2023-10-16T09:32:20.613 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-16T09:32:20.614 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428968 --owner mchangir --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} --name mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi -- /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428968/orig.config.yaml 2023-10-16T09:32:20.660 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T09:32:20.747 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428968 branch: wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: mchangir@redhat.com first_in_suite: false job_id: '7428968' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: c7471068426370a3754c2250e7729280233d552b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: c7471068426370a3754c2250e7729280233d552b kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: mgr-snap_schedule-support-subvol-and-group-arguments sha1: 7b3d12460841f3cbe9400ba1b68e3e20957ff531 owner: mchangir priority: 25 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 127 sha1: c7471068426370a3754c2250e7729280233d552b sleep_before_teardown: 0 suite: fs suite_branch: mgr-snap_schedule-support-subvol-and-group-arguments suite_path: /home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa suite_relpath: qa suite_repo: https://github.com/mchangir/ceph.git suite_sha1: 7b3d12460841f3cbe9400ba1b68e3e20957ff531 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi084.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJXxfKL1uHMlQo74v+R3GoiPXecQzV8P30woGckwQ84qfJ6EEXNgTDKH3hKwuDEEPsHSPFbQFO1y9ERH4QRY11M= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE2h+CtCtnk9zYAofCvqysHgWMJhS6NpRlokOe2x0VCCmhnx8iudgEX75GybhID5gv11OiYGzIRjlOPKuwng4PM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 8cdab074dcca9a68965bc5a50e9c30b691949723 timestamp: 2023-10-16_09:20:41 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-16T09:32:20.748 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa; will attempt to use it 2023-10-16T09:32:20.749 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks 2023-10-16T09:32:20.749 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-16T09:32:20.751 INFO:teuthology.task.internal:Checking packages... 2023-10-16T09:32:20.775 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c7471068426370a3754c2250e7729280233d552b' 2023-10-16T09:32:20.776 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-16T09:32:20.776 INFO:teuthology.packaging:ref: None 2023-10-16T09:32:20.777 INFO:teuthology.packaging:tag: None 2023-10-16T09:32:20.777 INFO:teuthology.packaging:branch: wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments 2023-10-16T09:32:20.777 INFO:teuthology.packaging:sha1: c7471068426370a3754c2250e7729280233d552b 2023-10-16T09:32:20.778 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments 2023-10-16T09:32:20.941 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6675.gc7471068 2023-10-16T09:32:20.941 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-16T09:32:20.947 INFO:teuthology.task.internal:no buildpackages task found 2023-10-16T09:32:20.948 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-16T09:32:20.953 INFO:teuthology.task.internal:Saving configuration 2023-10-16T09:32:20.975 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-16T09:32:20.981 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T09:32:21.008 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428968', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 09:26:43.499039', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJXxfKL1uHMlQo74v+R3GoiPXecQzV8P30woGckwQ84qfJ6EEXNgTDKH3hKwuDEEPsHSPFbQFO1y9ERH4QRY11M='} 2023-10-16T09:32:21.029 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428968', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 09:26:43.500630', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE2h+CtCtnk9zYAofCvqysHgWMJhS6NpRlokOe2x0VCCmhnx8iudgEX75GybhID5gv11OiYGzIRjlOPKuwng4PM='} 2023-10-16T09:32:21.030 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-16T09:32:21.037 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-16T09:32:21.037 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-16T09:32:21.037 INFO:teuthology.run_tasks:Running task console_log... 2023-10-16T09:32:21.104 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f090415b9d0>, signals=[15]) 2023-10-16T09:32:21.105 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-16T09:32:21.111 INFO:teuthology.task.internal:Opening connections... 2023-10-16T09:32:21.112 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T09:32:21.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T09:32:21.197 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2023-10-16T09:32:21.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T09:32:21.283 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-16T09:32:21.290 DEBUG:teuthology.orchestra.run.smithi084:> uname -m 2023-10-16T09:32:21.308 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64 2023-10-16T09:32:21.309 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release 2023-10-16T09:32:21.365 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="CentOS Stream" 2023-10-16T09:32:21.366 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="9" 2023-10-16T09:32:21.366 INFO:teuthology.orchestra.run.smithi084.stdout:ID="centos" 2023-10-16T09:32:21.366 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="rhel fedora" 2023-10-16T09:32:21.366 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="9" 2023-10-16T09:32:21.366 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el9" 2023-10-16T09:32:21.367 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-16T09:32:21.367 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31" 2023-10-16T09:32:21.367 INFO:teuthology.orchestra.run.smithi084.stdout:LOGO="fedora-logo-icon" 2023-10-16T09:32:21.367 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-16T09:32:21.368 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://centos.org/" 2023-10-16T09:32:21.368 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-16T09:32:21.368 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-16T09:32:21.368 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-16T09:32:21.369 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2023-10-16T09:32:21.395 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2023-10-16T09:32:21.413 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2023-10-16T09:32:21.414 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2023-10-16T09:32:21.468 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2023-10-16T09:32:21.469 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="9" 2023-10-16T09:32:21.469 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2023-10-16T09:32:21.469 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2023-10-16T09:32:21.469 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="9" 2023-10-16T09:32:21.470 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el9" 2023-10-16T09:32:21.470 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-16T09:32:21.470 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2023-10-16T09:32:21.470 INFO:teuthology.orchestra.run.smithi157.stdout:LOGO="fedora-logo-icon" 2023-10-16T09:32:21.470 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-16T09:32:21.471 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2023-10-16T09:32:21.471 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-16T09:32:21.471 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-16T09:32:21.471 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-16T09:32:21.472 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2023-10-16T09:32:21.495 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-16T09:32:21.505 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-16T09:32:21.510 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-16T09:32:21.510 DEBUG:teuthology.orchestra.run.smithi084:> test '!' -e /home/ubuntu/cephtest 2023-10-16T09:32:21.513 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2023-10-16T09:32:21.530 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-16T09:32:21.536 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-16T09:32:21.536 DEBUG:teuthology.orchestra.run.smithi084:> test -z $(ls -A /var/lib/ceph) 2023-10-16T09:32:21.569 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2023-10-16T09:32:21.599 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-16T09:32:21.654 INFO:teuthology.run_tasks:Running task kernel... 2023-10-16T09:32:21.668 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-16T09:32:21.669 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-10-16T09:32:21.669 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-10-16T09:32:21.669 DEBUG:teuthology.orchestra.run.smithi084:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-16T09:32:21.670 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-16T09:32:21.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T09:32:21.688 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2023-10-16T09:32:21.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T09:32:21.692 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2023-10-16T09:32:21.848 DEBUG:teuthology.task.kernel:sha1 for client.1 is 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T09:32:21.848 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "47672119149b103ec7cec2cc65c063ccbb001ac7"... 2023-10-16T09:32:21.849 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2023-10-16T09:32:21.866 INFO:teuthology.orchestra.run.smithi157.stdout:5.14.0-361.el9.x86_64 2023-10-16T09:32:21.867 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T09:32:21.867 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-16T09:32:21.868 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum upgrade -y linux-firmware 2023-10-16T09:32:22.001 DEBUG:teuthology.task.kernel:sha1 for client.0 is 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T09:32:22.002 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "47672119149b103ec7cec2cc65c063ccbb001ac7"... 2023-10-16T09:32:22.002 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2023-10-16T09:32:22.019 INFO:teuthology.orchestra.run.smithi084.stdout:5.14.0-361.el9.x86_64 2023-10-16T09:32:22.019 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T09:32:22.020 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-16T09:32:22.020 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum upgrade -y linux-firmware 2023-10-16T09:32:24.103 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - AppStream 10 MB/s | 6.3 MB 00:00 2023-10-16T09:32:24.364 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS-9 - AppStream 8.9 MB/s | 6.3 MB 00:00 2023-10-16T09:32:25.436 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - BaseOS 6.3 MB/s | 2.0 MB 00:00 2023-10-16T09:32:26.182 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS-9 - BaseOS 2.7 MB/s | 2.0 MB 00:00 2023-10-16T09:32:26.211 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 7.9 MB 00:00 2023-10-16T09:32:27.147 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 7.9 MB 00:00 2023-10-16T09:32:31.145 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 4.5 MB/s | 18 MB 00:03 2023-10-16T09:32:36.204 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 5.7 MB 00:01 2023-10-16T09:32:37.821 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 14 kB 00:00 2023-10-16T09:32:38.431 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 38 MB/s | 19 MB 00:00 2023-10-16T09:32:42.889 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2023-10-16T09:32:43.661 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 18 MB 00:15 2023-10-16T09:32:45.015 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-10-16T09:32:45.030 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-10-16T09:32:45.030 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-10-16T09:32:45.030 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-10-16T09:32:45.030 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2023-10-16T09:32:45.031 INFO:teuthology.orchestra.run.smithi157.stdout: linux-firmware noarch 20230926-140.el9 CentOS-BaseOS 326 M 2023-10-16T09:32:45.031 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:32:45.031 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-10-16T09:32:45.031 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-10-16T09:32:45.031 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 1 Package 2023-10-16T09:32:45.032 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:32:45.032 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 326 M 2023-10-16T09:32:45.032 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-10-16T09:32:47.703 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - CRB 7.8 MB/s | 5.7 MB 00:00 2023-10-16T09:32:49.245 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 14 kB 00:00 2023-10-16T09:32:51.258 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 10 MB/s | 19 MB 00:01 2023-10-16T09:32:54.064 INFO:teuthology.orchestra.run.smithi157.stdout:linux-firmware-20230926-140.el9.noarch.rpm 36 MB/s | 326 MB 00:09 2023-10-16T09:32:54.065 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-10-16T09:32:54.066 INFO:teuthology.orchestra.run.smithi157.stdout:Total 36 MB/s | 326 MB 00:09 2023-10-16T09:32:54.066 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-10-16T09:32:54.140 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-10-16T09:32:54.141 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-10-16T09:32:55.696 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2023-10-16T09:32:56.310 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-10-16T09:32:56.310 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-10-16T09:32:57.896 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2023-10-16T09:32:57.908 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2023-10-16T09:32:57.908 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2023-10-16T09:32:57.908 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2023-10-16T09:32:57.909 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrading: 2023-10-16T09:32:57.909 INFO:teuthology.orchestra.run.smithi084.stdout: linux-firmware noarch 20230926-140.el9 CentOS-BaseOS 326 M 2023-10-16T09:32:57.909 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:32:57.909 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2023-10-16T09:32:57.909 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2023-10-16T09:32:57.910 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrade 1 Package 2023-10-16T09:32:57.910 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:32:57.910 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 326 M 2023-10-16T09:32:57.910 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2023-10-16T09:33:01.966 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-10-16T09:33:02.399 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-16T09:33:02.486 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-16T09:33:03.417 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-10-16T09:33:03.417 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-16T09:33:03.823 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-16T09:33:03.824 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:33:03.824 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2023-10-16T09:33:03.824 INFO:teuthology.orchestra.run.smithi157.stdout: linux-firmware-20230926-140.el9.noarch 2023-10-16T09:33:03.824 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:33:03.824 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-10-16T09:33:04.094 INFO:teuthology.task.kernel:Downloading kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.1... 2023-10-16T09:33:04.094 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T09:33:04.336 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/9/flavors/default/x86_64/ 2023-10-16T09:33:04.337 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/9/flavors/default/x86_64/ --input-file=- 2023-10-16T09:33:04.340 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi157.front.sepia.ceph.com to complete... 2023-10-16T09:33:05.638 INFO:teuthology.orchestra.run.smithi084.stdout:linux-firmware-20230926-140.el9.noarch.rpm 42 MB/s | 326 MB 00:07 2023-10-16T09:33:05.639 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2023-10-16T09:33:05.639 INFO:teuthology.orchestra.run.smithi084.stdout:Total 42 MB/s | 326 MB 00:07 2023-10-16T09:33:05.640 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2023-10-16T09:33:05.724 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2023-10-16T09:33:05.724 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2023-10-16T09:33:07.912 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2023-10-16T09:33:07.912 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2023-10-16T09:33:13.300 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2023-10-16T09:33:13.679 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-16T09:33:13.784 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-16T09:33:14.749 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-10-16T09:33:14.750 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-16T09:33:15.127 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-16T09:33:15.127 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:33:15.128 INFO:teuthology.orchestra.run.smithi084.stdout:Upgraded: 2023-10-16T09:33:15.128 INFO:teuthology.orchestra.run.smithi084.stdout: linux-firmware-20230926-140.el9.noarch 2023-10-16T09:33:15.128 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:33:15.128 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2023-10-16T09:33:15.407 INFO:teuthology.task.kernel:Downloading kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.0... 2023-10-16T09:33:15.408 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T09:33:15.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16 09:33:15 URL:https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/9/flavors/default/x86_64/kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm [320683515/320683515] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-16T09:33:15.433 INFO:teuthology.orchestra.run.smithi157.stderr:FINISHED --2023-10-16 09:33:15-- 2023-10-16T09:33:15.433 INFO:teuthology.orchestra.run.smithi157.stderr:Total wall clock time: 11s 2023-10-16T09:33:15.433 INFO:teuthology.orchestra.run.smithi157.stderr:Downloaded: 1 files, 306M in 11s (28.5 MB/s) 2023-10-16T09:33:15.434 INFO:teuthology.task.kernel:Installing kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.1... 2023-10-16T09:33:15.435 DEBUG:teuthology.orchestra.run.smithi157:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-16T09:33:15.645 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/9/flavors/default/x86_64/ 2023-10-16T09:33:15.646 DEBUG:teuthology.orchestra.run.smithi084:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/9/flavors/default/x86_64/ --input-file=- 2023-10-16T09:33:15.648 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi084.front.sepia.ceph.com to complete... 2023-10-16T09:33:17.309 INFO:teuthology.orchestra.run.smithi157.stdout:Verifying... ######################################## 2023-10-16T09:33:17.357 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing... ######################################## 2023-10-16T09:33:17.436 INFO:teuthology.orchestra.run.smithi157.stdout:Updating / installing... 2023-10-16T09:33:24.540 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-6.6.0_rc5_g47672119149b-1 ######################################## 2023-10-16T09:33:26.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16 09:33:26 URL:https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/9/flavors/default/x86_64/kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm [320683515/320683515] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-16T09:33:26.554 INFO:teuthology.orchestra.run.smithi084.stderr:FINISHED --2023-10-16 09:33:26-- 2023-10-16T09:33:26.554 INFO:teuthology.orchestra.run.smithi084.stderr:Total wall clock time: 11s 2023-10-16T09:33:26.555 INFO:teuthology.orchestra.run.smithi084.stderr:Downloaded: 1 files, 306M in 11s (28.9 MB/s) 2023-10-16T09:33:26.556 INFO:teuthology.task.kernel:Installing kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.0... 2023-10-16T09:33:26.556 DEBUG:teuthology.orchestra.run.smithi084:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-16T09:33:28.406 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying... ######################################## 2023-10-16T09:33:28.455 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing... ######################################## 2023-10-16T09:33:28.534 INFO:teuthology.orchestra.run.smithi084.stdout:Updating / installing... 2023-10-16T09:33:35.666 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-6.6.0_rc5_g47672119149b-1 ######################################## 2023-10-16T09:33:39.619 INFO:teuthology.orchestra.run.smithi157.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-10-16T09:33:50.501 INFO:teuthology.orchestra.run.smithi084.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-10-16T09:34:00.855 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi157.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-16T09:34:00.856 DEBUG:teuthology.orchestra.run.smithi157:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-16T09:34:00.929 INFO:teuthology.orchestra.run.smithi157.stdout:/boot/System.map-6.6.0-rc5-g47672119149b 2023-10-16T09:34:00.929 INFO:teuthology.orchestra.run.smithi157.stdout:/boot/config-6.6.0-rc5-g47672119149b 2023-10-16T09:34:00.929 INFO:teuthology.orchestra.run.smithi157.stdout:/boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-16T09:34:00.929 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b 2023-10-16T09:34:00.930 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel 2023-10-16T09:34:00.930 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch 2023-10-16T09:34:00.930 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86 2023-10-16T09:34:00.930 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto 2023-10-16T09:34:00.931 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-16T09:34:00.931 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-16T09:34:00.931 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-16T09:34:00.931 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-16T09:34:00.932 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-16T09:34:00.932 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-16T09:34:00.932 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-16T09:34:00.932 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-16T09:34:00.932 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-16T09:34:00.933 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-16T09:34:00.933 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-16T09:34:00.933 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-16T09:34:00.933 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-16T09:34:00.934 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-16T09:34:00.934 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-16T09:34:00.934 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-16T09:34:00.934 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-16T09:34:00.935 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-16T09:34:00.935 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-16T09:34:00.935 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-16T09:34:00.935 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel 2023-10-16T09:34:00.935 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu 2023-10-16T09:34:00.936 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce 2023-10-16T09:34:00.936 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-16T09:34:00.936 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm 2023-10-16T09:34:00.936 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-16T09:34:00.937 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-16T09:34:00.937 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm.ko 2023-10-16T09:34:00.937 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto 2023-10-16T09:34:00.937 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ansi_cprng.ko 2023-10-16T09:34:00.938 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/anubis.ko 2023-10-16T09:34:00.938 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/arc4.ko 2023-10-16T09:34:00.938 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx 2023-10-16T09:34:00.938 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-16T09:34:00.939 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_pq.ko 2023-10-16T09:34:00.939 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-16T09:34:00.939 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_tx.ko 2023-10-16T09:34:00.939 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_xor.ko 2023-10-16T09:34:00.939 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/raid6test.ko 2023-10-16T09:34:00.940 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authenc.ko 2023-10-16T09:34:00.940 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authencesn.ko 2023-10-16T09:34:00.940 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blake2b_generic.ko 2023-10-16T09:34:00.940 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_common.ko 2023-10-16T09:34:00.940 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_generic.ko 2023-10-16T09:34:00.941 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/camellia_generic.ko 2023-10-16T09:34:00.941 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast5_generic.ko 2023-10-16T09:34:00.941 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast6_generic.ko 2023-10-16T09:34:00.941 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast_common.ko 2023-10-16T09:34:00.942 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ccm.ko 2023-10-16T09:34:00.942 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cmac.ko 2023-10-16T09:34:00.942 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crc32_generic.ko 2023-10-16T09:34:00.942 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_engine.ko 2023-10-16T09:34:00.943 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_user.ko 2023-10-16T09:34:00.943 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cts.ko 2023-10-16T09:34:00.943 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/des_generic.ko 2023-10-16T09:34:00.943 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/dh_generic.ko 2023-10-16T09:34:00.943 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/echainiv.ko 2023-10-16T09:34:00.944 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/essiv.ko 2023-10-16T09:34:00.944 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/fcrypt.ko 2023-10-16T09:34:00.944 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/khazad.ko 2023-10-16T09:34:00.944 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4.ko 2023-10-16T09:34:00.945 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4hc.ko 2023-10-16T09:34:00.945 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/md4.ko 2023-10-16T09:34:00.945 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/michael_mic.ko 2023-10-16T09:34:00.945 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcbc.ko 2023-10-16T09:34:00.945 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcrypt.ko 2023-10-16T09:34:00.946 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/rmd160.ko 2023-10-16T09:34:00.946 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/seed.ko 2023-10-16T09:34:00.946 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/serpent_generic.ko 2023-10-16T09:34:00.946 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tcrypt.ko 2023-10-16T09:34:00.946 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tea.ko 2023-10-16T09:34:00.947 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_common.ko 2023-10-16T09:34:00.947 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_generic.ko 2023-10-16T09:34:00.947 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/vmac.ko 2023-10-16T09:34:00.947 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/wp512.ko 2023-10-16T09:34:00.948 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xcbc.ko 2023-10-16T09:34:00.948 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xor.ko 2023-10-16T09:34:00.948 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xxhash_generic.ko 2023-10-16T09:34:00.948 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers 2023-10-16T09:34:00.949 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi 2023-10-16T09:34:00.949 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-16T09:34:00.949 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_pad.ko 2023-10-16T09:34:00.949 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/custom_method.ko 2023-10-16T09:34:00.950 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ec_sys.ko 2023-10-16T09:34:00.950 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/platform_profile.ko 2023-10-16T09:34:00.950 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbs.ko 2023-10-16T09:34:00.950 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbshc.ko 2023-10-16T09:34:00.951 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/video.ko 2023-10-16T09:34:00.951 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata 2023-10-16T09:34:00.951 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/acard-ahci.ko 2023-10-16T09:34:00.951 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci_platform.ko 2023-10-16T09:34:00.951 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ata_generic.ko 2023-10-16T09:34:00.952 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci_platform.ko 2023-10-16T09:34:00.952 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_acpi.ko 2023-10-16T09:34:00.952 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ali.ko 2023-10-16T09:34:00.952 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_amd.ko 2023-10-16T09:34:00.953 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_artop.ko 2023-10-16T09:34:00.953 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atiixp.ko 2023-10-16T09:34:00.953 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atp867x.ko 2023-10-16T09:34:00.953 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd640.ko 2023-10-16T09:34:00.954 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-16T09:34:00.954 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cypress.ko 2023-10-16T09:34:00.954 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_efar.ko 2023-10-16T09:34:00.954 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt366.ko 2023-10-16T09:34:00.954 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-16T09:34:00.955 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-16T09:34:00.955 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-16T09:34:00.955 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it8213.ko 2023-10-16T09:34:00.955 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it821x.ko 2023-10-16T09:34:00.956 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_jmicron.ko 2023-10-16T09:34:00.956 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_marvell.ko 2023-10-16T09:34:00.956 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_mpiix.ko 2023-10-16T09:34:00.956 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_netcell.ko 2023-10-16T09:34:00.957 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ninja32.ko 2023-10-16T09:34:00.957 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87410.ko 2023-10-16T09:34:00.957 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87415.ko 2023-10-16T09:34:00.957 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-16T09:34:00.958 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_opti.ko 2023-10-16T09:34:00.958 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_optidma.ko 2023-10-16T09:34:00.958 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-16T09:34:00.958 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-16T09:34:00.958 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-16T09:34:00.959 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_piccolo.ko 2023-10-16T09:34:00.959 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rdc.ko 2023-10-16T09:34:00.959 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sch.ko 2023-10-16T09:34:00.959 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_serverworks.ko 2023-10-16T09:34:00.960 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sil680.ko 2023-10-16T09:34:00.960 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sis.ko 2023-10-16T09:34:00.960 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-16T09:34:00.960 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_triflex.ko 2023-10-16T09:34:00.960 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_via.ko 2023-10-16T09:34:00.961 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pdc_adma.ko 2023-10-16T09:34:00.961 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_inic162x.ko 2023-10-16T09:34:00.961 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_mv.ko 2023-10-16T09:34:00.961 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_nv.ko 2023-10-16T09:34:00.962 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_promise.ko 2023-10-16T09:34:00.962 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_qstor.ko 2023-10-16T09:34:00.962 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil.ko 2023-10-16T09:34:00.962 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil24.ko 2023-10-16T09:34:00.963 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sis.ko 2023-10-16T09:34:00.963 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_svw.ko 2023-10-16T09:34:00.963 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sx4.ko 2023-10-16T09:34:00.963 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_uli.ko 2023-10-16T09:34:00.964 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_via.ko 2023-10-16T09:34:00.964 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_vsc.ko 2023-10-16T09:34:00.964 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay 2023-10-16T09:34:00.964 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-16T09:34:00.964 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base 2023-10-16T09:34:00.965 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap 2023-10-16T09:34:00.965 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-16T09:34:00.965 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-16T09:34:00.965 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma 2023-10-16T09:34:00.966 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/bcma.ko 2023-10-16T09:34:00.966 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block 2023-10-16T09:34:00.966 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe 2023-10-16T09:34:00.966 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/aoe.ko 2023-10-16T09:34:00.966 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/brd.ko 2023-10-16T09:34:00.967 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd 2023-10-16T09:34:00.967 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/drbd.ko 2023-10-16T09:34:00.967 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/floppy.ko 2023-10-16T09:34:00.968 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/loop.ko 2023-10-16T09:34:00.968 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx 2023-10-16T09:34:00.968 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-16T09:34:00.968 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/nbd.ko 2023-10-16T09:34:00.969 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk 2023-10-16T09:34:00.969 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-16T09:34:00.969 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/pktcdvd.ko 2023-10-16T09:34:00.969 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/rbd.ko 2023-10-16T09:34:00.970 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/virtio_blk.ko 2023-10-16T09:34:00.970 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback 2023-10-16T09:34:00.970 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-16T09:34:00.970 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkfront.ko 2023-10-16T09:34:00.970 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram 2023-10-16T09:34:00.971 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/zram.ko 2023-10-16T09:34:00.971 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char 2023-10-16T09:34:00.971 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hangcheck-timer.ko 2023-10-16T09:34:00.971 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random 2023-10-16T09:34:00.972 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-16T09:34:00.972 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-16T09:34:00.972 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-16T09:34:00.972 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-16T09:34:00.973 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-16T09:34:00.973 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi 2023-10-16T09:34:00.973 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-16T09:34:00.973 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-16T09:34:00.974 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-16T09:34:00.974 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-16T09:34:00.974 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-16T09:34:00.974 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-16T09:34:00.974 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/lp.ko 2023-10-16T09:34:00.975 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave 2023-10-16T09:34:00.975 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/mwave.ko 2023-10-16T09:34:00.975 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ppdev.ko 2023-10-16T09:34:00.975 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tlclk.ko 2023-10-16T09:34:00.975 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm 2023-10-16T09:34:00.976 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm.ko 2023-10-16T09:34:00.976 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-16T09:34:00.976 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-16T09:34:00.976 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-16T09:34:00.977 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-16T09:34:00.977 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-16T09:34:00.977 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/uv_mmtimer.ko 2023-10-16T09:34:00.977 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/virtio_console.ko 2023-10-16T09:34:00.977 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq 2023-10-16T09:34:00.978 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-16T09:34:00.978 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-16T09:34:00.978 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-16T09:34:00.978 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-16T09:34:00.978 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-16T09:34:00.979 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-16T09:34:00.979 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto 2023-10-16T09:34:00.979 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp 2023-10-16T09:34:00.979 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-16T09:34:00.979 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-16T09:34:00.980 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel 2023-10-16T09:34:00.980 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat 2023-10-16T09:34:00.980 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-16T09:34:00.980 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-16T09:34:00.980 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-16T09:34:00.981 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-16T09:34:00.981 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-aes.ko 2023-10-16T09:34:00.981 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-sha.ko 2023-10-16T09:34:00.981 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio 2023-10-16T09:34:00.982 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-16T09:34:00.982 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca 2023-10-16T09:34:00.982 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/dca.ko 2023-10-16T09:34:00.982 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq 2023-10-16T09:34:00.983 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-16T09:34:00.983 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma 2023-10-16T09:34:00.983 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw 2023-10-16T09:34:00.983 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-16T09:34:00.984 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat 2023-10-16T09:34:00.984 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-16T09:34:00.984 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac 2023-10-16T09:34:00.984 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/amd64_edac.ko 2023-10-16T09:34:00.984 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/e752x_edac.ko 2023-10-16T09:34:00.985 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-16T09:34:00.985 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3000_edac.ko 2023-10-16T09:34:00.985 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3200_edac.ko 2023-10-16T09:34:00.985 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5100_edac.ko 2023-10-16T09:34:00.985 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5400_edac.ko 2023-10-16T09:34:00.986 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7300_edac.ko 2023-10-16T09:34:00.986 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7core_edac.ko 2023-10-16T09:34:00.986 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i82975x_edac.ko 2023-10-16T09:34:00.986 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ie31200_edac.ko 2023-10-16T09:34:00.987 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/sb_edac.ko 2023-10-16T09:34:00.987 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/x38_edac.ko 2023-10-16T09:34:00.987 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware 2023-10-16T09:34:00.987 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/edd.ko 2023-10-16T09:34:00.987 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-16T09:34:00.988 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio 2023-10-16T09:34:00.988 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ich.ko 2023-10-16T09:34:00.988 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-16T09:34:00.988 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid 2023-10-16T09:34:00.988 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-appleir.ko 2023-10-16T09:34:00.989 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-aureal.ko 2023-10-16T09:34:00.989 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-axff.ko 2023-10-16T09:34:00.989 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-dr.ko 2023-10-16T09:34:00.989 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elecom.ko 2023-10-16T09:34:00.989 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elo.ko 2023-10-16T09:34:00.990 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-emsff.ko 2023-10-16T09:34:00.990 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gaff.ko 2023-10-16T09:34:00.990 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gt683r.ko 2023-10-16T09:34:00.990 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gyration.ko 2023-10-16T09:34:00.991 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-16T09:34:00.991 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-16T09:34:00.991 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtekff.ko 2023-10-16T09:34:00.991 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-hyperv.ko 2023-10-16T09:34:00.991 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-icade.ko 2023-10-16T09:34:00.992 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-keytouch.ko 2023-10-16T09:34:00.992 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kye.ko 2023-10-16T09:34:00.992 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lcpower.ko 2023-10-16T09:34:00.992 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-led.ko 2023-10-16T09:34:00.992 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lenovo.ko 2023-10-16T09:34:00.993 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-16T09:34:00.993 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-16T09:34:00.993 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-multitouch.ko 2023-10-16T09:34:00.993 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ortek.ko 2023-10-16T09:34:00.993 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-penmount.ko 2023-10-16T09:34:00.994 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-petalynx.ko 2023-10-16T09:34:00.994 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-picolcd.ko 2023-10-16T09:34:00.994 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-pl.ko 2023-10-16T09:34:00.994 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-plantronics.ko 2023-10-16T09:34:00.995 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-primax.ko 2023-10-16T09:34:00.995 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-rmi.ko 2023-10-16T09:34:00.995 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-16T09:34:00.995 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-16T09:34:00.996 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-16T09:34:00.996 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-16T09:34:00.996 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-16T09:34:00.996 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-16T09:34:00.996 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-16T09:34:00.997 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-16T09:34:00.997 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-16T09:34:00.997 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-16T09:34:00.997 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-16T09:34:00.998 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat.ko 2023-10-16T09:34:00.998 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-saitek.ko 2023-10-16T09:34:00.998 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-samsung.ko 2023-10-16T09:34:00.998 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-16T09:34:00.998 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sjoy.ko 2023-10-16T09:34:00.999 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sony.ko 2023-10-16T09:34:00.999 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-speedlink.ko 2023-10-16T09:34:00.999 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-steelseries.ko 2023-10-16T09:34:00.999 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sunplus.ko 2023-10-16T09:34:01.000 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-16T09:34:01.000 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tivo.ko 2023-10-16T09:34:01.000 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tmff.ko 2023-10-16T09:34:01.000 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-topseed.ko 2023-10-16T09:34:01.000 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-twinhan.ko 2023-10-16T09:34:01.001 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-uclogic.ko 2023-10-16T09:34:01.001 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-waltop.ko 2023-10-16T09:34:01.001 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-wiimote.ko 2023-10-16T09:34:01.001 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-xinmo.ko 2023-10-16T09:34:01.001 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zpff.ko 2023-10-16T09:34:01.002 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zydacron.ko 2023-10-16T09:34:01.002 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/uhid.ko 2023-10-16T09:34:01.002 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/wacom.ko 2023-10-16T09:34:01.002 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv 2023-10-16T09:34:01.002 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_balloon.ko 2023-10-16T09:34:01.003 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_utils.ko 2023-10-16T09:34:01.003 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_vmbus.ko 2023-10-16T09:34:01.003 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon 2023-10-16T09:34:01.003 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru.ko 2023-10-16T09:34:01.004 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru3.ko 2023-10-16T09:34:01.004 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-16T09:34:01.004 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7414.ko 2023-10-16T09:34:01.005 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7418.ko 2023-10-16T09:34:01.005 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adc128d818.ko 2023-10-16T09:34:01.005 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1025.ko 2023-10-16T09:34:01.005 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1026.ko 2023-10-16T09:34:01.006 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1029.ko 2023-10-16T09:34:01.006 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1031.ko 2023-10-16T09:34:01.006 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm9240.ko 2023-10-16T09:34:01.006 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ads7828.ko 2023-10-16T09:34:01.006 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7410.ko 2023-10-16T09:34:01.007 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7411.ko 2023-10-16T09:34:01.007 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7462.ko 2023-10-16T09:34:01.007 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7470.ko 2023-10-16T09:34:01.007 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7475.ko 2023-10-16T09:34:01.007 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7x10.ko 2023-10-16T09:34:01.008 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/amc6821.ko 2023-10-16T09:34:01.008 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/applesmc.ko 2023-10-16T09:34:01.008 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asb100.ko 2023-10-16T09:34:01.008 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asc7621.ko 2023-10-16T09:34:01.009 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-16T09:34:01.009 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/atxp1.ko 2023-10-16T09:34:01.009 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/coretemp.ko 2023-10-16T09:34:01.010 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-16T09:34:01.010 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dme1737.ko 2023-10-16T09:34:01.010 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds1621.ko 2023-10-16T09:34:01.010 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds620.ko 2023-10-16T09:34:01.011 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc1403.ko 2023-10-16T09:34:01.011 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc6w201.ko 2023-10-16T09:34:01.011 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71805f.ko 2023-10-16T09:34:01.011 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71882fg.ko 2023-10-16T09:34:01.012 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f75375s.ko 2023-10-16T09:34:01.012 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-16T09:34:01.012 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fschmd.ko 2023-10-16T09:34:01.012 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g760a.ko 2023-10-16T09:34:01.012 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g762.ko 2023-10-16T09:34:01.013 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl518sm.ko 2023-10-16T09:34:01.013 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl520sm.ko 2023-10-16T09:34:01.013 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-16T09:34:01.013 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-16T09:34:01.013 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-16T09:34:01.014 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmaem.ko 2023-10-16T09:34:01.014 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmpex.ko 2023-10-16T09:34:01.014 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina209.ko 2023-10-16T09:34:01.014 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina2xx.ko 2023-10-16T09:34:01.015 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/it87.ko 2023-10-16T09:34:01.015 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k10temp.ko 2023-10-16T09:34:01.015 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k8temp.ko 2023-10-16T09:34:01.015 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-16T09:34:01.015 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm63.ko 2023-10-16T09:34:01.016 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm73.ko 2023-10-16T09:34:01.016 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm75.ko 2023-10-16T09:34:01.016 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm77.ko 2023-10-16T09:34:01.016 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm78.ko 2023-10-16T09:34:01.016 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm80.ko 2023-10-16T09:34:01.017 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm83.ko 2023-10-16T09:34:01.017 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm85.ko 2023-10-16T09:34:01.017 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm87.ko 2023-10-16T09:34:01.017 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm90.ko 2023-10-16T09:34:01.018 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm92.ko 2023-10-16T09:34:01.018 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm93.ko 2023-10-16T09:34:01.018 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95234.ko 2023-10-16T09:34:01.018 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95241.ko 2023-10-16T09:34:01.018 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95245.ko 2023-10-16T09:34:01.019 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc2945.ko 2023-10-16T09:34:01.019 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4151.ko 2023-10-16T09:34:01.019 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4215.ko 2023-10-16T09:34:01.019 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4222.ko 2023-10-16T09:34:01.020 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4245.ko 2023-10-16T09:34:01.020 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4260.ko 2023-10-16T09:34:01.020 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4261.ko 2023-10-16T09:34:01.020 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max16065.ko 2023-10-16T09:34:01.020 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1619.ko 2023-10-16T09:34:01.021 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1668.ko 2023-10-16T09:34:01.021 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max197.ko 2023-10-16T09:34:01.021 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6639.ko 2023-10-16T09:34:01.021 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6650.ko 2023-10-16T09:34:01.021 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6697.ko 2023-10-16T09:34:01.022 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mcp3021.ko 2023-10-16T09:34:01.022 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6683.ko 2023-10-16T09:34:01.022 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-16T09:34:01.022 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775.ko 2023-10-16T09:34:01.023 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct7802.ko 2023-10-16T09:34:01.023 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87360.ko 2023-10-16T09:34:01.023 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87427.ko 2023-10-16T09:34:01.023 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pcf8591.ko 2023-10-16T09:34:01.023 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus 2023-10-16T09:34:01.024 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-16T09:34:01.024 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-16T09:34:01.024 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-16T09:34:01.024 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-16T09:34:01.024 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-16T09:34:01.025 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-16T09:34:01.025 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-16T09:34:01.025 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-16T09:34:01.025 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-16T09:34:01.025 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-16T09:34:01.026 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-16T09:34:01.026 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-16T09:34:01.026 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/powr1220.ko 2023-10-16T09:34:01.026 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5627.ko 2023-10-16T09:34:01.027 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5636.ko 2023-10-16T09:34:01.027 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-16T09:34:01.027 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht15.ko 2023-10-16T09:34:01.027 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht21.ko 2023-10-16T09:34:01.028 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/shtc1.ko 2023-10-16T09:34:01.028 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sis5595.ko 2023-10-16T09:34:01.028 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-16T09:34:01.028 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-16T09:34:01.029 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-16T09:34:01.029 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/thmc50.ko 2023-10-16T09:34:01.029 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp102.ko 2023-10-16T09:34:01.029 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp103.ko 2023-10-16T09:34:01.029 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp401.ko 2023-10-16T09:34:01.030 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp421.ko 2023-10-16T09:34:01.030 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-16T09:34:01.030 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via686a.ko 2023-10-16T09:34:01.030 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt1211.ko 2023-10-16T09:34:01.030 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt8231.ko 2023-10-16T09:34:01.031 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-16T09:34:01.031 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627hf.ko 2023-10-16T09:34:01.031 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83781d.ko 2023-10-16T09:34:01.031 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83791d.ko 2023-10-16T09:34:01.032 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83792d.ko 2023-10-16T09:34:01.032 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83793.ko 2023-10-16T09:34:01.032 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83795.ko 2023-10-16T09:34:01.032 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-16T09:34:01.032 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-16T09:34:01.033 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c 2023-10-16T09:34:01.033 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos 2023-10-16T09:34:01.033 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-16T09:34:01.033 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-16T09:34:01.033 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses 2023-10-16T09:34:01.034 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-16T09:34:01.034 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-16T09:34:01.034 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-16T09:34:01.034 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-16T09:34:01.034 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-16T09:34:01.035 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-16T09:34:01.035 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-16T09:34:01.035 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-16T09:34:01.035 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-16T09:34:01.036 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-16T09:34:01.036 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-16T09:34:01.036 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-16T09:34:01.036 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-16T09:34:01.036 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-16T09:34:01.037 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-16T09:34:01.037 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-16T09:34:01.037 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-16T09:34:01.037 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-16T09:34:01.037 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-16T09:34:01.038 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-16T09:34:01.038 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-16T09:34:01.038 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-16T09:34:01.038 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-16T09:34:01.039 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-dev.ko 2023-10-16T09:34:01.039 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-mux.ko 2023-10-16T09:34:01.039 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-16T09:34:01.039 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-16T09:34:01.039 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-stub.ko 2023-10-16T09:34:01.040 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband 2023-10-16T09:34:01.040 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core 2023-10-16T09:34:01.040 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-16T09:34:01.040 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-16T09:34:01.040 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-16T09:34:01.041 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-16T09:34:01.041 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-16T09:34:01.041 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-16T09:34:01.041 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-16T09:34:01.042 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw 2023-10-16T09:34:01.042 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4 2023-10-16T09:34:01.042 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-16T09:34:01.042 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4 2023-10-16T09:34:01.042 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-16T09:34:01.043 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5 2023-10-16T09:34:01.043 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-16T09:34:01.043 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca 2023-10-16T09:34:01.043 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-16T09:34:01.044 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp 2023-10-16T09:34:01.044 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib 2023-10-16T09:34:01.044 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-16T09:34:01.044 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser 2023-10-16T09:34:01.045 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-16T09:34:01.045 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert 2023-10-16T09:34:01.045 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-16T09:34:01.045 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp 2023-10-16T09:34:01.045 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-16T09:34:01.046 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt 2023-10-16T09:34:01.046 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-16T09:34:01.046 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input 2023-10-16T09:34:01.046 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport 2023-10-16T09:34:01.047 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-16T09:34:01.047 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-16T09:34:01.047 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/gameport.ko 2023-10-16T09:34:01.047 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/joydev.ko 2023-10-16T09:34:01.047 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard 2023-10-16T09:34:01.048 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-16T09:34:01.048 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse 2023-10-16T09:34:01.048 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/appletouch.ko 2023-10-16T09:34:01.048 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-16T09:34:01.048 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-16T09:34:01.049 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-16T09:34:01.049 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/sermouse.ko 2023-10-16T09:34:01.049 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-16T09:34:01.049 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-16T09:34:01.050 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-16T09:34:01.050 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4 2023-10-16T09:34:01.050 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-16T09:34:01.050 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio 2023-10-16T09:34:01.051 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-16T09:34:01.051 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-16T09:34:01.051 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-16T09:34:01.051 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serio_raw.ko 2023-10-16T09:34:01.052 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/sparse-keymap.ko 2023-10-16T09:34:01.052 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu 2023-10-16T09:34:01.052 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd 2023-10-16T09:34:01.052 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-16T09:34:01.052 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds 2023-10-16T09:34:01.053 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-blinkm.ko 2023-10-16T09:34:01.053 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3530.ko 2023-10-16T09:34:01.053 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp3944.ko 2023-10-16T09:34:01.053 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-ss4200.ko 2023-10-16T09:34:01.053 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger 2023-10-16T09:34:01.054 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-16T09:34:01.054 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-16T09:34:01.054 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-16T09:34:01.054 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-16T09:34:01.055 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-16T09:34:01.055 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-16T09:34:01.055 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-16T09:34:01.055 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-16T09:34:01.055 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md 2023-10-16T09:34:01.056 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache 2023-10-16T09:34:01.056 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/bcache.ko 2023-10-16T09:34:01.056 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bio-prison.ko 2023-10-16T09:34:01.056 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache-smq.ko 2023-10-16T09:34:01.056 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache.ko 2023-10-16T09:34:01.057 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-crypt.ko 2023-10-16T09:34:01.057 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-delay.ko 2023-10-16T09:34:01.057 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-flakey.ko 2023-10-16T09:34:01.057 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log-userspace.ko 2023-10-16T09:34:01.057 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-multipath.ko 2023-10-16T09:34:01.058 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-queue-length.ko 2023-10-16T09:34:01.058 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-raid.ko 2023-10-16T09:34:01.058 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-round-robin.ko 2023-10-16T09:34:01.058 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-service-time.ko 2023-10-16T09:34:01.059 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-switch.ko 2023-10-16T09:34:01.059 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-thin-pool.ko 2023-10-16T09:34:01.059 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-verity.ko 2023-10-16T09:34:01.059 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/faulty.ko 2023-10-16T09:34:01.059 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/linear.ko 2023-10-16T09:34:01.060 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/multipath.ko 2023-10-16T09:34:01.060 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data 2023-10-16T09:34:01.060 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-16T09:34:01.060 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid0.ko 2023-10-16T09:34:01.060 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid1.ko 2023-10-16T09:34:01.061 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid10.ko 2023-10-16T09:34:01.061 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid456.ko 2023-10-16T09:34:01.061 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message 2023-10-16T09:34:01.061 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion 2023-10-16T09:34:01.062 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptbase.ko 2023-10-16T09:34:01.062 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptctl.ko 2023-10-16T09:34:01.062 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptfc.ko 2023-10-16T09:34:01.062 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptlan.ko 2023-10-16T09:34:01.062 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptsas.ko 2023-10-16T09:34:01.063 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-16T09:34:01.063 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptspi.ko 2023-10-16T09:34:01.063 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd 2023-10-16T09:34:01.063 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_ich.ko 2023-10-16T09:34:01.063 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_sch.ko 2023-10-16T09:34:01.064 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mfd-core.ko 2023-10-16T09:34:01.064 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/sm501.ko 2023-10-16T09:34:01.064 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/viperboard.ko 2023-10-16T09:34:01.064 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/vx855.ko 2023-10-16T09:34:01.064 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wl1273-core.ko 2023-10-16T09:34:01.065 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc 2023-10-16T09:34:01.065 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl 2023-10-16T09:34:01.065 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-16T09:34:01.065 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds9802als.ko 2023-10-16T09:34:01.066 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds990x.ko 2023-10-16T09:34:01.066 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/bh1770glc.ko 2023-10-16T09:34:01.066 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710 2023-10-16T09:34:01.066 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/cb710.ko 2023-10-16T09:34:01.066 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom 2023-10-16T09:34:01.067 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/at24.ko 2023-10-16T09:34:01.067 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-16T09:34:01.067 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-16T09:34:01.067 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-16T09:34:01.068 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/enclosure.ko 2023-10-16T09:34:01.068 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hpilo.ko 2023-10-16T09:34:01.068 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29003.ko 2023-10-16T09:34:01.068 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29020.ko 2023-10-16T09:34:01.068 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d 2023-10-16T09:34:01.069 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-16T09:34:01.069 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-16T09:34:01.069 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei 2023-10-16T09:34:01.069 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-me.ko 2023-10-16T09:34:01.069 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-16T09:34:01.070 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei.ko 2023-10-16T09:34:01.070 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru 2023-10-16T09:34:01.070 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-16T09:34:01.070 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp 2023-10-16T09:34:01.071 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-16T09:34:01.071 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-16T09:34:01.071 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-16T09:34:01.071 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-16T09:34:01.071 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_core.ko 2023-10-16T09:34:01.072 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tsl2550.ko 2023-10-16T09:34:01.072 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_balloon.ko 2023-10-16T09:34:01.072 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci 2023-10-16T09:34:01.073 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-16T09:34:01.073 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net 2023-10-16T09:34:01.073 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding 2023-10-16T09:34:01.073 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/bonding.ko 2023-10-16T09:34:01.073 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa 2023-10-16T09:34:01.074 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53 2023-10-16T09:34:01.074 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-16T09:34:01.074 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-16T09:34:01.074 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-16T09:34:01.075 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-16T09:34:01.075 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-16T09:34:01.075 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dummy.ko 2023-10-16T09:34:01.075 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/eql.ko 2023-10-16T09:34:01.075 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet 2023-10-16T09:34:01.076 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com 2023-10-16T09:34:01.076 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-16T09:34:01.076 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-16T09:34:01.076 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-16T09:34:01.076 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-16T09:34:01.077 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390 2023-10-16T09:34:01.077 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-16T09:34:01.077 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-16T09:34:01.077 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-16T09:34:01.078 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-16T09:34:01.078 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec 2023-10-16T09:34:01.078 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-16T09:34:01.078 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere 2023-10-16T09:34:01.078 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-16T09:34:01.079 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon 2023-10-16T09:34:01.079 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-16T09:34:01.079 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera 2023-10-16T09:34:01.079 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-16T09:34:01.079 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd 2023-10-16T09:34:01.080 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-16T09:34:01.080 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-16T09:34:01.080 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-16T09:34:01.080 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros 2023-10-16T09:34:01.080 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx 2023-10-16T09:34:01.081 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-16T09:34:01.081 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-16T09:34:01.081 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-16T09:34:01.081 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-16T09:34:01.082 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-16T09:34:01.082 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-16T09:34:01.082 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-16T09:34:01.082 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-16T09:34:01.082 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom 2023-10-16T09:34:01.083 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-16T09:34:01.083 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-16T09:34:01.083 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-16T09:34:01.083 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-16T09:34:01.084 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-16T09:34:01.084 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-16T09:34:01.084 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-16T09:34:01.084 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-16T09:34:01.084 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade 2023-10-16T09:34:01.085 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna 2023-10-16T09:34:01.085 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-16T09:34:01.085 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio 2023-10-16T09:34:01.085 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-16T09:34:01.085 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-16T09:34:01.086 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-16T09:34:01.086 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-16T09:34:01.086 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-16T09:34:01.087 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-16T09:34:01.087 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-16T09:34:01.087 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-16T09:34:01.087 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-16T09:34:01.088 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-16T09:34:01.088 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco 2023-10-16T09:34:01.088 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic 2023-10-16T09:34:01.088 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-16T09:34:01.088 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec 2023-10-16T09:34:01.089 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip 2023-10-16T09:34:01.089 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-16T09:34:01.089 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-16T09:34:01.089 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-16T09:34:01.089 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-16T09:34:01.090 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-16T09:34:01.090 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-16T09:34:01.090 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink 2023-10-16T09:34:01.090 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-16T09:34:01.091 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-16T09:34:01.091 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dnet.ko 2023-10-16T09:34:01.091 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex 2023-10-16T09:34:01.091 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet 2023-10-16T09:34:01.091 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-16T09:34:01.092 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-16T09:34:01.092 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-16T09:34:01.092 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel 2023-10-16T09:34:01.092 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-16T09:34:01.093 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000 2023-10-16T09:34:01.093 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-16T09:34:01.093 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-16T09:34:01.093 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-16T09:34:01.094 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-16T09:34:01.094 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-16T09:34:01.094 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e 2023-10-16T09:34:01.094 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-16T09:34:01.095 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf 2023-10-16T09:34:01.095 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-16T09:34:01.095 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb 2023-10-16T09:34:01.095 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-16T09:34:01.095 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-16T09:34:01.096 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-16T09:34:01.096 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-16T09:34:01.096 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-16T09:34:01.096 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-16T09:34:01.096 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-16T09:34:01.097 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/jme.ko 2023-10-16T09:34:01.097 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell 2023-10-16T09:34:01.097 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-16T09:34:01.097 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-16T09:34:01.098 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-16T09:34:01.098 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox 2023-10-16T09:34:01.098 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-16T09:34:01.098 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-16T09:34:01.098 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-16T09:34:01.099 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-16T09:34:01.099 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-16T09:34:01.099 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-16T09:34:01.099 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel 2023-10-16T09:34:01.099 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-16T09:34:01.100 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom 2023-10-16T09:34:01.100 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-16T09:34:01.100 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-16T09:34:01.100 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi 2023-10-16T09:34:01.101 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-16T09:34:01.101 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-16T09:34:01.101 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion 2023-10-16T09:34:01.101 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-16T09:34:01.101 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia 2023-10-16T09:34:01.102 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-16T09:34:01.102 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines 2023-10-16T09:34:01.102 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-16T09:34:01.102 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-16T09:34:01.102 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic 2023-10-16T09:34:01.103 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-16T09:34:01.103 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-16T09:34:01.103 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-16T09:34:01.103 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-16T09:34:01.104 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-16T09:34:01.104 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc 2023-10-16T09:34:01.104 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-16T09:34:01.104 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek 2023-10-16T09:34:01.105 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-16T09:34:01.105 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-16T09:34:01.105 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-16T09:34:01.105 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-16T09:34:01.105 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker 2023-10-16T09:34:01.106 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-16T09:34:01.106 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc 2023-10-16T09:34:01.106 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-16T09:34:01.106 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan 2023-10-16T09:34:01.107 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-16T09:34:01.107 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis 2023-10-16T09:34:01.107 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-16T09:34:01.107 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-16T09:34:01.108 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc 2023-10-16T09:34:01.108 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-16T09:34:01.108 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-16T09:34:01.108 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-16T09:34:01.108 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-16T09:34:01.109 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro 2023-10-16T09:34:01.109 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-16T09:34:01.109 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-16T09:34:01.109 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-16T09:34:01.110 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun 2023-10-16T09:34:01.110 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-16T09:34:01.110 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-16T09:34:01.110 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-16T09:34:01.110 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-16T09:34:01.111 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti 2023-10-16T09:34:01.111 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-16T09:34:01.111 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti 2023-10-16T09:34:01.111 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-16T09:34:01.111 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via 2023-10-16T09:34:01.112 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-16T09:34:01.112 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-16T09:34:01.112 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet 2023-10-16T09:34:01.112 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-16T09:34:01.113 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-16T09:34:01.113 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom 2023-10-16T09:34:01.113 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-16T09:34:01.113 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/geneve.ko 2023-10-16T09:34:01.113 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv 2023-10-16T09:34:01.114 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-16T09:34:01.114 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154 2023-10-16T09:34:01.114 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-16T09:34:01.114 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ifb.ko 2023-10-16T09:34:01.114 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan 2023-10-16T09:34:01.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-16T09:34:01.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvlan.ko 2023-10-16T09:34:01.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvtap.ko 2023-10-16T09:34:01.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio 2023-10-16T09:34:01.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio.ko 2023-10-16T09:34:01.116 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-16T09:34:01.116 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-16T09:34:01.116 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-16T09:34:01.116 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mii.ko 2023-10-16T09:34:01.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/net_failover.ko 2023-10-16T09:34:01.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/netconsole.ko 2023-10-16T09:34:01.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/nlmon.ko 2023-10-16T09:34:01.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs 2023-10-16T09:34:01.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-16T09:34:01.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-16T09:34:01.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy 2023-10-16T09:34:01.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/amd.ko 2023-10-16T09:34:01.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-16T09:34:01.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-16T09:34:01.119 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-16T09:34:01.119 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-16T09:34:01.119 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/broadcom.ko 2023-10-16T09:34:01.119 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/cicada.ko 2023-10-16T09:34:01.120 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/davicom.ko 2023-10-16T09:34:01.120 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/dp83640.ko 2023-10-16T09:34:01.120 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/et1011c.ko 2023-10-16T09:34:01.120 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/icplus.ko 2023-10-16T09:34:01.120 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/lxt.ko 2023-10-16T09:34:01.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/marvell.ko 2023-10-16T09:34:01.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/micrel.ko 2023-10-16T09:34:01.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/national.ko 2023-10-16T09:34:01.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/phylink.ko 2023-10-16T09:34:01.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/qsemi.ko 2023-10-16T09:34:01.122 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/realtek.ko 2023-10-16T09:34:01.122 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/smsc.ko 2023-10-16T09:34:01.122 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-16T09:34:01.122 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/vitesse.ko 2023-10-16T09:34:01.122 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp 2023-10-16T09:34:01.123 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-16T09:34:01.123 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-16T09:34:01.123 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-16T09:34:01.123 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-16T09:34:01.124 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-16T09:34:01.124 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-16T09:34:01.124 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppoe.ko 2023-10-16T09:34:01.124 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppox.ko 2023-10-16T09:34:01.125 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pptp.ko 2023-10-16T09:34:01.125 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip 2023-10-16T09:34:01.125 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slhc.ko 2023-10-16T09:34:01.125 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slip.ko 2023-10-16T09:34:01.125 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sungem_phy.ko 2023-10-16T09:34:01.126 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tap.ko 2023-10-16T09:34:01.126 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team 2023-10-16T09:34:01.126 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team.ko 2023-10-16T09:34:01.126 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-16T09:34:01.127 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-16T09:34:01.127 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-16T09:34:01.127 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_random.ko 2023-10-16T09:34:01.127 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-16T09:34:01.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tun.ko 2023-10-16T09:34:01.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/veth.ko 2023-10-16T09:34:01.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/virtio_net.ko 2023-10-16T09:34:01.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3 2023-10-16T09:34:01.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-16T09:34:01.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan 2023-10-16T09:34:01.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-16T09:34:01.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback 2023-10-16T09:34:01.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-16T09:34:01.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netfront.ko 2023-10-16T09:34:01.130 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb 2023-10-16T09:34:01.130 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ntb.ko 2023-10-16T09:34:01.130 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme 2023-10-16T09:34:01.130 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host 2023-10-16T09:34:01.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-16T09:34:01.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme.ko 2023-10-16T09:34:01.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport 2023-10-16T09:34:01.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport.ko 2023-10-16T09:34:01.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_cs.ko 2023-10-16T09:34:01.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_pc.ko 2023-10-16T09:34:01.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_serial.ko 2023-10-16T09:34:01.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci 2023-10-16T09:34:01.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug 2023-10-16T09:34:01.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-16T09:34:01.133 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie 2023-10-16T09:34:01.133 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-16T09:34:01.133 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/xen-pcifront.ko 2023-10-16T09:34:01.133 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia 2023-10-16T09:34:01.134 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/i82092.ko 2023-10-16T09:34:01.134 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pd6729.ko 2023-10-16T09:34:01.134 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-16T09:34:01.134 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl 2023-10-16T09:34:01.134 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel 2023-10-16T09:34:01.135 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-16T09:34:01.135 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform 2023-10-16T09:34:01.135 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86 2023-10-16T09:34:01.135 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-16T09:34:01.135 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-16T09:34:01.136 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-16T09:34:01.136 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-16T09:34:01.136 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-16T09:34:01.136 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-16T09:34:01.136 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-16T09:34:01.137 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-16T09:34:01.137 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-16T09:34:01.137 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-16T09:34:01.137 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-16T09:34:01.138 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/hdaps.ko 2023-10-16T09:34:01.138 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel 2023-10-16T09:34:01.138 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-16T09:34:01.138 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-16T09:34:01.138 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-16T09:34:01.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-16T09:34:01.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-16T09:34:01.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-16T09:34:01.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-16T09:34:01.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-16T09:34:01.140 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-16T09:34:01.141 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-16T09:34:01.141 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-16T09:34:01.142 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi.ko 2023-10-16T09:34:01.142 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap 2023-10-16T09:34:01.142 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-16T09:34:01.142 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-16T09:34:01.142 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps 2023-10-16T09:34:01.143 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients 2023-10-16T09:34:01.143 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-16T09:34:01.143 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-16T09:34:01.143 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-16T09:34:01.143 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/pps_core.ko 2023-10-16T09:34:01.144 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp 2023-10-16T09:34:01.144 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp.ko 2023-10-16T09:34:01.144 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-16T09:34:01.144 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc 2023-10-16T09:34:01.145 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-16T09:34:01.145 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-16T09:34:01.145 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-16T09:34:01.145 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-16T09:34:01.145 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-16T09:34:01.146 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-16T09:34:01.146 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-16T09:34:01.146 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-16T09:34:01.146 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-16T09:34:01.147 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-16T09:34:01.147 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-16T09:34:01.147 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-16T09:34:01.147 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-16T09:34:01.147 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-16T09:34:01.148 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-16T09:34:01.148 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-16T09:34:01.148 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-16T09:34:01.148 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-16T09:34:01.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-16T09:34:01.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-16T09:34:01.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-16T09:34:01.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-16T09:34:01.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-16T09:34:01.150 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-16T09:34:01.150 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-16T09:34:01.150 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-16T09:34:01.150 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-16T09:34:01.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-16T09:34:01.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-16T09:34:01.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-16T09:34:01.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-16T09:34:01.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi 2023-10-16T09:34:01.152 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-16T09:34:01.152 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-sas.ko 2023-10-16T09:34:01.152 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-16T09:34:01.152 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/BusLogic.ko 2023-10-16T09:34:01.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/a100u2w.ko 2023-10-16T09:34:01.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid 2023-10-16T09:34:01.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-16T09:34:01.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/advansys.ko 2023-10-16T09:34:01.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx 2023-10-16T09:34:01.154 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-16T09:34:01.154 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-16T09:34:01.154 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/am53c974.ko 2023-10-16T09:34:01.154 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr 2023-10-16T09:34:01.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-16T09:34:01.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/atp870u.ko 2023-10-16T09:34:01.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi 2023-10-16T09:34:01.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-16T09:34:01.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa 2023-10-16T09:34:01.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-16T09:34:01.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc 2023-10-16T09:34:01.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-16T09:34:01.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i 2023-10-16T09:34:01.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-16T09:34:01.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ch.ko 2023-10-16T09:34:01.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor 2023-10-16T09:34:01.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-16T09:34:01.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi 2023-10-16T09:34:01.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-16T09:34:01.158 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-16T09:34:01.158 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-16T09:34:01.158 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-16T09:34:01.158 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-16T09:34:01.159 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dc395x.ko 2023-10-16T09:34:01.159 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler 2023-10-16T09:34:01.159 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-16T09:34:01.159 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-16T09:34:01.159 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-16T09:34:01.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-16T09:34:01.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dmx3191d.ko 2023-10-16T09:34:01.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r 2023-10-16T09:34:01.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-16T09:34:01.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esp_scsi.ko 2023-10-16T09:34:01.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe 2023-10-16T09:34:01.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-16T09:34:01.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-16T09:34:01.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic 2023-10-16T09:34:01.162 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-16T09:34:01.162 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hpsa.ko 2023-10-16T09:34:01.162 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hptiop.ko 2023-10-16T09:34:01.162 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-16T09:34:01.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/initio.ko 2023-10-16T09:34:01.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ipr.ko 2023-10-16T09:34:01.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ips.ko 2023-10-16T09:34:01.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci 2023-10-16T09:34:01.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/isci.ko 2023-10-16T09:34:01.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-16T09:34:01.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-16T09:34:01.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc 2023-10-16T09:34:01.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-16T09:34:01.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi.ko 2023-10-16T09:34:01.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-16T09:34:01.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas 2023-10-16T09:34:01.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-16T09:34:01.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc 2023-10-16T09:34:01.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-16T09:34:01.166 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid 2023-10-16T09:34:01.166 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid.ko 2023-10-16T09:34:01.166 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-16T09:34:01.166 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-16T09:34:01.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-16T09:34:01.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas 2023-10-16T09:34:01.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-16T09:34:01.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas 2023-10-16T09:34:01.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-16T09:34:01.168 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvumi.ko 2023-10-16T09:34:01.168 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001 2023-10-16T09:34:01.168 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-16T09:34:01.168 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pmcraid.ko 2023-10-16T09:34:01.168 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla1280.ko 2023-10-16T09:34:01.169 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx 2023-10-16T09:34:01.169 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-16T09:34:01.169 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-16T09:34:01.169 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx 2023-10-16T09:34:01.170 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-16T09:34:01.170 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/raid_class.ko 2023-10-16T09:34:01.170 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_debug.ko 2023-10-16T09:34:01.170 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-16T09:34:01.171 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-16T09:34:01.171 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-16T09:34:01.171 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-16T09:34:01.171 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-16T09:34:01.171 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ses.ko 2023-10-16T09:34:01.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/st.ko 2023-10-16T09:34:01.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/stex.ko 2023-10-16T09:34:01.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2 2023-10-16T09:34:01.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-16T09:34:01.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-16T09:34:01.173 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-16T09:34:01.173 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/wd719x.ko 2023-10-16T09:34:01.173 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb 2023-10-16T09:34:01.173 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ssb.ko 2023-10-16T09:34:01.173 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target 2023-10-16T09:34:01.174 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi 2023-10-16T09:34:01.174 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-16T09:34:01.174 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback 2023-10-16T09:34:01.174 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-16T09:34:01.174 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_file.ko 2023-10-16T09:34:01.175 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_iblock.ko 2023-10-16T09:34:01.175 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_mod.ko 2023-10-16T09:34:01.175 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_pscsi.ko 2023-10-16T09:34:01.175 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_user.ko 2023-10-16T09:34:01.176 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc 2023-10-16T09:34:01.176 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-16T09:34:01.176 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal 2023-10-16T09:34:01.176 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel 2023-10-16T09:34:01.176 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-16T09:34:01.177 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-16T09:34:01.177 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-16T09:34:01.177 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-16T09:34:01.177 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-16T09:34:01.177 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-16T09:34:01.178 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-16T09:34:01.178 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-16T09:34:01.178 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-16T09:34:01.178 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-16T09:34:01.178 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-16T09:34:01.179 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-16T09:34:01.179 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-16T09:34:01.179 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-16T09:34:01.179 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-16T09:34:01.179 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-16T09:34:01.180 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty 2023-10-16T09:34:01.180 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless 2023-10-16T09:34:01.180 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-16T09:34:01.180 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_gsm.ko 2023-10-16T09:34:01.181 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_hdlc.ko 2023-10-16T09:34:01.181 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/nozomi.ko 2023-10-16T09:34:01.181 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial 2023-10-16T09:34:01.181 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250 2023-10-16T09:34:01.181 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-16T09:34:01.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-16T09:34:01.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm 2023-10-16T09:34:01.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-16T09:34:01.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs 2023-10-16T09:34:01.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core 2023-10-16T09:34:01.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-16T09:34:01.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host 2023-10-16T09:34:01.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-16T09:34:01.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio 2023-10-16T09:34:01.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio.ko 2023-10-16T09:34:01.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_aec.ko 2023-10-16T09:34:01.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_cif.ko 2023-10-16T09:34:01.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-16T09:34:01.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_sercos3.ko 2023-10-16T09:34:01.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb 2023-10-16T09:34:01.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class 2023-10-16T09:34:01.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-16T09:34:01.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-16T09:34:01.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usblp.ko 2023-10-16T09:34:01.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usbtmc.ko 2023-10-16T09:34:01.186 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host 2023-10-16T09:34:01.186 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-16T09:34:01.186 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image 2023-10-16T09:34:01.186 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/mdc800.ko 2023-10-16T09:34:01.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/microtek.ko 2023-10-16T09:34:01.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc 2023-10-16T09:34:01.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/adutux.ko 2023-10-16T09:34:01.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-16T09:34:01.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi26.ko 2023-10-16T09:34:01.188 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi62.ko 2023-10-16T09:34:01.188 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ezusb.ko 2023-10-16T09:34:01.188 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/idmouse.ko 2023-10-16T09:34:01.188 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-16T09:34:01.188 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-16T09:34:01.189 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ldusb.ko 2023-10-16T09:34:01.189 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-16T09:34:01.189 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga 2023-10-16T09:34:01.189 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-16T09:34:01.189 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-16T09:34:01.190 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usb3503.ko 2023-10-16T09:34:01.190 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usblcd.ko 2023-10-16T09:34:01.190 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-16T09:34:01.190 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/uss720.ko 2023-10-16T09:34:01.190 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/yurex.ko 2023-10-16T09:34:01.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy 2023-10-16T09:34:01.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-16T09:34:01.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial 2023-10-16T09:34:01.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/aircable.ko 2023-10-16T09:34:01.192 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ark3116.ko 2023-10-16T09:34:01.192 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-16T09:34:01.192 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ch341.ko 2023-10-16T09:34:01.192 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cp210x.ko 2023-10-16T09:34:01.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-16T09:34:01.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-16T09:34:01.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-16T09:34:01.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/empeg.ko 2023-10-16T09:34:01.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-16T09:34:01.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-16T09:34:01.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-16T09:34:01.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_ti.ko 2023-10-16T09:34:01.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipaq.ko 2023-10-16T09:34:01.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipw.ko 2023-10-16T09:34:01.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-16T09:34:01.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-16T09:34:01.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan.ko 2023-10-16T09:34:01.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-16T09:34:01.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-16T09:34:01.196 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-16T09:34:01.196 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-16T09:34:01.196 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7720.ko 2023-10-16T09:34:01.196 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7840.ko 2023-10-16T09:34:01.196 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/navman.ko 2023-10-16T09:34:01.197 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/omninet.ko 2023-10-16T09:34:01.197 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/opticon.ko 2023-10-16T09:34:01.197 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/option.ko 2023-10-16T09:34:01.197 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/oti6858.ko 2023-10-16T09:34:01.198 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/pl2303.ko 2023-10-16T09:34:01.198 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcaux.ko 2023-10-16T09:34:01.198 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcserial.ko 2023-10-16T09:34:01.198 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/quatech2.ko 2023-10-16T09:34:01.198 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-16T09:34:01.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/sierra.ko 2023-10-16T09:34:01.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-16T09:34:01.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ssu100.ko 2023-10-16T09:34:01.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-16T09:34:01.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-16T09:34:01.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-16T09:34:01.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-16T09:34:01.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-16T09:34:01.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/visor.ko 2023-10-16T09:34:01.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-16T09:34:01.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-16T09:34:01.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage 2023-10-16T09:34:01.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/uas.ko 2023-10-16T09:34:01.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-16T09:34:01.202 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-16T09:34:01.202 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-16T09:34:01.202 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-16T09:34:01.202 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-16T09:34:01.202 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-16T09:34:01.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-16T09:34:01.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-16T09:34:01.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-16T09:34:01.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-16T09:34:01.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-16T09:34:01.204 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-16T09:34:01.204 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-16T09:34:01.204 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-16T09:34:01.204 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip 2023-10-16T09:34:01.204 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-16T09:34:01.205 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-16T09:34:01.205 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-16T09:34:01.205 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio 2023-10-16T09:34:01.205 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci 2023-10-16T09:34:01.205 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-16T09:34:01.206 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-16T09:34:01.206 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio.ko 2023-10-16T09:34:01.206 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-16T09:34:01.206 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost 2023-10-16T09:34:01.206 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost.ko 2023-10-16T09:34:01.207 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-16T09:34:01.207 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_net.ko 2023-10-16T09:34:01.207 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-16T09:34:01.207 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video 2023-10-16T09:34:01.208 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight 2023-10-16T09:34:01.208 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-16T09:34:01.208 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lcd.ko 2023-10-16T09:34:01.208 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-16T09:34:01.208 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio 2023-10-16T09:34:01.209 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio.ko 2023-10-16T09:34:01.209 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-16T09:34:01.209 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-16T09:34:01.209 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci.ko 2023-10-16T09:34:01.210 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-16T09:34:01.210 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-16T09:34:01.210 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_ring.ko 2023-10-16T09:34:01.210 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1 2023-10-16T09:34:01.210 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters 2023-10-16T09:34:01.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2482.ko 2023-10-16T09:34:01.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2490.ko 2023-10-16T09:34:01.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves 2023-10-16T09:34:01.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-16T09:34:01.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-16T09:34:01.212 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-16T09:34:01.212 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-16T09:34:01.212 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-16T09:34:01.212 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-16T09:34:01.213 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-16T09:34:01.213 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-16T09:34:01.213 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-16T09:34:01.213 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-16T09:34:01.214 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-16T09:34:01.214 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/wire.ko 2023-10-16T09:34:01.214 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog 2023-10-16T09:34:01.214 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-16T09:34:01.214 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-16T09:34:01.215 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-16T09:34:01.215 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/hpwdt.ko 2023-10-16T09:34:01.215 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/i6300esb.ko 2023-10-16T09:34:01.215 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-16T09:34:01.215 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-16T09:34:01.216 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-16T09:34:01.216 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ibmasr.ko 2023-10-16T09:34:01.216 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-16T09:34:01.216 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-16T09:34:01.216 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-16T09:34:01.217 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/machzwd.ko 2023-10-16T09:34:01.217 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/nv_tco.ko 2023-10-16T09:34:01.217 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-16T09:34:01.217 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-16T09:34:01.217 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-16T09:34:01.218 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-16T09:34:01.218 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/softdog.ko 2023-10-16T09:34:01.218 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-16T09:34:01.218 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/via_wdt.ko 2023-10-16T09:34:01.219 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-16T09:34:01.219 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-16T09:34:01.219 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-16T09:34:01.219 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-16T09:34:01.219 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-16T09:34:01.220 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen 2023-10-16T09:34:01.220 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-16T09:34:01.220 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-evtchn.ko 2023-10-16T09:34:01.220 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-16T09:34:01.221 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntdev.ko 2023-10-16T09:34:01.221 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback 2023-10-16T09:34:01.221 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-16T09:34:01.221 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-privcmd.ko 2023-10-16T09:34:01.221 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs 2023-10-16T09:34:01.222 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-16T09:34:01.222 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs 2023-10-16T09:34:01.222 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p 2023-10-16T09:34:01.222 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/9p.ko 2023-10-16T09:34:01.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs 2023-10-16T09:34:01.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/affs.ko 2023-10-16T09:34:01.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs 2023-10-16T09:34:01.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/befs.ko 2023-10-16T09:34:01.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/binfmt_misc.ko 2023-10-16T09:34:01.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs 2023-10-16T09:34:01.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/btrfs.ko 2023-10-16T09:34:01.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles 2023-10-16T09:34:01.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/cachefiles.ko 2023-10-16T09:34:01.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph 2023-10-16T09:34:01.225 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ceph.ko 2023-10-16T09:34:01.225 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda 2023-10-16T09:34:01.225 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/coda.ko 2023-10-16T09:34:01.225 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs 2023-10-16T09:34:01.225 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/cramfs.ko 2023-10-16T09:34:01.226 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm 2023-10-16T09:34:01.226 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/dlm.ko 2023-10-16T09:34:01.226 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs 2023-10-16T09:34:01.226 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-16T09:34:01.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs 2023-10-16T09:34:01.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/f2fs.ko 2023-10-16T09:34:01.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat 2023-10-16T09:34:01.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/fat.ko 2023-10-16T09:34:01.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/msdos.ko 2023-10-16T09:34:01.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/vfat.ko 2023-10-16T09:34:01.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache 2023-10-16T09:34:01.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/fscache.ko 2023-10-16T09:34:01.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse 2023-10-16T09:34:01.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/cuse.ko 2023-10-16T09:34:01.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/fuse.ko 2023-10-16T09:34:01.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2 2023-10-16T09:34:01.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/gfs2.ko 2023-10-16T09:34:01.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs 2023-10-16T09:34:01.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/hfs.ko 2023-10-16T09:34:01.230 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus 2023-10-16T09:34:01.230 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/hfsplus.ko 2023-10-16T09:34:01.230 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs 2023-10-16T09:34:01.230 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/isofs.ko 2023-10-16T09:34:01.230 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs 2023-10-16T09:34:01.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/jfs.ko 2023-10-16T09:34:01.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd 2023-10-16T09:34:01.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/lockd.ko 2023-10-16T09:34:01.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix 2023-10-16T09:34:01.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/minix.ko 2023-10-16T09:34:01.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs 2023-10-16T09:34:01.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/netfs.ko 2023-10-16T09:34:01.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs 2023-10-16T09:34:01.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout 2023-10-16T09:34:01.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-16T09:34:01.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout 2023-10-16T09:34:01.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-16T09:34:01.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout 2023-10-16T09:34:01.234 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-16T09:34:01.234 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfs.ko 2023-10-16T09:34:01.234 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv3.ko 2023-10-16T09:34:01.234 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv4.ko 2023-10-16T09:34:01.235 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common 2023-10-16T09:34:01.235 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/grace.ko 2023-10-16T09:34:01.235 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-16T09:34:01.235 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd 2023-10-16T09:34:01.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/nfsd.ko 2023-10-16T09:34:01.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2 2023-10-16T09:34:01.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/nilfs2.ko 2023-10-16T09:34:01.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls 2023-10-16T09:34:01.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-celtic.ko 2023-10-16T09:34:01.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-centeuro.ko 2023-10-16T09:34:01.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-croatian.ko 2023-10-16T09:34:01.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-cyrillic.ko 2023-10-16T09:34:01.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-gaelic.ko 2023-10-16T09:34:01.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-greek.ko 2023-10-16T09:34:01.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-iceland.ko 2023-10-16T09:34:01.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-inuit.ko 2023-10-16T09:34:01.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-roman.ko 2023-10-16T09:34:01.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-romanian.ko 2023-10-16T09:34:01.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-turkish.ko 2023-10-16T09:34:01.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1250.ko 2023-10-16T09:34:01.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1251.ko 2023-10-16T09:34:01.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1255.ko 2023-10-16T09:34:01.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp737.ko 2023-10-16T09:34:01.240 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp775.ko 2023-10-16T09:34:01.240 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp850.ko 2023-10-16T09:34:01.240 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp852.ko 2023-10-16T09:34:01.240 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp855.ko 2023-10-16T09:34:01.240 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp857.ko 2023-10-16T09:34:01.241 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp860.ko 2023-10-16T09:34:01.241 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp861.ko 2023-10-16T09:34:01.241 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp862.ko 2023-10-16T09:34:01.241 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp863.ko 2023-10-16T09:34:01.242 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp864.ko 2023-10-16T09:34:01.242 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp865.ko 2023-10-16T09:34:01.242 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp866.ko 2023-10-16T09:34:01.242 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp869.ko 2023-10-16T09:34:01.242 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp874.ko 2023-10-16T09:34:01.243 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp932.ko 2023-10-16T09:34:01.243 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp936.ko 2023-10-16T09:34:01.243 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp949.ko 2023-10-16T09:34:01.243 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp950.ko 2023-10-16T09:34:01.243 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_euc-jp.ko 2023-10-16T09:34:01.244 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-16T09:34:01.244 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-16T09:34:01.244 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-16T09:34:01.244 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-16T09:34:01.244 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-16T09:34:01.245 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-16T09:34:01.245 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-16T09:34:01.245 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-16T09:34:01.245 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-16T09:34:01.245 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-16T09:34:01.246 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-16T09:34:01.246 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-r.ko 2023-10-16T09:34:01.246 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-16T09:34:01.246 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-u.ko 2023-10-16T09:34:01.247 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-16T09:34:01.247 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_utf8.ko 2023-10-16T09:34:01.247 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2 2023-10-16T09:34:01.247 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster 2023-10-16T09:34:01.247 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-16T09:34:01.248 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm 2023-10-16T09:34:01.248 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-16T09:34:01.248 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs 2023-10-16T09:34:01.248 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-16T09:34:01.248 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2.ko 2023-10-16T09:34:01.249 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-16T09:34:01.249 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-16T09:34:01.249 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-16T09:34:01.249 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs 2023-10-16T09:34:01.249 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/overlay.ko 2023-10-16T09:34:01.250 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore 2023-10-16T09:34:01.250 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ramoops.ko 2023-10-16T09:34:01.250 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs 2023-10-16T09:34:01.250 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/reiserfs.ko 2023-10-16T09:34:01.250 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs 2023-10-16T09:34:01.251 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/romfs.ko 2023-10-16T09:34:01.251 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb 2023-10-16T09:34:01.251 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client 2023-10-16T09:34:01.251 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/cifs.ko 2023-10-16T09:34:01.252 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common 2023-10-16T09:34:01.252 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-16T09:34:01.252 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_md4.ko 2023-10-16T09:34:01.252 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs 2023-10-16T09:34:01.253 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/squashfs.ko 2023-10-16T09:34:01.253 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv 2023-10-16T09:34:01.253 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/sysv.ko 2023-10-16T09:34:01.253 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf 2023-10-16T09:34:01.253 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/udf.ko 2023-10-16T09:34:01.254 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs 2023-10-16T09:34:01.254 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ufs.ko 2023-10-16T09:34:01.254 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs 2023-10-16T09:34:01.254 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/xfs.ko 2023-10-16T09:34:01.254 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel 2023-10-16T09:34:01.255 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu 2023-10-16T09:34:01.255 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu/rcutorture.ko 2023-10-16T09:34:01.255 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/torture.ko 2023-10-16T09:34:01.255 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace 2023-10-16T09:34:01.256 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-16T09:34:01.256 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib 2023-10-16T09:34:01.256 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/asn1_encoder.ko 2023-10-16T09:34:01.256 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc-itu-t.ko 2023-10-16T09:34:01.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc8.ko 2023-10-16T09:34:01.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto 2023-10-16T09:34:01.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libarc4.ko 2023-10-16T09:34:01.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libdes.ko 2023-10-16T09:34:01.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/libcrc32c.ko 2023-10-16T09:34:01.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lru_cache.ko 2023-10-16T09:34:01.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4 2023-10-16T09:34:01.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4_compress.ko 2023-10-16T09:34:01.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-16T09:34:01.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math 2023-10-16T09:34:01.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/cordic.ko 2023-10-16T09:34:01.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6 2023-10-16T09:34:01.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/raid6_pq.ko 2023-10-16T09:34:01.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon 2023-10-16T09:34:01.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-16T09:34:01.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_bm.ko 2023-10-16T09:34:01.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_fsm.ko 2023-10-16T09:34:01.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_kmp.ko 2023-10-16T09:34:01.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd 2023-10-16T09:34:01.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd/zstd_compress.ko 2023-10-16T09:34:01.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm 2023-10-16T09:34:01.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/hwpoison-inject.ko 2023-10-16T09:34:01.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net 2023-10-16T09:34:01.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan 2023-10-16T09:34:01.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/6lowpan.ko 2023-10-16T09:34:01.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_dest.ko 2023-10-16T09:34:01.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-16T09:34:01.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_hop.ko 2023-10-16T09:34:01.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-16T09:34:01.263 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-16T09:34:01.263 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_routing.ko 2023-10-16T09:34:01.263 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_udp.ko 2023-10-16T09:34:01.263 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802 2023-10-16T09:34:01.263 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/garp.ko 2023-10-16T09:34:01.264 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/mrp.ko 2023-10-16T09:34:01.264 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/p8022.ko 2023-10-16T09:34:01.264 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/psnap.ko 2023-10-16T09:34:01.264 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/stp.ko 2023-10-16T09:34:01.264 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q 2023-10-16T09:34:01.265 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/8021q.ko 2023-10-16T09:34:01.265 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p 2023-10-16T09:34:01.265 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet.ko 2023-10-16T09:34:01.265 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_fd.ko 2023-10-16T09:34:01.266 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_rdma.ko 2023-10-16T09:34:01.266 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_virtio.ko 2023-10-16T09:34:01.266 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv 2023-10-16T09:34:01.266 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/batman-adv.ko 2023-10-16T09:34:01.266 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge 2023-10-16T09:34:01.267 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/br_netfilter.ko 2023-10-16T09:34:01.267 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/bridge.ko 2023-10-16T09:34:01.267 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter 2023-10-16T09:34:01.267 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-16T09:34:01.267 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-16T09:34:01.268 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-16T09:34:01.268 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-16T09:34:01.268 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-16T09:34:01.268 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-16T09:34:01.268 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-16T09:34:01.269 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-16T09:34:01.269 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-16T09:34:01.269 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-16T09:34:01.269 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-16T09:34:01.269 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-16T09:34:01.270 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-16T09:34:01.270 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-16T09:34:01.270 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-16T09:34:01.270 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-16T09:34:01.271 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-16T09:34:01.271 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-16T09:34:01.271 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-16T09:34:01.271 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-16T09:34:01.271 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-16T09:34:01.272 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-16T09:34:01.272 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-16T09:34:01.272 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-16T09:34:01.272 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph 2023-10-16T09:34:01.272 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/libceph.ko 2023-10-16T09:34:01.273 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core 2023-10-16T09:34:01.273 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/failover.ko 2023-10-16T09:34:01.273 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/pktgen.ko 2023-10-16T09:34:01.274 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp 2023-10-16T09:34:01.274 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp.ko 2023-10-16T09:34:01.274 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_diag.ko 2023-10-16T09:34:01.274 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv4.ko 2023-10-16T09:34:01.274 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv6.ko 2023-10-16T09:34:01.275 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver 2023-10-16T09:34:01.275 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-16T09:34:01.275 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa 2023-10-16T09:34:01.276 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/dsa_core.ko 2023-10-16T09:34:01.276 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_brcm.ko 2023-10-16T09:34:01.276 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_dsa.ko 2023-10-16T09:34:01.276 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_none.ko 2023-10-16T09:34:01.276 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_trailer.ko 2023-10-16T09:34:01.277 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154 2023-10-16T09:34:01.277 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan 2023-10-16T09:34:01.277 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-16T09:34:01.277 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154.ko 2023-10-16T09:34:01.278 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-16T09:34:01.278 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4 2023-10-16T09:34:01.278 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ah4.ko 2023-10-16T09:34:01.278 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/esp4.ko 2023-10-16T09:34:01.278 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/fou.ko 2023-10-16T09:34:01.279 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/gre.ko 2023-10-16T09:34:01.279 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/inet_diag.ko 2023-10-16T09:34:01.279 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_gre.ko 2023-10-16T09:34:01.279 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_tunnel.ko 2023-10-16T09:34:01.279 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_vti.ko 2023-10-16T09:34:01.280 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipcomp.ko 2023-10-16T09:34:01.280 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipip.ko 2023-10-16T09:34:01.280 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter 2023-10-16T09:34:01.280 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-16T09:34:01.280 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-16T09:34:01.281 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-16T09:34:01.281 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-16T09:34:01.281 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-16T09:34:01.281 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-16T09:34:01.281 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-16T09:34:01.282 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-16T09:34:01.282 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-16T09:34:01.282 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-16T09:34:01.282 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-16T09:34:01.282 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-16T09:34:01.283 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-16T09:34:01.283 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-16T09:34:01.283 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-16T09:34:01.283 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-16T09:34:01.283 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-16T09:34:01.284 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-16T09:34:01.284 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-16T09:34:01.284 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-16T09:34:01.284 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-16T09:34:01.285 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_bic.ko 2023-10-16T09:34:01.285 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-16T09:34:01.285 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_diag.ko 2023-10-16T09:34:01.285 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-16T09:34:01.285 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_htcp.ko 2023-10-16T09:34:01.286 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_hybla.ko 2023-10-16T09:34:01.286 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_illinois.ko 2023-10-16T09:34:01.286 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_lp.ko 2023-10-16T09:34:01.286 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_scalable.ko 2023-10-16T09:34:01.286 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_vegas.ko 2023-10-16T09:34:01.287 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_veno.ko 2023-10-16T09:34:01.287 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_westwood.ko 2023-10-16T09:34:01.287 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_yeah.ko 2023-10-16T09:34:01.287 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tunnel4.ko 2023-10-16T09:34:01.287 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_diag.ko 2023-10-16T09:34:01.288 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_tunnel.ko 2023-10-16T09:34:01.288 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-16T09:34:01.288 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6 2023-10-16T09:34:01.288 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ah6.ko 2023-10-16T09:34:01.289 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/esp6.ko 2023-10-16T09:34:01.289 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/fou6.ko 2023-10-16T09:34:01.289 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-16T09:34:01.289 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-16T09:34:01.289 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_vti.ko 2023-10-16T09:34:01.290 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ipcomp6.ko 2023-10-16T09:34:01.290 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter 2023-10-16T09:34:01.290 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-16T09:34:01.290 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-16T09:34:01.290 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-16T09:34:01.291 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-16T09:34:01.291 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-16T09:34:01.291 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-16T09:34:01.291 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-16T09:34:01.291 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-16T09:34:01.292 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-16T09:34:01.292 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-16T09:34:01.292 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-16T09:34:01.292 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-16T09:34:01.293 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-16T09:34:01.293 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-16T09:34:01.293 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-16T09:34:01.293 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-16T09:34:01.294 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-16T09:34:01.294 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-16T09:34:01.294 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-16T09:34:01.294 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-16T09:34:01.295 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-16T09:34:01.295 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-16T09:34:01.295 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-16T09:34:01.295 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-16T09:34:01.295 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/sit.ko 2023-10-16T09:34:01.296 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/tunnel6.ko 2023-10-16T09:34:01.296 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-16T09:34:01.296 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key 2023-10-16T09:34:01.296 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/af_key.ko 2023-10-16T09:34:01.297 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp 2023-10-16T09:34:01.297 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_core.ko 2023-10-16T09:34:01.297 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-16T09:34:01.297 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_eth.ko 2023-10-16T09:34:01.298 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip.ko 2023-10-16T09:34:01.298 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-16T09:34:01.298 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-16T09:34:01.298 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-16T09:34:01.298 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc 2023-10-16T09:34:01.299 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc.ko 2023-10-16T09:34:01.299 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154 2023-10-16T09:34:01.299 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/mac802154.ko 2023-10-16T09:34:01.299 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls 2023-10-16T09:34:01.300 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/mpls_gso.ko 2023-10-16T09:34:01.300 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter 2023-10-16T09:34:01.300 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset 2023-10-16T09:34:01.300 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-16T09:34:01.300 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-16T09:34:01.301 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-16T09:34:01.301 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-16T09:34:01.301 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-16T09:34:01.301 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-16T09:34:01.301 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-16T09:34:01.302 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-16T09:34:01.302 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-16T09:34:01.302 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-16T09:34:01.302 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-16T09:34:01.303 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-16T09:34:01.303 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-16T09:34:01.303 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-16T09:34:01.303 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-16T09:34:01.303 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-16T09:34:01.304 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs 2023-10-16T09:34:01.304 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-16T09:34:01.304 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-16T09:34:01.304 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-16T09:34:01.304 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-16T09:34:01.305 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-16T09:34:01.305 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-16T09:34:01.305 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-16T09:34:01.305 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-16T09:34:01.306 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-16T09:34:01.306 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-16T09:34:01.306 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-16T09:34:01.306 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-16T09:34:01.307 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-16T09:34:01.307 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-16T09:34:01.307 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conncount.ko 2023-10-16T09:34:01.307 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack.ko 2023-10-16T09:34:01.307 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-16T09:34:01.308 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-16T09:34:01.308 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-16T09:34:01.308 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-16T09:34:01.308 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-16T09:34:01.308 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-16T09:34:01.309 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-16T09:34:01.309 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-16T09:34:01.309 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-16T09:34:01.309 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-16T09:34:01.310 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-16T09:34:01.310 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-16T09:34:01.310 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-16T09:34:01.310 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table.ko 2023-10-16T09:34:01.311 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-16T09:34:01.311 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-16T09:34:01.311 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat.ko 2023-10-16T09:34:01.311 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-16T09:34:01.312 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-16T09:34:01.312 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-16T09:34:01.312 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-16T09:34:01.312 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-16T09:34:01.312 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-16T09:34:01.313 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_tables.ko 2023-10-16T09:34:01.313 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink.ko 2023-10-16T09:34:01.313 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-16T09:34:01.313 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-16T09:34:01.313 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-16T09:34:01.314 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-16T09:34:01.314 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-16T09:34:01.314 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_compat.ko 2023-10-16T09:34:01.315 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_ct.ko 2023-10-16T09:34:01.315 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-16T09:34:01.315 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fib.ko 2023-10-16T09:34:01.315 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-16T09:34:01.316 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_hash.ko 2023-10-16T09:34:01.316 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_limit.ko 2023-10-16T09:34:01.316 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_log.ko 2023-10-16T09:34:01.316 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_masq.ko 2023-10-16T09:34:01.317 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_nat.ko 2023-10-16T09:34:01.317 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_numgen.ko 2023-10-16T09:34:01.317 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_queue.ko 2023-10-16T09:34:01.317 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_quota.ko 2023-10-16T09:34:01.317 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_redir.ko 2023-10-16T09:34:01.318 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject.ko 2023-10-16T09:34:01.318 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-16T09:34:01.318 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_socket.ko 2023-10-16T09:34:01.318 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_tunnel.ko 2023-10-16T09:34:01.319 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-16T09:34:01.319 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-16T09:34:01.319 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-16T09:34:01.319 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-16T09:34:01.319 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CT.ko 2023-10-16T09:34:01.320 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_DSCP.ko 2023-10-16T09:34:01.320 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HL.ko 2023-10-16T09:34:01.320 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HMARK.ko 2023-10-16T09:34:01.320 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-16T09:34:01.320 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LED.ko 2023-10-16T09:34:01.321 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LOG.ko 2023-10-16T09:34:01.321 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-16T09:34:01.321 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-16T09:34:01.321 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-16T09:34:01.321 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-16T09:34:01.322 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-16T09:34:01.322 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-16T09:34:01.322 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-16T09:34:01.322 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-16T09:34:01.323 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-16T09:34:01.323 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TEE.ko 2023-10-16T09:34:01.323 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-16T09:34:01.323 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TRACE.ko 2023-10-16T09:34:01.323 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_addrtype.ko 2023-10-16T09:34:01.324 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_bpf.ko 2023-10-16T09:34:01.324 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cgroup.ko 2023-10-16T09:34:01.324 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cluster.ko 2023-10-16T09:34:01.324 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_comment.ko 2023-10-16T09:34:01.325 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connbytes.ko 2023-10-16T09:34:01.325 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlabel.ko 2023-10-16T09:34:01.325 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlimit.ko 2023-10-16T09:34:01.325 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connmark.ko 2023-10-16T09:34:01.325 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_conntrack.ko 2023-10-16T09:34:01.326 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cpu.ko 2023-10-16T09:34:01.326 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dccp.ko 2023-10-16T09:34:01.326 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_devgroup.ko 2023-10-16T09:34:01.326 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dscp.ko 2023-10-16T09:34:01.326 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ecn.ko 2023-10-16T09:34:01.327 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_esp.ko 2023-10-16T09:34:01.327 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-16T09:34:01.327 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_helper.ko 2023-10-16T09:34:01.327 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hl.ko 2023-10-16T09:34:01.327 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-16T09:34:01.328 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_iprange.ko 2023-10-16T09:34:01.328 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipvs.ko 2023-10-16T09:34:01.328 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_l2tp.ko 2023-10-16T09:34:01.328 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_length.ko 2023-10-16T09:34:01.329 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_limit.ko 2023-10-16T09:34:01.329 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mac.ko 2023-10-16T09:34:01.329 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mark.ko 2023-10-16T09:34:01.329 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_multiport.ko 2023-10-16T09:34:01.329 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nat.ko 2023-10-16T09:34:01.330 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nfacct.ko 2023-10-16T09:34:01.330 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_osf.ko 2023-10-16T09:34:01.330 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_owner.ko 2023-10-16T09:34:01.330 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_physdev.ko 2023-10-16T09:34:01.331 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_pkttype.ko 2023-10-16T09:34:01.331 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_policy.ko 2023-10-16T09:34:01.331 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_quota.ko 2023-10-16T09:34:01.331 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_rateest.ko 2023-10-16T09:34:01.331 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_realm.ko 2023-10-16T09:34:01.332 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_recent.ko 2023-10-16T09:34:01.332 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_sctp.ko 2023-10-16T09:34:01.332 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_set.ko 2023-10-16T09:34:01.332 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_socket.ko 2023-10-16T09:34:01.332 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_state.ko 2023-10-16T09:34:01.333 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_statistic.ko 2023-10-16T09:34:01.333 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_string.ko 2023-10-16T09:34:01.333 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-16T09:34:01.334 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_time.ko 2023-10-16T09:34:01.334 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_u32.ko 2023-10-16T09:34:01.334 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink 2023-10-16T09:34:01.334 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/netlink_diag.ko 2023-10-16T09:34:01.334 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh 2023-10-16T09:34:01.335 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/nsh.ko 2023-10-16T09:34:01.335 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch 2023-10-16T09:34:01.335 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/openvswitch.ko 2023-10-16T09:34:01.335 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-geneve.ko 2023-10-16T09:34:01.335 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-gre.ko 2023-10-16T09:34:01.336 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-16T09:34:01.336 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet 2023-10-16T09:34:01.336 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/af_packet_diag.ko 2023-10-16T09:34:01.336 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds 2023-10-16T09:34:01.337 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds.ko 2023-10-16T09:34:01.337 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_rdma.ko 2023-10-16T09:34:01.337 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_tcp.ko 2023-10-16T09:34:01.337 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched 2023-10-16T09:34:01.337 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_csum.ko 2023-10-16T09:34:01.338 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_gact.ko 2023-10-16T09:34:01.338 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_ipt.ko 2023-10-16T09:34:01.338 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_mirred.ko 2023-10-16T09:34:01.338 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_nat.ko 2023-10-16T09:34:01.339 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_pedit.ko 2023-10-16T09:34:01.339 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_police.ko 2023-10-16T09:34:01.339 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_simple.ko 2023-10-16T09:34:01.339 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_skbedit.ko 2023-10-16T09:34:01.340 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_vlan.ko 2023-10-16T09:34:01.340 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_basic.ko 2023-10-16T09:34:01.340 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_bpf.ko 2023-10-16T09:34:01.340 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_flow.ko 2023-10-16T09:34:01.340 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_fw.ko 2023-10-16T09:34:01.341 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_route.ko 2023-10-16T09:34:01.341 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_u32.ko 2023-10-16T09:34:01.341 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_cmp.ko 2023-10-16T09:34:01.341 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_ipset.ko 2023-10-16T09:34:01.341 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_meta.ko 2023-10-16T09:34:01.342 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_nbyte.ko 2023-10-16T09:34:01.342 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_text.ko 2023-10-16T09:34:01.342 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_u32.ko 2023-10-16T09:34:01.342 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_choke.ko 2023-10-16T09:34:01.342 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_codel.ko 2023-10-16T09:34:01.343 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_drr.ko 2023-10-16T09:34:01.343 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq.ko 2023-10-16T09:34:01.343 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_gred.ko 2023-10-16T09:34:01.343 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hfsc.ko 2023-10-16T09:34:01.345 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hhf.ko 2023-10-16T09:34:01.346 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_htb.ko 2023-10-16T09:34:01.346 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_ingress.ko 2023-10-16T09:34:01.346 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio.ko 2023-10-16T09:34:01.346 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-16T09:34:01.347 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_multiq.ko 2023-10-16T09:34:01.347 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_netem.ko 2023-10-16T09:34:01.347 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_pie.ko 2023-10-16T09:34:01.347 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_plug.ko 2023-10-16T09:34:01.348 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_prio.ko 2023-10-16T09:34:01.348 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_qfq.ko 2023-10-16T09:34:01.348 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_red.ko 2023-10-16T09:34:01.348 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfb.ko 2023-10-16T09:34:01.348 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfq.ko 2023-10-16T09:34:01.349 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_tbf.ko 2023-10-16T09:34:01.349 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_teql.ko 2023-10-16T09:34:01.349 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp 2023-10-16T09:34:01.349 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp.ko 2023-10-16T09:34:01.350 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp_diag.ko 2023-10-16T09:34:01.350 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc 2023-10-16T09:34:01.350 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss 2023-10-16T09:34:01.350 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-16T09:34:01.350 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-16T09:34:01.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/sunrpc.ko 2023-10-16T09:34:01.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma 2023-10-16T09:34:01.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-16T09:34:01.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc 2023-10-16T09:34:01.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/diag.ko 2023-10-16T09:34:01.352 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/tipc.ko 2023-10-16T09:34:01.352 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix 2023-10-16T09:34:01.352 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/unix_diag.ko 2023-10-16T09:34:01.352 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock 2023-10-16T09:34:01.352 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-16T09:34:01.353 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-16T09:34:01.353 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock.ko 2023-10-16T09:34:01.353 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-16T09:34:01.354 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-16T09:34:01.354 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm 2023-10-16T09:34:01.354 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-16T09:34:01.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security 2023-10-16T09:34:01.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys 2023-10-16T09:34:01.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys 2023-10-16T09:34:01.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-16T09:34:01.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys 2023-10-16T09:34:01.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-16T09:34:01.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt 2023-10-16T09:34:01.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib 2023-10-16T09:34:01.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/irqbypass.ko 2023-10-16T09:34:01.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias 2023-10-16T09:34:01.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias.bin 2023-10-16T09:34:01.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin 2023-10-16T09:34:01.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.alias.bin 2023-10-16T09:34:01.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.bin 2023-10-16T09:34:01.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.modinfo 2023-10-16T09:34:01.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep 2023-10-16T09:34:01.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep.bin 2023-10-16T09:34:01.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.devname 2023-10-16T09:34:01.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.order 2023-10-16T09:34:01.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.softdep 2023-10-16T09:34:01.359 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols 2023-10-16T09:34:01.359 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols.bin 2023-10-16T09:34:01.360 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g47672119149b 2023-10-16T09:34:01.360 DEBUG:teuthology.orchestra.run.smithi157:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-16T09:34:01.384 INFO:teuthology.orchestra.run.smithi157.stdout:postinstall scriptlet (using /bin/sh): 2023-10-16T09:34:01.385 INFO:teuthology.orchestra.run.smithi157.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g47672119149b -a -r /boot/System.map-6.6.0-rc5-g47672119149b ]; then 2023-10-16T09:34:01.385 INFO:teuthology.orchestra.run.smithi157.stdout:cp /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm 2023-10-16T09:34:01.385 INFO:teuthology.orchestra.run.smithi157.stdout:cp /boot/System.map-6.6.0-rc5-g47672119149b /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-16T09:34:01.385 INFO:teuthology.orchestra.run.smithi157.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/System.map-6.6.0-rc5-g47672119149b 2023-10-16T09:34:01.385 INFO:teuthology.orchestra.run.smithi157.stdout:/sbin/installkernel 6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-16T09:34:01.386 INFO:teuthology.orchestra.run.smithi157.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-16T09:34:01.386 INFO:teuthology.orchestra.run.smithi157.stdout:fi 2023-10-16T09:34:01.386 INFO:teuthology.orchestra.run.smithi157.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-16T09:34:01.386 INFO:teuthology.orchestra.run.smithi157.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-16T09:34:01.387 INFO:teuthology.orchestra.run.smithi157.stdout:new-kernel-pkg --remove 6.6.0-rc5-g47672119149b --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g47672119149b.img 2023-10-16T09:34:01.387 INFO:teuthology.orchestra.run.smithi157.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-16T09:34:01.387 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-install remove 6.6.0-rc5-g47672119149b 2023-10-16T09:34:01.387 INFO:teuthology.orchestra.run.smithi157.stdout:fi 2023-10-16T09:34:01.387 INFO:teuthology.orchestra.run.smithi157.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-16T09:34:01.388 INFO:teuthology.orchestra.run.smithi157.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-16T09:34:01.388 INFO:teuthology.orchestra.run.smithi157.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g47672119149b 2023-10-16T09:34:01.388 INFO:teuthology.orchestra.run.smithi157.stdout:fi 2023-10-16T09:34:01.390 DEBUG:teuthology.orchestra.run.smithi157:> sudo rpm -qi grub2-tools 2023-10-16T09:34:01.466 INFO:teuthology.orchestra.run.smithi157.stdout:Name : grub2-tools 2023-10-16T09:34:01.466 INFO:teuthology.orchestra.run.smithi157.stdout:Epoch : 1 2023-10-16T09:34:01.466 INFO:teuthology.orchestra.run.smithi157.stdout:Version : 2.06 2023-10-16T09:34:01.467 INFO:teuthology.orchestra.run.smithi157.stdout:Release : 68.el9 2023-10-16T09:34:01.467 INFO:teuthology.orchestra.run.smithi157.stdout:Architecture: x86_64 2023-10-16T09:34:01.467 INFO:teuthology.orchestra.run.smithi157.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-16T09:34:01.467 INFO:teuthology.orchestra.run.smithi157.stdout:Group : Unspecified 2023-10-16T09:34:01.468 INFO:teuthology.orchestra.run.smithi157.stdout:Size : 8264254 2023-10-16T09:34:01.468 INFO:teuthology.orchestra.run.smithi157.stdout:License : GPLv3+ 2023-10-16T09:34:01.468 INFO:teuthology.orchestra.run.smithi157.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-16T09:34:01.468 INFO:teuthology.orchestra.run.smithi157.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-16T09:34:01.468 INFO:teuthology.orchestra.run.smithi157.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-16T09:34:01.468 INFO:teuthology.orchestra.run.smithi157.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-16T09:34:01.469 INFO:teuthology.orchestra.run.smithi157.stdout:Packager : builder@centos.org 2023-10-16T09:34:01.469 INFO:teuthology.orchestra.run.smithi157.stdout:Vendor : CentOS 2023-10-16T09:34:01.469 INFO:teuthology.orchestra.run.smithi157.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-16T09:34:01.469 INFO:teuthology.orchestra.run.smithi157.stdout:Summary : Support tools for GRUB. 2023-10-16T09:34:01.469 INFO:teuthology.orchestra.run.smithi157.stdout:Description : 2023-10-16T09:34:01.469 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:34:01.470 INFO:teuthology.orchestra.run.smithi157.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-16T09:34:01.470 INFO:teuthology.orchestra.run.smithi157.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-16T09:34:01.470 INFO:teuthology.orchestra.run.smithi157.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-16T09:34:01.470 INFO:teuthology.orchestra.run.smithi157.stdout:hardware devices. 2023-10-16T09:34:01.470 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:34:01.471 INFO:teuthology.orchestra.run.smithi157.stdout:This subpackage provides tools for support of all platforms. 2023-10-16T09:34:01.472 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-16T09:34:01.472 INFO:teuthology.task.kernel:Updating grub on smithi157 to boot 6.6.0-rc5-g47672119149b 2023-10-16T09:34:01.472 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-16T09:34:02.096 INFO:teuthology.orchestra.run.smithi157.stderr:Generating grub configuration file ... 2023-10-16T09:34:03.477 INFO:teuthology.orchestra.run.smithi157.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-16T09:34:03.510 INFO:teuthology.orchestra.run.smithi157.stderr:done 2023-10-16T09:34:03.514 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2023-10-16T09:34:03.529 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.ep1DYxm5NN 2023-10-16T09:34:03.529 DEBUG:teuthology.orchestra.run.smithi157:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ep1DYxm5NN 2023-10-16T09:34:03.598 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.ep1DYxm5NN 2023-10-16T09:34:03.739 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.ep1DYxm5NN is 6KB 2023-10-16T09:34:03.788 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.ep1DYxm5NN 2023-10-16T09:34:03.802 DEBUG:teuthology.orchestra.run.smithi157:> sudo /bin/ls /boot/loader/entries || true 2023-10-16T09:34:03.876 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-16T09:34:03.876 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-16T09:34:03.876 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-16T09:34:03.877 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-6.6.0-rc5-g47672119149b.conf 2023-10-16T09:34:03.878 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-rc5-g47672119149b 2023-10-16T09:34:04.053 DEBUG:teuthology.orchestra.run.smithi157:> sudo shutdown -r now 2023-10-16T09:34:11.680 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi084.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-16T09:34:11.680 DEBUG:teuthology.orchestra.run.smithi084:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-16T09:34:11.754 INFO:teuthology.orchestra.run.smithi084.stdout:/boot/System.map-6.6.0-rc5-g47672119149b 2023-10-16T09:34:11.755 INFO:teuthology.orchestra.run.smithi084.stdout:/boot/config-6.6.0-rc5-g47672119149b 2023-10-16T09:34:11.755 INFO:teuthology.orchestra.run.smithi084.stdout:/boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-16T09:34:11.755 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b 2023-10-16T09:34:11.755 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel 2023-10-16T09:34:11.756 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch 2023-10-16T09:34:11.756 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86 2023-10-16T09:34:11.756 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto 2023-10-16T09:34:11.756 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-16T09:34:11.757 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-16T09:34:11.757 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-16T09:34:11.757 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-16T09:34:11.757 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-16T09:34:11.757 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-16T09:34:11.758 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-16T09:34:11.758 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-16T09:34:11.758 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-16T09:34:11.758 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-16T09:34:11.759 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-16T09:34:11.759 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-16T09:34:11.759 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-16T09:34:11.759 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-16T09:34:11.759 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-16T09:34:11.760 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-16T09:34:11.760 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-16T09:34:11.760 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-16T09:34:11.760 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-16T09:34:11.760 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-16T09:34:11.761 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel 2023-10-16T09:34:11.761 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu 2023-10-16T09:34:11.761 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce 2023-10-16T09:34:11.761 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-16T09:34:11.761 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm 2023-10-16T09:34:11.761 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-16T09:34:11.762 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-16T09:34:11.762 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm.ko 2023-10-16T09:34:11.762 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto 2023-10-16T09:34:11.762 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ansi_cprng.ko 2023-10-16T09:34:11.762 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/anubis.ko 2023-10-16T09:34:11.763 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/arc4.ko 2023-10-16T09:34:11.763 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx 2023-10-16T09:34:11.763 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-16T09:34:11.763 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_pq.ko 2023-10-16T09:34:11.763 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-16T09:34:11.764 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_tx.ko 2023-10-16T09:34:11.764 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_xor.ko 2023-10-16T09:34:11.764 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/raid6test.ko 2023-10-16T09:34:11.764 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authenc.ko 2023-10-16T09:34:11.764 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authencesn.ko 2023-10-16T09:34:11.765 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blake2b_generic.ko 2023-10-16T09:34:11.765 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_common.ko 2023-10-16T09:34:11.765 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_generic.ko 2023-10-16T09:34:11.765 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/camellia_generic.ko 2023-10-16T09:34:11.765 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast5_generic.ko 2023-10-16T09:34:11.766 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast6_generic.ko 2023-10-16T09:34:11.766 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast_common.ko 2023-10-16T09:34:11.766 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ccm.ko 2023-10-16T09:34:11.766 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cmac.ko 2023-10-16T09:34:11.766 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crc32_generic.ko 2023-10-16T09:34:11.766 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_engine.ko 2023-10-16T09:34:11.767 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_user.ko 2023-10-16T09:34:11.767 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cts.ko 2023-10-16T09:34:11.767 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/des_generic.ko 2023-10-16T09:34:11.767 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/dh_generic.ko 2023-10-16T09:34:11.767 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/echainiv.ko 2023-10-16T09:34:11.768 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/essiv.ko 2023-10-16T09:34:11.768 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/fcrypt.ko 2023-10-16T09:34:11.768 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/khazad.ko 2023-10-16T09:34:11.768 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4.ko 2023-10-16T09:34:11.768 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4hc.ko 2023-10-16T09:34:11.769 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/md4.ko 2023-10-16T09:34:11.769 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/michael_mic.ko 2023-10-16T09:34:11.769 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcbc.ko 2023-10-16T09:34:11.769 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcrypt.ko 2023-10-16T09:34:11.769 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/rmd160.ko 2023-10-16T09:34:11.770 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/seed.ko 2023-10-16T09:34:11.770 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/serpent_generic.ko 2023-10-16T09:34:11.770 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tcrypt.ko 2023-10-16T09:34:11.770 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tea.ko 2023-10-16T09:34:11.770 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_common.ko 2023-10-16T09:34:11.771 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_generic.ko 2023-10-16T09:34:11.771 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/vmac.ko 2023-10-16T09:34:11.771 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/wp512.ko 2023-10-16T09:34:11.771 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xcbc.ko 2023-10-16T09:34:11.771 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xor.ko 2023-10-16T09:34:11.771 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xxhash_generic.ko 2023-10-16T09:34:11.772 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers 2023-10-16T09:34:11.772 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi 2023-10-16T09:34:11.772 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-16T09:34:11.772 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_pad.ko 2023-10-16T09:34:11.772 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/custom_method.ko 2023-10-16T09:34:11.773 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ec_sys.ko 2023-10-16T09:34:11.773 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/platform_profile.ko 2023-10-16T09:34:11.773 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbs.ko 2023-10-16T09:34:11.773 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbshc.ko 2023-10-16T09:34:11.773 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/video.ko 2023-10-16T09:34:11.774 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata 2023-10-16T09:34:11.774 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/acard-ahci.ko 2023-10-16T09:34:11.774 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci_platform.ko 2023-10-16T09:34:11.774 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ata_generic.ko 2023-10-16T09:34:11.774 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci_platform.ko 2023-10-16T09:34:11.775 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_acpi.ko 2023-10-16T09:34:11.775 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ali.ko 2023-10-16T09:34:11.775 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_amd.ko 2023-10-16T09:34:11.775 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_artop.ko 2023-10-16T09:34:11.775 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atiixp.ko 2023-10-16T09:34:11.776 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atp867x.ko 2023-10-16T09:34:11.776 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd640.ko 2023-10-16T09:34:11.776 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-16T09:34:11.776 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cypress.ko 2023-10-16T09:34:11.776 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_efar.ko 2023-10-16T09:34:11.777 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt366.ko 2023-10-16T09:34:11.777 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-16T09:34:11.777 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-16T09:34:11.777 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-16T09:34:11.778 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it8213.ko 2023-10-16T09:34:11.778 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it821x.ko 2023-10-16T09:34:11.778 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_jmicron.ko 2023-10-16T09:34:11.778 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_marvell.ko 2023-10-16T09:34:11.778 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_mpiix.ko 2023-10-16T09:34:11.779 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_netcell.ko 2023-10-16T09:34:11.779 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ninja32.ko 2023-10-16T09:34:11.779 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87410.ko 2023-10-16T09:34:11.779 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87415.ko 2023-10-16T09:34:11.779 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-16T09:34:11.780 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_opti.ko 2023-10-16T09:34:11.780 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_optidma.ko 2023-10-16T09:34:11.780 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-16T09:34:11.780 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-16T09:34:11.780 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-16T09:34:11.781 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_piccolo.ko 2023-10-16T09:34:11.781 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rdc.ko 2023-10-16T09:34:11.781 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sch.ko 2023-10-16T09:34:11.781 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_serverworks.ko 2023-10-16T09:34:11.781 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sil680.ko 2023-10-16T09:34:11.782 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sis.ko 2023-10-16T09:34:11.782 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-16T09:34:11.782 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_triflex.ko 2023-10-16T09:34:11.782 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_via.ko 2023-10-16T09:34:11.783 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pdc_adma.ko 2023-10-16T09:34:11.783 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_inic162x.ko 2023-10-16T09:34:11.783 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_mv.ko 2023-10-16T09:34:11.783 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_nv.ko 2023-10-16T09:34:11.783 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_promise.ko 2023-10-16T09:34:11.784 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_qstor.ko 2023-10-16T09:34:11.784 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil.ko 2023-10-16T09:34:11.784 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil24.ko 2023-10-16T09:34:11.784 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sis.ko 2023-10-16T09:34:11.784 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_svw.ko 2023-10-16T09:34:11.785 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sx4.ko 2023-10-16T09:34:11.785 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_uli.ko 2023-10-16T09:34:11.785 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_via.ko 2023-10-16T09:34:11.785 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_vsc.ko 2023-10-16T09:34:11.786 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay 2023-10-16T09:34:11.786 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-16T09:34:11.786 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base 2023-10-16T09:34:11.786 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap 2023-10-16T09:34:11.787 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-16T09:34:11.787 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-16T09:34:11.787 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma 2023-10-16T09:34:11.787 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/bcma.ko 2023-10-16T09:34:11.787 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block 2023-10-16T09:34:11.788 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe 2023-10-16T09:34:11.788 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/aoe.ko 2023-10-16T09:34:11.788 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/brd.ko 2023-10-16T09:34:11.788 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd 2023-10-16T09:34:11.788 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/drbd.ko 2023-10-16T09:34:11.789 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/floppy.ko 2023-10-16T09:34:11.789 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/loop.ko 2023-10-16T09:34:11.789 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx 2023-10-16T09:34:11.789 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-16T09:34:11.789 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/nbd.ko 2023-10-16T09:34:11.789 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk 2023-10-16T09:34:11.790 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-16T09:34:11.790 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/pktcdvd.ko 2023-10-16T09:34:11.790 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/rbd.ko 2023-10-16T09:34:11.790 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/virtio_blk.ko 2023-10-16T09:34:11.790 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback 2023-10-16T09:34:11.791 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-16T09:34:11.791 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkfront.ko 2023-10-16T09:34:11.791 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram 2023-10-16T09:34:11.791 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/zram.ko 2023-10-16T09:34:11.791 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char 2023-10-16T09:34:11.792 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hangcheck-timer.ko 2023-10-16T09:34:11.792 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random 2023-10-16T09:34:11.792 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-16T09:34:11.792 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-16T09:34:11.792 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-16T09:34:11.793 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-16T09:34:11.793 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-16T09:34:11.793 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi 2023-10-16T09:34:11.793 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-16T09:34:11.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-16T09:34:11.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-16T09:34:11.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-16T09:34:11.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-16T09:34:11.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-16T09:34:11.795 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/lp.ko 2023-10-16T09:34:11.795 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave 2023-10-16T09:34:11.795 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/mwave.ko 2023-10-16T09:34:11.795 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ppdev.ko 2023-10-16T09:34:11.795 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tlclk.ko 2023-10-16T09:34:11.796 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm 2023-10-16T09:34:11.796 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm.ko 2023-10-16T09:34:11.796 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-16T09:34:11.796 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-16T09:34:11.796 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-16T09:34:11.797 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-16T09:34:11.797 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-16T09:34:11.797 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/uv_mmtimer.ko 2023-10-16T09:34:11.797 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/virtio_console.ko 2023-10-16T09:34:11.797 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq 2023-10-16T09:34:11.798 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-16T09:34:11.798 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-16T09:34:11.798 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-16T09:34:11.798 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-16T09:34:11.798 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-16T09:34:11.799 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-16T09:34:11.799 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto 2023-10-16T09:34:11.799 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp 2023-10-16T09:34:11.799 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-16T09:34:11.799 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-16T09:34:11.800 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel 2023-10-16T09:34:11.800 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat 2023-10-16T09:34:11.800 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-16T09:34:11.800 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-16T09:34:11.800 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-16T09:34:11.801 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-16T09:34:11.801 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-aes.ko 2023-10-16T09:34:11.801 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-sha.ko 2023-10-16T09:34:11.801 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio 2023-10-16T09:34:11.801 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-16T09:34:11.802 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca 2023-10-16T09:34:11.802 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/dca.ko 2023-10-16T09:34:11.802 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq 2023-10-16T09:34:11.802 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-16T09:34:11.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma 2023-10-16T09:34:11.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw 2023-10-16T09:34:11.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-16T09:34:11.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat 2023-10-16T09:34:11.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-16T09:34:11.804 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac 2023-10-16T09:34:11.804 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/amd64_edac.ko 2023-10-16T09:34:11.804 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/e752x_edac.ko 2023-10-16T09:34:11.804 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-16T09:34:11.804 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3000_edac.ko 2023-10-16T09:34:11.805 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3200_edac.ko 2023-10-16T09:34:11.805 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5100_edac.ko 2023-10-16T09:34:11.805 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5400_edac.ko 2023-10-16T09:34:11.805 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7300_edac.ko 2023-10-16T09:34:11.805 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7core_edac.ko 2023-10-16T09:34:11.806 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i82975x_edac.ko 2023-10-16T09:34:11.806 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ie31200_edac.ko 2023-10-16T09:34:11.806 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/sb_edac.ko 2023-10-16T09:34:11.806 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/x38_edac.ko 2023-10-16T09:34:11.806 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware 2023-10-16T09:34:11.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/edd.ko 2023-10-16T09:34:11.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-16T09:34:11.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio 2023-10-16T09:34:11.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ich.ko 2023-10-16T09:34:11.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-16T09:34:11.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid 2023-10-16T09:34:11.808 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-appleir.ko 2023-10-16T09:34:11.808 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-aureal.ko 2023-10-16T09:34:11.808 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-axff.ko 2023-10-16T09:34:11.808 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-dr.ko 2023-10-16T09:34:11.808 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elecom.ko 2023-10-16T09:34:11.809 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elo.ko 2023-10-16T09:34:11.809 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-emsff.ko 2023-10-16T09:34:11.809 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gaff.ko 2023-10-16T09:34:11.809 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gt683r.ko 2023-10-16T09:34:11.809 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gyration.ko 2023-10-16T09:34:11.810 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-16T09:34:11.810 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-16T09:34:11.810 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtekff.ko 2023-10-16T09:34:11.810 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-hyperv.ko 2023-10-16T09:34:11.810 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-icade.ko 2023-10-16T09:34:11.811 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-keytouch.ko 2023-10-16T09:34:11.811 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kye.ko 2023-10-16T09:34:11.811 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lcpower.ko 2023-10-16T09:34:11.811 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-led.ko 2023-10-16T09:34:11.811 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lenovo.ko 2023-10-16T09:34:11.811 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-16T09:34:11.812 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-16T09:34:11.812 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-multitouch.ko 2023-10-16T09:34:11.812 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ortek.ko 2023-10-16T09:34:11.812 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-penmount.ko 2023-10-16T09:34:11.812 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-petalynx.ko 2023-10-16T09:34:11.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-picolcd.ko 2023-10-16T09:34:11.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-pl.ko 2023-10-16T09:34:11.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-plantronics.ko 2023-10-16T09:34:11.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-primax.ko 2023-10-16T09:34:11.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-rmi.ko 2023-10-16T09:34:11.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-16T09:34:11.814 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-16T09:34:11.814 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-16T09:34:11.814 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-16T09:34:11.814 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-16T09:34:11.814 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-16T09:34:11.815 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-16T09:34:11.815 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-16T09:34:11.815 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-16T09:34:11.815 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-16T09:34:11.815 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-16T09:34:11.816 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat.ko 2023-10-16T09:34:11.816 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-saitek.ko 2023-10-16T09:34:11.816 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-samsung.ko 2023-10-16T09:34:11.816 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-16T09:34:11.816 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sjoy.ko 2023-10-16T09:34:11.817 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sony.ko 2023-10-16T09:34:11.817 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-speedlink.ko 2023-10-16T09:34:11.817 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-steelseries.ko 2023-10-16T09:34:11.817 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sunplus.ko 2023-10-16T09:34:11.817 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-16T09:34:11.818 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tivo.ko 2023-10-16T09:34:11.818 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tmff.ko 2023-10-16T09:34:11.818 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-topseed.ko 2023-10-16T09:34:11.818 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-twinhan.ko 2023-10-16T09:34:11.818 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-uclogic.ko 2023-10-16T09:34:11.818 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-waltop.ko 2023-10-16T09:34:11.819 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-wiimote.ko 2023-10-16T09:34:11.819 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-xinmo.ko 2023-10-16T09:34:11.819 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zpff.ko 2023-10-16T09:34:11.819 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zydacron.ko 2023-10-16T09:34:11.819 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/uhid.ko 2023-10-16T09:34:11.820 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/wacom.ko 2023-10-16T09:34:11.820 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv 2023-10-16T09:34:11.820 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_balloon.ko 2023-10-16T09:34:11.820 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_utils.ko 2023-10-16T09:34:11.820 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_vmbus.ko 2023-10-16T09:34:11.821 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon 2023-10-16T09:34:11.821 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru.ko 2023-10-16T09:34:11.821 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru3.ko 2023-10-16T09:34:11.821 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-16T09:34:11.821 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7414.ko 2023-10-16T09:34:11.822 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7418.ko 2023-10-16T09:34:11.822 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adc128d818.ko 2023-10-16T09:34:11.822 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1025.ko 2023-10-16T09:34:11.822 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1026.ko 2023-10-16T09:34:11.822 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1029.ko 2023-10-16T09:34:11.823 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1031.ko 2023-10-16T09:34:11.823 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm9240.ko 2023-10-16T09:34:11.823 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ads7828.ko 2023-10-16T09:34:11.823 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7410.ko 2023-10-16T09:34:11.823 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7411.ko 2023-10-16T09:34:11.824 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7462.ko 2023-10-16T09:34:11.824 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7470.ko 2023-10-16T09:34:11.824 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7475.ko 2023-10-16T09:34:11.824 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7x10.ko 2023-10-16T09:34:11.824 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/amc6821.ko 2023-10-16T09:34:11.824 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/applesmc.ko 2023-10-16T09:34:11.825 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asb100.ko 2023-10-16T09:34:11.825 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asc7621.ko 2023-10-16T09:34:11.825 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-16T09:34:11.825 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/atxp1.ko 2023-10-16T09:34:11.825 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/coretemp.ko 2023-10-16T09:34:11.826 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-16T09:34:11.826 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dme1737.ko 2023-10-16T09:34:11.826 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds1621.ko 2023-10-16T09:34:11.826 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds620.ko 2023-10-16T09:34:11.826 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc1403.ko 2023-10-16T09:34:11.827 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc6w201.ko 2023-10-16T09:34:11.827 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71805f.ko 2023-10-16T09:34:11.827 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71882fg.ko 2023-10-16T09:34:11.827 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f75375s.ko 2023-10-16T09:34:11.828 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-16T09:34:11.828 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fschmd.ko 2023-10-16T09:34:11.828 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g760a.ko 2023-10-16T09:34:11.828 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g762.ko 2023-10-16T09:34:11.828 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl518sm.ko 2023-10-16T09:34:11.829 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl520sm.ko 2023-10-16T09:34:11.829 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-16T09:34:11.829 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-16T09:34:11.829 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-16T09:34:11.829 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmaem.ko 2023-10-16T09:34:11.830 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmpex.ko 2023-10-16T09:34:11.830 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina209.ko 2023-10-16T09:34:11.830 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina2xx.ko 2023-10-16T09:34:11.830 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/it87.ko 2023-10-16T09:34:11.830 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k10temp.ko 2023-10-16T09:34:11.831 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k8temp.ko 2023-10-16T09:34:11.831 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-16T09:34:11.831 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm63.ko 2023-10-16T09:34:11.831 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm73.ko 2023-10-16T09:34:11.831 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm75.ko 2023-10-16T09:34:11.832 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm77.ko 2023-10-16T09:34:11.832 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm78.ko 2023-10-16T09:34:11.832 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm80.ko 2023-10-16T09:34:11.832 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm83.ko 2023-10-16T09:34:11.832 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm85.ko 2023-10-16T09:34:11.833 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm87.ko 2023-10-16T09:34:11.833 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm90.ko 2023-10-16T09:34:11.833 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm92.ko 2023-10-16T09:34:11.834 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm93.ko 2023-10-16T09:34:11.834 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95234.ko 2023-10-16T09:34:11.834 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95241.ko 2023-10-16T09:34:11.834 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95245.ko 2023-10-16T09:34:11.834 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc2945.ko 2023-10-16T09:34:11.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4151.ko 2023-10-16T09:34:11.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4215.ko 2023-10-16T09:34:11.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4222.ko 2023-10-16T09:34:11.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4245.ko 2023-10-16T09:34:11.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4260.ko 2023-10-16T09:34:11.836 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4261.ko 2023-10-16T09:34:11.836 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max16065.ko 2023-10-16T09:34:11.836 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1619.ko 2023-10-16T09:34:11.836 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1668.ko 2023-10-16T09:34:11.836 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max197.ko 2023-10-16T09:34:11.837 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6639.ko 2023-10-16T09:34:11.837 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6650.ko 2023-10-16T09:34:11.837 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6697.ko 2023-10-16T09:34:11.837 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mcp3021.ko 2023-10-16T09:34:11.837 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6683.ko 2023-10-16T09:34:11.837 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-16T09:34:11.838 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775.ko 2023-10-16T09:34:11.838 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct7802.ko 2023-10-16T09:34:11.838 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87360.ko 2023-10-16T09:34:11.838 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87427.ko 2023-10-16T09:34:11.838 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pcf8591.ko 2023-10-16T09:34:11.839 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus 2023-10-16T09:34:11.839 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-16T09:34:11.839 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-16T09:34:11.839 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-16T09:34:11.839 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-16T09:34:11.840 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-16T09:34:11.840 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-16T09:34:11.840 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-16T09:34:11.840 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-16T09:34:11.840 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-16T09:34:11.841 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-16T09:34:11.841 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-16T09:34:11.841 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-16T09:34:11.841 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/powr1220.ko 2023-10-16T09:34:11.841 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5627.ko 2023-10-16T09:34:11.842 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5636.ko 2023-10-16T09:34:11.842 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-16T09:34:11.842 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht15.ko 2023-10-16T09:34:11.842 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht21.ko 2023-10-16T09:34:11.843 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/shtc1.ko 2023-10-16T09:34:11.843 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sis5595.ko 2023-10-16T09:34:11.843 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-16T09:34:11.843 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-16T09:34:11.843 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-16T09:34:11.844 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/thmc50.ko 2023-10-16T09:34:11.844 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp102.ko 2023-10-16T09:34:11.844 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp103.ko 2023-10-16T09:34:11.844 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp401.ko 2023-10-16T09:34:11.845 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp421.ko 2023-10-16T09:34:11.845 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-16T09:34:11.845 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via686a.ko 2023-10-16T09:34:11.845 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt1211.ko 2023-10-16T09:34:11.846 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt8231.ko 2023-10-16T09:34:11.846 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-16T09:34:11.846 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627hf.ko 2023-10-16T09:34:11.846 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83781d.ko 2023-10-16T09:34:11.846 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83791d.ko 2023-10-16T09:34:11.847 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83792d.ko 2023-10-16T09:34:11.847 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83793.ko 2023-10-16T09:34:11.847 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83795.ko 2023-10-16T09:34:11.847 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-16T09:34:11.847 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-16T09:34:11.848 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c 2023-10-16T09:34:11.848 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos 2023-10-16T09:34:11.848 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-16T09:34:11.848 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-16T09:34:11.848 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses 2023-10-16T09:34:11.849 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-16T09:34:11.849 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-16T09:34:11.849 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-16T09:34:11.849 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-16T09:34:11.849 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-16T09:34:11.850 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-16T09:34:11.850 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-16T09:34:11.850 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-16T09:34:11.850 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-16T09:34:11.851 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-16T09:34:11.851 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-16T09:34:11.851 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-16T09:34:11.851 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-16T09:34:11.851 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-16T09:34:11.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-16T09:34:11.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-16T09:34:11.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-16T09:34:11.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-16T09:34:11.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-16T09:34:11.853 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-16T09:34:11.853 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-16T09:34:11.853 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-16T09:34:11.853 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-16T09:34:11.853 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-dev.ko 2023-10-16T09:34:11.854 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-mux.ko 2023-10-16T09:34:11.854 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-16T09:34:11.854 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-16T09:34:11.854 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-stub.ko 2023-10-16T09:34:11.854 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband 2023-10-16T09:34:11.855 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core 2023-10-16T09:34:11.855 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-16T09:34:11.855 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-16T09:34:11.855 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-16T09:34:11.855 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-16T09:34:11.856 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-16T09:34:11.856 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-16T09:34:11.856 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-16T09:34:11.856 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw 2023-10-16T09:34:11.856 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4 2023-10-16T09:34:11.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-16T09:34:11.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4 2023-10-16T09:34:11.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-16T09:34:11.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5 2023-10-16T09:34:11.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-16T09:34:11.858 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca 2023-10-16T09:34:11.858 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-16T09:34:11.858 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp 2023-10-16T09:34:11.858 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib 2023-10-16T09:34:11.859 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-16T09:34:11.859 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser 2023-10-16T09:34:11.859 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-16T09:34:11.859 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert 2023-10-16T09:34:11.859 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-16T09:34:11.860 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp 2023-10-16T09:34:11.860 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-16T09:34:11.860 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt 2023-10-16T09:34:11.860 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-16T09:34:11.860 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input 2023-10-16T09:34:11.861 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport 2023-10-16T09:34:11.861 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-16T09:34:11.861 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-16T09:34:11.861 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/gameport.ko 2023-10-16T09:34:11.861 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/joydev.ko 2023-10-16T09:34:11.862 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard 2023-10-16T09:34:11.862 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-16T09:34:11.862 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse 2023-10-16T09:34:11.862 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/appletouch.ko 2023-10-16T09:34:11.862 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-16T09:34:11.863 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-16T09:34:11.863 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-16T09:34:11.863 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/sermouse.ko 2023-10-16T09:34:11.863 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-16T09:34:11.863 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-16T09:34:11.864 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-16T09:34:11.864 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4 2023-10-16T09:34:11.864 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-16T09:34:11.864 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio 2023-10-16T09:34:11.864 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-16T09:34:11.865 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-16T09:34:11.865 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-16T09:34:11.865 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serio_raw.ko 2023-10-16T09:34:11.865 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/sparse-keymap.ko 2023-10-16T09:34:11.865 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu 2023-10-16T09:34:11.866 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd 2023-10-16T09:34:11.866 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-16T09:34:11.866 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds 2023-10-16T09:34:11.866 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-blinkm.ko 2023-10-16T09:34:11.867 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3530.ko 2023-10-16T09:34:11.867 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp3944.ko 2023-10-16T09:34:11.867 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-ss4200.ko 2023-10-16T09:34:11.867 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger 2023-10-16T09:34:11.867 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-16T09:34:11.868 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-16T09:34:11.868 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-16T09:34:11.868 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-16T09:34:11.868 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-16T09:34:11.868 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-16T09:34:11.869 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-16T09:34:11.869 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-16T09:34:11.869 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md 2023-10-16T09:34:11.869 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache 2023-10-16T09:34:11.870 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/bcache.ko 2023-10-16T09:34:11.870 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bio-prison.ko 2023-10-16T09:34:11.870 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache-smq.ko 2023-10-16T09:34:11.870 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache.ko 2023-10-16T09:34:11.870 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-crypt.ko 2023-10-16T09:34:11.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-delay.ko 2023-10-16T09:34:11.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-flakey.ko 2023-10-16T09:34:11.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log-userspace.ko 2023-10-16T09:34:11.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-multipath.ko 2023-10-16T09:34:11.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-queue-length.ko 2023-10-16T09:34:11.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-raid.ko 2023-10-16T09:34:11.872 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-round-robin.ko 2023-10-16T09:34:11.872 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-service-time.ko 2023-10-16T09:34:11.872 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-switch.ko 2023-10-16T09:34:11.872 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-thin-pool.ko 2023-10-16T09:34:11.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-verity.ko 2023-10-16T09:34:11.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/faulty.ko 2023-10-16T09:34:11.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/linear.ko 2023-10-16T09:34:11.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/multipath.ko 2023-10-16T09:34:11.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data 2023-10-16T09:34:11.874 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-16T09:34:11.874 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid0.ko 2023-10-16T09:34:11.874 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid1.ko 2023-10-16T09:34:11.874 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid10.ko 2023-10-16T09:34:11.874 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid456.ko 2023-10-16T09:34:11.875 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message 2023-10-16T09:34:11.875 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion 2023-10-16T09:34:11.875 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptbase.ko 2023-10-16T09:34:11.875 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptctl.ko 2023-10-16T09:34:11.875 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptfc.ko 2023-10-16T09:34:11.876 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptlan.ko 2023-10-16T09:34:11.876 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptsas.ko 2023-10-16T09:34:11.876 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-16T09:34:11.876 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptspi.ko 2023-10-16T09:34:11.876 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd 2023-10-16T09:34:11.877 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_ich.ko 2023-10-16T09:34:11.877 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_sch.ko 2023-10-16T09:34:11.877 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mfd-core.ko 2023-10-16T09:34:11.877 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/sm501.ko 2023-10-16T09:34:11.877 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/viperboard.ko 2023-10-16T09:34:11.878 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/vx855.ko 2023-10-16T09:34:11.878 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wl1273-core.ko 2023-10-16T09:34:11.878 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc 2023-10-16T09:34:11.879 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl 2023-10-16T09:34:11.879 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-16T09:34:11.879 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds9802als.ko 2023-10-16T09:34:11.879 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds990x.ko 2023-10-16T09:34:11.879 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/bh1770glc.ko 2023-10-16T09:34:11.880 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710 2023-10-16T09:34:11.880 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/cb710.ko 2023-10-16T09:34:11.880 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom 2023-10-16T09:34:11.880 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/at24.ko 2023-10-16T09:34:11.880 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-16T09:34:11.881 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-16T09:34:11.881 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-16T09:34:11.881 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/enclosure.ko 2023-10-16T09:34:11.881 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hpilo.ko 2023-10-16T09:34:11.882 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29003.ko 2023-10-16T09:34:11.882 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29020.ko 2023-10-16T09:34:11.882 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d 2023-10-16T09:34:11.882 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-16T09:34:11.883 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-16T09:34:11.883 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei 2023-10-16T09:34:11.883 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-me.ko 2023-10-16T09:34:11.883 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-16T09:34:11.883 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei.ko 2023-10-16T09:34:11.884 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru 2023-10-16T09:34:11.884 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-16T09:34:11.884 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp 2023-10-16T09:34:11.884 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-16T09:34:11.884 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-16T09:34:11.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-16T09:34:11.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-16T09:34:11.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_core.ko 2023-10-16T09:34:11.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tsl2550.ko 2023-10-16T09:34:11.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_balloon.ko 2023-10-16T09:34:11.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci 2023-10-16T09:34:11.886 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-16T09:34:11.886 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net 2023-10-16T09:34:11.886 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding 2023-10-16T09:34:11.886 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/bonding.ko 2023-10-16T09:34:11.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa 2023-10-16T09:34:11.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53 2023-10-16T09:34:11.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-16T09:34:11.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-16T09:34:11.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-16T09:34:11.888 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-16T09:34:11.888 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-16T09:34:11.888 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dummy.ko 2023-10-16T09:34:11.888 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/eql.ko 2023-10-16T09:34:11.888 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet 2023-10-16T09:34:11.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com 2023-10-16T09:34:11.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-16T09:34:11.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-16T09:34:11.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-16T09:34:11.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-16T09:34:11.890 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390 2023-10-16T09:34:11.890 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-16T09:34:11.890 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-16T09:34:11.890 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-16T09:34:11.890 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-16T09:34:11.891 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec 2023-10-16T09:34:11.891 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-16T09:34:11.891 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere 2023-10-16T09:34:11.891 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-16T09:34:11.891 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon 2023-10-16T09:34:11.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-16T09:34:11.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera 2023-10-16T09:34:11.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-16T09:34:11.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd 2023-10-16T09:34:11.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-16T09:34:11.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-16T09:34:11.893 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-16T09:34:11.893 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros 2023-10-16T09:34:11.893 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx 2023-10-16T09:34:11.893 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-16T09:34:11.893 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-16T09:34:11.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-16T09:34:11.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-16T09:34:11.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-16T09:34:11.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-16T09:34:11.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-16T09:34:11.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-16T09:34:11.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom 2023-10-16T09:34:11.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-16T09:34:11.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-16T09:34:11.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-16T09:34:11.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-16T09:34:11.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-16T09:34:11.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-16T09:34:11.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-16T09:34:11.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-16T09:34:11.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade 2023-10-16T09:34:11.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna 2023-10-16T09:34:11.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-16T09:34:11.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio 2023-10-16T09:34:11.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-16T09:34:11.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-16T09:34:11.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-16T09:34:11.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-16T09:34:11.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-16T09:34:11.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-16T09:34:11.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-16T09:34:11.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-16T09:34:11.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-16T09:34:11.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-16T09:34:11.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco 2023-10-16T09:34:11.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic 2023-10-16T09:34:11.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-16T09:34:11.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec 2023-10-16T09:34:11.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip 2023-10-16T09:34:11.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-16T09:34:11.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-16T09:34:11.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-16T09:34:11.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-16T09:34:11.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-16T09:34:11.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-16T09:34:11.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink 2023-10-16T09:34:11.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-16T09:34:11.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-16T09:34:11.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dnet.ko 2023-10-16T09:34:11.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex 2023-10-16T09:34:11.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet 2023-10-16T09:34:11.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-16T09:34:11.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-16T09:34:11.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-16T09:34:11.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel 2023-10-16T09:34:11.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-16T09:34:11.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000 2023-10-16T09:34:11.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-16T09:34:11.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-16T09:34:11.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-16T09:34:11.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-16T09:34:11.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-16T09:34:11.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e 2023-10-16T09:34:11.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-16T09:34:11.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf 2023-10-16T09:34:11.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-16T09:34:11.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb 2023-10-16T09:34:11.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-16T09:34:11.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-16T09:34:11.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-16T09:34:11.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-16T09:34:11.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-16T09:34:11.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-16T09:34:11.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-16T09:34:11.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/jme.ko 2023-10-16T09:34:11.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell 2023-10-16T09:34:11.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-16T09:34:11.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-16T09:34:11.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-16T09:34:11.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox 2023-10-16T09:34:11.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-16T09:34:11.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-16T09:34:11.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-16T09:34:11.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-16T09:34:11.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-16T09:34:11.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-16T09:34:11.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel 2023-10-16T09:34:11.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-16T09:34:11.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom 2023-10-16T09:34:11.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-16T09:34:11.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-16T09:34:11.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi 2023-10-16T09:34:11.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-16T09:34:11.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-16T09:34:11.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion 2023-10-16T09:34:11.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-16T09:34:11.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia 2023-10-16T09:34:11.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-16T09:34:11.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines 2023-10-16T09:34:11.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-16T09:34:11.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-16T09:34:11.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic 2023-10-16T09:34:11.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-16T09:34:11.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-16T09:34:11.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-16T09:34:11.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-16T09:34:11.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-16T09:34:11.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc 2023-10-16T09:34:11.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-16T09:34:11.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek 2023-10-16T09:34:11.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-16T09:34:11.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-16T09:34:11.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-16T09:34:11.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-16T09:34:11.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker 2023-10-16T09:34:11.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-16T09:34:11.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc 2023-10-16T09:34:11.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-16T09:34:11.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan 2023-10-16T09:34:11.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-16T09:34:11.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis 2023-10-16T09:34:11.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-16T09:34:11.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-16T09:34:11.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc 2023-10-16T09:34:11.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-16T09:34:11.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-16T09:34:11.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-16T09:34:11.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-16T09:34:11.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro 2023-10-16T09:34:11.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-16T09:34:11.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-16T09:34:11.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-16T09:34:11.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun 2023-10-16T09:34:11.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-16T09:34:11.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-16T09:34:11.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-16T09:34:11.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-16T09:34:11.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti 2023-10-16T09:34:11.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-16T09:34:11.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti 2023-10-16T09:34:11.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-16T09:34:11.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via 2023-10-16T09:34:11.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-16T09:34:11.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-16T09:34:11.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet 2023-10-16T09:34:11.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-16T09:34:11.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-16T09:34:11.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom 2023-10-16T09:34:11.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-16T09:34:11.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/geneve.ko 2023-10-16T09:34:11.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv 2023-10-16T09:34:11.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-16T09:34:11.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154 2023-10-16T09:34:11.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-16T09:34:11.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ifb.ko 2023-10-16T09:34:11.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan 2023-10-16T09:34:11.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-16T09:34:11.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvlan.ko 2023-10-16T09:34:11.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvtap.ko 2023-10-16T09:34:11.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio 2023-10-16T09:34:11.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio.ko 2023-10-16T09:34:11.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-16T09:34:11.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-16T09:34:11.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-16T09:34:11.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mii.ko 2023-10-16T09:34:11.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/net_failover.ko 2023-10-16T09:34:11.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/netconsole.ko 2023-10-16T09:34:11.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/nlmon.ko 2023-10-16T09:34:11.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs 2023-10-16T09:34:11.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-16T09:34:11.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-16T09:34:11.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy 2023-10-16T09:34:11.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/amd.ko 2023-10-16T09:34:11.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-16T09:34:11.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-16T09:34:11.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-16T09:34:11.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-16T09:34:11.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/broadcom.ko 2023-10-16T09:34:11.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/cicada.ko 2023-10-16T09:34:11.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/davicom.ko 2023-10-16T09:34:11.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/dp83640.ko 2023-10-16T09:34:11.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/et1011c.ko 2023-10-16T09:34:11.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/icplus.ko 2023-10-16T09:34:11.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/lxt.ko 2023-10-16T09:34:11.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/marvell.ko 2023-10-16T09:34:11.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/micrel.ko 2023-10-16T09:34:11.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/national.ko 2023-10-16T09:34:11.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/phylink.ko 2023-10-16T09:34:11.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/qsemi.ko 2023-10-16T09:34:11.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/realtek.ko 2023-10-16T09:34:11.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/smsc.ko 2023-10-16T09:34:11.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-16T09:34:11.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/vitesse.ko 2023-10-16T09:34:11.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp 2023-10-16T09:34:11.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-16T09:34:11.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-16T09:34:11.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-16T09:34:11.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-16T09:34:11.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-16T09:34:11.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-16T09:34:11.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppoe.ko 2023-10-16T09:34:11.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppox.ko 2023-10-16T09:34:11.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pptp.ko 2023-10-16T09:34:11.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip 2023-10-16T09:34:11.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slhc.ko 2023-10-16T09:34:11.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slip.ko 2023-10-16T09:34:11.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sungem_phy.ko 2023-10-16T09:34:11.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tap.ko 2023-10-16T09:34:11.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team 2023-10-16T09:34:11.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team.ko 2023-10-16T09:34:11.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-16T09:34:11.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-16T09:34:11.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-16T09:34:11.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_random.ko 2023-10-16T09:34:11.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-16T09:34:11.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tun.ko 2023-10-16T09:34:11.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/veth.ko 2023-10-16T09:34:11.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/virtio_net.ko 2023-10-16T09:34:11.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3 2023-10-16T09:34:11.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-16T09:34:11.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan 2023-10-16T09:34:11.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-16T09:34:11.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback 2023-10-16T09:34:11.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-16T09:34:11.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netfront.ko 2023-10-16T09:34:11.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb 2023-10-16T09:34:11.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ntb.ko 2023-10-16T09:34:11.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme 2023-10-16T09:34:11.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host 2023-10-16T09:34:11.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-16T09:34:11.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme.ko 2023-10-16T09:34:11.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport 2023-10-16T09:34:11.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport.ko 2023-10-16T09:34:11.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_cs.ko 2023-10-16T09:34:11.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_pc.ko 2023-10-16T09:34:11.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_serial.ko 2023-10-16T09:34:11.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci 2023-10-16T09:34:11.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug 2023-10-16T09:34:11.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-16T09:34:11.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie 2023-10-16T09:34:11.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-16T09:34:11.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/xen-pcifront.ko 2023-10-16T09:34:11.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia 2023-10-16T09:34:11.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/i82092.ko 2023-10-16T09:34:11.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pd6729.ko 2023-10-16T09:34:11.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-16T09:34:11.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl 2023-10-16T09:34:11.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel 2023-10-16T09:34:11.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-16T09:34:11.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform 2023-10-16T09:34:11.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86 2023-10-16T09:34:11.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-16T09:34:11.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-16T09:34:11.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-16T09:34:11.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-16T09:34:11.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-16T09:34:11.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-16T09:34:11.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-16T09:34:11.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-16T09:34:11.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-16T09:34:11.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-16T09:34:11.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-16T09:34:11.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/hdaps.ko 2023-10-16T09:34:11.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel 2023-10-16T09:34:11.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-16T09:34:11.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-16T09:34:11.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-16T09:34:11.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-16T09:34:11.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-16T09:34:11.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-16T09:34:11.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-16T09:34:11.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-16T09:34:11.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-16T09:34:11.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-16T09:34:11.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-16T09:34:11.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi.ko 2023-10-16T09:34:11.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap 2023-10-16T09:34:11.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-16T09:34:11.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-16T09:34:11.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps 2023-10-16T09:34:11.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients 2023-10-16T09:34:11.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-16T09:34:11.950 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-16T09:34:11.950 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-16T09:34:11.950 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/pps_core.ko 2023-10-16T09:34:11.950 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp 2023-10-16T09:34:11.950 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp.ko 2023-10-16T09:34:11.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-16T09:34:11.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc 2023-10-16T09:34:11.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-16T09:34:11.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-16T09:34:11.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-16T09:34:11.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-16T09:34:11.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-16T09:34:11.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-16T09:34:11.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-16T09:34:11.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-16T09:34:11.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-16T09:34:11.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-16T09:34:11.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-16T09:34:11.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-16T09:34:11.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-16T09:34:11.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-16T09:34:11.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-16T09:34:11.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-16T09:34:11.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-16T09:34:11.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-16T09:34:11.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-16T09:34:11.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-16T09:34:11.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-16T09:34:11.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-16T09:34:11.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-16T09:34:11.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-16T09:34:11.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-16T09:34:11.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-16T09:34:11.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-16T09:34:11.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-16T09:34:11.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-16T09:34:11.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-16T09:34:11.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-16T09:34:11.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi 2023-10-16T09:34:11.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-16T09:34:11.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-sas.ko 2023-10-16T09:34:11.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-16T09:34:11.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/BusLogic.ko 2023-10-16T09:34:11.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/a100u2w.ko 2023-10-16T09:34:11.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid 2023-10-16T09:34:11.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-16T09:34:11.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/advansys.ko 2023-10-16T09:34:11.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx 2023-10-16T09:34:11.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-16T09:34:11.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-16T09:34:11.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/am53c974.ko 2023-10-16T09:34:11.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr 2023-10-16T09:34:11.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-16T09:34:11.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/atp870u.ko 2023-10-16T09:34:11.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi 2023-10-16T09:34:11.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-16T09:34:11.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa 2023-10-16T09:34:11.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-16T09:34:11.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc 2023-10-16T09:34:11.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-16T09:34:11.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i 2023-10-16T09:34:11.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-16T09:34:11.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ch.ko 2023-10-16T09:34:11.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor 2023-10-16T09:34:11.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-16T09:34:11.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi 2023-10-16T09:34:11.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-16T09:34:11.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-16T09:34:11.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-16T09:34:11.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-16T09:34:11.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-16T09:34:11.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dc395x.ko 2023-10-16T09:34:11.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler 2023-10-16T09:34:11.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-16T09:34:11.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-16T09:34:11.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-16T09:34:11.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-16T09:34:11.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dmx3191d.ko 2023-10-16T09:34:11.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r 2023-10-16T09:34:11.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-16T09:34:11.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esp_scsi.ko 2023-10-16T09:34:11.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe 2023-10-16T09:34:11.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-16T09:34:11.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-16T09:34:11.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic 2023-10-16T09:34:11.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-16T09:34:11.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hpsa.ko 2023-10-16T09:34:11.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hptiop.ko 2023-10-16T09:34:11.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-16T09:34:11.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/initio.ko 2023-10-16T09:34:11.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ipr.ko 2023-10-16T09:34:11.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ips.ko 2023-10-16T09:34:11.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci 2023-10-16T09:34:11.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/isci.ko 2023-10-16T09:34:11.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-16T09:34:11.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-16T09:34:11.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc 2023-10-16T09:34:11.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-16T09:34:11.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi.ko 2023-10-16T09:34:11.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-16T09:34:11.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas 2023-10-16T09:34:11.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-16T09:34:11.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc 2023-10-16T09:34:11.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-16T09:34:11.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid 2023-10-16T09:34:11.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid.ko 2023-10-16T09:34:11.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-16T09:34:11.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-16T09:34:11.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-16T09:34:11.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas 2023-10-16T09:34:11.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-16T09:34:11.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas 2023-10-16T09:34:11.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-16T09:34:11.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvumi.ko 2023-10-16T09:34:11.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001 2023-10-16T09:34:11.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-16T09:34:11.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pmcraid.ko 2023-10-16T09:34:11.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla1280.ko 2023-10-16T09:34:11.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx 2023-10-16T09:34:11.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-16T09:34:11.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-16T09:34:11.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx 2023-10-16T09:34:11.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-16T09:34:11.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/raid_class.ko 2023-10-16T09:34:11.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_debug.ko 2023-10-16T09:34:11.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-16T09:34:11.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-16T09:34:11.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-16T09:34:11.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-16T09:34:11.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-16T09:34:11.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ses.ko 2023-10-16T09:34:11.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/st.ko 2023-10-16T09:34:11.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/stex.ko 2023-10-16T09:34:11.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2 2023-10-16T09:34:11.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-16T09:34:11.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-16T09:34:11.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-16T09:34:11.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/wd719x.ko 2023-10-16T09:34:11.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb 2023-10-16T09:34:11.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ssb.ko 2023-10-16T09:34:11.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target 2023-10-16T09:34:11.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi 2023-10-16T09:34:11.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-16T09:34:11.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback 2023-10-16T09:34:11.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-16T09:34:11.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_file.ko 2023-10-16T09:34:11.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_iblock.ko 2023-10-16T09:34:11.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_mod.ko 2023-10-16T09:34:11.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_pscsi.ko 2023-10-16T09:34:11.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_user.ko 2023-10-16T09:34:11.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc 2023-10-16T09:34:11.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-16T09:34:11.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal 2023-10-16T09:34:11.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel 2023-10-16T09:34:11.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-16T09:34:11.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-16T09:34:11.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-16T09:34:11.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-16T09:34:11.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-16T09:34:11.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-16T09:34:11.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-16T09:34:11.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-16T09:34:11.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-16T09:34:11.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-16T09:34:11.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-16T09:34:11.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-16T09:34:11.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-16T09:34:11.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-16T09:34:11.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-16T09:34:11.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-16T09:34:11.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty 2023-10-16T09:34:11.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless 2023-10-16T09:34:11.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-16T09:34:11.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_gsm.ko 2023-10-16T09:34:11.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_hdlc.ko 2023-10-16T09:34:11.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/nozomi.ko 2023-10-16T09:34:11.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial 2023-10-16T09:34:11.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250 2023-10-16T09:34:11.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-16T09:34:11.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-16T09:34:11.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm 2023-10-16T09:34:11.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-16T09:34:11.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs 2023-10-16T09:34:11.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core 2023-10-16T09:34:11.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-16T09:34:11.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host 2023-10-16T09:34:11.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-16T09:34:11.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio 2023-10-16T09:34:11.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio.ko 2023-10-16T09:34:11.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_aec.ko 2023-10-16T09:34:11.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_cif.ko 2023-10-16T09:34:11.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-16T09:34:11.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_sercos3.ko 2023-10-16T09:34:11.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb 2023-10-16T09:34:11.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class 2023-10-16T09:34:11.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-16T09:34:11.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-16T09:34:11.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usblp.ko 2023-10-16T09:34:11.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usbtmc.ko 2023-10-16T09:34:11.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host 2023-10-16T09:34:11.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-16T09:34:11.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image 2023-10-16T09:34:11.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/mdc800.ko 2023-10-16T09:34:11.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/microtek.ko 2023-10-16T09:34:11.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc 2023-10-16T09:34:11.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/adutux.ko 2023-10-16T09:34:11.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-16T09:34:11.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi26.ko 2023-10-16T09:34:11.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi62.ko 2023-10-16T09:34:11.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ezusb.ko 2023-10-16T09:34:11.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/idmouse.ko 2023-10-16T09:34:11.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-16T09:34:11.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-16T09:34:11.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ldusb.ko 2023-10-16T09:34:11.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-16T09:34:11.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga 2023-10-16T09:34:11.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-16T09:34:11.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-16T09:34:11.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usb3503.ko 2023-10-16T09:34:11.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usblcd.ko 2023-10-16T09:34:11.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-16T09:34:11.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/uss720.ko 2023-10-16T09:34:11.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/yurex.ko 2023-10-16T09:34:11.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy 2023-10-16T09:34:11.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-16T09:34:11.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial 2023-10-16T09:34:11.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/aircable.ko 2023-10-16T09:34:11.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ark3116.ko 2023-10-16T09:34:11.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-16T09:34:11.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ch341.ko 2023-10-16T09:34:11.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cp210x.ko 2023-10-16T09:34:11.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-16T09:34:11.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-16T09:34:11.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-16T09:34:11.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/empeg.ko 2023-10-16T09:34:11.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-16T09:34:11.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-16T09:34:11.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-16T09:34:11.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_ti.ko 2023-10-16T09:34:11.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipaq.ko 2023-10-16T09:34:11.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipw.ko 2023-10-16T09:34:11.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-16T09:34:11.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-16T09:34:11.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan.ko 2023-10-16T09:34:11.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-16T09:34:11.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-16T09:34:11.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-16T09:34:11.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-16T09:34:11.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7720.ko 2023-10-16T09:34:11.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7840.ko 2023-10-16T09:34:11.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/navman.ko 2023-10-16T09:34:11.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/omninet.ko 2023-10-16T09:34:11.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/opticon.ko 2023-10-16T09:34:11.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/option.ko 2023-10-16T09:34:11.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/oti6858.ko 2023-10-16T09:34:11.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/pl2303.ko 2023-10-16T09:34:12.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcaux.ko 2023-10-16T09:34:12.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcserial.ko 2023-10-16T09:34:12.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/quatech2.ko 2023-10-16T09:34:12.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-16T09:34:12.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/sierra.ko 2023-10-16T09:34:12.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-16T09:34:12.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ssu100.ko 2023-10-16T09:34:12.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-16T09:34:12.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-16T09:34:12.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-16T09:34:12.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-16T09:34:12.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-16T09:34:12.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/visor.ko 2023-10-16T09:34:12.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-16T09:34:12.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-16T09:34:12.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage 2023-10-16T09:34:12.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/uas.ko 2023-10-16T09:34:12.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-16T09:34:12.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-16T09:34:12.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-16T09:34:12.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-16T09:34:12.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-16T09:34:12.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-16T09:34:12.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-16T09:34:12.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-16T09:34:12.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-16T09:34:12.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-16T09:34:12.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-16T09:34:12.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-16T09:34:12.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-16T09:34:12.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-16T09:34:12.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip 2023-10-16T09:34:12.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-16T09:34:12.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-16T09:34:12.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-16T09:34:12.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio 2023-10-16T09:34:12.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci 2023-10-16T09:34:12.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-16T09:34:12.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-16T09:34:12.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio.ko 2023-10-16T09:34:12.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-16T09:34:12.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost 2023-10-16T09:34:12.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost.ko 2023-10-16T09:34:12.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-16T09:34:12.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_net.ko 2023-10-16T09:34:12.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-16T09:34:12.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video 2023-10-16T09:34:12.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight 2023-10-16T09:34:12.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-16T09:34:12.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lcd.ko 2023-10-16T09:34:12.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-16T09:34:12.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio 2023-10-16T09:34:12.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio.ko 2023-10-16T09:34:12.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-16T09:34:12.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-16T09:34:12.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci.ko 2023-10-16T09:34:12.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-16T09:34:12.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-16T09:34:12.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_ring.ko 2023-10-16T09:34:12.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1 2023-10-16T09:34:12.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters 2023-10-16T09:34:12.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2482.ko 2023-10-16T09:34:12.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2490.ko 2023-10-16T09:34:12.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves 2023-10-16T09:34:12.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-16T09:34:12.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-16T09:34:12.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-16T09:34:12.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-16T09:34:12.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-16T09:34:12.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-16T09:34:12.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-16T09:34:12.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-16T09:34:12.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-16T09:34:12.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-16T09:34:12.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-16T09:34:12.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/wire.ko 2023-10-16T09:34:12.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog 2023-10-16T09:34:12.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-16T09:34:12.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-16T09:34:12.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-16T09:34:12.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/hpwdt.ko 2023-10-16T09:34:12.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/i6300esb.ko 2023-10-16T09:34:12.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-16T09:34:12.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-16T09:34:12.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-16T09:34:12.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ibmasr.ko 2023-10-16T09:34:12.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-16T09:34:12.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-16T09:34:12.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-16T09:34:12.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/machzwd.ko 2023-10-16T09:34:12.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/nv_tco.ko 2023-10-16T09:34:12.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-16T09:34:12.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-16T09:34:12.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-16T09:34:12.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-16T09:34:12.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/softdog.ko 2023-10-16T09:34:12.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-16T09:34:12.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/via_wdt.ko 2023-10-16T09:34:12.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-16T09:34:12.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-16T09:34:12.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-16T09:34:12.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-16T09:34:12.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-16T09:34:12.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen 2023-10-16T09:34:12.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-16T09:34:12.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-evtchn.ko 2023-10-16T09:34:12.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-16T09:34:12.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntdev.ko 2023-10-16T09:34:12.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback 2023-10-16T09:34:12.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-16T09:34:12.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-privcmd.ko 2023-10-16T09:34:12.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs 2023-10-16T09:34:12.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-16T09:34:12.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs 2023-10-16T09:34:12.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p 2023-10-16T09:34:12.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/9p.ko 2023-10-16T09:34:12.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs 2023-10-16T09:34:12.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/affs.ko 2023-10-16T09:34:12.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs 2023-10-16T09:34:12.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/befs.ko 2023-10-16T09:34:12.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/binfmt_misc.ko 2023-10-16T09:34:12.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs 2023-10-16T09:34:12.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/btrfs.ko 2023-10-16T09:34:12.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles 2023-10-16T09:34:12.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/cachefiles.ko 2023-10-16T09:34:12.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph 2023-10-16T09:34:12.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ceph.ko 2023-10-16T09:34:12.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda 2023-10-16T09:34:12.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/coda.ko 2023-10-16T09:34:12.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs 2023-10-16T09:34:12.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/cramfs.ko 2023-10-16T09:34:12.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm 2023-10-16T09:34:12.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/dlm.ko 2023-10-16T09:34:12.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs 2023-10-16T09:34:12.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-16T09:34:12.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs 2023-10-16T09:34:12.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/f2fs.ko 2023-10-16T09:34:12.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat 2023-10-16T09:34:12.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/fat.ko 2023-10-16T09:34:12.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/msdos.ko 2023-10-16T09:34:12.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/vfat.ko 2023-10-16T09:34:12.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache 2023-10-16T09:34:12.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/fscache.ko 2023-10-16T09:34:12.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse 2023-10-16T09:34:12.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/cuse.ko 2023-10-16T09:34:12.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/fuse.ko 2023-10-16T09:34:12.030 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2 2023-10-16T09:34:12.030 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/gfs2.ko 2023-10-16T09:34:12.030 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs 2023-10-16T09:34:12.030 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/hfs.ko 2023-10-16T09:34:12.031 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus 2023-10-16T09:34:12.031 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/hfsplus.ko 2023-10-16T09:34:12.031 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs 2023-10-16T09:34:12.031 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/isofs.ko 2023-10-16T09:34:12.031 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs 2023-10-16T09:34:12.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/jfs.ko 2023-10-16T09:34:12.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd 2023-10-16T09:34:12.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/lockd.ko 2023-10-16T09:34:12.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix 2023-10-16T09:34:12.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/minix.ko 2023-10-16T09:34:12.033 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs 2023-10-16T09:34:12.033 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/netfs.ko 2023-10-16T09:34:12.033 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs 2023-10-16T09:34:12.033 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout 2023-10-16T09:34:12.033 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-16T09:34:12.034 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout 2023-10-16T09:34:12.034 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-16T09:34:12.034 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout 2023-10-16T09:34:12.034 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-16T09:34:12.034 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfs.ko 2023-10-16T09:34:12.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv3.ko 2023-10-16T09:34:12.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv4.ko 2023-10-16T09:34:12.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common 2023-10-16T09:34:12.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/grace.ko 2023-10-16T09:34:12.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-16T09:34:12.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd 2023-10-16T09:34:12.036 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/nfsd.ko 2023-10-16T09:34:12.036 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2 2023-10-16T09:34:12.036 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/nilfs2.ko 2023-10-16T09:34:12.036 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls 2023-10-16T09:34:12.036 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-celtic.ko 2023-10-16T09:34:12.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-centeuro.ko 2023-10-16T09:34:12.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-croatian.ko 2023-10-16T09:34:12.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-cyrillic.ko 2023-10-16T09:34:12.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-gaelic.ko 2023-10-16T09:34:12.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-greek.ko 2023-10-16T09:34:12.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-iceland.ko 2023-10-16T09:34:12.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-inuit.ko 2023-10-16T09:34:12.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-roman.ko 2023-10-16T09:34:12.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-romanian.ko 2023-10-16T09:34:12.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-turkish.ko 2023-10-16T09:34:12.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1250.ko 2023-10-16T09:34:12.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1251.ko 2023-10-16T09:34:12.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1255.ko 2023-10-16T09:34:12.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp737.ko 2023-10-16T09:34:12.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp775.ko 2023-10-16T09:34:12.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp850.ko 2023-10-16T09:34:12.040 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp852.ko 2023-10-16T09:34:12.040 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp855.ko 2023-10-16T09:34:12.040 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp857.ko 2023-10-16T09:34:12.040 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp860.ko 2023-10-16T09:34:12.040 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp861.ko 2023-10-16T09:34:12.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp862.ko 2023-10-16T09:34:12.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp863.ko 2023-10-16T09:34:12.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp864.ko 2023-10-16T09:34:12.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp865.ko 2023-10-16T09:34:12.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp866.ko 2023-10-16T09:34:12.042 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp869.ko 2023-10-16T09:34:12.042 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp874.ko 2023-10-16T09:34:12.042 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp932.ko 2023-10-16T09:34:12.042 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp936.ko 2023-10-16T09:34:12.043 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp949.ko 2023-10-16T09:34:12.043 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp950.ko 2023-10-16T09:34:12.043 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_euc-jp.ko 2023-10-16T09:34:12.043 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-16T09:34:12.043 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-16T09:34:12.043 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-16T09:34:12.044 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-16T09:34:12.044 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-16T09:34:12.044 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-16T09:34:12.044 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-16T09:34:12.044 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-16T09:34:12.045 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-16T09:34:12.045 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-16T09:34:12.045 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-16T09:34:12.045 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-r.ko 2023-10-16T09:34:12.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-16T09:34:12.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-u.ko 2023-10-16T09:34:12.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-16T09:34:12.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_utf8.ko 2023-10-16T09:34:12.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2 2023-10-16T09:34:12.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster 2023-10-16T09:34:12.047 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-16T09:34:12.047 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm 2023-10-16T09:34:12.047 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-16T09:34:12.047 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs 2023-10-16T09:34:12.048 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-16T09:34:12.048 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2.ko 2023-10-16T09:34:12.048 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-16T09:34:12.048 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-16T09:34:12.048 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-16T09:34:12.049 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs 2023-10-16T09:34:12.049 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/overlay.ko 2023-10-16T09:34:12.049 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore 2023-10-16T09:34:12.049 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ramoops.ko 2023-10-16T09:34:12.049 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs 2023-10-16T09:34:12.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/reiserfs.ko 2023-10-16T09:34:12.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs 2023-10-16T09:34:12.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/romfs.ko 2023-10-16T09:34:12.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb 2023-10-16T09:34:12.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client 2023-10-16T09:34:12.051 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/cifs.ko 2023-10-16T09:34:12.051 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common 2023-10-16T09:34:12.051 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-16T09:34:12.051 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_md4.ko 2023-10-16T09:34:12.051 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs 2023-10-16T09:34:12.052 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/squashfs.ko 2023-10-16T09:34:12.052 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv 2023-10-16T09:34:12.052 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/sysv.ko 2023-10-16T09:34:12.052 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf 2023-10-16T09:34:12.052 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/udf.ko 2023-10-16T09:34:12.053 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs 2023-10-16T09:34:12.053 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ufs.ko 2023-10-16T09:34:12.053 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs 2023-10-16T09:34:12.053 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/xfs.ko 2023-10-16T09:34:12.053 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel 2023-10-16T09:34:12.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu 2023-10-16T09:34:12.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu/rcutorture.ko 2023-10-16T09:34:12.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/torture.ko 2023-10-16T09:34:12.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace 2023-10-16T09:34:12.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-16T09:34:12.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib 2023-10-16T09:34:12.055 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/asn1_encoder.ko 2023-10-16T09:34:12.055 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc-itu-t.ko 2023-10-16T09:34:12.055 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc8.ko 2023-10-16T09:34:12.055 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto 2023-10-16T09:34:12.056 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libarc4.ko 2023-10-16T09:34:12.056 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libdes.ko 2023-10-16T09:34:12.056 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/libcrc32c.ko 2023-10-16T09:34:12.056 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lru_cache.ko 2023-10-16T09:34:12.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4 2023-10-16T09:34:12.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4_compress.ko 2023-10-16T09:34:12.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-16T09:34:12.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math 2023-10-16T09:34:12.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/cordic.ko 2023-10-16T09:34:12.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6 2023-10-16T09:34:12.058 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/raid6_pq.ko 2023-10-16T09:34:12.058 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon 2023-10-16T09:34:12.058 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-16T09:34:12.058 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_bm.ko 2023-10-16T09:34:12.058 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_fsm.ko 2023-10-16T09:34:12.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_kmp.ko 2023-10-16T09:34:12.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd 2023-10-16T09:34:12.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd/zstd_compress.ko 2023-10-16T09:34:12.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm 2023-10-16T09:34:12.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/hwpoison-inject.ko 2023-10-16T09:34:12.060 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net 2023-10-16T09:34:12.060 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan 2023-10-16T09:34:12.060 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/6lowpan.ko 2023-10-16T09:34:12.060 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_dest.ko 2023-10-16T09:34:12.060 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-16T09:34:12.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_hop.ko 2023-10-16T09:34:12.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-16T09:34:12.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-16T09:34:12.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_routing.ko 2023-10-16T09:34:12.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_udp.ko 2023-10-16T09:34:12.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802 2023-10-16T09:34:12.062 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/garp.ko 2023-10-16T09:34:12.062 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/mrp.ko 2023-10-16T09:34:12.062 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/p8022.ko 2023-10-16T09:34:12.062 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/psnap.ko 2023-10-16T09:34:12.063 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/stp.ko 2023-10-16T09:34:12.063 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q 2023-10-16T09:34:12.063 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/8021q.ko 2023-10-16T09:34:12.063 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p 2023-10-16T09:34:12.063 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet.ko 2023-10-16T09:34:12.064 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_fd.ko 2023-10-16T09:34:12.064 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_rdma.ko 2023-10-16T09:34:12.064 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_virtio.ko 2023-10-16T09:34:12.064 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv 2023-10-16T09:34:12.065 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/batman-adv.ko 2023-10-16T09:34:12.065 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge 2023-10-16T09:34:12.065 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/br_netfilter.ko 2023-10-16T09:34:12.065 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/bridge.ko 2023-10-16T09:34:12.065 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter 2023-10-16T09:34:12.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-16T09:34:12.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-16T09:34:12.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-16T09:34:12.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-16T09:34:12.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-16T09:34:12.067 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-16T09:34:12.067 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-16T09:34:12.067 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-16T09:34:12.067 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-16T09:34:12.067 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-16T09:34:12.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-16T09:34:12.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-16T09:34:12.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-16T09:34:12.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-16T09:34:12.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-16T09:34:12.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-16T09:34:12.069 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-16T09:34:12.069 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-16T09:34:12.069 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-16T09:34:12.069 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-16T09:34:12.069 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-16T09:34:12.070 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-16T09:34:12.070 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-16T09:34:12.070 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-16T09:34:12.070 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph 2023-10-16T09:34:12.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/libceph.ko 2023-10-16T09:34:12.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core 2023-10-16T09:34:12.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/failover.ko 2023-10-16T09:34:12.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/pktgen.ko 2023-10-16T09:34:12.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp 2023-10-16T09:34:12.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp.ko 2023-10-16T09:34:12.072 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_diag.ko 2023-10-16T09:34:12.072 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv4.ko 2023-10-16T09:34:12.072 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv6.ko 2023-10-16T09:34:12.072 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver 2023-10-16T09:34:12.072 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-16T09:34:12.073 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa 2023-10-16T09:34:12.073 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/dsa_core.ko 2023-10-16T09:34:12.073 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_brcm.ko 2023-10-16T09:34:12.073 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_dsa.ko 2023-10-16T09:34:12.073 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_none.ko 2023-10-16T09:34:12.074 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_trailer.ko 2023-10-16T09:34:12.074 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154 2023-10-16T09:34:12.074 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan 2023-10-16T09:34:12.074 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-16T09:34:12.074 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154.ko 2023-10-16T09:34:12.075 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-16T09:34:12.075 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4 2023-10-16T09:34:12.075 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ah4.ko 2023-10-16T09:34:12.075 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/esp4.ko 2023-10-16T09:34:12.075 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/fou.ko 2023-10-16T09:34:12.075 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/gre.ko 2023-10-16T09:34:12.076 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/inet_diag.ko 2023-10-16T09:34:12.076 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_gre.ko 2023-10-16T09:34:12.076 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_tunnel.ko 2023-10-16T09:34:12.076 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_vti.ko 2023-10-16T09:34:12.076 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipcomp.ko 2023-10-16T09:34:12.077 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipip.ko 2023-10-16T09:34:12.077 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter 2023-10-16T09:34:12.077 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-16T09:34:12.077 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-16T09:34:12.077 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-16T09:34:12.077 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-16T09:34:12.078 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-16T09:34:12.078 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-16T09:34:12.078 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-16T09:34:12.078 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-16T09:34:12.078 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-16T09:34:12.079 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-16T09:34:12.079 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-16T09:34:12.079 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-16T09:34:12.079 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-16T09:34:12.079 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-16T09:34:12.080 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-16T09:34:12.080 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-16T09:34:12.080 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-16T09:34:12.080 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-16T09:34:12.080 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-16T09:34:12.081 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-16T09:34:12.081 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-16T09:34:12.081 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_bic.ko 2023-10-16T09:34:12.081 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-16T09:34:12.081 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_diag.ko 2023-10-16T09:34:12.081 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-16T09:34:12.082 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_htcp.ko 2023-10-16T09:34:12.082 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_hybla.ko 2023-10-16T09:34:12.082 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_illinois.ko 2023-10-16T09:34:12.082 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_lp.ko 2023-10-16T09:34:12.082 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_scalable.ko 2023-10-16T09:34:12.083 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_vegas.ko 2023-10-16T09:34:12.083 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_veno.ko 2023-10-16T09:34:12.083 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_westwood.ko 2023-10-16T09:34:12.083 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_yeah.ko 2023-10-16T09:34:12.083 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tunnel4.ko 2023-10-16T09:34:12.084 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_diag.ko 2023-10-16T09:34:12.084 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_tunnel.ko 2023-10-16T09:34:12.084 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-16T09:34:12.084 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6 2023-10-16T09:34:12.084 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ah6.ko 2023-10-16T09:34:12.085 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/esp6.ko 2023-10-16T09:34:12.085 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/fou6.ko 2023-10-16T09:34:12.085 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-16T09:34:12.085 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-16T09:34:12.085 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_vti.ko 2023-10-16T09:34:12.086 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ipcomp6.ko 2023-10-16T09:34:12.086 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter 2023-10-16T09:34:12.086 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-16T09:34:12.086 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-16T09:34:12.087 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-16T09:34:12.087 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-16T09:34:12.087 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-16T09:34:12.087 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-16T09:34:12.088 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-16T09:34:12.088 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-16T09:34:12.088 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-16T09:34:12.088 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-16T09:34:12.088 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-16T09:34:12.089 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-16T09:34:12.089 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-16T09:34:12.089 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-16T09:34:12.089 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-16T09:34:12.089 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-16T09:34:12.090 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-16T09:34:12.090 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-16T09:34:12.090 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-16T09:34:12.090 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-16T09:34:12.090 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-16T09:34:12.091 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-16T09:34:12.091 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-16T09:34:12.091 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-16T09:34:12.091 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/sit.ko 2023-10-16T09:34:12.091 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/tunnel6.ko 2023-10-16T09:34:12.092 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-16T09:34:12.092 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key 2023-10-16T09:34:12.092 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/af_key.ko 2023-10-16T09:34:12.092 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp 2023-10-16T09:34:12.092 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_core.ko 2023-10-16T09:34:12.092 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-16T09:34:12.093 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_eth.ko 2023-10-16T09:34:12.093 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip.ko 2023-10-16T09:34:12.093 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-16T09:34:12.093 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-16T09:34:12.093 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-16T09:34:12.094 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc 2023-10-16T09:34:12.094 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc.ko 2023-10-16T09:34:12.094 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154 2023-10-16T09:34:12.094 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/mac802154.ko 2023-10-16T09:34:12.094 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls 2023-10-16T09:34:12.095 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/mpls_gso.ko 2023-10-16T09:34:12.095 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter 2023-10-16T09:34:12.095 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset 2023-10-16T09:34:12.095 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-16T09:34:12.095 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-16T09:34:12.096 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-16T09:34:12.096 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-16T09:34:12.096 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-16T09:34:12.096 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-16T09:34:12.096 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-16T09:34:12.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-16T09:34:12.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-16T09:34:12.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-16T09:34:12.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-16T09:34:12.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-16T09:34:12.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-16T09:34:12.098 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-16T09:34:12.098 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-16T09:34:12.098 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-16T09:34:12.098 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs 2023-10-16T09:34:12.098 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-16T09:34:12.099 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-16T09:34:12.099 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-16T09:34:12.099 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-16T09:34:12.099 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-16T09:34:12.099 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-16T09:34:12.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-16T09:34:12.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-16T09:34:12.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-16T09:34:12.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-16T09:34:12.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-16T09:34:12.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-16T09:34:12.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-16T09:34:12.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-16T09:34:12.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conncount.ko 2023-10-16T09:34:12.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack.ko 2023-10-16T09:34:12.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-16T09:34:12.102 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-16T09:34:12.102 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-16T09:34:12.102 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-16T09:34:12.102 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-16T09:34:12.102 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-16T09:34:12.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-16T09:34:12.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-16T09:34:12.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-16T09:34:12.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-16T09:34:12.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-16T09:34:12.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-16T09:34:12.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-16T09:34:12.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table.ko 2023-10-16T09:34:12.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-16T09:34:12.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-16T09:34:12.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat.ko 2023-10-16T09:34:12.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-16T09:34:12.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-16T09:34:12.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-16T09:34:12.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-16T09:34:12.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-16T09:34:12.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-16T09:34:12.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_tables.ko 2023-10-16T09:34:12.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink.ko 2023-10-16T09:34:12.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-16T09:34:12.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-16T09:34:12.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-16T09:34:12.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-16T09:34:12.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-16T09:34:12.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_compat.ko 2023-10-16T09:34:12.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_ct.ko 2023-10-16T09:34:12.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-16T09:34:12.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fib.ko 2023-10-16T09:34:12.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-16T09:34:12.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_hash.ko 2023-10-16T09:34:12.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_limit.ko 2023-10-16T09:34:12.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_log.ko 2023-10-16T09:34:12.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_masq.ko 2023-10-16T09:34:12.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_nat.ko 2023-10-16T09:34:12.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_numgen.ko 2023-10-16T09:34:12.110 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_queue.ko 2023-10-16T09:34:12.110 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_quota.ko 2023-10-16T09:34:12.110 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_redir.ko 2023-10-16T09:34:12.110 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject.ko 2023-10-16T09:34:12.110 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-16T09:34:12.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_socket.ko 2023-10-16T09:34:12.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_tunnel.ko 2023-10-16T09:34:12.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-16T09:34:12.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-16T09:34:12.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-16T09:34:12.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-16T09:34:12.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CT.ko 2023-10-16T09:34:12.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_DSCP.ko 2023-10-16T09:34:12.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HL.ko 2023-10-16T09:34:12.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HMARK.ko 2023-10-16T09:34:12.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-16T09:34:12.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LED.ko 2023-10-16T09:34:12.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LOG.ko 2023-10-16T09:34:12.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-16T09:34:12.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-16T09:34:12.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-16T09:34:12.114 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-16T09:34:12.114 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-16T09:34:12.114 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-16T09:34:12.114 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-16T09:34:12.114 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-16T09:34:12.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-16T09:34:12.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TEE.ko 2023-10-16T09:34:12.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-16T09:34:12.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TRACE.ko 2023-10-16T09:34:12.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_addrtype.ko 2023-10-16T09:34:12.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_bpf.ko 2023-10-16T09:34:12.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cgroup.ko 2023-10-16T09:34:12.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cluster.ko 2023-10-16T09:34:12.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_comment.ko 2023-10-16T09:34:12.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connbytes.ko 2023-10-16T09:34:12.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlabel.ko 2023-10-16T09:34:12.117 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlimit.ko 2023-10-16T09:34:12.117 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connmark.ko 2023-10-16T09:34:12.117 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_conntrack.ko 2023-10-16T09:34:12.118 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cpu.ko 2023-10-16T09:34:12.118 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dccp.ko 2023-10-16T09:34:12.118 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_devgroup.ko 2023-10-16T09:34:12.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dscp.ko 2023-10-16T09:34:12.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ecn.ko 2023-10-16T09:34:12.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_esp.ko 2023-10-16T09:34:12.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-16T09:34:12.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_helper.ko 2023-10-16T09:34:12.120 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hl.ko 2023-10-16T09:34:12.120 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-16T09:34:12.120 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_iprange.ko 2023-10-16T09:34:12.120 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipvs.ko 2023-10-16T09:34:12.121 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_l2tp.ko 2023-10-16T09:34:12.121 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_length.ko 2023-10-16T09:34:12.121 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_limit.ko 2023-10-16T09:34:12.121 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mac.ko 2023-10-16T09:34:12.122 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mark.ko 2023-10-16T09:34:12.122 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_multiport.ko 2023-10-16T09:34:12.122 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nat.ko 2023-10-16T09:34:12.122 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nfacct.ko 2023-10-16T09:34:12.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_osf.ko 2023-10-16T09:34:12.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_owner.ko 2023-10-16T09:34:12.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_physdev.ko 2023-10-16T09:34:12.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_pkttype.ko 2023-10-16T09:34:12.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_policy.ko 2023-10-16T09:34:12.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_quota.ko 2023-10-16T09:34:12.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_rateest.ko 2023-10-16T09:34:12.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_realm.ko 2023-10-16T09:34:12.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_recent.ko 2023-10-16T09:34:12.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_sctp.ko 2023-10-16T09:34:12.125 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_set.ko 2023-10-16T09:34:12.125 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_socket.ko 2023-10-16T09:34:12.125 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_state.ko 2023-10-16T09:34:12.125 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_statistic.ko 2023-10-16T09:34:12.126 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_string.ko 2023-10-16T09:34:12.126 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-16T09:34:12.126 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_time.ko 2023-10-16T09:34:12.126 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_u32.ko 2023-10-16T09:34:12.127 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink 2023-10-16T09:34:12.127 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/netlink_diag.ko 2023-10-16T09:34:12.127 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh 2023-10-16T09:34:12.127 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/nsh.ko 2023-10-16T09:34:12.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch 2023-10-16T09:34:12.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/openvswitch.ko 2023-10-16T09:34:12.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-geneve.ko 2023-10-16T09:34:12.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-gre.ko 2023-10-16T09:34:12.129 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-16T09:34:12.129 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet 2023-10-16T09:34:12.129 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/af_packet_diag.ko 2023-10-16T09:34:12.129 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds 2023-10-16T09:34:12.129 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds.ko 2023-10-16T09:34:12.130 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_rdma.ko 2023-10-16T09:34:12.130 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_tcp.ko 2023-10-16T09:34:12.130 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched 2023-10-16T09:34:12.130 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_csum.ko 2023-10-16T09:34:12.131 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_gact.ko 2023-10-16T09:34:12.131 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_ipt.ko 2023-10-16T09:34:12.131 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_mirred.ko 2023-10-16T09:34:12.131 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_nat.ko 2023-10-16T09:34:12.132 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_pedit.ko 2023-10-16T09:34:12.132 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_police.ko 2023-10-16T09:34:12.132 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_simple.ko 2023-10-16T09:34:12.132 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_skbedit.ko 2023-10-16T09:34:12.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_vlan.ko 2023-10-16T09:34:12.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_basic.ko 2023-10-16T09:34:12.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_bpf.ko 2023-10-16T09:34:12.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_flow.ko 2023-10-16T09:34:12.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_fw.ko 2023-10-16T09:34:12.134 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_route.ko 2023-10-16T09:34:12.134 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_u32.ko 2023-10-16T09:34:12.134 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_cmp.ko 2023-10-16T09:34:12.134 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_ipset.ko 2023-10-16T09:34:12.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_meta.ko 2023-10-16T09:34:12.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_nbyte.ko 2023-10-16T09:34:12.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_text.ko 2023-10-16T09:34:12.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_u32.ko 2023-10-16T09:34:12.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_choke.ko 2023-10-16T09:34:12.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_codel.ko 2023-10-16T09:34:12.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_drr.ko 2023-10-16T09:34:12.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq.ko 2023-10-16T09:34:12.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_gred.ko 2023-10-16T09:34:12.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hfsc.ko 2023-10-16T09:34:12.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hhf.ko 2023-10-16T09:34:12.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_htb.ko 2023-10-16T09:34:12.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_ingress.ko 2023-10-16T09:34:12.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio.ko 2023-10-16T09:34:12.138 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-16T09:34:12.138 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_multiq.ko 2023-10-16T09:34:12.138 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_netem.ko 2023-10-16T09:34:12.138 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_pie.ko 2023-10-16T09:34:12.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_plug.ko 2023-10-16T09:34:12.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_prio.ko 2023-10-16T09:34:12.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_qfq.ko 2023-10-16T09:34:12.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_red.ko 2023-10-16T09:34:12.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfb.ko 2023-10-16T09:34:12.140 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfq.ko 2023-10-16T09:34:12.140 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_tbf.ko 2023-10-16T09:34:12.140 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_teql.ko 2023-10-16T09:34:12.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp 2023-10-16T09:34:12.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp.ko 2023-10-16T09:34:12.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp_diag.ko 2023-10-16T09:34:12.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc 2023-10-16T09:34:12.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss 2023-10-16T09:34:12.142 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-16T09:34:12.142 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-16T09:34:12.142 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/sunrpc.ko 2023-10-16T09:34:12.142 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma 2023-10-16T09:34:12.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-16T09:34:12.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc 2023-10-16T09:34:12.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/diag.ko 2023-10-16T09:34:12.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/tipc.ko 2023-10-16T09:34:12.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix 2023-10-16T09:34:12.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/unix_diag.ko 2023-10-16T09:34:12.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock 2023-10-16T09:34:12.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-16T09:34:12.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-16T09:34:12.145 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock.ko 2023-10-16T09:34:12.145 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-16T09:34:12.145 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-16T09:34:12.145 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm 2023-10-16T09:34:12.146 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-16T09:34:12.146 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security 2023-10-16T09:34:12.146 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys 2023-10-16T09:34:12.146 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys 2023-10-16T09:34:12.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-16T09:34:12.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys 2023-10-16T09:34:12.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-16T09:34:12.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt 2023-10-16T09:34:12.148 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib 2023-10-16T09:34:12.148 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/irqbypass.ko 2023-10-16T09:34:12.148 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias 2023-10-16T09:34:12.148 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias.bin 2023-10-16T09:34:12.149 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin 2023-10-16T09:34:12.149 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.alias.bin 2023-10-16T09:34:12.149 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.bin 2023-10-16T09:34:12.149 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.modinfo 2023-10-16T09:34:12.150 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep 2023-10-16T09:34:12.150 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep.bin 2023-10-16T09:34:12.150 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.devname 2023-10-16T09:34:12.150 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.order 2023-10-16T09:34:12.150 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.softdep 2023-10-16T09:34:12.151 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols 2023-10-16T09:34:12.151 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols.bin 2023-10-16T09:34:12.152 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g47672119149b 2023-10-16T09:34:12.152 DEBUG:teuthology.orchestra.run.smithi084:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-16T09:34:12.175 INFO:teuthology.orchestra.run.smithi084.stdout:postinstall scriptlet (using /bin/sh): 2023-10-16T09:34:12.176 INFO:teuthology.orchestra.run.smithi084.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g47672119149b -a -r /boot/System.map-6.6.0-rc5-g47672119149b ]; then 2023-10-16T09:34:12.176 INFO:teuthology.orchestra.run.smithi084.stdout:cp /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm 2023-10-16T09:34:12.176 INFO:teuthology.orchestra.run.smithi084.stdout:cp /boot/System.map-6.6.0-rc5-g47672119149b /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-16T09:34:12.176 INFO:teuthology.orchestra.run.smithi084.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/System.map-6.6.0-rc5-g47672119149b 2023-10-16T09:34:12.177 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/installkernel 6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-16T09:34:12.177 INFO:teuthology.orchestra.run.smithi084.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-16T09:34:12.177 INFO:teuthology.orchestra.run.smithi084.stdout:fi 2023-10-16T09:34:12.177 INFO:teuthology.orchestra.run.smithi084.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-16T09:34:12.178 INFO:teuthology.orchestra.run.smithi084.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-16T09:34:12.178 INFO:teuthology.orchestra.run.smithi084.stdout:new-kernel-pkg --remove 6.6.0-rc5-g47672119149b --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g47672119149b.img 2023-10-16T09:34:12.178 INFO:teuthology.orchestra.run.smithi084.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-16T09:34:12.178 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-install remove 6.6.0-rc5-g47672119149b 2023-10-16T09:34:12.179 INFO:teuthology.orchestra.run.smithi084.stdout:fi 2023-10-16T09:34:12.179 INFO:teuthology.orchestra.run.smithi084.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-16T09:34:12.179 INFO:teuthology.orchestra.run.smithi084.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-16T09:34:12.179 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g47672119149b 2023-10-16T09:34:12.179 INFO:teuthology.orchestra.run.smithi084.stdout:fi 2023-10-16T09:34:12.181 DEBUG:teuthology.orchestra.run.smithi084:> sudo rpm -qi grub2-tools 2023-10-16T09:34:12.258 INFO:teuthology.orchestra.run.smithi084.stdout:Name : grub2-tools 2023-10-16T09:34:12.259 INFO:teuthology.orchestra.run.smithi084.stdout:Epoch : 1 2023-10-16T09:34:12.259 INFO:teuthology.orchestra.run.smithi084.stdout:Version : 2.06 2023-10-16T09:34:12.260 INFO:teuthology.orchestra.run.smithi084.stdout:Release : 68.el9 2023-10-16T09:34:12.260 INFO:teuthology.orchestra.run.smithi084.stdout:Architecture: x86_64 2023-10-16T09:34:12.260 INFO:teuthology.orchestra.run.smithi084.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-16T09:34:12.260 INFO:teuthology.orchestra.run.smithi084.stdout:Group : Unspecified 2023-10-16T09:34:12.260 INFO:teuthology.orchestra.run.smithi084.stdout:Size : 8264254 2023-10-16T09:34:12.261 INFO:teuthology.orchestra.run.smithi084.stdout:License : GPLv3+ 2023-10-16T09:34:12.261 INFO:teuthology.orchestra.run.smithi084.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-16T09:34:12.261 INFO:teuthology.orchestra.run.smithi084.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-16T09:34:12.261 INFO:teuthology.orchestra.run.smithi084.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-16T09:34:12.262 INFO:teuthology.orchestra.run.smithi084.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-16T09:34:12.262 INFO:teuthology.orchestra.run.smithi084.stdout:Packager : builder@centos.org 2023-10-16T09:34:12.262 INFO:teuthology.orchestra.run.smithi084.stdout:Vendor : CentOS 2023-10-16T09:34:12.262 INFO:teuthology.orchestra.run.smithi084.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-16T09:34:12.262 INFO:teuthology.orchestra.run.smithi084.stdout:Summary : Support tools for GRUB. 2023-10-16T09:34:12.263 INFO:teuthology.orchestra.run.smithi084.stdout:Description : 2023-10-16T09:34:12.263 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:34:12.263 INFO:teuthology.orchestra.run.smithi084.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-16T09:34:12.263 INFO:teuthology.orchestra.run.smithi084.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-16T09:34:12.264 INFO:teuthology.orchestra.run.smithi084.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-16T09:34:12.264 INFO:teuthology.orchestra.run.smithi084.stdout:hardware devices. 2023-10-16T09:34:12.264 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:34:12.265 INFO:teuthology.orchestra.run.smithi084.stdout:This subpackage provides tools for support of all platforms. 2023-10-16T09:34:12.266 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-16T09:34:12.267 INFO:teuthology.task.kernel:Updating grub on smithi084 to boot 6.6.0-rc5-g47672119149b 2023-10-16T09:34:12.267 DEBUG:teuthology.orchestra.run.smithi084:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-16T09:34:12.871 INFO:teuthology.orchestra.run.smithi084.stderr:Generating grub configuration file ... 2023-10-16T09:34:14.212 INFO:teuthology.orchestra.run.smithi084.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-16T09:34:14.248 INFO:teuthology.orchestra.run.smithi084.stderr:done 2023-10-16T09:34:14.251 DEBUG:teuthology.orchestra.run.smithi084:> mktemp 2023-10-16T09:34:14.266 INFO:teuthology.orchestra.run.smithi084.stdout:/tmp/tmp.uxLHMI8jEB 2023-10-16T09:34:14.266 DEBUG:teuthology.orchestra.run.smithi084:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.uxLHMI8jEB 2023-10-16T09:34:14.334 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0666 /tmp/tmp.uxLHMI8jEB 2023-10-16T09:34:14.473 DEBUG:teuthology.orchestra.remote:smithi084:/tmp/tmp.uxLHMI8jEB is 6KB 2023-10-16T09:34:14.523 DEBUG:teuthology.orchestra.run.smithi084:> rm -fr /tmp/tmp.uxLHMI8jEB 2023-10-16T09:34:14.538 DEBUG:teuthology.orchestra.run.smithi084:> sudo /bin/ls /boot/loader/entries || true 2023-10-16T09:34:14.609 INFO:teuthology.orchestra.run.smithi084.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-16T09:34:14.609 INFO:teuthology.orchestra.run.smithi084.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-16T09:34:14.610 INFO:teuthology.orchestra.run.smithi084.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-16T09:34:14.610 INFO:teuthology.orchestra.run.smithi084.stdout:15949e3c88704300882482eddfd7baae-6.6.0-rc5-g47672119149b.conf 2023-10-16T09:34:14.611 DEBUG:teuthology.orchestra.run.smithi084:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-rc5-g47672119149b 2023-10-16T09:34:14.780 DEBUG:teuthology.orchestra.run.smithi084:> sudo shutdown -r now 2023-10-16T09:34:34.074 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-16T09:34:34.075 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-16T09:34:34.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T09:34:44.794 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-16T09:34:44.795 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2023-10-16T09:34:44.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T09:34:52.538 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-10-16T09:35:01.546 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-16T09:35:01.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T09:35:03.290 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.84 2023-10-16T09:35:04.606 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-10-16T09:35:12.298 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2023-10-16T09:35:12.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T09:35:15.354 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.84 2023-10-16T09:35:16.608 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-16T09:35:16.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T09:35:19.681 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-10-16T09:35:27.362 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2023-10-16T09:35:27.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T09:35:34.686 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-16T09:35:34.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T09:35:53.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-10-16T09:35:59.279 DEBUG:teuthology.orchestra.run.smithi084:> true 2023-10-16T09:35:59.922 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2023-10-16T09:35:59.923 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-g47672119149b"... 2023-10-16T09:35:59.923 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2023-10-16T09:35:59.967 INFO:teuthology.orchestra.run.smithi084.stdout:6.6.0-rc5-g47672119149b 2023-10-16T09:35:59.968 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g47672119149b vs 6.6.0-rc5-g47672119149b 2023-10-16T09:35:59.968 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-16T09:35:59.968 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-16T09:36:00.970 DEBUG:teuthology.parallel:result is None 2023-10-16T09:36:11.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-16T09:36:11.217 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T09:36:11.682 DEBUG:teuthology.orchestra.run.smithi157:> true 2023-10-16T09:36:12.255 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-16T09:36:12.256 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-rc5-g47672119149b"... 2023-10-16T09:36:12.256 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2023-10-16T09:36:12.299 INFO:teuthology.orchestra.run.smithi157.stdout:6.6.0-rc5-g47672119149b 2023-10-16T09:36:12.299 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g47672119149b vs 6.6.0-rc5-g47672119149b 2023-10-16T09:36:12.300 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-16T09:36:12.300 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-16T09:36:13.301 DEBUG:teuthology.parallel:result is None 2023-10-16T09:36:13.301 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-16T09:36:13.311 INFO:teuthology.task.internal:Creating test directory... 2023-10-16T09:36:13.311 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-16T09:36:13.321 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-16T09:36:13.373 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-16T09:36:13.382 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-16T09:36:13.397 INFO:teuthology.task.internal:Creating archive directory... 2023-10-16T09:36:13.397 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-16T09:36:13.405 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-16T09:36:13.471 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-16T09:36:13.479 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-16T09:36:13.480 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-16T09:36:13.489 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-16T09:36:13.614 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T09:36:13.639 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T09:36:13.659 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T09:36:13.711 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T09:36:13.715 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-16T09:36:13.724 INFO:teuthology.task.internal:Configuring sudo... 2023-10-16T09:36:13.724 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-16T09:36:13.730 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-16T09:36:13.824 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-16T09:36:13.835 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-16T09:36:13.836 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-16T09:36:13.842 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-16T09:36:13.889 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T09:36:13.968 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T09:36:14.076 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T09:36:14.164 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T09:36:14.262 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:36:14.262 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-16T09:36:14.358 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T09:36:14.445 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T09:36:14.536 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T09:36:14.607 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T09:36:14.688 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:36:14.689 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-16T09:36:14.757 DEBUG:teuthology.orchestra.run.smithi084:> sudo service rsyslog restart 2023-10-16T09:36:14.766 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2023-10-16T09:36:14.896 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-16T09:36:14.908 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-16T09:36:15.385 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-16T09:36:15.393 INFO:teuthology.task.internal:Starting timer... 2023-10-16T09:36:15.394 INFO:teuthology.run_tasks:Running task pcp... 2023-10-16T09:36:15.407 INFO:teuthology.run_tasks:Running task selinux... 2023-10-16T09:36:15.417 DEBUG:teuthology.orchestra.run.smithi084:> sudo service auditd rotate 2023-10-16T09:36:15.543 INFO:teuthology.orchestra.run.smithi084.stdout:Rotating logs: 2023-10-16T09:36:15.547 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2023-10-16T09:36:15.685 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2023-10-16T09:36:15.689 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-16T09:36:15.689 DEBUG:teuthology.orchestra.run.smithi084:> /usr/sbin/getenforce 2023-10-16T09:36:15.749 INFO:teuthology.orchestra.run.smithi084.stdout:Permissive 2023-10-16T09:36:15.749 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2023-10-16T09:36:15.795 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2023-10-16T09:36:15.796 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi084.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2023-10-16T09:36:15.796 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-16T09:36:15.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T09:36:15.901 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-16T09:36:15.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T09:36:15.975 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-16T09:36:15.975 DEBUG:teuthology.orchestra.run.smithi084:> sudo /usr/sbin/setenforce permissive 2023-10-16T09:36:16.096 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2023-10-16T09:36:16.175 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-16T09:36:16.188 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-16T09:36:16.194 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-16T09:36:16.309 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-16T09:36:16.327 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-16T09:36:16.329 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi084.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-16T09:46:03.620 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi084.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2023-10-16T09:46:03.622 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2023-10-16T09:46:03.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T09:46:03.763 DEBUG:teuthology.orchestra.run.smithi084:> true 2023-10-16T09:46:03.916 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2023-10-16T09:46:03.917 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-16T09:46:03.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T09:46:04.050 DEBUG:teuthology.orchestra.run.smithi157:> true 2023-10-16T09:46:04.191 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-16T09:46:04.191 INFO:teuthology.run_tasks:Running task clock... 2023-10-16T09:46:04.202 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-16T09:46:04.202 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-16T09:46:04.203 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T09:46:04.212 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-16T09:46:04.212 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T09:46:04.311 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-16T09:46:04.316 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-16T09:46:04.359 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-16T09:46:04.368 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-16T09:46:04.448 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: ntpd: command not found 2023-10-16T09:46:04.463 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2023-10-16T09:46:04.487 INFO:teuthology.orchestra.run.smithi084.stdout:506 Cannot talk to daemon 2023-10-16T09:46:04.504 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2023-10-16T09:46:04.533 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-16T09:46:04.556 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-16T09:46:04.582 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-16T09:46:04.610 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-16T09:46:04.709 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2023-10-16T09:46:04.718 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-16T09:46:04.719 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2023-10-16T09:46:04.719 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T09:46:04.719 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T09:46:04.719 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T09:46:04.720 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T09:46:04.745 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2023-10-16T09:46:04.754 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-16T09:46:04.754 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2023-10-16T09:46:04.754 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T09:46:04.754 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T09:46:04.755 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T09:46:04.755 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T09:46:04.756 INFO:teuthology.run_tasks:Running task install... 2023-10-16T09:46:04.764 DEBUG:teuthology.task.install:project ceph 2023-10-16T09:46:04.765 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c7471068426370a3754c2250e7729280233d552b'}} 2023-10-16T09:46:04.765 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c7471068426370a3754c2250e7729280233d552b'} 2023-10-16T09:46:04.765 INFO:teuthology.task.install:Using flavor: default 2023-10-16T09:46:04.772 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-16T09:46:04.772 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-16T09:46:04.773 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c7471068426370a3754c2250e7729280233d552b', 'tag': None, 'wait_for_package': False} 2023-10-16T09:46:04.773 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c7471068426370a3754c2250e7729280233d552b 2023-10-16T09:46:04.775 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c7471068426370a3754c2250e7729280233d552b', 'tag': None, 'wait_for_package': False} 2023-10-16T09:46:04.776 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c7471068426370a3754c2250e7729280233d552b 2023-10-16T09:46:05.009 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/centos/9/flavors/default/ 2023-10-16T09:46:05.009 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6675.gc7471068 2023-10-16T09:46:05.013 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/centos/9/flavors/default/ 2023-10-16T09:46:05.014 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6675.gc7471068 2023-10-16T09:46:05.116 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-16T09:46:05.117 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:46:05.117 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-16T09:46:05.120 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-16T09:46:05.120 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:46:05.121 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-16T09:46:05.219 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-16T09:46:05.219 DEBUG:teuthology.orchestra.run.smithi084:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c7471068426370a3754c2250e7729280233d552b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-16T09:46:05.224 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-16T09:46:05.224 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c7471068426370a3754c2250e7729280233d552b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-16T09:46:05.359 DEBUG:teuthology.orchestra.run.smithi084:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-16T09:46:05.367 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-16T09:46:05.495 DEBUG:teuthology.orchestra.run.smithi084:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-16T09:46:05.498 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-16T09:46:05.588 INFO:teuthology.orchestra.run.smithi084.stdout:check_obsoletes = 1 2023-10-16T09:46:05.592 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum clean all 2023-10-16T09:46:05.594 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2023-10-16T09:46:05.598 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2023-10-16T09:46:05.950 INFO:teuthology.orchestra.run.smithi084.stdout:72 files removed 2023-10-16T09:46:05.951 INFO:teuthology.orchestra.run.smithi157.stdout:72 files removed 2023-10-16T09:46:05.988 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-16T09:46:05.993 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-16T09:46:06.908 INFO:teuthology.orchestra.run.smithi084.stdout:ceph packages for x86_64 192 kB/s | 78 kB 00:00 2023-10-16T09:46:07.035 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 145 kB/s | 78 kB 00:00 2023-10-16T09:46:07.425 INFO:teuthology.orchestra.run.smithi084.stdout:ceph noarch packages 22 kB/s | 9.5 kB 00:00 2023-10-16T09:46:07.535 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 23 kB/s | 9.5 kB 00:00 2023-10-16T09:46:07.934 INFO:teuthology.orchestra.run.smithi084.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2023-10-16T09:46:08.053 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2023-10-16T09:46:08.513 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS-9 - AppStream 12 MB/s | 6.3 MB 00:00 2023-10-16T09:46:08.706 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - AppStream 11 MB/s | 6.3 MB 00:00 2023-10-16T09:46:10.054 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS-9 - BaseOS 5.8 MB/s | 2.0 MB 00:00 2023-10-16T09:46:10.382 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - BaseOS 4.2 MB/s | 2.0 MB 00:00 2023-10-16T09:46:11.869 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - BaseOS 5.5 MB/s | 7.9 MB 00:01 2023-10-16T09:46:14.594 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 7.9 MB 00:03 2023-10-16T09:46:15.792 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - AppStream 6.4 MB/s | 18 MB 00:02 2023-10-16T09:46:20.378 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 3.8 MB/s | 18 MB 00:04 2023-10-16T09:46:21.076 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 5.7 MB 00:01 2023-10-16T09:46:23.546 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 14 kB 00:01 2023-10-16T09:46:24.887 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 16 MB/s | 19 MB 00:01 2023-10-16T09:46:25.220 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 5.7 MB 00:01 2023-10-16T09:46:26.738 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 14 kB 00:00 2023-10-16T09:46:28.714 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 11 MB/s | 19 MB 00:01 2023-10-16T09:46:30.120 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2023-10-16T09:46:32.645 INFO:teuthology.orchestra.run.smithi084.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-16T09:46:32.646 INFO:teuthology.orchestra.run.smithi084.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-16T09:46:32.667 INFO:teuthology.orchestra.run.smithi084.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-10-16T09:46:32.669 INFO:teuthology.orchestra.run.smithi084.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-10-16T09:46:32.670 INFO:teuthology.orchestra.run.smithi084.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-10-16T09:46:32.672 INFO:teuthology.orchestra.run.smithi084.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-10-16T09:46:32.674 INFO:teuthology.orchestra.run.smithi084.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-10-16T09:46:32.678 INFO:teuthology.orchestra.run.smithi084.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-10-16T09:46:32.679 INFO:teuthology.orchestra.run.smithi084.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-10-16T09:46:32.756 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2023-10-16T09:46:32.796 INFO:teuthology.orchestra.run.smithi084.stdout:============================================================================================ 2023-10-16T09:46:32.796 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2023-10-16T09:46:32.797 INFO:teuthology.orchestra.run.smithi084.stdout:============================================================================================ 2023-10-16T09:46:32.797 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2023-10-16T09:46:32.797 INFO:teuthology.orchestra.run.smithi084.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-10-16T09:46:32.797 INFO:teuthology.orchestra.run.smithi084.stdout: ceph x86_64 2:18.0.0-6675.gc7471068.el9 ceph 6.5 k 2023-10-16T09:46:32.797 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base x86_64 2:18.0.0-6675.gc7471068.el9 ceph 5.2 M 2023-10-16T09:46:32.797 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse x86_64 2:18.0.0-6675.gc7471068.el9 ceph 855 k 2023-10-16T09:46:32.798 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6675.gc7471068.el9 ceph 141 k 2023-10-16T09:46:32.798 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr x86_64 2:18.0.0-6675.gc7471068.el9 ceph 1.5 M 2023-10-16T09:46:32.798 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 132 k 2023-10-16T09:46:32.798 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 1.8 M 2023-10-16T09:46:32.799 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 7.4 M 2023-10-16T09:46:32.799 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 47 k 2023-10-16T09:46:32.799 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw x86_64 2:18.0.0-6675.gc7471068.el9 ceph 8.7 M 2023-10-16T09:46:32.799 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test x86_64 2:18.0.0-6675.gc7471068.el9 ceph 41 M 2023-10-16T09:46:32.799 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 262 k 2023-10-16T09:46:32.799 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 269 k 2023-10-16T09:46:32.800 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror x86_64 2:18.0.0-6675.gc7471068.el9 ceph 218 k 2023-10-16T09:46:32.800 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-top noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 18 k 2023-10-16T09:46:32.800 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-10-16T09:46:32.800 INFO:teuthology.orchestra.run.smithi084.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-10-16T09:46:32.800 INFO:teuthology.orchestra.run.smithi084.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-10-16T09:46:32.801 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel x86_64 2:18.0.0-6675.gc7471068.el9 ceph 31 k 2023-10-16T09:46:32.801 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2 x86_64 2:18.0.0-6675.gc7471068.el9 ceph 715 k 2023-10-16T09:46:32.801 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel x86_64 2:18.0.0-6675.gc7471068.el9 ceph 127 k 2023-10-16T09:46:32.801 INFO:teuthology.orchestra.run.smithi084.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-10-16T09:46:32.801 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-10-16T09:46:32.802 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-10-16T09:46:32.802 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-10-16T09:46:32.802 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-10-16T09:46:32.802 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-10-16T09:46:32.802 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs x86_64 2:18.0.0-6675.gc7471068.el9 ceph 162 k 2023-10-16T09:46:32.803 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados x86_64 2:18.0.0-6675.gc7471068.el9 ceph 320 k 2023-10-16T09:46:32.803 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd x86_64 2:18.0.0-6675.gc7471068.el9 ceph 297 k 2023-10-16T09:46:32.803 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw x86_64 2:18.0.0-6675.gc7471068.el9 ceph 99 k 2023-10-16T09:46:32.803 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse x86_64 2:18.0.0-6675.gc7471068.el9 ceph 85 k 2023-10-16T09:46:32.803 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror x86_64 2:18.0.0-6675.gc7471068.el9 ceph 3.0 M 2023-10-16T09:46:32.804 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd x86_64 2:18.0.0-6675.gc7471068.el9 ceph 168 k 2023-10-16T09:46:32.804 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-10-16T09:46:32.804 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrading: 2023-10-16T09:46:32.804 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager x86_64 1:1.45.4-1.el9 CentOS-BaseOS 2.2 M 2023-10-16T09:46:32.804 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-libnm x86_64 1:1.45.4-1.el9 CentOS-BaseOS 1.8 M 2023-10-16T09:46:32.805 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-team x86_64 1:1.45.4-1.el9 CentOS-BaseOS 41 k 2023-10-16T09:46:32.805 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-tui x86_64 1:1.45.4-1.el9 CentOS-BaseOS 244 k 2023-10-16T09:46:32.805 INFO:teuthology.orchestra.run.smithi084.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-10-16T09:46:32.805 INFO:teuthology.orchestra.run.smithi084.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-10-16T09:46:32.805 INFO:teuthology.orchestra.run.smithi084.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-10-16T09:46:32.805 INFO:teuthology.orchestra.run.smithi084.stdout: librados2 x86_64 2:18.0.0-6675.gc7471068.el9 ceph 3.3 M 2023-10-16T09:46:32.806 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1 x86_64 2:18.0.0-6675.gc7471068.el9 ceph 3.0 M 2023-10-16T09:46:32.806 INFO:teuthology.orchestra.run.smithi084.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-10-16T09:46:32.806 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2023-10-16T09:46:32.806 INFO:teuthology.orchestra.run.smithi084.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-10-16T09:46:32.806 INFO:teuthology.orchestra.run.smithi084.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-16T09:46:32.807 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common x86_64 2:18.0.0-6675.gc7471068.el9 ceph 19 M 2023-10-16T09:46:32.807 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 23 k 2023-10-16T09:46:32.807 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds x86_64 2:18.0.0-6675.gc7471068.el9 ceph 2.1 M 2023-10-16T09:46:32.807 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 243 k 2023-10-16T09:46:32.808 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon x86_64 2:18.0.0-6675.gc7471068.el9 ceph 4.4 M 2023-10-16T09:46:32.808 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd x86_64 2:18.0.0-6675.gc7471068.el9 ceph 18 M 2023-10-16T09:46:32.808 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 15 k 2023-10-16T09:46:32.808 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux x86_64 2:18.0.0-6675.gc7471068.el9 ceph 24 k 2023-10-16T09:46:32.808 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-16T09:46:32.808 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-16T09:46:32.809 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-16T09:46:32.809 INFO:teuthology.orchestra.run.smithi084.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-16T09:46:32.809 INFO:teuthology.orchestra.run.smithi084.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-16T09:46:32.809 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-16T09:46:32.810 INFO:teuthology.orchestra.run.smithi084.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-16T09:46:32.810 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-16T09:46:32.810 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-16T09:46:32.810 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite x86_64 2:18.0.0-6675.gc7471068.el9 ceph 163 k 2023-10-16T09:46:32.810 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-16T09:46:32.811 INFO:teuthology.orchestra.run.smithi084.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-16T09:46:32.811 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-16T09:46:32.811 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-16T09:46:32.811 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-16T09:46:32.811 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1 x86_64 2:18.0.0-6675.gc7471068.el9 ceph 473 k 2023-10-16T09:46:32.812 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-16T09:46:32.812 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2 x86_64 2:18.0.0-6675.gc7471068.el9 ceph 4.8 M 2023-10-16T09:46:32.812 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-16T09:46:32.812 INFO:teuthology.orchestra.run.smithi084.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-10-16T09:46:32.812 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-16T09:46:32.813 INFO:teuthology.orchestra.run.smithi084.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-16T09:46:32.813 INFO:teuthology.orchestra.run.smithi084.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-16T09:46:32.813 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-16T09:46:32.813 INFO:teuthology.orchestra.run.smithi084.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-16T09:46:32.813 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-10-16T09:46:32.814 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-16T09:46:32.814 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-16T09:46:32.814 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-16T09:46:32.815 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-16T09:46:32.815 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse x86_64 2:18.0.0-6675.gc7471068.el9 ceph 45 k 2023-10-16T09:46:32.815 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common x86_64 2:18.0.0-6675.gc7471068.el9 ceph 127 k 2023-10-16T09:46:32.816 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-16T09:46:32.816 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-16T09:46:32.816 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-16T09:46:32.817 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-16T09:46:32.817 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-16T09:46:32.817 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-16T09:46:32.817 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth noarch 1:2.23.2-1.el9 epel 196 k 2023-10-16T09:46:32.818 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-16T09:46:32.818 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-16T09:46:32.818 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-16T09:46:32.818 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-16T09:46:32.818 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-16T09:46:32.819 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-16T09:46:32.819 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-16T09:46:32.819 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-16T09:46:32.819 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-16T09:46:32.820 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-16T09:46:32.820 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-16T09:46:32.820 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-16T09:46:32.820 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-16T09:46:32.821 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-16T09:46:32.821 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-16T09:46:32.821 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-16T09:46:32.821 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-16T09:46:32.821 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-16T09:46:32.822 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-16T09:46:32.822 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-16T09:46:32.822 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-16T09:46:32.822 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-16T09:46:32.822 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-16T09:46:32.823 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-16T09:46:32.823 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-16T09:46:32.823 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-16T09:46:32.823 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-16T09:46:32.823 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-16T09:46:32.824 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-16T09:46:32.824 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-16T09:46:32.824 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-16T09:46:32.824 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-16T09:46:32.824 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-16T09:46:32.824 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-16T09:46:32.825 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-16T09:46:32.825 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-16T09:46:32.825 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3 noarch 1.26.5-3.el9 CentOS-BaseOS 188 k 2023-10-16T09:46:32.825 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-16T09:46:32.825 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-16T09:46:32.826 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-16T09:46:32.826 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-16T09:46:32.826 INFO:teuthology.orchestra.run.smithi084.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-16T09:46:32.826 INFO:teuthology.orchestra.run.smithi084.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-16T09:46:32.826 INFO:teuthology.orchestra.run.smithi084.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-16T09:46:32.827 INFO:teuthology.orchestra.run.smithi084.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-10-16T09:46:32.827 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-16T09:46:32.827 INFO:teuthology.orchestra.run.smithi084.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-10-16T09:46:32.827 INFO:teuthology.orchestra.run.smithi084.stdout:Installing weak dependencies: 2023-10-16T09:46:32.827 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-16T09:46:32.828 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:46:32.828 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2023-10-16T09:46:32.828 INFO:teuthology.orchestra.run.smithi084.stdout:============================================================================================ 2023-10-16T09:46:32.828 INFO:teuthology.orchestra.run.smithi084.stdout:Install 133 Packages 2023-10-16T09:46:32.828 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrade 10 Packages 2023-10-16T09:46:32.828 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:46:32.829 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 200 M 2023-10-16T09:46:32.829 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2023-10-16T09:46:33.254 INFO:teuthology.orchestra.run.smithi084.stdout:(1/143): ceph-18.0.0-6675.gc7471068.el9.x86_64. 24 kB/s | 6.5 kB 00:00 2023-10-16T09:46:33.629 INFO:teuthology.orchestra.run.smithi084.stdout:(2/143): ceph-fuse-18.0.0-6675.gc7471068.el9.x8 2.2 MB/s | 855 kB 00:00 2023-10-16T09:46:33.704 INFO:teuthology.orchestra.run.smithi084.stdout:(3/143): ceph-immutable-object-cache-18.0.0-667 1.9 MB/s | 141 kB 00:00 2023-10-16T09:46:33.897 INFO:teuthology.orchestra.run.smithi084.stdout:(4/143): ceph-base-18.0.0-6675.gc7471068.el9.x8 5.7 MB/s | 5.2 MB 00:00 2023-10-16T09:46:33.955 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2023-10-16T09:46:34.038 INFO:teuthology.orchestra.run.smithi084.stdout:(5/143): ceph-mgr-18.0.0-6675.gc7471068.el9.x86 10 MB/s | 1.5 MB 00:00 2023-10-16T09:46:34.264 INFO:teuthology.orchestra.run.smithi084.stdout:(6/143): ceph-mds-18.0.0-6675.gc7471068.el9.x86 3.7 MB/s | 2.1 MB 00:00 2023-10-16T09:46:34.389 INFO:teuthology.orchestra.run.smithi084.stdout:(7/143): ceph-mon-18.0.0-6675.gc7471068.el9.x86 13 MB/s | 4.4 MB 00:00 2023-10-16T09:46:35.053 INFO:teuthology.orchestra.run.smithi084.stdout:(8/143): ceph-radosgw-18.0.0-6675.gc7471068.el9 13 MB/s | 8.7 MB 00:00 2023-10-16T09:46:35.095 INFO:teuthology.orchestra.run.smithi084.stdout:(9/143): ceph-selinux-18.0.0-6675.gc7471068.el9 592 kB/s | 24 kB 00:00 2023-10-16T09:46:36.466 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-16T09:46:36.468 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-16T09:46:36.487 INFO:teuthology.orchestra.run.smithi157.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-10-16T09:46:36.488 INFO:teuthology.orchestra.run.smithi157.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-10-16T09:46:36.490 INFO:teuthology.orchestra.run.smithi157.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-10-16T09:46:36.491 INFO:teuthology.orchestra.run.smithi157.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-10-16T09:46:36.493 INFO:teuthology.orchestra.run.smithi157.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-10-16T09:46:36.497 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-10-16T09:46:36.498 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-10-16T09:46:36.577 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-10-16T09:46:36.592 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================================ 2023-10-16T09:46:36.593 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-10-16T09:46:36.593 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================================ 2023-10-16T09:46:36.593 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-10-16T09:46:36.593 INFO:teuthology.orchestra.run.smithi157.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-10-16T09:46:36.594 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:18.0.0-6675.gc7471068.el9 ceph 6.5 k 2023-10-16T09:46:36.594 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:18.0.0-6675.gc7471068.el9 ceph 5.2 M 2023-10-16T09:46:36.594 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:18.0.0-6675.gc7471068.el9 ceph 855 k 2023-10-16T09:46:36.594 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6675.gc7471068.el9 ceph 141 k 2023-10-16T09:46:36.595 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:18.0.0-6675.gc7471068.el9 ceph 1.5 M 2023-10-16T09:46:36.595 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 132 k 2023-10-16T09:46:36.595 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 1.8 M 2023-10-16T09:46:36.595 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 7.4 M 2023-10-16T09:46:36.595 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 47 k 2023-10-16T09:46:36.596 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:18.0.0-6675.gc7471068.el9 ceph 8.7 M 2023-10-16T09:46:36.596 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:18.0.0-6675.gc7471068.el9 ceph 41 M 2023-10-16T09:46:36.596 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 262 k 2023-10-16T09:46:36.596 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 269 k 2023-10-16T09:46:36.596 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror x86_64 2:18.0.0-6675.gc7471068.el9 ceph 218 k 2023-10-16T09:46:36.597 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 18 k 2023-10-16T09:46:36.597 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-10-16T09:46:36.597 INFO:teuthology.orchestra.run.smithi157.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-10-16T09:46:36.597 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-10-16T09:46:36.597 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:18.0.0-6675.gc7471068.el9 ceph 31 k 2023-10-16T09:46:36.597 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:18.0.0-6675.gc7471068.el9 ceph 715 k 2023-10-16T09:46:36.598 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:18.0.0-6675.gc7471068.el9 ceph 127 k 2023-10-16T09:46:36.598 INFO:teuthology.orchestra.run.smithi157.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-10-16T09:46:36.598 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-10-16T09:46:36.598 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-10-16T09:46:36.598 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-10-16T09:46:36.599 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-10-16T09:46:36.599 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-10-16T09:46:36.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:18.0.0-6675.gc7471068.el9 ceph 162 k 2023-10-16T09:46:36.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:18.0.0-6675.gc7471068.el9 ceph 320 k 2023-10-16T09:46:36.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:18.0.0-6675.gc7471068.el9 ceph 297 k 2023-10-16T09:46:36.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:18.0.0-6675.gc7471068.el9 ceph 99 k 2023-10-16T09:46:36.600 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:18.0.0-6675.gc7471068.el9 ceph 85 k 2023-10-16T09:46:36.600 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:18.0.0-6675.gc7471068.el9 ceph 3.0 M 2023-10-16T09:46:36.600 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:18.0.0-6675.gc7471068.el9 ceph 168 k 2023-10-16T09:46:36.600 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-10-16T09:46:36.600 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2023-10-16T09:46:36.601 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager x86_64 1:1.45.4-1.el9 CentOS-BaseOS 2.2 M 2023-10-16T09:46:36.601 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-libnm x86_64 1:1.45.4-1.el9 CentOS-BaseOS 1.8 M 2023-10-16T09:46:36.601 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-team x86_64 1:1.45.4-1.el9 CentOS-BaseOS 41 k 2023-10-16T09:46:36.601 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-tui x86_64 1:1.45.4-1.el9 CentOS-BaseOS 244 k 2023-10-16T09:46:36.601 INFO:teuthology.orchestra.run.smithi157.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-10-16T09:46:36.602 INFO:teuthology.orchestra.run.smithi157.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-10-16T09:46:36.602 INFO:teuthology.orchestra.run.smithi157.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-10-16T09:46:36.602 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:18.0.0-6675.gc7471068.el9 ceph 3.3 M 2023-10-16T09:46:36.602 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:18.0.0-6675.gc7471068.el9 ceph 3.0 M 2023-10-16T09:46:36.602 INFO:teuthology.orchestra.run.smithi157.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-10-16T09:46:36.603 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2023-10-16T09:46:36.603 INFO:teuthology.orchestra.run.smithi157.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-10-16T09:46:36.603 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-16T09:46:36.603 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:18.0.0-6675.gc7471068.el9 ceph 19 M 2023-10-16T09:46:36.603 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 23 k 2023-10-16T09:46:36.604 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:18.0.0-6675.gc7471068.el9 ceph 2.1 M 2023-10-16T09:46:36.604 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 243 k 2023-10-16T09:46:36.604 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:18.0.0-6675.gc7471068.el9 ceph 4.4 M 2023-10-16T09:46:36.604 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:18.0.0-6675.gc7471068.el9 ceph 18 M 2023-10-16T09:46:36.605 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6675.gc7471068.el9 ceph-noarch 15 k 2023-10-16T09:46:36.605 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:18.0.0-6675.gc7471068.el9 ceph 24 k 2023-10-16T09:46:36.605 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-16T09:46:36.605 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-16T09:46:36.606 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-16T09:46:36.606 INFO:teuthology.orchestra.run.smithi157.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-16T09:46:36.606 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-16T09:46:36.606 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-16T09:46:36.607 INFO:teuthology.orchestra.run.smithi157.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-16T09:46:36.607 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-16T09:46:36.607 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-16T09:46:36.607 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:18.0.0-6675.gc7471068.el9 ceph 163 k 2023-10-16T09:46:36.607 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-16T09:46:36.608 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-16T09:46:36.608 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-16T09:46:36.608 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-16T09:46:36.608 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-16T09:46:36.608 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:18.0.0-6675.gc7471068.el9 ceph 473 k 2023-10-16T09:46:36.609 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-16T09:46:36.609 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:18.0.0-6675.gc7471068.el9 ceph 4.8 M 2023-10-16T09:46:36.609 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-16T09:46:36.609 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-10-16T09:46:36.610 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-16T09:46:36.610 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-16T09:46:36.610 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-16T09:46:36.610 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-16T09:46:36.611 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-16T09:46:36.611 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-10-16T09:46:36.611 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-16T09:46:36.611 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-16T09:46:36.611 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-16T09:46:36.612 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-16T09:46:36.612 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:18.0.0-6675.gc7471068.el9 ceph 45 k 2023-10-16T09:46:36.612 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:18.0.0-6675.gc7471068.el9 ceph 127 k 2023-10-16T09:46:36.612 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-16T09:46:36.612 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-16T09:46:36.613 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-16T09:46:36.613 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-16T09:46:36.613 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-16T09:46:36.613 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-16T09:46:36.613 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:2.23.2-1.el9 epel 196 k 2023-10-16T09:46:36.613 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-16T09:46:36.614 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-16T09:46:36.614 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-16T09:46:36.614 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-16T09:46:36.615 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-16T09:46:36.615 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-16T09:46:36.615 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-16T09:46:36.615 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-16T09:46:36.616 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-16T09:46:36.616 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-16T09:46:36.616 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-16T09:46:36.616 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-16T09:46:36.616 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-16T09:46:36.617 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-16T09:46:36.617 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-16T09:46:36.617 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-16T09:46:36.618 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-16T09:46:36.618 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-16T09:46:36.618 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-16T09:46:36.618 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-16T09:46:36.618 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-16T09:46:36.619 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-16T09:46:36.619 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-16T09:46:36.619 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-16T09:46:36.619 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-16T09:46:36.619 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-16T09:46:36.620 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-16T09:46:36.620 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-16T09:46:36.620 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-16T09:46:36.620 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-16T09:46:36.620 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-16T09:46:36.621 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-16T09:46:36.621 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-16T09:46:36.621 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-16T09:46:36.621 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-16T09:46:36.621 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-16T09:46:36.621 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3 noarch 1.26.5-3.el9 CentOS-BaseOS 188 k 2023-10-16T09:46:36.622 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-16T09:46:36.622 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-16T09:46:36.622 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-16T09:46:36.622 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-16T09:46:36.623 INFO:teuthology.orchestra.run.smithi157.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-16T09:46:36.623 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-16T09:46:36.623 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-16T09:46:36.623 INFO:teuthology.orchestra.run.smithi157.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-10-16T09:46:36.623 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-16T09:46:36.624 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-10-16T09:46:36.624 INFO:teuthology.orchestra.run.smithi157.stdout:Installing weak dependencies: 2023-10-16T09:46:36.624 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-16T09:46:36.624 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:46:36.624 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-10-16T09:46:36.625 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================================ 2023-10-16T09:46:36.625 INFO:teuthology.orchestra.run.smithi157.stdout:Install 133 Packages 2023-10-16T09:46:36.625 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 10 Packages 2023-10-16T09:46:36.625 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:46:36.626 INFO:teuthology.orchestra.run.smithi084.stdout:(10/143): ceph-common-18.0.0-6675.gc7471068.el9 5.2 MB/s | 19 MB 00:03 2023-10-16T09:46:36.626 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 200 M 2023-10-16T09:46:36.627 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-10-16T09:46:36.670 INFO:teuthology.orchestra.run.smithi084.stdout:(11/143): cephfs-mirror-18.0.0-6675.gc7471068.e 3.2 MB/s | 218 kB 00:00 2023-10-16T09:46:36.938 INFO:teuthology.orchestra.run.smithi084.stdout:(12/143): ceph-osd-18.0.0-6675.gc7471068.el9.x8 6.6 MB/s | 18 MB 00:02 2023-10-16T09:46:36.963 INFO:teuthology.orchestra.run.smithi084.stdout:(13/143): libcephfs-devel-18.0.0-6675.gc7471068 107 kB/s | 31 kB 00:00 2023-10-16T09:46:37.038 INFO:teuthology.orchestra.run.smithi084.stdout:(14/143): libcephsqlite-18.0.0-6675.gc7471068.e 2.2 MB/s | 163 kB 00:00 2023-10-16T09:46:37.088 INFO:teuthology.orchestra.run.smithi084.stdout:(15/143): libcephfs2-18.0.0-6675.gc7471068.el9. 4.7 MB/s | 715 kB 00:00 2023-10-16T09:46:37.122 INFO:teuthology.orchestra.run.smithi084.stdout:(16/143): librados-devel-18.0.0-6675.gc7471068. 1.5 MB/s | 127 kB 00:00 2023-10-16T09:46:37.164 INFO:teuthology.orchestra.run.smithi084.stdout:(17/143): libradosstriper1-18.0.0-6675.gc747106 6.2 MB/s | 473 kB 00:00 2023-10-16T09:46:37.231 INFO:teuthology.orchestra.run.smithi084.stdout:(18/143): python3-ceph-argparse-18.0.0-6675.gc7 688 kB/s | 45 kB 00:00 2023-10-16T09:46:37.339 INFO:teuthology.orchestra.run.smithi084.stdout:(19/143): python3-ceph-common-18.0.0-6675.gc747 1.2 MB/s | 127 kB 00:00 2023-10-16T09:46:37.456 INFO:teuthology.orchestra.run.smithi084.stdout:(20/143): python3-cephfs-18.0.0-6675.gc7471068. 1.4 MB/s | 162 kB 00:00 2023-10-16T09:46:37.565 INFO:teuthology.orchestra.run.smithi084.stdout:(21/143): python3-rados-18.0.0-6675.gc7471068.e 2.9 MB/s | 320 kB 00:00 2023-10-16T09:46:37.673 INFO:teuthology.orchestra.run.smithi084.stdout:(22/143): python3-rbd-18.0.0-6675.gc7471068.el9 2.7 MB/s | 297 kB 00:00 2023-10-16T09:46:37.724 INFO:teuthology.orchestra.run.smithi157.stdout:(1/143): ceph-18.0.0-6675.gc7471068.el9.x86_64. 14 kB/s | 6.5 kB 00:00 2023-10-16T09:46:37.832 INFO:teuthology.orchestra.run.smithi084.stdout:(23/143): librgw2-18.0.0-6675.gc7471068.el9.x86 6.7 MB/s | 4.8 MB 00:00 2023-10-16T09:46:37.866 INFO:teuthology.orchestra.run.smithi084.stdout:(24/143): python3-rgw-18.0.0-6675.gc7471068.el9 520 kB/s | 99 kB 00:00 2023-10-16T09:46:37.899 INFO:teuthology.orchestra.run.smithi084.stdout:(25/143): rbd-fuse-18.0.0-6675.gc7471068.el9.x8 1.3 MB/s | 85 kB 00:00 2023-10-16T09:46:38.174 INFO:teuthology.orchestra.run.smithi157.stdout:(2/143): ceph-fuse-18.0.0-6675.gc7471068.el9.x8 1.9 MB/s | 855 kB 00:00 2023-10-16T09:46:38.240 INFO:teuthology.orchestra.run.smithi157.stdout:(3/143): ceph-immutable-object-cache-18.0.0-667 2.1 MB/s | 141 kB 00:00 2023-10-16T09:46:38.375 INFO:teuthology.orchestra.run.smithi084.stdout:(26/143): ceph-test-18.0.0-6675.gc7471068.el9.x 12 MB/s | 41 MB 00:03 2023-10-16T09:46:38.408 INFO:teuthology.orchestra.run.smithi084.stdout:(27/143): rbd-nbd-18.0.0-6675.gc7471068.el9.x86 332 kB/s | 168 kB 00:00 2023-10-16T09:46:38.433 INFO:teuthology.orchestra.run.smithi084.stdout:(28/143): ceph-grafana-dashboards-18.0.0-6675.g 404 kB/s | 23 kB 00:00 2023-10-16T09:46:38.475 INFO:teuthology.orchestra.run.smithi084.stdout:(29/143): ceph-mgr-cephadm-18.0.0-6675.gc747106 1.9 MB/s | 132 kB 00:00 2023-10-16T09:46:38.567 INFO:teuthology.orchestra.run.smithi084.stdout:(30/143): rbd-mirror-18.0.0-6675.gc7471068.el9. 4.3 MB/s | 3.0 MB 00:00 2023-10-16T09:46:38.709 INFO:teuthology.orchestra.run.smithi084.stdout:(31/143): ceph-mgr-dashboard-18.0.0-6675.gc7471 6.4 MB/s | 1.8 MB 00:00 2023-10-16T09:46:38.743 INFO:teuthology.orchestra.run.smithi084.stdout:(32/143): ceph-mgr-modules-core-18.0.0-6675.gc7 1.4 MB/s | 243 kB 00:00 2023-10-16T09:46:38.768 INFO:teuthology.orchestra.run.smithi084.stdout:(33/143): ceph-mgr-rook-18.0.0-6675.gc7471068.e 825 kB/s | 47 kB 00:00 2023-10-16T09:46:38.827 INFO:teuthology.orchestra.run.smithi084.stdout:(34/143): ceph-prometheus-alerts-18.0.0-6675.gc 177 kB/s | 15 kB 00:00 2023-10-16T09:46:38.860 INFO:teuthology.orchestra.run.smithi084.stdout:(35/143): ceph-volume-18.0.0-6675.gc7471068.el9 2.8 MB/s | 262 kB 00:00 2023-10-16T09:46:38.927 INFO:teuthology.orchestra.run.smithi084.stdout:(36/143): cephadm-18.0.0-6675.gc7471068.el9.noa 2.6 MB/s | 269 kB 00:00 2023-10-16T09:46:38.952 INFO:teuthology.orchestra.run.smithi084.stdout:(37/143): cephfs-top-18.0.0-6675.gc7471068.el9. 200 kB/s | 18 kB 00:00 2023-10-16T09:46:39.073 INFO:teuthology.orchestra.run.smithi157.stdout:(4/143): ceph-mds-18.0.0-6675.gc7471068.el9.x86 2.5 MB/s | 2.1 MB 00:00 2023-10-16T09:46:39.194 INFO:teuthology.orchestra.run.smithi084.stdout:(38/143): boost-program-options-1.75.0-8.el9.x8 440 kB/s | 106 kB 00:00 2023-10-16T09:46:39.206 INFO:teuthology.orchestra.run.smithi157.stdout:(5/143): ceph-base-18.0.0-6675.gc7471068.el9.x8 2.7 MB/s | 5.2 MB 00:01 2023-10-16T09:46:39.261 INFO:teuthology.orchestra.run.smithi084.stdout:(39/143): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 920 kB 00:00 2023-10-16T09:46:39.295 INFO:teuthology.orchestra.run.smithi084.stdout:(40/143): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 307 kB 00:00 2023-10-16T09:46:39.320 INFO:teuthology.orchestra.run.smithi084.stdout:(41/143): flexiblas-3.0.4-8.el9.x86_64.rpm 574 kB/s | 32 kB 00:00 2023-10-16T09:46:39.520 INFO:teuthology.orchestra.run.smithi084.stdout:(42/143): ceph-mgr-diskprediction-local-18.0.0- 7.1 MB/s | 7.4 MB 00:01 2023-10-16T09:46:39.596 INFO:teuthology.orchestra.run.smithi084.stdout:(43/143): flexiblas-netlib-3.0.4-8.el9.x86_64.r 10 MB/s | 3.0 MB 00:00 2023-10-16T09:46:39.621 INFO:teuthology.orchestra.run.smithi084.stdout:(44/143): flexiblas-openblas-openmp-3.0.4-8.el9 57 kB/s | 17 kB 00:00 2023-10-16T09:46:39.646 INFO:teuthology.orchestra.run.smithi084.stdout:(45/143): libaio-devel-0.3.111-13.el9.x86_64.rp 82 kB/s | 10 kB 00:00 2023-10-16T09:46:39.656 INFO:teuthology.orchestra.run.smithi157.stdout:(6/143): ceph-common-18.0.0-6675.gc7471068.el9. 7.9 MB/s | 19 MB 00:02 2023-10-16T09:46:39.680 INFO:teuthology.orchestra.run.smithi084.stdout:(46/143): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2023-10-16T09:46:39.730 INFO:teuthology.orchestra.run.smithi084.stdout:(47/143): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 661 kB 00:00 2023-10-16T09:46:39.757 INFO:teuthology.orchestra.run.smithi157.stdout:(7/143): ceph-mgr-18.0.0-6675.gc7471068.el9.x86 2.1 MB/s | 1.5 MB 00:00 2023-10-16T09:46:39.764 INFO:teuthology.orchestra.run.smithi084.stdout:(48/143): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 578 kB 00:00 2023-10-16T09:46:39.789 INFO:teuthology.orchestra.run.smithi084.stdout:(49/143): libzstd-devel-1.5.1-2.el9.x86_64.rpm 455 kB/s | 46 kB 00:00 2023-10-16T09:46:39.823 INFO:teuthology.orchestra.run.smithi084.stdout:(50/143): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2023-10-16T09:46:39.848 INFO:teuthology.orchestra.run.smithi084.stdout:(51/143): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.6 MB/s | 217 kB 00:00 2023-10-16T09:46:39.873 INFO:teuthology.orchestra.run.smithi084.stdout:(52/143): openblas-0.3.21-2.el9.x86_64.rpm 413 kB/s | 34 kB 00:00 2023-10-16T09:46:39.932 INFO:teuthology.orchestra.run.smithi157.stdout:(8/143): ceph-mon-18.0.0-6675.gc7471068.el9.x86 6.0 MB/s | 4.4 MB 00:00 2023-10-16T09:46:39.948 INFO:teuthology.orchestra.run.smithi084.stdout:(53/143): postgresql-13.11-1.el9.x86_64.rpm 15 MB/s | 1.5 MB 00:00 2023-10-16T09:46:40.007 INFO:teuthology.orchestra.run.smithi157.stdout:(9/143): ceph-selinux-18.0.0-6675.gc7471068.el9 326 kB/s | 24 kB 00:00 2023-10-16T09:46:40.074 INFO:teuthology.orchestra.run.smithi084.stdout:(54/143): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2023-10-16T09:46:40.107 INFO:teuthology.orchestra.run.smithi084.stdout:(55/143): postgresql-contrib-13.11-1.el9.x86_64 3.5 MB/s | 811 kB 00:00 2023-10-16T09:46:40.132 INFO:teuthology.orchestra.run.smithi084.stdout:(56/143): postgresql-private-libs-13.11-1.el9.x 793 kB/s | 134 kB 00:00 2023-10-16T09:46:40.199 INFO:teuthology.orchestra.run.smithi084.stdout:(57/143): python3-devel-3.9.17-2.el9.x86_64.rpm 3.1 MB/s | 205 kB 00:00 2023-10-16T09:46:40.491 INFO:teuthology.orchestra.run.smithi084.stdout:(58/143): postgresql-server-13.11-1.el9.x86_64. 16 MB/s | 5.7 MB 00:00 2023-10-16T09:46:40.599 INFO:teuthology.orchestra.run.smithi084.stdout:(59/143): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 5.8 MB 00:00 2023-10-16T09:46:40.633 INFO:teuthology.orchestra.run.smithi084.stdout:(60/143): python3-jinja2-2.11.3-4.el9.noarch.rp 534 kB/s | 228 kB 00:00 2023-10-16T09:46:40.658 INFO:teuthology.orchestra.run.smithi084.stdout:(61/143): python3-jsonpatch-1.21-16.el9.noarch. 425 kB/s | 24 kB 00:00 2023-10-16T09:46:40.683 INFO:teuthology.orchestra.run.smithi084.stdout:(62/143): python3-jsonpointer-2.0-4.el9.noarch. 219 kB/s | 18 kB 00:00 2023-10-16T09:46:40.708 INFO:teuthology.orchestra.run.smithi084.stdout:(63/143): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2023-10-16T09:46:40.745 INFO:teuthology.orchestra.run.smithi084.stdout:(64/143): python3-markupsafe-1.1.1-12.el9.x86_6 531 kB/s | 32 kB 00:00 2023-10-16T09:46:40.879 INFO:teuthology.orchestra.run.smithi084.stdout:(65/143): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2023-10-16T09:46:40.913 INFO:teuthology.orchestra.run.smithi084.stdout:(66/143): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 217 kB 00:00 2023-10-16T09:46:40.938 INFO:teuthology.orchestra.run.smithi084.stdout:(67/143): python3-oauthlib-3.1.1-5.el9.noarch.r 975 kB/s | 174 kB 00:00 2023-10-16T09:46:40.963 INFO:teuthology.orchestra.run.smithi084.stdout:(68/143): python3-prettytable-0.7.2-27.el9.noar 517 kB/s | 41 kB 00:00 2023-10-16T09:46:40.996 INFO:teuthology.orchestra.run.smithi084.stdout:(69/143): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2023-10-16T09:46:41.050 INFO:teuthology.orchestra.run.smithi157.stdout:(10/143): ceph-osd-18.0.0-6675.gc7471068.el9.x8 13 MB/s | 18 MB 00:01 2023-10-16T09:46:41.108 INFO:teuthology.orchestra.run.smithi157.stdout:(11/143): cephfs-mirror-18.0.0-6675.gc7471068.e 3.7 MB/s | 218 kB 00:00 2023-10-16T09:46:41.155 INFO:teuthology.orchestra.run.smithi084.stdout:(70/143): python3-pyasn1-modules-0.4.8-6.el9.no 976 kB/s | 211 kB 00:00 2023-10-16T09:46:41.166 INFO:teuthology.orchestra.run.smithi157.stdout:(12/143): libcephfs-devel-18.0.0-6675.gc7471068 541 kB/s | 31 kB 00:00 2023-10-16T09:46:41.180 INFO:teuthology.orchestra.run.smithi084.stdout:(71/143): python3-requests-oauthlib-1.3.0-12.el 198 kB/s | 43 kB 00:00 2023-10-16T09:46:41.205 INFO:teuthology.orchestra.run.smithi084.stdout:(72/143): python3-toml-0.10.2-6.el9.noarch.rpm 795 kB/s | 39 kB 00:00 2023-10-16T09:46:41.258 INFO:teuthology.orchestra.run.smithi157.stdout:(13/143): libcephfs2-18.0.0-6675.gc7471068.el9. 7.7 MB/s | 715 kB 00:00 2023-10-16T09:46:41.325 INFO:teuthology.orchestra.run.smithi157.stdout:(14/143): libcephsqlite-18.0.0-6675.gc7471068.e 2.4 MB/s | 163 kB 00:00 2023-10-16T09:46:41.376 INFO:teuthology.orchestra.run.smithi157.stdout:(15/143): librados-devel-18.0.0-6675.gc7471068. 2.5 MB/s | 127 kB 00:00 2023-10-16T09:46:41.418 INFO:teuthology.orchestra.run.smithi084.stdout:(73/143): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 300 kB 00:00 2023-10-16T09:46:41.442 INFO:teuthology.orchestra.run.smithi157.stdout:(16/143): libradosstriper1-18.0.0-6675.gc747106 7.0 MB/s | 473 kB 00:00 2023-10-16T09:46:41.531 INFO:teuthology.orchestra.run.smithi084.stdout:(74/143): uuid-1.6.2-55.el9.x86_64.rpm 182 kB/s | 56 kB 00:00 2023-10-16T09:46:41.748 INFO:teuthology.orchestra.run.smithi084.stdout:(75/143): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2023-10-16T09:46:41.773 INFO:teuthology.orchestra.run.smithi084.stdout:(76/143): xmlstarlet-1.6.1-20.el9.x86_64.rpm 184 kB/s | 63 kB 00:00 2023-10-16T09:46:41.798 INFO:teuthology.orchestra.run.smithi084.stdout:(77/143): fuse-2.9.9-15.el9.x86_64.rpm 307 kB/s | 79 kB 00:00 2023-10-16T09:46:41.826 INFO:teuthology.orchestra.run.smithi157.stdout:(17/143): librgw2-18.0.0-6675.gc7471068.el9.x86 12 MB/s | 4.8 MB 00:00 2023-10-16T09:46:41.840 INFO:teuthology.orchestra.run.smithi084.stdout:(78/143): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2023-10-16T09:46:41.868 INFO:teuthology.orchestra.run.smithi157.stdout:(18/143): python3-ceph-argparse-18.0.0-6675.gc7 1.1 MB/s | 45 kB 00:00 2023-10-16T09:46:41.882 INFO:teuthology.orchestra.run.smithi084.stdout:(79/143): libgfortran-11.4.1-2.3.el9.x86_64.rpm 7.3 MB/s | 803 kB 00:00 2023-10-16T09:46:41.907 INFO:teuthology.orchestra.run.smithi084.stdout:(80/143): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.2 MB/s | 241 kB 00:00 2023-10-16T09:46:41.918 INFO:teuthology.orchestra.run.smithi157.stdout:(19/143): python3-ceph-common-18.0.0-6675.gc747 2.5 MB/s | 127 kB 00:00 2023-10-16T09:46:41.933 INFO:teuthology.orchestra.run.smithi084.stdout:(81/143): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2023-10-16T09:46:41.966 INFO:teuthology.orchestra.run.smithi084.stdout:(82/143): python3-requests-2.25.1-7.el9.noarch. 1.9 MB/s | 113 kB 00:00 2023-10-16T09:46:41.968 INFO:teuthology.orchestra.run.smithi157.stdout:(20/143): python3-cephfs-18.0.0-6675.gc7471068. 3.2 MB/s | 162 kB 00:00 2023-10-16T09:46:41.991 INFO:teuthology.orchestra.run.smithi084.stdout:(83/143): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2023-10-16T09:46:42.025 INFO:teuthology.orchestra.run.smithi084.stdout:(84/143): python3-urllib3-1.26.5-3.el9.noarch.r 2.0 MB/s | 188 kB 00:00 2023-10-16T09:46:42.035 INFO:teuthology.orchestra.run.smithi157.stdout:(21/143): python3-rados-18.0.0-6675.gc7471068.e 4.8 MB/s | 320 kB 00:00 2023-10-16T09:46:42.103 INFO:teuthology.orchestra.run.smithi157.stdout:(22/143): python3-rbd-18.0.0-6675.gc7471068.el9 4.4 MB/s | 297 kB 00:00 2023-10-16T09:46:42.142 INFO:teuthology.orchestra.run.smithi084.stdout:(85/143): elfutils-libelf-devel-0.189-3.el9.x86 194 kB/s | 22 kB 00:00 2023-10-16T09:46:42.169 INFO:teuthology.orchestra.run.smithi157.stdout:(23/143): python3-rgw-18.0.0-6675.gc7471068.el9 1.5 MB/s | 99 kB 00:00 2023-10-16T09:46:42.234 INFO:teuthology.orchestra.run.smithi084.stdout:(86/143): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2023-10-16T09:46:42.236 INFO:teuthology.orchestra.run.smithi157.stdout:(24/143): rbd-fuse-18.0.0-6675.gc7471068.el9.x8 1.3 MB/s | 85 kB 00:00 2023-10-16T09:46:42.293 INFO:teuthology.orchestra.run.smithi084.stdout:(87/143): jq-1.6-15.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2023-10-16T09:46:42.359 INFO:teuthology.orchestra.run.smithi084.stdout:(88/143): librabbitmq-0.11.0-7.el9.x86_64.rpm 364 kB/s | 45 kB 00:00 2023-10-16T09:46:42.418 INFO:teuthology.orchestra.run.smithi084.stdout:(89/143): libuuid-devel-2.37.4-15.el9.x86_64.rp 205 kB/s | 26 kB 00:00 2023-10-16T09:46:42.527 INFO:teuthology.orchestra.run.smithi084.stdout:(90/143): python3-pytz-2021.1-5.el9.noarch.rpm 475 kB/s | 51 kB 00:00 2023-10-16T09:46:42.537 INFO:teuthology.orchestra.run.smithi157.stdout:(25/143): rbd-mirror-18.0.0-6675.gc7471068.el9. 10 MB/s | 3.0 MB 00:00 2023-10-16T09:46:42.602 INFO:teuthology.orchestra.run.smithi084.stdout:(91/143): python3-cryptography-36.0.1-4.el9.x86 1.9 MB/s | 1.2 MB 00:00 2023-10-16T09:46:42.660 INFO:teuthology.orchestra.run.smithi084.stdout:(92/143): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 650 kB/s | 86 kB 00:00 2023-10-16T09:46:42.695 INFO:teuthology.orchestra.run.smithi157.stdout:(26/143): ceph-radosgw-18.0.0-6675.gc7471068.el 2.9 MB/s | 8.7 MB 00:02 2023-10-16T09:46:42.702 INFO:teuthology.orchestra.run.smithi084.stdout:(93/143): zlib-devel-1.2.11-40.el9.x86_64.rpm 454 kB/s | 45 kB 00:00 2023-10-16T09:46:42.729 INFO:teuthology.orchestra.run.smithi157.stdout:(27/143): rbd-nbd-18.0.0-6675.gc7471068.el9.x86 882 kB/s | 168 kB 00:00 2023-10-16T09:46:42.771 INFO:teuthology.orchestra.run.smithi157.stdout:(28/143): ceph-grafana-dashboards-18.0.0-6675.g 312 kB/s | 23 kB 00:00 2023-10-16T09:46:42.796 INFO:teuthology.orchestra.run.smithi157.stdout:(29/143): ceph-mgr-cephadm-18.0.0-6675.gc747106 1.9 MB/s | 132 kB 00:00 2023-10-16T09:46:42.986 INFO:teuthology.orchestra.run.smithi084.stdout:(94/143): openssl-devel-3.0.7-24.el9.x86_64.rpm 6.6 MB/s | 4.1 MB 00:00 2023-10-16T09:46:43.136 INFO:teuthology.orchestra.run.smithi084.stdout:(95/143): fmt-8.1.1-5.el9.x86_64.rpm 233 kB/s | 111 kB 00:00 2023-10-16T09:46:43.228 INFO:teuthology.orchestra.run.smithi084.stdout:(96/143): libarrow-doc-9.0.0-9.el9.noarch.rpm 267 kB/s | 24 kB 00:00 2023-10-16T09:46:43.303 INFO:teuthology.orchestra.run.smithi084.stdout:(97/143): gperftools-libs-2.9.1-2.el9.x86_64.rp 515 kB/s | 309 kB 00:00 2023-10-16T09:46:43.328 INFO:teuthology.orchestra.run.smithi084.stdout:(98/143): liboath-2.6.7-2.el9.x86_64.rpm 488 kB/s | 49 kB 00:00 2023-10-16T09:46:43.371 INFO:teuthology.orchestra.run.smithi157.stdout:(30/143): ceph-mgr-dashboard-18.0.0-6675.gc7471 2.9 MB/s | 1.8 MB 00:00 2023-10-16T09:46:43.395 INFO:teuthology.orchestra.run.smithi084.stdout:(99/143): libunwind-1.6.2-1.el9.x86_64.rpm 738 kB/s | 67 kB 00:00 2023-10-16T09:46:43.505 INFO:teuthology.orchestra.run.smithi157.stdout:(31/143): ceph-mgr-diskprediction-local-18.0.0- 10 MB/s | 7.4 MB 00:00 2023-10-16T09:46:43.539 INFO:teuthology.orchestra.run.smithi157.stdout:(32/143): ceph-mgr-modules-core-18.0.0-6675.gc7 1.4 MB/s | 243 kB 00:00 2023-10-16T09:46:43.564 INFO:teuthology.orchestra.run.smithi157.stdout:(33/143): ceph-mgr-rook-18.0.0-6675.gc7471068.e 827 kB/s | 47 kB 00:00 2023-10-16T09:46:43.587 INFO:teuthology.orchestra.run.smithi084.stdout:(100/143): python3-asyncssh-2.13.2-3.el9.noarch 2.8 MB/s | 548 kB 00:00 2023-10-16T09:46:43.614 INFO:teuthology.orchestra.run.smithi157.stdout:(34/143): ceph-prometheus-alerts-18.0.0-6675.gc 197 kB/s | 15 kB 00:00 2023-10-16T09:46:43.665 INFO:teuthology.orchestra.run.smithi157.stdout:(35/143): ceph-volume-18.0.0-6675.gc7471068.el9 2.6 MB/s | 262 kB 00:00 2023-10-16T09:46:43.671 INFO:teuthology.orchestra.run.smithi084.stdout:(101/143): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.4 MB/s | 837 kB 00:00 2023-10-16T09:46:43.696 INFO:teuthology.orchestra.run.smithi084.stdout:(102/143): python3-bcrypt-3.2.2-1.el9.x86_64.rp 402 kB/s | 43 kB 00:00 2023-10-16T09:46:43.715 INFO:teuthology.orchestra.run.smithi157.stdout:(36/143): cephfs-top-18.0.0-6675.gc7471068.el9. 370 kB/s | 18 kB 00:00 2023-10-16T09:46:43.746 INFO:teuthology.orchestra.run.smithi084.stdout:(103/143): python3-cachetools-4.2.4-1.el9.noarc 433 kB/s | 32 kB 00:00 2023-10-16T09:46:43.748 INFO:teuthology.orchestra.run.smithi157.stdout:(37/143): cephadm-18.0.0-6675.gc7471068.el9.noa 2.0 MB/s | 269 kB 00:00 2023-10-16T09:46:43.780 INFO:teuthology.orchestra.run.smithi084.stdout:(104/143): python3-certifi-2021.10.8-2.el9.noar 181 kB/s | 15 kB 00:00 2023-10-16T09:46:43.905 INFO:teuthology.orchestra.run.smithi084.stdout:(105/143): libarrow-9.0.0-9.el9.x86_64.rpm 4.8 MB/s | 4.4 MB 00:00 2023-10-16T09:46:43.939 INFO:teuthology.orchestra.run.smithi084.stdout:(106/143): python3-cheroot-8.6.0-4.el9.noarch.r 898 kB/s | 172 kB 00:00 2023-10-16T09:46:43.966 INFO:teuthology.orchestra.run.smithi157.stdout:(38/143): boost-program-options-1.75.0-8.el9.x8 490 kB/s | 106 kB 00:00 2023-10-16T09:46:43.972 INFO:teuthology.orchestra.run.smithi084.stdout:(107/143): python3-cherrypy-18.6.1-2.el9.noarch 1.8 MB/s | 358 kB 00:00 2023-10-16T09:46:44.014 INFO:teuthology.orchestra.run.smithi084.stdout:(108/143): python3-google-auth-2.23.2-1.el9.noa 1.8 MB/s | 196 kB 00:00 2023-10-16T09:46:44.032 INFO:teuthology.orchestra.run.smithi084.stdout:(109/143): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2023-10-16T09:46:44.049 INFO:teuthology.orchestra.run.smithi157.stdout:(39/143): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 920 kB 00:00 2023-10-16T09:46:44.057 INFO:teuthology.orchestra.run.smithi084.stdout:(110/143): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2023-10-16T09:46:44.083 INFO:teuthology.orchestra.run.smithi157.stdout:(40/143): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 307 kB 00:00 2023-10-16T09:46:44.090 INFO:teuthology.orchestra.run.smithi084.stdout:(111/143): python3-jaraco-collections-3.0.0-8.e 309 kB/s | 23 kB 00:00 2023-10-16T09:46:44.107 INFO:teuthology.orchestra.run.smithi084.stdout:(112/143): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2023-10-16T09:46:44.108 INFO:teuthology.orchestra.run.smithi157.stdout:(41/143): flexiblas-3.0.4-8.el9.x86_64.rpm 572 kB/s | 32 kB 00:00 2023-10-16T09:46:44.133 INFO:teuthology.orchestra.run.smithi084.stdout:(113/143): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2023-10-16T09:46:44.166 INFO:teuthology.orchestra.run.smithi084.stdout:(114/143): python3-jwt+crypto-2.4.0-1.el9.noarc 121 kB/s | 9.0 kB 00:00 2023-10-16T09:46:44.191 INFO:teuthology.orchestra.run.smithi084.stdout:(115/143): python3-jwt-2.4.0-1.el9.noarch.rpm 491 kB/s | 41 kB 00:00 2023-10-16T09:46:44.217 INFO:teuthology.orchestra.run.smithi157.stdout:(42/143): flexiblas-netlib-3.0.4-8.el9.x86_64.r 23 MB/s | 3.0 MB 00:00 2023-10-16T09:46:44.241 INFO:teuthology.orchestra.run.smithi157.stdout:(43/143): flexiblas-openblas-openmp-3.0.4-8.el9 135 kB/s | 17 kB 00:00 2023-10-16T09:46:44.250 INFO:teuthology.orchestra.run.smithi084.stdout:(116/143): python3-logutils-0.3.5-21.el9.noarch 559 kB/s | 46 kB 00:00 2023-10-16T09:46:44.275 INFO:teuthology.orchestra.run.smithi084.stdout:(117/143): python3-more-itertools-8.12.0-2.el9. 950 kB/s | 79 kB 00:00 2023-10-16T09:46:44.325 INFO:teuthology.orchestra.run.smithi084.stdout:(118/143): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2023-10-16T09:46:44.350 INFO:teuthology.orchestra.run.smithi084.stdout:(119/143): python3-natsort-7.1.1-5.el9.noarch.r 580 kB/s | 58 kB 00:00 2023-10-16T09:46:44.384 INFO:teuthology.orchestra.run.smithi084.stdout:(120/143): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2023-10-16T09:46:44.409 INFO:teuthology.orchestra.run.smithi084.stdout:(121/143): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2023-10-16T09:46:44.426 INFO:teuthology.orchestra.run.smithi084.stdout:(122/143): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2023-10-16T09:46:44.460 INFO:teuthology.orchestra.run.smithi084.stdout:(123/143): python3-repoze-lru-0.7-16.el9.noarch 411 kB/s | 31 kB 00:00 2023-10-16T09:46:44.502 INFO:teuthology.orchestra.run.smithi084.stdout:(124/143): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2023-10-16T09:46:44.527 INFO:teuthology.orchestra.run.smithi084.stdout:(125/143): python3-rsa-4.9-2.el9.noarch.rpm 589 kB/s | 59 kB 00:00 2023-10-16T09:46:44.552 INFO:teuthology.orchestra.run.smithi084.stdout:(126/143): python3-tempora-5.0.0-2.el9.noarch.r 391 kB/s | 36 kB 00:00 2023-10-16T09:46:44.586 INFO:teuthology.orchestra.run.smithi084.stdout:(127/143): python3-typing-extensions-4.4.0-2.el 615 kB/s | 51 kB 00:00 2023-10-16T09:46:44.628 INFO:teuthology.orchestra.run.smithi084.stdout:(128/143): python3-webob-1.8.7-6.el9.noarch.rpm 2.3 MB/s | 230 kB 00:00 2023-10-16T09:46:44.645 INFO:teuthology.orchestra.run.smithi084.stdout:(129/143): python3-websocket-client-1.2.3-2.el9 980 kB/s | 90 kB 00:00 2023-10-16T09:46:44.687 INFO:teuthology.orchestra.run.smithi084.stdout:(130/143): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2023-10-16T09:46:44.700 INFO:teuthology.orchestra.run.smithi157.stdout:(44/143): ceph-test-18.0.0-6675.gc7471068.el9.x 8.7 MB/s | 41 MB 00:04 2023-10-16T09:46:44.712 INFO:teuthology.orchestra.run.smithi084.stdout:(131/143): python3-zc-lockfile-2.0-10.el9.noarc 240 kB/s | 20 kB 00:00 2023-10-16T09:46:44.725 INFO:teuthology.orchestra.run.smithi157.stdout:(45/143): libaio-devel-0.3.111-13.el9.x86_64.rp 21 kB/s | 10 kB 00:00 2023-10-16T09:46:44.745 INFO:teuthology.orchestra.run.smithi084.stdout:(132/143): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2023-10-16T09:46:44.751 INFO:teuthology.orchestra.run.smithi157.stdout:(46/143): libpmemobj-1.12.1-1.el9.x86_64.rpm 313 kB/s | 159 kB 00:00 2023-10-16T09:46:44.793 INFO:teuthology.orchestra.run.smithi157.stdout:(47/143): librdkafka-1.6.1-102.el9.x86_64.rpm 7.2 MB/s | 661 kB 00:00 2023-10-16T09:46:44.818 INFO:teuthology.orchestra.run.smithi157.stdout:(48/143): libzstd-devel-1.5.1-2.el9.x86_64.rpm 772 kB/s | 46 kB 00:00 2023-10-16T09:46:44.846 INFO:teuthology.orchestra.run.smithi084.stdout:(133/143): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2023-10-16T09:46:44.860 INFO:teuthology.orchestra.run.smithi157.stdout:(49/143): libtool-2.4.6-45.el9.x86_64.rpm 5.5 MB/s | 578 kB 00:00 2023-10-16T09:46:44.879 INFO:teuthology.orchestra.run.smithi084.stdout:(134/143): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2023-10-16T09:46:44.885 INFO:teuthology.orchestra.run.smithi157.stdout:(50/143): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.0 MB/s | 282 kB 00:00 2023-10-16T09:46:44.918 INFO:teuthology.orchestra.run.smithi157.stdout:(51/143): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.3 MB/s | 217 kB 00:00 2023-10-16T09:46:44.961 INFO:teuthology.orchestra.run.smithi157.stdout:(52/143): openblas-0.3.21-2.el9.x86_64.rpm 472 kB/s | 34 kB 00:00 2023-10-16T09:46:45.021 INFO:teuthology.orchestra.run.smithi084.stdout:(135/143): NetworkManager-1.45.4-1.el9.x86_64.r 16 MB/s | 2.2 MB 00:00 2023-10-16T09:46:45.111 INFO:teuthology.orchestra.run.smithi157.stdout:(53/143): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2023-10-16T09:46:45.121 INFO:teuthology.orchestra.run.smithi084.stdout:(136/143): librbd1-18.0.0-6675.gc7471068.el9.x8 8.1 MB/s | 3.0 MB 00:00 2023-10-16T09:46:45.161 INFO:teuthology.orchestra.run.smithi157.stdout:(54/143): postgresql-13.11-1.el9.x86_64.rpm 6.3 MB/s | 1.5 MB 00:00 2023-10-16T09:46:45.188 INFO:teuthology.orchestra.run.smithi084.stdout:(137/143): NetworkManager-libnm-1.45.4-1.el9.x8 11 MB/s | 1.8 MB 00:00 2023-10-16T09:46:45.194 INFO:teuthology.orchestra.run.smithi157.stdout:(55/143): postgresql-contrib-13.11-1.el9.x86_64 3.6 MB/s | 811 kB 00:00 2023-10-16T09:46:45.214 INFO:teuthology.orchestra.run.smithi084.stdout:(138/143): NetworkManager-team-1.45.4-1.el9.x86 536 kB/s | 41 kB 00:00 2023-10-16T09:46:45.220 INFO:teuthology.orchestra.run.smithi157.stdout:(56/143): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 134 kB 00:00 2023-10-16T09:46:45.256 INFO:teuthology.orchestra.run.smithi084.stdout:(139/143): NetworkManager-tui-1.45.4-1.el9.x86_ 3.6 MB/s | 244 kB 00:00 2023-10-16T09:46:45.289 INFO:teuthology.orchestra.run.smithi084.stdout:(140/143): acl-2.3.1-4.el9.x86_64.rpm 954 kB/s | 69 kB 00:00 2023-10-16T09:46:45.323 INFO:teuthology.orchestra.run.smithi084.stdout:(141/143): libacl-2.3.1-4.el9.x86_64.rpm 335 kB/s | 22 kB 00:00 2023-10-16T09:46:45.373 INFO:teuthology.orchestra.run.smithi084.stdout:(142/143): xfsdump-3.1.12-4.el9.x86_64.rpm 3.8 MB/s | 321 kB 00:00 2023-10-16T09:46:45.465 INFO:teuthology.orchestra.run.smithi084.stdout:(143/143): librados2-18.0.0-6675.gc7471068.el9. 4.4 MB/s | 3.3 MB 00:00 2023-10-16T09:46:45.470 INFO:teuthology.orchestra.run.smithi157.stdout:(57/143): python3-devel-3.9.17-2.el9.x86_64.rpm 824 kB/s | 205 kB 00:00 2023-10-16T09:46:45.479 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2023-10-16T09:46:45.479 INFO:teuthology.orchestra.run.smithi084.stdout:Total 16 MB/s | 200 MB 00:12 2023-10-16T09:46:45.581 INFO:teuthology.orchestra.run.smithi157.stdout:(58/143): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 5.8 MB 00:00 2023-10-16T09:46:45.810 INFO:teuthology.orchestra.run.smithi157.stdout:(59/143): postgresql-server-13.11-1.el9.x86_64. 9.3 MB/s | 5.7 MB 00:00 2023-10-16T09:46:45.840 INFO:teuthology.orchestra.run.smithi157.stdout:(60/143): python3-jinja2-2.11.3-4.el9.noarch.rp 621 kB/s | 228 kB 00:00 2023-10-16T09:46:45.865 INFO:teuthology.orchestra.run.smithi157.stdout:(61/143): python3-jsonpatch-1.21-16.el9.noarch. 88 kB/s | 24 kB 00:00 2023-10-16T09:46:45.890 INFO:teuthology.orchestra.run.smithi157.stdout:(62/143): python3-jsonpointer-2.0-4.el9.noarch. 370 kB/s | 18 kB 00:00 2023-10-16T09:46:45.915 INFO:teuthology.orchestra.run.smithi157.stdout:(63/143): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2023-10-16T09:46:45.924 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2023-10-16T09:46:45.941 INFO:teuthology.orchestra.run.smithi157.stdout:(64/143): python3-markupsafe-1.1.1-12.el9.x86_6 424 kB/s | 32 kB 00:00 2023-10-16T09:46:45.974 INFO:teuthology.orchestra.run.smithi157.stdout:(65/143): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 217 kB 00:00 2023-10-16T09:46:46.127 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2023-10-16T09:46:46.127 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2023-10-16T09:46:46.133 INFO:teuthology.orchestra.run.smithi157.stdout:(66/143): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 4.8 MB 00:00 2023-10-16T09:46:46.166 INFO:teuthology.orchestra.run.smithi157.stdout:(67/143): python3-oauthlib-3.1.1-5.el9.noarch.r 777 kB/s | 174 kB 00:00 2023-10-16T09:46:46.192 INFO:teuthology.orchestra.run.smithi157.stdout:(68/143): python3-prettytable-0.7.2-27.el9.noar 203 kB/s | 41 kB 00:00 2023-10-16T09:46:46.217 INFO:teuthology.orchestra.run.smithi157.stdout:(69/143): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2023-10-16T09:46:46.242 INFO:teuthology.orchestra.run.smithi157.stdout:(70/143): python3-requests-oauthlib-1.3.0-12.el 892 kB/s | 43 kB 00:00 2023-10-16T09:46:46.276 INFO:teuthology.orchestra.run.smithi157.stdout:(71/143): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 211 kB 00:00 2023-10-16T09:46:46.568 INFO:teuthology.orchestra.run.smithi157.stdout:(72/143): python3-toml-0.10.2-6.el9.noarch.rpm 120 kB/s | 39 kB 00:00 2023-10-16T09:46:46.618 INFO:teuthology.orchestra.run.smithi157.stdout:(73/143): socat-1.7.4.1-5.el9.x86_64.rpm 914 kB/s | 300 kB 00:00 2023-10-16T09:46:46.843 INFO:teuthology.orchestra.run.smithi157.stdout:(74/143): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2023-10-16T09:46:46.860 INFO:teuthology.orchestra.run.smithi157.stdout:(75/143): uuid-1.6.2-55.el9.x86_64.rpm 195 kB/s | 56 kB 00:00 2023-10-16T09:46:46.877 INFO:teuthology.orchestra.run.smithi157.stdout:(76/143): xmlstarlet-1.6.1-20.el9.x86_64.rpm 256 kB/s | 63 kB 00:00 2023-10-16T09:46:46.911 INFO:teuthology.orchestra.run.smithi157.stdout:(77/143): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2023-10-16T09:46:46.961 INFO:teuthology.orchestra.run.smithi157.stdout:(78/143): libgfortran-11.4.1-2.3.el9.x86_64.rpm 8.0 MB/s | 803 kB 00:00 2023-10-16T09:46:46.986 INFO:teuthology.orchestra.run.smithi157.stdout:(79/143): libquadmath-11.4.1-2.3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2023-10-16T09:46:47.020 INFO:teuthology.orchestra.run.smithi157.stdout:(80/143): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 241 kB 00:00 2023-10-16T09:46:47.045 INFO:teuthology.orchestra.run.smithi157.stdout:(81/143): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2023-10-16T09:46:47.070 INFO:teuthology.orchestra.run.smithi157.stdout:(82/143): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2023-10-16T09:46:47.096 INFO:teuthology.orchestra.run.smithi157.stdout:(83/143): python3-requests-2.25.1-7.el9.noarch. 1.5 MB/s | 113 kB 00:00 2023-10-16T09:46:47.129 INFO:teuthology.orchestra.run.smithi157.stdout:(84/143): python3-urllib3-1.26.5-3.el9.noarch.r 3.2 MB/s | 188 kB 00:00 2023-10-16T09:46:47.221 INFO:teuthology.orchestra.run.smithi157.stdout:(85/143): elfutils-libelf-devel-0.189-3.el9.x86 245 kB/s | 22 kB 00:00 2023-10-16T09:46:47.321 INFO:teuthology.orchestra.run.smithi157.stdout:(86/143): jq-1.6-15.el9.x86_64.rpm 1.8 MB/s | 187 kB 00:00 2023-10-16T09:46:47.357 INFO:teuthology.orchestra.run.smithi157.stdout:(87/143): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 689 kB 00:00 2023-10-16T09:46:47.374 INFO:teuthology.orchestra.run.smithi157.stdout:(88/143): librabbitmq-0.11.0-7.el9.x86_64.rpm 875 kB/s | 45 kB 00:00 2023-10-16T09:46:47.391 INFO:teuthology.orchestra.run.smithi157.stdout:(89/143): libuuid-devel-2.37.4-15.el9.x86_64.rp 783 kB/s | 26 kB 00:00 2023-10-16T09:46:47.433 INFO:teuthology.orchestra.run.smithi157.stdout:(90/143): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2023-10-16T09:46:47.483 INFO:teuthology.orchestra.run.smithi157.stdout:(91/143): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.7 MB/s | 86 kB 00:00 2023-10-16T09:46:47.517 INFO:teuthology.orchestra.run.smithi157.stdout:(92/143): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.4 MB/s | 45 kB 00:00 2023-10-16T09:46:47.642 INFO:teuthology.orchestra.run.smithi157.stdout:(93/143): openssl-devel-3.0.7-24.el9.x86_64.rpm 15 MB/s | 4.1 MB 00:00 2023-10-16T09:46:47.767 INFO:teuthology.orchestra.run.smithi157.stdout:(94/143): fmt-8.1.1-5.el9.x86_64.rpm 444 kB/s | 111 kB 00:00 2023-10-16T09:46:47.834 INFO:teuthology.orchestra.run.smithi157.stdout:(95/143): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.6 MB/s | 309 kB 00:00 2023-10-16T09:46:48.043 INFO:teuthology.orchestra.run.smithi157.stdout:(96/143): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2023-10-16T09:46:48.143 INFO:teuthology.orchestra.run.smithi157.stdout:(97/143): liboath-2.6.7-2.el9.x86_64.rpm 490 kB/s | 49 kB 00:00 2023-10-16T09:46:48.226 INFO:teuthology.orchestra.run.smithi157.stdout:(98/143): libarrow-doc-9.0.0-9.el9.noarch.rpm 62 kB/s | 24 kB 00:00 2023-10-16T09:46:48.243 INFO:teuthology.orchestra.run.smithi157.stdout:(99/143): libunwind-1.6.2-1.el9.x86_64.rpm 675 kB/s | 67 kB 00:00 2023-10-16T09:46:48.255 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2023-10-16T09:46:48.256 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2023-10-16T09:46:48.369 INFO:teuthology.orchestra.run.smithi157.stdout:(100/143): libarrow-9.0.0-9.el9.x86_64.rpm 7.4 MB/s | 4.4 MB 00:00 2023-10-16T09:46:48.486 INFO:teuthology.orchestra.run.smithi157.stdout:(101/143): python3-asyncssh-2.13.2-3.el9.noarch 2.2 MB/s | 548 kB 00:00 2023-10-16T09:46:48.511 INFO:teuthology.orchestra.run.smithi157.stdout:(102/143): python3-bcrypt-3.2.2-1.el9.x86_64.rp 308 kB/s | 43 kB 00:00 2023-10-16T09:46:48.528 INFO:teuthology.orchestra.run.smithi157.stdout:(103/143): python3-cachetools-4.2.4-1.el9.noarc 784 kB/s | 32 kB 00:00 2023-10-16T09:46:48.570 INFO:teuthology.orchestra.run.smithi157.stdout:(104/143): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.4 MB/s | 837 kB 00:00 2023-10-16T09:46:48.587 INFO:teuthology.orchestra.run.smithi157.stdout:(105/143): python3-certifi-2021.10.8-2.el9.noar 201 kB/s | 15 kB 00:00 2023-10-16T09:46:48.640 INFO:teuthology.orchestra.run.smithi157.stdout:(106/143): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2023-10-16T09:46:48.679 INFO:teuthology.orchestra.run.smithi157.stdout:(107/143): python3-google-auth-2.23.2-1.el9.noa 2.1 MB/s | 196 kB 00:00 2023-10-16T09:46:48.696 INFO:teuthology.orchestra.run.smithi157.stdout:(108/143): python3-jaraco-8.2.1-3.el9.noarch.rp 193 kB/s | 11 kB 00:00 2023-10-16T09:46:48.713 INFO:teuthology.orchestra.run.smithi157.stdout:(109/143): python3-jaraco-classes-3.2.1-5.el9.n 540 kB/s | 18 kB 00:00 2023-10-16T09:46:48.738 INFO:teuthology.orchestra.run.smithi157.stdout:(110/143): python3-jaraco-collections-3.0.0-8.e 563 kB/s | 23 kB 00:00 2023-10-16T09:46:48.763 INFO:teuthology.orchestra.run.smithi157.stdout:(111/143): python3-jaraco-functools-3.5.0-2.el9 395 kB/s | 19 kB 00:00 2023-10-16T09:46:48.797 INFO:teuthology.orchestra.run.smithi157.stdout:(112/143): python3-cherrypy-18.6.1-2.el9.noarch 1.5 MB/s | 358 kB 00:00 2023-10-16T09:46:48.822 INFO:teuthology.orchestra.run.smithi157.stdout:(113/143): python3-jaraco-text-3.2.0-6.el9.noar 237 kB/s | 20 kB 00:00 2023-10-16T09:46:48.847 INFO:teuthology.orchestra.run.smithi157.stdout:(114/143): python3-jwt+crypto-2.4.0-1.el9.noarc 109 kB/s | 9.0 kB 00:00 2023-10-16T09:46:48.873 INFO:teuthology.orchestra.run.smithi157.stdout:(115/143): python3-jwt-2.4.0-1.el9.noarch.rpm 832 kB/s | 41 kB 00:00 2023-10-16T09:46:50.439 INFO:teuthology.orchestra.run.smithi157.stdout:(116/143): python3-more-itertools-8.12.0-2.el9. 50 kB/s | 79 kB 00:01 2023-10-16T09:46:50.481 INFO:teuthology.orchestra.run.smithi157.stdout:(117/143): python3-kubernetes-26.1.0-2.el9.noar 631 kB/s | 1.0 MB 00:01 2023-10-16T09:46:50.531 INFO:teuthology.orchestra.run.smithi157.stdout:(118/143): python3-natsort-7.1.1-5.el9.noarch.r 634 kB/s | 58 kB 00:00 2023-10-16T09:46:50.565 INFO:teuthology.orchestra.run.smithi157.stdout:(119/143): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2023-10-16T09:46:50.590 INFO:teuthology.orchestra.run.smithi157.stdout:(120/143): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2023-10-16T09:46:50.616 INFO:teuthology.orchestra.run.smithi157.stdout:(121/143): python3-logutils-0.3.5-21.el9.noarch 26 kB/s | 46 kB 00:01 2023-10-16T09:46:50.641 INFO:teuthology.orchestra.run.smithi157.stdout:(122/143): python3-repoze-lru-0.7-16.el9.noarch 623 kB/s | 31 kB 00:00 2023-10-16T09:46:50.687 INFO:teuthology.orchestra.run.smithi157.stdout:(123/143): python3-pyOpenSSL-21.0.0-1.el9.noarc 745 kB/s | 90 kB 00:00 2023-10-16T09:46:50.717 INFO:teuthology.orchestra.run.smithi157.stdout:(124/143): python3-rsa-4.9-2.el9.noarch.rpm 792 kB/s | 59 kB 00:00 2023-10-16T09:46:50.792 INFO:teuthology.orchestra.run.smithi157.stdout:(125/143): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2023-10-16T09:46:50.850 INFO:teuthology.orchestra.run.smithi157.stdout:(126/143): python3-webob-1.8.7-6.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2023-10-16T09:46:50.875 INFO:teuthology.orchestra.run.smithi157.stdout:(127/143): python3-tempora-5.0.0-2.el9.noarch.r 191 kB/s | 36 kB 00:00 2023-10-16T09:46:50.901 INFO:teuthology.orchestra.run.smithi157.stdout:(128/143): python3-typing-extensions-4.4.0-2.el 278 kB/s | 51 kB 00:00 2023-10-16T09:46:50.926 INFO:teuthology.orchestra.run.smithi157.stdout:(129/143): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2023-10-16T09:46:50.976 INFO:teuthology.orchestra.run.smithi157.stdout:(130/143): re2-20211101-3.el9.x86_64.rpm 3.8 MB/s | 192 kB 00:00 2023-10-16T09:46:51.043 INFO:teuthology.orchestra.run.smithi157.stdout:(131/143): python3-werkzeug-2.0.3-3.el9.1.noarc 2.5 MB/s | 427 kB 00:00 2023-10-16T09:46:51.069 INFO:teuthology.orchestra.run.smithi157.stdout:(132/143): python3-zc-lockfile-2.0-10.el9.noarc 120 kB/s | 20 kB 00:00 2023-10-16T09:46:51.452 INFO:teuthology.orchestra.run.smithi157.stdout:(133/143): thrift-0.15.0-2.el9.x86_64.rpm 3.3 MB/s | 1.6 MB 00:00 2023-10-16T09:46:51.494 INFO:teuthology.orchestra.run.smithi157.stdout:(134/143): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2023-10-16T09:46:51.569 INFO:teuthology.orchestra.run.smithi157.stdout:(135/143): librbd1-18.0.0-6675.gc7471068.el9.x8 6.1 MB/s | 3.0 MB 00:00 2023-10-16T09:46:51.670 INFO:teuthology.orchestra.run.smithi157.stdout:(136/143): NetworkManager-1.45.4-1.el9.x86_64.r 13 MB/s | 2.2 MB 00:00 2023-10-16T09:46:51.737 INFO:teuthology.orchestra.run.smithi157.stdout:(137/143): NetworkManager-libnm-1.45.4-1.el9.x8 12 MB/s | 1.8 MB 00:00 2023-10-16T09:46:51.762 INFO:teuthology.orchestra.run.smithi157.stdout:(138/143): NetworkManager-team-1.45.4-1.el9.x86 529 kB/s | 41 kB 00:00 2023-10-16T09:46:51.795 INFO:teuthology.orchestra.run.smithi157.stdout:(139/143): NetworkManager-tui-1.45.4-1.el9.x86_ 4.2 MB/s | 244 kB 00:00 2023-10-16T09:46:51.821 INFO:teuthology.orchestra.run.smithi157.stdout:(140/143): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2023-10-16T09:46:51.846 INFO:teuthology.orchestra.run.smithi157.stdout:(141/143): libacl-2.3.1-4.el9.x86_64.rpm 451 kB/s | 22 kB 00:00 2023-10-16T09:46:51.879 INFO:teuthology.orchestra.run.smithi157.stdout:(142/143): xfsdump-3.1.12-4.el9.x86_64.rpm 5.5 MB/s | 321 kB 00:00 2023-10-16T09:46:51.997 INFO:teuthology.orchestra.run.smithi157.stdout:(143/143): librados2-18.0.0-6675.gc7471068.el9. 3.5 MB/s | 3.3 MB 00:00 2023-10-16T09:46:52.005 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-10-16T09:46:52.005 INFO:teuthology.orchestra.run.smithi157.stdout:Total 13 MB/s | 200 MB 00:15 2023-10-16T09:46:52.450 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-10-16T09:46:52.655 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-10-16T09:46:52.656 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-10-16T09:46:54.086 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2023-10-16T09:46:54.162 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/153 2023-10-16T09:46:54.201 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/153 2023-10-16T09:46:54.229 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/153 2023-10-16T09:46:54.336 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/153 2023-10-16T09:46:54.800 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-10-16T09:46:54.801 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-10-16T09:46:54.804 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/153 2023-10-16T09:46:54.926 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librados2-2:18.0.0-6675.gc7471068.el9.x86_64 6/153 2023-10-16T09:46:55.233 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:18.0.0-6675.gc7471068.el9.x86_64 6/153 2023-10-16T09:46:55.269 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs2-2:18.0.0-6675.gc7471068.el9.x86_64 7/153 2023-10-16T09:46:55.406 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephfs2-2:18.0.0-6675.gc7471068.el9.x86_64 7/153 2023-10-16T09:46:55.481 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rados-2:18.0.0-6675.gc7471068.el9.x86_64 8/153 2023-10-16T09:46:55.527 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/153 2023-10-16T09:46:55.791 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/153 2023-10-16T09:46:55.857 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/153 2023-10-16T09:46:55.907 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephsqlite-2:18.0.0-6675.gc7471068.el9.x86_64 12/153 2023-10-16T09:46:56.040 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6675.gc7471068.el9.x86_64 12/153 2023-10-16T09:46:56.108 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libradosstriper1-2:18.0.0-6675.gc7471068.el9.x86 13/153 2023-10-16T09:46:56.184 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6675.gc7471068.el9.x86 13/153 2023-10-16T09:46:56.539 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/153 2023-10-16T09:46:56.671 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-16T09:46:56.758 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-16T09:46:56.968 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-16T09:46:57.058 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-16T09:46:57.478 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-16T09:46:57.521 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/153 2023-10-16T09:46:57.568 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/153 2023-10-16T09:46:57.666 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/153 2023-10-16T09:46:57.744 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-urllib3-1.26.5-3.el9.noarch 20/153 2023-10-16T09:46:57.775 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/153 2023-10-16T09:46:57.828 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/153 2023-10-16T09:46:57.923 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/153 2023-10-16T09:46:57.967 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/153 2023-10-16T09:46:58.002 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/153 2023-10-16T09:46:58.083 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/153 2023-10-16T09:46:58.123 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-common-2:18.0.0-6675.gc7471068.el9. 27/153 2023-10-16T09:46:58.169 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-argparse-2:18.0.0-6675.gc7471068.el 28/153 2023-10-16T09:46:58.273 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cephfs-2:18.0.0-6675.gc7471068.el9.x86_6 29/153 2023-10-16T09:46:58.450 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/153 2023-10-16T09:46:58.523 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/153 2023-10-16T09:46:59.193 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/153 2023-10-16T09:46:59.288 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/153 2023-10-16T09:46:59.342 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/153 2023-10-16T09:46:59.381 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librados-devel-2:18.0.0-6675.gc7471068.el9.x86_6 35/153 2023-10-16T09:46:59.421 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/153 2023-10-16T09:46:59.594 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/153 2023-10-16T09:46:59.656 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/153 2023-10-16T09:46:59.725 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/153 2023-10-16T09:46:59.821 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/153 2023-10-16T09:46:59.857 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/153 2023-10-16T09:46:59.887 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/153 2023-10-16T09:46:59.955 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/153 2023-10-16T09:47:00.194 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/153 2023-10-16T09:47:00.262 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/153 2023-10-16T09:47:00.488 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/153 2023-10-16T09:47:00.530 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/153 2023-10-16T09:47:00.576 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/153 2023-10-16T09:47:00.704 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/153 2023-10-16T09:47:00.716 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-10-16T09:47:00.758 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-google-auth-1:2.23.2-1.el9.noarch 50/153 2023-10-16T09:47:00.786 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/153 2023-10-16T09:47:00.816 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/153 2023-10-16T09:47:00.817 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/153 2023-10-16T09:47:00.845 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/153 2023-10-16T09:47:00.854 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/153 2023-10-16T09:47:00.894 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/153 2023-10-16T09:47:00.936 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : fmt-8.1.1-5.el9.x86_64 54/153 2023-10-16T09:47:00.943 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/153 2023-10-16T09:47:00.989 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 55/153 2023-10-16T09:47:01.143 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 56/153 2023-10-16T09:47:01.204 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : automake-1.16.2-8.el9.noarch 57/153 2023-10-16T09:47:01.284 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ply-3.11-14.el9.noarch 58/153 2023-10-16T09:47:01.376 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 59/153 2023-10-16T09:47:01.418 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/153 2023-10-16T09:47:01.475 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:18.0.0-6675.gc7471068.el9.x86_64 6/153 2023-10-16T09:47:01.648 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 60/153 2023-10-16T09:47:01.681 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:18.0.0-6675.gc7471068.el9.x86_64 6/153 2023-10-16T09:47:01.706 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 61/153 2023-10-16T09:47:01.717 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:18.0.0-6675.gc7471068.el9.x86_64 7/153 2023-10-16T09:47:01.879 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 62/153 2023-10-16T09:47:01.890 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:18.0.0-6675.gc7471068.el9.x86_64 7/153 2023-10-16T09:47:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 63/153 2023-10-16T09:47:01.957 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:18.0.0-6675.gc7471068.el9.x86_64 8/153 2023-10-16T09:47:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 64/153 2023-10-16T09:47:01.995 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 65/153 2023-10-16T09:47:01.996 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/153 2023-10-16T09:47:02.039 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 66/153 2023-10-16T09:47:02.082 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : fuse-2.9.9-15.el9.x86_64 67/153 2023-10-16T09:47:02.136 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/153 2023-10-16T09:47:02.208 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/153 2023-10-16T09:47:02.245 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/153 2023-10-16T09:47:02.307 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 70/153 2023-10-16T09:47:02.321 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/153 2023-10-16T09:47:02.347 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:18.0.0-6675.gc7471068.el9.x86_64 12/153 2023-10-16T09:47:02.350 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/153 2023-10-16T09:47:02.479 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/153 2023-10-16T09:47:02.523 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 73/153 2023-10-16T09:47:02.528 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6675.gc7471068.el9.x86_64 12/153 2023-10-16T09:47:02.565 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 74/153 2023-10-16T09:47:02.573 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:18.0.0-6675.gc7471068.el9.x86 13/153 2023-10-16T09:47:02.629 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/153 2023-10-16T09:47:02.676 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 76/153 2023-10-16T09:47:02.716 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6675.gc7471068.el9.x86 13/153 2023-10-16T09:47:02.810 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 77/153 2023-10-16T09:47:02.856 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/153 2023-10-16T09:47:03.000 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/153 2023-10-16T09:47:03.050 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/153 2023-10-16T09:47:03.070 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/153 2023-10-16T09:47:03.195 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-16T09:47:03.333 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-16T09:47:03.544 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-16T09:47:03.590 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-16T09:47:04.050 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/153 2023-10-16T09:47:04.053 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-16T09:47:04.095 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/153 2023-10-16T09:47:04.111 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/153 2023-10-16T09:47:04.136 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/153 2023-10-16T09:47:04.152 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/153 2023-10-16T09:47:04.261 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/153 2023-10-16T09:47:04.275 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/153 2023-10-16T09:47:04.302 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 85/153 2023-10-16T09:47:04.335 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-urllib3-1.26.5-3.el9.noarch 20/153 2023-10-16T09:47:04.374 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/153 2023-10-16T09:47:04.428 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/153 2023-10-16T09:47:04.528 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/153 2023-10-16T09:47:04.531 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-0.3.21-2.el9.x86_64 86/153 2023-10-16T09:47:04.558 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 87/153 2023-10-16T09:47:04.584 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/153 2023-10-16T09:47:04.618 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/153 2023-10-16T09:47:04.651 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 88/153 2023-10-16T09:47:04.753 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/153 2023-10-16T09:47:04.790 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:18.0.0-6675.gc7471068.el9. 27/153 2023-10-16T09:47:04.842 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:18.0.0-6675.gc7471068.el 28/153 2023-10-16T09:47:04.954 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:18.0.0-6675.gc7471068.el9.x86_6 29/153 2023-10-16T09:47:05.142 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/153 2023-10-16T09:47:05.211 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/153 2023-10-16T09:47:05.449 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 89/153 2023-10-16T09:47:05.578 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 90/153 2023-10-16T09:47:05.902 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/153 2023-10-16T09:47:06.006 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/153 2023-10-16T09:47:06.102 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/153 2023-10-16T09:47:06.141 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:18.0.0-6675.gc7471068.el9.x86_6 35/153 2023-10-16T09:47:06.398 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/153 2023-10-16T09:47:06.584 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/153 2023-10-16T09:47:06.624 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/153 2023-10-16T09:47:06.711 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/153 2023-10-16T09:47:06.808 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/153 2023-10-16T09:47:06.850 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/153 2023-10-16T09:47:06.898 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/153 2023-10-16T09:47:06.966 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/153 2023-10-16T09:47:07.021 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/153 2023-10-16T09:47:07.083 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/153 2023-10-16T09:47:07.126 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 91/153 2023-10-16T09:47:07.236 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 92/153 2023-10-16T09:47:07.273 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/153 2023-10-16T09:47:07.280 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 93/153 2023-10-16T09:47:07.314 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : jq-1.6-15.el9.x86_64 94/153 2023-10-16T09:47:07.322 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/153 2023-10-16T09:47:07.355 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 95/153 2023-10-16T09:47:07.368 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/153 2023-10-16T09:47:07.505 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/153 2023-10-16T09:47:07.547 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:2.23.2-1.el9.noarch 50/153 2023-10-16T09:47:07.595 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/153 2023-10-16T09:47:07.629 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/153 2023-10-16T09:47:07.693 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/153 2023-10-16T09:47:07.721 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/153 2023-10-16T09:47:07.735 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fmt-8.1.1-5.el9.x86_64 54/153 2023-10-16T09:47:07.751 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librbd1-2:18.0.0-6675.gc7471068.el9.x86_64 97/153 2023-10-16T09:47:07.781 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 55/153 2023-10-16T09:47:07.871 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:18.0.0-6675.gc7471068.el9.x86_64 97/153 2023-10-16T09:47:07.898 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rbd-2:18.0.0-6675.gc7471068.el9.x86_64 98/153 2023-10-16T09:47:07.936 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 56/153 2023-10-16T09:47:07.951 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/153 2023-10-16T09:47:07.988 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : automake-1.16.2-8.el9.noarch 57/153 2023-10-16T09:47:08.071 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ply-3.11-14.el9.noarch 58/153 2023-10-16T09:47:08.162 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 59/153 2023-10-16T09:47:08.452 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 100/153 2023-10-16T09:47:08.478 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librgw2-2:18.0.0-6675.gc7471068.el9.x86_64 101/153 2023-10-16T09:47:08.481 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 60/153 2023-10-16T09:47:08.539 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 61/153 2023-10-16T09:47:08.570 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librgw2-2:18.0.0-6675.gc7471068.el9.x86_64 101/153 2023-10-16T09:47:08.597 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rgw-2:18.0.0-6675.gc7471068.el9.x86_64 102/153 2023-10-16T09:47:08.715 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 62/153 2023-10-16T09:47:08.776 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 63/153 2023-10-16T09:47:08.800 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 64/153 2023-10-16T09:47:08.840 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 65/153 2023-10-16T09:47:08.883 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 66/153 2023-10-16T09:47:08.926 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.9-15.el9.x86_64 67/153 2023-10-16T09:47:08.975 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/153 2023-10-16T09:47:09.051 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/153 2023-10-16T09:47:09.103 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 70/153 2023-10-16T09:47:09.142 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/153 2023-10-16T09:47:09.269 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/153 2023-10-16T09:47:09.314 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 73/153 2023-10-16T09:47:09.350 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 74/153 2023-10-16T09:47:09.411 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/153 2023-10-16T09:47:09.453 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 76/153 2023-10-16T09:47:09.596 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 77/153 2023-10-16T09:47:09.642 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/153 2023-10-16T09:47:09.783 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/153 2023-10-16T09:47:09.850 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/153 2023-10-16T09:47:10.778 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/153 2023-10-16T09:47:10.831 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/153 2023-10-16T09:47:10.864 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/153 2023-10-16T09:47:10.993 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/153 2023-10-16T09:47:11.039 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 85/153 2023-10-16T09:47:11.217 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:18.0.0-6675.gc7471068.el9.x86_64 103/153 2023-10-16T09:47:11.251 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-common-2:18.0.0-6675.gc7471068.el9.x86_64 103/153 2023-10-16T09:47:11.272 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.21-2.el9.x86_64 86/153 2023-10-16T09:47:11.329 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 87/153 2023-10-16T09:47:11.425 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 88/153 2023-10-16T09:47:11.893 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:18.0.0-6675.gc7471068.el9.x86_64 103/153 2023-10-16T09:47:11.938 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-base-2:18.0.0-6675.gc7471068.el9.x86_64 104/153 2023-10-16T09:47:12.095 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-base-2:18.0.0-6675.gc7471068.el9.x86_64 104/153 2023-10-16T09:47:12.096 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-16T09:47:12.096 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-16T09:47:12.096 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:47:12.139 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-selinux-2:18.0.0-6675.gc7471068.el9.x86_64 105/153 2023-10-16T09:47:12.245 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 89/153 2023-10-16T09:47:12.382 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 90/153 2023-10-16T09:47:13.931 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 91/153 2023-10-16T09:47:14.015 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 92/153 2023-10-16T09:47:14.052 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 93/153 2023-10-16T09:47:14.083 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : jq-1.6-15.el9.x86_64 94/153 2023-10-16T09:47:14.123 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 95/153 2023-10-16T09:47:14.483 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/153 2023-10-16T09:47:14.513 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:18.0.0-6675.gc7471068.el9.x86_64 97/153 2023-10-16T09:47:14.616 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:18.0.0-6675.gc7471068.el9.x86_64 97/153 2023-10-16T09:47:14.652 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:18.0.0-6675.gc7471068.el9.x86_64 98/153 2023-10-16T09:47:14.706 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/153 2023-10-16T09:47:15.180 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 100/153 2023-10-16T09:47:15.216 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:18.0.0-6675.gc7471068.el9.x86_64 101/153 2023-10-16T09:47:15.308 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:18.0.0-6675.gc7471068.el9.x86_64 101/153 2023-10-16T09:47:15.335 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:18.0.0-6675.gc7471068.el9.x86_64 102/153 2023-10-16T09:47:17.834 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:18.0.0-6675.gc7471068.el9.x86_64 103/153 2023-10-16T09:47:17.872 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:18.0.0-6675.gc7471068.el9.x86_64 103/153 2023-10-16T09:47:18.494 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:18.0.0-6675.gc7471068.el9.x86_64 103/153 2023-10-16T09:47:18.567 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:18.0.0-6675.gc7471068.el9.x86_64 104/153 2023-10-16T09:47:18.755 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:18.0.0-6675.gc7471068.el9.x86_64 104/153 2023-10-16T09:47:18.756 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-16T09:47:18.756 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-16T09:47:18.756 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:47:18.801 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:18.0.0-6675.gc7471068.el9.x86_64 105/153 2023-10-16T09:47:24.591 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6675.gc7471068.el9.x86_64 105/153 2023-10-16T09:47:24.591 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /sys 2023-10-16T09:47:24.591 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /proc 2023-10-16T09:47:24.592 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /mnt 2023-10-16T09:47:24.592 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /var/tmp 2023-10-16T09:47:24.592 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /home 2023-10-16T09:47:24.592 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /root 2023-10-16T09:47:24.592 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /tmp 2023-10-16T09:47:24.592 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:47:24.798 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-volume-2:18.0.0-6675.gc7471068.el9.noarch 106/153 2023-10-16T09:47:24.846 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-volume-2:18.0.0-6675.gc7471068.el9.noarch 106/153 2023-10-16T09:47:24.846 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-16T09:47:24.846 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-16T09:47:24.847 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:47:26.534 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-osd-2:18.0.0-6675.gc7471068.el9.x86_64 107/153 2023-10-16T09:47:26.618 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-osd-2:18.0.0-6675.gc7471068.el9.x86_64 107/153 2023-10-16T09:47:26.619 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-16T09:47:26.619 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-16T09:47:26.619 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-16T09:47:26.619 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-16T09:47:26.620 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:47:26.867 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mds-2:18.0.0-6675.gc7471068.el9.x86_64 108/153 2023-10-16T09:47:26.942 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mds-2:18.0.0-6675.gc7471068.el9.x86_64 108/153 2023-10-16T09:47:26.943 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-16T09:47:26.943 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-16T09:47:26.943 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-16T09:47:26.944 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-16T09:47:26.944 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:47:27.446 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mon-2:18.0.0-6675.gc7471068.el9.x86_64 109/153 2023-10-16T09:47:27.516 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mon-2:18.0.0-6675.gc7471068.el9.x86_64 109/153 2023-10-16T09:47:27.516 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-16T09:47:27.516 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-16T09:47:27.517 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-16T09:47:27.517 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-16T09:47:27.517 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:47:28.128 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephadm-2:18.0.0-6675.gc7471068.el9.noarch 110/153 2023-10-16T09:47:28.163 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephadm-2:18.0.0-6675.gc7471068.el9.noarch 110/153 2023-10-16T09:47:28.212 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6675.gc7471068.e 111/153 2023-10-16T09:47:28.323 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6675.gc7471068. 112/153 2023-10-16T09:47:28.373 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6675.gc7471068.el9.noa 113/153 2023-10-16T09:47:28.701 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6675.gc7471068.el9.noa 113/153 2023-10-16T09:47:28.732 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6675.gc7471068.el9.n 114/153 2023-10-16T09:47:29.861 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6675.gc7471068.el9.n 114/153 2023-10-16T09:47:29.883 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6675.gc74 115/153 2023-10-16T09:47:30.023 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6675.gc74 115/153 2023-10-16T09:47:30.198 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6675.gc7471068.el 116/153 2023-10-16T09:47:30.234 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-2:18.0.0-6675.gc7471068.el9.x86_64 117/153 2023-10-16T09:47:30.308 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6675.gc7471068.el9.x86_64 117/153 2023-10-16T09:47:30.309 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-16T09:47:30.309 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-16T09:47:30.309 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-16T09:47:30.309 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-16T09:47:30.309 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:47:30.360 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-rook-2:18.0.0-6675.gc7471068.el9.noarch 118/153 2023-10-16T09:47:30.396 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6675.gc7471068.el9.noarch 118/153 2023-10-16T09:47:31.317 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-2:18.0.0-6675.gc7471068.el9.x86_64 119/153 2023-10-16T09:47:31.350 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6675.gc7471068.el9.x86_64 105/153 2023-10-16T09:47:31.350 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2023-10-16T09:47:31.350 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2023-10-16T09:47:31.351 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2023-10-16T09:47:31.351 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2023-10-16T09:47:31.351 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2023-10-16T09:47:31.351 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /root 2023-10-16T09:47:31.351 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2023-10-16T09:47:31.352 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:47:31.353 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-radosgw-2:18.0.0-6675.gc7471068.el9.x86_64 120/153 2023-10-16T09:47:31.443 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6675.gc7471068.el9.x86_64 120/153 2023-10-16T09:47:31.443 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-16T09:47:31.444 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-16T09:47:31.444 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-16T09:47:31.444 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-16T09:47:31.444 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:47:31.496 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6675.gc7471 121/153 2023-10-16T09:47:31.563 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:18.0.0-6675.gc7471068.el9.noarch 106/153 2023-10-16T09:47:31.570 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6675.gc7471 121/153 2023-10-16T09:47:31.570 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-16T09:47:31.571 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-16T09:47:31.571 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:47:31.607 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:18.0.0-6675.gc7471068.el9.noarch 106/153 2023-10-16T09:47:31.607 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-16T09:47:31.607 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-16T09:47:31.608 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:47:31.646 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-mirror-2:18.0.0-6675.gc7471068.el9.x86_64 122/153 2023-10-16T09:47:31.724 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6675.gc7471068.el9.x86_64 122/153 2023-10-16T09:47:31.725 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-16T09:47:31.725 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-10-16T09:47:31.725 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-16T09:47:31.725 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-16T09:47:31.725 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:47:32.088 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-mirror-2:18.0.0-6675.gc7471068.el9.x86_64 123/153 2023-10-16T09:47:32.190 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6675.gc7471068.el9.x86_64 123/153 2023-10-16T09:47:32.191 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-16T09:47:32.191 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-16T09:47:32.191 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-16T09:47:32.191 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-16T09:47:32.192 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:47:33.299 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:18.0.0-6675.gc7471068.el9.x86_64 107/153 2023-10-16T09:47:33.410 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:18.0.0-6675.gc7471068.el9.x86_64 107/153 2023-10-16T09:47:33.410 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-16T09:47:33.410 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-16T09:47:33.411 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-16T09:47:33.411 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-16T09:47:33.411 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:47:33.649 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:18.0.0-6675.gc7471068.el9.x86_64 108/153 2023-10-16T09:47:33.714 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:18.0.0-6675.gc7471068.el9.x86_64 108/153 2023-10-16T09:47:33.714 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-16T09:47:33.715 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-16T09:47:33.715 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-16T09:47:33.715 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-16T09:47:33.715 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:47:34.200 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:18.0.0-6675.gc7471068.el9.x86_64 109/153 2023-10-16T09:47:34.285 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:18.0.0-6675.gc7471068.el9.x86_64 109/153 2023-10-16T09:47:34.285 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-16T09:47:34.285 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-16T09:47:34.285 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-16T09:47:34.286 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-16T09:47:34.286 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:47:35.009 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:18.0.0-6675.gc7471068.el9.noarch 110/153 2023-10-16T09:47:35.052 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:18.0.0-6675.gc7471068.el9.noarch 110/153 2023-10-16T09:47:35.098 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6675.gc7471068.e 111/153 2023-10-16T09:47:35.211 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6675.gc7471068. 112/153 2023-10-16T09:47:35.263 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6675.gc7471068.el9.noa 113/153 2023-10-16T09:47:35.589 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6675.gc7471068.el9.noa 113/153 2023-10-16T09:47:35.648 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6675.gc7471068.el9.n 114/153 2023-10-16T09:47:36.336 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-test-2:18.0.0-6675.gc7471068.el9.x86_64 124/153 2023-10-16T09:47:36.393 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-fuse-2:18.0.0-6675.gc7471068.el9.x86_64 125/153 2023-10-16T09:47:36.433 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-nbd-2:18.0.0-6675.gc7471068.el9.x86_64 126/153 2023-10-16T09:47:36.708 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 127/153 2023-10-16T09:47:36.781 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6675.gc7471068.el9.n 114/153 2023-10-16T09:47:36.815 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6675.gc74 115/153 2023-10-16T09:47:36.823 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 128/153 2023-10-16T09:47:36.893 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-fuse-2:18.0.0-6675.gc7471068.el9.x86_64 129/153 2023-10-16T09:47:36.941 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/153 2023-10-16T09:47:36.954 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6675.gc74 115/153 2023-10-16T09:47:36.972 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/153 2023-10-16T09:47:37.002 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs-devel-2:18.0.0-6675.gc7471068.el9.x86_ 132/153 2023-10-16T09:47:37.038 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-team-1:1.45.4-1.el9.x86_64 133/153 2023-10-16T09:47:37.122 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6675.gc7471068.el 116/153 2023-10-16T09:47:37.142 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-tui-1:1.45.4-1.el9.x86_64 134/153 2023-10-16T09:47:37.150 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:18.0.0-6675.gc7471068.el9.x86_64 117/153 2023-10-16T09:47:37.210 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 135/153 2023-10-16T09:47:37.221 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6675.gc7471068.el9.x86_64 117/153 2023-10-16T09:47:37.221 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-16T09:47:37.221 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-16T09:47:37.222 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-16T09:47:37.222 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-16T09:47:37.222 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:47:37.249 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 136/153 2023-10-16T09:47:37.276 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:18.0.0-6675.gc7471068.el9.noarch 118/153 2023-10-16T09:47:37.313 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6675.gc7471068.el9.noarch 118/153 2023-10-16T09:47:38.102 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-16T09:47:38.147 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-16T09:47:38.188 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:18.0.0-6675.gc7471068.el9.x86_64 119/153 2023-10-16T09:47:38.200 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-16T09:47:38.218 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:18.0.0-6675.gc7471068.el9.x86_64 120/153 2023-10-16T09:47:38.263 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-top-2:18.0.0-6675.gc7471068.el9.noarch 138/153 2023-10-16T09:47:38.287 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6675.gc7471068.el9.x86_64 120/153 2023-10-16T09:47:38.287 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-16T09:47:38.288 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-16T09:47:38.288 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-16T09:47:38.288 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-16T09:47:38.288 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:47:38.335 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6675.gc7471 121/153 2023-10-16T09:47:38.421 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6675.gc7471 121/153 2023-10-16T09:47:38.421 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-16T09:47:38.421 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-16T09:47:38.422 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:47:38.517 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-mirror-2:18.0.0-6675.gc7471068.el9.x86_64 122/153 2023-10-16T09:47:38.624 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6675.gc7471068.el9.x86_64 122/153 2023-10-16T09:47:38.625 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-16T09:47:38.625 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-10-16T09:47:38.625 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-16T09:47:38.625 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-16T09:47:38.625 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:47:39.004 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:18.0.0-6675.gc7471068.el9.x86_64 123/153 2023-10-16T09:47:39.088 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6675.gc7471068.el9.x86_64 123/153 2023-10-16T09:47:39.089 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-16T09:47:39.089 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-16T09:47:39.089 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-16T09:47:39.089 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-16T09:47:39.089 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:47:39.592 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 139/153 2023-10-16T09:47:39.694 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 140/153 2023-10-16T09:47:39.750 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 141/153 2023-10-16T09:47:39.879 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flex-2.6.4-9.el9.x86_64 142/153 2023-10-16T09:47:39.920 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : bison-3.7.4-5.el9.x86_64 143/153 2023-10-16T09:47:39.957 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 144/153 2023-10-16T09:47:39.958 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-16T09:47:40.076 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-16T09:47:40.117 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 146/153 2023-10-16T09:47:40.149 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 147/153 2023-10-16T09:47:40.182 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 148/153 2023-10-16T09:47:40.221 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-16T09:47:40.222 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-16T09:47:40.540 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-16T09:47:40.540 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-16T09:47:40.704 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-16T09:47:40.735 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-16T09:47:40.735 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-16T09:47:40.862 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-16T09:47:40.976 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-16T09:47:41.230 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-16T09:47:41.231 INFO:teuthology.orchestra.run.smithi084.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-16T09:47:41.231 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:47:43.243 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:18.0.0-6675.gc7471068.el9.x86_64 124/153 2023-10-16T09:47:43.299 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:18.0.0-6675.gc7471068.el9.x86_64 125/153 2023-10-16T09:47:43.331 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:18.0.0-6675.gc7471068.el9.x86_64 126/153 2023-10-16T09:47:43.626 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 127/153 2023-10-16T09:47:43.747 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 128/153 2023-10-16T09:47:43.806 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:18.0.0-6675.gc7471068.el9.x86_64 129/153 2023-10-16T09:47:43.858 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/153 2023-10-16T09:47:43.888 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/153 2023-10-16T09:47:43.918 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:18.0.0-6675.gc7471068.el9.x86_ 132/153 2023-10-16T09:47:43.954 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-team-1:1.45.4-1.el9.x86_64 133/153 2023-10-16T09:47:44.039 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-tui-1:1.45.4-1.el9.x86_64 134/153 2023-10-16T09:47:44.112 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 135/153 2023-10-16T09:47:44.155 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 136/153 2023-10-16T09:47:45.084 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-16T09:47:45.128 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-16T09:47:45.192 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-16T09:47:45.266 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-top-2:18.0.0-6675.gc7471068.el9.noarch 138/153 2023-10-16T09:47:45.690 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-2:18.0.0-6675.gc7471068.el9.x86_64 1/153 2023-10-16T09:47:45.690 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-base-2:18.0.0-6675.gc7471068.el9.x86_64 2/153 2023-10-16T09:47:45.690 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-common-2:18.0.0-6675.gc7471068.el9.x86_64 3/153 2023-10-16T09:47:45.690 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-fuse-2:18.0.0-6675.gc7471068.el9.x86_64 4/153 2023-10-16T09:47:45.691 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6675.gc7471 5/153 2023-10-16T09:47:45.691 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mds-2:18.0.0-6675.gc7471068.el9.x86_64 6/153 2023-10-16T09:47:45.691 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-2:18.0.0-6675.gc7471068.el9.x86_64 7/153 2023-10-16T09:47:45.691 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mon-2:18.0.0-6675.gc7471068.el9.x86_64 8/153 2023-10-16T09:47:45.691 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-osd-2:18.0.0-6675.gc7471068.el9.x86_64 9/153 2023-10-16T09:47:45.692 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-radosgw-2:18.0.0-6675.gc7471068.el9.x86_64 10/153 2023-10-16T09:47:45.692 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-selinux-2:18.0.0-6675.gc7471068.el9.x86_64 11/153 2023-10-16T09:47:45.692 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-test-2:18.0.0-6675.gc7471068.el9.x86_64 12/153 2023-10-16T09:47:45.692 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-mirror-2:18.0.0-6675.gc7471068.el9.x86_64 13/153 2023-10-16T09:47:45.692 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs-devel-2:18.0.0-6675.gc7471068.el9.x86_ 14/153 2023-10-16T09:47:45.693 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs2-2:18.0.0-6675.gc7471068.el9.x86_64 15/153 2023-10-16T09:47:45.693 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephsqlite-2:18.0.0-6675.gc7471068.el9.x86_64 16/153 2023-10-16T09:47:45.693 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados-devel-2:18.0.0-6675.gc7471068.el9.x86_6 17/153 2023-10-16T09:47:45.693 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libradosstriper1-2:18.0.0-6675.gc7471068.el9.x86 18/153 2023-10-16T09:47:45.693 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librgw2-2:18.0.0-6675.gc7471068.el9.x86_64 19/153 2023-10-16T09:47:45.694 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6675.gc7471068.el 20/153 2023-10-16T09:47:45.694 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-common-2:18.0.0-6675.gc7471068.el9. 21/153 2023-10-16T09:47:45.694 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cephfs-2:18.0.0-6675.gc7471068.el9.x86_6 22/153 2023-10-16T09:47:45.694 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rados-2:18.0.0-6675.gc7471068.el9.x86_64 23/153 2023-10-16T09:47:45.694 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rbd-2:18.0.0-6675.gc7471068.el9.x86_64 24/153 2023-10-16T09:47:45.695 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rgw-2:18.0.0-6675.gc7471068.el9.x86_64 25/153 2023-10-16T09:47:45.695 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-fuse-2:18.0.0-6675.gc7471068.el9.x86_64 26/153 2023-10-16T09:47:45.695 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-mirror-2:18.0.0-6675.gc7471068.el9.x86_64 27/153 2023-10-16T09:47:45.705 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-nbd-2:18.0.0-6675.gc7471068.el9.x86_64 28/153 2023-10-16T09:47:45.705 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6675.gc7471068. 29/153 2023-10-16T09:47:45.705 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6675.gc7471068.el9.noa 30/153 2023-10-16T09:47:45.706 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6675.gc7471068.el9.n 31/153 2023-10-16T09:47:45.706 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6675.gc74 32/153 2023-10-16T09:47:45.706 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6675.gc7471068.el 33/153 2023-10-16T09:47:45.706 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6675.gc7471068.el9.noarch 34/153 2023-10-16T09:47:45.706 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6675.gc7471068.e 35/153 2023-10-16T09:47:45.707 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-volume-2:18.0.0-6675.gc7471068.el9.noarch 36/153 2023-10-16T09:47:45.707 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephadm-2:18.0.0-6675.gc7471068.el9.noarch 37/153 2023-10-16T09:47:45.707 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-top-2:18.0.0-6675.gc7471068.el9.noarch 38/153 2023-10-16T09:47:45.707 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/153 2023-10-16T09:47:45.707 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/153 2023-10-16T09:47:45.708 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/153 2023-10-16T09:47:45.708 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/153 2023-10-16T09:47:45.708 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/153 2023-10-16T09:47:45.708 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/153 2023-10-16T09:47:45.708 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 45/153 2023-10-16T09:47:45.708 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 46/153 2023-10-16T09:47:45.709 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 47/153 2023-10-16T09:47:45.709 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 48/153 2023-10-16T09:47:45.709 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 49/153 2023-10-16T09:47:45.709 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 50/153 2023-10-16T09:47:45.709 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 51/153 2023-10-16T09:47:45.709 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 52/153 2023-10-16T09:47:45.710 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 53/153 2023-10-16T09:47:45.710 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-13.11-1.el9.x86_64 54/153 2023-10-16T09:47:45.710 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 55/153 2023-10-16T09:47:45.710 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 56/153 2023-10-16T09:47:45.710 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 57/153 2023-10-16T09:47:45.711 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/153 2023-10-16T09:47:45.711 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 59/153 2023-10-16T09:47:45.711 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 60/153 2023-10-16T09:47:45.711 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/153 2023-10-16T09:47:45.711 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/153 2023-10-16T09:47:45.712 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/153 2023-10-16T09:47:45.712 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/153 2023-10-16T09:47:45.712 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/153 2023-10-16T09:47:45.712 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/153 2023-10-16T09:47:45.712 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/153 2023-10-16T09:47:45.712 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 68/153 2023-10-16T09:47:45.713 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 69/153 2023-10-16T09:47:45.713 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 70/153 2023-10-16T09:47:45.713 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/153 2023-10-16T09:47:45.713 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/153 2023-10-16T09:47:45.713 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/153 2023-10-16T09:47:45.713 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/153 2023-10-16T09:47:45.714 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 75/153 2023-10-16T09:47:45.714 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/153 2023-10-16T09:47:45.714 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/153 2023-10-16T09:47:45.714 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 78/153 2023-10-16T09:47:45.714 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 79/153 2023-10-16T09:47:45.715 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 80/153 2023-10-16T09:47:45.715 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ply-3.11-14.el9.noarch 81/153 2023-10-16T09:47:45.715 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 82/153 2023-10-16T09:47:45.715 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 83/153 2023-10-16T09:47:45.715 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-urllib3-1.26.5-3.el9.noarch 84/153 2023-10-16T09:47:45.716 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 85/153 2023-10-16T09:47:45.716 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : automake-1.16.2-8.el9.noarch 86/153 2023-10-16T09:47:45.716 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 87/153 2023-10-16T09:47:45.716 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : jq-1.6-15.el9.x86_64 88/153 2023-10-16T09:47:45.716 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/153 2023-10-16T09:47:45.716 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/153 2023-10-16T09:47:45.717 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/153 2023-10-16T09:47:45.717 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/153 2023-10-16T09:47:45.717 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/153 2023-10-16T09:47:45.717 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/153 2023-10-16T09:47:45.717 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/153 2023-10-16T09:47:45.718 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/153 2023-10-16T09:47:45.718 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/153 2023-10-16T09:47:45.718 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/153 2023-10-16T09:47:45.718 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/153 2023-10-16T09:47:45.718 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/153 2023-10-16T09:47:45.719 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/153 2023-10-16T09:47:45.719 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/153 2023-10-16T09:47:45.719 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/153 2023-10-16T09:47:45.719 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/153 2023-10-16T09:47:45.719 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/153 2023-10-16T09:47:45.719 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/153 2023-10-16T09:47:45.720 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/153 2023-10-16T09:47:45.720 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-google-auth-1:2.23.2-1.el9.noarch 108/153 2023-10-16T09:47:45.720 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/153 2023-10-16T09:47:45.720 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/153 2023-10-16T09:47:45.720 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/153 2023-10-16T09:47:45.720 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/153 2023-10-16T09:47:45.721 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/153 2023-10-16T09:47:45.721 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/153 2023-10-16T09:47:45.721 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/153 2023-10-16T09:47:45.721 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/153 2023-10-16T09:47:45.721 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/153 2023-10-16T09:47:45.721 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/153 2023-10-16T09:47:45.722 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/153 2023-10-16T09:47:45.722 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/153 2023-10-16T09:47:45.722 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/153 2023-10-16T09:47:45.722 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/153 2023-10-16T09:47:45.722 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/153 2023-10-16T09:47:45.723 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/153 2023-10-16T09:47:45.723 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/153 2023-10-16T09:47:45.723 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/153 2023-10-16T09:47:45.723 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/153 2023-10-16T09:47:45.723 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/153 2023-10-16T09:47:45.723 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/153 2023-10-16T09:47:45.724 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/153 2023-10-16T09:47:45.724 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/153 2023-10-16T09:47:45.724 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/153 2023-10-16T09:47:45.724 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/153 2023-10-16T09:47:45.724 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:18.0.0-6675.gc7471068.el9.x86_64 134/153 2023-10-16T09:47:45.724 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/153 2023-10-16T09:47:45.725 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:18.0.0-6675.gc7471068.el9.x86_64 136/153 2023-10-16T09:47:45.725 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/153 2023-10-16T09:47:45.725 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 138/153 2023-10-16T09:47:45.725 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 139/153 2023-10-16T09:47:45.725 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-1:1.45.4-1.el9.x86_64 140/153 2023-10-16T09:47:45.725 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 141/153 2023-10-16T09:47:45.726 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 142/153 2023-10-16T09:47:45.726 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 143/153 2023-10-16T09:47:45.726 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-team-1:1.45.4-1.el9.x86_64 144/153 2023-10-16T09:47:45.726 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 145/153 2023-10-16T09:47:45.726 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-tui-1:1.45.4-1.el9.x86_64 146/153 2023-10-16T09:47:45.727 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 147/153 2023-10-16T09:47:45.727 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : acl-2.3.1-4.el9.x86_64 148/153 2023-10-16T09:47:45.727 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : acl-2.3.1-3.el9.x86_64 149/153 2023-10-16T09:47:45.727 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 150/153 2023-10-16T09:47:45.727 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-16T09:47:45.727 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 152/153 2023-10-16T09:47:46.710 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 139/153 2023-10-16T09:47:46.809 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 140/153 2023-10-16T09:47:46.853 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 141/153 2023-10-16T09:47:46.976 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flex-2.6.4-9.el9.x86_64 142/153 2023-10-16T09:47:47.012 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : bison-3.7.4-5.el9.x86_64 143/153 2023-10-16T09:47:47.049 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 144/153 2023-10-16T09:47:47.049 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-16T09:47:47.168 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-16T09:47:47.200 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 146/153 2023-10-16T09:47:47.233 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 147/153 2023-10-16T09:47:47.265 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 148/153 2023-10-16T09:47:47.304 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-16T09:47:47.305 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-16T09:47:47.640 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-16T09:47:47.640 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-16T09:47:47.769 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-16T09:47:47.769 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:47:47.769 INFO:teuthology.orchestra.run.smithi084.stdout:Upgraded: 2023-10-16T09:47:47.769 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-1:1.45.4-1.el9.x86_64 2023-10-16T09:47:47.770 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 2023-10-16T09:47:47.770 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-team-1:1.45.4-1.el9.x86_64 2023-10-16T09:47:47.770 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-tui-1:1.45.4-1.el9.x86_64 2023-10-16T09:47:47.770 INFO:teuthology.orchestra.run.smithi084.stdout: acl-2.3.1-4.el9.x86_64 2023-10-16T09:47:47.770 INFO:teuthology.orchestra.run.smithi084.stdout: libacl-2.3.1-4.el9.x86_64 2023-10-16T09:47:47.771 INFO:teuthology.orchestra.run.smithi084.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-10-16T09:47:47.771 INFO:teuthology.orchestra.run.smithi084.stdout: librados2-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.771 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.771 INFO:teuthology.orchestra.run.smithi084.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-10-16T09:47:47.771 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2023-10-16T09:47:47.772 INFO:teuthology.orchestra.run.smithi084.stdout: automake-1.16.2-8.el9.noarch 2023-10-16T09:47:47.772 INFO:teuthology.orchestra.run.smithi084.stdout: bison-3.7.4-5.el9.x86_64 2023-10-16T09:47:47.772 INFO:teuthology.orchestra.run.smithi084.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-16T09:47:47.772 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.772 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.773 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.773 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.773 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:47.773 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.773 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.774 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.774 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:47.774 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:47.774 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:47.774 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:47.775 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:47.775 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.775 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.775 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:47.775 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.776 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.776 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.776 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:47.776 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:47.776 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.776 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-top-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:47.777 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-10-16T09:47:47.777 INFO:teuthology.orchestra.run.smithi084.stdout: flex-2.6.4-9.el9.x86_64 2023-10-16T09:47:47.777 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-16T09:47:47.777 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-16T09:47:47.777 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-16T09:47:47.778 INFO:teuthology.orchestra.run.smithi084.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-16T09:47:47.778 INFO:teuthology.orchestra.run.smithi084.stdout: fuse-2.9.9-15.el9.x86_64 2023-10-16T09:47:47.778 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-16T09:47:47.778 INFO:teuthology.orchestra.run.smithi084.stdout: jq-1.6-15.el9.x86_64 2023-10-16T09:47:47.778 INFO:teuthology.orchestra.run.smithi084.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-10-16T09:47:47.779 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-16T09:47:47.779 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-16T09:47:47.779 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.779 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.779 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.779 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-16T09:47:47.780 INFO:teuthology.orchestra.run.smithi084.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-16T09:47:47.780 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-16T09:47:47.780 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-16T09:47:47.780 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-16T09:47:47.780 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.780 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.781 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-16T09:47:47.781 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.781 INFO:teuthology.orchestra.run.smithi084.stdout: libtool-2.4.6-45.el9.x86_64 2023-10-16T09:47:47.781 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-16T09:47:47.781 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-10-16T09:47:47.782 INFO:teuthology.orchestra.run.smithi084.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-10-16T09:47:47.782 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-16T09:47:47.782 INFO:teuthology.orchestra.run.smithi084.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-16T09:47:47.782 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-16T09:47:47.782 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-16T09:47:47.782 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-10-16T09:47:47.783 INFO:teuthology.orchestra.run.smithi084.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-16T09:47:47.783 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-13.11-1.el9.x86_64 2023-10-16T09:47:47.783 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-10-16T09:47:47.783 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-10-16T09:47:47.783 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-server-13.11-1.el9.x86_64 2023-10-16T09:47:47.784 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-16T09:47:47.784 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-16T09:47:47.784 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-16T09:47:47.784 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-16T09:47:47.784 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.785 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.785 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.785 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-16T09:47:47.785 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-16T09:47:47.785 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-16T09:47:47.785 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-16T09:47:47.786 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-16T09:47:47.786 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-16T09:47:47.786 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth-1:2.23.2-1.el9.noarch 2023-10-16T09:47:47.786 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-16T09:47:47.786 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-16T09:47:47.787 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-16T09:47:47.787 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-16T09:47:47.787 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-16T09:47:47.787 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-16T09:47:47.787 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-16T09:47:47.787 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-16T09:47:47.788 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-16T09:47:47.788 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-16T09:47:47.788 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-16T09:47:47.788 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-16T09:47:47.788 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-16T09:47:47.788 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-16T09:47:47.789 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-16T09:47:47.789 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-16T09:47:47.789 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-16T09:47:47.789 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-16T09:47:47.789 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-16T09:47:47.790 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-16T09:47:47.790 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply-3.11-14.el9.noarch 2023-10-16T09:47:47.790 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-16T09:47:47.790 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-16T09:47:47.790 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-16T09:47:47.791 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-16T09:47:47.791 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-16T09:47:47.791 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-16T09:47:47.791 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-16T09:47:47.791 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.791 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.792 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-16T09:47:47.792 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-16T09:47:47.792 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-16T09:47:47.792 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.792 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-16T09:47:47.792 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-16T09:47:47.793 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-16T09:47:47.793 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-16T09:47:47.793 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-16T09:47:47.793 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-16T09:47:47.793 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3-1.26.5-3.el9.noarch 2023-10-16T09:47:47.794 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-16T09:47:47.794 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-16T09:47:47.794 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-16T09:47:47.794 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-16T09:47:47.794 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.795 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.795 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:47.795 INFO:teuthology.orchestra.run.smithi084.stdout: re2-1:20211101-3.el9.x86_64 2023-10-16T09:47:47.795 INFO:teuthology.orchestra.run.smithi084.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-16T09:47:47.795 INFO:teuthology.orchestra.run.smithi084.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-16T09:47:47.795 INFO:teuthology.orchestra.run.smithi084.stdout: uuid-1.6.2-55.el9.x86_64 2023-10-16T09:47:47.796 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-10-16T09:47:47.796 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-16T09:47:47.796 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-16T09:47:47.797 INFO:teuthology.orchestra.run.smithi084.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-10-16T09:47:47.797 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:47:47.797 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2023-10-16T09:47:47.810 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-16T09:47:47.810 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-16T09:47:47.949 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-16T09:47:48.070 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-16T09:47:48.265 DEBUG:teuthology.parallel:result is None 2023-10-16T09:47:48.333 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-16T09:47:48.334 INFO:teuthology.orchestra.run.smithi157.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-16T09:47:48.334 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:47:52.914 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:18.0.0-6675.gc7471068.el9.x86_64 1/153 2023-10-16T09:47:52.915 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:18.0.0-6675.gc7471068.el9.x86_64 2/153 2023-10-16T09:47:52.915 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:18.0.0-6675.gc7471068.el9.x86_64 3/153 2023-10-16T09:47:52.915 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:18.0.0-6675.gc7471068.el9.x86_64 4/153 2023-10-16T09:47:52.915 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6675.gc7471 5/153 2023-10-16T09:47:52.916 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:18.0.0-6675.gc7471068.el9.x86_64 6/153 2023-10-16T09:47:52.916 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:18.0.0-6675.gc7471068.el9.x86_64 7/153 2023-10-16T09:47:52.916 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:18.0.0-6675.gc7471068.el9.x86_64 8/153 2023-10-16T09:47:52.916 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:18.0.0-6675.gc7471068.el9.x86_64 9/153 2023-10-16T09:47:52.916 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:18.0.0-6675.gc7471068.el9.x86_64 10/153 2023-10-16T09:47:52.916 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:18.0.0-6675.gc7471068.el9.x86_64 11/153 2023-10-16T09:47:52.917 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:18.0.0-6675.gc7471068.el9.x86_64 12/153 2023-10-16T09:47:52.917 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-mirror-2:18.0.0-6675.gc7471068.el9.x86_64 13/153 2023-10-16T09:47:52.917 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:18.0.0-6675.gc7471068.el9.x86_ 14/153 2023-10-16T09:47:52.917 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:18.0.0-6675.gc7471068.el9.x86_64 15/153 2023-10-16T09:47:52.917 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:18.0.0-6675.gc7471068.el9.x86_64 16/153 2023-10-16T09:47:52.918 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:18.0.0-6675.gc7471068.el9.x86_6 17/153 2023-10-16T09:47:52.918 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:18.0.0-6675.gc7471068.el9.x86 18/153 2023-10-16T09:47:52.918 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:18.0.0-6675.gc7471068.el9.x86_64 19/153 2023-10-16T09:47:52.918 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6675.gc7471068.el 20/153 2023-10-16T09:47:52.918 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:18.0.0-6675.gc7471068.el9. 21/153 2023-10-16T09:47:52.919 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:18.0.0-6675.gc7471068.el9.x86_6 22/153 2023-10-16T09:47:52.919 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:18.0.0-6675.gc7471068.el9.x86_64 23/153 2023-10-16T09:47:52.919 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:18.0.0-6675.gc7471068.el9.x86_64 24/153 2023-10-16T09:47:52.919 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:18.0.0-6675.gc7471068.el9.x86_64 25/153 2023-10-16T09:47:52.920 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:18.0.0-6675.gc7471068.el9.x86_64 26/153 2023-10-16T09:47:52.920 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:18.0.0-6675.gc7471068.el9.x86_64 27/153 2023-10-16T09:47:52.920 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:18.0.0-6675.gc7471068.el9.x86_64 28/153 2023-10-16T09:47:52.920 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6675.gc7471068. 29/153 2023-10-16T09:47:52.920 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6675.gc7471068.el9.noa 30/153 2023-10-16T09:47:52.930 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6675.gc7471068.el9.n 31/153 2023-10-16T09:47:52.930 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6675.gc74 32/153 2023-10-16T09:47:52.930 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6675.gc7471068.el 33/153 2023-10-16T09:47:52.931 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6675.gc7471068.el9.noarch 34/153 2023-10-16T09:47:52.931 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6675.gc7471068.e 35/153 2023-10-16T09:47:52.931 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:18.0.0-6675.gc7471068.el9.noarch 36/153 2023-10-16T09:47:52.931 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:18.0.0-6675.gc7471068.el9.noarch 37/153 2023-10-16T09:47:52.931 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-top-2:18.0.0-6675.gc7471068.el9.noarch 38/153 2023-10-16T09:47:52.932 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/153 2023-10-16T09:47:52.932 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/153 2023-10-16T09:47:52.932 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/153 2023-10-16T09:47:52.932 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/153 2023-10-16T09:47:52.932 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/153 2023-10-16T09:47:52.933 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/153 2023-10-16T09:47:52.933 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 45/153 2023-10-16T09:47:52.933 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 46/153 2023-10-16T09:47:52.933 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 47/153 2023-10-16T09:47:52.933 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 48/153 2023-10-16T09:47:52.934 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 49/153 2023-10-16T09:47:52.934 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 50/153 2023-10-16T09:47:52.934 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 51/153 2023-10-16T09:47:52.934 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 52/153 2023-10-16T09:47:52.934 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 53/153 2023-10-16T09:47:52.934 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-13.11-1.el9.x86_64 54/153 2023-10-16T09:47:52.935 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 55/153 2023-10-16T09:47:52.935 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 56/153 2023-10-16T09:47:52.935 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 57/153 2023-10-16T09:47:52.935 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/153 2023-10-16T09:47:52.935 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 59/153 2023-10-16T09:47:52.936 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 60/153 2023-10-16T09:47:52.936 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/153 2023-10-16T09:47:52.936 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/153 2023-10-16T09:47:52.936 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/153 2023-10-16T09:47:52.936 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/153 2023-10-16T09:47:52.936 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/153 2023-10-16T09:47:52.937 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/153 2023-10-16T09:47:52.937 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/153 2023-10-16T09:47:52.937 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 68/153 2023-10-16T09:47:52.937 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 69/153 2023-10-16T09:47:52.937 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 70/153 2023-10-16T09:47:52.938 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/153 2023-10-16T09:47:52.938 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/153 2023-10-16T09:47:52.938 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/153 2023-10-16T09:47:52.938 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/153 2023-10-16T09:47:52.938 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 75/153 2023-10-16T09:47:52.941 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/153 2023-10-16T09:47:52.941 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/153 2023-10-16T09:47:52.941 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 78/153 2023-10-16T09:47:52.941 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 79/153 2023-10-16T09:47:52.941 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 80/153 2023-10-16T09:47:52.942 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ply-3.11-14.el9.noarch 81/153 2023-10-16T09:47:52.942 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 82/153 2023-10-16T09:47:52.942 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 83/153 2023-10-16T09:47:52.942 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-urllib3-1.26.5-3.el9.noarch 84/153 2023-10-16T09:47:52.942 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 85/153 2023-10-16T09:47:52.943 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : automake-1.16.2-8.el9.noarch 86/153 2023-10-16T09:47:52.943 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 87/153 2023-10-16T09:47:52.943 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : jq-1.6-15.el9.x86_64 88/153 2023-10-16T09:47:52.943 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/153 2023-10-16T09:47:52.943 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/153 2023-10-16T09:47:52.944 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/153 2023-10-16T09:47:52.944 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/153 2023-10-16T09:47:52.944 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/153 2023-10-16T09:47:52.944 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/153 2023-10-16T09:47:52.944 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/153 2023-10-16T09:47:52.944 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/153 2023-10-16T09:47:52.945 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/153 2023-10-16T09:47:52.945 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/153 2023-10-16T09:47:52.945 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/153 2023-10-16T09:47:52.945 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/153 2023-10-16T09:47:52.945 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/153 2023-10-16T09:47:52.946 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/153 2023-10-16T09:47:52.946 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/153 2023-10-16T09:47:52.946 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/153 2023-10-16T09:47:52.946 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/153 2023-10-16T09:47:52.946 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/153 2023-10-16T09:47:52.946 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/153 2023-10-16T09:47:52.947 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:2.23.2-1.el9.noarch 108/153 2023-10-16T09:47:52.947 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/153 2023-10-16T09:47:52.947 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/153 2023-10-16T09:47:52.947 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/153 2023-10-16T09:47:52.947 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/153 2023-10-16T09:47:52.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/153 2023-10-16T09:47:52.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/153 2023-10-16T09:47:52.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/153 2023-10-16T09:47:52.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/153 2023-10-16T09:47:52.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/153 2023-10-16T09:47:52.948 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/153 2023-10-16T09:47:52.949 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/153 2023-10-16T09:47:52.949 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/153 2023-10-16T09:47:52.949 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/153 2023-10-16T09:47:52.949 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/153 2023-10-16T09:47:52.949 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/153 2023-10-16T09:47:52.950 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/153 2023-10-16T09:47:52.950 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/153 2023-10-16T09:47:52.950 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/153 2023-10-16T09:47:52.950 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/153 2023-10-16T09:47:52.950 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/153 2023-10-16T09:47:52.950 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/153 2023-10-16T09:47:52.951 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/153 2023-10-16T09:47:52.951 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/153 2023-10-16T09:47:52.951 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/153 2023-10-16T09:47:52.951 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/153 2023-10-16T09:47:52.951 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:18.0.0-6675.gc7471068.el9.x86_64 134/153 2023-10-16T09:47:52.951 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/153 2023-10-16T09:47:52.952 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:18.0.0-6675.gc7471068.el9.x86_64 136/153 2023-10-16T09:47:52.952 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/153 2023-10-16T09:47:52.952 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 138/153 2023-10-16T09:47:52.952 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 139/153 2023-10-16T09:47:52.952 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-1:1.45.4-1.el9.x86_64 140/153 2023-10-16T09:47:52.953 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 141/153 2023-10-16T09:47:52.953 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 142/153 2023-10-16T09:47:52.953 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 143/153 2023-10-16T09:47:52.953 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-team-1:1.45.4-1.el9.x86_64 144/153 2023-10-16T09:47:52.953 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 145/153 2023-10-16T09:47:52.953 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-tui-1:1.45.4-1.el9.x86_64 146/153 2023-10-16T09:47:52.954 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 147/153 2023-10-16T09:47:52.954 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : acl-2.3.1-4.el9.x86_64 148/153 2023-10-16T09:47:52.954 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : acl-2.3.1-3.el9.x86_64 149/153 2023-10-16T09:47:52.954 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 150/153 2023-10-16T09:47:52.954 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-16T09:47:52.955 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 152/153 2023-10-16T09:47:54.982 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-16T09:47:54.982 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:47:54.982 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2023-10-16T09:47:54.983 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-1:1.45.4-1.el9.x86_64 2023-10-16T09:47:54.983 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 2023-10-16T09:47:54.983 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-team-1:1.45.4-1.el9.x86_64 2023-10-16T09:47:54.983 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-tui-1:1.45.4-1.el9.x86_64 2023-10-16T09:47:54.983 INFO:teuthology.orchestra.run.smithi157.stdout: acl-2.3.1-4.el9.x86_64 2023-10-16T09:47:54.984 INFO:teuthology.orchestra.run.smithi157.stdout: libacl-2.3.1-4.el9.x86_64 2023-10-16T09:47:54.984 INFO:teuthology.orchestra.run.smithi157.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-10-16T09:47:54.984 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.984 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.984 INFO:teuthology.orchestra.run.smithi157.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-10-16T09:47:54.984 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-10-16T09:47:54.985 INFO:teuthology.orchestra.run.smithi157.stdout: automake-1.16.2-8.el9.noarch 2023-10-16T09:47:54.985 INFO:teuthology.orchestra.run.smithi157.stdout: bison-3.7.4-5.el9.x86_64 2023-10-16T09:47:54.985 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-16T09:47:54.985 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.985 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.986 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.986 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.986 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:54.986 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.986 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.987 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.987 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:54.987 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:54.987 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:54.987 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:54.988 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:54.988 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.988 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.988 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:54.988 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.988 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.989 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.989 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:54.989 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:54.989 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.989 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top-2:18.0.0-6675.gc7471068.el9.noarch 2023-10-16T09:47:54.990 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-10-16T09:47:54.990 INFO:teuthology.orchestra.run.smithi157.stdout: flex-2.6.4-9.el9.x86_64 2023-10-16T09:47:54.990 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-16T09:47:54.990 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-16T09:47:54.990 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-16T09:47:54.990 INFO:teuthology.orchestra.run.smithi157.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-16T09:47:54.991 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-2.9.9-15.el9.x86_64 2023-10-16T09:47:54.991 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-16T09:47:54.991 INFO:teuthology.orchestra.run.smithi157.stdout: jq-1.6-15.el9.x86_64 2023-10-16T09:47:54.991 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-10-16T09:47:54.991 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-16T09:47:54.992 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-16T09:47:54.992 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.992 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.992 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.992 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-16T09:47:54.992 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-16T09:47:54.993 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-16T09:47:54.993 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-16T09:47:54.993 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-16T09:47:54.993 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.993 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.994 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-16T09:47:54.994 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.994 INFO:teuthology.orchestra.run.smithi157.stdout: libtool-2.4.6-45.el9.x86_64 2023-10-16T09:47:54.994 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-16T09:47:54.994 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-10-16T09:47:54.994 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-10-16T09:47:54.995 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-16T09:47:54.995 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-16T09:47:54.995 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-16T09:47:54.995 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-16T09:47:54.995 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-10-16T09:47:54.995 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-16T09:47:54.996 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-13.11-1.el9.x86_64 2023-10-16T09:47:54.996 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-10-16T09:47:54.996 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-10-16T09:47:54.996 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server-13.11-1.el9.x86_64 2023-10-16T09:47:54.996 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-16T09:47:54.996 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-16T09:47:54.997 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-16T09:47:54.997 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-16T09:47:54.997 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.997 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.997 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:54.998 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-16T09:47:54.998 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-16T09:47:54.998 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-16T09:47:54.998 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-16T09:47:54.998 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-16T09:47:54.999 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-16T09:47:54.999 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:2.23.2-1.el9.noarch 2023-10-16T09:47:54.999 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-16T09:47:54.999 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-16T09:47:54.999 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-16T09:47:54.999 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-16T09:47:55.000 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-16T09:47:55.000 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-16T09:47:55.000 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-16T09:47:55.000 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-16T09:47:55.000 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-16T09:47:55.001 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-16T09:47:55.001 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-16T09:47:55.001 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-16T09:47:55.001 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-16T09:47:55.001 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-16T09:47:55.002 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-16T09:47:55.002 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-16T09:47:55.002 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-16T09:47:55.002 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-16T09:47:55.002 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-16T09:47:55.003 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-16T09:47:55.003 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply-3.11-14.el9.noarch 2023-10-16T09:47:55.003 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-16T09:47:55.003 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-16T09:47:55.003 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-16T09:47:55.004 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-16T09:47:55.004 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-16T09:47:55.004 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-16T09:47:55.004 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-16T09:47:55.004 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:55.005 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:55.005 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-16T09:47:55.005 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-16T09:47:55.005 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-16T09:47:55.005 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:55.006 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-16T09:47:55.006 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-16T09:47:55.006 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-16T09:47:55.006 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-16T09:47:55.006 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-16T09:47:55.007 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-16T09:47:55.007 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3-1.26.5-3.el9.noarch 2023-10-16T09:47:55.007 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-16T09:47:55.007 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-16T09:47:55.007 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-16T09:47:55.007 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-16T09:47:55.008 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:55.008 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:55.008 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:18.0.0-6675.gc7471068.el9.x86_64 2023-10-16T09:47:55.008 INFO:teuthology.orchestra.run.smithi157.stdout: re2-1:20211101-3.el9.x86_64 2023-10-16T09:47:55.008 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-16T09:47:55.009 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-16T09:47:55.009 INFO:teuthology.orchestra.run.smithi157.stdout: uuid-1.6.2-55.el9.x86_64 2023-10-16T09:47:55.009 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-10-16T09:47:55.009 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-16T09:47:55.010 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-10-16T09:47:55.010 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:47:55.010 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-10-16T09:47:55.526 DEBUG:teuthology.parallel:result is None 2023-10-16T09:47:55.527 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c7471068426370a3754c2250e7729280233d552b 2023-10-16T09:47:55.745 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-16T09:47:55.820 INFO:teuthology.orchestra.run.smithi084.stdout:18.0.0-6675.gc7471068.el9 2023-10-16T09:47:55.821 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6675.gc7471068.el9 2023-10-16T09:47:55.821 INFO:teuthology.task.install:The correct ceph version 18.0.0-6675.gc7471068 is installed. 2023-10-16T09:47:55.821 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c7471068426370a3754c2250e7729280233d552b 2023-10-16T09:47:55.954 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-16T09:47:56.014 INFO:teuthology.orchestra.run.smithi157.stdout:18.0.0-6675.gc7471068.el9 2023-10-16T09:47:56.015 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6675.gc7471068.el9 2023-10-16T09:47:56.015 INFO:teuthology.task.install:The correct ceph version 18.0.0-6675.gc7471068 is installed. 2023-10-16T09:47:56.015 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-16T09:47:56.016 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:47:56.016 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-16T09:47:56.119 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:47:56.119 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-16T09:47:56.201 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-16T09:47:56.201 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:47:56.201 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/daemon-helper 2023-10-16T09:47:56.304 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-16T09:47:56.446 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:47:56.447 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2023-10-16T09:47:56.523 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-16T09:47:56.641 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-16T09:47:56.641 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:47:56.642 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-16T09:47:56.745 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-16T09:47:56.887 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:47:56.888 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-16T09:47:56.977 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-16T09:47:57.092 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-16T09:47:57.092 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:47:57.093 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/stdin-killer 2023-10-16T09:47:57.185 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-16T09:47:57.268 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:47:57.269 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2023-10-16T09:47:57.352 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-16T09:47:57.467 INFO:teuthology.run_tasks:Running task ceph... 2023-10-16T09:47:57.527 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-16T09:47:57.528 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 777 /var/log/ceph 2023-10-16T09:47:57.534 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /var/log/ceph 2023-10-16T09:47:57.607 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-16T09:47:57.608 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-16T09:47:57.614 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-16T09:47:57.685 INFO:tasks.ceph:Creating extra log directories... 2023-10-16T09:47:57.686 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-16T09:47:57.691 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-16T09:47:57.771 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-16T09:47:57.771 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-16T09:47:57.772 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-16T09:47:57.772 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:47:57.772 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-16T09:47:57.821 DEBUG:teuthology.orchestra.run.smithi084:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-16T09:47:58.082 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-16T09:47:58.168 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:47:58.168 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-16T09:47:58.214 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-16T09:47:58.430 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-16T09:47:58.510 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-16T09:47:58.511 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-16T09:47:58.511 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428968', 'branch': 'wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7428968', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'c7471068426370a3754c2250e7729280233d552b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c7471068426370a3754c2250e7729280233d552b'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'mgr-snap_schedule-support-subvol-and-group-arguments', 'sha1': '7b3d12460841f3cbe9400ba1b68e3e20957ff531'}}, 'owner': 'mchangir', 'priority': 25, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 127, 'sha1': 'c7471068426370a3754c2250e7729280233d552b', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'mgr-snap_schedule-support-subvol-and-group-arguments', 'suite_path': '/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/mchangir/ceph.git', 'suite_sha1': '7b3d12460841f3cbe9400ba1b68e3e20957ff531', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi084.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJXxfKL1uHMlQo74v+R3GoiPXecQzV8P30woGckwQ84qfJ6EEXNgTDKH3hKwuDEEPsHSPFbQFO1y9ERH4QRY11M=', 'smithi157.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE2h+CtCtnk9zYAofCvqysHgWMJhS6NpRlokOe2x0VCCmhnx8iudgEX75GybhID5gv11OiYGzIRjlOPKuwng4PM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c7471068426370a3754c2250e7729280233d552b'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'c7471068426370a3754c2250e7729280233d552b', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '8cdab074dcca9a68965bc5a50e9c30b691949723', 'timestamp': '2023-10-16_09:20:41', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-16T09:47:58.512 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-16T09:47:58.521 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-16T09:47:58.566 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m0777 -- /var/run/ceph 2023-10-16T09:47:58.572 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/run/ceph 2023-10-16T09:47:58.689 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:47:58.689 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/scratch_devs of=/dev/stdout 2023-10-16T09:47:58.742 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-16T09:47:58.743 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_1 2023-10-16T09:47:58.793 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-16T09:47:58.793 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T09:47:58.794 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 633 Links: 1 2023-10-16T09:47:58.794 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T09:47:58.794 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T09:47:58.794 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2023-10-16 09:47:41.681564307 +0000 2023-10-16T09:47:58.795 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2023-10-16 09:45:20.812716419 +0000 2023-10-16T09:47:58.795 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2023-10-16 09:45:20.812716419 +0000 2023-10-16T09:47:58.795 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2023-10-16 09:45:20.812716419 +0000 2023-10-16T09:47:58.795 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-16T09:47:58.880 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2023-10-16T09:47:58.881 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2023-10-16T09:47:58.881 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000732283 s, 699 kB/s 2023-10-16T09:47:58.884 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-16T09:47:58.940 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_2 2023-10-16T09:47:58.990 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-16T09:47:58.990 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T09:47:58.990 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 653 Links: 1 2023-10-16T09:47:58.990 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T09:47:58.991 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T09:47:58.991 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2023-10-16 09:47:41.682564290 +0000 2023-10-16T09:47:58.991 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2023-10-16 09:45:21.825699031 +0000 2023-10-16T09:47:58.991 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2023-10-16 09:45:21.825699031 +0000 2023-10-16T09:47:58.991 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2023-10-16 09:45:21.825699031 +0000 2023-10-16T09:47:58.992 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-16T09:47:59.080 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2023-10-16T09:47:59.080 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2023-10-16T09:47:59.081 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000443009 s, 1.2 MB/s 2023-10-16T09:47:59.084 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-16T09:47:59.140 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_3 2023-10-16T09:47:59.187 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-16T09:47:59.188 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T09:47:59.188 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 669 Links: 1 2023-10-16T09:47:59.188 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T09:47:59.188 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T09:47:59.188 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2023-10-16 09:47:41.683564273 +0000 2023-10-16T09:47:59.189 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2023-10-16 09:45:22.839681627 +0000 2023-10-16T09:47:59.189 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2023-10-16 09:45:22.839681627 +0000 2023-10-16T09:47:59.189 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2023-10-16 09:45:22.839681627 +0000 2023-10-16T09:47:59.189 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-16T09:47:59.271 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2023-10-16T09:47:59.272 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2023-10-16T09:47:59.272 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000304852 s, 1.7 MB/s 2023-10-16T09:47:59.276 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-16T09:47:59.332 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_4 2023-10-16T09:47:59.383 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-16T09:47:59.384 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T09:47:59.384 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 691 Links: 1 2023-10-16T09:47:59.384 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T09:47:59.385 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T09:47:59.385 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2023-10-16 09:47:41.684564256 +0000 2023-10-16T09:47:59.385 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2023-10-16 09:45:23.845664359 +0000 2023-10-16T09:47:59.385 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2023-10-16 09:45:23.845664359 +0000 2023-10-16T09:47:59.385 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2023-10-16 09:45:23.845664359 +0000 2023-10-16T09:47:59.386 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-16T09:47:59.476 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2023-10-16T09:47:59.476 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2023-10-16T09:47:59.476 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000332319 s, 1.5 MB/s 2023-10-16T09:47:59.479 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-16T09:47:59.533 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T09:47:59.534 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:47:59.534 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2023-10-16T09:47:59.595 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-16T09:47:59.595 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2023-10-16T09:47:59.648 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-16T09:47:59.649 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T09:47:59.649 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 633 Links: 1 2023-10-16T09:47:59.649 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T09:47:59.649 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T09:47:59.650 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-10-16 09:47:48.792011788 +0000 2023-10-16T09:47:59.650 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-10-16 09:45:20.785192310 +0000 2023-10-16T09:47:59.650 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-10-16 09:45:20.786192292 +0000 2023-10-16T09:47:59.650 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2023-10-16 09:45:20.785192310 +0000 2023-10-16T09:47:59.651 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-16T09:47:59.740 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-10-16T09:47:59.740 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-10-16T09:47:59.740 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000291408 s, 1.8 MB/s 2023-10-16T09:47:59.743 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-16T09:47:59.803 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2023-10-16T09:47:59.852 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-16T09:47:59.852 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T09:47:59.852 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 652 Links: 1 2023-10-16T09:47:59.852 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T09:47:59.853 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T09:47:59.853 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-10-16 09:47:48.792011788 +0000 2023-10-16T09:47:59.853 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-10-16 09:45:21.819173587 +0000 2023-10-16T09:47:59.854 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-10-16 09:45:21.819173587 +0000 2023-10-16T09:47:59.854 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2023-10-16 09:45:21.819173587 +0000 2023-10-16T09:47:59.854 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-16T09:47:59.940 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-10-16T09:47:59.940 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-10-16T09:47:59.940 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000251805 s, 2.0 MB/s 2023-10-16T09:47:59.944 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-16T09:47:59.999 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2023-10-16T09:48:00.051 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-16T09:48:00.051 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T09:48:00.051 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 669 Links: 1 2023-10-16T09:48:00.051 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T09:48:00.052 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T09:48:00.052 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-10-16 09:47:48.794011752 +0000 2023-10-16T09:48:00.052 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-10-16 09:45:22.839155117 +0000 2023-10-16T09:48:00.052 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-10-16 09:45:22.839155117 +0000 2023-10-16T09:48:00.052 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2023-10-16 09:45:22.839155117 +0000 2023-10-16T09:48:00.053 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-16T09:48:00.142 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-10-16T09:48:00.142 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-10-16T09:48:00.143 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000253637 s, 2.0 MB/s 2023-10-16T09:48:00.146 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-16T09:48:00.204 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2023-10-16T09:48:00.253 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-16T09:48:00.253 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T09:48:00.253 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 689 Links: 1 2023-10-16T09:48:00.253 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T09:48:00.254 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T09:48:00.254 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-10-16 09:47:48.794011752 +0000 2023-10-16T09:48:00.254 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-10-16 09:45:23.853136757 +0000 2023-10-16T09:48:00.254 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-10-16 09:45:23.853136757 +0000 2023-10-16T09:48:00.254 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2023-10-16 09:45:23.853136757 +0000 2023-10-16T09:48:00.255 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-16T09:48:00.329 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-10-16T09:48:00.330 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-10-16T09:48:00.330 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000209335 s, 2.4 MB/s 2023-10-16T09:48:00.332 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-16T09:48:00.386 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T09:48:00.387 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi084.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-16T09:48:00.387 INFO:tasks.ceph:Generating config... 2023-10-16T09:48:00.388 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-16T09:48:00.389 INFO:tasks.ceph:[client] debug client = 20 2023-10-16T09:48:00.389 INFO:tasks.ceph:[client] debug ms = 1 2023-10-16T09:48:00.389 INFO:tasks.ceph:[client] pid file = 2023-10-16T09:48:00.389 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-16T09:48:00.390 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-16T09:48:00.390 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-16T09:48:00.390 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-16T09:48:00.391 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-10-16T09:48:00.391 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-16T09:48:00.391 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-16T09:48:00.391 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-16T09:48:00.392 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-16T09:48:00.392 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-16T09:48:00.392 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-16T09:48:00.392 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-16T09:48:00.393 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-16T09:48:00.393 INFO:tasks.ceph:[mgr] debug client = 20 2023-10-16T09:48:00.393 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-10-16T09:48:00.393 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-16T09:48:00.394 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-16T09:48:00.394 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-16T09:48:00.394 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-16T09:48:00.394 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-16T09:48:00.395 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-16T09:48:00.395 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-16T09:48:00.395 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-16T09:48:00.395 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-16T09:48:00.396 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-16T09:48:00.396 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-16T09:48:00.396 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-16T09:48:00.396 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-16T09:48:00.397 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-16T09:48:00.397 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-16T09:48:00.397 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-16T09:48:00.397 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-16T09:48:00.398 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-16T09:48:00.398 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-16T09:48:00.398 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-16T09:48:00.399 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-16T09:48:00.399 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-16T09:48:00.399 INFO:tasks.ceph:Setting up mon.a... 2023-10-16T09:48:00.400 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-16T09:48:00.550 INFO:teuthology.orchestra.run.smithi084.stdout:creating /etc/ceph/ceph.keyring 2023-10-16T09:48:00.558 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-16T09:48:00.696 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-16T09:48:00.784 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.84'), ('mon.b', '172.21.15.157'), ('mon.c', '[v2:172.21.15.157:3301,v1:172.21.15.157:6790]')] 2023-10-16T09:48:00.785 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.84,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-16T09:48:00.786 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:48:00.786 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-16T09:48:00.836 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.84 --add b 172.21.15.157 --addv c '[v2:172.21.15.157:3301,v1:172.21.15.157:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:48:00.992 INFO:teuthology.orchestra.run.smithi084.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:48:00.993 INFO:teuthology.orchestra.run.smithi084.stdout:monmaptool: generated fsid 63fb4d68-f881-4694-b880-d10369105c0e 2023-10-16T09:48:00.993 INFO:teuthology.orchestra.run.smithi084.stdout:setting min_mon_release = pacific 2023-10-16T09:48:00.993 INFO:teuthology.orchestra.run.smithi084.stdout:epoch 0 2023-10-16T09:48:00.993 INFO:teuthology.orchestra.run.smithi084.stdout:fsid 63fb4d68-f881-4694-b880-d10369105c0e 2023-10-16T09:48:00.994 INFO:teuthology.orchestra.run.smithi084.stdout:last_changed 2023-10-16T09:48:00.992065+0000 2023-10-16T09:48:00.994 INFO:teuthology.orchestra.run.smithi084.stdout:created 2023-10-16T09:48:00.992065+0000 2023-10-16T09:48:00.994 INFO:teuthology.orchestra.run.smithi084.stdout:min_mon_release 16 (pacific) 2023-10-16T09:48:00.994 INFO:teuthology.orchestra.run.smithi084.stdout:election_strategy: 1 2023-10-16T09:48:00.995 INFO:teuthology.orchestra.run.smithi084.stdout:0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2023-10-16T09:48:00.995 INFO:teuthology.orchestra.run.smithi084.stdout:1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.b 2023-10-16T09:48:00.995 INFO:teuthology.orchestra.run.smithi084.stdout:2: [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] mon.c 2023-10-16T09:48:00.995 INFO:teuthology.orchestra.run.smithi084.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-16T09:48:00.996 DEBUG:teuthology.orchestra.run.smithi084:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-16T09:48:01.047 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 63fb4d68-f881-4694-b880-d10369105c0e... 2023-10-16T09:48:01.048 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-16T09:48:01.053 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-16T09:48:01.210 INFO:teuthology.orchestra.run.smithi084.stdout:[global] 2023-10-16T09:48:01.210 INFO:teuthology.orchestra.run.smithi084.stdout: chdir = "" 2023-10-16T09:48:01.211 INFO:teuthology.orchestra.run.smithi084.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-16T09:48:01.211 INFO:teuthology.orchestra.run.smithi084.stdout: auth supported = cephx 2023-10-16T09:48:01.211 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.211 INFO:teuthology.orchestra.run.smithi084.stdout: filestore xattr use omap = true 2023-10-16T09:48:01.211 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.212 INFO:teuthology.orchestra.run.smithi084.stdout: mon clock drift allowed = 1.000 2023-10-16T09:48:01.212 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.212 INFO:teuthology.orchestra.run.smithi084.stdout: osd crush chooseleaf type = 0 2023-10-16T09:48:01.212 INFO:teuthology.orchestra.run.smithi084.stdout: auth debug = true 2023-10-16T09:48:01.212 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.213 INFO:teuthology.orchestra.run.smithi084.stdout: ms die on old message = true 2023-10-16T09:48:01.213 INFO:teuthology.orchestra.run.smithi084.stdout: ms die on bug = true 2023-10-16T09:48:01.213 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.213 INFO:teuthology.orchestra.run.smithi084.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-16T09:48:01.213 INFO:teuthology.orchestra.run.smithi084.stdout: mon pg warn max object skew = 0 2023-10-16T09:48:01.214 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.214 INFO:teuthology.orchestra.run.smithi084.stdout: # disable pg_autoscaler by default for new pools 2023-10-16T09:48:01.214 INFO:teuthology.orchestra.run.smithi084.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-16T09:48:01.214 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.214 INFO:teuthology.orchestra.run.smithi084.stdout: osd pool default size = 2 2023-10-16T09:48:01.215 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.215 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd allow primary affinity = true 2023-10-16T09:48:01.215 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd allow pg remap = true 2023-10-16T09:48:01.215 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on legacy crush tunables = false 2023-10-16T09:48:01.215 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on crush straw calc version zero = false 2023-10-16T09:48:01.216 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on no sortbitwise = false 2023-10-16T09:48:01.216 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on osd down out interval zero = false 2023-10-16T09:48:01.216 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on too few osds = false 2023-10-16T09:48:01.216 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-16T09:48:01.217 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-16T09:48:01.217 INFO:teuthology.orchestra.run.smithi084.stdout: mon_allow_pool_size_one = true 2023-10-16T09:48:01.217 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.217 INFO:teuthology.orchestra.run.smithi084.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-16T09:48:01.217 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.217 INFO:teuthology.orchestra.run.smithi084.stdout: osd default data pool replay window = 5 2023-10-16T09:48:01.218 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.218 INFO:teuthology.orchestra.run.smithi084.stdout: mon allow pool delete = true 2023-10-16T09:48:01.218 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.218 INFO:teuthology.orchestra.run.smithi084.stdout: mon cluster log file level = debug 2023-10-16T09:48:01.219 INFO:teuthology.orchestra.run.smithi084.stdout: debug asserts on shutdown = true 2023-10-16T09:48:01.219 INFO:teuthology.orchestra.run.smithi084.stdout: mon health detail to clog = false 2023-10-16T09:48:01.219 INFO:teuthology.orchestra.run.smithi084.stdout: mon host = "172.21.15.84,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2023-10-16T09:48:01.219 INFO:teuthology.orchestra.run.smithi084.stdout: ms die on skipped message = False 2023-10-16T09:48:01.219 INFO:teuthology.orchestra.run.smithi084.stdout: fsid = 63fb4d68-f881-4694-b880-d10369105c0e 2023-10-16T09:48:01.219 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.220 INFO:teuthology.orchestra.run.smithi084.stdout:[osd] 2023-10-16T09:48:01.220 INFO:teuthology.orchestra.run.smithi084.stdout: osd journal size = 100 2023-10-16T09:48:01.220 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.220 INFO:teuthology.orchestra.run.smithi084.stdout: osd scrub load threshold = 5.0 2023-10-16T09:48:01.220 INFO:teuthology.orchestra.run.smithi084.stdout: osd scrub max interval = 600 2023-10-16T09:48:01.220 INFO:teuthology.orchestra.run.smithi084.stdout: osd mclock profile = high_recovery_ops 2023-10-16T09:48:01.221 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.221 INFO:teuthology.orchestra.run.smithi084.stdout: osd recover clone overlap = true 2023-10-16T09:48:01.221 INFO:teuthology.orchestra.run.smithi084.stdout: osd recovery max chunk = 1048576 2023-10-16T09:48:01.221 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.221 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug shutdown = true 2023-10-16T09:48:01.222 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug op order = true 2023-10-16T09:48:01.222 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug verify stray on activate = true 2023-10-16T09:48:01.222 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.222 INFO:teuthology.orchestra.run.smithi084.stdout: osd open classes on start = true 2023-10-16T09:48:01.222 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug pg log writeout = true 2023-10-16T09:48:01.222 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.223 INFO:teuthology.orchestra.run.smithi084.stdout: osd deep scrub update digest min age = 30 2023-10-16T09:48:01.223 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.223 INFO:teuthology.orchestra.run.smithi084.stdout: osd map max advance = 10 2023-10-16T09:48:01.223 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.223 INFO:teuthology.orchestra.run.smithi084.stdout: journal zero on create = true 2023-10-16T09:48:01.224 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.224 INFO:teuthology.orchestra.run.smithi084.stdout: filestore ondisk finisher threads = 3 2023-10-16T09:48:01.224 INFO:teuthology.orchestra.run.smithi084.stdout: filestore apply finisher threads = 3 2023-10-16T09:48:01.224 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.224 INFO:teuthology.orchestra.run.smithi084.stdout: bdev debug aio = true 2023-10-16T09:48:01.224 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug misdirected ops = true 2023-10-16T09:48:01.225 INFO:teuthology.orchestra.run.smithi084.stdout: bdev async discard = True 2023-10-16T09:48:01.225 INFO:teuthology.orchestra.run.smithi084.stdout: bdev enable discard = True 2023-10-16T09:48:01.225 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore allocator = bitmap 2023-10-16T09:48:01.225 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore block size = 96636764160 2023-10-16T09:48:01.225 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore fsck on mount = True 2023-10-16T09:48:01.226 INFO:teuthology.orchestra.run.smithi084.stdout: debug bluefs = 1/20 2023-10-16T09:48:01.226 INFO:teuthology.orchestra.run.smithi084.stdout: debug bluestore = 1/20 2023-10-16T09:48:01.226 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2023-10-16T09:48:01.226 INFO:teuthology.orchestra.run.smithi084.stdout: debug osd = 20 2023-10-16T09:48:01.226 INFO:teuthology.orchestra.run.smithi084.stdout: debug rocksdb = 4/10 2023-10-16T09:48:01.226 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd backfillfull_ratio = 0.85 2023-10-16T09:48:01.227 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd full ratio = 0.9 2023-10-16T09:48:01.227 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd nearfull ratio = 0.8 2023-10-16T09:48:01.227 INFO:teuthology.orchestra.run.smithi084.stdout: osd failsafe full ratio = 0.95 2023-10-16T09:48:01.227 INFO:teuthology.orchestra.run.smithi084.stdout: osd objectstore = bluestore 2023-10-16T09:48:01.227 INFO:teuthology.orchestra.run.smithi084.stdout: osd op complaint time = 180 2023-10-16T09:48:01.228 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.228 INFO:teuthology.orchestra.run.smithi084.stdout:[mgr] 2023-10-16T09:48:01.228 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2023-10-16T09:48:01.228 INFO:teuthology.orchestra.run.smithi084.stdout: debug mgr = 20 2023-10-16T09:48:01.228 INFO:teuthology.orchestra.run.smithi084.stdout: debug mon = 20 2023-10-16T09:48:01.228 INFO:teuthology.orchestra.run.smithi084.stdout: debug auth = 20 2023-10-16T09:48:01.229 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min pgs per osd = 4 2023-10-16T09:48:01.229 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min bytes per osd = 10 2023-10-16T09:48:01.229 INFO:teuthology.orchestra.run.smithi084.stdout: mgr/telemetry/nag = false 2023-10-16T09:48:01.229 INFO:teuthology.orchestra.run.smithi084.stdout: debug client = 20 2023-10-16T09:48:01.229 INFO:teuthology.orchestra.run.smithi084.stdout: debug finisher = 20 2023-10-16T09:48:01.229 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.230 INFO:teuthology.orchestra.run.smithi084.stdout:[mon] 2023-10-16T09:48:01.230 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2023-10-16T09:48:01.230 INFO:teuthology.orchestra.run.smithi084.stdout: debug mon = 20 2023-10-16T09:48:01.230 INFO:teuthology.orchestra.run.smithi084.stdout: debug paxos = 20 2023-10-16T09:48:01.230 INFO:teuthology.orchestra.run.smithi084.stdout: debug auth = 20 2023-10-16T09:48:01.231 INFO:teuthology.orchestra.run.smithi084.stdout: mon data avail warn = 5 2023-10-16T09:48:01.231 INFO:teuthology.orchestra.run.smithi084.stdout: mon mgr mkfs grace = 240 2023-10-16T09:48:01.231 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min pgs per osd = 4 2023-10-16T09:48:01.231 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd reporter subtree level = osd 2023-10-16T09:48:01.231 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd prime pg temp = true 2023-10-16T09:48:01.232 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min bytes per osd = 10 2023-10-16T09:48:01.232 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.232 INFO:teuthology.orchestra.run.smithi084.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-16T09:48:01.232 INFO:teuthology.orchestra.run.smithi084.stdout: auth mon ticket ttl = 660 # 11m 2023-10-16T09:48:01.232 INFO:teuthology.orchestra.run.smithi084.stdout: auth service ticket ttl = 240 # 4m 2023-10-16T09:48:01.233 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.233 INFO:teuthology.orchestra.run.smithi084.stdout: # don't complain about insecure global_id in the test suite 2023-10-16T09:48:01.233 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-16T09:48:01.233 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-16T09:48:01.233 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.234 INFO:teuthology.orchestra.run.smithi084.stdout: # 1m isn't quite enough 2023-10-16T09:48:01.234 INFO:teuthology.orchestra.run.smithi084.stdout: mon_down_mkfs_grace = 2m 2023-10-16T09:48:01.234 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.234 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_filestore_osds = false 2023-10-16T09:48:01.234 INFO:teuthology.orchestra.run.smithi084.stdout: mon op complaint time = 120 2023-10-16T09:48:01.235 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:01.235 INFO:teuthology.orchestra.run.smithi084.stdout:[client] 2023-10-16T09:48:01.235 INFO:teuthology.orchestra.run.smithi084.stdout: rgw cache enabled = true 2023-10-16T09:48:01.235 INFO:teuthology.orchestra.run.smithi084.stdout: rgw enable ops log = true 2023-10-16T09:48:01.235 INFO:teuthology.orchestra.run.smithi084.stdout: rgw enable usage log = true 2023-10-16T09:48:01.236 INFO:teuthology.orchestra.run.smithi084.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-16T09:48:01.236 INFO:teuthology.orchestra.run.smithi084.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-16T09:48:01.236 INFO:teuthology.orchestra.run.smithi084.stdout: client mount timeout = 600 2023-10-16T09:48:01.236 INFO:teuthology.orchestra.run.smithi084.stdout: debug client = 20 2023-10-16T09:48:01.236 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2023-10-16T09:48:01.237 INFO:teuthology.orchestra.run.smithi084.stdout: pid file = "" 2023-10-16T09:48:01.237 INFO:teuthology.orchestra.run.smithi084.stdout: rados mon op timeout = 900 2023-10-16T09:48:01.237 INFO:teuthology.orchestra.run.smithi084.stdout: rados osd op timeout = 900 2023-10-16T09:48:01.237 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.a] 2023-10-16T09:48:01.237 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.b] 2023-10-16T09:48:01.237 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.c] 2023-10-16T09:48:01.238 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.a] 2023-10-16T09:48:01.238 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.c] 2023-10-16T09:48:01.238 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.b] 2023-10-16T09:48:01.238 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.d] 2023-10-16T09:48:01.239 INFO:teuthology.orchestra.run.smithi084.stdout:[mds] 2023-10-16T09:48:01.239 INFO:teuthology.orchestra.run.smithi084.stdout: debug mds = 20 2023-10-16T09:48:01.239 INFO:teuthology.orchestra.run.smithi084.stdout: debug mds balancer = 20 2023-10-16T09:48:01.239 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2023-10-16T09:48:01.239 INFO:teuthology.orchestra.run.smithi084.stdout: mds debug frag = True 2023-10-16T09:48:01.240 INFO:teuthology.orchestra.run.smithi084.stdout: mds debug scatterstat = True 2023-10-16T09:48:01.240 INFO:teuthology.orchestra.run.smithi084.stdout: mds op complaint time = 180 2023-10-16T09:48:01.240 INFO:teuthology.orchestra.run.smithi084.stdout: mds verify scatter = True 2023-10-16T09:48:01.240 INFO:teuthology.orchestra.run.smithi084.stdout: osd op complaint time = 180 2023-10-16T09:48:01.240 INFO:teuthology.orchestra.run.smithi084.stdout: rados mon op timeout = 900 2023-10-16T09:48:01.241 INFO:teuthology.orchestra.run.smithi084.stdout: rados osd op timeout = 900 2023-10-16T09:48:01.241 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2023-10-16T09:48:01.241 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2023-10-16T09:48:01.242 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-16T09:48:01.242 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2023-10-16T09:48:01.242 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.242 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2023-10-16T09:48:01.242 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.243 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2023-10-16T09:48:01.243 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.243 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2023-10-16T09:48:01.243 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2023-10-16T09:48:01.243 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.243 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2023-10-16T09:48:01.244 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2023-10-16T09:48:01.244 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.244 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-16T09:48:01.244 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2023-10-16T09:48:01.244 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.245 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2023-10-16T09:48:01.245 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-16T09:48:01.245 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.245 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2023-10-16T09:48:01.245 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.245 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2023-10-16T09:48:01.246 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2023-10-16T09:48:01.246 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2023-10-16T09:48:01.246 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2023-10-16T09:48:01.246 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2023-10-16T09:48:01.246 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2023-10-16T09:48:01.247 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2023-10-16T09:48:01.247 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-16T09:48:01.247 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-16T09:48:01.247 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2023-10-16T09:48:01.247 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.247 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-16T09:48:01.248 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.248 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2023-10-16T09:48:01.248 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.248 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2023-10-16T09:48:01.248 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.249 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2023-10-16T09:48:01.249 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2023-10-16T09:48:01.249 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2023-10-16T09:48:01.249 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.84,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2023-10-16T09:48:01.249 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on skipped message = False 2023-10-16T09:48:01.249 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = 63fb4d68-f881-4694-b880-d10369105c0e 2023-10-16T09:48:01.250 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.250 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2023-10-16T09:48:01.250 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2023-10-16T09:48:01.250 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.250 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2023-10-16T09:48:01.250 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2023-10-16T09:48:01.251 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2023-10-16T09:48:01.251 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.251 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2023-10-16T09:48:01.251 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2023-10-16T09:48:01.251 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.251 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2023-10-16T09:48:01.252 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2023-10-16T09:48:01.252 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2023-10-16T09:48:01.252 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.252 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2023-10-16T09:48:01.252 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2023-10-16T09:48:01.253 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.253 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2023-10-16T09:48:01.253 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.253 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2023-10-16T09:48:01.253 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.254 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2023-10-16T09:48:01.254 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.254 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2023-10-16T09:48:01.254 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2023-10-16T09:48:01.254 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.254 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2023-10-16T09:48:01.255 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2023-10-16T09:48:01.255 INFO:teuthology.orchestra.run.smithi157.stdout: bdev async discard = True 2023-10-16T09:48:01.255 INFO:teuthology.orchestra.run.smithi157.stdout: bdev enable discard = True 2023-10-16T09:48:01.255 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore allocator = bitmap 2023-10-16T09:48:01.255 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2023-10-16T09:48:01.256 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2023-10-16T09:48:01.256 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 1/20 2023-10-16T09:48:01.256 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 1/20 2023-10-16T09:48:01.256 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-10-16T09:48:01.256 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2023-10-16T09:48:01.257 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 4/10 2023-10-16T09:48:01.257 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2023-10-16T09:48:01.257 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2023-10-16T09:48:01.257 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2023-10-16T09:48:01.257 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2023-10-16T09:48:01.258 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2023-10-16T09:48:01.258 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2023-10-16T09:48:01.258 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.258 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2023-10-16T09:48:01.258 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-10-16T09:48:01.259 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2023-10-16T09:48:01.259 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2023-10-16T09:48:01.259 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2023-10-16T09:48:01.259 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2023-10-16T09:48:01.259 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2023-10-16T09:48:01.259 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2023-10-16T09:48:01.260 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2023-10-16T09:48:01.260 INFO:teuthology.orchestra.run.smithi157.stdout: debug finisher = 20 2023-10-16T09:48:01.260 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.260 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2023-10-16T09:48:01.260 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-10-16T09:48:01.260 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2023-10-16T09:48:01.261 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2023-10-16T09:48:01.261 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2023-10-16T09:48:01.261 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2023-10-16T09:48:01.261 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2023-10-16T09:48:01.261 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2023-10-16T09:48:01.261 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2023-10-16T09:48:01.262 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2023-10-16T09:48:01.262 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2023-10-16T09:48:01.262 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.262 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-16T09:48:01.262 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2023-10-16T09:48:01.263 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2023-10-16T09:48:01.263 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.263 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2023-10-16T09:48:01.263 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-16T09:48:01.263 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-16T09:48:01.263 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.264 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2023-10-16T09:48:01.264 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2023-10-16T09:48:01.264 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.264 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2023-10-16T09:48:01.264 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2023-10-16T09:48:01.264 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-16T09:48:01.265 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2023-10-16T09:48:01.265 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2023-10-16T09:48:01.265 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2023-10-16T09:48:01.265 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2023-10-16T09:48:01.265 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-16T09:48:01.266 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-16T09:48:01.266 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2023-10-16T09:48:01.266 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2023-10-16T09:48:01.266 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-10-16T09:48:01.266 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = "" 2023-10-16T09:48:01.267 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2023-10-16T09:48:01.267 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2023-10-16T09:48:01.267 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2023-10-16T09:48:01.267 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2023-10-16T09:48:01.267 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2023-10-16T09:48:01.268 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2023-10-16T09:48:01.268 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2023-10-16T09:48:01.268 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2023-10-16T09:48:01.268 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2023-10-16T09:48:01.268 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2023-10-16T09:48:01.268 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2023-10-16T09:48:01.269 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds balancer = 20 2023-10-16T09:48:01.269 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-10-16T09:48:01.269 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2023-10-16T09:48:01.269 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = True 2023-10-16T09:48:01.269 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2023-10-16T09:48:01.270 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = True 2023-10-16T09:48:01.270 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2023-10-16T09:48:01.270 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2023-10-16T09:48:01.270 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2023-10-16T09:48:01.278 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-16T09:48:01.278 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-16T09:48:01.432 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-16T09:48:01.433 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:48:01.433 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-16T09:48:01.484 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:48:01.484 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-16T09:48:01.582 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T09:48:01.583 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:48:01.583 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-16T09:48:01.583 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-16T09:48:01.732 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:48:01.732 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:48:01.829 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi157.front.sepia.ceph.com 2023-10-16T09:48:01.830 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:48:01.830 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-16T09:48:01.830 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-16T09:48:01.994 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:48:01.994 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:48:02.090 INFO:tasks.ceph:Setting up mon nodes... 2023-10-16T09:48:02.091 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-16T09:48:02.091 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-16T09:48:02.282 INFO:teuthology.orchestra.run.smithi084.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-16T09:48:02.292 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-16T09:48:02.494 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-16T09:48:02.505 INFO:tasks.ceph:Setting up mds nodes... 2023-10-16T09:48:02.506 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-16T09:48:02.695 INFO:teuthology.orchestra.run.smithi084.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-16T09:48:02.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-16T09:48:02.798 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-16T09:48:02.977 INFO:teuthology.orchestra.run.smithi084.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-16T09:48:02.986 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-16T09:48:03.077 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-16T09:48:03.272 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-16T09:48:03.281 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-16T09:48:03.381 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-16T09:48:03.571 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-16T09:48:03.578 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-16T09:48:03.661 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-16T09:48:03.662 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-16T09:48:03.790 INFO:teuthology.orchestra.run.smithi084.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-16T09:48:03.835 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-16T09:48:03.961 INFO:teuthology.orchestra.run.smithi084.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-16T09:48:04.016 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-16T09:48:04.129 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-16T09:48:04.172 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-16T09:48:04.288 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-16T09:48:04.335 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-16T09:48:04.335 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi084.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-16T09:48:04.336 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-16T09:48:04.437 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T09:48:04.437 INFO:tasks.ceph:role: osd.0 2023-10-16T09:48:04.438 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T09:48:04.438 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-16T09:48:04.535 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T09:48:04.535 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T09:48:04.536 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T09:48:04.536 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-16T09:48:04.536 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T09:48:04.536 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2023-10-16T09:48:04.536 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T09:48:04.537 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-16T09:48:04.537 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T09:48:04.537 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T09:48:05.041 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2023-10-16T09:48:05.049 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2023-10-16T09:48:05.049 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-16T09:48:05.239 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-16T09:48:05.347 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-16T09:48:05.437 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T09:48:05.437 INFO:tasks.ceph:role: osd.1 2023-10-16T09:48:05.438 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T09:48:05.438 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-16T09:48:05.539 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T09:48:05.540 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T09:48:05.540 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T09:48:05.540 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-16T09:48:05.540 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T09:48:05.540 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2023-10-16T09:48:05.541 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T09:48:05.541 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-16T09:48:05.541 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T09:48:05.541 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T09:48:06.030 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2023-10-16T09:48:06.036 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2023-10-16T09:48:06.036 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-16T09:48:06.179 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-16T09:48:06.276 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-16T09:48:06.367 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T09:48:06.367 INFO:tasks.ceph:role: osd.2 2023-10-16T09:48:06.367 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T09:48:06.367 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-16T09:48:06.468 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T09:48:06.468 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T09:48:06.468 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T09:48:06.468 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-16T09:48:06.468 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T09:48:06.469 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2023-10-16T09:48:06.469 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T09:48:06.469 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-16T09:48:06.469 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T09:48:06.469 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T09:48:06.966 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2023-10-16T09:48:06.973 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2023-10-16T09:48:06.973 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-16T09:48:07.141 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-16T09:48:07.245 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-16T09:48:07.341 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T09:48:07.341 INFO:tasks.ceph:role: osd.3 2023-10-16T09:48:07.342 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T09:48:07.342 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-16T09:48:07.443 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T09:48:07.444 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T09:48:07.444 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T09:48:07.444 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-16T09:48:07.444 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T09:48:07.445 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2023-10-16T09:48:07.445 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T09:48:07.445 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-16T09:48:07.445 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T09:48:07.445 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T09:48:07.889 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2023-10-16T09:48:07.895 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2023-10-16T09:48:07.895 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-16T09:48:08.036 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-16T09:48:08.136 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:48:08.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:08.289+0000 7f21c1c0c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-16T09:48:08.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:08.292+0000 7f21c1c0c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-16T09:48:08.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:08.292+0000 7f21c1c0c500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-16T09:48:08.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:08.292+0000 7f21c1c0c500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-16T09:48:08.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:08.292+0000 7f21c1c0c500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-16T09:48:08.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:08.295+0000 7f21c1c0c500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-16T09:48:11.463 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-16T09:48:11.557 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:48:11.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:11.690+0000 7fd35a01a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-16T09:48:11.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:11.691+0000 7fd35a01a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-16T09:48:11.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:11.691+0000 7fd35a01a500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-16T09:48:11.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:11.691+0000 7fd35a01a500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-16T09:48:11.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:11.691+0000 7fd35a01a500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-16T09:48:11.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:11.693+0000 7fd35a01a500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-16T09:48:14.619 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-16T09:48:14.720 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:48:14.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:14.858+0000 7f93b5d07500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-16T09:48:14.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:14.859+0000 7f93b5d07500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-16T09:48:14.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:14.859+0000 7f93b5d07500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-16T09:48:14.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:14.859+0000 7f93b5d07500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-16T09:48:14.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:14.859+0000 7f93b5d07500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-16T09:48:14.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:14.861+0000 7f93b5d07500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-16T09:48:18.057 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-16T09:48:18.157 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:48:18.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:18.296+0000 7f0a0aad6500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-16T09:48:18.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:18.297+0000 7f0a0aad6500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-16T09:48:18.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:18.297+0000 7f0a0aad6500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-16T09:48:18.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:18.297+0000 7f0a0aad6500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-16T09:48:18.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:18.297+0000 7f0a0aad6500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-16T09:48:18.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:18.299+0000 7f0a0aad6500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-16T09:48:21.217 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-16T09:48:21.319 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-16T09:48:21.425 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T09:48:21.425 INFO:tasks.ceph:role: osd.4 2023-10-16T09:48:21.426 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com 2023-10-16T09:48:21.426 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-16T09:48:21.517 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T09:48:21.517 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T09:48:21.518 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T09:48:21.518 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-16T09:48:21.518 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T09:48:21.518 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2023-10-16T09:48:21.518 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T09:48:21.519 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-16T09:48:21.519 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T09:48:21.519 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T09:48:21.551 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2023-10-16T09:48:21.556 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2023-10-16T09:48:21.557 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-16T09:48:21.673 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-16T09:48:21.759 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-16T09:48:21.844 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T09:48:21.845 INFO:tasks.ceph:role: osd.5 2023-10-16T09:48:21.845 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com 2023-10-16T09:48:21.845 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-16T09:48:21.940 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T09:48:21.941 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T09:48:21.941 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T09:48:21.941 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-16T09:48:21.941 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T09:48:21.942 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2023-10-16T09:48:21.942 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T09:48:21.942 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-16T09:48:21.942 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T09:48:21.942 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T09:48:21.974 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2023-10-16T09:48:21.981 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2023-10-16T09:48:21.981 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-16T09:48:22.095 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-16T09:48:22.178 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-16T09:48:22.271 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T09:48:22.271 INFO:tasks.ceph:role: osd.6 2023-10-16T09:48:22.271 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com 2023-10-16T09:48:22.272 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-16T09:48:22.366 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T09:48:22.367 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T09:48:22.367 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T09:48:22.367 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-16T09:48:22.367 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T09:48:22.367 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2023-10-16T09:48:22.368 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T09:48:22.368 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-16T09:48:22.368 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T09:48:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T09:48:22.400 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2023-10-16T09:48:22.405 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2023-10-16T09:48:22.405 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-16T09:48:22.521 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-16T09:48:22.612 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-16T09:48:22.693 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T09:48:22.693 INFO:tasks.ceph:role: osd.7 2023-10-16T09:48:22.693 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com 2023-10-16T09:48:22.694 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-16T09:48:22.779 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T09:48:22.780 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T09:48:22.780 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T09:48:22.780 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-16T09:48:22.780 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T09:48:22.780 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2023-10-16T09:48:22.781 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T09:48:22.781 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-16T09:48:22.781 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T09:48:22.781 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T09:48:22.813 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2023-10-16T09:48:22.818 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2023-10-16T09:48:22.819 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-16T09:48:22.930 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-16T09:48:23.018 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:48:23.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:23.167+0000 7fa915b24500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-16T09:48:23.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:23.168+0000 7fa915b24500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-16T09:48:23.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:23.168+0000 7fa915b24500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-16T09:48:23.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:23.168+0000 7fa915b24500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-16T09:48:23.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:23.168+0000 7fa915b24500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-16T09:48:23.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:23.171+0000 7fa915b24500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-16T09:48:26.362 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-16T09:48:26.466 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:48:26.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:26.606+0000 7f550080c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-16T09:48:26.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:26.607+0000 7f550080c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-16T09:48:26.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:26.607+0000 7f550080c500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-16T09:48:26.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:26.607+0000 7f550080c500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-16T09:48:26.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:26.607+0000 7f550080c500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-16T09:48:26.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:26.609+0000 7f550080c500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-16T09:48:28.513 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:48:28.523 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:48:29.786 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-16T09:48:29.883 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:48:30.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:30.016+0000 7f928080c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-16T09:48:30.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:30.017+0000 7f928080c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-16T09:48:30.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:30.017+0000 7f928080c500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-16T09:48:30.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:30.017+0000 7f928080c500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-16T09:48:30.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:30.017+0000 7f928080c500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-16T09:48:30.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:30.019+0000 7f928080c500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-16T09:48:33.174 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-16T09:48:33.270 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:48:33.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:33.410+0000 7fb46021f500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-16T09:48:33.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:33.411+0000 7fb46021f500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-16T09:48:33.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:33.411+0000 7fb46021f500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-16T09:48:33.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:33.411+0000 7fb46021f500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-16T09:48:33.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:33.411+0000 7fb46021f500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-16T09:48:33.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:33.413+0000 7fb46021f500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-16T09:48:36.587 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-16T09:48:36.690 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-16T09:48:36.690 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:48:36.691 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-16T09:48:36.795 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:48:36.796 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-16T09:48:36.892 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:48:36.892 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-16T09:48:36.987 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:48:36.987 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-16T09:48:37.123 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:48:37.124 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-16T09:48:37.220 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:48:37.221 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-16T09:48:37.317 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:48:37.317 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-16T09:48:37.409 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:48:37.410 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-16T09:48:37.515 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:48:37.515 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-16T09:48:37.608 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:48:37.608 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-16T09:48:37.688 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:48:37.688 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-16T09:48:37.832 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:48:37.832 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-16T09:48:37.926 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:48:37.926 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-16T09:48:38.022 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:48:38.022 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-16T09:48:38.119 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:48:38.120 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-16T09:48:38.177 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:48:38.177 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-16T09:48:38.227 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:48:38.228 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-16T09:48:38.289 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:48:38.289 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-16T09:48:38.342 INFO:tasks.ceph:Adding keys to all mons... 2023-10-16T09:48:38.343 DEBUG:teuthology.orchestra.run.smithi084:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-16T09:48:38.352 DEBUG:teuthology.orchestra.run.smithi157:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-16T09:48:38.439 INFO:teuthology.orchestra.run.smithi084.stdout:[mgr.y] 2023-10-16T09:48:38.439 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDSBi1lLxnKEBAA+a/pOHUp7Sg3m4weAust5w== 2023-10-16T09:48:38.439 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.a] 2023-10-16T09:48:38.440 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDSBi1l6dhjKRAA0pMvY6AK6qtFXDprNhwsqg== 2023-10-16T09:48:38.440 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.c] 2023-10-16T09:48:38.440 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDSBi1l73RBOhAAOMEPxfUAMHhn5Blyvd5oMw== 2023-10-16T09:48:38.440 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.0] 2023-10-16T09:48:38.440 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDYBi1l1zBLERAA3YLW3xIYQoRVw7KSkwN3Og== 2023-10-16T09:48:38.441 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.1] 2023-10-16T09:48:38.441 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDbBi1lFUUxKRAAQbus8SmYprJRmWTjoCznNA== 2023-10-16T09:48:38.441 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.2] 2023-10-16T09:48:38.441 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDeBi1l+9lBMxAArl6sb/h15uBG/a9lVH40Hg== 2023-10-16T09:48:38.441 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.3] 2023-10-16T09:48:38.442 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDiBi1l2rnEERAA3oRD71xa0fEOynM8zKuBEg== 2023-10-16T09:48:38.442 INFO:teuthology.orchestra.run.smithi084.stdout:[mgr.x] 2023-10-16T09:48:38.442 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDSBi1lPuJxHRAABAnGPVCrkq0JR4Y6iFGZlw== 2023-10-16T09:48:38.442 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.b] 2023-10-16T09:48:38.442 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lFpMxEBAA0TLPRt5q3FwwBvMG019cjQ== 2023-10-16T09:48:38.443 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.d] 2023-10-16T09:48:38.443 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lcTIRIhAA0+FWm4b4yw77LD3+WKbk7w== 2023-10-16T09:48:38.443 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.4] 2023-10-16T09:48:38.443 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDnBi1lR7oLChAAZto8fDovDbpOd3Fcnc7C6w== 2023-10-16T09:48:38.443 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.5] 2023-10-16T09:48:38.444 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDqBi1lmjg/JBAAlINKRGBc0/UWJJHMVFw57w== 2023-10-16T09:48:38.444 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.6] 2023-10-16T09:48:38.444 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDuBi1lu0kMARAAtuSeM8F3yr8CVrc3p1biaQ== 2023-10-16T09:48:38.444 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.7] 2023-10-16T09:48:38.444 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDxBi1lf6mNGBAAHGbqzJ0fmDxQIsasboHMfg== 2023-10-16T09:48:38.444 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2023-10-16T09:48:38.445 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ== 2023-10-16T09:48:38.445 INFO:teuthology.orchestra.run.smithi084.stdout:[client.2] 2023-10-16T09:48:38.445 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lA1BOORAAFNyfqgEF34MVR9YX3KBKkQ== 2023-10-16T09:48:38.445 INFO:teuthology.orchestra.run.smithi084.stdout:[client.1] 2023-10-16T09:48:38.445 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDUBi1lHU+4BxAAd8ET8hQ8Y+EUwdTJr9/+Ig== 2023-10-16T09:48:38.446 INFO:teuthology.orchestra.run.smithi084.stdout:[client.3] 2023-10-16T09:48:38.446 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDUBi1l848oERAANMfrZz9JrlgNVh7ZFbtnaA== 2023-10-16T09:48:38.447 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.y] 2023-10-16T09:48:38.448 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDSBi1lLxnKEBAA+a/pOHUp7Sg3m4weAust5w== 2023-10-16T09:48:38.448 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2023-10-16T09:48:38.448 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDSBi1l6dhjKRAA0pMvY6AK6qtFXDprNhwsqg== 2023-10-16T09:48:38.448 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2023-10-16T09:48:38.448 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDSBi1l73RBOhAAOMEPxfUAMHhn5Blyvd5oMw== 2023-10-16T09:48:38.449 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.0] 2023-10-16T09:48:38.449 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDYBi1l1zBLERAA3YLW3xIYQoRVw7KSkwN3Og== 2023-10-16T09:48:38.449 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.1] 2023-10-16T09:48:38.449 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDbBi1lFUUxKRAAQbus8SmYprJRmWTjoCznNA== 2023-10-16T09:48:38.449 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.2] 2023-10-16T09:48:38.449 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDeBi1l+9lBMxAArl6sb/h15uBG/a9lVH40Hg== 2023-10-16T09:48:38.450 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.3] 2023-10-16T09:48:38.450 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDiBi1l2rnEERAA3oRD71xa0fEOynM8zKuBEg== 2023-10-16T09:48:38.450 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.x] 2023-10-16T09:48:38.450 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDSBi1lPuJxHRAABAnGPVCrkq0JR4Y6iFGZlw== 2023-10-16T09:48:38.451 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2023-10-16T09:48:38.451 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDTBi1lFpMxEBAA0TLPRt5q3FwwBvMG019cjQ== 2023-10-16T09:48:38.451 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2023-10-16T09:48:38.451 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDTBi1lcTIRIhAA0+FWm4b4yw77LD3+WKbk7w== 2023-10-16T09:48:38.451 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.4] 2023-10-16T09:48:38.451 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDnBi1lR7oLChAAZto8fDovDbpOd3Fcnc7C6w== 2023-10-16T09:48:38.452 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.5] 2023-10-16T09:48:38.452 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDqBi1lmjg/JBAAlINKRGBc0/UWJJHMVFw57w== 2023-10-16T09:48:38.452 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.6] 2023-10-16T09:48:38.452 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDuBi1lu0kMARAAtuSeM8F3yr8CVrc3p1biaQ== 2023-10-16T09:48:38.452 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.7] 2023-10-16T09:48:38.453 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDxBi1lf6mNGBAAHGbqzJ0fmDxQIsasboHMfg== 2023-10-16T09:48:38.453 INFO:teuthology.orchestra.run.smithi157.stdout:[client.0] 2023-10-16T09:48:38.453 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ== 2023-10-16T09:48:38.453 INFO:teuthology.orchestra.run.smithi157.stdout:[client.2] 2023-10-16T09:48:38.453 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDTBi1lA1BOORAAFNyfqgEF34MVR9YX3KBKkQ== 2023-10-16T09:48:38.454 INFO:teuthology.orchestra.run.smithi157.stdout:[client.1] 2023-10-16T09:48:38.454 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDUBi1lHU+4BxAAd8ET8hQ8Y+EUwdTJr9/+Ig== 2023-10-16T09:48:38.454 INFO:teuthology.orchestra.run.smithi157.stdout:[client.3] 2023-10-16T09:48:38.454 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDUBi1l848oERAANMfrZz9JrlgNVh7ZFbtnaA== 2023-10-16T09:48:38.455 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T09:48:38.496 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T09:48:38.649 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T09:48:38.658 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T09:48:38.805 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T09:48:38.814 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T09:48:38.952 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:48:38.958 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:48:39.092 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:48:39.098 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:48:39.245 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:48:39.254 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:48:39.402 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:48:39.408 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:48:39.540 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T09:48:39.546 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T09:48:39.694 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T09:48:39.702 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T09:48:39.853 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T09:48:39.861 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T09:48:40.017 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:48:40.025 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:48:40.165 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:48:40.170 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:48:40.312 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:48:40.321 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:48:40.469 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:48:40.475 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:48:40.617 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T09:48:40.625 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T09:48:40.778 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T09:48:40.787 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T09:48:40.940 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T09:48:40.949 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T09:48:41.093 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T09:48:41.099 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T09:48:41.240 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-16T09:48:41.240 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-16T09:48:41.333 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-16T09:48:41.494 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-16T09:48:41.576 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-16T09:48:41.682 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-16T09:48:41.844 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-16T09:48:41.931 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-16T09:48:42.024 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-16T09:48:42.179 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-16T09:48:42.255 DEBUG:teuthology.orchestra.run.smithi084:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:48:42.263 DEBUG:teuthology.orchestra.run.smithi157:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:48:42.313 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-16T09:48:42.314 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-16T09:48:42.314 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-16T09:48:42.320 INFO:tasks.ceph.mon.a:Started 2023-10-16T09:48:42.320 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-16T09:48:42.321 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-16T09:48:42.362 INFO:tasks.ceph.mon.b:Started 2023-10-16T09:48:42.362 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-16T09:48:42.363 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-16T09:48:42.371 INFO:tasks.ceph.mon.c:Started 2023-10-16T09:48:42.371 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-16T09:48:42.371 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-16T09:48:42.372 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-16T09:48:42.378 INFO:tasks.ceph.mgr.y:Started 2023-10-16T09:48:42.378 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-16T09:48:42.378 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-16T09:48:42.383 INFO:tasks.ceph.mgr.x:Started 2023-10-16T09:48:42.383 INFO:tasks.ceph:Setting crush tunables to default 2023-10-16T09:48:42.384 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd crush tunables default 2023-10-16T09:48:42.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.560+0000 7f1f41444640 1 Processor -- start 2023-10-16T09:48:42.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.560+0000 7f1f41444640 1 -- start start 2023-10-16T09:48:42.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.561+0000 7f1f41444640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c0b3070 0x7f1f3c0cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:42.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.561+0000 7f1f41444640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1f3c0d1b10 0x7f1f3c0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:42.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.561+0000 7f1f41444640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f3c0b3940 con 0x7f1f3c0b3e70 2023-10-16T09:48:42.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.561+0000 7f1f41444640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f3c0b45e0 con 0x7f1f3c0d1b10 2023-10-16T09:48:42.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.561+0000 7f1f41444640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c0b3b90 con 0x7f1f3c0b3070 2023-10-16T09:48:42.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.562+0000 7f1f3affd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c0b3070 0x7f1f3c0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:42.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.562+0000 7f1f3b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1f3c0d1b10 0x7f1f3c0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:42.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.562+0000 7f1f3affd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c0b3070 0x7f1f3c0cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:42284/0 (socket says 172.21.15.84:42284) 2023-10-16T09:48:42.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.562+0000 7f1f3affd640 1 -- 172.21.15.84:0/1339797384 learned_addr learned my addr 172.21.15.84:0/1339797384 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:48:42.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.562+0000 7f1f3affd640 1 -- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c0b3070 msgr2=0x7f1f3c0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-16T09:48:42.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.562+0000 7f1f3affd640 1 -- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c0b3070 msgr2=0x7f1f3c0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T09:48:42.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.562+0000 7f1f3affd640 1 --2- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c0b3070 0x7f1f3c0cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T09:48:42.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.562+0000 7f1f3affd640 1 --2- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c0b3070 0x7f1f3c0cf500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:48:42.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.562+0000 7f1f3b7fe640 1 -- 172.21.15.84:0/1339797384 >> v2:172.21.15.157:3300/0 conn(0x7f1f3c0d1b10 msgr2=0x7f1f3c0cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-16T09:48:42.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.562+0000 7f1f3b7fe640 1 -- 172.21.15.84:0/1339797384 >> v2:172.21.15.157:3300/0 conn(0x7f1f3c0d1b10 msgr2=0x7f1f3c0cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T09:48:42.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.562+0000 7f1f3b7fe640 1 --2- 172.21.15.84:0/1339797384 >> v2:172.21.15.157:3300/0 conn(0x7f1f3c0d1b10 0x7f1f3c0cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T09:48:42.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.562+0000 7f1f3b7fe640 1 --2- 172.21.15.84:0/1339797384 >> v2:172.21.15.157:3300/0 conn(0x7f1f3c0d1b10 0x7f1f3c0cfa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:48:42.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.763+0000 7f1f3b7fe640 1 --2- 172.21.15.84:0/1339797384 >> v2:172.21.15.157:3300/0 conn(0x7f1f3c0d1b10 0x7f1f3c0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:42.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.764+0000 7f1f3affd640 1 --2- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c0b3070 0x7f1f3c0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:42.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.764+0000 7f1f3b7fe640 1 -- 172.21.15.84:0/1339797384 >> v2:172.21.15.157:3300/0 conn(0x7f1f3c0d1b10 msgr2=0x7f1f3c0cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-16T09:48:42.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.764+0000 7f1f3b7fe640 1 -- 172.21.15.84:0/1339797384 >> v2:172.21.15.157:3300/0 conn(0x7f1f3c0d1b10 msgr2=0x7f1f3c0cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T09:48:42.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.764+0000 7f1f3b7fe640 1 --2- 172.21.15.84:0/1339797384 >> v2:172.21.15.157:3300/0 conn(0x7f1f3c0d1b10 0x7f1f3c0cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T09:48:42.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.764+0000 7f1f3b7fe640 1 --2- 172.21.15.84:0/1339797384 >> v2:172.21.15.157:3300/0 conn(0x7f1f3c0d1b10 0x7f1f3c0cfa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:48:42.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.764+0000 7f1f3affd640 1 -- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c0b3070 msgr2=0x7f1f3c0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-16T09:48:42.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.764+0000 7f1f3affd640 1 -- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c0b3070 msgr2=0x7f1f3c0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T09:48:42.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.764+0000 7f1f3affd640 1 --2- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c0b3070 0x7f1f3c0cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T09:48:42.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.764+0000 7f1f3affd640 1 --2- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c0b3070 0x7f1f3c0cf500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:48:42.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.778+0000 7f1f3a7fc640 1 -- 172.21.15.84:0/1339797384 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3891473928 0 0) 0x7f1f3c0b3940 con 0x7f1f3c0b3e70 2023-10-16T09:48:42.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.778+0000 7f1f3a7fc640 1 -- 172.21.15.84:0/1339797384 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f28003520 con 0x7f1f3c0b3e70 2023-10-16T09:48:42.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.779+0000 7f1f3a7fc640 1 -- 172.21.15.84:0/1339797384 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1897574811 0 0) 0x7f1f28003520 con 0x7f1f3c0b3e70 2023-10-16T09:48:42.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.779+0000 7f1f3a7fc640 1 -- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c0b3070 msgr2=0x7f1f3c0cf500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T09:48:42.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.779+0000 7f1f3a7fc640 1 --2- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c0b3070 0x7f1f3c0cf500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:42.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.779+0000 7f1f3a7fc640 1 -- 172.21.15.84:0/1339797384 >> v2:172.21.15.157:3300/0 conn(0x7f1f3c0d1b10 msgr2=0x7f1f3c0cfa40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T09:48:42.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.779+0000 7f1f3a7fc640 1 --2- 172.21.15.84:0/1339797384 >> v2:172.21.15.157:3300/0 conn(0x7f1f3c0d1b10 0x7f1f3c0cfa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:42.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.779+0000 7f1f3a7fc640 1 -- 172.21.15.84:0/1339797384 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3c0d3840 con 0x7f1f3c0b3e70 2023-10-16T09:48:42.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.779+0000 7f1f3a7fc640 1 -- 172.21.15.84:0/1339797384 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f1f24001670 con 0x7f1f3c0b3e70 2023-10-16T09:48:42.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.779+0000 7f1f3a7fc640 1 -- 172.21.15.84:0/1339797384 >> v1:172.21.15.157:6789/0 conn(0x7f1f3c0b3e70 legacy=0x7f1f3c0c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:42.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.779+0000 7f1f3a7fc640 1 --2- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f28003970 0x7f1f28003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:42.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.779+0000 7f1f3a7fc640 1 --2- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c0b3070 0x7f1f280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:42.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.780+0000 7f1f3a7fc640 1 --2- 172.21.15.84:0/1339797384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f28004d00 0x7f1f28005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:42.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.780+0000 7f1f3a7fc640 1 -- 172.21.15.84:0/1339797384 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c0b45e0 con 0x7f1f28004d00 2023-10-16T09:48:42.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.780+0000 7f1f3a7fc640 1 -- 172.21.15.84:0/1339797384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c0b3b90 con 0x7f1f28003970 2023-10-16T09:48:42.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.780+0000 7f1f3a7fc640 1 -- 172.21.15.84:0/1339797384 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2800ba30 con 0x7f1f3c0b3070 2023-10-16T09:48:42.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.780+0000 7f1f3affd640 1 --2- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f28003970 0x7f1f28003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:42.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.781+0000 7f1f33fff640 1 --2- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c0b3070 0x7f1f280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:42.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.781+0000 7f1f3b7fe640 1 --2- 172.21.15.84:0/1339797384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f28004d00 0x7f1f28005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:42.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.782+0000 7f1f3affd640 1 -- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c0b3070 msgr2=0x7f1f280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:42.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.782+0000 7f1f3affd640 1 --2- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c0b3070 0x7f1f280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:42.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.782+0000 7f1f3affd640 1 -- 172.21.15.84:0/1339797384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f28004d00 msgr2=0x7f1f28005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:42.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.782+0000 7f1f3affd640 1 --2- 172.21.15.84:0/1339797384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f28004d00 0x7f1f28005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:42.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.782+0000 7f1f3affd640 1 -- 172.21.15.84:0/1339797384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f28003520 con 0x7f1f28003970 2023-10-16T09:48:42.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.784+0000 7f1f3affd640 1 --2- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f28003970 0x7f1f28003d90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00e8c0 tx=0x7f1f280057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15a141eac558a6c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:42.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.784+0000 7f1f3a7fc640 1 -- 172.21.15.84:0/1339797384 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c005b90 con 0x7f1f28003970 2023-10-16T09:48:42.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.784+0000 7f1f3a7fc640 1 -- 172.21.15.84:0/1339797384 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2c030070 con 0x7f1f28003970 2023-10-16T09:48:42.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.784+0000 7f1f3a7fc640 1 -- 172.21.15.84:0/1339797384 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c02c680 con 0x7f1f28003970 2023-10-16T09:48:42.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.785+0000 7f1f41444640 1 -- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f28003970 msgr2=0x7f1f28003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:42.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.785+0000 7f1f41444640 1 --2- 172.21.15.84:0/1339797384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f28003970 0x7f1f28003d90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00e8c0 tx=0x7f1f280057f0 comp rx=0 tx=0).stop 2023-10-16T09:48:42.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.785+0000 7f1f3b7fe640 1 -- 172.21.15.84:0/1339797384 reap_dead start 2023-10-16T09:48:42.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.785+0000 7f1f41444640 1 -- 172.21.15.84:0/1339797384 shutdown_connections 2023-10-16T09:48:42.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.785+0000 7f1f41444640 1 -- 172.21.15.84:0/1339797384 >> 172.21.15.84:0/1339797384 conn(0x7f1f3c0d2d20 msgr2=0x7f1f3c0d3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:42.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.786+0000 7f1f41444640 1 -- 172.21.15.84:0/1339797384 shutdown_connections 2023-10-16T09:48:42.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.786+0000 7f1f41444640 1 -- 172.21.15.84:0/1339797384 wait complete. 2023-10-16T09:48:42.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.788+0000 7f1f41444640 1 Processor -- start 2023-10-16T09:48:42.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.788+0000 7f1f41444640 1 -- start start 2023-10-16T09:48:42.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.790+0000 7f1f41444640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f28003970 0x7f1f3c15f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:42.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.790+0000 7f1f41444640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c160000 0x7f1f3c160480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:42.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.790+0000 7f1f41444640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c0cf1d0 0x7f1f3c0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:42.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.790+0000 7f1f41444640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c1609c0 con 0x7f1f3c0cf1d0 2023-10-16T09:48:42.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.790+0000 7f1f41444640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c0cfbc0 con 0x7f1f28003970 2023-10-16T09:48:42.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.790+0000 7f1f41444640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c15a030 con 0x7f1f3c160000 2023-10-16T09:48:42.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.790+0000 7f1f3affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f28003970 0x7f1f3c15f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:42.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.790+0000 7f1f3affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f28003970 0x7f1f3c15f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:46508/0 (socket says 172.21.15.84:46508) 2023-10-16T09:48:42.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.791+0000 7f1f3affd640 1 -- 172.21.15.84:0/457273912 learned_addr learned my addr 172.21.15.84:0/457273912 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:48:42.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.791+0000 7f1f3b7fe640 1 --2- 172.21.15.84:0/457273912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c0cf1d0 0x7f1f3c0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:42.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.791+0000 7f1f33fff640 1 --2- 172.21.15.84:0/457273912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c160000 0x7f1f3c160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:42.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.791+0000 7f1f3affd640 1 -- 172.21.15.84:0/457273912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c160000 msgr2=0x7f1f3c160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:42.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.791+0000 7f1f3affd640 1 --2- 172.21.15.84:0/457273912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c160000 0x7f1f3c160480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:42.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.791+0000 7f1f3affd640 1 -- 172.21.15.84:0/457273912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c0cf1d0 msgr2=0x7f1f3c0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:42.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.791+0000 7f1f3affd640 1 --2- 172.21.15.84:0/457273912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c0cf1d0 0x7f1f3c0cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:42.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.791+0000 7f1f3affd640 1 -- 172.21.15.84:0/457273912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3c15a310 con 0x7f1f28003970 2023-10-16T09:48:42.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.791+0000 7f1f3b7fe640 1 --2- 172.21.15.84:0/457273912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c0cf1d0 0x7f1f3c0cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:48:42.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.792+0000 7f1f3affd640 1 --2- 172.21.15.84:0/457273912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f28003970 0x7f1f3c15f970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00e9e0 tx=0x7f1f2c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:42.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.793+0000 7f1f337fe640 1 -- 172.21.15.84:0/457273912 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c02db20 con 0x7f1f28003970 2023-10-16T09:48:42.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.793+0000 7f1f337fe640 1 -- 172.21.15.84:0/457273912 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2c02dca0 con 0x7f1f28003970 2023-10-16T09:48:42.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.793+0000 7f1f337fe640 1 -- 172.21.15.84:0/457273912 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c006290 con 0x7f1f28003970 2023-10-16T09:48:42.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.793+0000 7f1f41444640 1 -- 172.21.15.84:0/457273912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3c15a5f0 con 0x7f1f28003970 2023-10-16T09:48:42.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.793+0000 7f1f41444640 1 -- 172.21.15.84:0/457273912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3c15abb0 con 0x7f1f28003970 2023-10-16T09:48:42.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.794+0000 7f1f337fe640 1 -- 172.21.15.84:0/457273912 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1f2c03f070 con 0x7f1f28003970 2023-10-16T09:48:42.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.794+0000 7f1f337fe640 1 -- 172.21.15.84:0/457273912 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1f2c032070 con 0x7f1f28003970 2023-10-16T09:48:42.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.795+0000 7f1f41444640 1 -- 172.21.15.84:0/457273912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3c0cfbc0 con 0x7f1f28003970 2023-10-16T09:48:42.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.799+0000 7f1f337fe640 1 -- 172.21.15.84:0/457273912 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f1f3c0cfbc0 con 0x7f1f28003970 2023-10-16T09:48:42.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.872+0000 7f1f41444640 1 -- 172.21.15.84:0/457273912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f1f3c15ad40 con 0x7f1f28003970 2023-10-16T09:48:42.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.879+0000 7f1f337fe640 1 -- 172.21.15.84:0/457273912 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1f2c048070 con 0x7f1f28003970 2023-10-16T09:48:42.881 INFO:teuthology.orchestra.run.smithi084.stderr:adjusted tunables profile to default 2023-10-16T09:48:42.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.881+0000 7f1f41444640 1 -- 172.21.15.84:0/457273912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f28003970 msgr2=0x7f1f3c15f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:42.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.881+0000 7f1f41444640 1 --2- 172.21.15.84:0/457273912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f28003970 0x7f1f3c15f970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00e9e0 tx=0x7f1f2c0046c0 comp rx=0 tx=0).stop 2023-10-16T09:48:42.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.881+0000 7f1f41444640 1 -- 172.21.15.84:0/457273912 shutdown_connections 2023-10-16T09:48:42.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.881+0000 7f1f41444640 1 --2- 172.21.15.84:0/457273912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c0cf1d0 0x7f1f3c0cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:42.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.881+0000 7f1f41444640 1 --2- 172.21.15.84:0/457273912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c160000 0x7f1f3c160480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:42.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.881+0000 7f1f41444640 1 --2- 172.21.15.84:0/457273912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f28003970 0x7f1f3c15f970 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:42.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.881+0000 7f1f41444640 1 -- 172.21.15.84:0/457273912 >> 172.21.15.84:0/457273912 conn(0x7f1f3c0d2d20 msgr2=0x7f1f3c0c69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:42.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.881+0000 7f1f41444640 1 -- 172.21.15.84:0/457273912 shutdown_connections 2023-10-16T09:48:42.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:42.881+0000 7f1f41444640 1 -- 172.21.15.84:0/457273912 wait complete. 2023-10-16T09:48:42.900 INFO:tasks.ceph:check_enable_crimson: False 2023-10-16T09:48:42.900 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-16T09:48:42.900 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:48:42.901 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-16T09:48:42.976 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:48:42.977 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-16T09:48:43.102 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:48:43.103 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-16T09:48:43.238 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:48:43.239 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-16T09:48:43.324 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:48:43.324 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-16T09:48:43.368 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:43.367+0000 7fa84ea2b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T09:48:43.416 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:48:43.417 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-16T09:48:43.435 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:43.433+0000 7fa84ea2b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T09:48:43.436 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:43.435+0000 7f1aece12040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T09:48:43.503 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:48:43.503 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-16T09:48:43.504 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:43.504+0000 7f1aece12040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T09:48:43.584 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:48:43.585 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-16T09:48:43.716 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 2e56c068-81b6-4d45-ae23-abd5ca91bf25 0 2023-10-16T09:48:43.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.915+0000 7f21394ad640 1 Processor -- start 2023-10-16T09:48:43.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.915+0000 7f21394ad640 1 -- start start 2023-10-16T09:48:43.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.916+0000 7f21394ad640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f213415cd70 0x7f21340d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:43.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.917+0000 7f21394ad640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f21340cded0 0x7f21340b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:43.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.917+0000 7f21394ad640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f213415d150 con 0x7f21340b8480 2023-10-16T09:48:43.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.917+0000 7f21394ad640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21340d1180 con 0x7f21340cded0 2023-10-16T09:48:43.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.917+0000 7f21394ad640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21340d1460 con 0x7f213415cd70 2023-10-16T09:48:43.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.917+0000 7f2133fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f213415cd70 0x7f21340d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:43.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.917+0000 7f2133fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f213415cd70 0x7f21340d2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51110/0 (socket says 172.21.15.157:51110) 2023-10-16T09:48:43.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.917+0000 7f2133fff640 1 -- 172.21.15.157:0/837072862 learned_addr learned my addr 172.21.15.157:0/837072862 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-16T09:48:43.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.917+0000 7f2138cac640 1 --2- 172.21.15.157:0/837072862 >> v2:172.21.15.84:3300/0 conn(0x7f21340cded0 0x7f21340b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:43.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.917+0000 7f2133fff640 1 -- 172.21.15.157:0/837072862 >> v2:172.21.15.84:3300/0 conn(0x7f21340cded0 msgr2=0x7f21340b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:43.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.917+0000 7f2133fff640 1 --2- 172.21.15.157:0/837072862 >> v2:172.21.15.84:3300/0 conn(0x7f21340cded0 0x7f21340b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:43.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.917+0000 7f2133fff640 1 -- 172.21.15.157:0/837072862 >> v1:172.21.15.84:6789/0 conn(0x7f21340b8480 legacy=0x7f21340cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:43.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.917+0000 7f2133fff640 1 -- 172.21.15.157:0/837072862 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21340d1740 con 0x7f213415cd70 2023-10-16T09:48:43.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.918+0000 7f2133fff640 1 --2- 172.21.15.157:0/837072862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f213415cd70 0x7f21340d2220 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f21340b56a0 tx=0x7f212402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee47ddc2274426a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:43.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.919+0000 7f21337fe640 1 -- 172.21.15.157:0/837072862 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124031870 con 0x7f213415cd70 2023-10-16T09:48:43.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.919+0000 7f21337fe640 1 -- 172.21.15.157:0/837072862 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21240319d0 con 0x7f213415cd70 2023-10-16T09:48:43.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.919+0000 7f21337fe640 1 -- 172.21.15.157:0/837072862 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124031d60 con 0x7f213415cd70 2023-10-16T09:48:43.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.919+0000 7f21394ad640 1 -- 172.21.15.157:0/837072862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f213415cd70 msgr2=0x7f21340d2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:43.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.919+0000 7f21394ad640 1 --2- 172.21.15.157:0/837072862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f213415cd70 0x7f21340d2220 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f21340b56a0 tx=0x7f212402b910 comp rx=0 tx=0).stop 2023-10-16T09:48:43.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.919+0000 7f21394ad640 1 -- 172.21.15.157:0/837072862 shutdown_connections 2023-10-16T09:48:43.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.919+0000 7f21394ad640 1 --2- 172.21.15.157:0/837072862 >> v2:172.21.15.84:3300/0 conn(0x7f21340cded0 0x7f21340b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:43.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.919+0000 7f21394ad640 1 --2- 172.21.15.157:0/837072862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f213415cd70 0x7f21340d2220 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:43.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.919+0000 7f21394ad640 1 -- 172.21.15.157:0/837072862 >> 172.21.15.157:0/837072862 conn(0x7f21340cf570 msgr2=0x7f21340cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:43.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.919+0000 7f21394ad640 1 -- 172.21.15.157:0/837072862 shutdown_connections 2023-10-16T09:48:43.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.919+0000 7f21394ad640 1 -- 172.21.15.157:0/837072862 wait complete. 2023-10-16T09:48:43.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.921+0000 7f21394ad640 1 Processor -- start 2023-10-16T09:48:43.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.921+0000 7f21394ad640 1 -- start start 2023-10-16T09:48:43.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.922+0000 7f21394ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21340b8480 0x7f21340a9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:43.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.922+0000 7f21394ad640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21340cded0 0x7f21341f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:43.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.922+0000 7f21394ad640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f213415cd70 0x7f21340b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:43.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.922+0000 7f21394ad640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213414a8a0 con 0x7f213415cd70 2023-10-16T09:48:43.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.922+0000 7f21394ad640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2134149c20 con 0x7f21340b8480 2023-10-16T09:48:43.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.922+0000 7f21394ad640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2134149f00 con 0x7f21340cded0 2023-10-16T09:48:43.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.922+0000 7f2133fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21340b8480 0x7f21340a9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:43.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.922+0000 7f212bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21340cded0 0x7f21341f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:43.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.922+0000 7f2133fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21340b8480 0x7f21340a9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51120/0 (socket says 172.21.15.157:51120) 2023-10-16T09:48:43.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.923+0000 7f2133fff640 1 -- 172.21.15.157:0/2225169659 learned_addr learned my addr 172.21.15.157:0/2225169659 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-16T09:48:43.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.923+0000 7f2138cac640 1 --2- 172.21.15.157:0/2225169659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f213415cd70 0x7f21340b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:43.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.923+0000 7f2133fff640 1 -- 172.21.15.157:0/2225169659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21340cded0 msgr2=0x7f21341f1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:43.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.923+0000 7f2133fff640 1 --2- 172.21.15.157:0/2225169659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21340cded0 0x7f21341f1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:43.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.923+0000 7f2133fff640 1 -- 172.21.15.157:0/2225169659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f213415cd70 msgr2=0x7f21340b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:43.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.923+0000 7f2133fff640 1 --2- 172.21.15.157:0/2225169659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f213415cd70 0x7f21340b2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:43.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.923+0000 7f2133fff640 1 -- 172.21.15.157:0/2225169659 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21240094d0 con 0x7f21340b8480 2023-10-16T09:48:43.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.923+0000 7f2133fff640 1 --2- 172.21.15.157:0/2225169659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21340b8480 0x7f21340a9be0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2124000c00 tx=0x7f21240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:43.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.923+0000 7f21317fa640 1 -- 172.21.15.157:0/2225169659 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212403abe0 con 0x7f21340b8480 2023-10-16T09:48:43.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.923+0000 7f21317fa640 1 -- 172.21.15.157:0/2225169659 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212403ad40 con 0x7f21340b8480 2023-10-16T09:48:43.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.923+0000 7f21394ad640 1 -- 172.21.15.157:0/2225169659 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213414a1e0 con 0x7f21340b8480 2023-10-16T09:48:43.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.924+0000 7f21317fa640 1 -- 172.21.15.157:0/2225169659 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124044610 con 0x7f21340b8480 2023-10-16T09:48:43.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.924+0000 7f21394ad640 1 -- 172.21.15.157:0/2225169659 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21340b2060 con 0x7f21340b8480 2023-10-16T09:48:43.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.924+0000 7f21317fa640 1 -- 172.21.15.157:0/2225169659 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2124044a70 con 0x7f21340b8480 2023-10-16T09:48:43.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.924+0000 7f21317fa640 1 -- 172.21.15.157:0/2225169659 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f212402d040 con 0x7f21340b8480 2023-10-16T09:48:43.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.925+0000 7f21394ad640 1 -- 172.21.15.157:0/2225169659 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f213404e9b0 con 0x7f21340b8480 2023-10-16T09:48:43.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:43.928+0000 7f21317fa640 1 -- 172.21.15.157:0/2225169659 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f2124035020 con 0x7f21340b8480 2023-10-16T09:48:44.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.006+0000 7f21394ad640 1 -- 172.21.15.157:0/2225169659 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2e56c068-81b6-4d45-ae23-abd5ca91bf25", "id": 0} v 0) v1 -- 0x7f213415c320 con 0x7f21340b8480 2023-10-16T09:48:44.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.013+0000 7f21317fa640 1 -- 172.21.15.157:0/2225169659 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2e56c068-81b6-4d45-ae23-abd5ca91bf25", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f212404d450 con 0x7f21340b8480 2023-10-16T09:48:44.014 INFO:teuthology.orchestra.run.smithi157.stdout:0 2023-10-16T09:48:44.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.015+0000 7f21394ad640 1 -- 172.21.15.157:0/2225169659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21340b8480 msgr2=0x7f21340a9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:44.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.015+0000 7f21394ad640 1 --2- 172.21.15.157:0/2225169659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21340b8480 0x7f21340a9be0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2124000c00 tx=0x7f21240046c0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.015+0000 7f21394ad640 1 -- 172.21.15.157:0/2225169659 shutdown_connections 2023-10-16T09:48:44.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.015+0000 7f21394ad640 1 --2- 172.21.15.157:0/2225169659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f213415cd70 0x7f21340b2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.015+0000 7f21394ad640 1 --2- 172.21.15.157:0/2225169659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21340cded0 0x7f21341f1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.015+0000 7f21394ad640 1 --2- 172.21.15.157:0/2225169659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21340b8480 0x7f21340a9be0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.015+0000 7f21394ad640 1 -- 172.21.15.157:0/2225169659 >> 172.21.15.157:0/2225169659 conn(0x7f21340cf570 msgr2=0x7f21340d2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:44.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.015+0000 7f21394ad640 1 -- 172.21.15.157:0/2225169659 shutdown_connections 2023-10-16T09:48:44.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.015+0000 7f21394ad640 1 -- 172.21.15.157:0/2225169659 wait complete. 2023-10-16T09:48:44.031 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:44.029+0000 7fa84ea2b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T09:48:44.032 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 306aa004-b7d8-4c60-b346-925524c4d79f 1 2023-10-16T09:48:44.091 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:44.090+0000 7fa84ea2b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T09:48:44.142 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:44.140+0000 7fa84ea2b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T09:48:44.146 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:44.146+0000 7f1aece12040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T09:48:44.203 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:44.202+0000 7fa84ea2b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T09:48:44.210 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:44.210+0000 7f1aece12040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T09:48:44.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.224+0000 7f5184d68640 1 Processor -- start 2023-10-16T09:48:44.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.225+0000 7f5184d68640 1 -- start start 2023-10-16T09:48:44.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.226+0000 7f5184d68640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f518006c850 0x7f51800ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:44.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.226+0000 7f5184d68640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5180070360 con 0x7f5180159ab0 2023-10-16T09:48:44.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.226+0000 7f5184d68640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51800cd1f0 con 0x7f5180159e90 2023-10-16T09:48:44.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.226+0000 7f5184d68640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518015d890 con 0x7f518006c850 2023-10-16T09:48:44.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.226+0000 7f5175d9b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5180159e90 0x7f518006c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37526/0 (socket says 172.21.15.157:37526) 2023-10-16T09:48:44.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.226+0000 7f5175d9b640 1 -- 172.21.15.157:0/2983836577 learned_addr learned my addr 172.21.15.157:0/2983836577 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-16T09:48:44.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.227+0000 7f517ed9d640 1 --2- 172.21.15.157:0/2983836577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f518006c850 0x7f51800ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:44.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.227+0000 7f517dd9b640 1 -- 172.21.15.157:0/2983836577 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694897298 0 0) 0x7f51800cd1f0 con 0x7f5180159e90 2023-10-16T09:48:44.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.227+0000 7f517dd9b640 1 -- 172.21.15.157:0/2983836577 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f516c003520 con 0x7f5180159e90 2023-10-16T09:48:44.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.227+0000 7f517ed9d640 1 -- 172.21.15.157:0/2983836577 >> v1:172.21.15.157:6789/0 conn(0x7f5180159e90 legacy=0x7f518006c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:44.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.227+0000 7f517ed9d640 1 -- 172.21.15.157:0/2983836577 >> v1:172.21.15.84:6789/0 conn(0x7f5180159ab0 legacy=0x7f518006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:44.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.227+0000 7f517ed9d640 1 -- 172.21.15.157:0/2983836577 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51800cd4d0 con 0x7f518006c850 2023-10-16T09:48:44.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.228+0000 7f517ed9d640 1 --2- 172.21.15.157:0/2983836577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f518006c850 0x7f51800ce370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5178009dc0 tx=0x7f517802be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b221292ff5ac3bc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:44.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.228+0000 7f517dd9b640 1 -- 172.21.15.157:0/2983836577 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517802cbb0 con 0x7f518006c850 2023-10-16T09:48:44.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.228+0000 7f517dd9b640 1 -- 172.21.15.157:0/2983836577 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517802cd10 con 0x7f518006c850 2023-10-16T09:48:44.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.228+0000 7f517dd9b640 1 -- 172.21.15.157:0/2983836577 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178034660 con 0x7f518006c850 2023-10-16T09:48:44.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.229+0000 7f5184d68640 1 -- 172.21.15.157:0/2983836577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f518006c850 msgr2=0x7f51800ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:44.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.229+0000 7f5184d68640 1 --2- 172.21.15.157:0/2983836577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f518006c850 0x7f51800ce370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5178009dc0 tx=0x7f517802be00 comp rx=0 tx=0).stop 2023-10-16T09:48:44.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.229+0000 7f5184d68640 1 -- 172.21.15.157:0/2983836577 shutdown_connections 2023-10-16T09:48:44.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.229+0000 7f5184d68640 1 --2- 172.21.15.157:0/2983836577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f518006c850 0x7f51800ce370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.229+0000 7f5184d68640 1 -- 172.21.15.157:0/2983836577 >> 172.21.15.157:0/2983836577 conn(0x7f51800ce9b0 msgr2=0x7f51800cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:44.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.229+0000 7f5184d68640 1 -- 172.21.15.157:0/2983836577 shutdown_connections 2023-10-16T09:48:44.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.229+0000 7f5184d68640 1 -- 172.21.15.157:0/2983836577 wait complete. 2023-10-16T09:48:44.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.231+0000 7f5184d68640 1 Processor -- start 2023-10-16T09:48:44.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.231+0000 7f5184d68640 1 -- start start 2023-10-16T09:48:44.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.232+0000 7f5184d68640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f518006c850 0x7f51800acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:44.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.232+0000 7f5184d68640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5180159ab0 0x7f518014a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:44.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.232+0000 7f5184d68640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5180159e90 0x7f518014ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:44.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.232+0000 7f5184d68640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518014bfd0 con 0x7f518006c850 2023-10-16T09:48:44.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.232+0000 7f5184d68640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518014daa0 con 0x7f5180159e90 2023-10-16T09:48:44.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.232+0000 7f5184d68640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518014b0a0 con 0x7f5180159ab0 2023-10-16T09:48:44.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.232+0000 7f517ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5180159e90 0x7f518014ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:44.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.232+0000 7f5175d9b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5180159ab0 0x7f518014a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:44.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.232+0000 7f517ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5180159e90 0x7f518014ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51130/0 (socket says 172.21.15.157:51130) 2023-10-16T09:48:44.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.232+0000 7f517ed9d640 1 -- 172.21.15.157:0/3356378561 learned_addr learned my addr 172.21.15.157:0/3356378561 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-16T09:48:44.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.232+0000 7f517e59c640 1 --2- 172.21.15.157:0/3356378561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f518006c850 0x7f51800acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:44.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.233+0000 7f517ed9d640 1 -- 172.21.15.157:0/3356378561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5180159ab0 msgr2=0x7f518014a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:44.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.233+0000 7f517ed9d640 1 --2- 172.21.15.157:0/3356378561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5180159ab0 0x7f518014a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.233+0000 7f517ed9d640 1 -- 172.21.15.157:0/3356378561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f518006c850 msgr2=0x7f51800acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:44.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.233+0000 7f517ed9d640 1 --2- 172.21.15.157:0/3356378561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f518006c850 0x7f51800acb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.233+0000 7f517ed9d640 1 -- 172.21.15.157:0/3356378561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5178009a70 con 0x7f5180159e90 2023-10-16T09:48:44.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.233+0000 7f517ed9d640 1 --2- 172.21.15.157:0/3356378561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5180159e90 0x7f518014ab60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5178009dc0 tx=0x7f5178004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:44.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.233+0000 7f51777fe640 1 -- 172.21.15.157:0/3356378561 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178040650 con 0x7f5180159e90 2023-10-16T09:48:44.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.233+0000 7f51777fe640 1 -- 172.21.15.157:0/3356378561 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51780407b0 con 0x7f5180159e90 2023-10-16T09:48:44.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.233+0000 7f51777fe640 1 -- 172.21.15.157:0/3356378561 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178040b60 con 0x7f5180159e90 2023-10-16T09:48:44.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.233+0000 7f5184d68640 1 -- 172.21.15.157:0/3356378561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518014b350 con 0x7f5180159e90 2023-10-16T09:48:44.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.233+0000 7f5184d68640 1 -- 172.21.15.157:0/3356378561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518014b910 con 0x7f5180159e90 2023-10-16T09:48:44.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.234+0000 7f51777fe640 1 -- 172.21.15.157:0/3356378561 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5178049400 con 0x7f5180159e90 2023-10-16T09:48:44.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.234+0000 7f51777fe640 1 -- 172.21.15.157:0/3356378561 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f517803e7b0 con 0x7f5180159e90 2023-10-16T09:48:44.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.235+0000 7f5184d68640 1 -- 172.21.15.157:0/3356378561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518004e160 con 0x7f5180159e90 2023-10-16T09:48:44.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.237+0000 7f51777fe640 1 -- 172.21.15.157:0/3356378561 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f5178067b40 con 0x7f5180159e90 2023-10-16T09:48:44.254 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:44.252+0000 7fa84ea2b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T09:48:44.266 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:44.265+0000 7f1aece12040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T09:48:44.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.315+0000 7f5184d68640 1 -- 172.21.15.157:0/3356378561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "306aa004-b7d8-4c60-b346-925524c4d79f", "id": 1} v 0) v1 -- 0x7f51800ccb60 con 0x7f5180159e90 2023-10-16T09:48:44.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.322+0000 7f51777fe640 1 -- 172.21.15.157:0/3356378561 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "306aa004-b7d8-4c60-b346-925524c4d79f", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5178056030 con 0x7f5180159e90 2023-10-16T09:48:44.323 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-10-16T09:48:44.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.324+0000 7f5184d68640 1 -- 172.21.15.157:0/3356378561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5180159e90 msgr2=0x7f518014ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:44.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.324+0000 7f5184d68640 1 --2- 172.21.15.157:0/3356378561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5180159e90 0x7f518014ab60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5178009dc0 tx=0x7f5178004a40 comp rx=0 tx=0).stop 2023-10-16T09:48:44.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.324+0000 7f5184d68640 1 -- 172.21.15.157:0/3356378561 shutdown_connections 2023-10-16T09:48:44.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.324+0000 7f5184d68640 1 --2- 172.21.15.157:0/3356378561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5180159e90 0x7f518014ab60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.324+0000 7f5184d68640 1 --2- 172.21.15.157:0/3356378561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5180159ab0 0x7f518014a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.324+0000 7f5184d68640 1 --2- 172.21.15.157:0/3356378561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f518006c850 0x7f51800acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.324+0000 7f5184d68640 1 -- 172.21.15.157:0/3356378561 >> 172.21.15.157:0/3356378561 conn(0x7f51800ce9b0 msgr2=0x7f51800ae3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:44.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.324+0000 7f5184d68640 1 -- 172.21.15.157:0/3356378561 shutdown_connections 2023-10-16T09:48:44.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.324+0000 7f5184d68640 1 -- 172.21.15.157:0/3356378561 wait complete. 2023-10-16T09:48:44.341 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 807fffa0-14de-4fac-bfc9-ef94715af8f0 2 2023-10-16T09:48:44.348 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:44.344+0000 7f1aece12040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T09:48:44.367 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:44.365+0000 7fa84ea2b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T09:48:44.397 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:44.397+0000 7f1aece12040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T09:48:44.446 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:44.445+0000 7fa84ea2b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T09:48:44.500 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:44.498+0000 7fa84ea2b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T09:48:44.516 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:44.516+0000 7f1aece12040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T09:48:44.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.536+0000 7f954a20d640 1 Processor -- start 2023-10-16T09:48:44.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.536+0000 7f954a20d640 1 -- start start 2023-10-16T09:48:44.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.538+0000 7f954a20d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f954415b320 0x7f95440cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:44.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.538+0000 7f954a20d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f954415b9d0 con 0x7f954406c040 2023-10-16T09:48:44.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.538+0000 7f954a20d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f954415bcb0 con 0x7f95440ad970 2023-10-16T09:48:44.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.538+0000 7f954a20d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954415b7d0 con 0x7f954415b320 2023-10-16T09:48:44.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.538+0000 7f9549a0c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f954415b320 0x7f95440cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:44.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.538+0000 7f9548a0a640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f95440ad970 0x7f95440cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37528/0 (socket says 172.21.15.157:37528) 2023-10-16T09:48:44.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.538+0000 7f9548a0a640 1 -- 172.21.15.157:0/3930621729 learned_addr learned my addr 172.21.15.157:0/3930621729 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-16T09:48:44.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.539+0000 7f9549a0c640 1 -- 172.21.15.157:0/3930621729 >> v1:172.21.15.157:6789/0 conn(0x7f95440ad970 legacy=0x7f95440cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:44.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.539+0000 7f9549a0c640 1 -- 172.21.15.157:0/3930621729 >> v1:172.21.15.84:6789/0 conn(0x7f954406c040 legacy=0x7f954406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:44.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.539+0000 7f953bfff640 1 -- 172.21.15.157:0/3930621729 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2434417592 0 0) 0x7f954415bcb0 con 0x7f95440ad970 2023-10-16T09:48:44.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.539+0000 7f9549a0c640 1 -- 172.21.15.157:0/3930621729 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954415bf90 con 0x7f954415b320 2023-10-16T09:48:44.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.540+0000 7f9549a0c640 1 --2- 172.21.15.157:0/3930621729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f954415b320 0x7f95440cd180 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f953c009dc0 tx=0x7f953c02bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=899e5d705199ab14 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:44.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.540+0000 7f953bfff640 1 -- 172.21.15.157:0/3930621729 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c02ccc0 con 0x7f954415b320 2023-10-16T09:48:44.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.540+0000 7f953bfff640 1 -- 172.21.15.157:0/3930621729 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c02ce20 con 0x7f954415b320 2023-10-16T09:48:44.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.540+0000 7f953bfff640 1 -- 172.21.15.157:0/3930621729 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c034660 con 0x7f954415b320 2023-10-16T09:48:44.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.541+0000 7f954a20d640 1 -- 172.21.15.157:0/3930621729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f954415b320 msgr2=0x7f95440cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:44.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.541+0000 7f954a20d640 1 --2- 172.21.15.157:0/3930621729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f954415b320 0x7f95440cd180 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f953c009dc0 tx=0x7f953c02bf10 comp rx=0 tx=0).stop 2023-10-16T09:48:44.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.541+0000 7f954a20d640 1 -- 172.21.15.157:0/3930621729 shutdown_connections 2023-10-16T09:48:44.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.541+0000 7f954a20d640 1 --2- 172.21.15.157:0/3930621729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f954415b320 0x7f95440cd180 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.541+0000 7f954a20d640 1 -- 172.21.15.157:0/3930621729 >> 172.21.15.157:0/3930621729 conn(0x7f95440ce6a0 msgr2=0x7f95440ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:44.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.541+0000 7f954a20d640 1 -- 172.21.15.157:0/3930621729 shutdown_connections 2023-10-16T09:48:44.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.541+0000 7f954a20d640 1 -- 172.21.15.157:0/3930621729 wait complete. 2023-10-16T09:48:44.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.543+0000 7f954a20d640 1 Processor -- start 2023-10-16T09:48:44.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.543+0000 7f954a20d640 1 -- start start 2023-10-16T09:48:44.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.544+0000 7f954a20d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f954406c040 0x7f9544152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:44.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.544+0000 7f954a20d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95440ad970 0x7f95440aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:44.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.544+0000 7f954a20d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f954415b320 0x7f95440af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:44.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.544+0000 7f954a20d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9544153350 con 0x7f95440ad970 2023-10-16T09:48:44.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.544+0000 7f954a20d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95440c7360 con 0x7f954406c040 2023-10-16T09:48:44.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.544+0000 7f954a20d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95440c2c60 con 0x7f954415b320 2023-10-16T09:48:44.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.544+0000 7f954920b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f954406c040 0x7f9544152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:44.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.544+0000 7f9549a0c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f954415b320 0x7f95440af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:44.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.544+0000 7f9549a0c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f954415b320 0x7f95440af4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:58604/0 (socket says 172.21.15.157:58604) 2023-10-16T09:48:44.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.544+0000 7f954920b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f954406c040 0x7f9544152e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51142/0 (socket says 172.21.15.157:51142) 2023-10-16T09:48:44.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.544+0000 7f9549a0c640 1 -- 172.21.15.157:0/1505445336 learned_addr learned my addr 172.21.15.157:0/1505445336 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-16T09:48:44.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.544+0000 7f9548a0a640 1 --2- 172.21.15.157:0/1505445336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95440ad970 0x7f95440aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:44.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.544+0000 7f9549a0c640 1 -- 172.21.15.157:0/1505445336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f954406c040 msgr2=0x7f9544152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:44.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.544+0000 7f9549a0c640 1 --2- 172.21.15.157:0/1505445336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f954406c040 0x7f9544152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.544+0000 7f9549a0c640 1 -- 172.21.15.157:0/1505445336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95440ad970 msgr2=0x7f95440aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:44.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.544+0000 7f9549a0c640 1 --2- 172.21.15.157:0/1505445336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95440ad970 0x7f95440aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.545+0000 7f9549a0c640 1 -- 172.21.15.157:0/1505445336 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c009a70 con 0x7f954415b320 2023-10-16T09:48:44.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.545+0000 7f9549a0c640 1 --2- 172.21.15.157:0/1505445336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f954415b320 0x7f95440af4a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f953c009eb0 tx=0x7f953c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:44.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.545+0000 7f9539ffb640 1 -- 172.21.15.157:0/1505445336 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c03e610 con 0x7f954415b320 2023-10-16T09:48:44.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.545+0000 7f9539ffb640 1 -- 172.21.15.157:0/1505445336 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c03e770 con 0x7f954415b320 2023-10-16T09:48:44.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.545+0000 7f9539ffb640 1 -- 172.21.15.157:0/1505445336 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c03eae0 con 0x7f954415b320 2023-10-16T09:48:44.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.545+0000 7f954a20d640 1 -- 172.21.15.157:0/1505445336 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95440af9e0 con 0x7f954415b320 2023-10-16T09:48:44.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.545+0000 7f954a20d640 1 -- 172.21.15.157:0/1505445336 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95440c8200 con 0x7f954415b320 2023-10-16T09:48:44.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.546+0000 7f9539ffb640 1 -- 172.21.15.157:0/1505445336 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f953c03b050 con 0x7f954415b320 2023-10-16T09:48:44.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.546+0000 7f9539ffb640 1 -- 172.21.15.157:0/1505445336 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f953c048d10 con 0x7f954415b320 2023-10-16T09:48:44.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.546+0000 7f954a20d640 1 -- 172.21.15.157:0/1505445336 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954406b330 con 0x7f954415b320 2023-10-16T09:48:44.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.549+0000 7f9539ffb640 1 -- 172.21.15.157:0/1505445336 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f953c04d060 con 0x7f954415b320 2023-10-16T09:48:44.560 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:44.558+0000 7fa84ea2b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T09:48:44.601 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:44.601+0000 7f1aece12040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T09:48:44.614 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:44.612+0000 7fa84ea2b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T09:48:44.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.625+0000 7f954a20d640 1 -- 172.21.15.157:0/1505445336 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "807fffa0-14de-4fac-bfc9-ef94715af8f0", "id": 2} v 0) v1 -- 0x7f95440c2df0 con 0x7f954415b320 2023-10-16T09:48:44.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.633+0000 7f9539ffb640 1 -- 172.21.15.157:0/1505445336 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "807fffa0-14de-4fac-bfc9-ef94715af8f0", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f953c044070 con 0x7f954415b320 2023-10-16T09:48:44.635 INFO:teuthology.orchestra.run.smithi157.stdout:2 2023-10-16T09:48:44.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.634+0000 7f954a20d640 1 -- 172.21.15.157:0/1505445336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f954415b320 msgr2=0x7f95440af4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:44.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.634+0000 7f954a20d640 1 --2- 172.21.15.157:0/1505445336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f954415b320 0x7f95440af4a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f953c009eb0 tx=0x7f953c004ab0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.634+0000 7f954a20d640 1 -- 172.21.15.157:0/1505445336 shutdown_connections 2023-10-16T09:48:44.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.634+0000 7f954a20d640 1 --2- 172.21.15.157:0/1505445336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f954415b320 0x7f95440af4a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.635+0000 7f954a20d640 1 --2- 172.21.15.157:0/1505445336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95440ad970 0x7f95440aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.635+0000 7f954a20d640 1 --2- 172.21.15.157:0/1505445336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f954406c040 0x7f9544152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.635+0000 7f954a20d640 1 -- 172.21.15.157:0/1505445336 >> 172.21.15.157:0/1505445336 conn(0x7f95440ce6a0 msgr2=0x7f95440cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:44.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.635+0000 7f954a20d640 1 -- 172.21.15.157:0/1505445336 shutdown_connections 2023-10-16T09:48:44.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.635+0000 7f954a20d640 1 -- 172.21.15.157:0/1505445336 wait complete. 2023-10-16T09:48:44.653 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 300bbe2b-e619-4d94-963f-1ba6e567f666 3 2023-10-16T09:48:44.659 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:44.657+0000 7f1aece12040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T09:48:44.722 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:44.720+0000 7f1aece12040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T09:48:44.781 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:44.780+0000 7f1aece12040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T09:48:44.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.840+0000 7fa94ffff640 1 Processor -- start 2023-10-16T09:48:44.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.841+0000 7fa94ffff640 1 -- start start 2023-10-16T09:48:44.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.842+0000 7fa94ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa95015cd70 0x7fa9500d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:44.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.842+0000 7fa94ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa9500b8480 0x7fa9500cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:44.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.842+0000 7fa94ffff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95015d150 con 0x7fa9500d0be0 2023-10-16T09:48:44.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.842+0000 7fa94ffff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9500d1180 con 0x7fa9500b8480 2023-10-16T09:48:44.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.842+0000 7fa94ffff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9500d1460 con 0x7fa95015cd70 2023-10-16T09:48:44.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.843+0000 7fa94effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa95015cd70 0x7fa9500d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:44.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.843+0000 7fa94effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa95015cd70 0x7fa9500d2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51150/0 (socket says 172.21.15.157:51150) 2023-10-16T09:48:44.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.843+0000 7fa94effd640 1 -- 172.21.15.157:0/705799910 learned_addr learned my addr 172.21.15.157:0/705799910 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-16T09:48:44.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.843+0000 7fa947fff640 1 --2- 172.21.15.157:0/705799910 >> v2:172.21.15.84:3300/0 conn(0x7fa9500b8480 0x7fa9500cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:44.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.843+0000 7fa94effd640 1 -- 172.21.15.157:0/705799910 >> v2:172.21.15.84:3300/0 conn(0x7fa9500b8480 msgr2=0x7fa9500cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:44.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.843+0000 7fa94effd640 1 --2- 172.21.15.157:0/705799910 >> v2:172.21.15.84:3300/0 conn(0x7fa9500b8480 0x7fa9500cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.843+0000 7fa94effd640 1 -- 172.21.15.157:0/705799910 >> v1:172.21.15.84:6789/0 conn(0x7fa9500d0be0 legacy=0x7fa9500b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:44.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.843+0000 7fa94effd640 1 -- 172.21.15.157:0/705799910 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9500d1740 con 0x7fa95015cd70 2023-10-16T09:48:44.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.844+0000 7fa94effd640 1 --2- 172.21.15.157:0/705799910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa95015cd70 0x7fa9500d2220 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa9500b56a0 tx=0x7fa94002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=862b1d44587296d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:44.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.845+0000 7fa94e7fc640 1 -- 172.21.15.157:0/705799910 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940031870 con 0x7fa95015cd70 2023-10-16T09:48:44.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.845+0000 7fa94e7fc640 1 -- 172.21.15.157:0/705799910 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9400319d0 con 0x7fa95015cd70 2023-10-16T09:48:44.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.845+0000 7fa94e7fc640 1 -- 172.21.15.157:0/705799910 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940031d60 con 0x7fa95015cd70 2023-10-16T09:48:44.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.845+0000 7fa94ffff640 1 -- 172.21.15.157:0/705799910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa95015cd70 msgr2=0x7fa9500d2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:44.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.845+0000 7fa94ffff640 1 --2- 172.21.15.157:0/705799910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa95015cd70 0x7fa9500d2220 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa9500b56a0 tx=0x7fa94002b910 comp rx=0 tx=0).stop 2023-10-16T09:48:44.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.845+0000 7fa94ffff640 1 -- 172.21.15.157:0/705799910 shutdown_connections 2023-10-16T09:48:44.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.845+0000 7fa94ffff640 1 --2- 172.21.15.157:0/705799910 >> v2:172.21.15.84:3300/0 conn(0x7fa9500b8480 0x7fa9500cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.845+0000 7fa94ffff640 1 --2- 172.21.15.157:0/705799910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa95015cd70 0x7fa9500d2220 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.845+0000 7fa94ffff640 1 -- 172.21.15.157:0/705799910 >> 172.21.15.157:0/705799910 conn(0x7fa9500cf570 msgr2=0x7fa9500cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:44.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.845+0000 7fa94ffff640 1 -- 172.21.15.157:0/705799910 shutdown_connections 2023-10-16T09:48:44.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.845+0000 7fa94ffff640 1 -- 172.21.15.157:0/705799910 wait complete. 2023-10-16T09:48:44.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.847+0000 7fa94ffff640 1 Processor -- start 2023-10-16T09:48:44.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.847+0000 7fa94ffff640 1 -- start start 2023-10-16T09:48:44.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.848+0000 7fa94ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9500b8480 0x7fa9500b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:44.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.848+0000 7fa94ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9500d0be0 0x7fa95014f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:44.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.848+0000 7fa94ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa95015cd70 0x7fa9500cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:44.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.848+0000 7fa94ffff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9501f2470 con 0x7fa9500b8480 2023-10-16T09:48:44.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.848+0000 7fa94ffff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9500cc460 con 0x7fa9500d0be0 2023-10-16T09:48:44.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.848+0000 7fa94ffff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa950154460 con 0x7fa95015cd70 2023-10-16T09:48:44.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.848+0000 7fa947fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9500d0be0 0x7fa95014f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:44.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.848+0000 7fa94f7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa95015cd70 0x7fa9500cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:44.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.848+0000 7fa947fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9500d0be0 0x7fa95014f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51450/0 (socket says 172.21.15.157:51450) 2023-10-16T09:48:44.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.848+0000 7fa947fff640 1 -- 172.21.15.157:0/4009660189 learned_addr learned my addr 172.21.15.157:0/4009660189 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-16T09:48:44.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.849+0000 7fa94effd640 1 --2- 172.21.15.157:0/4009660189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9500b8480 0x7fa9500b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:44.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.849+0000 7fa947fff640 1 -- 172.21.15.157:0/4009660189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa95015cd70 msgr2=0x7fa9500cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:44.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.849+0000 7fa947fff640 1 --2- 172.21.15.157:0/4009660189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa95015cd70 0x7fa9500cbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.849+0000 7fa947fff640 1 -- 172.21.15.157:0/4009660189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9500b8480 msgr2=0x7fa9500b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:44.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.849+0000 7fa947fff640 1 --2- 172.21.15.157:0/4009660189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9500b8480 0x7fa9500b2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.849+0000 7fa947fff640 1 -- 172.21.15.157:0/4009660189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9400094d0 con 0x7fa9500d0be0 2023-10-16T09:48:44.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.849+0000 7fa947fff640 1 --2- 172.21.15.157:0/4009660189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9500d0be0 0x7fa95014f670 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa93800d5d0 tx=0x7fa93800da90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:44.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.849+0000 7fa9477fe640 1 -- 172.21.15.157:0/4009660189 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9380125f0 con 0x7fa9500d0be0 2023-10-16T09:48:44.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.849+0000 7fa9477fe640 1 -- 172.21.15.157:0/4009660189 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa938012770 con 0x7fa9500d0be0 2023-10-16T09:48:44.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.849+0000 7fa9477fe640 1 -- 172.21.15.157:0/4009660189 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa938012b20 con 0x7fa9500d0be0 2023-10-16T09:48:44.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.849+0000 7fa94ffff640 1 -- 172.21.15.157:0/4009660189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa950154740 con 0x7fa9500d0be0 2023-10-16T09:48:44.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.850+0000 7fa94ffff640 1 -- 172.21.15.157:0/4009660189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95014fbb0 con 0x7fa9500d0be0 2023-10-16T09:48:44.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.850+0000 7fa9477fe640 1 -- 172.21.15.157:0/4009660189 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa93801e460 con 0x7fa9500d0be0 2023-10-16T09:48:44.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.850+0000 7fa9477fe640 1 -- 172.21.15.157:0/4009660189 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fa938014a50 con 0x7fa9500d0be0 2023-10-16T09:48:44.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.851+0000 7fa94ffff640 1 -- 172.21.15.157:0/4009660189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95004e9b0 con 0x7fa9500d0be0 2023-10-16T09:48:44.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.853+0000 7fa9477fe640 1 -- 172.21.15.157:0/4009660189 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fa938014cb0 con 0x7fa9500d0be0 2023-10-16T09:48:44.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.931+0000 7fa94ffff640 1 -- 172.21.15.157:0/4009660189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "300bbe2b-e619-4d94-963f-1ba6e567f666", "id": 3} v 0) v1 -- 0x7fa9500b7600 con 0x7fa9500d0be0 2023-10-16T09:48:44.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.938+0000 7fa9477fe640 1 -- 172.21.15.157:0/4009660189 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "300bbe2b-e619-4d94-963f-1ba6e567f666", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa9380145c0 con 0x7fa9500d0be0 2023-10-16T09:48:44.940 INFO:teuthology.orchestra.run.smithi157.stdout:3 2023-10-16T09:48:44.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.939+0000 7fa94ffff640 1 -- 172.21.15.157:0/4009660189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9500d0be0 msgr2=0x7fa95014f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:44.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.939+0000 7fa94ffff640 1 --2- 172.21.15.157:0/4009660189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9500d0be0 0x7fa95014f670 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa93800d5d0 tx=0x7fa93800da90 comp rx=0 tx=0).stop 2023-10-16T09:48:44.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.939+0000 7fa94ffff640 1 -- 172.21.15.157:0/4009660189 shutdown_connections 2023-10-16T09:48:44.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.939+0000 7fa94ffff640 1 --2- 172.21.15.157:0/4009660189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa95015cd70 0x7fa9500cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.940+0000 7fa94ffff640 1 --2- 172.21.15.157:0/4009660189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9500d0be0 0x7fa95014f670 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.940+0000 7fa94ffff640 1 --2- 172.21.15.157:0/4009660189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9500b8480 0x7fa9500b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:44.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.940+0000 7fa94ffff640 1 -- 172.21.15.157:0/4009660189 >> 172.21.15.157:0/4009660189 conn(0x7fa9500cf570 msgr2=0x7fa9500cfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:44.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.940+0000 7fa94ffff640 1 -- 172.21.15.157:0/4009660189 shutdown_connections 2023-10-16T09:48:44.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:44.940+0000 7fa94ffff640 1 -- 172.21.15.157:0/4009660189 wait complete. 2023-10-16T09:48:44.958 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 9067ece2-38cc-475c-981a-fe25c90cf9bd 4 2023-10-16T09:48:45.066 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:45.064+0000 7fa84ea2b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T09:48:45.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.150+0000 7f76e3fff640 1 Processor -- start 2023-10-16T09:48:45.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.150+0000 7f76e3fff640 1 -- start start 2023-10-16T09:48:45.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.152+0000 7f76e3fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f76e415cd70 0x7f76e40d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:45.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.152+0000 7f76e3fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76e40b8480 0x7f76e40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:45.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.152+0000 7f76e3fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e415d150 con 0x7f76e40d0be0 2023-10-16T09:48:45.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.152+0000 7f76e3fff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e40d1180 con 0x7f76e415cd70 2023-10-16T09:48:45.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.152+0000 7f76e3fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e40d1460 con 0x7f76e40b8480 2023-10-16T09:48:45.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.152+0000 7f76e27fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76e40b8480 0x7f76e40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:45.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.152+0000 7f76e2ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f76e415cd70 0x7f76e40d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:45.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.152+0000 7f76e27fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76e40b8480 0x7f76e40cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:38254/0 (socket says 172.21.15.157:38254) 2023-10-16T09:48:45.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.152+0000 7f76e27fc640 1 -- 172.21.15.157:0/1153741191 learned_addr learned my addr 172.21.15.157:0/1153741191 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-16T09:48:45.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.153+0000 7f76e27fc640 1 -- 172.21.15.157:0/1153741191 >> v2:172.21.15.157:3300/0 conn(0x7f76e415cd70 msgr2=0x7f76e40d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:45.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.153+0000 7f76e27fc640 1 --2- 172.21.15.157:0/1153741191 >> v2:172.21.15.157:3300/0 conn(0x7f76e415cd70 0x7f76e40d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.153+0000 7f76e27fc640 1 -- 172.21.15.157:0/1153741191 >> v1:172.21.15.84:6789/0 conn(0x7f76e40d0be0 legacy=0x7f76e40b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:45.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.153+0000 7f76e27fc640 1 -- 172.21.15.157:0/1153741191 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e40d1740 con 0x7f76e40b8480 2023-10-16T09:48:45.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.153+0000 7f76e2ffd640 1 --2- 172.21.15.157:0/1153741191 >> v2:172.21.15.157:3300/0 conn(0x7f76e415cd70 0x7f76e40d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:48:45.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.154+0000 7f76e27fc640 1 --2- 172.21.15.157:0/1153741191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76e40b8480 0x7f76e40cd7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009870 tx=0x7f76d802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b895c7e30097bb25 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:45.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.154+0000 7f76e1ffb640 1 -- 172.21.15.157:0/1153741191 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8030840 con 0x7f76e40b8480 2023-10-16T09:48:45.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.154+0000 7f76e1ffb640 1 -- 172.21.15.157:0/1153741191 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d80309a0 con 0x7f76e40b8480 2023-10-16T09:48:45.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.154+0000 7f76e1ffb640 1 -- 172.21.15.157:0/1153741191 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8030d30 con 0x7f76e40b8480 2023-10-16T09:48:45.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.155+0000 7f76e3fff640 1 -- 172.21.15.157:0/1153741191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76e40b8480 msgr2=0x7f76e40cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:45.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.155+0000 7f76e3fff640 1 --2- 172.21.15.157:0/1153741191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76e40b8480 0x7f76e40cd7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009870 tx=0x7f76d802b980 comp rx=0 tx=0).stop 2023-10-16T09:48:45.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.155+0000 7f76e3fff640 1 -- 172.21.15.157:0/1153741191 shutdown_connections 2023-10-16T09:48:45.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.155+0000 7f76e3fff640 1 --2- 172.21.15.157:0/1153741191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76e40b8480 0x7f76e40cd7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.155+0000 7f76e3fff640 1 --2- 172.21.15.157:0/1153741191 >> v2:172.21.15.157:3300/0 conn(0x7f76e415cd70 0x7f76e40d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.155+0000 7f76e3fff640 1 -- 172.21.15.157:0/1153741191 >> 172.21.15.157:0/1153741191 conn(0x7f76e40cf570 msgr2=0x7f76e40cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:45.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.155+0000 7f76e3fff640 1 -- 172.21.15.157:0/1153741191 shutdown_connections 2023-10-16T09:48:45.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.155+0000 7f76e3fff640 1 -- 172.21.15.157:0/1153741191 wait complete. 2023-10-16T09:48:45.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.156+0000 7f76e3fff640 1 Processor -- start 2023-10-16T09:48:45.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.157+0000 7f76e3fff640 1 -- start start 2023-10-16T09:48:45.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.157+0000 7f76e3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76e40b8480 0x7f76e414a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:45.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.158+0000 7f76e3fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76e40d0be0 0x7f76e4148b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:45.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.158+0000 7f76e3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76e415cd70 0x7f76e41f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:45.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.158+0000 7f76e3fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e4149450 con 0x7f76e415cd70 2023-10-16T09:48:45.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.158+0000 7f76e3fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e4149730 con 0x7f76e40b8480 2023-10-16T09:48:45.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.158+0000 7f76e3fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e4149a10 con 0x7f76e40d0be0 2023-10-16T09:48:45.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.158+0000 7f76e2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76e40b8480 0x7f76e414a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:45.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.158+0000 7f76e27fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76e40d0be0 0x7f76e4148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:45.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.158+0000 7f76e2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76e40b8480 0x7f76e414a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51464/0 (socket says 172.21.15.157:51464) 2023-10-16T09:48:45.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.158+0000 7f76e2ffd640 1 -- 172.21.15.157:0/3815567053 learned_addr learned my addr 172.21.15.157:0/3815567053 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-16T09:48:45.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.158+0000 7f76e37fe640 1 --2- 172.21.15.157:0/3815567053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76e415cd70 0x7f76e41f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:45.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.158+0000 7f76e27fc640 1 -- 172.21.15.157:0/3815567053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76e40b8480 msgr2=0x7f76e414a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:45.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.158+0000 7f76e27fc640 1 --2- 172.21.15.157:0/3815567053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76e40b8480 0x7f76e414a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.158+0000 7f76e27fc640 1 -- 172.21.15.157:0/3815567053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76e415cd70 msgr2=0x7f76e41f1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:45.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.158+0000 7f76e27fc640 1 --2- 172.21.15.157:0/3815567053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76e415cd70 0x7f76e41f1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.158+0000 7f76e27fc640 1 -- 172.21.15.157:0/3815567053 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d4009400 con 0x7f76e40d0be0 2023-10-16T09:48:45.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.158+0000 7f76e37fe640 1 --2- 172.21.15.157:0/3815567053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76e415cd70 0x7f76e41f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:48:45.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.158+0000 7f76e2ffd640 1 --2- 172.21.15.157:0/3815567053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76e40b8480 0x7f76e414a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:48:45.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.159+0000 7f76e27fc640 1 --2- 172.21.15.157:0/3815567053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76e40d0be0 0x7f76e4148b50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f76d8033000 tx=0x7f76d8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:45.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.159+0000 7f76cb7fe640 1 -- 172.21.15.157:0/3815567053 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8039a10 con 0x7f76e40d0be0 2023-10-16T09:48:45.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.159+0000 7f76cb7fe640 1 -- 172.21.15.157:0/3815567053 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d8039b70 con 0x7f76e40d0be0 2023-10-16T09:48:45.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.159+0000 7f76cb7fe640 1 -- 172.21.15.157:0/3815567053 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8043410 con 0x7f76e40d0be0 2023-10-16T09:48:45.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.159+0000 7f76e3fff640 1 -- 172.21.15.157:0/3815567053 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d80094d0 con 0x7f76e40d0be0 2023-10-16T09:48:45.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.159+0000 7f76e3fff640 1 -- 172.21.15.157:0/3815567053 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e4149fd0 con 0x7f76e40d0be0 2023-10-16T09:48:45.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.159+0000 7f76cb7fe640 1 -- 172.21.15.157:0/3815567053 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f76d8043870 con 0x7f76e40d0be0 2023-10-16T09:48:45.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.159+0000 7f76cb7fe640 1 -- 172.21.15.157:0/3815567053 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f76d802f8a0 con 0x7f76e40d0be0 2023-10-16T09:48:45.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.160+0000 7f76e3fff640 1 -- 172.21.15.157:0/3815567053 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e404e9b0 con 0x7f76e40d0be0 2023-10-16T09:48:45.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.165+0000 7f76cb7fe640 1 -- 172.21.15.157:0/3815567053 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f76d8048070 con 0x7f76e40d0be0 2023-10-16T09:48:45.225 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:45.224+0000 7fa84ea2b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T09:48:45.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.243+0000 7f76e3fff640 1 -- 172.21.15.157:0/3815567053 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9067ece2-38cc-475c-981a-fe25c90cf9bd", "id": 4} v 0) v1 -- 0x7f76e4149b50 con 0x7f76e40d0be0 2023-10-16T09:48:45.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.250+0000 7f76cb7fe640 1 -- 172.21.15.157:0/3815567053 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9067ece2-38cc-475c-981a-fe25c90cf9bd", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f76e4149b50 con 0x7f76e40d0be0 2023-10-16T09:48:45.252 INFO:teuthology.orchestra.run.smithi157.stdout:4 2023-10-16T09:48:45.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.252+0000 7f76e3fff640 1 -- 172.21.15.157:0/3815567053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76e40d0be0 msgr2=0x7f76e4148b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:45.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.252+0000 7f76e3fff640 1 --2- 172.21.15.157:0/3815567053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76e40d0be0 0x7f76e4148b50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f76d8033000 tx=0x7f76d8004730 comp rx=0 tx=0).stop 2023-10-16T09:48:45.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.252+0000 7f76e3fff640 1 -- 172.21.15.157:0/3815567053 shutdown_connections 2023-10-16T09:48:45.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.252+0000 7f76e3fff640 1 --2- 172.21.15.157:0/3815567053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76e415cd70 0x7f76e41f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.252+0000 7f76e3fff640 1 --2- 172.21.15.157:0/3815567053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76e40d0be0 0x7f76e4148b50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.252+0000 7f76e3fff640 1 --2- 172.21.15.157:0/3815567053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76e40b8480 0x7f76e414a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.252+0000 7f76e3fff640 1 -- 172.21.15.157:0/3815567053 >> 172.21.15.157:0/3815567053 conn(0x7f76e40cf570 msgr2=0x7f76e415fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:45.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.252+0000 7f76e3fff640 1 -- 172.21.15.157:0/3815567053 shutdown_connections 2023-10-16T09:48:45.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.252+0000 7f76e3fff640 1 -- 172.21.15.157:0/3815567053 wait complete. 2023-10-16T09:48:45.263 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:45.261+0000 7f1aece12040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T09:48:45.269 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 796e804a-e843-4356-8e30-252ce0dcc18a 5 2023-10-16T09:48:45.350 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:45.348+0000 7fa84ea2b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T09:48:45.403 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:45.401+0000 7fa84ea2b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T09:48:45.431 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:45.430+0000 7f1aece12040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T09:48:45.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.462+0000 7f81a5c60640 1 Processor -- start 2023-10-16T09:48:45.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.462+0000 7f81a5c60640 1 -- start start 2023-10-16T09:48:45.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.463+0000 7f81a5c60640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81a00b8480 0x7f81a00cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:45.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.464+0000 7f81a5c60640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a015d150 con 0x7f81a00d0be0 2023-10-16T09:48:45.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.464+0000 7f81a5c60640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a00d1180 con 0x7f81a015cd70 2023-10-16T09:48:45.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.464+0000 7f81a5c60640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00d1460 con 0x7f81a00b8480 2023-10-16T09:48:45.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.464+0000 7f819ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81a00b8480 0x7f81a00cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:45.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.464+0000 7f81a4c5e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f81a015cd70 0x7f81a00d2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:46454/0 (socket says 172.21.15.157:46454) 2023-10-16T09:48:45.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.464+0000 7f81a4c5e640 1 -- 172.21.15.157:0/3683647810 learned_addr learned my addr 172.21.15.157:0/3683647810 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-16T09:48:45.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.464+0000 7f819ffff640 1 -- 172.21.15.157:0/3683647810 >> v1:172.21.15.157:6789/0 conn(0x7f81a015cd70 legacy=0x7f81a00d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:45.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.464+0000 7f819ffff640 1 -- 172.21.15.157:0/3683647810 >> v1:172.21.15.84:6789/0 conn(0x7f81a00d0be0 legacy=0x7f81a00b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:45.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.464+0000 7f819ffff640 1 -- 172.21.15.157:0/3683647810 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a00d1740 con 0x7f81a00b8480 2023-10-16T09:48:45.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.464+0000 7f819f7fe640 1 -- 172.21.15.157:0/3683647810 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2680228267 0 0) 0x7f81a00d1180 con 0x7f81a015cd70 2023-10-16T09:48:45.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.465+0000 7f819ffff640 1 --2- 172.21.15.157:0/3683647810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81a00b8480 0x7f81a00cd7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f818c0097d0 tx=0x7f818c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=843e243586802b28 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:45.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.466+0000 7f819f7fe640 1 -- 172.21.15.157:0/3683647810 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c031800 con 0x7f81a00b8480 2023-10-16T09:48:45.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.466+0000 7f819f7fe640 1 -- 172.21.15.157:0/3683647810 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818c031960 con 0x7f81a00b8480 2023-10-16T09:48:45.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.466+0000 7f819f7fe640 1 -- 172.21.15.157:0/3683647810 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c031cf0 con 0x7f81a00b8480 2023-10-16T09:48:45.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.466+0000 7f81a5c60640 1 -- 172.21.15.157:0/3683647810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81a00b8480 msgr2=0x7f81a00cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:45.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.466+0000 7f81a5c60640 1 --2- 172.21.15.157:0/3683647810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81a00b8480 0x7f81a00cd7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f818c0097d0 tx=0x7f818c02b8a0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.466+0000 7f81a5c60640 1 -- 172.21.15.157:0/3683647810 shutdown_connections 2023-10-16T09:48:45.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.466+0000 7f81a5c60640 1 --2- 172.21.15.157:0/3683647810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81a00b8480 0x7f81a00cd7c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.466+0000 7f81a5c60640 1 -- 172.21.15.157:0/3683647810 >> 172.21.15.157:0/3683647810 conn(0x7f81a00cf570 msgr2=0x7f81a00cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:45.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.466+0000 7f81a5c60640 1 -- 172.21.15.157:0/3683647810 shutdown_connections 2023-10-16T09:48:45.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.467+0000 7f81a5c60640 1 -- 172.21.15.157:0/3683647810 wait complete. 2023-10-16T09:48:45.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.468+0000 7f81a5c60640 1 Processor -- start 2023-10-16T09:48:45.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.468+0000 7f81a5c60640 1 -- start start 2023-10-16T09:48:45.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.469+0000 7f81a5c60640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81a00b8480 0x7f81a015faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:45.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.469+0000 7f81a5c60640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81a00d0be0 0x7f81a00c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:45.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.469+0000 7f81a5c60640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81a015cd70 0x7f81a00ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:45.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.469+0000 7f81a5c60640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00c81c0 con 0x7f81a015cd70 2023-10-16T09:48:45.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.469+0000 7f81a5c60640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00c84a0 con 0x7f81a00b8480 2023-10-16T09:48:45.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.469+0000 7f81a5c60640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00c8780 con 0x7f81a00d0be0 2023-10-16T09:48:45.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.470+0000 7f81a4c5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81a00b8480 0x7f81a015faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:45.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.470+0000 7f81a4c5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81a00b8480 0x7f81a015faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51470/0 (socket says 172.21.15.157:51470) 2023-10-16T09:48:45.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.470+0000 7f81a4c5e640 1 -- 172.21.15.157:0/3884392948 learned_addr learned my addr 172.21.15.157:0/3884392948 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-16T09:48:45.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.470+0000 7f819ffff640 1 --2- 172.21.15.157:0/3884392948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81a00d0be0 0x7f81a00c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:45.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.470+0000 7f81a545f640 1 --2- 172.21.15.157:0/3884392948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81a015cd70 0x7f81a00ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:45.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.470+0000 7f81a4c5e640 1 -- 172.21.15.157:0/3884392948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81a00d0be0 msgr2=0x7f81a00c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:45.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.470+0000 7f81a4c5e640 1 --2- 172.21.15.157:0/3884392948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81a00d0be0 0x7f81a00c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.470+0000 7f81a4c5e640 1 -- 172.21.15.157:0/3884392948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81a015cd70 msgr2=0x7f81a00ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:45.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.470+0000 7f81a4c5e640 1 --2- 172.21.15.157:0/3884392948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81a015cd70 0x7f81a00ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.470+0000 7f81a4c5e640 1 -- 172.21.15.157:0/3884392948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c009400 con 0x7f81a00b8480 2023-10-16T09:48:45.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.470+0000 7f81a4c5e640 1 --2- 172.21.15.157:0/3884392948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81a00b8480 0x7f81a015faf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f819400c4d0 tx=0x7f819400c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:45.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.471+0000 7f819d7fa640 1 -- 172.21.15.157:0/3884392948 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81940074e0 con 0x7f81a00b8480 2023-10-16T09:48:45.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.471+0000 7f819d7fa640 1 -- 172.21.15.157:0/3884392948 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8194007660 con 0x7f81a00b8480 2023-10-16T09:48:45.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.471+0000 7f819d7fa640 1 -- 172.21.15.157:0/3884392948 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194004a20 con 0x7f81a00b8480 2023-10-16T09:48:45.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.471+0000 7f81a5c60640 1 -- 172.21.15.157:0/3884392948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a00c8a60 con 0x7f81a00b8480 2023-10-16T09:48:45.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.471+0000 7f81a5c60640 1 -- 172.21.15.157:0/3884392948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a00c7e40 con 0x7f81a00b8480 2023-10-16T09:48:45.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.471+0000 7f819d7fa640 1 -- 172.21.15.157:0/3884392948 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8194012400 con 0x7f81a00b8480 2023-10-16T09:48:45.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.471+0000 7f819d7fa640 1 -- 172.21.15.157:0/3884392948 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f8194012c30 con 0x7f81a00b8480 2023-10-16T09:48:45.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.472+0000 7f81a5c60640 1 -- 172.21.15.157:0/3884392948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a0142d40 con 0x7f81a00b8480 2023-10-16T09:48:45.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.474+0000 7f819d7fa640 1 -- 172.21.15.157:0/3884392948 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f8194018070 con 0x7f81a00b8480 2023-10-16T09:48:45.513 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:45.511+0000 7fa84ea2b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:48:45.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.552+0000 7f81a5c60640 1 -- 172.21.15.157:0/3884392948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "796e804a-e843-4356-8e30-252ce0dcc18a", "id": 5} v 0) v1 -- 0x7f81a004e9b0 con 0x7f81a00b8480 2023-10-16T09:48:45.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.558+0000 7f819d7fa640 1 -- 172.21.15.157:0/3884392948 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "796e804a-e843-4356-8e30-252ce0dcc18a", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f81940126b0 con 0x7f81a00b8480 2023-10-16T09:48:45.560 INFO:teuthology.orchestra.run.smithi157.stdout:5 2023-10-16T09:48:45.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.560+0000 7f81a5c60640 1 -- 172.21.15.157:0/3884392948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81a00b8480 msgr2=0x7f81a015faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:45.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.560+0000 7f81a5c60640 1 --2- 172.21.15.157:0/3884392948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81a00b8480 0x7f81a015faf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f819400c4d0 tx=0x7f819400c990 comp rx=0 tx=0).stop 2023-10-16T09:48:45.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.560+0000 7f81a5c60640 1 -- 172.21.15.157:0/3884392948 shutdown_connections 2023-10-16T09:48:45.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.560+0000 7f81a5c60640 1 --2- 172.21.15.157:0/3884392948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81a015cd70 0x7f81a00ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.560+0000 7f81a5c60640 1 --2- 172.21.15.157:0/3884392948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81a00d0be0 0x7f81a00c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.560+0000 7f81a5c60640 1 --2- 172.21.15.157:0/3884392948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81a00b8480 0x7f81a015faf0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.560+0000 7f81a5c60640 1 -- 172.21.15.157:0/3884392948 >> 172.21.15.157:0/3884392948 conn(0x7f81a00cf570 msgr2=0x7f81a015bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:45.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.560+0000 7f81a5c60640 1 -- 172.21.15.157:0/3884392948 shutdown_connections 2023-10-16T09:48:45.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.560+0000 7f81a5c60640 1 -- 172.21.15.157:0/3884392948 wait complete. 2023-10-16T09:48:45.573 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:45.571+0000 7f1aece12040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T09:48:45.578 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 38f95207-b5e2-4d7f-8d92-002ab5421e5b 6 2023-10-16T09:48:45.624 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:45.622+0000 7fa84ea2b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T09:48:45.629 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:45.628+0000 7f1aece12040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T09:48:45.745 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:45.743+0000 7f1aece12040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:48:45.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.770+0000 7f1c3746d640 1 Processor -- start 2023-10-16T09:48:45.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.771+0000 7f1c3746d640 1 -- start start 2023-10-16T09:48:45.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.772+0000 7f1c3746d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1c3015cd70 0x7f1c300d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:45.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.772+0000 7f1c3746d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c300b8480 0x7f1c300cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:45.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.772+0000 7f1c3746d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c3015d150 con 0x7f1c300d0be0 2023-10-16T09:48:45.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.772+0000 7f1c3746d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c300d1180 con 0x7f1c3015cd70 2023-10-16T09:48:45.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.772+0000 7f1c3746d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c300d1460 con 0x7f1c300b8480 2023-10-16T09:48:45.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.772+0000 7f1c35c6a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c300b8480 0x7f1c300cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:45.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.772+0000 7f1c35c6a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c300b8480 0x7f1c300cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:38294/0 (socket says 172.21.15.157:38294) 2023-10-16T09:48:45.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.772+0000 7f1c35c6a640 1 -- 172.21.15.157:0/78803588 learned_addr learned my addr 172.21.15.157:0/78803588 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-16T09:48:45.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.773+0000 7f1c3646b640 1 --2- 172.21.15.157:0/78803588 >> v2:172.21.15.157:3300/0 conn(0x7f1c3015cd70 0x7f1c300d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:45.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.773+0000 7f1c35c6a640 1 -- 172.21.15.157:0/78803588 >> v2:172.21.15.157:3300/0 conn(0x7f1c3015cd70 msgr2=0x7f1c300d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:45.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.773+0000 7f1c35c6a640 1 --2- 172.21.15.157:0/78803588 >> v2:172.21.15.157:3300/0 conn(0x7f1c3015cd70 0x7f1c300d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.773+0000 7f1c35c6a640 1 -- 172.21.15.157:0/78803588 >> v1:172.21.15.157:6789/0 conn(0x7f1c300d0be0 legacy=0x7f1c300b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:45.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.773+0000 7f1c35c6a640 1 -- 172.21.15.157:0/78803588 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c300d1740 con 0x7f1c300b8480 2023-10-16T09:48:45.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.774+0000 7f1c35c6a640 1 --2- 172.21.15.157:0/78803588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c300b8480 0x7f1c300cd7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009870 tx=0x7f1c2002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e01250ed24b8637c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:45.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.774+0000 7f1c35469640 1 -- 172.21.15.157:0/78803588 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20030840 con 0x7f1c300b8480 2023-10-16T09:48:45.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.774+0000 7f1c35469640 1 -- 172.21.15.157:0/78803588 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c200309a0 con 0x7f1c300b8480 2023-10-16T09:48:45.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.774+0000 7f1c35469640 1 -- 172.21.15.157:0/78803588 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20030d30 con 0x7f1c300b8480 2023-10-16T09:48:45.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.775+0000 7f1c3746d640 1 -- 172.21.15.157:0/78803588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c300b8480 msgr2=0x7f1c300cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:45.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.775+0000 7f1c3746d640 1 --2- 172.21.15.157:0/78803588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c300b8480 0x7f1c300cd7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009870 tx=0x7f1c2002b980 comp rx=0 tx=0).stop 2023-10-16T09:48:45.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.775+0000 7f1c3746d640 1 -- 172.21.15.157:0/78803588 shutdown_connections 2023-10-16T09:48:45.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.775+0000 7f1c3746d640 1 --2- 172.21.15.157:0/78803588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c300b8480 0x7f1c300cd7c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.775+0000 7f1c3746d640 1 --2- 172.21.15.157:0/78803588 >> v2:172.21.15.157:3300/0 conn(0x7f1c3015cd70 0x7f1c300d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.775+0000 7f1c3746d640 1 -- 172.21.15.157:0/78803588 >> 172.21.15.157:0/78803588 conn(0x7f1c300cf570 msgr2=0x7f1c300cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:45.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.775+0000 7f1c3746d640 1 -- 172.21.15.157:0/78803588 shutdown_connections 2023-10-16T09:48:45.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.775+0000 7f1c3746d640 1 -- 172.21.15.157:0/78803588 wait complete. 2023-10-16T09:48:45.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.777+0000 7f1c3746d640 1 Processor -- start 2023-10-16T09:48:45.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.777+0000 7f1c3746d640 1 -- start start 2023-10-16T09:48:45.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.778+0000 7f1c3746d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c300b8480 0x7f1c3015d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:45.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.778+0000 7f1c3746d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c300d0be0 0x7f1c301f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:45.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.778+0000 7f1c3746d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c3015cd70 0x7f1c3006bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:45.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.778+0000 7f1c3746d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c301f56f0 con 0x7f1c300b8480 2023-10-16T09:48:45.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.778+0000 7f1c3746d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c300c66a0 con 0x7f1c3015cd70 2023-10-16T09:48:45.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.778+0000 7f1c3746d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c300c6980 con 0x7f1c300d0be0 2023-10-16T09:48:45.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.779+0000 7f1c35c6a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c300d0be0 0x7f1c301f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:45.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.779+0000 7f1c36c6c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c3015cd70 0x7f1c3006bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:45.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.779+0000 7f1c35c6a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c300d0be0 0x7f1c301f5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:38298/0 (socket says 172.21.15.157:38298) 2023-10-16T09:48:45.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.779+0000 7f1c35c6a640 1 -- 172.21.15.157:0/457608554 learned_addr learned my addr 172.21.15.157:0/457608554 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-16T09:48:45.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.779+0000 7f1c3646b640 1 --2- 172.21.15.157:0/457608554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c300b8480 0x7f1c3015d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:45.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.779+0000 7f1c35c6a640 1 -- 172.21.15.157:0/457608554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c3015cd70 msgr2=0x7f1c3006bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:45.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.779+0000 7f1c35c6a640 1 --2- 172.21.15.157:0/457608554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c3015cd70 0x7f1c3006bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.779+0000 7f1c35c6a640 1 -- 172.21.15.157:0/457608554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c300b8480 msgr2=0x7f1c3015d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:45.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.779+0000 7f1c35c6a640 1 --2- 172.21.15.157:0/457608554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c300b8480 0x7f1c3015d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.779+0000 7f1c35c6a640 1 -- 172.21.15.157:0/457608554 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c200094d0 con 0x7f1c300d0be0 2023-10-16T09:48:45.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.779+0000 7f1c3646b640 1 --2- 172.21.15.157:0/457608554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c300b8480 0x7f1c3015d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:48:45.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.779+0000 7f1c36c6c640 1 --2- 172.21.15.157:0/457608554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c3015cd70 0x7f1c3006bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:48:45.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.779+0000 7f1c35c6a640 1 --2- 172.21.15.157:0/457608554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c300d0be0 0x7f1c301f5ed0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1c20000c00 tx=0x7f1c20004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:45.785 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:45.783+0000 7fa84ea2b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:48:45.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.781+0000 7f1c26ffd640 1 -- 172.21.15.157:0/457608554 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20039a90 con 0x7f1c300d0be0 2023-10-16T09:48:45.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.781+0000 7f1c26ffd640 1 -- 172.21.15.157:0/457608554 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c20039bf0 con 0x7f1c300d0be0 2023-10-16T09:48:45.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.781+0000 7f1c26ffd640 1 -- 172.21.15.157:0/457608554 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20043410 con 0x7f1c300d0be0 2023-10-16T09:48:45.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.781+0000 7f1c3746d640 1 -- 172.21.15.157:0/457608554 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c301f59b0 con 0x7f1c300d0be0 2023-10-16T09:48:45.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.781+0000 7f1c3746d640 1 -- 172.21.15.157:0/457608554 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3006c110 con 0x7f1c300d0be0 2023-10-16T09:48:45.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.782+0000 7f1c26ffd640 1 -- 172.21.15.157:0/457608554 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1c20043870 con 0x7f1c300d0be0 2023-10-16T09:48:45.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.782+0000 7f1c26ffd640 1 -- 172.21.15.157:0/457608554 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f1c2004c3f0 con 0x7f1c300d0be0 2023-10-16T09:48:45.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.783+0000 7f1c3746d640 1 -- 172.21.15.157:0/457608554 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c301f5ba0 con 0x7f1c300d0be0 2023-10-16T09:48:45.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.785+0000 7f1c26ffd640 1 -- 172.21.15.157:0/457608554 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f1c20048070 con 0x7f1c300d0be0 2023-10-16T09:48:45.835 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:45.833+0000 7fa84ea2b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T09:48:45.863 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:45.861+0000 7f1aece12040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T09:48:45.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.863+0000 7f1c3746d640 1 -- 172.21.15.157:0/457608554 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "38f95207-b5e2-4d7f-8d92-002ab5421e5b", "id": 6} v 0) v1 -- 0x7f1c3004e9b0 con 0x7f1c300d0be0 2023-10-16T09:48:45.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.870+0000 7f1c26ffd640 1 -- 172.21.15.157:0/457608554 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "38f95207-b5e2-4d7f-8d92-002ab5421e5b", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1c20034070 con 0x7f1c300d0be0 2023-10-16T09:48:45.872 INFO:teuthology.orchestra.run.smithi157.stdout:6 2023-10-16T09:48:45.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.871+0000 7f1c3746d640 1 -- 172.21.15.157:0/457608554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c300d0be0 msgr2=0x7f1c301f5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:45.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.871+0000 7f1c3746d640 1 --2- 172.21.15.157:0/457608554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c300d0be0 0x7f1c301f5ed0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1c20000c00 tx=0x7f1c20004730 comp rx=0 tx=0).stop 2023-10-16T09:48:45.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.872+0000 7f1c3746d640 1 -- 172.21.15.157:0/457608554 shutdown_connections 2023-10-16T09:48:45.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.872+0000 7f1c3746d640 1 --2- 172.21.15.157:0/457608554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c3015cd70 0x7f1c3006bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.872+0000 7f1c3746d640 1 --2- 172.21.15.157:0/457608554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c300d0be0 0x7f1c301f5ed0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.872+0000 7f1c3746d640 1 --2- 172.21.15.157:0/457608554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c300b8480 0x7f1c3015d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:45.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.872+0000 7f1c3746d640 1 -- 172.21.15.157:0/457608554 >> 172.21.15.157:0/457608554 conn(0x7f1c300cf570 msgr2=0x7f1c300b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:45.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.872+0000 7f1c3746d640 1 -- 172.21.15.157:0/457608554 shutdown_connections 2023-10-16T09:48:45.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:45.872+0000 7f1c3746d640 1 -- 172.21.15.157:0/457608554 wait complete. 2023-10-16T09:48:45.889 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 4f0b5649-ac0f-4deb-bb8d-cf3107fe891b 7 2023-10-16T09:48:45.992 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:45.990+0000 7fa84ea2b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T09:48:46.032 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:46.030+0000 7f1aece12040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:48:46.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.072+0000 7fadc5290640 1 Processor -- start 2023-10-16T09:48:46.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.072+0000 7fadc5290640 1 -- start start 2023-10-16T09:48:46.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.073+0000 7fadc5290640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadc00cded0 0x7fadc00b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:46.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.073+0000 7fadc5290640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc015d150 con 0x7fadc00b8480 2023-10-16T09:48:46.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.073+0000 7fadc5290640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc00d1180 con 0x7fadc015cd70 2023-10-16T09:48:46.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.073+0000 7fadc5290640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc00d1460 con 0x7fadc00cded0 2023-10-16T09:48:46.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.073+0000 7fadbffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fadc015cd70 0x7fadc00d2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:46464/0 (socket says 172.21.15.157:46464) 2023-10-16T09:48:46.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.073+0000 7fadbffff640 1 -- 172.21.15.157:0/2559469850 learned_addr learned my addr 172.21.15.157:0/2559469850 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-16T09:48:46.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.073+0000 7fadc4a8f640 1 --2- 172.21.15.157:0/2559469850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadc00cded0 0x7fadc00b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:46.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.074+0000 7fadbf7fe640 1 -- 172.21.15.157:0/2559469850 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364167453 0 0) 0x7fadc00d1180 con 0x7fadc015cd70 2023-10-16T09:48:46.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.074+0000 7fadbf7fe640 1 -- 172.21.15.157:0/2559469850 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadac003520 con 0x7fadc015cd70 2023-10-16T09:48:46.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.074+0000 7fadc4a8f640 1 -- 172.21.15.157:0/2559469850 >> v1:172.21.15.157:6789/0 conn(0x7fadc015cd70 legacy=0x7fadc00d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:46.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.074+0000 7fadc4a8f640 1 -- 172.21.15.157:0/2559469850 >> v1:172.21.15.84:6789/0 conn(0x7fadc00b8480 legacy=0x7fadc00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:46.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.074+0000 7fadc4a8f640 1 -- 172.21.15.157:0/2559469850 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc00d1740 con 0x7fadc00cded0 2023-10-16T09:48:46.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.075+0000 7fadc4a8f640 1 --2- 172.21.15.157:0/2559469850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadc00cded0 0x7fadc00b5c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009dc0 tx=0x7fadb802bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8b8680ae28f128f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:46.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.075+0000 7fadbf7fe640 1 -- 172.21.15.157:0/2559469850 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb802ccc0 con 0x7fadc00cded0 2023-10-16T09:48:46.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.075+0000 7fadbf7fe640 1 -- 172.21.15.157:0/2559469850 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb802ce20 con 0x7fadc00cded0 2023-10-16T09:48:46.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.075+0000 7fadbf7fe640 1 -- 172.21.15.157:0/2559469850 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8034660 con 0x7fadc00cded0 2023-10-16T09:48:46.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.076+0000 7fadc5290640 1 -- 172.21.15.157:0/2559469850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadc00cded0 msgr2=0x7fadc00b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:46.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.076+0000 7fadc5290640 1 --2- 172.21.15.157:0/2559469850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadc00cded0 0x7fadc00b5c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009dc0 tx=0x7fadb802bf10 comp rx=0 tx=0).stop 2023-10-16T09:48:46.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.076+0000 7fadc5290640 1 -- 172.21.15.157:0/2559469850 shutdown_connections 2023-10-16T09:48:46.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.076+0000 7fadc5290640 1 --2- 172.21.15.157:0/2559469850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadc00cded0 0x7fadc00b5c90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:46.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.076+0000 7fadc5290640 1 -- 172.21.15.157:0/2559469850 >> 172.21.15.157:0/2559469850 conn(0x7fadc00cf570 msgr2=0x7fadc00cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:46.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.076+0000 7fadc5290640 1 -- 172.21.15.157:0/2559469850 shutdown_connections 2023-10-16T09:48:46.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.076+0000 7fadc5290640 1 -- 172.21.15.157:0/2559469850 wait complete. 2023-10-16T09:48:46.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.078+0000 7fadc5290640 1 Processor -- start 2023-10-16T09:48:46.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.078+0000 7fadc5290640 1 -- start start 2023-10-16T09:48:46.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.079+0000 7fadc5290640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc00b8480 0x7fadc00b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:46.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.079+0000 7fadc5290640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadc00cded0 0x7fadc015ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:46.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.079+0000 7fadc5290640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadc015cd70 0x7fadc014d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:46.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.079+0000 7fadc5290640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc01f30c0 con 0x7fadc00cded0 2023-10-16T09:48:46.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.079+0000 7fadc5290640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc015f0f0 con 0x7fadc00b8480 2023-10-16T09:48:46.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.079+0000 7fadc5290640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc014da00 con 0x7fadc015cd70 2023-10-16T09:48:46.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.079+0000 7fadc4a8f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadc015cd70 0x7fadc014d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:46.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.079+0000 7fadbffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc00b8480 0x7fadc00b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:46.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.079+0000 7fadc4a8f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadc015cd70 0x7fadc014d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:38316/0 (socket says 172.21.15.157:38316) 2023-10-16T09:48:46.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.079+0000 7fadc4a8f640 1 -- 172.21.15.157:0/3791377711 learned_addr learned my addr 172.21.15.157:0/3791377711 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-16T09:48:46.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.079+0000 7fadb7fff640 1 --2- 172.21.15.157:0/3791377711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadc00cded0 0x7fadc015ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:46.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.079+0000 7fadc4a8f640 1 -- 172.21.15.157:0/3791377711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc00b8480 msgr2=0x7fadc00b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:46.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.079+0000 7fadc4a8f640 1 --2- 172.21.15.157:0/3791377711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc00b8480 0x7fadc00b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:46.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.079+0000 7fadc4a8f640 1 -- 172.21.15.157:0/3791377711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadc00cded0 msgr2=0x7fadc015ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:46.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.079+0000 7fadc4a8f640 1 --2- 172.21.15.157:0/3791377711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadc00cded0 0x7fadc015ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:46.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.079+0000 7fadc4a8f640 1 -- 172.21.15.157:0/3791377711 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb8009a70 con 0x7fadc015cd70 2023-10-16T09:48:46.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.080+0000 7fadc4a8f640 1 --2- 172.21.15.157:0/3791377711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadc015cd70 0x7fadc014d4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009ee0 tx=0x7fadb8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:46.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.080+0000 7fadbd7fa640 1 -- 172.21.15.157:0/3791377711 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8040650 con 0x7fadc015cd70 2023-10-16T09:48:46.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.080+0000 7fadbd7fa640 1 -- 172.21.15.157:0/3791377711 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb80407b0 con 0x7fadc015cd70 2023-10-16T09:48:46.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.080+0000 7fadbd7fa640 1 -- 172.21.15.157:0/3791377711 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8040b60 con 0x7fadc015cd70 2023-10-16T09:48:46.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.080+0000 7fadc5290640 1 -- 172.21.15.157:0/3791377711 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc01f2540 con 0x7fadc015cd70 2023-10-16T09:48:46.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.080+0000 7fadc5290640 1 -- 172.21.15.157:0/3791377711 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc01f2b00 con 0x7fadc015cd70 2023-10-16T09:48:46.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.080+0000 7fadbd7fa640 1 -- 172.21.15.157:0/3791377711 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fadb8049400 con 0x7fadc015cd70 2023-10-16T09:48:46.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.080+0000 7fadbd7fa640 1 -- 172.21.15.157:0/3791377711 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fadb8049d10 con 0x7fadc015cd70 2023-10-16T09:48:46.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.081+0000 7fadc5290640 1 -- 172.21.15.157:0/3791377711 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc015d3d0 con 0x7fadc015cd70 2023-10-16T09:48:46.088 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:46.083+0000 7f1aece12040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T09:48:46.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.083+0000 7fadbd7fa640 1 -- 172.21.15.157:0/3791377711 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fadb8068b40 con 0x7fadc015cd70 2023-10-16T09:48:46.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.161+0000 7fadc5290640 1 -- 172.21.15.157:0/3791377711 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4f0b5649-ac0f-4deb-bb8d-cf3107fe891b", "id": 7} v 0) v1 -- 0x7fadc00742a0 con 0x7fadc015cd70 2023-10-16T09:48:46.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.169+0000 7fadbd7fa640 1 -- 172.21.15.157:0/3791377711 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4f0b5649-ac0f-4deb-bb8d-cf3107fe891b", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fadb8057030 con 0x7fadc015cd70 2023-10-16T09:48:46.171 INFO:teuthology.orchestra.run.smithi157.stdout:7 2023-10-16T09:48:46.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.170+0000 7fadc5290640 1 -- 172.21.15.157:0/3791377711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadc015cd70 msgr2=0x7fadc014d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:46.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.170+0000 7fadc5290640 1 --2- 172.21.15.157:0/3791377711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadc015cd70 0x7fadc014d4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009ee0 tx=0x7fadb8004ab0 comp rx=0 tx=0).stop 2023-10-16T09:48:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.171+0000 7fadc5290640 1 -- 172.21.15.157:0/3791377711 shutdown_connections 2023-10-16T09:48:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.171+0000 7fadc5290640 1 --2- 172.21.15.157:0/3791377711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadc015cd70 0x7fadc014d4c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.171+0000 7fadc5290640 1 --2- 172.21.15.157:0/3791377711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadc00cded0 0x7fadc015ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.171+0000 7fadc5290640 1 --2- 172.21.15.157:0/3791377711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc00b8480 0x7fadc00b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.171+0000 7fadc5290640 1 -- 172.21.15.157:0/3791377711 >> 172.21.15.157:0/3791377711 conn(0x7fadc00cf570 msgr2=0x7fadc015bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:46.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.171+0000 7fadc5290640 1 -- 172.21.15.157:0/3791377711 shutdown_connections 2023-10-16T09:48:46.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-16T09:48:46.171+0000 7fadc5290640 1 -- 172.21.15.157:0/3791377711 wait complete. 2023-10-16T09:48:46.188 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-16T09:48:46.189 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-16T09:48:46.198 INFO:tasks.ceph.osd.0:Started 2023-10-16T09:48:46.198 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-16T09:48:46.199 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-16T09:48:46.207 INFO:tasks.ceph.osd.1:Started 2023-10-16T09:48:46.208 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-16T09:48:46.208 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-16T09:48:46.214 INFO:tasks.ceph.osd.2:Started 2023-10-16T09:48:46.215 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-16T09:48:46.215 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-16T09:48:46.221 INFO:tasks.ceph.osd.3:Started 2023-10-16T09:48:46.221 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-16T09:48:46.221 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-16T09:48:46.226 INFO:tasks.ceph.osd.4:Started 2023-10-16T09:48:46.227 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-16T09:48:46.227 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-16T09:48:46.231 INFO:tasks.ceph.osd.5:Started 2023-10-16T09:48:46.231 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-16T09:48:46.232 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-16T09:48:46.237 INFO:tasks.ceph.osd.6:Started 2023-10-16T09:48:46.238 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-16T09:48:46.238 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-16T09:48:46.243 INFO:tasks.ceph.osd.7:Started 2023-10-16T09:48:46.243 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:48:46.256 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:46.255+0000 7f1aece12040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T09:48:46.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.468+0000 7f2beb6bb640 1 Processor -- start 2023-10-16T09:48:46.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.469+0000 7f2beb6bb640 1 -- start start 2023-10-16T09:48:46.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.470+0000 7f2beb6bb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2be4159b20 0x7f2be406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:46.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.470+0000 7f2beb6bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2be40b0b20 0x7f2be406fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:46.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.470+0000 7f2beb6bb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be406f7f0 con 0x7f2be4159f00 2023-10-16T09:48:46.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.470+0000 7f2beb6bb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be406c5e0 con 0x7f2be4159b20 2023-10-16T09:48:46.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.470+0000 7f2beb6bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be406c8c0 con 0x7f2be40b0b20 2023-10-16T09:48:46.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.470+0000 7f2be9457640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2be4159b20 0x7f2be406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:46.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.470+0000 7f2be8c56640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2be4159f00 0x7f2be40b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60470/0 (socket says 172.21.15.84:60470) 2023-10-16T09:48:46.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.470+0000 7f2be8c56640 1 -- 172.21.15.84:0/3637390613 learned_addr learned my addr 172.21.15.84:0/3637390613 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:48:46.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.470+0000 7f2be9457640 1 --2- 172.21.15.84:0/3637390613 >> v2:172.21.15.84:3300/0 conn(0x7f2be4159b20 0x7f2be406bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49496/0 (socket says 172.21.15.84:49496) 2023-10-16T09:48:46.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.470+0000 7f2be9c58640 1 --2- 172.21.15.84:0/3637390613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2be40b0b20 0x7f2be406fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:46.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.470+0000 7f2bdbfff640 1 -- 172.21.15.84:0/3637390613 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071500678 0 0) 0x7f2be406f7f0 con 0x7f2be4159f00 2023-10-16T09:48:46.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.471+0000 7f2be9457640 1 -- 172.21.15.84:0/3637390613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2be40b0b20 msgr2=0x7f2be406fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:46.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.471+0000 7f2be9457640 1 --2- 172.21.15.84:0/3637390613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2be40b0b20 0x7f2be406fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:46.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.471+0000 7f2be9457640 1 -- 172.21.15.84:0/3637390613 >> v1:172.21.15.84:6789/0 conn(0x7f2be4159f00 legacy=0x7f2be40b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:46.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.471+0000 7f2be9457640 1 -- 172.21.15.84:0/3637390613 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be406cba0 con 0x7f2be4159b20 2023-10-16T09:48:46.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.472+0000 7f2be9457640 1 --2- 172.21.15.84:0/3637390613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2be4159b20 0x7f2be406bbe0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2be000ba40 tx=0x7f2be002e080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a337f80ad23a3e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:46.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.472+0000 7f2bdbfff640 1 -- 172.21.15.84:0/3637390613 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be002ee30 con 0x7f2be4159b20 2023-10-16T09:48:46.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.472+0000 7f2bdbfff640 1 -- 172.21.15.84:0/3637390613 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be0036440 con 0x7f2be4159b20 2023-10-16T09:48:46.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.472+0000 7f2bdbfff640 1 -- 172.21.15.84:0/3637390613 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be00367d0 con 0x7f2be4159b20 2023-10-16T09:48:46.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.473+0000 7f2beb6bb640 1 -- 172.21.15.84:0/3637390613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2be4159b20 msgr2=0x7f2be406bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:46.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.473+0000 7f2beb6bb640 1 --2- 172.21.15.84:0/3637390613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2be4159b20 0x7f2be406bbe0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2be000ba40 tx=0x7f2be002e080 comp rx=0 tx=0).stop 2023-10-16T09:48:46.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.473+0000 7f2beb6bb640 1 -- 172.21.15.84:0/3637390613 shutdown_connections 2023-10-16T09:48:46.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.473+0000 7f2beb6bb640 1 --2- 172.21.15.84:0/3637390613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2be40b0b20 0x7f2be406fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:46.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.473+0000 7f2beb6bb640 1 --2- 172.21.15.84:0/3637390613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2be4159b20 0x7f2be406bbe0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:46.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.473+0000 7f2beb6bb640 1 -- 172.21.15.84:0/3637390613 >> 172.21.15.84:0/3637390613 conn(0x7f2be40cb410 msgr2=0x7f2be40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:46.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.473+0000 7f2beb6bb640 1 -- 172.21.15.84:0/3637390613 shutdown_connections 2023-10-16T09:48:46.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.473+0000 7f2beb6bb640 1 -- 172.21.15.84:0/3637390613 wait complete. 2023-10-16T09:48:46.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.475+0000 7f2beb6bb640 1 Processor -- start 2023-10-16T09:48:46.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.475+0000 7f2beb6bb640 1 -- start start 2023-10-16T09:48:46.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.476+0000 7f2beb6bb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2be40b0b20 0x7f2be4148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:46.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.476+0000 7f2beb6bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2be4159b20 0x7f2be4147e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:46.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.476+0000 7f2beb6bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be4159f00 0x7f2be40ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:46.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.476+0000 7f2beb6bb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be4148e70 con 0x7f2be40b0b20 2023-10-16T09:48:46.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.476+0000 7f2beb6bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40b1370 con 0x7f2be4159f00 2023-10-16T09:48:46.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.476+0000 7f2beb6bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40b1650 con 0x7f2be4159b20 2023-10-16T09:48:46.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.476+0000 7f2be9457640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2be40b0b20 0x7f2be4148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:46.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.476+0000 7f2be9457640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2be40b0b20 0x7f2be4148930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49506/0 (socket says 172.21.15.84:49506) 2023-10-16T09:48:46.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.476+0000 7f2be9457640 1 -- 172.21.15.84:0/3856487992 learned_addr learned my addr 172.21.15.84:0/3856487992 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:48:46.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.476+0000 7f2be8c56640 1 --2- 172.21.15.84:0/3856487992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2be4159b20 0x7f2be4147e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:46.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.476+0000 7f2be9c58640 1 --2- 172.21.15.84:0/3856487992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be4159f00 0x7f2be40ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:46.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.476+0000 7f2be9457640 1 -- 172.21.15.84:0/3856487992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2be4159b20 msgr2=0x7f2be4147e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:46.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.476+0000 7f2be9457640 1 --2- 172.21.15.84:0/3856487992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2be4159b20 0x7f2be4147e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:46.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.476+0000 7f2be9457640 1 -- 172.21.15.84:0/3856487992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be4159f00 msgr2=0x7f2be40ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:46.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.476+0000 7f2be9457640 1 --2- 172.21.15.84:0/3856487992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be4159f00 0x7f2be40ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:46.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.476+0000 7f2be9457640 1 -- 172.21.15.84:0/3856487992 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be000b6a0 con 0x7f2be40b0b20 2023-10-16T09:48:46.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.477+0000 7f2be9457640 1 --2- 172.21.15.84:0/3856487992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2be40b0b20 0x7f2be4148930 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2be0000c00 tx=0x7f2be0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:46.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.477+0000 7f2bd9ffb640 1 -- 172.21.15.84:0/3856487992 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be0004230 con 0x7f2be40b0b20 2023-10-16T09:48:46.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.477+0000 7f2bd9ffb640 1 -- 172.21.15.84:0/3856487992 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be0004390 con 0x7f2be40b0b20 2023-10-16T09:48:46.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.477+0000 7f2bd9ffb640 1 -- 172.21.15.84:0/3856487992 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be0041930 con 0x7f2be40b0b20 2023-10-16T09:48:46.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.477+0000 7f2beb6bb640 1 -- 172.21.15.84:0/3856487992 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be40b1930 con 0x7f2be40b0b20 2023-10-16T09:48:46.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.477+0000 7f2beb6bb640 1 -- 172.21.15.84:0/3856487992 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be40b1ef0 con 0x7f2be40b0b20 2023-10-16T09:48:46.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.477+0000 7f2bd9ffb640 1 -- 172.21.15.84:0/3856487992 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2be003d040 con 0x7f2be40b0b20 2023-10-16T09:48:46.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.478+0000 7f2bd9ffb640 1 -- 172.21.15.84:0/3856487992 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f2be0041d90 con 0x7f2be40b0b20 2023-10-16T09:48:46.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.478+0000 7f2beb6bb640 1 -- 172.21.15.84:0/3856487992 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be404e160 con 0x7f2be40b0b20 2023-10-16T09:48:46.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.481+0000 7f2bd9ffb640 1 -- 172.21.15.84:0/3856487992 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f2be003b070 con 0x7f2be40b0b20 2023-10-16T09:48:46.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.545+0000 7f2beb6bb640 1 -- 172.21.15.84:0/3856487992 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2be40b10b0 con 0x7f2be40b0b20 2023-10-16T09:48:46.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.545+0000 7f2bd9ffb640 1 -- 172.21.15.84:0/3856487992 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f2be0003d40 con 0x7f2be40b0b20 2023-10-16T09:48:46.547 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:46.547 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":10,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:48:46.163691+0000","last_up_change":"0.000000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:48:46.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.547+0000 7f2beb6bb640 1 -- 172.21.15.84:0/3856487992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2be40b0b20 msgr2=0x7f2be4148930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:46.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.547+0000 7f2beb6bb640 1 --2- 172.21.15.84:0/3856487992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2be40b0b20 0x7f2be4148930 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2be0000c00 tx=0x7f2be0004730 comp rx=0 tx=0).stop 2023-10-16T09:48:46.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.547+0000 7f2beb6bb640 1 -- 172.21.15.84:0/3856487992 shutdown_connections 2023-10-16T09:48:46.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.547+0000 7f2beb6bb640 1 --2- 172.21.15.84:0/3856487992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be4159f00 0x7f2be40ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:46.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.547+0000 7f2beb6bb640 1 --2- 172.21.15.84:0/3856487992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2be4159b20 0x7f2be4147e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:46.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.547+0000 7f2beb6bb640 1 --2- 172.21.15.84:0/3856487992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2be40b0b20 0x7f2be4148930 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:46.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.547+0000 7f2beb6bb640 1 -- 172.21.15.84:0/3856487992 >> 172.21.15.84:0/3856487992 conn(0x7f2be40cb410 msgr2=0x7f2be40cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:46.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.547+0000 7f2beb6bb640 1 -- 172.21.15.84:0/3856487992 shutdown_connections 2023-10-16T09:48:46.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:46.547+0000 7f2beb6bb640 1 -- 172.21.15.84:0/3856487992 wait complete. 2023-10-16T09:48:46.564 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-16T09:48:46.565 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-16T09:48:46.681 INFO:tasks.ceph.osd.5.smithi157.stderr:2023-10-16T09:48:46.679+0000 7f8fd9e16500 -1 Falling back to public interface 2023-10-16T09:48:46.917 INFO:tasks.ceph.osd.0.smithi084.stderr:2023-10-16T09:48:46.915+0000 7f272d54a500 -1 Falling back to public interface 2023-10-16T09:48:46.933 INFO:tasks.ceph.osd.1.smithi084.stderr:2023-10-16T09:48:46.932+0000 7f987f00f500 -1 Falling back to public interface 2023-10-16T09:48:46.937 INFO:tasks.ceph.osd.4.smithi157.stderr:2023-10-16T09:48:46.936+0000 7f59b7e12500 -1 Falling back to public interface 2023-10-16T09:48:46.941 INFO:tasks.ceph.osd.3.smithi084.stderr:2023-10-16T09:48:46.939+0000 7ff374010500 -1 Falling back to public interface 2023-10-16T09:48:46.942 INFO:tasks.ceph.osd.2.smithi084.stderr:2023-10-16T09:48:46.940+0000 7f3008e0c500 -1 Falling back to public interface 2023-10-16T09:48:46.954 INFO:tasks.ceph.osd.6.smithi157.stderr:2023-10-16T09:48:46.953+0000 7ff80161a500 -1 Falling back to public interface 2023-10-16T09:48:46.957 INFO:tasks.ceph.osd.7.smithi157.stderr:2023-10-16T09:48:46.956+0000 7f88dc574500 -1 Falling back to public interface 2023-10-16T09:48:46.962 INFO:tasks.ceph.osd.7.smithi157.stderr:2023-10-16T09:48:46.960+0000 7f88db51b640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6822/0: (98) Address already in use 2023-10-16T09:48:47.083 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:47.081+0000 7fa84ea2b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T09:48:47.306 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:47.304+0000 7f1aece12040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T09:48:47.370 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-16T09:48:47.388 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:47.386+0000 7fa84ea2b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T09:48:47.450 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:47.449+0000 7fa84ea2b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T09:48:47.539 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.537+0000 7f2020533640 1 Processor -- start 2023-10-16T09:48:47.539 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.537+0000 7f2020533640 1 -- start start 2023-10-16T09:48:47.541 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.538+0000 7f2020533640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20180afef0 0x7f20180cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:47.541 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.538+0000 7f2020533640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f201806aeb0 0x7f201806b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:47.541 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.538+0000 7f2020533640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20180d0180 con 0x7f20180b24a0 2023-10-16T09:48:47.541 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.538+0000 7f2020533640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f201806b870 con 0x7f201806aeb0 2023-10-16T09:48:47.541 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.538+0000 7f2020533640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20180cfea0 con 0x7f20180afef0 2023-10-16T09:48:47.542 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.539+0000 7f201dace640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f201806aeb0 0x7f201806b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:47.542 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.539+0000 7f201dace640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f201806aeb0 0x7f201806b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49508/0 (socket says 172.21.15.84:49508) 2023-10-16T09:48:47.542 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.539+0000 7f201dace640 1 -- 172.21.15.84:0/932584653 learned_addr learned my addr 172.21.15.84:0/932584653 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:48:47.542 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.539+0000 7f201ead0640 1 --1- 172.21.15.84:0/932584653 >> v1:172.21.15.84:6789/0 conn(0x7f20180b24a0 0x7f20180b0fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60482/0 (socket says 172.21.15.84:60482) 2023-10-16T09:48:47.542 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.539+0000 7f201e2cf640 1 --2- 172.21.15.84:0/932584653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20180afef0 0x7f20180cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:47.543 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.539+0000 7f201dace640 1 -- 172.21.15.84:0/932584653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20180afef0 msgr2=0x7f20180cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:47.543 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.539+0000 7f201dace640 1 --2- 172.21.15.84:0/932584653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20180afef0 0x7f20180cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:47.543 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.539+0000 7f201dace640 1 -- 172.21.15.84:0/932584653 >> v1:172.21.15.84:6789/0 conn(0x7f20180b24a0 legacy=0x7f20180b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:47.543 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.539+0000 7f201d2cd640 1 -- 172.21.15.84:0/932584653 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005154558 0 0) 0x7f20180d0180 con 0x7f20180b24a0 2023-10-16T09:48:47.543 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.539+0000 7f201dace640 1 -- 172.21.15.84:0/932584653 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20180b04d0 con 0x7f201806aeb0 2023-10-16T09:48:47.545 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.541+0000 7f201dace640 1 --2- 172.21.15.84:0/932584653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f201806aeb0 0x7f201806b330 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2008009870 tx=0x7f200802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb66ba9b5d32e494 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:47.545 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.541+0000 7f201d2cd640 1 -- 172.21.15.84:0/932584653 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20080317c0 con 0x7f201806aeb0 2023-10-16T09:48:47.545 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.541+0000 7f201d2cd640 1 -- 172.21.15.84:0/932584653 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2008031920 con 0x7f201806aeb0 2023-10-16T09:48:47.545 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.541+0000 7f201d2cd640 1 -- 172.21.15.84:0/932584653 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008031cb0 con 0x7f201806aeb0 2023-10-16T09:48:47.545 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.542+0000 7f2020533640 1 -- 172.21.15.84:0/932584653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f201806aeb0 msgr2=0x7f201806b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:47.546 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.542+0000 7f2020533640 1 --2- 172.21.15.84:0/932584653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f201806aeb0 0x7f201806b330 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2008009870 tx=0x7f200802b860 comp rx=0 tx=0).stop 2023-10-16T09:48:47.546 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.542+0000 7f2020533640 1 -- 172.21.15.84:0/932584653 shutdown_connections 2023-10-16T09:48:47.547 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.542+0000 7f2020533640 1 --2- 172.21.15.84:0/932584653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f201806aeb0 0x7f201806b330 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:47.547 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.542+0000 7f2020533640 1 --2- 172.21.15.84:0/932584653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20180afef0 0x7f20180cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:47.547 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.542+0000 7f2020533640 1 -- 172.21.15.84:0/932584653 >> 172.21.15.84:0/932584653 conn(0x7f20180d0ad0 msgr2=0x7f20180d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:47.547 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.542+0000 7f2020533640 1 -- 172.21.15.84:0/932584653 shutdown_connections 2023-10-16T09:48:47.548 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.542+0000 7f2020533640 1 -- 172.21.15.84:0/932584653 wait complete. 2023-10-16T09:48:47.550 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.543+0000 7f2020533640 1 Processor -- start 2023-10-16T09:48:47.550 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.544+0000 7f2020533640 1 -- start start 2023-10-16T09:48:47.550 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.544+0000 7f2020533640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f201806aeb0 0x7f20180c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:47.550 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.545+0000 7f2020533640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20180afef0 0x7f201806dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:47.551 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.545+0000 7f2020533640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20180b24a0 0x7f201806e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:47.551 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.545+0000 7f2020533640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20180b5980 con 0x7f201806aeb0 2023-10-16T09:48:47.551 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.545+0000 7f2020533640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20180b4670 con 0x7f20180afef0 2023-10-16T09:48:47.551 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.545+0000 7f2020533640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20180b56a0 con 0x7f20180b24a0 2023-10-16T09:48:47.551 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.545+0000 7f201e2cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f201806aeb0 0x7f20180c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:47.552 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.545+0000 7f201e2cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f201806aeb0 0x7f20180c6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49510/0 (socket says 172.21.15.84:49510) 2023-10-16T09:48:47.552 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.545+0000 7f201e2cf640 1 -- 172.21.15.84:0/4089075328 learned_addr learned my addr 172.21.15.84:0/4089075328 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:48:47.552 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.545+0000 7f201ead0640 1 --2- 172.21.15.84:0/4089075328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20180b24a0 0x7f201806e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:47.552 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.545+0000 7f201e2cf640 1 -- 172.21.15.84:0/4089075328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20180b24a0 msgr2=0x7f201806e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:47.552 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.545+0000 7f201e2cf640 1 --2- 172.21.15.84:0/4089075328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20180b24a0 0x7f201806e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:47.553 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.545+0000 7f201e2cf640 1 -- 172.21.15.84:0/4089075328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20180afef0 msgr2=0x7f201806dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:47.553 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.545+0000 7f201e2cf640 1 --2- 172.21.15.84:0/4089075328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20180afef0 0x7f201806dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:47.553 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.545+0000 7f201e2cf640 1 -- 172.21.15.84:0/4089075328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20080094d0 con 0x7f201806aeb0 2023-10-16T09:48:47.553 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.545+0000 7f201e2cf640 1 --2- 172.21.15.84:0/4089075328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f201806aeb0 0x7f20180c6ea0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f201400e8e0 tx=0x7f201400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:47.553 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.546+0000 7f200effd640 1 -- 172.21.15.84:0/4089075328 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201400c8e0 con 0x7f201806aeb0 2023-10-16T09:48:47.554 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.546+0000 7f200effd640 1 -- 172.21.15.84:0/4089075328 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201400ca40 con 0x7f201806aeb0 2023-10-16T09:48:47.554 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.546+0000 7f200effd640 1 -- 172.21.15.84:0/4089075328 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201400cdd0 con 0x7f201806aeb0 2023-10-16T09:48:47.554 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.546+0000 7f2020533640 1 -- 172.21.15.84:0/4089075328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20181f5ac0 con 0x7f201806aeb0 2023-10-16T09:48:47.554 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.546+0000 7f2020533640 1 -- 172.21.15.84:0/4089075328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20181f6080 con 0x7f201806aeb0 2023-10-16T09:48:47.554 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.546+0000 7f200effd640 1 -- 172.21.15.84:0/4089075328 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2014013770 con 0x7f201806aeb0 2023-10-16T09:48:47.555 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.546+0000 7f200effd640 1 -- 172.21.15.84:0/4089075328 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f201401c450 con 0x7f201806aeb0 2023-10-16T09:48:47.555 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.547+0000 7f2020533640 1 -- 172.21.15.84:0/4089075328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20180b2180 con 0x7f201806aeb0 2023-10-16T09:48:47.555 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.550+0000 7f200effd640 1 -- 172.21.15.84:0/4089075328 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f2014021060 con 0x7f201806aeb0 2023-10-16T09:48:47.571 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:47.570+0000 7fa84ea2b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T09:48:47.580 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.578+0000 7f200effd640 1 -- 172.21.15.84:0/4089075328 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 2) v1 ==== 44566+0+0 (secure 0 0 0) 0x7f2014013a20 con 0x7f201806aeb0 2023-10-16T09:48:47.620 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:47.618+0000 7f1aece12040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T09:48:47.632 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.630+0000 7f2020533640 1 -- 172.21.15.84:0/4089075328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20181f5920 con 0x7f201806aeb0 2023-10-16T09:48:47.633 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.631+0000 7f200effd640 1 -- 172.21.15.84:0/4089075328 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f20181f5920 con 0x7f201806aeb0 2023-10-16T09:48:47.633 INFO:teuthology.misc.health.smithi084.stdout: 2023-10-16T09:48:47.633 INFO:teuthology.misc.health.smithi084.stdout:{"epoch":10,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:48:46.163691+0000","last_up_change":"0.000000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:48:47.634 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.632+0000 7f2020533640 1 -- 172.21.15.84:0/4089075328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f201806aeb0 msgr2=0x7f20180c6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:47.635 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.632+0000 7f2020533640 1 --2- 172.21.15.84:0/4089075328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f201806aeb0 0x7f20180c6ea0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f201400e8e0 tx=0x7f201400eda0 comp rx=0 tx=0).stop 2023-10-16T09:48:47.635 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.632+0000 7f2020533640 1 -- 172.21.15.84:0/4089075328 shutdown_connections 2023-10-16T09:48:47.635 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.632+0000 7f2020533640 1 --2- 172.21.15.84:0/4089075328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20180b24a0 0x7f201806e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:47.635 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.632+0000 7f2020533640 1 --2- 172.21.15.84:0/4089075328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20180afef0 0x7f201806dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:47.636 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.632+0000 7f2020533640 1 --2- 172.21.15.84:0/4089075328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f201806aeb0 0x7f20180c6ea0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:47.636 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.632+0000 7f2020533640 1 -- 172.21.15.84:0/4089075328 >> 172.21.15.84:0/4089075328 conn(0x7f20180d0ad0 msgr2=0x7f20180af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:47.636 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.632+0000 7f2020533640 1 -- 172.21.15.84:0/4089075328 shutdown_connections 2023-10-16T09:48:47.636 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:47.632+0000 7f2020533640 1 -- 172.21.15.84:0/4089075328 wait complete. 2023-10-16T09:48:47.646 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-16T09:48:47.681 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:47.680+0000 7f1aece12040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T09:48:47.797 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:48:47.796+0000 7f1aece12040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T09:48:48.605 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:48.603+0000 7fa7ef6ab640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-16T09:48:48.606 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:48:48.604+0000 7fa7ef6ab640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-16T09:48:50.208 INFO:tasks.ceph.osd.5.smithi157.stderr:2023-10-16T09:48:50.206+0000 7f8fd9e16500 -1 osd.5 0 log_to_monitors true 2023-10-16T09:48:50.439 INFO:tasks.ceph.osd.0.smithi084.stderr:2023-10-16T09:48:50.437+0000 7f272d54a500 -1 osd.0 0 log_to_monitors true 2023-10-16T09:48:50.477 INFO:tasks.ceph.osd.1.smithi084.stderr:2023-10-16T09:48:50.475+0000 7f987f00f500 -1 osd.1 0 log_to_monitors true 2023-10-16T09:48:50.477 INFO:tasks.ceph.osd.4.smithi157.stderr:2023-10-16T09:48:50.475+0000 7f59b7e12500 -1 osd.4 0 log_to_monitors true 2023-10-16T09:48:50.525 INFO:tasks.ceph.osd.3.smithi084.stderr:2023-10-16T09:48:50.524+0000 7ff374010500 -1 osd.3 0 log_to_monitors true 2023-10-16T09:48:50.541 INFO:tasks.ceph.osd.6.smithi157.stderr:2023-10-16T09:48:50.540+0000 7ff80161a500 -1 osd.6 0 log_to_monitors true 2023-10-16T09:48:50.548 INFO:tasks.ceph.osd.2.smithi084.stderr:2023-10-16T09:48:50.546+0000 7f3008e0c500 -1 osd.2 0 log_to_monitors true 2023-10-16T09:48:50.555 INFO:tasks.ceph.osd.7.smithi157.stderr:2023-10-16T09:48:50.553+0000 7f88dc574500 -1 osd.7 0 log_to_monitors true 2023-10-16T09:48:53.323 INFO:tasks.ceph.osd.5.smithi157.stderr:2023-10-16T09:48:53.321+0000 7f8fd4f59640 -1 osd.5 0 waiting for initial osdmap 2023-10-16T09:48:53.339 INFO:tasks.ceph.osd.5.smithi157.stderr:2023-10-16T09:48:53.337+0000 7f8fcd58c640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T09:48:53.340 INFO:tasks.ceph.osd.6.smithi157.stderr:2023-10-16T09:48:53.338+0000 7ff7fc858640 -1 osd.6 0 waiting for initial osdmap 2023-10-16T09:48:53.360 INFO:tasks.ceph.osd.6.smithi157.stderr:2023-10-16T09:48:53.358+0000 7ff7f4e8b640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T09:48:53.367 INFO:tasks.ceph.osd.7.smithi157.stderr:2023-10-16T09:48:53.366+0000 7f88d7cdd640 -1 osd.7 0 waiting for initial osdmap 2023-10-16T09:48:53.369 INFO:tasks.ceph.osd.4.smithi157.stderr:2023-10-16T09:48:53.368+0000 7f59b471e640 -1 osd.4 0 waiting for initial osdmap 2023-10-16T09:48:53.387 INFO:tasks.ceph.osd.4.smithi157.stderr:2023-10-16T09:48:53.385+0000 7f59ab517640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T09:48:53.388 INFO:tasks.ceph.osd.7.smithi157.stderr:2023-10-16T09:48:53.387+0000 7f88d0310640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T09:48:53.980 INFO:tasks.ceph.osd.2.smithi084.stderr:2023-10-16T09:48:53.979+0000 7f3005daf640 -1 osd.2 0 waiting for initial osdmap 2023-10-16T09:48:53.991 INFO:tasks.ceph.osd.1.smithi084.stderr:2023-10-16T09:48:53.989+0000 7f987a09b640 -1 osd.1 0 waiting for initial osdmap 2023-10-16T09:48:53.998 INFO:tasks.ceph.osd.2.smithi084.stderr:2023-10-16T09:48:53.996+0000 7f2ffcba8640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T09:48:54.010 INFO:tasks.ceph.osd.0.smithi084.stderr:2023-10-16T09:48:54.008+0000 7f2728cb5640 -1 osd.0 0 waiting for initial osdmap 2023-10-16T09:48:54.010 INFO:tasks.ceph.osd.1.smithi084.stderr:2023-10-16T09:48:54.008+0000 7f98726ce640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T09:48:54.022 INFO:tasks.ceph.osd.0.smithi084.stderr:2023-10-16T09:48:54.021+0000 7f27212e8640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T09:48:54.037 INFO:tasks.ceph.osd.3.smithi084.stderr:2023-10-16T09:48:54.036+0000 7ff36f11d640 -1 osd.3 0 waiting for initial osdmap 2023-10-16T09:48:54.053 INFO:tasks.ceph.osd.3.smithi084.stderr:2023-10-16T09:48:54.051+0000 7ff367750640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T09:48:54.454 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-16T09:48:54.638 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.636+0000 7fcf26a83640 1 Processor -- start 2023-10-16T09:48:54.638 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.636+0000 7fcf26a83640 1 -- start start 2023-10-16T09:48:54.639 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.637+0000 7fcf26a83640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf201615f0 0x7fcf2015db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:54.639 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.637+0000 7fcf26a83640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2015e3e0 con 0x7fcf200cd0c0 2023-10-16T09:48:54.639 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.637+0000 7fcf26a83640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf200cd4a0 con 0x7fcf200b0b20 2023-10-16T09:48:54.640 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.637+0000 7fcf26a83640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2015e100 con 0x7fcf201615f0 2023-10-16T09:48:54.640 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.637+0000 7fcf2481f640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fcf200cd0c0 0x7fcf200b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60494/0 (socket says 172.21.15.84:60494) 2023-10-16T09:48:54.640 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.637+0000 7fcf2481f640 1 -- 172.21.15.84:0/4164418355 learned_addr learned my addr 172.21.15.84:0/4164418355 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:48:54.642 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.638+0000 7fcf1f7fe640 1 -- 172.21.15.84:0/4164418355 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828755883 0 0) 0x7fcf2015e3e0 con 0x7fcf200cd0c0 2023-10-16T09:48:54.642 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.638+0000 7fcf25020640 1 --2- 172.21.15.84:0/4164418355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf201615f0 0x7fcf2015db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:54.642 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.638+0000 7fcf1f7fe640 1 -- 172.21.15.84:0/4164418355 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf10003520 con 0x7fcf200cd0c0 2023-10-16T09:48:54.643 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.638+0000 7fcf1f7fe640 1 -- 172.21.15.84:0/4164418355 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1358719634 0 0) 0x7fcf10003520 con 0x7fcf200cd0c0 2023-10-16T09:48:54.643 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.638+0000 7fcf1f7fe640 1 -- 172.21.15.84:0/4164418355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf201615f0 msgr2=0x7fcf2015db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:54.643 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.638+0000 7fcf1f7fe640 1 --2- 172.21.15.84:0/4164418355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf201615f0 0x7fcf2015db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:54.643 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.638+0000 7fcf1f7fe640 1 -- 172.21.15.84:0/4164418355 >> v1:172.21.15.157:6789/0 conn(0x7fcf200b0b20 legacy=0x7fcf2015d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:54.643 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.638+0000 7fcf1f7fe640 1 -- 172.21.15.84:0/4164418355 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf20160b10 con 0x7fcf200cd0c0 2023-10-16T09:48:54.644 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.638+0000 7fcf1f7fe640 1 -- 172.21.15.84:0/4164418355 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fcf14001670 con 0x7fcf200cd0c0 2023-10-16T09:48:54.644 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.638+0000 7fcf1f7fe640 1 -- 172.21.15.84:0/4164418355 >> v1:172.21.15.84:6789/0 conn(0x7fcf200cd0c0 legacy=0x7fcf200b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:54.644 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.638+0000 7fcf1f7fe640 1 --2- 172.21.15.84:0/4164418355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf201615f0 0x7fcf100039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:54.644 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.638+0000 7fcf1f7fe640 1 --2- 172.21.15.84:0/4164418355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf10003f70 0x7fcf100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:54.644 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.638+0000 7fcf1f7fe640 1 --2- 172.21.15.84:0/4164418355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf10004d00 0x7fcf10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:54.644 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.638+0000 7fcf1f7fe640 1 -- 172.21.15.84:0/4164418355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2015e100 con 0x7fcf10003f70 2023-10-16T09:48:54.645 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.638+0000 7fcf1f7fe640 1 -- 172.21.15.84:0/4164418355 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1000bb70 con 0x7fcf10004d00 2023-10-16T09:48:54.645 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.638+0000 7fcf1f7fe640 1 -- 172.21.15.84:0/4164418355 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1000bd30 con 0x7fcf201615f0 2023-10-16T09:48:54.645 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.639+0000 7fcf1ffff640 1 --2- 172.21.15.84:0/4164418355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf10003f70 0x7fcf100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:54.645 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.639+0000 7fcf1ffff640 1 -- 172.21.15.84:0/4164418355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf201615f0 msgr2=0x7fcf100039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:54.645 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.639+0000 7fcf1ffff640 1 --2- 172.21.15.84:0/4164418355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf201615f0 0x7fcf100039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:54.646 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.639+0000 7fcf25020640 1 --2- 172.21.15.84:0/4164418355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf10004d00 0x7fcf10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:54.646 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.639+0000 7fcf1ffff640 1 -- 172.21.15.84:0/4164418355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf10004d00 msgr2=0x7fcf10005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:54.646 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.639+0000 7fcf1ffff640 1 --2- 172.21.15.84:0/4164418355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf10004d00 0x7fcf10005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:54.646 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.639+0000 7fcf1ffff640 1 -- 172.21.15.84:0/4164418355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf10003520 con 0x7fcf10003f70 2023-10-16T09:48:54.647 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.640+0000 7fcf1ffff640 1 --2- 172.21.15.84:0/4164418355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf10003f70 0x7fcf100047c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c001750 tx=0x7fcf100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b073ae6eb3b5b9a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:54.650 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.641+0000 7fcf1f7fe640 1 -- 172.21.15.84:0/4164418355 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c0052d0 con 0x7fcf10003f70 2023-10-16T09:48:54.650 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.641+0000 7fcf1f7fe640 1 -- 172.21.15.84:0/4164418355 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c02e050 con 0x7fcf10003f70 2023-10-16T09:48:54.650 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.641+0000 7fcf1f7fe640 1 -- 172.21.15.84:0/4164418355 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c003830 con 0x7fcf10003f70 2023-10-16T09:48:54.650 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.642+0000 7fcf26a83640 1 -- 172.21.15.84:0/4164418355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf10003f70 msgr2=0x7fcf100047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:54.651 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.642+0000 7fcf26a83640 1 --2- 172.21.15.84:0/4164418355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf10003f70 0x7fcf100047c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c001750 tx=0x7fcf100057f0 comp rx=0 tx=0).stop 2023-10-16T09:48:54.651 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.642+0000 7fcf25020640 1 -- 172.21.15.84:0/4164418355 reap_dead start 2023-10-16T09:48:54.651 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.642+0000 7fcf26a83640 1 -- 172.21.15.84:0/4164418355 shutdown_connections 2023-10-16T09:48:54.651 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.642+0000 7fcf26a83640 1 -- 172.21.15.84:0/4164418355 >> 172.21.15.84:0/4164418355 conn(0x7fcf200cb410 msgr2=0x7fcf200cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:54.652 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.642+0000 7fcf26a83640 1 -- 172.21.15.84:0/4164418355 shutdown_connections 2023-10-16T09:48:54.652 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.642+0000 7fcf26a83640 1 -- 172.21.15.84:0/4164418355 wait complete. 2023-10-16T09:48:54.652 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.643+0000 7fcf26a83640 1 Processor -- start 2023-10-16T09:48:54.652 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.644+0000 7fcf26a83640 1 -- start start 2023-10-16T09:48:54.652 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.644+0000 7fcf26a83640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf10003f70 0x7fcf20154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:54.653 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.644+0000 7fcf26a83640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf200b2950 0x7fcf200b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:54.653 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.645+0000 7fcf26a83640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf20148830 0x7fcf20148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:54.653 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.645+0000 7fcf26a83640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf200b44b0 con 0x7fcf200b2950 2023-10-16T09:48:54.653 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.645+0000 7fcf26a83640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf200b3410 con 0x7fcf10003f70 2023-10-16T09:48:54.653 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.645+0000 7fcf26a83640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf200b36f0 con 0x7fcf20148830 2023-10-16T09:48:54.653 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.645+0000 7fcf1ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf200b2950 0x7fcf200b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:54.654 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.645+0000 7fcf1ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf200b2950 0x7fcf200b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49612/0 (socket says 172.21.15.84:49612) 2023-10-16T09:48:54.654 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.645+0000 7fcf1ffff640 1 -- 172.21.15.84:0/3999181442 learned_addr learned my addr 172.21.15.84:0/3999181442 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:48:54.654 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.645+0000 7fcf25020640 1 --2- 172.21.15.84:0/3999181442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf20148830 0x7fcf20148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:54.654 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.645+0000 7fcf2481f640 1 --2- 172.21.15.84:0/3999181442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf10003f70 0x7fcf20154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:54.654 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.645+0000 7fcf1ffff640 1 -- 172.21.15.84:0/3999181442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf20148830 msgr2=0x7fcf20148cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:54.655 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.645+0000 7fcf1ffff640 1 --2- 172.21.15.84:0/3999181442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf20148830 0x7fcf20148cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:54.655 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.645+0000 7fcf1ffff640 1 -- 172.21.15.84:0/3999181442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf10003f70 msgr2=0x7fcf20154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:54.655 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.645+0000 7fcf1ffff640 1 --2- 172.21.15.84:0/3999181442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf10003f70 0x7fcf20154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:54.655 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.645+0000 7fcf1ffff640 1 -- 172.21.15.84:0/3999181442 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf200b39d0 con 0x7fcf200b2950 2023-10-16T09:48:54.656 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.645+0000 7fcf25020640 1 --2- 172.21.15.84:0/3999181442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf20148830 0x7fcf20148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:48:54.656 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.645+0000 7fcf1ffff640 1 --2- 172.21.15.84:0/3999181442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf200b2950 0x7fcf200b2dd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c008fd0 tx=0x7fcf0c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:54.656 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.646+0000 7fcf1d7fa640 1 -- 172.21.15.84:0/3999181442 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c0076c0 con 0x7fcf200b2950 2023-10-16T09:48:54.656 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.646+0000 7fcf1d7fa640 1 -- 172.21.15.84:0/3999181442 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c007a30 con 0x7fcf200b2950 2023-10-16T09:48:54.656 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.646+0000 7fcf1d7fa640 1 -- 172.21.15.84:0/3999181442 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c006c00 con 0x7fcf200b2950 2023-10-16T09:48:54.657 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.646+0000 7fcf26a83640 1 -- 172.21.15.84:0/3999181442 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf200b3cb0 con 0x7fcf200b2950 2023-10-16T09:48:54.657 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.646+0000 7fcf26a83640 1 -- 172.21.15.84:0/3999181442 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf200adc60 con 0x7fcf200b2950 2023-10-16T09:48:54.657 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.647+0000 7fcf26a83640 1 -- 172.21.15.84:0/3999181442 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf2004e160 con 0x7fcf200b2950 2023-10-16T09:48:54.657 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.648+0000 7fcf1d7fa640 1 -- 172.21.15.84:0/3999181442 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7fcf0c0035f0 con 0x7fcf200b2950 2023-10-16T09:48:54.658 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.649+0000 7fcf1d7fa640 1 --2- 172.21.15.84:0/3999181442 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fcf0406ade0 0x7fcf0406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:54.658 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.649+0000 7fcf1d7fa640 1 -- 172.21.15.84:0/3999181442 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fcf0c0aaff0 con 0x7fcf200b2950 2023-10-16T09:48:54.659 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.649+0000 7fcf2481f640 1 --2- 172.21.15.84:0/3999181442 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fcf0406ade0 0x7fcf0406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:54.659 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.649+0000 7fcf2481f640 1 --2- 172.21.15.84:0/3999181442 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fcf0406ade0 0x7fcf0406d2a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcf14000fb0 tx=0x7fcf14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:54.659 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.654+0000 7fcf1d7fa640 1 -- 172.21.15.84:0/3999181442 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcf0c076780 con 0x7fcf200b2950 2023-10-16T09:48:54.834 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.832+0000 7fcf26a83640 1 -- 172.21.15.84:0/3999181442 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf200b4910 con 0x7fcf200b2950 2023-10-16T09:48:54.835 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.834+0000 7fcf1d7fa640 1 -- 172.21.15.84:0/3999181442 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10884 (secure 0 0 0) 0x7fcf0c04b090 con 0x7fcf200b2950 2023-10-16T09:48:54.836 INFO:teuthology.misc.health.smithi084.stdout: 2023-10-16T09:48:54.836 INFO:teuthology.misc.health.smithi084.stdout:{"epoch":13,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:48:54.329282+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:48:54.838 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.836+0000 7fcf26a83640 1 -- 172.21.15.84:0/3999181442 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fcf0406ade0 msgr2=0x7fcf0406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:54.838 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.836+0000 7fcf26a83640 1 --2- 172.21.15.84:0/3999181442 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fcf0406ade0 0x7fcf0406d2a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcf14000fb0 tx=0x7fcf14009040 comp rx=0 tx=0).stop 2023-10-16T09:48:54.838 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.836+0000 7fcf26a83640 1 -- 172.21.15.84:0/3999181442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf200b2950 msgr2=0x7fcf200b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:54.839 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.836+0000 7fcf26a83640 1 --2- 172.21.15.84:0/3999181442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf200b2950 0x7fcf200b2dd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c008fd0 tx=0x7fcf0c001120 comp rx=0 tx=0).stop 2023-10-16T09:48:54.839 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.836+0000 7fcf26a83640 1 -- 172.21.15.84:0/3999181442 shutdown_connections 2023-10-16T09:48:54.839 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.836+0000 7fcf26a83640 1 --2- 172.21.15.84:0/3999181442 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fcf0406ade0 0x7fcf0406d2a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:54.839 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.836+0000 7fcf26a83640 1 --2- 172.21.15.84:0/3999181442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf20148830 0x7fcf20148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:54.840 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.836+0000 7fcf26a83640 1 --2- 172.21.15.84:0/3999181442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf200b2950 0x7fcf200b2dd0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:54.840 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.836+0000 7fcf26a83640 1 --2- 172.21.15.84:0/3999181442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf10003f70 0x7fcf20154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:54.840 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.836+0000 7fcf26a83640 1 -- 172.21.15.84:0/3999181442 >> 172.21.15.84:0/3999181442 conn(0x7fcf200cb410 msgr2=0x7fcf2015f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:54.840 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.836+0000 7fcf26a83640 1 -- 172.21.15.84:0/3999181442 shutdown_connections 2023-10-16T09:48:54.840 INFO:teuthology.misc.health.smithi084.stderr:2023-10-16T09:48:54.836+0000 7fcf26a83640 1 -- 172.21.15.84:0/3999181442 wait complete. 2023-10-16T09:48:54.851 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-16T09:48:54.852 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-16T09:48:54.852 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T09:48:54.853 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T09:48:54.861 INFO:tasks.ceph.mds.a:Started 2023-10-16T09:48:54.862 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T09:48:54.862 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T09:48:54.870 INFO:tasks.ceph.mds.c:Started 2023-10-16T09:48:54.870 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T09:48:54.871 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T09:48:54.880 INFO:tasks.ceph.mds.b:Started 2023-10-16T09:48:54.880 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T09:48:54.881 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T09:48:54.891 INFO:tasks.ceph.mds.d:Started 2023-10-16T09:48:54.891 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-16T09:48:54.891 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:48:55.008 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T09:48:55.011 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T09:48:55.043 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T09:48:55.043 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-10-16T09:48:55.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.064+0000 7f89ef426640 1 Processor -- start 2023-10-16T09:48:55.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.064+0000 7f89ef426640 1 -- start start 2023-10-16T09:48:55.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.065+0000 7f89ef426640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f89e815aaa0 0x7f89e815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:55.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.065+0000 7f89ef426640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e815ed60 con 0x7f89e815ae80 2023-10-16T09:48:55.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.065+0000 7f89ef426640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e80707e0 con 0x7f89e80ce390 2023-10-16T09:48:55.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.065+0000 7f89ef426640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e806b810 con 0x7f89e815aaa0 2023-10-16T09:48:55.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.066+0000 7f89ed1c2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f89e815aaa0 0x7f89e815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:55.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.066+0000 7f89ec9c1640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f89e815ae80 0x7f89e80cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60520/0 (socket says 172.21.15.84:60520) 2023-10-16T09:48:55.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.066+0000 7f89ec9c1640 1 -- 172.21.15.84:0/1358044059 learned_addr learned my addr 172.21.15.84:0/1358044059 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:48:55.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.066+0000 7f89ed1c2640 1 -- 172.21.15.84:0/1358044059 >> v1:172.21.15.157:6789/0 conn(0x7f89e80ce390 legacy=0x7f89e8070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:55.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.066+0000 7f89ed1c2640 1 -- 172.21.15.84:0/1358044059 >> v1:172.21.15.84:6789/0 conn(0x7f89e815ae80 legacy=0x7f89e80cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:55.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.066+0000 7f89ed1c2640 1 -- 172.21.15.84:0/1358044059 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e815ef90 con 0x7f89e815aaa0 2023-10-16T09:48:55.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.066+0000 7f89dffff640 1 -- 172.21.15.84:0/1358044059 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2375804185 0 0) 0x7f89e815ed60 con 0x7f89e815ae80 2023-10-16T09:48:55.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.067+0000 7f89ed1c2640 1 --2- 172.21.15.84:0/1358044059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e815aaa0 0x7f89e815e820 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f89e40097a0 tx=0x7f89e402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c28f5827993358 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:55.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.067+0000 7f89dffff640 1 -- 172.21.15.84:0/1358044059 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e4031830 con 0x7f89e815aaa0 2023-10-16T09:48:55.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.067+0000 7f89dffff640 1 -- 172.21.15.84:0/1358044059 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e4031990 con 0x7f89e815aaa0 2023-10-16T09:48:55.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.067+0000 7f89dffff640 1 -- 172.21.15.84:0/1358044059 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e4031d20 con 0x7f89e815aaa0 2023-10-16T09:48:55.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.068+0000 7f89ef426640 1 -- 172.21.15.84:0/1358044059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e815aaa0 msgr2=0x7f89e815e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:55.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.068+0000 7f89ef426640 1 --2- 172.21.15.84:0/1358044059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e815aaa0 0x7f89e815e820 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f89e40097a0 tx=0x7f89e402b8d0 comp rx=0 tx=0).stop 2023-10-16T09:48:55.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.068+0000 7f89ef426640 1 -- 172.21.15.84:0/1358044059 shutdown_connections 2023-10-16T09:48:55.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.068+0000 7f89ef426640 1 --2- 172.21.15.84:0/1358044059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e815aaa0 0x7f89e815e820 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:55.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.068+0000 7f89ef426640 1 -- 172.21.15.84:0/1358044059 >> 172.21.15.84:0/1358044059 conn(0x7f89e80cb410 msgr2=0x7f89e80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:55.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.068+0000 7f89ef426640 1 -- 172.21.15.84:0/1358044059 shutdown_connections 2023-10-16T09:48:55.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.068+0000 7f89ef426640 1 -- 172.21.15.84:0/1358044059 wait complete. 2023-10-16T09:48:55.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.069+0000 7f89ef426640 1 Processor -- start 2023-10-16T09:48:55.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.070+0000 7f89ef426640 1 -- start start 2023-10-16T09:48:55.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.070+0000 7f89ef426640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89e80ce390 0x7f89e80c2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:55.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.070+0000 7f89ef426640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89e815aaa0 0x7f89e80c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:55.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.070+0000 7f89ef426640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e815ae80 0x7f89e8157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:55.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.070+0000 7f89ef426640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e815c610 con 0x7f89e815ae80 2023-10-16T09:48:55.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.070+0000 7f89ef426640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80c2ae0 con 0x7f89e80ce390 2023-10-16T09:48:55.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.070+0000 7f89ef426640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80c2dc0 con 0x7f89e815aaa0 2023-10-16T09:48:55.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.071+0000 7f89ed9c3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e815ae80 0x7f89e8157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:55.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.071+0000 7f89ed9c3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e815ae80 0x7f89e8157490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49642/0 (socket says 172.21.15.84:49642) 2023-10-16T09:48:55.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.071+0000 7f89ed9c3640 1 -- 172.21.15.84:0/188058995 learned_addr learned my addr 172.21.15.84:0/188058995 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:48:55.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.071+0000 7f89ed1c2640 1 --2- 172.21.15.84:0/188058995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89e80ce390 0x7f89e80c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:55.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.071+0000 7f89ec9c1640 1 --2- 172.21.15.84:0/188058995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89e815aaa0 0x7f89e80c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:55.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.071+0000 7f89ed9c3640 1 -- 172.21.15.84:0/188058995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89e815aaa0 msgr2=0x7f89e80c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:55.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.071+0000 7f89ed9c3640 1 --2- 172.21.15.84:0/188058995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89e815aaa0 0x7f89e80c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:55.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.071+0000 7f89ed9c3640 1 -- 172.21.15.84:0/188058995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89e80ce390 msgr2=0x7f89e80c2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:55.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.071+0000 7f89ed9c3640 1 --2- 172.21.15.84:0/188058995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89e80ce390 0x7f89e80c2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:55.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.071+0000 7f89ed9c3640 1 -- 172.21.15.84:0/188058995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4009400 con 0x7f89e815ae80 2023-10-16T09:48:55.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.071+0000 7f89ec9c1640 1 --2- 172.21.15.84:0/188058995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89e815aaa0 0x7f89e80c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:48:55.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.071+0000 7f89ed9c3640 1 --2- 172.21.15.84:0/188058995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e815ae80 0x7f89e8157490 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f89e000c780 tx=0x7f89e000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:55.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.071+0000 7f89ddffb640 1 -- 172.21.15.84:0/188058995 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0004b60 con 0x7f89e815ae80 2023-10-16T09:48:55.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.071+0000 7f89ddffb640 1 -- 172.21.15.84:0/188058995 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e000ce90 con 0x7f89e815ae80 2023-10-16T09:48:55.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.071+0000 7f89ddffb640 1 -- 172.21.15.84:0/188058995 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e000a850 con 0x7f89e815ae80 2023-10-16T09:48:55.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.071+0000 7f89ef426640 1 -- 172.21.15.84:0/188058995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e80c30a0 con 0x7f89e815ae80 2023-10-16T09:48:55.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.072+0000 7f89ef426640 1 -- 172.21.15.84:0/188058995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e80c3660 con 0x7f89e815ae80 2023-10-16T09:48:55.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.073+0000 7f89ef426640 1 -- 172.21.15.84:0/188058995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e80c3880 con 0x7f89e815ae80 2023-10-16T09:48:55.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.074+0000 7f89ddffb640 1 -- 172.21.15.84:0/188058995 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f89e000a9b0 con 0x7f89e815ae80 2023-10-16T09:48:55.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.074+0000 7f89ddffb640 1 --2- 172.21.15.84:0/188058995 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f89cc06ade0 0x7f89cc06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:55.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.074+0000 7f89ddffb640 1 -- 172.21.15.84:0/188058995 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f89e00891f0 con 0x7f89e815ae80 2023-10-16T09:48:55.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.075+0000 7f89ed1c2640 1 --2- 172.21.15.84:0/188058995 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f89cc06ade0 0x7f89cc06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:55.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.075+0000 7f89ed1c2640 1 --2- 172.21.15.84:0/188058995 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f89cc06ade0 0x7f89cc06d2a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f89e4036000 tx=0x7f89e403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:55.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.078+0000 7f89ddffb640 1 -- 172.21.15.84:0/188058995 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f89e0054850 con 0x7f89e815ae80 2023-10-16T09:48:55.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.265+0000 7f89ef426640 1 -- 172.21.15.84:0/188058995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89e804e160 con 0x7f89e815ae80 2023-10-16T09:48:55.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.266+0000 7f89ddffb640 1 -- 172.21.15.84:0/188058995 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10884 (secure 0 0 0) 0x7f89e0057e70 con 0x7f89e815ae80 2023-10-16T09:48:55.268 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:55.268 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":13,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:48:54.329282+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:48:55.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.269+0000 7f89ef426640 1 -- 172.21.15.84:0/188058995 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f89cc06ade0 msgr2=0x7f89cc06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:55.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.269+0000 7f89ef426640 1 --2- 172.21.15.84:0/188058995 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f89cc06ade0 0x7f89cc06d2a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f89e4036000 tx=0x7f89e403c040 comp rx=0 tx=0).stop 2023-10-16T09:48:55.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.269+0000 7f89ef426640 1 -- 172.21.15.84:0/188058995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e815ae80 msgr2=0x7f89e8157490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:55.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.269+0000 7f89ef426640 1 --2- 172.21.15.84:0/188058995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e815ae80 0x7f89e8157490 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f89e000c780 tx=0x7f89e000cc40 comp rx=0 tx=0).stop 2023-10-16T09:48:55.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.269+0000 7f89ef426640 1 -- 172.21.15.84:0/188058995 shutdown_connections 2023-10-16T09:48:55.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.269+0000 7f89ef426640 1 --2- 172.21.15.84:0/188058995 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f89cc06ade0 0x7f89cc06d2a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:55.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.269+0000 7f89ef426640 1 --2- 172.21.15.84:0/188058995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e815ae80 0x7f89e8157490 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:55.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.269+0000 7f89ef426640 1 --2- 172.21.15.84:0/188058995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89e815aaa0 0x7f89e80c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:55.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.269+0000 7f89ef426640 1 --2- 172.21.15.84:0/188058995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89e80ce390 0x7f89e80c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:55.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.269+0000 7f89ef426640 1 -- 172.21.15.84:0/188058995 >> 172.21.15.84:0/188058995 conn(0x7f89e80cb410 msgr2=0x7f89e80b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:55.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.269+0000 7f89ef426640 1 -- 172.21.15.84:0/188058995 shutdown_connections 2023-10-16T09:48:55.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.269+0000 7f89ef426640 1 -- 172.21.15.84:0/188058995 wait complete. 2023-10-16T09:48:55.286 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2023-10-16T09:48:55.287 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:48:55.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.473+0000 7f22400f2640 1 Processor -- start 2023-10-16T09:48:55.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.473+0000 7f22400f2640 1 -- start start 2023-10-16T09:48:55.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.475+0000 7f22400f2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f223806c3f0 0x7f223806c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:55.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.475+0000 7f22400f2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815e0c0 0x7f22380b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:55.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.475+0000 7f22400f2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22380cd630 con 0x7f223815e540 2023-10-16T09:48:55.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.475+0000 7f22400f2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22380ce1b0 con 0x7f223806c3f0 2023-10-16T09:48:55.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.475+0000 7f22400f2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223806cd10 con 0x7f223815e0c0 2023-10-16T09:48:55.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.476+0000 7f223d68d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815e0c0 0x7f22380b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:55.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.476+0000 7f223d68d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815e0c0 0x7f22380b4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:59978/0 (socket says 172.21.15.84:59978) 2023-10-16T09:48:55.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.476+0000 7f223d68d640 1 -- 172.21.15.84:0/3855982452 learned_addr learned my addr 172.21.15.84:0/3855982452 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:48:55.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.476+0000 7f223de8e640 1 --2- 172.21.15.84:0/3855982452 >> v2:172.21.15.157:3300/0 conn(0x7f223806c3f0 0x7f223806c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:55.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.476+0000 7f223d68d640 1 -- 172.21.15.84:0/3855982452 >> v2:172.21.15.157:3300/0 conn(0x7f223806c3f0 msgr2=0x7f223806c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:55.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.476+0000 7f223d68d640 1 --2- 172.21.15.84:0/3855982452 >> v2:172.21.15.157:3300/0 conn(0x7f223806c3f0 0x7f223806c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:55.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.476+0000 7f223d68d640 1 -- 172.21.15.84:0/3855982452 >> v1:172.21.15.157:6789/0 conn(0x7f223815e540 legacy=0x7f22380b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:55.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.476+0000 7f223d68d640 1 -- 172.21.15.84:0/3855982452 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22380d2170 con 0x7f223815e0c0 2023-10-16T09:48:55.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.478+0000 7f223d68d640 1 --2- 172.21.15.84:0/3855982452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815e0c0 0x7f22380b4730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2228009870 tx=0x7f222802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e28fb6af080a161e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:55.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.478+0000 7f223ce8c640 1 -- 172.21.15.84:0/3855982452 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228030840 con 0x7f223815e0c0 2023-10-16T09:48:55.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.478+0000 7f223ce8c640 1 -- 172.21.15.84:0/3855982452 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22280309a0 con 0x7f223815e0c0 2023-10-16T09:48:55.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.478+0000 7f223ce8c640 1 -- 172.21.15.84:0/3855982452 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228030d30 con 0x7f223815e0c0 2023-10-16T09:48:55.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.479+0000 7f22400f2640 1 -- 172.21.15.84:0/3855982452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815e0c0 msgr2=0x7f22380b4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:55.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.479+0000 7f22400f2640 1 --2- 172.21.15.84:0/3855982452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815e0c0 0x7f22380b4730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2228009870 tx=0x7f222802b980 comp rx=0 tx=0).stop 2023-10-16T09:48:55.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.479+0000 7f22400f2640 1 -- 172.21.15.84:0/3855982452 shutdown_connections 2023-10-16T09:48:55.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.479+0000 7f22400f2640 1 --2- 172.21.15.84:0/3855982452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815e0c0 0x7f22380b4730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:55.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.479+0000 7f22400f2640 1 --2- 172.21.15.84:0/3855982452 >> v2:172.21.15.157:3300/0 conn(0x7f223806c3f0 0x7f223806c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:55.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.479+0000 7f22400f2640 1 -- 172.21.15.84:0/3855982452 >> 172.21.15.84:0/3855982452 conn(0x7f22380d2cb0 msgr2=0x7f22380d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:55.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.479+0000 7f22400f2640 1 -- 172.21.15.84:0/3855982452 shutdown_connections 2023-10-16T09:48:55.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.479+0000 7f22400f2640 1 -- 172.21.15.84:0/3855982452 wait complete. 2023-10-16T09:48:55.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.481+0000 7f22400f2640 1 Processor -- start 2023-10-16T09:48:55.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.481+0000 7f22400f2640 1 -- start start 2023-10-16T09:48:55.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.482+0000 7f22400f2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f223806c3f0 0x7f22380c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:55.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.482+0000 7f22400f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f223815e0c0 0x7f22380c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:55.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.482+0000 7f22400f2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815e540 0x7f223815c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:55.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.482+0000 7f22400f2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22380cb7e0 con 0x7f223806c3f0 2023-10-16T09:48:55.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.482+0000 7f22400f2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223815ce10 con 0x7f223815e0c0 2023-10-16T09:48:55.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.482+0000 7f22400f2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223815d0f0 con 0x7f223815e540 2023-10-16T09:48:55.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.483+0000 7f223de8e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f223806c3f0 0x7f22380c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:55.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.483+0000 7f223de8e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f223806c3f0 0x7f22380c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49654/0 (socket says 172.21.15.84:49654) 2023-10-16T09:48:55.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.483+0000 7f223de8e640 1 -- 172.21.15.84:0/288127955 learned_addr learned my addr 172.21.15.84:0/288127955 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:48:55.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.483+0000 7f223d68d640 1 --2- 172.21.15.84:0/288127955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f223815e0c0 0x7f22380c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:55.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.483+0000 7f223e68f640 1 --2- 172.21.15.84:0/288127955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815e540 0x7f223815c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:55.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.483+0000 7f223de8e640 1 -- 172.21.15.84:0/288127955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815e540 msgr2=0x7f223815c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:55.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.483+0000 7f223de8e640 1 --2- 172.21.15.84:0/288127955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815e540 0x7f223815c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:55.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.483+0000 7f223de8e640 1 -- 172.21.15.84:0/288127955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f223815e0c0 msgr2=0x7f22380c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:55.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.483+0000 7f223de8e640 1 --2- 172.21.15.84:0/288127955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f223815e0c0 0x7f22380c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:55.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.483+0000 7f223de8e640 1 -- 172.21.15.84:0/288127955 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22280094d0 con 0x7f223806c3f0 2023-10-16T09:48:55.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.483+0000 7f223d68d640 1 --2- 172.21.15.84:0/288127955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f223815e0c0 0x7f22380c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:48:55.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.483+0000 7f223de8e640 1 --2- 172.21.15.84:0/288127955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f223806c3f0 0x7f22380c9fd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f223400e870 tx=0x7f223400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:55.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.484+0000 7f222e7fc640 1 -- 172.21.15.84:0/288127955 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22340094f0 con 0x7f223806c3f0 2023-10-16T09:48:55.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.484+0000 7f222e7fc640 1 -- 172.21.15.84:0/288127955 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223400c940 con 0x7f223806c3f0 2023-10-16T09:48:55.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.484+0000 7f22400f2640 1 -- 172.21.15.84:0/288127955 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22380cba60 con 0x7f223806c3f0 2023-10-16T09:48:55.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.484+0000 7f222e7fc640 1 -- 172.21.15.84:0/288127955 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223400ccf0 con 0x7f223806c3f0 2023-10-16T09:48:55.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.484+0000 7f22400f2640 1 -- 172.21.15.84:0/288127955 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223815cb00 con 0x7f223806c3f0 2023-10-16T09:48:55.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.485+0000 7f22400f2640 1 -- 172.21.15.84:0/288127955 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22380cd860 con 0x7f223806c3f0 2023-10-16T09:48:55.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.486+0000 7f222e7fc640 1 -- 172.21.15.84:0/288127955 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f2234013470 con 0x7f223806c3f0 2023-10-16T09:48:55.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.487+0000 7f222e7fc640 1 --2- 172.21.15.84:0/288127955 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f221c06ade0 0x7f221c06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:55.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.487+0000 7f222e7fc640 1 -- 172.21.15.84:0/288127955 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f22340891f0 con 0x7f223806c3f0 2023-10-16T09:48:55.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.487+0000 7f223d68d640 1 --2- 172.21.15.84:0/288127955 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f221c06ade0 0x7f221c06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:55.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.488+0000 7f223d68d640 1 --2- 172.21.15.84:0/288127955 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f221c06ade0 0x7f221c06d2a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2228036000 tx=0x7f222803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:55.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.491+0000 7f222e7fc640 1 -- 172.21.15.84:0/288127955 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f22380cd860 con 0x7f223806c3f0 2023-10-16T09:48:55.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.721+0000 7f22400f2640 1 -- 172.21.15.84:0/288127955 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22380cd860 con 0x7f223806c3f0 2023-10-16T09:48:55.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.722+0000 7f222e7fc640 1 -- 172.21.15.84:0/288127955 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f2234021540 con 0x7f223806c3f0 2023-10-16T09:48:55.724 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:55.724 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3716581310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3716581310},{"type":"v1","addr":"172.21.15.84:6835","nonce":3716581310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-16T09:48:55.724 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 2 2023-10-16T09:48:55.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.724+0000 7f22400f2640 1 -- 172.21.15.84:0/288127955 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f221c06ade0 msgr2=0x7f221c06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:55.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.724+0000 7f22400f2640 1 --2- 172.21.15.84:0/288127955 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f221c06ade0 0x7f221c06d2a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2228036000 tx=0x7f222803b040 comp rx=0 tx=0).stop 2023-10-16T09:48:55.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.724+0000 7f22400f2640 1 -- 172.21.15.84:0/288127955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f223806c3f0 msgr2=0x7f22380c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:55.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.724+0000 7f22400f2640 1 --2- 172.21.15.84:0/288127955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f223806c3f0 0x7f22380c9fd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f223400e870 tx=0x7f223400ed30 comp rx=0 tx=0).stop 2023-10-16T09:48:55.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.724+0000 7f22400f2640 1 -- 172.21.15.84:0/288127955 shutdown_connections 2023-10-16T09:48:55.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.724+0000 7f22400f2640 1 --2- 172.21.15.84:0/288127955 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f221c06ade0 0x7f221c06d2a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:55.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.724+0000 7f22400f2640 1 --2- 172.21.15.84:0/288127955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815e540 0x7f223815c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:55.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.724+0000 7f22400f2640 1 --2- 172.21.15.84:0/288127955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f223815e0c0 0x7f22380c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:55.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.724+0000 7f22400f2640 1 --2- 172.21.15.84:0/288127955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f223806c3f0 0x7f22380c9fd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:55.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.724+0000 7f22400f2640 1 -- 172.21.15.84:0/288127955 >> 172.21.15.84:0/288127955 conn(0x7f22380d2cb0 msgr2=0x7f22380b81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:55.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.725+0000 7f22400f2640 1 -- 172.21.15.84:0/288127955 shutdown_connections 2023-10-16T09:48:55.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:55.725+0000 7f22400f2640 1 -- 172.21.15.84:0/288127955 wait complete. 2023-10-16T09:48:57.744 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:48:57.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.947+0000 7f312af2c640 1 Processor -- start 2023-10-16T09:48:57.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.947+0000 7f312af2c640 1 -- start start 2023-10-16T09:48:57.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.948+0000 7f312af2c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31240b8480 0x7f31240cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:57.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.949+0000 7f312af2c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312415d150 con 0x7f312415cd70 2023-10-16T09:48:57.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.949+0000 7f312af2c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31240d1180 con 0x7f31240d0be0 2023-10-16T09:48:57.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.949+0000 7f312af2c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31240d1460 con 0x7f31240b8480 2023-10-16T09:48:57.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.949+0000 7f3129f2a640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f312415cd70 0x7f31240d2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52826/0 (socket says 172.21.15.84:52826) 2023-10-16T09:48:57.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.949+0000 7f3129f2a640 1 -- 172.21.15.84:0/83288471 learned_addr learned my addr 172.21.15.84:0/83288471 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:48:57.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.949+0000 7f3128f28640 1 -- 172.21.15.84:0/83288471 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3779903429 0 0) 0x7f312415d150 con 0x7f312415cd70 2023-10-16T09:48:57.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.949+0000 7f3129729640 1 --2- 172.21.15.84:0/83288471 >> v2:172.21.15.157:3300/0 conn(0x7f31240b8480 0x7f31240cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:57.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.949+0000 7f3128f28640 1 -- 172.21.15.84:0/83288471 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3110003520 con 0x7f312415cd70 2023-10-16T09:48:57.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.950+0000 7f3128f28640 1 -- 172.21.15.84:0/83288471 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2525732645 0 0) 0x7f3110003520 con 0x7f312415cd70 2023-10-16T09:48:57.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.950+0000 7f3128f28640 1 -- 172.21.15.84:0/83288471 >> v2:172.21.15.157:3300/0 conn(0x7f31240b8480 msgr2=0x7f31240cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:57.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.950+0000 7f3128f28640 1 --2- 172.21.15.84:0/83288471 >> v2:172.21.15.157:3300/0 conn(0x7f31240b8480 0x7f31240cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:57.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.950+0000 7f3128f28640 1 -- 172.21.15.84:0/83288471 >> v1:172.21.15.157:6789/0 conn(0x7f31240d0be0 legacy=0x7f31240b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:57.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.950+0000 7f3128f28640 1 -- 172.21.15.84:0/83288471 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31240d1740 con 0x7f312415cd70 2023-10-16T09:48:57.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.950+0000 7f3128f28640 1 -- 172.21.15.84:0/83288471 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f3120001670 con 0x7f312415cd70 2023-10-16T09:48:57.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.950+0000 7f3128f28640 1 -- 172.21.15.84:0/83288471 >> v1:172.21.15.84:6789/0 conn(0x7f312415cd70 legacy=0x7f31240d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:57.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.950+0000 7f3128f28640 1 --2- 172.21.15.84:0/83288471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3110003970 0x7f3110003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:57.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.950+0000 7f3128f28640 1 --2- 172.21.15.84:0/83288471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31100047c0 0x7f3110004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:57.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.950+0000 7f3128f28640 1 --2- 172.21.15.84:0/83288471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110005180 0x7f311000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:57.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.950+0000 7f3128f28640 1 -- 172.21.15.84:0/83288471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31240d1460 con 0x7f3110005180 2023-10-16T09:48:57.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.950+0000 7f3128f28640 1 -- 172.21.15.84:0/83288471 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311000c120 con 0x7f31100047c0 2023-10-16T09:48:57.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.950+0000 7f3128f28640 1 -- 172.21.15.84:0/83288471 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311000c2e0 con 0x7f3110003970 2023-10-16T09:48:57.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.950+0000 7f312a72b640 1 --2- 172.21.15.84:0/83288471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110005180 0x7f311000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:57.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.950+0000 7f3129729640 1 --2- 172.21.15.84:0/83288471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31100047c0 0x7f3110004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:57.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.950+0000 7f3129f2a640 1 --2- 172.21.15.84:0/83288471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3110003970 0x7f3110003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:57.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.951+0000 7f312a72b640 1 -- 172.21.15.84:0/83288471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3110003970 msgr2=0x7f3110003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:57.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.951+0000 7f312a72b640 1 --2- 172.21.15.84:0/83288471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3110003970 0x7f3110003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:57.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.951+0000 7f312a72b640 1 -- 172.21.15.84:0/83288471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31100047c0 msgr2=0x7f3110004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:57.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.951+0000 7f312a72b640 1 --2- 172.21.15.84:0/83288471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31100047c0 0x7f3110004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:57.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.951+0000 7f312a72b640 1 -- 172.21.15.84:0/83288471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3110003520 con 0x7f3110005180 2023-10-16T09:48:57.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.951+0000 7f3129f2a640 1 --2- 172.21.15.84:0/83288471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3110003970 0x7f3110003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:48:57.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.951+0000 7f3129729640 1 --2- 172.21.15.84:0/83288471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31100047c0 0x7f3110004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:48:57.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.951+0000 7f312a72b640 1 -- 172.21.15.84:0/83288471 reap_dead start 2023-10-16T09:48:57.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.952+0000 7f312a72b640 1 --2- 172.21.15.84:0/83288471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110005180 0x7f311000b980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f311c001e50 tx=0x7f31100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1762223af9e26fb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:57.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.952+0000 7f3128f28640 1 -- 172.21.15.84:0/83288471 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c003b70 con 0x7f3110005180 2023-10-16T09:48:57.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.952+0000 7f3128f28640 1 -- 172.21.15.84:0/83288471 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c003d10 con 0x7f3110005180 2023-10-16T09:48:57.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.952+0000 7f3128f28640 1 -- 172.21.15.84:0/83288471 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c005310 con 0x7f3110005180 2023-10-16T09:48:57.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.952+0000 7f312af2c640 1 -- 172.21.15.84:0/83288471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110005180 msgr2=0x7f311000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:57.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.952+0000 7f312af2c640 1 --2- 172.21.15.84:0/83288471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110005180 0x7f311000b980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f311c001e50 tx=0x7f31100057f0 comp rx=0 tx=0).stop 2023-10-16T09:48:57.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.952+0000 7f312af2c640 1 -- 172.21.15.84:0/83288471 shutdown_connections 2023-10-16T09:48:57.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.952+0000 7f312af2c640 1 --2- 172.21.15.84:0/83288471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110005180 0x7f311000b980 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:57.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.953+0000 7f312af2c640 1 -- 172.21.15.84:0/83288471 >> 172.21.15.84:0/83288471 conn(0x7f31240cf570 msgr2=0x7f31240cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:57.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.953+0000 7f312af2c640 1 -- 172.21.15.84:0/83288471 shutdown_connections 2023-10-16T09:48:57.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.953+0000 7f312af2c640 1 -- 172.21.15.84:0/83288471 wait complete. 2023-10-16T09:48:57.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.954+0000 7f312af2c640 1 Processor -- start 2023-10-16T09:48:57.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.954+0000 7f312af2c640 1 -- start start 2023-10-16T09:48:57.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.955+0000 7f312af2c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110005180 0x7f3124150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:57.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.955+0000 7f312af2c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f312415f850 0x7f31240ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:57.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.955+0000 7f312af2c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31240cab10 0x7f31240cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:57.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.955+0000 7f312af2c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312415f430 con 0x7f3110005180 2023-10-16T09:48:57.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.955+0000 7f312af2c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312415e7b0 con 0x7f31240cab10 2023-10-16T09:48:57.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.955+0000 7f312af2c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312415ea90 con 0x7f312415f850 2023-10-16T09:48:57.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.956+0000 7f3129f2a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110005180 0x7f3124150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:57.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.956+0000 7f3129f2a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110005180 0x7f3124150f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52720/0 (socket says 172.21.15.84:52720) 2023-10-16T09:48:57.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.956+0000 7f3129f2a640 1 -- 172.21.15.84:0/3122175026 learned_addr learned my addr 172.21.15.84:0/3122175026 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:48:57.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.956+0000 7f312a72b640 1 --2- 172.21.15.84:0/3122175026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31240cab10 0x7f31240cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:57.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.956+0000 7f3129729640 1 --2- 172.21.15.84:0/3122175026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f312415f850 0x7f31240ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:57.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.956+0000 7f3129f2a640 1 -- 172.21.15.84:0/3122175026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f312415f850 msgr2=0x7f31240ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:57.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.956+0000 7f3129f2a640 1 --2- 172.21.15.84:0/3122175026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f312415f850 0x7f31240ca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:57.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.956+0000 7f3129f2a640 1 -- 172.21.15.84:0/3122175026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31240cab10 msgr2=0x7f31240cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:57.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.956+0000 7f3129f2a640 1 --2- 172.21.15.84:0/3122175026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31240cab10 0x7f31240cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:57.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.956+0000 7f3129f2a640 1 -- 172.21.15.84:0/3122175026 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312415ed70 con 0x7f3110005180 2023-10-16T09:48:57.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.956+0000 7f3129729640 1 --2- 172.21.15.84:0/3122175026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f312415f850 0x7f31240ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:48:57.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.956+0000 7f3129f2a640 1 --2- 172.21.15.84:0/3122175026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110005180 0x7f3124150f70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3120001200 tx=0x7f312000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:57.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.957+0000 7f311a7fc640 1 -- 172.21.15.84:0/3122175026 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31200074e0 con 0x7f3110005180 2023-10-16T09:48:57.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.957+0000 7f311a7fc640 1 -- 172.21.15.84:0/3122175026 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3120007640 con 0x7f3110005180 2023-10-16T09:48:57.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.957+0000 7f311a7fc640 1 -- 172.21.15.84:0/3122175026 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31200049e0 con 0x7f3110005180 2023-10-16T09:48:57.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.957+0000 7f312af2c640 1 -- 172.21.15.84:0/3122175026 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f312415f050 con 0x7f3110005180 2023-10-16T09:48:57.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.957+0000 7f312af2c640 1 -- 172.21.15.84:0/3122175026 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312415e460 con 0x7f3110005180 2023-10-16T09:48:57.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.958+0000 7f312af2c640 1 -- 172.21.15.84:0/3122175026 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31240caef0 con 0x7f3110005180 2023-10-16T09:48:57.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.959+0000 7f311a7fc640 1 -- 172.21.15.84:0/3122175026 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f3120004b40 con 0x7f3110005180 2023-10-16T09:48:57.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.960+0000 7f311a7fc640 1 --2- 172.21.15.84:0/3122175026 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f310806ade0 0x7f310806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:57.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.960+0000 7f311a7fc640 1 -- 172.21.15.84:0/3122175026 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3120088b80 con 0x7f3110005180 2023-10-16T09:48:57.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.960+0000 7f3129729640 1 --2- 172.21.15.84:0/3122175026 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f310806ade0 0x7f310806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:57.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.961+0000 7f3129729640 1 --2- 172.21.15.84:0/3122175026 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f310806ade0 0x7f310806d2a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3114006f80 tx=0x7f3114008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:57.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:57.965+0000 7f311a7fc640 1 -- 172.21.15.84:0/3122175026 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f31240caef0 con 0x7f3110005180 2023-10-16T09:48:58.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.203+0000 7f312af2c640 1 -- 172.21.15.84:0/3122175026 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31240caef0 con 0x7f3110005180 2023-10-16T09:48:58.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.204+0000 7f311a7fc640 1 -- 172.21.15.84:0/3122175026 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7f31240caef0 con 0x7f3110005180 2023-10-16T09:48:58.206 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:58.207 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2978888131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2978888131},{"type":"v1","addr":"172.21.15.157:6834","nonce":2978888131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3716581310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3716581310},{"type":"v1","addr":"172.21.15.84:6835","nonce":3716581310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2761479872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2761479872},{"type":"v1","addr":"172.21.15.84:6837","nonce":2761479872}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/848754825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":848754825},{"type":"v1","addr":"172.21.15.157:6835","nonce":848754825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-16T09:48:58.207 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 3 2023-10-16T09:48:58.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.207+0000 7f312af2c640 1 -- 172.21.15.84:0/3122175026 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f310806ade0 msgr2=0x7f310806d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:58.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.207+0000 7f312af2c640 1 --2- 172.21.15.84:0/3122175026 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f310806ade0 0x7f310806d2a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3114006f80 tx=0x7f3114008040 comp rx=0 tx=0).stop 2023-10-16T09:48:58.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.207+0000 7f312af2c640 1 -- 172.21.15.84:0/3122175026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110005180 msgr2=0x7f3124150f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:58.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.207+0000 7f312af2c640 1 --2- 172.21.15.84:0/3122175026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110005180 0x7f3124150f70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3120001200 tx=0x7f312000dd40 comp rx=0 tx=0).stop 2023-10-16T09:48:58.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.207+0000 7f312af2c640 1 -- 172.21.15.84:0/3122175026 shutdown_connections 2023-10-16T09:48:58.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.207+0000 7f312af2c640 1 --2- 172.21.15.84:0/3122175026 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f310806ade0 0x7f310806d2a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.207+0000 7f312af2c640 1 --2- 172.21.15.84:0/3122175026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31240cab10 0x7f31240cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.207+0000 7f312af2c640 1 --2- 172.21.15.84:0/3122175026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f312415f850 0x7f31240ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.207+0000 7f312af2c640 1 --2- 172.21.15.84:0/3122175026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110005180 0x7f3124150f70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.207+0000 7f312af2c640 1 -- 172.21.15.84:0/3122175026 >> 172.21.15.84:0/3122175026 conn(0x7f31240cf570 msgr2=0x7f312415bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:58.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.207+0000 7f312af2c640 1 -- 172.21.15.84:0/3122175026 shutdown_connections 2023-10-16T09:48:58.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.207+0000 7f312af2c640 1 -- 172.21.15.84:0/3122175026 wait complete. 2023-10-16T09:48:58.227 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:48:58.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.413+0000 7f4268f97640 1 Processor -- start 2023-10-16T09:48:58.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.413+0000 7f4268f97640 1 -- start start 2023-10-16T09:48:58.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.414+0000 7f4268f97640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42640cd0c0 0x7f42641615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:58.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.414+0000 7f4268f97640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f426415db00 0x7f42640b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:58.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.414+0000 7f4268f97640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42640b0db0 con 0x7f42640b0410 2023-10-16T09:48:58.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.414+0000 7f4268f97640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426415e3e0 con 0x7f426415db00 2023-10-16T09:48:58.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.414+0000 7f4268f97640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42640cd4a0 con 0x7f42640cd0c0 2023-10-16T09:48:58.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.415+0000 7f426259c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42640cd0c0 0x7f42641615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:58.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.415+0000 7f4262d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f426415db00 0x7f42640b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:58.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.415+0000 7f426259c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42640cd0c0 0x7f42641615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:33008/0 (socket says 172.21.15.84:33008) 2023-10-16T09:48:58.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.415+0000 7f426259c640 1 -- 172.21.15.84:0/4242154170 learned_addr learned my addr 172.21.15.84:0/4242154170 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:48:58.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.415+0000 7f4262d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f426415db00 0x7f42640b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:51058/0 (socket says 172.21.15.84:51058) 2023-10-16T09:48:58.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.416+0000 7f426259c640 1 -- 172.21.15.84:0/4242154170 >> v2:172.21.15.157:3300/0 conn(0x7f426415db00 msgr2=0x7f42640b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:58.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.416+0000 7f426259c640 1 --2- 172.21.15.84:0/4242154170 >> v2:172.21.15.157:3300/0 conn(0x7f426415db00 0x7f42640b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.416+0000 7f426259c640 1 -- 172.21.15.84:0/4242154170 >> v1:172.21.15.157:6789/0 conn(0x7f42640b0410 legacy=0x7f426415d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:58.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.416+0000 7f426259c640 1 -- 172.21.15.84:0/4242154170 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426415e100 con 0x7f42640cd0c0 2023-10-16T09:48:58.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.416+0000 7f4262d9d640 1 --2- 172.21.15.84:0/4242154170 >> v2:172.21.15.157:3300/0 conn(0x7f426415db00 0x7f42640b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:48:58.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.417+0000 7f426259c640 1 --2- 172.21.15.84:0/4242154170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42640cd0c0 0x7f42641615f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4254009870 tx=0x7f425402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17c22e403bccba2b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:58.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.418+0000 7f4261d9b640 1 -- 172.21.15.84:0/4242154170 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254031640 con 0x7f42640cd0c0 2023-10-16T09:48:58.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.418+0000 7f4261d9b640 1 -- 172.21.15.84:0/4242154170 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42540317a0 con 0x7f42640cd0c0 2023-10-16T09:48:58.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.418+0000 7f4261d9b640 1 -- 172.21.15.84:0/4242154170 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254031b30 con 0x7f42640cd0c0 2023-10-16T09:48:58.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.418+0000 7f4268f97640 1 -- 172.21.15.84:0/4242154170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42640cd0c0 msgr2=0x7f42641615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:58.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.418+0000 7f4268f97640 1 --2- 172.21.15.84:0/4242154170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42640cd0c0 0x7f42641615f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4254009870 tx=0x7f425402b8f0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.418+0000 7f4268f97640 1 -- 172.21.15.84:0/4242154170 shutdown_connections 2023-10-16T09:48:58.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.418+0000 7f4268f97640 1 --2- 172.21.15.84:0/4242154170 >> v2:172.21.15.157:3300/0 conn(0x7f426415db00 0x7f42640b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.418+0000 7f4268f97640 1 --2- 172.21.15.84:0/4242154170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42640cd0c0 0x7f42641615f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.418+0000 7f4268f97640 1 -- 172.21.15.84:0/4242154170 >> 172.21.15.84:0/4242154170 conn(0x7f42640cb410 msgr2=0x7f42640cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:58.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.418+0000 7f4268f97640 1 -- 172.21.15.84:0/4242154170 shutdown_connections 2023-10-16T09:48:58.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.418+0000 7f4268f97640 1 -- 172.21.15.84:0/4242154170 wait complete. 2023-10-16T09:48:58.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.420+0000 7f4268f97640 1 Processor -- start 2023-10-16T09:48:58.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.420+0000 7f4268f97640 1 -- start start 2023-10-16T09:48:58.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.421+0000 7f4268f97640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42640b0410 0x7f426406f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:58.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.421+0000 7f4268f97640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42640cd0c0 0x7f42640c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:58.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.421+0000 7f4268f97640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f426415db00 0x7f42641e69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:58.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.421+0000 7f4268f97640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42640c7220 con 0x7f42640cd0c0 2023-10-16T09:48:58.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.421+0000 7f4268f97640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42640c7500 con 0x7f426415db00 2023-10-16T09:48:58.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.421+0000 7f4268f97640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42640c77e0 con 0x7f42640b0410 2023-10-16T09:48:58.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.422+0000 7f425bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42640cd0c0 0x7f42640c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:58.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.422+0000 7f425bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42640cd0c0 0x7f42640c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52728/0 (socket says 172.21.15.84:52728) 2023-10-16T09:48:58.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.422+0000 7f425bfff640 1 -- 172.21.15.84:0/3009553325 learned_addr learned my addr 172.21.15.84:0/3009553325 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:48:58.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.422+0000 7f426259c640 1 --2- 172.21.15.84:0/3009553325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42640b0410 0x7f426406f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:58.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.422+0000 7f4262d9d640 1 --2- 172.21.15.84:0/3009553325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f426415db00 0x7f42641e69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:58.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.422+0000 7f425bfff640 1 -- 172.21.15.84:0/3009553325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42640b0410 msgr2=0x7f426406f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:58.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.422+0000 7f425bfff640 1 --2- 172.21.15.84:0/3009553325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42640b0410 0x7f426406f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.422+0000 7f425bfff640 1 -- 172.21.15.84:0/3009553325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f426415db00 msgr2=0x7f42641e69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:58.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.422+0000 7f425bfff640 1 --2- 172.21.15.84:0/3009553325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f426415db00 0x7f42641e69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.422+0000 7f425bfff640 1 -- 172.21.15.84:0/3009553325 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c009a70 con 0x7f42640cd0c0 2023-10-16T09:48:58.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.422+0000 7f426259c640 1 --2- 172.21.15.84:0/3009553325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42640b0410 0x7f426406f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:48:58.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.422+0000 7f425bfff640 1 --2- 172.21.15.84:0/3009553325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42640cd0c0 0x7f42640c82c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f424c00c3c0 tx=0x7f424c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:58.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.423+0000 7f425affd640 1 -- 172.21.15.84:0/3009553325 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c0052d0 con 0x7f42640cd0c0 2023-10-16T09:48:58.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.423+0000 7f425affd640 1 -- 172.21.15.84:0/3009553325 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424c005450 con 0x7f42640cd0c0 2023-10-16T09:48:58.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.423+0000 7f4268f97640 1 -- 172.21.15.84:0/3009553325 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42540094d0 con 0x7f42640cd0c0 2023-10-16T09:48:58.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.423+0000 7f425affd640 1 -- 172.21.15.84:0/3009553325 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c007260 con 0x7f42640cd0c0 2023-10-16T09:48:58.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.423+0000 7f4268f97640 1 -- 172.21.15.84:0/3009553325 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42640c7da0 con 0x7f42640cd0c0 2023-10-16T09:48:58.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.424+0000 7f4268f97640 1 -- 172.21.15.84:0/3009553325 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42640cbd20 con 0x7f42640cd0c0 2023-10-16T09:48:58.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.425+0000 7f425affd640 1 -- 172.21.15.84:0/3009553325 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f424c0073c0 con 0x7f42640cd0c0 2023-10-16T09:48:58.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.426+0000 7f425affd640 1 --2- 172.21.15.84:0/3009553325 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f424406ade0 0x7f424406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:58.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.426+0000 7f425affd640 1 -- 172.21.15.84:0/3009553325 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f424c088680 con 0x7f42640cd0c0 2023-10-16T09:48:58.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.426+0000 7f426259c640 1 --2- 172.21.15.84:0/3009553325 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f424406ade0 0x7f424406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:58.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.427+0000 7f426259c640 1 --2- 172.21.15.84:0/3009553325 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f424406ade0 0x7f424406d2a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4254036000 tx=0x7f425403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:58.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.434+0000 7f425affd640 1 -- 172.21.15.84:0/3009553325 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f424c053a30 con 0x7f42640cd0c0 2023-10-16T09:48:58.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.635+0000 7f4268f97640 1 -- 172.21.15.84:0/3009553325 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f426404e160 con 0x7f42640cd0c0 2023-10-16T09:48:58.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.636+0000 7f425affd640 1 -- 172.21.15.84:0/3009553325 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12826 (secure 0 0 0) 0x7f424c080e60 con 0x7f42640cd0c0 2023-10-16T09:48:58.639 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:58.639 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":17,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:48:58.527013+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:48:58.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.639+0000 7f4268f97640 1 -- 172.21.15.84:0/3009553325 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f424406ade0 msgr2=0x7f424406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:58.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.639+0000 7f4268f97640 1 --2- 172.21.15.84:0/3009553325 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f424406ade0 0x7f424406d2a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4254036000 tx=0x7f425403c040 comp rx=0 tx=0).stop 2023-10-16T09:48:58.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.639+0000 7f4268f97640 1 -- 172.21.15.84:0/3009553325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42640cd0c0 msgr2=0x7f42640c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:58.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.639+0000 7f4268f97640 1 --2- 172.21.15.84:0/3009553325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42640cd0c0 0x7f42640c82c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f424c00c3c0 tx=0x7f424c00c880 comp rx=0 tx=0).stop 2023-10-16T09:48:58.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.639+0000 7f4268f97640 1 -- 172.21.15.84:0/3009553325 shutdown_connections 2023-10-16T09:48:58.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.639+0000 7f4268f97640 1 --2- 172.21.15.84:0/3009553325 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f424406ade0 0x7f424406d2a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.639+0000 7f4268f97640 1 --2- 172.21.15.84:0/3009553325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f426415db00 0x7f42641e69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.639+0000 7f4268f97640 1 --2- 172.21.15.84:0/3009553325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42640cd0c0 0x7f42640c82c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.639+0000 7f4268f97640 1 --2- 172.21.15.84:0/3009553325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42640b0410 0x7f426406f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.639+0000 7f4268f97640 1 -- 172.21.15.84:0/3009553325 >> 172.21.15.84:0/3009553325 conn(0x7f42640cb410 msgr2=0x7f42640c9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:58.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.639+0000 7f4268f97640 1 -- 172.21.15.84:0/3009553325 shutdown_connections 2023-10-16T09:48:58.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.639+0000 7f4268f97640 1 -- 172.21.15.84:0/3009553325 wait complete. 2023-10-16T09:48:58.658 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:48:58.659 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:48:58.672 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:48:58.710 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:48:58.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.844+0000 7f35b62f2640 1 Processor -- start 2023-10-16T09:48:58.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.844+0000 7f35b62f2640 1 -- start start 2023-10-16T09:48:58.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.845+0000 7f35b62f2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35b015d330 0x7f35b01618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:58.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.845+0000 7f35b62f2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35b006c7d0 0x7f35b0065ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:58.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.845+0000 7f35b62f2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b00d0890 con 0x7f35b006cc80 2023-10-16T09:48:58.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.845+0000 7f35b62f2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b006b1e0 con 0x7f35b015d330 2023-10-16T09:48:58.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.845+0000 7f35b62f2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b00d05b0 con 0x7f35b006c7d0 2023-10-16T09:48:58.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.846+0000 7f35b488f640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f35b006cc80 0x7f35b00ce9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52840/0 (socket says 172.21.15.84:52840) 2023-10-16T09:48:58.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.846+0000 7f35b488f640 1 -- 172.21.15.84:0/1902070432 learned_addr learned my addr 172.21.15.84:0/1902070432 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:48:58.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.846+0000 7f35affff640 1 --2- 172.21.15.84:0/1902070432 >> v2:172.21.15.157:3300/0 conn(0x7f35b015d330 0x7f35b01618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:58.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.846+0000 7f35aeffd640 1 -- 172.21.15.84:0/1902070432 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4153429868 0 0) 0x7f35b00d0890 con 0x7f35b006cc80 2023-10-16T09:48:58.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.846+0000 7f35aeffd640 1 -- 172.21.15.84:0/1902070432 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3598003520 con 0x7f35b006cc80 2023-10-16T09:48:58.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.846+0000 7f35af7fe640 1 --2- 172.21.15.84:0/1902070432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35b006c7d0 0x7f35b0065ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:58.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.846+0000 7f35aeffd640 1 -- 172.21.15.84:0/1902070432 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3654949442 0 0) 0x7f3598003520 con 0x7f35b006cc80 2023-10-16T09:48:58.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.846+0000 7f35aeffd640 1 -- 172.21.15.84:0/1902070432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35b006c7d0 msgr2=0x7f35b0065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:58.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.846+0000 7f35aeffd640 1 --2- 172.21.15.84:0/1902070432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35b006c7d0 0x7f35b0065ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.846+0000 7f35aeffd640 1 -- 172.21.15.84:0/1902070432 >> v2:172.21.15.157:3300/0 conn(0x7f35b015d330 msgr2=0x7f35b01618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:58.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.846+0000 7f35aeffd640 1 --2- 172.21.15.84:0/1902070432 >> v2:172.21.15.157:3300/0 conn(0x7f35b015d330 0x7f35b01618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.846+0000 7f35aeffd640 1 -- 172.21.15.84:0/1902070432 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b0160dd0 con 0x7f35b006cc80 2023-10-16T09:48:58.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.846+0000 7f35aeffd640 1 -- 172.21.15.84:0/1902070432 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f35a80019f0 con 0x7f35b006cc80 2023-10-16T09:48:58.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.846+0000 7f35aeffd640 1 -- 172.21.15.84:0/1902070432 >> v1:172.21.15.84:6789/0 conn(0x7f35b006cc80 legacy=0x7f35b00ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:58.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.847+0000 7f35aeffd640 1 --2- 172.21.15.84:0/1902070432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3598003970 0x7f3598003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:58.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.847+0000 7f35aeffd640 1 --2- 172.21.15.84:0/1902070432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35980047c0 0x7f3598004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:58.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.847+0000 7f35aeffd640 1 --2- 172.21.15.84:0/1902070432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35b006c7d0 0x7f3598005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:58.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.847+0000 7f35aeffd640 1 -- 172.21.15.84:0/1902070432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b006b1e0 con 0x7f3598003970 2023-10-16T09:48:58.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.847+0000 7f35aeffd640 1 -- 172.21.15.84:0/1902070432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b00d05b0 con 0x7f35980047c0 2023-10-16T09:48:58.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.847+0000 7f35aeffd640 1 -- 172.21.15.84:0/1902070432 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359800ba60 con 0x7f35b006c7d0 2023-10-16T09:48:58.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.847+0000 7f35affff640 1 --2- 172.21.15.84:0/1902070432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3598003970 0x7f3598003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:58.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.847+0000 7f35af7fe640 1 --2- 172.21.15.84:0/1902070432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35980047c0 0x7f3598004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:58.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.847+0000 7f35b488f640 1 --2- 172.21.15.84:0/1902070432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35b006c7d0 0x7f3598005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:58.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.847+0000 7f35affff640 1 -- 172.21.15.84:0/1902070432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35b006c7d0 msgr2=0x7f3598005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:58.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.847+0000 7f35affff640 1 --2- 172.21.15.84:0/1902070432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35b006c7d0 0x7f3598005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.847+0000 7f35affff640 1 -- 172.21.15.84:0/1902070432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35980047c0 msgr2=0x7f3598004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:58.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.847+0000 7f35affff640 1 --2- 172.21.15.84:0/1902070432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35980047c0 0x7f3598004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.847+0000 7f35affff640 1 -- 172.21.15.84:0/1902070432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3598003520 con 0x7f3598003970 2023-10-16T09:48:58.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.847+0000 7f35af7fe640 1 --2- 172.21.15.84:0/1902070432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35980047c0 0x7f3598004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:48:58.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.848+0000 7f35affff640 1 --2- 172.21.15.84:0/1902070432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3598003970 0x7f3598003d90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f35a0009a90 tx=0x7f35980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538dfa8d4e4231b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:58.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.849+0000 7f35aeffd640 1 -- 172.21.15.84:0/1902070432 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a002c600 con 0x7f3598003970 2023-10-16T09:48:58.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.849+0000 7f35aeffd640 1 -- 172.21.15.84:0/1902070432 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a002c760 con 0x7f3598003970 2023-10-16T09:48:58.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.849+0000 7f35aeffd640 1 -- 172.21.15.84:0/1902070432 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a002caf0 con 0x7f3598003970 2023-10-16T09:48:58.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.849+0000 7f35b62f2640 1 -- 172.21.15.84:0/1902070432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3598003970 msgr2=0x7f3598003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:58.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.849+0000 7f35b62f2640 1 --2- 172.21.15.84:0/1902070432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3598003970 0x7f3598003d90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f35a0009a90 tx=0x7f35980057f0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.849+0000 7f35b488f640 1 -- 172.21.15.84:0/1902070432 reap_dead start 2023-10-16T09:48:58.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.849+0000 7f35b62f2640 1 -- 172.21.15.84:0/1902070432 shutdown_connections 2023-10-16T09:48:58.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.849+0000 7f35b62f2640 1 -- 172.21.15.84:0/1902070432 >> 172.21.15.84:0/1902070432 conn(0x7f35b00d0ab0 msgr2=0x7f35b00d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:58.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.850+0000 7f35b62f2640 1 -- 172.21.15.84:0/1902070432 shutdown_connections 2023-10-16T09:48:58.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.850+0000 7f35b62f2640 1 -- 172.21.15.84:0/1902070432 wait complete. 2023-10-16T09:48:58.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.851+0000 7f35b62f2640 1 Processor -- start 2023-10-16T09:48:58.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.851+0000 7f35b62f2640 1 -- start start 2023-10-16T09:48:58.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.852+0000 7f35b62f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598003970 0x7f35b01564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:58.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.852+0000 7f35b62f2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35b00b5600 0x7f35b0156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:58.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.852+0000 7f35b62f2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35b01570e0 0x7f35b015d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:58.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.852+0000 7f35b62f2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b0156a30 con 0x7f35b00b5600 2023-10-16T09:48:58.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.852+0000 7f35b62f2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b00b4280 con 0x7f3598003970 2023-10-16T09:48:58.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.852+0000 7f35b62f2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b00b4560 con 0x7f35b01570e0 2023-10-16T09:48:58.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.853+0000 7f35af7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35b00b5600 0x7f35b0156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:58.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.853+0000 7f35af7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35b00b5600 0x7f35b0156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52740/0 (socket says 172.21.15.84:52740) 2023-10-16T09:48:58.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.853+0000 7f35af7fe640 1 -- 172.21.15.84:0/438288814 learned_addr learned my addr 172.21.15.84:0/438288814 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:48:58.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.853+0000 7f35affff640 1 --2- 172.21.15.84:0/438288814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598003970 0x7f35b01564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:58.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.853+0000 7f35b488f640 1 --2- 172.21.15.84:0/438288814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35b01570e0 0x7f35b015d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:58.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.853+0000 7f35af7fe640 1 -- 172.21.15.84:0/438288814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35b01570e0 msgr2=0x7f35b015d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:58.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.853+0000 7f35af7fe640 1 --2- 172.21.15.84:0/438288814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35b01570e0 0x7f35b015d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.853+0000 7f35af7fe640 1 -- 172.21.15.84:0/438288814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598003970 msgr2=0x7f35b01564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:58.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.853+0000 7f35af7fe640 1 --2- 172.21.15.84:0/438288814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598003970 0x7f35b01564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:58.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.853+0000 7f35af7fe640 1 -- 172.21.15.84:0/438288814 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b00b4840 con 0x7f35b00b5600 2023-10-16T09:48:58.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.853+0000 7f35b488f640 1 --2- 172.21.15.84:0/438288814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35b01570e0 0x7f35b015d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:48:58.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.853+0000 7f35af7fe640 1 --2- 172.21.15.84:0/438288814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35b00b5600 0x7f35b0156ba0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f35a4006a90 tx=0x7f35a4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:58.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.854+0000 7f35acff9640 1 -- 172.21.15.84:0/438288814 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4014870 con 0x7f35b00b5600 2023-10-16T09:48:58.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.854+0000 7f35acff9640 1 -- 172.21.15.84:0/438288814 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a40149d0 con 0x7f35b00b5600 2023-10-16T09:48:58.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.854+0000 7f35b62f2640 1 -- 172.21.15.84:0/438288814 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b00b4b20 con 0x7f35b00b5600 2023-10-16T09:48:58.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.854+0000 7f35acff9640 1 -- 172.21.15.84:0/438288814 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4014d60 con 0x7f35b00b5600 2023-10-16T09:48:58.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.854+0000 7f35b62f2640 1 -- 172.21.15.84:0/438288814 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b00b50e0 con 0x7f35b00b5600 2023-10-16T09:48:58.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.855+0000 7f35b62f2640 1 -- 172.21.15.84:0/438288814 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b004e160 con 0x7f35b00b5600 2023-10-16T09:48:58.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.857+0000 7f35acff9640 1 -- 172.21.15.84:0/438288814 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f35a4015410 con 0x7f35b00b5600 2023-10-16T09:48:58.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.858+0000 7f35acff9640 1 --2- 172.21.15.84:0/438288814 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f359c06ae10 0x7f359c06d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:58.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.858+0000 7f35acff9640 1 -- 172.21.15.84:0/438288814 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f35a4089920 con 0x7f35b00b5600 2023-10-16T09:48:58.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.858+0000 7f35affff640 1 --2- 172.21.15.84:0/438288814 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f359c06ae10 0x7f359c06d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:58.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.859+0000 7f35affff640 1 --2- 172.21.15.84:0/438288814 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f359c06ae10 0x7f359c06d2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35a0036000 tx=0x7f35a0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:58.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:58.862+0000 7f35acff9640 1 -- 172.21.15.84:0/438288814 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f35a4054de0 con 0x7f35b00b5600 2023-10-16T09:48:59.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.050+0000 7f35b62f2640 1 -- 172.21.15.84:0/438288814 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35b006c3b0 con 0x7f35b00b5600 2023-10-16T09:48:59.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.050+0000 7f35acff9640 1 -- 172.21.15.84:0/438288814 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35a4058400 con 0x7f35b00b5600 2023-10-16T09:48:59.053 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T09:48:59.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.053+0000 7f35b62f2640 1 -- 172.21.15.84:0/438288814 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f359c06ae10 msgr2=0x7f359c06d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:59.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.053+0000 7f35b62f2640 1 --2- 172.21.15.84:0/438288814 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f359c06ae10 0x7f359c06d2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35a0036000 tx=0x7f35a0030000 comp rx=0 tx=0).stop 2023-10-16T09:48:59.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.053+0000 7f35b62f2640 1 -- 172.21.15.84:0/438288814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35b00b5600 msgr2=0x7f35b0156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:59.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.053+0000 7f35b62f2640 1 --2- 172.21.15.84:0/438288814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35b00b5600 0x7f35b0156ba0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f35a4006a90 tx=0x7f35a4004610 comp rx=0 tx=0).stop 2023-10-16T09:48:59.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.053+0000 7f35b62f2640 1 -- 172.21.15.84:0/438288814 shutdown_connections 2023-10-16T09:48:59.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.053+0000 7f35b62f2640 1 --2- 172.21.15.84:0/438288814 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f359c06ae10 0x7f359c06d2d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:59.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.053+0000 7f35b62f2640 1 --2- 172.21.15.84:0/438288814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35b01570e0 0x7f35b015d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:59.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.053+0000 7f35b62f2640 1 --2- 172.21.15.84:0/438288814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35b00b5600 0x7f35b0156ba0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:59.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.053+0000 7f35b62f2640 1 --2- 172.21.15.84:0/438288814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598003970 0x7f35b01564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:59.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.053+0000 7f35b62f2640 1 -- 172.21.15.84:0/438288814 >> 172.21.15.84:0/438288814 conn(0x7f35b00d0ab0 msgr2=0x7f35b015af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:59.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.053+0000 7f35b62f2640 1 -- 172.21.15.84:0/438288814 shutdown_connections 2023-10-16T09:48:59.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.053+0000 7f35b62f2640 1 -- 172.21.15.84:0/438288814 wait complete. 2023-10-16T09:48:59.071 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T09:48:59.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.323+0000 7fe54bfff640 1 Processor -- start 2023-10-16T09:48:59.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.323+0000 7fe54bfff640 1 -- start start 2023-10-16T09:48:59.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.324+0000 7fe54bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe54c0ad970 0x7fe54c0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:59.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.324+0000 7fe54bfff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe54c15b9d0 con 0x7fe54c0ccfb0 2023-10-16T09:48:59.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.324+0000 7fe54bfff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe54c15bcb0 con 0x7fe54c06c040 2023-10-16T09:48:59.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.324+0000 7fe54bfff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c15bf90 con 0x7fe54c0ad970 2023-10-16T09:48:59.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.325+0000 7fe54b7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe54c0ccfb0 0x7fe54c0cd440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52848/0 (socket says 172.21.15.84:52848) 2023-10-16T09:48:59.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.325+0000 7fe54b7fe640 1 -- 172.21.15.84:0/634540440 learned_addr learned my addr 172.21.15.84:0/634540440 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:48:59.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.325+0000 7fe543fff640 1 --2- 172.21.15.84:0/634540440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe54c0ad970 0x7fe54c0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:59.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.325+0000 7fe54a7fc640 1 -- 172.21.15.84:0/634540440 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243435139 0 0) 0x7fe54c15b9d0 con 0x7fe54c0ccfb0 2023-10-16T09:48:59.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.325+0000 7fe54a7fc640 1 -- 172.21.15.84:0/634540440 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe538003520 con 0x7fe54c0ccfb0 2023-10-16T09:48:59.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.325+0000 7fe54a7fc640 1 -- 172.21.15.84:0/634540440 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3137557790 0 0) 0x7fe538003520 con 0x7fe54c0ccfb0 2023-10-16T09:48:59.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.325+0000 7fe54a7fc640 1 -- 172.21.15.84:0/634540440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe54c0ad970 msgr2=0x7fe54c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:59.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.325+0000 7fe54a7fc640 1 --2- 172.21.15.84:0/634540440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe54c0ad970 0x7fe54c0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:59.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.325+0000 7fe54a7fc640 1 -- 172.21.15.84:0/634540440 >> v1:172.21.15.157:6789/0 conn(0x7fe54c06c040 legacy=0x7fe54c06c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:59.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.325+0000 7fe54a7fc640 1 -- 172.21.15.84:0/634540440 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c0cc270 con 0x7fe54c0ccfb0 2023-10-16T09:48:59.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.325+0000 7fe54a7fc640 1 -- 172.21.15.84:0/634540440 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fe5440019f0 con 0x7fe54c0ccfb0 2023-10-16T09:48:59.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.326+0000 7fe54a7fc640 1 -- 172.21.15.84:0/634540440 >> v1:172.21.15.84:6789/0 conn(0x7fe54c0ccfb0 legacy=0x7fe54c0cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:59.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.326+0000 7fe54a7fc640 1 --2- 172.21.15.84:0/634540440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5380039a0 0x7fe538003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:59.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.326+0000 7fe54a7fc640 1 --2- 172.21.15.84:0/634540440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe54c0ad970 0x7fe5380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:59.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.326+0000 7fe54a7fc640 1 --2- 172.21.15.84:0/634540440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe538004d00 0x7fe538005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:59.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.326+0000 7fe54a7fc640 1 -- 172.21.15.84:0/634540440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c15bf90 con 0x7fe5380039a0 2023-10-16T09:48:59.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.326+0000 7fe54a7fc640 1 -- 172.21.15.84:0/634540440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53800ba60 con 0x7fe538004d00 2023-10-16T09:48:59.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.326+0000 7fe54a7fc640 1 -- 172.21.15.84:0/634540440 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53800bc20 con 0x7fe54c0ad970 2023-10-16T09:48:59.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.326+0000 7fe54affd640 1 --2- 172.21.15.84:0/634540440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5380039a0 0x7fe538003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:59.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.326+0000 7fe543fff640 1 --2- 172.21.15.84:0/634540440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe54c0ad970 0x7fe5380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:59.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.326+0000 7fe54b7fe640 1 --2- 172.21.15.84:0/634540440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe538004d00 0x7fe538005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:59.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.326+0000 7fe54affd640 1 -- 172.21.15.84:0/634540440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe54c0ad970 msgr2=0x7fe5380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:59.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.326+0000 7fe54affd640 1 --2- 172.21.15.84:0/634540440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe54c0ad970 0x7fe5380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:59.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.326+0000 7fe54affd640 1 -- 172.21.15.84:0/634540440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe538004d00 msgr2=0x7fe538005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:48:59.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.326+0000 7fe54affd640 1 --2- 172.21.15.84:0/634540440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe538004d00 0x7fe538005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:59.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.326+0000 7fe54affd640 1 -- 172.21.15.84:0/634540440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe538003520 con 0x7fe5380039a0 2023-10-16T09:48:59.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.326+0000 7fe54b7fe640 1 --2- 172.21.15.84:0/634540440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe538004d00 0x7fe538005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:48:59.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.328+0000 7fe54affd640 1 --2- 172.21.15.84:0/634540440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5380039a0 0x7fe538003dc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe53c001750 tx=0x7fe5380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8def0d185b6052 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:59.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.328+0000 7fe54a7fc640 1 -- 172.21.15.84:0/634540440 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53c0052d0 con 0x7fe5380039a0 2023-10-16T09:48:59.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.328+0000 7fe54a7fc640 1 -- 172.21.15.84:0/634540440 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53c02e050 con 0x7fe5380039a0 2023-10-16T09:48:59.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.328+0000 7fe54a7fc640 1 -- 172.21.15.84:0/634540440 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53c0035f0 con 0x7fe5380039a0 2023-10-16T09:48:59.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.328+0000 7fe54bfff640 1 -- 172.21.15.84:0/634540440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5380039a0 msgr2=0x7fe538003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:59.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.328+0000 7fe54bfff640 1 --2- 172.21.15.84:0/634540440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5380039a0 0x7fe538003dc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe53c001750 tx=0x7fe5380057f0 comp rx=0 tx=0).stop 2023-10-16T09:48:59.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.328+0000 7fe54b7fe640 1 -- 172.21.15.84:0/634540440 reap_dead start 2023-10-16T09:48:59.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.328+0000 7fe54bfff640 1 -- 172.21.15.84:0/634540440 shutdown_connections 2023-10-16T09:48:59.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.328+0000 7fe54bfff640 1 -- 172.21.15.84:0/634540440 >> 172.21.15.84:0/634540440 conn(0x7fe54c0ce6a0 msgr2=0x7fe54c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:59.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.328+0000 7fe54bfff640 1 -- 172.21.15.84:0/634540440 shutdown_connections 2023-10-16T09:48:59.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.328+0000 7fe54bfff640 1 -- 172.21.15.84:0/634540440 wait complete. 2023-10-16T09:48:59.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.330+0000 7fe54bfff640 1 Processor -- start 2023-10-16T09:48:59.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.330+0000 7fe54bfff640 1 -- start start 2023-10-16T09:48:59.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.331+0000 7fe54bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5380039a0 0x7fe54c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:59.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.331+0000 7fe54bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe54c1f5de0 0x7fe54c152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:59.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.331+0000 7fe54bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe54c0c7790 0x7fe54c0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:59.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.331+0000 7fe54bfff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c153480 con 0x7fe54c0c7790 2023-10-16T09:48:59.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.331+0000 7fe54bfff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c1f6270 con 0x7fe5380039a0 2023-10-16T09:48:59.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.331+0000 7fe54bfff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c1f6550 con 0x7fe54c1f5de0 2023-10-16T09:48:59.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.332+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe54c0c7790 0x7fe54c0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:59.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.332+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe54c0c7790 0x7fe54c0ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52766/0 (socket says 172.21.15.84:52766) 2023-10-16T09:48:59.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.332+0000 7fe54b7fe640 1 -- 172.21.15.84:0/3836181039 learned_addr learned my addr 172.21.15.84:0/3836181039 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:48:59.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.332+0000 7fe54b7fe640 1 -- 172.21.15.84:0/3836181039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe54c1f5de0 msgr2=0x7fe54c152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:59.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.332+0000 7fe54b7fe640 1 --2- 172.21.15.84:0/3836181039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe54c1f5de0 0x7fe54c152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:59.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.332+0000 7fe54affd640 1 --2- 172.21.15.84:0/3836181039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5380039a0 0x7fe54c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:59.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.332+0000 7fe54b7fe640 1 -- 172.21.15.84:0/3836181039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5380039a0 msgr2=0x7fe54c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:59.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.332+0000 7fe54b7fe640 1 --2- 172.21.15.84:0/3836181039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5380039a0 0x7fe54c0ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:59.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.332+0000 7fe54b7fe640 1 -- 172.21.15.84:0/3836181039 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c0cdcd0 con 0x7fe54c0c7790 2023-10-16T09:48:59.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.332+0000 7fe54b7fe640 1 --2- 172.21.15.84:0/3836181039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe54c0c7790 0x7fe54c0ac890 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0b1290 tx=0x7fe54400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:59.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.332+0000 7fe5437fe640 1 -- 172.21.15.84:0/3836181039 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe544004d60 con 0x7fe54c0c7790 2023-10-16T09:48:59.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.332+0000 7fe5437fe640 1 -- 172.21.15.84:0/3836181039 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54400bb80 con 0x7fe54c0c7790 2023-10-16T09:48:59.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.332+0000 7fe54bfff640 1 -- 172.21.15.84:0/3836181039 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54c1f6830 con 0x7fe54c0c7790 2023-10-16T09:48:59.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.333+0000 7fe5437fe640 1 -- 172.21.15.84:0/3836181039 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe544007110 con 0x7fe54c0c7790 2023-10-16T09:48:59.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.333+0000 7fe54bfff640 1 -- 172.21.15.84:0/3836181039 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54c0c7c40 con 0x7fe54c0c7790 2023-10-16T09:48:59.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.334+0000 7fe54bfff640 1 -- 172.21.15.84:0/3836181039 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54c06b330 con 0x7fe54c0c7790 2023-10-16T09:48:59.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.335+0000 7fe5437fe640 1 -- 172.21.15.84:0/3836181039 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7fe544007270 con 0x7fe54c0c7790 2023-10-16T09:48:59.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.336+0000 7fe5437fe640 1 --2- 172.21.15.84:0/3836181039 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe52c06ade0 0x7fe52c06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:59.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.336+0000 7fe5437fe640 1 -- 172.21.15.84:0/3836181039 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe544089360 con 0x7fe54c0c7790 2023-10-16T09:48:59.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.336+0000 7fe54affd640 1 --2- 172.21.15.84:0/3836181039 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe52c06ade0 0x7fe52c06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:59.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.337+0000 7fe54affd640 1 --2- 172.21.15.84:0/3836181039 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe52c06ade0 0x7fe52c06d2a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe53c033000 tx=0x7fe53c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:59.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.340+0000 7fe5437fe640 1 -- 172.21.15.84:0/3836181039 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe5440548d0 con 0x7fe54c0c7790 2023-10-16T09:48:59.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.537+0000 7fe54bfff640 1 -- 172.21.15.84:0/3836181039 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe54c04e9b0 con 0x7fe54c0c7790 2023-10-16T09:48:59.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.544+0000 7fe5437fe640 1 -- 172.21.15.84:0/3836181039 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe544057ef0 con 0x7fe54c0c7790 2023-10-16T09:48:59.545 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:48:59.546 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T09:48:59.546 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:48:59.546 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-16T09:48:59.546 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-16T09:48:59.547 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:48:59.547 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T09:48:59.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.546+0000 7fe54bfff640 1 -- 172.21.15.84:0/3836181039 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe52c06ade0 msgr2=0x7fe52c06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:59.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.546+0000 7fe54bfff640 1 --2- 172.21.15.84:0/3836181039 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe52c06ade0 0x7fe52c06d2a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe53c033000 tx=0x7fe53c009030 comp rx=0 tx=0).stop 2023-10-16T09:48:59.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.546+0000 7fe54bfff640 1 -- 172.21.15.84:0/3836181039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe54c0c7790 msgr2=0x7fe54c0ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:59.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.546+0000 7fe54bfff640 1 --2- 172.21.15.84:0/3836181039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe54c0c7790 0x7fe54c0ac890 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0b1290 tx=0x7fe54400b4a0 comp rx=0 tx=0).stop 2023-10-16T09:48:59.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.546+0000 7fe54bfff640 1 -- 172.21.15.84:0/3836181039 shutdown_connections 2023-10-16T09:48:59.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.546+0000 7fe54bfff640 1 --2- 172.21.15.84:0/3836181039 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe52c06ade0 0x7fe52c06d2a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:59.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.546+0000 7fe54bfff640 1 --2- 172.21.15.84:0/3836181039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe54c0c7790 0x7fe54c0ac890 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:59.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.546+0000 7fe54bfff640 1 --2- 172.21.15.84:0/3836181039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe54c1f5de0 0x7fe54c152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:59.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.546+0000 7fe54bfff640 1 --2- 172.21.15.84:0/3836181039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5380039a0 0x7fe54c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:59.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.546+0000 7fe54bfff640 1 -- 172.21.15.84:0/3836181039 >> 172.21.15.84:0/3836181039 conn(0x7fe54c0ce6a0 msgr2=0x7fe54c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:59.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.546+0000 7fe54bfff640 1 -- 172.21.15.84:0/3836181039 shutdown_connections 2023-10-16T09:48:59.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.546+0000 7fe54bfff640 1 -- 172.21.15.84:0/3836181039 wait complete. 2023-10-16T09:48:59.566 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T09:48:59.567 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T09:48:59.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.783+0000 7fbec1d5a640 1 Processor -- start 2023-10-16T09:48:59.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.783+0000 7fbec1d5a640 1 -- start start 2023-10-16T09:48:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.784+0000 7fbec1d5a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbebc0cac20 0x7fbebc0b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.784+0000 7fbec1d5a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbebc15b310 con 0x7fbebc0b0360 2023-10-16T09:48:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.784+0000 7fbec1d5a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbebc06f640 con 0x7fbebc15ac10 2023-10-16T09:48:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.784+0000 7fbec1d5a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbebc15f800 con 0x7fbebc0cac20 2023-10-16T09:48:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.785+0000 7fbebbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbebc0cac20 0x7fbebc0b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.785+0000 7fbebbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbebc0cac20 0x7fbebc0b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52782/0 (socket says 172.21.15.84:52782) 2023-10-16T09:48:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.785+0000 7fbeb3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbebc0b0360 0x7fbebc0ca510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52862/0 (socket says 172.21.15.84:52862) 2023-10-16T09:48:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.785+0000 7fbebbfff640 1 -- 172.21.15.84:0/2705406426 learned_addr learned my addr 172.21.15.84:0/2705406426 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:48:59.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.785+0000 7fbebbfff640 1 -- 172.21.15.84:0/2705406426 >> v1:172.21.15.157:6789/0 conn(0x7fbebc15ac10 legacy=0x7fbebc06c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:59.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.785+0000 7fbebbfff640 1 -- 172.21.15.84:0/2705406426 >> v1:172.21.15.84:6789/0 conn(0x7fbebc0b0360 legacy=0x7fbebc0ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:59.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.785+0000 7fbebbfff640 1 -- 172.21.15.84:0/2705406426 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc15b5b0 con 0x7fbebc0cac20 2023-10-16T09:48:59.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.785+0000 7fbebaffd640 1 -- 172.21.15.84:0/2705406426 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588446268 0 0) 0x7fbebc15b310 con 0x7fbebc0b0360 2023-10-16T09:48:59.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.786+0000 7fbebbfff640 1 --2- 172.21.15.84:0/2705406426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbebc0cac20 0x7fbebc0b07f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009e50 tx=0x7fbeb402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09ec7b5ce1a9ef8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:59.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.787+0000 7fbebaffd640 1 -- 172.21.15.84:0/2705406426 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb402cb20 con 0x7fbebc0cac20 2023-10-16T09:48:59.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.787+0000 7fbebaffd640 1 -- 172.21.15.84:0/2705406426 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb402cc80 con 0x7fbebc0cac20 2023-10-16T09:48:59.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.787+0000 7fbebaffd640 1 -- 172.21.15.84:0/2705406426 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb40343f0 con 0x7fbebc0cac20 2023-10-16T09:48:59.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.787+0000 7fbec1d5a640 1 -- 172.21.15.84:0/2705406426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbebc0cac20 msgr2=0x7fbebc0b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:59.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.787+0000 7fbec1d5a640 1 --2- 172.21.15.84:0/2705406426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbebc0cac20 0x7fbebc0b07f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009e50 tx=0x7fbeb402bf80 comp rx=0 tx=0).stop 2023-10-16T09:48:59.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.787+0000 7fbec1d5a640 1 -- 172.21.15.84:0/2705406426 shutdown_connections 2023-10-16T09:48:59.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.787+0000 7fbec1d5a640 1 --2- 172.21.15.84:0/2705406426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbebc0cac20 0x7fbebc0b07f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:59.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.787+0000 7fbec1d5a640 1 -- 172.21.15.84:0/2705406426 >> 172.21.15.84:0/2705406426 conn(0x7fbebc0cb360 msgr2=0x7fbebc0cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:48:59.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.787+0000 7fbec1d5a640 1 -- 172.21.15.84:0/2705406426 shutdown_connections 2023-10-16T09:48:59.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.788+0000 7fbec1d5a640 1 -- 172.21.15.84:0/2705406426 wait complete. 2023-10-16T09:48:59.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.789+0000 7fbec1d5a640 1 Processor -- start 2023-10-16T09:48:59.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.789+0000 7fbec1d5a640 1 -- start start 2023-10-16T09:48:59.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.790+0000 7fbec1d5a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc0b0360 0x7fbebc14bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:59.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.790+0000 7fbec1d5a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbebc0cac20 0x7fbebc15e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:59.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.790+0000 7fbec1d5a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbebc15ac10 0x7fbebc15eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:59.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.790+0000 7fbec1d5a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc0ce570 con 0x7fbebc15ac10 2023-10-16T09:48:59.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.790+0000 7fbec1d5a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc14c2e0 con 0x7fbebc0b0360 2023-10-16T09:48:59.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.790+0000 7fbec1d5a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc0cdcd0 con 0x7fbebc0cac20 2023-10-16T09:48:59.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.790+0000 7fbebbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbebc15ac10 0x7fbebc15eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:59.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.791+0000 7fbebbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbebc15ac10 0x7fbebc15eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52786/0 (socket says 172.21.15.84:52786) 2023-10-16T09:48:59.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.791+0000 7fbebbfff640 1 -- 172.21.15.84:0/351805140 learned_addr learned my addr 172.21.15.84:0/351805140 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:48:59.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.791+0000 7fbeb3fff640 1 --2- 172.21.15.84:0/351805140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbebc0cac20 0x7fbebc15e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:59.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.791+0000 7fbebbfff640 1 -- 172.21.15.84:0/351805140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbebc0cac20 msgr2=0x7fbebc15e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:59.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.791+0000 7fbebbfff640 1 --2- 172.21.15.84:0/351805140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbebc0cac20 0x7fbebc15e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:59.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.791+0000 7fbebbfff640 1 -- 172.21.15.84:0/351805140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc0b0360 msgr2=0x7fbebc14bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:48:59.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.791+0000 7fbebb7fe640 1 --2- 172.21.15.84:0/351805140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc0b0360 0x7fbebc14bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:59.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.791+0000 7fbebbfff640 1 --2- 172.21.15.84:0/351805140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc0b0360 0x7fbebc14bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:48:59.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.791+0000 7fbebbfff640 1 -- 172.21.15.84:0/351805140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb4009b30 con 0x7fbebc15ac10 2023-10-16T09:48:59.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.791+0000 7fbebb7fe640 1 --2- 172.21.15.84:0/351805140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc0b0360 0x7fbebc14bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:48:59.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.791+0000 7fbebbfff640 1 --2- 172.21.15.84:0/351805140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbebc15ac10 0x7fbebc15eb50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4036040 tx=0x7fbeb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:59.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.792+0000 7fbeb8ff9640 1 -- 172.21.15.84:0/351805140 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4009510 con 0x7fbebc15ac10 2023-10-16T09:48:59.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.792+0000 7fbeb8ff9640 1 -- 172.21.15.84:0/351805140 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb4007e50 con 0x7fbebc15ac10 2023-10-16T09:48:59.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.792+0000 7fbec1d5a640 1 -- 172.21.15.84:0/351805140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbebc0cdfb0 con 0x7fbebc15ac10 2023-10-16T09:48:59.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.792+0000 7fbeb8ff9640 1 -- 172.21.15.84:0/351805140 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb403e690 con 0x7fbebc15ac10 2023-10-16T09:48:59.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.792+0000 7fbec1d5a640 1 -- 172.21.15.84:0/351805140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbebc0b16a0 con 0x7fbebc15ac10 2023-10-16T09:48:59.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.793+0000 7fbec1d5a640 1 -- 172.21.15.84:0/351805140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbebc0b1500 con 0x7fbebc15ac10 2023-10-16T09:48:59.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.794+0000 7fbeb8ff9640 1 -- 172.21.15.84:0/351805140 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7fbeb403e7f0 con 0x7fbebc15ac10 2023-10-16T09:48:59.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.795+0000 7fbeb8ff9640 1 --2- 172.21.15.84:0/351805140 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbea006ae10 0x7fbea006d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:48:59.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.795+0000 7fbeb8ff9640 1 -- 172.21.15.84:0/351805140 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbeb40abc00 con 0x7fbebc15ac10 2023-10-16T09:48:59.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.795+0000 7fbebb7fe640 1 --2- 172.21.15.84:0/351805140 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbea006ae10 0x7fbea006d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:48:59.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.796+0000 7fbebb7fe640 1 --2- 172.21.15.84:0/351805140 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbea006ae10 0x7fbea006d2d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbeac000f60 tx=0x7fbeac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:48:59.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.800+0000 7fbeb8ff9640 1 -- 172.21.15.84:0/351805140 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbebc0b1500 con 0x7fbebc15ac10 2023-10-16T09:48:59.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:48:59.994+0000 7fbec1d5a640 1 -- 172.21.15.84:0/351805140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbebc0b1500 con 0x7fbebc15ac10 2023-10-16T09:49:00.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:00.868+0000 7fbeb8ff9640 1 -- 172.21.15.84:0/351805140 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbebc0b1500 con 0x7fbebc15ac10 2023-10-16T09:49:00.870 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-16T09:49:00.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:00.872+0000 7fbec1d5a640 1 -- 172.21.15.84:0/351805140 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbea006ae10 msgr2=0x7fbea006d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:00.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:00.872+0000 7fbec1d5a640 1 --2- 172.21.15.84:0/351805140 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbea006ae10 0x7fbea006d2d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbeac000f60 tx=0x7fbeac009040 comp rx=0 tx=0).stop 2023-10-16T09:49:00.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:00.872+0000 7fbec1d5a640 1 -- 172.21.15.84:0/351805140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbebc15ac10 msgr2=0x7fbebc15eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:00.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:00.872+0000 7fbec1d5a640 1 --2- 172.21.15.84:0/351805140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbebc15ac10 0x7fbebc15eb50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4036040 tx=0x7fbeb4004a40 comp rx=0 tx=0).stop 2023-10-16T09:49:00.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:00.874+0000 7fbec1d5a640 1 -- 172.21.15.84:0/351805140 shutdown_connections 2023-10-16T09:49:00.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:00.874+0000 7fbec1d5a640 1 --2- 172.21.15.84:0/351805140 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbea006ae10 0x7fbea006d2d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:00.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:00.874+0000 7fbec1d5a640 1 --2- 172.21.15.84:0/351805140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbebc15ac10 0x7fbebc15eb50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:00.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:00.874+0000 7fbec1d5a640 1 --2- 172.21.15.84:0/351805140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbebc0cac20 0x7fbebc15e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:00.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:00.874+0000 7fbec1d5a640 1 --2- 172.21.15.84:0/351805140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc0b0360 0x7fbebc14bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:00.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:00.874+0000 7fbec1d5a640 1 -- 172.21.15.84:0/351805140 >> 172.21.15.84:0/351805140 conn(0x7fbebc0cb360 msgr2=0x7fbebc0b07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:00.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:00.874+0000 7fbec1d5a640 1 -- 172.21.15.84:0/351805140 shutdown_connections 2023-10-16T09:49:00.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:00.874+0000 7fbec1d5a640 1 -- 172.21.15.84:0/351805140 wait complete. 2023-10-16T09:49:00.906 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T09:49:01.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.100+0000 7fc277533640 1 Processor -- start 2023-10-16T09:49:01.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.100+0000 7fc277533640 1 -- start start 2023-10-16T09:49:01.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.102+0000 7fc277533640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc27006ecd0 0x7fc270070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:01.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.102+0000 7fc277533640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc270071360 0x7fc2700cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:01.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.102+0000 7fc277533640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc270070660 con 0x7fc27015a970 2023-10-16T09:49:01.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.102+0000 7fc277533640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27015dc60 con 0x7fc270071360 2023-10-16T09:49:01.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.102+0000 7fc277533640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27015df40 con 0x7fc27006ecd0 2023-10-16T09:49:01.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.102+0000 7fc2752cf640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc27015a970 0x7fc27015d430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52876/0 (socket says 172.21.15.84:52876) 2023-10-16T09:49:01.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.102+0000 7fc2752cf640 1 -- 172.21.15.84:0/2003489368 learned_addr learned my addr 172.21.15.84:0/2003489368 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:01.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.102+0000 7fc274ace640 1 --2- 172.21.15.84:0/2003489368 >> v2:172.21.15.157:3300/0 conn(0x7fc27006ecd0 0x7fc270070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:01.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.102+0000 7fc275ad0640 1 --2- 172.21.15.84:0/2003489368 >> v2:172.21.15.84:3300/0 conn(0x7fc270071360 0x7fc2700cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:01.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.103+0000 7fc26ffff640 1 -- 172.21.15.84:0/2003489368 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1100540679 0 0) 0x7fc270070660 con 0x7fc27015a970 2023-10-16T09:49:01.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.103+0000 7fc26ffff640 1 -- 172.21.15.84:0/2003489368 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc258003520 con 0x7fc27015a970 2023-10-16T09:49:01.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.103+0000 7fc275ad0640 1 -- 172.21.15.84:0/2003489368 >> v2:172.21.15.157:3300/0 conn(0x7fc27006ecd0 msgr2=0x7fc270070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:01.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.103+0000 7fc275ad0640 1 --2- 172.21.15.84:0/2003489368 >> v2:172.21.15.157:3300/0 conn(0x7fc27006ecd0 0x7fc270070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:01.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.103+0000 7fc275ad0640 1 -- 172.21.15.84:0/2003489368 >> v1:172.21.15.84:6789/0 conn(0x7fc27015a970 legacy=0x7fc27015d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:01.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.103+0000 7fc26ffff640 1 -- 172.21.15.84:0/2003489368 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2069919529 0 0) 0x7fc258003520 con 0x7fc27015a970 2023-10-16T09:49:01.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.103+0000 7fc275ad0640 1 -- 172.21.15.84:0/2003489368 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27015e220 con 0x7fc270071360 2023-10-16T09:49:01.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.104+0000 7fc275ad0640 1 --2- 172.21.15.84:0/2003489368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc270071360 0x7fc2700cd880 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc270071800 tx=0x7fc26802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7cfecdf08669ae server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:01.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.104+0000 7fc26ffff640 1 -- 172.21.15.84:0/2003489368 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2600019f0 con 0x7fc270071360 2023-10-16T09:49:01.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.104+0000 7fc26ffff640 1 -- 172.21.15.84:0/2003489368 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc268033440 con 0x7fc270071360 2023-10-16T09:49:01.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.105+0000 7fc26ffff640 1 -- 172.21.15.84:0/2003489368 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2680337d0 con 0x7fc270071360 2023-10-16T09:49:01.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.105+0000 7fc277533640 1 -- 172.21.15.84:0/2003489368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc270071360 msgr2=0x7fc2700cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:01.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.105+0000 7fc277533640 1 --2- 172.21.15.84:0/2003489368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc270071360 0x7fc2700cd880 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc270071800 tx=0x7fc26802b7a0 comp rx=0 tx=0).stop 2023-10-16T09:49:01.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.105+0000 7fc277533640 1 -- 172.21.15.84:0/2003489368 shutdown_connections 2023-10-16T09:49:01.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.105+0000 7fc277533640 1 --2- 172.21.15.84:0/2003489368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc270071360 0x7fc2700cd880 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:01.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.105+0000 7fc277533640 1 --2- 172.21.15.84:0/2003489368 >> v2:172.21.15.157:3300/0 conn(0x7fc27006ecd0 0x7fc270070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:01.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.105+0000 7fc277533640 1 -- 172.21.15.84:0/2003489368 >> 172.21.15.84:0/2003489368 conn(0x7fc2700cfaa0 msgr2=0x7fc2700cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:01.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.105+0000 7fc277533640 1 -- 172.21.15.84:0/2003489368 shutdown_connections 2023-10-16T09:49:01.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.105+0000 7fc277533640 1 -- 172.21.15.84:0/2003489368 wait complete. 2023-10-16T09:49:01.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.107+0000 7fc277533640 1 Processor -- start 2023-10-16T09:49:01.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.107+0000 7fc277533640 1 -- start start 2023-10-16T09:49:01.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.108+0000 7fc277533640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc27006ecd0 0x7fc270151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:01.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.108+0000 7fc277533640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc27015a970 0x7fc2700ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:01.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.108+0000 7fc277533640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2700ad7e0 0x7fc2700c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:01.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.108+0000 7fc277533640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2700ae240 con 0x7fc27015a970 2023-10-16T09:49:01.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.108+0000 7fc277533640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2700c35a0 con 0x7fc2700ad7e0 2023-10-16T09:49:01.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.108+0000 7fc277533640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2701522d0 con 0x7fc27006ecd0 2023-10-16T09:49:01.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.108+0000 7fc274ace640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc27015a970 0x7fc2700ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:01.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.109+0000 7fc274ace640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc27015a970 0x7fc2700ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52804/0 (socket says 172.21.15.84:52804) 2023-10-16T09:49:01.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.109+0000 7fc274ace640 1 -- 172.21.15.84:0/1955826855 learned_addr learned my addr 172.21.15.84:0/1955826855 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:01.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.109+0000 7fc274ace640 1 -- 172.21.15.84:0/1955826855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc27006ecd0 msgr2=0x7fc270151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:01.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.109+0000 7fc274ace640 1 --2- 172.21.15.84:0/1955826855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc27006ecd0 0x7fc270151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:01.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.109+0000 7fc274ace640 1 -- 172.21.15.84:0/1955826855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2700ad7e0 msgr2=0x7fc2700c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:01.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.109+0000 7fc274ace640 1 --2- 172.21.15.84:0/1955826855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2700ad7e0 0x7fc2700c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:01.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.109+0000 7fc274ace640 1 -- 172.21.15.84:0/1955826855 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc268009400 con 0x7fc27015a970 2023-10-16T09:49:01.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.109+0000 7fc274ace640 1 --2- 172.21.15.84:0/1955826855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc27015a970 0x7fc2700ad2a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc26400e9b0 tx=0x7fc26400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:01.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.109+0000 7fc26dffb640 1 -- 172.21.15.84:0/1955826855 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264009670 con 0x7fc27015a970 2023-10-16T09:49:01.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.109+0000 7fc26dffb640 1 -- 172.21.15.84:0/1955826855 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26400c9a0 con 0x7fc27015a970 2023-10-16T09:49:01.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.109+0000 7fc26dffb640 1 -- 172.21.15.84:0/1955826855 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26400cd50 con 0x7fc27015a970 2023-10-16T09:49:01.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.109+0000 7fc277533640 1 -- 172.21.15.84:0/1955826855 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2700adc80 con 0x7fc27015a970 2023-10-16T09:49:01.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.110+0000 7fc277533640 1 -- 172.21.15.84:0/1955826855 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27006cc30 con 0x7fc27015a970 2023-10-16T09:49:01.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.110+0000 7fc277533640 1 -- 172.21.15.84:0/1955826855 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27006ca90 con 0x7fc27015a970 2023-10-16T09:49:01.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.111+0000 7fc26dffb640 1 -- 172.21.15.84:0/1955826855 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc264013470 con 0x7fc27015a970 2023-10-16T09:49:01.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.113+0000 7fc26dffb640 1 --2- 172.21.15.84:0/1955826855 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc25c06ae70 0x7fc25c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:01.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.113+0000 7fc26dffb640 1 -- 172.21.15.84:0/1955826855 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fc264089780 con 0x7fc27015a970 2023-10-16T09:49:01.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.113+0000 7fc2752cf640 1 --2- 172.21.15.84:0/1955826855 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc25c06ae70 0x7fc25c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:01.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.114+0000 7fc2752cf640 1 --2- 172.21.15.84:0/1955826855 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc25c06ae70 0x7fc25c06d330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc2701f5a30 tx=0x7fc260001420 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:01.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.118+0000 7fc26dffb640 1 -- 172.21.15.84:0/1955826855 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc27006ca90 con 0x7fc27015a970 2023-10-16T09:49:01.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.307+0000 7fc277533640 1 -- 172.21.15.84:0/1955826855 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc27006ca90 con 0x7fc27015a970 2023-10-16T09:49:01.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.885+0000 7fc26dffb640 1 -- 172.21.15.84:0/1955826855 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc27006ca90 con 0x7fc27015a970 2023-10-16T09:49:01.887 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-16T09:49:01.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.888+0000 7fc277533640 1 -- 172.21.15.84:0/1955826855 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc25c06ae70 msgr2=0x7fc25c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:01.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.888+0000 7fc277533640 1 --2- 172.21.15.84:0/1955826855 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc25c06ae70 0x7fc25c06d330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc2701f5a30 tx=0x7fc260001420 comp rx=0 tx=0).stop 2023-10-16T09:49:01.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.888+0000 7fc277533640 1 -- 172.21.15.84:0/1955826855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc27015a970 msgr2=0x7fc2700ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:01.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.888+0000 7fc277533640 1 --2- 172.21.15.84:0/1955826855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc27015a970 0x7fc2700ad2a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc26400e9b0 tx=0x7fc26400ee70 comp rx=0 tx=0).stop 2023-10-16T09:49:01.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.889+0000 7fc277533640 1 -- 172.21.15.84:0/1955826855 shutdown_connections 2023-10-16T09:49:01.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.889+0000 7fc277533640 1 --2- 172.21.15.84:0/1955826855 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc25c06ae70 0x7fc25c06d330 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:01.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.889+0000 7fc277533640 1 --2- 172.21.15.84:0/1955826855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2700ad7e0 0x7fc2700c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:01.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.889+0000 7fc277533640 1 --2- 172.21.15.84:0/1955826855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc27015a970 0x7fc2700ad2a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:01.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.889+0000 7fc277533640 1 --2- 172.21.15.84:0/1955826855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc27006ecd0 0x7fc270151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:01.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.889+0000 7fc277533640 1 -- 172.21.15.84:0/1955826855 >> 172.21.15.84:0/1955826855 conn(0x7fc2700cfaa0 msgr2=0x7fc2700d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:01.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.889+0000 7fc277533640 1 -- 172.21.15.84:0/1955826855 shutdown_connections 2023-10-16T09:49:01.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:01.889+0000 7fc277533640 1 -- 172.21.15.84:0/1955826855 wait complete. 2023-10-16T09:49:01.923 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T09:49:02.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.109+0000 7fb5f559e640 1 Processor -- start 2023-10-16T09:49:02.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.110+0000 7fb5f559e640 1 -- start start 2023-10-16T09:49:02.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.111+0000 7fb5f559e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb5f006c040 0x7fb5f006c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:02.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.111+0000 7fb5f559e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f00cda80 0x7fb5f015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:02.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.111+0000 7fb5f559e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f006c960 con 0x7fb5f00ad970 2023-10-16T09:49:02.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.111+0000 7fb5f559e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f015b9d0 con 0x7fb5f006c040 2023-10-16T09:49:02.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.111+0000 7fb5f559e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f015bcb0 con 0x7fb5f00cda80 2023-10-16T09:49:02.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.112+0000 7fb5f4d9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f00cda80 0x7fb5f015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:02.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.112+0000 7fb5f4d9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f00cda80 0x7fb5f015ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:33130/0 (socket says 172.21.15.84:33130) 2023-10-16T09:49:02.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.112+0000 7fb5f4d9d640 1 -- 172.21.15.84:0/1798387358 learned_addr learned my addr 172.21.15.84:0/1798387358 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:02.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.112+0000 7fb5effff640 1 --2- 172.21.15.84:0/1798387358 >> v2:172.21.15.157:3300/0 conn(0x7fb5f006c040 0x7fb5f006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:02.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.113+0000 7fb5f4d9d640 1 -- 172.21.15.84:0/1798387358 >> v2:172.21.15.157:3300/0 conn(0x7fb5f006c040 msgr2=0x7fb5f006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:02.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.113+0000 7fb5f4d9d640 1 --2- 172.21.15.84:0/1798387358 >> v2:172.21.15.157:3300/0 conn(0x7fb5f006c040 0x7fb5f006c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:02.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.113+0000 7fb5f4d9d640 1 -- 172.21.15.84:0/1798387358 >> v1:172.21.15.157:6789/0 conn(0x7fb5f00ad970 legacy=0x7fb5f015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:02.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.113+0000 7fb5f4d9d640 1 -- 172.21.15.84:0/1798387358 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f015bf90 con 0x7fb5f00cda80 2023-10-16T09:49:02.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.114+0000 7fb5f4d9d640 1 --2- 172.21.15.84:0/1798387358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f00cda80 0x7fb5f015ed90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009e50 tx=0x7fb5e802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b756b41b5d3cebb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:02.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.114+0000 7fb5ef7fe640 1 -- 172.21.15.84:0/1798387358 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e802cb00 con 0x7fb5f00cda80 2023-10-16T09:49:02.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.114+0000 7fb5ef7fe640 1 -- 172.21.15.84:0/1798387358 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e802cc60 con 0x7fb5f00cda80 2023-10-16T09:49:02.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.114+0000 7fb5ef7fe640 1 -- 172.21.15.84:0/1798387358 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80343f0 con 0x7fb5f00cda80 2023-10-16T09:49:02.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.115+0000 7fb5f559e640 1 -- 172.21.15.84:0/1798387358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f00cda80 msgr2=0x7fb5f015ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:02.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.115+0000 7fb5f559e640 1 --2- 172.21.15.84:0/1798387358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f00cda80 0x7fb5f015ed90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009e50 tx=0x7fb5e802bf60 comp rx=0 tx=0).stop 2023-10-16T09:49:02.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.115+0000 7fb5f559e640 1 -- 172.21.15.84:0/1798387358 shutdown_connections 2023-10-16T09:49:02.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.115+0000 7fb5f559e640 1 --2- 172.21.15.84:0/1798387358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f00cda80 0x7fb5f015ed90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:02.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.115+0000 7fb5f559e640 1 --2- 172.21.15.84:0/1798387358 >> v2:172.21.15.157:3300/0 conn(0x7fb5f006c040 0x7fb5f006c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:02.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.115+0000 7fb5f559e640 1 -- 172.21.15.84:0/1798387358 >> 172.21.15.84:0/1798387358 conn(0x7fb5f00ce6a0 msgr2=0x7fb5f00ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:02.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.115+0000 7fb5f559e640 1 -- 172.21.15.84:0/1798387358 shutdown_connections 2023-10-16T09:49:02.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.115+0000 7fb5f559e640 1 -- 172.21.15.84:0/1798387358 wait complete. 2023-10-16T09:49:02.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.117+0000 7fb5f559e640 1 Processor -- start 2023-10-16T09:49:02.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.118+0000 7fb5f559e640 1 -- start start 2023-10-16T09:49:02.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.119+0000 7fb5f559e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f006c040 0x7fb5f00acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:02.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.119+0000 7fb5f559e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5f00ad970 0x7fb5f00ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:02.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.119+0000 7fb5f559e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f00cda80 0x7fb5f00ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:02.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.119+0000 7fb5f559e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f014e370 con 0x7fb5f00ad970 2023-10-16T09:49:02.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.119+0000 7fb5f559e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f00c9730 con 0x7fb5f00cda80 2023-10-16T09:49:02.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.119+0000 7fb5f559e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f01f67a0 con 0x7fb5f006c040 2023-10-16T09:49:02.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.119+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5f00ad970 0x7fb5f00ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:02.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.119+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5f00ad970 0x7fb5f00ca440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52814/0 (socket says 172.21.15.84:52814) 2023-10-16T09:49:02.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.119+0000 7fb5e7fff640 1 -- 172.21.15.84:0/255851828 learned_addr learned my addr 172.21.15.84:0/255851828 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:02.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.119+0000 7fb5effff640 1 --2- 172.21.15.84:0/255851828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f006c040 0x7fb5f00acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:02.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.120+0000 7fb5f4d9d640 1 --2- 172.21.15.84:0/255851828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f00cda80 0x7fb5f00ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:02.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.120+0000 7fb5e7fff640 1 -- 172.21.15.84:0/255851828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f006c040 msgr2=0x7fb5f00acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:02.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.120+0000 7fb5e7fff640 1 --2- 172.21.15.84:0/255851828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f006c040 0x7fb5f00acb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:02.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.120+0000 7fb5e7fff640 1 -- 172.21.15.84:0/255851828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f00cda80 msgr2=0x7fb5f00ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:02.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.120+0000 7fb5e7fff640 1 --2- 172.21.15.84:0/255851828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f00cda80 0x7fb5f00ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:02.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.120+0000 7fb5e7fff640 1 -- 172.21.15.84:0/255851828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e8009b30 con 0x7fb5f00ad970 2023-10-16T09:49:02.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.120+0000 7fb5e7fff640 1 --2- 172.21.15.84:0/255851828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5f00ad970 0x7fb5f00ca440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800c3c0 tx=0x7fb5d800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:02.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.120+0000 7fb5ed7fa640 1 -- 172.21.15.84:0/255851828 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d80052d0 con 0x7fb5f00ad970 2023-10-16T09:49:02.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.120+0000 7fb5ed7fa640 1 -- 172.21.15.84:0/255851828 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d8005450 con 0x7fb5f00ad970 2023-10-16T09:49:02.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.120+0000 7fb5ed7fa640 1 -- 172.21.15.84:0/255851828 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8003830 con 0x7fb5f00ad970 2023-10-16T09:49:02.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.120+0000 7fb5f559e640 1 -- 172.21.15.84:0/255851828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f00c9980 con 0x7fb5f00ad970 2023-10-16T09:49:02.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.121+0000 7fb5f559e640 1 -- 172.21.15.84:0/255851828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f00c9f40 con 0x7fb5f00ad970 2023-10-16T09:49:02.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.122+0000 7fb5f559e640 1 -- 172.21.15.84:0/255851828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f006b330 con 0x7fb5f00ad970 2023-10-16T09:49:02.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.123+0000 7fb5ed7fa640 1 -- 172.21.15.84:0/255851828 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb5d8003990 con 0x7fb5f00ad970 2023-10-16T09:49:02.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.124+0000 7fb5ed7fa640 1 --2- 172.21.15.84:0/255851828 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb5d006ae70 0x7fb5d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:02.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.124+0000 7fb5ed7fa640 1 -- 172.21.15.84:0/255851828 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fb5d8088a00 con 0x7fb5f00ad970 2023-10-16T09:49:02.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.124+0000 7fb5effff640 1 --2- 172.21.15.84:0/255851828 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb5d006ae70 0x7fb5d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:02.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.125+0000 7fb5effff640 1 --2- 172.21.15.84:0/255851828 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb5d006ae70 0x7fb5d006d330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009aa0 tx=0x7fb5e0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:02.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.129+0000 7fb5ed7fa640 1 -- 172.21.15.84:0/255851828 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb5d8053bc0 con 0x7fb5f00ad970 2023-10-16T09:49:02.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.348+0000 7fb5f559e640 1 -- 172.21.15.84:0/255851828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb5f004e9b0 con 0x7fb5f00ad970 2023-10-16T09:49:02.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.888+0000 7fb5ed7fa640 1 -- 172.21.15.84:0/255851828 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb5d80170c0 con 0x7fb5f00ad970 2023-10-16T09:49:02.891 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 2 and data pool 3 2023-10-16T09:49:02.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.891+0000 7fb5f559e640 1 -- 172.21.15.84:0/255851828 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb5d006ae70 msgr2=0x7fb5d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:02.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.891+0000 7fb5f559e640 1 --2- 172.21.15.84:0/255851828 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb5d006ae70 0x7fb5d006d330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009aa0 tx=0x7fb5e0009190 comp rx=0 tx=0).stop 2023-10-16T09:49:02.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.892+0000 7fb5f559e640 1 -- 172.21.15.84:0/255851828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5f00ad970 msgr2=0x7fb5f00ca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:02.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.892+0000 7fb5f559e640 1 --2- 172.21.15.84:0/255851828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5f00ad970 0x7fb5f00ca440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800c3c0 tx=0x7fb5d800c880 comp rx=0 tx=0).stop 2023-10-16T09:49:02.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.892+0000 7fb5f559e640 1 -- 172.21.15.84:0/255851828 shutdown_connections 2023-10-16T09:49:02.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.892+0000 7fb5f559e640 1 --2- 172.21.15.84:0/255851828 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb5d006ae70 0x7fb5d006d330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:02.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.892+0000 7fb5f559e640 1 --2- 172.21.15.84:0/255851828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5f00cda80 0x7fb5f00ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:02.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.892+0000 7fb5f559e640 1 --2- 172.21.15.84:0/255851828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5f00ad970 0x7fb5f00ca440 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:02.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.892+0000 7fb5f559e640 1 --2- 172.21.15.84:0/255851828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5f006c040 0x7fb5f00acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:02.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.892+0000 7fb5f559e640 1 -- 172.21.15.84:0/255851828 >> 172.21.15.84:0/255851828 conn(0x7fb5f00ce6a0 msgr2=0x7fb5f015e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:02.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.892+0000 7fb5f559e640 1 -- 172.21.15.84:0/255851828 shutdown_connections 2023-10-16T09:49:02.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:02.892+0000 7fb5f559e640 1 -- 172.21.15.84:0/255851828 wait complete. 2023-10-16T09:49:02.916 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:03.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.102+0000 7f164b52c640 1 Processor -- start 2023-10-16T09:49:03.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.102+0000 7f164b52c640 1 -- start start 2023-10-16T09:49:03.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.103+0000 7f164b52c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f16440ce390 0x7f164415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:03.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.103+0000 7f164b52c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164415ed60 con 0x7f164415ae80 2023-10-16T09:49:03.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.103+0000 7f164b52c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16440707e0 con 0x7f164415aaa0 2023-10-16T09:49:03.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.103+0000 7f164b52c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f164406b810 con 0x7f16440ce390 2023-10-16T09:49:03.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.103+0000 7f1648ac7640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f164415ae80 0x7f1644070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52888/0 (socket says 172.21.15.84:52888) 2023-10-16T09:49:03.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.103+0000 7f1648ac7640 1 -- 172.21.15.84:0/3341147224 learned_addr learned my addr 172.21.15.84:0/3341147224 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:03.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.103+0000 7f1649ac9640 1 --2- 172.21.15.84:0/3341147224 >> v2:172.21.15.84:3300/0 conn(0x7f16440ce390 0x7f164415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:03.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.103+0000 7f163bfff640 1 -- 172.21.15.84:0/3341147224 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695947648 0 0) 0x7f164415ed60 con 0x7f164415ae80 2023-10-16T09:49:03.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.103+0000 7f163bfff640 1 -- 172.21.15.84:0/3341147224 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1630003520 con 0x7f164415ae80 2023-10-16T09:49:03.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.104+0000 7f1649ac9640 1 -- 172.21.15.84:0/3341147224 >> v1:172.21.15.157:6789/0 conn(0x7f164415aaa0 legacy=0x7f16440cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:03.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.104+0000 7f1649ac9640 1 -- 172.21.15.84:0/3341147224 >> v1:172.21.15.84:6789/0 conn(0x7f164415ae80 legacy=0x7f1644070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:03.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.104+0000 7f1649ac9640 1 -- 172.21.15.84:0/3341147224 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164415ef90 con 0x7f16440ce390 2023-10-16T09:49:03.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.105+0000 7f1649ac9640 1 --2- 172.21.15.84:0/3341147224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16440ce390 0x7f164415e820 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f163c0097a0 tx=0x7f163c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93bade8f5413b22 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:03.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.105+0000 7f163bfff640 1 -- 172.21.15.84:0/3341147224 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c030610 con 0x7f16440ce390 2023-10-16T09:49:03.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.105+0000 7f163bfff640 1 -- 172.21.15.84:0/3341147224 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163c030770 con 0x7f16440ce390 2023-10-16T09:49:03.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.105+0000 7f163bfff640 1 -- 172.21.15.84:0/3341147224 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c030b00 con 0x7f16440ce390 2023-10-16T09:49:03.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.105+0000 7f164b52c640 1 -- 172.21.15.84:0/3341147224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16440ce390 msgr2=0x7f164415e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:03.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.105+0000 7f164b52c640 1 --2- 172.21.15.84:0/3341147224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16440ce390 0x7f164415e820 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f163c0097a0 tx=0x7f163c02b960 comp rx=0 tx=0).stop 2023-10-16T09:49:03.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.105+0000 7f164b52c640 1 -- 172.21.15.84:0/3341147224 shutdown_connections 2023-10-16T09:49:03.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.105+0000 7f164b52c640 1 --2- 172.21.15.84:0/3341147224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16440ce390 0x7f164415e820 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.105+0000 7f164b52c640 1 -- 172.21.15.84:0/3341147224 >> 172.21.15.84:0/3341147224 conn(0x7f16440cb410 msgr2=0x7f16440cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:03.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.106+0000 7f164b52c640 1 -- 172.21.15.84:0/3341147224 shutdown_connections 2023-10-16T09:49:03.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.106+0000 7f164b52c640 1 -- 172.21.15.84:0/3341147224 wait complete. 2023-10-16T09:49:03.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.107+0000 7f164b52c640 1 Processor -- start 2023-10-16T09:49:03.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.107+0000 7f164b52c640 1 -- start start 2023-10-16T09:49:03.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.108+0000 7f164b52c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16440ce390 0x7f16440c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:03.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.108+0000 7f164b52c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f164415aaa0 0x7f1644158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:03.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.108+0000 7f164b52c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f164415ae80 0x7f16440c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:03.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.108+0000 7f164b52c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1644159e60 con 0x7f16440ce390 2023-10-16T09:49:03.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.108+0000 7f164b52c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16440c76a0 con 0x7f164415ae80 2023-10-16T09:49:03.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.108+0000 7f164b52c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1644159380 con 0x7f164415aaa0 2023-10-16T09:49:03.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.108+0000 7f16492c8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16440ce390 0x7f16440c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:03.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.108+0000 7f16492c8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16440ce390 0x7f16440c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52828/0 (socket says 172.21.15.84:52828) 2023-10-16T09:49:03.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.108+0000 7f16492c8640 1 -- 172.21.15.84:0/3710941639 learned_addr learned my addr 172.21.15.84:0/3710941639 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:03.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.109+0000 7f16492c8640 1 -- 172.21.15.84:0/3710941639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f164415aaa0 msgr2=0x7f1644158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:03.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.109+0000 7f1648ac7640 1 --2- 172.21.15.84:0/3710941639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f164415aaa0 0x7f1644158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:03.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.109+0000 7f16492c8640 1 --2- 172.21.15.84:0/3710941639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f164415aaa0 0x7f1644158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.109+0000 7f1649ac9640 1 --2- 172.21.15.84:0/3710941639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f164415ae80 0x7f16440c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:03.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.109+0000 7f16492c8640 1 -- 172.21.15.84:0/3710941639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f164415ae80 msgr2=0x7f16440c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:03.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.109+0000 7f16492c8640 1 --2- 172.21.15.84:0/3710941639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f164415ae80 0x7f16440c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.109+0000 7f16492c8640 1 -- 172.21.15.84:0/3710941639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c009400 con 0x7f16440ce390 2023-10-16T09:49:03.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.109+0000 7f16492c8640 1 --2- 172.21.15.84:0/3710941639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16440ce390 0x7f16440c7b30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f164000c450 tx=0x7f164000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:03.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.109+0000 7f1639ffb640 1 -- 172.21.15.84:0/3710941639 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16400052d0 con 0x7f16440ce390 2023-10-16T09:49:03.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.109+0000 7f1639ffb640 1 -- 172.21.15.84:0/3710941639 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1640005430 con 0x7f16440ce390 2023-10-16T09:49:03.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.109+0000 7f164b52c640 1 -- 172.21.15.84:0/3710941639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16440c67b0 con 0x7f16440ce390 2023-10-16T09:49:03.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.109+0000 7f1639ffb640 1 -- 172.21.15.84:0/3710941639 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1640012870 con 0x7f16440ce390 2023-10-16T09:49:03.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.110+0000 7f164b52c640 1 -- 172.21.15.84:0/3710941639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16440c6d70 con 0x7f16440ce390 2023-10-16T09:49:03.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.111+0000 7f164b52c640 1 -- 172.21.15.84:0/3710941639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164404e160 con 0x7f16440ce390 2023-10-16T09:49:03.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.112+0000 7f1639ffb640 1 -- 172.21.15.84:0/3710941639 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f16400129d0 con 0x7f16440ce390 2023-10-16T09:49:03.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.113+0000 7f1639ffb640 1 --2- 172.21.15.84:0/3710941639 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f162806ae70 0x7f162806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:03.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.113+0000 7f1639ffb640 1 -- 172.21.15.84:0/3710941639 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1640089130 con 0x7f16440ce390 2023-10-16T09:49:03.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.113+0000 7f1648ac7640 1 --2- 172.21.15.84:0/3710941639 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f162806ae70 0x7f162806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:03.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.114+0000 7f1648ac7640 1 --2- 172.21.15.84:0/3710941639 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f162806ae70 0x7f162806d330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f16440c66e0 tx=0x7f1634009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:03.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.118+0000 7f1639ffb640 1 -- 172.21.15.84:0/3710941639 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f16400541e0 con 0x7f16440ce390 2023-10-16T09:49:03.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.311+0000 7f164b52c640 1 -- 172.21.15.84:0/3710941639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16440cdc60 con 0x7f16440ce390 2023-10-16T09:49:03.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.314+0000 7f1639ffb640 1 -- 172.21.15.84:0/3710941639 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16494 (secure 0 0 0) 0x7f1640057800 con 0x7f16440ce390 2023-10-16T09:49:03.317 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:03.317 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":20,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:02.350182+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:03.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.317+0000 7f164b52c640 1 -- 172.21.15.84:0/3710941639 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f162806ae70 msgr2=0x7f162806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:03.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.317+0000 7f164b52c640 1 --2- 172.21.15.84:0/3710941639 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f162806ae70 0x7f162806d330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f16440c66e0 tx=0x7f1634009040 comp rx=0 tx=0).stop 2023-10-16T09:49:03.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.317+0000 7f164b52c640 1 -- 172.21.15.84:0/3710941639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16440ce390 msgr2=0x7f16440c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:03.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.317+0000 7f164b52c640 1 --2- 172.21.15.84:0/3710941639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16440ce390 0x7f16440c7b30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f164000c450 tx=0x7f164000c910 comp rx=0 tx=0).stop 2023-10-16T09:49:03.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.317+0000 7f164b52c640 1 -- 172.21.15.84:0/3710941639 shutdown_connections 2023-10-16T09:49:03.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.317+0000 7f164b52c640 1 --2- 172.21.15.84:0/3710941639 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f162806ae70 0x7f162806d330 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.317+0000 7f164b52c640 1 --2- 172.21.15.84:0/3710941639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f164415ae80 0x7f16440c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.317+0000 7f164b52c640 1 --2- 172.21.15.84:0/3710941639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f164415aaa0 0x7f1644158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.317+0000 7f164b52c640 1 --2- 172.21.15.84:0/3710941639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16440ce390 0x7f16440c7b30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.317+0000 7f164b52c640 1 -- 172.21.15.84:0/3710941639 >> 172.21.15.84:0/3710941639 conn(0x7f16440cb410 msgr2=0x7f16440b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:03.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.317+0000 7f164b52c640 1 -- 172.21.15.84:0/3710941639 shutdown_connections 2023-10-16T09:49:03.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.317+0000 7f164b52c640 1 -- 172.21.15.84:0/3710941639 wait complete. 2023-10-16T09:49:03.336 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:03.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.550+0000 7f59271e2640 1 Processor -- start 2023-10-16T09:49:03.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.551+0000 7f59271e2640 1 -- start start 2023-10-16T09:49:03.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.552+0000 7f59271e2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5920159f00 0x7f59200b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:03.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.552+0000 7f59271e2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f592015d880 0x7f59200b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:03.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.552+0000 7f59271e2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592006f7f0 con 0x7f5920159b20 2023-10-16T09:49:03.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.552+0000 7f59271e2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592006c5e0 con 0x7f5920159f00 2023-10-16T09:49:03.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.552+0000 7f59271e2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592006c8c0 con 0x7f592015d880 2023-10-16T09:49:03.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.552+0000 7f591ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5920159f00 0x7f59200b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:03.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.552+0000 7f591ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5920159f00 0x7f59200b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52844/0 (socket says 172.21.15.84:52844) 2023-10-16T09:49:03.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.552+0000 7f591ffff640 1 -- 172.21.15.84:0/1650656095 learned_addr learned my addr 172.21.15.84:0/1650656095 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:03.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.552+0000 7f591ffff640 1 -- 172.21.15.84:0/1650656095 >> v2:172.21.15.157:3300/0 conn(0x7f592015d880 msgr2=0x7f59200b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:03.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.552+0000 7f591ffff640 1 --2- 172.21.15.84:0/1650656095 >> v2:172.21.15.157:3300/0 conn(0x7f592015d880 0x7f59200b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.552+0000 7f591ffff640 1 -- 172.21.15.84:0/1650656095 >> v1:172.21.15.157:6789/0 conn(0x7f5920159b20 legacy=0x7f592006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:03.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.552+0000 7f591ffff640 1 -- 172.21.15.84:0/1650656095 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592006cba0 con 0x7f5920159f00 2023-10-16T09:49:03.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.553+0000 7f591ffff640 1 --2- 172.21.15.84:0/1650656095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5920159f00 0x7f59200b0410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f59200c8bb0 tx=0x7f591002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b773c029450720 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:03.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.554+0000 7f591f7fe640 1 -- 172.21.15.84:0/1650656095 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910031870 con 0x7f5920159f00 2023-10-16T09:49:03.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.554+0000 7f591f7fe640 1 -- 172.21.15.84:0/1650656095 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59100319d0 con 0x7f5920159f00 2023-10-16T09:49:03.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.554+0000 7f591f7fe640 1 -- 172.21.15.84:0/1650656095 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910031d60 con 0x7f5920159f00 2023-10-16T09:49:03.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.554+0000 7f59271e2640 1 -- 172.21.15.84:0/1650656095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5920159f00 msgr2=0x7f59200b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:03.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.554+0000 7f59271e2640 1 --2- 172.21.15.84:0/1650656095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5920159f00 0x7f59200b0410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f59200c8bb0 tx=0x7f591002b910 comp rx=0 tx=0).stop 2023-10-16T09:49:03.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.554+0000 7f59271e2640 1 -- 172.21.15.84:0/1650656095 shutdown_connections 2023-10-16T09:49:03.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.554+0000 7f59271e2640 1 --2- 172.21.15.84:0/1650656095 >> v2:172.21.15.157:3300/0 conn(0x7f592015d880 0x7f59200b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.554+0000 7f59271e2640 1 --2- 172.21.15.84:0/1650656095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5920159f00 0x7f59200b0410 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.554+0000 7f59271e2640 1 -- 172.21.15.84:0/1650656095 >> 172.21.15.84:0/1650656095 conn(0x7f59200cb410 msgr2=0x7f59200cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:03.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.554+0000 7f59271e2640 1 -- 172.21.15.84:0/1650656095 shutdown_connections 2023-10-16T09:49:03.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.554+0000 7f59271e2640 1 -- 172.21.15.84:0/1650656095 wait complete. 2023-10-16T09:49:03.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.556+0000 7f59271e2640 1 Processor -- start 2023-10-16T09:49:03.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.556+0000 7f59271e2640 1 -- start start 2023-10-16T09:49:03.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.557+0000 7f59271e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5920159b20 0x7f5920148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:03.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.557+0000 7f59271e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5920159f00 0x7f5920157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:03.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.557+0000 7f59271e2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f592015d880 0x7f5920157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:03.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.557+0000 7f59271e2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5920149380 con 0x7f5920159b20 2023-10-16T09:49:03.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.557+0000 7f59271e2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59200c30d0 con 0x7f5920159f00 2023-10-16T09:49:03.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.557+0000 7f59271e2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59200c3560 con 0x7f592015d880 2023-10-16T09:49:03.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.557+0000 7f5924f7e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5920159b20 0x7f5920148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:03.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.557+0000 7f5924f7e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5920159b20 0x7f5920148e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52856/0 (socket says 172.21.15.84:52856) 2023-10-16T09:49:03.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.557+0000 7f5924f7e640 1 -- 172.21.15.84:0/2590795403 learned_addr learned my addr 172.21.15.84:0/2590795403 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:03.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.558+0000 7f5924f7e640 1 -- 172.21.15.84:0/2590795403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f592015d880 msgr2=0x7f5920157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:03.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.558+0000 7f5924f7e640 1 --2- 172.21.15.84:0/2590795403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f592015d880 0x7f5920157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.558+0000 7f5924f7e640 1 -- 172.21.15.84:0/2590795403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5920159f00 msgr2=0x7f5920157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:03.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.558+0000 7f5924f7e640 1 --2- 172.21.15.84:0/2590795403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5920159f00 0x7f5920157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.558+0000 7f5924f7e640 1 -- 172.21.15.84:0/2590795403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59100094d0 con 0x7f5920159b20 2023-10-16T09:49:03.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.558+0000 7f5924f7e640 1 --2- 172.21.15.84:0/2590795403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5920159b20 0x7f5920148e40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f590800b3e0 tx=0x7f590800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:03.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.558+0000 7f591d7fa640 1 -- 172.21.15.84:0/2590795403 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5908011830 con 0x7f5920159b20 2023-10-16T09:49:03.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.558+0000 7f591d7fa640 1 -- 172.21.15.84:0/2590795403 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59080119b0 con 0x7f5920159b20 2023-10-16T09:49:03.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.558+0000 7f591d7fa640 1 -- 172.21.15.84:0/2590795403 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5908011d60 con 0x7f5920159b20 2023-10-16T09:49:03.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.558+0000 7f59271e2640 1 -- 172.21.15.84:0/2590795403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59200c24c0 con 0x7f5920159b20 2023-10-16T09:49:03.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.559+0000 7f59271e2640 1 -- 172.21.15.84:0/2590795403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59200c2a80 con 0x7f5920159b20 2023-10-16T09:49:03.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.560+0000 7f59271e2640 1 -- 172.21.15.84:0/2590795403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592006c270 con 0x7f5920159b20 2023-10-16T09:49:03.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.560+0000 7f591d7fa640 1 -- 172.21.15.84:0/2590795403 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f590801a400 con 0x7f5920159b20 2023-10-16T09:49:03.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.561+0000 7f591d7fa640 1 --2- 172.21.15.84:0/2590795403 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f590006aea0 0x7f590006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:03.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.561+0000 7f591d7fa640 1 -- 172.21.15.84:0/2590795403 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5908089880 con 0x7f5920159b20 2023-10-16T09:49:03.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.561+0000 7f591ffff640 1 --2- 172.21.15.84:0/2590795403 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f590006aea0 0x7f590006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:03.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.562+0000 7f591ffff640 1 --2- 172.21.15.84:0/2590795403 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f590006aea0 0x7f590006d360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5910036000 tx=0x7f5910030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:03.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.566+0000 7f591d7fa640 1 -- 172.21.15.84:0/2590795403 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5908054890 con 0x7f5920159b20 2023-10-16T09:49:03.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.747+0000 7f59271e2640 1 -- 172.21.15.84:0/2590795403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f592004e160 con 0x7f5920159b20 2023-10-16T09:49:03.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.749+0000 7f591d7fa640 1 -- 172.21.15.84:0/2590795403 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f5908015090 con 0x7f5920159b20 2023-10-16T09:49:03.751 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:03.751 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":21,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:03.529261+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:03.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.751+0000 7f59271e2640 1 -- 172.21.15.84:0/2590795403 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f590006aea0 msgr2=0x7f590006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:03.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.751+0000 7f59271e2640 1 --2- 172.21.15.84:0/2590795403 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f590006aea0 0x7f590006d360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5910036000 tx=0x7f5910030000 comp rx=0 tx=0).stop 2023-10-16T09:49:03.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.751+0000 7f59271e2640 1 -- 172.21.15.84:0/2590795403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5920159b20 msgr2=0x7f5920148e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:03.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.751+0000 7f59271e2640 1 --2- 172.21.15.84:0/2590795403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5920159b20 0x7f5920148e40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f590800b3e0 tx=0x7f590800b8a0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.751+0000 7f59271e2640 1 -- 172.21.15.84:0/2590795403 shutdown_connections 2023-10-16T09:49:03.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.751+0000 7f59271e2640 1 --2- 172.21.15.84:0/2590795403 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f590006aea0 0x7f590006d360 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.751+0000 7f59271e2640 1 --2- 172.21.15.84:0/2590795403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f592015d880 0x7f5920157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.751+0000 7f59271e2640 1 --2- 172.21.15.84:0/2590795403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5920159f00 0x7f5920157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.751+0000 7f59271e2640 1 --2- 172.21.15.84:0/2590795403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5920159b20 0x7f5920148e40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.751+0000 7f59271e2640 1 -- 172.21.15.84:0/2590795403 >> 172.21.15.84:0/2590795403 conn(0x7f59200cb410 msgr2=0x7f59200cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:03.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.753+0000 7f59271e2640 1 -- 172.21.15.84:0/2590795403 shutdown_connections 2023-10-16T09:49:03.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.753+0000 7f59271e2640 1 -- 172.21.15.84:0/2590795403 wait complete. 2023-10-16T09:49:03.771 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T09:49:03.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.988+0000 7fbc91b78640 1 Processor -- start 2023-10-16T09:49:03.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.989+0000 7fbc91b78640 1 -- start start 2023-10-16T09:49:03.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.990+0000 7fbc91b78640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbc8c15af80 0x7fbc8c0cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:03.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.990+0000 7fbc91b78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc8c065ab0 0x7fbc8c065f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:03.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.990+0000 7fbc91b78640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc8c15eff0 con 0x7fbc8c06faf0 2023-10-16T09:49:03.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.990+0000 7fbc91b78640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc8c066480 con 0x7fbc8c15af80 2023-10-16T09:49:03.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.990+0000 7fbc91b78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c0d0930 con 0x7fbc8c065ab0 2023-10-16T09:49:03.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.990+0000 7fbc8affd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbc8c06faf0 0x7fbc8c15f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52892/0 (socket says 172.21.15.84:52892) 2023-10-16T09:49:03.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.990+0000 7fbc8affd640 1 -- 172.21.15.84:0/3470249341 learned_addr learned my addr 172.21.15.84:0/3470249341 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:03.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8bfff640 1 --2- 172.21.15.84:0/3470249341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc8c065ab0 0x7fbc8c065f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:03.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8a7fc640 1 -- 172.21.15.84:0/3470249341 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1370874703 0 0) 0x7fbc8c15eff0 con 0x7fbc8c06faf0 2023-10-16T09:49:03.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8a7fc640 1 -- 172.21.15.84:0/3470249341 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc78003520 con 0x7fbc8c06faf0 2023-10-16T09:49:03.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8b7fe640 1 --2- 172.21.15.84:0/3470249341 >> v2:172.21.15.157:3300/0 conn(0x7fbc8c15af80 0x7fbc8c0cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:03.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8a7fc640 1 -- 172.21.15.84:0/3470249341 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 24208098 0 0) 0x7fbc78003520 con 0x7fbc8c06faf0 2023-10-16T09:49:03.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8a7fc640 1 -- 172.21.15.84:0/3470249341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc8c065ab0 msgr2=0x7fbc8c065f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:03.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8a7fc640 1 --2- 172.21.15.84:0/3470249341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc8c065ab0 0x7fbc8c065f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8a7fc640 1 -- 172.21.15.84:0/3470249341 >> v2:172.21.15.157:3300/0 conn(0x7fbc8c15af80 msgr2=0x7fbc8c0cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:03.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8a7fc640 1 --2- 172.21.15.84:0/3470249341 >> v2:172.21.15.157:3300/0 conn(0x7fbc8c15af80 0x7fbc8c0cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8a7fc640 1 -- 172.21.15.84:0/3470249341 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8c15ecc0 con 0x7fbc8c06faf0 2023-10-16T09:49:03.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8a7fc640 1 -- 172.21.15.84:0/3470249341 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fbc74001670 con 0x7fbc8c06faf0 2023-10-16T09:49:03.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8bfff640 1 --2- 172.21.15.84:0/3470249341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc8c065ab0 0x7fbc8c065f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:03.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8a7fc640 1 -- 172.21.15.84:0/3470249341 >> v1:172.21.15.84:6789/0 conn(0x7fbc8c06faf0 legacy=0x7fbc8c15f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:03.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8a7fc640 1 --2- 172.21.15.84:0/3470249341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc780039a0 0x7fbc78003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:03.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8a7fc640 1 --2- 172.21.15.84:0/3470249341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc780047c0 0x7fbc78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:03.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8a7fc640 1 --2- 172.21.15.84:0/3470249341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc8c065ab0 0x7fbc78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:03.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8a7fc640 1 -- 172.21.15.84:0/3470249341 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c066480 con 0x7fbc780047c0 2023-10-16T09:49:03.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8a7fc640 1 -- 172.21.15.84:0/3470249341 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c0d0930 con 0x7fbc780039a0 2023-10-16T09:49:03.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.991+0000 7fbc8a7fc640 1 -- 172.21.15.84:0/3470249341 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7800bae0 con 0x7fbc8c065ab0 2023-10-16T09:49:03.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.992+0000 7fbc8affd640 1 --2- 172.21.15.84:0/3470249341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc780047c0 0x7fbc78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:03.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.992+0000 7fbc8bfff640 1 --2- 172.21.15.84:0/3470249341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc8c065ab0 0x7fbc78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:03.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.992+0000 7fbc8affd640 1 -- 172.21.15.84:0/3470249341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc8c065ab0 msgr2=0x7fbc78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:03.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.992+0000 7fbc8affd640 1 --2- 172.21.15.84:0/3470249341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc8c065ab0 0x7fbc78005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.992+0000 7fbc8affd640 1 -- 172.21.15.84:0/3470249341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc780039a0 msgr2=0x7fbc78003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:03.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.992+0000 7fbc8affd640 1 --2- 172.21.15.84:0/3470249341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc780039a0 0x7fbc78003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:03.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.992+0000 7fbc8affd640 1 -- 172.21.15.84:0/3470249341 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc78003520 con 0x7fbc780047c0 2023-10-16T09:49:03.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.993+0000 7fbc8affd640 1 --2- 172.21.15.84:0/3470249341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc780047c0 0x7fbc78004c40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400c240 tx=0x7fbc780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3dd9d34258476d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:04.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.993+0000 7fbc8a7fc640 1 -- 172.21.15.84:0/3470249341 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc74005ad0 con 0x7fbc780047c0 2023-10-16T09:49:04.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.993+0000 7fbc8a7fc640 1 -- 172.21.15.84:0/3470249341 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc74005c30 con 0x7fbc780047c0 2023-10-16T09:49:04.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.993+0000 7fbc8a7fc640 1 -- 172.21.15.84:0/3470249341 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7402c8b0 con 0x7fbc780047c0 2023-10-16T09:49:04.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.994+0000 7fbc91b78640 1 -- 172.21.15.84:0/3470249341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc780047c0 msgr2=0x7fbc78004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:04.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.994+0000 7fbc91b78640 1 --2- 172.21.15.84:0/3470249341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc780047c0 0x7fbc78004c40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400c240 tx=0x7fbc780057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:04.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.994+0000 7fbc8bfff640 1 -- 172.21.15.84:0/3470249341 reap_dead start 2023-10-16T09:49:04.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.994+0000 7fbc91b78640 1 -- 172.21.15.84:0/3470249341 shutdown_connections 2023-10-16T09:49:04.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.994+0000 7fbc91b78640 1 -- 172.21.15.84:0/3470249341 >> 172.21.15.84:0/3470249341 conn(0x7fbc8c0d0ab0 msgr2=0x7fbc8c0d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:04.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.994+0000 7fbc91b78640 1 -- 172.21.15.84:0/3470249341 shutdown_connections 2023-10-16T09:49:04.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.994+0000 7fbc91b78640 1 -- 172.21.15.84:0/3470249341 wait complete. 2023-10-16T09:49:04.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.996+0000 7fbc91b78640 1 Processor -- start 2023-10-16T09:49:04.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.996+0000 7fbc91b78640 1 -- start start 2023-10-16T09:49:04.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.997+0000 7fbc91b78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc780047c0 0x7fbc8c158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:04.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.997+0000 7fbc91b78640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc8c1f6160 0x7fbc8c1f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:04.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.997+0000 7fbc91b78640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc8c0ac900 0x7fbc8c0acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:04.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.997+0000 7fbc91b78640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c0ad2c0 con 0x7fbc8c0ac900 2023-10-16T09:49:04.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.997+0000 7fbc91b78640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c0c8b40 con 0x7fbc8c1f6160 2023-10-16T09:49:04.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.997+0000 7fbc91b78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c1f6c20 con 0x7fbc780047c0 2023-10-16T09:49:04.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.997+0000 7fbc8bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc8c0ac900 0x7fbc8c0acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:04.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.997+0000 7fbc8bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc8c0ac900 0x7fbc8c0acd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52870/0 (socket says 172.21.15.84:52870) 2023-10-16T09:49:04.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.997+0000 7fbc8bfff640 1 -- 172.21.15.84:0/2917494312 learned_addr learned my addr 172.21.15.84:0/2917494312 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:04.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.997+0000 7fbc8b7fe640 1 --2- 172.21.15.84:0/2917494312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc780047c0 0x7fbc8c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:04.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.998+0000 7fbc8bfff640 1 -- 172.21.15.84:0/2917494312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc780047c0 msgr2=0x7fbc8c158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:04.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.998+0000 7fbc8bfff640 1 --2- 172.21.15.84:0/2917494312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc780047c0 0x7fbc8c158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:04.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.998+0000 7fbc8bfff640 1 -- 172.21.15.84:0/2917494312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc8c1f6160 msgr2=0x7fbc8c1f65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:04.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.998+0000 7fbc8bfff640 1 --2- 172.21.15.84:0/2917494312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc8c1f6160 0x7fbc8c1f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:04.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.998+0000 7fbc8bfff640 1 -- 172.21.15.84:0/2917494312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8c1f6f00 con 0x7fbc8c0ac900 2023-10-16T09:49:04.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.998+0000 7fbc8b7fe640 1 --2- 172.21.15.84:0/2917494312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc780047c0 0x7fbc8c158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:04.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.998+0000 7fbc8bfff640 1 --2- 172.21.15.84:0/2917494312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc8c0ac900 0x7fbc8c0acd80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0b36a0 tx=0x7fbc7c00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:04.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.998+0000 7fbc83fff640 1 -- 172.21.15.84:0/2917494312 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c004230 con 0x7fbc8c0ac900 2023-10-16T09:49:04.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.998+0000 7fbc83fff640 1 -- 172.21.15.84:0/2917494312 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7c00c070 con 0x7fbc8c0ac900 2023-10-16T09:49:04.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.998+0000 7fbc91b78640 1 -- 172.21.15.84:0/2917494312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8c0c8de0 con 0x7fbc8c0ac900 2023-10-16T09:49:04.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.998+0000 7fbc83fff640 1 -- 172.21.15.84:0/2917494312 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c003610 con 0x7fbc8c0ac900 2023-10-16T09:49:04.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.998+0000 7fbc91b78640 1 -- 172.21.15.84:0/2917494312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8c15e160 con 0x7fbc8c0ac900 2023-10-16T09:49:04.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:03.999+0000 7fbc91b78640 1 -- 172.21.15.84:0/2917494312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8c0c9300 con 0x7fbc8c0ac900 2023-10-16T09:49:04.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.000+0000 7fbc83fff640 1 -- 172.21.15.84:0/2917494312 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbc7c003770 con 0x7fbc8c0ac900 2023-10-16T09:49:04.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.001+0000 7fbc83fff640 1 --2- 172.21.15.84:0/2917494312 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbc6c06aea0 0x7fbc6c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:04.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.001+0000 7fbc83fff640 1 -- 172.21.15.84:0/2917494312 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc7c0892a0 con 0x7fbc8c0ac900 2023-10-16T09:49:04.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.001+0000 7fbc8b7fe640 1 --2- 172.21.15.84:0/2917494312 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbc6c06aea0 0x7fbc6c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:04.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.002+0000 7fbc8b7fe640 1 --2- 172.21.15.84:0/2917494312 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbc6c06aea0 0x7fbc6c06d360 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbc840070b0 tx=0x7fbc8400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:04.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.005+0000 7fbc83fff640 1 -- 172.21.15.84:0/2917494312 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbc8c0c9300 con 0x7fbc8c0ac900 2023-10-16T09:49:04.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.236+0000 7fbc91b78640 1 -- 172.21.15.84:0/2917494312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbc8c0c9300 con 0x7fbc8c0ac900 2023-10-16T09:49:04.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.911+0000 7fbc83fff640 1 -- 172.21.15.84:0/2917494312 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbc8c0c9300 con 0x7fbc8c0ac900 2023-10-16T09:49:04.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.915+0000 7fbc91b78640 1 -- 172.21.15.84:0/2917494312 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbc6c06aea0 msgr2=0x7fbc6c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:04.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.915+0000 7fbc91b78640 1 --2- 172.21.15.84:0/2917494312 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbc6c06aea0 0x7fbc6c06d360 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbc840070b0 tx=0x7fbc8400b040 comp rx=0 tx=0).stop 2023-10-16T09:49:04.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.915+0000 7fbc91b78640 1 -- 172.21.15.84:0/2917494312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc8c0ac900 msgr2=0x7fbc8c0acd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:04.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.915+0000 7fbc91b78640 1 --2- 172.21.15.84:0/2917494312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc8c0ac900 0x7fbc8c0acd80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0b36a0 tx=0x7fbc7c00f670 comp rx=0 tx=0).stop 2023-10-16T09:49:04.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.915+0000 7fbc91b78640 1 -- 172.21.15.84:0/2917494312 shutdown_connections 2023-10-16T09:49:04.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.915+0000 7fbc91b78640 1 --2- 172.21.15.84:0/2917494312 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbc6c06aea0 0x7fbc6c06d360 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:04.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.915+0000 7fbc91b78640 1 --2- 172.21.15.84:0/2917494312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc8c0ac900 0x7fbc8c0acd80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:04.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.915+0000 7fbc91b78640 1 --2- 172.21.15.84:0/2917494312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc8c1f6160 0x7fbc8c1f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:04.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.915+0000 7fbc91b78640 1 --2- 172.21.15.84:0/2917494312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc780047c0 0x7fbc8c158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:04.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.915+0000 7fbc91b78640 1 -- 172.21.15.84:0/2917494312 >> 172.21.15.84:0/2917494312 conn(0x7fbc8c0d0ab0 msgr2=0x7fbc8c06aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:04.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.916+0000 7fbc91b78640 1 -- 172.21.15.84:0/2917494312 shutdown_connections 2023-10-16T09:49:04.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:04.916+0000 7fbc91b78640 1 -- 172.21.15.84:0/2917494312 wait complete. 2023-10-16T09:49:04.937 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T09:49:04.937 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T09:49:05.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.151+0000 7ffb961ba640 1 Processor -- start 2023-10-16T09:49:05.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.152+0000 7ffb961ba640 1 -- start start 2023-10-16T09:49:05.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.153+0000 7ffb961ba640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb9015cd70 0x7ffb900d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:05.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.153+0000 7ffb961ba640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb9015d150 con 0x7ffb900b8480 2023-10-16T09:49:05.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.153+0000 7ffb961ba640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb900d1180 con 0x7ffb900cded0 2023-10-16T09:49:05.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.153+0000 7ffb961ba640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb900d1460 con 0x7ffb9015cd70 2023-10-16T09:49:05.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.153+0000 7ffb949b7640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ffb900b8480 0x7ffb900cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52902/0 (socket says 172.21.15.84:52902) 2023-10-16T09:49:05.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.153+0000 7ffb949b7640 1 -- 172.21.15.84:0/3736745881 learned_addr learned my addr 172.21.15.84:0/3736745881 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:05.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.154+0000 7ffb951b8640 1 --2- 172.21.15.84:0/3736745881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb9015cd70 0x7ffb900d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:05.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.154+0000 7ffb8ffff640 1 -- 172.21.15.84:0/3736745881 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2410229318 0 0) 0x7ffb9015d150 con 0x7ffb900b8480 2023-10-16T09:49:05.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.154+0000 7ffb8ffff640 1 -- 172.21.15.84:0/3736745881 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb78003520 con 0x7ffb900b8480 2023-10-16T09:49:05.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.154+0000 7ffb8ffff640 1 -- 172.21.15.84:0/3736745881 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4052348219 0 0) 0x7ffb78003520 con 0x7ffb900b8480 2023-10-16T09:49:05.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.154+0000 7ffb8ffff640 1 -- 172.21.15.84:0/3736745881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb9015cd70 msgr2=0x7ffb900d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:05.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.154+0000 7ffb8ffff640 1 --2- 172.21.15.84:0/3736745881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb9015cd70 0x7ffb900d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:05.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.154+0000 7ffb8ffff640 1 -- 172.21.15.84:0/3736745881 >> v1:172.21.15.157:6789/0 conn(0x7ffb900cded0 legacy=0x7ffb900b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:05.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.154+0000 7ffb8ffff640 1 -- 172.21.15.84:0/3736745881 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb900d1740 con 0x7ffb900b8480 2023-10-16T09:49:05.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.154+0000 7ffb8ffff640 1 -- 172.21.15.84:0/3736745881 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ffb84001670 con 0x7ffb900b8480 2023-10-16T09:49:05.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.154+0000 7ffb8ffff640 1 -- 172.21.15.84:0/3736745881 >> v1:172.21.15.84:6789/0 conn(0x7ffb900b8480 legacy=0x7ffb900cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:05.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.154+0000 7ffb8ffff640 1 --2- 172.21.15.84:0/3736745881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb780039a0 0x7ffb78003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:05.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.154+0000 7ffb8ffff640 1 --2- 172.21.15.84:0/3736745881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb9015cd70 0x7ffb780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:05.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.154+0000 7ffb8ffff640 1 --2- 172.21.15.84:0/3736745881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb78004d00 0x7ffb78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:05.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.154+0000 7ffb8ffff640 1 -- 172.21.15.84:0/3736745881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb900d1460 con 0x7ffb780039a0 2023-10-16T09:49:05.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.154+0000 7ffb8ffff640 1 -- 172.21.15.84:0/3736745881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7800ba30 con 0x7ffb78004d00 2023-10-16T09:49:05.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.154+0000 7ffb8ffff640 1 -- 172.21.15.84:0/3736745881 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7800bbf0 con 0x7ffb9015cd70 2023-10-16T09:49:05.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.155+0000 7ffb951b8640 1 --2- 172.21.15.84:0/3736745881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb780039a0 0x7ffb78003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:05.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.155+0000 7ffb951b8640 1 -- 172.21.15.84:0/3736745881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb9015cd70 msgr2=0x7ffb780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:05.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.155+0000 7ffb949b7640 1 --2- 172.21.15.84:0/3736745881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb9015cd70 0x7ffb780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:05.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.155+0000 7ffb951b8640 1 --2- 172.21.15.84:0/3736745881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb9015cd70 0x7ffb780047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:05.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.155+0000 7ffb959b9640 1 --2- 172.21.15.84:0/3736745881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb78004d00 0x7ffb78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:05.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.155+0000 7ffb951b8640 1 -- 172.21.15.84:0/3736745881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb78004d00 msgr2=0x7ffb78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:05.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.155+0000 7ffb951b8640 1 --2- 172.21.15.84:0/3736745881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb78004d00 0x7ffb78005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:05.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.155+0000 7ffb951b8640 1 -- 172.21.15.84:0/3736745881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb78003520 con 0x7ffb780039a0 2023-10-16T09:49:05.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.155+0000 7ffb959b9640 1 --2- 172.21.15.84:0/3736745881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb78004d00 0x7ffb78005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:05.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.156+0000 7ffb951b8640 1 --2- 172.21.15.84:0/3736745881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb780039a0 0x7ffb78003dc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009f90 tx=0x7ffb780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e5a8e9edd2ebce server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:05.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.156+0000 7ffb8ffff640 1 -- 172.21.15.84:0/3736745881 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8002cbd0 con 0x7ffb780039a0 2023-10-16T09:49:05.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.156+0000 7ffb8ffff640 1 -- 172.21.15.84:0/3736745881 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8000f040 con 0x7ffb780039a0 2023-10-16T09:49:05.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.156+0000 7ffb8ffff640 1 -- 172.21.15.84:0/3736745881 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb80034440 con 0x7ffb780039a0 2023-10-16T09:49:05.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.157+0000 7ffb961ba640 1 -- 172.21.15.84:0/3736745881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb780039a0 msgr2=0x7ffb78003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:05.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.157+0000 7ffb961ba640 1 --2- 172.21.15.84:0/3736745881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb780039a0 0x7ffb78003dc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009f90 tx=0x7ffb780057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:05.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.157+0000 7ffb959b9640 1 -- 172.21.15.84:0/3736745881 reap_dead start 2023-10-16T09:49:05.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.157+0000 7ffb961ba640 1 -- 172.21.15.84:0/3736745881 shutdown_connections 2023-10-16T09:49:05.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.157+0000 7ffb961ba640 1 -- 172.21.15.84:0/3736745881 >> 172.21.15.84:0/3736745881 conn(0x7ffb900cf570 msgr2=0x7ffb900cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:05.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.157+0000 7ffb961ba640 1 -- 172.21.15.84:0/3736745881 shutdown_connections 2023-10-16T09:49:05.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.157+0000 7ffb961ba640 1 -- 172.21.15.84:0/3736745881 wait complete. 2023-10-16T09:49:05.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.159+0000 7ffb961ba640 1 Processor -- start 2023-10-16T09:49:05.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.159+0000 7ffb961ba640 1 -- start start 2023-10-16T09:49:05.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.160+0000 7ffb961ba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb780039a0 0x7ffb900ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:05.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.160+0000 7ffb961ba640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb900c9bc0 0x7ffb900b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:05.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.160+0000 7ffb961ba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb90156350 0x7ffb900b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:05.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.160+0000 7ffb961ba640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb90158440 con 0x7ffb90156350 2023-10-16T09:49:05.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.160+0000 7ffb961ba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb90157e30 con 0x7ffb780039a0 2023-10-16T09:49:05.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.160+0000 7ffb961ba640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb900caac0 con 0x7ffb900c9bc0 2023-10-16T09:49:05.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.160+0000 7ffb959b9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb90156350 0x7ffb900b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:05.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.160+0000 7ffb959b9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb90156350 0x7ffb900b40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52898/0 (socket says 172.21.15.84:52898) 2023-10-16T09:49:05.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.160+0000 7ffb959b9640 1 -- 172.21.15.84:0/1015611827 learned_addr learned my addr 172.21.15.84:0/1015611827 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:05.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.161+0000 7ffb949b7640 1 --2- 172.21.15.84:0/1015611827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb900c9bc0 0x7ffb900b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:05.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.161+0000 7ffb951b8640 1 --2- 172.21.15.84:0/1015611827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb780039a0 0x7ffb900ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:05.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.161+0000 7ffb959b9640 1 -- 172.21.15.84:0/1015611827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb900c9bc0 msgr2=0x7ffb900b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:05.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.161+0000 7ffb959b9640 1 --2- 172.21.15.84:0/1015611827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb900c9bc0 0x7ffb900b3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:05.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.161+0000 7ffb959b9640 1 -- 172.21.15.84:0/1015611827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb780039a0 msgr2=0x7ffb900ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:05.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.161+0000 7ffb959b9640 1 --2- 172.21.15.84:0/1015611827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb780039a0 0x7ffb900ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:05.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.161+0000 7ffb959b9640 1 -- 172.21.15.84:0/1015611827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb900cb040 con 0x7ffb90156350 2023-10-16T09:49:05.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.161+0000 7ffb959b9640 1 --2- 172.21.15.84:0/1015611827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb90156350 0x7ffb900b40b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffb90066a80 tx=0x7ffb8800d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:05.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.161+0000 7ffb8dffb640 1 -- 172.21.15.84:0/1015611827 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8800b070 con 0x7ffb90156350 2023-10-16T09:49:05.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.161+0000 7ffb8dffb640 1 -- 172.21.15.84:0/1015611827 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb880054a0 con 0x7ffb90156350 2023-10-16T09:49:05.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.161+0000 7ffb961ba640 1 -- 172.21.15.84:0/1015611827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb900cad60 con 0x7ffb90156350 2023-10-16T09:49:05.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.161+0000 7ffb8dffb640 1 -- 172.21.15.84:0/1015611827 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb88007270 con 0x7ffb90156350 2023-10-16T09:49:05.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.162+0000 7ffb961ba640 1 -- 172.21.15.84:0/1015611827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb90156890 con 0x7ffb90156350 2023-10-16T09:49:05.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.163+0000 7ffb961ba640 1 -- 172.21.15.84:0/1015611827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb90157c90 con 0x7ffb90156350 2023-10-16T09:49:05.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.164+0000 7ffb8dffb640 1 -- 172.21.15.84:0/1015611827 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ffb880073d0 con 0x7ffb90156350 2023-10-16T09:49:05.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.165+0000 7ffb8dffb640 1 --2- 172.21.15.84:0/1015611827 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffb7c06aea0 0x7ffb7c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:05.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.165+0000 7ffb8dffb640 1 -- 172.21.15.84:0/1015611827 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb88089160 con 0x7ffb90156350 2023-10-16T09:49:05.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.165+0000 7ffb951b8640 1 --2- 172.21.15.84:0/1015611827 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffb7c06aea0 0x7ffb7c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:05.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.166+0000 7ffb951b8640 1 --2- 172.21.15.84:0/1015611827 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffb7c06aea0 0x7ffb7c06d360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffb80036000 tx=0x7ffb8002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:05.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.169+0000 7ffb8dffb640 1 -- 172.21.15.84:0/1015611827 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffb90157c90 con 0x7ffb90156350 2023-10-16T09:49:05.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.401+0000 7ffb961ba640 1 -- 172.21.15.84:0/1015611827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ffb90157c90 con 0x7ffb90156350 2023-10-16T09:49:05.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.921+0000 7ffb8dffb640 1 -- 172.21.15.84:0/1015611827 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7ffb90157c90 con 0x7ffb90156350 2023-10-16T09:49:05.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.926+0000 7ffb961ba640 1 -- 172.21.15.84:0/1015611827 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffb7c06aea0 msgr2=0x7ffb7c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:05.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.926+0000 7ffb961ba640 1 --2- 172.21.15.84:0/1015611827 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffb7c06aea0 0x7ffb7c06d360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffb80036000 tx=0x7ffb8002e000 comp rx=0 tx=0).stop 2023-10-16T09:49:05.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.926+0000 7ffb961ba640 1 -- 172.21.15.84:0/1015611827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb90156350 msgr2=0x7ffb900b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:05.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.926+0000 7ffb961ba640 1 --2- 172.21.15.84:0/1015611827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb90156350 0x7ffb900b40b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffb90066a80 tx=0x7ffb8800d890 comp rx=0 tx=0).stop 2023-10-16T09:49:05.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.926+0000 7ffb961ba640 1 -- 172.21.15.84:0/1015611827 shutdown_connections 2023-10-16T09:49:05.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.926+0000 7ffb961ba640 1 --2- 172.21.15.84:0/1015611827 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffb7c06aea0 0x7ffb7c06d360 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:05.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.926+0000 7ffb961ba640 1 --2- 172.21.15.84:0/1015611827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb90156350 0x7ffb900b40b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:05.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.926+0000 7ffb961ba640 1 --2- 172.21.15.84:0/1015611827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb900c9bc0 0x7ffb900b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:05.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.926+0000 7ffb961ba640 1 --2- 172.21.15.84:0/1015611827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb780039a0 0x7ffb900ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:05.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.926+0000 7ffb961ba640 1 -- 172.21.15.84:0/1015611827 >> 172.21.15.84:0/1015611827 conn(0x7ffb900cf570 msgr2=0x7ffb900b7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:05.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.926+0000 7ffb961ba640 1 -- 172.21.15.84:0/1015611827 shutdown_connections 2023-10-16T09:49:05.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:05.926+0000 7ffb961ba640 1 -- 172.21.15.84:0/1015611827 wait complete. 2023-10-16T09:49:05.948 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T09:49:05.948 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:49:06.043 INFO:tasks.ceph.mds.b.smithi157.stderr:2023-10-16T09:49:06.041+0000 7f43c3384640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T09:49:06.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.161+0000 7f6540893640 1 Processor -- start 2023-10-16T09:49:06.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.161+0000 7f6540893640 1 -- start start 2023-10-16T09:49:06.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.162+0000 7f6540893640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f653c159950 0x7f653c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:06.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.162+0000 7f6540893640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f653c070450 0x7f653c0708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:06.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.162+0000 7f6540893640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c06c100 0x7f653c06c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:06.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.162+0000 7f6540893640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f653c070e20 con 0x7f653c070450 2023-10-16T09:49:06.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.162+0000 7f6540893640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f653c15e490 con 0x7f653c159950 2023-10-16T09:49:06.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.162+0000 7f6540893640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653c15aef0 con 0x7f653c06c100 2023-10-16T09:49:06.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.162+0000 7f6533fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f653c070450 0x7f653c0708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:06.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.162+0000 7f6533fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f653c070450 0x7f653c0708e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59398/0 (socket says 172.21.15.84:59398) 2023-10-16T09:49:06.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.162+0000 7f6533fff640 1 -- 172.21.15.84:0/2514691489 learned_addr learned my addr 172.21.15.84:0/2514691489 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:06.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.163+0000 7f653b59e640 1 --2- 172.21.15.84:0/2514691489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c06c100 0x7f653c06c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:06.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.163+0000 7f653ad9d640 1 --2- 172.21.15.84:0/2514691489 >> v2:172.21.15.157:3300/0 conn(0x7f653c159950 0x7f653c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:06.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.163+0000 7f6533fff640 1 -- 172.21.15.84:0/2514691489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c06c100 msgr2=0x7f653c06c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:06.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.163+0000 7f6533fff640 1 --2- 172.21.15.84:0/2514691489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c06c100 0x7f653c06c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.163+0000 7f6533fff640 1 -- 172.21.15.84:0/2514691489 >> v2:172.21.15.157:3300/0 conn(0x7f653c159950 msgr2=0x7f653c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:06.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.163+0000 7f6533fff640 1 --2- 172.21.15.84:0/2514691489 >> v2:172.21.15.157:3300/0 conn(0x7f653c159950 0x7f653c15a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.163+0000 7f6533fff640 1 -- 172.21.15.84:0/2514691489 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c15d910 con 0x7f653c070450 2023-10-16T09:49:06.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.164+0000 7f6533fff640 1 --2- 172.21.15.84:0/2514691489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f653c070450 0x7f653c0708e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f653c0ca950 tx=0x7f652402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692cf9676aa1c28c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:06.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.164+0000 7f653a59c640 1 -- 172.21.15.84:0/2514691489 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65240317c0 con 0x7f653c070450 2023-10-16T09:49:06.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.164+0000 7f653a59c640 1 -- 172.21.15.84:0/2514691489 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6524031920 con 0x7f653c070450 2023-10-16T09:49:06.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.164+0000 7f653a59c640 1 -- 172.21.15.84:0/2514691489 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6524031cb0 con 0x7f653c070450 2023-10-16T09:49:06.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.165+0000 7f6540893640 1 -- 172.21.15.84:0/2514691489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f653c070450 msgr2=0x7f653c0708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:06.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.165+0000 7f6540893640 1 --2- 172.21.15.84:0/2514691489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f653c070450 0x7f653c0708e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f653c0ca950 tx=0x7f652402b860 comp rx=0 tx=0).stop 2023-10-16T09:49:06.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.165+0000 7f6540893640 1 -- 172.21.15.84:0/2514691489 shutdown_connections 2023-10-16T09:49:06.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.165+0000 7f6540893640 1 --2- 172.21.15.84:0/2514691489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f653c070450 0x7f653c0708e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.165+0000 7f6540893640 1 --2- 172.21.15.84:0/2514691489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c06c100 0x7f653c06c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.165+0000 7f6540893640 1 --2- 172.21.15.84:0/2514691489 >> v2:172.21.15.157:3300/0 conn(0x7f653c159950 0x7f653c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.165+0000 7f6540893640 1 -- 172.21.15.84:0/2514691489 >> 172.21.15.84:0/2514691489 conn(0x7f653c0ce6a0 msgr2=0x7f653c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:06.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.165+0000 7f6540893640 1 -- 172.21.15.84:0/2514691489 shutdown_connections 2023-10-16T09:49:06.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.165+0000 7f6540893640 1 -- 172.21.15.84:0/2514691489 wait complete. 2023-10-16T09:49:06.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.167+0000 7f6540893640 1 Processor -- start 2023-10-16T09:49:06.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.167+0000 7f6540893640 1 -- start start 2023-10-16T09:49:06.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.168+0000 7f6540893640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f653c06c100 0x7f653c0c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:06.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.168+0000 7f6540893640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f653c070450 0x7f653c0ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:06.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.168+0000 7f6540893640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c159950 0x7f653c0cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:06.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.168+0000 7f6540893640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c0c6f80 con 0x7f653c06c100 2023-10-16T09:49:06.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.168+0000 7f6540893640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c0c6130 con 0x7f653c070450 2023-10-16T09:49:06.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.168+0000 7f6540893640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653c0c6410 con 0x7f653c159950 2023-10-16T09:49:06.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.168+0000 7f653ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f653c06c100 0x7f653c0c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:06.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.168+0000 7f653ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f653c06c100 0x7f653c0c5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59400/0 (socket says 172.21.15.84:59400) 2023-10-16T09:49:06.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.168+0000 7f653ad9d640 1 -- 172.21.15.84:0/1074035794 learned_addr learned my addr 172.21.15.84:0/1074035794 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:06.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.168+0000 7f6533fff640 1 --2- 172.21.15.84:0/1074035794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f653c070450 0x7f653c0ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:06.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.168+0000 7f653ad9d640 1 -- 172.21.15.84:0/1074035794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c159950 msgr2=0x7f653c0cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:06.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.168+0000 7f653ad9d640 1 --2- 172.21.15.84:0/1074035794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c159950 0x7f653c0cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.168+0000 7f653ad9d640 1 -- 172.21.15.84:0/1074035794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f653c070450 msgr2=0x7f653c0ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:06.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.168+0000 7f653ad9d640 1 --2- 172.21.15.84:0/1074035794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f653c070450 0x7f653c0ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.168+0000 7f653ad9d640 1 -- 172.21.15.84:0/1074035794 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65240094d0 con 0x7f653c06c100 2023-10-16T09:49:06.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.169+0000 7f653ad9d640 1 --2- 172.21.15.84:0/1074035794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f653c06c100 0x7f653c0c5770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f652c00c7f0 tx=0x7f652c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:06.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.169+0000 7f65337fe640 1 -- 172.21.15.84:0/1074035794 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c004220 con 0x7f653c06c100 2023-10-16T09:49:06.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.169+0000 7f65337fe640 1 -- 172.21.15.84:0/1074035794 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652c0043a0 con 0x7f653c06c100 2023-10-16T09:49:06.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.169+0000 7f65337fe640 1 -- 172.21.15.84:0/1074035794 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c007860 con 0x7f653c06c100 2023-10-16T09:49:06.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.169+0000 7f6540893640 1 -- 172.21.15.84:0/1074035794 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653c0c66f0 con 0x7f653c06c100 2023-10-16T09:49:06.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.169+0000 7f6540893640 1 -- 172.21.15.84:0/1074035794 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653c0c4770 con 0x7f653c06c100 2023-10-16T09:49:06.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.170+0000 7f6540893640 1 -- 172.21.15.84:0/1074035794 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f653c0c4960 con 0x7f653c06c100 2023-10-16T09:49:06.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.171+0000 7f65337fe640 1 -- 172.21.15.84:0/1074035794 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f652c007bd0 con 0x7f653c06c100 2023-10-16T09:49:06.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.172+0000 7f65337fe640 1 --2- 172.21.15.84:0/1074035794 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f651c06ae70 0x7f651c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:06.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.172+0000 7f65337fe640 1 -- 172.21.15.84:0/1074035794 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f652c089550 con 0x7f653c06c100 2023-10-16T09:49:06.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.172+0000 7f6533fff640 1 --2- 172.21.15.84:0/1074035794 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f651c06ae70 0x7f651c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:06.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.173+0000 7f6533fff640 1 --2- 172.21.15.84:0/1074035794 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f651c06ae70 0x7f651c06d330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6524036000 tx=0x7f652403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:06.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.176+0000 7f65337fe640 1 -- 172.21.15.84:0/1074035794 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f653c0c4960 con 0x7f653c06c100 2023-10-16T09:49:06.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.401+0000 7f6540893640 1 -- 172.21.15.84:0/1074035794 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f653c0c4960 con 0x7f653c06c100 2023-10-16T09:49:06.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.402+0000 7f65337fe640 1 -- 172.21.15.84:0/1074035794 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f653c0c4960 con 0x7f653c06c100 2023-10-16T09:49:06.403 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:06.403 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2978888131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2978888131},{"type":"v1","addr":"172.21.15.157:6834","nonce":2978888131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3716581310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3716581310},{"type":"v1","addr":"172.21.15.84:6835","nonce":3716581310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2761479872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2761479872},{"type":"v1","addr":"172.21.15.84:6837","nonce":2761479872}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:49:02.350624+0000","modified":"2023-10-16T09:49:05.402785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/848754825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":848754825},{"type":"v1","addr":"172.21.15.157:6835","nonce":848754825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T09:49:06.404 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-10-16T09:49:06.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.404+0000 7f6540893640 1 -- 172.21.15.84:0/1074035794 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f651c06ae70 msgr2=0x7f651c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:06.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.404+0000 7f6540893640 1 --2- 172.21.15.84:0/1074035794 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f651c06ae70 0x7f651c06d330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6524036000 tx=0x7f652403d040 comp rx=0 tx=0).stop 2023-10-16T09:49:06.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.404+0000 7f6540893640 1 -- 172.21.15.84:0/1074035794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f653c06c100 msgr2=0x7f653c0c5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:06.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.404+0000 7f6540893640 1 --2- 172.21.15.84:0/1074035794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f653c06c100 0x7f653c0c5770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f652c00c7f0 tx=0x7f652c00ccb0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.404+0000 7f6540893640 1 -- 172.21.15.84:0/1074035794 shutdown_connections 2023-10-16T09:49:06.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.404+0000 7f6540893640 1 --2- 172.21.15.84:0/1074035794 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f651c06ae70 0x7f651c06d330 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.404+0000 7f6540893640 1 --2- 172.21.15.84:0/1074035794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c159950 0x7f653c0cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.404+0000 7f6540893640 1 --2- 172.21.15.84:0/1074035794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f653c070450 0x7f653c0ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.404+0000 7f6540893640 1 --2- 172.21.15.84:0/1074035794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f653c06c100 0x7f653c0c5770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.404+0000 7f6540893640 1 -- 172.21.15.84:0/1074035794 >> 172.21.15.84:0/1074035794 conn(0x7f653c0ce6a0 msgr2=0x7f653c0c94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:06.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.405+0000 7f6540893640 1 -- 172.21.15.84:0/1074035794 shutdown_connections 2023-10-16T09:49:06.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.405+0000 7f6540893640 1 -- 172.21.15.84:0/1074035794 wait complete. 2023-10-16T09:49:06.422 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:06.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.625+0000 7fbe709a4640 1 Processor -- start 2023-10-16T09:49:06.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.625+0000 7fbe709a4640 1 -- start start 2023-10-16T09:49:06.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.627+0000 7fbe709a4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbe6c06c040 0x7fbe6c06c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:06.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.627+0000 7fbe709a4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe6c0ad970 0x7fbe6c15e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:06.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.627+0000 7fbe709a4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe6c06c960 con 0x7fbe6c15ebc0 2023-10-16T09:49:06.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.627+0000 7fbe709a4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe6c15b9d0 con 0x7fbe6c06c040 2023-10-16T09:49:06.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.627+0000 7fbe709a4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c15bcb0 con 0x7fbe6c0ad970 2023-10-16T09:49:06.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.627+0000 7fbe63fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe6c0ad970 0x7fbe6c15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:06.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.627+0000 7fbe63fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe6c0ad970 0x7fbe6c15e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:36544/0 (socket says 172.21.15.84:36544) 2023-10-16T09:49:06.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.627+0000 7fbe63fff640 1 -- 172.21.15.84:0/3977599280 learned_addr learned my addr 172.21.15.84:0/3977599280 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:06.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.628+0000 7fbe6ad9d640 1 --2- 172.21.15.84:0/3977599280 >> v2:172.21.15.157:3300/0 conn(0x7fbe6c06c040 0x7fbe6c06c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:06.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.628+0000 7fbe63fff640 1 -- 172.21.15.84:0/3977599280 >> v2:172.21.15.157:3300/0 conn(0x7fbe6c06c040 msgr2=0x7fbe6c06c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:06.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.628+0000 7fbe63fff640 1 --2- 172.21.15.84:0/3977599280 >> v2:172.21.15.157:3300/0 conn(0x7fbe6c06c040 0x7fbe6c06c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.628+0000 7fbe63fff640 1 -- 172.21.15.84:0/3977599280 >> v1:172.21.15.157:6789/0 conn(0x7fbe6c15ebc0 legacy=0x7fbe6c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:06.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.628+0000 7fbe6a59c640 1 -- 172.21.15.84:0/3977599280 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995712761 0 0) 0x7fbe6c06c960 con 0x7fbe6c15ebc0 2023-10-16T09:49:06.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.628+0000 7fbe63fff640 1 -- 172.21.15.84:0/3977599280 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c15bf90 con 0x7fbe6c0ad970 2023-10-16T09:49:06.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.630+0000 7fbe63fff640 1 --2- 172.21.15.84:0/3977599280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe6c0ad970 0x7fbe6c15e680 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbe540098a0 tx=0x7fbe5402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fde570494ccbe851 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:06.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.630+0000 7fbe6a59c640 1 -- 172.21.15.84:0/3977599280 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe54030840 con 0x7fbe6c0ad970 2023-10-16T09:49:06.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.630+0000 7fbe6a59c640 1 -- 172.21.15.84:0/3977599280 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe540309a0 con 0x7fbe6c0ad970 2023-10-16T09:49:06.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.630+0000 7fbe6a59c640 1 -- 172.21.15.84:0/3977599280 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe54030d30 con 0x7fbe6c0ad970 2023-10-16T09:49:06.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.631+0000 7fbe709a4640 1 -- 172.21.15.84:0/3977599280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe6c0ad970 msgr2=0x7fbe6c15e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:06.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.631+0000 7fbe709a4640 1 --2- 172.21.15.84:0/3977599280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe6c0ad970 0x7fbe6c15e680 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbe540098a0 tx=0x7fbe5402b980 comp rx=0 tx=0).stop 2023-10-16T09:49:06.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.631+0000 7fbe709a4640 1 -- 172.21.15.84:0/3977599280 shutdown_connections 2023-10-16T09:49:06.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.631+0000 7fbe709a4640 1 --2- 172.21.15.84:0/3977599280 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe6c0ad970 0x7fbe6c15e680 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.631+0000 7fbe709a4640 1 --2- 172.21.15.84:0/3977599280 >> v2:172.21.15.157:3300/0 conn(0x7fbe6c06c040 0x7fbe6c06c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.631+0000 7fbe709a4640 1 -- 172.21.15.84:0/3977599280 >> 172.21.15.84:0/3977599280 conn(0x7fbe6c0ce6a0 msgr2=0x7fbe6c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:06.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.631+0000 7fbe709a4640 1 -- 172.21.15.84:0/3977599280 shutdown_connections 2023-10-16T09:49:06.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.631+0000 7fbe709a4640 1 -- 172.21.15.84:0/3977599280 wait complete. 2023-10-16T09:49:06.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.633+0000 7fbe709a4640 1 Processor -- start 2023-10-16T09:49:06.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.633+0000 7fbe709a4640 1 -- start start 2023-10-16T09:49:06.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.634+0000 7fbe709a4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe6c06c040 0x7fbe6c153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:06.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.634+0000 7fbe709a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c0ad970 0x7fbe6c0c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:06.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.634+0000 7fbe709a4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe6c15ebc0 0x7fbe6c15d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:06.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.634+0000 7fbe709a4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c0c9730 con 0x7fbe6c0ad970 2023-10-16T09:49:06.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.634+0000 7fbe709a4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c153810 con 0x7fbe6c06c040 2023-10-16T09:49:06.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.634+0000 7fbe709a4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c153530 con 0x7fbe6c15ebc0 2023-10-16T09:49:06.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.634+0000 7fbe63fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c0ad970 0x7fbe6c0c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:06.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.634+0000 7fbe63fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c0ad970 0x7fbe6c0c9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59412/0 (socket says 172.21.15.84:59412) 2023-10-16T09:49:06.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.634+0000 7fbe63fff640 1 -- 172.21.15.84:0/414576772 learned_addr learned my addr 172.21.15.84:0/414576772 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:06.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.634+0000 7fbe6ad9d640 1 --2- 172.21.15.84:0/414576772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe6c06c040 0x7fbe6c153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:06.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.634+0000 7fbe6b59e640 1 --2- 172.21.15.84:0/414576772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe6c15ebc0 0x7fbe6c15d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:06.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.634+0000 7fbe63fff640 1 -- 172.21.15.84:0/414576772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe6c15ebc0 msgr2=0x7fbe6c15d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:06.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.634+0000 7fbe63fff640 1 --2- 172.21.15.84:0/414576772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe6c15ebc0 0x7fbe6c15d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.634+0000 7fbe63fff640 1 -- 172.21.15.84:0/414576772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe6c06c040 msgr2=0x7fbe6c153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:06.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.634+0000 7fbe63fff640 1 --2- 172.21.15.84:0/414576772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe6c06c040 0x7fbe6c153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.634+0000 7fbe63fff640 1 -- 172.21.15.84:0/414576772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe540094d0 con 0x7fbe6c0ad970 2023-10-16T09:49:06.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.634+0000 7fbe6ad9d640 1 --2- 172.21.15.84:0/414576772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe6c06c040 0x7fbe6c153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:06.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.635+0000 7fbe63fff640 1 --2- 172.21.15.84:0/414576772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c0ad970 0x7fbe6c0c9b50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbe54000c00 tx=0x7fbe54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:06.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.635+0000 7fbe637fe640 1 -- 172.21.15.84:0/414576772 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe54039a90 con 0x7fbe6c0ad970 2023-10-16T09:49:06.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.635+0000 7fbe637fe640 1 -- 172.21.15.84:0/414576772 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe54039bf0 con 0x7fbe6c0ad970 2023-10-16T09:49:06.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.635+0000 7fbe637fe640 1 -- 172.21.15.84:0/414576772 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe54043410 con 0x7fbe6c0ad970 2023-10-16T09:49:06.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.635+0000 7fbe709a4640 1 -- 172.21.15.84:0/414576772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe6c0c8d90 con 0x7fbe6c0ad970 2023-10-16T09:49:06.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.635+0000 7fbe709a4640 1 -- 172.21.15.84:0/414576772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6c0c9350 con 0x7fbe6c0ad970 2023-10-16T09:49:06.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.636+0000 7fbe709a4640 1 -- 172.21.15.84:0/414576772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6c06b330 con 0x7fbe6c0ad970 2023-10-16T09:49:06.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.637+0000 7fbe637fe640 1 -- 172.21.15.84:0/414576772 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbe5402f390 con 0x7fbe6c0ad970 2023-10-16T09:49:06.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.638+0000 7fbe637fe640 1 --2- 172.21.15.84:0/414576772 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbe4c06ae70 0x7fbe4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:06.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.638+0000 7fbe637fe640 1 -- 172.21.15.84:0/414576772 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe540ab810 con 0x7fbe6c0ad970 2023-10-16T09:49:06.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.638+0000 7fbe6ad9d640 1 --2- 172.21.15.84:0/414576772 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbe4c06ae70 0x7fbe4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:06.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.639+0000 7fbe6ad9d640 1 --2- 172.21.15.84:0/414576772 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbe4c06ae70 0x7fbe4c06d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c006fd0 tx=0x7fbe5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:06.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.642+0000 7fbe637fe640 1 -- 172.21.15.84:0/414576772 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbe540768c0 con 0x7fbe6c0ad970 2023-10-16T09:49:06.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.821+0000 7fbe709a4640 1 -- 172.21.15.84:0/414576772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe6c0c9870 con 0x7fbe6c0ad970 2023-10-16T09:49:06.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.823+0000 7fbe637fe640 1 -- 172.21.15.84:0/414576772 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fbe54079ee0 con 0x7fbe6c0ad970 2023-10-16T09:49:06.825 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:06.825 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":21,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:03.529261+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:06.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.825+0000 7fbe709a4640 1 -- 172.21.15.84:0/414576772 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbe4c06ae70 msgr2=0x7fbe4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:06.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.825+0000 7fbe709a4640 1 --2- 172.21.15.84:0/414576772 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbe4c06ae70 0x7fbe4c06d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c006fd0 tx=0x7fbe5c008040 comp rx=0 tx=0).stop 2023-10-16T09:49:06.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.825+0000 7fbe709a4640 1 -- 172.21.15.84:0/414576772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c0ad970 msgr2=0x7fbe6c0c9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:06.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.825+0000 7fbe709a4640 1 --2- 172.21.15.84:0/414576772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c0ad970 0x7fbe6c0c9b50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbe54000c00 tx=0x7fbe54004730 comp rx=0 tx=0).stop 2023-10-16T09:49:06.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.826+0000 7fbe709a4640 1 -- 172.21.15.84:0/414576772 shutdown_connections 2023-10-16T09:49:06.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.826+0000 7fbe709a4640 1 --2- 172.21.15.84:0/414576772 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbe4c06ae70 0x7fbe4c06d330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.826+0000 7fbe709a4640 1 --2- 172.21.15.84:0/414576772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe6c15ebc0 0x7fbe6c15d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.826+0000 7fbe709a4640 1 --2- 172.21.15.84:0/414576772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c0ad970 0x7fbe6c0c9b50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.826+0000 7fbe709a4640 1 --2- 172.21.15.84:0/414576772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe6c06c040 0x7fbe6c153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:06.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.826+0000 7fbe709a4640 1 -- 172.21.15.84:0/414576772 >> 172.21.15.84:0/414576772 conn(0x7fbe6c0ce6a0 msgr2=0x7fbe6c0cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:06.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.826+0000 7fbe709a4640 1 -- 172.21.15.84:0/414576772 shutdown_connections 2023-10-16T09:49:06.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:06.826+0000 7fbe709a4640 1 -- 172.21.15.84:0/414576772 wait complete. 2023-10-16T09:49:06.846 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T09:49:06.847 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:49:07.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.056+0000 7f594bd0f640 1 Processor -- start 2023-10-16T09:49:07.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.057+0000 7f594bd0f640 1 -- start start 2023-10-16T09:49:07.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.058+0000 7f594bd0f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59440cfc10 0x7f59440d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:07.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.058+0000 7f594bd0f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f59440b3e70 0x7f59440d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:07.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.058+0000 7f594bd0f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59440b3940 con 0x7f59440b3070 2023-10-16T09:49:07.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.058+0000 7f594bd0f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59440b45e0 con 0x7f59440b3e70 2023-10-16T09:49:07.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.058+0000 7f594bd0f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59440c7880 con 0x7f59440cfc10 2023-10-16T09:49:07.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.058+0000 7f5949aab640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f59440b3070 0x7f59440cf500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60996/0 (socket says 172.21.15.84:60996) 2023-10-16T09:49:07.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.058+0000 7f5949aab640 1 -- 172.21.15.84:0/667040036 learned_addr learned my addr 172.21.15.84:0/667040036 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:07.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.058+0000 7f594a2ac640 1 --2- 172.21.15.84:0/667040036 >> v2:172.21.15.84:3300/0 conn(0x7f59440b3e70 0x7f59440d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:07.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.059+0000 7f59492aa640 1 --2- 172.21.15.84:0/667040036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59440cfc10 0x7f59440d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:07.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.059+0000 7f594a2ac640 1 -- 172.21.15.84:0/667040036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59440cfc10 msgr2=0x7f59440d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:07.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.059+0000 7f594a2ac640 1 --2- 172.21.15.84:0/667040036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59440cfc10 0x7f59440d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.059+0000 7f594a2ac640 1 -- 172.21.15.84:0/667040036 >> v1:172.21.15.84:6789/0 conn(0x7f59440b3070 legacy=0x7f59440cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:07.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.059+0000 7f594a2ac640 1 -- 172.21.15.84:0/667040036 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59440b3b90 con 0x7f59440b3e70 2023-10-16T09:49:07.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.059+0000 7f5948aa9640 1 -- 172.21.15.84:0/667040036 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276706616 0 0) 0x7f59440b3940 con 0x7f59440b3070 2023-10-16T09:49:07.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.060+0000 7f594a2ac640 1 --2- 172.21.15.84:0/667040036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59440b3e70 0x7f59440d1040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f593c0097a0 tx=0x7f593c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4952c3610be86ed9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:07.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.060+0000 7f5948aa9640 1 -- 172.21.15.84:0/667040036 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c031830 con 0x7f59440b3e70 2023-10-16T09:49:07.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.060+0000 7f5948aa9640 1 -- 172.21.15.84:0/667040036 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c031990 con 0x7f59440b3e70 2023-10-16T09:49:07.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.060+0000 7f5948aa9640 1 -- 172.21.15.84:0/667040036 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c031d20 con 0x7f59440b3e70 2023-10-16T09:49:07.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.061+0000 7f594bd0f640 1 -- 172.21.15.84:0/667040036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59440b3e70 msgr2=0x7f59440d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:07.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.061+0000 7f594bd0f640 1 --2- 172.21.15.84:0/667040036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59440b3e70 0x7f59440d1040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f593c0097a0 tx=0x7f593c02b8d0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.061+0000 7f594bd0f640 1 -- 172.21.15.84:0/667040036 shutdown_connections 2023-10-16T09:49:07.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.061+0000 7f594bd0f640 1 --2- 172.21.15.84:0/667040036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59440b3e70 0x7f59440d1040 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.061+0000 7f594bd0f640 1 --2- 172.21.15.84:0/667040036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59440cfc10 0x7f59440d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.061+0000 7f594bd0f640 1 -- 172.21.15.84:0/667040036 >> 172.21.15.84:0/667040036 conn(0x7f59440d2d20 msgr2=0x7f59440d3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:07.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.061+0000 7f594bd0f640 1 -- 172.21.15.84:0/667040036 shutdown_connections 2023-10-16T09:49:07.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.061+0000 7f594bd0f640 1 -- 172.21.15.84:0/667040036 wait complete. 2023-10-16T09:49:07.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.063+0000 7f594bd0f640 1 Processor -- start 2023-10-16T09:49:07.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.063+0000 7f594bd0f640 1 -- start start 2023-10-16T09:49:07.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.064+0000 7f594bd0f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59440b3070 0x7f594415e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:07.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.064+0000 7f594bd0f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59440b3e70 0x7f59440b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:07.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.064+0000 7f594bd0f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59440cfc10 0x7f59440b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:07.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.064+0000 7f594bd0f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59440cbda0 con 0x7f59440b3070 2023-10-16T09:49:07.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.064+0000 7f594bd0f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59440cbf90 con 0x7f59440b3e70 2023-10-16T09:49:07.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.064+0000 7f594bd0f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59440cc270 con 0x7f59440cfc10 2023-10-16T09:49:07.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.064+0000 7f5949aab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59440b3070 0x7f594415e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:07.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.064+0000 7f5949aab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59440b3070 0x7f594415e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59422/0 (socket says 172.21.15.84:59422) 2023-10-16T09:49:07.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.064+0000 7f5949aab640 1 -- 172.21.15.84:0/3833547436 learned_addr learned my addr 172.21.15.84:0/3833547436 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:07.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.064+0000 7f5949aab640 1 -- 172.21.15.84:0/3833547436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59440cfc10 msgr2=0x7f59440b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:07.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.064+0000 7f5949aab640 1 --2- 172.21.15.84:0/3833547436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59440cfc10 0x7f59440b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.064+0000 7f5949aab640 1 -- 172.21.15.84:0/3833547436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59440b3e70 msgr2=0x7f59440b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:07.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.064+0000 7f5949aab640 1 --2- 172.21.15.84:0/3833547436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59440b3e70 0x7f59440b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.064+0000 7f5949aab640 1 -- 172.21.15.84:0/3833547436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c009400 con 0x7f59440b3070 2023-10-16T09:49:07.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.065+0000 7f5949aab640 1 --2- 172.21.15.84:0/3833547436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59440b3070 0x7f594415e200 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5938001460 tx=0x7f593800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:07.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.065+0000 7f59427fc640 1 -- 172.21.15.84:0/3833547436 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593800a610 con 0x7f59440b3070 2023-10-16T09:49:07.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.065+0000 7f59427fc640 1 -- 172.21.15.84:0/3833547436 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593800a770 con 0x7f59440b3070 2023-10-16T09:49:07.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.065+0000 7f59427fc640 1 -- 172.21.15.84:0/3833547436 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593800aba0 con 0x7f59440b3070 2023-10-16T09:49:07.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.065+0000 7f594bd0f640 1 -- 172.21.15.84:0/3833547436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59440cc550 con 0x7f59440b3070 2023-10-16T09:49:07.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.065+0000 7f594bd0f640 1 -- 172.21.15.84:0/3833547436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59440ccb10 con 0x7f59440b3070 2023-10-16T09:49:07.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.066+0000 7f594bd0f640 1 -- 172.21.15.84:0/3833547436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59440ccd30 con 0x7f59440b3070 2023-10-16T09:49:07.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.067+0000 7f59427fc640 1 -- 172.21.15.84:0/3833547436 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f593800ad00 con 0x7f59440b3070 2023-10-16T09:49:07.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.068+0000 7f59427fc640 1 --2- 172.21.15.84:0/3833547436 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f592806ae70 0x7f592806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:07.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.068+0000 7f59427fc640 1 -- 172.21.15.84:0/3833547436 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5938088f90 con 0x7f59440b3070 2023-10-16T09:49:07.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.068+0000 7f59492aa640 1 --2- 172.21.15.84:0/3833547436 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f592806ae70 0x7f592806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:07.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.069+0000 7f59492aa640 1 --2- 172.21.15.84:0/3833547436 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f592806ae70 0x7f592806d330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5930006fd0 tx=0x7f5930008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:07.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.072+0000 7f59427fc640 1 -- 172.21.15.84:0/3833547436 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5938054030 con 0x7f59440b3070 2023-10-16T09:49:07.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.260+0000 7f594bd0f640 1 -- 172.21.15.84:0/3833547436 --> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f59440d3b20 con 0x7f592806ae70 2023-10-16T09:49:07.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.269+0000 7f59427fc640 1 -- 172.21.15.84:0/3833547436 <== mgr.4107 v2:172.21.15.84:6832/3010246666 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264514 (secure 0 0 0) 0x7f59300090c0 con 0x7f592806ae70 2023-10-16T09:49:07.274 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:07.276 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T09:49:07.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.274+0000 7f594bd0f640 1 -- 172.21.15.84:0/3833547436 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f592806ae70 msgr2=0x7f592806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:07.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.274+0000 7f594bd0f640 1 --2- 172.21.15.84:0/3833547436 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f592806ae70 0x7f592806d330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5930006fd0 tx=0x7f5930008040 comp rx=0 tx=0).stop 2023-10-16T09:49:07.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.274+0000 7f594bd0f640 1 -- 172.21.15.84:0/3833547436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59440b3070 msgr2=0x7f594415e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:07.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.274+0000 7f594bd0f640 1 --2- 172.21.15.84:0/3833547436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59440b3070 0x7f594415e200 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5938001460 tx=0x7f593800cd20 comp rx=0 tx=0).stop 2023-10-16T09:49:07.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.274+0000 7f594bd0f640 1 -- 172.21.15.84:0/3833547436 shutdown_connections 2023-10-16T09:49:07.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.274+0000 7f594bd0f640 1 --2- 172.21.15.84:0/3833547436 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f592806ae70 0x7f592806d330 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.274+0000 7f594bd0f640 1 --2- 172.21.15.84:0/3833547436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59440cfc10 0x7f59440b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.274+0000 7f594bd0f640 1 --2- 172.21.15.84:0/3833547436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59440b3e70 0x7f59440b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.274+0000 7f594bd0f640 1 --2- 172.21.15.84:0/3833547436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59440b3070 0x7f594415e200 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.274+0000 7f594bd0f640 1 -- 172.21.15.84:0/3833547436 >> 172.21.15.84:0/3833547436 conn(0x7f59440d2d20 msgr2=0x7f59440c7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:07.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.274+0000 7f594bd0f640 1 -- 172.21.15.84:0/3833547436 shutdown_connections 2023-10-16T09:49:07.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.274+0000 7f594bd0f640 1 -- 172.21.15.84:0/3833547436 wait complete. 2023-10-16T09:49:07.294 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-10-16T09:49:05.589367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236084,"kb_used_data":2908,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738636,"statfs":{"total":773094113280,"available":772852363264,"internally_reserved":0,"allocated":2977792,"data_stored":1783808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.717843"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942979+0000","last_change":"2023-10-16T09:49:02.569411+0000","last_active":"2023-10-16T09:49:03.942979+0000","last_peered":"2023-10-16T09:49:03.942979+0000","last_clean":"2023-10-16T09:49:03.942979+0000","last_became_active":"2023-10-16T09:49:02.569101+0000","last_became_peered":"2023-10-16T09:49:02.569101+0000","last_unstale":"2023-10-16T09:49:03.942979+0000","last_undegraded":"2023-10-16T09:49:03.942979+0000","last_fullsized":"2023-10-16T09:49:03.942979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:20:01.929361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937690+0000","last_change":"2023-10-16T09:49:02.570326+0000","last_active":"2023-10-16T09:49:03.937690+0000","last_peered":"2023-10-16T09:49:03.937690+0000","last_clean":"2023-10-16T09:49:03.937690+0000","last_became_active":"2023-10-16T09:49:02.569866+0000","last_became_peered":"2023-10-16T09:49:02.569866+0000","last_unstale":"2023-10-16T09:49:03.937690+0000","last_undegraded":"2023-10-16T09:49:03.937690+0000","last_fullsized":"2023-10-16T09:49:03.937690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:10.503481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940359+0000","last_change":"2023-10-16T09:49:02.566217+0000","last_active":"2023-10-16T09:49:03.940359+0000","last_peered":"2023-10-16T09:49:03.940359+0000","last_clean":"2023-10-16T09:49:03.940359+0000","last_became_active":"2023-10-16T09:49:02.565843+0000","last_became_peered":"2023-10-16T09:49:02.565843+0000","last_unstale":"2023-10-16T09:49:03.940359+0000","last_undegraded":"2023-10-16T09:49:03.940359+0000","last_fullsized":"2023-10-16T09:49:03.940359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540898+0000","last_change":"2023-10-16T09:49:02.580605+0000","last_active":"2023-10-16T09:49:03.540898+0000","last_peered":"2023-10-16T09:49:03.540898+0000","last_clean":"2023-10-16T09:49:03.540898+0000","last_became_active":"2023-10-16T09:49:02.573845+0000","last_became_peered":"2023-10-16T09:49:02.573845+0000","last_unstale":"2023-10-16T09:49:03.540898+0000","last_undegraded":"2023-10-16T09:49:03.540898+0000","last_fullsized":"2023-10-16T09:49:03.540898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:05.838911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941895+0000","last_change":"2023-10-16T09:49:02.563238+0000","last_active":"2023-10-16T09:49:03.941895+0000","last_peered":"2023-10-16T09:49:03.941895+0000","last_clean":"2023-10-16T09:49:03.941895+0000","last_became_active":"2023-10-16T09:49:02.562849+0000","last_became_peered":"2023-10-16T09:49:02.562849+0000","last_unstale":"2023-10-16T09:49:03.941895+0000","last_undegraded":"2023-10-16T09:49:03.941895+0000","last_fullsized":"2023-10-16T09:49:03.941895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:55.226843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540148+0000","last_change":"2023-10-16T09:49:02.572385+0000","last_active":"2023-10-16T09:49:03.540148+0000","last_peered":"2023-10-16T09:49:03.540148+0000","last_clean":"2023-10-16T09:49:03.540148+0000","last_became_active":"2023-10-16T09:49:02.571270+0000","last_became_peered":"2023-10-16T09:49:02.571270+0000","last_unstale":"2023-10-16T09:49:03.540148+0000","last_undegraded":"2023-10-16T09:49:03.540148+0000","last_fullsized":"2023-10-16T09:49:03.540148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941710+0000","last_change":"2023-10-16T09:49:02.570612+0000","last_active":"2023-10-16T09:49:03.941710+0000","last_peered":"2023-10-16T09:49:03.941710+0000","last_clean":"2023-10-16T09:49:03.941710+0000","last_became_active":"2023-10-16T09:49:02.570330+0000","last_became_peered":"2023-10-16T09:49:02.570330+0000","last_unstale":"2023-10-16T09:49:03.941710+0000","last_undegraded":"2023-10-16T09:49:03.941710+0000","last_fullsized":"2023-10-16T09:49:03.941710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:07.724831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540222+0000","last_change":"2023-10-16T09:49:02.572301+0000","last_active":"2023-10-16T09:49:03.540222+0000","last_peered":"2023-10-16T09:49:03.540222+0000","last_clean":"2023-10-16T09:49:03.540222+0000","last_became_active":"2023-10-16T09:49:02.571934+0000","last_became_peered":"2023-10-16T09:49:02.571934+0000","last_unstale":"2023-10-16T09:49:03.540222+0000","last_undegraded":"2023-10-16T09:49:03.540222+0000","last_fullsized":"2023-10-16T09:49:03.540222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:51.981410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.943019+0000","last_change":"2023-10-16T09:49:02.580905+0000","last_active":"2023-10-16T09:49:03.943019+0000","last_peered":"2023-10-16T09:49:03.943019+0000","last_clean":"2023-10-16T09:49:03.943019+0000","last_became_active":"2023-10-16T09:49:02.577694+0000","last_became_peered":"2023-10-16T09:49:02.577694+0000","last_unstale":"2023-10-16T09:49:03.943019+0000","last_undegraded":"2023-10-16T09:49:03.943019+0000","last_fullsized":"2023-10-16T09:49:03.943019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:02:14.799393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938295+0000","last_change":"2023-10-16T09:49:02.576191+0000","last_active":"2023-10-16T09:49:03.938295+0000","last_peered":"2023-10-16T09:49:03.938295+0000","last_clean":"2023-10-16T09:49:03.938295+0000","last_became_active":"2023-10-16T09:49:02.575810+0000","last_became_peered":"2023-10-16T09:49:02.575810+0000","last_unstale":"2023-10-16T09:49:03.938295+0000","last_undegraded":"2023-10-16T09:49:03.938295+0000","last_fullsized":"2023-10-16T09:49:03.938295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937913+0000","last_change":"2023-10-16T09:49:02.577871+0000","last_active":"2023-10-16T09:49:03.937913+0000","last_peered":"2023-10-16T09:49:03.937913+0000","last_clean":"2023-10-16T09:49:03.937913+0000","last_became_active":"2023-10-16T09:49:02.577580+0000","last_became_peered":"2023-10-16T09:49:02.577580+0000","last_unstale":"2023-10-16T09:49:03.937913+0000","last_undegraded":"2023-10-16T09:49:03.937913+0000","last_fullsized":"2023-10-16T09:49:03.937913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940615+0000","last_change":"2023-10-16T09:49:02.575626+0000","last_active":"2023-10-16T09:49:03.940615+0000","last_peered":"2023-10-16T09:49:03.940615+0000","last_clean":"2023-10-16T09:49:03.940615+0000","last_became_active":"2023-10-16T09:49:02.569588+0000","last_became_peered":"2023-10-16T09:49:02.569588+0000","last_unstale":"2023-10-16T09:49:03.940615+0000","last_undegraded":"2023-10-16T09:49:03.940615+0000","last_fullsized":"2023-10-16T09:49:03.940615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:28.559387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541264+0000","last_change":"2023-10-16T09:49:02.562597+0000","last_active":"2023-10-16T09:49:03.541264+0000","last_peered":"2023-10-16T09:49:03.541264+0000","last_clean":"2023-10-16T09:49:03.541264+0000","last_became_active":"2023-10-16T09:49:02.562322+0000","last_became_peered":"2023-10-16T09:49:02.562322+0000","last_unstale":"2023-10-16T09:49:03.541264+0000","last_undegraded":"2023-10-16T09:49:03.541264+0000","last_fullsized":"2023-10-16T09:49:03.541264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:51.170491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939007+0000","last_change":"2023-10-16T09:49:02.568210+0000","last_active":"2023-10-16T09:49:03.939007+0000","last_peered":"2023-10-16T09:49:03.939007+0000","last_clean":"2023-10-16T09:49:03.939007+0000","last_became_active":"2023-10-16T09:49:02.567330+0000","last_became_peered":"2023-10-16T09:49:02.567330+0000","last_unstale":"2023-10-16T09:49:03.939007+0000","last_undegraded":"2023-10-16T09:49:03.939007+0000","last_fullsized":"2023-10-16T09:49:03.939007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:17.275794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938071+0000","last_change":"2023-10-16T09:49:02.561360+0000","last_active":"2023-10-16T09:49:03.938071+0000","last_peered":"2023-10-16T09:49:03.938071+0000","last_clean":"2023-10-16T09:49:03.938071+0000","last_became_active":"2023-10-16T09:49:02.560831+0000","last_became_peered":"2023-10-16T09:49:02.560831+0000","last_unstale":"2023-10-16T09:49:03.938071+0000","last_undegraded":"2023-10-16T09:49:03.938071+0000","last_fullsized":"2023-10-16T09:49:03.938071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539917+0000","last_change":"2023-10-16T09:49:02.571717+0000","last_active":"2023-10-16T09:49:03.539917+0000","last_peered":"2023-10-16T09:49:03.539917+0000","last_clean":"2023-10-16T09:49:03.539917+0000","last_became_active":"2023-10-16T09:49:02.571311+0000","last_became_peered":"2023-10-16T09:49:02.571311+0000","last_unstale":"2023-10-16T09:49:03.539917+0000","last_undegraded":"2023-10-16T09:49:03.539917+0000","last_fullsized":"2023-10-16T09:49:03.539917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541271+0000","last_change":"2023-10-16T09:49:02.582038+0000","last_active":"2023-10-16T09:49:03.541271+0000","last_peered":"2023-10-16T09:49:03.541271+0000","last_clean":"2023-10-16T09:49:03.541271+0000","last_became_active":"2023-10-16T09:49:02.581780+0000","last_became_peered":"2023-10-16T09:49:02.581780+0000","last_unstale":"2023-10-16T09:49:03.541271+0000","last_undegraded":"2023-10-16T09:49:03.541271+0000","last_fullsized":"2023-10-16T09:49:03.541271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:45.572589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539221+0000","last_change":"2023-10-16T09:49:02.563686+0000","last_active":"2023-10-16T09:49:03.539221+0000","last_peered":"2023-10-16T09:49:03.539221+0000","last_clean":"2023-10-16T09:49:03.539221+0000","last_became_active":"2023-10-16T09:49:02.562728+0000","last_became_peered":"2023-10-16T09:49:02.562728+0000","last_unstale":"2023-10-16T09:49:03.539221+0000","last_undegraded":"2023-10-16T09:49:03.539221+0000","last_fullsized":"2023-10-16T09:49:03.539221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:58.154049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942765+0000","last_change":"2023-10-16T09:49:02.569238+0000","last_active":"2023-10-16T09:49:03.942765+0000","last_peered":"2023-10-16T09:49:03.942765+0000","last_clean":"2023-10-16T09:49:03.942765+0000","last_became_active":"2023-10-16T09:49:02.568292+0000","last_became_peered":"2023-10-16T09:49:02.568292+0000","last_unstale":"2023-10-16T09:49:03.942765+0000","last_undegraded":"2023-10-16T09:49:03.942765+0000","last_fullsized":"2023-10-16T09:49:03.942765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:46.937999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937917+0000","last_change":"2023-10-16T09:49:02.563047+0000","last_active":"2023-10-16T09:49:03.937917+0000","last_peered":"2023-10-16T09:49:03.937917+0000","last_clean":"2023-10-16T09:49:03.937917+0000","last_became_active":"2023-10-16T09:49:02.562532+0000","last_became_peered":"2023-10-16T09:49:02.562532+0000","last_unstale":"2023-10-16T09:49:03.937917+0000","last_undegraded":"2023-10-16T09:49:03.937917+0000","last_fullsized":"2023-10-16T09:49:03.937917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:33.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939622+0000","last_change":"2023-10-16T09:49:02.569454+0000","last_active":"2023-10-16T09:49:03.939622+0000","last_peered":"2023-10-16T09:49:03.939622+0000","last_clean":"2023-10-16T09:49:03.939622+0000","last_became_active":"2023-10-16T09:49:02.569216+0000","last_became_peered":"2023-10-16T09:49:02.569216+0000","last_unstale":"2023-10-16T09:49:03.939622+0000","last_undegraded":"2023-10-16T09:49:03.939622+0000","last_fullsized":"2023-10-16T09:49:03.939622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:47.112566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540091+0000","last_change":"2023-10-16T09:49:02.572101+0000","last_active":"2023-10-16T09:49:03.540091+0000","last_peered":"2023-10-16T09:49:03.540091+0000","last_clean":"2023-10-16T09:49:03.540091+0000","last_became_active":"2023-10-16T09:49:02.570939+0000","last_became_peered":"2023-10-16T09:49:02.570939+0000","last_unstale":"2023-10-16T09:49:03.540091+0000","last_undegraded":"2023-10-16T09:49:03.540091+0000","last_fullsized":"2023-10-16T09:49:03.540091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540377+0000","last_change":"2023-10-16T09:49:02.562443+0000","last_active":"2023-10-16T09:49:03.540377+0000","last_peered":"2023-10-16T09:49:03.540377+0000","last_clean":"2023-10-16T09:49:03.540377+0000","last_became_active":"2023-10-16T09:49:02.562184+0000","last_became_peered":"2023-10-16T09:49:02.562184+0000","last_unstale":"2023-10-16T09:49:03.540377+0000","last_undegraded":"2023-10-16T09:49:03.540377+0000","last_fullsized":"2023-10-16T09:49:03.540377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:47.112566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540214+0000","last_change":"2023-10-16T09:49:02.566157+0000","last_active":"2023-10-16T09:49:03.540214+0000","last_peered":"2023-10-16T09:49:03.540214+0000","last_clean":"2023-10-16T09:49:03.540214+0000","last_became_active":"2023-10-16T09:49:02.565808+0000","last_became_peered":"2023-10-16T09:49:02.565808+0000","last_unstale":"2023-10-16T09:49:03.540214+0000","last_undegraded":"2023-10-16T09:49:03.540214+0000","last_fullsized":"2023-10-16T09:49:03.540214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:51.170491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540739+0000","last_change":"2023-10-16T09:49:02.570442+0000","last_active":"2023-10-16T09:49:03.540739+0000","last_peered":"2023-10-16T09:49:03.540739+0000","last_clean":"2023-10-16T09:49:03.540739+0000","last_became_active":"2023-10-16T09:49:02.570173+0000","last_became_peered":"2023-10-16T09:49:02.570173+0000","last_unstale":"2023-10-16T09:49:03.540739+0000","last_undegraded":"2023-10-16T09:49:03.540739+0000","last_fullsized":"2023-10-16T09:49:03.540739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938295+0000","last_change":"2023-10-16T09:49:02.565763+0000","last_active":"2023-10-16T09:49:03.938295+0000","last_peered":"2023-10-16T09:49:03.938295+0000","last_clean":"2023-10-16T09:49:03.938295+0000","last_became_active":"2023-10-16T09:49:02.565248+0000","last_became_peered":"2023-10-16T09:49:02.565248+0000","last_unstale":"2023-10-16T09:49:03.938295+0000","last_undegraded":"2023-10-16T09:49:03.938295+0000","last_fullsized":"2023-10-16T09:49:03.938295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938715+0000","last_change":"2023-10-16T09:49:02.556364+0000","last_active":"2023-10-16T09:49:03.938715+0000","last_peered":"2023-10-16T09:49:03.938715+0000","last_clean":"2023-10-16T09:49:03.938715+0000","last_became_active":"2023-10-16T09:49:02.556055+0000","last_became_peered":"2023-10-16T09:49:02.556055+0000","last_unstale":"2023-10-16T09:49:03.938715+0000","last_undegraded":"2023-10-16T09:49:03.938715+0000","last_fullsized":"2023-10-16T09:49:03.938715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541726+0000","last_change":"2023-10-16T09:49:02.564976+0000","last_active":"2023-10-16T09:49:03.541726+0000","last_peered":"2023-10-16T09:49:03.541726+0000","last_clean":"2023-10-16T09:49:03.541726+0000","last_became_active":"2023-10-16T09:49:02.564798+0000","last_became_peered":"2023-10-16T09:49:02.564798+0000","last_unstale":"2023-10-16T09:49:03.541726+0000","last_undegraded":"2023-10-16T09:49:03.541726+0000","last_fullsized":"2023-10-16T09:49:03.541726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:39:22.668380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939780+0000","last_change":"2023-10-16T09:49:02.561691+0000","last_active":"2023-10-16T09:49:03.939780+0000","last_peered":"2023-10-16T09:49:03.939780+0000","last_clean":"2023-10-16T09:49:03.939780+0000","last_became_active":"2023-10-16T09:49:02.561408+0000","last_became_peered":"2023-10-16T09:49:02.561408+0000","last_unstale":"2023-10-16T09:49:03.939780+0000","last_undegraded":"2023-10-16T09:49:03.939780+0000","last_fullsized":"2023-10-16T09:49:03.939780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:10.503481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541186+0000","last_change":"2023-10-16T09:49:02.572702+0000","last_active":"2023-10-16T09:49:03.541186+0000","last_peered":"2023-10-16T09:49:03.541186+0000","last_clean":"2023-10-16T09:49:03.541186+0000","last_became_active":"2023-10-16T09:49:02.571811+0000","last_became_peered":"2023-10-16T09:49:02.571811+0000","last_unstale":"2023-10-16T09:49:03.541186+0000","last_undegraded":"2023-10-16T09:49:03.541186+0000","last_fullsized":"2023-10-16T09:49:03.541186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:08.814427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.538838+0000","last_change":"2023-10-16T09:49:02.576101+0000","last_active":"2023-10-16T09:49:03.538838+0000","last_peered":"2023-10-16T09:49:03.538838+0000","last_clean":"2023-10-16T09:49:03.538838+0000","last_became_active":"2023-10-16T09:49:02.575844+0000","last_became_peered":"2023-10-16T09:49:02.575844+0000","last_unstale":"2023-10-16T09:49:03.538838+0000","last_undegraded":"2023-10-16T09:49:03.538838+0000","last_fullsized":"2023-10-16T09:49:03.538838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:45.119851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937812+0000","last_change":"2023-10-16T09:49:02.568425+0000","last_active":"2023-10-16T09:49:03.937812+0000","last_peered":"2023-10-16T09:49:03.937812+0000","last_clean":"2023-10-16T09:49:03.937812+0000","last_became_active":"2023-10-16T09:49:02.568091+0000","last_became_peered":"2023-10-16T09:49:02.568091+0000","last_unstale":"2023-10-16T09:49:03.937812+0000","last_undegraded":"2023-10-16T09:49:03.937812+0000","last_fullsized":"2023-10-16T09:49:03.937812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:10.503481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940047+0000","last_change":"2023-10-16T09:49:02.559306+0000","last_active":"2023-10-16T09:49:03.940047+0000","last_peered":"2023-10-16T09:49:03.940047+0000","last_clean":"2023-10-16T09:49:03.940047+0000","last_became_active":"2023-10-16T09:49:02.558667+0000","last_became_peered":"2023-10-16T09:49:02.558667+0000","last_unstale":"2023-10-16T09:49:03.940047+0000","last_undegraded":"2023-10-16T09:49:03.940047+0000","last_fullsized":"2023-10-16T09:49:03.940047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:38:53.130875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541754+0000","last_change":"2023-10-16T09:49:01.918393+0000","last_active":"2023-10-16T09:49:03.541754+0000","last_peered":"2023-10-16T09:49:03.541754+0000","last_clean":"2023-10-16T09:49:03.541754+0000","last_became_active":"2023-10-16T09:49:01.918085+0000","last_became_peered":"2023-10-16T09:49:01.918085+0000","last_unstale":"2023-10-16T09:49:03.541754+0000","last_undegraded":"2023-10-16T09:49:03.541754+0000","last_fullsized":"2023-10-16T09:49:03.541754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:24.118554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939938+0000","last_change":"2023-10-16T09:49:02.562307+0000","last_active":"2023-10-16T09:49:03.939938+0000","last_peered":"2023-10-16T09:49:03.939938+0000","last_clean":"2023-10-16T09:49:03.939938+0000","last_became_active":"2023-10-16T09:49:02.562013+0000","last_became_peered":"2023-10-16T09:49:02.562013+0000","last_unstale":"2023-10-16T09:49:03.939938+0000","last_undegraded":"2023-10-16T09:49:03.939938+0000","last_fullsized":"2023-10-16T09:49:03.939938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937479+0000","last_change":"2023-10-16T09:49:01.931642+0000","last_active":"2023-10-16T09:49:03.937479+0000","last_peered":"2023-10-16T09:49:03.937479+0000","last_clean":"2023-10-16T09:49:03.937479+0000","last_became_active":"2023-10-16T09:49:01.929759+0000","last_became_peered":"2023-10-16T09:49:01.929759+0000","last_unstale":"2023-10-16T09:49:03.937479+0000","last_undegraded":"2023-10-16T09:49:03.937479+0000","last_fullsized":"2023-10-16T09:49:03.937479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:56.824003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540519+0000","last_change":"2023-10-16T09:49:01.925581+0000","last_active":"2023-10-16T09:49:03.540519+0000","last_peered":"2023-10-16T09:49:03.540519+0000","last_clean":"2023-10-16T09:49:03.540519+0000","last_became_active":"2023-10-16T09:49:01.925329+0000","last_became_peered":"2023-10-16T09:49:01.925329+0000","last_unstale":"2023-10-16T09:49:03.540519+0000","last_undegraded":"2023-10-16T09:49:03.540519+0000","last_fullsized":"2023-10-16T09:49:03.540519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:12:13.066628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540403+0000","last_change":"2023-10-16T09:49:02.566394+0000","last_active":"2023-10-16T09:49:03.540403+0000","last_peered":"2023-10-16T09:49:03.540403+0000","last_clean":"2023-10-16T09:49:03.540403+0000","last_became_active":"2023-10-16T09:49:02.566059+0000","last_became_peered":"2023-10-16T09:49:02.566059+0000","last_unstale":"2023-10-16T09:49:03.540403+0000","last_undegraded":"2023-10-16T09:49:03.540403+0000","last_fullsized":"2023-10-16T09:49:03.540403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:10.503481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540187+0000","last_change":"2023-10-16T09:49:01.933426+0000","last_active":"2023-10-16T09:49:03.540187+0000","last_peered":"2023-10-16T09:49:03.540187+0000","last_clean":"2023-10-16T09:49:03.540187+0000","last_became_active":"2023-10-16T09:49:01.930300+0000","last_became_peered":"2023-10-16T09:49:01.930300+0000","last_unstale":"2023-10-16T09:49:03.540187+0000","last_undegraded":"2023-10-16T09:49:03.540187+0000","last_fullsized":"2023-10-16T09:49:03.540187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:27.229341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937721+0000","last_change":"2023-10-16T09:49:02.555653+0000","last_active":"2023-10-16T09:49:03.937721+0000","last_peered":"2023-10-16T09:49:03.937721+0000","last_clean":"2023-10-16T09:49:03.937721+0000","last_became_active":"2023-10-16T09:49:02.555099+0000","last_became_peered":"2023-10-16T09:49:02.555099+0000","last_unstale":"2023-10-16T09:49:03.937721+0000","last_undegraded":"2023-10-16T09:49:03.937721+0000","last_fullsized":"2023-10-16T09:49:03.937721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:33.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938399+0000","last_change":"2023-10-16T09:49:01.937059+0000","last_active":"2023-10-16T09:49:03.938399+0000","last_peered":"2023-10-16T09:49:03.938399+0000","last_clean":"2023-10-16T09:49:03.938399+0000","last_became_active":"2023-10-16T09:49:01.936784+0000","last_became_peered":"2023-10-16T09:49:01.936784+0000","last_unstale":"2023-10-16T09:49:03.938399+0000","last_undegraded":"2023-10-16T09:49:03.938399+0000","last_fullsized":"2023-10-16T09:49:03.938399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:24.118554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540473+0000","last_change":"2023-10-16T09:49:02.575679+0000","last_active":"2023-10-16T09:49:03.540473+0000","last_peered":"2023-10-16T09:49:03.540473+0000","last_clean":"2023-10-16T09:49:03.540473+0000","last_became_active":"2023-10-16T09:49:02.575367+0000","last_became_peered":"2023-10-16T09:49:02.575367+0000","last_unstale":"2023-10-16T09:49:03.540473+0000","last_undegraded":"2023-10-16T09:49:03.540473+0000","last_fullsized":"2023-10-16T09:49:03.540473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:54:44.787181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540422+0000","last_change":"2023-10-16T09:49:01.934686+0000","last_active":"2023-10-16T09:49:03.540422+0000","last_peered":"2023-10-16T09:49:03.540422+0000","last_clean":"2023-10-16T09:49:03.540422+0000","last_became_active":"2023-10-16T09:49:01.934414+0000","last_became_peered":"2023-10-16T09:49:01.934414+0000","last_unstale":"2023-10-16T09:49:03.540422+0000","last_undegraded":"2023-10-16T09:49:03.540422+0000","last_fullsized":"2023-10-16T09:49:03.540422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:09.173435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940582+0000","last_change":"2023-10-16T09:49:02.573683+0000","last_active":"2023-10-16T09:49:03.940582+0000","last_peered":"2023-10-16T09:49:03.940582+0000","last_clean":"2023-10-16T09:49:03.940582+0000","last_became_active":"2023-10-16T09:49:02.573331+0000","last_became_peered":"2023-10-16T09:49:02.573331+0000","last_unstale":"2023-10-16T09:49:03.940582+0000","last_undegraded":"2023-10-16T09:49:03.940582+0000","last_fullsized":"2023-10-16T09:49:03.940582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938642+0000","last_change":"2023-10-16T09:49:01.922705+0000","last_active":"2023-10-16T09:49:03.938642+0000","last_peered":"2023-10-16T09:49:03.938642+0000","last_clean":"2023-10-16T09:49:03.938642+0000","last_became_active":"2023-10-16T09:49:01.922292+0000","last_became_peered":"2023-10-16T09:49:01.922292+0000","last_unstale":"2023-10-16T09:49:03.938642+0000","last_undegraded":"2023-10-16T09:49:03.938642+0000","last_fullsized":"2023-10-16T09:49:03.938642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:50.651364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940794+0000","last_change":"2023-10-16T09:49:02.567153+0000","last_active":"2023-10-16T09:49:03.940794+0000","last_peered":"2023-10-16T09:49:03.940794+0000","last_clean":"2023-10-16T09:49:03.940794+0000","last_became_active":"2023-10-16T09:49:02.566840+0000","last_became_peered":"2023-10-16T09:49:02.566840+0000","last_unstale":"2023-10-16T09:49:03.940794+0000","last_undegraded":"2023-10-16T09:49:03.940794+0000","last_fullsized":"2023-10-16T09:49:03.940794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:58.154049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937813+0000","last_change":"2023-10-16T09:49:01.930609+0000","last_active":"2023-10-16T09:49:03.937813+0000","last_peered":"2023-10-16T09:49:03.937813+0000","last_clean":"2023-10-16T09:49:03.937813+0000","last_became_active":"2023-10-16T09:49:01.930235+0000","last_became_peered":"2023-10-16T09:49:01.930235+0000","last_unstale":"2023-10-16T09:49:03.937813+0000","last_undegraded":"2023-10-16T09:49:03.937813+0000","last_fullsized":"2023-10-16T09:49:03.937813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:29:30.922262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942436+0000","last_change":"2023-10-16T09:49:02.577180+0000","last_active":"2023-10-16T09:49:03.942436+0000","last_peered":"2023-10-16T09:49:03.942436+0000","last_clean":"2023-10-16T09:49:03.942436+0000","last_became_active":"2023-10-16T09:49:02.576534+0000","last_became_peered":"2023-10-16T09:49:02.576534+0000","last_unstale":"2023-10-16T09:49:03.942436+0000","last_undegraded":"2023-10-16T09:49:03.942436+0000","last_fullsized":"2023-10-16T09:49:03.942436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:45.572589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940062+0000","last_change":"2023-10-16T09:49:01.918540+0000","last_active":"2023-10-16T09:49:03.940062+0000","last_peered":"2023-10-16T09:49:03.940062+0000","last_clean":"2023-10-16T09:49:03.940062+0000","last_became_active":"2023-10-16T09:49:01.917673+0000","last_became_peered":"2023-10-16T09:49:01.917673+0000","last_unstale":"2023-10-16T09:49:03.940062+0000","last_undegraded":"2023-10-16T09:49:03.940062+0000","last_fullsized":"2023-10-16T09:49:03.940062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:41.350109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939226+0000","last_change":"2023-10-16T09:49:02.570004+0000","last_active":"2023-10-16T09:49:03.939226+0000","last_peered":"2023-10-16T09:49:03.939226+0000","last_clean":"2023-10-16T09:49:03.939226+0000","last_became_active":"2023-10-16T09:49:02.569583+0000","last_became_peered":"2023-10-16T09:49:02.569583+0000","last_unstale":"2023-10-16T09:49:03.939226+0000","last_undegraded":"2023-10-16T09:49:03.939226+0000","last_fullsized":"2023-10-16T09:49:03.939226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:51.981410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540696+0000","last_change":"2023-10-16T09:49:01.932631+0000","last_active":"2023-10-16T09:49:03.540696+0000","last_peered":"2023-10-16T09:49:03.540696+0000","last_clean":"2023-10-16T09:49:03.540696+0000","last_became_active":"2023-10-16T09:49:01.928924+0000","last_became_peered":"2023-10-16T09:49:01.928924+0000","last_unstale":"2023-10-16T09:49:03.540696+0000","last_undegraded":"2023-10-16T09:49:03.540696+0000","last_fullsized":"2023-10-16T09:49:03.540696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:15.945748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942055+0000","last_change":"2023-10-16T09:49:02.569260+0000","last_active":"2023-10-16T09:49:03.942055+0000","last_peered":"2023-10-16T09:49:03.942055+0000","last_clean":"2023-10-16T09:49:03.942055+0000","last_became_active":"2023-10-16T09:49:02.568903+0000","last_became_peered":"2023-10-16T09:49:02.568903+0000","last_unstale":"2023-10-16T09:49:03.942055+0000","last_undegraded":"2023-10-16T09:49:03.942055+0000","last_fullsized":"2023-10-16T09:49:03.942055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:08.814427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937182+0000","last_change":"2023-10-16T09:49:01.930319+0000","last_active":"2023-10-16T09:49:03.937182+0000","last_peered":"2023-10-16T09:49:03.937182+0000","last_clean":"2023-10-16T09:49:03.937182+0000","last_became_active":"2023-10-16T09:49:01.928990+0000","last_became_peered":"2023-10-16T09:49:01.928990+0000","last_unstale":"2023-10-16T09:49:03.937182+0000","last_undegraded":"2023-10-16T09:49:03.937182+0000","last_fullsized":"2023-10-16T09:49:03.937182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:43.789805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937440+0000","last_change":"2023-10-16T09:49:02.568938+0000","last_active":"2023-10-16T09:49:03.937440+0000","last_peered":"2023-10-16T09:49:03.937440+0000","last_clean":"2023-10-16T09:49:03.937440+0000","last_became_active":"2023-10-16T09:49:02.568222+0000","last_became_peered":"2023-10-16T09:49:02.568222+0000","last_unstale":"2023-10-16T09:49:03.937440+0000","last_undegraded":"2023-10-16T09:49:03.937440+0000","last_fullsized":"2023-10-16T09:49:03.937440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:47.112566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.538731+0000","last_change":"2023-10-16T09:49:01.913071+0000","last_active":"2023-10-16T09:49:03.538731+0000","last_peered":"2023-10-16T09:49:03.538731+0000","last_clean":"2023-10-16T09:49:03.538731+0000","last_became_active":"2023-10-16T09:49:01.912124+0000","last_became_peered":"2023-10-16T09:49:01.912124+0000","last_unstale":"2023-10-16T09:49:03.538731+0000","last_undegraded":"2023-10-16T09:49:03.538731+0000","last_fullsized":"2023-10-16T09:49:03.538731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:16.897247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941822+0000","last_change":"2023-10-16T09:49:02.577394+0000","last_active":"2023-10-16T09:49:03.941822+0000","last_peered":"2023-10-16T09:49:03.941822+0000","last_clean":"2023-10-16T09:49:03.941822+0000","last_became_active":"2023-10-16T09:49:02.577042+0000","last_became_peered":"2023-10-16T09:49:02.577042+0000","last_unstale":"2023-10-16T09:49:03.941822+0000","last_undegraded":"2023-10-16T09:49:03.941822+0000","last_fullsized":"2023-10-16T09:49:03.941822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:05.838911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941102+0000","last_change":"2023-10-16T09:49:01.918367+0000","last_active":"2023-10-16T09:49:03.941102+0000","last_peered":"2023-10-16T09:49:03.941102+0000","last_clean":"2023-10-16T09:49:03.941102+0000","last_became_active":"2023-10-16T09:49:01.918021+0000","last_became_peered":"2023-10-16T09:49:01.918021+0000","last_unstale":"2023-10-16T09:49:03.941102+0000","last_undegraded":"2023-10-16T09:49:03.941102+0000","last_fullsized":"2023-10-16T09:49:03.941102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:25.972592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942743+0000","last_change":"2023-10-16T09:49:02.567390+0000","last_active":"2023-10-16T09:49:03.942743+0000","last_peered":"2023-10-16T09:49:03.942743+0000","last_clean":"2023-10-16T09:49:03.942743+0000","last_became_active":"2023-10-16T09:49:02.566827+0000","last_became_peered":"2023-10-16T09:49:02.566827+0000","last_unstale":"2023-10-16T09:49:03.942743+0000","last_undegraded":"2023-10-16T09:49:03.942743+0000","last_fullsized":"2023-10-16T09:49:03.942743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:40:56.166477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539098+0000","last_change":"2023-10-16T09:49:01.904674+0000","last_active":"2023-10-16T09:49:03.539098+0000","last_peered":"2023-10-16T09:49:03.539098+0000","last_clean":"2023-10-16T09:49:03.539098+0000","last_became_active":"2023-10-16T09:49:01.904380+0000","last_became_peered":"2023-10-16T09:49:01.904380+0000","last_unstale":"2023-10-16T09:49:03.539098+0000","last_undegraded":"2023-10-16T09:49:03.539098+0000","last_fullsized":"2023-10-16T09:49:03.539098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:25.972592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.943170+0000","last_change":"2023-10-16T09:49:02.575633+0000","last_active":"2023-10-16T09:49:03.943170+0000","last_peered":"2023-10-16T09:49:03.943170+0000","last_clean":"2023-10-16T09:49:03.943170+0000","last_became_active":"2023-10-16T09:49:02.573187+0000","last_became_peered":"2023-10-16T09:49:02.573187+0000","last_unstale":"2023-10-16T09:49:03.943170+0000","last_undegraded":"2023-10-16T09:49:03.943170+0000","last_fullsized":"2023-10-16T09:49:03.943170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:12.370135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938903+0000","last_change":"2023-10-16T09:49:01.911986+0000","last_active":"2023-10-16T09:49:03.938903+0000","last_peered":"2023-10-16T09:49:03.938903+0000","last_clean":"2023-10-16T09:49:03.938903+0000","last_became_active":"2023-10-16T09:49:01.911700+0000","last_became_peered":"2023-10-16T09:49:01.911700+0000","last_unstale":"2023-10-16T09:49:03.938903+0000","last_undegraded":"2023-10-16T09:49:03.938903+0000","last_fullsized":"2023-10-16T09:49:03.938903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:45.782520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540965+0000","last_change":"2023-10-16T09:49:02.574594+0000","last_active":"2023-10-16T09:49:03.540965+0000","last_peered":"2023-10-16T09:49:03.540965+0000","last_clean":"2023-10-16T09:49:03.540965+0000","last_became_active":"2023-10-16T09:49:02.573419+0000","last_became_peered":"2023-10-16T09:49:02.573419+0000","last_unstale":"2023-10-16T09:49:03.540965+0000","last_undegraded":"2023-10-16T09:49:03.540965+0000","last_fullsized":"2023-10-16T09:49:03.540965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:55.226843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539291+0000","last_change":"2023-10-16T09:49:01.934196+0000","last_active":"2023-10-16T09:49:03.539291+0000","last_peered":"2023-10-16T09:49:03.539291+0000","last_clean":"2023-10-16T09:49:03.539291+0000","last_became_active":"2023-10-16T09:49:01.933922+0000","last_became_peered":"2023-10-16T09:49:01.933922+0000","last_unstale":"2023-10-16T09:49:03.539291+0000","last_undegraded":"2023-10-16T09:49:03.539291+0000","last_fullsized":"2023-10-16T09:49:03.539291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:10:19.173392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942391+0000","last_change":"2023-10-16T09:49:02.577848+0000","last_active":"2023-10-16T09:49:03.942391+0000","last_peered":"2023-10-16T09:49:03.942391+0000","last_clean":"2023-10-16T09:49:03.942391+0000","last_became_active":"2023-10-16T09:49:02.577571+0000","last_became_peered":"2023-10-16T09:49:02.577571+0000","last_unstale":"2023-10-16T09:49:03.942391+0000","last_undegraded":"2023-10-16T09:49:03.942391+0000","last_fullsized":"2023-10-16T09:49:03.942391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:12:32.747232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T09:49:02.548285+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T09:49:02.548285+0000","last_peered":"2023-10-16T09:49:02.548285+0000","last_clean":"2023-10-16T09:49:02.548285+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T09:49:02.548285+0000","last_undegraded":"2023-10-16T09:49:02.548285+0000","last_fullsized":"2023-10-16T09:49:02.548285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539903+0000","last_change":"2023-10-16T09:49:01.922671+0000","last_active":"2023-10-16T09:49:03.539903+0000","last_peered":"2023-10-16T09:49:03.539903+0000","last_clean":"2023-10-16T09:49:03.539903+0000","last_became_active":"2023-10-16T09:49:01.922356+0000","last_became_peered":"2023-10-16T09:49:01.922356+0000","last_unstale":"2023-10-16T09:49:03.539903+0000","last_undegraded":"2023-10-16T09:49:03.539903+0000","last_fullsized":"2023-10-16T09:49:03.539903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:10:19.173392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.542081+0000","last_change":"2023-10-16T09:49:02.572185+0000","last_active":"2023-10-16T09:49:03.542081+0000","last_peered":"2023-10-16T09:49:03.542081+0000","last_clean":"2023-10-16T09:49:03.542081+0000","last_became_active":"2023-10-16T09:49:02.571953+0000","last_became_peered":"2023-10-16T09:49:02.571953+0000","last_unstale":"2023-10-16T09:49:03.542081+0000","last_undegraded":"2023-10-16T09:49:03.542081+0000","last_fullsized":"2023-10-16T09:49:03.542081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:07.724831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941421+0000","last_change":"2023-10-16T09:49:01.925742+0000","last_active":"2023-10-16T09:49:03.941421+0000","last_peered":"2023-10-16T09:49:03.941421+0000","last_clean":"2023-10-16T09:49:03.941421+0000","last_became_active":"2023-10-16T09:49:01.925438+0000","last_became_peered":"2023-10-16T09:49:01.925438+0000","last_unstale":"2023-10-16T09:49:03.941421+0000","last_undegraded":"2023-10-16T09:49:03.941421+0000","last_fullsized":"2023-10-16T09:49:03.941421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:56.824003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938454+0000","last_change":"2023-10-16T09:49:02.562256+0000","last_active":"2023-10-16T09:49:03.938454+0000","last_peered":"2023-10-16T09:49:03.938454+0000","last_clean":"2023-10-16T09:49:03.938454+0000","last_became_active":"2023-10-16T09:49:02.561942+0000","last_became_peered":"2023-10-16T09:49:02.561942+0000","last_unstale":"2023-10-16T09:49:03.938454+0000","last_undegraded":"2023-10-16T09:49:03.938454+0000","last_fullsized":"2023-10-16T09:49:03.938454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:12:14.396674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.538971+0000","last_change":"2023-10-16T09:49:01.905306+0000","last_active":"2023-10-16T09:49:03.538971+0000","last_peered":"2023-10-16T09:49:03.538971+0000","last_clean":"2023-10-16T09:49:03.538971+0000","last_became_active":"2023-10-16T09:49:01.904664+0000","last_became_peered":"2023-10-16T09:49:01.904664+0000","last_unstale":"2023-10-16T09:49:03.538971+0000","last_undegraded":"2023-10-16T09:49:03.538971+0000","last_fullsized":"2023-10-16T09:49:03.538971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:38:51.800830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939222+0000","last_change":"2023-10-16T09:49:02.551068+0000","last_active":"2023-10-16T09:49:03.939222+0000","last_peered":"2023-10-16T09:49:03.939222+0000","last_clean":"2023-10-16T09:49:03.939222+0000","last_became_active":"2023-10-16T09:49:02.550772+0000","last_became_peered":"2023-10-16T09:49:02.550772+0000","last_unstale":"2023-10-16T09:49:03.939222+0000","last_undegraded":"2023-10-16T09:49:03.939222+0000","last_fullsized":"2023-10-16T09:49:03.939222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:51.170491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540961+0000","last_change":"2023-10-16T09:49:01.932431+0000","last_active":"2023-10-16T09:49:03.540961+0000","last_peered":"2023-10-16T09:49:03.540961+0000","last_clean":"2023-10-16T09:49:03.540961+0000","last_became_active":"2023-10-16T09:49:01.927598+0000","last_became_peered":"2023-10-16T09:49:01.927598+0000","last_unstale":"2023-10-16T09:49:03.540961+0000","last_undegraded":"2023-10-16T09:49:03.540961+0000","last_fullsized":"2023-10-16T09:49:03.540961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:50.651364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940795+0000","last_change":"2023-10-16T09:49:02.571518+0000","last_active":"2023-10-16T09:49:03.940795+0000","last_peered":"2023-10-16T09:49:03.940795+0000","last_clean":"2023-10-16T09:49:03.940795+0000","last_became_active":"2023-10-16T09:49:02.571243+0000","last_became_peered":"2023-10-16T09:49:02.571243+0000","last_unstale":"2023-10-16T09:49:03.940795+0000","last_undegraded":"2023-10-16T09:49:03.940795+0000","last_fullsized":"2023-10-16T09:49:03.940795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:45.119851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937987+0000","last_change":"2023-10-16T09:49:01.928253+0000","last_active":"2023-10-16T09:49:03.937987+0000","last_peered":"2023-10-16T09:49:03.937987+0000","last_clean":"2023-10-16T09:49:03.937987+0000","last_became_active":"2023-10-16T09:49:01.927808+0000","last_became_peered":"2023-10-16T09:49:01.927808+0000","last_unstale":"2023-10-16T09:49:03.937987+0000","last_undegraded":"2023-10-16T09:49:03.937987+0000","last_fullsized":"2023-10-16T09:49:03.937987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:41.350109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541226+0000","last_change":"2023-10-16T09:49:02.575272+0000","last_active":"2023-10-16T09:49:03.541226+0000","last_peered":"2023-10-16T09:49:03.541226+0000","last_clean":"2023-10-16T09:49:03.541226+0000","last_became_active":"2023-10-16T09:49:02.574888+0000","last_became_peered":"2023-10-16T09:49:02.574888+0000","last_unstale":"2023-10-16T09:49:03.541226+0000","last_undegraded":"2023-10-16T09:49:03.541226+0000","last_fullsized":"2023-10-16T09:49:03.541226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:12:32.747232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942771+0000","last_change":"2023-10-16T09:49:01.931740+0000","last_active":"2023-10-16T09:49:03.942771+0000","last_peered":"2023-10-16T09:49:03.942771+0000","last_clean":"2023-10-16T09:49:03.942771+0000","last_became_active":"2023-10-16T09:49:01.931370+0000","last_became_peered":"2023-10-16T09:49:01.931370+0000","last_unstale":"2023-10-16T09:49:03.942771+0000","last_undegraded":"2023-10-16T09:49:03.942771+0000","last_fullsized":"2023-10-16T09:49:03.942771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:49.840445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937766+0000","last_change":"2023-10-16T09:49:02.553220+0000","last_active":"2023-10-16T09:49:03.937766+0000","last_peered":"2023-10-16T09:49:03.937766+0000","last_clean":"2023-10-16T09:49:03.937766+0000","last_became_active":"2023-10-16T09:49:02.552703+0000","last_became_peered":"2023-10-16T09:49:02.552703+0000","last_unstale":"2023-10-16T09:49:03.937766+0000","last_undegraded":"2023-10-16T09:49:03.937766+0000","last_fullsized":"2023-10-16T09:49:03.937766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937784+0000","last_change":"2023-10-16T09:49:01.928063+0000","last_active":"2023-10-16T09:49:03.937784+0000","last_peered":"2023-10-16T09:49:03.937784+0000","last_clean":"2023-10-16T09:49:03.937784+0000","last_became_active":"2023-10-16T09:49:01.927770+0000","last_became_peered":"2023-10-16T09:49:01.927770+0000","last_unstale":"2023-10-16T09:49:03.937784+0000","last_undegraded":"2023-10-16T09:49:03.937784+0000","last_fullsized":"2023-10-16T09:49:03.937784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:12:13.066628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540860+0000","last_change":"2023-10-16T09:49:02.562667+0000","last_active":"2023-10-16T09:49:03.540860+0000","last_peered":"2023-10-16T09:49:03.540860+0000","last_clean":"2023-10-16T09:49:03.540860+0000","last_became_active":"2023-10-16T09:49:02.562361+0000","last_became_peered":"2023-10-16T09:49:02.562361+0000","last_unstale":"2023-10-16T09:49:03.540860+0000","last_undegraded":"2023-10-16T09:49:03.540860+0000","last_fullsized":"2023-10-16T09:49:03.540860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938522+0000","last_change":"2023-10-16T09:49:01.930319+0000","last_active":"2023-10-16T09:49:03.938522+0000","last_peered":"2023-10-16T09:49:03.938522+0000","last_clean":"2023-10-16T09:49:03.938522+0000","last_became_active":"2023-10-16T09:49:01.929983+0000","last_became_peered":"2023-10-16T09:49:01.929983+0000","last_unstale":"2023-10-16T09:49:03.938522+0000","last_undegraded":"2023-10-16T09:49:03.938522+0000","last_fullsized":"2023-10-16T09:49:03.938522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:12:13.066628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939914+0000","last_change":"2023-10-16T09:49:02.567693+0000","last_active":"2023-10-16T09:49:03.939914+0000","last_peered":"2023-10-16T09:49:03.939914+0000","last_clean":"2023-10-16T09:49:03.939914+0000","last_became_active":"2023-10-16T09:49:02.566400+0000","last_became_peered":"2023-10-16T09:49:02.566400+0000","last_unstale":"2023-10-16T09:49:03.939914+0000","last_undegraded":"2023-10-16T09:49:03.939914+0000","last_fullsized":"2023-10-16T09:49:03.939914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:33.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941454+0000","last_change":"2023-10-16T09:49:01.929977+0000","last_active":"2023-10-16T09:49:03.941454+0000","last_peered":"2023-10-16T09:49:03.941454+0000","last_clean":"2023-10-16T09:49:03.941454+0000","last_became_active":"2023-10-16T09:49:01.928703+0000","last_became_peered":"2023-10-16T09:49:01.928703+0000","last_unstale":"2023-10-16T09:49:03.941454+0000","last_undegraded":"2023-10-16T09:49:03.941454+0000","last_fullsized":"2023-10-16T09:49:03.941454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:43.789805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940325+0000","last_change":"2023-10-16T09:49:02.558651+0000","last_active":"2023-10-16T09:49:03.940325+0000","last_peered":"2023-10-16T09:49:03.940325+0000","last_clean":"2023-10-16T09:49:03.940325+0000","last_became_active":"2023-10-16T09:49:02.554370+0000","last_became_peered":"2023-10-16T09:49:02.554370+0000","last_unstale":"2023-10-16T09:49:03.940325+0000","last_undegraded":"2023-10-16T09:49:03.940325+0000","last_fullsized":"2023-10-16T09:49:03.940325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:51:17.179081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938321+0000","last_change":"2023-10-16T09:49:01.918105+0000","last_active":"2023-10-16T09:49:03.938321+0000","last_peered":"2023-10-16T09:49:03.938321+0000","last_clean":"2023-10-16T09:49:03.938321+0000","last_became_active":"2023-10-16T09:49:01.917625+0000","last_became_peered":"2023-10-16T09:49:01.917625+0000","last_unstale":"2023-10-16T09:49:03.938321+0000","last_undegraded":"2023-10-16T09:49:03.938321+0000","last_fullsized":"2023-10-16T09:49:03.938321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:24.118554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540666+0000","last_change":"2023-10-16T09:49:02.556320+0000","last_active":"2023-10-16T09:49:03.540666+0000","last_peered":"2023-10-16T09:49:03.540666+0000","last_clean":"2023-10-16T09:49:03.540666+0000","last_became_active":"2023-10-16T09:49:02.555897+0000","last_became_peered":"2023-10-16T09:49:02.555897+0000","last_unstale":"2023-10-16T09:49:03.540666+0000","last_undegraded":"2023-10-16T09:49:03.540666+0000","last_fullsized":"2023-10-16T09:49:03.540666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:33.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938761+0000","last_change":"2023-10-16T09:49:01.915222+0000","last_active":"2023-10-16T09:49:03.938761+0000","last_peered":"2023-10-16T09:49:03.938761+0000","last_clean":"2023-10-16T09:49:03.938761+0000","last_became_active":"2023-10-16T09:49:01.914843+0000","last_became_peered":"2023-10-16T09:49:01.914843+0000","last_unstale":"2023-10-16T09:49:03.938761+0000","last_undegraded":"2023-10-16T09:49:03.938761+0000","last_fullsized":"2023-10-16T09:49:03.938761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:51:15.849036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541605+0000","last_change":"2023-10-16T09:49:02.567819+0000","last_active":"2023-10-16T09:49:03.541605+0000","last_peered":"2023-10-16T09:49:03.541605+0000","last_clean":"2023-10-16T09:49:03.541605+0000","last_became_active":"2023-10-16T09:49:02.567548+0000","last_became_peered":"2023-10-16T09:49:02.567548+0000","last_unstale":"2023-10-16T09:49:03.541605+0000","last_undegraded":"2023-10-16T09:49:03.541605+0000","last_fullsized":"2023-10-16T09:49:03.541605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:54:44.787181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942035+0000","last_change":"2023-10-16T09:49:01.924408+0000","last_active":"2023-10-16T09:49:03.942035+0000","last_peered":"2023-10-16T09:49:03.942035+0000","last_clean":"2023-10-16T09:49:03.942035+0000","last_became_active":"2023-10-16T09:49:01.922411+0000","last_became_peered":"2023-10-16T09:49:01.922411+0000","last_unstale":"2023-10-16T09:49:03.942035+0000","last_undegraded":"2023-10-16T09:49:03.942035+0000","last_fullsized":"2023-10-16T09:49:03.942035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:24.118554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937385+0000","last_change":"2023-10-16T09:49:02.560941+0000","last_active":"2023-10-16T09:49:03.937385+0000","last_peered":"2023-10-16T09:49:03.937385+0000","last_clean":"2023-10-16T09:49:03.937385+0000","last_became_active":"2023-10-16T09:49:02.560673+0000","last_became_peered":"2023-10-16T09:49:02.560673+0000","last_unstale":"2023-10-16T09:49:03.937385+0000","last_undegraded":"2023-10-16T09:49:03.937385+0000","last_fullsized":"2023-10-16T09:49:03.937385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:51.981410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540856+0000","last_change":"2023-10-16T09:49:02.575954+0000","last_active":"2023-10-16T09:49:03.540856+0000","last_peered":"2023-10-16T09:49:03.540856+0000","last_clean":"2023-10-16T09:49:03.540856+0000","last_became_active":"2023-10-16T09:49:02.575517+0000","last_became_peered":"2023-10-16T09:49:02.575517+0000","last_unstale":"2023-10-16T09:49:03.540856+0000","last_undegraded":"2023-10-16T09:49:03.540856+0000","last_fullsized":"2023-10-16T09:49:03.540856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:07.724831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942265+0000","last_change":"2023-10-16T09:49:01.933911+0000","last_active":"2023-10-16T09:49:03.942265+0000","last_peered":"2023-10-16T09:49:03.942265+0000","last_clean":"2023-10-16T09:49:03.942265+0000","last_became_active":"2023-10-16T09:49:01.933640+0000","last_became_peered":"2023-10-16T09:49:01.933640+0000","last_unstale":"2023-10-16T09:49:03.942265+0000","last_undegraded":"2023-10-16T09:49:03.942265+0000","last_fullsized":"2023-10-16T09:49:03.942265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:27.229341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540639+0000","last_change":"2023-10-16T09:49:02.580360+0000","last_active":"2023-10-16T09:49:03.540639+0000","last_peered":"2023-10-16T09:49:03.540639+0000","last_clean":"2023-10-16T09:49:03.540639+0000","last_became_active":"2023-10-16T09:49:02.574225+0000","last_became_peered":"2023-10-16T09:49:02.574225+0000","last_unstale":"2023-10-16T09:49:03.540639+0000","last_undegraded":"2023-10-16T09:49:03.540639+0000","last_fullsized":"2023-10-16T09:49:03.540639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:39:22.668380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942632+0000","last_change":"2023-10-16T09:49:01.931685+0000","last_active":"2023-10-16T09:49:03.942632+0000","last_peered":"2023-10-16T09:49:03.942632+0000","last_clean":"2023-10-16T09:49:03.942632+0000","last_became_active":"2023-10-16T09:49:01.930733+0000","last_became_peered":"2023-10-16T09:49:01.930733+0000","last_unstale":"2023-10-16T09:49:03.942632+0000","last_undegraded":"2023-10-16T09:49:03.942632+0000","last_fullsized":"2023-10-16T09:49:03.942632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:09.173435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940714+0000","last_change":"2023-10-16T09:49:02.573451+0000","last_active":"2023-10-16T09:49:03.940714+0000","last_peered":"2023-10-16T09:49:03.940714+0000","last_clean":"2023-10-16T09:49:03.940714+0000","last_became_active":"2023-10-16T09:49:02.572970+0000","last_became_peered":"2023-10-16T09:49:02.572970+0000","last_unstale":"2023-10-16T09:49:03.940714+0000","last_undegraded":"2023-10-16T09:49:03.940714+0000","last_fullsized":"2023-10-16T09:49:03.940714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:51.170491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540855+0000","last_change":"2023-10-16T09:49:01.920578+0000","last_active":"2023-10-16T09:49:03.540855+0000","last_peered":"2023-10-16T09:49:03.540855+0000","last_clean":"2023-10-16T09:49:03.540855+0000","last_became_active":"2023-10-16T09:49:01.919963+0000","last_became_peered":"2023-10-16T09:49:01.919963+0000","last_unstale":"2023-10-16T09:49:03.540855+0000","last_undegraded":"2023-10-16T09:49:03.540855+0000","last_fullsized":"2023-10-16T09:49:03.540855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:15.945748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540674+0000","last_change":"2023-10-16T09:49:02.563034+0000","last_active":"2023-10-16T09:49:03.540674+0000","last_peered":"2023-10-16T09:49:03.540674+0000","last_clean":"2023-10-16T09:49:03.540674+0000","last_became_active":"2023-10-16T09:49:02.562680+0000","last_became_peered":"2023-10-16T09:49:02.562680+0000","last_unstale":"2023-10-16T09:49:03.540674+0000","last_undegraded":"2023-10-16T09:49:03.540674+0000","last_fullsized":"2023-10-16T09:49:03.540674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940780+0000","last_change":"2023-10-16T09:49:01.919451+0000","last_active":"2023-10-16T09:49:03.940780+0000","last_peered":"2023-10-16T09:49:03.940780+0000","last_clean":"2023-10-16T09:49:03.940780+0000","last_became_active":"2023-10-16T09:49:01.919106+0000","last_became_peered":"2023-10-16T09:49:01.919106+0000","last_unstale":"2023-10-16T09:49:03.940780+0000","last_undegraded":"2023-10-16T09:49:03.940780+0000","last_fullsized":"2023-10-16T09:49:03.940780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:29:30.922262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":815104,"data_stored":666519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5629999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8280000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0819999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1299999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.224}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.722}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.621}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7949999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3320000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.54}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7889999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7470000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.982}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.024}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3879999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":77181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.806}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0019999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3029999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1680000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2549999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4630000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2309999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8340000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9710000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1869999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.288}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.496}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":811008,"data_stored":660275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5429999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:49:07.300 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:49:07.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.537+0000 7fdc7b378640 1 Processor -- start 2023-10-16T09:49:07.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.538+0000 7fdc7b378640 1 -- start start 2023-10-16T09:49:07.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.539+0000 7fdc7b378640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdc74160420 0x7fdc740ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:07.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.539+0000 7fdc7b378640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc74160800 0x7fdc740cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:07.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.539+0000 7fdc7b378640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc740adc40 0x7fdc7406adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:07.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.539+0000 7fdc7b378640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc740cde30 con 0x7fdc74160420 2023-10-16T09:49:07.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.539+0000 7fdc7b378640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7406e170 con 0x7fdc740adc40 2023-10-16T09:49:07.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.539+0000 7fdc7b378640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc740cd460 con 0x7fdc74160800 2023-10-16T09:49:07.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.539+0000 7fdc79114640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdc74160420 0x7fdc740ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:07.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.539+0000 7fdc79114640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdc74160420 0x7fdc740ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59432/0 (socket says 172.21.15.84:59432) 2023-10-16T09:49:07.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.539+0000 7fdc79114640 1 -- 172.21.15.84:0/399139465 learned_addr learned my addr 172.21.15.84:0/399139465 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:07.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.540+0000 7fdc79114640 1 -- 172.21.15.84:0/399139465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc74160800 msgr2=0x7fdc740cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:07.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.540+0000 7fdc78913640 1 --2- 172.21.15.84:0/399139465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc74160800 0x7fdc740cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:07.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.540+0000 7fdc79915640 1 --2- 172.21.15.84:0/399139465 >> v2:172.21.15.157:3300/0 conn(0x7fdc740adc40 0x7fdc7406adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:07.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.540+0000 7fdc79114640 1 --2- 172.21.15.84:0/399139465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc74160800 0x7fdc740cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.540+0000 7fdc79114640 1 -- 172.21.15.84:0/399139465 >> v2:172.21.15.157:3300/0 conn(0x7fdc740adc40 msgr2=0x7fdc7406adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:07.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.540+0000 7fdc79114640 1 --2- 172.21.15.84:0/399139465 >> v2:172.21.15.157:3300/0 conn(0x7fdc740adc40 0x7fdc7406adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.540+0000 7fdc79114640 1 -- 172.21.15.84:0/399139465 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7406e3c0 con 0x7fdc74160420 2023-10-16T09:49:07.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.541+0000 7fdc79114640 1 --2- 172.21.15.84:0/399139465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc74160420 0x7fdc740ae170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdc740b1ae0 tx=0x7fdc7002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b088e113166dad server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:07.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.541+0000 7fdc6bfff640 1 -- 172.21.15.84:0/399139465 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc70031870 con 0x7fdc74160420 2023-10-16T09:49:07.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.541+0000 7fdc6bfff640 1 -- 172.21.15.84:0/399139465 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc700319d0 con 0x7fdc74160420 2023-10-16T09:49:07.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.541+0000 7fdc6bfff640 1 -- 172.21.15.84:0/399139465 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc70031d60 con 0x7fdc74160420 2023-10-16T09:49:07.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.542+0000 7fdc7b378640 1 -- 172.21.15.84:0/399139465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc74160420 msgr2=0x7fdc740ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:07.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.542+0000 7fdc7b378640 1 --2- 172.21.15.84:0/399139465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc74160420 0x7fdc740ae170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdc740b1ae0 tx=0x7fdc7002b910 comp rx=0 tx=0).stop 2023-10-16T09:49:07.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.542+0000 7fdc7b378640 1 -- 172.21.15.84:0/399139465 shutdown_connections 2023-10-16T09:49:07.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.542+0000 7fdc7b378640 1 --2- 172.21.15.84:0/399139465 >> v2:172.21.15.157:3300/0 conn(0x7fdc740adc40 0x7fdc7406adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.542+0000 7fdc7b378640 1 --2- 172.21.15.84:0/399139465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc74160800 0x7fdc740cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.542+0000 7fdc7b378640 1 --2- 172.21.15.84:0/399139465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc74160420 0x7fdc740ae170 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.542+0000 7fdc7b378640 1 -- 172.21.15.84:0/399139465 >> 172.21.15.84:0/399139465 conn(0x7fdc740b0d30 msgr2=0x7fdc740b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:07.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.542+0000 7fdc7b378640 1 -- 172.21.15.84:0/399139465 shutdown_connections 2023-10-16T09:49:07.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.542+0000 7fdc7b378640 1 -- 172.21.15.84:0/399139465 wait complete. 2023-10-16T09:49:07.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.543+0000 7fdc7b378640 1 Processor -- start 2023-10-16T09:49:07.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.544+0000 7fdc7b378640 1 -- start start 2023-10-16T09:49:07.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.545+0000 7fdc7b378640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc740adc40 0x7fdc740c4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:07.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.545+0000 7fdc7b378640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc74160420 0x7fdc741f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:07.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.545+0000 7fdc7b378640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc74160800 0x7fdc740c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:07.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.545+0000 7fdc7b378640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc741f28d0 con 0x7fdc740adc40 2023-10-16T09:49:07.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.545+0000 7fdc7b378640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc740c5a40 con 0x7fdc74160800 2023-10-16T09:49:07.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.545+0000 7fdc7b378640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc740c4ec0 con 0x7fdc74160420 2023-10-16T09:49:07.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.545+0000 7fdc79114640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc740adc40 0x7fdc740c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:07.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.545+0000 7fdc79114640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc740adc40 0x7fdc740c4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59434/0 (socket says 172.21.15.84:59434) 2023-10-16T09:49:07.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.545+0000 7fdc79114640 1 -- 172.21.15.84:0/1541889728 learned_addr learned my addr 172.21.15.84:0/1541889728 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:07.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.545+0000 7fdc79915640 1 --2- 172.21.15.84:0/1541889728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc74160800 0x7fdc740c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:07.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.545+0000 7fdc78913640 1 --2- 172.21.15.84:0/1541889728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc74160420 0x7fdc741f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:07.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.545+0000 7fdc79114640 1 -- 172.21.15.84:0/1541889728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc74160420 msgr2=0x7fdc741f2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:07.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.545+0000 7fdc79114640 1 --2- 172.21.15.84:0/1541889728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc74160420 0x7fdc741f2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.545+0000 7fdc79114640 1 -- 172.21.15.84:0/1541889728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc74160800 msgr2=0x7fdc740c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:07.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.545+0000 7fdc79114640 1 --2- 172.21.15.84:0/1541889728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc74160800 0x7fdc740c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.545+0000 7fdc79114640 1 -- 172.21.15.84:0/1541889728 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc700094d0 con 0x7fdc740adc40 2023-10-16T09:49:07.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.545+0000 7fdc78913640 1 --2- 172.21.15.84:0/1541889728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc74160420 0x7fdc741f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:07.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.546+0000 7fdc79114640 1 --2- 172.21.15.84:0/1541889728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc740adc40 0x7fdc740c4800 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdc70000c00 tx=0x7fdc700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:07.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.546+0000 7fdc69ffb640 1 -- 172.21.15.84:0/1541889728 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc70031840 con 0x7fdc740adc40 2023-10-16T09:49:07.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.546+0000 7fdc69ffb640 1 -- 172.21.15.84:0/1541889728 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc700443f0 con 0x7fdc740adc40 2023-10-16T09:49:07.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.546+0000 7fdc69ffb640 1 -- 172.21.15.84:0/1541889728 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc700447a0 con 0x7fdc740adc40 2023-10-16T09:49:07.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.546+0000 7fdc7b378640 1 -- 172.21.15.84:0/1541889728 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc740c51a0 con 0x7fdc740adc40 2023-10-16T09:49:07.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.546+0000 7fdc7b378640 1 -- 172.21.15.84:0/1541889728 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc740c5760 con 0x7fdc740adc40 2023-10-16T09:49:07.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.547+0000 7fdc7b378640 1 -- 172.21.15.84:0/1541889728 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7404e160 con 0x7fdc740adc40 2023-10-16T09:49:07.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.548+0000 7fdc69ffb640 1 -- 172.21.15.84:0/1541889728 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdc7004d450 con 0x7fdc740adc40 2023-10-16T09:49:07.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.549+0000 7fdc69ffb640 1 --2- 172.21.15.84:0/1541889728 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdc5806ae70 0x7fdc5806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:07.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.549+0000 7fdc69ffb640 1 -- 172.21.15.84:0/1541889728 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc7002d040 con 0x7fdc740adc40 2023-10-16T09:49:07.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.549+0000 7fdc78913640 1 --2- 172.21.15.84:0/1541889728 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdc5806ae70 0x7fdc5806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:07.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.550+0000 7fdc78913640 1 --2- 172.21.15.84:0/1541889728 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdc5806ae70 0x7fdc5806d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdc7409f850 tx=0x7fdc64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:07.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.554+0000 7fdc69ffb640 1 -- 172.21.15.84:0/1541889728 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdc70076e80 con 0x7fdc740adc40 2023-10-16T09:49:07.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.742+0000 7fdc7b378640 1 -- 172.21.15.84:0/1541889728 --> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdc740cce10 con 0x7fdc5806ae70 2023-10-16T09:49:07.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.762+0000 7fdc69ffb640 1 -- 172.21.15.84:0/1541889728 <== mgr.4107 v2:172.21.15.84:6832/3010246666 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264514 (secure 0 0 0) 0x7fdc6400c040 con 0x7fdc5806ae70 2023-10-16T09:49:07.767 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:07.768 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T09:49:07.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.768+0000 7fdc7b378640 1 -- 172.21.15.84:0/1541889728 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdc5806ae70 msgr2=0x7fdc5806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:07.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.768+0000 7fdc7b378640 1 --2- 172.21.15.84:0/1541889728 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdc5806ae70 0x7fdc5806d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdc7409f850 tx=0x7fdc64008040 comp rx=0 tx=0).stop 2023-10-16T09:49:07.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.768+0000 7fdc7b378640 1 -- 172.21.15.84:0/1541889728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc740adc40 msgr2=0x7fdc740c4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:07.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.768+0000 7fdc7b378640 1 --2- 172.21.15.84:0/1541889728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc740adc40 0x7fdc740c4800 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdc70000c00 tx=0x7fdc700046c0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.769+0000 7fdc7b378640 1 -- 172.21.15.84:0/1541889728 shutdown_connections 2023-10-16T09:49:07.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.769+0000 7fdc7b378640 1 --2- 172.21.15.84:0/1541889728 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdc5806ae70 0x7fdc5806d330 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.769+0000 7fdc7b378640 1 --2- 172.21.15.84:0/1541889728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc74160800 0x7fdc740c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.769+0000 7fdc7b378640 1 --2- 172.21.15.84:0/1541889728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc74160420 0x7fdc741f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.769+0000 7fdc7b378640 1 --2- 172.21.15.84:0/1541889728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc740adc40 0x7fdc740c4800 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:07.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.769+0000 7fdc7b378640 1 -- 172.21.15.84:0/1541889728 >> 172.21.15.84:0/1541889728 conn(0x7fdc740b0d30 msgr2=0x7fdc740af1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:07.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.769+0000 7fdc7b378640 1 -- 172.21.15.84:0/1541889728 shutdown_connections 2023-10-16T09:49:07.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:07.769+0000 7fdc7b378640 1 -- 172.21.15.84:0/1541889728 wait complete. 2023-10-16T09:49:07.787 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-16T09:49:07.590260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236084,"kb_used_data":2908,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738636,"statfs":{"total":773094113280,"available":772852363264,"internally_reserved":0,"allocated":2977792,"data_stored":1783808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001893"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942979+0000","last_change":"2023-10-16T09:49:02.569411+0000","last_active":"2023-10-16T09:49:03.942979+0000","last_peered":"2023-10-16T09:49:03.942979+0000","last_clean":"2023-10-16T09:49:03.942979+0000","last_became_active":"2023-10-16T09:49:02.569101+0000","last_became_peered":"2023-10-16T09:49:02.569101+0000","last_unstale":"2023-10-16T09:49:03.942979+0000","last_undegraded":"2023-10-16T09:49:03.942979+0000","last_fullsized":"2023-10-16T09:49:03.942979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:20:01.929361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937690+0000","last_change":"2023-10-16T09:49:02.570326+0000","last_active":"2023-10-16T09:49:03.937690+0000","last_peered":"2023-10-16T09:49:03.937690+0000","last_clean":"2023-10-16T09:49:03.937690+0000","last_became_active":"2023-10-16T09:49:02.569866+0000","last_became_peered":"2023-10-16T09:49:02.569866+0000","last_unstale":"2023-10-16T09:49:03.937690+0000","last_undegraded":"2023-10-16T09:49:03.937690+0000","last_fullsized":"2023-10-16T09:49:03.937690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:10.503481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940359+0000","last_change":"2023-10-16T09:49:02.566217+0000","last_active":"2023-10-16T09:49:03.940359+0000","last_peered":"2023-10-16T09:49:03.940359+0000","last_clean":"2023-10-16T09:49:03.940359+0000","last_became_active":"2023-10-16T09:49:02.565843+0000","last_became_peered":"2023-10-16T09:49:02.565843+0000","last_unstale":"2023-10-16T09:49:03.940359+0000","last_undegraded":"2023-10-16T09:49:03.940359+0000","last_fullsized":"2023-10-16T09:49:03.940359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540898+0000","last_change":"2023-10-16T09:49:02.580605+0000","last_active":"2023-10-16T09:49:03.540898+0000","last_peered":"2023-10-16T09:49:03.540898+0000","last_clean":"2023-10-16T09:49:03.540898+0000","last_became_active":"2023-10-16T09:49:02.573845+0000","last_became_peered":"2023-10-16T09:49:02.573845+0000","last_unstale":"2023-10-16T09:49:03.540898+0000","last_undegraded":"2023-10-16T09:49:03.540898+0000","last_fullsized":"2023-10-16T09:49:03.540898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:05.838911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941895+0000","last_change":"2023-10-16T09:49:02.563238+0000","last_active":"2023-10-16T09:49:03.941895+0000","last_peered":"2023-10-16T09:49:03.941895+0000","last_clean":"2023-10-16T09:49:03.941895+0000","last_became_active":"2023-10-16T09:49:02.562849+0000","last_became_peered":"2023-10-16T09:49:02.562849+0000","last_unstale":"2023-10-16T09:49:03.941895+0000","last_undegraded":"2023-10-16T09:49:03.941895+0000","last_fullsized":"2023-10-16T09:49:03.941895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:55.226843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540148+0000","last_change":"2023-10-16T09:49:02.572385+0000","last_active":"2023-10-16T09:49:03.540148+0000","last_peered":"2023-10-16T09:49:03.540148+0000","last_clean":"2023-10-16T09:49:03.540148+0000","last_became_active":"2023-10-16T09:49:02.571270+0000","last_became_peered":"2023-10-16T09:49:02.571270+0000","last_unstale":"2023-10-16T09:49:03.540148+0000","last_undegraded":"2023-10-16T09:49:03.540148+0000","last_fullsized":"2023-10-16T09:49:03.540148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941710+0000","last_change":"2023-10-16T09:49:02.570612+0000","last_active":"2023-10-16T09:49:03.941710+0000","last_peered":"2023-10-16T09:49:03.941710+0000","last_clean":"2023-10-16T09:49:03.941710+0000","last_became_active":"2023-10-16T09:49:02.570330+0000","last_became_peered":"2023-10-16T09:49:02.570330+0000","last_unstale":"2023-10-16T09:49:03.941710+0000","last_undegraded":"2023-10-16T09:49:03.941710+0000","last_fullsized":"2023-10-16T09:49:03.941710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:07.724831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540222+0000","last_change":"2023-10-16T09:49:02.572301+0000","last_active":"2023-10-16T09:49:03.540222+0000","last_peered":"2023-10-16T09:49:03.540222+0000","last_clean":"2023-10-16T09:49:03.540222+0000","last_became_active":"2023-10-16T09:49:02.571934+0000","last_became_peered":"2023-10-16T09:49:02.571934+0000","last_unstale":"2023-10-16T09:49:03.540222+0000","last_undegraded":"2023-10-16T09:49:03.540222+0000","last_fullsized":"2023-10-16T09:49:03.540222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:51.981410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.943019+0000","last_change":"2023-10-16T09:49:02.580905+0000","last_active":"2023-10-16T09:49:03.943019+0000","last_peered":"2023-10-16T09:49:03.943019+0000","last_clean":"2023-10-16T09:49:03.943019+0000","last_became_active":"2023-10-16T09:49:02.577694+0000","last_became_peered":"2023-10-16T09:49:02.577694+0000","last_unstale":"2023-10-16T09:49:03.943019+0000","last_undegraded":"2023-10-16T09:49:03.943019+0000","last_fullsized":"2023-10-16T09:49:03.943019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:02:14.799393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938295+0000","last_change":"2023-10-16T09:49:02.576191+0000","last_active":"2023-10-16T09:49:03.938295+0000","last_peered":"2023-10-16T09:49:03.938295+0000","last_clean":"2023-10-16T09:49:03.938295+0000","last_became_active":"2023-10-16T09:49:02.575810+0000","last_became_peered":"2023-10-16T09:49:02.575810+0000","last_unstale":"2023-10-16T09:49:03.938295+0000","last_undegraded":"2023-10-16T09:49:03.938295+0000","last_fullsized":"2023-10-16T09:49:03.938295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937913+0000","last_change":"2023-10-16T09:49:02.577871+0000","last_active":"2023-10-16T09:49:03.937913+0000","last_peered":"2023-10-16T09:49:03.937913+0000","last_clean":"2023-10-16T09:49:03.937913+0000","last_became_active":"2023-10-16T09:49:02.577580+0000","last_became_peered":"2023-10-16T09:49:02.577580+0000","last_unstale":"2023-10-16T09:49:03.937913+0000","last_undegraded":"2023-10-16T09:49:03.937913+0000","last_fullsized":"2023-10-16T09:49:03.937913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940615+0000","last_change":"2023-10-16T09:49:02.575626+0000","last_active":"2023-10-16T09:49:03.940615+0000","last_peered":"2023-10-16T09:49:03.940615+0000","last_clean":"2023-10-16T09:49:03.940615+0000","last_became_active":"2023-10-16T09:49:02.569588+0000","last_became_peered":"2023-10-16T09:49:02.569588+0000","last_unstale":"2023-10-16T09:49:03.940615+0000","last_undegraded":"2023-10-16T09:49:03.940615+0000","last_fullsized":"2023-10-16T09:49:03.940615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:28.559387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541264+0000","last_change":"2023-10-16T09:49:02.562597+0000","last_active":"2023-10-16T09:49:03.541264+0000","last_peered":"2023-10-16T09:49:03.541264+0000","last_clean":"2023-10-16T09:49:03.541264+0000","last_became_active":"2023-10-16T09:49:02.562322+0000","last_became_peered":"2023-10-16T09:49:02.562322+0000","last_unstale":"2023-10-16T09:49:03.541264+0000","last_undegraded":"2023-10-16T09:49:03.541264+0000","last_fullsized":"2023-10-16T09:49:03.541264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:51.170491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939007+0000","last_change":"2023-10-16T09:49:02.568210+0000","last_active":"2023-10-16T09:49:03.939007+0000","last_peered":"2023-10-16T09:49:03.939007+0000","last_clean":"2023-10-16T09:49:03.939007+0000","last_became_active":"2023-10-16T09:49:02.567330+0000","last_became_peered":"2023-10-16T09:49:02.567330+0000","last_unstale":"2023-10-16T09:49:03.939007+0000","last_undegraded":"2023-10-16T09:49:03.939007+0000","last_fullsized":"2023-10-16T09:49:03.939007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:17.275794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938071+0000","last_change":"2023-10-16T09:49:02.561360+0000","last_active":"2023-10-16T09:49:03.938071+0000","last_peered":"2023-10-16T09:49:03.938071+0000","last_clean":"2023-10-16T09:49:03.938071+0000","last_became_active":"2023-10-16T09:49:02.560831+0000","last_became_peered":"2023-10-16T09:49:02.560831+0000","last_unstale":"2023-10-16T09:49:03.938071+0000","last_undegraded":"2023-10-16T09:49:03.938071+0000","last_fullsized":"2023-10-16T09:49:03.938071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539917+0000","last_change":"2023-10-16T09:49:02.571717+0000","last_active":"2023-10-16T09:49:03.539917+0000","last_peered":"2023-10-16T09:49:03.539917+0000","last_clean":"2023-10-16T09:49:03.539917+0000","last_became_active":"2023-10-16T09:49:02.571311+0000","last_became_peered":"2023-10-16T09:49:02.571311+0000","last_unstale":"2023-10-16T09:49:03.539917+0000","last_undegraded":"2023-10-16T09:49:03.539917+0000","last_fullsized":"2023-10-16T09:49:03.539917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541271+0000","last_change":"2023-10-16T09:49:02.582038+0000","last_active":"2023-10-16T09:49:03.541271+0000","last_peered":"2023-10-16T09:49:03.541271+0000","last_clean":"2023-10-16T09:49:03.541271+0000","last_became_active":"2023-10-16T09:49:02.581780+0000","last_became_peered":"2023-10-16T09:49:02.581780+0000","last_unstale":"2023-10-16T09:49:03.541271+0000","last_undegraded":"2023-10-16T09:49:03.541271+0000","last_fullsized":"2023-10-16T09:49:03.541271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:45.572589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539221+0000","last_change":"2023-10-16T09:49:02.563686+0000","last_active":"2023-10-16T09:49:03.539221+0000","last_peered":"2023-10-16T09:49:03.539221+0000","last_clean":"2023-10-16T09:49:03.539221+0000","last_became_active":"2023-10-16T09:49:02.562728+0000","last_became_peered":"2023-10-16T09:49:02.562728+0000","last_unstale":"2023-10-16T09:49:03.539221+0000","last_undegraded":"2023-10-16T09:49:03.539221+0000","last_fullsized":"2023-10-16T09:49:03.539221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:58.154049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942765+0000","last_change":"2023-10-16T09:49:02.569238+0000","last_active":"2023-10-16T09:49:03.942765+0000","last_peered":"2023-10-16T09:49:03.942765+0000","last_clean":"2023-10-16T09:49:03.942765+0000","last_became_active":"2023-10-16T09:49:02.568292+0000","last_became_peered":"2023-10-16T09:49:02.568292+0000","last_unstale":"2023-10-16T09:49:03.942765+0000","last_undegraded":"2023-10-16T09:49:03.942765+0000","last_fullsized":"2023-10-16T09:49:03.942765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:46.937999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937917+0000","last_change":"2023-10-16T09:49:02.563047+0000","last_active":"2023-10-16T09:49:03.937917+0000","last_peered":"2023-10-16T09:49:03.937917+0000","last_clean":"2023-10-16T09:49:03.937917+0000","last_became_active":"2023-10-16T09:49:02.562532+0000","last_became_peered":"2023-10-16T09:49:02.562532+0000","last_unstale":"2023-10-16T09:49:03.937917+0000","last_undegraded":"2023-10-16T09:49:03.937917+0000","last_fullsized":"2023-10-16T09:49:03.937917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:33.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939622+0000","last_change":"2023-10-16T09:49:02.569454+0000","last_active":"2023-10-16T09:49:03.939622+0000","last_peered":"2023-10-16T09:49:03.939622+0000","last_clean":"2023-10-16T09:49:03.939622+0000","last_became_active":"2023-10-16T09:49:02.569216+0000","last_became_peered":"2023-10-16T09:49:02.569216+0000","last_unstale":"2023-10-16T09:49:03.939622+0000","last_undegraded":"2023-10-16T09:49:03.939622+0000","last_fullsized":"2023-10-16T09:49:03.939622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:47.112566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540091+0000","last_change":"2023-10-16T09:49:02.572101+0000","last_active":"2023-10-16T09:49:03.540091+0000","last_peered":"2023-10-16T09:49:03.540091+0000","last_clean":"2023-10-16T09:49:03.540091+0000","last_became_active":"2023-10-16T09:49:02.570939+0000","last_became_peered":"2023-10-16T09:49:02.570939+0000","last_unstale":"2023-10-16T09:49:03.540091+0000","last_undegraded":"2023-10-16T09:49:03.540091+0000","last_fullsized":"2023-10-16T09:49:03.540091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540377+0000","last_change":"2023-10-16T09:49:02.562443+0000","last_active":"2023-10-16T09:49:03.540377+0000","last_peered":"2023-10-16T09:49:03.540377+0000","last_clean":"2023-10-16T09:49:03.540377+0000","last_became_active":"2023-10-16T09:49:02.562184+0000","last_became_peered":"2023-10-16T09:49:02.562184+0000","last_unstale":"2023-10-16T09:49:03.540377+0000","last_undegraded":"2023-10-16T09:49:03.540377+0000","last_fullsized":"2023-10-16T09:49:03.540377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:47.112566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540214+0000","last_change":"2023-10-16T09:49:02.566157+0000","last_active":"2023-10-16T09:49:03.540214+0000","last_peered":"2023-10-16T09:49:03.540214+0000","last_clean":"2023-10-16T09:49:03.540214+0000","last_became_active":"2023-10-16T09:49:02.565808+0000","last_became_peered":"2023-10-16T09:49:02.565808+0000","last_unstale":"2023-10-16T09:49:03.540214+0000","last_undegraded":"2023-10-16T09:49:03.540214+0000","last_fullsized":"2023-10-16T09:49:03.540214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:51.170491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540739+0000","last_change":"2023-10-16T09:49:02.570442+0000","last_active":"2023-10-16T09:49:03.540739+0000","last_peered":"2023-10-16T09:49:03.540739+0000","last_clean":"2023-10-16T09:49:03.540739+0000","last_became_active":"2023-10-16T09:49:02.570173+0000","last_became_peered":"2023-10-16T09:49:02.570173+0000","last_unstale":"2023-10-16T09:49:03.540739+0000","last_undegraded":"2023-10-16T09:49:03.540739+0000","last_fullsized":"2023-10-16T09:49:03.540739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938295+0000","last_change":"2023-10-16T09:49:02.565763+0000","last_active":"2023-10-16T09:49:03.938295+0000","last_peered":"2023-10-16T09:49:03.938295+0000","last_clean":"2023-10-16T09:49:03.938295+0000","last_became_active":"2023-10-16T09:49:02.565248+0000","last_became_peered":"2023-10-16T09:49:02.565248+0000","last_unstale":"2023-10-16T09:49:03.938295+0000","last_undegraded":"2023-10-16T09:49:03.938295+0000","last_fullsized":"2023-10-16T09:49:03.938295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938715+0000","last_change":"2023-10-16T09:49:02.556364+0000","last_active":"2023-10-16T09:49:03.938715+0000","last_peered":"2023-10-16T09:49:03.938715+0000","last_clean":"2023-10-16T09:49:03.938715+0000","last_became_active":"2023-10-16T09:49:02.556055+0000","last_became_peered":"2023-10-16T09:49:02.556055+0000","last_unstale":"2023-10-16T09:49:03.938715+0000","last_undegraded":"2023-10-16T09:49:03.938715+0000","last_fullsized":"2023-10-16T09:49:03.938715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541726+0000","last_change":"2023-10-16T09:49:02.564976+0000","last_active":"2023-10-16T09:49:03.541726+0000","last_peered":"2023-10-16T09:49:03.541726+0000","last_clean":"2023-10-16T09:49:03.541726+0000","last_became_active":"2023-10-16T09:49:02.564798+0000","last_became_peered":"2023-10-16T09:49:02.564798+0000","last_unstale":"2023-10-16T09:49:03.541726+0000","last_undegraded":"2023-10-16T09:49:03.541726+0000","last_fullsized":"2023-10-16T09:49:03.541726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:39:22.668380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939780+0000","last_change":"2023-10-16T09:49:02.561691+0000","last_active":"2023-10-16T09:49:03.939780+0000","last_peered":"2023-10-16T09:49:03.939780+0000","last_clean":"2023-10-16T09:49:03.939780+0000","last_became_active":"2023-10-16T09:49:02.561408+0000","last_became_peered":"2023-10-16T09:49:02.561408+0000","last_unstale":"2023-10-16T09:49:03.939780+0000","last_undegraded":"2023-10-16T09:49:03.939780+0000","last_fullsized":"2023-10-16T09:49:03.939780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:10.503481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541186+0000","last_change":"2023-10-16T09:49:02.572702+0000","last_active":"2023-10-16T09:49:03.541186+0000","last_peered":"2023-10-16T09:49:03.541186+0000","last_clean":"2023-10-16T09:49:03.541186+0000","last_became_active":"2023-10-16T09:49:02.571811+0000","last_became_peered":"2023-10-16T09:49:02.571811+0000","last_unstale":"2023-10-16T09:49:03.541186+0000","last_undegraded":"2023-10-16T09:49:03.541186+0000","last_fullsized":"2023-10-16T09:49:03.541186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:08.814427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.538838+0000","last_change":"2023-10-16T09:49:02.576101+0000","last_active":"2023-10-16T09:49:03.538838+0000","last_peered":"2023-10-16T09:49:03.538838+0000","last_clean":"2023-10-16T09:49:03.538838+0000","last_became_active":"2023-10-16T09:49:02.575844+0000","last_became_peered":"2023-10-16T09:49:02.575844+0000","last_unstale":"2023-10-16T09:49:03.538838+0000","last_undegraded":"2023-10-16T09:49:03.538838+0000","last_fullsized":"2023-10-16T09:49:03.538838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:45.119851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937812+0000","last_change":"2023-10-16T09:49:02.568425+0000","last_active":"2023-10-16T09:49:03.937812+0000","last_peered":"2023-10-16T09:49:03.937812+0000","last_clean":"2023-10-16T09:49:03.937812+0000","last_became_active":"2023-10-16T09:49:02.568091+0000","last_became_peered":"2023-10-16T09:49:02.568091+0000","last_unstale":"2023-10-16T09:49:03.937812+0000","last_undegraded":"2023-10-16T09:49:03.937812+0000","last_fullsized":"2023-10-16T09:49:03.937812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:10.503481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940047+0000","last_change":"2023-10-16T09:49:02.559306+0000","last_active":"2023-10-16T09:49:03.940047+0000","last_peered":"2023-10-16T09:49:03.940047+0000","last_clean":"2023-10-16T09:49:03.940047+0000","last_became_active":"2023-10-16T09:49:02.558667+0000","last_became_peered":"2023-10-16T09:49:02.558667+0000","last_unstale":"2023-10-16T09:49:03.940047+0000","last_undegraded":"2023-10-16T09:49:03.940047+0000","last_fullsized":"2023-10-16T09:49:03.940047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:38:53.130875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541754+0000","last_change":"2023-10-16T09:49:01.918393+0000","last_active":"2023-10-16T09:49:03.541754+0000","last_peered":"2023-10-16T09:49:03.541754+0000","last_clean":"2023-10-16T09:49:03.541754+0000","last_became_active":"2023-10-16T09:49:01.918085+0000","last_became_peered":"2023-10-16T09:49:01.918085+0000","last_unstale":"2023-10-16T09:49:03.541754+0000","last_undegraded":"2023-10-16T09:49:03.541754+0000","last_fullsized":"2023-10-16T09:49:03.541754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:24.118554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939938+0000","last_change":"2023-10-16T09:49:02.562307+0000","last_active":"2023-10-16T09:49:03.939938+0000","last_peered":"2023-10-16T09:49:03.939938+0000","last_clean":"2023-10-16T09:49:03.939938+0000","last_became_active":"2023-10-16T09:49:02.562013+0000","last_became_peered":"2023-10-16T09:49:02.562013+0000","last_unstale":"2023-10-16T09:49:03.939938+0000","last_undegraded":"2023-10-16T09:49:03.939938+0000","last_fullsized":"2023-10-16T09:49:03.939938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937479+0000","last_change":"2023-10-16T09:49:01.931642+0000","last_active":"2023-10-16T09:49:03.937479+0000","last_peered":"2023-10-16T09:49:03.937479+0000","last_clean":"2023-10-16T09:49:03.937479+0000","last_became_active":"2023-10-16T09:49:01.929759+0000","last_became_peered":"2023-10-16T09:49:01.929759+0000","last_unstale":"2023-10-16T09:49:03.937479+0000","last_undegraded":"2023-10-16T09:49:03.937479+0000","last_fullsized":"2023-10-16T09:49:03.937479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:56.824003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540519+0000","last_change":"2023-10-16T09:49:01.925581+0000","last_active":"2023-10-16T09:49:03.540519+0000","last_peered":"2023-10-16T09:49:03.540519+0000","last_clean":"2023-10-16T09:49:03.540519+0000","last_became_active":"2023-10-16T09:49:01.925329+0000","last_became_peered":"2023-10-16T09:49:01.925329+0000","last_unstale":"2023-10-16T09:49:03.540519+0000","last_undegraded":"2023-10-16T09:49:03.540519+0000","last_fullsized":"2023-10-16T09:49:03.540519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:12:13.066628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540403+0000","last_change":"2023-10-16T09:49:02.566394+0000","last_active":"2023-10-16T09:49:03.540403+0000","last_peered":"2023-10-16T09:49:03.540403+0000","last_clean":"2023-10-16T09:49:03.540403+0000","last_became_active":"2023-10-16T09:49:02.566059+0000","last_became_peered":"2023-10-16T09:49:02.566059+0000","last_unstale":"2023-10-16T09:49:03.540403+0000","last_undegraded":"2023-10-16T09:49:03.540403+0000","last_fullsized":"2023-10-16T09:49:03.540403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:10.503481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540187+0000","last_change":"2023-10-16T09:49:01.933426+0000","last_active":"2023-10-16T09:49:03.540187+0000","last_peered":"2023-10-16T09:49:03.540187+0000","last_clean":"2023-10-16T09:49:03.540187+0000","last_became_active":"2023-10-16T09:49:01.930300+0000","last_became_peered":"2023-10-16T09:49:01.930300+0000","last_unstale":"2023-10-16T09:49:03.540187+0000","last_undegraded":"2023-10-16T09:49:03.540187+0000","last_fullsized":"2023-10-16T09:49:03.540187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:27.229341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937721+0000","last_change":"2023-10-16T09:49:02.555653+0000","last_active":"2023-10-16T09:49:03.937721+0000","last_peered":"2023-10-16T09:49:03.937721+0000","last_clean":"2023-10-16T09:49:03.937721+0000","last_became_active":"2023-10-16T09:49:02.555099+0000","last_became_peered":"2023-10-16T09:49:02.555099+0000","last_unstale":"2023-10-16T09:49:03.937721+0000","last_undegraded":"2023-10-16T09:49:03.937721+0000","last_fullsized":"2023-10-16T09:49:03.937721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:33.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938399+0000","last_change":"2023-10-16T09:49:01.937059+0000","last_active":"2023-10-16T09:49:03.938399+0000","last_peered":"2023-10-16T09:49:03.938399+0000","last_clean":"2023-10-16T09:49:03.938399+0000","last_became_active":"2023-10-16T09:49:01.936784+0000","last_became_peered":"2023-10-16T09:49:01.936784+0000","last_unstale":"2023-10-16T09:49:03.938399+0000","last_undegraded":"2023-10-16T09:49:03.938399+0000","last_fullsized":"2023-10-16T09:49:03.938399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:24.118554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540473+0000","last_change":"2023-10-16T09:49:02.575679+0000","last_active":"2023-10-16T09:49:03.540473+0000","last_peered":"2023-10-16T09:49:03.540473+0000","last_clean":"2023-10-16T09:49:03.540473+0000","last_became_active":"2023-10-16T09:49:02.575367+0000","last_became_peered":"2023-10-16T09:49:02.575367+0000","last_unstale":"2023-10-16T09:49:03.540473+0000","last_undegraded":"2023-10-16T09:49:03.540473+0000","last_fullsized":"2023-10-16T09:49:03.540473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:54:44.787181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540422+0000","last_change":"2023-10-16T09:49:01.934686+0000","last_active":"2023-10-16T09:49:03.540422+0000","last_peered":"2023-10-16T09:49:03.540422+0000","last_clean":"2023-10-16T09:49:03.540422+0000","last_became_active":"2023-10-16T09:49:01.934414+0000","last_became_peered":"2023-10-16T09:49:01.934414+0000","last_unstale":"2023-10-16T09:49:03.540422+0000","last_undegraded":"2023-10-16T09:49:03.540422+0000","last_fullsized":"2023-10-16T09:49:03.540422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:09.173435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940582+0000","last_change":"2023-10-16T09:49:02.573683+0000","last_active":"2023-10-16T09:49:03.940582+0000","last_peered":"2023-10-16T09:49:03.940582+0000","last_clean":"2023-10-16T09:49:03.940582+0000","last_became_active":"2023-10-16T09:49:02.573331+0000","last_became_peered":"2023-10-16T09:49:02.573331+0000","last_unstale":"2023-10-16T09:49:03.940582+0000","last_undegraded":"2023-10-16T09:49:03.940582+0000","last_fullsized":"2023-10-16T09:49:03.940582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938642+0000","last_change":"2023-10-16T09:49:01.922705+0000","last_active":"2023-10-16T09:49:03.938642+0000","last_peered":"2023-10-16T09:49:03.938642+0000","last_clean":"2023-10-16T09:49:03.938642+0000","last_became_active":"2023-10-16T09:49:01.922292+0000","last_became_peered":"2023-10-16T09:49:01.922292+0000","last_unstale":"2023-10-16T09:49:03.938642+0000","last_undegraded":"2023-10-16T09:49:03.938642+0000","last_fullsized":"2023-10-16T09:49:03.938642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:50.651364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940794+0000","last_change":"2023-10-16T09:49:02.567153+0000","last_active":"2023-10-16T09:49:03.940794+0000","last_peered":"2023-10-16T09:49:03.940794+0000","last_clean":"2023-10-16T09:49:03.940794+0000","last_became_active":"2023-10-16T09:49:02.566840+0000","last_became_peered":"2023-10-16T09:49:02.566840+0000","last_unstale":"2023-10-16T09:49:03.940794+0000","last_undegraded":"2023-10-16T09:49:03.940794+0000","last_fullsized":"2023-10-16T09:49:03.940794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:58.154049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937813+0000","last_change":"2023-10-16T09:49:01.930609+0000","last_active":"2023-10-16T09:49:03.937813+0000","last_peered":"2023-10-16T09:49:03.937813+0000","last_clean":"2023-10-16T09:49:03.937813+0000","last_became_active":"2023-10-16T09:49:01.930235+0000","last_became_peered":"2023-10-16T09:49:01.930235+0000","last_unstale":"2023-10-16T09:49:03.937813+0000","last_undegraded":"2023-10-16T09:49:03.937813+0000","last_fullsized":"2023-10-16T09:49:03.937813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:29:30.922262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942436+0000","last_change":"2023-10-16T09:49:02.577180+0000","last_active":"2023-10-16T09:49:03.942436+0000","last_peered":"2023-10-16T09:49:03.942436+0000","last_clean":"2023-10-16T09:49:03.942436+0000","last_became_active":"2023-10-16T09:49:02.576534+0000","last_became_peered":"2023-10-16T09:49:02.576534+0000","last_unstale":"2023-10-16T09:49:03.942436+0000","last_undegraded":"2023-10-16T09:49:03.942436+0000","last_fullsized":"2023-10-16T09:49:03.942436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:45.572589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940062+0000","last_change":"2023-10-16T09:49:01.918540+0000","last_active":"2023-10-16T09:49:03.940062+0000","last_peered":"2023-10-16T09:49:03.940062+0000","last_clean":"2023-10-16T09:49:03.940062+0000","last_became_active":"2023-10-16T09:49:01.917673+0000","last_became_peered":"2023-10-16T09:49:01.917673+0000","last_unstale":"2023-10-16T09:49:03.940062+0000","last_undegraded":"2023-10-16T09:49:03.940062+0000","last_fullsized":"2023-10-16T09:49:03.940062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:41.350109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939226+0000","last_change":"2023-10-16T09:49:02.570004+0000","last_active":"2023-10-16T09:49:03.939226+0000","last_peered":"2023-10-16T09:49:03.939226+0000","last_clean":"2023-10-16T09:49:03.939226+0000","last_became_active":"2023-10-16T09:49:02.569583+0000","last_became_peered":"2023-10-16T09:49:02.569583+0000","last_unstale":"2023-10-16T09:49:03.939226+0000","last_undegraded":"2023-10-16T09:49:03.939226+0000","last_fullsized":"2023-10-16T09:49:03.939226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:51.981410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540696+0000","last_change":"2023-10-16T09:49:01.932631+0000","last_active":"2023-10-16T09:49:03.540696+0000","last_peered":"2023-10-16T09:49:03.540696+0000","last_clean":"2023-10-16T09:49:03.540696+0000","last_became_active":"2023-10-16T09:49:01.928924+0000","last_became_peered":"2023-10-16T09:49:01.928924+0000","last_unstale":"2023-10-16T09:49:03.540696+0000","last_undegraded":"2023-10-16T09:49:03.540696+0000","last_fullsized":"2023-10-16T09:49:03.540696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:15.945748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942055+0000","last_change":"2023-10-16T09:49:02.569260+0000","last_active":"2023-10-16T09:49:03.942055+0000","last_peered":"2023-10-16T09:49:03.942055+0000","last_clean":"2023-10-16T09:49:03.942055+0000","last_became_active":"2023-10-16T09:49:02.568903+0000","last_became_peered":"2023-10-16T09:49:02.568903+0000","last_unstale":"2023-10-16T09:49:03.942055+0000","last_undegraded":"2023-10-16T09:49:03.942055+0000","last_fullsized":"2023-10-16T09:49:03.942055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:08.814427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937182+0000","last_change":"2023-10-16T09:49:01.930319+0000","last_active":"2023-10-16T09:49:03.937182+0000","last_peered":"2023-10-16T09:49:03.937182+0000","last_clean":"2023-10-16T09:49:03.937182+0000","last_became_active":"2023-10-16T09:49:01.928990+0000","last_became_peered":"2023-10-16T09:49:01.928990+0000","last_unstale":"2023-10-16T09:49:03.937182+0000","last_undegraded":"2023-10-16T09:49:03.937182+0000","last_fullsized":"2023-10-16T09:49:03.937182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:43.789805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937440+0000","last_change":"2023-10-16T09:49:02.568938+0000","last_active":"2023-10-16T09:49:03.937440+0000","last_peered":"2023-10-16T09:49:03.937440+0000","last_clean":"2023-10-16T09:49:03.937440+0000","last_became_active":"2023-10-16T09:49:02.568222+0000","last_became_peered":"2023-10-16T09:49:02.568222+0000","last_unstale":"2023-10-16T09:49:03.937440+0000","last_undegraded":"2023-10-16T09:49:03.937440+0000","last_fullsized":"2023-10-16T09:49:03.937440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:47.112566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.538731+0000","last_change":"2023-10-16T09:49:01.913071+0000","last_active":"2023-10-16T09:49:03.538731+0000","last_peered":"2023-10-16T09:49:03.538731+0000","last_clean":"2023-10-16T09:49:03.538731+0000","last_became_active":"2023-10-16T09:49:01.912124+0000","last_became_peered":"2023-10-16T09:49:01.912124+0000","last_unstale":"2023-10-16T09:49:03.538731+0000","last_undegraded":"2023-10-16T09:49:03.538731+0000","last_fullsized":"2023-10-16T09:49:03.538731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:16.897247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941822+0000","last_change":"2023-10-16T09:49:02.577394+0000","last_active":"2023-10-16T09:49:03.941822+0000","last_peered":"2023-10-16T09:49:03.941822+0000","last_clean":"2023-10-16T09:49:03.941822+0000","last_became_active":"2023-10-16T09:49:02.577042+0000","last_became_peered":"2023-10-16T09:49:02.577042+0000","last_unstale":"2023-10-16T09:49:03.941822+0000","last_undegraded":"2023-10-16T09:49:03.941822+0000","last_fullsized":"2023-10-16T09:49:03.941822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:05.838911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941102+0000","last_change":"2023-10-16T09:49:01.918367+0000","last_active":"2023-10-16T09:49:03.941102+0000","last_peered":"2023-10-16T09:49:03.941102+0000","last_clean":"2023-10-16T09:49:03.941102+0000","last_became_active":"2023-10-16T09:49:01.918021+0000","last_became_peered":"2023-10-16T09:49:01.918021+0000","last_unstale":"2023-10-16T09:49:03.941102+0000","last_undegraded":"2023-10-16T09:49:03.941102+0000","last_fullsized":"2023-10-16T09:49:03.941102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:25.972592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942743+0000","last_change":"2023-10-16T09:49:02.567390+0000","last_active":"2023-10-16T09:49:03.942743+0000","last_peered":"2023-10-16T09:49:03.942743+0000","last_clean":"2023-10-16T09:49:03.942743+0000","last_became_active":"2023-10-16T09:49:02.566827+0000","last_became_peered":"2023-10-16T09:49:02.566827+0000","last_unstale":"2023-10-16T09:49:03.942743+0000","last_undegraded":"2023-10-16T09:49:03.942743+0000","last_fullsized":"2023-10-16T09:49:03.942743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:40:56.166477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539098+0000","last_change":"2023-10-16T09:49:01.904674+0000","last_active":"2023-10-16T09:49:03.539098+0000","last_peered":"2023-10-16T09:49:03.539098+0000","last_clean":"2023-10-16T09:49:03.539098+0000","last_became_active":"2023-10-16T09:49:01.904380+0000","last_became_peered":"2023-10-16T09:49:01.904380+0000","last_unstale":"2023-10-16T09:49:03.539098+0000","last_undegraded":"2023-10-16T09:49:03.539098+0000","last_fullsized":"2023-10-16T09:49:03.539098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:25.972592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.943170+0000","last_change":"2023-10-16T09:49:02.575633+0000","last_active":"2023-10-16T09:49:03.943170+0000","last_peered":"2023-10-16T09:49:03.943170+0000","last_clean":"2023-10-16T09:49:03.943170+0000","last_became_active":"2023-10-16T09:49:02.573187+0000","last_became_peered":"2023-10-16T09:49:02.573187+0000","last_unstale":"2023-10-16T09:49:03.943170+0000","last_undegraded":"2023-10-16T09:49:03.943170+0000","last_fullsized":"2023-10-16T09:49:03.943170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:12.370135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938903+0000","last_change":"2023-10-16T09:49:01.911986+0000","last_active":"2023-10-16T09:49:03.938903+0000","last_peered":"2023-10-16T09:49:03.938903+0000","last_clean":"2023-10-16T09:49:03.938903+0000","last_became_active":"2023-10-16T09:49:01.911700+0000","last_became_peered":"2023-10-16T09:49:01.911700+0000","last_unstale":"2023-10-16T09:49:03.938903+0000","last_undegraded":"2023-10-16T09:49:03.938903+0000","last_fullsized":"2023-10-16T09:49:03.938903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:45.782520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540965+0000","last_change":"2023-10-16T09:49:02.574594+0000","last_active":"2023-10-16T09:49:03.540965+0000","last_peered":"2023-10-16T09:49:03.540965+0000","last_clean":"2023-10-16T09:49:03.540965+0000","last_became_active":"2023-10-16T09:49:02.573419+0000","last_became_peered":"2023-10-16T09:49:02.573419+0000","last_unstale":"2023-10-16T09:49:03.540965+0000","last_undegraded":"2023-10-16T09:49:03.540965+0000","last_fullsized":"2023-10-16T09:49:03.540965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:55.226843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539291+0000","last_change":"2023-10-16T09:49:01.934196+0000","last_active":"2023-10-16T09:49:03.539291+0000","last_peered":"2023-10-16T09:49:03.539291+0000","last_clean":"2023-10-16T09:49:03.539291+0000","last_became_active":"2023-10-16T09:49:01.933922+0000","last_became_peered":"2023-10-16T09:49:01.933922+0000","last_unstale":"2023-10-16T09:49:03.539291+0000","last_undegraded":"2023-10-16T09:49:03.539291+0000","last_fullsized":"2023-10-16T09:49:03.539291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:10:19.173392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942391+0000","last_change":"2023-10-16T09:49:02.577848+0000","last_active":"2023-10-16T09:49:03.942391+0000","last_peered":"2023-10-16T09:49:03.942391+0000","last_clean":"2023-10-16T09:49:03.942391+0000","last_became_active":"2023-10-16T09:49:02.577571+0000","last_became_peered":"2023-10-16T09:49:02.577571+0000","last_unstale":"2023-10-16T09:49:03.942391+0000","last_undegraded":"2023-10-16T09:49:03.942391+0000","last_fullsized":"2023-10-16T09:49:03.942391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:12:32.747232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T09:49:02.548285+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T09:49:02.548285+0000","last_peered":"2023-10-16T09:49:02.548285+0000","last_clean":"2023-10-16T09:49:02.548285+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T09:49:02.548285+0000","last_undegraded":"2023-10-16T09:49:02.548285+0000","last_fullsized":"2023-10-16T09:49:02.548285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539903+0000","last_change":"2023-10-16T09:49:01.922671+0000","last_active":"2023-10-16T09:49:03.539903+0000","last_peered":"2023-10-16T09:49:03.539903+0000","last_clean":"2023-10-16T09:49:03.539903+0000","last_became_active":"2023-10-16T09:49:01.922356+0000","last_became_peered":"2023-10-16T09:49:01.922356+0000","last_unstale":"2023-10-16T09:49:03.539903+0000","last_undegraded":"2023-10-16T09:49:03.539903+0000","last_fullsized":"2023-10-16T09:49:03.539903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:10:19.173392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.542081+0000","last_change":"2023-10-16T09:49:02.572185+0000","last_active":"2023-10-16T09:49:03.542081+0000","last_peered":"2023-10-16T09:49:03.542081+0000","last_clean":"2023-10-16T09:49:03.542081+0000","last_became_active":"2023-10-16T09:49:02.571953+0000","last_became_peered":"2023-10-16T09:49:02.571953+0000","last_unstale":"2023-10-16T09:49:03.542081+0000","last_undegraded":"2023-10-16T09:49:03.542081+0000","last_fullsized":"2023-10-16T09:49:03.542081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:07.724831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941421+0000","last_change":"2023-10-16T09:49:01.925742+0000","last_active":"2023-10-16T09:49:03.941421+0000","last_peered":"2023-10-16T09:49:03.941421+0000","last_clean":"2023-10-16T09:49:03.941421+0000","last_became_active":"2023-10-16T09:49:01.925438+0000","last_became_peered":"2023-10-16T09:49:01.925438+0000","last_unstale":"2023-10-16T09:49:03.941421+0000","last_undegraded":"2023-10-16T09:49:03.941421+0000","last_fullsized":"2023-10-16T09:49:03.941421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:56.824003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938454+0000","last_change":"2023-10-16T09:49:02.562256+0000","last_active":"2023-10-16T09:49:03.938454+0000","last_peered":"2023-10-16T09:49:03.938454+0000","last_clean":"2023-10-16T09:49:03.938454+0000","last_became_active":"2023-10-16T09:49:02.561942+0000","last_became_peered":"2023-10-16T09:49:02.561942+0000","last_unstale":"2023-10-16T09:49:03.938454+0000","last_undegraded":"2023-10-16T09:49:03.938454+0000","last_fullsized":"2023-10-16T09:49:03.938454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:12:14.396674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.538971+0000","last_change":"2023-10-16T09:49:01.905306+0000","last_active":"2023-10-16T09:49:03.538971+0000","last_peered":"2023-10-16T09:49:03.538971+0000","last_clean":"2023-10-16T09:49:03.538971+0000","last_became_active":"2023-10-16T09:49:01.904664+0000","last_became_peered":"2023-10-16T09:49:01.904664+0000","last_unstale":"2023-10-16T09:49:03.538971+0000","last_undegraded":"2023-10-16T09:49:03.538971+0000","last_fullsized":"2023-10-16T09:49:03.538971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:38:51.800830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939222+0000","last_change":"2023-10-16T09:49:02.551068+0000","last_active":"2023-10-16T09:49:03.939222+0000","last_peered":"2023-10-16T09:49:03.939222+0000","last_clean":"2023-10-16T09:49:03.939222+0000","last_became_active":"2023-10-16T09:49:02.550772+0000","last_became_peered":"2023-10-16T09:49:02.550772+0000","last_unstale":"2023-10-16T09:49:03.939222+0000","last_undegraded":"2023-10-16T09:49:03.939222+0000","last_fullsized":"2023-10-16T09:49:03.939222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:51.170491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540961+0000","last_change":"2023-10-16T09:49:01.932431+0000","last_active":"2023-10-16T09:49:03.540961+0000","last_peered":"2023-10-16T09:49:03.540961+0000","last_clean":"2023-10-16T09:49:03.540961+0000","last_became_active":"2023-10-16T09:49:01.927598+0000","last_became_peered":"2023-10-16T09:49:01.927598+0000","last_unstale":"2023-10-16T09:49:03.540961+0000","last_undegraded":"2023-10-16T09:49:03.540961+0000","last_fullsized":"2023-10-16T09:49:03.540961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:50.651364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940795+0000","last_change":"2023-10-16T09:49:02.571518+0000","last_active":"2023-10-16T09:49:03.940795+0000","last_peered":"2023-10-16T09:49:03.940795+0000","last_clean":"2023-10-16T09:49:03.940795+0000","last_became_active":"2023-10-16T09:49:02.571243+0000","last_became_peered":"2023-10-16T09:49:02.571243+0000","last_unstale":"2023-10-16T09:49:03.940795+0000","last_undegraded":"2023-10-16T09:49:03.940795+0000","last_fullsized":"2023-10-16T09:49:03.940795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:45.119851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937987+0000","last_change":"2023-10-16T09:49:01.928253+0000","last_active":"2023-10-16T09:49:03.937987+0000","last_peered":"2023-10-16T09:49:03.937987+0000","last_clean":"2023-10-16T09:49:03.937987+0000","last_became_active":"2023-10-16T09:49:01.927808+0000","last_became_peered":"2023-10-16T09:49:01.927808+0000","last_unstale":"2023-10-16T09:49:03.937987+0000","last_undegraded":"2023-10-16T09:49:03.937987+0000","last_fullsized":"2023-10-16T09:49:03.937987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:41.350109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541226+0000","last_change":"2023-10-16T09:49:02.575272+0000","last_active":"2023-10-16T09:49:03.541226+0000","last_peered":"2023-10-16T09:49:03.541226+0000","last_clean":"2023-10-16T09:49:03.541226+0000","last_became_active":"2023-10-16T09:49:02.574888+0000","last_became_peered":"2023-10-16T09:49:02.574888+0000","last_unstale":"2023-10-16T09:49:03.541226+0000","last_undegraded":"2023-10-16T09:49:03.541226+0000","last_fullsized":"2023-10-16T09:49:03.541226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:12:32.747232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942771+0000","last_change":"2023-10-16T09:49:01.931740+0000","last_active":"2023-10-16T09:49:03.942771+0000","last_peered":"2023-10-16T09:49:03.942771+0000","last_clean":"2023-10-16T09:49:03.942771+0000","last_became_active":"2023-10-16T09:49:01.931370+0000","last_became_peered":"2023-10-16T09:49:01.931370+0000","last_unstale":"2023-10-16T09:49:03.942771+0000","last_undegraded":"2023-10-16T09:49:03.942771+0000","last_fullsized":"2023-10-16T09:49:03.942771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:49.840445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937766+0000","last_change":"2023-10-16T09:49:02.553220+0000","last_active":"2023-10-16T09:49:03.937766+0000","last_peered":"2023-10-16T09:49:03.937766+0000","last_clean":"2023-10-16T09:49:03.937766+0000","last_became_active":"2023-10-16T09:49:02.552703+0000","last_became_peered":"2023-10-16T09:49:02.552703+0000","last_unstale":"2023-10-16T09:49:03.937766+0000","last_undegraded":"2023-10-16T09:49:03.937766+0000","last_fullsized":"2023-10-16T09:49:03.937766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937784+0000","last_change":"2023-10-16T09:49:01.928063+0000","last_active":"2023-10-16T09:49:03.937784+0000","last_peered":"2023-10-16T09:49:03.937784+0000","last_clean":"2023-10-16T09:49:03.937784+0000","last_became_active":"2023-10-16T09:49:01.927770+0000","last_became_peered":"2023-10-16T09:49:01.927770+0000","last_unstale":"2023-10-16T09:49:03.937784+0000","last_undegraded":"2023-10-16T09:49:03.937784+0000","last_fullsized":"2023-10-16T09:49:03.937784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:12:13.066628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540860+0000","last_change":"2023-10-16T09:49:02.562667+0000","last_active":"2023-10-16T09:49:03.540860+0000","last_peered":"2023-10-16T09:49:03.540860+0000","last_clean":"2023-10-16T09:49:03.540860+0000","last_became_active":"2023-10-16T09:49:02.562361+0000","last_became_peered":"2023-10-16T09:49:02.562361+0000","last_unstale":"2023-10-16T09:49:03.540860+0000","last_undegraded":"2023-10-16T09:49:03.540860+0000","last_fullsized":"2023-10-16T09:49:03.540860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938522+0000","last_change":"2023-10-16T09:49:01.930319+0000","last_active":"2023-10-16T09:49:03.938522+0000","last_peered":"2023-10-16T09:49:03.938522+0000","last_clean":"2023-10-16T09:49:03.938522+0000","last_became_active":"2023-10-16T09:49:01.929983+0000","last_became_peered":"2023-10-16T09:49:01.929983+0000","last_unstale":"2023-10-16T09:49:03.938522+0000","last_undegraded":"2023-10-16T09:49:03.938522+0000","last_fullsized":"2023-10-16T09:49:03.938522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:12:13.066628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939914+0000","last_change":"2023-10-16T09:49:02.567693+0000","last_active":"2023-10-16T09:49:03.939914+0000","last_peered":"2023-10-16T09:49:03.939914+0000","last_clean":"2023-10-16T09:49:03.939914+0000","last_became_active":"2023-10-16T09:49:02.566400+0000","last_became_peered":"2023-10-16T09:49:02.566400+0000","last_unstale":"2023-10-16T09:49:03.939914+0000","last_undegraded":"2023-10-16T09:49:03.939914+0000","last_fullsized":"2023-10-16T09:49:03.939914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:33.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941454+0000","last_change":"2023-10-16T09:49:01.929977+0000","last_active":"2023-10-16T09:49:03.941454+0000","last_peered":"2023-10-16T09:49:03.941454+0000","last_clean":"2023-10-16T09:49:03.941454+0000","last_became_active":"2023-10-16T09:49:01.928703+0000","last_became_peered":"2023-10-16T09:49:01.928703+0000","last_unstale":"2023-10-16T09:49:03.941454+0000","last_undegraded":"2023-10-16T09:49:03.941454+0000","last_fullsized":"2023-10-16T09:49:03.941454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:43.789805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940325+0000","last_change":"2023-10-16T09:49:02.558651+0000","last_active":"2023-10-16T09:49:03.940325+0000","last_peered":"2023-10-16T09:49:03.940325+0000","last_clean":"2023-10-16T09:49:03.940325+0000","last_became_active":"2023-10-16T09:49:02.554370+0000","last_became_peered":"2023-10-16T09:49:02.554370+0000","last_unstale":"2023-10-16T09:49:03.940325+0000","last_undegraded":"2023-10-16T09:49:03.940325+0000","last_fullsized":"2023-10-16T09:49:03.940325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:51:17.179081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938321+0000","last_change":"2023-10-16T09:49:01.918105+0000","last_active":"2023-10-16T09:49:03.938321+0000","last_peered":"2023-10-16T09:49:03.938321+0000","last_clean":"2023-10-16T09:49:03.938321+0000","last_became_active":"2023-10-16T09:49:01.917625+0000","last_became_peered":"2023-10-16T09:49:01.917625+0000","last_unstale":"2023-10-16T09:49:03.938321+0000","last_undegraded":"2023-10-16T09:49:03.938321+0000","last_fullsized":"2023-10-16T09:49:03.938321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:24.118554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540666+0000","last_change":"2023-10-16T09:49:02.556320+0000","last_active":"2023-10-16T09:49:03.540666+0000","last_peered":"2023-10-16T09:49:03.540666+0000","last_clean":"2023-10-16T09:49:03.540666+0000","last_became_active":"2023-10-16T09:49:02.555897+0000","last_became_peered":"2023-10-16T09:49:02.555897+0000","last_unstale":"2023-10-16T09:49:03.540666+0000","last_undegraded":"2023-10-16T09:49:03.540666+0000","last_fullsized":"2023-10-16T09:49:03.540666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:33.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938761+0000","last_change":"2023-10-16T09:49:01.915222+0000","last_active":"2023-10-16T09:49:03.938761+0000","last_peered":"2023-10-16T09:49:03.938761+0000","last_clean":"2023-10-16T09:49:03.938761+0000","last_became_active":"2023-10-16T09:49:01.914843+0000","last_became_peered":"2023-10-16T09:49:01.914843+0000","last_unstale":"2023-10-16T09:49:03.938761+0000","last_undegraded":"2023-10-16T09:49:03.938761+0000","last_fullsized":"2023-10-16T09:49:03.938761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:51:15.849036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541605+0000","last_change":"2023-10-16T09:49:02.567819+0000","last_active":"2023-10-16T09:49:03.541605+0000","last_peered":"2023-10-16T09:49:03.541605+0000","last_clean":"2023-10-16T09:49:03.541605+0000","last_became_active":"2023-10-16T09:49:02.567548+0000","last_became_peered":"2023-10-16T09:49:02.567548+0000","last_unstale":"2023-10-16T09:49:03.541605+0000","last_undegraded":"2023-10-16T09:49:03.541605+0000","last_fullsized":"2023-10-16T09:49:03.541605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:54:44.787181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942035+0000","last_change":"2023-10-16T09:49:01.924408+0000","last_active":"2023-10-16T09:49:03.942035+0000","last_peered":"2023-10-16T09:49:03.942035+0000","last_clean":"2023-10-16T09:49:03.942035+0000","last_became_active":"2023-10-16T09:49:01.922411+0000","last_became_peered":"2023-10-16T09:49:01.922411+0000","last_unstale":"2023-10-16T09:49:03.942035+0000","last_undegraded":"2023-10-16T09:49:03.942035+0000","last_fullsized":"2023-10-16T09:49:03.942035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:24.118554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937385+0000","last_change":"2023-10-16T09:49:02.560941+0000","last_active":"2023-10-16T09:49:03.937385+0000","last_peered":"2023-10-16T09:49:03.937385+0000","last_clean":"2023-10-16T09:49:03.937385+0000","last_became_active":"2023-10-16T09:49:02.560673+0000","last_became_peered":"2023-10-16T09:49:02.560673+0000","last_unstale":"2023-10-16T09:49:03.937385+0000","last_undegraded":"2023-10-16T09:49:03.937385+0000","last_fullsized":"2023-10-16T09:49:03.937385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:51.981410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540856+0000","last_change":"2023-10-16T09:49:02.575954+0000","last_active":"2023-10-16T09:49:03.540856+0000","last_peered":"2023-10-16T09:49:03.540856+0000","last_clean":"2023-10-16T09:49:03.540856+0000","last_became_active":"2023-10-16T09:49:02.575517+0000","last_became_peered":"2023-10-16T09:49:02.575517+0000","last_unstale":"2023-10-16T09:49:03.540856+0000","last_undegraded":"2023-10-16T09:49:03.540856+0000","last_fullsized":"2023-10-16T09:49:03.540856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:07.724831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942265+0000","last_change":"2023-10-16T09:49:01.933911+0000","last_active":"2023-10-16T09:49:03.942265+0000","last_peered":"2023-10-16T09:49:03.942265+0000","last_clean":"2023-10-16T09:49:03.942265+0000","last_became_active":"2023-10-16T09:49:01.933640+0000","last_became_peered":"2023-10-16T09:49:01.933640+0000","last_unstale":"2023-10-16T09:49:03.942265+0000","last_undegraded":"2023-10-16T09:49:03.942265+0000","last_fullsized":"2023-10-16T09:49:03.942265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:27.229341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540639+0000","last_change":"2023-10-16T09:49:02.580360+0000","last_active":"2023-10-16T09:49:03.540639+0000","last_peered":"2023-10-16T09:49:03.540639+0000","last_clean":"2023-10-16T09:49:03.540639+0000","last_became_active":"2023-10-16T09:49:02.574225+0000","last_became_peered":"2023-10-16T09:49:02.574225+0000","last_unstale":"2023-10-16T09:49:03.540639+0000","last_undegraded":"2023-10-16T09:49:03.540639+0000","last_fullsized":"2023-10-16T09:49:03.540639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:39:22.668380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942632+0000","last_change":"2023-10-16T09:49:01.931685+0000","last_active":"2023-10-16T09:49:03.942632+0000","last_peered":"2023-10-16T09:49:03.942632+0000","last_clean":"2023-10-16T09:49:03.942632+0000","last_became_active":"2023-10-16T09:49:01.930733+0000","last_became_peered":"2023-10-16T09:49:01.930733+0000","last_unstale":"2023-10-16T09:49:03.942632+0000","last_undegraded":"2023-10-16T09:49:03.942632+0000","last_fullsized":"2023-10-16T09:49:03.942632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:09.173435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940714+0000","last_change":"2023-10-16T09:49:02.573451+0000","last_active":"2023-10-16T09:49:03.940714+0000","last_peered":"2023-10-16T09:49:03.940714+0000","last_clean":"2023-10-16T09:49:03.940714+0000","last_became_active":"2023-10-16T09:49:02.572970+0000","last_became_peered":"2023-10-16T09:49:02.572970+0000","last_unstale":"2023-10-16T09:49:03.940714+0000","last_undegraded":"2023-10-16T09:49:03.940714+0000","last_fullsized":"2023-10-16T09:49:03.940714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:51.170491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540855+0000","last_change":"2023-10-16T09:49:01.920578+0000","last_active":"2023-10-16T09:49:03.540855+0000","last_peered":"2023-10-16T09:49:03.540855+0000","last_clean":"2023-10-16T09:49:03.540855+0000","last_became_active":"2023-10-16T09:49:01.919963+0000","last_became_peered":"2023-10-16T09:49:01.919963+0000","last_unstale":"2023-10-16T09:49:03.540855+0000","last_undegraded":"2023-10-16T09:49:03.540855+0000","last_fullsized":"2023-10-16T09:49:03.540855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:15.945748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540674+0000","last_change":"2023-10-16T09:49:02.563034+0000","last_active":"2023-10-16T09:49:03.540674+0000","last_peered":"2023-10-16T09:49:03.540674+0000","last_clean":"2023-10-16T09:49:03.540674+0000","last_became_active":"2023-10-16T09:49:02.562680+0000","last_became_peered":"2023-10-16T09:49:02.562680+0000","last_unstale":"2023-10-16T09:49:03.540674+0000","last_undegraded":"2023-10-16T09:49:03.540674+0000","last_fullsized":"2023-10-16T09:49:03.540674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940780+0000","last_change":"2023-10-16T09:49:01.919451+0000","last_active":"2023-10-16T09:49:03.940780+0000","last_peered":"2023-10-16T09:49:03.940780+0000","last_clean":"2023-10-16T09:49:03.940780+0000","last_became_active":"2023-10-16T09:49:01.919106+0000","last_became_peered":"2023-10-16T09:49:01.919106+0000","last_unstale":"2023-10-16T09:49:03.940780+0000","last_undegraded":"2023-10-16T09:49:03.940780+0000","last_fullsized":"2023-10-16T09:49:03.940780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:29:30.922262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":815104,"data_stored":666519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5629999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8280000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0819999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1299999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.224}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.722}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.621}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7949999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3320000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.54}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7889999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7470000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.982}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.024}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3879999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":77181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.806}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0019999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3029999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1680000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2549999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4630000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2309999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8340000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9710000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1869999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.288}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.496}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":811008,"data_stored":660275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5429999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:49:07.793 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T09:49:07.793 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-16T09:49:07.793 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-16T09:49:07.794 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-16T09:49:07.794 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-16T09:49:08.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.040+0000 7f4a91c59640 1 Processor -- start 2023-10-16T09:49:08.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.041+0000 7f4a91c59640 1 -- start start 2023-10-16T09:49:08.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.042+0000 7f4a91c59640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4a8c0ca0a0 0x7f4a8c06c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:08.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.042+0000 7f4a91c59640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a8c15d910 con 0x7f4a8c159950 2023-10-16T09:49:08.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.042+0000 7f4a91c59640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a8c15dbf0 con 0x7f4a8c070b60 2023-10-16T09:49:08.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.042+0000 7f4a91c59640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a8c15e490 con 0x7f4a8c0ca0a0 2023-10-16T09:49:08.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.042+0000 7f4a90c57640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4a8c159950 0x7f4a8c070450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32768/0 (socket says 172.21.15.84:32768) 2023-10-16T09:49:08.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.042+0000 7f4a90c57640 1 -- 172.21.15.84:0/86837164 learned_addr learned my addr 172.21.15.84:0/86837164 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:08.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.042+0000 7f4a91458640 1 --2- 172.21.15.84:0/86837164 >> v2:172.21.15.84:3300/0 conn(0x7f4a8c0ca0a0 0x7f4a8c06c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:08.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.043+0000 7f4a8b7fe640 1 -- 172.21.15.84:0/86837164 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149995570 0 0) 0x7f4a8c15d910 con 0x7f4a8c159950 2023-10-16T09:49:08.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.043+0000 7f4a8b7fe640 1 -- 172.21.15.84:0/86837164 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a7c003520 con 0x7f4a8c159950 2023-10-16T09:49:08.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.043+0000 7f4a91458640 1 -- 172.21.15.84:0/86837164 >> v1:172.21.15.157:6789/0 conn(0x7f4a8c070b60 legacy=0x7f4a8c06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:08.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.043+0000 7f4a91458640 1 -- 172.21.15.84:0/86837164 >> v1:172.21.15.84:6789/0 conn(0x7f4a8c159950 legacy=0x7f4a8c070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:08.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.043+0000 7f4a91458640 1 -- 172.21.15.84:0/86837164 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c15ded0 con 0x7f4a8c0ca0a0 2023-10-16T09:49:08.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.044+0000 7f4a91458640 1 --2- 172.21.15.84:0/86837164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a8c0ca0a0 0x7f4a8c06c810 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009d90 tx=0x7f4a8402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29ea1f1aebd1ee6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:08.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.045+0000 7f4a8b7fe640 1 -- 172.21.15.84:0/86837164 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8402cc60 con 0x7f4a8c0ca0a0 2023-10-16T09:49:08.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.045+0000 7f4a8b7fe640 1 -- 172.21.15.84:0/86837164 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8402cdc0 con 0x7f4a8c0ca0a0 2023-10-16T09:49:08.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.045+0000 7f4a8b7fe640 1 -- 172.21.15.84:0/86837164 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84034660 con 0x7f4a8c0ca0a0 2023-10-16T09:49:08.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.045+0000 7f4a91c59640 1 -- 172.21.15.84:0/86837164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a8c0ca0a0 msgr2=0x7f4a8c06c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:08.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.045+0000 7f4a91c59640 1 --2- 172.21.15.84:0/86837164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a8c0ca0a0 0x7f4a8c06c810 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009d90 tx=0x7f4a8402beb0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.045+0000 7f4a91c59640 1 -- 172.21.15.84:0/86837164 shutdown_connections 2023-10-16T09:49:08.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.045+0000 7f4a91c59640 1 --2- 172.21.15.84:0/86837164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a8c0ca0a0 0x7f4a8c06c810 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.045+0000 7f4a91c59640 1 -- 172.21.15.84:0/86837164 >> 172.21.15.84:0/86837164 conn(0x7f4a8c0ce6a0 msgr2=0x7f4a8c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:08.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.046+0000 7f4a91c59640 1 -- 172.21.15.84:0/86837164 shutdown_connections 2023-10-16T09:49:08.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.046+0000 7f4a91c59640 1 -- 172.21.15.84:0/86837164 wait complete. 2023-10-16T09:49:08.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.047+0000 7f4a91c59640 1 Processor -- start 2023-10-16T09:49:08.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.047+0000 7f4a91c59640 1 -- start start 2023-10-16T09:49:08.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.049+0000 7f4a91c59640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a8c070b60 0x7f4a8c1574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:08.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.049+0000 7f4a91c59640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a8c0ca0a0 0x7f4a8c1579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:08.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.049+0000 7f4a91c59640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8c159950 0x7f4a8c14a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:08.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.049+0000 7f4a91c59640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c14b6e0 con 0x7f4a8c0ca0a0 2023-10-16T09:49:08.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.049+0000 7f4a91c59640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c0c21b0 con 0x7f4a8c070b60 2023-10-16T09:49:08.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.049+0000 7f4a91c59640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c148e30 con 0x7f4a8c159950 2023-10-16T09:49:08.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.049+0000 7f4a8bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a8c0ca0a0 0x7f4a8c1579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:08.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.049+0000 7f4a8bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a8c0ca0a0 0x7f4a8c1579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59446/0 (socket says 172.21.15.84:59446) 2023-10-16T09:49:08.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.049+0000 7f4a8bfff640 1 -- 172.21.15.84:0/3135012071 learned_addr learned my addr 172.21.15.84:0/3135012071 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:08.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.049+0000 7f4a90c57640 1 --2- 172.21.15.84:0/3135012071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a8c070b60 0x7f4a8c1574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:08.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.049+0000 7f4a8bfff640 1 -- 172.21.15.84:0/3135012071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8c159950 msgr2=0x7f4a8c14a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:08.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.049+0000 7f4a8bfff640 1 --2- 172.21.15.84:0/3135012071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8c159950 0x7f4a8c14a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.049+0000 7f4a8bfff640 1 -- 172.21.15.84:0/3135012071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a8c070b60 msgr2=0x7f4a8c1574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:08.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.049+0000 7f4a8bfff640 1 --2- 172.21.15.84:0/3135012071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a8c070b60 0x7f4a8c1574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.049+0000 7f4a8bfff640 1 -- 172.21.15.84:0/3135012071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a84009a70 con 0x7f4a8c0ca0a0 2023-10-16T09:49:08.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.050+0000 7f4a8bfff640 1 --2- 172.21.15.84:0/3135012071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a8c0ca0a0 0x7f4a8c1579f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800c480 tx=0x7f4a7800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:08.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.050+0000 7f4a897fa640 1 -- 172.21.15.84:0/3135012071 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a780052d0 con 0x7f4a8c0ca0a0 2023-10-16T09:49:08.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.050+0000 7f4a897fa640 1 -- 172.21.15.84:0/3135012071 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a78005430 con 0x7f4a8c0ca0a0 2023-10-16T09:49:08.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.050+0000 7f4a91c59640 1 -- 172.21.15.84:0/3135012071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8c14b400 con 0x7f4a8c0ca0a0 2023-10-16T09:49:08.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.050+0000 7f4a897fa640 1 -- 172.21.15.84:0/3135012071 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a78012870 con 0x7f4a8c0ca0a0 2023-10-16T09:49:08.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.050+0000 7f4a91c59640 1 -- 172.21.15.84:0/3135012071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8c148ac0 con 0x7f4a8c0ca0a0 2023-10-16T09:49:08.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.051+0000 7f4a91c59640 1 -- 172.21.15.84:0/3135012071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8c148640 con 0x7f4a8c0ca0a0 2023-10-16T09:49:08.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.052+0000 7f4a897fa640 1 -- 172.21.15.84:0/3135012071 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4a780129d0 con 0x7f4a8c0ca0a0 2023-10-16T09:49:08.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.053+0000 7f4a897fa640 1 --2- 172.21.15.84:0/3135012071 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4a7006ae70 0x7f4a7006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:08.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.053+0000 7f4a897fa640 1 -- 172.21.15.84:0/3135012071 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a780891c0 con 0x7f4a8c0ca0a0 2023-10-16T09:49:08.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.053+0000 7f4a90c57640 1 --2- 172.21.15.84:0/3135012071 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4a7006ae70 0x7f4a7006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:08.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.054+0000 7f4a90c57640 1 --2- 172.21.15.84:0/3135012071 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4a7006ae70 0x7f4a7006d330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4a800011e0 tx=0x7f4a80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:08.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.057+0000 7f4a897fa640 1 -- 172.21.15.84:0/3135012071 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4a8c148640 con 0x7f4a8c0ca0a0 2023-10-16T09:49:08.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.296+0000 7f4a91c59640 1 -- 172.21.15.84:0/3135012071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f4a8c148640 con 0x7f4a8c0ca0a0 2023-10-16T09:49:08.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.301+0000 7f4a897fa640 1 -- 172.21.15.84:0/3135012071 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171029 (secure 0 0 0) 0x7f4a8c148640 con 0x7f4a8c0ca0a0 2023-10-16T09:49:08.305 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:08.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.306+0000 7f4a91c59640 1 -- 172.21.15.84:0/3135012071 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4a7006ae70 msgr2=0x7f4a7006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:08.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.306+0000 7f4a91c59640 1 --2- 172.21.15.84:0/3135012071 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4a7006ae70 0x7f4a7006d330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4a800011e0 tx=0x7f4a80009040 comp rx=0 tx=0).stop 2023-10-16T09:49:08.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.306+0000 7f4a91c59640 1 -- 172.21.15.84:0/3135012071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a8c0ca0a0 msgr2=0x7f4a8c1579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:08.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.306+0000 7f4a91c59640 1 --2- 172.21.15.84:0/3135012071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a8c0ca0a0 0x7f4a8c1579f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800c480 tx=0x7f4a7800c940 comp rx=0 tx=0).stop 2023-10-16T09:49:08.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.306+0000 7f4a91c59640 1 -- 172.21.15.84:0/3135012071 shutdown_connections 2023-10-16T09:49:08.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.306+0000 7f4a91c59640 1 --2- 172.21.15.84:0/3135012071 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4a7006ae70 0x7f4a7006d330 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.306+0000 7f4a91c59640 1 --2- 172.21.15.84:0/3135012071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8c159950 0x7f4a8c14a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.306+0000 7f4a91c59640 1 --2- 172.21.15.84:0/3135012071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a8c0ca0a0 0x7f4a8c1579f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.306+0000 7f4a91c59640 1 --2- 172.21.15.84:0/3135012071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a8c070b60 0x7f4a8c1574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.306+0000 7f4a91c59640 1 -- 172.21.15.84:0/3135012071 >> 172.21.15.84:0/3135012071 conn(0x7f4a8c0ce6a0 msgr2=0x7f4a8c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:08.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.306+0000 7f4a91c59640 1 -- 172.21.15.84:0/3135012071 shutdown_connections 2023-10-16T09:49:08.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.306+0000 7f4a91c59640 1 -- 172.21.15.84:0/3135012071 wait complete. 2023-10-16T09:49:08.325 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":5,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3010246666},{"type":"v1","addr":"172.21.15.84:6833","nonce":3010246666}]},"active_addr":"172.21.15.84:6833/3010246666","active_change":"2023-10-16T09:48:47.572733+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4114,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":1060139177}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":1628587200}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":3968394661}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":982275204}]}]} 2023-10-16T09:49:08.329 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-16T09:49:08.329 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-16T09:49:08.329 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:08.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.536+0000 7fc753fff640 1 Processor -- start 2023-10-16T09:49:08.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.536+0000 7fc753fff640 1 -- start start 2023-10-16T09:49:08.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.537+0000 7fc753fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc75406c040 0x7fc75406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:08.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.537+0000 7fc753fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7540ad970 0x7fc75415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:08.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.538+0000 7fc753fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc75406c960 con 0x7fc75415ebc0 2023-10-16T09:49:08.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.538+0000 7fc753fff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc75415b9d0 con 0x7fc75406c040 2023-10-16T09:49:08.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.538+0000 7fc753fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75415bcb0 con 0x7fc7540ad970 2023-10-16T09:49:08.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.538+0000 7fc7537fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc75415ebc0 0x7fc7540cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32774/0 (socket says 172.21.15.84:32774) 2023-10-16T09:49:08.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.538+0000 7fc7537fe640 1 -- 172.21.15.84:0/100708164 learned_addr learned my addr 172.21.15.84:0/100708164 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:08.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.538+0000 7fc7527fc640 1 --2- 172.21.15.84:0/100708164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7540ad970 0x7fc75415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:08.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.538+0000 7fc752ffd640 1 --2- 172.21.15.84:0/100708164 >> v2:172.21.15.157:3300/0 conn(0x7fc75406c040 0x7fc75406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:08.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.538+0000 7fc751ffb640 1 -- 172.21.15.84:0/100708164 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 32222772 0 0) 0x7fc75406c960 con 0x7fc75415ebc0 2023-10-16T09:49:08.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.538+0000 7fc751ffb640 1 -- 172.21.15.84:0/100708164 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc73c003520 con 0x7fc75415ebc0 2023-10-16T09:49:08.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.539+0000 7fc751ffb640 1 -- 172.21.15.84:0/100708164 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 787479458 0 0) 0x7fc73c003520 con 0x7fc75415ebc0 2023-10-16T09:49:08.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.539+0000 7fc751ffb640 1 -- 172.21.15.84:0/100708164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7540ad970 msgr2=0x7fc75415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:08.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.539+0000 7fc751ffb640 1 --2- 172.21.15.84:0/100708164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7540ad970 0x7fc75415e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.539+0000 7fc751ffb640 1 -- 172.21.15.84:0/100708164 >> v2:172.21.15.157:3300/0 conn(0x7fc75406c040 msgr2=0x7fc75406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:08.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.539+0000 7fc751ffb640 1 --2- 172.21.15.84:0/100708164 >> v2:172.21.15.157:3300/0 conn(0x7fc75406c040 0x7fc75406c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.539+0000 7fc751ffb640 1 -- 172.21.15.84:0/100708164 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75415bf90 con 0x7fc75415ebc0 2023-10-16T09:49:08.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.539+0000 7fc751ffb640 1 -- 172.21.15.84:0/100708164 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc74c0019f0 con 0x7fc75415ebc0 2023-10-16T09:49:08.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.539+0000 7fc751ffb640 1 -- 172.21.15.84:0/100708164 >> v1:172.21.15.84:6789/0 conn(0x7fc75415ebc0 legacy=0x7fc7540cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:08.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.539+0000 7fc751ffb640 1 --2- 172.21.15.84:0/100708164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73c003970 0x7fc73c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:08.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.539+0000 7fc751ffb640 1 --2- 172.21.15.84:0/100708164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7540ad970 0x7fc73c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:08.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.539+0000 7fc751ffb640 1 --2- 172.21.15.84:0/100708164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73c004d00 0x7fc73c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:08.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.539+0000 7fc751ffb640 1 -- 172.21.15.84:0/100708164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75415b9d0 con 0x7fc73c004d00 2023-10-16T09:49:08.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.539+0000 7fc751ffb640 1 -- 172.21.15.84:0/100708164 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75415bcb0 con 0x7fc73c003970 2023-10-16T09:49:08.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.539+0000 7fc751ffb640 1 -- 172.21.15.84:0/100708164 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c00b9b0 con 0x7fc7540ad970 2023-10-16T09:49:08.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.539+0000 7fc752ffd640 1 --2- 172.21.15.84:0/100708164 >> v2:172.21.15.157:3300/0 conn(0x7fc75406c040 0x7fc75406c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:08.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.539+0000 7fc7537fe640 1 --2- 172.21.15.84:0/100708164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73c004d00 0x7fc73c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:08.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.539+0000 7fc7527fc640 1 --2- 172.21.15.84:0/100708164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7540ad970 0x7fc73c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:08.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.540+0000 7fc7537fe640 1 -- 172.21.15.84:0/100708164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7540ad970 msgr2=0x7fc73c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:08.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.540+0000 7fc7537fe640 1 --2- 172.21.15.84:0/100708164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7540ad970 0x7fc73c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.540+0000 7fc7537fe640 1 -- 172.21.15.84:0/100708164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73c003970 msgr2=0x7fc73c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:08.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.540+0000 7fc7537fe640 1 --2- 172.21.15.84:0/100708164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73c003970 0x7fc73c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.540+0000 7fc7537fe640 1 -- 172.21.15.84:0/100708164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73c003520 con 0x7fc73c004d00 2023-10-16T09:49:08.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.541+0000 7fc7537fe640 1 --2- 172.21.15.84:0/100708164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73c004d00 0x7fc73c005180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00a830 tx=0x7fc73c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615b2ebccd47a418 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:08.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.542+0000 7fc751ffb640 1 -- 172.21.15.84:0/100708164 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0072d0 con 0x7fc73c004d00 2023-10-16T09:49:08.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.542+0000 7fc751ffb640 1 -- 172.21.15.84:0/100708164 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c02f070 con 0x7fc73c004d00 2023-10-16T09:49:08.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.542+0000 7fc751ffb640 1 -- 172.21.15.84:0/100708164 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c007690 con 0x7fc73c004d00 2023-10-16T09:49:08.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.542+0000 7fc753fff640 1 -- 172.21.15.84:0/100708164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73c004d00 msgr2=0x7fc73c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:08.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.542+0000 7fc753fff640 1 --2- 172.21.15.84:0/100708164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73c004d00 0x7fc73c005180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00a830 tx=0x7fc73c0057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.542+0000 7fc7537fe640 1 -- 172.21.15.84:0/100708164 reap_dead start 2023-10-16T09:49:08.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.542+0000 7fc753fff640 1 -- 172.21.15.84:0/100708164 shutdown_connections 2023-10-16T09:49:08.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.542+0000 7fc753fff640 1 -- 172.21.15.84:0/100708164 >> 172.21.15.84:0/100708164 conn(0x7fc7540ce6a0 msgr2=0x7fc7540ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:08.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.543+0000 7fc753fff640 1 -- 172.21.15.84:0/100708164 shutdown_connections 2023-10-16T09:49:08.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.543+0000 7fc753fff640 1 -- 172.21.15.84:0/100708164 wait complete. 2023-10-16T09:49:08.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.544+0000 7fc753fff640 1 Processor -- start 2023-10-16T09:49:08.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.544+0000 7fc753fff640 1 -- start start 2023-10-16T09:49:08.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.545+0000 7fc753fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73c004d00 0x7fc7540c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:08.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.545+0000 7fc753fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7540c96c0 0x7fc7540c2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:08.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.545+0000 7fc753fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7540c9b40 0x7fc7540ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:08.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.545+0000 7fc753fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7541502d0 con 0x7fc73c004d00 2023-10-16T09:49:08.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.545+0000 7fc753fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7541e5310 con 0x7fc7540c9b40 2023-10-16T09:49:08.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.545+0000 7fc753fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7541e4460 con 0x7fc7540c96c0 2023-10-16T09:49:08.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.546+0000 7fc752ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73c004d00 0x7fc7540c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:08.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.546+0000 7fc752ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73c004d00 0x7fc7540c4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59452/0 (socket says 172.21.15.84:59452) 2023-10-16T09:49:08.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.546+0000 7fc752ffd640 1 -- 172.21.15.84:0/1898420780 learned_addr learned my addr 172.21.15.84:0/1898420780 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:08.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.546+0000 7fc752ffd640 1 -- 172.21.15.84:0/1898420780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7540c96c0 msgr2=0x7fc7540c2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:08.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.546+0000 7fc752ffd640 1 --2- 172.21.15.84:0/1898420780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7540c96c0 0x7fc7540c2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.546+0000 7fc752ffd640 1 -- 172.21.15.84:0/1898420780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7540c9b40 msgr2=0x7fc7540ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:08.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.546+0000 7fc752ffd640 1 --2- 172.21.15.84:0/1898420780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7540c9b40 0x7fc7540ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.546+0000 7fc752ffd640 1 -- 172.21.15.84:0/1898420780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7541e4700 con 0x7fc73c004d00 2023-10-16T09:49:08.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.546+0000 7fc752ffd640 1 --2- 172.21.15.84:0/1898420780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73c004d00 0x7fc7540c4470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc74400e810 tx=0x7fc74400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:08.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.547+0000 7fc73b7fe640 1 -- 172.21.15.84:0/1898420780 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744004030 con 0x7fc73c004d00 2023-10-16T09:49:08.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.547+0000 7fc73b7fe640 1 -- 172.21.15.84:0/1898420780 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7440041b0 con 0x7fc73c004d00 2023-10-16T09:49:08.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.547+0000 7fc73b7fe640 1 -- 172.21.15.84:0/1898420780 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74400c650 con 0x7fc73c004d00 2023-10-16T09:49:08.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.547+0000 7fc753fff640 1 -- 172.21.15.84:0/1898420780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7541e49e0 con 0x7fc73c004d00 2023-10-16T09:49:08.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.547+0000 7fc753fff640 1 -- 172.21.15.84:0/1898420780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7541e4fa0 con 0x7fc73c004d00 2023-10-16T09:49:08.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.548+0000 7fc753fff640 1 -- 172.21.15.84:0/1898420780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc75406b330 con 0x7fc73c004d00 2023-10-16T09:49:08.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.549+0000 7fc73b7fe640 1 -- 172.21.15.84:0/1898420780 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc74400c7b0 con 0x7fc73c004d00 2023-10-16T09:49:08.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.551+0000 7fc73b7fe640 1 --2- 172.21.15.84:0/1898420780 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc74006ae70 0x7fc74006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:08.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.551+0000 7fc73b7fe640 1 -- 172.21.15.84:0/1898420780 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc744089270 con 0x7fc73c004d00 2023-10-16T09:49:08.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.552+0000 7fc7527fc640 1 --2- 172.21.15.84:0/1898420780 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc74006ae70 0x7fc74006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:08.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.552+0000 7fc7527fc640 1 --2- 172.21.15.84:0/1898420780 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc74006ae70 0x7fc74006d330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc7540c8510 tx=0x7fc748008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:08.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.556+0000 7fc73b7fe640 1 -- 172.21.15.84:0/1898420780 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc744054310 con 0x7fc73c004d00 2023-10-16T09:49:08.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.748+0000 7fc753fff640 1 -- 172.21.15.84:0/1898420780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc75404e9b0 con 0x7fc73c004d00 2023-10-16T09:49:08.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.751+0000 7fc73b7fe640 1 -- 172.21.15.84:0/1898420780 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fc74405ce60 con 0x7fc73c004d00 2023-10-16T09:49:08.753 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:08.753 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":21,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:03.529261+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:08.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.754+0000 7fc753fff640 1 -- 172.21.15.84:0/1898420780 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc74006ae70 msgr2=0x7fc74006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:08.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.754+0000 7fc753fff640 1 --2- 172.21.15.84:0/1898420780 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc74006ae70 0x7fc74006d330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc7540c8510 tx=0x7fc748008040 comp rx=0 tx=0).stop 2023-10-16T09:49:08.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.754+0000 7fc753fff640 1 -- 172.21.15.84:0/1898420780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73c004d00 msgr2=0x7fc7540c4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:08.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.754+0000 7fc753fff640 1 --2- 172.21.15.84:0/1898420780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73c004d00 0x7fc7540c4470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc74400e810 tx=0x7fc74400ecd0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.754+0000 7fc753fff640 1 -- 172.21.15.84:0/1898420780 shutdown_connections 2023-10-16T09:49:08.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.754+0000 7fc753fff640 1 --2- 172.21.15.84:0/1898420780 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc74006ae70 0x7fc74006d330 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.754+0000 7fc753fff640 1 --2- 172.21.15.84:0/1898420780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7540c9b40 0x7fc7540ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.754+0000 7fc753fff640 1 --2- 172.21.15.84:0/1898420780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7540c96c0 0x7fc7540c2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.754+0000 7fc753fff640 1 --2- 172.21.15.84:0/1898420780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73c004d00 0x7fc7540c4470 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.754+0000 7fc753fff640 1 -- 172.21.15.84:0/1898420780 >> 172.21.15.84:0/1898420780 conn(0x7fc7540ce6a0 msgr2=0x7fc75415c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:08.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.754+0000 7fc753fff640 1 -- 172.21.15.84:0/1898420780 shutdown_connections 2023-10-16T09:49:08.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.754+0000 7fc753fff640 1 -- 172.21.15.84:0/1898420780 wait complete. 2023-10-16T09:49:08.774 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-16T09:49:08.775 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:08.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.988+0000 7f9f3658a640 1 Processor -- start 2023-10-16T09:49:08.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.988+0000 7f9f3658a640 1 -- start start 2023-10-16T09:49:08.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.990+0000 7f9f3658a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9f30159b20 0x7f9f3006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:08.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.990+0000 7f9f3658a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9f30159f00 0x7f9f300b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:08.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.990+0000 7f9f3658a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f3006f7f0 con 0x7f9f3006c120 2023-10-16T09:49:08.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.990+0000 7f9f3658a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f3006c5e0 con 0x7f9f30159f00 2023-10-16T09:49:08.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.990+0000 7f9f3658a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f3006c8c0 con 0x7f9f30159b20 2023-10-16T09:49:08.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.990+0000 7f9f277fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9f30159f00 0x7f9f300b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:08.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.990+0000 7f9f277fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9f30159f00 0x7f9f300b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59460/0 (socket says 172.21.15.84:59460) 2023-10-16T09:49:08.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.990+0000 7f9f277fe640 1 -- 172.21.15.84:0/1889308719 learned_addr learned my addr 172.21.15.84:0/1889308719 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:08.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.991+0000 7f9f2ffff640 1 --2- 172.21.15.84:0/1889308719 >> v2:172.21.15.157:3300/0 conn(0x7f9f30159b20 0x7f9f3006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:08.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.991+0000 7f9f277fe640 1 -- 172.21.15.84:0/1889308719 >> v2:172.21.15.157:3300/0 conn(0x7f9f30159b20 msgr2=0x7f9f3006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:08.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.991+0000 7f9f277fe640 1 --2- 172.21.15.84:0/1889308719 >> v2:172.21.15.157:3300/0 conn(0x7f9f30159b20 0x7f9f3006bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.991+0000 7f9f277fe640 1 -- 172.21.15.84:0/1889308719 >> v1:172.21.15.84:6789/0 conn(0x7f9f3006c120 legacy=0x7f9f300cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:08.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.991+0000 7f9f2f7fe640 1 -- 172.21.15.84:0/1889308719 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2564094360 0 0) 0x7f9f3006f7f0 con 0x7f9f3006c120 2023-10-16T09:49:08.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.991+0000 7f9f277fe640 1 -- 172.21.15.84:0/1889308719 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3006cba0 con 0x7f9f30159f00 2023-10-16T09:49:08.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.992+0000 7f9f277fe640 1 --2- 172.21.15.84:0/1889308719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f30159f00 0x7f9f300b0410 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9f300c9340 tx=0x7f9f1802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9fe32d03780ef8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:08.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.992+0000 7f9f2f7fe640 1 -- 172.21.15.84:0/1889308719 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18030860 con 0x7f9f30159f00 2023-10-16T09:49:08.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.992+0000 7f9f2f7fe640 1 -- 172.21.15.84:0/1889308719 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f180309c0 con 0x7f9f30159f00 2023-10-16T09:49:08.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.992+0000 7f9f2f7fe640 1 -- 172.21.15.84:0/1889308719 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18030d50 con 0x7f9f30159f00 2023-10-16T09:49:08.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.993+0000 7f9f3658a640 1 -- 172.21.15.84:0/1889308719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f30159f00 msgr2=0x7f9f300b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:08.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.993+0000 7f9f3658a640 1 --2- 172.21.15.84:0/1889308719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f30159f00 0x7f9f300b0410 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9f300c9340 tx=0x7f9f1802b9a0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.993+0000 7f9f3658a640 1 -- 172.21.15.84:0/1889308719 shutdown_connections 2023-10-16T09:49:08.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.993+0000 7f9f3658a640 1 --2- 172.21.15.84:0/1889308719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f30159f00 0x7f9f300b0410 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.993+0000 7f9f3658a640 1 --2- 172.21.15.84:0/1889308719 >> v2:172.21.15.157:3300/0 conn(0x7f9f30159b20 0x7f9f3006bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:08.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.993+0000 7f9f3658a640 1 -- 172.21.15.84:0/1889308719 >> 172.21.15.84:0/1889308719 conn(0x7f9f300cb410 msgr2=0x7f9f300cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:08.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.993+0000 7f9f3658a640 1 -- 172.21.15.84:0/1889308719 shutdown_connections 2023-10-16T09:49:08.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.993+0000 7f9f3658a640 1 -- 172.21.15.84:0/1889308719 wait complete. 2023-10-16T09:49:08.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.995+0000 7f9f3658a640 1 Processor -- start 2023-10-16T09:49:08.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.995+0000 7f9f3658a640 1 -- start start 2023-10-16T09:49:09.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.996+0000 7f9f3658a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f3006c120 0x7f9f3015a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.996+0000 7f9f3658a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f30159b20 0x7f9f3014e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.996+0000 7f9f3658a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f30159f00 0x7f9f30070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.996+0000 7f9f3658a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300c44c0 con 0x7f9f30159f00 2023-10-16T09:49:09.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.996+0000 7f9f3658a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3015a820 con 0x7f9f3006c120 2023-10-16T09:49:09.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.996+0000 7f9f3658a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300c3840 con 0x7f9f30159b20 2023-10-16T09:49:09.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.996+0000 7f9f34b27640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f30159f00 0x7f9f30070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.996+0000 7f9f34b27640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f30159f00 0x7f9f30070090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59464/0 (socket says 172.21.15.84:59464) 2023-10-16T09:49:09.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.996+0000 7f9f34b27640 1 -- 172.21.15.84:0/25341737 learned_addr learned my addr 172.21.15.84:0/25341737 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:09.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.996+0000 7f9f277fe640 1 --2- 172.21.15.84:0/25341737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f30159b20 0x7f9f3014e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.996+0000 7f9f2ffff640 1 --2- 172.21.15.84:0/25341737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f3006c120 0x7f9f3015a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.996+0000 7f9f34b27640 1 -- 172.21.15.84:0/25341737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f30159b20 msgr2=0x7f9f3014e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.996+0000 7f9f34b27640 1 --2- 172.21.15.84:0/25341737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f30159b20 0x7f9f3014e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.996+0000 7f9f34b27640 1 -- 172.21.15.84:0/25341737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f3006c120 msgr2=0x7f9f3015a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.996+0000 7f9f34b27640 1 --2- 172.21.15.84:0/25341737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f3006c120 0x7f9f3015a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.996+0000 7f9f34b27640 1 -- 172.21.15.84:0/25341737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f180094d0 con 0x7f9f30159f00 2023-10-16T09:49:09.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.997+0000 7f9f34b27640 1 --2- 172.21.15.84:0/25341737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f30159f00 0x7f9f30070090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800c7b0 tx=0x7f9f2800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.997+0000 7f9f2d7fa640 1 -- 172.21.15.84:0/25341737 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f28003ba0 con 0x7f9f30159f00 2023-10-16T09:49:09.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.997+0000 7f9f2d7fa640 1 -- 172.21.15.84:0/25341737 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f28003d00 con 0x7f9f30159f00 2023-10-16T09:49:09.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.997+0000 7f9f2d7fa640 1 -- 172.21.15.84:0/25341737 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f28005270 con 0x7f9f30159f00 2023-10-16T09:49:09.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.997+0000 7f9f3658a640 1 -- 172.21.15.84:0/25341737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f300c3b20 con 0x7f9f30159f00 2023-10-16T09:49:09.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.997+0000 7f9f3658a640 1 -- 172.21.15.84:0/25341737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f300c40e0 con 0x7f9f30159f00 2023-10-16T09:49:09.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.998+0000 7f9f3658a640 1 -- 172.21.15.84:0/25341737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f3006bf90 con 0x7f9f30159f00 2023-10-16T09:49:09.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:08.999+0000 7f9f2d7fa640 1 -- 172.21.15.84:0/25341737 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9f280053d0 con 0x7f9f30159f00 2023-10-16T09:49:09.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.000+0000 7f9f2d7fa640 1 --2- 172.21.15.84:0/25341737 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9f1406ae70 0x7f9f1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.000+0000 7f9f2d7fa640 1 -- 172.21.15.84:0/25341737 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f280893b0 con 0x7f9f30159f00 2023-10-16T09:49:09.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.000+0000 7f9f2ffff640 1 --2- 172.21.15.84:0/25341737 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9f1406ae70 0x7f9f1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.001+0000 7f9f2ffff640 1 --2- 172.21.15.84:0/25341737 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9f1406ae70 0x7f9f1406d330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9f20006fd0 tx=0x7f9f20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.005+0000 7f9f2d7fa640 1 -- 172.21.15.84:0/25341737 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9f28054460 con 0x7f9f30159f00 2023-10-16T09:49:09.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.197+0000 7f9f3658a640 1 -- 172.21.15.84:0/25341737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f300c4600 con 0x7f9f30159f00 2023-10-16T09:49:09.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.199+0000 7f9f2d7fa640 1 -- 172.21.15.84:0/25341737 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f9f28057a80 con 0x7f9f30159f00 2023-10-16T09:49:09.200 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:09.200 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":21,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:03.529261+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:09.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.201+0000 7f9f3658a640 1 -- 172.21.15.84:0/25341737 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9f1406ae70 msgr2=0x7f9f1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.201+0000 7f9f3658a640 1 --2- 172.21.15.84:0/25341737 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9f1406ae70 0x7f9f1406d330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9f20006fd0 tx=0x7f9f20008040 comp rx=0 tx=0).stop 2023-10-16T09:49:09.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.201+0000 7f9f3658a640 1 -- 172.21.15.84:0/25341737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f30159f00 msgr2=0x7f9f30070090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.201+0000 7f9f3658a640 1 --2- 172.21.15.84:0/25341737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f30159f00 0x7f9f30070090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800c7b0 tx=0x7f9f2800cbb0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.202+0000 7f9f3658a640 1 -- 172.21.15.84:0/25341737 shutdown_connections 2023-10-16T09:49:09.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.202+0000 7f9f3658a640 1 --2- 172.21.15.84:0/25341737 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9f1406ae70 0x7f9f1406d330 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.202+0000 7f9f3658a640 1 --2- 172.21.15.84:0/25341737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f30159f00 0x7f9f30070090 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.202+0000 7f9f3658a640 1 --2- 172.21.15.84:0/25341737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f30159b20 0x7f9f3014e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.202+0000 7f9f3658a640 1 --2- 172.21.15.84:0/25341737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f3006c120 0x7f9f3015a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.202+0000 7f9f3658a640 1 -- 172.21.15.84:0/25341737 >> 172.21.15.84:0/25341737 conn(0x7f9f300cb410 msgr2=0x7f9f300cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:09.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.202+0000 7f9f3658a640 1 -- 172.21.15.84:0/25341737 shutdown_connections 2023-10-16T09:49:09.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.202+0000 7f9f3658a640 1 -- 172.21.15.84:0/25341737 wait complete. 2023-10-16T09:49:09.220 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-16T09:49:09.221 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-16T09:49:09.221 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-16T09:49:09.222 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-16T09:49:09.222 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-16T09:49:09.223 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-16T09:49:09.223 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-16T09:49:09.224 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-16T09:49:09.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.526+0000 7f050ec90640 1 Processor -- start 2023-10-16T09:49:09.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.526+0000 7f050ec90640 1 -- start start 2023-10-16T09:49:09.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.526+0000 7fd003ee0640 1 Processor -- start 2023-10-16T09:49:09.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.527+0000 7fd003ee0640 1 -- start start 2023-10-16T09:49:09.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.527+0000 7f050ec90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05000e18f0 0x7f05000062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.528+0000 7f050ec90640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f05000e1cd0 0x7f05000eb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.528+0000 7f050ec90640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0500006b20 con 0x7f0500038630 2023-10-16T09:49:09.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.528+0000 7f050ec90640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0500006e00 con 0x7f05000e1cd0 2023-10-16T09:49:09.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.528+0000 7f050ec90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05000070e0 con 0x7f05000e18f0 2023-10-16T09:49:09.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.528+0000 7f050dc8e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05000e18f0 0x7f05000062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.528+0000 7fc02df78640 1 Processor -- start 2023-10-16T09:49:09.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.528+0000 7fc02df78640 1 -- start start 2023-10-16T09:49:09.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.528+0000 7f050dc8e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05000e18f0 0x7f05000062d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:36666/0 (socket says 172.21.15.84:36666) 2023-10-16T09:49:09.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.528+0000 7f050dc8e640 1 -- 172.21.15.84:0/3224320408 learned_addr learned my addr 172.21.15.84:0/3224320408 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:09.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.529+0000 7fd003ee0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcffc0b4580 0x7fcffc0b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.529+0000 7fd003ee0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcffc0b5c30 0x7fcffc0b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.529+0000 7fd003ee0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcffc0b4f20 con 0x7fcffc15cf10 2023-10-16T09:49:09.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.529+0000 7fd003ee0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcffc0d2250 con 0x7fcffc0b5c30 2023-10-16T09:49:09.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.529+0000 7fd003ee0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc0d1640 con 0x7fcffc0b4580 2023-10-16T09:49:09.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.530+0000 7fd001c7c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcffc15cf10 0x7fcffc0d0b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:57360/0 (socket says 172.21.15.84:57360) 2023-10-16T09:49:09.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.530+0000 7fc02df78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0280c1dd0 0x7fc0280ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.530+0000 7fd00147b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcffc0b4580 0x7fcffc0b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.531+0000 7fc02df78640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0280c8dd0 con 0x7fc0280adc70 2023-10-16T09:49:09.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.531+0000 7fc02df78640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0280cd880 con 0x7fc02806fb70 2023-10-16T09:49:09.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.531+0000 7fc02df78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0280cdb60 con 0x7fc0280c1dd0 2023-10-16T09:49:09.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.531+0000 7fd00147b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcffc0b4580 0x7fcffc0b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:36670/0 (socket says 172.21.15.84:36670) 2023-10-16T09:49:09.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.531+0000 7fd00147b640 1 -- 172.21.15.84:0/2794796356 learned_addr learned my addr 172.21.15.84:0/2794796356 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:09.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.531+0000 7fd00247d640 1 --2- 172.21.15.84:0/2794796356 >> v2:172.21.15.84:3300/0 conn(0x7fcffc0b5c30 0x7fcffc0b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.530+0000 7f050dc8e640 1 -- 172.21.15.84:0/3224320408 >> v2:172.21.15.157:3300/0 conn(0x7f05000e1cd0 msgr2=0x7f05000eb170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T09:49:09.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.530+0000 7f050dc8e640 1 --2- 172.21.15.84:0/3224320408 >> v2:172.21.15.157:3300/0 conn(0x7f05000e1cd0 0x7f05000eb170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.530+0000 7f050dc8e640 1 -- 172.21.15.84:0/3224320408 >> v1:172.21.15.84:6789/0 conn(0x7f0500038630 legacy=0x7f0500057cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.531+0000 7fc0277fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0280c1dd0 0x7fc0280ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.530+0000 7f050dc8e640 1 -- 172.21.15.84:0/3224320408 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05000073c0 con 0x7f05000e18f0 2023-10-16T09:49:09.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.531+0000 7fc0277fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0280c1dd0 0x7fc0280ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:36684/0 (socket says 172.21.15.84:36684) 2023-10-16T09:49:09.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.531+0000 7fc026ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc0280adc70 0x7fc02806f460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32800/0 (socket says 172.21.15.84:32800) 2023-10-16T09:49:09.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.532+0000 7fc0277fe640 1 -- 172.21.15.84:0/116239214 learned_addr learned my addr 172.21.15.84:0/116239214 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:09.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.532+0000 7f050dc8e640 1 --2- 172.21.15.84:0/3224320408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05000e18f0 0x7f05000062d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0504009870 tx=0x7f050402b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf3976e64e7a061 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.532+0000 7fd00247d640 1 -- 172.21.15.84:0/2794796356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcffc0b4580 msgr2=0x7fcffc0b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:09.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.532+0000 7fd00247d640 1 --2- 172.21.15.84:0/2794796356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcffc0b4580 0x7fcffc0b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.532+0000 7fd00247d640 1 -- 172.21.15.84:0/2794796356 >> v1:172.21.15.157:6789/0 conn(0x7fcffc15cf10 legacy=0x7fcffc0d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.532+0000 7fd00247d640 1 -- 172.21.15.84:0/2794796356 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcffc0d1920 con 0x7fcffc0b5c30 2023-10-16T09:49:09.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.532+0000 7fc0267fc640 1 -- 172.21.15.84:0/116239214 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498685604 0 0) 0x7fc0280c8dd0 con 0x7fc0280adc70 2023-10-16T09:49:09.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.532+0000 7fc0267fc640 1 -- 172.21.15.84:0/116239214 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc010003520 con 0x7fc0280adc70 2023-10-16T09:49:09.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.533+0000 7f050cc8c640 1 -- 172.21.15.84:0/3224320408 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0504030860 con 0x7f05000e18f0 2023-10-16T09:49:09.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.533+0000 7f050cc8c640 1 -- 172.21.15.84:0/3224320408 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05040309c0 con 0x7f05000e18f0 2023-10-16T09:49:09.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.533+0000 7f050cc8c640 1 -- 172.21.15.84:0/3224320408 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0504030d50 con 0x7f05000e18f0 2023-10-16T09:49:09.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.533+0000 7f050ec90640 1 -- 172.21.15.84:0/3224320408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05000e18f0 msgr2=0x7f05000062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.533+0000 7f050ec90640 1 --2- 172.21.15.84:0/3224320408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05000e18f0 0x7f05000062d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0504009870 tx=0x7f050402b9a0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.534+0000 7fc0277fe640 1 -- 172.21.15.84:0/116239214 >> v1:172.21.15.157:6789/0 conn(0x7fc02806fb70 legacy=0x7fc0280c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.534+0000 7fc0277fe640 1 -- 172.21.15.84:0/116239214 >> v1:172.21.15.84:6789/0 conn(0x7fc0280adc70 legacy=0x7fc02806f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.534+0000 7fc0277fe640 1 -- 172.21.15.84:0/116239214 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0280c9070 con 0x7fc0280c1dd0 2023-10-16T09:49:09.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.534+0000 7f050ec90640 1 -- 172.21.15.84:0/3224320408 shutdown_connections 2023-10-16T09:49:09.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.534+0000 7f050ec90640 1 --2- 172.21.15.84:0/3224320408 >> v2:172.21.15.157:3300/0 conn(0x7f05000e1cd0 0x7f05000eb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.534+0000 7f050ec90640 1 --2- 172.21.15.84:0/3224320408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05000e18f0 0x7f05000062d0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0504009870 tx=0x7f050402b9a0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.534+0000 7f050ec90640 1 -- 172.21.15.84:0/3224320408 >> 172.21.15.84:0/3224320408 conn(0x7f05000590c0 msgr2=0x7f05000594d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:09.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.535+0000 7f050ec90640 1 -- 172.21.15.84:0/3224320408 shutdown_connections 2023-10-16T09:49:09.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.535+0000 7f050ec90640 1 -- 172.21.15.84:0/3224320408 wait complete. 2023-10-16T09:49:09.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.535+0000 7fd00247d640 1 --2- 172.21.15.84:0/2794796356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcffc0b5c30 0x7fcffc0b60c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009d90 tx=0x7fcff402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5edcb99e5efc262 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.535+0000 7fab768de640 1 Processor -- start 2023-10-16T09:49:09.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.535+0000 7fd000c7a640 1 -- 172.21.15.84:0/2794796356 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff402cc50 con 0x7fcffc0b5c30 2023-10-16T09:49:09.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.535+0000 7fd000c7a640 1 -- 172.21.15.84:0/2794796356 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff402cdb0 con 0x7fcffc0b5c30 2023-10-16T09:49:09.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.535+0000 7fd000c7a640 1 -- 172.21.15.84:0/2794796356 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff4034660 con 0x7fcffc0b5c30 2023-10-16T09:49:09.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.535+0000 7fc0277fe640 1 --2- 172.21.15.84:0/116239214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0280c1dd0 0x7fc0280ccfa0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc0180098a0 tx=0x7fc01802ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=362a428cc8461e9e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.535+0000 7fab768de640 1 -- start start 2023-10-16T09:49:09.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.536+0000 7fc0267fc640 1 -- 172.21.15.84:0/116239214 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01802f830 con 0x7fc0280c1dd0 2023-10-16T09:49:09.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.536+0000 7fc0267fc640 1 -- 172.21.15.84:0/116239214 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01802f990 con 0x7fc0280c1dd0 2023-10-16T09:49:09.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.536+0000 7fc0267fc640 1 -- 172.21.15.84:0/116239214 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01802fd00 con 0x7fc0280c1dd0 2023-10-16T09:49:09.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.536+0000 7fd003ee0640 1 -- 172.21.15.84:0/2794796356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcffc0b5c30 msgr2=0x7fcffc0b60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.536+0000 7fd003ee0640 1 --2- 172.21.15.84:0/2794796356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcffc0b5c30 0x7fcffc0b60c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009d90 tx=0x7fcff402bea0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.536+0000 7f2ba2478640 1 Processor -- start 2023-10-16T09:49:09.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.537+0000 7fc02df78640 1 -- 172.21.15.84:0/116239214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0280c1dd0 msgr2=0x7fc0280ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.537+0000 7fc02df78640 1 --2- 172.21.15.84:0/116239214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0280c1dd0 0x7fc0280ccfa0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc0180098a0 tx=0x7fc01802ba10 comp rx=0 tx=0).stop 2023-10-16T09:49:09.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.537+0000 7f050ec90640 1 Processor -- start 2023-10-16T09:49:09.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.537+0000 7fab768de640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fab680e78e0 0x7fab6804f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.537+0000 7fab768de640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fab680eb2f0 0x7fab680eb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.537+0000 7fab768de640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab68053010 0x7fab680534a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.537+0000 7fab768de640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab68050370 con 0x7fab680eb2f0 2023-10-16T09:49:09.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.537+0000 7fab768de640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab680064b0 con 0x7fab680e78e0 2023-10-16T09:49:09.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.537+0000 7fab768de640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab680ebc70 con 0x7fab68053010 2023-10-16T09:49:09.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.537+0000 7f2ba2478640 1 -- start start 2023-10-16T09:49:09.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.537+0000 7fe452901640 1 Processor -- start 2023-10-16T09:49:09.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.538+0000 7f050ec90640 1 -- start start 2023-10-16T09:49:09.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.538+0000 7fab6f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fab680eb2f0 0x7fab680eb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.538+0000 7fc02df78640 1 -- 172.21.15.84:0/116239214 shutdown_connections 2023-10-16T09:49:09.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.538+0000 7fc02df78640 1 --2- 172.21.15.84:0/116239214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0280c1dd0 0x7fc0280ccfa0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.538+0000 7fc02df78640 1 -- 172.21.15.84:0/116239214 >> 172.21.15.84:0/116239214 conn(0x7fc0280ce6e0 msgr2=0x7fc0280ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:09.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.538+0000 7fab6f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fab680eb2f0 0x7fab680eb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59476/0 (socket says 172.21.15.84:59476) 2023-10-16T09:49:09.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.538+0000 7fab6f7fe640 1 -- 172.21.15.84:0/1281630272 learned_addr learned my addr 172.21.15.84:0/1281630272 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:09.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.538+0000 7fab6ffff640 1 --2- 172.21.15.84:0/1281630272 >> v2:172.21.15.157:3300/0 conn(0x7fab680e78e0 0x7fab6804f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.538+0000 7fc02df78640 1 -- 172.21.15.84:0/116239214 shutdown_connections 2023-10-16T09:49:09.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.538+0000 7fd003ee0640 1 -- 172.21.15.84:0/2794796356 shutdown_connections 2023-10-16T09:49:09.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.538+0000 7fd003ee0640 1 --2- 172.21.15.84:0/2794796356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcffc0b5c30 0x7fcffc0b60c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.538+0000 7fd003ee0640 1 --2- 172.21.15.84:0/2794796356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcffc0b4580 0x7fcffc0b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.538+0000 7fd003ee0640 1 -- 172.21.15.84:0/2794796356 >> 172.21.15.84:0/2794796356 conn(0x7fcffc0cf580 msgr2=0x7fcffc0cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:09.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.538+0000 7fab74e7b640 1 --2- 172.21.15.84:0/1281630272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab68053010 0x7fab680534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.538+0000 7fc02df78640 1 -- 172.21.15.84:0/116239214 wait complete. 2023-10-16T09:49:09.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.539+0000 7f2ba2478640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2b9c0cded0 0x7f2b9c0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.539+0000 7f2ba2478640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b9c15d150 con 0x7f2b9c15cd70 2023-10-16T09:49:09.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.539+0000 7f2ba2478640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b9c0d1180 con 0x7f2b9c0b8480 2023-10-16T09:49:09.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.539+0000 7f2ba2478640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9c0d1460 con 0x7f2b9c0cded0 2023-10-16T09:49:09.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.539+0000 7fd003ee0640 1 -- 172.21.15.84:0/2794796356 shutdown_connections 2023-10-16T09:49:09.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.539+0000 7f2ba0c75640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2b9c0b8480 0x7f2b9c0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:57384/0 (socket says 172.21.15.84:57384) 2023-10-16T09:49:09.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.539+0000 7f2ba0c75640 1 -- 172.21.15.84:0/277629916 learned_addr learned my addr 172.21.15.84:0/277629916 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:09.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.539+0000 7f050ec90640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0500038630 0x7f05000cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.539+0000 7f050ec90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05000e1cd0 0x7f050003de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.539+0000 7f050ec90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05000cd8b0 0x7f050003e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.539+0000 7f050ec90640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05000e9b60 con 0x7f0500038630 2023-10-16T09:49:09.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.539+0000 7f050ec90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05000e8cb0 con 0x7f05000cd8b0 2023-10-16T09:49:09.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.539+0000 7f050ec90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05000e8f50 con 0x7f05000e1cd0 2023-10-16T09:49:09.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.540+0000 7f050d48d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05000e1cd0 0x7f050003de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.540+0000 7f050d48d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05000e1cd0 0x7f050003de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:36708/0 (socket says 172.21.15.84:36708) 2023-10-16T09:49:09.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.540+0000 7f050d48d640 1 -- 172.21.15.84:0/457905920 learned_addr learned my addr 172.21.15.84:0/457905920 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:09.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.540+0000 7f050dc8e640 1 --2- 172.21.15.84:0/457905920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0500038630 0x7f05000cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.540+0000 7f050e48f640 1 --2- 172.21.15.84:0/457905920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05000cd8b0 0x7f050003e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.538+0000 7fe452901640 1 -- start start 2023-10-16T09:49:09.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.540+0000 7fab6f7fe640 1 -- 172.21.15.84:0/1281630272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab68053010 msgr2=0x7fab680534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:09.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.540+0000 7fab6f7fe640 1 --2- 172.21.15.84:0/1281630272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab68053010 0x7fab680534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.540+0000 7fe452901640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe44c0caa80 0x7fe44c069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.540+0000 7fe452901640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe44c0caec0 con 0x7fe44c06a2e0 2023-10-16T09:49:09.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.540+0000 7fe452901640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe44c0cb140 con 0x7fe44c0addb0 2023-10-16T09:49:09.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.540+0000 7fe452901640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c161a50 con 0x7fe44c0caa80 2023-10-16T09:49:09.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.540+0000 7fab6f7fe640 1 -- 172.21.15.84:0/1281630272 >> v2:172.21.15.157:3300/0 conn(0x7fab680e78e0 msgr2=0x7fab6804f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:09.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.540+0000 7fab6f7fe640 1 --2- 172.21.15.84:0/1281630272 >> v2:172.21.15.157:3300/0 conn(0x7fab680e78e0 0x7fab6804f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.540+0000 7fab6f7fe640 1 -- 172.21.15.84:0/1281630272 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6804fdb0 con 0x7fab680eb2f0 2023-10-16T09:49:09.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.540+0000 7fab6ffff640 1 --2- 172.21.15.84:0/1281630272 >> v2:172.21.15.157:3300/0 conn(0x7fab680e78e0 0x7fab6804f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:09.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.539+0000 7fd003ee0640 1 -- 172.21.15.84:0/2794796356 wait complete. 2023-10-16T09:49:09.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.540+0000 7f2ba1c77640 1 --2- 172.21.15.84:0/277629916 >> v2:172.21.15.84:3300/0 conn(0x7f2b9c0cded0 0x7f2b9c0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.541+0000 7fe44b7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe44c0caa80 0x7fe44c069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.541+0000 7fe450e9e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe44c06a2e0 0x7fe44c1650d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32822/0 (socket says 172.21.15.84:32822) 2023-10-16T09:49:09.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.541+0000 7fe44b7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe44c0caa80 0x7fe44c069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:36716/0 (socket says 172.21.15.84:36716) 2023-10-16T09:49:09.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.541+0000 7fe450e9e640 1 -- 172.21.15.84:0/474467385 learned_addr learned my addr 172.21.15.84:0/474467385 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:09.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.541+0000 7fc02df78640 1 Processor -- start 2023-10-16T09:49:09.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.541+0000 7f2b93fff640 1 -- 172.21.15.84:0/277629916 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1299820071 0 0) 0x7f2b9c0d1180 con 0x7f2b9c0b8480 2023-10-16T09:49:09.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.542+0000 7fc02df78640 1 -- start start 2023-10-16T09:49:09.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.542+0000 7f2b93fff640 1 -- 172.21.15.84:0/277629916 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b88003520 con 0x7f2b9c0b8480 2023-10-16T09:49:09.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.542+0000 7fe44b7fe640 1 -- 172.21.15.84:0/474467385 >> v1:172.21.15.157:6789/0 conn(0x7fe44c0addb0 legacy=0x7fe44c0c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.542+0000 7fe44b7fe640 1 -- 172.21.15.84:0/474467385 >> v1:172.21.15.84:6789/0 conn(0x7fe44c06a2e0 legacy=0x7fe44c1650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.542+0000 7fe44b7fe640 1 -- 172.21.15.84:0/474467385 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c06b030 con 0x7fe44c0caa80 2023-10-16T09:49:09.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.542+0000 7fab6f7fe640 1 --2- 172.21.15.84:0/1281630272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab680eb2f0 0x7fab680eb730 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fab6800bda0 tx=0x7fab5c02b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36de18ef8331bc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.542+0000 7f050dc8e640 1 -- 172.21.15.84:0/457905920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05000e1cd0 msgr2=0x7f050003de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.542+0000 7f050dc8e640 1 --2- 172.21.15.84:0/457905920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05000e1cd0 0x7f050003de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.542+0000 7f050dc8e640 1 -- 172.21.15.84:0/457905920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05000cd8b0 msgr2=0x7f050003e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.542+0000 7f050dc8e640 1 --2- 172.21.15.84:0/457905920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05000cd8b0 0x7f050003e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.542+0000 7f050dc8e640 1 -- 172.21.15.84:0/457905920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05040094d0 con 0x7f0500038630 2023-10-16T09:49:09.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.543+0000 7f050dc8e640 1 --2- 172.21.15.84:0/457905920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0500038630 0x7f05000cd370 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0504033000 tx=0x7f0504004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.543+0000 7fc02df78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc02806fb70 0x7fc0281f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.543+0000 7fc02df78640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0280adc70 0x7fc028161a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.543+0000 7f2ba1c77640 1 -- 172.21.15.84:0/277629916 >> v1:172.21.15.157:6789/0 conn(0x7f2b9c0b8480 legacy=0x7f2b9c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.543+0000 7f2ba1c77640 1 -- 172.21.15.84:0/277629916 >> v1:172.21.15.84:6789/0 conn(0x7f2b9c15cd70 legacy=0x7f2b9c0d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.543+0000 7f2ba1c77640 1 -- 172.21.15.84:0/277629916 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9c0d1740 con 0x7f2b9c0cded0 2023-10-16T09:49:09.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.543+0000 7fc02df78640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0280c1dd0 0x7fc0280b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.543+0000 7fc02df78640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0281617e0 con 0x7fc0280c1dd0 2023-10-16T09:49:09.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.543+0000 7fc02df78640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0281620e0 con 0x7fc0280adc70 2023-10-16T09:49:09.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.543+0000 7fc02df78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0281623c0 con 0x7fc02806fb70 2023-10-16T09:49:09.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.544+0000 7fc026ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0280adc70 0x7fc028161a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.544+0000 7fd003ee0640 1 Processor -- start 2023-10-16T09:49:09.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.544+0000 7f04fe7fc640 1 -- 172.21.15.84:0/457905920 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0504047070 con 0x7f0500038630 2023-10-16T09:49:09.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.544+0000 7f04fe7fc640 1 -- 172.21.15.84:0/457905920 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0504039d90 con 0x7f0500038630 2023-10-16T09:49:09.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.544+0000 7f04fe7fc640 1 -- 172.21.15.84:0/457905920 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0504043620 con 0x7f0500038630 2023-10-16T09:49:09.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.544+0000 7fc027fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0280c1dd0 0x7fc0280b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.544+0000 7fc026ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0280adc70 0x7fc028161a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:54964/0 (socket says 172.21.15.84:54964) 2023-10-16T09:49:09.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.544+0000 7fc026ffd640 1 -- 172.21.15.84:0/272379975 learned_addr learned my addr 172.21.15.84:0/272379975 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:09.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.544+0000 7fc0277fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc02806fb70 0x7fc0281f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.544+0000 7fd003ee0640 1 -- start start 2023-10-16T09:49:09.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.544+0000 7f050ec90640 1 -- 172.21.15.84:0/457905920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05000e9230 con 0x7f0500038630 2023-10-16T09:49:09.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.544+0000 7f050ec90640 1 -- 172.21.15.84:0/457905920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05000e97f0 con 0x7f0500038630 2023-10-16T09:49:09.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.545+0000 7fc0277fe640 1 -- 172.21.15.84:0/272379975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0280adc70 msgr2=0x7fc028161a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.545+0000 7fc0277fe640 1 --2- 172.21.15.84:0/272379975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0280adc70 0x7fc028161a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.545+0000 7fc0277fe640 1 -- 172.21.15.84:0/272379975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0280c1dd0 msgr2=0x7fc0280b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.545+0000 7fc0277fe640 1 --2- 172.21.15.84:0/272379975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0280c1dd0 0x7fc0280b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.545+0000 7fc0277fe640 1 -- 172.21.15.84:0/272379975 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0180094d0 con 0x7fc02806fb70 2023-10-16T09:49:09.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.544+0000 7fe44b7fe640 1 --2- 172.21.15.84:0/474467385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe44c0caa80 0x7fe44c069da0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4400097d0 tx=0x7fe44002b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd92f1827e136d76 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.546+0000 7fd003ee0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcffc0b4580 0x7fcffc15c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.545+0000 7f2ba1c77640 1 --2- 172.21.15.84:0/277629916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b9c0cded0 0x7f2b9c0b5c90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0097a0 tx=0x7f2b8c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51829c61e5c8b489 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.545+0000 7fe44affd640 1 -- 172.21.15.84:0/474467385 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440030800 con 0x7fe44c0caa80 2023-10-16T09:49:09.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.545+0000 7fe44affd640 1 -- 172.21.15.84:0/474467385 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe440030960 con 0x7fe44c0caa80 2023-10-16T09:49:09.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.545+0000 7fe44affd640 1 -- 172.21.15.84:0/474467385 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440030cf0 con 0x7fe44c0caa80 2023-10-16T09:49:09.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.546+0000 7fd003ee0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcffc0b5c30 0x7fcffc0c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.546+0000 7fe452901640 1 -- 172.21.15.84:0/474467385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe44c0caa80 msgr2=0x7fe44c069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.546+0000 7fe452901640 1 --2- 172.21.15.84:0/474467385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe44c0caa80 0x7fe44c069da0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4400097d0 tx=0x7fe44002b940 comp rx=0 tx=0).stop 2023-10-16T09:49:09.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.546+0000 7f050ec90640 1 -- 172.21.15.84:0/457905920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0500007bc0 con 0x7f0500038630 2023-10-16T09:49:09.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.546+0000 7fd003ee0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcffc15cf10 0x7fcffc0c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.546+0000 7fd003ee0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc0c87a0 con 0x7fcffc0b5c30 2023-10-16T09:49:09.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.546+0000 7fd003ee0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc1490d0 con 0x7fcffc0b4580 2023-10-16T09:49:09.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.546+0000 7fd003ee0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc1493b0 con 0x7fcffc15cf10 2023-10-16T09:49:09.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.546+0000 7fe452901640 1 -- 172.21.15.84:0/474467385 shutdown_connections 2023-10-16T09:49:09.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.546+0000 7fe452901640 1 --2- 172.21.15.84:0/474467385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe44c0caa80 0x7fe44c069da0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.546+0000 7fe452901640 1 -- 172.21.15.84:0/474467385 >> 172.21.15.84:0/474467385 conn(0x7fe44c0ce9b0 msgr2=0x7fe44c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:09.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.547+0000 7fe452901640 1 -- 172.21.15.84:0/474467385 shutdown_connections 2023-10-16T09:49:09.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.547+0000 7fe452901640 1 -- 172.21.15.84:0/474467385 wait complete. 2023-10-16T09:49:09.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.547+0000 7f4125dc9640 1 Processor -- start 2023-10-16T09:49:09.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.547+0000 7fd001c7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcffc0b4580 0x7fcffc15c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.547+0000 7fd00147b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcffc0b5c30 0x7fcffc0c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.547+0000 7f4125dc9640 1 -- start start 2023-10-16T09:49:09.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.546+0000 7fc0277fe640 1 --2- 172.21.15.84:0/272379975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc02806fb70 0x7fc0281f6620 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc018033000 tx=0x7fc0180046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.547+0000 7fd001c7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcffc0b4580 0x7fcffc15c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:54972/0 (socket says 172.21.15.84:54972) 2023-10-16T09:49:09.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.547+0000 7fab6effd640 1 -- 172.21.15.84:0/1281630272 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0317f0 con 0x7fab680eb2f0 2023-10-16T09:49:09.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.547+0000 7fd001c7c640 1 -- 172.21.15.84:0/3984178739 learned_addr learned my addr 172.21.15.84:0/3984178739 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:09.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.547+0000 7fd00247d640 1 --2- 172.21.15.84:0/3984178739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcffc15cf10 0x7fcffc0c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.548+0000 7fd001c7c640 1 -- 172.21.15.84:0/3984178739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcffc15cf10 msgr2=0x7fcffc0c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.547+0000 7fab6effd640 1 -- 172.21.15.84:0/1281630272 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c031950 con 0x7fab680eb2f0 2023-10-16T09:49:09.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.547+0000 7f9d789ce640 1 Processor -- start 2023-10-16T09:49:09.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.547+0000 7fab6effd640 1 -- 172.21.15.84:0/1281630272 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c031ce0 con 0x7fab680eb2f0 2023-10-16T09:49:09.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.548+0000 7fd001c7c640 1 --2- 172.21.15.84:0/3984178739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcffc15cf10 0x7fcffc0c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.548+0000 7fd001c7c640 1 -- 172.21.15.84:0/3984178739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcffc0b5c30 msgr2=0x7fcffc0c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.548+0000 7fd001c7c640 1 --2- 172.21.15.84:0/3984178739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcffc0b5c30 0x7fcffc0c65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.548+0000 7fd001c7c640 1 -- 172.21.15.84:0/3984178739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff4009a70 con 0x7fcffc0b4580 2023-10-16T09:49:09.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.548+0000 7fd00147b640 1 --2- 172.21.15.84:0/3984178739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcffc0b5c30 0x7fcffc0c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:49:09.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.548+0000 7fab768de640 1 -- 172.21.15.84:0/1281630272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab680eb2f0 msgr2=0x7fab680eb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.548+0000 7fab768de640 1 --2- 172.21.15.84:0/1281630272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab680eb2f0 0x7fab680eb730 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fab6800bda0 tx=0x7fab5c02b890 comp rx=0 tx=0).stop 2023-10-16T09:49:09.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.548+0000 7f9d789ce640 1 -- start start 2023-10-16T09:49:09.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7fab768de640 1 -- 172.21.15.84:0/1281630272 shutdown_connections 2023-10-16T09:49:09.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7fab768de640 1 --2- 172.21.15.84:0/1281630272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab680eb2f0 0x7fab680eb730 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7fab768de640 1 --2- 172.21.15.84:0/1281630272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab68053010 0x7fab680534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7fab768de640 1 --2- 172.21.15.84:0/1281630272 >> v2:172.21.15.157:3300/0 conn(0x7fab680e78e0 0x7fab6804f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7f4125dc9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f41200cf710 0x7f41200b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7fab768de640 1 -- 172.21.15.84:0/1281630272 >> 172.21.15.84:0/1281630272 conn(0x7fab68054c40 msgr2=0x7fab68055020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:09.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7f4125dc9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41200d18c0 con 0x7f41200b6cd0 2023-10-16T09:49:09.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7fe452901640 1 Processor -- start 2023-10-16T09:49:09.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7fab768de640 1 -- 172.21.15.84:0/1281630272 shutdown_connections 2023-10-16T09:49:09.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7f4125dc9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41200d3980 con 0x7f41200cfaf0 2023-10-16T09:49:09.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7f4125dc9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41200d3c60 con 0x7f41200cf710 2023-10-16T09:49:09.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7fe452901640 1 -- start start 2023-10-16T09:49:09.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7f411f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f41200cf710 0x7f41200b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7f411f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f41200cf710 0x7f41200b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59526/0 (socket says 172.21.15.84:59526) 2023-10-16T09:49:09.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7f411f7fe640 1 -- 172.21.15.84:0/2054013373 learned_addr learned my addr 172.21.15.84:0/2054013373 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:09.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7f2b93fff640 1 -- 172.21.15.84:0/277629916 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c030610 con 0x7f2b9c0cded0 2023-10-16T09:49:09.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7f2b93fff640 1 -- 172.21.15.84:0/277629916 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8c030770 con 0x7f2b9c0cded0 2023-10-16T09:49:09.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7f2b93fff640 1 -- 172.21.15.84:0/277629916 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c030b00 con 0x7f2b9c0cded0 2023-10-16T09:49:09.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.549+0000 7fd001c7c640 1 --2- 172.21.15.84:0/3984178739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcffc0b4580 0x7fcffc15c360 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcff800c3e0 tx=0x7fcff800c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.550+0000 7fab768de640 1 -- 172.21.15.84:0/1281630272 wait complete. 2023-10-16T09:49:09.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.550+0000 7f411f7fe640 1 -- 172.21.15.84:0/2054013373 >> v1:172.21.15.157:6789/0 conn(0x7f41200cfaf0 legacy=0x7f41200d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:09.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.550+0000 7f411f7fe640 1 -- 172.21.15.84:0/2054013373 >> v1:172.21.15.84:6789/0 conn(0x7f41200b6cd0 legacy=0x7f41200d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:09.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.550+0000 7f411f7fe640 1 -- 172.21.15.84:0/2054013373 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41200ce630 con 0x7f41200cf710 2023-10-16T09:49:09.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.550+0000 7f2ba2478640 1 -- 172.21.15.84:0/277629916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b9c0cded0 msgr2=0x7f2b9c0b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.550+0000 7f2ba2478640 1 --2- 172.21.15.84:0/277629916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b9c0cded0 0x7f2b9c0b5c90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0097a0 tx=0x7f2b8c02b960 comp rx=0 tx=0).stop 2023-10-16T09:49:09.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.550+0000 7f2ba2478640 1 -- 172.21.15.84:0/277629916 shutdown_connections 2023-10-16T09:49:09.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.550+0000 7fe452901640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe44c06a2e0 0x7fe44c0c5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.551+0000 7f2ba2478640 1 --2- 172.21.15.84:0/277629916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b9c0cded0 0x7f2b9c0b5c90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.550+0000 7fe452901640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe44c0addb0 0x7fe44c0b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.550+0000 7fe452901640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe44c0caa80 0x7fe44c0b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.551+0000 7f2ba2478640 1 -- 172.21.15.84:0/277629916 >> 172.21.15.84:0/277629916 conn(0x7f2b9c0cf570 msgr2=0x7f2b9c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:09.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.550+0000 7fe452901640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c0c3360 con 0x7fe44c0caa80 2023-10-16T09:49:09.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.550+0000 7fe452901640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c0c2940 con 0x7fe44c06a2e0 2023-10-16T09:49:09.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.550+0000 7fe452901640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c0c5840 con 0x7fe44c0addb0 2023-10-16T09:49:09.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.551+0000 7f9d789ce640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d7406c850 0x7f9d740ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.551+0000 7f9d789ce640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d74070360 con 0x7f9d74159e90 2023-10-16T09:49:09.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.551+0000 7f9d789ce640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d740cd1f0 con 0x7f9d74159ab0 2023-10-16T09:49:09.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.551+0000 7f9d789ce640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7415d890 con 0x7f9d7406c850 2023-10-16T09:49:09.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.551+0000 7fe44b7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe44c0addb0 0x7fe44c0b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.551+0000 7fe44b7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe44c0addb0 0x7fe44c0b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:36750/0 (socket says 172.21.15.84:36750) 2023-10-16T09:49:09.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.551+0000 7fe44b7fe640 1 -- 172.21.15.84:0/1723890860 learned_addr learned my addr 172.21.15.84:0/1723890860 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:09.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.551+0000 7f2ba2478640 1 -- 172.21.15.84:0/277629916 shutdown_connections 2023-10-16T09:49:09.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7f9d72d9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d7406c850 0x7f9d740ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7f9d72d9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d7406c850 0x7f9d740ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:36766/0 (socket says 172.21.15.84:36766) 2023-10-16T09:49:09.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7f9d72d9d640 1 -- 172.21.15.84:0/1385268369 learned_addr learned my addr 172.21.15.84:0/1385268369 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:09.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7fe450e9e640 1 --2- 172.21.15.84:0/1723890860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe44c0caa80 0x7fe44c0b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7fe44bfff640 1 --2- 172.21.15.84:0/1723890860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe44c06a2e0 0x7fe44c0c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7fcff27fc640 1 -- 172.21.15.84:0/3984178739 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff80074e0 con 0x7fcffc0b4580 2023-10-16T09:49:09.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7fcff27fc640 1 -- 172.21.15.84:0/3984178739 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff8007640 con 0x7fcffc0b4580 2023-10-16T09:49:09.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7fcff27fc640 1 -- 172.21.15.84:0/3984178739 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff80049e0 con 0x7fcffc0b4580 2023-10-16T09:49:09.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7fd003ee0640 1 -- 172.21.15.84:0/3984178739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcffc0c8430 con 0x7fcffc0b4580 2023-10-16T09:49:09.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7fc00ffff640 1 -- 172.21.15.84:0/272379975 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01803c070 con 0x7fc02806fb70 2023-10-16T09:49:09.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7fc00ffff640 1 -- 172.21.15.84:0/272379975 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc018003610 con 0x7fc02806fb70 2023-10-16T09:49:09.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7fc00ffff640 1 -- 172.21.15.84:0/272379975 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0180039a0 con 0x7fc02806fb70 2023-10-16T09:49:09.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7fc02df78640 1 -- 172.21.15.84:0/272379975 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0281626a0 con 0x7fc02806fb70 2023-10-16T09:49:09.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7fc02df78640 1 -- 172.21.15.84:0/272379975 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0280c3300 con 0x7fc02806fb70 2023-10-16T09:49:09.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.554+0000 7fc02df78640 1 -- 172.21.15.84:0/272379975 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc0280cde40 con 0x7fc02806fb70 2023-10-16T09:49:09.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.551+0000 7f2ba2478640 1 -- 172.21.15.84:0/277629916 wait complete. 2023-10-16T09:49:09.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7fe44b7fe640 1 -- 172.21.15.84:0/1723890860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe44c06a2e0 msgr2=0x7fe44c0c5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7fe44b7fe640 1 --2- 172.21.15.84:0/1723890860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe44c06a2e0 0x7fe44c0c5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7fe44b7fe640 1 -- 172.21.15.84:0/1723890860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe44c0caa80 msgr2=0x7fe44c0b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7fe44b7fe640 1 --2- 172.21.15.84:0/1723890860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe44c0caa80 0x7fe44c0b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7fe44b7fe640 1 -- 172.21.15.84:0/1723890860 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe440009400 con 0x7fe44c0addb0 2023-10-16T09:49:09.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.553+0000 7fe450e9e640 1 --2- 172.21.15.84:0/1723890860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe44c0caa80 0x7fe44c0b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:49:09.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.553+0000 7fe44b7fe640 1 --2- 172.21.15.84:0/1723890860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe44c0addb0 0x7fe44c0b06f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe4400098f0 tx=0x7fe4400046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.553+0000 7fe448ff9640 1 -- 172.21.15.84:0/1723890860 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44003e070 con 0x7fe44c0addb0 2023-10-16T09:49:09.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.553+0000 7fe448ff9640 1 -- 172.21.15.84:0/1723890860 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe440039c90 con 0x7fe44c0addb0 2023-10-16T09:49:09.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.554+0000 7fe452901640 1 -- 172.21.15.84:0/1723890860 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe44c0c2660 con 0x7fe44c0addb0 2023-10-16T09:49:09.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.554+0000 7fe448ff9640 1 -- 172.21.15.84:0/1723890860 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4400423f0 con 0x7fe44c0addb0 2023-10-16T09:49:09.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.554+0000 7fe452901640 1 -- 172.21.15.84:0/1723890860 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe44c0c3080 con 0x7fe44c0addb0 2023-10-16T09:49:09.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.552+0000 7fab768de640 1 Processor -- start 2023-10-16T09:49:09.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.553+0000 7fab768de640 1 -- start start 2023-10-16T09:49:09.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.553+0000 7f411f7fe640 1 --2- 172.21.15.84:0/2054013373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41200cf710 0x7f41200b2d10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4110009850 tx=0x7f411002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a32d276f4650e1b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.554+0000 7f411e7fc640 1 -- 172.21.15.84:0/2054013373 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4110004230 con 0x7f41200cf710 2023-10-16T09:49:09.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.554+0000 7f411e7fc640 1 -- 172.21.15.84:0/2054013373 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4110004390 con 0x7f41200cf710 2023-10-16T09:49:09.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.554+0000 7f411e7fc640 1 -- 172.21.15.84:0/2054013373 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4110003810 con 0x7f41200cf710 2023-10-16T09:49:09.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.554+0000 7f9d72d9d640 1 -- 172.21.15.84:0/1385268369 >> v1:172.21.15.157:6789/0 conn(0x7f9d74159ab0 legacy=0x7f9d7406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.554+0000 7f9d72d9d640 1 -- 172.21.15.84:0/1385268369 >> v1:172.21.15.84:6789/0 conn(0x7f9d74159e90 legacy=0x7f9d7406c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.554+0000 7f9d7159a640 1 -- 172.21.15.84:0/1385268369 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141487260 0 0) 0x7f9d74070360 con 0x7f9d74159e90 2023-10-16T09:49:09.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.554+0000 7f9d72d9d640 1 -- 172.21.15.84:0/1385268369 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d740cd4d0 con 0x7f9d7406c850 2023-10-16T09:49:09.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.555+0000 7f4125dc9640 1 -- 172.21.15.84:0/2054013373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41200cf710 msgr2=0x7f41200b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.555+0000 7f4125dc9640 1 --2- 172.21.15.84:0/2054013373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41200cf710 0x7f41200b2d10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4110009850 tx=0x7f411002b860 comp rx=0 tx=0).stop 2023-10-16T09:49:09.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.555+0000 7f4125dc9640 1 -- 172.21.15.84:0/2054013373 shutdown_connections 2023-10-16T09:49:09.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.555+0000 7f4125dc9640 1 --2- 172.21.15.84:0/2054013373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41200cf710 0x7f41200b2d10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.555+0000 7f4125dc9640 1 -- 172.21.15.84:0/2054013373 >> 172.21.15.84:0/2054013373 conn(0x7f41200d2cc0 msgr2=0x7f41200d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:09.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.555+0000 7f4125dc9640 1 -- 172.21.15.84:0/2054013373 shutdown_connections 2023-10-16T09:49:09.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.555+0000 7f4125dc9640 1 -- 172.21.15.84:0/2054013373 wait complete. 2023-10-16T09:49:09.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.554+0000 7f04fe7fc640 1 -- 172.21.15.84:0/457905920 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f050404c410 con 0x7f0500038630 2023-10-16T09:49:09.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.556+0000 7fd003ee0640 1 -- 172.21.15.84:0/3984178739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcffc0c8050 con 0x7fcffc0b4580 2023-10-16T09:49:09.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.556+0000 7fe452901640 1 -- 172.21.15.84:0/1723890860 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe44c06b310 con 0x7fe44c0addb0 2023-10-16T09:49:09.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.556+0000 7f04fe7fc640 1 --2- 172.21.15.84:0/457905920 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f04ec06af40 0x7f04ec06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.556+0000 7f04fe7fc640 1 -- 172.21.15.84:0/457905920 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f050403e070 con 0x7f0500038630 2023-10-16T09:49:09.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.557+0000 7f04fe7fc640 1 --2- 172.21.15.84:0/457905920 >> [v2:172.21.15.84:6800/2077434097,v1:172.21.15.84:6801/2077434097] conn(0x7f04ec073160 0x7f04ec075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.557+0000 7f04fe7fc640 1 -- 172.21.15.84:0/457905920 --> [v2:172.21.15.84:6800/2077434097,v1:172.21.15.84:6801/2077434097] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f04ec075c30 con 0x7f04ec073160 2023-10-16T09:49:09.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.557+0000 7f04fe7fc640 1 -- 172.21.15.84:0/457905920 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05040acec0 con 0x7f0500038630 2023-10-16T09:49:09.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.557+0000 7f050e48f640 1 --2- 172.21.15.84:0/457905920 >> [v2:172.21.15.84:6800/2077434097,v1:172.21.15.84:6801/2077434097] conn(0x7f04ec073160 0x7f04ec075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.555+0000 7f2ba2478640 1 Processor -- start 2023-10-16T09:49:09.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.557+0000 7f2ba2478640 1 -- start start 2023-10-16T09:49:09.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.557+0000 7fab768de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab68053010 0x7fab680e2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.557+0000 7fab768de640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab680e78e0 0x7fab68171870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.557+0000 7fab768de640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab680eb2f0 0x7fab68171db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.557+0000 7fab768de640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab680e51e0 con 0x7fab68053010 2023-10-16T09:49:09.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.557+0000 7fab768de640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab681722f0 con 0x7fab680e78e0 2023-10-16T09:49:09.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.557+0000 7fab768de640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab68172870 con 0x7fab680eb2f0 2023-10-16T09:49:09.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.556+0000 7f9d72d9d640 1 --2- 172.21.15.84:0/1385268369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d7406c850 0x7f9d740ce370 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00d180 tx=0x7f9d6c02e0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=924632a3b436af58 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.557+0000 7f050e48f640 1 --2- 172.21.15.84:0/457905920 >> [v2:172.21.15.84:6800/2077434097,v1:172.21.15.84:6801/2077434097] conn(0x7f04ec073160 0x7f04ec075580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.557+0000 7fc00ffff640 1 -- 172.21.15.84:0/272379975 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc018040070 con 0x7fc02806fb70 2023-10-16T09:49:09.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.558+0000 7f9d7159a640 1 -- 172.21.15.84:0/1385268369 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c02ea60 con 0x7f9d7406c850 2023-10-16T09:49:09.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.558+0000 7fab6f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab680e78e0 0x7fab68171870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.558+0000 7fab6f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab680e78e0 0x7fab68171870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:54992/0 (socket says 172.21.15.84:54992) 2023-10-16T09:49:09.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.558+0000 7fab6f7fe640 1 -- 172.21.15.84:0/1689208241 learned_addr learned my addr 172.21.15.84:0/1689208241 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:09.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.558+0000 7f9d7159a640 1 -- 172.21.15.84:0/1385268369 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c02ebc0 con 0x7f9d7406c850 2023-10-16T09:49:09.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.558+0000 7fab6ffff640 1 --2- 172.21.15.84:0/1689208241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab68053010 0x7fab680e2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.558+0000 7f9d7159a640 1 -- 172.21.15.84:0/1385268369 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c009600 con 0x7f9d7406c850 2023-10-16T09:49:09.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.558+0000 7f4125dc9640 1 Processor -- start 2023-10-16T09:49:09.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.558+0000 7f4125dc9640 1 -- start start 2023-10-16T09:49:09.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.558+0000 7f04fe7fc640 1 -- 172.21.15.84:0/457905920 <== osd.0 v2:172.21.15.84:6800/2077434097 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f04ec075c30 con 0x7f04ec073160 2023-10-16T09:49:09.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.559+0000 7fc00ffff640 1 --2- 172.21.15.84:0/272379975 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc01406ae70 0x7fc01406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.559+0000 7fab6f7fe640 1 -- 172.21.15.84:0/1689208241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab680eb2f0 msgr2=0x7fab68171db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.559+0000 7fab6f7fe640 1 --2- 172.21.15.84:0/1689208241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab680eb2f0 0x7fab68171db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.559+0000 7fab6f7fe640 1 -- 172.21.15.84:0/1689208241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab68053010 msgr2=0x7fab680e2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.559+0000 7f9d789ce640 1 -- 172.21.15.84:0/1385268369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d7406c850 msgr2=0x7f9d740ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.559+0000 7f9d789ce640 1 --2- 172.21.15.84:0/1385268369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d7406c850 0x7f9d740ce370 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00d180 tx=0x7f9d6c02e0c0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.559+0000 7f050d48d640 1 --2- 172.21.15.84:0/457905920 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f04ec06af40 0x7f04ec06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.559+0000 7fab6f7fe640 1 --2- 172.21.15.84:0/1689208241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab68053010 0x7fab680e2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.559+0000 7fab6f7fe640 1 -- 172.21.15.84:0/1689208241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c0094d0 con 0x7fab680e78e0 2023-10-16T09:49:09.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.559+0000 7fab6ffff640 1 --2- 172.21.15.84:0/1689208241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab68053010 0x7fab680e2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:49:09.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.559+0000 7fc00ffff640 1 -- 172.21.15.84:0/272379975 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0180ab3a0 con 0x7fc02806fb70 2023-10-16T09:49:09.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.559+0000 7fc00ffff640 1 --2- 172.21.15.84:0/272379975 >> [v2:172.21.15.84:6818/338331808,v1:172.21.15.84:6821/338331808] conn(0x7fc014073090 0x7fc0140754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.559+0000 7fc00ffff640 1 -- 172.21.15.84:0/272379975 --> [v2:172.21.15.84:6818/338331808,v1:172.21.15.84:6821/338331808] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc014075b60 con 0x7fc014073090 2023-10-16T09:49:09.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.559+0000 7fc00ffff640 1 -- 172.21.15.84:0/272379975 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0180ab750 con 0x7fc02806fb70 2023-10-16T09:49:09.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.559+0000 7f2ba2478640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b9c0b8480 0x7f2b9c1f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.560+0000 7f9d789ce640 1 -- 172.21.15.84:0/1385268369 shutdown_connections 2023-10-16T09:49:09.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.560+0000 7f9d789ce640 1 --2- 172.21.15.84:0/1385268369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d7406c850 0x7f9d740ce370 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.560+0000 7f9d789ce640 1 -- 172.21.15.84:0/1385268369 >> 172.21.15.84:0/1385268369 conn(0x7f9d740ce9b0 msgr2=0x7f9d740cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:09.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.559+0000 7fc026ffd640 1 --2- 172.21.15.84:0/272379975 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc01406ae70 0x7fc01406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.560+0000 7fab6f7fe640 1 --2- 172.21.15.84:0/1689208241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab680e78e0 0x7fab68171870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0090b0 tx=0x7fab5c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.560+0000 7fc027fff640 1 --2- 172.21.15.84:0/272379975 >> [v2:172.21.15.84:6818/338331808,v1:172.21.15.84:6821/338331808] conn(0x7fc014073090 0x7fc0140754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.560+0000 7f4125dc9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41200b6cd0 0x7f41200cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.560+0000 7f2ba2478640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b9c0cded0 0x7f2b9c1f4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.560+0000 7fab6cff9640 1 -- 172.21.15.84:0/1689208241 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c032890 con 0x7fab680e78e0 2023-10-16T09:49:09.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.560+0000 7fab6cff9640 1 -- 172.21.15.84:0/1689208241 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c0329f0 con 0x7fab680e78e0 2023-10-16T09:49:09.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.560+0000 7fab6cff9640 1 -- 172.21.15.84:0/1689208241 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c032da0 con 0x7fab680e78e0 2023-10-16T09:49:09.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.561+0000 7f4125dc9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41200cf710 0x7f41201f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.561+0000 7fab768de640 1 -- 172.21.15.84:0/1689208241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab68172590 con 0x7fab680e78e0 2023-10-16T09:49:09.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.562+0000 7f050d48d640 1 --2- 172.21.15.84:0/457905920 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f04ec06af40 0x7f04ec06d400 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0508069020 tx=0x7f050806c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.561+0000 7fcff27fc640 1 -- 172.21.15.84:0/3984178739 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcff8004b40 con 0x7fcffc0b4580 2023-10-16T09:49:09.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.561+0000 7f2ba2478640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b9c15cd70 0x7f2b9c202570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.562+0000 7f2ba1476640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b9c0b8480 0x7f2b9c1f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.562+0000 7f2ba2478640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c1f5860 con 0x7f2b9c0cded0 2023-10-16T09:49:09.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.560+0000 7fe448ff9640 1 -- 172.21.15.84:0/1723890860 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe440042550 con 0x7fe44c0addb0 2023-10-16T09:49:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.561+0000 7fe448ff9640 1 --2- 172.21.15.84:0/1723890860 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe43806ae70 0x7fe43806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.561+0000 7fe448ff9640 1 -- 172.21.15.84:0/1723890860 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4400abb90 con 0x7fe44c0addb0 2023-10-16T09:49:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.562+0000 7fe448ff9640 1 --2- 172.21.15.84:0/1723890860 >> [v2:172.21.15.157:6808/2704622262,v1:172.21.15.157:6809/2704622262] conn(0x7fe438073010 0x7fe438075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.562+0000 7fe448ff9640 1 -- 172.21.15.84:0/1723890860 --> [v2:172.21.15.157:6808/2704622262,v1:172.21.15.157:6809/2704622262] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe438075ae0 con 0x7fe438073010 2023-10-16T09:49:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.562+0000 7fe448ff9640 1 -- 172.21.15.84:0/1723890860 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4400b02a0 con 0x7fe44c0addb0 2023-10-16T09:49:09.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.561+0000 7fab768de640 1 -- 172.21.15.84:0/1689208241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab680e56f0 con 0x7fab680e78e0 2023-10-16T09:49:09.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.562+0000 7fab768de640 1 -- 172.21.15.84:0/1689208241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab68006140 con 0x7fab680e78e0 2023-10-16T09:49:09.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.561+0000 7f4125dc9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41200cfaf0 0x7f41200b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.562+0000 7f411f7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41200b6cd0 0x7f41200cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.562+0000 7f411ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41200cfaf0 0x7f41200b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.562+0000 7f411f7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41200b6cd0 0x7f41200cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:36788/0 (socket says 172.21.15.84:36788) 2023-10-16T09:49:09.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.562+0000 7f411ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41200cfaf0 0x7f41200b1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59570/0 (socket says 172.21.15.84:59570) 2023-10-16T09:49:09.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.562+0000 7f4125dc9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412015a3a0 con 0x7f41200cfaf0 2023-10-16T09:49:09.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.562+0000 7f4125dc9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41200cc890 con 0x7f41200cf710 2023-10-16T09:49:09.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.562+0000 7f411f7fe640 1 -- 172.21.15.84:0/4267825939 learned_addr learned my addr 172.21.15.84:0/4267825939 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:09.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.562+0000 7f4125dc9640 1 -- 172.21.15.84:0/4267825939 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41201f3ab0 con 0x7f41200b6cd0 2023-10-16T09:49:09.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.561+0000 7f9d789ce640 1 -- 172.21.15.84:0/1385268369 shutdown_connections 2023-10-16T09:49:09.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.561+0000 7f9d789ce640 1 -- 172.21.15.84:0/1385268369 wait complete. 2023-10-16T09:49:09.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.562+0000 7f2ba2478640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c15dd50 con 0x7f2b9c0b8480 2023-10-16T09:49:09.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.563+0000 7f2ba2478640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c15ed90 con 0x7f2b9c15cd70 2023-10-16T09:49:09.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.563+0000 7f2ba1476640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b9c0b8480 0x7f2b9c1f5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:55000/0 (socket says 172.21.15.84:55000) 2023-10-16T09:49:09.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.563+0000 7f2ba1476640 1 -- 172.21.15.84:0/2903684370 learned_addr learned my addr 172.21.15.84:0/2903684370 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:09.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.561+0000 7fc026ffd640 1 --2- 172.21.15.84:0/272379975 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc01406ae70 0x7fc01406d330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc0280ca470 tx=0x7fc01c007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.561+0000 7fc027fff640 1 --2- 172.21.15.84:0/272379975 >> [v2:172.21.15.84:6818/338331808,v1:172.21.15.84:6821/338331808] conn(0x7fc014073090 0x7fc0140754b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.563+0000 7fc00ffff640 1 -- 172.21.15.84:0/272379975 <== osd.2 v2:172.21.15.84:6818/338331808 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fc014075b60 con 0x7fc014073090 2023-10-16T09:49:09.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.563+0000 7f2ba1c77640 1 --2- 172.21.15.84:0/2903684370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b9c15cd70 0x7f2b9c202570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.563+0000 7f2ba0c75640 1 --2- 172.21.15.84:0/2903684370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b9c0cded0 0x7f2b9c1f4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.563+0000 7fe44bfff640 1 --2- 172.21.15.84:0/1723890860 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe43806ae70 0x7fe43806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.563+0000 7f411effd640 1 --2- 172.21.15.84:0/4267825939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41200cf710 0x7f41201f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.563+0000 7fe450e9e640 1 --2- 172.21.15.84:0/1723890860 >> [v2:172.21.15.157:6808/2704622262,v1:172.21.15.157:6809/2704622262] conn(0x7fe438073010 0x7fe438075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.564+0000 7f411ffff640 1 -- 172.21.15.84:0/4267825939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41200b6cd0 msgr2=0x7f41200cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.564+0000 7f411ffff640 1 --2- 172.21.15.84:0/4267825939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41200b6cd0 0x7f41200cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.564+0000 7f411ffff640 1 -- 172.21.15.84:0/4267825939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41200cf710 msgr2=0x7f41201f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.564+0000 7f411ffff640 1 --2- 172.21.15.84:0/4267825939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41200cf710 0x7f41201f5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.564+0000 7f411ffff640 1 -- 172.21.15.84:0/4267825939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4110009460 con 0x7f41200cfaf0 2023-10-16T09:49:09.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.563+0000 7fd003ee0640 1 -- 172.21.15.84:0/3984178739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcffc0d1ee0 con 0x7fcffc0b4580 2023-10-16T09:49:09.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.564+0000 7f2ba0c75640 1 -- 172.21.15.84:0/2903684370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b9c15cd70 msgr2=0x7f2b9c202570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.564+0000 7f2ba0c75640 1 --2- 172.21.15.84:0/2903684370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b9c15cd70 0x7f2b9c202570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.564+0000 7f2ba0c75640 1 -- 172.21.15.84:0/2903684370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b9c0b8480 msgr2=0x7f2b9c1f5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.564+0000 7f2ba0c75640 1 --2- 172.21.15.84:0/2903684370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b9c0b8480 0x7f2b9c1f5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.564+0000 7f2ba0c75640 1 -- 172.21.15.84:0/2903684370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c009400 con 0x7f2b9c0cded0 2023-10-16T09:49:09.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.564+0000 7fe44bfff640 1 --2- 172.21.15.84:0/1723890860 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe43806ae70 0x7fe43806d330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe43c000f60 tx=0x7fe43c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.564+0000 7f411ffff640 1 --2- 172.21.15.84:0/4267825939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41200cfaf0 0x7f41200b1a60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f411800c780 tx=0x7f411800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.565+0000 7f4107fff640 1 -- 172.21.15.84:0/4267825939 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411800aa50 con 0x7f41200cfaf0 2023-10-16T09:49:09.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.565+0000 7fcff27fc640 1 --2- 172.21.15.84:0/3984178739 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fcfe006af40 0x7fcfe006d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.565+0000 7f4107fff640 1 -- 172.21.15.84:0/4267825939 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411800abb0 con 0x7f41200cfaf0 2023-10-16T09:49:09.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.565+0000 7fab6cff9640 1 -- 172.21.15.84:0/1689208241 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fab5c04d3f0 con 0x7fab680e78e0 2023-10-16T09:49:09.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.565+0000 7f2ba0c75640 1 --2- 172.21.15.84:0/2903684370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b9c0cded0 0x7f2b9c1f4980 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400cbc0 tx=0x7f2b9400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.565+0000 7f2b91ffb640 1 -- 172.21.15.84:0/2903684370 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9400aca0 con 0x7f2b9c0cded0 2023-10-16T09:49:09.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.565+0000 7f4107fff640 1 -- 172.21.15.84:0/4267825939 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411800ce30 con 0x7f41200cfaf0 2023-10-16T09:49:09.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.565+0000 7f4125dc9640 1 -- 172.21.15.84:0/4267825939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41201f3d90 con 0x7f41200cfaf0 2023-10-16T09:49:09.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.565+0000 7f4125dc9640 1 -- 172.21.15.84:0/4267825939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41201f4350 con 0x7f41200cfaf0 2023-10-16T09:49:09.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.565+0000 7fe450e9e640 1 --2- 172.21.15.84:0/1723890860 >> [v2:172.21.15.157:6808/2704622262,v1:172.21.15.157:6809/2704622262] conn(0x7fe438073010 0x7fe438075430 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.565+0000 7f2b91ffb640 1 -- 172.21.15.84:0/2903684370 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9400ae00 con 0x7f2b9c0cded0 2023-10-16T09:49:09.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.566+0000 7f4125dc9640 1 -- 172.21.15.84:0/4267825939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f41200ce910 con 0x7f41200cfaf0 2023-10-16T09:49:09.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.565+0000 7f9d789ce640 1 Processor -- start 2023-10-16T09:49:09.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.566+0000 7f9d789ce640 1 -- start start 2023-10-16T09:49:09.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.566+0000 7f2ba1476640 1 --2- 172.21.15.84:0/2903684370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b9c0b8480 0x7f2b9c1f5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:09.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.565+0000 7fcff27fc640 1 -- 172.21.15.84:0/3984178739 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcff8088890 con 0x7fcffc0b4580 2023-10-16T09:49:09.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.566+0000 7f2ba2478640 1 -- 172.21.15.84:0/2903684370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9c15e280 con 0x7f2b9c0cded0 2023-10-16T09:49:09.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.566+0000 7f2ba2478640 1 -- 172.21.15.84:0/2903684370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9c15e840 con 0x7f2b9c0cded0 2023-10-16T09:49:09.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.566+0000 7f2b91ffb640 1 -- 172.21.15.84:0/2903684370 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94012620 con 0x7f2b9c0cded0 2023-10-16T09:49:09.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.567+0000 7fd00147b640 1 --2- 172.21.15.84:0/3984178739 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fcfe006af40 0x7fcfe006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.568+0000 7fcff27fc640 1 --2- 172.21.15.84:0/3984178739 >> [v2:172.21.15.84:6808/805077222,v1:172.21.15.84:6809/805077222] conn(0x7fcfe00730e0 0x7fcfe0075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.568+0000 7fcff27fc640 1 -- 172.21.15.84:0/3984178739 --> [v2:172.21.15.84:6808/805077222,v1:172.21.15.84:6809/805077222] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcfe0075bb0 con 0x7fcfe00730e0 2023-10-16T09:49:09.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.568+0000 7fcff27fc640 1 -- 172.21.15.84:0/3984178739 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcff8093070 con 0x7fcffc0b4580 2023-10-16T09:49:09.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.568+0000 7fab6cff9640 1 --2- 172.21.15.84:0/1689208241 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fab5406af40 0x7fab5406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.568+0000 7f9d789ce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d7406c850 0x7f9d740c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.568+0000 7fe448ff9640 1 -- 172.21.15.84:0/1723890860 <== osd.4 v2:172.21.15.157:6808/2704622262 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe438075ae0 con 0x7fe438073010 2023-10-16T09:49:09.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.568+0000 7f9d789ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d74159ab0 0x7f9d7415cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.568+0000 7fab6cff9640 1 -- 172.21.15.84:0/1689208241 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab5c0ab6a0 con 0x7fab680e78e0 2023-10-16T09:49:09.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.568+0000 7f9d789ce640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d74159e90 0x7f9d7415bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.568+0000 7fd00247d640 1 --2- 172.21.15.84:0/3984178739 >> [v2:172.21.15.84:6808/805077222,v1:172.21.15.84:6809/805077222] conn(0x7fcfe00730e0 0x7fcfe0075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.569+0000 7f4107fff640 1 -- 172.21.15.84:0/4267825939 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4118012420 con 0x7f41200cfaf0 2023-10-16T09:49:09.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.569+0000 7fab6cff9640 1 --2- 172.21.15.84:0/1689208241 >> [v2:172.21.15.157:6800/3654970329,v1:172.21.15.157:6801/3654970329] conn(0x7fab54073160 0x7fab54075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.569+0000 7fab6cff9640 1 -- 172.21.15.84:0/1689208241 --> [v2:172.21.15.157:6800/3654970329,v1:172.21.15.157:6801/3654970329] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab54075c30 con 0x7fab54073160 2023-10-16T09:49:09.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.569+0000 7fab6cff9640 1 -- 172.21.15.84:0/1689208241 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab5c0aba50 con 0x7fab680e78e0 2023-10-16T09:49:09.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.569+0000 7fab6ffff640 1 --2- 172.21.15.84:0/1689208241 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fab5406af40 0x7fab5406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.569+0000 7fd00247d640 1 --2- 172.21.15.84:0/3984178739 >> [v2:172.21.15.84:6808/805077222,v1:172.21.15.84:6809/805077222] conn(0x7fcfe00730e0 0x7fcfe0075500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.569+0000 7f9d789ce640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d740b19b0 con 0x7f9d74159ab0 2023-10-16T09:49:09.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.569+0000 7f9d789ce640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7414e480 con 0x7f9d7406c850 2023-10-16T09:49:09.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.569+0000 7f9d789ce640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d740c4520 con 0x7f9d74159e90 2023-10-16T09:49:09.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.569+0000 7f9d7259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d7406c850 0x7f9d740c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.570+0000 7f9d7259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d7406c850 0x7f9d740c3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:55014/0 (socket says 172.21.15.84:55014) 2023-10-16T09:49:09.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.570+0000 7f9d7259c640 1 -- 172.21.15.84:0/1613192397 learned_addr learned my addr 172.21.15.84:0/1613192397 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:09.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.569+0000 7f9d72d9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d74159e90 0x7f9d7415bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.570+0000 7fab74e7b640 1 --2- 172.21.15.84:0/1689208241 >> [v2:172.21.15.157:6800/3654970329,v1:172.21.15.157:6801/3654970329] conn(0x7fab54073160 0x7fab54075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.570+0000 7f4107fff640 1 --2- 172.21.15.84:0/4267825939 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f410c06af10 0x7f410c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.570+0000 7f4107fff640 1 -- 172.21.15.84:0/4267825939 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4118089fd0 con 0x7f41200cfaf0 2023-10-16T09:49:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.569+0000 7f2ba2478640 1 -- 172.21.15.84:0/2903684370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2b7c000fc0 con 0x7f2b9c0cded0 2023-10-16T09:49:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.571+0000 7f4107fff640 1 --2- 172.21.15.84:0/4267825939 >> [v2:172.21.15.157:6816/899624983,v1:172.21.15.157:6817/899624983] conn(0x7f410c073130 0x7f410c075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.571+0000 7f4107fff640 1 -- 172.21.15.84:0/4267825939 --> [v2:172.21.15.157:6816/899624983,v1:172.21.15.157:6817/899624983] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f410c075c00 con 0x7f410c073130 2023-10-16T09:49:09.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.571+0000 7f4107fff640 1 -- 172.21.15.84:0/4267825939 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f411808a380 con 0x7f41200cfaf0 2023-10-16T09:49:09.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.571+0000 7f2b91ffb640 1 -- 172.21.15.84:0/2903684370 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2b94007070 con 0x7f2b9c0cded0 2023-10-16T09:49:09.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.571+0000 7fcff27fc640 1 -- 172.21.15.84:0/3984178739 <== osd.1 v2:172.21.15.84:6808/805077222 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fcfe0075bb0 con 0x7fcfe00730e0 2023-10-16T09:49:09.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.570+0000 7f9d71d9b640 1 --2- 172.21.15.84:0/1613192397 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d74159ab0 0x7f9d7415cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.571+0000 7f9d72d9d640 1 -- 172.21.15.84:0/1613192397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d7406c850 msgr2=0x7f9d740c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.571+0000 7f9d72d9d640 1 --2- 172.21.15.84:0/1613192397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d7406c850 0x7f9d740c3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.571+0000 7f9d72d9d640 1 -- 172.21.15.84:0/1613192397 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d74159ab0 msgr2=0x7f9d7415cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.571+0000 7fab74e7b640 1 --2- 172.21.15.84:0/1689208241 >> [v2:172.21.15.157:6800/3654970329,v1:172.21.15.157:6801/3654970329] conn(0x7fab54073160 0x7fab54075580 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.572+0000 7f9d72d9d640 1 --2- 172.21.15.84:0/1613192397 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d74159ab0 0x7f9d7415cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.572+0000 7f9d72d9d640 1 -- 172.21.15.84:0/1613192397 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c00bd40 con 0x7f9d74159e90 2023-10-16T09:49:09.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.572+0000 7f411f7fe640 1 --2- 172.21.15.84:0/4267825939 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f410c06af10 0x7f410c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.572+0000 7f411effd640 1 --2- 172.21.15.84:0/4267825939 >> [v2:172.21.15.157:6816/899624983,v1:172.21.15.157:6817/899624983] conn(0x7f410c073130 0x7f410c075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.572+0000 7fab6ffff640 1 --2- 172.21.15.84:0/1689208241 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fab5406af40 0x7fab5406d400 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fab64009db0 tx=0x7fab64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.572+0000 7f9d71d9b640 1 --2- 172.21.15.84:0/1613192397 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d74159ab0 0x7f9d7415cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:09.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.573+0000 7f9d72d9d640 1 --2- 172.21.15.84:0/1613192397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d74159e90 0x7f9d7415bde0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c034000 tx=0x7f9d6c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.573+0000 7f411effd640 1 --2- 172.21.15.84:0/4267825939 >> [v2:172.21.15.157:6816/899624983,v1:172.21.15.157:6817/899624983] conn(0x7f410c073130 0x7f410c075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.573+0000 7f9d62ffd640 1 -- 172.21.15.84:0/1613192397 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c02ea60 con 0x7f9d74159e90 2023-10-16T09:49:09.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.573+0000 7f9d62ffd640 1 -- 172.21.15.84:0/1613192397 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c036820 con 0x7f9d74159e90 2023-10-16T09:49:09.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.573+0000 7f9d62ffd640 1 -- 172.21.15.84:0/1613192397 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c036bd0 con 0x7f9d74159e90 2023-10-16T09:49:09.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.573+0000 7f9d789ce640 1 -- 172.21.15.84:0/1613192397 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d740c4800 con 0x7f9d74159e90 2023-10-16T09:49:09.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.574+0000 7fd00147b640 1 --2- 172.21.15.84:0/3984178739 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fcfe006af40 0x7fcfe006d400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcffc152970 tx=0x7fcfec009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.574+0000 7f9d789ce640 1 -- 172.21.15.84:0/1613192397 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7415d4a0 con 0x7f9d74159e90 2023-10-16T09:49:09.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.575+0000 7fab6cff9640 1 -- 172.21.15.84:0/1689208241 <== osd.5 v2:172.21.15.157:6800/3654970329 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fab54075c30 con 0x7fab54073160 2023-10-16T09:49:09.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.575+0000 7f4107fff640 1 -- 172.21.15.84:0/4267825939 <== osd.6 v2:172.21.15.157:6816/899624983 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f410c075c00 con 0x7f410c073130 2023-10-16T09:49:09.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.574+0000 7f2b91ffb640 1 --2- 172.21.15.84:0/2903684370 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2b8006af40 0x7f2b8006d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.575+0000 7f2b91ffb640 1 -- 172.21.15.84:0/2903684370 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b94089500 con 0x7f2b9c0cded0 2023-10-16T09:49:09.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.575+0000 7f2b91ffb640 1 --2- 172.21.15.84:0/2903684370 >> [v2:172.21.15.84:6816/1933430277,v1:172.21.15.84:6817/1933430277] conn(0x7f2b80073160 0x7f2b80075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.575+0000 7f2b91ffb640 1 -- 172.21.15.84:0/2903684370 --> [v2:172.21.15.84:6816/1933430277,v1:172.21.15.84:6817/1933430277] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b80075c30 con 0x7f2b80073160 2023-10-16T09:49:09.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.575+0000 7f2b91ffb640 1 -- 172.21.15.84:0/2903684370 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b9408bb20 con 0x7f2b9c0cded0 2023-10-16T09:49:09.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.575+0000 7f2ba1476640 1 --2- 172.21.15.84:0/2903684370 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2b8006af40 0x7f2b8006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.576+0000 7f9d789ce640 1 -- 172.21.15.84:0/1613192397 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9d740cda90 con 0x7f9d74159e90 2023-10-16T09:49:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.576+0000 7f2ba1476640 1 --2- 172.21.15.84:0/2903684370 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2b8006af40 0x7f2b8006d400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2b98001790 tx=0x7f2b98009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.576+0000 7f411f7fe640 1 --2- 172.21.15.84:0/4267825939 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f410c06af10 0x7f410c06d3d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4110009850 tx=0x7f4110008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.578+0000 7f9d62ffd640 1 -- 172.21.15.84:0/1613192397 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9d6c036d30 con 0x7f9d74159e90 2023-10-16T09:49:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.579+0000 7f9d62ffd640 1 --2- 172.21.15.84:0/1613192397 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9d5406ae70 0x7f9d5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.580+0000 7f9d62ffd640 1 -- 172.21.15.84:0/1613192397 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d6c0acc20 con 0x7f9d74159e90 2023-10-16T09:49:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.580+0000 7f9d62ffd640 1 --2- 172.21.15.84:0/1613192397 >> [v2:172.21.15.157:6819/1752891116,v1:172.21.15.157:6821/1752891116] conn(0x7f9d54073010 0x7f9d54075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.580+0000 7f9d62ffd640 1 -- 172.21.15.84:0/1613192397 --> [v2:172.21.15.157:6819/1752891116,v1:172.21.15.157:6821/1752891116] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d54075ae0 con 0x7f9d54073010 2023-10-16T09:49:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.580+0000 7f9d62ffd640 1 -- 172.21.15.84:0/1613192397 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d6c0acfd0 con 0x7f9d74159e90 2023-10-16T09:49:09.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.580+0000 7f2ba1c77640 1 --2- 172.21.15.84:0/2903684370 >> [v2:172.21.15.84:6816/1933430277,v1:172.21.15.84:6817/1933430277] conn(0x7f2b80073160 0x7f2b80075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.580+0000 7f9d7259c640 1 --2- 172.21.15.84:0/1613192397 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9d5406ae70 0x7f9d5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.581+0000 7f9d71d9b640 1 --2- 172.21.15.84:0/1613192397 >> [v2:172.21.15.157:6819/1752891116,v1:172.21.15.157:6821/1752891116] conn(0x7f9d54073010 0x7f9d54075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:09.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.581+0000 7f2ba1c77640 1 --2- 172.21.15.84:0/2903684370 >> [v2:172.21.15.84:6816/1933430277,v1:172.21.15.84:6817/1933430277] conn(0x7f2b80073160 0x7f2b80075580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.581+0000 7f9d7259c640 1 --2- 172.21.15.84:0/1613192397 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9d5406ae70 0x7f9d5406d330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9d640017f0 tx=0x7f9d64009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.583+0000 7f2b91ffb640 1 -- 172.21.15.84:0/2903684370 <== osd.3 v2:172.21.15.84:6816/1933430277 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f2b80075c30 con 0x7f2b80073160 2023-10-16T09:49:09.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.583+0000 7f9d71d9b640 1 --2- 172.21.15.84:0/1613192397 >> [v2:172.21.15.157:6819/1752891116,v1:172.21.15.157:6821/1752891116] conn(0x7f9d54073010 0x7f9d54075430 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:09.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.586+0000 7f9d62ffd640 1 -- 172.21.15.84:0/1613192397 <== osd.7 v2:172.21.15.157:6819/1752891116 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f9d54075ae0 con 0x7f9d54073010 2023-10-16T09:49:09.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.591+0000 7f050ec90640 1 -- 172.21.15.84:0/457905920 --> [v2:172.21.15.84:6800/2077434097,v1:172.21.15.84:6801/2077434097] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f05000076a0 con 0x7f04ec073160 2023-10-16T09:49:09.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.594+0000 7f04fe7fc640 1 -- 172.21.15.84:0/457905920 <== osd.0 v2:172.21.15.84:6800/2077434097 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f04f8001240 con 0x7f04ec073160 2023-10-16T09:49:09.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.594+0000 7fc02df78640 1 -- 172.21.15.84:0/272379975 --> [v2:172.21.15.84:6818/338331808,v1:172.21.15.84:6821/338331808] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc02804e9b0 con 0x7fc014073090 2023-10-16T09:49:09.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.594+0000 7f050ec90640 1 -- 172.21.15.84:0/457905920 >> [v2:172.21.15.84:6800/2077434097,v1:172.21.15.84:6801/2077434097] conn(0x7f04ec073160 msgr2=0x7f04ec075580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.594+0000 7f050ec90640 1 --2- 172.21.15.84:0/457905920 >> [v2:172.21.15.84:6800/2077434097,v1:172.21.15.84:6801/2077434097] conn(0x7f04ec073160 0x7f04ec075580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.595+0000 7fd003ee0640 1 -- 172.21.15.84:0/3984178739 --> [v2:172.21.15.84:6808/805077222,v1:172.21.15.84:6809/805077222] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcffc0d1c00 con 0x7fcfe00730e0 2023-10-16T09:49:09.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.595+0000 7fc00ffff640 1 -- 172.21.15.84:0/272379975 <== osd.2 v2:172.21.15.84:6818/338331808 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc02804e9b0 con 0x7fc014073090 2023-10-16T09:49:09.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.595+0000 7f050ec90640 1 -- 172.21.15.84:0/457905920 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f04ec06af40 msgr2=0x7f04ec06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.595+0000 7f050ec90640 1 --2- 172.21.15.84:0/457905920 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f04ec06af40 0x7f04ec06d400 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0508069020 tx=0x7f050806c040 comp rx=0 tx=0).stop 2023-10-16T09:49:09.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.595+0000 7f050ec90640 1 -- 172.21.15.84:0/457905920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0500038630 msgr2=0x7f05000cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.595+0000 7f050ec90640 1 --2- 172.21.15.84:0/457905920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0500038630 0x7f05000cd370 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0504033000 tx=0x7f0504004730 comp rx=0 tx=0).stop 2023-10-16T09:49:09.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.596+0000 7f050e48f640 1 -- 172.21.15.84:0/457905920 reap_dead start 2023-10-16T09:49:09.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.596+0000 7f050ec90640 1 -- 172.21.15.84:0/457905920 shutdown_connections 2023-10-16T09:49:09.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.596+0000 7f050ec90640 1 -- 172.21.15.84:0/457905920 >> 172.21.15.84:0/457905920 conn(0x7f05000590c0 msgr2=0x7f0500056fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:09.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.596+0000 7f050ec90640 1 -- 172.21.15.84:0/457905920 shutdown_connections 2023-10-16T09:49:09.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.596+0000 7f050ec90640 1 -- 172.21.15.84:0/457905920 wait complete. 2023-10-16T09:49:09.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.596+0000 7fc02df78640 1 -- 172.21.15.84:0/272379975 >> [v2:172.21.15.84:6818/338331808,v1:172.21.15.84:6821/338331808] conn(0x7fc014073090 msgr2=0x7fc0140754b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.596+0000 7fc02df78640 1 --2- 172.21.15.84:0/272379975 >> [v2:172.21.15.84:6818/338331808,v1:172.21.15.84:6821/338331808] conn(0x7fc014073090 0x7fc0140754b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.596+0000 7fc02df78640 1 -- 172.21.15.84:0/272379975 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc01406ae70 msgr2=0x7fc01406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.596+0000 7fc02df78640 1 --2- 172.21.15.84:0/272379975 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc01406ae70 0x7fc01406d330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc0280ca470 tx=0x7fc01c007c10 comp rx=0 tx=0).stop 2023-10-16T09:49:09.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.596+0000 7fc02df78640 1 -- 172.21.15.84:0/272379975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc02806fb70 msgr2=0x7fc0281f6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.596+0000 7fc02df78640 1 --2- 172.21.15.84:0/272379975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc02806fb70 0x7fc0281f6620 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc018033000 tx=0x7fc0180046c0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.596+0000 7fc027fff640 1 -- 172.21.15.84:0/272379975 reap_dead start 2023-10-16T09:49:09.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.597+0000 7fc02df78640 1 -- 172.21.15.84:0/272379975 shutdown_connections 2023-10-16T09:49:09.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.597+0000 7fc02df78640 1 -- 172.21.15.84:0/272379975 >> 172.21.15.84:0/272379975 conn(0x7fc0280ce6e0 msgr2=0x7fc0280aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:09.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.597+0000 7fc02df78640 1 -- 172.21.15.84:0/272379975 shutdown_connections 2023-10-16T09:49:09.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.598+0000 7fcff27fc640 1 -- 172.21.15.84:0/3984178739 <== osd.1 v2:172.21.15.84:6808/805077222 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcff4034440 con 0x7fcfe00730e0 2023-10-16T09:49:09.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.598+0000 7fc02df78640 1 -- 172.21.15.84:0/272379975 wait complete. 2023-10-16T09:49:09.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.599+0000 7fd003ee0640 1 -- 172.21.15.84:0/3984178739 >> [v2:172.21.15.84:6808/805077222,v1:172.21.15.84:6809/805077222] conn(0x7fcfe00730e0 msgr2=0x7fcfe0075500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.599+0000 7fd003ee0640 1 --2- 172.21.15.84:0/3984178739 >> [v2:172.21.15.84:6808/805077222,v1:172.21.15.84:6809/805077222] conn(0x7fcfe00730e0 0x7fcfe0075500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.599+0000 7fd003ee0640 1 -- 172.21.15.84:0/3984178739 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fcfe006af40 msgr2=0x7fcfe006d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.599+0000 7fd003ee0640 1 --2- 172.21.15.84:0/3984178739 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fcfe006af40 0x7fcfe006d400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcffc152970 tx=0x7fcfec009240 comp rx=0 tx=0).stop 2023-10-16T09:49:09.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.599+0000 7fd003ee0640 1 -- 172.21.15.84:0/3984178739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcffc0b4580 msgr2=0x7fcffc15c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.599+0000 7fd003ee0640 1 --2- 172.21.15.84:0/3984178739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcffc0b4580 0x7fcffc15c360 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcff800c3e0 tx=0x7fcff800c8a0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.599+0000 7fd00247d640 1 -- 172.21.15.84:0/3984178739 reap_dead start 2023-10-16T09:49:09.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.599+0000 7fd003ee0640 1 -- 172.21.15.84:0/3984178739 shutdown_connections 2023-10-16T09:49:09.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.599+0000 7fd003ee0640 1 -- 172.21.15.84:0/3984178739 >> 172.21.15.84:0/3984178739 conn(0x7fcffc0cf580 msgr2=0x7fcffc0b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:09.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.601+0000 7fd003ee0640 1 -- 172.21.15.84:0/3984178739 shutdown_connections 2023-10-16T09:49:09.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.601+0000 7fd003ee0640 1 -- 172.21.15.84:0/3984178739 wait complete. 2023-10-16T09:49:09.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.603+0000 7fe452901640 1 -- 172.21.15.84:0/1723890860 --> [v2:172.21.15.157:6808/2704622262,v1:172.21.15.157:6809/2704622262] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe44c06a750 con 0x7fe438073010 2023-10-16T09:49:09.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.604+0000 7fe448ff9640 1 -- 172.21.15.84:0/1723890860 <== osd.4 v2:172.21.15.157:6808/2704622262 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe444002010 con 0x7fe438073010 2023-10-16T09:49:09.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.605+0000 7fe452901640 1 -- 172.21.15.84:0/1723890860 >> [v2:172.21.15.157:6808/2704622262,v1:172.21.15.157:6809/2704622262] conn(0x7fe438073010 msgr2=0x7fe438075430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.605+0000 7fe452901640 1 --2- 172.21.15.84:0/1723890860 >> [v2:172.21.15.157:6808/2704622262,v1:172.21.15.157:6809/2704622262] conn(0x7fe438073010 0x7fe438075430 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.605+0000 7fe452901640 1 -- 172.21.15.84:0/1723890860 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe43806ae70 msgr2=0x7fe43806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.605+0000 7fe452901640 1 --2- 172.21.15.84:0/1723890860 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe43806ae70 0x7fe43806d330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe43c000f60 tx=0x7fe43c009040 comp rx=0 tx=0).stop 2023-10-16T09:49:09.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.605+0000 7fe452901640 1 -- 172.21.15.84:0/1723890860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe44c0addb0 msgr2=0x7fe44c0b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.605+0000 7fe452901640 1 --2- 172.21.15.84:0/1723890860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe44c0addb0 0x7fe44c0b06f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe4400098f0 tx=0x7fe4400046c0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.605+0000 7fe450e9e640 1 -- 172.21.15.84:0/1723890860 reap_dead start 2023-10-16T09:49:09.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.605+0000 7fab768de640 1 -- 172.21.15.84:0/1689208241 --> [v2:172.21.15.157:6800/3654970329,v1:172.21.15.157:6801/3654970329] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fab68050090 con 0x7fab54073160 2023-10-16T09:49:09.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.606+0000 7fe452901640 1 -- 172.21.15.84:0/1723890860 shutdown_connections 2023-10-16T09:49:09.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.606+0000 7fe452901640 1 -- 172.21.15.84:0/1723890860 >> 172.21.15.84:0/1723890860 conn(0x7fe44c0ce9b0 msgr2=0x7fe44c0cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:09.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.606+0000 7fe452901640 1 -- 172.21.15.84:0/1723890860 shutdown_connections 2023-10-16T09:49:09.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.606+0000 7fe452901640 1 -- 172.21.15.84:0/1723890860 wait complete. 2023-10-16T09:49:09.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.606+0000 7fab6cff9640 1 -- 172.21.15.84:0/1689208241 <== osd.5 v2:172.21.15.157:6800/3654970329 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fab54075c30 con 0x7fab54073160 2023-10-16T09:49:09.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.607+0000 7f4125dc9640 1 -- 172.21.15.84:0/4267825939 --> [v2:172.21.15.157:6816/899624983,v1:172.21.15.157:6817/899624983] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f41200b38e0 con 0x7f410c073130 2023-10-16T09:49:09.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.608+0000 7fab768de640 1 -- 172.21.15.84:0/1689208241 >> [v2:172.21.15.157:6800/3654970329,v1:172.21.15.157:6801/3654970329] conn(0x7fab54073160 msgr2=0x7fab54075580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.608+0000 7fab768de640 1 --2- 172.21.15.84:0/1689208241 >> [v2:172.21.15.157:6800/3654970329,v1:172.21.15.157:6801/3654970329] conn(0x7fab54073160 0x7fab54075580 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.608+0000 7fab768de640 1 -- 172.21.15.84:0/1689208241 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fab5406af40 msgr2=0x7fab5406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.608+0000 7fab768de640 1 --2- 172.21.15.84:0/1689208241 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fab5406af40 0x7fab5406d400 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fab64009db0 tx=0x7fab64008040 comp rx=0 tx=0).stop 2023-10-16T09:49:09.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.608+0000 7fab768de640 1 -- 172.21.15.84:0/1689208241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab680e78e0 msgr2=0x7fab68171870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.608+0000 7fab768de640 1 --2- 172.21.15.84:0/1689208241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab680e78e0 0x7fab68171870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0090b0 tx=0x7fab5c0046c0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.608+0000 7fab74e7b640 1 -- 172.21.15.84:0/1689208241 reap_dead start 2023-10-16T09:49:09.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.608+0000 7f4107fff640 1 -- 172.21.15.84:0/4267825939 <== osd.6 v2:172.21.15.157:6816/899624983 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4114009550 con 0x7f410c073130 2023-10-16T09:49:09.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.610+0000 7fab768de640 1 -- 172.21.15.84:0/1689208241 shutdown_connections 2023-10-16T09:49:09.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.610+0000 7f4125dc9640 1 -- 172.21.15.84:0/4267825939 >> [v2:172.21.15.157:6816/899624983,v1:172.21.15.157:6817/899624983] conn(0x7f410c073130 msgr2=0x7f410c075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.610+0000 7f4125dc9640 1 --2- 172.21.15.84:0/4267825939 >> [v2:172.21.15.157:6816/899624983,v1:172.21.15.157:6817/899624983] conn(0x7f410c073130 0x7f410c075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.610+0000 7fab768de640 1 -- 172.21.15.84:0/1689208241 >> 172.21.15.84:0/1689208241 conn(0x7fab68054c40 msgr2=0x7fab680e9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:09.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.611+0000 7fab768de640 1 -- 172.21.15.84:0/1689208241 shutdown_connections 2023-10-16T09:49:09.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.611+0000 7f4125dc9640 1 -- 172.21.15.84:0/4267825939 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f410c06af10 msgr2=0x7f410c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.611+0000 7f4125dc9640 1 --2- 172.21.15.84:0/4267825939 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f410c06af10 0x7f410c06d3d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4110009850 tx=0x7f4110008030 comp rx=0 tx=0).stop 2023-10-16T09:49:09.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.611+0000 7f4125dc9640 1 -- 172.21.15.84:0/4267825939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41200cfaf0 msgr2=0x7f41200b1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.611+0000 7f4125dc9640 1 --2- 172.21.15.84:0/4267825939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41200cfaf0 0x7f41200b1a60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f411800c780 tx=0x7f411800cc40 comp rx=0 tx=0).stop 2023-10-16T09:49:09.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.611+0000 7fab768de640 1 -- 172.21.15.84:0/1689208241 wait complete. 2023-10-16T09:49:09.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.611+0000 7f411ffff640 1 -- 172.21.15.84:0/4267825939 reap_dead start 2023-10-16T09:49:09.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.611+0000 7f4125dc9640 1 -- 172.21.15.84:0/4267825939 shutdown_connections 2023-10-16T09:49:09.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.611+0000 7f4125dc9640 1 -- 172.21.15.84:0/4267825939 >> 172.21.15.84:0/4267825939 conn(0x7f41200d2cc0 msgr2=0x7f412006bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:09.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.612+0000 7f4125dc9640 1 -- 172.21.15.84:0/4267825939 shutdown_connections 2023-10-16T09:49:09.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.612+0000 7f4125dc9640 1 -- 172.21.15.84:0/4267825939 wait complete. 2023-10-16T09:49:09.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.616+0000 7f2ba2478640 1 -- 172.21.15.84:0/2903684370 --> [v2:172.21.15.84:6816/1933430277,v1:172.21.15.84:6817/1933430277] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2b7c002cb0 con 0x7f2b80073160 2023-10-16T09:49:09.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.617+0000 7f2b91ffb640 1 -- 172.21.15.84:0/2903684370 <== osd.3 v2:172.21.15.84:6816/1933430277 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2b7c002cb0 con 0x7f2b80073160 2023-10-16T09:49:09.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.619+0000 7f2ba2478640 1 -- 172.21.15.84:0/2903684370 >> [v2:172.21.15.84:6816/1933430277,v1:172.21.15.84:6817/1933430277] conn(0x7f2b80073160 msgr2=0x7f2b80075580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.619+0000 7f2ba2478640 1 --2- 172.21.15.84:0/2903684370 >> [v2:172.21.15.84:6816/1933430277,v1:172.21.15.84:6817/1933430277] conn(0x7f2b80073160 0x7f2b80075580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.710 INFO:teuthology.orchestra.run.smithi084.stdout:55834574852 2023-10-16T09:49:09.711 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-16T09:49:09.712 INFO:teuthology.orchestra.run.smithi084.stdout:55834574853 2023-10-16T09:49:09.712 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-16T09:49:09.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.622+0000 7f2ba2478640 1 -- 172.21.15.84:0/2903684370 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2b8006af40 msgr2=0x7f2b8006d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.622+0000 7f2ba2478640 1 --2- 172.21.15.84:0/2903684370 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2b8006af40 0x7f2b8006d400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2b98001790 tx=0x7f2b98009040 comp rx=0 tx=0).stop 2023-10-16T09:49:09.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.622+0000 7f2ba2478640 1 -- 172.21.15.84:0/2903684370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b9c0cded0 msgr2=0x7f2b9c1f4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.622+0000 7f2ba2478640 1 --2- 172.21.15.84:0/2903684370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b9c0cded0 0x7f2b9c1f4980 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400cbc0 tx=0x7f2b9400a060 comp rx=0 tx=0).stop 2023-10-16T09:49:09.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.622+0000 7f2ba1c77640 1 -- 172.21.15.84:0/2903684370 reap_dead start 2023-10-16T09:49:09.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.622+0000 7f2ba2478640 1 -- 172.21.15.84:0/2903684370 shutdown_connections 2023-10-16T09:49:09.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.622+0000 7f2ba2478640 1 -- 172.21.15.84:0/2903684370 >> 172.21.15.84:0/2903684370 conn(0x7f2b9c0cf570 msgr2=0x7f2b9c0b6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:09.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.623+0000 7f2ba2478640 1 -- 172.21.15.84:0/2903684370 shutdown_connections 2023-10-16T09:49:09.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.623+0000 7f2ba2478640 1 -- 172.21.15.84:0/2903684370 wait complete. 2023-10-16T09:49:09.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.624+0000 7f9d789ce640 1 -- 172.21.15.84:0/1613192397 --> [v2:172.21.15.157:6819/1752891116,v1:172.21.15.157:6821/1752891116] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9d740cd7b0 con 0x7f9d54073010 2023-10-16T09:49:09.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.625+0000 7f9d62ffd640 1 -- 172.21.15.84:0/1613192397 <== osd.7 v2:172.21.15.157:6819/1752891116 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9d680013d0 con 0x7f9d54073010 2023-10-16T09:49:09.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.626+0000 7f9d789ce640 1 -- 172.21.15.84:0/1613192397 >> [v2:172.21.15.157:6819/1752891116,v1:172.21.15.157:6821/1752891116] conn(0x7f9d54073010 msgr2=0x7f9d54075430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.626+0000 7f9d789ce640 1 --2- 172.21.15.84:0/1613192397 >> [v2:172.21.15.157:6819/1752891116,v1:172.21.15.157:6821/1752891116] conn(0x7f9d54073010 0x7f9d54075430 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:09.717 INFO:teuthology.orchestra.run.smithi084.stdout:55834574852 2023-10-16T09:49:09.717 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-16T09:49:09.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.629+0000 7f9d789ce640 1 -- 172.21.15.84:0/1613192397 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9d5406ae70 msgr2=0x7f9d5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.629+0000 7f9d789ce640 1 --2- 172.21.15.84:0/1613192397 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9d5406ae70 0x7f9d5406d330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9d640017f0 tx=0x7f9d64009040 comp rx=0 tx=0).stop 2023-10-16T09:49:09.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.629+0000 7f9d789ce640 1 -- 172.21.15.84:0/1613192397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d74159e90 msgr2=0x7f9d7415bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:09.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.629+0000 7f9d789ce640 1 --2- 172.21.15.84:0/1613192397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d74159e90 0x7f9d7415bde0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c034000 tx=0x7f9d6c004a40 comp rx=0 tx=0).stop 2023-10-16T09:49:09.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.629+0000 7f9d72d9d640 1 -- 172.21.15.84:0/1613192397 reap_dead start 2023-10-16T09:49:09.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.629+0000 7f9d789ce640 1 -- 172.21.15.84:0/1613192397 shutdown_connections 2023-10-16T09:49:09.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.629+0000 7f9d789ce640 1 -- 172.21.15.84:0/1613192397 >> 172.21.15.84:0/1613192397 conn(0x7f9d740ce9b0 msgr2=0x7f9d741575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:09.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.629+0000 7f9d789ce640 1 -- 172.21.15.84:0/1613192397 shutdown_connections 2023-10-16T09:49:09.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:09.629+0000 7f9d789ce640 1 -- 172.21.15.84:0/1613192397 wait complete. 2023-10-16T09:49:09.720 INFO:teuthology.orchestra.run.smithi084.stdout:55834574854 2023-10-16T09:49:09.721 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-16T09:49:09.722 INFO:teuthology.orchestra.run.smithi084.stdout:55834574853 2023-10-16T09:49:09.722 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-16T09:49:09.723 INFO:teuthology.orchestra.run.smithi084.stdout:55834574853 2023-10-16T09:49:09.723 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-16T09:49:09.724 INFO:teuthology.orchestra.run.smithi084.stdout:55834574853 2023-10-16T09:49:09.724 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-16T09:49:09.725 INFO:teuthology.orchestra.run.smithi084.stdout:55834574853 2023-10-16T09:49:09.726 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-16T09:49:10.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.039+0000 7ff319c09640 1 Processor -- start 2023-10-16T09:49:10.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.039+0000 7ff319c09640 1 -- start start 2023-10-16T09:49:10.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.041+0000 7ff319c09640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff314158eb0 0x7ff31406a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.042+0000 7ff313fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff3140cf200 0x7ff3140c9c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32850/0 (socket says 172.21.15.84:32850) 2023-10-16T09:49:10.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.042+0000 7ff313fff640 1 -- 172.21.15.84:0/2956265356 learned_addr learned my addr 172.21.15.84:0/2956265356 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:10.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.042+0000 7ff3137fe640 1 --2- 172.21.15.84:0/2956265356 >> v2:172.21.15.157:3300/0 conn(0x7ff314158eb0 0x7ff31406a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.043+0000 7ff319c09640 1 -- 172.21.15.84:0/2956265356 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3140cc2b0 con 0x7ff3140cf200 2023-10-16T09:49:10.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.043+0000 7ff319c09640 1 -- 172.21.15.84:0/2956265356 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3140cc590 con 0x7ff31406a670 2023-10-16T09:49:10.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.043+0000 7ff319c09640 1 -- 172.21.15.84:0/2956265356 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3140cc870 con 0x7ff314158eb0 2023-10-16T09:49:10.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.044+0000 7ff3137fe640 1 -- 172.21.15.84:0/2956265356 >> v1:172.21.15.157:6789/0 conn(0x7ff31406a670 legacy=0x7ff3140cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.044+0000 7ff3127fc640 1 -- 172.21.15.84:0/2956265356 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 704630248 0 0) 0x7ff3140cc2b0 con 0x7ff3140cf200 2023-10-16T09:49:10.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.045+0000 7ff3137fe640 1 -- 172.21.15.84:0/2956265356 >> v1:172.21.15.84:6789/0 conn(0x7ff3140cf200 legacy=0x7ff3140c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.045+0000 7ff3137fe640 1 -- 172.21.15.84:0/2956265356 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3140ccb50 con 0x7ff314158eb0 2023-10-16T09:49:10.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.047+0000 7ff3137fe640 1 --2- 172.21.15.84:0/2956265356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff314158eb0 0x7ff31406a130 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff3080097a0 tx=0x7ff30802ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c24a3bc626335b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.047+0000 7ff3127fc640 1 -- 172.21.15.84:0/2956265356 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30802da60 con 0x7ff314158eb0 2023-10-16T09:49:10.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.048+0000 7ff3127fc640 1 -- 172.21.15.84:0/2956265356 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30802dbc0 con 0x7ff314158eb0 2023-10-16T09:49:10.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.049+0000 7ff319c09640 1 -- 172.21.15.84:0/2956265356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff314158eb0 msgr2=0x7ff31406a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.049+0000 7ff319c09640 1 --2- 172.21.15.84:0/2956265356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff314158eb0 0x7ff31406a130 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff3080097a0 tx=0x7ff30802ccb0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.049+0000 7ff319c09640 1 -- 172.21.15.84:0/2956265356 shutdown_connections 2023-10-16T09:49:10.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.049+0000 7ff319c09640 1 --2- 172.21.15.84:0/2956265356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff314158eb0 0x7ff31406a130 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.049+0000 7ff319c09640 1 -- 172.21.15.84:0/2956265356 >> 172.21.15.84:0/2956265356 conn(0x7ff3140ce6e0 msgr2=0x7ff3140ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:10.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.050+0000 7ff319c09640 1 -- 172.21.15.84:0/2956265356 shutdown_connections 2023-10-16T09:49:10.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.050+0000 7ff319c09640 1 -- 172.21.15.84:0/2956265356 wait complete. 2023-10-16T09:49:10.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.051+0000 7f9eb737c640 1 Processor -- start 2023-10-16T09:49:10.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.052+0000 7f9eb737c640 1 -- start start 2023-10-16T09:49:10.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.053+0000 7ff319c09640 1 Processor -- start 2023-10-16T09:49:10.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.053+0000 7ff319c09640 1 -- start start 2023-10-16T09:49:10.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.055+0000 7ff319c09640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff31406a670 0x7ff31415b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.055+0000 7ff319c09640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3140cf200 0x7ff314157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.055+0000 7ff319c09640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff314158eb0 0x7ff3141f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.055+0000 7ff319c09640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3141f36a0 con 0x7ff31406a670 2023-10-16T09:49:10.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.055+0000 7ff319c09640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3141f3cf0 con 0x7ff3140cf200 2023-10-16T09:49:10.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.055+0000 7ff319c09640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3141f3fd0 con 0x7ff314158eb0 2023-10-16T09:49:10.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.056+0000 7ff313fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff314158eb0 0x7ff3141f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.056+0000 7ff313fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff314158eb0 0x7ff3141f3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:36830/0 (socket says 172.21.15.84:36830) 2023-10-16T09:49:10.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.054+0000 7f9eb737c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb00acfa0 0x7f9eb006fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.054+0000 7f9eb737c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9eb0070380 0x7f9eb016d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.054+0000 7f9eb737c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb008add0 con 0x7f9eb008a9f0 2023-10-16T09:49:10.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.054+0000 7f9eb737c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb00c8e60 con 0x7f9eb0070380 2023-10-16T09:49:10.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.054+0000 7f9eb737c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb0070c30 con 0x7f9eb00acfa0 2023-10-16T09:49:10.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.055+0000 7f9eb4917640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb00acfa0 0x7f9eb006fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.055+0000 7f9eb4917640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb00acfa0 0x7f9eb006fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:36814/0 (socket says 172.21.15.84:36814) 2023-10-16T09:49:10.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.055+0000 7f9eb4917640 1 -- 172.21.15.84:0/4056344967 learned_addr learned my addr 172.21.15.84:0/4056344967 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:10.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.055+0000 7f9eb5919640 1 --2- 172.21.15.84:0/4056344967 >> v2:172.21.15.157:3300/0 conn(0x7f9eb0070380 0x7f9eb016d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.056+0000 7f9eb4917640 1 -- 172.21.15.84:0/4056344967 >> v2:172.21.15.157:3300/0 conn(0x7f9eb0070380 msgr2=0x7f9eb016d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:10.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.056+0000 7f9eb4917640 1 --2- 172.21.15.84:0/4056344967 >> v2:172.21.15.157:3300/0 conn(0x7f9eb0070380 0x7f9eb016d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.056+0000 7f9eb4917640 1 -- 172.21.15.84:0/4056344967 >> v1:172.21.15.157:6789/0 conn(0x7f9eb008a9f0 legacy=0x7f9eb00ac890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T09:49:10.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.056+0000 7f9eb4917640 1 -- 172.21.15.84:0/4056344967 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb0070f10 con 0x7f9eb00acfa0 2023-10-16T09:49:10.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.056+0000 7ff312ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3140cf200 0x7ff314157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.057+0000 7ff312ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3140cf200 0x7ff314157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:55036/0 (socket says 172.21.15.84:55036) 2023-10-16T09:49:10.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.057+0000 7ff312ffd640 1 -- 172.21.15.84:0/602933260 learned_addr learned my addr 172.21.15.84:0/602933260 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:10.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.057+0000 7ff3137fe640 1 --2- 172.21.15.84:0/602933260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff31406a670 0x7ff31415b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.058+0000 7f6018d35640 1 Processor -- start 2023-10-16T09:49:10.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.058+0000 7ff312ffd640 1 -- 172.21.15.84:0/602933260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff314158eb0 msgr2=0x7ff3141f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.058+0000 7f9eb4917640 1 --2- 172.21.15.84:0/4056344967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb00acfa0 0x7f9eb006fe40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8014a00 tx=0x7f9ea803d1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29cf396220446a83 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.058+0000 7f6018d35640 1 -- start start 2023-10-16T09:49:10.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.058+0000 7ff312ffd640 1 --2- 172.21.15.84:0/602933260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff314158eb0 0x7ff3141f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.058+0000 7ff312ffd640 1 -- 172.21.15.84:0/602933260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff31406a670 msgr2=0x7ff31415b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.058+0000 7ff312ffd640 1 --2- 172.21.15.84:0/602933260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff31406a670 0x7ff31415b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.058+0000 7ff312ffd640 1 -- 172.21.15.84:0/602933260 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff308009400 con 0x7ff3140cf200 2023-10-16T09:49:10.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.058+0000 7ff3137fe640 1 --2- 172.21.15.84:0/602933260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff31406a670 0x7ff31415b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:10.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.059+0000 7f9ea7fff640 1 -- 172.21.15.84:0/4056344967 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea80110d0 con 0x7f9eb00acfa0 2023-10-16T09:49:10.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.059+0000 7f9ea7fff640 1 -- 172.21.15.84:0/4056344967 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea8011230 con 0x7f9eb00acfa0 2023-10-16T09:49:10.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.059+0000 7f9ea7fff640 1 -- 172.21.15.84:0/4056344967 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea80115c0 con 0x7f9eb00acfa0 2023-10-16T09:49:10.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.059+0000 7ff312ffd640 1 --2- 172.21.15.84:0/602933260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3140cf200 0x7ff314157a50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00c790 tx=0x7ff30c00cc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.059+0000 7ff2fbfff640 1 -- 172.21.15.84:0/602933260 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c00cea0 con 0x7ff3140cf200 2023-10-16T09:49:10.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.060+0000 7f9eb737c640 1 -- 172.21.15.84:0/4056344967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb00acfa0 msgr2=0x7f9eb006fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.060+0000 7f9eb737c640 1 --2- 172.21.15.84:0/4056344967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb00acfa0 0x7f9eb006fe40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8014a00 tx=0x7f9ea803d1f0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.060+0000 7ff2fbfff640 1 -- 172.21.15.84:0/602933260 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30c006d50 con 0x7ff3140cf200 2023-10-16T09:49:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.060+0000 7f9eb737c640 1 -- 172.21.15.84:0/4056344967 shutdown_connections 2023-10-16T09:49:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.060+0000 7f9eb737c640 1 --2- 172.21.15.84:0/4056344967 >> v2:172.21.15.157:3300/0 conn(0x7f9eb0070380 0x7f9eb016d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.060+0000 7f9eb737c640 1 --2- 172.21.15.84:0/4056344967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb00acfa0 0x7f9eb006fe40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.060+0000 7f9eb737c640 1 -- 172.21.15.84:0/4056344967 >> 172.21.15.84:0/4056344967 conn(0x7f9eb00ce6e0 msgr2=0x7f9eb00ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.060+0000 7f9eb737c640 1 -- 172.21.15.84:0/4056344967 shutdown_connections 2023-10-16T09:49:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.061+0000 7f6018d35640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f600c0e0ed0 0x7f600c038780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.060+0000 7ff2fbfff640 1 -- 172.21.15.84:0/602933260 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c003bb0 con 0x7ff3140cf200 2023-10-16T09:49:10.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.060+0000 7ff319c09640 1 -- 172.21.15.84:0/602933260 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3141f42b0 con 0x7ff3140cf200 2023-10-16T09:49:10.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.060+0000 7ff319c09640 1 -- 172.21.15.84:0/602933260 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3141e26c0 con 0x7ff3140cf200 2023-10-16T09:49:10.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.062+0000 7f6018d35640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f600c039fa0 0x7f600c03a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.062+0000 7f6018d35640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600c050fb0 con 0x7f600c04eac0 2023-10-16T09:49:10.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.062+0000 7f6018d35640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600c04f800 con 0x7f600c0e0ed0 2023-10-16T09:49:10.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.063+0000 7f6018d35640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c04fae0 con 0x7f600c039fa0 2023-10-16T09:49:10.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.060+0000 7f9eb737c640 1 -- 172.21.15.84:0/4056344967 wait complete. 2023-10-16T09:49:10.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.063+0000 7ff319c09640 1 -- 172.21.15.84:0/602933260 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3141f47d0 con 0x7ff3140cf200 2023-10-16T09:49:10.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.063+0000 7f9eb737c640 1 Processor -- start 2023-10-16T09:49:10.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.062+0000 7f6012ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f600c04eac0 0x7f600c0508a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32854/0 (socket says 172.21.15.84:32854) 2023-10-16T09:49:10.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.062+0000 7f60137fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f600c0e0ed0 0x7f600c038780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.063+0000 7f6012ffd640 1 -- 172.21.15.84:0/4093669144 learned_addr learned my addr 172.21.15.84:0/4093669144 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:10.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.063+0000 7f6013fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f600c039fa0 0x7f600c03a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.063+0000 7f9eb737c640 1 -- start start 2023-10-16T09:49:10.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.064+0000 7f60137fe640 1 -- 172.21.15.84:0/4093669144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f600c039fa0 msgr2=0x7f600c03a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:10.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.064+0000 7f60137fe640 1 --2- 172.21.15.84:0/4093669144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f600c039fa0 0x7f600c03a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.064+0000 7f60137fe640 1 -- 172.21.15.84:0/4093669144 >> v1:172.21.15.84:6789/0 conn(0x7f600c04eac0 legacy=0x7f600c0508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.064+0000 7f60137fe640 1 -- 172.21.15.84:0/4093669144 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c04fdc0 con 0x7f600c0e0ed0 2023-10-16T09:49:10.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.065+0000 7f9eb737c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb0070380 0x7f9eb016ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.065+0000 7f9eb737c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9eb008a9f0 0x7f9eb00c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.066+0000 7f9eb737c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb00acfa0 0x7f9eb00c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.065+0000 7ff2fbfff640 1 -- 172.21.15.84:0/602933260 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff30c003d10 con 0x7ff3140cf200 2023-10-16T09:49:10.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.065+0000 7f60137fe640 1 --2- 172.21.15.84:0/4093669144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f600c0e0ed0 0x7f600c038780 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f60080097a0 tx=0x7f600802b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea17a405c3fbc96 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.066+0000 7f9eb5118640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb0070380 0x7f9eb016ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.066+0000 7f9eb5118640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb0070380 0x7f9eb016ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:36860/0 (socket says 172.21.15.84:36860) 2023-10-16T09:49:10.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.066+0000 7f9eb5118640 1 -- 172.21.15.84:0/1507136532 learned_addr learned my addr 172.21.15.84:0/1507136532 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:10.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.066+0000 7f9eb737c640 1 -- 172.21.15.84:0/1507136532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb00c8310 con 0x7f9eb008a9f0 2023-10-16T09:49:10.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.066+0000 7f9eb4917640 1 --2- 172.21.15.84:0/1507136532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9eb008a9f0 0x7f9eb00c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.066+0000 7f9eb737c640 1 -- 172.21.15.84:0/1507136532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb00c63f0 con 0x7f9eb00acfa0 2023-10-16T09:49:10.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.066+0000 7f9eb737c640 1 -- 172.21.15.84:0/1507136532 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb016b8a0 con 0x7f9eb0070380 2023-10-16T09:49:10.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.067+0000 7f9eb5919640 1 --2- 172.21.15.84:0/1507136532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb00acfa0 0x7f9eb00c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.067+0000 7f60127fc640 1 -- 172.21.15.84:0/4093669144 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6008030710 con 0x7f600c0e0ed0 2023-10-16T09:49:10.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.067+0000 7ff2fbfff640 1 --2- 172.21.15.84:0/602933260 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff30006ae70 0x7ff30006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.067+0000 7ff2fbfff640 1 -- 172.21.15.84:0/602933260 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff30c088a60 con 0x7ff3140cf200 2023-10-16T09:49:10.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.067+0000 7f60127fc640 1 -- 172.21.15.84:0/4093669144 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600802e050 con 0x7f600c0e0ed0 2023-10-16T09:49:10.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.067+0000 7f60127fc640 1 -- 172.21.15.84:0/4093669144 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6008030a90 con 0x7f600c0e0ed0 2023-10-16T09:49:10.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.067+0000 7f9eb5919640 1 -- 172.21.15.84:0/1507136532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb0070380 msgr2=0x7f9eb016ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.068+0000 7ff3137fe640 1 --2- 172.21.15.84:0/602933260 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff30006ae70 0x7ff30006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.067+0000 7f9eb5919640 1 --2- 172.21.15.84:0/1507136532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb0070380 0x7f9eb016ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.067+0000 7f9eb5919640 1 -- 172.21.15.84:0/1507136532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9eb008a9f0 msgr2=0x7f9eb00c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.068+0000 7f9eb5919640 1 --2- 172.21.15.84:0/1507136532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9eb008a9f0 0x7f9eb00c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.068+0000 7f9eb5919640 1 -- 172.21.15.84:0/1507136532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea8014660 con 0x7f9eb00acfa0 2023-10-16T09:49:10.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.068+0000 7f9eb4917640 1 --2- 172.21.15.84:0/1507136532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9eb008a9f0 0x7f9eb00c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:49:10.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.068+0000 7f6018d35640 1 -- 172.21.15.84:0/4093669144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f600c0e0ed0 msgr2=0x7f600c038780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.068+0000 7f6018d35640 1 --2- 172.21.15.84:0/4093669144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f600c0e0ed0 0x7f600c038780 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f60080097a0 tx=0x7f600802b5d0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.069+0000 7f6018d35640 1 -- 172.21.15.84:0/4093669144 shutdown_connections 2023-10-16T09:49:10.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.069+0000 7f6018d35640 1 --2- 172.21.15.84:0/4093669144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f600c039fa0 0x7f600c03a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.069+0000 7f6018d35640 1 --2- 172.21.15.84:0/4093669144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f600c0e0ed0 0x7f600c038780 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.069+0000 7f6018d35640 1 -- 172.21.15.84:0/4093669144 >> 172.21.15.84:0/4093669144 conn(0x7f600c054c10 msgr2=0x7f600c054ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:10.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.069+0000 7ff3137fe640 1 --2- 172.21.15.84:0/602933260 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff30006ae70 0x7ff30006d330 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff308037000 tx=0x7ff308031000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.069+0000 7f6018d35640 1 -- 172.21.15.84:0/4093669144 shutdown_connections 2023-10-16T09:49:10.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.069+0000 7f6018d35640 1 -- 172.21.15.84:0/4093669144 wait complete. 2023-10-16T09:49:10.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.070+0000 7f9eb5919640 1 --2- 172.21.15.84:0/1507136532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb00acfa0 0x7f9eb00c6810 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000e7f0 tx=0x7f9ea000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.072+0000 7f6018d35640 1 Processor -- start 2023-10-16T09:49:10.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.072+0000 7f6018d35640 1 -- start start 2023-10-16T09:49:10.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.073+0000 7f29c316a640 1 Processor -- start 2023-10-16T09:49:10.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.073+0000 7f29c316a640 1 -- start start 2023-10-16T09:49:10.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.073+0000 7f6018d35640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f600c039fa0 0x7f600c0527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.073+0000 7f6018d35640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f600c04eac0 0x7f600c0d29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.073+0000 7f6018d35640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f600c0e0ed0 0x7f600c033560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.073+0000 7f6018d35640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c033aa0 con 0x7f600c0e0ed0 2023-10-16T09:49:10.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.073+0000 7f6018d35640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0cfbc0 con 0x7f600c04eac0 2023-10-16T09:49:10.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.073+0000 7f6018d35640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c053ae0 con 0x7f600c039fa0 2023-10-16T09:49:10.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.074+0000 7f6013fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f600c0e0ed0 0x7f600c033560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.074+0000 7f6013fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f600c0e0ed0 0x7f600c033560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59624/0 (socket says 172.21.15.84:59624) 2023-10-16T09:49:10.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.074+0000 7f6013fff640 1 -- 172.21.15.84:0/2426694830 learned_addr learned my addr 172.21.15.84:0/2426694830 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:10.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.074+0000 7f60137fe640 1 --2- 172.21.15.84:0/2426694830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f600c039fa0 0x7f600c0527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.074+0000 7f6012ffd640 1 --2- 172.21.15.84:0/2426694830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f600c04eac0 0x7f600c0d29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.075+0000 7f6013fff640 1 -- 172.21.15.84:0/2426694830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f600c039fa0 msgr2=0x7f600c0527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.075+0000 7f6013fff640 1 --2- 172.21.15.84:0/2426694830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f600c039fa0 0x7f600c0527c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.075+0000 7f6013fff640 1 -- 172.21.15.84:0/2426694830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f600c04eac0 msgr2=0x7f600c0d29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.075+0000 7f29c316a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29bc15cd70 0x7f29bc0d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.075+0000 7f29c316a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29bc0cded0 0x7f29bc0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.075+0000 7f29c316a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29bc15d150 con 0x7f29bc0b8480 2023-10-16T09:49:10.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.075+0000 7f29c316a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29bc0d1180 con 0x7f29bc0cded0 2023-10-16T09:49:10.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.075+0000 7f29c316a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0d1460 con 0x7f29bc15cd70 2023-10-16T09:49:10.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.075+0000 7f6013fff640 1 --2- 172.21.15.84:0/2426694830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f600c04eac0 0x7f600c0d29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.075+0000 7f6013fff640 1 -- 172.21.15.84:0/2426694830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6008009400 con 0x7f600c0e0ed0 2023-10-16T09:49:10.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.075+0000 7f60137fe640 1 --2- 172.21.15.84:0/2426694830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f600c039fa0 0x7f600c0527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:10.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.075+0000 7f6012ffd640 1 --2- 172.21.15.84:0/2426694830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f600c04eac0 0x7f600c0d29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:10.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.075+0000 7f6013fff640 1 --2- 172.21.15.84:0/2426694830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f600c0e0ed0 0x7f600c033560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f60140687d0 tx=0x7f6014068000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.076+0000 7f29c2168640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29bc15cd70 0x7f29bc0d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.076+0000 7f5ffffff640 1 -- 172.21.15.84:0/2426694830 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601408f6a0 con 0x7f600c0e0ed0 2023-10-16T09:49:10.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.076+0000 7f5ffffff640 1 -- 172.21.15.84:0/2426694830 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601408f800 con 0x7f600c0e0ed0 2023-10-16T09:49:10.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.076+0000 7f29c1967640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f29bc0b8480 0x7f29bc0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:57456/0 (socket says 172.21.15.84:57456) 2023-10-16T09:49:10.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.076+0000 7f29c1967640 1 -- 172.21.15.84:0/3661580861 learned_addr learned my addr 172.21.15.84:0/3661580861 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:10.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.076+0000 7f5ffffff640 1 -- 172.21.15.84:0/2426694830 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601408fbb0 con 0x7f600c0e0ed0 2023-10-16T09:49:10.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.076+0000 7f6018d35640 1 -- 172.21.15.84:0/2426694830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600c052ed0 con 0x7f600c0e0ed0 2023-10-16T09:49:10.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.076+0000 7f29c2969640 1 --2- 172.21.15.84:0/3661580861 >> v2:172.21.15.157:3300/0 conn(0x7f29bc0cded0 0x7f29bc0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.076+0000 7fc9a9778640 1 Processor -- start 2023-10-16T09:49:10.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.076+0000 7f6018d35640 1 -- 172.21.15.84:0/2426694830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600c053490 con 0x7f600c0e0ed0 2023-10-16T09:49:10.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.077+0000 7fc9a9778640 1 -- start start 2023-10-16T09:49:10.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.077+0000 7f29c2168640 1 -- 172.21.15.84:0/3661580861 >> v2:172.21.15.157:3300/0 conn(0x7f29bc0cded0 msgr2=0x7f29bc0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:10.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.077+0000 7f29c2168640 1 --2- 172.21.15.84:0/3661580861 >> v2:172.21.15.157:3300/0 conn(0x7f29bc0cded0 0x7f29bc0b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.077+0000 7f29c2168640 1 -- 172.21.15.84:0/3661580861 >> v1:172.21.15.157:6789/0 conn(0x7f29bc0b8480 legacy=0x7f29bc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.077+0000 7f29c2168640 1 -- 172.21.15.84:0/3661580861 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc0d1740 con 0x7f29bc15cd70 2023-10-16T09:49:10.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.078+0000 7f29c2168640 1 --2- 172.21.15.84:0/3661580861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29bc15cd70 0x7f29bc0d2220 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f29b80097a0 tx=0x7f29b802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e4eb40d08e45bbd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.078+0000 7fc9a9778640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc9a4159e40 0x7fc9a40cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.078+0000 7f5ffffff640 1 -- 172.21.15.84:0/2426694830 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f601408fd10 con 0x7f600c0e0ed0 2023-10-16T09:49:10.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.079+0000 7fc9a9778640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9a40ca780 0x7fc9a40705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.079+0000 7fc9a9778640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a40c3610 con 0x7fc9a4070b00 2023-10-16T09:49:10.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.079+0000 7fc9a9778640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a415a220 con 0x7fc9a4159e40 2023-10-16T09:49:10.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.079+0000 7fc9a9778640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a40cafd0 con 0x7fc9a40ca780 2023-10-16T09:49:10.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.079+0000 7fc9a2ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc9a4159e40 0x7fc9a40cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.079+0000 7fc9a2ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc9a4159e40 0x7fc9a40cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59640/0 (socket says 172.21.15.84:59640) 2023-10-16T09:49:10.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.079+0000 7fc9a2ffd640 1 -- 172.21.15.84:0/1312110273 learned_addr learned my addr 172.21.15.84:0/1312110273 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:10.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.079+0000 7fc9a27fc640 1 --2- 172.21.15.84:0/1312110273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9a40ca780 0x7fc9a40705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.080+0000 7f29c1166640 1 -- 172.21.15.84:0/3661580861 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b802d040 con 0x7f29bc15cd70 2023-10-16T09:49:10.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.080+0000 7fc9a2ffd640 1 -- 172.21.15.84:0/1312110273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9a40ca780 msgr2=0x7fc9a40705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:10.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.080+0000 7fc9a2ffd640 1 --2- 172.21.15.84:0/1312110273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9a40ca780 0x7fc9a40705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.080+0000 7fc9a2ffd640 1 -- 172.21.15.84:0/1312110273 >> v1:172.21.15.157:6789/0 conn(0x7fc9a4070b00 legacy=0x7fc9a406ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:10.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.080+0000 7f29c1166640 1 -- 172.21.15.84:0/3661580861 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b8031810 con 0x7f29bc15cd70 2023-10-16T09:49:10.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.080+0000 7f29c1166640 1 -- 172.21.15.84:0/3661580861 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8031ba0 con 0x7f29bc15cd70 2023-10-16T09:49:10.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.080+0000 7f5ffffff640 1 --2- 172.21.15.84:0/2426694830 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f5ff806ae70 0x7f5ff806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.080+0000 7f5ffffff640 1 -- 172.21.15.84:0/2426694830 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6014103080 con 0x7f600c0e0ed0 2023-10-16T09:49:10.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.080+0000 7fc9a2ffd640 1 -- 172.21.15.84:0/1312110273 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a40cb2b0 con 0x7fc9a4159e40 2023-10-16T09:49:10.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.081+0000 7f60137fe640 1 --2- 172.21.15.84:0/2426694830 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f5ff806ae70 0x7f5ff806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.081+0000 7f29c316a640 1 -- 172.21.15.84:0/3661580861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29bc15cd70 msgr2=0x7f29bc0d2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.081+0000 7f29c316a640 1 --2- 172.21.15.84:0/3661580861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29bc15cd70 0x7f29bc0d2220 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f29b80097a0 tx=0x7f29b802b8b0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.081+0000 7f29c316a640 1 -- 172.21.15.84:0/3661580861 shutdown_connections 2023-10-16T09:49:10.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.081+0000 7f29c316a640 1 --2- 172.21.15.84:0/3661580861 >> v2:172.21.15.157:3300/0 conn(0x7f29bc0cded0 0x7f29bc0b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.081+0000 7f29c316a640 1 --2- 172.21.15.84:0/3661580861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29bc15cd70 0x7f29bc0d2220 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f29b80097a0 tx=0x7f29b802b8b0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.081+0000 7f29c316a640 1 -- 172.21.15.84:0/3661580861 >> 172.21.15.84:0/3661580861 conn(0x7f29bc0cf570 msgr2=0x7f29bc0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:10.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.081+0000 7fc9a27fc640 1 --2- 172.21.15.84:0/1312110273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9a40ca780 0x7fc9a40705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:10.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.081+0000 7f6018d35640 1 -- 172.21.15.84:0/2426694830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600c053c50 con 0x7f600c0e0ed0 2023-10-16T09:49:10.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.081+0000 7f29c316a640 1 -- 172.21.15.84:0/3661580861 shutdown_connections 2023-10-16T09:49:10.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.082+0000 7fc9a2ffd640 1 --2- 172.21.15.84:0/1312110273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a4159e40 0x7fc9a40cda70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009870 tx=0x7fc98c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e83ce41c4a5c608 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.082+0000 7f60137fe640 1 --2- 172.21.15.84:0/2426694830 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f5ff806ae70 0x7f5ff806d330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6008034000 tx=0x7f6008008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.082+0000 7f29c316a640 1 -- 172.21.15.84:0/3661580861 wait complete. 2023-10-16T09:49:10.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.083+0000 7f6a86b73640 1 Processor -- start 2023-10-16T09:49:10.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.083+0000 7f6a86b73640 1 -- start start 2023-10-16T09:49:10.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.083+0000 7f29c316a640 1 Processor -- start 2023-10-16T09:49:10.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.084+0000 7f29c316a640 1 -- start start 2023-10-16T09:49:10.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.084+0000 7f9ea5ffb640 1 -- 172.21.15.84:0/1507136532 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea000c730 con 0x7f9eb00acfa0 2023-10-16T09:49:10.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.084+0000 7f9ea5ffb640 1 -- 172.21.15.84:0/1507136532 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea000c890 con 0x7f9eb00acfa0 2023-10-16T09:49:10.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.084+0000 7f9ea5ffb640 1 -- 172.21.15.84:0/1507136532 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea000cc80 con 0x7f9eb00acfa0 2023-10-16T09:49:10.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.084+0000 7f9eb737c640 1 -- 172.21.15.84:0/1507136532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb00c38f0 con 0x7f9eb00acfa0 2023-10-16T09:49:10.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.084+0000 7f9eb737c640 1 -- 172.21.15.84:0/1507136532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb00c3610 con 0x7f9eb00acfa0 2023-10-16T09:49:10.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.085+0000 7f6a86b73640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6a800ce370 0x7f6a80159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.085+0000 7f6a86b73640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a800700b0 0x7f6a8015e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.085+0000 7f6a86b73640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a800cd1f0 con 0x7f6a8006fc50 2023-10-16T09:49:10.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.085+0000 7f6a86b73640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a8015d890 con 0x7f6a800ce370 2023-10-16T09:49:10.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.085+0000 7f6a86b73640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a800cd4d0 con 0x7f6a800700b0 2023-10-16T09:49:10.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.085+0000 7f6a77fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6a8006fc50 0x7f6a8015dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32856/0 (socket says 172.21.15.84:32856) 2023-10-16T09:49:10.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.085+0000 7f6a77fff640 1 -- 172.21.15.84:0/1743653899 learned_addr learned my addr 172.21.15.84:0/1743653899 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:10.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.086+0000 7ff2fbfff640 1 -- 172.21.15.84:0/602933260 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff3141f47d0 con 0x7ff3140cf200 2023-10-16T09:49:10.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.085+0000 7f3eae7e2640 1 Processor -- start 2023-10-16T09:49:10.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.085+0000 7f3eae7e2640 1 -- start start 2023-10-16T09:49:10.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.086+0000 7f6a8490f640 1 --2- 172.21.15.84:0/1743653899 >> v2:172.21.15.84:3300/0 conn(0x7f6a800ce370 0x7f6a80159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.086+0000 7f6a85110640 1 --2- 172.21.15.84:0/1743653899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a800700b0 0x7f6a8015e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.086+0000 7f29c316a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29bc0b8480 0x7f29bc0b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.086+0000 7f29c316a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29bc0cded0 0x7f29bc15da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.086+0000 7f29c316a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29bc15ffe0 0x7f29bc15df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.086+0000 7f29c2969640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29bc15ffe0 0x7f29bc15df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.086+0000 7f3eae7e2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea0033fc0 0x7f3ea0053150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.086+0000 7f3eae7e2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3ea000c1f0 0x7f3ea0050230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.086+0000 7f3eae7e2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea00543f0 con 0x7f3ea00e2e10 2023-10-16T09:49:10.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.086+0000 7f3eae7e2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea00343a0 con 0x7f3ea000c1f0 2023-10-16T09:49:10.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.086+0000 7f3eae7e2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00537e0 con 0x7f3ea0033fc0 2023-10-16T09:49:10.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.086+0000 7f29c2969640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29bc15ffe0 0x7f29bc15df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59664/0 (socket says 172.21.15.84:59664) 2023-10-16T09:49:10.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.086+0000 7f29c2969640 1 -- 172.21.15.84:0/4096139593 learned_addr learned my addr 172.21.15.84:0/4096139593 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:10.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.086+0000 7f29c1967640 1 --2- 172.21.15.84:0/4096139593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29bc0cded0 0x7f29bc15da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.087+0000 7f29c2168640 1 --2- 172.21.15.84:0/4096139593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29bc0b8480 0x7f29bc0b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.087+0000 7f29c316a640 1 -- 172.21.15.84:0/4096139593 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0b6fd0 con 0x7f29bc15ffe0 2023-10-16T09:49:10.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.087+0000 7f6a8490f640 1 -- 172.21.15.84:0/1743653899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a800700b0 msgr2=0x7f6a8015e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:10.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.087+0000 7f6a8490f640 1 --2- 172.21.15.84:0/1743653899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a800700b0 0x7f6a8015e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.087+0000 7f6a8490f640 1 -- 172.21.15.84:0/1743653899 >> v1:172.21.15.84:6789/0 conn(0x7f6a8006fc50 legacy=0x7f6a8015dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.087+0000 7f6a8490f640 1 -- 172.21.15.84:0/1743653899 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a800cd7b0 con 0x7f6a800ce370 2023-10-16T09:49:10.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.087+0000 7f3eadfe1640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3ea00e2e10 0x7f3ea00e32a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32868/0 (socket says 172.21.15.84:32868) 2023-10-16T09:49:10.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.087+0000 7f3eadfe1640 1 -- 172.21.15.84:0/777160693 learned_addr learned my addr 172.21.15.84:0/777160693 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:10.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.088+0000 7f3ead7e0640 1 --2- 172.21.15.84:0/777160693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea0033fc0 0x7f3ea0053150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.087+0000 7f9eb737c640 1 -- 172.21.15.84:0/1507136532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb00c6530 con 0x7f9eb00acfa0 2023-10-16T09:49:10.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.089+0000 7f29c316a640 1 -- 172.21.15.84:0/4096139593 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0b72b0 con 0x7f29bc0b8480 2023-10-16T09:49:10.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.089+0000 7f5ffffff640 1 -- 172.21.15.84:0/2426694830 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f60140ce090 con 0x7f600c0e0ed0 2023-10-16T09:49:10.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.089+0000 7fc9a1ffb640 1 -- 172.21.15.84:0/1312110273 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c031870 con 0x7fc9a4159e40 2023-10-16T09:49:10.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.089+0000 7fc9a1ffb640 1 -- 172.21.15.84:0/1312110273 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98c0319d0 con 0x7fc9a4159e40 2023-10-16T09:49:10.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.089+0000 7fc9a1ffb640 1 -- 172.21.15.84:0/1312110273 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c031da0 con 0x7fc9a4159e40 2023-10-16T09:49:10.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.089+0000 7f29c316a640 1 -- 172.21.15.84:0/4096139593 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0c74a0 con 0x7f29bc0cded0 2023-10-16T09:49:10.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.090+0000 7f6a8490f640 1 --2- 172.21.15.84:0/1743653899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a800ce370 0x7f6a80159ab0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0097a0 tx=0x7f6a7c02b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d1b1ae348c29e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.090+0000 7fc9a9778640 1 -- 172.21.15.84:0/1312110273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a4159e40 msgr2=0x7fc9a40cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.090+0000 7fc9a9778640 1 --2- 172.21.15.84:0/1312110273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a4159e40 0x7fc9a40cda70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009870 tx=0x7fc98c02b910 comp rx=0 tx=0).stop 2023-10-16T09:49:10.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.090+0000 7f29c1967640 1 -- 172.21.15.84:0/4096139593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29bc0b8480 msgr2=0x7f29bc0b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.090+0000 7f29c1967640 1 --2- 172.21.15.84:0/4096139593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29bc0b8480 0x7f29bc0b6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.090+0000 7f29c1967640 1 -- 172.21.15.84:0/4096139593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29bc15ffe0 msgr2=0x7f29bc15df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.090+0000 7f29c1967640 1 --2- 172.21.15.84:0/4096139593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29bc15ffe0 0x7f29bc15df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.090+0000 7f29c1967640 1 -- 172.21.15.84:0/4096139593 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b8009400 con 0x7f29bc0cded0 2023-10-16T09:49:10.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.090+0000 7f29c2969640 1 --2- 172.21.15.84:0/4096139593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29bc15ffe0 0x7f29bc15df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:49:10.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.090+0000 7f6a777fe640 1 -- 172.21.15.84:0/1743653899 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c031520 con 0x7f6a800ce370 2023-10-16T09:49:10.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.090+0000 7f6a777fe640 1 -- 172.21.15.84:0/1743653899 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7c031680 con 0x7f6a800ce370 2023-10-16T09:49:10.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.090+0000 7f6a777fe640 1 -- 172.21.15.84:0/1743653899 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c031a50 con 0x7f6a800ce370 2023-10-16T09:49:10.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.089+0000 7f3ead7e0640 1 -- 172.21.15.84:0/777160693 >> v2:172.21.15.157:3300/0 conn(0x7f3ea000c1f0 msgr2=0x7f3ea0050230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T09:49:10.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.089+0000 7f3ead7e0640 1 --2- 172.21.15.84:0/777160693 >> v2:172.21.15.157:3300/0 conn(0x7f3ea000c1f0 0x7f3ea0050230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.089+0000 7f3ead7e0640 1 -- 172.21.15.84:0/777160693 >> v1:172.21.15.84:6789/0 conn(0x7f3ea00e2e10 legacy=0x7f3ea00e32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.089+0000 7f3ead7e0640 1 -- 172.21.15.84:0/777160693 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea0053ac0 con 0x7f3ea0033fc0 2023-10-16T09:49:10.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.089+0000 7f3e9ffff640 1 -- 172.21.15.84:0/777160693 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3173363260 0 0) 0x7f3ea00543f0 con 0x7f3ea00e2e10 2023-10-16T09:49:10.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.091+0000 7f9ea5ffb640 1 -- 172.21.15.84:0/1507136532 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9ea0013470 con 0x7f9eb00acfa0 2023-10-16T09:49:10.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.091+0000 7fc9a9778640 1 -- 172.21.15.84:0/1312110273 shutdown_connections 2023-10-16T09:49:10.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.091+0000 7fc9a9778640 1 --2- 172.21.15.84:0/1312110273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9a40ca780 0x7fc9a40705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.091+0000 7fc9a9778640 1 --2- 172.21.15.84:0/1312110273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a4159e40 0x7fc9a40cda70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.091+0000 7fc9a9778640 1 -- 172.21.15.84:0/1312110273 >> 172.21.15.84:0/1312110273 conn(0x7fc9a40cfac0 msgr2=0x7fc9a40cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:10.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.091+0000 7f29c2168640 1 --2- 172.21.15.84:0/4096139593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29bc0b8480 0x7f29bc0b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:10.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.091+0000 7f29c1967640 1 --2- 172.21.15.84:0/4096139593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29bc0cded0 0x7f29bc15da50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00c440 tx=0x7f29ac001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.091+0000 7f6a86b73640 1 -- 172.21.15.84:0/1743653899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a800ce370 msgr2=0x7f6a80159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.091+0000 7f6a86b73640 1 --2- 172.21.15.84:0/1743653899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a800ce370 0x7f6a80159ab0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0097a0 tx=0x7f6a7c02b7c0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.091+0000 7fc9a9778640 1 -- 172.21.15.84:0/1312110273 shutdown_connections 2023-10-16T09:49:10.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.091+0000 7fc9a9778640 1 -- 172.21.15.84:0/1312110273 wait complete. 2023-10-16T09:49:10.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.091+0000 7f29b2ffd640 1 -- 172.21.15.84:0/4096139593 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac005ce0 con 0x7f29bc0cded0 2023-10-16T09:49:10.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.092+0000 7f29b2ffd640 1 -- 172.21.15.84:0/4096139593 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac005e40 con 0x7f29bc0cded0 2023-10-16T09:49:10.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.092+0000 7f29c316a640 1 -- 172.21.15.84:0/4096139593 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29bc0c7780 con 0x7f29bc0cded0 2023-10-16T09:49:10.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.092+0000 7f29c316a640 1 -- 172.21.15.84:0/4096139593 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29bc0b6ba0 con 0x7f29bc0cded0 2023-10-16T09:49:10.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.092+0000 7f29b2ffd640 1 -- 172.21.15.84:0/4096139593 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac0072d0 con 0x7f29bc0cded0 2023-10-16T09:49:10.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.092+0000 7f6a86b73640 1 -- 172.21.15.84:0/1743653899 shutdown_connections 2023-10-16T09:49:10.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.092+0000 7f6a86b73640 1 --2- 172.21.15.84:0/1743653899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a800700b0 0x7f6a8015e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.092+0000 7f6a86b73640 1 --2- 172.21.15.84:0/1743653899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a800ce370 0x7f6a80159ab0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.092+0000 7f6a86b73640 1 -- 172.21.15.84:0/1743653899 >> 172.21.15.84:0/1743653899 conn(0x7f6a800ce9b0 msgr2=0x7f6a800cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:10.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.092+0000 7f3ead7e0640 1 --2- 172.21.15.84:0/777160693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea0033fc0 0x7f3ea0053150 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40097a0 tx=0x7f3ea402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f68d3b7d8a615b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.092+0000 7f3e9ffff640 1 -- 172.21.15.84:0/777160693 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4031640 con 0x7f3ea0033fc0 2023-10-16T09:49:10.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.092+0000 7f3e9ffff640 1 -- 172.21.15.84:0/777160693 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea40317a0 con 0x7f3ea0033fc0 2023-10-16T09:49:10.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.092+0000 7f3e9ffff640 1 -- 172.21.15.84:0/777160693 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4031b30 con 0x7f3ea0033fc0 2023-10-16T09:49:10.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.092+0000 7f6a86b73640 1 -- 172.21.15.84:0/1743653899 shutdown_connections 2023-10-16T09:49:10.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.092+0000 7f6a86b73640 1 -- 172.21.15.84:0/1743653899 wait complete. 2023-10-16T09:49:10.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.093+0000 7f3eae7e2640 1 -- 172.21.15.84:0/777160693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea0033fc0 msgr2=0x7f3ea0053150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.093+0000 7f3eae7e2640 1 --2- 172.21.15.84:0/777160693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea0033fc0 0x7f3ea0053150 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40097a0 tx=0x7f3ea402b8f0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.093+0000 7f29c316a640 1 -- 172.21.15.84:0/4096139593 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a4005250 con 0x7f29bc0cded0 2023-10-16T09:49:10.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.093+0000 7f9ea5ffb640 1 --2- 172.21.15.84:0/1507136532 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9e9406ae70 0x7f9e9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.093+0000 7f9ea5ffb640 1 -- 172.21.15.84:0/1507136532 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ea0089580 con 0x7f9eb00acfa0 2023-10-16T09:49:10.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.094+0000 7f9eb5118640 1 --2- 172.21.15.84:0/1507136532 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9e9406ae70 0x7f9e9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.094+0000 7fc9a9778640 1 Processor -- start 2023-10-16T09:49:10.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.095+0000 7f9eb5118640 1 --2- 172.21.15.84:0/1507136532 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9e9406ae70 0x7f9e9406d330 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00c51f0 tx=0x7f9eac009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.094+0000 7fc9a9778640 1 -- start start 2023-10-16T09:49:10.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.096+0000 7fc9a9778640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9a4070b00 0x7fc9a415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.096+0000 7fc9a9778640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9a40ca780 0x7fc9a40c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.096+0000 7fc9a9778640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a4159e40 0x7fc9a4201eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.096+0000 7fc9a9778640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a40c7740 con 0x7fc9a4159e40 2023-10-16T09:49:10.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.096+0000 7fc9a9778640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a40c79a0 con 0x7fc9a4070b00 2023-10-16T09:49:10.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.096+0000 7fc9a9778640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a40c7c80 con 0x7fc9a40ca780 2023-10-16T09:49:10.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.096+0000 7f3eae7e2640 1 -- 172.21.15.84:0/777160693 shutdown_connections 2023-10-16T09:49:10.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.096+0000 7f3eae7e2640 1 --2- 172.21.15.84:0/777160693 >> v2:172.21.15.157:3300/0 conn(0x7f3ea000c1f0 0x7f3ea0050230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.096+0000 7f3eae7e2640 1 --2- 172.21.15.84:0/777160693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea0033fc0 0x7f3ea0053150 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.096+0000 7f3eae7e2640 1 -- 172.21.15.84:0/777160693 >> 172.21.15.84:0/777160693 conn(0x7f3ea0054c10 msgr2=0x7f3ea0054ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:10.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.096+0000 7f3eae7e2640 1 -- 172.21.15.84:0/777160693 shutdown_connections 2023-10-16T09:49:10.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.097+0000 7fc9a27fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9a40ca780 0x7fc9a40c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.097+0000 7fc9a27fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9a40ca780 0x7fc9a40c7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:36912/0 (socket says 172.21.15.84:36912) 2023-10-16T09:49:10.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.097+0000 7fc9a27fc640 1 -- 172.21.15.84:0/2307186348 learned_addr learned my addr 172.21.15.84:0/2307186348 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:10.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.097+0000 7fc9a37fe640 1 --2- 172.21.15.84:0/2307186348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a4159e40 0x7fc9a4201eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.097+0000 7f29b2ffd640 1 -- 172.21.15.84:0/4096139593 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f29ac0074f0 con 0x7f29bc0cded0 2023-10-16T09:49:10.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.098+0000 7fc9a27fc640 1 -- 172.21.15.84:0/2307186348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9a4070b00 msgr2=0x7fc9a415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.098+0000 7fc9a27fc640 1 --2- 172.21.15.84:0/2307186348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9a4070b00 0x7fc9a415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.098+0000 7fc9a27fc640 1 -- 172.21.15.84:0/2307186348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a4159e40 msgr2=0x7fc9a4201eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.098+0000 7fc9a27fc640 1 --2- 172.21.15.84:0/2307186348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a4159e40 0x7fc9a4201eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.098+0000 7fc9a27fc640 1 -- 172.21.15.84:0/2307186348 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98c0094d0 con 0x7fc9a40ca780 2023-10-16T09:49:10.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.098+0000 7fc9a37fe640 1 --2- 172.21.15.84:0/2307186348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a4159e40 0x7fc9a4201eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:49:10.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.098+0000 7f29b2ffd640 1 --2- 172.21.15.84:0/4096139593 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f29a006ae70 0x7f29a006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.098+0000 7f29b2ffd640 1 -- 172.21.15.84:0/4096139593 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29ac088cf0 con 0x7f29bc0cded0 2023-10-16T09:49:10.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.099+0000 7f29c2168640 1 --2- 172.21.15.84:0/4096139593 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f29a006ae70 0x7f29a006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.096+0000 7f3eae7e2640 1 -- 172.21.15.84:0/777160693 wait complete. 2023-10-16T09:49:10.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.099+0000 7f6a86b73640 1 Processor -- start 2023-10-16T09:49:10.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.099+0000 7fc9a27fc640 1 --2- 172.21.15.84:0/2307186348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9a40ca780 0x7fc9a40c7200 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc99400d7d0 tx=0x7fc99400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.099+0000 7f6a86b73640 1 -- start start 2023-10-16T09:49:10.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.099+0000 7f3eae7e2640 1 Processor -- start 2023-10-16T09:49:10.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.100+0000 7f29c2168640 1 --2- 172.21.15.84:0/4096139593 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f29a006ae70 0x7f29a006d330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f29b8036000 tx=0x7f29b803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.099+0000 7f3eae7e2640 1 -- start start 2023-10-16T09:49:10.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.101+0000 7f6a86b73640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8006fc50 0x7f6a800c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.101+0000 7f6a86b73640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a800700b0 0x7f6a800c0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.101+0000 7f6a86b73640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a800ce370 0x7f6a800c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.101+0000 7f6a86b73640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a800c3090 con 0x7f6a800ce370 2023-10-16T09:49:10.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.101+0000 7f6a86b73640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a800c3370 con 0x7f6a800700b0 2023-10-16T09:49:10.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.101+0000 7f6a86b73640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a800c3650 con 0x7f6a8006fc50 2023-10-16T09:49:10.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.101+0000 7f6a85110640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a800ce370 0x7f6a800c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.101+0000 7f6a85110640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a800ce370 0x7f6a800c3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59678/0 (socket says 172.21.15.84:59678) 2023-10-16T09:49:10.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.101+0000 7f6a85110640 1 -- 172.21.15.84:0/1324759032 learned_addr learned my addr 172.21.15.84:0/1324759032 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:10.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.101+0000 7f6a77fff640 1 --2- 172.21.15.84:0/1324759032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a800700b0 0x7f6a800c0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.101+0000 7f6a8490f640 1 --2- 172.21.15.84:0/1324759032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8006fc50 0x7f6a800c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.102+0000 7f6a85110640 1 -- 172.21.15.84:0/1324759032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8006fc50 msgr2=0x7f6a800c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.102+0000 7f6a85110640 1 --2- 172.21.15.84:0/1324759032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8006fc50 0x7f6a800c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.102+0000 7f6a85110640 1 -- 172.21.15.84:0/1324759032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a800700b0 msgr2=0x7f6a800c0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.102+0000 7f6a85110640 1 --2- 172.21.15.84:0/1324759032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a800700b0 0x7f6a800c0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.102+0000 7f6a85110640 1 -- 172.21.15.84:0/1324759032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c009400 con 0x7f6a800ce370 2023-10-16T09:49:10.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.102+0000 7f6a8490f640 1 --2- 172.21.15.84:0/1324759032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8006fc50 0x7f6a800c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:10.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.102+0000 7f6a77fff640 1 --2- 172.21.15.84:0/1324759032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a800700b0 0x7f6a800c0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:10.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.102+0000 7f743a8bb640 1 Processor -- start 2023-10-16T09:49:10.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.102+0000 7f3eae7e2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea000c1f0 0x7f3ea00d4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.102+0000 7f3eae7e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea0033fc0 0x7f3ea0041820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.102+0000 7f3eae7e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea00e2e10 0x7f3ea0041d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.102+0000 7f3eae7e2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea0048150 con 0x7f3ea0033fc0 2023-10-16T09:49:10.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.102+0000 7f3eae7e2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00d0490 con 0x7f3ea00e2e10 2023-10-16T09:49:10.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.102+0000 7f3eae7e2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00d0a50 con 0x7f3ea000c1f0 2023-10-16T09:49:10.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.103+0000 7f6a85110640 1 --2- 172.21.15.84:0/1324759032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a800ce370 0x7f6a800c3ca0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6a7800efc0 tx=0x7f6a7800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.103+0000 7f6a757fa640 1 -- 172.21.15.84:0/1324759032 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78009ba0 con 0x7f6a800ce370 2023-10-16T09:49:10.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.103+0000 7f6a757fa640 1 -- 172.21.15.84:0/1324759032 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7800f040 con 0x7f6a800ce370 2023-10-16T09:49:10.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.103+0000 7f6a757fa640 1 -- 172.21.15.84:0/1324759032 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a780146c0 con 0x7f6a800ce370 2023-10-16T09:49:10.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.103+0000 7f6a86b73640 1 -- 172.21.15.84:0/1324759032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a800c8690 con 0x7f6a800ce370 2023-10-16T09:49:10.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.103+0000 7f3eacfdf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea0033fc0 0x7f3ea0041820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.103+0000 7f3ead7e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea000c1f0 0x7f3ea00d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.103+0000 7f3eacfdf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea0033fc0 0x7f3ea0041820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59686/0 (socket says 172.21.15.84:59686) 2023-10-16T09:49:10.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.103+0000 7f3eacfdf640 1 -- 172.21.15.84:0/2262489329 learned_addr learned my addr 172.21.15.84:0/2262489329 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:10.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.104+0000 7f3eacfdf640 1 -- 172.21.15.84:0/2262489329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea000c1f0 msgr2=0x7f3ea00d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.104+0000 7f3eacfdf640 1 --2- 172.21.15.84:0/2262489329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea000c1f0 0x7f3ea00d4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.104+0000 7f3eacfdf640 1 -- 172.21.15.84:0/2262489329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea00e2e10 msgr2=0x7f3ea0041d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:49:10.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.103+0000 7f6a86b73640 1 -- 172.21.15.84:0/1324759032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a800c0b30 con 0x7f6a800ce370 2023-10-16T09:49:10.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.102+0000 7f743a8bb640 1 -- start start 2023-10-16T09:49:10.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.104+0000 7f3eacfdf640 1 --2- 172.21.15.84:0/2262489329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea00e2e10 0x7f3ea0041d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.104+0000 7f3eacfdf640 1 -- 172.21.15.84:0/2262489329 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea4009400 con 0x7f3ea0033fc0 2023-10-16T09:49:10.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.104+0000 7f743a8bb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f74340cfaf0 0x7f74340b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.104+0000 7f743a8bb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74340d3980 con 0x7f74340b3420 2023-10-16T09:49:10.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.104+0000 7f743a8bb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74340ce630 con 0x7f74340cf710 2023-10-16T09:49:10.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.104+0000 7f743a8bb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74340d3c60 con 0x7f74340cfaf0 2023-10-16T09:49:10.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.105+0000 7f742bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f74340b3420 0x7f74340b5cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32870/0 (socket says 172.21.15.84:32870) 2023-10-16T09:49:10.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.105+0000 7f742bfff640 1 -- 172.21.15.84:0/1695302255 learned_addr learned my addr 172.21.15.84:0/1695302255 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:10.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.104+0000 7f6a86b73640 1 -- 172.21.15.84:0/1324759032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a800c31d0 con 0x7f6a800ce370 2023-10-16T09:49:10.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.105+0000 7f3eacfdf640 1 --2- 172.21.15.84:0/2262489329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea0033fc0 0x7f3ea0041820 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800b4a0 tx=0x7f3e9800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.105+0000 7f7438e58640 1 --2- 172.21.15.84:0/1695302255 >> v2:172.21.15.157:3300/0 conn(0x7f74340cfaf0 0x7f74340b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.105+0000 7f74337fe640 1 -- 172.21.15.84:0/1695302255 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3961410262 0 0) 0x7f74340d3980 con 0x7f74340b3420 2023-10-16T09:49:10.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.105+0000 7f74337fe640 1 -- 172.21.15.84:0/1695302255 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7420003520 con 0x7f74340b3420 2023-10-16T09:49:10.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.105+0000 7f3e9dffb640 1 -- 172.21.15.84:0/2262489329 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98010610 con 0x7f3ea0033fc0 2023-10-16T09:49:10.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.105+0000 7f3e9dffb640 1 -- 172.21.15.84:0/2262489329 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e98010770 con 0x7f3ea0033fc0 2023-10-16T09:49:10.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.105+0000 7f3e9dffb640 1 -- 172.21.15.84:0/2262489329 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98010b20 con 0x7f3ea0033fc0 2023-10-16T09:49:10.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.105+0000 7f3eae7e2640 1 -- 172.21.15.84:0/2262489329 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea00d0ee0 con 0x7f3ea0033fc0 2023-10-16T09:49:10.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.106+0000 7f3eae7e2640 1 -- 172.21.15.84:0/2262489329 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea00d2a50 con 0x7f3ea0033fc0 2023-10-16T09:49:10.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.106+0000 7f7438e58640 1 -- 172.21.15.84:0/1695302255 >> v1:172.21.15.157:6789/0 conn(0x7f74340cf710 legacy=0x7f74340b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.106+0000 7f7438e58640 1 -- 172.21.15.84:0/1695302255 >> v1:172.21.15.84:6789/0 conn(0x7f74340b3420 legacy=0x7f74340b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.106+0000 7f7438e58640 1 -- 172.21.15.84:0/1695302255 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74340ce910 con 0x7f74340cfaf0 2023-10-16T09:49:10.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.107+0000 7f3eae7e2640 1 -- 172.21.15.84:0/2262489329 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea00d0be0 con 0x7f3ea0033fc0 2023-10-16T09:49:10.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.107+0000 7f7438e58640 1 --2- 172.21.15.84:0/1695302255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74340cfaf0 0x7f74340b63d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f742c009d90 tx=0x7f742c02bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be690cc5cbe9d3bd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.108+0000 7f9ea5ffb640 1 -- 172.21.15.84:0/1507136532 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9ea00546d0 con 0x7f9eb00acfa0 2023-10-16T09:49:10.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.108+0000 7f74337fe640 1 -- 172.21.15.84:0/1695302255 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c02cc80 con 0x7f74340cfaf0 2023-10-16T09:49:10.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.108+0000 7f74337fe640 1 -- 172.21.15.84:0/1695302255 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742c02cde0 con 0x7f74340cfaf0 2023-10-16T09:49:10.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.108+0000 7f74337fe640 1 -- 172.21.15.84:0/1695302255 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c034660 con 0x7f74340cfaf0 2023-10-16T09:49:10.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.108+0000 7f6a757fa640 1 -- 172.21.15.84:0/1324759032 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6a78014820 con 0x7f6a800ce370 2023-10-16T09:49:10.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.109+0000 7f743a8bb640 1 -- 172.21.15.84:0/1695302255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74340cfaf0 msgr2=0x7f74340b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.109+0000 7f743a8bb640 1 --2- 172.21.15.84:0/1695302255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74340cfaf0 0x7f74340b63d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f742c009d90 tx=0x7f742c02bed0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.109+0000 7f743a8bb640 1 -- 172.21.15.84:0/1695302255 shutdown_connections 2023-10-16T09:49:10.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.109+0000 7f743a8bb640 1 --2- 172.21.15.84:0/1695302255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74340cfaf0 0x7f74340b63d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.109+0000 7f743a8bb640 1 -- 172.21.15.84:0/1695302255 >> 172.21.15.84:0/1695302255 conn(0x7f74340d2cc0 msgr2=0x7f74340d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:10.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.109+0000 7f743a8bb640 1 -- 172.21.15.84:0/1695302255 shutdown_connections 2023-10-16T09:49:10.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.109+0000 7f743a8bb640 1 -- 172.21.15.84:0/1695302255 wait complete. 2023-10-16T09:49:10.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.110+0000 7f6a757fa640 1 --2- 172.21.15.84:0/1324759032 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6a6c06ae70 0x7f6a6c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.110+0000 7f6a757fa640 1 -- 172.21.15.84:0/1324759032 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a7808a5d0 con 0x7f6a800ce370 2023-10-16T09:49:10.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.110+0000 7f6a8490f640 1 --2- 172.21.15.84:0/1324759032 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6a6c06ae70 0x7f6a6c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.111+0000 7f743a8bb640 1 Processor -- start 2023-10-16T09:49:10.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.111+0000 7f743a8bb640 1 -- start start 2023-10-16T09:49:10.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.111+0000 7fc99b7fe640 1 -- 172.21.15.84:0/2307186348 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994004030 con 0x7fc9a40ca780 2023-10-16T09:49:10.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.111+0000 7fc99b7fe640 1 -- 172.21.15.84:0/2307186348 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc994004190 con 0x7fc9a40ca780 2023-10-16T09:49:10.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.111+0000 7fc99b7fe640 1 -- 172.21.15.84:0/2307186348 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994004520 con 0x7fc9a40ca780 2023-10-16T09:49:10.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.111+0000 7fc9a9778640 1 -- 172.21.15.84:0/2307186348 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a40c7f60 con 0x7fc9a40ca780 2023-10-16T09:49:10.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.111+0000 7fc9a9778640 1 -- 172.21.15.84:0/2307186348 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a40c8520 con 0x7fc9a40ca780 2023-10-16T09:49:10.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.112+0000 7f743a8bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74340b3420 0x7f7434150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.112+0000 7f6a8490f640 1 --2- 172.21.15.84:0/1324759032 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6a6c06ae70 0x7f6a6c06d330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c035000 tx=0x7f6a7c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.112+0000 7f743a8bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74340cf710 0x7f743409c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.112+0000 7f743a8bb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74340cfaf0 0x7f74340b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.112+0000 7f743a8bb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74340a0020 con 0x7f74340cfaf0 2023-10-16T09:49:10.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.112+0000 7f743a8bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74340cc5b0 con 0x7f74340b3420 2023-10-16T09:49:10.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.112+0000 7f743a8bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74340a0270 con 0x7f74340cf710 2023-10-16T09:49:10.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.113+0000 7f29b2ffd640 1 -- 172.21.15.84:0/4096139593 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f29ac053e30 con 0x7f29bc0cded0 2023-10-16T09:49:10.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.113+0000 7f7433fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74340b3420 0x7f7434150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.113+0000 7f7433fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74340b3420 0x7f7434150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:55142/0 (socket says 172.21.15.84:55142) 2023-10-16T09:49:10.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.113+0000 7f7433fff640 1 -- 172.21.15.84:0/2734093616 learned_addr learned my addr 172.21.15.84:0/2734093616 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:10.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.113+0000 7f7438e58640 1 --2- 172.21.15.84:0/2734093616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74340cfaf0 0x7f74340b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.113+0000 7f742bfff640 1 --2- 172.21.15.84:0/2734093616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74340cf710 0x7f743409c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.114+0000 7f7433fff640 1 -- 172.21.15.84:0/2734093616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74340cf710 msgr2=0x7f743409c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.114+0000 7f7433fff640 1 --2- 172.21.15.84:0/2734093616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74340cf710 0x7f743409c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.114+0000 7f7433fff640 1 -- 172.21.15.84:0/2734093616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74340cfaf0 msgr2=0x7f74340b5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.114+0000 7fc9997fa640 1 -- 172.21.15.84:0/2307186348 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a40c8740 con 0x7fc9a40ca780 2023-10-16T09:49:10.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.114+0000 7f7433fff640 1 --2- 172.21.15.84:0/2734093616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74340cfaf0 0x7f74340b5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.114+0000 7f7433fff640 1 -- 172.21.15.84:0/2734093616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c009a70 con 0x7f74340b3420 2023-10-16T09:49:10.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.114+0000 7f7438e58640 1 --2- 172.21.15.84:0/2734093616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74340cfaf0 0x7f74340b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:10.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.115+0000 7f7433fff640 1 --2- 172.21.15.84:0/2734093616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74340b3420 0x7f7434150030 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f742400c450 tx=0x7f742400c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.115+0000 7f74317fa640 1 -- 172.21.15.84:0/2734093616 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74240052d0 con 0x7f74340b3420 2023-10-16T09:49:10.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.115+0000 7f74317fa640 1 -- 172.21.15.84:0/2734093616 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7424005430 con 0x7f74340b3420 2023-10-16T09:49:10.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.115+0000 7f74317fa640 1 -- 172.21.15.84:0/2734093616 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424012870 con 0x7f74340b3420 2023-10-16T09:49:10.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.115+0000 7f743a8bb640 1 -- 172.21.15.84:0/2734093616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74340a0550 con 0x7f74340b3420 2023-10-16T09:49:10.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.116+0000 7fc99b7fe640 1 -- 172.21.15.84:0/2307186348 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc9940096a0 con 0x7fc9a40ca780 2023-10-16T09:49:10.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.117+0000 7f743a8bb640 1 -- 172.21.15.84:0/2734093616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7434150570 con 0x7f74340b3420 2023-10-16T09:49:10.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.117+0000 7fc99b7fe640 1 --2- 172.21.15.84:0/2307186348 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc98406af40 0x7fc98406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.118+0000 7fc99b7fe640 1 -- 172.21.15.84:0/2307186348 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc994089770 con 0x7fc9a40ca780 2023-10-16T09:49:10.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.118+0000 7f743a8bb640 1 -- 172.21.15.84:0/2734093616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74340a0a70 con 0x7f74340b3420 2023-10-16T09:49:10.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.118+0000 7f6a757fa640 1 -- 172.21.15.84:0/1324759032 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6a78055660 con 0x7f6a800ce370 2023-10-16T09:49:10.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.118+0000 7fc9a2ffd640 1 --2- 172.21.15.84:0/2307186348 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc98406af40 0x7fc98406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.119+0000 7fc9a2ffd640 1 --2- 172.21.15.84:0/2307186348 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc98406af40 0x7fc98406d400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc98c036000 tx=0x7fc98c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.119+0000 7f74317fa640 1 -- 172.21.15.84:0/2734093616 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f74240129d0 con 0x7f74340b3420 2023-10-16T09:49:10.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.121+0000 7f74317fa640 1 --2- 172.21.15.84:0/2734093616 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f741406ae70 0x7f741406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.121+0000 7f74317fa640 1 -- 172.21.15.84:0/2734093616 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7424089110 con 0x7f74340b3420 2023-10-16T09:49:10.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.121+0000 7f742bfff640 1 --2- 172.21.15.84:0/2734093616 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f741406ae70 0x7f741406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.123+0000 7f742bfff640 1 --2- 172.21.15.84:0/2734093616 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f741406ae70 0x7f741406d330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f74340b2a90 tx=0x7f741c007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.123+0000 7f3e9dffb640 1 -- 172.21.15.84:0/2262489329 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3e98010c80 con 0x7f3ea0033fc0 2023-10-16T09:49:10.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.125+0000 7f3e9dffb640 1 --2- 172.21.15.84:0/2262489329 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3e8c06ae70 0x7f3e8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:10.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.125+0000 7f3e9dffb640 1 -- 172.21.15.84:0/2262489329 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e980895d0 con 0x7f3ea0033fc0 2023-10-16T09:49:10.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.125+0000 7f3ead7e0640 1 --2- 172.21.15.84:0/2262489329 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3e8c06ae70 0x7f3e8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:10.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.126+0000 7f3e9dffb640 1 -- 172.21.15.84:0/2262489329 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3e98054710 con 0x7f3ea0033fc0 2023-10-16T09:49:10.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.127+0000 7f3ead7e0640 1 --2- 172.21.15.84:0/2262489329 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3e8c06ae70 0x7f3e8c06d330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4036000 tx=0x7f3ea403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:10.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.133+0000 7fc99b7fe640 1 -- 172.21.15.84:0/2307186348 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc994054800 con 0x7fc9a40ca780 2023-10-16T09:49:10.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.134+0000 7f74317fa640 1 -- 172.21.15.84:0/2734093616 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f74340a0a70 con 0x7f74340b3420 2023-10-16T09:49:10.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.500+0000 7f6018d35640 1 -- 172.21.15.84:0/2426694830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f600c005490 con 0x7f600c0e0ed0 2023-10-16T09:49:10.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.501+0000 7ff319c09640 1 -- 172.21.15.84:0/602933260 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff3141f47d0 con 0x7ff3140cf200 2023-10-16T09:49:10.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.501+0000 7f5ffffff640 1 -- 172.21.15.84:0/2426694830 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6014097070 con 0x7f600c0e0ed0 2023-10-16T09:49:10.503 INFO:teuthology.orchestra.run.smithi084.stdout:55834574851 2023-10-16T09:49:10.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.502+0000 7ff2fbfff640 1 -- 172.21.15.84:0/602933260 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff3141f47d0 con 0x7ff3140cf200 2023-10-16T09:49:10.503 INFO:teuthology.orchestra.run.smithi084.stdout:55834574851 2023-10-16T09:49:10.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.506+0000 7ff319c09640 1 -- 172.21.15.84:0/602933260 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff30006ae70 msgr2=0x7ff30006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.506+0000 7ff319c09640 1 --2- 172.21.15.84:0/602933260 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff30006ae70 0x7ff30006d330 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff308037000 tx=0x7ff308031000 comp rx=0 tx=0).stop 2023-10-16T09:49:10.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.506+0000 7ff319c09640 1 -- 172.21.15.84:0/602933260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3140cf200 msgr2=0x7ff314157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.506+0000 7ff319c09640 1 --2- 172.21.15.84:0/602933260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3140cf200 0x7ff314157a50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00c790 tx=0x7ff30c00cc50 comp rx=0 tx=0).stop 2023-10-16T09:49:10.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.507+0000 7ff319c09640 1 -- 172.21.15.84:0/602933260 shutdown_connections 2023-10-16T09:49:10.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.507+0000 7ff319c09640 1 --2- 172.21.15.84:0/602933260 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff30006ae70 0x7ff30006d330 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.507+0000 7ff319c09640 1 --2- 172.21.15.84:0/602933260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff314158eb0 0x7ff3141f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.507+0000 7ff319c09640 1 --2- 172.21.15.84:0/602933260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3140cf200 0x7ff314157a50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.507+0000 7ff319c09640 1 --2- 172.21.15.84:0/602933260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff31406a670 0x7ff31415b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.507+0000 7ff319c09640 1 -- 172.21.15.84:0/602933260 >> 172.21.15.84:0/602933260 conn(0x7ff3140ce6e0 msgr2=0x7ff3140ceac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:10.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.507+0000 7ff319c09640 1 -- 172.21.15.84:0/602933260 shutdown_connections 2023-10-16T09:49:10.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.507+0000 7ff319c09640 1 -- 172.21.15.84:0/602933260 wait complete. 2023-10-16T09:49:10.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.508+0000 7f5ffdffb640 1 -- 172.21.15.84:0/2426694830 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f5ff806ae70 msgr2=0x7f5ff806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.508+0000 7f5ffdffb640 1 --2- 172.21.15.84:0/2426694830 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f5ff806ae70 0x7f5ff806d330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6008034000 tx=0x7f6008008030 comp rx=0 tx=0).stop 2023-10-16T09:49:10.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.508+0000 7f5ffdffb640 1 -- 172.21.15.84:0/2426694830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f600c0e0ed0 msgr2=0x7f600c033560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.508+0000 7f5ffdffb640 1 --2- 172.21.15.84:0/2426694830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f600c0e0ed0 0x7f600c033560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f60140687d0 tx=0x7f6014068000 comp rx=0 tx=0).stop 2023-10-16T09:49:10.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.508+0000 7f5ffdffb640 1 -- 172.21.15.84:0/2426694830 shutdown_connections 2023-10-16T09:49:10.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.508+0000 7f5ffdffb640 1 --2- 172.21.15.84:0/2426694830 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f5ff806ae70 0x7f5ff806d330 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.508+0000 7f5ffdffb640 1 --2- 172.21.15.84:0/2426694830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f600c0e0ed0 0x7f600c033560 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.508+0000 7f5ffdffb640 1 --2- 172.21.15.84:0/2426694830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f600c04eac0 0x7f600c0d29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.509+0000 7f5ffdffb640 1 --2- 172.21.15.84:0/2426694830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f600c039fa0 0x7f600c0527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.509+0000 7f5ffdffb640 1 -- 172.21.15.84:0/2426694830 >> 172.21.15.84:0/2426694830 conn(0x7f600c054c10 msgr2=0x7f600c0071f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:10.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.512+0000 7f5ffdffb640 1 -- 172.21.15.84:0/2426694830 shutdown_connections 2023-10-16T09:49:10.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.512+0000 7f5ffdffb640 1 -- 172.21.15.84:0/2426694830 wait complete. 2023-10-16T09:49:10.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.521+0000 7f29c316a640 1 -- 172.21.15.84:0/4096139593 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f29a40057d0 con 0x7f29bc0cded0 2023-10-16T09:49:10.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.522+0000 7f29b2ffd640 1 -- 172.21.15.84:0/4096139593 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f29ac017090 con 0x7f29bc0cded0 2023-10-16T09:49:10.523 INFO:teuthology.orchestra.run.smithi084.stdout:55834574853 2023-10-16T09:49:10.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.525+0000 7f29c316a640 1 -- 172.21.15.84:0/4096139593 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f29a006ae70 msgr2=0x7f29a006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.525+0000 7f29c316a640 1 --2- 172.21.15.84:0/4096139593 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f29a006ae70 0x7f29a006d330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f29b8036000 tx=0x7f29b803c040 comp rx=0 tx=0).stop 2023-10-16T09:49:10.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.525+0000 7f29c316a640 1 -- 172.21.15.84:0/4096139593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29bc0cded0 msgr2=0x7f29bc15da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.525+0000 7f29c316a640 1 --2- 172.21.15.84:0/4096139593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29bc0cded0 0x7f29bc15da50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00c440 tx=0x7f29ac001e70 comp rx=0 tx=0).stop 2023-10-16T09:49:10.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.526+0000 7f29c316a640 1 -- 172.21.15.84:0/4096139593 shutdown_connections 2023-10-16T09:49:10.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.526+0000 7f29c316a640 1 --2- 172.21.15.84:0/4096139593 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f29a006ae70 0x7f29a006d330 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f29b8036000 tx=0x7f29b803c040 comp rx=0 tx=0).stop 2023-10-16T09:49:10.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.526+0000 7f29c316a640 1 --2- 172.21.15.84:0/4096139593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29bc15ffe0 0x7f29bc15df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.526+0000 7f29c316a640 1 --2- 172.21.15.84:0/4096139593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29bc0cded0 0x7f29bc15da50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.526+0000 7f29c316a640 1 --2- 172.21.15.84:0/4096139593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29bc0b8480 0x7f29bc0b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.526+0000 7f29c316a640 1 -- 172.21.15.84:0/4096139593 >> 172.21.15.84:0/4096139593 conn(0x7f29bc0cf570 msgr2=0x7f29bc0b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:10.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.527+0000 7f29c316a640 1 -- 172.21.15.84:0/4096139593 shutdown_connections 2023-10-16T09:49:10.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.527+0000 7f29c316a640 1 -- 172.21.15.84:0/4096139593 wait complete. 2023-10-16T09:49:10.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.529+0000 7f9eb737c640 1 -- 172.21.15.84:0/1507136532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9eb004e9b0 con 0x7f9eb00acfa0 2023-10-16T09:49:10.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.530+0000 7f9ea5ffb640 1 -- 172.21.15.84:0/1507136532 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9ea005e070 con 0x7f9eb00acfa0 2023-10-16T09:49:10.531 INFO:teuthology.orchestra.run.smithi084.stdout:55834574852 2023-10-16T09:49:10.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.533+0000 7f9eb737c640 1 -- 172.21.15.84:0/1507136532 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9e9406ae70 msgr2=0x7f9e9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.533+0000 7f9eb737c640 1 --2- 172.21.15.84:0/1507136532 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9e9406ae70 0x7f9e9406d330 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00c51f0 tx=0x7f9eac009190 comp rx=0 tx=0).stop 2023-10-16T09:49:10.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.533+0000 7f9eb737c640 1 -- 172.21.15.84:0/1507136532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb00acfa0 msgr2=0x7f9eb00c6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.533+0000 7f9eb737c640 1 --2- 172.21.15.84:0/1507136532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb00acfa0 0x7f9eb00c6810 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000e7f0 tx=0x7f9ea000ecb0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.534+0000 7f9eb737c640 1 -- 172.21.15.84:0/1507136532 shutdown_connections 2023-10-16T09:49:10.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.534+0000 7f9eb737c640 1 --2- 172.21.15.84:0/1507136532 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9e9406ae70 0x7f9e9406d330 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.534+0000 7f9eb737c640 1 --2- 172.21.15.84:0/1507136532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb00acfa0 0x7f9eb00c6810 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.534+0000 7f9eb737c640 1 --2- 172.21.15.84:0/1507136532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9eb008a9f0 0x7f9eb00c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.534+0000 7f9eb737c640 1 --2- 172.21.15.84:0/1507136532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb0070380 0x7f9eb016ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.534+0000 7f9eb737c640 1 -- 172.21.15.84:0/1507136532 >> 172.21.15.84:0/1507136532 conn(0x7f9eb00ce6e0 msgr2=0x7f9eb0170cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:10.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.534+0000 7f9eb737c640 1 -- 172.21.15.84:0/1507136532 shutdown_connections 2023-10-16T09:49:10.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.534+0000 7f9eb737c640 1 -- 172.21.15.84:0/1507136532 wait complete. 2023-10-16T09:49:10.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.2 2023-10-16T09:49:10.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.536+0000 7f3eae7e2640 1 -- 172.21.15.84:0/2262489329 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3ea0005490 con 0x7f3ea0033fc0 2023-10-16T09:49:10.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.537+0000 7f3e9dffb640 1 -- 172.21.15.84:0/2262489329 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3e98014090 con 0x7f3ea0033fc0 2023-10-16T09:49:10.540 INFO:teuthology.orchestra.run.smithi084.stdout:55834574852 2023-10-16T09:49:10.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.540+0000 7f3eae7e2640 1 -- 172.21.15.84:0/2262489329 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3e8c06ae70 msgr2=0x7f3e8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.540+0000 7f3eae7e2640 1 --2- 172.21.15.84:0/2262489329 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3e8c06ae70 0x7f3e8c06d330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4036000 tx=0x7f3ea403d040 comp rx=0 tx=0).stop 2023-10-16T09:49:10.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.540+0000 7f3eae7e2640 1 -- 172.21.15.84:0/2262489329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea0033fc0 msgr2=0x7f3ea0041820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.540+0000 7f3eae7e2640 1 --2- 172.21.15.84:0/2262489329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea0033fc0 0x7f3ea0041820 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800b4a0 tx=0x7f3e9800b960 comp rx=0 tx=0).stop 2023-10-16T09:49:10.542 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.1 2023-10-16T09:49:10.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.541+0000 7f3eae7e2640 1 -- 172.21.15.84:0/2262489329 shutdown_connections 2023-10-16T09:49:10.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.541+0000 7f3eae7e2640 1 --2- 172.21.15.84:0/2262489329 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3e8c06ae70 0x7f3e8c06d330 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.541+0000 7f3eae7e2640 1 --2- 172.21.15.84:0/2262489329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea00e2e10 0x7f3ea0041d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.541+0000 7f3eae7e2640 1 --2- 172.21.15.84:0/2262489329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea0033fc0 0x7f3ea0041820 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.541+0000 7f3eae7e2640 1 --2- 172.21.15.84:0/2262489329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea000c1f0 0x7f3ea00d4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.541+0000 7f3eae7e2640 1 -- 172.21.15.84:0/2262489329 >> 172.21.15.84:0/2262489329 conn(0x7f3ea0054c10 msgr2=0x7f3ea0035c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:10.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.542+0000 7f3eae7e2640 1 -- 172.21.15.84:0/2262489329 shutdown_connections 2023-10-16T09:49:10.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.542+0000 7f3eae7e2640 1 -- 172.21.15.84:0/2262489329 wait complete. 2023-10-16T09:49:10.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.550+0000 7f6a86b73640 1 -- 172.21.15.84:0/1324759032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f6a800c0b30 con 0x7f6a800ce370 2023-10-16T09:49:10.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.550+0000 7fc9997fa640 1 -- 172.21.15.84:0/2307186348 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fc9a404e160 con 0x7fc9a40ca780 2023-10-16T09:49:10.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.553+0000 7f6a757fa640 1 -- 172.21.15.84:0/1324759032 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6a800c0b30 con 0x7f6a800ce370 2023-10-16T09:49:10.554 INFO:teuthology.orchestra.run.smithi084.stdout:55834574852 2023-10-16T09:49:10.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.554+0000 7f743a8bb640 1 -- 172.21.15.84:0/2734093616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f74340a0a70 con 0x7f74340b3420 2023-10-16T09:49:10.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.554+0000 7fc99b7fe640 1 -- 172.21.15.84:0/2307186348 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc994057e20 con 0x7fc9a40ca780 2023-10-16T09:49:10.556 INFO:teuthology.orchestra.run.smithi084.stdout:55834574852 2023-10-16T09:49:10.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.555+0000 7f74317fa640 1 -- 172.21.15.84:0/2734093616 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f74340a0a70 con 0x7f74340b3420 2023-10-16T09:49:10.557 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2023-10-16T09:49:10.557 INFO:teuthology.orchestra.run.smithi084.stdout:55834574852 2023-10-16T09:49:10.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.556+0000 7f6a86b73640 1 -- 172.21.15.84:0/1324759032 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6a6c06ae70 msgr2=0x7f6a6c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.556+0000 7f6a86b73640 1 --2- 172.21.15.84:0/1324759032 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6a6c06ae70 0x7f6a6c06d330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c035000 tx=0x7f6a7c030000 comp rx=0 tx=0).stop 2023-10-16T09:49:10.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.556+0000 7f6a86b73640 1 -- 172.21.15.84:0/1324759032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a800ce370 msgr2=0x7f6a800c3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.556+0000 7f6a86b73640 1 --2- 172.21.15.84:0/1324759032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a800ce370 0x7f6a800c3ca0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6a7800efc0 tx=0x7f6a7800c490 comp rx=0 tx=0).stop 2023-10-16T09:49:10.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.556+0000 7f6a86b73640 1 -- 172.21.15.84:0/1324759032 shutdown_connections 2023-10-16T09:49:10.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.556+0000 7f6a86b73640 1 --2- 172.21.15.84:0/1324759032 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6a6c06ae70 0x7f6a6c06d330 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.556+0000 7f6a86b73640 1 --2- 172.21.15.84:0/1324759032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a800ce370 0x7f6a800c3ca0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.556+0000 7f6a86b73640 1 --2- 172.21.15.84:0/1324759032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a800700b0 0x7f6a800c0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.556+0000 7f6a86b73640 1 --2- 172.21.15.84:0/1324759032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8006fc50 0x7f6a800c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.556+0000 7f6a86b73640 1 -- 172.21.15.84:0/1324759032 >> 172.21.15.84:0/1324759032 conn(0x7f6a800ce9b0 msgr2=0x7f6a8006b760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:10.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.558+0000 7fc9997fa640 1 -- 172.21.15.84:0/2307186348 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc98406af40 msgr2=0x7fc98406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.558+0000 7fc9997fa640 1 --2- 172.21.15.84:0/2307186348 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc98406af40 0x7fc98406d400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc98c036000 tx=0x7fc98c030000 comp rx=0 tx=0).stop 2023-10-16T09:49:10.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.558+0000 7fc9997fa640 1 -- 172.21.15.84:0/2307186348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9a40ca780 msgr2=0x7fc9a40c7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.558+0000 7fc9997fa640 1 --2- 172.21.15.84:0/2307186348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9a40ca780 0x7fc9a40c7200 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc99400d7d0 tx=0x7fc99400dc90 comp rx=0 tx=0).stop 2023-10-16T09:49:10.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.558+0000 7fc9997fa640 1 -- 172.21.15.84:0/2307186348 shutdown_connections 2023-10-16T09:49:10.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.558+0000 7fc9997fa640 1 --2- 172.21.15.84:0/2307186348 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc98406af40 0x7fc98406d400 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.558+0000 7fc9997fa640 1 --2- 172.21.15.84:0/2307186348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a4159e40 0x7fc9a4201eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.558+0000 7fc9997fa640 1 --2- 172.21.15.84:0/2307186348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9a40ca780 0x7fc9a40c7200 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.558+0000 7fc9997fa640 1 --2- 172.21.15.84:0/2307186348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9a4070b00 0x7fc9a415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.558+0000 7fc9997fa640 1 -- 172.21.15.84:0/2307186348 >> 172.21.15.84:0/2307186348 conn(0x7fc9a40cfac0 msgr2=0x7fc9a415e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:10.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.558+0000 7fc9997fa640 1 -- 172.21.15.84:0/2307186348 shutdown_connections 2023-10-16T09:49:10.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.558+0000 7fc9997fa640 1 -- 172.21.15.84:0/2307186348 wait complete. 2023-10-16T09:49:10.563 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2023-10-16T09:49:10.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.559+0000 7f743a8bb640 1 -- 172.21.15.84:0/2734093616 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f741406ae70 msgr2=0x7f741406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.559+0000 7f743a8bb640 1 --2- 172.21.15.84:0/2734093616 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f741406ae70 0x7f741406d330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f74340b2a90 tx=0x7f741c007d30 comp rx=0 tx=0).stop 2023-10-16T09:49:10.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.559+0000 7f743a8bb640 1 -- 172.21.15.84:0/2734093616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74340b3420 msgr2=0x7f7434150030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:10.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.559+0000 7f743a8bb640 1 --2- 172.21.15.84:0/2734093616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74340b3420 0x7f7434150030 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f742400c450 tx=0x7f742400c910 comp rx=0 tx=0).stop 2023-10-16T09:49:10.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.559+0000 7f743a8bb640 1 -- 172.21.15.84:0/2734093616 shutdown_connections 2023-10-16T09:49:10.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.559+0000 7f743a8bb640 1 --2- 172.21.15.84:0/2734093616 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f741406ae70 0x7f741406d330 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.559+0000 7f743a8bb640 1 --2- 172.21.15.84:0/2734093616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74340cfaf0 0x7f74340b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.559+0000 7f743a8bb640 1 --2- 172.21.15.84:0/2734093616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74340cf710 0x7f743409c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.559+0000 7f743a8bb640 1 --2- 172.21.15.84:0/2734093616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74340b3420 0x7f7434150030 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:10.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.559+0000 7f743a8bb640 1 -- 172.21.15.84:0/2734093616 >> 172.21.15.84:0/2734093616 conn(0x7f74340d2cc0 msgr2=0x7f743406bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:10.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.559+0000 7f743a8bb640 1 -- 172.21.15.84:0/2734093616 shutdown_connections 2023-10-16T09:49:10.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.559+0000 7f743a8bb640 1 -- 172.21.15.84:0/2734093616 wait complete. 2023-10-16T09:49:10.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.559+0000 7f6a86b73640 1 -- 172.21.15.84:0/1324759032 shutdown_connections 2023-10-16T09:49:10.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:10.559+0000 7f6a86b73640 1 -- 172.21.15.84:0/1324759032 wait complete. 2023-10-16T09:49:10.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2023-10-16T09:49:10.581 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2023-10-16T09:49:10.581 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2023-10-16T09:49:10.582 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2023-10-16T09:49:11.538 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-16T09:49:11.543 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-16T09:49:11.557 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-16T09:49:11.564 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-16T09:49:11.569 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-16T09:49:11.582 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-16T09:49:11.582 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-16T09:49:11.583 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-16T09:49:11.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.824+0000 7f7d3a590640 1 Processor -- start 2023-10-16T09:49:11.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.824+0000 7f7d3a590640 1 -- start start 2023-10-16T09:49:11.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.826+0000 7f7d3a590640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7d3415cf10 0x7f7d340d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.826+0000 7f7d3a590640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d340d10c0 0x7f7d340b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.826+0000 7f7d3a590640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d340b4ac0 0x7f7d3406b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.826+0000 7f7d3a590640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d340b4f50 con 0x7f7d3415cf10 2023-10-16T09:49:11.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.826+0000 7f7d3a590640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d340d2250 con 0x7f7d340d10c0 2023-10-16T09:49:11.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.826+0000 7f7d3a590640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d340d1640 con 0x7f7d340b4ac0 2023-10-16T09:49:11.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.826+0000 7f7d33fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7d3415cf10 0x7f7d340d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.826+0000 7f7d33fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7d3415cf10 0x7f7d340d0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59704/0 (socket says 172.21.15.84:59704) 2023-10-16T09:49:11.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.826+0000 7f7d33fff640 1 -- 172.21.15.84:0/3166130550 learned_addr learned my addr 172.21.15.84:0/3166130550 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:11.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.827+0000 7f7d33fff640 1 -- 172.21.15.84:0/3166130550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d340b4ac0 msgr2=0x7f7d3406b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:11.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.827+0000 7f7d33fff640 1 --2- 172.21.15.84:0/3166130550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d340b4ac0 0x7f7d3406b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.827+0000 7f7d33fff640 1 -- 172.21.15.84:0/3166130550 >> v2:172.21.15.157:3300/0 conn(0x7f7d340d10c0 msgr2=0x7f7d340b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:11.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.827+0000 7f7d33fff640 1 --2- 172.21.15.84:0/3166130550 >> v2:172.21.15.157:3300/0 conn(0x7f7d340d10c0 0x7f7d340b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.827+0000 7f7d33fff640 1 -- 172.21.15.84:0/3166130550 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d340d1920 con 0x7f7d3415cf10 2023-10-16T09:49:11.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.830+0000 7f7d33fff640 1 --2- 172.21.15.84:0/3166130550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d3415cf10 0x7f7d340d0b80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7d340cd0b0 tx=0x7f7d2802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68700919850c613e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.830+0000 7f7d32ffd640 1 -- 172.21.15.84:0/3166130550 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28030610 con 0x7f7d3415cf10 2023-10-16T09:49:11.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.831+0000 7f7d32ffd640 1 -- 172.21.15.84:0/3166130550 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d28030770 con 0x7f7d3415cf10 2023-10-16T09:49:11.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.831+0000 7f7d32ffd640 1 -- 172.21.15.84:0/3166130550 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28030b40 con 0x7f7d3415cf10 2023-10-16T09:49:11.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.832+0000 7f7d3a590640 1 -- 172.21.15.84:0/3166130550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d3415cf10 msgr2=0x7f7d340d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.832+0000 7f7d3a590640 1 --2- 172.21.15.84:0/3166130550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d3415cf10 0x7f7d340d0b80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7d340cd0b0 tx=0x7f7d2802b960 comp rx=0 tx=0).stop 2023-10-16T09:49:11.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.833+0000 7f7d3a590640 1 -- 172.21.15.84:0/3166130550 shutdown_connections 2023-10-16T09:49:11.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.833+0000 7f7d3a590640 1 --2- 172.21.15.84:0/3166130550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d340b4ac0 0x7f7d3406b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.833+0000 7f7d3a590640 1 --2- 172.21.15.84:0/3166130550 >> v2:172.21.15.157:3300/0 conn(0x7f7d340d10c0 0x7f7d340b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.833+0000 7f7d3a590640 1 --2- 172.21.15.84:0/3166130550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d3415cf10 0x7f7d340d0b80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.833+0000 7f7d3a590640 1 -- 172.21.15.84:0/3166130550 >> 172.21.15.84:0/3166130550 conn(0x7f7d340cf580 msgr2=0x7f7d340cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:11.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.834+0000 7f7d3a590640 1 -- 172.21.15.84:0/3166130550 shutdown_connections 2023-10-16T09:49:11.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.834+0000 7f7d3a590640 1 -- 172.21.15.84:0/3166130550 wait complete. 2023-10-16T09:49:11.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.837+0000 7f7d3a590640 1 Processor -- start 2023-10-16T09:49:11.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.838+0000 7f7d3a590640 1 -- start start 2023-10-16T09:49:11.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.840+0000 7f7d3a590640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d340b4ac0 0x7f7d34148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.840+0000 7f7d3a590640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d340d10c0 0x7f7d340cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.840+0000 7f7d3a590640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3415cf10 0x7f7d34154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.841+0000 7f7d3a590640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d341494b0 con 0x7f7d340d10c0 2023-10-16T09:49:11.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.841+0000 7f7d3a590640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d34152860 con 0x7f7d3415cf10 2023-10-16T09:49:11.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.841+0000 7f7d3a590640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d340c8910 con 0x7f7d340b4ac0 2023-10-16T09:49:11.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.841+0000 7f7d33fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d340b4ac0 0x7f7d34148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.841+0000 7f7d38b2d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3415cf10 0x7f7d34154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.841+0000 7f7d38b2d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3415cf10 0x7f7d34154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:55156/0 (socket says 172.21.15.84:55156) 2023-10-16T09:49:11.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.841+0000 7f7d38b2d640 1 -- 172.21.15.84:0/3792055076 learned_addr learned my addr 172.21.15.84:0/3792055076 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:11.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.842+0000 7f7d38b2d640 1 -- 172.21.15.84:0/3792055076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d340b4ac0 msgr2=0x7f7d34148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.842+0000 7f7d38b2d640 1 --2- 172.21.15.84:0/3792055076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d340b4ac0 0x7f7d34148f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.842+0000 7f7d38b2d640 1 -- 172.21.15.84:0/3792055076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d340d10c0 msgr2=0x7f7d340cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.842+0000 7f7d38b2d640 1 --2- 172.21.15.84:0/3792055076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d340d10c0 0x7f7d340cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.842+0000 7f7d38b2d640 1 -- 172.21.15.84:0/3792055076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d280094d0 con 0x7f7d3415cf10 2023-10-16T09:49:11.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.844+0000 7f7d38b2d640 1 --2- 172.21.15.84:0/3792055076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3415cf10 0x7f7d34154580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400b970 tx=0x7f7d2400be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.844+0000 7f7d30ff9640 1 -- 172.21.15.84:0/3792055076 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2400cb40 con 0x7f7d3415cf10 2023-10-16T09:49:11.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.844+0000 7f7d30ff9640 1 -- 172.21.15.84:0/3792055076 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2400ccc0 con 0x7f7d3415cf10 2023-10-16T09:49:11.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.844+0000 7f7d30ff9640 1 -- 172.21.15.84:0/3792055076 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24014680 con 0x7f7d3415cf10 2023-10-16T09:49:11.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.845+0000 7f7d3a590640 1 -- 172.21.15.84:0/3792055076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d340c8bf0 con 0x7f7d3415cf10 2023-10-16T09:49:11.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.845+0000 7f7d3a590640 1 -- 172.21.15.84:0/3792055076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d340c9c70 con 0x7f7d3415cf10 2023-10-16T09:49:11.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.846+0000 7f7d3a590640 1 -- 172.21.15.84:0/3792055076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d340d26c0 con 0x7f7d3415cf10 2023-10-16T09:49:11.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.850+0000 7f7d30ff9640 1 -- 172.21.15.84:0/3792055076 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7d24007370 con 0x7f7d3415cf10 2023-10-16T09:49:11.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.851+0000 7f7d30ff9640 1 --2- 172.21.15.84:0/3792055076 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7d1806ae40 0x7f7d1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.851+0000 7f7d30ff9640 1 -- 172.21.15.84:0/3792055076 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d24089850 con 0x7f7d3415cf10 2023-10-16T09:49:11.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.853+0000 7f7d33fff640 1 --2- 172.21.15.84:0/3792055076 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7d1806ae40 0x7f7d1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.854+0000 7f7d33fff640 1 --2- 172.21.15.84:0/3792055076 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7d1806ae40 0x7f7d1806d300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7d28036000 tx=0x7f7d28008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.864+0000 7f7d30ff9640 1 -- 172.21.15.84:0/3792055076 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7d24054900 con 0x7f7d3415cf10 2023-10-16T09:49:11.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.880+0000 7f3903683640 1 Processor -- start 2023-10-16T09:49:11.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.880+0000 7f3903683640 1 -- start start 2023-10-16T09:49:11.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.882+0000 7f3903683640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f38fc0661c0 0x7f38fc06faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.882+0000 7f3903683640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38fc15eff0 con 0x7f38fc0cb1c0 2023-10-16T09:49:11.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.882+0000 7f3903683640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38fc15ecc0 con 0x7f38fc15af80 2023-10-16T09:49:11.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.882+0000 7f3903683640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc0d0930 con 0x7f38fc0661c0 2023-10-16T09:49:11.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.883+0000 7f3900c1e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f38fc0cb1c0 0x7f38fc065ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32874/0 (socket says 172.21.15.84:32874) 2023-10-16T09:49:11.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.883+0000 7f3900c1e640 1 -- 172.21.15.84:0/3594803399 learned_addr learned my addr 172.21.15.84:0/3594803399 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:11.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.884+0000 7f3901c20640 1 --2- 172.21.15.84:0/3594803399 >> v2:172.21.15.157:3300/0 conn(0x7f38fc0661c0 0x7f38fc06faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.884+0000 7f38f3fff640 1 -- 172.21.15.84:0/3594803399 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83058196 0 0) 0x7f38fc15eff0 con 0x7f38fc0cb1c0 2023-10-16T09:49:11.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.884+0000 7f38f3fff640 1 -- 172.21.15.84:0/3594803399 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38e8003520 con 0x7f38fc0cb1c0 2023-10-16T09:49:11.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.884+0000 7f38f3fff640 1 -- 172.21.15.84:0/3594803399 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 196386565 0 0) 0x7f38fc15ecc0 con 0x7f38fc15af80 2023-10-16T09:49:11.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.885+0000 7f38f3fff640 1 -- 172.21.15.84:0/3594803399 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38fc15eff0 con 0x7f38fc15af80 2023-10-16T09:49:11.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.885+0000 7f3901c20640 1 -- 172.21.15.84:0/3594803399 >> v1:172.21.15.157:6789/0 conn(0x7f38fc15af80 legacy=0x7f38fc15f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.885+0000 7f3901c20640 1 -- 172.21.15.84:0/3594803399 >> v1:172.21.15.84:6789/0 conn(0x7f38fc0cb1c0 legacy=0x7f38fc065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.885+0000 7f3901c20640 1 -- 172.21.15.84:0/3594803399 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc15b360 con 0x7f38fc0661c0 2023-10-16T09:49:11.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.887+0000 7f3901c20640 1 --2- 172.21.15.84:0/3594803399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38fc0661c0 0x7f38fc06faf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009d90 tx=0x7f38f402be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebb863dafe69b23b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.888+0000 7f38f3fff640 1 -- 172.21.15.84:0/3594803399 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f402cbf0 con 0x7f38fc0661c0 2023-10-16T09:49:11.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.888+0000 7f38f3fff640 1 -- 172.21.15.84:0/3594803399 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f402cd50 con 0x7f38fc0661c0 2023-10-16T09:49:11.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.888+0000 7f38f3fff640 1 -- 172.21.15.84:0/3594803399 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4034660 con 0x7f38fc0661c0 2023-10-16T09:49:11.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.889+0000 7ffb26c07640 1 Processor -- start 2023-10-16T09:49:11.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.889+0000 7f3903683640 1 -- 172.21.15.84:0/3594803399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38fc0661c0 msgr2=0x7f38fc06faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.889+0000 7ffb26c07640 1 -- start start 2023-10-16T09:49:11.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.889+0000 7f3903683640 1 --2- 172.21.15.84:0/3594803399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38fc0661c0 0x7f38fc06faf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009d90 tx=0x7f38f402be40 comp rx=0 tx=0).stop 2023-10-16T09:49:11.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.890+0000 7f3903683640 1 -- 172.21.15.84:0/3594803399 shutdown_connections 2023-10-16T09:49:11.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.890+0000 7f3903683640 1 --2- 172.21.15.84:0/3594803399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38fc0661c0 0x7f38fc06faf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.890+0000 7f3903683640 1 -- 172.21.15.84:0/3594803399 >> 172.21.15.84:0/3594803399 conn(0x7f38fc0d0ab0 msgr2=0x7f38fc0d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:11.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.890+0000 7f3903683640 1 -- 172.21.15.84:0/3594803399 shutdown_connections 2023-10-16T09:49:11.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.891+0000 7ffb26c07640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffb200b3420 0x7ffb200b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.891+0000 7ffb26c07640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ffb200cfaf0 0x7ffb200b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.891+0000 7f3903683640 1 -- 172.21.15.84:0/3594803399 wait complete. 2023-10-16T09:49:11.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.893+0000 7f3903683640 1 Processor -- start 2023-10-16T09:49:11.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.891+0000 7ffb26c07640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb200d3980 con 0x7ffb200cf710 2023-10-16T09:49:11.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.891+0000 7ffb26c07640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb200d3c60 con 0x7ffb200cfaf0 2023-10-16T09:49:11.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.891+0000 7ffb26c07640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb200ce630 con 0x7ffb200b3420 2023-10-16T09:49:11.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.892+0000 7ffb249a3640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ffb200cf710 0x7ffb200b2d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32882/0 (socket says 172.21.15.84:32882) 2023-10-16T09:49:11.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.892+0000 7ffb249a3640 1 -- 172.21.15.84:0/1367138432 learned_addr learned my addr 172.21.15.84:0/1367138432 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:11.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.892+0000 7ffb1ffff640 1 --2- 172.21.15.84:0/1367138432 >> v2:172.21.15.157:3300/0 conn(0x7ffb200b3420 0x7ffb200b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.892+0000 7ffb251a4640 1 --2- 172.21.15.84:0/1367138432 >> v2:172.21.15.84:3300/0 conn(0x7ffb200cfaf0 0x7ffb200b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.893+0000 7ffb1f7fe640 1 -- 172.21.15.84:0/1367138432 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 546309043 0 0) 0x7ffb200d3980 con 0x7ffb200cf710 2023-10-16T09:49:11.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.893+0000 7ffb1f7fe640 1 -- 172.21.15.84:0/1367138432 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb08003520 con 0x7ffb200cf710 2023-10-16T09:49:11.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.893+0000 7ffb251a4640 1 -- 172.21.15.84:0/1367138432 >> v2:172.21.15.157:3300/0 conn(0x7ffb200b3420 msgr2=0x7ffb200b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:11.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.893+0000 7ffb251a4640 1 --2- 172.21.15.84:0/1367138432 >> v2:172.21.15.157:3300/0 conn(0x7ffb200b3420 0x7ffb200b5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.893+0000 7ffb251a4640 1 -- 172.21.15.84:0/1367138432 >> v1:172.21.15.84:6789/0 conn(0x7ffb200cf710 legacy=0x7ffb200b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.893+0000 7ffb251a4640 1 -- 172.21.15.84:0/1367138432 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb200ce910 con 0x7ffb200cfaf0 2023-10-16T09:49:11.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.894+0000 7f3903683640 1 -- start start 2023-10-16T09:49:11.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.895+0000 7ffb251a4640 1 --2- 172.21.15.84:0/1367138432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb200cfaf0 0x7ffb200b6200 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffb2008d210 tx=0x7ffb14034070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb191241899a2c40 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.895+0000 7ffb1f7fe640 1 -- 172.21.15.84:0/1367138432 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14034e20 con 0x7ffb200cfaf0 2023-10-16T09:49:11.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.895+0000 7fdb9f59e640 1 Processor -- start 2023-10-16T09:49:11.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.895+0000 7ffb1f7fe640 1 -- 172.21.15.84:0/1367138432 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1403c440 con 0x7ffb200cfaf0 2023-10-16T09:49:11.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.896+0000 7ffb1f7fe640 1 -- 172.21.15.84:0/1367138432 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14034e20 con 0x7ffb200cfaf0 2023-10-16T09:49:11.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.896+0000 7f3903683640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38fc0661c0 0x7f38fc1ff270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.897+0000 7ffb26c07640 1 -- 172.21.15.84:0/1367138432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb200cfaf0 msgr2=0x7ffb200b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.897+0000 7ffb26c07640 1 --2- 172.21.15.84:0/1367138432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb200cfaf0 0x7ffb200b6200 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffb2008d210 tx=0x7ffb14034070 comp rx=0 tx=0).stop 2023-10-16T09:49:11.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.897+0000 7f3903683640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38fc0cb1c0 0x7f38fc0b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.897+0000 7ffb26c07640 1 -- 172.21.15.84:0/1367138432 shutdown_connections 2023-10-16T09:49:11.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.897+0000 7f3903683640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38fc15af80 0x7f38fc0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.897+0000 7ffb26c07640 1 --2- 172.21.15.84:0/1367138432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb200cfaf0 0x7ffb200b6200 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.897+0000 7ffb26c07640 1 --2- 172.21.15.84:0/1367138432 >> v2:172.21.15.157:3300/0 conn(0x7ffb200b3420 0x7ffb200b5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.897+0000 7ffb26c07640 1 -- 172.21.15.84:0/1367138432 >> 172.21.15.84:0/1367138432 conn(0x7ffb200d2cc0 msgr2=0x7ffb200d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:11.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.896+0000 7fdb9f59e640 1 -- start start 2023-10-16T09:49:11.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.898+0000 7f390141f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38fc0661c0 0x7f38fc1ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.898+0000 7f3901c20640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38fc15af80 0x7f38fc0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.898+0000 7ffb26c07640 1 -- 172.21.15.84:0/1367138432 shutdown_connections 2023-10-16T09:49:11.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.898+0000 7f3901c20640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38fc15af80 0x7f38fc0cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59734/0 (socket says 172.21.15.84:59734) 2023-10-16T09:49:11.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.898+0000 7ffb26c07640 1 -- 172.21.15.84:0/1367138432 wait complete. 2023-10-16T09:49:11.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.898+0000 7f2eb47ee640 1 Processor -- start 2023-10-16T09:49:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.898+0000 7f390141f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38fc0661c0 0x7f38fc1ff270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:55182/0 (socket says 172.21.15.84:55182) 2023-10-16T09:49:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.898+0000 7f3901c20640 1 -- 172.21.15.84:0/991990821 learned_addr learned my addr 172.21.15.84:0/991990821 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.898+0000 7f3903683640 1 -- 172.21.15.84:0/991990821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0b0780 con 0x7f38fc15af80 2023-10-16T09:49:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.899+0000 7fdb9f59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb980eb2c0 0x7fdb980eb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.899+0000 7fdb9e59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdb980e78b0 0x7fdb9804f260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32886/0 (socket says 172.21.15.84:32886) 2023-10-16T09:49:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.898+0000 7f3903683640 1 -- 172.21.15.84:0/991990821 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0afb00 con 0x7f38fc0661c0 2023-10-16T09:49:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.899+0000 7fdb9f59e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb98050340 con 0x7fdb980e78b0 2023-10-16T09:49:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.898+0000 7f3903683640 1 -- 172.21.15.84:0/991990821 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0afde0 con 0x7f38fc0cb1c0 2023-10-16T09:49:11.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.899+0000 7fdb9f59e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb98006480 con 0x7fdb98006910 2023-10-16T09:49:11.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.899+0000 7fdb9f59e640 1 -- 172.21.15.84:0/3464452645 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb980ebc40 con 0x7fdb980eb2c0 2023-10-16T09:49:11.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.899+0000 7fdb9e59c640 1 -- 172.21.15.84:0/3464452645 learned_addr learned my addr 172.21.15.84:0/3464452645 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:11.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.898+0000 7f2eb47ee640 1 -- start start 2023-10-16T09:49:11.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.900+0000 7f2eb47ee640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2eac158f60 0x7f2eac0ade50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.900+0000 7f2eb47ee640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2eac06a4b0 0x7f2eac0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.900+0000 7f2eb47ee640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eac06cf50 con 0x7f2eac0ce1c0 2023-10-16T09:49:11.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.900+0000 7f2eb47ee640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eac15d970 con 0x7f2eac06a4b0 2023-10-16T09:49:11.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.900+0000 7f2eb47ee640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac15ccf0 con 0x7f2eac158f60 2023-10-16T09:49:11.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.898+0000 7f3900c1e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38fc0cb1c0 0x7f38fc0b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.901+0000 7f390141f640 1 -- 172.21.15.84:0/991990821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38fc0cb1c0 msgr2=0x7f38fc0b5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.901+0000 7f390141f640 1 --2- 172.21.15.84:0/991990821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38fc0cb1c0 0x7f38fc0b5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.901+0000 7f390141f640 1 -- 172.21.15.84:0/991990821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38fc15af80 msgr2=0x7f38fc0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.901+0000 7f390141f640 1 --2- 172.21.15.84:0/991990821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38fc15af80 0x7f38fc0cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.901+0000 7f390141f640 1 -- 172.21.15.84:0/991990821 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f4009a70 con 0x7f38fc0661c0 2023-10-16T09:49:11.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.900+0000 7fdb9dd9b640 1 --2- 172.21.15.84:0/3464452645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb980eb2c0 0x7fdb980eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.901+0000 7fdb9dd9b640 1 -- 172.21.15.84:0/3464452645 >> v1:172.21.15.157:6789/0 conn(0x7fdb98006910 legacy=0x7fdb98006da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.901+0000 7f2eb2d8b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2eac06a4b0 0x7f2eac0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.901+0000 7f2eb258a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2eac158f60 0x7f2eac0ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.901+0000 7ffb26c07640 1 Processor -- start 2023-10-16T09:49:11.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.901+0000 7ffb26c07640 1 -- start start 2023-10-16T09:49:11.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.901+0000 7f2eb258a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2eac158f60 0x7f2eac0ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:36984/0 (socket says 172.21.15.84:36984) 2023-10-16T09:49:11.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.902+0000 7f2eb2d8b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2eac06a4b0 0x7f2eac0cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59736/0 (socket says 172.21.15.84:59736) 2023-10-16T09:49:11.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.902+0000 7f2eb2d8b640 1 -- 172.21.15.84:0/2479991241 learned_addr learned my addr 172.21.15.84:0/2479991241 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:11.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.902+0000 7fdb9d59a640 1 -- 172.21.15.84:0/3464452645 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051883015 0 0) 0x7fdb98006480 con 0x7fdb98006910 2023-10-16T09:49:11.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.902+0000 7fdb9dd9b640 1 -- 172.21.15.84:0/3464452645 >> v1:172.21.15.84:6789/0 conn(0x7fdb980e78b0 legacy=0x7fdb9804f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.902+0000 7fdb9dd9b640 1 -- 172.21.15.84:0/3464452645 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9804fd80 con 0x7fdb980eb2c0 2023-10-16T09:49:11.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.902+0000 7f390141f640 1 --2- 172.21.15.84:0/991990821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38fc0661c0 0x7f38fc1ff270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00c400 tx=0x7f38ec00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.903+0000 7ffb26c07640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb200b3420 0x7ffb200c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.903+0000 7ffb26c07640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb200cf710 0x7ffb2015aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.903+0000 7ffb26c07640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb200cfaf0 0x7ffb20159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.903+0000 7ffb26c07640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb20159b00 con 0x7ffb200b3420 2023-10-16T09:49:11.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.903+0000 7ffb26c07640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb20159de0 con 0x7ffb200cf710 2023-10-16T09:49:11.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.903+0000 7ffb26c07640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2015a0c0 con 0x7ffb200cfaf0 2023-10-16T09:49:11.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.903+0000 7f2eb2d8b640 1 -- 172.21.15.84:0/2479991241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2eac158f60 msgr2=0x7f2eac0ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:11.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.903+0000 7f2eb1588640 1 -- 172.21.15.84:0/2479991241 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1607107517 0 0) 0x7f2eac06cf50 con 0x7f2eac0ce1c0 2023-10-16T09:49:11.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.903+0000 7f2eb2d8b640 1 --2- 172.21.15.84:0/2479991241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2eac158f60 0x7f2eac0ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.903+0000 7f2eb2d8b640 1 -- 172.21.15.84:0/2479991241 >> v1:172.21.15.84:6789/0 conn(0x7f2eac0ce1c0 legacy=0x7f2eac069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.904+0000 7f2eb2d8b640 1 -- 172.21.15.84:0/2479991241 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac15cfd0 con 0x7f2eac06a4b0 2023-10-16T09:49:11.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.903+0000 7f38f1ffb640 1 -- 172.21.15.84:0/991990821 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec0052d0 con 0x7f38fc0661c0 2023-10-16T09:49:11.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.903+0000 7f3903683640 1 -- 172.21.15.84:0/991990821 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38fc0b00c0 con 0x7f38fc0661c0 2023-10-16T09:49:11.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.903+0000 7f3903683640 1 -- 172.21.15.84:0/991990821 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38fc0af720 con 0x7f38fc0661c0 2023-10-16T09:49:11.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.904+0000 7ffb1ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb200cf710 0x7ffb2015aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.904+0000 7ffb1ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb200cf710 0x7ffb2015aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:55194/0 (socket says 172.21.15.84:55194) 2023-10-16T09:49:11.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.904+0000 7ffb1ffff640 1 -- 172.21.15.84:0/1901300169 learned_addr learned my addr 172.21.15.84:0/1901300169 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:11.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.904+0000 7f38f1ffb640 1 -- 172.21.15.84:0/991990821 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec005430 con 0x7f38fc0661c0 2023-10-16T09:49:11.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.904+0000 7f38f1ffb640 1 -- 172.21.15.84:0/991990821 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec003830 con 0x7f38fc0661c0 2023-10-16T09:49:11.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.904+0000 7ffb249a3640 1 --2- 172.21.15.84:0/1901300169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb200b3420 0x7ffb200c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.905+0000 7ffb251a4640 1 --2- 172.21.15.84:0/1901300169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb200cfaf0 0x7ffb20159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.905+0000 7f2eb258a640 1 --2- 172.21.15.84:0/2479991241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2eac158f60 0x7f2eac0ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:11.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.905+0000 7ffb1ffff640 1 -- 172.21.15.84:0/1901300169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb200cfaf0 msgr2=0x7ffb20159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.905+0000 7ffb1ffff640 1 --2- 172.21.15.84:0/1901300169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb200cfaf0 0x7ffb20159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.905+0000 7ffb1ffff640 1 -- 172.21.15.84:0/1901300169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb200b3420 msgr2=0x7ffb200c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.905+0000 7ffb1ffff640 1 --2- 172.21.15.84:0/1901300169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb200b3420 0x7ffb200c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.905+0000 7ffb1ffff640 1 -- 172.21.15.84:0/1901300169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb14013a10 con 0x7ffb200cf710 2023-10-16T09:49:11.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.905+0000 7fdb9dd9b640 1 --2- 172.21.15.84:0/3464452645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb980eb2c0 0x7fdb980eb700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdb940097d0 tx=0x7fdb9402b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ef07d37c199a4a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.905+0000 7f2eb2d8b640 1 --2- 172.21.15.84:0/2479991241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2eac06a4b0 0x7f2eac0cc660 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4009bd0 tx=0x7f2ea402d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6c808e7b28735a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.905+0000 7f3903683640 1 -- 172.21.15.84:0/991990821 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38dc005250 con 0x7f38fc0661c0 2023-10-16T09:49:11.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.905+0000 7fc76b002640 1 Processor -- start 2023-10-16T09:49:11.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.906+0000 7fc76b002640 1 -- start start 2023-10-16T09:49:11.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.906+0000 7f2eb1588640 1 -- 172.21.15.84:0/2479991241 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40353f0 con 0x7f2eac06a4b0 2023-10-16T09:49:11.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.906+0000 7ffb1ffff640 1 --2- 172.21.15.84:0/1901300169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb200cf710 0x7ffb2015aba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800efc0 tx=0x7ffb1800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.906+0000 7f2eb1588640 1 -- 172.21.15.84:0/2479991241 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea4035550 con 0x7f2eac06a4b0 2023-10-16T09:49:11.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.906+0000 7f2eb1588640 1 -- 172.21.15.84:0/2479991241 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40353f0 con 0x7f2eac06a4b0 2023-10-16T09:49:11.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.906+0000 7fdb9d59a640 1 -- 172.21.15.84:0/3464452645 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94030710 con 0x7fdb980eb2c0 2023-10-16T09:49:11.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.906+0000 7fdb9d59a640 1 -- 172.21.15.84:0/3464452645 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9402e050 con 0x7fdb980eb2c0 2023-10-16T09:49:11.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.906+0000 7fdb9d59a640 1 -- 172.21.15.84:0/3464452645 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94030a90 con 0x7fdb980eb2c0 2023-10-16T09:49:11.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.906+0000 7f7d83fff640 1 Processor -- start 2023-10-16T09:49:11.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.907+0000 7ffb1d7fa640 1 -- 172.21.15.84:0/1901300169 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb18009b80 con 0x7ffb200cf710 2023-10-16T09:49:11.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.907+0000 7ffb1d7fa640 1 -- 172.21.15.84:0/1901300169 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1800f040 con 0x7ffb200cf710 2023-10-16T09:49:11.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.907+0000 7ffb1d7fa640 1 -- 172.21.15.84:0/1901300169 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb18009b80 con 0x7ffb200cf710 2023-10-16T09:49:11.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.907+0000 7f2eb47ee640 1 -- 172.21.15.84:0/2479991241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2eac06a4b0 msgr2=0x7f2eac0cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.907+0000 7f2eb47ee640 1 --2- 172.21.15.84:0/2479991241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2eac06a4b0 0x7f2eac0cc660 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4009bd0 tx=0x7f2ea402d360 comp rx=0 tx=0).stop 2023-10-16T09:49:11.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.907+0000 7f7d83fff640 1 -- start start 2023-10-16T09:49:11.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.907+0000 7fdb9f59e640 1 -- 172.21.15.84:0/3464452645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb980eb2c0 msgr2=0x7fdb980eb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.907+0000 7fdb9f59e640 1 --2- 172.21.15.84:0/3464452645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb980eb2c0 0x7fdb980eb700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdb940097d0 tx=0x7fdb9402b710 comp rx=0 tx=0).stop 2023-10-16T09:49:11.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.907+0000 7ffb26c07640 1 -- 172.21.15.84:0/1901300169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2015a3a0 con 0x7ffb200cf710 2023-10-16T09:49:11.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.907+0000 7f2eb47ee640 1 -- 172.21.15.84:0/2479991241 shutdown_connections 2023-10-16T09:49:11.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.907+0000 7f2eb47ee640 1 --2- 172.21.15.84:0/2479991241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2eac06a4b0 0x7f2eac0cc660 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.907+0000 7f2eb47ee640 1 --2- 172.21.15.84:0/2479991241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2eac158f60 0x7f2eac0ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.907+0000 7f2eb47ee640 1 -- 172.21.15.84:0/2479991241 >> 172.21.15.84:0/2479991241 conn(0x7f2eac0ce9b0 msgr2=0x7f2eac0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:11.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.908+0000 7f2eb47ee640 1 -- 172.21.15.84:0/2479991241 shutdown_connections 2023-10-16T09:49:11.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.908+0000 7f2eb47ee640 1 -- 172.21.15.84:0/2479991241 wait complete. 2023-10-16T09:49:11.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.908+0000 7ffb26c07640 1 -- 172.21.15.84:0/1901300169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb20155520 con 0x7ffb200cf710 2023-10-16T09:49:11.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.908+0000 7fdb9f59e640 1 -- 172.21.15.84:0/3464452645 shutdown_connections 2023-10-16T09:49:11.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.908+0000 7fdb9f59e640 1 --2- 172.21.15.84:0/3464452645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb980eb2c0 0x7fdb980eb700 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.908+0000 7fdb9f59e640 1 -- 172.21.15.84:0/3464452645 >> 172.21.15.84:0/3464452645 conn(0x7fdb98054c10 msgr2=0x7fdb98054ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:11.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.908+0000 7f7d83fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c0e77a0 0x7f7d7c04f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.908+0000 7f7d83fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d7c0eb1b0 0x7f7d7c0eb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.909+0000 7f7d83fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7c050230 con 0x7f7d7c052ed0 2023-10-16T09:49:11.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.909+0000 7f7d83fff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d7c006370 con 0x7f7d7c0eb1b0 2023-10-16T09:49:11.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.909+0000 7f7d83fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0ebb30 con 0x7f7d7c0e77a0 2023-10-16T09:49:11.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.909+0000 7fc76b002640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7640cda70 0x7fc76406aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.909+0000 7fc76b002640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc76406b3f0 0x7fc76406ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.909+0000 7fc76b002640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7640c3610 con 0x7fc764159e40 2023-10-16T09:49:11.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.909+0000 7f38f1ffb640 1 -- 172.21.15.84:0/991990821 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f38ec003990 con 0x7f38fc0661c0 2023-10-16T09:49:11.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.909+0000 7fc76b002640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc76406b880 con 0x7fc76406b3f0 2023-10-16T09:49:11.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.909+0000 7fc76b002640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7640cdeb0 con 0x7fc7640cda70 2023-10-16T09:49:11.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.909+0000 7fc768d9e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc764159e40 0x7fc7640ca780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32906/0 (socket says 172.21.15.84:32906) 2023-10-16T09:49:11.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.909+0000 7fc768d9e640 1 -- 172.21.15.84:0/786296110 learned_addr learned my addr 172.21.15.84:0/786296110 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:11.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.909+0000 7f7d837fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7d7c052ed0 0x7f7d7c053360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32914/0 (socket says 172.21.15.84:32914) 2023-10-16T09:49:11.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.909+0000 7fc763fff640 1 --2- 172.21.15.84:0/786296110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7640cda70 0x7fc76406aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7f7d827fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d7c0eb1b0 0x7f7d7c0eb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7fc7637fe640 1 -- 172.21.15.84:0/786296110 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3474182144 0 0) 0x7fc7640c3610 con 0x7fc764159e40 2023-10-16T09:49:11.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7f7d827fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d7c0eb1b0 0x7f7d7c0eb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:55206/0 (socket says 172.21.15.84:55206) 2023-10-16T09:49:11.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7f7d827fc640 1 -- 172.21.15.84:0/4189749178 learned_addr learned my addr 172.21.15.84:0/4189749178 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:11.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7fc7637fe640 1 -- 172.21.15.84:0/786296110 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc750003520 con 0x7fc764159e40 2023-10-16T09:49:11.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7f38f1ffb640 1 --2- 172.21.15.84:0/991990821 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f38e006ae70 0x7f38e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7f7d82ffd640 1 --2- 172.21.15.84:0/4189749178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c0e77a0 0x7f7d7c04f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.909+0000 7fdb9f59e640 1 -- 172.21.15.84:0/3464452645 shutdown_connections 2023-10-16T09:49:11.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7fdb9f59e640 1 -- 172.21.15.84:0/3464452645 wait complete. 2023-10-16T09:49:11.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7fc7637fe640 1 -- 172.21.15.84:0/786296110 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1113795948 0 0) 0x7fc750003520 con 0x7fc764159e40 2023-10-16T09:49:11.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7fc7637fe640 1 -- 172.21.15.84:0/786296110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7640cda70 msgr2=0x7fc76406aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:11.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7fc7637fe640 1 --2- 172.21.15.84:0/786296110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7640cda70 0x7fc76406aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7fc7637fe640 1 -- 172.21.15.84:0/786296110 >> v2:172.21.15.84:3300/0 conn(0x7fc76406b3f0 msgr2=0x7fc76406ccc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T09:49:11.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7fc7637fe640 1 --2- 172.21.15.84:0/786296110 >> v2:172.21.15.84:3300/0 conn(0x7fc76406b3f0 0x7fc76406ccc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7fc7637fe640 1 -- 172.21.15.84:0/786296110 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76415a220 con 0x7fc764159e40 2023-10-16T09:49:11.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7fc7637fe640 1 -- 172.21.15.84:0/786296110 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc74c001670 con 0x7fc764159e40 2023-10-16T09:49:11.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7fc7637fe640 1 -- 172.21.15.84:0/786296110 >> v1:172.21.15.84:6789/0 conn(0x7fc764159e40 legacy=0x7fc7640ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7fc7637fe640 1 --2- 172.21.15.84:0/786296110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc750003970 0x7fc750003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7fc7637fe640 1 --2- 172.21.15.84:0/786296110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7500047c0 0x7fc750004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7fc7637fe640 1 --2- 172.21.15.84:0/786296110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc750005180 0x7fc75000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7fc7637fe640 1 -- 172.21.15.84:0/786296110 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76406b880 con 0x7fc750003970 2023-10-16T09:49:11.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7fc7637fe640 1 -- 172.21.15.84:0/786296110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7640cdeb0 con 0x7fc750005180 2023-10-16T09:49:11.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7fc7637fe640 1 -- 172.21.15.84:0/786296110 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75000c020 con 0x7fc7500047c0 2023-10-16T09:49:11.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.910+0000 7f38f1ffb640 1 -- 172.21.15.84:0/991990821 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38ec088ff0 con 0x7f38fc0661c0 2023-10-16T09:49:11.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.911+0000 7f7d827fc640 1 -- 172.21.15.84:0/4189749178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c0e77a0 msgr2=0x7f7d7c04f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:11.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.911+0000 7f7d827fc640 1 --2- 172.21.15.84:0/4189749178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c0e77a0 0x7f7d7c04f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.911+0000 7f7d827fc640 1 -- 172.21.15.84:0/4189749178 >> v1:172.21.15.84:6789/0 conn(0x7f7d7c052ed0 legacy=0x7f7d7c053360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.911+0000 7f7d827fc640 1 -- 172.21.15.84:0/4189749178 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c007810 con 0x7f7d7c0eb1b0 2023-10-16T09:49:11.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.911+0000 7f3900c1e640 1 --2- 172.21.15.84:0/991990821 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f38e006ae70 0x7f38e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.911+0000 7fc768d9e640 1 --2- 172.21.15.84:0/786296110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc750003970 0x7fc750003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.911+0000 7fc76959f640 1 --2- 172.21.15.84:0/786296110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc750005180 0x7fc75000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.912+0000 7f3900c1e640 1 --2- 172.21.15.84:0/991990821 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f38e006ae70 0x7f38e006d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0b11f0 tx=0x7f38f8007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.912+0000 7ffb26c07640 1 -- 172.21.15.84:0/1901300169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2015a8c0 con 0x7ffb200cf710 2023-10-16T09:49:11.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.912+0000 7fc763fff640 1 --2- 172.21.15.84:0/786296110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7500047c0 0x7fc750004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.912+0000 7f2eb47ee640 1 Processor -- start 2023-10-16T09:49:11.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.912+0000 7f2eb47ee640 1 -- start start 2023-10-16T09:49:11.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.913+0000 7fdb9f59e640 1 Processor -- start 2023-10-16T09:49:11.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.912+0000 7fc76959f640 1 -- 172.21.15.84:0/786296110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7500047c0 msgr2=0x7fc750004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:11.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.912+0000 7fc76959f640 1 --2- 172.21.15.84:0/786296110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7500047c0 0x7fc750004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.912+0000 7fc76959f640 1 -- 172.21.15.84:0/786296110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc750003970 msgr2=0x7fc750003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:11.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.912+0000 7fc76959f640 1 --2- 172.21.15.84:0/786296110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc750003970 0x7fc750003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.912+0000 7fc76959f640 1 -- 172.21.15.84:0/786296110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc750003520 con 0x7fc750005180 2023-10-16T09:49:11.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.913+0000 7f2eb47ee640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2eac06a4b0 0x7f2eac152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.914+0000 7f2eb47ee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2eac0ce1c0 0x7f2eac06f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.914+0000 7f2eb47ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2eac158f60 0x7f2eac06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.914+0000 7f2eb47ee640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac0c4320 con 0x7f2eac158f60 2023-10-16T09:49:11.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.914+0000 7f2eb47ee640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac0c4600 con 0x7f2eac0ce1c0 2023-10-16T09:49:11.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.914+0000 7f2eb47ee640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac0c48e0 con 0x7f2eac06a4b0 2023-10-16T09:49:11.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.913+0000 7fdb9f59e640 1 -- start start 2023-10-16T09:49:11.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.914+0000 7f2eb2d8b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2eac158f60 0x7f2eac06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.914+0000 7f2eb2d8b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2eac158f60 0x7f2eac06fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59762/0 (socket says 172.21.15.84:59762) 2023-10-16T09:49:11.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.914+0000 7f2eb2d8b640 1 -- 172.21.15.84:0/2497595141 learned_addr learned my addr 172.21.15.84:0/2497595141 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:11.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.914+0000 7f2eb258a640 1 --2- 172.21.15.84:0/2497595141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2eac06a4b0 0x7f2eac152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.914+0000 7fdb9f59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb98006910 0x7fdb980e96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.914+0000 7fdb9f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb980e78b0 0x7fdb9817c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.914+0000 7f2eb2d8b640 1 -- 172.21.15.84:0/2497595141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2eac06a4b0 msgr2=0x7f2eac152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.914+0000 7f2eb2d8b640 1 --2- 172.21.15.84:0/2497595141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2eac06a4b0 0x7f2eac152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.915+0000 7f2eb2d8b640 1 -- 172.21.15.84:0/2497595141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2eac0ce1c0 msgr2=0x7f2eac06f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.915+0000 7f2eb1d89640 1 --2- 172.21.15.84:0/2497595141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2eac0ce1c0 0x7f2eac06f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.915+0000 7f2eb2d8b640 1 --2- 172.21.15.84:0/2497595141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2eac0ce1c0 0x7f2eac06f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.915+0000 7f2eb2d8b640 1 -- 172.21.15.84:0/2497595141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea4009830 con 0x7f2eac158f60 2023-10-16T09:49:11.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.915+0000 7f2eb1d89640 1 --2- 172.21.15.84:0/2497595141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2eac0ce1c0 0x7f2eac06f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:11.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.915+0000 7fc76959f640 1 --2- 172.21.15.84:0/786296110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc750005180 0x7fc75000b980 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc754009370 tx=0x7fc7500057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e9da404848017db server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.915+0000 7fdb9f59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb980eb2c0 0x7fdb980e5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.915+0000 7fdb9f59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb980e3940 con 0x7fdb980e78b0 2023-10-16T09:49:11.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.915+0000 7fdb9f59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb980e3be0 con 0x7fdb980eb2c0 2023-10-16T09:49:11.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.915+0000 7fdb9f59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb980e9c30 con 0x7fdb98006910 2023-10-16T09:49:11.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.915+0000 7fdb9e59c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb98006910 0x7fdb980e96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.915+0000 7fdb9dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb980e78b0 0x7fdb9817c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.915+0000 7fdb9dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb980e78b0 0x7fdb9817c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59778/0 (socket says 172.21.15.84:59778) 2023-10-16T09:49:11.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.915+0000 7fdb9dd9b640 1 -- 172.21.15.84:0/1643732984 learned_addr learned my addr 172.21.15.84:0/1643732984 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:11.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.915+0000 7f7d827fc640 1 --2- 172.21.15.84:0/4189749178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d7c0eb1b0 0x7f7d7c0eb5f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009870 tx=0x7f7d7802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc695aae9fdf362a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.915+0000 7fdb9ed9d640 1 --2- 172.21.15.84:0/1643732984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb980eb2c0 0x7fdb980e5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.916+0000 7fdb9dd9b640 1 -- 172.21.15.84:0/1643732984 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb98006910 msgr2=0x7fdb980e96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.916+0000 7fdb9dd9b640 1 --2- 172.21.15.84:0/1643732984 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb98006910 0x7fdb980e96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.916+0000 7fdb9dd9b640 1 -- 172.21.15.84:0/1643732984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb980eb2c0 msgr2=0x7fdb980e5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.916+0000 7fdb9dd9b640 1 --2- 172.21.15.84:0/1643732984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb980eb2c0 0x7fdb980e5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.916+0000 7fdb9dd9b640 1 -- 172.21.15.84:0/1643732984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb94009400 con 0x7fdb980e78b0 2023-10-16T09:49:11.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.916+0000 7f2eb2d8b640 1 --2- 172.21.15.84:0/2497595141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2eac158f60 0x7f2eac06fc90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4037040 tx=0x7f2ea4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.916+0000 7f2ea2ffd640 1 -- 172.21.15.84:0/2497595141 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40353f0 con 0x7f2eac158f60 2023-10-16T09:49:11.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.916+0000 7f2ea2ffd640 1 -- 172.21.15.84:0/2497595141 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea4035e80 con 0x7f2eac158f60 2023-10-16T09:49:11.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.916+0000 7f2ea2ffd640 1 -- 172.21.15.84:0/2497595141 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40095b0 con 0x7f2eac158f60 2023-10-16T09:49:11.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.916+0000 7fdb9dd9b640 1 --2- 172.21.15.84:0/1643732984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb980e78b0 0x7fdb9817c5f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009720 tx=0x7fdb940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.916+0000 7f2eb47ee640 1 -- 172.21.15.84:0/2497595141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eac0c4bc0 con 0x7f2eac158f60 2023-10-16T09:49:11.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.916+0000 7f2eb47ee640 1 -- 172.21.15.84:0/2497595141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eac153360 con 0x7f2eac158f60 2023-10-16T09:49:11.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.916+0000 7fdb8effd640 1 -- 172.21.15.84:0/1643732984 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb940329d0 con 0x7fdb980e78b0 2023-10-16T09:49:11.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.917+0000 7fdb9f59e640 1 -- 172.21.15.84:0/1643732984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb980e9eb0 con 0x7fdb980e78b0 2023-10-16T09:49:11.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.917+0000 7fdb8effd640 1 -- 172.21.15.84:0/1643732984 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb94032b30 con 0x7fdb980e78b0 2023-10-16T09:49:11.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.917+0000 7fdb8effd640 1 -- 172.21.15.84:0/1643732984 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9403d480 con 0x7fdb980e78b0 2023-10-16T09:49:11.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.917+0000 7fdb9f59e640 1 -- 172.21.15.84:0/1643732984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb980ea470 con 0x7fdb980e78b0 2023-10-16T09:49:11.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.918+0000 7f2eb47ee640 1 -- 172.21.15.84:0/2497595141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eac04e160 con 0x7f2eac158f60 2023-10-16T09:49:11.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.919+0000 7fdb8cff9640 1 -- 172.21.15.84:0/1643732984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb78005250 con 0x7fdb980e78b0 2023-10-16T09:49:11.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.919+0000 7f2ea2ffd640 1 -- 172.21.15.84:0/2497595141 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2ea4004260 con 0x7f2eac158f60 2023-10-16T09:49:11.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.919+0000 7eff78389640 1 Processor -- start 2023-10-16T09:49:11.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.919+0000 7eff78389640 1 -- start start 2023-10-16T09:49:11.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.920+0000 7f2ea2ffd640 1 --2- 172.21.15.84:0/2497595141 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2e9006aef0 0x7f2e9006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.920+0000 7f2ea2ffd640 1 -- 172.21.15.84:0/2497595141 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ea40af4e0 con 0x7f2eac158f60 2023-10-16T09:49:11.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.920+0000 7fdb8effd640 1 -- 172.21.15.84:0/1643732984 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdb94039040 con 0x7fdb980e78b0 2023-10-16T09:49:11.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.920+0000 7f2eb258a640 1 --2- 172.21.15.84:0/2497595141 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2e9006aef0 0x7f2e9006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.921+0000 7eff78389640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7eff700ad920 0x7eff700add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.921+0000 7eff78389640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7eff700ce2b0 0x7eff70069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.921+0000 7eff78389640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff7006ba40 con 0x7eff7006a2e0 2023-10-16T09:49:11.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.921+0000 7eff78389640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff7006c000 con 0x7eff700ce2b0 2023-10-16T09:49:11.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.921+0000 7eff78389640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff7006a750 con 0x7eff700ad920 2023-10-16T09:49:11.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.921+0000 7eff75924640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7eff700ce2b0 0x7eff70069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.921+0000 7eff75924640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7eff700ce2b0 0x7eff70069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59788/0 (socket says 172.21.15.84:59788) 2023-10-16T09:49:11.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.921+0000 7eff75924640 1 -- 172.21.15.84:0/1411230374 learned_addr learned my addr 172.21.15.84:0/1411230374 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:11.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.922+0000 7eff76125640 1 --2- 172.21.15.84:0/1411230374 >> v2:172.21.15.157:3300/0 conn(0x7eff700ad920 0x7eff700add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.922+0000 7fdb8effd640 1 --2- 172.21.15.84:0/1643732984 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdb8006aef0 0x7fdb8006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.922+0000 7fdb8effd640 1 -- 172.21.15.84:0/1643732984 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb940ab5a0 con 0x7fdb980e78b0 2023-10-16T09:49:11.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.922+0000 7eff75924640 1 -- 172.21.15.84:0/1411230374 >> v2:172.21.15.157:3300/0 conn(0x7eff700ad920 msgr2=0x7eff700add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:11.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.922+0000 7eff75924640 1 --2- 172.21.15.84:0/1411230374 >> v2:172.21.15.157:3300/0 conn(0x7eff700ad920 0x7eff700add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.922+0000 7eff75924640 1 -- 172.21.15.84:0/1411230374 >> v1:172.21.15.84:6789/0 conn(0x7eff7006a2e0 legacy=0x7eff701651d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.922+0000 7eff75924640 1 -- 172.21.15.84:0/1411230374 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7006bd20 con 0x7eff700ce2b0 2023-10-16T09:49:11.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.922+0000 7fdb9e59c640 1 --2- 172.21.15.84:0/1643732984 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdb8006aef0 0x7fdb8006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.922+0000 7f2eb258a640 1 --2- 172.21.15.84:0/2497595141 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2e9006aef0 0x7f2e9006d3b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8006fd0 tx=0x7f2ea8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.923+0000 7f38f1ffb640 1 -- 172.21.15.84:0/991990821 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f38ec054130 con 0x7f38fc0661c0 2023-10-16T09:49:11.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.923+0000 7f7d81ffb640 1 -- 172.21.15.84:0/4189749178 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78031640 con 0x7f7d7c0eb1b0 2023-10-16T09:49:11.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.923+0000 7f7d81ffb640 1 -- 172.21.15.84:0/4189749178 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d780317a0 con 0x7f7d7c0eb1b0 2023-10-16T09:49:11.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.923+0000 7fc7637fe640 1 -- 172.21.15.84:0/786296110 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75403b600 con 0x7fc750005180 2023-10-16T09:49:11.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.923+0000 7f7d81ffb640 1 -- 172.21.15.84:0/4189749178 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78031b30 con 0x7f7d7c0eb1b0 2023-10-16T09:49:11.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.923+0000 7fc7637fe640 1 -- 172.21.15.84:0/786296110 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75403b780 con 0x7fc750005180 2023-10-16T09:49:11.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.923+0000 7fc7637fe640 1 -- 172.21.15.84:0/786296110 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75403bb30 con 0x7fc750005180 2023-10-16T09:49:11.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.923+0000 7fdb9e59c640 1 --2- 172.21.15.84:0/1643732984 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdb8006aef0 0x7fdb8006d3b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdb90000f60 tx=0x7fdb90007e20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.924+0000 7f7d83fff640 1 -- 172.21.15.84:0/4189749178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d7c0eb1b0 msgr2=0x7f7d7c0eb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.924+0000 7f7d83fff640 1 --2- 172.21.15.84:0/4189749178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d7c0eb1b0 0x7f7d7c0eb5f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009870 tx=0x7f7d7802b8f0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.924+0000 7fc76b002640 1 -- 172.21.15.84:0/786296110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc750005180 msgr2=0x7fc75000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.924+0000 7fc76b002640 1 --2- 172.21.15.84:0/786296110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc750005180 0x7fc75000b980 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc754009370 tx=0x7fc7500057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.924+0000 7fc76959f640 1 -- 172.21.15.84:0/786296110 reap_dead start 2023-10-16T09:49:11.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.924+0000 7eff75924640 1 --2- 172.21.15.84:0/1411230374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff700ce2b0 0x7eff70069da0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7eff700cf4f0 tx=0x7eff6002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cce714537e15a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.924+0000 7f7d83fff640 1 -- 172.21.15.84:0/4189749178 shutdown_connections 2023-10-16T09:49:11.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.924+0000 7f7d83fff640 1 --2- 172.21.15.84:0/4189749178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d7c0eb1b0 0x7f7d7c0eb5f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.924+0000 7f7d83fff640 1 --2- 172.21.15.84:0/4189749178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c0e77a0 0x7f7d7c04f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.924+0000 7f7d83fff640 1 -- 172.21.15.84:0/4189749178 >> 172.21.15.84:0/4189749178 conn(0x7f7d7c054b00 msgr2=0x7f7d7c054ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:11.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.927+0000 7fc76b002640 1 -- 172.21.15.84:0/786296110 shutdown_connections 2023-10-16T09:49:11.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.927+0000 7fc76b002640 1 -- 172.21.15.84:0/786296110 >> 172.21.15.84:0/786296110 conn(0x7fc7640cfac0 msgr2=0x7fc7640cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:11.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.927+0000 7f7d83fff640 1 -- 172.21.15.84:0/4189749178 shutdown_connections 2023-10-16T09:49:11.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.926+0000 7ffb1d7fa640 1 -- 172.21.15.84:0/1901300169 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ffb18014480 con 0x7ffb200cf710 2023-10-16T09:49:11.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.927+0000 7ffb1d7fa640 1 --2- 172.21.15.84:0/1901300169 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffb0c06aed0 0x7ffb0c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.927+0000 7ffb249a3640 1 --2- 172.21.15.84:0/1901300169 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffb0c06aed0 0x7ffb0c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.927+0000 7ffb1d7fa640 1 -- 172.21.15.84:0/1901300169 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb180597f0 con 0x7ffb200cf710 2023-10-16T09:49:11.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.927+0000 7f7d83fff640 1 -- 172.21.15.84:0/4189749178 wait complete. 2023-10-16T09:49:11.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.927+0000 7fc76b002640 1 -- 172.21.15.84:0/786296110 shutdown_connections 2023-10-16T09:49:11.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.927+0000 7f2ea2ffd640 1 -- 172.21.15.84:0/2497595141 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2ea4076a40 con 0x7f2eac158f60 2023-10-16T09:49:11.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.928+0000 7fc76b002640 1 -- 172.21.15.84:0/786296110 wait complete. 2023-10-16T09:49:11.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.928+0000 7ffb249a3640 1 --2- 172.21.15.84:0/1901300169 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffb0c06aed0 0x7ffb0c06d390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffb10001360 tx=0x7ffb10009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.930+0000 7f7d83fff640 1 Processor -- start 2023-10-16T09:49:11.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.930+0000 7fc76b002640 1 Processor -- start 2023-10-16T09:49:11.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.931+0000 7f7d83fff640 1 -- start start 2023-10-16T09:49:11.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.931+0000 7fc76b002640 1 -- start start 2023-10-16T09:49:11.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.932+0000 7fc76b002640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc750005180 0x7fc7641edc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.932+0000 7fc76b002640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7641510b0 0x7fc764151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.932+0000 7fc76b002640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc764152eb0 0x7fc764153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.932+0000 7fc76b002640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc764153870 con 0x7fc7641510b0 2023-10-16T09:49:11.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.932+0000 7fc76b002640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7641ee1c0 con 0x7fc764152eb0 2023-10-16T09:49:11.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.932+0000 7fc76b002640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc764151c20 con 0x7fc750005180 2023-10-16T09:49:11.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.932+0000 7f7d83fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c052ed0 0x7f7d7c17a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.932+0000 7f7d83fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d7c0e77a0 0x7f7d7c0dda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.932+0000 7f7d83fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d7c0eb1b0 0x7f7d7c0ddf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:11.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.932+0000 7f7d83fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c17d390 con 0x7f7d7c0e77a0 2023-10-16T09:49:11.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.932+0000 7f7d83fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0dec00 con 0x7f7d7c0eb1b0 2023-10-16T09:49:11.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.932+0000 7f7d83fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0de640 con 0x7f7d7c052ed0 2023-10-16T09:49:11.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.933+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d7c0eb1b0 0x7f7d7c0ddf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.933+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d7c0eb1b0 0x7f7d7c0ddf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:55248/0 (socket says 172.21.15.84:55248) 2023-10-16T09:49:11.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.933+0000 7f7d837fe640 1 -- 172.21.15.84:0/3299918574 learned_addr learned my addr 172.21.15.84:0/3299918574 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:11.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.933+0000 7fc768d9e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc750005180 0x7fc7641edc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.933+0000 7fc763fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7641510b0 0x7fc764151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.933+0000 7fc763fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7641510b0 0x7fc764151530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59804/0 (socket says 172.21.15.84:59804) 2023-10-16T09:49:11.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.933+0000 7f7d82ffd640 1 --2- 172.21.15.84:0/3299918574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c052ed0 0x7f7d7c17a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.933+0000 7f7d827fc640 1 --2- 172.21.15.84:0/3299918574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d7c0e77a0 0x7f7d7c0dda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.933+0000 7fc76959f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc764152eb0 0x7fc764153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:11.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.933+0000 7fc768d9e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc750005180 0x7fc7641edc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:37058/0 (socket says 172.21.15.84:37058) 2023-10-16T09:49:11.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.934+0000 7fc768d9e640 1 -- 172.21.15.84:0/2107106810 learned_addr learned my addr 172.21.15.84:0/2107106810 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:11.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.934+0000 7f7d837fe640 1 -- 172.21.15.84:0/3299918574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c052ed0 msgr2=0x7f7d7c17a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.934+0000 7f7d837fe640 1 --2- 172.21.15.84:0/3299918574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c052ed0 0x7f7d7c17a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.934+0000 7f7d837fe640 1 -- 172.21.15.84:0/3299918574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d7c0e77a0 msgr2=0x7f7d7c0dda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.934+0000 7f7d837fe640 1 --2- 172.21.15.84:0/3299918574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d7c0e77a0 0x7f7d7c0dda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.934+0000 7f7d837fe640 1 -- 172.21.15.84:0/3299918574 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d780094d0 con 0x7f7d7c0eb1b0 2023-10-16T09:49:11.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.934+0000 7f7d837fe640 1 --2- 172.21.15.84:0/3299918574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d7c0eb1b0 0x7f7d7c0ddf70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d84000b60 tx=0x7f7d840671b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.935+0000 7fc763fff640 1 -- 172.21.15.84:0/2107106810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc750005180 msgr2=0x7fc7641edc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.935+0000 7fc763fff640 1 --2- 172.21.15.84:0/2107106810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc750005180 0x7fc7641edc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.935+0000 7fc763fff640 1 -- 172.21.15.84:0/2107106810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc764152eb0 msgr2=0x7fc764153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.935+0000 7fc763fff640 1 --2- 172.21.15.84:0/2107106810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc764152eb0 0x7fc764153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.935+0000 7fc763fff640 1 -- 172.21.15.84:0/2107106810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc764151f00 con 0x7fc7641510b0 2023-10-16T09:49:11.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.935+0000 7eff75123640 1 -- 172.21.15.84:0/1411230374 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60030860 con 0x7eff700ce2b0 2023-10-16T09:49:11.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.935+0000 7eff75123640 1 -- 172.21.15.84:0/1411230374 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff600309c0 con 0x7eff700ce2b0 2023-10-16T09:49:11.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.935+0000 7eff75123640 1 -- 172.21.15.84:0/1411230374 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60030d50 con 0x7eff700ce2b0 2023-10-16T09:49:11.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.935+0000 7fc76959f640 1 --2- 172.21.15.84:0/2107106810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc764152eb0 0x7fc764153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:49:11.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.935+0000 7fdb8effd640 1 -- 172.21.15.84:0/1643732984 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdb94076990 con 0x7fdb980e78b0 2023-10-16T09:49:11.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.935+0000 7fc768d9e640 1 --2- 172.21.15.84:0/2107106810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc750005180 0x7fc7641edc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:49:11.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.936+0000 7fc763fff640 1 --2- 172.21.15.84:0/2107106810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7641510b0 0x7fc764151530 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc7640cc510 tx=0x7fc75c0085d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:11.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.936+0000 7eff78389640 1 -- 172.21.15.84:0/1411230374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff700ce2b0 msgr2=0x7eff70069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:11.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.936+0000 7eff78389640 1 --2- 172.21.15.84:0/1411230374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff700ce2b0 0x7eff70069da0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7eff700cf4f0 tx=0x7eff6002b9a0 comp rx=0 tx=0).stop 2023-10-16T09:49:11.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.937+0000 7eff78389640 1 -- 172.21.15.84:0/1411230374 shutdown_connections 2023-10-16T09:49:12.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.937+0000 7eff78389640 1 --2- 172.21.15.84:0/1411230374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff700ce2b0 0x7eff70069da0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.937+0000 7eff78389640 1 --2- 172.21.15.84:0/1411230374 >> v2:172.21.15.157:3300/0 conn(0x7eff700ad920 0x7eff700add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.937+0000 7eff78389640 1 -- 172.21.15.84:0/1411230374 >> 172.21.15.84:0/1411230374 conn(0x7eff700ce9b0 msgr2=0x7eff700cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:12.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.937+0000 7fc7617fa640 1 -- 172.21.15.84:0/2107106810 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c010050 con 0x7fc7641510b0 2023-10-16T09:49:12.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.937+0000 7fc7617fa640 1 -- 172.21.15.84:0/2107106810 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75c011040 con 0x7fc7641510b0 2023-10-16T09:49:12.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.937+0000 7fc7617fa640 1 -- 172.21.15.84:0/2107106810 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c017650 con 0x7fc7641510b0 2023-10-16T09:49:12.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.937+0000 7fc76b002640 1 -- 172.21.15.84:0/2107106810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7641521e0 con 0x7fc7641510b0 2023-10-16T09:49:12.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.938+0000 7eff78389640 1 -- 172.21.15.84:0/1411230374 shutdown_connections 2023-10-16T09:49:12.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.938+0000 7ffb1d7fa640 1 -- 172.21.15.84:0/1901300169 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffb18055bc0 con 0x7ffb200cf710 2023-10-16T09:49:12.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.938+0000 7eff78389640 1 -- 172.21.15.84:0/1411230374 wait complete. 2023-10-16T09:49:12.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.938+0000 7fc76b002640 1 -- 172.21.15.84:0/2107106810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76415a690 con 0x7fc7641510b0 2023-10-16T09:49:12.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.940+0000 7fc76b002640 1 -- 172.21.15.84:0/2107106810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76404e160 con 0x7fc7641510b0 2023-10-16T09:49:12.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.938+0000 7f7d6bfff640 1 -- 172.21.15.84:0/3299918574 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d840925b0 con 0x7f7d7c0eb1b0 2023-10-16T09:49:12.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.938+0000 7f7d6bfff640 1 -- 172.21.15.84:0/3299918574 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d84092730 con 0x7f7d7c0eb1b0 2023-10-16T09:49:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.938+0000 7f7d6bfff640 1 -- 172.21.15.84:0/3299918574 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d84092b20 con 0x7f7d7c0eb1b0 2023-10-16T09:49:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.942+0000 7fc7617fa640 1 -- 172.21.15.84:0/2107106810 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc75c0179c0 con 0x7fc7641510b0 2023-10-16T09:49:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.942+0000 7f7d83fff640 1 -- 172.21.15.84:0/3299918574 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7c0de920 con 0x7f7d7c0eb1b0 2023-10-16T09:49:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.942+0000 7f7d83fff640 1 -- 172.21.15.84:0/3299918574 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7c17afc0 con 0x7f7d7c0eb1b0 2023-10-16T09:49:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.944+0000 7f7d83fff640 1 -- 172.21.15.84:0/3299918574 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7c005340 con 0x7f7d7c0eb1b0 2023-10-16T09:49:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.941+0000 7eff78389640 1 Processor -- start 2023-10-16T09:49:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.943+0000 7eff78389640 1 -- start start 2023-10-16T09:49:12.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.943+0000 7fc7617fa640 1 --2- 172.21.15.84:0/2107106810 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc74406aec0 0x7fc74406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:12.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.943+0000 7fc7617fa640 1 -- 172.21.15.84:0/2107106810 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc75c08a1c0 con 0x7fc7641510b0 2023-10-16T09:49:12.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.946+0000 7eff78389640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff7006a2e0 0x7eff700af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:12.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.946+0000 7eff78389640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff700ad920 0x7eff700c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:12.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.946+0000 7fc768d9e640 1 --2- 172.21.15.84:0/2107106810 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc74406aec0 0x7fc74406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:12.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.946+0000 7eff78389640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff700ce2b0 0x7eff700b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:12.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.946+0000 7eff78389640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff700b0be0 con 0x7eff700ce2b0 2023-10-16T09:49:12.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.946+0000 7eff78389640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7015b850 con 0x7eff700ad920 2023-10-16T09:49:12.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.946+0000 7eff78389640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff700aff60 con 0x7eff7006a2e0 2023-10-16T09:49:12.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.946+0000 7f7d6bfff640 1 -- 172.21.15.84:0/3299918574 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7d84092c80 con 0x7f7d7c0eb1b0 2023-10-16T09:49:12.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.947+0000 7eff76125640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff7006a2e0 0x7eff700af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:12.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.947+0000 7eff75924640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff700ad920 0x7eff700c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:12.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.947+0000 7fc768d9e640 1 --2- 172.21.15.84:0/2107106810 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc74406aec0 0x7fc74406d380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc74c007fd0 tx=0x7fc74c001df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:12.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.947+0000 7eff75924640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff700ad920 0x7eff700c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:55278/0 (socket says 172.21.15.84:55278) 2023-10-16T09:49:12.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.947+0000 7eff75924640 1 -- 172.21.15.84:0/4088927642 learned_addr learned my addr 172.21.15.84:0/4088927642 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:12.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.947+0000 7eff76926640 1 --2- 172.21.15.84:0/4088927642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff700ce2b0 0x7eff700b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:12.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.948+0000 7f7d6bfff640 1 --2- 172.21.15.84:0/3299918574 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7d7006aec0 0x7f7d7006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:12.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.948+0000 7eff75924640 1 -- 172.21.15.84:0/4088927642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff7006a2e0 msgr2=0x7eff700af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:12.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.948+0000 7eff75924640 1 --2- 172.21.15.84:0/4088927642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff7006a2e0 0x7eff700af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.948+0000 7eff75924640 1 -- 172.21.15.84:0/4088927642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff700ce2b0 msgr2=0x7eff700b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:12.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.948+0000 7eff75924640 1 --2- 172.21.15.84:0/4088927642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff700ce2b0 0x7eff700b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.948+0000 7eff75924640 1 -- 172.21.15.84:0/4088927642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff600094d0 con 0x7eff700ad920 2023-10-16T09:49:12.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.949+0000 7eff75924640 1 --2- 172.21.15.84:0/4088927642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff700ad920 0x7eff700c6af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff60000c00 tx=0x7eff60004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.949+0000 7f7d6bfff640 1 -- 172.21.15.84:0/3299918574 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d84103b00 con 0x7f7d7c0eb1b0 2023-10-16T09:49:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.949+0000 7f7d82ffd640 1 --2- 172.21.15.84:0/3299918574 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7d7006aec0 0x7f7d7006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.950+0000 7f7d82ffd640 1 --2- 172.21.15.84:0/3299918574 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7d7006aec0 0x7f7d7006d380 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c0536b0 tx=0x7f7d74009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.952+0000 7fc7617fa640 1 -- 172.21.15.84:0/2107106810 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc75c055390 con 0x7fc7641510b0 2023-10-16T09:49:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.958+0000 7eff66ffd640 1 -- 172.21.15.84:0/4088927642 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60039c90 con 0x7eff700ad920 2023-10-16T09:49:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.958+0000 7eff66ffd640 1 -- 172.21.15.84:0/4088927642 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff60039df0 con 0x7eff700ad920 2023-10-16T09:49:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.958+0000 7eff66ffd640 1 -- 172.21.15.84:0/4088927642 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60043640 con 0x7eff700ad920 2023-10-16T09:49:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.958+0000 7eff78389640 1 -- 172.21.15.84:0/4088927642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff700b0240 con 0x7eff700ad920 2023-10-16T09:49:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.958+0000 7eff78389640 1 -- 172.21.15.84:0/4088927642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff700b0800 con 0x7eff700ad920 2023-10-16T09:49:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.959+0000 7f7d6bfff640 1 -- 172.21.15.84:0/3299918574 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7d840cedd0 con 0x7f7d7c0eb1b0 2023-10-16T09:49:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.961+0000 7eff78389640 1 -- 172.21.15.84:0/4088927642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff58005250 con 0x7eff700ad920 2023-10-16T09:49:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.963+0000 7eff66ffd640 1 -- 172.21.15.84:0/4088927642 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7eff600437a0 con 0x7eff700ad920 2023-10-16T09:49:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.964+0000 7eff66ffd640 1 --2- 172.21.15.84:0/4088927642 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7eff5406ae70 0x7eff5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.965+0000 7eff66ffd640 1 -- 172.21.15.84:0/4088927642 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff6003e070 con 0x7eff700ad920 2023-10-16T09:49:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.966+0000 7eff76125640 1 --2- 172.21.15.84:0/4088927642 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7eff5406ae70 0x7eff5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.967+0000 7eff76125640 1 --2- 172.21.15.84:0/4088927642 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7eff5406ae70 0x7eff5406d330 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009b90 tx=0x7eff6c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:11.979+0000 7eff66ffd640 1 -- 172.21.15.84:0/4088927642 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7eff60077670 con 0x7eff700ad920 2023-10-16T09:49:12.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.241+0000 7f7d3a590640 1 -- 172.21.15.84:0/3792055076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7d340d26c0 con 0x7f7d3415cf10 2023-10-16T09:49:12.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.242+0000 7f7d30ff9640 1 -- 172.21.15.84:0/3792055076 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7d340d26c0 con 0x7f7d3415cf10 2023-10-16T09:49:12.243 INFO:teuthology.orchestra.run.smithi084.stdout:55834574851 2023-10-16T09:49:12.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.245+0000 7f7d3a590640 1 -- 172.21.15.84:0/3792055076 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7d1806ae40 msgr2=0x7f7d1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:12.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.245+0000 7f7d3a590640 1 --2- 172.21.15.84:0/3792055076 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7d1806ae40 0x7f7d1806d300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7d28036000 tx=0x7f7d28008030 comp rx=0 tx=0).stop 2023-10-16T09:49:12.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.246+0000 7f7d3a590640 1 -- 172.21.15.84:0/3792055076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3415cf10 msgr2=0x7f7d34154580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:12.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.246+0000 7f7d3a590640 1 --2- 172.21.15.84:0/3792055076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3415cf10 0x7f7d34154580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400b970 tx=0x7f7d2400be30 comp rx=0 tx=0).stop 2023-10-16T09:49:12.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.246+0000 7f7d3a590640 1 -- 172.21.15.84:0/3792055076 shutdown_connections 2023-10-16T09:49:12.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.246+0000 7f7d3a590640 1 --2- 172.21.15.84:0/3792055076 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7d1806ae40 0x7f7d1806d300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.247+0000 7f7d3a590640 1 --2- 172.21.15.84:0/3792055076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d3415cf10 0x7f7d34154580 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.247+0000 7f7d3a590640 1 --2- 172.21.15.84:0/3792055076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d340d10c0 0x7f7d340cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.247+0000 7f7d3a590640 1 --2- 172.21.15.84:0/3792055076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d340b4ac0 0x7f7d34148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.247+0000 7f7d3a590640 1 -- 172.21.15.84:0/3792055076 >> 172.21.15.84:0/3792055076 conn(0x7f7d340cf580 msgr2=0x7f7d340b6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:12.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.247+0000 7f7d3a590640 1 -- 172.21.15.84:0/3792055076 shutdown_connections 2023-10-16T09:49:12.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.247+0000 7f7d3a590640 1 -- 172.21.15.84:0/3792055076 wait complete. 2023-10-16T09:49:12.275 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.2 2023-10-16T09:49:12.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.324+0000 7eff78389640 1 -- 172.21.15.84:0/4088927642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7eff58005500 con 0x7eff700ad920 2023-10-16T09:49:12.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.325+0000 7eff66ffd640 1 -- 172.21.15.84:0/4088927642 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7eff600aae60 con 0x7eff700ad920 2023-10-16T09:49:12.326 INFO:teuthology.orchestra.run.smithi084.stdout:55834574852 2023-10-16T09:49:12.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.327+0000 7eff78389640 1 -- 172.21.15.84:0/4088927642 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7eff5406ae70 msgr2=0x7eff5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:12.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.328+0000 7eff78389640 1 --2- 172.21.15.84:0/4088927642 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7eff5406ae70 0x7eff5406d330 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009b90 tx=0x7eff6c009190 comp rx=0 tx=0).stop 2023-10-16T09:49:12.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.328+0000 7eff78389640 1 -- 172.21.15.84:0/4088927642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff700ad920 msgr2=0x7eff700c6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:12.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.328+0000 7eff78389640 1 --2- 172.21.15.84:0/4088927642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff700ad920 0x7eff700c6af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff60000c00 tx=0x7eff60004730 comp rx=0 tx=0).stop 2023-10-16T09:49:12.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.328+0000 7eff78389640 1 -- 172.21.15.84:0/4088927642 shutdown_connections 2023-10-16T09:49:12.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.328+0000 7eff78389640 1 --2- 172.21.15.84:0/4088927642 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7eff5406ae70 0x7eff5406d330 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.328+0000 7eff78389640 1 --2- 172.21.15.84:0/4088927642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff700ce2b0 0x7eff700b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.328+0000 7eff78389640 1 --2- 172.21.15.84:0/4088927642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff700ad920 0x7eff700c6af0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.328+0000 7eff78389640 1 --2- 172.21.15.84:0/4088927642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff7006a2e0 0x7eff700af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.328+0000 7eff78389640 1 -- 172.21.15.84:0/4088927642 >> 172.21.15.84:0/4088927642 conn(0x7eff700ce9b0 msgr2=0x7eff7006f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:12.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.328+0000 7eff78389640 1 -- 172.21.15.84:0/4088927642 shutdown_connections 2023-10-16T09:49:12.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.328+0000 7eff78389640 1 -- 172.21.15.84:0/4088927642 wait complete. 2023-10-16T09:49:12.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.343+0000 7f3903683640 1 -- 172.21.15.84:0/991990821 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f38dc0057d0 con 0x7f38fc0661c0 2023-10-16T09:49:12.345 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2023-10-16T09:49:12.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.344+0000 7f38f1ffb640 1 -- 172.21.15.84:0/991990821 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f38ec017020 con 0x7f38fc0661c0 2023-10-16T09:49:12.346 INFO:teuthology.orchestra.run.smithi084.stdout:55834574851 2023-10-16T09:49:12.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.345+0000 7fdb8cff9640 1 -- 172.21.15.84:0/1643732984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdb780057d0 con 0x7fdb980e78b0 2023-10-16T09:49:12.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.346+0000 7fdb8effd640 1 -- 172.21.15.84:0/1643732984 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdb94079fb0 con 0x7fdb980e78b0 2023-10-16T09:49:12.347 INFO:teuthology.orchestra.run.smithi084.stdout:55834574852 2023-10-16T09:49:12.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.347+0000 7f3903683640 1 -- 172.21.15.84:0/991990821 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f38e006ae70 msgr2=0x7f38e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:12.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.347+0000 7f3903683640 1 --2- 172.21.15.84:0/991990821 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f38e006ae70 0x7f38e006d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0b11f0 tx=0x7f38f8007de0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.347+0000 7f3903683640 1 -- 172.21.15.84:0/991990821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38fc0661c0 msgr2=0x7f38fc1ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:12.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.347+0000 7f3903683640 1 --2- 172.21.15.84:0/991990821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38fc0661c0 0x7f38fc1ff270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00c400 tx=0x7f38ec00c8c0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.348+0000 7f3903683640 1 -- 172.21.15.84:0/991990821 shutdown_connections 2023-10-16T09:49:12.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.348+0000 7f3903683640 1 --2- 172.21.15.84:0/991990821 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f38e006ae70 0x7f38e006d330 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.348+0000 7f3903683640 1 --2- 172.21.15.84:0/991990821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38fc15af80 0x7f38fc0cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.348+0000 7f3903683640 1 --2- 172.21.15.84:0/991990821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38fc0cb1c0 0x7f38fc0b5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.348+0000 7f3903683640 1 --2- 172.21.15.84:0/991990821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38fc0661c0 0x7f38fc1ff270 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00c400 tx=0x7f38ec00c8c0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.348+0000 7f3903683640 1 -- 172.21.15.84:0/991990821 >> 172.21.15.84:0/991990821 conn(0x7f38fc0d0ab0 msgr2=0x7f38fc15bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:12.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.348+0000 7f3903683640 1 -- 172.21.15.84:0/991990821 shutdown_connections 2023-10-16T09:49:12.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.348+0000 7f3903683640 1 -- 172.21.15.84:0/991990821 wait complete. 2023-10-16T09:49:12.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.348+0000 7fdb8cff9640 1 -- 172.21.15.84:0/1643732984 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdb8006aef0 msgr2=0x7fdb8006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:12.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.348+0000 7fdb8cff9640 1 --2- 172.21.15.84:0/1643732984 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdb8006aef0 0x7fdb8006d3b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdb90000f60 tx=0x7fdb90007e20 comp rx=0 tx=0).stop 2023-10-16T09:49:12.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.348+0000 7fdb8cff9640 1 -- 172.21.15.84:0/1643732984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb980e78b0 msgr2=0x7fdb9817c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:12.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.348+0000 7fdb8cff9640 1 --2- 172.21.15.84:0/1643732984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb980e78b0 0x7fdb9817c5f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009720 tx=0x7fdb940046c0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.348+0000 7fdb8cff9640 1 -- 172.21.15.84:0/1643732984 shutdown_connections 2023-10-16T09:49:12.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.348+0000 7fdb8cff9640 1 --2- 172.21.15.84:0/1643732984 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdb8006aef0 0x7fdb8006d3b0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdb90000f60 tx=0x7fdb90007e20 comp rx=0 tx=0).stop 2023-10-16T09:49:12.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.348+0000 7fdb8cff9640 1 --2- 172.21.15.84:0/1643732984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb980eb2c0 0x7fdb980e5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.348+0000 7fdb8cff9640 1 --2- 172.21.15.84:0/1643732984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb980e78b0 0x7fdb9817c5f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.348+0000 7fdb8cff9640 1 --2- 172.21.15.84:0/1643732984 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb98006910 0x7fdb980e96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.348+0000 7fdb8cff9640 1 -- 172.21.15.84:0/1643732984 >> 172.21.15.84:0/1643732984 conn(0x7fdb98054c10 msgr2=0x7fdb9804d240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:12.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.349+0000 7fdb8cff9640 1 -- 172.21.15.84:0/1643732984 shutdown_connections 2023-10-16T09:49:12.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.349+0000 7fdb8cff9640 1 -- 172.21.15.84:0/1643732984 wait complete. 2023-10-16T09:49:12.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.350+0000 7f2eb47ee640 1 -- 172.21.15.84:0/2497595141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2eac06c9e0 con 0x7f2eac158f60 2023-10-16T09:49:12.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.351+0000 7f2ea2ffd640 1 -- 172.21.15.84:0/2497595141 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2ea407a060 con 0x7f2eac158f60 2023-10-16T09:49:12.355 INFO:teuthology.orchestra.run.smithi084.stdout:55834574852 2023-10-16T09:49:12.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.354+0000 7f2eb47ee640 1 -- 172.21.15.84:0/2497595141 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2e9006aef0 msgr2=0x7f2e9006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:12.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.354+0000 7f2eb47ee640 1 --2- 172.21.15.84:0/2497595141 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2e9006aef0 0x7f2e9006d3b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8006fd0 tx=0x7f2ea8008040 comp rx=0 tx=0).stop 2023-10-16T09:49:12.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.354+0000 7f2eb47ee640 1 -- 172.21.15.84:0/2497595141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2eac158f60 msgr2=0x7f2eac06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:12.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.354+0000 7f2eb47ee640 1 --2- 172.21.15.84:0/2497595141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2eac158f60 0x7f2eac06fc90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4037040 tx=0x7f2ea4004a40 comp rx=0 tx=0).stop 2023-10-16T09:49:12.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.355+0000 7f2eb47ee640 1 -- 172.21.15.84:0/2497595141 shutdown_connections 2023-10-16T09:49:12.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.355+0000 7f2eb47ee640 1 --2- 172.21.15.84:0/2497595141 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2e9006aef0 0x7f2e9006d3b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.355+0000 7f2eb47ee640 1 --2- 172.21.15.84:0/2497595141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2eac158f60 0x7f2eac06fc90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.355+0000 7f2eb47ee640 1 --2- 172.21.15.84:0/2497595141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2eac0ce1c0 0x7f2eac06f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.355+0000 7f2eb47ee640 1 --2- 172.21.15.84:0/2497595141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2eac06a4b0 0x7f2eac152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.355+0000 7f2eb47ee640 1 -- 172.21.15.84:0/2497595141 >> 172.21.15.84:0/2497595141 conn(0x7f2eac0ce9b0 msgr2=0x7f2eac1539e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:12.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.355+0000 7f2eb47ee640 1 -- 172.21.15.84:0/2497595141 shutdown_connections 2023-10-16T09:49:12.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.355+0000 7f2eb47ee640 1 -- 172.21.15.84:0/2497595141 wait complete. 2023-10-16T09:49:12.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.367+0000 7ffb26c07640 1 -- 172.21.15.84:0/1901300169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ffb2015a8c0 con 0x7ffb200cf710 2023-10-16T09:49:12.369 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.1 2023-10-16T09:49:12.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.367+0000 7ffb1d7fa640 1 -- 172.21.15.84:0/1901300169 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffb2015a8c0 con 0x7ffb200cf710 2023-10-16T09:49:12.369 INFO:teuthology.orchestra.run.smithi084.stdout:55834574853 2023-10-16T09:49:12.370 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2023-10-16T09:49:12.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.370+0000 7ffb26c07640 1 -- 172.21.15.84:0/1901300169 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffb0c06aed0 msgr2=0x7ffb0c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:12.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.370+0000 7ffb26c07640 1 --2- 172.21.15.84:0/1901300169 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffb0c06aed0 0x7ffb0c06d390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffb10001360 tx=0x7ffb10009040 comp rx=0 tx=0).stop 2023-10-16T09:49:12.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.370+0000 7ffb26c07640 1 -- 172.21.15.84:0/1901300169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb200cf710 msgr2=0x7ffb2015aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:12.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.370+0000 7ffb26c07640 1 --2- 172.21.15.84:0/1901300169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb200cf710 0x7ffb2015aba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800efc0 tx=0x7ffb1800c490 comp rx=0 tx=0).stop 2023-10-16T09:49:12.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.370+0000 7ffb26c07640 1 -- 172.21.15.84:0/1901300169 shutdown_connections 2023-10-16T09:49:12.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.370+0000 7ffb26c07640 1 --2- 172.21.15.84:0/1901300169 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffb0c06aed0 0x7ffb0c06d390 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.370+0000 7ffb26c07640 1 --2- 172.21.15.84:0/1901300169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb200cfaf0 0x7ffb20159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.370+0000 7ffb26c07640 1 --2- 172.21.15.84:0/1901300169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb200cf710 0x7ffb2015aba0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.370+0000 7ffb26c07640 1 --2- 172.21.15.84:0/1901300169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb200b3420 0x7ffb200c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.370+0000 7ffb26c07640 1 -- 172.21.15.84:0/1901300169 >> 172.21.15.84:0/1901300169 conn(0x7ffb200d2cc0 msgr2=0x7ffb200b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:12.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.370+0000 7ffb26c07640 1 -- 172.21.15.84:0/1901300169 shutdown_connections 2023-10-16T09:49:12.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.370+0000 7ffb26c07640 1 -- 172.21.15.84:0/1901300169 wait complete. 2023-10-16T09:49:12.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.371+0000 7fc76b002640 1 -- 172.21.15.84:0/2107106810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fc7640c9610 con 0x7fc7641510b0 2023-10-16T09:49:12.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.372+0000 7fc7617fa640 1 -- 172.21.15.84:0/2107106810 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc75c0589b0 con 0x7fc7641510b0 2023-10-16T09:49:12.375 INFO:teuthology.orchestra.run.smithi084.stdout:55834574852 2023-10-16T09:49:12.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.374+0000 7fc76b002640 1 -- 172.21.15.84:0/2107106810 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc74406aec0 msgr2=0x7fc74406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:12.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.374+0000 7fc76b002640 1 --2- 172.21.15.84:0/2107106810 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc74406aec0 0x7fc74406d380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc74c007fd0 tx=0x7fc74c001df0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.374+0000 7fc76b002640 1 -- 172.21.15.84:0/2107106810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7641510b0 msgr2=0x7fc764151530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:12.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.374+0000 7fc76b002640 1 --2- 172.21.15.84:0/2107106810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7641510b0 0x7fc764151530 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc7640cc510 tx=0x7fc75c0085d0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.375+0000 7fc76b002640 1 -- 172.21.15.84:0/2107106810 shutdown_connections 2023-10-16T09:49:12.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.375+0000 7fc76b002640 1 --2- 172.21.15.84:0/2107106810 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc74406aec0 0x7fc74406d380 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.375+0000 7fc76b002640 1 --2- 172.21.15.84:0/2107106810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc764152eb0 0x7fc764153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.375+0000 7fc76b002640 1 --2- 172.21.15.84:0/2107106810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7641510b0 0x7fc764151530 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.375+0000 7fc76b002640 1 --2- 172.21.15.84:0/2107106810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc750005180 0x7fc7641edc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.375+0000 7fc76b002640 1 -- 172.21.15.84:0/2107106810 >> 172.21.15.84:0/2107106810 conn(0x7fc7640cfac0 msgr2=0x7fc76406ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:12.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.375+0000 7fc76b002640 1 -- 172.21.15.84:0/2107106810 shutdown_connections 2023-10-16T09:49:12.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.375+0000 7fc76b002640 1 -- 172.21.15.84:0/2107106810 wait complete. 2023-10-16T09:49:12.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2023-10-16T09:49:12.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.381+0000 7f7d83fff640 1 -- 172.21.15.84:0/3299918574 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7d7c005fd0 con 0x7f7d7c0eb1b0 2023-10-16T09:49:12.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.382+0000 7f7d6bfff640 1 -- 172.21.15.84:0/3299918574 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7d8408e090 con 0x7f7d7c0eb1b0 2023-10-16T09:49:12.383 INFO:teuthology.orchestra.run.smithi084.stdout:55834574852 2023-10-16T09:49:12.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.385+0000 7f7d83fff640 1 -- 172.21.15.84:0/3299918574 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7d7006aec0 msgr2=0x7f7d7006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:12.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.385+0000 7f7d83fff640 1 --2- 172.21.15.84:0/3299918574 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7d7006aec0 0x7f7d7006d380 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c0536b0 tx=0x7f7d74009190 comp rx=0 tx=0).stop 2023-10-16T09:49:12.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.385+0000 7f7d83fff640 1 -- 172.21.15.84:0/3299918574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d7c0eb1b0 msgr2=0x7f7d7c0ddf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:12.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.385+0000 7f7d83fff640 1 --2- 172.21.15.84:0/3299918574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d7c0eb1b0 0x7f7d7c0ddf70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d84000b60 tx=0x7f7d840671b0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.385+0000 7f7d83fff640 1 -- 172.21.15.84:0/3299918574 shutdown_connections 2023-10-16T09:49:12.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.385+0000 7f7d83fff640 1 --2- 172.21.15.84:0/3299918574 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7d7006aec0 0x7f7d7006d380 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.385+0000 7f7d83fff640 1 --2- 172.21.15.84:0/3299918574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d7c0eb1b0 0x7f7d7c0ddf70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.385+0000 7f7d83fff640 1 --2- 172.21.15.84:0/3299918574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d7c0e77a0 0x7f7d7c0dda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.385+0000 7f7d83fff640 1 --2- 172.21.15.84:0/3299918574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d7c052ed0 0x7f7d7c17a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:12.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.385+0000 7f7d83fff640 1 -- 172.21.15.84:0/3299918574 >> 172.21.15.84:0/3299918574 conn(0x7f7d7c054b00 msgr2=0x7f7d7c04d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:12.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.385+0000 7f7d83fff640 1 -- 172.21.15.84:0/3299918574 shutdown_connections 2023-10-16T09:49:12.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:12.385+0000 7f7d83fff640 1 -- 172.21.15.84:0/3299918574 wait complete. 2023-10-16T09:49:12.392 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2023-10-16T09:49:12.393 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2023-10-16T09:49:12.403 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2023-10-16T09:49:13.277 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-16T09:49:13.346 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-16T09:49:13.370 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-16T09:49:13.371 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-16T09:49:13.381 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-16T09:49:13.393 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-16T09:49:13.394 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-16T09:49:13.403 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-16T09:49:13.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.528+0000 7f00f621f640 1 Processor -- start 2023-10-16T09:49:13.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.528+0000 7f00f621f640 1 -- start start 2023-10-16T09:49:13.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.530+0000 7f00f621f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00f00ce1c0 0x7f00f00ade50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.531+0000 7f00ef7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f00f0158f60 0x7f00f0069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:57512/0 (socket says 172.21.15.84:57512) 2023-10-16T09:49:13.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.531+0000 7f00ef7fe640 1 -- 172.21.15.84:0/465456006 learned_addr learned my addr 172.21.15.84:0/465456006 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:13.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.531+0000 7f00f621f640 1 -- 172.21.15.84:0/465456006 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00f006cf50 con 0x7f00f006a4b0 2023-10-16T09:49:13.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.531+0000 7f00f621f640 1 -- 172.21.15.84:0/465456006 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00f015ccf0 con 0x7f00f0158f60 2023-10-16T09:49:13.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.532+0000 7f00f621f640 1 -- 172.21.15.84:0/465456006 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f015d970 con 0x7f00f00ce1c0 2023-10-16T09:49:13.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.532+0000 7f00effff640 1 --2- 172.21.15.84:0/465456006 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00f00ce1c0 0x7f00f00ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.532+0000 7f00ee7fc640 1 -- 172.21.15.84:0/465456006 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912429548 0 0) 0x7f00f015ccf0 con 0x7f00f0158f60 2023-10-16T09:49:13.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.532+0000 7f00ee7fc640 1 -- 172.21.15.84:0/465456006 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00d8003440 con 0x7f00f0158f60 2023-10-16T09:49:13.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.533+0000 7f00effff640 1 -- 172.21.15.84:0/465456006 >> v1:172.21.15.157:6789/0 conn(0x7f00f0158f60 legacy=0x7f00f0069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.534+0000 7f00ee7fc640 1 -- 172.21.15.84:0/465456006 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 902754825 0 0) 0x7f00d8003440 con 0x7f00f0158f60 2023-10-16T09:49:13.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.534+0000 7f00effff640 1 -- 172.21.15.84:0/465456006 >> v1:172.21.15.84:6789/0 conn(0x7f00f006a4b0 legacy=0x7f00f00cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.534+0000 7f00effff640 1 -- 172.21.15.84:0/465456006 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f015cfd0 con 0x7f00f00ce1c0 2023-10-16T09:49:13.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.536+0000 7f00effff640 1 --2- 172.21.15.84:0/465456006 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00f00ce1c0 0x7f00f00ade50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009dc0 tx=0x7f00e802bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9da3487f1f4ea099 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.536+0000 7f00ee7fc640 1 -- 172.21.15.84:0/465456006 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e4001670 con 0x7f00f00ce1c0 2023-10-16T09:49:13.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.538+0000 7f00ee7fc640 1 -- 172.21.15.84:0/465456006 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e802c810 con 0x7f00f00ce1c0 2023-10-16T09:49:13.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.538+0000 7f00ee7fc640 1 -- 172.21.15.84:0/465456006 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e802cb80 con 0x7f00f00ce1c0 2023-10-16T09:49:13.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.539+0000 7f00f621f640 1 -- 172.21.15.84:0/465456006 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00f00ce1c0 msgr2=0x7f00f00ade50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.539+0000 7f00f621f640 1 --2- 172.21.15.84:0/465456006 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00f00ce1c0 0x7f00f00ade50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009dc0 tx=0x7f00e802bce0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.539+0000 7f00f621f640 1 -- 172.21.15.84:0/465456006 shutdown_connections 2023-10-16T09:49:13.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.539+0000 7f00f621f640 1 --2- 172.21.15.84:0/465456006 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00f00ce1c0 0x7f00f00ade50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.539+0000 7f00f621f640 1 -- 172.21.15.84:0/465456006 >> 172.21.15.84:0/465456006 conn(0x7f00f00ce9b0 msgr2=0x7f00f00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:13.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.539+0000 7f00f621f640 1 -- 172.21.15.84:0/465456006 shutdown_connections 2023-10-16T09:49:13.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.539+0000 7f00f621f640 1 -- 172.21.15.84:0/465456006 wait complete. 2023-10-16T09:49:13.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.543+0000 7f00f621f640 1 Processor -- start 2023-10-16T09:49:13.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.543+0000 7f00f621f640 1 -- start start 2023-10-16T09:49:13.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.545+0000 7f00f621f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00f006a4b0 0x7f00f0159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.545+0000 7f00f621f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00f00ce1c0 0x7f00f01514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.546+0000 7f00ef7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00f006a4b0 0x7f00f0159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.546+0000 7f00ef7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00f006a4b0 0x7f00f0159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:37076/0 (socket says 172.21.15.84:37076) 2023-10-16T09:49:13.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.547+0000 7f00eeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00f00ce1c0 0x7f00f01514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.547+0000 7f00ef7fe640 1 -- 172.21.15.84:0/1772082248 learned_addr learned my addr 172.21.15.84:0/1772082248 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:13.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.547+0000 7f00f621f640 1 --2- 172.21.15.84:0/1772082248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00f0158f60 0x7f00f015a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.547+0000 7f00f621f640 1 -- 172.21.15.84:0/1772082248 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f00afbe0 con 0x7f00f00ce1c0 2023-10-16T09:49:13.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.547+0000 7f00f621f640 1 -- 172.21.15.84:0/1772082248 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f0159620 con 0x7f00f0158f60 2023-10-16T09:49:13.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.548+0000 7f00f621f640 1 -- 172.21.15.84:0/1772082248 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f00afe30 con 0x7f00f006a4b0 2023-10-16T09:49:13.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.548+0000 7f00effff640 1 --2- 172.21.15.84:0/1772082248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00f0158f60 0x7f00f015a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.549+0000 7f00eeffd640 1 -- 172.21.15.84:0/1772082248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00f006a4b0 msgr2=0x7f00f0159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.549+0000 7f00eeffd640 1 --2- 172.21.15.84:0/1772082248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00f006a4b0 0x7f00f0159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.549+0000 7f00eeffd640 1 -- 172.21.15.84:0/1772082248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00f0158f60 msgr2=0x7f00f015a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.549+0000 7f00eeffd640 1 --2- 172.21.15.84:0/1772082248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00f0158f60 0x7f00f015a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.549+0000 7f00eeffd640 1 -- 172.21.15.84:0/1772082248 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e8009a70 con 0x7f00f00ce1c0 2023-10-16T09:49:13.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.550+0000 7f00eeffd640 1 --2- 172.21.15.84:0/1772082248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00f00ce1c0 0x7f00f01514c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f00f00ad680 tx=0x7f00e000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.551+0000 7f00d7fff640 1 -- 172.21.15.84:0/1772082248 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e00052d0 con 0x7f00f00ce1c0 2023-10-16T09:49:13.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.551+0000 7f00d7fff640 1 -- 172.21.15.84:0/1772082248 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e0005430 con 0x7f00f00ce1c0 2023-10-16T09:49:13.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.551+0000 7f00f621f640 1 -- 172.21.15.84:0/1772082248 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f00b0110 con 0x7f00f00ce1c0 2023-10-16T09:49:13.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.551+0000 7f00f621f640 1 -- 172.21.15.84:0/1772082248 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f0151a00 con 0x7f00f00ce1c0 2023-10-16T09:49:13.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.552+0000 7f00d7fff640 1 -- 172.21.15.84:0/1772082248 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e0003830 con 0x7f00f00ce1c0 2023-10-16T09:49:13.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.553+0000 7f00f621f640 1 -- 172.21.15.84:0/1772082248 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f0159b00 con 0x7f00f00ce1c0 2023-10-16T09:49:13.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.555+0000 7f00d7fff640 1 -- 172.21.15.84:0/1772082248 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f00e0003990 con 0x7f00f00ce1c0 2023-10-16T09:49:13.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.556+0000 7f00d7fff640 1 --2- 172.21.15.84:0/1772082248 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f00dc06ae70 0x7f00dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.557+0000 7f00d7fff640 1 -- 172.21.15.84:0/1772082248 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00e0088b80 con 0x7f00f00ce1c0 2023-10-16T09:49:13.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.557+0000 7f00ef7fe640 1 --2- 172.21.15.84:0/1772082248 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f00dc06ae70 0x7f00dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.559+0000 7f00ef7fe640 1 --2- 172.21.15.84:0/1772082248 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f00dc06ae70 0x7f00dc06d330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f00e40017a0 tx=0x7f00e40016f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.567+0000 7f00d7fff640 1 -- 172.21.15.84:0/1772082248 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f00e0053c40 con 0x7f00f00ce1c0 2023-10-16T09:49:13.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.610+0000 7fde8b59e640 1 Processor -- start 2023-10-16T09:49:13.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.610+0000 7fde8b59e640 1 -- start start 2023-10-16T09:49:13.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.612+0000 7fde8b59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fde8c0c9850 0x7fde8c0cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.613+0000 7fde8b59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8c06f440 0x7fde8c0c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.613+0000 7fde8b59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fde8c0ca170 0x7fde8c06a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.613+0000 7fde8a59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fde8c0c9850 0x7fde8c0cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.613+0000 7fde8a59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fde8c0c9850 0x7fde8c0cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:55286/0 (socket says 172.21.15.84:55286) 2023-10-16T09:49:13.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.613+0000 7fde8a59c640 1 -- 172.21.15.84:0/3251420786 learned_addr learned my addr 172.21.15.84:0/3251420786 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:13.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.614+0000 7fde8b59e640 1 -- 172.21.15.84:0/3251420786 --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8c0cdbc0 con 0x7fde8c0ca170 2023-10-16T09:49:13.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.614+0000 7fde8b59e640 1 -- 172.21.15.84:0/3251420786 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8c0cc5e0 con 0x7fde8c0c9850 2023-10-16T09:49:13.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.614+0000 7fde89d9b640 1 --2- 172.21.15.84:0/3251420786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8c06f440 0x7fde8c0c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.614+0000 7fde8b59e640 1 -- 172.21.15.84:0/3251420786 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c0cc8c0 con 0x7fde8c06f440 2023-10-16T09:49:13.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.614+0000 7fde8ad9d640 1 --2- 172.21.15.84:0/3251420786 >> v2:172.21.15.84:3300/0 conn(0x7fde8c0ca170 0x7fde8c06a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.615+0000 7fde8a59c640 1 -- 172.21.15.84:0/3251420786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8c06f440 msgr2=0x7fde8c0c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:13.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.615+0000 7fde8a59c640 1 --2- 172.21.15.84:0/3251420786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8c06f440 0x7fde8c0c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.615+0000 7fde8a59c640 1 -- 172.21.15.84:0/3251420786 >> v2:172.21.15.84:3300/0 conn(0x7fde8c0ca170 msgr2=0x7fde8c06a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:13.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.618+0000 7fde8a59c640 1 --2- 172.21.15.84:0/3251420786 >> v2:172.21.15.84:3300/0 conn(0x7fde8c0ca170 0x7fde8c06a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.618+0000 7fde8a59c640 1 -- 172.21.15.84:0/3251420786 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8c0ccba0 con 0x7fde8c0c9850 2023-10-16T09:49:13.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.618+0000 7fde8ad9d640 1 --2- 172.21.15.84:0/3251420786 >> v2:172.21.15.84:3300/0 conn(0x7fde8c0ca170 0x7fde8c06a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:49:13.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.620+0000 7fde8a59c640 1 --2- 172.21.15.84:0/3251420786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde8c0c9850 0x7fde8c0cd680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fde8c0cf6f0 tx=0x7fde8002b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adecfccc8be07835 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.620+0000 7fde8959a640 1 -- 172.21.15.84:0/3251420786 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde80007ce0 con 0x7fde8c0c9850 2023-10-16T09:49:13.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.621+0000 7fde8959a640 1 -- 172.21.15.84:0/3251420786 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde80007e40 con 0x7fde8c0c9850 2023-10-16T09:49:13.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.621+0000 7fde8959a640 1 -- 172.21.15.84:0/3251420786 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde800092f0 con 0x7fde8c0c9850 2023-10-16T09:49:13.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.622+0000 7fde8b59e640 1 -- 172.21.15.84:0/3251420786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde8c0c9850 msgr2=0x7fde8c0cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.622+0000 7fde8b59e640 1 --2- 172.21.15.84:0/3251420786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde8c0c9850 0x7fde8c0cd680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fde8c0cf6f0 tx=0x7fde8002b930 comp rx=0 tx=0).stop 2023-10-16T09:49:13.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.622+0000 7fde8b59e640 1 -- 172.21.15.84:0/3251420786 shutdown_connections 2023-10-16T09:49:13.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.622+0000 7fde8b59e640 1 --2- 172.21.15.84:0/3251420786 >> v2:172.21.15.84:3300/0 conn(0x7fde8c0ca170 0x7fde8c06a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.622+0000 7fde8b59e640 1 --2- 172.21.15.84:0/3251420786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8c06f440 0x7fde8c0c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.622+0000 7fde8b59e640 1 --2- 172.21.15.84:0/3251420786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde8c0c9850 0x7fde8c0cd680 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.622+0000 7fde8b59e640 1 -- 172.21.15.84:0/3251420786 >> 172.21.15.84:0/3251420786 conn(0x7fde8c0ce6a0 msgr2=0x7fde8c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:13.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.622+0000 7fde8b59e640 1 -- 172.21.15.84:0/3251420786 shutdown_connections 2023-10-16T09:49:13.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.622+0000 7fde8b59e640 1 -- 172.21.15.84:0/3251420786 wait complete. 2023-10-16T09:49:13.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.625+0000 7fde8b59e640 1 Processor -- start 2023-10-16T09:49:13.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.626+0000 7fde8b59e640 1 -- start start 2023-10-16T09:49:13.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.628+0000 7fde8b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde8c06f440 0x7fde8c1ff840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.629+0000 7fde8b59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8c0c9850 0x7fde8c0c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.629+0000 7fde8b59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde8c0ca170 0x7fde8c1fe260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.629+0000 7fde8b59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c1ffd80 con 0x7fde8c06f440 2023-10-16T09:49:13.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.629+0000 7fde8b59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c1e4760 con 0x7fde8c0ca170 2023-10-16T09:49:13.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.629+0000 7fde8b59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c1e4a00 con 0x7fde8c0c9850 2023-10-16T09:49:13.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.630+0000 7fde89d9b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8c0c9850 0x7fde8c0c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.630+0000 7fde8ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde8c0ca170 0x7fde8c1fe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.630+0000 7fde8ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde8c0ca170 0x7fde8c1fe260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:55300/0 (socket says 172.21.15.84:55300) 2023-10-16T09:49:13.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.630+0000 7fde8ad9d640 1 -- 172.21.15.84:0/694718665 learned_addr learned my addr 172.21.15.84:0/694718665 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:13.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.630+0000 7fde8a59c640 1 --2- 172.21.15.84:0/694718665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde8c06f440 0x7fde8c1ff840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.631+0000 7fde8ad9d640 1 -- 172.21.15.84:0/694718665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8c0c9850 msgr2=0x7fde8c0c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.631+0000 7fde8ad9d640 1 --2- 172.21.15.84:0/694718665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8c0c9850 0x7fde8c0c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.631+0000 7fde8ad9d640 1 -- 172.21.15.84:0/694718665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde8c06f440 msgr2=0x7fde8c1ff840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.631+0000 7fde8ad9d640 1 --2- 172.21.15.84:0/694718665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde8c06f440 0x7fde8c1ff840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.631+0000 7fde8ad9d640 1 -- 172.21.15.84:0/694718665 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde84009a70 con 0x7fde8c0ca170 2023-10-16T09:49:13.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.631+0000 7fde8a59c640 1 --2- 172.21.15.84:0/694718665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde8c06f440 0x7fde8c1ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:13.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.632+0000 7fde8ad9d640 1 --2- 172.21.15.84:0/694718665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde8c0ca170 0x7fde8c1fe260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fde8400e580 tx=0x7fde8400ea40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.632+0000 7fde7affd640 1 -- 172.21.15.84:0/694718665 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84003990 con 0x7fde8c0ca170 2023-10-16T09:49:13.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.632+0000 7fde7affd640 1 -- 172.21.15.84:0/694718665 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde84003af0 con 0x7fde8c0ca170 2023-10-16T09:49:13.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.633+0000 7fde7affd640 1 -- 172.21.15.84:0/694718665 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84003e60 con 0x7fde8c0ca170 2023-10-16T09:49:13.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.634+0000 7fde8b59e640 1 -- 172.21.15.84:0/694718665 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde80009df0 con 0x7fde8c0ca170 2023-10-16T09:49:13.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.634+0000 7fde8b59e640 1 -- 172.21.15.84:0/694718665 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8c1e4fc0 con 0x7fde8c0ca170 2023-10-16T09:49:13.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.636+0000 7fde8b59e640 1 -- 172.21.15.84:0/694718665 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8c065dc0 con 0x7fde8c0ca170 2023-10-16T09:49:13.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.638+0000 7fde7affd640 1 -- 172.21.15.84:0/694718665 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fde84010680 con 0x7fde8c0ca170 2023-10-16T09:49:13.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.639+0000 7fde7affd640 1 --2- 172.21.15.84:0/694718665 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fde6c06aec0 0x7fde6c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.639+0000 7fde7affd640 1 -- 172.21.15.84:0/694718665 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde8408a620 con 0x7fde8c0ca170 2023-10-16T09:49:13.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.640+0000 7fde8a59c640 1 --2- 172.21.15.84:0/694718665 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fde6c06aec0 0x7fde6c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.641+0000 7fde8a59c640 1 --2- 172.21.15.84:0/694718665 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fde6c06aec0 0x7fde6c06d380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fde80035000 tx=0x7fde80009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.654+0000 7fde7affd640 1 -- 172.21.15.84:0/694718665 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fde840556c0 con 0x7fde8c0ca170 2023-10-16T09:49:13.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.663+0000 7f6b32c0e640 1 Processor -- start 2023-10-16T09:49:13.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.663+0000 7f6b32c0e640 1 -- start start 2023-10-16T09:49:13.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.666+0000 7f6b32c0e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b2c0b20f0 0x7f6b2c065ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.666+0000 7f6b32c0e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6b2c0b5150 0x7f6b2c15c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.666+0000 7f6b32c0e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2c0cd790 con 0x7f6b2c065ff0 2023-10-16T09:49:13.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.666+0000 7f6b32c0e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2c0ce310 con 0x7f6b2c0b5150 2023-10-16T09:49:13.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.666+0000 7f6b32c0e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c066430 con 0x7f6b2c0b20f0 2023-10-16T09:49:13.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.667+0000 7f6b2bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6b2c065ff0 0x7f6b2c0b4a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32944/0 (socket says 172.21.15.84:32944) 2023-10-16T09:49:13.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.667+0000 7f6b311ab640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6b2c0b5150 0x7f6b2c15c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.668+0000 7f6b311ab640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6b2c0b5150 0x7f6b2c15c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59846/0 (socket says 172.21.15.84:59846) 2023-10-16T09:49:13.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.668+0000 7f6c363c9640 1 Processor -- start 2023-10-16T09:49:13.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.668+0000 7f6b2bfff640 1 -- 172.21.15.84:0/692196721 learned_addr learned my addr 172.21.15.84:0/692196721 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:13.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.668+0000 7f6c363c9640 1 -- start start 2023-10-16T09:49:13.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.669+0000 7f6b309aa640 1 --2- 172.21.15.84:0/692196721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b2c0b20f0 0x7f6b2c065ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.670+0000 7f6c363c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c3015d410 0x7f6c300d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.670+0000 7f6c363c9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6c300cd260 0x7f6c300cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.670+0000 7f6c363c9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c300b5d90 con 0x7f6c3015d030 2023-10-16T09:49:13.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.670+0000 7f6c363c9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c300d2b50 con 0x7f6c300cd260 2023-10-16T09:49:13.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.670+0000 7f6c363c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c30160fc0 con 0x7f6c3015d410 2023-10-16T09:49:13.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.671+0000 7f6c2f7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c3015d410 0x7f6c300d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.671+0000 7f6c34966640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6c300cd260 0x7f6c300cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.671+0000 7f6c34966640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6c300cd260 0x7f6c300cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59862/0 (socket says 172.21.15.84:59862) 2023-10-16T09:49:13.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.671+0000 7f6c34966640 1 -- 172.21.15.84:0/2878044568 learned_addr learned my addr 172.21.15.84:0/2878044568 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:13.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.669+0000 7f6b311ab640 1 -- 172.21.15.84:0/692196721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b2c0b20f0 msgr2=0x7f6b2c065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:13.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.669+0000 7f6b311ab640 1 --2- 172.21.15.84:0/692196721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b2c0b20f0 0x7f6b2c065ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.670+0000 7f6b311ab640 1 -- 172.21.15.84:0/692196721 >> v1:172.21.15.84:6789/0 conn(0x7f6b2c065ff0 legacy=0x7f6b2c0b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.670+0000 7f6b311ab640 1 -- 172.21.15.84:0/692196721 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c0b5580 con 0x7f6b2c0b5150 2023-10-16T09:49:13.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.672+0000 7fac49187640 1 Processor -- start 2023-10-16T09:49:13.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.672+0000 7f6b311ab640 1 --2- 172.21.15.84:0/692196721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b2c0b5150 0x7f6b2c15c960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009d90 tx=0x7f6b1c02bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fac2a340164803 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.672+0000 7fac49187640 1 -- start start 2023-10-16T09:49:13.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.673+0000 7f6b2b7fe640 1 -- 172.21.15.84:0/692196721 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c02c980 con 0x7f6b2c0b5150 2023-10-16T09:49:13.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.673+0000 7f6b2b7fe640 1 -- 172.21.15.84:0/692196721 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1c02cae0 con 0x7f6b2c0b5150 2023-10-16T09:49:13.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.672+0000 7f6c34966640 1 -- 172.21.15.84:0/2878044568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c3015d410 msgr2=0x7f6c300d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:13.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.672+0000 7f6c34966640 1 --2- 172.21.15.84:0/2878044568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c3015d410 0x7f6c300d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.672+0000 7f6c34966640 1 -- 172.21.15.84:0/2878044568 >> v1:172.21.15.157:6789/0 conn(0x7f6c3015d030 legacy=0x7f6c30161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:13.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.672+0000 7f6c34966640 1 -- 172.21.15.84:0/2878044568 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c300d25d0 con 0x7f6c300cd260 2023-10-16T09:49:13.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.673+0000 7f6b2b7fe640 1 -- 172.21.15.84:0/692196721 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c02ce70 con 0x7f6b2c0b5150 2023-10-16T09:49:13.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.674+0000 7f6c34966640 1 --2- 172.21.15.84:0/2878044568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c300cd260 0x7f6c300cd6f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009870 tx=0x7f6c2402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ef3b24106b39e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.675+0000 7fac49187640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fac3c0df320 0x7fac3c0060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.674+0000 7f6c2effd640 1 -- 172.21.15.84:0/2878044568 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24030860 con 0x7f6c300cd260 2023-10-16T09:49:13.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.675+0000 7f6c2effd640 1 -- 172.21.15.84:0/2878044568 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c240309c0 con 0x7f6c300cd260 2023-10-16T09:49:13.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.675+0000 7f6c2effd640 1 -- 172.21.15.84:0/2878044568 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24030d30 con 0x7f6c300cd260 2023-10-16T09:49:13.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.675+0000 7fac49187640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac3c0e2e10 0x7fac3c0e32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.675+0000 7fac49187640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac3c0df700 con 0x7fac3c0077e0 2023-10-16T09:49:13.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.675+0000 7f6b32c0e640 1 -- 172.21.15.84:0/692196721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b2c0b5150 msgr2=0x7f6b2c15c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.675+0000 7f6b32c0e640 1 --2- 172.21.15.84:0/692196721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b2c0b5150 0x7f6b2c15c960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009d90 tx=0x7f6b1c02bc50 comp rx=0 tx=0).stop 2023-10-16T09:49:13.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.675+0000 7fac49187640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac3c0e37e0 con 0x7fac3c0df320 2023-10-16T09:49:13.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.675+0000 7fac49187640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c039170 con 0x7fac3c0e2e10 2023-10-16T09:49:13.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.675+0000 7fac43fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac3c0e2e10 0x7fac3c0e32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.676+0000 7f6c363c9640 1 -- 172.21.15.84:0/2878044568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c300cd260 msgr2=0x7f6c300cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.676+0000 7f6c363c9640 1 --2- 172.21.15.84:0/2878044568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c300cd260 0x7f6c300cd6f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009870 tx=0x7f6c2402b9a0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.675+0000 7fac42ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fac3c0077e0 0x7fac3c053fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:57534/0 (socket says 172.21.15.84:57534) 2023-10-16T09:49:13.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.675+0000 7fac42ffd640 1 -- 172.21.15.84:0/808704342 learned_addr learned my addr 172.21.15.84:0/808704342 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:13.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.676+0000 7fac437fe640 1 --2- 172.21.15.84:0/808704342 >> v2:172.21.15.157:3300/0 conn(0x7fac3c0df320 0x7fac3c0060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.676+0000 7f6b32c0e640 1 -- 172.21.15.84:0/692196721 shutdown_connections 2023-10-16T09:49:13.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.676+0000 7f6b32c0e640 1 --2- 172.21.15.84:0/692196721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b2c0b5150 0x7f6b2c15c960 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.676+0000 7f6b32c0e640 1 --2- 172.21.15.84:0/692196721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b2c0b20f0 0x7f6b2c065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.676+0000 7f6b32c0e640 1 -- 172.21.15.84:0/692196721 >> 172.21.15.84:0/692196721 conn(0x7f6b2c0d2d10 msgr2=0x7f6b2c0d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:13.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.676+0000 7f6c363c9640 1 -- 172.21.15.84:0/2878044568 shutdown_connections 2023-10-16T09:49:13.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.676+0000 7f6c363c9640 1 --2- 172.21.15.84:0/2878044568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c300cd260 0x7f6c300cd6f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.676+0000 7f6c363c9640 1 --2- 172.21.15.84:0/2878044568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c3015d410 0x7f6c300d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.676+0000 7f6c363c9640 1 -- 172.21.15.84:0/2878044568 >> 172.21.15.84:0/2878044568 conn(0x7f6c300cf720 msgr2=0x7f6c300cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:13.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.676+0000 7f6c363c9640 1 -- 172.21.15.84:0/2878044568 shutdown_connections 2023-10-16T09:49:13.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.676+0000 7f6c363c9640 1 -- 172.21.15.84:0/2878044568 wait complete. 2023-10-16T09:49:13.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.677+0000 7f6b32c0e640 1 -- 172.21.15.84:0/692196721 shutdown_connections 2023-10-16T09:49:13.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.677+0000 7f6b32c0e640 1 -- 172.21.15.84:0/692196721 wait complete. 2023-10-16T09:49:13.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.677+0000 7fac4a3e9640 1 -- 172.21.15.84:0/808704342 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525468326 0 0) 0x7fac3c0df700 con 0x7fac3c0077e0 2023-10-16T09:49:13.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.677+0000 7fac4a3e9640 1 -- 172.21.15.84:0/808704342 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac34003520 con 0x7fac3c0077e0 2023-10-16T09:49:13.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.677+0000 7fac437fe640 1 -- 172.21.15.84:0/808704342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac3c0e2e10 msgr2=0x7fac3c0e32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:13.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.677+0000 7fac437fe640 1 --2- 172.21.15.84:0/808704342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac3c0e2e10 0x7fac3c0e32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.677+0000 7fac437fe640 1 -- 172.21.15.84:0/808704342 >> v1:172.21.15.157:6789/0 conn(0x7fac3c0077e0 legacy=0x7fac3c053fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.677+0000 7fac437fe640 1 -- 172.21.15.84:0/808704342 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3c006740 con 0x7fac3c0df320 2023-10-16T09:49:13.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.679+0000 7fac437fe640 1 --2- 172.21.15.84:0/808704342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac3c0df320 0x7fac3c0060b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fac4404f940 tx=0x7fac440a7d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15b87417aaba2edf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.679+0000 7f6c363c9640 1 Processor -- start 2023-10-16T09:49:13.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.679+0000 7f6c363c9640 1 -- start start 2023-10-16T09:49:13.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.680+0000 7fac4a3e9640 1 -- 172.21.15.84:0/808704342 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac440a8be0 con 0x7fac3c0df320 2023-10-16T09:49:13.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.680+0000 7fac4a3e9640 1 -- 172.21.15.84:0/808704342 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac440a8d80 con 0x7fac3c0df320 2023-10-16T09:49:13.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.680+0000 7fac4a3e9640 1 -- 172.21.15.84:0/808704342 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac440b0640 con 0x7fac3c0df320 2023-10-16T09:49:13.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.681+0000 7f6b32c0e640 1 Processor -- start 2023-10-16T09:49:13.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.681+0000 7f6c363c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c300cd260 0x7f6c300c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.682+0000 7f6c363c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c3015d030 0x7f6c30065d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.681+0000 7f6b32c0e640 1 -- start start 2023-10-16T09:49:13.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.682+0000 7fac49187640 1 -- 172.21.15.84:0/808704342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac3c0df320 msgr2=0x7fac3c0060b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.682+0000 7fac49187640 1 --2- 172.21.15.84:0/808704342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac3c0df320 0x7fac3c0060b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fac4404f940 tx=0x7fac440a7d70 comp rx=0 tx=0).stop 2023-10-16T09:49:13.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.682+0000 7f6c363c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c3015d410 0x7f6c300cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.682+0000 7f6c2f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c3015d030 0x7f6c30065d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.682+0000 7f6b32c0e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b2c065ff0 0x7f6b2c161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.682+0000 7f6b32c0e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b2c0b20f0 0x7f6b2c0ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.682+0000 7f6b32c0e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b2c0b5150 0x7f6b2c0cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.682+0000 7f6b32c0e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0cb6b0 con 0x7f6b2c065ff0 2023-10-16T09:49:13.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.682+0000 7f6b32c0e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0cada0 con 0x7f6b2c0b5150 2023-10-16T09:49:13.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.682+0000 7f6b32c0e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c1624f0 con 0x7f6b2c0b20f0 2023-10-16T09:49:13.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.683+0000 7f6b309aa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b2c065ff0 0x7f6b2c161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.682+0000 7f6c2ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c300cd260 0x7f6c300c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.682+0000 7f6c2f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c3015d030 0x7f6c30065d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59872/0 (socket says 172.21.15.84:59872) 2023-10-16T09:49:13.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.683+0000 7f6c2ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c300cd260 0x7f6c300c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:55320/0 (socket says 172.21.15.84:55320) 2023-10-16T09:49:13.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.683+0000 7f6c363c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300c6a50 con 0x7f6c3015d030 2023-10-16T09:49:13.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.683+0000 7f6c2f7fe640 1 -- 172.21.15.84:0/1206184196 learned_addr learned my addr 172.21.15.84:0/1206184196 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:13.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.683+0000 7f6c363c9640 1 -- 172.21.15.84:0/1206184196 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3014f2c0 con 0x7f6c300cd260 2023-10-16T09:49:13.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.683+0000 7f6c363c9640 1 -- 172.21.15.84:0/1206184196 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300c6ca0 con 0x7f6c3015d410 2023-10-16T09:49:13.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.683+0000 7f6b2bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b2c0b20f0 0x7f6b2c0ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.683+0000 7f6b2bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b2c0b20f0 0x7f6b2c0ccb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:37134/0 (socket says 172.21.15.84:37134) 2023-10-16T09:49:13.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.683+0000 7f6b2bfff640 1 -- 172.21.15.84:0/1225528834 learned_addr learned my addr 172.21.15.84:0/1225528834 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:13.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.683+0000 7f6b311ab640 1 --2- 172.21.15.84:0/1225528834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b2c0b5150 0x7f6b2c0cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.683+0000 7f6b309aa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b2c065ff0 0x7f6b2c161fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59880/0 (socket says 172.21.15.84:59880) 2023-10-16T09:49:13.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.684+0000 7f6c34966640 1 --2- 172.21.15.84:0/1206184196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c3015d410 0x7f6c300cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.683+0000 7fac49187640 1 -- 172.21.15.84:0/808704342 shutdown_connections 2023-10-16T09:49:13.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.683+0000 7fac49187640 1 --2- 172.21.15.84:0/808704342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac3c0e2e10 0x7fac3c0e32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.683+0000 7fac49187640 1 --2- 172.21.15.84:0/808704342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac3c0df320 0x7fac3c0060b0 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fac4404f940 tx=0x7fac440a7d70 comp rx=0 tx=0).stop 2023-10-16T09:49:13.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.683+0000 7fac49187640 1 -- 172.21.15.84:0/808704342 >> 172.21.15.84:0/808704342 conn(0x7fac3c054ce0 msgr2=0x7fac3c0550f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:13.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.684+0000 7fac49187640 1 -- 172.21.15.84:0/808704342 shutdown_connections 2023-10-16T09:49:13.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.684+0000 7fac49187640 1 -- 172.21.15.84:0/808704342 wait complete. 2023-10-16T09:49:13.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.684+0000 7f6c2f7fe640 1 -- 172.21.15.84:0/1206184196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c3015d410 msgr2=0x7f6c300cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.684+0000 7f6c2f7fe640 1 --2- 172.21.15.84:0/1206184196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c3015d410 0x7f6c300cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.684+0000 7f6c2f7fe640 1 -- 172.21.15.84:0/1206184196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c300cd260 msgr2=0x7f6c300c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.684+0000 7f6c2f7fe640 1 --2- 172.21.15.84:0/1206184196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c300cd260 0x7f6c300c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.684+0000 7f6c2f7fe640 1 -- 172.21.15.84:0/1206184196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c240094d0 con 0x7f6c3015d030 2023-10-16T09:49:13.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.685+0000 7f6b309aa640 1 -- 172.21.15.84:0/1225528834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b2c0b20f0 msgr2=0x7f6b2c0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.685+0000 7f6b309aa640 1 --2- 172.21.15.84:0/1225528834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b2c0b20f0 0x7f6b2c0ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.685+0000 7f6b309aa640 1 -- 172.21.15.84:0/1225528834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b2c0b5150 msgr2=0x7f6b2c0cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.685+0000 7f6b309aa640 1 --2- 172.21.15.84:0/1225528834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b2c0b5150 0x7f6b2c0cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.685+0000 7f6b309aa640 1 -- 172.21.15.84:0/1225528834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c009a70 con 0x7f6b2c065ff0 2023-10-16T09:49:13.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.686+0000 7f6b309aa640 1 --2- 172.21.15.84:0/1225528834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b2c065ff0 0x7f6b2c161fb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400e8b0 tx=0x7f6b2400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.685+0000 7f6c2f7fe640 1 --2- 172.21.15.84:0/1206184196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c3015d030 0x7f6c30065d20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000e9b0 tx=0x7f6c2000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.686+0000 7f6c2cff9640 1 -- 172.21.15.84:0/1206184196 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20009670 con 0x7f6c3015d030 2023-10-16T09:49:13.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.686+0000 7f6b297fa640 1 -- 172.21.15.84:0/1225528834 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400c940 con 0x7f6b2c065ff0 2023-10-16T09:49:13.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.687+0000 7f6b297fa640 1 -- 172.21.15.84:0/1225528834 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2400caa0 con 0x7f6b2c065ff0 2023-10-16T09:49:13.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.687+0000 7f6b297fa640 1 -- 172.21.15.84:0/1225528834 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400c940 con 0x7f6b2c065ff0 2023-10-16T09:49:13.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.687+0000 7f6c2cff9640 1 -- 172.21.15.84:0/1206184196 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2000c9a0 con 0x7f6c3015d030 2023-10-16T09:49:13.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.687+0000 7f6c2cff9640 1 -- 172.21.15.84:0/1206184196 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000cd50 con 0x7f6c3015d030 2023-10-16T09:49:13.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.687+0000 7f6c363c9640 1 -- 172.21.15.84:0/1206184196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c300c6f80 con 0x7f6c3015d030 2023-10-16T09:49:13.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.687+0000 7f6b32c0e640 1 -- 172.21.15.84:0/1225528834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2c06f7b0 con 0x7f6b2c065ff0 2023-10-16T09:49:13.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.687+0000 7fac49187640 1 Processor -- start 2023-10-16T09:49:13.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.687+0000 7fac49187640 1 -- start start 2023-10-16T09:49:13.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.688+0000 7f6b32c0e640 1 -- 172.21.15.84:0/1225528834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c0cb2d0 con 0x7f6b2c065ff0 2023-10-16T09:49:13.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.690+0000 7fac49187640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac3c0077e0 0x7fac3c04acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.690+0000 7fac49187640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac3c0e2e10 0x7fac3c048a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.690+0000 7fac49187640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac3c16d460 0x7fac3c16d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.690+0000 7fac49187640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c04b210 con 0x7fac3c0e2e10 2023-10-16T09:49:13.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.690+0000 7fac49187640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c04a1c0 con 0x7fac3c0077e0 2023-10-16T09:49:13.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.690+0000 7fac49187640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c178ec0 con 0x7fac3c16d460 2023-10-16T09:49:13.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.687+0000 7f6c363c9640 1 -- 172.21.15.84:0/1206184196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c30150710 con 0x7f6c3015d030 2023-10-16T09:49:13.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.689+0000 7f6c363c9640 1 -- 172.21.15.84:0/1206184196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3004e160 con 0x7f6c3015d030 2023-10-16T09:49:13.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.692+0000 7fac42ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac3c0e2e10 0x7fac3c048a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.692+0000 7fac437fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac3c0077e0 0x7fac3c04acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.692+0000 7fac42ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac3c0e2e10 0x7fac3c048a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59894/0 (socket says 172.21.15.84:59894) 2023-10-16T09:49:13.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.692+0000 7fac42ffd640 1 -- 172.21.15.84:0/2976888441 learned_addr learned my addr 172.21.15.84:0/2976888441 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:13.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.692+0000 7f6c2cff9640 1 -- 172.21.15.84:0/1206184196 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6c20013470 con 0x7f6c3015d030 2023-10-16T09:49:13.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.692+0000 7fac42ffd640 1 -- 172.21.15.84:0/2976888441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac3c16d460 msgr2=0x7fac3c16d8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-16T09:49:13.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.692+0000 7fac42ffd640 1 --2- 172.21.15.84:0/2976888441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac3c16d460 0x7fac3c16d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.692+0000 7fac42ffd640 1 -- 172.21.15.84:0/2976888441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac3c0077e0 msgr2=0x7fac3c04acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.693+0000 7fac42ffd640 1 --2- 172.21.15.84:0/2976888441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac3c0077e0 0x7fac3c04acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.693+0000 7fac42ffd640 1 -- 172.21.15.84:0/2976888441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4404e6d0 con 0x7fac3c0e2e10 2023-10-16T09:49:13.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.693+0000 7fac42ffd640 1 --2- 172.21.15.84:0/2976888441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac3c0e2e10 0x7fac3c048a80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fac3000c500 tx=0x7fac3000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.694+0000 7f6c2cff9640 1 --2- 172.21.15.84:0/1206184196 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6c1c06af40 0x7f6c1c06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.694+0000 7f6c2cff9640 1 -- 172.21.15.84:0/1206184196 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c200897b0 con 0x7f6c3015d030 2023-10-16T09:49:13.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.694+0000 7f6b297fa640 1 -- 172.21.15.84:0/1225528834 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6b2400cc10 con 0x7f6b2c065ff0 2023-10-16T09:49:13.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.695+0000 7f6c2ffff640 1 --2- 172.21.15.84:0/1206184196 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6c1c06af40 0x7f6c1c06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.696+0000 7f6b32c0e640 1 -- 172.21.15.84:0/1225528834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2c0cb7f0 con 0x7f6b2c065ff0 2023-10-16T09:49:13.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.696+0000 7f6c2ffff640 1 --2- 172.21.15.84:0/1206184196 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6c1c06af40 0x7f6c1c06d400 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6c28005570 tx=0x7f6c28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.697+0000 7f6b297fa640 1 --2- 172.21.15.84:0/1225528834 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6b0c06aed0 0x7f6b0c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.697+0000 7f6b297fa640 1 -- 172.21.15.84:0/1225528834 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b24089330 con 0x7f6b2c065ff0 2023-10-16T09:49:13.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.698+0000 7f6b2bfff640 1 --2- 172.21.15.84:0/1225528834 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6b0c06aed0 0x7f6b0c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.699+0000 7f6b2bfff640 1 --2- 172.21.15.84:0/1225528834 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6b0c06aed0 0x7f6b0c06d390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b14007c90 tx=0x7f6b14007be0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.702+0000 7fac40ff9640 1 -- 172.21.15.84:0/2976888441 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30005da0 con 0x7fac3c0e2e10 2023-10-16T09:49:13.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.702+0000 7fac49187640 1 -- 172.21.15.84:0/2976888441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac3c049400 con 0x7fac3c0e2e10 2023-10-16T09:49:13.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.703+0000 7f6c2cff9640 1 -- 172.21.15.84:0/1206184196 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6c200548d0 con 0x7f6c3015d030 2023-10-16T09:49:13.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.703+0000 7fac49187640 1 -- 172.21.15.84:0/2976888441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac3c0499c0 con 0x7fac3c0e2e10 2023-10-16T09:49:13.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.704+0000 7fac49187640 1 -- 172.21.15.84:0/2976888441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac3c008bf0 con 0x7fac3c0e2e10 2023-10-16T09:49:13.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.711+0000 7fac40ff9640 1 -- 172.21.15.84:0/2976888441 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac300070f0 con 0x7fac3c0e2e10 2023-10-16T09:49:13.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.711+0000 7fac40ff9640 1 -- 172.21.15.84:0/2976888441 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30005da0 con 0x7fac3c0e2e10 2023-10-16T09:49:13.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.715+0000 7fac40ff9640 1 -- 172.21.15.84:0/2976888441 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fac30007260 con 0x7fac3c0e2e10 2023-10-16T09:49:13.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.717+0000 7fac40ff9640 1 --2- 172.21.15.84:0/2976888441 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fac2806aed0 0x7fac2806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.717+0000 7fac40ff9640 1 -- 172.21.15.84:0/2976888441 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac30088bf0 con 0x7fac3c0e2e10 2023-10-16T09:49:13.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.719+0000 7fac437fe640 1 --2- 172.21.15.84:0/2976888441 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fac2806aed0 0x7fac2806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.717+0000 7f6b297fa640 1 -- 172.21.15.84:0/1225528834 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6b2c0cb7f0 con 0x7f6b2c065ff0 2023-10-16T09:49:13.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.720+0000 7fac437fe640 1 --2- 172.21.15.84:0/2976888441 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fac2806aed0 0x7fac2806d390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fac3c055de0 tx=0x7fac440ac000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.722+0000 7fac40ff9640 1 -- 172.21.15.84:0/2976888441 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fac30053dc0 con 0x7fac3c0e2e10 2023-10-16T09:49:13.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.743+0000 7f4bca1a9640 1 Processor -- start 2023-10-16T09:49:13.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.744+0000 7f4bca1a9640 1 -- start start 2023-10-16T09:49:13.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.744+0000 7ff45e75a640 1 Processor -- start 2023-10-16T09:49:13.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.746+0000 7f4bca1a9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4bc40b3420 0x7f4bc40b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.746+0000 7f4bca1a9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc40d3980 con 0x7f4bc40cf710 2023-10-16T09:49:13.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.746+0000 7f4bca1a9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc40ce630 con 0x7f4bc40cfaf0 2023-10-16T09:49:13.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.746+0000 7f4bca1a9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc40d3c60 con 0x7f4bc40b3420 2023-10-16T09:49:13.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.744+0000 7ff45e75a640 1 -- start start 2023-10-16T09:49:13.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.746+0000 7ff45e75a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff4580cf710 0x7ff4580b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.746+0000 7ff45e75a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4580cfaf0 0x7ff4580b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.746+0000 7ff45e75a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4580d3980 con 0x7ff4580b38e0 2023-10-16T09:49:13.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.746+0000 7f4bc37fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4bc40cf710 0x7f4bc40b2d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32952/0 (socket says 172.21.15.84:32952) 2023-10-16T09:49:13.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.746+0000 7f4bc37fe640 1 -- 172.21.15.84:0/362436675 learned_addr learned my addr 172.21.15.84:0/362436675 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:13.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.746+0000 7ff45e75a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4580d3c60 con 0x7ff4580cf710 2023-10-16T09:49:13.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.746+0000 7ff45e75a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4580ce630 con 0x7ff4580cfaf0 2023-10-16T09:49:13.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.747+0000 7ff457fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff4580cf710 0x7ff4580b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.747+0000 7ff4577fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4580cfaf0 0x7ff4580b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.747+0000 7ff4577fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4580cfaf0 0x7ff4580b3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:37166/0 (socket says 172.21.15.84:37166) 2023-10-16T09:49:13.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.747+0000 7ff4577fe640 1 -- 172.21.15.84:0/3805626150 learned_addr learned my addr 172.21.15.84:0/3805626150 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:13.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.748+0000 7ff4577fe640 1 -- 172.21.15.84:0/3805626150 >> v2:172.21.15.84:3300/0 conn(0x7ff4580cf710 msgr2=0x7ff4580b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:13.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.748+0000 7f4bc27fc640 1 -- 172.21.15.84:0/362436675 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547563083 0 0) 0x7f4bc40ce630 con 0x7f4bc40cfaf0 2023-10-16T09:49:13.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.748+0000 7f4bc27fc640 1 -- 172.21.15.84:0/362436675 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bac003520 con 0x7f4bc40cfaf0 2023-10-16T09:49:13.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.748+0000 7f4bc2ffd640 1 --2- 172.21.15.84:0/362436675 >> v2:172.21.15.157:3300/0 conn(0x7f4bc40b3420 0x7f4bc40b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.748+0000 7f4bc27fc640 1 -- 172.21.15.84:0/362436675 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3157056617 0 0) 0x7f4bc40d3980 con 0x7f4bc40cf710 2023-10-16T09:49:13.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.748+0000 7f4bc27fc640 1 -- 172.21.15.84:0/362436675 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc40ce630 con 0x7f4bc40cf710 2023-10-16T09:49:13.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.749+0000 7f4bc27fc640 1 -- 172.21.15.84:0/362436675 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4143244867 0 0) 0x7f4bc40ce630 con 0x7f4bc40cf710 2023-10-16T09:49:13.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.749+0000 7f4bc27fc640 1 -- 172.21.15.84:0/362436675 >> v2:172.21.15.157:3300/0 conn(0x7f4bc40b3420 msgr2=0x7f4bc40b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:13.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.749+0000 7f4bc27fc640 1 --2- 172.21.15.84:0/362436675 >> v2:172.21.15.157:3300/0 conn(0x7f4bc40b3420 0x7f4bc40b5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.749+0000 7f4bc27fc640 1 -- 172.21.15.84:0/362436675 >> v1:172.21.15.157:6789/0 conn(0x7f4bc40cfaf0 legacy=0x7f4bc40b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.749+0000 7f4bc27fc640 1 -- 172.21.15.84:0/362436675 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc40ce910 con 0x7f4bc40cf710 2023-10-16T09:49:13.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.749+0000 7f4bc27fc640 1 -- 172.21.15.84:0/362436675 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f4bb40016c0 con 0x7f4bc40cf710 2023-10-16T09:49:13.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.749+0000 7f4bc27fc640 1 -- 172.21.15.84:0/362436675 >> v1:172.21.15.84:6789/0 conn(0x7f4bc40cf710 legacy=0x7f4bc40b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.749+0000 7f4bc27fc640 1 --2- 172.21.15.84:0/362436675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bac003b00 0x7f4bac003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.749+0000 7f4bc27fc640 1 --2- 172.21.15.84:0/362436675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bac0047c0 0x7f4bac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.749+0000 7f4bc27fc640 1 --2- 172.21.15.84:0/362436675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bac005180 0x7f4bac014190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.749+0000 7f4bc27fc640 1 -- 172.21.15.84:0/362436675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40d3c60 con 0x7f4bac0047c0 2023-10-16T09:49:13.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.749+0000 7f4bc27fc640 1 -- 172.21.15.84:0/362436675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac0148c0 con 0x7f4bac005180 2023-10-16T09:49:13.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.749+0000 7f4bc27fc640 1 -- 172.21.15.84:0/362436675 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac014a80 con 0x7f4bac003b00 2023-10-16T09:49:13.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.750+0000 7f4bc2ffd640 1 --2- 172.21.15.84:0/362436675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bac0047c0 0x7f4bac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.748+0000 7ff4577fe640 1 --2- 172.21.15.84:0/3805626150 >> v2:172.21.15.84:3300/0 conn(0x7ff4580cf710 0x7ff4580b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.748+0000 7ff4577fe640 1 -- 172.21.15.84:0/3805626150 >> v1:172.21.15.157:6789/0 conn(0x7ff4580b38e0 legacy=0x7ff4580d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.748+0000 7ff4577fe640 1 -- 172.21.15.84:0/3805626150 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4580ce910 con 0x7ff4580cfaf0 2023-10-16T09:49:13.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.748+0000 7ff457fff640 1 --2- 172.21.15.84:0/3805626150 >> v2:172.21.15.84:3300/0 conn(0x7ff4580cf710 0x7ff4580b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:49:13.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.751+0000 7f4bc2ffd640 1 -- 172.21.15.84:0/362436675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bac003b00 msgr2=0x7f4bac003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:13.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.751+0000 7f4bc2ffd640 1 --2- 172.21.15.84:0/362436675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bac003b00 0x7f4bac003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.751+0000 7f4bc2ffd640 1 -- 172.21.15.84:0/362436675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bac005180 msgr2=0x7f4bac014190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T09:49:13.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.751+0000 7f4bc2ffd640 1 --2- 172.21.15.84:0/362436675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bac005180 0x7f4bac014190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.751+0000 7f4bc2ffd640 1 -- 172.21.15.84:0/362436675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc40ce910 con 0x7f4bac0047c0 2023-10-16T09:49:13.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.752+0000 7ff4577fe640 1 --2- 172.21.15.84:0/3805626150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4580cfaf0 0x7ff4580b3250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff440009e50 tx=0x7ff44002bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f05a5553b3da139 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.752+0000 7f4bc2ffd640 1 --2- 172.21.15.84:0/362436675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bac0047c0 0x7f4bac004c40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8009ac0 tx=0x7f4bac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5a299d735e717f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.752+0000 7f4bc3fff640 1 -- 172.21.15.84:0/362436675 reap_dead start 2023-10-16T09:49:13.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.753+0000 7ff456ffd640 1 -- 172.21.15.84:0/3805626150 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44002ca80 con 0x7ff4580cfaf0 2023-10-16T09:49:13.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.753+0000 7ff456ffd640 1 -- 172.21.15.84:0/3805626150 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44002cbe0 con 0x7ff4580cfaf0 2023-10-16T09:49:13.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.753+0000 7ff456ffd640 1 -- 172.21.15.84:0/3805626150 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4400343f0 con 0x7ff4580cfaf0 2023-10-16T09:49:13.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.753+0000 7f4bc27fc640 1 -- 172.21.15.84:0/362436675 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb802c6f0 con 0x7f4bac0047c0 2023-10-16T09:49:13.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.753+0000 7f4bc27fc640 1 -- 172.21.15.84:0/362436675 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb802c850 con 0x7f4bac0047c0 2023-10-16T09:49:13.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.753+0000 7f4bc27fc640 1 -- 172.21.15.84:0/362436675 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb802cbe0 con 0x7f4bac0047c0 2023-10-16T09:49:13.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.754+0000 7f4bca1a9640 1 -- 172.21.15.84:0/362436675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bac0047c0 msgr2=0x7f4bac004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.754+0000 7f4bca1a9640 1 --2- 172.21.15.84:0/362436675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bac0047c0 0x7f4bac004c40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8009ac0 tx=0x7f4bac0057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.754+0000 7f4bca1a9640 1 -- 172.21.15.84:0/362436675 shutdown_connections 2023-10-16T09:49:13.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.754+0000 7f4bca1a9640 1 --2- 172.21.15.84:0/362436675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bac0047c0 0x7f4bac004c40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.754+0000 7f4bca1a9640 1 -- 172.21.15.84:0/362436675 >> 172.21.15.84:0/362436675 conn(0x7f4bc40d2cc0 msgr2=0x7f4bc40d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:13.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.754+0000 7f4bca1a9640 1 -- 172.21.15.84:0/362436675 shutdown_connections 2023-10-16T09:49:13.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.754+0000 7f4bca1a9640 1 -- 172.21.15.84:0/362436675 wait complete. 2023-10-16T09:49:13.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.755+0000 7ff45e75a640 1 -- 172.21.15.84:0/3805626150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4580cfaf0 msgr2=0x7ff4580b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.755+0000 7ff45e75a640 1 --2- 172.21.15.84:0/3805626150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4580cfaf0 0x7ff4580b3250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff440009e50 tx=0x7ff44002bee0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.755+0000 7f54e8b05640 1 Processor -- start 2023-10-16T09:49:13.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.755+0000 7ff45e75a640 1 -- 172.21.15.84:0/3805626150 shutdown_connections 2023-10-16T09:49:13.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.755+0000 7ff45e75a640 1 --2- 172.21.15.84:0/3805626150 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4580cfaf0 0x7ff4580b3250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.755+0000 7ff45e75a640 1 --2- 172.21.15.84:0/3805626150 >> v2:172.21.15.84:3300/0 conn(0x7ff4580cf710 0x7ff4580b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.755+0000 7ff45e75a640 1 -- 172.21.15.84:0/3805626150 >> 172.21.15.84:0/3805626150 conn(0x7ff4580d2cc0 msgr2=0x7ff4580d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:13.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.755+0000 7f54e8b05640 1 -- start start 2023-10-16T09:49:13.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.757+0000 7ff45e75a640 1 -- 172.21.15.84:0/3805626150 shutdown_connections 2023-10-16T09:49:13.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.756+0000 7f54e8b05640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54e40cb1c0 0x7f54e406faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.756+0000 7f54e8b05640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e4065ab0 0x7f54e4065f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.757+0000 7f54e1d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54e40cb1c0 0x7f54e406faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.757+0000 7ff45e75a640 1 -- 172.21.15.84:0/3805626150 wait complete. 2023-10-16T09:49:13.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.757+0000 7f54e1d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54e40cb1c0 0x7f54e406faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:55362/0 (socket says 172.21.15.84:55362) 2023-10-16T09:49:13.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.757+0000 7f54e1d9b640 1 -- 172.21.15.84:0/2602166248 learned_addr learned my addr 172.21.15.84:0/2602166248 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:13.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.757+0000 7f54e8b05640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e415eff0 con 0x7f54e415af80 2023-10-16T09:49:13.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.757+0000 7f54e8b05640 1 -- 172.21.15.84:0/2602166248 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54e4066480 con 0x7f54e40cb1c0 2023-10-16T09:49:13.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.757+0000 7f54e8b05640 1 -- 172.21.15.84:0/2602166248 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40d0930 con 0x7f54e4065ab0 2023-10-16T09:49:13.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.758+0000 7f4bca1a9640 1 Processor -- start 2023-10-16T09:49:13.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.758+0000 7f54e159a640 1 -- 172.21.15.84:0/2602166248 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335065399 0 0) 0x7f54e415eff0 con 0x7f54e415af80 2023-10-16T09:49:13.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.758+0000 7f54e159a640 1 -- 172.21.15.84:0/2602166248 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54cc003520 con 0x7f54e415af80 2023-10-16T09:49:13.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.758+0000 7f54e1d9b640 1 -- 172.21.15.84:0/2602166248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e4065ab0 msgr2=0x7f54e4065f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:13.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.758+0000 7f54e1d9b640 1 --2- 172.21.15.84:0/2602166248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e4065ab0 0x7f54e4065f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.758+0000 7f54e1d9b640 1 -- 172.21.15.84:0/2602166248 >> v1:172.21.15.84:6789/0 conn(0x7f54e415af80 legacy=0x7f54e415f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.758+0000 7f54e1d9b640 1 -- 172.21.15.84:0/2602166248 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e415ecc0 con 0x7f54e40cb1c0 2023-10-16T09:49:13.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.759+0000 7f54e1d9b640 1 --2- 172.21.15.84:0/2602166248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e40cb1c0 0x7f54e406faf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f54d40098a0 tx=0x7f54d402b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8473c2eb0636ec56 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.758+0000 7f4bca1a9640 1 -- start start 2023-10-16T09:49:13.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.759+0000 7f4bca1a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bc415d980 0x7f4bc415aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.759+0000 7f4bca1a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc415dd60 0x7f4bc40b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.759+0000 7f4bca1a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc40b04a0 0x7f4bc40cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.759+0000 7f4bca1a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40b0010 con 0x7f4bc40b04a0 2023-10-16T09:49:13.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.760+0000 7f4bca1a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40b2220 con 0x7f4bc415dd60 2023-10-16T09:49:13.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.760+0000 7f4bca1a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40af9c0 con 0x7f4bc415d980 2023-10-16T09:49:13.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.760+0000 7f54e159a640 1 -- 172.21.15.84:0/2602166248 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d4031890 con 0x7f54e40cb1c0 2023-10-16T09:49:13.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.760+0000 7f4bc2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc415dd60 0x7f4bc40b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.760+0000 7f4bc2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc415dd60 0x7f4bc40b19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:55366/0 (socket says 172.21.15.84:55366) 2023-10-16T09:49:13.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.760+0000 7f4bc2ffd640 1 -- 172.21.15.84:0/70047323 learned_addr learned my addr 172.21.15.84:0/70047323 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:13.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.760+0000 7f4bc37fe640 1 --2- 172.21.15.84:0/70047323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bc415d980 0x7f4bc415aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.760+0000 7f4bc3fff640 1 --2- 172.21.15.84:0/70047323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc40b04a0 0x7f4bc40cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.760+0000 7f54e159a640 1 -- 172.21.15.84:0/2602166248 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d40319f0 con 0x7f54e40cb1c0 2023-10-16T09:49:13.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.760+0000 7f54e159a640 1 -- 172.21.15.84:0/2602166248 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d4031d60 con 0x7f54e40cb1c0 2023-10-16T09:49:13.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.760+0000 7ff45e75a640 1 Processor -- start 2023-10-16T09:49:13.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.761+0000 7f4bc37fe640 1 -- 172.21.15.84:0/70047323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc415dd60 msgr2=0x7f4bc40b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.761+0000 7f4bc37fe640 1 --2- 172.21.15.84:0/70047323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc415dd60 0x7f4bc40b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.761+0000 7f4bc37fe640 1 -- 172.21.15.84:0/70047323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc40b04a0 msgr2=0x7f4bc40cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.762+0000 7f4bc37fe640 1 --2- 172.21.15.84:0/70047323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc40b04a0 0x7f4bc40cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.762+0000 7f4bc37fe640 1 -- 172.21.15.84:0/70047323 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc40afca0 con 0x7f4bc415d980 2023-10-16T09:49:13.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.762+0000 7ff45e75a640 1 -- start start 2023-10-16T09:49:13.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.761+0000 7f54e8b05640 1 -- 172.21.15.84:0/2602166248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e40cb1c0 msgr2=0x7f54e406faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.761+0000 7f54e8b05640 1 --2- 172.21.15.84:0/2602166248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e40cb1c0 0x7f54e406faf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f54d40098a0 tx=0x7f54d402b930 comp rx=0 tx=0).stop 2023-10-16T09:49:13.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.761+0000 7f54e8b05640 1 -- 172.21.15.84:0/2602166248 shutdown_connections 2023-10-16T09:49:13.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.761+0000 7f54e8b05640 1 --2- 172.21.15.84:0/2602166248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e4065ab0 0x7f54e4065f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.761+0000 7f54e8b05640 1 --2- 172.21.15.84:0/2602166248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e40cb1c0 0x7f54e406faf0 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f54d40098a0 tx=0x7f54d402b930 comp rx=0 tx=0).stop 2023-10-16T09:49:13.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.761+0000 7f54e8b05640 1 -- 172.21.15.84:0/2602166248 >> 172.21.15.84:0/2602166248 conn(0x7f54e40d0ab0 msgr2=0x7f54e40d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:13.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.762+0000 7f4bc2ffd640 1 --2- 172.21.15.84:0/70047323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc415dd60 0x7f4bc40b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:49:13.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.762+0000 7f4bc3fff640 1 --2- 172.21.15.84:0/70047323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc40b04a0 0x7f4bc40cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:49:13.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.762+0000 7f4bc37fe640 1 --2- 172.21.15.84:0/70047323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bc415d980 0x7f4bc415aba0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400c7c0 tx=0x7f4bb400cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.763+0000 7f4babfff640 1 -- 172.21.15.84:0/70047323 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40074e0 con 0x7f4bc415d980 2023-10-16T09:49:13.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.763+0000 7f54e8b05640 1 -- 172.21.15.84:0/2602166248 shutdown_connections 2023-10-16T09:49:13.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.763+0000 7f4babfff640 1 -- 172.21.15.84:0/70047323 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb4007660 con 0x7f4bc415d980 2023-10-16T09:49:13.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.763+0000 7f4babfff640 1 -- 172.21.15.84:0/70047323 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4004a20 con 0x7f4bc415d980 2023-10-16T09:49:13.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.763+0000 7f4bca1a9640 1 -- 172.21.15.84:0/70047323 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc40b1f30 con 0x7f4bc415d980 2023-10-16T09:49:13.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.763+0000 7f4bca1a9640 1 -- 172.21.15.84:0/70047323 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc40b0900 con 0x7f4bc415d980 2023-10-16T09:49:13.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.764+0000 7ff45e75a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4580b38e0 0x7ff4581f6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.764+0000 7ff45e75a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4580cf710 0x7ff458157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.764+0000 7ff45e75a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4580cfaf0 0x7ff4580c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.764+0000 7ff45e75a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4580b0b70 con 0x7ff4580cfaf0 2023-10-16T09:49:13.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.764+0000 7ff45e75a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4580b0e10 con 0x7ff4580cf710 2023-10-16T09:49:13.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.764+0000 7ff45e75a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4580b10f0 con 0x7ff4580b38e0 2023-10-16T09:49:13.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.765+0000 7ff4577fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4580cf710 0x7ff458157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.765+0000 7ff4577fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4580cf710 0x7ff458157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:55376/0 (socket says 172.21.15.84:55376) 2023-10-16T09:49:13.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.765+0000 7ff4577fe640 1 -- 172.21.15.84:0/2056812348 learned_addr learned my addr 172.21.15.84:0/2056812348 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:13.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.765+0000 7ff4577fe640 1 -- 172.21.15.84:0/2056812348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4580b38e0 msgr2=0x7ff4581f6f30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:49:13.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.766+0000 7f54e8b05640 1 -- 172.21.15.84:0/2602166248 wait complete. 2023-10-16T09:49:13.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.766+0000 7ff45ccf7640 1 --2- 172.21.15.84:0/2056812348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4580cfaf0 0x7ff4580c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.766+0000 7ff4577fe640 1 --2- 172.21.15.84:0/2056812348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4580b38e0 0x7ff4581f6f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.766+0000 7ff4577fe640 1 -- 172.21.15.84:0/2056812348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4580cfaf0 msgr2=0x7ff4580c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.766+0000 7ff4577fe640 1 --2- 172.21.15.84:0/2056812348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4580cfaf0 0x7ff4580c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.765+0000 7f4bca1a9640 1 -- 172.21.15.84:0/70047323 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc40d25d0 con 0x7f4bc415d980 2023-10-16T09:49:13.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.766+0000 7ff4577fe640 1 -- 172.21.15.84:0/2056812348 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4480093a0 con 0x7ff4580cf710 2023-10-16T09:49:13.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.766+0000 7ff45ccf7640 1 --2- 172.21.15.84:0/2056812348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4580cfaf0 0x7ff4580c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:13.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.766+0000 7ff4577fe640 1 --2- 172.21.15.84:0/2056812348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4580cf710 0x7ff458157f60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff440000f80 tx=0x7ff440004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.767+0000 7ff454ff9640 1 -- 172.21.15.84:0/2056812348 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44003e450 con 0x7ff4580cf710 2023-10-16T09:49:13.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.767+0000 7ff454ff9640 1 -- 172.21.15.84:0/2056812348 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44003e5f0 con 0x7ff4580cf710 2023-10-16T09:49:13.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.767+0000 7ff454ff9640 1 -- 172.21.15.84:0/2056812348 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44003e9a0 con 0x7ff4580cf710 2023-10-16T09:49:13.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.767+0000 7ff45e75a640 1 -- 172.21.15.84:0/2056812348 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff440009b30 con 0x7ff4580cf710 2023-10-16T09:49:13.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.767+0000 7ff45e75a640 1 -- 172.21.15.84:0/2056812348 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4580b16b0 con 0x7ff4580cf710 2023-10-16T09:49:13.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.769+0000 7f54e8b05640 1 Processor -- start 2023-10-16T09:49:13.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.768+0000 7f4babfff640 1 -- 172.21.15.84:0/70047323 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4bb4004b80 con 0x7f4bc415d980 2023-10-16T09:49:13.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.769+0000 7f4babfff640 1 --2- 172.21.15.84:0/70047323 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4bb006ae70 0x7f4bb006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.770+0000 7f4babfff640 1 -- 172.21.15.84:0/70047323 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bb4057b70 con 0x7f4bc415d980 2023-10-16T09:49:13.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.770+0000 7f4bc2ffd640 1 --2- 172.21.15.84:0/70047323 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4bb006ae70 0x7f4bb006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.770+0000 7f54e8b05640 1 -- start start 2023-10-16T09:49:13.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.770+0000 7ff44e7fc640 1 -- 172.21.15.84:0/2056812348 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff45804e160 con 0x7ff4580cf710 2023-10-16T09:49:13.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.771+0000 7f54e8b05640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e4065ab0 0x7f54e40b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.771+0000 7f54e8b05640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e415af80 0x7f54e4151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.771+0000 7f54e8b05640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54e414dde0 0x7f54e4152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.771+0000 7f54e8b05640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e414e5e0 con 0x7f54e414dde0 2023-10-16T09:49:13.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.771+0000 7f54e8b05640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e4152e00 con 0x7f54e4065ab0 2023-10-16T09:49:13.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.771+0000 7f54e8b05640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e41530e0 con 0x7f54e415af80 2023-10-16T09:49:13.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.771+0000 7f54e2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54e414dde0 0x7f54e4152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.772+0000 7f54e259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e4065ab0 0x7f54e40b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.772+0000 7f54e1d9b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e415af80 0x7f54e4151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.772+0000 7f54e1d9b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e415af80 0x7f54e4151cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:37202/0 (socket says 172.21.15.84:37202) 2023-10-16T09:49:13.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.772+0000 7f54e1d9b640 1 -- 172.21.15.84:0/1388576311 learned_addr learned my addr 172.21.15.84:0/1388576311 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:13.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.772+0000 7ff454ff9640 1 -- 172.21.15.84:0/2056812348 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff44002cd50 con 0x7ff4580cf710 2023-10-16T09:49:13.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.772+0000 7f54e1d9b640 1 -- 172.21.15.84:0/1388576311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e4065ab0 msgr2=0x7f54e40b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.772+0000 7f54e1d9b640 1 --2- 172.21.15.84:0/1388576311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e4065ab0 0x7f54e40b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.772+0000 7f54e1d9b640 1 -- 172.21.15.84:0/1388576311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54e414dde0 msgr2=0x7f54e4152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.773+0000 7ff454ff9640 1 --2- 172.21.15.84:0/2056812348 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff43c06ae70 0x7ff43c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.774+0000 7ff454ff9640 1 -- 172.21.15.84:0/2056812348 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff44007b4a0 con 0x7ff4580cf710 2023-10-16T09:49:13.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.771+0000 7f54e2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54e414dde0 0x7f54e4152200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59936/0 (socket says 172.21.15.84:59936) 2023-10-16T09:49:13.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.774+0000 7f54e1d9b640 1 --2- 172.21.15.84:0/1388576311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54e414dde0 0x7f54e4152200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.774+0000 7f54e1d9b640 1 -- 172.21.15.84:0/1388576311 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d40094d0 con 0x7f54e415af80 2023-10-16T09:49:13.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.774+0000 7ff457fff640 1 --2- 172.21.15.84:0/2056812348 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff43c06ae70 0x7ff43c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.774+0000 7f54e2d9d640 1 --2- 172.21.15.84:0/1388576311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54e414dde0 0x7f54e4152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:13.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.774+0000 7f4bc2ffd640 1 --2- 172.21.15.84:0/70047323 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4bb006ae70 0x7f4bb006d330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4bc415e290 tx=0x7f4bb8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.775+0000 7ff457fff640 1 --2- 172.21.15.84:0/2056812348 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff43c06ae70 0x7ff43c06d330 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff448009ad0 tx=0x7ff448009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.775+0000 7f54e1d9b640 1 --2- 172.21.15.84:0/1388576311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e415af80 0x7f54e4151cc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f54d40097f0 tx=0x7f54d40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.783+0000 7f54d2ffd640 1 -- 172.21.15.84:0/1388576311 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d4003810 con 0x7f54e415af80 2023-10-16T09:49:13.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.783+0000 7f54d2ffd640 1 -- 172.21.15.84:0/1388576311 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d403f070 con 0x7f54e415af80 2023-10-16T09:49:13.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.783+0000 7f54d2ffd640 1 -- 172.21.15.84:0/1388576311 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d4003bb0 con 0x7f54e415af80 2023-10-16T09:49:13.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.783+0000 7f4babfff640 1 -- 172.21.15.84:0/70047323 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4bb4054640 con 0x7f4bc415d980 2023-10-16T09:49:13.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.783+0000 7f54e8b05640 1 -- 172.21.15.84:0/1388576311 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e41533c0 con 0x7f54e415af80 2023-10-16T09:49:13.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.784+0000 7f54e8b05640 1 -- 172.21.15.84:0/1388576311 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e40b2cf0 con 0x7f54e415af80 2023-10-16T09:49:13.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.785+0000 7f54e8b05640 1 -- 172.21.15.84:0/1388576311 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e40b2f70 con 0x7f54e415af80 2023-10-16T09:49:13.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.788+0000 7f54d2ffd640 1 -- 172.21.15.84:0/1388576311 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f54d4003d10 con 0x7f54e415af80 2023-10-16T09:49:13.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.788+0000 7ff454ff9640 1 -- 172.21.15.84:0/2056812348 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff440077870 con 0x7ff4580cf710 2023-10-16T09:49:13.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.789+0000 7f54d2ffd640 1 --2- 172.21.15.84:0/1388576311 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f54c406aec0 0x7f54c406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:13.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.789+0000 7f54d2ffd640 1 -- 172.21.15.84:0/1388576311 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54d4041070 con 0x7f54e415af80 2023-10-16T09:49:13.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.790+0000 7f54e259c640 1 --2- 172.21.15.84:0/1388576311 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f54c406aec0 0x7f54c406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:13.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.792+0000 7f54e259c640 1 --2- 172.21.15.84:0/1388576311 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f54c406aec0 0x7f54c406d380 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f54dc001d10 tx=0x7f54dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:13.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.802+0000 7f54d2ffd640 1 -- 172.21.15.84:0/1388576311 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f54d4077050 con 0x7f54e415af80 2023-10-16T09:49:13.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.973+0000 7f00f621f640 1 -- 172.21.15.84:0/1772082248 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f00f004e160 con 0x7f00f00ce1c0 2023-10-16T09:49:13.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.974+0000 7f00d7fff640 1 -- 172.21.15.84:0/1772082248 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f00e0074e60 con 0x7f00f00ce1c0 2023-10-16T09:49:13.976 INFO:teuthology.orchestra.run.smithi084.stdout:55834574853 2023-10-16T09:49:13.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.978+0000 7f00f621f640 1 -- 172.21.15.84:0/1772082248 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f00dc06ae70 msgr2=0x7f00dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.978+0000 7f00f621f640 1 --2- 172.21.15.84:0/1772082248 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f00dc06ae70 0x7f00dc06d330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f00e40017a0 tx=0x7f00e40016f0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.978+0000 7f00f621f640 1 -- 172.21.15.84:0/1772082248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00f00ce1c0 msgr2=0x7f00f01514c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:13.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.978+0000 7f00f621f640 1 --2- 172.21.15.84:0/1772082248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00f00ce1c0 0x7f00f01514c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f00f00ad680 tx=0x7f00e000c820 comp rx=0 tx=0).stop 2023-10-16T09:49:13.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.979+0000 7f00f621f640 1 -- 172.21.15.84:0/1772082248 shutdown_connections 2023-10-16T09:49:13.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.979+0000 7f00f621f640 1 --2- 172.21.15.84:0/1772082248 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f00dc06ae70 0x7f00dc06d330 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.979+0000 7f00f621f640 1 --2- 172.21.15.84:0/1772082248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00f0158f60 0x7f00f015a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.979+0000 7f00f621f640 1 --2- 172.21.15.84:0/1772082248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00f00ce1c0 0x7f00f01514c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.979+0000 7f00f621f640 1 --2- 172.21.15.84:0/1772082248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00f006a4b0 0x7f00f0159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:13.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.979+0000 7f00f621f640 1 -- 172.21.15.84:0/1772082248 >> 172.21.15.84:0/1772082248 conn(0x7f00f00ce9b0 msgr2=0x7f00f006ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:13.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.980+0000 7f00f621f640 1 -- 172.21.15.84:0/1772082248 shutdown_connections 2023-10-16T09:49:13.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:13.980+0000 7f00f621f640 1 -- 172.21.15.84:0/1772082248 wait complete. 2023-10-16T09:49:14.008 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.2 2023-10-16T09:49:14.009 DEBUG:teuthology.parallel:result is None 2023-10-16T09:49:14.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.061+0000 7fac49187640 1 -- 172.21.15.84:0/2976888441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fac3c005520 con 0x7fac3c0e2e10 2023-10-16T09:49:14.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.062+0000 7fac40ff9640 1 -- 172.21.15.84:0/2976888441 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fac300573e0 con 0x7fac3c0e2e10 2023-10-16T09:49:14.064 INFO:teuthology.orchestra.run.smithi084.stdout:55834574854 2023-10-16T09:49:14.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.064+0000 7fde8b59e640 1 -- 172.21.15.84:0/694718665 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fde8c04e9b0 con 0x7fde8c0ca170 2023-10-16T09:49:14.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.064+0000 7fac49187640 1 -- 172.21.15.84:0/2976888441 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fac2806aed0 msgr2=0x7fac2806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.064+0000 7fac49187640 1 --2- 172.21.15.84:0/2976888441 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fac2806aed0 0x7fac2806d390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fac3c055de0 tx=0x7fac440ac000 comp rx=0 tx=0).stop 2023-10-16T09:49:14.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.064+0000 7fac49187640 1 -- 172.21.15.84:0/2976888441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac3c0e2e10 msgr2=0x7fac3c048a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.064+0000 7fac49187640 1 --2- 172.21.15.84:0/2976888441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac3c0e2e10 0x7fac3c048a80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fac3000c500 tx=0x7fac3000c9c0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.064+0000 7fac49187640 1 -- 172.21.15.84:0/2976888441 shutdown_connections 2023-10-16T09:49:14.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.064+0000 7fac49187640 1 --2- 172.21.15.84:0/2976888441 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fac2806aed0 0x7fac2806d390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.064+0000 7fac49187640 1 --2- 172.21.15.84:0/2976888441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac3c16d460 0x7fac3c16d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.064+0000 7fac49187640 1 --2- 172.21.15.84:0/2976888441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac3c0e2e10 0x7fac3c048a80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.064+0000 7fac49187640 1 --2- 172.21.15.84:0/2976888441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac3c0077e0 0x7fac3c04acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.064+0000 7fac49187640 1 -- 172.21.15.84:0/2976888441 >> 172.21.15.84:0/2976888441 conn(0x7fac3c054ce0 msgr2=0x7fac3c005f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:14.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.065+0000 7fde7affd640 1 -- 172.21.15.84:0/694718665 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fde84015020 con 0x7fde8c0ca170 2023-10-16T09:49:14.069 INFO:teuthology.orchestra.run.smithi084.stdout:55834574854 2023-10-16T09:49:14.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.065+0000 7fac49187640 1 -- 172.21.15.84:0/2976888441 shutdown_connections 2023-10-16T09:49:14.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.065+0000 7fac49187640 1 -- 172.21.15.84:0/2976888441 wait complete. 2023-10-16T09:49:14.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.068+0000 7fde8b59e640 1 -- 172.21.15.84:0/694718665 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fde6c06aec0 msgr2=0x7fde6c06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.068+0000 7fde8b59e640 1 --2- 172.21.15.84:0/694718665 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fde6c06aec0 0x7fde6c06d380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fde80035000 tx=0x7fde80009000 comp rx=0 tx=0).stop 2023-10-16T09:49:14.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.068+0000 7fde8b59e640 1 -- 172.21.15.84:0/694718665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde8c0ca170 msgr2=0x7fde8c1fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.068+0000 7fde8b59e640 1 --2- 172.21.15.84:0/694718665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde8c0ca170 0x7fde8c1fe260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fde8400e580 tx=0x7fde8400ea40 comp rx=0 tx=0).stop 2023-10-16T09:49:14.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.069+0000 7fde8b59e640 1 -- 172.21.15.84:0/694718665 shutdown_connections 2023-10-16T09:49:14.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.069+0000 7fde8b59e640 1 --2- 172.21.15.84:0/694718665 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fde6c06aec0 0x7fde6c06d380 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.069+0000 7fde8b59e640 1 --2- 172.21.15.84:0/694718665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde8c0ca170 0x7fde8c1fe260 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.069+0000 7fde8b59e640 1 --2- 172.21.15.84:0/694718665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8c0c9850 0x7fde8c0c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.069+0000 7fde8b59e640 1 --2- 172.21.15.84:0/694718665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde8c06f440 0x7fde8c1ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.069+0000 7fde8b59e640 1 -- 172.21.15.84:0/694718665 >> 172.21.15.84:0/694718665 conn(0x7fde8c0ce6a0 msgr2=0x7fde8c0ac910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:14.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.069+0000 7fde8b59e640 1 -- 172.21.15.84:0/694718665 shutdown_connections 2023-10-16T09:49:14.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.072+0000 7fde8b59e640 1 -- 172.21.15.84:0/694718665 wait complete. 2023-10-16T09:49:14.084 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.3 2023-10-16T09:49:14.084 DEBUG:teuthology.parallel:result is None 2023-10-16T09:49:14.098 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.5 2023-10-16T09:49:14.098 DEBUG:teuthology.parallel:result is None 2023-10-16T09:49:14.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.107+0000 7f6b32c0e640 1 -- 172.21.15.84:0/1225528834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6b2c0cb7f0 con 0x7f6b2c065ff0 2023-10-16T09:49:14.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.107+0000 7f6b297fa640 1 -- 172.21.15.84:0/1225528834 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6b240544a0 con 0x7f6b2c065ff0 2023-10-16T09:49:14.110 INFO:teuthology.orchestra.run.smithi084.stdout:55834574853 2023-10-16T09:49:14.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.111+0000 7f6b32c0e640 1 -- 172.21.15.84:0/1225528834 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6b0c06aed0 msgr2=0x7f6b0c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.111+0000 7f6b32c0e640 1 --2- 172.21.15.84:0/1225528834 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6b0c06aed0 0x7f6b0c06d390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b14007c90 tx=0x7f6b14007be0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.111+0000 7f6b32c0e640 1 -- 172.21.15.84:0/1225528834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b2c065ff0 msgr2=0x7f6b2c161fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.111+0000 7f6b32c0e640 1 --2- 172.21.15.84:0/1225528834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b2c065ff0 0x7f6b2c161fb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400e8b0 tx=0x7f6b2400ed70 comp rx=0 tx=0).stop 2023-10-16T09:49:14.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.111+0000 7f6b32c0e640 1 -- 172.21.15.84:0/1225528834 shutdown_connections 2023-10-16T09:49:14.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.111+0000 7f6b32c0e640 1 --2- 172.21.15.84:0/1225528834 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6b0c06aed0 0x7f6b0c06d390 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.111+0000 7f6b32c0e640 1 --2- 172.21.15.84:0/1225528834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b2c0b5150 0x7f6b2c0cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.111+0000 7f6b32c0e640 1 --2- 172.21.15.84:0/1225528834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b2c0b20f0 0x7f6b2c0ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.111+0000 7f6b32c0e640 1 --2- 172.21.15.84:0/1225528834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b2c065ff0 0x7f6b2c161fb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.111+0000 7f6b32c0e640 1 -- 172.21.15.84:0/1225528834 >> 172.21.15.84:0/1225528834 conn(0x7f6b2c0d2d10 msgr2=0x7f6b2c0b3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:14.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.111+0000 7f6b32c0e640 1 -- 172.21.15.84:0/1225528834 shutdown_connections 2023-10-16T09:49:14.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.111+0000 7f6b32c0e640 1 -- 172.21.15.84:0/1225528834 wait complete. 2023-10-16T09:49:14.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.117+0000 7f6c363c9640 1 -- 172.21.15.84:0/1206184196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6c300c9aa0 con 0x7f6c3015d030 2023-10-16T09:49:14.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.117+0000 7f6c2cff9640 1 -- 172.21.15.84:0/1206184196 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6c20057ef0 con 0x7f6c3015d030 2023-10-16T09:49:14.119 INFO:teuthology.orchestra.run.smithi084.stdout:55834574854 2023-10-16T09:49:14.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.120+0000 7f6c363c9640 1 -- 172.21.15.84:0/1206184196 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6c1c06af40 msgr2=0x7f6c1c06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.120+0000 7f6c363c9640 1 --2- 172.21.15.84:0/1206184196 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6c1c06af40 0x7f6c1c06d400 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6c28005570 tx=0x7f6c28009040 comp rx=0 tx=0).stop 2023-10-16T09:49:14.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.120+0000 7f6c363c9640 1 -- 172.21.15.84:0/1206184196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c3015d030 msgr2=0x7f6c30065d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.120+0000 7f6c363c9640 1 --2- 172.21.15.84:0/1206184196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c3015d030 0x7f6c30065d20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000e9b0 tx=0x7f6c2000ee70 comp rx=0 tx=0).stop 2023-10-16T09:49:14.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.121+0000 7f6c363c9640 1 -- 172.21.15.84:0/1206184196 shutdown_connections 2023-10-16T09:49:14.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.121+0000 7f6c363c9640 1 --2- 172.21.15.84:0/1206184196 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6c1c06af40 0x7f6c1c06d400 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.121+0000 7f6c363c9640 1 --2- 172.21.15.84:0/1206184196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c3015d410 0x7f6c300cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.121+0000 7f6c363c9640 1 --2- 172.21.15.84:0/1206184196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c3015d030 0x7f6c30065d20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.121+0000 7f6c363c9640 1 --2- 172.21.15.84:0/1206184196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c300cd260 0x7f6c300c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.121+0000 7f6c363c9640 1 -- 172.21.15.84:0/1206184196 >> 172.21.15.84:0/1206184196 conn(0x7f6c300cf720 msgr2=0x7f6c300b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:14.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.121+0000 7f6c363c9640 1 -- 172.21.15.84:0/1206184196 shutdown_connections 2023-10-16T09:49:14.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.121+0000 7f6c363c9640 1 -- 172.21.15.84:0/1206184196 wait complete. 2023-10-16T09:49:14.138 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.1 2023-10-16T09:49:14.138 DEBUG:teuthology.parallel:result is None 2023-10-16T09:49:14.139 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.0 2023-10-16T09:49:14.139 DEBUG:teuthology.parallel:result is None 2023-10-16T09:49:14.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.140+0000 7ff44e7fc640 1 -- 172.21.15.84:0/2056812348 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7ff4580cd980 con 0x7ff4580cf710 2023-10-16T09:49:14.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.141+0000 7ff454ff9640 1 -- 172.21.15.84:0/2056812348 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff440077a10 con 0x7ff4580cf710 2023-10-16T09:49:14.143 INFO:teuthology.orchestra.run.smithi084.stdout:55834574854 2023-10-16T09:49:14.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.143+0000 7ff44e7fc640 1 -- 172.21.15.84:0/2056812348 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff43c06ae70 msgr2=0x7ff43c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.143+0000 7ff44e7fc640 1 --2- 172.21.15.84:0/2056812348 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff43c06ae70 0x7ff43c06d330 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff448009ad0 tx=0x7ff448009190 comp rx=0 tx=0).stop 2023-10-16T09:49:14.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.143+0000 7ff44e7fc640 1 -- 172.21.15.84:0/2056812348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4580cf710 msgr2=0x7ff458157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.143+0000 7ff44e7fc640 1 --2- 172.21.15.84:0/2056812348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4580cf710 0x7ff458157f60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff440000f80 tx=0x7ff440004a40 comp rx=0 tx=0).stop 2023-10-16T09:49:14.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.143+0000 7ff44e7fc640 1 -- 172.21.15.84:0/2056812348 shutdown_connections 2023-10-16T09:49:14.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.144+0000 7ff44e7fc640 1 --2- 172.21.15.84:0/2056812348 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff43c06ae70 0x7ff43c06d330 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.144+0000 7ff44e7fc640 1 --2- 172.21.15.84:0/2056812348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4580cfaf0 0x7ff4580c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.144+0000 7ff44e7fc640 1 --2- 172.21.15.84:0/2056812348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4580cf710 0x7ff458157f60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.144+0000 7ff44e7fc640 1 --2- 172.21.15.84:0/2056812348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4580b38e0 0x7ff4581f6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.144+0000 7ff44e7fc640 1 -- 172.21.15.84:0/2056812348 >> 172.21.15.84:0/2056812348 conn(0x7ff4580d2cc0 msgr2=0x7ff4580d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:14.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.144+0000 7ff44e7fc640 1 -- 172.21.15.84:0/2056812348 shutdown_connections 2023-10-16T09:49:14.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.144+0000 7ff44e7fc640 1 -- 172.21.15.84:0/2056812348 wait complete. 2023-10-16T09:49:14.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.158+0000 7f4bca1a9640 1 -- 172.21.15.84:0/70047323 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f4bc404e160 con 0x7f4bc415d980 2023-10-16T09:49:14.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.159+0000 7f4babfff640 1 -- 172.21.15.84:0/70047323 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4bb4018070 con 0x7f4bc415d980 2023-10-16T09:49:14.161 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-10-16T09:49:14.161 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.7 2023-10-16T09:49:14.162 DEBUG:teuthology.parallel:result is None 2023-10-16T09:49:14.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.161+0000 7f4bca1a9640 1 -- 172.21.15.84:0/70047323 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4bb006ae70 msgr2=0x7f4bb006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.161+0000 7f4bca1a9640 1 --2- 172.21.15.84:0/70047323 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4bb006ae70 0x7f4bb006d330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4bc415e290 tx=0x7f4bb8030000 comp rx=0 tx=0).stop 2023-10-16T09:49:14.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.161+0000 7f4bca1a9640 1 -- 172.21.15.84:0/70047323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bc415d980 msgr2=0x7f4bc415aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.161+0000 7f4bca1a9640 1 --2- 172.21.15.84:0/70047323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bc415d980 0x7f4bc415aba0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400c7c0 tx=0x7f4bb400cc80 comp rx=0 tx=0).stop 2023-10-16T09:49:14.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.161+0000 7f4bca1a9640 1 -- 172.21.15.84:0/70047323 shutdown_connections 2023-10-16T09:49:14.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.161+0000 7f4bca1a9640 1 --2- 172.21.15.84:0/70047323 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4bb006ae70 0x7f4bb006d330 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.161+0000 7f4bca1a9640 1 --2- 172.21.15.84:0/70047323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc40b04a0 0x7f4bc40cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.161+0000 7f4bca1a9640 1 --2- 172.21.15.84:0/70047323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc415dd60 0x7f4bc40b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.161+0000 7f4bca1a9640 1 --2- 172.21.15.84:0/70047323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bc415d980 0x7f4bc415aba0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.161+0000 7f4bca1a9640 1 -- 172.21.15.84:0/70047323 >> 172.21.15.84:0/70047323 conn(0x7f4bc40d2cc0 msgr2=0x7f4bc4160000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:14.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.161+0000 7f4bca1a9640 1 -- 172.21.15.84:0/70047323 shutdown_connections 2023-10-16T09:49:14.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.161+0000 7f4bca1a9640 1 -- 172.21.15.84:0/70047323 wait complete. 2023-10-16T09:49:14.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.168+0000 7f54e8b05640 1 -- 172.21.15.84:0/1388576311 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f54e40b0dd0 con 0x7f54e415af80 2023-10-16T09:49:14.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.169+0000 7f54d2ffd640 1 -- 172.21.15.84:0/1388576311 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f54e40b0dd0 con 0x7f54e415af80 2023-10-16T09:49:14.171 INFO:teuthology.orchestra.run.smithi084.stdout:55834574854 2023-10-16T09:49:14.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.171+0000 7f54e8b05640 1 -- 172.21.15.84:0/1388576311 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f54c406aec0 msgr2=0x7f54c406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.171+0000 7f54e8b05640 1 --2- 172.21.15.84:0/1388576311 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f54c406aec0 0x7f54c406d380 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f54dc001d10 tx=0x7f54dc009040 comp rx=0 tx=0).stop 2023-10-16T09:49:14.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.172+0000 7f54e8b05640 1 -- 172.21.15.84:0/1388576311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e415af80 msgr2=0x7f54e4151cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.172+0000 7f54e8b05640 1 --2- 172.21.15.84:0/1388576311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e415af80 0x7f54e4151cc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f54d40097f0 tx=0x7f54d40046c0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.172+0000 7f54e8b05640 1 -- 172.21.15.84:0/1388576311 shutdown_connections 2023-10-16T09:49:14.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.172+0000 7f54e8b05640 1 --2- 172.21.15.84:0/1388576311 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f54c406aec0 0x7f54c406d380 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.172+0000 7f54e8b05640 1 --2- 172.21.15.84:0/1388576311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54e414dde0 0x7f54e4152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.172+0000 7f54e8b05640 1 --2- 172.21.15.84:0/1388576311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e415af80 0x7f54e4151cc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.172+0000 7f54e8b05640 1 --2- 172.21.15.84:0/1388576311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e4065ab0 0x7f54e40b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.172+0000 7f54e8b05640 1 -- 172.21.15.84:0/1388576311 >> 172.21.15.84:0/1388576311 conn(0x7f54e40d0ab0 msgr2=0x7f54e40cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:14.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.172+0000 7f54e8b05640 1 -- 172.21.15.84:0/1388576311 shutdown_connections 2023-10-16T09:49:14.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.172+0000 7f54e8b05640 1 -- 172.21.15.84:0/1388576311 wait complete. 2023-10-16T09:49:14.180 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.4 2023-10-16T09:49:14.180 DEBUG:teuthology.parallel:result is None 2023-10-16T09:49:14.190 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.6 2023-10-16T09:49:14.190 DEBUG:teuthology.parallel:result is None 2023-10-16T09:49:14.191 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-16T09:49:14.191 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:49:14.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.439+0000 7f31e60e0640 1 Processor -- start 2023-10-16T09:49:14.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.439+0000 7f31e60e0640 1 -- start start 2023-10-16T09:49:14.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.440+0000 7f31e60e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31e015aaa0 0x7f31e015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:14.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.440+0000 7f31e60e0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f31e00ce390 0x7f31e0070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:14.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.440+0000 7f31e60e0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e015ed60 con 0x7f31e015ae80 2023-10-16T09:49:14.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.440+0000 7f31e60e0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e006b810 con 0x7f31e00ce390 2023-10-16T09:49:14.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.440+0000 7f31e60e0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e00707e0 con 0x7f31e015aaa0 2023-10-16T09:49:14.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.440+0000 7f31dffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f31e00ce390 0x7f31e0070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:14.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.441+0000 7f31dffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f31e00ce390 0x7f31e0070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59940/0 (socket says 172.21.15.84:59940) 2023-10-16T09:49:14.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.441+0000 7f31dffff640 1 -- 172.21.15.84:0/828103157 learned_addr learned my addr 172.21.15.84:0/828103157 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:14.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.441+0000 7f31df7fe640 1 --2- 172.21.15.84:0/828103157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31e015aaa0 0x7f31e015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:14.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.441+0000 7f31dffff640 1 -- 172.21.15.84:0/828103157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31e015aaa0 msgr2=0x7f31e015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:14.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.441+0000 7f31dffff640 1 --2- 172.21.15.84:0/828103157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31e015aaa0 0x7f31e015e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.441+0000 7f31dffff640 1 -- 172.21.15.84:0/828103157 >> v1:172.21.15.157:6789/0 conn(0x7f31e015ae80 legacy=0x7f31e00cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:14.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.441+0000 7f31dffff640 1 -- 172.21.15.84:0/828103157 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e015ef90 con 0x7f31e00ce390 2023-10-16T09:49:14.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.442+0000 7f31dffff640 1 --2- 172.21.15.84:0/828103157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e00ce390 0x7f31e0070f50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009e50 tx=0x7f31d802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd00679fd728a9e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:14.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.442+0000 7f31deffd640 1 -- 172.21.15.84:0/828103157 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d802ca70 con 0x7f31e00ce390 2023-10-16T09:49:14.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.442+0000 7f31deffd640 1 -- 172.21.15.84:0/828103157 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d802cbd0 con 0x7f31e00ce390 2023-10-16T09:49:14.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.442+0000 7f31deffd640 1 -- 172.21.15.84:0/828103157 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d80343f0 con 0x7f31e00ce390 2023-10-16T09:49:14.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.443+0000 7f31e60e0640 1 -- 172.21.15.84:0/828103157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e00ce390 msgr2=0x7f31e0070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.443+0000 7f31e60e0640 1 --2- 172.21.15.84:0/828103157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e00ce390 0x7f31e0070f50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009e50 tx=0x7f31d802bed0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.443+0000 7f31e60e0640 1 -- 172.21.15.84:0/828103157 shutdown_connections 2023-10-16T09:49:14.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.443+0000 7f31e60e0640 1 --2- 172.21.15.84:0/828103157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e00ce390 0x7f31e0070f50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.443+0000 7f31e60e0640 1 --2- 172.21.15.84:0/828103157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31e015aaa0 0x7f31e015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.443+0000 7f31e60e0640 1 -- 172.21.15.84:0/828103157 >> 172.21.15.84:0/828103157 conn(0x7f31e00cb410 msgr2=0x7f31e00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:14.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.443+0000 7f31e60e0640 1 -- 172.21.15.84:0/828103157 shutdown_connections 2023-10-16T09:49:14.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.443+0000 7f31e60e0640 1 -- 172.21.15.84:0/828103157 wait complete. 2023-10-16T09:49:14.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.444+0000 7f31e60e0640 1 Processor -- start 2023-10-16T09:49:14.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.445+0000 7f31e60e0640 1 -- start start 2023-10-16T09:49:14.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.445+0000 7f31e60e0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31e00ce390 0x7f31e015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:14.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.445+0000 7f31e60e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31e015aaa0 0x7f31e01509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:14.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.445+0000 7f31e60e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e015ae80 0x7f31e015b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:14.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.445+0000 7f31e60e0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e015c630 con 0x7f31e015ae80 2023-10-16T09:49:14.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.445+0000 7f31e60e0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e0151290 con 0x7f31e00ce390 2023-10-16T09:49:14.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.445+0000 7f31e60e0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e015ba20 con 0x7f31e015aaa0 2023-10-16T09:49:14.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.446+0000 7f31dffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e015ae80 0x7f31e015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:14.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.446+0000 7f31dffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e015ae80 0x7f31e015b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59944/0 (socket says 172.21.15.84:59944) 2023-10-16T09:49:14.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.446+0000 7f31dffff640 1 -- 172.21.15.84:0/2003258640 learned_addr learned my addr 172.21.15.84:0/2003258640 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:14.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.446+0000 7f31df7fe640 1 --2- 172.21.15.84:0/2003258640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31e00ce390 0x7f31e015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:14.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.446+0000 7f31d7fff640 1 --2- 172.21.15.84:0/2003258640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31e015aaa0 0x7f31e01509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:14.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.446+0000 7f31dffff640 1 -- 172.21.15.84:0/2003258640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31e015aaa0 msgr2=0x7f31e01509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.446+0000 7f31dffff640 1 --2- 172.21.15.84:0/2003258640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31e015aaa0 0x7f31e01509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.446+0000 7f31dffff640 1 -- 172.21.15.84:0/2003258640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31e00ce390 msgr2=0x7f31e015cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.446+0000 7f31dffff640 1 --2- 172.21.15.84:0/2003258640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31e00ce390 0x7f31e015cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.446+0000 7f31dffff640 1 -- 172.21.15.84:0/2003258640 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d8009b30 con 0x7f31e015ae80 2023-10-16T09:49:14.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.446+0000 7f31df7fe640 1 --2- 172.21.15.84:0/2003258640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31e00ce390 0x7f31e015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:14.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.446+0000 7f31d7fff640 1 --2- 172.21.15.84:0/2003258640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31e015aaa0 0x7f31e01509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:14.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.446+0000 7f31dffff640 1 --2- 172.21.15.84:0/2003258640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e015ae80 0x7f31e015b320 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009e50 tx=0x7f31d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:14.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.447+0000 7f31dcff9640 1 -- 172.21.15.84:0/2003258640 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8009510 con 0x7f31e015ae80 2023-10-16T09:49:14.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.447+0000 7f31dcff9640 1 -- 172.21.15.84:0/2003258640 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d803e450 con 0x7f31e015ae80 2023-10-16T09:49:14.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.447+0000 7f31e60e0640 1 -- 172.21.15.84:0/2003258640 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e015bd00 con 0x7f31e015ae80 2023-10-16T09:49:14.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.447+0000 7f31dcff9640 1 -- 172.21.15.84:0/2003258640 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d803e7e0 con 0x7f31e015ae80 2023-10-16T09:49:14.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.447+0000 7f31e60e0640 1 -- 172.21.15.84:0/2003258640 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e015c2c0 con 0x7f31e015ae80 2023-10-16T09:49:14.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.448+0000 7f31e60e0640 1 -- 172.21.15.84:0/2003258640 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e015c7e0 con 0x7f31e015ae80 2023-10-16T09:49:14.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.449+0000 7f31dcff9640 1 -- 172.21.15.84:0/2003258640 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f31d803e940 con 0x7f31e015ae80 2023-10-16T09:49:14.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.450+0000 7f31dcff9640 1 --2- 172.21.15.84:0/2003258640 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f31c006ae70 0x7f31c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:14.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.450+0000 7f31dcff9640 1 -- 172.21.15.84:0/2003258640 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31d80ac3c0 con 0x7f31e015ae80 2023-10-16T09:49:14.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.450+0000 7f31df7fe640 1 --2- 172.21.15.84:0/2003258640 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f31c006ae70 0x7f31c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:14.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.450+0000 7f31df7fe640 1 --2- 172.21.15.84:0/2003258640 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f31c006ae70 0x7f31c006d330 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f31d0006fd0 tx=0x7f31d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:14.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.454+0000 7f31dcff9640 1 -- 172.21.15.84:0/2003258640 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f31e015c7e0 con 0x7f31e015ae80 2023-10-16T09:49:14.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.627+0000 7f31e60e0640 1 -- 172.21.15.84:0/2003258640 --> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f31e015f550 con 0x7f31c006ae70 2023-10-16T09:49:14.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.635+0000 7f31dcff9640 1 -- 172.21.15.84:0/2003258640 <== mgr.4107 v2:172.21.15.84:6832/3010246666 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264273 (secure 0 0 0) 0x7f31d00095f0 con 0x7f31c006ae70 2023-10-16T09:49:14.641 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:14.642 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T09:49:14.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.641+0000 7f31e60e0640 1 -- 172.21.15.84:0/2003258640 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f31c006ae70 msgr2=0x7f31c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.641+0000 7f31e60e0640 1 --2- 172.21.15.84:0/2003258640 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f31c006ae70 0x7f31c006d330 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f31d0006fd0 tx=0x7f31d0008040 comp rx=0 tx=0).stop 2023-10-16T09:49:14.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.641+0000 7f31e60e0640 1 -- 172.21.15.84:0/2003258640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e015ae80 msgr2=0x7f31e015b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.641+0000 7f31e60e0640 1 --2- 172.21.15.84:0/2003258640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e015ae80 0x7f31e015b320 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009e50 tx=0x7f31d8004a40 comp rx=0 tx=0).stop 2023-10-16T09:49:14.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.641+0000 7f31e60e0640 1 -- 172.21.15.84:0/2003258640 shutdown_connections 2023-10-16T09:49:14.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.641+0000 7f31e60e0640 1 --2- 172.21.15.84:0/2003258640 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f31c006ae70 0x7f31c006d330 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.641+0000 7f31e60e0640 1 --2- 172.21.15.84:0/2003258640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e015ae80 0x7f31e015b320 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.641+0000 7f31e60e0640 1 --2- 172.21.15.84:0/2003258640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31e015aaa0 0x7f31e01509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.641+0000 7f31e60e0640 1 --2- 172.21.15.84:0/2003258640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31e00ce390 0x7f31e015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.642+0000 7f31e60e0640 1 -- 172.21.15.84:0/2003258640 >> 172.21.15.84:0/2003258640 conn(0x7f31e00cb410 msgr2=0x7f31e006f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:14.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.642+0000 7f31e60e0640 1 -- 172.21.15.84:0/2003258640 shutdown_connections 2023-10-16T09:49:14.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.642+0000 7f31e60e0640 1 -- 172.21.15.84:0/2003258640 wait complete. 2023-10-16T09:49:14.660 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-16T09:49:13.596231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236100,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738620,"statfs":{"total":773094113280,"available":772852346880,"internally_reserved":0,"allocated":2990080,"data_stored":1790108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.060358"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942979+0000","last_change":"2023-10-16T09:49:02.569411+0000","last_active":"2023-10-16T09:49:03.942979+0000","last_peered":"2023-10-16T09:49:03.942979+0000","last_clean":"2023-10-16T09:49:03.942979+0000","last_became_active":"2023-10-16T09:49:02.569101+0000","last_became_peered":"2023-10-16T09:49:02.569101+0000","last_unstale":"2023-10-16T09:49:03.942979+0000","last_undegraded":"2023-10-16T09:49:03.942979+0000","last_fullsized":"2023-10-16T09:49:03.942979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:20:01.929361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937690+0000","last_change":"2023-10-16T09:49:02.570326+0000","last_active":"2023-10-16T09:49:03.937690+0000","last_peered":"2023-10-16T09:49:03.937690+0000","last_clean":"2023-10-16T09:49:03.937690+0000","last_became_active":"2023-10-16T09:49:02.569866+0000","last_became_peered":"2023-10-16T09:49:02.569866+0000","last_unstale":"2023-10-16T09:49:03.937690+0000","last_undegraded":"2023-10-16T09:49:03.937690+0000","last_fullsized":"2023-10-16T09:49:03.937690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:10.503481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940359+0000","last_change":"2023-10-16T09:49:02.566217+0000","last_active":"2023-10-16T09:49:03.940359+0000","last_peered":"2023-10-16T09:49:03.940359+0000","last_clean":"2023-10-16T09:49:03.940359+0000","last_became_active":"2023-10-16T09:49:02.565843+0000","last_became_peered":"2023-10-16T09:49:02.565843+0000","last_unstale":"2023-10-16T09:49:03.940359+0000","last_undegraded":"2023-10-16T09:49:03.940359+0000","last_fullsized":"2023-10-16T09:49:03.940359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540898+0000","last_change":"2023-10-16T09:49:02.580605+0000","last_active":"2023-10-16T09:49:03.540898+0000","last_peered":"2023-10-16T09:49:03.540898+0000","last_clean":"2023-10-16T09:49:03.540898+0000","last_became_active":"2023-10-16T09:49:02.573845+0000","last_became_peered":"2023-10-16T09:49:02.573845+0000","last_unstale":"2023-10-16T09:49:03.540898+0000","last_undegraded":"2023-10-16T09:49:03.540898+0000","last_fullsized":"2023-10-16T09:49:03.540898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:05.838911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941895+0000","last_change":"2023-10-16T09:49:02.563238+0000","last_active":"2023-10-16T09:49:03.941895+0000","last_peered":"2023-10-16T09:49:03.941895+0000","last_clean":"2023-10-16T09:49:03.941895+0000","last_became_active":"2023-10-16T09:49:02.562849+0000","last_became_peered":"2023-10-16T09:49:02.562849+0000","last_unstale":"2023-10-16T09:49:03.941895+0000","last_undegraded":"2023-10-16T09:49:03.941895+0000","last_fullsized":"2023-10-16T09:49:03.941895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:55.226843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540148+0000","last_change":"2023-10-16T09:49:02.572385+0000","last_active":"2023-10-16T09:49:03.540148+0000","last_peered":"2023-10-16T09:49:03.540148+0000","last_clean":"2023-10-16T09:49:03.540148+0000","last_became_active":"2023-10-16T09:49:02.571270+0000","last_became_peered":"2023-10-16T09:49:02.571270+0000","last_unstale":"2023-10-16T09:49:03.540148+0000","last_undegraded":"2023-10-16T09:49:03.540148+0000","last_fullsized":"2023-10-16T09:49:03.540148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941710+0000","last_change":"2023-10-16T09:49:02.570612+0000","last_active":"2023-10-16T09:49:03.941710+0000","last_peered":"2023-10-16T09:49:03.941710+0000","last_clean":"2023-10-16T09:49:03.941710+0000","last_became_active":"2023-10-16T09:49:02.570330+0000","last_became_peered":"2023-10-16T09:49:02.570330+0000","last_unstale":"2023-10-16T09:49:03.941710+0000","last_undegraded":"2023-10-16T09:49:03.941710+0000","last_fullsized":"2023-10-16T09:49:03.941710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:07.724831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540222+0000","last_change":"2023-10-16T09:49:02.572301+0000","last_active":"2023-10-16T09:49:03.540222+0000","last_peered":"2023-10-16T09:49:03.540222+0000","last_clean":"2023-10-16T09:49:03.540222+0000","last_became_active":"2023-10-16T09:49:02.571934+0000","last_became_peered":"2023-10-16T09:49:02.571934+0000","last_unstale":"2023-10-16T09:49:03.540222+0000","last_undegraded":"2023-10-16T09:49:03.540222+0000","last_fullsized":"2023-10-16T09:49:03.540222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:51.981410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.943019+0000","last_change":"2023-10-16T09:49:02.580905+0000","last_active":"2023-10-16T09:49:03.943019+0000","last_peered":"2023-10-16T09:49:03.943019+0000","last_clean":"2023-10-16T09:49:03.943019+0000","last_became_active":"2023-10-16T09:49:02.577694+0000","last_became_peered":"2023-10-16T09:49:02.577694+0000","last_unstale":"2023-10-16T09:49:03.943019+0000","last_undegraded":"2023-10-16T09:49:03.943019+0000","last_fullsized":"2023-10-16T09:49:03.943019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:02:14.799393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938295+0000","last_change":"2023-10-16T09:49:02.576191+0000","last_active":"2023-10-16T09:49:03.938295+0000","last_peered":"2023-10-16T09:49:03.938295+0000","last_clean":"2023-10-16T09:49:03.938295+0000","last_became_active":"2023-10-16T09:49:02.575810+0000","last_became_peered":"2023-10-16T09:49:02.575810+0000","last_unstale":"2023-10-16T09:49:03.938295+0000","last_undegraded":"2023-10-16T09:49:03.938295+0000","last_fullsized":"2023-10-16T09:49:03.938295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937913+0000","last_change":"2023-10-16T09:49:02.577871+0000","last_active":"2023-10-16T09:49:03.937913+0000","last_peered":"2023-10-16T09:49:03.937913+0000","last_clean":"2023-10-16T09:49:03.937913+0000","last_became_active":"2023-10-16T09:49:02.577580+0000","last_became_peered":"2023-10-16T09:49:02.577580+0000","last_unstale":"2023-10-16T09:49:03.937913+0000","last_undegraded":"2023-10-16T09:49:03.937913+0000","last_fullsized":"2023-10-16T09:49:03.937913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940615+0000","last_change":"2023-10-16T09:49:02.575626+0000","last_active":"2023-10-16T09:49:03.940615+0000","last_peered":"2023-10-16T09:49:03.940615+0000","last_clean":"2023-10-16T09:49:03.940615+0000","last_became_active":"2023-10-16T09:49:02.569588+0000","last_became_peered":"2023-10-16T09:49:02.569588+0000","last_unstale":"2023-10-16T09:49:03.940615+0000","last_undegraded":"2023-10-16T09:49:03.940615+0000","last_fullsized":"2023-10-16T09:49:03.940615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:28.559387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541264+0000","last_change":"2023-10-16T09:49:02.562597+0000","last_active":"2023-10-16T09:49:03.541264+0000","last_peered":"2023-10-16T09:49:03.541264+0000","last_clean":"2023-10-16T09:49:03.541264+0000","last_became_active":"2023-10-16T09:49:02.562322+0000","last_became_peered":"2023-10-16T09:49:02.562322+0000","last_unstale":"2023-10-16T09:49:03.541264+0000","last_undegraded":"2023-10-16T09:49:03.541264+0000","last_fullsized":"2023-10-16T09:49:03.541264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:51.170491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939007+0000","last_change":"2023-10-16T09:49:02.568210+0000","last_active":"2023-10-16T09:49:03.939007+0000","last_peered":"2023-10-16T09:49:03.939007+0000","last_clean":"2023-10-16T09:49:03.939007+0000","last_became_active":"2023-10-16T09:49:02.567330+0000","last_became_peered":"2023-10-16T09:49:02.567330+0000","last_unstale":"2023-10-16T09:49:03.939007+0000","last_undegraded":"2023-10-16T09:49:03.939007+0000","last_fullsized":"2023-10-16T09:49:03.939007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:17.275794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938071+0000","last_change":"2023-10-16T09:49:02.561360+0000","last_active":"2023-10-16T09:49:03.938071+0000","last_peered":"2023-10-16T09:49:03.938071+0000","last_clean":"2023-10-16T09:49:03.938071+0000","last_became_active":"2023-10-16T09:49:02.560831+0000","last_became_peered":"2023-10-16T09:49:02.560831+0000","last_unstale":"2023-10-16T09:49:03.938071+0000","last_undegraded":"2023-10-16T09:49:03.938071+0000","last_fullsized":"2023-10-16T09:49:03.938071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539917+0000","last_change":"2023-10-16T09:49:02.571717+0000","last_active":"2023-10-16T09:49:03.539917+0000","last_peered":"2023-10-16T09:49:03.539917+0000","last_clean":"2023-10-16T09:49:03.539917+0000","last_became_active":"2023-10-16T09:49:02.571311+0000","last_became_peered":"2023-10-16T09:49:02.571311+0000","last_unstale":"2023-10-16T09:49:03.539917+0000","last_undegraded":"2023-10-16T09:49:03.539917+0000","last_fullsized":"2023-10-16T09:49:03.539917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541271+0000","last_change":"2023-10-16T09:49:02.582038+0000","last_active":"2023-10-16T09:49:03.541271+0000","last_peered":"2023-10-16T09:49:03.541271+0000","last_clean":"2023-10-16T09:49:03.541271+0000","last_became_active":"2023-10-16T09:49:02.581780+0000","last_became_peered":"2023-10-16T09:49:02.581780+0000","last_unstale":"2023-10-16T09:49:03.541271+0000","last_undegraded":"2023-10-16T09:49:03.541271+0000","last_fullsized":"2023-10-16T09:49:03.541271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:45.572589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539221+0000","last_change":"2023-10-16T09:49:02.563686+0000","last_active":"2023-10-16T09:49:03.539221+0000","last_peered":"2023-10-16T09:49:03.539221+0000","last_clean":"2023-10-16T09:49:03.539221+0000","last_became_active":"2023-10-16T09:49:02.562728+0000","last_became_peered":"2023-10-16T09:49:02.562728+0000","last_unstale":"2023-10-16T09:49:03.539221+0000","last_undegraded":"2023-10-16T09:49:03.539221+0000","last_fullsized":"2023-10-16T09:49:03.539221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:58.154049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942765+0000","last_change":"2023-10-16T09:49:02.569238+0000","last_active":"2023-10-16T09:49:03.942765+0000","last_peered":"2023-10-16T09:49:03.942765+0000","last_clean":"2023-10-16T09:49:03.942765+0000","last_became_active":"2023-10-16T09:49:02.568292+0000","last_became_peered":"2023-10-16T09:49:02.568292+0000","last_unstale":"2023-10-16T09:49:03.942765+0000","last_undegraded":"2023-10-16T09:49:03.942765+0000","last_fullsized":"2023-10-16T09:49:03.942765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:46.937999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937917+0000","last_change":"2023-10-16T09:49:02.563047+0000","last_active":"2023-10-16T09:49:03.937917+0000","last_peered":"2023-10-16T09:49:03.937917+0000","last_clean":"2023-10-16T09:49:03.937917+0000","last_became_active":"2023-10-16T09:49:02.562532+0000","last_became_peered":"2023-10-16T09:49:02.562532+0000","last_unstale":"2023-10-16T09:49:03.937917+0000","last_undegraded":"2023-10-16T09:49:03.937917+0000","last_fullsized":"2023-10-16T09:49:03.937917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:33.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939622+0000","last_change":"2023-10-16T09:49:02.569454+0000","last_active":"2023-10-16T09:49:03.939622+0000","last_peered":"2023-10-16T09:49:03.939622+0000","last_clean":"2023-10-16T09:49:03.939622+0000","last_became_active":"2023-10-16T09:49:02.569216+0000","last_became_peered":"2023-10-16T09:49:02.569216+0000","last_unstale":"2023-10-16T09:49:03.939622+0000","last_undegraded":"2023-10-16T09:49:03.939622+0000","last_fullsized":"2023-10-16T09:49:03.939622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:47.112566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540091+0000","last_change":"2023-10-16T09:49:02.572101+0000","last_active":"2023-10-16T09:49:03.540091+0000","last_peered":"2023-10-16T09:49:03.540091+0000","last_clean":"2023-10-16T09:49:03.540091+0000","last_became_active":"2023-10-16T09:49:02.570939+0000","last_became_peered":"2023-10-16T09:49:02.570939+0000","last_unstale":"2023-10-16T09:49:03.540091+0000","last_undegraded":"2023-10-16T09:49:03.540091+0000","last_fullsized":"2023-10-16T09:49:03.540091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540377+0000","last_change":"2023-10-16T09:49:02.562443+0000","last_active":"2023-10-16T09:49:03.540377+0000","last_peered":"2023-10-16T09:49:03.540377+0000","last_clean":"2023-10-16T09:49:03.540377+0000","last_became_active":"2023-10-16T09:49:02.562184+0000","last_became_peered":"2023-10-16T09:49:02.562184+0000","last_unstale":"2023-10-16T09:49:03.540377+0000","last_undegraded":"2023-10-16T09:49:03.540377+0000","last_fullsized":"2023-10-16T09:49:03.540377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:47.112566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540214+0000","last_change":"2023-10-16T09:49:02.566157+0000","last_active":"2023-10-16T09:49:03.540214+0000","last_peered":"2023-10-16T09:49:03.540214+0000","last_clean":"2023-10-16T09:49:03.540214+0000","last_became_active":"2023-10-16T09:49:02.565808+0000","last_became_peered":"2023-10-16T09:49:02.565808+0000","last_unstale":"2023-10-16T09:49:03.540214+0000","last_undegraded":"2023-10-16T09:49:03.540214+0000","last_fullsized":"2023-10-16T09:49:03.540214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:51.170491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540739+0000","last_change":"2023-10-16T09:49:02.570442+0000","last_active":"2023-10-16T09:49:03.540739+0000","last_peered":"2023-10-16T09:49:03.540739+0000","last_clean":"2023-10-16T09:49:03.540739+0000","last_became_active":"2023-10-16T09:49:02.570173+0000","last_became_peered":"2023-10-16T09:49:02.570173+0000","last_unstale":"2023-10-16T09:49:03.540739+0000","last_undegraded":"2023-10-16T09:49:03.540739+0000","last_fullsized":"2023-10-16T09:49:03.540739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938295+0000","last_change":"2023-10-16T09:49:02.565763+0000","last_active":"2023-10-16T09:49:03.938295+0000","last_peered":"2023-10-16T09:49:03.938295+0000","last_clean":"2023-10-16T09:49:03.938295+0000","last_became_active":"2023-10-16T09:49:02.565248+0000","last_became_peered":"2023-10-16T09:49:02.565248+0000","last_unstale":"2023-10-16T09:49:03.938295+0000","last_undegraded":"2023-10-16T09:49:03.938295+0000","last_fullsized":"2023-10-16T09:49:03.938295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938715+0000","last_change":"2023-10-16T09:49:02.556364+0000","last_active":"2023-10-16T09:49:03.938715+0000","last_peered":"2023-10-16T09:49:03.938715+0000","last_clean":"2023-10-16T09:49:03.938715+0000","last_became_active":"2023-10-16T09:49:02.556055+0000","last_became_peered":"2023-10-16T09:49:02.556055+0000","last_unstale":"2023-10-16T09:49:03.938715+0000","last_undegraded":"2023-10-16T09:49:03.938715+0000","last_fullsized":"2023-10-16T09:49:03.938715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541726+0000","last_change":"2023-10-16T09:49:02.564976+0000","last_active":"2023-10-16T09:49:03.541726+0000","last_peered":"2023-10-16T09:49:03.541726+0000","last_clean":"2023-10-16T09:49:03.541726+0000","last_became_active":"2023-10-16T09:49:02.564798+0000","last_became_peered":"2023-10-16T09:49:02.564798+0000","last_unstale":"2023-10-16T09:49:03.541726+0000","last_undegraded":"2023-10-16T09:49:03.541726+0000","last_fullsized":"2023-10-16T09:49:03.541726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:39:22.668380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939780+0000","last_change":"2023-10-16T09:49:02.561691+0000","last_active":"2023-10-16T09:49:03.939780+0000","last_peered":"2023-10-16T09:49:03.939780+0000","last_clean":"2023-10-16T09:49:03.939780+0000","last_became_active":"2023-10-16T09:49:02.561408+0000","last_became_peered":"2023-10-16T09:49:02.561408+0000","last_unstale":"2023-10-16T09:49:03.939780+0000","last_undegraded":"2023-10-16T09:49:03.939780+0000","last_fullsized":"2023-10-16T09:49:03.939780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:10.503481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541186+0000","last_change":"2023-10-16T09:49:02.572702+0000","last_active":"2023-10-16T09:49:03.541186+0000","last_peered":"2023-10-16T09:49:03.541186+0000","last_clean":"2023-10-16T09:49:03.541186+0000","last_became_active":"2023-10-16T09:49:02.571811+0000","last_became_peered":"2023-10-16T09:49:02.571811+0000","last_unstale":"2023-10-16T09:49:03.541186+0000","last_undegraded":"2023-10-16T09:49:03.541186+0000","last_fullsized":"2023-10-16T09:49:03.541186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:08.814427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.538838+0000","last_change":"2023-10-16T09:49:02.576101+0000","last_active":"2023-10-16T09:49:03.538838+0000","last_peered":"2023-10-16T09:49:03.538838+0000","last_clean":"2023-10-16T09:49:03.538838+0000","last_became_active":"2023-10-16T09:49:02.575844+0000","last_became_peered":"2023-10-16T09:49:02.575844+0000","last_unstale":"2023-10-16T09:49:03.538838+0000","last_undegraded":"2023-10-16T09:49:03.538838+0000","last_fullsized":"2023-10-16T09:49:03.538838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:45.119851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937812+0000","last_change":"2023-10-16T09:49:02.568425+0000","last_active":"2023-10-16T09:49:03.937812+0000","last_peered":"2023-10-16T09:49:03.937812+0000","last_clean":"2023-10-16T09:49:03.937812+0000","last_became_active":"2023-10-16T09:49:02.568091+0000","last_became_peered":"2023-10-16T09:49:02.568091+0000","last_unstale":"2023-10-16T09:49:03.937812+0000","last_undegraded":"2023-10-16T09:49:03.937812+0000","last_fullsized":"2023-10-16T09:49:03.937812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:10.503481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940047+0000","last_change":"2023-10-16T09:49:02.559306+0000","last_active":"2023-10-16T09:49:03.940047+0000","last_peered":"2023-10-16T09:49:03.940047+0000","last_clean":"2023-10-16T09:49:03.940047+0000","last_became_active":"2023-10-16T09:49:02.558667+0000","last_became_peered":"2023-10-16T09:49:02.558667+0000","last_unstale":"2023-10-16T09:49:03.940047+0000","last_undegraded":"2023-10-16T09:49:03.940047+0000","last_fullsized":"2023-10-16T09:49:03.940047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:38:53.130875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541754+0000","last_change":"2023-10-16T09:49:01.918393+0000","last_active":"2023-10-16T09:49:03.541754+0000","last_peered":"2023-10-16T09:49:03.541754+0000","last_clean":"2023-10-16T09:49:03.541754+0000","last_became_active":"2023-10-16T09:49:01.918085+0000","last_became_peered":"2023-10-16T09:49:01.918085+0000","last_unstale":"2023-10-16T09:49:03.541754+0000","last_undegraded":"2023-10-16T09:49:03.541754+0000","last_fullsized":"2023-10-16T09:49:03.541754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:24.118554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939938+0000","last_change":"2023-10-16T09:49:02.562307+0000","last_active":"2023-10-16T09:49:03.939938+0000","last_peered":"2023-10-16T09:49:03.939938+0000","last_clean":"2023-10-16T09:49:03.939938+0000","last_became_active":"2023-10-16T09:49:02.562013+0000","last_became_peered":"2023-10-16T09:49:02.562013+0000","last_unstale":"2023-10-16T09:49:03.939938+0000","last_undegraded":"2023-10-16T09:49:03.939938+0000","last_fullsized":"2023-10-16T09:49:03.939938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937479+0000","last_change":"2023-10-16T09:49:01.931642+0000","last_active":"2023-10-16T09:49:03.937479+0000","last_peered":"2023-10-16T09:49:03.937479+0000","last_clean":"2023-10-16T09:49:03.937479+0000","last_became_active":"2023-10-16T09:49:01.929759+0000","last_became_peered":"2023-10-16T09:49:01.929759+0000","last_unstale":"2023-10-16T09:49:03.937479+0000","last_undegraded":"2023-10-16T09:49:03.937479+0000","last_fullsized":"2023-10-16T09:49:03.937479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:56.824003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540519+0000","last_change":"2023-10-16T09:49:01.925581+0000","last_active":"2023-10-16T09:49:03.540519+0000","last_peered":"2023-10-16T09:49:03.540519+0000","last_clean":"2023-10-16T09:49:03.540519+0000","last_became_active":"2023-10-16T09:49:01.925329+0000","last_became_peered":"2023-10-16T09:49:01.925329+0000","last_unstale":"2023-10-16T09:49:03.540519+0000","last_undegraded":"2023-10-16T09:49:03.540519+0000","last_fullsized":"2023-10-16T09:49:03.540519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:12:13.066628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540403+0000","last_change":"2023-10-16T09:49:02.566394+0000","last_active":"2023-10-16T09:49:03.540403+0000","last_peered":"2023-10-16T09:49:03.540403+0000","last_clean":"2023-10-16T09:49:03.540403+0000","last_became_active":"2023-10-16T09:49:02.566059+0000","last_became_peered":"2023-10-16T09:49:02.566059+0000","last_unstale":"2023-10-16T09:49:03.540403+0000","last_undegraded":"2023-10-16T09:49:03.540403+0000","last_fullsized":"2023-10-16T09:49:03.540403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:10.503481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540187+0000","last_change":"2023-10-16T09:49:01.933426+0000","last_active":"2023-10-16T09:49:03.540187+0000","last_peered":"2023-10-16T09:49:03.540187+0000","last_clean":"2023-10-16T09:49:03.540187+0000","last_became_active":"2023-10-16T09:49:01.930300+0000","last_became_peered":"2023-10-16T09:49:01.930300+0000","last_unstale":"2023-10-16T09:49:03.540187+0000","last_undegraded":"2023-10-16T09:49:03.540187+0000","last_fullsized":"2023-10-16T09:49:03.540187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:27.229341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937721+0000","last_change":"2023-10-16T09:49:02.555653+0000","last_active":"2023-10-16T09:49:03.937721+0000","last_peered":"2023-10-16T09:49:03.937721+0000","last_clean":"2023-10-16T09:49:03.937721+0000","last_became_active":"2023-10-16T09:49:02.555099+0000","last_became_peered":"2023-10-16T09:49:02.555099+0000","last_unstale":"2023-10-16T09:49:03.937721+0000","last_undegraded":"2023-10-16T09:49:03.937721+0000","last_fullsized":"2023-10-16T09:49:03.937721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:33.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938399+0000","last_change":"2023-10-16T09:49:01.937059+0000","last_active":"2023-10-16T09:49:03.938399+0000","last_peered":"2023-10-16T09:49:03.938399+0000","last_clean":"2023-10-16T09:49:03.938399+0000","last_became_active":"2023-10-16T09:49:01.936784+0000","last_became_peered":"2023-10-16T09:49:01.936784+0000","last_unstale":"2023-10-16T09:49:03.938399+0000","last_undegraded":"2023-10-16T09:49:03.938399+0000","last_fullsized":"2023-10-16T09:49:03.938399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:24.118554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540473+0000","last_change":"2023-10-16T09:49:02.575679+0000","last_active":"2023-10-16T09:49:03.540473+0000","last_peered":"2023-10-16T09:49:03.540473+0000","last_clean":"2023-10-16T09:49:03.540473+0000","last_became_active":"2023-10-16T09:49:02.575367+0000","last_became_peered":"2023-10-16T09:49:02.575367+0000","last_unstale":"2023-10-16T09:49:03.540473+0000","last_undegraded":"2023-10-16T09:49:03.540473+0000","last_fullsized":"2023-10-16T09:49:03.540473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:54:44.787181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540422+0000","last_change":"2023-10-16T09:49:01.934686+0000","last_active":"2023-10-16T09:49:03.540422+0000","last_peered":"2023-10-16T09:49:03.540422+0000","last_clean":"2023-10-16T09:49:03.540422+0000","last_became_active":"2023-10-16T09:49:01.934414+0000","last_became_peered":"2023-10-16T09:49:01.934414+0000","last_unstale":"2023-10-16T09:49:03.540422+0000","last_undegraded":"2023-10-16T09:49:03.540422+0000","last_fullsized":"2023-10-16T09:49:03.540422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:09.173435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940582+0000","last_change":"2023-10-16T09:49:02.573683+0000","last_active":"2023-10-16T09:49:03.940582+0000","last_peered":"2023-10-16T09:49:03.940582+0000","last_clean":"2023-10-16T09:49:03.940582+0000","last_became_active":"2023-10-16T09:49:02.573331+0000","last_became_peered":"2023-10-16T09:49:02.573331+0000","last_unstale":"2023-10-16T09:49:03.940582+0000","last_undegraded":"2023-10-16T09:49:03.940582+0000","last_fullsized":"2023-10-16T09:49:03.940582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938642+0000","last_change":"2023-10-16T09:49:01.922705+0000","last_active":"2023-10-16T09:49:03.938642+0000","last_peered":"2023-10-16T09:49:03.938642+0000","last_clean":"2023-10-16T09:49:03.938642+0000","last_became_active":"2023-10-16T09:49:01.922292+0000","last_became_peered":"2023-10-16T09:49:01.922292+0000","last_unstale":"2023-10-16T09:49:03.938642+0000","last_undegraded":"2023-10-16T09:49:03.938642+0000","last_fullsized":"2023-10-16T09:49:03.938642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:50.651364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940794+0000","last_change":"2023-10-16T09:49:02.567153+0000","last_active":"2023-10-16T09:49:03.940794+0000","last_peered":"2023-10-16T09:49:03.940794+0000","last_clean":"2023-10-16T09:49:03.940794+0000","last_became_active":"2023-10-16T09:49:02.566840+0000","last_became_peered":"2023-10-16T09:49:02.566840+0000","last_unstale":"2023-10-16T09:49:03.940794+0000","last_undegraded":"2023-10-16T09:49:03.940794+0000","last_fullsized":"2023-10-16T09:49:03.940794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:58.154049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937813+0000","last_change":"2023-10-16T09:49:01.930609+0000","last_active":"2023-10-16T09:49:03.937813+0000","last_peered":"2023-10-16T09:49:03.937813+0000","last_clean":"2023-10-16T09:49:03.937813+0000","last_became_active":"2023-10-16T09:49:01.930235+0000","last_became_peered":"2023-10-16T09:49:01.930235+0000","last_unstale":"2023-10-16T09:49:03.937813+0000","last_undegraded":"2023-10-16T09:49:03.937813+0000","last_fullsized":"2023-10-16T09:49:03.937813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:29:30.922262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942436+0000","last_change":"2023-10-16T09:49:02.577180+0000","last_active":"2023-10-16T09:49:03.942436+0000","last_peered":"2023-10-16T09:49:03.942436+0000","last_clean":"2023-10-16T09:49:03.942436+0000","last_became_active":"2023-10-16T09:49:02.576534+0000","last_became_peered":"2023-10-16T09:49:02.576534+0000","last_unstale":"2023-10-16T09:49:03.942436+0000","last_undegraded":"2023-10-16T09:49:03.942436+0000","last_fullsized":"2023-10-16T09:49:03.942436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:45.572589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940062+0000","last_change":"2023-10-16T09:49:01.918540+0000","last_active":"2023-10-16T09:49:03.940062+0000","last_peered":"2023-10-16T09:49:03.940062+0000","last_clean":"2023-10-16T09:49:03.940062+0000","last_became_active":"2023-10-16T09:49:01.917673+0000","last_became_peered":"2023-10-16T09:49:01.917673+0000","last_unstale":"2023-10-16T09:49:03.940062+0000","last_undegraded":"2023-10-16T09:49:03.940062+0000","last_fullsized":"2023-10-16T09:49:03.940062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:41.350109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939226+0000","last_change":"2023-10-16T09:49:02.570004+0000","last_active":"2023-10-16T09:49:03.939226+0000","last_peered":"2023-10-16T09:49:03.939226+0000","last_clean":"2023-10-16T09:49:03.939226+0000","last_became_active":"2023-10-16T09:49:02.569583+0000","last_became_peered":"2023-10-16T09:49:02.569583+0000","last_unstale":"2023-10-16T09:49:03.939226+0000","last_undegraded":"2023-10-16T09:49:03.939226+0000","last_fullsized":"2023-10-16T09:49:03.939226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:51.981410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540696+0000","last_change":"2023-10-16T09:49:01.932631+0000","last_active":"2023-10-16T09:49:03.540696+0000","last_peered":"2023-10-16T09:49:03.540696+0000","last_clean":"2023-10-16T09:49:03.540696+0000","last_became_active":"2023-10-16T09:49:01.928924+0000","last_became_peered":"2023-10-16T09:49:01.928924+0000","last_unstale":"2023-10-16T09:49:03.540696+0000","last_undegraded":"2023-10-16T09:49:03.540696+0000","last_fullsized":"2023-10-16T09:49:03.540696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:15.945748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942055+0000","last_change":"2023-10-16T09:49:02.569260+0000","last_active":"2023-10-16T09:49:03.942055+0000","last_peered":"2023-10-16T09:49:03.942055+0000","last_clean":"2023-10-16T09:49:03.942055+0000","last_became_active":"2023-10-16T09:49:02.568903+0000","last_became_peered":"2023-10-16T09:49:02.568903+0000","last_unstale":"2023-10-16T09:49:03.942055+0000","last_undegraded":"2023-10-16T09:49:03.942055+0000","last_fullsized":"2023-10-16T09:49:03.942055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:08.814427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937182+0000","last_change":"2023-10-16T09:49:01.930319+0000","last_active":"2023-10-16T09:49:03.937182+0000","last_peered":"2023-10-16T09:49:03.937182+0000","last_clean":"2023-10-16T09:49:03.937182+0000","last_became_active":"2023-10-16T09:49:01.928990+0000","last_became_peered":"2023-10-16T09:49:01.928990+0000","last_unstale":"2023-10-16T09:49:03.937182+0000","last_undegraded":"2023-10-16T09:49:03.937182+0000","last_fullsized":"2023-10-16T09:49:03.937182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:43.789805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937440+0000","last_change":"2023-10-16T09:49:02.568938+0000","last_active":"2023-10-16T09:49:03.937440+0000","last_peered":"2023-10-16T09:49:03.937440+0000","last_clean":"2023-10-16T09:49:03.937440+0000","last_became_active":"2023-10-16T09:49:02.568222+0000","last_became_peered":"2023-10-16T09:49:02.568222+0000","last_unstale":"2023-10-16T09:49:03.937440+0000","last_undegraded":"2023-10-16T09:49:03.937440+0000","last_fullsized":"2023-10-16T09:49:03.937440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:47.112566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.538731+0000","last_change":"2023-10-16T09:49:01.913071+0000","last_active":"2023-10-16T09:49:03.538731+0000","last_peered":"2023-10-16T09:49:03.538731+0000","last_clean":"2023-10-16T09:49:03.538731+0000","last_became_active":"2023-10-16T09:49:01.912124+0000","last_became_peered":"2023-10-16T09:49:01.912124+0000","last_unstale":"2023-10-16T09:49:03.538731+0000","last_undegraded":"2023-10-16T09:49:03.538731+0000","last_fullsized":"2023-10-16T09:49:03.538731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:16.897247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941822+0000","last_change":"2023-10-16T09:49:02.577394+0000","last_active":"2023-10-16T09:49:03.941822+0000","last_peered":"2023-10-16T09:49:03.941822+0000","last_clean":"2023-10-16T09:49:03.941822+0000","last_became_active":"2023-10-16T09:49:02.577042+0000","last_became_peered":"2023-10-16T09:49:02.577042+0000","last_unstale":"2023-10-16T09:49:03.941822+0000","last_undegraded":"2023-10-16T09:49:03.941822+0000","last_fullsized":"2023-10-16T09:49:03.941822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:05.838911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941102+0000","last_change":"2023-10-16T09:49:01.918367+0000","last_active":"2023-10-16T09:49:03.941102+0000","last_peered":"2023-10-16T09:49:03.941102+0000","last_clean":"2023-10-16T09:49:03.941102+0000","last_became_active":"2023-10-16T09:49:01.918021+0000","last_became_peered":"2023-10-16T09:49:01.918021+0000","last_unstale":"2023-10-16T09:49:03.941102+0000","last_undegraded":"2023-10-16T09:49:03.941102+0000","last_fullsized":"2023-10-16T09:49:03.941102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:25.972592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942743+0000","last_change":"2023-10-16T09:49:02.567390+0000","last_active":"2023-10-16T09:49:03.942743+0000","last_peered":"2023-10-16T09:49:03.942743+0000","last_clean":"2023-10-16T09:49:03.942743+0000","last_became_active":"2023-10-16T09:49:02.566827+0000","last_became_peered":"2023-10-16T09:49:02.566827+0000","last_unstale":"2023-10-16T09:49:03.942743+0000","last_undegraded":"2023-10-16T09:49:03.942743+0000","last_fullsized":"2023-10-16T09:49:03.942743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:40:56.166477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539098+0000","last_change":"2023-10-16T09:49:01.904674+0000","last_active":"2023-10-16T09:49:03.539098+0000","last_peered":"2023-10-16T09:49:03.539098+0000","last_clean":"2023-10-16T09:49:03.539098+0000","last_became_active":"2023-10-16T09:49:01.904380+0000","last_became_peered":"2023-10-16T09:49:01.904380+0000","last_unstale":"2023-10-16T09:49:03.539098+0000","last_undegraded":"2023-10-16T09:49:03.539098+0000","last_fullsized":"2023-10-16T09:49:03.539098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:25.972592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.943170+0000","last_change":"2023-10-16T09:49:02.575633+0000","last_active":"2023-10-16T09:49:03.943170+0000","last_peered":"2023-10-16T09:49:03.943170+0000","last_clean":"2023-10-16T09:49:03.943170+0000","last_became_active":"2023-10-16T09:49:02.573187+0000","last_became_peered":"2023-10-16T09:49:02.573187+0000","last_unstale":"2023-10-16T09:49:03.943170+0000","last_undegraded":"2023-10-16T09:49:03.943170+0000","last_fullsized":"2023-10-16T09:49:03.943170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:12.370135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938903+0000","last_change":"2023-10-16T09:49:01.911986+0000","last_active":"2023-10-16T09:49:03.938903+0000","last_peered":"2023-10-16T09:49:03.938903+0000","last_clean":"2023-10-16T09:49:03.938903+0000","last_became_active":"2023-10-16T09:49:01.911700+0000","last_became_peered":"2023-10-16T09:49:01.911700+0000","last_unstale":"2023-10-16T09:49:03.938903+0000","last_undegraded":"2023-10-16T09:49:03.938903+0000","last_fullsized":"2023-10-16T09:49:03.938903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:45.782520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540965+0000","last_change":"2023-10-16T09:49:02.574594+0000","last_active":"2023-10-16T09:49:03.540965+0000","last_peered":"2023-10-16T09:49:03.540965+0000","last_clean":"2023-10-16T09:49:03.540965+0000","last_became_active":"2023-10-16T09:49:02.573419+0000","last_became_peered":"2023-10-16T09:49:02.573419+0000","last_unstale":"2023-10-16T09:49:03.540965+0000","last_undegraded":"2023-10-16T09:49:03.540965+0000","last_fullsized":"2023-10-16T09:49:03.540965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:55.226843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539291+0000","last_change":"2023-10-16T09:49:01.934196+0000","last_active":"2023-10-16T09:49:03.539291+0000","last_peered":"2023-10-16T09:49:03.539291+0000","last_clean":"2023-10-16T09:49:03.539291+0000","last_became_active":"2023-10-16T09:49:01.933922+0000","last_became_peered":"2023-10-16T09:49:01.933922+0000","last_unstale":"2023-10-16T09:49:03.539291+0000","last_undegraded":"2023-10-16T09:49:03.539291+0000","last_fullsized":"2023-10-16T09:49:03.539291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:10:19.173392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942391+0000","last_change":"2023-10-16T09:49:02.577848+0000","last_active":"2023-10-16T09:49:03.942391+0000","last_peered":"2023-10-16T09:49:03.942391+0000","last_clean":"2023-10-16T09:49:03.942391+0000","last_became_active":"2023-10-16T09:49:02.577571+0000","last_became_peered":"2023-10-16T09:49:02.577571+0000","last_unstale":"2023-10-16T09:49:03.942391+0000","last_undegraded":"2023-10-16T09:49:03.942391+0000","last_fullsized":"2023-10-16T09:49:03.942391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:12:32.747232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T09:49:02.548285+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T09:49:02.548285+0000","last_peered":"2023-10-16T09:49:02.548285+0000","last_clean":"2023-10-16T09:49:02.548285+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T09:49:02.548285+0000","last_undegraded":"2023-10-16T09:49:02.548285+0000","last_fullsized":"2023-10-16T09:49:02.548285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539903+0000","last_change":"2023-10-16T09:49:01.922671+0000","last_active":"2023-10-16T09:49:03.539903+0000","last_peered":"2023-10-16T09:49:03.539903+0000","last_clean":"2023-10-16T09:49:03.539903+0000","last_became_active":"2023-10-16T09:49:01.922356+0000","last_became_peered":"2023-10-16T09:49:01.922356+0000","last_unstale":"2023-10-16T09:49:03.539903+0000","last_undegraded":"2023-10-16T09:49:03.539903+0000","last_fullsized":"2023-10-16T09:49:03.539903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:10:19.173392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.542081+0000","last_change":"2023-10-16T09:49:02.572185+0000","last_active":"2023-10-16T09:49:03.542081+0000","last_peered":"2023-10-16T09:49:03.542081+0000","last_clean":"2023-10-16T09:49:03.542081+0000","last_became_active":"2023-10-16T09:49:02.571953+0000","last_became_peered":"2023-10-16T09:49:02.571953+0000","last_unstale":"2023-10-16T09:49:03.542081+0000","last_undegraded":"2023-10-16T09:49:03.542081+0000","last_fullsized":"2023-10-16T09:49:03.542081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:07.724831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941421+0000","last_change":"2023-10-16T09:49:01.925742+0000","last_active":"2023-10-16T09:49:03.941421+0000","last_peered":"2023-10-16T09:49:03.941421+0000","last_clean":"2023-10-16T09:49:03.941421+0000","last_became_active":"2023-10-16T09:49:01.925438+0000","last_became_peered":"2023-10-16T09:49:01.925438+0000","last_unstale":"2023-10-16T09:49:03.941421+0000","last_undegraded":"2023-10-16T09:49:03.941421+0000","last_fullsized":"2023-10-16T09:49:03.941421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:56.824003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938454+0000","last_change":"2023-10-16T09:49:02.562256+0000","last_active":"2023-10-16T09:49:03.938454+0000","last_peered":"2023-10-16T09:49:03.938454+0000","last_clean":"2023-10-16T09:49:03.938454+0000","last_became_active":"2023-10-16T09:49:02.561942+0000","last_became_peered":"2023-10-16T09:49:02.561942+0000","last_unstale":"2023-10-16T09:49:03.938454+0000","last_undegraded":"2023-10-16T09:49:03.938454+0000","last_fullsized":"2023-10-16T09:49:03.938454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:12:14.396674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.538971+0000","last_change":"2023-10-16T09:49:01.905306+0000","last_active":"2023-10-16T09:49:03.538971+0000","last_peered":"2023-10-16T09:49:03.538971+0000","last_clean":"2023-10-16T09:49:03.538971+0000","last_became_active":"2023-10-16T09:49:01.904664+0000","last_became_peered":"2023-10-16T09:49:01.904664+0000","last_unstale":"2023-10-16T09:49:03.538971+0000","last_undegraded":"2023-10-16T09:49:03.538971+0000","last_fullsized":"2023-10-16T09:49:03.538971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:38:51.800830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939222+0000","last_change":"2023-10-16T09:49:02.551068+0000","last_active":"2023-10-16T09:49:03.939222+0000","last_peered":"2023-10-16T09:49:03.939222+0000","last_clean":"2023-10-16T09:49:03.939222+0000","last_became_active":"2023-10-16T09:49:02.550772+0000","last_became_peered":"2023-10-16T09:49:02.550772+0000","last_unstale":"2023-10-16T09:49:03.939222+0000","last_undegraded":"2023-10-16T09:49:03.939222+0000","last_fullsized":"2023-10-16T09:49:03.939222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:51.170491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540961+0000","last_change":"2023-10-16T09:49:01.932431+0000","last_active":"2023-10-16T09:49:03.540961+0000","last_peered":"2023-10-16T09:49:03.540961+0000","last_clean":"2023-10-16T09:49:03.540961+0000","last_became_active":"2023-10-16T09:49:01.927598+0000","last_became_peered":"2023-10-16T09:49:01.927598+0000","last_unstale":"2023-10-16T09:49:03.540961+0000","last_undegraded":"2023-10-16T09:49:03.540961+0000","last_fullsized":"2023-10-16T09:49:03.540961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:50.651364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940795+0000","last_change":"2023-10-16T09:49:02.571518+0000","last_active":"2023-10-16T09:49:03.940795+0000","last_peered":"2023-10-16T09:49:03.940795+0000","last_clean":"2023-10-16T09:49:03.940795+0000","last_became_active":"2023-10-16T09:49:02.571243+0000","last_became_peered":"2023-10-16T09:49:02.571243+0000","last_unstale":"2023-10-16T09:49:03.940795+0000","last_undegraded":"2023-10-16T09:49:03.940795+0000","last_fullsized":"2023-10-16T09:49:03.940795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:45.119851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937987+0000","last_change":"2023-10-16T09:49:01.928253+0000","last_active":"2023-10-16T09:49:03.937987+0000","last_peered":"2023-10-16T09:49:03.937987+0000","last_clean":"2023-10-16T09:49:03.937987+0000","last_became_active":"2023-10-16T09:49:01.927808+0000","last_became_peered":"2023-10-16T09:49:01.927808+0000","last_unstale":"2023-10-16T09:49:03.937987+0000","last_undegraded":"2023-10-16T09:49:03.937987+0000","last_fullsized":"2023-10-16T09:49:03.937987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:41.350109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541226+0000","last_change":"2023-10-16T09:49:02.575272+0000","last_active":"2023-10-16T09:49:03.541226+0000","last_peered":"2023-10-16T09:49:03.541226+0000","last_clean":"2023-10-16T09:49:03.541226+0000","last_became_active":"2023-10-16T09:49:02.574888+0000","last_became_peered":"2023-10-16T09:49:02.574888+0000","last_unstale":"2023-10-16T09:49:03.541226+0000","last_undegraded":"2023-10-16T09:49:03.541226+0000","last_fullsized":"2023-10-16T09:49:03.541226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:12:32.747232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942771+0000","last_change":"2023-10-16T09:49:01.931740+0000","last_active":"2023-10-16T09:49:03.942771+0000","last_peered":"2023-10-16T09:49:03.942771+0000","last_clean":"2023-10-16T09:49:03.942771+0000","last_became_active":"2023-10-16T09:49:01.931370+0000","last_became_peered":"2023-10-16T09:49:01.931370+0000","last_unstale":"2023-10-16T09:49:03.942771+0000","last_undegraded":"2023-10-16T09:49:03.942771+0000","last_fullsized":"2023-10-16T09:49:03.942771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:49.840445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937766+0000","last_change":"2023-10-16T09:49:02.553220+0000","last_active":"2023-10-16T09:49:03.937766+0000","last_peered":"2023-10-16T09:49:03.937766+0000","last_clean":"2023-10-16T09:49:03.937766+0000","last_became_active":"2023-10-16T09:49:02.552703+0000","last_became_peered":"2023-10-16T09:49:02.552703+0000","last_unstale":"2023-10-16T09:49:03.937766+0000","last_undegraded":"2023-10-16T09:49:03.937766+0000","last_fullsized":"2023-10-16T09:49:03.937766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:04.627058+0000","last_change":"2023-10-16T09:49:01.928063+0000","last_active":"2023-10-16T09:49:04.627058+0000","last_peered":"2023-10-16T09:49:04.627058+0000","last_clean":"2023-10-16T09:49:04.627058+0000","last_became_active":"2023-10-16T09:49:01.927770+0000","last_became_peered":"2023-10-16T09:49:01.927770+0000","last_unstale":"2023-10-16T09:49:04.627058+0000","last_undegraded":"2023-10-16T09:49:04.627058+0000","last_fullsized":"2023-10-16T09:49:04.627058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:12:13.066628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540860+0000","last_change":"2023-10-16T09:49:02.562667+0000","last_active":"2023-10-16T09:49:03.540860+0000","last_peered":"2023-10-16T09:49:03.540860+0000","last_clean":"2023-10-16T09:49:03.540860+0000","last_became_active":"2023-10-16T09:49:02.562361+0000","last_became_peered":"2023-10-16T09:49:02.562361+0000","last_unstale":"2023-10-16T09:49:03.540860+0000","last_undegraded":"2023-10-16T09:49:03.540860+0000","last_fullsized":"2023-10-16T09:49:03.540860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938522+0000","last_change":"2023-10-16T09:49:01.930319+0000","last_active":"2023-10-16T09:49:03.938522+0000","last_peered":"2023-10-16T09:49:03.938522+0000","last_clean":"2023-10-16T09:49:03.938522+0000","last_became_active":"2023-10-16T09:49:01.929983+0000","last_became_peered":"2023-10-16T09:49:01.929983+0000","last_unstale":"2023-10-16T09:49:03.938522+0000","last_undegraded":"2023-10-16T09:49:03.938522+0000","last_fullsized":"2023-10-16T09:49:03.938522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:12:13.066628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939914+0000","last_change":"2023-10-16T09:49:02.567693+0000","last_active":"2023-10-16T09:49:03.939914+0000","last_peered":"2023-10-16T09:49:03.939914+0000","last_clean":"2023-10-16T09:49:03.939914+0000","last_became_active":"2023-10-16T09:49:02.566400+0000","last_became_peered":"2023-10-16T09:49:02.566400+0000","last_unstale":"2023-10-16T09:49:03.939914+0000","last_undegraded":"2023-10-16T09:49:03.939914+0000","last_fullsized":"2023-10-16T09:49:03.939914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:33.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941454+0000","last_change":"2023-10-16T09:49:01.929977+0000","last_active":"2023-10-16T09:49:03.941454+0000","last_peered":"2023-10-16T09:49:03.941454+0000","last_clean":"2023-10-16T09:49:03.941454+0000","last_became_active":"2023-10-16T09:49:01.928703+0000","last_became_peered":"2023-10-16T09:49:01.928703+0000","last_unstale":"2023-10-16T09:49:03.941454+0000","last_undegraded":"2023-10-16T09:49:03.941454+0000","last_fullsized":"2023-10-16T09:49:03.941454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:43.789805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940325+0000","last_change":"2023-10-16T09:49:02.558651+0000","last_active":"2023-10-16T09:49:03.940325+0000","last_peered":"2023-10-16T09:49:03.940325+0000","last_clean":"2023-10-16T09:49:03.940325+0000","last_became_active":"2023-10-16T09:49:02.554370+0000","last_became_peered":"2023-10-16T09:49:02.554370+0000","last_unstale":"2023-10-16T09:49:03.940325+0000","last_undegraded":"2023-10-16T09:49:03.940325+0000","last_fullsized":"2023-10-16T09:49:03.940325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:51:17.179081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938321+0000","last_change":"2023-10-16T09:49:01.918105+0000","last_active":"2023-10-16T09:49:03.938321+0000","last_peered":"2023-10-16T09:49:03.938321+0000","last_clean":"2023-10-16T09:49:03.938321+0000","last_became_active":"2023-10-16T09:49:01.917625+0000","last_became_peered":"2023-10-16T09:49:01.917625+0000","last_unstale":"2023-10-16T09:49:03.938321+0000","last_undegraded":"2023-10-16T09:49:03.938321+0000","last_fullsized":"2023-10-16T09:49:03.938321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:24.118554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540666+0000","last_change":"2023-10-16T09:49:02.556320+0000","last_active":"2023-10-16T09:49:03.540666+0000","last_peered":"2023-10-16T09:49:03.540666+0000","last_clean":"2023-10-16T09:49:03.540666+0000","last_became_active":"2023-10-16T09:49:02.555897+0000","last_became_peered":"2023-10-16T09:49:02.555897+0000","last_unstale":"2023-10-16T09:49:03.540666+0000","last_undegraded":"2023-10-16T09:49:03.540666+0000","last_fullsized":"2023-10-16T09:49:03.540666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:33.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938761+0000","last_change":"2023-10-16T09:49:01.915222+0000","last_active":"2023-10-16T09:49:03.938761+0000","last_peered":"2023-10-16T09:49:03.938761+0000","last_clean":"2023-10-16T09:49:03.938761+0000","last_became_active":"2023-10-16T09:49:01.914843+0000","last_became_peered":"2023-10-16T09:49:01.914843+0000","last_unstale":"2023-10-16T09:49:03.938761+0000","last_undegraded":"2023-10-16T09:49:03.938761+0000","last_fullsized":"2023-10-16T09:49:03.938761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:51:15.849036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541605+0000","last_change":"2023-10-16T09:49:02.567819+0000","last_active":"2023-10-16T09:49:03.541605+0000","last_peered":"2023-10-16T09:49:03.541605+0000","last_clean":"2023-10-16T09:49:03.541605+0000","last_became_active":"2023-10-16T09:49:02.567548+0000","last_became_peered":"2023-10-16T09:49:02.567548+0000","last_unstale":"2023-10-16T09:49:03.541605+0000","last_undegraded":"2023-10-16T09:49:03.541605+0000","last_fullsized":"2023-10-16T09:49:03.541605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:54:44.787181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942035+0000","last_change":"2023-10-16T09:49:01.924408+0000","last_active":"2023-10-16T09:49:03.942035+0000","last_peered":"2023-10-16T09:49:03.942035+0000","last_clean":"2023-10-16T09:49:03.942035+0000","last_became_active":"2023-10-16T09:49:01.922411+0000","last_became_peered":"2023-10-16T09:49:01.922411+0000","last_unstale":"2023-10-16T09:49:03.942035+0000","last_undegraded":"2023-10-16T09:49:03.942035+0000","last_fullsized":"2023-10-16T09:49:03.942035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:24.118554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937385+0000","last_change":"2023-10-16T09:49:02.560941+0000","last_active":"2023-10-16T09:49:03.937385+0000","last_peered":"2023-10-16T09:49:03.937385+0000","last_clean":"2023-10-16T09:49:03.937385+0000","last_became_active":"2023-10-16T09:49:02.560673+0000","last_became_peered":"2023-10-16T09:49:02.560673+0000","last_unstale":"2023-10-16T09:49:03.937385+0000","last_undegraded":"2023-10-16T09:49:03.937385+0000","last_fullsized":"2023-10-16T09:49:03.937385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:51.981410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540856+0000","last_change":"2023-10-16T09:49:02.575954+0000","last_active":"2023-10-16T09:49:03.540856+0000","last_peered":"2023-10-16T09:49:03.540856+0000","last_clean":"2023-10-16T09:49:03.540856+0000","last_became_active":"2023-10-16T09:49:02.575517+0000","last_became_peered":"2023-10-16T09:49:02.575517+0000","last_unstale":"2023-10-16T09:49:03.540856+0000","last_undegraded":"2023-10-16T09:49:03.540856+0000","last_fullsized":"2023-10-16T09:49:03.540856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:07.724831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942265+0000","last_change":"2023-10-16T09:49:01.933911+0000","last_active":"2023-10-16T09:49:03.942265+0000","last_peered":"2023-10-16T09:49:03.942265+0000","last_clean":"2023-10-16T09:49:03.942265+0000","last_became_active":"2023-10-16T09:49:01.933640+0000","last_became_peered":"2023-10-16T09:49:01.933640+0000","last_unstale":"2023-10-16T09:49:03.942265+0000","last_undegraded":"2023-10-16T09:49:03.942265+0000","last_fullsized":"2023-10-16T09:49:03.942265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:27.229341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540639+0000","last_change":"2023-10-16T09:49:02.580360+0000","last_active":"2023-10-16T09:49:03.540639+0000","last_peered":"2023-10-16T09:49:03.540639+0000","last_clean":"2023-10-16T09:49:03.540639+0000","last_became_active":"2023-10-16T09:49:02.574225+0000","last_became_peered":"2023-10-16T09:49:02.574225+0000","last_unstale":"2023-10-16T09:49:03.540639+0000","last_undegraded":"2023-10-16T09:49:03.540639+0000","last_fullsized":"2023-10-16T09:49:03.540639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:39:22.668380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942632+0000","last_change":"2023-10-16T09:49:01.931685+0000","last_active":"2023-10-16T09:49:03.942632+0000","last_peered":"2023-10-16T09:49:03.942632+0000","last_clean":"2023-10-16T09:49:03.942632+0000","last_became_active":"2023-10-16T09:49:01.930733+0000","last_became_peered":"2023-10-16T09:49:01.930733+0000","last_unstale":"2023-10-16T09:49:03.942632+0000","last_undegraded":"2023-10-16T09:49:03.942632+0000","last_fullsized":"2023-10-16T09:49:03.942632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:09.173435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940714+0000","last_change":"2023-10-16T09:49:02.573451+0000","last_active":"2023-10-16T09:49:03.940714+0000","last_peered":"2023-10-16T09:49:03.940714+0000","last_clean":"2023-10-16T09:49:03.940714+0000","last_became_active":"2023-10-16T09:49:02.572970+0000","last_became_peered":"2023-10-16T09:49:02.572970+0000","last_unstale":"2023-10-16T09:49:03.940714+0000","last_undegraded":"2023-10-16T09:49:03.940714+0000","last_fullsized":"2023-10-16T09:49:03.940714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:51.170491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540855+0000","last_change":"2023-10-16T09:49:01.920578+0000","last_active":"2023-10-16T09:49:03.540855+0000","last_peered":"2023-10-16T09:49:03.540855+0000","last_clean":"2023-10-16T09:49:03.540855+0000","last_became_active":"2023-10-16T09:49:01.919963+0000","last_became_peered":"2023-10-16T09:49:01.919963+0000","last_unstale":"2023-10-16T09:49:03.540855+0000","last_undegraded":"2023-10-16T09:49:03.540855+0000","last_fullsized":"2023-10-16T09:49:03.540855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:15.945748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540674+0000","last_change":"2023-10-16T09:49:02.563034+0000","last_active":"2023-10-16T09:49:03.540674+0000","last_peered":"2023-10-16T09:49:03.540674+0000","last_clean":"2023-10-16T09:49:03.540674+0000","last_became_active":"2023-10-16T09:49:02.562680+0000","last_became_peered":"2023-10-16T09:49:02.562680+0000","last_unstale":"2023-10-16T09:49:03.540674+0000","last_undegraded":"2023-10-16T09:49:03.540674+0000","last_fullsized":"2023-10-16T09:49:03.540674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940780+0000","last_change":"2023-10-16T09:49:01.919451+0000","last_active":"2023-10-16T09:49:03.940780+0000","last_peered":"2023-10-16T09:49:03.940780+0000","last_clean":"2023-10-16T09:49:03.940780+0000","last_became_active":"2023-10-16T09:49:01.919106+0000","last_became_peered":"2023-10-16T09:49:01.919106+0000","last_unstale":"2023-10-16T09:49:03.940780+0000","last_undegraded":"2023-10-16T09:49:03.940780+0000","last_fullsized":"2023-10-16T09:49:03.940780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:29:30.922262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":815104,"data_stored":666519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4540000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.399}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.355}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.224}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.722}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.621}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7949999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7290000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8129999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.028}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9850000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":77181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8640000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.74}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6379999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8280000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0819999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9119999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6059999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6319999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7770000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.377}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.667}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7410000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8169999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.782}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":823296,"data_stored":666575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4830000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.643}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.605}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.548}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5229999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:49:14.666 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:49:14.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.908+0000 7f75b6f2c640 1 Processor -- start 2023-10-16T09:49:14.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.908+0000 7f75b6f2c640 1 -- start start 2023-10-16T09:49:14.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.909+0000 7f75b6f2c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f75b0159b20 0x7f75b006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:14.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.909+0000 7f75b6f2c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75b0159f00 0x7f75b00b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:14.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.909+0000 7f75b6f2c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b006f7f0 con 0x7f75b006c120 2023-10-16T09:49:14.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.909+0000 7f75b6f2c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b006c5e0 con 0x7f75b0159b20 2023-10-16T09:49:14.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.909+0000 7f75b6f2c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b006c8c0 con 0x7f75b0159f00 2023-10-16T09:49:14.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.909+0000 7f75b54c9640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f75b006c120 0x7f75b00cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32972/0 (socket says 172.21.15.84:32972) 2023-10-16T09:49:14.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.909+0000 7f75b54c9640 1 -- 172.21.15.84:0/1428667143 learned_addr learned my addr 172.21.15.84:0/1428667143 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:14.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75af7fe640 1 -- 172.21.15.84:0/1428667143 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098121952 0 0) 0x7f75b006f7f0 con 0x7f75b006c120 2023-10-16T09:49:14.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75affff640 1 --2- 172.21.15.84:0/1428667143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75b0159f00 0x7f75b00b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:14.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75af7fe640 1 -- 172.21.15.84:0/1428667143 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f759c003520 con 0x7f75b006c120 2023-10-16T09:49:14.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75b4cc8640 1 --2- 172.21.15.84:0/1428667143 >> v2:172.21.15.157:3300/0 conn(0x7f75b0159b20 0x7f75b006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:14.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75af7fe640 1 -- 172.21.15.84:0/1428667143 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1442539752 0 0) 0x7f759c003520 con 0x7f75b006c120 2023-10-16T09:49:14.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75af7fe640 1 -- 172.21.15.84:0/1428667143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75b0159f00 msgr2=0x7f75b00b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:14.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75af7fe640 1 --2- 172.21.15.84:0/1428667143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75b0159f00 0x7f75b00b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75af7fe640 1 -- 172.21.15.84:0/1428667143 >> v2:172.21.15.157:3300/0 conn(0x7f75b0159b20 msgr2=0x7f75b006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:14.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75af7fe640 1 --2- 172.21.15.84:0/1428667143 >> v2:172.21.15.157:3300/0 conn(0x7f75b0159b20 0x7f75b006bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75af7fe640 1 -- 172.21.15.84:0/1428667143 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b006cba0 con 0x7f75b006c120 2023-10-16T09:49:14.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75af7fe640 1 -- 172.21.15.84:0/1428667143 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f75a8001670 con 0x7f75b006c120 2023-10-16T09:49:14.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75af7fe640 1 -- 172.21.15.84:0/1428667143 >> v1:172.21.15.84:6789/0 conn(0x7f75b006c120 legacy=0x7f75b00cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75af7fe640 1 --2- 172.21.15.84:0/1428667143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f759c003970 0x7f759c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:14.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75af7fe640 1 --2- 172.21.15.84:0/1428667143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f759c0047c0 0x7f759c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:14.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75af7fe640 1 --2- 172.21.15.84:0/1428667143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75b0159f00 0x7f759c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:14.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75af7fe640 1 -- 172.21.15.84:0/1428667143 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b006c5e0 con 0x7f759c0047c0 2023-10-16T09:49:14.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75af7fe640 1 -- 172.21.15.84:0/1428667143 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b006c8c0 con 0x7f759c003970 2023-10-16T09:49:14.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75af7fe640 1 -- 172.21.15.84:0/1428667143 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759c00b980 con 0x7f75b0159f00 2023-10-16T09:49:14.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.910+0000 7f75b4cc8640 1 --2- 172.21.15.84:0/1428667143 >> v2:172.21.15.157:3300/0 conn(0x7f75b0159b20 0x7f75b006bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:14.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.911+0000 7f75affff640 1 --2- 172.21.15.84:0/1428667143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f759c0047c0 0x7f759c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:14.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.911+0000 7f75affff640 1 -- 172.21.15.84:0/1428667143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75b0159f00 msgr2=0x7f759c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:14.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.911+0000 7f75affff640 1 --2- 172.21.15.84:0/1428667143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75b0159f00 0x7f759c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.911+0000 7f75affff640 1 -- 172.21.15.84:0/1428667143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f759c003970 msgr2=0x7f759c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:14.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.911+0000 7f75affff640 1 --2- 172.21.15.84:0/1428667143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f759c003970 0x7f759c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.911+0000 7f75affff640 1 -- 172.21.15.84:0/1428667143 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c003520 con 0x7f759c0047c0 2023-10-16T09:49:14.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.912+0000 7f75affff640 1 --2- 172.21.15.84:0/1428667143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f759c0047c0 0x7f759c004c40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7598009a90 tx=0x7f759c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d21313787fda91 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:14.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.912+0000 7f75af7fe640 1 -- 172.21.15.84:0/1428667143 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759802c6b0 con 0x7f759c0047c0 2023-10-16T09:49:14.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.912+0000 7f75af7fe640 1 -- 172.21.15.84:0/1428667143 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759802c810 con 0x7f759c0047c0 2023-10-16T09:49:14.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.912+0000 7f75af7fe640 1 -- 172.21.15.84:0/1428667143 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759802cba0 con 0x7f759c0047c0 2023-10-16T09:49:14.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.913+0000 7f75b6f2c640 1 -- 172.21.15.84:0/1428667143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f759c0047c0 msgr2=0x7f759c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.913+0000 7f75b6f2c640 1 --2- 172.21.15.84:0/1428667143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f759c0047c0 0x7f759c004c40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7598009a90 tx=0x7f759c0057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.913+0000 7f75b54c9640 1 -- 172.21.15.84:0/1428667143 reap_dead start 2023-10-16T09:49:14.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.913+0000 7f75b6f2c640 1 -- 172.21.15.84:0/1428667143 shutdown_connections 2023-10-16T09:49:14.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.913+0000 7f75b6f2c640 1 -- 172.21.15.84:0/1428667143 >> 172.21.15.84:0/1428667143 conn(0x7f75b00cb410 msgr2=0x7f75b00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:14.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.913+0000 7f75b6f2c640 1 -- 172.21.15.84:0/1428667143 shutdown_connections 2023-10-16T09:49:14.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.913+0000 7f75b6f2c640 1 -- 172.21.15.84:0/1428667143 wait complete. 2023-10-16T09:49:14.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.914+0000 7f75b6f2c640 1 Processor -- start 2023-10-16T09:49:14.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.915+0000 7f75b6f2c640 1 -- start start 2023-10-16T09:49:14.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.915+0000 7f75b6f2c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f759c0047c0 0x7f75b015cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:14.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.915+0000 7f75b6f2c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b00b3a10 0x7f75b0150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:14.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.915+0000 7f75b6f2c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b00c24c0 0x7f75b00c2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:14.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.915+0000 7f75b6f2c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b015d5c0 con 0x7f75b00b3a10 2023-10-16T09:49:14.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.915+0000 7f75b6f2c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00c2e80 con 0x7f75b00c24c0 2023-10-16T09:49:14.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.915+0000 7f75b6f2c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b01f4d20 con 0x7f759c0047c0 2023-10-16T09:49:14.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.916+0000 7f75affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b00b3a10 0x7f75b0150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:14.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.916+0000 7f75affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b00b3a10 0x7f75b0150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59954/0 (socket says 172.21.15.84:59954) 2023-10-16T09:49:14.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.916+0000 7f75affff640 1 -- 172.21.15.84:0/771687090 learned_addr learned my addr 172.21.15.84:0/771687090 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:14.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.916+0000 7f75affff640 1 -- 172.21.15.84:0/771687090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f759c0047c0 msgr2=0x7f75b015cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.916+0000 7f75affff640 1 --2- 172.21.15.84:0/771687090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f759c0047c0 0x7f75b015cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.916+0000 7f75affff640 1 -- 172.21.15.84:0/771687090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b00c24c0 msgr2=0x7f75b00c2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:14.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.916+0000 7f75affff640 1 --2- 172.21.15.84:0/771687090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b00c24c0 0x7f75b00c2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:14.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.916+0000 7f75affff640 1 -- 172.21.15.84:0/771687090 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b0151400 con 0x7f75b00b3a10 2023-10-16T09:49:14.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.916+0000 7f75affff640 1 --2- 172.21.15.84:0/771687090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b00b3a10 0x7f75b0150ec0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7598037040 tx=0x7f7598004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:14.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.917+0000 7f75ad7fa640 1 -- 172.21.15.84:0/771687090 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7598003810 con 0x7f75b00b3a10 2023-10-16T09:49:14.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.917+0000 7f75ad7fa640 1 -- 172.21.15.84:0/771687090 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7598003970 con 0x7f75b00b3a10 2023-10-16T09:49:14.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.917+0000 7f75ad7fa640 1 -- 172.21.15.84:0/771687090 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7598003ce0 con 0x7f75b00b3a10 2023-10-16T09:49:14.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.917+0000 7f75b6f2c640 1 -- 172.21.15.84:0/771687090 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b01f3f60 con 0x7f75b00b3a10 2023-10-16T09:49:14.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.917+0000 7f75b6f2c640 1 -- 172.21.15.84:0/771687090 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b01f4520 con 0x7f75b00b3a10 2023-10-16T09:49:14.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.918+0000 7f75b6f2c640 1 -- 172.21.15.84:0/771687090 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b004e160 con 0x7f75b00b3a10 2023-10-16T09:49:14.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.919+0000 7f75ad7fa640 1 -- 172.21.15.84:0/771687090 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7598034440 con 0x7f75b00b3a10 2023-10-16T09:49:14.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.920+0000 7f75ad7fa640 1 --2- 172.21.15.84:0/771687090 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f759006aea0 0x7f759006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:14.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.920+0000 7f75ad7fa640 1 -- 172.21.15.84:0/771687090 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75980ab7c0 con 0x7f75b00b3a10 2023-10-16T09:49:14.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.920+0000 7f75b4cc8640 1 --2- 172.21.15.84:0/771687090 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f759006aea0 0x7f759006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:14.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.920+0000 7f75b4cc8640 1 --2- 172.21.15.84:0/771687090 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f759006aea0 0x7f759006d360 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f75a00094c0 tx=0x7f75a0009690 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:14.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:14.924+0000 7f75ad7fa640 1 -- 172.21.15.84:0/771687090 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7598076910 con 0x7f75b00b3a10 2023-10-16T09:49:15.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.099+0000 7f75b6f2c640 1 -- 172.21.15.84:0/771687090 --> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f75b015d5c0 con 0x7f759006aea0 2023-10-16T09:49:15.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.119+0000 7f75ad7fa640 1 -- 172.21.15.84:0/771687090 <== mgr.4107 v2:172.21.15.84:6832/3010246666 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264273 (secure 0 0 0) 0x7f75b015d5c0 con 0x7f759006aea0 2023-10-16T09:49:15.124 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:15.125 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T09:49:15.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.124+0000 7f75b6f2c640 1 -- 172.21.15.84:0/771687090 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f759006aea0 msgr2=0x7f759006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:15.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.124+0000 7f75b6f2c640 1 --2- 172.21.15.84:0/771687090 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f759006aea0 0x7f759006d360 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f75a00094c0 tx=0x7f75a0009690 comp rx=0 tx=0).stop 2023-10-16T09:49:15.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.124+0000 7f75b6f2c640 1 -- 172.21.15.84:0/771687090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b00b3a10 msgr2=0x7f75b0150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:15.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.124+0000 7f75b6f2c640 1 --2- 172.21.15.84:0/771687090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b00b3a10 0x7f75b0150ec0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7598037040 tx=0x7f7598004730 comp rx=0 tx=0).stop 2023-10-16T09:49:15.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.124+0000 7f75b6f2c640 1 -- 172.21.15.84:0/771687090 shutdown_connections 2023-10-16T09:49:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.124+0000 7f75b6f2c640 1 --2- 172.21.15.84:0/771687090 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f759006aea0 0x7f759006d360 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.124+0000 7f75b6f2c640 1 --2- 172.21.15.84:0/771687090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b00c24c0 0x7f75b00c2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.124+0000 7f75b6f2c640 1 --2- 172.21.15.84:0/771687090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b00b3a10 0x7f75b0150ec0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.124+0000 7f75b6f2c640 1 --2- 172.21.15.84:0/771687090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f759c0047c0 0x7f75b015cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.124+0000 7f75b6f2c640 1 -- 172.21.15.84:0/771687090 >> 172.21.15.84:0/771687090 conn(0x7f75b00cb410 msgr2=0x7f75b00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:15.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.124+0000 7f75b6f2c640 1 -- 172.21.15.84:0/771687090 shutdown_connections 2023-10-16T09:49:15.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.125+0000 7f75b6f2c640 1 -- 172.21.15.84:0/771687090 wait complete. 2023-10-16T09:49:15.143 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-16T09:49:13.596231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236100,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738620,"statfs":{"total":773094113280,"available":772852346880,"internally_reserved":0,"allocated":2990080,"data_stored":1790108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.060358"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942979+0000","last_change":"2023-10-16T09:49:02.569411+0000","last_active":"2023-10-16T09:49:03.942979+0000","last_peered":"2023-10-16T09:49:03.942979+0000","last_clean":"2023-10-16T09:49:03.942979+0000","last_became_active":"2023-10-16T09:49:02.569101+0000","last_became_peered":"2023-10-16T09:49:02.569101+0000","last_unstale":"2023-10-16T09:49:03.942979+0000","last_undegraded":"2023-10-16T09:49:03.942979+0000","last_fullsized":"2023-10-16T09:49:03.942979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:20:01.929361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937690+0000","last_change":"2023-10-16T09:49:02.570326+0000","last_active":"2023-10-16T09:49:03.937690+0000","last_peered":"2023-10-16T09:49:03.937690+0000","last_clean":"2023-10-16T09:49:03.937690+0000","last_became_active":"2023-10-16T09:49:02.569866+0000","last_became_peered":"2023-10-16T09:49:02.569866+0000","last_unstale":"2023-10-16T09:49:03.937690+0000","last_undegraded":"2023-10-16T09:49:03.937690+0000","last_fullsized":"2023-10-16T09:49:03.937690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:10.503481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940359+0000","last_change":"2023-10-16T09:49:02.566217+0000","last_active":"2023-10-16T09:49:03.940359+0000","last_peered":"2023-10-16T09:49:03.940359+0000","last_clean":"2023-10-16T09:49:03.940359+0000","last_became_active":"2023-10-16T09:49:02.565843+0000","last_became_peered":"2023-10-16T09:49:02.565843+0000","last_unstale":"2023-10-16T09:49:03.940359+0000","last_undegraded":"2023-10-16T09:49:03.940359+0000","last_fullsized":"2023-10-16T09:49:03.940359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540898+0000","last_change":"2023-10-16T09:49:02.580605+0000","last_active":"2023-10-16T09:49:03.540898+0000","last_peered":"2023-10-16T09:49:03.540898+0000","last_clean":"2023-10-16T09:49:03.540898+0000","last_became_active":"2023-10-16T09:49:02.573845+0000","last_became_peered":"2023-10-16T09:49:02.573845+0000","last_unstale":"2023-10-16T09:49:03.540898+0000","last_undegraded":"2023-10-16T09:49:03.540898+0000","last_fullsized":"2023-10-16T09:49:03.540898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:05.838911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941895+0000","last_change":"2023-10-16T09:49:02.563238+0000","last_active":"2023-10-16T09:49:03.941895+0000","last_peered":"2023-10-16T09:49:03.941895+0000","last_clean":"2023-10-16T09:49:03.941895+0000","last_became_active":"2023-10-16T09:49:02.562849+0000","last_became_peered":"2023-10-16T09:49:02.562849+0000","last_unstale":"2023-10-16T09:49:03.941895+0000","last_undegraded":"2023-10-16T09:49:03.941895+0000","last_fullsized":"2023-10-16T09:49:03.941895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:55.226843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540148+0000","last_change":"2023-10-16T09:49:02.572385+0000","last_active":"2023-10-16T09:49:03.540148+0000","last_peered":"2023-10-16T09:49:03.540148+0000","last_clean":"2023-10-16T09:49:03.540148+0000","last_became_active":"2023-10-16T09:49:02.571270+0000","last_became_peered":"2023-10-16T09:49:02.571270+0000","last_unstale":"2023-10-16T09:49:03.540148+0000","last_undegraded":"2023-10-16T09:49:03.540148+0000","last_fullsized":"2023-10-16T09:49:03.540148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941710+0000","last_change":"2023-10-16T09:49:02.570612+0000","last_active":"2023-10-16T09:49:03.941710+0000","last_peered":"2023-10-16T09:49:03.941710+0000","last_clean":"2023-10-16T09:49:03.941710+0000","last_became_active":"2023-10-16T09:49:02.570330+0000","last_became_peered":"2023-10-16T09:49:02.570330+0000","last_unstale":"2023-10-16T09:49:03.941710+0000","last_undegraded":"2023-10-16T09:49:03.941710+0000","last_fullsized":"2023-10-16T09:49:03.941710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:07.724831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540222+0000","last_change":"2023-10-16T09:49:02.572301+0000","last_active":"2023-10-16T09:49:03.540222+0000","last_peered":"2023-10-16T09:49:03.540222+0000","last_clean":"2023-10-16T09:49:03.540222+0000","last_became_active":"2023-10-16T09:49:02.571934+0000","last_became_peered":"2023-10-16T09:49:02.571934+0000","last_unstale":"2023-10-16T09:49:03.540222+0000","last_undegraded":"2023-10-16T09:49:03.540222+0000","last_fullsized":"2023-10-16T09:49:03.540222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:51.981410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.943019+0000","last_change":"2023-10-16T09:49:02.580905+0000","last_active":"2023-10-16T09:49:03.943019+0000","last_peered":"2023-10-16T09:49:03.943019+0000","last_clean":"2023-10-16T09:49:03.943019+0000","last_became_active":"2023-10-16T09:49:02.577694+0000","last_became_peered":"2023-10-16T09:49:02.577694+0000","last_unstale":"2023-10-16T09:49:03.943019+0000","last_undegraded":"2023-10-16T09:49:03.943019+0000","last_fullsized":"2023-10-16T09:49:03.943019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:02:14.799393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938295+0000","last_change":"2023-10-16T09:49:02.576191+0000","last_active":"2023-10-16T09:49:03.938295+0000","last_peered":"2023-10-16T09:49:03.938295+0000","last_clean":"2023-10-16T09:49:03.938295+0000","last_became_active":"2023-10-16T09:49:02.575810+0000","last_became_peered":"2023-10-16T09:49:02.575810+0000","last_unstale":"2023-10-16T09:49:03.938295+0000","last_undegraded":"2023-10-16T09:49:03.938295+0000","last_fullsized":"2023-10-16T09:49:03.938295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937913+0000","last_change":"2023-10-16T09:49:02.577871+0000","last_active":"2023-10-16T09:49:03.937913+0000","last_peered":"2023-10-16T09:49:03.937913+0000","last_clean":"2023-10-16T09:49:03.937913+0000","last_became_active":"2023-10-16T09:49:02.577580+0000","last_became_peered":"2023-10-16T09:49:02.577580+0000","last_unstale":"2023-10-16T09:49:03.937913+0000","last_undegraded":"2023-10-16T09:49:03.937913+0000","last_fullsized":"2023-10-16T09:49:03.937913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940615+0000","last_change":"2023-10-16T09:49:02.575626+0000","last_active":"2023-10-16T09:49:03.940615+0000","last_peered":"2023-10-16T09:49:03.940615+0000","last_clean":"2023-10-16T09:49:03.940615+0000","last_became_active":"2023-10-16T09:49:02.569588+0000","last_became_peered":"2023-10-16T09:49:02.569588+0000","last_unstale":"2023-10-16T09:49:03.940615+0000","last_undegraded":"2023-10-16T09:49:03.940615+0000","last_fullsized":"2023-10-16T09:49:03.940615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:28.559387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541264+0000","last_change":"2023-10-16T09:49:02.562597+0000","last_active":"2023-10-16T09:49:03.541264+0000","last_peered":"2023-10-16T09:49:03.541264+0000","last_clean":"2023-10-16T09:49:03.541264+0000","last_became_active":"2023-10-16T09:49:02.562322+0000","last_became_peered":"2023-10-16T09:49:02.562322+0000","last_unstale":"2023-10-16T09:49:03.541264+0000","last_undegraded":"2023-10-16T09:49:03.541264+0000","last_fullsized":"2023-10-16T09:49:03.541264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:51.170491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939007+0000","last_change":"2023-10-16T09:49:02.568210+0000","last_active":"2023-10-16T09:49:03.939007+0000","last_peered":"2023-10-16T09:49:03.939007+0000","last_clean":"2023-10-16T09:49:03.939007+0000","last_became_active":"2023-10-16T09:49:02.567330+0000","last_became_peered":"2023-10-16T09:49:02.567330+0000","last_unstale":"2023-10-16T09:49:03.939007+0000","last_undegraded":"2023-10-16T09:49:03.939007+0000","last_fullsized":"2023-10-16T09:49:03.939007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:17.275794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938071+0000","last_change":"2023-10-16T09:49:02.561360+0000","last_active":"2023-10-16T09:49:03.938071+0000","last_peered":"2023-10-16T09:49:03.938071+0000","last_clean":"2023-10-16T09:49:03.938071+0000","last_became_active":"2023-10-16T09:49:02.560831+0000","last_became_peered":"2023-10-16T09:49:02.560831+0000","last_unstale":"2023-10-16T09:49:03.938071+0000","last_undegraded":"2023-10-16T09:49:03.938071+0000","last_fullsized":"2023-10-16T09:49:03.938071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539917+0000","last_change":"2023-10-16T09:49:02.571717+0000","last_active":"2023-10-16T09:49:03.539917+0000","last_peered":"2023-10-16T09:49:03.539917+0000","last_clean":"2023-10-16T09:49:03.539917+0000","last_became_active":"2023-10-16T09:49:02.571311+0000","last_became_peered":"2023-10-16T09:49:02.571311+0000","last_unstale":"2023-10-16T09:49:03.539917+0000","last_undegraded":"2023-10-16T09:49:03.539917+0000","last_fullsized":"2023-10-16T09:49:03.539917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541271+0000","last_change":"2023-10-16T09:49:02.582038+0000","last_active":"2023-10-16T09:49:03.541271+0000","last_peered":"2023-10-16T09:49:03.541271+0000","last_clean":"2023-10-16T09:49:03.541271+0000","last_became_active":"2023-10-16T09:49:02.581780+0000","last_became_peered":"2023-10-16T09:49:02.581780+0000","last_unstale":"2023-10-16T09:49:03.541271+0000","last_undegraded":"2023-10-16T09:49:03.541271+0000","last_fullsized":"2023-10-16T09:49:03.541271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:45.572589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539221+0000","last_change":"2023-10-16T09:49:02.563686+0000","last_active":"2023-10-16T09:49:03.539221+0000","last_peered":"2023-10-16T09:49:03.539221+0000","last_clean":"2023-10-16T09:49:03.539221+0000","last_became_active":"2023-10-16T09:49:02.562728+0000","last_became_peered":"2023-10-16T09:49:02.562728+0000","last_unstale":"2023-10-16T09:49:03.539221+0000","last_undegraded":"2023-10-16T09:49:03.539221+0000","last_fullsized":"2023-10-16T09:49:03.539221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:58.154049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942765+0000","last_change":"2023-10-16T09:49:02.569238+0000","last_active":"2023-10-16T09:49:03.942765+0000","last_peered":"2023-10-16T09:49:03.942765+0000","last_clean":"2023-10-16T09:49:03.942765+0000","last_became_active":"2023-10-16T09:49:02.568292+0000","last_became_peered":"2023-10-16T09:49:02.568292+0000","last_unstale":"2023-10-16T09:49:03.942765+0000","last_undegraded":"2023-10-16T09:49:03.942765+0000","last_fullsized":"2023-10-16T09:49:03.942765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:46.937999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937917+0000","last_change":"2023-10-16T09:49:02.563047+0000","last_active":"2023-10-16T09:49:03.937917+0000","last_peered":"2023-10-16T09:49:03.937917+0000","last_clean":"2023-10-16T09:49:03.937917+0000","last_became_active":"2023-10-16T09:49:02.562532+0000","last_became_peered":"2023-10-16T09:49:02.562532+0000","last_unstale":"2023-10-16T09:49:03.937917+0000","last_undegraded":"2023-10-16T09:49:03.937917+0000","last_fullsized":"2023-10-16T09:49:03.937917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:33.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939622+0000","last_change":"2023-10-16T09:49:02.569454+0000","last_active":"2023-10-16T09:49:03.939622+0000","last_peered":"2023-10-16T09:49:03.939622+0000","last_clean":"2023-10-16T09:49:03.939622+0000","last_became_active":"2023-10-16T09:49:02.569216+0000","last_became_peered":"2023-10-16T09:49:02.569216+0000","last_unstale":"2023-10-16T09:49:03.939622+0000","last_undegraded":"2023-10-16T09:49:03.939622+0000","last_fullsized":"2023-10-16T09:49:03.939622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:47.112566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540091+0000","last_change":"2023-10-16T09:49:02.572101+0000","last_active":"2023-10-16T09:49:03.540091+0000","last_peered":"2023-10-16T09:49:03.540091+0000","last_clean":"2023-10-16T09:49:03.540091+0000","last_became_active":"2023-10-16T09:49:02.570939+0000","last_became_peered":"2023-10-16T09:49:02.570939+0000","last_unstale":"2023-10-16T09:49:03.540091+0000","last_undegraded":"2023-10-16T09:49:03.540091+0000","last_fullsized":"2023-10-16T09:49:03.540091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540377+0000","last_change":"2023-10-16T09:49:02.562443+0000","last_active":"2023-10-16T09:49:03.540377+0000","last_peered":"2023-10-16T09:49:03.540377+0000","last_clean":"2023-10-16T09:49:03.540377+0000","last_became_active":"2023-10-16T09:49:02.562184+0000","last_became_peered":"2023-10-16T09:49:02.562184+0000","last_unstale":"2023-10-16T09:49:03.540377+0000","last_undegraded":"2023-10-16T09:49:03.540377+0000","last_fullsized":"2023-10-16T09:49:03.540377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:47.112566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540214+0000","last_change":"2023-10-16T09:49:02.566157+0000","last_active":"2023-10-16T09:49:03.540214+0000","last_peered":"2023-10-16T09:49:03.540214+0000","last_clean":"2023-10-16T09:49:03.540214+0000","last_became_active":"2023-10-16T09:49:02.565808+0000","last_became_peered":"2023-10-16T09:49:02.565808+0000","last_unstale":"2023-10-16T09:49:03.540214+0000","last_undegraded":"2023-10-16T09:49:03.540214+0000","last_fullsized":"2023-10-16T09:49:03.540214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:51.170491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540739+0000","last_change":"2023-10-16T09:49:02.570442+0000","last_active":"2023-10-16T09:49:03.540739+0000","last_peered":"2023-10-16T09:49:03.540739+0000","last_clean":"2023-10-16T09:49:03.540739+0000","last_became_active":"2023-10-16T09:49:02.570173+0000","last_became_peered":"2023-10-16T09:49:02.570173+0000","last_unstale":"2023-10-16T09:49:03.540739+0000","last_undegraded":"2023-10-16T09:49:03.540739+0000","last_fullsized":"2023-10-16T09:49:03.540739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:05.333911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938295+0000","last_change":"2023-10-16T09:49:02.565763+0000","last_active":"2023-10-16T09:49:03.938295+0000","last_peered":"2023-10-16T09:49:03.938295+0000","last_clean":"2023-10-16T09:49:03.938295+0000","last_became_active":"2023-10-16T09:49:02.565248+0000","last_became_peered":"2023-10-16T09:49:02.565248+0000","last_unstale":"2023-10-16T09:49:03.938295+0000","last_undegraded":"2023-10-16T09:49:03.938295+0000","last_fullsized":"2023-10-16T09:49:03.938295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938715+0000","last_change":"2023-10-16T09:49:02.556364+0000","last_active":"2023-10-16T09:49:03.938715+0000","last_peered":"2023-10-16T09:49:03.938715+0000","last_clean":"2023-10-16T09:49:03.938715+0000","last_became_active":"2023-10-16T09:49:02.556055+0000","last_became_peered":"2023-10-16T09:49:02.556055+0000","last_unstale":"2023-10-16T09:49:03.938715+0000","last_undegraded":"2023-10-16T09:49:03.938715+0000","last_fullsized":"2023-10-16T09:49:03.938715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541726+0000","last_change":"2023-10-16T09:49:02.564976+0000","last_active":"2023-10-16T09:49:03.541726+0000","last_peered":"2023-10-16T09:49:03.541726+0000","last_clean":"2023-10-16T09:49:03.541726+0000","last_became_active":"2023-10-16T09:49:02.564798+0000","last_became_peered":"2023-10-16T09:49:02.564798+0000","last_unstale":"2023-10-16T09:49:03.541726+0000","last_undegraded":"2023-10-16T09:49:03.541726+0000","last_fullsized":"2023-10-16T09:49:03.541726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:39:22.668380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939780+0000","last_change":"2023-10-16T09:49:02.561691+0000","last_active":"2023-10-16T09:49:03.939780+0000","last_peered":"2023-10-16T09:49:03.939780+0000","last_clean":"2023-10-16T09:49:03.939780+0000","last_became_active":"2023-10-16T09:49:02.561408+0000","last_became_peered":"2023-10-16T09:49:02.561408+0000","last_unstale":"2023-10-16T09:49:03.939780+0000","last_undegraded":"2023-10-16T09:49:03.939780+0000","last_fullsized":"2023-10-16T09:49:03.939780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:10.503481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541186+0000","last_change":"2023-10-16T09:49:02.572702+0000","last_active":"2023-10-16T09:49:03.541186+0000","last_peered":"2023-10-16T09:49:03.541186+0000","last_clean":"2023-10-16T09:49:03.541186+0000","last_became_active":"2023-10-16T09:49:02.571811+0000","last_became_peered":"2023-10-16T09:49:02.571811+0000","last_unstale":"2023-10-16T09:49:03.541186+0000","last_undegraded":"2023-10-16T09:49:03.541186+0000","last_fullsized":"2023-10-16T09:49:03.541186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:08.814427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.538838+0000","last_change":"2023-10-16T09:49:02.576101+0000","last_active":"2023-10-16T09:49:03.538838+0000","last_peered":"2023-10-16T09:49:03.538838+0000","last_clean":"2023-10-16T09:49:03.538838+0000","last_became_active":"2023-10-16T09:49:02.575844+0000","last_became_peered":"2023-10-16T09:49:02.575844+0000","last_unstale":"2023-10-16T09:49:03.538838+0000","last_undegraded":"2023-10-16T09:49:03.538838+0000","last_fullsized":"2023-10-16T09:49:03.538838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:45.119851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937812+0000","last_change":"2023-10-16T09:49:02.568425+0000","last_active":"2023-10-16T09:49:03.937812+0000","last_peered":"2023-10-16T09:49:03.937812+0000","last_clean":"2023-10-16T09:49:03.937812+0000","last_became_active":"2023-10-16T09:49:02.568091+0000","last_became_peered":"2023-10-16T09:49:02.568091+0000","last_unstale":"2023-10-16T09:49:03.937812+0000","last_undegraded":"2023-10-16T09:49:03.937812+0000","last_fullsized":"2023-10-16T09:49:03.937812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:10.503481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940047+0000","last_change":"2023-10-16T09:49:02.559306+0000","last_active":"2023-10-16T09:49:03.940047+0000","last_peered":"2023-10-16T09:49:03.940047+0000","last_clean":"2023-10-16T09:49:03.940047+0000","last_became_active":"2023-10-16T09:49:02.558667+0000","last_became_peered":"2023-10-16T09:49:02.558667+0000","last_unstale":"2023-10-16T09:49:03.940047+0000","last_undegraded":"2023-10-16T09:49:03.940047+0000","last_fullsized":"2023-10-16T09:49:03.940047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:38:53.130875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541754+0000","last_change":"2023-10-16T09:49:01.918393+0000","last_active":"2023-10-16T09:49:03.541754+0000","last_peered":"2023-10-16T09:49:03.541754+0000","last_clean":"2023-10-16T09:49:03.541754+0000","last_became_active":"2023-10-16T09:49:01.918085+0000","last_became_peered":"2023-10-16T09:49:01.918085+0000","last_unstale":"2023-10-16T09:49:03.541754+0000","last_undegraded":"2023-10-16T09:49:03.541754+0000","last_fullsized":"2023-10-16T09:49:03.541754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:24.118554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939938+0000","last_change":"2023-10-16T09:49:02.562307+0000","last_active":"2023-10-16T09:49:03.939938+0000","last_peered":"2023-10-16T09:49:03.939938+0000","last_clean":"2023-10-16T09:49:03.939938+0000","last_became_active":"2023-10-16T09:49:02.562013+0000","last_became_peered":"2023-10-16T09:49:02.562013+0000","last_unstale":"2023-10-16T09:49:03.939938+0000","last_undegraded":"2023-10-16T09:49:03.939938+0000","last_fullsized":"2023-10-16T09:49:03.939938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937479+0000","last_change":"2023-10-16T09:49:01.931642+0000","last_active":"2023-10-16T09:49:03.937479+0000","last_peered":"2023-10-16T09:49:03.937479+0000","last_clean":"2023-10-16T09:49:03.937479+0000","last_became_active":"2023-10-16T09:49:01.929759+0000","last_became_peered":"2023-10-16T09:49:01.929759+0000","last_unstale":"2023-10-16T09:49:03.937479+0000","last_undegraded":"2023-10-16T09:49:03.937479+0000","last_fullsized":"2023-10-16T09:49:03.937479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:56.824003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540519+0000","last_change":"2023-10-16T09:49:01.925581+0000","last_active":"2023-10-16T09:49:03.540519+0000","last_peered":"2023-10-16T09:49:03.540519+0000","last_clean":"2023-10-16T09:49:03.540519+0000","last_became_active":"2023-10-16T09:49:01.925329+0000","last_became_peered":"2023-10-16T09:49:01.925329+0000","last_unstale":"2023-10-16T09:49:03.540519+0000","last_undegraded":"2023-10-16T09:49:03.540519+0000","last_fullsized":"2023-10-16T09:49:03.540519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:12:13.066628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540403+0000","last_change":"2023-10-16T09:49:02.566394+0000","last_active":"2023-10-16T09:49:03.540403+0000","last_peered":"2023-10-16T09:49:03.540403+0000","last_clean":"2023-10-16T09:49:03.540403+0000","last_became_active":"2023-10-16T09:49:02.566059+0000","last_became_peered":"2023-10-16T09:49:02.566059+0000","last_unstale":"2023-10-16T09:49:03.540403+0000","last_undegraded":"2023-10-16T09:49:03.540403+0000","last_fullsized":"2023-10-16T09:49:03.540403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:10.503481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540187+0000","last_change":"2023-10-16T09:49:01.933426+0000","last_active":"2023-10-16T09:49:03.540187+0000","last_peered":"2023-10-16T09:49:03.540187+0000","last_clean":"2023-10-16T09:49:03.540187+0000","last_became_active":"2023-10-16T09:49:01.930300+0000","last_became_peered":"2023-10-16T09:49:01.930300+0000","last_unstale":"2023-10-16T09:49:03.540187+0000","last_undegraded":"2023-10-16T09:49:03.540187+0000","last_fullsized":"2023-10-16T09:49:03.540187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:27.229341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937721+0000","last_change":"2023-10-16T09:49:02.555653+0000","last_active":"2023-10-16T09:49:03.937721+0000","last_peered":"2023-10-16T09:49:03.937721+0000","last_clean":"2023-10-16T09:49:03.937721+0000","last_became_active":"2023-10-16T09:49:02.555099+0000","last_became_peered":"2023-10-16T09:49:02.555099+0000","last_unstale":"2023-10-16T09:49:03.937721+0000","last_undegraded":"2023-10-16T09:49:03.937721+0000","last_fullsized":"2023-10-16T09:49:03.937721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:33.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938399+0000","last_change":"2023-10-16T09:49:01.937059+0000","last_active":"2023-10-16T09:49:03.938399+0000","last_peered":"2023-10-16T09:49:03.938399+0000","last_clean":"2023-10-16T09:49:03.938399+0000","last_became_active":"2023-10-16T09:49:01.936784+0000","last_became_peered":"2023-10-16T09:49:01.936784+0000","last_unstale":"2023-10-16T09:49:03.938399+0000","last_undegraded":"2023-10-16T09:49:03.938399+0000","last_fullsized":"2023-10-16T09:49:03.938399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:24.118554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540473+0000","last_change":"2023-10-16T09:49:02.575679+0000","last_active":"2023-10-16T09:49:03.540473+0000","last_peered":"2023-10-16T09:49:03.540473+0000","last_clean":"2023-10-16T09:49:03.540473+0000","last_became_active":"2023-10-16T09:49:02.575367+0000","last_became_peered":"2023-10-16T09:49:02.575367+0000","last_unstale":"2023-10-16T09:49:03.540473+0000","last_undegraded":"2023-10-16T09:49:03.540473+0000","last_fullsized":"2023-10-16T09:49:03.540473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:54:44.787181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540422+0000","last_change":"2023-10-16T09:49:01.934686+0000","last_active":"2023-10-16T09:49:03.540422+0000","last_peered":"2023-10-16T09:49:03.540422+0000","last_clean":"2023-10-16T09:49:03.540422+0000","last_became_active":"2023-10-16T09:49:01.934414+0000","last_became_peered":"2023-10-16T09:49:01.934414+0000","last_unstale":"2023-10-16T09:49:03.540422+0000","last_undegraded":"2023-10-16T09:49:03.540422+0000","last_fullsized":"2023-10-16T09:49:03.540422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:09.173435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940582+0000","last_change":"2023-10-16T09:49:02.573683+0000","last_active":"2023-10-16T09:49:03.940582+0000","last_peered":"2023-10-16T09:49:03.940582+0000","last_clean":"2023-10-16T09:49:03.940582+0000","last_became_active":"2023-10-16T09:49:02.573331+0000","last_became_peered":"2023-10-16T09:49:02.573331+0000","last_unstale":"2023-10-16T09:49:03.940582+0000","last_undegraded":"2023-10-16T09:49:03.940582+0000","last_fullsized":"2023-10-16T09:49:03.940582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938642+0000","last_change":"2023-10-16T09:49:01.922705+0000","last_active":"2023-10-16T09:49:03.938642+0000","last_peered":"2023-10-16T09:49:03.938642+0000","last_clean":"2023-10-16T09:49:03.938642+0000","last_became_active":"2023-10-16T09:49:01.922292+0000","last_became_peered":"2023-10-16T09:49:01.922292+0000","last_unstale":"2023-10-16T09:49:03.938642+0000","last_undegraded":"2023-10-16T09:49:03.938642+0000","last_fullsized":"2023-10-16T09:49:03.938642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:50.651364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940794+0000","last_change":"2023-10-16T09:49:02.567153+0000","last_active":"2023-10-16T09:49:03.940794+0000","last_peered":"2023-10-16T09:49:03.940794+0000","last_clean":"2023-10-16T09:49:03.940794+0000","last_became_active":"2023-10-16T09:49:02.566840+0000","last_became_peered":"2023-10-16T09:49:02.566840+0000","last_unstale":"2023-10-16T09:49:03.940794+0000","last_undegraded":"2023-10-16T09:49:03.940794+0000","last_fullsized":"2023-10-16T09:49:03.940794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:58.154049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937813+0000","last_change":"2023-10-16T09:49:01.930609+0000","last_active":"2023-10-16T09:49:03.937813+0000","last_peered":"2023-10-16T09:49:03.937813+0000","last_clean":"2023-10-16T09:49:03.937813+0000","last_became_active":"2023-10-16T09:49:01.930235+0000","last_became_peered":"2023-10-16T09:49:01.930235+0000","last_unstale":"2023-10-16T09:49:03.937813+0000","last_undegraded":"2023-10-16T09:49:03.937813+0000","last_fullsized":"2023-10-16T09:49:03.937813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:29:30.922262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942436+0000","last_change":"2023-10-16T09:49:02.577180+0000","last_active":"2023-10-16T09:49:03.942436+0000","last_peered":"2023-10-16T09:49:03.942436+0000","last_clean":"2023-10-16T09:49:03.942436+0000","last_became_active":"2023-10-16T09:49:02.576534+0000","last_became_peered":"2023-10-16T09:49:02.576534+0000","last_unstale":"2023-10-16T09:49:03.942436+0000","last_undegraded":"2023-10-16T09:49:03.942436+0000","last_fullsized":"2023-10-16T09:49:03.942436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:45.572589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940062+0000","last_change":"2023-10-16T09:49:01.918540+0000","last_active":"2023-10-16T09:49:03.940062+0000","last_peered":"2023-10-16T09:49:03.940062+0000","last_clean":"2023-10-16T09:49:03.940062+0000","last_became_active":"2023-10-16T09:49:01.917673+0000","last_became_peered":"2023-10-16T09:49:01.917673+0000","last_unstale":"2023-10-16T09:49:03.940062+0000","last_undegraded":"2023-10-16T09:49:03.940062+0000","last_fullsized":"2023-10-16T09:49:03.940062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:41.350109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939226+0000","last_change":"2023-10-16T09:49:02.570004+0000","last_active":"2023-10-16T09:49:03.939226+0000","last_peered":"2023-10-16T09:49:03.939226+0000","last_clean":"2023-10-16T09:49:03.939226+0000","last_became_active":"2023-10-16T09:49:02.569583+0000","last_became_peered":"2023-10-16T09:49:02.569583+0000","last_unstale":"2023-10-16T09:49:03.939226+0000","last_undegraded":"2023-10-16T09:49:03.939226+0000","last_fullsized":"2023-10-16T09:49:03.939226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:51.981410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540696+0000","last_change":"2023-10-16T09:49:01.932631+0000","last_active":"2023-10-16T09:49:03.540696+0000","last_peered":"2023-10-16T09:49:03.540696+0000","last_clean":"2023-10-16T09:49:03.540696+0000","last_became_active":"2023-10-16T09:49:01.928924+0000","last_became_peered":"2023-10-16T09:49:01.928924+0000","last_unstale":"2023-10-16T09:49:03.540696+0000","last_undegraded":"2023-10-16T09:49:03.540696+0000","last_fullsized":"2023-10-16T09:49:03.540696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:15.945748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942055+0000","last_change":"2023-10-16T09:49:02.569260+0000","last_active":"2023-10-16T09:49:03.942055+0000","last_peered":"2023-10-16T09:49:03.942055+0000","last_clean":"2023-10-16T09:49:03.942055+0000","last_became_active":"2023-10-16T09:49:02.568903+0000","last_became_peered":"2023-10-16T09:49:02.568903+0000","last_unstale":"2023-10-16T09:49:03.942055+0000","last_undegraded":"2023-10-16T09:49:03.942055+0000","last_fullsized":"2023-10-16T09:49:03.942055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:08.814427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937182+0000","last_change":"2023-10-16T09:49:01.930319+0000","last_active":"2023-10-16T09:49:03.937182+0000","last_peered":"2023-10-16T09:49:03.937182+0000","last_clean":"2023-10-16T09:49:03.937182+0000","last_became_active":"2023-10-16T09:49:01.928990+0000","last_became_peered":"2023-10-16T09:49:01.928990+0000","last_unstale":"2023-10-16T09:49:03.937182+0000","last_undegraded":"2023-10-16T09:49:03.937182+0000","last_fullsized":"2023-10-16T09:49:03.937182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:43.789805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937440+0000","last_change":"2023-10-16T09:49:02.568938+0000","last_active":"2023-10-16T09:49:03.937440+0000","last_peered":"2023-10-16T09:49:03.937440+0000","last_clean":"2023-10-16T09:49:03.937440+0000","last_became_active":"2023-10-16T09:49:02.568222+0000","last_became_peered":"2023-10-16T09:49:02.568222+0000","last_unstale":"2023-10-16T09:49:03.937440+0000","last_undegraded":"2023-10-16T09:49:03.937440+0000","last_fullsized":"2023-10-16T09:49:03.937440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:47.112566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.538731+0000","last_change":"2023-10-16T09:49:01.913071+0000","last_active":"2023-10-16T09:49:03.538731+0000","last_peered":"2023-10-16T09:49:03.538731+0000","last_clean":"2023-10-16T09:49:03.538731+0000","last_became_active":"2023-10-16T09:49:01.912124+0000","last_became_peered":"2023-10-16T09:49:01.912124+0000","last_unstale":"2023-10-16T09:49:03.538731+0000","last_undegraded":"2023-10-16T09:49:03.538731+0000","last_fullsized":"2023-10-16T09:49:03.538731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:16.897247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941822+0000","last_change":"2023-10-16T09:49:02.577394+0000","last_active":"2023-10-16T09:49:03.941822+0000","last_peered":"2023-10-16T09:49:03.941822+0000","last_clean":"2023-10-16T09:49:03.941822+0000","last_became_active":"2023-10-16T09:49:02.577042+0000","last_became_peered":"2023-10-16T09:49:02.577042+0000","last_unstale":"2023-10-16T09:49:03.941822+0000","last_undegraded":"2023-10-16T09:49:03.941822+0000","last_fullsized":"2023-10-16T09:49:03.941822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:05.838911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941102+0000","last_change":"2023-10-16T09:49:01.918367+0000","last_active":"2023-10-16T09:49:03.941102+0000","last_peered":"2023-10-16T09:49:03.941102+0000","last_clean":"2023-10-16T09:49:03.941102+0000","last_became_active":"2023-10-16T09:49:01.918021+0000","last_became_peered":"2023-10-16T09:49:01.918021+0000","last_unstale":"2023-10-16T09:49:03.941102+0000","last_undegraded":"2023-10-16T09:49:03.941102+0000","last_fullsized":"2023-10-16T09:49:03.941102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:25.972592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942743+0000","last_change":"2023-10-16T09:49:02.567390+0000","last_active":"2023-10-16T09:49:03.942743+0000","last_peered":"2023-10-16T09:49:03.942743+0000","last_clean":"2023-10-16T09:49:03.942743+0000","last_became_active":"2023-10-16T09:49:02.566827+0000","last_became_peered":"2023-10-16T09:49:02.566827+0000","last_unstale":"2023-10-16T09:49:03.942743+0000","last_undegraded":"2023-10-16T09:49:03.942743+0000","last_fullsized":"2023-10-16T09:49:03.942743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:40:56.166477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539098+0000","last_change":"2023-10-16T09:49:01.904674+0000","last_active":"2023-10-16T09:49:03.539098+0000","last_peered":"2023-10-16T09:49:03.539098+0000","last_clean":"2023-10-16T09:49:03.539098+0000","last_became_active":"2023-10-16T09:49:01.904380+0000","last_became_peered":"2023-10-16T09:49:01.904380+0000","last_unstale":"2023-10-16T09:49:03.539098+0000","last_undegraded":"2023-10-16T09:49:03.539098+0000","last_fullsized":"2023-10-16T09:49:03.539098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:25.972592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.943170+0000","last_change":"2023-10-16T09:49:02.575633+0000","last_active":"2023-10-16T09:49:03.943170+0000","last_peered":"2023-10-16T09:49:03.943170+0000","last_clean":"2023-10-16T09:49:03.943170+0000","last_became_active":"2023-10-16T09:49:02.573187+0000","last_became_peered":"2023-10-16T09:49:02.573187+0000","last_unstale":"2023-10-16T09:49:03.943170+0000","last_undegraded":"2023-10-16T09:49:03.943170+0000","last_fullsized":"2023-10-16T09:49:03.943170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:12.370135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938903+0000","last_change":"2023-10-16T09:49:01.911986+0000","last_active":"2023-10-16T09:49:03.938903+0000","last_peered":"2023-10-16T09:49:03.938903+0000","last_clean":"2023-10-16T09:49:03.938903+0000","last_became_active":"2023-10-16T09:49:01.911700+0000","last_became_peered":"2023-10-16T09:49:01.911700+0000","last_unstale":"2023-10-16T09:49:03.938903+0000","last_undegraded":"2023-10-16T09:49:03.938903+0000","last_fullsized":"2023-10-16T09:49:03.938903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:45.782520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540965+0000","last_change":"2023-10-16T09:49:02.574594+0000","last_active":"2023-10-16T09:49:03.540965+0000","last_peered":"2023-10-16T09:49:03.540965+0000","last_clean":"2023-10-16T09:49:03.540965+0000","last_became_active":"2023-10-16T09:49:02.573419+0000","last_became_peered":"2023-10-16T09:49:02.573419+0000","last_unstale":"2023-10-16T09:49:03.540965+0000","last_undegraded":"2023-10-16T09:49:03.540965+0000","last_fullsized":"2023-10-16T09:49:03.540965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:55.226843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539291+0000","last_change":"2023-10-16T09:49:01.934196+0000","last_active":"2023-10-16T09:49:03.539291+0000","last_peered":"2023-10-16T09:49:03.539291+0000","last_clean":"2023-10-16T09:49:03.539291+0000","last_became_active":"2023-10-16T09:49:01.933922+0000","last_became_peered":"2023-10-16T09:49:01.933922+0000","last_unstale":"2023-10-16T09:49:03.539291+0000","last_undegraded":"2023-10-16T09:49:03.539291+0000","last_fullsized":"2023-10-16T09:49:03.539291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:10:19.173392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942391+0000","last_change":"2023-10-16T09:49:02.577848+0000","last_active":"2023-10-16T09:49:03.942391+0000","last_peered":"2023-10-16T09:49:03.942391+0000","last_clean":"2023-10-16T09:49:03.942391+0000","last_became_active":"2023-10-16T09:49:02.577571+0000","last_became_peered":"2023-10-16T09:49:02.577571+0000","last_unstale":"2023-10-16T09:49:03.942391+0000","last_undegraded":"2023-10-16T09:49:03.942391+0000","last_fullsized":"2023-10-16T09:49:03.942391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:12:32.747232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T09:49:02.548285+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T09:49:02.548285+0000","last_peered":"2023-10-16T09:49:02.548285+0000","last_clean":"2023-10-16T09:49:02.548285+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T09:49:02.548285+0000","last_undegraded":"2023-10-16T09:49:02.548285+0000","last_fullsized":"2023-10-16T09:49:02.548285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.539903+0000","last_change":"2023-10-16T09:49:01.922671+0000","last_active":"2023-10-16T09:49:03.539903+0000","last_peered":"2023-10-16T09:49:03.539903+0000","last_clean":"2023-10-16T09:49:03.539903+0000","last_became_active":"2023-10-16T09:49:01.922356+0000","last_became_peered":"2023-10-16T09:49:01.922356+0000","last_unstale":"2023-10-16T09:49:03.539903+0000","last_undegraded":"2023-10-16T09:49:03.539903+0000","last_fullsized":"2023-10-16T09:49:03.539903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:10:19.173392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.542081+0000","last_change":"2023-10-16T09:49:02.572185+0000","last_active":"2023-10-16T09:49:03.542081+0000","last_peered":"2023-10-16T09:49:03.542081+0000","last_clean":"2023-10-16T09:49:03.542081+0000","last_became_active":"2023-10-16T09:49:02.571953+0000","last_became_peered":"2023-10-16T09:49:02.571953+0000","last_unstale":"2023-10-16T09:49:03.542081+0000","last_undegraded":"2023-10-16T09:49:03.542081+0000","last_fullsized":"2023-10-16T09:49:03.542081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:07.724831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941421+0000","last_change":"2023-10-16T09:49:01.925742+0000","last_active":"2023-10-16T09:49:03.941421+0000","last_peered":"2023-10-16T09:49:03.941421+0000","last_clean":"2023-10-16T09:49:03.941421+0000","last_became_active":"2023-10-16T09:49:01.925438+0000","last_became_peered":"2023-10-16T09:49:01.925438+0000","last_unstale":"2023-10-16T09:49:03.941421+0000","last_undegraded":"2023-10-16T09:49:03.941421+0000","last_fullsized":"2023-10-16T09:49:03.941421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:56.824003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938454+0000","last_change":"2023-10-16T09:49:02.562256+0000","last_active":"2023-10-16T09:49:03.938454+0000","last_peered":"2023-10-16T09:49:03.938454+0000","last_clean":"2023-10-16T09:49:03.938454+0000","last_became_active":"2023-10-16T09:49:02.561942+0000","last_became_peered":"2023-10-16T09:49:02.561942+0000","last_unstale":"2023-10-16T09:49:03.938454+0000","last_undegraded":"2023-10-16T09:49:03.938454+0000","last_fullsized":"2023-10-16T09:49:03.938454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:12:14.396674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.538971+0000","last_change":"2023-10-16T09:49:01.905306+0000","last_active":"2023-10-16T09:49:03.538971+0000","last_peered":"2023-10-16T09:49:03.538971+0000","last_clean":"2023-10-16T09:49:03.538971+0000","last_became_active":"2023-10-16T09:49:01.904664+0000","last_became_peered":"2023-10-16T09:49:01.904664+0000","last_unstale":"2023-10-16T09:49:03.538971+0000","last_undegraded":"2023-10-16T09:49:03.538971+0000","last_fullsized":"2023-10-16T09:49:03.538971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:38:51.800830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939222+0000","last_change":"2023-10-16T09:49:02.551068+0000","last_active":"2023-10-16T09:49:03.939222+0000","last_peered":"2023-10-16T09:49:03.939222+0000","last_clean":"2023-10-16T09:49:03.939222+0000","last_became_active":"2023-10-16T09:49:02.550772+0000","last_became_peered":"2023-10-16T09:49:02.550772+0000","last_unstale":"2023-10-16T09:49:03.939222+0000","last_undegraded":"2023-10-16T09:49:03.939222+0000","last_fullsized":"2023-10-16T09:49:03.939222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:51.170491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540961+0000","last_change":"2023-10-16T09:49:01.932431+0000","last_active":"2023-10-16T09:49:03.540961+0000","last_peered":"2023-10-16T09:49:03.540961+0000","last_clean":"2023-10-16T09:49:03.540961+0000","last_became_active":"2023-10-16T09:49:01.927598+0000","last_became_peered":"2023-10-16T09:49:01.927598+0000","last_unstale":"2023-10-16T09:49:03.540961+0000","last_undegraded":"2023-10-16T09:49:03.540961+0000","last_fullsized":"2023-10-16T09:49:03.540961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:50.651364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940795+0000","last_change":"2023-10-16T09:49:02.571518+0000","last_active":"2023-10-16T09:49:03.940795+0000","last_peered":"2023-10-16T09:49:03.940795+0000","last_clean":"2023-10-16T09:49:03.940795+0000","last_became_active":"2023-10-16T09:49:02.571243+0000","last_became_peered":"2023-10-16T09:49:02.571243+0000","last_unstale":"2023-10-16T09:49:03.940795+0000","last_undegraded":"2023-10-16T09:49:03.940795+0000","last_fullsized":"2023-10-16T09:49:03.940795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:45.119851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937987+0000","last_change":"2023-10-16T09:49:01.928253+0000","last_active":"2023-10-16T09:49:03.937987+0000","last_peered":"2023-10-16T09:49:03.937987+0000","last_clean":"2023-10-16T09:49:03.937987+0000","last_became_active":"2023-10-16T09:49:01.927808+0000","last_became_peered":"2023-10-16T09:49:01.927808+0000","last_unstale":"2023-10-16T09:49:03.937987+0000","last_undegraded":"2023-10-16T09:49:03.937987+0000","last_fullsized":"2023-10-16T09:49:03.937987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:41.350109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541226+0000","last_change":"2023-10-16T09:49:02.575272+0000","last_active":"2023-10-16T09:49:03.541226+0000","last_peered":"2023-10-16T09:49:03.541226+0000","last_clean":"2023-10-16T09:49:03.541226+0000","last_became_active":"2023-10-16T09:49:02.574888+0000","last_became_peered":"2023-10-16T09:49:02.574888+0000","last_unstale":"2023-10-16T09:49:03.541226+0000","last_undegraded":"2023-10-16T09:49:03.541226+0000","last_fullsized":"2023-10-16T09:49:03.541226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:12:32.747232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942771+0000","last_change":"2023-10-16T09:49:01.931740+0000","last_active":"2023-10-16T09:49:03.942771+0000","last_peered":"2023-10-16T09:49:03.942771+0000","last_clean":"2023-10-16T09:49:03.942771+0000","last_became_active":"2023-10-16T09:49:01.931370+0000","last_became_peered":"2023-10-16T09:49:01.931370+0000","last_unstale":"2023-10-16T09:49:03.942771+0000","last_undegraded":"2023-10-16T09:49:03.942771+0000","last_fullsized":"2023-10-16T09:49:03.942771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:49.840445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937766+0000","last_change":"2023-10-16T09:49:02.553220+0000","last_active":"2023-10-16T09:49:03.937766+0000","last_peered":"2023-10-16T09:49:03.937766+0000","last_clean":"2023-10-16T09:49:03.937766+0000","last_became_active":"2023-10-16T09:49:02.552703+0000","last_became_peered":"2023-10-16T09:49:02.552703+0000","last_unstale":"2023-10-16T09:49:03.937766+0000","last_undegraded":"2023-10-16T09:49:03.937766+0000","last_fullsized":"2023-10-16T09:49:03.937766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:04.627058+0000","last_change":"2023-10-16T09:49:01.928063+0000","last_active":"2023-10-16T09:49:04.627058+0000","last_peered":"2023-10-16T09:49:04.627058+0000","last_clean":"2023-10-16T09:49:04.627058+0000","last_became_active":"2023-10-16T09:49:01.927770+0000","last_became_peered":"2023-10-16T09:49:01.927770+0000","last_unstale":"2023-10-16T09:49:04.627058+0000","last_undegraded":"2023-10-16T09:49:04.627058+0000","last_fullsized":"2023-10-16T09:49:04.627058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:12:13.066628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540860+0000","last_change":"2023-10-16T09:49:02.562667+0000","last_active":"2023-10-16T09:49:03.540860+0000","last_peered":"2023-10-16T09:49:03.540860+0000","last_clean":"2023-10-16T09:49:03.540860+0000","last_became_active":"2023-10-16T09:49:02.562361+0000","last_became_peered":"2023-10-16T09:49:02.562361+0000","last_unstale":"2023-10-16T09:49:03.540860+0000","last_undegraded":"2023-10-16T09:49:03.540860+0000","last_fullsized":"2023-10-16T09:49:03.540860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:17.917386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938522+0000","last_change":"2023-10-16T09:49:01.930319+0000","last_active":"2023-10-16T09:49:03.938522+0000","last_peered":"2023-10-16T09:49:03.938522+0000","last_clean":"2023-10-16T09:49:03.938522+0000","last_became_active":"2023-10-16T09:49:01.929983+0000","last_became_peered":"2023-10-16T09:49:01.929983+0000","last_unstale":"2023-10-16T09:49:03.938522+0000","last_undegraded":"2023-10-16T09:49:03.938522+0000","last_fullsized":"2023-10-16T09:49:03.938522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:12:13.066628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.939914+0000","last_change":"2023-10-16T09:49:02.567693+0000","last_active":"2023-10-16T09:49:03.939914+0000","last_peered":"2023-10-16T09:49:03.939914+0000","last_clean":"2023-10-16T09:49:03.939914+0000","last_became_active":"2023-10-16T09:49:02.566400+0000","last_became_peered":"2023-10-16T09:49:02.566400+0000","last_unstale":"2023-10-16T09:49:03.939914+0000","last_undegraded":"2023-10-16T09:49:03.939914+0000","last_fullsized":"2023-10-16T09:49:03.939914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:33.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.941454+0000","last_change":"2023-10-16T09:49:01.929977+0000","last_active":"2023-10-16T09:49:03.941454+0000","last_peered":"2023-10-16T09:49:03.941454+0000","last_clean":"2023-10-16T09:49:03.941454+0000","last_became_active":"2023-10-16T09:49:01.928703+0000","last_became_peered":"2023-10-16T09:49:01.928703+0000","last_unstale":"2023-10-16T09:49:03.941454+0000","last_undegraded":"2023-10-16T09:49:03.941454+0000","last_fullsized":"2023-10-16T09:49:03.941454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:43.789805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940325+0000","last_change":"2023-10-16T09:49:02.558651+0000","last_active":"2023-10-16T09:49:03.940325+0000","last_peered":"2023-10-16T09:49:03.940325+0000","last_clean":"2023-10-16T09:49:03.940325+0000","last_became_active":"2023-10-16T09:49:02.554370+0000","last_became_peered":"2023-10-16T09:49:02.554370+0000","last_unstale":"2023-10-16T09:49:03.940325+0000","last_undegraded":"2023-10-16T09:49:03.940325+0000","last_fullsized":"2023-10-16T09:49:03.940325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:51:17.179081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938321+0000","last_change":"2023-10-16T09:49:01.918105+0000","last_active":"2023-10-16T09:49:03.938321+0000","last_peered":"2023-10-16T09:49:03.938321+0000","last_clean":"2023-10-16T09:49:03.938321+0000","last_became_active":"2023-10-16T09:49:01.917625+0000","last_became_peered":"2023-10-16T09:49:01.917625+0000","last_unstale":"2023-10-16T09:49:03.938321+0000","last_undegraded":"2023-10-16T09:49:03.938321+0000","last_fullsized":"2023-10-16T09:49:03.938321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:24.118554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540666+0000","last_change":"2023-10-16T09:49:02.556320+0000","last_active":"2023-10-16T09:49:03.540666+0000","last_peered":"2023-10-16T09:49:03.540666+0000","last_clean":"2023-10-16T09:49:03.540666+0000","last_became_active":"2023-10-16T09:49:02.555897+0000","last_became_peered":"2023-10-16T09:49:02.555897+0000","last_unstale":"2023-10-16T09:49:03.540666+0000","last_undegraded":"2023-10-16T09:49:03.540666+0000","last_fullsized":"2023-10-16T09:49:03.540666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:33.133616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.938761+0000","last_change":"2023-10-16T09:49:01.915222+0000","last_active":"2023-10-16T09:49:03.938761+0000","last_peered":"2023-10-16T09:49:03.938761+0000","last_clean":"2023-10-16T09:49:03.938761+0000","last_became_active":"2023-10-16T09:49:01.914843+0000","last_became_peered":"2023-10-16T09:49:01.914843+0000","last_unstale":"2023-10-16T09:49:03.938761+0000","last_undegraded":"2023-10-16T09:49:03.938761+0000","last_fullsized":"2023-10-16T09:49:03.938761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:51:15.849036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.541605+0000","last_change":"2023-10-16T09:49:02.567819+0000","last_active":"2023-10-16T09:49:03.541605+0000","last_peered":"2023-10-16T09:49:03.541605+0000","last_clean":"2023-10-16T09:49:03.541605+0000","last_became_active":"2023-10-16T09:49:02.567548+0000","last_became_peered":"2023-10-16T09:49:02.567548+0000","last_unstale":"2023-10-16T09:49:03.541605+0000","last_undegraded":"2023-10-16T09:49:03.541605+0000","last_fullsized":"2023-10-16T09:49:03.541605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:54:44.787181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942035+0000","last_change":"2023-10-16T09:49:01.924408+0000","last_active":"2023-10-16T09:49:03.942035+0000","last_peered":"2023-10-16T09:49:03.942035+0000","last_clean":"2023-10-16T09:49:03.942035+0000","last_became_active":"2023-10-16T09:49:01.922411+0000","last_became_peered":"2023-10-16T09:49:01.922411+0000","last_unstale":"2023-10-16T09:49:03.942035+0000","last_undegraded":"2023-10-16T09:49:03.942035+0000","last_fullsized":"2023-10-16T09:49:03.942035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:24.118554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.937385+0000","last_change":"2023-10-16T09:49:02.560941+0000","last_active":"2023-10-16T09:49:03.937385+0000","last_peered":"2023-10-16T09:49:03.937385+0000","last_clean":"2023-10-16T09:49:03.937385+0000","last_became_active":"2023-10-16T09:49:02.560673+0000","last_became_peered":"2023-10-16T09:49:02.560673+0000","last_unstale":"2023-10-16T09:49:03.937385+0000","last_undegraded":"2023-10-16T09:49:03.937385+0000","last_fullsized":"2023-10-16T09:49:03.937385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:51.981410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540856+0000","last_change":"2023-10-16T09:49:02.575954+0000","last_active":"2023-10-16T09:49:03.540856+0000","last_peered":"2023-10-16T09:49:03.540856+0000","last_clean":"2023-10-16T09:49:03.540856+0000","last_became_active":"2023-10-16T09:49:02.575517+0000","last_became_peered":"2023-10-16T09:49:02.575517+0000","last_unstale":"2023-10-16T09:49:03.540856+0000","last_undegraded":"2023-10-16T09:49:03.540856+0000","last_fullsized":"2023-10-16T09:49:03.540856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:07.724831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942265+0000","last_change":"2023-10-16T09:49:01.933911+0000","last_active":"2023-10-16T09:49:03.942265+0000","last_peered":"2023-10-16T09:49:03.942265+0000","last_clean":"2023-10-16T09:49:03.942265+0000","last_became_active":"2023-10-16T09:49:01.933640+0000","last_became_peered":"2023-10-16T09:49:01.933640+0000","last_unstale":"2023-10-16T09:49:03.942265+0000","last_undegraded":"2023-10-16T09:49:03.942265+0000","last_fullsized":"2023-10-16T09:49:03.942265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:27.229341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540639+0000","last_change":"2023-10-16T09:49:02.580360+0000","last_active":"2023-10-16T09:49:03.540639+0000","last_peered":"2023-10-16T09:49:03.540639+0000","last_clean":"2023-10-16T09:49:03.540639+0000","last_became_active":"2023-10-16T09:49:02.574225+0000","last_became_peered":"2023-10-16T09:49:02.574225+0000","last_unstale":"2023-10-16T09:49:03.540639+0000","last_undegraded":"2023-10-16T09:49:03.540639+0000","last_fullsized":"2023-10-16T09:49:03.540639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:39:22.668380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.942632+0000","last_change":"2023-10-16T09:49:01.931685+0000","last_active":"2023-10-16T09:49:03.942632+0000","last_peered":"2023-10-16T09:49:03.942632+0000","last_clean":"2023-10-16T09:49:03.942632+0000","last_became_active":"2023-10-16T09:49:01.930733+0000","last_became_peered":"2023-10-16T09:49:01.930733+0000","last_unstale":"2023-10-16T09:49:03.942632+0000","last_undegraded":"2023-10-16T09:49:03.942632+0000","last_fullsized":"2023-10-16T09:49:03.942632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:09.173435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940714+0000","last_change":"2023-10-16T09:49:02.573451+0000","last_active":"2023-10-16T09:49:03.940714+0000","last_peered":"2023-10-16T09:49:03.940714+0000","last_clean":"2023-10-16T09:49:03.940714+0000","last_became_active":"2023-10-16T09:49:02.572970+0000","last_became_peered":"2023-10-16T09:49:02.572970+0000","last_unstale":"2023-10-16T09:49:03.940714+0000","last_undegraded":"2023-10-16T09:49:03.940714+0000","last_fullsized":"2023-10-16T09:49:03.940714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:51.170491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540855+0000","last_change":"2023-10-16T09:49:01.920578+0000","last_active":"2023-10-16T09:49:03.540855+0000","last_peered":"2023-10-16T09:49:03.540855+0000","last_clean":"2023-10-16T09:49:03.540855+0000","last_became_active":"2023-10-16T09:49:01.919963+0000","last_became_peered":"2023-10-16T09:49:01.919963+0000","last_unstale":"2023-10-16T09:49:03.540855+0000","last_undegraded":"2023-10-16T09:49:03.540855+0000","last_fullsized":"2023-10-16T09:49:03.540855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:15.945748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.540674+0000","last_change":"2023-10-16T09:49:02.563034+0000","last_active":"2023-10-16T09:49:03.540674+0000","last_peered":"2023-10-16T09:49:03.540674+0000","last_clean":"2023-10-16T09:49:03.540674+0000","last_became_active":"2023-10-16T09:49:02.562680+0000","last_became_peered":"2023-10-16T09:49:02.562680+0000","last_unstale":"2023-10-16T09:49:03.540674+0000","last_undegraded":"2023-10-16T09:49:03.540674+0000","last_fullsized":"2023-10-16T09:49:03.540674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:01.870501+0000","last_clean_scrub_stamp":"2023-10-16T09:49:01.870501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:11.944499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:49:03.940780+0000","last_change":"2023-10-16T09:49:01.919451+0000","last_active":"2023-10-16T09:49:03.940780+0000","last_peered":"2023-10-16T09:49:03.940780+0000","last_clean":"2023-10-16T09:49:03.940780+0000","last_became_active":"2023-10-16T09:49:01.919106+0000","last_became_peered":"2023-10-16T09:49:01.919106+0000","last_unstale":"2023-10-16T09:49:03.940780+0000","last_undegraded":"2023-10-16T09:49:03.940780+0000","last_fullsized":"2023-10-16T09:49:03.940780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:49:00.540455+0000","last_clean_scrub_stamp":"2023-10-16T09:49:00.540455+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:29:30.922262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":815104,"data_stored":666519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4540000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.399}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.355}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.224}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.722}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.621}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7949999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7290000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8129999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.028}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9850000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":77181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8640000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.74}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6379999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8280000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0819999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9119999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6059999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6319999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7770000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.377}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.667}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7410000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8169999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.782}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":823296,"data_stored":666575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4830000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.643}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.605}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.548}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5229999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:49:15.148 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-16T09:49:15.149 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-16T09:49:15.149 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-16T09:49:15.149 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-16T09:49:15.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.387+0000 7f5c66c75640 1 Processor -- start 2023-10-16T09:49:15.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.387+0000 7f5c66c75640 1 -- start start 2023-10-16T09:49:15.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.388+0000 7f5c66c75640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5c6015ad70 0x7f5c60069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:15.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.388+0000 7f5c66c75640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c600c9ca0 0x7f5c6006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:15.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.389+0000 7f5c66c75640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c600ca3b0 0x7f5c600ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:15.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.389+0000 7f5c66c75640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c600cad80 con 0x7f5c6015ad70 2023-10-16T09:49:15.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.389+0000 7f5c66c75640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c600cb030 con 0x7f5c600ca3b0 2023-10-16T09:49:15.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.389+0000 7f5c66c75640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c600c91c0 con 0x7f5c600c9ca0 2023-10-16T09:49:15.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.389+0000 7f5c64a11640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5c6015ad70 0x7f5c60069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:15.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.389+0000 7f5c64a11640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5c6015ad70 0x7f5c60069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59968/0 (socket says 172.21.15.84:59968) 2023-10-16T09:49:15.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.389+0000 7f5c64a11640 1 -- 172.21.15.84:0/3421749328 learned_addr learned my addr 172.21.15.84:0/3421749328 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:15.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.389+0000 7f5c65212640 1 --2- 172.21.15.84:0/3421749328 >> v2:172.21.15.157:3300/0 conn(0x7f5c600ca3b0 0x7f5c600ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:15.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.389+0000 7f5c5ffff640 1 --2- 172.21.15.84:0/3421749328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c600c9ca0 0x7f5c6006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:15.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.389+0000 7f5c64a11640 1 -- 172.21.15.84:0/3421749328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c600c9ca0 msgr2=0x7f5c6006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:15.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.389+0000 7f5c64a11640 1 --2- 172.21.15.84:0/3421749328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c600c9ca0 0x7f5c6006a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.389+0000 7f5c64a11640 1 -- 172.21.15.84:0/3421749328 >> v2:172.21.15.157:3300/0 conn(0x7f5c600ca3b0 msgr2=0x7f5c600ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:15.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.389+0000 7f5c64a11640 1 --2- 172.21.15.84:0/3421749328 >> v2:172.21.15.157:3300/0 conn(0x7f5c600ca3b0 0x7f5c600ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.389+0000 7f5c64a11640 1 -- 172.21.15.84:0/3421749328 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c600c94a0 con 0x7f5c6015ad70 2023-10-16T09:49:15.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.389+0000 7f5c5ffff640 1 --2- 172.21.15.84:0/3421749328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c600c9ca0 0x7f5c6006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:15.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.391+0000 7f5c64a11640 1 --2- 172.21.15.84:0/3421749328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c6015ad70 0x7f5c60069da0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5c600cf820 tx=0x7f5c4802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83951e2ad78cca4c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:15.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.391+0000 7f5c5f7fe640 1 -- 172.21.15.84:0/3421749328 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48031870 con 0x7f5c6015ad70 2023-10-16T09:49:15.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.391+0000 7f5c5f7fe640 1 -- 172.21.15.84:0/3421749328 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c480319d0 con 0x7f5c6015ad70 2023-10-16T09:49:15.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.391+0000 7f5c5f7fe640 1 -- 172.21.15.84:0/3421749328 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48031d60 con 0x7f5c6015ad70 2023-10-16T09:49:15.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.392+0000 7f5c66c75640 1 -- 172.21.15.84:0/3421749328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c6015ad70 msgr2=0x7f5c60069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:15.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.392+0000 7f5c66c75640 1 --2- 172.21.15.84:0/3421749328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c6015ad70 0x7f5c60069da0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5c600cf820 tx=0x7f5c4802b910 comp rx=0 tx=0).stop 2023-10-16T09:49:15.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.392+0000 7f5c66c75640 1 -- 172.21.15.84:0/3421749328 shutdown_connections 2023-10-16T09:49:15.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.392+0000 7f5c66c75640 1 --2- 172.21.15.84:0/3421749328 >> v2:172.21.15.157:3300/0 conn(0x7f5c600ca3b0 0x7f5c600ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.392+0000 7f5c66c75640 1 --2- 172.21.15.84:0/3421749328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c600c9ca0 0x7f5c6006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.392+0000 7f5c66c75640 1 --2- 172.21.15.84:0/3421749328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c6015ad70 0x7f5c60069da0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.392+0000 7f5c66c75640 1 -- 172.21.15.84:0/3421749328 >> 172.21.15.84:0/3421749328 conn(0x7f5c600ce9b0 msgr2=0x7f5c600cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:15.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.392+0000 7f5c66c75640 1 -- 172.21.15.84:0/3421749328 shutdown_connections 2023-10-16T09:49:15.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.392+0000 7f5c66c75640 1 -- 172.21.15.84:0/3421749328 wait complete. 2023-10-16T09:49:15.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.393+0000 7f5c66c75640 1 Processor -- start 2023-10-16T09:49:15.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.394+0000 7f5c66c75640 1 -- start start 2023-10-16T09:49:15.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.394+0000 7f5c66c75640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c600c9ca0 0x7f5c600c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:15.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.394+0000 7f5c66c75640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c600ca3b0 0x7f5c6014be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:15.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.394+0000 7f5c66c75640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c6015ad70 0x7f5c600b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:15.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.394+0000 7f5c66c75640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6014c360 con 0x7f5c600c9ca0 2023-10-16T09:49:15.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.394+0000 7f5c66c75640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c600b09c0 con 0x7f5c600ca3b0 2023-10-16T09:49:15.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.394+0000 7f5c66c75640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c600b0c90 con 0x7f5c6015ad70 2023-10-16T09:49:15.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.395+0000 7f5c64a11640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c600c9ca0 0x7f5c600c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:15.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.395+0000 7f5c64a11640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c600c9ca0 0x7f5c600c85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59970/0 (socket says 172.21.15.84:59970) 2023-10-16T09:49:15.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.395+0000 7f5c64a11640 1 -- 172.21.15.84:0/2405178679 learned_addr learned my addr 172.21.15.84:0/2405178679 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:15.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.395+0000 7f5c5ffff640 1 --2- 172.21.15.84:0/2405178679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c600ca3b0 0x7f5c6014be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:15.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.395+0000 7f5c65212640 1 --2- 172.21.15.84:0/2405178679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c6015ad70 0x7f5c600b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:15.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.395+0000 7f5c64a11640 1 -- 172.21.15.84:0/2405178679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c6015ad70 msgr2=0x7f5c600b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:15.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.395+0000 7f5c64a11640 1 --2- 172.21.15.84:0/2405178679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c6015ad70 0x7f5c600b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.395+0000 7f5c64a11640 1 -- 172.21.15.84:0/2405178679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c600ca3b0 msgr2=0x7f5c6014be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:15.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.395+0000 7f5c64a11640 1 --2- 172.21.15.84:0/2405178679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c600ca3b0 0x7f5c6014be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.395+0000 7f5c64a11640 1 -- 172.21.15.84:0/2405178679 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c480094d0 con 0x7f5c600c9ca0 2023-10-16T09:49:15.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.395+0000 7f5c65212640 1 --2- 172.21.15.84:0/2405178679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c6015ad70 0x7f5c600b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:15.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.395+0000 7f5c64a11640 1 --2- 172.21.15.84:0/2405178679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c600c9ca0 0x7f5c600c85c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5c48000c00 tx=0x7f5c480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:15.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.396+0000 7f5c5d7fa640 1 -- 172.21.15.84:0/2405178679 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4803abe0 con 0x7f5c600c9ca0 2023-10-16T09:49:15.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.396+0000 7f5c5d7fa640 1 -- 172.21.15.84:0/2405178679 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4803ad40 con 0x7f5c600c9ca0 2023-10-16T09:49:15.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.396+0000 7f5c5d7fa640 1 -- 172.21.15.84:0/2405178679 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48044610 con 0x7f5c600c9ca0 2023-10-16T09:49:15.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.396+0000 7f5c66c75640 1 -- 172.21.15.84:0/2405178679 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c600b0f70 con 0x7f5c600c9ca0 2023-10-16T09:49:15.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.396+0000 7f5c66c75640 1 -- 172.21.15.84:0/2405178679 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c600b1530 con 0x7f5c600c9ca0 2023-10-16T09:49:15.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.397+0000 7f5c66c75640 1 -- 172.21.15.84:0/2405178679 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6004e160 con 0x7f5c600c9ca0 2023-10-16T09:49:15.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.398+0000 7f5c5d7fa640 1 -- 172.21.15.84:0/2405178679 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5c4804d450 con 0x7f5c600c9ca0 2023-10-16T09:49:15.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.398+0000 7f5c5d7fa640 1 --2- 172.21.15.84:0/2405178679 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f5c4006aea0 0x7f5c4006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:15.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.399+0000 7f5c5d7fa640 1 -- 172.21.15.84:0/2405178679 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c4802d040 con 0x7f5c600c9ca0 2023-10-16T09:49:15.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.399+0000 7f5c5ffff640 1 --2- 172.21.15.84:0/2405178679 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f5c4006aea0 0x7f5c4006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:15.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.399+0000 7f5c5ffff640 1 --2- 172.21.15.84:0/2405178679 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f5c4006aea0 0x7f5c4006d360 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5c60159790 tx=0x7f5c500079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:15.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.403+0000 7f5c5d7fa640 1 -- 172.21.15.84:0/2405178679 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5c48076c50 con 0x7f5c600c9ca0 2023-10-16T09:49:15.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.617+0000 7f5c66c75640 1 -- 172.21.15.84:0/2405178679 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5c6015d090 con 0x7f5c600c9ca0 2023-10-16T09:49:15.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.617+0000 7f5c5d7fa640 1 -- 172.21.15.84:0/2405178679 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5c4807a270 con 0x7f5c600c9ca0 2023-10-16T09:49:15.620 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:15.620 INFO:teuthology.orchestra.run.smithi084.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-16T09:49:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.620+0000 7f5c66c75640 1 -- 172.21.15.84:0/2405178679 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f5c4006aea0 msgr2=0x7f5c4006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.620+0000 7f5c66c75640 1 --2- 172.21.15.84:0/2405178679 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f5c4006aea0 0x7f5c4006d360 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5c60159790 tx=0x7f5c500079c0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.620+0000 7f5c66c75640 1 -- 172.21.15.84:0/2405178679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c600c9ca0 msgr2=0x7f5c600c85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.620+0000 7f5c66c75640 1 --2- 172.21.15.84:0/2405178679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c600c9ca0 0x7f5c600c85c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5c48000c00 tx=0x7f5c480046c0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.620+0000 7f5c66c75640 1 -- 172.21.15.84:0/2405178679 shutdown_connections 2023-10-16T09:49:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.620+0000 7f5c66c75640 1 --2- 172.21.15.84:0/2405178679 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f5c4006aea0 0x7f5c4006d360 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.620+0000 7f5c66c75640 1 --2- 172.21.15.84:0/2405178679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c6015ad70 0x7f5c600b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.620+0000 7f5c66c75640 1 --2- 172.21.15.84:0/2405178679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c600ca3b0 0x7f5c6014be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.620+0000 7f5c66c75640 1 --2- 172.21.15.84:0/2405178679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c600c9ca0 0x7f5c600c85c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.620+0000 7f5c66c75640 1 -- 172.21.15.84:0/2405178679 >> 172.21.15.84:0/2405178679 conn(0x7f5c600ce9b0 msgr2=0x7f5c60070d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:15.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.620+0000 7f5c66c75640 1 -- 172.21.15.84:0/2405178679 shutdown_connections 2023-10-16T09:49:15.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.620+0000 7f5c66c75640 1 -- 172.21.15.84:0/2405178679 wait complete. 2023-10-16T09:49:15.639 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-16T09:49:15.639 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:15.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.842+0000 7f3c48e56640 1 Processor -- start 2023-10-16T09:49:15.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.843+0000 7f3c48e56640 1 -- start start 2023-10-16T09:49:15.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.844+0000 7f3c48e56640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3c44159b20 0x7f3c4406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:15.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.844+0000 7f3c48e56640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c4406f7f0 con 0x7f3c440b0b20 2023-10-16T09:49:15.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.844+0000 7f3c48e56640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c4406c5e0 con 0x7f3c44159f00 2023-10-16T09:49:15.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.844+0000 7f3c48e56640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c4406c8c0 con 0x7f3c44159b20 2023-10-16T09:49:15.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.844+0000 7f3c4259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3c44159b20 0x7f3c4406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:15.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.844+0000 7f3c4259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3c44159b20 0x7f3c4406bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33842/0 (socket says 172.21.15.84:33842) 2023-10-16T09:49:15.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.844+0000 7f3c42d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3c440b0b20 0x7f3c4406fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36784/0 (socket says 172.21.15.84:36784) 2023-10-16T09:49:15.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.844+0000 7f3c4259c640 1 -- 172.21.15.84:0/568142892 learned_addr learned my addr 172.21.15.84:0/568142892 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:15.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.845+0000 7f3c4259c640 1 -- 172.21.15.84:0/568142892 >> v1:172.21.15.157:6789/0 conn(0x7f3c44159f00 legacy=0x7f3c440b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:15.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.845+0000 7f3c4259c640 1 -- 172.21.15.84:0/568142892 >> v1:172.21.15.84:6789/0 conn(0x7f3c440b0b20 legacy=0x7f3c4406fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:15.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.845+0000 7f3c4259c640 1 -- 172.21.15.84:0/568142892 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4406cba0 con 0x7f3c44159b20 2023-10-16T09:49:15.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.845+0000 7f3c4159a640 1 -- 172.21.15.84:0/568142892 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3374620563 0 0) 0x7f3c4406f7f0 con 0x7f3c440b0b20 2023-10-16T09:49:15.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.846+0000 7f3c4259c640 1 --2- 172.21.15.84:0/568142892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c44159b20 0x7f3c4406bbe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009e50 tx=0x7f3c3402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b947c70549be4bd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:15.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.846+0000 7f3c4159a640 1 -- 172.21.15.84:0/568142892 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3402cb10 con 0x7f3c44159b20 2023-10-16T09:49:15.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.846+0000 7f3c4159a640 1 -- 172.21.15.84:0/568142892 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3402cc70 con 0x7f3c44159b20 2023-10-16T09:49:15.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.846+0000 7f3c4159a640 1 -- 172.21.15.84:0/568142892 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c340343f0 con 0x7f3c44159b20 2023-10-16T09:49:15.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.847+0000 7f3c48e56640 1 -- 172.21.15.84:0/568142892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c44159b20 msgr2=0x7f3c4406bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:15.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.847+0000 7f3c48e56640 1 --2- 172.21.15.84:0/568142892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c44159b20 0x7f3c4406bbe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009e50 tx=0x7f3c3402bf70 comp rx=0 tx=0).stop 2023-10-16T09:49:15.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.847+0000 7f3c48e56640 1 -- 172.21.15.84:0/568142892 shutdown_connections 2023-10-16T09:49:15.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.847+0000 7f3c48e56640 1 --2- 172.21.15.84:0/568142892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c44159b20 0x7f3c4406bbe0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.847+0000 7f3c48e56640 1 -- 172.21.15.84:0/568142892 >> 172.21.15.84:0/568142892 conn(0x7f3c440cb410 msgr2=0x7f3c440cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:15.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.847+0000 7f3c48e56640 1 -- 172.21.15.84:0/568142892 shutdown_connections 2023-10-16T09:49:15.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.847+0000 7f3c48e56640 1 -- 172.21.15.84:0/568142892 wait complete. 2023-10-16T09:49:15.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.849+0000 7f3c48e56640 1 Processor -- start 2023-10-16T09:49:15.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.849+0000 7f3c48e56640 1 -- start start 2023-10-16T09:49:15.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.850+0000 7f3c48e56640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c440b0b20 0x7f3c4415bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:15.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.850+0000 7f3c48e56640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c44159b20 0x7f3c4415a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:15.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.850+0000 7f3c48e56640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c44159f00 0x7f3c440af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:15.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.850+0000 7f3c48e56640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4415c090 con 0x7f3c44159b20 2023-10-16T09:49:15.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.850+0000 7f3c48e56640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4415a850 con 0x7f3c44159f00 2023-10-16T09:49:15.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.850+0000 7f3c48e56640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4415aad0 con 0x7f3c440b0b20 2023-10-16T09:49:15.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.850+0000 7f3c41d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c44159b20 0x7f3c4415a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:15.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.850+0000 7f3c41d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c44159b20 0x7f3c4415a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33848/0 (socket says 172.21.15.84:33848) 2023-10-16T09:49:15.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.850+0000 7f3c41d9b640 1 -- 172.21.15.84:0/299807571 learned_addr learned my addr 172.21.15.84:0/299807571 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:15.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.851+0000 7f3c4259c640 1 --2- 172.21.15.84:0/299807571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c440b0b20 0x7f3c4415bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:15.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.851+0000 7f3c41d9b640 1 -- 172.21.15.84:0/299807571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c440b0b20 msgr2=0x7f3c4415bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:15.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.851+0000 7f3c41d9b640 1 --2- 172.21.15.84:0/299807571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c440b0b20 0x7f3c4415bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.851+0000 7f3c41d9b640 1 -- 172.21.15.84:0/299807571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c44159f00 msgr2=0x7f3c440af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:15.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.851+0000 7f3c41d9b640 1 --2- 172.21.15.84:0/299807571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c44159f00 0x7f3c440af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:15.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.851+0000 7f3c41d9b640 1 -- 172.21.15.84:0/299807571 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c34009b30 con 0x7f3c44159b20 2023-10-16T09:49:15.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.851+0000 7f3c41d9b640 1 --2- 172.21.15.84:0/299807571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c44159b20 0x7f3c4415a310 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00c400 tx=0x7f3c2c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:15.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.851+0000 7f3c3affd640 1 -- 172.21.15.84:0/299807571 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c011790 con 0x7f3c44159b20 2023-10-16T09:49:15.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.851+0000 7f3c3affd640 1 -- 172.21.15.84:0/299807571 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c011930 con 0x7f3c44159b20 2023-10-16T09:49:15.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.851+0000 7f3c3affd640 1 -- 172.21.15.84:0/299807571 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c011ce0 con 0x7f3c44159b20 2023-10-16T09:49:15.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.851+0000 7f3c48e56640 1 -- 172.21.15.84:0/299807571 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4415adb0 con 0x7f3c44159b20 2023-10-16T09:49:15.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.851+0000 7f3c48e56640 1 -- 172.21.15.84:0/299807571 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4415b370 con 0x7f3c44159b20 2023-10-16T09:49:15.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.852+0000 7f3c48e56640 1 -- 172.21.15.84:0/299807571 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c4404e160 con 0x7f3c44159b20 2023-10-16T09:49:15.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.853+0000 7f3c3affd640 1 -- 172.21.15.84:0/299807571 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3c2c01c460 con 0x7f3c44159b20 2023-10-16T09:49:15.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.854+0000 7f3c3affd640 1 --2- 172.21.15.84:0/299807571 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3c2406ae70 0x7f3c2406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:15.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.854+0000 7f3c3affd640 1 -- 172.21.15.84:0/299807571 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c2c0896e0 con 0x7f3c44159b20 2023-10-16T09:49:15.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.854+0000 7f3c4259c640 1 --2- 172.21.15.84:0/299807571 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3c2406ae70 0x7f3c2406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:15.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.855+0000 7f3c4259c640 1 --2- 172.21.15.84:0/299807571 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3c2406ae70 0x7f3c2406d330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3c34035000 tx=0x7f3c34030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:15.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:15.858+0000 7f3c3affd640 1 -- 172.21.15.84:0/299807571 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3c2c054660 con 0x7f3c44159b20 2023-10-16T09:49:16.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.035+0000 7f3c48e56640 1 -- 172.21.15.84:0/299807571 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c440b10b0 con 0x7f3c44159b20 2023-10-16T09:49:16.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.037+0000 7f3c3affd640 1 -- 172.21.15.84:0/299807571 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f3c2c057c80 con 0x7f3c44159b20 2023-10-16T09:49:16.040 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:16.040 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":21,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:03.529261+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:16.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.040+0000 7f3c48e56640 1 -- 172.21.15.84:0/299807571 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3c2406ae70 msgr2=0x7f3c2406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:16.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.040+0000 7f3c48e56640 1 --2- 172.21.15.84:0/299807571 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3c2406ae70 0x7f3c2406d330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3c34035000 tx=0x7f3c34030000 comp rx=0 tx=0).stop 2023-10-16T09:49:16.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.040+0000 7f3c48e56640 1 -- 172.21.15.84:0/299807571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c44159b20 msgr2=0x7f3c4415a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:16.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.040+0000 7f3c48e56640 1 --2- 172.21.15.84:0/299807571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c44159b20 0x7f3c4415a310 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00c400 tx=0x7f3c2c00c8c0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.040+0000 7f3c48e56640 1 -- 172.21.15.84:0/299807571 shutdown_connections 2023-10-16T09:49:16.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.040+0000 7f3c48e56640 1 --2- 172.21.15.84:0/299807571 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3c2406ae70 0x7f3c2406d330 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.040+0000 7f3c48e56640 1 --2- 172.21.15.84:0/299807571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c44159f00 0x7f3c440af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.040+0000 7f3c48e56640 1 --2- 172.21.15.84:0/299807571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c44159b20 0x7f3c4415a310 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.040+0000 7f3c48e56640 1 --2- 172.21.15.84:0/299807571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c440b0b20 0x7f3c4415bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.040+0000 7f3c48e56640 1 -- 172.21.15.84:0/299807571 >> 172.21.15.84:0/299807571 conn(0x7f3c440cb410 msgr2=0x7f3c440ce8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:16.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.040+0000 7f3c48e56640 1 -- 172.21.15.84:0/299807571 shutdown_connections 2023-10-16T09:49:16.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.040+0000 7f3c48e56640 1 -- 172.21.15.84:0/299807571 wait complete. 2023-10-16T09:49:16.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:49:00.048649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:49:01.361365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:49:16.061 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:49:16.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.270+0000 7fbecbfff640 1 Processor -- start 2023-10-16T09:49:16.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.270+0000 7fbecbfff640 1 -- start start 2023-10-16T09:49:16.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.272+0000 7fbecbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbecc0b2080 0x7fbecc0d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:16.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.272+0000 7fbecbfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbecc0cfc40 0x7fbecc0b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:16.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.272+0000 7fbecbfff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbecc0d0970 con 0x7fbecc0b1ca0 2023-10-16T09:49:16.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.272+0000 7fbecbfff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbecc0d0ea0 con 0x7fbecc0b2080 2023-10-16T09:49:16.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.272+0000 7fbecbfff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbecc0d0bc0 con 0x7fbecc0cfc40 2023-10-16T09:49:16.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.272+0000 7fbec27fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbecc0b2080 0x7fbecc0d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:16.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.273+0000 7fbec27fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbecc0b2080 0x7fbecc0d36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33854/0 (socket says 172.21.15.84:33854) 2023-10-16T09:49:16.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.273+0000 7fbec27fc640 1 -- 172.21.15.84:0/1834129906 learned_addr learned my addr 172.21.15.84:0/1834129906 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:16.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.273+0000 7fbecb7fe640 1 --2- 172.21.15.84:0/1834129906 >> v2:172.21.15.157:3300/0 conn(0x7fbecc0cfc40 0x7fbecc0b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:16.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.273+0000 7fbec27fc640 1 -- 172.21.15.84:0/1834129906 >> v2:172.21.15.157:3300/0 conn(0x7fbecc0cfc40 msgr2=0x7fbecc0b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:16.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.273+0000 7fbec27fc640 1 --2- 172.21.15.84:0/1834129906 >> v2:172.21.15.157:3300/0 conn(0x7fbecc0cfc40 0x7fbecc0b24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.273+0000 7fbeca7fc640 1 -- 172.21.15.84:0/1834129906 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227911295 0 0) 0x7fbecc0d0970 con 0x7fbecc0b1ca0 2023-10-16T09:49:16.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.273+0000 7fbec27fc640 1 -- 172.21.15.84:0/1834129906 >> v1:172.21.15.84:6789/0 conn(0x7fbecc0b1ca0 legacy=0x7fbecc0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:16.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.273+0000 7fbec27fc640 1 -- 172.21.15.84:0/1834129906 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc0cd940 con 0x7fbecc0b2080 2023-10-16T09:49:16.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.274+0000 7fbec27fc640 1 --2- 172.21.15.84:0/1834129906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbecc0b2080 0x7fbecc0d36b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0d1d30 tx=0x7fbeb402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbabc6df466efadd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:16.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.275+0000 7fbeca7fc640 1 -- 172.21.15.84:0/1834129906 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb402f850 con 0x7fbecc0b2080 2023-10-16T09:49:16.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.275+0000 7fbeca7fc640 1 -- 172.21.15.84:0/1834129906 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb402f9b0 con 0x7fbecc0b2080 2023-10-16T09:49:16.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.275+0000 7fbeca7fc640 1 -- 172.21.15.84:0/1834129906 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb402fd40 con 0x7fbecc0b2080 2023-10-16T09:49:16.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.275+0000 7fbecbfff640 1 -- 172.21.15.84:0/1834129906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbecc0b2080 msgr2=0x7fbecc0d36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:16.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.275+0000 7fbecbfff640 1 --2- 172.21.15.84:0/1834129906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbecc0b2080 0x7fbecc0d36b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0d1d30 tx=0x7fbeb402ba30 comp rx=0 tx=0).stop 2023-10-16T09:49:16.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.275+0000 7fbecbfff640 1 -- 172.21.15.84:0/1834129906 shutdown_connections 2023-10-16T09:49:16.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.275+0000 7fbecbfff640 1 --2- 172.21.15.84:0/1834129906 >> v2:172.21.15.157:3300/0 conn(0x7fbecc0cfc40 0x7fbecc0b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.275+0000 7fbecbfff640 1 --2- 172.21.15.84:0/1834129906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbecc0b2080 0x7fbecc0d36b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.275+0000 7fbecbfff640 1 -- 172.21.15.84:0/1834129906 >> 172.21.15.84:0/1834129906 conn(0x7fbecc0d2b90 msgr2=0x7fbecc0d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:16.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.275+0000 7fbecbfff640 1 -- 172.21.15.84:0/1834129906 shutdown_connections 2023-10-16T09:49:16.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.275+0000 7fbecbfff640 1 -- 172.21.15.84:0/1834129906 wait complete. 2023-10-16T09:49:16.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.277+0000 7fbecbfff640 1 Processor -- start 2023-10-16T09:49:16.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.277+0000 7fbecbfff640 1 -- start start 2023-10-16T09:49:16.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.278+0000 7fbecbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbecc0b1ca0 0x7fbecc0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:16.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.278+0000 7fbecbfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbecc0b2080 0x7fbecc15d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:16.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.278+0000 7fbecbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbecc0cfc40 0x7fbecc155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:16.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.278+0000 7fbecbfff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc15e4c0 con 0x7fbecc0b1ca0 2023-10-16T09:49:16.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.278+0000 7fbecbfff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0c7920 con 0x7fbecc0cfc40 2023-10-16T09:49:16.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.278+0000 7fbecbfff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc15dc20 con 0x7fbecc0b2080 2023-10-16T09:49:16.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.278+0000 7fbecaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbecc0b1ca0 0x7fbecc0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:16.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.278+0000 7fbecaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbecc0b1ca0 0x7fbecc0cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33868/0 (socket says 172.21.15.84:33868) 2023-10-16T09:49:16.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.278+0000 7fbecaffd640 1 -- 172.21.15.84:0/508086420 learned_addr learned my addr 172.21.15.84:0/508086420 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:16.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.279+0000 7fbecaffd640 1 -- 172.21.15.84:0/508086420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbecc0b2080 msgr2=0x7fbecc15d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:16.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.279+0000 7fbecaffd640 1 --2- 172.21.15.84:0/508086420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbecc0b2080 0x7fbecc15d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.279+0000 7fbecaffd640 1 -- 172.21.15.84:0/508086420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbecc0cfc40 msgr2=0x7fbecc155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:16.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.279+0000 7fbecaffd640 1 --2- 172.21.15.84:0/508086420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbecc0cfc40 0x7fbecc155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.279+0000 7fbecaffd640 1 -- 172.21.15.84:0/508086420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb40094d0 con 0x7fbecc0b1ca0 2023-10-16T09:49:16.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.279+0000 7fbecaffd640 1 --2- 172.21.15.84:0/508086420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbecc0b1ca0 0x7fbecc0cc4c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00c820 tx=0x7fbebc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:16.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.279+0000 7fbec3fff640 1 -- 172.21.15.84:0/508086420 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc003da0 con 0x7fbecc0b1ca0 2023-10-16T09:49:16.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.279+0000 7fbec3fff640 1 -- 172.21.15.84:0/508086420 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbebc005070 con 0x7fbecc0b1ca0 2023-10-16T09:49:16.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.279+0000 7fbecbfff640 1 -- 172.21.15.84:0/508086420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbecc15df00 con 0x7fbecc0b1ca0 2023-10-16T09:49:16.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.279+0000 7fbec3fff640 1 -- 172.21.15.84:0/508086420 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc005400 con 0x7fbecc0b1ca0 2023-10-16T09:49:16.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.280+0000 7fbecbfff640 1 -- 172.21.15.84:0/508086420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbecc15d8d0 con 0x7fbecc0b1ca0 2023-10-16T09:49:16.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.281+0000 7fbecbfff640 1 -- 172.21.15.84:0/508086420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbecc0ceb10 con 0x7fbecc0b1ca0 2023-10-16T09:49:16.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.282+0000 7fbec3fff640 1 -- 172.21.15.84:0/508086420 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbebc005560 con 0x7fbecc0b1ca0 2023-10-16T09:49:16.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.283+0000 7fbec3fff640 1 --2- 172.21.15.84:0/508086420 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbeac06ae70 0x7fbeac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:16.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.283+0000 7fbec3fff640 1 -- 172.21.15.84:0/508086420 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbebc088e00 con 0x7fbecc0b1ca0 2023-10-16T09:49:16.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.283+0000 7fbec27fc640 1 --2- 172.21.15.84:0/508086420 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbeac06ae70 0x7fbeac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:16.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.284+0000 7fbec27fc640 1 --2- 172.21.15.84:0/508086420 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbeac06ae70 0x7fbeac06d330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4036000 tx=0x7fbeb4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:16.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.287+0000 7fbec3fff640 1 -- 172.21.15.84:0/508086420 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbecc0ceb10 con 0x7fbecc0b1ca0 2023-10-16T09:49:16.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.477+0000 7fbecbfff640 1 -- 172.21.15.84:0/508086420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbecc0ceb10 con 0x7fbecc0b1ca0 2023-10-16T09:49:16.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.478+0000 7fbec3fff640 1 -- 172.21.15.84:0/508086420 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbecc0ceb10 con 0x7fbecc0b1ca0 2023-10-16T09:49:16.480 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T09:49:16.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.481+0000 7fbecbfff640 1 -- 172.21.15.84:0/508086420 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbeac06ae70 msgr2=0x7fbeac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:16.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.481+0000 7fbecbfff640 1 --2- 172.21.15.84:0/508086420 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbeac06ae70 0x7fbeac06d330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4036000 tx=0x7fbeb4030000 comp rx=0 tx=0).stop 2023-10-16T09:49:16.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.481+0000 7fbecbfff640 1 -- 172.21.15.84:0/508086420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbecc0b1ca0 msgr2=0x7fbecc0cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:16.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.481+0000 7fbecbfff640 1 --2- 172.21.15.84:0/508086420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbecc0b1ca0 0x7fbecc0cc4c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00c820 tx=0x7fbebc00cce0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.481+0000 7fbecbfff640 1 -- 172.21.15.84:0/508086420 shutdown_connections 2023-10-16T09:49:16.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.481+0000 7fbecbfff640 1 --2- 172.21.15.84:0/508086420 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbeac06ae70 0x7fbeac06d330 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.481+0000 7fbecbfff640 1 --2- 172.21.15.84:0/508086420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbecc0cfc40 0x7fbecc155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.481+0000 7fbecbfff640 1 --2- 172.21.15.84:0/508086420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbecc0b2080 0x7fbecc15d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.481+0000 7fbecbfff640 1 --2- 172.21.15.84:0/508086420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbecc0b1ca0 0x7fbecc0cc4c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.481+0000 7fbecbfff640 1 -- 172.21.15.84:0/508086420 >> 172.21.15.84:0/508086420 conn(0x7fbecc0d2b90 msgr2=0x7fbecc06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:16.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.481+0000 7fbecbfff640 1 -- 172.21.15.84:0/508086420 shutdown_connections 2023-10-16T09:49:16.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.481+0000 7fbecbfff640 1 -- 172.21.15.84:0/508086420 wait complete. 2023-10-16T09:49:16.500 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:49:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.715+0000 7f0f48a75640 1 Processor -- start 2023-10-16T09:49:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.715+0000 7f0f48a75640 1 -- start start 2023-10-16T09:49:16.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.717+0000 7f0f48a75640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f4406a4b0 0x7f0f440ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:16.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.717+0000 7f0f48a75640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0f440c9ca0 0x7f0f440ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:16.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.717+0000 7f0f48a75640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f440cb030 con 0x7f0f4415ad70 2023-10-16T09:49:16.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.717+0000 7f0f48a75640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f440cae30 con 0x7f0f440c9ca0 2023-10-16T09:49:16.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.717+0000 7f0f48a75640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f440c91c0 con 0x7f0f4406a4b0 2023-10-16T09:49:16.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.717+0000 7f0f4259c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0f4415ad70 0x7f0f44069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36804/0 (socket says 172.21.15.84:36804) 2023-10-16T09:49:16.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.717+0000 7f0f4259c640 1 -- 172.21.15.84:0/3071385281 learned_addr learned my addr 172.21.15.84:0/3071385281 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:16.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.717+0000 7f0f3bfff640 1 --2- 172.21.15.84:0/3071385281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f4406a4b0 0x7f0f440ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:16.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.717+0000 7f0f41d9b640 1 -- 172.21.15.84:0/3071385281 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3398562532 0 0) 0x7f0f440cb030 con 0x7f0f4415ad70 2023-10-16T09:49:16.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.717+0000 7f0f41d9b640 1 -- 172.21.15.84:0/3071385281 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f30003520 con 0x7f0f4415ad70 2023-10-16T09:49:16.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.718+0000 7f0f42d9d640 1 --2- 172.21.15.84:0/3071385281 >> v2:172.21.15.157:3300/0 conn(0x7f0f440c9ca0 0x7f0f440ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:16.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.718+0000 7f0f41d9b640 1 -- 172.21.15.84:0/3071385281 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4256683556 0 0) 0x7f0f30003520 con 0x7f0f4415ad70 2023-10-16T09:49:16.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.718+0000 7f0f41d9b640 1 -- 172.21.15.84:0/3071385281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f4406a4b0 msgr2=0x7f0f440ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:16.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.718+0000 7f0f41d9b640 1 --2- 172.21.15.84:0/3071385281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f4406a4b0 0x7f0f440ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.718+0000 7f0f41d9b640 1 -- 172.21.15.84:0/3071385281 >> v2:172.21.15.157:3300/0 conn(0x7f0f440c9ca0 msgr2=0x7f0f440ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:16.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.718+0000 7f0f41d9b640 1 --2- 172.21.15.84:0/3071385281 >> v2:172.21.15.157:3300/0 conn(0x7f0f440c9ca0 0x7f0f440ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.718+0000 7f0f41d9b640 1 -- 172.21.15.84:0/3071385281 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f440c94a0 con 0x7f0f4415ad70 2023-10-16T09:49:16.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.718+0000 7f0f41d9b640 1 -- 172.21.15.84:0/3071385281 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f0f34001670 con 0x7f0f4415ad70 2023-10-16T09:49:16.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.718+0000 7f0f41d9b640 1 -- 172.21.15.84:0/3071385281 >> v1:172.21.15.84:6789/0 conn(0x7f0f4415ad70 legacy=0x7f0f44069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:16.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.718+0000 7f0f41d9b640 1 --2- 172.21.15.84:0/3071385281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f30003970 0x7f0f30003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:16.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.718+0000 7f0f41d9b640 1 --2- 172.21.15.84:0/3071385281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f300047c0 0x7f0f30004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:16.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.718+0000 7f0f41d9b640 1 --2- 172.21.15.84:0/3071385281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f30005180 0x7f0f3000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:16.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.718+0000 7f0f41d9b640 1 -- 172.21.15.84:0/3071385281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f440cae30 con 0x7f0f30005180 2023-10-16T09:49:16.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.718+0000 7f0f41d9b640 1 -- 172.21.15.84:0/3071385281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f440c91c0 con 0x7f0f30003970 2023-10-16T09:49:16.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.718+0000 7f0f41d9b640 1 -- 172.21.15.84:0/3071385281 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3000c020 con 0x7f0f300047c0 2023-10-16T09:49:16.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.718+0000 7f0f3bfff640 1 --2- 172.21.15.84:0/3071385281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f4406a4b0 0x7f0f440ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:16.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.718+0000 7f0f42d9d640 1 --2- 172.21.15.84:0/3071385281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f30005180 0x7f0f3000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:16.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.719+0000 7f0f4259c640 1 --2- 172.21.15.84:0/3071385281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f30003970 0x7f0f30003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:16.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.719+0000 7f0f42d9d640 1 -- 172.21.15.84:0/3071385281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f300047c0 msgr2=0x7f0f30004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:16.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.719+0000 7f0f42d9d640 1 --2- 172.21.15.84:0/3071385281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f300047c0 0x7f0f30004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.719+0000 7f0f42d9d640 1 -- 172.21.15.84:0/3071385281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f30003970 msgr2=0x7f0f30003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:16.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.719+0000 7f0f42d9d640 1 --2- 172.21.15.84:0/3071385281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f30003970 0x7f0f30003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.719+0000 7f0f42d9d640 1 -- 172.21.15.84:0/3071385281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f30003520 con 0x7f0f30005180 2023-10-16T09:49:16.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.720+0000 7f0f42d9d640 1 --2- 172.21.15.84:0/3071385281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f30005180 0x7f0f3000b980 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c007360 tx=0x7f0f300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9e055d8c6f24ce server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:16.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.720+0000 7f0f41d9b640 1 -- 172.21.15.84:0/3071385281 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c02ccf0 con 0x7f0f30005180 2023-10-16T09:49:16.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.720+0000 7f0f41d9b640 1 -- 172.21.15.84:0/3071385281 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3c02ce50 con 0x7f0f30005180 2023-10-16T09:49:16.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.720+0000 7f0f41d9b640 1 -- 172.21.15.84:0/3071385281 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c034660 con 0x7f0f30005180 2023-10-16T09:49:16.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.720+0000 7f0f48a75640 1 -- 172.21.15.84:0/3071385281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f30005180 msgr2=0x7f0f3000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:16.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.720+0000 7f0f48a75640 1 --2- 172.21.15.84:0/3071385281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f30005180 0x7f0f3000b980 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c007360 tx=0x7f0f300057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.720+0000 7f0f42d9d640 1 -- 172.21.15.84:0/3071385281 reap_dead start 2023-10-16T09:49:16.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.721+0000 7f0f48a75640 1 -- 172.21.15.84:0/3071385281 shutdown_connections 2023-10-16T09:49:16.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.721+0000 7f0f48a75640 1 -- 172.21.15.84:0/3071385281 >> 172.21.15.84:0/3071385281 conn(0x7f0f440ce9b0 msgr2=0x7f0f440cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:16.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.721+0000 7f0f48a75640 1 -- 172.21.15.84:0/3071385281 shutdown_connections 2023-10-16T09:49:16.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.721+0000 7f0f48a75640 1 -- 172.21.15.84:0/3071385281 wait complete. 2023-10-16T09:49:16.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.722+0000 7f0f48a75640 1 Processor -- start 2023-10-16T09:49:16.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.723+0000 7f0f48a75640 1 -- start start 2023-10-16T09:49:16.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.723+0000 7f0f48a75640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f30005180 0x7f0f4414ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:16.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.723+0000 7f0f48a75640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f440c7c70 0x7f0f440c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:16.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.724+0000 7f0f48a75640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f440c5fc0 0x7f0f4414f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:16.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.724+0000 7f0f48a75640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f440c6450 con 0x7f0f440c5fc0 2023-10-16T09:49:16.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.724+0000 7f0f48a75640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4414d370 con 0x7f0f30005180 2023-10-16T09:49:16.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.724+0000 7f0f48a75640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f440c8110 con 0x7f0f440c7c70 2023-10-16T09:49:16.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.724+0000 7f0f42d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f440c5fc0 0x7f0f4414f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:16.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.724+0000 7f0f42d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f440c5fc0 0x7f0f4414f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33884/0 (socket says 172.21.15.84:33884) 2023-10-16T09:49:16.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.724+0000 7f0f42d9d640 1 -- 172.21.15.84:0/3892671404 learned_addr learned my addr 172.21.15.84:0/3892671404 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:16.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.724+0000 7f0f3bfff640 1 --2- 172.21.15.84:0/3892671404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f440c7c70 0x7f0f440c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:16.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.724+0000 7f0f4259c640 1 --2- 172.21.15.84:0/3892671404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f30005180 0x7f0f4414ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:16.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.724+0000 7f0f42d9d640 1 -- 172.21.15.84:0/3892671404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f440c7c70 msgr2=0x7f0f440c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:16.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.724+0000 7f0f42d9d640 1 --2- 172.21.15.84:0/3892671404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f440c7c70 0x7f0f440c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.724+0000 7f0f42d9d640 1 -- 172.21.15.84:0/3892671404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f30005180 msgr2=0x7f0f4414ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:16.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.724+0000 7f0f42d9d640 1 --2- 172.21.15.84:0/3892671404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f30005180 0x7f0f4414ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.724+0000 7f0f42d9d640 1 -- 172.21.15.84:0/3892671404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f440c83f0 con 0x7f0f440c5fc0 2023-10-16T09:49:16.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.724+0000 7f0f42d9d640 1 --2- 172.21.15.84:0/3892671404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f440c5fc0 0x7f0f4414f8a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c02dfd0 tx=0x7f0f3c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:16.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.725+0000 7f0f3affd640 1 -- 172.21.15.84:0/3892671404 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c045070 con 0x7f0f440c5fc0 2023-10-16T09:49:16.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.725+0000 7f0f3affd640 1 -- 172.21.15.84:0/3892671404 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3c034e60 con 0x7f0f440c5fc0 2023-10-16T09:49:16.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.725+0000 7f0f3affd640 1 -- 172.21.15.84:0/3892671404 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c03d620 con 0x7f0f440c5fc0 2023-10-16T09:49:16.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.725+0000 7f0f48a75640 1 -- 172.21.15.84:0/3892671404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f440c86d0 con 0x7f0f440c5fc0 2023-10-16T09:49:16.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.725+0000 7f0f48a75640 1 -- 172.21.15.84:0/3892671404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f440c8c90 con 0x7f0f440c5fc0 2023-10-16T09:49:16.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.726+0000 7f0f48a75640 1 -- 172.21.15.84:0/3892671404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f4404e160 con 0x7f0f440c5fc0 2023-10-16T09:49:16.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.727+0000 7f0f3affd640 1 -- 172.21.15.84:0/3892671404 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0f3c03b040 con 0x7f0f440c5fc0 2023-10-16T09:49:16.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.728+0000 7f0f3affd640 1 --2- 172.21.15.84:0/3892671404 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f2406aea0 0x7f0f2406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:16.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.728+0000 7f0f3affd640 1 -- 172.21.15.84:0/3892671404 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f3c0aa6e0 con 0x7f0f440c5fc0 2023-10-16T09:49:16.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.728+0000 7f0f4259c640 1 --2- 172.21.15.84:0/3892671404 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f2406aea0 0x7f0f2406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:16.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.729+0000 7f0f4259c640 1 --2- 172.21.15.84:0/3892671404 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f2406aea0 0x7f0f2406d360 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f34001eb0 tx=0x7f0f34000f50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:16.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.732+0000 7f0f3affd640 1 -- 172.21.15.84:0/3892671404 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0f3c0756f0 con 0x7f0f440c5fc0 2023-10-16T09:49:16.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.921+0000 7f0f48a75640 1 -- 172.21.15.84:0/3892671404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0f44069f20 con 0x7f0f440c5fc0 2023-10-16T09:49:16.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.922+0000 7f0f3affd640 1 -- 172.21.15.84:0/3892671404 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0f3c078d10 con 0x7f0f440c5fc0 2023-10-16T09:49:16.924 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T09:49:16.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.924+0000 7f0f48a75640 1 -- 172.21.15.84:0/3892671404 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f2406aea0 msgr2=0x7f0f2406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:16.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.924+0000 7f0f48a75640 1 --2- 172.21.15.84:0/3892671404 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f2406aea0 0x7f0f2406d360 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f34001eb0 tx=0x7f0f34000f50 comp rx=0 tx=0).stop 2023-10-16T09:49:16.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.924+0000 7f0f48a75640 1 -- 172.21.15.84:0/3892671404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f440c5fc0 msgr2=0x7f0f4414f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:16.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.924+0000 7f0f48a75640 1 --2- 172.21.15.84:0/3892671404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f440c5fc0 0x7f0f4414f8a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c02dfd0 tx=0x7f0f3c004ab0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.925+0000 7f0f48a75640 1 -- 172.21.15.84:0/3892671404 shutdown_connections 2023-10-16T09:49:16.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.925+0000 7f0f48a75640 1 --2- 172.21.15.84:0/3892671404 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f2406aea0 0x7f0f2406d360 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.925+0000 7f0f48a75640 1 --2- 172.21.15.84:0/3892671404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f440c5fc0 0x7f0f4414f8a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.925+0000 7f0f48a75640 1 --2- 172.21.15.84:0/3892671404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f440c7c70 0x7f0f440c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.925+0000 7f0f48a75640 1 --2- 172.21.15.84:0/3892671404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f30005180 0x7f0f4414ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:16.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.925+0000 7f0f48a75640 1 -- 172.21.15.84:0/3892671404 >> 172.21.15.84:0/3892671404 conn(0x7f0f440ce9b0 msgr2=0x7f0f4415dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:16.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.925+0000 7f0f48a75640 1 -- 172.21.15.84:0/3892671404 shutdown_connections 2023-10-16T09:49:16.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:16.925+0000 7f0f48a75640 1 -- 172.21.15.84:0/3892671404 wait complete. 2023-10-16T09:49:16.944 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:49:17.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.155+0000 7f1449a0e640 1 Processor -- start 2023-10-16T09:49:17.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.155+0000 7f1449a0e640 1 -- start start 2023-10-16T09:49:17.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.157+0000 7f1449a0e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f144406b660 0x7f144406ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:17.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.157+0000 7f1449a0e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14440cc6b0 con 0x7f14440c9860 2023-10-16T09:49:17.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.157+0000 7f1449a0e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14441599b0 con 0x7f14441428c0 2023-10-16T09:49:17.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.157+0000 7f1449a0e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144406c290 con 0x7f144406b660 2023-10-16T09:49:17.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.157+0000 7f14437fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f14440c9860 0x7f14440ccb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36816/0 (socket says 172.21.15.84:36816) 2023-10-16T09:49:17.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.157+0000 7f14437fe640 1 -- 172.21.15.84:0/862007161 learned_addr learned my addr 172.21.15.84:0/862007161 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:17.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.158+0000 7f1442ffd640 1 --2- 172.21.15.84:0/862007161 >> v2:172.21.15.157:3300/0 conn(0x7f144406b660 0x7f144406ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:17.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.158+0000 7f14427fc640 1 -- 172.21.15.84:0/862007161 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2707413774 0 0) 0x7f14440cc6b0 con 0x7f14440c9860 2023-10-16T09:49:17.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.158+0000 7f14427fc640 1 -- 172.21.15.84:0/862007161 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1430003440 con 0x7f14440c9860 2023-10-16T09:49:17.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.158+0000 7f14427fc640 1 -- 172.21.15.84:0/862007161 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2029991621 0 0) 0x7f1430003440 con 0x7f14440c9860 2023-10-16T09:49:17.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.158+0000 7f14427fc640 1 -- 172.21.15.84:0/862007161 >> v2:172.21.15.157:3300/0 conn(0x7f144406b660 msgr2=0x7f144406ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:17.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.158+0000 7f14427fc640 1 --2- 172.21.15.84:0/862007161 >> v2:172.21.15.157:3300/0 conn(0x7f144406b660 0x7f144406ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.158+0000 7f14427fc640 1 -- 172.21.15.84:0/862007161 >> v1:172.21.15.157:6789/0 conn(0x7f14441428c0 legacy=0x7f14440c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:17.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.158+0000 7f14427fc640 1 -- 172.21.15.84:0/862007161 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144406c570 con 0x7f14440c9860 2023-10-16T09:49:17.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.158+0000 7f14427fc640 1 -- 172.21.15.84:0/862007161 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f143c0019f0 con 0x7f14440c9860 2023-10-16T09:49:17.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.158+0000 7f14427fc640 1 -- 172.21.15.84:0/862007161 >> v1:172.21.15.84:6789/0 conn(0x7f14440c9860 legacy=0x7f14440ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:17.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.158+0000 7f14427fc640 1 --2- 172.21.15.84:0/862007161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1430003ab0 0x7f1430003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:17.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.158+0000 7f14427fc640 1 --2- 172.21.15.84:0/862007161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1430008850 0x7f143000ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:17.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.158+0000 7f14427fc640 1 --2- 172.21.15.84:0/862007161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f143000b530 0x7f143000d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:17.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.158+0000 7f14427fc640 1 -- 172.21.15.84:0/862007161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144406c290 con 0x7f1430008850 2023-10-16T09:49:17.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.159+0000 7f14427fc640 1 -- 172.21.15.84:0/862007161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143000e2d0 con 0x7f143000b530 2023-10-16T09:49:17.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.159+0000 7f14427fc640 1 -- 172.21.15.84:0/862007161 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143000e490 con 0x7f1430003ab0 2023-10-16T09:49:17.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.159+0000 7f143a7fc640 1 --2- 172.21.15.84:0/862007161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1430008850 0x7f143000ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:17.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.159+0000 7f1442ffd640 1 --2- 172.21.15.84:0/862007161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1430003ab0 0x7f1430003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:17.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.159+0000 7f14437fe640 1 --2- 172.21.15.84:0/862007161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f143000b530 0x7f143000d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:17.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.159+0000 7f143a7fc640 1 -- 172.21.15.84:0/862007161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1430003ab0 msgr2=0x7f1430003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:17.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.159+0000 7f143a7fc640 1 --2- 172.21.15.84:0/862007161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1430003ab0 0x7f1430003ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.159+0000 7f143a7fc640 1 -- 172.21.15.84:0/862007161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f143000b530 msgr2=0x7f143000d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:17.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.159+0000 7f143a7fc640 1 --2- 172.21.15.84:0/862007161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f143000b530 0x7f143000d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.159+0000 7f143a7fc640 1 -- 172.21.15.84:0/862007161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1430003440 con 0x7f1430008850 2023-10-16T09:49:17.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.159+0000 7f14437fe640 1 --2- 172.21.15.84:0/862007161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f143000b530 0x7f143000d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:17.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.159+0000 7f14437fe640 1 -- 172.21.15.84:0/862007161 reap_dead start 2023-10-16T09:49:17.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.160+0000 7f143a7fc640 1 --2- 172.21.15.84:0/862007161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1430008850 0x7f143000ace0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f142c001750 tx=0x7f14300036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cfa93a239fc4adb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:17.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.160+0000 7f14427fc640 1 -- 172.21.15.84:0/862007161 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c0052d0 con 0x7f1430008850 2023-10-16T09:49:17.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.160+0000 7f14427fc640 1 -- 172.21.15.84:0/862007161 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c02e050 con 0x7f1430008850 2023-10-16T09:49:17.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.160+0000 7f14427fc640 1 -- 172.21.15.84:0/862007161 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c003830 con 0x7f1430008850 2023-10-16T09:49:17.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.161+0000 7f1449a0e640 1 -- 172.21.15.84:0/862007161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1430008850 msgr2=0x7f143000ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:17.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.161+0000 7f1449a0e640 1 --2- 172.21.15.84:0/862007161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1430008850 0x7f143000ace0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f142c001750 tx=0x7f14300036f0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.161+0000 7f1449a0e640 1 -- 172.21.15.84:0/862007161 shutdown_connections 2023-10-16T09:49:17.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.161+0000 7f1449a0e640 1 --2- 172.21.15.84:0/862007161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1430008850 0x7f143000ace0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.161+0000 7f1449a0e640 1 -- 172.21.15.84:0/862007161 >> 172.21.15.84:0/862007161 conn(0x7f14440ce9b0 msgr2=0x7f14440cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:17.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.161+0000 7f1449a0e640 1 -- 172.21.15.84:0/862007161 shutdown_connections 2023-10-16T09:49:17.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.161+0000 7f1449a0e640 1 -- 172.21.15.84:0/862007161 wait complete. 2023-10-16T09:49:17.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.163+0000 7f1449a0e640 1 Processor -- start 2023-10-16T09:49:17.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.163+0000 7f1449a0e640 1 -- start start 2023-10-16T09:49:17.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.164+0000 7f1449a0e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1430008850 0x7f14440c4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:17.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.164+0000 7f1449a0e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14440ac8c0 0x7f14440acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:17.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.164+0000 7f1449a0e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14440c7030 0x7f14440c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:17.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.164+0000 7f1449a0e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14441f6af0 con 0x7f14440ac8c0 2023-10-16T09:49:17.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.164+0000 7f1449a0e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14440ad280 con 0x7f1430008850 2023-10-16T09:49:17.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.164+0000 7f1449a0e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14441f5e70 con 0x7f14440c7030 2023-10-16T09:49:17.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.164+0000 7f143a7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14440ac8c0 0x7f14440acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:17.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.164+0000 7f143a7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14440ac8c0 0x7f14440acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33906/0 (socket says 172.21.15.84:33906) 2023-10-16T09:49:17.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.164+0000 7f143a7fc640 1 -- 172.21.15.84:0/3152086397 learned_addr learned my addr 172.21.15.84:0/3152086397 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:17.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.165+0000 7f14437fe640 1 --2- 172.21.15.84:0/3152086397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14440c7030 0x7f14440c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:17.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.165+0000 7f143a7fc640 1 -- 172.21.15.84:0/3152086397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14440c7030 msgr2=0x7f14440c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:17.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.165+0000 7f143a7fc640 1 --2- 172.21.15.84:0/3152086397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14440c7030 0x7f14440c74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.165+0000 7f143a7fc640 1 -- 172.21.15.84:0/3152086397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1430008850 msgr2=0x7f14440c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:17.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.165+0000 7f143a7fc640 1 --2- 172.21.15.84:0/3152086397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1430008850 0x7f14440c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.165+0000 7f143a7fc640 1 -- 172.21.15.84:0/3152086397 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14441f6150 con 0x7f14440ac8c0 2023-10-16T09:49:17.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.165+0000 7f143a7fc640 1 --2- 172.21.15.84:0/3152086397 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14440ac8c0 0x7f14440acd40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f142c02bc00 tx=0x7f142c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:17.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.165+0000 7f1440ff9640 1 -- 172.21.15.84:0/3152086397 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c0076c0 con 0x7f14440ac8c0 2023-10-16T09:49:17.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.165+0000 7f1440ff9640 1 -- 172.21.15.84:0/3152086397 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c007a30 con 0x7f14440ac8c0 2023-10-16T09:49:17.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.165+0000 7f1449a0e640 1 -- 172.21.15.84:0/3152086397 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14441f6430 con 0x7f14440ac8c0 2023-10-16T09:49:17.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.165+0000 7f1440ff9640 1 -- 172.21.15.84:0/3152086397 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c006c20 con 0x7f14440ac8c0 2023-10-16T09:49:17.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.166+0000 7f1449a0e640 1 -- 172.21.15.84:0/3152086397 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14440c47b0 con 0x7f14440ac8c0 2023-10-16T09:49:17.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.167+0000 7f1449a0e640 1 -- 172.21.15.84:0/3152086397 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14440adcd0 con 0x7f14440ac8c0 2023-10-16T09:49:17.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.168+0000 7f1440ff9640 1 -- 172.21.15.84:0/3152086397 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f142c0035f0 con 0x7f14440ac8c0 2023-10-16T09:49:17.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.169+0000 7f1440ff9640 1 --2- 172.21.15.84:0/3152086397 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f142406aea0 0x7f142406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:17.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.169+0000 7f1440ff9640 1 -- 172.21.15.84:0/3152086397 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f142c0ab140 con 0x7f14440ac8c0 2023-10-16T09:49:17.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.169+0000 7f1442ffd640 1 --2- 172.21.15.84:0/3152086397 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f142406aea0 0x7f142406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:17.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.169+0000 7f1442ffd640 1 --2- 172.21.15.84:0/3152086397 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f142406aea0 0x7f142406d360 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f14440c96c0 tx=0x7f1434008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:17.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.173+0000 7f1440ff9640 1 -- 172.21.15.84:0/3152086397 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f142c079d10 con 0x7f14440ac8c0 2023-10-16T09:49:17.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.364+0000 7f1449a0e640 1 -- 172.21.15.84:0/3152086397 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f144404e160 con 0x7f14440ac8c0 2023-10-16T09:49:17.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.364+0000 7f1440ff9640 1 -- 172.21.15.84:0/3152086397 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f142c0798a0 con 0x7f14440ac8c0 2023-10-16T09:49:17.366 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T09:49:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.367+0000 7f1449a0e640 1 -- 172.21.15.84:0/3152086397 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f142406aea0 msgr2=0x7f142406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.367+0000 7f1449a0e640 1 --2- 172.21.15.84:0/3152086397 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f142406aea0 0x7f142406d360 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f14440c96c0 tx=0x7f1434008040 comp rx=0 tx=0).stop 2023-10-16T09:49:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.367+0000 7f1449a0e640 1 -- 172.21.15.84:0/3152086397 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14440ac8c0 msgr2=0x7f14440acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.367+0000 7f1449a0e640 1 --2- 172.21.15.84:0/3152086397 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14440ac8c0 0x7f14440acd40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f142c02bc00 tx=0x7f142c001120 comp rx=0 tx=0).stop 2023-10-16T09:49:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.367+0000 7f1449a0e640 1 -- 172.21.15.84:0/3152086397 shutdown_connections 2023-10-16T09:49:17.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.367+0000 7f1449a0e640 1 --2- 172.21.15.84:0/3152086397 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f142406aea0 0x7f142406d360 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.367+0000 7f1449a0e640 1 --2- 172.21.15.84:0/3152086397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14440c7030 0x7f14440c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.367+0000 7f1449a0e640 1 --2- 172.21.15.84:0/3152086397 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14440ac8c0 0x7f14440acd40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.367+0000 7f1449a0e640 1 --2- 172.21.15.84:0/3152086397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1430008850 0x7f14440c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.367+0000 7f1449a0e640 1 -- 172.21.15.84:0/3152086397 >> 172.21.15.84:0/3152086397 conn(0x7f14440ce9b0 msgr2=0x7f144406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:17.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.367+0000 7f1449a0e640 1 -- 172.21.15.84:0/3152086397 shutdown_connections 2023-10-16T09:49:17.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.367+0000 7f1449a0e640 1 -- 172.21.15.84:0/3152086397 wait complete. 2023-10-16T09:49:17.388 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:49:17.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.594+0000 7fe88359e640 1 Processor -- start 2023-10-16T09:49:17.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.594+0000 7fe88359e640 1 -- start start 2023-10-16T09:49:17.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.596+0000 7fe88359e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe88406c040 0x7fe88406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:17.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.596+0000 7fe88359e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe8840ad970 0x7fe88415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:17.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.596+0000 7fe88359e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88406c960 con 0x7fe88415ebc0 2023-10-16T09:49:17.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.596+0000 7fe88359e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe88415b9d0 con 0x7fe8840ad970 2023-10-16T09:49:17.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.596+0000 7fe88359e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe88415bcb0 con 0x7fe88406c040 2023-10-16T09:49:17.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.596+0000 7fe87bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe8840ad970 0x7fe88415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:17.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.596+0000 7fe87bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe8840ad970 0x7fe88415e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33922/0 (socket says 172.21.15.84:33922) 2023-10-16T09:49:17.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.596+0000 7fe87bfff640 1 -- 172.21.15.84:0/3303086633 learned_addr learned my addr 172.21.15.84:0/3303086633 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:17.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.596+0000 7fe88259c640 1 --2- 172.21.15.84:0/3303086633 >> v2:172.21.15.157:3300/0 conn(0x7fe88406c040 0x7fe88406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:17.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.596+0000 7fe87bfff640 1 -- 172.21.15.84:0/3303086633 >> v2:172.21.15.157:3300/0 conn(0x7fe88406c040 msgr2=0x7fe88406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:17.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.596+0000 7fe87bfff640 1 --2- 172.21.15.84:0/3303086633 >> v2:172.21.15.157:3300/0 conn(0x7fe88406c040 0x7fe88406c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.596+0000 7fe87bfff640 1 -- 172.21.15.84:0/3303086633 >> v1:172.21.15.157:6789/0 conn(0x7fe88415ebc0 legacy=0x7fe8840cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:17.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.596+0000 7fe87bfff640 1 -- 172.21.15.84:0/3303086633 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88415bf90 con 0x7fe8840ad970 2023-10-16T09:49:17.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.597+0000 7fe87bfff640 1 --2- 172.21.15.84:0/3303086633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8840ad970 0x7fe88415e680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8840cf350 tx=0x7fe86c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59fc2ce20e17738 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:17.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.598+0000 7fe881d9b640 1 -- 172.21.15.84:0/3303086633 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c031870 con 0x7fe8840ad970 2023-10-16T09:49:17.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.598+0000 7fe881d9b640 1 -- 172.21.15.84:0/3303086633 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86c0319d0 con 0x7fe8840ad970 2023-10-16T09:49:17.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.598+0000 7fe881d9b640 1 -- 172.21.15.84:0/3303086633 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c031d60 con 0x7fe8840ad970 2023-10-16T09:49:17.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.598+0000 7fe88359e640 1 -- 172.21.15.84:0/3303086633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8840ad970 msgr2=0x7fe88415e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:17.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.598+0000 7fe88359e640 1 --2- 172.21.15.84:0/3303086633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8840ad970 0x7fe88415e680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8840cf350 tx=0x7fe86c02b910 comp rx=0 tx=0).stop 2023-10-16T09:49:17.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.598+0000 7fe88359e640 1 -- 172.21.15.84:0/3303086633 shutdown_connections 2023-10-16T09:49:17.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.598+0000 7fe88359e640 1 --2- 172.21.15.84:0/3303086633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8840ad970 0x7fe88415e680 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.598+0000 7fe88359e640 1 --2- 172.21.15.84:0/3303086633 >> v2:172.21.15.157:3300/0 conn(0x7fe88406c040 0x7fe88406c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.598+0000 7fe88359e640 1 -- 172.21.15.84:0/3303086633 >> 172.21.15.84:0/3303086633 conn(0x7fe8840ce6a0 msgr2=0x7fe8840ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:17.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.599+0000 7fe88359e640 1 -- 172.21.15.84:0/3303086633 shutdown_connections 2023-10-16T09:49:17.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.599+0000 7fe88359e640 1 -- 172.21.15.84:0/3303086633 wait complete. 2023-10-16T09:49:17.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.600+0000 7fe88359e640 1 Processor -- start 2023-10-16T09:49:17.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.600+0000 7fe88359e640 1 -- start start 2023-10-16T09:49:17.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.601+0000 7fe88359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88406c040 0x7fe8840ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:17.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.601+0000 7fe88359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8840ad970 0x7fe8840c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:17.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.601+0000 7fe88359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe88415ebc0 0x7fe8841539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:17.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.601+0000 7fe88359e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8840aee90 con 0x7fe88415ebc0 2023-10-16T09:49:17.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.601+0000 7fe88359e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8840ca430 con 0x7fe8840ad970 2023-10-16T09:49:17.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.601+0000 7fe88359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8840c9380 con 0x7fe88406c040 2023-10-16T09:49:17.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.601+0000 7fe882d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe88415ebc0 0x7fe8841539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:17.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.601+0000 7fe882d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe88415ebc0 0x7fe8841539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33924/0 (socket says 172.21.15.84:33924) 2023-10-16T09:49:17.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.601+0000 7fe882d9d640 1 -- 172.21.15.84:0/4268334121 learned_addr learned my addr 172.21.15.84:0/4268334121 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:17.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.602+0000 7fe88259c640 1 --2- 172.21.15.84:0/4268334121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88406c040 0x7fe8840ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:17.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.602+0000 7fe882d9d640 1 -- 172.21.15.84:0/4268334121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88406c040 msgr2=0x7fe8840ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:17.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.602+0000 7fe882d9d640 1 --2- 172.21.15.84:0/4268334121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88406c040 0x7fe8840ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.602+0000 7fe882d9d640 1 -- 172.21.15.84:0/4268334121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8840ad970 msgr2=0x7fe8840c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:17.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.602+0000 7fe882d9d640 1 --2- 172.21.15.84:0/4268334121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8840ad970 0x7fe8840c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.602+0000 7fe882d9d640 1 -- 172.21.15.84:0/4268334121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c0094d0 con 0x7fe88415ebc0 2023-10-16T09:49:17.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.602+0000 7fe882d9d640 1 --2- 172.21.15.84:0/4268334121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe88415ebc0 0x7fe8841539f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00c780 tx=0x7fe87c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:17.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.602+0000 7fe87affd640 1 -- 172.21.15.84:0/4268334121 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c004b60 con 0x7fe88415ebc0 2023-10-16T09:49:17.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.602+0000 7fe87affd640 1 -- 172.21.15.84:0/4268334121 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87c00ce90 con 0x7fe88415ebc0 2023-10-16T09:49:17.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.602+0000 7fe87affd640 1 -- 172.21.15.84:0/4268334121 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c00a850 con 0x7fe88415ebc0 2023-10-16T09:49:17.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.602+0000 7fe88359e640 1 -- 172.21.15.84:0/4268334121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8840c9660 con 0x7fe88415ebc0 2023-10-16T09:49:17.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.603+0000 7fe88359e640 1 -- 172.21.15.84:0/4268334121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8840c9c20 con 0x7fe88415ebc0 2023-10-16T09:49:17.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.604+0000 7fe88359e640 1 -- 172.21.15.84:0/4268334121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe88406b330 con 0x7fe88415ebc0 2023-10-16T09:49:17.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.605+0000 7fe87affd640 1 -- 172.21.15.84:0/4268334121 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe87c00a9b0 con 0x7fe88415ebc0 2023-10-16T09:49:17.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.605+0000 7fe87affd640 1 --2- 172.21.15.84:0/4268334121 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe86406ae70 0x7fe86406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:17.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.606+0000 7fe87affd640 1 -- 172.21.15.84:0/4268334121 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe87c089760 con 0x7fe88415ebc0 2023-10-16T09:49:17.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.606+0000 7fe88259c640 1 --2- 172.21.15.84:0/4268334121 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe86406ae70 0x7fe86406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:17.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.606+0000 7fe88259c640 1 --2- 172.21.15.84:0/4268334121 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe86406ae70 0x7fe86406d330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe874006fd0 tx=0x7fe874008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:17.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.610+0000 7fe87affd640 1 -- 172.21.15.84:0/4268334121 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe87c0548a0 con 0x7fe88415ebc0 2023-10-16T09:49:17.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.824+0000 7fe88359e640 1 -- 172.21.15.84:0/4268334121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8840aebf0 con 0x7fe88415ebc0 2023-10-16T09:49:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.825+0000 7fe87affd640 1 -- 172.21.15.84:0/4268334121 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7fe8840aebf0 con 0x7fe88415ebc0 2023-10-16T09:49:17.827 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:17.828 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2978888131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2978888131},{"type":"v1","addr":"172.21.15.157:6834","nonce":2978888131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3716581310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3716581310},{"type":"v1","addr":"172.21.15.84:6835","nonce":3716581310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2761479872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2761479872},{"type":"v1","addr":"172.21.15.84:6837","nonce":2761479872}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:49:02.350624+0000","modified":"2023-10-16T09:49:05.402785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/848754825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":848754825},{"type":"v1","addr":"172.21.15.157:6835","nonce":848754825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T09:49:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-10-16T09:49:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.828+0000 7fe88359e640 1 -- 172.21.15.84:0/4268334121 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe86406ae70 msgr2=0x7fe86406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.828+0000 7fe88359e640 1 --2- 172.21.15.84:0/4268334121 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe86406ae70 0x7fe86406d330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe874006fd0 tx=0x7fe874008040 comp rx=0 tx=0).stop 2023-10-16T09:49:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.828+0000 7fe88359e640 1 -- 172.21.15.84:0/4268334121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe88415ebc0 msgr2=0x7fe8841539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.828+0000 7fe88359e640 1 --2- 172.21.15.84:0/4268334121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe88415ebc0 0x7fe8841539f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00c780 tx=0x7fe87c00cc40 comp rx=0 tx=0).stop 2023-10-16T09:49:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.828+0000 7fe88359e640 1 -- 172.21.15.84:0/4268334121 shutdown_connections 2023-10-16T09:49:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.828+0000 7fe88359e640 1 --2- 172.21.15.84:0/4268334121 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe86406ae70 0x7fe86406d330 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.828+0000 7fe88359e640 1 --2- 172.21.15.84:0/4268334121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe88415ebc0 0x7fe8841539f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.828+0000 7fe88359e640 1 --2- 172.21.15.84:0/4268334121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8840ad970 0x7fe8840c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.828+0000 7fe88359e640 1 --2- 172.21.15.84:0/4268334121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88406c040 0x7fe8840ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.828+0000 7fe88359e640 1 -- 172.21.15.84:0/4268334121 >> 172.21.15.84:0/4268334121 conn(0x7fe8840ce6a0 msgr2=0x7fe8840cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.828+0000 7fe88359e640 1 -- 172.21.15.84:0/4268334121 shutdown_connections 2023-10-16T09:49:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:17.828+0000 7fe88359e640 1 -- 172.21.15.84:0/4268334121 wait complete. 2023-10-16T09:49:17.848 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:18.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.054+0000 7f097011d640 1 Processor -- start 2023-10-16T09:49:18.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.054+0000 7f097011d640 1 -- start start 2023-10-16T09:49:18.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.055+0000 7f097011d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f096815aaa0 0x7f096815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:18.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.055+0000 7f097011d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f09680ce390 0x7f0968070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:18.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.055+0000 7f097011d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f096815ed60 con 0x7f096815ae80 2023-10-16T09:49:18.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.055+0000 7f097011d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f096806b810 con 0x7f09680ce390 2023-10-16T09:49:18.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.055+0000 7f097011d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09680707e0 con 0x7f096815aaa0 2023-10-16T09:49:18.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.056+0000 7f096deb9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f096815aaa0 0x7f096815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:18.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.056+0000 7f096deb9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f096815aaa0 0x7f096815e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:36340/0 (socket says 172.21.15.84:36340) 2023-10-16T09:49:18.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.056+0000 7f096deb9640 1 -- 172.21.15.84:0/1873538502 learned_addr learned my addr 172.21.15.84:0/1873538502 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:18.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.056+0000 7f096e6ba640 1 --2- 172.21.15.84:0/1873538502 >> v2:172.21.15.157:3300/0 conn(0x7f09680ce390 0x7f0968070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:18.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.057+0000 7f096deb9640 1 -- 172.21.15.84:0/1873538502 >> v2:172.21.15.157:3300/0 conn(0x7f09680ce390 msgr2=0x7f0968070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:18.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.057+0000 7f096deb9640 1 --2- 172.21.15.84:0/1873538502 >> v2:172.21.15.157:3300/0 conn(0x7f09680ce390 0x7f0968070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.057+0000 7f096deb9640 1 -- 172.21.15.84:0/1873538502 >> v1:172.21.15.157:6789/0 conn(0x7f096815ae80 legacy=0x7f09680cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:18.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.057+0000 7f096deb9640 1 -- 172.21.15.84:0/1873538502 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096815ef90 con 0x7f096815aaa0 2023-10-16T09:49:18.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.058+0000 7f096deb9640 1 --2- 172.21.15.84:0/1873538502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f096815aaa0 0x7f096815e820 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0964009870 tx=0x7f096402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e55b3122f684da84 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:18.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.058+0000 7f096ceb7640 1 -- 172.21.15.84:0/1873538502 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0964030840 con 0x7f096815aaa0 2023-10-16T09:49:18.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.059+0000 7f096ceb7640 1 -- 172.21.15.84:0/1873538502 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09640309a0 con 0x7f096815aaa0 2023-10-16T09:49:18.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.059+0000 7f096ceb7640 1 -- 172.21.15.84:0/1873538502 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0964030d30 con 0x7f096815aaa0 2023-10-16T09:49:18.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.059+0000 7f097011d640 1 -- 172.21.15.84:0/1873538502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f096815aaa0 msgr2=0x7f096815e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.059+0000 7f097011d640 1 --2- 172.21.15.84:0/1873538502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f096815aaa0 0x7f096815e820 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0964009870 tx=0x7f096402b980 comp rx=0 tx=0).stop 2023-10-16T09:49:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.059+0000 7f097011d640 1 -- 172.21.15.84:0/1873538502 shutdown_connections 2023-10-16T09:49:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.059+0000 7f097011d640 1 --2- 172.21.15.84:0/1873538502 >> v2:172.21.15.157:3300/0 conn(0x7f09680ce390 0x7f0968070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.059+0000 7f097011d640 1 --2- 172.21.15.84:0/1873538502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f096815aaa0 0x7f096815e820 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.059+0000 7f097011d640 1 -- 172.21.15.84:0/1873538502 >> 172.21.15.84:0/1873538502 conn(0x7f09680cb410 msgr2=0x7f09680cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.059+0000 7f097011d640 1 -- 172.21.15.84:0/1873538502 shutdown_connections 2023-10-16T09:49:18.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.059+0000 7f097011d640 1 -- 172.21.15.84:0/1873538502 wait complete. 2023-10-16T09:49:18.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.061+0000 7f097011d640 1 Processor -- start 2023-10-16T09:49:18.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.062+0000 7f097011d640 1 -- start start 2023-10-16T09:49:18.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.063+0000 7f097011d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09680ce390 0x7f09680c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:18.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.063+0000 7f097011d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f096815aaa0 0x7f09681f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:18.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.063+0000 7f097011d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096815ae80 0x7f09681f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:18.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.063+0000 7f097011d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09681f4090 con 0x7f096815aaa0 2023-10-16T09:49:18.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.063+0000 7f097011d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09680c3870 con 0x7f096815ae80 2023-10-16T09:49:18.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.063+0000 7f097011d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09680c3b50 con 0x7f09680ce390 2023-10-16T09:49:18.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.063+0000 7f096d6b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f096815aaa0 0x7f09681f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:18.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.063+0000 7f096d6b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f096815aaa0 0x7f09681f2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33932/0 (socket says 172.21.15.84:33932) 2023-10-16T09:49:18.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.063+0000 7f096d6b8640 1 -- 172.21.15.84:0/2393220357 learned_addr learned my addr 172.21.15.84:0/2393220357 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:18.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.063+0000 7f096deb9640 1 --2- 172.21.15.84:0/2393220357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09680ce390 0x7f09680c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:18.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.064+0000 7f096e6ba640 1 --2- 172.21.15.84:0/2393220357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096815ae80 0x7f09681f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:18.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.064+0000 7f096d6b8640 1 -- 172.21.15.84:0/2393220357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09680ce390 msgr2=0x7f09680c2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:18.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.064+0000 7f096d6b8640 1 --2- 172.21.15.84:0/2393220357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09680ce390 0x7f09680c2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.064+0000 7f096d6b8640 1 -- 172.21.15.84:0/2393220357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096815ae80 msgr2=0x7f09681f2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:18.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.064+0000 7f096d6b8640 1 --2- 172.21.15.84:0/2393220357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096815ae80 0x7f09681f2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.064+0000 7f096d6b8640 1 -- 172.21.15.84:0/2393220357 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09640094d0 con 0x7f096815aaa0 2023-10-16T09:49:18.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.064+0000 7f096d6b8640 1 --2- 172.21.15.84:0/2393220357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f096815aaa0 0x7f09681f2720 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f095800c400 tx=0x7f095800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:18.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.064+0000 7f095e7fc640 1 -- 172.21.15.84:0/2393220357 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09580052d0 con 0x7f096815aaa0 2023-10-16T09:49:18.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.064+0000 7f095e7fc640 1 -- 172.21.15.84:0/2393220357 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0958005450 con 0x7f096815aaa0 2023-10-16T09:49:18.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.064+0000 7f095e7fc640 1 -- 172.21.15.84:0/2393220357 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0958007260 con 0x7f096815aaa0 2023-10-16T09:49:18.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.064+0000 7f097011d640 1 -- 172.21.15.84:0/2393220357 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09681f3cb0 con 0x7f096815aaa0 2023-10-16T09:49:18.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.064+0000 7f097011d640 1 -- 172.21.15.84:0/2393220357 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09680c33f0 con 0x7f096815aaa0 2023-10-16T09:49:18.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.066+0000 7f097011d640 1 -- 172.21.15.84:0/2393220357 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09681f41d0 con 0x7f096815aaa0 2023-10-16T09:49:18.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.066+0000 7f095e7fc640 1 -- 172.21.15.84:0/2393220357 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f09580073c0 con 0x7f096815aaa0 2023-10-16T09:49:18.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.067+0000 7f095e7fc640 1 --2- 172.21.15.84:0/2393220357 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f094c06ae70 0x7f094c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:18.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.068+0000 7f095e7fc640 1 -- 172.21.15.84:0/2393220357 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0958088980 con 0x7f096815aaa0 2023-10-16T09:49:18.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.068+0000 7f096deb9640 1 --2- 172.21.15.84:0/2393220357 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f094c06ae70 0x7f094c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:18.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.068+0000 7f096deb9640 1 --2- 172.21.15.84:0/2393220357 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f094c06ae70 0x7f094c06d330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0964036000 tx=0x7f0964008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:18.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.072+0000 7f095e7fc640 1 -- 172.21.15.84:0/2393220357 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f09681f41d0 con 0x7f096815aaa0 2023-10-16T09:49:18.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.268+0000 7f097011d640 1 -- 172.21.15.84:0/2393220357 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09681f41d0 con 0x7f096815aaa0 2023-10-16T09:49:18.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.270+0000 7f095e7fc640 1 -- 172.21.15.84:0/2393220357 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f0958080e60 con 0x7f096815aaa0 2023-10-16T09:49:18.272 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:18.272 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":21,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:03.529261+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:18.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.272+0000 7f097011d640 1 -- 172.21.15.84:0/2393220357 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f094c06ae70 msgr2=0x7f094c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:18.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.272+0000 7f097011d640 1 --2- 172.21.15.84:0/2393220357 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f094c06ae70 0x7f094c06d330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0964036000 tx=0x7f0964008030 comp rx=0 tx=0).stop 2023-10-16T09:49:18.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.272+0000 7f097011d640 1 -- 172.21.15.84:0/2393220357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f096815aaa0 msgr2=0x7f09681f2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:18.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.272+0000 7f097011d640 1 --2- 172.21.15.84:0/2393220357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f096815aaa0 0x7f09681f2720 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f095800c400 tx=0x7f095800c8c0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.273+0000 7f097011d640 1 -- 172.21.15.84:0/2393220357 shutdown_connections 2023-10-16T09:49:18.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.273+0000 7f097011d640 1 --2- 172.21.15.84:0/2393220357 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f094c06ae70 0x7f094c06d330 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.273+0000 7f097011d640 1 --2- 172.21.15.84:0/2393220357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096815ae80 0x7f09681f2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.273+0000 7f097011d640 1 --2- 172.21.15.84:0/2393220357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f096815aaa0 0x7f09681f2720 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.273+0000 7f097011d640 1 --2- 172.21.15.84:0/2393220357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09680ce390 0x7f09680c2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.273+0000 7f097011d640 1 -- 172.21.15.84:0/2393220357 >> 172.21.15.84:0/2393220357 conn(0x7f09680cb410 msgr2=0x7f096806f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:18.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.273+0000 7f097011d640 1 -- 172.21.15.84:0/2393220357 shutdown_connections 2023-10-16T09:49:18.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.273+0000 7f097011d640 1 -- 172.21.15.84:0/2393220357 wait complete. 2023-10-16T09:49:18.292 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:49:00.048649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:49:01.361365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:49:18.293 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:49:18.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.505+0000 7f505f802640 1 Processor -- start 2023-10-16T09:49:18.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.505+0000 7f505f802640 1 -- start start 2023-10-16T09:49:18.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.507+0000 7f505f802640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f505815ad70 0x7f5058069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:18.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.507+0000 7f505f802640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f50580caac0 0x7f505806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:18.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.507+0000 7f505f802640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50580cb030 con 0x7f50580c9ca0 2023-10-16T09:49:18.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.507+0000 7f505f802640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50580c91c0 con 0x7f50580caac0 2023-10-16T09:49:18.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.507+0000 7f505f802640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50580c94a0 con 0x7f505815ad70 2023-10-16T09:49:18.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.507+0000 7f505dd9f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f50580caac0 0x7f505806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:18.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.507+0000 7f505dd9f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f50580caac0 0x7f505806a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33946/0 (socket says 172.21.15.84:33946) 2023-10-16T09:49:18.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.507+0000 7f505dd9f640 1 -- 172.21.15.84:0/3428902089 learned_addr learned my addr 172.21.15.84:0/3428902089 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:18.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.507+0000 7f505d59e640 1 --2- 172.21.15.84:0/3428902089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f505815ad70 0x7f5058069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:18.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.507+0000 7f505dd9f640 1 -- 172.21.15.84:0/3428902089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f505815ad70 msgr2=0x7f5058069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:18.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.507+0000 7f505dd9f640 1 --2- 172.21.15.84:0/3428902089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f505815ad70 0x7f5058069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.507+0000 7f505dd9f640 1 -- 172.21.15.84:0/3428902089 >> v1:172.21.15.157:6789/0 conn(0x7f50580c9ca0 legacy=0x7f50580ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:18.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.507+0000 7f505dd9f640 1 -- 172.21.15.84:0/3428902089 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50580c9780 con 0x7f50580caac0 2023-10-16T09:49:18.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.508+0000 7f505d59e640 1 --2- 172.21.15.84:0/3428902089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f505815ad70 0x7f5058069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:18.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.508+0000 7f505dd9f640 1 --2- 172.21.15.84:0/3428902089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50580caac0 0x7f505806a2e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5050009e50 tx=0x7f505002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e6ae5fffea31ed server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:18.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.509+0000 7f504ffff640 1 -- 172.21.15.84:0/3428902089 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505002ca80 con 0x7f50580caac0 2023-10-16T09:49:18.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.509+0000 7f504ffff640 1 -- 172.21.15.84:0/3428902089 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505002cbe0 con 0x7f50580caac0 2023-10-16T09:49:18.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.509+0000 7f504ffff640 1 -- 172.21.15.84:0/3428902089 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50500343f0 con 0x7f50580caac0 2023-10-16T09:49:18.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.509+0000 7f505f802640 1 -- 172.21.15.84:0/3428902089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50580caac0 msgr2=0x7f505806a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:18.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.509+0000 7f505f802640 1 --2- 172.21.15.84:0/3428902089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50580caac0 0x7f505806a2e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5050009e50 tx=0x7f505002bee0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.510+0000 7f505f802640 1 -- 172.21.15.84:0/3428902089 shutdown_connections 2023-10-16T09:49:18.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.510+0000 7f505f802640 1 --2- 172.21.15.84:0/3428902089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50580caac0 0x7f505806a2e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.510+0000 7f505f802640 1 --2- 172.21.15.84:0/3428902089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f505815ad70 0x7f5058069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.510+0000 7f505f802640 1 -- 172.21.15.84:0/3428902089 >> 172.21.15.84:0/3428902089 conn(0x7f50580ce9b0 msgr2=0x7f50580cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:18.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.510+0000 7f505f802640 1 -- 172.21.15.84:0/3428902089 shutdown_connections 2023-10-16T09:49:18.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.510+0000 7f505f802640 1 -- 172.21.15.84:0/3428902089 wait complete. 2023-10-16T09:49:18.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.511+0000 7f505f802640 1 Processor -- start 2023-10-16T09:49:18.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.511+0000 7f505f802640 1 -- start start 2023-10-16T09:49:18.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.512+0000 7f505f802640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50580c9ca0 0x7f50581545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:18.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.512+0000 7f505f802640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50580caac0 0x7f505815cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:18.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.512+0000 7f505f802640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f505815ad70 0x7f505815d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:18.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.512+0000 7f505f802640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50581571b0 con 0x7f50580c9ca0 2023-10-16T09:49:18.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.512+0000 7f505f802640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50580c6fd0 con 0x7f50580caac0 2023-10-16T09:49:18.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.512+0000 7f505f802640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5058156880 con 0x7f505815ad70 2023-10-16T09:49:18.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.513+0000 7f505d59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50580c9ca0 0x7f50581545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:18.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.513+0000 7f505d59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50580c9ca0 0x7f50581545c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33962/0 (socket says 172.21.15.84:33962) 2023-10-16T09:49:18.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.513+0000 7f505d59e640 1 -- 172.21.15.84:0/1480099257 learned_addr learned my addr 172.21.15.84:0/1480099257 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:18.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.513+0000 7f505dd9f640 1 --2- 172.21.15.84:0/1480099257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f505815ad70 0x7f505815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:18.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.513+0000 7f505cd9d640 1 --2- 172.21.15.84:0/1480099257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50580caac0 0x7f505815cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:18.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.513+0000 7f505d59e640 1 -- 172.21.15.84:0/1480099257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f505815ad70 msgr2=0x7f505815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:18.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.513+0000 7f505d59e640 1 --2- 172.21.15.84:0/1480099257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f505815ad70 0x7f505815d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.513+0000 7f505d59e640 1 -- 172.21.15.84:0/1480099257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50580caac0 msgr2=0x7f505815cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:18.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.513+0000 7f505d59e640 1 --2- 172.21.15.84:0/1480099257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50580caac0 0x7f505815cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.513+0000 7f505d59e640 1 -- 172.21.15.84:0/1480099257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5050009b30 con 0x7f50580c9ca0 2023-10-16T09:49:18.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.513+0000 7f505d59e640 1 --2- 172.21.15.84:0/1480099257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50580c9ca0 0x7f50581545c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f505400c7f0 tx=0x7f505400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:18.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.514+0000 7f504dffb640 1 -- 172.21.15.84:0/1480099257 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054007820 con 0x7f50580c9ca0 2023-10-16T09:49:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.514+0000 7f504dffb640 1 -- 172.21.15.84:0/1480099257 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5054007980 con 0x7f50580c9ca0 2023-10-16T09:49:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.514+0000 7f504dffb640 1 -- 172.21.15.84:0/1480099257 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054007cf0 con 0x7f50580c9ca0 2023-10-16T09:49:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.514+0000 7f505f802640 1 -- 172.21.15.84:0/1480099257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5058156b60 con 0x7f50580c9ca0 2023-10-16T09:49:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.514+0000 7f505f802640 1 -- 172.21.15.84:0/1480099257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5058154b00 con 0x7f50580c9ca0 2023-10-16T09:49:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.515+0000 7f505f802640 1 -- 172.21.15.84:0/1480099257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505804e160 con 0x7f50580c9ca0 2023-10-16T09:49:18.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.516+0000 7f504dffb640 1 -- 172.21.15.84:0/1480099257 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f50540113f0 con 0x7f50580c9ca0 2023-10-16T09:49:18.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.517+0000 7f504dffb640 1 --2- 172.21.15.84:0/1480099257 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f503c06ae70 0x7f503c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:18.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.517+0000 7f504dffb640 1 -- 172.21.15.84:0/1480099257 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50540898b0 con 0x7f50580c9ca0 2023-10-16T09:49:18.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.517+0000 7f505cd9d640 1 --2- 172.21.15.84:0/1480099257 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f503c06ae70 0x7f503c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:18.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.517+0000 7f505cd9d640 1 --2- 172.21.15.84:0/1480099257 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f503c06ae70 0x7f503c06d330 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5058159810 tx=0x7f5048009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:18.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.521+0000 7f504dffb640 1 -- 172.21.15.84:0/1480099257 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f50540549d0 con 0x7f50580c9ca0 2023-10-16T09:49:18.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.724+0000 7f505f802640 1 -- 172.21.15.84:0/1480099257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50580ad430 con 0x7f50580c9ca0 2023-10-16T09:49:18.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.725+0000 7f504dffb640 1 -- 172.21.15.84:0/1480099257 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f505401f0a0 con 0x7f50580c9ca0 2023-10-16T09:49:18.727 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T09:49:18.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.728+0000 7f505f802640 1 -- 172.21.15.84:0/1480099257 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f503c06ae70 msgr2=0x7f503c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:18.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.728+0000 7f505f802640 1 --2- 172.21.15.84:0/1480099257 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f503c06ae70 0x7f503c06d330 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5058159810 tx=0x7f5048009040 comp rx=0 tx=0).stop 2023-10-16T09:49:18.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.728+0000 7f505f802640 1 -- 172.21.15.84:0/1480099257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50580c9ca0 msgr2=0x7f50581545c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:18.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.728+0000 7f505f802640 1 --2- 172.21.15.84:0/1480099257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50580c9ca0 0x7f50581545c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f505400c7f0 tx=0x7f505400ccb0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.728+0000 7f505f802640 1 -- 172.21.15.84:0/1480099257 shutdown_connections 2023-10-16T09:49:18.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.728+0000 7f505f802640 1 --2- 172.21.15.84:0/1480099257 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f503c06ae70 0x7f503c06d330 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.728+0000 7f505f802640 1 --2- 172.21.15.84:0/1480099257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f505815ad70 0x7f505815d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.728+0000 7f505f802640 1 --2- 172.21.15.84:0/1480099257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50580caac0 0x7f505815cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.728+0000 7f505f802640 1 --2- 172.21.15.84:0/1480099257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50580c9ca0 0x7f50581545c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.728+0000 7f505f802640 1 -- 172.21.15.84:0/1480099257 >> 172.21.15.84:0/1480099257 conn(0x7f50580ce9b0 msgr2=0x7f505806b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:18.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.729+0000 7f505f802640 1 -- 172.21.15.84:0/1480099257 shutdown_connections 2023-10-16T09:49:18.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.729+0000 7f505f802640 1 -- 172.21.15.84:0/1480099257 wait complete. 2023-10-16T09:49:18.748 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:49:18.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.952+0000 7fc4f3a56640 1 Processor -- start 2023-10-16T09:49:18.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.953+0000 7fc4f3a56640 1 -- start start 2023-10-16T09:49:18.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.954+0000 7fc4f3a56640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc4ec15ad70 0x7fc4ec069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:18.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.954+0000 7fc4f3a56640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4ec0c9ca0 0x7fc4ec06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:18.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.954+0000 7fc4f3a56640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4ec0cb030 con 0x7fc4ec0ca3b0 2023-10-16T09:49:18.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.954+0000 7fc4f3a56640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4ec0c91c0 con 0x7fc4ec15ad70 2023-10-16T09:49:18.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.954+0000 7fc4f3a56640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4ec0c94a0 con 0x7fc4ec0c9ca0 2023-10-16T09:49:18.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.954+0000 7fc4f17f2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc4ec15ad70 0x7fc4ec069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:18.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.954+0000 7fc4f17f2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc4ec15ad70 0x7fc4ec069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33964/0 (socket says 172.21.15.84:33964) 2023-10-16T09:49:18.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.954+0000 7fc4f17f2640 1 -- 172.21.15.84:0/1001961929 learned_addr learned my addr 172.21.15.84:0/1001961929 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:18.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.954+0000 7fc4f1ff3640 1 --1- 172.21.15.84:0/1001961929 >> v1:172.21.15.84:6789/0 conn(0x7fc4ec0ca3b0 0x7fc4ec0ca840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36820/0 (socket says 172.21.15.84:36820) 2023-10-16T09:49:18.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.955+0000 7fc4f0ff1640 1 --2- 172.21.15.84:0/1001961929 >> v2:172.21.15.157:3300/0 conn(0x7fc4ec0c9ca0 0x7fc4ec06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:18.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.955+0000 7fc4f17f2640 1 -- 172.21.15.84:0/1001961929 >> v2:172.21.15.157:3300/0 conn(0x7fc4ec0c9ca0 msgr2=0x7fc4ec06a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:18.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.955+0000 7fc4f17f2640 1 --2- 172.21.15.84:0/1001961929 >> v2:172.21.15.157:3300/0 conn(0x7fc4ec0c9ca0 0x7fc4ec06a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.955+0000 7fc4f17f2640 1 -- 172.21.15.84:0/1001961929 >> v1:172.21.15.84:6789/0 conn(0x7fc4ec0ca3b0 legacy=0x7fc4ec0ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:18.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.955+0000 7fc4f17f2640 1 -- 172.21.15.84:0/1001961929 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ec0c9780 con 0x7fc4ec15ad70 2023-10-16T09:49:18.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.955+0000 7fc4e3fff640 1 -- 172.21.15.84:0/1001961929 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193654255 0 0) 0x7fc4ec0cb030 con 0x7fc4ec0ca3b0 2023-10-16T09:49:18.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.956+0000 7fc4f17f2640 1 --2- 172.21.15.84:0/1001961929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4ec15ad70 0x7fc4ec069da0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec0ce870 tx=0x7fc4e802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99725469dc410d95 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:18.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.956+0000 7fc4e3fff640 1 -- 172.21.15.84:0/1001961929 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8030860 con 0x7fc4ec15ad70 2023-10-16T09:49:18.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.956+0000 7fc4e3fff640 1 -- 172.21.15.84:0/1001961929 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e80309c0 con 0x7fc4ec15ad70 2023-10-16T09:49:18.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.956+0000 7fc4e3fff640 1 -- 172.21.15.84:0/1001961929 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8030d50 con 0x7fc4ec15ad70 2023-10-16T09:49:18.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.957+0000 7fc4f3a56640 1 -- 172.21.15.84:0/1001961929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4ec15ad70 msgr2=0x7fc4ec069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:18.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.957+0000 7fc4f3a56640 1 --2- 172.21.15.84:0/1001961929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4ec15ad70 0x7fc4ec069da0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec0ce870 tx=0x7fc4e802b9a0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.957+0000 7fc4f3a56640 1 -- 172.21.15.84:0/1001961929 shutdown_connections 2023-10-16T09:49:18.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.957+0000 7fc4f3a56640 1 --2- 172.21.15.84:0/1001961929 >> v2:172.21.15.157:3300/0 conn(0x7fc4ec0c9ca0 0x7fc4ec06a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.957+0000 7fc4f3a56640 1 --2- 172.21.15.84:0/1001961929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4ec15ad70 0x7fc4ec069da0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.957+0000 7fc4f3a56640 1 -- 172.21.15.84:0/1001961929 >> 172.21.15.84:0/1001961929 conn(0x7fc4ec0ce9b0 msgr2=0x7fc4ec0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:18.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.957+0000 7fc4f3a56640 1 -- 172.21.15.84:0/1001961929 shutdown_connections 2023-10-16T09:49:18.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.957+0000 7fc4f3a56640 1 -- 172.21.15.84:0/1001961929 wait complete. 2023-10-16T09:49:18.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.959+0000 7fc4f3a56640 1 Processor -- start 2023-10-16T09:49:18.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.959+0000 7fc4f3a56640 1 -- start start 2023-10-16T09:49:18.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.960+0000 7fc4f3a56640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4ec0c9ca0 0x7fc4ec1f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:18.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.960+0000 7fc4f3a56640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4ec0ca3b0 0x7fc4ec157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:18.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.960+0000 7fc4f3a56640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4ec15ad70 0x7fc4ec15b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:18.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.960+0000 7fc4f3a56640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec0c51d0 con 0x7fc4ec15ad70 2023-10-16T09:49:18.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.960+0000 7fc4f3a56640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec158520 con 0x7fc4ec0c9ca0 2023-10-16T09:49:18.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.960+0000 7fc4f3a56640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec1f3e40 con 0x7fc4ec0ca3b0 2023-10-16T09:49:18.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.960+0000 7fc4f1ff3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4ec15ad70 0x7fc4ec15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:18.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.960+0000 7fc4f1ff3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4ec15ad70 0x7fc4ec15b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33968/0 (socket says 172.21.15.84:33968) 2023-10-16T09:49:18.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.960+0000 7fc4f1ff3640 1 -- 172.21.15.84:0/2103511680 learned_addr learned my addr 172.21.15.84:0/2103511680 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:18.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.961+0000 7fc4f17f2640 1 --2- 172.21.15.84:0/2103511680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4ec0c9ca0 0x7fc4ec1f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:18.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.961+0000 7fc4f0ff1640 1 --2- 172.21.15.84:0/2103511680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4ec0ca3b0 0x7fc4ec157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:18.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.961+0000 7fc4f1ff3640 1 -- 172.21.15.84:0/2103511680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4ec0ca3b0 msgr2=0x7fc4ec157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:18.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.961+0000 7fc4f1ff3640 1 --2- 172.21.15.84:0/2103511680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4ec0ca3b0 0x7fc4ec157fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.961+0000 7fc4f1ff3640 1 -- 172.21.15.84:0/2103511680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4ec0c9ca0 msgr2=0x7fc4ec1f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:18.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.961+0000 7fc4f1ff3640 1 --2- 172.21.15.84:0/2103511680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4ec0c9ca0 0x7fc4ec1f3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:18.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.961+0000 7fc4f1ff3640 1 -- 172.21.15.84:0/2103511680 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e80094d0 con 0x7fc4ec15ad70 2023-10-16T09:49:18.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.961+0000 7fc4f17f2640 1 --2- 172.21.15.84:0/2103511680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4ec0c9ca0 0x7fc4ec1f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:18.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.961+0000 7fc4f1ff3640 1 --2- 172.21.15.84:0/2103511680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4ec15ad70 0x7fc4ec15b450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400c7e0 tx=0x7fc4e400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:18.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.961+0000 7fc4e1ffb640 1 -- 172.21.15.84:0/2103511680 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e400a640 con 0x7fc4ec15ad70 2023-10-16T09:49:18.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.961+0000 7fc4e1ffb640 1 -- 172.21.15.84:0/2103511680 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e400a7c0 con 0x7fc4ec15ad70 2023-10-16T09:49:18.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.961+0000 7fc4e1ffb640 1 -- 172.21.15.84:0/2103511680 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e400ab70 con 0x7fc4ec15ad70 2023-10-16T09:49:18.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.961+0000 7fc4f3a56640 1 -- 172.21.15.84:0/2103511680 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ec1f4120 con 0x7fc4ec15ad70 2023-10-16T09:49:18.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.961+0000 7fc4f3a56640 1 -- 172.21.15.84:0/2103511680 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ec0c5c20 con 0x7fc4ec15ad70 2023-10-16T09:49:18.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.962+0000 7fc4f3a56640 1 -- 172.21.15.84:0/2103511680 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4ec1f4640 con 0x7fc4ec15ad70 2023-10-16T09:49:18.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.963+0000 7fc4e1ffb640 1 -- 172.21.15.84:0/2103511680 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc4e400acd0 con 0x7fc4ec15ad70 2023-10-16T09:49:18.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.964+0000 7fc4e1ffb640 1 --2- 172.21.15.84:0/2103511680 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc4d006ae70 0x7fc4d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:18.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.964+0000 7fc4e1ffb640 1 -- 172.21.15.84:0/2103511680 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4e4088cd0 con 0x7fc4ec15ad70 2023-10-16T09:49:18.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.964+0000 7fc4f17f2640 1 --2- 172.21.15.84:0/2103511680 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc4d006ae70 0x7fc4d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:18.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.965+0000 7fc4f17f2640 1 --2- 172.21.15.84:0/2103511680 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc4d006ae70 0x7fc4d006d330 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8036000 tx=0x7fc4e803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:18.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:18.968+0000 7fc4e1ffb640 1 -- 172.21.15.84:0/2103511680 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc4ec1f4640 con 0x7fc4ec15ad70 2023-10-16T09:49:19.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.160+0000 7fc4f3a56640 1 -- 172.21.15.84:0/2103511680 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc4ec0c5a80 con 0x7fc4ec15ad70 2023-10-16T09:49:19.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.160+0000 7fc4e1ffb640 1 -- 172.21.15.84:0/2103511680 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc4ec0c5a80 con 0x7fc4ec15ad70 2023-10-16T09:49:19.162 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T09:49:19.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.163+0000 7fc4f3a56640 1 -- 172.21.15.84:0/2103511680 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc4d006ae70 msgr2=0x7fc4d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:19.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.163+0000 7fc4f3a56640 1 --2- 172.21.15.84:0/2103511680 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc4d006ae70 0x7fc4d006d330 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8036000 tx=0x7fc4e803b040 comp rx=0 tx=0).stop 2023-10-16T09:49:19.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.163+0000 7fc4f3a56640 1 -- 172.21.15.84:0/2103511680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4ec15ad70 msgr2=0x7fc4ec15b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:19.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.163+0000 7fc4f3a56640 1 --2- 172.21.15.84:0/2103511680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4ec15ad70 0x7fc4ec15b450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400c7e0 tx=0x7fc4e400cca0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.163+0000 7fc4f3a56640 1 -- 172.21.15.84:0/2103511680 shutdown_connections 2023-10-16T09:49:19.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.163+0000 7fc4f3a56640 1 --2- 172.21.15.84:0/2103511680 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc4d006ae70 0x7fc4d006d330 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.163+0000 7fc4f3a56640 1 --2- 172.21.15.84:0/2103511680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4ec15ad70 0x7fc4ec15b450 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.163+0000 7fc4f3a56640 1 --2- 172.21.15.84:0/2103511680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4ec0ca3b0 0x7fc4ec157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.163+0000 7fc4f3a56640 1 --2- 172.21.15.84:0/2103511680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4ec0c9ca0 0x7fc4ec1f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.163+0000 7fc4f3a56640 1 -- 172.21.15.84:0/2103511680 >> 172.21.15.84:0/2103511680 conn(0x7fc4ec0ce9b0 msgr2=0x7fc4ec0b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:19.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.164+0000 7fc4f3a56640 1 -- 172.21.15.84:0/2103511680 shutdown_connections 2023-10-16T09:49:19.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.164+0000 7fc4f3a56640 1 -- 172.21.15.84:0/2103511680 wait complete. 2023-10-16T09:49:19.184 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:49:19.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.398+0000 7efe4983f640 1 Processor -- start 2023-10-16T09:49:19.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.398+0000 7efe4983f640 1 -- start start 2023-10-16T09:49:19.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.400+0000 7efe4983f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efe4415ad70 0x7efe44069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:19.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.400+0000 7efe4983f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe440c9ca0 0x7efe4406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:19.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.400+0000 7efe4983f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe440cb030 con 0x7efe440ca3b0 2023-10-16T09:49:19.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.400+0000 7efe4983f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe440c91c0 con 0x7efe4415ad70 2023-10-16T09:49:19.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.400+0000 7efe4983f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe440c94a0 con 0x7efe440c9ca0 2023-10-16T09:49:19.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.400+0000 7efe427fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe440c9ca0 0x7efe4406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:19.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.400+0000 7efe427fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe440c9ca0 0x7efe4406a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:36378/0 (socket says 172.21.15.84:36378) 2023-10-16T09:49:19.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.400+0000 7efe427fc640 1 -- 172.21.15.84:0/817957678 learned_addr learned my addr 172.21.15.84:0/817957678 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:19.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.400+0000 7efe437fe640 1 --1- 172.21.15.84:0/817957678 >> v1:172.21.15.157:6789/0 conn(0x7efe440ca3b0 0x7efe440ca840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:45124/0 (socket says 172.21.15.84:45124) 2023-10-16T09:49:19.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.400+0000 7efe42ffd640 1 --2- 172.21.15.84:0/817957678 >> v2:172.21.15.157:3300/0 conn(0x7efe4415ad70 0x7efe44069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:19.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.401+0000 7efe41ffb640 1 -- 172.21.15.84:0/817957678 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2137142141 0 0) 0x7efe440cb030 con 0x7efe440ca3b0 2023-10-16T09:49:19.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.401+0000 7efe427fc640 1 -- 172.21.15.84:0/817957678 >> v2:172.21.15.157:3300/0 conn(0x7efe4415ad70 msgr2=0x7efe44069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:19.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.401+0000 7efe427fc640 1 --2- 172.21.15.84:0/817957678 >> v2:172.21.15.157:3300/0 conn(0x7efe4415ad70 0x7efe44069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.401+0000 7efe427fc640 1 -- 172.21.15.84:0/817957678 >> v1:172.21.15.157:6789/0 conn(0x7efe440ca3b0 legacy=0x7efe440ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:19.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.401+0000 7efe427fc640 1 -- 172.21.15.84:0/817957678 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe440c9780 con 0x7efe440c9ca0 2023-10-16T09:49:19.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.402+0000 7efe427fc640 1 --2- 172.21.15.84:0/817957678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe440c9ca0 0x7efe4406a2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efe38009870 tx=0x7efe3802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c76232ece27e97e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:19.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.403+0000 7efe41ffb640 1 -- 172.21.15.84:0/817957678 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38030840 con 0x7efe440c9ca0 2023-10-16T09:49:19.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.403+0000 7efe41ffb640 1 -- 172.21.15.84:0/817957678 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe380309a0 con 0x7efe440c9ca0 2023-10-16T09:49:19.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.403+0000 7efe41ffb640 1 -- 172.21.15.84:0/817957678 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38030d30 con 0x7efe440c9ca0 2023-10-16T09:49:19.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.404+0000 7efe4983f640 1 -- 172.21.15.84:0/817957678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe440c9ca0 msgr2=0x7efe4406a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:19.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.404+0000 7efe4983f640 1 --2- 172.21.15.84:0/817957678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe440c9ca0 0x7efe4406a2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efe38009870 tx=0x7efe3802b980 comp rx=0 tx=0).stop 2023-10-16T09:49:19.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.404+0000 7efe4983f640 1 -- 172.21.15.84:0/817957678 shutdown_connections 2023-10-16T09:49:19.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.404+0000 7efe4983f640 1 --2- 172.21.15.84:0/817957678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe440c9ca0 0x7efe4406a2e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.404+0000 7efe4983f640 1 --2- 172.21.15.84:0/817957678 >> v2:172.21.15.157:3300/0 conn(0x7efe4415ad70 0x7efe44069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.404+0000 7efe4983f640 1 -- 172.21.15.84:0/817957678 >> 172.21.15.84:0/817957678 conn(0x7efe440ce9b0 msgr2=0x7efe440cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:19.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.404+0000 7efe4983f640 1 -- 172.21.15.84:0/817957678 shutdown_connections 2023-10-16T09:49:19.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.404+0000 7efe4983f640 1 -- 172.21.15.84:0/817957678 wait complete. 2023-10-16T09:49:19.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.406+0000 7efe4983f640 1 Processor -- start 2023-10-16T09:49:19.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.406+0000 7efe4983f640 1 -- start start 2023-10-16T09:49:19.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.407+0000 7efe4983f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe440c9ca0 0x7efe44156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:19.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.407+0000 7efe4983f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe440ca3b0 0x7efe440c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:19.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.407+0000 7efe4983f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe4415ad70 0x7efe44070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:19.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.407+0000 7efe4983f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4415c810 con 0x7efe4415ad70 2023-10-16T09:49:19.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.407+0000 7efe4983f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe440c6630 con 0x7efe440c9ca0 2023-10-16T09:49:19.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.407+0000 7efe4983f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe440c7490 con 0x7efe440ca3b0 2023-10-16T09:49:19.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.407+0000 7efe437fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe4415ad70 0x7efe44070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:19.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.407+0000 7efe437fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe4415ad70 0x7efe44070d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33978/0 (socket says 172.21.15.84:33978) 2023-10-16T09:49:19.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.407+0000 7efe437fe640 1 -- 172.21.15.84:0/1641314198 learned_addr learned my addr 172.21.15.84:0/1641314198 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:19.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.407+0000 7efe42ffd640 1 --2- 172.21.15.84:0/1641314198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe440c9ca0 0x7efe44156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:19.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.407+0000 7efe427fc640 1 --2- 172.21.15.84:0/1641314198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe440ca3b0 0x7efe440c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:19.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.407+0000 7efe437fe640 1 -- 172.21.15.84:0/1641314198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe440ca3b0 msgr2=0x7efe440c5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:19.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.407+0000 7efe437fe640 1 --2- 172.21.15.84:0/1641314198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe440ca3b0 0x7efe440c5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.407+0000 7efe437fe640 1 -- 172.21.15.84:0/1641314198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe440c9ca0 msgr2=0x7efe44156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:19.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.407+0000 7efe437fe640 1 --2- 172.21.15.84:0/1641314198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe440c9ca0 0x7efe44156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.407+0000 7efe437fe640 1 -- 172.21.15.84:0/1641314198 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe380094d0 con 0x7efe4415ad70 2023-10-16T09:49:19.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.407+0000 7efe427fc640 1 --2- 172.21.15.84:0/1641314198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe440ca3b0 0x7efe440c5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:19.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.407+0000 7efe42ffd640 1 --2- 172.21.15.84:0/1641314198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe440c9ca0 0x7efe44156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:19.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.408+0000 7efe437fe640 1 --2- 172.21.15.84:0/1641314198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe4415ad70 0x7efe44070d60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00c7e0 tx=0x7efe3c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:19.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.408+0000 7efe2bfff640 1 -- 172.21.15.84:0/1641314198 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c00a640 con 0x7efe4415ad70 2023-10-16T09:49:19.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.408+0000 7efe2bfff640 1 -- 172.21.15.84:0/1641314198 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c00a7c0 con 0x7efe4415ad70 2023-10-16T09:49:19.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.408+0000 7efe2bfff640 1 -- 172.21.15.84:0/1641314198 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c00ab70 con 0x7efe4415ad70 2023-10-16T09:49:19.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.408+0000 7efe4983f640 1 -- 172.21.15.84:0/1641314198 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe441574a0 con 0x7efe4415ad70 2023-10-16T09:49:19.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.408+0000 7efe4983f640 1 -- 172.21.15.84:0/1641314198 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe440c6e40 con 0x7efe4415ad70 2023-10-16T09:49:19.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.409+0000 7efe4983f640 1 -- 172.21.15.84:0/1641314198 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe4404e160 con 0x7efe4415ad70 2023-10-16T09:49:19.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.410+0000 7efe2bfff640 1 -- 172.21.15.84:0/1641314198 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efe3c00acd0 con 0x7efe4415ad70 2023-10-16T09:49:19.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.411+0000 7efe2bfff640 1 --2- 172.21.15.84:0/1641314198 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7efe3006ae70 0x7efe3006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:19.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.411+0000 7efe2bfff640 1 -- 172.21.15.84:0/1641314198 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe3c088cd0 con 0x7efe4415ad70 2023-10-16T09:49:19.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.411+0000 7efe42ffd640 1 --2- 172.21.15.84:0/1641314198 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7efe3006ae70 0x7efe3006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:19.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.412+0000 7efe42ffd640 1 --2- 172.21.15.84:0/1641314198 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7efe3006ae70 0x7efe3006d330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efe34009af0 tx=0x7efe34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:19.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.416+0000 7efe2bfff640 1 -- 172.21.15.84:0/1641314198 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efe3c053ce0 con 0x7efe4415ad70 2023-10-16T09:49:19.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.608+0000 7efe4983f640 1 -- 172.21.15.84:0/1641314198 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe44158820 con 0x7efe4415ad70 2023-10-16T09:49:19.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.609+0000 7efe2bfff640 1 -- 172.21.15.84:0/1641314198 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7efe3c018070 con 0x7efe4415ad70 2023-10-16T09:49:19.611 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T09:49:19.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.611+0000 7efe4983f640 1 -- 172.21.15.84:0/1641314198 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7efe3006ae70 msgr2=0x7efe3006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:19.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.611+0000 7efe4983f640 1 --2- 172.21.15.84:0/1641314198 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7efe3006ae70 0x7efe3006d330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efe34009af0 tx=0x7efe34008040 comp rx=0 tx=0).stop 2023-10-16T09:49:19.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.612+0000 7efe4983f640 1 -- 172.21.15.84:0/1641314198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe4415ad70 msgr2=0x7efe44070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:19.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.612+0000 7efe4983f640 1 --2- 172.21.15.84:0/1641314198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe4415ad70 0x7efe44070d60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00c7e0 tx=0x7efe3c00cca0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.612+0000 7efe4983f640 1 -- 172.21.15.84:0/1641314198 shutdown_connections 2023-10-16T09:49:19.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.612+0000 7efe4983f640 1 --2- 172.21.15.84:0/1641314198 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7efe3006ae70 0x7efe3006d330 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.612+0000 7efe4983f640 1 --2- 172.21.15.84:0/1641314198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe4415ad70 0x7efe44070d60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.612+0000 7efe4983f640 1 --2- 172.21.15.84:0/1641314198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe440ca3b0 0x7efe440c5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.612+0000 7efe4983f640 1 --2- 172.21.15.84:0/1641314198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe440c9ca0 0x7efe44156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.612+0000 7efe4983f640 1 -- 172.21.15.84:0/1641314198 >> 172.21.15.84:0/1641314198 conn(0x7efe440ce9b0 msgr2=0x7efe440afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:19.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.612+0000 7efe4983f640 1 -- 172.21.15.84:0/1641314198 shutdown_connections 2023-10-16T09:49:19.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.612+0000 7efe4983f640 1 -- 172.21.15.84:0/1641314198 wait complete. 2023-10-16T09:49:19.631 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:49:19.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.841+0000 7ff013875640 1 Processor -- start 2023-10-16T09:49:19.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.841+0000 7ff013875640 1 -- start start 2023-10-16T09:49:19.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.842+0000 7ff013875640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff00c0ce390 0x7ff00c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:19.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.842+0000 7ff013875640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff00c15ed60 con 0x7ff00c15aaa0 2023-10-16T09:49:19.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.843+0000 7ff013875640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff00c0707e0 con 0x7ff00c15ae80 2023-10-16T09:49:19.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.843+0000 7ff013875640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff00c06b810 con 0x7ff00c0ce390 2023-10-16T09:49:19.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.843+0000 7ff011611640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff00c15aaa0 0x7ff00c0cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36828/0 (socket says 172.21.15.84:36828) 2023-10-16T09:49:19.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.843+0000 7ff011611640 1 -- 172.21.15.84:0/1604070051 learned_addr learned my addr 172.21.15.84:0/1604070051 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:19.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.843+0000 7ff011e12640 1 --2- 172.21.15.84:0/1604070051 >> v2:172.21.15.84:3300/0 conn(0x7ff00c0ce390 0x7ff00c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:19.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.843+0000 7ff003fff640 1 -- 172.21.15.84:0/1604070051 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198240910 0 0) 0x7ff00c15ed60 con 0x7ff00c15aaa0 2023-10-16T09:49:19.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.843+0000 7ff003fff640 1 -- 172.21.15.84:0/1604070051 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feff8003520 con 0x7ff00c15aaa0 2023-10-16T09:49:19.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.843+0000 7ff011e12640 1 -- 172.21.15.84:0/1604070051 >> v1:172.21.15.157:6789/0 conn(0x7ff00c15ae80 legacy=0x7ff00c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:19.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.843+0000 7ff011e12640 1 -- 172.21.15.84:0/1604070051 >> v1:172.21.15.84:6789/0 conn(0x7ff00c15aaa0 legacy=0x7ff00c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:19.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.843+0000 7ff011e12640 1 -- 172.21.15.84:0/1604070051 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00c15ef90 con 0x7ff00c0ce390 2023-10-16T09:49:19.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.844+0000 7ff011e12640 1 --2- 172.21.15.84:0/1604070051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff00c0ce390 0x7ff00c15e820 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff004009d90 tx=0x7ff00402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe372241b2a1b9a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:19.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.845+0000 7ff003fff640 1 -- 172.21.15.84:0/1604070051 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00402cc60 con 0x7ff00c0ce390 2023-10-16T09:49:19.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.845+0000 7ff003fff640 1 -- 172.21.15.84:0/1604070051 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00402cdc0 con 0x7ff00c0ce390 2023-10-16T09:49:19.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.845+0000 7ff003fff640 1 -- 172.21.15.84:0/1604070051 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004034660 con 0x7ff00c0ce390 2023-10-16T09:49:19.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.845+0000 7ff013875640 1 -- 172.21.15.84:0/1604070051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff00c0ce390 msgr2=0x7ff00c15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:19.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.845+0000 7ff013875640 1 --2- 172.21.15.84:0/1604070051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff00c0ce390 0x7ff00c15e820 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff004009d90 tx=0x7ff00402beb0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.845+0000 7ff013875640 1 -- 172.21.15.84:0/1604070051 shutdown_connections 2023-10-16T09:49:19.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.845+0000 7ff013875640 1 --2- 172.21.15.84:0/1604070051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff00c0ce390 0x7ff00c15e820 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.845+0000 7ff013875640 1 -- 172.21.15.84:0/1604070051 >> 172.21.15.84:0/1604070051 conn(0x7ff00c0cb410 msgr2=0x7ff00c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:19.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.845+0000 7ff013875640 1 -- 172.21.15.84:0/1604070051 shutdown_connections 2023-10-16T09:49:19.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.845+0000 7ff013875640 1 -- 172.21.15.84:0/1604070051 wait complete. 2023-10-16T09:49:19.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.847+0000 7ff013875640 1 Processor -- start 2023-10-16T09:49:19.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.847+0000 7ff013875640 1 -- start start 2023-10-16T09:49:19.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.848+0000 7ff013875640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff00c0ce390 0x7ff00c0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:19.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.848+0000 7ff013875640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff00c15aaa0 0x7ff00c15d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:19.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.848+0000 7ff013875640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff00c15ae80 0x7ff00c154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:19.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.848+0000 7ff013875640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00c1590f0 con 0x7ff00c15ae80 2023-10-16T09:49:19.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.848+0000 7ff013875640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00c159340 con 0x7ff00c15aaa0 2023-10-16T09:49:19.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.848+0000 7ff013875640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff00c159620 con 0x7ff00c0ce390 2023-10-16T09:49:19.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.848+0000 7ff011e12640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff00c15ae80 0x7ff00c154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:19.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.848+0000 7ff011e12640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff00c15ae80 0x7ff00c154fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34000/0 (socket says 172.21.15.84:34000) 2023-10-16T09:49:19.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.848+0000 7ff011e12640 1 -- 172.21.15.84:0/987842545 learned_addr learned my addr 172.21.15.84:0/987842545 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:19.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.849+0000 7ff010e10640 1 --2- 172.21.15.84:0/987842545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff00c15aaa0 0x7ff00c15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:19.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.849+0000 7ff011611640 1 --2- 172.21.15.84:0/987842545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff00c0ce390 0x7ff00c0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:19.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.849+0000 7ff011e12640 1 -- 172.21.15.84:0/987842545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff00c0ce390 msgr2=0x7ff00c0c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:19.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.849+0000 7ff011e12640 1 --2- 172.21.15.84:0/987842545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff00c0ce390 0x7ff00c0c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.849+0000 7ff011e12640 1 -- 172.21.15.84:0/987842545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff00c15aaa0 msgr2=0x7ff00c15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:19.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.849+0000 7ff011e12640 1 --2- 172.21.15.84:0/987842545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff00c15aaa0 0x7ff00c15d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:19.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.849+0000 7ff011e12640 1 -- 172.21.15.84:0/987842545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff004009a70 con 0x7ff00c15ae80 2023-10-16T09:49:19.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.849+0000 7ff011e12640 1 --2- 172.21.15.84:0/987842545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff00c15ae80 0x7ff00c154fb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff004009d90 tx=0x7ff004004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:19.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.849+0000 7ff001ffb640 1 -- 172.21.15.84:0/987842545 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004040650 con 0x7ff00c15ae80 2023-10-16T09:49:19.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.849+0000 7ff001ffb640 1 -- 172.21.15.84:0/987842545 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0040407b0 con 0x7ff00c15ae80 2023-10-16T09:49:19.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.849+0000 7ff001ffb640 1 -- 172.21.15.84:0/987842545 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004040b60 con 0x7ff00c15ae80 2023-10-16T09:49:19.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.849+0000 7ff013875640 1 -- 172.21.15.84:0/987842545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff00c159900 con 0x7ff00c15ae80 2023-10-16T09:49:19.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.850+0000 7ff013875640 1 -- 172.21.15.84:0/987842545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff00c159ec0 con 0x7ff00c15ae80 2023-10-16T09:49:19.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.851+0000 7ff013875640 1 -- 172.21.15.84:0/987842545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff00c0b1c50 con 0x7ff00c15ae80 2023-10-16T09:49:19.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.851+0000 7ff001ffb640 1 -- 172.21.15.84:0/987842545 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff004040cc0 con 0x7ff00c15ae80 2023-10-16T09:49:19.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.852+0000 7ff001ffb640 1 --2- 172.21.15.84:0/987842545 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7feff006ae70 0x7feff006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:19.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.852+0000 7ff001ffb640 1 -- 172.21.15.84:0/987842545 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff0040ac490 con 0x7ff00c15ae80 2023-10-16T09:49:19.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.853+0000 7ff011611640 1 --2- 172.21.15.84:0/987842545 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7feff006ae70 0x7feff006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:19.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.853+0000 7ff011611640 1 --2- 172.21.15.84:0/987842545 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7feff006ae70 0x7feff006d330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff0080011e0 tx=0x7ff008009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:19.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:19.857+0000 7ff001ffb640 1 -- 172.21.15.84:0/987842545 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff004077560 con 0x7ff00c15ae80 2023-10-16T09:49:20.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.084+0000 7ff013875640 1 -- 172.21.15.84:0/987842545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff00c04e160 con 0x7ff00c15ae80 2023-10-16T09:49:20.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.085+0000 7ff001ffb640 1 -- 172.21.15.84:0/987842545 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7ff004009450 con 0x7ff00c15ae80 2023-10-16T09:49:20.087 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:20.088 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2978888131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2978888131},{"type":"v1","addr":"172.21.15.157:6834","nonce":2978888131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3716581310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3716581310},{"type":"v1","addr":"172.21.15.84:6835","nonce":3716581310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2761479872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2761479872},{"type":"v1","addr":"172.21.15.84:6837","nonce":2761479872}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:49:02.350624+0000","modified":"2023-10-16T09:49:05.402785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/848754825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":848754825},{"type":"v1","addr":"172.21.15.157:6835","nonce":848754825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T09:49:20.088 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-10-16T09:49:20.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.088+0000 7ff013875640 1 -- 172.21.15.84:0/987842545 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7feff006ae70 msgr2=0x7feff006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:20.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.088+0000 7ff013875640 1 --2- 172.21.15.84:0/987842545 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7feff006ae70 0x7feff006d330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff0080011e0 tx=0x7ff008009040 comp rx=0 tx=0).stop 2023-10-16T09:49:20.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.088+0000 7ff013875640 1 -- 172.21.15.84:0/987842545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff00c15ae80 msgr2=0x7ff00c154fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:20.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.088+0000 7ff013875640 1 --2- 172.21.15.84:0/987842545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff00c15ae80 0x7ff00c154fb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff004009d90 tx=0x7ff004004a40 comp rx=0 tx=0).stop 2023-10-16T09:49:20.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.088+0000 7ff013875640 1 -- 172.21.15.84:0/987842545 shutdown_connections 2023-10-16T09:49:20.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.088+0000 7ff013875640 1 --2- 172.21.15.84:0/987842545 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7feff006ae70 0x7feff006d330 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.088+0000 7ff013875640 1 --2- 172.21.15.84:0/987842545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff00c15ae80 0x7ff00c154fb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.088+0000 7ff013875640 1 --2- 172.21.15.84:0/987842545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff00c15aaa0 0x7ff00c15d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.088+0000 7ff013875640 1 --2- 172.21.15.84:0/987842545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff00c0ce390 0x7ff00c0c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.088+0000 7ff013875640 1 -- 172.21.15.84:0/987842545 >> 172.21.15.84:0/987842545 conn(0x7ff00c0cb410 msgr2=0x7ff00c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:20.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.088+0000 7ff013875640 1 -- 172.21.15.84:0/987842545 shutdown_connections 2023-10-16T09:49:20.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.088+0000 7ff013875640 1 -- 172.21.15.84:0/987842545 wait complete. 2023-10-16T09:49:20.108 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:20.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.318+0000 7f80c8c34640 1 Processor -- start 2023-10-16T09:49:20.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.318+0000 7f80c8c34640 1 -- start start 2023-10-16T09:49:20.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.320+0000 7f80c8c34640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c40cd0c0 0x7f80c41615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:20.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.320+0000 7f80c8c34640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f80c40b0410 0x7f80c40b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:20.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.320+0000 7f80c8c34640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f80c415d3f0 0x7f80c415d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:20.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.320+0000 7f80c8c34640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c40b0db0 con 0x7f80c40b0410 2023-10-16T09:49:20.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.320+0000 7f80c8c34640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c415e3e0 con 0x7f80c415d3f0 2023-10-16T09:49:20.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.320+0000 7f80c8c34640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c40cd4a0 con 0x7f80c40cd0c0 2023-10-16T09:49:20.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.320+0000 7f80bbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f80c40b0410 0x7f80c40b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:20.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.320+0000 7f80bbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f80c40b0410 0x7f80c40b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34014/0 (socket says 172.21.15.84:34014) 2023-10-16T09:49:20.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.320+0000 7f80bbfff640 1 -- 172.21.15.84:0/3691101401 learned_addr learned my addr 172.21.15.84:0/3691101401 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.320+0000 7f80c2d9d640 1 --2- 172.21.15.84:0/3691101401 >> v2:172.21.15.157:3300/0 conn(0x7f80c415d3f0 0x7f80c415d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.320+0000 7f80c259c640 1 --2- 172.21.15.84:0/3691101401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c40cd0c0 0x7f80c41615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.321+0000 7f80bbfff640 1 -- 172.21.15.84:0/3691101401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c40cd0c0 msgr2=0x7f80c41615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:20.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.321+0000 7f80bbfff640 1 --2- 172.21.15.84:0/3691101401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c40cd0c0 0x7f80c41615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.321+0000 7f80bbfff640 1 -- 172.21.15.84:0/3691101401 >> v2:172.21.15.157:3300/0 conn(0x7f80c415d3f0 msgr2=0x7f80c415d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:20.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.321+0000 7f80bbfff640 1 --2- 172.21.15.84:0/3691101401 >> v2:172.21.15.157:3300/0 conn(0x7f80c415d3f0 0x7f80c415d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.321+0000 7f80bbfff640 1 -- 172.21.15.84:0/3691101401 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c415e100 con 0x7f80c40b0410 2023-10-16T09:49:20.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.322+0000 7f80bbfff640 1 --2- 172.21.15.84:0/3691101401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80c40b0410 0x7f80c40b0870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f80c406da30 tx=0x7f80ac02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fafacbece2f776e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:20.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.322+0000 7f80c1d9b640 1 -- 172.21.15.84:0/3691101401 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac0317e0 con 0x7f80c40b0410 2023-10-16T09:49:20.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.322+0000 7f80c1d9b640 1 -- 172.21.15.84:0/3691101401 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ac031940 con 0x7f80c40b0410 2023-10-16T09:49:20.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.322+0000 7f80c1d9b640 1 -- 172.21.15.84:0/3691101401 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac031cd0 con 0x7f80c40b0410 2023-10-16T09:49:20.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.323+0000 7f80c8c34640 1 -- 172.21.15.84:0/3691101401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80c40b0410 msgr2=0x7f80c40b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:20.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.323+0000 7f80c8c34640 1 --2- 172.21.15.84:0/3691101401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80c40b0410 0x7f80c40b0870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f80c406da30 tx=0x7f80ac02b880 comp rx=0 tx=0).stop 2023-10-16T09:49:20.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.323+0000 7f80c8c34640 1 -- 172.21.15.84:0/3691101401 shutdown_connections 2023-10-16T09:49:20.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.323+0000 7f80c8c34640 1 --2- 172.21.15.84:0/3691101401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80c40b0410 0x7f80c40b0870 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.323+0000 7f80c8c34640 1 --2- 172.21.15.84:0/3691101401 >> v2:172.21.15.157:3300/0 conn(0x7f80c415d3f0 0x7f80c415d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.323+0000 7f80c8c34640 1 --2- 172.21.15.84:0/3691101401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c40cd0c0 0x7f80c41615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.323+0000 7f80c8c34640 1 -- 172.21.15.84:0/3691101401 >> 172.21.15.84:0/3691101401 conn(0x7f80c40cb410 msgr2=0x7f80c40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:20.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.323+0000 7f80c8c34640 1 -- 172.21.15.84:0/3691101401 shutdown_connections 2023-10-16T09:49:20.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.323+0000 7f80c8c34640 1 -- 172.21.15.84:0/3691101401 wait complete. 2023-10-16T09:49:20.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.325+0000 7f80c8c34640 1 Processor -- start 2023-10-16T09:49:20.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.326+0000 7f80c8c34640 1 -- start start 2023-10-16T09:49:20.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.327+0000 7f80c8c34640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c40b0410 0x7f80c40b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:20.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.327+0000 7f80c8c34640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80c40cd0c0 0x7f80c41f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:20.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.327+0000 7f80c8c34640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c415d3f0 0x7f80c41f2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:20.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.327+0000 7f80c8c34640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c41f2b60 con 0x7f80c40cd0c0 2023-10-16T09:49:20.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.327+0000 7f80c8c34640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c40b0190 con 0x7f80c415d3f0 2023-10-16T09:49:20.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.327+0000 7f80c8c34640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c40af580 con 0x7f80c40b0410 2023-10-16T09:49:20.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.327+0000 7f80bbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80c40cd0c0 0x7f80c41f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:20.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.327+0000 7f80bbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80c40cd0c0 0x7f80c41f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34016/0 (socket says 172.21.15.84:34016) 2023-10-16T09:49:20.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.327+0000 7f80bbfff640 1 -- 172.21.15.84:0/574362509 learned_addr learned my addr 172.21.15.84:0/574362509 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:20.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.327+0000 7f80c2d9d640 1 --2- 172.21.15.84:0/574362509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c415d3f0 0x7f80c41f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:20.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.327+0000 7f80c259c640 1 --2- 172.21.15.84:0/574362509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c40b0410 0x7f80c40b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:20.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.327+0000 7f80bbfff640 1 -- 172.21.15.84:0/574362509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c40b0410 msgr2=0x7f80c40b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:20.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.327+0000 7f80bbfff640 1 --2- 172.21.15.84:0/574362509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c40b0410 0x7f80c40b2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.327+0000 7f80bbfff640 1 -- 172.21.15.84:0/574362509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c415d3f0 msgr2=0x7f80c41f2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:20.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.327+0000 7f80bbfff640 1 --2- 172.21.15.84:0/574362509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c415d3f0 0x7f80c41f2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.327+0000 7f80bbfff640 1 -- 172.21.15.84:0/574362509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ac0094d0 con 0x7f80c40cd0c0 2023-10-16T09:49:20.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.328+0000 7f80bbfff640 1 --2- 172.21.15.84:0/574362509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80c40cd0c0 0x7f80c41f20e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f80ac000c00 tx=0x7f80ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:20.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.328+0000 7f80baffd640 1 -- 172.21.15.84:0/574362509 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac032880 con 0x7f80c40cd0c0 2023-10-16T09:49:20.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.328+0000 7f80baffd640 1 -- 172.21.15.84:0/574362509 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ac032a20 con 0x7f80c40cd0c0 2023-10-16T09:49:20.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.328+0000 7f80c8c34640 1 -- 172.21.15.84:0/574362509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c40af860 con 0x7f80c40cd0c0 2023-10-16T09:49:20.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.328+0000 7f80baffd640 1 -- 172.21.15.84:0/574362509 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac032dd0 con 0x7f80c40cd0c0 2023-10-16T09:49:20.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.328+0000 7f80c8c34640 1 -- 172.21.15.84:0/574362509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c40afe20 con 0x7f80c40cd0c0 2023-10-16T09:49:20.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.329+0000 7f80c8c34640 1 -- 172.21.15.84:0/574362509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c41f2cd0 con 0x7f80c40cd0c0 2023-10-16T09:49:20.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.330+0000 7f80baffd640 1 -- 172.21.15.84:0/574362509 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f80ac04e450 con 0x7f80c40cd0c0 2023-10-16T09:49:20.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.331+0000 7f80baffd640 1 --2- 172.21.15.84:0/574362509 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f80a406aea0 0x7f80a406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:20.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.331+0000 7f80baffd640 1 -- 172.21.15.84:0/574362509 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80ac0ab710 con 0x7f80c40cd0c0 2023-10-16T09:49:20.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.331+0000 7f80c259c640 1 --2- 172.21.15.84:0/574362509 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f80a406aea0 0x7f80a406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:20.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.332+0000 7f80c259c640 1 --2- 172.21.15.84:0/574362509 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f80a406aea0 0x7f80a406d360 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f80b4006fd0 tx=0x7f80b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:20.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.335+0000 7f80baffd640 1 -- 172.21.15.84:0/574362509 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f80ac076880 con 0x7f80c40cd0c0 2023-10-16T09:49:20.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.531+0000 7f80c8c34640 1 -- 172.21.15.84:0/574362509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80c404e160 con 0x7f80c40cd0c0 2023-10-16T09:49:20.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.533+0000 7f80baffd640 1 -- 172.21.15.84:0/574362509 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f80ac079ea0 con 0x7f80c40cd0c0 2023-10-16T09:49:20.535 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:20.535 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":21,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:03.529261+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:20.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.535+0000 7f80c8c34640 1 -- 172.21.15.84:0/574362509 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f80a406aea0 msgr2=0x7f80a406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:20.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.535+0000 7f80c8c34640 1 --2- 172.21.15.84:0/574362509 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f80a406aea0 0x7f80a406d360 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f80b4006fd0 tx=0x7f80b4008040 comp rx=0 tx=0).stop 2023-10-16T09:49:20.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.535+0000 7f80c8c34640 1 -- 172.21.15.84:0/574362509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80c40cd0c0 msgr2=0x7f80c41f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:20.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.535+0000 7f80c8c34640 1 --2- 172.21.15.84:0/574362509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80c40cd0c0 0x7f80c41f20e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f80ac000c00 tx=0x7f80ac0046c0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.535+0000 7f80c8c34640 1 -- 172.21.15.84:0/574362509 shutdown_connections 2023-10-16T09:49:20.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.535+0000 7f80c8c34640 1 --2- 172.21.15.84:0/574362509 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f80a406aea0 0x7f80a406d360 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.535+0000 7f80c8c34640 1 --2- 172.21.15.84:0/574362509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c415d3f0 0x7f80c41f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.535+0000 7f80c8c34640 1 --2- 172.21.15.84:0/574362509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80c40cd0c0 0x7f80c41f20e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.535+0000 7f80c8c34640 1 --2- 172.21.15.84:0/574362509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c40b0410 0x7f80c40b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.535+0000 7f80c8c34640 1 -- 172.21.15.84:0/574362509 >> 172.21.15.84:0/574362509 conn(0x7f80c40cb410 msgr2=0x7f80c40c9770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:20.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.536+0000 7f80c8c34640 1 -- 172.21.15.84:0/574362509 shutdown_connections 2023-10-16T09:49:20.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.536+0000 7f80c8c34640 1 -- 172.21.15.84:0/574362509 wait complete. 2023-10-16T09:49:20.556 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:49:20.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.764+0000 7f0805eb4640 1 Processor -- start 2023-10-16T09:49:20.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.764+0000 7f0805eb4640 1 -- start start 2023-10-16T09:49:20.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.765+0000 7f0805eb4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0800159b20 0x7f080006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:20.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.765+0000 7f0805eb4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0800159f00 0x7f08000b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:20.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.765+0000 7f0805eb4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f080006f7f0 con 0x7f080006c120 2023-10-16T09:49:20.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.765+0000 7f0805eb4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f080006c5e0 con 0x7f0800159f00 2023-10-16T09:49:20.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.765+0000 7f0805eb4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080006c8c0 con 0x7f0800159b20 2023-10-16T09:49:20.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.766+0000 7f07ff7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0800159b20 0x7f080006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:20.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.766+0000 7f07fffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f080006c120 0x7f08000cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:45148/0 (socket says 172.21.15.84:45148) 2023-10-16T09:49:20.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.766+0000 7f07fffff640 1 -- 172.21.15.84:0/3345543443 learned_addr learned my addr 172.21.15.84:0/3345543443 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:20.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.766+0000 7f07feffd640 1 --2- 172.21.15.84:0/3345543443 >> v2:172.21.15.157:3300/0 conn(0x7f0800159f00 0x7f08000b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:20.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.767+0000 7f07feffd640 1 -- 172.21.15.84:0/3345543443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0800159b20 msgr2=0x7f080006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:20.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.767+0000 7f07feffd640 1 --2- 172.21.15.84:0/3345543443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0800159b20 0x7f080006bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.767+0000 7f07feffd640 1 -- 172.21.15.84:0/3345543443 >> v1:172.21.15.157:6789/0 conn(0x7f080006c120 legacy=0x7f08000cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:20.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.767+0000 7f07feffd640 1 -- 172.21.15.84:0/3345543443 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080006cba0 con 0x7f0800159f00 2023-10-16T09:49:20.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.768+0000 7f07feffd640 1 --2- 172.21.15.84:0/3345543443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0800159f00 0x7f08000b0410 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f07e80097a0 tx=0x7f07e802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aae1a572dce668b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:20.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.769+0000 7f07fe7fc640 1 -- 172.21.15.84:0/3345543443 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8031830 con 0x7f0800159f00 2023-10-16T09:49:20.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.769+0000 7f07fe7fc640 1 -- 172.21.15.84:0/3345543443 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e8031990 con 0x7f0800159f00 2023-10-16T09:49:20.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.769+0000 7f07fe7fc640 1 -- 172.21.15.84:0/3345543443 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8031d20 con 0x7f0800159f00 2023-10-16T09:49:20.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.770+0000 7f0805eb4640 1 -- 172.21.15.84:0/3345543443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0800159f00 msgr2=0x7f08000b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:20.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.770+0000 7f0805eb4640 1 --2- 172.21.15.84:0/3345543443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0800159f00 0x7f08000b0410 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f07e80097a0 tx=0x7f07e802b8d0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.770+0000 7f0805eb4640 1 -- 172.21.15.84:0/3345543443 shutdown_connections 2023-10-16T09:49:20.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.770+0000 7f0805eb4640 1 --2- 172.21.15.84:0/3345543443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0800159f00 0x7f08000b0410 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.770+0000 7f0805eb4640 1 --2- 172.21.15.84:0/3345543443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0800159b20 0x7f080006bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.770+0000 7f0805eb4640 1 -- 172.21.15.84:0/3345543443 >> 172.21.15.84:0/3345543443 conn(0x7f08000cb410 msgr2=0x7f08000cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:20.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.770+0000 7f0805eb4640 1 -- 172.21.15.84:0/3345543443 shutdown_connections 2023-10-16T09:49:20.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.770+0000 7f0805eb4640 1 -- 172.21.15.84:0/3345543443 wait complete. 2023-10-16T09:49:20.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.772+0000 7f0805eb4640 1 Processor -- start 2023-10-16T09:49:20.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.772+0000 7f0805eb4640 1 -- start start 2023-10-16T09:49:20.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.773+0000 7f0805eb4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f080006c120 0x7f080014f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:20.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.773+0000 7f0805eb4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0800159b20 0x7f08000c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:20.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.773+0000 7f0805eb4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0800159f00 0x7f08000ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:20.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.773+0000 7f0805eb4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080014e0a0 con 0x7f080006c120 2023-10-16T09:49:20.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.773+0000 7f0805eb4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080014e380 con 0x7f0800159f00 2023-10-16T09:49:20.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.773+0000 7f0805eb4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080014e660 con 0x7f0800159b20 2023-10-16T09:49:20.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.774+0000 7f07ff7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f080006c120 0x7f080014f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:20.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.774+0000 7f07ff7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f080006c120 0x7f080014f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34020/0 (socket says 172.21.15.84:34020) 2023-10-16T09:49:20.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.774+0000 7f07ff7fe640 1 -- 172.21.15.84:0/1260618580 learned_addr learned my addr 172.21.15.84:0/1260618580 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:20.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.774+0000 7f07feffd640 1 --2- 172.21.15.84:0/1260618580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0800159b20 0x7f08000c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:20.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.774+0000 7f07fffff640 1 --2- 172.21.15.84:0/1260618580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0800159f00 0x7f08000ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:20.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.774+0000 7f07ff7fe640 1 -- 172.21.15.84:0/1260618580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0800159b20 msgr2=0x7f08000c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:20.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.774+0000 7f07ff7fe640 1 --2- 172.21.15.84:0/1260618580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0800159b20 0x7f08000c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.774+0000 7f07ff7fe640 1 -- 172.21.15.84:0/1260618580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0800159f00 msgr2=0x7f08000ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:20.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.774+0000 7f07ff7fe640 1 --2- 172.21.15.84:0/1260618580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0800159f00 0x7f08000ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:20.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.774+0000 7f07ff7fe640 1 -- 172.21.15.84:0/1260618580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e8009400 con 0x7f080006c120 2023-10-16T09:49:20.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.774+0000 7f07feffd640 1 --2- 172.21.15.84:0/1260618580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0800159b20 0x7f08000c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:20.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.775+0000 7f07ff7fe640 1 --2- 172.21.15.84:0/1260618580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f080006c120 0x7f080014f140 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f07f000efc0 tx=0x7f07f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:20.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.775+0000 7f07f7fff640 1 -- 172.21.15.84:0/1260618580 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0014480 con 0x7f080006c120 2023-10-16T09:49:20.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.775+0000 7f07f7fff640 1 -- 172.21.15.84:0/1260618580 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f000f040 con 0x7f080006c120 2023-10-16T09:49:20.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.775+0000 7f0805eb4640 1 -- 172.21.15.84:0/1260618580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f080014e940 con 0x7f080006c120 2023-10-16T09:49:20.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.775+0000 7f07f7fff640 1 -- 172.21.15.84:0/1260618580 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0014800 con 0x7f080006c120 2023-10-16T09:49:20.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.775+0000 7f0805eb4640 1 -- 172.21.15.84:0/1260618580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08000c84e0 con 0x7f080006c120 2023-10-16T09:49:20.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.776+0000 7f0805eb4640 1 -- 172.21.15.84:0/1260618580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f080004e160 con 0x7f080006c120 2023-10-16T09:49:20.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.777+0000 7f07f7fff640 1 -- 172.21.15.84:0/1260618580 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f07f0014960 con 0x7f080006c120 2023-10-16T09:49:20.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.779+0000 7f07f7fff640 1 --2- 172.21.15.84:0/1260618580 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f07e006aea0 0x7f07e006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:20.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.779+0000 7f07f7fff640 1 -- 172.21.15.84:0/1260618580 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07f008a260 con 0x7f080006c120 2023-10-16T09:49:20.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.779+0000 7f07feffd640 1 --2- 172.21.15.84:0/1260618580 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f07e006aea0 0x7f07e006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:20.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.780+0000 7f07feffd640 1 --2- 172.21.15.84:0/1260618580 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f07e006aea0 0x7f07e006d360 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f07e8036000 tx=0x7f07e803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:20.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:20.783+0000 7f07f7fff640 1 -- 172.21.15.84:0/1260618580 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f07f0055270 con 0x7f080006c120 2023-10-16T09:49:21.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.014+0000 7f0805eb4640 1 -- 172.21.15.84:0/1260618580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08000b10b0 con 0x7f080006c120 2023-10-16T09:49:21.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.015+0000 7f07f7fff640 1 -- 172.21.15.84:0/1260618580 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f07f0058890 con 0x7f080006c120 2023-10-16T09:49:21.018 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:21.018 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2978888131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2978888131},{"type":"v1","addr":"172.21.15.157:6834","nonce":2978888131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3716581310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3716581310},{"type":"v1","addr":"172.21.15.84:6835","nonce":3716581310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2761479872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2761479872},{"type":"v1","addr":"172.21.15.84:6837","nonce":2761479872}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:49:02.350624+0000","modified":"2023-10-16T09:49:05.402785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/848754825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":848754825},{"type":"v1","addr":"172.21.15.157:6835","nonce":848754825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T09:49:21.018 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-10-16T09:49:21.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.018+0000 7f0805eb4640 1 -- 172.21.15.84:0/1260618580 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f07e006aea0 msgr2=0x7f07e006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:21.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.018+0000 7f0805eb4640 1 --2- 172.21.15.84:0/1260618580 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f07e006aea0 0x7f07e006d360 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f07e8036000 tx=0x7f07e803c040 comp rx=0 tx=0).stop 2023-10-16T09:49:21.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.018+0000 7f0805eb4640 1 -- 172.21.15.84:0/1260618580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f080006c120 msgr2=0x7f080014f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:21.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.018+0000 7f0805eb4640 1 --2- 172.21.15.84:0/1260618580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f080006c120 0x7f080014f140 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f07f000efc0 tx=0x7f07f000c490 comp rx=0 tx=0).stop 2023-10-16T09:49:21.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.018+0000 7f0805eb4640 1 -- 172.21.15.84:0/1260618580 shutdown_connections 2023-10-16T09:49:21.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.018+0000 7f0805eb4640 1 --2- 172.21.15.84:0/1260618580 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f07e006aea0 0x7f07e006d360 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.018+0000 7f0805eb4640 1 --2- 172.21.15.84:0/1260618580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0800159f00 0x7f08000ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.018+0000 7f0805eb4640 1 --2- 172.21.15.84:0/1260618580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0800159b20 0x7f08000c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.018+0000 7f0805eb4640 1 --2- 172.21.15.84:0/1260618580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f080006c120 0x7f080014f140 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.018+0000 7f0805eb4640 1 -- 172.21.15.84:0/1260618580 >> 172.21.15.84:0/1260618580 conn(0x7f08000cb410 msgr2=0x7f08000cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:21.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.019+0000 7f0805eb4640 1 -- 172.21.15.84:0/1260618580 shutdown_connections 2023-10-16T09:49:21.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.019+0000 7f0805eb4640 1 -- 172.21.15.84:0/1260618580 wait complete. 2023-10-16T09:49:21.442 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T09:49:02.350624+0000', 'modified': '2023-10-16T09:49:05.402785+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4261}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4261': {'gid': 4261, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/848754825', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6832', 'nonce': 848754825}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 848754825}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T09:49:21.442 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T09:49:21.442 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-16T09:49:21.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.660+0000 7fb829389640 1 Processor -- start 2023-10-16T09:49:21.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.661+0000 7fb829389640 1 -- start start 2023-10-16T09:49:21.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.662+0000 7fb829389640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8240ad350 0x7fb824069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.662+0000 7fb829389640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb82406a2e0 0x7fb8240c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.662+0000 7fb829389640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb82406a6f0 con 0x7fb82415d150 2023-10-16T09:49:21.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.662+0000 7fb829389640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb82415d530 con 0x7fb82406a2e0 2023-10-16T09:49:21.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.662+0000 7fb829389640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8240ac890 con 0x7fb8240ad350 2023-10-16T09:49:21.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.663+0000 7fb8237fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb82406a2e0 0x7fb8240c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:21.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.663+0000 7fb822ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb82415d150 0x7fb824160b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36840/0 (socket says 172.21.15.84:36840) 2023-10-16T09:49:21.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.663+0000 7fb8237fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb82406a2e0 0x7fb8240c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34034/0 (socket says 172.21.15.84:34034) 2023-10-16T09:49:21.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.663+0000 7fb822ffd640 1 -- 172.21.15.84:0/682487541 learned_addr learned my addr 172.21.15.84:0/682487541 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:21.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.663+0000 7fb81bfff640 1 --2- 172.21.15.84:0/682487541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8240ad350 0x7fb824069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:21.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.663+0000 7fb8237fe640 1 -- 172.21.15.84:0/682487541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8240ad350 msgr2=0x7fb824069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:21.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.663+0000 7fb8237fe640 1 --2- 172.21.15.84:0/682487541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8240ad350 0x7fb824069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.663+0000 7fb8237fe640 1 -- 172.21.15.84:0/682487541 >> v1:172.21.15.84:6789/0 conn(0x7fb82415d150 legacy=0x7fb824160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:21.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.663+0000 7fb8237fe640 1 -- 172.21.15.84:0/682487541 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb824161650 con 0x7fb82406a2e0 2023-10-16T09:49:21.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.663+0000 7fb8227fc640 1 -- 172.21.15.84:0/682487541 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408656891 0 0) 0x7fb82406a6f0 con 0x7fb82415d150 2023-10-16T09:49:21.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.664+0000 7fb8237fe640 1 --2- 172.21.15.84:0/682487541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb82406a2e0 0x7fb8240c9000 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009d90 tx=0x7fb81c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b565f0059baa6e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:21.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.665+0000 7fb8227fc640 1 -- 172.21.15.84:0/682487541 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c02cc50 con 0x7fb82406a2e0 2023-10-16T09:49:21.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.665+0000 7fb8227fc640 1 -- 172.21.15.84:0/682487541 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c02cdb0 con 0x7fb82406a2e0 2023-10-16T09:49:21.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.665+0000 7fb8227fc640 1 -- 172.21.15.84:0/682487541 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c034660 con 0x7fb82406a2e0 2023-10-16T09:49:21.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.665+0000 7fb829389640 1 -- 172.21.15.84:0/682487541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb82406a2e0 msgr2=0x7fb8240c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:21.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.665+0000 7fb829389640 1 --2- 172.21.15.84:0/682487541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb82406a2e0 0x7fb8240c9000 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009d90 tx=0x7fb81c02bea0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.665+0000 7fb829389640 1 -- 172.21.15.84:0/682487541 shutdown_connections 2023-10-16T09:49:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.665+0000 7fb829389640 1 --2- 172.21.15.84:0/682487541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb82406a2e0 0x7fb8240c9000 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.665+0000 7fb829389640 1 --2- 172.21.15.84:0/682487541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8240ad350 0x7fb824069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.665+0000 7fb829389640 1 -- 172.21.15.84:0/682487541 >> 172.21.15.84:0/682487541 conn(0x7fb8240ce9b0 msgr2=0x7fb8240cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.665+0000 7fb829389640 1 -- 172.21.15.84:0/682487541 shutdown_connections 2023-10-16T09:49:21.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.665+0000 7fb829389640 1 -- 172.21.15.84:0/682487541 wait complete. 2023-10-16T09:49:21.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.667+0000 7fb829389640 1 Processor -- start 2023-10-16T09:49:21.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.667+0000 7fb829389640 1 -- start start 2023-10-16T09:49:21.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.668+0000 7fb829389640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb82406a2e0 0x7fb8241f3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.668+0000 7fb829389640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8240ad350 0x7fb824158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.668+0000 7fb829389640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb82415d150 0x7fb8241f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.668+0000 7fb829389640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8241f3750 con 0x7fb82415d150 2023-10-16T09:49:21.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.668+0000 7fb829389640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8241f2750 con 0x7fb82406a2e0 2023-10-16T09:49:21.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.668+0000 7fb829389640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb824159330 con 0x7fb8240ad350 2023-10-16T09:49:21.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.668+0000 7fb8237fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb82415d150 0x7fb8241f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:21.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.668+0000 7fb8237fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb82415d150 0x7fb8241f5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34044/0 (socket says 172.21.15.84:34044) 2023-10-16T09:49:21.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.668+0000 7fb8237fe640 1 -- 172.21.15.84:0/3769580543 learned_addr learned my addr 172.21.15.84:0/3769580543 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:21.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.669+0000 7fb81bfff640 1 --2- 172.21.15.84:0/3769580543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8240ad350 0x7fb824158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:21.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.669+0000 7fb8237fe640 1 -- 172.21.15.84:0/3769580543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8240ad350 msgr2=0x7fb824158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:21.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.669+0000 7fb8237fe640 1 --2- 172.21.15.84:0/3769580543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8240ad350 0x7fb824158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.669+0000 7fb8237fe640 1 -- 172.21.15.84:0/3769580543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb82406a2e0 msgr2=0x7fb8241f3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:21.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.669+0000 7fb8237fe640 1 --2- 172.21.15.84:0/3769580543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb82406a2e0 0x7fb8241f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.669+0000 7fb8237fe640 1 -- 172.21.15.84:0/3769580543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c009a70 con 0x7fb82415d150 2023-10-16T09:49:21.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.669+0000 7fb8237fe640 1 --2- 172.21.15.84:0/3769580543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb82415d150 0x7fb8241f5d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009d90 tx=0x7fb81c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:21.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.669+0000 7fb81b7fe640 1 -- 172.21.15.84:0/3769580543 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c003b90 con 0x7fb82415d150 2023-10-16T09:49:21.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.670+0000 7fb81b7fe640 1 -- 172.21.15.84:0/3769580543 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c003cf0 con 0x7fb82415d150 2023-10-16T09:49:21.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.670+0000 7fb829389640 1 -- 172.21.15.84:0/3769580543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8241f67c0 con 0x7fb82415d150 2023-10-16T09:49:21.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.670+0000 7fb81b7fe640 1 -- 172.21.15.84:0/3769580543 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c004270 con 0x7fb82415d150 2023-10-16T09:49:21.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.670+0000 7fb829389640 1 -- 172.21.15.84:0/3769580543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8241f2a10 con 0x7fb82415d150 2023-10-16T09:49:21.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.672+0000 7fb81b7fe640 1 -- 172.21.15.84:0/3769580543 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb81c03b050 con 0x7fb82415d150 2023-10-16T09:49:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.673+0000 7fb81b7fe640 1 --2- 172.21.15.84:0/3769580543 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb80406aea0 0x7fb80406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.673+0000 7fb81b7fe640 1 -- 172.21.15.84:0/3769580543 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb81c0abbd0 con 0x7fb82415d150 2023-10-16T09:49:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.673+0000 7fb822ffd640 1 --2- 172.21.15.84:0/3769580543 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb80406aea0 0x7fb80406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.674+0000 7fb82bffc740 1 Processor -- start 2023-10-16T09:49:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.674+0000 7fb82bffc740 1 -- start start 2023-10-16T09:49:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.674+0000 7fb822ffd640 1 --2- 172.21.15.84:0/3769580543 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb80406aea0 0x7fb80406d360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb814007fd0 tx=0x7fb814007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.675+0000 7fb82bffc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b1cb8af80 0x564b1cb5de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.675+0000 7fb82bffc740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x564b1cb8b360 0x564b1cb8dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.675+0000 7fb82bffc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb5e3c0 0x564b1cb8e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.675+0000 7fb82bffc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1ca7e7a0 con 0x564b1cb5e3c0 2023-10-16T09:49:21.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.675+0000 7fb82bffc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1cad0740 con 0x564b1cb8af80 2023-10-16T09:49:21.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.675+0000 7fb82bffc740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b1cacde30 con 0x564b1cb8b360 2023-10-16T09:49:21.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.675+0000 7fb8237fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb5e3c0 0x564b1cb8e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:21.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.675+0000 7fb8237fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb5e3c0 0x564b1cb8e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34054/0 (socket says 172.21.15.84:34054) 2023-10-16T09:49:21.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.675+0000 7fb8237fe640 1 -- 172.21.15.84:0/3805252897 learned_addr learned my addr 172.21.15.84:0/3805252897 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:21.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.675+0000 7fb8237fe640 1 -- 172.21.15.84:0/3805252897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x564b1cb8b360 msgr2=0x564b1cb8dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:21.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.675+0000 7fb8237fe640 1 --2- 172.21.15.84:0/3805252897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x564b1cb8b360 0x564b1cb8dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.675+0000 7fb8237fe640 1 -- 172.21.15.84:0/3805252897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b1cb8af80 msgr2=0x564b1cb5de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:21.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.675+0000 7fb8237fe640 1 --2- 172.21.15.84:0/3805252897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b1cb8af80 0x564b1cb5de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.675+0000 7fb8237fe640 1 -- 172.21.15.84:0/3805252897 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b1ca8d910 con 0x564b1cb5e3c0 2023-10-16T09:49:21.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.676+0000 7fb8237fe640 1 --2- 172.21.15.84:0/3805252897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb5e3c0 0x564b1cb8e250 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb81c082700 tx=0x7fb81c07a350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6daa64a0e4c13246 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:21.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.676+0000 7fb818ff9640 1 -- 172.21.15.84:0/3805252897 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c07a930 con 0x564b1cb5e3c0 2023-10-16T09:49:21.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.676+0000 7fb818ff9640 1 -- 172.21.15.84:0/3805252897 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c07ea80 con 0x564b1cb5e3c0 2023-10-16T09:49:21.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.676+0000 7fb818ff9640 1 -- 172.21.15.84:0/3805252897 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c082c30 con 0x564b1cb5e3c0 2023-10-16T09:49:21.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.676+0000 7fb82bffc740 1 -- 172.21.15.84:0/3805252897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb5e3c0 msgr2=0x564b1cb8e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:21.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.676+0000 7fb82bffc740 1 --2- 172.21.15.84:0/3805252897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb5e3c0 0x564b1cb8e250 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb81c082700 tx=0x7fb81c07a350 comp rx=0 tx=0).stop 2023-10-16T09:49:21.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.677+0000 7fb82bffc740 1 -- 172.21.15.84:0/3805252897 shutdown_connections 2023-10-16T09:49:21.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.677+0000 7fb82bffc740 1 --2- 172.21.15.84:0/3805252897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb5e3c0 0x564b1cb8e250 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.677+0000 7fb82bffc740 1 --2- 172.21.15.84:0/3805252897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x564b1cb8b360 0x564b1cb8dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.677+0000 7fb82bffc740 1 --2- 172.21.15.84:0/3805252897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b1cb8af80 0x564b1cb5de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.677+0000 7fb82bffc740 1 -- 172.21.15.84:0/3805252897 >> 172.21.15.84:0/3805252897 conn(0x564b1cb60190 msgr2=0x564b1cba8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:21.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.677+0000 7fb82bffc740 1 -- 172.21.15.84:0/3805252897 shutdown_connections 2023-10-16T09:49:21.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.677+0000 7fb82bffc740 1 -- 172.21.15.84:0/3805252897 wait complete. 2023-10-16T09:49:21.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.677+0000 7fb82bffc740 1 -- start start 2023-10-16T09:49:21.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.678+0000 7fb82bffc740 1 Processor -- start 2023-10-16T09:49:21.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.679+0000 7fb82bffc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb8af80 0x564b1cba1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.679+0000 7fb82bffc740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x564b1cb8b360 0x564b1cb9f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.679+0000 7fb82bffc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b1cb79cf0 0x564b1cb7a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.679+0000 7fb82bffc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1cac9720 con 0x564b1cb8af80 2023-10-16T09:49:21.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.679+0000 7fb82bffc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1ca67970 con 0x564b1cb79cf0 2023-10-16T09:49:21.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.679+0000 7fb82bffc740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b1ca67610 con 0x564b1cb8b360 2023-10-16T09:49:21.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.679+0000 7fb82bffc740 10 client.0 _ll_register_callbacks cb 0x564b1ca899d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T09:49:21.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.679+0000 7fb822ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb8af80 0x564b1cba1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:21.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.679+0000 7fb822ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb8af80 0x564b1cba1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34064/0 (socket says 172.21.15.84:34064) 2023-10-16T09:49:21.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.679+0000 7fb822ffd640 1 -- 172.21.15.84:0/2012597533 learned_addr learned my addr 172.21.15.84:0/2012597533 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:21.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.679+0000 7fb822ffd640 1 -- 172.21.15.84:0/2012597533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x564b1cb8b360 msgr2=0x564b1cb9f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:21.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.679+0000 7fb822ffd640 1 --2- 172.21.15.84:0/2012597533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x564b1cb8b360 0x564b1cb9f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.679+0000 7fb822ffd640 1 -- 172.21.15.84:0/2012597533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b1cb79cf0 msgr2=0x564b1cb7a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:21.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.679+0000 7fb822ffd640 1 --2- 172.21.15.84:0/2012597533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b1cb79cf0 0x564b1cb7a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.679+0000 7fb822ffd640 1 -- 172.21.15.84:0/2012597533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8241f2a10 con 0x564b1cb8af80 2023-10-16T09:49:21.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.680+0000 7fb822ffd640 1 --2- 172.21.15.84:0/2012597533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb8af80 0x564b1cba1790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb814005da0 tx=0x7fb814016820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f054e8e6ebd6b3a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:21.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.680+0000 7fb801ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T09:49:21.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.680+0000 7fb801ffb640 1 -- 172.21.15.84:0/2012597533 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81401b070 con 0x564b1cb8af80 2023-10-16T09:49:21.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.680+0000 7fb801ffb640 1 -- 172.21.15.84:0/2012597533 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb814016dd0 con 0x564b1cb8af80 2023-10-16T09:49:21.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.680+0000 7fb82bffc740 1 -- 172.21.15.84:0/2012597533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b1caf0160 con 0x564b1cb8af80 2023-10-16T09:49:21.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.680+0000 7fb801ffb640 1 -- 172.21.15.84:0/2012597533 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb814016110 con 0x564b1cb8af80 2023-10-16T09:49:21.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.680+0000 7fb82bffc740 10 client.4692 fetch_fsmap learned FSMap version 8 2023-10-16T09:49:21.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.680+0000 7fb82bffc740 1 -- 172.21.15.84:0/2012597533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x564b1cac7da0 con 0x564b1cb8af80 2023-10-16T09:49:21.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.680+0000 7fb801ffb640 1 -- 172.21.15.84:0/2012597533 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564b1caf0160 con 0x564b1cb8af80 2023-10-16T09:49:21.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.681+0000 7fb801ffb640 1 -- 172.21.15.84:0/2012597533 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb814012440 con 0x564b1cb8af80 2023-10-16T09:49:21.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.681+0000 7fb82bffc740 10 client.4692 fetch_fsmap finished waiting for FSMap version 8 2023-10-16T09:49:21.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.681+0000 7fb82bffc740 10 client.4692 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T09:49:21.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.681+0000 7fb82bffc740 20 client.4692 populate_metadata read hostname 'smithi084' 2023-10-16T09:49:21.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.681+0000 7fb82bffc740 1 --2- 172.21.15.84:0/2012597533 >> [v2:172.21.15.157:6832/848754825,v1:172.21.15.157:6835/848754825] conn(0x564b1cb88c30 0x564b1cbc5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.681+0000 7fb82bffc740 4 client.4692 mds_command: new command op to 4261 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T09:49:21.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.681+0000 7fb82bffc740 1 -- 172.21.15.84:0/2012597533 --> [v2:172.21.15.157:6832/848754825,v1:172.21.15.157:6835/848754825] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b1ca9bcc0 con 0x564b1cb88c30 2023-10-16T09:49:21.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.681+0000 7fb81bfff640 1 --2- 172.21.15.84:0/2012597533 >> [v2:172.21.15.157:6832/848754825,v1:172.21.15.157:6835/848754825] conn(0x564b1cb88c30 0x564b1cbc5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:21.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.682+0000 7fb801ffb640 10 client.4692 ms_handle_connect on v2:172.21.15.157:6832/848754825 2023-10-16T09:49:21.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.682+0000 7fb81bfff640 1 --2- 172.21.15.84:0/2012597533 >> [v2:172.21.15.157:6832/848754825,v1:172.21.15.157:6835/848754825] conn(0x564b1cb88c30 0x564b1cbc5cd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b6642ca7fbeb4db server_cookie=612d513f47ae6522 in_seq=0 out_seq=0 2023-10-16T09:49:21.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.684+0000 7fb801ffb640 1 -- 172.21.15.84:0/2012597533 <== mds.0 v2:172.21.15.157:6832/848754825 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564b1ca9bcc0 con 0x564b1cb88c30 2023-10-16T09:49:21.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.684+0000 7fb801ffb640 10 client.4692 handle_command_reply: tid=0 2023-10-16T09:49:21.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.685+0000 7fb82bffc740 1 client.4692 shutdown 2023-10-16T09:49:21.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.685+0000 7fb82bffc740 1 -- 172.21.15.84:0/2012597533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb8af80 msgr2=0x564b1cba1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:21.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.685+0000 7fb82bffc740 1 --2- 172.21.15.84:0/2012597533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb8af80 0x564b1cba1790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb814005da0 tx=0x7fb814016820 comp rx=0 tx=0).stop 2023-10-16T09:49:21.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.685+0000 7fb82bffc740 1 -- 172.21.15.84:0/2012597533 shutdown_connections 2023-10-16T09:49:21.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.685+0000 7fb82bffc740 1 --2- 172.21.15.84:0/2012597533 >> [v2:172.21.15.157:6832/848754825,v1:172.21.15.157:6835/848754825] conn(0x564b1cb88c30 0x564b1cbc5cd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.685+0000 7fb82bffc740 1 --2- 172.21.15.84:0/2012597533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b1cb79cf0 0x564b1cb7a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.685+0000 7fb82bffc740 1 --2- 172.21.15.84:0/2012597533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x564b1cb8b360 0x564b1cb9f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.685+0000 7fb801ffb640 0 client.4692 ms_handle_reset on v2:172.21.15.157:6832/848754825 2023-10-16T09:49:21.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.685+0000 7fb82bffc740 1 --2- 172.21.15.84:0/2012597533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb8af80 0x564b1cba1790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.685+0000 7fb82bffc740 1 -- 172.21.15.84:0/2012597533 >> 172.21.15.84:0/2012597533 conn(0x564b1cb5e3c0 msgr2=0x564b1cb61e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:21.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.685+0000 7fb82bffc740 1 -- 172.21.15.84:0/2012597533 shutdown_connections 2023-10-16T09:49:21.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.685+0000 7fb82bffc740 1 -- 172.21.15.84:0/2012597533 wait complete. 2023-10-16T09:49:21.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.685+0000 7fb82bffc740 20 client.4692 trim_cache size 0 max 16384 2023-10-16T09:49:21.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.702+0000 7fb82bffc740 1 Processor -- start 2023-10-16T09:49:21.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.702+0000 7fb82bffc740 1 -- start start 2023-10-16T09:49:21.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.703+0000 7fb82bffc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb79cf0 0x564b1cbca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.703+0000 7fb82bffc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b1cb8af80 0x564b1cb79400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.703+0000 7fb82bffc740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x564b1cb8b360 0x564b1cbac220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.703+0000 7fb82bffc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1c82df20 con 0x564b1cb79cf0 2023-10-16T09:49:21.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.703+0000 7fb82bffc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1ca6f5c0 con 0x564b1cb8af80 2023-10-16T09:49:21.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.703+0000 7fb82bffc740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b1cba1f70 con 0x564b1cb8b360 2023-10-16T09:49:21.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.703+0000 7fb822ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb79cf0 0x564b1cbca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:21.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.703+0000 7fb822ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb79cf0 0x564b1cbca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34076/0 (socket says 172.21.15.84:34076) 2023-10-16T09:49:21.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.703+0000 7fb822ffd640 1 -- 172.21.15.84:0/1750931029 learned_addr learned my addr 172.21.15.84:0/1750931029 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:21.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.704+0000 7fb822ffd640 1 -- 172.21.15.84:0/1750931029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x564b1cb8b360 msgr2=0x564b1cbac220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:21.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.704+0000 7fb822ffd640 1 --2- 172.21.15.84:0/1750931029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x564b1cb8b360 0x564b1cbac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.704+0000 7fb822ffd640 1 -- 172.21.15.84:0/1750931029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b1cb8af80 msgr2=0x564b1cb79400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:21.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.704+0000 7fb822ffd640 1 --2- 172.21.15.84:0/1750931029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b1cb8af80 0x564b1cb79400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.704+0000 7fb822ffd640 1 -- 172.21.15.84:0/1750931029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b1ca66b80 con 0x564b1cb79cf0 2023-10-16T09:49:21.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.704+0000 7fb822ffd640 1 --2- 172.21.15.84:0/1750931029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb79cf0 0x564b1cbca2d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb814016860 tx=0x7fb81401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9042d1ce64afb0b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:21.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.705+0000 7fb8197fa640 1 -- 172.21.15.84:0/1750931029 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814016d20 con 0x564b1cb79cf0 2023-10-16T09:49:21.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.705+0000 7fb8197fa640 1 -- 172.21.15.84:0/1750931029 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb814016e80 con 0x564b1cb79cf0 2023-10-16T09:49:21.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.705+0000 7fb8197fa640 1 -- 172.21.15.84:0/1750931029 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8140160e0 con 0x564b1cb79cf0 2023-10-16T09:49:21.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.705+0000 7fb82bffc740 1 -- 172.21.15.84:0/1750931029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb79cf0 msgr2=0x564b1cbca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:21.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.705+0000 7fb82bffc740 1 --2- 172.21.15.84:0/1750931029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb79cf0 0x564b1cbca2d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb814016860 tx=0x7fb81401b000 comp rx=0 tx=0).stop 2023-10-16T09:49:21.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.705+0000 7fb82bffc740 1 -- 172.21.15.84:0/1750931029 shutdown_connections 2023-10-16T09:49:21.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.705+0000 7fb82bffc740 1 --2- 172.21.15.84:0/1750931029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x564b1cb8b360 0x564b1cbac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.705+0000 7fb82bffc740 1 --2- 172.21.15.84:0/1750931029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b1cb8af80 0x564b1cb79400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.705+0000 7fb82bffc740 1 --2- 172.21.15.84:0/1750931029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb79cf0 0x564b1cbca2d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.705+0000 7fb82bffc740 1 -- 172.21.15.84:0/1750931029 >> 172.21.15.84:0/1750931029 conn(0x564b1cb5e3c0 msgr2=0x564b1cb9f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:21.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.705+0000 7fb82bffc740 1 -- 172.21.15.84:0/1750931029 shutdown_connections 2023-10-16T09:49:21.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.706+0000 7fb82bffc740 1 -- 172.21.15.84:0/1750931029 wait complete. 2023-10-16T09:49:21.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.706+0000 7fb82bffc740 1 -- start start 2023-10-16T09:49:21.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.706+0000 7fb82bffc740 1 Processor -- start 2023-10-16T09:49:21.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.707+0000 7fb82bffc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb79cf0 0x564b1cba0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.707+0000 7fb82bffc740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x564b1cb8af80 0x564b1cb895c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.707+0000 7fb82bffc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b1cb8b360 0x564b1cb90cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.707+0000 7fb82bffc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1cb89ef0 con 0x564b1cb79cf0 2023-10-16T09:49:21.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.707+0000 7fb82bffc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1cbb73e0 con 0x564b1cb8b360 2023-10-16T09:49:21.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.707+0000 7fb82bffc740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b1cbc1fd0 con 0x564b1cb8af80 2023-10-16T09:49:21.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.707+0000 7fb82bffc740 10 client.0 _ll_register_callbacks cb 0x7fb814001390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T09:49:21.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.708+0000 7fb822ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb79cf0 0x564b1cba0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:21.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.708+0000 7fb822ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb79cf0 0x564b1cba0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34078/0 (socket says 172.21.15.84:34078) 2023-10-16T09:49:21.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.708+0000 7fb822ffd640 1 -- 172.21.15.84:0/2237699653 learned_addr learned my addr 172.21.15.84:0/2237699653 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:21.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.708+0000 7fb81bfff640 1 --2- 172.21.15.84:0/2237699653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x564b1cb8af80 0x564b1cb895c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:21.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.708+0000 7fb822ffd640 1 -- 172.21.15.84:0/2237699653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x564b1cb8af80 msgr2=0x564b1cb895c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:21.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.708+0000 7fb822ffd640 1 --2- 172.21.15.84:0/2237699653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x564b1cb8af80 0x564b1cb895c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.708+0000 7fb822ffd640 1 -- 172.21.15.84:0/2237699653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b1cb8b360 msgr2=0x564b1cb90cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:21.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.708+0000 7fb822ffd640 1 --2- 172.21.15.84:0/2237699653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b1cb8b360 0x564b1cb90cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.708+0000 7fb822ffd640 1 -- 172.21.15.84:0/2237699653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b1cac7da0 con 0x564b1cb79cf0 2023-10-16T09:49:21.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.708+0000 7fb822ffd640 1 --2- 172.21.15.84:0/2237699653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb79cf0 0x564b1cba0d00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb814014010 tx=0x7fb814010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6317aa33c17695 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:21.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.708+0000 7fb801ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T09:49:21.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.709+0000 7fb801ffb640 1 -- 172.21.15.84:0/2237699653 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81401b040 con 0x564b1cb79cf0 2023-10-16T09:49:21.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.709+0000 7fb801ffb640 1 -- 172.21.15.84:0/2237699653 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb814030070 con 0x564b1cb79cf0 2023-10-16T09:49:21.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.709+0000 7fb82bffc740 1 -- 172.21.15.84:0/2237699653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b1cac2700 con 0x564b1cb79cf0 2023-10-16T09:49:21.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.709+0000 7fb801ffb640 1 -- 172.21.15.84:0/2237699653 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8140160e0 con 0x564b1cb79cf0 2023-10-16T09:49:21.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.709+0000 7fb82bffc740 10 client.4698 fetch_fsmap learned FSMap version 8 2023-10-16T09:49:21.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.709+0000 7fb82bffc740 1 -- 172.21.15.84:0/2237699653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x564b1ca8c170 con 0x564b1cb79cf0 2023-10-16T09:49:21.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.709+0000 7fb801ffb640 1 -- 172.21.15.84:0/2237699653 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564b1cac2700 con 0x564b1cb79cf0 2023-10-16T09:49:21.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.709+0000 7fb801ffb640 1 -- 172.21.15.84:0/2237699653 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb814012900 con 0x564b1cb79cf0 2023-10-16T09:49:21.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.709+0000 7fb82bffc740 10 client.4698 fetch_fsmap finished waiting for FSMap version 8 2023-10-16T09:49:21.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.709+0000 7fb82bffc740 10 client.4698 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T09:49:21.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.709+0000 7fb82bffc740 20 client.4698 populate_metadata read hostname 'smithi084' 2023-10-16T09:49:21.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.709+0000 7fb82bffc740 1 --2- 172.21.15.84:0/2237699653 >> [v2:172.21.15.157:6832/848754825,v1:172.21.15.157:6835/848754825] conn(0x564b1cc10c40 0x564b1cbc9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:21.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.709+0000 7fb82bffc740 4 client.4698 mds_command: new command op to 4261 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T09:49:21.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.709+0000 7fb82bffc740 1 -- 172.21.15.84:0/2237699653 --> [v2:172.21.15.157:6832/848754825,v1:172.21.15.157:6835/848754825] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564b1cac2420 con 0x564b1cc10c40 2023-10-16T09:49:21.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.710+0000 7fb81bfff640 1 --2- 172.21.15.84:0/2237699653 >> [v2:172.21.15.157:6832/848754825,v1:172.21.15.157:6835/848754825] conn(0x564b1cc10c40 0x564b1cbc9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:21.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.711+0000 7fb81bfff640 1 --2- 172.21.15.84:0/2237699653 >> [v2:172.21.15.157:6832/848754825,v1:172.21.15.157:6835/848754825] conn(0x564b1cc10c40 0x564b1cbc9cf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c5920b32d8061a7 server_cookie=4141d1788b387fde in_seq=0 out_seq=0 2023-10-16T09:49:21.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.711+0000 7fb801ffb640 10 client.4698 ms_handle_connect on v2:172.21.15.157:6832/848754825 2023-10-16T09:49:21.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.711+0000 7fb801ffb640 1 -- 172.21.15.84:0/2237699653 <== mds.0 v2:172.21.15.157:6832/848754825 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x564b1cac2420 con 0x564b1cc10c40 2023-10-16T09:49:21.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.711+0000 7fb801ffb640 10 client.4698 handle_command_reply: tid=0 2023-10-16T09:49:21.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.711+0000 7fb82bffc740 1 client.4698 shutdown 2023-10-16T09:49:21.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.712+0000 7fb82bffc740 1 -- 172.21.15.84:0/2237699653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb79cf0 msgr2=0x564b1cba0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:21.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.712+0000 7fb82bffc740 1 --2- 172.21.15.84:0/2237699653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb79cf0 0x564b1cba0d00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb814014010 tx=0x7fb814010000 comp rx=0 tx=0).stop 2023-10-16T09:49:21.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.712+0000 7fb82bffc740 1 -- 172.21.15.84:0/2237699653 shutdown_connections 2023-10-16T09:49:21.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.712+0000 7fb82bffc740 1 --2- 172.21.15.84:0/2237699653 >> [v2:172.21.15.157:6832/848754825,v1:172.21.15.157:6835/848754825] conn(0x564b1cc10c40 0x564b1cbc9cf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.712+0000 7fb82bffc740 1 --2- 172.21.15.84:0/2237699653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b1cb8b360 0x564b1cb90cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.712+0000 7fb82bffc740 1 --2- 172.21.15.84:0/2237699653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x564b1cb8af80 0x564b1cb895c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.712+0000 7fb82bffc740 1 --2- 172.21.15.84:0/2237699653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b1cb79cf0 0x564b1cba0d00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.712+0000 7fb801ffb640 0 client.4698 ms_handle_reset on v2:172.21.15.157:6832/848754825 2023-10-16T09:49:21.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.712+0000 7fb82bffc740 1 -- 172.21.15.84:0/2237699653 >> 172.21.15.84:0/2237699653 conn(0x564b1cb5e3c0 msgr2=0x564b1cbac220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:21.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.712+0000 7fb82bffc740 1 -- 172.21.15.84:0/2237699653 shutdown_connections 2023-10-16T09:49:21.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.712+0000 7fb82bffc740 1 -- 172.21.15.84:0/2237699653 wait complete. 2023-10-16T09:49:21.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.712+0000 7fb82bffc740 20 client.4698 trim_cache size 0 max 16384 2023-10-16T09:49:21.724 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T09:49:21.724 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "63fb4d68-f881-4694-b880-d10369105c0e", 2023-10-16T09:49:21.725 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-16T09:49:21.725 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 4261, 2023-10-16T09:49:21.725 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-16T09:49:21.725 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-16T09:49:21.725 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-16T09:49:21.725 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 18.820226886, 2023-10-16T09:49:21.726 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 8, 2023-10-16T09:49:21.726 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 21, 2023-10-16T09:49:21.726 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 21, 2023-10-16T09:49:21.726 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 26.665649961 2023-10-16T09:49:21.726 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T09:49:21.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.713+0000 7fb829389640 1 -- 172.21.15.84:0/3769580543 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb80406aea0 msgr2=0x7fb80406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:21.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.713+0000 7fb829389640 1 --2- 172.21.15.84:0/3769580543 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb80406aea0 0x7fb80406d360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb814007fd0 tx=0x7fb814007920 comp rx=0 tx=0).stop 2023-10-16T09:49:21.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.713+0000 7fb829389640 1 -- 172.21.15.84:0/3769580543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb82415d150 msgr2=0x7fb8241f5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:21.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.713+0000 7fb829389640 1 --2- 172.21.15.84:0/3769580543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb82415d150 0x7fb8241f5d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009d90 tx=0x7fb81c004a40 comp rx=0 tx=0).stop 2023-10-16T09:49:21.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.713+0000 7fb829389640 1 -- 172.21.15.84:0/3769580543 shutdown_connections 2023-10-16T09:49:21.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.713+0000 7fb829389640 1 --2- 172.21.15.84:0/3769580543 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb80406aea0 0x7fb80406d360 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.713+0000 7fb829389640 1 --2- 172.21.15.84:0/3769580543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb82415d150 0x7fb8241f5d40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.713+0000 7fb829389640 1 --2- 172.21.15.84:0/3769580543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8240ad350 0x7fb824158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.713+0000 7fb829389640 1 --2- 172.21.15.84:0/3769580543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb82406a2e0 0x7fb8241f3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:21.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.713+0000 7fb829389640 1 -- 172.21.15.84:0/3769580543 >> 172.21.15.84:0/3769580543 conn(0x7fb8240ce9b0 msgr2=0x7fb824159fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:21.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.713+0000 7fb829389640 1 -- 172.21.15.84:0/3769580543 shutdown_connections 2023-10-16T09:49:21.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:21.713+0000 7fb829389640 1 -- 172.21.15.84:0/3769580543 wait complete. 2023-10-16T09:49:21.731 INFO:teuthology.run_tasks:Running task kclient... 2023-10-16T09:49:21.754 INFO:tasks.kclient:Mounting kernel clients... 2023-10-16T09:49:21.754 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-16T09:49:21.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:21.755 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T09:49:21.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:21.813 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link delete ceph-brx 2023-10-16T09:49:21.917 INFO:teuthology.orchestra.run.smithi084.stderr:Cannot find device "ceph-brx" 2023-10-16T09:49:21.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T09:49:21.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:21.921 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T09:49:21.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:21.976 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link delete ceph-brx 2023-10-16T09:49:22.071 INFO:teuthology.orchestra.run.smithi084.stderr:Cannot find device "ceph-brx" 2023-10-16T09:49:22.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T09:49:22.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:22.075 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-10-16T09:49:22.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:22.134 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2023-10-16T09:49:22.235 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2023-10-16T09:49:22.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T09:49:22.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:22.240 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-10-16T09:49:22.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:22.293 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2023-10-16T09:49:22.390 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2023-10-16T09:49:22.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T09:49:22.394 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-10-16T09:49:22.395 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T09:49:22.395 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T09:49:22.395 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T09:49:22.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.615+0000 7f7b8d015640 1 Processor -- start 2023-10-16T09:49:22.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.615+0000 7f7b8d015640 1 -- start start 2023-10-16T09:49:22.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.616+0000 7f7b8d015640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7b88159f00 0x7f7b880b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:22.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.617+0000 7f7b8d015640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7b8815d880 0x7f7b880b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:22.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.617+0000 7f7b8d015640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b8806f7f0 con 0x7f7b88159b20 2023-10-16T09:49:22.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.617+0000 7f7b8d015640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b8806c5e0 con 0x7f7b8815d880 2023-10-16T09:49:22.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.617+0000 7f7b8d015640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b8806c8c0 con 0x7f7b88159f00 2023-10-16T09:49:22.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.617+0000 7f7b8759e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7b8815d880 0x7f7b880b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:22.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.617+0000 7f7b8759e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7b8815d880 0x7f7b880b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34084/0 (socket says 172.21.15.84:34084) 2023-10-16T09:49:22.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.617+0000 7f7b8759e640 1 -- 172.21.15.84:0/2666513546 learned_addr learned my addr 172.21.15.84:0/2666513546 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:22.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.617+0000 7f7b8659c640 1 --2- 172.21.15.84:0/2666513546 >> v2:172.21.15.157:3300/0 conn(0x7f7b88159f00 0x7f7b880b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:22.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.617+0000 7f7b8759e640 1 -- 172.21.15.84:0/2666513546 >> v2:172.21.15.157:3300/0 conn(0x7f7b88159f00 msgr2=0x7f7b880b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:22.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.617+0000 7f7b8759e640 1 --2- 172.21.15.84:0/2666513546 >> v2:172.21.15.157:3300/0 conn(0x7f7b88159f00 0x7f7b880b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:22.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.617+0000 7f7b8759e640 1 -- 172.21.15.84:0/2666513546 >> v1:172.21.15.157:6789/0 conn(0x7f7b88159b20 legacy=0x7f7b8806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:22.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.617+0000 7f7b8759e640 1 -- 172.21.15.84:0/2666513546 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8806cba0 con 0x7f7b8815d880 2023-10-16T09:49:22.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.617+0000 7f7b8659c640 1 --2- 172.21.15.84:0/2666513546 >> v2:172.21.15.157:3300/0 conn(0x7f7b88159f00 0x7f7b880b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:22.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.618+0000 7f7b8759e640 1 --2- 172.21.15.84:0/2666513546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b8815d880 0x7f7b880b0950 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7b8815d770 tx=0x7f7b8002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50cc97738ca53b5d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:22.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.619+0000 7f7b85d9b640 1 -- 172.21.15.84:0/2666513546 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8002ca80 con 0x7f7b8815d880 2023-10-16T09:49:22.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.619+0000 7f7b85d9b640 1 -- 172.21.15.84:0/2666513546 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8002cbe0 con 0x7f7b8815d880 2023-10-16T09:49:22.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.619+0000 7f7b85d9b640 1 -- 172.21.15.84:0/2666513546 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b800343f0 con 0x7f7b8815d880 2023-10-16T09:49:22.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.619+0000 7f7b8d015640 1 -- 172.21.15.84:0/2666513546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b8815d880 msgr2=0x7f7b880b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:22.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.619+0000 7f7b8d015640 1 --2- 172.21.15.84:0/2666513546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b8815d880 0x7f7b880b0950 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7b8815d770 tx=0x7f7b8002bee0 comp rx=0 tx=0).stop 2023-10-16T09:49:22.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.619+0000 7f7b8d015640 1 -- 172.21.15.84:0/2666513546 shutdown_connections 2023-10-16T09:49:22.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.619+0000 7f7b8d015640 1 --2- 172.21.15.84:0/2666513546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b8815d880 0x7f7b880b0950 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:22.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.619+0000 7f7b8d015640 1 --2- 172.21.15.84:0/2666513546 >> v2:172.21.15.157:3300/0 conn(0x7f7b88159f00 0x7f7b880b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:22.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.619+0000 7f7b8d015640 1 -- 172.21.15.84:0/2666513546 >> 172.21.15.84:0/2666513546 conn(0x7f7b880cb410 msgr2=0x7f7b880cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:22.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.620+0000 7f7b8d015640 1 -- 172.21.15.84:0/2666513546 shutdown_connections 2023-10-16T09:49:22.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.620+0000 7f7b8d015640 1 -- 172.21.15.84:0/2666513546 wait complete. 2023-10-16T09:49:22.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.621+0000 7f7b8d015640 1 Processor -- start 2023-10-16T09:49:22.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.621+0000 7f7b8d015640 1 -- start start 2023-10-16T09:49:22.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.622+0000 7f7b8d015640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b88159b20 0x7f7b8815a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:22.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.622+0000 7f7b8d015640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b88159f00 0x7f7b8815c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:22.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.622+0000 7f7b8d015640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b8815d880 0x7f7b880c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:22.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.622+0000 7f7b8d015640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8815acc0 con 0x7f7b88159f00 2023-10-16T09:49:22.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.622+0000 7f7b8d015640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b880c5db0 con 0x7f7b88159b20 2023-10-16T09:49:22.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.622+0000 7f7b8d015640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b880c6090 con 0x7f7b8815d880 2023-10-16T09:49:22.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.623+0000 7f7b8659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b88159f00 0x7f7b8815c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:22.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.623+0000 7f7b8659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b88159f00 0x7f7b8815c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34090/0 (socket says 172.21.15.84:34090) 2023-10-16T09:49:22.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.623+0000 7f7b8659c640 1 -- 172.21.15.84:0/4285275038 learned_addr learned my addr 172.21.15.84:0/4285275038 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:22.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.623+0000 7f7b86d9d640 1 --2- 172.21.15.84:0/4285275038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b88159b20 0x7f7b8815a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:22.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.623+0000 7f7b8659c640 1 -- 172.21.15.84:0/4285275038 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b8815d880 msgr2=0x7f7b880c6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:22.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.623+0000 7f7b8659c640 1 --2- 172.21.15.84:0/4285275038 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b8815d880 0x7f7b880c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:22.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.623+0000 7f7b8659c640 1 -- 172.21.15.84:0/4285275038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b88159b20 msgr2=0x7f7b8815a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:22.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.623+0000 7f7b8659c640 1 --2- 172.21.15.84:0/4285275038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b88159b20 0x7f7b8815a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:22.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.623+0000 7f7b8659c640 1 -- 172.21.15.84:0/4285275038 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b80009b30 con 0x7f7b88159f00 2023-10-16T09:49:22.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.623+0000 7f7b86d9d640 1 --2- 172.21.15.84:0/4285275038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b88159b20 0x7f7b8815a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:22.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.623+0000 7f7b8659c640 1 --2- 172.21.15.84:0/4285275038 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b88159f00 0x7f7b8815c310 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00c7c0 tx=0x7f7b7c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:22.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.624+0000 7f7b6f7fe640 1 -- 172.21.15.84:0/4285275038 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c007820 con 0x7f7b88159f00 2023-10-16T09:49:22.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.624+0000 7f7b6f7fe640 1 -- 172.21.15.84:0/4285275038 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7c007980 con 0x7f7b88159f00 2023-10-16T09:49:22.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.624+0000 7f7b8d015640 1 -- 172.21.15.84:0/4285275038 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b880c6370 con 0x7f7b88159f00 2023-10-16T09:49:22.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.624+0000 7f7b6f7fe640 1 -- 172.21.15.84:0/4285275038 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c007d10 con 0x7f7b88159f00 2023-10-16T09:49:22.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.624+0000 7f7b8d015640 1 -- 172.21.15.84:0/4285275038 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b880c6930 con 0x7f7b88159f00 2023-10-16T09:49:22.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.625+0000 7f7b8d015640 1 -- 172.21.15.84:0/4285275038 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b8806c270 con 0x7f7b88159f00 2023-10-16T09:49:22.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.626+0000 7f7b6f7fe640 1 -- 172.21.15.84:0/4285275038 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7b7c0113f0 con 0x7f7b88159f00 2023-10-16T09:49:22.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.627+0000 7f7b6f7fe640 1 --2- 172.21.15.84:0/4285275038 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7b7406aea0 0x7f7b7406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:22.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.627+0000 7f7b6f7fe640 1 -- 172.21.15.84:0/4285275038 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b7c089790 con 0x7f7b88159f00 2023-10-16T09:49:22.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.627+0000 7f7b86d9d640 1 --2- 172.21.15.84:0/4285275038 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7b7406aea0 0x7f7b7406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:22.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.628+0000 7f7b86d9d640 1 --2- 172.21.15.84:0/4285275038 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7b7406aea0 0x7f7b7406d360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800a830 tx=0x7f7b78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:22.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.632+0000 7f7b6f7fe640 1 -- 172.21.15.84:0/4285275038 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7b7c0547b0 con 0x7f7b88159f00 2023-10-16T09:49:22.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.863+0000 7f7b8d015640 1 -- 172.21.15.84:0/4285275038 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7b8804e160 con 0x7f7b88159f00 2023-10-16T09:49:22.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.864+0000 7f7b6f7fe640 1 -- 172.21.15.84:0/4285275038 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7b7c01f070 con 0x7f7b88159f00 2023-10-16T09:49:22.866 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T09:49:22.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.867+0000 7f7b8d015640 1 -- 172.21.15.84:0/4285275038 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7b7406aea0 msgr2=0x7f7b7406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:22.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.867+0000 7f7b8d015640 1 --2- 172.21.15.84:0/4285275038 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7b7406aea0 0x7f7b7406d360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800a830 tx=0x7f7b78008040 comp rx=0 tx=0).stop 2023-10-16T09:49:22.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.867+0000 7f7b8d015640 1 -- 172.21.15.84:0/4285275038 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b88159f00 msgr2=0x7f7b8815c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:22.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.867+0000 7f7b8d015640 1 --2- 172.21.15.84:0/4285275038 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b88159f00 0x7f7b8815c310 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00c7c0 tx=0x7f7b7c00cc80 comp rx=0 tx=0).stop 2023-10-16T09:49:22.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.867+0000 7f7b8d015640 1 -- 172.21.15.84:0/4285275038 shutdown_connections 2023-10-16T09:49:22.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.867+0000 7f7b8d015640 1 --2- 172.21.15.84:0/4285275038 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7b7406aea0 0x7f7b7406d360 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:22.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.867+0000 7f7b8d015640 1 --2- 172.21.15.84:0/4285275038 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b8815d880 0x7f7b880c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:22.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.867+0000 7f7b8d015640 1 --2- 172.21.15.84:0/4285275038 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b88159f00 0x7f7b8815c310 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:22.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.867+0000 7f7b8d015640 1 --2- 172.21.15.84:0/4285275038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b88159b20 0x7f7b8815a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:22.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.867+0000 7f7b8d015640 1 -- 172.21.15.84:0/4285275038 >> 172.21.15.84:0/4285275038 conn(0x7f7b880cb410 msgr2=0x7f7b880cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:22.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.867+0000 7f7b8d015640 1 -- 172.21.15.84:0/4285275038 shutdown_connections 2023-10-16T09:49:22.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:22.867+0000 7f7b8d015640 1 -- 172.21.15.84:0/4285275038 wait complete. 2023-10-16T09:49:22.887 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T09:49:22.887 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T09:49:22.887 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-16T09:49:22.888 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T09:49:22.888 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T09:49:22.888 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T09:49:22.889 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T09:49:22.889 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T09:49:22.889 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T09:49:22.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:22.890 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-16T09:49:22.940 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:49:22.941 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:49:22.941 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:49:22.941 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:22.941 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T09:49:22.941 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:22.942 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:49:22.942 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:49:22.942 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-16T09:49:22.942 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:49:22.942 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:49:22.943 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-16T09:49:22.943 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:49:22.943 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:49:22.943 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T09:49:22.943 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:49:22.944 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T09:49:22.944 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42388sec preferred_lft 42388sec 2023-10-16T09:49:22.944 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-16T09:49:22.944 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:22.945 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-16T09:49:22.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:22.946 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T09:49:22.946 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T09:49:22.946 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link add name ceph-brx type bridge 2023-10-16T09:49:22.947 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip addr flush dev ceph-brx 2023-10-16T09:49:22.947 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set ceph-brx up 2023-10-16T09:49:22.947 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-16T09:49:22.947 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T09:49:23.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:23.394 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T09:49:23.482 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-16T09:49:23.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:23.486 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-16T09:49:23.536 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-16T09:49:23.536 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T09:49:23.536 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T09:49:23.536 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T09:49:23.536 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T09:49:23.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:23.537 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T09:49:23.537 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T09:49:23.538 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T09:49:23.538 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T09:49:23.538 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T09:49:23.539 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T09:49:24.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:24.241 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T09:49:24.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:24.299 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list-id 2023-10-16T09:49:24.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:24.356 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T09:49:24.356 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T09:49:24.356 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T09:49:24.357 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-16T09:49:24.357 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T09:49:24.502 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-16T09:49:24.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:24.503 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T09:49:24.503 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T09:49:24.503 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-16T09:49:24.503 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-16T09:49:24.504 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-16T09:49:24.504 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-16T09:49:24.504 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-16T09:49:24.505 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T09:49:24.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:24.783 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T09:49:24.783 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T09:49:24.783 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-16T09:49:24.784 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set dev brx.0 master ceph-brx 2023-10-16T09:49:24.784 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T09:49:24.943 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:49:24.943 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T09:49:24.988 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T09:49:24.989 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:49:24.989 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T09:49:25.086 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-16T09:49:25.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:25.087 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T09:49:25.238 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T09:49:25.238 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T09:49:25.680 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T09:49:25.680 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T09:49:25.681 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T09:49:25.681 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with new device syntax: 0@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T09:49:25.681 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T09:49:25.683 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-16T09:49:25.683 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2023-10-16T09:49:25.683 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T09:49:25.683 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T09:49:25.684 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T09:49:25.686 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T09:49:25.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:25.687 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T09:49:25.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:25.774 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:49:25.848 INFO:teuthology.orchestra.run.smithi084.stdout:client4552 2023-10-16T09:49:25.849 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:49:25.849 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:49:25.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:25.907 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:49:25.982 INFO:teuthology.orchestra.run.smithi084.stdout:client4552 2023-10-16T09:49:25.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:25.983 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:49:26.063 INFO:teuthology.orchestra.run.smithi084.stdout:client4552 2023-10-16T09:49:26.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:26.064 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/63fb4d68-f881-4694-b880-d10369105c0e.client4552/status') 2023-10-16T09:49:26.187 INFO:teuthology.orchestra.run.smithi084.stdout:instance: client.4552 (3)172.21.15.84:0/284188939 2023-10-16T09:49:26.188 INFO:teuthology.orchestra.run.smithi084.stdout:blocklisted: false 2023-10-16T09:49:26.188 INFO:teuthology.orchestra.run.smithi084.stderr:0+1 records in 2023-10-16T09:49:26.188 INFO:teuthology.orchestra.run.smithi084.stderr:0+1 records out 2023-10-16T09:49:26.188 INFO:teuthology.orchestra.run.smithi084.stderr:69 bytes copied, 9.3472e-05 s, 738 kB/s 2023-10-16T09:49:26.192 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-10-16T09:49:26.193 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T09:49:26.193 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-16T09:49:26.193 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T09:49:26.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.407+0000 7fe60cc61640 1 Processor -- start 2023-10-16T09:49:26.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.407+0000 7fe60cc61640 1 -- start start 2023-10-16T09:49:26.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.409+0000 7fe60cc61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6080cd0c0 0x7fe6081615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:26.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.409+0000 7fe60cc61640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe60815e3e0 con 0x7fe60815db00 2023-10-16T09:49:26.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.409+0000 7fe60cc61640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6080cd4a0 con 0x7fe6080b0410 2023-10-16T09:49:26.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.409+0000 7fe60cc61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60815e100 con 0x7fe6080cd0c0 2023-10-16T09:49:26.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.409+0000 7fe606d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe60815db00 0x7fe60806f550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33514/0 (socket says 172.21.15.84:33514) 2023-10-16T09:49:26.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.409+0000 7fe606d9d640 1 -- 172.21.15.84:0/2488129693 learned_addr learned my addr 172.21.15.84:0/2488129693 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:26.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.410+0000 7fe60659c640 1 --2- 172.21.15.84:0/2488129693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6080cd0c0 0x7fe6081615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:26.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.410+0000 7fe60559a640 1 -- 172.21.15.84:0/2488129693 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276762336 0 0) 0x7fe60815e3e0 con 0x7fe60815db00 2023-10-16T09:49:26.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.410+0000 7fe60559a640 1 -- 172.21.15.84:0/2488129693 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5f4003520 con 0x7fe60815db00 2023-10-16T09:49:26.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.410+0000 7fe60559a640 1 -- 172.21.15.84:0/2488129693 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3909092258 0 0) 0x7fe5f4003520 con 0x7fe60815db00 2023-10-16T09:49:26.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.410+0000 7fe60559a640 1 -- 172.21.15.84:0/2488129693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6080cd0c0 msgr2=0x7fe6081615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:26.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.410+0000 7fe60559a640 1 --2- 172.21.15.84:0/2488129693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6080cd0c0 0x7fe6081615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:26.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.410+0000 7fe60559a640 1 -- 172.21.15.84:0/2488129693 >> v1:172.21.15.157:6789/0 conn(0x7fe6080b0410 legacy=0x7fe60815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:26.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.410+0000 7fe60559a640 1 -- 172.21.15.84:0/2488129693 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe608160b10 con 0x7fe60815db00 2023-10-16T09:49:26.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.410+0000 7fe60559a640 1 -- 172.21.15.84:0/2488129693 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fe6000019f0 con 0x7fe60815db00 2023-10-16T09:49:26.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.410+0000 7fe60659c640 1 --2- 172.21.15.84:0/2488129693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6080cd0c0 0x7fe6081615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:26.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.410+0000 7fe60559a640 1 -- 172.21.15.84:0/2488129693 >> v1:172.21.15.84:6789/0 conn(0x7fe60815db00 legacy=0x7fe60806f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:26.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.410+0000 7fe60559a640 1 --2- 172.21.15.84:0/2488129693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6080cd0c0 0x7fe5f40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:26.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.410+0000 7fe60559a640 1 --2- 172.21.15.84:0/2488129693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f4003f90 0x7fe5f40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:26.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.410+0000 7fe60559a640 1 --2- 172.21.15.84:0/2488129693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f4004d00 0x7fe5f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:26.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.410+0000 7fe60559a640 1 -- 172.21.15.84:0/2488129693 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60815e100 con 0x7fe5f4003f90 2023-10-16T09:49:26.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.410+0000 7fe60559a640 1 -- 172.21.15.84:0/2488129693 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f400bb70 con 0x7fe5f4004d00 2023-10-16T09:49:26.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.410+0000 7fe60559a640 1 -- 172.21.15.84:0/2488129693 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f400bd30 con 0x7fe6080cd0c0 2023-10-16T09:49:26.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.410+0000 7fe605d9b640 1 --2- 172.21.15.84:0/2488129693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f4003f90 0x7fe5f40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:26.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.411+0000 7fe60659c640 1 --2- 172.21.15.84:0/2488129693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6080cd0c0 0x7fe5f40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:26.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.411+0000 7fe606d9d640 1 --2- 172.21.15.84:0/2488129693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f4004d00 0x7fe5f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:26.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.411+0000 7fe605d9b640 1 -- 172.21.15.84:0/2488129693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6080cd0c0 msgr2=0x7fe5f40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:26.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.411+0000 7fe605d9b640 1 --2- 172.21.15.84:0/2488129693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6080cd0c0 0x7fe5f40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:26.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.411+0000 7fe605d9b640 1 -- 172.21.15.84:0/2488129693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f4004d00 msgr2=0x7fe5f4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:26.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.411+0000 7fe605d9b640 1 --2- 172.21.15.84:0/2488129693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f4004d00 0x7fe5f4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:26.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.411+0000 7fe605d9b640 1 -- 172.21.15.84:0/2488129693 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f4003520 con 0x7fe5f4003f90 2023-10-16T09:49:26.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.411+0000 7fe606d9d640 1 --2- 172.21.15.84:0/2488129693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f4004d00 0x7fe5f4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:26.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.412+0000 7fe605d9b640 1 --2- 172.21.15.84:0/2488129693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f4003f90 0x7fe5f40047c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0001750 tx=0x7fe5f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a41c89720ab9a02 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:26.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.412+0000 7fe60559a640 1 -- 172.21.15.84:0/2488129693 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f00052d0 con 0x7fe5f4003f90 2023-10-16T09:49:26.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.412+0000 7fe60559a640 1 -- 172.21.15.84:0/2488129693 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f002e050 con 0x7fe5f4003f90 2023-10-16T09:49:26.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.412+0000 7fe60559a640 1 -- 172.21.15.84:0/2488129693 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0003830 con 0x7fe5f4003f90 2023-10-16T09:49:26.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.413+0000 7fe60cc61640 1 -- 172.21.15.84:0/2488129693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f4003f90 msgr2=0x7fe5f40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:26.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.413+0000 7fe60cc61640 1 --2- 172.21.15.84:0/2488129693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f4003f90 0x7fe5f40047c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0001750 tx=0x7fe5f40057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:26.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.413+0000 7fe606d9d640 1 -- 172.21.15.84:0/2488129693 reap_dead start 2023-10-16T09:49:26.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.413+0000 7fe60cc61640 1 -- 172.21.15.84:0/2488129693 shutdown_connections 2023-10-16T09:49:26.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.413+0000 7fe60cc61640 1 -- 172.21.15.84:0/2488129693 >> 172.21.15.84:0/2488129693 conn(0x7fe6080cb410 msgr2=0x7fe6080cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:26.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.413+0000 7fe60cc61640 1 -- 172.21.15.84:0/2488129693 shutdown_connections 2023-10-16T09:49:26.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.413+0000 7fe60cc61640 1 -- 172.21.15.84:0/2488129693 wait complete. 2023-10-16T09:49:26.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.415+0000 7fe60cc61640 1 Processor -- start 2023-10-16T09:49:26.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.415+0000 7fe60cc61640 1 -- start start 2023-10-16T09:49:26.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.416+0000 7fe60cc61640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f4003f90 0x7fe60814cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:26.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.416+0000 7fe60cc61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6080adab0 0x7fe6080adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:26.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.416+0000 7fe60cc61640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6080c47e0 0x7fe6080c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:26.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.416+0000 7fe60cc61640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6080c51a0 con 0x7fe5f4003f90 2023-10-16T09:49:26.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.416+0000 7fe60cc61640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6080ae620 con 0x7fe6080c47e0 2023-10-16T09:49:26.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.416+0000 7fe60cc61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6080ae900 con 0x7fe6080adab0 2023-10-16T09:49:26.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.416+0000 7fe60659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f4003f90 0x7fe60814cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:26.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.416+0000 7fe60659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f4003f90 0x7fe60814cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33934/0 (socket says 172.21.15.84:33934) 2023-10-16T09:49:26.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.416+0000 7fe60659c640 1 -- 172.21.15.84:0/1640290108 learned_addr learned my addr 172.21.15.84:0/1640290108 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:26.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.416+0000 7fe60659c640 1 -- 172.21.15.84:0/1640290108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6080adab0 msgr2=0x7fe6080adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:26.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.416+0000 7fe60659c640 1 --2- 172.21.15.84:0/1640290108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6080adab0 0x7fe6080adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:26.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.416+0000 7fe60659c640 1 -- 172.21.15.84:0/1640290108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6080c47e0 msgr2=0x7fe6080c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:26.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.416+0000 7fe60659c640 1 --2- 172.21.15.84:0/1640290108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6080c47e0 0x7fe6080c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:26.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.416+0000 7fe60659c640 1 -- 172.21.15.84:0/1640290108 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6080aebe0 con 0x7fe5f4003f90 2023-10-16T09:49:26.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.417+0000 7fe60659c640 1 --2- 172.21.15.84:0/1640290108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f4003f90 0x7fe60814cce0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8010860 tx=0x7fe5f8010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:26.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.417+0000 7fe5feffd640 1 -- 172.21.15.84:0/1640290108 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f800cb40 con 0x7fe5f4003f90 2023-10-16T09:49:26.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.417+0000 7fe5feffd640 1 -- 172.21.15.84:0/1640290108 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f800cca0 con 0x7fe5f4003f90 2023-10-16T09:49:26.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.417+0000 7fe5feffd640 1 -- 172.21.15.84:0/1640290108 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8015690 con 0x7fe5f4003f90 2023-10-16T09:49:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.417+0000 7fe60cc61640 1 -- 172.21.15.84:0/1640290108 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6080aeec0 con 0x7fe5f4003f90 2023-10-16T09:49:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.417+0000 7fe60cc61640 1 -- 172.21.15.84:0/1640290108 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60815a4c0 con 0x7fe5f4003f90 2023-10-16T09:49:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.418+0000 7fe60cc61640 1 -- 172.21.15.84:0/1640290108 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6080aea40 con 0x7fe5f4003f90 2023-10-16T09:49:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.419+0000 7fe5feffd640 1 -- 172.21.15.84:0/1640290108 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe5f80157f0 con 0x7fe5f4003f90 2023-10-16T09:49:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.421+0000 7fe5feffd640 1 --2- 172.21.15.84:0/1640290108 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe5e806ae70 0x7fe5e806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:26.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.421+0000 7fe5feffd640 1 -- 172.21.15.84:0/1640290108 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5f8089c90 con 0x7fe5f4003f90 2023-10-16T09:49:26.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.421+0000 7fe605d9b640 1 --2- 172.21.15.84:0/1640290108 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe5e806ae70 0x7fe5e806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:26.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.422+0000 7fe605d9b640 1 --2- 172.21.15.84:0/1640290108 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe5e806ae70 0x7fe5e806d330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0033000 tx=0x7fe5f002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:26.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.425+0000 7fe5feffd640 1 -- 172.21.15.84:0/1640290108 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe6080aea40 con 0x7fe5f4003f90 2023-10-16T09:49:26.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.665+0000 7fe60cc61640 1 -- 172.21.15.84:0/1640290108 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe6080aea40 con 0x7fe5f4003f90 2023-10-16T09:49:26.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.665+0000 7fe5feffd640 1 -- 172.21.15.84:0/1640290108 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe6080aea40 con 0x7fe5f4003f90 2023-10-16T09:49:26.667 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T09:49:26.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.668+0000 7fe60cc61640 1 -- 172.21.15.84:0/1640290108 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe5e806ae70 msgr2=0x7fe5e806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:26.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.668+0000 7fe60cc61640 1 --2- 172.21.15.84:0/1640290108 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe5e806ae70 0x7fe5e806d330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0033000 tx=0x7fe5f002f000 comp rx=0 tx=0).stop 2023-10-16T09:49:26.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.668+0000 7fe60cc61640 1 -- 172.21.15.84:0/1640290108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f4003f90 msgr2=0x7fe60814cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:26.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.668+0000 7fe60cc61640 1 --2- 172.21.15.84:0/1640290108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f4003f90 0x7fe60814cce0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8010860 tx=0x7fe5f8010d20 comp rx=0 tx=0).stop 2023-10-16T09:49:26.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.668+0000 7fe60cc61640 1 -- 172.21.15.84:0/1640290108 shutdown_connections 2023-10-16T09:49:26.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.668+0000 7fe60cc61640 1 --2- 172.21.15.84:0/1640290108 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe5e806ae70 0x7fe5e806d330 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:26.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.668+0000 7fe60cc61640 1 --2- 172.21.15.84:0/1640290108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6080c47e0 0x7fe6080c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:26.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.668+0000 7fe60cc61640 1 --2- 172.21.15.84:0/1640290108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6080adab0 0x7fe6080adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:26.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.668+0000 7fe60cc61640 1 --2- 172.21.15.84:0/1640290108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f4003f90 0x7fe60814cce0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:26.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.668+0000 7fe60cc61640 1 -- 172.21.15.84:0/1640290108 >> 172.21.15.84:0/1640290108 conn(0x7fe6080cb410 msgr2=0x7fe60815b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:26.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.668+0000 7fe60cc61640 1 -- 172.21.15.84:0/1640290108 shutdown_connections 2023-10-16T09:49:26.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:26.668+0000 7fe60cc61640 1 -- 172.21.15.84:0/1640290108 wait complete. 2023-10-16T09:49:26.687 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T09:49:26.687 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T09:49:26.687 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-16T09:49:26.687 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-16T09:49:26.688 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-16T09:49:26.688 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T09:49:26.688 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T09:49:26.688 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T09:49:26.688 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-16T09:49:26.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:26.689 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-16T09:49:26.737 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:49:26.737 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:49:26.737 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:49:26.737 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:26.738 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T09:49:26.738 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:26.738 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:49:26.738 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:49:26.738 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-16T09:49:26.739 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:49:26.739 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:49:26.739 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-16T09:49:26.739 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:49:26.739 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:49:26.740 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T09:49:26.740 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:49:26.740 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T09:49:26.740 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42384sec preferred_lft 42384sec 2023-10-16T09:49:26.740 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-16T09:49:26.740 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:26.741 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T09:49:26.741 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:49:26.741 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T09:49:26.741 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:26.741 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::20a8:eaff:fe9d:5cea/64 scope link 2023-10-16T09:49:26.742 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:26.742 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T09:49:26.742 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T09:49:26.742 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1854:13ff:fe47:be92/64 scope link 2023-10-16T09:49:26.742 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:26.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:26.743 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T09:49:26.870 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-16T09:49:26.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:26.874 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-16T09:49:26.929 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-16T09:49:26.929 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T09:49:26.929 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T09:49:26.929 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T09:49:26.930 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T09:49:26.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:26.931 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T09:49:26.931 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T09:49:26.932 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T09:49:26.932 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T09:49:26.932 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T09:49:26.932 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T09:49:27.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:27.113 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T09:49:27.163 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T09:49:27.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:27.165 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list-id 2023-10-16T09:49:27.216 INFO:teuthology.orchestra.run.smithi084.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-16T09:49:27.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:27.218 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T09:49:27.218 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T09:49:27.218 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T09:49:27.218 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-16T09:49:27.219 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T09:49:27.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:27.363 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-16T09:49:27.455 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:49:27.455 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:49:27.455 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:49:27.455 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:27.456 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T09:49:27.456 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:27.456 INFO:teuthology.orchestra.run.smithi084.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T09:49:27.456 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 52:8d:1f:7e:34:de brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T09:49:27.456 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T09:49:27.457 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:27.457 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::508d:1fff:fe7e:34de/64 scope link 2023-10-16T09:49:27.457 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:27.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:27.459 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-16T09:49:27.555 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:49:27.556 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:49:27.556 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:49:27.556 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:27.556 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T09:49:27.556 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:27.556 INFO:teuthology.orchestra.run.smithi084.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T09:49:27.557 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 52:8d:1f:7e:34:de brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T09:49:27.557 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T09:49:27.557 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:27.557 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::508d:1fff:fe7e:34de/64 scope link 2023-10-16T09:49:27.557 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:27.560 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-16T09:49:27.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:27.561 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T09:49:27.561 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T09:49:27.561 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-16T09:49:27.562 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-16T09:49:27.562 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-16T09:49:27.562 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-16T09:49:27.562 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-16T09:49:27.563 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T09:49:27.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:27.838 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T09:49:27.838 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T09:49:27.838 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.1 up 2023-10-16T09:49:27.839 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set dev brx.1 master ceph-brx 2023-10-16T09:49:27.839 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T09:49:28.003 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:49:28.003 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-16T09:49:28.053 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-16T09:49:28.053 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:49:28.053 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-16T09:49:28.100 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-10-16T09:49:28.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:28.101 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T09:49:28.238 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T09:49:28.238 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T09:49:28.274 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T09:49:28.274 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T09:49:28.274 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T09:49:28.274 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with new device syntax: 2@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T09:49:28.274 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T09:49:28.276 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2023-10-16T09:49:28.276 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2023-10-16T09:49:28.276 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T09:49:28.276 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T09:49:28.276 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T09:49:28.279 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T09:49:28.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:28.280 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-16T09:49:28.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:28.358 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:49:28.411 INFO:teuthology.orchestra.run.smithi084.stdout:client4469 2023-10-16T09:49:28.411 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:49:28.412 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:49:28.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:28.454 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:49:28.552 INFO:teuthology.orchestra.run.smithi084.stdout:client4469 2023-10-16T09:49:28.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:28.553 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:49:28.607 INFO:teuthology.orchestra.run.smithi084.stdout:client4469 2023-10-16T09:49:28.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:28.607 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/63fb4d68-f881-4694-b880-d10369105c0e.client4469/status') 2023-10-16T09:49:28.688 INFO:teuthology.orchestra.run.smithi084.stdout:instance: client.4469 (3)172.21.15.84:0/3090410770 2023-10-16T09:49:28.688 INFO:teuthology.orchestra.run.smithi084.stdout:blocklisted: false 2023-10-16T09:49:28.689 INFO:teuthology.orchestra.run.smithi084.stderr:0+1 records in 2023-10-16T09:49:28.689 INFO:teuthology.orchestra.run.smithi084.stderr:0+1 records out 2023-10-16T09:49:28.689 INFO:teuthology.orchestra.run.smithi084.stderr:70 bytes copied, 7.7954e-05 s, 898 kB/s 2023-10-16T09:49:28.692 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-10-16T09:49:28.692 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T09:49:28.692 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-16T09:49:28.692 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T09:49:28.840 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:49:28.846 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:49:28.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.875+0000 7fd255dc9640 1 Processor -- start 2023-10-16T09:49:28.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.875+0000 7fd255dc9640 1 -- start start 2023-10-16T09:49:28.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.876+0000 7fd255dc9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd25015aaa0 0x7fd25015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:28.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.876+0000 7fd255dc9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd25015ed60 con 0x7fd25015ae80 2023-10-16T09:49:28.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.877+0000 7fd255dc9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2500707e0 con 0x7fd2500ce390 2023-10-16T09:49:28.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.877+0000 7fd255dc9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25006b810 con 0x7fd25015aaa0 2023-10-16T09:49:28.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.877+0000 7fd246ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd25015ae80 0x7fd2500cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33526/0 (socket says 172.21.15.84:33526) 2023-10-16T09:49:28.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.877+0000 7fd246ffd640 1 -- 172.21.15.84:0/3041706472 learned_addr learned my addr 172.21.15.84:0/3041706472 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:28.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.877+0000 7fd24f7fe640 1 --2- 172.21.15.84:0/3041706472 >> v2:172.21.15.157:3300/0 conn(0x7fd25015aaa0 0x7fd25015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:28.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.877+0000 7fd24effd640 1 -- 172.21.15.84:0/3041706472 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322226718 0 0) 0x7fd25015ed60 con 0x7fd25015ae80 2023-10-16T09:49:28.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.877+0000 7fd24f7fe640 1 -- 172.21.15.84:0/3041706472 >> v1:172.21.15.157:6789/0 conn(0x7fd2500ce390 legacy=0x7fd250070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:28.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.877+0000 7fd24f7fe640 1 -- 172.21.15.84:0/3041706472 >> v1:172.21.15.84:6789/0 conn(0x7fd25015ae80 legacy=0x7fd2500cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:28.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.877+0000 7fd24f7fe640 1 -- 172.21.15.84:0/3041706472 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25015ef90 con 0x7fd25015aaa0 2023-10-16T09:49:28.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.879+0000 7fd24f7fe640 1 --2- 172.21.15.84:0/3041706472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015aaa0 0x7fd25015e820 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd240009d90 tx=0x7fd24002bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab2acc57e0f2d634 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:28.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.879+0000 7fd24effd640 1 -- 172.21.15.84:0/3041706472 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24002cd00 con 0x7fd25015aaa0 2023-10-16T09:49:28.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.879+0000 7fd24effd640 1 -- 172.21.15.84:0/3041706472 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24002ce60 con 0x7fd25015aaa0 2023-10-16T09:49:28.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.879+0000 7fd24effd640 1 -- 172.21.15.84:0/3041706472 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd240034660 con 0x7fd25015aaa0 2023-10-16T09:49:28.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.880+0000 7fd255dc9640 1 -- 172.21.15.84:0/3041706472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015aaa0 msgr2=0x7fd25015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:28.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.880+0000 7fd255dc9640 1 --2- 172.21.15.84:0/3041706472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015aaa0 0x7fd25015e820 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd240009d90 tx=0x7fd24002bf50 comp rx=0 tx=0).stop 2023-10-16T09:49:28.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.880+0000 7fd255dc9640 1 -- 172.21.15.84:0/3041706472 shutdown_connections 2023-10-16T09:49:28.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.880+0000 7fd255dc9640 1 --2- 172.21.15.84:0/3041706472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015aaa0 0x7fd25015e820 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:28.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.880+0000 7fd255dc9640 1 -- 172.21.15.84:0/3041706472 >> 172.21.15.84:0/3041706472 conn(0x7fd2500cb410 msgr2=0x7fd2500cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:28.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.880+0000 7fd255dc9640 1 -- 172.21.15.84:0/3041706472 shutdown_connections 2023-10-16T09:49:28.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.880+0000 7fd255dc9640 1 -- 172.21.15.84:0/3041706472 wait complete. 2023-10-16T09:49:28.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.882+0000 7fd255dc9640 1 Processor -- start 2023-10-16T09:49:28.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.882+0000 7fd255dc9640 1 -- start start 2023-10-16T09:49:28.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.883+0000 7fd255dc9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2500ce390 0x7fd25015cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:28.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.883+0000 7fd255dc9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd25015aaa0 0x7fd25015b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:28.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.883+0000 7fd255dc9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015ae80 0x7fd2500c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:28.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.883+0000 7fd255dc9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25015bd00 con 0x7fd25015aaa0 2023-10-16T09:49:28.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.883+0000 7fd255dc9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25015bfe0 con 0x7fd25015ae80 2023-10-16T09:49:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.883+0000 7fd255dc9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25015c2c0 con 0x7fd2500ce390 2023-10-16T09:49:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.883+0000 7fd246ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd25015aaa0 0x7fd25015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.884+0000 7fd246ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd25015aaa0 0x7fd25015b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33940/0 (socket says 172.21.15.84:33940) 2023-10-16T09:49:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.884+0000 7fd246ffd640 1 -- 172.21.15.84:0/1045006937 learned_addr learned my addr 172.21.15.84:0/1045006937 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.884+0000 7fd24f7fe640 1 --2- 172.21.15.84:0/1045006937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2500ce390 0x7fd25015cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.884+0000 7fd24ffff640 1 --2- 172.21.15.84:0/1045006937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015ae80 0x7fd2500c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.884+0000 7fd246ffd640 1 -- 172.21.15.84:0/1045006937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2500ce390 msgr2=0x7fd25015cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.884+0000 7fd246ffd640 1 --2- 172.21.15.84:0/1045006937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2500ce390 0x7fd25015cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.884+0000 7fd246ffd640 1 -- 172.21.15.84:0/1045006937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015ae80 msgr2=0x7fd2500c44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.884+0000 7fd246ffd640 1 --2- 172.21.15.84:0/1045006937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015ae80 0x7fd2500c44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.884+0000 7fd246ffd640 1 -- 172.21.15.84:0/1045006937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd240009a70 con 0x7fd25015aaa0 2023-10-16T09:49:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.884+0000 7fd24ffff640 1 --2- 172.21.15.84:0/1045006937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015ae80 0x7fd2500c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.884+0000 7fd246ffd640 1 --2- 172.21.15.84:0/1045006937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd25015aaa0 0x7fd25015b320 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd23800c3c0 tx=0x7fd23800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.885+0000 7fd24cff9640 1 -- 172.21.15.84:0/1045006937 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2380074e0 con 0x7fd25015aaa0 2023-10-16T09:49:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.885+0000 7fd24cff9640 1 -- 172.21.15.84:0/1045006937 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd238007640 con 0x7fd25015aaa0 2023-10-16T09:49:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.885+0000 7fd255dc9640 1 -- 172.21.15.84:0/1045006937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25015c5a0 con 0x7fd25015aaa0 2023-10-16T09:49:28.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.885+0000 7fd24cff9640 1 -- 172.21.15.84:0/1045006937 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238004a20 con 0x7fd25015aaa0 2023-10-16T09:49:28.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.885+0000 7fd255dc9640 1 -- 172.21.15.84:0/1045006937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd250157ee0 con 0x7fd25015aaa0 2023-10-16T09:49:28.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.886+0000 7fd255dc9640 1 -- 172.21.15.84:0/1045006937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd250158100 con 0x7fd25015aaa0 2023-10-16T09:49:28.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.887+0000 7fd24cff9640 1 -- 172.21.15.84:0/1045006937 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd238004b80 con 0x7fd25015aaa0 2023-10-16T09:49:28.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.889+0000 7fd24cff9640 1 --2- 172.21.15.84:0/1045006937 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd23006ae70 0x7fd23006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:28.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.889+0000 7fd24cff9640 1 -- 172.21.15.84:0/1045006937 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd238089290 con 0x7fd25015aaa0 2023-10-16T09:49:28.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.889+0000 7fd24f7fe640 1 --2- 172.21.15.84:0/1045006937 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd23006ae70 0x7fd23006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:28.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.890+0000 7fd24f7fe640 1 --2- 172.21.15.84:0/1045006937 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd23006ae70 0x7fd23006d330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd240036000 tx=0x7fd240030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:28.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:28.897+0000 7fd24cff9640 1 -- 172.21.15.84:0/1045006937 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd2380543d0 con 0x7fd25015aaa0 2023-10-16T09:49:29.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:29.128+0000 7fd255dc9640 1 -- 172.21.15.84:0/1045006937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd25015cac0 con 0x7fd25015aaa0 2023-10-16T09:49:29.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:29.129+0000 7fd24cff9640 1 -- 172.21.15.84:0/1045006937 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd25015cac0 con 0x7fd25015aaa0 2023-10-16T09:49:29.131 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T09:49:29.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:29.131+0000 7fd255dc9640 1 -- 172.21.15.84:0/1045006937 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd23006ae70 msgr2=0x7fd23006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:29.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:29.131+0000 7fd255dc9640 1 --2- 172.21.15.84:0/1045006937 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd23006ae70 0x7fd23006d330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd240036000 tx=0x7fd240030000 comp rx=0 tx=0).stop 2023-10-16T09:49:29.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:29.131+0000 7fd255dc9640 1 -- 172.21.15.84:0/1045006937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd25015aaa0 msgr2=0x7fd25015b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:29.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:29.131+0000 7fd255dc9640 1 --2- 172.21.15.84:0/1045006937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd25015aaa0 0x7fd25015b320 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd23800c3c0 tx=0x7fd23800c880 comp rx=0 tx=0).stop 2023-10-16T09:49:29.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:29.131+0000 7fd255dc9640 1 -- 172.21.15.84:0/1045006937 shutdown_connections 2023-10-16T09:49:29.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:29.131+0000 7fd255dc9640 1 --2- 172.21.15.84:0/1045006937 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd23006ae70 0x7fd23006d330 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:29.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:29.131+0000 7fd255dc9640 1 --2- 172.21.15.84:0/1045006937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015ae80 0x7fd2500c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:29.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:29.131+0000 7fd255dc9640 1 --2- 172.21.15.84:0/1045006937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd25015aaa0 0x7fd25015b320 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:29.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:29.131+0000 7fd255dc9640 1 --2- 172.21.15.84:0/1045006937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2500ce390 0x7fd25015cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:29.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:29.131+0000 7fd255dc9640 1 -- 172.21.15.84:0/1045006937 >> 172.21.15.84:0/1045006937 conn(0x7fd2500cb410 msgr2=0x7fd2500cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:29.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:29.131+0000 7fd255dc9640 1 -- 172.21.15.84:0/1045006937 shutdown_connections 2023-10-16T09:49:29.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:29.132+0000 7fd255dc9640 1 -- 172.21.15.84:0/1045006937 wait complete. 2023-10-16T09:49:29.150 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T09:49:29.151 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T09:49:29.151 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-10-16T09:49:29.151 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-16T09:49:29.152 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-16T09:49:29.152 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T09:49:29.152 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T09:49:29.152 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T09:49:29.153 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-16T09:49:29.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:29.153 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-10-16T09:49:29.212 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:49:29.212 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:49:29.213 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:49:29.213 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:29.213 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-10-16T09:49:29.213 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:29.214 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:49:29.214 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:49:29.214 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-10-16T09:49:29.214 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:49:29.214 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:49:29.215 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-10-16T09:49:29.215 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:49:29.215 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:49:29.215 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T09:49:29.216 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:49:29.216 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T09:49:29.216 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42394sec preferred_lft 42394sec 2023-10-16T09:49:29.216 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2023-10-16T09:49:29.216 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:29.217 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-16T09:49:29.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:29.218 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-10-16T09:49:29.218 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-10-16T09:49:29.218 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add name ceph-brx type bridge 2023-10-16T09:49:29.219 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr flush dev ceph-brx 2023-10-16T09:49:29.219 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set ceph-brx up 2023-10-16T09:49:29.219 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-16T09:49:29.220 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-10-16T09:49:29.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:29.686 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T09:49:29.762 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-10-16T09:49:29.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:29.766 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-10-16T09:49:29.826 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-10-16T09:49:29.826 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T09:49:29.827 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T09:49:29.827 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T09:49:29.827 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T09:49:29.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:29.828 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-10-16T09:49:29.828 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-10-16T09:49:29.828 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T09:49:29.829 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T09:49:29.829 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T09:49:29.829 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-10-16T09:49:30.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:30.573 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-10-16T09:49:30.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:30.629 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2023-10-16T09:49:30.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:30.682 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-10-16T09:49:30.682 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-10-16T09:49:30.683 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T09:49:30.683 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-16T09:49:30.683 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-10-16T09:49:30.831 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-16T09:49:30.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:30.832 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-10-16T09:49:30.832 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-10-16T09:49:30.832 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-16T09:49:30.832 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-16T09:49:30.833 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-16T09:49:30.833 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-16T09:49:30.833 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-16T09:49:30.833 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-10-16T09:49:31.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:31.113 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-10-16T09:49:31.113 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-10-16T09:49:31.113 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2023-10-16T09:49:31.114 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.0 master ceph-brx 2023-10-16T09:49:31.114 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-10-16T09:49:31.274 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:49:31.274 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-16T09:49:31.321 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-16T09:49:31.321 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:49:31.321 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-16T09:49:31.418 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-16T09:49:31.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:31.419 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T09:49:31.571 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T09:49:31.571 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T09:49:32.042 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T09:49:32.042 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T09:49:32.042 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T09:49:32.043 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T09:49:32.043 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T09:49:32.044 INFO:teuthology.orchestra.run.smithi157.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-16T09:49:32.044 INFO:teuthology.orchestra.run.smithi157.stdout: You just mounted a file system that supports labels which does not 2023-10-16T09:49:32.044 INFO:teuthology.orchestra.run.smithi157.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T09:49:32.044 INFO:teuthology.orchestra.run.smithi157.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T09:49:32.045 INFO:teuthology.orchestra.run.smithi157.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T09:49:32.051 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T09:49:32.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:32.051 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-16T09:49:32.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:32.134 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:49:32.191 INFO:teuthology.orchestra.run.smithi157.stdout:client4475 2023-10-16T09:49:32.192 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:49:32.192 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:49:32.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:32.243 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:49:32.298 INFO:teuthology.orchestra.run.smithi157.stdout:client4475 2023-10-16T09:49:32.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:32.299 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:49:32.354 INFO:teuthology.orchestra.run.smithi157.stdout:client4475 2023-10-16T09:49:32.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:32.355 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/63fb4d68-f881-4694-b880-d10369105c0e.client4475/status') 2023-10-16T09:49:32.449 INFO:teuthology.orchestra.run.smithi157.stdout:instance: client.4475 (3)192.168.0.1:0/2519576531 2023-10-16T09:49:32.449 INFO:teuthology.orchestra.run.smithi157.stdout:blocklisted: false 2023-10-16T09:49:32.449 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records in 2023-10-16T09:49:32.450 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records out 2023-10-16T09:49:32.450 INFO:teuthology.orchestra.run.smithi157.stderr:69 bytes copied, 7.5545e-05 s, 913 kB/s 2023-10-16T09:49:32.453 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-10-16T09:49:32.453 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T09:49:32.454 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-16T09:49:32.454 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T09:49:32.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.674+0000 7f987b475640 1 Processor -- start 2023-10-16T09:49:32.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.674+0000 7f987b475640 1 -- start start 2023-10-16T09:49:32.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.676+0000 7f987b475640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f98740adb50 0x7f987406f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:32.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.676+0000 7f987b475640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9874071450 con 0x7f98740b0b20 2023-10-16T09:49:32.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.676+0000 7f987b475640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98740cc600 con 0x7f9874070fd0 2023-10-16T09:49:32.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.676+0000 7f987b475640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98740cc8e0 con 0x7f98740adb50 2023-10-16T09:49:32.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.676+0000 7f9879a12640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f98740b0b20 0x7f9874164f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33542/0 (socket says 172.21.15.84:33542) 2023-10-16T09:49:32.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.676+0000 7f9879a12640 1 -- 172.21.15.84:0/1276170973 learned_addr learned my addr 172.21.15.84:0/1276170973 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:32.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.676+0000 7f986bfff640 1 -- 172.21.15.84:0/1276170973 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3564128999 0 0) 0x7f9874071450 con 0x7f98740b0b20 2023-10-16T09:49:32.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.677+0000 7f9879211640 1 --2- 172.21.15.84:0/1276170973 >> v2:172.21.15.157:3300/0 conn(0x7f98740adb50 0x7f987406f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:32.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.677+0000 7f986bfff640 1 -- 172.21.15.84:0/1276170973 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9860003520 con 0x7f98740b0b20 2023-10-16T09:49:32.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.677+0000 7f986bfff640 1 -- 172.21.15.84:0/1276170973 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3420995504 0 0) 0x7f9860003520 con 0x7f98740b0b20 2023-10-16T09:49:32.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.677+0000 7f986bfff640 1 -- 172.21.15.84:0/1276170973 >> v2:172.21.15.157:3300/0 conn(0x7f98740adb50 msgr2=0x7f987406f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:32.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.677+0000 7f986bfff640 1 --2- 172.21.15.84:0/1276170973 >> v2:172.21.15.157:3300/0 conn(0x7f98740adb50 0x7f987406f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:32.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.677+0000 7f986bfff640 1 -- 172.21.15.84:0/1276170973 >> v1:172.21.15.157:6789/0 conn(0x7f9874070fd0 legacy=0x7f98740b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:32.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.677+0000 7f986bfff640 1 -- 172.21.15.84:0/1276170973 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98740ccbc0 con 0x7f98740b0b20 2023-10-16T09:49:32.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.677+0000 7f986bfff640 1 -- 172.21.15.84:0/1276170973 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f986c0019f0 con 0x7f98740b0b20 2023-10-16T09:49:32.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.677+0000 7f986bfff640 1 -- 172.21.15.84:0/1276170973 >> v1:172.21.15.84:6789/0 conn(0x7f98740b0b20 legacy=0x7f9874164f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:32.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.677+0000 7f986bfff640 1 --2- 172.21.15.84:0/1276170973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98600039c0 0x7f9860003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:32.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.677+0000 7f986bfff640 1 --2- 172.21.15.84:0/1276170973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98600047c0 0x7f9860004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:32.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.677+0000 7f986bfff640 1 --2- 172.21.15.84:0/1276170973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98600051d0 0x7f986000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:32.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.677+0000 7f986bfff640 1 -- 172.21.15.84:0/1276170973 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98740cc8e0 con 0x7f98600051d0 2023-10-16T09:49:32.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.677+0000 7f986bfff640 1 -- 172.21.15.84:0/1276170973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986000c100 con 0x7f98600039c0 2023-10-16T09:49:32.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.677+0000 7f986bfff640 1 -- 172.21.15.84:0/1276170973 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986000c2c0 con 0x7f98600047c0 2023-10-16T09:49:32.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.677+0000 7f9879a12640 1 --2- 172.21.15.84:0/1276170973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98600051d0 0x7f986000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:32.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.678+0000 7f9879211640 1 --2- 172.21.15.84:0/1276170973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98600039c0 0x7f9860003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:32.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.678+0000 7f9878a10640 1 --2- 172.21.15.84:0/1276170973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98600047c0 0x7f9860004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:32.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.678+0000 7f9879a12640 1 -- 172.21.15.84:0/1276170973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98600047c0 msgr2=0x7f9860004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:32.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.678+0000 7f9879a12640 1 --2- 172.21.15.84:0/1276170973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98600047c0 0x7f9860004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:32.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.678+0000 7f9879a12640 1 -- 172.21.15.84:0/1276170973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98600039c0 msgr2=0x7f9860003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:32.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.678+0000 7f9879a12640 1 --2- 172.21.15.84:0/1276170973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98600039c0 0x7f9860003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:32.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.678+0000 7f9879a12640 1 -- 172.21.15.84:0/1276170973 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9860003520 con 0x7f98600051d0 2023-10-16T09:49:32.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.678+0000 7f9878a10640 1 --2- 172.21.15.84:0/1276170973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98600047c0 0x7f9860004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:32.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.679+0000 7f9879a12640 1 -- 172.21.15.84:0/1276170973 reap_dead start 2023-10-16T09:49:32.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.679+0000 7f9879a12640 1 --2- 172.21.15.84:0/1276170973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98600051d0 0x7f986000b980 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9874161a30 tx=0x7f98600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7dc114602773e24 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:32.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.679+0000 7f986bfff640 1 -- 172.21.15.84:0/1276170973 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c034630 con 0x7f98600051d0 2023-10-16T09:49:32.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.679+0000 7f986bfff640 1 -- 172.21.15.84:0/1276170973 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c02f070 con 0x7f98600051d0 2023-10-16T09:49:32.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.679+0000 7f986bfff640 1 -- 172.21.15.84:0/1276170973 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c0349d0 con 0x7f98600051d0 2023-10-16T09:49:32.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.680+0000 7f987b475640 1 -- 172.21.15.84:0/1276170973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98600051d0 msgr2=0x7f986000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:32.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.680+0000 7f987b475640 1 --2- 172.21.15.84:0/1276170973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98600051d0 0x7f986000b980 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9874161a30 tx=0x7f98600057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:32.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.680+0000 7f987b475640 1 -- 172.21.15.84:0/1276170973 shutdown_connections 2023-10-16T09:49:32.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.680+0000 7f987b475640 1 --2- 172.21.15.84:0/1276170973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98600051d0 0x7f986000b980 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:32.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.680+0000 7f987b475640 1 -- 172.21.15.84:0/1276170973 >> 172.21.15.84:0/1276170973 conn(0x7f98740cb410 msgr2=0x7f98740cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:32.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.680+0000 7f987b475640 1 -- 172.21.15.84:0/1276170973 shutdown_connections 2023-10-16T09:49:32.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.680+0000 7f987b475640 1 -- 172.21.15.84:0/1276170973 wait complete. 2023-10-16T09:49:32.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.682+0000 7f987b475640 1 Processor -- start 2023-10-16T09:49:32.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.682+0000 7f987b475640 1 -- start start 2023-10-16T09:49:32.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.683+0000 7f987b475640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98600051d0 0x7f987415feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:32.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.683+0000 7f987b475640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98741f7db0 0x7f98740cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:32.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.683+0000 7f987b475640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98741f8210 0x7f98740cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:32.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.683+0000 7f987b475640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98740c25b0 con 0x7f98741f7db0 2023-10-16T09:49:32.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.683+0000 7f987b475640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98740c2890 con 0x7f98600051d0 2023-10-16T09:49:32.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.683+0000 7f987b475640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98740c2b70 con 0x7f98741f8210 2023-10-16T09:49:32.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.683+0000 7f9878a10640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98741f7db0 0x7f98740cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:32.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.683+0000 7f9878a10640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98741f7db0 0x7f98740cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33952/0 (socket says 172.21.15.84:33952) 2023-10-16T09:49:32.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.683+0000 7f9878a10640 1 -- 172.21.15.84:0/3069110947 learned_addr learned my addr 172.21.15.84:0/3069110947 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:32.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.683+0000 7f9878a10640 1 -- 172.21.15.84:0/3069110947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98741f8210 msgr2=0x7f98740cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:32.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.683+0000 7f9878a10640 1 --2- 172.21.15.84:0/3069110947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98741f8210 0x7f98740cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:32.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.683+0000 7f9878a10640 1 -- 172.21.15.84:0/3069110947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98600051d0 msgr2=0x7f987415feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:32.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.683+0000 7f9878a10640 1 --2- 172.21.15.84:0/3069110947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98600051d0 0x7f987415feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:32.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.683+0000 7f9878a10640 1 -- 172.21.15.84:0/3069110947 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98740c2e50 con 0x7f98741f7db0 2023-10-16T09:49:32.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.684+0000 7f9878a10640 1 --2- 172.21.15.84:0/3069110947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98741f7db0 0x7f98740cd9e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f986400d590 tx=0x7f986400da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:32.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.684+0000 7f9869ffb640 1 -- 172.21.15.84:0/3069110947 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864007220 con 0x7f98741f7db0 2023-10-16T09:49:32.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.684+0000 7f9869ffb640 1 -- 172.21.15.84:0/3069110947 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98640073a0 con 0x7f98741f7db0 2023-10-16T09:49:32.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.684+0000 7f9869ffb640 1 -- 172.21.15.84:0/3069110947 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98640067d0 con 0x7f98741f7db0 2023-10-16T09:49:32.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.684+0000 7f987b475640 1 -- 172.21.15.84:0/3069110947 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98741603f0 con 0x7f98741f7db0 2023-10-16T09:49:32.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.684+0000 7f987b475640 1 -- 172.21.15.84:0/3069110947 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98740c4a70 con 0x7f98741f7db0 2023-10-16T09:49:32.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.685+0000 7f987b475640 1 -- 172.21.15.84:0/3069110947 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98740c3090 con 0x7f98741f7db0 2023-10-16T09:49:32.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.686+0000 7f9869ffb640 1 -- 172.21.15.84:0/3069110947 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9864006930 con 0x7f98741f7db0 2023-10-16T09:49:32.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.688+0000 7f9869ffb640 1 --2- 172.21.15.84:0/3069110947 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f985806ae70 0x7f985806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:32.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.688+0000 7f9869ffb640 1 -- 172.21.15.84:0/3069110947 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98640892d0 con 0x7f98741f7db0 2023-10-16T09:49:32.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.688+0000 7f9879211640 1 --2- 172.21.15.84:0/3069110947 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f985806ae70 0x7f985806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:32.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.689+0000 7f9879211640 1 --2- 172.21.15.84:0/3069110947 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f985806ae70 0x7f985806d330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9870006fd0 tx=0x7f987000d280 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:32.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.692+0000 7f9869ffb640 1 -- 172.21.15.84:0/3069110947 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9864054410 con 0x7f98741f7db0 2023-10-16T09:49:32.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.924+0000 7f987b475640 1 -- 172.21.15.84:0/3069110947 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f98740c26f0 con 0x7f98741f7db0 2023-10-16T09:49:32.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.925+0000 7f9869ffb640 1 -- 172.21.15.84:0/3069110947 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f98740c26f0 con 0x7f98741f7db0 2023-10-16T09:49:32.927 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T09:49:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.928+0000 7f987b475640 1 -- 172.21.15.84:0/3069110947 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f985806ae70 msgr2=0x7f985806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.928+0000 7f987b475640 1 --2- 172.21.15.84:0/3069110947 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f985806ae70 0x7f985806d330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9870006fd0 tx=0x7f987000d280 comp rx=0 tx=0).stop 2023-10-16T09:49:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.928+0000 7f987b475640 1 -- 172.21.15.84:0/3069110947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98741f7db0 msgr2=0x7f98740cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.928+0000 7f987b475640 1 --2- 172.21.15.84:0/3069110947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98741f7db0 0x7f98740cd9e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f986400d590 tx=0x7f986400da50 comp rx=0 tx=0).stop 2023-10-16T09:49:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.928+0000 7f987b475640 1 -- 172.21.15.84:0/3069110947 shutdown_connections 2023-10-16T09:49:32.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.928+0000 7f987b475640 1 --2- 172.21.15.84:0/3069110947 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f985806ae70 0x7f985806d330 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:32.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.928+0000 7f987b475640 1 --2- 172.21.15.84:0/3069110947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98741f8210 0x7f98740cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:32.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.928+0000 7f987b475640 1 --2- 172.21.15.84:0/3069110947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98741f7db0 0x7f98740cd9e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:32.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.928+0000 7f987b475640 1 --2- 172.21.15.84:0/3069110947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98600051d0 0x7f987415feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:32.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.928+0000 7f987b475640 1 -- 172.21.15.84:0/3069110947 >> 172.21.15.84:0/3069110947 conn(0x7f98740cb410 msgr2=0x7f98740af320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.928+0000 7f987b475640 1 -- 172.21.15.84:0/3069110947 shutdown_connections 2023-10-16T09:49:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:32.928+0000 7f987b475640 1 -- 172.21.15.84:0/3069110947 wait complete. 2023-10-16T09:49:32.949 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T09:49:32.949 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T09:49:32.949 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-10-16T09:49:32.949 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-16T09:49:32.950 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-16T09:49:32.950 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T09:49:32.950 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T09:49:32.950 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T09:49:32.951 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-16T09:49:32.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:32.951 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-10-16T09:49:33.014 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:49:33.015 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:49:33.015 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:49:33.015 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:33.015 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-10-16T09:49:33.016 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:33.016 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:49:33.016 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:49:33.016 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-10-16T09:49:33.016 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:49:33.017 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:49:33.017 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-10-16T09:49:33.017 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:49:33.017 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:49:33.017 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T09:49:33.017 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:49:33.018 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T09:49:33.018 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42390sec preferred_lft 42390sec 2023-10-16T09:49:33.018 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2023-10-16T09:49:33.018 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:33.018 INFO:teuthology.orchestra.run.smithi157.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T09:49:33.019 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether ea:46:54:4d:79:2c brd ff:ff:ff:ff:ff:ff 2023-10-16T09:49:33.019 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T09:49:33.019 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:33.019 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::28bf:7eff:fec6:ab65/64 scope link 2023-10-16T09:49:33.019 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:33.020 INFO:teuthology.orchestra.run.smithi157.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T09:49:33.020 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether ea:46:54:4d:79:2c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T09:49:33.020 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::e846:54ff:fe4d:792c/64 scope link 2023-10-16T09:49:33.020 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:33.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:33.021 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T09:49:33.116 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-10-16T09:49:33.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:33.121 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-10-16T09:49:33.178 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-10-16T09:49:33.178 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T09:49:33.178 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T09:49:33.178 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T09:49:33.178 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T09:49:33.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:33.180 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-10-16T09:49:33.180 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-10-16T09:49:33.180 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T09:49:33.181 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T09:49:33.181 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T09:49:33.181 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-10-16T09:49:33.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:33.369 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-10-16T09:49:33.422 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-16T09:49:33.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:33.424 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2023-10-16T09:49:33.474 INFO:teuthology.orchestra.run.smithi157.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-16T09:49:33.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:33.475 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-10-16T09:49:33.476 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-10-16T09:49:33.476 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T09:49:33.476 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-16T09:49:33.476 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-10-16T09:49:33.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:33.618 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-16T09:49:33.711 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:49:33.712 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:49:33.712 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:49:33.712 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:33.712 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-10-16T09:49:33.713 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:33.713 INFO:teuthology.orchestra.run.smithi157.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T09:49:33.713 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether ce:22:3c:30:e7:89 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T09:49:33.713 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T09:49:33.713 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:33.714 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::cc22:3cff:fe30:e789/64 scope link 2023-10-16T09:49:33.714 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:33.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:33.716 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-16T09:49:33.811 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:49:33.812 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:49:33.812 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:49:33.812 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:33.812 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-10-16T09:49:33.812 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:33.813 INFO:teuthology.orchestra.run.smithi157.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T09:49:33.813 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether ce:22:3c:30:e7:89 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T09:49:33.813 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T09:49:33.813 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:33.813 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::cc22:3cff:fe30:e789/64 scope link 2023-10-16T09:49:33.813 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:49:33.815 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-16T09:49:33.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:33.816 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-10-16T09:49:33.816 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-10-16T09:49:33.817 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-16T09:49:33.817 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-16T09:49:33.817 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-16T09:49:33.817 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-16T09:49:33.818 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-16T09:49:33.818 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-10-16T09:49:34.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:34.097 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-10-16T09:49:34.097 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-10-16T09:49:34.097 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.1 up 2023-10-16T09:49:34.098 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.1 master ceph-brx 2023-10-16T09:49:34.098 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-10-16T09:49:34.267 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:49:34.267 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-16T09:49:34.319 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-16T09:49:34.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:49:34.319 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-16T09:49:34.370 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-10-16T09:49:34.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:34.370 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T09:49:34.510 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T09:49:34.510 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T09:49:34.544 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T09:49:34.545 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T09:49:34.545 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T09:49:34.545 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 3@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T09:49:34.545 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T09:49:34.546 INFO:teuthology.orchestra.run.smithi157.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2023-10-16T09:49:34.546 INFO:teuthology.orchestra.run.smithi157.stdout: You just mounted a file system that supports labels which does not 2023-10-16T09:49:34.547 INFO:teuthology.orchestra.run.smithi157.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T09:49:34.547 INFO:teuthology.orchestra.run.smithi157.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T09:49:34.547 INFO:teuthology.orchestra.run.smithi157.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T09:49:34.551 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T09:49:34.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:34.551 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-16T09:49:34.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:34.637 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:49:34.692 INFO:teuthology.orchestra.run.smithi157.stdout:client4478 2023-10-16T09:49:34.692 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:49:34.692 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:49:34.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:34.742 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:49:34.843 INFO:teuthology.orchestra.run.smithi157.stdout:client4478 2023-10-16T09:49:34.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:34.844 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:49:34.900 INFO:teuthology.orchestra.run.smithi157.stdout:client4478 2023-10-16T09:49:34.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:34.901 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/63fb4d68-f881-4694-b880-d10369105c0e.client4478/status') 2023-10-16T09:49:34.992 INFO:teuthology.orchestra.run.smithi157.stdout:instance: client.4478 (3)192.168.0.2:0/103737647 2023-10-16T09:49:34.993 INFO:teuthology.orchestra.run.smithi157.stdout:blocklisted: false 2023-10-16T09:49:34.993 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records in 2023-10-16T09:49:34.993 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records out 2023-10-16T09:49:34.993 INFO:teuthology.orchestra.run.smithi157.stderr:68 bytes copied, 7.5466e-05 s, 901 kB/s 2023-10-16T09:49:34.996 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-16T09:49:35.015 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:35.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.239+0000 7fe24a81f640 1 Processor -- start 2023-10-16T09:49:35.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.240+0000 7fe24a81f640 1 -- start start 2023-10-16T09:49:35.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.241+0000 7fe24a81f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe2440cd0c0 0x7fe2441615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:35.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.241+0000 7fe24a81f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe24415db00 0x7fe2440b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:35.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.241+0000 7fe24a81f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24415e3e0 con 0x7fe2440b0410 2023-10-16T09:49:35.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.241+0000 7fe24a81f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2440b0de0 con 0x7fe2440cd0c0 2023-10-16T09:49:35.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.241+0000 7fe24a81f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2440cd4a0 con 0x7fe24415db00 2023-10-16T09:49:35.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.241+0000 7fe243fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe2440cd0c0 0x7fe2441615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:35.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.242+0000 7fe243fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe2440cd0c0 0x7fe2441615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33960/0 (socket says 172.21.15.84:33960) 2023-10-16T09:49:35.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.242+0000 7fe243fff640 1 -- 172.21.15.84:0/3915250829 learned_addr learned my addr 172.21.15.84:0/3915250829 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:35.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.241+0000 7fe2437fe640 1 --1- 172.21.15.84:0/3915250829 >> v1:172.21.15.84:6789/0 conn(0x7fe2440b0410 0x7fe24415d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33550/0 (socket says 172.21.15.84:33550) 2023-10-16T09:49:35.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.242+0000 7fe248dbc640 1 --2- 172.21.15.84:0/3915250829 >> v2:172.21.15.157:3300/0 conn(0x7fe24415db00 0x7fe2440b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:35.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.242+0000 7fe243fff640 1 -- 172.21.15.84:0/3915250829 >> v2:172.21.15.157:3300/0 conn(0x7fe24415db00 msgr2=0x7fe2440b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:35.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.242+0000 7fe243fff640 1 --2- 172.21.15.84:0/3915250829 >> v2:172.21.15.157:3300/0 conn(0x7fe24415db00 0x7fe2440b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.242+0000 7fe243fff640 1 -- 172.21.15.84:0/3915250829 >> v1:172.21.15.84:6789/0 conn(0x7fe2440b0410 legacy=0x7fe24415d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:35.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.242+0000 7fe243fff640 1 -- 172.21.15.84:0/3915250829 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24415e100 con 0x7fe2440cd0c0 2023-10-16T09:49:35.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.243+0000 7fe243fff640 1 --2- 172.21.15.84:0/3915250829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2440cd0c0 0x7fe2441615f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe24415afc0 tx=0x7fe22c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b524cb4234320d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:35.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.243+0000 7fe242ffd640 1 -- 172.21.15.84:0/3915250829 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c031870 con 0x7fe2440cd0c0 2023-10-16T09:49:35.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.243+0000 7fe242ffd640 1 -- 172.21.15.84:0/3915250829 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c0319d0 con 0x7fe2440cd0c0 2023-10-16T09:49:35.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.243+0000 7fe242ffd640 1 -- 172.21.15.84:0/3915250829 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c031d60 con 0x7fe2440cd0c0 2023-10-16T09:49:35.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.244+0000 7fe24a81f640 1 -- 172.21.15.84:0/3915250829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2440cd0c0 msgr2=0x7fe2441615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:35.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.244+0000 7fe24a81f640 1 --2- 172.21.15.84:0/3915250829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2440cd0c0 0x7fe2441615f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe24415afc0 tx=0x7fe22c02b910 comp rx=0 tx=0).stop 2023-10-16T09:49:35.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.244+0000 7fe24a81f640 1 -- 172.21.15.84:0/3915250829 shutdown_connections 2023-10-16T09:49:35.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.244+0000 7fe24a81f640 1 --2- 172.21.15.84:0/3915250829 >> v2:172.21.15.157:3300/0 conn(0x7fe24415db00 0x7fe2440b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.244+0000 7fe24a81f640 1 --2- 172.21.15.84:0/3915250829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2440cd0c0 0x7fe2441615f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.244+0000 7fe24a81f640 1 -- 172.21.15.84:0/3915250829 >> 172.21.15.84:0/3915250829 conn(0x7fe2440cb410 msgr2=0x7fe2440cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:35.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.244+0000 7fe24a81f640 1 -- 172.21.15.84:0/3915250829 shutdown_connections 2023-10-16T09:49:35.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.244+0000 7fe24a81f640 1 -- 172.21.15.84:0/3915250829 wait complete. 2023-10-16T09:49:35.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.246+0000 7fe24a81f640 1 Processor -- start 2023-10-16T09:49:35.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.246+0000 7fe24a81f640 1 -- start start 2023-10-16T09:49:35.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.247+0000 7fe24a81f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2440b0410 0x7fe2441f6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:35.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.247+0000 7fe24a81f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2440cd0c0 0x7fe244152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:35.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.247+0000 7fe24a81f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe24415db00 0x7fe24415f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:35.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.247+0000 7fe24a81f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe244160810 con 0x7fe24415db00 2023-10-16T09:49:35.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.247+0000 7fe24a81f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2441f2db0 con 0x7fe2440cd0c0 2023-10-16T09:49:35.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.247+0000 7fe24a81f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24415f840 con 0x7fe2440b0410 2023-10-16T09:49:35.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.247+0000 7fe248dbc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe24415db00 0x7fe24415f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:35.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.247+0000 7fe248dbc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe24415db00 0x7fe24415f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33966/0 (socket says 172.21.15.84:33966) 2023-10-16T09:49:35.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.247+0000 7fe248dbc640 1 -- 172.21.15.84:0/1786016394 learned_addr learned my addr 172.21.15.84:0/1786016394 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:35.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.248+0000 7fe248dbc640 1 -- 172.21.15.84:0/1786016394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2440b0410 msgr2=0x7fe2441f6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:35.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.248+0000 7fe2437fe640 1 --2- 172.21.15.84:0/1786016394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2440cd0c0 0x7fe244152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:35.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.248+0000 7fe243fff640 1 --2- 172.21.15.84:0/1786016394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2440b0410 0x7fe2441f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:35.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.248+0000 7fe248dbc640 1 --2- 172.21.15.84:0/1786016394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2440b0410 0x7fe2441f6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.248+0000 7fe248dbc640 1 -- 172.21.15.84:0/1786016394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2440cd0c0 msgr2=0x7fe244152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:35.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.248+0000 7fe248dbc640 1 --2- 172.21.15.84:0/1786016394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2440cd0c0 0x7fe244152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.248+0000 7fe248dbc640 1 -- 172.21.15.84:0/1786016394 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c0094d0 con 0x7fe24415db00 2023-10-16T09:49:35.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.248+0000 7fe248dbc640 1 --2- 172.21.15.84:0/1786016394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe24415db00 0x7fe24415f300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00d800 tx=0x7fe23c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:35.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.248+0000 7fe240ff9640 1 -- 172.21.15.84:0/1786016394 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c00b6c0 con 0x7fe24415db00 2023-10-16T09:49:35.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.248+0000 7fe240ff9640 1 -- 172.21.15.84:0/1786016394 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23c00b840 con 0x7fe24415db00 2023-10-16T09:49:35.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.248+0000 7fe240ff9640 1 -- 172.21.15.84:0/1786016394 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c00bbf0 con 0x7fe24415db00 2023-10-16T09:49:35.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.248+0000 7fe24a81f640 1 -- 172.21.15.84:0/1786016394 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2441f7000 con 0x7fe24415db00 2023-10-16T09:49:35.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.249+0000 7fe24a81f640 1 -- 172.21.15.84:0/1786016394 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2441f6a40 con 0x7fe24415db00 2023-10-16T09:49:35.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.250+0000 7fe24a81f640 1 -- 172.21.15.84:0/1786016394 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2441f2f20 con 0x7fe24415db00 2023-10-16T09:49:35.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.251+0000 7fe240ff9640 1 -- 172.21.15.84:0/1786016394 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe23c00bd50 con 0x7fe24415db00 2023-10-16T09:49:35.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.251+0000 7fe240ff9640 1 --2- 172.21.15.84:0/1786016394 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe22406ae70 0x7fe22406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:35.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.252+0000 7fe240ff9640 1 -- 172.21.15.84:0/1786016394 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe23c089c40 con 0x7fe24415db00 2023-10-16T09:49:35.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.252+0000 7fe243fff640 1 --2- 172.21.15.84:0/1786016394 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe22406ae70 0x7fe22406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:35.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.252+0000 7fe243fff640 1 --2- 172.21.15.84:0/1786016394 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe22406ae70 0x7fe22406d330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe22c036000 tx=0x7fe22c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:35.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.256+0000 7fe240ff9640 1 -- 172.21.15.84:0/1786016394 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe23c054c50 con 0x7fe24415db00 2023-10-16T09:49:35.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.448+0000 7fe24a81f640 1 -- 172.21.15.84:0/1786016394 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe244160dc0 con 0x7fe24415db00 2023-10-16T09:49:35.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.450+0000 7fe240ff9640 1 -- 172.21.15.84:0/1786016394 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fe23c058270 con 0x7fe24415db00 2023-10-16T09:49:35.452 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:35.453 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":21,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:03.529261+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:35.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.453+0000 7fe24a81f640 1 -- 172.21.15.84:0/1786016394 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe22406ae70 msgr2=0x7fe22406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:35.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.453+0000 7fe24a81f640 1 --2- 172.21.15.84:0/1786016394 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe22406ae70 0x7fe22406d330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe22c036000 tx=0x7fe22c03c040 comp rx=0 tx=0).stop 2023-10-16T09:49:35.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.453+0000 7fe24a81f640 1 -- 172.21.15.84:0/1786016394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe24415db00 msgr2=0x7fe24415f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:35.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.453+0000 7fe24a81f640 1 --2- 172.21.15.84:0/1786016394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe24415db00 0x7fe24415f300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00d800 tx=0x7fe23c00dcc0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.453+0000 7fe24a81f640 1 -- 172.21.15.84:0/1786016394 shutdown_connections 2023-10-16T09:49:35.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.453+0000 7fe24a81f640 1 --2- 172.21.15.84:0/1786016394 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe22406ae70 0x7fe22406d330 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.453+0000 7fe24a81f640 1 --2- 172.21.15.84:0/1786016394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe24415db00 0x7fe24415f300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.453+0000 7fe24a81f640 1 --2- 172.21.15.84:0/1786016394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2440cd0c0 0x7fe244152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.453+0000 7fe24a81f640 1 --2- 172.21.15.84:0/1786016394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2440b0410 0x7fe2441f6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.453+0000 7fe24a81f640 1 -- 172.21.15.84:0/1786016394 >> 172.21.15.84:0/1786016394 conn(0x7fe2440cb410 msgr2=0x7fe2440c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:35.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.453+0000 7fe24a81f640 1 -- 172.21.15.84:0/1786016394 shutdown_connections 2023-10-16T09:49:35.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.453+0000 7fe24a81f640 1 -- 172.21.15.84:0/1786016394 wait complete. 2023-10-16T09:49:35.474 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:49:00.048649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:49:01.361365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:49:35.475 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:49:35.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.688+0000 7fe2d7ab4640 1 Processor -- start 2023-10-16T09:49:35.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.688+0000 7fe2d7ab4640 1 -- start start 2023-10-16T09:49:35.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.689+0000 7fe2d7ab4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe2d015ad70 0x7fe2d0069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:35.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.689+0000 7fe2d7ab4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2d00c9ca0 0x7fe2d006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:35.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.689+0000 7fe2d7ab4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d00cb030 con 0x7fe2d00ca3b0 2023-10-16T09:49:35.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.689+0000 7fe2d7ab4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d00c91c0 con 0x7fe2d015ad70 2023-10-16T09:49:35.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.689+0000 7fe2d7ab4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d00c94a0 con 0x7fe2d00c9ca0 2023-10-16T09:49:35.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.690+0000 7fe2d6051640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe2d00ca3b0 0x7fe2d00ca840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53546/0 (socket says 172.21.15.84:53546) 2023-10-16T09:49:35.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.690+0000 7fe2d6051640 1 -- 172.21.15.84:0/3653786153 learned_addr learned my addr 172.21.15.84:0/3653786153 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:35.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.690+0000 7fe2d5850640 1 --2- 172.21.15.84:0/3653786153 >> v2:172.21.15.157:3300/0 conn(0x7fe2d015ad70 0x7fe2d0069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:35.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.690+0000 7fe2d484e640 1 -- 172.21.15.84:0/3653786153 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872293177 0 0) 0x7fe2d00cb030 con 0x7fe2d00ca3b0 2023-10-16T09:49:35.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.690+0000 7fe2d484e640 1 -- 172.21.15.84:0/3653786153 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2bc003520 con 0x7fe2d00ca3b0 2023-10-16T09:49:35.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.690+0000 7fe2d504f640 1 --2- 172.21.15.84:0/3653786153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2d00c9ca0 0x7fe2d006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:35.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.690+0000 7fe2d484e640 1 -- 172.21.15.84:0/3653786153 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2499611792 0 0) 0x7fe2bc003520 con 0x7fe2d00ca3b0 2023-10-16T09:49:35.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.690+0000 7fe2d484e640 1 -- 172.21.15.84:0/3653786153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2d00c9ca0 msgr2=0x7fe2d006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:35.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.690+0000 7fe2d484e640 1 --2- 172.21.15.84:0/3653786153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2d00c9ca0 0x7fe2d006a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.690+0000 7fe2d484e640 1 -- 172.21.15.84:0/3653786153 >> v2:172.21.15.157:3300/0 conn(0x7fe2d015ad70 msgr2=0x7fe2d0069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:35.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.690+0000 7fe2d484e640 1 --2- 172.21.15.84:0/3653786153 >> v2:172.21.15.157:3300/0 conn(0x7fe2d015ad70 0x7fe2d0069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.690+0000 7fe2d484e640 1 -- 172.21.15.84:0/3653786153 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d00c9780 con 0x7fe2d00ca3b0 2023-10-16T09:49:35.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.691+0000 7fe2d484e640 1 -- 172.21.15.84:0/3653786153 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fe2c80019f0 con 0x7fe2d00ca3b0 2023-10-16T09:49:35.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.691+0000 7fe2d484e640 1 -- 172.21.15.84:0/3653786153 >> v1:172.21.15.84:6789/0 conn(0x7fe2d00ca3b0 legacy=0x7fe2d00ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:35.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.691+0000 7fe2d484e640 1 --2- 172.21.15.84:0/3653786153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2bc003970 0x7fe2bc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:35.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.691+0000 7fe2d484e640 1 --2- 172.21.15.84:0/3653786153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2d00c9ca0 0x7fe2bc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:35.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.691+0000 7fe2d484e640 1 --2- 172.21.15.84:0/3653786153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2bc004d00 0x7fe2bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:35.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.691+0000 7fe2d484e640 1 -- 172.21.15.84:0/3653786153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d00c91c0 con 0x7fe2bc004d00 2023-10-16T09:49:35.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.691+0000 7fe2d484e640 1 -- 172.21.15.84:0/3653786153 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d00c94a0 con 0x7fe2bc003970 2023-10-16T09:49:35.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.691+0000 7fe2d484e640 1 -- 172.21.15.84:0/3653786153 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc00b980 con 0x7fe2d00c9ca0 2023-10-16T09:49:35.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.691+0000 7fe2d6051640 1 --2- 172.21.15.84:0/3653786153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2bc004d00 0x7fe2bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:35.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.691+0000 7fe2d5850640 1 --2- 172.21.15.84:0/3653786153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2bc003970 0x7fe2bc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:35.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.691+0000 7fe2d6051640 1 -- 172.21.15.84:0/3653786153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2d00c9ca0 msgr2=0x7fe2bc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:35.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.691+0000 7fe2d6051640 1 --2- 172.21.15.84:0/3653786153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2d00c9ca0 0x7fe2bc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.691+0000 7fe2d6051640 1 -- 172.21.15.84:0/3653786153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2bc003970 msgr2=0x7fe2bc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:35.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.691+0000 7fe2d6051640 1 --2- 172.21.15.84:0/3653786153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2bc003970 0x7fe2bc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.691+0000 7fe2d6051640 1 -- 172.21.15.84:0/3653786153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc003520 con 0x7fe2bc004d00 2023-10-16T09:49:35.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.692+0000 7fe2d6051640 1 --2- 172.21.15.84:0/3653786153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2bc004d00 0x7fe2bc005180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800a900 tx=0x7fe2bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e34f747d5dbc0a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:35.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.693+0000 7fe2d484e640 1 -- 172.21.15.84:0/3653786153 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8034630 con 0x7fe2bc004d00 2023-10-16T09:49:35.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.693+0000 7fe2d484e640 1 -- 172.21.15.84:0/3653786153 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c802f070 con 0x7fe2bc004d00 2023-10-16T09:49:35.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.693+0000 7fe2d484e640 1 -- 172.21.15.84:0/3653786153 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80349d0 con 0x7fe2bc004d00 2023-10-16T09:49:35.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.693+0000 7fe2d7ab4640 1 -- 172.21.15.84:0/3653786153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2bc004d00 msgr2=0x7fe2bc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:35.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.693+0000 7fe2d7ab4640 1 --2- 172.21.15.84:0/3653786153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2bc004d00 0x7fe2bc005180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800a900 tx=0x7fe2bc0057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.693+0000 7fe2d6051640 1 -- 172.21.15.84:0/3653786153 reap_dead start 2023-10-16T09:49:35.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.693+0000 7fe2d7ab4640 1 -- 172.21.15.84:0/3653786153 shutdown_connections 2023-10-16T09:49:35.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.693+0000 7fe2d7ab4640 1 -- 172.21.15.84:0/3653786153 >> 172.21.15.84:0/3653786153 conn(0x7fe2d00ce9b0 msgr2=0x7fe2d00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:35.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.693+0000 7fe2d7ab4640 1 -- 172.21.15.84:0/3653786153 shutdown_connections 2023-10-16T09:49:35.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.693+0000 7fe2d7ab4640 1 -- 172.21.15.84:0/3653786153 wait complete. 2023-10-16T09:49:35.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.695+0000 7fe2d7ab4640 1 Processor -- start 2023-10-16T09:49:35.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.695+0000 7fe2d7ab4640 1 -- start start 2023-10-16T09:49:35.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.696+0000 7fe2d7ab4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2bc004d00 0x7fe2d00c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:35.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.696+0000 7fe2d7ab4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2d00ad280 0x7fe2d01f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:35.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.696+0000 7fe2d7ab4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d00ad700 0x7fe2d01f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:35.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.696+0000 7fe2d7ab4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d00cd190 con 0x7fe2d00ad280 2023-10-16T09:49:35.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.696+0000 7fe2d7ab4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d00cc580 con 0x7fe2d00ad700 2023-10-16T09:49:35.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.696+0000 7fe2d7ab4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d00cc860 con 0x7fe2bc004d00 2023-10-16T09:49:35.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.696+0000 7fe2d504f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2d00ad280 0x7fe2d01f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:35.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.697+0000 7fe2d504f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2d00ad280 0x7fe2d01f5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34530/0 (socket says 172.21.15.84:34530) 2023-10-16T09:49:35.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.697+0000 7fe2d504f640 1 -- 172.21.15.84:0/2892434307 learned_addr learned my addr 172.21.15.84:0/2892434307 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:35.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.697+0000 7fe2d5850640 1 --2- 172.21.15.84:0/2892434307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2bc004d00 0x7fe2d00c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:35.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.697+0000 7fe2d6051640 1 --2- 172.21.15.84:0/2892434307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d00ad700 0x7fe2d01f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:35.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.697+0000 7fe2d504f640 1 -- 172.21.15.84:0/2892434307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2bc004d00 msgr2=0x7fe2d00c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:35.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.697+0000 7fe2d504f640 1 --2- 172.21.15.84:0/2892434307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2bc004d00 0x7fe2d00c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.697+0000 7fe2d504f640 1 -- 172.21.15.84:0/2892434307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d00ad700 msgr2=0x7fe2d01f5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:35.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.697+0000 7fe2d504f640 1 --2- 172.21.15.84:0/2892434307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d00ad700 0x7fe2d01f5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.697+0000 7fe2d504f640 1 -- 172.21.15.84:0/2892434307 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d00ccb40 con 0x7fe2d00ad280 2023-10-16T09:49:35.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.697+0000 7fe2d504f640 1 --2- 172.21.15.84:0/2892434307 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2d00ad280 0x7fe2d01f5950 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000d800 tx=0x7fe2c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:35.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.697+0000 7fe2c67fc640 1 -- 172.21.15.84:0/2892434307 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c000b6c0 con 0x7fe2d00ad280 2023-10-16T09:49:35.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.697+0000 7fe2c67fc640 1 -- 172.21.15.84:0/2892434307 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c000b820 con 0x7fe2d00ad280 2023-10-16T09:49:35.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.697+0000 7fe2d7ab4640 1 -- 172.21.15.84:0/2892434307 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d00cce20 con 0x7fe2d00ad280 2023-10-16T09:49:35.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.698+0000 7fe2c67fc640 1 -- 172.21.15.84:0/2892434307 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c000bbb0 con 0x7fe2d00ad280 2023-10-16T09:49:35.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.698+0000 7fe2d7ab4640 1 -- 172.21.15.84:0/2892434307 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d00b0480 con 0x7fe2d00ad280 2023-10-16T09:49:35.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.699+0000 7fe2d7ab4640 1 -- 172.21.15.84:0/2892434307 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d00cd300 con 0x7fe2d00ad280 2023-10-16T09:49:35.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.700+0000 7fe2c67fc640 1 -- 172.21.15.84:0/2892434307 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe2c000bd10 con 0x7fe2d00ad280 2023-10-16T09:49:35.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.701+0000 7fe2c67fc640 1 --2- 172.21.15.84:0/2892434307 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe2b406aea0 0x7fe2b406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:35.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.701+0000 7fe2c67fc640 1 -- 172.21.15.84:0/2892434307 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2c00897a0 con 0x7fe2d00ad280 2023-10-16T09:49:35.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.701+0000 7fe2d5850640 1 --2- 172.21.15.84:0/2892434307 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe2b406aea0 0x7fe2b406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:35.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.702+0000 7fe2d5850640 1 --2- 172.21.15.84:0/2892434307 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe2b406aea0 0x7fe2b406d360 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc0099d0 tx=0x7fe2cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:35.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.706+0000 7fe2c67fc640 1 -- 172.21.15.84:0/2892434307 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe2c0054850 con 0x7fe2d00ad280 2023-10-16T09:49:35.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.891+0000 7fe2d7ab4640 1 -- 172.21.15.84:0/2892434307 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2d004e160 con 0x7fe2d00ad280 2023-10-16T09:49:35.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.891+0000 7fe2c67fc640 1 -- 172.21.15.84:0/2892434307 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2c0057e70 con 0x7fe2d00ad280 2023-10-16T09:49:35.893 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T09:49:35.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.894+0000 7fe2d7ab4640 1 -- 172.21.15.84:0/2892434307 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe2b406aea0 msgr2=0x7fe2b406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:35.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.894+0000 7fe2d7ab4640 1 --2- 172.21.15.84:0/2892434307 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe2b406aea0 0x7fe2b406d360 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc0099d0 tx=0x7fe2cc008040 comp rx=0 tx=0).stop 2023-10-16T09:49:35.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.894+0000 7fe2d7ab4640 1 -- 172.21.15.84:0/2892434307 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2d00ad280 msgr2=0x7fe2d01f5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:35.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.894+0000 7fe2d7ab4640 1 --2- 172.21.15.84:0/2892434307 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2d00ad280 0x7fe2d01f5950 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000d800 tx=0x7fe2c000dcc0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.894+0000 7fe2d7ab4640 1 -- 172.21.15.84:0/2892434307 shutdown_connections 2023-10-16T09:49:35.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.894+0000 7fe2d7ab4640 1 --2- 172.21.15.84:0/2892434307 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe2b406aea0 0x7fe2b406d360 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.894+0000 7fe2d7ab4640 1 --2- 172.21.15.84:0/2892434307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d00ad700 0x7fe2d01f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.894+0000 7fe2d7ab4640 1 --2- 172.21.15.84:0/2892434307 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2d00ad280 0x7fe2d01f5950 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.894+0000 7fe2d7ab4640 1 --2- 172.21.15.84:0/2892434307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2bc004d00 0x7fe2d00c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:35.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.894+0000 7fe2d7ab4640 1 -- 172.21.15.84:0/2892434307 >> 172.21.15.84:0/2892434307 conn(0x7fe2d00ce9b0 msgr2=0x7fe2d015d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:35.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.894+0000 7fe2d7ab4640 1 -- 172.21.15.84:0/2892434307 shutdown_connections 2023-10-16T09:49:35.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:35.894+0000 7fe2d7ab4640 1 -- 172.21.15.84:0/2892434307 wait complete. 2023-10-16T09:49:35.913 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:49:36.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.122+0000 7fad2d634640 1 Processor -- start 2023-10-16T09:49:36.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.122+0000 7fad2d634640 1 -- start start 2023-10-16T09:49:36.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.124+0000 7fad2d634640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fad28159e90 0x7fad280ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.124+0000 7fad2d634640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2806c140 0x7fad2806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.124+0000 7fad2d634640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad28070360 con 0x7fad28159ab0 2023-10-16T09:49:36.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.124+0000 7fad2d634640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2806cb10 con 0x7fad28159e90 2023-10-16T09:49:36.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.124+0000 7fad2d634640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2815d890 con 0x7fad2806c140 2023-10-16T09:49:36.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.124+0000 7fad26ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fad28159ab0 0x7fad2806fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53562/0 (socket says 172.21.15.84:53562) 2023-10-16T09:49:36.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.124+0000 7fad26ffd640 1 -- 172.21.15.84:0/3446681808 learned_addr learned my addr 172.21.15.84:0/3446681808 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:36.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad25ffb640 1 -- 172.21.15.84:0/3446681808 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806920551 0 0) 0x7fad28070360 con 0x7fad28159ab0 2023-10-16T09:49:36.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad277fe640 1 --2- 172.21.15.84:0/3446681808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2806c140 0x7fad2806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:36.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad25ffb640 1 -- 172.21.15.84:0/3446681808 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad10005730 con 0x7fad28159ab0 2023-10-16T09:49:36.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad267fc640 1 --2- 172.21.15.84:0/3446681808 >> v2:172.21.15.157:3300/0 conn(0x7fad28159e90 0x7fad280ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:36.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad25ffb640 1 -- 172.21.15.84:0/3446681808 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2296445935 0 0) 0x7fad10005730 con 0x7fad28159ab0 2023-10-16T09:49:36.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad25ffb640 1 -- 172.21.15.84:0/3446681808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2806c140 msgr2=0x7fad2806c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:36.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad25ffb640 1 --2- 172.21.15.84:0/3446681808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2806c140 0x7fad2806c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad25ffb640 1 -- 172.21.15.84:0/3446681808 >> v2:172.21.15.157:3300/0 conn(0x7fad28159e90 msgr2=0x7fad280ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:36.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad25ffb640 1 --2- 172.21.15.84:0/3446681808 >> v2:172.21.15.157:3300/0 conn(0x7fad28159e90 0x7fad280ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad25ffb640 1 -- 172.21.15.84:0/3446681808 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad280cd1f0 con 0x7fad28159ab0 2023-10-16T09:49:36.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad25ffb640 1 -- 172.21.15.84:0/3446681808 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fad18001670 con 0x7fad28159ab0 2023-10-16T09:49:36.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad25ffb640 1 -- 172.21.15.84:0/3446681808 >> v1:172.21.15.84:6789/0 conn(0x7fad28159ab0 legacy=0x7fad2806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:36.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad25ffb640 1 --2- 172.21.15.84:0/3446681808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad10005b80 0x7fad10005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad25ffb640 1 --2- 172.21.15.84:0/3446681808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad100069d0 0x7fad10006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad25ffb640 1 --2- 172.21.15.84:0/3446681808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2806c140 0x7fad10007460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad25ffb640 1 -- 172.21.15.84:0/3446681808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2806cb10 con 0x7fad10005b80 2023-10-16T09:49:36.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad25ffb640 1 -- 172.21.15.84:0/3446681808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2815d890 con 0x7fad100069d0 2023-10-16T09:49:36.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad25ffb640 1 -- 172.21.15.84:0/3446681808 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1000ddd0 con 0x7fad2806c140 2023-10-16T09:49:36.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.125+0000 7fad26ffd640 1 --2- 172.21.15.84:0/3446681808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad10005b80 0x7fad10005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:36.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.126+0000 7fad267fc640 1 --2- 172.21.15.84:0/3446681808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad100069d0 0x7fad10006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:36.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.126+0000 7fad277fe640 1 --2- 172.21.15.84:0/3446681808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2806c140 0x7fad10007460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:36.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.126+0000 7fad26ffd640 1 -- 172.21.15.84:0/3446681808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2806c140 msgr2=0x7fad10007460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:36.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.126+0000 7fad26ffd640 1 --2- 172.21.15.84:0/3446681808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2806c140 0x7fad10007460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.126+0000 7fad26ffd640 1 -- 172.21.15.84:0/3446681808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad100069d0 msgr2=0x7fad10006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:36.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.126+0000 7fad26ffd640 1 --2- 172.21.15.84:0/3446681808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad100069d0 0x7fad10006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.126+0000 7fad26ffd640 1 -- 172.21.15.84:0/3446681808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad10005730 con 0x7fad10005b80 2023-10-16T09:49:36.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.127+0000 7fad26ffd640 1 --2- 172.21.15.84:0/3446681808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad10005b80 0x7fad10005fa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fad1800c0e0 tx=0x7fad10007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5d31b1cd5f2549 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:36.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.127+0000 7fad25ffb640 1 -- 172.21.15.84:0/3446681808 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18005ad0 con 0x7fad10005b80 2023-10-16T09:49:36.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.127+0000 7fad25ffb640 1 -- 172.21.15.84:0/3446681808 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad18005c30 con 0x7fad10005b80 2023-10-16T09:49:36.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.127+0000 7fad25ffb640 1 -- 172.21.15.84:0/3446681808 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18007070 con 0x7fad10005b80 2023-10-16T09:49:36.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.128+0000 7fad2d634640 1 -- 172.21.15.84:0/3446681808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad10005b80 msgr2=0x7fad10005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:36.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.128+0000 7fad2d634640 1 --2- 172.21.15.84:0/3446681808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad10005b80 0x7fad10005fa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fad1800c0e0 tx=0x7fad10007a00 comp rx=0 tx=0).stop 2023-10-16T09:49:36.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.128+0000 7fad277fe640 1 -- 172.21.15.84:0/3446681808 reap_dead start 2023-10-16T09:49:36.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.128+0000 7fad2d634640 1 -- 172.21.15.84:0/3446681808 shutdown_connections 2023-10-16T09:49:36.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.128+0000 7fad2d634640 1 -- 172.21.15.84:0/3446681808 >> 172.21.15.84:0/3446681808 conn(0x7fad280ce9b0 msgr2=0x7fad280cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:36.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.128+0000 7fad2d634640 1 -- 172.21.15.84:0/3446681808 shutdown_connections 2023-10-16T09:49:36.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.128+0000 7fad2d634640 1 -- 172.21.15.84:0/3446681808 wait complete. 2023-10-16T09:49:36.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.129+0000 7fad2d634640 1 Processor -- start 2023-10-16T09:49:36.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.130+0000 7fad2d634640 1 -- start start 2023-10-16T09:49:36.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.130+0000 7fad2d634640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad10005b80 0x7fad28159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.130+0000 7fad2d634640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad280c8ad0 0x7fad28155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.130+0000 7fad2d634640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad280c98b0 0x7fad280ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.130+0000 7fad2d634640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad28159550 con 0x7fad10005b80 2023-10-16T09:49:36.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.130+0000 7fad2d634640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad280c7d10 con 0x7fad280c98b0 2023-10-16T09:49:36.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.130+0000 7fad2d634640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad280c7ff0 con 0x7fad280c8ad0 2023-10-16T09:49:36.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.131+0000 7fad26ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad10005b80 0x7fad28159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:36.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.131+0000 7fad26ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad10005b80 0x7fad28159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34554/0 (socket says 172.21.15.84:34554) 2023-10-16T09:49:36.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.131+0000 7fad26ffd640 1 -- 172.21.15.84:0/3658280958 learned_addr learned my addr 172.21.15.84:0/3658280958 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:36.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.131+0000 7fad267fc640 1 --2- 172.21.15.84:0/3658280958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad280c8ad0 0x7fad28155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:36.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.131+0000 7fad26ffd640 1 -- 172.21.15.84:0/3658280958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad280c8ad0 msgr2=0x7fad28155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:36.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.131+0000 7fad26ffd640 1 --2- 172.21.15.84:0/3658280958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad280c8ad0 0x7fad28155840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.131+0000 7fad26ffd640 1 -- 172.21.15.84:0/3658280958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad280c98b0 msgr2=0x7fad280ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:36.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.131+0000 7fad26ffd640 1 --2- 172.21.15.84:0/3658280958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad280c98b0 0x7fad280ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.131+0000 7fad26ffd640 1 -- 172.21.15.84:0/3658280958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad280c82d0 con 0x7fad10005b80 2023-10-16T09:49:36.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.131+0000 7fad26ffd640 1 --2- 172.21.15.84:0/3658280958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad10005b80 0x7fad28159010 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fad1802c3b0 tx=0x7fad280cdd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:36.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.132+0000 7fad0f7fe640 1 -- 172.21.15.84:0/3658280958 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18007480 con 0x7fad10005b80 2023-10-16T09:49:36.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.132+0000 7fad0f7fe640 1 -- 172.21.15.84:0/3658280958 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad180075e0 con 0x7fad10005b80 2023-10-16T09:49:36.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.132+0000 7fad0f7fe640 1 -- 172.21.15.84:0/3658280958 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18007790 con 0x7fad10005b80 2023-10-16T09:49:36.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.132+0000 7fad2d634640 1 -- 172.21.15.84:0/3658280958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad280c85b0 con 0x7fad10005b80 2023-10-16T09:49:36.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.132+0000 7fad2d634640 1 -- 172.21.15.84:0/3658280958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad280c7950 con 0x7fad10005b80 2023-10-16T09:49:36.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.133+0000 7fad2d634640 1 -- 172.21.15.84:0/3658280958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2804e160 con 0x7fad10005b80 2023-10-16T09:49:36.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.134+0000 7fad0f7fe640 1 -- 172.21.15.84:0/3658280958 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fad18032070 con 0x7fad10005b80 2023-10-16T09:49:36.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.135+0000 7fad0f7fe640 1 --2- 172.21.15.84:0/3658280958 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fad1406ae70 0x7fad1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.135+0000 7fad0f7fe640 1 -- 172.21.15.84:0/3658280958 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad1803c040 con 0x7fad10005b80 2023-10-16T09:49:36.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.136+0000 7fad267fc640 1 --2- 172.21.15.84:0/3658280958 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fad1406ae70 0x7fad1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:36.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.136+0000 7fad267fc640 1 --2- 172.21.15.84:0/3658280958 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fad1406ae70 0x7fad1406d330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fad281f5c60 tx=0x7fad1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:36.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.140+0000 7fad0f7fe640 1 -- 172.21.15.84:0/3658280958 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fad18075760 con 0x7fad10005b80 2023-10-16T09:49:36.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.324+0000 7fad2d634640 1 -- 172.21.15.84:0/3658280958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fad281574d0 con 0x7fad10005b80 2023-10-16T09:49:36.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.325+0000 7fad0f7fe640 1 -- 172.21.15.84:0/3658280958 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fad18078d80 con 0x7fad10005b80 2023-10-16T09:49:36.327 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T09:49:36.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.327+0000 7fad2d634640 1 -- 172.21.15.84:0/3658280958 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fad1406ae70 msgr2=0x7fad1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:36.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.327+0000 7fad2d634640 1 --2- 172.21.15.84:0/3658280958 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fad1406ae70 0x7fad1406d330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fad281f5c60 tx=0x7fad1c008040 comp rx=0 tx=0).stop 2023-10-16T09:49:36.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.327+0000 7fad2d634640 1 -- 172.21.15.84:0/3658280958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad10005b80 msgr2=0x7fad28159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:36.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.327+0000 7fad2d634640 1 --2- 172.21.15.84:0/3658280958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad10005b80 0x7fad28159010 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fad1802c3b0 tx=0x7fad280cdd70 comp rx=0 tx=0).stop 2023-10-16T09:49:36.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.328+0000 7fad2d634640 1 -- 172.21.15.84:0/3658280958 shutdown_connections 2023-10-16T09:49:36.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.328+0000 7fad2d634640 1 --2- 172.21.15.84:0/3658280958 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fad1406ae70 0x7fad1406d330 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.328+0000 7fad2d634640 1 --2- 172.21.15.84:0/3658280958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad280c98b0 0x7fad280ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.328+0000 7fad2d634640 1 --2- 172.21.15.84:0/3658280958 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad280c8ad0 0x7fad28155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.328+0000 7fad2d634640 1 --2- 172.21.15.84:0/3658280958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad10005b80 0x7fad28159010 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.328+0000 7fad2d634640 1 -- 172.21.15.84:0/3658280958 >> 172.21.15.84:0/3658280958 conn(0x7fad280ce9b0 msgr2=0x7fad280aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:36.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.328+0000 7fad2d634640 1 -- 172.21.15.84:0/3658280958 shutdown_connections 2023-10-16T09:49:36.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.328+0000 7fad2d634640 1 -- 172.21.15.84:0/3658280958 wait complete. 2023-10-16T09:49:36.347 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:49:36.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.553+0000 7f4496acf640 1 Processor -- start 2023-10-16T09:49:36.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.554+0000 7f4496acf640 1 -- start start 2023-10-16T09:49:36.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.555+0000 7f4496acf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f449015ad70 0x7f4490069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.555+0000 7f4496acf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f44900caac0 0x7f449006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.555+0000 7f4496acf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44900cb030 con 0x7f44900c9ca0 2023-10-16T09:49:36.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.555+0000 7f4496acf640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44900c91c0 con 0x7f44900caac0 2023-10-16T09:49:36.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.555+0000 7f4496acf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44900c94a0 con 0x7f449015ad70 2023-10-16T09:49:36.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.555+0000 7f449506c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f44900caac0 0x7f449006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:36.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.555+0000 7f448ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f44900c9ca0 0x7f44900ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53572/0 (socket says 172.21.15.84:53572) 2023-10-16T09:49:36.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.555+0000 7f449506c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f44900caac0 0x7f449006a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34566/0 (socket says 172.21.15.84:34566) 2023-10-16T09:49:36.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.555+0000 7f448ffff640 1 -- 172.21.15.84:0/1969656534 learned_addr learned my addr 172.21.15.84:0/1969656534 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:36.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.555+0000 7f449486b640 1 --2- 172.21.15.84:0/1969656534 >> v2:172.21.15.157:3300/0 conn(0x7f449015ad70 0x7f4490069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:36.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.555+0000 7f449506c640 1 -- 172.21.15.84:0/1969656534 >> v2:172.21.15.157:3300/0 conn(0x7f449015ad70 msgr2=0x7f4490069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:36.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.556+0000 7f449506c640 1 --2- 172.21.15.84:0/1969656534 >> v2:172.21.15.157:3300/0 conn(0x7f449015ad70 0x7f4490069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.556+0000 7f449506c640 1 -- 172.21.15.84:0/1969656534 >> v1:172.21.15.84:6789/0 conn(0x7f44900c9ca0 legacy=0x7f44900ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:36.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.556+0000 7f449506c640 1 -- 172.21.15.84:0/1969656534 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44900c9780 con 0x7f44900caac0 2023-10-16T09:49:36.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.557+0000 7f449506c640 1 --2- 172.21.15.84:0/1969656534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44900caac0 0x7f449006a2e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f44900cf780 tx=0x7f448802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2735156f7f7bbf9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:36.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.557+0000 7f448f7fe640 1 -- 172.21.15.84:0/1969656534 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448802cc70 con 0x7f44900caac0 2023-10-16T09:49:36.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.557+0000 7f448f7fe640 1 -- 172.21.15.84:0/1969656534 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448802cdd0 con 0x7f44900caac0 2023-10-16T09:49:36.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.557+0000 7f448f7fe640 1 -- 172.21.15.84:0/1969656534 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4488034660 con 0x7f44900caac0 2023-10-16T09:49:36.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.557+0000 7f4496acf640 1 -- 172.21.15.84:0/1969656534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44900caac0 msgr2=0x7f449006a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:36.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.557+0000 7f4496acf640 1 --2- 172.21.15.84:0/1969656534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44900caac0 0x7f449006a2e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f44900cf780 tx=0x7f448802bec0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.558+0000 7f4496acf640 1 -- 172.21.15.84:0/1969656534 shutdown_connections 2023-10-16T09:49:36.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.558+0000 7f4496acf640 1 --2- 172.21.15.84:0/1969656534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44900caac0 0x7f449006a2e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.558+0000 7f4496acf640 1 --2- 172.21.15.84:0/1969656534 >> v2:172.21.15.157:3300/0 conn(0x7f449015ad70 0x7f4490069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.558+0000 7f4496acf640 1 -- 172.21.15.84:0/1969656534 >> 172.21.15.84:0/1969656534 conn(0x7f44900ce9b0 msgr2=0x7f44900cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:36.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.558+0000 7f4496acf640 1 -- 172.21.15.84:0/1969656534 shutdown_connections 2023-10-16T09:49:36.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.558+0000 7f4496acf640 1 -- 172.21.15.84:0/1969656534 wait complete. 2023-10-16T09:49:36.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.559+0000 7f4496acf640 1 Processor -- start 2023-10-16T09:49:36.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.559+0000 7f4496acf640 1 -- start start 2023-10-16T09:49:36.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.560+0000 7f4496acf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44900c9ca0 0x7f44900b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.560+0000 7f4496acf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44900caac0 0x7f44900c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.560+0000 7f4496acf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449015ad70 0x7f44901f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.560+0000 7f4496acf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44900c69d0 con 0x7f44900c9ca0 2023-10-16T09:49:36.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.560+0000 7f4496acf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44900c6ca0 con 0x7f44900caac0 2023-10-16T09:49:36.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.560+0000 7f4496acf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44900c6f80 con 0x7f449015ad70 2023-10-16T09:49:36.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.561+0000 7f449486b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44900c9ca0 0x7f44900b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:36.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.561+0000 7f449486b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44900c9ca0 0x7f44900b3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34582/0 (socket says 172.21.15.84:34582) 2023-10-16T09:49:36.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.561+0000 7f449486b640 1 -- 172.21.15.84:0/3645106895 learned_addr learned my addr 172.21.15.84:0/3645106895 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:36.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.561+0000 7f449486b640 1 -- 172.21.15.84:0/3645106895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449015ad70 msgr2=0x7f44901f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:36.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.561+0000 7f449486b640 1 --2- 172.21.15.84:0/3645106895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449015ad70 0x7f44901f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.561+0000 7f449486b640 1 -- 172.21.15.84:0/3645106895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44900caac0 msgr2=0x7f44900c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:36.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.561+0000 7f448ffff640 1 --2- 172.21.15.84:0/3645106895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44900caac0 0x7f44900c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:36.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.561+0000 7f449486b640 1 --2- 172.21.15.84:0/3645106895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44900caac0 0x7f44900c6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.561+0000 7f449486b640 1 -- 172.21.15.84:0/3645106895 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4488009a70 con 0x7f44900c9ca0 2023-10-16T09:49:36.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.561+0000 7f449486b640 1 --2- 172.21.15.84:0/3645106895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44900c9ca0 0x7f44900b3a30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f447800c7f0 tx=0x7f447800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:36.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.562+0000 7f448d7fa640 1 -- 172.21.15.84:0/3645106895 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478004220 con 0x7f44900c9ca0 2023-10-16T09:49:36.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.562+0000 7f448d7fa640 1 -- 172.21.15.84:0/3645106895 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4478004380 con 0x7f44900c9ca0 2023-10-16T09:49:36.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.562+0000 7f448d7fa640 1 -- 172.21.15.84:0/3645106895 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478007840 con 0x7f44900c9ca0 2023-10-16T09:49:36.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.562+0000 7f4496acf640 1 -- 172.21.15.84:0/3645106895 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44900c7260 con 0x7f44900c9ca0 2023-10-16T09:49:36.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.562+0000 7f4496acf640 1 -- 172.21.15.84:0/3645106895 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44900c7820 con 0x7f44900c9ca0 2023-10-16T09:49:36.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.563+0000 7f4496acf640 1 -- 172.21.15.84:0/3645106895 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f449004e160 con 0x7f44900c9ca0 2023-10-16T09:49:36.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.564+0000 7f448d7fa640 1 -- 172.21.15.84:0/3645106895 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4478007bb0 con 0x7f44900c9ca0 2023-10-16T09:49:36.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.565+0000 7f448d7fa640 1 --2- 172.21.15.84:0/3645106895 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f447006ae70 0x7f447006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.565+0000 7f448d7fa640 1 -- 172.21.15.84:0/3645106895 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4478089570 con 0x7f44900c9ca0 2023-10-16T09:49:36.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.565+0000 7f448ffff640 1 --2- 172.21.15.84:0/3645106895 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f447006ae70 0x7f447006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:36.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.566+0000 7f448ffff640 1 --2- 172.21.15.84:0/3645106895 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f447006ae70 0x7f447006d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f44900ce0b0 tx=0x7f4480009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:36.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.569+0000 7f448d7fa640 1 -- 172.21.15.84:0/3645106895 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4478054620 con 0x7f44900c9ca0 2023-10-16T09:49:36.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.753+0000 7f4496acf640 1 -- 172.21.15.84:0/3645106895 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f44900ad4a0 con 0x7f44900c9ca0 2023-10-16T09:49:36.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.754+0000 7f448d7fa640 1 -- 172.21.15.84:0/3645106895 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f447805e030 con 0x7f44900c9ca0 2023-10-16T09:49:36.756 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T09:49:36.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.756+0000 7f4496acf640 1 -- 172.21.15.84:0/3645106895 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f447006ae70 msgr2=0x7f447006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:36.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.756+0000 7f4496acf640 1 --2- 172.21.15.84:0/3645106895 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f447006ae70 0x7f447006d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f44900ce0b0 tx=0x7f4480009040 comp rx=0 tx=0).stop 2023-10-16T09:49:36.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.756+0000 7f4496acf640 1 -- 172.21.15.84:0/3645106895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44900c9ca0 msgr2=0x7f44900b3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:36.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.756+0000 7f4496acf640 1 --2- 172.21.15.84:0/3645106895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44900c9ca0 0x7f44900b3a30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f447800c7f0 tx=0x7f447800ccb0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.756+0000 7f4496acf640 1 -- 172.21.15.84:0/3645106895 shutdown_connections 2023-10-16T09:49:36.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.756+0000 7f4496acf640 1 --2- 172.21.15.84:0/3645106895 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f447006ae70 0x7f447006d330 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.756+0000 7f4496acf640 1 --2- 172.21.15.84:0/3645106895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f449015ad70 0x7f44901f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.757+0000 7f4496acf640 1 --2- 172.21.15.84:0/3645106895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44900caac0 0x7f44900c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.757+0000 7f4496acf640 1 --2- 172.21.15.84:0/3645106895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44900c9ca0 0x7f44900b3a30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.757+0000 7f4496acf640 1 -- 172.21.15.84:0/3645106895 >> 172.21.15.84:0/3645106895 conn(0x7f44900ce9b0 msgr2=0x7f449006b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:36.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.757+0000 7f4496acf640 1 -- 172.21.15.84:0/3645106895 shutdown_connections 2023-10-16T09:49:36.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.757+0000 7f4496acf640 1 -- 172.21.15.84:0/3645106895 wait complete. 2023-10-16T09:49:36.777 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:36.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.983+0000 7f5150ba9640 1 Processor -- start 2023-10-16T09:49:36.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.983+0000 7f5150ba9640 1 -- start start 2023-10-16T09:49:36.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.984+0000 7f5150ba9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f514c159b20 0x7f514c06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.984+0000 7f5150ba9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514c06f7f0 con 0x7f514c0b0b20 2023-10-16T09:49:36.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.984+0000 7f5150ba9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514c06c5e0 con 0x7f514c159f00 2023-10-16T09:49:36.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.984+0000 7f5150ba9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514c06c8c0 con 0x7f514c159b20 2023-10-16T09:49:36.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.985+0000 7f514ad9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f514c0b0b20 0x7f514c06fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53588/0 (socket says 172.21.15.84:53588) 2023-10-16T09:49:36.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.985+0000 7f514ad9d640 1 -- 172.21.15.84:0/1480290549 learned_addr learned my addr 172.21.15.84:0/1480290549 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:36.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.985+0000 7f514a59c640 1 --2- 172.21.15.84:0/1480290549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f514c159b20 0x7f514c06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:36.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.985+0000 7f514959a640 1 -- 172.21.15.84:0/1480290549 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608308578 0 0) 0x7f514c06f7f0 con 0x7f514c0b0b20 2023-10-16T09:49:36.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.985+0000 7f514959a640 1 -- 172.21.15.84:0/1480290549 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5134003520 con 0x7f514c0b0b20 2023-10-16T09:49:36.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f514959a640 1 -- 172.21.15.84:0/1480290549 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3318514855 0 0) 0x7f5134003520 con 0x7f514c0b0b20 2023-10-16T09:49:36.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f514959a640 1 -- 172.21.15.84:0/1480290549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f514c159b20 msgr2=0x7f514c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:36.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f514959a640 1 --2- 172.21.15.84:0/1480290549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f514c159b20 0x7f514c06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f514959a640 1 -- 172.21.15.84:0/1480290549 >> v1:172.21.15.157:6789/0 conn(0x7f514c159f00 legacy=0x7f514c0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:36.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f514959a640 1 -- 172.21.15.84:0/1480290549 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c06cba0 con 0x7f514c0b0b20 2023-10-16T09:49:36.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f514959a640 1 -- 172.21.15.84:0/1480290549 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f51440019f0 con 0x7f514c0b0b20 2023-10-16T09:49:36.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f514959a640 1 -- 172.21.15.84:0/1480290549 >> v1:172.21.15.84:6789/0 conn(0x7f514c0b0b20 legacy=0x7f514c06fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:36.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f514959a640 1 --2- 172.21.15.84:0/1480290549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f514c159b20 0x7f51340039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f514959a640 1 --2- 172.21.15.84:0/1480290549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5134003f70 0x7f51340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f514959a640 1 --2- 172.21.15.84:0/1480290549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5134004d00 0x7f5134005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:36.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f514959a640 1 -- 172.21.15.84:0/1480290549 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c06c8c0 con 0x7f5134003f70 2023-10-16T09:49:36.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f514959a640 1 -- 172.21.15.84:0/1480290549 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513400bac0 con 0x7f5134004d00 2023-10-16T09:49:36.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f514959a640 1 -- 172.21.15.84:0/1480290549 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513400de90 con 0x7f514c159b20 2023-10-16T09:49:36.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f5149d9b640 1 --2- 172.21.15.84:0/1480290549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5134003f70 0x7f51340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:36.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f514a59c640 1 --2- 172.21.15.84:0/1480290549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f514c159b20 0x7f51340039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:36.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f5149d9b640 1 -- 172.21.15.84:0/1480290549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f514c159b20 msgr2=0x7f51340039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:36.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f5149d9b640 1 --2- 172.21.15.84:0/1480290549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f514c159b20 0x7f51340039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f5149d9b640 1 -- 172.21.15.84:0/1480290549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5134004d00 msgr2=0x7f5134005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:36.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f5149d9b640 1 --2- 172.21.15.84:0/1480290549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5134004d00 0x7f5134005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.986+0000 7f5149d9b640 1 -- 172.21.15.84:0/1480290549 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5134003520 con 0x7f5134003f70 2023-10-16T09:49:36.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.987+0000 7f5149d9b640 1 --2- 172.21.15.84:0/1480290549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5134003f70 0x7f51340047c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5140001720 tx=0x7f51340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460c95b4303fb07b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:36.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.988+0000 7f514959a640 1 -- 172.21.15.84:0/1480290549 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51400052d0 con 0x7f5134003f70 2023-10-16T09:49:36.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.988+0000 7f514959a640 1 -- 172.21.15.84:0/1480290549 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514002e050 con 0x7f5134003f70 2023-10-16T09:49:36.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.988+0000 7f514959a640 1 -- 172.21.15.84:0/1480290549 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140003830 con 0x7f5134003f70 2023-10-16T09:49:36.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.988+0000 7f5150ba9640 1 -- 172.21.15.84:0/1480290549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5134003f70 msgr2=0x7f51340047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:36.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.988+0000 7f5150ba9640 1 --2- 172.21.15.84:0/1480290549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5134003f70 0x7f51340047c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5140001720 tx=0x7f51340057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:36.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.988+0000 7f514ad9d640 1 -- 172.21.15.84:0/1480290549 reap_dead start 2023-10-16T09:49:36.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.988+0000 7f5150ba9640 1 -- 172.21.15.84:0/1480290549 shutdown_connections 2023-10-16T09:49:36.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.988+0000 7f5150ba9640 1 -- 172.21.15.84:0/1480290549 >> 172.21.15.84:0/1480290549 conn(0x7f514c0cb410 msgr2=0x7f514c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:36.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.988+0000 7f5150ba9640 1 -- 172.21.15.84:0/1480290549 shutdown_connections 2023-10-16T09:49:36.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.988+0000 7f5150ba9640 1 -- 172.21.15.84:0/1480290549 wait complete. 2023-10-16T09:49:37.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.990+0000 7f5150ba9640 1 Processor -- start 2023-10-16T09:49:37.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.990+0000 7f5150ba9640 1 -- start start 2023-10-16T09:49:37.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.991+0000 7f5150ba9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5134003f70 0x7f514c155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:37.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.991+0000 7f5150ba9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f514c0b3c80 0x7f514c0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:37.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.991+0000 7f5150ba9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f514c0c7d60 0x7f514c0c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:37.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.991+0000 7f5150ba9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c0b4640 con 0x7f514c0b3c80 2023-10-16T09:49:37.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.991+0000 7f5150ba9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c0c5a60 con 0x7f5134003f70 2023-10-16T09:49:37.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.991+0000 7f5150ba9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514c0c5d40 con 0x7f514c0c7d60 2023-10-16T09:49:37.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.991+0000 7f5149d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f514c0b3c80 0x7f514c0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:37.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.991+0000 7f5149d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f514c0b3c80 0x7f514c0b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34610/0 (socket says 172.21.15.84:34610) 2023-10-16T09:49:37.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.991+0000 7f5149d9b640 1 -- 172.21.15.84:0/1520841433 learned_addr learned my addr 172.21.15.84:0/1520841433 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:37.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.992+0000 7f5149d9b640 1 -- 172.21.15.84:0/1520841433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f514c0c7d60 msgr2=0x7f514c0c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:37.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.992+0000 7f5149d9b640 1 --2- 172.21.15.84:0/1520841433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f514c0c7d60 0x7f514c0c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.992+0000 7f514a59c640 1 --2- 172.21.15.84:0/1520841433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5134003f70 0x7f514c155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:37.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.992+0000 7f5149d9b640 1 -- 172.21.15.84:0/1520841433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5134003f70 msgr2=0x7f514c155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:37.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.992+0000 7f5149d9b640 1 --2- 172.21.15.84:0/1520841433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5134003f70 0x7f514c155820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.992+0000 7f5149d9b640 1 -- 172.21.15.84:0/1520841433 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c0c6020 con 0x7f514c0b3c80 2023-10-16T09:49:37.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.992+0000 7f5149d9b640 1 --2- 172.21.15.84:0/1520841433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f514c0b3c80 0x7f514c0b4100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5140008fd0 tx=0x7f5140001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:37.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.992+0000 7f513affd640 1 -- 172.21.15.84:0/1520841433 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140007a30 con 0x7f514c0b3c80 2023-10-16T09:49:37.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.992+0000 7f513affd640 1 -- 172.21.15.84:0/1520841433 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51400069e0 con 0x7f514c0b3c80 2023-10-16T09:49:37.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.992+0000 7f513affd640 1 -- 172.21.15.84:0/1520841433 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140006d90 con 0x7f514c0b3c80 2023-10-16T09:49:37.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.993+0000 7f5150ba9640 1 -- 172.21.15.84:0/1520841433 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514c0c6300 con 0x7f514c0b3c80 2023-10-16T09:49:37.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.993+0000 7f5150ba9640 1 -- 172.21.15.84:0/1520841433 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514c155d60 con 0x7f514c0b3c80 2023-10-16T09:49:37.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.994+0000 7f5150ba9640 1 -- 172.21.15.84:0/1520841433 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514c0c6950 con 0x7f514c0b3c80 2023-10-16T09:49:37.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.995+0000 7f513affd640 1 -- 172.21.15.84:0/1520841433 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f51400035f0 con 0x7f514c0b3c80 2023-10-16T09:49:37.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.996+0000 7f513affd640 1 --2- 172.21.15.84:0/1520841433 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f512c06ae70 0x7f512c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:37.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.996+0000 7f513affd640 1 -- 172.21.15.84:0/1520841433 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51400ab630 con 0x7f514c0b3c80 2023-10-16T09:49:37.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.996+0000 7f514a59c640 1 --2- 172.21.15.84:0/1520841433 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f512c06ae70 0x7f512c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:37.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:36.997+0000 7f514a59c640 1 --2- 172.21.15.84:0/1520841433 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f512c06ae70 0x7f512c06d330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f513c009910 tx=0x7f513c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:37.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.000+0000 7f513affd640 1 -- 172.21.15.84:0/1520841433 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5140076760 con 0x7f514c0b3c80 2023-10-16T09:49:37.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.192+0000 7f5150ba9640 1 -- 172.21.15.84:0/1520841433 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f514c04e160 con 0x7f514c0b3c80 2023-10-16T09:49:37.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.196+0000 7f513affd640 1 -- 172.21.15.84:0/1520841433 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f5140079d80 con 0x7f514c0b3c80 2023-10-16T09:49:37.198 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:37.198 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":21,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:03.529261+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:37.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.198+0000 7f5150ba9640 1 -- 172.21.15.84:0/1520841433 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f512c06ae70 msgr2=0x7f512c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:37.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.198+0000 7f5150ba9640 1 --2- 172.21.15.84:0/1520841433 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f512c06ae70 0x7f512c06d330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f513c009910 tx=0x7f513c008040 comp rx=0 tx=0).stop 2023-10-16T09:49:37.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.198+0000 7f5150ba9640 1 -- 172.21.15.84:0/1520841433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f514c0b3c80 msgr2=0x7f514c0b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:37.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.198+0000 7f5150ba9640 1 --2- 172.21.15.84:0/1520841433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f514c0b3c80 0x7f514c0b4100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5140008fd0 tx=0x7f5140001120 comp rx=0 tx=0).stop 2023-10-16T09:49:37.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.199+0000 7f5150ba9640 1 -- 172.21.15.84:0/1520841433 shutdown_connections 2023-10-16T09:49:37.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.199+0000 7f5150ba9640 1 --2- 172.21.15.84:0/1520841433 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f512c06ae70 0x7f512c06d330 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.199+0000 7f5150ba9640 1 --2- 172.21.15.84:0/1520841433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f514c0c7d60 0x7f514c0c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.199+0000 7f5150ba9640 1 --2- 172.21.15.84:0/1520841433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f514c0b3c80 0x7f514c0b4100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.199+0000 7f5150ba9640 1 --2- 172.21.15.84:0/1520841433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5134003f70 0x7f514c155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.199+0000 7f5150ba9640 1 -- 172.21.15.84:0/1520841433 >> 172.21.15.84:0/1520841433 conn(0x7f514c0cb410 msgr2=0x7f514c158ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:37.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.199+0000 7f5150ba9640 1 -- 172.21.15.84:0/1520841433 shutdown_connections 2023-10-16T09:49:37.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.199+0000 7f5150ba9640 1 -- 172.21.15.84:0/1520841433 wait complete. 2023-10-16T09:49:37.220 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:49:00.048649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:49:01.361365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:49:37.220 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:49:37.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.437+0000 7f4436a4b640 1 Processor -- start 2023-10-16T09:49:37.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.437+0000 7f4436a4b640 1 -- start start 2023-10-16T09:49:37.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.439+0000 7f4436a4b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f443015ad70 0x7f4430069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:37.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.439+0000 7f4436a4b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f44300c9ca0 0x7f443006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:37.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.439+0000 7f4436a4b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44300cb030 con 0x7f44300ca3b0 2023-10-16T09:49:37.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.439+0000 7f4436a4b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44300c91c0 con 0x7f44300c9ca0 2023-10-16T09:49:37.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.439+0000 7f4436a4b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44300c94a0 con 0x7f443015ad70 2023-10-16T09:49:37.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.439+0000 7f4434fe8640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f44300ca3b0 0x7f44300ca840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53604/0 (socket says 172.21.15.84:53604) 2023-10-16T09:49:37.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.439+0000 7f4434fe8640 1 -- 172.21.15.84:0/1858173333 learned_addr learned my addr 172.21.15.84:0/1858173333 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:37.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.439+0000 7f4427fff640 1 --2- 172.21.15.84:0/1858173333 >> v2:172.21.15.157:3300/0 conn(0x7f44300c9ca0 0x7f443006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:37.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442ffff640 1 --2- 172.21.15.84:0/1858173333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f443015ad70 0x7f4430069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:37.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442f7fe640 1 -- 172.21.15.84:0/1858173333 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1407527218 0 0) 0x7f44300cb030 con 0x7f44300ca3b0 2023-10-16T09:49:37.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442f7fe640 1 -- 172.21.15.84:0/1858173333 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f441c003520 con 0x7f44300ca3b0 2023-10-16T09:49:37.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442f7fe640 1 -- 172.21.15.84:0/1858173333 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1017613518 0 0) 0x7f441c003520 con 0x7f44300ca3b0 2023-10-16T09:49:37.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442f7fe640 1 -- 172.21.15.84:0/1858173333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f443015ad70 msgr2=0x7f4430069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:37.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442f7fe640 1 --2- 172.21.15.84:0/1858173333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f443015ad70 0x7f4430069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442f7fe640 1 -- 172.21.15.84:0/1858173333 >> v2:172.21.15.157:3300/0 conn(0x7f44300c9ca0 msgr2=0x7f443006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:37.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442f7fe640 1 --2- 172.21.15.84:0/1858173333 >> v2:172.21.15.157:3300/0 conn(0x7f44300c9ca0 0x7f443006a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442f7fe640 1 -- 172.21.15.84:0/1858173333 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44300c9780 con 0x7f44300ca3b0 2023-10-16T09:49:37.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442f7fe640 1 -- 172.21.15.84:0/1858173333 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f44280019f0 con 0x7f44300ca3b0 2023-10-16T09:49:37.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442ffff640 1 --2- 172.21.15.84:0/1858173333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f443015ad70 0x7f4430069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:37.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442f7fe640 1 -- 172.21.15.84:0/1858173333 >> v1:172.21.15.84:6789/0 conn(0x7f44300ca3b0 legacy=0x7f44300ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:37.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442f7fe640 1 --2- 172.21.15.84:0/1858173333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f441c003970 0x7f441c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:37.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442f7fe640 1 --2- 172.21.15.84:0/1858173333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f443015ad70 0x7f441c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:37.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442f7fe640 1 --2- 172.21.15.84:0/1858173333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f441c004d00 0x7f441c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:37.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442f7fe640 1 -- 172.21.15.84:0/1858173333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44300c91c0 con 0x7f441c003970 2023-10-16T09:49:37.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442f7fe640 1 -- 172.21.15.84:0/1858173333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44300c94a0 con 0x7f441c004d00 2023-10-16T09:49:37.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442f7fe640 1 -- 172.21.15.84:0/1858173333 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441c00b980 con 0x7f443015ad70 2023-10-16T09:49:37.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.440+0000 7f442ffff640 1 --2- 172.21.15.84:0/1858173333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f441c003970 0x7f441c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:37.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.441+0000 7f442ffff640 1 -- 172.21.15.84:0/1858173333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f443015ad70 msgr2=0x7f441c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:37.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.441+0000 7f442ffff640 1 --2- 172.21.15.84:0/1858173333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f443015ad70 0x7f441c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.441+0000 7f442ffff640 1 -- 172.21.15.84:0/1858173333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f441c004d00 msgr2=0x7f441c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:37.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.441+0000 7f442ffff640 1 --2- 172.21.15.84:0/1858173333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f441c004d00 0x7f441c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.441+0000 7f442ffff640 1 -- 172.21.15.84:0/1858173333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c003520 con 0x7f441c003970 2023-10-16T09:49:37.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.442+0000 7f442ffff640 1 --2- 172.21.15.84:0/1858173333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f441c003970 0x7f441c003d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f44200098a0 tx=0x7f441c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9724d0ef87915a4f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:37.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.442+0000 7f442f7fe640 1 -- 172.21.15.84:0/1858173333 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4420004230 con 0x7f441c003970 2023-10-16T09:49:37.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.442+0000 7f442f7fe640 1 -- 172.21.15.84:0/1858173333 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4420004390 con 0x7f441c003970 2023-10-16T09:49:37.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.442+0000 7f442f7fe640 1 -- 172.21.15.84:0/1858173333 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4420003830 con 0x7f441c003970 2023-10-16T09:49:37.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.443+0000 7f4436a4b640 1 -- 172.21.15.84:0/1858173333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f441c003970 msgr2=0x7f441c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:37.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.443+0000 7f4436a4b640 1 --2- 172.21.15.84:0/1858173333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f441c003970 0x7f441c003d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f44200098a0 tx=0x7f441c0057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.443+0000 7f4434fe8640 1 -- 172.21.15.84:0/1858173333 reap_dead start 2023-10-16T09:49:37.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.443+0000 7f4436a4b640 1 -- 172.21.15.84:0/1858173333 shutdown_connections 2023-10-16T09:49:37.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.443+0000 7f4436a4b640 1 -- 172.21.15.84:0/1858173333 >> 172.21.15.84:0/1858173333 conn(0x7f44300ce9b0 msgr2=0x7f44300cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:37.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.443+0000 7f4436a4b640 1 -- 172.21.15.84:0/1858173333 shutdown_connections 2023-10-16T09:49:37.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.443+0000 7f4436a4b640 1 -- 172.21.15.84:0/1858173333 wait complete. 2023-10-16T09:49:37.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.445+0000 7f4436a4b640 1 Processor -- start 2023-10-16T09:49:37.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.445+0000 7f4436a4b640 1 -- start start 2023-10-16T09:49:37.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.446+0000 7f4436a4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f441c003970 0x7f44300c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:37.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.446+0000 7f4436a4b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44300c3750 0x7f443014eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:37.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.446+0000 7f4436a4b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44300c77c0 0x7f4430158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:37.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.446+0000 7f4436a4b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44300c53e0 con 0x7f44300c3750 2023-10-16T09:49:37.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.446+0000 7f4436a4b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44300c88a0 con 0x7f441c003970 2023-10-16T09:49:37.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.446+0000 7f4436a4b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44300c85c0 con 0x7f44300c77c0 2023-10-16T09:49:37.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.446+0000 7f4427fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44300c3750 0x7f443014eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:37.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.446+0000 7f4427fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44300c3750 0x7f443014eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34634/0 (socket says 172.21.15.84:34634) 2023-10-16T09:49:37.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.446+0000 7f4427fff640 1 -- 172.21.15.84:0/997448448 learned_addr learned my addr 172.21.15.84:0/997448448 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:37.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.446+0000 7f4427fff640 1 -- 172.21.15.84:0/997448448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44300c77c0 msgr2=0x7f4430158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:37.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.446+0000 7f4427fff640 1 --2- 172.21.15.84:0/997448448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44300c77c0 0x7f4430158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.446+0000 7f4427fff640 1 -- 172.21.15.84:0/997448448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f441c003970 msgr2=0x7f44300c4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:37.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.446+0000 7f442ffff640 1 --2- 172.21.15.84:0/997448448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f441c003970 0x7f44300c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:37.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.446+0000 7f4427fff640 1 --2- 172.21.15.84:0/997448448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f441c003970 0x7f44300c4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.446+0000 7f4427fff640 1 -- 172.21.15.84:0/997448448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44300c6ce0 con 0x7f44300c3750 2023-10-16T09:49:37.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.447+0000 7f4427fff640 1 --2- 172.21.15.84:0/997448448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44300c3750 0x7f443014eb20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f441800e860 tx=0x7f441800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:37.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.447+0000 7f442d7fa640 1 -- 172.21.15.84:0/997448448 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418009600 con 0x7f44300c3750 2023-10-16T09:49:37.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.447+0000 7f442d7fa640 1 -- 172.21.15.84:0/997448448 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441800c940 con 0x7f44300c3750 2023-10-16T09:49:37.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.447+0000 7f4436a4b640 1 -- 172.21.15.84:0/997448448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44300c6fc0 con 0x7f44300c3750 2023-10-16T09:49:37.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.447+0000 7f442d7fa640 1 -- 172.21.15.84:0/997448448 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441800ccd0 con 0x7f44300c3750 2023-10-16T09:49:37.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.447+0000 7f4436a4b640 1 -- 172.21.15.84:0/997448448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443014f060 con 0x7f44300c3750 2023-10-16T09:49:37.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.448+0000 7f4436a4b640 1 -- 172.21.15.84:0/997448448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f443004e160 con 0x7f44300c3750 2023-10-16T09:49:37.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.449+0000 7f442d7fa640 1 -- 172.21.15.84:0/997448448 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4418013470 con 0x7f44300c3750 2023-10-16T09:49:37.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.450+0000 7f442d7fa640 1 --2- 172.21.15.84:0/997448448 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f441006ae70 0x7f441006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:37.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.450+0000 7f442d7fa640 1 -- 172.21.15.84:0/997448448 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4418089740 con 0x7f44300c3750 2023-10-16T09:49:37.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.450+0000 7f442ffff640 1 --2- 172.21.15.84:0/997448448 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f441006ae70 0x7f441006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:37.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.451+0000 7f442ffff640 1 --2- 172.21.15.84:0/997448448 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f441006ae70 0x7f441006d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4420035000 tx=0x7f442003a040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:37.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.454+0000 7f442d7fa640 1 -- 172.21.15.84:0/997448448 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f44180547e0 con 0x7f44300c3750 2023-10-16T09:49:37.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.647+0000 7f4436a4b640 1 -- 172.21.15.84:0/997448448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f44300736e0 con 0x7f44300c3750 2023-10-16T09:49:37.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.647+0000 7f442d7fa640 1 -- 172.21.15.84:0/997448448 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f44180170a0 con 0x7f44300c3750 2023-10-16T09:49:37.650 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T09:49:37.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.650+0000 7f4436a4b640 1 -- 172.21.15.84:0/997448448 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f441006ae70 msgr2=0x7f441006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:37.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.650+0000 7f4436a4b640 1 --2- 172.21.15.84:0/997448448 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f441006ae70 0x7f441006d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4420035000 tx=0x7f442003a040 comp rx=0 tx=0).stop 2023-10-16T09:49:37.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.650+0000 7f4436a4b640 1 -- 172.21.15.84:0/997448448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44300c3750 msgr2=0x7f443014eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:37.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.650+0000 7f4436a4b640 1 --2- 172.21.15.84:0/997448448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44300c3750 0x7f443014eb20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f441800e860 tx=0x7f441800ed20 comp rx=0 tx=0).stop 2023-10-16T09:49:37.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.651+0000 7f4436a4b640 1 -- 172.21.15.84:0/997448448 shutdown_connections 2023-10-16T09:49:37.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.651+0000 7f4436a4b640 1 --2- 172.21.15.84:0/997448448 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f441006ae70 0x7f441006d330 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.651+0000 7f4436a4b640 1 --2- 172.21.15.84:0/997448448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44300c77c0 0x7f4430158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.651+0000 7f4436a4b640 1 --2- 172.21.15.84:0/997448448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44300c3750 0x7f443014eb20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.651+0000 7f4436a4b640 1 --2- 172.21.15.84:0/997448448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f441c003970 0x7f44300c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.651+0000 7f4436a4b640 1 -- 172.21.15.84:0/997448448 >> 172.21.15.84:0/997448448 conn(0x7f44300ce9b0 msgr2=0x7f44300cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:37.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.651+0000 7f4436a4b640 1 -- 172.21.15.84:0/997448448 shutdown_connections 2023-10-16T09:49:37.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.651+0000 7f4436a4b640 1 -- 172.21.15.84:0/997448448 wait complete. 2023-10-16T09:49:37.671 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:49:37.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.878+0000 7f59559ac640 1 Processor -- start 2023-10-16T09:49:37.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.878+0000 7f59559ac640 1 -- start start 2023-10-16T09:49:37.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.879+0000 7f59559ac640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f595015c9f0 0x7f59500c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:37.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.879+0000 7f59559ac640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5950160a90 con 0x7f59500c9230 2023-10-16T09:49:37.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.879+0000 7f59559ac640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59500cd900 con 0x7f595015dde0 2023-10-16T09:49:37.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.879+0000 7f59559ac640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59500cdec0 con 0x7f595015c9f0 2023-10-16T09:49:37.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.880+0000 7f59549aa640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f595015c9f0 0x7f59500c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:37.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.880+0000 7f59549aa640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f595015c9f0 0x7f59500c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34636/0 (socket says 172.21.15.84:34636) 2023-10-16T09:49:37.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.880+0000 7f59549aa640 1 -- 172.21.15.84:0/2117613508 learned_addr learned my addr 172.21.15.84:0/2117613508 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:37.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.880+0000 7f594ffff640 1 --1- 172.21.15.84:0/2117613508 >> v1:172.21.15.84:6789/0 conn(0x7f59500c9230 0x7f59500709e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53606/0 (socket says 172.21.15.84:53606) 2023-10-16T09:49:37.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.880+0000 7f59549aa640 1 -- 172.21.15.84:0/2117613508 >> v1:172.21.15.157:6789/0 conn(0x7f595015dde0 legacy=0x7f59500ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:37.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.880+0000 7f59549aa640 1 -- 172.21.15.84:0/2117613508 >> v1:172.21.15.84:6789/0 conn(0x7f59500c9230 legacy=0x7f59500709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:37.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.880+0000 7f59549aa640 1 -- 172.21.15.84:0/2117613508 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59500cdbe0 con 0x7f595015c9f0 2023-10-16T09:49:37.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.880+0000 7f594f7fe640 1 -- 172.21.15.84:0/2117613508 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409183892 0 0) 0x7f5950160a90 con 0x7f59500c9230 2023-10-16T09:49:37.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.881+0000 7f59549aa640 1 --2- 172.21.15.84:0/2117613508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f595015c9f0 0x7f59500c8cf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5944009870 tx=0x7f594402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e40631fb160684 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:37.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.881+0000 7f594f7fe640 1 -- 172.21.15.84:0/2117613508 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5944030860 con 0x7f595015c9f0 2023-10-16T09:49:37.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.882+0000 7f594f7fe640 1 -- 172.21.15.84:0/2117613508 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59440309c0 con 0x7f595015c9f0 2023-10-16T09:49:37.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.882+0000 7f594f7fe640 1 -- 172.21.15.84:0/2117613508 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5944030d50 con 0x7f595015c9f0 2023-10-16T09:49:37.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.882+0000 7f59559ac640 1 -- 172.21.15.84:0/2117613508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f595015c9f0 msgr2=0x7f59500c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:37.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.882+0000 7f59559ac640 1 --2- 172.21.15.84:0/2117613508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f595015c9f0 0x7f59500c8cf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5944009870 tx=0x7f594402b9a0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.882+0000 7f59559ac640 1 -- 172.21.15.84:0/2117613508 shutdown_connections 2023-10-16T09:49:37.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.882+0000 7f59559ac640 1 --2- 172.21.15.84:0/2117613508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f595015c9f0 0x7f59500c8cf0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.882+0000 7f59559ac640 1 -- 172.21.15.84:0/2117613508 >> 172.21.15.84:0/2117613508 conn(0x7f59500ce6a0 msgr2=0x7f59500ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:37.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.882+0000 7f59559ac640 1 -- 172.21.15.84:0/2117613508 shutdown_connections 2023-10-16T09:49:37.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.882+0000 7f59559ac640 1 -- 172.21.15.84:0/2117613508 wait complete. 2023-10-16T09:49:37.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.884+0000 7f59559ac640 1 Processor -- start 2023-10-16T09:49:37.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.884+0000 7f59559ac640 1 -- start start 2023-10-16T09:49:37.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.885+0000 7f59559ac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59500c9230 0x7f59501f5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:37.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.885+0000 7f59559ac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f595015c9f0 0x7f59501f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:37.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.885+0000 7f59559ac640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f595015dde0 0x7f595006a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:37.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.885+0000 7f59559ac640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5950148f90 con 0x7f59500c9230 2023-10-16T09:49:37.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.885+0000 7f59559ac640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595006b6a0 con 0x7f595015c9f0 2023-10-16T09:49:37.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.885+0000 7f59559ac640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595006b980 con 0x7f595015dde0 2023-10-16T09:49:37.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.885+0000 7f59549aa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59500c9230 0x7f59501f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:37.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.885+0000 7f59549aa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59500c9230 0x7f59501f5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34642/0 (socket says 172.21.15.84:34642) 2023-10-16T09:49:37.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.885+0000 7f59549aa640 1 -- 172.21.15.84:0/994848948 learned_addr learned my addr 172.21.15.84:0/994848948 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:37.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.886+0000 7f59549aa640 1 -- 172.21.15.84:0/994848948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f595015dde0 msgr2=0x7f595006a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:37.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.886+0000 7f59549aa640 1 --2- 172.21.15.84:0/994848948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f595015dde0 0x7f595006a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.886+0000 7f59549aa640 1 -- 172.21.15.84:0/994848948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f595015c9f0 msgr2=0x7f59501f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:37.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.886+0000 7f59549aa640 1 --2- 172.21.15.84:0/994848948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f595015c9f0 0x7f59501f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:37.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.886+0000 7f59549aa640 1 -- 172.21.15.84:0/994848948 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59440094d0 con 0x7f59500c9230 2023-10-16T09:49:37.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.886+0000 7f59549aa640 1 --2- 172.21.15.84:0/994848948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59500c9230 0x7f59501f5730 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5944009870 tx=0x7f5944004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:37.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.886+0000 7f594d7fa640 1 -- 172.21.15.84:0/994848948 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5944030810 con 0x7f59500c9230 2023-10-16T09:49:37.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.886+0000 7f594d7fa640 1 -- 172.21.15.84:0/994848948 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5944043400 con 0x7f59500c9230 2023-10-16T09:49:37.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.887+0000 7f59559ac640 1 -- 172.21.15.84:0/994848948 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59501488d0 con 0x7f59500c9230 2023-10-16T09:49:37.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.887+0000 7f594d7fa640 1 -- 172.21.15.84:0/994848948 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5944043790 con 0x7f59500c9230 2023-10-16T09:49:37.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.887+0000 7f59559ac640 1 -- 172.21.15.84:0/994848948 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59501482b0 con 0x7f59500c9230 2023-10-16T09:49:37.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.888+0000 7f59559ac640 1 -- 172.21.15.84:0/994848948 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595006bbc0 con 0x7f59500c9230 2023-10-16T09:49:37.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.889+0000 7f594d7fa640 1 -- 172.21.15.84:0/994848948 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f59440438f0 con 0x7f59500c9230 2023-10-16T09:49:37.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.890+0000 7f594d7fa640 1 --2- 172.21.15.84:0/994848948 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f593c06aea0 0x7f593c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:37.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.890+0000 7f594d7fa640 1 -- 172.21.15.84:0/994848948 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f594403e070 con 0x7f59500c9230 2023-10-16T09:49:37.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.890+0000 7f594ffff640 1 --2- 172.21.15.84:0/994848948 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f593c06aea0 0x7f593c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:37.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.891+0000 7f594ffff640 1 --2- 172.21.15.84:0/994848948 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f593c06aea0 0x7f593c06d360 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f59400011e0 tx=0x7f5940007940 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:37.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:37.894+0000 7f594d7fa640 1 -- 172.21.15.84:0/994848948 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5944077c30 con 0x7f59500c9230 2023-10-16T09:49:38.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.083+0000 7f59559ac640 1 -- 172.21.15.84:0/994848948 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f595004e9b0 con 0x7f59500c9230 2023-10-16T09:49:38.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.084+0000 7f594d7fa640 1 -- 172.21.15.84:0/994848948 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5944034020 con 0x7f59500c9230 2023-10-16T09:49:38.086 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T09:49:38.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.087+0000 7f59559ac640 1 -- 172.21.15.84:0/994848948 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f593c06aea0 msgr2=0x7f593c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:38.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.087+0000 7f59559ac640 1 --2- 172.21.15.84:0/994848948 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f593c06aea0 0x7f593c06d360 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f59400011e0 tx=0x7f5940007940 comp rx=0 tx=0).stop 2023-10-16T09:49:38.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.087+0000 7f59559ac640 1 -- 172.21.15.84:0/994848948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59500c9230 msgr2=0x7f59501f5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:38.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.087+0000 7f59559ac640 1 --2- 172.21.15.84:0/994848948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59500c9230 0x7f59501f5730 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5944009870 tx=0x7f5944004730 comp rx=0 tx=0).stop 2023-10-16T09:49:38.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.087+0000 7f59559ac640 1 -- 172.21.15.84:0/994848948 shutdown_connections 2023-10-16T09:49:38.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.087+0000 7f59559ac640 1 --2- 172.21.15.84:0/994848948 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f593c06aea0 0x7f593c06d360 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.087+0000 7f59559ac640 1 --2- 172.21.15.84:0/994848948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f595015dde0 0x7f595006a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.087+0000 7f59559ac640 1 --2- 172.21.15.84:0/994848948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f595015c9f0 0x7f59501f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.087+0000 7f59559ac640 1 --2- 172.21.15.84:0/994848948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59500c9230 0x7f59501f5730 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.087+0000 7f59559ac640 1 -- 172.21.15.84:0/994848948 >> 172.21.15.84:0/994848948 conn(0x7f59500ce6a0 msgr2=0x7f595015ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:38.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.087+0000 7f59559ac640 1 -- 172.21.15.84:0/994848948 shutdown_connections 2023-10-16T09:49:38.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.087+0000 7f59559ac640 1 -- 172.21.15.84:0/994848948 wait complete. 2023-10-16T09:49:38.107 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:49:38.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.318+0000 7f7fa80d4640 1 Processor -- start 2023-10-16T09:49:38.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.318+0000 7f7fa80d4640 1 -- start start 2023-10-16T09:49:38.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.320+0000 7f7fa80d4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7fa015ad70 0x7f7fa0069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:38.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.320+0000 7f7fa80d4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7fa00c9ca0 0x7f7fa006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:38.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.320+0000 7f7fa80d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fa00ca3b0 0x7f7fa00ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:38.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.320+0000 7f7fa80d4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa00cad80 con 0x7f7fa00c9ca0 2023-10-16T09:49:38.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.320+0000 7f7fa80d4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa00cb030 con 0x7f7fa015ad70 2023-10-16T09:49:38.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.320+0000 7f7fa80d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa00c91c0 con 0x7f7fa00ca3b0 2023-10-16T09:49:38.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.320+0000 7f7fa566f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7fa00c9ca0 0x7f7fa006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:38.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.320+0000 7f7fa566f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7fa00c9ca0 0x7f7fa006a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34654/0 (socket says 172.21.15.84:34654) 2023-10-16T09:49:38.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.320+0000 7f7fa566f640 1 -- 172.21.15.84:0/3063296760 learned_addr learned my addr 172.21.15.84:0/3063296760 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:38.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.321+0000 7f7fa6671640 1 --2- 172.21.15.84:0/3063296760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fa00ca3b0 0x7f7fa00ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:38.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.321+0000 7f7fa5e70640 1 --2- 172.21.15.84:0/3063296760 >> v2:172.21.15.157:3300/0 conn(0x7f7fa015ad70 0x7f7fa0069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:38.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.321+0000 7f7fa566f640 1 -- 172.21.15.84:0/3063296760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fa00ca3b0 msgr2=0x7f7fa00ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:38.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.321+0000 7f7fa566f640 1 --2- 172.21.15.84:0/3063296760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fa00ca3b0 0x7f7fa00ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.321+0000 7f7fa566f640 1 -- 172.21.15.84:0/3063296760 >> v2:172.21.15.157:3300/0 conn(0x7f7fa015ad70 msgr2=0x7f7fa0069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:38.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.321+0000 7f7fa566f640 1 --2- 172.21.15.84:0/3063296760 >> v2:172.21.15.157:3300/0 conn(0x7f7fa015ad70 0x7f7fa0069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.321+0000 7f7fa566f640 1 -- 172.21.15.84:0/3063296760 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa00c94a0 con 0x7f7fa00c9ca0 2023-10-16T09:49:38.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.322+0000 7f7fa566f640 1 --2- 172.21.15.84:0/3063296760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fa00c9ca0 0x7f7fa006a2e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00cf780 tx=0x7f7f9002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa9401ff0bfa57f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:38.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.322+0000 7f7fa4e6e640 1 -- 172.21.15.84:0/3063296760 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90031870 con 0x7f7fa00c9ca0 2023-10-16T09:49:38.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.322+0000 7f7fa4e6e640 1 -- 172.21.15.84:0/3063296760 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f900319d0 con 0x7f7fa00c9ca0 2023-10-16T09:49:38.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.322+0000 7f7fa4e6e640 1 -- 172.21.15.84:0/3063296760 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90031d60 con 0x7f7fa00c9ca0 2023-10-16T09:49:38.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.323+0000 7f7fa80d4640 1 -- 172.21.15.84:0/3063296760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fa00c9ca0 msgr2=0x7f7fa006a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:38.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.323+0000 7f7fa80d4640 1 --2- 172.21.15.84:0/3063296760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fa00c9ca0 0x7f7fa006a2e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00cf780 tx=0x7f7f9002b910 comp rx=0 tx=0).stop 2023-10-16T09:49:38.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.323+0000 7f7fa80d4640 1 -- 172.21.15.84:0/3063296760 shutdown_connections 2023-10-16T09:49:38.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.323+0000 7f7fa80d4640 1 --2- 172.21.15.84:0/3063296760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fa00c9ca0 0x7f7fa006a2e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.323+0000 7f7fa80d4640 1 --2- 172.21.15.84:0/3063296760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fa00ca3b0 0x7f7fa00ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.323+0000 7f7fa80d4640 1 --2- 172.21.15.84:0/3063296760 >> v2:172.21.15.157:3300/0 conn(0x7f7fa015ad70 0x7f7fa0069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.323+0000 7f7fa80d4640 1 -- 172.21.15.84:0/3063296760 >> 172.21.15.84:0/3063296760 conn(0x7f7fa00ce9b0 msgr2=0x7f7fa00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:38.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.323+0000 7f7fa80d4640 1 -- 172.21.15.84:0/3063296760 shutdown_connections 2023-10-16T09:49:38.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.323+0000 7f7fa80d4640 1 -- 172.21.15.84:0/3063296760 wait complete. 2023-10-16T09:49:38.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.325+0000 7f7fa80d4640 1 Processor -- start 2023-10-16T09:49:38.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.325+0000 7f7fa80d4640 1 -- start start 2023-10-16T09:49:38.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.326+0000 7f7fa80d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fa00c9ca0 0x7f7fa00c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:38.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.326+0000 7f7fa80d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fa00ca3b0 0x7f7fa00c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:38.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.326+0000 7f7fa80d4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fa015ad70 0x7f7fa01593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:38.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.326+0000 7f7fa80d4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa00c6ea0 con 0x7f7fa015ad70 2023-10-16T09:49:38.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.326+0000 7f7fa80d4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa00c5fc0 con 0x7f7fa00c9ca0 2023-10-16T09:49:38.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.326+0000 7f7fa80d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa00c6290 con 0x7f7fa00ca3b0 2023-10-16T09:49:38.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.326+0000 7f7fa6671640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fa015ad70 0x7f7fa01593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:38.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.326+0000 7f7fa6671640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fa015ad70 0x7f7fa01593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34660/0 (socket says 172.21.15.84:34660) 2023-10-16T09:49:38.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.326+0000 7f7fa6671640 1 -- 172.21.15.84:0/2592743344 learned_addr learned my addr 172.21.15.84:0/2592743344 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:38.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.326+0000 7f7fa5e70640 1 --2- 172.21.15.84:0/2592743344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fa00c9ca0 0x7f7fa00c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:38.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.326+0000 7f7fa566f640 1 --2- 172.21.15.84:0/2592743344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fa00ca3b0 0x7f7fa00c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:38.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.326+0000 7f7fa6671640 1 -- 172.21.15.84:0/2592743344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fa00ca3b0 msgr2=0x7f7fa00c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:38.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.326+0000 7f7fa6671640 1 --2- 172.21.15.84:0/2592743344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fa00ca3b0 0x7f7fa00c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.326+0000 7f7fa6671640 1 -- 172.21.15.84:0/2592743344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fa00c9ca0 msgr2=0x7f7fa00c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:38.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.326+0000 7f7fa6671640 1 --2- 172.21.15.84:0/2592743344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fa00c9ca0 0x7f7fa00c5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.326+0000 7f7fa6671640 1 -- 172.21.15.84:0/2592743344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f900094d0 con 0x7f7fa015ad70 2023-10-16T09:49:38.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.327+0000 7f7fa6671640 1 --2- 172.21.15.84:0/2592743344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fa015ad70 0x7f7fa01593d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7f98007e60 tx=0x7f7f9800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:38.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.327+0000 7f7f967fc640 1 -- 172.21.15.84:0/2592743344 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98014620 con 0x7f7fa015ad70 2023-10-16T09:49:38.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.327+0000 7f7f967fc640 1 -- 172.21.15.84:0/2592743344 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9800ee90 con 0x7f7fa015ad70 2023-10-16T09:49:38.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.327+0000 7f7f967fc640 1 -- 172.21.15.84:0/2592743344 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f980149e0 con 0x7f7fa015ad70 2023-10-16T09:49:38.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.327+0000 7f7fa80d4640 1 -- 172.21.15.84:0/2592743344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa00c6570 con 0x7f7fa015ad70 2023-10-16T09:49:38.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.327+0000 7f7fa80d4640 1 -- 172.21.15.84:0/2592743344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa00c6b30 con 0x7f7fa015ad70 2023-10-16T09:49:38.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.328+0000 7f7fa80d4640 1 -- 172.21.15.84:0/2592743344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa00c7010 con 0x7f7fa015ad70 2023-10-16T09:49:38.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.329+0000 7f7f967fc640 1 -- 172.21.15.84:0/2592743344 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7f98014b40 con 0x7f7fa015ad70 2023-10-16T09:49:38.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.330+0000 7f7f967fc640 1 --2- 172.21.15.84:0/2592743344 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7f8406aef0 0x7f7f8406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:38.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.330+0000 7f7f967fc640 1 -- 172.21.15.84:0/2592743344 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f98089fd0 con 0x7f7fa015ad70 2023-10-16T09:49:38.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.330+0000 7f7fa5e70640 1 --2- 172.21.15.84:0/2592743344 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7f8406aef0 0x7f7f8406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:38.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.331+0000 7f7fa5e70640 1 --2- 172.21.15.84:0/2592743344 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7f8406aef0 0x7f7f8406d3b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c006fd0 tx=0x7f7f9c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:38.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.334+0000 7f7f967fc640 1 -- 172.21.15.84:0/2592743344 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7f98054fd0 con 0x7f7fa015ad70 2023-10-16T09:49:38.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.526+0000 7f7fa80d4640 1 -- 172.21.15.84:0/2592743344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7fa004e160 con 0x7f7fa015ad70 2023-10-16T09:49:38.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.527+0000 7f7f967fc640 1 -- 172.21.15.84:0/2592743344 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7f980585f0 con 0x7f7fa015ad70 2023-10-16T09:49:38.529 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T09:49:38.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.530+0000 7f7fa80d4640 1 -- 172.21.15.84:0/2592743344 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7f8406aef0 msgr2=0x7f7f8406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:38.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.530+0000 7f7fa80d4640 1 --2- 172.21.15.84:0/2592743344 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7f8406aef0 0x7f7f8406d3b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c006fd0 tx=0x7f7f9c008040 comp rx=0 tx=0).stop 2023-10-16T09:49:38.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.530+0000 7f7fa80d4640 1 -- 172.21.15.84:0/2592743344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fa015ad70 msgr2=0x7f7fa01593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:38.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.530+0000 7f7fa80d4640 1 --2- 172.21.15.84:0/2592743344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fa015ad70 0x7f7fa01593d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7f98007e60 tx=0x7f7f9800e4b0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.530+0000 7f7fa80d4640 1 -- 172.21.15.84:0/2592743344 shutdown_connections 2023-10-16T09:49:38.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.530+0000 7f7fa80d4640 1 --2- 172.21.15.84:0/2592743344 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7f8406aef0 0x7f7f8406d3b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.530+0000 7f7fa80d4640 1 --2- 172.21.15.84:0/2592743344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fa015ad70 0x7f7fa01593d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.530+0000 7f7fa80d4640 1 --2- 172.21.15.84:0/2592743344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7fa00ca3b0 0x7f7fa00c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.530+0000 7f7fa80d4640 1 --2- 172.21.15.84:0/2592743344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fa00c9ca0 0x7f7fa00c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.530+0000 7f7fa80d4640 1 -- 172.21.15.84:0/2592743344 >> 172.21.15.84:0/2592743344 conn(0x7f7fa00ce9b0 msgr2=0x7f7fa00ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:38.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.530+0000 7f7fa80d4640 1 -- 172.21.15.84:0/2592743344 shutdown_connections 2023-10-16T09:49:38.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.530+0000 7f7fa80d4640 1 -- 172.21.15.84:0/2592743344 wait complete. 2023-10-16T09:49:38.551 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:38.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.758+0000 7f7e382d4640 1 Processor -- start 2023-10-16T09:49:38.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.758+0000 7f7e382d4640 1 -- start start 2023-10-16T09:49:38.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.759+0000 7f7e382d4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7e301615f0 0x7f7e3015db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:38.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.760+0000 7f7e382d4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e3015e3e0 con 0x7f7e300cd0c0 2023-10-16T09:49:38.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.760+0000 7f7e382d4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e300cd4a0 con 0x7f7e300b0b20 2023-10-16T09:49:38.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.760+0000 7f7e382d4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3015e100 con 0x7f7e301615f0 2023-10-16T09:49:38.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.760+0000 7f7e36070640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7e300cd0c0 0x7f7e300b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53614/0 (socket says 172.21.15.84:53614) 2023-10-16T09:49:38.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.760+0000 7f7e36070640 1 -- 172.21.15.84:0/1785084122 learned_addr learned my addr 172.21.15.84:0/1785084122 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:38.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.760+0000 7f7e36871640 1 --2- 172.21.15.84:0/1785084122 >> v2:172.21.15.157:3300/0 conn(0x7f7e301615f0 0x7f7e3015db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:38.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.760+0000 7f7e3506e640 1 -- 172.21.15.84:0/1785084122 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 41359066 0 0) 0x7f7e3015e3e0 con 0x7f7e300cd0c0 2023-10-16T09:49:38.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.760+0000 7f7e3506e640 1 -- 172.21.15.84:0/1785084122 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e1c003520 con 0x7f7e300cd0c0 2023-10-16T09:49:38.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.761+0000 7f7e3506e640 1 -- 172.21.15.84:0/1785084122 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 711211724 0 0) 0x7f7e1c003520 con 0x7f7e300cd0c0 2023-10-16T09:49:38.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.761+0000 7f7e3506e640 1 -- 172.21.15.84:0/1785084122 >> v2:172.21.15.157:3300/0 conn(0x7f7e301615f0 msgr2=0x7f7e3015db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:38.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.761+0000 7f7e3506e640 1 --2- 172.21.15.84:0/1785084122 >> v2:172.21.15.157:3300/0 conn(0x7f7e301615f0 0x7f7e3015db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.761+0000 7f7e3506e640 1 -- 172.21.15.84:0/1785084122 >> v1:172.21.15.157:6789/0 conn(0x7f7e300b0b20 legacy=0x7f7e3015d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:38.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.761+0000 7f7e3506e640 1 -- 172.21.15.84:0/1785084122 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e30160b10 con 0x7f7e300cd0c0 2023-10-16T09:49:38.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.761+0000 7f7e3506e640 1 -- 172.21.15.84:0/1785084122 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f7e2c001670 con 0x7f7e300cd0c0 2023-10-16T09:49:38.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.761+0000 7f7e3506e640 1 -- 172.21.15.84:0/1785084122 >> v1:172.21.15.84:6789/0 conn(0x7f7e300cd0c0 legacy=0x7f7e300b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:38.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.761+0000 7f7e3506e640 1 --2- 172.21.15.84:0/1785084122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e1c0039c0 0x7f7e1c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:38.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.761+0000 7f7e3506e640 1 --2- 172.21.15.84:0/1785084122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e1c0047c0 0x7f7e1c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:38.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.761+0000 7f7e3506e640 1 --2- 172.21.15.84:0/1785084122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c0051d0 0x7f7e1c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:38.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.761+0000 7f7e3506e640 1 -- 172.21.15.84:0/1785084122 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3015e100 con 0x7f7e1c0047c0 2023-10-16T09:49:38.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.761+0000 7f7e3506e640 1 -- 172.21.15.84:0/1785084122 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c00c120 con 0x7f7e1c0051d0 2023-10-16T09:49:38.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.761+0000 7f7e3506e640 1 -- 172.21.15.84:0/1785084122 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c00c2e0 con 0x7f7e1c0039c0 2023-10-16T09:49:38.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.761+0000 7f7e3586f640 1 --2- 172.21.15.84:0/1785084122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e1c0047c0 0x7f7e1c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:38.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.762+0000 7f7e36070640 1 --2- 172.21.15.84:0/1785084122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e1c0039c0 0x7f7e1c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:38.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.762+0000 7f7e36871640 1 --2- 172.21.15.84:0/1785084122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c0051d0 0x7f7e1c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:38.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.762+0000 7f7e3586f640 1 -- 172.21.15.84:0/1785084122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e1c0039c0 msgr2=0x7f7e1c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:38.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.762+0000 7f7e3586f640 1 --2- 172.21.15.84:0/1785084122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e1c0039c0 0x7f7e1c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.762+0000 7f7e3586f640 1 -- 172.21.15.84:0/1785084122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c0051d0 msgr2=0x7f7e1c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:38.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.762+0000 7f7e3586f640 1 --2- 172.21.15.84:0/1785084122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c0051d0 0x7f7e1c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.762+0000 7f7e36871640 1 -- 172.21.15.84:0/1785084122 reap_dead start 2023-10-16T09:49:38.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.762+0000 7f7e3586f640 1 -- 172.21.15.84:0/1785084122 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e1c003520 con 0x7f7e1c0047c0 2023-10-16T09:49:38.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.763+0000 7f7e3586f640 1 --2- 172.21.15.84:0/1785084122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e1c0047c0 0x7f7e1c004c40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7e20001720 tx=0x7f7e1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6bb57214fccbe0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:38.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.763+0000 7f7e3506e640 1 -- 172.21.15.84:0/1785084122 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e200052d0 con 0x7f7e1c0047c0 2023-10-16T09:49:38.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.763+0000 7f7e3506e640 1 -- 172.21.15.84:0/1785084122 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2002e050 con 0x7f7e1c0047c0 2023-10-16T09:49:38.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.763+0000 7f7e3506e640 1 -- 172.21.15.84:0/1785084122 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e20003830 con 0x7f7e1c0047c0 2023-10-16T09:49:38.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.763+0000 7f7e382d4640 1 -- 172.21.15.84:0/1785084122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e1c0047c0 msgr2=0x7f7e1c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:38.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.763+0000 7f7e382d4640 1 --2- 172.21.15.84:0/1785084122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e1c0047c0 0x7f7e1c004c40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7e20001720 tx=0x7f7e1c0057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.764+0000 7f7e382d4640 1 -- 172.21.15.84:0/1785084122 shutdown_connections 2023-10-16T09:49:38.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.764+0000 7f7e382d4640 1 --2- 172.21.15.84:0/1785084122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e1c0047c0 0x7f7e1c004c40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.764+0000 7f7e382d4640 1 -- 172.21.15.84:0/1785084122 >> 172.21.15.84:0/1785084122 conn(0x7f7e300cb410 msgr2=0x7f7e300cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:38.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.764+0000 7f7e382d4640 1 -- 172.21.15.84:0/1785084122 shutdown_connections 2023-10-16T09:49:38.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.764+0000 7f7e382d4640 1 -- 172.21.15.84:0/1785084122 wait complete. 2023-10-16T09:49:38.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.765+0000 7f7e382d4640 1 Processor -- start 2023-10-16T09:49:38.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.765+0000 7f7e382d4640 1 -- start start 2023-10-16T09:49:38.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.766+0000 7f7e382d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c0047c0 0x7f7e301f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:38.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.766+0000 7f7e382d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e300b1550 0x7f7e300b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:38.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.766+0000 7f7e382d4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e3015f5e0 0x7f7e3015fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:38.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.766+0000 7f7e382d4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3015ffa0 con 0x7f7e3015f5e0 2023-10-16T09:49:38.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.766+0000 7f7e382d4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e30153720 con 0x7f7e1c0047c0 2023-10-16T09:49:38.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.766+0000 7f7e382d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e300c5a60 con 0x7f7e300b1550 2023-10-16T09:49:38.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.767+0000 7f7e36871640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e3015f5e0 0x7f7e3015fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:38.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.767+0000 7f7e36871640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e3015f5e0 0x7f7e3015fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34676/0 (socket says 172.21.15.84:34676) 2023-10-16T09:49:38.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.767+0000 7f7e36871640 1 -- 172.21.15.84:0/3517228944 learned_addr learned my addr 172.21.15.84:0/3517228944 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:38.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.767+0000 7f7e36070640 1 --2- 172.21.15.84:0/3517228944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c0047c0 0x7f7e301f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:38.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.767+0000 7f7e3586f640 1 --2- 172.21.15.84:0/3517228944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e300b1550 0x7f7e300b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:38.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.767+0000 7f7e36871640 1 -- 172.21.15.84:0/3517228944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e300b1550 msgr2=0x7f7e300b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:38.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.767+0000 7f7e36871640 1 --2- 172.21.15.84:0/3517228944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e300b1550 0x7f7e300b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.767+0000 7f7e36871640 1 -- 172.21.15.84:0/3517228944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c0047c0 msgr2=0x7f7e301f22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:38.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.767+0000 7f7e36871640 1 --2- 172.21.15.84:0/3517228944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c0047c0 0x7f7e301f22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.767+0000 7f7e36871640 1 -- 172.21.15.84:0/3517228944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e300b2010 con 0x7f7e3015f5e0 2023-10-16T09:49:38.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.767+0000 7f7e36871640 1 --2- 172.21.15.84:0/3517228944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e3015f5e0 0x7f7e3015fa60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e300c8f70 tx=0x7f7e2800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:38.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.768+0000 7f7e26ffd640 1 -- 172.21.15.84:0/3517228944 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28016420 con 0x7f7e3015f5e0 2023-10-16T09:49:38.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.768+0000 7f7e26ffd640 1 -- 172.21.15.84:0/3517228944 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e28016580 con 0x7f7e3015f5e0 2023-10-16T09:49:38.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.768+0000 7f7e26ffd640 1 -- 172.21.15.84:0/3517228944 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28016930 con 0x7f7e3015f5e0 2023-10-16T09:49:38.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.768+0000 7f7e382d4640 1 -- 172.21.15.84:0/3517228944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e300b22f0 con 0x7f7e3015f5e0 2023-10-16T09:49:38.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.768+0000 7f7e382d4640 1 -- 172.21.15.84:0/3517228944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e300c5f90 con 0x7f7e3015f5e0 2023-10-16T09:49:38.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.769+0000 7f7e382d4640 1 -- 172.21.15.84:0/3517228944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3004e160 con 0x7f7e3015f5e0 2023-10-16T09:49:38.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.770+0000 7f7e26ffd640 1 -- 172.21.15.84:0/3517228944 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7e28016a90 con 0x7f7e3015f5e0 2023-10-16T09:49:38.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.771+0000 7f7e26ffd640 1 --2- 172.21.15.84:0/3517228944 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7e1406ae70 0x7f7e1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:38.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.771+0000 7f7e26ffd640 1 -- 172.21.15.84:0/3517228944 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e2808a4c0 con 0x7f7e3015f5e0 2023-10-16T09:49:38.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.771+0000 7f7e36070640 1 --2- 172.21.15.84:0/3517228944 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7e1406ae70 0x7f7e1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:38.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.772+0000 7f7e36070640 1 --2- 172.21.15.84:0/3517228944 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7e1406ae70 0x7f7e1406d330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c000ff0 tx=0x7f7e2c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:38.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.775+0000 7f7e26ffd640 1 -- 172.21.15.84:0/3517228944 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7e28055600 con 0x7f7e3015f5e0 2023-10-16T09:49:38.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.954+0000 7f7e382d4640 1 -- 172.21.15.84:0/3517228944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e30156e60 con 0x7f7e3015f5e0 2023-10-16T09:49:38.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.957+0000 7f7e26ffd640 1 -- 172.21.15.84:0/3517228944 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f7e28058c20 con 0x7f7e3015f5e0 2023-10-16T09:49:38.959 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:38.960 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":21,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:03.529261+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:38.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.960+0000 7f7e382d4640 1 -- 172.21.15.84:0/3517228944 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7e1406ae70 msgr2=0x7f7e1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:38.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.960+0000 7f7e382d4640 1 --2- 172.21.15.84:0/3517228944 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7e1406ae70 0x7f7e1406d330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c000ff0 tx=0x7f7e2c009040 comp rx=0 tx=0).stop 2023-10-16T09:49:38.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.960+0000 7f7e382d4640 1 -- 172.21.15.84:0/3517228944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e3015f5e0 msgr2=0x7f7e3015fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:38.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.960+0000 7f7e382d4640 1 --2- 172.21.15.84:0/3517228944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e3015f5e0 0x7f7e3015fa60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e300c8f70 tx=0x7f7e2800f520 comp rx=0 tx=0).stop 2023-10-16T09:49:38.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.960+0000 7f7e382d4640 1 -- 172.21.15.84:0/3517228944 shutdown_connections 2023-10-16T09:49:38.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.960+0000 7f7e382d4640 1 --2- 172.21.15.84:0/3517228944 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7e1406ae70 0x7f7e1406d330 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.960+0000 7f7e382d4640 1 --2- 172.21.15.84:0/3517228944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e3015f5e0 0x7f7e3015fa60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.960+0000 7f7e382d4640 1 --2- 172.21.15.84:0/3517228944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e300b1550 0x7f7e300b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.960+0000 7f7e382d4640 1 --2- 172.21.15.84:0/3517228944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c0047c0 0x7f7e301f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:38.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.960+0000 7f7e382d4640 1 -- 172.21.15.84:0/3517228944 >> 172.21.15.84:0/3517228944 conn(0x7f7e300cb410 msgr2=0x7f7e300cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:38.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.960+0000 7f7e382d4640 1 -- 172.21.15.84:0/3517228944 shutdown_connections 2023-10-16T09:49:38.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:38.960+0000 7f7e382d4640 1 -- 172.21.15.84:0/3517228944 wait complete. 2023-10-16T09:49:38.981 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:49:00.048649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:49:01.361365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:49:38.981 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:49:39.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.195+0000 7f390534e640 1 Processor -- start 2023-10-16T09:49:39.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.195+0000 7f390534e640 1 -- start start 2023-10-16T09:49:39.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.196+0000 7f390534e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f390015ad70 0x7f3900069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:39.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.196+0000 7f390534e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f39000c9ca0 0x7f390006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:39.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.197+0000 7f390534e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39000cb030 con 0x7f39000ca3b0 2023-10-16T09:49:39.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.197+0000 7f390534e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39000c91c0 con 0x7f390015ad70 2023-10-16T09:49:39.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.197+0000 7f390534e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39000c94a0 con 0x7f39000c9ca0 2023-10-16T09:49:39.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.197+0000 7f38feffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f390015ad70 0x7f3900069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:39.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.197+0000 7f38feffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f390015ad70 0x7f3900069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34682/0 (socket says 172.21.15.84:34682) 2023-10-16T09:49:39.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.197+0000 7f38feffd640 1 -- 172.21.15.84:0/4159389049 learned_addr learned my addr 172.21.15.84:0/4159389049 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:39.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.197+0000 7f38feffd640 1 -- 172.21.15.84:0/4159389049 >> v2:172.21.15.157:3300/0 conn(0x7f39000c9ca0 msgr2=0x7f390006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:39.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.197+0000 7f38feffd640 1 --2- 172.21.15.84:0/4159389049 >> v2:172.21.15.157:3300/0 conn(0x7f39000c9ca0 0x7f390006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.197+0000 7f38feffd640 1 -- 172.21.15.84:0/4159389049 >> v1:172.21.15.84:6789/0 conn(0x7f39000ca3b0 legacy=0x7f39000ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:39.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.197+0000 7f38feffd640 1 -- 172.21.15.84:0/4159389049 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39000c9780 con 0x7f390015ad70 2023-10-16T09:49:39.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.198+0000 7f38feffd640 1 --2- 172.21.15.84:0/4159389049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f390015ad70 0x7f3900069da0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f39000ce870 tx=0x7f38f002b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172d729e74ca96be server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:39.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.198+0000 7f38fdffb640 1 -- 172.21.15.84:0/4159389049 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0030870 con 0x7f390015ad70 2023-10-16T09:49:39.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.199+0000 7f38fdffb640 1 -- 172.21.15.84:0/4159389049 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f00309d0 con 0x7f390015ad70 2023-10-16T09:49:39.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.199+0000 7f38fdffb640 1 -- 172.21.15.84:0/4159389049 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0030d60 con 0x7f390015ad70 2023-10-16T09:49:39.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.199+0000 7f390534e640 1 -- 172.21.15.84:0/4159389049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f390015ad70 msgr2=0x7f3900069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:39.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.199+0000 7f390534e640 1 --2- 172.21.15.84:0/4159389049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f390015ad70 0x7f3900069da0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f39000ce870 tx=0x7f38f002b9b0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.199+0000 7f390534e640 1 -- 172.21.15.84:0/4159389049 shutdown_connections 2023-10-16T09:49:39.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.199+0000 7f390534e640 1 --2- 172.21.15.84:0/4159389049 >> v2:172.21.15.157:3300/0 conn(0x7f39000c9ca0 0x7f390006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.199+0000 7f390534e640 1 --2- 172.21.15.84:0/4159389049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f390015ad70 0x7f3900069da0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.199+0000 7f390534e640 1 -- 172.21.15.84:0/4159389049 >> 172.21.15.84:0/4159389049 conn(0x7f39000ce9b0 msgr2=0x7f39000cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:39.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.199+0000 7f390534e640 1 -- 172.21.15.84:0/4159389049 shutdown_connections 2023-10-16T09:49:39.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.199+0000 7f390534e640 1 -- 172.21.15.84:0/4159389049 wait complete. 2023-10-16T09:49:39.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.201+0000 7f390534e640 1 Processor -- start 2023-10-16T09:49:39.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.201+0000 7f390534e640 1 -- start start 2023-10-16T09:49:39.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.202+0000 7f390534e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39000c9ca0 0x7f390014ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:39.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.202+0000 7f390534e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39000ca3b0 0x7f39001f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:39.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.202+0000 7f390534e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390015ad70 0x7f39001f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:39.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.202+0000 7f390534e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39001f33f0 con 0x7f39000ca3b0 2023-10-16T09:49:39.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.202+0000 7f390534e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390014e410 con 0x7f390015ad70 2023-10-16T09:49:39.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.202+0000 7f390534e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39001f5f50 con 0x7f39000c9ca0 2023-10-16T09:49:39.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.202+0000 7f38fe7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39000ca3b0 0x7f39001f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:39.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.202+0000 7f38fe7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39000ca3b0 0x7f39001f63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34688/0 (socket says 172.21.15.84:34688) 2023-10-16T09:49:39.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.202+0000 7f38fe7fc640 1 -- 172.21.15.84:0/1213227133 learned_addr learned my addr 172.21.15.84:0/1213227133 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:39.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.203+0000 7f38fe7fc640 1 -- 172.21.15.84:0/1213227133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39000c9ca0 msgr2=0x7f390014ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:39.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.203+0000 7f38fe7fc640 1 --2- 172.21.15.84:0/1213227133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39000c9ca0 0x7f390014ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.203+0000 7f38fe7fc640 1 -- 172.21.15.84:0/1213227133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390015ad70 msgr2=0x7f39001f2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:39.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.203+0000 7f38fe7fc640 1 --2- 172.21.15.84:0/1213227133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390015ad70 0x7f39001f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.203+0000 7f38fe7fc640 1 -- 172.21.15.84:0/1213227133 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f00094d0 con 0x7f39000ca3b0 2023-10-16T09:49:39.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.203+0000 7f38fe7fc640 1 --2- 172.21.15.84:0/1213227133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39000ca3b0 0x7f39001f63e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f38f400b4a0 tx=0x7f38f400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:39.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.203+0000 7f38e77fe640 1 -- 172.21.15.84:0/1213227133 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4010610 con 0x7f39000ca3b0 2023-10-16T09:49:39.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.203+0000 7f38e77fe640 1 -- 172.21.15.84:0/1213227133 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f4010770 con 0x7f39000ca3b0 2023-10-16T09:49:39.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.203+0000 7f38e77fe640 1 -- 172.21.15.84:0/1213227133 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4010b20 con 0x7f39000ca3b0 2023-10-16T09:49:39.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.203+0000 7f390534e640 1 -- 172.21.15.84:0/1213227133 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39001f6920 con 0x7f39000ca3b0 2023-10-16T09:49:39.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.203+0000 7f390534e640 1 -- 172.21.15.84:0/1213227133 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39001f3110 con 0x7f39000ca3b0 2023-10-16T09:49:39.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.204+0000 7f390534e640 1 -- 172.21.15.84:0/1213227133 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39001f5a40 con 0x7f39000ca3b0 2023-10-16T09:49:39.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.205+0000 7f38e77fe640 1 -- 172.21.15.84:0/1213227133 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f38f4010c80 con 0x7f39000ca3b0 2023-10-16T09:49:39.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.206+0000 7f38e77fe640 1 --2- 172.21.15.84:0/1213227133 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f38ec06ae70 0x7f38ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:39.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.206+0000 7f38e77fe640 1 -- 172.21.15.84:0/1213227133 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38f4089310 con 0x7f39000ca3b0 2023-10-16T09:49:39.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.207+0000 7f38feffd640 1 --2- 172.21.15.84:0/1213227133 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f38ec06ae70 0x7f38ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:39.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.207+0000 7f38feffd640 1 --2- 172.21.15.84:0/1213227133 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f38ec06ae70 0x7f38ec06d330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f38f0036000 tx=0x7f38f003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:39.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.211+0000 7f38e77fe640 1 -- 172.21.15.84:0/1213227133 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f39001f5a40 con 0x7f39000ca3b0 2023-10-16T09:49:39.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.402+0000 7f390534e640 1 -- 172.21.15.84:0/1213227133 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39001f5a40 con 0x7f39000ca3b0 2023-10-16T09:49:39.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.403+0000 7f38e77fe640 1 -- 172.21.15.84:0/1213227133 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f39001f5a40 con 0x7f39000ca3b0 2023-10-16T09:49:39.405 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T09:49:39.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.405+0000 7f390534e640 1 -- 172.21.15.84:0/1213227133 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f38ec06ae70 msgr2=0x7f38ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:39.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.406+0000 7f390534e640 1 --2- 172.21.15.84:0/1213227133 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f38ec06ae70 0x7f38ec06d330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f38f0036000 tx=0x7f38f003b040 comp rx=0 tx=0).stop 2023-10-16T09:49:39.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.406+0000 7f390534e640 1 -- 172.21.15.84:0/1213227133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39000ca3b0 msgr2=0x7f39001f63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:39.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.406+0000 7f390534e640 1 --2- 172.21.15.84:0/1213227133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39000ca3b0 0x7f39001f63e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f38f400b4a0 tx=0x7f38f400b960 comp rx=0 tx=0).stop 2023-10-16T09:49:39.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.406+0000 7f390534e640 1 -- 172.21.15.84:0/1213227133 shutdown_connections 2023-10-16T09:49:39.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.406+0000 7f390534e640 1 --2- 172.21.15.84:0/1213227133 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f38ec06ae70 0x7f38ec06d330 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.406+0000 7f390534e640 1 --2- 172.21.15.84:0/1213227133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390015ad70 0x7f39001f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.406+0000 7f390534e640 1 --2- 172.21.15.84:0/1213227133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39000ca3b0 0x7f39001f63e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.406+0000 7f390534e640 1 --2- 172.21.15.84:0/1213227133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39000c9ca0 0x7f390014ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.406+0000 7f390534e640 1 -- 172.21.15.84:0/1213227133 >> 172.21.15.84:0/1213227133 conn(0x7f39000ce9b0 msgr2=0x7f39000ca820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:39.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.406+0000 7f390534e640 1 -- 172.21.15.84:0/1213227133 shutdown_connections 2023-10-16T09:49:39.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.406+0000 7f390534e640 1 -- 172.21.15.84:0/1213227133 wait complete. 2023-10-16T09:49:39.427 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:49:39.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.641+0000 7fbe8de1c640 1 Processor -- start 2023-10-16T09:49:39.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.642+0000 7fbe8de1c640 1 -- start start 2023-10-16T09:49:39.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.643+0000 7fbe8de1c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe8806ade0 0x7fbe8806b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:39.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.643+0000 7fbe8de1c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe880cdc30 con 0x7fbe880c9800 2023-10-16T09:49:39.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.643+0000 7fbe8de1c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe880ccd80 con 0x7fbe88071b30 2023-10-16T09:49:39.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.643+0000 7fbe8de1c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8806b780 con 0x7fbe8806ade0 2023-10-16T09:49:39.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.643+0000 7fbe87fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbe880c9800 0x7fbe880ce0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53630/0 (socket says 172.21.15.84:53630) 2023-10-16T09:49:39.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.643+0000 7fbe87fff640 1 -- 172.21.15.84:0/1474778339 learned_addr learned my addr 172.21.15.84:0/1474778339 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:39.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.644+0000 7fbe7effd640 1 --2- 172.21.15.84:0/1474778339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe8806ade0 0x7fbe8806b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:39.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.644+0000 7fbe86ffd640 1 -- 172.21.15.84:0/1474778339 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625861134 0 0) 0x7fbe880cdc30 con 0x7fbe880c9800 2023-10-16T09:49:39.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.644+0000 7fbe86ffd640 1 -- 172.21.15.84:0/1474778339 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe74003440 con 0x7fbe880c9800 2023-10-16T09:49:39.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.644+0000 7fbe86ffd640 1 -- 172.21.15.84:0/1474778339 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3785256859 0 0) 0x7fbe74003440 con 0x7fbe880c9800 2023-10-16T09:49:39.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.644+0000 7fbe86ffd640 1 -- 172.21.15.84:0/1474778339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe8806ade0 msgr2=0x7fbe8806b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:39.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.644+0000 7fbe86ffd640 1 --2- 172.21.15.84:0/1474778339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe8806ade0 0x7fbe8806b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.644+0000 7fbe86ffd640 1 -- 172.21.15.84:0/1474778339 >> v1:172.21.15.157:6789/0 conn(0x7fbe88071b30 legacy=0x7fbe8806fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:39.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.644+0000 7fbe86ffd640 1 -- 172.21.15.84:0/1474778339 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe880cd020 con 0x7fbe880c9800 2023-10-16T09:49:39.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.644+0000 7fbe86ffd640 1 -- 172.21.15.84:0/1474778339 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fbe80001670 con 0x7fbe880c9800 2023-10-16T09:49:39.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.644+0000 7fbe86ffd640 1 -- 172.21.15.84:0/1474778339 >> v1:172.21.15.84:6789/0 conn(0x7fbe880c9800 legacy=0x7fbe880ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:39.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.644+0000 7fbe86ffd640 1 --2- 172.21.15.84:0/1474778339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe74003a90 0x7fbe74003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:39.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.644+0000 7fbe86ffd640 1 --2- 172.21.15.84:0/1474778339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe74008850 0x7fbe7400acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:39.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.644+0000 7fbe86ffd640 1 --2- 172.21.15.84:0/1474778339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe8806ade0 0x7fbe7400d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:39.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.644+0000 7fbe86ffd640 1 -- 172.21.15.84:0/1474778339 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8806b780 con 0x7fbe74003a90 2023-10-16T09:49:39.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.644+0000 7fbe86ffd640 1 -- 172.21.15.84:0/1474778339 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7400dc70 con 0x7fbe74008850 2023-10-16T09:49:39.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.644+0000 7fbe86ffd640 1 -- 172.21.15.84:0/1474778339 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7400de30 con 0x7fbe8806ade0 2023-10-16T09:49:39.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.645+0000 7fbe877fe640 1 --2- 172.21.15.84:0/1474778339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe74003a90 0x7fbe74003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:39.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.645+0000 7fbe877fe640 1 -- 172.21.15.84:0/1474778339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe8806ade0 msgr2=0x7fbe7400d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:39.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.645+0000 7fbe877fe640 1 --2- 172.21.15.84:0/1474778339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe8806ade0 0x7fbe7400d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.645+0000 7fbe877fe640 1 -- 172.21.15.84:0/1474778339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe74008850 msgr2=0x7fbe7400acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:39.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.645+0000 7fbe877fe640 1 --2- 172.21.15.84:0/1474778339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe74008850 0x7fbe7400acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.645+0000 7fbe877fe640 1 -- 172.21.15.84:0/1474778339 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe74003440 con 0x7fbe74003a90 2023-10-16T09:49:39.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.646+0000 7fbe877fe640 1 --2- 172.21.15.84:0/1474778339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe74003a90 0x7fbe74003eb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbe78001750 tx=0x7fbe740036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa5d7d9d165b279 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:39.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.646+0000 7fbe86ffd640 1 -- 172.21.15.84:0/1474778339 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe780052d0 con 0x7fbe74003a90 2023-10-16T09:49:39.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.646+0000 7fbe86ffd640 1 -- 172.21.15.84:0/1474778339 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7802e050 con 0x7fbe74003a90 2023-10-16T09:49:39.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.646+0000 7fbe86ffd640 1 -- 172.21.15.84:0/1474778339 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe780035f0 con 0x7fbe74003a90 2023-10-16T09:49:39.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.647+0000 7fbe8de1c640 1 -- 172.21.15.84:0/1474778339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe74003a90 msgr2=0x7fbe74003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:39.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.647+0000 7fbe8de1c640 1 --2- 172.21.15.84:0/1474778339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe74003a90 0x7fbe74003eb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbe78001750 tx=0x7fbe740036f0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.647+0000 7fbe87fff640 1 -- 172.21.15.84:0/1474778339 reap_dead start 2023-10-16T09:49:39.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.647+0000 7fbe8de1c640 1 -- 172.21.15.84:0/1474778339 shutdown_connections 2023-10-16T09:49:39.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.647+0000 7fbe8de1c640 1 -- 172.21.15.84:0/1474778339 >> 172.21.15.84:0/1474778339 conn(0x7fbe8806bc10 msgr2=0x7fbe8806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:39.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.647+0000 7fbe8de1c640 1 -- 172.21.15.84:0/1474778339 shutdown_connections 2023-10-16T09:49:39.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.647+0000 7fbe8de1c640 1 -- 172.21.15.84:0/1474778339 wait complete. 2023-10-16T09:49:39.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.649+0000 7fbe8de1c640 1 Processor -- start 2023-10-16T09:49:39.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.649+0000 7fbe8de1c640 1 -- start start 2023-10-16T09:49:39.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.650+0000 7fbe8de1c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe74003a90 0x7fbe8815a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:39.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.650+0000 7fbe8de1c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe881e3a00 0x7fbe880ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:39.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.650+0000 7fbe8de1c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe880c34f0 0x7fbe880c3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:39.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.650+0000 7fbe8de1c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8815bb60 con 0x7fbe880c34f0 2023-10-16T09:49:39.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.650+0000 7fbe8de1c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe880ae800 con 0x7fbe74003a90 2023-10-16T09:49:39.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.650+0000 7fbe8de1c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8815aee0 con 0x7fbe881e3a00 2023-10-16T09:49:39.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.650+0000 7fbe87fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe880c34f0 0x7fbe880c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:39.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.650+0000 7fbe87fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe880c34f0 0x7fbe880c3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34710/0 (socket says 172.21.15.84:34710) 2023-10-16T09:49:39.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.650+0000 7fbe87fff640 1 -- 172.21.15.84:0/1156126805 learned_addr learned my addr 172.21.15.84:0/1156126805 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:39.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.651+0000 7fbe87fff640 1 -- 172.21.15.84:0/1156126805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe881e3a00 msgr2=0x7fbe880ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:39.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.651+0000 7fbe87fff640 1 --2- 172.21.15.84:0/1156126805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe881e3a00 0x7fbe880ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.651+0000 7fbe87fff640 1 -- 172.21.15.84:0/1156126805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe74003a90 msgr2=0x7fbe8815a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:39.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.651+0000 7fbe87fff640 1 --2- 172.21.15.84:0/1156126805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe74003a90 0x7fbe8815a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.651+0000 7fbe87fff640 1 -- 172.21.15.84:0/1156126805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8815b1c0 con 0x7fbe880c34f0 2023-10-16T09:49:39.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.651+0000 7fbe87fff640 1 --2- 172.21.15.84:0/1156126805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe880c34f0 0x7fbe880c3970 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbe880c9be0 tx=0x7fbe8000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:39.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.651+0000 7fbe84ff9640 1 -- 172.21.15.84:0/1156126805 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe800074e0 con 0x7fbe880c34f0 2023-10-16T09:49:39.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.651+0000 7fbe84ff9640 1 -- 172.21.15.84:0/1156126805 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe80007660 con 0x7fbe880c34f0 2023-10-16T09:49:39.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.651+0000 7fbe84ff9640 1 -- 172.21.15.84:0/1156126805 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe80004a20 con 0x7fbe880c34f0 2023-10-16T09:49:39.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.651+0000 7fbe8de1c640 1 -- 172.21.15.84:0/1156126805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8815b4a0 con 0x7fbe880c34f0 2023-10-16T09:49:39.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.651+0000 7fbe8de1c640 1 -- 172.21.15.84:0/1156126805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe881e3e60 con 0x7fbe880c34f0 2023-10-16T09:49:39.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.652+0000 7fbe8de1c640 1 -- 172.21.15.84:0/1156126805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8804e160 con 0x7fbe880c34f0 2023-10-16T09:49:39.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.653+0000 7fbe84ff9640 1 -- 172.21.15.84:0/1156126805 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbe80004b80 con 0x7fbe880c34f0 2023-10-16T09:49:39.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.654+0000 7fbe84ff9640 1 --2- 172.21.15.84:0/1156126805 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbe6806aea0 0x7fbe6806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:39.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.654+0000 7fbe84ff9640 1 -- 172.21.15.84:0/1156126805 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe80088b00 con 0x7fbe880c34f0 2023-10-16T09:49:39.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.654+0000 7fbe877fe640 1 --2- 172.21.15.84:0/1156126805 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbe6806aea0 0x7fbe6806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:39.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.655+0000 7fbe877fe640 1 --2- 172.21.15.84:0/1156126805 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbe6806aea0 0x7fbe6806d360 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbe78033000 tx=0x7fbe7802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:39.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.659+0000 7fbe84ff9640 1 -- 172.21.15.84:0/1156126805 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbe8008ae60 con 0x7fbe880c34f0 2023-10-16T09:49:39.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.853+0000 7fbe8de1c640 1 -- 172.21.15.84:0/1156126805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbe88159410 con 0x7fbe880c34f0 2023-10-16T09:49:39.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.854+0000 7fbe84ff9640 1 -- 172.21.15.84:0/1156126805 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbe80053ba0 con 0x7fbe880c34f0 2023-10-16T09:49:39.856 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T09:49:39.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.856+0000 7fbe8de1c640 1 -- 172.21.15.84:0/1156126805 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbe6806aea0 msgr2=0x7fbe6806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:39.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.856+0000 7fbe8de1c640 1 --2- 172.21.15.84:0/1156126805 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbe6806aea0 0x7fbe6806d360 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbe78033000 tx=0x7fbe7802f000 comp rx=0 tx=0).stop 2023-10-16T09:49:39.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.856+0000 7fbe8de1c640 1 -- 172.21.15.84:0/1156126805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe880c34f0 msgr2=0x7fbe880c3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:39.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.856+0000 7fbe8de1c640 1 --2- 172.21.15.84:0/1156126805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe880c34f0 0x7fbe880c3970 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbe880c9be0 tx=0x7fbe8000c960 comp rx=0 tx=0).stop 2023-10-16T09:49:39.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.856+0000 7fbe8de1c640 1 -- 172.21.15.84:0/1156126805 shutdown_connections 2023-10-16T09:49:39.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.856+0000 7fbe8de1c640 1 --2- 172.21.15.84:0/1156126805 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fbe6806aea0 0x7fbe6806d360 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.856+0000 7fbe8de1c640 1 --2- 172.21.15.84:0/1156126805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe880c34f0 0x7fbe880c3970 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.856+0000 7fbe8de1c640 1 --2- 172.21.15.84:0/1156126805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe881e3a00 0x7fbe880ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.856+0000 7fbe8de1c640 1 --2- 172.21.15.84:0/1156126805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe74003a90 0x7fbe8815a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:39.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.856+0000 7fbe8de1c640 1 -- 172.21.15.84:0/1156126805 >> 172.21.15.84:0/1156126805 conn(0x7fbe8806bc10 msgr2=0x7fbe880aec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:39.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.857+0000 7fbe8de1c640 1 -- 172.21.15.84:0/1156126805 shutdown_connections 2023-10-16T09:49:39.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:39.857+0000 7fbe8de1c640 1 -- 172.21.15.84:0/1156126805 wait complete. 2023-10-16T09:49:39.877 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:49:40.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.079+0000 7f4a1fcd7640 1 Processor -- start 2023-10-16T09:49:40.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.080+0000 7f4a1fcd7640 1 -- start start 2023-10-16T09:49:40.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.081+0000 7f4a1fcd7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4a1815ad70 0x7f4a18069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.081+0000 7f4a1fcd7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180c9ca0 0x7f4a1806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.081+0000 7f4a1fcd7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a180cb030 con 0x7f4a180ca3b0 2023-10-16T09:49:40.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.081+0000 7f4a1fcd7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a180c91c0 con 0x7f4a1815ad70 2023-10-16T09:49:40.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.081+0000 7f4a1fcd7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180c94a0 con 0x7f4a180c9ca0 2023-10-16T09:49:40.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.082+0000 7f4a1da73640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4a1815ad70 0x7f4a18069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.082+0000 7f4a1d272640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180c9ca0 0x7f4a1806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.082+0000 7f4a1da73640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4a1815ad70 0x7f4a18069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:59948/0 (socket says 172.21.15.84:59948) 2023-10-16T09:49:40.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.082+0000 7f4a1da73640 1 -- 172.21.15.84:0/1248312055 learned_addr learned my addr 172.21.15.84:0/1248312055 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:40.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.083+0000 7f4a1da73640 1 -- 172.21.15.84:0/1248312055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180c9ca0 msgr2=0x7f4a1806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:40.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.083+0000 7f4a1da73640 1 --2- 172.21.15.84:0/1248312055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180c9ca0 0x7f4a1806a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.083+0000 7f4a1da73640 1 -- 172.21.15.84:0/1248312055 >> v1:172.21.15.157:6789/0 conn(0x7f4a180ca3b0 legacy=0x7f4a180ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:40.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.083+0000 7f4a1da73640 1 -- 172.21.15.84:0/1248312055 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a180c9780 con 0x7f4a1815ad70 2023-10-16T09:49:40.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.083+0000 7f4a1d272640 1 --2- 172.21.15.84:0/1248312055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180c9ca0 0x7f4a1806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:49:40.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.084+0000 7f4a1da73640 1 --2- 172.21.15.84:0/1248312055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a1815ad70 0x7f4a18069da0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009870 tx=0x7f4a0c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=168b6c2edca9ab5d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:40.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.085+0000 7f4a1ca71640 1 -- 172.21.15.84:0/1248312055 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c030840 con 0x7f4a1815ad70 2023-10-16T09:49:40.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.085+0000 7f4a1ca71640 1 -- 172.21.15.84:0/1248312055 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0c0309a0 con 0x7f4a1815ad70 2023-10-16T09:49:40.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.085+0000 7f4a1ca71640 1 -- 172.21.15.84:0/1248312055 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c030d30 con 0x7f4a1815ad70 2023-10-16T09:49:40.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.086+0000 7f4a1fcd7640 1 -- 172.21.15.84:0/1248312055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a1815ad70 msgr2=0x7f4a18069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:40.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.086+0000 7f4a1fcd7640 1 --2- 172.21.15.84:0/1248312055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a1815ad70 0x7f4a18069da0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009870 tx=0x7f4a0c02b980 comp rx=0 tx=0).stop 2023-10-16T09:49:40.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.086+0000 7f4a1fcd7640 1 -- 172.21.15.84:0/1248312055 shutdown_connections 2023-10-16T09:49:40.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.086+0000 7f4a1fcd7640 1 --2- 172.21.15.84:0/1248312055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180c9ca0 0x7f4a1806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.086+0000 7f4a1fcd7640 1 --2- 172.21.15.84:0/1248312055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a1815ad70 0x7f4a18069da0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.086+0000 7f4a1fcd7640 1 -- 172.21.15.84:0/1248312055 >> 172.21.15.84:0/1248312055 conn(0x7f4a180ce9b0 msgr2=0x7f4a180cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:40.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.086+0000 7f4a1fcd7640 1 -- 172.21.15.84:0/1248312055 shutdown_connections 2023-10-16T09:49:40.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.086+0000 7f4a1fcd7640 1 -- 172.21.15.84:0/1248312055 wait complete. 2023-10-16T09:49:40.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.088+0000 7f4a1fcd7640 1 Processor -- start 2023-10-16T09:49:40.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.088+0000 7f4a1fcd7640 1 -- start start 2023-10-16T09:49:40.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.089+0000 7f4a1fcd7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180c9ca0 0x7f4a1815dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.089+0000 7f4a1fcd7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a180ca3b0 0x7f4a181f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.089+0000 7f4a1fcd7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a1815ad70 0x7f4a180b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.089+0000 7f4a1fcd7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180c4d70 con 0x7f4a1815ad70 2023-10-16T09:49:40.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.089+0000 7f4a1fcd7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180c35e0 con 0x7f4a180ca3b0 2023-10-16T09:49:40.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.089+0000 7f4a1fcd7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180c4130 con 0x7f4a180c9ca0 2023-10-16T09:49:40.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.089+0000 7f4a1e274640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a1815ad70 0x7f4a180b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.089+0000 7f4a1e274640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a1815ad70 0x7f4a180b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34714/0 (socket says 172.21.15.84:34714) 2023-10-16T09:49:40.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.089+0000 7f4a1e274640 1 -- 172.21.15.84:0/2542919594 learned_addr learned my addr 172.21.15.84:0/2542919594 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:40.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.090+0000 7f4a1d272640 1 --2- 172.21.15.84:0/2542919594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a180ca3b0 0x7f4a181f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.090+0000 7f4a1da73640 1 --2- 172.21.15.84:0/2542919594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180c9ca0 0x7f4a1815dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.090+0000 7f4a1e274640 1 -- 172.21.15.84:0/2542919594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180c9ca0 msgr2=0x7f4a1815dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:40.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.090+0000 7f4a1e274640 1 --2- 172.21.15.84:0/2542919594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180c9ca0 0x7f4a1815dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.090+0000 7f4a1e274640 1 -- 172.21.15.84:0/2542919594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a180ca3b0 msgr2=0x7f4a181f34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:40.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.090+0000 7f4a1e274640 1 --2- 172.21.15.84:0/2542919594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a180ca3b0 0x7f4a181f34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.090+0000 7f4a1e274640 1 -- 172.21.15.84:0/2542919594 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a14009a70 con 0x7f4a1815ad70 2023-10-16T09:49:40.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.090+0000 7f4a1da73640 1 --2- 172.21.15.84:0/2542919594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180c9ca0 0x7f4a1815dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:40.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.090+0000 7f4a1d272640 1 --2- 172.21.15.84:0/2542919594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a180ca3b0 0x7f4a181f34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:40.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.090+0000 7f4a1e274640 1 --2- 172.21.15.84:0/2542919594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a1815ad70 0x7f4a180b3080 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000c400 tx=0x7f4a1000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:40.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.091+0000 7f4a0a7fc640 1 -- 172.21.15.84:0/2542919594 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a100052d0 con 0x7f4a1815ad70 2023-10-16T09:49:40.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.091+0000 7f4a0a7fc640 1 -- 172.21.15.84:0/2542919594 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a10005430 con 0x7f4a1815ad70 2023-10-16T09:49:40.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.091+0000 7f4a0a7fc640 1 -- 172.21.15.84:0/2542919594 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10003810 con 0x7f4a1815ad70 2023-10-16T09:49:40.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.091+0000 7f4a1fcd7640 1 -- 172.21.15.84:0/2542919594 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0c0094d0 con 0x7f4a1815ad70 2023-10-16T09:49:40.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.091+0000 7f4a1fcd7640 1 -- 172.21.15.84:0/2542919594 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a180c43d0 con 0x7f4a1815ad70 2023-10-16T09:49:40.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.092+0000 7f4a1fcd7640 1 -- 172.21.15.84:0/2542919594 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a180c4eb0 con 0x7f4a1815ad70 2023-10-16T09:49:40.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.093+0000 7f4a0a7fc640 1 -- 172.21.15.84:0/2542919594 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4a10003970 con 0x7f4a1815ad70 2023-10-16T09:49:40.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.094+0000 7f4a0a7fc640 1 --2- 172.21.15.84:0/2542919594 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f49fc06ae70 0x7f49fc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.094+0000 7f4a0a7fc640 1 -- 172.21.15.84:0/2542919594 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a10088b30 con 0x7f4a1815ad70 2023-10-16T09:49:40.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.094+0000 7f4a1da73640 1 --2- 172.21.15.84:0/2542919594 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f49fc06ae70 0x7f49fc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.095+0000 7f4a1da73640 1 --2- 172.21.15.84:0/2542919594 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f49fc06ae70 0x7f49fc06d330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c036000 tx=0x7f4a0c03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:40.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.098+0000 7f4a0a7fc640 1 -- 172.21.15.84:0/2542919594 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4a180c4eb0 con 0x7f4a1815ad70 2023-10-16T09:49:40.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.288+0000 7f4a1fcd7640 1 -- 172.21.15.84:0/2542919594 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a180c4eb0 con 0x7f4a1815ad70 2023-10-16T09:49:40.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.289+0000 7f4a0a7fc640 1 -- 172.21.15.84:0/2542919594 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a180c4eb0 con 0x7f4a1815ad70 2023-10-16T09:49:40.291 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T09:49:40.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.292+0000 7f4a1fcd7640 1 -- 172.21.15.84:0/2542919594 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f49fc06ae70 msgr2=0x7f49fc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:40.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.292+0000 7f4a1fcd7640 1 --2- 172.21.15.84:0/2542919594 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f49fc06ae70 0x7f49fc06d330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c036000 tx=0x7f4a0c03b040 comp rx=0 tx=0).stop 2023-10-16T09:49:40.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.292+0000 7f4a1fcd7640 1 -- 172.21.15.84:0/2542919594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a1815ad70 msgr2=0x7f4a180b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:40.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.292+0000 7f4a1fcd7640 1 --2- 172.21.15.84:0/2542919594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a1815ad70 0x7f4a180b3080 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000c400 tx=0x7f4a1000c8c0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.292+0000 7f4a1fcd7640 1 -- 172.21.15.84:0/2542919594 shutdown_connections 2023-10-16T09:49:40.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.292+0000 7f4a1fcd7640 1 --2- 172.21.15.84:0/2542919594 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f49fc06ae70 0x7f49fc06d330 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.292+0000 7f4a1fcd7640 1 --2- 172.21.15.84:0/2542919594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a1815ad70 0x7f4a180b3080 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.292+0000 7f4a1fcd7640 1 --2- 172.21.15.84:0/2542919594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a180ca3b0 0x7f4a181f34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.292+0000 7f4a1fcd7640 1 --2- 172.21.15.84:0/2542919594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180c9ca0 0x7f4a1815dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.292+0000 7f4a1fcd7640 1 -- 172.21.15.84:0/2542919594 >> 172.21.15.84:0/2542919594 conn(0x7f4a180ce9b0 msgr2=0x7f4a180ad950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:40.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.293+0000 7f4a1fcd7640 1 -- 172.21.15.84:0/2542919594 shutdown_connections 2023-10-16T09:49:40.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.293+0000 7f4a1fcd7640 1 -- 172.21.15.84:0/2542919594 wait complete. 2023-10-16T09:49:40.313 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:40.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.524+0000 7fef8ffff640 1 Processor -- start 2023-10-16T09:49:40.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.524+0000 7fef8ffff640 1 -- start start 2023-10-16T09:49:40.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.525+0000 7fef8ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef90159950 0x7fef9015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.526+0000 7fef8ffff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9015aef0 con 0x7fef9006c810 2023-10-16T09:49:40.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.526+0000 7fef8ffff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9006cc80 con 0x7fef90070450 2023-10-16T09:49:40.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.526+0000 7fef8ffff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9015e490 con 0x7fef90159950 2023-10-16T09:49:40.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.526+0000 7fef8f7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fef9006c810 0x7fef900c9090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53632/0 (socket says 172.21.15.84:53632) 2023-10-16T09:49:40.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.526+0000 7fef8f7fe640 1 -- 172.21.15.84:0/2616698043 learned_addr learned my addr 172.21.15.84:0/2616698043 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:40.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.526+0000 7fef8effd640 1 --2- 172.21.15.84:0/2616698043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef90159950 0x7fef9015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.526+0000 7fef8e7fc640 1 -- 172.21.15.84:0/2616698043 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889867878 0 0) 0x7fef88001490 con 0x7fef9006c810 2023-10-16T09:49:40.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.526+0000 7fef8e7fc640 1 -- 172.21.15.84:0/2616698043 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef7c003520 con 0x7fef9006c810 2023-10-16T09:49:40.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.527+0000 7fef8e7fc640 1 -- 172.21.15.84:0/2616698043 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3203669301 0 0) 0x7fef7c003520 con 0x7fef9006c810 2023-10-16T09:49:40.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.527+0000 7fef8e7fc640 1 -- 172.21.15.84:0/2616698043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef90159950 msgr2=0x7fef9015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:40.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.527+0000 7fef8e7fc640 1 --2- 172.21.15.84:0/2616698043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef90159950 0x7fef9015a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.527+0000 7fef8e7fc640 1 -- 172.21.15.84:0/2616698043 >> v1:172.21.15.157:6789/0 conn(0x7fef90070450 legacy=0x7fef9006c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:40.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.527+0000 7fef8e7fc640 1 -- 172.21.15.84:0/2616698043 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9015d910 con 0x7fef9006c810 2023-10-16T09:49:40.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.527+0000 7fef8e7fc640 1 -- 172.21.15.84:0/2616698043 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fef88001b80 con 0x7fef9006c810 2023-10-16T09:49:40.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.527+0000 7fef8e7fc640 1 -- 172.21.15.84:0/2616698043 >> v1:172.21.15.84:6789/0 conn(0x7fef9006c810 legacy=0x7fef900c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:40.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.527+0000 7fef8e7fc640 1 --2- 172.21.15.84:0/2616698043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef7c0039a0 0x7fef7c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.527+0000 7fef8e7fc640 1 --2- 172.21.15.84:0/2616698043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef90159950 0x7fef7c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.527+0000 7fef8e7fc640 1 --2- 172.21.15.84:0/2616698043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c004d00 0x7fef7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.527+0000 7fef8e7fc640 1 -- 172.21.15.84:0/2616698043 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9015e490 con 0x7fef7c0039a0 2023-10-16T09:49:40.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.527+0000 7fef8e7fc640 1 -- 172.21.15.84:0/2616698043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c00bb10 con 0x7fef7c004d00 2023-10-16T09:49:40.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.527+0000 7fef8e7fc640 1 -- 172.21.15.84:0/2616698043 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c00bcd0 con 0x7fef90159950 2023-10-16T09:49:40.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.527+0000 7fef8effd640 1 --2- 172.21.15.84:0/2616698043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef7c0039a0 0x7fef7c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.528+0000 7fef867fc640 1 --2- 172.21.15.84:0/2616698043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef90159950 0x7fef7c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.528+0000 7fef8effd640 1 -- 172.21.15.84:0/2616698043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef90159950 msgr2=0x7fef7c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:40.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.528+0000 7fef8effd640 1 --2- 172.21.15.84:0/2616698043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef90159950 0x7fef7c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.528+0000 7fef8effd640 1 -- 172.21.15.84:0/2616698043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c004d00 msgr2=0x7fef7c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:40.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.528+0000 7fef8f7fe640 1 --2- 172.21.15.84:0/2616698043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c004d00 0x7fef7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.528+0000 7fef8effd640 1 --2- 172.21.15.84:0/2616698043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c004d00 0x7fef7c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.528+0000 7fef8effd640 1 -- 172.21.15.84:0/2616698043 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c003520 con 0x7fef7c0039a0 2023-10-16T09:49:40.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.528+0000 7fef867fc640 1 --2- 172.21.15.84:0/2616698043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef90159950 0x7fef7c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:40.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.529+0000 7fef8effd640 1 --2- 172.21.15.84:0/2616698043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef7c0039a0 0x7fef7c003dc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef80009ac0 tx=0x7fef7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddfc0316207a593 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:40.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.529+0000 7fef8e7fc640 1 -- 172.21.15.84:0/2616698043 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8002c6b0 con 0x7fef7c0039a0 2023-10-16T09:49:40.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.529+0000 7fef8e7fc640 1 -- 172.21.15.84:0/2616698043 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8002c810 con 0x7fef7c0039a0 2023-10-16T09:49:40.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.529+0000 7fef8e7fc640 1 -- 172.21.15.84:0/2616698043 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8002cba0 con 0x7fef7c0039a0 2023-10-16T09:49:40.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.529+0000 7fef8ffff640 1 -- 172.21.15.84:0/2616698043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef7c0039a0 msgr2=0x7fef7c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:40.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.529+0000 7fef8ffff640 1 --2- 172.21.15.84:0/2616698043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef7c0039a0 0x7fef7c003dc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef80009ac0 tx=0x7fef7c0057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.529+0000 7fef8f7fe640 1 -- 172.21.15.84:0/2616698043 reap_dead start 2023-10-16T09:49:40.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.530+0000 7fef8ffff640 1 -- 172.21.15.84:0/2616698043 shutdown_connections 2023-10-16T09:49:40.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.530+0000 7fef8ffff640 1 -- 172.21.15.84:0/2616698043 >> 172.21.15.84:0/2616698043 conn(0x7fef900ce6a0 msgr2=0x7fef900ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:40.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.530+0000 7fef8ffff640 1 -- 172.21.15.84:0/2616698043 shutdown_connections 2023-10-16T09:49:40.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.530+0000 7fef8ffff640 1 -- 172.21.15.84:0/2616698043 wait complete. 2023-10-16T09:49:40.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.532+0000 7fef8ffff640 1 Processor -- start 2023-10-16T09:49:40.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.532+0000 7fef8ffff640 1 -- start start 2023-10-16T09:49:40.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.533+0000 7fef8ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c0039a0 0x7fef9014aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.533+0000 7fef8ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef9006ada0 0x7fef9006b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.533+0000 7fef8ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef900c2760 0x7fef900c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.533+0000 7fef8ffff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9006b740 con 0x7fef900c2760 2023-10-16T09:49:40.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.533+0000 7fef8ffff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef900b23b0 con 0x7fef7c0039a0 2023-10-16T09:49:40.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.533+0000 7fef8ffff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef900c3500 con 0x7fef9006ada0 2023-10-16T09:49:40.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.533+0000 7fef8f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef900c2760 0x7fef900c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.533+0000 7fef8f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef900c2760 0x7fef900c2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34740/0 (socket says 172.21.15.84:34740) 2023-10-16T09:49:40.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.533+0000 7fef8f7fe640 1 -- 172.21.15.84:0/2817658950 learned_addr learned my addr 172.21.15.84:0/2817658950 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:40.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.533+0000 7fef8effd640 1 --2- 172.21.15.84:0/2817658950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c0039a0 0x7fef9014aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.533+0000 7fef867fc640 1 --2- 172.21.15.84:0/2817658950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef9006ada0 0x7fef9006b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.533+0000 7fef8f7fe640 1 -- 172.21.15.84:0/2817658950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef9006ada0 msgr2=0x7fef9006b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:40.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.533+0000 7fef8f7fe640 1 --2- 172.21.15.84:0/2817658950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef9006ada0 0x7fef9006b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.533+0000 7fef8f7fe640 1 -- 172.21.15.84:0/2817658950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c0039a0 msgr2=0x7fef9014aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:40.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.533+0000 7fef8f7fe640 1 --2- 172.21.15.84:0/2817658950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c0039a0 0x7fef9014aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.533+0000 7fef8f7fe640 1 -- 172.21.15.84:0/2817658950 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef900b2650 con 0x7fef900c2760 2023-10-16T09:49:40.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.533+0000 7fef867fc640 1 --2- 172.21.15.84:0/2817658950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef9006ada0 0x7fef9006b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:40.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.534+0000 7fef8f7fe640 1 --2- 172.21.15.84:0/2817658950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef900c2760 0x7fef900c2be0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fef900ca950 tx=0x7fef8800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.534+0000 7fef87fff640 1 -- 172.21.15.84:0/2817658950 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88004d60 con 0x7fef900c2760 2023-10-16T09:49:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.534+0000 7fef87fff640 1 -- 172.21.15.84:0/2817658950 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8800bb80 con 0x7fef900c2760 2023-10-16T09:49:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.534+0000 7fef87fff640 1 -- 172.21.15.84:0/2817658950 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88007070 con 0x7fef900c2760 2023-10-16T09:49:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.534+0000 7fef8ffff640 1 -- 172.21.15.84:0/2817658950 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef900b2930 con 0x7fef900c2760 2023-10-16T09:49:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.534+0000 7fef8ffff640 1 -- 172.21.15.84:0/2817658950 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9014b400 con 0x7fef900c2760 2023-10-16T09:49:40.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.535+0000 7fef8ffff640 1 -- 172.21.15.84:0/2817658950 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef900cc250 con 0x7fef900c2760 2023-10-16T09:49:40.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.536+0000 7fef87fff640 1 -- 172.21.15.84:0/2817658950 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fef880071d0 con 0x7fef900c2760 2023-10-16T09:49:40.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.537+0000 7fef87fff640 1 --2- 172.21.15.84:0/2817658950 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fef7006ae70 0x7fef7006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.537+0000 7fef87fff640 1 -- 172.21.15.84:0/2817658950 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef880898a0 con 0x7fef900c2760 2023-10-16T09:49:40.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.537+0000 7fef8effd640 1 --2- 172.21.15.84:0/2817658950 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fef7006ae70 0x7fef7006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.538+0000 7fef8effd640 1 --2- 172.21.15.84:0/2817658950 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fef7006ae70 0x7fef7006d330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fef80036000 tx=0x7fef80030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:40.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.541+0000 7fef87fff640 1 -- 172.21.15.84:0/2817658950 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fef880548b0 con 0x7fef900c2760 2023-10-16T09:49:40.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.738+0000 7fef8ffff640 1 -- 172.21.15.84:0/2817658950 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef9004e9b0 con 0x7fef900c2760 2023-10-16T09:49:40.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.740+0000 7fef87fff640 1 -- 172.21.15.84:0/2817658950 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fef88057ed0 con 0x7fef900c2760 2023-10-16T09:49:40.742 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:40.742 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":21,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:03.529261+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:40.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.742+0000 7fef8ffff640 1 -- 172.21.15.84:0/2817658950 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fef7006ae70 msgr2=0x7fef7006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:40.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.742+0000 7fef8ffff640 1 --2- 172.21.15.84:0/2817658950 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fef7006ae70 0x7fef7006d330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fef80036000 tx=0x7fef80030000 comp rx=0 tx=0).stop 2023-10-16T09:49:40.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.742+0000 7fef8ffff640 1 -- 172.21.15.84:0/2817658950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef900c2760 msgr2=0x7fef900c2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:40.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.742+0000 7fef8ffff640 1 --2- 172.21.15.84:0/2817658950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef900c2760 0x7fef900c2be0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fef900ca950 tx=0x7fef8800b4a0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.743+0000 7fef8ffff640 1 -- 172.21.15.84:0/2817658950 shutdown_connections 2023-10-16T09:49:40.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.743+0000 7fef8ffff640 1 --2- 172.21.15.84:0/2817658950 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fef7006ae70 0x7fef7006d330 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.743+0000 7fef8ffff640 1 --2- 172.21.15.84:0/2817658950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef900c2760 0x7fef900c2be0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.743+0000 7fef8ffff640 1 --2- 172.21.15.84:0/2817658950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef9006ada0 0x7fef9006b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.743+0000 7fef8ffff640 1 --2- 172.21.15.84:0/2817658950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c0039a0 0x7fef9014aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.743+0000 7fef8ffff640 1 -- 172.21.15.84:0/2817658950 >> 172.21.15.84:0/2817658950 conn(0x7fef900ce6a0 msgr2=0x7fef900ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:40.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.743+0000 7fef8ffff640 1 -- 172.21.15.84:0/2817658950 shutdown_connections 2023-10-16T09:49:40.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.743+0000 7fef8ffff640 1 -- 172.21.15.84:0/2817658950 wait complete. 2023-10-16T09:49:40.763 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:49:00.048649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:49:01.361365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:49:40.764 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:49:40.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.976+0000 7fc75f59e640 1 Processor -- start 2023-10-16T09:49:40.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.976+0000 7fc75f59e640 1 -- start start 2023-10-16T09:49:40.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.978+0000 7fc75f59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc7600b2080 0x7fc7600d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.978+0000 7fc75f59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7600cfc40 0x7fc7600b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.978+0000 7fc75f59e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7600d0970 con 0x7fc7600b1ca0 2023-10-16T09:49:40.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.978+0000 7fc75f59e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7600d0ea0 con 0x7fc7600b2080 2023-10-16T09:49:40.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.978+0000 7fc75f59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7600d0bc0 con 0x7fc7600cfc40 2023-10-16T09:49:40.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.978+0000 7fc757fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc7600b2080 0x7fc7600d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.978+0000 7fc757fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc7600b2080 0x7fc7600d36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34750/0 (socket says 172.21.15.84:34750) 2023-10-16T09:49:40.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.978+0000 7fc757fff640 1 -- 172.21.15.84:0/679731470 learned_addr learned my addr 172.21.15.84:0/679731470 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:40.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.979+0000 7fc75ed9d640 1 --2- 172.21.15.84:0/679731470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7600cfc40 0x7fc7600b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.979+0000 7fc757fff640 1 -- 172.21.15.84:0/679731470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7600cfc40 msgr2=0x7fc7600b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:40.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.979+0000 7fc757fff640 1 --2- 172.21.15.84:0/679731470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7600cfc40 0x7fc7600b24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.979+0000 7fc757fff640 1 -- 172.21.15.84:0/679731470 >> v1:172.21.15.157:6789/0 conn(0x7fc7600b1ca0 legacy=0x7fc7600ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:40.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.979+0000 7fc757fff640 1 -- 172.21.15.84:0/679731470 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7600cd940 con 0x7fc7600b2080 2023-10-16T09:49:40.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.979+0000 7fc75ed9d640 1 --2- 172.21.15.84:0/679731470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7600cfc40 0x7fc7600b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:40.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.980+0000 7fc757fff640 1 --2- 172.21.15.84:0/679731470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7600b2080 0x7fc7600d36b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc748009870 tx=0x7fc74802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918b976a00d7637a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:40.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.980+0000 7fc75dd9b640 1 -- 172.21.15.84:0/679731470 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748030840 con 0x7fc7600b2080 2023-10-16T09:49:40.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.980+0000 7fc75dd9b640 1 -- 172.21.15.84:0/679731470 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7480309a0 con 0x7fc7600b2080 2023-10-16T09:49:40.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.980+0000 7fc75dd9b640 1 -- 172.21.15.84:0/679731470 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748030d30 con 0x7fc7600b2080 2023-10-16T09:49:40.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.981+0000 7fc75f59e640 1 -- 172.21.15.84:0/679731470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7600b2080 msgr2=0x7fc7600d36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:40.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.981+0000 7fc75f59e640 1 --2- 172.21.15.84:0/679731470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7600b2080 0x7fc7600d36b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc748009870 tx=0x7fc74802b980 comp rx=0 tx=0).stop 2023-10-16T09:49:40.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.981+0000 7fc75f59e640 1 -- 172.21.15.84:0/679731470 shutdown_connections 2023-10-16T09:49:40.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.981+0000 7fc75f59e640 1 --2- 172.21.15.84:0/679731470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7600cfc40 0x7fc7600b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.981+0000 7fc75f59e640 1 --2- 172.21.15.84:0/679731470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7600b2080 0x7fc7600d36b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.981+0000 7fc75f59e640 1 -- 172.21.15.84:0/679731470 >> 172.21.15.84:0/679731470 conn(0x7fc7600d2b90 msgr2=0x7fc7600d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:40.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.981+0000 7fc75f59e640 1 -- 172.21.15.84:0/679731470 shutdown_connections 2023-10-16T09:49:40.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.981+0000 7fc75f59e640 1 -- 172.21.15.84:0/679731470 wait complete. 2023-10-16T09:49:40.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.982+0000 7fc75f59e640 1 Processor -- start 2023-10-16T09:49:40.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.983+0000 7fc75f59e640 1 -- start start 2023-10-16T09:49:40.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.984+0000 7fc75f59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7600b1ca0 0x7fc76009e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.984+0000 7fc75f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7600b2080 0x7fc76015d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.984+0000 7fc75f59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7600cfc40 0x7fc76009fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.984+0000 7fc75f59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7600c8740 con 0x7fc7600b2080 2023-10-16T09:49:40.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.984+0000 7fc75f59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76009d460 con 0x7fc7600b1ca0 2023-10-16T09:49:40.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.984+0000 7fc75f59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76009d740 con 0x7fc7600cfc40 2023-10-16T09:49:40.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.984+0000 7fc757fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7600b2080 0x7fc76015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.984+0000 7fc757fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7600b2080 0x7fc76015d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34766/0 (socket says 172.21.15.84:34766) 2023-10-16T09:49:40.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.984+0000 7fc757fff640 1 -- 172.21.15.84:0/4046380855 learned_addr learned my addr 172.21.15.84:0/4046380855 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:40.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.984+0000 7fc75e59c640 1 --2- 172.21.15.84:0/4046380855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7600b1ca0 0x7fc76009e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.984+0000 7fc757fff640 1 -- 172.21.15.84:0/4046380855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7600cfc40 msgr2=0x7fc76009fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:40.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.984+0000 7fc757fff640 1 --2- 172.21.15.84:0/4046380855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7600cfc40 0x7fc76009fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.984+0000 7fc757fff640 1 -- 172.21.15.84:0/4046380855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7600b1ca0 msgr2=0x7fc76009e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:40.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.984+0000 7fc757fff640 1 --2- 172.21.15.84:0/4046380855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7600b1ca0 0x7fc76009e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:40.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.984+0000 7fc757fff640 1 -- 172.21.15.84:0/4046380855 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7480094d0 con 0x7fc7600b2080 2023-10-16T09:49:40.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.984+0000 7fc75e59c640 1 --2- 172.21.15.84:0/4046380855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7600b1ca0 0x7fc76009e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:40.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.985+0000 7fc757fff640 1 --2- 172.21.15.84:0/4046380855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7600b2080 0x7fc76015d380 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc748009870 tx=0x7fc748004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:40.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.985+0000 7fc756ffd640 1 -- 172.21.15.84:0/4046380855 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7480307f0 con 0x7fc7600b2080 2023-10-16T09:49:40.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.985+0000 7fc756ffd640 1 -- 172.21.15.84:0/4046380855 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc748039d90 con 0x7fc7600b2080 2023-10-16T09:49:40.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.985+0000 7fc756ffd640 1 -- 172.21.15.84:0/4046380855 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748043630 con 0x7fc7600b2080 2023-10-16T09:49:40.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.985+0000 7fc75f59e640 1 -- 172.21.15.84:0/4046380855 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76009da20 con 0x7fc7600b2080 2023-10-16T09:49:40.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.985+0000 7fc75f59e640 1 -- 172.21.15.84:0/4046380855 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76009dfe0 con 0x7fc7600b2080 2023-10-16T09:49:40.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.986+0000 7fc75f59e640 1 -- 172.21.15.84:0/4046380855 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76004e9b0 con 0x7fc7600b2080 2023-10-16T09:49:40.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.987+0000 7fc756ffd640 1 -- 172.21.15.84:0/4046380855 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc748042390 con 0x7fc7600b2080 2023-10-16T09:49:40.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.988+0000 7fc756ffd640 1 --2- 172.21.15.84:0/4046380855 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc74006ae70 0x7fc74006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:40.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.988+0000 7fc756ffd640 1 -- 172.21.15.84:0/4046380855 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7480aba00 con 0x7fc7600b2080 2023-10-16T09:49:40.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.988+0000 7fc75e59c640 1 --2- 172.21.15.84:0/4046380855 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc74006ae70 0x7fc74006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:40.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.989+0000 7fc75e59c640 1 --2- 172.21.15.84:0/4046380855 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc74006ae70 0x7fc74006d330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc75000a830 tx=0x7fc750008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:40.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:40.992+0000 7fc756ffd640 1 -- 172.21.15.84:0/4046380855 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc748076aa0 con 0x7fc7600b2080 2023-10-16T09:49:41.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.183+0000 7fc75f59e640 1 -- 172.21.15.84:0/4046380855 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc760073a40 con 0x7fc7600b2080 2023-10-16T09:49:41.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.183+0000 7fc756ffd640 1 -- 172.21.15.84:0/4046380855 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc748034020 con 0x7fc7600b2080 2023-10-16T09:49:41.185 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T09:49:41.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.186+0000 7fc75f59e640 1 -- 172.21.15.84:0/4046380855 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc74006ae70 msgr2=0x7fc74006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:41.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.186+0000 7fc75f59e640 1 --2- 172.21.15.84:0/4046380855 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc74006ae70 0x7fc74006d330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc75000a830 tx=0x7fc750008040 comp rx=0 tx=0).stop 2023-10-16T09:49:41.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.186+0000 7fc75f59e640 1 -- 172.21.15.84:0/4046380855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7600b2080 msgr2=0x7fc76015d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:41.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.186+0000 7fc75f59e640 1 --2- 172.21.15.84:0/4046380855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7600b2080 0x7fc76015d380 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc748009870 tx=0x7fc748004730 comp rx=0 tx=0).stop 2023-10-16T09:49:41.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.186+0000 7fc75f59e640 1 -- 172.21.15.84:0/4046380855 shutdown_connections 2023-10-16T09:49:41.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.186+0000 7fc75f59e640 1 --2- 172.21.15.84:0/4046380855 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc74006ae70 0x7fc74006d330 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:41.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.186+0000 7fc75f59e640 1 --2- 172.21.15.84:0/4046380855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7600cfc40 0x7fc76009fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:41.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.186+0000 7fc75f59e640 1 --2- 172.21.15.84:0/4046380855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7600b2080 0x7fc76015d380 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:41.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.186+0000 7fc75f59e640 1 --2- 172.21.15.84:0/4046380855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7600b1ca0 0x7fc76009e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:41.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.186+0000 7fc75f59e640 1 -- 172.21.15.84:0/4046380855 >> 172.21.15.84:0/4046380855 conn(0x7fc7600d2b90 msgr2=0x7fc7600afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:41.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.186+0000 7fc75f59e640 1 -- 172.21.15.84:0/4046380855 shutdown_connections 2023-10-16T09:49:41.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.187+0000 7fc75f59e640 1 -- 172.21.15.84:0/4046380855 wait complete. 2023-10-16T09:49:41.207 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:49:41.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.423+0000 7f20ccd5e640 1 Processor -- start 2023-10-16T09:49:41.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.423+0000 7f20ccd5e640 1 -- start start 2023-10-16T09:49:41.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.425+0000 7f20ccd5e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f20c815ad70 0x7f20c8069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:41.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.425+0000 7f20ccd5e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c80cb030 con 0x7f20c80c9ca0 2023-10-16T09:49:41.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.425+0000 7f20ccd5e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c80c91c0 con 0x7f20c80caac0 2023-10-16T09:49:41.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.425+0000 7f20ccd5e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c80c94a0 con 0x7f20c815ad70 2023-10-16T09:49:41.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.425+0000 7f20c5d9b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f20c80c9ca0 0x7f20c80ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53640/0 (socket says 172.21.15.84:53640) 2023-10-16T09:49:41.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.425+0000 7f20c5d9b640 1 -- 172.21.15.84:0/3223471636 learned_addr learned my addr 172.21.15.84:0/3223471636 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:41.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.425+0000 7f20c659c640 1 --2- 172.21.15.84:0/3223471636 >> v2:172.21.15.157:3300/0 conn(0x7f20c815ad70 0x7f20c8069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:41.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.426+0000 7f20c559a640 1 -- 172.21.15.84:0/3223471636 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3162824925 0 0) 0x7f20c80cb030 con 0x7f20c80c9ca0 2023-10-16T09:49:41.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.426+0000 7f20c559a640 1 -- 172.21.15.84:0/3223471636 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20b4003520 con 0x7f20c80c9ca0 2023-10-16T09:49:41.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.426+0000 7f20c559a640 1 -- 172.21.15.84:0/3223471636 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1648127911 0 0) 0x7f20b4003520 con 0x7f20c80c9ca0 2023-10-16T09:49:41.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.426+0000 7f20c659c640 1 -- 172.21.15.84:0/3223471636 >> v1:172.21.15.157:6789/0 conn(0x7f20c80caac0 legacy=0x7f20c80ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:41.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.426+0000 7f20c659c640 1 -- 172.21.15.84:0/3223471636 >> v1:172.21.15.84:6789/0 conn(0x7f20c80c9ca0 legacy=0x7f20c80ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:41.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.426+0000 7f20c659c640 1 -- 172.21.15.84:0/3223471636 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c80c9780 con 0x7f20c815ad70 2023-10-16T09:49:41.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.427+0000 7f20c659c640 1 --2- 172.21.15.84:0/3223471636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c815ad70 0x7f20c8069da0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009d90 tx=0x7f20b802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52fe38d014934bb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:41.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.428+0000 7f20c559a640 1 -- 172.21.15.84:0/3223471636 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0001670 con 0x7f20c815ad70 2023-10-16T09:49:41.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.428+0000 7f20c559a640 1 -- 172.21.15.84:0/3223471636 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b802c940 con 0x7f20c815ad70 2023-10-16T09:49:41.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.428+0000 7f20c559a640 1 -- 172.21.15.84:0/3223471636 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b802ccd0 con 0x7f20c815ad70 2023-10-16T09:49:41.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.429+0000 7f20ccd5e640 1 -- 172.21.15.84:0/3223471636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c815ad70 msgr2=0x7f20c8069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:41.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.429+0000 7f20ccd5e640 1 --2- 172.21.15.84:0/3223471636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c815ad70 0x7f20c8069da0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009d90 tx=0x7f20b802be10 comp rx=0 tx=0).stop 2023-10-16T09:49:41.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.429+0000 7f20ccd5e640 1 -- 172.21.15.84:0/3223471636 shutdown_connections 2023-10-16T09:49:41.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.429+0000 7f20ccd5e640 1 --2- 172.21.15.84:0/3223471636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c815ad70 0x7f20c8069da0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:41.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.429+0000 7f20ccd5e640 1 -- 172.21.15.84:0/3223471636 >> 172.21.15.84:0/3223471636 conn(0x7f20c80ce9b0 msgr2=0x7f20c80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:41.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.429+0000 7f20ccd5e640 1 -- 172.21.15.84:0/3223471636 shutdown_connections 2023-10-16T09:49:41.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.429+0000 7f20ccd5e640 1 -- 172.21.15.84:0/3223471636 wait complete. 2023-10-16T09:49:41.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.430+0000 7f20ccd5e640 1 Processor -- start 2023-10-16T09:49:41.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.431+0000 7f20ccd5e640 1 -- start start 2023-10-16T09:49:41.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.431+0000 7f20ccd5e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20c80c9ca0 0x7f20c8151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:41.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.431+0000 7f20ccd5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c80caac0 0x7f20c80adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:41.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.432+0000 7f20ccd5e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c815ad70 0x7f20c80ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:41.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.432+0000 7f20ccd5e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c815b800 con 0x7f20c80c9ca0 2023-10-16T09:49:41.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.432+0000 7f20ccd5e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c815db00 con 0x7f20c80caac0 2023-10-16T09:49:41.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.432+0000 7f20ccd5e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c8152330 con 0x7f20c815ad70 2023-10-16T09:49:41.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.432+0000 7f20c659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20c80c9ca0 0x7f20c8151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:41.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.432+0000 7f20c659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20c80c9ca0 0x7f20c8151df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34774/0 (socket says 172.21.15.84:34774) 2023-10-16T09:49:41.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.432+0000 7f20c659c640 1 -- 172.21.15.84:0/1802844528 learned_addr learned my addr 172.21.15.84:0/1802844528 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:41.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.432+0000 7f20c5d9b640 1 --2- 172.21.15.84:0/1802844528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c80caac0 0x7f20c80adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:41.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.432+0000 7f20c659c640 1 -- 172.21.15.84:0/1802844528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c815ad70 msgr2=0x7f20c80ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:41.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.432+0000 7f20c659c640 1 --2- 172.21.15.84:0/1802844528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c815ad70 0x7f20c80ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:41.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.432+0000 7f20c659c640 1 -- 172.21.15.84:0/1802844528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c80caac0 msgr2=0x7f20c80adbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:41.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.432+0000 7f20c659c640 1 --2- 172.21.15.84:0/1802844528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c80caac0 0x7f20c80adbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:41.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.432+0000 7f20c659c640 1 -- 172.21.15.84:0/1802844528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b8009a70 con 0x7f20c80c9ca0 2023-10-16T09:49:41.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.432+0000 7f20c5d9b640 1 --2- 172.21.15.84:0/1802844528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c80caac0 0x7f20c80adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:41.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.433+0000 7f20c659c640 1 --2- 172.21.15.84:0/1802844528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20c80c9ca0 0x7f20c8151df0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f20b8037040 tx=0x7f20b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:41.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.433+0000 7f20beffd640 1 -- 172.21.15.84:0/1802844528 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8009450 con 0x7f20c80c9ca0 2023-10-16T09:49:41.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.433+0000 7f20beffd640 1 -- 172.21.15.84:0/1802844528 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b8003990 con 0x7f20c80c9ca0 2023-10-16T09:49:41.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.433+0000 7f20beffd640 1 -- 172.21.15.84:0/1802844528 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8003d40 con 0x7f20c80c9ca0 2023-10-16T09:49:41.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.433+0000 7f20ccd5e640 1 -- 172.21.15.84:0/1802844528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c815cf80 con 0x7f20c80c9ca0 2023-10-16T09:49:41.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.433+0000 7f20ccd5e640 1 -- 172.21.15.84:0/1802844528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c815d540 con 0x7f20c80c9ca0 2023-10-16T09:49:41.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.434+0000 7f20ccd5e640 1 -- 172.21.15.84:0/1802844528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c804e160 con 0x7f20c80c9ca0 2023-10-16T09:49:41.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.435+0000 7f20beffd640 1 -- 172.21.15.84:0/1802844528 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f20b803b050 con 0x7f20c80c9ca0 2023-10-16T09:49:41.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.436+0000 7f20beffd640 1 --2- 172.21.15.84:0/1802844528 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f20a806aea0 0x7f20a806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:41.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.436+0000 7f20beffd640 1 -- 172.21.15.84:0/1802844528 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20b80abc40 con 0x7f20c80c9ca0 2023-10-16T09:49:41.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.436+0000 7f20c5d9b640 1 --2- 172.21.15.84:0/1802844528 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f20a806aea0 0x7f20a806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:41.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.437+0000 7f20c5d9b640 1 --2- 172.21.15.84:0/1802844528 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f20a806aea0 0x7f20a806d360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f20b00011e0 tx=0x7f20b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:41.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.441+0000 7f20beffd640 1 -- 172.21.15.84:0/1802844528 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f20b8076e10 con 0x7f20c80c9ca0 2023-10-16T09:49:41.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.632+0000 7f20ccd5e640 1 -- 172.21.15.84:0/1802844528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20c80c8cc0 con 0x7f20c80c9ca0 2023-10-16T09:49:41.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.633+0000 7f20beffd640 1 -- 172.21.15.84:0/1802844528 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f20b804d020 con 0x7f20c80c9ca0 2023-10-16T09:49:41.635 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T09:49:41.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.636+0000 7f20ccd5e640 1 -- 172.21.15.84:0/1802844528 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f20a806aea0 msgr2=0x7f20a806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:41.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.636+0000 7f20ccd5e640 1 --2- 172.21.15.84:0/1802844528 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f20a806aea0 0x7f20a806d360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f20b00011e0 tx=0x7f20b0009040 comp rx=0 tx=0).stop 2023-10-16T09:49:41.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.636+0000 7f20ccd5e640 1 -- 172.21.15.84:0/1802844528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20c80c9ca0 msgr2=0x7f20c8151df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:41.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.636+0000 7f20ccd5e640 1 --2- 172.21.15.84:0/1802844528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20c80c9ca0 0x7f20c8151df0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f20b8037040 tx=0x7f20b8004a40 comp rx=0 tx=0).stop 2023-10-16T09:49:41.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.636+0000 7f20ccd5e640 1 -- 172.21.15.84:0/1802844528 shutdown_connections 2023-10-16T09:49:41.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.636+0000 7f20ccd5e640 1 --2- 172.21.15.84:0/1802844528 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f20a806aea0 0x7f20a806d360 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:41.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.636+0000 7f20ccd5e640 1 --2- 172.21.15.84:0/1802844528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c815ad70 0x7f20c80ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:41.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.636+0000 7f20ccd5e640 1 --2- 172.21.15.84:0/1802844528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c80caac0 0x7f20c80adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:41.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.636+0000 7f20ccd5e640 1 --2- 172.21.15.84:0/1802844528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20c80c9ca0 0x7f20c8151df0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:41.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.636+0000 7f20ccd5e640 1 -- 172.21.15.84:0/1802844528 >> 172.21.15.84:0/1802844528 conn(0x7f20c80ce9b0 msgr2=0x7f20c80b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:41.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.636+0000 7f20ccd5e640 1 -- 172.21.15.84:0/1802844528 shutdown_connections 2023-10-16T09:49:41.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.636+0000 7f20ccd5e640 1 -- 172.21.15.84:0/1802844528 wait complete. 2023-10-16T09:49:41.657 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:49:41.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.858+0000 7ff4866c6640 1 Processor -- start 2023-10-16T09:49:41.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.859+0000 7ff4866c6640 1 -- start start 2023-10-16T09:49:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.860+0000 7ff4866c6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff48015ad70 0x7ff480069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.860+0000 7ff4866c6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4800c9ca0 0x7ff48006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.860+0000 7ff4866c6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4800cb030 con 0x7ff4800ca3b0 2023-10-16T09:49:41.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.860+0000 7ff4866c6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4800c91c0 con 0x7ff48015ad70 2023-10-16T09:49:41.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.860+0000 7ff4866c6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4800c94a0 con 0x7ff4800c9ca0 2023-10-16T09:49:41.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.860+0000 7ff47ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff48015ad70 0x7ff480069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:41.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.860+0000 7ff47ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff48015ad70 0x7ff480069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34782/0 (socket says 172.21.15.84:34782) 2023-10-16T09:49:41.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.860+0000 7ff484c63640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff4800ca3b0 0x7ff4800ca840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53648/0 (socket says 172.21.15.84:53648) 2023-10-16T09:49:41.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.861+0000 7ff47ffff640 1 -- 172.21.15.84:0/3696839194 learned_addr learned my addr 172.21.15.84:0/3696839194 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:41.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.861+0000 7ff47f7fe640 1 --2- 172.21.15.84:0/3696839194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4800c9ca0 0x7ff48006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:41.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.861+0000 7ff47ffff640 1 -- 172.21.15.84:0/3696839194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4800c9ca0 msgr2=0x7ff48006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:41.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.861+0000 7ff47ffff640 1 --2- 172.21.15.84:0/3696839194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4800c9ca0 0x7ff48006a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:41.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.861+0000 7ff47ffff640 1 -- 172.21.15.84:0/3696839194 >> v1:172.21.15.84:6789/0 conn(0x7ff4800ca3b0 legacy=0x7ff4800ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:41.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.861+0000 7ff47ffff640 1 -- 172.21.15.84:0/3696839194 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4800c9780 con 0x7ff48015ad70 2023-10-16T09:49:41.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.861+0000 7ff47f7fe640 1 --2- 172.21.15.84:0/3696839194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4800c9ca0 0x7ff48006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:41.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.861+0000 7ff47effd640 1 -- 172.21.15.84:0/3696839194 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3153164718 0 0) 0x7ff4800cb030 con 0x7ff4800ca3b0 2023-10-16T09:49:41.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.862+0000 7ff47ffff640 1 --2- 172.21.15.84:0/3696839194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48015ad70 0x7ff480069da0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff468009870 tx=0x7ff46802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ce65a489ae5336 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:41.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.863+0000 7ff47effd640 1 -- 172.21.15.84:0/3696839194 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468030840 con 0x7ff48015ad70 2023-10-16T09:49:41.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.863+0000 7ff47effd640 1 -- 172.21.15.84:0/3696839194 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4680309a0 con 0x7ff48015ad70 2023-10-16T09:49:41.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.863+0000 7ff47effd640 1 -- 172.21.15.84:0/3696839194 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468030d30 con 0x7ff48015ad70 2023-10-16T09:49:41.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.863+0000 7ff4866c6640 1 -- 172.21.15.84:0/3696839194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48015ad70 msgr2=0x7ff480069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:41.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.863+0000 7ff4866c6640 1 --2- 172.21.15.84:0/3696839194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48015ad70 0x7ff480069da0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff468009870 tx=0x7ff46802b980 comp rx=0 tx=0).stop 2023-10-16T09:49:41.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.863+0000 7ff4866c6640 1 -- 172.21.15.84:0/3696839194 shutdown_connections 2023-10-16T09:49:41.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.863+0000 7ff4866c6640 1 --2- 172.21.15.84:0/3696839194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4800c9ca0 0x7ff48006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:41.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.863+0000 7ff4866c6640 1 --2- 172.21.15.84:0/3696839194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48015ad70 0x7ff480069da0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:41.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.863+0000 7ff4866c6640 1 -- 172.21.15.84:0/3696839194 >> 172.21.15.84:0/3696839194 conn(0x7ff4800ce9b0 msgr2=0x7ff4800cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:41.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.863+0000 7ff4866c6640 1 -- 172.21.15.84:0/3696839194 shutdown_connections 2023-10-16T09:49:41.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.863+0000 7ff4866c6640 1 -- 172.21.15.84:0/3696839194 wait complete. 2023-10-16T09:49:41.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.865+0000 7ff4866c6640 1 Processor -- start 2023-10-16T09:49:41.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.865+0000 7ff4866c6640 1 -- start start 2023-10-16T09:49:41.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.866+0000 7ff4866c6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4800c9ca0 0x7ff480150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:41.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.866+0000 7ff4866c6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4800ca3b0 0x7ff4800c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:41.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.866+0000 7ff4866c6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48015ad70 0x7ff4800c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:41.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.866+0000 7ff4866c6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48014f860 con 0x7ff48015ad70 2023-10-16T09:49:41.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.866+0000 7ff4866c6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4800c6780 con 0x7ff4800c9ca0 2023-10-16T09:49:41.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.866+0000 7ff4866c6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4800c5bb0 con 0x7ff4800ca3b0 2023-10-16T09:49:41.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.866+0000 7ff484c63640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48015ad70 0x7ff4800c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:41.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.866+0000 7ff484c63640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48015ad70 0x7ff4800c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34792/0 (socket says 172.21.15.84:34792) 2023-10-16T09:49:41.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.866+0000 7ff484c63640 1 -- 172.21.15.84:0/1662981935 learned_addr learned my addr 172.21.15.84:0/1662981935 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:41.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.867+0000 7ff47f7fe640 1 --2- 172.21.15.84:0/1662981935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4800ca3b0 0x7ff4800c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:41.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.867+0000 7ff484c63640 1 -- 172.21.15.84:0/1662981935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4800ca3b0 msgr2=0x7ff4800c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:41.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.867+0000 7ff484c63640 1 --2- 172.21.15.84:0/1662981935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4800ca3b0 0x7ff4800c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:41.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.867+0000 7ff484c63640 1 -- 172.21.15.84:0/1662981935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4800c9ca0 msgr2=0x7ff480150720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:41.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.867+0000 7ff484c63640 1 --2- 172.21.15.84:0/1662981935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4800c9ca0 0x7ff480150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:41.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.867+0000 7ff484c63640 1 -- 172.21.15.84:0/1662981935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4680094d0 con 0x7ff48015ad70 2023-10-16T09:49:41.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.867+0000 7ff484c63640 1 --2- 172.21.15.84:0/1662981935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48015ad70 0x7ff4800c7150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff47800c450 tx=0x7ff47800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:41.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.867+0000 7ff47cff9640 1 -- 172.21.15.84:0/1662981935 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4780074e0 con 0x7ff48015ad70 2023-10-16T09:49:41.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.867+0000 7ff47cff9640 1 -- 172.21.15.84:0/1662981935 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff478007660 con 0x7ff48015ad70 2023-10-16T09:49:41.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.867+0000 7ff47cff9640 1 -- 172.21.15.84:0/1662981935 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff478004a20 con 0x7ff48015ad70 2023-10-16T09:49:41.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.867+0000 7ff4866c6640 1 -- 172.21.15.84:0/1662981935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4800c5e50 con 0x7ff48015ad70 2023-10-16T09:49:41.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.868+0000 7ff4866c6640 1 -- 172.21.15.84:0/1662981935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4800c6410 con 0x7ff48015ad70 2023-10-16T09:49:41.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.869+0000 7ff4866c6640 1 -- 172.21.15.84:0/1662981935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48004e160 con 0x7ff48015ad70 2023-10-16T09:49:41.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.869+0000 7ff47cff9640 1 -- 172.21.15.84:0/1662981935 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff478004b80 con 0x7ff48015ad70 2023-10-16T09:49:41.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.870+0000 7ff47cff9640 1 --2- 172.21.15.84:0/1662981935 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff46006ae70 0x7ff46006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:41.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.870+0000 7ff47cff9640 1 -- 172.21.15.84:0/1662981935 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4780886f0 con 0x7ff48015ad70 2023-10-16T09:49:41.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.871+0000 7ff47ffff640 1 --2- 172.21.15.84:0/1662981935 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff46006ae70 0x7ff46006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:41.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.871+0000 7ff47ffff640 1 --2- 172.21.15.84:0/1662981935 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff46006ae70 0x7ff46006d330 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff468036000 tx=0x7ff468030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:41.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:41.874+0000 7ff47cff9640 1 -- 172.21.15.84:0/1662981935 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff478053790 con 0x7ff48015ad70 2023-10-16T09:49:42.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.071+0000 7ff4866c6640 1 -- 172.21.15.84:0/1662981935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4800c68f0 con 0x7ff48015ad70 2023-10-16T09:49:42.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.072+0000 7ff47cff9640 1 -- 172.21.15.84:0/1662981935 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff478056db0 con 0x7ff48015ad70 2023-10-16T09:49:42.074 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T09:49:42.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.074+0000 7ff4866c6640 1 -- 172.21.15.84:0/1662981935 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff46006ae70 msgr2=0x7ff46006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:42.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.074+0000 7ff4866c6640 1 --2- 172.21.15.84:0/1662981935 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff46006ae70 0x7ff46006d330 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff468036000 tx=0x7ff468030000 comp rx=0 tx=0).stop 2023-10-16T09:49:42.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.075+0000 7ff4866c6640 1 -- 172.21.15.84:0/1662981935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48015ad70 msgr2=0x7ff4800c7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:42.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.075+0000 7ff4866c6640 1 --2- 172.21.15.84:0/1662981935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48015ad70 0x7ff4800c7150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff47800c450 tx=0x7ff47800c910 comp rx=0 tx=0).stop 2023-10-16T09:49:42.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.075+0000 7ff4866c6640 1 -- 172.21.15.84:0/1662981935 shutdown_connections 2023-10-16T09:49:42.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.075+0000 7ff4866c6640 1 --2- 172.21.15.84:0/1662981935 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff46006ae70 0x7ff46006d330 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.075+0000 7ff4866c6640 1 --2- 172.21.15.84:0/1662981935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48015ad70 0x7ff4800c7150 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.075+0000 7ff4866c6640 1 --2- 172.21.15.84:0/1662981935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4800ca3b0 0x7ff4800c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.075+0000 7ff4866c6640 1 --2- 172.21.15.84:0/1662981935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4800c9ca0 0x7ff480150720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.075+0000 7ff4866c6640 1 -- 172.21.15.84:0/1662981935 >> 172.21.15.84:0/1662981935 conn(0x7ff4800ce9b0 msgr2=0x7ff4800ca840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:42.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.075+0000 7ff4866c6640 1 -- 172.21.15.84:0/1662981935 shutdown_connections 2023-10-16T09:49:42.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.075+0000 7ff4866c6640 1 -- 172.21.15.84:0/1662981935 wait complete. 2023-10-16T09:49:42.106 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-10-16T09:49:42.107 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T09:49:42.107 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:42.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.321+0000 7f631aabb640 1 Processor -- start 2023-10-16T09:49:42.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.321+0000 7f631aabb640 1 -- start start 2023-10-16T09:49:42.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.323+0000 7f631aabb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f631415ae80 0x7f631415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:42.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.323+0000 7f631aabb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f631415ed60 con 0x7f63140ce390 2023-10-16T09:49:42.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.323+0000 7f631aabb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63140707e0 con 0x7f631415aaa0 2023-10-16T09:49:42.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.323+0000 7f631aabb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631406b810 con 0x7f631415ae80 2023-10-16T09:49:42.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.323+0000 7f6319058640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f63140ce390 0x7f6314070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53654/0 (socket says 172.21.15.84:53654) 2023-10-16T09:49:42.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.323+0000 7f6319058640 1 -- 172.21.15.84:0/2339217615 learned_addr learned my addr 172.21.15.84:0/2339217615 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:42.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.324+0000 7f6313fff640 1 --2- 172.21.15.84:0/2339217615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f631415ae80 0x7f631415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:42.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.324+0000 7f63137fe640 1 -- 172.21.15.84:0/2339217615 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448940672 0 0) 0x7f631415ed60 con 0x7f63140ce390 2023-10-16T09:49:42.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.324+0000 7f63137fe640 1 -- 172.21.15.84:0/2339217615 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6304003520 con 0x7f63140ce390 2023-10-16T09:49:42.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.324+0000 7f63137fe640 1 -- 172.21.15.84:0/2339217615 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2232960111 0 0) 0x7f6304003520 con 0x7f63140ce390 2023-10-16T09:49:42.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.324+0000 7f63137fe640 1 -- 172.21.15.84:0/2339217615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f631415ae80 msgr2=0x7f631415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:42.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.324+0000 7f63137fe640 1 --2- 172.21.15.84:0/2339217615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f631415ae80 0x7f631415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.324+0000 7f63137fe640 1 -- 172.21.15.84:0/2339217615 >> v1:172.21.15.157:6789/0 conn(0x7f631415aaa0 legacy=0x7f63140cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:42.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.324+0000 7f63137fe640 1 -- 172.21.15.84:0/2339217615 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631415ef90 con 0x7f63140ce390 2023-10-16T09:49:42.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.324+0000 7f63137fe640 1 -- 172.21.15.84:0/2339217615 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f630c0019f0 con 0x7f63140ce390 2023-10-16T09:49:42.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.324+0000 7f63137fe640 1 -- 172.21.15.84:0/2339217615 >> v1:172.21.15.84:6789/0 conn(0x7f63140ce390 legacy=0x7f6314070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:42.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.324+0000 7f63137fe640 1 --2- 172.21.15.84:0/2339217615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63040039a0 0x7f6304003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:42.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.324+0000 7f63137fe640 1 --2- 172.21.15.84:0/2339217615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f631415ae80 0x7f63040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:42.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.325+0000 7f63137fe640 1 --2- 172.21.15.84:0/2339217615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6304004d00 0x7f6304005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:42.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.325+0000 7f63137fe640 1 -- 172.21.15.84:0/2339217615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631406b810 con 0x7f6304004d00 2023-10-16T09:49:42.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.325+0000 7f63137fe640 1 -- 172.21.15.84:0/2339217615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630400ba30 con 0x7f63040039a0 2023-10-16T09:49:42.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.325+0000 7f63137fe640 1 -- 172.21.15.84:0/2339217615 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630400bbf0 con 0x7f631415ae80 2023-10-16T09:49:42.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.325+0000 7f6319058640 1 --2- 172.21.15.84:0/2339217615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6304004d00 0x7f6304005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:42.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.325+0000 7f6318857640 1 --2- 172.21.15.84:0/2339217615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63040039a0 0x7f6304003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:42.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.325+0000 7f6313fff640 1 --2- 172.21.15.84:0/2339217615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f631415ae80 0x7f63040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:42.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.325+0000 7f6319058640 1 -- 172.21.15.84:0/2339217615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f631415ae80 msgr2=0x7f63040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:42.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.325+0000 7f6319058640 1 --2- 172.21.15.84:0/2339217615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f631415ae80 0x7f63040047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.325+0000 7f6319058640 1 -- 172.21.15.84:0/2339217615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63040039a0 msgr2=0x7f6304003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:42.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.325+0000 7f6319058640 1 --2- 172.21.15.84:0/2339217615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63040039a0 0x7f6304003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.325+0000 7f6319058640 1 -- 172.21.15.84:0/2339217615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6304003520 con 0x7f6304004d00 2023-10-16T09:49:42.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.325+0000 7f6313fff640 1 --2- 172.21.15.84:0/2339217615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f631415ae80 0x7f63040047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:42.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.326+0000 7f6319058640 1 --2- 172.21.15.84:0/2339217615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6304004d00 0x7f6304005180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f630c00a830 tx=0x7f63040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58306ffcc7503731 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:42.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.327+0000 7f63137fe640 1 -- 172.21.15.84:0/2339217615 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c02cea0 con 0x7f6304004d00 2023-10-16T09:49:42.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.327+0000 7f63137fe640 1 -- 172.21.15.84:0/2339217615 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630c02f070 con 0x7f6304004d00 2023-10-16T09:49:42.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.327+0000 7f63137fe640 1 -- 172.21.15.84:0/2339217615 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c034890 con 0x7f6304004d00 2023-10-16T09:49:42.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.327+0000 7f631aabb640 1 -- 172.21.15.84:0/2339217615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6304004d00 msgr2=0x7f6304005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:42.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.327+0000 7f631aabb640 1 --2- 172.21.15.84:0/2339217615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6304004d00 0x7f6304005180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f630c00a830 tx=0x7f63040057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.327+0000 7f6319058640 1 -- 172.21.15.84:0/2339217615 reap_dead start 2023-10-16T09:49:42.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.327+0000 7f631aabb640 1 -- 172.21.15.84:0/2339217615 shutdown_connections 2023-10-16T09:49:42.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.327+0000 7f631aabb640 1 -- 172.21.15.84:0/2339217615 >> 172.21.15.84:0/2339217615 conn(0x7f63140cb410 msgr2=0x7f63140cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:42.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.328+0000 7f631aabb640 1 -- 172.21.15.84:0/2339217615 shutdown_connections 2023-10-16T09:49:42.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.328+0000 7f631aabb640 1 -- 172.21.15.84:0/2339217615 wait complete. 2023-10-16T09:49:42.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.329+0000 7f631aabb640 1 Processor -- start 2023-10-16T09:49:42.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.329+0000 7f631aabb640 1 -- start start 2023-10-16T09:49:42.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.330+0000 7f631aabb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6304004d00 0x7f63140c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:42.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.330+0000 7f631aabb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f631415b4d0 0x7f631414eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:42.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.330+0000 7f631aabb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63140b1ed0 0x7f63140b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:42.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.330+0000 7f631aabb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63140b28a0 con 0x7f631415b4d0 2023-10-16T09:49:42.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.330+0000 7f631aabb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63140c4b30 con 0x7f63140b1ed0 2023-10-16T09:49:42.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.330+0000 7f631aabb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631415ba90 con 0x7f6304004d00 2023-10-16T09:49:42.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.331+0000 7f6313fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f631415b4d0 0x7f631414eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:42.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.331+0000 7f6313fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f631415b4d0 0x7f631414eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34810/0 (socket says 172.21.15.84:34810) 2023-10-16T09:49:42.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.331+0000 7f6313fff640 1 -- 172.21.15.84:0/2690431583 learned_addr learned my addr 172.21.15.84:0/2690431583 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:42.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.331+0000 7f6319058640 1 --2- 172.21.15.84:0/2690431583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63140b1ed0 0x7f63140b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:42.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.331+0000 7f6318857640 1 --2- 172.21.15.84:0/2690431583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6304004d00 0x7f63140c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:42.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.331+0000 7f6313fff640 1 -- 172.21.15.84:0/2690431583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6304004d00 msgr2=0x7f63140c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:42.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.331+0000 7f6313fff640 1 --2- 172.21.15.84:0/2690431583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6304004d00 0x7f63140c45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.331+0000 7f6313fff640 1 -- 172.21.15.84:0/2690431583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63140b1ed0 msgr2=0x7f63140b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:42.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.331+0000 7f6313fff640 1 --2- 172.21.15.84:0/2690431583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63140b1ed0 0x7f63140b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.331+0000 7f6313fff640 1 -- 172.21.15.84:0/2690431583 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631415bd70 con 0x7f631415b4d0 2023-10-16T09:49:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.331+0000 7f6318857640 1 --2- 172.21.15.84:0/2690431583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6304004d00 0x7f63140c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.331+0000 7f6313fff640 1 --2- 172.21.15.84:0/2690431583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f631415b4d0 0x7f631414eea0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f63000108b0 tx=0x7f6300010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.332+0000 7f63117fa640 1 -- 172.21.15.84:0/2690431583 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630000eb60 con 0x7f631415b4d0 2023-10-16T09:49:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.332+0000 7f63117fa640 1 -- 172.21.15.84:0/2690431583 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630000ecc0 con 0x7f631415b4d0 2023-10-16T09:49:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.332+0000 7f63117fa640 1 -- 172.21.15.84:0/2690431583 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630001c400 con 0x7f631415b4d0 2023-10-16T09:49:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.332+0000 7f631aabb640 1 -- 172.21.15.84:0/2690431583 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631415c050 con 0x7f631415b4d0 2023-10-16T09:49:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.332+0000 7f631aabb640 1 -- 172.21.15.84:0/2690431583 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631415c610 con 0x7f631415b4d0 2023-10-16T09:49:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.333+0000 7f631aabb640 1 -- 172.21.15.84:0/2690431583 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631404e160 con 0x7f631415b4d0 2023-10-16T09:49:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.334+0000 7f63117fa640 1 -- 172.21.15.84:0/2690431583 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f630001c560 con 0x7f631415b4d0 2023-10-16T09:49:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.335+0000 7f63117fa640 1 --2- 172.21.15.84:0/2690431583 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f62f806ae70 0x7f62f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:42.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.335+0000 7f63117fa640 1 -- 172.21.15.84:0/2690431583 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6300089c20 con 0x7f631415b4d0 2023-10-16T09:49:42.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.335+0000 7f6318857640 1 --2- 172.21.15.84:0/2690431583 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f62f806ae70 0x7f62f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:42.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.336+0000 7f6318857640 1 --2- 172.21.15.84:0/2690431583 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f62f806ae70 0x7f62f806d330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6308004be0 tx=0x7f6308009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:42.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.339+0000 7f63117fa640 1 -- 172.21.15.84:0/2690431583 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6300054d70 con 0x7f631415b4d0 2023-10-16T09:49:42.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.531+0000 7f631aabb640 1 -- 172.21.15.84:0/2690431583 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63140cdc60 con 0x7f631415b4d0 2023-10-16T09:49:42.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.533+0000 7f63117fa640 1 -- 172.21.15.84:0/2690431583 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f6300058390 con 0x7f631415b4d0 2023-10-16T09:49:42.536 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:42.536 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":21,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:03.529261+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:42.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.536+0000 7f631aabb640 1 -- 172.21.15.84:0/2690431583 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f62f806ae70 msgr2=0x7f62f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:42.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.536+0000 7f631aabb640 1 --2- 172.21.15.84:0/2690431583 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f62f806ae70 0x7f62f806d330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6308004be0 tx=0x7f6308009040 comp rx=0 tx=0).stop 2023-10-16T09:49:42.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.536+0000 7f631aabb640 1 -- 172.21.15.84:0/2690431583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f631415b4d0 msgr2=0x7f631414eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:42.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.536+0000 7f631aabb640 1 --2- 172.21.15.84:0/2690431583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f631415b4d0 0x7f631414eea0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f63000108b0 tx=0x7f6300010d70 comp rx=0 tx=0).stop 2023-10-16T09:49:42.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.536+0000 7f631aabb640 1 -- 172.21.15.84:0/2690431583 shutdown_connections 2023-10-16T09:49:42.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.536+0000 7f631aabb640 1 --2- 172.21.15.84:0/2690431583 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f62f806ae70 0x7f62f806d330 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.536+0000 7f631aabb640 1 --2- 172.21.15.84:0/2690431583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63140b1ed0 0x7f63140b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.536+0000 7f631aabb640 1 --2- 172.21.15.84:0/2690431583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f631415b4d0 0x7f631414eea0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.536+0000 7f631aabb640 1 --2- 172.21.15.84:0/2690431583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6304004d00 0x7f63140c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.536+0000 7f631aabb640 1 -- 172.21.15.84:0/2690431583 >> 172.21.15.84:0/2690431583 conn(0x7f63140cb410 msgr2=0x7f6314159e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:42.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.536+0000 7f631aabb640 1 -- 172.21.15.84:0/2690431583 shutdown_connections 2023-10-16T09:49:42.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.536+0000 7f631aabb640 1 -- 172.21.15.84:0/2690431583 wait complete. 2023-10-16T09:49:42.557 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:49:00.048649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:49:01.361365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:49:42.558 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:49:42.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.766+0000 7f54c80d4640 1 Processor -- start 2023-10-16T09:49:42.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.767+0000 7f54c80d4640 1 -- start start 2023-10-16T09:49:42.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.768+0000 7f54c80d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c015ad70 0x7f54c0069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:42.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.768+0000 7f54c80d4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54c00caac0 0x7f54c006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:42.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.768+0000 7f54c80d4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c00cb030 con 0x7f54c00c9ca0 2023-10-16T09:49:42.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.768+0000 7f54c80d4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54c00c91c0 con 0x7f54c00caac0 2023-10-16T09:49:42.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.768+0000 7f54c80d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c00c94a0 con 0x7f54c015ad70 2023-10-16T09:49:42.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.768+0000 7f54c566f640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f54c00c9ca0 0x7f54c00ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53656/0 (socket says 172.21.15.84:53656) 2023-10-16T09:49:42.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.768+0000 7f54c566f640 1 -- 172.21.15.84:0/874890775 learned_addr learned my addr 172.21.15.84:0/874890775 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:42.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c4e6e640 1 -- 172.21.15.84:0/874890775 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3872631094 0 0) 0x7f54c00cb030 con 0x7f54c00c9ca0 2023-10-16T09:49:42.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c4e6e640 1 -- 172.21.15.84:0/874890775 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54ac003520 con 0x7f54c00c9ca0 2023-10-16T09:49:42.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c5e70640 1 --2- 172.21.15.84:0/874890775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c015ad70 0x7f54c0069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:42.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c6671640 1 --2- 172.21.15.84:0/874890775 >> v2:172.21.15.157:3300/0 conn(0x7f54c00caac0 0x7f54c006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:42.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c4e6e640 1 -- 172.21.15.84:0/874890775 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3626067736 0 0) 0x7f54ac003520 con 0x7f54c00c9ca0 2023-10-16T09:49:42.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c4e6e640 1 -- 172.21.15.84:0/874890775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c015ad70 msgr2=0x7f54c0069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:42.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c4e6e640 1 --2- 172.21.15.84:0/874890775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c015ad70 0x7f54c0069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c4e6e640 1 -- 172.21.15.84:0/874890775 >> v2:172.21.15.157:3300/0 conn(0x7f54c00caac0 msgr2=0x7f54c006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:42.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c4e6e640 1 --2- 172.21.15.84:0/874890775 >> v2:172.21.15.157:3300/0 conn(0x7f54c00caac0 0x7f54c006a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c4e6e640 1 -- 172.21.15.84:0/874890775 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c00c9780 con 0x7f54c00c9ca0 2023-10-16T09:49:42.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c4e6e640 1 -- 172.21.15.84:0/874890775 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f54b0001670 con 0x7f54c00c9ca0 2023-10-16T09:49:42.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c4e6e640 1 -- 172.21.15.84:0/874890775 >> v1:172.21.15.84:6789/0 conn(0x7f54c00c9ca0 legacy=0x7f54c00ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:42.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c4e6e640 1 --2- 172.21.15.84:0/874890775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c015ad70 0x7f54ac003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:42.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c4e6e640 1 --2- 172.21.15.84:0/874890775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54ac003ff0 0x7f54ac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:42.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c4e6e640 1 --2- 172.21.15.84:0/874890775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54ac004d00 0x7f54ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:42.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c4e6e640 1 -- 172.21.15.84:0/874890775 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c00c91c0 con 0x7f54ac004d00 2023-10-16T09:49:42.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c4e6e640 1 -- 172.21.15.84:0/874890775 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c00c94a0 con 0x7f54ac003ff0 2023-10-16T09:49:42.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c4e6e640 1 -- 172.21.15.84:0/874890775 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac00bae0 con 0x7f54c015ad70 2023-10-16T09:49:42.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.769+0000 7f54c6671640 1 --2- 172.21.15.84:0/874890775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54ac004d00 0x7f54ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:42.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.770+0000 7f54c6671640 1 -- 172.21.15.84:0/874890775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c015ad70 msgr2=0x7f54ac003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:42.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.770+0000 7f54c6671640 1 --2- 172.21.15.84:0/874890775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c015ad70 0x7f54ac003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.770+0000 7f54c6671640 1 -- 172.21.15.84:0/874890775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54ac003ff0 msgr2=0x7f54ac0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:42.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.770+0000 7f54c6671640 1 --2- 172.21.15.84:0/874890775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54ac003ff0 0x7f54ac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.770+0000 7f54c6671640 1 -- 172.21.15.84:0/874890775 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac003520 con 0x7f54ac004d00 2023-10-16T09:49:42.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.771+0000 7f54c6671640 1 --2- 172.21.15.84:0/874890775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54ac004d00 0x7f54ac005180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f54b8007360 tx=0x7f54ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198bc9b56aa08ad2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:42.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.771+0000 7f54c4e6e640 1 -- 172.21.15.84:0/874890775 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b802ccf0 con 0x7f54ac004d00 2023-10-16T09:49:42.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.771+0000 7f54c4e6e640 1 -- 172.21.15.84:0/874890775 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b802ce50 con 0x7f54ac004d00 2023-10-16T09:49:42.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.771+0000 7f54c4e6e640 1 -- 172.21.15.84:0/874890775 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8034660 con 0x7f54ac004d00 2023-10-16T09:49:42.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.772+0000 7f54c80d4640 1 -- 172.21.15.84:0/874890775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54ac004d00 msgr2=0x7f54ac005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:42.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.772+0000 7f54c80d4640 1 --2- 172.21.15.84:0/874890775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54ac004d00 0x7f54ac005180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f54b8007360 tx=0x7f54ac0057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.772+0000 7f54c6671640 1 -- 172.21.15.84:0/874890775 reap_dead start 2023-10-16T09:49:42.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.772+0000 7f54c80d4640 1 -- 172.21.15.84:0/874890775 shutdown_connections 2023-10-16T09:49:42.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.772+0000 7f54c80d4640 1 -- 172.21.15.84:0/874890775 >> 172.21.15.84:0/874890775 conn(0x7f54c00ce9b0 msgr2=0x7f54c00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:42.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.772+0000 7f54c80d4640 1 -- 172.21.15.84:0/874890775 shutdown_connections 2023-10-16T09:49:42.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.772+0000 7f54c80d4640 1 -- 172.21.15.84:0/874890775 wait complete. 2023-10-16T09:49:42.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.773+0000 7f54c80d4640 1 Processor -- start 2023-10-16T09:49:42.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.774+0000 7f54c80d4640 1 -- start start 2023-10-16T09:49:42.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.774+0000 7f54c80d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54ac004d00 0x7f54c0153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:42.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.774+0000 7f54c80d4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54c00aef90 0x7f54c006b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:42.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.774+0000 7f54c80d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c00af3f0 0x7f54c006b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:42.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.774+0000 7f54c80d4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c00aec80 con 0x7f54c00aef90 2023-10-16T09:49:42.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.774+0000 7f54c80d4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c015b450 con 0x7f54ac004d00 2023-10-16T09:49:42.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.774+0000 7f54c80d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c015b730 con 0x7f54c00af3f0 2023-10-16T09:49:42.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.775+0000 7f54c566f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54c00aef90 0x7f54c006b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:42.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.775+0000 7f54c566f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54c00aef90 0x7f54c006b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34828/0 (socket says 172.21.15.84:34828) 2023-10-16T09:49:42.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.775+0000 7f54c566f640 1 -- 172.21.15.84:0/897920207 learned_addr learned my addr 172.21.15.84:0/897920207 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:42.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.775+0000 7f54c5e70640 1 --2- 172.21.15.84:0/897920207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54ac004d00 0x7f54c0153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:42.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.775+0000 7f54c566f640 1 -- 172.21.15.84:0/897920207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c00af3f0 msgr2=0x7f54c006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:42.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.775+0000 7f54c566f640 1 --2- 172.21.15.84:0/897920207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c00af3f0 0x7f54c006b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.775+0000 7f54c566f640 1 -- 172.21.15.84:0/897920207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54ac004d00 msgr2=0x7f54c0153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:42.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.775+0000 7f54c566f640 1 --2- 172.21.15.84:0/897920207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54ac004d00 0x7f54c0153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.775+0000 7f54c566f640 1 -- 172.21.15.84:0/897920207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c015ba10 con 0x7f54c00aef90 2023-10-16T09:49:42.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.775+0000 7f54c566f640 1 --2- 172.21.15.84:0/897920207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54c00aef90 0x7f54c006b030 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f54b000c7a0 tx=0x7f54b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:42.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.775+0000 7f54b67fc640 1 -- 172.21.15.84:0/897920207 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b00074e0 con 0x7f54c00aef90 2023-10-16T09:49:42.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.776+0000 7f54b67fc640 1 -- 172.21.15.84:0/897920207 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b0007640 con 0x7f54c00aef90 2023-10-16T09:49:42.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.776+0000 7f54b67fc640 1 -- 172.21.15.84:0/897920207 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b0004a20 con 0x7f54c00aef90 2023-10-16T09:49:42.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.776+0000 7f54c80d4640 1 -- 172.21.15.84:0/897920207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c015bcf0 con 0x7f54c00aef90 2023-10-16T09:49:42.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.776+0000 7f54c80d4640 1 -- 172.21.15.84:0/897920207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c015c2b0 con 0x7f54c00aef90 2023-10-16T09:49:42.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.777+0000 7f54c80d4640 1 -- 172.21.15.84:0/897920207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c004e160 con 0x7f54c00aef90 2023-10-16T09:49:42.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.778+0000 7f54b67fc640 1 -- 172.21.15.84:0/897920207 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f54b0004c40 con 0x7f54c00aef90 2023-10-16T09:49:42.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.778+0000 7f54b67fc640 1 --2- 172.21.15.84:0/897920207 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f54a406ae70 0x7f54a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:42.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.779+0000 7f54b67fc640 1 -- 172.21.15.84:0/897920207 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54b0088b30 con 0x7f54c00aef90 2023-10-16T09:49:42.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.779+0000 7f54c5e70640 1 --2- 172.21.15.84:0/897920207 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f54a406ae70 0x7f54a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:42.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.779+0000 7f54c5e70640 1 --2- 172.21.15.84:0/897920207 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f54a406ae70 0x7f54a406d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f54bc006a90 tx=0x7f54bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:42.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.783+0000 7f54b67fc640 1 -- 172.21.15.84:0/897920207 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f54b0053be0 con 0x7f54c00aef90 2023-10-16T09:49:42.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.959+0000 7f54c80d4640 1 -- 172.21.15.84:0/897920207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f54c00c55a0 con 0x7f54c00aef90 2023-10-16T09:49:42.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.960+0000 7f54b67fc640 1 -- 172.21.15.84:0/897920207 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f54b0018070 con 0x7f54c00aef90 2023-10-16T09:49:42.962 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T09:49:42.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.963+0000 7f54c80d4640 1 -- 172.21.15.84:0/897920207 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f54a406ae70 msgr2=0x7f54a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:42.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.963+0000 7f54c80d4640 1 --2- 172.21.15.84:0/897920207 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f54a406ae70 0x7f54a406d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f54bc006a90 tx=0x7f54bc008040 comp rx=0 tx=0).stop 2023-10-16T09:49:42.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.963+0000 7f54c80d4640 1 -- 172.21.15.84:0/897920207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54c00aef90 msgr2=0x7f54c006b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:42.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.963+0000 7f54c80d4640 1 --2- 172.21.15.84:0/897920207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54c00aef90 0x7f54c006b030 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f54b000c7a0 tx=0x7f54b000cc60 comp rx=0 tx=0).stop 2023-10-16T09:49:42.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.963+0000 7f54c80d4640 1 -- 172.21.15.84:0/897920207 shutdown_connections 2023-10-16T09:49:42.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.963+0000 7f54c80d4640 1 --2- 172.21.15.84:0/897920207 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f54a406ae70 0x7f54a406d330 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.963+0000 7f54c80d4640 1 --2- 172.21.15.84:0/897920207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c00af3f0 0x7f54c006b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.963+0000 7f54c80d4640 1 --2- 172.21.15.84:0/897920207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54c00aef90 0x7f54c006b030 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.963+0000 7f54c80d4640 1 --2- 172.21.15.84:0/897920207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54ac004d00 0x7f54c0153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:42.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.963+0000 7f54c80d4640 1 -- 172.21.15.84:0/897920207 >> 172.21.15.84:0/897920207 conn(0x7f54c00ce9b0 msgr2=0x7f54c00cd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:42.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.963+0000 7f54c80d4640 1 -- 172.21.15.84:0/897920207 shutdown_connections 2023-10-16T09:49:42.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:42.963+0000 7f54c80d4640 1 -- 172.21.15.84:0/897920207 wait complete. 2023-10-16T09:49:42.983 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:49:43.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.195+0000 7fe96ffb0640 1 Processor -- start 2023-10-16T09:49:43.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.195+0000 7fe96ffb0640 1 -- start start 2023-10-16T09:49:43.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.197+0000 7fe96ffb0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe96806c850 0x7fe9680ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:43.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.197+0000 7fe96ffb0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe968070360 con 0x7fe968159e90 2023-10-16T09:49:43.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.197+0000 7fe96ffb0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9680cd1f0 con 0x7fe968159ab0 2023-10-16T09:49:43.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.197+0000 7fe96ffb0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96815d890 con 0x7fe96806c850 2023-10-16T09:49:43.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.197+0000 7fe96d54b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe968159e90 0x7fe96806c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53658/0 (socket says 172.21.15.84:53658) 2023-10-16T09:49:43.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.197+0000 7fe96d54b640 1 -- 172.21.15.84:0/147385091 learned_addr learned my addr 172.21.15.84:0/147385091 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:43.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.197+0000 7fe96e54d640 1 --2- 172.21.15.84:0/147385091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe96806c850 0x7fe9680ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:43.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.198+0000 7fe96cd4a640 1 -- 172.21.15.84:0/147385091 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3436106347 0 0) 0x7fe968070360 con 0x7fe968159e90 2023-10-16T09:49:43.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.198+0000 7fe96cd4a640 1 -- 172.21.15.84:0/147385091 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe954003520 con 0x7fe968159e90 2023-10-16T09:49:43.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.198+0000 7fe96cd4a640 1 -- 172.21.15.84:0/147385091 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4099718842 0 0) 0x7fe954003520 con 0x7fe968159e90 2023-10-16T09:49:43.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.198+0000 7fe96cd4a640 1 -- 172.21.15.84:0/147385091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe96806c850 msgr2=0x7fe9680ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:43.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.198+0000 7fe96cd4a640 1 --2- 172.21.15.84:0/147385091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe96806c850 0x7fe9680ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.198+0000 7fe96cd4a640 1 -- 172.21.15.84:0/147385091 >> v1:172.21.15.157:6789/0 conn(0x7fe968159ab0 legacy=0x7fe96806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:43.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.198+0000 7fe96cd4a640 1 -- 172.21.15.84:0/147385091 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9680cd4d0 con 0x7fe968159e90 2023-10-16T09:49:43.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.198+0000 7fe96cd4a640 1 -- 172.21.15.84:0/147385091 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fe958001670 con 0x7fe968159e90 2023-10-16T09:49:43.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.198+0000 7fe96cd4a640 1 -- 172.21.15.84:0/147385091 >> v1:172.21.15.84:6789/0 conn(0x7fe968159e90 legacy=0x7fe96806c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:43.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.198+0000 7fe96cd4a640 1 --2- 172.21.15.84:0/147385091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9540039a0 0x7fe954003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:43.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.198+0000 7fe96cd4a640 1 --2- 172.21.15.84:0/147385091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9540047c0 0x7fe954004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:43.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.198+0000 7fe96cd4a640 1 --2- 172.21.15.84:0/147385091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe96806c850 0x7fe9540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:43.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.198+0000 7fe96cd4a640 1 -- 172.21.15.84:0/147385091 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96815d890 con 0x7fe9540039a0 2023-10-16T09:49:43.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.198+0000 7fe96cd4a640 1 -- 172.21.15.84:0/147385091 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95400bae0 con 0x7fe9540047c0 2023-10-16T09:49:43.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.198+0000 7fe96cd4a640 1 -- 172.21.15.84:0/147385091 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95400bca0 con 0x7fe96806c850 2023-10-16T09:49:43.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.198+0000 7fe96dd4c640 1 --2- 172.21.15.84:0/147385091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9540039a0 0x7fe954003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:43.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.199+0000 7fe96d54b640 1 --2- 172.21.15.84:0/147385091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9540047c0 0x7fe954004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:43.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.199+0000 7fe96dd4c640 1 -- 172.21.15.84:0/147385091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe96806c850 msgr2=0x7fe9540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:43.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.199+0000 7fe96dd4c640 1 --2- 172.21.15.84:0/147385091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe96806c850 0x7fe9540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.199+0000 7fe96dd4c640 1 -- 172.21.15.84:0/147385091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9540047c0 msgr2=0x7fe954004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:43.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.199+0000 7fe96dd4c640 1 --2- 172.21.15.84:0/147385091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9540047c0 0x7fe954004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.199+0000 7fe96dd4c640 1 -- 172.21.15.84:0/147385091 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe954003520 con 0x7fe9540039a0 2023-10-16T09:49:43.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.200+0000 7fe96dd4c640 1 --2- 172.21.15.84:0/147385091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9540039a0 0x7fe954003d80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe964001860 tx=0x7fe9540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c66d96cf6fe515a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:43.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.200+0000 7fe96cd4a640 1 -- 172.21.15.84:0/147385091 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964033630 con 0x7fe9540039a0 2023-10-16T09:49:43.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.200+0000 7fe96cd4a640 1 -- 172.21.15.84:0/147385091 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9640337b0 con 0x7fe9540039a0 2023-10-16T09:49:43.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.200+0000 7fe96cd4a640 1 -- 172.21.15.84:0/147385091 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964033b60 con 0x7fe9540039a0 2023-10-16T09:49:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.201+0000 7fe96ffb0640 1 -- 172.21.15.84:0/147385091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9540039a0 msgr2=0x7fe954003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.201+0000 7fe96ffb0640 1 --2- 172.21.15.84:0/147385091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9540039a0 0x7fe954003d80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe964001860 tx=0x7fe9540057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.201+0000 7fe96e54d640 1 -- 172.21.15.84:0/147385091 reap_dead start 2023-10-16T09:49:43.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.201+0000 7fe96ffb0640 1 -- 172.21.15.84:0/147385091 shutdown_connections 2023-10-16T09:49:43.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.201+0000 7fe96ffb0640 1 -- 172.21.15.84:0/147385091 >> 172.21.15.84:0/147385091 conn(0x7fe9680ce9b0 msgr2=0x7fe9680cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:43.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.201+0000 7fe96ffb0640 1 -- 172.21.15.84:0/147385091 shutdown_connections 2023-10-16T09:49:43.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.201+0000 7fe96ffb0640 1 -- 172.21.15.84:0/147385091 wait complete. 2023-10-16T09:49:43.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.203+0000 7fe96ffb0640 1 Processor -- start 2023-10-16T09:49:43.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.203+0000 7fe96ffb0640 1 -- start start 2023-10-16T09:49:43.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.204+0000 7fe96ffb0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9540039a0 0x7fe9681e2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:43.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.204+0000 7fe96ffb0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe968157f60 0x7fe96815c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:43.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.204+0000 7fe96ffb0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9681583e0 0x7fe96815c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:43.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.204+0000 7fe96ffb0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe968157ad0 con 0x7fe9540039a0 2023-10-16T09:49:43.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.204+0000 7fe96ffb0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9681e34d0 con 0x7fe968157f60 2023-10-16T09:49:43.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.204+0000 7fe96ffb0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9681e5990 con 0x7fe9681583e0 2023-10-16T09:49:43.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.204+0000 7fe96dd4c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9540039a0 0x7fe9681e2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:43.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.204+0000 7fe96dd4c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9540039a0 0x7fe9681e2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34844/0 (socket says 172.21.15.84:34844) 2023-10-16T09:49:43.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.204+0000 7fe96dd4c640 1 -- 172.21.15.84:0/845620402 learned_addr learned my addr 172.21.15.84:0/845620402 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:43.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.204+0000 7fe96dd4c640 1 -- 172.21.15.84:0/845620402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9681583e0 msgr2=0x7fe96815c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:43.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.204+0000 7fe96dd4c640 1 --2- 172.21.15.84:0/845620402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9681583e0 0x7fe96815c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.205+0000 7fe96dd4c640 1 -- 172.21.15.84:0/845620402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe968157f60 msgr2=0x7fe96815c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:43.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.205+0000 7fe96dd4c640 1 --2- 172.21.15.84:0/845620402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe968157f60 0x7fe96815c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.205+0000 7fe96dd4c640 1 -- 172.21.15.84:0/845620402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9681571a0 con 0x7fe9540039a0 2023-10-16T09:49:43.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.205+0000 7fe96dd4c640 1 --2- 172.21.15.84:0/845620402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9540039a0 0x7fe9681e2f90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe964008fd0 tx=0x7fe964001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:43.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.205+0000 7fe95e7fc640 1 -- 172.21.15.84:0/845620402 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9640035f0 con 0x7fe9540039a0 2023-10-16T09:49:43.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.205+0000 7fe95e7fc640 1 -- 172.21.15.84:0/845620402 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe964003750 con 0x7fe9540039a0 2023-10-16T09:49:43.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.205+0000 7fe95e7fc640 1 -- 172.21.15.84:0/845620402 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964003ae0 con 0x7fe9540039a0 2023-10-16T09:49:43.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.205+0000 7fe96ffb0640 1 -- 172.21.15.84:0/845620402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe968157480 con 0x7fe9540039a0 2023-10-16T09:49:43.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.205+0000 7fe96ffb0640 1 -- 172.21.15.84:0/845620402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe968144a30 con 0x7fe9540039a0 2023-10-16T09:49:43.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.206+0000 7fe96ffb0640 1 -- 172.21.15.84:0/845620402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96804e160 con 0x7fe9540039a0 2023-10-16T09:49:43.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.207+0000 7fe95e7fc640 1 -- 172.21.15.84:0/845620402 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe964003c40 con 0x7fe9540039a0 2023-10-16T09:49:43.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.208+0000 7fe95e7fc640 1 --2- 172.21.15.84:0/845620402 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe94c06aea0 0x7fe94c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:43.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.208+0000 7fe95e7fc640 1 -- 172.21.15.84:0/845620402 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9640ab950 con 0x7fe9540039a0 2023-10-16T09:49:43.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.208+0000 7fe96d54b640 1 --2- 172.21.15.84:0/845620402 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe94c06aea0 0x7fe94c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:43.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.209+0000 7fe96d54b640 1 --2- 172.21.15.84:0/845620402 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe94c06aea0 0x7fe94c06d360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe958001200 tx=0x7fe958009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:43.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.213+0000 7fe95e7fc640 1 -- 172.21.15.84:0/845620402 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe964076b10 con 0x7fe9540039a0 2023-10-16T09:49:43.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.402+0000 7fe96ffb0640 1 -- 172.21.15.84:0/845620402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9680ccb60 con 0x7fe9540039a0 2023-10-16T09:49:43.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.403+0000 7fe95e7fc640 1 -- 172.21.15.84:0/845620402 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe96404ee80 con 0x7fe9540039a0 2023-10-16T09:49:43.405 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T09:49:43.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.405+0000 7fe96ffb0640 1 -- 172.21.15.84:0/845620402 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe94c06aea0 msgr2=0x7fe94c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:43.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.405+0000 7fe96ffb0640 1 --2- 172.21.15.84:0/845620402 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe94c06aea0 0x7fe94c06d360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe958001200 tx=0x7fe958009040 comp rx=0 tx=0).stop 2023-10-16T09:49:43.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.405+0000 7fe96ffb0640 1 -- 172.21.15.84:0/845620402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9540039a0 msgr2=0x7fe9681e2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:43.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.405+0000 7fe96ffb0640 1 --2- 172.21.15.84:0/845620402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9540039a0 0x7fe9681e2f90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe964008fd0 tx=0x7fe964001230 comp rx=0 tx=0).stop 2023-10-16T09:49:43.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.405+0000 7fe96ffb0640 1 -- 172.21.15.84:0/845620402 shutdown_connections 2023-10-16T09:49:43.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.405+0000 7fe96ffb0640 1 --2- 172.21.15.84:0/845620402 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe94c06aea0 0x7fe94c06d360 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.405+0000 7fe96ffb0640 1 --2- 172.21.15.84:0/845620402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9681583e0 0x7fe96815c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.405+0000 7fe96ffb0640 1 --2- 172.21.15.84:0/845620402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe968157f60 0x7fe96815c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.405+0000 7fe96ffb0640 1 --2- 172.21.15.84:0/845620402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9540039a0 0x7fe9681e2f90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.405+0000 7fe96ffb0640 1 -- 172.21.15.84:0/845620402 >> 172.21.15.84:0/845620402 conn(0x7fe9680ce9b0 msgr2=0x7fe968158f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:43.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.406+0000 7fe96ffb0640 1 -- 172.21.15.84:0/845620402 shutdown_connections 2023-10-16T09:49:43.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.406+0000 7fe96ffb0640 1 -- 172.21.15.84:0/845620402 wait complete. 2023-10-16T09:49:43.425 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:49:43.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.631+0000 7fe7ff341640 1 Processor -- start 2023-10-16T09:49:43.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.631+0000 7fe7ff341640 1 -- start start 2023-10-16T09:49:43.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.633+0000 7fe7ff341640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ad70 0x7fe7f8069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:43.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.633+0000 7fe7ff341640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe7f80caac0 0x7fe7f806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:43.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.633+0000 7fe7ff341640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f80cb030 con 0x7fe7f80c9ca0 2023-10-16T09:49:43.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.633+0000 7fe7ff341640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f80c91c0 con 0x7fe7f80caac0 2023-10-16T09:49:43.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.633+0000 7fe7ff341640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f80c94a0 con 0x7fe7f815ad70 2023-10-16T09:49:43.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.633+0000 7fe7fc8dc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe7f80c9ca0 0x7fe7f80ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53670/0 (socket says 172.21.15.84:53670) 2023-10-16T09:49:43.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.633+0000 7fe7fc8dc640 1 -- 172.21.15.84:0/839918278 learned_addr learned my addr 172.21.15.84:0/839918278 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:43.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.633+0000 7fe7fd8de640 1 --2- 172.21.15.84:0/839918278 >> v2:172.21.15.84:3300/0 conn(0x7fe7f80caac0 0x7fe7f806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:43.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.633+0000 7fe7fd0dd640 1 --2- 172.21.15.84:0/839918278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ad70 0x7fe7f8069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:43.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.633+0000 7fe7fd8de640 1 -- 172.21.15.84:0/839918278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ad70 msgr2=0x7fe7f8069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:43.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.633+0000 7fe7fd8de640 1 --2- 172.21.15.84:0/839918278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ad70 0x7fe7f8069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.633+0000 7fe7effff640 1 -- 172.21.15.84:0/839918278 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2354543018 0 0) 0x7fe7f80cb030 con 0x7fe7f80c9ca0 2023-10-16T09:49:43.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.634+0000 7fe7fd8de640 1 -- 172.21.15.84:0/839918278 >> v1:172.21.15.84:6789/0 conn(0x7fe7f80c9ca0 legacy=0x7fe7f80ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:43.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.634+0000 7fe7fd8de640 1 -- 172.21.15.84:0/839918278 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f80c9780 con 0x7fe7f80caac0 2023-10-16T09:49:43.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.635+0000 7fe7fd8de640 1 --2- 172.21.15.84:0/839918278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80caac0 0x7fe7f806a2e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009d90 tx=0x7fe7f002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9848ddf54ede120 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:43.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.635+0000 7fe7effff640 1 -- 172.21.15.84:0/839918278 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f002cc60 con 0x7fe7f80caac0 2023-10-16T09:49:43.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.635+0000 7fe7effff640 1 -- 172.21.15.84:0/839918278 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f002cdc0 con 0x7fe7f80caac0 2023-10-16T09:49:43.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.635+0000 7fe7effff640 1 -- 172.21.15.84:0/839918278 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0034660 con 0x7fe7f80caac0 2023-10-16T09:49:43.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.635+0000 7fe7ff341640 1 -- 172.21.15.84:0/839918278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80caac0 msgr2=0x7fe7f806a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:43.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.635+0000 7fe7ff341640 1 --2- 172.21.15.84:0/839918278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80caac0 0x7fe7f806a2e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009d90 tx=0x7fe7f002beb0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.636+0000 7fe7ff341640 1 -- 172.21.15.84:0/839918278 shutdown_connections 2023-10-16T09:49:43.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.636+0000 7fe7ff341640 1 --2- 172.21.15.84:0/839918278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80caac0 0x7fe7f806a2e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.636+0000 7fe7ff341640 1 --2- 172.21.15.84:0/839918278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ad70 0x7fe7f8069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.636+0000 7fe7ff341640 1 -- 172.21.15.84:0/839918278 >> 172.21.15.84:0/839918278 conn(0x7fe7f80ce9b0 msgr2=0x7fe7f80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:43.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.636+0000 7fe7ff341640 1 -- 172.21.15.84:0/839918278 shutdown_connections 2023-10-16T09:49:43.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.636+0000 7fe7ff341640 1 -- 172.21.15.84:0/839918278 wait complete. 2023-10-16T09:49:43.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.638+0000 7fe7ff341640 1 Processor -- start 2023-10-16T09:49:43.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.638+0000 7fe7ff341640 1 -- start start 2023-10-16T09:49:43.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.639+0000 7fe7ff341640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f80c9ca0 0x7fe7f8157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:43.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.639+0000 7fe7ff341640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80caac0 0x7fe7f81f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:43.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.639+0000 7fe7ff341640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ad70 0x7fe7f81f3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:43.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.639+0000 7fe7ff341640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f80c8a00 con 0x7fe7f80caac0 2023-10-16T09:49:43.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.639+0000 7fe7ff341640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f80c81f0 con 0x7fe7f80c9ca0 2023-10-16T09:49:43.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.639+0000 7fe7ff341640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f80a86b0 con 0x7fe7f815ad70 2023-10-16T09:49:43.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.639+0000 7fe7fc8dc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80caac0 0x7fe7f81f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:43.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.639+0000 7fe7fc8dc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80caac0 0x7fe7f81f3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34866/0 (socket says 172.21.15.84:34866) 2023-10-16T09:49:43.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.639+0000 7fe7fc8dc640 1 -- 172.21.15.84:0/2765230691 learned_addr learned my addr 172.21.15.84:0/2765230691 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:43.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.640+0000 7fe7fd0dd640 1 --2- 172.21.15.84:0/2765230691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f80c9ca0 0x7fe7f8157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:43.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.640+0000 7fe7fc8dc640 1 -- 172.21.15.84:0/2765230691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ad70 msgr2=0x7fe7f81f3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:43.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.640+0000 7fe7fc8dc640 1 --2- 172.21.15.84:0/2765230691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ad70 0x7fe7f81f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.640+0000 7fe7fc8dc640 1 -- 172.21.15.84:0/2765230691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f80c9ca0 msgr2=0x7fe7f8157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:43.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.640+0000 7fe7fc8dc640 1 --2- 172.21.15.84:0/2765230691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f80c9ca0 0x7fe7f8157fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.640+0000 7fe7fc8dc640 1 -- 172.21.15.84:0/2765230691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f0009a70 con 0x7fe7f80caac0 2023-10-16T09:49:43.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.640+0000 7fe7fc8dc640 1 --2- 172.21.15.84:0/2765230691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80caac0 0x7fe7f81f3450 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800c500 tx=0x7fe7e800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:43.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.640+0000 7fe7edffb640 1 -- 172.21.15.84:0/2765230691 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8007560 con 0x7fe7f80caac0 2023-10-16T09:49:43.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.640+0000 7fe7edffb640 1 -- 172.21.15.84:0/2765230691 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e80047e0 con 0x7fe7f80caac0 2023-10-16T09:49:43.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.640+0000 7fe7edffb640 1 -- 172.21.15.84:0/2765230691 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8004b90 con 0x7fe7f80caac0 2023-10-16T09:49:43.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.640+0000 7fe7ff341640 1 -- 172.21.15.84:0/2765230691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f8158520 con 0x7fe7f80caac0 2023-10-16T09:49:43.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.640+0000 7fe7ff341640 1 -- 172.21.15.84:0/2765230691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f80c8720 con 0x7fe7f80caac0 2023-10-16T09:49:43.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.641+0000 7fe7ff341640 1 -- 172.21.15.84:0/2765230691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f80a8480 con 0x7fe7f80caac0 2023-10-16T09:49:43.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.642+0000 7fe7edffb640 1 -- 172.21.15.84:0/2765230691 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe7e8004cf0 con 0x7fe7f80caac0 2023-10-16T09:49:43.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.643+0000 7fe7edffb640 1 --2- 172.21.15.84:0/2765230691 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe7dc06ae70 0x7fe7dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:43.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.643+0000 7fe7edffb640 1 -- 172.21.15.84:0/2765230691 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7e8088800 con 0x7fe7f80caac0 2023-10-16T09:49:43.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.644+0000 7fe7fd0dd640 1 --2- 172.21.15.84:0/2765230691 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe7dc06ae70 0x7fe7dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:43.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.644+0000 7fe7fd0dd640 1 --2- 172.21.15.84:0/2765230691 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe7dc06ae70 0x7fe7dc06d330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4006fd0 tx=0x7fe7f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:43.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.647+0000 7fe7edffb640 1 -- 172.21.15.84:0/2765230691 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe7f80a8480 con 0x7fe7f80caac0 2023-10-16T09:49:43.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.839+0000 7fe7ff341640 1 -- 172.21.15.84:0/2765230691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe7f80a8480 con 0x7fe7f80caac0 2023-10-16T09:49:43.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.840+0000 7fe7edffb640 1 -- 172.21.15.84:0/2765230691 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe7e80180a0 con 0x7fe7f80caac0 2023-10-16T09:49:43.842 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T09:49:43.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.842+0000 7fe7ff341640 1 -- 172.21.15.84:0/2765230691 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe7dc06ae70 msgr2=0x7fe7dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:43.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.842+0000 7fe7ff341640 1 --2- 172.21.15.84:0/2765230691 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe7dc06ae70 0x7fe7dc06d330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4006fd0 tx=0x7fe7f4008040 comp rx=0 tx=0).stop 2023-10-16T09:49:43.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.843+0000 7fe7ff341640 1 -- 172.21.15.84:0/2765230691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80caac0 msgr2=0x7fe7f81f3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:43.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.843+0000 7fe7ff341640 1 --2- 172.21.15.84:0/2765230691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80caac0 0x7fe7f81f3450 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800c500 tx=0x7fe7e800c9c0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.843+0000 7fe7ff341640 1 -- 172.21.15.84:0/2765230691 shutdown_connections 2023-10-16T09:49:43.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.843+0000 7fe7ff341640 1 --2- 172.21.15.84:0/2765230691 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe7dc06ae70 0x7fe7dc06d330 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.843+0000 7fe7ff341640 1 --2- 172.21.15.84:0/2765230691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ad70 0x7fe7f81f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.843+0000 7fe7ff341640 1 --2- 172.21.15.84:0/2765230691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80caac0 0x7fe7f81f3450 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.843+0000 7fe7ff341640 1 --2- 172.21.15.84:0/2765230691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f80c9ca0 0x7fe7f8157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:43.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.843+0000 7fe7ff341640 1 -- 172.21.15.84:0/2765230691 >> 172.21.15.84:0/2765230691 conn(0x7fe7f80ce9b0 msgr2=0x7fe7f8159d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:43.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.843+0000 7fe7ff341640 1 -- 172.21.15.84:0/2765230691 shutdown_connections 2023-10-16T09:49:43.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:43.843+0000 7fe7ff341640 1 -- 172.21.15.84:0/2765230691 wait complete. 2023-10-16T09:49:43.863 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-10-16T09:49:44.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.068+0000 7fd574f43640 1 Processor -- start 2023-10-16T09:49:44.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.068+0000 7fd574f43640 1 -- start start 2023-10-16T09:49:44.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.069+0000 7fd574f43640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd57015a9c0 0x7fd570160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:44.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.069+0000 7fd574f43640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd5700cf850 0x7fd570071c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:44.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.069+0000 7fd574f43640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd57006cef0 con 0x7fd5700721a0 2023-10-16T09:49:44.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.069+0000 7fd574f43640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd57015ada0 con 0x7fd5700cf850 2023-10-16T09:49:44.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.069+0000 7fd574f43640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57015d350 con 0x7fd57015a9c0 2023-10-16T09:49:44.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.070+0000 7fd56f7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd57015a9c0 0x7fd570160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:44.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.070+0000 7fd56effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd5700cf850 0x7fd570071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:44.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.070+0000 7fd56effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd5700cf850 0x7fd570071c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:60084/0 (socket says 172.21.15.84:60084) 2023-10-16T09:49:44.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.070+0000 7fd56effd640 1 -- 172.21.15.84:0/829206012 learned_addr learned my addr 172.21.15.84:0/829206012 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:44.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.071+0000 7fd56f7fe640 1 -- 172.21.15.84:0/829206012 >> v2:172.21.15.157:3300/0 conn(0x7fd5700cf850 msgr2=0x7fd570071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:44.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.071+0000 7fd56f7fe640 1 --2- 172.21.15.84:0/829206012 >> v2:172.21.15.157:3300/0 conn(0x7fd5700cf850 0x7fd570071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:44.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.071+0000 7fd56f7fe640 1 -- 172.21.15.84:0/829206012 >> v1:172.21.15.157:6789/0 conn(0x7fd5700721a0 legacy=0x7fd5700b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:44.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.071+0000 7fd56f7fe640 1 -- 172.21.15.84:0/829206012 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5700cfcf0 con 0x7fd57015a9c0 2023-10-16T09:49:44.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.073+0000 7fd56f7fe640 1 --2- 172.21.15.84:0/829206012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd57015a9c0 0x7fd570160940 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd560009d90 tx=0x7fd56002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23b9cc95e7664ba8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:44.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.073+0000 7fd56e7fc640 1 -- 172.21.15.84:0/829206012 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56002cbc0 con 0x7fd57015a9c0 2023-10-16T09:49:44.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.073+0000 7fd56e7fc640 1 -- 172.21.15.84:0/829206012 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56002cd20 con 0x7fd57015a9c0 2023-10-16T09:49:44.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.073+0000 7fd56e7fc640 1 -- 172.21.15.84:0/829206012 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560034660 con 0x7fd57015a9c0 2023-10-16T09:49:44.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.074+0000 7fd574f43640 1 -- 172.21.15.84:0/829206012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd57015a9c0 msgr2=0x7fd570160940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:44.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.074+0000 7fd574f43640 1 --2- 172.21.15.84:0/829206012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd57015a9c0 0x7fd570160940 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd560009d90 tx=0x7fd56002be10 comp rx=0 tx=0).stop 2023-10-16T09:49:44.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.074+0000 7fd574f43640 1 -- 172.21.15.84:0/829206012 shutdown_connections 2023-10-16T09:49:44.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.074+0000 7fd574f43640 1 --2- 172.21.15.84:0/829206012 >> v2:172.21.15.157:3300/0 conn(0x7fd5700cf850 0x7fd570071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:44.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.074+0000 7fd574f43640 1 --2- 172.21.15.84:0/829206012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd57015a9c0 0x7fd570160940 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:44.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.074+0000 7fd574f43640 1 -- 172.21.15.84:0/829206012 >> 172.21.15.84:0/829206012 conn(0x7fd5700cd370 msgr2=0x7fd5700cd750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:44.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.074+0000 7fd574f43640 1 -- 172.21.15.84:0/829206012 shutdown_connections 2023-10-16T09:49:44.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.074+0000 7fd574f43640 1 -- 172.21.15.84:0/829206012 wait complete. 2023-10-16T09:49:44.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.076+0000 7fd574f43640 1 Processor -- start 2023-10-16T09:49:44.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.076+0000 7fd574f43640 1 -- start start 2023-10-16T09:49:44.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.077+0000 7fd574f43640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5700721a0 0x7fd5700b6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:44.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.077+0000 7fd574f43640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5700cf850 0x7fd570153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:44.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.077+0000 7fd574f43640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd57015a9c0 0x7fd5701e8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:44.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.077+0000 7fd574f43640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5701e9620 con 0x7fd5700cf850 2023-10-16T09:49:44.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.077+0000 7fd574f43640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700b6570 con 0x7fd57015a9c0 2023-10-16T09:49:44.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.077+0000 7fd574f43640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5701e6e20 con 0x7fd5700721a0 2023-10-16T09:49:44.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.077+0000 7fd56effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5700cf850 0x7fd570153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:44.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.077+0000 7fd56effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5700cf850 0x7fd570153350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34880/0 (socket says 172.21.15.84:34880) 2023-10-16T09:49:44.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.077+0000 7fd56effd640 1 -- 172.21.15.84:0/86864434 learned_addr learned my addr 172.21.15.84:0/86864434 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:44.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.077+0000 7fd56ffff640 1 --2- 172.21.15.84:0/86864434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd57015a9c0 0x7fd5701e8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:44.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.077+0000 7fd56f7fe640 1 --2- 172.21.15.84:0/86864434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5700721a0 0x7fd5700b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:44.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.078+0000 7fd56effd640 1 -- 172.21.15.84:0/86864434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5700721a0 msgr2=0x7fd5700b6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:44.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.078+0000 7fd56effd640 1 --2- 172.21.15.84:0/86864434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5700721a0 0x7fd5700b6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:44.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.078+0000 7fd56effd640 1 -- 172.21.15.84:0/86864434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd57015a9c0 msgr2=0x7fd5701e8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:44.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.078+0000 7fd56effd640 1 --2- 172.21.15.84:0/86864434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd57015a9c0 0x7fd5701e8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:44.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.078+0000 7fd56effd640 1 -- 172.21.15.84:0/86864434 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd560009a70 con 0x7fd5700cf850 2023-10-16T09:49:44.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.078+0000 7fd56f7fe640 1 --2- 172.21.15.84:0/86864434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5700721a0 0x7fd5700b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:44.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.078+0000 7fd56effd640 1 --2- 172.21.15.84:0/86864434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5700cf850 0x7fd570153350 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd55800e840 tx=0x7fd55800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:44.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.078+0000 7fd567fff640 1 -- 172.21.15.84:0/86864434 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55800c8e0 con 0x7fd5700cf850 2023-10-16T09:49:44.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.078+0000 7fd567fff640 1 -- 172.21.15.84:0/86864434 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55800ca40 con 0x7fd5700cf850 2023-10-16T09:49:44.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.078+0000 7fd567fff640 1 -- 172.21.15.84:0/86864434 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55800cdf0 con 0x7fd5700cf850 2023-10-16T09:49:44.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.078+0000 7fd574f43640 1 -- 172.21.15.84:0/86864434 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5701e6b40 con 0x7fd5700cf850 2023-10-16T09:49:44.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.078+0000 7fd574f43640 1 -- 172.21.15.84:0/86864434 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5701e9340 con 0x7fd5700cf850 2023-10-16T09:49:44.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.080+0000 7fd574f43640 1 -- 172.21.15.84:0/86864434 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57004e9b0 con 0x7fd5700cf850 2023-10-16T09:49:44.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.080+0000 7fd567fff640 1 -- 172.21.15.84:0/86864434 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd558013470 con 0x7fd5700cf850 2023-10-16T09:49:44.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.081+0000 7fd567fff640 1 --2- 172.21.15.84:0/86864434 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd55006ae70 0x7fd55006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:44.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.081+0000 7fd567fff640 1 -- 172.21.15.84:0/86864434 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd558089650 con 0x7fd5700cf850 2023-10-16T09:49:44.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.082+0000 7fd56f7fe640 1 --2- 172.21.15.84:0/86864434 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd55006ae70 0x7fd55006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:44.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.082+0000 7fd56f7fe640 1 --2- 172.21.15.84:0/86864434 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd55006ae70 0x7fd55006d330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd560036000 tx=0x7fd560030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:44.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.086+0000 7fd567fff640 1 -- 172.21.15.84:0/86864434 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd558054790 con 0x7fd5700cf850 2023-10-16T09:49:44.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.350+0000 7fd574f43640 1 -- 172.21.15.84:0/86864434 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7fd5700b66b0 con 0x7fd5700cf850 2023-10-16T09:49:44.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.728+0000 7fd567fff640 1 -- 172.21.15.84:0/86864434 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v57) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd558057db0 con 0x7fd5700cf850 2023-10-16T09:49:44.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.731+0000 7fd574f43640 1 -- 172.21.15.84:0/86864434 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd55006ae70 msgr2=0x7fd55006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:44.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.731+0000 7fd574f43640 1 --2- 172.21.15.84:0/86864434 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd55006ae70 0x7fd55006d330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd560036000 tx=0x7fd560030000 comp rx=0 tx=0).stop 2023-10-16T09:49:44.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.731+0000 7fd574f43640 1 -- 172.21.15.84:0/86864434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5700cf850 msgr2=0x7fd570153350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:44.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.731+0000 7fd574f43640 1 --2- 172.21.15.84:0/86864434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5700cf850 0x7fd570153350 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd55800e840 tx=0x7fd55800ed00 comp rx=0 tx=0).stop 2023-10-16T09:49:44.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.731+0000 7fd574f43640 1 -- 172.21.15.84:0/86864434 shutdown_connections 2023-10-16T09:49:44.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.731+0000 7fd574f43640 1 --2- 172.21.15.84:0/86864434 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd55006ae70 0x7fd55006d330 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:44.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.731+0000 7fd574f43640 1 --2- 172.21.15.84:0/86864434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd57015a9c0 0x7fd5701e8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:44.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.731+0000 7fd574f43640 1 --2- 172.21.15.84:0/86864434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5700cf850 0x7fd570153350 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:44.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.731+0000 7fd574f43640 1 --2- 172.21.15.84:0/86864434 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5700721a0 0x7fd5700b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:44.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.731+0000 7fd574f43640 1 -- 172.21.15.84:0/86864434 >> 172.21.15.84:0/86864434 conn(0x7fd5700cd370 msgr2=0x7fd5700cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:44.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.732+0000 7fd574f43640 1 -- 172.21.15.84:0/86864434 shutdown_connections 2023-10-16T09:49:44.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.732+0000 7fd574f43640 1 -- 172.21.15.84:0/86864434 wait complete. 2023-10-16T09:49:44.753 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T09:49:44.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.968+0000 7fd8ae95a640 1 Processor -- start 2023-10-16T09:49:44.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.968+0000 7fd8ae95a640 1 -- start start 2023-10-16T09:49:44.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.970+0000 7fd8ae95a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd8a80caa80 0x7fd8a8069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:44.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.970+0000 7fd8ae95a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a806a2e0 0x7fd8a81650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:44.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.970+0000 7fd8ae95a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a806a730 con 0x7fd8a80addb0 2023-10-16T09:49:44.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.970+0000 7fd8ae95a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8a8161a50 con 0x7fd8a80caa80 2023-10-16T09:49:44.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.970+0000 7fd8ae95a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80caef0 con 0x7fd8a806a2e0 2023-10-16T09:49:44.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.970+0000 7fd89ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd8a80caa80 0x7fd8a8069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:44.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.970+0000 7fd89ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd8a80caa80 0x7fd8a8069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34886/0 (socket says 172.21.15.84:34886) 2023-10-16T09:49:44.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.971+0000 7fd89ffff640 1 -- 172.21.15.84:0/3585198005 learned_addr learned my addr 172.21.15.84:0/3585198005 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:44.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.971+0000 7fd8acef7640 1 --2- 172.21.15.84:0/3585198005 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a806a2e0 0x7fd8a81650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:44.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.971+0000 7fd89ffff640 1 -- 172.21.15.84:0/3585198005 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a806a2e0 msgr2=0x7fd8a81650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:44.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.971+0000 7fd89ffff640 1 --2- 172.21.15.84:0/3585198005 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a806a2e0 0x7fd8a81650d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:44.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.971+0000 7fd89ffff640 1 -- 172.21.15.84:0/3585198005 >> v1:172.21.15.157:6789/0 conn(0x7fd8a80addb0 legacy=0x7fd8a80c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:44.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.971+0000 7fd89ffff640 1 -- 172.21.15.84:0/3585198005 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a80cb140 con 0x7fd8a80caa80 2023-10-16T09:49:44.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.972+0000 7fd89ffff640 1 --2- 172.21.15.84:0/3585198005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a80caa80 0x7fd8a8069da0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd890009870 tx=0x7fd89002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33044942f0bb2dff server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:44.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.973+0000 7fd8a77fe640 1 -- 172.21.15.84:0/3585198005 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd890030840 con 0x7fd8a80caa80 2023-10-16T09:49:44.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.973+0000 7fd8a77fe640 1 -- 172.21.15.84:0/3585198005 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8900309a0 con 0x7fd8a80caa80 2023-10-16T09:49:44.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.973+0000 7fd8a77fe640 1 -- 172.21.15.84:0/3585198005 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd890030d30 con 0x7fd8a80caa80 2023-10-16T09:49:44.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.973+0000 7fd8ae95a640 1 -- 172.21.15.84:0/3585198005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a80caa80 msgr2=0x7fd8a8069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:44.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.973+0000 7fd8ae95a640 1 --2- 172.21.15.84:0/3585198005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a80caa80 0x7fd8a8069da0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd890009870 tx=0x7fd89002b980 comp rx=0 tx=0).stop 2023-10-16T09:49:44.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.973+0000 7fd8ae95a640 1 -- 172.21.15.84:0/3585198005 shutdown_connections 2023-10-16T09:49:44.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.973+0000 7fd8ae95a640 1 --2- 172.21.15.84:0/3585198005 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a806a2e0 0x7fd8a81650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:44.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.973+0000 7fd8ae95a640 1 --2- 172.21.15.84:0/3585198005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a80caa80 0x7fd8a8069da0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:44.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.973+0000 7fd8ae95a640 1 -- 172.21.15.84:0/3585198005 >> 172.21.15.84:0/3585198005 conn(0x7fd8a80ce9b0 msgr2=0x7fd8a80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:44.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.973+0000 7fd8ae95a640 1 -- 172.21.15.84:0/3585198005 shutdown_connections 2023-10-16T09:49:44.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.973+0000 7fd8ae95a640 1 -- 172.21.15.84:0/3585198005 wait complete. 2023-10-16T09:49:44.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.975+0000 7fd8ae95a640 1 Processor -- start 2023-10-16T09:49:44.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.975+0000 7fd8ae95a640 1 -- start start 2023-10-16T09:49:44.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.976+0000 7fd8ae95a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a806a2e0 0x7fd8a815b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:44.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.976+0000 7fd8ae95a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a80addb0 0x7fd8a80b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:44.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.976+0000 7fd8ae95a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80caa80 0x7fd8a80b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:44.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.976+0000 7fd8ae95a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a815eb60 con 0x7fd8a806a2e0 2023-10-16T09:49:44.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.976+0000 7fd8ae95a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a815dfe0 con 0x7fd8a80addb0 2023-10-16T09:49:44.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.976+0000 7fd8ae95a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a815e2c0 con 0x7fd8a80caa80 2023-10-16T09:49:44.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.976+0000 7fd8a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a806a2e0 0x7fd8a815b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:44.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.976+0000 7fd8a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a806a2e0 0x7fd8a815b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34898/0 (socket says 172.21.15.84:34898) 2023-10-16T09:49:44.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.976+0000 7fd8a7fff640 1 -- 172.21.15.84:0/3508454591 learned_addr learned my addr 172.21.15.84:0/3508454591 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:44.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.977+0000 7fd8acef7640 1 --2- 172.21.15.84:0/3508454591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80caa80 0x7fd8a80b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:44.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.977+0000 7fd89ffff640 1 --2- 172.21.15.84:0/3508454591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a80addb0 0x7fd8a80b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:44.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.977+0000 7fd8a7fff640 1 -- 172.21.15.84:0/3508454591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80caa80 msgr2=0x7fd8a80b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:44.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.977+0000 7fd8a7fff640 1 --2- 172.21.15.84:0/3508454591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80caa80 0x7fd8a80b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:44.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.977+0000 7fd8a7fff640 1 -- 172.21.15.84:0/3508454591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a80addb0 msgr2=0x7fd8a80b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:44.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.977+0000 7fd8a7fff640 1 --2- 172.21.15.84:0/3508454591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a80addb0 0x7fd8a80b0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:44.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.977+0000 7fd8a7fff640 1 -- 172.21.15.84:0/3508454591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8900094d0 con 0x7fd8a806a2e0 2023-10-16T09:49:44.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.977+0000 7fd8a7fff640 1 --2- 172.21.15.84:0/3508454591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a806a2e0 0x7fd8a815b1a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd89800c450 tx=0x7fd89800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:44.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.977+0000 7fd8a57fa640 1 -- 172.21.15.84:0/3508454591 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8980052d0 con 0x7fd8a806a2e0 2023-10-16T09:49:44.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.977+0000 7fd8a57fa640 1 -- 172.21.15.84:0/3508454591 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd898005430 con 0x7fd8a806a2e0 2023-10-16T09:49:44.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.978+0000 7fd8ae95a640 1 -- 172.21.15.84:0/3508454591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a815e5a0 con 0x7fd8a806a2e0 2023-10-16T09:49:44.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.978+0000 7fd8a57fa640 1 -- 172.21.15.84:0/3508454591 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd898012870 con 0x7fd8a806a2e0 2023-10-16T09:49:44.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.978+0000 7fd8ae95a640 1 -- 172.21.15.84:0/3508454591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a80b2650 con 0x7fd8a806a2e0 2023-10-16T09:49:44.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.979+0000 7fd8ae95a640 1 -- 172.21.15.84:0/3508454591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a804e160 con 0x7fd8a806a2e0 2023-10-16T09:49:44.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.980+0000 7fd8a57fa640 1 -- 172.21.15.84:0/3508454591 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd8980129d0 con 0x7fd8a806a2e0 2023-10-16T09:49:44.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.981+0000 7fd8a57fa640 1 --2- 172.21.15.84:0/3508454591 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd88806aea0 0x7fd88806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:44.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.981+0000 7fd8a57fa640 1 -- 172.21.15.84:0/3508454591 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd898088f80 con 0x7fd8a806a2e0 2023-10-16T09:49:44.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.981+0000 7fd89ffff640 1 --2- 172.21.15.84:0/3508454591 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd88806aea0 0x7fd88806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:44.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.982+0000 7fd89ffff640 1 --2- 172.21.15.84:0/3508454591 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd88806aea0 0x7fd88806d360 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd890036000 tx=0x7fd89003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:44.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:44.985+0000 7fd8a57fa640 1 -- 172.21.15.84:0/3508454591 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd898054020 con 0x7fd8a806a2e0 2023-10-16T09:49:45.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.171+0000 7fd8ae95a640 1 -- 172.21.15.84:0/3508454591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd8a806a730 con 0x7fd8a806a2e0 2023-10-16T09:49:45.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.177+0000 7fd8a57fa640 1 -- 172.21.15.84:0/3508454591 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd8a806a730 con 0x7fd8a806a2e0 2023-10-16T09:49:45.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.180+0000 7fd8ae95a640 1 -- 172.21.15.84:0/3508454591 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd88806aea0 msgr2=0x7fd88806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:45.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.180+0000 7fd8ae95a640 1 --2- 172.21.15.84:0/3508454591 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd88806aea0 0x7fd88806d360 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd890036000 tx=0x7fd89003b040 comp rx=0 tx=0).stop 2023-10-16T09:49:45.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.180+0000 7fd8ae95a640 1 -- 172.21.15.84:0/3508454591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a806a2e0 msgr2=0x7fd8a815b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:45.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.180+0000 7fd8ae95a640 1 --2- 172.21.15.84:0/3508454591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a806a2e0 0x7fd8a815b1a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd89800c450 tx=0x7fd89800c910 comp rx=0 tx=0).stop 2023-10-16T09:49:45.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.180+0000 7fd8ae95a640 1 -- 172.21.15.84:0/3508454591 shutdown_connections 2023-10-16T09:49:45.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.180+0000 7fd8ae95a640 1 --2- 172.21.15.84:0/3508454591 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd88806aea0 0x7fd88806d360 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:45.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.180+0000 7fd8ae95a640 1 --2- 172.21.15.84:0/3508454591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80caa80 0x7fd8a80b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:45.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.180+0000 7fd8ae95a640 1 --2- 172.21.15.84:0/3508454591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a80addb0 0x7fd8a80b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:45.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.180+0000 7fd8ae95a640 1 --2- 172.21.15.84:0/3508454591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a806a2e0 0x7fd8a815b1a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:45.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.180+0000 7fd8ae95a640 1 -- 172.21.15.84:0/3508454591 >> 172.21.15.84:0/3508454591 conn(0x7fd8a80ce9b0 msgr2=0x7fd8a80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:45.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.180+0000 7fd8ae95a640 1 -- 172.21.15.84:0/3508454591 shutdown_connections 2023-10-16T09:49:45.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.180+0000 7fd8ae95a640 1 -- 172.21.15.84:0/3508454591 wait complete. 2023-10-16T09:49:45.198 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T09:49:45.198 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T09:49:45.321 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T09:49:45.448 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T09:49:45.449 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:49:45.449 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:49:45.506 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:49:45.507 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:49:45.556 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:49:45.557 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:49:45.656 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:45.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.859+0000 7f88ffe75640 1 Processor -- start 2023-10-16T09:49:45.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.859+0000 7f88ffe75640 1 -- start start 2023-10-16T09:49:45.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.860+0000 7f88ffe75640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f88f80b0b20 0x7f88f81615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:45.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.860+0000 7f88ffe75640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f815e3e0 con 0x7f88f80cd0c0 2023-10-16T09:49:45.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.860+0000 7f88ffe75640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f80cd4a0 con 0x7f88f815d3f0 2023-10-16T09:49:45.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.860+0000 7f88ffe75640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f815e100 con 0x7f88f80b0b20 2023-10-16T09:49:45.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.861+0000 7f88fd410640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f88f80b0b20 0x7f88f81615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:45.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.861+0000 7f88fd410640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f88f80b0b20 0x7f88f81615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59658/0 (socket says 172.21.15.84:59658) 2023-10-16T09:49:45.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.861+0000 7f88fd410640 1 -- 172.21.15.84:0/3778731216 learned_addr learned my addr 172.21.15.84:0/3778731216 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:45.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.861+0000 7f88fdc11640 1 --1- 172.21.15.84:0/3778731216 >> v1:172.21.15.84:6789/0 conn(0x7f88f80cd0c0 0x7f88f80b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47132/0 (socket says 172.21.15.84:47132) 2023-10-16T09:49:45.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.861+0000 7f88fd410640 1 -- 172.21.15.84:0/3778731216 >> v1:172.21.15.157:6789/0 conn(0x7f88f815d3f0 legacy=0x7f88f815d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:45.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.861+0000 7f88fd410640 1 -- 172.21.15.84:0/3778731216 >> v1:172.21.15.84:6789/0 conn(0x7f88f80cd0c0 legacy=0x7f88f80b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:45.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.861+0000 7f88fd410640 1 -- 172.21.15.84:0/3778731216 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f8160b10 con 0x7f88f80b0b20 2023-10-16T09:49:45.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.862+0000 7f88fd410640 1 --2- 172.21.15.84:0/3778731216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88f80b0b20 0x7f88f81615f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009870 tx=0x7f88e802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c244e9056b357d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:45.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.862+0000 7f88fcc0f640 1 -- 172.21.15.84:0/3778731216 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8031860 con 0x7f88f80b0b20 2023-10-16T09:49:45.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.862+0000 7f88fcc0f640 1 -- 172.21.15.84:0/3778731216 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e80319c0 con 0x7f88f80b0b20 2023-10-16T09:49:45.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.862+0000 7f88fcc0f640 1 -- 172.21.15.84:0/3778731216 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8031d50 con 0x7f88f80b0b20 2023-10-16T09:49:45.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.863+0000 7f88ffe75640 1 -- 172.21.15.84:0/3778731216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88f80b0b20 msgr2=0x7f88f81615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:45.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.863+0000 7f88ffe75640 1 --2- 172.21.15.84:0/3778731216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88f80b0b20 0x7f88f81615f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009870 tx=0x7f88e802b900 comp rx=0 tx=0).stop 2023-10-16T09:49:45.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.863+0000 7f88ffe75640 1 -- 172.21.15.84:0/3778731216 shutdown_connections 2023-10-16T09:49:45.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.863+0000 7f88ffe75640 1 --2- 172.21.15.84:0/3778731216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88f80b0b20 0x7f88f81615f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:45.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.863+0000 7f88ffe75640 1 -- 172.21.15.84:0/3778731216 >> 172.21.15.84:0/3778731216 conn(0x7f88f80cb410 msgr2=0x7f88f80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:45.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.863+0000 7f88ffe75640 1 -- 172.21.15.84:0/3778731216 shutdown_connections 2023-10-16T09:49:45.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.863+0000 7f88ffe75640 1 -- 172.21.15.84:0/3778731216 wait complete. 2023-10-16T09:49:45.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.865+0000 7f88ffe75640 1 Processor -- start 2023-10-16T09:49:45.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.865+0000 7f88ffe75640 1 -- start start 2023-10-16T09:49:45.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.866+0000 7f88ffe75640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88f80b0b20 0x7f88f814e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:45.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.866+0000 7f88ffe75640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88f80cd0c0 0x7f88f80ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:45.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.866+0000 7f88ffe75640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f815d3f0 0x7f88f80aee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:45.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.866+0000 7f88ffe75640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f8160220 con 0x7f88f80b0b20 2023-10-16T09:49:45.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.866+0000 7f88ffe75640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f8152860 con 0x7f88f815d3f0 2023-10-16T09:49:45.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.866+0000 7f88ffe75640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f815f610 con 0x7f88f80cd0c0 2023-10-16T09:49:45.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.866+0000 7f88fdc11640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88f80b0b20 0x7f88f814e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:45.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.866+0000 7f88fdc11640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88f80b0b20 0x7f88f814e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59672/0 (socket says 172.21.15.84:59672) 2023-10-16T09:49:45.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.866+0000 7f88fdc11640 1 -- 172.21.15.84:0/4100119930 learned_addr learned my addr 172.21.15.84:0/4100119930 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:45.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.866+0000 7f88fdc11640 1 -- 172.21.15.84:0/4100119930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88f80cd0c0 msgr2=0x7f88f80ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:45.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.866+0000 7f88fdc11640 1 --2- 172.21.15.84:0/4100119930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88f80cd0c0 0x7f88f80ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:45.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.866+0000 7f88fdc11640 1 -- 172.21.15.84:0/4100119930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f815d3f0 msgr2=0x7f88f80aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:45.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.866+0000 7f88fdc11640 1 --2- 172.21.15.84:0/4100119930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f815d3f0 0x7f88f80aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:45.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.866+0000 7f88fdc11640 1 -- 172.21.15.84:0/4100119930 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e80094d0 con 0x7f88f80b0b20 2023-10-16T09:49:45.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.867+0000 7f88fdc11640 1 --2- 172.21.15.84:0/4100119930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88f80b0b20 0x7f88f814e7b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f88f400c500 tx=0x7f88f400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:45.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.867+0000 7f88ee7fc640 1 -- 172.21.15.84:0/4100119930 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4007500 con 0x7f88f80b0b20 2023-10-16T09:49:45.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.867+0000 7f88ee7fc640 1 -- 172.21.15.84:0/4100119930 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f4007660 con 0x7f88f80b0b20 2023-10-16T09:49:45.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.867+0000 7f88ffe75640 1 -- 172.21.15.84:0/4100119930 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f815f8f0 con 0x7f88f80b0b20 2023-10-16T09:49:45.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.867+0000 7f88ee7fc640 1 -- 172.21.15.84:0/4100119930 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f40049e0 con 0x7f88f80b0b20 2023-10-16T09:49:45.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.867+0000 7f88ffe75640 1 -- 172.21.15.84:0/4100119930 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f815feb0 con 0x7f88f80b0b20 2023-10-16T09:49:45.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.868+0000 7f88ffe75640 1 -- 172.21.15.84:0/4100119930 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f804e160 con 0x7f88f80b0b20 2023-10-16T09:49:45.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.869+0000 7f88ee7fc640 1 -- 172.21.15.84:0/4100119930 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f88f4004b40 con 0x7f88f80b0b20 2023-10-16T09:49:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.870+0000 7f88ee7fc640 1 --2- 172.21.15.84:0/4100119930 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f88dc06ae70 0x7f88dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.870+0000 7f88ee7fc640 1 -- 172.21.15.84:0/4100119930 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88f4088670 con 0x7f88f80b0b20 2023-10-16T09:49:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.870+0000 7f88fd410640 1 --2- 172.21.15.84:0/4100119930 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f88dc06ae70 0x7f88dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.871+0000 7f88fd410640 1 --2- 172.21.15.84:0/4100119930 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f88dc06ae70 0x7f88dc06d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f88e8036000 tx=0x7f88e8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:45.874+0000 7f88ee7fc640 1 -- 172.21.15.84:0/4100119930 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f88f4053860 con 0x7f88f80b0b20 2023-10-16T09:49:46.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.054+0000 7f88ffe75640 1 -- 172.21.15.84:0/4100119930 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88f80b4910 con 0x7f88f80b0b20 2023-10-16T09:49:46.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.055+0000 7f88ee7fc640 1 -- 172.21.15.84:0/4100119930 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f88f4018090 con 0x7f88f80b0b20 2023-10-16T09:49:46.057 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:46.058 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":21,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:03.529261+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:46.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.058+0000 7f88ffe75640 1 -- 172.21.15.84:0/4100119930 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f88dc06ae70 msgr2=0x7f88dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:46.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.058+0000 7f88ffe75640 1 --2- 172.21.15.84:0/4100119930 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f88dc06ae70 0x7f88dc06d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f88e8036000 tx=0x7f88e8030000 comp rx=0 tx=0).stop 2023-10-16T09:49:46.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.058+0000 7f88ffe75640 1 -- 172.21.15.84:0/4100119930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88f80b0b20 msgr2=0x7f88f814e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:46.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.058+0000 7f88ffe75640 1 --2- 172.21.15.84:0/4100119930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88f80b0b20 0x7f88f814e7b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f88f400c500 tx=0x7f88f400c9c0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.058+0000 7f88ffe75640 1 -- 172.21.15.84:0/4100119930 shutdown_connections 2023-10-16T09:49:46.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.058+0000 7f88ffe75640 1 --2- 172.21.15.84:0/4100119930 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f88dc06ae70 0x7f88dc06d330 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.058+0000 7f88ffe75640 1 --2- 172.21.15.84:0/4100119930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f815d3f0 0x7f88f80aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.058+0000 7f88ffe75640 1 --2- 172.21.15.84:0/4100119930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88f80cd0c0 0x7f88f80ae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.058+0000 7f88ffe75640 1 --2- 172.21.15.84:0/4100119930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88f80b0b20 0x7f88f814e7b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.058+0000 7f88ffe75640 1 -- 172.21.15.84:0/4100119930 >> 172.21.15.84:0/4100119930 conn(0x7f88f80cb410 msgr2=0x7f88f806f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:46.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.058+0000 7f88ffe75640 1 -- 172.21.15.84:0/4100119930 shutdown_connections 2023-10-16T09:49:46.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.058+0000 7f88ffe75640 1 -- 172.21.15.84:0/4100119930 wait complete. 2023-10-16T09:49:46.077 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:49:00.048649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:49:01.361365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:49:46.078 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:49:46.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.285+0000 7f263c43f640 1 Processor -- start 2023-10-16T09:49:46.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.285+0000 7f263c43f640 1 -- start start 2023-10-16T09:49:46.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.287+0000 7f263c43f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f263415d150 0x7f26340ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:46.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.287+0000 7f263c43f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2634160b90 0x7f2634160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:46.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.287+0000 7f263c43f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2634069da0 0x7f263406a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:46.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.287+0000 7f263c43f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f263415d530 con 0x7f2634069da0 2023-10-16T09:49:46.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.287+0000 7f263c43f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f263406a6f0 con 0x7f2634160b90 2023-10-16T09:49:46.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.287+0000 7f263c43f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26340ac890 con 0x7f263415d150 2023-10-16T09:49:46.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.287+0000 7f263a9dc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2634069da0 0x7f263406a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:46.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.287+0000 7f263a9dc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2634069da0 0x7f263406a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59676/0 (socket says 172.21.15.84:59676) 2023-10-16T09:49:46.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.287+0000 7f263a9dc640 1 -- 172.21.15.84:0/174607833 learned_addr learned my addr 172.21.15.84:0/174607833 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:46.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.288+0000 7f26399da640 1 --2- 172.21.15.84:0/174607833 >> v2:172.21.15.157:3300/0 conn(0x7f2634160b90 0x7f2634160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:46.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.288+0000 7f263a1db640 1 --2- 172.21.15.84:0/174607833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f263415d150 0x7f26340ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:46.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.288+0000 7f263a9dc640 1 -- 172.21.15.84:0/174607833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f263415d150 msgr2=0x7f26340ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:46.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.288+0000 7f263a9dc640 1 --2- 172.21.15.84:0/174607833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f263415d150 0x7f26340ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.288+0000 7f263a9dc640 1 -- 172.21.15.84:0/174607833 >> v2:172.21.15.157:3300/0 conn(0x7f2634160b90 msgr2=0x7f2634160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:46.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.288+0000 7f263a9dc640 1 --2- 172.21.15.84:0/174607833 >> v2:172.21.15.157:3300/0 conn(0x7f2634160b90 0x7f2634160fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.288+0000 7f263a9dc640 1 -- 172.21.15.84:0/174607833 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2634161650 con 0x7f2634069da0 2023-10-16T09:49:46.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.289+0000 7f263a9dc640 1 --2- 172.21.15.84:0/174607833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2634069da0 0x7f263406a1b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f26340cf820 tx=0x7f262c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4407a48b936a3f34 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:46.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.289+0000 7f26391d9640 1 -- 172.21.15.84:0/174607833 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c031860 con 0x7f2634069da0 2023-10-16T09:49:46.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.289+0000 7f26391d9640 1 -- 172.21.15.84:0/174607833 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c0319c0 con 0x7f2634069da0 2023-10-16T09:49:46.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.289+0000 7f26391d9640 1 -- 172.21.15.84:0/174607833 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c031d50 con 0x7f2634069da0 2023-10-16T09:49:46.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.290+0000 7f263c43f640 1 -- 172.21.15.84:0/174607833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2634069da0 msgr2=0x7f263406a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:46.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.290+0000 7f263c43f640 1 --2- 172.21.15.84:0/174607833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2634069da0 0x7f263406a1b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f26340cf820 tx=0x7f262c02b900 comp rx=0 tx=0).stop 2023-10-16T09:49:46.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.290+0000 7f263c43f640 1 -- 172.21.15.84:0/174607833 shutdown_connections 2023-10-16T09:49:46.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.290+0000 7f263c43f640 1 --2- 172.21.15.84:0/174607833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2634069da0 0x7f263406a1b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.290+0000 7f263c43f640 1 --2- 172.21.15.84:0/174607833 >> v2:172.21.15.157:3300/0 conn(0x7f2634160b90 0x7f2634160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.290+0000 7f263c43f640 1 --2- 172.21.15.84:0/174607833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f263415d150 0x7f26340ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.290+0000 7f263c43f640 1 -- 172.21.15.84:0/174607833 >> 172.21.15.84:0/174607833 conn(0x7f26340ce9b0 msgr2=0x7f26340cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:46.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.290+0000 7f263c43f640 1 -- 172.21.15.84:0/174607833 shutdown_connections 2023-10-16T09:49:46.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.290+0000 7f263c43f640 1 -- 172.21.15.84:0/174607833 wait complete. 2023-10-16T09:49:46.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.292+0000 7f263c43f640 1 Processor -- start 2023-10-16T09:49:46.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.292+0000 7f263c43f640 1 -- start start 2023-10-16T09:49:46.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.293+0000 7f263c43f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2634069da0 0x7f26341e8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:46.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.293+0000 7f263c43f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f263415d150 0x7f26341e69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:46.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.293+0000 7f263c43f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2634160b90 0x7f26341e6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:46.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.293+0000 7f263c43f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26341e81a0 con 0x7f263415d150 2023-10-16T09:49:46.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.293+0000 7f263c43f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26341560e0 con 0x7f2634069da0 2023-10-16T09:49:46.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.293+0000 7f263c43f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26341e7520 con 0x7f2634160b90 2023-10-16T09:49:46.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.293+0000 7f26399da640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f263415d150 0x7f26341e69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:46.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.293+0000 7f26399da640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f263415d150 0x7f26341e69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59688/0 (socket says 172.21.15.84:59688) 2023-10-16T09:49:46.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.293+0000 7f26399da640 1 -- 172.21.15.84:0/811984232 learned_addr learned my addr 172.21.15.84:0/811984232 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:46.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.293+0000 7f26399da640 1 -- 172.21.15.84:0/811984232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2634160b90 msgr2=0x7f26341e6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:46.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.293+0000 7f26399da640 1 --2- 172.21.15.84:0/811984232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2634160b90 0x7f26341e6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.293+0000 7f26399da640 1 -- 172.21.15.84:0/811984232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2634069da0 msgr2=0x7f26341e8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:46.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.293+0000 7f26399da640 1 --2- 172.21.15.84:0/811984232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2634069da0 0x7f26341e8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.294+0000 7f26399da640 1 -- 172.21.15.84:0/811984232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c0094d0 con 0x7f263415d150 2023-10-16T09:49:46.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.294+0000 7f26399da640 1 --2- 172.21.15.84:0/811984232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f263415d150 0x7f26341e69b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f262000d800 tx=0x7f262000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:46.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.294+0000 7f2632ffd640 1 -- 172.21.15.84:0/811984232 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262000b6c0 con 0x7f263415d150 2023-10-16T09:49:46.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.294+0000 7f2632ffd640 1 -- 172.21.15.84:0/811984232 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262000b820 con 0x7f263415d150 2023-10-16T09:49:46.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.294+0000 7f263c43f640 1 -- 172.21.15.84:0/811984232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26341e7800 con 0x7f263415d150 2023-10-16T09:49:46.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.294+0000 7f2632ffd640 1 -- 172.21.15.84:0/811984232 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262000bbb0 con 0x7f263415d150 2023-10-16T09:49:46.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.294+0000 7f263c43f640 1 -- 172.21.15.84:0/811984232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26341e7dc0 con 0x7f263415d150 2023-10-16T09:49:46.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.295+0000 7f263c43f640 1 -- 172.21.15.84:0/811984232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263404e160 con 0x7f263415d150 2023-10-16T09:49:46.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.296+0000 7f2632ffd640 1 -- 172.21.15.84:0/811984232 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f262000bd10 con 0x7f263415d150 2023-10-16T09:49:46.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.297+0000 7f2632ffd640 1 --2- 172.21.15.84:0/811984232 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f261806ae70 0x7f261806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:46.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.297+0000 7f2632ffd640 1 -- 172.21.15.84:0/811984232 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2620089bc0 con 0x7f263415d150 2023-10-16T09:49:46.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.298+0000 7f263a1db640 1 --2- 172.21.15.84:0/811984232 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f261806ae70 0x7f261806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:46.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.298+0000 7f263a1db640 1 --2- 172.21.15.84:0/811984232 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f261806ae70 0x7f261806d330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2628009540 tx=0x7f2628009450 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:46.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.301+0000 7f2632ffd640 1 -- 172.21.15.84:0/811984232 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2620054c60 con 0x7f263415d150 2023-10-16T09:49:46.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.491+0000 7f263c43f640 1 -- 172.21.15.84:0/811984232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f26340ade10 con 0x7f263415d150 2023-10-16T09:49:46.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.491+0000 7f2632ffd640 1 -- 172.21.15.84:0/811984232 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2620058280 con 0x7f263415d150 2023-10-16T09:49:46.494 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T09:49:46.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.494+0000 7f263c43f640 1 -- 172.21.15.84:0/811984232 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f261806ae70 msgr2=0x7f261806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:46.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.494+0000 7f263c43f640 1 --2- 172.21.15.84:0/811984232 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f261806ae70 0x7f261806d330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2628009540 tx=0x7f2628009450 comp rx=0 tx=0).stop 2023-10-16T09:49:46.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.494+0000 7f263c43f640 1 -- 172.21.15.84:0/811984232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f263415d150 msgr2=0x7f26341e69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:46.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.494+0000 7f263c43f640 1 --2- 172.21.15.84:0/811984232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f263415d150 0x7f26341e69b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f262000d800 tx=0x7f262000dcc0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.494+0000 7f263c43f640 1 -- 172.21.15.84:0/811984232 shutdown_connections 2023-10-16T09:49:46.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.494+0000 7f263c43f640 1 --2- 172.21.15.84:0/811984232 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f261806ae70 0x7f261806d330 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.494+0000 7f263c43f640 1 --2- 172.21.15.84:0/811984232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2634160b90 0x7f26341e6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.494+0000 7f263c43f640 1 --2- 172.21.15.84:0/811984232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f263415d150 0x7f26341e69b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.494+0000 7f263c43f640 1 --2- 172.21.15.84:0/811984232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2634069da0 0x7f26341e8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.494+0000 7f263c43f640 1 -- 172.21.15.84:0/811984232 >> 172.21.15.84:0/811984232 conn(0x7f26340ce9b0 msgr2=0x7f2634070ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:46.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.494+0000 7f263c43f640 1 -- 172.21.15.84:0/811984232 shutdown_connections 2023-10-16T09:49:46.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.494+0000 7f263c43f640 1 -- 172.21.15.84:0/811984232 wait complete. 2023-10-16T09:49:46.514 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:49:46.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.721+0000 7f4b766e5640 1 Processor -- start 2023-10-16T09:49:46.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.722+0000 7f4b766e5640 1 -- start start 2023-10-16T09:49:46.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.723+0000 7f4b766e5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4b70159ab0 0x7f4b700ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:46.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.723+0000 7f4b766e5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b7006c140 0x7f4b7006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:46.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.723+0000 7f4b766e5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b70070360 con 0x7f4b70159e90 2023-10-16T09:49:46.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.723+0000 7f4b766e5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b7006cb10 con 0x7f4b70159ab0 2023-10-16T09:49:46.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.723+0000 7f4b766e5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7015d890 con 0x7f4b7006c140 2023-10-16T09:49:46.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.724+0000 7f4b6ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4b70159ab0 0x7f4b700ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:46.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.724+0000 7f4b6ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4b70159ab0 0x7f4b700ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59700/0 (socket says 172.21.15.84:59700) 2023-10-16T09:49:46.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.724+0000 7f4b6ffff640 1 -- 172.21.15.84:0/1660794318 learned_addr learned my addr 172.21.15.84:0/1660794318 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:46.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.724+0000 7f4b74c82640 1 --2- 172.21.15.84:0/1660794318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b7006c140 0x7f4b7006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:46.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.724+0000 7f4b6ffff640 1 -- 172.21.15.84:0/1660794318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b7006c140 msgr2=0x7f4b7006c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:46.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.724+0000 7f4b6ffff640 1 --2- 172.21.15.84:0/1660794318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b7006c140 0x7f4b7006c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.724+0000 7f4b6ffff640 1 -- 172.21.15.84:0/1660794318 >> v1:172.21.15.157:6789/0 conn(0x7f4b70159e90 legacy=0x7f4b7006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:46.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.724+0000 7f4b6ffff640 1 -- 172.21.15.84:0/1660794318 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b700cd1f0 con 0x7f4b70159ab0 2023-10-16T09:49:46.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.725+0000 7f4b6ffff640 1 --2- 172.21.15.84:0/1660794318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b70159ab0 0x7f4b700ce370 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009870 tx=0x7f4b6002d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b33ed83cfa039b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:46.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.726+0000 7f4b6effd640 1 -- 172.21.15.84:0/1660794318 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b60031810 con 0x7f4b70159ab0 2023-10-16T09:49:46.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.726+0000 7f4b6effd640 1 -- 172.21.15.84:0/1660794318 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b60031970 con 0x7f4b70159ab0 2023-10-16T09:49:46.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.726+0000 7f4b6effd640 1 -- 172.21.15.84:0/1660794318 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b60031d00 con 0x7f4b70159ab0 2023-10-16T09:49:46.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.726+0000 7f4b766e5640 1 -- 172.21.15.84:0/1660794318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b70159ab0 msgr2=0x7f4b700ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:46.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.726+0000 7f4b766e5640 1 --2- 172.21.15.84:0/1660794318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b70159ab0 0x7f4b700ce370 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009870 tx=0x7f4b6002d9f0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.726+0000 7f4b766e5640 1 -- 172.21.15.84:0/1660794318 shutdown_connections 2023-10-16T09:49:46.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.726+0000 7f4b766e5640 1 --2- 172.21.15.84:0/1660794318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b7006c140 0x7f4b7006c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.726+0000 7f4b766e5640 1 --2- 172.21.15.84:0/1660794318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b70159ab0 0x7f4b700ce370 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.726+0000 7f4b766e5640 1 -- 172.21.15.84:0/1660794318 >> 172.21.15.84:0/1660794318 conn(0x7f4b700ce9b0 msgr2=0x7f4b700cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:46.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.726+0000 7f4b766e5640 1 -- 172.21.15.84:0/1660794318 shutdown_connections 2023-10-16T09:49:46.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.726+0000 7f4b766e5640 1 -- 172.21.15.84:0/1660794318 wait complete. 2023-10-16T09:49:46.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.728+0000 7f4b766e5640 1 Processor -- start 2023-10-16T09:49:46.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.728+0000 7f4b766e5640 1 -- start start 2023-10-16T09:49:46.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.729+0000 7f4b766e5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b7006c140 0x7f4b700c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:46.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.729+0000 7f4b766e5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b70159ab0 0x7f4b701567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:46.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.729+0000 7f4b766e5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b70159e90 0x7f4b701f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:46.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.729+0000 7f4b766e5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b700c2e70 con 0x7f4b70159ab0 2023-10-16T09:49:46.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.729+0000 7f4b766e5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b700c3110 con 0x7f4b70159e90 2023-10-16T09:49:46.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.729+0000 7f4b766e5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b700c33f0 con 0x7f4b7006c140 2023-10-16T09:49:46.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.729+0000 7f4b6f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b70159ab0 0x7f4b701567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:46.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.730+0000 7f4b6f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b70159ab0 0x7f4b701567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59714/0 (socket says 172.21.15.84:59714) 2023-10-16T09:49:46.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.730+0000 7f4b6f7fe640 1 -- 172.21.15.84:0/469093194 learned_addr learned my addr 172.21.15.84:0/469093194 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:46.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.730+0000 7f4b74c82640 1 --2- 172.21.15.84:0/469093194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b70159e90 0x7f4b701f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:46.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.730+0000 7f4b6f7fe640 1 -- 172.21.15.84:0/469093194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b7006c140 msgr2=0x7f4b700c24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:46.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.730+0000 7f4b6ffff640 1 --2- 172.21.15.84:0/469093194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b7006c140 0x7f4b700c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:46.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.730+0000 7f4b6f7fe640 1 --2- 172.21.15.84:0/469093194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b7006c140 0x7f4b700c24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.730+0000 7f4b6f7fe640 1 -- 172.21.15.84:0/469093194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b70159e90 msgr2=0x7f4b701f5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:46.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.730+0000 7f4b6f7fe640 1 --2- 172.21.15.84:0/469093194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b70159e90 0x7f4b701f5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.730+0000 7f4b6f7fe640 1 -- 172.21.15.84:0/469093194 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b600094d0 con 0x7f4b70159ab0 2023-10-16T09:49:46.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.730+0000 7f4b74c82640 1 --2- 172.21.15.84:0/469093194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b70159e90 0x7f4b701f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:46.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.730+0000 7f4b6ffff640 1 --2- 172.21.15.84:0/469093194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b7006c140 0x7f4b700c24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:46.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.730+0000 7f4b6f7fe640 1 --2- 172.21.15.84:0/469093194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b70159ab0 0x7f4b701567c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800c7c0 tx=0x7f4b5800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:46.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.731+0000 7f4b6cff9640 1 -- 172.21.15.84:0/469093194 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5800aa50 con 0x7f4b70159ab0 2023-10-16T09:49:46.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.731+0000 7f4b6cff9640 1 -- 172.21.15.84:0/469093194 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5800abb0 con 0x7f4b70159ab0 2023-10-16T09:49:46.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.731+0000 7f4b766e5640 1 -- 172.21.15.84:0/469093194 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b701f2a20 con 0x7f4b70159ab0 2023-10-16T09:49:46.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.731+0000 7f4b6cff9640 1 -- 172.21.15.84:0/469093194 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58012420 con 0x7f4b70159ab0 2023-10-16T09:49:46.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.731+0000 7f4b766e5640 1 -- 172.21.15.84:0/469093194 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b70156d00 con 0x7f4b70159ab0 2023-10-16T09:49:46.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.732+0000 7f4b766e5640 1 -- 172.21.15.84:0/469093194 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b701f2c40 con 0x7f4b70159ab0 2023-10-16T09:49:46.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.734+0000 7f4b6cff9640 1 -- 172.21.15.84:0/469093194 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4b58012580 con 0x7f4b70159ab0 2023-10-16T09:49:46.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.735+0000 7f4b6cff9640 1 --2- 172.21.15.84:0/469093194 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4b5406ae70 0x7f4b5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:46.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.735+0000 7f4b6cff9640 1 -- 172.21.15.84:0/469093194 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b58089520 con 0x7f4b70159ab0 2023-10-16T09:49:46.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.735+0000 7f4b6ffff640 1 --2- 172.21.15.84:0/469093194 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4b5406ae70 0x7f4b5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:46.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.736+0000 7f4b6ffff640 1 --2- 172.21.15.84:0/469093194 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4b5406ae70 0x7f4b5406d330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4b60038000 tx=0x7f4b6003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:46.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.740+0000 7f4b6cff9640 1 -- 172.21.15.84:0/469093194 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4b58054660 con 0x7f4b70159ab0 2023-10-16T09:49:46.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.933+0000 7f4b766e5640 1 -- 172.21.15.84:0/469093194 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4b7004e160 con 0x7f4b70159ab0 2023-10-16T09:49:46.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.933+0000 7f4b6cff9640 1 -- 172.21.15.84:0/469093194 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4b58057c80 con 0x7f4b70159ab0 2023-10-16T09:49:46.936 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T09:49:46.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.936+0000 7f4b766e5640 1 -- 172.21.15.84:0/469093194 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4b5406ae70 msgr2=0x7f4b5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:46.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.936+0000 7f4b766e5640 1 --2- 172.21.15.84:0/469093194 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4b5406ae70 0x7f4b5406d330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4b60038000 tx=0x7f4b6003b040 comp rx=0 tx=0).stop 2023-10-16T09:49:46.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.937+0000 7f4b766e5640 1 -- 172.21.15.84:0/469093194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b70159ab0 msgr2=0x7f4b701567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:46.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.937+0000 7f4b766e5640 1 --2- 172.21.15.84:0/469093194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b70159ab0 0x7f4b701567c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800c7c0 tx=0x7f4b5800cc80 comp rx=0 tx=0).stop 2023-10-16T09:49:46.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.937+0000 7f4b766e5640 1 -- 172.21.15.84:0/469093194 shutdown_connections 2023-10-16T09:49:46.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.937+0000 7f4b766e5640 1 --2- 172.21.15.84:0/469093194 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4b5406ae70 0x7f4b5406d330 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.937+0000 7f4b766e5640 1 --2- 172.21.15.84:0/469093194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b70159e90 0x7f4b701f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.937+0000 7f4b766e5640 1 --2- 172.21.15.84:0/469093194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b70159ab0 0x7f4b701567c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.937+0000 7f4b766e5640 1 --2- 172.21.15.84:0/469093194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b7006c140 0x7f4b700c24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:46.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.937+0000 7f4b766e5640 1 -- 172.21.15.84:0/469093194 >> 172.21.15.84:0/469093194 conn(0x7f4b700ce9b0 msgr2=0x7f4b700b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:46.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.937+0000 7f4b766e5640 1 -- 172.21.15.84:0/469093194 shutdown_connections 2023-10-16T09:49:46.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:46.937+0000 7f4b766e5640 1 -- 172.21.15.84:0/469093194 wait complete. 2023-10-16T09:49:46.957 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:49:47.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.159+0000 7f0b763e9640 1 Processor -- start 2023-10-16T09:49:47.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.159+0000 7f0b763e9640 1 -- start start 2023-10-16T09:49:47.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.161+0000 7f0b763e9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b700c9150 0x7f0b700c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:47.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.161+0000 7f0b763e9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b700c9af0 con 0x7f0b701428c0 2023-10-16T09:49:47.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.161+0000 7f0b763e9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b700cc6b0 con 0x7f0b7006b660 2023-10-16T09:49:47.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.161+0000 7f0b763e9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7006a4b0 con 0x7f0b700c9150 2023-10-16T09:49:47.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.161+0000 7f0b6ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0b701428c0 0x7f0b70069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47144/0 (socket says 172.21.15.84:47144) 2023-10-16T09:49:47.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.161+0000 7f0b6ffff640 1 -- 172.21.15.84:0/2228253030 learned_addr learned my addr 172.21.15.84:0/2228253030 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:47.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.161+0000 7f0b6effd640 1 -- 172.21.15.84:0/2228253030 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3215331862 0 0) 0x7f0b700c9af0 con 0x7f0b701428c0 2023-10-16T09:49:47.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.161+0000 7f0b6effd640 1 -- 172.21.15.84:0/2228253030 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b5c003520 con 0x7f0b701428c0 2023-10-16T09:49:47.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.161+0000 7f0b6f7fe640 1 --2- 172.21.15.84:0/2228253030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b700c9150 0x7f0b700c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:47.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.162+0000 7f0b6effd640 1 -- 172.21.15.84:0/2228253030 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2085391852 0 0) 0x7f0b5c003520 con 0x7f0b701428c0 2023-10-16T09:49:47.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.162+0000 7f0b6effd640 1 -- 172.21.15.84:0/2228253030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b700c9150 msgr2=0x7f0b700c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:47.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.162+0000 7f0b6effd640 1 --2- 172.21.15.84:0/2228253030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b700c9150 0x7f0b700c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.162+0000 7f0b6effd640 1 -- 172.21.15.84:0/2228253030 >> v1:172.21.15.157:6789/0 conn(0x7f0b7006b660 legacy=0x7f0b701643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:47.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.162+0000 7f0b6effd640 1 -- 172.21.15.84:0/2228253030 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b701599b0 con 0x7f0b701428c0 2023-10-16T09:49:47.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.162+0000 7f0b6effd640 1 -- 172.21.15.84:0/2228253030 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f0b680019f0 con 0x7f0b701428c0 2023-10-16T09:49:47.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.162+0000 7f0b6effd640 1 -- 172.21.15.84:0/2228253030 >> v1:172.21.15.84:6789/0 conn(0x7f0b701428c0 legacy=0x7f0b70069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:47.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.162+0000 7f0b6effd640 1 --2- 172.21.15.84:0/2228253030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b5c0039a0 0x7f0b5c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:47.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.162+0000 7f0b6effd640 1 --2- 172.21.15.84:0/2228253030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b700c9150 0x7f0b5c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:47.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.162+0000 7f0b6effd640 1 --2- 172.21.15.84:0/2228253030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5c004d00 0x7f0b5c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:47.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.162+0000 7f0b6effd640 1 -- 172.21.15.84:0/2228253030 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7006a4b0 con 0x7f0b5c004d00 2023-10-16T09:49:47.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.162+0000 7f0b6effd640 1 -- 172.21.15.84:0/2228253030 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c00dd70 con 0x7f0b5c0039a0 2023-10-16T09:49:47.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.162+0000 7f0b6effd640 1 -- 172.21.15.84:0/2228253030 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c00df30 con 0x7f0b700c9150 2023-10-16T09:49:47.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.162+0000 7f0b74986640 1 --2- 172.21.15.84:0/2228253030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5c004d00 0x7f0b5c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:47.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.162+0000 7f0b6ffff640 1 --2- 172.21.15.84:0/2228253030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b5c0039a0 0x7f0b5c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:47.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.163+0000 7f0b74986640 1 -- 172.21.15.84:0/2228253030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b700c9150 msgr2=0x7f0b5c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:47.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.163+0000 7f0b74986640 1 --2- 172.21.15.84:0/2228253030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b700c9150 0x7f0b5c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.163+0000 7f0b74986640 1 -- 172.21.15.84:0/2228253030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b5c0039a0 msgr2=0x7f0b5c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:47.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.163+0000 7f0b74986640 1 --2- 172.21.15.84:0/2228253030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b5c0039a0 0x7f0b5c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.163+0000 7f0b74986640 1 -- 172.21.15.84:0/2228253030 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c003520 con 0x7f0b5c004d00 2023-10-16T09:49:47.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.163+0000 7f0b6ffff640 1 --2- 172.21.15.84:0/2228253030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b5c0039a0 0x7f0b5c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:47.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.164+0000 7f0b74986640 1 --2- 172.21.15.84:0/2228253030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5c004d00 0x7f0b5c005180 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0b60001750 tx=0x7f0b5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa7fc7adcc3b7be server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:47.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.164+0000 7f0b6effd640 1 -- 172.21.15.84:0/2228253030 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b600052d0 con 0x7f0b5c004d00 2023-10-16T09:49:47.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.164+0000 7f0b6effd640 1 -- 172.21.15.84:0/2228253030 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6002e050 con 0x7f0b5c004d00 2023-10-16T09:49:47.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.164+0000 7f0b6effd640 1 -- 172.21.15.84:0/2228253030 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b600035f0 con 0x7f0b5c004d00 2023-10-16T09:49:47.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.164+0000 7f0b763e9640 1 -- 172.21.15.84:0/2228253030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5c004d00 msgr2=0x7f0b5c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:47.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.164+0000 7f0b763e9640 1 --2- 172.21.15.84:0/2228253030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5c004d00 0x7f0b5c005180 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0b60001750 tx=0x7f0b5c0057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.164+0000 7f0b74986640 1 -- 172.21.15.84:0/2228253030 reap_dead start 2023-10-16T09:49:47.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.165+0000 7f0b763e9640 1 -- 172.21.15.84:0/2228253030 shutdown_connections 2023-10-16T09:49:47.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.165+0000 7f0b763e9640 1 -- 172.21.15.84:0/2228253030 >> 172.21.15.84:0/2228253030 conn(0x7f0b700ce9b0 msgr2=0x7f0b700cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:47.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.165+0000 7f0b763e9640 1 -- 172.21.15.84:0/2228253030 shutdown_connections 2023-10-16T09:49:47.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.165+0000 7f0b763e9640 1 -- 172.21.15.84:0/2228253030 wait complete. 2023-10-16T09:49:47.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.166+0000 7f0b763e9640 1 Processor -- start 2023-10-16T09:49:47.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.167+0000 7f0b763e9640 1 -- start start 2023-10-16T09:49:47.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.167+0000 7f0b763e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5c004d00 0x7f0b70157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:47.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.167+0000 7f0b763e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b700c5a80 0x7f0b700c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:47.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.167+0000 7f0b763e9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b700c6e60 0x7f0b700c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:47.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.167+0000 7f0b763e9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700c7820 con 0x7f0b5c004d00 2023-10-16T09:49:47.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.167+0000 7f0b763e9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700c65b0 con 0x7f0b700c5a80 2023-10-16T09:49:47.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.167+0000 7f0b763e9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700c6890 con 0x7f0b700c6e60 2023-10-16T09:49:47.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.168+0000 7f0b6ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5c004d00 0x7f0b70157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:47.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.168+0000 7f0b6ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5c004d00 0x7f0b70157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59738/0 (socket says 172.21.15.84:59738) 2023-10-16T09:49:47.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.168+0000 7f0b6ffff640 1 -- 172.21.15.84:0/186260983 learned_addr learned my addr 172.21.15.84:0/186260983 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:47.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.168+0000 7f0b6ffff640 1 -- 172.21.15.84:0/186260983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b700c6e60 msgr2=0x7f0b700c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:47.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.168+0000 7f0b6ffff640 1 --2- 172.21.15.84:0/186260983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b700c6e60 0x7f0b700c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.168+0000 7f0b6ffff640 1 -- 172.21.15.84:0/186260983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b700c5a80 msgr2=0x7f0b700c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:47.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.168+0000 7f0b6ffff640 1 --2- 172.21.15.84:0/186260983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b700c5a80 0x7f0b700c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.168+0000 7f0b6ffff640 1 -- 172.21.15.84:0/186260983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b700b1670 con 0x7f0b5c004d00 2023-10-16T09:49:47.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.168+0000 7f0b6ffff640 1 --2- 172.21.15.84:0/186260983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5c004d00 0x7f0b70157c70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0b68002120 tx=0x7f0b6800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:47.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.169+0000 7f0b6cff9640 1 -- 172.21.15.84:0/186260983 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68004d60 con 0x7f0b5c004d00 2023-10-16T09:49:47.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.169+0000 7f0b6cff9640 1 -- 172.21.15.84:0/186260983 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b680070d0 con 0x7f0b5c004d00 2023-10-16T09:49:47.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.169+0000 7f0b6cff9640 1 -- 172.21.15.84:0/186260983 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68007440 con 0x7f0b5c004d00 2023-10-16T09:49:47.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.169+0000 7f0b763e9640 1 -- 172.21.15.84:0/186260983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b700b1950 con 0x7f0b5c004d00 2023-10-16T09:49:47.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.169+0000 7f0b763e9640 1 -- 172.21.15.84:0/186260983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b700c6b70 con 0x7f0b5c004d00 2023-10-16T09:49:47.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.170+0000 7f0b763e9640 1 -- 172.21.15.84:0/186260983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b700b39f0 con 0x7f0b5c004d00 2023-10-16T09:49:47.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.171+0000 7f0b6cff9640 1 -- 172.21.15.84:0/186260983 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0b680075a0 con 0x7f0b5c004d00 2023-10-16T09:49:47.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.171+0000 7f0b6cff9640 1 --2- 172.21.15.84:0/186260983 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0b5006aea0 0x7f0b5006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:47.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.172+0000 7f0b6cff9640 1 -- 172.21.15.84:0/186260983 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b68089780 con 0x7f0b5c004d00 2023-10-16T09:49:47.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.172+0000 7f0b6f7fe640 1 --2- 172.21.15.84:0/186260983 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0b5006aea0 0x7f0b5006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:47.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.172+0000 7f0b6f7fe640 1 --2- 172.21.15.84:0/186260983 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0b5006aea0 0x7f0b5006d360 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0b70070d60 tx=0x7f0b58008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:47.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.176+0000 7f0b6cff9640 1 -- 172.21.15.84:0/186260983 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0b68054790 con 0x7f0b5c004d00 2023-10-16T09:49:47.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.355+0000 7f0b763e9640 1 -- 172.21.15.84:0/186260983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b7004e160 con 0x7f0b5c004d00 2023-10-16T09:49:47.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.356+0000 7f0b6cff9640 1 -- 172.21.15.84:0/186260983 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0b680190a0 con 0x7f0b5c004d00 2023-10-16T09:49:47.358 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T09:49:47.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.358+0000 7f0b763e9640 1 -- 172.21.15.84:0/186260983 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0b5006aea0 msgr2=0x7f0b5006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:47.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.358+0000 7f0b763e9640 1 --2- 172.21.15.84:0/186260983 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0b5006aea0 0x7f0b5006d360 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0b70070d60 tx=0x7f0b58008040 comp rx=0 tx=0).stop 2023-10-16T09:49:47.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.358+0000 7f0b763e9640 1 -- 172.21.15.84:0/186260983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5c004d00 msgr2=0x7f0b70157c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:47.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.358+0000 7f0b763e9640 1 --2- 172.21.15.84:0/186260983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5c004d00 0x7f0b70157c70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0b68002120 tx=0x7f0b6800b4a0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.358+0000 7f0b763e9640 1 -- 172.21.15.84:0/186260983 shutdown_connections 2023-10-16T09:49:47.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.358+0000 7f0b763e9640 1 --2- 172.21.15.84:0/186260983 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0b5006aea0 0x7f0b5006d360 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.358+0000 7f0b763e9640 1 --2- 172.21.15.84:0/186260983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b700c6e60 0x7f0b700c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.358+0000 7f0b763e9640 1 --2- 172.21.15.84:0/186260983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b700c5a80 0x7f0b700c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.358+0000 7f0b763e9640 1 --2- 172.21.15.84:0/186260983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5c004d00 0x7f0b70157c70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.358+0000 7f0b763e9640 1 -- 172.21.15.84:0/186260983 >> 172.21.15.84:0/186260983 conn(0x7f0b700ce9b0 msgr2=0x7f0b700ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:47.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.358+0000 7f0b763e9640 1 -- 172.21.15.84:0/186260983 shutdown_connections 2023-10-16T09:49:47.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.359+0000 7f0b763e9640 1 -- 172.21.15.84:0/186260983 wait complete. 2023-10-16T09:49:47.378 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T09:49:47.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.581+0000 7fd8aec34640 1 Processor -- start 2023-10-16T09:49:47.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.581+0000 7fd8aec34640 1 -- start start 2023-10-16T09:49:47.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.583+0000 7fd8aec34640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd8a815ad70 0x7fd8a8069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:47.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.583+0000 7fd8aec34640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80c9ca0 0x7fd8a806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:47.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.583+0000 7fd8aec34640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a80cb030 con 0x7fd8a80ca3b0 2023-10-16T09:49:47.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.583+0000 7fd8aec34640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8a80c91c0 con 0x7fd8a815ad70 2023-10-16T09:49:47.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.583+0000 7fd8aec34640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80c94a0 con 0x7fd8a80c9ca0 2023-10-16T09:49:47.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.583+0000 7fd8ac9d0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd8a815ad70 0x7fd8a8069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:47.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.583+0000 7fd8ac9d0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd8a815ad70 0x7fd8a8069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59750/0 (socket says 172.21.15.84:59750) 2023-10-16T09:49:47.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.583+0000 7fd8ac9d0640 1 -- 172.21.15.84:0/1852801154 learned_addr learned my addr 172.21.15.84:0/1852801154 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:47.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.584+0000 7fd8a7fff640 1 --2- 172.21.15.84:0/1852801154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80c9ca0 0x7fd8a806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:47.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.584+0000 7fd8ac9d0640 1 -- 172.21.15.84:0/1852801154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80c9ca0 msgr2=0x7fd8a806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:47.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.584+0000 7fd8ac9d0640 1 --2- 172.21.15.84:0/1852801154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80c9ca0 0x7fd8a806a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.584+0000 7fd8ac9d0640 1 -- 172.21.15.84:0/1852801154 >> v1:172.21.15.157:6789/0 conn(0x7fd8a80ca3b0 legacy=0x7fd8a80ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:47.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.584+0000 7fd8ac9d0640 1 -- 172.21.15.84:0/1852801154 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a80c9780 con 0x7fd8a815ad70 2023-10-16T09:49:47.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.585+0000 7fd8ac9d0640 1 --2- 172.21.15.84:0/1852801154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a815ad70 0x7fd8a8069da0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd898009e50 tx=0x7fd89802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72081b309c436abb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:47.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.585+0000 7fd8a77fe640 1 -- 172.21.15.84:0/1852801154 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89802ca70 con 0x7fd8a815ad70 2023-10-16T09:49:47.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.585+0000 7fd8a77fe640 1 -- 172.21.15.84:0/1852801154 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89802cbd0 con 0x7fd8a815ad70 2023-10-16T09:49:47.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.585+0000 7fd8a77fe640 1 -- 172.21.15.84:0/1852801154 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8980343f0 con 0x7fd8a815ad70 2023-10-16T09:49:47.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.586+0000 7fd8aec34640 1 -- 172.21.15.84:0/1852801154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a815ad70 msgr2=0x7fd8a8069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:47.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.586+0000 7fd8aec34640 1 --2- 172.21.15.84:0/1852801154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a815ad70 0x7fd8a8069da0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd898009e50 tx=0x7fd89802bed0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.586+0000 7fd8aec34640 1 -- 172.21.15.84:0/1852801154 shutdown_connections 2023-10-16T09:49:47.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.586+0000 7fd8aec34640 1 --2- 172.21.15.84:0/1852801154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80c9ca0 0x7fd8a806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.586+0000 7fd8aec34640 1 --2- 172.21.15.84:0/1852801154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a815ad70 0x7fd8a8069da0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.586+0000 7fd8aec34640 1 -- 172.21.15.84:0/1852801154 >> 172.21.15.84:0/1852801154 conn(0x7fd8a80ce9b0 msgr2=0x7fd8a80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:47.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.586+0000 7fd8aec34640 1 -- 172.21.15.84:0/1852801154 shutdown_connections 2023-10-16T09:49:47.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.586+0000 7fd8aec34640 1 -- 172.21.15.84:0/1852801154 wait complete. 2023-10-16T09:49:47.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.588+0000 7fd8aec34640 1 Processor -- start 2023-10-16T09:49:47.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.588+0000 7fd8aec34640 1 -- start start 2023-10-16T09:49:47.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.589+0000 7fd8aec34640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a80c9ca0 0x7fd8a814b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:47.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.589+0000 7fd8aec34640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80ca3b0 0x7fd8a8159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:47.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.589+0000 7fd8aec34640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a815ad70 0x7fd8a8069da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:47.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.589+0000 7fd8aec34640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a815d9a0 con 0x7fd8a815ad70 2023-10-16T09:49:47.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.589+0000 7fd8aec34640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a815b450 con 0x7fd8a80c9ca0 2023-10-16T09:49:47.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.589+0000 7fd8aec34640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a8158ed0 con 0x7fd8a80ca3b0 2023-10-16T09:49:47.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.590+0000 7fd8ad1d1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a815ad70 0x7fd8a8069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:47.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.590+0000 7fd8ad1d1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a815ad70 0x7fd8a8069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59766/0 (socket says 172.21.15.84:59766) 2023-10-16T09:49:47.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.590+0000 7fd8ad1d1640 1 -- 172.21.15.84:0/1282948701 learned_addr learned my addr 172.21.15.84:0/1282948701 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:47.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.590+0000 7fd8ac9d0640 1 --2- 172.21.15.84:0/1282948701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a80c9ca0 0x7fd8a814b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:47.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.590+0000 7fd8a7fff640 1 --2- 172.21.15.84:0/1282948701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80ca3b0 0x7fd8a8159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:47.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.590+0000 7fd8ad1d1640 1 -- 172.21.15.84:0/1282948701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80ca3b0 msgr2=0x7fd8a8159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:47.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.590+0000 7fd8ad1d1640 1 --2- 172.21.15.84:0/1282948701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80ca3b0 0x7fd8a8159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.590+0000 7fd8ad1d1640 1 -- 172.21.15.84:0/1282948701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a80c9ca0 msgr2=0x7fd8a814b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:47.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.590+0000 7fd8ad1d1640 1 --2- 172.21.15.84:0/1282948701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a80c9ca0 0x7fd8a814b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.590+0000 7fd8ad1d1640 1 -- 172.21.15.84:0/1282948701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd898009b30 con 0x7fd8a815ad70 2023-10-16T09:49:47.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.590+0000 7fd8ac9d0640 1 --2- 172.21.15.84:0/1282948701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a80c9ca0 0x7fd8a814b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:47.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.591+0000 7fd8ad1d1640 1 --2- 172.21.15.84:0/1282948701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a815ad70 0x7fd8a8069da0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000c440 tx=0x7fd8a000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:47.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.591+0000 7fd8a57fa640 1 -- 172.21.15.84:0/1282948701 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0011790 con 0x7fd8a815ad70 2023-10-16T09:49:47.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.591+0000 7fd8a57fa640 1 -- 172.21.15.84:0/1282948701 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a00118f0 con 0x7fd8a815ad70 2023-10-16T09:49:47.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.591+0000 7fd8a57fa640 1 -- 172.21.15.84:0/1282948701 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0011ca0 con 0x7fd8a815ad70 2023-10-16T09:49:47.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.591+0000 7fd8aec34640 1 -- 172.21.15.84:0/1282948701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a815b6a0 con 0x7fd8a815ad70 2023-10-16T09:49:47.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.591+0000 7fd8aec34640 1 -- 172.21.15.84:0/1282948701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a815bc60 con 0x7fd8a815ad70 2023-10-16T09:49:47.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.592+0000 7fd8aec34640 1 -- 172.21.15.84:0/1282948701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a80ad480 con 0x7fd8a815ad70 2023-10-16T09:49:47.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.593+0000 7fd8a57fa640 1 -- 172.21.15.84:0/1282948701 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd8a001c460 con 0x7fd8a815ad70 2023-10-16T09:49:47.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.594+0000 7fd8a57fa640 1 --2- 172.21.15.84:0/1282948701 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd88806aea0 0x7fd88806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:47.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.594+0000 7fd8a57fa640 1 -- 172.21.15.84:0/1282948701 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8a00895c0 con 0x7fd8a815ad70 2023-10-16T09:49:47.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.594+0000 7fd8ac9d0640 1 --2- 172.21.15.84:0/1282948701 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd88806aea0 0x7fd88806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:47.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.595+0000 7fd8ac9d0640 1 --2- 172.21.15.84:0/1282948701 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd88806aea0 0x7fd88806d360 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd898035000 tx=0x7fd898030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:47.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.599+0000 7fd8a57fa640 1 -- 172.21.15.84:0/1282948701 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd8a00545d0 con 0x7fd8a815ad70 2023-10-16T09:49:47.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.790+0000 7fd8aec34640 1 -- 172.21.15.84:0/1282948701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd8a804e160 con 0x7fd8a815ad70 2023-10-16T09:49:47.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.790+0000 7fd8a57fa640 1 -- 172.21.15.84:0/1282948701 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd8a0057bf0 con 0x7fd8a815ad70 2023-10-16T09:49:47.793 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-16T09:49:47.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.793+0000 7fd8aec34640 1 -- 172.21.15.84:0/1282948701 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd88806aea0 msgr2=0x7fd88806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:47.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.793+0000 7fd8aec34640 1 --2- 172.21.15.84:0/1282948701 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd88806aea0 0x7fd88806d360 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd898035000 tx=0x7fd898030000 comp rx=0 tx=0).stop 2023-10-16T09:49:47.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.793+0000 7fd8aec34640 1 -- 172.21.15.84:0/1282948701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a815ad70 msgr2=0x7fd8a8069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:47.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.793+0000 7fd8aec34640 1 --2- 172.21.15.84:0/1282948701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a815ad70 0x7fd8a8069da0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000c440 tx=0x7fd8a000c900 comp rx=0 tx=0).stop 2023-10-16T09:49:47.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.793+0000 7fd8aec34640 1 -- 172.21.15.84:0/1282948701 shutdown_connections 2023-10-16T09:49:47.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.793+0000 7fd8aec34640 1 --2- 172.21.15.84:0/1282948701 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd88806aea0 0x7fd88806d360 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.793+0000 7fd8aec34640 1 --2- 172.21.15.84:0/1282948701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a815ad70 0x7fd8a8069da0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.793+0000 7fd8aec34640 1 --2- 172.21.15.84:0/1282948701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a80ca3b0 0x7fd8a8159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.793+0000 7fd8aec34640 1 --2- 172.21.15.84:0/1282948701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8a80c9ca0 0x7fd8a814b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:47.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.793+0000 7fd8aec34640 1 -- 172.21.15.84:0/1282948701 >> 172.21.15.84:0/1282948701 conn(0x7fd8a80ce9b0 msgr2=0x7fd8a80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:47.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.793+0000 7fd8aec34640 1 -- 172.21.15.84:0/1282948701 shutdown_connections 2023-10-16T09:49:47.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:47.793+0000 7fd8aec34640 1 -- 172.21.15.84:0/1282948701 wait complete. 2023-10-16T09:49:47.812 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T09:49:47.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:47.813 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-16T09:49:47.910 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T09:49:47.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:47.911 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T09:49:47.974 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:49:47.975 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:49:48.037 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:49:48.037 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:49:48.091 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:49:48.092 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:49:48.149 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:48.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.370+0000 7fb29ce6f640 1 Processor -- start 2023-10-16T09:49:48.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.370+0000 7fb29ce6f640 1 -- start start 2023-10-16T09:49:48.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.372+0000 7fb29ce6f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb29815ae80 0x7fb29815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:48.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.372+0000 7fb29ce6f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2980ce390 0x7fb298070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:48.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.372+0000 7fb29ce6f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb29815ed60 con 0x7fb29815aaa0 2023-10-16T09:49:48.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.372+0000 7fb29ce6f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb29806b810 con 0x7fb29815ae80 2023-10-16T09:49:48.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.372+0000 7fb29ce6f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980707e0 con 0x7fb2980ce390 2023-10-16T09:49:48.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.373+0000 7fb296d9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2980ce390 0x7fb298070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:48.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.373+0000 7fb29659c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb29815aaa0 0x7fb2980cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:35618/0 (socket says 172.21.15.84:35618) 2023-10-16T09:49:48.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.373+0000 7fb28dd9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb29815ae80 0x7fb29815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:48.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.373+0000 7fb29659c640 1 -- 172.21.15.84:0/1911658435 learned_addr learned my addr 172.21.15.84:0/1911658435 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:48.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.373+0000 7fb296d9d640 1 -- 172.21.15.84:0/1911658435 >> v2:172.21.15.157:3300/0 conn(0x7fb29815ae80 msgr2=0x7fb29815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:48.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.373+0000 7fb296d9d640 1 --2- 172.21.15.84:0/1911658435 >> v2:172.21.15.157:3300/0 conn(0x7fb29815ae80 0x7fb29815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:48.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.373+0000 7fb296d9d640 1 -- 172.21.15.84:0/1911658435 >> v1:172.21.15.157:6789/0 conn(0x7fb29815aaa0 legacy=0x7fb2980cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:48.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.373+0000 7fb296d9d640 1 -- 172.21.15.84:0/1911658435 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29815ef90 con 0x7fb2980ce390 2023-10-16T09:49:48.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.375+0000 7fb296d9d640 1 --2- 172.21.15.84:0/1911658435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2980ce390 0x7fb298070f50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb290009d90 tx=0x7fb29002beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a487dca0952e9dda server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:48.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.376+0000 7fb295d9b640 1 -- 172.21.15.84:0/1911658435 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29002cc60 con 0x7fb2980ce390 2023-10-16T09:49:48.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.376+0000 7fb295d9b640 1 -- 172.21.15.84:0/1911658435 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29002cdc0 con 0x7fb2980ce390 2023-10-16T09:49:48.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.376+0000 7fb295d9b640 1 -- 172.21.15.84:0/1911658435 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb290034660 con 0x7fb2980ce390 2023-10-16T09:49:48.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.376+0000 7fb29ce6f640 1 -- 172.21.15.84:0/1911658435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2980ce390 msgr2=0x7fb298070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:48.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.376+0000 7fb29ce6f640 1 --2- 172.21.15.84:0/1911658435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2980ce390 0x7fb298070f50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb290009d90 tx=0x7fb29002beb0 comp rx=0 tx=0).stop 2023-10-16T09:49:48.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.377+0000 7fb29ce6f640 1 -- 172.21.15.84:0/1911658435 shutdown_connections 2023-10-16T09:49:48.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.377+0000 7fb29ce6f640 1 --2- 172.21.15.84:0/1911658435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2980ce390 0x7fb298070f50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:48.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.377+0000 7fb29ce6f640 1 --2- 172.21.15.84:0/1911658435 >> v2:172.21.15.157:3300/0 conn(0x7fb29815ae80 0x7fb29815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:48.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.377+0000 7fb29ce6f640 1 -- 172.21.15.84:0/1911658435 >> 172.21.15.84:0/1911658435 conn(0x7fb2980cb410 msgr2=0x7fb2980cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:48.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.377+0000 7fb29ce6f640 1 -- 172.21.15.84:0/1911658435 shutdown_connections 2023-10-16T09:49:48.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.377+0000 7fb29ce6f640 1 -- 172.21.15.84:0/1911658435 wait complete. 2023-10-16T09:49:48.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.379+0000 7fb29ce6f640 1 Processor -- start 2023-10-16T09:49:48.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.379+0000 7fb29ce6f640 1 -- start start 2023-10-16T09:49:48.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.380+0000 7fb29ce6f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2980ce390 0x7fb2980c2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:48.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.380+0000 7fb29ce6f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb29815aaa0 0x7fb29814c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:48.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.380+0000 7fb29ce6f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb29815ae80 0x7fb29815d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:48.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.380+0000 7fb29ce6f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980c4d70 con 0x7fb29815aaa0 2023-10-16T09:49:48.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.380+0000 7fb29ce6f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980c40f0 con 0x7fb29815ae80 2023-10-16T09:49:48.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.380+0000 7fb29ce6f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980c43d0 con 0x7fb2980ce390 2023-10-16T09:49:48.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.381+0000 7fb28dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb29815aaa0 0x7fb29814c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:48.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.381+0000 7fb28dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb29815aaa0 0x7fb29814c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59770/0 (socket says 172.21.15.84:59770) 2023-10-16T09:49:48.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.381+0000 7fb28dd9b640 1 -- 172.21.15.84:0/2992238100 learned_addr learned my addr 172.21.15.84:0/2992238100 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:48.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.381+0000 7fb29659c640 1 --2- 172.21.15.84:0/2992238100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2980ce390 0x7fb2980c2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:48.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.381+0000 7fb296d9d640 1 --2- 172.21.15.84:0/2992238100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb29815ae80 0x7fb29815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:48.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.381+0000 7fb28dd9b640 1 -- 172.21.15.84:0/2992238100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2980ce390 msgr2=0x7fb2980c2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:48.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.381+0000 7fb28dd9b640 1 --2- 172.21.15.84:0/2992238100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2980ce390 0x7fb2980c2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:48.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.381+0000 7fb28dd9b640 1 -- 172.21.15.84:0/2992238100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb29815ae80 msgr2=0x7fb29815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:48.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.381+0000 7fb28dd9b640 1 --2- 172.21.15.84:0/2992238100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb29815ae80 0x7fb29815d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:48.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.381+0000 7fb28dd9b640 1 -- 172.21.15.84:0/2992238100 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb290009a70 con 0x7fb29815aaa0 2023-10-16T09:49:48.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.381+0000 7fb296d9d640 1 --2- 172.21.15.84:0/2992238100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb29815ae80 0x7fb29815d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:48.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.381+0000 7fb28dd9b640 1 --2- 172.21.15.84:0/2992238100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb29815aaa0 0x7fb29814c520 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb28000e840 tx=0x7fb28000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:48.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.382+0000 7fb28f7fe640 1 -- 172.21.15.84:0/2992238100 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2800094f0 con 0x7fb29815aaa0 2023-10-16T09:49:48.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.382+0000 7fb28f7fe640 1 -- 172.21.15.84:0/2992238100 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28000c940 con 0x7fb29815aaa0 2023-10-16T09:49:48.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.382+0000 7fb28f7fe640 1 -- 172.21.15.84:0/2992238100 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28000ccd0 con 0x7fb29815aaa0 2023-10-16T09:49:48.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.382+0000 7fb29ce6f640 1 -- 172.21.15.84:0/2992238100 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2980c46b0 con 0x7fb29815aaa0 2023-10-16T09:49:48.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.382+0000 7fb29ce6f640 1 -- 172.21.15.84:0/2992238100 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2980c3430 con 0x7fb29815aaa0 2023-10-16T09:49:48.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.383+0000 7fb29ce6f640 1 -- 172.21.15.84:0/2992238100 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2980c4eb0 con 0x7fb29815aaa0 2023-10-16T09:49:48.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.384+0000 7fb28f7fe640 1 -- 172.21.15.84:0/2992238100 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb280013470 con 0x7fb29815aaa0 2023-10-16T09:49:48.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.385+0000 7fb28f7fe640 1 --2- 172.21.15.84:0/2992238100 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb27806ae70 0x7fb27806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:48.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.385+0000 7fb28f7fe640 1 -- 172.21.15.84:0/2992238100 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb280089780 con 0x7fb29815aaa0 2023-10-16T09:49:48.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.386+0000 7fb29659c640 1 --2- 172.21.15.84:0/2992238100 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb27806ae70 0x7fb27806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:48.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.386+0000 7fb29659c640 1 --2- 172.21.15.84:0/2992238100 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb27806ae70 0x7fb27806d330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb288000f60 tx=0x7fb288009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:48.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.390+0000 7fb28f7fe640 1 -- 172.21.15.84:0/2992238100 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb2980c4eb0 con 0x7fb29815aaa0 2023-10-16T09:49:48.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.591+0000 7fb29ce6f640 1 -- 172.21.15.84:0/2992238100 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2980c4eb0 con 0x7fb29815aaa0 2023-10-16T09:49:48.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.593+0000 7fb28f7fe640 1 -- 172.21.15.84:0/2992238100 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16730 (secure 0 0 0) 0x7fb2980c4eb0 con 0x7fb29815aaa0 2023-10-16T09:49:48.595 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:48.595 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":22,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:48.543855+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.27","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:48.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.595+0000 7fb29ce6f640 1 -- 172.21.15.84:0/2992238100 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb27806ae70 msgr2=0x7fb27806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:48.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.595+0000 7fb29ce6f640 1 --2- 172.21.15.84:0/2992238100 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb27806ae70 0x7fb27806d330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb288000f60 tx=0x7fb288009040 comp rx=0 tx=0).stop 2023-10-16T09:49:48.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.595+0000 7fb29ce6f640 1 -- 172.21.15.84:0/2992238100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb29815aaa0 msgr2=0x7fb29814c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:48.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.595+0000 7fb29ce6f640 1 --2- 172.21.15.84:0/2992238100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb29815aaa0 0x7fb29814c520 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb28000e840 tx=0x7fb28000ed00 comp rx=0 tx=0).stop 2023-10-16T09:49:48.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.595+0000 7fb29ce6f640 1 -- 172.21.15.84:0/2992238100 shutdown_connections 2023-10-16T09:49:48.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.595+0000 7fb29ce6f640 1 --2- 172.21.15.84:0/2992238100 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb27806ae70 0x7fb27806d330 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:48.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.595+0000 7fb29ce6f640 1 --2- 172.21.15.84:0/2992238100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb29815ae80 0x7fb29815d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:48.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.595+0000 7fb29ce6f640 1 --2- 172.21.15.84:0/2992238100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb29815aaa0 0x7fb29814c520 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:48.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.595+0000 7fb29ce6f640 1 --2- 172.21.15.84:0/2992238100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2980ce390 0x7fb2980c2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:48.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.595+0000 7fb29ce6f640 1 -- 172.21.15.84:0/2992238100 >> 172.21.15.84:0/2992238100 conn(0x7fb2980cb410 msgr2=0x7fb29806f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:48.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.596+0000 7fb29ce6f640 1 -- 172.21.15.84:0/2992238100 shutdown_connections 2023-10-16T09:49:48.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.596+0000 7fb29ce6f640 1 -- 172.21.15.84:0/2992238100 wait complete. 2023-10-16T09:49:48.616 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:49:00.048649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:49:01.361365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:49:48.617 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:49:48.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.827+0000 7f009e875640 1 Processor -- start 2023-10-16T09:49:48.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.828+0000 7f009e875640 1 -- start start 2023-10-16T09:49:48.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.829+0000 7f009e875640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f009815d330 0x7f00981618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:48.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.829+0000 7f009e875640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f00980661c0 0x7f00980ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:48.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.829+0000 7f009e875640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00980d0890 con 0x7f009806c7d0 2023-10-16T09:49:48.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.829+0000 7f009e875640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f009806b1e0 con 0x7f00980661c0 2023-10-16T09:49:48.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.829+0000 7f009e875640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00980d05b0 con 0x7f009815d330 2023-10-16T09:49:48.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.830+0000 7f009ce12640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f00980661c0 0x7f00980ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:48.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.830+0000 7f009ce12640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f00980661c0 0x7f00980ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59782/0 (socket says 172.21.15.84:59782) 2023-10-16T09:49:48.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.830+0000 7f009ce12640 1 -- 172.21.15.84:0/4173143774 learned_addr learned my addr 172.21.15.84:0/4173143774 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:48.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.830+0000 7f0097fff640 1 --2- 172.21.15.84:0/4173143774 >> v2:172.21.15.157:3300/0 conn(0x7f009815d330 0x7f00981618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:48.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.830+0000 7f009ce12640 1 -- 172.21.15.84:0/4173143774 >> v2:172.21.15.157:3300/0 conn(0x7f009815d330 msgr2=0x7f00981618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:48.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.830+0000 7f009ce12640 1 --2- 172.21.15.84:0/4173143774 >> v2:172.21.15.157:3300/0 conn(0x7f009815d330 0x7f00981618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:48.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.830+0000 7f009ce12640 1 -- 172.21.15.84:0/4173143774 >> v1:172.21.15.157:6789/0 conn(0x7f009806c7d0 legacy=0x7f0098065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:48.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.830+0000 7f009ce12640 1 -- 172.21.15.84:0/4173143774 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0098160dd0 con 0x7f00980661c0 2023-10-16T09:49:48.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.831+0000 7f009ce12640 1 --2- 172.21.15.84:0/4173143774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00980661c0 0x7f00980ce9b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f00980744f0 tx=0x7f009002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab2279e01c5fcc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:48.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.832+0000 7f0096ffd640 1 -- 172.21.15.84:0/4173143774 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0090031870 con 0x7f00980661c0 2023-10-16T09:49:48.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.832+0000 7f0096ffd640 1 -- 172.21.15.84:0/4173143774 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00900319d0 con 0x7f00980661c0 2023-10-16T09:49:48.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.832+0000 7f0096ffd640 1 -- 172.21.15.84:0/4173143774 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0090031d60 con 0x7f00980661c0 2023-10-16T09:49:48.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.832+0000 7f009e875640 1 -- 172.21.15.84:0/4173143774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00980661c0 msgr2=0x7f00980ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:48.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.832+0000 7f009e875640 1 --2- 172.21.15.84:0/4173143774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00980661c0 0x7f00980ce9b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f00980744f0 tx=0x7f009002b910 comp rx=0 tx=0).stop 2023-10-16T09:49:48.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.832+0000 7f009e875640 1 -- 172.21.15.84:0/4173143774 shutdown_connections 2023-10-16T09:49:48.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.832+0000 7f009e875640 1 --2- 172.21.15.84:0/4173143774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00980661c0 0x7f00980ce9b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:48.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.833+0000 7f009e875640 1 --2- 172.21.15.84:0/4173143774 >> v2:172.21.15.157:3300/0 conn(0x7f009815d330 0x7f00981618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:48.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.833+0000 7f009e875640 1 -- 172.21.15.84:0/4173143774 >> 172.21.15.84:0/4173143774 conn(0x7f00980d0ab0 msgr2=0x7f00980d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:48.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.833+0000 7f009e875640 1 -- 172.21.15.84:0/4173143774 shutdown_connections 2023-10-16T09:49:48.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.833+0000 7f009e875640 1 -- 172.21.15.84:0/4173143774 wait complete. 2023-10-16T09:49:48.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.834+0000 7f009e875640 1 Processor -- start 2023-10-16T09:49:48.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.835+0000 7f009e875640 1 -- start start 2023-10-16T09:49:48.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.835+0000 7f009e875640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00980661c0 0x7f0098160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:48.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.836+0000 7f009e875640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f009806c7d0 0x7f00981584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:48.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.836+0000 7f009e875640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f009815d330 0x7f00981589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:48.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.836+0000 7f009e875640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0098158f20 con 0x7f009815d330 2023-10-16T09:49:48.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.836+0000 7f009e875640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009815f720 con 0x7f00980661c0 2023-10-16T09:49:48.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.836+0000 7f009e875640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00980c4550 con 0x7f009806c7d0 2023-10-16T09:49:48.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.836+0000 7f009ce12640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f009815d330 0x7f00981589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:48.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.836+0000 7f009ce12640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f009815d330 0x7f00981589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59796/0 (socket says 172.21.15.84:59796) 2023-10-16T09:49:48.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.836+0000 7f009ce12640 1 -- 172.21.15.84:0/1401557264 learned_addr learned my addr 172.21.15.84:0/1401557264 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:48.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.836+0000 7f0097fff640 1 --2- 172.21.15.84:0/1401557264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00980661c0 0x7f0098160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:48.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.836+0000 7f00977fe640 1 --2- 172.21.15.84:0/1401557264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f009806c7d0 0x7f00981584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:48.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.836+0000 7f009ce12640 1 -- 172.21.15.84:0/1401557264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f009806c7d0 msgr2=0x7f00981584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:48.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.836+0000 7f009ce12640 1 --2- 172.21.15.84:0/1401557264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f009806c7d0 0x7f00981584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:48.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.836+0000 7f009ce12640 1 -- 172.21.15.84:0/1401557264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00980661c0 msgr2=0x7f0098160040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:48.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.836+0000 7f009ce12640 1 --2- 172.21.15.84:0/1401557264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00980661c0 0x7f0098160040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:48.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.836+0000 7f009ce12640 1 -- 172.21.15.84:0/1401557264 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00900094d0 con 0x7f009815d330 2023-10-16T09:49:48.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.836+0000 7f0097fff640 1 --2- 172.21.15.84:0/1401557264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00980661c0 0x7f0098160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:48.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.836+0000 7f00977fe640 1 --2- 172.21.15.84:0/1401557264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f009806c7d0 0x7f00981584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:48.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.837+0000 7f009ce12640 1 --2- 172.21.15.84:0/1401557264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f009815d330 0x7f00981589e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f00900090b0 tx=0x7f00900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:48.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.837+0000 7f0094ff9640 1 -- 172.21.15.84:0/1401557264 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009003ac60 con 0x7f009815d330 2023-10-16T09:49:48.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.837+0000 7f0094ff9640 1 -- 172.21.15.84:0/1401557264 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009003adc0 con 0x7f009815d330 2023-10-16T09:49:48.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.837+0000 7f0094ff9640 1 -- 172.21.15.84:0/1401557264 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0090044630 con 0x7f009815d330 2023-10-16T09:49:48.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.837+0000 7f009e875640 1 -- 172.21.15.84:0/1401557264 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00980c47a0 con 0x7f009815d330 2023-10-16T09:49:48.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.837+0000 7f009e875640 1 -- 172.21.15.84:0/1401557264 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00980c4ed0 con 0x7f009815d330 2023-10-16T09:49:48.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.838+0000 7f009e875640 1 -- 172.21.15.84:0/1401557264 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f009804e160 con 0x7f009815d330 2023-10-16T09:49:48.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.839+0000 7f0094ff9640 1 -- 172.21.15.84:0/1401557264 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0090044790 con 0x7f009815d330 2023-10-16T09:49:48.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.840+0000 7f0094ff9640 1 --2- 172.21.15.84:0/1401557264 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f007806ae70 0x7f007806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:48.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.840+0000 7f0094ff9640 1 -- 172.21.15.84:0/1401557264 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f009002d040 con 0x7f009815d330 2023-10-16T09:49:48.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.840+0000 7f0097fff640 1 --2- 172.21.15.84:0/1401557264 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f007806ae70 0x7f007806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:48.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.841+0000 7f0097fff640 1 --2- 172.21.15.84:0/1401557264 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f007806ae70 0x7f007806d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f009815e330 tx=0x7f008800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:48.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:48.845+0000 7f0094ff9640 1 -- 172.21.15.84:0/1401557264 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0090076cd0 con 0x7f009815d330 2023-10-16T09:49:49.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.040+0000 7f009e875640 1 -- 172.21.15.84:0/1401557264 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f009806f3a0 con 0x7f009815d330 2023-10-16T09:49:49.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.041+0000 7f0094ff9640 1 -- 172.21.15.84:0/1401557264 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0090035070 con 0x7f009815d330 2023-10-16T09:49:49.042 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T09:49:49.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.044+0000 7f009e875640 1 -- 172.21.15.84:0/1401557264 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f007806ae70 msgr2=0x7f007806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:49.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.044+0000 7f009e875640 1 --2- 172.21.15.84:0/1401557264 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f007806ae70 0x7f007806d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f009815e330 tx=0x7f008800b040 comp rx=0 tx=0).stop 2023-10-16T09:49:49.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.044+0000 7f009e875640 1 -- 172.21.15.84:0/1401557264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f009815d330 msgr2=0x7f00981589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:49.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.044+0000 7f009e875640 1 --2- 172.21.15.84:0/1401557264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f009815d330 0x7f00981589e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f00900090b0 tx=0x7f00900046c0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.044+0000 7f009e875640 1 -- 172.21.15.84:0/1401557264 shutdown_connections 2023-10-16T09:49:49.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.044+0000 7f009e875640 1 --2- 172.21.15.84:0/1401557264 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f007806ae70 0x7f007806d330 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.044+0000 7f009e875640 1 --2- 172.21.15.84:0/1401557264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f009815d330 0x7f00981589e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.044+0000 7f009e875640 1 --2- 172.21.15.84:0/1401557264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f009806c7d0 0x7f00981584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.044+0000 7f009e875640 1 --2- 172.21.15.84:0/1401557264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00980661c0 0x7f0098160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.044+0000 7f009e875640 1 -- 172.21.15.84:0/1401557264 >> 172.21.15.84:0/1401557264 conn(0x7f00980d0ab0 msgr2=0x7f0098159a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:49.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.044+0000 7f009e875640 1 -- 172.21.15.84:0/1401557264 shutdown_connections 2023-10-16T09:49:49.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.044+0000 7f009e875640 1 -- 172.21.15.84:0/1401557264 wait complete. 2023-10-16T09:49:49.065 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:49:49.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.279+0000 7f4885733640 1 Processor -- start 2023-10-16T09:49:49.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.280+0000 7f4885733640 1 -- start start 2023-10-16T09:49:49.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.281+0000 7f4885733640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f488015ad70 0x7f4880069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:49.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.281+0000 7f4885733640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f48800caac0 0x7f488006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:49.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.282+0000 7f4885733640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48800cb030 con 0x7f48800c9ca0 2023-10-16T09:49:49.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.282+0000 7f4885733640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48800c91c0 con 0x7f488015ad70 2023-10-16T09:49:49.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.282+0000 7f4885733640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48800c94a0 con 0x7f48800caac0 2023-10-16T09:49:49.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.282+0000 7f487effd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f488015ad70 0x7f4880069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:49.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.282+0000 7f487effd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f488015ad70 0x7f4880069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59802/0 (socket says 172.21.15.84:59802) 2023-10-16T09:49:49.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.282+0000 7f487effd640 1 -- 172.21.15.84:0/1671658691 learned_addr learned my addr 172.21.15.84:0/1671658691 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:49.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.282+0000 7f48767fc640 1 --1- 172.21.15.84:0/1671658691 >> v1:172.21.15.84:6789/0 conn(0x7f48800c9ca0 0x7f48800ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47152/0 (socket says 172.21.15.84:47152) 2023-10-16T09:49:49.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.282+0000 7f487f7fe640 1 --2- 172.21.15.84:0/1671658691 >> v2:172.21.15.157:3300/0 conn(0x7f48800caac0 0x7f488006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:49.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.282+0000 7f487effd640 1 -- 172.21.15.84:0/1671658691 >> v2:172.21.15.157:3300/0 conn(0x7f48800caac0 msgr2=0x7f488006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:49.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.282+0000 7f487effd640 1 --2- 172.21.15.84:0/1671658691 >> v2:172.21.15.157:3300/0 conn(0x7f48800caac0 0x7f488006a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.282+0000 7f487effd640 1 -- 172.21.15.84:0/1671658691 >> v1:172.21.15.84:6789/0 conn(0x7f48800c9ca0 legacy=0x7f48800ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:49.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.282+0000 7f487effd640 1 -- 172.21.15.84:0/1671658691 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48800c9780 con 0x7f488015ad70 2023-10-16T09:49:49.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.282+0000 7f487f7fe640 1 --2- 172.21.15.84:0/1671658691 >> v2:172.21.15.157:3300/0 conn(0x7f48800caac0 0x7f488006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:49.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.283+0000 7f487effd640 1 --2- 172.21.15.84:0/1671658691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f488015ad70 0x7f4880069da0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f488006d090 tx=0x7f487002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bd557881780751 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:49.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.284+0000 7f487e7fc640 1 -- 172.21.15.84:0/1671658691 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487002cb10 con 0x7f488015ad70 2023-10-16T09:49:49.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.284+0000 7f487e7fc640 1 -- 172.21.15.84:0/1671658691 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487002cc70 con 0x7f488015ad70 2023-10-16T09:49:49.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.284+0000 7f487e7fc640 1 -- 172.21.15.84:0/1671658691 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48700343f0 con 0x7f488015ad70 2023-10-16T09:49:49.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.284+0000 7f4885733640 1 -- 172.21.15.84:0/1671658691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f488015ad70 msgr2=0x7f4880069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:49.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.284+0000 7f4885733640 1 --2- 172.21.15.84:0/1671658691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f488015ad70 0x7f4880069da0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f488006d090 tx=0x7f487002bf70 comp rx=0 tx=0).stop 2023-10-16T09:49:49.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.284+0000 7f4885733640 1 -- 172.21.15.84:0/1671658691 shutdown_connections 2023-10-16T09:49:49.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.284+0000 7f4885733640 1 --2- 172.21.15.84:0/1671658691 >> v2:172.21.15.157:3300/0 conn(0x7f48800caac0 0x7f488006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.284+0000 7f4885733640 1 --2- 172.21.15.84:0/1671658691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f488015ad70 0x7f4880069da0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.284+0000 7f4885733640 1 -- 172.21.15.84:0/1671658691 >> 172.21.15.84:0/1671658691 conn(0x7f48800ce9b0 msgr2=0x7f48800cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:49.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.285+0000 7f4885733640 1 -- 172.21.15.84:0/1671658691 shutdown_connections 2023-10-16T09:49:49.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.285+0000 7f4885733640 1 -- 172.21.15.84:0/1671658691 wait complete. 2023-10-16T09:49:49.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.286+0000 7f4885733640 1 Processor -- start 2023-10-16T09:49:49.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.286+0000 7f4885733640 1 -- start start 2023-10-16T09:49:49.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.287+0000 7f4885733640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48800c9ca0 0x7f48800c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:49.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.287+0000 7f4885733640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48800caac0 0x7f48801576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:49.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.287+0000 7f4885733640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488015ad70 0x7f48800ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:49.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.287+0000 7f4885733640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488015d310 con 0x7f48800caac0 2023-10-16T09:49:49.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.287+0000 7f4885733640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48800c6200 con 0x7f488015ad70 2023-10-16T09:49:49.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.287+0000 7f4885733640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4880157c10 con 0x7f48800c9ca0 2023-10-16T09:49:49.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.288+0000 7f48767fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48800caac0 0x7f48801576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:49.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.288+0000 7f48767fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48800caac0 0x7f48801576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59814/0 (socket says 172.21.15.84:59814) 2023-10-16T09:49:49.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.288+0000 7f48767fc640 1 -- 172.21.15.84:0/595792623 learned_addr learned my addr 172.21.15.84:0/595792623 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:49.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.288+0000 7f487f7fe640 1 --2- 172.21.15.84:0/595792623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488015ad70 0x7f48800ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:49.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.288+0000 7f48767fc640 1 -- 172.21.15.84:0/595792623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48800c9ca0 msgr2=0x7f48800c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:49.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.288+0000 7f48767fc640 1 --2- 172.21.15.84:0/595792623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48800c9ca0 0x7f48800c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.288+0000 7f48767fc640 1 -- 172.21.15.84:0/595792623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488015ad70 msgr2=0x7f48800ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:49.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.288+0000 7f48767fc640 1 --2- 172.21.15.84:0/595792623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488015ad70 0x7f48800ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.288+0000 7f48767fc640 1 -- 172.21.15.84:0/595792623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4870009b30 con 0x7f48800caac0 2023-10-16T09:49:49.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.288+0000 7f48767fc640 1 --2- 172.21.15.84:0/595792623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48800caac0 0x7f48801576d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f486800c4d0 tx=0x7f486800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:49.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.289+0000 7f4877fff640 1 -- 172.21.15.84:0/595792623 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868007480 con 0x7f48800caac0 2023-10-16T09:49:49.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.289+0000 7f4877fff640 1 -- 172.21.15.84:0/595792623 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48680075e0 con 0x7f48800caac0 2023-10-16T09:49:49.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.289+0000 7f4877fff640 1 -- 172.21.15.84:0/595792623 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48680049e0 con 0x7f48800caac0 2023-10-16T09:49:49.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.289+0000 7f4885733640 1 -- 172.21.15.84:0/595792623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488015cf70 con 0x7f48800caac0 2023-10-16T09:49:49.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.289+0000 7f4885733640 1 -- 172.21.15.84:0/595792623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488015b4f0 con 0x7f48800caac0 2023-10-16T09:49:49.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.290+0000 7f4885733640 1 -- 172.21.15.84:0/595792623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48800ad490 con 0x7f48800caac0 2023-10-16T09:49:49.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.291+0000 7f4877fff640 1 -- 172.21.15.84:0/595792623 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4868004b40 con 0x7f48800caac0 2023-10-16T09:49:49.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.292+0000 7f4877fff640 1 --2- 172.21.15.84:0/595792623 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f486006ae70 0x7f486006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:49.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.292+0000 7f4877fff640 1 -- 172.21.15.84:0/595792623 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4868088810 con 0x7f48800caac0 2023-10-16T09:49:49.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.292+0000 7f487effd640 1 --2- 172.21.15.84:0/595792623 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f486006ae70 0x7f486006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:49.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.293+0000 7f487effd640 1 --2- 172.21.15.84:0/595792623 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f486006ae70 0x7f486006d330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4870035000 tx=0x7f487002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:49.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.297+0000 7f4877fff640 1 -- 172.21.15.84:0/595792623 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4868053850 con 0x7f48800caac0 2023-10-16T09:49:49.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.489+0000 7f4885733640 1 -- 172.21.15.84:0/595792623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f488015d450 con 0x7f48800caac0 2023-10-16T09:49:49.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.490+0000 7f4877fff640 1 -- 172.21.15.84:0/595792623 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4868018020 con 0x7f48800caac0 2023-10-16T09:49:49.492 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T09:49:49.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.493+0000 7f4885733640 1 -- 172.21.15.84:0/595792623 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f486006ae70 msgr2=0x7f486006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:49.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.493+0000 7f4885733640 1 --2- 172.21.15.84:0/595792623 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f486006ae70 0x7f486006d330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4870035000 tx=0x7f487002e000 comp rx=0 tx=0).stop 2023-10-16T09:49:49.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.493+0000 7f4885733640 1 -- 172.21.15.84:0/595792623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48800caac0 msgr2=0x7f48801576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.493+0000 7f4885733640 1 --2- 172.21.15.84:0/595792623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48800caac0 0x7f48801576d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f486800c4d0 tx=0x7f486800c990 comp rx=0 tx=0).stop 2023-10-16T09:49:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.493+0000 7f4885733640 1 -- 172.21.15.84:0/595792623 shutdown_connections 2023-10-16T09:49:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.493+0000 7f4885733640 1 --2- 172.21.15.84:0/595792623 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f486006ae70 0x7f486006d330 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.493+0000 7f4885733640 1 --2- 172.21.15.84:0/595792623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488015ad70 0x7f48800ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.493+0000 7f4885733640 1 --2- 172.21.15.84:0/595792623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48800caac0 0x7f48801576d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.493+0000 7f4885733640 1 --2- 172.21.15.84:0/595792623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48800c9ca0 0x7f48800c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.493+0000 7f4885733640 1 -- 172.21.15.84:0/595792623 >> 172.21.15.84:0/595792623 conn(0x7f48800ce9b0 msgr2=0x7f48800afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:49.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.493+0000 7f4885733640 1 -- 172.21.15.84:0/595792623 shutdown_connections 2023-10-16T09:49:49.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.493+0000 7f4885733640 1 -- 172.21.15.84:0/595792623 wait complete. 2023-10-16T09:49:49.513 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:49:49.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.722+0000 7f3b107d5640 1 Processor -- start 2023-10-16T09:49:49.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.723+0000 7f3b107d5640 1 -- start start 2023-10-16T09:49:49.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.724+0000 7f3b107d5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3b0806a4b0 0x7f3b080ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:49.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.724+0000 7f3b107d5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b080c9ca0 0x7f3b080ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:49.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.724+0000 7f3b107d5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b080cb030 con 0x7f3b0815ad70 2023-10-16T09:49:49.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.724+0000 7f3b107d5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b080cae30 con 0x7f3b0806a4b0 2023-10-16T09:49:49.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.724+0000 7f3b107d5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b080c91c0 con 0x7f3b080c9ca0 2023-10-16T09:49:49.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.725+0000 7f3b0dd70640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3b0806a4b0 0x7f3b080ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:49.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.725+0000 7f3b0dd70640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3b0806a4b0 0x7f3b080ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59816/0 (socket says 172.21.15.84:59816) 2023-10-16T09:49:49.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.725+0000 7f3b0dd70640 1 -- 172.21.15.84:0/968870567 learned_addr learned my addr 172.21.15.84:0/968870567 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:49.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.725+0000 7f3b0ed72640 1 --2- 172.21.15.84:0/968870567 >> v2:172.21.15.157:3300/0 conn(0x7f3b080c9ca0 0x7f3b080ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:49.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.725+0000 7f3b0dd70640 1 -- 172.21.15.84:0/968870567 >> v2:172.21.15.157:3300/0 conn(0x7f3b080c9ca0 msgr2=0x7f3b080ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:49.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.725+0000 7f3b0dd70640 1 --2- 172.21.15.84:0/968870567 >> v2:172.21.15.157:3300/0 conn(0x7f3b080c9ca0 0x7f3b080ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.725+0000 7f3b0dd70640 1 -- 172.21.15.84:0/968870567 >> v1:172.21.15.157:6789/0 conn(0x7f3b0815ad70 legacy=0x7f3b08069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:49.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.725+0000 7f3b0dd70640 1 -- 172.21.15.84:0/968870567 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b080c94a0 con 0x7f3b0806a4b0 2023-10-16T09:49:49.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.725+0000 7f3b0ed72640 1 --2- 172.21.15.84:0/968870567 >> v2:172.21.15.157:3300/0 conn(0x7f3b080c9ca0 0x7f3b080ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:49.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.726+0000 7f3b0dd70640 1 --2- 172.21.15.84:0/968870567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b0806a4b0 0x7f3b080ca3b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3b0806f270 tx=0x7f3af802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9d3aa746771848 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:49.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.727+0000 7f3b0d56f640 1 -- 172.21.15.84:0/968870567 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8031870 con 0x7f3b0806a4b0 2023-10-16T09:49:49.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.727+0000 7f3b0d56f640 1 -- 172.21.15.84:0/968870567 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af80319d0 con 0x7f3b0806a4b0 2023-10-16T09:49:49.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.727+0000 7f3b0d56f640 1 -- 172.21.15.84:0/968870567 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8031d60 con 0x7f3b0806a4b0 2023-10-16T09:49:49.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.727+0000 7f3b107d5640 1 -- 172.21.15.84:0/968870567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b0806a4b0 msgr2=0x7f3b080ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:49.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.727+0000 7f3b107d5640 1 --2- 172.21.15.84:0/968870567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b0806a4b0 0x7f3b080ca3b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3b0806f270 tx=0x7f3af802b910 comp rx=0 tx=0).stop 2023-10-16T09:49:49.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.728+0000 7f3b107d5640 1 -- 172.21.15.84:0/968870567 shutdown_connections 2023-10-16T09:49:49.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.728+0000 7f3b107d5640 1 --2- 172.21.15.84:0/968870567 >> v2:172.21.15.157:3300/0 conn(0x7f3b080c9ca0 0x7f3b080ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.728+0000 7f3b107d5640 1 --2- 172.21.15.84:0/968870567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b0806a4b0 0x7f3b080ca3b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.728+0000 7f3b107d5640 1 -- 172.21.15.84:0/968870567 >> 172.21.15.84:0/968870567 conn(0x7f3b080ce9b0 msgr2=0x7f3b080cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:49.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.728+0000 7f3b107d5640 1 -- 172.21.15.84:0/968870567 shutdown_connections 2023-10-16T09:49:49.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.728+0000 7f3b107d5640 1 -- 172.21.15.84:0/968870567 wait complete. 2023-10-16T09:49:49.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.729+0000 7f3b107d5640 1 Processor -- start 2023-10-16T09:49:49.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.730+0000 7f3b107d5640 1 -- start start 2023-10-16T09:49:49.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.731+0000 7f3b107d5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b0806a4b0 0x7f3b081f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:49.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.731+0000 7f3b107d5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b080c9ca0 0x7f3b08158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:49.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.731+0000 7f3b107d5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b0815ad70 0x7f3b080aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:49.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.731+0000 7f3b107d5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b081f69a0 con 0x7f3b080c9ca0 2023-10-16T09:49:49.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.731+0000 7f3b107d5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b080acac0 con 0x7f3b0815ad70 2023-10-16T09:49:49.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.731+0000 7f3b107d5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b08159330 con 0x7f3b0806a4b0 2023-10-16T09:49:49.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.731+0000 7f3b0dd70640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b080c9ca0 0x7f3b08158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:49.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.731+0000 7f3b0dd70640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b080c9ca0 0x7f3b08158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59832/0 (socket says 172.21.15.84:59832) 2023-10-16T09:49:49.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.731+0000 7f3b0dd70640 1 -- 172.21.15.84:0/403113200 learned_addr learned my addr 172.21.15.84:0/403113200 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:49.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.731+0000 7f3b0dd70640 1 -- 172.21.15.84:0/403113200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b0806a4b0 msgr2=0x7f3b081f5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:49.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.731+0000 7f3b0e571640 1 --2- 172.21.15.84:0/403113200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b0806a4b0 0x7f3b081f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:49.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.731+0000 7f3b0dd70640 1 --2- 172.21.15.84:0/403113200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b0806a4b0 0x7f3b081f5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.731+0000 7f3b0ed72640 1 --2- 172.21.15.84:0/403113200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b0815ad70 0x7f3b080aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:49.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.731+0000 7f3b0dd70640 1 -- 172.21.15.84:0/403113200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b0815ad70 msgr2=0x7f3b080aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:49.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.731+0000 7f3b0dd70640 1 --2- 172.21.15.84:0/403113200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b0815ad70 0x7f3b080aebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.731+0000 7f3b0dd70640 1 -- 172.21.15.84:0/403113200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af80094d0 con 0x7f3b080c9ca0 2023-10-16T09:49:49.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.731+0000 7f3b0e571640 1 --2- 172.21.15.84:0/403113200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b0806a4b0 0x7f3b081f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:49.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.732+0000 7f3b0dd70640 1 --2- 172.21.15.84:0/403113200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b080c9ca0 0x7f3b08158870 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3af8000c00 tx=0x7f3af80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:49.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.732+0000 7f3afeffd640 1 -- 172.21.15.84:0/403113200 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8031840 con 0x7f3b080c9ca0 2023-10-16T09:49:49.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.732+0000 7f3afeffd640 1 -- 172.21.15.84:0/403113200 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af80443f0 con 0x7f3b080c9ca0 2023-10-16T09:49:49.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.732+0000 7f3afeffd640 1 -- 172.21.15.84:0/403113200 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af80447a0 con 0x7f3b080c9ca0 2023-10-16T09:49:49.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.732+0000 7f3b107d5640 1 -- 172.21.15.84:0/403113200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b08158db0 con 0x7f3b080c9ca0 2023-10-16T09:49:49.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.732+0000 7f3b107d5640 1 -- 172.21.15.84:0/403113200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b081f63e0 con 0x7f3b080c9ca0 2023-10-16T09:49:49.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.733+0000 7f3b107d5640 1 -- 172.21.15.84:0/403113200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b080ac890 con 0x7f3b080c9ca0 2023-10-16T09:49:49.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.734+0000 7f3afeffd640 1 -- 172.21.15.84:0/403113200 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3af804d450 con 0x7f3b080c9ca0 2023-10-16T09:49:49.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.735+0000 7f3afeffd640 1 --2- 172.21.15.84:0/403113200 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3aec06aec0 0x7f3aec06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:49.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.735+0000 7f3afeffd640 1 -- 172.21.15.84:0/403113200 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3af802d040 con 0x7f3b080c9ca0 2023-10-16T09:49:49.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.735+0000 7f3b0e571640 1 --2- 172.21.15.84:0/403113200 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3aec06aec0 0x7f3aec06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:49.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.736+0000 7f3b0e571640 1 --2- 172.21.15.84:0/403113200 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3aec06aec0 0x7f3aec06d380 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400a830 tx=0x7f3b04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:49.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.740+0000 7f3afeffd640 1 -- 172.21.15.84:0/403113200 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3af8076f00 con 0x7f3b080c9ca0 2023-10-16T09:49:49.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.935+0000 7f3b107d5640 1 -- 172.21.15.84:0/403113200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3b080acc30 con 0x7f3b080c9ca0 2023-10-16T09:49:49.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.936+0000 7f3afeffd640 1 -- 172.21.15.84:0/403113200 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3af8035020 con 0x7f3b080c9ca0 2023-10-16T09:49:49.938 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T09:49:49.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.939+0000 7f3b107d5640 1 -- 172.21.15.84:0/403113200 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3aec06aec0 msgr2=0x7f3aec06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:49.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.939+0000 7f3b107d5640 1 --2- 172.21.15.84:0/403113200 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3aec06aec0 0x7f3aec06d380 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400a830 tx=0x7f3b04008040 comp rx=0 tx=0).stop 2023-10-16T09:49:49.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.939+0000 7f3b107d5640 1 -- 172.21.15.84:0/403113200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b080c9ca0 msgr2=0x7f3b08158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:49.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.939+0000 7f3b107d5640 1 --2- 172.21.15.84:0/403113200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b080c9ca0 0x7f3b08158870 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3af8000c00 tx=0x7f3af80046c0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.939+0000 7f3b107d5640 1 -- 172.21.15.84:0/403113200 shutdown_connections 2023-10-16T09:49:49.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.939+0000 7f3b107d5640 1 --2- 172.21.15.84:0/403113200 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3aec06aec0 0x7f3aec06d380 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.939+0000 7f3b107d5640 1 --2- 172.21.15.84:0/403113200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b0815ad70 0x7f3b080aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.939+0000 7f3b107d5640 1 --2- 172.21.15.84:0/403113200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b080c9ca0 0x7f3b08158870 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.939+0000 7f3b107d5640 1 --2- 172.21.15.84:0/403113200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b0806a4b0 0x7f3b081f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:49.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.939+0000 7f3b107d5640 1 -- 172.21.15.84:0/403113200 >> 172.21.15.84:0/403113200 conn(0x7f3b080ce9b0 msgr2=0x7f3b080cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:49.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.939+0000 7f3b107d5640 1 -- 172.21.15.84:0/403113200 shutdown_connections 2023-10-16T09:49:49.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:49.939+0000 7f3b107d5640 1 -- 172.21.15.84:0/403113200 wait complete. 2023-10-16T09:49:49.960 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T09:49:50.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.173+0000 7fae0e4e6640 1 Processor -- start 2023-10-16T09:49:50.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.173+0000 7fae0e4e6640 1 -- start start 2023-10-16T09:49:50.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.175+0000 7fae0e4e6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae0806c040 0x7fae0806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:50.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.175+0000 7fae0e4e6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fae080ad970 0x7fae0815e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:50.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.175+0000 7fae0e4e6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0806c960 con 0x7fae0815ebc0 2023-10-16T09:49:50.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.175+0000 7fae0e4e6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0815b9d0 con 0x7fae080ad970 2023-10-16T09:49:50.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.175+0000 7fae0e4e6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0815bcb0 con 0x7fae0806c040 2023-10-16T09:49:50.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.175+0000 7fae0dce5640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fae0815ebc0 0x7fae080cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47154/0 (socket says 172.21.15.84:47154) 2023-10-16T09:49:50.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.175+0000 7fae0dce5640 1 -- 172.21.15.84:0/1521787455 learned_addr learned my addr 172.21.15.84:0/1521787455 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:50.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.175+0000 7fae0d4e4640 1 --2- 172.21.15.84:0/1521787455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae0806c040 0x7fae0806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:50.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.175+0000 7fae0cce3640 1 --2- 172.21.15.84:0/1521787455 >> v2:172.21.15.157:3300/0 conn(0x7fae080ad970 0x7fae0815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:50.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.175+0000 7fadfffff640 1 -- 172.21.15.84:0/1521787455 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2721972313 0 0) 0x7fae0806c960 con 0x7fae0815ebc0 2023-10-16T09:49:50.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.176+0000 7fadfffff640 1 -- 172.21.15.84:0/1521787455 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadf4003520 con 0x7fae0815ebc0 2023-10-16T09:49:50.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.176+0000 7fadfffff640 1 -- 172.21.15.84:0/1521787455 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1891856159 0 0) 0x7fadf4003520 con 0x7fae0815ebc0 2023-10-16T09:49:50.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.176+0000 7fadfffff640 1 -- 172.21.15.84:0/1521787455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae0806c040 msgr2=0x7fae0806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:50.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.176+0000 7fadfffff640 1 --2- 172.21.15.84:0/1521787455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae0806c040 0x7fae0806c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:50.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.176+0000 7fadfffff640 1 -- 172.21.15.84:0/1521787455 >> v2:172.21.15.157:3300/0 conn(0x7fae080ad970 msgr2=0x7fae0815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:50.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.176+0000 7fadfffff640 1 --2- 172.21.15.84:0/1521787455 >> v2:172.21.15.157:3300/0 conn(0x7fae080ad970 0x7fae0815e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:50.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.176+0000 7fadfffff640 1 -- 172.21.15.84:0/1521787455 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0815bf90 con 0x7fae0815ebc0 2023-10-16T09:49:50.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.176+0000 7fadfffff640 1 -- 172.21.15.84:0/1521787455 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fae000019f0 con 0x7fae0815ebc0 2023-10-16T09:49:50.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.176+0000 7fae0cce3640 1 --2- 172.21.15.84:0/1521787455 >> v2:172.21.15.157:3300/0 conn(0x7fae080ad970 0x7fae0815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:49:50.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.176+0000 7fadfffff640 1 -- 172.21.15.84:0/1521787455 >> v1:172.21.15.84:6789/0 conn(0x7fae0815ebc0 legacy=0x7fae080cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:50.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.176+0000 7fadfffff640 1 --2- 172.21.15.84:0/1521787455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadf4003970 0x7fadf4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:50.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.176+0000 7fadfffff640 1 --2- 172.21.15.84:0/1521787455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae0806c040 0x7fadf40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:50.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.176+0000 7fadfffff640 1 --2- 172.21.15.84:0/1521787455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf4004d00 0x7fadf4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:50.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.176+0000 7fadfffff640 1 -- 172.21.15.84:0/1521787455 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0815b9d0 con 0x7fadf4004d00 2023-10-16T09:49:50.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.176+0000 7fadfffff640 1 -- 172.21.15.84:0/1521787455 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0815bcb0 con 0x7fadf4003970 2023-10-16T09:49:50.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.176+0000 7fadfffff640 1 -- 172.21.15.84:0/1521787455 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf400ba00 con 0x7fae0806c040 2023-10-16T09:49:50.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.176+0000 7fae0dce5640 1 --2- 172.21.15.84:0/1521787455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf4004d00 0x7fadf4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:50.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.177+0000 7fae0cce3640 1 --2- 172.21.15.84:0/1521787455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae0806c040 0x7fadf40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:50.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.177+0000 7fae0d4e4640 1 --2- 172.21.15.84:0/1521787455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadf4003970 0x7fadf4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:50.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.177+0000 7fae0dce5640 1 -- 172.21.15.84:0/1521787455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae0806c040 msgr2=0x7fadf40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:50.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.177+0000 7fae0dce5640 1 --2- 172.21.15.84:0/1521787455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae0806c040 0x7fadf40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:50.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.177+0000 7fae0dce5640 1 -- 172.21.15.84:0/1521787455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadf4003970 msgr2=0x7fadf4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:50.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.177+0000 7fae0dce5640 1 --2- 172.21.15.84:0/1521787455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadf4003970 0x7fadf4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:50.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.177+0000 7fae0dce5640 1 -- 172.21.15.84:0/1521787455 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf4003520 con 0x7fadf4004d00 2023-10-16T09:49:50.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.177+0000 7fae0d4e4640 1 --2- 172.21.15.84:0/1521787455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadf4003970 0x7fadf4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:50.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.178+0000 7fae0dce5640 1 --2- 172.21.15.84:0/1521787455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf4004d00 0x7fadf4005180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fae0000a760 tx=0x7fadf40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4503e51ef1963f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:50.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.178+0000 7fadfffff640 1 -- 172.21.15.84:0/1521787455 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae00034630 con 0x7fadf4004d00 2023-10-16T09:49:50.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.178+0000 7fadfffff640 1 -- 172.21.15.84:0/1521787455 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0002f070 con 0x7fadf4004d00 2023-10-16T09:49:50.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.178+0000 7fadfffff640 1 -- 172.21.15.84:0/1521787455 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae00034a10 con 0x7fadf4004d00 2023-10-16T09:49:50.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.179+0000 7fae0e4e6640 1 -- 172.21.15.84:0/1521787455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf4004d00 msgr2=0x7fadf4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:50.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.179+0000 7fae0e4e6640 1 --2- 172.21.15.84:0/1521787455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf4004d00 0x7fadf4005180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fae0000a760 tx=0x7fadf40057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:50.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.179+0000 7fae0dce5640 1 -- 172.21.15.84:0/1521787455 reap_dead start 2023-10-16T09:49:50.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.179+0000 7fae0e4e6640 1 -- 172.21.15.84:0/1521787455 shutdown_connections 2023-10-16T09:49:50.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.179+0000 7fae0e4e6640 1 -- 172.21.15.84:0/1521787455 >> 172.21.15.84:0/1521787455 conn(0x7fae080ce6a0 msgr2=0x7fae080ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:50.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.179+0000 7fae0e4e6640 1 -- 172.21.15.84:0/1521787455 shutdown_connections 2023-10-16T09:49:50.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.179+0000 7fae0e4e6640 1 -- 172.21.15.84:0/1521787455 wait complete. 2023-10-16T09:49:50.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.181+0000 7fae0e4e6640 1 Processor -- start 2023-10-16T09:49:50.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.181+0000 7fae0e4e6640 1 -- start start 2023-10-16T09:49:50.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.182+0000 7fae0e4e6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf4004d00 0x7fae0814c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:50.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.182+0000 7fae0e4e6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae08155040 0x7fae08154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:50.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.182+0000 7fae0e4e6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae080c7120 0x7fae080c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:50.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.182+0000 7fae0e4e6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae080c7ae0 con 0x7fadf4004d00 2023-10-16T09:49:50.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.182+0000 7fae0e4e6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0814c770 con 0x7fae08155040 2023-10-16T09:49:50.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.182+0000 7fae0e4e6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae081532a0 con 0x7fae080c7120 2023-10-16T09:49:50.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.182+0000 7fae0d4e4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf4004d00 0x7fae0814c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:50.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.182+0000 7fae0d4e4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf4004d00 0x7fae0814c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59850/0 (socket says 172.21.15.84:59850) 2023-10-16T09:49:50.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.182+0000 7fae0d4e4640 1 -- 172.21.15.84:0/1961843990 learned_addr learned my addr 172.21.15.84:0/1961843990 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:50.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.182+0000 7fae0d4e4640 1 -- 172.21.15.84:0/1961843990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae080c7120 msgr2=0x7fae080c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:50.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.182+0000 7fae0d4e4640 1 --2- 172.21.15.84:0/1961843990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae080c7120 0x7fae080c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:50.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.182+0000 7fae0d4e4640 1 -- 172.21.15.84:0/1961843990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae08155040 msgr2=0x7fae08154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:50.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.182+0000 7fae0d4e4640 1 --2- 172.21.15.84:0/1961843990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae08155040 0x7fae08154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:50.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.182+0000 7fae0d4e4640 1 -- 172.21.15.84:0/1961843990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae08153580 con 0x7fadf4004d00 2023-10-16T09:49:50.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.183+0000 7fae0d4e4640 1 --2- 172.21.15.84:0/1961843990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf4004d00 0x7fae0814c230 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fae04010890 tx=0x7fae04010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:50.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.183+0000 7fadfdffb640 1 -- 172.21.15.84:0/1961843990 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0400ea80 con 0x7fadf4004d00 2023-10-16T09:49:50.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.183+0000 7fadfdffb640 1 -- 172.21.15.84:0/1961843990 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0400ebe0 con 0x7fadf4004d00 2023-10-16T09:49:50.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.183+0000 7fadfdffb640 1 -- 172.21.15.84:0/1961843990 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0401c400 con 0x7fadf4004d00 2023-10-16T09:49:50.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.183+0000 7fae0e4e6640 1 -- 172.21.15.84:0/1961843990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae08153b00 con 0x7fadf4004d00 2023-10-16T09:49:50.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.183+0000 7fae0e4e6640 1 -- 172.21.15.84:0/1961843990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae08152f50 con 0x7fadf4004d00 2023-10-16T09:49:50.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.184+0000 7fae0e4e6640 1 -- 172.21.15.84:0/1961843990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0806b330 con 0x7fadf4004d00 2023-10-16T09:49:50.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.185+0000 7fadfdffb640 1 -- 172.21.15.84:0/1961843990 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fae0401c560 con 0x7fadf4004d00 2023-10-16T09:49:50.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.186+0000 7fadfdffb640 1 --2- 172.21.15.84:0/1961843990 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fadec06aea0 0x7fadec06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:50.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.186+0000 7fadfdffb640 1 -- 172.21.15.84:0/1961843990 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fae04089a30 con 0x7fadf4004d00 2023-10-16T09:49:50.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.186+0000 7fae0cce3640 1 --2- 172.21.15.84:0/1961843990 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fadec06aea0 0x7fadec06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:50.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.187+0000 7fae0cce3640 1 --2- 172.21.15.84:0/1961843990 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fadec06aea0 0x7fadec06d360 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fae080c7db0 tx=0x7fadf800c2d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:50.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.191+0000 7fadfdffb640 1 -- 172.21.15.84:0/1961843990 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fae04054ba0 con 0x7fadf4004d00 2023-10-16T09:49:50.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.395+0000 7fae0e4e6640 1 -- 172.21.15.84:0/1961843990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fae0804e9b0 con 0x7fadf4004d00 2023-10-16T09:49:50.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.396+0000 7fadfdffb640 1 -- 172.21.15.84:0/1961843990 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fae040581c0 con 0x7fadf4004d00 2023-10-16T09:49:50.398 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-16T09:49:50.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.399+0000 7fae0e4e6640 1 -- 172.21.15.84:0/1961843990 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fadec06aea0 msgr2=0x7fadec06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:50.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.399+0000 7fae0e4e6640 1 --2- 172.21.15.84:0/1961843990 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fadec06aea0 0x7fadec06d360 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fae080c7db0 tx=0x7fadf800c2d0 comp rx=0 tx=0).stop 2023-10-16T09:49:50.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.399+0000 7fae0e4e6640 1 -- 172.21.15.84:0/1961843990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf4004d00 msgr2=0x7fae0814c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:50.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.399+0000 7fae0e4e6640 1 --2- 172.21.15.84:0/1961843990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf4004d00 0x7fae0814c230 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fae04010890 tx=0x7fae04010d50 comp rx=0 tx=0).stop 2023-10-16T09:49:50.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.399+0000 7fae0e4e6640 1 -- 172.21.15.84:0/1961843990 shutdown_connections 2023-10-16T09:49:50.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.399+0000 7fae0e4e6640 1 --2- 172.21.15.84:0/1961843990 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fadec06aea0 0x7fadec06d360 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:50.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.399+0000 7fae0e4e6640 1 --2- 172.21.15.84:0/1961843990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae080c7120 0x7fae080c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:50.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.399+0000 7fae0e4e6640 1 --2- 172.21.15.84:0/1961843990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae08155040 0x7fae08154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:50.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.399+0000 7fae0e4e6640 1 --2- 172.21.15.84:0/1961843990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf4004d00 0x7fae0814c230 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:50.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.399+0000 7fae0e4e6640 1 -- 172.21.15.84:0/1961843990 >> 172.21.15.84:0/1961843990 conn(0x7fae080ce6a0 msgr2=0x7fae0815c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:50.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.399+0000 7fae0e4e6640 1 -- 172.21.15.84:0/1961843990 shutdown_connections 2023-10-16T09:49:50.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:50.399+0000 7fae0e4e6640 1 -- 172.21.15.84:0/1961843990 wait complete. 2023-10-16T09:49:50.418 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-16T09:49:50.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:50.419 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-16T09:49:50.525 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-10-16T09:49:50.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:50.526 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-16T09:49:50.582 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:49:50.582 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:49:50.642 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:49:50.642 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:49:50.693 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:49:50.693 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:49:50.790 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:51.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.005+0000 7fc7e9705640 1 Processor -- start 2023-10-16T09:49:51.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.005+0000 7fc7e9705640 1 -- start start 2023-10-16T09:49:51.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.007+0000 7fc7e9705640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7e406b4c0 0x7fc7e40ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.007+0000 7fc7e9705640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7e4160800 0x7fc7e40cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.007+0000 7fc7e9705640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e406e170 con 0x7fc7e4160420 2023-10-16T09:49:51.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.007+0000 7fc7e9705640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7e40cde30 con 0x7fc7e406b4c0 2023-10-16T09:49:51.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.007+0000 7fc7e9705640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e40cd460 con 0x7fc7e4160800 2023-10-16T09:49:51.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.007+0000 7fc7e37fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7e4160800 0x7fc7e40cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.008+0000 7fc7e2ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc7e4160420 0x7fc7e406adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:35630/0 (socket says 172.21.15.84:35630) 2023-10-16T09:49:51.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.008+0000 7fc7e2ffd640 1 -- 172.21.15.84:0/3456014378 learned_addr learned my addr 172.21.15.84:0/3456014378 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:51.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.008+0000 7fc7da7fc640 1 --2- 172.21.15.84:0/3456014378 >> v2:172.21.15.157:3300/0 conn(0x7fc7e406b4c0 0x7fc7e40ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.008+0000 7fc7e27fc640 1 -- 172.21.15.84:0/3456014378 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896794029 0 0) 0x7fc7e406e170 con 0x7fc7e4160420 2023-10-16T09:49:51.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.008+0000 7fc7e37fe640 1 -- 172.21.15.84:0/3456014378 >> v2:172.21.15.157:3300/0 conn(0x7fc7e406b4c0 msgr2=0x7fc7e40ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:51.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.008+0000 7fc7e37fe640 1 --2- 172.21.15.84:0/3456014378 >> v2:172.21.15.157:3300/0 conn(0x7fc7e406b4c0 0x7fc7e40ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.008+0000 7fc7e37fe640 1 -- 172.21.15.84:0/3456014378 >> v1:172.21.15.157:6789/0 conn(0x7fc7e4160420 legacy=0x7fc7e406adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:51.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.008+0000 7fc7e37fe640 1 -- 172.21.15.84:0/3456014378 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e406e3c0 con 0x7fc7e4160800 2023-10-16T09:49:51.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.010+0000 7fc7e37fe640 1 --2- 172.21.15.84:0/3456014378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7e4160800 0x7fc7e40cd8f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc009d90 tx=0x7fc7dc02beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f1861fbd7319e93 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:51.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.010+0000 7fc7e27fc640 1 -- 172.21.15.84:0/3456014378 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc02cc60 con 0x7fc7e4160800 2023-10-16T09:49:51.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.010+0000 7fc7e27fc640 1 -- 172.21.15.84:0/3456014378 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7dc02cdc0 con 0x7fc7e4160800 2023-10-16T09:49:51.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.010+0000 7fc7e27fc640 1 -- 172.21.15.84:0/3456014378 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc034660 con 0x7fc7e4160800 2023-10-16T09:49:51.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.011+0000 7fc7e9705640 1 -- 172.21.15.84:0/3456014378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7e4160800 msgr2=0x7fc7e40cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:51.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.011+0000 7fc7e9705640 1 --2- 172.21.15.84:0/3456014378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7e4160800 0x7fc7e40cd8f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc009d90 tx=0x7fc7dc02beb0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.011+0000 7fc7e9705640 1 -- 172.21.15.84:0/3456014378 shutdown_connections 2023-10-16T09:49:51.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.011+0000 7fc7e9705640 1 --2- 172.21.15.84:0/3456014378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7e4160800 0x7fc7e40cd8f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.011+0000 7fc7e9705640 1 --2- 172.21.15.84:0/3456014378 >> v2:172.21.15.157:3300/0 conn(0x7fc7e406b4c0 0x7fc7e40ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.011+0000 7fc7e9705640 1 -- 172.21.15.84:0/3456014378 >> 172.21.15.84:0/3456014378 conn(0x7fc7e40b0d30 msgr2=0x7fc7e40b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:51.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.011+0000 7fc7e9705640 1 -- 172.21.15.84:0/3456014378 shutdown_connections 2023-10-16T09:49:51.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.011+0000 7fc7e9705640 1 -- 172.21.15.84:0/3456014378 wait complete. 2023-10-16T09:49:51.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.013+0000 7fc7e9705640 1 Processor -- start 2023-10-16T09:49:51.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.013+0000 7fc7e9705640 1 -- start start 2023-10-16T09:49:51.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.014+0000 7fc7e9705640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7e406b4c0 0x7fc7e41f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.014+0000 7fc7e9705640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7e4160420 0x7fc7e41f2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.014+0000 7fc7e9705640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7e4160800 0x7fc7e409e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.014+0000 7fc7e9705640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e409ecd0 con 0x7fc7e4160420 2023-10-16T09:49:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.014+0000 7fc7e9705640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e41f27a0 con 0x7fc7e406b4c0 2023-10-16T09:49:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.014+0000 7fc7e9705640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e409e9f0 con 0x7fc7e4160800 2023-10-16T09:49:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.014+0000 7fc7da7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7e4160420 0x7fc7e41f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.014+0000 7fc7da7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7e4160420 0x7fc7e41f2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59866/0 (socket says 172.21.15.84:59866) 2023-10-16T09:49:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.014+0000 7fc7da7fc640 1 -- 172.21.15.84:0/141144274 learned_addr learned my addr 172.21.15.84:0/141144274 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.015+0000 7fc7e2ffd640 1 --2- 172.21.15.84:0/141144274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7e406b4c0 0x7fc7e41f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.015+0000 7fc7e37fe640 1 --2- 172.21.15.84:0/141144274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7e4160800 0x7fc7e409e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.015+0000 7fc7da7fc640 1 -- 172.21.15.84:0/141144274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7e4160800 msgr2=0x7fc7e409e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:51.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.015+0000 7fc7da7fc640 1 --2- 172.21.15.84:0/141144274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7e4160800 0x7fc7e409e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.015+0000 7fc7da7fc640 1 -- 172.21.15.84:0/141144274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7e406b4c0 msgr2=0x7fc7e41f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:51.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.015+0000 7fc7da7fc640 1 --2- 172.21.15.84:0/141144274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7e406b4c0 0x7fc7e41f3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.015+0000 7fc7da7fc640 1 -- 172.21.15.84:0/141144274 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7dc009a70 con 0x7fc7e4160420 2023-10-16T09:49:51.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.015+0000 7fc7e2ffd640 1 --2- 172.21.15.84:0/141144274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7e406b4c0 0x7fc7e41f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:51.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.015+0000 7fc7da7fc640 1 --2- 172.21.15.84:0/141144274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7e4160420 0x7fc7e41f2260 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00e9b0 tx=0x7fc7cc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:51.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.015+0000 7fc7dbfff640 1 -- 172.21.15.84:0/141144274 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc009670 con 0x7fc7e4160420 2023-10-16T09:49:51.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.015+0000 7fc7dbfff640 1 -- 172.21.15.84:0/141144274 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc00c9a0 con 0x7fc7e4160420 2023-10-16T09:49:51.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.015+0000 7fc7dbfff640 1 -- 172.21.15.84:0/141144274 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc00cd50 con 0x7fc7e4160420 2023-10-16T09:49:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.015+0000 7fc7e9705640 1 -- 172.21.15.84:0/141144274 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e40c7990 con 0x7fc7e4160420 2023-10-16T09:49:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.015+0000 7fc7e9705640 1 -- 172.21.15.84:0/141144274 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e40c7f50 con 0x7fc7e4160420 2023-10-16T09:49:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.016+0000 7fc7e9705640 1 -- 172.21.15.84:0/141144274 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e40c77f0 con 0x7fc7e4160420 2023-10-16T09:49:51.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.017+0000 7fc7dbfff640 1 -- 172.21.15.84:0/141144274 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc7cc013470 con 0x7fc7e4160420 2023-10-16T09:49:51.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.018+0000 7fc7dbfff640 1 --2- 172.21.15.84:0/141144274 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc7c406ae70 0x7fc7c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.018+0000 7fc7dbfff640 1 -- 172.21.15.84:0/141144274 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc7cc089810 con 0x7fc7e4160420 2023-10-16T09:49:51.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.019+0000 7fc7e2ffd640 1 --2- 172.21.15.84:0/141144274 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc7c406ae70 0x7fc7c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.019+0000 7fc7e2ffd640 1 --2- 172.21.15.84:0/141144274 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc7c406ae70 0x7fc7c406d330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4000f60 tx=0x7fc7d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:51.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.023+0000 7fc7dbfff640 1 -- 172.21.15.84:0/141144274 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc7e40c77f0 con 0x7fc7e4160420 2023-10-16T09:49:51.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.217+0000 7fc7e9705640 1 -- 172.21.15.84:0/141144274 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7e40c77f0 con 0x7fc7e4160420 2023-10-16T09:49:51.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.220+0000 7fc7dbfff640 1 -- 172.21.15.84:0/141144274 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16730 (secure 0 0 0) 0x7fc7e40c77f0 con 0x7fc7e4160420 2023-10-16T09:49:51.222 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:51.223 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":23,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:49.552121+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.27","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:51.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.222+0000 7fc7e9705640 1 -- 172.21.15.84:0/141144274 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc7c406ae70 msgr2=0x7fc7c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:51.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.222+0000 7fc7e9705640 1 --2- 172.21.15.84:0/141144274 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc7c406ae70 0x7fc7c406d330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4000f60 tx=0x7fc7d4009040 comp rx=0 tx=0).stop 2023-10-16T09:49:51.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.222+0000 7fc7e9705640 1 -- 172.21.15.84:0/141144274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7e4160420 msgr2=0x7fc7e41f2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:51.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.222+0000 7fc7e9705640 1 --2- 172.21.15.84:0/141144274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7e4160420 0x7fc7e41f2260 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00e9b0 tx=0x7fc7cc00ee70 comp rx=0 tx=0).stop 2023-10-16T09:49:51.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.222+0000 7fc7e9705640 1 -- 172.21.15.84:0/141144274 shutdown_connections 2023-10-16T09:49:51.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.222+0000 7fc7e9705640 1 --2- 172.21.15.84:0/141144274 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc7c406ae70 0x7fc7c406d330 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.222+0000 7fc7e9705640 1 --2- 172.21.15.84:0/141144274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7e4160800 0x7fc7e409e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.222+0000 7fc7e9705640 1 --2- 172.21.15.84:0/141144274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7e4160420 0x7fc7e41f2260 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.223+0000 7fc7e9705640 1 --2- 172.21.15.84:0/141144274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7e406b4c0 0x7fc7e41f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.223+0000 7fc7e9705640 1 -- 172.21.15.84:0/141144274 >> 172.21.15.84:0/141144274 conn(0x7fc7e40b0d30 msgr2=0x7fc7e40b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:51.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.223+0000 7fc7e9705640 1 -- 172.21.15.84:0/141144274 shutdown_connections 2023-10-16T09:49:51.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.223+0000 7fc7e9705640 1 -- 172.21.15.84:0/141144274 wait complete. 2023-10-16T09:49:51.242 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:49:00.048649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:49:01.361365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:49:51.243 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:49:51.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.459+0000 7f3fc359e640 1 Processor -- start 2023-10-16T09:49:51.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.460+0000 7f3fc359e640 1 -- start start 2023-10-16T09:49:51.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.461+0000 7f3fc359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fc40b2080 0x7f3fc40d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.461+0000 7f3fc359e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc40d0970 con 0x7f3fc40cfc40 2023-10-16T09:49:51.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.461+0000 7f3fc359e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc40d0ea0 con 0x7f3fc40b1ca0 2023-10-16T09:49:51.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.461+0000 7f3fc359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc40d0bc0 con 0x7f3fc40b2080 2023-10-16T09:49:51.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.461+0000 7f3fc2d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3fc40cfc40 0x7f3fc40b24c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47162/0 (socket says 172.21.15.84:47162) 2023-10-16T09:49:51.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.461+0000 7f3fc2d9d640 1 -- 172.21.15.84:0/4214609939 learned_addr learned my addr 172.21.15.84:0/4214609939 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:51.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.462+0000 7f3fc1d9b640 1 --2- 172.21.15.84:0/4214609939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fc40b2080 0x7f3fc40d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.462+0000 7f3fc159a640 1 -- 172.21.15.84:0/4214609939 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259523011 0 0) 0x7f3fc40d0970 con 0x7f3fc40cfc40 2023-10-16T09:49:51.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.462+0000 7f3fc159a640 1 -- 172.21.15.84:0/4214609939 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fb0003520 con 0x7f3fc40cfc40 2023-10-16T09:49:51.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.462+0000 7f3fc159a640 1 -- 172.21.15.84:0/4214609939 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2038401669 0 0) 0x7f3fb0003520 con 0x7f3fc40cfc40 2023-10-16T09:49:51.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.462+0000 7f3fc159a640 1 -- 172.21.15.84:0/4214609939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fc40b2080 msgr2=0x7f3fc40d36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:51.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.462+0000 7f3fc159a640 1 --2- 172.21.15.84:0/4214609939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fc40b2080 0x7f3fc40d36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.462+0000 7f3fc159a640 1 -- 172.21.15.84:0/4214609939 >> v1:172.21.15.157:6789/0 conn(0x7f3fc40b1ca0 legacy=0x7f3fc40ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:51.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.462+0000 7f3fc159a640 1 -- 172.21.15.84:0/4214609939 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc40cd940 con 0x7f3fc40cfc40 2023-10-16T09:49:51.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.462+0000 7f3fc159a640 1 -- 172.21.15.84:0/4214609939 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f3fbc001670 con 0x7f3fc40cfc40 2023-10-16T09:49:51.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.462+0000 7f3fc159a640 1 -- 172.21.15.84:0/4214609939 >> v1:172.21.15.84:6789/0 conn(0x7f3fc40cfc40 legacy=0x7f3fc40b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:51.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.462+0000 7f3fc159a640 1 --2- 172.21.15.84:0/4214609939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fb00039a0 0x7f3fb0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.462+0000 7f3fc159a640 1 --2- 172.21.15.84:0/4214609939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fc40b2080 0x7f3fb00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.462+0000 7f3fc159a640 1 --2- 172.21.15.84:0/4214609939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb0004d00 0x7f3fb0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.463+0000 7f3fc159a640 1 -- 172.21.15.84:0/4214609939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc40d0bc0 con 0x7f3fb00039a0 2023-10-16T09:49:51.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.463+0000 7f3fc159a640 1 -- 172.21.15.84:0/4214609939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb000baf0 con 0x7f3fb0004d00 2023-10-16T09:49:51.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.463+0000 7f3fc159a640 1 -- 172.21.15.84:0/4214609939 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb000bcb0 con 0x7f3fc40b2080 2023-10-16T09:49:51.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.463+0000 7f3fc259c640 1 --2- 172.21.15.84:0/4214609939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fb00039a0 0x7f3fb0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.463+0000 7f3fc259c640 1 -- 172.21.15.84:0/4214609939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fc40b2080 msgr2=0x7f3fb00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:51.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.463+0000 7f3fc259c640 1 --2- 172.21.15.84:0/4214609939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fc40b2080 0x7f3fb00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.463+0000 7f3fc259c640 1 -- 172.21.15.84:0/4214609939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb0004d00 msgr2=0x7f3fb0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:51.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.463+0000 7f3fc259c640 1 --2- 172.21.15.84:0/4214609939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb0004d00 0x7f3fb0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.463+0000 7f3fc259c640 1 -- 172.21.15.84:0/4214609939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb0003520 con 0x7f3fb00039a0 2023-10-16T09:49:51.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.464+0000 7f3fc259c640 1 --2- 172.21.15.84:0/4214609939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fb00039a0 0x7f3fb0003dc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4001e20 tx=0x7f3fb00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4b015ead65da77 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:51.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.464+0000 7f3fc159a640 1 -- 172.21.15.84:0/4214609939 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4003b70 con 0x7f3fb00039a0 2023-10-16T09:49:51.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.465+0000 7f3fc159a640 1 -- 172.21.15.84:0/4214609939 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb4003cd0 con 0x7f3fb00039a0 2023-10-16T09:49:51.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.465+0000 7f3fc159a640 1 -- 172.21.15.84:0/4214609939 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4005310 con 0x7f3fb00039a0 2023-10-16T09:49:51.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.465+0000 7f3fc359e640 1 -- 172.21.15.84:0/4214609939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fb00039a0 msgr2=0x7f3fb0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:51.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.465+0000 7f3fc359e640 1 --2- 172.21.15.84:0/4214609939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fb00039a0 0x7f3fb0003dc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4001e20 tx=0x7f3fb00057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.465+0000 7f3fc2d9d640 1 -- 172.21.15.84:0/4214609939 reap_dead start 2023-10-16T09:49:51.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.465+0000 7f3fc359e640 1 -- 172.21.15.84:0/4214609939 shutdown_connections 2023-10-16T09:49:51.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.465+0000 7f3fc359e640 1 -- 172.21.15.84:0/4214609939 >> 172.21.15.84:0/4214609939 conn(0x7f3fc40d2b90 msgr2=0x7f3fc40d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:51.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.465+0000 7f3fc359e640 1 -- 172.21.15.84:0/4214609939 shutdown_connections 2023-10-16T09:49:51.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.465+0000 7f3fc359e640 1 -- 172.21.15.84:0/4214609939 wait complete. 2023-10-16T09:49:51.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.467+0000 7f3fc359e640 1 Processor -- start 2023-10-16T09:49:51.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.467+0000 7f3fc359e640 1 -- start start 2023-10-16T09:49:51.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.468+0000 7f3fc359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb00039a0 0x7f3fc40cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.468+0000 7f3fc359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fc40b5700 0x7f3fc41e8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.468+0000 7f3fc359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fc40c7810 0x7f3fc41f6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.468+0000 7f3fc359e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc40cd010 con 0x7f3fc40c7810 2023-10-16T09:49:51.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.468+0000 7f3fc359e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc40b8a60 con 0x7f3fb00039a0 2023-10-16T09:49:51.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.468+0000 7f3fc359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc41f6120 con 0x7f3fc40b5700 2023-10-16T09:49:51.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.468+0000 7f3fc2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fc40c7810 0x7f3fc41f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.468+0000 7f3fc2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fc40c7810 0x7f3fc41f6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59874/0 (socket says 172.21.15.84:59874) 2023-10-16T09:49:51.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.468+0000 7f3fc2d9d640 1 -- 172.21.15.84:0/1326432733 learned_addr learned my addr 172.21.15.84:0/1326432733 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:51.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.469+0000 7f3fc1d9b640 1 --2- 172.21.15.84:0/1326432733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fc40b5700 0x7f3fc41e8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.469+0000 7f3fc259c640 1 --2- 172.21.15.84:0/1326432733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb00039a0 0x7f3fc40cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.469+0000 7f3fc2d9d640 1 -- 172.21.15.84:0/1326432733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fc40b5700 msgr2=0x7f3fc41e8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:51.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.469+0000 7f3fc2d9d640 1 --2- 172.21.15.84:0/1326432733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fc40b5700 0x7f3fc41e8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.469+0000 7f3fc2d9d640 1 -- 172.21.15.84:0/1326432733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb00039a0 msgr2=0x7f3fc40cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:51.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.469+0000 7f3fc2d9d640 1 --2- 172.21.15.84:0/1326432733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb00039a0 0x7f3fc40cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.469+0000 7f3fc2d9d640 1 -- 172.21.15.84:0/1326432733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc40ccd70 con 0x7f3fc40c7810 2023-10-16T09:49:51.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.469+0000 7f3fc259c640 1 --2- 172.21.15.84:0/1326432733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb00039a0 0x7f3fc40cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:51.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.469+0000 7f3fc2d9d640 1 --2- 172.21.15.84:0/1326432733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fc40c7810 0x7f3fc41f6600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40b56d0 tx=0x7f3fbc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:51.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.469+0000 7f3fbaffd640 1 -- 172.21.15.84:0/1326432733 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc004bf0 con 0x7f3fc40c7810 2023-10-16T09:49:51.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.469+0000 7f3fbaffd640 1 -- 172.21.15.84:0/1326432733 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fbc004d70 con 0x7f3fc40c7810 2023-10-16T09:49:51.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.470+0000 7f3fc359e640 1 -- 172.21.15.84:0/1326432733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc40b61b0 con 0x7f3fc40c7810 2023-10-16T09:49:51.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.470+0000 7f3fbaffd640 1 -- 172.21.15.84:0/1326432733 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc007270 con 0x7f3fc40c7810 2023-10-16T09:49:51.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.470+0000 7f3fc359e640 1 -- 172.21.15.84:0/1326432733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc40b6770 con 0x7f3fc40c7810 2023-10-16T09:49:51.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.471+0000 7f3fc359e640 1 -- 172.21.15.84:0/1326432733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc41f6320 con 0x7f3fc40c7810 2023-10-16T09:49:51.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.472+0000 7f3fbaffd640 1 -- 172.21.15.84:0/1326432733 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3fbc0073d0 con 0x7f3fc40c7810 2023-10-16T09:49:51.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.473+0000 7f3fbaffd640 1 --2- 172.21.15.84:0/1326432733 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3fa406aea0 0x7f3fa406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.473+0000 7f3fbaffd640 1 -- 172.21.15.84:0/1326432733 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3fbc088820 con 0x7f3fc40c7810 2023-10-16T09:49:51.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.473+0000 7f3fc259c640 1 --2- 172.21.15.84:0/1326432733 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3fa406aea0 0x7f3fa406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.474+0000 7f3fc259c640 1 --2- 172.21.15.84:0/1326432733 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3fa406aea0 0x7f3fa406d360 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4034000 tx=0x7f3fb4039040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:51.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.477+0000 7f3fbaffd640 1 -- 172.21.15.84:0/1326432733 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3fc41f6320 con 0x7f3fc40c7810 2023-10-16T09:49:51.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.675+0000 7f3fc359e640 1 -- 172.21.15.84:0/1326432733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3fc41f62e0 con 0x7f3fc40c7810 2023-10-16T09:49:51.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.676+0000 7f3fbaffd640 1 -- 172.21.15.84:0/1326432733 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3fbc0537b0 con 0x7f3fc40c7810 2023-10-16T09:49:51.678 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T09:49:51.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.679+0000 7f3fc359e640 1 -- 172.21.15.84:0/1326432733 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3fa406aea0 msgr2=0x7f3fa406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:51.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.679+0000 7f3fc359e640 1 --2- 172.21.15.84:0/1326432733 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3fa406aea0 0x7f3fa406d360 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4034000 tx=0x7f3fb4039040 comp rx=0 tx=0).stop 2023-10-16T09:49:51.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.679+0000 7f3fc359e640 1 -- 172.21.15.84:0/1326432733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fc40c7810 msgr2=0x7f3fc41f6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:51.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.679+0000 7f3fc359e640 1 --2- 172.21.15.84:0/1326432733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fc40c7810 0x7f3fc41f6600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40b56d0 tx=0x7f3fbc00c960 comp rx=0 tx=0).stop 2023-10-16T09:49:51.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.679+0000 7f3fc359e640 1 -- 172.21.15.84:0/1326432733 shutdown_connections 2023-10-16T09:49:51.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.679+0000 7f3fc359e640 1 --2- 172.21.15.84:0/1326432733 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f3fa406aea0 0x7f3fa406d360 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.679+0000 7f3fc359e640 1 --2- 172.21.15.84:0/1326432733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fc40c7810 0x7f3fc41f6600 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.679+0000 7f3fc359e640 1 --2- 172.21.15.84:0/1326432733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fc40b5700 0x7f3fc41e8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.679+0000 7f3fc359e640 1 --2- 172.21.15.84:0/1326432733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb00039a0 0x7f3fc40cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.679+0000 7f3fc359e640 1 -- 172.21.15.84:0/1326432733 >> 172.21.15.84:0/1326432733 conn(0x7f3fc40d2b90 msgr2=0x7f3fc415fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:51.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.679+0000 7f3fc359e640 1 -- 172.21.15.84:0/1326432733 shutdown_connections 2023-10-16T09:49:51.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.679+0000 7f3fc359e640 1 -- 172.21.15.84:0/1326432733 wait complete. 2023-10-16T09:49:51.700 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:49:51.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.916+0000 7fd2725a9640 1 Processor -- start 2023-10-16T09:49:51.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.916+0000 7fd2725a9640 1 -- start start 2023-10-16T09:49:51.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.917+0000 7fd2725a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c06a4b0 0x7fd26c0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.918+0000 7fd2725a9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26c0cb030 con 0x7fd26c15ad70 2023-10-16T09:49:51.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.918+0000 7fd2725a9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26c0c91c0 con 0x7fd26c0c9ca0 2023-10-16T09:49:51.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.918+0000 7fd2725a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c0c94a0 con 0x7fd26c06a4b0 2023-10-16T09:49:51.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.918+0000 7fd26bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd26c15ad70 0x7fd26c069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47164/0 (socket says 172.21.15.84:47164) 2023-10-16T09:49:51.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.918+0000 7fd26bfff640 1 -- 172.21.15.84:0/3741403167 learned_addr learned my addr 172.21.15.84:0/3741403167 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:51.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.918+0000 7fd26b7fe640 1 --2- 172.21.15.84:0/3741403167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c06a4b0 0x7fd26c0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.919+0000 7fd26affd640 1 -- 172.21.15.84:0/3741403167 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604318815 0 0) 0x7fd26c0cb030 con 0x7fd26c15ad70 2023-10-16T09:49:51.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.919+0000 7fd26affd640 1 -- 172.21.15.84:0/3741403167 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd254003520 con 0x7fd26c15ad70 2023-10-16T09:49:51.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.919+0000 7fd26affd640 1 -- 172.21.15.84:0/3741403167 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1358565307 0 0) 0x7fd254003520 con 0x7fd26c15ad70 2023-10-16T09:49:51.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.919+0000 7fd26affd640 1 -- 172.21.15.84:0/3741403167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c06a4b0 msgr2=0x7fd26c0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:51.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.919+0000 7fd26affd640 1 --2- 172.21.15.84:0/3741403167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c06a4b0 0x7fd26c0ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.919+0000 7fd26affd640 1 -- 172.21.15.84:0/3741403167 >> v1:172.21.15.157:6789/0 conn(0x7fd26c0c9ca0 legacy=0x7fd26c0ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:51.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.919+0000 7fd26affd640 1 -- 172.21.15.84:0/3741403167 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c0c9780 con 0x7fd26c15ad70 2023-10-16T09:49:51.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.919+0000 7fd26affd640 1 -- 172.21.15.84:0/3741403167 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd25c001670 con 0x7fd26c15ad70 2023-10-16T09:49:51.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.919+0000 7fd26affd640 1 -- 172.21.15.84:0/3741403167 >> v1:172.21.15.84:6789/0 conn(0x7fd26c15ad70 legacy=0x7fd26c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:51.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.919+0000 7fd26affd640 1 --2- 172.21.15.84:0/3741403167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c06a4b0 0x7fd2540039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.919+0000 7fd26affd640 1 --2- 172.21.15.84:0/3741403167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd254003ee0 0x7fd2540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.919+0000 7fd26affd640 1 --2- 172.21.15.84:0/3741403167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd254004d00 0x7fd254005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.919+0000 7fd26affd640 1 -- 172.21.15.84:0/3741403167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c0c94a0 con 0x7fd254004d00 2023-10-16T09:49:51.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.919+0000 7fd26affd640 1 -- 172.21.15.84:0/3741403167 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25400ba30 con 0x7fd254003ee0 2023-10-16T09:49:51.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.919+0000 7fd26affd640 1 -- 172.21.15.84:0/3741403167 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25400bbf0 con 0x7fd26c06a4b0 2023-10-16T09:49:51.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.919+0000 7fd270b46640 1 --2- 172.21.15.84:0/3741403167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd254004d00 0x7fd254005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.920+0000 7fd26bfff640 1 --2- 172.21.15.84:0/3741403167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c06a4b0 0x7fd2540039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.920+0000 7fd26b7fe640 1 --2- 172.21.15.84:0/3741403167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd254003ee0 0x7fd2540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.920+0000 7fd270b46640 1 -- 172.21.15.84:0/3741403167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c06a4b0 msgr2=0x7fd2540039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:51.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.920+0000 7fd270b46640 1 --2- 172.21.15.84:0/3741403167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c06a4b0 0x7fd2540039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.920+0000 7fd270b46640 1 -- 172.21.15.84:0/3741403167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd254003ee0 msgr2=0x7fd2540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:51.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.920+0000 7fd270b46640 1 --2- 172.21.15.84:0/3741403167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd254003ee0 0x7fd2540047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.920+0000 7fd270b46640 1 -- 172.21.15.84:0/3741403167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd254003520 con 0x7fd254004d00 2023-10-16T09:49:51.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.920+0000 7fd26b7fe640 1 --2- 172.21.15.84:0/3741403167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd254003ee0 0x7fd2540047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:51.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.921+0000 7fd270b46640 1 --2- 172.21.15.84:0/3741403167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd254004d00 0x7fd254005180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd264001f60 tx=0x7fd2540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0374a707b38976 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:51.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.921+0000 7fd26affd640 1 -- 172.21.15.84:0/3741403167 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26402d6f0 con 0x7fd254004d00 2023-10-16T09:49:51.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.921+0000 7fd26affd640 1 -- 172.21.15.84:0/3741403167 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26402d870 con 0x7fd254004d00 2023-10-16T09:49:51.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.921+0000 7fd26affd640 1 -- 172.21.15.84:0/3741403167 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26402dc20 con 0x7fd254004d00 2023-10-16T09:49:51.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.922+0000 7fd2725a9640 1 -- 172.21.15.84:0/3741403167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd254004d00 msgr2=0x7fd254005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:51.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.922+0000 7fd2725a9640 1 --2- 172.21.15.84:0/3741403167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd254004d00 0x7fd254005180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd264001f60 tx=0x7fd2540057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.922+0000 7fd270b46640 1 -- 172.21.15.84:0/3741403167 reap_dead start 2023-10-16T09:49:51.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.922+0000 7fd2725a9640 1 -- 172.21.15.84:0/3741403167 shutdown_connections 2023-10-16T09:49:51.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.922+0000 7fd2725a9640 1 -- 172.21.15.84:0/3741403167 >> 172.21.15.84:0/3741403167 conn(0x7fd26c0ce9b0 msgr2=0x7fd26c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:51.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.922+0000 7fd2725a9640 1 -- 172.21.15.84:0/3741403167 shutdown_connections 2023-10-16T09:49:51.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.922+0000 7fd2725a9640 1 -- 172.21.15.84:0/3741403167 wait complete. 2023-10-16T09:49:51.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.924+0000 7fd2725a9640 1 Processor -- start 2023-10-16T09:49:51.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.924+0000 7fd2725a9640 1 -- start start 2023-10-16T09:49:51.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.925+0000 7fd2725a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd254004d00 0x7fd26c14e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.925+0000 7fd2725a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c1f1e30 0x7fd26c15dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.925+0000 7fd2725a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd26c0af4a0 0x7fd26c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.925+0000 7fd2725a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c1f2ee0 con 0x7fd26c0af4a0 2023-10-16T09:49:51.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.925+0000 7fd2725a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c0afe90 con 0x7fd254004d00 2023-10-16T09:49:51.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.925+0000 7fd2725a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c0cc9b0 con 0x7fd26c1f1e30 2023-10-16T09:49:51.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.925+0000 7fd270b46640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd26c0af4a0 0x7fd26c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.925+0000 7fd270b46640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd26c0af4a0 0x7fd26c0af950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59882/0 (socket says 172.21.15.84:59882) 2023-10-16T09:49:51.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.925+0000 7fd270b46640 1 -- 172.21.15.84:0/848876325 learned_addr learned my addr 172.21.15.84:0/848876325 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:51.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.926+0000 7fd26b7fe640 1 --2- 172.21.15.84:0/848876325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c1f1e30 0x7fd26c15dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.926+0000 7fd270b46640 1 -- 172.21.15.84:0/848876325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c1f1e30 msgr2=0x7fd26c15dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:51.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.926+0000 7fd26bfff640 1 --2- 172.21.15.84:0/848876325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd254004d00 0x7fd26c14e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.926+0000 7fd270b46640 1 --2- 172.21.15.84:0/848876325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c1f1e30 0x7fd26c15dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.926+0000 7fd270b46640 1 -- 172.21.15.84:0/848876325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd254004d00 msgr2=0x7fd26c14e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:51.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.926+0000 7fd270b46640 1 --2- 172.21.15.84:0/848876325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd254004d00 0x7fd26c14e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:51.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.926+0000 7fd270b46640 1 -- 172.21.15.84:0/848876325 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c1f2360 con 0x7fd26c0af4a0 2023-10-16T09:49:51.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.926+0000 7fd26b7fe640 1 --2- 172.21.15.84:0/848876325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c1f1e30 0x7fd26c15dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:51.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.926+0000 7fd270b46640 1 --2- 172.21.15.84:0/848876325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd26c0af4a0 0x7fd26c0af950 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd264002080 tx=0x7fd2640015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:51.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.927+0000 7fd268ff9640 1 -- 172.21.15.84:0/848876325 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264003970 con 0x7fd26c0af4a0 2023-10-16T09:49:51.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.927+0000 7fd268ff9640 1 -- 172.21.15.84:0/848876325 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd264003ad0 con 0x7fd26c0af4a0 2023-10-16T09:49:51.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.927+0000 7fd268ff9640 1 -- 172.21.15.84:0/848876325 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264003e60 con 0x7fd26c0af4a0 2023-10-16T09:49:51.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.927+0000 7fd2725a9640 1 -- 172.21.15.84:0/848876325 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26c1f2640 con 0x7fd26c0af4a0 2023-10-16T09:49:51.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.927+0000 7fd2725a9640 1 -- 172.21.15.84:0/848876325 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26c1f2c00 con 0x7fd26c0af4a0 2023-10-16T09:49:51.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.928+0000 7fd2725a9640 1 -- 172.21.15.84:0/848876325 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26c0adba0 con 0x7fd26c0af4a0 2023-10-16T09:49:51.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.929+0000 7fd268ff9640 1 -- 172.21.15.84:0/848876325 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd264005070 con 0x7fd26c0af4a0 2023-10-16T09:49:51.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.930+0000 7fd268ff9640 1 --2- 172.21.15.84:0/848876325 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd25806ae70 0x7fd25806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:51.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.930+0000 7fd268ff9640 1 -- 172.21.15.84:0/848876325 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fd264031070 con 0x7fd26c0af4a0 2023-10-16T09:49:51.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.930+0000 7fd26bfff640 1 --2- 172.21.15.84:0/848876325 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd25806ae70 0x7fd25806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:51.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.931+0000 7fd26bfff640 1 --2- 172.21.15.84:0/848876325 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd25806ae70 0x7fd25806d330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd25c001bd0 tx=0x7fd25c000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:51.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:51.934+0000 7fd268ff9640 1 -- 172.21.15.84:0/848876325 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd264076630 con 0x7fd26c0af4a0 2023-10-16T09:49:52.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.125+0000 7fd2725a9640 1 -- 172.21.15.84:0/848876325 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd26c04e160 con 0x7fd26c0af4a0 2023-10-16T09:49:52.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.125+0000 7fd268ff9640 1 -- 172.21.15.84:0/848876325 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd264079c50 con 0x7fd26c0af4a0 2023-10-16T09:49:52.127 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T09:49:52.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.128+0000 7fd2725a9640 1 -- 172.21.15.84:0/848876325 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd25806ae70 msgr2=0x7fd25806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:52.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.128+0000 7fd2725a9640 1 --2- 172.21.15.84:0/848876325 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd25806ae70 0x7fd25806d330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd25c001bd0 tx=0x7fd25c000ed0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.128+0000 7fd2725a9640 1 -- 172.21.15.84:0/848876325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd26c0af4a0 msgr2=0x7fd26c0af950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:52.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.128+0000 7fd2725a9640 1 --2- 172.21.15.84:0/848876325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd26c0af4a0 0x7fd26c0af950 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd264002080 tx=0x7fd2640015b0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.128+0000 7fd2725a9640 1 -- 172.21.15.84:0/848876325 shutdown_connections 2023-10-16T09:49:52.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.128+0000 7fd2725a9640 1 --2- 172.21.15.84:0/848876325 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd25806ae70 0x7fd25806d330 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.128+0000 7fd2725a9640 1 --2- 172.21.15.84:0/848876325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd26c0af4a0 0x7fd26c0af950 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.128+0000 7fd2725a9640 1 --2- 172.21.15.84:0/848876325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd26c1f1e30 0x7fd26c15dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.128+0000 7fd2725a9640 1 --2- 172.21.15.84:0/848876325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd254004d00 0x7fd26c14e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.128+0000 7fd2725a9640 1 -- 172.21.15.84:0/848876325 >> 172.21.15.84:0/848876325 conn(0x7fd26c0ce9b0 msgr2=0x7fd26c0b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:52.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.129+0000 7fd2725a9640 1 -- 172.21.15.84:0/848876325 shutdown_connections 2023-10-16T09:49:52.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.129+0000 7fd2725a9640 1 -- 172.21.15.84:0/848876325 wait complete. 2023-10-16T09:49:52.149 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:49:52.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.361+0000 7f077ea7c640 1 Processor -- start 2023-10-16T09:49:52.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.362+0000 7f077ea7c640 1 -- start start 2023-10-16T09:49:52.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.363+0000 7f077ea7c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0778160ad0 0x7f07780ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:52.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.363+0000 7f077ea7c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f07780ce350 0x7f0778160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:52.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.363+0000 7f077ea7c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07780ae260 0x7f07780ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:52.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.363+0000 7f077ea7c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07780ce800 con 0x7f0778160ad0 2023-10-16T09:49:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.363+0000 7f077ea7c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07780cdec0 con 0x7f07780ce350 2023-10-16T09:49:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.363+0000 7f077ea7c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07780acdd0 con 0x7f07780ae260 2023-10-16T09:49:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.363+0000 7f077c818640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0778160ad0 0x7f07780ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.364+0000 7f077c818640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0778160ad0 0x7f07780ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59890/0 (socket says 172.21.15.84:59890) 2023-10-16T09:49:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.364+0000 7f077c818640 1 -- 172.21.15.84:0/3138936305 learned_addr learned my addr 172.21.15.84:0/3138936305 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:52.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.364+0000 7f077d019640 1 --2- 172.21.15.84:0/3138936305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07780ae260 0x7f07780ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:52.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.364+0000 7f077c818640 1 -- 172.21.15.84:0/3138936305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07780ae260 msgr2=0x7f07780ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:52.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.364+0000 7f077c818640 1 --2- 172.21.15.84:0/3138936305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07780ae260 0x7f07780ccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.364+0000 7f077c818640 1 -- 172.21.15.84:0/3138936305 >> v2:172.21.15.157:3300/0 conn(0x7f07780ce350 msgr2=0x7f0778160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.364+0000 7f0777fff640 1 --2- 172.21.15.84:0/3138936305 >> v2:172.21.15.157:3300/0 conn(0x7f07780ce350 0x7f0778160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.364+0000 7f077c818640 1 --2- 172.21.15.84:0/3138936305 >> v2:172.21.15.157:3300/0 conn(0x7f07780ce350 0x7f0778160eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.364+0000 7f077c818640 1 -- 172.21.15.84:0/3138936305 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07781613f0 con 0x7f0778160ad0 2023-10-16T09:49:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.364+0000 7f0777fff640 1 --2- 172.21.15.84:0/3138936305 >> v2:172.21.15.157:3300/0 conn(0x7f07780ce350 0x7f0778160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.366+0000 7f077c818640 1 --2- 172.21.15.84:0/3138936305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0778160ad0 0x7f07780ac890 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f077806f6e0 tx=0x7f076802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc5ecfa6f7e73e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:52.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.366+0000 7f07777fe640 1 -- 172.21.15.84:0/3138936305 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076802ca80 con 0x7f0778160ad0 2023-10-16T09:49:52.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.366+0000 7f07777fe640 1 -- 172.21.15.84:0/3138936305 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076802cbe0 con 0x7f0778160ad0 2023-10-16T09:49:52.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.366+0000 7f07777fe640 1 -- 172.21.15.84:0/3138936305 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680343f0 con 0x7f0778160ad0 2023-10-16T09:49:52.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.367+0000 7f077ea7c640 1 -- 172.21.15.84:0/3138936305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0778160ad0 msgr2=0x7f07780ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:52.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.367+0000 7f077ea7c640 1 --2- 172.21.15.84:0/3138936305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0778160ad0 0x7f07780ac890 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f077806f6e0 tx=0x7f076802bee0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.367+0000 7f077ea7c640 1 -- 172.21.15.84:0/3138936305 shutdown_connections 2023-10-16T09:49:52.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.367+0000 7f077ea7c640 1 --2- 172.21.15.84:0/3138936305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07780ae260 0x7f07780ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.367+0000 7f077ea7c640 1 --2- 172.21.15.84:0/3138936305 >> v2:172.21.15.157:3300/0 conn(0x7f07780ce350 0x7f0778160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.367+0000 7f077ea7c640 1 --2- 172.21.15.84:0/3138936305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0778160ad0 0x7f07780ac890 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.367+0000 7f077ea7c640 1 -- 172.21.15.84:0/3138936305 >> 172.21.15.84:0/3138936305 conn(0x7f077806bc10 msgr2=0x7f077806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:52.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.367+0000 7f077ea7c640 1 -- 172.21.15.84:0/3138936305 shutdown_connections 2023-10-16T09:49:52.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.367+0000 7f077ea7c640 1 -- 172.21.15.84:0/3138936305 wait complete. 2023-10-16T09:49:52.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.369+0000 7f077ea7c640 1 Processor -- start 2023-10-16T09:49:52.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.370+0000 7f077ea7c640 1 -- start start 2023-10-16T09:49:52.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.371+0000 7f077ea7c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07780ae260 0x7f07780c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:52.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.371+0000 7f077ea7c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07780ce350 0x7f0778158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:52.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.371+0000 7f077ea7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0778160ad0 0x7f07780c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:52.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.371+0000 7f077ea7c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077806b0a0 con 0x7f07780ae260 2023-10-16T09:49:52.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.371+0000 7f077ea7c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077806a7b0 con 0x7f0778160ad0 2023-10-16T09:49:52.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.371+0000 7f077ea7c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07780c8f30 con 0x7f07780ce350 2023-10-16T09:49:52.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.371+0000 7f077c818640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07780ae260 0x7f07780c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:52.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.371+0000 7f077c818640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07780ae260 0x7f07780c8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59898/0 (socket says 172.21.15.84:59898) 2023-10-16T09:49:52.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.371+0000 7f077c818640 1 -- 172.21.15.84:0/3292433607 learned_addr learned my addr 172.21.15.84:0/3292433607 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:52.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.372+0000 7f0777fff640 1 --2- 172.21.15.84:0/3292433607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07780ce350 0x7f0778158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:52.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.372+0000 7f077c818640 1 -- 172.21.15.84:0/3292433607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07780ce350 msgr2=0x7f0778158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:52.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.372+0000 7f077c818640 1 --2- 172.21.15.84:0/3292433607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07780ce350 0x7f0778158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.372+0000 7f077d019640 1 --2- 172.21.15.84:0/3292433607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0778160ad0 0x7f07780c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:52.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.372+0000 7f077c818640 1 -- 172.21.15.84:0/3292433607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0778160ad0 msgr2=0x7f07780c6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:52.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.372+0000 7f077c818640 1 --2- 172.21.15.84:0/3292433607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0778160ad0 0x7f07780c6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.372+0000 7f077c818640 1 -- 172.21.15.84:0/3292433607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0768009b30 con 0x7f07780ae260 2023-10-16T09:49:52.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.372+0000 7f077c818640 1 --2- 172.21.15.84:0/3292433607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07780ae260 0x7f07780c8890 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0768000f80 tx=0x7f0768004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:52.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.372+0000 7f07757fa640 1 -- 172.21.15.84:0/3292433607 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076803e450 con 0x7f07780ae260 2023-10-16T09:49:52.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.373+0000 7f07757fa640 1 -- 172.21.15.84:0/3292433607 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076803e5f0 con 0x7f07780ae260 2023-10-16T09:49:52.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.373+0000 7f07757fa640 1 -- 172.21.15.84:0/3292433607 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076803e9a0 con 0x7f07780ae260 2023-10-16T09:49:52.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.373+0000 7f077ea7c640 1 -- 172.21.15.84:0/3292433607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077806aa50 con 0x7f07780ae260 2023-10-16T09:49:52.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.373+0000 7f077ea7c640 1 -- 172.21.15.84:0/3292433607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07780c7390 con 0x7f07780ae260 2023-10-16T09:49:52.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.374+0000 7f077ea7c640 1 -- 172.21.15.84:0/3292433607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077804e160 con 0x7f07780ae260 2023-10-16T09:49:52.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.375+0000 7f07757fa640 1 -- 172.21.15.84:0/3292433607 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f076802cd50 con 0x7f07780ae260 2023-10-16T09:49:52.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.376+0000 7f07757fa640 1 --2- 172.21.15.84:0/3292433607 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f075806aea0 0x7f075806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:52.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.376+0000 7f07757fa640 1 -- 172.21.15.84:0/3292433607 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f07680ac1e0 con 0x7f07780ae260 2023-10-16T09:49:52.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.377+0000 7f0777fff640 1 --2- 172.21.15.84:0/3292433607 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f075806aea0 0x7f075806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:52.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.377+0000 7f0777fff640 1 --2- 172.21.15.84:0/3292433607 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f075806aea0 0x7f075806d360 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f07780c8680 tx=0x7f0760008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:52.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.381+0000 7f07757fa640 1 -- 172.21.15.84:0/3292433607 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f07680773c0 con 0x7f07780ae260 2023-10-16T09:49:52.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.573+0000 7f077ea7c640 1 -- 172.21.15.84:0/3292433607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0778065d60 con 0x7f07780ae260 2023-10-16T09:49:52.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.573+0000 7f07757fa640 1 -- 172.21.15.84:0/3292433607 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f076807a9e0 con 0x7f07780ae260 2023-10-16T09:49:52.576 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T09:49:52.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.576+0000 7f077ea7c640 1 -- 172.21.15.84:0/3292433607 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f075806aea0 msgr2=0x7f075806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:52.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.576+0000 7f077ea7c640 1 --2- 172.21.15.84:0/3292433607 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f075806aea0 0x7f075806d360 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f07780c8680 tx=0x7f0760008040 comp rx=0 tx=0).stop 2023-10-16T09:49:52.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.576+0000 7f077ea7c640 1 -- 172.21.15.84:0/3292433607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07780ae260 msgr2=0x7f07780c8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:52.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.576+0000 7f077ea7c640 1 --2- 172.21.15.84:0/3292433607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07780ae260 0x7f07780c8890 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0768000f80 tx=0x7f0768004a40 comp rx=0 tx=0).stop 2023-10-16T09:49:52.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.577+0000 7f077ea7c640 1 -- 172.21.15.84:0/3292433607 shutdown_connections 2023-10-16T09:49:52.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.577+0000 7f077ea7c640 1 --2- 172.21.15.84:0/3292433607 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f075806aea0 0x7f075806d360 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.577+0000 7f077ea7c640 1 --2- 172.21.15.84:0/3292433607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0778160ad0 0x7f07780c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.577+0000 7f077ea7c640 1 --2- 172.21.15.84:0/3292433607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07780ce350 0x7f0778158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.577+0000 7f077ea7c640 1 --2- 172.21.15.84:0/3292433607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07780ae260 0x7f07780c8890 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.577+0000 7f077ea7c640 1 -- 172.21.15.84:0/3292433607 >> 172.21.15.84:0/3292433607 conn(0x7f077806bc10 msgr2=0x7f07780ccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:52.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.577+0000 7f077ea7c640 1 -- 172.21.15.84:0/3292433607 shutdown_connections 2023-10-16T09:49:52.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.577+0000 7f077ea7c640 1 -- 172.21.15.84:0/3292433607 wait complete. 2023-10-16T09:49:52.598 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T09:49:52.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.810+0000 7f65000d0640 1 Processor -- start 2023-10-16T09:49:52.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.811+0000 7f65000d0640 1 -- start start 2023-10-16T09:49:52.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.812+0000 7f65000d0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64f806a4b0 0x7f64f80ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:52.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.812+0000 7f65000d0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f80cb030 con 0x7f64f815ad70 2023-10-16T09:49:52.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.812+0000 7f65000d0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f80c91c0 con 0x7f64f80c9ca0 2023-10-16T09:49:52.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.812+0000 7f65000d0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f80c94a0 con 0x7f64f806a4b0 2023-10-16T09:49:52.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.812+0000 7f64fde6c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f64f815ad70 0x7f64f8069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47172/0 (socket says 172.21.15.84:47172) 2023-10-16T09:49:52.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.812+0000 7f64fde6c640 1 -- 172.21.15.84:0/3199969947 learned_addr learned my addr 172.21.15.84:0/3199969947 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:52.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.813+0000 7f64fd66b640 1 --2- 172.21.15.84:0/3199969947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64f806a4b0 0x7f64f80ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:52.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.813+0000 7f64fce6a640 1 -- 172.21.15.84:0/3199969947 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42312456 0 0) 0x7f64f80cb030 con 0x7f64f815ad70 2023-10-16T09:49:52.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.813+0000 7f64fce6a640 1 -- 172.21.15.84:0/3199969947 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64e4003520 con 0x7f64f815ad70 2023-10-16T09:49:52.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.813+0000 7f64fce6a640 1 -- 172.21.15.84:0/3199969947 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 935268276 0 0) 0x7f64e4003520 con 0x7f64f815ad70 2023-10-16T09:49:52.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.813+0000 7f64fce6a640 1 -- 172.21.15.84:0/3199969947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64f806a4b0 msgr2=0x7f64f80ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:52.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.813+0000 7f64fce6a640 1 --2- 172.21.15.84:0/3199969947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64f806a4b0 0x7f64f80ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.813+0000 7f64fce6a640 1 -- 172.21.15.84:0/3199969947 >> v1:172.21.15.157:6789/0 conn(0x7f64f80c9ca0 legacy=0x7f64f80ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:52.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.813+0000 7f64fce6a640 1 -- 172.21.15.84:0/3199969947 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f80c9780 con 0x7f64f815ad70 2023-10-16T09:49:52.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.813+0000 7f64fce6a640 1 -- 172.21.15.84:0/3199969947 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f64f4001670 con 0x7f64f815ad70 2023-10-16T09:49:52.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.813+0000 7f64fce6a640 1 -- 172.21.15.84:0/3199969947 >> v1:172.21.15.84:6789/0 conn(0x7f64f815ad70 legacy=0x7f64f8069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:52.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.813+0000 7f64fce6a640 1 --2- 172.21.15.84:0/3199969947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64f806a4b0 0x7f64e40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:52.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.814+0000 7f64fce6a640 1 --2- 172.21.15.84:0/3199969947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e4003f20 0x7f64e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:52.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.814+0000 7f64fce6a640 1 --2- 172.21.15.84:0/3199969947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64e4004d00 0x7f64e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:52.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.814+0000 7f64fce6a640 1 -- 172.21.15.84:0/3199969947 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f80c94a0 con 0x7f64e4004d00 2023-10-16T09:49:52.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.814+0000 7f64fce6a640 1 -- 172.21.15.84:0/3199969947 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e400ba60 con 0x7f64e4003f20 2023-10-16T09:49:52.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.814+0000 7f64fce6a640 1 -- 172.21.15.84:0/3199969947 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e400bc20 con 0x7f64f806a4b0 2023-10-16T09:49:52.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.814+0000 7f64fe66d640 1 --2- 172.21.15.84:0/3199969947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64e4004d00 0x7f64e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:52.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.814+0000 7f64fde6c640 1 --2- 172.21.15.84:0/3199969947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64f806a4b0 0x7f64e40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:52.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.814+0000 7f64fe66d640 1 -- 172.21.15.84:0/3199969947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64f806a4b0 msgr2=0x7f64e40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:52.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.814+0000 7f64fe66d640 1 --2- 172.21.15.84:0/3199969947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64f806a4b0 0x7f64e40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.814+0000 7f64fe66d640 1 -- 172.21.15.84:0/3199969947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e4003f20 msgr2=0x7f64e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:52.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.814+0000 7f64fe66d640 1 --2- 172.21.15.84:0/3199969947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e4003f20 0x7f64e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.814+0000 7f64fe66d640 1 -- 172.21.15.84:0/3199969947 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e4003520 con 0x7f64e4004d00 2023-10-16T09:49:52.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.815+0000 7f64fe66d640 1 --2- 172.21.15.84:0/3199969947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64e4004d00 0x7f64e4005180 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f64f0001e50 tx=0x7f64e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a72071a9ed4058 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:52.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.816+0000 7f64fce6a640 1 -- 172.21.15.84:0/3199969947 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f0003b70 con 0x7f64e4004d00 2023-10-16T09:49:52.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.816+0000 7f64fce6a640 1 -- 172.21.15.84:0/3199969947 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f0003cd0 con 0x7f64e4004d00 2023-10-16T09:49:52.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.816+0000 7f64fce6a640 1 -- 172.21.15.84:0/3199969947 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f0005310 con 0x7f64e4004d00 2023-10-16T09:49:52.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.816+0000 7f65000d0640 1 -- 172.21.15.84:0/3199969947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64e4004d00 msgr2=0x7f64e4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:52.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.816+0000 7f65000d0640 1 --2- 172.21.15.84:0/3199969947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64e4004d00 0x7f64e4005180 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f64f0001e50 tx=0x7f64e40057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.816+0000 7f64fe66d640 1 -- 172.21.15.84:0/3199969947 reap_dead start 2023-10-16T09:49:52.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.816+0000 7f65000d0640 1 -- 172.21.15.84:0/3199969947 shutdown_connections 2023-10-16T09:49:52.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.816+0000 7f65000d0640 1 -- 172.21.15.84:0/3199969947 >> 172.21.15.84:0/3199969947 conn(0x7f64f80ce9b0 msgr2=0x7f64f80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:52.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.816+0000 7f65000d0640 1 -- 172.21.15.84:0/3199969947 shutdown_connections 2023-10-16T09:49:52.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.816+0000 7f65000d0640 1 -- 172.21.15.84:0/3199969947 wait complete. 2023-10-16T09:49:52.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.818+0000 7f65000d0640 1 Processor -- start 2023-10-16T09:49:52.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.818+0000 7f65000d0640 1 -- start start 2023-10-16T09:49:52.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.819+0000 7f65000d0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e4004d00 0x7f64f81566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:52.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.819+0000 7f65000d0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64f8157270 0x7f64f80cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:52.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.819+0000 7f65000d0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64f8156cb0 0x7f64f80ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:52.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.819+0000 7f65000d0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f815cda0 con 0x7f64f8157270 2023-10-16T09:49:52.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.819+0000 7f65000d0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f815bd00 con 0x7f64e4004d00 2023-10-16T09:49:52.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.819+0000 7f65000d0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f815bfe0 con 0x7f64f8156cb0 2023-10-16T09:49:52.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.819+0000 7f64fd66b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64f8157270 0x7f64f80cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:52.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.819+0000 7f64fd66b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64f8157270 0x7f64f80cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59912/0 (socket says 172.21.15.84:59912) 2023-10-16T09:49:52.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.819+0000 7f64fd66b640 1 -- 172.21.15.84:0/1134131530 learned_addr learned my addr 172.21.15.84:0/1134131530 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:52.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.820+0000 7f64fd66b640 1 -- 172.21.15.84:0/1134131530 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64f8156cb0 msgr2=0x7f64f80ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:52.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.820+0000 7f64fd66b640 1 --2- 172.21.15.84:0/1134131530 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64f8156cb0 0x7f64f80ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.820+0000 7f64fd66b640 1 -- 172.21.15.84:0/1134131530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e4004d00 msgr2=0x7f64f81566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:52.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.820+0000 7f64fd66b640 1 --2- 172.21.15.84:0/1134131530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e4004d00 0x7f64f81566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:52.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.820+0000 7f64fd66b640 1 -- 172.21.15.84:0/1134131530 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f815c2c0 con 0x7f64f8157270 2023-10-16T09:49:52.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.820+0000 7f64fd66b640 1 --2- 172.21.15.84:0/1134131530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64f8157270 0x7f64f80cdba0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f64e800e7c0 tx=0x7f64e800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:52.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.820+0000 7f64ee7fc640 1 -- 172.21.15.84:0/1134131530 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e800eed0 con 0x7f64f8157270 2023-10-16T09:49:52.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.820+0000 7f64ee7fc640 1 -- 172.21.15.84:0/1134131530 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e800c840 con 0x7f64f8157270 2023-10-16T09:49:52.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.820+0000 7f64ee7fc640 1 -- 172.21.15.84:0/1134131530 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e800cbf0 con 0x7f64f8157270 2023-10-16T09:49:52.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.820+0000 7f65000d0640 1 -- 172.21.15.84:0/1134131530 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f815c5a0 con 0x7f64f8157270 2023-10-16T09:49:52.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.821+0000 7f65000d0640 1 -- 172.21.15.84:0/1134131530 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f815b920 con 0x7f64f8157270 2023-10-16T09:49:52.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.822+0000 7f65000d0640 1 -- 172.21.15.84:0/1134131530 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f806c9f0 con 0x7f64f8157270 2023-10-16T09:49:52.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.823+0000 7f64ee7fc640 1 -- 172.21.15.84:0/1134131530 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f64e800cd50 con 0x7f64f8157270 2023-10-16T09:49:52.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.823+0000 7f64ee7fc640 1 --2- 172.21.15.84:0/1134131530 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f64dc06aea0 0x7f64dc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:52.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.823+0000 7f64ee7fc640 1 -- 172.21.15.84:0/1134131530 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f64e8089790 con 0x7f64f8157270 2023-10-16T09:49:52.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.824+0000 7f64fde6c640 1 --2- 172.21.15.84:0/1134131530 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f64dc06aea0 0x7f64dc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:52.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.824+0000 7f64fde6c640 1 --2- 172.21.15.84:0/1134131530 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f64dc06aea0 0x7f64dc06d360 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f64f4000fb0 tx=0x7f64f40014b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:52.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:52.828+0000 7f64ee7fc640 1 -- 172.21.15.84:0/1134131530 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f64e8054690 con 0x7f64f8157270 2023-10-16T09:49:53.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.033+0000 7f65000d0640 1 -- 172.21.15.84:0/1134131530 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f64f804e160 con 0x7f64f8157270 2023-10-16T09:49:53.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.033+0000 7f64ee7fc640 1 -- 172.21.15.84:0/1134131530 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f64e8057cb0 con 0x7f64f8157270 2023-10-16T09:49:53.035 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-16T09:49:53.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.036+0000 7f65000d0640 1 -- 172.21.15.84:0/1134131530 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f64dc06aea0 msgr2=0x7f64dc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:53.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.036+0000 7f65000d0640 1 --2- 172.21.15.84:0/1134131530 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f64dc06aea0 0x7f64dc06d360 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f64f4000fb0 tx=0x7f64f40014b0 comp rx=0 tx=0).stop 2023-10-16T09:49:53.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.036+0000 7f65000d0640 1 -- 172.21.15.84:0/1134131530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64f8157270 msgr2=0x7f64f80cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:53.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.036+0000 7f65000d0640 1 --2- 172.21.15.84:0/1134131530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64f8157270 0x7f64f80cdba0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f64e800e7c0 tx=0x7f64e800ec80 comp rx=0 tx=0).stop 2023-10-16T09:49:53.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.036+0000 7f65000d0640 1 -- 172.21.15.84:0/1134131530 shutdown_connections 2023-10-16T09:49:53.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.036+0000 7f65000d0640 1 --2- 172.21.15.84:0/1134131530 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f64dc06aea0 0x7f64dc06d360 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:53.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.036+0000 7f65000d0640 1 --2- 172.21.15.84:0/1134131530 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64f8156cb0 0x7f64f80ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:53.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.036+0000 7f65000d0640 1 --2- 172.21.15.84:0/1134131530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64f8157270 0x7f64f80cdba0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:53.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.036+0000 7f65000d0640 1 --2- 172.21.15.84:0/1134131530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e4004d00 0x7f64f81566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:53.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.036+0000 7f65000d0640 1 -- 172.21.15.84:0/1134131530 >> 172.21.15.84:0/1134131530 conn(0x7f64f80ce9b0 msgr2=0x7f64f80ca1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:53.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.036+0000 7f65000d0640 1 -- 172.21.15.84:0/1134131530 shutdown_connections 2023-10-16T09:49:53.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.037+0000 7f65000d0640 1 -- 172.21.15.84:0/1134131530 wait complete. 2023-10-16T09:49:53.056 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-16T09:49:53.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:53.057 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-16T09:49:53.156 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T09:49:53.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:53.157 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-16T09:49:53.209 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:49:53.209 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:49:53.270 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:49:53.270 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:49:53.323 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T09:49:53.323 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:49:53.377 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:53.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.602+0000 7f6db982d640 1 Processor -- start 2023-10-16T09:49:53.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.602+0000 7f6db982d640 1 -- start start 2023-10-16T09:49:53.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.604+0000 7f6db982d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6db415ae80 0x7f6db415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:53.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.604+0000 7f6db982d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6db40ce390 0x7f6db4070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:53.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.604+0000 7f6db982d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db415ed60 con 0x7f6db415aaa0 2023-10-16T09:49:53.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.604+0000 7f6db982d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db406b810 con 0x7f6db415ae80 2023-10-16T09:49:53.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.604+0000 7f6db982d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db40707e0 con 0x7f6db40ce390 2023-10-16T09:49:53.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.604+0000 7f6daa7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6db415ae80 0x7f6db415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:53.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.604+0000 7f6daa7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6db415ae80 0x7f6db415e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59926/0 (socket says 172.21.15.84:59926) 2023-10-16T09:49:53.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.604+0000 7f6daa7fc640 1 -- 172.21.15.84:0/3160142486 learned_addr learned my addr 172.21.15.84:0/3160142486 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:53.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.605+0000 7f6daa7fc640 1 -- 172.21.15.84:0/3160142486 >> v2:172.21.15.157:3300/0 conn(0x7f6db40ce390 msgr2=0x7f6db4070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:53.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.605+0000 7f6daa7fc640 1 --2- 172.21.15.84:0/3160142486 >> v2:172.21.15.157:3300/0 conn(0x7f6db40ce390 0x7f6db4070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:53.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.605+0000 7f6daa7fc640 1 -- 172.21.15.84:0/3160142486 >> v1:172.21.15.157:6789/0 conn(0x7f6db415aaa0 legacy=0x7f6db40cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:53.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.605+0000 7f6daa7fc640 1 -- 172.21.15.84:0/3160142486 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db415ef90 con 0x7f6db415ae80 2023-10-16T09:49:53.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.606+0000 7f6daa7fc640 1 --2- 172.21.15.84:0/3160142486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6db415ae80 0x7f6db415e820 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6db40c2490 tx=0x7f6d9c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dbc4b1532db7def server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:53.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.606+0000 7f6db27fc640 1 -- 172.21.15.84:0/3160142486 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c031870 con 0x7f6db415ae80 2023-10-16T09:49:53.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.606+0000 7f6db27fc640 1 -- 172.21.15.84:0/3160142486 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9c0319d0 con 0x7f6db415ae80 2023-10-16T09:49:53.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.606+0000 7f6db27fc640 1 -- 172.21.15.84:0/3160142486 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c031d60 con 0x7f6db415ae80 2023-10-16T09:49:53.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.607+0000 7f6db982d640 1 -- 172.21.15.84:0/3160142486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6db415ae80 msgr2=0x7f6db415e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:53.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.607+0000 7f6db982d640 1 --2- 172.21.15.84:0/3160142486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6db415ae80 0x7f6db415e820 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6db40c2490 tx=0x7f6d9c02b910 comp rx=0 tx=0).stop 2023-10-16T09:49:53.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.607+0000 7f6db982d640 1 -- 172.21.15.84:0/3160142486 shutdown_connections 2023-10-16T09:49:53.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.607+0000 7f6db982d640 1 --2- 172.21.15.84:0/3160142486 >> v2:172.21.15.157:3300/0 conn(0x7f6db40ce390 0x7f6db4070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:53.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.607+0000 7f6db982d640 1 --2- 172.21.15.84:0/3160142486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6db415ae80 0x7f6db415e820 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:53.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.607+0000 7f6db982d640 1 -- 172.21.15.84:0/3160142486 >> 172.21.15.84:0/3160142486 conn(0x7f6db40cb410 msgr2=0x7f6db40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:53.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.607+0000 7f6db982d640 1 -- 172.21.15.84:0/3160142486 shutdown_connections 2023-10-16T09:49:53.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.607+0000 7f6db982d640 1 -- 172.21.15.84:0/3160142486 wait complete. 2023-10-16T09:49:53.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.609+0000 7f6db982d640 1 Processor -- start 2023-10-16T09:49:53.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.610+0000 7f6db982d640 1 -- start start 2023-10-16T09:49:53.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.611+0000 7f6db982d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6db40ce390 0x7f6db4147690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:53.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.611+0000 7f6db982d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6db415aaa0 0x7f6db4156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:53.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.611+0000 7f6db982d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db415ae80 0x7f6db40ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:53.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.611+0000 7f6db982d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db4156cb0 con 0x7f6db415aaa0 2023-10-16T09:49:53.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.611+0000 7f6db982d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db4147f10 con 0x7f6db415ae80 2023-10-16T09:49:53.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.611+0000 7f6db982d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db41481f0 con 0x7f6db40ce390 2023-10-16T09:49:53.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.611+0000 7f6daa7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6db415aaa0 0x7f6db4156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:53.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.611+0000 7f6daa7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6db415aaa0 0x7f6db4156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59932/0 (socket says 172.21.15.84:59932) 2023-10-16T09:49:53.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.611+0000 7f6daa7fc640 1 -- 172.21.15.84:0/3795403071 learned_addr learned my addr 172.21.15.84:0/3795403071 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:53.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.611+0000 7f6db2ffd640 1 --2- 172.21.15.84:0/3795403071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6db40ce390 0x7f6db4147690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:53.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.611+0000 7f6daa7fc640 1 -- 172.21.15.84:0/3795403071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6db40ce390 msgr2=0x7f6db4147690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:53.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.611+0000 7f6daa7fc640 1 --2- 172.21.15.84:0/3795403071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6db40ce390 0x7f6db4147690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:53.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.612+0000 7f6daa7fc640 1 -- 172.21.15.84:0/3795403071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db415ae80 msgr2=0x7f6db40ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:53.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.612+0000 7f6daa7fc640 1 --2- 172.21.15.84:0/3795403071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db415ae80 0x7f6db40ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:53.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.612+0000 7f6daa7fc640 1 -- 172.21.15.84:0/3795403071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c0094d0 con 0x7f6db415aaa0 2023-10-16T09:49:53.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.612+0000 7f6daa7fc640 1 --2- 172.21.15.84:0/3795403071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6db415aaa0 0x7f6db4156770 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c000c00 tx=0x7f6d9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:53.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.612+0000 7f6db0ff9640 1 -- 172.21.15.84:0/3795403071 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c031840 con 0x7f6db415aaa0 2023-10-16T09:49:53.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.612+0000 7f6db0ff9640 1 -- 172.21.15.84:0/3795403071 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9c0443f0 con 0x7f6db415aaa0 2023-10-16T09:49:53.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.612+0000 7f6db982d640 1 -- 172.21.15.84:0/3795403071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db41484d0 con 0x7f6db415aaa0 2023-10-16T09:49:53.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.612+0000 7f6db0ff9640 1 -- 172.21.15.84:0/3795403071 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c0447a0 con 0x7f6db415aaa0 2023-10-16T09:49:53.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.612+0000 7f6db982d640 1 -- 172.21.15.84:0/3795403071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db4148a90 con 0x7f6db415aaa0 2023-10-16T09:49:53.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.613+0000 7f6db982d640 1 -- 172.21.15.84:0/3795403071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db4148c80 con 0x7f6db415aaa0 2023-10-16T09:49:53.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.614+0000 7f6db0ff9640 1 -- 172.21.15.84:0/3795403071 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6d9c04d450 con 0x7f6db415aaa0 2023-10-16T09:49:53.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.615+0000 7f6db0ff9640 1 --2- 172.21.15.84:0/3795403071 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6d9406ae70 0x7f6d9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:53.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.615+0000 7f6db0ff9640 1 -- 172.21.15.84:0/3795403071 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f6d9c02d040 con 0x7f6db415aaa0 2023-10-16T09:49:53.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.616+0000 7f6db2ffd640 1 --2- 172.21.15.84:0/3795403071 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6d9406ae70 0x7f6d9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:53.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.616+0000 7f6db2ffd640 1 --2- 172.21.15.84:0/3795403071 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6d9406ae70 0x7f6d9406d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6db41f5fc0 tx=0x7f6da4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:53.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.620+0000 7f6db0ff9640 1 -- 172.21.15.84:0/3795403071 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6d9c076e50 con 0x7f6db415aaa0 2023-10-16T09:49:53.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.812+0000 7f6db982d640 1 -- 172.21.15.84:0/3795403071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6db4148330 con 0x7f6db415aaa0 2023-10-16T09:49:53.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.813+0000 7f6db0ff9640 1 -- 172.21.15.84:0/3795403071 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16730 (secure 0 0 0) 0x7f6db4148330 con 0x7f6db415aaa0 2023-10-16T09:49:53.816 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:53.816 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":23,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:49.552121+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.27","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:53.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.816+0000 7f6db982d640 1 -- 172.21.15.84:0/3795403071 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6d9406ae70 msgr2=0x7f6d9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:53.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.816+0000 7f6db982d640 1 --2- 172.21.15.84:0/3795403071 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6d9406ae70 0x7f6d9406d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6db41f5fc0 tx=0x7f6da4009040 comp rx=0 tx=0).stop 2023-10-16T09:49:53.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.816+0000 7f6db982d640 1 -- 172.21.15.84:0/3795403071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6db415aaa0 msgr2=0x7f6db4156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:53.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.816+0000 7f6db982d640 1 --2- 172.21.15.84:0/3795403071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6db415aaa0 0x7f6db4156770 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c000c00 tx=0x7f6d9c0046c0 comp rx=0 tx=0).stop 2023-10-16T09:49:53.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.817+0000 7f6db982d640 1 -- 172.21.15.84:0/3795403071 shutdown_connections 2023-10-16T09:49:53.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.817+0000 7f6db982d640 1 --2- 172.21.15.84:0/3795403071 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6d9406ae70 0x7f6d9406d330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:53.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.817+0000 7f6db982d640 1 --2- 172.21.15.84:0/3795403071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db415ae80 0x7f6db40ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:53.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.817+0000 7f6db982d640 1 --2- 172.21.15.84:0/3795403071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6db415aaa0 0x7f6db4156770 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:53.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.817+0000 7f6db982d640 1 --2- 172.21.15.84:0/3795403071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6db40ce390 0x7f6db4147690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:53.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.817+0000 7f6db982d640 1 -- 172.21.15.84:0/3795403071 >> 172.21.15.84:0/3795403071 conn(0x7f6db40cb410 msgr2=0x7f6db40b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:53.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.817+0000 7f6db982d640 1 -- 172.21.15.84:0/3795403071 shutdown_connections 2023-10-16T09:49:53.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:53.817+0000 7f6db982d640 1 -- 172.21.15.84:0/3795403071 wait complete. 2023-10-16T09:49:53.839 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:49:00.048649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:49:01.361365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:49:53.840 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:49:54.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.049+0000 7f1bf33e9640 1 Processor -- start 2023-10-16T09:49:54.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.050+0000 7f1bf33e9640 1 -- start start 2023-10-16T09:49:54.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.051+0000 7f1bf33e9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1bec15cc00 0x7f1bec0ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.051+0000 7f1bf33e9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1bec0cd260 0x7f1bec0ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.051+0000 7f1bf33e9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bec0ce480 con 0x7f1bec0ad670 2023-10-16T09:49:54.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.051+0000 7f1bf33e9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bec0ca950 con 0x7f1bec15cc00 2023-10-16T09:49:54.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.051+0000 7f1bf33e9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bec0cd7b0 con 0x7f1bec0cd260 2023-10-16T09:49:54.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.052+0000 7f1bf23e7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1bec15cc00 0x7f1bec0ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:54.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.052+0000 7f1bf23e7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1bec15cc00 0x7f1bec0ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59946/0 (socket says 172.21.15.84:59946) 2023-10-16T09:49:54.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.052+0000 7f1bf23e7640 1 -- 172.21.15.84:0/3488209542 learned_addr learned my addr 172.21.15.84:0/3488209542 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:54.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.052+0000 7f1bf1be6640 1 --2- 172.21.15.84:0/3488209542 >> v2:172.21.15.157:3300/0 conn(0x7f1bec0cd260 0x7f1bec0ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:54.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.052+0000 7f1bf23e7640 1 -- 172.21.15.84:0/3488209542 >> v2:172.21.15.157:3300/0 conn(0x7f1bec0cd260 msgr2=0x7f1bec0ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:54.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.052+0000 7f1bf23e7640 1 --2- 172.21.15.84:0/3488209542 >> v2:172.21.15.157:3300/0 conn(0x7f1bec0cd260 0x7f1bec0ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.052+0000 7f1bf23e7640 1 -- 172.21.15.84:0/3488209542 >> v1:172.21.15.157:6789/0 conn(0x7f1bec0ad670 legacy=0x7f1bec0c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:54.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.052+0000 7f1bf23e7640 1 -- 172.21.15.84:0/3488209542 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec0cda90 con 0x7f1bec15cc00 2023-10-16T09:49:54.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.053+0000 7f1bf23e7640 1 --2- 172.21.15.84:0/3488209542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bec15cc00 0x7f1bec0ca410 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009870 tx=0x7f1be802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538e5098fa62f388 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:54.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.054+0000 7f1bf13e5640 1 -- 172.21.15.84:0/3488209542 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be8031870 con 0x7f1bec15cc00 2023-10-16T09:49:54.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.054+0000 7f1bf13e5640 1 -- 172.21.15.84:0/3488209542 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be80319d0 con 0x7f1bec15cc00 2023-10-16T09:49:54.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.054+0000 7f1bf13e5640 1 -- 172.21.15.84:0/3488209542 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be8031d60 con 0x7f1bec15cc00 2023-10-16T09:49:54.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.054+0000 7f1bf33e9640 1 -- 172.21.15.84:0/3488209542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bec15cc00 msgr2=0x7f1bec0ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:54.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.054+0000 7f1bf33e9640 1 --2- 172.21.15.84:0/3488209542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bec15cc00 0x7f1bec0ca410 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009870 tx=0x7f1be802b910 comp rx=0 tx=0).stop 2023-10-16T09:49:54.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.054+0000 7f1bf33e9640 1 -- 172.21.15.84:0/3488209542 shutdown_connections 2023-10-16T09:49:54.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.054+0000 7f1bf33e9640 1 --2- 172.21.15.84:0/3488209542 >> v2:172.21.15.157:3300/0 conn(0x7f1bec0cd260 0x7f1bec0ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.054+0000 7f1bf33e9640 1 --2- 172.21.15.84:0/3488209542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bec15cc00 0x7f1bec0ca410 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.054+0000 7f1bf33e9640 1 -- 172.21.15.84:0/3488209542 >> 172.21.15.84:0/3488209542 conn(0x7f1bec0b0170 msgr2=0x7f1bec0b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:54.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.054+0000 7f1bf33e9640 1 -- 172.21.15.84:0/3488209542 shutdown_connections 2023-10-16T09:49:54.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.054+0000 7f1bf33e9640 1 -- 172.21.15.84:0/3488209542 wait complete. 2023-10-16T09:49:54.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.056+0000 7f1bf33e9640 1 Processor -- start 2023-10-16T09:49:54.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.056+0000 7f1bf33e9640 1 -- start start 2023-10-16T09:49:54.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.057+0000 7f1bf33e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bec0ad670 0x7f1bec15b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.057+0000 7f1bf33e9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bec0cd260 0x7f1bec15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.057+0000 7f1bf33e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bec15cc00 0x7f1bec15f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.057+0000 7f1bf33e9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec0c2490 con 0x7f1bec15cc00 2023-10-16T09:49:54.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.057+0000 7f1bf33e9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec0b1e50 con 0x7f1bec0ad670 2023-10-16T09:49:54.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.057+0000 7f1bf33e9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec15bce0 con 0x7f1bec0cd260 2023-10-16T09:49:54.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.057+0000 7f1bf2be8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bec15cc00 0x7f1bec15f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:54.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.057+0000 7f1bf2be8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bec15cc00 0x7f1bec15f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59956/0 (socket says 172.21.15.84:59956) 2023-10-16T09:49:54.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.057+0000 7f1bf2be8640 1 -- 172.21.15.84:0/1764923860 learned_addr learned my addr 172.21.15.84:0/1764923860 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:54.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.058+0000 7f1bf1be6640 1 --2- 172.21.15.84:0/1764923860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bec0cd260 0x7f1bec15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:54.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.058+0000 7f1bf23e7640 1 --2- 172.21.15.84:0/1764923860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bec0ad670 0x7f1bec15b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:54.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.058+0000 7f1bf2be8640 1 -- 172.21.15.84:0/1764923860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bec0cd260 msgr2=0x7f1bec15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:54.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.058+0000 7f1bf2be8640 1 --2- 172.21.15.84:0/1764923860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bec0cd260 0x7f1bec15efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.058+0000 7f1bf2be8640 1 -- 172.21.15.84:0/1764923860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bec0ad670 msgr2=0x7f1bec15b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:54.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.058+0000 7f1bf2be8640 1 --2- 172.21.15.84:0/1764923860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bec0ad670 0x7f1bec15b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.058+0000 7f1bf2be8640 1 -- 172.21.15.84:0/1764923860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be80094d0 con 0x7f1bec15cc00 2023-10-16T09:49:54.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.058+0000 7f1bf23e7640 1 --2- 172.21.15.84:0/1764923860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bec0ad670 0x7f1bec15b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:54.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.058+0000 7f1bf2be8640 1 --2- 172.21.15.84:0/1764923860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bec15cc00 0x7f1bec15f4e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1be400c6d0 tx=0x7f1be400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:54.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.059+0000 7f1be2ffd640 1 -- 172.21.15.84:0/1764923860 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be400a840 con 0x7f1bec15cc00 2023-10-16T09:49:54.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.059+0000 7f1be2ffd640 1 -- 172.21.15.84:0/1764923860 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be400a9a0 con 0x7f1bec15cc00 2023-10-16T09:49:54.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.059+0000 7f1be2ffd640 1 -- 172.21.15.84:0/1764923860 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be400ad90 con 0x7f1bec15cc00 2023-10-16T09:49:54.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.059+0000 7f1bf33e9640 1 -- 172.21.15.84:0/1764923860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bec0b1b70 con 0x7f1bec15cc00 2023-10-16T09:49:54.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.059+0000 7f1bf33e9640 1 -- 172.21.15.84:0/1764923860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec1e6a70 con 0x7f1bec15cc00 2023-10-16T09:49:54.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.060+0000 7f1bf33e9640 1 -- 172.21.15.84:0/1764923860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bec04e9b0 con 0x7f1bec15cc00 2023-10-16T09:49:54.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.061+0000 7f1be2ffd640 1 -- 172.21.15.84:0/1764923860 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1be4012630 con 0x7f1bec15cc00 2023-10-16T09:49:54.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.062+0000 7f1be2ffd640 1 --2- 172.21.15.84:0/1764923860 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f1bd006ae70 0x7f1bd006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.062+0000 7f1be2ffd640 1 -- 172.21.15.84:0/1764923860 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f1be4089d20 con 0x7f1bec15cc00 2023-10-16T09:49:54.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.062+0000 7f1bf23e7640 1 --2- 172.21.15.84:0/1764923860 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f1bd006ae70 0x7f1bd006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:54.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.063+0000 7f1bf23e7640 1 --2- 172.21.15.84:0/1764923860 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f1bd006ae70 0x7f1bd006d330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1be8036000 tx=0x7f1be803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:54.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.066+0000 7f1be2ffd640 1 -- 172.21.15.84:0/1764923860 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1be4054df0 con 0x7f1bec15cc00 2023-10-16T09:49:54.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.257+0000 7f1bf33e9640 1 -- 172.21.15.84:0/1764923860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1bec06c460 con 0x7f1bec15cc00 2023-10-16T09:49:54.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.258+0000 7f1be2ffd640 1 -- 172.21.15.84:0/1764923860 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1be4016070 con 0x7f1bec15cc00 2023-10-16T09:49:54.260 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T09:49:54.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.260+0000 7f1bf33e9640 1 -- 172.21.15.84:0/1764923860 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f1bd006ae70 msgr2=0x7f1bd006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:54.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.260+0000 7f1bf33e9640 1 --2- 172.21.15.84:0/1764923860 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f1bd006ae70 0x7f1bd006d330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1be8036000 tx=0x7f1be803c040 comp rx=0 tx=0).stop 2023-10-16T09:49:54.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.260+0000 7f1bf33e9640 1 -- 172.21.15.84:0/1764923860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bec15cc00 msgr2=0x7f1bec15f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:54.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.260+0000 7f1bf33e9640 1 --2- 172.21.15.84:0/1764923860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bec15cc00 0x7f1bec15f4e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1be400c6d0 tx=0x7f1be400cb90 comp rx=0 tx=0).stop 2023-10-16T09:49:54.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.260+0000 7f1bf33e9640 1 -- 172.21.15.84:0/1764923860 shutdown_connections 2023-10-16T09:49:54.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.260+0000 7f1bf33e9640 1 --2- 172.21.15.84:0/1764923860 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f1bd006ae70 0x7f1bd006d330 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.260+0000 7f1bf33e9640 1 --2- 172.21.15.84:0/1764923860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bec15cc00 0x7f1bec15f4e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.260+0000 7f1bf33e9640 1 --2- 172.21.15.84:0/1764923860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bec0cd260 0x7f1bec15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.260+0000 7f1bf33e9640 1 --2- 172.21.15.84:0/1764923860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bec0ad670 0x7f1bec15b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.260+0000 7f1bf33e9640 1 -- 172.21.15.84:0/1764923860 >> 172.21.15.84:0/1764923860 conn(0x7f1bec0b0170 msgr2=0x7f1bec0cc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:54.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.261+0000 7f1bf33e9640 1 -- 172.21.15.84:0/1764923860 shutdown_connections 2023-10-16T09:49:54.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.261+0000 7f1bf33e9640 1 -- 172.21.15.84:0/1764923860 wait complete. 2023-10-16T09:49:54.279 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:49:54.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.487+0000 7f317539e640 1 Processor -- start 2023-10-16T09:49:54.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.488+0000 7f317539e640 1 -- start start 2023-10-16T09:49:54.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.489+0000 7f317539e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f317015ad70 0x7f3170069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.489+0000 7f317539e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f31700c9ca0 0x7f317006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.489+0000 7f317539e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31700cb030 con 0x7f31700ca3b0 2023-10-16T09:49:54.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.489+0000 7f317539e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31700c91c0 con 0x7f31700c9ca0 2023-10-16T09:49:54.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.489+0000 7f317539e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31700c94a0 con 0x7f317015ad70 2023-10-16T09:49:54.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.490+0000 7f316e7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f31700c9ca0 0x7f317006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:54.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.490+0000 7f316f7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f31700ca3b0 0x7f31700ca840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47180/0 (socket says 172.21.15.84:47180) 2023-10-16T09:49:54.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.490+0000 7f316f7fe640 1 -- 172.21.15.84:0/1624186392 learned_addr learned my addr 172.21.15.84:0/1624186392 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:54.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.490+0000 7f316e7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f31700c9ca0 0x7f317006a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59966/0 (socket says 172.21.15.84:59966) 2023-10-16T09:49:54.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.490+0000 7f316effd640 1 --2- 172.21.15.84:0/1624186392 >> v2:172.21.15.157:3300/0 conn(0x7f317015ad70 0x7f3170069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:54.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.490+0000 7f316e7fc640 1 -- 172.21.15.84:0/1624186392 >> v2:172.21.15.157:3300/0 conn(0x7f317015ad70 msgr2=0x7f3170069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:54.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.490+0000 7f316e7fc640 1 --2- 172.21.15.84:0/1624186392 >> v2:172.21.15.157:3300/0 conn(0x7f317015ad70 0x7f3170069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.490+0000 7f316e7fc640 1 -- 172.21.15.84:0/1624186392 >> v1:172.21.15.84:6789/0 conn(0x7f31700ca3b0 legacy=0x7f31700ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:54.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.490+0000 7f316e7fc640 1 -- 172.21.15.84:0/1624186392 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31700c9780 con 0x7f31700c9ca0 2023-10-16T09:49:54.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.491+0000 7f316e7fc640 1 --2- 172.21.15.84:0/1624186392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31700c9ca0 0x7f317006a2e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f31700cf780 tx=0x7f315802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecd75014a476e6f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:54.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.491+0000 7f316dffb640 1 -- 172.21.15.84:0/1624186392 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158031830 con 0x7f31700c9ca0 2023-10-16T09:49:54.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.491+0000 7f316dffb640 1 -- 172.21.15.84:0/1624186392 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3158031990 con 0x7f31700c9ca0 2023-10-16T09:49:54.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.491+0000 7f316dffb640 1 -- 172.21.15.84:0/1624186392 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158031d20 con 0x7f31700c9ca0 2023-10-16T09:49:54.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.492+0000 7f317539e640 1 -- 172.21.15.84:0/1624186392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31700c9ca0 msgr2=0x7f317006a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:54.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.492+0000 7f317539e640 1 --2- 172.21.15.84:0/1624186392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31700c9ca0 0x7f317006a2e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f31700cf780 tx=0x7f315802b8d0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.492+0000 7f317539e640 1 -- 172.21.15.84:0/1624186392 shutdown_connections 2023-10-16T09:49:54.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.492+0000 7f317539e640 1 --2- 172.21.15.84:0/1624186392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31700c9ca0 0x7f317006a2e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.492+0000 7f317539e640 1 --2- 172.21.15.84:0/1624186392 >> v2:172.21.15.157:3300/0 conn(0x7f317015ad70 0x7f3170069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.492+0000 7f317539e640 1 -- 172.21.15.84:0/1624186392 >> 172.21.15.84:0/1624186392 conn(0x7f31700ce9b0 msgr2=0x7f31700cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:54.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.492+0000 7f317539e640 1 -- 172.21.15.84:0/1624186392 shutdown_connections 2023-10-16T09:49:54.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.492+0000 7f317539e640 1 -- 172.21.15.84:0/1624186392 wait complete. 2023-10-16T09:49:54.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.495+0000 7f317539e640 1 Processor -- start 2023-10-16T09:49:54.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.495+0000 7f317539e640 1 -- start start 2023-10-16T09:49:54.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.496+0000 7f317539e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31700c9ca0 0x7f31700c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.496+0000 7f317539e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31700ca3b0 0x7f3170150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.496+0000 7f317539e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f317015ad70 0x7f31700cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.496+0000 7f317539e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3170150b80 con 0x7f31700ca3b0 2023-10-16T09:49:54.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.496+0000 7f317539e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31700cd580 con 0x7f317015ad70 2023-10-16T09:49:54.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.496+0000 7f317539e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31700c74c0 con 0x7f31700c9ca0 2023-10-16T09:49:54.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.496+0000 7f316e7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31700ca3b0 0x7f3170150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:54.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.496+0000 7f316e7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31700ca3b0 0x7f3170150640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59978/0 (socket says 172.21.15.84:59978) 2023-10-16T09:49:54.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.496+0000 7f316e7fc640 1 -- 172.21.15.84:0/1118276120 learned_addr learned my addr 172.21.15.84:0/1118276120 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:54.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.497+0000 7f316e7fc640 1 -- 172.21.15.84:0/1118276120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31700c9ca0 msgr2=0x7f31700c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:54.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.497+0000 7f316e7fc640 1 --2- 172.21.15.84:0/1118276120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31700c9ca0 0x7f31700c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.497+0000 7f316e7fc640 1 -- 172.21.15.84:0/1118276120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f317015ad70 msgr2=0x7f31700cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:54.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.497+0000 7f316e7fc640 1 --2- 172.21.15.84:0/1118276120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f317015ad70 0x7f31700cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.497+0000 7f316e7fc640 1 -- 172.21.15.84:0/1118276120 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3158009400 con 0x7f31700ca3b0 2023-10-16T09:49:54.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.497+0000 7f316e7fc640 1 --2- 172.21.15.84:0/1118276120 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31700ca3b0 0x7f3170150640 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3158009890 tx=0x7f31580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:54.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.497+0000 7f31677fe640 1 -- 172.21.15.84:0/1118276120 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315803ba70 con 0x7f31700ca3b0 2023-10-16T09:49:54.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.497+0000 7f31677fe640 1 -- 172.21.15.84:0/1118276120 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315803bbd0 con 0x7f31700ca3b0 2023-10-16T09:49:54.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.497+0000 7f31677fe640 1 -- 172.21.15.84:0/1118276120 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31580453f0 con 0x7f31700ca3b0 2023-10-16T09:49:54.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.497+0000 7f317539e640 1 -- 172.21.15.84:0/1118276120 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31700c77a0 con 0x7f31700ca3b0 2023-10-16T09:49:54.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.498+0000 7f317539e640 1 -- 172.21.15.84:0/1118276120 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31700c7d60 con 0x7f31700ca3b0 2023-10-16T09:49:54.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.499+0000 7f317539e640 1 -- 172.21.15.84:0/1118276120 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f317004e160 con 0x7f31700ca3b0 2023-10-16T09:49:54.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.499+0000 7f31677fe640 1 -- 172.21.15.84:0/1118276120 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3158045550 con 0x7f31700ca3b0 2023-10-16T09:49:54.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.500+0000 7f31677fe640 1 --2- 172.21.15.84:0/1118276120 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f315006ae70 0x7f315006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.500+0000 7f31677fe640 1 -- 172.21.15.84:0/1118276120 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f31580aba90 con 0x7f31700ca3b0 2023-10-16T09:49:54.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.501+0000 7f316effd640 1 --2- 172.21.15.84:0/1118276120 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f315006ae70 0x7f315006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:54.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.501+0000 7f316effd640 1 --2- 172.21.15.84:0/1118276120 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f315006ae70 0x7f315006d330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3160007dc0 tx=0x7f31600072f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:54.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.505+0000 7f31677fe640 1 -- 172.21.15.84:0/1118276120 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3158076be0 con 0x7f31700ca3b0 2023-10-16T09:49:54.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.697+0000 7f317539e640 1 -- 172.21.15.84:0/1118276120 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f31700cf1a0 con 0x7f31700ca3b0 2023-10-16T09:49:54.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.698+0000 7f31677fe640 1 -- 172.21.15.84:0/1118276120 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f315807a200 con 0x7f31700ca3b0 2023-10-16T09:49:54.700 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T09:49:54.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.700+0000 7f317539e640 1 -- 172.21.15.84:0/1118276120 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f315006ae70 msgr2=0x7f315006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:54.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.700+0000 7f317539e640 1 --2- 172.21.15.84:0/1118276120 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f315006ae70 0x7f315006d330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3160007dc0 tx=0x7f31600072f0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.700+0000 7f317539e640 1 -- 172.21.15.84:0/1118276120 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31700ca3b0 msgr2=0x7f3170150640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:54.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.700+0000 7f317539e640 1 --2- 172.21.15.84:0/1118276120 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31700ca3b0 0x7f3170150640 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3158009890 tx=0x7f31580046c0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.700+0000 7f317539e640 1 -- 172.21.15.84:0/1118276120 shutdown_connections 2023-10-16T09:49:54.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.701+0000 7f317539e640 1 --2- 172.21.15.84:0/1118276120 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f315006ae70 0x7f315006d330 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.701+0000 7f317539e640 1 --2- 172.21.15.84:0/1118276120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f317015ad70 0x7f31700cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.701+0000 7f317539e640 1 --2- 172.21.15.84:0/1118276120 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31700ca3b0 0x7f3170150640 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.701+0000 7f317539e640 1 --2- 172.21.15.84:0/1118276120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31700c9ca0 0x7f31700c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.701+0000 7f317539e640 1 -- 172.21.15.84:0/1118276120 >> 172.21.15.84:0/1118276120 conn(0x7f31700ce9b0 msgr2=0x7f3170070d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:54.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.701+0000 7f317539e640 1 -- 172.21.15.84:0/1118276120 shutdown_connections 2023-10-16T09:49:54.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.701+0000 7f317539e640 1 -- 172.21.15.84:0/1118276120 wait complete. 2023-10-16T09:49:54.720 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:49:54.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.926+0000 7f7d8f487640 1 Processor -- start 2023-10-16T09:49:54.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.927+0000 7f7d8f487640 1 -- start start 2023-10-16T09:49:54.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.928+0000 7f7d8f487640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d88160ad0 0x7f7d880ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.928+0000 7f7d8f487640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d880ce350 0x7f7d88160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.928+0000 7f7d8f487640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7d880ae260 0x7f7d880ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.928+0000 7f7d8f487640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d880ce800 con 0x7f7d880ae260 2023-10-16T09:49:54.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.928+0000 7f7d8f487640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d880cdec0 con 0x7f7d88160ad0 2023-10-16T09:49:54.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.928+0000 7f7d8f487640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880acdd0 con 0x7f7d880ce350 2023-10-16T09:49:54.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.928+0000 7f7d8da24640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7d880ae260 0x7f7d880ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:54.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.928+0000 7f7d8da24640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7d880ae260 0x7f7d880ccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59990/0 (socket says 172.21.15.84:59990) 2023-10-16T09:49:54.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.928+0000 7f7d8da24640 1 -- 172.21.15.84:0/1449420227 learned_addr learned my addr 172.21.15.84:0/1449420227 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:54.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.929+0000 7f7d8ca22640 1 --2- 172.21.15.84:0/1449420227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d880ce350 0x7f7d88160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:54.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.929+0000 7f7d8d223640 1 --2- 172.21.15.84:0/1449420227 >> v2:172.21.15.157:3300/0 conn(0x7f7d88160ad0 0x7f7d880ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:54.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.929+0000 7f7d8da24640 1 -- 172.21.15.84:0/1449420227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d880ce350 msgr2=0x7f7d88160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:54.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.929+0000 7f7d8da24640 1 --2- 172.21.15.84:0/1449420227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d880ce350 0x7f7d88160eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.929+0000 7f7d8da24640 1 -- 172.21.15.84:0/1449420227 >> v2:172.21.15.157:3300/0 conn(0x7f7d88160ad0 msgr2=0x7f7d880ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:54.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.929+0000 7f7d8da24640 1 --2- 172.21.15.84:0/1449420227 >> v2:172.21.15.157:3300/0 conn(0x7f7d88160ad0 0x7f7d880ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.929+0000 7f7d8da24640 1 -- 172.21.15.84:0/1449420227 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d881613f0 con 0x7f7d880ae260 2023-10-16T09:49:54.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.930+0000 7f7d8da24640 1 --2- 172.21.15.84:0/1449420227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d880ae260 0x7f7d880ccd80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7d880721f0 tx=0x7f7d8002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf199d375907f9c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:54.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.930+0000 7f7d87fff640 1 -- 172.21.15.84:0/1449420227 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80031870 con 0x7f7d880ae260 2023-10-16T09:49:54.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.930+0000 7f7d87fff640 1 -- 172.21.15.84:0/1449420227 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d800319d0 con 0x7f7d880ae260 2023-10-16T09:49:54.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.930+0000 7f7d87fff640 1 -- 172.21.15.84:0/1449420227 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80031d60 con 0x7f7d880ae260 2023-10-16T09:49:54.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.931+0000 7f7d8f487640 1 -- 172.21.15.84:0/1449420227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d880ae260 msgr2=0x7f7d880ccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:54.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.931+0000 7f7d8f487640 1 --2- 172.21.15.84:0/1449420227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d880ae260 0x7f7d880ccd80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7d880721f0 tx=0x7f7d8002b910 comp rx=0 tx=0).stop 2023-10-16T09:49:54.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.931+0000 7f7d8f487640 1 -- 172.21.15.84:0/1449420227 shutdown_connections 2023-10-16T09:49:54.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.931+0000 7f7d8f487640 1 --2- 172.21.15.84:0/1449420227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d880ae260 0x7f7d880ccd80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.931+0000 7f7d8f487640 1 --2- 172.21.15.84:0/1449420227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d880ce350 0x7f7d88160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.931+0000 7f7d8f487640 1 --2- 172.21.15.84:0/1449420227 >> v2:172.21.15.157:3300/0 conn(0x7f7d88160ad0 0x7f7d880ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.931+0000 7f7d8f487640 1 -- 172.21.15.84:0/1449420227 >> 172.21.15.84:0/1449420227 conn(0x7f7d8806bc10 msgr2=0x7f7d8806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:54.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.931+0000 7f7d8f487640 1 -- 172.21.15.84:0/1449420227 shutdown_connections 2023-10-16T09:49:54.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.931+0000 7f7d8f487640 1 -- 172.21.15.84:0/1449420227 wait complete. 2023-10-16T09:49:54.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.933+0000 7f7d8f487640 1 Processor -- start 2023-10-16T09:49:54.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.933+0000 7f7d8f487640 1 -- start start 2023-10-16T09:49:54.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.934+0000 7f7d8f487640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d880ae260 0x7f7d880c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.934+0000 7f7d8f487640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d880ce350 0x7f7d881584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.934+0000 7f7d8f487640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d88160ad0 0x7f7d880b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.934+0000 7f7d8f487640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880c2ef0 con 0x7f7d88160ad0 2023-10-16T09:49:54.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.934+0000 7f7d8f487640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880c3140 con 0x7f7d880ae260 2023-10-16T09:49:54.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.934+0000 7f7d8f487640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880c3420 con 0x7f7d880ce350 2023-10-16T09:49:54.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.934+0000 7f7d8da24640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d88160ad0 0x7f7d880b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:54.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.934+0000 7f7d8da24640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d88160ad0 0x7f7d880b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59994/0 (socket says 172.21.15.84:59994) 2023-10-16T09:49:54.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.934+0000 7f7d8da24640 1 -- 172.21.15.84:0/85595893 learned_addr learned my addr 172.21.15.84:0/85595893 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:54.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.934+0000 7f7d8da24640 1 -- 172.21.15.84:0/85595893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d880ce350 msgr2=0x7f7d881584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:54.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.934+0000 7f7d8da24640 1 --2- 172.21.15.84:0/85595893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d880ce350 0x7f7d881584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.934+0000 7f7d8d223640 1 --2- 172.21.15.84:0/85595893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d880ae260 0x7f7d880c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:54.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.934+0000 7f7d8da24640 1 -- 172.21.15.84:0/85595893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d880ae260 msgr2=0x7f7d880c41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:54.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.934+0000 7f7d8da24640 1 --2- 172.21.15.84:0/85595893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d880ae260 0x7f7d880c41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:54.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.934+0000 7f7d8da24640 1 -- 172.21.15.84:0/85595893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d800094d0 con 0x7f7d88160ad0 2023-10-16T09:49:54.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.935+0000 7f7d8da24640 1 --2- 172.21.15.84:0/85595893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d88160ad0 0x7f7d880b44b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7d800090b0 tx=0x7f7d800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:54.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.935+0000 7f7d85ffb640 1 -- 172.21.15.84:0/85595893 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80031800 con 0x7f7d88160ad0 2023-10-16T09:49:54.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.935+0000 7f7d85ffb640 1 -- 172.21.15.84:0/85595893 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d800443f0 con 0x7f7d88160ad0 2023-10-16T09:49:54.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.935+0000 7f7d85ffb640 1 -- 172.21.15.84:0/85595893 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d800447a0 con 0x7f7d88160ad0 2023-10-16T09:49:54.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.935+0000 7f7d8f487640 1 -- 172.21.15.84:0/85595893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d880c3700 con 0x7f7d88160ad0 2023-10-16T09:49:54.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.935+0000 7f7d8f487640 1 -- 172.21.15.84:0/85595893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d880c3cc0 con 0x7f7d88160ad0 2023-10-16T09:49:54.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.936+0000 7f7d8f487640 1 -- 172.21.15.84:0/85595893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d880c3eb0 con 0x7f7d88160ad0 2023-10-16T09:49:54.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.937+0000 7f7d85ffb640 1 -- 172.21.15.84:0/85595893 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7d8004d450 con 0x7f7d88160ad0 2023-10-16T09:49:54.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.938+0000 7f7d85ffb640 1 --2- 172.21.15.84:0/85595893 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7d7406ae70 0x7f7d7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:54.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.938+0000 7f7d85ffb640 1 -- 172.21.15.84:0/85595893 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f7d8002d040 con 0x7f7d88160ad0 2023-10-16T09:49:54.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.938+0000 7f7d8d223640 1 --2- 172.21.15.84:0/85595893 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7d7406ae70 0x7f7d7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:54.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.939+0000 7f7d8d223640 1 --2- 172.21.15.84:0/85595893 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7d7406ae70 0x7f7d7406d330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7d78008ef0 tx=0x7f7d78008440 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:54.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:54.942+0000 7f7d85ffb640 1 -- 172.21.15.84:0/85595893 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7d80076db0 con 0x7f7d88160ad0 2023-10-16T09:49:55.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.119+0000 7f7d8f487640 1 -- 172.21.15.84:0/85595893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7d8804e160 con 0x7f7d88160ad0 2023-10-16T09:49:55.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.120+0000 7f7d85ffb640 1 -- 172.21.15.84:0/85595893 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7d80035020 con 0x7f7d88160ad0 2023-10-16T09:49:55.122 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T09:49:55.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.122+0000 7f7d8f487640 1 -- 172.21.15.84:0/85595893 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7d7406ae70 msgr2=0x7f7d7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:55.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.122+0000 7f7d8f487640 1 --2- 172.21.15.84:0/85595893 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7d7406ae70 0x7f7d7406d330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7d78008ef0 tx=0x7f7d78008440 comp rx=0 tx=0).stop 2023-10-16T09:49:55.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.122+0000 7f7d8f487640 1 -- 172.21.15.84:0/85595893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d88160ad0 msgr2=0x7f7d880b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:55.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.122+0000 7f7d8f487640 1 --2- 172.21.15.84:0/85595893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d88160ad0 0x7f7d880b44b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7d800090b0 tx=0x7f7d800046c0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.122+0000 7f7d8f487640 1 -- 172.21.15.84:0/85595893 shutdown_connections 2023-10-16T09:49:55.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.122+0000 7f7d8f487640 1 --2- 172.21.15.84:0/85595893 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7d7406ae70 0x7f7d7406d330 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.122+0000 7f7d8f487640 1 --2- 172.21.15.84:0/85595893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d88160ad0 0x7f7d880b44b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.122+0000 7f7d8f487640 1 --2- 172.21.15.84:0/85595893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d880ce350 0x7f7d881584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.122+0000 7f7d8f487640 1 --2- 172.21.15.84:0/85595893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d880ae260 0x7f7d880c41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.122+0000 7f7d8f487640 1 -- 172.21.15.84:0/85595893 >> 172.21.15.84:0/85595893 conn(0x7f7d8806bc10 msgr2=0x7f7d880ca4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:55.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.123+0000 7f7d8f487640 1 -- 172.21.15.84:0/85595893 shutdown_connections 2023-10-16T09:49:55.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.123+0000 7f7d8f487640 1 -- 172.21.15.84:0/85595893 wait complete. 2023-10-16T09:49:55.143 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T09:49:55.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.352+0000 7fe16947c640 1 Processor -- start 2023-10-16T09:49:55.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.352+0000 7fe16947c640 1 -- start start 2023-10-16T09:49:55.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.353+0000 7fe16947c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe164071280 0x7fe1640afca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:55.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.353+0000 7fe16947c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1640b01e0 0x7fe1640cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:55.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.353+0000 7fe16947c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1640b1720 con 0x7fe164159890 2023-10-16T09:49:55.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.353+0000 7fe16947c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1640b1a00 con 0x7fe164071280 2023-10-16T09:49:55.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.353+0000 7fe16947c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1640b1ce0 con 0x7fe1640b01e0 2023-10-16T09:49:55.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.354+0000 7fe163fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe164159890 0x7fe164070b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47194/0 (socket says 172.21.15.84:47194) 2023-10-16T09:49:55.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.354+0000 7fe163fff640 1 -- 172.21.15.84:0/3995261442 learned_addr learned my addr 172.21.15.84:0/3995261442 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:55.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.354+0000 7fe1637fe640 1 --2- 172.21.15.84:0/3995261442 >> v2:172.21.15.84:3300/0 conn(0x7fe164071280 0x7fe1640afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:55.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.354+0000 7fe1637fe640 1 -- 172.21.15.84:0/3995261442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1640b01e0 msgr2=0x7fe1640cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:55.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.354+0000 7fe1637fe640 1 --2- 172.21.15.84:0/3995261442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1640b01e0 0x7fe1640cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.354+0000 7fe1637fe640 1 -- 172.21.15.84:0/3995261442 >> v1:172.21.15.84:6789/0 conn(0x7fe164159890 legacy=0x7fe164070b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:55.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.354+0000 7fe1637fe640 1 -- 172.21.15.84:0/3995261442 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1640b1fc0 con 0x7fe164071280 2023-10-16T09:49:55.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.355+0000 7fe1637fe640 1 --2- 172.21.15.84:0/3995261442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe164071280 0x7fe1640afca0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe154009d90 tx=0x7fe15402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ded5e487e17a59 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:55.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.356+0000 7fe162ffd640 1 -- 172.21.15.84:0/3995261442 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15402cc60 con 0x7fe164071280 2023-10-16T09:49:55.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.356+0000 7fe162ffd640 1 -- 172.21.15.84:0/3995261442 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15402cdc0 con 0x7fe164071280 2023-10-16T09:49:55.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.356+0000 7fe162ffd640 1 -- 172.21.15.84:0/3995261442 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154034660 con 0x7fe164071280 2023-10-16T09:49:55.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.356+0000 7fe16947c640 1 -- 172.21.15.84:0/3995261442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe164071280 msgr2=0x7fe1640afca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:55.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.356+0000 7fe16947c640 1 --2- 172.21.15.84:0/3995261442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe164071280 0x7fe1640afca0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe154009d90 tx=0x7fe15402beb0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.356+0000 7fe16947c640 1 -- 172.21.15.84:0/3995261442 shutdown_connections 2023-10-16T09:49:55.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.356+0000 7fe16947c640 1 --2- 172.21.15.84:0/3995261442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1640b01e0 0x7fe1640cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.356+0000 7fe16947c640 1 --2- 172.21.15.84:0/3995261442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe164071280 0x7fe1640afca0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.356+0000 7fe16947c640 1 -- 172.21.15.84:0/3995261442 >> 172.21.15.84:0/3995261442 conn(0x7fe164069da0 msgr2=0x7fe16406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:55.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.356+0000 7fe16947c640 1 -- 172.21.15.84:0/3995261442 shutdown_connections 2023-10-16T09:49:55.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.356+0000 7fe16947c640 1 -- 172.21.15.84:0/3995261442 wait complete. 2023-10-16T09:49:55.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.358+0000 7fe16947c640 1 Processor -- start 2023-10-16T09:49:55.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.358+0000 7fe16947c640 1 -- start start 2023-10-16T09:49:55.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.359+0000 7fe16947c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe164071280 0x7fe16414b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:55.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.359+0000 7fe16947c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1640b01e0 0x7fe16406c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:55.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.359+0000 7fe16947c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe164159890 0x7fe164142ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:55.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.359+0000 7fe16947c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16414a930 con 0x7fe1640b01e0 2023-10-16T09:49:55.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.359+0000 7fe16947c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe164149db0 con 0x7fe164159890 2023-10-16T09:49:55.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.359+0000 7fe16947c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16414a090 con 0x7fe164071280 2023-10-16T09:49:55.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.359+0000 7fe1637fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1640b01e0 0x7fe16406c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:55.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.359+0000 7fe1637fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1640b01e0 0x7fe16406c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60014/0 (socket says 172.21.15.84:60014) 2023-10-16T09:49:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.359+0000 7fe1637fe640 1 -- 172.21.15.84:0/4013619224 learned_addr learned my addr 172.21.15.84:0/4013619224 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.360+0000 7fe163fff640 1 --2- 172.21.15.84:0/4013619224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe164071280 0x7fe16414b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.360+0000 7fe1637fe640 1 -- 172.21.15.84:0/4013619224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe164071280 msgr2=0x7fe16414b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.360+0000 7fe1637fe640 1 --2- 172.21.15.84:0/4013619224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe164071280 0x7fe16414b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.360+0000 7fe1637fe640 1 -- 172.21.15.84:0/4013619224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe164159890 msgr2=0x7fe164142ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.360+0000 7fe168c7b640 1 --2- 172.21.15.84:0/4013619224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe164159890 0x7fe164142ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.360+0000 7fe1637fe640 1 --2- 172.21.15.84:0/4013619224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe164159890 0x7fe164142ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.360+0000 7fe1637fe640 1 -- 172.21.15.84:0/4013619224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe154009a70 con 0x7fe1640b01e0 2023-10-16T09:49:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.360+0000 7fe1637fe640 1 --2- 172.21.15.84:0/4013619224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1640b01e0 0x7fe16406c820 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe154037040 tx=0x7fe154004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.360+0000 7fe160ff9640 1 -- 172.21.15.84:0/4013619224 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154040650 con 0x7fe1640b01e0 2023-10-16T09:49:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.361+0000 7fe160ff9640 1 -- 172.21.15.84:0/4013619224 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1540407b0 con 0x7fe1640b01e0 2023-10-16T09:49:55.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.361+0000 7fe160ff9640 1 -- 172.21.15.84:0/4013619224 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154040b60 con 0x7fe1640b01e0 2023-10-16T09:49:55.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.361+0000 7fe16947c640 1 -- 172.21.15.84:0/4013619224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16414a370 con 0x7fe1640b01e0 2023-10-16T09:49:55.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.361+0000 7fe16947c640 1 -- 172.21.15.84:0/4013619224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16414af20 con 0x7fe1640b01e0 2023-10-16T09:49:55.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.362+0000 7fe16947c640 1 -- 172.21.15.84:0/4013619224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16404e9b0 con 0x7fe1640b01e0 2023-10-16T09:49:55.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.363+0000 7fe160ff9640 1 -- 172.21.15.84:0/4013619224 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe154040cc0 con 0x7fe1640b01e0 2023-10-16T09:49:55.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.364+0000 7fe160ff9640 1 --2- 172.21.15.84:0/4013619224 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe14406ae70 0x7fe14406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:55.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.364+0000 7fe160ff9640 1 -- 172.21.15.84:0/4013619224 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe1540ac690 con 0x7fe1640b01e0 2023-10-16T09:49:55.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.364+0000 7fe163fff640 1 --2- 172.21.15.84:0/4013619224 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe14406ae70 0x7fe14406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:55.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.365+0000 7fe163fff640 1 --2- 172.21.15.84:0/4013619224 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe14406ae70 0x7fe14406d330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe14c005880 tx=0x7fe14c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:55.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.368+0000 7fe160ff9640 1 -- 172.21.15.84:0/4013619224 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe154077620 con 0x7fe1640b01e0 2023-10-16T09:49:55.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.573+0000 7fe16947c640 1 -- 172.21.15.84:0/4013619224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe1640ca560 con 0x7fe1640b01e0 2023-10-16T09:49:55.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.574+0000 7fe160ff9640 1 -- 172.21.15.84:0/4013619224 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe154081080 con 0x7fe1640b01e0 2023-10-16T09:49:55.576 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-16T09:49:55.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.576+0000 7fe16947c640 1 -- 172.21.15.84:0/4013619224 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe14406ae70 msgr2=0x7fe14406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:55.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.576+0000 7fe16947c640 1 --2- 172.21.15.84:0/4013619224 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe14406ae70 0x7fe14406d330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe14c005880 tx=0x7fe14c009040 comp rx=0 tx=0).stop 2023-10-16T09:49:55.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.576+0000 7fe16947c640 1 -- 172.21.15.84:0/4013619224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1640b01e0 msgr2=0x7fe16406c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:55.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.576+0000 7fe16947c640 1 --2- 172.21.15.84:0/4013619224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1640b01e0 0x7fe16406c820 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe154037040 tx=0x7fe154004a40 comp rx=0 tx=0).stop 2023-10-16T09:49:55.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.576+0000 7fe16947c640 1 -- 172.21.15.84:0/4013619224 shutdown_connections 2023-10-16T09:49:55.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.576+0000 7fe16947c640 1 --2- 172.21.15.84:0/4013619224 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fe14406ae70 0x7fe14406d330 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.576+0000 7fe16947c640 1 --2- 172.21.15.84:0/4013619224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe164159890 0x7fe164142ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.576+0000 7fe16947c640 1 --2- 172.21.15.84:0/4013619224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1640b01e0 0x7fe16406c820 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.576+0000 7fe16947c640 1 --2- 172.21.15.84:0/4013619224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe164071280 0x7fe16414b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.576+0000 7fe16947c640 1 -- 172.21.15.84:0/4013619224 >> 172.21.15.84:0/4013619224 conn(0x7fe164069da0 msgr2=0x7fe16406fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:55.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.577+0000 7fe16947c640 1 -- 172.21.15.84:0/4013619224 shutdown_connections 2023-10-16T09:49:55.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.577+0000 7fe16947c640 1 -- 172.21.15.84:0/4013619224 wait complete. 2023-10-16T09:49:55.597 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-16T09:49:55.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:55.597 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-16T09:49:55.707 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-10-16T09:49:55.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:49:55.708 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-16T09:49:55.759 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T09:49:55.759 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:49:55.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.983+0000 7fb396800640 1 Processor -- start 2023-10-16T09:49:55.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.983+0000 7fb396800640 1 -- start start 2023-10-16T09:49:55.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.984+0000 7fb396800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb3900cd0c0 0x7fb3901615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:55.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.985+0000 7fb396800640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb3900b0410 0x7fb3900b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:55.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.985+0000 7fb396800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb39015d3f0 0x7fb39015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:55.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.985+0000 7fb396800640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3900b0de0 con 0x7fb3900b0410 2023-10-16T09:49:55.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.985+0000 7fb396800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb39015e3e0 con 0x7fb3900cd0c0 2023-10-16T09:49:55.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.985+0000 7fb396800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900cd4a0 con 0x7fb39015d3f0 2023-10-16T09:49:55.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.985+0000 7fb38f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb3900b0410 0x7fb3900b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:55.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.985+0000 7fb38f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb3900b0410 0x7fb3900b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38188/0 (socket says 172.21.15.84:38188) 2023-10-16T09:49:55.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.985+0000 7fb38f7fe640 1 -- 172.21.15.84:0/3939954000 learned_addr learned my addr 172.21.15.84:0/3939954000 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:55.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.985+0000 7fb394d9d640 1 --2- 172.21.15.84:0/3939954000 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb39015d3f0 0x7fb39015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:55.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.985+0000 7fb38ffff640 1 --2- 172.21.15.84:0/3939954000 >> v2:172.21.15.157:3300/0 conn(0x7fb3900cd0c0 0x7fb3901615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:55.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.986+0000 7fb38f7fe640 1 -- 172.21.15.84:0/3939954000 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb39015d3f0 msgr2=0x7fb39015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:55.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.986+0000 7fb38f7fe640 1 --2- 172.21.15.84:0/3939954000 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb39015d3f0 0x7fb39015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.986+0000 7fb38f7fe640 1 -- 172.21.15.84:0/3939954000 >> v2:172.21.15.157:3300/0 conn(0x7fb3900cd0c0 msgr2=0x7fb3901615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:55.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.986+0000 7fb38f7fe640 1 --2- 172.21.15.84:0/3939954000 >> v2:172.21.15.157:3300/0 conn(0x7fb3900cd0c0 0x7fb3901615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.986+0000 7fb38f7fe640 1 -- 172.21.15.84:0/3939954000 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39015e100 con 0x7fb3900b0410 2023-10-16T09:49:55.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.987+0000 7fb38f7fe640 1 --2- 172.21.15.84:0/3939954000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3900b0410 0x7fb3900b08a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb3900cc4e0 tx=0x7fb38402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5690954882309fd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:55.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.987+0000 7fb38effd640 1 -- 172.21.15.84:0/3939954000 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3840317e0 con 0x7fb3900b0410 2023-10-16T09:49:55.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.987+0000 7fb38effd640 1 -- 172.21.15.84:0/3939954000 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb384031940 con 0x7fb3900b0410 2023-10-16T09:49:55.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.987+0000 7fb38effd640 1 -- 172.21.15.84:0/3939954000 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb384031cd0 con 0x7fb3900b0410 2023-10-16T09:49:55.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.988+0000 7fb396800640 1 -- 172.21.15.84:0/3939954000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3900b0410 msgr2=0x7fb3900b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:55.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.988+0000 7fb396800640 1 --2- 172.21.15.84:0/3939954000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3900b0410 0x7fb3900b08a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb3900cc4e0 tx=0x7fb38402b880 comp rx=0 tx=0).stop 2023-10-16T09:49:55.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.988+0000 7fb396800640 1 -- 172.21.15.84:0/3939954000 shutdown_connections 2023-10-16T09:49:55.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.988+0000 7fb396800640 1 --2- 172.21.15.84:0/3939954000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3900b0410 0x7fb3900b08a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.988+0000 7fb396800640 1 --2- 172.21.15.84:0/3939954000 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb39015d3f0 0x7fb39015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.988+0000 7fb396800640 1 --2- 172.21.15.84:0/3939954000 >> v2:172.21.15.157:3300/0 conn(0x7fb3900cd0c0 0x7fb3901615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.988+0000 7fb396800640 1 -- 172.21.15.84:0/3939954000 >> 172.21.15.84:0/3939954000 conn(0x7fb3900cb410 msgr2=0x7fb3900cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:55.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.988+0000 7fb396800640 1 -- 172.21.15.84:0/3939954000 shutdown_connections 2023-10-16T09:49:55.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.988+0000 7fb396800640 1 -- 172.21.15.84:0/3939954000 wait complete. 2023-10-16T09:49:55.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.990+0000 7fb396800640 1 Processor -- start 2023-10-16T09:49:55.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.990+0000 7fb396800640 1 -- start start 2023-10-16T09:49:55.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.991+0000 7fb396800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3900b0410 0x7fb390155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:55.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.991+0000 7fb396800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3900cd0c0 0x7fb390155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:55.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.991+0000 7fb396800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb39015d3f0 0x7fb3900c2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:55.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.991+0000 7fb396800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb390151790 con 0x7fb3900b0410 2023-10-16T09:49:55.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.991+0000 7fb396800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900c34e0 con 0x7fb39015d3f0 2023-10-16T09:49:55.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.991+0000 7fb396800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3901561d0 con 0x7fb3900cd0c0 2023-10-16T09:49:55.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.992+0000 7fb38ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3900b0410 0x7fb390155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:55.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.992+0000 7fb38ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3900b0410 0x7fb390155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38200/0 (socket says 172.21.15.84:38200) 2023-10-16T09:49:55.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.992+0000 7fb38ffff640 1 -- 172.21.15.84:0/723259565 learned_addr learned my addr 172.21.15.84:0/723259565 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:55.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.992+0000 7fb38ffff640 1 -- 172.21.15.84:0/723259565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3900cd0c0 msgr2=0x7fb390155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:55.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.992+0000 7fb38ffff640 1 --2- 172.21.15.84:0/723259565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3900cd0c0 0x7fb390155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.992+0000 7fb38ffff640 1 -- 172.21.15.84:0/723259565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb39015d3f0 msgr2=0x7fb3900c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:55.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.992+0000 7fb38ffff640 1 --2- 172.21.15.84:0/723259565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb39015d3f0 0x7fb3900c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:55.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.992+0000 7fb38ffff640 1 -- 172.21.15.84:0/723259565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3840094d0 con 0x7fb3900b0410 2023-10-16T09:49:55.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.992+0000 7fb38ffff640 1 --2- 172.21.15.84:0/723259565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3900b0410 0x7fb390155660 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb38000d800 tx=0x7fb38000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:55.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.993+0000 7fb38cff9640 1 -- 172.21.15.84:0/723259565 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3800096a0 con 0x7fb3900b0410 2023-10-16T09:49:55.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.993+0000 7fb38cff9640 1 -- 172.21.15.84:0/723259565 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb380009800 con 0x7fb3900b0410 2023-10-16T09:49:55.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.993+0000 7fb396800640 1 -- 172.21.15.84:0/723259565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3901564b0 con 0x7fb3900b0410 2023-10-16T09:49:56.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.993+0000 7fb38cff9640 1 -- 172.21.15.84:0/723259565 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380009990 con 0x7fb3900b0410 2023-10-16T09:49:56.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.993+0000 7fb396800640 1 -- 172.21.15.84:0/723259565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb390156a70 con 0x7fb3900b0410 2023-10-16T09:49:56.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.994+0000 7fb396800640 1 -- 172.21.15.84:0/723259565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb39004e160 con 0x7fb3900b0410 2023-10-16T09:49:56.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.995+0000 7fb38cff9640 1 -- 172.21.15.84:0/723259565 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb380009af0 con 0x7fb3900b0410 2023-10-16T09:49:56.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.997+0000 7fb38cff9640 1 --2- 172.21.15.84:0/723259565 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb37c06ae70 0x7fb37c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:56.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.997+0000 7fb38cff9640 1 -- 172.21.15.84:0/723259565 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb380089b60 con 0x7fb3900b0410 2023-10-16T09:49:56.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.997+0000 7fb38f7fe640 1 --2- 172.21.15.84:0/723259565 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb37c06ae70 0x7fb37c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:56.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:55.998+0000 7fb38f7fe640 1 --2- 172.21.15.84:0/723259565 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb37c06ae70 0x7fb37c06d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb384036000 tx=0x7fb384030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:56.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.001+0000 7fb38cff9640 1 -- 172.21.15.84:0/723259565 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb380054c40 con 0x7fb3900b0410 2023-10-16T09:49:56.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.232+0000 7fb396800640 1 -- 172.21.15.84:0/723259565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3900b48f0 con 0x7fb3900b0410 2023-10-16T09:49:56.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.233+0000 7fb38cff9640 1 -- 172.21.15.84:0/723259565 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7fb380058260 con 0x7fb3900b0410 2023-10-16T09:49:56.235 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:56.235 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2978888131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2978888131},{"type":"v1","addr":"172.21.15.157:6834","nonce":2978888131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3716581310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3716581310},{"type":"v1","addr":"172.21.15.84:6835","nonce":3716581310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2761479872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2761479872},{"type":"v1","addr":"172.21.15.84:6837","nonce":2761479872}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:49:02.350624+0000","modified":"2023-10-16T09:49:05.402785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/848754825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":848754825},{"type":"v1","addr":"172.21.15.157:6835","nonce":848754825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T09:49:56.236 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-10-16T09:49:56.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.235+0000 7fb396800640 1 -- 172.21.15.84:0/723259565 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb37c06ae70 msgr2=0x7fb37c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:56.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.235+0000 7fb396800640 1 --2- 172.21.15.84:0/723259565 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb37c06ae70 0x7fb37c06d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb384036000 tx=0x7fb384030000 comp rx=0 tx=0).stop 2023-10-16T09:49:56.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.235+0000 7fb396800640 1 -- 172.21.15.84:0/723259565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3900b0410 msgr2=0x7fb390155660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:56.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.235+0000 7fb396800640 1 --2- 172.21.15.84:0/723259565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3900b0410 0x7fb390155660 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb38000d800 tx=0x7fb38000dcc0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.236+0000 7fb396800640 1 -- 172.21.15.84:0/723259565 shutdown_connections 2023-10-16T09:49:56.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.236+0000 7fb396800640 1 --2- 172.21.15.84:0/723259565 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb37c06ae70 0x7fb37c06d330 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.236+0000 7fb396800640 1 --2- 172.21.15.84:0/723259565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb39015d3f0 0x7fb3900c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.236+0000 7fb396800640 1 --2- 172.21.15.84:0/723259565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3900cd0c0 0x7fb390155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.236+0000 7fb396800640 1 --2- 172.21.15.84:0/723259565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3900b0410 0x7fb390155660 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.236+0000 7fb396800640 1 -- 172.21.15.84:0/723259565 >> 172.21.15.84:0/723259565 conn(0x7fb3900cb410 msgr2=0x7fb3900cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:56.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.236+0000 7fb396800640 1 -- 172.21.15.84:0/723259565 shutdown_connections 2023-10-16T09:49:56.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.236+0000 7fb396800640 1 -- 172.21.15.84:0/723259565 wait complete. 2023-10-16T09:49:56.257 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:56.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.465+0000 7fb46a27c640 1 Processor -- start 2023-10-16T09:49:56.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.465+0000 7fb46a27c640 1 -- start start 2023-10-16T09:49:56.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.466+0000 7fb46a27c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46415aaa0 0x7fb46415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:56.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.466+0000 7fb46a27c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb4640ce390 0x7fb464070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:56.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.467+0000 7fb46a27c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46415ed60 con 0x7fb46415ae80 2023-10-16T09:49:56.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.467+0000 7fb46a27c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46406b810 con 0x7fb4640ce390 2023-10-16T09:49:56.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.467+0000 7fb46a27c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4640707e0 con 0x7fb46415aaa0 2023-10-16T09:49:56.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.467+0000 7fb463fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46415aaa0 0x7fb46415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:56.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.467+0000 7fb468819640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb4640ce390 0x7fb464070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:56.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.467+0000 7fb463fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46415aaa0 0x7fb46415e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:49936/0 (socket says 172.21.15.84:49936) 2023-10-16T09:49:56.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.467+0000 7fb463fff640 1 -- 172.21.15.84:0/1618197966 learned_addr learned my addr 172.21.15.84:0/1618197966 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:56.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.468+0000 7fb468819640 1 -- 172.21.15.84:0/1618197966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46415aaa0 msgr2=0x7fb46415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:56.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.468+0000 7fb468819640 1 --2- 172.21.15.84:0/1618197966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46415aaa0 0x7fb46415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.468+0000 7fb468819640 1 -- 172.21.15.84:0/1618197966 >> v1:172.21.15.157:6789/0 conn(0x7fb46415ae80 legacy=0x7fb4640cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:56.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.468+0000 7fb468819640 1 -- 172.21.15.84:0/1618197966 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46415ef90 con 0x7fb4640ce390 2023-10-16T09:49:56.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.468+0000 7fb463fff640 1 --2- 172.21.15.84:0/1618197966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46415aaa0 0x7fb46415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:49:56.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.469+0000 7fb468819640 1 --2- 172.21.15.84:0/1618197966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4640ce390 0x7fb464070f50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009d90 tx=0x7fb45c02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ed1dd1aaccfd09b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:56.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.470+0000 7fb462ffd640 1 -- 172.21.15.84:0/1618197966 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c02cc60 con 0x7fb4640ce390 2023-10-16T09:49:56.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.470+0000 7fb462ffd640 1 -- 172.21.15.84:0/1618197966 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c02cdc0 con 0x7fb4640ce390 2023-10-16T09:49:56.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.470+0000 7fb462ffd640 1 -- 172.21.15.84:0/1618197966 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c034660 con 0x7fb4640ce390 2023-10-16T09:49:56.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.470+0000 7fb46a27c640 1 -- 172.21.15.84:0/1618197966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4640ce390 msgr2=0x7fb464070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:56.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.470+0000 7fb46a27c640 1 --2- 172.21.15.84:0/1618197966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4640ce390 0x7fb464070f50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009d90 tx=0x7fb45c02beb0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.470+0000 7fb46a27c640 1 -- 172.21.15.84:0/1618197966 shutdown_connections 2023-10-16T09:49:56.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.470+0000 7fb46a27c640 1 --2- 172.21.15.84:0/1618197966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4640ce390 0x7fb464070f50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.470+0000 7fb46a27c640 1 --2- 172.21.15.84:0/1618197966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46415aaa0 0x7fb46415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.470+0000 7fb46a27c640 1 -- 172.21.15.84:0/1618197966 >> 172.21.15.84:0/1618197966 conn(0x7fb4640cb410 msgr2=0x7fb4640cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:56.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.471+0000 7fb46a27c640 1 -- 172.21.15.84:0/1618197966 shutdown_connections 2023-10-16T09:49:56.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.471+0000 7fb46a27c640 1 -- 172.21.15.84:0/1618197966 wait complete. 2023-10-16T09:49:56.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.472+0000 7fb46a27c640 1 Processor -- start 2023-10-16T09:49:56.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.472+0000 7fb46a27c640 1 -- start start 2023-10-16T09:49:56.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.473+0000 7fb46a27c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4640ce390 0x7fb4641569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:56.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.473+0000 7fb46a27c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46415aaa0 0x7fb4640c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:56.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.473+0000 7fb46a27c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb46415ae80 0x7fb4640c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:56.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.473+0000 7fb46a27c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb464147d30 con 0x7fb4640ce390 2023-10-16T09:49:56.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.473+0000 7fb46a27c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb464159ad0 con 0x7fb46415ae80 2023-10-16T09:49:56.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.473+0000 7fb46a27c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb464159510 con 0x7fb46415aaa0 2023-10-16T09:49:56.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.474+0000 7fb463fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4640ce390 0x7fb4641569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:56.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.474+0000 7fb463fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4640ce390 0x7fb4641569a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38210/0 (socket says 172.21.15.84:38210) 2023-10-16T09:49:56.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.474+0000 7fb463fff640 1 -- 172.21.15.84:0/2300753269 learned_addr learned my addr 172.21.15.84:0/2300753269 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:56.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.474+0000 7fb4637fe640 1 --2- 172.21.15.84:0/2300753269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46415aaa0 0x7fb4640c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:56.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.474+0000 7fb468819640 1 --2- 172.21.15.84:0/2300753269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb46415ae80 0x7fb4640c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:56.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.474+0000 7fb463fff640 1 -- 172.21.15.84:0/2300753269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46415aaa0 msgr2=0x7fb4640c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:56.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.474+0000 7fb463fff640 1 --2- 172.21.15.84:0/2300753269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46415aaa0 0x7fb4640c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.474+0000 7fb463fff640 1 -- 172.21.15.84:0/2300753269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb46415ae80 msgr2=0x7fb4640c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:56.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.474+0000 7fb463fff640 1 --2- 172.21.15.84:0/2300753269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb46415ae80 0x7fb4640c4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.474+0000 7fb463fff640 1 -- 172.21.15.84:0/2300753269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4540094d0 con 0x7fb4640ce390 2023-10-16T09:49:56.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.474+0000 7fb463fff640 1 --2- 172.21.15.84:0/2300753269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4640ce390 0x7fb4641569a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb45400ea00 tx=0x7fb45400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:56.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.475+0000 7fb460ff9640 1 -- 172.21.15.84:0/2300753269 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45400ca60 con 0x7fb4640ce390 2023-10-16T09:49:56.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.475+0000 7fb460ff9640 1 -- 172.21.15.84:0/2300753269 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45400cbc0 con 0x7fb4640ce390 2023-10-16T09:49:56.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.475+0000 7fb460ff9640 1 -- 172.21.15.84:0/2300753269 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454013470 con 0x7fb4640ce390 2023-10-16T09:49:56.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.475+0000 7fb46a27c640 1 -- 172.21.15.84:0/2300753269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45c009a70 con 0x7fb4640ce390 2023-10-16T09:49:56.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.475+0000 7fb46a27c640 1 -- 172.21.15.84:0/2300753269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4641479c0 con 0x7fb4640ce390 2023-10-16T09:49:56.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.476+0000 7fb46a27c640 1 -- 172.21.15.84:0/2300753269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46404e160 con 0x7fb4640ce390 2023-10-16T09:49:56.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.477+0000 7fb460ff9640 1 -- 172.21.15.84:0/2300753269 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb4540135d0 con 0x7fb4640ce390 2023-10-16T09:49:56.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.478+0000 7fb460ff9640 1 --2- 172.21.15.84:0/2300753269 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb45006ae70 0x7fb45006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:56.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.479+0000 7fb460ff9640 1 -- 172.21.15.84:0/2300753269 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb4540899e0 con 0x7fb4640ce390 2023-10-16T09:49:56.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.479+0000 7fb4637fe640 1 --2- 172.21.15.84:0/2300753269 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb45006ae70 0x7fb45006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:56.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.479+0000 7fb4637fe640 1 --2- 172.21.15.84:0/2300753269 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb45006ae70 0x7fb45006d330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb46406d740 tx=0x7fb458009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:56.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.483+0000 7fb460ff9640 1 -- 172.21.15.84:0/2300753269 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb454054a00 con 0x7fb4640ce390 2023-10-16T09:49:56.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.673+0000 7fb46a27c640 1 -- 172.21.15.84:0/2300753269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4640cdc60 con 0x7fb4640ce390 2023-10-16T09:49:56.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.676+0000 7fb460ff9640 1 -- 172.21.15.84:0/2300753269 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16730 (secure 0 0 0) 0x7fb454017090 con 0x7fb4640ce390 2023-10-16T09:49:56.679 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:56.680 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":23,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:49.552121+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.27","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:56.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.679+0000 7fb46a27c640 1 -- 172.21.15.84:0/2300753269 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb45006ae70 msgr2=0x7fb45006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:56.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.679+0000 7fb46a27c640 1 --2- 172.21.15.84:0/2300753269 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb45006ae70 0x7fb45006d330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb46406d740 tx=0x7fb458009040 comp rx=0 tx=0).stop 2023-10-16T09:49:56.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.679+0000 7fb46a27c640 1 -- 172.21.15.84:0/2300753269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4640ce390 msgr2=0x7fb4641569a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:56.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.679+0000 7fb46a27c640 1 --2- 172.21.15.84:0/2300753269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4640ce390 0x7fb4641569a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb45400ea00 tx=0x7fb45400eec0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.679+0000 7fb46a27c640 1 -- 172.21.15.84:0/2300753269 shutdown_connections 2023-10-16T09:49:56.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.679+0000 7fb46a27c640 1 --2- 172.21.15.84:0/2300753269 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb45006ae70 0x7fb45006d330 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.679+0000 7fb46a27c640 1 --2- 172.21.15.84:0/2300753269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb46415ae80 0x7fb4640c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.679+0000 7fb46a27c640 1 --2- 172.21.15.84:0/2300753269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46415aaa0 0x7fb4640c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.679+0000 7fb46a27c640 1 --2- 172.21.15.84:0/2300753269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4640ce390 0x7fb4641569a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.679+0000 7fb46a27c640 1 -- 172.21.15.84:0/2300753269 >> 172.21.15.84:0/2300753269 conn(0x7fb4640cb410 msgr2=0x7fb4640cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:56.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.679+0000 7fb46a27c640 1 -- 172.21.15.84:0/2300753269 shutdown_connections 2023-10-16T09:49:56.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.679+0000 7fb46a27c640 1 -- 172.21.15.84:0/2300753269 wait complete. 2023-10-16T09:49:56.698 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:49:00.048649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:49:01.361365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:49:56.699 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:49:56.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.905+0000 7f28ab505640 1 Processor -- start 2023-10-16T09:49:56.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.905+0000 7f28ab505640 1 -- start start 2023-10-16T09:49:56.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.907+0000 7f28ab505640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f28a415ad70 0x7f28a4069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:56.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.907+0000 7f28ab505640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f28a40c9ca0 0x7f28a406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:56.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.907+0000 7f28ab505640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28a40ca3b0 0x7f28a40ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:56.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.907+0000 7f28ab505640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a40cad80 con 0x7f28a40c9ca0 2023-10-16T09:49:56.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.907+0000 7f28ab505640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a40cb030 con 0x7f28a415ad70 2023-10-16T09:49:56.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.907+0000 7f28ab505640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a40c91c0 con 0x7f28a40ca3b0 2023-10-16T09:49:56.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.907+0000 7f28a8aa0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f28a40c9ca0 0x7f28a406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:56.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.907+0000 7f28a8aa0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f28a40c9ca0 0x7f28a406a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38224/0 (socket says 172.21.15.84:38224) 2023-10-16T09:49:56.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.907+0000 7f28a8aa0640 1 -- 172.21.15.84:0/2150906764 learned_addr learned my addr 172.21.15.84:0/2150906764 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:56.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.907+0000 7f28a92a1640 1 --2- 172.21.15.84:0/2150906764 >> v2:172.21.15.157:3300/0 conn(0x7f28a415ad70 0x7f28a4069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:56.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.907+0000 7f28a9aa2640 1 --2- 172.21.15.84:0/2150906764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28a40ca3b0 0x7f28a40ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:56.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.908+0000 7f28a8aa0640 1 -- 172.21.15.84:0/2150906764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28a40ca3b0 msgr2=0x7f28a40ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:56.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.908+0000 7f28a8aa0640 1 --2- 172.21.15.84:0/2150906764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28a40ca3b0 0x7f28a40ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.908+0000 7f28a8aa0640 1 -- 172.21.15.84:0/2150906764 >> v2:172.21.15.157:3300/0 conn(0x7f28a415ad70 msgr2=0x7f28a4069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:56.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.908+0000 7f28a8aa0640 1 --2- 172.21.15.84:0/2150906764 >> v2:172.21.15.157:3300/0 conn(0x7f28a415ad70 0x7f28a4069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.908+0000 7f28a8aa0640 1 -- 172.21.15.84:0/2150906764 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a40c94a0 con 0x7f28a40c9ca0 2023-10-16T09:49:56.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.908+0000 7f28a9aa2640 1 --2- 172.21.15.84:0/2150906764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28a40ca3b0 0x7f28a40ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:56.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.908+0000 7f28a92a1640 1 --2- 172.21.15.84:0/2150906764 >> v2:172.21.15.157:3300/0 conn(0x7f28a415ad70 0x7f28a4069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:56.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.909+0000 7f28a8aa0640 1 --2- 172.21.15.84:0/2150906764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a40c9ca0 0x7f28a406a2e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f28a40cf780 tx=0x7f289002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb8bf0aed2d3ba3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:56.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.909+0000 7f28a3fff640 1 -- 172.21.15.84:0/2150906764 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890031870 con 0x7f28a40c9ca0 2023-10-16T09:49:56.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.909+0000 7f28a3fff640 1 -- 172.21.15.84:0/2150906764 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28900319d0 con 0x7f28a40c9ca0 2023-10-16T09:49:56.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.909+0000 7f28a3fff640 1 -- 172.21.15.84:0/2150906764 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890031d60 con 0x7f28a40c9ca0 2023-10-16T09:49:56.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.910+0000 7f28ab505640 1 -- 172.21.15.84:0/2150906764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a40c9ca0 msgr2=0x7f28a406a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:56.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.910+0000 7f28ab505640 1 --2- 172.21.15.84:0/2150906764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a40c9ca0 0x7f28a406a2e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f28a40cf780 tx=0x7f289002b910 comp rx=0 tx=0).stop 2023-10-16T09:49:56.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.910+0000 7f28ab505640 1 -- 172.21.15.84:0/2150906764 shutdown_connections 2023-10-16T09:49:56.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.910+0000 7f28ab505640 1 --2- 172.21.15.84:0/2150906764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a40c9ca0 0x7f28a406a2e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.910+0000 7f28ab505640 1 --2- 172.21.15.84:0/2150906764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28a40ca3b0 0x7f28a40ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.910+0000 7f28ab505640 1 --2- 172.21.15.84:0/2150906764 >> v2:172.21.15.157:3300/0 conn(0x7f28a415ad70 0x7f28a4069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.910+0000 7f28ab505640 1 -- 172.21.15.84:0/2150906764 >> 172.21.15.84:0/2150906764 conn(0x7f28a40ce9b0 msgr2=0x7f28a40cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:56.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.910+0000 7f28ab505640 1 -- 172.21.15.84:0/2150906764 shutdown_connections 2023-10-16T09:49:56.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.910+0000 7f28ab505640 1 -- 172.21.15.84:0/2150906764 wait complete. 2023-10-16T09:49:56.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.912+0000 7f28ab505640 1 Processor -- start 2023-10-16T09:49:56.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.912+0000 7f28ab505640 1 -- start start 2023-10-16T09:49:56.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.913+0000 7f28ab505640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28a40c9ca0 0x7f28a40c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:56.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.913+0000 7f28ab505640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a40ca3b0 0x7f28a40c3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:56.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.913+0000 7f28ab505640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28a415ad70 0x7f28a4159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:56.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.913+0000 7f28ab505640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a40c8360 con 0x7f28a40ca3b0 2023-10-16T09:49:56.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.913+0000 7f28ab505640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a40c34b0 con 0x7f28a40c9ca0 2023-10-16T09:49:56.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.913+0000 7f28ab505640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a40c3e80 con 0x7f28a415ad70 2023-10-16T09:49:56.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.913+0000 7f28a8aa0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a40ca3b0 0x7f28a40c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:56.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.913+0000 7f28a8aa0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a40ca3b0 0x7f28a40c3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38232/0 (socket says 172.21.15.84:38232) 2023-10-16T09:49:56.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.913+0000 7f28a8aa0640 1 -- 172.21.15.84:0/2484381602 learned_addr learned my addr 172.21.15.84:0/2484381602 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:56.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.913+0000 7f28a92a1640 1 --2- 172.21.15.84:0/2484381602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28a40c9ca0 0x7f28a40c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:56.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.913+0000 7f28a8aa0640 1 -- 172.21.15.84:0/2484381602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28a415ad70 msgr2=0x7f28a4159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:56.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.913+0000 7f28a8aa0640 1 --2- 172.21.15.84:0/2484381602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28a415ad70 0x7f28a4159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.913+0000 7f28a8aa0640 1 -- 172.21.15.84:0/2484381602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28a40c9ca0 msgr2=0x7f28a40c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:56.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.913+0000 7f28a8aa0640 1 --2- 172.21.15.84:0/2484381602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28a40c9ca0 0x7f28a40c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:56.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.913+0000 7f28a8aa0640 1 -- 172.21.15.84:0/2484381602 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28900094d0 con 0x7f28a40ca3b0 2023-10-16T09:49:56.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.914+0000 7f28a8aa0640 1 --2- 172.21.15.84:0/2484381602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a40ca3b0 0x7f28a40c3940 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f289002b950 tx=0x7f28900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:56.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.914+0000 7f28a1ffb640 1 -- 172.21.15.84:0/2484381602 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890031860 con 0x7f28a40ca3b0 2023-10-16T09:49:56.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.914+0000 7f28a1ffb640 1 -- 172.21.15.84:0/2484381602 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28900443f0 con 0x7f28a40ca3b0 2023-10-16T09:49:56.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.914+0000 7f28ab505640 1 -- 172.21.15.84:0/2484381602 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a40c28a0 con 0x7f28a40ca3b0 2023-10-16T09:49:56.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.914+0000 7f28a1ffb640 1 -- 172.21.15.84:0/2484381602 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28900447a0 con 0x7f28a40ca3b0 2023-10-16T09:49:56.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.914+0000 7f28ab505640 1 -- 172.21.15.84:0/2484381602 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a40c2e60 con 0x7f28a40ca3b0 2023-10-16T09:49:56.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.915+0000 7f28ab505640 1 -- 172.21.15.84:0/2484381602 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a40c3640 con 0x7f28a40ca3b0 2023-10-16T09:49:56.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.916+0000 7f28a1ffb640 1 -- 172.21.15.84:0/2484381602 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f289004d450 con 0x7f28a40ca3b0 2023-10-16T09:49:56.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.917+0000 7f28a1ffb640 1 --2- 172.21.15.84:0/2484381602 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f288806aea0 0x7f288806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:56.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.917+0000 7f28a1ffb640 1 -- 172.21.15.84:0/2484381602 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f289002d040 con 0x7f28a40ca3b0 2023-10-16T09:49:56.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.917+0000 7f28a92a1640 1 --2- 172.21.15.84:0/2484381602 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f288806aea0 0x7f288806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:56.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.918+0000 7f28a92a1640 1 --2- 172.21.15.84:0/2484381602 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f288806aea0 0x7f288806d360 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f28a415da90 tx=0x7f2898008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:56.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:56.922+0000 7f28a1ffb640 1 -- 172.21.15.84:0/2484381602 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2890076e80 con 0x7f28a40ca3b0 2023-10-16T09:49:57.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.098+0000 7f28ab505640 1 -- 172.21.15.84:0/2484381602 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f28a404e160 con 0x7f28a40ca3b0 2023-10-16T09:49:57.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.099+0000 7f28a1ffb640 1 -- 172.21.15.84:0/2484381602 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2890035070 con 0x7f28a40ca3b0 2023-10-16T09:49:57.101 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T09:49:57.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.102+0000 7f28ab505640 1 -- 172.21.15.84:0/2484381602 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f288806aea0 msgr2=0x7f288806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:57.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.102+0000 7f28ab505640 1 --2- 172.21.15.84:0/2484381602 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f288806aea0 0x7f288806d360 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f28a415da90 tx=0x7f2898008040 comp rx=0 tx=0).stop 2023-10-16T09:49:57.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.102+0000 7f28ab505640 1 -- 172.21.15.84:0/2484381602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a40ca3b0 msgr2=0x7f28a40c3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:57.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.102+0000 7f28ab505640 1 --2- 172.21.15.84:0/2484381602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a40ca3b0 0x7f28a40c3940 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f289002b950 tx=0x7f28900046c0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.102+0000 7f28ab505640 1 -- 172.21.15.84:0/2484381602 shutdown_connections 2023-10-16T09:49:57.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.102+0000 7f28ab505640 1 --2- 172.21.15.84:0/2484381602 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f288806aea0 0x7f288806d360 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.102+0000 7f28ab505640 1 --2- 172.21.15.84:0/2484381602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28a415ad70 0x7f28a4159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.102+0000 7f28ab505640 1 --2- 172.21.15.84:0/2484381602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a40ca3b0 0x7f28a40c3940 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.102+0000 7f28ab505640 1 --2- 172.21.15.84:0/2484381602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28a40c9ca0 0x7f28a40c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.102+0000 7f28ab505640 1 -- 172.21.15.84:0/2484381602 >> 172.21.15.84:0/2484381602 conn(0x7f28a40ce9b0 msgr2=0x7f28a40ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:57.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.102+0000 7f28ab505640 1 -- 172.21.15.84:0/2484381602 shutdown_connections 2023-10-16T09:49:57.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.102+0000 7f28ab505640 1 -- 172.21.15.84:0/2484381602 wait complete. 2023-10-16T09:49:57.121 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:49:57.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.335+0000 7fa13e51d640 1 Processor -- start 2023-10-16T09:49:57.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.335+0000 7fa13e51d640 1 -- start start 2023-10-16T09:49:57.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.336+0000 7fa13e51d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa13815ad70 0x7fa138069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:57.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.336+0000 7fa13e51d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1380caac0 0x7fa13806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:57.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.336+0000 7fa13e51d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1380cb030 con 0x7fa1380c9ca0 2023-10-16T09:49:57.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.336+0000 7fa13e51d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1380c91c0 con 0x7fa13815ad70 2023-10-16T09:49:57.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.336+0000 7fa13e51d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1380c94a0 con 0x7fa1380caac0 2023-10-16T09:49:57.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.336+0000 7fa137fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa13815ad70 0x7fa138069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:57.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.337+0000 7fa12ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa1380c9ca0 0x7fa1380ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36200/0 (socket says 172.21.15.84:36200) 2023-10-16T09:49:57.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.337+0000 7fa12ffff640 1 -- 172.21.15.84:0/2974454722 learned_addr learned my addr 172.21.15.84:0/2974454722 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:57.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.337+0000 7fa137fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa13815ad70 0x7fa138069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38244/0 (socket says 172.21.15.84:38244) 2023-10-16T09:49:57.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.337+0000 7fa13caba640 1 --2- 172.21.15.84:0/2974454722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1380caac0 0x7fa13806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.337+0000 7fa137fff640 1 -- 172.21.15.84:0/2974454722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1380caac0 msgr2=0x7fa13806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.337+0000 7fa137fff640 1 --2- 172.21.15.84:0/2974454722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1380caac0 0x7fa13806a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.337+0000 7fa137fff640 1 -- 172.21.15.84:0/2974454722 >> v1:172.21.15.84:6789/0 conn(0x7fa1380c9ca0 legacy=0x7fa1380ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.337+0000 7fa137fff640 1 -- 172.21.15.84:0/2974454722 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1380c9780 con 0x7fa13815ad70 2023-10-16T09:49:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.337+0000 7fa13caba640 1 --2- 172.21.15.84:0/2974454722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1380caac0 0x7fa13806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:57.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.337+0000 7fa1377fe640 1 -- 172.21.15.84:0/2974454722 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197668365 0 0) 0x7fa1380cb030 con 0x7fa1380c9ca0 2023-10-16T09:49:57.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.338+0000 7fa137fff640 1 --2- 172.21.15.84:0/2974454722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa13815ad70 0x7fa138069da0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa1280097a0 tx=0x7fa12802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d5aaf0503e8f39 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:57.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.338+0000 7fa1377fe640 1 -- 172.21.15.84:0/2974454722 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12802d040 con 0x7fa13815ad70 2023-10-16T09:49:57.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.339+0000 7fa1377fe640 1 -- 172.21.15.84:0/2974454722 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa128031810 con 0x7fa13815ad70 2023-10-16T09:49:57.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.339+0000 7fa1377fe640 1 -- 172.21.15.84:0/2974454722 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa128031ba0 con 0x7fa13815ad70 2023-10-16T09:49:57.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.339+0000 7fa13e51d640 1 -- 172.21.15.84:0/2974454722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa13815ad70 msgr2=0x7fa138069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:57.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.339+0000 7fa13e51d640 1 --2- 172.21.15.84:0/2974454722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa13815ad70 0x7fa138069da0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa1280097a0 tx=0x7fa12802b8b0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.339+0000 7fa13e51d640 1 -- 172.21.15.84:0/2974454722 shutdown_connections 2023-10-16T09:49:57.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.339+0000 7fa13e51d640 1 --2- 172.21.15.84:0/2974454722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1380caac0 0x7fa13806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.339+0000 7fa13e51d640 1 --2- 172.21.15.84:0/2974454722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa13815ad70 0x7fa138069da0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.339+0000 7fa13e51d640 1 -- 172.21.15.84:0/2974454722 >> 172.21.15.84:0/2974454722 conn(0x7fa1380ce9b0 msgr2=0x7fa1380cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:57.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.339+0000 7fa13e51d640 1 -- 172.21.15.84:0/2974454722 shutdown_connections 2023-10-16T09:49:57.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.339+0000 7fa13e51d640 1 -- 172.21.15.84:0/2974454722 wait complete. 2023-10-16T09:49:57.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.341+0000 7fa13e51d640 1 Processor -- start 2023-10-16T09:49:57.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.341+0000 7fa13e51d640 1 -- start start 2023-10-16T09:49:57.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.342+0000 7fa13e51d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1380c9ca0 0x7fa138148090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:57.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.342+0000 7fa13e51d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1380caac0 0x7fa1380c4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:57.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.342+0000 7fa13e51d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa13815ad70 0x7fa1380c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:57.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.342+0000 7fa13e51d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13814c0c0 con 0x7fa13815ad70 2023-10-16T09:49:57.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.342+0000 7fa13e51d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1381485d0 con 0x7fa1380c9ca0 2023-10-16T09:49:57.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.342+0000 7fa13e51d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13814bd50 con 0x7fa1380caac0 2023-10-16T09:49:57.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.343+0000 7fa13caba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa13815ad70 0x7fa1380c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:57.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.343+0000 7fa13caba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa13815ad70 0x7fa1380c7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38250/0 (socket says 172.21.15.84:38250) 2023-10-16T09:49:57.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.343+0000 7fa13caba640 1 -- 172.21.15.84:0/3326294767 learned_addr learned my addr 172.21.15.84:0/3326294767 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:57.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.343+0000 7fa137fff640 1 --2- 172.21.15.84:0/3326294767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1380c9ca0 0x7fa138148090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:57.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.343+0000 7fa13caba640 1 -- 172.21.15.84:0/3326294767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1380caac0 msgr2=0x7fa1380c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:57.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.343+0000 7fa13caba640 1 --2- 172.21.15.84:0/3326294767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1380caac0 0x7fa1380c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.343+0000 7fa13caba640 1 -- 172.21.15.84:0/3326294767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1380c9ca0 msgr2=0x7fa138148090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:57.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.343+0000 7fa13caba640 1 --2- 172.21.15.84:0/3326294767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1380c9ca0 0x7fa138148090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.343+0000 7fa13caba640 1 -- 172.21.15.84:0/3326294767 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa128009400 con 0x7fa13815ad70 2023-10-16T09:49:57.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.343+0000 7fa13caba640 1 --2- 172.21.15.84:0/3326294767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa13815ad70 0x7fa1380c7c50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa13000cf20 tx=0x7fa13000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:57.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.343+0000 7fa1357fa640 1 -- 172.21.15.84:0/3326294767 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13000d840 con 0x7fa13815ad70 2023-10-16T09:49:57.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.343+0000 7fa1357fa640 1 -- 172.21.15.84:0/3326294767 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13000d9a0 con 0x7fa13815ad70 2023-10-16T09:49:57.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.343+0000 7fa1357fa640 1 -- 172.21.15.84:0/3326294767 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13000dd50 con 0x7fa13815ad70 2023-10-16T09:49:57.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.344+0000 7fa13e51d640 1 -- 172.21.15.84:0/3326294767 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1380c4d60 con 0x7fa13815ad70 2023-10-16T09:49:57.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.344+0000 7fa13e51d640 1 -- 172.21.15.84:0/3326294767 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1380c2760 con 0x7fa13815ad70 2023-10-16T09:49:57.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.345+0000 7fa13e51d640 1 -- 172.21.15.84:0/3326294767 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13814c200 con 0x7fa13815ad70 2023-10-16T09:49:57.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.346+0000 7fa1357fa640 1 -- 172.21.15.84:0/3326294767 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa13001e3b0 con 0x7fa13815ad70 2023-10-16T09:49:57.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.346+0000 7fa1357fa640 1 --2- 172.21.15.84:0/3326294767 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fa11806ae70 0x7fa11806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:57.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.347+0000 7fa1357fa640 1 -- 172.21.15.84:0/3326294767 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa13008a120 con 0x7fa13815ad70 2023-10-16T09:49:57.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.347+0000 7fa137fff640 1 --2- 172.21.15.84:0/3326294767 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fa11806ae70 0x7fa11806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:57.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.348+0000 7fa137fff640 1 --2- 172.21.15.84:0/3326294767 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fa11806ae70 0x7fa11806d330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa128036000 tx=0x7fa12803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:57.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.351+0000 7fa1357fa640 1 -- 172.21.15.84:0/3326294767 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa13814c200 con 0x7fa13815ad70 2023-10-16T09:49:57.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.547+0000 7fa13e51d640 1 -- 172.21.15.84:0/3326294767 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa13814c200 con 0x7fa13815ad70 2023-10-16T09:49:57.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.549+0000 7fa1357fa640 1 -- 172.21.15.84:0/3326294767 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa13814c200 con 0x7fa13815ad70 2023-10-16T09:49:57.550 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T09:49:57.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.551+0000 7fa13e51d640 1 -- 172.21.15.84:0/3326294767 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fa11806ae70 msgr2=0x7fa11806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:57.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.551+0000 7fa13e51d640 1 --2- 172.21.15.84:0/3326294767 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fa11806ae70 0x7fa11806d330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa128036000 tx=0x7fa12803c040 comp rx=0 tx=0).stop 2023-10-16T09:49:57.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.551+0000 7fa13e51d640 1 -- 172.21.15.84:0/3326294767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa13815ad70 msgr2=0x7fa1380c7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:57.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.551+0000 7fa13e51d640 1 --2- 172.21.15.84:0/3326294767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa13815ad70 0x7fa1380c7c50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa13000cf20 tx=0x7fa13000a7d0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.552+0000 7fa13e51d640 1 -- 172.21.15.84:0/3326294767 shutdown_connections 2023-10-16T09:49:57.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.552+0000 7fa13e51d640 1 --2- 172.21.15.84:0/3326294767 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fa11806ae70 0x7fa11806d330 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.552+0000 7fa13e51d640 1 --2- 172.21.15.84:0/3326294767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa13815ad70 0x7fa1380c7c50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.552+0000 7fa13e51d640 1 --2- 172.21.15.84:0/3326294767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1380caac0 0x7fa1380c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.552+0000 7fa13e51d640 1 --2- 172.21.15.84:0/3326294767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1380c9ca0 0x7fa138148090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.552+0000 7fa13e51d640 1 -- 172.21.15.84:0/3326294767 >> 172.21.15.84:0/3326294767 conn(0x7fa1380ce9b0 msgr2=0x7fa138070d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:57.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.552+0000 7fa13e51d640 1 -- 172.21.15.84:0/3326294767 shutdown_connections 2023-10-16T09:49:57.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.552+0000 7fa13e51d640 1 -- 172.21.15.84:0/3326294767 wait complete. 2023-10-16T09:49:57.572 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:49:57.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.786+0000 7fcaa43a9640 1 Processor -- start 2023-10-16T09:49:57.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.787+0000 7fcaa43a9640 1 -- start start 2023-10-16T09:49:57.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.788+0000 7fcaa43a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca9c15d150 0x7fca9c0ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:57.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.788+0000 7fcaa43a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fca9c06a4b0 0x7fca9c160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:57.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.788+0000 7fcaa43a9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca9c0ac890 con 0x7fca9c160b90 2023-10-16T09:49:57.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.788+0000 7fcaa43a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca9c15d530 con 0x7fca9c06a4b0 2023-10-16T09:49:57.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.788+0000 7fcaa43a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c161650 con 0x7fca9c15d150 2023-10-16T09:49:57.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.788+0000 7fcaa2946640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fca9c06a4b0 0x7fca9c160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:57.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.788+0000 7fcaa1944640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fca9c160b90 0x7fca9c069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36206/0 (socket says 172.21.15.84:36206) 2023-10-16T09:49:57.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.788+0000 7fcaa1944640 1 -- 172.21.15.84:0/3011183383 learned_addr learned my addr 172.21.15.84:0/3011183383 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:57.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.788+0000 7fcaa2946640 1 --2- 172.21.15.84:0/3011183383 >> v2:172.21.15.84:3300/0 conn(0x7fca9c06a4b0 0x7fca9c160fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38254/0 (socket says 172.21.15.84:38254) 2023-10-16T09:49:57.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.789+0000 7fcaa2145640 1 --2- 172.21.15.84:0/3011183383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca9c15d150 0x7fca9c0ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:57.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.789+0000 7fcaa2946640 1 -- 172.21.15.84:0/3011183383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca9c15d150 msgr2=0x7fca9c0ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:57.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.789+0000 7fcaa2946640 1 --2- 172.21.15.84:0/3011183383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca9c15d150 0x7fca9c0ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.789+0000 7fcaa2946640 1 -- 172.21.15.84:0/3011183383 >> v1:172.21.15.84:6789/0 conn(0x7fca9c160b90 legacy=0x7fca9c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:57.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.789+0000 7fcaa2946640 1 -- 172.21.15.84:0/3011183383 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c161930 con 0x7fca9c06a4b0 2023-10-16T09:49:57.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.790+0000 7fcaa2946640 1 --2- 172.21.15.84:0/3011183383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca9c06a4b0 0x7fca9c160fb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fca94009d90 tx=0x7fca9402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbe854a2b490fd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:57.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.790+0000 7fcaa1143640 1 -- 172.21.15.84:0/3011183383 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9402cc50 con 0x7fca9c06a4b0 2023-10-16T09:49:57.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.790+0000 7fcaa1143640 1 -- 172.21.15.84:0/3011183383 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9402cdb0 con 0x7fca9c06a4b0 2023-10-16T09:49:57.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.790+0000 7fcaa1143640 1 -- 172.21.15.84:0/3011183383 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca94034660 con 0x7fca9c06a4b0 2023-10-16T09:49:57.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.791+0000 7fcaa43a9640 1 -- 172.21.15.84:0/3011183383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca9c06a4b0 msgr2=0x7fca9c160fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:57.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.791+0000 7fcaa43a9640 1 --2- 172.21.15.84:0/3011183383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca9c06a4b0 0x7fca9c160fb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fca94009d90 tx=0x7fca9402bea0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.791+0000 7fcaa43a9640 1 -- 172.21.15.84:0/3011183383 shutdown_connections 2023-10-16T09:49:57.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.791+0000 7fcaa43a9640 1 --2- 172.21.15.84:0/3011183383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca9c06a4b0 0x7fca9c160fb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.791+0000 7fcaa43a9640 1 --2- 172.21.15.84:0/3011183383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca9c15d150 0x7fca9c0ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.791+0000 7fcaa43a9640 1 -- 172.21.15.84:0/3011183383 >> 172.21.15.84:0/3011183383 conn(0x7fca9c0ce9b0 msgr2=0x7fca9c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:57.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.791+0000 7fcaa43a9640 1 -- 172.21.15.84:0/3011183383 shutdown_connections 2023-10-16T09:49:57.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.791+0000 7fcaa43a9640 1 -- 172.21.15.84:0/3011183383 wait complete. 2023-10-16T09:49:57.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.793+0000 7fcaa43a9640 1 Processor -- start 2023-10-16T09:49:57.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.793+0000 7fcaa43a9640 1 -- start start 2023-10-16T09:49:57.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.794+0000 7fcaa43a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca9c06a4b0 0x7fca9c0c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:57.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.794+0000 7fcaa43a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca9c15d150 0x7fca9c146750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:57.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.794+0000 7fcaa43a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca9c160b90 0x7fca9c1470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:57.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.794+0000 7fcaa43a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0c4430 con 0x7fca9c06a4b0 2023-10-16T09:49:57.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.794+0000 7fcaa43a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0ae5b0 con 0x7fca9c160b90 2023-10-16T09:49:57.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.794+0000 7fcaa43a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0ae890 con 0x7fca9c15d150 2023-10-16T09:49:57.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.794+0000 7fcaa2145640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca9c06a4b0 0x7fca9c0c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:57.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.794+0000 7fcaa2145640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca9c06a4b0 0x7fca9c0c3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38256/0 (socket says 172.21.15.84:38256) 2023-10-16T09:49:57.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.794+0000 7fcaa2145640 1 -- 172.21.15.84:0/2647377630 learned_addr learned my addr 172.21.15.84:0/2647377630 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:57.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.795+0000 7fcaa2145640 1 -- 172.21.15.84:0/2647377630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca9c15d150 msgr2=0x7fca9c146750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:57.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.795+0000 7fcaa2145640 1 --2- 172.21.15.84:0/2647377630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca9c15d150 0x7fca9c146750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.795+0000 7fcaa2145640 1 -- 172.21.15.84:0/2647377630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca9c160b90 msgr2=0x7fca9c1470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:57.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.795+0000 7fcaa2145640 1 --2- 172.21.15.84:0/2647377630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca9c160b90 0x7fca9c1470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:57.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.795+0000 7fcaa2145640 1 -- 172.21.15.84:0/2647377630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca94009a70 con 0x7fca9c06a4b0 2023-10-16T09:49:57.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.795+0000 7fcaa2145640 1 --2- 172.21.15.84:0/2647377630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca9c06a4b0 0x7fca9c0c3ef0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fca9800d800 tx=0x7fca9800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:57.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.795+0000 7fca92ffd640 1 -- 172.21.15.84:0/2647377630 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9800b6c0 con 0x7fca9c06a4b0 2023-10-16T09:49:57.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.795+0000 7fca92ffd640 1 -- 172.21.15.84:0/2647377630 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9800b820 con 0x7fca9c06a4b0 2023-10-16T09:49:57.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.795+0000 7fca92ffd640 1 -- 172.21.15.84:0/2647377630 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9800bbb0 con 0x7fca9c06a4b0 2023-10-16T09:49:57.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.795+0000 7fcaa43a9640 1 -- 172.21.15.84:0/2647377630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9c0c3410 con 0x7fca9c06a4b0 2023-10-16T09:49:57.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.795+0000 7fcaa43a9640 1 -- 172.21.15.84:0/2647377630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9c0c39d0 con 0x7fca9c06a4b0 2023-10-16T09:49:57.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.796+0000 7fcaa43a9640 1 -- 172.21.15.84:0/2647377630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca9c0c3b60 con 0x7fca9c06a4b0 2023-10-16T09:49:57.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.797+0000 7fca92ffd640 1 -- 172.21.15.84:0/2647377630 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fca9800bd10 con 0x7fca9c06a4b0 2023-10-16T09:49:57.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.798+0000 7fca92ffd640 1 --2- 172.21.15.84:0/2647377630 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fca8006ae70 0x7fca8006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:57.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.798+0000 7fca92ffd640 1 -- 172.21.15.84:0/2647377630 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fca98089c20 con 0x7fca9c06a4b0 2023-10-16T09:49:57.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.798+0000 7fcaa1944640 1 --2- 172.21.15.84:0/2647377630 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fca8006ae70 0x7fca8006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:57.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.799+0000 7fcaa1944640 1 --2- 172.21.15.84:0/2647377630 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fca8006ae70 0x7fca8006d330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0c3340 tx=0x7fca8c007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:57.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.803+0000 7fca92ffd640 1 -- 172.21.15.84:0/2647377630 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fca98054c40 con 0x7fca9c06a4b0 2023-10-16T09:49:57.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.995+0000 7fcaa43a9640 1 -- 172.21.15.84:0/2647377630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fca9c04e160 con 0x7fca9c06a4b0 2023-10-16T09:49:57.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.996+0000 7fca92ffd640 1 -- 172.21.15.84:0/2647377630 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fca98058260 con 0x7fca9c06a4b0 2023-10-16T09:49:57.998 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T09:49:58.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.999+0000 7fcaa43a9640 1 -- 172.21.15.84:0/2647377630 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fca8006ae70 msgr2=0x7fca8006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:58.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.999+0000 7fcaa43a9640 1 --2- 172.21.15.84:0/2647377630 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fca8006ae70 0x7fca8006d330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0c3340 tx=0x7fca8c007a40 comp rx=0 tx=0).stop 2023-10-16T09:49:58.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.999+0000 7fcaa43a9640 1 -- 172.21.15.84:0/2647377630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca9c06a4b0 msgr2=0x7fca9c0c3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:58.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.999+0000 7fcaa43a9640 1 --2- 172.21.15.84:0/2647377630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca9c06a4b0 0x7fca9c0c3ef0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fca9800d800 tx=0x7fca9800dcc0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.999+0000 7fcaa43a9640 1 -- 172.21.15.84:0/2647377630 shutdown_connections 2023-10-16T09:49:58.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.999+0000 7fcaa43a9640 1 --2- 172.21.15.84:0/2647377630 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fca8006ae70 0x7fca8006d330 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.999+0000 7fcaa43a9640 1 --2- 172.21.15.84:0/2647377630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca9c160b90 0x7fca9c1470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.999+0000 7fcaa43a9640 1 --2- 172.21.15.84:0/2647377630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca9c15d150 0x7fca9c146750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.999+0000 7fcaa43a9640 1 --2- 172.21.15.84:0/2647377630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca9c06a4b0 0x7fca9c0c3ef0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:57.999+0000 7fcaa43a9640 1 -- 172.21.15.84:0/2647377630 >> 172.21.15.84:0/2647377630 conn(0x7fca9c0ce9b0 msgr2=0x7fca9c15a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:58.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.000+0000 7fcaa43a9640 1 -- 172.21.15.84:0/2647377630 shutdown_connections 2023-10-16T09:49:58.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.000+0000 7fcaa43a9640 1 -- 172.21.15.84:0/2647377630 wait complete. 2023-10-16T09:49:58.020 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:49:58.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.226+0000 7efd3e415640 1 Processor -- start 2023-10-16T09:49:58.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.226+0000 7efd3e415640 1 -- start start 2023-10-16T09:49:58.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.228+0000 7efd3e415640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efd3815aaa0 0x7efd3815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:58.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.228+0000 7efd3e415640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efd380ce390 0x7efd38070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:58.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.228+0000 7efd3e415640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3815ed60 con 0x7efd3815ae80 2023-10-16T09:49:58.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.228+0000 7efd3e415640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3806b810 con 0x7efd380ce390 2023-10-16T09:49:58.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.228+0000 7efd3e415640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd380707e0 con 0x7efd3815aaa0 2023-10-16T09:49:58.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.228+0000 7efd3c9b2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efd380ce390 0x7efd38070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:58.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.228+0000 7efd3c9b2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efd380ce390 0x7efd38070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38266/0 (socket says 172.21.15.84:38266) 2023-10-16T09:49:58.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.228+0000 7efd3c9b2640 1 -- 172.21.15.84:0/3740407634 learned_addr learned my addr 172.21.15.84:0/3740407634 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:58.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.229+0000 7efd37fff640 1 --2- 172.21.15.84:0/3740407634 >> v2:172.21.15.157:3300/0 conn(0x7efd3815aaa0 0x7efd3815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:58.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.229+0000 7efd3c9b2640 1 -- 172.21.15.84:0/3740407634 >> v2:172.21.15.157:3300/0 conn(0x7efd3815aaa0 msgr2=0x7efd3815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:58.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.229+0000 7efd3c9b2640 1 --2- 172.21.15.84:0/3740407634 >> v2:172.21.15.157:3300/0 conn(0x7efd3815aaa0 0x7efd3815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.229+0000 7efd3c9b2640 1 -- 172.21.15.84:0/3740407634 >> v1:172.21.15.84:6789/0 conn(0x7efd3815ae80 legacy=0x7efd380cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:58.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.229+0000 7efd3c9b2640 1 -- 172.21.15.84:0/3740407634 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3815ef90 con 0x7efd380ce390 2023-10-16T09:49:58.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.229+0000 7efd377fe640 1 -- 172.21.15.84:0/3740407634 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812775458 0 0) 0x7efd3815ed60 con 0x7efd3815ae80 2023-10-16T09:49:58.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.230+0000 7efd3c9b2640 1 --2- 172.21.15.84:0/3740407634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd380ce390 0x7efd38070f50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efd380cd990 tx=0x7efd3002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfef4584ebeb91c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:58.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.230+0000 7efd377fe640 1 -- 172.21.15.84:0/3740407634 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3002cb20 con 0x7efd380ce390 2023-10-16T09:49:58.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.230+0000 7efd377fe640 1 -- 172.21.15.84:0/3740407634 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3002cc80 con 0x7efd380ce390 2023-10-16T09:49:58.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.230+0000 7efd377fe640 1 -- 172.21.15.84:0/3740407634 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd300343f0 con 0x7efd380ce390 2023-10-16T09:49:58.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.231+0000 7efd3e415640 1 -- 172.21.15.84:0/3740407634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd380ce390 msgr2=0x7efd38070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:58.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.231+0000 7efd3e415640 1 --2- 172.21.15.84:0/3740407634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd380ce390 0x7efd38070f50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efd380cd990 tx=0x7efd3002bf80 comp rx=0 tx=0).stop 2023-10-16T09:49:58.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.231+0000 7efd3e415640 1 -- 172.21.15.84:0/3740407634 shutdown_connections 2023-10-16T09:49:58.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.231+0000 7efd3e415640 1 --2- 172.21.15.84:0/3740407634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd380ce390 0x7efd38070f50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.231+0000 7efd3e415640 1 --2- 172.21.15.84:0/3740407634 >> v2:172.21.15.157:3300/0 conn(0x7efd3815aaa0 0x7efd3815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.231+0000 7efd3e415640 1 -- 172.21.15.84:0/3740407634 >> 172.21.15.84:0/3740407634 conn(0x7efd380cb410 msgr2=0x7efd380cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:58.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.231+0000 7efd3e415640 1 -- 172.21.15.84:0/3740407634 shutdown_connections 2023-10-16T09:49:58.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.231+0000 7efd3e415640 1 -- 172.21.15.84:0/3740407634 wait complete. 2023-10-16T09:49:58.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.233+0000 7efd3e415640 1 Processor -- start 2023-10-16T09:49:58.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.233+0000 7efd3e415640 1 -- start start 2023-10-16T09:49:58.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.234+0000 7efd3e415640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd380ce390 0x7efd3815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:58.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.234+0000 7efd3e415640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd3815aaa0 0x7efd38156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:58.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.234+0000 7efd3e415640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd3815ae80 0x7efd38149a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:58.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.234+0000 7efd3e415640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd38159b30 con 0x7efd380ce390 2023-10-16T09:49:58.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.234+0000 7efd3e415640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd380c3710 con 0x7efd3815ae80 2023-10-16T09:49:58.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.234+0000 7efd3e415640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd38159290 con 0x7efd3815aaa0 2023-10-16T09:49:58.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.234+0000 7efd37fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd380ce390 0x7efd3815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:58.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.234+0000 7efd37fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd380ce390 0x7efd3815dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38270/0 (socket says 172.21.15.84:38270) 2023-10-16T09:49:58.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.234+0000 7efd37fff640 1 -- 172.21.15.84:0/4230670774 learned_addr learned my addr 172.21.15.84:0/4230670774 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:58.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.235+0000 7efd37fff640 1 -- 172.21.15.84:0/4230670774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd3815aaa0 msgr2=0x7efd38156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:58.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.235+0000 7efd2ffff640 1 --2- 172.21.15.84:0/4230670774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd3815aaa0 0x7efd38156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:58.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.235+0000 7efd3c9b2640 1 --2- 172.21.15.84:0/4230670774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd3815ae80 0x7efd38149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:58.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.235+0000 7efd37fff640 1 --2- 172.21.15.84:0/4230670774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd3815aaa0 0x7efd38156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.235+0000 7efd37fff640 1 -- 172.21.15.84:0/4230670774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd3815ae80 msgr2=0x7efd38149a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:58.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.235+0000 7efd37fff640 1 --2- 172.21.15.84:0/4230670774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd3815ae80 0x7efd38149a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.235+0000 7efd37fff640 1 -- 172.21.15.84:0/4230670774 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd30009b30 con 0x7efd380ce390 2023-10-16T09:49:58.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.235+0000 7efd3c9b2640 1 --2- 172.21.15.84:0/4230670774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd3815ae80 0x7efd38149a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:58.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.235+0000 7efd37fff640 1 --2- 172.21.15.84:0/4230670774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd380ce390 0x7efd3815dc90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efd2800d830 tx=0x7efd2800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:58.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.235+0000 7efd357fa640 1 -- 172.21.15.84:0/4230670774 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2800b6c0 con 0x7efd380ce390 2023-10-16T09:49:58.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.236+0000 7efd357fa640 1 -- 172.21.15.84:0/4230670774 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2800b820 con 0x7efd380ce390 2023-10-16T09:49:58.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.236+0000 7efd357fa640 1 -- 172.21.15.84:0/4230670774 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2800bbb0 con 0x7efd380ce390 2023-10-16T09:49:58.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.236+0000 7efd3e415640 1 -- 172.21.15.84:0/4230670774 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd38159570 con 0x7efd380ce390 2023-10-16T09:49:58.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.236+0000 7efd3e415640 1 -- 172.21.15.84:0/4230670774 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3815e1d0 con 0x7efd380ce390 2023-10-16T09:49:58.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.237+0000 7efd3e415640 1 -- 172.21.15.84:0/4230670774 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3804e160 con 0x7efd380ce390 2023-10-16T09:49:58.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.238+0000 7efd357fa640 1 -- 172.21.15.84:0/4230670774 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efd2801e460 con 0x7efd380ce390 2023-10-16T09:49:58.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.239+0000 7efd357fa640 1 --2- 172.21.15.84:0/4230670774 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7efd1806aea0 0x7efd1806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:58.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.239+0000 7efd357fa640 1 -- 172.21.15.84:0/4230670774 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7efd28089d90 con 0x7efd380ce390 2023-10-16T09:49:58.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.239+0000 7efd2ffff640 1 --2- 172.21.15.84:0/4230670774 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7efd1806aea0 0x7efd1806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:58.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.240+0000 7efd2ffff640 1 --2- 172.21.15.84:0/4230670774 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7efd1806aea0 0x7efd1806d360 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd380b3d50 tx=0x7efd20009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:58.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.243+0000 7efd357fa640 1 -- 172.21.15.84:0/4230670774 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efd28054e50 con 0x7efd380ce390 2023-10-16T09:49:58.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.476+0000 7efd3e415640 1 -- 172.21.15.84:0/4230670774 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd380c34e0 con 0x7efd380ce390 2023-10-16T09:49:58.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.477+0000 7efd357fa640 1 -- 172.21.15.84:0/4230670774 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7efd28058470 con 0x7efd380ce390 2023-10-16T09:49:58.479 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:58.479 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2978888131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2978888131},{"type":"v1","addr":"172.21.15.157:6834","nonce":2978888131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3716581310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3716581310},{"type":"v1","addr":"172.21.15.84:6835","nonce":3716581310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2761479872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2761479872},{"type":"v1","addr":"172.21.15.84:6837","nonce":2761479872}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:49:02.350624+0000","modified":"2023-10-16T09:49:05.402785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/848754825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":848754825},{"type":"v1","addr":"172.21.15.157:6835","nonce":848754825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T09:49:58.480 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-10-16T09:49:58.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.479+0000 7efd3e415640 1 -- 172.21.15.84:0/4230670774 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7efd1806aea0 msgr2=0x7efd1806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:58.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.479+0000 7efd3e415640 1 --2- 172.21.15.84:0/4230670774 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7efd1806aea0 0x7efd1806d360 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd380b3d50 tx=0x7efd20009040 comp rx=0 tx=0).stop 2023-10-16T09:49:58.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.479+0000 7efd3e415640 1 -- 172.21.15.84:0/4230670774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd380ce390 msgr2=0x7efd3815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:58.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.479+0000 7efd3e415640 1 --2- 172.21.15.84:0/4230670774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd380ce390 0x7efd3815dc90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efd2800d830 tx=0x7efd2800dcf0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.479+0000 7efd3e415640 1 -- 172.21.15.84:0/4230670774 shutdown_connections 2023-10-16T09:49:58.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.479+0000 7efd3e415640 1 --2- 172.21.15.84:0/4230670774 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7efd1806aea0 0x7efd1806d360 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.479+0000 7efd3e415640 1 --2- 172.21.15.84:0/4230670774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd3815ae80 0x7efd38149a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.479+0000 7efd3e415640 1 --2- 172.21.15.84:0/4230670774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd3815aaa0 0x7efd38156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.479+0000 7efd3e415640 1 --2- 172.21.15.84:0/4230670774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd380ce390 0x7efd3815dc90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.479+0000 7efd3e415640 1 -- 172.21.15.84:0/4230670774 >> 172.21.15.84:0/4230670774 conn(0x7efd380cb410 msgr2=0x7efd380b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:58.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.479+0000 7efd3e415640 1 -- 172.21.15.84:0/4230670774 shutdown_connections 2023-10-16T09:49:58.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.480+0000 7efd3e415640 1 -- 172.21.15.84:0/4230670774 wait complete. 2023-10-16T09:49:58.498 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:49:58.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.703+0000 7fc5f3b3a640 1 Processor -- start 2023-10-16T09:49:58.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.703+0000 7fc5f3b3a640 1 -- start start 2023-10-16T09:49:58.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.704+0000 7fc5f3b3a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc5ec0ce390 0x7fc5ec15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:58.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.704+0000 7fc5f3b3a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ec15ed60 con 0x7fc5ec15ae80 2023-10-16T09:49:58.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.704+0000 7fc5f3b3a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ec0707e0 con 0x7fc5ec15aaa0 2023-10-16T09:49:58.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.704+0000 7fc5f3b3a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ec06b810 con 0x7fc5ec0ce390 2023-10-16T09:49:58.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.705+0000 7fc5f10d5640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc5ec15ae80 0x7fc5ec070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36226/0 (socket says 172.21.15.84:36226) 2023-10-16T09:49:58.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.705+0000 7fc5f10d5640 1 -- 172.21.15.84:0/1109028917 learned_addr learned my addr 172.21.15.84:0/1109028917 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:58.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.705+0000 7fc5f08d4640 1 -- 172.21.15.84:0/1109028917 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614452018 0 0) 0x7fc5ec15ed60 con 0x7fc5ec15ae80 2023-10-16T09:49:58.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.705+0000 7fc5f08d4640 1 -- 172.21.15.84:0/1109028917 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5d8003520 con 0x7fc5ec15ae80 2023-10-16T09:49:58.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.705+0000 7fc5f20d7640 1 --2- 172.21.15.84:0/1109028917 >> v2:172.21.15.157:3300/0 conn(0x7fc5ec0ce390 0x7fc5ec15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:58.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.705+0000 7fc5f08d4640 1 -- 172.21.15.84:0/1109028917 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 247133698 0 0) 0x7fc5d8003520 con 0x7fc5ec15ae80 2023-10-16T09:49:58.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.706+0000 7fc5f08d4640 1 -- 172.21.15.84:0/1109028917 >> v2:172.21.15.157:3300/0 conn(0x7fc5ec0ce390 msgr2=0x7fc5ec15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:58.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.706+0000 7fc5f08d4640 1 --2- 172.21.15.84:0/1109028917 >> v2:172.21.15.157:3300/0 conn(0x7fc5ec0ce390 0x7fc5ec15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.706+0000 7fc5f08d4640 1 -- 172.21.15.84:0/1109028917 >> v1:172.21.15.157:6789/0 conn(0x7fc5ec15aaa0 legacy=0x7fc5ec0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:58.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.706+0000 7fc5f08d4640 1 -- 172.21.15.84:0/1109028917 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec15ef90 con 0x7fc5ec15ae80 2023-10-16T09:49:58.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.706+0000 7fc5f08d4640 1 -- 172.21.15.84:0/1109028917 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc5dc001670 con 0x7fc5ec15ae80 2023-10-16T09:49:58.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.706+0000 7fc5f08d4640 1 -- 172.21.15.84:0/1109028917 >> v1:172.21.15.84:6789/0 conn(0x7fc5ec15ae80 legacy=0x7fc5ec070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:58.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.706+0000 7fc5f08d4640 1 --2- 172.21.15.84:0/1109028917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d8003970 0x7fc5d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:58.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.706+0000 7fc5f08d4640 1 --2- 172.21.15.84:0/1109028917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5d80047c0 0x7fc5d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:58.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.706+0000 7fc5f08d4640 1 --2- 172.21.15.84:0/1109028917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5d8005180 0x7fc5d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:58.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.706+0000 7fc5f08d4640 1 -- 172.21.15.84:0/1109028917 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec06b810 con 0x7fc5d8005180 2023-10-16T09:49:58.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.706+0000 7fc5f08d4640 1 -- 172.21.15.84:0/1109028917 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d800c0d0 con 0x7fc5d8003970 2023-10-16T09:49:58.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.706+0000 7fc5f08d4640 1 -- 172.21.15.84:0/1109028917 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d800c290 con 0x7fc5d80047c0 2023-10-16T09:49:58.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.706+0000 7fc5f20d7640 1 --2- 172.21.15.84:0/1109028917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5d8005180 0x7fc5d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:58.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.706+0000 7fc5f20d7640 1 -- 172.21.15.84:0/1109028917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5d80047c0 msgr2=0x7fc5d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:58.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.706+0000 7fc5f20d7640 1 --2- 172.21.15.84:0/1109028917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5d80047c0 0x7fc5d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.706+0000 7fc5f20d7640 1 -- 172.21.15.84:0/1109028917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d8003970 msgr2=0x7fc5d8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.706+0000 7fc5f20d7640 1 --2- 172.21.15.84:0/1109028917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d8003970 0x7fc5d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.706+0000 7fc5f20d7640 1 -- 172.21.15.84:0/1109028917 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d8003520 con 0x7fc5d8005180 2023-10-16T09:49:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.707+0000 7fc5f20d7640 1 -- 172.21.15.84:0/1109028917 reap_dead start 2023-10-16T09:49:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.707+0000 7fc5f20d7640 1 --2- 172.21.15.84:0/1109028917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5d8005180 0x7fc5d800b980 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4007360 tx=0x7fc5d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b95571f64c5cc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.708+0000 7fc5f08d4640 1 -- 172.21.15.84:0/1109028917 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e402ccf0 con 0x7fc5d8005180 2023-10-16T09:49:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.708+0000 7fc5f08d4640 1 -- 172.21.15.84:0/1109028917 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e402ce90 con 0x7fc5d8005180 2023-10-16T09:49:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.708+0000 7fc5f08d4640 1 -- 172.21.15.84:0/1109028917 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4003bd0 con 0x7fc5d8005180 2023-10-16T09:49:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.708+0000 7fc5f3b3a640 1 -- 172.21.15.84:0/1109028917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5d8005180 msgr2=0x7fc5d800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.708+0000 7fc5f3b3a640 1 --2- 172.21.15.84:0/1109028917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5d8005180 0x7fc5d800b980 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4007360 tx=0x7fc5d80057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.708+0000 7fc5f3b3a640 1 -- 172.21.15.84:0/1109028917 shutdown_connections 2023-10-16T09:49:58.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.708+0000 7fc5f3b3a640 1 --2- 172.21.15.84:0/1109028917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5d8005180 0x7fc5d800b980 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.708+0000 7fc5f3b3a640 1 -- 172.21.15.84:0/1109028917 >> 172.21.15.84:0/1109028917 conn(0x7fc5ec0cb410 msgr2=0x7fc5ec0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:58.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.708+0000 7fc5f3b3a640 1 -- 172.21.15.84:0/1109028917 shutdown_connections 2023-10-16T09:49:58.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.708+0000 7fc5f3b3a640 1 -- 172.21.15.84:0/1109028917 wait complete. 2023-10-16T09:49:58.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.710+0000 7fc5f3b3a640 1 Processor -- start 2023-10-16T09:49:58.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.710+0000 7fc5f3b3a640 1 -- start start 2023-10-16T09:49:58.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.711+0000 7fc5f3b3a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d8005180 0x7fc5ec1f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:58.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.711+0000 7fc5f3b3a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5ec15b3b0 0x7fc5ec1f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:58.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.711+0000 7fc5f3b3a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5ec15d960 0x7fc5ec15dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:58.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.711+0000 7fc5f3b3a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec15bc50 con 0x7fc5ec15d960 2023-10-16T09:49:58.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.711+0000 7fc5f3b3a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec15bf30 con 0x7fc5d8005180 2023-10-16T09:49:58.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.711+0000 7fc5f3b3a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec15c210 con 0x7fc5ec15b3b0 2023-10-16T09:49:58.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.711+0000 7fc5f20d7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5ec15d960 0x7fc5ec15dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:58.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.711+0000 7fc5f20d7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5ec15d960 0x7fc5ec15dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38274/0 (socket says 172.21.15.84:38274) 2023-10-16T09:49:58.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.712+0000 7fc5f20d7640 1 -- 172.21.15.84:0/3430262701 learned_addr learned my addr 172.21.15.84:0/3430262701 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:58.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.712+0000 7fc5f10d5640 1 --2- 172.21.15.84:0/3430262701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5ec15b3b0 0x7fc5ec1f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:58.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.712+0000 7fc5f18d6640 1 --2- 172.21.15.84:0/3430262701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d8005180 0x7fc5ec1f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:58.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.712+0000 7fc5f20d7640 1 -- 172.21.15.84:0/3430262701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5ec15b3b0 msgr2=0x7fc5ec1f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:58.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.712+0000 7fc5f20d7640 1 --2- 172.21.15.84:0/3430262701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5ec15b3b0 0x7fc5ec1f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.712+0000 7fc5f20d7640 1 -- 172.21.15.84:0/3430262701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d8005180 msgr2=0x7fc5ec1f2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:58.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.712+0000 7fc5f20d7640 1 --2- 172.21.15.84:0/3430262701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d8005180 0x7fc5ec1f2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.712+0000 7fc5f20d7640 1 -- 172.21.15.84:0/3430262701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec1f5630 con 0x7fc5ec15d960 2023-10-16T09:49:58.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.712+0000 7fc5f10d5640 1 --2- 172.21.15.84:0/3430262701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5ec15b3b0 0x7fc5ec1f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:58.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.712+0000 7fc5f18d6640 1 --2- 172.21.15.84:0/3430262701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d8005180 0x7fc5ec1f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:58.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.712+0000 7fc5f20d7640 1 --2- 172.21.15.84:0/3430262701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5ec15d960 0x7fc5ec15dde0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc5e402c760 tx=0x7fc5e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:58.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.713+0000 7fc5e27fc640 1 -- 172.21.15.84:0/3430262701 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4039070 con 0x7fc5ec15d960 2023-10-16T09:49:58.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.713+0000 7fc5e27fc640 1 -- 172.21.15.84:0/3430262701 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e4003ed0 con 0x7fc5ec15d960 2023-10-16T09:49:58.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.713+0000 7fc5e27fc640 1 -- 172.21.15.84:0/3430262701 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4004820 con 0x7fc5ec15d960 2023-10-16T09:49:58.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.713+0000 7fc5f3b3a640 1 -- 172.21.15.84:0/3430262701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ec0ac890 con 0x7fc5ec15d960 2023-10-16T09:49:58.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.713+0000 7fc5f3b3a640 1 -- 172.21.15.84:0/3430262701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ec0acd50 con 0x7fc5ec15d960 2023-10-16T09:49:58.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.714+0000 7fc5f3b3a640 1 -- 172.21.15.84:0/3430262701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ec04e160 con 0x7fc5ec15d960 2023-10-16T09:49:58.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.715+0000 7fc5e27fc640 1 -- 172.21.15.84:0/3430262701 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc5e4034430 con 0x7fc5ec15d960 2023-10-16T09:49:58.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.716+0000 7fc5e27fc640 1 --2- 172.21.15.84:0/3430262701 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc5d006ae70 0x7fc5d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:58.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.716+0000 7fc5e27fc640 1 -- 172.21.15.84:0/3430262701 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc5e40aa4e0 con 0x7fc5ec15d960 2023-10-16T09:49:58.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.716+0000 7fc5f18d6640 1 --2- 172.21.15.84:0/3430262701 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc5d006ae70 0x7fc5d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:58.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.717+0000 7fc5f18d6640 1 --2- 172.21.15.84:0/3430262701 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc5d006ae70 0x7fc5d006d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8001660 tx=0x7fc5e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:58.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.720+0000 7fc5e27fc640 1 -- 172.21.15.84:0/3430262701 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc5e4075500 con 0x7fc5ec15d960 2023-10-16T09:49:58.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.915+0000 7fc5f3b3a640 1 -- 172.21.15.84:0/3430262701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5ec15c3d0 con 0x7fc5ec15d960 2023-10-16T09:49:58.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.917+0000 7fc5e27fc640 1 -- 172.21.15.84:0/3430262701 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16730 (secure 0 0 0) 0x7fc5e4078b20 con 0x7fc5ec15d960 2023-10-16T09:49:58.920 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:58.920 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":23,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:49.552121+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.27","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:49:58.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.920+0000 7fc5f3b3a640 1 -- 172.21.15.84:0/3430262701 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc5d006ae70 msgr2=0x7fc5d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:58.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.920+0000 7fc5f3b3a640 1 --2- 172.21.15.84:0/3430262701 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc5d006ae70 0x7fc5d006d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8001660 tx=0x7fc5e8009040 comp rx=0 tx=0).stop 2023-10-16T09:49:58.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.920+0000 7fc5f3b3a640 1 -- 172.21.15.84:0/3430262701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5ec15d960 msgr2=0x7fc5ec15dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:58.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.920+0000 7fc5f3b3a640 1 --2- 172.21.15.84:0/3430262701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5ec15d960 0x7fc5ec15dde0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc5e402c760 tx=0x7fc5e4004ab0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.921+0000 7fc5f3b3a640 1 -- 172.21.15.84:0/3430262701 shutdown_connections 2023-10-16T09:49:58.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.921+0000 7fc5f3b3a640 1 --2- 172.21.15.84:0/3430262701 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc5d006ae70 0x7fc5d006d330 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.921+0000 7fc5f3b3a640 1 --2- 172.21.15.84:0/3430262701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5ec15d960 0x7fc5ec15dde0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.921+0000 7fc5f3b3a640 1 --2- 172.21.15.84:0/3430262701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5ec15b3b0 0x7fc5ec1f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.921+0000 7fc5f3b3a640 1 --2- 172.21.15.84:0/3430262701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d8005180 0x7fc5ec1f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:58.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.921+0000 7fc5f3b3a640 1 -- 172.21.15.84:0/3430262701 >> 172.21.15.84:0/3430262701 conn(0x7fc5ec0cb410 msgr2=0x7fc5ec06cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:58.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.921+0000 7fc5f3b3a640 1 -- 172.21.15.84:0/3430262701 shutdown_connections 2023-10-16T09:49:58.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:58.921+0000 7fc5f3b3a640 1 -- 172.21.15.84:0/3430262701 wait complete. 2023-10-16T09:49:58.942 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T09:49:58.942 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:49:58.979 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:49:58.994 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:49:59.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.141+0000 7feefa7a9640 1 Processor -- start 2023-10-16T09:49:59.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.141+0000 7feefa7a9640 1 -- start start 2023-10-16T09:49:59.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.142+0000 7feefa7a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feef415ae80 0x7feef415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:59.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.142+0000 7feefa7a9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef415ed60 con 0x7feef40ce390 2023-10-16T09:49:59.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.142+0000 7feefa7a9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef40707e0 con 0x7feef415aaa0 2023-10-16T09:49:59.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.142+0000 7feefa7a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef406b810 con 0x7feef415ae80 2023-10-16T09:49:59.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.143+0000 7feef8d46640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7feef40ce390 0x7feef4070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36234/0 (socket says 172.21.15.84:36234) 2023-10-16T09:49:59.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.143+0000 7feef8d46640 1 -- 172.21.15.84:0/1241702493 learned_addr learned my addr 172.21.15.84:0/1241702493 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:59.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.143+0000 7feeebfff640 1 --2- 172.21.15.84:0/1241702493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feef415ae80 0x7feef415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:59.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.143+0000 7feef37fe640 1 -- 172.21.15.84:0/1241702493 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998864013 0 0) 0x7feef415ed60 con 0x7feef40ce390 2023-10-16T09:49:59.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.143+0000 7feef37fe640 1 -- 172.21.15.84:0/1241702493 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feee0003520 con 0x7feef40ce390 2023-10-16T09:49:59.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.144+0000 7feef37fe640 1 -- 172.21.15.84:0/1241702493 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3539645693 0 0) 0x7feee0003520 con 0x7feef40ce390 2023-10-16T09:49:59.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.144+0000 7feef37fe640 1 -- 172.21.15.84:0/1241702493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feef415ae80 msgr2=0x7feef415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:59.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.144+0000 7feef37fe640 1 --2- 172.21.15.84:0/1241702493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feef415ae80 0x7feef415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.144+0000 7feef37fe640 1 -- 172.21.15.84:0/1241702493 >> v1:172.21.15.157:6789/0 conn(0x7feef415aaa0 legacy=0x7feef40cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:59.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.144+0000 7feef37fe640 1 -- 172.21.15.84:0/1241702493 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef415ef90 con 0x7feef40ce390 2023-10-16T09:49:59.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.144+0000 7feef37fe640 1 -- 172.21.15.84:0/1241702493 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7feeec0019f0 con 0x7feef40ce390 2023-10-16T09:49:59.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.144+0000 7feef37fe640 1 -- 172.21.15.84:0/1241702493 >> v1:172.21.15.84:6789/0 conn(0x7feef40ce390 legacy=0x7feef4070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:59.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.144+0000 7feef37fe640 1 --2- 172.21.15.84:0/1241702493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feef415ae80 0x7feee00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:59.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.144+0000 7feef37fe640 1 --2- 172.21.15.84:0/1241702493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feee0003ee0 0x7feee00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:59.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.144+0000 7feef37fe640 1 --2- 172.21.15.84:0/1241702493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee0004d00 0x7feee0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:59.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.144+0000 7feef37fe640 1 -- 172.21.15.84:0/1241702493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef406b810 con 0x7feee0004d00 2023-10-16T09:49:59.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.144+0000 7feef37fe640 1 -- 172.21.15.84:0/1241702493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee000ba30 con 0x7feee0003ee0 2023-10-16T09:49:59.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.144+0000 7feef37fe640 1 -- 172.21.15.84:0/1241702493 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee000bbf0 con 0x7feef415ae80 2023-10-16T09:49:59.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.144+0000 7feef8d46640 1 --2- 172.21.15.84:0/1241702493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee0004d00 0x7feee0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:59.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.144+0000 7feef3fff640 1 --2- 172.21.15.84:0/1241702493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feef415ae80 0x7feee00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:59.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.144+0000 7feeebfff640 1 --2- 172.21.15.84:0/1241702493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feee0003ee0 0x7feee00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:59.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.145+0000 7feef8d46640 1 -- 172.21.15.84:0/1241702493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feef415ae80 msgr2=0x7feee00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:59.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.145+0000 7feef8d46640 1 --2- 172.21.15.84:0/1241702493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feef415ae80 0x7feee00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.145+0000 7feef8d46640 1 -- 172.21.15.84:0/1241702493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feee0003ee0 msgr2=0x7feee00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:59.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.145+0000 7feef8d46640 1 --2- 172.21.15.84:0/1241702493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feee0003ee0 0x7feee00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.145+0000 7feef8d46640 1 -- 172.21.15.84:0/1241702493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee0003520 con 0x7feee0004d00 2023-10-16T09:49:59.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.145+0000 7feeebfff640 1 --2- 172.21.15.84:0/1241702493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feee0003ee0 0x7feee00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:59.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.146+0000 7feef8d46640 1 --2- 172.21.15.84:0/1241702493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee0004d00 0x7feee0005180 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7feeec00a810 tx=0x7feee00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a621bfe54b69c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:59.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.146+0000 7feef37fe640 1 -- 172.21.15.84:0/1241702493 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec034630 con 0x7feee0004d00 2023-10-16T09:49:59.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.146+0000 7feef37fe640 1 -- 172.21.15.84:0/1241702493 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec02f070 con 0x7feee0004d00 2023-10-16T09:49:59.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.146+0000 7feef37fe640 1 -- 172.21.15.84:0/1241702493 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0349d0 con 0x7feee0004d00 2023-10-16T09:49:59.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.146+0000 7feefa7a9640 1 -- 172.21.15.84:0/1241702493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee0004d00 msgr2=0x7feee0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:59.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.146+0000 7feefa7a9640 1 --2- 172.21.15.84:0/1241702493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee0004d00 0x7feee0005180 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7feeec00a810 tx=0x7feee00057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.146+0000 7feef8d46640 1 -- 172.21.15.84:0/1241702493 reap_dead start 2023-10-16T09:49:59.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.147+0000 7feefa7a9640 1 -- 172.21.15.84:0/1241702493 shutdown_connections 2023-10-16T09:49:59.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.147+0000 7feefa7a9640 1 -- 172.21.15.84:0/1241702493 >> 172.21.15.84:0/1241702493 conn(0x7feef40cb410 msgr2=0x7feef40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:59.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.147+0000 7feefa7a9640 1 -- 172.21.15.84:0/1241702493 shutdown_connections 2023-10-16T09:49:59.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.147+0000 7feefa7a9640 1 -- 172.21.15.84:0/1241702493 wait complete. 2023-10-16T09:49:59.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.148+0000 7feefa7a9640 1 Processor -- start 2023-10-16T09:49:59.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.148+0000 7feefa7a9640 1 -- start start 2023-10-16T09:49:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.149+0000 7feefa7a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee0004d00 0x7feef414ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.149+0000 7feefa7a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feef4154f50 0x7feef40ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.149+0000 7feefa7a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feef40acdd0 0x7feef40c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.149+0000 7feefa7a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef414d3c0 con 0x7feee0004d00 2023-10-16T09:49:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.149+0000 7feefa7a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef40ad250 con 0x7feef40acdd0 2023-10-16T09:49:59.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.149+0000 7feefa7a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef40c7be0 con 0x7feef4154f50 2023-10-16T09:49:59.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.150+0000 7feef3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee0004d00 0x7feef414ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:59.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.150+0000 7feef3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee0004d00 0x7feef414ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38286/0 (socket says 172.21.15.84:38286) 2023-10-16T09:49:59.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.150+0000 7feef3fff640 1 -- 172.21.15.84:0/2156940611 learned_addr learned my addr 172.21.15.84:0/2156940611 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:59.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.150+0000 7feef3fff640 1 -- 172.21.15.84:0/2156940611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feef4154f50 msgr2=0x7feef40ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:59.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.150+0000 7feef3fff640 1 --2- 172.21.15.84:0/2156940611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feef4154f50 0x7feef40ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.150+0000 7feef3fff640 1 -- 172.21.15.84:0/2156940611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feef40acdd0 msgr2=0x7feef40c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:59.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.150+0000 7feef3fff640 1 --2- 172.21.15.84:0/2156940611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feef40acdd0 0x7feef40c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.150+0000 7feef3fff640 1 -- 172.21.15.84:0/2156940611 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef40c7ec0 con 0x7feee0004d00 2023-10-16T09:49:59.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.150+0000 7feef3fff640 1 --2- 172.21.15.84:0/2156940611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee0004d00 0x7feef414ce80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7feee400e7d0 tx=0x7feee400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:59.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.151+0000 7feef17fa640 1 -- 172.21.15.84:0/2156940611 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee40050d0 con 0x7feee0004d00 2023-10-16T09:49:59.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.151+0000 7feef17fa640 1 -- 172.21.15.84:0/2156940611 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee4005230 con 0x7feee0004d00 2023-10-16T09:49:59.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.151+0000 7feef17fa640 1 -- 172.21.15.84:0/2156940611 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee400c640 con 0x7feee0004d00 2023-10-16T09:49:59.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.151+0000 7feefa7a9640 1 -- 172.21.15.84:0/2156940611 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef40c81a0 con 0x7feee0004d00 2023-10-16T09:49:59.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.151+0000 7feefa7a9640 1 -- 172.21.15.84:0/2156940611 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef40c8760 con 0x7feee0004d00 2023-10-16T09:49:59.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.152+0000 7feefa7a9640 1 -- 172.21.15.84:0/2156940611 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef404e160 con 0x7feee0004d00 2023-10-16T09:49:59.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.153+0000 7feef17fa640 1 -- 172.21.15.84:0/2156940611 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feee400c7a0 con 0x7feee0004d00 2023-10-16T09:49:59.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.154+0000 7feef17fa640 1 --2- 172.21.15.84:0/2156940611 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7feed406ae70 0x7feed406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:59.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.154+0000 7feef17fa640 1 -- 172.21.15.84:0/2156940611 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7feee4089520 con 0x7feee0004d00 2023-10-16T09:49:59.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.154+0000 7feeebfff640 1 --2- 172.21.15.84:0/2156940611 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7feed406ae70 0x7feed406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:59.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.155+0000 7feeebfff640 1 --2- 172.21.15.84:0/2156940611 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7feed406ae70 0x7feed406d330 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feef406d7c0 tx=0x7feedc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:59.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.158+0000 7feef17fa640 1 -- 172.21.15.84:0/2156940611 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7feee4054550 con 0x7feee0004d00 2023-10-16T09:49:59.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.374+0000 7feefa7a9640 1 -- 172.21.15.84:0/2156940611 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feef40cdc60 con 0x7feee0004d00 2023-10-16T09:49:59.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.376+0000 7feef17fa640 1 -- 172.21.15.84:0/2156940611 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7feee40012d0 con 0x7feee0004d00 2023-10-16T09:49:59.378 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:59.378 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2978888131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2978888131},{"type":"v1","addr":"172.21.15.157:6834","nonce":2978888131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3716581310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3716581310},{"type":"v1","addr":"172.21.15.84:6835","nonce":3716581310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2761479872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2761479872},{"type":"v1","addr":"172.21.15.84:6837","nonce":2761479872}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:49:02.350624+0000","modified":"2023-10-16T09:49:05.402785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/848754825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":848754825},{"type":"v1","addr":"172.21.15.157:6835","nonce":848754825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T09:49:59.378 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-10-16T09:49:59.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.378+0000 7feefa7a9640 1 -- 172.21.15.84:0/2156940611 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7feed406ae70 msgr2=0x7feed406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:59.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.378+0000 7feefa7a9640 1 --2- 172.21.15.84:0/2156940611 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7feed406ae70 0x7feed406d330 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feef406d7c0 tx=0x7feedc008040 comp rx=0 tx=0).stop 2023-10-16T09:49:59.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.378+0000 7feefa7a9640 1 -- 172.21.15.84:0/2156940611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee0004d00 msgr2=0x7feef414ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:59.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.378+0000 7feefa7a9640 1 --2- 172.21.15.84:0/2156940611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee0004d00 0x7feef414ce80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7feee400e7d0 tx=0x7feee400ec90 comp rx=0 tx=0).stop 2023-10-16T09:49:59.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.378+0000 7feefa7a9640 1 -- 172.21.15.84:0/2156940611 shutdown_connections 2023-10-16T09:49:59.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.378+0000 7feefa7a9640 1 --2- 172.21.15.84:0/2156940611 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7feed406ae70 0x7feed406d330 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.378+0000 7feefa7a9640 1 --2- 172.21.15.84:0/2156940611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feef40acdd0 0x7feef40c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.378+0000 7feefa7a9640 1 --2- 172.21.15.84:0/2156940611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feef4154f50 0x7feef40ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.378+0000 7feefa7a9640 1 --2- 172.21.15.84:0/2156940611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee0004d00 0x7feef414ce80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.378+0000 7feefa7a9640 1 -- 172.21.15.84:0/2156940611 >> 172.21.15.84:0/2156940611 conn(0x7feef40cb410 msgr2=0x7feef406ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:59.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.379+0000 7feefa7a9640 1 -- 172.21.15.84:0/2156940611 shutdown_connections 2023-10-16T09:49:59.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.379+0000 7feefa7a9640 1 -- 172.21.15.84:0/2156940611 wait complete. 2023-10-16T09:49:59.398 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:49:59.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.645+0000 7fa55897c640 1 Processor -- start 2023-10-16T09:49:59.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.645+0000 7fa55897c640 1 -- start start 2023-10-16T09:49:59.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.646+0000 7fa55897c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa554160800 0x7fa5540ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:59.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.646+0000 7fa55897c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa55406e170 con 0x7fa554160420 2023-10-16T09:49:59.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.646+0000 7fa55897c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5540cd460 con 0x7fa55406b4c0 2023-10-16T09:49:59.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.646+0000 7fa55897c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55406e3c0 con 0x7fa554160800 2023-10-16T09:49:59.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.646+0000 7fa55259c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa554160420 0x7fa55406adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36238/0 (socket says 172.21.15.84:36238) 2023-10-16T09:49:59.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.646+0000 7fa55259c640 1 -- 172.21.15.84:0/444834129 learned_addr learned my addr 172.21.15.84:0/444834129 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:49:59.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.647+0000 7fa552d9d640 1 --2- 172.21.15.84:0/444834129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa554160800 0x7fa5540ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:59.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.647+0000 7fa551d9b640 1 -- 172.21.15.84:0/444834129 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696617698 0 0) 0x7fa55406e170 con 0x7fa554160420 2023-10-16T09:49:59.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.647+0000 7fa551d9b640 1 -- 172.21.15.84:0/444834129 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa540003520 con 0x7fa554160420 2023-10-16T09:49:59.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.647+0000 7fa551d9b640 1 -- 172.21.15.84:0/444834129 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3425432467 0 0) 0x7fa540003520 con 0x7fa554160420 2023-10-16T09:49:59.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.647+0000 7fa551d9b640 1 -- 172.21.15.84:0/444834129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa554160800 msgr2=0x7fa5540ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:59.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.647+0000 7fa551d9b640 1 --2- 172.21.15.84:0/444834129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa554160800 0x7fa5540ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.647+0000 7fa551d9b640 1 -- 172.21.15.84:0/444834129 >> v1:172.21.15.157:6789/0 conn(0x7fa55406b4c0 legacy=0x7fa554070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:59.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.647+0000 7fa551d9b640 1 -- 172.21.15.84:0/444834129 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55406e6a0 con 0x7fa554160420 2023-10-16T09:49:59.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.647+0000 7fa551d9b640 1 -- 172.21.15.84:0/444834129 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fa544001670 con 0x7fa554160420 2023-10-16T09:49:59.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.647+0000 7fa551d9b640 1 -- 172.21.15.84:0/444834129 >> v1:172.21.15.84:6789/0 conn(0x7fa554160420 legacy=0x7fa55406adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:59.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.647+0000 7fa551d9b640 1 --2- 172.21.15.84:0/444834129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa554160800 0x7fa5400039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:59.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.647+0000 7fa551d9b640 1 --2- 172.21.15.84:0/444834129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa540003f20 0x7fa5400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:59.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.647+0000 7fa551d9b640 1 --2- 172.21.15.84:0/444834129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa540004d00 0x7fa540005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:59.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.648+0000 7fa551d9b640 1 -- 172.21.15.84:0/444834129 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55406e3c0 con 0x7fa540003f20 2023-10-16T09:49:59.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.648+0000 7fa551d9b640 1 -- 172.21.15.84:0/444834129 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54000ba60 con 0x7fa540004d00 2023-10-16T09:49:59.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.648+0000 7fa551d9b640 1 -- 172.21.15.84:0/444834129 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54000bc20 con 0x7fa554160800 2023-10-16T09:49:59.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.648+0000 7fa54bfff640 1 --2- 172.21.15.84:0/444834129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa540003f20 0x7fa5400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:59.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.648+0000 7fa552d9d640 1 --2- 172.21.15.84:0/444834129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa540004d00 0x7fa540005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:59.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.648+0000 7fa55259c640 1 --2- 172.21.15.84:0/444834129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa554160800 0x7fa5400039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:59.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.648+0000 7fa54bfff640 1 -- 172.21.15.84:0/444834129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa554160800 msgr2=0x7fa5400039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:59.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.648+0000 7fa54bfff640 1 --2- 172.21.15.84:0/444834129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa554160800 0x7fa5400039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.648+0000 7fa54bfff640 1 -- 172.21.15.84:0/444834129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa540004d00 msgr2=0x7fa540005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:49:59.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.648+0000 7fa54bfff640 1 --2- 172.21.15.84:0/444834129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa540004d00 0x7fa540005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.648+0000 7fa54bfff640 1 -- 172.21.15.84:0/444834129 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa540003520 con 0x7fa540003f20 2023-10-16T09:49:59.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.648+0000 7fa55259c640 1 --2- 172.21.15.84:0/444834129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa554160800 0x7fa5400039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:49:59.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.650+0000 7fa54bfff640 1 --2- 172.21.15.84:0/444834129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa540003f20 0x7fa5400047c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa53c001750 tx=0x7fa5400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3075ca26491d11 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:59.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.650+0000 7fa551d9b640 1 -- 172.21.15.84:0/444834129 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53c0052d0 con 0x7fa540003f20 2023-10-16T09:49:59.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.650+0000 7fa551d9b640 1 -- 172.21.15.84:0/444834129 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53c02e050 con 0x7fa540003f20 2023-10-16T09:49:59.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.650+0000 7fa551d9b640 1 -- 172.21.15.84:0/444834129 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53c003830 con 0x7fa540003f20 2023-10-16T09:49:59.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.650+0000 7fa55897c640 1 -- 172.21.15.84:0/444834129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa540003f20 msgr2=0x7fa5400047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:59.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.650+0000 7fa55897c640 1 --2- 172.21.15.84:0/444834129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa540003f20 0x7fa5400047c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa53c001750 tx=0x7fa5400057f0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.650+0000 7fa552d9d640 1 -- 172.21.15.84:0/444834129 reap_dead start 2023-10-16T09:49:59.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.650+0000 7fa55897c640 1 -- 172.21.15.84:0/444834129 shutdown_connections 2023-10-16T09:49:59.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.650+0000 7fa55897c640 1 -- 172.21.15.84:0/444834129 >> 172.21.15.84:0/444834129 conn(0x7fa5540b0d30 msgr2=0x7fa5540b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:59.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.651+0000 7fa55897c640 1 -- 172.21.15.84:0/444834129 shutdown_connections 2023-10-16T09:49:59.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.651+0000 7fa55897c640 1 -- 172.21.15.84:0/444834129 wait complete. 2023-10-16T09:49:59.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.652+0000 7fa55897c640 1 Processor -- start 2023-10-16T09:49:59.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.652+0000 7fa55897c640 1 -- start start 2023-10-16T09:49:59.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.653+0000 7fa55897c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa540003f20 0x7fa55415a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:59.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.653+0000 7fa55897c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa55409e6f0 0x7fa55409dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:59.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.653+0000 7fa55897c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5540cddb0 0x7fa5540ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:59.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.653+0000 7fa55897c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55409f810 con 0x7fa540003f20 2023-10-16T09:49:59.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.653+0000 7fa55897c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5540ce770 con 0x7fa5540cddb0 2023-10-16T09:49:59.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.653+0000 7fa55897c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55415a6a0 con 0x7fa55409e6f0 2023-10-16T09:49:59.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.654+0000 7fa55259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa540003f20 0x7fa55415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:59.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.654+0000 7fa55259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa540003f20 0x7fa55415a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38308/0 (socket says 172.21.15.84:38308) 2023-10-16T09:49:59.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.654+0000 7fa55259c640 1 -- 172.21.15.84:0/3935822232 learned_addr learned my addr 172.21.15.84:0/3935822232 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:49:59.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.654+0000 7fa54bfff640 1 --2- 172.21.15.84:0/3935822232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa55409e6f0 0x7fa55409dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:59.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.654+0000 7fa552d9d640 1 --2- 172.21.15.84:0/3935822232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5540cddb0 0x7fa5540ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:59.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.654+0000 7fa55259c640 1 -- 172.21.15.84:0/3935822232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa55409e6f0 msgr2=0x7fa55409dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:59.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.654+0000 7fa55259c640 1 --2- 172.21.15.84:0/3935822232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa55409e6f0 0x7fa55409dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.654+0000 7fa55259c640 1 -- 172.21.15.84:0/3935822232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5540cddb0 msgr2=0x7fa5540ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:59.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.654+0000 7fa55259c640 1 --2- 172.21.15.84:0/3935822232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5540cddb0 0x7fa5540ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.654+0000 7fa55259c640 1 -- 172.21.15.84:0/3935822232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55409e2b0 con 0x7fa540003f20 2023-10-16T09:49:59.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.655+0000 7fa55259c640 1 --2- 172.21.15.84:0/3935822232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa540003f20 0x7fa55415a160 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa54400d7e0 tx=0x7fa54400dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:59.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.655+0000 7fa54affd640 1 -- 172.21.15.84:0/3935822232 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5440074e0 con 0x7fa540003f20 2023-10-16T09:49:59.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.655+0000 7fa54affd640 1 -- 172.21.15.84:0/3935822232 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa544007640 con 0x7fa540003f20 2023-10-16T09:49:59.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.655+0000 7fa54affd640 1 -- 172.21.15.84:0/3935822232 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5440049e0 con 0x7fa540003f20 2023-10-16T09:49:59.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.655+0000 7fa55897c640 1 -- 172.21.15.84:0/3935822232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55409eb90 con 0x7fa540003f20 2023-10-16T09:49:59.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.655+0000 7fa55897c640 1 -- 172.21.15.84:0/3935822232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55409f150 con 0x7fa540003f20 2023-10-16T09:49:59.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.656+0000 7fa55897c640 1 -- 172.21.15.84:0/3935822232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55404e160 con 0x7fa540003f20 2023-10-16T09:49:59.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.657+0000 7fa54affd640 1 -- 172.21.15.84:0/3935822232 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa544004b40 con 0x7fa540003f20 2023-10-16T09:49:59.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.658+0000 7fa54affd640 1 --2- 172.21.15.84:0/3935822232 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fa53406aea0 0x7fa53406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:49:59.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.659+0000 7fa54affd640 1 -- 172.21.15.84:0/3935822232 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa544089090 con 0x7fa540003f20 2023-10-16T09:49:59.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.659+0000 7fa54bfff640 1 --2- 172.21.15.84:0/3935822232 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fa53406aea0 0x7fa53406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:49:59.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.659+0000 7fa54bfff640 1 --2- 172.21.15.84:0/3935822232 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fa53406aea0 0x7fa53406d360 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa53c033000 tx=0x7fa53c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:49:59.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.663+0000 7fa54affd640 1 -- 172.21.15.84:0/3935822232 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa544054020 con 0x7fa540003f20 2023-10-16T09:49:59.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.893+0000 7fa55897c640 1 -- 172.21.15.84:0/3935822232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5540b51c0 con 0x7fa540003f20 2023-10-16T09:49:59.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.894+0000 7fa54affd640 1 -- 172.21.15.84:0/3935822232 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7fa544057640 con 0x7fa540003f20 2023-10-16T09:49:59.896 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:49:59.896 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2978888131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2978888131},{"type":"v1","addr":"172.21.15.157:6834","nonce":2978888131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3716581310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3716581310},{"type":"v1","addr":"172.21.15.84:6835","nonce":3716581310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2761479872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2761479872},{"type":"v1","addr":"172.21.15.84:6837","nonce":2761479872}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:49:02.350624+0000","modified":"2023-10-16T09:49:05.402785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/848754825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":848754825},{"type":"v1","addr":"172.21.15.157:6835","nonce":848754825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T09:49:59.897 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-10-16T09:49:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.896+0000 7fa55897c640 1 -- 172.21.15.84:0/3935822232 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fa53406aea0 msgr2=0x7fa53406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.897+0000 7fa55897c640 1 --2- 172.21.15.84:0/3935822232 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fa53406aea0 0x7fa53406d360 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa53c033000 tx=0x7fa53c009030 comp rx=0 tx=0).stop 2023-10-16T09:49:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.897+0000 7fa55897c640 1 -- 172.21.15.84:0/3935822232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa540003f20 msgr2=0x7fa55415a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:49:59.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.897+0000 7fa55897c640 1 --2- 172.21.15.84:0/3935822232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa540003f20 0x7fa55415a160 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa54400d7e0 tx=0x7fa54400dca0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.897+0000 7fa55897c640 1 -- 172.21.15.84:0/3935822232 shutdown_connections 2023-10-16T09:49:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.897+0000 7fa55897c640 1 --2- 172.21.15.84:0/3935822232 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fa53406aea0 0x7fa53406d360 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.897+0000 7fa55897c640 1 --2- 172.21.15.84:0/3935822232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5540cddb0 0x7fa5540ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.897+0000 7fa55897c640 1 --2- 172.21.15.84:0/3935822232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa55409e6f0 0x7fa55409dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.897+0000 7fa55897c640 1 --2- 172.21.15.84:0/3935822232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa540003f20 0x7fa55415a160 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:49:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.897+0000 7fa55897c640 1 -- 172.21.15.84:0/3935822232 >> 172.21.15.84:0/3935822232 conn(0x7fa5540b0d30 msgr2=0x7fa5540b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:49:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.897+0000 7fa55897c640 1 -- 172.21.15.84:0/3935822232 shutdown_connections 2023-10-16T09:49:59.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:49:59.897+0000 7fa55897c640 1 -- 172.21.15.84:0/3935822232 wait complete. 2023-10-16T09:49:59.918 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:50:00.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.122+0000 7f7295590640 1 Processor -- start 2023-10-16T09:50:00.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.122+0000 7f7295590640 1 -- start start 2023-10-16T09:50:00.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.124+0000 7f7295590640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7290070fd0 0x7f72900b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:00.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.124+0000 7f7295590640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f72900adf30 0x7f72900b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:00.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.124+0000 7f7295590640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7290071450 con 0x7f72900adb50 2023-10-16T09:50:00.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.124+0000 7f7295590640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72900cc600 con 0x7f72900adf30 2023-10-16T09:50:00.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.124+0000 7f7295590640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72900cc8e0 con 0x7f7290070fd0 2023-10-16T09:50:00.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.124+0000 7f728effd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f72900adb50 0x7f729006f550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36250/0 (socket says 172.21.15.84:36250) 2023-10-16T09:50:00.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.124+0000 7f728effd640 1 -- 172.21.15.84:0/2009450460 learned_addr learned my addr 172.21.15.84:0/2009450460 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:00.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.124+0000 7f728f7fe640 1 --2- 172.21.15.84:0/2009450460 >> v2:172.21.15.84:3300/0 conn(0x7f72900adf30 0x7f72900b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:00.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.124+0000 7f728dffb640 1 -- 172.21.15.84:0/2009450460 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3175776713 0 0) 0x7f7290071450 con 0x7f72900adb50 2023-10-16T09:50:00.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.125+0000 7f728e7fc640 1 --2- 172.21.15.84:0/2009450460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7290070fd0 0x7f72900b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:00.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.125+0000 7f728dffb640 1 -- 172.21.15.84:0/2009450460 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7278003520 con 0x7f72900adb50 2023-10-16T09:50:00.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.125+0000 7f728f7fe640 1 -- 172.21.15.84:0/2009450460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7290070fd0 msgr2=0x7f72900b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:00.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.125+0000 7f728f7fe640 1 --2- 172.21.15.84:0/2009450460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7290070fd0 0x7f72900b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.125+0000 7f728f7fe640 1 -- 172.21.15.84:0/2009450460 >> v1:172.21.15.84:6789/0 conn(0x7f72900adb50 legacy=0x7f729006f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:00.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.125+0000 7f728f7fe640 1 -- 172.21.15.84:0/2009450460 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72900ccbc0 con 0x7f72900adf30 2023-10-16T09:50:00.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.126+0000 7f728f7fe640 1 --2- 172.21.15.84:0/2009450460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72900adf30 0x7f72900b0950 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7288009d90 tx=0x7f728802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7092859804b921a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:00.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.126+0000 7f728dffb640 1 -- 172.21.15.84:0/2009450460 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728802cc60 con 0x7f72900adf30 2023-10-16T09:50:00.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.126+0000 7f728dffb640 1 -- 172.21.15.84:0/2009450460 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728802cdc0 con 0x7f72900adf30 2023-10-16T09:50:00.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.126+0000 7f728dffb640 1 -- 172.21.15.84:0/2009450460 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7288034660 con 0x7f72900adf30 2023-10-16T09:50:00.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.127+0000 7f7295590640 1 -- 172.21.15.84:0/2009450460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72900adf30 msgr2=0x7f72900b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:00.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.127+0000 7f7295590640 1 --2- 172.21.15.84:0/2009450460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72900adf30 0x7f72900b0950 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7288009d90 tx=0x7f728802beb0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.127+0000 7f7295590640 1 -- 172.21.15.84:0/2009450460 shutdown_connections 2023-10-16T09:50:00.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.127+0000 7f7295590640 1 --2- 172.21.15.84:0/2009450460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72900adf30 0x7f72900b0950 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.127+0000 7f7295590640 1 --2- 172.21.15.84:0/2009450460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7290070fd0 0x7f72900b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.127+0000 7f7295590640 1 -- 172.21.15.84:0/2009450460 >> 172.21.15.84:0/2009450460 conn(0x7f72900cb410 msgr2=0x7f72900cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:00.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.127+0000 7f7295590640 1 -- 172.21.15.84:0/2009450460 shutdown_connections 2023-10-16T09:50:00.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.127+0000 7f7295590640 1 -- 172.21.15.84:0/2009450460 wait complete. 2023-10-16T09:50:00.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.128+0000 7f7295590640 1 Processor -- start 2023-10-16T09:50:00.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.129+0000 7f7295590640 1 -- start start 2023-10-16T09:50:00.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.130+0000 7f7295590640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7290070fd0 0x7f72901faa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:00.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.130+0000 7f7295590640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72900adb50 0x7f72901faf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:00.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.130+0000 7f7295590640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72900adf30 0x7f72900bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:00.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.130+0000 7f7295590640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72900c0cc0 con 0x7f72900adf30 2023-10-16T09:50:00.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.130+0000 7f7295590640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72900c00b0 con 0x7f7290070fd0 2023-10-16T09:50:00.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.130+0000 7f7295590640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72900c0390 con 0x7f72900adb50 2023-10-16T09:50:00.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.130+0000 7f728f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72900adf30 0x7f72900bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:00.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.130+0000 7f728f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72900adf30 0x7f72900bf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38326/0 (socket says 172.21.15.84:38326) 2023-10-16T09:50:00.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.130+0000 7f728f7fe640 1 -- 172.21.15.84:0/4287326051 learned_addr learned my addr 172.21.15.84:0/4287326051 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:00.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.130+0000 7f728f7fe640 1 -- 172.21.15.84:0/4287326051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72900adb50 msgr2=0x7f72901faf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:00.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.130+0000 7f728effd640 1 --2- 172.21.15.84:0/4287326051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7290070fd0 0x7f72901faa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:00.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.130+0000 7f728e7fc640 1 --2- 172.21.15.84:0/4287326051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72900adb50 0x7f72901faf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:00.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.130+0000 7f728f7fe640 1 --2- 172.21.15.84:0/4287326051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72900adb50 0x7f72901faf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.130+0000 7f728f7fe640 1 -- 172.21.15.84:0/4287326051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7290070fd0 msgr2=0x7f72901faa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:00.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.130+0000 7f728f7fe640 1 --2- 172.21.15.84:0/4287326051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7290070fd0 0x7f72901faa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.130+0000 7f728f7fe640 1 -- 172.21.15.84:0/4287326051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7288009a70 con 0x7f72900adf30 2023-10-16T09:50:00.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.130+0000 7f728e7fc640 1 --2- 172.21.15.84:0/4287326051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72900adb50 0x7f72901faf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:00.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.131+0000 7f728f7fe640 1 --2- 172.21.15.84:0/4287326051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72900adf30 0x7f72900bf2c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7288009d90 tx=0x7f7288004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:00.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.131+0000 7f72777fe640 1 -- 172.21.15.84:0/4287326051 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7288040650 con 0x7f72900adf30 2023-10-16T09:50:00.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.131+0000 7f72777fe640 1 -- 172.21.15.84:0/4287326051 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72880407b0 con 0x7f72900adf30 2023-10-16T09:50:00.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.131+0000 7f7295590640 1 -- 172.21.15.84:0/4287326051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72900c0670 con 0x7f72900adf30 2023-10-16T09:50:00.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.131+0000 7f72777fe640 1 -- 172.21.15.84:0/4287326051 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7288040b60 con 0x7f72900adf30 2023-10-16T09:50:00.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.131+0000 7f7295590640 1 -- 172.21.15.84:0/4287326051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72900c1150 con 0x7f72900adf30 2023-10-16T09:50:00.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.132+0000 7f7295590640 1 -- 172.21.15.84:0/4287326051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72900c0e30 con 0x7f72900adf30 2023-10-16T09:50:00.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.133+0000 7f72777fe640 1 -- 172.21.15.84:0/4287326051 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7288040cc0 con 0x7f72900adf30 2023-10-16T09:50:00.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.134+0000 7f72777fe640 1 --2- 172.21.15.84:0/4287326051 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f727c06ae70 0x7f727c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:00.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.134+0000 7f72777fe640 1 -- 172.21.15.84:0/4287326051 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f72880ac4b0 con 0x7f72900adf30 2023-10-16T09:50:00.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.134+0000 7f728effd640 1 --2- 172.21.15.84:0/4287326051 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f727c06ae70 0x7f727c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:00.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.135+0000 7f728effd640 1 --2- 172.21.15.84:0/4287326051 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f727c06ae70 0x7f727c06d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f72800011e0 tx=0x7f7280009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:00.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.138+0000 7f72777fe640 1 -- 172.21.15.84:0/4287326051 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7288077440 con 0x7f72900adf30 2023-10-16T09:50:00.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.331+0000 7f7295590640 1 -- 172.21.15.84:0/4287326051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f729004e160 con 0x7f72900adf30 2023-10-16T09:50:00.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.334+0000 7f72777fe640 1 -- 172.21.15.84:0/4287326051 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16730 (secure 0 0 0) 0x7f7288009450 con 0x7f72900adf30 2023-10-16T09:50:00.336 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:50:00.336 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":23,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:49:49.552121+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:49:00.048649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:49:01.361365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.27","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:50:00.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.336+0000 7f7295590640 1 -- 172.21.15.84:0/4287326051 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f727c06ae70 msgr2=0x7f727c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:00.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.336+0000 7f7295590640 1 --2- 172.21.15.84:0/4287326051 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f727c06ae70 0x7f727c06d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f72800011e0 tx=0x7f7280009040 comp rx=0 tx=0).stop 2023-10-16T09:50:00.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.336+0000 7f7295590640 1 -- 172.21.15.84:0/4287326051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72900adf30 msgr2=0x7f72900bf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:00.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.336+0000 7f7295590640 1 --2- 172.21.15.84:0/4287326051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72900adf30 0x7f72900bf2c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7288009d90 tx=0x7f7288004a40 comp rx=0 tx=0).stop 2023-10-16T09:50:00.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.337+0000 7f7295590640 1 -- 172.21.15.84:0/4287326051 shutdown_connections 2023-10-16T09:50:00.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.337+0000 7f7295590640 1 --2- 172.21.15.84:0/4287326051 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f727c06ae70 0x7f727c06d330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.337+0000 7f7295590640 1 --2- 172.21.15.84:0/4287326051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72900adf30 0x7f72900bf2c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.337+0000 7f7295590640 1 --2- 172.21.15.84:0/4287326051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72900adb50 0x7f72901faf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.337+0000 7f7295590640 1 --2- 172.21.15.84:0/4287326051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7290070fd0 0x7f72901faa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.337+0000 7f7295590640 1 -- 172.21.15.84:0/4287326051 >> 172.21.15.84:0/4287326051 conn(0x7f72900cb410 msgr2=0x7f72900b05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:00.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.337+0000 7f7295590640 1 -- 172.21.15.84:0/4287326051 shutdown_connections 2023-10-16T09:50:00.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.337+0000 7f7295590640 1 -- 172.21.15.84:0/4287326051 wait complete. 2023-10-16T09:50:00.357 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T09:50:00.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.561+0000 7f47ca621640 1 Processor -- start 2023-10-16T09:50:00.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.561+0000 7f47ca621640 1 -- start start 2023-10-16T09:50:00.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.563+0000 7f47ca621640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f47c406c040 0x7f47c406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:00.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.563+0000 7f47ca621640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c415f4a0 con 0x7f47c40ad970 2023-10-16T09:50:00.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.563+0000 7f47ca621640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c406c960 con 0x7f47c40cda80 2023-10-16T09:50:00.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.563+0000 7f47ca621640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c415b9d0 con 0x7f47c406c040 2023-10-16T09:50:00.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.563+0000 7f47c8e1e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f47c40ad970 0x7f47c415e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36262/0 (socket says 172.21.15.84:36262) 2023-10-16T09:50:00.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.563+0000 7f47c8e1e640 1 -- 172.21.15.84:0/507075468 learned_addr learned my addr 172.21.15.84:0/507075468 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:00.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.564+0000 7f47bbfff640 1 -- 172.21.15.84:0/507075468 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105091902 0 0) 0x7f47c415f4a0 con 0x7f47c40ad970 2023-10-16T09:50:00.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.564+0000 7f47bbfff640 1 -- 172.21.15.84:0/507075468 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47b0003520 con 0x7f47c40ad970 2023-10-16T09:50:00.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.564+0000 7f47c961f640 1 --2- 172.21.15.84:0/507075468 >> v2:172.21.15.157:3300/0 conn(0x7f47c406c040 0x7f47c406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:00.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.564+0000 7f47bbfff640 1 -- 172.21.15.84:0/507075468 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2496067642 0 0) 0x7f47b0003520 con 0x7f47c40ad970 2023-10-16T09:50:00.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.564+0000 7f47bbfff640 1 -- 172.21.15.84:0/507075468 >> v2:172.21.15.157:3300/0 conn(0x7f47c406c040 msgr2=0x7f47c406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:00.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.564+0000 7f47bbfff640 1 --2- 172.21.15.84:0/507075468 >> v2:172.21.15.157:3300/0 conn(0x7f47c406c040 0x7f47c406c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.564+0000 7f47bbfff640 1 -- 172.21.15.84:0/507075468 >> v1:172.21.15.157:6789/0 conn(0x7f47c40cda80 legacy=0x7f47c415ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:00.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.564+0000 7f47bbfff640 1 -- 172.21.15.84:0/507075468 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c415bcb0 con 0x7f47c40ad970 2023-10-16T09:50:00.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.564+0000 7f47bbfff640 1 -- 172.21.15.84:0/507075468 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f47b4001670 con 0x7f47c40ad970 2023-10-16T09:50:00.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.564+0000 7f47bbfff640 1 -- 172.21.15.84:0/507075468 >> v1:172.21.15.84:6789/0 conn(0x7f47c40ad970 legacy=0x7f47c415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:00.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.564+0000 7f47bbfff640 1 --2- 172.21.15.84:0/507075468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b0003970 0x7f47b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:00.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.564+0000 7f47bbfff640 1 --2- 172.21.15.84:0/507075468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47b00047c0 0x7f47b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:00.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.564+0000 7f47bbfff640 1 --2- 172.21.15.84:0/507075468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0005180 0x7f47b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:00.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.564+0000 7f47bbfff640 1 -- 172.21.15.84:0/507075468 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c415b9d0 con 0x7f47b0005180 2023-10-16T09:50:00.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.564+0000 7f47bbfff640 1 -- 172.21.15.84:0/507075468 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b000c0d0 con 0x7f47b0003970 2023-10-16T09:50:00.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.564+0000 7f47bbfff640 1 -- 172.21.15.84:0/507075468 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b000c290 con 0x7f47b00047c0 2023-10-16T09:50:00.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.565+0000 7f47c9e20640 1 --2- 172.21.15.84:0/507075468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0005180 0x7f47b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:00.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.565+0000 7f47c961f640 1 --2- 172.21.15.84:0/507075468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b0003970 0x7f47b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:00.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.565+0000 7f47c9e20640 1 -- 172.21.15.84:0/507075468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47b00047c0 msgr2=0x7f47b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:00.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.565+0000 7f47c8e1e640 1 --2- 172.21.15.84:0/507075468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47b00047c0 0x7f47b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:00.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.565+0000 7f47c9e20640 1 --2- 172.21.15.84:0/507075468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47b00047c0 0x7f47b0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.565+0000 7f47c9e20640 1 -- 172.21.15.84:0/507075468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b0003970 msgr2=0x7f47b0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:00.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.565+0000 7f47c9e20640 1 --2- 172.21.15.84:0/507075468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b0003970 0x7f47b0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.565+0000 7f47c9e20640 1 -- 172.21.15.84:0/507075468 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b0003520 con 0x7f47b0005180 2023-10-16T09:50:00.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.565+0000 7f47c8e1e640 1 --2- 172.21.15.84:0/507075468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47b00047c0 0x7f47b0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:00.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.566+0000 7f47c9e20640 1 -- 172.21.15.84:0/507075468 reap_dead start 2023-10-16T09:50:00.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.566+0000 7f47c9e20640 1 --2- 172.21.15.84:0/507075468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0005180 0x7f47b000b980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f47bc001e50 tx=0x7f47b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33383e83163fd2b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:00.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.566+0000 7f47bbfff640 1 -- 172.21.15.84:0/507075468 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc003b70 con 0x7f47b0005180 2023-10-16T09:50:00.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.566+0000 7f47bbfff640 1 -- 172.21.15.84:0/507075468 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47bc003cf0 con 0x7f47b0005180 2023-10-16T09:50:00.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.566+0000 7f47bbfff640 1 -- 172.21.15.84:0/507075468 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc005310 con 0x7f47b0005180 2023-10-16T09:50:00.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.567+0000 7f47ca621640 1 -- 172.21.15.84:0/507075468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0005180 msgr2=0x7f47b000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:00.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.567+0000 7f47ca621640 1 --2- 172.21.15.84:0/507075468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0005180 0x7f47b000b980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f47bc001e50 tx=0x7f47b00057f0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.567+0000 7f47ca621640 1 -- 172.21.15.84:0/507075468 shutdown_connections 2023-10-16T09:50:00.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.567+0000 7f47ca621640 1 --2- 172.21.15.84:0/507075468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0005180 0x7f47b000b980 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.567+0000 7f47ca621640 1 -- 172.21.15.84:0/507075468 >> 172.21.15.84:0/507075468 conn(0x7f47c40ce6a0 msgr2=0x7f47c40ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:00.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.568+0000 7f47ca621640 1 -- 172.21.15.84:0/507075468 shutdown_connections 2023-10-16T09:50:00.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.568+0000 7f47ca621640 1 -- 172.21.15.84:0/507075468 wait complete. 2023-10-16T09:50:00.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.569+0000 7f47ca621640 1 Processor -- start 2023-10-16T09:50:00.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.569+0000 7f47ca621640 1 -- start start 2023-10-16T09:50:00.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.570+0000 7f47ca621640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0005180 0x7f47c40a8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:00.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.570+0000 7f47ca621640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47c4158160 0x7f47c414a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:00.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.570+0000 7f47ca621640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c4157290 0x7f47c41f0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:00.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.570+0000 7f47ca621640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c4158900 con 0x7f47b0005180 2023-10-16T09:50:00.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.570+0000 7f47ca621640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c41564d0 con 0x7f47c4157290 2023-10-16T09:50:00.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.570+0000 7f47ca621640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c41567b0 con 0x7f47c4158160 2023-10-16T09:50:00.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.571+0000 7f47c961f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0005180 0x7f47c40a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:00.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.571+0000 7f47c961f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0005180 0x7f47c40a8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38346/0 (socket says 172.21.15.84:38346) 2023-10-16T09:50:00.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.571+0000 7f47c961f640 1 -- 172.21.15.84:0/3382049165 learned_addr learned my addr 172.21.15.84:0/3382049165 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:00.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.571+0000 7f47c9e20640 1 --2- 172.21.15.84:0/3382049165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c4157290 0x7f47c41f0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:00.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.571+0000 7f47c8e1e640 1 --2- 172.21.15.84:0/3382049165 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47c4158160 0x7f47c414a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:00.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.571+0000 7f47c961f640 1 -- 172.21.15.84:0/3382049165 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47c4158160 msgr2=0x7f47c414a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:00.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.571+0000 7f47c961f640 1 --2- 172.21.15.84:0/3382049165 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47c4158160 0x7f47c414a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.571+0000 7f47c961f640 1 -- 172.21.15.84:0/3382049165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c4157290 msgr2=0x7f47c41f0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:00.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.571+0000 7f47c961f640 1 --2- 172.21.15.84:0/3382049165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c4157290 0x7f47c41f0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.571+0000 7f47c961f640 1 -- 172.21.15.84:0/3382049165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c4156a90 con 0x7f47b0005180 2023-10-16T09:50:00.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.571+0000 7f47c961f640 1 --2- 172.21.15.84:0/3382049165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0005180 0x7f47c40a8480 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f47c000f580 tx=0x7f47c000fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:00.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.572+0000 7f47b9ffb640 1 -- 172.21.15.84:0/3382049165 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0013810 con 0x7f47b0005180 2023-10-16T09:50:00.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.572+0000 7f47b9ffb640 1 -- 172.21.15.84:0/3382049165 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c0013970 con 0x7f47b0005180 2023-10-16T09:50:00.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.572+0000 7f47b9ffb640 1 -- 172.21.15.84:0/3382049165 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0013ce0 con 0x7f47b0005180 2023-10-16T09:50:00.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.572+0000 7f47ca621640 1 -- 172.21.15.84:0/3382049165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c4156d70 con 0x7f47b0005180 2023-10-16T09:50:00.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.572+0000 7f47ca621640 1 -- 172.21.15.84:0/3382049165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c414ae10 con 0x7f47b0005180 2023-10-16T09:50:00.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.573+0000 7f47ca621640 1 -- 172.21.15.84:0/3382049165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c406b330 con 0x7f47b0005180 2023-10-16T09:50:00.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.574+0000 7f47b9ffb640 1 -- 172.21.15.84:0/3382049165 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f47c001f460 con 0x7f47b0005180 2023-10-16T09:50:00.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.575+0000 7f47b9ffb640 1 --2- 172.21.15.84:0/3382049165 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f47a806aea0 0x7f47a806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:00.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.575+0000 7f47b9ffb640 1 -- 172.21.15.84:0/3382049165 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f47c0089a90 con 0x7f47b0005180 2023-10-16T09:50:00.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.575+0000 7f47c8e1e640 1 --2- 172.21.15.84:0/3382049165 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f47a806aea0 0x7f47a806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:00.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.575+0000 7f47c8e1e640 1 --2- 172.21.15.84:0/3382049165 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f47a806aea0 0x7f47a806d360 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f47c406af50 tx=0x7f47b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:00.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.582+0000 7f47b9ffb640 1 -- 172.21.15.84:0/3382049165 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f47c0054a30 con 0x7f47b0005180 2023-10-16T09:50:00.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.812+0000 7f47ca621640 1 -- 172.21.15.84:0/3382049165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f47c404e9b0 con 0x7f47b0005180 2023-10-16T09:50:00.815 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-16T09:50:00.813+0000 7fca0c7ff640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T09:50:00.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.817+0000 7f47b9ffb640 1 -- 172.21.15.84:0/3382049165 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f47c0058050 con 0x7f47b0005180 2023-10-16T09:50:00.819 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T09:50:00.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.820+0000 7f47ca621640 1 -- 172.21.15.84:0/3382049165 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f47a806aea0 msgr2=0x7f47a806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:00.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.820+0000 7f47ca621640 1 --2- 172.21.15.84:0/3382049165 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f47a806aea0 0x7f47a806d360 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f47c406af50 tx=0x7f47b4009040 comp rx=0 tx=0).stop 2023-10-16T09:50:00.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.820+0000 7f47ca621640 1 -- 172.21.15.84:0/3382049165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0005180 msgr2=0x7f47c40a8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:00.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.820+0000 7f47ca621640 1 --2- 172.21.15.84:0/3382049165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0005180 0x7f47c40a8480 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f47c000f580 tx=0x7f47c000fa40 comp rx=0 tx=0).stop 2023-10-16T09:50:00.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.820+0000 7f47ca621640 1 -- 172.21.15.84:0/3382049165 shutdown_connections 2023-10-16T09:50:00.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.820+0000 7f47ca621640 1 --2- 172.21.15.84:0/3382049165 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f47a806aea0 0x7f47a806d360 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.820+0000 7f47ca621640 1 --2- 172.21.15.84:0/3382049165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c4157290 0x7f47c41f0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.820+0000 7f47ca621640 1 --2- 172.21.15.84:0/3382049165 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47c4158160 0x7f47c414a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.820+0000 7f47ca621640 1 --2- 172.21.15.84:0/3382049165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0005180 0x7f47c40a8480 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:00.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.821+0000 7f47ca621640 1 -- 172.21.15.84:0/3382049165 >> 172.21.15.84:0/3382049165 conn(0x7f47c40ce6a0 msgr2=0x7f47c406f700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:00.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.821+0000 7f47ca621640 1 -- 172.21.15.84:0/3382049165 shutdown_connections 2023-10-16T09:50:00.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:00.821+0000 7f47ca621640 1 -- 172.21.15.84:0/3382049165 wait complete. 2023-10-16T09:50:00.841 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T09:50:00.910 INFO:tasks.ceph.mds.b.smithi157.stderr: -7424> 2023-10-16T09:49:06.041+0000 7f43c3384640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T09:50:00.937 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T09:50:01.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.129+0000 7f7f0d99e640 1 Processor -- start 2023-10-16T09:50:01.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.129+0000 7f7f0d99e640 1 -- start start 2023-10-16T09:50:01.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.131+0000 7f7f0d99e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f0815cd70 0x7f7f080d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:01.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.131+0000 7f7f0d99e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7f080b8480 0x7f7f080cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:01.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.131+0000 7f7f0d99e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f0815d150 con 0x7f7f080d0be0 2023-10-16T09:50:01.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.131+0000 7f7f0d99e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f080d1180 con 0x7f7f080b8480 2023-10-16T09:50:01.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.131+0000 7f7f0d99e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080d1460 con 0x7f7f0815cd70 2023-10-16T09:50:01.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.131+0000 7f7f0c99c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f0815cd70 0x7f7f080d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:01.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.131+0000 7f7f0c99c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f0815cd70 0x7f7f080d2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:50106/0 (socket says 172.21.15.84:50106) 2023-10-16T09:50:01.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.131+0000 7f7f0c99c640 1 -- 172.21.15.84:0/145416606 learned_addr learned my addr 172.21.15.84:0/145416606 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:01.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.131+0000 7f7f07fff640 1 --2- 172.21.15.84:0/145416606 >> v2:172.21.15.157:3300/0 conn(0x7f7f080b8480 0x7f7f080cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:01.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.132+0000 7f7f0c99c640 1 -- 172.21.15.84:0/145416606 >> v2:172.21.15.157:3300/0 conn(0x7f7f080b8480 msgr2=0x7f7f080cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:01.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.132+0000 7f7f0c99c640 1 --2- 172.21.15.84:0/145416606 >> v2:172.21.15.157:3300/0 conn(0x7f7f080b8480 0x7f7f080cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:01.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.132+0000 7f7f0c99c640 1 -- 172.21.15.84:0/145416606 >> v1:172.21.15.157:6789/0 conn(0x7f7f080d0be0 legacy=0x7f7f080b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:01.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.132+0000 7f7f0c99c640 1 -- 172.21.15.84:0/145416606 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f080d1740 con 0x7f7f0815cd70 2023-10-16T09:50:01.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.132+0000 7f7f07fff640 1 --2- 172.21.15.84:0/145416606 >> v2:172.21.15.157:3300/0 conn(0x7f7f080b8480 0x7f7f080cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:50:01.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.133+0000 7f7f0c99c640 1 --2- 172.21.15.84:0/145416606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f0815cd70 0x7f7f080d2220 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009870 tx=0x7f7efc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfb0944764eed8b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:01.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.133+0000 7f7f077fe640 1 -- 172.21.15.84:0/145416606 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc030840 con 0x7f7f0815cd70 2023-10-16T09:50:01.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.133+0000 7f7f077fe640 1 -- 172.21.15.84:0/145416606 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7efc0309a0 con 0x7f7f0815cd70 2023-10-16T09:50:01.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.133+0000 7f7f077fe640 1 -- 172.21.15.84:0/145416606 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc030d30 con 0x7f7f0815cd70 2023-10-16T09:50:01.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.134+0000 7f7f0d99e640 1 -- 172.21.15.84:0/145416606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f0815cd70 msgr2=0x7f7f080d2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:01.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.134+0000 7f7f0d99e640 1 --2- 172.21.15.84:0/145416606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f0815cd70 0x7f7f080d2220 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009870 tx=0x7f7efc02b980 comp rx=0 tx=0).stop 2023-10-16T09:50:01.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.134+0000 7f7f0d99e640 1 -- 172.21.15.84:0/145416606 shutdown_connections 2023-10-16T09:50:01.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.134+0000 7f7f0d99e640 1 --2- 172.21.15.84:0/145416606 >> v2:172.21.15.157:3300/0 conn(0x7f7f080b8480 0x7f7f080cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:01.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.134+0000 7f7f0d99e640 1 --2- 172.21.15.84:0/145416606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f0815cd70 0x7f7f080d2220 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:01.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.134+0000 7f7f0d99e640 1 -- 172.21.15.84:0/145416606 >> 172.21.15.84:0/145416606 conn(0x7f7f080cf570 msgr2=0x7f7f080cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:01.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.134+0000 7f7f0d99e640 1 -- 172.21.15.84:0/145416606 shutdown_connections 2023-10-16T09:50:01.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.134+0000 7f7f0d99e640 1 -- 172.21.15.84:0/145416606 wait complete. 2023-10-16T09:50:01.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.136+0000 7f7f0d99e640 1 Processor -- start 2023-10-16T09:50:01.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.136+0000 7f7f0d99e640 1 -- start start 2023-10-16T09:50:01.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.137+0000 7f7f0d99e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f080b8480 0x7f7f080c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:01.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.137+0000 7f7f0d99e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f080d0be0 0x7f7f08151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:01.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.137+0000 7f7f0d99e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f0815cd70 0x7f7f0815edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:01.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.137+0000 7f7f0d99e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f081f1ef0 con 0x7f7f080d0be0 2023-10-16T09:50:01.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.137+0000 7f7f0d99e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080c7d20 con 0x7f7f0815cd70 2023-10-16T09:50:01.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.137+0000 7f7f0d99e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080c8000 con 0x7f7f080b8480 2023-10-16T09:50:01.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.137+0000 7f7f07fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f080d0be0 0x7f7f08151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:01.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.137+0000 7f7f07fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f080d0be0 0x7f7f08151900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38358/0 (socket says 172.21.15.84:38358) 2023-10-16T09:50:01.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.137+0000 7f7f07fff640 1 -- 172.21.15.84:0/2134024405 learned_addr learned my addr 172.21.15.84:0/2134024405 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:01.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.137+0000 7f7f07fff640 1 -- 172.21.15.84:0/2134024405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f080b8480 msgr2=0x7f7f080c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:01.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.138+0000 7f7f0c99c640 1 --2- 172.21.15.84:0/2134024405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f080b8480 0x7f7f080c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:01.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.138+0000 7f7f0d19d640 1 --2- 172.21.15.84:0/2134024405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f0815cd70 0x7f7f0815edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:01.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.138+0000 7f7f07fff640 1 --2- 172.21.15.84:0/2134024405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f080b8480 0x7f7f080c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:01.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.138+0000 7f7f07fff640 1 -- 172.21.15.84:0/2134024405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f0815cd70 msgr2=0x7f7f0815edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:01.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.138+0000 7f7f07fff640 1 --2- 172.21.15.84:0/2134024405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f0815cd70 0x7f7f0815edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:01.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.138+0000 7f7f07fff640 1 -- 172.21.15.84:0/2134024405 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef4009400 con 0x7f7f080d0be0 2023-10-16T09:50:01.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.138+0000 7f7f0d19d640 1 --2- 172.21.15.84:0/2134024405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f0815cd70 0x7f7f0815edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:01.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.138+0000 7f7f07fff640 1 --2- 172.21.15.84:0/2134024405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f080d0be0 0x7f7f08151900 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400e960 tx=0x7f7ef400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:01.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.138+0000 7f7f057fa640 1 -- 172.21.15.84:0/2134024405 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef400ca00 con 0x7f7f080d0be0 2023-10-16T09:50:01.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.138+0000 7f7f057fa640 1 -- 172.21.15.84:0/2134024405 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef400cb60 con 0x7f7f080d0be0 2023-10-16T09:50:01.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.138+0000 7f7f0d99e640 1 -- 172.21.15.84:0/2134024405 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7efc0094d0 con 0x7f7f080d0be0 2023-10-16T09:50:01.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.138+0000 7f7f057fa640 1 -- 172.21.15.84:0/2134024405 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4013470 con 0x7f7f080d0be0 2023-10-16T09:50:01.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.138+0000 7f7f0d99e640 1 -- 172.21.15.84:0/2134024405 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f080c85c0 con 0x7f7f080d0be0 2023-10-16T09:50:01.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.139+0000 7f7f0d99e640 1 -- 172.21.15.84:0/2134024405 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0804e9b0 con 0x7f7f080d0be0 2023-10-16T09:50:01.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.140+0000 7f7f057fa640 1 -- 172.21.15.84:0/2134024405 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7ef40135d0 con 0x7f7f080d0be0 2023-10-16T09:50:01.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.141+0000 7f7f057fa640 1 --2- 172.21.15.84:0/2134024405 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7eec06ae70 0x7f7eec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:01.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.141+0000 7f7f057fa640 1 -- 172.21.15.84:0/2134024405 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f7ef4089a50 con 0x7f7f080d0be0 2023-10-16T09:50:01.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.141+0000 7f7f0c99c640 1 --2- 172.21.15.84:0/2134024405 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7eec06ae70 0x7f7eec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:01.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.142+0000 7f7f0c99c640 1 --2- 172.21.15.84:0/2134024405 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7eec06ae70 0x7f7eec06d330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7efc036000 tx=0x7f7efc03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:01.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.145+0000 7f7f057fa640 1 -- 172.21.15.84:0/2134024405 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7ef4054990 con 0x7f7f080d0be0 2023-10-16T09:50:01.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.374+0000 7f7f0d99e640 1 -- 172.21.15.84:0/2134024405 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7f080b6870 con 0x7f7f080d0be0 2023-10-16T09:50:01.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.826+0000 7f7f057fa640 1 -- 172.21.15.84:0/2134024405 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7ef4057fb0 con 0x7f7f080d0be0 2023-10-16T09:50:01.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.830+0000 7f7f0d99e640 1 -- 172.21.15.84:0/2134024405 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7eec06ae70 msgr2=0x7f7eec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:01.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.830+0000 7f7f0d99e640 1 --2- 172.21.15.84:0/2134024405 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7eec06ae70 0x7f7eec06d330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7efc036000 tx=0x7f7efc03b040 comp rx=0 tx=0).stop 2023-10-16T09:50:01.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.830+0000 7f7f0d99e640 1 -- 172.21.15.84:0/2134024405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f080d0be0 msgr2=0x7f7f08151900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:01.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.830+0000 7f7f0d99e640 1 --2- 172.21.15.84:0/2134024405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f080d0be0 0x7f7f08151900 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400e960 tx=0x7f7ef400ee20 comp rx=0 tx=0).stop 2023-10-16T09:50:01.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.830+0000 7f7f0d99e640 1 -- 172.21.15.84:0/2134024405 shutdown_connections 2023-10-16T09:50:01.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.830+0000 7f7f0d99e640 1 --2- 172.21.15.84:0/2134024405 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f7eec06ae70 0x7f7eec06d330 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:01.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.830+0000 7f7f0d99e640 1 --2- 172.21.15.84:0/2134024405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f0815cd70 0x7f7f0815edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:01.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.830+0000 7f7f0d99e640 1 --2- 172.21.15.84:0/2134024405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f080d0be0 0x7f7f08151900 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:01.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.830+0000 7f7f0d99e640 1 --2- 172.21.15.84:0/2134024405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f080b8480 0x7f7f080c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:01.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.830+0000 7f7f0d99e640 1 -- 172.21.15.84:0/2134024405 >> 172.21.15.84:0/2134024405 conn(0x7f7f080cf570 msgr2=0x7f7f080b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:01.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.830+0000 7f7f0d99e640 1 -- 172.21.15.84:0/2134024405 shutdown_connections 2023-10-16T09:50:01.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:01.830+0000 7f7f0d99e640 1 -- 172.21.15.84:0/2134024405 wait complete. 2023-10-16T09:50:01.853 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T09:50:02.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.067+0000 7f106f59e640 1 Processor -- start 2023-10-16T09:50:02.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.067+0000 7f106f59e640 1 -- start start 2023-10-16T09:50:02.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.068+0000 7f106f59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f107015b320 0x7f10700cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:02.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.068+0000 7f106f59e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107015b9d0 con 0x7f107006c040 2023-10-16T09:50:02.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.068+0000 7f106f59e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107015bcb0 con 0x7f10700ad970 2023-10-16T09:50:02.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.068+0000 7f106f59e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f107015b7d0 con 0x7f107015b320 2023-10-16T09:50:02.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.068+0000 7f106e59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f107006c040 0x7f107006c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36274/0 (socket says 172.21.15.84:36274) 2023-10-16T09:50:02.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.068+0000 7f106e59c640 1 -- 172.21.15.84:0/3775549889 learned_addr learned my addr 172.21.15.84:0/3775549889 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:02.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.068+0000 7f106d59a640 1 -- 172.21.15.84:0/3775549889 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 730309723 0 0) 0x7f107015b9d0 con 0x7f107006c040 2023-10-16T09:50:02.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.068+0000 7f106d59a640 1 -- 172.21.15.84:0/3775549889 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1058003520 con 0x7f107006c040 2023-10-16T09:50:02.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.068+0000 7f106ed9d640 1 --2- 172.21.15.84:0/3775549889 >> v2:172.21.15.157:3300/0 conn(0x7f107015b320 0x7f10700cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:02.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.069+0000 7f106d59a640 1 -- 172.21.15.84:0/3775549889 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2849607087 0 0) 0x7f1058003520 con 0x7f107006c040 2023-10-16T09:50:02.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.069+0000 7f106d59a640 1 -- 172.21.15.84:0/3775549889 >> v2:172.21.15.157:3300/0 conn(0x7f107015b320 msgr2=0x7f10700cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:02.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.069+0000 7f106d59a640 1 --2- 172.21.15.84:0/3775549889 >> v2:172.21.15.157:3300/0 conn(0x7f107015b320 0x7f10700cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:02.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.069+0000 7f106d59a640 1 -- 172.21.15.84:0/3775549889 >> v1:172.21.15.157:6789/0 conn(0x7f10700ad970 legacy=0x7f10700cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:02.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.069+0000 7f106d59a640 1 -- 172.21.15.84:0/3775549889 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107015bf90 con 0x7f107006c040 2023-10-16T09:50:02.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.069+0000 7f106d59a640 1 -- 172.21.15.84:0/3775549889 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f1060001670 con 0x7f107006c040 2023-10-16T09:50:02.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.069+0000 7f106d59a640 1 -- 172.21.15.84:0/3775549889 >> v1:172.21.15.84:6789/0 conn(0x7f107006c040 legacy=0x7f107006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:02.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.069+0000 7f106d59a640 1 --2- 172.21.15.84:0/3775549889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10580039f0 0x7f1058003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:02.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.069+0000 7f106d59a640 1 --2- 172.21.15.84:0/3775549889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10580047c0 0x7f1058004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:02.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.069+0000 7f106d59a640 1 --2- 172.21.15.84:0/3775549889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10580051f0 0x7f105800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:02.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.069+0000 7f106d59a640 1 -- 172.21.15.84:0/3775549889 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107015b7d0 con 0x7f10580039f0 2023-10-16T09:50:02.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.069+0000 7f106d59a640 1 -- 172.21.15.84:0/3775549889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105800c150 con 0x7f10580047c0 2023-10-16T09:50:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.069+0000 7f106d59a640 1 -- 172.21.15.84:0/3775549889 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105800c310 con 0x7f10580051f0 2023-10-16T09:50:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.069+0000 7f106e59c640 1 --2- 172.21.15.84:0/3775549889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10580039f0 0x7f1058003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.070+0000 7f106e59c640 1 -- 172.21.15.84:0/3775549889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10580051f0 msgr2=0x7f105800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.070+0000 7f106e59c640 1 --2- 172.21.15.84:0/3775549889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10580051f0 0x7f105800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.070+0000 7f106e59c640 1 -- 172.21.15.84:0/3775549889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10580047c0 msgr2=0x7f1058004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:02.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.070+0000 7f106dd9b640 1 --2- 172.21.15.84:0/3775549889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10580047c0 0x7f1058004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.070+0000 7f106e59c640 1 --2- 172.21.15.84:0/3775549889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10580047c0 0x7f1058004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.070+0000 7f106e59c640 1 -- 172.21.15.84:0/3775549889 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1058003520 con 0x7f10580039f0 2023-10-16T09:50:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.070+0000 7f106ed9d640 1 -- 172.21.15.84:0/3775549889 reap_dead start 2023-10-16T09:50:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.070+0000 7f106dd9b640 1 --2- 172.21.15.84:0/3775549889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10580047c0 0x7f1058004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:02.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.071+0000 7f106e59c640 1 --2- 172.21.15.84:0/3775549889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10580039f0 0x7f1058003df0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f106000c190 tx=0x7f10580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619c47522fad4012 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:02.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.071+0000 7f106d59a640 1 -- 172.21.15.84:0/3775549889 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060005ad0 con 0x7f10580039f0 2023-10-16T09:50:02.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.071+0000 7f106d59a640 1 -- 172.21.15.84:0/3775549889 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1060005c30 con 0x7f10580039f0 2023-10-16T09:50:02.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.071+0000 7f106d59a640 1 -- 172.21.15.84:0/3775549889 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060007070 con 0x7f10580039f0 2023-10-16T09:50:02.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.071+0000 7f106f59e640 1 -- 172.21.15.84:0/3775549889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10580039f0 msgr2=0x7f1058003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:02.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.071+0000 7f106f59e640 1 --2- 172.21.15.84:0/3775549889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10580039f0 0x7f1058003df0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f106000c190 tx=0x7f10580057f0 comp rx=0 tx=0).stop 2023-10-16T09:50:02.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.072+0000 7f106f59e640 1 -- 172.21.15.84:0/3775549889 shutdown_connections 2023-10-16T09:50:02.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.072+0000 7f106f59e640 1 --2- 172.21.15.84:0/3775549889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10580039f0 0x7f1058003df0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:02.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.072+0000 7f106f59e640 1 -- 172.21.15.84:0/3775549889 >> 172.21.15.84:0/3775549889 conn(0x7f10700ce6a0 msgr2=0x7f10700ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:02.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.072+0000 7f106f59e640 1 -- 172.21.15.84:0/3775549889 shutdown_connections 2023-10-16T09:50:02.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.072+0000 7f106f59e640 1 -- 172.21.15.84:0/3775549889 wait complete. 2023-10-16T09:50:02.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.073+0000 7f106f59e640 1 Processor -- start 2023-10-16T09:50:02.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.073+0000 7f106f59e640 1 -- start start 2023-10-16T09:50:02.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.074+0000 7f106f59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10580039f0 0x7f10700ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:02.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.074+0000 7f106f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f107014fc30 0x7f10700ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:02.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.074+0000 7f106f59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10700acdd0 0x7f10701f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:02.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.074+0000 7f106f59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10700ad250 con 0x7f107014fc30 2023-10-16T09:50:02.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.074+0000 7f106f59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10701f6bd0 con 0x7f10580039f0 2023-10-16T09:50:02.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.074+0000 7f106f59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1070150090 con 0x7f10700acdd0 2023-10-16T09:50:02.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.075+0000 7f106dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f107014fc30 0x7f10700ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:02.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.075+0000 7f106dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f107014fc30 0x7f10700ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38374/0 (socket says 172.21.15.84:38374) 2023-10-16T09:50:02.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.075+0000 7f106dd9b640 1 -- 172.21.15.84:0/588417115 learned_addr learned my addr 172.21.15.84:0/588417115 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:02.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.075+0000 7f106dd9b640 1 -- 172.21.15.84:0/588417115 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10700acdd0 msgr2=0x7f10701f5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:02.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.075+0000 7f106dd9b640 1 --2- 172.21.15.84:0/588417115 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10700acdd0 0x7f10701f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:02.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.075+0000 7f106dd9b640 1 -- 172.21.15.84:0/588417115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10580039f0 msgr2=0x7f10700ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:02.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.075+0000 7f106dd9b640 1 --2- 172.21.15.84:0/588417115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10580039f0 0x7f10700ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:02.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.075+0000 7f106dd9b640 1 -- 172.21.15.84:0/588417115 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10700c9b70 con 0x7f107014fc30 2023-10-16T09:50:02.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.075+0000 7f106dd9b640 1 --2- 172.21.15.84:0/588417115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f107014fc30 0x7f10700ac890 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f106400d400 tx=0x7f106400d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:02.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.076+0000 7f105effd640 1 -- 172.21.15.84:0/588417115 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064007020 con 0x7f107014fc30 2023-10-16T09:50:02.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.076+0000 7f105effd640 1 -- 172.21.15.84:0/588417115 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10640071a0 con 0x7f107014fc30 2023-10-16T09:50:02.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.076+0000 7f105effd640 1 -- 172.21.15.84:0/588417115 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064007550 con 0x7f107014fc30 2023-10-16T09:50:02.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.076+0000 7f106f59e640 1 -- 172.21.15.84:0/588417115 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10700c9e50 con 0x7f107014fc30 2023-10-16T09:50:02.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.076+0000 7f106f59e640 1 -- 172.21.15.84:0/588417115 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10700ca410 con 0x7f107014fc30 2023-10-16T09:50:02.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.077+0000 7f106f59e640 1 -- 172.21.15.84:0/588417115 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f107006b330 con 0x7f107014fc30 2023-10-16T09:50:02.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.078+0000 7f105effd640 1 -- 172.21.15.84:0/588417115 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f10640067d0 con 0x7f107014fc30 2023-10-16T09:50:02.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.079+0000 7f105effd640 1 --2- 172.21.15.84:0/588417115 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f105006aea0 0x7f105006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:02.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.079+0000 7f105effd640 1 -- 172.21.15.84:0/588417115 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f10640894f0 con 0x7f107014fc30 2023-10-16T09:50:02.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.079+0000 7f106e59c640 1 --2- 172.21.15.84:0/588417115 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f105006aea0 0x7f105006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:02.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.080+0000 7f106e59c640 1 --2- 172.21.15.84:0/588417115 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f105006aea0 0x7f105006d360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1060034000 tx=0x7f1060007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:02.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.083+0000 7f105effd640 1 -- 172.21.15.84:0/588417115 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f10640544c0 con 0x7f107014fc30 2023-10-16T09:50:02.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.277+0000 7f106f59e640 1 -- 172.21.15.84:0/588417115 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f10700ca630 con 0x7f107014fc30 2023-10-16T09:50:02.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.831+0000 7f105effd640 1 -- 172.21.15.84:0/588417115 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1064057ae0 con 0x7f107014fc30 2023-10-16T09:50:02.832 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-16T09:50:02.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.835+0000 7f106f59e640 1 -- 172.21.15.84:0/588417115 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f105006aea0 msgr2=0x7f105006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:02.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.835+0000 7f106f59e640 1 --2- 172.21.15.84:0/588417115 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f105006aea0 0x7f105006d360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1060034000 tx=0x7f1060007000 comp rx=0 tx=0).stop 2023-10-16T09:50:02.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.835+0000 7f106f59e640 1 -- 172.21.15.84:0/588417115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f107014fc30 msgr2=0x7f10700ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:02.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.835+0000 7f106f59e640 1 --2- 172.21.15.84:0/588417115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f107014fc30 0x7f10700ac890 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f106400d400 tx=0x7f106400d8c0 comp rx=0 tx=0).stop 2023-10-16T09:50:02.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.835+0000 7f106f59e640 1 -- 172.21.15.84:0/588417115 shutdown_connections 2023-10-16T09:50:02.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.835+0000 7f106f59e640 1 --2- 172.21.15.84:0/588417115 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f105006aea0 0x7f105006d360 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:02.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.835+0000 7f106f59e640 1 --2- 172.21.15.84:0/588417115 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10700acdd0 0x7f10701f5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:02.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.835+0000 7f106f59e640 1 --2- 172.21.15.84:0/588417115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f107014fc30 0x7f10700ac890 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:02.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.835+0000 7f106f59e640 1 --2- 172.21.15.84:0/588417115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10580039f0 0x7f10700ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:02.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.836+0000 7f106f59e640 1 -- 172.21.15.84:0/588417115 >> 172.21.15.84:0/588417115 conn(0x7f10700ce6a0 msgr2=0x7f10700c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:02.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.836+0000 7f106f59e640 1 -- 172.21.15.84:0/588417115 shutdown_connections 2023-10-16T09:50:02.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:02.836+0000 7f106f59e640 1 -- 172.21.15.84:0/588417115 wait complete. 2023-10-16T09:50:02.860 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T09:50:03.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.078+0000 7f8b305ce640 1 Processor -- start 2023-10-16T09:50:03.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.078+0000 7f8b305ce640 1 -- start start 2023-10-16T09:50:03.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.080+0000 7f8b305ce640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b280c1d30 0x7f8b28165aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:03.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.080+0000 7f8b305ce640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b280c8fa0 0x7f8b280cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:03.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.080+0000 7f8b305ce640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b28165610 con 0x7f8b280b0360 2023-10-16T09:50:03.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.080+0000 7f8b305ce640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b280ce370 con 0x7f8b280c8fa0 2023-10-16T09:50:03.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.080+0000 7f8b305ce640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b280ce650 con 0x7f8b280c1d30 2023-10-16T09:50:03.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.081+0000 7f8b2e36a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b280c1d30 0x7f8b28165aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:03.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.081+0000 7f8b2e36a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b280c1d30 0x7f8b28165aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:50134/0 (socket says 172.21.15.84:50134) 2023-10-16T09:50:03.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.081+0000 7f8b2e36a640 1 -- 172.21.15.84:0/416664407 learned_addr learned my addr 172.21.15.84:0/416664407 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:03.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.081+0000 7f8b2db69640 1 --2- 172.21.15.84:0/416664407 >> v2:172.21.15.157:3300/0 conn(0x7f8b280c8fa0 0x7f8b280cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:03.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.081+0000 7f8b2e36a640 1 -- 172.21.15.84:0/416664407 >> v2:172.21.15.157:3300/0 conn(0x7f8b280c8fa0 msgr2=0x7f8b280cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:03.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.081+0000 7f8b2e36a640 1 --2- 172.21.15.84:0/416664407 >> v2:172.21.15.157:3300/0 conn(0x7f8b280c8fa0 0x7f8b280cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:03.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.081+0000 7f8b2e36a640 1 -- 172.21.15.84:0/416664407 >> v1:172.21.15.157:6789/0 conn(0x7f8b280b0360 legacy=0x7f8b2806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:03.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.081+0000 7f8b2e36a640 1 -- 172.21.15.84:0/416664407 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b28164fc0 con 0x7f8b280c1d30 2023-10-16T09:50:03.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.083+0000 7f8b2e36a640 1 --2- 172.21.15.84:0/416664407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b280c1d30 0x7f8b28165aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009870 tx=0x7f8b2402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=548819871468d354 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:03.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.083+0000 7f8b2d368640 1 -- 172.21.15.84:0/416664407 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24030840 con 0x7f8b280c1d30 2023-10-16T09:50:03.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.084+0000 7f8b2d368640 1 -- 172.21.15.84:0/416664407 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b240309a0 con 0x7f8b280c1d30 2023-10-16T09:50:03.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.084+0000 7f8b2d368640 1 -- 172.21.15.84:0/416664407 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24030d30 con 0x7f8b280c1d30 2023-10-16T09:50:03.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.084+0000 7f8b305ce640 1 -- 172.21.15.84:0/416664407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b280c1d30 msgr2=0x7f8b28165aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:03.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.084+0000 7f8b305ce640 1 --2- 172.21.15.84:0/416664407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b280c1d30 0x7f8b28165aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009870 tx=0x7f8b2402b980 comp rx=0 tx=0).stop 2023-10-16T09:50:03.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.084+0000 7f8b305ce640 1 -- 172.21.15.84:0/416664407 shutdown_connections 2023-10-16T09:50:03.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.084+0000 7f8b305ce640 1 --2- 172.21.15.84:0/416664407 >> v2:172.21.15.157:3300/0 conn(0x7f8b280c8fa0 0x7f8b280cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:03.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.084+0000 7f8b305ce640 1 --2- 172.21.15.84:0/416664407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b280c1d30 0x7f8b28165aa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:03.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.084+0000 7f8b305ce640 1 -- 172.21.15.84:0/416664407 >> 172.21.15.84:0/416664407 conn(0x7f8b280cb360 msgr2=0x7f8b280cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:03.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.084+0000 7f8b305ce640 1 -- 172.21.15.84:0/416664407 shutdown_connections 2023-10-16T09:50:03.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.084+0000 7f8b305ce640 1 -- 172.21.15.84:0/416664407 wait complete. 2023-10-16T09:50:03.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.086+0000 7f8b305ce640 1 Processor -- start 2023-10-16T09:50:03.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.086+0000 7f8b305ce640 1 -- start start 2023-10-16T09:50:03.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.087+0000 7f8b305ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b280b0360 0x7f8b2815f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:03.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.087+0000 7f8b305ce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b280c1d30 0x7f8b280c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:03.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.087+0000 7f8b305ce640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b280c8fa0 0x7f8b280cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:03.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.087+0000 7f8b305ce640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b28164330 con 0x7f8b280b0360 2023-10-16T09:50:03.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.087+0000 7f8b305ce640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b28163290 con 0x7f8b280c1d30 2023-10-16T09:50:03.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.087+0000 7f8b305ce640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b28163570 con 0x7f8b280c8fa0 2023-10-16T09:50:03.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.087+0000 7f8b2e36a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b280b0360 0x7f8b2815f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:03.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.087+0000 7f8b2e36a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b280b0360 0x7f8b2815f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38386/0 (socket says 172.21.15.84:38386) 2023-10-16T09:50:03.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.087+0000 7f8b2e36a640 1 -- 172.21.15.84:0/2071904098 learned_addr learned my addr 172.21.15.84:0/2071904098 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:03.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.088+0000 7f8b2eb6b640 1 --2- 172.21.15.84:0/2071904098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b280c8fa0 0x7f8b280cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:03.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.088+0000 7f8b2db69640 1 --2- 172.21.15.84:0/2071904098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b280c1d30 0x7f8b280c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:03.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.088+0000 7f8b2e36a640 1 -- 172.21.15.84:0/2071904098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b280c8fa0 msgr2=0x7f8b280cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:03.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.088+0000 7f8b2e36a640 1 --2- 172.21.15.84:0/2071904098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b280c8fa0 0x7f8b280cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:03.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.088+0000 7f8b2e36a640 1 -- 172.21.15.84:0/2071904098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b280c1d30 msgr2=0x7f8b280c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:03.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.088+0000 7f8b2e36a640 1 --2- 172.21.15.84:0/2071904098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b280c1d30 0x7f8b280c4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:03.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.088+0000 7f8b2e36a640 1 -- 172.21.15.84:0/2071904098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b240094d0 con 0x7f8b280b0360 2023-10-16T09:50:03.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.088+0000 7f8b2eb6b640 1 --2- 172.21.15.84:0/2071904098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b280c8fa0 0x7f8b280cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:03.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.088+0000 7f8b2db69640 1 --2- 172.21.15.84:0/2071904098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b280c1d30 0x7f8b280c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:03.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.088+0000 7f8b2e36a640 1 --2- 172.21.15.84:0/2071904098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b280b0360 0x7f8b2815f080 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8b24000c00 tx=0x7f8b24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:03.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.089+0000 7f8b1affd640 1 -- 172.21.15.84:0/2071904098 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24039c90 con 0x7f8b280b0360 2023-10-16T09:50:03.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.089+0000 7f8b1affd640 1 -- 172.21.15.84:0/2071904098 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b24039df0 con 0x7f8b280b0360 2023-10-16T09:50:03.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.089+0000 7f8b305ce640 1 -- 172.21.15.84:0/2071904098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b28163850 con 0x7f8b280b0360 2023-10-16T09:50:03.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.089+0000 7f8b1affd640 1 -- 172.21.15.84:0/2071904098 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24043640 con 0x7f8b280b0360 2023-10-16T09:50:03.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.089+0000 7f8b305ce640 1 -- 172.21.15.84:0/2071904098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b28163e10 con 0x7f8b280b0360 2023-10-16T09:50:03.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.090+0000 7f8b305ce640 1 -- 172.21.15.84:0/2071904098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b28165aa0 con 0x7f8b280b0360 2023-10-16T09:50:03.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.091+0000 7f8b1affd640 1 -- 172.21.15.84:0/2071904098 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8b240437a0 con 0x7f8b280b0360 2023-10-16T09:50:03.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.092+0000 7f8b1affd640 1 --2- 172.21.15.84:0/2071904098 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f8b0c06aea0 0x7f8b0c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:03.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.092+0000 7f8b1affd640 1 -- 172.21.15.84:0/2071904098 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f8b2403e070 con 0x7f8b280b0360 2023-10-16T09:50:03.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.092+0000 7f8b2db69640 1 --2- 172.21.15.84:0/2071904098 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f8b0c06aea0 0x7f8b0c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:03.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.093+0000 7f8b2db69640 1 --2- 172.21.15.84:0/2071904098 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f8b0c06aea0 0x7f8b0c06d360 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8b28164580 tx=0x7f8b1c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:03.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.097+0000 7f8b1affd640 1 -- 172.21.15.84:0/2071904098 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8b240776e0 con 0x7f8b280b0360 2023-10-16T09:50:03.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.289+0000 7f8b305ce640 1 -- 172.21.15.84:0/2071904098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b2804e160 con 0x7f8b280b0360 2023-10-16T09:50:03.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.843+0000 7f8b1affd640 1 -- 172.21.15.84:0/2071904098 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8b2407ad00 con 0x7f8b280b0360 2023-10-16T09:50:03.844 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-16T09:50:03.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.846+0000 7f8b305ce640 1 -- 172.21.15.84:0/2071904098 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f8b0c06aea0 msgr2=0x7f8b0c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:03.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.846+0000 7f8b305ce640 1 --2- 172.21.15.84:0/2071904098 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f8b0c06aea0 0x7f8b0c06d360 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8b28164580 tx=0x7f8b1c00b040 comp rx=0 tx=0).stop 2023-10-16T09:50:03.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.847+0000 7f8b305ce640 1 -- 172.21.15.84:0/2071904098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b280b0360 msgr2=0x7f8b2815f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:03.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.847+0000 7f8b305ce640 1 --2- 172.21.15.84:0/2071904098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b280b0360 0x7f8b2815f080 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8b24000c00 tx=0x7f8b24004730 comp rx=0 tx=0).stop 2023-10-16T09:50:03.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.847+0000 7f8b305ce640 1 -- 172.21.15.84:0/2071904098 shutdown_connections 2023-10-16T09:50:03.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.847+0000 7f8b305ce640 1 --2- 172.21.15.84:0/2071904098 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f8b0c06aea0 0x7f8b0c06d360 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:03.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.847+0000 7f8b305ce640 1 --2- 172.21.15.84:0/2071904098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b280c8fa0 0x7f8b280cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:03.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.847+0000 7f8b305ce640 1 --2- 172.21.15.84:0/2071904098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b280c1d30 0x7f8b280c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:03.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.847+0000 7f8b305ce640 1 --2- 172.21.15.84:0/2071904098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b280b0360 0x7f8b2815f080 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:03.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.847+0000 7f8b305ce640 1 -- 172.21.15.84:0/2071904098 >> 172.21.15.84:0/2071904098 conn(0x7f8b280cb360 msgr2=0x7f8b280aef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:03.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.847+0000 7f8b305ce640 1 -- 172.21.15.84:0/2071904098 shutdown_connections 2023-10-16T09:50:03.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:03.847+0000 7f8b305ce640 1 -- 172.21.15.84:0/2071904098 wait complete. 2023-10-16T09:50:03.873 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T09:50:03.873 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T09:50:03.873 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T09:50:03.874 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:50:03.874 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T09:50:03.874 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T09:50:03.875 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T09:50:03.875 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:50:03.875 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T09:50:03.875 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T09:50:03.876 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T09:50:03.876 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:50:03.876 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T09:50:03.876 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T09:50:03.877 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T09:50:03.877 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:50:03.891 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T09:50:03.892 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T09:50:03.893 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T09:50:03.893 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T09:50:03.897 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T09:50:03.897 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T09:50:03.902 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T09:50:03.902 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T09:50:03.940 INFO:tasks.ceph.mds.a:Started 2023-10-16T09:50:03.941 DEBUG:teuthology.parallel:result is None 2023-10-16T09:50:03.947 INFO:tasks.ceph.mds.c:Started 2023-10-16T09:50:03.947 DEBUG:teuthology.parallel:result is None 2023-10-16T09:50:03.949 INFO:tasks.ceph.mds.b:Started 2023-10-16T09:50:03.950 DEBUG:teuthology.parallel:result is None 2023-10-16T09:50:03.957 INFO:tasks.ceph.mds.d:Started 2023-10-16T09:50:03.957 DEBUG:teuthology.parallel:result is None 2023-10-16T09:50:03.957 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T09:50:04.102 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T09:50:04.104 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T09:50:04.119 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T09:50:04.132 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-10-16T09:50:04.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.149+0000 7f9f017a5640 1 Processor -- start 2023-10-16T09:50:04.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.150+0000 7f9f017a5640 1 -- start start 2023-10-16T09:50:04.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.151+0000 7f9f017a5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9efc15ad70 0x7f9efc069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:04.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.151+0000 7f9f017a5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9efc0caac0 0x7f9efc06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:04.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.151+0000 7f9f017a5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9efc0cb030 con 0x7f9efc0c9ca0 2023-10-16T09:50:04.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.151+0000 7f9f017a5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9efc0c91c0 con 0x7f9efc0caac0 2023-10-16T09:50:04.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.151+0000 7f9f017a5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc0c94a0 con 0x7f9efc15ad70 2023-10-16T09:50:04.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.151+0000 7f9efb7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9efc0caac0 0x7f9efc06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:04.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.151+0000 7f9efb7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9efc0caac0 0x7f9efc06a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38438/0 (socket says 172.21.15.84:38438) 2023-10-16T09:50:04.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.151+0000 7f9efb7fe640 1 -- 172.21.15.84:0/3134983510 learned_addr learned my addr 172.21.15.84:0/3134983510 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:04.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.151+0000 7f9efaffd640 1 --2- 172.21.15.84:0/3134983510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9efc15ad70 0x7f9efc069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:04.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.152+0000 7f9efb7fe640 1 -- 172.21.15.84:0/3134983510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9efc15ad70 msgr2=0x7f9efc069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:04.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.152+0000 7f9efb7fe640 1 --2- 172.21.15.84:0/3134983510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9efc15ad70 0x7f9efc069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:04.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.152+0000 7f9efb7fe640 1 -- 172.21.15.84:0/3134983510 >> v1:172.21.15.157:6789/0 conn(0x7f9efc0c9ca0 legacy=0x7f9efc0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:04.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.152+0000 7f9efb7fe640 1 -- 172.21.15.84:0/3134983510 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc0c9780 con 0x7f9efc0caac0 2023-10-16T09:50:04.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.152+0000 7f9efa7fc640 1 -- 172.21.15.84:0/3134983510 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1012865877 0 0) 0x7f9efc0cb030 con 0x7f9efc0c9ca0 2023-10-16T09:50:04.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.153+0000 7f9efb7fe640 1 --2- 172.21.15.84:0/3134983510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9efc0caac0 0x7f9efc06a2e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009e50 tx=0x7f9ef402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7067dff1a76047 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:04.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.153+0000 7f9efa7fc640 1 -- 172.21.15.84:0/3134983510 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef402cb00 con 0x7f9efc0caac0 2023-10-16T09:50:04.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.153+0000 7f9efa7fc640 1 -- 172.21.15.84:0/3134983510 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef402cc60 con 0x7f9efc0caac0 2023-10-16T09:50:04.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.153+0000 7f9efa7fc640 1 -- 172.21.15.84:0/3134983510 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef40343f0 con 0x7f9efc0caac0 2023-10-16T09:50:04.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.154+0000 7f9f017a5640 1 -- 172.21.15.84:0/3134983510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9efc0caac0 msgr2=0x7f9efc06a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:04.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.154+0000 7f9f017a5640 1 --2- 172.21.15.84:0/3134983510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9efc0caac0 0x7f9efc06a2e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009e50 tx=0x7f9ef402bf60 comp rx=0 tx=0).stop 2023-10-16T09:50:04.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.154+0000 7f9f017a5640 1 -- 172.21.15.84:0/3134983510 shutdown_connections 2023-10-16T09:50:04.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.154+0000 7f9f017a5640 1 --2- 172.21.15.84:0/3134983510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9efc0caac0 0x7f9efc06a2e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:04.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.154+0000 7f9f017a5640 1 --2- 172.21.15.84:0/3134983510 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9efc15ad70 0x7f9efc069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:04.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.154+0000 7f9f017a5640 1 -- 172.21.15.84:0/3134983510 >> 172.21.15.84:0/3134983510 conn(0x7f9efc0ce9b0 msgr2=0x7f9efc0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:04.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.154+0000 7f9f017a5640 1 -- 172.21.15.84:0/3134983510 shutdown_connections 2023-10-16T09:50:04.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.154+0000 7f9f017a5640 1 -- 172.21.15.84:0/3134983510 wait complete. 2023-10-16T09:50:04.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.155+0000 7f9f017a5640 1 Processor -- start 2023-10-16T09:50:04.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.156+0000 7f9f017a5640 1 -- start start 2023-10-16T09:50:04.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.157+0000 7f9f017a5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9efc0c9ca0 0x7f9efc1f0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:04.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.157+0000 7f9f017a5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9efc0caac0 0x7f9efc15bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:04.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.157+0000 7f9f017a5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9efc15ad70 0x7f9efc14bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:04.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.157+0000 7f9f017a5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc1f0740 con 0x7f9efc15ad70 2023-10-16T09:50:04.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.157+0000 7f9f017a5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc0c6b60 con 0x7f9efc0caac0 2023-10-16T09:50:04.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.157+0000 7f9f017a5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc0c6e40 con 0x7f9efc0c9ca0 2023-10-16T09:50:04.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.157+0000 7f9efb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9efc15ad70 0x7f9efc14bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:04.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.157+0000 7f9efaffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9efc0c9ca0 0x7f9efc1f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:04.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.157+0000 7f9efb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9efc15ad70 0x7f9efc14bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38442/0 (socket says 172.21.15.84:38442) 2023-10-16T09:50:04.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.157+0000 7f9efb7fe640 1 -- 172.21.15.84:0/3898941986 learned_addr learned my addr 172.21.15.84:0/3898941986 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:04.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.157+0000 7f9ef3fff640 1 --2- 172.21.15.84:0/3898941986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9efc0caac0 0x7f9efc15bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:04.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.157+0000 7f9efb7fe640 1 -- 172.21.15.84:0/3898941986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9efc0c9ca0 msgr2=0x7f9efc1f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:04.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.157+0000 7f9efb7fe640 1 --2- 172.21.15.84:0/3898941986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9efc0c9ca0 0x7f9efc1f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:04.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.157+0000 7f9efb7fe640 1 -- 172.21.15.84:0/3898941986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9efc0caac0 msgr2=0x7f9efc15bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:04.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.157+0000 7f9efb7fe640 1 --2- 172.21.15.84:0/3898941986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9efc0caac0 0x7f9efc15bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:04.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.157+0000 7f9efb7fe640 1 -- 172.21.15.84:0/3898941986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef4009b30 con 0x7f9efc15ad70 2023-10-16T09:50:04.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.158+0000 7f9efb7fe640 1 --2- 172.21.15.84:0/3898941986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9efc15ad70 0x7f9efc14bc60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009e50 tx=0x7f9ef4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:04.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.158+0000 7f9ef37fe640 1 -- 172.21.15.84:0/3898941986 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4009510 con 0x7f9efc15ad70 2023-10-16T09:50:04.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.158+0000 7f9ef37fe640 1 -- 172.21.15.84:0/3898941986 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef403e450 con 0x7f9efc15ad70 2023-10-16T09:50:04.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.158+0000 7f9ef37fe640 1 -- 172.21.15.84:0/3898941986 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef403e800 con 0x7f9efc15ad70 2023-10-16T09:50:04.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.158+0000 7f9f017a5640 1 -- 172.21.15.84:0/3898941986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9efc0c7120 con 0x7f9efc15ad70 2023-10-16T09:50:04.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.158+0000 7f9f017a5640 1 -- 172.21.15.84:0/3898941986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9efc0c76e0 con 0x7f9efc15ad70 2023-10-16T09:50:04.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.159+0000 7f9f017a5640 1 -- 172.21.15.84:0/3898941986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9efc04e160 con 0x7f9efc15ad70 2023-10-16T09:50:04.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.160+0000 7f9ef37fe640 1 -- 172.21.15.84:0/3898941986 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9ef403e960 con 0x7f9efc15ad70 2023-10-16T09:50:04.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.161+0000 7f9ef37fe640 1 --2- 172.21.15.84:0/3898941986 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9edc06ae70 0x7f9edc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:04.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.161+0000 7f9ef37fe640 1 -- 172.21.15.84:0/3898941986 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9ef40abf80 con 0x7f9efc15ad70 2023-10-16T09:50:04.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.161+0000 7f9efaffd640 1 --2- 172.21.15.84:0/3898941986 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9edc06ae70 0x7f9edc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:04.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.162+0000 7f9efaffd640 1 --2- 172.21.15.84:0/3898941986 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9edc06ae70 0x7f9edc06d330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009aa0 tx=0x7f9eec009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:04.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.166+0000 7f9ef37fe640 1 -- 172.21.15.84:0/3898941986 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9ef4077340 con 0x7f9efc15ad70 2023-10-16T09:50:04.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.371+0000 7f9f017a5640 1 -- 172.21.15.84:0/3898941986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9efc06d6e0 con 0x7f9efc15ad70 2023-10-16T09:50:04.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.866+0000 7f9ef37fe640 1 -- 172.21.15.84:0/3898941986 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9ef407a960 con 0x7f9efc15ad70 2023-10-16T09:50:04.867 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-10-16T09:50:04.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.869+0000 7f9f017a5640 1 -- 172.21.15.84:0/3898941986 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9edc06ae70 msgr2=0x7f9edc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:04.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.869+0000 7f9f017a5640 1 --2- 172.21.15.84:0/3898941986 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9edc06ae70 0x7f9edc06d330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009aa0 tx=0x7f9eec009190 comp rx=0 tx=0).stop 2023-10-16T09:50:04.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.869+0000 7f9f017a5640 1 -- 172.21.15.84:0/3898941986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9efc15ad70 msgr2=0x7f9efc14bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:04.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.869+0000 7f9f017a5640 1 --2- 172.21.15.84:0/3898941986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9efc15ad70 0x7f9efc14bc60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009e50 tx=0x7f9ef4004a40 comp rx=0 tx=0).stop 2023-10-16T09:50:04.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.869+0000 7f9f017a5640 1 -- 172.21.15.84:0/3898941986 shutdown_connections 2023-10-16T09:50:04.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.869+0000 7f9f017a5640 1 --2- 172.21.15.84:0/3898941986 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9edc06ae70 0x7f9edc06d330 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:04.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.869+0000 7f9f017a5640 1 --2- 172.21.15.84:0/3898941986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9efc15ad70 0x7f9efc14bc60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:04.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.869+0000 7f9f017a5640 1 --2- 172.21.15.84:0/3898941986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9efc0caac0 0x7f9efc15bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:04.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.869+0000 7f9f017a5640 1 --2- 172.21.15.84:0/3898941986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9efc0c9ca0 0x7f9efc1f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:04.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.869+0000 7f9f017a5640 1 -- 172.21.15.84:0/3898941986 >> 172.21.15.84:0/3898941986 conn(0x7f9efc0ce9b0 msgr2=0x7f9efc0b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:04.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.869+0000 7f9f017a5640 1 -- 172.21.15.84:0/3898941986 shutdown_connections 2023-10-16T09:50:04.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:04.870+0000 7f9f017a5640 1 -- 172.21.15.84:0/3898941986 wait complete. 2023-10-16T09:50:04.893 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T09:50:05.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.101+0000 7f2b21ebb640 1 Processor -- start 2023-10-16T09:50:05.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.101+0000 7f2b21ebb640 1 -- start start 2023-10-16T09:50:05.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.103+0000 7f2b21ebb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b1c070fd0 0x7f2b1c0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:05.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.103+0000 7f2b21ebb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2b1c0adf30 0x7f2b1c0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:05.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.103+0000 7f2b21ebb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b1c071450 con 0x7f2b1c0adb50 2023-10-16T09:50:05.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.103+0000 7f2b21ebb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b1c0cc600 con 0x7f2b1c0adf30 2023-10-16T09:50:05.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.103+0000 7f2b21ebb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c0cc8e0 con 0x7f2b1c070fd0 2023-10-16T09:50:05.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.104+0000 7f2b1bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2b1c0adf30 0x7f2b1c0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:05.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.104+0000 7f2b1bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2b1c0adf30 0x7f2b1c0b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:33786/0 (socket says 172.21.15.84:33786) 2023-10-16T09:50:05.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.104+0000 7f2b1bfff640 1 -- 172.21.15.84:0/2189863089 learned_addr learned my addr 172.21.15.84:0/2189863089 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:05.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.104+0000 7f2b12ffd640 1 --2- 172.21.15.84:0/2189863089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b1c070fd0 0x7f2b1c0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:05.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.105+0000 7f2b1affd640 1 -- 172.21.15.84:0/2189863089 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977410358 0 0) 0x7f2b1c071450 con 0x7f2b1c0adb50 2023-10-16T09:50:05.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.105+0000 7f2b12ffd640 1 -- 172.21.15.84:0/2189863089 >> v2:172.21.15.157:3300/0 conn(0x7f2b1c0adf30 msgr2=0x7f2b1c0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:05.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.105+0000 7f2b12ffd640 1 --2- 172.21.15.84:0/2189863089 >> v2:172.21.15.157:3300/0 conn(0x7f2b1c0adf30 0x7f2b1c0b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.105+0000 7f2b12ffd640 1 -- 172.21.15.84:0/2189863089 >> v1:172.21.15.157:6789/0 conn(0x7f2b1c0adb50 legacy=0x7f2b1c06f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:05.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.105+0000 7f2b12ffd640 1 -- 172.21.15.84:0/2189863089 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c0ccbc0 con 0x7f2b1c070fd0 2023-10-16T09:50:05.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.106+0000 7f2b12ffd640 1 --2- 172.21.15.84:0/2189863089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b1c070fd0 0x7f2b1c0b0410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2b040097a0 tx=0x7f2b0402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32750e167c0c4d0f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:05.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.107+0000 7f2b1affd640 1 -- 172.21.15.84:0/2189863089 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0402d040 con 0x7f2b1c070fd0 2023-10-16T09:50:05.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.107+0000 7f2b1affd640 1 -- 172.21.15.84:0/2189863089 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b04031810 con 0x7f2b1c070fd0 2023-10-16T09:50:05.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.107+0000 7f2b1affd640 1 -- 172.21.15.84:0/2189863089 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b04031ba0 con 0x7f2b1c070fd0 2023-10-16T09:50:05.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.108+0000 7f2b21ebb640 1 -- 172.21.15.84:0/2189863089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b1c070fd0 msgr2=0x7f2b1c0b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:05.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.108+0000 7f2b21ebb640 1 --2- 172.21.15.84:0/2189863089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b1c070fd0 0x7f2b1c0b0410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2b040097a0 tx=0x7f2b0402b8b0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.108+0000 7f2b21ebb640 1 -- 172.21.15.84:0/2189863089 shutdown_connections 2023-10-16T09:50:05.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.108+0000 7f2b21ebb640 1 --2- 172.21.15.84:0/2189863089 >> v2:172.21.15.157:3300/0 conn(0x7f2b1c0adf30 0x7f2b1c0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.108+0000 7f2b21ebb640 1 --2- 172.21.15.84:0/2189863089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b1c070fd0 0x7f2b1c0b0410 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.108+0000 7f2b21ebb640 1 -- 172.21.15.84:0/2189863089 >> 172.21.15.84:0/2189863089 conn(0x7f2b1c0cb410 msgr2=0x7f2b1c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:05.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.108+0000 7f2b21ebb640 1 -- 172.21.15.84:0/2189863089 shutdown_connections 2023-10-16T09:50:05.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.108+0000 7f2b21ebb640 1 -- 172.21.15.84:0/2189863089 wait complete. 2023-10-16T09:50:05.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.110+0000 7f2b21ebb640 1 Processor -- start 2023-10-16T09:50:05.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.110+0000 7f2b21ebb640 1 -- start start 2023-10-16T09:50:05.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.111+0000 7f2b21ebb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b1c070fd0 0x7f2b1c15f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:05.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.111+0000 7f2b21ebb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b1c0adb50 0x7f2b1c1f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:05.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.111+0000 7f2b21ebb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b1c0adf30 0x7f2b1c0b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:05.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.111+0000 7f2b21ebb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c0c0bb0 con 0x7f2b1c070fd0 2023-10-16T09:50:05.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.111+0000 7f2b21ebb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c0b27a0 con 0x7f2b1c0adf30 2023-10-16T09:50:05.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.111+0000 7f2b21ebb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c1f78e0 con 0x7f2b1c0adb50 2023-10-16T09:50:05.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.111+0000 7f2b1b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b1c070fd0 0x7f2b1c15f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:05.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.111+0000 7f2b1b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b1c070fd0 0x7f2b1c15f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38450/0 (socket says 172.21.15.84:38450) 2023-10-16T09:50:05.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.111+0000 7f2b1b7fe640 1 -- 172.21.15.84:0/3040074663 learned_addr learned my addr 172.21.15.84:0/3040074663 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:05.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.111+0000 7f2b1b7fe640 1 -- 172.21.15.84:0/3040074663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b1c0adb50 msgr2=0x7f2b1c1f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:05.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.111+0000 7f2b1b7fe640 1 --2- 172.21.15.84:0/3040074663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b1c0adb50 0x7f2b1c1f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.111+0000 7f2b1bfff640 1 --2- 172.21.15.84:0/3040074663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b1c0adf30 0x7f2b1c0b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:05.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.111+0000 7f2b1b7fe640 1 -- 172.21.15.84:0/3040074663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b1c0adf30 msgr2=0x7f2b1c0b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:05.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.111+0000 7f2b1b7fe640 1 --2- 172.21.15.84:0/3040074663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b1c0adf30 0x7f2b1c0b2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.111+0000 7f2b1b7fe640 1 -- 172.21.15.84:0/3040074663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b04009400 con 0x7f2b1c070fd0 2023-10-16T09:50:05.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.112+0000 7f2b1b7fe640 1 --2- 172.21.15.84:0/3040074663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b1c070fd0 0x7f2b1c15f8c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00c4b0 tx=0x7f2b0c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:05.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.112+0000 7f2b18ff9640 1 -- 172.21.15.84:0/3040074663 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c005480 con 0x7f2b1c070fd0 2023-10-16T09:50:05.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.112+0000 7f2b18ff9640 1 -- 172.21.15.84:0/3040074663 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0c0035f0 con 0x7f2b1c070fd0 2023-10-16T09:50:05.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.112+0000 7f2b18ff9640 1 -- 172.21.15.84:0/3040074663 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c0039e0 con 0x7f2b1c070fd0 2023-10-16T09:50:05.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.112+0000 7f2b21ebb640 1 -- 172.21.15.84:0/3040074663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b1c0b1b90 con 0x7f2b1c070fd0 2023-10-16T09:50:05.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.112+0000 7f2b21ebb640 1 -- 172.21.15.84:0/3040074663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1c0b2150 con 0x7f2b1c070fd0 2023-10-16T09:50:05.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.113+0000 7f2b21ebb640 1 -- 172.21.15.84:0/3040074663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b1c04e160 con 0x7f2b1c070fd0 2023-10-16T09:50:05.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.114+0000 7f2b18ff9640 1 -- 172.21.15.84:0/3040074663 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2b0c003b40 con 0x7f2b1c070fd0 2023-10-16T09:50:05.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.115+0000 7f2b18ff9640 1 --2- 172.21.15.84:0/3040074663 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2afc06aea0 0x7f2afc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:05.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.115+0000 7f2b18ff9640 1 -- 172.21.15.84:0/3040074663 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2b0c0887c0 con 0x7f2b1c070fd0 2023-10-16T09:50:05.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.115+0000 7f2b12ffd640 1 --2- 172.21.15.84:0/3040074663 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2afc06aea0 0x7f2afc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:05.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.116+0000 7f2b12ffd640 1 --2- 172.21.15.84:0/3040074663 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2afc06aea0 0x7f2afc06d360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2b04036000 tx=0x7f2b04030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:05.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.119+0000 7f2b18ff9640 1 -- 172.21.15.84:0/3040074663 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2b0c053c50 con 0x7f2b1c070fd0 2023-10-16T09:50:05.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.379+0000 7f2b21ebb640 1 -- 172.21.15.84:0/3040074663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2b1c0b4210 con 0x7f2b1c070fd0 2023-10-16T09:50:05.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.380+0000 7f2b18ff9640 1 -- 172.21.15.84:0/3040074663 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f2b0c017020 con 0x7f2b1c070fd0 2023-10-16T09:50:05.381 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:50:05.381 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T09:50:05.382 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-10-16T09:50:05.382 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.382 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-10-16T09:50:05.382 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l6dhjKRAA0pMvY6AK6qtFXDprNhwsqg==", 2023-10-16T09:50:05.382 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.383 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T09:50:05.383 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T09:50:05.383 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T09:50:05.383 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T09:50:05.384 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.384 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.384 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.384 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-10-16T09:50:05.384 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lFpMxEBAA0TLPRt5q3FwwBvMG019cjQ==", 2023-10-16T09:50:05.385 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.385 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T09:50:05.385 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T09:50:05.385 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T09:50:05.385 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T09:50:05.386 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.386 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.386 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.386 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-10-16T09:50:05.386 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l73RBOhAAOMEPxfUAMHhn5Blyvd5oMw==", 2023-10-16T09:50:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T09:50:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T09:50:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T09:50:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T09:50:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.388 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.388 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.388 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-10-16T09:50:05.388 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lcTIRIhAA0+FWm4b4yw77LD3+WKbk7w==", 2023-10-16T09:50:05.388 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T09:50:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T09:50:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T09:50:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T09:50:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-10-16T09:50:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDYBi1l1zBLERAA3YLW3xIYQoRVw7KSkwN3Og==", 2023-10-16T09:50:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T09:50:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T09:50:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T09:50:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.392 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-10-16T09:50:05.392 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDbBi1lFUUxKRAAQbus8SmYprJRmWTjoCznNA==", 2023-10-16T09:50:05.392 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.392 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T09:50:05.392 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T09:50:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T09:50:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-10-16T09:50:05.394 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDeBi1l+9lBMxAArl6sb/h15uBG/a9lVH40Hg==", 2023-10-16T09:50:05.394 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.394 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T09:50:05.394 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T09:50:05.395 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T09:50:05.395 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.395 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.395 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.395 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-10-16T09:50:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDiBi1l2rnEERAA3oRD71xa0fEOynM8zKuBEg==", 2023-10-16T09:50:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T09:50:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T09:50:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T09:50:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.397 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.397 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.397 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-10-16T09:50:05.397 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDnBi1lR7oLChAAZto8fDovDbpOd3Fcnc7C6w==", 2023-10-16T09:50:05.397 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.397 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T09:50:05.398 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T09:50:05.398 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T09:50:05.398 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.398 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.398 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-10-16T09:50:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDqBi1lmjg/JBAAlINKRGBc0/UWJJHMVFw57w==", 2023-10-16T09:50:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T09:50:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T09:50:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T09:50:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-10-16T09:50:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDuBi1lu0kMARAAtuSeM8F3yr8CVrc3p1biaQ==", 2023-10-16T09:50:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T09:50:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T09:50:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T09:50:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-10-16T09:50:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDxBi1lf6mNGBAAHGbqzJ0fmDxQIsasboHMfg==", 2023-10-16T09:50:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T09:50:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T09:50:05.403 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T09:50:05.403 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.403 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.403 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.403 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-10-16T09:50:05.404 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ==", 2023-10-16T09:50:05.404 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.404 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T09:50:05.404 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T09:50:05.404 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T09:50:05.404 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T09:50:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-10-16T09:50:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1lHU+4BxAAd8ET8hQ8Y+EUwdTJr9/+Ig==", 2023-10-16T09:50:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.406 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T09:50:05.406 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T09:50:05.406 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T09:50:05.406 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T09:50:05.407 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.407 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.407 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.407 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-10-16T09:50:05.407 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lA1BOORAAFNyfqgEF34MVR9YX3KBKkQ==", 2023-10-16T09:50:05.407 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T09:50:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T09:50:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T09:50:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T09:50:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-10-16T09:50:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1l848oERAANMfrZz9JrlgNVh7ZFbtnaA==", 2023-10-16T09:50:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T09:50:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T09:50:05.410 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T09:50:05.410 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T09:50:05.410 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.410 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.410 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-10-16T09:50:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDRBi1lTAxTGRAAMptxnxy3Qit/cTCXie6CrQ==", 2023-10-16T09:50:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T09:50:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T09:50:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T09:50:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T09:50:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.bootstrap-mds", 2023-10-16T09:50:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQD6Bi1lRa+0LBAAd9LjDuOW/+64ZeCSF7EwbA==", 2023-10-16T09:50:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile bootstrap-mds" 2023-10-16T09:50:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.414 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.bootstrap-mgr", 2023-10-16T09:50:05.414 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQD6Bi1lBs20LBAApzIs6ouIzMGWnEeJu6YTLw==", 2023-10-16T09:50:05.414 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.414 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-16T09:50:05.414 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.bootstrap-osd", 2023-10-16T09:50:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQD6Bi1l5OW0LBAA6XunITn8bteo9YMIxGDbHg==", 2023-10-16T09:50:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile bootstrap-osd" 2023-10-16T09:50:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.bootstrap-rbd", 2023-10-16T09:50:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQD6Bi1l9P20LBAAKwcoGmGLv6/6C7AAqKVObw==", 2023-10-16T09:50:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-16T09:50:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-16T09:50:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQD6Bi1lNha1LBAAyHjzQINWOj39Obz4xXRtJA==", 2023-10-16T09:50:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-16T09:50:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.bootstrap-rgw", 2023-10-16T09:50:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQD6Bi1lCDC1LBAAOICOjfqXJaEkpkZYolJQIw==", 2023-10-16T09:50:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-16T09:50:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-10-16T09:50:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lPuJxHRAABAnGPVCrkq0JR4Y6iFGZlw==", 2023-10-16T09:50:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T09:50:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T09:50:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T09:50:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T09:50:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-10-16T09:50:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lLxnKEBAA+a/pOHUp7Sg3m4weAust5w==", 2023-10-16T09:50:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T09:50:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T09:50:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T09:50:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T09:50:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.423 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:05.423 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-10-16T09:50:05.423 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T09:50:05.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.383+0000 7f2b21ebb640 1 -- 172.21.15.84:0/3040074663 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2afc06aea0 msgr2=0x7f2afc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:05.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.383+0000 7f2b21ebb640 1 --2- 172.21.15.84:0/3040074663 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2afc06aea0 0x7f2afc06d360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2b04036000 tx=0x7f2b04030000 comp rx=0 tx=0).stop 2023-10-16T09:50:05.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.383+0000 7f2b21ebb640 1 -- 172.21.15.84:0/3040074663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b1c070fd0 msgr2=0x7f2b1c15f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:05.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.383+0000 7f2b21ebb640 1 --2- 172.21.15.84:0/3040074663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b1c070fd0 0x7f2b1c15f8c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00c4b0 tx=0x7f2b0c00c970 comp rx=0 tx=0).stop 2023-10-16T09:50:05.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.383+0000 7f2b21ebb640 1 -- 172.21.15.84:0/3040074663 shutdown_connections 2023-10-16T09:50:05.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.383+0000 7f2b21ebb640 1 --2- 172.21.15.84:0/3040074663 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2afc06aea0 0x7f2afc06d360 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.383+0000 7f2b21ebb640 1 --2- 172.21.15.84:0/3040074663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b1c0adf30 0x7f2b1c0b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.383+0000 7f2b21ebb640 1 --2- 172.21.15.84:0/3040074663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b1c0adb50 0x7f2b1c1f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.383+0000 7f2b21ebb640 1 --2- 172.21.15.84:0/3040074663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b1c070fd0 0x7f2b1c15f8c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.383+0000 7f2b21ebb640 1 -- 172.21.15.84:0/3040074663 >> 172.21.15.84:0/3040074663 conn(0x7f2b1c0cb410 msgr2=0x7f2b1c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:05.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.384+0000 7f2b21ebb640 1 -- 172.21.15.84:0/3040074663 shutdown_connections 2023-10-16T09:50:05.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.384+0000 7f2b21ebb640 1 -- 172.21.15.84:0/3040074663 wait complete. 2023-10-16T09:50:05.427 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-16T09:50:05.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.642+0000 7fb8bb4e5640 1 Processor -- start 2023-10-16T09:50:05.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.643+0000 7fb8bb4e5640 1 -- start start 2023-10-16T09:50:05.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.644+0000 7fb8bb4e5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8b415cd70 0x7fb8b40d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:05.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.644+0000 7fb8bb4e5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb8b40cded0 0x7fb8b40b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:05.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.644+0000 7fb8bb4e5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b415d150 con 0x7fb8b40b8480 2023-10-16T09:50:05.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.644+0000 7fb8bb4e5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b40d1180 con 0x7fb8b40cded0 2023-10-16T09:50:05.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.644+0000 7fb8bb4e5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b40d1460 con 0x7fb8b415cd70 2023-10-16T09:50:05.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.644+0000 7fb8bace4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb8b40cded0 0x7fb8b40b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:05.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.644+0000 7fb8bace4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb8b40cded0 0x7fb8b40b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46856/0 (socket says 172.21.15.84:46856) 2023-10-16T09:50:05.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.644+0000 7fb8bace4640 1 -- 172.21.15.84:0/257470206 learned_addr learned my addr 172.21.15.84:0/257470206 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:05.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.645+0000 7fb8bace4640 1 -- 172.21.15.84:0/257470206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8b415cd70 msgr2=0x7fb8b40d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:05.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.645+0000 7fb8bace4640 1 --2- 172.21.15.84:0/257470206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8b415cd70 0x7fb8b40d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.645+0000 7fb8bace4640 1 -- 172.21.15.84:0/257470206 >> v1:172.21.15.157:6789/0 conn(0x7fb8b40b8480 legacy=0x7fb8b40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:05.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.645+0000 7fb8bace4640 1 -- 172.21.15.84:0/257470206 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b40d1740 con 0x7fb8b40cded0 2023-10-16T09:50:05.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.646+0000 7fb8bace4640 1 --2- 172.21.15.84:0/257470206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8b40cded0 0x7fb8b40b5c90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009870 tx=0x7fb8ac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719a005924b3617f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:05.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.646+0000 7fb8b94e1640 1 -- 172.21.15.84:0/257470206 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac031640 con 0x7fb8b40cded0 2023-10-16T09:50:05.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.646+0000 7fb8b94e1640 1 -- 172.21.15.84:0/257470206 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ac0317a0 con 0x7fb8b40cded0 2023-10-16T09:50:05.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.646+0000 7fb8b94e1640 1 -- 172.21.15.84:0/257470206 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac031b30 con 0x7fb8b40cded0 2023-10-16T09:50:05.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.647+0000 7fb8bb4e5640 1 -- 172.21.15.84:0/257470206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8b40cded0 msgr2=0x7fb8b40b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:05.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.647+0000 7fb8bb4e5640 1 --2- 172.21.15.84:0/257470206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8b40cded0 0x7fb8b40b5c90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009870 tx=0x7fb8ac02b8f0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.647+0000 7fb8bb4e5640 1 -- 172.21.15.84:0/257470206 shutdown_connections 2023-10-16T09:50:05.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.647+0000 7fb8bb4e5640 1 --2- 172.21.15.84:0/257470206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8b40cded0 0x7fb8b40b5c90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.647+0000 7fb8bb4e5640 1 --2- 172.21.15.84:0/257470206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8b415cd70 0x7fb8b40d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.647+0000 7fb8bb4e5640 1 -- 172.21.15.84:0/257470206 >> 172.21.15.84:0/257470206 conn(0x7fb8b40cf570 msgr2=0x7fb8b40cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:05.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.647+0000 7fb8bb4e5640 1 -- 172.21.15.84:0/257470206 shutdown_connections 2023-10-16T09:50:05.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.647+0000 7fb8bb4e5640 1 -- 172.21.15.84:0/257470206 wait complete. 2023-10-16T09:50:05.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.649+0000 7fb8bb4e5640 1 Processor -- start 2023-10-16T09:50:05.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.649+0000 7fb8bb4e5640 1 -- start start 2023-10-16T09:50:05.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.650+0000 7fb8bb4e5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8b40b8480 0x7fb8b41f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:05.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.650+0000 7fb8bb4e5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8b40cded0 0x7fb8b41f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:05.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.650+0000 7fb8bb4e5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8b415cd70 0x7fb8b40c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:05.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.650+0000 7fb8bb4e5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b415df30 con 0x7fb8b40cded0 2023-10-16T09:50:05.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.650+0000 7fb8bb4e5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b415d340 con 0x7fb8b40b8480 2023-10-16T09:50:05.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.650+0000 7fb8bb4e5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b415d590 con 0x7fb8b415cd70 2023-10-16T09:50:05.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.650+0000 7fb8b9ce2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8b40cded0 0x7fb8b41f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:05.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.650+0000 7fb8b9ce2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8b40cded0 0x7fb8b41f4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46866/0 (socket says 172.21.15.84:46866) 2023-10-16T09:50:05.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.650+0000 7fb8b9ce2640 1 -- 172.21.15.84:0/819860014 learned_addr learned my addr 172.21.15.84:0/819860014 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:05.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.651+0000 7fb8b9ce2640 1 -- 172.21.15.84:0/819860014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8b415cd70 msgr2=0x7fb8b40c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:05.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.651+0000 7fb8b9ce2640 1 --2- 172.21.15.84:0/819860014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8b415cd70 0x7fb8b40c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.651+0000 7fb8b9ce2640 1 -- 172.21.15.84:0/819860014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8b40b8480 msgr2=0x7fb8b41f31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:05.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.651+0000 7fb8ba4e3640 1 --2- 172.21.15.84:0/819860014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8b40b8480 0x7fb8b41f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:05.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.651+0000 7fb8b9ce2640 1 --2- 172.21.15.84:0/819860014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8b40b8480 0x7fb8b41f31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.651+0000 7fb8b9ce2640 1 -- 172.21.15.84:0/819860014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ac0094d0 con 0x7fb8b40cded0 2023-10-16T09:50:05.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.651+0000 7fb8ba4e3640 1 --2- 172.21.15.84:0/819860014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8b40b8480 0x7fb8b41f31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:05.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.651+0000 7fb8b9ce2640 1 --2- 172.21.15.84:0/819860014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8b40cded0 0x7fb8b41f4d80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000c410 tx=0x7fb8a000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:05.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.651+0000 7fb8b2ffd640 1 -- 172.21.15.84:0/819860014 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a00035f0 con 0x7fb8b40cded0 2023-10-16T09:50:05.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.651+0000 7fb8b2ffd640 1 -- 172.21.15.84:0/819860014 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a0003750 con 0x7fb8b40cded0 2023-10-16T09:50:05.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.651+0000 7fb8b2ffd640 1 -- 172.21.15.84:0/819860014 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0003b40 con 0x7fb8b40cded0 2023-10-16T09:50:05.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.651+0000 7fb8bb4e5640 1 -- 172.21.15.84:0/819860014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b415d870 con 0x7fb8b40cded0 2023-10-16T09:50:05.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.652+0000 7fb8bb4e5640 1 -- 172.21.15.84:0/819860014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b41f3720 con 0x7fb8b40cded0 2023-10-16T09:50:05.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.653+0000 7fb8bb4e5640 1 -- 172.21.15.84:0/819860014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b415e070 con 0x7fb8b40cded0 2023-10-16T09:50:05.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.654+0000 7fb8b2ffd640 1 -- 172.21.15.84:0/819860014 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb8a0003ca0 con 0x7fb8b40cded0 2023-10-16T09:50:05.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.654+0000 7fb8b2ffd640 1 --2- 172.21.15.84:0/819860014 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb89806ae70 0x7fb89806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:05.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.655+0000 7fb8b2ffd640 1 -- 172.21.15.84:0/819860014 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb8a00884e0 con 0x7fb8b40cded0 2023-10-16T09:50:05.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.655+0000 7fb8ba4e3640 1 --2- 172.21.15.84:0/819860014 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb89806ae70 0x7fb89806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:05.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.655+0000 7fb8ba4e3640 1 --2- 172.21.15.84:0/819860014 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb89806ae70 0x7fb89806d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8007eb0 tx=0x7fb8a80073c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:05.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.659+0000 7fb8b2ffd640 1 -- 172.21.15.84:0/819860014 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb8b415e070 con 0x7fb8b40cded0 2023-10-16T09:50:05.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.929+0000 7fb8bb4e5640 1 -- 172.21.15.84:0/819860014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fb8b415e070 con 0x7fb8b40cded0 2023-10-16T09:50:05.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.933+0000 7fb8b2ffd640 1 -- 172.21.15.84:0/819860014 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb8b415e070 con 0x7fb8b40cded0 2023-10-16T09:50:05.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.935+0000 7fb8bb4e5640 1 -- 172.21.15.84:0/819860014 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb89806ae70 msgr2=0x7fb89806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:05.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.935+0000 7fb8bb4e5640 1 --2- 172.21.15.84:0/819860014 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb89806ae70 0x7fb89806d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8007eb0 tx=0x7fb8a80073c0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.936+0000 7fb8bb4e5640 1 -- 172.21.15.84:0/819860014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8b40cded0 msgr2=0x7fb8b41f4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:05.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.936+0000 7fb8bb4e5640 1 --2- 172.21.15.84:0/819860014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8b40cded0 0x7fb8b41f4d80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000c410 tx=0x7fb8a000c8d0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.936+0000 7fb8bb4e5640 1 -- 172.21.15.84:0/819860014 shutdown_connections 2023-10-16T09:50:05.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.936+0000 7fb8bb4e5640 1 --2- 172.21.15.84:0/819860014 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb89806ae70 0x7fb89806d330 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.936+0000 7fb8bb4e5640 1 --2- 172.21.15.84:0/819860014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8b415cd70 0x7fb8b40c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.936+0000 7fb8bb4e5640 1 --2- 172.21.15.84:0/819860014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8b40cded0 0x7fb8b41f4d80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.936+0000 7fb8bb4e5640 1 --2- 172.21.15.84:0/819860014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8b40b8480 0x7fb8b41f31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:05.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.936+0000 7fb8bb4e5640 1 -- 172.21.15.84:0/819860014 >> 172.21.15.84:0/819860014 conn(0x7fb8b40cf570 msgr2=0x7fb8b40b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:05.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.936+0000 7fb8bb4e5640 1 -- 172.21.15.84:0/819860014 shutdown_connections 2023-10-16T09:50:05.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:05.936+0000 7fb8bb4e5640 1 -- 172.21.15.84:0/819860014 wait complete. 2023-10-16T09:50:05.954 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-16T09:50:06.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.162+0000 7f2f1bfff640 1 Processor -- start 2023-10-16T09:50:06.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.162+0000 7f2f1bfff640 1 -- start start 2023-10-16T09:50:06.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.163+0000 7f2f1bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2f1c159950 0x7f2f1c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:06.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.164+0000 7f2f1bfff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f1c15aef0 con 0x7f2f1c06c810 2023-10-16T09:50:06.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.164+0000 7f2f1bfff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f1c15d910 con 0x7f2f1c070450 2023-10-16T09:50:06.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.164+0000 7f2f1bfff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f1c15e490 con 0x7f2f1c159950 2023-10-16T09:50:06.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.164+0000 7f2f1b7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2f1c06c810 0x7f2f1c0c9090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48560/0 (socket says 172.21.15.84:48560) 2023-10-16T09:50:06.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.164+0000 7f2f1b7fe640 1 -- 172.21.15.84:0/3455979934 learned_addr learned my addr 172.21.15.84:0/3455979934 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:06.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.164+0000 7f2f1affd640 1 --2- 172.21.15.84:0/3455979934 >> v2:172.21.15.157:3300/0 conn(0x7f2f1c159950 0x7f2f1c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:06.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.164+0000 7f2f1a7fc640 1 -- 172.21.15.84:0/3455979934 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217846545 0 0) 0x7f2f14001490 con 0x7f2f1c06c810 2023-10-16T09:50:06.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.165+0000 7f2f1a7fc640 1 -- 172.21.15.84:0/3455979934 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f08003520 con 0x7f2f1c06c810 2023-10-16T09:50:06.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.165+0000 7f2f1a7fc640 1 -- 172.21.15.84:0/3455979934 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3531122663 0 0) 0x7f2f08003520 con 0x7f2f1c06c810 2023-10-16T09:50:06.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.165+0000 7f2f1a7fc640 1 -- 172.21.15.84:0/3455979934 >> v2:172.21.15.157:3300/0 conn(0x7f2f1c159950 msgr2=0x7f2f1c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:06.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.165+0000 7f2f1a7fc640 1 --2- 172.21.15.84:0/3455979934 >> v2:172.21.15.157:3300/0 conn(0x7f2f1c159950 0x7f2f1c15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.165+0000 7f2f1a7fc640 1 -- 172.21.15.84:0/3455979934 >> v1:172.21.15.157:6789/0 conn(0x7f2f1c070450 legacy=0x7f2f1c06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:06.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.165+0000 7f2f1a7fc640 1 -- 172.21.15.84:0/3455979934 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c15dbf0 con 0x7f2f1c06c810 2023-10-16T09:50:06.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.165+0000 7f2f1a7fc640 1 -- 172.21.15.84:0/3455979934 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f2f14001b80 con 0x7f2f1c06c810 2023-10-16T09:50:06.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.165+0000 7f2f1a7fc640 1 -- 172.21.15.84:0/3455979934 >> v1:172.21.15.84:6789/0 conn(0x7f2f1c06c810 legacy=0x7f2f1c0c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:06.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.165+0000 7f2f1a7fc640 1 --2- 172.21.15.84:0/3455979934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f080039c0 0x7f2f08003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:06.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.165+0000 7f2f1a7fc640 1 --2- 172.21.15.84:0/3455979934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f080047c0 0x7f2f08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:06.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.165+0000 7f2f1a7fc640 1 --2- 172.21.15.84:0/3455979934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f080051d0 0x7f2f0800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:06.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.165+0000 7f2f1a7fc640 1 -- 172.21.15.84:0/3455979934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c15e490 con 0x7f2f080047c0 2023-10-16T09:50:06.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.165+0000 7f2f1a7fc640 1 -- 172.21.15.84:0/3455979934 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0800c120 con 0x7f2f080039c0 2023-10-16T09:50:06.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.165+0000 7f2f1a7fc640 1 -- 172.21.15.84:0/3455979934 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0800c2e0 con 0x7f2f080051d0 2023-10-16T09:50:06.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.165+0000 7f2f127fc640 1 --2- 172.21.15.84:0/3455979934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f080047c0 0x7f2f08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:06.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.166+0000 7f2f1affd640 1 --2- 172.21.15.84:0/3455979934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f080039c0 0x7f2f08003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:06.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.166+0000 7f2f1b7fe640 1 --2- 172.21.15.84:0/3455979934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f080051d0 0x7f2f0800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:06.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.166+0000 7f2f127fc640 1 -- 172.21.15.84:0/3455979934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f080051d0 msgr2=0x7f2f0800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:06.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.166+0000 7f2f127fc640 1 --2- 172.21.15.84:0/3455979934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f080051d0 0x7f2f0800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.166+0000 7f2f127fc640 1 -- 172.21.15.84:0/3455979934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f080039c0 msgr2=0x7f2f08003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:06.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.166+0000 7f2f127fc640 1 --2- 172.21.15.84:0/3455979934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f080039c0 0x7f2f08003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.166+0000 7f2f127fc640 1 -- 172.21.15.84:0/3455979934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f08003520 con 0x7f2f080047c0 2023-10-16T09:50:06.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.166+0000 7f2f1b7fe640 1 -- 172.21.15.84:0/3455979934 reap_dead start 2023-10-16T09:50:06.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.167+0000 7f2f127fc640 1 --2- 172.21.15.84:0/3455979934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f080047c0 0x7f2f08004c40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2f04001720 tx=0x7f2f080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca4fb7254600b79 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:06.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.167+0000 7f2f1a7fc640 1 -- 172.21.15.84:0/3455979934 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f040052d0 con 0x7f2f080047c0 2023-10-16T09:50:06.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.167+0000 7f2f1a7fc640 1 -- 172.21.15.84:0/3455979934 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0402e050 con 0x7f2f080047c0 2023-10-16T09:50:06.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.167+0000 7f2f1a7fc640 1 -- 172.21.15.84:0/3455979934 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04003830 con 0x7f2f080047c0 2023-10-16T09:50:06.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.168+0000 7f2f1bfff640 1 -- 172.21.15.84:0/3455979934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f080047c0 msgr2=0x7f2f08004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:06.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.168+0000 7f2f1bfff640 1 --2- 172.21.15.84:0/3455979934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f080047c0 0x7f2f08004c40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2f04001720 tx=0x7f2f080057f0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.168+0000 7f2f1bfff640 1 -- 172.21.15.84:0/3455979934 shutdown_connections 2023-10-16T09:50:06.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.168+0000 7f2f1bfff640 1 --2- 172.21.15.84:0/3455979934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f080047c0 0x7f2f08004c40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.168+0000 7f2f1bfff640 1 -- 172.21.15.84:0/3455979934 >> 172.21.15.84:0/3455979934 conn(0x7f2f1c0ce6a0 msgr2=0x7f2f1c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:06.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.168+0000 7f2f1bfff640 1 -- 172.21.15.84:0/3455979934 shutdown_connections 2023-10-16T09:50:06.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.168+0000 7f2f1bfff640 1 -- 172.21.15.84:0/3455979934 wait complete. 2023-10-16T09:50:06.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.170+0000 7f2f1bfff640 1 Processor -- start 2023-10-16T09:50:06.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.170+0000 7f2f1bfff640 1 -- start start 2023-10-16T09:50:06.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.171+0000 7f2f1bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f080047c0 0x7f2f1c0b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:06.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.171+0000 7f2f1bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f1c0b12c0 0x7f2f1c1472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:06.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.171+0000 7f2f1bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f1c0c8050 0x7f2f1c157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:06.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.171+0000 7f2f1bfff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0b1980 con 0x7f2f1c0b12c0 2023-10-16T09:50:06.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.171+0000 7f2f1bfff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c147830 con 0x7f2f080047c0 2023-10-16T09:50:06.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.171+0000 7f2f1bfff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0b1720 con 0x7f2f1c0c8050 2023-10-16T09:50:06.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.171+0000 7f2f127fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f1c0b12c0 0x7f2f1c1472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:06.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.171+0000 7f2f127fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f1c0b12c0 0x7f2f1c1472f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46888/0 (socket says 172.21.15.84:46888) 2023-10-16T09:50:06.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.171+0000 7f2f127fc640 1 -- 172.21.15.84:0/468110911 learned_addr learned my addr 172.21.15.84:0/468110911 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:06.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.171+0000 7f2f1b7fe640 1 --2- 172.21.15.84:0/468110911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f1c0c8050 0x7f2f1c157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:06.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.171+0000 7f2f1affd640 1 --2- 172.21.15.84:0/468110911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f080047c0 0x7f2f1c0b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:06.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.171+0000 7f2f127fc640 1 -- 172.21.15.84:0/468110911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f1c0c8050 msgr2=0x7f2f1c157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:06.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.171+0000 7f2f127fc640 1 --2- 172.21.15.84:0/468110911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f1c0c8050 0x7f2f1c157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.171+0000 7f2f127fc640 1 -- 172.21.15.84:0/468110911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f080047c0 msgr2=0x7f2f1c0b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:06.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.171+0000 7f2f127fc640 1 --2- 172.21.15.84:0/468110911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f080047c0 0x7f2f1c0b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.172+0000 7f2f127fc640 1 -- 172.21.15.84:0/468110911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c0b30e0 con 0x7f2f1c0b12c0 2023-10-16T09:50:06.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.172+0000 7f2f127fc640 1 --2- 172.21.15.84:0/468110911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f1c0b12c0 0x7f2f1c1472f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2f04005040 tx=0x7f2f04001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:06.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.172+0000 7f2f13fff640 1 -- 172.21.15.84:0/468110911 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f040054c0 con 0x7f2f1c0b12c0 2023-10-16T09:50:06.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.172+0000 7f2f13fff640 1 -- 172.21.15.84:0/468110911 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f040069e0 con 0x7f2f1c0b12c0 2023-10-16T09:50:06.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.172+0000 7f2f1bfff640 1 -- 172.21.15.84:0/468110911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1c0b33c0 con 0x7f2f1c0b12c0 2023-10-16T09:50:06.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.172+0000 7f2f13fff640 1 -- 172.21.15.84:0/468110911 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04006db0 con 0x7f2f1c0b12c0 2023-10-16T09:50:06.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.172+0000 7f2f1bfff640 1 -- 172.21.15.84:0/468110911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1c0b3980 con 0x7f2f1c0b12c0 2023-10-16T09:50:06.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.173+0000 7f2f1bfff640 1 -- 172.21.15.84:0/468110911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1c04e9b0 con 0x7f2f1c0b12c0 2023-10-16T09:50:06.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.174+0000 7f2f13fff640 1 -- 172.21.15.84:0/468110911 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2f040035f0 con 0x7f2f1c0b12c0 2023-10-16T09:50:06.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.175+0000 7f2f13fff640 1 --2- 172.21.15.84:0/468110911 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2efc06ae70 0x7f2efc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:06.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.175+0000 7f2f13fff640 1 -- 172.21.15.84:0/468110911 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2f040ab0e0 con 0x7f2f1c0b12c0 2023-10-16T09:50:06.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.175+0000 7f2f1affd640 1 --2- 172.21.15.84:0/468110911 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2efc06ae70 0x7f2efc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:06.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.176+0000 7f2f1affd640 1 --2- 172.21.15.84:0/468110911 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2efc06ae70 0x7f2efc06d330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c063ab0 tx=0x7f2f0c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:06.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.179+0000 7f2f13fff640 1 -- 172.21.15.84:0/468110911 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2f040763b0 con 0x7f2f1c0b12c0 2023-10-16T09:50:06.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.439+0000 7f2f1bfff640 1 -- 172.21.15.84:0/468110911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f2f1c06c5c0 con 0x7f2f1c0b12c0 2023-10-16T09:50:06.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.443+0000 7f2f13fff640 1 -- 172.21.15.84:0/468110911 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2f040799d0 con 0x7f2f1c0b12c0 2023-10-16T09:50:06.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.445+0000 7f2f1bfff640 1 -- 172.21.15.84:0/468110911 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2efc06ae70 msgr2=0x7f2efc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:06.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.445+0000 7f2f1bfff640 1 --2- 172.21.15.84:0/468110911 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2efc06ae70 0x7f2efc06d330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c063ab0 tx=0x7f2f0c008040 comp rx=0 tx=0).stop 2023-10-16T09:50:06.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.445+0000 7f2f1bfff640 1 -- 172.21.15.84:0/468110911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f1c0b12c0 msgr2=0x7f2f1c1472f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:06.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.445+0000 7f2f1bfff640 1 --2- 172.21.15.84:0/468110911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f1c0b12c0 0x7f2f1c1472f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2f04005040 tx=0x7f2f04001120 comp rx=0 tx=0).stop 2023-10-16T09:50:06.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.445+0000 7f2f1bfff640 1 -- 172.21.15.84:0/468110911 shutdown_connections 2023-10-16T09:50:06.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.445+0000 7f2f1bfff640 1 --2- 172.21.15.84:0/468110911 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f2efc06ae70 0x7f2efc06d330 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.445+0000 7f2f1bfff640 1 --2- 172.21.15.84:0/468110911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f1c0c8050 0x7f2f1c157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.445+0000 7f2f1bfff640 1 --2- 172.21.15.84:0/468110911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f1c0b12c0 0x7f2f1c1472f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.445+0000 7f2f1bfff640 1 --2- 172.21.15.84:0/468110911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f080047c0 0x7f2f1c0b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.445+0000 7f2f1bfff640 1 -- 172.21.15.84:0/468110911 >> 172.21.15.84:0/468110911 conn(0x7f2f1c0ce6a0 msgr2=0x7f2f1c06ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:06.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.446+0000 7f2f1bfff640 1 -- 172.21.15.84:0/468110911 shutdown_connections 2023-10-16T09:50:06.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.446+0000 7f2f1bfff640 1 -- 172.21.15.84:0/468110911 wait complete. 2023-10-16T09:50:06.465 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-16T09:50:06.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.680+0000 7fb576675640 1 Processor -- start 2023-10-16T09:50:06.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.680+0000 7fb576675640 1 -- start start 2023-10-16T09:50:06.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.681+0000 7fb576675640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb570165e90 0x7fb5700cb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:06.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.681+0000 7fb576675640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb570065ab0 0x7fb570065f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:06.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.681+0000 7fb576675640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5700cd3a0 con 0x7fb5700ccfc0 2023-10-16T09:50:06.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.681+0000 7fb576675640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb57006cee0 con 0x7fb570065ab0 2023-10-16T09:50:06.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.681+0000 7fb576675640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5700ceb30 con 0x7fb570165e90 2023-10-16T09:50:06.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.682+0000 7fb56ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb5700ccfc0 0x7fb57006c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48572/0 (socket says 172.21.15.84:48572) 2023-10-16T09:50:06.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.682+0000 7fb56ffff640 1 -- 172.21.15.84:0/4013809033 learned_addr learned my addr 172.21.15.84:0/4013809033 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:06.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.682+0000 7fb574c12640 1 --2- 172.21.15.84:0/4013809033 >> v2:172.21.15.84:3300/0 conn(0x7fb570065ab0 0x7fb570065f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:06.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.682+0000 7fb56f7fe640 1 --2- 172.21.15.84:0/4013809033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb570165e90 0x7fb5700cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:06.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.682+0000 7fb56effd640 1 -- 172.21.15.84:0/4013809033 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26624379 0 0) 0x7fb5700cd3a0 con 0x7fb5700ccfc0 2023-10-16T09:50:06.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.682+0000 7fb56effd640 1 -- 172.21.15.84:0/4013809033 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb55c003520 con 0x7fb5700ccfc0 2023-10-16T09:50:06.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.682+0000 7fb574c12640 1 -- 172.21.15.84:0/4013809033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb570165e90 msgr2=0x7fb5700cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:06.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.682+0000 7fb574c12640 1 --2- 172.21.15.84:0/4013809033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb570165e90 0x7fb5700cb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.682+0000 7fb574c12640 1 -- 172.21.15.84:0/4013809033 >> v1:172.21.15.84:6789/0 conn(0x7fb5700ccfc0 legacy=0x7fb57006c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:06.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.682+0000 7fb574c12640 1 -- 172.21.15.84:0/4013809033 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb570066450 con 0x7fb570065ab0 2023-10-16T09:50:06.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.683+0000 7fb574c12640 1 --2- 172.21.15.84:0/4013809033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb570065ab0 0x7fb570065f10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb5680097a0 tx=0x7fb56802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40ef630b451030d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:06.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.684+0000 7fb56effd640 1 -- 172.21.15.84:0/4013809033 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56802d040 con 0x7fb570065ab0 2023-10-16T09:50:06.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.684+0000 7fb56effd640 1 -- 172.21.15.84:0/4013809033 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb568031810 con 0x7fb570065ab0 2023-10-16T09:50:06.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.684+0000 7fb56effd640 1 -- 172.21.15.84:0/4013809033 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568031ba0 con 0x7fb570065ab0 2023-10-16T09:50:06.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.684+0000 7fb576675640 1 -- 172.21.15.84:0/4013809033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb570065ab0 msgr2=0x7fb570065f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:06.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.684+0000 7fb576675640 1 --2- 172.21.15.84:0/4013809033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb570065ab0 0x7fb570065f10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb5680097a0 tx=0x7fb56802b8b0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.684+0000 7fb576675640 1 -- 172.21.15.84:0/4013809033 shutdown_connections 2023-10-16T09:50:06.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.684+0000 7fb576675640 1 --2- 172.21.15.84:0/4013809033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb570065ab0 0x7fb570065f10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.684+0000 7fb576675640 1 --2- 172.21.15.84:0/4013809033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb570165e90 0x7fb5700cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.684+0000 7fb576675640 1 -- 172.21.15.84:0/4013809033 >> 172.21.15.84:0/4013809033 conn(0x7fb5700d0ab0 msgr2=0x7fb5700d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:06.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.684+0000 7fb576675640 1 -- 172.21.15.84:0/4013809033 shutdown_connections 2023-10-16T09:50:06.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.684+0000 7fb576675640 1 -- 172.21.15.84:0/4013809033 wait complete. 2023-10-16T09:50:06.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.686+0000 7fb576675640 1 Processor -- start 2023-10-16T09:50:06.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.686+0000 7fb576675640 1 -- start start 2023-10-16T09:50:06.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.687+0000 7fb576675640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb570065ab0 0x7fb5701f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:06.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.687+0000 7fb576675640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5700ccfc0 0x7fb5700b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:06.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.687+0000 7fb576675640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5700b4620 0x7fb570157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:06.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.687+0000 7fb576675640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5700b4ba0 con 0x7fb5700b4620 2023-10-16T09:50:06.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.687+0000 7fb576675640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5700b5950 con 0x7fb570065ab0 2023-10-16T09:50:06.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.687+0000 7fb576675640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5700b5020 con 0x7fb5700ccfc0 2023-10-16T09:50:06.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.688+0000 7fb574c12640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5700b4620 0x7fb570157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:06.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.688+0000 7fb574c12640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5700b4620 0x7fb570157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46906/0 (socket says 172.21.15.84:46906) 2023-10-16T09:50:06.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.688+0000 7fb574c12640 1 -- 172.21.15.84:0/2786663284 learned_addr learned my addr 172.21.15.84:0/2786663284 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:06.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.688+0000 7fb56ffff640 1 --2- 172.21.15.84:0/2786663284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb570065ab0 0x7fb5701f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:06.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.688+0000 7fb56f7fe640 1 --2- 172.21.15.84:0/2786663284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5700ccfc0 0x7fb5700b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:06.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.688+0000 7fb574c12640 1 -- 172.21.15.84:0/2786663284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5700ccfc0 msgr2=0x7fb5700b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:06.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.688+0000 7fb574c12640 1 --2- 172.21.15.84:0/2786663284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5700ccfc0 0x7fb5700b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.688+0000 7fb574c12640 1 -- 172.21.15.84:0/2786663284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb570065ab0 msgr2=0x7fb5701f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:06.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.688+0000 7fb574c12640 1 --2- 172.21.15.84:0/2786663284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb570065ab0 0x7fb5701f6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.688+0000 7fb574c12640 1 -- 172.21.15.84:0/2786663284 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb568009400 con 0x7fb5700b4620 2023-10-16T09:50:06.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.688+0000 7fb56ffff640 1 --2- 172.21.15.84:0/2786663284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb570065ab0 0x7fb5701f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:06.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.688+0000 7fb574c12640 1 --2- 172.21.15.84:0/2786663284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5700b4620 0x7fb570157fb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb5680097a0 tx=0x7fb5680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:06.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.689+0000 7fb56cff9640 1 -- 172.21.15.84:0/2786663284 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568006e10 con 0x7fb5700b4620 2023-10-16T09:50:06.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.689+0000 7fb56cff9640 1 -- 172.21.15.84:0/2786663284 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56803ace0 con 0x7fb5700b4620 2023-10-16T09:50:06.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.689+0000 7fb56cff9640 1 -- 172.21.15.84:0/2786663284 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568044640 con 0x7fb5700b4620 2023-10-16T09:50:06.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.689+0000 7fb576675640 1 -- 172.21.15.84:0/2786663284 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5700b5300 con 0x7fb5700b4620 2023-10-16T09:50:06.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.689+0000 7fb576675640 1 -- 172.21.15.84:0/2786663284 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5701535d0 con 0x7fb5700b4620 2023-10-16T09:50:06.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.690+0000 7fb576675640 1 -- 172.21.15.84:0/2786663284 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5700c40e0 con 0x7fb5700b4620 2023-10-16T09:50:06.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.691+0000 7fb56cff9640 1 -- 172.21.15.84:0/2786663284 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb56804d440 con 0x7fb5700b4620 2023-10-16T09:50:06.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.692+0000 7fb56cff9640 1 --2- 172.21.15.84:0/2786663284 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb55006ae70 0x7fb55006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:06.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.692+0000 7fb56cff9640 1 -- 172.21.15.84:0/2786663284 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb5680ab500 con 0x7fb5700b4620 2023-10-16T09:50:06.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.692+0000 7fb56ffff640 1 --2- 172.21.15.84:0/2786663284 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb55006ae70 0x7fb55006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:06.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.693+0000 7fb56ffff640 1 --2- 172.21.15.84:0/2786663284 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb55006ae70 0x7fb55006d330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb560001560 tx=0x7fb560009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:06.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.696+0000 7fb56cff9640 1 -- 172.21.15.84:0/2786663284 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb5680768f0 con 0x7fb5700b4620 2023-10-16T09:50:06.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.954+0000 7fb576675640 1 -- 172.21.15.84:0/2786663284 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fb57004e160 con 0x7fb5700b4620 2023-10-16T09:50:06.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.958+0000 7fb56cff9640 1 -- 172.21.15.84:0/2786663284 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb568079f10 con 0x7fb5700b4620 2023-10-16T09:50:06.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.960+0000 7fb576675640 1 -- 172.21.15.84:0/2786663284 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb55006ae70 msgr2=0x7fb55006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:06.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.960+0000 7fb576675640 1 --2- 172.21.15.84:0/2786663284 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb55006ae70 0x7fb55006d330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb560001560 tx=0x7fb560009040 comp rx=0 tx=0).stop 2023-10-16T09:50:06.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.960+0000 7fb576675640 1 -- 172.21.15.84:0/2786663284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5700b4620 msgr2=0x7fb570157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:06.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.960+0000 7fb576675640 1 --2- 172.21.15.84:0/2786663284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5700b4620 0x7fb570157fb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb5680097a0 tx=0x7fb5680046c0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.960+0000 7fb576675640 1 -- 172.21.15.84:0/2786663284 shutdown_connections 2023-10-16T09:50:06.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.960+0000 7fb576675640 1 --2- 172.21.15.84:0/2786663284 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fb55006ae70 0x7fb55006d330 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.960+0000 7fb576675640 1 --2- 172.21.15.84:0/2786663284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5700b4620 0x7fb570157fb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.960+0000 7fb576675640 1 --2- 172.21.15.84:0/2786663284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5700ccfc0 0x7fb5700b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.960+0000 7fb576675640 1 --2- 172.21.15.84:0/2786663284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb570065ab0 0x7fb5701f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:06.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.960+0000 7fb576675640 1 -- 172.21.15.84:0/2786663284 >> 172.21.15.84:0/2786663284 conn(0x7fb5700d0ab0 msgr2=0x7fb5700cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:06.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.961+0000 7fb576675640 1 -- 172.21.15.84:0/2786663284 shutdown_connections 2023-10-16T09:50:06.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:06.961+0000 7fb576675640 1 -- 172.21.15.84:0/2786663284 wait complete. 2023-10-16T09:50:06.977 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-16T09:50:07.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.180+0000 7fcc0a3fb640 1 Processor -- start 2023-10-16T09:50:07.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.180+0000 7fcc0a3fb640 1 -- start start 2023-10-16T09:50:07.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.182+0000 7fcc0a3fb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcc0415d150 0x7fcc040ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:07.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.182+0000 7fcc0a3fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc0406a4b0 0x7fcc04160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:07.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.182+0000 7fcc0a3fb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc040ac890 con 0x7fcc04160b90 2023-10-16T09:50:07.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.182+0000 7fcc0a3fb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0415d530 con 0x7fcc0415d150 2023-10-16T09:50:07.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.182+0000 7fcc0a3fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc04161650 con 0x7fcc0406a4b0 2023-10-16T09:50:07.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.183+0000 7fcbfbfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fcc04160b90 0x7fcc04069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48578/0 (socket says 172.21.15.84:48578) 2023-10-16T09:50:07.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.183+0000 7fcbfbfff640 1 -- 172.21.15.84:0/2595637144 learned_addr learned my addr 172.21.15.84:0/2595637144 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:07.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.183+0000 7fcc03fff640 1 --2- 172.21.15.84:0/2595637144 >> v2:172.21.15.157:3300/0 conn(0x7fcc0415d150 0x7fcc040ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:07.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.183+0000 7fcc08998640 1 --2- 172.21.15.84:0/2595637144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc0406a4b0 0x7fcc04160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:07.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.183+0000 7fcc037fe640 1 -- 172.21.15.84:0/2595637144 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351324601 0 0) 0x7fcc040ac890 con 0x7fcc04160b90 2023-10-16T09:50:07.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.183+0000 7fcc037fe640 1 -- 172.21.15.84:0/2595637144 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbf0003520 con 0x7fcc04160b90 2023-10-16T09:50:07.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.183+0000 7fcc037fe640 1 -- 172.21.15.84:0/2595637144 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2856814579 0 0) 0x7fcbf0003520 con 0x7fcc04160b90 2023-10-16T09:50:07.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.183+0000 7fcc037fe640 1 -- 172.21.15.84:0/2595637144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc0406a4b0 msgr2=0x7fcc04160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:07.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.183+0000 7fcc037fe640 1 --2- 172.21.15.84:0/2595637144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc0406a4b0 0x7fcc04160fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.183+0000 7fcc037fe640 1 -- 172.21.15.84:0/2595637144 >> v2:172.21.15.157:3300/0 conn(0x7fcc0415d150 msgr2=0x7fcc040ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:07.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.183+0000 7fcc037fe640 1 --2- 172.21.15.84:0/2595637144 >> v2:172.21.15.157:3300/0 conn(0x7fcc0415d150 0x7fcc040ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.183+0000 7fcc037fe640 1 -- 172.21.15.84:0/2595637144 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc04161930 con 0x7fcc04160b90 2023-10-16T09:50:07.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.183+0000 7fcc037fe640 1 -- 172.21.15.84:0/2595637144 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fcbec001670 con 0x7fcc04160b90 2023-10-16T09:50:07.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.184+0000 7fcc037fe640 1 -- 172.21.15.84:0/2595637144 >> v1:172.21.15.84:6789/0 conn(0x7fcc04160b90 legacy=0x7fcc04069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:07.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.184+0000 7fcc037fe640 1 --2- 172.21.15.84:0/2595637144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc0406a4b0 0x7fcbf0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:07.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.184+0000 7fcc037fe640 1 --2- 172.21.15.84:0/2595637144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbf0003ef0 0x7fcbf00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:07.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.184+0000 7fcc037fe640 1 --2- 172.21.15.84:0/2595637144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf0004d00 0x7fcbf0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:07.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.184+0000 7fcc037fe640 1 -- 172.21.15.84:0/2595637144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0415d530 con 0x7fcbf0003ef0 2023-10-16T09:50:07.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.184+0000 7fcc037fe640 1 -- 172.21.15.84:0/2595637144 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc04161650 con 0x7fcbf0004d00 2023-10-16T09:50:07.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.184+0000 7fcc037fe640 1 -- 172.21.15.84:0/2595637144 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf000b9d0 con 0x7fcc0406a4b0 2023-10-16T09:50:07.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.184+0000 7fcbfbfff640 1 --2- 172.21.15.84:0/2595637144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbf0003ef0 0x7fcbf00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:07.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.184+0000 7fcc03fff640 1 --2- 172.21.15.84:0/2595637144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc0406a4b0 0x7fcbf0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:07.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.184+0000 7fcc08998640 1 --2- 172.21.15.84:0/2595637144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf0004d00 0x7fcbf0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:07.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.184+0000 7fcbfbfff640 1 -- 172.21.15.84:0/2595637144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc0406a4b0 msgr2=0x7fcbf0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:07.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.184+0000 7fcbfbfff640 1 --2- 172.21.15.84:0/2595637144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc0406a4b0 0x7fcbf0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.184+0000 7fcbfbfff640 1 -- 172.21.15.84:0/2595637144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf0004d00 msgr2=0x7fcbf0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:07.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.184+0000 7fcbfbfff640 1 --2- 172.21.15.84:0/2595637144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf0004d00 0x7fcbf0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.184+0000 7fcbfbfff640 1 -- 172.21.15.84:0/2595637144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf0003520 con 0x7fcbf0003ef0 2023-10-16T09:50:07.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.186+0000 7fcbfbfff640 1 --2- 172.21.15.84:0/2595637144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbf0003ef0 0x7fcbf00047c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00c270 tx=0x7fcbf00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d83ed68cad5b65 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:07.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.186+0000 7fcc037fe640 1 -- 172.21.15.84:0/2595637144 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec02cae0 con 0x7fcbf0003ef0 2023-10-16T09:50:07.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.186+0000 7fcc037fe640 1 -- 172.21.15.84:0/2595637144 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbec02cc40 con 0x7fcbf0003ef0 2023-10-16T09:50:07.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.186+0000 7fcc037fe640 1 -- 172.21.15.84:0/2595637144 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec005af0 con 0x7fcbf0003ef0 2023-10-16T09:50:07.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.186+0000 7fcc0a3fb640 1 -- 172.21.15.84:0/2595637144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbf0003ef0 msgr2=0x7fcbf00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:07.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.186+0000 7fcc0a3fb640 1 --2- 172.21.15.84:0/2595637144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbf0003ef0 0x7fcbf00047c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00c270 tx=0x7fcbf00057f0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.186+0000 7fcc08998640 1 -- 172.21.15.84:0/2595637144 reap_dead start 2023-10-16T09:50:07.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.186+0000 7fcc0a3fb640 1 -- 172.21.15.84:0/2595637144 shutdown_connections 2023-10-16T09:50:07.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.187+0000 7fcc0a3fb640 1 -- 172.21.15.84:0/2595637144 >> 172.21.15.84:0/2595637144 conn(0x7fcc040ce9b0 msgr2=0x7fcc040cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:07.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.187+0000 7fcc0a3fb640 1 -- 172.21.15.84:0/2595637144 shutdown_connections 2023-10-16T09:50:07.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.187+0000 7fcc0a3fb640 1 -- 172.21.15.84:0/2595637144 wait complete. 2023-10-16T09:50:07.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.188+0000 7fcc0a3fb640 1 Processor -- start 2023-10-16T09:50:07.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.188+0000 7fcc0a3fb640 1 -- start start 2023-10-16T09:50:07.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.189+0000 7fcc0a3fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf0003ef0 0x7fcc04154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:07.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.189+0000 7fcc0a3fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc040b0ae0 0x7fcc040b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:07.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.189+0000 7fcc0a3fb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc040b3f30 0x7fcc041f6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:07.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.189+0000 7fcc0a3fb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040c4ec0 con 0x7fcc040b3f30 2023-10-16T09:50:07.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.189+0000 7fcc0a3fb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040b22b0 con 0x7fcbf0003ef0 2023-10-16T09:50:07.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.189+0000 7fcc0a3fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040c42b0 con 0x7fcc040b0ae0 2023-10-16T09:50:07.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.190+0000 7fcc08998640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc040b3f30 0x7fcc041f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:07.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.190+0000 7fcc08998640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc040b3f30 0x7fcc041f6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46926/0 (socket says 172.21.15.84:46926) 2023-10-16T09:50:07.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.190+0000 7fcc08998640 1 -- 172.21.15.84:0/438684957 learned_addr learned my addr 172.21.15.84:0/438684957 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:07.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.190+0000 7fcc03fff640 1 --2- 172.21.15.84:0/438684957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf0003ef0 0x7fcc04154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:07.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.190+0000 7fcbfbfff640 1 --2- 172.21.15.84:0/438684957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc040b0ae0 0x7fcc040b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:07.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.190+0000 7fcc08998640 1 -- 172.21.15.84:0/438684957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc040b0ae0 msgr2=0x7fcc040b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:07.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.190+0000 7fcc08998640 1 --2- 172.21.15.84:0/438684957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc040b0ae0 0x7fcc040b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.190+0000 7fcc08998640 1 -- 172.21.15.84:0/438684957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf0003ef0 msgr2=0x7fcc04154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:07.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.190+0000 7fcc08998640 1 --2- 172.21.15.84:0/438684957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf0003ef0 0x7fcc04154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.190+0000 7fcc08998640 1 -- 172.21.15.84:0/438684957 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040c4590 con 0x7fcc040b3f30 2023-10-16T09:50:07.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.191+0000 7fcc08998640 1 --2- 172.21.15.84:0/438684957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc040b3f30 0x7fcc041f6150 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcc040c8f90 tx=0x7fcbfc00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:07.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.191+0000 7fcc017fa640 1 -- 172.21.15.84:0/438684957 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc017400 con 0x7fcc040b3f30 2023-10-16T09:50:07.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.191+0000 7fcc017fa640 1 -- 172.21.15.84:0/438684957 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc011040 con 0x7fcc040b3f30 2023-10-16T09:50:07.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.191+0000 7fcc017fa640 1 -- 172.21.15.84:0/438684957 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc0177c0 con 0x7fcc040b3f30 2023-10-16T09:50:07.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.191+0000 7fcc0a3fb640 1 -- 172.21.15.84:0/438684957 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc040c4870 con 0x7fcc040b3f30 2023-10-16T09:50:07.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.191+0000 7fcc0a3fb640 1 -- 172.21.15.84:0/438684957 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc040b43b0 con 0x7fcc040b3f30 2023-10-16T09:50:07.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.192+0000 7fcc0a3fb640 1 -- 172.21.15.84:0/438684957 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0404e160 con 0x7fcc040b3f30 2023-10-16T09:50:07.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.193+0000 7fcc017fa640 1 -- 172.21.15.84:0/438684957 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcbfc017920 con 0x7fcc040b3f30 2023-10-16T09:50:07.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.194+0000 7fcc017fa640 1 --2- 172.21.15.84:0/438684957 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fcbe406ae70 0x7fcbe406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:07.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.194+0000 7fcc017fa640 1 -- 172.21.15.84:0/438684957 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcbfc0897f0 con 0x7fcc040b3f30 2023-10-16T09:50:07.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.194+0000 7fcc03fff640 1 --2- 172.21.15.84:0/438684957 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fcbe406ae70 0x7fcbe406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:07.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.195+0000 7fcc03fff640 1 --2- 172.21.15.84:0/438684957 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fcbe406ae70 0x7fcbe406d330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009a10 tx=0x7fcbf4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:07.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.199+0000 7fcc017fa640 1 -- 172.21.15.84:0/438684957 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcbfc054b50 con 0x7fcc040b3f30 2023-10-16T09:50:07.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.460+0000 7fcc0a3fb640 1 -- 172.21.15.84:0/438684957 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fcc040cdbf0 con 0x7fcc040b3f30 2023-10-16T09:50:07.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.464+0000 7fcc017fa640 1 -- 172.21.15.84:0/438684957 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fcbfc058170 con 0x7fcc040b3f30 2023-10-16T09:50:07.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.467+0000 7fcc0a3fb640 1 -- 172.21.15.84:0/438684957 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fcbe406ae70 msgr2=0x7fcbe406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:07.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.467+0000 7fcc0a3fb640 1 --2- 172.21.15.84:0/438684957 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fcbe406ae70 0x7fcbe406d330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009a10 tx=0x7fcbf4008040 comp rx=0 tx=0).stop 2023-10-16T09:50:07.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.467+0000 7fcc0a3fb640 1 -- 172.21.15.84:0/438684957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc040b3f30 msgr2=0x7fcc041f6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:07.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.467+0000 7fcc0a3fb640 1 --2- 172.21.15.84:0/438684957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc040b3f30 0x7fcc041f6150 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcc040c8f90 tx=0x7fcbfc00f4a0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.467+0000 7fcc0a3fb640 1 -- 172.21.15.84:0/438684957 shutdown_connections 2023-10-16T09:50:07.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.467+0000 7fcc0a3fb640 1 --2- 172.21.15.84:0/438684957 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fcbe406ae70 0x7fcbe406d330 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.467+0000 7fcc0a3fb640 1 --2- 172.21.15.84:0/438684957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc040b3f30 0x7fcc041f6150 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.467+0000 7fcc0a3fb640 1 --2- 172.21.15.84:0/438684957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc040b0ae0 0x7fcc040b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.467+0000 7fcc0a3fb640 1 --2- 172.21.15.84:0/438684957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf0003ef0 0x7fcc04154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.467+0000 7fcc0a3fb640 1 -- 172.21.15.84:0/438684957 >> 172.21.15.84:0/438684957 conn(0x7fcc040ce9b0 msgr2=0x7fcc040c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:07.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.467+0000 7fcc0a3fb640 1 -- 172.21.15.84:0/438684957 shutdown_connections 2023-10-16T09:50:07.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.467+0000 7fcc0a3fb640 1 -- 172.21.15.84:0/438684957 wait complete. 2023-10-16T09:50:07.487 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-16T09:50:07.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.692+0000 7f4886b4e640 1 Processor -- start 2023-10-16T09:50:07.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.693+0000 7f4886b4e640 1 -- start start 2023-10-16T09:50:07.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.694+0000 7f4886b4e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f488015d030 0x7f48800d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:07.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.694+0000 7f4886b4e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f488015d410 0x7f4880161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:07.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.694+0000 7f4886b4e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48800b5d90 con 0x7f4880161990 2023-10-16T09:50:07.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.694+0000 7f4886b4e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48800d2b50 con 0x7f488015d030 2023-10-16T09:50:07.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.694+0000 7f4886b4e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4880160fc0 con 0x7f488015d410 2023-10-16T09:50:07.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.695+0000 7f487ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f488015d410 0x7f4880161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:07.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.695+0000 7f48848ea640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f488015d030 0x7f48800d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:07.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.695+0000 7f487ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f488015d410 0x7f4880161450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:46292/0 (socket says 172.21.15.84:46292) 2023-10-16T09:50:07.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.695+0000 7f487ffff640 1 -- 172.21.15.84:0/2782090432 learned_addr learned my addr 172.21.15.84:0/2782090432 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:07.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.696+0000 7f487ffff640 1 -- 172.21.15.84:0/2782090432 >> v2:172.21.15.157:3300/0 conn(0x7f488015d030 msgr2=0x7f48800d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:07.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.696+0000 7f487ffff640 1 --2- 172.21.15.84:0/2782090432 >> v2:172.21.15.157:3300/0 conn(0x7f488015d030 0x7f48800d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.696+0000 7f487ffff640 1 -- 172.21.15.84:0/2782090432 >> v1:172.21.15.157:6789/0 conn(0x7f4880161990 legacy=0x7f48800cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:07.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.696+0000 7f487ffff640 1 -- 172.21.15.84:0/2782090432 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48800d25d0 con 0x7f488015d410 2023-10-16T09:50:07.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.697+0000 7f487ffff640 1 --2- 172.21.15.84:0/2782090432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f488015d410 0x7f4880161450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4870009870 tx=0x7f487002ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1008782e77f2db0a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:07.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.697+0000 7f487f7fe640 1 -- 172.21.15.84:0/2782090432 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487002f840 con 0x7f488015d410 2023-10-16T09:50:07.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.698+0000 7f487f7fe640 1 -- 172.21.15.84:0/2782090432 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487002f9a0 con 0x7f488015d410 2023-10-16T09:50:07.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.698+0000 7f487f7fe640 1 -- 172.21.15.84:0/2782090432 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487002fd30 con 0x7f488015d410 2023-10-16T09:50:07.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.698+0000 7f4886b4e640 1 -- 172.21.15.84:0/2782090432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f488015d410 msgr2=0x7f4880161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:07.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.698+0000 7f4886b4e640 1 --2- 172.21.15.84:0/2782090432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f488015d410 0x7f4880161450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4870009870 tx=0x7f487002ba20 comp rx=0 tx=0).stop 2023-10-16T09:50:07.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.698+0000 7f4886b4e640 1 -- 172.21.15.84:0/2782090432 shutdown_connections 2023-10-16T09:50:07.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.698+0000 7f4886b4e640 1 --2- 172.21.15.84:0/2782090432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f488015d410 0x7f4880161450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.698+0000 7f4886b4e640 1 --2- 172.21.15.84:0/2782090432 >> v2:172.21.15.157:3300/0 conn(0x7f488015d030 0x7f48800d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.698+0000 7f4886b4e640 1 -- 172.21.15.84:0/2782090432 >> 172.21.15.84:0/2782090432 conn(0x7f48800cf720 msgr2=0x7f48800cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:07.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.699+0000 7f4886b4e640 1 -- 172.21.15.84:0/2782090432 shutdown_connections 2023-10-16T09:50:07.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.699+0000 7f4886b4e640 1 -- 172.21.15.84:0/2782090432 wait complete. 2023-10-16T09:50:07.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.701+0000 7f4886b4e640 1 Processor -- start 2023-10-16T09:50:07.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.701+0000 7f4886b4e640 1 -- start start 2023-10-16T09:50:07.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.702+0000 7f4886b4e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488015d030 0x7f48801f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:07.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.702+0000 7f4886b4e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f488015d410 0x7f48801583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:07.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.702+0000 7f4886b4e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4880161990 0x7f488015d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:07.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.702+0000 7f4886b4e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48801f6ea0 con 0x7f4880161990 2023-10-16T09:50:07.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.702+0000 7f4886b4e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48800c9f10 con 0x7f488015d030 2023-10-16T09:50:07.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.702+0000 7f4886b4e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48801f6600 con 0x7f488015d410 2023-10-16T09:50:07.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.703+0000 7f48850eb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4880161990 0x7f488015d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:07.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.703+0000 7f48850eb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4880161990 0x7f488015d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46930/0 (socket says 172.21.15.84:46930) 2023-10-16T09:50:07.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.703+0000 7f48850eb640 1 -- 172.21.15.84:0/1960651180 learned_addr learned my addr 172.21.15.84:0/1960651180 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:07.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.703+0000 7f487ffff640 1 --2- 172.21.15.84:0/1960651180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f488015d410 0x7f48801583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:07.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.703+0000 7f48848ea640 1 --2- 172.21.15.84:0/1960651180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488015d030 0x7f48801f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:07.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.703+0000 7f48850eb640 1 -- 172.21.15.84:0/1960651180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f488015d410 msgr2=0x7f48801583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:07.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.703+0000 7f48850eb640 1 --2- 172.21.15.84:0/1960651180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f488015d410 0x7f48801583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.703+0000 7f48850eb640 1 -- 172.21.15.84:0/1960651180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488015d030 msgr2=0x7f48801f5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:07.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.703+0000 7f48850eb640 1 --2- 172.21.15.84:0/1960651180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488015d030 0x7f48801f5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.703+0000 7f48850eb640 1 -- 172.21.15.84:0/1960651180 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48700094d0 con 0x7f4880161990 2023-10-16T09:50:07.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.703+0000 7f48850eb640 1 --2- 172.21.15.84:0/1960651180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4880161990 0x7f488015d890 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f487800c6e0 tx=0x7f487800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:07.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.704+0000 7f487d7fa640 1 -- 172.21.15.84:0/1960651180 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4878003b70 con 0x7f4880161990 2023-10-16T09:50:07.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.704+0000 7f487d7fa640 1 -- 172.21.15.84:0/1960651180 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4878003cf0 con 0x7f4880161990 2023-10-16T09:50:07.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.704+0000 7f487d7fa640 1 -- 172.21.15.84:0/1960651180 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4878005310 con 0x7f4880161990 2023-10-16T09:50:07.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.704+0000 7f4886b4e640 1 -- 172.21.15.84:0/1960651180 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48801f68e0 con 0x7f4880161990 2023-10-16T09:50:07.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.704+0000 7f4886b4e640 1 -- 172.21.15.84:0/1960651180 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48800cc870 con 0x7f4880161990 2023-10-16T09:50:07.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.705+0000 7f4886b4e640 1 -- 172.21.15.84:0/1960651180 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48800c9ce0 con 0x7f4880161990 2023-10-16T09:50:07.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.706+0000 7f487d7fa640 1 -- 172.21.15.84:0/1960651180 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4878005470 con 0x7f4880161990 2023-10-16T09:50:07.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.707+0000 7f487d7fa640 1 --2- 172.21.15.84:0/1960651180 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f486c06aea0 0x7f486c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:07.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.707+0000 7f487d7fa640 1 -- 172.21.15.84:0/1960651180 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4878088f80 con 0x7f4880161990 2023-10-16T09:50:07.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.707+0000 7f48848ea640 1 --2- 172.21.15.84:0/1960651180 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f486c06aea0 0x7f486c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:07.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.708+0000 7f48848ea640 1 --2- 172.21.15.84:0/1960651180 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f486c06aea0 0x7f486c06d360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4874009ad0 tx=0x7f4874008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:07.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.712+0000 7f487d7fa640 1 -- 172.21.15.84:0/1960651180 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f48780bbfe0 con 0x7f4880161990 2023-10-16T09:50:07.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.969+0000 7f4886b4e640 1 -- 172.21.15.84:0/1960651180 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f488004e160 con 0x7f4880161990 2023-10-16T09:50:07.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.973+0000 7f487d7fa640 1 -- 172.21.15.84:0/1960651180 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4878003e60 con 0x7f4880161990 2023-10-16T09:50:07.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.975+0000 7f4886b4e640 1 -- 172.21.15.84:0/1960651180 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f486c06aea0 msgr2=0x7f486c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:07.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.975+0000 7f4886b4e640 1 --2- 172.21.15.84:0/1960651180 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f486c06aea0 0x7f486c06d360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4874009ad0 tx=0x7f4874008040 comp rx=0 tx=0).stop 2023-10-16T09:50:07.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.975+0000 7f4886b4e640 1 -- 172.21.15.84:0/1960651180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4880161990 msgr2=0x7f488015d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:07.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.975+0000 7f4886b4e640 1 --2- 172.21.15.84:0/1960651180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4880161990 0x7f488015d890 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f487800c6e0 tx=0x7f487800cba0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.975+0000 7f4886b4e640 1 -- 172.21.15.84:0/1960651180 shutdown_connections 2023-10-16T09:50:07.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.975+0000 7f4886b4e640 1 --2- 172.21.15.84:0/1960651180 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f486c06aea0 0x7f486c06d360 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.975+0000 7f4886b4e640 1 --2- 172.21.15.84:0/1960651180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4880161990 0x7f488015d890 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.975+0000 7f4886b4e640 1 --2- 172.21.15.84:0/1960651180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f488015d410 0x7f48801583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.975+0000 7f4886b4e640 1 --2- 172.21.15.84:0/1960651180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488015d030 0x7f48801f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:07.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.975+0000 7f4886b4e640 1 -- 172.21.15.84:0/1960651180 >> 172.21.15.84:0/1960651180 conn(0x7f48800cf720 msgr2=0x7f488015b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:07.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.975+0000 7f4886b4e640 1 -- 172.21.15.84:0/1960651180 shutdown_connections 2023-10-16T09:50:07.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:07.975+0000 7f4886b4e640 1 -- 172.21.15.84:0/1960651180 wait complete. 2023-10-16T09:50:07.992 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-16T09:50:08.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.196+0000 7fdc206e0640 1 Processor -- start 2023-10-16T09:50:08.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.197+0000 7fdc206e0640 1 -- start start 2023-10-16T09:50:08.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.198+0000 7fdc206e0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc180ad350 0x7fdc18069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:08.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.198+0000 7fdc206e0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1806a730 con 0x7fdc1806a2e0 2023-10-16T09:50:08.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.198+0000 7fdc206e0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc180ac890 con 0x7fdc1815d150 2023-10-16T09:50:08.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.198+0000 7fdc206e0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1815d530 con 0x7fdc180ad350 2023-10-16T09:50:08.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.198+0000 7fdc1ec7d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdc1806a2e0 0x7fdc180c9000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48588/0 (socket says 172.21.15.84:48588) 2023-10-16T09:50:08.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.199+0000 7fdc1ec7d640 1 -- 172.21.15.84:0/20815475 learned_addr learned my addr 172.21.15.84:0/20815475 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:08.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.199+0000 7fdc1dc7b640 1 --2- 172.21.15.84:0/20815475 >> v2:172.21.15.157:3300/0 conn(0x7fdc180ad350 0x7fdc18069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:08.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.199+0000 7fdc1d47a640 1 -- 172.21.15.84:0/20815475 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891653021 0 0) 0x7fdc10001490 con 0x7fdc1806a2e0 2023-10-16T09:50:08.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.199+0000 7fdc1d47a640 1 -- 172.21.15.84:0/20815475 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc04003520 con 0x7fdc1806a2e0 2023-10-16T09:50:08.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.199+0000 7fdc1d47a640 1 -- 172.21.15.84:0/20815475 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 821017393 0 0) 0x7fdc04003520 con 0x7fdc1806a2e0 2023-10-16T09:50:08.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.199+0000 7fdc1d47a640 1 -- 172.21.15.84:0/20815475 >> v2:172.21.15.157:3300/0 conn(0x7fdc180ad350 msgr2=0x7fdc18069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:08.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.199+0000 7fdc1d47a640 1 --2- 172.21.15.84:0/20815475 >> v2:172.21.15.157:3300/0 conn(0x7fdc180ad350 0x7fdc18069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.199+0000 7fdc1d47a640 1 -- 172.21.15.84:0/20815475 >> v1:172.21.15.157:6789/0 conn(0x7fdc1815d150 legacy=0x7fdc18160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:08.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.199+0000 7fdc1d47a640 1 -- 172.21.15.84:0/20815475 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc180ad7a0 con 0x7fdc1806a2e0 2023-10-16T09:50:08.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.199+0000 7fdc1d47a640 1 -- 172.21.15.84:0/20815475 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fdc10001b80 con 0x7fdc1806a2e0 2023-10-16T09:50:08.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.199+0000 7fdc1d47a640 1 -- 172.21.15.84:0/20815475 >> v1:172.21.15.84:6789/0 conn(0x7fdc1806a2e0 legacy=0x7fdc180c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:08.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.199+0000 7fdc1d47a640 1 --2- 172.21.15.84:0/20815475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc040039c0 0x7fdc04003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:08.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.200+0000 7fdc1d47a640 1 --2- 172.21.15.84:0/20815475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc040047c0 0x7fdc04004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:08.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.200+0000 7fdc1d47a640 1 --2- 172.21.15.84:0/20815475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc04005180 0x7fdc0400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:08.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.200+0000 7fdc1d47a640 1 -- 172.21.15.84:0/20815475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1815d530 con 0x7fdc040039c0 2023-10-16T09:50:08.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.200+0000 7fdc1d47a640 1 -- 172.21.15.84:0/20815475 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0400c080 con 0x7fdc040047c0 2023-10-16T09:50:08.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.200+0000 7fdc1d47a640 1 -- 172.21.15.84:0/20815475 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0400c240 con 0x7fdc04005180 2023-10-16T09:50:08.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.200+0000 7fdc1e47c640 1 --2- 172.21.15.84:0/20815475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc040039c0 0x7fdc04003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:08.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.200+0000 7fdc1e47c640 1 -- 172.21.15.84:0/20815475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc04005180 msgr2=0x7fdc0400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:08.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.200+0000 7fdc1e47c640 1 --2- 172.21.15.84:0/20815475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc04005180 0x7fdc0400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.200+0000 7fdc1e47c640 1 -- 172.21.15.84:0/20815475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc040047c0 msgr2=0x7fdc04004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:08.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.200+0000 7fdc1e47c640 1 --2- 172.21.15.84:0/20815475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc040047c0 0x7fdc04004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.200+0000 7fdc1e47c640 1 -- 172.21.15.84:0/20815475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc04003520 con 0x7fdc040039c0 2023-10-16T09:50:08.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.200+0000 7fdc1ec7d640 1 -- 172.21.15.84:0/20815475 reap_dead start 2023-10-16T09:50:08.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.201+0000 7fdc1e47c640 1 --2- 172.21.15.84:0/20815475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc040039c0 0x7fdc04003de0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdc14001750 tx=0x7fdc040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59624a452298c81 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:08.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.202+0000 7fdc1d47a640 1 -- 172.21.15.84:0/20815475 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc140052d0 con 0x7fdc040039c0 2023-10-16T09:50:08.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.202+0000 7fdc1d47a640 1 -- 172.21.15.84:0/20815475 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1402e050 con 0x7fdc040039c0 2023-10-16T09:50:08.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.202+0000 7fdc1d47a640 1 -- 172.21.15.84:0/20815475 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14003830 con 0x7fdc040039c0 2023-10-16T09:50:08.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.202+0000 7fdc206e0640 1 -- 172.21.15.84:0/20815475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc040039c0 msgr2=0x7fdc04003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:08.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.202+0000 7fdc206e0640 1 --2- 172.21.15.84:0/20815475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc040039c0 0x7fdc04003de0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdc14001750 tx=0x7fdc040057f0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.202+0000 7fdc206e0640 1 -- 172.21.15.84:0/20815475 shutdown_connections 2023-10-16T09:50:08.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.202+0000 7fdc206e0640 1 --2- 172.21.15.84:0/20815475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc040039c0 0x7fdc04003de0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.202+0000 7fdc206e0640 1 -- 172.21.15.84:0/20815475 >> 172.21.15.84:0/20815475 conn(0x7fdc180ce9b0 msgr2=0x7fdc180cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:08.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.202+0000 7fdc206e0640 1 -- 172.21.15.84:0/20815475 shutdown_connections 2023-10-16T09:50:08.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.202+0000 7fdc206e0640 1 -- 172.21.15.84:0/20815475 wait complete. 2023-10-16T09:50:08.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.204+0000 7fdc206e0640 1 Processor -- start 2023-10-16T09:50:08.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.204+0000 7fdc206e0640 1 -- start start 2023-10-16T09:50:08.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.205+0000 7fdc206e0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc040039c0 0x7fdc1814aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:08.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.205+0000 7fdc206e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc181595c0 0x7fdc180b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:08.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.205+0000 7fdc206e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc180afc80 0x7fdc180b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:08.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.205+0000 7fdc206e0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc180b0640 con 0x7fdc181595c0 2023-10-16T09:50:08.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.205+0000 7fdc206e0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc180b3250 con 0x7fdc040039c0 2023-10-16T09:50:08.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.205+0000 7fdc206e0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc180b21b0 con 0x7fdc180afc80 2023-10-16T09:50:08.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.205+0000 7fdc1dc7b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc181595c0 0x7fdc180b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:08.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.205+0000 7fdc1dc7b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc181595c0 0x7fdc180b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46944/0 (socket says 172.21.15.84:46944) 2023-10-16T09:50:08.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.205+0000 7fdc1dc7b640 1 -- 172.21.15.84:0/4114559447 learned_addr learned my addr 172.21.15.84:0/4114559447 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:08.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.206+0000 7fdc1e47c640 1 --2- 172.21.15.84:0/4114559447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc040039c0 0x7fdc1814aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:08.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.206+0000 7fdc1ec7d640 1 --2- 172.21.15.84:0/4114559447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc180afc80 0x7fdc180b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:08.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.206+0000 7fdc1dc7b640 1 -- 172.21.15.84:0/4114559447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc180afc80 msgr2=0x7fdc180b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:08.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.206+0000 7fdc1dc7b640 1 --2- 172.21.15.84:0/4114559447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc180afc80 0x7fdc180b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.206+0000 7fdc1dc7b640 1 -- 172.21.15.84:0/4114559447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc040039c0 msgr2=0x7fdc1814aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:08.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.206+0000 7fdc1dc7b640 1 --2- 172.21.15.84:0/4114559447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc040039c0 0x7fdc1814aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.206+0000 7fdc1dc7b640 1 -- 172.21.15.84:0/4114559447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc180b2490 con 0x7fdc181595c0 2023-10-16T09:50:08.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.206+0000 7fdc1ec7d640 1 --2- 172.21.15.84:0/4114559447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc180afc80 0x7fdc180b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:08.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.206+0000 7fdc1dc7b640 1 --2- 172.21.15.84:0/4114559447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc181595c0 0x7fdc180b18d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800e860 tx=0x7fdc0800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:08.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.206+0000 7fdc0effd640 1 -- 172.21.15.84:0/4114559447 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08009600 con 0x7fdc181595c0 2023-10-16T09:50:08.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.207+0000 7fdc0effd640 1 -- 172.21.15.84:0/4114559447 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0800c940 con 0x7fdc181595c0 2023-10-16T09:50:08.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.207+0000 7fdc0effd640 1 -- 172.21.15.84:0/4114559447 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0800ccd0 con 0x7fdc181595c0 2023-10-16T09:50:08.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.207+0000 7fdc206e0640 1 -- 172.21.15.84:0/4114559447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc180b2770 con 0x7fdc181595c0 2023-10-16T09:50:08.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.207+0000 7fdc206e0640 1 -- 172.21.15.84:0/4114559447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc180b2d30 con 0x7fdc181595c0 2023-10-16T09:50:08.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.208+0000 7fdc206e0640 1 -- 172.21.15.84:0/4114559447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1804e160 con 0x7fdc181595c0 2023-10-16T09:50:08.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.209+0000 7fdc0effd640 1 -- 172.21.15.84:0/4114559447 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdc08013470 con 0x7fdc181595c0 2023-10-16T09:50:08.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.210+0000 7fdc0effd640 1 --2- 172.21.15.84:0/4114559447 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdbfc06ae70 0x7fdbfc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:08.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.210+0000 7fdc0effd640 1 -- 172.21.15.84:0/4114559447 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdc080894c0 con 0x7fdc181595c0 2023-10-16T09:50:08.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.210+0000 7fdc1e47c640 1 --2- 172.21.15.84:0/4114559447 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdbfc06ae70 0x7fdbfc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:08.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.211+0000 7fdc1e47c640 1 --2- 172.21.15.84:0/4114559447 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdbfc06ae70 0x7fdbfc06d330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdc14033000 tx=0x7fdc14009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:08.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.214+0000 7fdc0effd640 1 -- 172.21.15.84:0/4114559447 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdc08054830 con 0x7fdc181595c0 2023-10-16T09:50:08.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.475+0000 7fdc206e0640 1 -- 172.21.15.84:0/4114559447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fdc180ace20 con 0x7fdc181595c0 2023-10-16T09:50:08.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.479+0000 7fdc0effd640 1 -- 172.21.15.84:0/4114559447 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7fdc08057e50 con 0x7fdc181595c0 2023-10-16T09:50:08.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.482+0000 7fdc206e0640 1 -- 172.21.15.84:0/4114559447 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdbfc06ae70 msgr2=0x7fdbfc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:08.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.482+0000 7fdc206e0640 1 --2- 172.21.15.84:0/4114559447 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdbfc06ae70 0x7fdbfc06d330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdc14033000 tx=0x7fdc14009030 comp rx=0 tx=0).stop 2023-10-16T09:50:08.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.482+0000 7fdc206e0640 1 -- 172.21.15.84:0/4114559447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc181595c0 msgr2=0x7fdc180b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:08.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.482+0000 7fdc206e0640 1 --2- 172.21.15.84:0/4114559447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc181595c0 0x7fdc180b18d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800e860 tx=0x7fdc0800ed20 comp rx=0 tx=0).stop 2023-10-16T09:50:08.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.482+0000 7fdc206e0640 1 -- 172.21.15.84:0/4114559447 shutdown_connections 2023-10-16T09:50:08.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.482+0000 7fdc206e0640 1 --2- 172.21.15.84:0/4114559447 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdbfc06ae70 0x7fdbfc06d330 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.482+0000 7fdc206e0640 1 --2- 172.21.15.84:0/4114559447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc180afc80 0x7fdc180b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.482+0000 7fdc206e0640 1 --2- 172.21.15.84:0/4114559447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc181595c0 0x7fdc180b18d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.482+0000 7fdc206e0640 1 --2- 172.21.15.84:0/4114559447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc040039c0 0x7fdc1814aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.482+0000 7fdc206e0640 1 -- 172.21.15.84:0/4114559447 >> 172.21.15.84:0/4114559447 conn(0x7fdc180ce9b0 msgr2=0x7fdc18070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:08.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.482+0000 7fdc206e0640 1 -- 172.21.15.84:0/4114559447 shutdown_connections 2023-10-16T09:50:08.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.482+0000 7fdc206e0640 1 -- 172.21.15.84:0/4114559447 wait complete. 2023-10-16T09:50:08.500 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:50:08.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.711+0000 7f9a18aa2640 1 Processor -- start 2023-10-16T09:50:08.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.712+0000 7f9a18aa2640 1 -- start start 2023-10-16T09:50:08.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.713+0000 7f9a18aa2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9a140cbd90 0x7f9a1406e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:08.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.713+0000 7f9a18aa2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a1406c5f0 0x7f9a1406ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:08.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.713+0000 7f9a18aa2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1406cf60 con 0x7f9a1415e550 2023-10-16T09:50:08.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.713+0000 7f9a18aa2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a1406ea00 con 0x7f9a140cbd90 2023-10-16T09:50:08.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.713+0000 7f9a18aa2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1406eca0 con 0x7f9a1406c5f0 2023-10-16T09:50:08.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.713+0000 7f9a1259c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9a1415e550 0x7f9a140caee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48598/0 (socket says 172.21.15.84:48598) 2023-10-16T09:50:08.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.713+0000 7f9a1259c640 1 -- 172.21.15.84:0/1316457387 learned_addr learned my addr 172.21.15.84:0/1316457387 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:08.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.714+0000 7f9a1159a640 1 -- 172.21.15.84:0/1316457387 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3153907163 0 0) 0x7f9a1406cf60 con 0x7f9a1415e550 2023-10-16T09:50:08.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.714+0000 7f9a1159a640 1 -- 172.21.15.84:0/1316457387 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a00003520 con 0x7f9a1415e550 2023-10-16T09:50:08.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.714+0000 7f9a11d9b640 1 --2- 172.21.15.84:0/1316457387 >> v2:172.21.15.157:3300/0 conn(0x7f9a140cbd90 0x7f9a1406e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:08.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.714+0000 7f9a12d9d640 1 --2- 172.21.15.84:0/1316457387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a1406c5f0 0x7f9a1406ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:08.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.714+0000 7f9a1159a640 1 -- 172.21.15.84:0/1316457387 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4053394562 0 0) 0x7f9a00003520 con 0x7f9a1415e550 2023-10-16T09:50:08.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.714+0000 7f9a1159a640 1 -- 172.21.15.84:0/1316457387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a1406c5f0 msgr2=0x7f9a1406ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:08.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.714+0000 7f9a1159a640 1 --2- 172.21.15.84:0/1316457387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a1406c5f0 0x7f9a1406ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.714+0000 7f9a1159a640 1 -- 172.21.15.84:0/1316457387 >> v2:172.21.15.157:3300/0 conn(0x7f9a140cbd90 msgr2=0x7f9a1406e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:08.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.714+0000 7f9a1159a640 1 --2- 172.21.15.84:0/1316457387 >> v2:172.21.15.157:3300/0 conn(0x7f9a140cbd90 0x7f9a1406e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.714+0000 7f9a1159a640 1 -- 172.21.15.84:0/1316457387 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a140ca400 con 0x7f9a1415e550 2023-10-16T09:50:08.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.714+0000 7f9a1159a640 1 -- 172.21.15.84:0/1316457387 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f9a040019f0 con 0x7f9a1415e550 2023-10-16T09:50:08.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.714+0000 7f9a1159a640 1 -- 172.21.15.84:0/1316457387 >> v1:172.21.15.84:6789/0 conn(0x7f9a1415e550 legacy=0x7f9a140caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:08.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.714+0000 7f9a1159a640 1 --2- 172.21.15.84:0/1316457387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a1406c5f0 0x7f9a00003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:08.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.714+0000 7f9a1159a640 1 --2- 172.21.15.84:0/1316457387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a00003ff0 0x7f9a000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:08.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.714+0000 7f9a1159a640 1 --2- 172.21.15.84:0/1316457387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a00004d00 0x7f9a00005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:08.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.714+0000 7f9a1159a640 1 -- 172.21.15.84:0/1316457387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1406ea00 con 0x7f9a00004d00 2023-10-16T09:50:08.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.714+0000 7f9a1159a640 1 -- 172.21.15.84:0/1316457387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1406eca0 con 0x7f9a00003ff0 2023-10-16T09:50:08.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.714+0000 7f9a1159a640 1 -- 172.21.15.84:0/1316457387 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0000bb90 con 0x7f9a1406c5f0 2023-10-16T09:50:08.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.715+0000 7f9a12d9d640 1 --2- 172.21.15.84:0/1316457387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a00004d00 0x7f9a00005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:08.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.715+0000 7f9a11d9b640 1 --2- 172.21.15.84:0/1316457387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a00003ff0 0x7f9a000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:08.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.715+0000 7f9a1259c640 1 --2- 172.21.15.84:0/1316457387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a1406c5f0 0x7f9a00003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:08.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.715+0000 7f9a12d9d640 1 -- 172.21.15.84:0/1316457387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a1406c5f0 msgr2=0x7f9a00003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:08.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.715+0000 7f9a12d9d640 1 --2- 172.21.15.84:0/1316457387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a1406c5f0 0x7f9a00003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.715+0000 7f9a12d9d640 1 -- 172.21.15.84:0/1316457387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a00003ff0 msgr2=0x7f9a000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:08.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.715+0000 7f9a12d9d640 1 --2- 172.21.15.84:0/1316457387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a00003ff0 0x7f9a000047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.715+0000 7f9a12d9d640 1 -- 172.21.15.84:0/1316457387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a00003520 con 0x7f9a00004d00 2023-10-16T09:50:08.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.715+0000 7f9a1259c640 1 --2- 172.21.15.84:0/1316457387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a1406c5f0 0x7f9a00003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:08.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.716+0000 7f9a12d9d640 1 --2- 172.21.15.84:0/1316457387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a00004d00 0x7f9a00005180 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00ba30 tx=0x7f9a000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8197390c57e9f1f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:08.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.716+0000 7f9a1159a640 1 -- 172.21.15.84:0/1316457387 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c032630 con 0x7f9a00004d00 2023-10-16T09:50:08.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.716+0000 7f9a1159a640 1 -- 172.21.15.84:0/1316457387 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c032790 con 0x7f9a00004d00 2023-10-16T09:50:08.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.717+0000 7f9a1159a640 1 -- 172.21.15.84:0/1316457387 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c032b20 con 0x7f9a00004d00 2023-10-16T09:50:08.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.717+0000 7f9a18aa2640 1 -- 172.21.15.84:0/1316457387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a00004d00 msgr2=0x7f9a00005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:08.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.717+0000 7f9a18aa2640 1 --2- 172.21.15.84:0/1316457387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a00004d00 0x7f9a00005180 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00ba30 tx=0x7f9a000057f0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.717+0000 7f9a12d9d640 1 -- 172.21.15.84:0/1316457387 reap_dead start 2023-10-16T09:50:08.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.717+0000 7f9a18aa2640 1 -- 172.21.15.84:0/1316457387 shutdown_connections 2023-10-16T09:50:08.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.717+0000 7f9a18aa2640 1 -- 172.21.15.84:0/1316457387 >> 172.21.15.84:0/1316457387 conn(0x7f9a140cfaa0 msgr2=0x7f9a140cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:08.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.717+0000 7f9a18aa2640 1 -- 172.21.15.84:0/1316457387 shutdown_connections 2023-10-16T09:50:08.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.717+0000 7f9a18aa2640 1 -- 172.21.15.84:0/1316457387 wait complete. 2023-10-16T09:50:08.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.719+0000 7f9a18aa2640 1 Processor -- start 2023-10-16T09:50:08.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.719+0000 7f9a18aa2640 1 -- start start 2023-10-16T09:50:08.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.720+0000 7f9a18aa2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a00004d00 0x7f9a141e3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:08.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.720+0000 7f9a18aa2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a141e52d0 0x7f9a14151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:08.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.720+0000 7f9a18aa2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a14144d90 0x7f9a14145240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:08.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.720+0000 7f9a18aa2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a141e5000 con 0x7f9a14144d90 2023-10-16T09:50:08.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.720+0000 7f9a18aa2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140c6b70 con 0x7f9a141e52d0 2023-10-16T09:50:08.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.720+0000 7f9a18aa2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a141e43f0 con 0x7f9a00004d00 2023-10-16T09:50:08.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.720+0000 7f9a12d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a14144d90 0x7f9a14145240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:08.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.720+0000 7f9a12d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a14144d90 0x7f9a14145240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46958/0 (socket says 172.21.15.84:46958) 2023-10-16T09:50:08.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.720+0000 7f9a12d9d640 1 -- 172.21.15.84:0/3230058772 learned_addr learned my addr 172.21.15.84:0/3230058772 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:08.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.721+0000 7f9a11d9b640 1 --2- 172.21.15.84:0/3230058772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a141e52d0 0x7f9a14151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:08.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.721+0000 7f9a12d9d640 1 -- 172.21.15.84:0/3230058772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a00004d00 msgr2=0x7f9a141e3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:08.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.721+0000 7f9a1259c640 1 --2- 172.21.15.84:0/3230058772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a00004d00 0x7f9a141e3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:08.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.721+0000 7f9a12d9d640 1 --2- 172.21.15.84:0/3230058772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a00004d00 0x7f9a141e3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.721+0000 7f9a12d9d640 1 -- 172.21.15.84:0/3230058772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a141e52d0 msgr2=0x7f9a14151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:08.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.721+0000 7f9a12d9d640 1 --2- 172.21.15.84:0/3230058772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a141e52d0 0x7f9a14151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.721+0000 7f9a12d9d640 1 -- 172.21.15.84:0/3230058772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a141e46d0 con 0x7f9a14144d90 2023-10-16T09:50:08.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.721+0000 7f9a12d9d640 1 --2- 172.21.15.84:0/3230058772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a14144d90 0x7f9a14145240 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00bb50 tx=0x7f9a0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:08.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.721+0000 7f9a0affd640 1 -- 172.21.15.84:0/3230058772 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c004020 con 0x7f9a14144d90 2023-10-16T09:50:08.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.721+0000 7f9a0affd640 1 -- 172.21.15.84:0/3230058772 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c004180 con 0x7f9a14144d90 2023-10-16T09:50:08.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.721+0000 7f9a18aa2640 1 -- 172.21.15.84:0/3230058772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a141e49b0 con 0x7f9a14144d90 2023-10-16T09:50:08.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.721+0000 7f9a0affd640 1 -- 172.21.15.84:0/3230058772 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c0044f0 con 0x7f9a14144d90 2023-10-16T09:50:08.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.722+0000 7f9a18aa2640 1 -- 172.21.15.84:0/3230058772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a14145780 con 0x7f9a14144d90 2023-10-16T09:50:08.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.723+0000 7f9a18aa2640 1 -- 172.21.15.84:0/3230058772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1404e160 con 0x7f9a14144d90 2023-10-16T09:50:08.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.723+0000 7f9a0affd640 1 -- 172.21.15.84:0/3230058772 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9a0c03b630 con 0x7f9a14144d90 2023-10-16T09:50:08.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.724+0000 7f9a0affd640 1 --2- 172.21.15.84:0/3230058772 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f99f406aea0 0x7f99f406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:08.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.724+0000 7f9a0affd640 1 -- 172.21.15.84:0/3230058772 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9a0c0ab2b0 con 0x7f9a14144d90 2023-10-16T09:50:08.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.725+0000 7f9a1259c640 1 --2- 172.21.15.84:0/3230058772 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f99f406aea0 0x7f99f406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:08.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.725+0000 7f9a1259c640 1 --2- 172.21.15.84:0/3230058772 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f99f406aea0 0x7f99f406d360 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9a04001b20 tx=0x7f9a04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:08.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.729+0000 7f9a0affd640 1 -- 172.21.15.84:0/3230058772 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9a0c076610 con 0x7f9a14144d90 2023-10-16T09:50:08.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.924+0000 7f9a18aa2640 1 -- 172.21.15.84:0/3230058772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a140ca990 con 0x7f9a14144d90 2023-10-16T09:50:08.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.925+0000 7f9a0affd640 1 -- 172.21.15.84:0/3230058772 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12833 (secure 0 0 0) 0x7f9a0c079c30 con 0x7f9a14144d90 2023-10-16T09:50:08.926 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:50:08.927 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":30,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:50:04.882444+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:50:08.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.928+0000 7f9a18aa2640 1 -- 172.21.15.84:0/3230058772 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f99f406aea0 msgr2=0x7f99f406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:08.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.928+0000 7f9a18aa2640 1 --2- 172.21.15.84:0/3230058772 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f99f406aea0 0x7f99f406d360 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9a04001b20 tx=0x7f9a04009040 comp rx=0 tx=0).stop 2023-10-16T09:50:08.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.928+0000 7f9a18aa2640 1 -- 172.21.15.84:0/3230058772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a14144d90 msgr2=0x7f9a14145240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:08.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.928+0000 7f9a18aa2640 1 --2- 172.21.15.84:0/3230058772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a14144d90 0x7f9a14145240 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00bb50 tx=0x7f9a0c004730 comp rx=0 tx=0).stop 2023-10-16T09:50:08.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.928+0000 7f9a18aa2640 1 -- 172.21.15.84:0/3230058772 shutdown_connections 2023-10-16T09:50:08.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.928+0000 7f9a18aa2640 1 --2- 172.21.15.84:0/3230058772 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f99f406aea0 0x7f99f406d360 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.928+0000 7f9a18aa2640 1 --2- 172.21.15.84:0/3230058772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a14144d90 0x7f9a14145240 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.928+0000 7f9a18aa2640 1 --2- 172.21.15.84:0/3230058772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a141e52d0 0x7f9a14151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.928+0000 7f9a18aa2640 1 --2- 172.21.15.84:0/3230058772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a00004d00 0x7f9a141e3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:08.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.928+0000 7f9a18aa2640 1 -- 172.21.15.84:0/3230058772 >> 172.21.15.84:0/3230058772 conn(0x7f9a140cfaa0 msgr2=0x7f9a1415ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:08.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.928+0000 7f9a18aa2640 1 -- 172.21.15.84:0/3230058772 shutdown_connections 2023-10-16T09:50:08.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:08.928+0000 7f9a18aa2640 1 -- 172.21.15.84:0/3230058772 wait complete. 2023-10-16T09:50:08.949 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:50:08.949 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:50:09.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.160+0000 7f5474ca2640 1 Processor -- start 2023-10-16T09:50:09.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.160+0000 7f5474ca2640 1 -- start start 2023-10-16T09:50:09.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.162+0000 7f5474ca2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f547006a4b0 0x7f54700c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:09.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.162+0000 7f5474ca2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54700ac890 con 0x7f547015d150 2023-10-16T09:50:09.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.162+0000 7f5474ca2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54700ad7a0 con 0x7f54700ad350 2023-10-16T09:50:09.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.162+0000 7f5474ca2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f547015d530 con 0x7f547006a4b0 2023-10-16T09:50:09.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.162+0000 7f546e59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f547015d150 0x7f5470160b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48612/0 (socket says 172.21.15.84:48612) 2023-10-16T09:50:09.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.162+0000 7f546e59c640 1 -- 172.21.15.84:0/597491160 learned_addr learned my addr 172.21.15.84:0/597491160 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:09.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.162+0000 7f546ed9d640 1 --2- 172.21.15.84:0/597491160 >> v2:172.21.15.84:3300/0 conn(0x7f547006a4b0 0x7f54700c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:09.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.163+0000 7f546ed9d640 1 -- 172.21.15.84:0/597491160 >> v1:172.21.15.157:6789/0 conn(0x7f54700ad350 legacy=0x7f5470069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:09.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.163+0000 7f546ed9d640 1 -- 172.21.15.84:0/597491160 >> v1:172.21.15.84:6789/0 conn(0x7f547015d150 legacy=0x7f5470160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:09.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.163+0000 7f546ed9d640 1 -- 172.21.15.84:0/597491160 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5470161650 con 0x7f547006a4b0 2023-10-16T09:50:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.164+0000 7f546ed9d640 1 --2- 172.21.15.84:0/597491160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f547006a4b0 0x7f54700c9000 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5468009d90 tx=0x7f546802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c431768605f8bf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.164+0000 7f546dd9b640 1 -- 172.21.15.84:0/597491160 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546802cc60 con 0x7f547006a4b0 2023-10-16T09:50:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.164+0000 7f546dd9b640 1 -- 172.21.15.84:0/597491160 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546802cdc0 con 0x7f547006a4b0 2023-10-16T09:50:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.164+0000 7f546dd9b640 1 -- 172.21.15.84:0/597491160 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468034660 con 0x7f547006a4b0 2023-10-16T09:50:09.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.165+0000 7f5474ca2640 1 -- 172.21.15.84:0/597491160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f547006a4b0 msgr2=0x7f54700c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:09.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.165+0000 7f5474ca2640 1 --2- 172.21.15.84:0/597491160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f547006a4b0 0x7f54700c9000 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5468009d90 tx=0x7f546802beb0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.165+0000 7f5474ca2640 1 -- 172.21.15.84:0/597491160 shutdown_connections 2023-10-16T09:50:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.165+0000 7f5474ca2640 1 --2- 172.21.15.84:0/597491160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f547006a4b0 0x7f54700c9000 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.165+0000 7f5474ca2640 1 -- 172.21.15.84:0/597491160 >> 172.21.15.84:0/597491160 conn(0x7f54700ce9b0 msgr2=0x7f54700cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.165+0000 7f5474ca2640 1 -- 172.21.15.84:0/597491160 shutdown_connections 2023-10-16T09:50:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.165+0000 7f5474ca2640 1 -- 172.21.15.84:0/597491160 wait complete. 2023-10-16T09:50:09.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.167+0000 7f5474ca2640 1 Processor -- start 2023-10-16T09:50:09.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.167+0000 7f5474ca2640 1 -- start start 2023-10-16T09:50:09.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.168+0000 7f5474ca2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f547006a4b0 0x7f54701f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:09.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.168+0000 7f5474ca2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54700ad350 0x7f54700b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:09.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.168+0000 7f5474ca2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f547015d150 0x7f54700b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:09.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.168+0000 7f5474ca2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54700c7e40 con 0x7f547006a4b0 2023-10-16T09:50:09.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.168+0000 7f5474ca2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54700c7270 con 0x7f547015d150 2023-10-16T09:50:09.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.168+0000 7f5474ca2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54700c7510 con 0x7f54700ad350 2023-10-16T09:50:09.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.168+0000 7f546e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f547006a4b0 0x7f54701f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:09.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.168+0000 7f546e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f547006a4b0 0x7f54701f33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46964/0 (socket says 172.21.15.84:46964) 2023-10-16T09:50:09.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.168+0000 7f546e59c640 1 -- 172.21.15.84:0/1942250641 learned_addr learned my addr 172.21.15.84:0/1942250641 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:09.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.168+0000 7f546e59c640 1 -- 172.21.15.84:0/1942250641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54700ad350 msgr2=0x7f54700b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:09.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.168+0000 7f546e59c640 1 --2- 172.21.15.84:0/1942250641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54700ad350 0x7f54700b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.168+0000 7f546e59c640 1 -- 172.21.15.84:0/1942250641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f547015d150 msgr2=0x7f54700b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:09.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.168+0000 7f546e59c640 1 --2- 172.21.15.84:0/1942250641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f547015d150 0x7f54700b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.168+0000 7f546e59c640 1 -- 172.21.15.84:0/1942250641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5468009a70 con 0x7f547006a4b0 2023-10-16T09:50:09.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.169+0000 7f546e59c640 1 --2- 172.21.15.84:0/1942250641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f547006a4b0 0x7f54701f33f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f546000c4a0 tx=0x7f546000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:09.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.169+0000 7f5466ffd640 1 -- 172.21.15.84:0/1942250641 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460005ce0 con 0x7f547006a4b0 2023-10-16T09:50:09.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.169+0000 7f5466ffd640 1 -- 172.21.15.84:0/1942250641 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5460005e40 con 0x7f547006a4b0 2023-10-16T09:50:09.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.169+0000 7f5466ffd640 1 -- 172.21.15.84:0/1942250641 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460007270 con 0x7f547006a4b0 2023-10-16T09:50:09.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.169+0000 7f5474ca2640 1 -- 172.21.15.84:0/1942250641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54700c77f0 con 0x7f547006a4b0 2023-10-16T09:50:09.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.169+0000 7f5474ca2640 1 -- 172.21.15.84:0/1942250641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54700c82d0 con 0x7f547006a4b0 2023-10-16T09:50:09.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.170+0000 7f5474ca2640 1 -- 172.21.15.84:0/1942250641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54700c58d0 con 0x7f547006a4b0 2023-10-16T09:50:09.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.171+0000 7f5466ffd640 1 -- 172.21.15.84:0/1942250641 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5460007490 con 0x7f547006a4b0 2023-10-16T09:50:09.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.172+0000 7f5466ffd640 1 --2- 172.21.15.84:0/1942250641 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f545006ae70 0x7f545006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:09.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.172+0000 7f5466ffd640 1 -- 172.21.15.84:0/1942250641 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5460088770 con 0x7f547006a4b0 2023-10-16T09:50:09.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.172+0000 7f5467fff640 1 --2- 172.21.15.84:0/1942250641 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f545006ae70 0x7f545006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:09.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.173+0000 7f5467fff640 1 --2- 172.21.15.84:0/1942250641 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f545006ae70 0x7f545006d330 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f54700ca080 tx=0x7f54580079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:09.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.176+0000 7f5466ffd640 1 -- 172.21.15.84:0/1942250641 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5460053b80 con 0x7f547006a4b0 2023-10-16T09:50:09.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.366+0000 7f5474ca2640 1 -- 172.21.15.84:0/1942250641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f547004e160 con 0x7f547006a4b0 2023-10-16T09:50:09.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.367+0000 7f5466ffd640 1 -- 172.21.15.84:0/1942250641 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f54600170a0 con 0x7f547006a4b0 2023-10-16T09:50:09.368 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T09:50:09.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.369+0000 7f5474ca2640 1 -- 172.21.15.84:0/1942250641 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f545006ae70 msgr2=0x7f545006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:09.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.369+0000 7f5474ca2640 1 --2- 172.21.15.84:0/1942250641 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f545006ae70 0x7f545006d330 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f54700ca080 tx=0x7f54580079b0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.369+0000 7f5474ca2640 1 -- 172.21.15.84:0/1942250641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f547006a4b0 msgr2=0x7f54701f33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:09.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.369+0000 7f5474ca2640 1 --2- 172.21.15.84:0/1942250641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f547006a4b0 0x7f54701f33f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f546000c4a0 tx=0x7f546000c960 comp rx=0 tx=0).stop 2023-10-16T09:50:09.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.369+0000 7f5474ca2640 1 -- 172.21.15.84:0/1942250641 shutdown_connections 2023-10-16T09:50:09.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.369+0000 7f5474ca2640 1 --2- 172.21.15.84:0/1942250641 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f545006ae70 0x7f545006d330 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.369+0000 7f5474ca2640 1 --2- 172.21.15.84:0/1942250641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f547015d150 0x7f54700b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.370+0000 7f5474ca2640 1 --2- 172.21.15.84:0/1942250641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54700ad350 0x7f54700b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.370+0000 7f5474ca2640 1 --2- 172.21.15.84:0/1942250641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f547006a4b0 0x7f54701f33f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.370+0000 7f5474ca2640 1 -- 172.21.15.84:0/1942250641 >> 172.21.15.84:0/1942250641 conn(0x7f54700ce9b0 msgr2=0x7f547015a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:09.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.370+0000 7f5474ca2640 1 -- 172.21.15.84:0/1942250641 shutdown_connections 2023-10-16T09:50:09.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.370+0000 7f5474ca2640 1 -- 172.21.15.84:0/1942250641 wait complete. 2023-10-16T09:50:09.390 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T09:50:09.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.603+0000 7f0ddb141640 1 Processor -- start 2023-10-16T09:50:09.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.604+0000 7f0ddb141640 1 -- start start 2023-10-16T09:50:09.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.605+0000 7f0ddb141640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dd415aaa0 0x7f0dd415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:09.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.605+0000 7f0ddb141640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0dd415ae80 0x7f0dd40cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:09.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.605+0000 7f0ddb141640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0dd40ccaa0 0x7f0dd4070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:09.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.605+0000 7f0ddb141640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd406b810 con 0x7f0dd40ccaa0 2023-10-16T09:50:09.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.605+0000 7f0ddb141640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd415ed60 con 0x7f0dd415ae80 2023-10-16T09:50:09.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.605+0000 7f0ddb141640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40707e0 con 0x7f0dd415aaa0 2023-10-16T09:50:09.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.605+0000 7f0dd96de640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0dd40ccaa0 0x7f0dd4070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:09.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.605+0000 7f0dd96de640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0dd40ccaa0 0x7f0dd4070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46978/0 (socket says 172.21.15.84:46978) 2023-10-16T09:50:09.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.605+0000 7f0dd96de640 1 -- 172.21.15.84:0/4108798847 learned_addr learned my addr 172.21.15.84:0/4108798847 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:09.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.606+0000 7f0dd3fff640 1 --2- 172.21.15.84:0/4108798847 >> v2:172.21.15.157:3300/0 conn(0x7f0dd415ae80 0x7f0dd40cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:09.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.606+0000 7f0dd8edd640 1 --2- 172.21.15.84:0/4108798847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dd415aaa0 0x7f0dd415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:09.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.606+0000 7f0dd96de640 1 -- 172.21.15.84:0/4108798847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dd415aaa0 msgr2=0x7f0dd415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:09.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.606+0000 7f0dd96de640 1 --2- 172.21.15.84:0/4108798847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dd415aaa0 0x7f0dd415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.606+0000 7f0dd96de640 1 -- 172.21.15.84:0/4108798847 >> v2:172.21.15.157:3300/0 conn(0x7f0dd415ae80 msgr2=0x7f0dd40cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:09.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.606+0000 7f0dd96de640 1 --2- 172.21.15.84:0/4108798847 >> v2:172.21.15.157:3300/0 conn(0x7f0dd415ae80 0x7f0dd40cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.606+0000 7f0dd96de640 1 -- 172.21.15.84:0/4108798847 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd415ef90 con 0x7f0dd40ccaa0 2023-10-16T09:50:09.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.606+0000 7f0dd8edd640 1 --2- 172.21.15.84:0/4108798847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dd415aaa0 0x7f0dd415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:09.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.606+0000 7f0dd3fff640 1 --2- 172.21.15.84:0/4108798847 >> v2:172.21.15.157:3300/0 conn(0x7f0dd415ae80 0x7f0dd40cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:09.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.607+0000 7f0dd96de640 1 --2- 172.21.15.84:0/4108798847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dd40ccaa0 0x7f0dd4070f50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40cb270 tx=0x7f0dcc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42856efc025e53ad server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:09.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.607+0000 7f0dd37fe640 1 -- 172.21.15.84:0/4108798847 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc031870 con 0x7f0dd40ccaa0 2023-10-16T09:50:09.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.607+0000 7f0dd37fe640 1 -- 172.21.15.84:0/4108798847 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dcc0319d0 con 0x7f0dd40ccaa0 2023-10-16T09:50:09.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.607+0000 7f0dd37fe640 1 -- 172.21.15.84:0/4108798847 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc031d60 con 0x7f0dd40ccaa0 2023-10-16T09:50:09.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.608+0000 7f0ddb141640 1 -- 172.21.15.84:0/4108798847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dd40ccaa0 msgr2=0x7f0dd4070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:09.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.608+0000 7f0ddb141640 1 --2- 172.21.15.84:0/4108798847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dd40ccaa0 0x7f0dd4070f50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40cb270 tx=0x7f0dcc02b910 comp rx=0 tx=0).stop 2023-10-16T09:50:09.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.608+0000 7f0ddb141640 1 -- 172.21.15.84:0/4108798847 shutdown_connections 2023-10-16T09:50:09.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.608+0000 7f0ddb141640 1 --2- 172.21.15.84:0/4108798847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dd40ccaa0 0x7f0dd4070f50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.608+0000 7f0ddb141640 1 --2- 172.21.15.84:0/4108798847 >> v2:172.21.15.157:3300/0 conn(0x7f0dd415ae80 0x7f0dd40cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.608+0000 7f0ddb141640 1 --2- 172.21.15.84:0/4108798847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dd415aaa0 0x7f0dd415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.608+0000 7f0ddb141640 1 -- 172.21.15.84:0/4108798847 >> 172.21.15.84:0/4108798847 conn(0x7f0dd40cb410 msgr2=0x7f0dd40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:09.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.608+0000 7f0ddb141640 1 -- 172.21.15.84:0/4108798847 shutdown_connections 2023-10-16T09:50:09.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.608+0000 7f0ddb141640 1 -- 172.21.15.84:0/4108798847 wait complete. 2023-10-16T09:50:09.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.610+0000 7f0ddb141640 1 Processor -- start 2023-10-16T09:50:09.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.610+0000 7f0ddb141640 1 -- start start 2023-10-16T09:50:09.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.611+0000 7f0ddb141640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dd40ccaa0 0x7f0dd40c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:09.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.611+0000 7f0ddb141640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dd415aaa0 0x7f0dd4155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:09.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.611+0000 7f0ddb141640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd415ae80 0x7f0dd40af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:09.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.611+0000 7f0ddb141640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40c3b60 con 0x7f0dd415aaa0 2023-10-16T09:50:09.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.611+0000 7f0ddb141640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd415bf60 con 0x7f0dd415ae80 2023-10-16T09:50:09.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.611+0000 7f0ddb141640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd415c240 con 0x7f0dd40ccaa0 2023-10-16T09:50:09.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.611+0000 7f0dd3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dd415aaa0 0x7f0dd4155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:09.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.611+0000 7f0dd3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dd415aaa0 0x7f0dd4155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46994/0 (socket says 172.21.15.84:46994) 2023-10-16T09:50:09.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.611+0000 7f0dd3fff640 1 -- 172.21.15.84:0/870352175 learned_addr learned my addr 172.21.15.84:0/870352175 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:09.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.612+0000 7f0dd3fff640 1 -- 172.21.15.84:0/870352175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dd40ccaa0 msgr2=0x7f0dd40c2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:09.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.612+0000 7f0dd3fff640 1 --2- 172.21.15.84:0/870352175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dd40ccaa0 0x7f0dd40c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.612+0000 7f0dd3fff640 1 -- 172.21.15.84:0/870352175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd415ae80 msgr2=0x7f0dd40af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:09.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.612+0000 7f0dd3fff640 1 --2- 172.21.15.84:0/870352175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd415ae80 0x7f0dd40af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.612+0000 7f0dd3fff640 1 -- 172.21.15.84:0/870352175 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc0094d0 con 0x7f0dd415aaa0 2023-10-16T09:50:09.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.612+0000 7f0dd3fff640 1 --2- 172.21.15.84:0/870352175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dd415aaa0 0x7f0dd4155e30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800c7f0 tx=0x7f0dc800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:09.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.612+0000 7f0dd17fa640 1 -- 172.21.15.84:0/870352175 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8007a30 con 0x7f0dd415aaa0 2023-10-16T09:50:09.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.612+0000 7f0dd17fa640 1 -- 172.21.15.84:0/870352175 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc8007b90 con 0x7f0dd415aaa0 2023-10-16T09:50:09.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.612+0000 7f0dd17fa640 1 -- 172.21.15.84:0/870352175 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc800ce40 con 0x7f0dd415aaa0 2023-10-16T09:50:09.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.612+0000 7f0ddb141640 1 -- 172.21.15.84:0/870352175 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd415c520 con 0x7f0dd415aaa0 2023-10-16T09:50:09.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.612+0000 7f0ddb141640 1 -- 172.21.15.84:0/870352175 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd40c3410 con 0x7f0dd415aaa0 2023-10-16T09:50:09.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.614+0000 7f0ddb141640 1 -- 172.21.15.84:0/870352175 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd40c4190 con 0x7f0dd415aaa0 2023-10-16T09:50:09.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.614+0000 7f0dd17fa640 1 -- 172.21.15.84:0/870352175 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0dc80113f0 con 0x7f0dd415aaa0 2023-10-16T09:50:09.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.615+0000 7f0dd17fa640 1 --2- 172.21.15.84:0/870352175 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0dc006ae70 0x7f0dc006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:09.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.615+0000 7f0dd17fa640 1 -- 172.21.15.84:0/870352175 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0dc80894c0 con 0x7f0dd415aaa0 2023-10-16T09:50:09.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.616+0000 7f0dd8edd640 1 --2- 172.21.15.84:0/870352175 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0dc006ae70 0x7f0dc006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:09.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.616+0000 7f0dd8edd640 1 --2- 172.21.15.84:0/870352175 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0dc006ae70 0x7f0dc006d330 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4007130 tx=0x7f0dc4007300 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:09.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.620+0000 7f0dd17fa640 1 -- 172.21.15.84:0/870352175 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0dd40c4190 con 0x7f0dd415aaa0 2023-10-16T09:50:09.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.813+0000 7f0ddb141640 1 -- 172.21.15.84:0/870352175 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0dd40c4190 con 0x7f0dd415aaa0 2023-10-16T09:50:09.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.814+0000 7f0dd17fa640 1 -- 172.21.15.84:0/870352175 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0dd40c4190 con 0x7f0dd415aaa0 2023-10-16T09:50:09.815 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:50:09.815 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T09:50:09.815 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:09.816 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-16T09:50:09.816 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-16T09:50:09.816 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:09.816 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T09:50:09.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.816+0000 7f0ddb141640 1 -- 172.21.15.84:0/870352175 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0dc006ae70 msgr2=0x7f0dc006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:09.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.816+0000 7f0ddb141640 1 --2- 172.21.15.84:0/870352175 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0dc006ae70 0x7f0dc006d330 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4007130 tx=0x7f0dc4007300 comp rx=0 tx=0).stop 2023-10-16T09:50:09.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.816+0000 7f0ddb141640 1 -- 172.21.15.84:0/870352175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dd415aaa0 msgr2=0x7f0dd4155e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:09.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.816+0000 7f0ddb141640 1 --2- 172.21.15.84:0/870352175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dd415aaa0 0x7f0dd4155e30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800c7f0 tx=0x7f0dc800cbf0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.816+0000 7f0ddb141640 1 -- 172.21.15.84:0/870352175 shutdown_connections 2023-10-16T09:50:09.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.816+0000 7f0ddb141640 1 --2- 172.21.15.84:0/870352175 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0dc006ae70 0x7f0dc006d330 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.816+0000 7f0ddb141640 1 --2- 172.21.15.84:0/870352175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd415ae80 0x7f0dd40af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.816+0000 7f0ddb141640 1 --2- 172.21.15.84:0/870352175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dd415aaa0 0x7f0dd4155e30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.816+0000 7f0ddb141640 1 --2- 172.21.15.84:0/870352175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dd40ccaa0 0x7f0dd40c2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:09.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.816+0000 7f0ddb141640 1 -- 172.21.15.84:0/870352175 >> 172.21.15.84:0/870352175 conn(0x7f0dd40cb410 msgr2=0x7f0dd40b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:09.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.817+0000 7f0ddb141640 1 -- 172.21.15.84:0/870352175 shutdown_connections 2023-10-16T09:50:09.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:09.817+0000 7f0ddb141640 1 -- 172.21.15.84:0/870352175 wait complete. 2023-10-16T09:50:09.836 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T09:50:09.836 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T09:50:10.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.038+0000 7fdedb844640 1 Processor -- start 2023-10-16T09:50:10.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.038+0000 7fdedb844640 1 -- start start 2023-10-16T09:50:10.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.040+0000 7fdedb844640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fded40cca70 0x7fded40cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:10.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.040+0000 7fdedb844640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fded40cfa00 0x7fded415c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:10.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.040+0000 7fdedb844640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded406bbc0 con 0x7fded406cc60 2023-10-16T09:50:10.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.040+0000 7fdedb844640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded406bea0 con 0x7fded40cfa00 2023-10-16T09:50:10.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.040+0000 7fdedb844640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded406c180 con 0x7fded40cca70 2023-10-16T09:50:10.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.040+0000 7fded9de1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fded40cfa00 0x7fded415c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:10.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.040+0000 7fded9de1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fded40cfa00 0x7fded415c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46996/0 (socket says 172.21.15.84:46996) 2023-10-16T09:50:10.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.040+0000 7fded9de1640 1 -- 172.21.15.84:0/2220702588 learned_addr learned my addr 172.21.15.84:0/2220702588 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:10.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.041+0000 7fded9de1640 1 -- 172.21.15.84:0/2220702588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fded40cca70 msgr2=0x7fded40cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:10.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.041+0000 7fded9de1640 1 --2- 172.21.15.84:0/2220702588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fded40cca70 0x7fded40cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:10.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.041+0000 7fded9de1640 1 -- 172.21.15.84:0/2220702588 >> v1:172.21.15.157:6789/0 conn(0x7fded406cc60 legacy=0x7fded40cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:10.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.041+0000 7fded9de1640 1 -- 172.21.15.84:0/2220702588 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded406c460 con 0x7fded40cfa00 2023-10-16T09:50:10.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.042+0000 7fded9de1640 1 --2- 172.21.15.84:0/2220702588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fded40cfa00 0x7fded415c730 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdecc009870 tx=0x7fdecc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729a7e4d2f4ba70e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:10.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.042+0000 7fded3fff640 1 -- 172.21.15.84:0/2220702588 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc031640 con 0x7fded40cfa00 2023-10-16T09:50:10.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.042+0000 7fded3fff640 1 -- 172.21.15.84:0/2220702588 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdecc0317a0 con 0x7fded40cfa00 2023-10-16T09:50:10.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.042+0000 7fded3fff640 1 -- 172.21.15.84:0/2220702588 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc031b30 con 0x7fded40cfa00 2023-10-16T09:50:10.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.043+0000 7fdedb844640 1 -- 172.21.15.84:0/2220702588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fded40cfa00 msgr2=0x7fded415c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:10.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.043+0000 7fdedb844640 1 --2- 172.21.15.84:0/2220702588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fded40cfa00 0x7fded415c730 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdecc009870 tx=0x7fdecc02b8f0 comp rx=0 tx=0).stop 2023-10-16T09:50:10.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.043+0000 7fdedb844640 1 -- 172.21.15.84:0/2220702588 shutdown_connections 2023-10-16T09:50:10.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.043+0000 7fdedb844640 1 --2- 172.21.15.84:0/2220702588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fded40cfa00 0x7fded415c730 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:10.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.043+0000 7fdedb844640 1 --2- 172.21.15.84:0/2220702588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fded40cca70 0x7fded40cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:10.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.043+0000 7fdedb844640 1 -- 172.21.15.84:0/2220702588 >> 172.21.15.84:0/2220702588 conn(0x7fded4099e30 msgr2=0x7fded40d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:10.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.043+0000 7fdedb844640 1 -- 172.21.15.84:0/2220702588 shutdown_connections 2023-10-16T09:50:10.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.043+0000 7fdedb844640 1 -- 172.21.15.84:0/2220702588 wait complete. 2023-10-16T09:50:10.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.045+0000 7fdedb844640 1 Processor -- start 2023-10-16T09:50:10.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.045+0000 7fdedb844640 1 -- start start 2023-10-16T09:50:10.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.046+0000 7fdedb844640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fded406cc60 0x7fded4157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:10.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.046+0000 7fdedb844640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded40cca70 0x7fded40af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:10.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.046+0000 7fdedb844640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fded40cfa00 0x7fded40af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:10.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.046+0000 7fdedb844640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded40c7b90 con 0x7fded406cc60 2023-10-16T09:50:10.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.046+0000 7fdedb844640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded40c89f0 con 0x7fded40cca70 2023-10-16T09:50:10.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.046+0000 7fdedb844640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded40c7de0 con 0x7fded40cfa00 2023-10-16T09:50:10.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.046+0000 7fded95e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fded406cc60 0x7fded4157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:10.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.046+0000 7fded95e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fded406cc60 0x7fded4157a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47010/0 (socket says 172.21.15.84:47010) 2023-10-16T09:50:10.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.047+0000 7fded95e0640 1 -- 172.21.15.84:0/415378281 learned_addr learned my addr 172.21.15.84:0/415378281 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:10.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.047+0000 7fded95e0640 1 -- 172.21.15.84:0/415378281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fded40cfa00 msgr2=0x7fded40af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:10.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.047+0000 7fded95e0640 1 --2- 172.21.15.84:0/415378281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fded40cfa00 0x7fded40af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:10.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.047+0000 7fded95e0640 1 -- 172.21.15.84:0/415378281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded40cca70 msgr2=0x7fded40af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:10.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.047+0000 7fded95e0640 1 --2- 172.21.15.84:0/415378281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded40cca70 0x7fded40af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:10.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.047+0000 7fded95e0640 1 -- 172.21.15.84:0/415378281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc0094d0 con 0x7fded406cc60 2023-10-16T09:50:10.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.047+0000 7fded95e0640 1 --2- 172.21.15.84:0/415378281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fded406cc60 0x7fded4157a60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdec800ba80 tx=0x7fdec800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:10.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.047+0000 7fded1ffb640 1 -- 172.21.15.84:0/415378281 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec800cc00 con 0x7fded406cc60 2023-10-16T09:50:10.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.048+0000 7fded1ffb640 1 -- 172.21.15.84:0/415378281 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec800cd80 con 0x7fded406cc60 2023-10-16T09:50:10.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.048+0000 7fdedb844640 1 -- 172.21.15.84:0/415378281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded40c80c0 con 0x7fded406cc60 2023-10-16T09:50:10.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.048+0000 7fded1ffb640 1 -- 172.21.15.84:0/415378281 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8014680 con 0x7fded406cc60 2023-10-16T09:50:10.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.048+0000 7fdedb844640 1 -- 172.21.15.84:0/415378281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded40c8680 con 0x7fded406cc60 2023-10-16T09:50:10.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.049+0000 7fdedb844640 1 -- 172.21.15.84:0/415378281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded40cc220 con 0x7fded406cc60 2023-10-16T09:50:10.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.050+0000 7fded1ffb640 1 -- 172.21.15.84:0/415378281 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdec801d3b0 con 0x7fded406cc60 2023-10-16T09:50:10.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.051+0000 7fded1ffb640 1 --2- 172.21.15.84:0/415378281 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdeb806ae70 0x7fdeb806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:10.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.051+0000 7fded1ffb640 1 -- 172.21.15.84:0/415378281 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdec8089340 con 0x7fded406cc60 2023-10-16T09:50:10.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.051+0000 7fded8ddf640 1 --2- 172.21.15.84:0/415378281 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdeb806ae70 0x7fdeb806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:10.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.052+0000 7fded8ddf640 1 --2- 172.21.15.84:0/415378281 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdeb806ae70 0x7fdeb806d330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fded40cbf90 tx=0x7fdec0007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:10.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.055+0000 7fded1ffb640 1 -- 172.21.15.84:0/415378281 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdec80546a0 con 0x7fded406cc60 2023-10-16T09:50:10.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.246+0000 7fdedb844640 1 -- 172.21.15.84:0/415378281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fded404e160 con 0x7fded406cc60 2023-10-16T09:50:10.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.898+0000 7fded1ffb640 1 -- 172.21.15.84:0/415378281 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdec8057cc0 con 0x7fded406cc60 2023-10-16T09:50:10.900 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-16T09:50:10.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.902+0000 7fdedb844640 1 -- 172.21.15.84:0/415378281 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdeb806ae70 msgr2=0x7fdeb806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:10.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.902+0000 7fdedb844640 1 --2- 172.21.15.84:0/415378281 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdeb806ae70 0x7fdeb806d330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fded40cbf90 tx=0x7fdec0007990 comp rx=0 tx=0).stop 2023-10-16T09:50:10.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.902+0000 7fdedb844640 1 -- 172.21.15.84:0/415378281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fded406cc60 msgr2=0x7fded4157a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:10.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.902+0000 7fdedb844640 1 --2- 172.21.15.84:0/415378281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fded406cc60 0x7fded4157a60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdec800ba80 tx=0x7fdec800bf40 comp rx=0 tx=0).stop 2023-10-16T09:50:10.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.902+0000 7fdedb844640 1 -- 172.21.15.84:0/415378281 shutdown_connections 2023-10-16T09:50:10.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.902+0000 7fdedb844640 1 --2- 172.21.15.84:0/415378281 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fdeb806ae70 0x7fdeb806d330 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:10.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.902+0000 7fdedb844640 1 --2- 172.21.15.84:0/415378281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fded40cfa00 0x7fded40af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:10.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.902+0000 7fdedb844640 1 --2- 172.21.15.84:0/415378281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded40cca70 0x7fded40af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:10.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.902+0000 7fdedb844640 1 --2- 172.21.15.84:0/415378281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fded406cc60 0x7fded4157a60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:10.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.902+0000 7fdedb844640 1 -- 172.21.15.84:0/415378281 >> 172.21.15.84:0/415378281 conn(0x7fded4099e30 msgr2=0x7fded415fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:10.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.903+0000 7fdedb844640 1 -- 172.21.15.84:0/415378281 shutdown_connections 2023-10-16T09:50:10.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:10.903+0000 7fdedb844640 1 -- 172.21.15.84:0/415378281 wait complete. 2023-10-16T09:50:10.928 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T09:50:11.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.132+0000 7f1a60101640 1 Processor -- start 2023-10-16T09:50:11.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.133+0000 7f1a60101640 1 -- start start 2023-10-16T09:50:11.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.134+0000 7f1a60101640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1a58160420 0x7f1a580ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:11.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.134+0000 7f1a60101640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a58160800 0x7f1a580cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:11.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.134+0000 7f1a60101640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1a580adc40 0x7f1a5806adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:11.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.134+0000 7f1a60101640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a580cde30 con 0x7f1a580adc40 2023-10-16T09:50:11.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.134+0000 7f1a60101640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a5806e170 con 0x7f1a58160420 2023-10-16T09:50:11.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.134+0000 7f1a60101640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a580cd460 con 0x7f1a58160800 2023-10-16T09:50:11.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.134+0000 7f1a5e69e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1a580adc40 0x7f1a5806adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:11.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.134+0000 7f1a5e69e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1a580adc40 0x7f1a5806adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47012/0 (socket says 172.21.15.84:47012) 2023-10-16T09:50:11.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.134+0000 7f1a5e69e640 1 -- 172.21.15.84:0/3255390378 learned_addr learned my addr 172.21.15.84:0/3255390378 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:11.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.134+0000 7f1a5de9d640 1 --2- 172.21.15.84:0/3255390378 >> v2:172.21.15.157:3300/0 conn(0x7f1a58160420 0x7f1a580ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:11.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.135+0000 7f1a5d69c640 1 --2- 172.21.15.84:0/3255390378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a58160800 0x7f1a580cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:11.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.135+0000 7f1a5e69e640 1 -- 172.21.15.84:0/3255390378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a58160800 msgr2=0x7f1a580cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:11.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.135+0000 7f1a5e69e640 1 --2- 172.21.15.84:0/3255390378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a58160800 0x7f1a580cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:11.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.135+0000 7f1a5e69e640 1 -- 172.21.15.84:0/3255390378 >> v2:172.21.15.157:3300/0 conn(0x7f1a58160420 msgr2=0x7f1a580ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:11.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.135+0000 7f1a5e69e640 1 --2- 172.21.15.84:0/3255390378 >> v2:172.21.15.157:3300/0 conn(0x7f1a58160420 0x7f1a580ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:11.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.135+0000 7f1a5e69e640 1 -- 172.21.15.84:0/3255390378 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5806e3c0 con 0x7f1a580adc40 2023-10-16T09:50:11.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.135+0000 7f1a5de9d640 1 --2- 172.21.15.84:0/3255390378 >> v2:172.21.15.157:3300/0 conn(0x7f1a58160420 0x7f1a580ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:11.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.136+0000 7f1a5e69e640 1 --2- 172.21.15.84:0/3255390378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a580adc40 0x7f1a5806adb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1a5806f470 tx=0x7f1a5002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3ea475f847c0fc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:11.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.136+0000 7f1a5ce9b640 1 -- 172.21.15.84:0/3255390378 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5002ca70 con 0x7f1a580adc40 2023-10-16T09:50:11.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.136+0000 7f1a5ce9b640 1 -- 172.21.15.84:0/3255390378 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5002cbd0 con 0x7f1a580adc40 2023-10-16T09:50:11.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.136+0000 7f1a5ce9b640 1 -- 172.21.15.84:0/3255390378 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a500343f0 con 0x7f1a580adc40 2023-10-16T09:50:11.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.137+0000 7f1a60101640 1 -- 172.21.15.84:0/3255390378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a580adc40 msgr2=0x7f1a5806adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:11.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.137+0000 7f1a60101640 1 --2- 172.21.15.84:0/3255390378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a580adc40 0x7f1a5806adb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1a5806f470 tx=0x7f1a5002bed0 comp rx=0 tx=0).stop 2023-10-16T09:50:11.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.137+0000 7f1a60101640 1 -- 172.21.15.84:0/3255390378 shutdown_connections 2023-10-16T09:50:11.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.137+0000 7f1a60101640 1 --2- 172.21.15.84:0/3255390378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a580adc40 0x7f1a5806adb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:11.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.137+0000 7f1a60101640 1 --2- 172.21.15.84:0/3255390378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a58160800 0x7f1a580cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:11.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.137+0000 7f1a60101640 1 --2- 172.21.15.84:0/3255390378 >> v2:172.21.15.157:3300/0 conn(0x7f1a58160420 0x7f1a580ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:11.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.137+0000 7f1a60101640 1 -- 172.21.15.84:0/3255390378 >> 172.21.15.84:0/3255390378 conn(0x7f1a580b0d30 msgr2=0x7f1a580b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:11.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.137+0000 7f1a60101640 1 -- 172.21.15.84:0/3255390378 shutdown_connections 2023-10-16T09:50:11.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.137+0000 7f1a60101640 1 -- 172.21.15.84:0/3255390378 wait complete. 2023-10-16T09:50:11.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.139+0000 7f1a60101640 1 Processor -- start 2023-10-16T09:50:11.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.139+0000 7f1a60101640 1 -- start start 2023-10-16T09:50:11.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.140+0000 7f1a60101640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a580adc40 0x7f1a581f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:11.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.140+0000 7f1a60101640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a58160420 0x7f1a5809d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:11.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.140+0000 7f1a60101640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a58160800 0x7f1a5809d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:11.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.140+0000 7f1a60101640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5809ee40 con 0x7f1a58160800 2023-10-16T09:50:11.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.140+0000 7f1a60101640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a580c8d10 con 0x7f1a58160420 2023-10-16T09:50:11.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.140+0000 7f1a60101640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a581f5a00 con 0x7f1a580adc40 2023-10-16T09:50:11.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.140+0000 7f1a5e69e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a58160800 0x7f1a5809d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:11.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.140+0000 7f1a5e69e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a58160800 0x7f1a5809d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47014/0 (socket says 172.21.15.84:47014) 2023-10-16T09:50:11.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.140+0000 7f1a5e69e640 1 -- 172.21.15.84:0/225960765 learned_addr learned my addr 172.21.15.84:0/225960765 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:11.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.140+0000 7f1a5d69c640 1 --2- 172.21.15.84:0/225960765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a58160420 0x7f1a5809d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:11.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.140+0000 7f1a5de9d640 1 --2- 172.21.15.84:0/225960765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a580adc40 0x7f1a581f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:11.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.141+0000 7f1a5e69e640 1 -- 172.21.15.84:0/225960765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a580adc40 msgr2=0x7f1a581f54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:11.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.141+0000 7f1a5e69e640 1 --2- 172.21.15.84:0/225960765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a580adc40 0x7f1a581f54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:11.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.141+0000 7f1a5e69e640 1 -- 172.21.15.84:0/225960765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a58160420 msgr2=0x7f1a5809d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:11.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.141+0000 7f1a5e69e640 1 --2- 172.21.15.84:0/225960765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a58160420 0x7f1a5809d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:11.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.141+0000 7f1a5e69e640 1 -- 172.21.15.84:0/225960765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a50009b30 con 0x7f1a58160800 2023-10-16T09:50:11.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.141+0000 7f1a5d69c640 1 --2- 172.21.15.84:0/225960765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a58160420 0x7f1a5809d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:11.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.141+0000 7f1a5e69e640 1 --2- 172.21.15.84:0/225960765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a58160800 0x7f1a5809d8f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009f40 tx=0x7f1a50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:11.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.141+0000 7f1a4a7fc640 1 -- 172.21.15.84:0/225960765 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5003e450 con 0x7f1a58160800 2023-10-16T09:50:11.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.141+0000 7f1a4a7fc640 1 -- 172.21.15.84:0/225960765 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5003e5d0 con 0x7f1a58160800 2023-10-16T09:50:11.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.141+0000 7f1a4a7fc640 1 -- 172.21.15.84:0/225960765 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5003e980 con 0x7f1a58160800 2023-10-16T09:50:11.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.141+0000 7f1a60101640 1 -- 172.21.15.84:0/225960765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a580c8a30 con 0x7f1a58160800 2023-10-16T09:50:11.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.141+0000 7f1a60101640 1 -- 172.21.15.84:0/225960765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a580b2e90 con 0x7f1a58160800 2023-10-16T09:50:11.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.143+0000 7f1a60101640 1 -- 172.21.15.84:0/225960765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a580b2a10 con 0x7f1a58160800 2023-10-16T09:50:11.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.144+0000 7f1a4a7fc640 1 -- 172.21.15.84:0/225960765 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1a5003eae0 con 0x7f1a58160800 2023-10-16T09:50:11.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.145+0000 7f1a4a7fc640 1 --2- 172.21.15.84:0/225960765 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f1a3c06ae70 0x7f1a3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:11.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.145+0000 7f1a4a7fc640 1 -- 172.21.15.84:0/225960765 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f1a500ac390 con 0x7f1a58160800 2023-10-16T09:50:11.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.145+0000 7f1a5de9d640 1 --2- 172.21.15.84:0/225960765 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f1a3c06ae70 0x7f1a3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:11.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.146+0000 7f1a5de9d640 1 --2- 172.21.15.84:0/225960765 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f1a3c06ae70 0x7f1a3c06d330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400a990 tx=0x7f1a54008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:11.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.150+0000 7f1a4a7fc640 1 -- 172.21.15.84:0/225960765 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1a580b2a10 con 0x7f1a58160800 2023-10-16T09:50:11.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.343+0000 7f1a60101640 1 -- 172.21.15.84:0/225960765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1a580b2a10 con 0x7f1a58160800 2023-10-16T09:50:11.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.911+0000 7f1a4a7fc640 1 -- 172.21.15.84:0/225960765 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1a580b2a10 con 0x7f1a58160800 2023-10-16T09:50:11.914 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-16T09:50:11.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.915+0000 7f1a60101640 1 -- 172.21.15.84:0/225960765 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f1a3c06ae70 msgr2=0x7f1a3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:11.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.915+0000 7f1a60101640 1 --2- 172.21.15.84:0/225960765 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f1a3c06ae70 0x7f1a3c06d330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400a990 tx=0x7f1a54008040 comp rx=0 tx=0).stop 2023-10-16T09:50:11.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.915+0000 7f1a60101640 1 -- 172.21.15.84:0/225960765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a58160800 msgr2=0x7f1a5809d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:11.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.915+0000 7f1a60101640 1 --2- 172.21.15.84:0/225960765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a58160800 0x7f1a5809d8f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009f40 tx=0x7f1a50004a40 comp rx=0 tx=0).stop 2023-10-16T09:50:11.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.915+0000 7f1a60101640 1 -- 172.21.15.84:0/225960765 shutdown_connections 2023-10-16T09:50:11.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.915+0000 7f1a60101640 1 --2- 172.21.15.84:0/225960765 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f1a3c06ae70 0x7f1a3c06d330 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:11.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.915+0000 7f1a60101640 1 --2- 172.21.15.84:0/225960765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a58160800 0x7f1a5809d8f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:11.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.915+0000 7f1a60101640 1 --2- 172.21.15.84:0/225960765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a58160420 0x7f1a5809d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:11.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.915+0000 7f1a60101640 1 --2- 172.21.15.84:0/225960765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a580adc40 0x7f1a581f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:11.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.915+0000 7f1a60101640 1 -- 172.21.15.84:0/225960765 >> 172.21.15.84:0/225960765 conn(0x7f1a580b0d30 msgr2=0x7f1a580af620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:11.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.916+0000 7f1a60101640 1 -- 172.21.15.84:0/225960765 shutdown_connections 2023-10-16T09:50:11.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:11.916+0000 7f1a60101640 1 -- 172.21.15.84:0/225960765 wait complete. 2023-10-16T09:50:11.952 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T09:50:12.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.136+0000 7ff1b359e640 1 Processor -- start 2023-10-16T09:50:12.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.136+0000 7ff1b359e640 1 -- start start 2023-10-16T09:50:12.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.137+0000 7ff1b359e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff1b406c040 0x7ff1b406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:12.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.137+0000 7ff1b359e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff1b40ad970 0x7ff1b415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:12.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.137+0000 7ff1b359e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b406c960 con 0x7ff1b415ebc0 2023-10-16T09:50:12.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.137+0000 7ff1b359e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b415b9d0 con 0x7ff1b40ad970 2023-10-16T09:50:12.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.137+0000 7ff1b359e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b415bcb0 con 0x7ff1b406c040 2023-10-16T09:50:12.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.138+0000 7ff1a9d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff1b40ad970 0x7ff1b415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:12.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.138+0000 7ff1a9d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff1b40ad970 0x7ff1b415e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47018/0 (socket says 172.21.15.84:47018) 2023-10-16T09:50:12.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.138+0000 7ff1a9d9b640 1 -- 172.21.15.84:0/3161941340 learned_addr learned my addr 172.21.15.84:0/3161941340 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:12.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.138+0000 7ff1b259c640 1 --2- 172.21.15.84:0/3161941340 >> v2:172.21.15.157:3300/0 conn(0x7ff1b406c040 0x7ff1b406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:12.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.138+0000 7ff1a9d9b640 1 -- 172.21.15.84:0/3161941340 >> v2:172.21.15.157:3300/0 conn(0x7ff1b406c040 msgr2=0x7ff1b406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:12.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.138+0000 7ff1a9d9b640 1 --2- 172.21.15.84:0/3161941340 >> v2:172.21.15.157:3300/0 conn(0x7ff1b406c040 0x7ff1b406c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:12.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.138+0000 7ff1a9d9b640 1 -- 172.21.15.84:0/3161941340 >> v1:172.21.15.157:6789/0 conn(0x7ff1b415ebc0 legacy=0x7ff1b40cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:12.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.138+0000 7ff1a9d9b640 1 -- 172.21.15.84:0/3161941340 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b415bf90 con 0x7ff1b40ad970 2023-10-16T09:50:12.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.139+0000 7ff1a9d9b640 1 --2- 172.21.15.84:0/3161941340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1b40ad970 0x7ff1b415e680 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40cf350 tx=0x7ff19c02b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf5ed0eed65d36 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:12.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.139+0000 7ff1b1d9b640 1 -- 172.21.15.84:0/3161941340 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c031890 con 0x7ff1b40ad970 2023-10-16T09:50:12.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.140+0000 7ff1b1d9b640 1 -- 172.21.15.84:0/3161941340 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19c0319f0 con 0x7ff1b40ad970 2023-10-16T09:50:12.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.140+0000 7ff1b1d9b640 1 -- 172.21.15.84:0/3161941340 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c031d80 con 0x7ff1b40ad970 2023-10-16T09:50:12.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.140+0000 7ff1b359e640 1 -- 172.21.15.84:0/3161941340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1b40ad970 msgr2=0x7ff1b415e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:12.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.140+0000 7ff1b359e640 1 --2- 172.21.15.84:0/3161941340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1b40ad970 0x7ff1b415e680 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40cf350 tx=0x7ff19c02b930 comp rx=0 tx=0).stop 2023-10-16T09:50:12.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.140+0000 7ff1b359e640 1 -- 172.21.15.84:0/3161941340 shutdown_connections 2023-10-16T09:50:12.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.140+0000 7ff1b359e640 1 --2- 172.21.15.84:0/3161941340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1b40ad970 0x7ff1b415e680 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:12.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.140+0000 7ff1b359e640 1 --2- 172.21.15.84:0/3161941340 >> v2:172.21.15.157:3300/0 conn(0x7ff1b406c040 0x7ff1b406c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:12.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.140+0000 7ff1b359e640 1 -- 172.21.15.84:0/3161941340 >> 172.21.15.84:0/3161941340 conn(0x7ff1b40ce6a0 msgr2=0x7ff1b40ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:12.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.140+0000 7ff1b359e640 1 -- 172.21.15.84:0/3161941340 shutdown_connections 2023-10-16T09:50:12.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.140+0000 7ff1b359e640 1 -- 172.21.15.84:0/3161941340 wait complete. 2023-10-16T09:50:12.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.142+0000 7ff1b359e640 1 Processor -- start 2023-10-16T09:50:12.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.142+0000 7ff1b359e640 1 -- start start 2023-10-16T09:50:12.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.143+0000 7ff1b359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1b406c040 0x7ff1b40acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:12.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.143+0000 7ff1b359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1b40ad970 0x7ff1b41efc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:12.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.143+0000 7ff1b359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1b415ebc0 0x7ff1b40aa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:12.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.143+0000 7ff1b359e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40b1980 con 0x7ff1b406c040 2023-10-16T09:50:12.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.143+0000 7ff1b359e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40ad500 con 0x7ff1b415ebc0 2023-10-16T09:50:12.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.143+0000 7ff1b359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b4150250 con 0x7ff1b40ad970 2023-10-16T09:50:12.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.143+0000 7ff1b259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1b406c040 0x7ff1b40acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:12.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.144+0000 7ff1b259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1b406c040 0x7ff1b40acfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47028/0 (socket says 172.21.15.84:47028) 2023-10-16T09:50:12.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.144+0000 7ff1b259c640 1 -- 172.21.15.84:0/846152789 learned_addr learned my addr 172.21.15.84:0/846152789 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:12.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.144+0000 7ff1a9d9b640 1 --2- 172.21.15.84:0/846152789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1b40ad970 0x7ff1b41efc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:12.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.144+0000 7ff1b2d9d640 1 --2- 172.21.15.84:0/846152789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1b415ebc0 0x7ff1b40aa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:12.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.144+0000 7ff1b259c640 1 -- 172.21.15.84:0/846152789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1b40ad970 msgr2=0x7ff1b41efc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:12.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.144+0000 7ff1b259c640 1 --2- 172.21.15.84:0/846152789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1b40ad970 0x7ff1b41efc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:12.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.144+0000 7ff1b259c640 1 -- 172.21.15.84:0/846152789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1b415ebc0 msgr2=0x7ff1b40aa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:12.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.144+0000 7ff1b259c640 1 --2- 172.21.15.84:0/846152789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1b415ebc0 0x7ff1b40aa3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:12.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.144+0000 7ff1b259c640 1 -- 172.21.15.84:0/846152789 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c0094d0 con 0x7ff1b406c040 2023-10-16T09:50:12.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.144+0000 7ff1b259c640 1 --2- 172.21.15.84:0/846152789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1b406c040 0x7ff1b40acfc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400c820 tx=0x7ff1a400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:12.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.145+0000 7ff1ab7fe640 1 -- 172.21.15.84:0/846152789 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4004020 con 0x7ff1b406c040 2023-10-16T09:50:12.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.145+0000 7ff1ab7fe640 1 -- 172.21.15.84:0/846152789 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a4004180 con 0x7ff1b406c040 2023-10-16T09:50:12.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.145+0000 7ff1ab7fe640 1 -- 172.21.15.84:0/846152789 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4007830 con 0x7ff1b406c040 2023-10-16T09:50:12.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.145+0000 7ff1b359e640 1 -- 172.21.15.84:0/846152789 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b40b4250 con 0x7ff1b406c040 2023-10-16T09:50:12.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.145+0000 7ff1b359e640 1 -- 172.21.15.84:0/846152789 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b40b1330 con 0x7ff1b406c040 2023-10-16T09:50:12.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.146+0000 7ff1b359e640 1 -- 172.21.15.84:0/846152789 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b406b330 con 0x7ff1b406c040 2023-10-16T09:50:12.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.147+0000 7ff1ab7fe640 1 -- 172.21.15.84:0/846152789 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff1a4007990 con 0x7ff1b406c040 2023-10-16T09:50:12.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.148+0000 7ff1ab7fe640 1 --2- 172.21.15.84:0/846152789 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff19406aec0 0x7ff19406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:12.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.148+0000 7ff1ab7fe640 1 -- 172.21.15.84:0/846152789 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5609+0+0 (secure 0 0 0) 0x7ff1a4089270 con 0x7ff1b406c040 2023-10-16T09:50:12.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.148+0000 7ff1a9d9b640 1 --2- 172.21.15.84:0/846152789 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff19406aec0 0x7ff19406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:12.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.149+0000 7ff1a9d9b640 1 --2- 172.21.15.84:0/846152789 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff19406aec0 0x7ff19406d380 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff19c036000 tx=0x7ff19c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:12.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.153+0000 7ff1ab7fe640 1 -- 172.21.15.84:0/846152789 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff1a4054360 con 0x7ff1b406c040 2023-10-16T09:50:12.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.377+0000 7ff1b359e640 1 -- 172.21.15.84:0/846152789 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff1b40adee0 con 0x7ff1b406c040 2023-10-16T09:50:12.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.914+0000 7ff1ab7fe640 1 -- 172.21.15.84:0/846152789 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff1a4057980 con 0x7ff1b406c040 2023-10-16T09:50:12.917 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 4 and data pool 5 2023-10-16T09:50:12.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.918+0000 7ff1b359e640 1 -- 172.21.15.84:0/846152789 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff19406aec0 msgr2=0x7ff19406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:12.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.918+0000 7ff1b359e640 1 --2- 172.21.15.84:0/846152789 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff19406aec0 0x7ff19406d380 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff19c036000 tx=0x7ff19c03c040 comp rx=0 tx=0).stop 2023-10-16T09:50:12.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.918+0000 7ff1b359e640 1 -- 172.21.15.84:0/846152789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1b406c040 msgr2=0x7ff1b40acfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:12.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.918+0000 7ff1b359e640 1 --2- 172.21.15.84:0/846152789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1b406c040 0x7ff1b40acfc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400c820 tx=0x7ff1a400cce0 comp rx=0 tx=0).stop 2023-10-16T09:50:12.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.918+0000 7ff1b359e640 1 -- 172.21.15.84:0/846152789 shutdown_connections 2023-10-16T09:50:12.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.918+0000 7ff1b359e640 1 --2- 172.21.15.84:0/846152789 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ff19406aec0 0x7ff19406d380 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:12.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.918+0000 7ff1b359e640 1 --2- 172.21.15.84:0/846152789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1b415ebc0 0x7ff1b40aa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:12.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.918+0000 7ff1b359e640 1 --2- 172.21.15.84:0/846152789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1b40ad970 0x7ff1b41efc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:12.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.918+0000 7ff1b359e640 1 --2- 172.21.15.84:0/846152789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1b406c040 0x7ff1b40acfc0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:12.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.918+0000 7ff1b359e640 1 -- 172.21.15.84:0/846152789 >> 172.21.15.84:0/846152789 conn(0x7ff1b40ce6a0 msgr2=0x7ff1b415de80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:12.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.918+0000 7ff1b359e640 1 -- 172.21.15.84:0/846152789 shutdown_connections 2023-10-16T09:50:12.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:12.918+0000 7ff1b359e640 1 -- 172.21.15.84:0/846152789 wait complete. 2023-10-16T09:50:12.945 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:50:13.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.133+0000 7fece9c4b640 1 Processor -- start 2023-10-16T09:50:13.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.133+0000 7fece9c4b640 1 -- start start 2023-10-16T09:50:13.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.134+0000 7fece9c4b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece4159b20 0x7fece406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:13.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.134+0000 7fece9c4b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fece4159f00 0x7fece40b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:13.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.134+0000 7fece9c4b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece406f7f0 con 0x7fece406c120 2023-10-16T09:50:13.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.134+0000 7fece9c4b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece406c5e0 con 0x7fece4159f00 2023-10-16T09:50:13.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.134+0000 7fece9c4b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece406c8c0 con 0x7fece4159b20 2023-10-16T09:50:13.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.135+0000 7fece3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fece406c120 0x7fece40cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48626/0 (socket says 172.21.15.84:48626) 2023-10-16T09:50:13.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.135+0000 7fece3fff640 1 -- 172.21.15.84:0/2885801201 learned_addr learned my addr 172.21.15.84:0/2885801201 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:13.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.135+0000 7fecdbfff640 1 --2- 172.21.15.84:0/2885801201 >> v2:172.21.15.157:3300/0 conn(0x7fece4159f00 0x7fece40b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:13.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.135+0000 7fece2ffd640 1 -- 172.21.15.84:0/2885801201 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197177019 0 0) 0x7fece406f7f0 con 0x7fece406c120 2023-10-16T09:50:13.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.135+0000 7fece37fe640 1 --2- 172.21.15.84:0/2885801201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece4159b20 0x7fece406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:13.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.135+0000 7fece2ffd640 1 -- 172.21.15.84:0/2885801201 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecd0003520 con 0x7fece406c120 2023-10-16T09:50:13.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.135+0000 7fece2ffd640 1 -- 172.21.15.84:0/2885801201 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2888661427 0 0) 0x7fecd0003520 con 0x7fece406c120 2023-10-16T09:50:13.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.135+0000 7fece2ffd640 1 -- 172.21.15.84:0/2885801201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece4159b20 msgr2=0x7fece406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:13.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.135+0000 7fece2ffd640 1 --2- 172.21.15.84:0/2885801201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece4159b20 0x7fece406bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.135+0000 7fece2ffd640 1 -- 172.21.15.84:0/2885801201 >> v2:172.21.15.157:3300/0 conn(0x7fece4159f00 msgr2=0x7fece40b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:13.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fece2ffd640 1 --2- 172.21.15.84:0/2885801201 >> v2:172.21.15.157:3300/0 conn(0x7fece4159f00 0x7fece40b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fece2ffd640 1 -- 172.21.15.84:0/2885801201 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece406cba0 con 0x7fece406c120 2023-10-16T09:50:13.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fece2ffd640 1 -- 172.21.15.84:0/2885801201 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fecdc0019f0 con 0x7fece406c120 2023-10-16T09:50:13.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fece2ffd640 1 -- 172.21.15.84:0/2885801201 >> v1:172.21.15.84:6789/0 conn(0x7fece406c120 legacy=0x7fece40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:13.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fece2ffd640 1 --2- 172.21.15.84:0/2885801201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd0003970 0x7fecd0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:13.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fece2ffd640 1 --2- 172.21.15.84:0/2885801201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece4159b20 0x7fecd00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:13.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fece2ffd640 1 --2- 172.21.15.84:0/2885801201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd0004d00 0x7fecd0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:13.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fece2ffd640 1 -- 172.21.15.84:0/2885801201 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece406c5e0 con 0x7fecd0003970 2023-10-16T09:50:13.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fece2ffd640 1 -- 172.21.15.84:0/2885801201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece406c8c0 con 0x7fecd0004d00 2023-10-16T09:50:13.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fece2ffd640 1 -- 172.21.15.84:0/2885801201 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd000dc10 con 0x7fece4159b20 2023-10-16T09:50:13.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fece37fe640 1 --2- 172.21.15.84:0/2885801201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd0003970 0x7fecd0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:13.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fecdbfff640 1 --2- 172.21.15.84:0/2885801201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece4159b20 0x7fecd00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:13.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fece3fff640 1 --2- 172.21.15.84:0/2885801201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd0004d00 0x7fecd0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:13.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fece37fe640 1 -- 172.21.15.84:0/2885801201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece4159b20 msgr2=0x7fecd00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:13.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fece37fe640 1 --2- 172.21.15.84:0/2885801201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece4159b20 0x7fecd00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fece37fe640 1 -- 172.21.15.84:0/2885801201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd0004d00 msgr2=0x7fecd0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:13.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fece37fe640 1 --2- 172.21.15.84:0/2885801201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd0004d00 0x7fecd0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fece37fe640 1 -- 172.21.15.84:0/2885801201 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd0003520 con 0x7fecd0003970 2023-10-16T09:50:13.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fecdbfff640 1 --2- 172.21.15.84:0/2885801201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece4159b20 0x7fecd00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:13.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.136+0000 7fece3fff640 1 --2- 172.21.15.84:0/2885801201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd0004d00 0x7fecd0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:13.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.138+0000 7fece37fe640 1 --2- 172.21.15.84:0/2885801201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd0003970 0x7fecd0003d90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fecd400ba30 tx=0x7fecd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6b5dc91209b62d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:13.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.138+0000 7fece2ffd640 1 -- 172.21.15.84:0/2885801201 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4032630 con 0x7fecd0003970 2023-10-16T09:50:13.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.138+0000 7fece2ffd640 1 -- 172.21.15.84:0/2885801201 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd4032790 con 0x7fecd0003970 2023-10-16T09:50:13.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.138+0000 7fece2ffd640 1 -- 172.21.15.84:0/2885801201 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4032b20 con 0x7fecd0003970 2023-10-16T09:50:13.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.138+0000 7fece9c4b640 1 -- 172.21.15.84:0/2885801201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd0003970 msgr2=0x7fecd0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:13.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.138+0000 7fece9c4b640 1 --2- 172.21.15.84:0/2885801201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd0003970 0x7fecd0003d90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fecd400ba30 tx=0x7fecd00057f0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.138+0000 7fece3fff640 1 -- 172.21.15.84:0/2885801201 reap_dead start 2023-10-16T09:50:13.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.138+0000 7fece9c4b640 1 -- 172.21.15.84:0/2885801201 shutdown_connections 2023-10-16T09:50:13.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.138+0000 7fece9c4b640 1 -- 172.21.15.84:0/2885801201 >> 172.21.15.84:0/2885801201 conn(0x7fece40cb410 msgr2=0x7fece40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:13.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.138+0000 7fece9c4b640 1 -- 172.21.15.84:0/2885801201 shutdown_connections 2023-10-16T09:50:13.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.138+0000 7fece9c4b640 1 -- 172.21.15.84:0/2885801201 wait complete. 2023-10-16T09:50:13.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.140+0000 7fece9c4b640 1 Processor -- start 2023-10-16T09:50:13.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.140+0000 7fece9c4b640 1 -- start start 2023-10-16T09:50:13.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.141+0000 7fece9c4b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd0003970 0x7fece4154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:13.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.141+0000 7fece9c4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece415c320 0x7fece41543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:13.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.141+0000 7fece9c4b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece40c2b80 0x7fece40c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:13.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.141+0000 7fece9c4b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece415cb00 con 0x7fecd0003970 2023-10-16T09:50:13.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.141+0000 7fece9c4b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece415cde0 con 0x7fece415c320 2023-10-16T09:50:13.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.141+0000 7fece9c4b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece415d0c0 con 0x7fece40c2b80 2023-10-16T09:50:13.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.141+0000 7fece37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd0003970 0x7fece4154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:13.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.141+0000 7fece37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd0003970 0x7fece4154b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47042/0 (socket says 172.21.15.84:47042) 2023-10-16T09:50:13.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.141+0000 7fece37fe640 1 -- 172.21.15.84:0/2636199960 learned_addr learned my addr 172.21.15.84:0/2636199960 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:13.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.142+0000 7fece3fff640 1 --2- 172.21.15.84:0/2636199960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece40c2b80 0x7fece40c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:13.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.142+0000 7fece37fe640 1 -- 172.21.15.84:0/2636199960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece40c2b80 msgr2=0x7fece40c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:13.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.142+0000 7fecdbfff640 1 --2- 172.21.15.84:0/2636199960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece415c320 0x7fece41543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:13.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.142+0000 7fece37fe640 1 --2- 172.21.15.84:0/2636199960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece40c2b80 0x7fece40c2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.142+0000 7fece37fe640 1 -- 172.21.15.84:0/2636199960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece415c320 msgr2=0x7fece41543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:13.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.142+0000 7fece37fe640 1 --2- 172.21.15.84:0/2636199960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece415c320 0x7fece41543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.142+0000 7fece37fe640 1 -- 172.21.15.84:0/2636199960 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece40c2660 con 0x7fecd0003970 2023-10-16T09:50:13.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.142+0000 7fece37fe640 1 --2- 172.21.15.84:0/2636199960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd0003970 0x7fece4154b80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fecd400bb50 tx=0x7fecd4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:13.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.142+0000 7fece0ff9640 1 -- 172.21.15.84:0/2636199960 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4004220 con 0x7fecd0003970 2023-10-16T09:50:13.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.143+0000 7fece0ff9640 1 -- 172.21.15.84:0/2636199960 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd4004380 con 0x7fecd0003970 2023-10-16T09:50:13.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.143+0000 7fece9c4b640 1 -- 172.21.15.84:0/2636199960 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece4154910 con 0x7fecd0003970 2023-10-16T09:50:13.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.143+0000 7fece0ff9640 1 -- 172.21.15.84:0/2636199960 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd403c870 con 0x7fecd0003970 2023-10-16T09:50:13.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.143+0000 7fece9c4b640 1 -- 172.21.15.84:0/2636199960 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece40c3500 con 0x7fecd0003970 2023-10-16T09:50:13.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.144+0000 7fece9c4b640 1 -- 172.21.15.84:0/2636199960 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece40c24c0 con 0x7fecd0003970 2023-10-16T09:50:13.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.145+0000 7fece0ff9640 1 -- 172.21.15.84:0/2636199960 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fecd4044070 con 0x7fecd0003970 2023-10-16T09:50:13.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.146+0000 7fece0ff9640 1 --2- 172.21.15.84:0/2636199960 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fecc406ae70 0x7fecc406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:13.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.146+0000 7fece0ff9640 1 -- 172.21.15.84:0/2636199960 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fecd40ab660 con 0x7fecd0003970 2023-10-16T09:50:13.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.146+0000 7fecdbfff640 1 --2- 172.21.15.84:0/2636199960 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fecc406ae70 0x7fecc406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:13.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.147+0000 7fecdbfff640 1 --2- 172.21.15.84:0/2636199960 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fecc406ae70 0x7fecc406d330 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fece40c8280 tx=0x7feccc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:13.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.151+0000 7fece0ff9640 1 -- 172.21.15.84:0/2636199960 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fece40c24c0 con 0x7fecd0003970 2023-10-16T09:50:13.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.341+0000 7fece9c4b640 1 -- 172.21.15.84:0/2636199960 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fece40c24c0 con 0x7fecd0003970 2023-10-16T09:50:13.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.344+0000 7fece0ff9640 1 -- 172.21.15.84:0/2636199960 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16558 (secure 0 0 0) 0x7fece40c24c0 con 0x7fecd0003970 2023-10-16T09:50:13.346 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:50:13.346 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":33,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:50:12.379609+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:50:10.297512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-16T09:50:11.394493+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:50:13.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.346+0000 7fece9c4b640 1 -- 172.21.15.84:0/2636199960 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fecc406ae70 msgr2=0x7fecc406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:13.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.346+0000 7fece9c4b640 1 --2- 172.21.15.84:0/2636199960 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fecc406ae70 0x7fecc406d330 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fece40c8280 tx=0x7feccc008040 comp rx=0 tx=0).stop 2023-10-16T09:50:13.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.346+0000 7fece9c4b640 1 -- 172.21.15.84:0/2636199960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd0003970 msgr2=0x7fece4154b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:13.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.346+0000 7fece9c4b640 1 --2- 172.21.15.84:0/2636199960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd0003970 0x7fece4154b80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fecd400bb50 tx=0x7fecd4004730 comp rx=0 tx=0).stop 2023-10-16T09:50:13.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.347+0000 7fece9c4b640 1 -- 172.21.15.84:0/2636199960 shutdown_connections 2023-10-16T09:50:13.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.347+0000 7fece9c4b640 1 --2- 172.21.15.84:0/2636199960 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fecc406ae70 0x7fecc406d330 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.347+0000 7fece9c4b640 1 --2- 172.21.15.84:0/2636199960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fece40c2b80 0x7fece40c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.347+0000 7fece9c4b640 1 --2- 172.21.15.84:0/2636199960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece415c320 0x7fece41543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.347+0000 7fece9c4b640 1 --2- 172.21.15.84:0/2636199960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd0003970 0x7fece4154b80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.347+0000 7fece9c4b640 1 -- 172.21.15.84:0/2636199960 >> 172.21.15.84:0/2636199960 conn(0x7fece40cb410 msgr2=0x7fece40aedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:13.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.347+0000 7fece9c4b640 1 -- 172.21.15.84:0/2636199960 shutdown_connections 2023-10-16T09:50:13.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.347+0000 7fece9c4b640 1 -- 172.21.15.84:0/2636199960 wait complete. 2023-10-16T09:50:13.366 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:50:13.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.577+0000 7f338b548640 1 Processor -- start 2023-10-16T09:50:13.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.577+0000 7f338b548640 1 -- start start 2023-10-16T09:50:13.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.578+0000 7f338b548640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3384160420 0x7f33840ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:13.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.578+0000 7f338b548640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f338406b4c0 0x7f33840cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:13.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.578+0000 7f338b548640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f338406e170 con 0x7f3384160800 2023-10-16T09:50:13.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.578+0000 7f338b548640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33840cde30 con 0x7f3384160420 2023-10-16T09:50:13.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.578+0000 7f338b548640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33840cd460 con 0x7f338406b4c0 2023-10-16T09:50:13.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.579+0000 7f33892e4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3384160420 0x7f33840ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:13.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.579+0000 7f33892e4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3384160420 0x7f33840ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47050/0 (socket says 172.21.15.84:47050) 2023-10-16T09:50:13.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.579+0000 7f33892e4640 1 -- 172.21.15.84:0/2165183125 learned_addr learned my addr 172.21.15.84:0/2165183125 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:13.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.579+0000 7f3389ae5640 1 --2- 172.21.15.84:0/2165183125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f338406b4c0 0x7f33840cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:13.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.579+0000 7f33892e4640 1 -- 172.21.15.84:0/2165183125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f338406b4c0 msgr2=0x7f33840cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:13.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.579+0000 7f33892e4640 1 --2- 172.21.15.84:0/2165183125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f338406b4c0 0x7f33840cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.579+0000 7f33892e4640 1 -- 172.21.15.84:0/2165183125 >> v1:172.21.15.157:6789/0 conn(0x7f3384160800 legacy=0x7f338406adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:13.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.579+0000 7f33892e4640 1 -- 172.21.15.84:0/2165183125 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338406e3c0 con 0x7f3384160420 2023-10-16T09:50:13.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.581+0000 7f33892e4640 1 --2- 172.21.15.84:0/2165183125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3384160420 0x7f33840ae170 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3380009870 tx=0x7f338002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de31e0ebc0a78c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:13.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.581+0000 7f337bfff640 1 -- 172.21.15.84:0/2165183125 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3380031640 con 0x7f3384160420 2023-10-16T09:50:13.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.581+0000 7f337bfff640 1 -- 172.21.15.84:0/2165183125 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33800317a0 con 0x7f3384160420 2023-10-16T09:50:13.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.581+0000 7f337bfff640 1 -- 172.21.15.84:0/2165183125 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3380031b30 con 0x7f3384160420 2023-10-16T09:50:13.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.582+0000 7f338b548640 1 -- 172.21.15.84:0/2165183125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3384160420 msgr2=0x7f33840ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:13.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.582+0000 7f338b548640 1 --2- 172.21.15.84:0/2165183125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3384160420 0x7f33840ae170 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3380009870 tx=0x7f338002b8f0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.582+0000 7f338b548640 1 -- 172.21.15.84:0/2165183125 shutdown_connections 2023-10-16T09:50:13.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.582+0000 7f338b548640 1 --2- 172.21.15.84:0/2165183125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f338406b4c0 0x7f33840cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.582+0000 7f338b548640 1 --2- 172.21.15.84:0/2165183125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3384160420 0x7f33840ae170 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.582+0000 7f338b548640 1 -- 172.21.15.84:0/2165183125 >> 172.21.15.84:0/2165183125 conn(0x7f33840b0d30 msgr2=0x7f33840b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:13.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.582+0000 7f338b548640 1 -- 172.21.15.84:0/2165183125 shutdown_connections 2023-10-16T09:50:13.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.582+0000 7f338b548640 1 -- 172.21.15.84:0/2165183125 wait complete. 2023-10-16T09:50:13.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.584+0000 7f338b548640 1 Processor -- start 2023-10-16T09:50:13.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.584+0000 7f338b548640 1 -- start start 2023-10-16T09:50:13.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.585+0000 7f338b548640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f338406b4c0 0x7f33840c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:13.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.585+0000 7f338b548640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3384160420 0x7f33840b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:13.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.585+0000 7f338b548640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3384160800 0x7f33840ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:13.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.585+0000 7f338b548640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338409f4b0 con 0x7f338406b4c0 2023-10-16T09:50:13.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.585+0000 7f338b548640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33840b4af0 con 0x7f3384160800 2023-10-16T09:50:13.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.585+0000 7f338b548640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33840b1b30 con 0x7f3384160420 2023-10-16T09:50:13.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.585+0000 7f33892e4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f338406b4c0 0x7f33840c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:13.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.585+0000 7f33892e4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f338406b4c0 0x7f33840c8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47066/0 (socket says 172.21.15.84:47066) 2023-10-16T09:50:13.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.585+0000 7f33892e4640 1 -- 172.21.15.84:0/1438867942 learned_addr learned my addr 172.21.15.84:0/1438867942 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:13.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.585+0000 7f33892e4640 1 -- 172.21.15.84:0/1438867942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3384160420 msgr2=0x7f33840b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:13.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.585+0000 7f33892e4640 1 --2- 172.21.15.84:0/1438867942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3384160420 0x7f33840b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.586+0000 7f33892e4640 1 -- 172.21.15.84:0/1438867942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3384160800 msgr2=0x7f33840ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:13.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.586+0000 7f33892e4640 1 --2- 172.21.15.84:0/1438867942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3384160800 0x7f33840ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.586+0000 7f33892e4640 1 -- 172.21.15.84:0/1438867942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33800094d0 con 0x7f338406b4c0 2023-10-16T09:50:13.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.586+0000 7f33892e4640 1 --2- 172.21.15.84:0/1438867942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f338406b4c0 0x7f33840c8f50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3380009870 tx=0x7f33800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:13.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.586+0000 7f3379ffb640 1 -- 172.21.15.84:0/1438867942 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338003ac60 con 0x7f338406b4c0 2023-10-16T09:50:13.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.586+0000 7f3379ffb640 1 -- 172.21.15.84:0/1438867942 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338003adc0 con 0x7f338406b4c0 2023-10-16T09:50:13.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.586+0000 7f338b548640 1 -- 172.21.15.84:0/1438867942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338409e830 con 0x7f338406b4c0 2023-10-16T09:50:13.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.586+0000 7f3379ffb640 1 -- 172.21.15.84:0/1438867942 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3380044630 con 0x7f338406b4c0 2023-10-16T09:50:13.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.586+0000 7f338b548640 1 -- 172.21.15.84:0/1438867942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338409edf0 con 0x7f338406b4c0 2023-10-16T09:50:13.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.587+0000 7f338b548640 1 -- 172.21.15.84:0/1438867942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33840c7770 con 0x7f338406b4c0 2023-10-16T09:50:13.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.588+0000 7f3379ffb640 1 -- 172.21.15.84:0/1438867942 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f338002f050 con 0x7f338406b4c0 2023-10-16T09:50:13.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.589+0000 7f3379ffb640 1 --2- 172.21.15.84:0/1438867942 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f336806ae70 0x7f336806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:13.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.589+0000 7f3379ffb640 1 -- 172.21.15.84:0/1438867942 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33800ab580 con 0x7f338406b4c0 2023-10-16T09:50:13.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.589+0000 7f3388ae3640 1 --2- 172.21.15.84:0/1438867942 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f336806ae70 0x7f336806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:13.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.590+0000 7f3388ae3640 1 --2- 172.21.15.84:0/1438867942 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f336806ae70 0x7f336806d330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3374006fd0 tx=0x7f3374006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:13.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.593+0000 7f3379ffb640 1 -- 172.21.15.84:0/1438867942 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3380076590 con 0x7f338406b4c0 2023-10-16T09:50:13.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.786+0000 7f338b548640 1 -- 172.21.15.84:0/1438867942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f338404e160 con 0x7f338406b4c0 2023-10-16T09:50:13.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.788+0000 7f3379ffb640 1 -- 172.21.15.84:0/1438867942 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16545 (secure 0 0 0) 0x7f3380031e60 con 0x7f338406b4c0 2023-10-16T09:50:13.790 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:50:13.791 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":34,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:50:13.553433+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:50:10.297512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-16T09:50:11.394493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:50:13.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.790+0000 7f338b548640 1 -- 172.21.15.84:0/1438867942 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f336806ae70 msgr2=0x7f336806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:13.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.790+0000 7f338b548640 1 --2- 172.21.15.84:0/1438867942 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f336806ae70 0x7f336806d330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3374006fd0 tx=0x7f3374006bc0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.790+0000 7f338b548640 1 -- 172.21.15.84:0/1438867942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f338406b4c0 msgr2=0x7f33840c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:13.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.790+0000 7f338b548640 1 --2- 172.21.15.84:0/1438867942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f338406b4c0 0x7f33840c8f50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3380009870 tx=0x7f33800046c0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.791+0000 7f338b548640 1 -- 172.21.15.84:0/1438867942 shutdown_connections 2023-10-16T09:50:13.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.791+0000 7f338b548640 1 --2- 172.21.15.84:0/1438867942 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f336806ae70 0x7f336806d330 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.791+0000 7f338b548640 1 --2- 172.21.15.84:0/1438867942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3384160800 0x7f33840ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.791+0000 7f338b548640 1 --2- 172.21.15.84:0/1438867942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3384160420 0x7f33840b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.791+0000 7f338b548640 1 --2- 172.21.15.84:0/1438867942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f338406b4c0 0x7f33840c8f50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:13.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.791+0000 7f338b548640 1 -- 172.21.15.84:0/1438867942 >> 172.21.15.84:0/1438867942 conn(0x7f33840b0d30 msgr2=0x7f33840cdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:13.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.791+0000 7f338b548640 1 -- 172.21.15.84:0/1438867942 shutdown_connections 2023-10-16T09:50:13.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:13.791+0000 7f338b548640 1 -- 172.21.15.84:0/1438867942 wait complete. 2023-10-16T09:50:13.813 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T09:50:14.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.020+0000 7ffbaa9e9640 1 Processor -- start 2023-10-16T09:50:14.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.020+0000 7ffbaa9e9640 1 -- start start 2023-10-16T09:50:14.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.021+0000 7ffbaa9e9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ffba406a4b0 0x7ffba40c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:14.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.022+0000 7ffbaa9e9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba40ac890 con 0x7ffba40ad350 2023-10-16T09:50:14.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.022+0000 7ffbaa9e9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba40ad7a0 con 0x7ffba415d150 2023-10-16T09:50:14.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.022+0000 7ffbaa9e9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba415d530 con 0x7ffba406a4b0 2023-10-16T09:50:14.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.022+0000 7ffba37fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ffba40ad350 0x7ffba4069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48634/0 (socket says 172.21.15.84:48634) 2023-10-16T09:50:14.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.022+0000 7ffba37fe640 1 -- 172.21.15.84:0/736813415 learned_addr learned my addr 172.21.15.84:0/736813415 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:14.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.022+0000 7ffba8f86640 1 --2- 172.21.15.84:0/736813415 >> v2:172.21.15.84:3300/0 conn(0x7ffba406a4b0 0x7ffba40c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:14.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.022+0000 7ffba8f86640 1 -- 172.21.15.84:0/736813415 >> v1:172.21.15.157:6789/0 conn(0x7ffba415d150 legacy=0x7ffba4160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:14.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.022+0000 7ffba8f86640 1 -- 172.21.15.84:0/736813415 >> v1:172.21.15.84:6789/0 conn(0x7ffba40ad350 legacy=0x7ffba4069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:14.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.023+0000 7ffba8f86640 1 -- 172.21.15.84:0/736813415 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba4161650 con 0x7ffba406a4b0 2023-10-16T09:50:14.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.022+0000 7ffba2ffd640 1 -- 172.21.15.84:0/736813415 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898080080 0 0) 0x7ffba40ac890 con 0x7ffba40ad350 2023-10-16T09:50:14.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.024+0000 7ffba8f86640 1 --2- 172.21.15.84:0/736813415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffba406a4b0 0x7ffba40c9000 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009d90 tx=0x7ffb9c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1647242658107f22 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:14.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.024+0000 7ffba2ffd640 1 -- 172.21.15.84:0/736813415 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c02cc60 con 0x7ffba406a4b0 2023-10-16T09:50:14.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.024+0000 7ffba2ffd640 1 -- 172.21.15.84:0/736813415 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9c02cdc0 con 0x7ffba406a4b0 2023-10-16T09:50:14.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.024+0000 7ffba2ffd640 1 -- 172.21.15.84:0/736813415 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c034660 con 0x7ffba406a4b0 2023-10-16T09:50:14.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.024+0000 7ffbaa9e9640 1 -- 172.21.15.84:0/736813415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffba406a4b0 msgr2=0x7ffba40c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:14.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.024+0000 7ffbaa9e9640 1 --2- 172.21.15.84:0/736813415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffba406a4b0 0x7ffba40c9000 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009d90 tx=0x7ffb9c02beb0 comp rx=0 tx=0).stop 2023-10-16T09:50:14.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.025+0000 7ffbaa9e9640 1 -- 172.21.15.84:0/736813415 shutdown_connections 2023-10-16T09:50:14.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.025+0000 7ffbaa9e9640 1 --2- 172.21.15.84:0/736813415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffba406a4b0 0x7ffba40c9000 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:14.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.025+0000 7ffbaa9e9640 1 -- 172.21.15.84:0/736813415 >> 172.21.15.84:0/736813415 conn(0x7ffba40ce9b0 msgr2=0x7ffba40cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:14.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.025+0000 7ffbaa9e9640 1 -- 172.21.15.84:0/736813415 shutdown_connections 2023-10-16T09:50:14.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.025+0000 7ffbaa9e9640 1 -- 172.21.15.84:0/736813415 wait complete. 2023-10-16T09:50:14.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.026+0000 7ffbaa9e9640 1 Processor -- start 2023-10-16T09:50:14.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.027+0000 7ffbaa9e9640 1 -- start start 2023-10-16T09:50:14.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.027+0000 7ffbaa9e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba406a4b0 0x7ffba41550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:14.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.027+0000 7ffbaa9e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffba40ad350 0x7ffba40b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:14.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.028+0000 7ffbaa9e9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba415d150 0x7ffba41e7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:14.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.028+0000 7ffbaa9e9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba4156090 con 0x7ffba40ad350 2023-10-16T09:50:14.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.028+0000 7ffbaa9e9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba40c5cb0 con 0x7ffba406a4b0 2023-10-16T09:50:14.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.028+0000 7ffbaa9e9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba40b32e0 con 0x7ffba415d150 2023-10-16T09:50:14.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.028+0000 7ffba37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffba40ad350 0x7ffba40b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:14.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.028+0000 7ffba37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffba40ad350 0x7ffba40b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47086/0 (socket says 172.21.15.84:47086) 2023-10-16T09:50:14.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.028+0000 7ffba37fe640 1 -- 172.21.15.84:0/2325923244 learned_addr learned my addr 172.21.15.84:0/2325923244 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:14.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.028+0000 7ffba8f86640 1 --2- 172.21.15.84:0/2325923244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba415d150 0x7ffba41e7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:14.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.028+0000 7ffba3fff640 1 --2- 172.21.15.84:0/2325923244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba406a4b0 0x7ffba41550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:14.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.028+0000 7ffba37fe640 1 -- 172.21.15.84:0/2325923244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba415d150 msgr2=0x7ffba41e7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:14.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.028+0000 7ffba37fe640 1 --2- 172.21.15.84:0/2325923244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba415d150 0x7ffba41e7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:14.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.028+0000 7ffba37fe640 1 -- 172.21.15.84:0/2325923244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba406a4b0 msgr2=0x7ffba41550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:14.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.028+0000 7ffba37fe640 1 --2- 172.21.15.84:0/2325923244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba406a4b0 0x7ffba41550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:14.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.028+0000 7ffba37fe640 1 -- 172.21.15.84:0/2325923244 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c009a70 con 0x7ffba40ad350 2023-10-16T09:50:14.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.028+0000 7ffba8f86640 1 --2- 172.21.15.84:0/2325923244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba415d150 0x7ffba41e7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:14.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.028+0000 7ffba3fff640 1 --2- 172.21.15.84:0/2325923244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba406a4b0 0x7ffba41550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:14.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.029+0000 7ffba37fe640 1 --2- 172.21.15.84:0/2325923244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffba40ad350 0x7ffba40b3de0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800c4d0 tx=0x7ffb9800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:14.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.029+0000 7ffba0ff9640 1 -- 172.21.15.84:0/2325923244 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb980074e0 con 0x7ffba40ad350 2023-10-16T09:50:14.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.029+0000 7ffba0ff9640 1 -- 172.21.15.84:0/2325923244 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb98007660 con 0x7ffba40ad350 2023-10-16T09:50:14.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.029+0000 7ffba0ff9640 1 -- 172.21.15.84:0/2325923244 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98004a20 con 0x7ffba40ad350 2023-10-16T09:50:14.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.029+0000 7ffbaa9e9640 1 -- 172.21.15.84:0/2325923244 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba4155610 con 0x7ffba40ad350 2023-10-16T09:50:14.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.029+0000 7ffbaa9e9640 1 -- 172.21.15.84:0/2325923244 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba40b2d20 con 0x7ffba40ad350 2023-10-16T09:50:14.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.030+0000 7ffbaa9e9640 1 -- 172.21.15.84:0/2325923244 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba40c5e20 con 0x7ffba40ad350 2023-10-16T09:50:14.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.031+0000 7ffba0ff9640 1 -- 172.21.15.84:0/2325923244 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ffb98004b80 con 0x7ffba40ad350 2023-10-16T09:50:14.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.032+0000 7ffba0ff9640 1 --2- 172.21.15.84:0/2325923244 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffb9006ae70 0x7ffb9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:14.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.032+0000 7ffba0ff9640 1 -- 172.21.15.84:0/2325923244 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb98088720 con 0x7ffba40ad350 2023-10-16T09:50:14.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.032+0000 7ffba3fff640 1 --2- 172.21.15.84:0/2325923244 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffb9006ae70 0x7ffb9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:14.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.033+0000 7ffba3fff640 1 --2- 172.21.15.84:0/2325923244 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffb9006ae70 0x7ffb9006d330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffba40b14a0 tx=0x7ffb94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:14.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.036+0000 7ffba0ff9640 1 -- 172.21.15.84:0/2325923244 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffb98053860 con 0x7ffba40ad350 2023-10-16T09:50:14.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.269+0000 7ffbaa9e9640 1 -- 172.21.15.84:0/2325923244 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffba404e160 con 0x7ffba40ad350 2023-10-16T09:50:14.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.945+0000 7ffba0ff9640 1 -- 172.21.15.84:0/2325923244 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffb980180b0 con 0x7ffba40ad350 2023-10-16T09:50:14.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.949+0000 7ffbaa9e9640 1 -- 172.21.15.84:0/2325923244 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffb9006ae70 msgr2=0x7ffb9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:14.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.949+0000 7ffbaa9e9640 1 --2- 172.21.15.84:0/2325923244 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffb9006ae70 0x7ffb9006d330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffba40b14a0 tx=0x7ffb94009040 comp rx=0 tx=0).stop 2023-10-16T09:50:14.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.949+0000 7ffbaa9e9640 1 -- 172.21.15.84:0/2325923244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffba40ad350 msgr2=0x7ffba40b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:14.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.949+0000 7ffbaa9e9640 1 --2- 172.21.15.84:0/2325923244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffba40ad350 0x7ffba40b3de0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800c4d0 tx=0x7ffb9800c990 comp rx=0 tx=0).stop 2023-10-16T09:50:14.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.949+0000 7ffbaa9e9640 1 -- 172.21.15.84:0/2325923244 shutdown_connections 2023-10-16T09:50:14.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.949+0000 7ffbaa9e9640 1 --2- 172.21.15.84:0/2325923244 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffb9006ae70 0x7ffb9006d330 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:14.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.949+0000 7ffbaa9e9640 1 --2- 172.21.15.84:0/2325923244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba415d150 0x7ffba41e7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:14.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.949+0000 7ffbaa9e9640 1 --2- 172.21.15.84:0/2325923244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffba40ad350 0x7ffba40b3de0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:14.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.949+0000 7ffbaa9e9640 1 --2- 172.21.15.84:0/2325923244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba406a4b0 0x7ffba41550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:14.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.949+0000 7ffbaa9e9640 1 -- 172.21.15.84:0/2325923244 >> 172.21.15.84:0/2325923244 conn(0x7ffba40ce9b0 msgr2=0x7ffba406f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:14.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.950+0000 7ffbaa9e9640 1 -- 172.21.15.84:0/2325923244 shutdown_connections 2023-10-16T09:50:14.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:14.950+0000 7ffbaa9e9640 1 -- 172.21.15.84:0/2325923244 wait complete. 2023-10-16T09:50:14.972 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T09:50:14.972 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T09:50:15.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.188+0000 7f006359e640 1 Processor -- start 2023-10-16T09:50:15.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.188+0000 7f006359e640 1 -- start start 2023-10-16T09:50:15.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.190+0000 7f006359e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f006415c9f0 0x7f00640c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:15.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.190+0000 7f006359e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f00640c9230 0x7f00640709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:15.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.190+0000 7f006359e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0064160a90 con 0x7f0064070f20 2023-10-16T09:50:15.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.190+0000 7f006359e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00640cdec0 con 0x7f006415c9f0 2023-10-16T09:50:15.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.190+0000 7f006359e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00640cd900 con 0x7f00640c9230 2023-10-16T09:50:15.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.190+0000 7f006259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f006415c9f0 0x7f00640c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:15.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.190+0000 7f0062d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0064070f20 0x7f00640ac890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48644/0 (socket says 172.21.15.84:48644) 2023-10-16T09:50:15.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.190+0000 7f0062d9d640 1 -- 172.21.15.84:0/45872352 learned_addr learned my addr 172.21.15.84:0/45872352 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:15.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.191+0000 7f0059d9b640 1 --2- 172.21.15.84:0/45872352 >> v2:172.21.15.157:3300/0 conn(0x7f00640c9230 0x7f00640709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:15.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.191+0000 7f0061d9b640 1 -- 172.21.15.84:0/45872352 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877981775 0 0) 0x7f0064160a90 con 0x7f0064070f20 2023-10-16T09:50:15.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.191+0000 7f0061d9b640 1 -- 172.21.15.84:0/45872352 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0050003520 con 0x7f0064070f20 2023-10-16T09:50:15.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.191+0000 7f006259c640 1 -- 172.21.15.84:0/45872352 >> v2:172.21.15.157:3300/0 conn(0x7f00640c9230 msgr2=0x7f00640709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:15.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.191+0000 7f006259c640 1 --2- 172.21.15.84:0/45872352 >> v2:172.21.15.157:3300/0 conn(0x7f00640c9230 0x7f00640709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:15.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.191+0000 7f006259c640 1 -- 172.21.15.84:0/45872352 >> v1:172.21.15.84:6789/0 conn(0x7f0064070f20 legacy=0x7f00640ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:15.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.191+0000 7f006259c640 1 -- 172.21.15.84:0/45872352 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00640cdbe0 con 0x7f006415c9f0 2023-10-16T09:50:15.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.192+0000 7f006259c640 1 --2- 172.21.15.84:0/45872352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f006415c9f0 0x7f00640c8cf0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f00640b1260 tx=0x7f005402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1905ad61a35d9fa6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:15.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.192+0000 7f0061d9b640 1 -- 172.21.15.84:0/45872352 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005402cc70 con 0x7f006415c9f0 2023-10-16T09:50:15.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.192+0000 7f0061d9b640 1 -- 172.21.15.84:0/45872352 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005402cdd0 con 0x7f006415c9f0 2023-10-16T09:50:15.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.192+0000 7f0061d9b640 1 -- 172.21.15.84:0/45872352 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0054034660 con 0x7f006415c9f0 2023-10-16T09:50:15.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.193+0000 7f006359e640 1 -- 172.21.15.84:0/45872352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f006415c9f0 msgr2=0x7f00640c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:15.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.193+0000 7f006359e640 1 --2- 172.21.15.84:0/45872352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f006415c9f0 0x7f00640c8cf0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f00640b1260 tx=0x7f005402bec0 comp rx=0 tx=0).stop 2023-10-16T09:50:15.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.193+0000 7f006359e640 1 -- 172.21.15.84:0/45872352 shutdown_connections 2023-10-16T09:50:15.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.193+0000 7f006359e640 1 --2- 172.21.15.84:0/45872352 >> v2:172.21.15.157:3300/0 conn(0x7f00640c9230 0x7f00640709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:15.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.193+0000 7f006359e640 1 --2- 172.21.15.84:0/45872352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f006415c9f0 0x7f00640c8cf0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:15.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.193+0000 7f006359e640 1 -- 172.21.15.84:0/45872352 >> 172.21.15.84:0/45872352 conn(0x7f00640ce6a0 msgr2=0x7f00640ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:15.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.193+0000 7f006359e640 1 -- 172.21.15.84:0/45872352 shutdown_connections 2023-10-16T09:50:15.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.193+0000 7f006359e640 1 -- 172.21.15.84:0/45872352 wait complete. 2023-10-16T09:50:15.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.195+0000 7f006359e640 1 Processor -- start 2023-10-16T09:50:15.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.195+0000 7f006359e640 1 -- start start 2023-10-16T09:50:15.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.196+0000 7f006359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0064070f20 0x7f00640c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:15.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.196+0000 7f006359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00640c9230 0x7f00640c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:15.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.196+0000 7f006359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f006415c9f0 0x7f00641496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:15.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.196+0000 7f006359e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00640c3630 con 0x7f00640c9230 2023-10-16T09:50:15.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.196+0000 7f006359e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00640c2710 con 0x7f0064070f20 2023-10-16T09:50:15.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.196+0000 7f006359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00640c29b0 con 0x7f006415c9f0 2023-10-16T09:50:15.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.196+0000 7f0059d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00640c9230 0x7f00640c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:15.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.196+0000 7f0059d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00640c9230 0x7f00640c7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47114/0 (socket says 172.21.15.84:47114) 2023-10-16T09:50:15.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.196+0000 7f0059d9b640 1 -- 172.21.15.84:0/3868892236 learned_addr learned my addr 172.21.15.84:0/3868892236 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:15.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.197+0000 7f0059d9b640 1 -- 172.21.15.84:0/3868892236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f006415c9f0 msgr2=0x7f00641496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:15.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.197+0000 7f0062d9d640 1 --2- 172.21.15.84:0/3868892236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f006415c9f0 0x7f00641496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:15.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.197+0000 7f006259c640 1 --2- 172.21.15.84:0/3868892236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0064070f20 0x7f00640c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:15.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.197+0000 7f0059d9b640 1 --2- 172.21.15.84:0/3868892236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f006415c9f0 0x7f00641496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:15.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.197+0000 7f0059d9b640 1 -- 172.21.15.84:0/3868892236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0064070f20 msgr2=0x7f00640c21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:15.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.197+0000 7f0059d9b640 1 --2- 172.21.15.84:0/3868892236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0064070f20 0x7f00640c21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:15.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.197+0000 7f0059d9b640 1 -- 172.21.15.84:0/3868892236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0054009a70 con 0x7f00640c9230 2023-10-16T09:50:15.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.197+0000 7f0062d9d640 1 --2- 172.21.15.84:0/3868892236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f006415c9f0 0x7f00641496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:15.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.197+0000 7f0059d9b640 1 --2- 172.21.15.84:0/3868892236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00640c9230 0x7f00640c7bb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f004c00d800 tx=0x7f004c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:15.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.197+0000 7f005b7fe640 1 -- 172.21.15.84:0/3868892236 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c00b6c0 con 0x7f00640c9230 2023-10-16T09:50:15.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.197+0000 7f005b7fe640 1 -- 172.21.15.84:0/3868892236 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c00b820 con 0x7f00640c9230 2023-10-16T09:50:15.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.197+0000 7f006359e640 1 -- 172.21.15.84:0/3868892236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00640c2c90 con 0x7f00640c9230 2023-10-16T09:50:15.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.197+0000 7f005b7fe640 1 -- 172.21.15.84:0/3868892236 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c00bbb0 con 0x7f00640c9230 2023-10-16T09:50:15.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.198+0000 7f006359e640 1 -- 172.21.15.84:0/3868892236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00640c3250 con 0x7f00640c9230 2023-10-16T09:50:15.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.199+0000 7f006359e640 1 -- 172.21.15.84:0/3868892236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006404e9b0 con 0x7f00640c9230 2023-10-16T09:50:15.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.200+0000 7f005b7fe640 1 -- 172.21.15.84:0/3868892236 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f004c00bd10 con 0x7f00640c9230 2023-10-16T09:50:15.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.201+0000 7f005b7fe640 1 --2- 172.21.15.84:0/3868892236 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f004406ae70 0x7f004406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:15.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.201+0000 7f005b7fe640 1 -- 172.21.15.84:0/3868892236 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f004c089b70 con 0x7f00640c9230 2023-10-16T09:50:15.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.201+0000 7f006259c640 1 --2- 172.21.15.84:0/3868892236 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f004406ae70 0x7f004406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:15.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.202+0000 7f006259c640 1 --2- 172.21.15.84:0/3868892236 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f004406ae70 0x7f004406d330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0054036000 tx=0x7f0054030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:15.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.206+0000 7f005b7fe640 1 -- 172.21.15.84:0/3868892236 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f004c054c20 con 0x7f00640c9230 2023-10-16T09:50:15.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.436+0000 7f006359e640 1 -- 172.21.15.84:0/3868892236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f00640cd4f0 con 0x7f00640c9230 2023-10-16T09:50:15.885 INFO:tasks.ceph.mds.b.smithi157.stderr:2023-10-16T09:50:15.883+0000 7f7ca4c77640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T09:50:15.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.949+0000 7f005b7fe640 1 -- 172.21.15.84:0/3868892236 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f004c058240 con 0x7f00640c9230 2023-10-16T09:50:15.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.954+0000 7f006359e640 1 -- 172.21.15.84:0/3868892236 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f004406ae70 msgr2=0x7f004406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:15.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.954+0000 7f006359e640 1 --2- 172.21.15.84:0/3868892236 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f004406ae70 0x7f004406d330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0054036000 tx=0x7f0054030000 comp rx=0 tx=0).stop 2023-10-16T09:50:15.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.954+0000 7f006359e640 1 -- 172.21.15.84:0/3868892236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00640c9230 msgr2=0x7f00640c7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:15.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.954+0000 7f006359e640 1 --2- 172.21.15.84:0/3868892236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00640c9230 0x7f00640c7bb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f004c00d800 tx=0x7f004c00dcc0 comp rx=0 tx=0).stop 2023-10-16T09:50:15.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.954+0000 7f006359e640 1 -- 172.21.15.84:0/3868892236 shutdown_connections 2023-10-16T09:50:15.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.954+0000 7f006359e640 1 --2- 172.21.15.84:0/3868892236 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f004406ae70 0x7f004406d330 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:15.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.954+0000 7f006359e640 1 --2- 172.21.15.84:0/3868892236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f006415c9f0 0x7f00641496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:15.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.954+0000 7f006359e640 1 --2- 172.21.15.84:0/3868892236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00640c9230 0x7f00640c7bb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:15.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.954+0000 7f006359e640 1 --2- 172.21.15.84:0/3868892236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0064070f20 0x7f00640c21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:15.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.954+0000 7f006359e640 1 -- 172.21.15.84:0/3868892236 >> 172.21.15.84:0/3868892236 conn(0x7f00640ce6a0 msgr2=0x7f006415b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:15.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.954+0000 7f006359e640 1 -- 172.21.15.84:0/3868892236 shutdown_connections 2023-10-16T09:50:15.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:15.954+0000 7f006359e640 1 -- 172.21.15.84:0/3868892236 wait complete. 2023-10-16T09:50:15.979 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T09:50:15.979 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:50:16.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.196+0000 7f4b23002640 1 Processor -- start 2023-10-16T09:50:16.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.196+0000 7f4b23002640 1 -- start start 2023-10-16T09:50:16.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.197+0000 7f4b23002640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1c15aaa0 0x7f4b1c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:16.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.197+0000 7f4b23002640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4b1c15ae80 0x7f4b1c0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:16.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.198+0000 7f4b23002640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b1c15ed60 con 0x7f4b1c0ccaa0 2023-10-16T09:50:16.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.198+0000 7f4b23002640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b1c06b810 con 0x7f4b1c15ae80 2023-10-16T09:50:16.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.198+0000 7f4b23002640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c0707e0 con 0x7f4b1c15aaa0 2023-10-16T09:50:16.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.198+0000 7f4b2159f640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4b1c0ccaa0 0x7f4b1c070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33538/0 (socket says 172.21.15.84:33538) 2023-10-16T09:50:16.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.198+0000 7f4b2159f640 1 -- 172.21.15.84:0/1600742521 learned_addr learned my addr 172.21.15.84:0/1600742521 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:16.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.198+0000 7f4b20d9e640 1 --2- 172.21.15.84:0/1600742521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1c15aaa0 0x7f4b1c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:16.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.198+0000 7f4b1bfff640 1 --2- 172.21.15.84:0/1600742521 >> v2:172.21.15.157:3300/0 conn(0x7f4b1c15ae80 0x7f4b1c0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:16.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.198+0000 7f4b1b7fe640 1 -- 172.21.15.84:0/1600742521 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2641913308 0 0) 0x7f4b1c15ed60 con 0x7f4b1c0ccaa0 2023-10-16T09:50:16.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.198+0000 7f4b1b7fe640 1 -- 172.21.15.84:0/1600742521 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b0c003520 con 0x7f4b1c0ccaa0 2023-10-16T09:50:16.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.199+0000 7f4b1b7fe640 1 -- 172.21.15.84:0/1600742521 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3344759155 0 0) 0x7f4b0c003520 con 0x7f4b1c0ccaa0 2023-10-16T09:50:16.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.199+0000 7f4b1b7fe640 1 -- 172.21.15.84:0/1600742521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1c15aaa0 msgr2=0x7f4b1c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:16.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.199+0000 7f4b1b7fe640 1 --2- 172.21.15.84:0/1600742521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1c15aaa0 0x7f4b1c15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.199+0000 7f4b1b7fe640 1 -- 172.21.15.84:0/1600742521 >> v2:172.21.15.157:3300/0 conn(0x7f4b1c15ae80 msgr2=0x7f4b1c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:16.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.199+0000 7f4b1b7fe640 1 --2- 172.21.15.84:0/1600742521 >> v2:172.21.15.157:3300/0 conn(0x7f4b1c15ae80 0x7f4b1c0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.199+0000 7f4b1b7fe640 1 -- 172.21.15.84:0/1600742521 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c15ef90 con 0x7f4b1c0ccaa0 2023-10-16T09:50:16.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.199+0000 7f4b1b7fe640 1 -- 172.21.15.84:0/1600742521 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f4b140019f0 con 0x7f4b1c0ccaa0 2023-10-16T09:50:16.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.199+0000 7f4b1b7fe640 1 -- 172.21.15.84:0/1600742521 >> v1:172.21.15.84:6789/0 conn(0x7f4b1c0ccaa0 legacy=0x7f4b1c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:16.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.199+0000 7f4b1b7fe640 1 --2- 172.21.15.84:0/1600742521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b0c003970 0x7f4b0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:16.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.199+0000 7f4b1b7fe640 1 --2- 172.21.15.84:0/1600742521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b0c0047c0 0x7f4b0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:16.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.199+0000 7f4b1b7fe640 1 --2- 172.21.15.84:0/1600742521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1c15aaa0 0x7f4b0c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:16.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.199+0000 7f4b1b7fe640 1 -- 172.21.15.84:0/1600742521 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c06b810 con 0x7f4b0c0047c0 2023-10-16T09:50:16.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.199+0000 7f4b1b7fe640 1 -- 172.21.15.84:0/1600742521 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c0707e0 con 0x7f4b0c003970 2023-10-16T09:50:16.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.199+0000 7f4b1b7fe640 1 -- 172.21.15.84:0/1600742521 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c00b980 con 0x7f4b1c15aaa0 2023-10-16T09:50:16.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.199+0000 7f4b1bfff640 1 --2- 172.21.15.84:0/1600742521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b0c0047c0 0x7f4b0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:16.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.200+0000 7f4b20d9e640 1 --2- 172.21.15.84:0/1600742521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b0c003970 0x7f4b0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:16.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.200+0000 7f4b2159f640 1 --2- 172.21.15.84:0/1600742521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1c15aaa0 0x7f4b0c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:16.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.200+0000 7f4b1bfff640 1 -- 172.21.15.84:0/1600742521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1c15aaa0 msgr2=0x7f4b0c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:16.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.200+0000 7f4b1bfff640 1 --2- 172.21.15.84:0/1600742521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1c15aaa0 0x7f4b0c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.200+0000 7f4b1bfff640 1 -- 172.21.15.84:0/1600742521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b0c003970 msgr2=0x7f4b0c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:16.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.200+0000 7f4b1bfff640 1 --2- 172.21.15.84:0/1600742521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b0c003970 0x7f4b0c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.200+0000 7f4b1bfff640 1 -- 172.21.15.84:0/1600742521 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0c003520 con 0x7f4b0c0047c0 2023-10-16T09:50:16.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.201+0000 7f4b1bfff640 1 --2- 172.21.15.84:0/1600742521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b0c0047c0 0x7f4b0c004c40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009ac0 tx=0x7f4b0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5688db698bd7c85b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:16.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.201+0000 7f4b1b7fe640 1 -- 172.21.15.84:0/1600742521 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0802c710 con 0x7f4b0c0047c0 2023-10-16T09:50:16.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.201+0000 7f4b1b7fe640 1 -- 172.21.15.84:0/1600742521 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0802c870 con 0x7f4b0c0047c0 2023-10-16T09:50:16.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.201+0000 7f4b1b7fe640 1 -- 172.21.15.84:0/1600742521 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0802cc00 con 0x7f4b0c0047c0 2023-10-16T09:50:16.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.202+0000 7f4b23002640 1 -- 172.21.15.84:0/1600742521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b0c0047c0 msgr2=0x7f4b0c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:16.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.202+0000 7f4b23002640 1 --2- 172.21.15.84:0/1600742521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b0c0047c0 0x7f4b0c004c40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009ac0 tx=0x7f4b0c0057f0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.202+0000 7f4b2159f640 1 -- 172.21.15.84:0/1600742521 reap_dead start 2023-10-16T09:50:16.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.202+0000 7f4b23002640 1 -- 172.21.15.84:0/1600742521 shutdown_connections 2023-10-16T09:50:16.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.202+0000 7f4b23002640 1 -- 172.21.15.84:0/1600742521 >> 172.21.15.84:0/1600742521 conn(0x7f4b1c0cb410 msgr2=0x7f4b1c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:16.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.202+0000 7f4b23002640 1 -- 172.21.15.84:0/1600742521 shutdown_connections 2023-10-16T09:50:16.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.202+0000 7f4b23002640 1 -- 172.21.15.84:0/1600742521 wait complete. 2023-10-16T09:50:16.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.204+0000 7f4b23002640 1 Processor -- start 2023-10-16T09:50:16.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.204+0000 7f4b23002640 1 -- start start 2023-10-16T09:50:16.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.205+0000 7f4b23002640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b0c0047c0 0x7f4b1c0c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:16.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.205+0000 7f4b23002640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b1c14cc50 0x7f4b1c0afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:16.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.205+0000 7f4b23002640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1c14d0d0 0x7f4b1c0aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:16.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.205+0000 7f4b23002640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c157380 con 0x7f4b1c14cc50 2023-10-16T09:50:16.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.205+0000 7f4b23002640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c0c3c70 con 0x7f4b0c0047c0 2023-10-16T09:50:16.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.205+0000 7f4b23002640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c0c4640 con 0x7f4b1c14d0d0 2023-10-16T09:50:16.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.205+0000 7f4b1bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b1c14cc50 0x7f4b1c0afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:16.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.205+0000 7f4b1bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b1c14cc50 0x7f4b1c0afa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37546/0 (socket says 172.21.15.84:37546) 2023-10-16T09:50:16.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.205+0000 7f4b1bfff640 1 -- 172.21.15.84:0/2238230914 learned_addr learned my addr 172.21.15.84:0/2238230914 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:16.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.205+0000 7f4b2159f640 1 --2- 172.21.15.84:0/2238230914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1c14d0d0 0x7f4b1c0aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:16.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.205+0000 7f4b20d9e640 1 --2- 172.21.15.84:0/2238230914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b0c0047c0 0x7f4b1c0c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:16.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.205+0000 7f4b1bfff640 1 -- 172.21.15.84:0/2238230914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1c14d0d0 msgr2=0x7f4b1c0aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:16.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.205+0000 7f4b1bfff640 1 --2- 172.21.15.84:0/2238230914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1c14d0d0 0x7f4b1c0aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.205+0000 7f4b1bfff640 1 -- 172.21.15.84:0/2238230914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b0c0047c0 msgr2=0x7f4b1c0c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:16.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.206+0000 7f4b1bfff640 1 --2- 172.21.15.84:0/2238230914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b0c0047c0 0x7f4b1c0c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.206+0000 7f4b1bfff640 1 -- 172.21.15.84:0/2238230914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c0c3060 con 0x7f4b1c14cc50 2023-10-16T09:50:16.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.206+0000 7f4b20d9e640 1 --2- 172.21.15.84:0/2238230914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b0c0047c0 0x7f4b1c0c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:16.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.206+0000 7f4b1bfff640 1 --2- 172.21.15.84:0/2238230914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b1c14cc50 0x7f4b1c0afa40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4b0802c2b0 tx=0x7f4b08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:16.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.206+0000 7f4b197fa640 1 -- 172.21.15.84:0/2238230914 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b08003810 con 0x7f4b1c14cc50 2023-10-16T09:50:16.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.206+0000 7f4b197fa640 1 -- 172.21.15.84:0/2238230914 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0802de90 con 0x7f4b1c14cc50 2023-10-16T09:50:16.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.206+0000 7f4b197fa640 1 -- 172.21.15.84:0/2238230914 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b08003b70 con 0x7f4b1c14cc50 2023-10-16T09:50:16.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.206+0000 7f4b23002640 1 -- 172.21.15.84:0/2238230914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1c0c3340 con 0x7f4b1c14cc50 2023-10-16T09:50:16.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.206+0000 7f4b23002640 1 -- 172.21.15.84:0/2238230914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1c0c3900 con 0x7f4b1c14cc50 2023-10-16T09:50:16.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.207+0000 7f4b23002640 1 -- 172.21.15.84:0/2238230914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b1c0c3e20 con 0x7f4b1c14cc50 2023-10-16T09:50:16.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.208+0000 7f4b197fa640 1 -- 172.21.15.84:0/2238230914 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4b08003cd0 con 0x7f4b1c14cc50 2023-10-16T09:50:16.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.209+0000 7f4b197fa640 1 --2- 172.21.15.84:0/2238230914 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4b0006ae70 0x7f4b0006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:16.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.209+0000 7f4b197fa640 1 -- 172.21.15.84:0/2238230914 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b080ab670 con 0x7f4b1c14cc50 2023-10-16T09:50:16.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.210+0000 7f4b20d9e640 1 --2- 172.21.15.84:0/2238230914 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4b0006ae70 0x7f4b0006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:16.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.210+0000 7f4b20d9e640 1 --2- 172.21.15.84:0/2238230914 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4b0006ae70 0x7f4b0006d330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009910 tx=0x7f4b10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:16.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.214+0000 7f4b197fa640 1 -- 172.21.15.84:0/2238230914 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4b1c0c3e20 con 0x7f4b1c14cc50 2023-10-16T09:50:16.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.444+0000 7f4b23002640 1 -- 172.21.15.84:0/2238230914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b1c0c3de0 con 0x7f4b1c14cc50 2023-10-16T09:50:16.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.445+0000 7f4b197fa640 1 -- 172.21.15.84:0/2238230914 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f4b08076830 con 0x7f4b1c14cc50 2023-10-16T09:50:16.447 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:50:16.448 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3662158203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3662158203},{"type":"v1","addr":"172.21.15.157:6835","nonce":3662158203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5091,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2502120537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2502120537},{"type":"v1","addr":"172.21.15.84:6836","nonce":2502120537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2715651376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2715651376},{"type":"v1","addr":"172.21.15.84:6837","nonce":2715651376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:50:12.380180+0000","modified":"2023-10-16T09:50:15.438597+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6833/62459899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":62459899},{"type":"v1","addr":"172.21.15.157:6833","nonce":62459899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T09:50:16.448 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 17 2023-10-16T09:50:16.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.448+0000 7f4b23002640 1 -- 172.21.15.84:0/2238230914 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4b0006ae70 msgr2=0x7f4b0006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:16.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.448+0000 7f4b23002640 1 --2- 172.21.15.84:0/2238230914 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4b0006ae70 0x7f4b0006d330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009910 tx=0x7f4b10008040 comp rx=0 tx=0).stop 2023-10-16T09:50:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.448+0000 7f4b23002640 1 -- 172.21.15.84:0/2238230914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b1c14cc50 msgr2=0x7f4b1c0afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.448+0000 7f4b23002640 1 --2- 172.21.15.84:0/2238230914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b1c14cc50 0x7f4b1c0afa40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4b0802c2b0 tx=0x7f4b08004730 comp rx=0 tx=0).stop 2023-10-16T09:50:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.448+0000 7f4b23002640 1 -- 172.21.15.84:0/2238230914 shutdown_connections 2023-10-16T09:50:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.448+0000 7f4b23002640 1 --2- 172.21.15.84:0/2238230914 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f4b0006ae70 0x7f4b0006d330 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.448+0000 7f4b23002640 1 --2- 172.21.15.84:0/2238230914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1c14d0d0 0x7f4b1c0aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.448+0000 7f4b23002640 1 --2- 172.21.15.84:0/2238230914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b1c14cc50 0x7f4b1c0afa40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.448+0000 7f4b23002640 1 --2- 172.21.15.84:0/2238230914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b0c0047c0 0x7f4b1c0c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.448+0000 7f4b23002640 1 -- 172.21.15.84:0/2238230914 >> 172.21.15.84:0/2238230914 conn(0x7f4b1c0cb410 msgr2=0x7f4b1c0ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.448+0000 7f4b23002640 1 -- 172.21.15.84:0/2238230914 shutdown_connections 2023-10-16T09:50:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.449+0000 7f4b23002640 1 -- 172.21.15.84:0/2238230914 wait complete. 2023-10-16T09:50:16.468 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:50:16.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.675+0000 7f427c18a640 1 Processor -- start 2023-10-16T09:50:16.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.675+0000 7f427c18a640 1 -- start start 2023-10-16T09:50:16.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.677+0000 7f427c18a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4274159b20 0x7f427406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:16.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.677+0000 7f427c18a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f427406f7f0 con 0x7f4274159f00 2023-10-16T09:50:16.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.677+0000 7f427c18a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f427406c5e0 con 0x7f42740b0b20 2023-10-16T09:50:16.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.677+0000 7f427c18a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427406c8c0 con 0x7f4274159b20 2023-10-16T09:50:16.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.677+0000 7f4279725640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4274159f00 0x7f42740b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33548/0 (socket says 172.21.15.84:33548) 2023-10-16T09:50:16.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.677+0000 7f4279725640 1 -- 172.21.15.84:0/2625406705 learned_addr learned my addr 172.21.15.84:0/2625406705 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:16.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.677+0000 7f4279f26640 1 --2- 172.21.15.84:0/2625406705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4274159b20 0x7f427406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:16.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.677+0000 7f4278f24640 1 -- 172.21.15.84:0/2625406705 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684435811 0 0) 0x7f427406f7f0 con 0x7f4274159f00 2023-10-16T09:50:16.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.678+0000 7f4278f24640 1 -- 172.21.15.84:0/2625406705 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4260003520 con 0x7f4274159f00 2023-10-16T09:50:16.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.678+0000 7f4278f24640 1 -- 172.21.15.84:0/2625406705 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 959669628 0 0) 0x7f4260003520 con 0x7f4274159f00 2023-10-16T09:50:16.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.678+0000 7f4278f24640 1 -- 172.21.15.84:0/2625406705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4274159b20 msgr2=0x7f427406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:16.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.678+0000 7f4278f24640 1 --2- 172.21.15.84:0/2625406705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4274159b20 0x7f427406bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.678+0000 7f4278f24640 1 -- 172.21.15.84:0/2625406705 >> v1:172.21.15.157:6789/0 conn(0x7f42740b0b20 legacy=0x7f427406fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:16.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.678+0000 7f4278f24640 1 -- 172.21.15.84:0/2625406705 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427406cba0 con 0x7f4274159f00 2023-10-16T09:50:16.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.678+0000 7f4278f24640 1 -- 172.21.15.84:0/2625406705 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f4264001670 con 0x7f4274159f00 2023-10-16T09:50:16.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.678+0000 7f4278f24640 1 -- 172.21.15.84:0/2625406705 >> v1:172.21.15.84:6789/0 conn(0x7f4274159f00 legacy=0x7f42740b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:16.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.678+0000 7f4278f24640 1 --2- 172.21.15.84:0/2625406705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4274159b20 0x7f42600039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:16.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.678+0000 7f4278f24640 1 --2- 172.21.15.84:0/2625406705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4260003f20 0x7f42600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:16.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.678+0000 7f4278f24640 1 --2- 172.21.15.84:0/2625406705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4260004d00 0x7f4260005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:16.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.678+0000 7f4278f24640 1 -- 172.21.15.84:0/2625406705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427406c8c0 con 0x7f4260003f20 2023-10-16T09:50:16.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.678+0000 7f4278f24640 1 -- 172.21.15.84:0/2625406705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426000ba60 con 0x7f4260004d00 2023-10-16T09:50:16.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.678+0000 7f4278f24640 1 -- 172.21.15.84:0/2625406705 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426000de30 con 0x7f4274159b20 2023-10-16T09:50:16.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.678+0000 7f4279725640 1 --2- 172.21.15.84:0/2625406705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4260003f20 0x7f42600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:16.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.679+0000 7f427a727640 1 --2- 172.21.15.84:0/2625406705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4260004d00 0x7f4260005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:16.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.679+0000 7f4279f26640 1 --2- 172.21.15.84:0/2625406705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4274159b20 0x7f42600039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:16.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.679+0000 7f4279725640 1 -- 172.21.15.84:0/2625406705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4274159b20 msgr2=0x7f42600039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:16.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.679+0000 7f4279725640 1 --2- 172.21.15.84:0/2625406705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4274159b20 0x7f42600039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.679+0000 7f4279725640 1 -- 172.21.15.84:0/2625406705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4260004d00 msgr2=0x7f4260005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:16.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.679+0000 7f4279725640 1 --2- 172.21.15.84:0/2625406705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4260004d00 0x7f4260005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.679+0000 7f4279725640 1 -- 172.21.15.84:0/2625406705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4260003520 con 0x7f4260003f20 2023-10-16T09:50:16.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.680+0000 7f4279725640 1 --2- 172.21.15.84:0/2625406705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4260003f20 0x7f42600047c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f426400c210 tx=0x7f42600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68b1d3f34332b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:16.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.680+0000 7f4278f24640 1 -- 172.21.15.84:0/2625406705 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4264005ad0 con 0x7f4260003f20 2023-10-16T09:50:16.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.680+0000 7f4278f24640 1 -- 172.21.15.84:0/2625406705 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4264005c30 con 0x7f4260003f20 2023-10-16T09:50:16.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.680+0000 7f4278f24640 1 -- 172.21.15.84:0/2625406705 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426402cad0 con 0x7f4260003f20 2023-10-16T09:50:16.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.681+0000 7f427c18a640 1 -- 172.21.15.84:0/2625406705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4260003f20 msgr2=0x7f42600047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:16.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.681+0000 7f427c18a640 1 --2- 172.21.15.84:0/2625406705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4260003f20 0x7f42600047c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f426400c210 tx=0x7f42600057f0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.681+0000 7f427a727640 1 -- 172.21.15.84:0/2625406705 reap_dead start 2023-10-16T09:50:16.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.681+0000 7f427c18a640 1 -- 172.21.15.84:0/2625406705 shutdown_connections 2023-10-16T09:50:16.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.681+0000 7f427c18a640 1 -- 172.21.15.84:0/2625406705 >> 172.21.15.84:0/2625406705 conn(0x7f42740cb410 msgr2=0x7f42740cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:16.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.681+0000 7f427c18a640 1 -- 172.21.15.84:0/2625406705 shutdown_connections 2023-10-16T09:50:16.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.681+0000 7f427c18a640 1 -- 172.21.15.84:0/2625406705 wait complete. 2023-10-16T09:50:16.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.683+0000 7f427c18a640 1 Processor -- start 2023-10-16T09:50:16.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.683+0000 7f427c18a640 1 -- start start 2023-10-16T09:50:16.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.684+0000 7f427c18a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4260003f20 0x7f4274150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:16.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.684+0000 7f427c18a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4274151220 0x7f4274154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:16.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.684+0000 7f427c18a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42740cc5d0 0x7f42740cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:16.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.684+0000 7f427c18a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42740c8a40 con 0x7f42740cc5d0 2023-10-16T09:50:16.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.684+0000 7f427c18a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4274155080 con 0x7f4260003f20 2023-10-16T09:50:16.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.684+0000 7f427c18a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42740ccf90 con 0x7f4274151220 2023-10-16T09:50:16.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.684+0000 7f427a727640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42740cc5d0 0x7f42740cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:16.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.684+0000 7f427a727640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42740cc5d0 0x7f42740cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37558/0 (socket says 172.21.15.84:37558) 2023-10-16T09:50:16.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.684+0000 7f427a727640 1 -- 172.21.15.84:0/3583017052 learned_addr learned my addr 172.21.15.84:0/3583017052 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:16.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.685+0000 7f4279f26640 1 --2- 172.21.15.84:0/3583017052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4260003f20 0x7f4274150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:16.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.685+0000 7f4279725640 1 --2- 172.21.15.84:0/3583017052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4274151220 0x7f4274154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:16.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.685+0000 7f427a727640 1 -- 172.21.15.84:0/3583017052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4274151220 msgr2=0x7f4274154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:16.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.685+0000 7f427a727640 1 --2- 172.21.15.84:0/3583017052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4274151220 0x7f4274154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.685+0000 7f427a727640 1 -- 172.21.15.84:0/3583017052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4260003f20 msgr2=0x7f4274150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:16.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.685+0000 7f427a727640 1 --2- 172.21.15.84:0/3583017052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4260003f20 0x7f4274150cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.685+0000 7f427a727640 1 -- 172.21.15.84:0/3583017052 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4274155290 con 0x7f42740cc5d0 2023-10-16T09:50:16.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.685+0000 7f4279725640 1 --2- 172.21.15.84:0/3583017052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4274151220 0x7f4274154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:16.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.685+0000 7f427a727640 1 --2- 172.21.15.84:0/3583017052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42740cc5d0 0x7f42740cca50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f42740b4ad0 tx=0x7f426c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:16.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.685+0000 7f426a7fc640 1 -- 172.21.15.84:0/3583017052 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c003970 con 0x7f42740cc5d0 2023-10-16T09:50:16.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.685+0000 7f426a7fc640 1 -- 172.21.15.84:0/3583017052 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c003ad0 con 0x7f42740cc5d0 2023-10-16T09:50:16.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.685+0000 7f426a7fc640 1 -- 172.21.15.84:0/3583017052 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c003e80 con 0x7f42740cc5d0 2023-10-16T09:50:16.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.685+0000 7f427c18a640 1 -- 172.21.15.84:0/3583017052 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42740c7160 con 0x7f42740cc5d0 2023-10-16T09:50:16.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.686+0000 7f427c18a640 1 -- 172.21.15.84:0/3583017052 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42740b1dd0 con 0x7f42740cc5d0 2023-10-16T09:50:16.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.687+0000 7f427c18a640 1 -- 172.21.15.84:0/3583017052 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42740ad110 con 0x7f42740cc5d0 2023-10-16T09:50:16.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.687+0000 7f426a7fc640 1 -- 172.21.15.84:0/3583017052 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f426c0050d0 con 0x7f42740cc5d0 2023-10-16T09:50:16.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.688+0000 7f426a7fc640 1 --2- 172.21.15.84:0/3583017052 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f425806ae70 0x7f425806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:16.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.688+0000 7f426a7fc640 1 -- 172.21.15.84:0/3583017052 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f426c0893e0 con 0x7f42740cc5d0 2023-10-16T09:50:16.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.689+0000 7f4279f26640 1 --2- 172.21.15.84:0/3583017052 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f425806ae70 0x7f425806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:16.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.689+0000 7f4279f26640 1 --2- 172.21.15.84:0/3583017052 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f425806ae70 0x7f425806d330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4270009950 tx=0x7f4270008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:16.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.693+0000 7f426a7fc640 1 -- 172.21.15.84:0/3583017052 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f426c054480 con 0x7f42740cc5d0 2023-10-16T09:50:16.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.889+0000 7f427c18a640 1 -- 172.21.15.84:0/3583017052 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f427404e160 con 0x7f42740cc5d0 2023-10-16T09:50:16.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.892+0000 7f426a7fc640 1 -- 172.21.15.84:0/3583017052 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16545 (secure 0 0 0) 0x7f426c057aa0 con 0x7f42740cc5d0 2023-10-16T09:50:16.894 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:50:16.895 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":34,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T09:50:13.553433+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:50:10.297512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-16T09:50:11.394493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:50:16.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.895+0000 7f427c18a640 1 -- 172.21.15.84:0/3583017052 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f425806ae70 msgr2=0x7f425806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:16.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.895+0000 7f427c18a640 1 --2- 172.21.15.84:0/3583017052 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f425806ae70 0x7f425806d330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4270009950 tx=0x7f4270008040 comp rx=0 tx=0).stop 2023-10-16T09:50:16.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.895+0000 7f427c18a640 1 -- 172.21.15.84:0/3583017052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42740cc5d0 msgr2=0x7f42740cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:16.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.895+0000 7f427c18a640 1 --2- 172.21.15.84:0/3583017052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42740cc5d0 0x7f42740cca50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f42740b4ad0 tx=0x7f426c00ec10 comp rx=0 tx=0).stop 2023-10-16T09:50:16.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.895+0000 7f427c18a640 1 -- 172.21.15.84:0/3583017052 shutdown_connections 2023-10-16T09:50:16.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.895+0000 7f427c18a640 1 --2- 172.21.15.84:0/3583017052 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f425806ae70 0x7f425806d330 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.895+0000 7f427c18a640 1 --2- 172.21.15.84:0/3583017052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42740cc5d0 0x7f42740cca50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.895+0000 7f427c18a640 1 --2- 172.21.15.84:0/3583017052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4274151220 0x7f4274154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.895+0000 7f427c18a640 1 --2- 172.21.15.84:0/3583017052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4260003f20 0x7f4274150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:16.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.895+0000 7f427c18a640 1 -- 172.21.15.84:0/3583017052 >> 172.21.15.84:0/3583017052 conn(0x7f42740cb410 msgr2=0x7f42740b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:16.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.895+0000 7f427c18a640 1 -- 172.21.15.84:0/3583017052 shutdown_connections 2023-10-16T09:50:16.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:16.895+0000 7f427c18a640 1 -- 172.21.15.84:0/3583017052 wait complete. 2023-10-16T09:50:16.915 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T09:50:16.915 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:50:17.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.130+0000 7fc8a1aea640 1 Processor -- start 2023-10-16T09:50:17.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.130+0000 7fc8a1aea640 1 -- start start 2023-10-16T09:50:17.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.132+0000 7fc8a1aea640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc89c0b8480 0x7fc89c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:17.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.132+0000 7fc8a1aea640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc89c0d0be0 0x7fc89c0cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:17.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.132+0000 7fc8a1aea640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c15d150 con 0x7fc89c15cd70 2023-10-16T09:50:17.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.132+0000 7fc8a1aea640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89c0d1180 con 0x7fc89c0b8480 2023-10-16T09:50:17.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.132+0000 7fc8a1aea640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0d1460 con 0x7fc89c0d0be0 2023-10-16T09:50:17.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.132+0000 7fc89bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc89c0b8480 0x7fc89c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:17.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.132+0000 7fc89bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc89c0b8480 0x7fc89c0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37570/0 (socket says 172.21.15.84:37570) 2023-10-16T09:50:17.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.132+0000 7fc89bfff640 1 -- 172.21.15.84:0/2580670233 learned_addr learned my addr 172.21.15.84:0/2580670233 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:17.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.133+0000 7fc8a12e9640 1 --2- 172.21.15.84:0/2580670233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc89c0d0be0 0x7fc89c0cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:17.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.133+0000 7fc89bfff640 1 -- 172.21.15.84:0/2580670233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc89c0d0be0 msgr2=0x7fc89c0cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:17.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.133+0000 7fc89bfff640 1 --2- 172.21.15.84:0/2580670233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc89c0d0be0 0x7fc89c0cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:17.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.133+0000 7fc89bfff640 1 -- 172.21.15.84:0/2580670233 >> v1:172.21.15.157:6789/0 conn(0x7fc89c15cd70 legacy=0x7fc89c0d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:17.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.133+0000 7fc89bfff640 1 -- 172.21.15.84:0/2580670233 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c0d1740 con 0x7fc89c0b8480 2023-10-16T09:50:17.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.133+0000 7fc8a12e9640 1 --2- 172.21.15.84:0/2580670233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc89c0d0be0 0x7fc89c0cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:17.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.134+0000 7fc89bfff640 1 --2- 172.21.15.84:0/2580670233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c0b8480 0x7fc89c0cd7c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc884009e50 tx=0x7fc88402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9000abf97834c1e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:17.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.134+0000 7fc89b7fe640 1 -- 172.21.15.84:0/2580670233 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88402ca80 con 0x7fc89c0b8480 2023-10-16T09:50:17.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.134+0000 7fc89b7fe640 1 -- 172.21.15.84:0/2580670233 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88402cbe0 con 0x7fc89c0b8480 2023-10-16T09:50:17.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.134+0000 7fc89b7fe640 1 -- 172.21.15.84:0/2580670233 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8840343f0 con 0x7fc89c0b8480 2023-10-16T09:50:17.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.135+0000 7fc8a1aea640 1 -- 172.21.15.84:0/2580670233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c0b8480 msgr2=0x7fc89c0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:17.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.135+0000 7fc8a1aea640 1 --2- 172.21.15.84:0/2580670233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c0b8480 0x7fc89c0cd7c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc884009e50 tx=0x7fc88402bee0 comp rx=0 tx=0).stop 2023-10-16T09:50:17.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.135+0000 7fc8a1aea640 1 -- 172.21.15.84:0/2580670233 shutdown_connections 2023-10-16T09:50:17.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.135+0000 7fc8a1aea640 1 --2- 172.21.15.84:0/2580670233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc89c0d0be0 0x7fc89c0cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:17.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.135+0000 7fc8a1aea640 1 --2- 172.21.15.84:0/2580670233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c0b8480 0x7fc89c0cd7c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:17.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.135+0000 7fc8a1aea640 1 -- 172.21.15.84:0/2580670233 >> 172.21.15.84:0/2580670233 conn(0x7fc89c0cf570 msgr2=0x7fc89c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:17.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.135+0000 7fc8a1aea640 1 -- 172.21.15.84:0/2580670233 shutdown_connections 2023-10-16T09:50:17.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.135+0000 7fc8a1aea640 1 -- 172.21.15.84:0/2580670233 wait complete. 2023-10-16T09:50:17.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.137+0000 7fc8a1aea640 1 Processor -- start 2023-10-16T09:50:17.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.137+0000 7fc8a1aea640 1 -- start start 2023-10-16T09:50:17.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.138+0000 7fc8a1aea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c0b8480 0x7fc89c0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:17.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.138+0000 7fc8a1aea640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc89c0d0be0 0x7fc89c1f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.138+0000 7fc8a1aea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc89c15cd70 0x7fc89c157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.138+0000 7fc8a1aea640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c156f50 con 0x7fc89c0b8480 2023-10-16T09:50:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.138+0000 7fc8a1aea640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c159b90 con 0x7fc89c15cd70 2023-10-16T09:50:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.138+0000 7fc8a1aea640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c156c70 con 0x7fc89c0d0be0 2023-10-16T09:50:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.138+0000 7fc8a0ae8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c0b8480 0x7fc89c0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.138+0000 7fc8a0ae8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c0b8480 0x7fc89c0b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37584/0 (socket says 172.21.15.84:37584) 2023-10-16T09:50:17.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.138+0000 7fc8a0ae8640 1 -- 172.21.15.84:0/2950397350 learned_addr learned my addr 172.21.15.84:0/2950397350 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:17.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.139+0000 7fc89bfff640 1 --2- 172.21.15.84:0/2950397350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc89c0d0be0 0x7fc89c1f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:17.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.139+0000 7fc8a0ae8640 1 -- 172.21.15.84:0/2950397350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc89c0d0be0 msgr2=0x7fc89c1f1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:17.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.139+0000 7fc8a0ae8640 1 --2- 172.21.15.84:0/2950397350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc89c0d0be0 0x7fc89c1f1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:17.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.139+0000 7fc8a0ae8640 1 -- 172.21.15.84:0/2950397350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc89c15cd70 msgr2=0x7fc89c157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:17.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.139+0000 7fc8a0ae8640 1 --2- 172.21.15.84:0/2950397350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc89c15cd70 0x7fc89c157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.139+0000 7fc8a0ae8640 1 -- 172.21.15.84:0/2950397350 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc884009b30 con 0x7fc89c0b8480 2023-10-16T09:50:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.139+0000 7fc8a0ae8640 1 --2- 172.21.15.84:0/2950397350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c0b8480 0x7fc89c0b4680 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00c410 tx=0x7fc88c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.139+0000 7fc8997fa640 1 -- 172.21.15.84:0/2950397350 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c0035f0 con 0x7fc89c0b8480 2023-10-16T09:50:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.139+0000 7fc8997fa640 1 -- 172.21.15.84:0/2950397350 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c003750 con 0x7fc89c0b8480 2023-10-16T09:50:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.139+0000 7fc8a1aea640 1 -- 172.21.15.84:0/2950397350 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89c0b3ba0 con 0x7fc89c0b8480 2023-10-16T09:50:17.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.139+0000 7fc8997fa640 1 -- 172.21.15.84:0/2950397350 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c003b40 con 0x7fc89c0b8480 2023-10-16T09:50:17.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.139+0000 7fc8a1aea640 1 -- 172.21.15.84:0/2950397350 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c0b4160 con 0x7fc89c0b8480 2023-10-16T09:50:17.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.141+0000 7fc8a1aea640 1 -- 172.21.15.84:0/2950397350 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89c0d28f0 con 0x7fc89c0b8480 2023-10-16T09:50:17.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.142+0000 7fc8997fa640 1 -- 172.21.15.84:0/2950397350 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc88c003ca0 con 0x7fc89c0b8480 2023-10-16T09:50:17.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.144+0000 7fc8997fa640 1 --2- 172.21.15.84:0/2950397350 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc88006aea0 0x7fc88006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:17.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.144+0000 7fc8997fa640 1 -- 172.21.15.84:0/2950397350 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc88c0888f0 con 0x7fc89c0b8480 2023-10-16T09:50:17.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.144+0000 7fc89bfff640 1 --2- 172.21.15.84:0/2950397350 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc88006aea0 0x7fc88006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:17.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.145+0000 7fc89bfff640 1 --2- 172.21.15.84:0/2950397350 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc88006aea0 0x7fc88006d360 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc884035000 tx=0x7fc884030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:17.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.148+0000 7fc8997fa640 1 -- 172.21.15.84:0/2950397350 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc88c053900 con 0x7fc89c0b8480 2023-10-16T09:50:17.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.335+0000 7fc8a1aea640 1 -- 172.21.15.84:0/2950397350 --> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc89c0d1d00 con 0x7fc88006aea0 2023-10-16T09:50:17.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.347+0000 7fc8997fa640 1 -- 172.21.15.84:0/2950397350 <== mgr.4107 v2:172.21.15.84:6832/3010246666 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286730 (secure 0 0 0) 0x7fc8840343f0 con 0x7fc88006aea0 2023-10-16T09:50:17.352 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:50:17.354 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T09:50:17.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.352+0000 7fc8a1aea640 1 -- 172.21.15.84:0/2950397350 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc88006aea0 msgr2=0x7fc88006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:17.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.352+0000 7fc8a1aea640 1 --2- 172.21.15.84:0/2950397350 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc88006aea0 0x7fc88006d360 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc884035000 tx=0x7fc884030000 comp rx=0 tx=0).stop 2023-10-16T09:50:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.352+0000 7fc8a1aea640 1 -- 172.21.15.84:0/2950397350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c0b8480 msgr2=0x7fc89c0b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.352+0000 7fc8a1aea640 1 --2- 172.21.15.84:0/2950397350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c0b8480 0x7fc89c0b4680 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00c410 tx=0x7fc88c00c8d0 comp rx=0 tx=0).stop 2023-10-16T09:50:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.352+0000 7fc8a1aea640 1 -- 172.21.15.84:0/2950397350 shutdown_connections 2023-10-16T09:50:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.352+0000 7fc8a1aea640 1 --2- 172.21.15.84:0/2950397350 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc88006aea0 0x7fc88006d360 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.352+0000 7fc8a1aea640 1 --2- 172.21.15.84:0/2950397350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc89c15cd70 0x7fc89c157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.352+0000 7fc8a1aea640 1 --2- 172.21.15.84:0/2950397350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc89c0d0be0 0x7fc89c1f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.352+0000 7fc8a1aea640 1 --2- 172.21.15.84:0/2950397350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c0b8480 0x7fc89c0b4680 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.352+0000 7fc8a1aea640 1 -- 172.21.15.84:0/2950397350 >> 172.21.15.84:0/2950397350 conn(0x7fc89c0cf570 msgr2=0x7fc89c0b73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.352+0000 7fc8a1aea640 1 -- 172.21.15.84:0/2950397350 shutdown_connections 2023-10-16T09:50:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.352+0000 7fc8a1aea640 1 -- 172.21.15.84:0/2950397350 wait complete. 2023-10-16T09:50:17.372 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2023-10-16T09:50:15.614377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237060,"kb_used_data":4208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737660,"statfs":{"total":773094113280,"available":772851363840,"internally_reserved":0,"allocated":4308992,"data_stored":2385620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.711250"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.566246+0000","last_change":"2023-10-16T09:50:12.596909+0000","last_active":"2023-10-16T09:50:13.566246+0000","last_peered":"2023-10-16T09:50:13.566246+0000","last_clean":"2023-10-16T09:50:13.566246+0000","last_became_active":"2023-10-16T09:50:12.596628+0000","last_became_peered":"2023-10-16T09:50:12.596628+0000","last_unstale":"2023-10-16T09:50:13.566246+0000","last_undegraded":"2023-10-16T09:50:13.566246+0000","last_fullsized":"2023-10-16T09:50:13.566246+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:58.217542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089952+0000","last_change":"2023-10-16T09:50:12.592882+0000","last_active":"2023-10-16T09:50:14.089952+0000","last_peered":"2023-10-16T09:50:14.089952+0000","last_clean":"2023-10-16T09:50:14.089952+0000","last_became_active":"2023-10-16T09:50:12.592375+0000","last_became_peered":"2023-10-16T09:50:12.592375+0000","last_unstale":"2023-10-16T09:50:14.089952+0000","last_undegraded":"2023-10-16T09:50:14.089952+0000","last_fullsized":"2023-10-16T09:50:14.089952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:26.528764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089717+0000","last_change":"2023-10-16T09:50:12.602127+0000","last_active":"2023-10-16T09:50:14.089717+0000","last_peered":"2023-10-16T09:50:14.089717+0000","last_clean":"2023-10-16T09:50:14.089717+0000","last_became_active":"2023-10-16T09:50:12.601856+0000","last_became_peered":"2023-10-16T09:50:12.601856+0000","last_unstale":"2023-10-16T09:50:14.089717+0000","last_undegraded":"2023-10-16T09:50:14.089717+0000","last_fullsized":"2023-10-16T09:50:14.089717+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:34.869138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.802897+0000","last_change":"2023-10-16T09:50:12.589977+0000","last_active":"2023-10-16T09:50:13.802897+0000","last_peered":"2023-10-16T09:50:13.802897+0000","last_clean":"2023-10-16T09:50:13.802897+0000","last_became_active":"2023-10-16T09:50:12.589662+0000","last_became_peered":"2023-10-16T09:50:12.589662+0000","last_unstale":"2023-10-16T09:50:13.802897+0000","last_undegraded":"2023-10-16T09:50:13.802897+0000","last_fullsized":"2023-10-16T09:50:13.802897+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:35.334811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.561893+0000","last_change":"2023-10-16T09:50:12.586223+0000","last_active":"2023-10-16T09:50:13.561893+0000","last_peered":"2023-10-16T09:50:13.561893+0000","last_clean":"2023-10-16T09:50:13.561893+0000","last_became_active":"2023-10-16T09:50:12.585830+0000","last_became_peered":"2023-10-16T09:50:12.585830+0000","last_unstale":"2023-10-16T09:50:13.561893+0000","last_undegraded":"2023-10-16T09:50:13.561893+0000","last_fullsized":"2023-10-16T09:50:13.561893+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:06:42.423962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.088892+0000","last_change":"2023-10-16T09:50:12.596153+0000","last_active":"2023-10-16T09:50:14.088892+0000","last_peered":"2023-10-16T09:50:14.088892+0000","last_clean":"2023-10-16T09:50:14.088892+0000","last_became_active":"2023-10-16T09:50:12.595887+0000","last_became_peered":"2023-10-16T09:50:12.595887+0000","last_unstale":"2023-10-16T09:50:14.088892+0000","last_undegraded":"2023-10-16T09:50:14.088892+0000","last_fullsized":"2023-10-16T09:50:14.088892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:56:52.867292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.564531+0000","last_change":"2023-10-16T09:50:12.599333+0000","last_active":"2023-10-16T09:50:13.564531+0000","last_peered":"2023-10-16T09:50:13.564531+0000","last_clean":"2023-10-16T09:50:13.564531+0000","last_became_active":"2023-10-16T09:50:12.598339+0000","last_became_peered":"2023-10-16T09:50:12.598339+0000","last_unstale":"2023-10-16T09:50:13.564531+0000","last_undegraded":"2023-10-16T09:50:13.564531+0000","last_fullsized":"2023-10-16T09:50:13.564531+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:55.517446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.563504+0000","last_change":"2023-10-16T09:50:12.588032+0000","last_active":"2023-10-16T09:50:13.563504+0000","last_peered":"2023-10-16T09:50:13.563504+0000","last_clean":"2023-10-16T09:50:13.563504+0000","last_became_active":"2023-10-16T09:50:12.587591+0000","last_became_peered":"2023-10-16T09:50:12.587591+0000","last_unstale":"2023-10-16T09:50:13.563504+0000","last_undegraded":"2023-10-16T09:50:13.563504+0000","last_fullsized":"2023-10-16T09:50:13.563504+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:01.222630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910905+0000","last_change":"2023-10-16T09:50:12.588544+0000","last_active":"2023-10-16T09:50:13.910905+0000","last_peered":"2023-10-16T09:50:13.910905+0000","last_clean":"2023-10-16T09:50:13.910905+0000","last_became_active":"2023-10-16T09:50:12.588091+0000","last_became_peered":"2023-10-16T09:50:12.588091+0000","last_unstale":"2023-10-16T09:50:13.910905+0000","last_undegraded":"2023-10-16T09:50:13.910905+0000","last_fullsized":"2023-10-16T09:50:13.910905+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:35.334811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.803485+0000","last_change":"2023-10-16T09:50:12.588302+0000","last_active":"2023-10-16T09:50:13.803485+0000","last_peered":"2023-10-16T09:50:13.803485+0000","last_clean":"2023-10-16T09:50:13.803485+0000","last_became_active":"2023-10-16T09:50:12.586374+0000","last_became_peered":"2023-10-16T09:50:12.586374+0000","last_unstale":"2023-10-16T09:50:13.803485+0000","last_undegraded":"2023-10-16T09:50:13.803485+0000","last_fullsized":"2023-10-16T09:50:13.803485+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:17:15.300568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.565582+0000","last_change":"2023-10-16T09:50:12.589723+0000","last_active":"2023-10-16T09:50:13.565582+0000","last_peered":"2023-10-16T09:50:13.565582+0000","last_clean":"2023-10-16T09:50:13.565582+0000","last_became_active":"2023-10-16T09:50:12.588705+0000","last_became_peered":"2023-10-16T09:50:12.588705+0000","last_unstale":"2023-10-16T09:50:13.565582+0000","last_undegraded":"2023-10-16T09:50:13.565582+0000","last_fullsized":"2023-10-16T09:50:13.565582+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:15.603817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090184+0000","last_change":"2023-10-16T09:50:12.595733+0000","last_active":"2023-10-16T09:50:14.090184+0000","last_peered":"2023-10-16T09:50:14.090184+0000","last_clean":"2023-10-16T09:50:14.090184+0000","last_became_active":"2023-10-16T09:50:12.594960+0000","last_became_peered":"2023-10-16T09:50:12.594960+0000","last_unstale":"2023-10-16T09:50:14.090184+0000","last_undegraded":"2023-10-16T09:50:14.090184+0000","last_fullsized":"2023-10-16T09:50:14.090184+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:40:06.173040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.562059+0000","last_change":"2023-10-16T09:50:12.580073+0000","last_active":"2023-10-16T09:50:13.562059+0000","last_peered":"2023-10-16T09:50:13.562059+0000","last_clean":"2023-10-16T09:50:13.562059+0000","last_became_active":"2023-10-16T09:50:12.579795+0000","last_became_peered":"2023-10-16T09:50:12.579795+0000","last_unstale":"2023-10-16T09:50:13.562059+0000","last_undegraded":"2023-10-16T09:50:13.562059+0000","last_fullsized":"2023-10-16T09:50:13.562059+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:27.666006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089119+0000","last_change":"2023-10-16T09:50:12.586337+0000","last_active":"2023-10-16T09:50:14.089119+0000","last_peered":"2023-10-16T09:50:14.089119+0000","last_clean":"2023-10-16T09:50:14.089119+0000","last_became_active":"2023-10-16T09:50:12.585617+0000","last_became_peered":"2023-10-16T09:50:12.585617+0000","last_unstale":"2023-10-16T09:50:14.089119+0000","last_undegraded":"2023-10-16T09:50:14.089119+0000","last_fullsized":"2023-10-16T09:50:14.089119+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:40:06.173040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089405+0000","last_change":"2023-10-16T09:50:12.601453+0000","last_active":"2023-10-16T09:50:14.089405+0000","last_peered":"2023-10-16T09:50:14.089405+0000","last_clean":"2023-10-16T09:50:14.089405+0000","last_became_active":"2023-10-16T09:50:12.601178+0000","last_became_peered":"2023-10-16T09:50:12.601178+0000","last_unstale":"2023-10-16T09:50:14.089405+0000","last_undegraded":"2023-10-16T09:50:14.089405+0000","last_fullsized":"2023-10-16T09:50:14.089405+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:47:57.680638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.564499+0000","last_change":"2023-10-16T09:50:12.600618+0000","last_active":"2023-10-16T09:50:13.564499+0000","last_peered":"2023-10-16T09:50:13.564499+0000","last_clean":"2023-10-16T09:50:13.564499+0000","last_became_active":"2023-10-16T09:50:12.600211+0000","last_became_peered":"2023-10-16T09:50:12.600211+0000","last_unstale":"2023-10-16T09:50:13.564499+0000","last_undegraded":"2023-10-16T09:50:13.564499+0000","last_fullsized":"2023-10-16T09:50:13.564499+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:59:06.130674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.564369+0000","last_change":"2023-10-16T09:50:12.593082+0000","last_active":"2023-10-16T09:50:13.564369+0000","last_peered":"2023-10-16T09:50:13.564369+0000","last_clean":"2023-10-16T09:50:13.564369+0000","last_became_active":"2023-10-16T09:50:12.592786+0000","last_became_peered":"2023-10-16T09:50:12.592786+0000","last_unstale":"2023-10-16T09:50:13.564369+0000","last_undegraded":"2023-10-16T09:50:13.564369+0000","last_fullsized":"2023-10-16T09:50:13.564369+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:26.528764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090277+0000","last_change":"2023-10-16T09:50:12.584808+0000","last_active":"2023-10-16T09:50:14.090277+0000","last_peered":"2023-10-16T09:50:14.090277+0000","last_clean":"2023-10-16T09:50:14.090277+0000","last_became_active":"2023-10-16T09:50:12.584474+0000","last_became_peered":"2023-10-16T09:50:12.584474+0000","last_unstale":"2023-10-16T09:50:14.090277+0000","last_undegraded":"2023-10-16T09:50:14.090277+0000","last_fullsized":"2023-10-16T09:50:14.090277+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:59:06.130674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090287+0000","last_change":"2023-10-16T09:50:12.590207+0000","last_active":"2023-10-16T09:50:14.090287+0000","last_peered":"2023-10-16T09:50:14.090287+0000","last_clean":"2023-10-16T09:50:14.090287+0000","last_became_active":"2023-10-16T09:50:12.588887+0000","last_became_peered":"2023-10-16T09:50:12.588887+0000","last_unstale":"2023-10-16T09:50:14.090287+0000","last_undegraded":"2023-10-16T09:50:14.090287+0000","last_fullsized":"2023-10-16T09:50:14.090287+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:39.157970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.562209+0000","last_change":"2023-10-16T09:50:12.587777+0000","last_active":"2023-10-16T09:50:13.562209+0000","last_peered":"2023-10-16T09:50:13.562209+0000","last_clean":"2023-10-16T09:50:13.562209+0000","last_became_active":"2023-10-16T09:50:12.587497+0000","last_became_peered":"2023-10-16T09:50:12.587497+0000","last_unstale":"2023-10-16T09:50:13.562209+0000","last_undegraded":"2023-10-16T09:50:13.562209+0000","last_fullsized":"2023-10-16T09:50:13.562209+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:09:40.888338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910140+0000","last_change":"2023-10-16T09:50:12.596464+0000","last_active":"2023-10-16T09:50:13.910140+0000","last_peered":"2023-10-16T09:50:13.910140+0000","last_clean":"2023-10-16T09:50:13.910140+0000","last_became_active":"2023-10-16T09:50:12.595509+0000","last_became_peered":"2023-10-16T09:50:12.595509+0000","last_unstale":"2023-10-16T09:50:13.910140+0000","last_undegraded":"2023-10-16T09:50:13.910140+0000","last_fullsized":"2023-10-16T09:50:13.910140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:55:02.113527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910050+0000","last_change":"2023-10-16T09:50:12.593140+0000","last_active":"2023-10-16T09:50:13.910050+0000","last_peered":"2023-10-16T09:50:13.910050+0000","last_clean":"2023-10-16T09:50:13.910050+0000","last_became_active":"2023-10-16T09:50:12.592837+0000","last_became_peered":"2023-10-16T09:50:12.592837+0000","last_unstale":"2023-10-16T09:50:13.910050+0000","last_undegraded":"2023-10-16T09:50:13.910050+0000","last_fullsized":"2023-10-16T09:50:13.910050+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:56:39.838249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.088664+0000","last_change":"2023-10-16T09:50:12.584277+0000","last_active":"2023-10-16T09:50:14.088664+0000","last_peered":"2023-10-16T09:50:14.088664+0000","last_clean":"2023-10-16T09:50:14.088664+0000","last_became_active":"2023-10-16T09:50:12.583278+0000","last_became_peered":"2023-10-16T09:50:12.583278+0000","last_unstale":"2023-10-16T09:50:14.088664+0000","last_undegraded":"2023-10-16T09:50:14.088664+0000","last_fullsized":"2023-10-16T09:50:14.088664+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:58:32.894323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089407+0000","last_change":"2023-10-16T09:50:12.597752+0000","last_active":"2023-10-16T09:50:14.089407+0000","last_peered":"2023-10-16T09:50:14.089407+0000","last_clean":"2023-10-16T09:50:14.089407+0000","last_became_active":"2023-10-16T09:50:12.595650+0000","last_became_peered":"2023-10-16T09:50:12.595650+0000","last_unstale":"2023-10-16T09:50:14.089407+0000","last_undegraded":"2023-10-16T09:50:14.089407+0000","last_fullsized":"2023-10-16T09:50:14.089407+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:18:26.654045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.803607+0000","last_change":"2023-10-16T09:50:12.585217+0000","last_active":"2023-10-16T09:50:13.803607+0000","last_peered":"2023-10-16T09:50:13.803607+0000","last_clean":"2023-10-16T09:50:13.803607+0000","last_became_active":"2023-10-16T09:50:12.584946+0000","last_became_peered":"2023-10-16T09:50:12.584946+0000","last_unstale":"2023-10-16T09:50:13.803607+0000","last_undegraded":"2023-10-16T09:50:13.803607+0000","last_fullsized":"2023-10-16T09:50:13.803607+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:34.869138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090554+0000","last_change":"2023-10-16T09:50:12.586890+0000","last_active":"2023-10-16T09:50:14.090554+0000","last_peered":"2023-10-16T09:50:14.090554+0000","last_clean":"2023-10-16T09:50:14.090554+0000","last_became_active":"2023-10-16T09:50:12.586570+0000","last_became_peered":"2023-10-16T09:50:12.586570+0000","last_unstale":"2023-10-16T09:50:14.090554+0000","last_undegraded":"2023-10-16T09:50:14.090554+0000","last_fullsized":"2023-10-16T09:50:14.090554+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:55.517446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.803275+0000","last_change":"2023-10-16T09:50:12.591299+0000","last_active":"2023-10-16T09:50:13.803275+0000","last_peered":"2023-10-16T09:50:13.803275+0000","last_clean":"2023-10-16T09:50:13.803275+0000","last_became_active":"2023-10-16T09:50:12.590387+0000","last_became_peered":"2023-10-16T09:50:12.590387+0000","last_unstale":"2023-10-16T09:50:13.803275+0000","last_undegraded":"2023-10-16T09:50:13.803275+0000","last_fullsized":"2023-10-16T09:50:13.803275+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:18:26.654045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090259+0000","last_change":"2023-10-16T09:50:12.591942+0000","last_active":"2023-10-16T09:50:14.090259+0000","last_peered":"2023-10-16T09:50:14.090259+0000","last_clean":"2023-10-16T09:50:14.090259+0000","last_became_active":"2023-10-16T09:50:12.591651+0000","last_became_peered":"2023-10-16T09:50:12.591651+0000","last_unstale":"2023-10-16T09:50:14.090259+0000","last_undegraded":"2023-10-16T09:50:14.090259+0000","last_fullsized":"2023-10-16T09:50:14.090259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:27.920687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910232+0000","last_change":"2023-10-16T09:50:12.597897+0000","last_active":"2023-10-16T09:50:13.910232+0000","last_peered":"2023-10-16T09:50:13.910232+0000","last_clean":"2023-10-16T09:50:13.910232+0000","last_became_active":"2023-10-16T09:50:12.597382+0000","last_became_peered":"2023-10-16T09:50:12.597382+0000","last_unstale":"2023-10-16T09:50:13.910232+0000","last_undegraded":"2023-10-16T09:50:13.910232+0000","last_fullsized":"2023-10-16T09:50:13.910232+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:35.275931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.563737+0000","last_change":"2023-10-16T09:50:12.600791+0000","last_active":"2023-10-16T09:50:13.563737+0000","last_peered":"2023-10-16T09:50:13.563737+0000","last_clean":"2023-10-16T09:50:13.563737+0000","last_became_active":"2023-10-16T09:50:12.600521+0000","last_became_peered":"2023-10-16T09:50:12.600521+0000","last_unstale":"2023-10-16T09:50:13.563737+0000","last_undegraded":"2023-10-16T09:50:13.563737+0000","last_fullsized":"2023-10-16T09:50:13.563737+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:39.821588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.565969+0000","last_change":"2023-10-16T09:50:12.581079+0000","last_active":"2023-10-16T09:50:13.565969+0000","last_peered":"2023-10-16T09:50:13.565969+0000","last_clean":"2023-10-16T09:50:13.565969+0000","last_became_active":"2023-10-16T09:50:12.580696+0000","last_became_peered":"2023-10-16T09:50:12.580696+0000","last_unstale":"2023-10-16T09:50:13.565969+0000","last_undegraded":"2023-10-16T09:50:13.565969+0000","last_fullsized":"2023-10-16T09:50:13.565969+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:55:27.040013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.564004+0000","last_change":"2023-10-16T09:50:12.598141+0000","last_active":"2023-10-16T09:50:13.564004+0000","last_peered":"2023-10-16T09:50:13.564004+0000","last_clean":"2023-10-16T09:50:13.564004+0000","last_became_active":"2023-10-16T09:50:12.593867+0000","last_became_peered":"2023-10-16T09:50:12.593867+0000","last_unstale":"2023-10-16T09:50:13.564004+0000","last_undegraded":"2023-10-16T09:50:13.564004+0000","last_fullsized":"2023-10-16T09:50:13.564004+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:27.920687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089749+0000","last_change":"2023-10-16T09:50:12.583938+0000","last_active":"2023-10-16T09:50:14.089749+0000","last_peered":"2023-10-16T09:50:14.089749+0000","last_clean":"2023-10-16T09:50:14.089749+0000","last_became_active":"2023-10-16T09:50:12.583645+0000","last_became_peered":"2023-10-16T09:50:12.583645+0000","last_unstale":"2023-10-16T09:50:14.089749+0000","last_undegraded":"2023-10-16T09:50:14.089749+0000","last_fullsized":"2023-10-16T09:50:14.089749+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:51.573863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.566397+0000","last_change":"2023-10-16T09:50:11.955907+0000","last_active":"2023-10-16T09:50:13.566397+0000","last_peered":"2023-10-16T09:50:13.566397+0000","last_clean":"2023-10-16T09:50:13.566397+0000","last_became_active":"2023-10-16T09:50:11.955616+0000","last_became_peered":"2023-10-16T09:50:11.955616+0000","last_unstale":"2023-10-16T09:50:13.566397+0000","last_undegraded":"2023-10-16T09:50:13.566397+0000","last_fullsized":"2023-10-16T09:50:13.566397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:43.336545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.566264+0000","last_change":"2023-10-16T09:50:12.590779+0000","last_active":"2023-10-16T09:50:13.566264+0000","last_peered":"2023-10-16T09:50:13.566264+0000","last_clean":"2023-10-16T09:50:13.566264+0000","last_became_active":"2023-10-16T09:50:12.590485+0000","last_became_peered":"2023-10-16T09:50:12.590485+0000","last_unstale":"2023-10-16T09:50:13.566264+0000","last_undegraded":"2023-10-16T09:50:13.566264+0000","last_fullsized":"2023-10-16T09:50:13.566264+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:58:32.894323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089947+0000","last_change":"2023-10-16T09:50:11.949795+0000","last_active":"2023-10-16T09:50:14.089947+0000","last_peered":"2023-10-16T09:50:14.089947+0000","last_clean":"2023-10-16T09:50:14.089947+0000","last_became_active":"2023-10-16T09:50:11.949558+0000","last_became_peered":"2023-10-16T09:50:11.949558+0000","last_unstale":"2023-10-16T09:50:14.089947+0000","last_undegraded":"2023-10-16T09:50:14.089947+0000","last_fullsized":"2023-10-16T09:50:14.089947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:55:26.024831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910602+0000","last_change":"2023-10-16T09:50:12.597787+0000","last_active":"2023-10-16T09:50:13.910602+0000","last_peered":"2023-10-16T09:50:13.910602+0000","last_clean":"2023-10-16T09:50:13.910602+0000","last_became_active":"2023-10-16T09:50:12.597406+0000","last_became_peered":"2023-10-16T09:50:12.597406+0000","last_unstale":"2023-10-16T09:50:13.910602+0000","last_undegraded":"2023-10-16T09:50:13.910602+0000","last_fullsized":"2023-10-16T09:50:13.910602+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:47.181025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090485+0000","last_change":"2023-10-16T09:50:11.942585+0000","last_active":"2023-10-16T09:50:14.090485+0000","last_peered":"2023-10-16T09:50:14.090485+0000","last_clean":"2023-10-16T09:50:14.090485+0000","last_became_active":"2023-10-16T09:50:11.942333+0000","last_became_peered":"2023-10-16T09:50:11.942333+0000","last_unstale":"2023-10-16T09:50:14.090485+0000","last_undegraded":"2023-10-16T09:50:14.090485+0000","last_fullsized":"2023-10-16T09:50:14.090485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:46.165842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.803228+0000","last_change":"2023-10-16T09:50:12.585893+0000","last_active":"2023-10-16T09:50:13.803228+0000","last_peered":"2023-10-16T09:50:13.803228+0000","last_clean":"2023-10-16T09:50:13.803228+0000","last_became_active":"2023-10-16T09:50:12.585616+0000","last_became_peered":"2023-10-16T09:50:12.585616+0000","last_unstale":"2023-10-16T09:50:13.803228+0000","last_undegraded":"2023-10-16T09:50:13.803228+0000","last_fullsized":"2023-10-16T09:50:13.803228+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:47:57.680638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.562364+0000","last_change":"2023-10-16T09:50:11.942077+0000","last_active":"2023-10-16T09:50:13.562364+0000","last_peered":"2023-10-16T09:50:13.562364+0000","last_clean":"2023-10-16T09:50:13.562364+0000","last_became_active":"2023-10-16T09:50:11.941696+0000","last_became_peered":"2023-10-16T09:50:11.941696+0000","last_unstale":"2023-10-16T09:50:13.562364+0000","last_undegraded":"2023-10-16T09:50:13.562364+0000","last_fullsized":"2023-10-16T09:50:13.562364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:10:45.453057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.563777+0000","last_change":"2023-10-16T09:50:12.599549+0000","last_active":"2023-10-16T09:50:13.563777+0000","last_peered":"2023-10-16T09:50:13.563777+0000","last_clean":"2023-10-16T09:50:13.563777+0000","last_became_active":"2023-10-16T09:50:12.599254+0000","last_became_peered":"2023-10-16T09:50:12.599254+0000","last_unstale":"2023-10-16T09:50:13.563777+0000","last_undegraded":"2023-10-16T09:50:13.563777+0000","last_fullsized":"2023-10-16T09:50:13.563777+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:56:52.867292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910427+0000","last_change":"2023-10-16T09:50:11.948043+0000","last_active":"2023-10-16T09:50:13.910427+0000","last_peered":"2023-10-16T09:50:13.910427+0000","last_clean":"2023-10-16T09:50:13.910427+0000","last_became_active":"2023-10-16T09:50:11.947746+0000","last_became_peered":"2023-10-16T09:50:11.947746+0000","last_unstale":"2023-10-16T09:50:13.910427+0000","last_undegraded":"2023-10-16T09:50:13.910427+0000","last_fullsized":"2023-10-16T09:50:13.910427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:20.357575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089461+0000","last_change":"2023-10-16T09:50:12.589951+0000","last_active":"2023-10-16T09:50:14.089461+0000","last_peered":"2023-10-16T09:50:14.089461+0000","last_clean":"2023-10-16T09:50:14.089461+0000","last_became_active":"2023-10-16T09:50:12.589588+0000","last_became_peered":"2023-10-16T09:50:12.589588+0000","last_unstale":"2023-10-16T09:50:14.089461+0000","last_undegraded":"2023-10-16T09:50:14.089461+0000","last_fullsized":"2023-10-16T09:50:14.089461+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:17:15.300568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910183+0000","last_change":"2023-10-16T09:50:11.936818+0000","last_active":"2023-10-16T09:50:13.910183+0000","last_peered":"2023-10-16T09:50:13.910183+0000","last_clean":"2023-10-16T09:50:13.910183+0000","last_became_active":"2023-10-16T09:50:11.936296+0000","last_became_peered":"2023-10-16T09:50:11.936296+0000","last_unstale":"2023-10-16T09:50:13.910183+0000","last_undegraded":"2023-10-16T09:50:13.910183+0000","last_fullsized":"2023-10-16T09:50:13.910183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:06:41.408779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.909811+0000","last_change":"2023-10-16T09:50:12.583979+0000","last_active":"2023-10-16T09:50:13.909811+0000","last_peered":"2023-10-16T09:50:13.909811+0000","last_clean":"2023-10-16T09:50:13.909811+0000","last_became_active":"2023-10-16T09:50:12.583666+0000","last_became_peered":"2023-10-16T09:50:12.583666+0000","last_unstale":"2023-10-16T09:50:13.909811+0000","last_undegraded":"2023-10-16T09:50:13.909811+0000","last_fullsized":"2023-10-16T09:50:13.909811+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:51.573863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.088563+0000","last_change":"2023-10-16T09:50:11.936387+0000","last_active":"2023-10-16T09:50:14.088563+0000","last_peered":"2023-10-16T09:50:14.088563+0000","last_clean":"2023-10-16T09:50:14.088563+0000","last_became_active":"2023-10-16T09:50:11.935830+0000","last_became_peered":"2023-10-16T09:50:11.935830+0000","last_unstale":"2023-10-16T09:50:14.088563+0000","last_undegraded":"2023-10-16T09:50:14.088563+0000","last_fullsized":"2023-10-16T09:50:14.088563+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:18:25.638863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.802455+0000","last_change":"2023-10-16T09:50:12.595196+0000","last_active":"2023-10-16T09:50:13.802455+0000","last_peered":"2023-10-16T09:50:13.802455+0000","last_clean":"2023-10-16T09:50:13.802455+0000","last_became_active":"2023-10-16T09:50:12.594843+0000","last_became_peered":"2023-10-16T09:50:12.594843+0000","last_unstale":"2023-10-16T09:50:13.802455+0000","last_undegraded":"2023-10-16T09:50:13.802455+0000","last_fullsized":"2023-10-16T09:50:13.802455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:34.134204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.909806+0000","last_change":"2023-10-16T09:50:11.947404+0000","last_active":"2023-10-16T09:50:13.909806+0000","last_peered":"2023-10-16T09:50:13.909806+0000","last_clean":"2023-10-16T09:50:13.909806+0000","last_became_active":"2023-10-16T09:50:11.947080+0000","last_became_peered":"2023-10-16T09:50:11.947080+0000","last_unstale":"2023-10-16T09:50:13.909806+0000","last_undegraded":"2023-10-16T09:50:13.909806+0000","last_fullsized":"2023-10-16T09:50:13.909806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:51:15.834863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090178+0000","last_change":"2023-10-16T09:50:12.601729+0000","last_active":"2023-10-16T09:50:14.090178+0000","last_peered":"2023-10-16T09:50:14.090178+0000","last_clean":"2023-10-16T09:50:14.090178+0000","last_became_active":"2023-10-16T09:50:12.601454+0000","last_became_peered":"2023-10-16T09:50:12.601454+0000","last_unstale":"2023-10-16T09:50:14.090178+0000","last_undegraded":"2023-10-16T09:50:14.090178+0000","last_fullsized":"2023-10-16T09:50:14.090178+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:56:52.867292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090624+0000","last_change":"2023-10-16T09:50:11.949935+0000","last_active":"2023-10-16T09:50:14.090624+0000","last_peered":"2023-10-16T09:50:14.090624+0000","last_clean":"2023-10-16T09:50:14.090624+0000","last_became_active":"2023-10-16T09:50:11.949614+0000","last_became_peered":"2023-10-16T09:50:11.949614+0000","last_unstale":"2023-10-16T09:50:14.090624+0000","last_undegraded":"2023-10-16T09:50:14.090624+0000","last_fullsized":"2023-10-16T09:50:14.090624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:47:56.665456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090717+0000","last_change":"2023-10-16T09:50:12.588890+0000","last_active":"2023-10-16T09:50:14.090717+0000","last_peered":"2023-10-16T09:50:14.090717+0000","last_clean":"2023-10-16T09:50:14.090717+0000","last_became_active":"2023-10-16T09:50:12.588606+0000","last_became_peered":"2023-10-16T09:50:12.588606+0000","last_unstale":"2023-10-16T09:50:14.090717+0000","last_undegraded":"2023-10-16T09:50:14.090717+0000","last_fullsized":"2023-10-16T09:50:14.090717+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:15.603817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090349+0000","last_change":"2023-10-16T09:50:11.940747+0000","last_active":"2023-10-16T09:50:14.090349+0000","last_peered":"2023-10-16T09:50:14.090349+0000","last_clean":"2023-10-16T09:50:14.090349+0000","last_became_active":"2023-10-16T09:50:11.940443+0000","last_became_peered":"2023-10-16T09:50:11.940443+0000","last_unstale":"2023-10-16T09:50:14.090349+0000","last_undegraded":"2023-10-16T09:50:14.090349+0000","last_fullsized":"2023-10-16T09:50:14.090349+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:57.202359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089295+0000","last_change":"2023-10-16T09:50:12.600921+0000","last_active":"2023-10-16T09:50:14.089295+0000","last_peered":"2023-10-16T09:50:14.089295+0000","last_clean":"2023-10-16T09:50:14.089295+0000","last_became_active":"2023-10-16T09:50:12.600711+0000","last_became_peered":"2023-10-16T09:50:12.600711+0000","last_unstale":"2023-10-16T09:50:14.089295+0000","last_undegraded":"2023-10-16T09:50:14.089295+0000","last_fullsized":"2023-10-16T09:50:14.089295+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:27.920687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090046+0000","last_change":"2023-10-16T09:50:11.951018+0000","last_active":"2023-10-16T09:50:14.090046+0000","last_peered":"2023-10-16T09:50:14.090046+0000","last_clean":"2023-10-16T09:50:14.090046+0000","last_became_active":"2023-10-16T09:50:11.950683+0000","last_became_peered":"2023-10-16T09:50:11.950683+0000","last_unstale":"2023-10-16T09:50:14.090046+0000","last_undegraded":"2023-10-16T09:50:14.090046+0000","last_fullsized":"2023-10-16T09:50:14.090046+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:34.319629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089747+0000","last_change":"2023-10-16T09:50:12.601903+0000","last_active":"2023-10-16T09:50:14.089747+0000","last_peered":"2023-10-16T09:50:14.089747+0000","last_clean":"2023-10-16T09:50:14.089747+0000","last_became_active":"2023-10-16T09:50:12.601633+0000","last_became_peered":"2023-10-16T09:50:12.601633+0000","last_unstale":"2023-10-16T09:50:14.089747+0000","last_undegraded":"2023-10-16T09:50:14.089747+0000","last_fullsized":"2023-10-16T09:50:14.089747+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:01.222630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089396+0000","last_change":"2023-10-16T09:50:11.942305+0000","last_active":"2023-10-16T09:50:14.089396+0000","last_peered":"2023-10-16T09:50:14.089396+0000","last_clean":"2023-10-16T09:50:14.089396+0000","last_became_active":"2023-10-16T09:50:11.940994+0000","last_became_peered":"2023-10-16T09:50:11.940994+0000","last_unstale":"2023-10-16T09:50:14.089396+0000","last_undegraded":"2023-10-16T09:50:14.089396+0000","last_fullsized":"2023-10-16T09:50:14.089396+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:34.260749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.802604+0000","last_change":"2023-10-16T09:50:12.595577+0000","last_active":"2023-10-16T09:50:13.802604+0000","last_peered":"2023-10-16T09:50:13.802604+0000","last_clean":"2023-10-16T09:50:13.802604+0000","last_became_active":"2023-10-16T09:50:12.595320+0000","last_became_peered":"2023-10-16T09:50:12.595320+0000","last_unstale":"2023-10-16T09:50:13.802604+0000","last_undegraded":"2023-10-16T09:50:13.802604+0000","last_fullsized":"2023-10-16T09:50:13.802604+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:47.181025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910534+0000","last_change":"2023-10-16T09:50:11.946954+0000","last_active":"2023-10-16T09:50:13.910534+0000","last_peered":"2023-10-16T09:50:13.910534+0000","last_clean":"2023-10-16T09:50:13.910534+0000","last_became_active":"2023-10-16T09:50:11.946582+0000","last_became_peered":"2023-10-16T09:50:11.946582+0000","last_unstale":"2023-10-16T09:50:13.910534+0000","last_undegraded":"2023-10-16T09:50:13.910534+0000","last_fullsized":"2023-10-16T09:50:13.910534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:36.724821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089917+0000","last_change":"2023-10-16T09:50:12.590565+0000","last_active":"2023-10-16T09:50:14.089917+0000","last_peered":"2023-10-16T09:50:14.089917+0000","last_clean":"2023-10-16T09:50:14.089917+0000","last_became_active":"2023-10-16T09:50:12.589994+0000","last_became_peered":"2023-10-16T09:50:12.589994+0000","last_unstale":"2023-10-16T09:50:14.089917+0000","last_undegraded":"2023-10-16T09:50:14.089917+0000","last_fullsized":"2023-10-16T09:50:14.089917+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:23.391598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.802719+0000","last_change":"2023-10-16T09:50:11.945770+0000","last_active":"2023-10-16T09:50:13.802719+0000","last_peered":"2023-10-16T09:50:13.802719+0000","last_clean":"2023-10-16T09:50:13.802719+0000","last_became_active":"2023-10-16T09:50:11.945552+0000","last_became_peered":"2023-10-16T09:50:11.945552+0000","last_unstale":"2023-10-16T09:50:13.802719+0000","last_undegraded":"2023-10-16T09:50:13.802719+0000","last_fullsized":"2023-10-16T09:50:13.802719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:57.202359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.563882+0000","last_change":"2023-10-16T09:50:12.595849+0000","last_active":"2023-10-16T09:50:13.563882+0000","last_peered":"2023-10-16T09:50:13.563882+0000","last_clean":"2023-10-16T09:50:13.563882+0000","last_became_active":"2023-10-16T09:50:12.591954+0000","last_became_peered":"2023-10-16T09:50:12.591954+0000","last_unstale":"2023-10-16T09:50:13.563882+0000","last_undegraded":"2023-10-16T09:50:13.563882+0000","last_fullsized":"2023-10-16T09:50:13.563882+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:40:06.173040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.088636+0000","last_change":"2023-10-16T09:50:11.950355+0000","last_active":"2023-10-16T09:50:14.088636+0000","last_peered":"2023-10-16T09:50:14.088636+0000","last_clean":"2023-10-16T09:50:14.088636+0000","last_became_active":"2023-10-16T09:50:11.950091+0000","last_became_peered":"2023-10-16T09:50:11.950091+0000","last_unstale":"2023-10-16T09:50:14.088636+0000","last_undegraded":"2023-10-16T09:50:14.088636+0000","last_fullsized":"2023-10-16T09:50:14.088636+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:01:46.047914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.566082+0000","last_change":"2023-10-16T09:50:12.591751+0000","last_active":"2023-10-16T09:50:13.566082+0000","last_peered":"2023-10-16T09:50:13.566082+0000","last_clean":"2023-10-16T09:50:13.566082+0000","last_became_active":"2023-10-16T09:50:12.591379+0000","last_became_peered":"2023-10-16T09:50:12.591379+0000","last_unstale":"2023-10-16T09:50:13.566082+0000","last_undegraded":"2023-10-16T09:50:13.566082+0000","last_fullsized":"2023-10-16T09:50:13.566082+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:05:27.268406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.088262+0000","last_change":"2023-10-16T09:50:11.935845+0000","last_active":"2023-10-16T09:50:14.088262+0000","last_peered":"2023-10-16T09:50:14.088262+0000","last_clean":"2023-10-16T09:50:14.088262+0000","last_became_active":"2023-10-16T09:50:11.935559+0000","last_became_peered":"2023-10-16T09:50:11.935559+0000","last_unstale":"2023-10-16T09:50:14.088262+0000","last_undegraded":"2023-10-16T09:50:14.088262+0000","last_fullsized":"2023-10-16T09:50:14.088262+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:30.221634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910360+0000","last_change":"2023-10-16T09:50:12.594596+0000","last_active":"2023-10-16T09:50:13.910360+0000","last_peered":"2023-10-16T09:50:13.910360+0000","last_clean":"2023-10-16T09:50:13.910360+0000","last_became_active":"2023-10-16T09:50:12.592376+0000","last_became_peered":"2023-10-16T09:50:12.592376+0000","last_unstale":"2023-10-16T09:50:13.910360+0000","last_undegraded":"2023-10-16T09:50:13.910360+0000","last_fullsized":"2023-10-16T09:50:13.910360+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:34.134204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.803818+0000","last_change":"2023-10-16T09:50:11.953184+0000","last_active":"2023-10-16T09:50:13.803818+0000","last_peered":"2023-10-16T09:50:13.803818+0000","last_clean":"2023-10-16T09:50:13.803818+0000","last_became_active":"2023-10-16T09:50:11.952647+0000","last_became_peered":"2023-10-16T09:50:11.952647+0000","last_unstale":"2023-10-16T09:50:13.803818+0000","last_undegraded":"2023-10-16T09:50:13.803818+0000","last_fullsized":"2023-10-16T09:50:13.803818+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:55:01.098345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090074+0000","last_change":"2023-10-16T09:50:12.589925+0000","last_active":"2023-10-16T09:50:14.090074+0000","last_peered":"2023-10-16T09:50:14.090074+0000","last_clean":"2023-10-16T09:50:14.090074+0000","last_became_active":"2023-10-16T09:50:12.589688+0000","last_became_peered":"2023-10-16T09:50:12.589688+0000","last_unstale":"2023-10-16T09:50:14.090074+0000","last_undegraded":"2023-10-16T09:50:14.090074+0000","last_fullsized":"2023-10-16T09:50:14.090074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:39.821588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910613+0000","last_change":"2023-10-16T09:50:11.943119+0000","last_active":"2023-10-16T09:50:13.910613+0000","last_peered":"2023-10-16T09:50:13.910613+0000","last_clean":"2023-10-16T09:50:13.910613+0000","last_became_active":"2023-10-16T09:50:11.942800+0000","last_became_peered":"2023-10-16T09:50:11.942800+0000","last_unstale":"2023-10-16T09:50:13.910613+0000","last_undegraded":"2023-10-16T09:50:13.910613+0000","last_fullsized":"2023-10-16T09:50:13.910613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:36.404480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.802925+0000","last_change":"2023-10-16T09:50:12.589887+0000","last_active":"2023-10-16T09:50:13.802925+0000","last_peered":"2023-10-16T09:50:13.802925+0000","last_clean":"2023-10-16T09:50:13.802925+0000","last_became_active":"2023-10-16T09:50:12.589589+0000","last_became_peered":"2023-10-16T09:50:12.589589+0000","last_unstale":"2023-10-16T09:50:13.802925+0000","last_undegraded":"2023-10-16T09:50:13.802925+0000","last_fullsized":"2023-10-16T09:50:13.802925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:01:47.063096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910671+0000","last_change":"2023-10-16T09:50:11.940537+0000","last_active":"2023-10-16T09:50:13.910671+0000","last_peered":"2023-10-16T09:50:13.910671+0000","last_clean":"2023-10-16T09:50:13.910671+0000","last_became_active":"2023-10-16T09:50:11.940046+0000","last_became_peered":"2023-10-16T09:50:11.940046+0000","last_unstale":"2023-10-16T09:50:13.910671+0000","last_undegraded":"2023-10-16T09:50:13.910671+0000","last_fullsized":"2023-10-16T09:50:13.910671+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:55:26.024831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.562741+0000","last_change":"2023-10-16T09:50:12.583467+0000","last_active":"2023-10-16T09:50:13.562741+0000","last_peered":"2023-10-16T09:50:13.562741+0000","last_clean":"2023-10-16T09:50:13.562741+0000","last_became_active":"2023-10-16T09:50:12.583099+0000","last_became_peered":"2023-10-16T09:50:12.583099+0000","last_unstale":"2023-10-16T09:50:13.562741+0000","last_undegraded":"2023-10-16T09:50:13.562741+0000","last_fullsized":"2023-10-16T09:50:13.562741+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:37.740003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910763+0000","last_change":"2023-10-16T09:50:11.938384+0000","last_active":"2023-10-16T09:50:13.910763+0000","last_peered":"2023-10-16T09:50:13.910763+0000","last_clean":"2023-10-16T09:50:13.910763+0000","last_became_active":"2023-10-16T09:50:11.937795+0000","last_became_peered":"2023-10-16T09:50:11.937795+0000","last_unstale":"2023-10-16T09:50:13.910763+0000","last_undegraded":"2023-10-16T09:50:13.910763+0000","last_fullsized":"2023-10-16T09:50:13.910763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:57.202359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.561991+0000","last_change":"2023-10-16T09:50:12.582768+0000","last_active":"2023-10-16T09:50:13.561991+0000","last_peered":"2023-10-16T09:50:13.561991+0000","last_clean":"2023-10-16T09:50:13.561991+0000","last_became_active":"2023-10-16T09:50:12.582416+0000","last_became_peered":"2023-10-16T09:50:12.582416+0000","last_unstale":"2023-10-16T09:50:13.561991+0000","last_undegraded":"2023-10-16T09:50:13.561991+0000","last_fullsized":"2023-10-16T09:50:13.561991+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:33:32.255338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089664+0000","last_change":"2023-10-16T09:50:11.944903+0000","last_active":"2023-10-16T09:50:14.089664+0000","last_peered":"2023-10-16T09:50:14.089664+0000","last_clean":"2023-10-16T09:50:14.089664+0000","last_became_active":"2023-10-16T09:50:11.944617+0000","last_became_peered":"2023-10-16T09:50:11.944617+0000","last_unstale":"2023-10-16T09:50:14.089664+0000","last_undegraded":"2023-10-16T09:50:14.089664+0000","last_fullsized":"2023-10-16T09:50:14.089664+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:43.336545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.562042+0000","last_change":"2023-10-16T09:50:12.587254+0000","last_active":"2023-10-16T09:50:13.562042+0000","last_peered":"2023-10-16T09:50:13.562042+0000","last_clean":"2023-10-16T09:50:13.562042+0000","last_became_active":"2023-10-16T09:50:12.586970+0000","last_became_peered":"2023-10-16T09:50:12.586970+0000","last_unstale":"2023-10-16T09:50:13.562042+0000","last_undegraded":"2023-10-16T09:50:13.562042+0000","last_fullsized":"2023-10-16T09:50:13.562042+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:14:50.109642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.565405+0000","last_change":"2023-10-16T09:50:11.954696+0000","last_active":"2023-10-16T09:50:13.565405+0000","last_peered":"2023-10-16T09:50:13.565405+0000","last_clean":"2023-10-16T09:50:13.565405+0000","last_became_active":"2023-10-16T09:50:11.951148+0000","last_became_peered":"2023-10-16T09:50:11.951148+0000","last_unstale":"2023-10-16T09:50:13.565405+0000","last_undegraded":"2023-10-16T09:50:13.565405+0000","last_fullsized":"2023-10-16T09:50:13.565405+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:09:39.873155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.561833+0000","last_change":"2023-10-16T09:50:12.584513+0000","last_active":"2023-10-16T09:50:13.561833+0000","last_peered":"2023-10-16T09:50:13.561833+0000","last_clean":"2023-10-16T09:50:13.561833+0000","last_became_active":"2023-10-16T09:50:12.583987+0000","last_became_peered":"2023-10-16T09:50:12.583987+0000","last_unstale":"2023-10-16T09:50:13.561833+0000","last_undegraded":"2023-10-16T09:50:13.561833+0000","last_fullsized":"2023-10-16T09:50:13.561833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:59.998733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.909752+0000","last_change":"2023-10-16T09:50:11.937491+0000","last_active":"2023-10-16T09:50:13.909752+0000","last_peered":"2023-10-16T09:50:13.909752+0000","last_clean":"2023-10-16T09:50:13.909752+0000","last_became_active":"2023-10-16T09:50:11.936899+0000","last_became_peered":"2023-10-16T09:50:11.936899+0000","last_unstale":"2023-10-16T09:50:13.909752+0000","last_undegraded":"2023-10-16T09:50:13.909752+0000","last_fullsized":"2023-10-16T09:50:13.909752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:14:49.094460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.564182+0000","last_change":"2023-10-16T09:50:12.592848+0000","last_active":"2023-10-16T09:50:13.564182+0000","last_peered":"2023-10-16T09:50:13.564182+0000","last_clean":"2023-10-16T09:50:13.564182+0000","last_became_active":"2023-10-16T09:50:12.592438+0000","last_became_peered":"2023-10-16T09:50:12.592438+0000","last_unstale":"2023-10-16T09:50:13.564182+0000","last_undegraded":"2023-10-16T09:50:13.564182+0000","last_fullsized":"2023-10-16T09:50:13.564182+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:23.391598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089250+0000","last_change":"2023-10-16T09:50:11.952836+0000","last_active":"2023-10-16T09:50:14.089250+0000","last_peered":"2023-10-16T09:50:14.089250+0000","last_clean":"2023-10-16T09:50:14.089250+0000","last_became_active":"2023-10-16T09:50:11.952496+0000","last_became_peered":"2023-10-16T09:50:11.952496+0000","last_unstale":"2023-10-16T09:50:14.089250+0000","last_undegraded":"2023-10-16T09:50:14.089250+0000","last_fullsized":"2023-10-16T09:50:14.089250+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:33.119022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.562725+0000","last_change":"2023-10-16T09:50:12.593896+0000","last_active":"2023-10-16T09:50:13.562725+0000","last_peered":"2023-10-16T09:50:13.562725+0000","last_clean":"2023-10-16T09:50:13.562725+0000","last_became_active":"2023-10-16T09:50:12.592759+0000","last_became_peered":"2023-10-16T09:50:12.592759+0000","last_unstale":"2023-10-16T09:50:13.562725+0000","last_undegraded":"2023-10-16T09:50:13.562725+0000","last_fullsized":"2023-10-16T09:50:13.562725+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:21.372758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.803791+0000","last_change":"2023-10-16T09:50:11.954471+0000","last_active":"2023-10-16T09:50:13.803791+0000","last_peered":"2023-10-16T09:50:13.803791+0000","last_clean":"2023-10-16T09:50:13.803791+0000","last_became_active":"2023-10-16T09:50:11.954090+0000","last_became_peered":"2023-10-16T09:50:11.954090+0000","last_unstale":"2023-10-16T09:50:13.803791+0000","last_undegraded":"2023-10-16T09:50:13.803791+0000","last_fullsized":"2023-10-16T09:50:13.803791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:56:38.823066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090298+0000","last_change":"2023-10-16T09:50:12.594329+0000","last_active":"2023-10-16T09:50:14.090298+0000","last_peered":"2023-10-16T09:50:14.090298+0000","last_clean":"2023-10-16T09:50:14.090298+0000","last_became_active":"2023-10-16T09:50:12.594032+0000","last_became_peered":"2023-10-16T09:50:12.594032+0000","last_unstale":"2023-10-16T09:50:14.090298+0000","last_undegraded":"2023-10-16T09:50:14.090298+0000","last_fullsized":"2023-10-16T09:50:14.090298+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:23.391598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.129883+0000","last_change":"2023-10-16T09:50:11.944544+0000","last_active":"2023-10-16T09:50:14.129883+0000","last_peered":"2023-10-16T09:50:14.129883+0000","last_clean":"2023-10-16T09:50:14.129883+0000","last_became_active":"2023-10-16T09:50:11.944188+0000","last_became_peered":"2023-10-16T09:50:11.944188+0000","last_unstale":"2023-10-16T09:50:14.129883+0000","last_undegraded":"2023-10-16T09:50:14.129883+0000","last_fullsized":"2023-10-16T09:50:14.129883+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:05:26.253224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089040+0000","last_change":"2023-10-16T09:50:12.580392+0000","last_active":"2023-10-16T09:50:14.089040+0000","last_peered":"2023-10-16T09:50:14.089040+0000","last_clean":"2023-10-16T09:50:14.089040+0000","last_became_active":"2023-10-16T09:50:12.579880+0000","last_became_peered":"2023-10-16T09:50:12.579880+0000","last_unstale":"2023-10-16T09:50:14.089040+0000","last_undegraded":"2023-10-16T09:50:14.089040+0000","last_fullsized":"2023-10-16T09:50:14.089040+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:31.236817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.803327+0000","last_change":"2023-10-16T09:50:11.939773+0000","last_active":"2023-10-16T09:50:13.803327+0000","last_peered":"2023-10-16T09:50:13.803327+0000","last_clean":"2023-10-16T09:50:13.803327+0000","last_became_active":"2023-10-16T09:50:11.939489+0000","last_became_peered":"2023-10-16T09:50:11.939489+0000","last_unstale":"2023-10-16T09:50:13.803327+0000","last_undegraded":"2023-10-16T09:50:13.803327+0000","last_fullsized":"2023-10-16T09:50:13.803327+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:50.558681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.565052+0000","last_change":"2023-10-16T09:50:12.584035+0000","last_active":"2023-10-16T09:50:13.565052+0000","last_peered":"2023-10-16T09:50:13.565052+0000","last_clean":"2023-10-16T09:50:13.565052+0000","last_became_active":"2023-10-16T09:50:12.583748+0000","last_became_peered":"2023-10-16T09:50:12.583748+0000","last_unstale":"2023-10-16T09:50:13.565052+0000","last_undegraded":"2023-10-16T09:50:13.565052+0000","last_fullsized":"2023-10-16T09:50:13.565052+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:37.419663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090011+0000","last_change":"2023-10-16T09:50:11.950606+0000","last_active":"2023-10-16T09:50:14.090011+0000","last_peered":"2023-10-16T09:50:14.090011+0000","last_clean":"2023-10-16T09:50:14.090011+0000","last_became_active":"2023-10-16T09:50:11.949747+0000","last_became_peered":"2023-10-16T09:50:11.949747+0000","last_unstale":"2023-10-16T09:50:14.090011+0000","last_undegraded":"2023-10-16T09:50:14.090011+0000","last_fullsized":"2023-10-16T09:50:14.090011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:36.404480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090250+0000","last_change":"2023-10-16T09:50:12.583623+0000","last_active":"2023-10-16T09:50:14.090250+0000","last_peered":"2023-10-16T09:50:14.090250+0000","last_clean":"2023-10-16T09:50:14.090250+0000","last_became_active":"2023-10-16T09:50:12.581070+0000","last_became_peered":"2023-10-16T09:50:12.581070+0000","last_unstale":"2023-10-16T09:50:14.090250+0000","last_undegraded":"2023-10-16T09:50:14.090250+0000","last_fullsized":"2023-10-16T09:50:14.090250+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:56:39.838249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.802705+0000","last_change":"2023-10-16T09:50:11.947795+0000","last_active":"2023-10-16T09:50:13.802705+0000","last_peered":"2023-10-16T09:50:13.802705+0000","last_clean":"2023-10-16T09:50:13.802705+0000","last_became_active":"2023-10-16T09:50:11.947532+0000","last_became_peered":"2023-10-16T09:50:11.947532+0000","last_unstale":"2023-10-16T09:50:13.802705+0000","last_undegraded":"2023-10-16T09:50:13.802705+0000","last_fullsized":"2023-10-16T09:50:13.802705+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:58:31.879140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.088285+0000","last_change":"2023-10-16T09:50:12.582893+0000","last_active":"2023-10-16T09:50:14.088285+0000","last_peered":"2023-10-16T09:50:14.088285+0000","last_clean":"2023-10-16T09:50:14.088285+0000","last_became_active":"2023-10-16T09:50:12.581938+0000","last_became_peered":"2023-10-16T09:50:12.581938+0000","last_unstale":"2023-10-16T09:50:14.088285+0000","last_undegraded":"2023-10-16T09:50:14.088285+0000","last_fullsized":"2023-10-16T09:50:14.088285+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:39.821588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910280+0000","last_change":"2023-10-16T09:50:11.944661+0000","last_active":"2023-10-16T09:50:13.910280+0000","last_peered":"2023-10-16T09:50:13.910280+0000","last_clean":"2023-10-16T09:50:13.910280+0000","last_became_active":"2023-10-16T09:50:11.944268+0000","last_became_peered":"2023-10-16T09:50:11.944268+0000","last_unstale":"2023-10-16T09:50:13.910280+0000","last_undegraded":"2023-10-16T09:50:13.910280+0000","last_fullsized":"2023-10-16T09:50:13.910280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:26.650824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.909722+0000","last_change":"2023-10-16T09:50:12.596045+0000","last_active":"2023-10-16T09:50:13.909722+0000","last_peered":"2023-10-16T09:50:13.909722+0000","last_clean":"2023-10-16T09:50:13.909722+0000","last_became_active":"2023-10-16T09:50:12.595584+0000","last_became_peered":"2023-10-16T09:50:12.595584+0000","last_unstale":"2023-10-16T09:50:13.909722+0000","last_undegraded":"2023-10-16T09:50:13.909722+0000","last_fullsized":"2023-10-16T09:50:13.909722+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:31.236817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.088146+0000","last_change":"2023-10-16T09:50:11.950425+0000","last_active":"2023-10-16T09:50:14.088146+0000","last_peered":"2023-10-16T09:50:14.088146+0000","last_clean":"2023-10-16T09:50:14.088146+0000","last_became_active":"2023-10-16T09:50:11.950120+0000","last_became_peered":"2023-10-16T09:50:11.950120+0000","last_unstale":"2023-10-16T09:50:14.088146+0000","last_undegraded":"2023-10-16T09:50:14.088146+0000","last_fullsized":"2023-10-16T09:50:14.088146+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:58:31.879140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.564504+0000","last_change":"2023-10-16T09:50:12.588657+0000","last_active":"2023-10-16T09:50:13.564504+0000","last_peered":"2023-10-16T09:50:13.564504+0000","last_clean":"2023-10-16T09:50:13.564504+0000","last_became_active":"2023-10-16T09:50:12.588359+0000","last_became_peered":"2023-10-16T09:50:12.588359+0000","last_unstale":"2023-10-16T09:50:13.564504+0000","last_undegraded":"2023-10-16T09:50:13.564504+0000","last_fullsized":"2023-10-16T09:50:13.564504+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:39.157970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.565670+0000","last_change":"2023-10-16T09:50:11.955005+0000","last_active":"2023-10-16T09:50:13.565670+0000","last_peered":"2023-10-16T09:50:13.565670+0000","last_clean":"2023-10-16T09:50:13.565670+0000","last_became_active":"2023-10-16T09:50:11.954691+0000","last_became_peered":"2023-10-16T09:50:11.954691+0000","last_unstale":"2023-10-16T09:50:13.565670+0000","last_undegraded":"2023-10-16T09:50:13.565670+0000","last_fullsized":"2023-10-16T09:50:13.565670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:06:41.408779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":176,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:50:12.580351+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T09:50:12.580351+0000","last_peered":"2023-10-16T09:50:12.580351+0000","last_clean":"2023-10-16T09:50:12.580351+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T09:50:12.580351+0000","last_undegraded":"2023-10-16T09:50:12.580351+0000","last_fullsized":"2023-10-16T09:50:12.580351+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":983040,"data_stored":742041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:49:54 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":3.694,"5min":3.694,"15min":3.694},"last":1.214}]},{"osd":1,"last update":"Mon Oct 16 09:49:54 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.155}]},{"osd":2,"last update":"Mon Oct 16 09:49:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.7759999999999998,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":1.375}]},{"osd":3,"last update":"Mon Oct 16 09:49:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":1.087}]},{"osd":4,"last update":"Mon Oct 16 09:49:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.6659999999999999}]},{"osd":5,"last update":"Mon Oct 16 09:49:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"min":{"1min":1.02,"5min":1.02,"15min":1.02},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.575}]},{"osd":6,"last update":"Mon Oct 16 09:49:54 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"min":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"max":{"1min":2.6259999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":1.625},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"min":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.5670000000000002,"5min":2.5670000000000002,"15min":2.5670000000000002},"last":1.4830000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":393216,"data_stored":152199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:49:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.04,"5min":2.04,"15min":2.04},"last":0.65400000000000003},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.86799999999999999}]},{"osd":1,"last update":"Mon Oct 16 09:49:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.145},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.0960000000000001}]},{"osd":2,"last update":"Mon Oct 16 09:49:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.413},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.371}]},{"osd":3,"last update":"Mon Oct 16 09:49:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.0249999999999999}]},{"osd":4,"last update":"Mon Oct 16 09:49:55 2023","interfaces":[{"interface":"back","average":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":3.4249999999999998,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":1.321}]},{"osd":5,"last update":"Mon Oct 16 09:49:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":1.512},{"interface":"front","average":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.4930000000000001}]},{"osd":7,"last update":"Mon Oct 16 09:49:55 2023","interfaces":[{"interface":"back","average":{"1min":1.746,"5min":1.746,"15min":1.746},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.8679999999999999,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":1.605},{"interface":"front","average":{"1min":1.742,"5min":1.742,"15min":1.742},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":3.5419999999999998,"5min":3.5419999999999998,"15min":3.5419999999999998},"last":1.5740000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":393216,"data_stored":152255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.325},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":1.1479999999999999}]},{"osd":1,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.4569999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.026}]},{"osd":2,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.468},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.4379999999999999}]},{"osd":3,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.25},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.361}]},{"osd":4,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.659,"5min":1.659,"15min":1.659},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.4019999999999999}]},{"osd":6,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.556,"15min":1.556},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.3580000000000001,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.208}]},{"osd":7,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":1.798,"15min":1.798},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.831,"5min":1.831,"15min":1.831},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.931,"5min":2.931,"15min":2.931},"last":1.2889999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":393216,"data_stored":151687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":4.2640000000000002,"5min":4.2640000000000002,"15min":4.2640000000000002},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":4.391,"5min":4.391,"15min":4.391},"last":0.72399999999999998}]},{"osd":1,"last update":"Mon Oct 16 09:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.2410000000000001}]},{"osd":2,"last update":"Mon Oct 16 09:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.1160000000000001,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.0840000000000001}]},{"osd":3,"last update":"Mon Oct 16 09:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":3.262,"5min":3.262,"15min":3.262},"last":1.042},{"interface":"front","average":{"1min":1.649,"5min":1.649,"15min":1.649},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":3.3050000000000002,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":1.4990000000000001}]},{"osd":5,"last update":"Mon Oct 16 09:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.9780000000000002,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":3.669,"5min":3.669,"15min":3.669},"last":1.548}]},{"osd":6,"last update":"Mon Oct 16 09:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.8090000000000002,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":1.714},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":3.2280000000000002,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.284}]},{"osd":7,"last update":"Mon Oct 16 09:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":3.5150000000000001,"5min":3.5150000000000001,"15min":3.5150000000000001},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.913,"5min":1.913,"15min":1.913},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":3.9849999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.6180000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":146917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.67200000000000004}]},{"osd":1,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.093},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.94399999999999995}]},{"osd":2,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":1.2410000000000001}]},{"osd":4,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.393},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.1930000000000001}]},{"osd":5,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.133},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.157}]},{"osd":6,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.462,"15min":1.462},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.2609999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.581,"15min":1.581},"min":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.659,"5min":1.659,"15min":1.659},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"max":{"1min":2.379,"5min":2.379,"15min":2.379},"last":1.351}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":389120,"data_stored":151653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.96199999999999997}]},{"osd":1,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.6059999999999999,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.89300000000000002}]},{"osd":3,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.7269999999999999,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.86699999999999999}]},{"osd":4,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.1950000000000001}]},{"osd":5,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.419,"15min":1.419},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":2.7770000000000001,"5min":2.7770000000000001,"15min":2.7770000000000001},"last":1.244},{"interface":"front","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.952,"5min":2.952,"15min":2.952},"last":1.1319999999999999}]},{"osd":6,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.8180000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.5}]},{"osd":7,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":3.746,"5min":3.746,"15min":3.746},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.2629999999999999,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":1.4830000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":153217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.82599999999999996}]},{"osd":2,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.78900000000000003}]},{"osd":3,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.8109999999999999,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":0.68600000000000005}]},{"osd":4,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.516,"5min":1.516,"15min":1.516},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.851,"5min":2.851,"15min":2.851},"last":1.391}]},{"osd":5,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.7120000000000002,"5min":2.7120000000000002,"15min":2.7120000000000002},"last":1.119},{"interface":"front","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":1.268}]},{"osd":6,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.522,"15min":1.522},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.75,"5min":2.75,"15min":2.75},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.669,"5min":2.669,"15min":2.669},"last":1.3520000000000001}]},{"osd":7,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.8799999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":3.1400000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":1.4790000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":735651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 09:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Mon Oct 16 09:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.48899999999999999}]},{"osd":3,"last update":"Mon Oct 16 09:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":0.51200000000000001}]},{"osd":4,"last update":"Mon Oct 16 09:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.206}]},{"osd":5,"last update":"Mon Oct 16 09:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.8159999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.9009999999999998,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.996}]},{"osd":6,"last update":"Mon Oct 16 09:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.7160000000000002,"5min":2.7160000000000002,"15min":2.7160000000000002},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":1.0169999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":1.155}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:50:17.378 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:50:17.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.635+0000 7fd703ba9640 1 Processor -- start 2023-10-16T09:50:17.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.635+0000 7fd703ba9640 1 -- start start 2023-10-16T09:50:17.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.636+0000 7fd703ba9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd6fc159f00 0x7fd6fc0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:17.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.637+0000 7fd703ba9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6fc06f7f0 con 0x7fd6fc15d880 2023-10-16T09:50:17.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.637+0000 7fd703ba9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6fc06c5e0 con 0x7fd6fc159b20 2023-10-16T09:50:17.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.637+0000 7fd703ba9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6fc06c8c0 con 0x7fd6fc159f00 2023-10-16T09:50:17.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.637+0000 7fd702146640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd6fc15d880 0x7fd6fc06fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33556/0 (socket says 172.21.15.84:33556) 2023-10-16T09:50:17.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.637+0000 7fd702146640 1 -- 172.21.15.84:0/2378668144 learned_addr learned my addr 172.21.15.84:0/2378668144 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:17.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.637+0000 7fd701144640 1 --2- 172.21.15.84:0/2378668144 >> v2:172.21.15.84:3300/0 conn(0x7fd6fc159f00 0x7fd6fc0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:17.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.637+0000 7fd700943640 1 -- 172.21.15.84:0/2378668144 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378357266 0 0) 0x7fd6fc06f7f0 con 0x7fd6fc15d880 2023-10-16T09:50:17.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.637+0000 7fd700943640 1 -- 172.21.15.84:0/2378668144 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6e8003520 con 0x7fd6fc15d880 2023-10-16T09:50:17.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.638+0000 7fd701144640 1 -- 172.21.15.84:0/2378668144 >> v1:172.21.15.157:6789/0 conn(0x7fd6fc159b20 legacy=0x7fd6fc06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:17.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.638+0000 7fd701144640 1 -- 172.21.15.84:0/2378668144 >> v1:172.21.15.84:6789/0 conn(0x7fd6fc15d880 legacy=0x7fd6fc06fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:17.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.638+0000 7fd701144640 1 -- 172.21.15.84:0/2378668144 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc06cba0 con 0x7fd6fc159f00 2023-10-16T09:50:17.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.639+0000 7fd701144640 1 --2- 172.21.15.84:0/2378668144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc159f00 0x7fd6fc0b0410 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0097a0 tx=0x7fd6ec02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9959880d2482220 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:17.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.639+0000 7fd700943640 1 -- 172.21.15.84:0/2378668144 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec030610 con 0x7fd6fc159f00 2023-10-16T09:50:17.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.639+0000 7fd700943640 1 -- 172.21.15.84:0/2378668144 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ec030770 con 0x7fd6fc159f00 2023-10-16T09:50:17.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.639+0000 7fd700943640 1 -- 172.21.15.84:0/2378668144 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec030b00 con 0x7fd6fc159f00 2023-10-16T09:50:17.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.639+0000 7fd703ba9640 1 -- 172.21.15.84:0/2378668144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc159f00 msgr2=0x7fd6fc0b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:17.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.639+0000 7fd703ba9640 1 --2- 172.21.15.84:0/2378668144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc159f00 0x7fd6fc0b0410 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0097a0 tx=0x7fd6ec02b960 comp rx=0 tx=0).stop 2023-10-16T09:50:17.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.640+0000 7fd703ba9640 1 -- 172.21.15.84:0/2378668144 shutdown_connections 2023-10-16T09:50:17.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.640+0000 7fd703ba9640 1 --2- 172.21.15.84:0/2378668144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc159f00 0x7fd6fc0b0410 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:17.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.640+0000 7fd703ba9640 1 -- 172.21.15.84:0/2378668144 >> 172.21.15.84:0/2378668144 conn(0x7fd6fc0cb410 msgr2=0x7fd6fc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:17.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.640+0000 7fd703ba9640 1 -- 172.21.15.84:0/2378668144 shutdown_connections 2023-10-16T09:50:17.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.640+0000 7fd703ba9640 1 -- 172.21.15.84:0/2378668144 wait complete. 2023-10-16T09:50:17.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.641+0000 7fd703ba9640 1 Processor -- start 2023-10-16T09:50:17.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.642+0000 7fd703ba9640 1 -- start start 2023-10-16T09:50:17.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.642+0000 7fd703ba9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6fc159b20 0x7fd6fc0c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:17.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.642+0000 7fd703ba9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6fc159f00 0x7fd6fc14f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:17.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.642+0000 7fd703ba9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc15d880 0x7fd6fc0c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:17.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.642+0000 7fd703ba9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0c6d70 con 0x7fd6fc15d880 2023-10-16T09:50:17.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.642+0000 7fd703ba9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0c5490 con 0x7fd6fc159f00 2023-10-16T09:50:17.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.642+0000 7fd703ba9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0c60f0 con 0x7fd6fc159b20 2023-10-16T09:50:17.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.643+0000 7fd702146640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc15d880 0x7fd6fc0c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:17.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.643+0000 7fd702146640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc15d880 0x7fd6fc0c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37604/0 (socket says 172.21.15.84:37604) 2023-10-16T09:50:17.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.643+0000 7fd702146640 1 -- 172.21.15.84:0/3787532345 learned_addr learned my addr 172.21.15.84:0/3787532345 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:17.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.643+0000 7fd701945640 1 --2- 172.21.15.84:0/3787532345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6fc159b20 0x7fd6fc0c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:17.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.643+0000 7fd701144640 1 --2- 172.21.15.84:0/3787532345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6fc159f00 0x7fd6fc14f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:17.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.643+0000 7fd702146640 1 -- 172.21.15.84:0/3787532345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6fc159b20 msgr2=0x7fd6fc0c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:17.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.643+0000 7fd702146640 1 --2- 172.21.15.84:0/3787532345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6fc159b20 0x7fd6fc0c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:17.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.643+0000 7fd702146640 1 -- 172.21.15.84:0/3787532345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6fc159f00 msgr2=0x7fd6fc14f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:17.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.643+0000 7fd702146640 1 --2- 172.21.15.84:0/3787532345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6fc159f00 0x7fd6fc14f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:17.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.643+0000 7fd702146640 1 -- 172.21.15.84:0/3787532345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec009400 con 0x7fd6fc15d880 2023-10-16T09:50:17.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.643+0000 7fd701945640 1 --2- 172.21.15.84:0/3787532345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6fc159b20 0x7fd6fc0c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:17.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.643+0000 7fd702146640 1 --2- 172.21.15.84:0/3787532345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc15d880 0x7fd6fc0c7190 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400cb70 tx=0x7fd6f400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:17.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.644+0000 7fd6f27fc640 1 -- 172.21.15.84:0/3787532345 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400ad40 con 0x7fd6fc15d880 2023-10-16T09:50:17.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.644+0000 7fd6f27fc640 1 -- 172.21.15.84:0/3787532345 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f4012400 con 0x7fd6fc15d880 2023-10-16T09:50:17.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.644+0000 7fd6f27fc640 1 -- 172.21.15.84:0/3787532345 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400aea0 con 0x7fd6fc15d880 2023-10-16T09:50:17.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.644+0000 7fd703ba9640 1 -- 172.21.15.84:0/3787532345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6fc0c63d0 con 0x7fd6fc15d880 2023-10-16T09:50:17.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.644+0000 7fd703ba9640 1 -- 172.21.15.84:0/3787532345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6fc0c6990 con 0x7fd6fc15d880 2023-10-16T09:50:17.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.645+0000 7fd703ba9640 1 -- 172.21.15.84:0/3787532345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6fc0c6eb0 con 0x7fd6fc15d880 2023-10-16T09:50:17.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.646+0000 7fd6f27fc640 1 -- 172.21.15.84:0/3787532345 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd6f4007070 con 0x7fd6fc15d880 2023-10-16T09:50:17.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.647+0000 7fd6f27fc640 1 --2- 172.21.15.84:0/3787532345 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd6e006ae70 0x7fd6e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:17.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.647+0000 7fd6f27fc640 1 -- 172.21.15.84:0/3787532345 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6f40895a0 con 0x7fd6fc15d880 2023-10-16T09:50:17.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.647+0000 7fd701945640 1 --2- 172.21.15.84:0/3787532345 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd6e006ae70 0x7fd6e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:17.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.648+0000 7fd701945640 1 --2- 172.21.15.84:0/3787532345 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd6e006ae70 0x7fd6e006d330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8000f60 tx=0x7fd6f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:17.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.652+0000 7fd6f27fc640 1 -- 172.21.15.84:0/3787532345 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd6fc0c6eb0 con 0x7fd6fc15d880 2023-10-16T09:50:17.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.840+0000 7fd703ba9640 1 -- 172.21.15.84:0/3787532345 --> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6fc06c2f0 con 0x7fd6e006ae70 2023-10-16T09:50:17.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.856+0000 7fd6f27fc640 1 -- 172.21.15.84:0/3787532345 <== mgr.4107 v2:172.21.15.84:6832/3010246666 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286730 (secure 0 0 0) 0x7fd6f8001c90 con 0x7fd6e006ae70 2023-10-16T09:50:17.862 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:50:17.863 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T09:50:17.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.861+0000 7fd703ba9640 1 -- 172.21.15.84:0/3787532345 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd6e006ae70 msgr2=0x7fd6e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:17.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.861+0000 7fd703ba9640 1 --2- 172.21.15.84:0/3787532345 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd6e006ae70 0x7fd6e006d330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8000f60 tx=0x7fd6f8009040 comp rx=0 tx=0).stop 2023-10-16T09:50:17.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.861+0000 7fd703ba9640 1 -- 172.21.15.84:0/3787532345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc15d880 msgr2=0x7fd6fc0c7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:17.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.861+0000 7fd703ba9640 1 --2- 172.21.15.84:0/3787532345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc15d880 0x7fd6fc0c7190 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400cb70 tx=0x7fd6f400a430 comp rx=0 tx=0).stop 2023-10-16T09:50:17.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.861+0000 7fd703ba9640 1 -- 172.21.15.84:0/3787532345 shutdown_connections 2023-10-16T09:50:17.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.861+0000 7fd703ba9640 1 --2- 172.21.15.84:0/3787532345 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fd6e006ae70 0x7fd6e006d330 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:17.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.861+0000 7fd703ba9640 1 --2- 172.21.15.84:0/3787532345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc15d880 0x7fd6fc0c7190 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:17.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.861+0000 7fd703ba9640 1 --2- 172.21.15.84:0/3787532345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6fc159f00 0x7fd6fc14f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:17.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.861+0000 7fd703ba9640 1 --2- 172.21.15.84:0/3787532345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6fc159b20 0x7fd6fc0c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:17.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.861+0000 7fd703ba9640 1 -- 172.21.15.84:0/3787532345 >> 172.21.15.84:0/3787532345 conn(0x7fd6fc0cb410 msgr2=0x7fd6fc0cc540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:17.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.862+0000 7fd703ba9640 1 -- 172.21.15.84:0/3787532345 shutdown_connections 2023-10-16T09:50:17.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:17.862+0000 7fd703ba9640 1 -- 172.21.15.84:0/3787532345 wait complete. 2023-10-16T09:50:17.883 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2023-10-16T09:50:17.615154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237060,"kb_used_data":4208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737660,"statfs":{"total":773094113280,"available":772851363840,"internally_reserved":0,"allocated":4308992,"data_stored":2385620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001660"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.566246+0000","last_change":"2023-10-16T09:50:12.596909+0000","last_active":"2023-10-16T09:50:13.566246+0000","last_peered":"2023-10-16T09:50:13.566246+0000","last_clean":"2023-10-16T09:50:13.566246+0000","last_became_active":"2023-10-16T09:50:12.596628+0000","last_became_peered":"2023-10-16T09:50:12.596628+0000","last_unstale":"2023-10-16T09:50:13.566246+0000","last_undegraded":"2023-10-16T09:50:13.566246+0000","last_fullsized":"2023-10-16T09:50:13.566246+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:58.217542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089952+0000","last_change":"2023-10-16T09:50:12.592882+0000","last_active":"2023-10-16T09:50:14.089952+0000","last_peered":"2023-10-16T09:50:14.089952+0000","last_clean":"2023-10-16T09:50:14.089952+0000","last_became_active":"2023-10-16T09:50:12.592375+0000","last_became_peered":"2023-10-16T09:50:12.592375+0000","last_unstale":"2023-10-16T09:50:14.089952+0000","last_undegraded":"2023-10-16T09:50:14.089952+0000","last_fullsized":"2023-10-16T09:50:14.089952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:26.528764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089717+0000","last_change":"2023-10-16T09:50:12.602127+0000","last_active":"2023-10-16T09:50:14.089717+0000","last_peered":"2023-10-16T09:50:14.089717+0000","last_clean":"2023-10-16T09:50:14.089717+0000","last_became_active":"2023-10-16T09:50:12.601856+0000","last_became_peered":"2023-10-16T09:50:12.601856+0000","last_unstale":"2023-10-16T09:50:14.089717+0000","last_undegraded":"2023-10-16T09:50:14.089717+0000","last_fullsized":"2023-10-16T09:50:14.089717+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:34.869138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.802897+0000","last_change":"2023-10-16T09:50:12.589977+0000","last_active":"2023-10-16T09:50:13.802897+0000","last_peered":"2023-10-16T09:50:13.802897+0000","last_clean":"2023-10-16T09:50:13.802897+0000","last_became_active":"2023-10-16T09:50:12.589662+0000","last_became_peered":"2023-10-16T09:50:12.589662+0000","last_unstale":"2023-10-16T09:50:13.802897+0000","last_undegraded":"2023-10-16T09:50:13.802897+0000","last_fullsized":"2023-10-16T09:50:13.802897+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:35.334811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.561893+0000","last_change":"2023-10-16T09:50:12.586223+0000","last_active":"2023-10-16T09:50:13.561893+0000","last_peered":"2023-10-16T09:50:13.561893+0000","last_clean":"2023-10-16T09:50:13.561893+0000","last_became_active":"2023-10-16T09:50:12.585830+0000","last_became_peered":"2023-10-16T09:50:12.585830+0000","last_unstale":"2023-10-16T09:50:13.561893+0000","last_undegraded":"2023-10-16T09:50:13.561893+0000","last_fullsized":"2023-10-16T09:50:13.561893+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:06:42.423962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.088892+0000","last_change":"2023-10-16T09:50:12.596153+0000","last_active":"2023-10-16T09:50:14.088892+0000","last_peered":"2023-10-16T09:50:14.088892+0000","last_clean":"2023-10-16T09:50:14.088892+0000","last_became_active":"2023-10-16T09:50:12.595887+0000","last_became_peered":"2023-10-16T09:50:12.595887+0000","last_unstale":"2023-10-16T09:50:14.088892+0000","last_undegraded":"2023-10-16T09:50:14.088892+0000","last_fullsized":"2023-10-16T09:50:14.088892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:56:52.867292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.564531+0000","last_change":"2023-10-16T09:50:12.599333+0000","last_active":"2023-10-16T09:50:13.564531+0000","last_peered":"2023-10-16T09:50:13.564531+0000","last_clean":"2023-10-16T09:50:13.564531+0000","last_became_active":"2023-10-16T09:50:12.598339+0000","last_became_peered":"2023-10-16T09:50:12.598339+0000","last_unstale":"2023-10-16T09:50:13.564531+0000","last_undegraded":"2023-10-16T09:50:13.564531+0000","last_fullsized":"2023-10-16T09:50:13.564531+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:55.517446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.563504+0000","last_change":"2023-10-16T09:50:12.588032+0000","last_active":"2023-10-16T09:50:13.563504+0000","last_peered":"2023-10-16T09:50:13.563504+0000","last_clean":"2023-10-16T09:50:13.563504+0000","last_became_active":"2023-10-16T09:50:12.587591+0000","last_became_peered":"2023-10-16T09:50:12.587591+0000","last_unstale":"2023-10-16T09:50:13.563504+0000","last_undegraded":"2023-10-16T09:50:13.563504+0000","last_fullsized":"2023-10-16T09:50:13.563504+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:01.222630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910905+0000","last_change":"2023-10-16T09:50:12.588544+0000","last_active":"2023-10-16T09:50:13.910905+0000","last_peered":"2023-10-16T09:50:13.910905+0000","last_clean":"2023-10-16T09:50:13.910905+0000","last_became_active":"2023-10-16T09:50:12.588091+0000","last_became_peered":"2023-10-16T09:50:12.588091+0000","last_unstale":"2023-10-16T09:50:13.910905+0000","last_undegraded":"2023-10-16T09:50:13.910905+0000","last_fullsized":"2023-10-16T09:50:13.910905+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:35.334811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.803485+0000","last_change":"2023-10-16T09:50:12.588302+0000","last_active":"2023-10-16T09:50:13.803485+0000","last_peered":"2023-10-16T09:50:13.803485+0000","last_clean":"2023-10-16T09:50:13.803485+0000","last_became_active":"2023-10-16T09:50:12.586374+0000","last_became_peered":"2023-10-16T09:50:12.586374+0000","last_unstale":"2023-10-16T09:50:13.803485+0000","last_undegraded":"2023-10-16T09:50:13.803485+0000","last_fullsized":"2023-10-16T09:50:13.803485+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:17:15.300568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.565582+0000","last_change":"2023-10-16T09:50:12.589723+0000","last_active":"2023-10-16T09:50:13.565582+0000","last_peered":"2023-10-16T09:50:13.565582+0000","last_clean":"2023-10-16T09:50:13.565582+0000","last_became_active":"2023-10-16T09:50:12.588705+0000","last_became_peered":"2023-10-16T09:50:12.588705+0000","last_unstale":"2023-10-16T09:50:13.565582+0000","last_undegraded":"2023-10-16T09:50:13.565582+0000","last_fullsized":"2023-10-16T09:50:13.565582+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:15.603817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090184+0000","last_change":"2023-10-16T09:50:12.595733+0000","last_active":"2023-10-16T09:50:14.090184+0000","last_peered":"2023-10-16T09:50:14.090184+0000","last_clean":"2023-10-16T09:50:14.090184+0000","last_became_active":"2023-10-16T09:50:12.594960+0000","last_became_peered":"2023-10-16T09:50:12.594960+0000","last_unstale":"2023-10-16T09:50:14.090184+0000","last_undegraded":"2023-10-16T09:50:14.090184+0000","last_fullsized":"2023-10-16T09:50:14.090184+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:40:06.173040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.562059+0000","last_change":"2023-10-16T09:50:12.580073+0000","last_active":"2023-10-16T09:50:13.562059+0000","last_peered":"2023-10-16T09:50:13.562059+0000","last_clean":"2023-10-16T09:50:13.562059+0000","last_became_active":"2023-10-16T09:50:12.579795+0000","last_became_peered":"2023-10-16T09:50:12.579795+0000","last_unstale":"2023-10-16T09:50:13.562059+0000","last_undegraded":"2023-10-16T09:50:13.562059+0000","last_fullsized":"2023-10-16T09:50:13.562059+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:27.666006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089119+0000","last_change":"2023-10-16T09:50:12.586337+0000","last_active":"2023-10-16T09:50:14.089119+0000","last_peered":"2023-10-16T09:50:14.089119+0000","last_clean":"2023-10-16T09:50:14.089119+0000","last_became_active":"2023-10-16T09:50:12.585617+0000","last_became_peered":"2023-10-16T09:50:12.585617+0000","last_unstale":"2023-10-16T09:50:14.089119+0000","last_undegraded":"2023-10-16T09:50:14.089119+0000","last_fullsized":"2023-10-16T09:50:14.089119+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:40:06.173040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089405+0000","last_change":"2023-10-16T09:50:12.601453+0000","last_active":"2023-10-16T09:50:14.089405+0000","last_peered":"2023-10-16T09:50:14.089405+0000","last_clean":"2023-10-16T09:50:14.089405+0000","last_became_active":"2023-10-16T09:50:12.601178+0000","last_became_peered":"2023-10-16T09:50:12.601178+0000","last_unstale":"2023-10-16T09:50:14.089405+0000","last_undegraded":"2023-10-16T09:50:14.089405+0000","last_fullsized":"2023-10-16T09:50:14.089405+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:47:57.680638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.564499+0000","last_change":"2023-10-16T09:50:12.600618+0000","last_active":"2023-10-16T09:50:13.564499+0000","last_peered":"2023-10-16T09:50:13.564499+0000","last_clean":"2023-10-16T09:50:13.564499+0000","last_became_active":"2023-10-16T09:50:12.600211+0000","last_became_peered":"2023-10-16T09:50:12.600211+0000","last_unstale":"2023-10-16T09:50:13.564499+0000","last_undegraded":"2023-10-16T09:50:13.564499+0000","last_fullsized":"2023-10-16T09:50:13.564499+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:59:06.130674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.564369+0000","last_change":"2023-10-16T09:50:12.593082+0000","last_active":"2023-10-16T09:50:13.564369+0000","last_peered":"2023-10-16T09:50:13.564369+0000","last_clean":"2023-10-16T09:50:13.564369+0000","last_became_active":"2023-10-16T09:50:12.592786+0000","last_became_peered":"2023-10-16T09:50:12.592786+0000","last_unstale":"2023-10-16T09:50:13.564369+0000","last_undegraded":"2023-10-16T09:50:13.564369+0000","last_fullsized":"2023-10-16T09:50:13.564369+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:26.528764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090277+0000","last_change":"2023-10-16T09:50:12.584808+0000","last_active":"2023-10-16T09:50:14.090277+0000","last_peered":"2023-10-16T09:50:14.090277+0000","last_clean":"2023-10-16T09:50:14.090277+0000","last_became_active":"2023-10-16T09:50:12.584474+0000","last_became_peered":"2023-10-16T09:50:12.584474+0000","last_unstale":"2023-10-16T09:50:14.090277+0000","last_undegraded":"2023-10-16T09:50:14.090277+0000","last_fullsized":"2023-10-16T09:50:14.090277+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:59:06.130674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090287+0000","last_change":"2023-10-16T09:50:12.590207+0000","last_active":"2023-10-16T09:50:14.090287+0000","last_peered":"2023-10-16T09:50:14.090287+0000","last_clean":"2023-10-16T09:50:14.090287+0000","last_became_active":"2023-10-16T09:50:12.588887+0000","last_became_peered":"2023-10-16T09:50:12.588887+0000","last_unstale":"2023-10-16T09:50:14.090287+0000","last_undegraded":"2023-10-16T09:50:14.090287+0000","last_fullsized":"2023-10-16T09:50:14.090287+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:39.157970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.562209+0000","last_change":"2023-10-16T09:50:12.587777+0000","last_active":"2023-10-16T09:50:13.562209+0000","last_peered":"2023-10-16T09:50:13.562209+0000","last_clean":"2023-10-16T09:50:13.562209+0000","last_became_active":"2023-10-16T09:50:12.587497+0000","last_became_peered":"2023-10-16T09:50:12.587497+0000","last_unstale":"2023-10-16T09:50:13.562209+0000","last_undegraded":"2023-10-16T09:50:13.562209+0000","last_fullsized":"2023-10-16T09:50:13.562209+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:09:40.888338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910140+0000","last_change":"2023-10-16T09:50:12.596464+0000","last_active":"2023-10-16T09:50:13.910140+0000","last_peered":"2023-10-16T09:50:13.910140+0000","last_clean":"2023-10-16T09:50:13.910140+0000","last_became_active":"2023-10-16T09:50:12.595509+0000","last_became_peered":"2023-10-16T09:50:12.595509+0000","last_unstale":"2023-10-16T09:50:13.910140+0000","last_undegraded":"2023-10-16T09:50:13.910140+0000","last_fullsized":"2023-10-16T09:50:13.910140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:55:02.113527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910050+0000","last_change":"2023-10-16T09:50:12.593140+0000","last_active":"2023-10-16T09:50:13.910050+0000","last_peered":"2023-10-16T09:50:13.910050+0000","last_clean":"2023-10-16T09:50:13.910050+0000","last_became_active":"2023-10-16T09:50:12.592837+0000","last_became_peered":"2023-10-16T09:50:12.592837+0000","last_unstale":"2023-10-16T09:50:13.910050+0000","last_undegraded":"2023-10-16T09:50:13.910050+0000","last_fullsized":"2023-10-16T09:50:13.910050+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:56:39.838249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.088664+0000","last_change":"2023-10-16T09:50:12.584277+0000","last_active":"2023-10-16T09:50:14.088664+0000","last_peered":"2023-10-16T09:50:14.088664+0000","last_clean":"2023-10-16T09:50:14.088664+0000","last_became_active":"2023-10-16T09:50:12.583278+0000","last_became_peered":"2023-10-16T09:50:12.583278+0000","last_unstale":"2023-10-16T09:50:14.088664+0000","last_undegraded":"2023-10-16T09:50:14.088664+0000","last_fullsized":"2023-10-16T09:50:14.088664+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:58:32.894323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089407+0000","last_change":"2023-10-16T09:50:12.597752+0000","last_active":"2023-10-16T09:50:14.089407+0000","last_peered":"2023-10-16T09:50:14.089407+0000","last_clean":"2023-10-16T09:50:14.089407+0000","last_became_active":"2023-10-16T09:50:12.595650+0000","last_became_peered":"2023-10-16T09:50:12.595650+0000","last_unstale":"2023-10-16T09:50:14.089407+0000","last_undegraded":"2023-10-16T09:50:14.089407+0000","last_fullsized":"2023-10-16T09:50:14.089407+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:18:26.654045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.803607+0000","last_change":"2023-10-16T09:50:12.585217+0000","last_active":"2023-10-16T09:50:13.803607+0000","last_peered":"2023-10-16T09:50:13.803607+0000","last_clean":"2023-10-16T09:50:13.803607+0000","last_became_active":"2023-10-16T09:50:12.584946+0000","last_became_peered":"2023-10-16T09:50:12.584946+0000","last_unstale":"2023-10-16T09:50:13.803607+0000","last_undegraded":"2023-10-16T09:50:13.803607+0000","last_fullsized":"2023-10-16T09:50:13.803607+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:34.869138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090554+0000","last_change":"2023-10-16T09:50:12.586890+0000","last_active":"2023-10-16T09:50:14.090554+0000","last_peered":"2023-10-16T09:50:14.090554+0000","last_clean":"2023-10-16T09:50:14.090554+0000","last_became_active":"2023-10-16T09:50:12.586570+0000","last_became_peered":"2023-10-16T09:50:12.586570+0000","last_unstale":"2023-10-16T09:50:14.090554+0000","last_undegraded":"2023-10-16T09:50:14.090554+0000","last_fullsized":"2023-10-16T09:50:14.090554+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:55.517446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.803275+0000","last_change":"2023-10-16T09:50:12.591299+0000","last_active":"2023-10-16T09:50:13.803275+0000","last_peered":"2023-10-16T09:50:13.803275+0000","last_clean":"2023-10-16T09:50:13.803275+0000","last_became_active":"2023-10-16T09:50:12.590387+0000","last_became_peered":"2023-10-16T09:50:12.590387+0000","last_unstale":"2023-10-16T09:50:13.803275+0000","last_undegraded":"2023-10-16T09:50:13.803275+0000","last_fullsized":"2023-10-16T09:50:13.803275+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:18:26.654045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090259+0000","last_change":"2023-10-16T09:50:12.591942+0000","last_active":"2023-10-16T09:50:14.090259+0000","last_peered":"2023-10-16T09:50:14.090259+0000","last_clean":"2023-10-16T09:50:14.090259+0000","last_became_active":"2023-10-16T09:50:12.591651+0000","last_became_peered":"2023-10-16T09:50:12.591651+0000","last_unstale":"2023-10-16T09:50:14.090259+0000","last_undegraded":"2023-10-16T09:50:14.090259+0000","last_fullsized":"2023-10-16T09:50:14.090259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:27.920687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910232+0000","last_change":"2023-10-16T09:50:12.597897+0000","last_active":"2023-10-16T09:50:13.910232+0000","last_peered":"2023-10-16T09:50:13.910232+0000","last_clean":"2023-10-16T09:50:13.910232+0000","last_became_active":"2023-10-16T09:50:12.597382+0000","last_became_peered":"2023-10-16T09:50:12.597382+0000","last_unstale":"2023-10-16T09:50:13.910232+0000","last_undegraded":"2023-10-16T09:50:13.910232+0000","last_fullsized":"2023-10-16T09:50:13.910232+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:35.275931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.563737+0000","last_change":"2023-10-16T09:50:12.600791+0000","last_active":"2023-10-16T09:50:13.563737+0000","last_peered":"2023-10-16T09:50:13.563737+0000","last_clean":"2023-10-16T09:50:13.563737+0000","last_became_active":"2023-10-16T09:50:12.600521+0000","last_became_peered":"2023-10-16T09:50:12.600521+0000","last_unstale":"2023-10-16T09:50:13.563737+0000","last_undegraded":"2023-10-16T09:50:13.563737+0000","last_fullsized":"2023-10-16T09:50:13.563737+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:39.821588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.565969+0000","last_change":"2023-10-16T09:50:12.581079+0000","last_active":"2023-10-16T09:50:13.565969+0000","last_peered":"2023-10-16T09:50:13.565969+0000","last_clean":"2023-10-16T09:50:13.565969+0000","last_became_active":"2023-10-16T09:50:12.580696+0000","last_became_peered":"2023-10-16T09:50:12.580696+0000","last_unstale":"2023-10-16T09:50:13.565969+0000","last_undegraded":"2023-10-16T09:50:13.565969+0000","last_fullsized":"2023-10-16T09:50:13.565969+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:55:27.040013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.564004+0000","last_change":"2023-10-16T09:50:12.598141+0000","last_active":"2023-10-16T09:50:13.564004+0000","last_peered":"2023-10-16T09:50:13.564004+0000","last_clean":"2023-10-16T09:50:13.564004+0000","last_became_active":"2023-10-16T09:50:12.593867+0000","last_became_peered":"2023-10-16T09:50:12.593867+0000","last_unstale":"2023-10-16T09:50:13.564004+0000","last_undegraded":"2023-10-16T09:50:13.564004+0000","last_fullsized":"2023-10-16T09:50:13.564004+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:27.920687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089749+0000","last_change":"2023-10-16T09:50:12.583938+0000","last_active":"2023-10-16T09:50:14.089749+0000","last_peered":"2023-10-16T09:50:14.089749+0000","last_clean":"2023-10-16T09:50:14.089749+0000","last_became_active":"2023-10-16T09:50:12.583645+0000","last_became_peered":"2023-10-16T09:50:12.583645+0000","last_unstale":"2023-10-16T09:50:14.089749+0000","last_undegraded":"2023-10-16T09:50:14.089749+0000","last_fullsized":"2023-10-16T09:50:14.089749+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:51.573863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.566397+0000","last_change":"2023-10-16T09:50:11.955907+0000","last_active":"2023-10-16T09:50:13.566397+0000","last_peered":"2023-10-16T09:50:13.566397+0000","last_clean":"2023-10-16T09:50:13.566397+0000","last_became_active":"2023-10-16T09:50:11.955616+0000","last_became_peered":"2023-10-16T09:50:11.955616+0000","last_unstale":"2023-10-16T09:50:13.566397+0000","last_undegraded":"2023-10-16T09:50:13.566397+0000","last_fullsized":"2023-10-16T09:50:13.566397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:43.336545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.566264+0000","last_change":"2023-10-16T09:50:12.590779+0000","last_active":"2023-10-16T09:50:13.566264+0000","last_peered":"2023-10-16T09:50:13.566264+0000","last_clean":"2023-10-16T09:50:13.566264+0000","last_became_active":"2023-10-16T09:50:12.590485+0000","last_became_peered":"2023-10-16T09:50:12.590485+0000","last_unstale":"2023-10-16T09:50:13.566264+0000","last_undegraded":"2023-10-16T09:50:13.566264+0000","last_fullsized":"2023-10-16T09:50:13.566264+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:58:32.894323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089947+0000","last_change":"2023-10-16T09:50:11.949795+0000","last_active":"2023-10-16T09:50:14.089947+0000","last_peered":"2023-10-16T09:50:14.089947+0000","last_clean":"2023-10-16T09:50:14.089947+0000","last_became_active":"2023-10-16T09:50:11.949558+0000","last_became_peered":"2023-10-16T09:50:11.949558+0000","last_unstale":"2023-10-16T09:50:14.089947+0000","last_undegraded":"2023-10-16T09:50:14.089947+0000","last_fullsized":"2023-10-16T09:50:14.089947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:55:26.024831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910602+0000","last_change":"2023-10-16T09:50:12.597787+0000","last_active":"2023-10-16T09:50:13.910602+0000","last_peered":"2023-10-16T09:50:13.910602+0000","last_clean":"2023-10-16T09:50:13.910602+0000","last_became_active":"2023-10-16T09:50:12.597406+0000","last_became_peered":"2023-10-16T09:50:12.597406+0000","last_unstale":"2023-10-16T09:50:13.910602+0000","last_undegraded":"2023-10-16T09:50:13.910602+0000","last_fullsized":"2023-10-16T09:50:13.910602+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:47.181025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090485+0000","last_change":"2023-10-16T09:50:11.942585+0000","last_active":"2023-10-16T09:50:14.090485+0000","last_peered":"2023-10-16T09:50:14.090485+0000","last_clean":"2023-10-16T09:50:14.090485+0000","last_became_active":"2023-10-16T09:50:11.942333+0000","last_became_peered":"2023-10-16T09:50:11.942333+0000","last_unstale":"2023-10-16T09:50:14.090485+0000","last_undegraded":"2023-10-16T09:50:14.090485+0000","last_fullsized":"2023-10-16T09:50:14.090485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:46.165842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.803228+0000","last_change":"2023-10-16T09:50:12.585893+0000","last_active":"2023-10-16T09:50:13.803228+0000","last_peered":"2023-10-16T09:50:13.803228+0000","last_clean":"2023-10-16T09:50:13.803228+0000","last_became_active":"2023-10-16T09:50:12.585616+0000","last_became_peered":"2023-10-16T09:50:12.585616+0000","last_unstale":"2023-10-16T09:50:13.803228+0000","last_undegraded":"2023-10-16T09:50:13.803228+0000","last_fullsized":"2023-10-16T09:50:13.803228+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:47:57.680638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.562364+0000","last_change":"2023-10-16T09:50:11.942077+0000","last_active":"2023-10-16T09:50:13.562364+0000","last_peered":"2023-10-16T09:50:13.562364+0000","last_clean":"2023-10-16T09:50:13.562364+0000","last_became_active":"2023-10-16T09:50:11.941696+0000","last_became_peered":"2023-10-16T09:50:11.941696+0000","last_unstale":"2023-10-16T09:50:13.562364+0000","last_undegraded":"2023-10-16T09:50:13.562364+0000","last_fullsized":"2023-10-16T09:50:13.562364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:10:45.453057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.563777+0000","last_change":"2023-10-16T09:50:12.599549+0000","last_active":"2023-10-16T09:50:13.563777+0000","last_peered":"2023-10-16T09:50:13.563777+0000","last_clean":"2023-10-16T09:50:13.563777+0000","last_became_active":"2023-10-16T09:50:12.599254+0000","last_became_peered":"2023-10-16T09:50:12.599254+0000","last_unstale":"2023-10-16T09:50:13.563777+0000","last_undegraded":"2023-10-16T09:50:13.563777+0000","last_fullsized":"2023-10-16T09:50:13.563777+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:56:52.867292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910427+0000","last_change":"2023-10-16T09:50:11.948043+0000","last_active":"2023-10-16T09:50:13.910427+0000","last_peered":"2023-10-16T09:50:13.910427+0000","last_clean":"2023-10-16T09:50:13.910427+0000","last_became_active":"2023-10-16T09:50:11.947746+0000","last_became_peered":"2023-10-16T09:50:11.947746+0000","last_unstale":"2023-10-16T09:50:13.910427+0000","last_undegraded":"2023-10-16T09:50:13.910427+0000","last_fullsized":"2023-10-16T09:50:13.910427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:20.357575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089461+0000","last_change":"2023-10-16T09:50:12.589951+0000","last_active":"2023-10-16T09:50:14.089461+0000","last_peered":"2023-10-16T09:50:14.089461+0000","last_clean":"2023-10-16T09:50:14.089461+0000","last_became_active":"2023-10-16T09:50:12.589588+0000","last_became_peered":"2023-10-16T09:50:12.589588+0000","last_unstale":"2023-10-16T09:50:14.089461+0000","last_undegraded":"2023-10-16T09:50:14.089461+0000","last_fullsized":"2023-10-16T09:50:14.089461+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:17:15.300568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910183+0000","last_change":"2023-10-16T09:50:11.936818+0000","last_active":"2023-10-16T09:50:13.910183+0000","last_peered":"2023-10-16T09:50:13.910183+0000","last_clean":"2023-10-16T09:50:13.910183+0000","last_became_active":"2023-10-16T09:50:11.936296+0000","last_became_peered":"2023-10-16T09:50:11.936296+0000","last_unstale":"2023-10-16T09:50:13.910183+0000","last_undegraded":"2023-10-16T09:50:13.910183+0000","last_fullsized":"2023-10-16T09:50:13.910183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:06:41.408779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.909811+0000","last_change":"2023-10-16T09:50:12.583979+0000","last_active":"2023-10-16T09:50:13.909811+0000","last_peered":"2023-10-16T09:50:13.909811+0000","last_clean":"2023-10-16T09:50:13.909811+0000","last_became_active":"2023-10-16T09:50:12.583666+0000","last_became_peered":"2023-10-16T09:50:12.583666+0000","last_unstale":"2023-10-16T09:50:13.909811+0000","last_undegraded":"2023-10-16T09:50:13.909811+0000","last_fullsized":"2023-10-16T09:50:13.909811+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:51.573863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.088563+0000","last_change":"2023-10-16T09:50:11.936387+0000","last_active":"2023-10-16T09:50:14.088563+0000","last_peered":"2023-10-16T09:50:14.088563+0000","last_clean":"2023-10-16T09:50:14.088563+0000","last_became_active":"2023-10-16T09:50:11.935830+0000","last_became_peered":"2023-10-16T09:50:11.935830+0000","last_unstale":"2023-10-16T09:50:14.088563+0000","last_undegraded":"2023-10-16T09:50:14.088563+0000","last_fullsized":"2023-10-16T09:50:14.088563+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:18:25.638863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.802455+0000","last_change":"2023-10-16T09:50:12.595196+0000","last_active":"2023-10-16T09:50:13.802455+0000","last_peered":"2023-10-16T09:50:13.802455+0000","last_clean":"2023-10-16T09:50:13.802455+0000","last_became_active":"2023-10-16T09:50:12.594843+0000","last_became_peered":"2023-10-16T09:50:12.594843+0000","last_unstale":"2023-10-16T09:50:13.802455+0000","last_undegraded":"2023-10-16T09:50:13.802455+0000","last_fullsized":"2023-10-16T09:50:13.802455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:34.134204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.909806+0000","last_change":"2023-10-16T09:50:11.947404+0000","last_active":"2023-10-16T09:50:13.909806+0000","last_peered":"2023-10-16T09:50:13.909806+0000","last_clean":"2023-10-16T09:50:13.909806+0000","last_became_active":"2023-10-16T09:50:11.947080+0000","last_became_peered":"2023-10-16T09:50:11.947080+0000","last_unstale":"2023-10-16T09:50:13.909806+0000","last_undegraded":"2023-10-16T09:50:13.909806+0000","last_fullsized":"2023-10-16T09:50:13.909806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:51:15.834863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090178+0000","last_change":"2023-10-16T09:50:12.601729+0000","last_active":"2023-10-16T09:50:14.090178+0000","last_peered":"2023-10-16T09:50:14.090178+0000","last_clean":"2023-10-16T09:50:14.090178+0000","last_became_active":"2023-10-16T09:50:12.601454+0000","last_became_peered":"2023-10-16T09:50:12.601454+0000","last_unstale":"2023-10-16T09:50:14.090178+0000","last_undegraded":"2023-10-16T09:50:14.090178+0000","last_fullsized":"2023-10-16T09:50:14.090178+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:56:52.867292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090624+0000","last_change":"2023-10-16T09:50:11.949935+0000","last_active":"2023-10-16T09:50:14.090624+0000","last_peered":"2023-10-16T09:50:14.090624+0000","last_clean":"2023-10-16T09:50:14.090624+0000","last_became_active":"2023-10-16T09:50:11.949614+0000","last_became_peered":"2023-10-16T09:50:11.949614+0000","last_unstale":"2023-10-16T09:50:14.090624+0000","last_undegraded":"2023-10-16T09:50:14.090624+0000","last_fullsized":"2023-10-16T09:50:14.090624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:47:56.665456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090717+0000","last_change":"2023-10-16T09:50:12.588890+0000","last_active":"2023-10-16T09:50:14.090717+0000","last_peered":"2023-10-16T09:50:14.090717+0000","last_clean":"2023-10-16T09:50:14.090717+0000","last_became_active":"2023-10-16T09:50:12.588606+0000","last_became_peered":"2023-10-16T09:50:12.588606+0000","last_unstale":"2023-10-16T09:50:14.090717+0000","last_undegraded":"2023-10-16T09:50:14.090717+0000","last_fullsized":"2023-10-16T09:50:14.090717+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:15.603817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090349+0000","last_change":"2023-10-16T09:50:11.940747+0000","last_active":"2023-10-16T09:50:14.090349+0000","last_peered":"2023-10-16T09:50:14.090349+0000","last_clean":"2023-10-16T09:50:14.090349+0000","last_became_active":"2023-10-16T09:50:11.940443+0000","last_became_peered":"2023-10-16T09:50:11.940443+0000","last_unstale":"2023-10-16T09:50:14.090349+0000","last_undegraded":"2023-10-16T09:50:14.090349+0000","last_fullsized":"2023-10-16T09:50:14.090349+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:57.202359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089295+0000","last_change":"2023-10-16T09:50:12.600921+0000","last_active":"2023-10-16T09:50:14.089295+0000","last_peered":"2023-10-16T09:50:14.089295+0000","last_clean":"2023-10-16T09:50:14.089295+0000","last_became_active":"2023-10-16T09:50:12.600711+0000","last_became_peered":"2023-10-16T09:50:12.600711+0000","last_unstale":"2023-10-16T09:50:14.089295+0000","last_undegraded":"2023-10-16T09:50:14.089295+0000","last_fullsized":"2023-10-16T09:50:14.089295+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:27.920687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090046+0000","last_change":"2023-10-16T09:50:11.951018+0000","last_active":"2023-10-16T09:50:14.090046+0000","last_peered":"2023-10-16T09:50:14.090046+0000","last_clean":"2023-10-16T09:50:14.090046+0000","last_became_active":"2023-10-16T09:50:11.950683+0000","last_became_peered":"2023-10-16T09:50:11.950683+0000","last_unstale":"2023-10-16T09:50:14.090046+0000","last_undegraded":"2023-10-16T09:50:14.090046+0000","last_fullsized":"2023-10-16T09:50:14.090046+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:34.319629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089747+0000","last_change":"2023-10-16T09:50:12.601903+0000","last_active":"2023-10-16T09:50:14.089747+0000","last_peered":"2023-10-16T09:50:14.089747+0000","last_clean":"2023-10-16T09:50:14.089747+0000","last_became_active":"2023-10-16T09:50:12.601633+0000","last_became_peered":"2023-10-16T09:50:12.601633+0000","last_unstale":"2023-10-16T09:50:14.089747+0000","last_undegraded":"2023-10-16T09:50:14.089747+0000","last_fullsized":"2023-10-16T09:50:14.089747+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:01.222630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089396+0000","last_change":"2023-10-16T09:50:11.942305+0000","last_active":"2023-10-16T09:50:14.089396+0000","last_peered":"2023-10-16T09:50:14.089396+0000","last_clean":"2023-10-16T09:50:14.089396+0000","last_became_active":"2023-10-16T09:50:11.940994+0000","last_became_peered":"2023-10-16T09:50:11.940994+0000","last_unstale":"2023-10-16T09:50:14.089396+0000","last_undegraded":"2023-10-16T09:50:14.089396+0000","last_fullsized":"2023-10-16T09:50:14.089396+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:34.260749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.802604+0000","last_change":"2023-10-16T09:50:12.595577+0000","last_active":"2023-10-16T09:50:13.802604+0000","last_peered":"2023-10-16T09:50:13.802604+0000","last_clean":"2023-10-16T09:50:13.802604+0000","last_became_active":"2023-10-16T09:50:12.595320+0000","last_became_peered":"2023-10-16T09:50:12.595320+0000","last_unstale":"2023-10-16T09:50:13.802604+0000","last_undegraded":"2023-10-16T09:50:13.802604+0000","last_fullsized":"2023-10-16T09:50:13.802604+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:47.181025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910534+0000","last_change":"2023-10-16T09:50:11.946954+0000","last_active":"2023-10-16T09:50:13.910534+0000","last_peered":"2023-10-16T09:50:13.910534+0000","last_clean":"2023-10-16T09:50:13.910534+0000","last_became_active":"2023-10-16T09:50:11.946582+0000","last_became_peered":"2023-10-16T09:50:11.946582+0000","last_unstale":"2023-10-16T09:50:13.910534+0000","last_undegraded":"2023-10-16T09:50:13.910534+0000","last_fullsized":"2023-10-16T09:50:13.910534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:36.724821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089917+0000","last_change":"2023-10-16T09:50:12.590565+0000","last_active":"2023-10-16T09:50:14.089917+0000","last_peered":"2023-10-16T09:50:14.089917+0000","last_clean":"2023-10-16T09:50:14.089917+0000","last_became_active":"2023-10-16T09:50:12.589994+0000","last_became_peered":"2023-10-16T09:50:12.589994+0000","last_unstale":"2023-10-16T09:50:14.089917+0000","last_undegraded":"2023-10-16T09:50:14.089917+0000","last_fullsized":"2023-10-16T09:50:14.089917+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:23.391598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.802719+0000","last_change":"2023-10-16T09:50:11.945770+0000","last_active":"2023-10-16T09:50:13.802719+0000","last_peered":"2023-10-16T09:50:13.802719+0000","last_clean":"2023-10-16T09:50:13.802719+0000","last_became_active":"2023-10-16T09:50:11.945552+0000","last_became_peered":"2023-10-16T09:50:11.945552+0000","last_unstale":"2023-10-16T09:50:13.802719+0000","last_undegraded":"2023-10-16T09:50:13.802719+0000","last_fullsized":"2023-10-16T09:50:13.802719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:57.202359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.563882+0000","last_change":"2023-10-16T09:50:12.595849+0000","last_active":"2023-10-16T09:50:13.563882+0000","last_peered":"2023-10-16T09:50:13.563882+0000","last_clean":"2023-10-16T09:50:13.563882+0000","last_became_active":"2023-10-16T09:50:12.591954+0000","last_became_peered":"2023-10-16T09:50:12.591954+0000","last_unstale":"2023-10-16T09:50:13.563882+0000","last_undegraded":"2023-10-16T09:50:13.563882+0000","last_fullsized":"2023-10-16T09:50:13.563882+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:40:06.173040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.088636+0000","last_change":"2023-10-16T09:50:11.950355+0000","last_active":"2023-10-16T09:50:14.088636+0000","last_peered":"2023-10-16T09:50:14.088636+0000","last_clean":"2023-10-16T09:50:14.088636+0000","last_became_active":"2023-10-16T09:50:11.950091+0000","last_became_peered":"2023-10-16T09:50:11.950091+0000","last_unstale":"2023-10-16T09:50:14.088636+0000","last_undegraded":"2023-10-16T09:50:14.088636+0000","last_fullsized":"2023-10-16T09:50:14.088636+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:01:46.047914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.566082+0000","last_change":"2023-10-16T09:50:12.591751+0000","last_active":"2023-10-16T09:50:13.566082+0000","last_peered":"2023-10-16T09:50:13.566082+0000","last_clean":"2023-10-16T09:50:13.566082+0000","last_became_active":"2023-10-16T09:50:12.591379+0000","last_became_peered":"2023-10-16T09:50:12.591379+0000","last_unstale":"2023-10-16T09:50:13.566082+0000","last_undegraded":"2023-10-16T09:50:13.566082+0000","last_fullsized":"2023-10-16T09:50:13.566082+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:05:27.268406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.088262+0000","last_change":"2023-10-16T09:50:11.935845+0000","last_active":"2023-10-16T09:50:14.088262+0000","last_peered":"2023-10-16T09:50:14.088262+0000","last_clean":"2023-10-16T09:50:14.088262+0000","last_became_active":"2023-10-16T09:50:11.935559+0000","last_became_peered":"2023-10-16T09:50:11.935559+0000","last_unstale":"2023-10-16T09:50:14.088262+0000","last_undegraded":"2023-10-16T09:50:14.088262+0000","last_fullsized":"2023-10-16T09:50:14.088262+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:30.221634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910360+0000","last_change":"2023-10-16T09:50:12.594596+0000","last_active":"2023-10-16T09:50:13.910360+0000","last_peered":"2023-10-16T09:50:13.910360+0000","last_clean":"2023-10-16T09:50:13.910360+0000","last_became_active":"2023-10-16T09:50:12.592376+0000","last_became_peered":"2023-10-16T09:50:12.592376+0000","last_unstale":"2023-10-16T09:50:13.910360+0000","last_undegraded":"2023-10-16T09:50:13.910360+0000","last_fullsized":"2023-10-16T09:50:13.910360+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:34.134204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.803818+0000","last_change":"2023-10-16T09:50:11.953184+0000","last_active":"2023-10-16T09:50:13.803818+0000","last_peered":"2023-10-16T09:50:13.803818+0000","last_clean":"2023-10-16T09:50:13.803818+0000","last_became_active":"2023-10-16T09:50:11.952647+0000","last_became_peered":"2023-10-16T09:50:11.952647+0000","last_unstale":"2023-10-16T09:50:13.803818+0000","last_undegraded":"2023-10-16T09:50:13.803818+0000","last_fullsized":"2023-10-16T09:50:13.803818+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:55:01.098345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090074+0000","last_change":"2023-10-16T09:50:12.589925+0000","last_active":"2023-10-16T09:50:14.090074+0000","last_peered":"2023-10-16T09:50:14.090074+0000","last_clean":"2023-10-16T09:50:14.090074+0000","last_became_active":"2023-10-16T09:50:12.589688+0000","last_became_peered":"2023-10-16T09:50:12.589688+0000","last_unstale":"2023-10-16T09:50:14.090074+0000","last_undegraded":"2023-10-16T09:50:14.090074+0000","last_fullsized":"2023-10-16T09:50:14.090074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:39.821588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910613+0000","last_change":"2023-10-16T09:50:11.943119+0000","last_active":"2023-10-16T09:50:13.910613+0000","last_peered":"2023-10-16T09:50:13.910613+0000","last_clean":"2023-10-16T09:50:13.910613+0000","last_became_active":"2023-10-16T09:50:11.942800+0000","last_became_peered":"2023-10-16T09:50:11.942800+0000","last_unstale":"2023-10-16T09:50:13.910613+0000","last_undegraded":"2023-10-16T09:50:13.910613+0000","last_fullsized":"2023-10-16T09:50:13.910613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:36.404480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.802925+0000","last_change":"2023-10-16T09:50:12.589887+0000","last_active":"2023-10-16T09:50:13.802925+0000","last_peered":"2023-10-16T09:50:13.802925+0000","last_clean":"2023-10-16T09:50:13.802925+0000","last_became_active":"2023-10-16T09:50:12.589589+0000","last_became_peered":"2023-10-16T09:50:12.589589+0000","last_unstale":"2023-10-16T09:50:13.802925+0000","last_undegraded":"2023-10-16T09:50:13.802925+0000","last_fullsized":"2023-10-16T09:50:13.802925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:01:47.063096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910671+0000","last_change":"2023-10-16T09:50:11.940537+0000","last_active":"2023-10-16T09:50:13.910671+0000","last_peered":"2023-10-16T09:50:13.910671+0000","last_clean":"2023-10-16T09:50:13.910671+0000","last_became_active":"2023-10-16T09:50:11.940046+0000","last_became_peered":"2023-10-16T09:50:11.940046+0000","last_unstale":"2023-10-16T09:50:13.910671+0000","last_undegraded":"2023-10-16T09:50:13.910671+0000","last_fullsized":"2023-10-16T09:50:13.910671+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:55:26.024831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.562741+0000","last_change":"2023-10-16T09:50:12.583467+0000","last_active":"2023-10-16T09:50:13.562741+0000","last_peered":"2023-10-16T09:50:13.562741+0000","last_clean":"2023-10-16T09:50:13.562741+0000","last_became_active":"2023-10-16T09:50:12.583099+0000","last_became_peered":"2023-10-16T09:50:12.583099+0000","last_unstale":"2023-10-16T09:50:13.562741+0000","last_undegraded":"2023-10-16T09:50:13.562741+0000","last_fullsized":"2023-10-16T09:50:13.562741+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:37.740003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910763+0000","last_change":"2023-10-16T09:50:11.938384+0000","last_active":"2023-10-16T09:50:13.910763+0000","last_peered":"2023-10-16T09:50:13.910763+0000","last_clean":"2023-10-16T09:50:13.910763+0000","last_became_active":"2023-10-16T09:50:11.937795+0000","last_became_peered":"2023-10-16T09:50:11.937795+0000","last_unstale":"2023-10-16T09:50:13.910763+0000","last_undegraded":"2023-10-16T09:50:13.910763+0000","last_fullsized":"2023-10-16T09:50:13.910763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:57.202359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.561991+0000","last_change":"2023-10-16T09:50:12.582768+0000","last_active":"2023-10-16T09:50:13.561991+0000","last_peered":"2023-10-16T09:50:13.561991+0000","last_clean":"2023-10-16T09:50:13.561991+0000","last_became_active":"2023-10-16T09:50:12.582416+0000","last_became_peered":"2023-10-16T09:50:12.582416+0000","last_unstale":"2023-10-16T09:50:13.561991+0000","last_undegraded":"2023-10-16T09:50:13.561991+0000","last_fullsized":"2023-10-16T09:50:13.561991+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:33:32.255338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089664+0000","last_change":"2023-10-16T09:50:11.944903+0000","last_active":"2023-10-16T09:50:14.089664+0000","last_peered":"2023-10-16T09:50:14.089664+0000","last_clean":"2023-10-16T09:50:14.089664+0000","last_became_active":"2023-10-16T09:50:11.944617+0000","last_became_peered":"2023-10-16T09:50:11.944617+0000","last_unstale":"2023-10-16T09:50:14.089664+0000","last_undegraded":"2023-10-16T09:50:14.089664+0000","last_fullsized":"2023-10-16T09:50:14.089664+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:43.336545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.562042+0000","last_change":"2023-10-16T09:50:12.587254+0000","last_active":"2023-10-16T09:50:13.562042+0000","last_peered":"2023-10-16T09:50:13.562042+0000","last_clean":"2023-10-16T09:50:13.562042+0000","last_became_active":"2023-10-16T09:50:12.586970+0000","last_became_peered":"2023-10-16T09:50:12.586970+0000","last_unstale":"2023-10-16T09:50:13.562042+0000","last_undegraded":"2023-10-16T09:50:13.562042+0000","last_fullsized":"2023-10-16T09:50:13.562042+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:14:50.109642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.565405+0000","last_change":"2023-10-16T09:50:11.954696+0000","last_active":"2023-10-16T09:50:13.565405+0000","last_peered":"2023-10-16T09:50:13.565405+0000","last_clean":"2023-10-16T09:50:13.565405+0000","last_became_active":"2023-10-16T09:50:11.951148+0000","last_became_peered":"2023-10-16T09:50:11.951148+0000","last_unstale":"2023-10-16T09:50:13.565405+0000","last_undegraded":"2023-10-16T09:50:13.565405+0000","last_fullsized":"2023-10-16T09:50:13.565405+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:09:39.873155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.561833+0000","last_change":"2023-10-16T09:50:12.584513+0000","last_active":"2023-10-16T09:50:13.561833+0000","last_peered":"2023-10-16T09:50:13.561833+0000","last_clean":"2023-10-16T09:50:13.561833+0000","last_became_active":"2023-10-16T09:50:12.583987+0000","last_became_peered":"2023-10-16T09:50:12.583987+0000","last_unstale":"2023-10-16T09:50:13.561833+0000","last_undegraded":"2023-10-16T09:50:13.561833+0000","last_fullsized":"2023-10-16T09:50:13.561833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:59.998733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.909752+0000","last_change":"2023-10-16T09:50:11.937491+0000","last_active":"2023-10-16T09:50:13.909752+0000","last_peered":"2023-10-16T09:50:13.909752+0000","last_clean":"2023-10-16T09:50:13.909752+0000","last_became_active":"2023-10-16T09:50:11.936899+0000","last_became_peered":"2023-10-16T09:50:11.936899+0000","last_unstale":"2023-10-16T09:50:13.909752+0000","last_undegraded":"2023-10-16T09:50:13.909752+0000","last_fullsized":"2023-10-16T09:50:13.909752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:14:49.094460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.564182+0000","last_change":"2023-10-16T09:50:12.592848+0000","last_active":"2023-10-16T09:50:13.564182+0000","last_peered":"2023-10-16T09:50:13.564182+0000","last_clean":"2023-10-16T09:50:13.564182+0000","last_became_active":"2023-10-16T09:50:12.592438+0000","last_became_peered":"2023-10-16T09:50:12.592438+0000","last_unstale":"2023-10-16T09:50:13.564182+0000","last_undegraded":"2023-10-16T09:50:13.564182+0000","last_fullsized":"2023-10-16T09:50:13.564182+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:23.391598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089250+0000","last_change":"2023-10-16T09:50:11.952836+0000","last_active":"2023-10-16T09:50:14.089250+0000","last_peered":"2023-10-16T09:50:14.089250+0000","last_clean":"2023-10-16T09:50:14.089250+0000","last_became_active":"2023-10-16T09:50:11.952496+0000","last_became_peered":"2023-10-16T09:50:11.952496+0000","last_unstale":"2023-10-16T09:50:14.089250+0000","last_undegraded":"2023-10-16T09:50:14.089250+0000","last_fullsized":"2023-10-16T09:50:14.089250+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:33.119022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.562725+0000","last_change":"2023-10-16T09:50:12.593896+0000","last_active":"2023-10-16T09:50:13.562725+0000","last_peered":"2023-10-16T09:50:13.562725+0000","last_clean":"2023-10-16T09:50:13.562725+0000","last_became_active":"2023-10-16T09:50:12.592759+0000","last_became_peered":"2023-10-16T09:50:12.592759+0000","last_unstale":"2023-10-16T09:50:13.562725+0000","last_undegraded":"2023-10-16T09:50:13.562725+0000","last_fullsized":"2023-10-16T09:50:13.562725+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:21.372758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.803791+0000","last_change":"2023-10-16T09:50:11.954471+0000","last_active":"2023-10-16T09:50:13.803791+0000","last_peered":"2023-10-16T09:50:13.803791+0000","last_clean":"2023-10-16T09:50:13.803791+0000","last_became_active":"2023-10-16T09:50:11.954090+0000","last_became_peered":"2023-10-16T09:50:11.954090+0000","last_unstale":"2023-10-16T09:50:13.803791+0000","last_undegraded":"2023-10-16T09:50:13.803791+0000","last_fullsized":"2023-10-16T09:50:13.803791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:56:38.823066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090298+0000","last_change":"2023-10-16T09:50:12.594329+0000","last_active":"2023-10-16T09:50:14.090298+0000","last_peered":"2023-10-16T09:50:14.090298+0000","last_clean":"2023-10-16T09:50:14.090298+0000","last_became_active":"2023-10-16T09:50:12.594032+0000","last_became_peered":"2023-10-16T09:50:12.594032+0000","last_unstale":"2023-10-16T09:50:14.090298+0000","last_undegraded":"2023-10-16T09:50:14.090298+0000","last_fullsized":"2023-10-16T09:50:14.090298+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:23.391598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.129883+0000","last_change":"2023-10-16T09:50:11.944544+0000","last_active":"2023-10-16T09:50:14.129883+0000","last_peered":"2023-10-16T09:50:14.129883+0000","last_clean":"2023-10-16T09:50:14.129883+0000","last_became_active":"2023-10-16T09:50:11.944188+0000","last_became_peered":"2023-10-16T09:50:11.944188+0000","last_unstale":"2023-10-16T09:50:14.129883+0000","last_undegraded":"2023-10-16T09:50:14.129883+0000","last_fullsized":"2023-10-16T09:50:14.129883+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:05:26.253224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.089040+0000","last_change":"2023-10-16T09:50:12.580392+0000","last_active":"2023-10-16T09:50:14.089040+0000","last_peered":"2023-10-16T09:50:14.089040+0000","last_clean":"2023-10-16T09:50:14.089040+0000","last_became_active":"2023-10-16T09:50:12.579880+0000","last_became_peered":"2023-10-16T09:50:12.579880+0000","last_unstale":"2023-10-16T09:50:14.089040+0000","last_undegraded":"2023-10-16T09:50:14.089040+0000","last_fullsized":"2023-10-16T09:50:14.089040+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:31.236817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.803327+0000","last_change":"2023-10-16T09:50:11.939773+0000","last_active":"2023-10-16T09:50:13.803327+0000","last_peered":"2023-10-16T09:50:13.803327+0000","last_clean":"2023-10-16T09:50:13.803327+0000","last_became_active":"2023-10-16T09:50:11.939489+0000","last_became_peered":"2023-10-16T09:50:11.939489+0000","last_unstale":"2023-10-16T09:50:13.803327+0000","last_undegraded":"2023-10-16T09:50:13.803327+0000","last_fullsized":"2023-10-16T09:50:13.803327+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:50.558681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.565052+0000","last_change":"2023-10-16T09:50:12.584035+0000","last_active":"2023-10-16T09:50:13.565052+0000","last_peered":"2023-10-16T09:50:13.565052+0000","last_clean":"2023-10-16T09:50:13.565052+0000","last_became_active":"2023-10-16T09:50:12.583748+0000","last_became_peered":"2023-10-16T09:50:12.583748+0000","last_unstale":"2023-10-16T09:50:13.565052+0000","last_undegraded":"2023-10-16T09:50:13.565052+0000","last_fullsized":"2023-10-16T09:50:13.565052+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:37.419663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090011+0000","last_change":"2023-10-16T09:50:11.950606+0000","last_active":"2023-10-16T09:50:14.090011+0000","last_peered":"2023-10-16T09:50:14.090011+0000","last_clean":"2023-10-16T09:50:14.090011+0000","last_became_active":"2023-10-16T09:50:11.949747+0000","last_became_peered":"2023-10-16T09:50:11.949747+0000","last_unstale":"2023-10-16T09:50:14.090011+0000","last_undegraded":"2023-10-16T09:50:14.090011+0000","last_fullsized":"2023-10-16T09:50:14.090011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:36.404480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.090250+0000","last_change":"2023-10-16T09:50:12.583623+0000","last_active":"2023-10-16T09:50:14.090250+0000","last_peered":"2023-10-16T09:50:14.090250+0000","last_clean":"2023-10-16T09:50:14.090250+0000","last_became_active":"2023-10-16T09:50:12.581070+0000","last_became_peered":"2023-10-16T09:50:12.581070+0000","last_unstale":"2023-10-16T09:50:14.090250+0000","last_undegraded":"2023-10-16T09:50:14.090250+0000","last_fullsized":"2023-10-16T09:50:14.090250+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:56:39.838249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.802705+0000","last_change":"2023-10-16T09:50:11.947795+0000","last_active":"2023-10-16T09:50:13.802705+0000","last_peered":"2023-10-16T09:50:13.802705+0000","last_clean":"2023-10-16T09:50:13.802705+0000","last_became_active":"2023-10-16T09:50:11.947532+0000","last_became_peered":"2023-10-16T09:50:11.947532+0000","last_unstale":"2023-10-16T09:50:13.802705+0000","last_undegraded":"2023-10-16T09:50:13.802705+0000","last_fullsized":"2023-10-16T09:50:13.802705+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:58:31.879140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.088285+0000","last_change":"2023-10-16T09:50:12.582893+0000","last_active":"2023-10-16T09:50:14.088285+0000","last_peered":"2023-10-16T09:50:14.088285+0000","last_clean":"2023-10-16T09:50:14.088285+0000","last_became_active":"2023-10-16T09:50:12.581938+0000","last_became_peered":"2023-10-16T09:50:12.581938+0000","last_unstale":"2023-10-16T09:50:14.088285+0000","last_undegraded":"2023-10-16T09:50:14.088285+0000","last_fullsized":"2023-10-16T09:50:14.088285+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:39.821588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.910280+0000","last_change":"2023-10-16T09:50:11.944661+0000","last_active":"2023-10-16T09:50:13.910280+0000","last_peered":"2023-10-16T09:50:13.910280+0000","last_clean":"2023-10-16T09:50:13.910280+0000","last_became_active":"2023-10-16T09:50:11.944268+0000","last_became_peered":"2023-10-16T09:50:11.944268+0000","last_unstale":"2023-10-16T09:50:13.910280+0000","last_undegraded":"2023-10-16T09:50:13.910280+0000","last_fullsized":"2023-10-16T09:50:13.910280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:26.650824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.909722+0000","last_change":"2023-10-16T09:50:12.596045+0000","last_active":"2023-10-16T09:50:13.909722+0000","last_peered":"2023-10-16T09:50:13.909722+0000","last_clean":"2023-10-16T09:50:13.909722+0000","last_became_active":"2023-10-16T09:50:12.595584+0000","last_became_peered":"2023-10-16T09:50:12.595584+0000","last_unstale":"2023-10-16T09:50:13.909722+0000","last_undegraded":"2023-10-16T09:50:13.909722+0000","last_fullsized":"2023-10-16T09:50:13.909722+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:31.236817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:14.088146+0000","last_change":"2023-10-16T09:50:11.950425+0000","last_active":"2023-10-16T09:50:14.088146+0000","last_peered":"2023-10-16T09:50:14.088146+0000","last_clean":"2023-10-16T09:50:14.088146+0000","last_became_active":"2023-10-16T09:50:11.950120+0000","last_became_peered":"2023-10-16T09:50:11.950120+0000","last_unstale":"2023-10-16T09:50:14.088146+0000","last_undegraded":"2023-10-16T09:50:14.088146+0000","last_fullsized":"2023-10-16T09:50:14.088146+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:58:31.879140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.564504+0000","last_change":"2023-10-16T09:50:12.588657+0000","last_active":"2023-10-16T09:50:13.564504+0000","last_peered":"2023-10-16T09:50:13.564504+0000","last_clean":"2023-10-16T09:50:13.564504+0000","last_became_active":"2023-10-16T09:50:12.588359+0000","last_became_peered":"2023-10-16T09:50:12.588359+0000","last_unstale":"2023-10-16T09:50:13.564504+0000","last_undegraded":"2023-10-16T09:50:13.564504+0000","last_fullsized":"2023-10-16T09:50:13.564504+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:11.899635+0000","last_clean_scrub_stamp":"2023-10-16T09:50:11.899635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:39.157970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:50:13.565670+0000","last_change":"2023-10-16T09:50:11.955005+0000","last_active":"2023-10-16T09:50:13.565670+0000","last_peered":"2023-10-16T09:50:13.565670+0000","last_clean":"2023-10-16T09:50:13.565670+0000","last_became_active":"2023-10-16T09:50:11.954691+0000","last_became_peered":"2023-10-16T09:50:11.954691+0000","last_unstale":"2023-10-16T09:50:13.565670+0000","last_undegraded":"2023-10-16T09:50:13.565670+0000","last_fullsized":"2023-10-16T09:50:13.565670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:50:10.884453+0000","last_clean_scrub_stamp":"2023-10-16T09:50:10.884453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:06:41.408779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":176,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:50:12.580351+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T09:50:12.580351+0000","last_peered":"2023-10-16T09:50:12.580351+0000","last_clean":"2023-10-16T09:50:12.580351+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T09:50:12.580351+0000","last_undegraded":"2023-10-16T09:50:12.580351+0000","last_fullsized":"2023-10-16T09:50:12.580351+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":983040,"data_stored":742041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:49:54 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":3.694,"5min":3.694,"15min":3.694},"last":1.214}]},{"osd":1,"last update":"Mon Oct 16 09:49:54 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.155}]},{"osd":2,"last update":"Mon Oct 16 09:49:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.7759999999999998,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":1.375}]},{"osd":3,"last update":"Mon Oct 16 09:49:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":1.087}]},{"osd":4,"last update":"Mon Oct 16 09:49:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.6659999999999999}]},{"osd":5,"last update":"Mon Oct 16 09:49:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"min":{"1min":1.02,"5min":1.02,"15min":1.02},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.575}]},{"osd":6,"last update":"Mon Oct 16 09:49:54 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"min":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"max":{"1min":2.6259999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":1.625},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"min":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.5670000000000002,"5min":2.5670000000000002,"15min":2.5670000000000002},"last":1.4830000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":393216,"data_stored":152199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:49:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.04,"5min":2.04,"15min":2.04},"last":0.65400000000000003},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.86799999999999999}]},{"osd":1,"last update":"Mon Oct 16 09:49:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.145},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.0960000000000001}]},{"osd":2,"last update":"Mon Oct 16 09:49:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.413},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.371}]},{"osd":3,"last update":"Mon Oct 16 09:49:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.0249999999999999}]},{"osd":4,"last update":"Mon Oct 16 09:49:55 2023","interfaces":[{"interface":"back","average":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":3.4249999999999998,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":1.321}]},{"osd":5,"last update":"Mon Oct 16 09:49:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":1.512},{"interface":"front","average":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.4930000000000001}]},{"osd":7,"last update":"Mon Oct 16 09:49:55 2023","interfaces":[{"interface":"back","average":{"1min":1.746,"5min":1.746,"15min":1.746},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.8679999999999999,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":1.605},{"interface":"front","average":{"1min":1.742,"5min":1.742,"15min":1.742},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":3.5419999999999998,"5min":3.5419999999999998,"15min":3.5419999999999998},"last":1.5740000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":393216,"data_stored":152255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.325},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":1.1479999999999999}]},{"osd":1,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.4569999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.026}]},{"osd":2,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.468},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.4379999999999999}]},{"osd":3,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.25},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.361}]},{"osd":4,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.659,"5min":1.659,"15min":1.659},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.4019999999999999}]},{"osd":6,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.556,"15min":1.556},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.3580000000000001,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.208}]},{"osd":7,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":1.798,"15min":1.798},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.831,"5min":1.831,"15min":1.831},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.931,"5min":2.931,"15min":2.931},"last":1.2889999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":393216,"data_stored":151687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":4.2640000000000002,"5min":4.2640000000000002,"15min":4.2640000000000002},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":4.391,"5min":4.391,"15min":4.391},"last":0.72399999999999998}]},{"osd":1,"last update":"Mon Oct 16 09:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.2410000000000001}]},{"osd":2,"last update":"Mon Oct 16 09:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.1160000000000001,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.0840000000000001}]},{"osd":3,"last update":"Mon Oct 16 09:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":3.262,"5min":3.262,"15min":3.262},"last":1.042},{"interface":"front","average":{"1min":1.649,"5min":1.649,"15min":1.649},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":3.3050000000000002,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":1.4990000000000001}]},{"osd":5,"last update":"Mon Oct 16 09:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.9780000000000002,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":3.669,"5min":3.669,"15min":3.669},"last":1.548}]},{"osd":6,"last update":"Mon Oct 16 09:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.8090000000000002,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":1.714},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":3.2280000000000002,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.284}]},{"osd":7,"last update":"Mon Oct 16 09:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":3.5150000000000001,"5min":3.5150000000000001,"15min":3.5150000000000001},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.913,"5min":1.913,"15min":1.913},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":3.9849999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.6180000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":146917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.67200000000000004}]},{"osd":1,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.093},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.94399999999999995}]},{"osd":2,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":1.2410000000000001}]},{"osd":4,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.393},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.1930000000000001}]},{"osd":5,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.133},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.157}]},{"osd":6,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.462,"15min":1.462},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.2609999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.581,"15min":1.581},"min":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.659,"5min":1.659,"15min":1.659},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"max":{"1min":2.379,"5min":2.379,"15min":2.379},"last":1.351}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":389120,"data_stored":151653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.96199999999999997}]},{"osd":1,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.6059999999999999,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.89300000000000002}]},{"osd":3,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.7269999999999999,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.86699999999999999}]},{"osd":4,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.1950000000000001}]},{"osd":5,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.419,"15min":1.419},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":2.7770000000000001,"5min":2.7770000000000001,"15min":2.7770000000000001},"last":1.244},{"interface":"front","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.952,"5min":2.952,"15min":2.952},"last":1.1319999999999999}]},{"osd":6,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.8180000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.5}]},{"osd":7,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":3.746,"5min":3.746,"15min":3.746},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.2629999999999999,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":1.4830000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":153217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.82599999999999996}]},{"osd":2,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.78900000000000003}]},{"osd":3,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.8109999999999999,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":0.68600000000000005}]},{"osd":4,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.516,"5min":1.516,"15min":1.516},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.851,"5min":2.851,"15min":2.851},"last":1.391}]},{"osd":5,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.7120000000000002,"5min":2.7120000000000002,"15min":2.7120000000000002},"last":1.119},{"interface":"front","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":1.268}]},{"osd":6,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.522,"15min":1.522},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.75,"5min":2.75,"15min":2.75},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.669,"5min":2.669,"15min":2.669},"last":1.3520000000000001}]},{"osd":7,"last update":"Mon Oct 16 09:49:57 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.8799999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":3.1400000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":1.4790000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":735651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 09:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Mon Oct 16 09:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.48899999999999999}]},{"osd":3,"last update":"Mon Oct 16 09:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":0.51200000000000001}]},{"osd":4,"last update":"Mon Oct 16 09:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.206}]},{"osd":5,"last update":"Mon Oct 16 09:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.8159999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.9009999999999998,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.996}]},{"osd":6,"last update":"Mon Oct 16 09:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.7160000000000002,"5min":2.7160000000000002,"15min":2.7160000000000002},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":1.0169999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":1.155}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:50:17.889 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T09:50:17.890 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T09:50:18.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.149+0000 7ffa9d075640 1 Processor -- start 2023-10-16T09:50:18.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.149+0000 7ffa9d075640 1 -- start start 2023-10-16T09:50:18.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.150+0000 7ffa9d075640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ffa9815cb50 0x7ffa98161180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:18.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.150+0000 7ffa9d075640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa980b4c80 0x7ffa980d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:18.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.150+0000 7ffa9d075640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa981616c0 con 0x7ffa980b1690 2023-10-16T09:50:18.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.150+0000 7ffa9d075640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa980cdbf0 con 0x7ffa9815cb50 2023-10-16T09:50:18.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.150+0000 7ffa9d075640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa980cde90 con 0x7ffa980b4c80 2023-10-16T09:50:18.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.151+0000 7ffa97fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ffa9815cb50 0x7ffa98161180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:18.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.151+0000 7ffa97fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ffa9815cb50 0x7ffa98161180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37610/0 (socket says 172.21.15.84:37610) 2023-10-16T09:50:18.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.151+0000 7ffa97fff640 1 -- 172.21.15.84:0/3466769189 learned_addr learned my addr 172.21.15.84:0/3466769189 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:18.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.151+0000 7ffa8f7fe640 1 --1- 172.21.15.84:0/3466769189 >> v1:172.21.15.84:6789/0 conn(0x7ffa980b1690 0x7ffa980b4570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33570/0 (socket says 172.21.15.84:33570) 2023-10-16T09:50:18.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.151+0000 7ffa97fff640 1 -- 172.21.15.84:0/3466769189 >> v2:172.21.15.157:3300/0 conn(0x7ffa980b4c80 msgr2=0x7ffa980d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:18.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.151+0000 7ffa97fff640 1 --2- 172.21.15.84:0/3466769189 >> v2:172.21.15.157:3300/0 conn(0x7ffa980b4c80 0x7ffa980d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:18.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.151+0000 7ffa97fff640 1 -- 172.21.15.84:0/3466769189 >> v1:172.21.15.84:6789/0 conn(0x7ffa980b1690 legacy=0x7ffa980b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:18.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.151+0000 7ffa97fff640 1 -- 172.21.15.84:0/3466769189 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa980ce170 con 0x7ffa9815cb50 2023-10-16T09:50:18.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.152+0000 7ffa97fff640 1 --2- 172.21.15.84:0/3466769189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa9815cb50 0x7ffa98161180 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ffa980b1ae0 tx=0x7ffa8802b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d1970538df8c47 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:18.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.153+0000 7ffa977fe640 1 -- 172.21.15.84:0/3466769189 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa880317f0 con 0x7ffa9815cb50 2023-10-16T09:50:18.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.153+0000 7ffa977fe640 1 -- 172.21.15.84:0/3466769189 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa88031950 con 0x7ffa9815cb50 2023-10-16T09:50:18.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.153+0000 7ffa977fe640 1 -- 172.21.15.84:0/3466769189 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88031ce0 con 0x7ffa9815cb50 2023-10-16T09:50:18.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.153+0000 7ffa9d075640 1 -- 172.21.15.84:0/3466769189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa9815cb50 msgr2=0x7ffa98161180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:18.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.153+0000 7ffa9d075640 1 --2- 172.21.15.84:0/3466769189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa9815cb50 0x7ffa98161180 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ffa980b1ae0 tx=0x7ffa8802b890 comp rx=0 tx=0).stop 2023-10-16T09:50:18.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.153+0000 7ffa9d075640 1 -- 172.21.15.84:0/3466769189 shutdown_connections 2023-10-16T09:50:18.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.153+0000 7ffa9d075640 1 --2- 172.21.15.84:0/3466769189 >> v2:172.21.15.157:3300/0 conn(0x7ffa980b4c80 0x7ffa980d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:18.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.153+0000 7ffa9d075640 1 --2- 172.21.15.84:0/3466769189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa9815cb50 0x7ffa98161180 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:18.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.153+0000 7ffa9d075640 1 -- 172.21.15.84:0/3466769189 >> 172.21.15.84:0/3466769189 conn(0x7ffa980cf570 msgr2=0x7ffa980cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:18.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.153+0000 7ffa9d075640 1 -- 172.21.15.84:0/3466769189 shutdown_connections 2023-10-16T09:50:18.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.153+0000 7ffa9d075640 1 -- 172.21.15.84:0/3466769189 wait complete. 2023-10-16T09:50:18.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.155+0000 7ffa9d075640 1 Processor -- start 2023-10-16T09:50:18.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.155+0000 7ffa9d075640 1 -- start start 2023-10-16T09:50:18.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.156+0000 7ffa9d075640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa980b1690 0x7ffa980b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:18.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.156+0000 7ffa9d075640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa980b4c80 0x7ffa981f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:18.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.156+0000 7ffa9d075640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9815cb50 0x7ffa981f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:18.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.156+0000 7ffa9d075640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa980b10b0 con 0x7ffa980b4c80 2023-10-16T09:50:18.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.156+0000 7ffa9d075640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa980c7f50 con 0x7ffa980b1690 2023-10-16T09:50:18.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.156+0000 7ffa9d075640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa981f3950 con 0x7ffa9815cb50 2023-10-16T09:50:18.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.156+0000 7ffa8f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa980b4c80 0x7ffa981f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:18.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.156+0000 7ffa8f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa980b4c80 0x7ffa981f5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37618/0 (socket says 172.21.15.84:37618) 2023-10-16T09:50:18.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.156+0000 7ffa8f7fe640 1 -- 172.21.15.84:0/1639266780 learned_addr learned my addr 172.21.15.84:0/1639266780 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:18.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.157+0000 7ffa97fff640 1 --2- 172.21.15.84:0/1639266780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa980b1690 0x7ffa980b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:18.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.157+0000 7ffa9c874640 1 --2- 172.21.15.84:0/1639266780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9815cb50 0x7ffa981f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:18.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.157+0000 7ffa8f7fe640 1 -- 172.21.15.84:0/1639266780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9815cb50 msgr2=0x7ffa981f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:18.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.157+0000 7ffa8f7fe640 1 --2- 172.21.15.84:0/1639266780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9815cb50 0x7ffa981f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:18.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.157+0000 7ffa8f7fe640 1 -- 172.21.15.84:0/1639266780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa980b1690 msgr2=0x7ffa980b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:18.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.157+0000 7ffa8f7fe640 1 --2- 172.21.15.84:0/1639266780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa980b1690 0x7ffa980b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:18.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.157+0000 7ffa8f7fe640 1 -- 172.21.15.84:0/1639266780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa880094d0 con 0x7ffa980b4c80 2023-10-16T09:50:18.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.157+0000 7ffa97fff640 1 --2- 172.21.15.84:0/1639266780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa980b1690 0x7ffa980b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:18.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.157+0000 7ffa9c874640 1 --2- 172.21.15.84:0/1639266780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9815cb50 0x7ffa981f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:18.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.157+0000 7ffa8f7fe640 1 --2- 172.21.15.84:0/1639266780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa980b4c80 0x7ffa981f5d80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000c800 tx=0x7ffa8000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:18.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.158+0000 7ffa957fa640 1 -- 172.21.15.84:0/1639266780 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8000a640 con 0x7ffa980b4c80 2023-10-16T09:50:18.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.158+0000 7ffa957fa640 1 -- 172.21.15.84:0/1639266780 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8000a7a0 con 0x7ffa980b4c80 2023-10-16T09:50:18.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.158+0000 7ffa957fa640 1 -- 172.21.15.84:0/1639266780 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8000ab50 con 0x7ffa980b4c80 2023-10-16T09:50:18.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.158+0000 7ffa9d075640 1 -- 172.21.15.84:0/1639266780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa981f3c30 con 0x7ffa980b4c80 2023-10-16T09:50:18.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.158+0000 7ffa9d075640 1 -- 172.21.15.84:0/1639266780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa981f41f0 con 0x7ffa980b4c80 2023-10-16T09:50:18.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.159+0000 7ffa9d075640 1 -- 172.21.15.84:0/1639266780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa981f4410 con 0x7ffa980b4c80 2023-10-16T09:50:18.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.160+0000 7ffa957fa640 1 -- 172.21.15.84:0/1639266780 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ffa8000acb0 con 0x7ffa980b4c80 2023-10-16T09:50:18.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.161+0000 7ffa957fa640 1 --2- 172.21.15.84:0/1639266780 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffa7806ae70 0x7ffa7806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:18.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.161+0000 7ffa957fa640 1 -- 172.21.15.84:0/1639266780 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa80088e20 con 0x7ffa980b4c80 2023-10-16T09:50:18.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.161+0000 7ffa97fff640 1 --2- 172.21.15.84:0/1639266780 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffa7806ae70 0x7ffa7806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:18.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.162+0000 7ffa97fff640 1 --2- 172.21.15.84:0/1639266780 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffa7806ae70 0x7ffa7806d330 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffa88036000 tx=0x7ffa8803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:18.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.165+0000 7ffa957fa640 1 -- 172.21.15.84:0/1639266780 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffa80057970 con 0x7ffa980b4c80 2023-10-16T09:50:18.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.425+0000 7ffa9d075640 1 -- 172.21.15.84:0/1639266780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ffa9804e9b0 con 0x7ffa980b4c80 2023-10-16T09:50:18.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.429+0000 7ffa957fa640 1 -- 172.21.15.84:0/1639266780 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7ffa80017020 con 0x7ffa980b4c80 2023-10-16T09:50:18.432 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2023-10-16T09:50:18.432 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ== 2023-10-16T09:50:18.432 INFO:teuthology.orchestra.run.smithi084.stdout: caps mds = "allow" 2023-10-16T09:50:18.432 INFO:teuthology.orchestra.run.smithi084.stdout: caps mon = "allow r" 2023-10-16T09:50:18.432 INFO:teuthology.orchestra.run.smithi084.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T09:50:18.433 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-16T09:50:18.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.433+0000 7ffa9d075640 1 -- 172.21.15.84:0/1639266780 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffa7806ae70 msgr2=0x7ffa7806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:18.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.433+0000 7ffa9d075640 1 --2- 172.21.15.84:0/1639266780 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffa7806ae70 0x7ffa7806d330 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffa88036000 tx=0x7ffa8803d040 comp rx=0 tx=0).stop 2023-10-16T09:50:18.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.433+0000 7ffa9d075640 1 -- 172.21.15.84:0/1639266780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa980b4c80 msgr2=0x7ffa981f5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:18.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.433+0000 7ffa9d075640 1 --2- 172.21.15.84:0/1639266780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa980b4c80 0x7ffa981f5d80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000c800 tx=0x7ffa8000ccc0 comp rx=0 tx=0).stop 2023-10-16T09:50:18.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.433+0000 7ffa9d075640 1 -- 172.21.15.84:0/1639266780 shutdown_connections 2023-10-16T09:50:18.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.433+0000 7ffa9d075640 1 --2- 172.21.15.84:0/1639266780 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7ffa7806ae70 0x7ffa7806d330 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:18.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.433+0000 7ffa9d075640 1 --2- 172.21.15.84:0/1639266780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9815cb50 0x7ffa981f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:18.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.433+0000 7ffa9d075640 1 --2- 172.21.15.84:0/1639266780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa980b4c80 0x7ffa981f5d80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:18.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.433+0000 7ffa9d075640 1 --2- 172.21.15.84:0/1639266780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa980b1690 0x7ffa980b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:18.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.433+0000 7ffa9d075640 1 -- 172.21.15.84:0/1639266780 >> 172.21.15.84:0/1639266780 conn(0x7ffa980cf570 msgr2=0x7ffa980d2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:18.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.433+0000 7ffa9d075640 1 -- 172.21.15.84:0/1639266780 shutdown_connections 2023-10-16T09:50:18.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.433+0000 7ffa9d075640 1 -- 172.21.15.84:0/1639266780 wait complete. 2023-10-16T09:50:18.452 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:50:18.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.666+0000 7f468f2d1640 1 Processor -- start 2023-10-16T09:50:18.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.666+0000 7f468f2d1640 1 -- start start 2023-10-16T09:50:18.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.667+0000 7f468f2d1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f468815c9f0 0x7f46880c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:18.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.667+0000 7f468f2d1640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4688160a90 con 0x7f46880c9230 2023-10-16T09:50:18.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.667+0000 7f468f2d1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46880cd900 con 0x7f468815dde0 2023-10-16T09:50:18.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.667+0000 7f468f2d1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46880cdec0 con 0x7f468815c9f0 2023-10-16T09:50:18.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.668+0000 7f468e2cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f468815c9f0 0x7f46880c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:18.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.668+0000 7f468e2cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f468815c9f0 0x7f46880c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37622/0 (socket says 172.21.15.84:37622) 2023-10-16T09:50:18.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.668+0000 7f468e2cf640 1 -- 172.21.15.84:0/1237692493 learned_addr learned my addr 172.21.15.84:0/1237692493 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:18.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.668+0000 7f468dace640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f46880c9230 0x7f46880709e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33572/0 (socket says 172.21.15.84:33572) 2023-10-16T09:50:18.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.668+0000 7f468e2cf640 1 -- 172.21.15.84:0/1237692493 >> v1:172.21.15.157:6789/0 conn(0x7f468815dde0 legacy=0x7f46880ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:18.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.668+0000 7f468e2cf640 1 -- 172.21.15.84:0/1237692493 >> v1:172.21.15.84:6789/0 conn(0x7f46880c9230 legacy=0x7f46880709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:18.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.668+0000 7f468e2cf640 1 -- 172.21.15.84:0/1237692493 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46880cdbe0 con 0x7f468815c9f0 2023-10-16T09:50:18.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.669+0000 7f468e2cf640 1 --2- 172.21.15.84:0/1237692493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468815c9f0 0x7f46880c8cf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4684009870 tx=0x7f468402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59205536cef528eb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:18.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.669+0000 7f468d2cd640 1 -- 172.21.15.84:0/1237692493 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684030860 con 0x7f468815c9f0 2023-10-16T09:50:18.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.669+0000 7f468d2cd640 1 -- 172.21.15.84:0/1237692493 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46840309c0 con 0x7f468815c9f0 2023-10-16T09:50:18.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.670+0000 7f468d2cd640 1 -- 172.21.15.84:0/1237692493 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684030d50 con 0x7f468815c9f0 2023-10-16T09:50:18.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.670+0000 7f468f2d1640 1 -- 172.21.15.84:0/1237692493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468815c9f0 msgr2=0x7f46880c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:18.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.670+0000 7f468f2d1640 1 --2- 172.21.15.84:0/1237692493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468815c9f0 0x7f46880c8cf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4684009870 tx=0x7f468402b9a0 comp rx=0 tx=0).stop 2023-10-16T09:50:18.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.670+0000 7f468f2d1640 1 -- 172.21.15.84:0/1237692493 shutdown_connections 2023-10-16T09:50:18.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.670+0000 7f468f2d1640 1 --2- 172.21.15.84:0/1237692493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468815c9f0 0x7f46880c8cf0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:18.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.670+0000 7f468f2d1640 1 -- 172.21.15.84:0/1237692493 >> 172.21.15.84:0/1237692493 conn(0x7f46880ce6a0 msgr2=0x7f46880ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:18.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.670+0000 7f468f2d1640 1 -- 172.21.15.84:0/1237692493 shutdown_connections 2023-10-16T09:50:18.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.670+0000 7f468f2d1640 1 -- 172.21.15.84:0/1237692493 wait complete. 2023-10-16T09:50:18.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.672+0000 7f468f2d1640 1 Processor -- start 2023-10-16T09:50:18.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.672+0000 7f468f2d1640 1 -- start start 2023-10-16T09:50:18.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.673+0000 7f468f2d1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46880c9230 0x7f4688153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:18.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.673+0000 7f468f2d1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f468815c9f0 0x7f4688157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:18.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.673+0000 7f468f2d1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468815dde0 0x7f4688157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:18.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.673+0000 7f468f2d1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46880c2af0 con 0x7f468815dde0 2023-10-16T09:50:18.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.673+0000 7f468f2d1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46881f2930 con 0x7f468815c9f0 2023-10-16T09:50:18.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.673+0000 7f468f2d1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46880c24a0 con 0x7f46880c9230 2023-10-16T09:50:18.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.673+0000 7f468ead0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468815dde0 0x7f4688157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:18.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.674+0000 7f468ead0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468815dde0 0x7f4688157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37634/0 (socket says 172.21.15.84:37634) 2023-10-16T09:50:18.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.674+0000 7f468ead0640 1 -- 172.21.15.84:0/2948848088 learned_addr learned my addr 172.21.15.84:0/2948848088 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:18.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.674+0000 7f468e2cf640 1 --2- 172.21.15.84:0/2948848088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46880c9230 0x7f4688153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:18.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.674+0000 7f468ead0640 1 -- 172.21.15.84:0/2948848088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46880c9230 msgr2=0x7f4688153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:18.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.674+0000 7f468ead0640 1 --2- 172.21.15.84:0/2948848088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46880c9230 0x7f4688153720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:18.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.674+0000 7f468ead0640 1 -- 172.21.15.84:0/2948848088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f468815c9f0 msgr2=0x7f4688157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:18.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.674+0000 7f468ead0640 1 --2- 172.21.15.84:0/2948848088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f468815c9f0 0x7f4688157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:18.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.674+0000 7f468ead0640 1 -- 172.21.15.84:0/2948848088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46840094d0 con 0x7f468815dde0 2023-10-16T09:50:18.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.674+0000 7f468e2cf640 1 --2- 172.21.15.84:0/2948848088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46880c9230 0x7f4688153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:18.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.674+0000 7f468ead0640 1 --2- 172.21.15.84:0/2948848088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468815dde0 0x7f4688157f90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f468000b960 tx=0x7f468000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:18.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.675+0000 7f467effd640 1 -- 172.21.15.84:0/2948848088 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468000cb20 con 0x7f468815dde0 2023-10-16T09:50:18.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.675+0000 7f467effd640 1 -- 172.21.15.84:0/2948848088 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468000cc80 con 0x7f468815dde0 2023-10-16T09:50:18.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.675+0000 7f467effd640 1 -- 172.21.15.84:0/2948848088 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4680014440 con 0x7f468815dde0 2023-10-16T09:50:18.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.675+0000 7f468f2d1640 1 -- 172.21.15.84:0/2948848088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46881541e0 con 0x7f468815dde0 2023-10-16T09:50:18.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.675+0000 7f468f2d1640 1 -- 172.21.15.84:0/2948848088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4688153c60 con 0x7f468815dde0 2023-10-16T09:50:18.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.676+0000 7f468f2d1640 1 -- 172.21.15.84:0/2948848088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46880c2c30 con 0x7f468815dde0 2023-10-16T09:50:18.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.677+0000 7f467effd640 1 -- 172.21.15.84:0/2948848088 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f468001d3b0 con 0x7f468815dde0 2023-10-16T09:50:18.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.678+0000 7f467effd640 1 --2- 172.21.15.84:0/2948848088 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f466c06ae70 0x7f466c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:18.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.678+0000 7f467effd640 1 -- 172.21.15.84:0/2948848088 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46800895f0 con 0x7f468815dde0 2023-10-16T09:50:18.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.678+0000 7f468e2cf640 1 --2- 172.21.15.84:0/2948848088 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f466c06ae70 0x7f466c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:18.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.679+0000 7f468e2cf640 1 --2- 172.21.15.84:0/2948848088 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f466c06ae70 0x7f466c06d330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4684036000 tx=0x7f468403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:18.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.682+0000 7f467effd640 1 -- 172.21.15.84:0/2948848088 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f46880c2c30 con 0x7f468815dde0 2023-10-16T09:50:18.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.911+0000 7f468f2d1640 1 -- 172.21.15.84:0/2948848088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46880c2c30 con 0x7f468815dde0 2023-10-16T09:50:18.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.912+0000 7f467effd640 1 -- 172.21.15.84:0/2948848088 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f46880c2c30 con 0x7f468815dde0 2023-10-16T09:50:18.914 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T09:50:18.914 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3662158203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3662158203},{"type":"v1","addr":"172.21.15.157:6835","nonce":3662158203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5091,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2502120537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2502120537},{"type":"v1","addr":"172.21.15.84:6836","nonce":2502120537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2715651376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2715651376},{"type":"v1","addr":"172.21.15.84:6837","nonce":2715651376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:50:12.380180+0000","modified":"2023-10-16T09:50:15.438597+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6833/62459899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":62459899},{"type":"v1","addr":"172.21.15.157:6833","nonce":62459899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T09:50:18.915 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 17 2023-10-16T09:50:18.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.914+0000 7f468f2d1640 1 -- 172.21.15.84:0/2948848088 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f466c06ae70 msgr2=0x7f466c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:18.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.914+0000 7f468f2d1640 1 --2- 172.21.15.84:0/2948848088 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f466c06ae70 0x7f466c06d330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4684036000 tx=0x7f468403b040 comp rx=0 tx=0).stop 2023-10-16T09:50:18.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.914+0000 7f468f2d1640 1 -- 172.21.15.84:0/2948848088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468815dde0 msgr2=0x7f4688157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:18.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.914+0000 7f468f2d1640 1 --2- 172.21.15.84:0/2948848088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468815dde0 0x7f4688157f90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f468000b960 tx=0x7f468000be20 comp rx=0 tx=0).stop 2023-10-16T09:50:18.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.915+0000 7f468f2d1640 1 -- 172.21.15.84:0/2948848088 shutdown_connections 2023-10-16T09:50:18.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.915+0000 7f468f2d1640 1 --2- 172.21.15.84:0/2948848088 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f466c06ae70 0x7f466c06d330 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:18.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.915+0000 7f468f2d1640 1 --2- 172.21.15.84:0/2948848088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f468815dde0 0x7f4688157f90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:18.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.915+0000 7f468f2d1640 1 --2- 172.21.15.84:0/2948848088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f468815c9f0 0x7f4688157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:18.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.915+0000 7f468f2d1640 1 --2- 172.21.15.84:0/2948848088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46880c9230 0x7f4688153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:18.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.915+0000 7f468f2d1640 1 -- 172.21.15.84:0/2948848088 >> 172.21.15.84:0/2948848088 conn(0x7f46880ce6a0 msgr2=0x7f46880ae100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:18.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.915+0000 7f468f2d1640 1 -- 172.21.15.84:0/2948848088 shutdown_connections 2023-10-16T09:50:18.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:18.915+0000 7f468f2d1640 1 -- 172.21.15.84:0/2948848088 wait complete. 2023-10-16T09:50:19.337 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T09:50:12.380180+0000', 'modified': '2023-10-16T09:50:15.438597+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5100}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5100': {'gid': 5100, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.157:6833/62459899', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6832', 'nonce': 62459899}, {'type': 'v1', 'addr': '172.21.15.157:6833', 'nonce': 62459899}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T09:50:19.338 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T09:50:19.339 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-16T09:50:19.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.559+0000 7f317b141640 1 Processor -- start 2023-10-16T09:50:19.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.559+0000 7f317b141640 1 -- start start 2023-10-16T09:50:19.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.560+0000 7f317b141640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f317415ad70 0x7f3174069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.560+0000 7f317b141640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31740caac0 0x7f317406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.561+0000 7f317b141640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31740cb030 con 0x7f31740c9ca0 2023-10-16T09:50:19.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.561+0000 7f317b141640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31740c91c0 con 0x7f317415ad70 2023-10-16T09:50:19.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.561+0000 7f317b141640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31740c94a0 con 0x7f31740caac0 2023-10-16T09:50:19.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.561+0000 7f3178edd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f317415ad70 0x7f3174069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.561+0000 7f3178edd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f317415ad70 0x7f3174069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37646/0 (socket says 172.21.15.84:37646) 2023-10-16T09:50:19.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.561+0000 7f3178edd640 1 -- 172.21.15.84:0/1027848456 learned_addr learned my addr 172.21.15.84:0/1027848456 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:19.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.561+0000 7f3173fff640 1 --1- 172.21.15.84:0/1027848456 >> v1:172.21.15.84:6789/0 conn(0x7f31740c9ca0 0x7f31740ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33584/0 (socket says 172.21.15.84:33584) 2023-10-16T09:50:19.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.561+0000 7f31796de640 1 --2- 172.21.15.84:0/1027848456 >> v2:172.21.15.157:3300/0 conn(0x7f31740caac0 0x7f317406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.561+0000 7f3178edd640 1 -- 172.21.15.84:0/1027848456 >> v2:172.21.15.157:3300/0 conn(0x7f31740caac0 msgr2=0x7f317406a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:19.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.561+0000 7f3178edd640 1 --2- 172.21.15.84:0/1027848456 >> v2:172.21.15.157:3300/0 conn(0x7f31740caac0 0x7f317406a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.561+0000 7f3178edd640 1 -- 172.21.15.84:0/1027848456 >> v1:172.21.15.84:6789/0 conn(0x7f31740c9ca0 legacy=0x7f31740ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:19.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.561+0000 7f31737fe640 1 -- 172.21.15.84:0/1027848456 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1604839221 0 0) 0x7f31740cb030 con 0x7f31740c9ca0 2023-10-16T09:50:19.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.561+0000 7f3178edd640 1 -- 172.21.15.84:0/1027848456 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31740c9780 con 0x7f317415ad70 2023-10-16T09:50:19.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.563+0000 7f3178edd640 1 --2- 172.21.15.84:0/1027848456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f317415ad70 0x7f3174069da0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f317406d090 tx=0x7f316802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e338151b6346df server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:19.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.565+0000 7f31737fe640 1 -- 172.21.15.84:0/1027848456 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168030860 con 0x7f317415ad70 2023-10-16T09:50:19.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.566+0000 7f31737fe640 1 -- 172.21.15.84:0/1027848456 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31680309c0 con 0x7f317415ad70 2023-10-16T09:50:19.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.566+0000 7f31737fe640 1 -- 172.21.15.84:0/1027848456 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168030d50 con 0x7f317415ad70 2023-10-16T09:50:19.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.566+0000 7f317b141640 1 -- 172.21.15.84:0/1027848456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f317415ad70 msgr2=0x7f3174069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:19.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.566+0000 7f317b141640 1 --2- 172.21.15.84:0/1027848456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f317415ad70 0x7f3174069da0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f317406d090 tx=0x7f316802b9a0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.566+0000 7f317b141640 1 -- 172.21.15.84:0/1027848456 shutdown_connections 2023-10-16T09:50:19.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.566+0000 7f317b141640 1 --2- 172.21.15.84:0/1027848456 >> v2:172.21.15.157:3300/0 conn(0x7f31740caac0 0x7f317406a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.566+0000 7f317b141640 1 --2- 172.21.15.84:0/1027848456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f317415ad70 0x7f3174069da0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.566+0000 7f317b141640 1 -- 172.21.15.84:0/1027848456 >> 172.21.15.84:0/1027848456 conn(0x7f31740ce9b0 msgr2=0x7f31740cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:19.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.566+0000 7f317b141640 1 -- 172.21.15.84:0/1027848456 shutdown_connections 2023-10-16T09:50:19.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.566+0000 7f317b141640 1 -- 172.21.15.84:0/1027848456 wait complete. 2023-10-16T09:50:19.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.568+0000 7f317b141640 1 Processor -- start 2023-10-16T09:50:19.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.568+0000 7f317b141640 1 -- start start 2023-10-16T09:50:19.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.569+0000 7f317b141640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31740c9ca0 0x7f31740b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.569+0000 7f317b141640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31740caac0 0x7f31740c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.569+0000 7f317b141640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f317415ad70 0x7f31740c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.569+0000 7f317b141640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31740c6530 con 0x7f31740caac0 2023-10-16T09:50:19.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.569+0000 7f317b141640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317415c7d0 con 0x7f31740c9ca0 2023-10-16T09:50:19.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.569+0000 7f317b141640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31740c67a0 con 0x7f317415ad70 2023-10-16T09:50:19.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.570+0000 7f3173fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31740caac0 0x7f31740c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.570+0000 7f3173fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31740caac0 0x7f31740c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37648/0 (socket says 172.21.15.84:37648) 2023-10-16T09:50:19.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.570+0000 7f3173fff640 1 -- 172.21.15.84:0/2641223427 learned_addr learned my addr 172.21.15.84:0/2641223427 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:19.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.570+0000 7f3173fff640 1 -- 172.21.15.84:0/2641223427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f317415ad70 msgr2=0x7f31740c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:19.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.570+0000 7f3173fff640 1 --2- 172.21.15.84:0/2641223427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f317415ad70 0x7f31740c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.570+0000 7f3173fff640 1 -- 172.21.15.84:0/2641223427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31740c9ca0 msgr2=0x7f31740b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:19.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.570+0000 7f3173fff640 1 --2- 172.21.15.84:0/2641223427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31740c9ca0 0x7f31740b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.570+0000 7f3173fff640 1 -- 172.21.15.84:0/2641223427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31680094d0 con 0x7f31740caac0 2023-10-16T09:50:19.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.570+0000 7f3173fff640 1 --2- 172.21.15.84:0/2641223427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31740caac0 0x7f31740c6fa0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f316000c4d0 tx=0x7f316000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:19.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.571+0000 7f31717fa640 1 -- 172.21.15.84:0/2641223427 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31600074e0 con 0x7f31740caac0 2023-10-16T09:50:19.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.571+0000 7f31717fa640 1 -- 172.21.15.84:0/2641223427 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3160007660 con 0x7f31740caac0 2023-10-16T09:50:19.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.571+0000 7f31717fa640 1 -- 172.21.15.84:0/2641223427 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160004a20 con 0x7f31740caac0 2023-10-16T09:50:19.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.571+0000 7f317b141640 1 -- 172.21.15.84:0/2641223427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31740c6a80 con 0x7f31740caac0 2023-10-16T09:50:19.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.571+0000 7f317b141640 1 -- 172.21.15.84:0/2641223427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31740afe90 con 0x7f31740caac0 2023-10-16T09:50:19.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.573+0000 7f31717fa640 1 -- 172.21.15.84:0/2641223427 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3160004b80 con 0x7f31740caac0 2023-10-16T09:50:19.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.574+0000 7f31717fa640 1 --2- 172.21.15.84:0/2641223427 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f315806ae70 0x7f315806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.574+0000 7f31717fa640 1 -- 172.21.15.84:0/2641223427 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3160088850 con 0x7f31740caac0 2023-10-16T09:50:19.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.575+0000 7f3178edd640 1 --2- 172.21.15.84:0/2641223427 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f315806ae70 0x7f315806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.575+0000 7f317ddb1740 1 Processor -- start 2023-10-16T09:50:19.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.575+0000 7f317ddb1740 1 -- start start 2023-10-16T09:50:19.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.575+0000 7f3178edd640 1 --2- 172.21.15.84:0/2641223427 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f315806ae70 0x7f315806d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3168036000 tx=0x7f3168030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:19.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.576+0000 7f317ddb1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d33f80 0x560164d06e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.576+0000 7f317ddb1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d34360 0x560164d36d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.576+0000 7f317ddb1740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d073c0 0x560164d37250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.576+0000 7f317ddb1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x560164c277a0 con 0x560164d34360 2023-10-16T09:50:19.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.576+0000 7f317ddb1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560164c79740 con 0x560164d33f80 2023-10-16T09:50:19.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.576+0000 7f317ddb1740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x560164c76e30 con 0x560164d073c0 2023-10-16T09:50:19.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.576+0000 7f3173fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d34360 0x560164d36d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.576+0000 7f3173fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d34360 0x560164d36d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37664/0 (socket says 172.21.15.84:37664) 2023-10-16T09:50:19.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.576+0000 7f3173fff640 1 -- 172.21.15.84:0/4217252488 learned_addr learned my addr 172.21.15.84:0/4217252488 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:19.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.577+0000 7f3173fff640 1 -- 172.21.15.84:0/4217252488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d073c0 msgr2=0x560164d37250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:19.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.577+0000 7f3173fff640 1 --2- 172.21.15.84:0/4217252488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d073c0 0x560164d37250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.577+0000 7f3173fff640 1 -- 172.21.15.84:0/4217252488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d33f80 msgr2=0x560164d06e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:19.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.577+0000 7f3173fff640 1 --2- 172.21.15.84:0/4217252488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d33f80 0x560164d06e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.577+0000 7f3173fff640 1 -- 172.21.15.84:0/4217252488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560164c36910 con 0x560164d34360 2023-10-16T09:50:19.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.577+0000 7f3173fff640 1 --2- 172.21.15.84:0/4217252488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d34360 0x560164d36d10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3160057220 tx=0x7f3160057ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57de37339c7cb01d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:19.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.577+0000 7f315e7fc640 1 -- 172.21.15.84:0/4217252488 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316005f820 con 0x560164d34360 2023-10-16T09:50:19.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.577+0000 7f315e7fc640 1 -- 172.21.15.84:0/4217252488 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3160088040 con 0x560164d34360 2023-10-16T09:50:19.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.577+0000 7f315e7fc640 1 -- 172.21.15.84:0/4217252488 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160053c30 con 0x560164d34360 2023-10-16T09:50:19.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.578+0000 7f317ddb1740 1 -- 172.21.15.84:0/4217252488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d34360 msgr2=0x560164d36d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:19.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.578+0000 7f317ddb1740 1 --2- 172.21.15.84:0/4217252488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d34360 0x560164d36d10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3160057220 tx=0x7f3160057ce0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.578+0000 7f317ddb1740 1 -- 172.21.15.84:0/4217252488 shutdown_connections 2023-10-16T09:50:19.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.578+0000 7f317ddb1740 1 --2- 172.21.15.84:0/4217252488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d073c0 0x560164d37250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.578+0000 7f317ddb1740 1 --2- 172.21.15.84:0/4217252488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d34360 0x560164d36d10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.578+0000 7f317ddb1740 1 --2- 172.21.15.84:0/4217252488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d33f80 0x560164d06e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.578+0000 7f317ddb1740 1 -- 172.21.15.84:0/4217252488 >> 172.21.15.84:0/4217252488 conn(0x560164d09190 msgr2=0x560164d51d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:19.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.578+0000 7f317ddb1740 1 -- 172.21.15.84:0/4217252488 shutdown_connections 2023-10-16T09:50:19.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.578+0000 7f317ddb1740 1 -- 172.21.15.84:0/4217252488 wait complete. 2023-10-16T09:50:19.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.579+0000 7f317ddb1740 1 -- start start 2023-10-16T09:50:19.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.579+0000 7f317ddb1740 1 Processor -- start 2023-10-16T09:50:19.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.580+0000 7f317ddb1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d33f80 0x560164d39780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.580+0000 7f317ddb1740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d34360 0x560164d4b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.580+0000 7f317ddb1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d4c340 0x560164d38220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.580+0000 7f317ddb1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x560164c72720 con 0x560164d4c340 2023-10-16T09:50:19.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.580+0000 7f317ddb1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560164c10970 con 0x560164d33f80 2023-10-16T09:50:19.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.580+0000 7f317ddb1740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x560164c10610 con 0x560164d34360 2023-10-16T09:50:19.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.580+0000 7f317ddb1740 10 client.0 _ll_register_callbacks cb 0x560164c329d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T09:50:19.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.580+0000 7f31796de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d4c340 0x560164d38220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.580+0000 7f31796de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d4c340 0x560164d38220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37666/0 (socket says 172.21.15.84:37666) 2023-10-16T09:50:19.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.580+0000 7f31796de640 1 -- 172.21.15.84:0/3519651981 learned_addr learned my addr 172.21.15.84:0/3519651981 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:19.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.581+0000 7f31796de640 1 -- 172.21.15.84:0/3519651981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d34360 msgr2=0x560164d4b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:19.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.581+0000 7f3178edd640 1 --2- 172.21.15.84:0/3519651981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d33f80 0x560164d39780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.581+0000 7f31796de640 1 --2- 172.21.15.84:0/3519651981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d34360 0x560164d4b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.581+0000 7f31796de640 1 -- 172.21.15.84:0/3519651981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d33f80 msgr2=0x560164d39780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:19.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.581+0000 7f31796de640 1 --2- 172.21.15.84:0/3519651981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d33f80 0x560164d39780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.581+0000 7f31796de640 1 -- 172.21.15.84:0/3519651981 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31740afe90 con 0x560164d4c340 2023-10-16T09:50:19.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.581+0000 7f31796de640 1 --2- 172.21.15.84:0/3519651981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d4c340 0x560164d38220 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f316c00c9b0 tx=0x7f316c00ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99949dd7f1bd9184 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:19.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.581+0000 7f31537fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T09:50:19.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.581+0000 7f31537fe640 1 -- 172.21.15.84:0/3519651981 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c00ac20 con 0x560164d4c340 2023-10-16T09:50:19.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.581+0000 7f31537fe640 1 -- 172.21.15.84:0/3519651981 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f316c012070 con 0x560164d4c340 2023-10-16T09:50:19.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.581+0000 7f317ddb1740 1 -- 172.21.15.84:0/3519651981 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560164c99160 con 0x560164d4c340 2023-10-16T09:50:19.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.581+0000 7f31537fe640 1 -- 172.21.15.84:0/3519651981 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f316c00ad80 con 0x560164d4c340 2023-10-16T09:50:19.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.581+0000 7f317ddb1740 10 client.5280 fetch_fsmap learned FSMap version 17 2023-10-16T09:50:19.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.581+0000 7f317ddb1740 1 -- 172.21.15.84:0/3519651981 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x560164c70da0 con 0x560164d4c340 2023-10-16T09:50:19.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.582+0000 7f31537fe640 1 -- 172.21.15.84:0/3519651981 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560164c99160 con 0x560164d4c340 2023-10-16T09:50:19.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.582+0000 7f31537fe640 1 -- 172.21.15.84:0/3519651981 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f316c016420 con 0x560164d4c340 2023-10-16T09:50:19.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.582+0000 7f317ddb1740 10 client.5280 fetch_fsmap finished waiting for FSMap version 17 2023-10-16T09:50:19.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.582+0000 7f317ddb1740 10 client.5280 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T09:50:19.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.582+0000 7f317ddb1740 20 client.5280 populate_metadata read hostname 'smithi084' 2023-10-16T09:50:19.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.582+0000 7f317ddb1740 1 --2- 172.21.15.84:0/3519651981 >> [v2:172.21.15.157:6832/62459899,v1:172.21.15.157:6833/62459899] conn(0x560164d6b330 0x560164d6f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.582+0000 7f317ddb1740 4 client.5280 mds_command: new command op to 5100 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T09:50:19.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.582+0000 7f317ddb1740 1 -- 172.21.15.84:0/3519651981 --> [v2:172.21.15.157:6832/62459899,v1:172.21.15.157:6833/62459899] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560164c44cc0 con 0x560164d6b330 2023-10-16T09:50:19.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.583+0000 7f3178edd640 1 --2- 172.21.15.84:0/3519651981 >> [v2:172.21.15.157:6832/62459899,v1:172.21.15.157:6833/62459899] conn(0x560164d6b330 0x560164d6f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.584+0000 7f3178edd640 1 --2- 172.21.15.84:0/3519651981 >> [v2:172.21.15.157:6832/62459899,v1:172.21.15.157:6833/62459899] conn(0x560164d6b330 0x560164d6f0d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8b4be6a81b2e8a6 server_cookie=9dd8408bc98cb9ec in_seq=0 out_seq=0 2023-10-16T09:50:19.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.584+0000 7f31537fe640 10 client.5280 ms_handle_connect on v2:172.21.15.157:6832/62459899 2023-10-16T09:50:19.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.585+0000 7f31537fe640 1 -- 172.21.15.84:0/3519651981 <== mds.0 v2:172.21.15.157:6832/62459899 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560164c44cc0 con 0x560164d6b330 2023-10-16T09:50:19.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.585+0000 7f31537fe640 10 client.5280 handle_command_reply: tid=0 2023-10-16T09:50:19.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.586+0000 7f317ddb1740 1 client.5280 shutdown 2023-10-16T09:50:19.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.586+0000 7f317ddb1740 1 -- 172.21.15.84:0/3519651981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d4c340 msgr2=0x560164d38220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:19.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.586+0000 7f317ddb1740 1 --2- 172.21.15.84:0/3519651981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d4c340 0x560164d38220 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f316c00c9b0 tx=0x7f316c00ce70 comp rx=0 tx=0).stop 2023-10-16T09:50:19.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.586+0000 7f317ddb1740 1 -- 172.21.15.84:0/3519651981 shutdown_connections 2023-10-16T09:50:19.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.586+0000 7f317ddb1740 1 --2- 172.21.15.84:0/3519651981 >> [v2:172.21.15.157:6832/62459899,v1:172.21.15.157:6833/62459899] conn(0x560164d6b330 0x560164d6f0d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.586+0000 7f317ddb1740 1 --2- 172.21.15.84:0/3519651981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d4c340 0x560164d38220 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.586+0000 7f317ddb1740 1 --2- 172.21.15.84:0/3519651981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d34360 0x560164d4b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.586+0000 7f317ddb1740 1 --2- 172.21.15.84:0/3519651981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d33f80 0x560164d39780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.586+0000 7f31537fe640 0 client.5280 ms_handle_reset on v2:172.21.15.157:6832/62459899 2023-10-16T09:50:19.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.586+0000 7f317ddb1740 1 -- 172.21.15.84:0/3519651981 >> 172.21.15.84:0/3519651981 conn(0x560164d073c0 msgr2=0x560164d4bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:19.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.586+0000 7f317ddb1740 1 -- 172.21.15.84:0/3519651981 shutdown_connections 2023-10-16T09:50:19.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.586+0000 7f317ddb1740 1 -- 172.21.15.84:0/3519651981 wait complete. 2023-10-16T09:50:19.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.586+0000 7f317ddb1740 20 client.5280 trim_cache size 0 max 16384 2023-10-16T09:50:19.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.602+0000 7f317ddb1740 1 Processor -- start 2023-10-16T09:50:19.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.603+0000 7f317ddb1740 1 -- start start 2023-10-16T09:50:19.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.603+0000 7f317ddb1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d33f80 0x560164d27d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.603+0000 7f317ddb1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d34360 0x560164d39420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.603+0000 7f317ddb1740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d4c340 0x560164d6a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.603+0000 7f317ddb1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601649d6f20 con 0x560164d33f80 2023-10-16T09:50:19.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.603+0000 7f317ddb1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560164d37280 con 0x560164d34360 2023-10-16T09:50:19.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.603+0000 7f317ddb1740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x560164d0a6b0 con 0x560164d4c340 2023-10-16T09:50:19.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.604+0000 7f3178edd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d33f80 0x560164d27d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.604+0000 7f3178edd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d33f80 0x560164d27d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37672/0 (socket says 172.21.15.84:37672) 2023-10-16T09:50:19.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.604+0000 7f3178edd640 1 -- 172.21.15.84:0/2208057316 learned_addr learned my addr 172.21.15.84:0/2208057316 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:19.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.604+0000 7f3173fff640 1 --2- 172.21.15.84:0/2208057316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d34360 0x560164d39420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.604+0000 7f31796de640 1 --2- 172.21.15.84:0/2208057316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d4c340 0x560164d6a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.604+0000 7f3178edd640 1 -- 172.21.15.84:0/2208057316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d4c340 msgr2=0x560164d6a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:19.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.604+0000 7f3178edd640 1 --2- 172.21.15.84:0/2208057316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d4c340 0x560164d6a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.604+0000 7f3178edd640 1 -- 172.21.15.84:0/2208057316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d34360 msgr2=0x560164d39420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:19.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.605+0000 7f3178edd640 1 --2- 172.21.15.84:0/2208057316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d34360 0x560164d39420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.605+0000 7f3178edd640 1 -- 172.21.15.84:0/2208057316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560164c0fb80 con 0x560164d33f80 2023-10-16T09:50:19.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.605+0000 7f3178edd640 1 --2- 172.21.15.84:0/2208057316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d33f80 0x560164d27d80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f316802bfb0 tx=0x7f3168031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bef0346c1c660f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:19.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.605+0000 7f315effd640 1 -- 172.21.15.84:0/2208057316 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316803e070 con 0x560164d33f80 2023-10-16T09:50:19.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.605+0000 7f315effd640 1 -- 172.21.15.84:0/2208057316 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316804a040 con 0x560164d33f80 2023-10-16T09:50:19.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.605+0000 7f315effd640 1 -- 172.21.15.84:0/2208057316 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168030d80 con 0x560164d33f80 2023-10-16T09:50:19.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.606+0000 7f317ddb1740 1 -- 172.21.15.84:0/2208057316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d33f80 msgr2=0x560164d27d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:19.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.606+0000 7f317ddb1740 1 --2- 172.21.15.84:0/2208057316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d33f80 0x560164d27d80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f316802bfb0 tx=0x7f3168031000 comp rx=0 tx=0).stop 2023-10-16T09:50:19.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.606+0000 7f317ddb1740 1 -- 172.21.15.84:0/2208057316 shutdown_connections 2023-10-16T09:50:19.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.606+0000 7f317ddb1740 1 --2- 172.21.15.84:0/2208057316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d4c340 0x560164d6a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.606+0000 7f317ddb1740 1 --2- 172.21.15.84:0/2208057316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d34360 0x560164d39420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.606+0000 7f317ddb1740 1 --2- 172.21.15.84:0/2208057316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d33f80 0x560164d27d80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.606+0000 7f317ddb1740 1 -- 172.21.15.84:0/2208057316 >> 172.21.15.84:0/2208057316 conn(0x560164d073c0 msgr2=0x560164d0df40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:19.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.606+0000 7f317ddb1740 1 -- 172.21.15.84:0/2208057316 shutdown_connections 2023-10-16T09:50:19.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.606+0000 7f317ddb1740 1 -- 172.21.15.84:0/2208057316 wait complete. 2023-10-16T09:50:19.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.607+0000 7f317ddb1740 1 -- start start 2023-10-16T09:50:19.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.607+0000 7f317ddb1740 1 Processor -- start 2023-10-16T09:50:19.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.608+0000 7f317ddb1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d33f80 0x560164db45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.608+0000 7f317ddb1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d34360 0x560164d6a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.608+0000 7f317ddb1740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d4c340 0x560164d6a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.608+0000 7f317ddb1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x560164d73be0 con 0x560164d34360 2023-10-16T09:50:19.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.608+0000 7f317ddb1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560164d51800 con 0x560164d33f80 2023-10-16T09:50:19.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.608+0000 7f317ddb1740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x560164d529a0 con 0x560164d4c340 2023-10-16T09:50:19.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.608+0000 7f317ddb1740 10 client.0 _ll_register_callbacks cb 0x7f316c007620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T09:50:19.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.608+0000 7f3173fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d34360 0x560164d6a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.608+0000 7f3173fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d34360 0x560164d6a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37674/0 (socket says 172.21.15.84:37674) 2023-10-16T09:50:19.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.608+0000 7f3173fff640 1 -- 172.21.15.84:0/3037539032 learned_addr learned my addr 172.21.15.84:0/3037539032 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:19.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.609+0000 7f31796de640 1 --2- 172.21.15.84:0/3037539032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d4c340 0x560164d6a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.609+0000 7f3178edd640 1 --2- 172.21.15.84:0/3037539032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d33f80 0x560164db45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.609+0000 7f3173fff640 1 -- 172.21.15.84:0/3037539032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d4c340 msgr2=0x560164d6a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:19.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.609+0000 7f3173fff640 1 --2- 172.21.15.84:0/3037539032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d4c340 0x560164d6a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.609+0000 7f3173fff640 1 -- 172.21.15.84:0/3037539032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d33f80 msgr2=0x560164db45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:19.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.609+0000 7f3173fff640 1 --2- 172.21.15.84:0/3037539032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d33f80 0x560164db45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.609+0000 7f3173fff640 1 -- 172.21.15.84:0/3037539032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31740c9780 con 0x560164d34360 2023-10-16T09:50:19.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.609+0000 7f31796de640 1 --2- 172.21.15.84:0/3037539032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d4c340 0x560164d6a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:19.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.609+0000 7f3178edd640 1 --2- 172.21.15.84:0/3037539032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d33f80 0x560164db45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:19.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.609+0000 7f3173fff640 1 --2- 172.21.15.84:0/3037539032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d34360 0x560164d6a480 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3160057dd0 tx=0x7f3160000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91d01739a23f514 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:19.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.609+0000 7f31537fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T09:50:19.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.609+0000 7f31537fe640 1 -- 172.21.15.84:0/3037539032 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160085040 con 0x560164d34360 2023-10-16T09:50:19.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.609+0000 7f31537fe640 1 -- 172.21.15.84:0/3037539032 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f316005f590 con 0x560164d34360 2023-10-16T09:50:19.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.609+0000 7f317ddb1740 1 -- 172.21.15.84:0/3037539032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3160088040 con 0x560164d34360 2023-10-16T09:50:19.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.609+0000 7f31537fe640 1 -- 172.21.15.84:0/3037539032 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f316005f8a0 con 0x560164d34360 2023-10-16T09:50:19.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.609+0000 7f317ddb1740 10 client.5286 fetch_fsmap learned FSMap version 17 2023-10-16T09:50:19.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.609+0000 7f317ddb1740 1 -- 172.21.15.84:0/3037539032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x560164c35170 con 0x560164d34360 2023-10-16T09:50:19.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.610+0000 7f31537fe640 1 -- 172.21.15.84:0/3037539032 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3160088040 con 0x560164d34360 2023-10-16T09:50:19.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.610+0000 7f31537fe640 1 -- 172.21.15.84:0/3037539032 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f316005b150 con 0x560164d34360 2023-10-16T09:50:19.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.610+0000 7f317ddb1740 10 client.5286 fetch_fsmap finished waiting for FSMap version 17 2023-10-16T09:50:19.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.610+0000 7f317ddb1740 10 client.5286 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T09:50:19.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.610+0000 7f317ddb1740 20 client.5286 populate_metadata read hostname 'smithi084' 2023-10-16T09:50:19.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.610+0000 7f317ddb1740 1 --2- 172.21.15.84:0/3037539032 >> [v2:172.21.15.157:6832/62459899,v1:172.21.15.157:6833/62459899] conn(0x560164d556e0 0x560164d55ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.610+0000 7f317ddb1740 4 client.5286 mds_command: new command op to 5100 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T09:50:19.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.610+0000 7f317ddb1740 1 -- 172.21.15.84:0/3037539032 --> [v2:172.21.15.157:6832/62459899,v1:172.21.15.157:6833/62459899] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560164c6b700 con 0x560164d556e0 2023-10-16T09:50:19.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.611+0000 7f3178edd640 1 --2- 172.21.15.84:0/3037539032 >> [v2:172.21.15.157:6832/62459899,v1:172.21.15.157:6833/62459899] conn(0x560164d556e0 0x560164d55ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.611+0000 7f3178edd640 1 --2- 172.21.15.84:0/3037539032 >> [v2:172.21.15.157:6832/62459899,v1:172.21.15.157:6833/62459899] conn(0x560164d556e0 0x560164d55ac0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=777190dba3f6365c server_cookie=8588545e5c96d1af in_seq=0 out_seq=0 2023-10-16T09:50:19.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.611+0000 7f31537fe640 10 client.5286 ms_handle_connect on v2:172.21.15.157:6832/62459899 2023-10-16T09:50:19.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.612+0000 7f31537fe640 1 -- 172.21.15.84:0/3037539032 <== mds.0 v2:172.21.15.157:6832/62459899 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x560164c6b700 con 0x560164d556e0 2023-10-16T09:50:19.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.612+0000 7f31537fe640 10 client.5286 handle_command_reply: tid=0 2023-10-16T09:50:19.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.612+0000 7f317ddb1740 1 client.5286 shutdown 2023-10-16T09:50:19.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.612+0000 7f317ddb1740 1 -- 172.21.15.84:0/3037539032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d34360 msgr2=0x560164d6a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:19.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.612+0000 7f317ddb1740 1 --2- 172.21.15.84:0/3037539032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d34360 0x560164d6a480 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3160057dd0 tx=0x7f3160000ed0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.612+0000 7f317ddb1740 1 -- 172.21.15.84:0/3037539032 shutdown_connections 2023-10-16T09:50:19.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.612+0000 7f317ddb1740 1 --2- 172.21.15.84:0/3037539032 >> [v2:172.21.15.157:6832/62459899,v1:172.21.15.157:6833/62459899] conn(0x560164d556e0 0x560164d55ac0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.613+0000 7f317ddb1740 1 --2- 172.21.15.84:0/3037539032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560164d4c340 0x560164d6a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.613+0000 7f317ddb1740 1 --2- 172.21.15.84:0/3037539032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560164d34360 0x560164d6a480 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.613+0000 7f31537fe640 0 client.5286 ms_handle_reset on v2:172.21.15.157:6832/62459899 2023-10-16T09:50:19.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.613+0000 7f317ddb1740 1 --2- 172.21.15.84:0/3037539032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560164d33f80 0x560164db45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.613+0000 7f317ddb1740 1 -- 172.21.15.84:0/3037539032 >> 172.21.15.84:0/3037539032 conn(0x560164d073c0 msgr2=0x560164d0aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:19.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.613+0000 7f317ddb1740 1 -- 172.21.15.84:0/3037539032 shutdown_connections 2023-10-16T09:50:19.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.613+0000 7f317ddb1740 1 -- 172.21.15.84:0/3037539032 wait complete. 2023-10-16T09:50:19.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.613+0000 7f317ddb1740 20 client.5286 trim_cache size 0 max 16384 2023-10-16T09:50:19.626 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T09:50:19.626 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "63fb4d68-f881-4694-b880-d10369105c0e", 2023-10-16T09:50:19.627 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-16T09:50:19.627 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 5100, 2023-10-16T09:50:19.627 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-16T09:50:19.627 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-16T09:50:19.627 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-16T09:50:19.627 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 6.6949373410000002, 2023-10-16T09:50:19.628 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 17, 2023-10-16T09:50:19.628 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 34, 2023-10-16T09:50:19.628 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 34, 2023-10-16T09:50:19.628 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 15.490745309999999 2023-10-16T09:50:19.628 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T09:50:19.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.613+0000 7f317b141640 1 -- 172.21.15.84:0/2641223427 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f315806ae70 msgr2=0x7f315806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:19.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.613+0000 7f317b141640 1 --2- 172.21.15.84:0/2641223427 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f315806ae70 0x7f315806d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3168036000 tx=0x7f3168030000 comp rx=0 tx=0).stop 2023-10-16T09:50:19.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.613+0000 7f317b141640 1 -- 172.21.15.84:0/2641223427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31740caac0 msgr2=0x7f31740c6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:19.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.613+0000 7f317b141640 1 --2- 172.21.15.84:0/2641223427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31740caac0 0x7f31740c6fa0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f316000c4d0 tx=0x7f316000c990 comp rx=0 tx=0).stop 2023-10-16T09:50:19.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.613+0000 7f317b141640 1 -- 172.21.15.84:0/2641223427 shutdown_connections 2023-10-16T09:50:19.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.613+0000 7f317b141640 1 --2- 172.21.15.84:0/2641223427 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f315806ae70 0x7f315806d330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.613+0000 7f317b141640 1 --2- 172.21.15.84:0/2641223427 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f317415ad70 0x7f31740c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.613+0000 7f317b141640 1 --2- 172.21.15.84:0/2641223427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31740caac0 0x7f31740c6fa0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.613+0000 7f317b141640 1 --2- 172.21.15.84:0/2641223427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31740c9ca0 0x7f31740b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.613+0000 7f317b141640 1 -- 172.21.15.84:0/2641223427 >> 172.21.15.84:0/2641223427 conn(0x7f31740ce9b0 msgr2=0x7f31740cc540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:19.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.614+0000 7f317b141640 1 -- 172.21.15.84:0/2641223427 shutdown_connections 2023-10-16T09:50:19.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.614+0000 7f317b141640 1 -- 172.21.15.84:0/2641223427 wait complete. 2023-10-16T09:50:19.634 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T09:50:19.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.844+0000 7f9a9a97c640 1 Processor -- start 2023-10-16T09:50:19.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.845+0000 7f9a9a97c640 1 -- start start 2023-10-16T09:50:19.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.846+0000 7f9a9a97c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415ae80 0x7f9a9415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.846+0000 7f9a9a97c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a9415ed60 con 0x7f9a940ce390 2023-10-16T09:50:19.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.846+0000 7f9a9a97c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a940707e0 con 0x7f9a9415aaa0 2023-10-16T09:50:19.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.846+0000 7f9a9a97c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9406b810 con 0x7f9a9415ae80 2023-10-16T09:50:19.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.846+0000 7f9a98f19640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9a940ce390 0x7f9a94070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33600/0 (socket says 172.21.15.84:33600) 2023-10-16T09:50:19.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.846+0000 7f9a98f19640 1 -- 172.21.15.84:0/3089152204 learned_addr learned my addr 172.21.15.84:0/3089152204 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:19.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.847+0000 7f9a937fe640 1 --2- 172.21.15.84:0/3089152204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415ae80 0x7f9a9415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.847+0000 7f9a92ffd640 1 -- 172.21.15.84:0/3089152204 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768510990 0 0) 0x7f9a9415ed60 con 0x7f9a940ce390 2023-10-16T09:50:19.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.847+0000 7f9a92ffd640 1 -- 172.21.15.84:0/3089152204 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a80003520 con 0x7f9a940ce390 2023-10-16T09:50:19.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.847+0000 7f9a92ffd640 1 -- 172.21.15.84:0/3089152204 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2558565611 0 0) 0x7f9a80003520 con 0x7f9a940ce390 2023-10-16T09:50:19.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.847+0000 7f9a92ffd640 1 -- 172.21.15.84:0/3089152204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415ae80 msgr2=0x7f9a9415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:19.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.847+0000 7f9a92ffd640 1 --2- 172.21.15.84:0/3089152204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415ae80 0x7f9a9415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.847+0000 7f9a92ffd640 1 -- 172.21.15.84:0/3089152204 >> v1:172.21.15.157:6789/0 conn(0x7f9a9415aaa0 legacy=0x7f9a940cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:19.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.847+0000 7f9a92ffd640 1 -- 172.21.15.84:0/3089152204 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9415ef90 con 0x7f9a940ce390 2023-10-16T09:50:19.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.847+0000 7f9a92ffd640 1 -- 172.21.15.84:0/3089152204 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f9a8c001670 con 0x7f9a940ce390 2023-10-16T09:50:19.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.847+0000 7f9a92ffd640 1 -- 172.21.15.84:0/3089152204 >> v1:172.21.15.84:6789/0 conn(0x7f9a940ce390 legacy=0x7f9a94070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:19.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.847+0000 7f9a92ffd640 1 --2- 172.21.15.84:0/3089152204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415ae80 0x7f9a800039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.847+0000 7f9a92ffd640 1 --2- 172.21.15.84:0/3089152204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a80004020 0x7f9a800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.847+0000 7f9a92ffd640 1 --2- 172.21.15.84:0/3089152204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a80004d00 0x7f9a80005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.847+0000 7f9a92ffd640 1 -- 172.21.15.84:0/3089152204 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9406b810 con 0x7f9a80004020 2023-10-16T09:50:19.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.847+0000 7f9a92ffd640 1 -- 172.21.15.84:0/3089152204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8000bbc0 con 0x7f9a80004d00 2023-10-16T09:50:19.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.847+0000 7f9a92ffd640 1 -- 172.21.15.84:0/3089152204 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8000bd80 con 0x7f9a9415ae80 2023-10-16T09:50:19.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.848+0000 7f9a937fe640 1 --2- 172.21.15.84:0/3089152204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a80004020 0x7f9a800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.848+0000 7f9a93fff640 1 --2- 172.21.15.84:0/3089152204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415ae80 0x7f9a800039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.848+0000 7f9a98f19640 1 --2- 172.21.15.84:0/3089152204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a80004d00 0x7f9a80005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.848+0000 7f9a937fe640 1 -- 172.21.15.84:0/3089152204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415ae80 msgr2=0x7f9a800039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:19.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.848+0000 7f9a937fe640 1 --2- 172.21.15.84:0/3089152204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415ae80 0x7f9a800039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.848+0000 7f9a937fe640 1 -- 172.21.15.84:0/3089152204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a80004d00 msgr2=0x7f9a80005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:19.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.848+0000 7f9a937fe640 1 --2- 172.21.15.84:0/3089152204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a80004d00 0x7f9a80005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.848+0000 7f9a937fe640 1 -- 172.21.15.84:0/3089152204 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a80003520 con 0x7f9a80004020 2023-10-16T09:50:19.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.849+0000 7f9a937fe640 1 --2- 172.21.15.84:0/3089152204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a80004020 0x7f9a800047c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00ba30 tx=0x7f9a800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efaea345a62ff4b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:19.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.850+0000 7f9a92ffd640 1 -- 172.21.15.84:0/3089152204 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c032640 con 0x7f9a80004020 2023-10-16T09:50:19.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.850+0000 7f9a92ffd640 1 -- 172.21.15.84:0/3089152204 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c0327a0 con 0x7f9a80004020 2023-10-16T09:50:19.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.850+0000 7f9a92ffd640 1 -- 172.21.15.84:0/3089152204 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c032b30 con 0x7f9a80004020 2023-10-16T09:50:19.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.850+0000 7f9a9a97c640 1 -- 172.21.15.84:0/3089152204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a80004020 msgr2=0x7f9a800047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:19.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.850+0000 7f9a9a97c640 1 --2- 172.21.15.84:0/3089152204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a80004020 0x7f9a800047c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00ba30 tx=0x7f9a800057f0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.850+0000 7f9a98f19640 1 -- 172.21.15.84:0/3089152204 reap_dead start 2023-10-16T09:50:19.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.850+0000 7f9a9a97c640 1 -- 172.21.15.84:0/3089152204 shutdown_connections 2023-10-16T09:50:19.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.850+0000 7f9a9a97c640 1 -- 172.21.15.84:0/3089152204 >> 172.21.15.84:0/3089152204 conn(0x7f9a940cb410 msgr2=0x7f9a940cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:19.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.850+0000 7f9a9a97c640 1 -- 172.21.15.84:0/3089152204 shutdown_connections 2023-10-16T09:50:19.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.850+0000 7f9a9a97c640 1 -- 172.21.15.84:0/3089152204 wait complete. 2023-10-16T09:50:19.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.853+0000 7f9a9a97c640 1 Processor -- start 2023-10-16T09:50:19.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.853+0000 7f9a9a97c640 1 -- start start 2023-10-16T09:50:19.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.854+0000 7f9a9a97c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a80004020 0x7f9a940c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.854+0000 7f9a9a97c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415c290 0x7f9a9415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.854+0000 7f9a9a97c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a9415de30 0x7f9a941568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.854+0000 7f9a9a97c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a941f22a0 con 0x7f9a80004020 2023-10-16T09:50:19.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.854+0000 7f9a9a97c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a941f16b0 con 0x7f9a9415de30 2023-10-16T09:50:19.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.854+0000 7f9a9a97c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a941f1900 con 0x7f9a9415c290 2023-10-16T09:50:19.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.854+0000 7f9a93fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a80004020 0x7f9a940c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.854+0000 7f9a93fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a80004020 0x7f9a940c4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37688/0 (socket says 172.21.15.84:37688) 2023-10-16T09:50:19.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.854+0000 7f9a93fff640 1 -- 172.21.15.84:0/313749606 learned_addr learned my addr 172.21.15.84:0/313749606 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:19.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.855+0000 7f9a93fff640 1 -- 172.21.15.84:0/313749606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415c290 msgr2=0x7f9a9415d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:19.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.855+0000 7f9a93fff640 1 --2- 172.21.15.84:0/313749606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415c290 0x7f9a9415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.855+0000 7f9a93fff640 1 -- 172.21.15.84:0/313749606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a9415de30 msgr2=0x7f9a941568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:19.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.855+0000 7f9a93fff640 1 --2- 172.21.15.84:0/313749606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a9415de30 0x7f9a941568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:19.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.855+0000 7f9a93fff640 1 -- 172.21.15.84:0/313749606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a941f1be0 con 0x7f9a80004020 2023-10-16T09:50:19.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.855+0000 7f9a93fff640 1 --2- 172.21.15.84:0/313749606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a80004020 0x7f9a940c4ca0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400e840 tx=0x7f9a8400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:19.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.855+0000 7f9a90ff9640 1 -- 172.21.15.84:0/313749606 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84003b80 con 0x7f9a80004020 2023-10-16T09:50:19.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.855+0000 7f9a90ff9640 1 -- 172.21.15.84:0/313749606 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a84003ce0 con 0x7f9a80004020 2023-10-16T09:50:19.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.855+0000 7f9a90ff9640 1 -- 172.21.15.84:0/313749606 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84005310 con 0x7f9a80004020 2023-10-16T09:50:19.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.855+0000 7f9a9a97c640 1 -- 172.21.15.84:0/313749606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a941f1ec0 con 0x7f9a80004020 2023-10-16T09:50:19.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.855+0000 7f9a9a97c640 1 -- 172.21.15.84:0/313749606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a94156e30 con 0x7f9a80004020 2023-10-16T09:50:19.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.857+0000 7f9a9a97c640 1 -- 172.21.15.84:0/313749606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a9415e260 con 0x7f9a80004020 2023-10-16T09:50:19.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.857+0000 7f9a90ff9640 1 -- 172.21.15.84:0/313749606 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9a84005470 con 0x7f9a80004020 2023-10-16T09:50:19.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.858+0000 7f9a90ff9640 1 --2- 172.21.15.84:0/313749606 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9a7406ae70 0x7f9a7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:19.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.859+0000 7f9a90ff9640 1 -- 172.21.15.84:0/313749606 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a84089650 con 0x7f9a80004020 2023-10-16T09:50:19.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.859+0000 7f9a937fe640 1 --2- 172.21.15.84:0/313749606 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9a7406ae70 0x7f9a7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:19.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.859+0000 7f9a937fe640 1 --2- 172.21.15.84:0/313749606 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9a7406ae70 0x7f9a7406d330 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00ba00 tx=0x7f9a7c032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:19.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:19.863+0000 7f9a90ff9640 1 -- 172.21.15.84:0/313749606 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9a84054790 con 0x7f9a80004020 2023-10-16T09:50:20.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:20.098+0000 7f9a9a97c640 1 -- 172.21.15.84:0/313749606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9a9404e160 con 0x7f9a80004020 2023-10-16T09:50:20.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:20.099+0000 7f9a90ff9640 1 -- 172.21.15.84:0/313749606 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9a84057db0 con 0x7f9a80004020 2023-10-16T09:50:20.101 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T09:50:20.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:20.101+0000 7f9a9a97c640 1 -- 172.21.15.84:0/313749606 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9a7406ae70 msgr2=0x7f9a7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:20.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:20.101+0000 7f9a9a97c640 1 --2- 172.21.15.84:0/313749606 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9a7406ae70 0x7f9a7406d330 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00ba00 tx=0x7f9a7c032000 comp rx=0 tx=0).stop 2023-10-16T09:50:20.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:20.101+0000 7f9a9a97c640 1 -- 172.21.15.84:0/313749606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a80004020 msgr2=0x7f9a940c4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:20.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:20.101+0000 7f9a9a97c640 1 --2- 172.21.15.84:0/313749606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a80004020 0x7f9a940c4ca0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400e840 tx=0x7f9a8400ed00 comp rx=0 tx=0).stop 2023-10-16T09:50:20.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:20.101+0000 7f9a9a97c640 1 -- 172.21.15.84:0/313749606 shutdown_connections 2023-10-16T09:50:20.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:20.101+0000 7f9a9a97c640 1 --2- 172.21.15.84:0/313749606 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9a7406ae70 0x7f9a7406d330 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:20.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:20.101+0000 7f9a9a97c640 1 --2- 172.21.15.84:0/313749606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a9415de30 0x7f9a941568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:20.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:20.101+0000 7f9a9a97c640 1 --2- 172.21.15.84:0/313749606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415c290 0x7f9a9415d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:20.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:20.101+0000 7f9a9a97c640 1 --2- 172.21.15.84:0/313749606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a80004020 0x7f9a940c4ca0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:20.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:20.101+0000 7f9a9a97c640 1 -- 172.21.15.84:0/313749606 >> 172.21.15.84:0/313749606 conn(0x7f9a940cb410 msgr2=0x7f9a9406ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:20.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:20.101+0000 7f9a9a97c640 1 -- 172.21.15.84:0/313749606 shutdown_connections 2023-10-16T09:50:20.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:20.101+0000 7f9a9a97c640 1 -- 172.21.15.84:0/313749606 wait complete. 2023-10-16T09:50:20.121 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T09:50:20.122 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T09:50:20.122 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-16T09:50:20.123 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T09:50:20.123 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T09:50:20.123 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T09:50:20.123 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T09:50:20.124 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T09:50:20.124 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T09:50:20.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:20.125 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-16T09:50:20.177 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:50:20.177 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:50:20.177 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:50:20.177 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:50:20.178 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T09:50:20.178 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:50:20.178 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:50:20.178 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:50:20.178 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-16T09:50:20.179 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:50:20.179 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:50:20.179 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-16T09:50:20.179 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:50:20.179 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:50:20.180 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T09:50:20.180 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:50:20.180 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T09:50:20.180 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42331sec preferred_lft 42331sec 2023-10-16T09:50:20.180 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-16T09:50:20.181 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:50:20.181 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T09:50:20.181 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:50:20.181 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T09:50:20.181 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:50:20.181 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::20a8:eaff:fe9d:5cea/64 scope link 2023-10-16T09:50:20.182 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:50:20.182 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T09:50:20.182 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T09:50:20.182 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1854:13ff:fe47:be92/64 scope link 2023-10-16T09:50:20.182 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:50:20.183 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T09:50:20.183 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether ee:d5:41:a3:a5:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T09:50:20.183 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ecd5:41ff:fea3:a570/64 scope link 2023-10-16T09:50:20.183 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:50:20.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:20.185 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T09:50:20.278 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-16T09:50:20.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:20.283 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-16T09:50:20.337 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-16T09:50:20.337 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T09:50:20.337 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T09:50:20.338 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T09:50:20.338 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T09:50:20.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:20.339 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T09:50:20.339 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T09:50:20.339 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T09:50:20.340 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T09:50:20.340 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T09:50:20.340 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T09:50:20.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:20.529 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T09:50:20.582 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T09:50:20.582 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T09:50:20.583 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T09:50:20.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:20.584 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-16T09:50:20.670 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:50:20.670 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T09:50:20.722 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T09:50:20.722 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:50:20.722 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T09:50:20.779 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-16T09:50:20.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:20.779 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T09:50:20.912 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T09:50:20.912 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T09:50:20.948 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T09:50:20.948 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T09:50:20.948 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T09:50:20.948 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with new device syntax: 0@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T09:50:20.949 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T09:50:20.953 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-16T09:50:20.953 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2023-10-16T09:50:20.953 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T09:50:20.953 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T09:50:20.954 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T09:50:20.958 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T09:50:20.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:20.959 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T09:50:21.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:21.046 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:50:21.119 INFO:teuthology.orchestra.run.smithi084.stdout:client4682 2023-10-16T09:50:21.119 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T09:50:21.119 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:50:21.176 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T09:50:21.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.388+0000 7f6e03275640 1 Processor -- start 2023-10-16T09:50:21.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.388+0000 7f6e03275640 1 -- start start 2023-10-16T09:50:21.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.389+0000 7f6e03275640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6dfc159ab0 0x7f6dfc0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:21.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.389+0000 7f6e03275640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6dfc06c140 0x7f6dfc06c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:21.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.389+0000 7f6e03275640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dfc06cad0 con 0x7f6dfc159e90 2023-10-16T09:50:21.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.389+0000 7f6e03275640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dfc15d890 con 0x7f6dfc06c140 2023-10-16T09:50:21.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.389+0000 7f6e03275640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dfc070360 con 0x7f6dfc159ab0 2023-10-16T09:50:21.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.390+0000 7f6e01812640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6dfc06c140 0x7f6dfc06c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:21.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.390+0000 7f6e00810640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6dfc159e90 0x7f6dfc06fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33612/0 (socket says 172.21.15.84:33612) 2023-10-16T09:50:21.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.390+0000 7f6e01812640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6dfc06c140 0x7f6dfc06c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37692/0 (socket says 172.21.15.84:37692) 2023-10-16T09:50:21.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.390+0000 7f6e00810640 1 -- 172.21.15.84:0/688378503 learned_addr learned my addr 172.21.15.84:0/688378503 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:21.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.390+0000 7f6e01011640 1 --2- 172.21.15.84:0/688378503 >> v2:172.21.15.157:3300/0 conn(0x7f6dfc159ab0 0x7f6dfc0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:21.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.390+0000 7f6e01812640 1 -- 172.21.15.84:0/688378503 >> v2:172.21.15.157:3300/0 conn(0x7f6dfc159ab0 msgr2=0x7f6dfc0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:21.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.390+0000 7f6e01812640 1 --2- 172.21.15.84:0/688378503 >> v2:172.21.15.157:3300/0 conn(0x7f6dfc159ab0 0x7f6dfc0ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:21.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.390+0000 7f6e01812640 1 -- 172.21.15.84:0/688378503 >> v1:172.21.15.84:6789/0 conn(0x7f6dfc159e90 legacy=0x7f6dfc06fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:21.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.390+0000 7f6e01812640 1 -- 172.21.15.84:0/688378503 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc0cd1f0 con 0x7f6dfc06c140 2023-10-16T09:50:21.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.392+0000 7f6e01812640 1 --2- 172.21.15.84:0/688378503 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dfc06c140 0x7f6dfc06c590 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0cfa00 tx=0x7f6df402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfae833ef5a0b06 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:21.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.392+0000 7f6df3fff640 1 -- 172.21.15.84:0/688378503 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df402cb60 con 0x7f6dfc06c140 2023-10-16T09:50:21.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.392+0000 7f6df3fff640 1 -- 172.21.15.84:0/688378503 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df402ccc0 con 0x7f6dfc06c140 2023-10-16T09:50:21.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.392+0000 7f6df3fff640 1 -- 172.21.15.84:0/688378503 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df4034660 con 0x7f6dfc06c140 2023-10-16T09:50:21.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.393+0000 7f6e03275640 1 -- 172.21.15.84:0/688378503 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dfc06c140 msgr2=0x7f6dfc06c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:21.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.393+0000 7f6e03275640 1 --2- 172.21.15.84:0/688378503 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dfc06c140 0x7f6dfc06c590 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0cfa00 tx=0x7f6df402bdb0 comp rx=0 tx=0).stop 2023-10-16T09:50:21.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.393+0000 7f6e03275640 1 -- 172.21.15.84:0/688378503 shutdown_connections 2023-10-16T09:50:21.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.393+0000 7f6e03275640 1 --2- 172.21.15.84:0/688378503 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dfc06c140 0x7f6dfc06c590 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:21.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.393+0000 7f6e03275640 1 --2- 172.21.15.84:0/688378503 >> v2:172.21.15.157:3300/0 conn(0x7f6dfc159ab0 0x7f6dfc0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:21.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.393+0000 7f6e03275640 1 -- 172.21.15.84:0/688378503 >> 172.21.15.84:0/688378503 conn(0x7f6dfc0ce9b0 msgr2=0x7f6dfc0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:21.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.393+0000 7f6e03275640 1 -- 172.21.15.84:0/688378503 shutdown_connections 2023-10-16T09:50:21.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.393+0000 7f6e03275640 1 -- 172.21.15.84:0/688378503 wait complete. 2023-10-16T09:50:21.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.395+0000 7f6e03275640 1 Processor -- start 2023-10-16T09:50:21.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.396+0000 7f6e03275640 1 -- start start 2023-10-16T09:50:21.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.397+0000 7f6e03275640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dfc06c140 0x7f6dfc156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:21.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.397+0000 7f6e03275640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dfc159ab0 0x7f6dfc1f0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:21.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.397+0000 7f6e03275640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dfc159e90 0x7f6dfc15bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:21.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.397+0000 7f6e03275640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc1e3710 con 0x7f6dfc06c140 2023-10-16T09:50:21.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.397+0000 7f6e03275640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc1e2b00 con 0x7f6dfc159e90 2023-10-16T09:50:21.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.397+0000 7f6e03275640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc1e2de0 con 0x7f6dfc159ab0 2023-10-16T09:50:21.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.397+0000 7f6e01011640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dfc06c140 0x7f6dfc156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:21.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.397+0000 7f6e01011640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dfc06c140 0x7f6dfc156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37702/0 (socket says 172.21.15.84:37702) 2023-10-16T09:50:21.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.397+0000 7f6e01011640 1 -- 172.21.15.84:0/38263262 learned_addr learned my addr 172.21.15.84:0/38263262 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:21.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.398+0000 7f6e01011640 1 -- 172.21.15.84:0/38263262 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dfc159ab0 msgr2=0x7f6dfc1f0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:21.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.398+0000 7f6e01011640 1 --2- 172.21.15.84:0/38263262 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dfc159ab0 0x7f6dfc1f0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:21.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.398+0000 7f6e01011640 1 -- 172.21.15.84:0/38263262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dfc159e90 msgr2=0x7f6dfc15bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:21.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.398+0000 7f6e01011640 1 --2- 172.21.15.84:0/38263262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dfc159e90 0x7f6dfc15bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:21.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.398+0000 7f6e01011640 1 -- 172.21.15.84:0/38263262 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df4009a70 con 0x7f6dfc06c140 2023-10-16T09:50:21.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.398+0000 7f6e01011640 1 --2- 172.21.15.84:0/38263262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dfc06c140 0x7f6dfc156ba0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6df800d800 tx=0x7f6df800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:21.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.398+0000 7f6df1ffb640 1 -- 172.21.15.84:0/38263262 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df800b6c0 con 0x7f6dfc06c140 2023-10-16T09:50:21.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.398+0000 7f6df1ffb640 1 -- 172.21.15.84:0/38263262 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df800b820 con 0x7f6dfc06c140 2023-10-16T09:50:21.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.398+0000 7f6df1ffb640 1 -- 172.21.15.84:0/38263262 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df800bbb0 con 0x7f6dfc06c140 2023-10-16T09:50:21.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.398+0000 7f6e03275640 1 -- 172.21.15.84:0/38263262 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dfc1e30c0 con 0x7f6dfc06c140 2023-10-16T09:50:21.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.398+0000 7f6e03275640 1 -- 172.21.15.84:0/38263262 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dfc157440 con 0x7f6dfc06c140 2023-10-16T09:50:21.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.400+0000 7f6e03275640 1 -- 172.21.15.84:0/38263262 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dfc04e160 con 0x7f6dfc06c140 2023-10-16T09:50:21.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.401+0000 7f6df1ffb640 1 -- 172.21.15.84:0/38263262 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6df800bd10 con 0x7f6dfc06c140 2023-10-16T09:50:21.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.402+0000 7f6df1ffb640 1 --2- 172.21.15.84:0/38263262 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6de006ae70 0x7f6de006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:21.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.402+0000 7f6df1ffb640 1 -- 172.21.15.84:0/38263262 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6df8089b70 con 0x7f6dfc06c140 2023-10-16T09:50:21.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.402+0000 7f6e00810640 1 --2- 172.21.15.84:0/38263262 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6de006ae70 0x7f6de006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:21.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.403+0000 7f6e00810640 1 --2- 172.21.15.84:0/38263262 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6de006ae70 0x7f6de006d330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc157860 tx=0x7f6dec007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:21.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.411+0000 7f6df1ffb640 1 -- 172.21.15.84:0/38263262 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6df8054c20 con 0x7f6dfc06c140 2023-10-16T09:50:21.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.630+0000 7f6e03275640 1 -- 172.21.15.84:0/38263262 --> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6dfc0cd7b0 con 0x7f6de006ae70 2023-10-16T09:50:21.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.633+0000 7f6df1ffb640 1 -- 172.21.15.84:0/38263262 <== mgr.4107 v2:172.21.15.84:6832/3010246666 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6dec0058b0 con 0x7f6de006ae70 2023-10-16T09:50:21.635 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T09:50:21.635 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T09:50:21.635 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephfs" 2023-10-16T09:50:21.636 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T09:50:21.636 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T09:50:21.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.636+0000 7f6e03275640 1 -- 172.21.15.84:0/38263262 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6de006ae70 msgr2=0x7f6de006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:21.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.636+0000 7f6e03275640 1 --2- 172.21.15.84:0/38263262 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6de006ae70 0x7f6de006d330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc157860 tx=0x7f6dec007a40 comp rx=0 tx=0).stop 2023-10-16T09:50:21.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.636+0000 7f6e03275640 1 -- 172.21.15.84:0/38263262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dfc06c140 msgr2=0x7f6dfc156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:21.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.636+0000 7f6e03275640 1 --2- 172.21.15.84:0/38263262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dfc06c140 0x7f6dfc156ba0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6df800d800 tx=0x7f6df800dcc0 comp rx=0 tx=0).stop 2023-10-16T09:50:21.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.636+0000 7f6e03275640 1 -- 172.21.15.84:0/38263262 shutdown_connections 2023-10-16T09:50:21.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.636+0000 7f6e03275640 1 --2- 172.21.15.84:0/38263262 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f6de006ae70 0x7f6de006d330 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:21.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.636+0000 7f6e03275640 1 --2- 172.21.15.84:0/38263262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dfc159e90 0x7f6dfc15bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:21.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.636+0000 7f6e03275640 1 --2- 172.21.15.84:0/38263262 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dfc159ab0 0x7f6dfc1f0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:21.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.636+0000 7f6e03275640 1 --2- 172.21.15.84:0/38263262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dfc06c140 0x7f6dfc156ba0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:21.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.636+0000 7f6e03275640 1 -- 172.21.15.84:0/38263262 >> 172.21.15.84:0/38263262 conn(0x7f6dfc0ce9b0 msgr2=0x7f6dfc158f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:21.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.636+0000 7f6e03275640 1 -- 172.21.15.84:0/38263262 shutdown_connections 2023-10-16T09:50:21.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.636+0000 7f6e03275640 1 -- 172.21.15.84:0/38263262 wait complete. 2023-10-16T09:50:21.656 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T09:50:21.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.855+0000 7fc78bb33640 1 Processor -- start 2023-10-16T09:50:21.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.855+0000 7fc78bb33640 1 -- start start 2023-10-16T09:50:21.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.856+0000 7fc78bb33640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc784159e90 0x7fc7840ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:21.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.856+0000 7fc78bb33640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc78406c140 0x7fc78406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:21.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.856+0000 7fc78bb33640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc784070360 con 0x7fc784159ab0 2023-10-16T09:50:21.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.856+0000 7fc78bb33640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc78406cb10 con 0x7fc784159e90 2023-10-16T09:50:21.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.856+0000 7fc78bb33640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78415d890 con 0x7fc78406c140 2023-10-16T09:50:21.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.856+0000 7fc7898cf640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc784159ab0 0x7fc78406fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33628/0 (socket says 172.21.15.84:33628) 2023-10-16T09:50:21.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc7898cf640 1 -- 172.21.15.84:0/749367694 learned_addr learned my addr 172.21.15.84:0/749367694 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:21.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc7890ce640 1 --2- 172.21.15.84:0/749367694 >> v2:172.21.15.157:3300/0 conn(0x7fc784159e90 0x7fc7840ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:21.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc78a0d0640 1 --2- 172.21.15.84:0/749367694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc78406c140 0x7fc78406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:21.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc7888cd640 1 -- 172.21.15.84:0/749367694 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402497959 0 0) 0x7fc784070360 con 0x7fc784159ab0 2023-10-16T09:50:21.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc7888cd640 1 -- 172.21.15.84:0/749367694 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc770003520 con 0x7fc784159ab0 2023-10-16T09:50:21.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc7888cd640 1 -- 172.21.15.84:0/749367694 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1656002370 0 0) 0x7fc770003520 con 0x7fc784159ab0 2023-10-16T09:50:21.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc7888cd640 1 -- 172.21.15.84:0/749367694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc78406c140 msgr2=0x7fc78406c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:21.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc7888cd640 1 --2- 172.21.15.84:0/749367694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc78406c140 0x7fc78406c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:21.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc7888cd640 1 -- 172.21.15.84:0/749367694 >> v2:172.21.15.157:3300/0 conn(0x7fc784159e90 msgr2=0x7fc7840ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:21.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc7888cd640 1 --2- 172.21.15.84:0/749367694 >> v2:172.21.15.157:3300/0 conn(0x7fc784159e90 0x7fc7840ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:21.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc7888cd640 1 -- 172.21.15.84:0/749367694 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7840cd1f0 con 0x7fc784159ab0 2023-10-16T09:50:21.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc7888cd640 1 -- 172.21.15.84:0/749367694 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc780001670 con 0x7fc784159ab0 2023-10-16T09:50:21.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc7888cd640 1 -- 172.21.15.84:0/749367694 >> v1:172.21.15.84:6789/0 conn(0x7fc784159ab0 legacy=0x7fc78406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc7888cd640 1 --2- 172.21.15.84:0/749367694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc770003970 0x7fc770003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc7888cd640 1 --2- 172.21.15.84:0/749367694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc770004320 0x7fc7700047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc7888cd640 1 --2- 172.21.15.84:0/749367694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc770005080 0x7fc77000ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc7888cd640 1 -- 172.21.15.84:0/749367694 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78406cb10 con 0x7fc770004320 2023-10-16T09:50:21.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc7888cd640 1 -- 172.21.15.84:0/749367694 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78415d890 con 0x7fc770003970 2023-10-16T09:50:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.857+0000 7fc7888cd640 1 -- 172.21.15.84:0/749367694 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc770007c10 con 0x7fc770005080 2023-10-16T09:50:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.858+0000 7fc78a0d0640 1 --2- 172.21.15.84:0/749367694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc78406c140 0x7fc78406c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:50:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.858+0000 7fc7890ce640 1 --2- 172.21.15.84:0/749367694 >> v2:172.21.15.157:3300/0 conn(0x7fc784159e90 0x7fc7840ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.858+0000 7fc7890ce640 1 --2- 172.21.15.84:0/749367694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc770004320 0x7fc7700047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:21.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.858+0000 7fc7898cf640 1 --2- 172.21.15.84:0/749367694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc770003970 0x7fc770003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:21.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.858+0000 7fc78a0d0640 1 --2- 172.21.15.84:0/749367694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc770005080 0x7fc77000ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:21.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.858+0000 7fc7890ce640 1 -- 172.21.15.84:0/749367694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc770005080 msgr2=0x7fc77000ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:21.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.858+0000 7fc7890ce640 1 --2- 172.21.15.84:0/749367694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc770005080 0x7fc77000ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:21.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.858+0000 7fc7890ce640 1 -- 172.21.15.84:0/749367694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc770003970 msgr2=0x7fc770003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:21.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.858+0000 7fc7890ce640 1 --2- 172.21.15.84:0/749367694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc770003970 0x7fc770003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.858+0000 7fc7890ce640 1 -- 172.21.15.84:0/749367694 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc770003520 con 0x7fc770004320 2023-10-16T09:50:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.860+0000 7fc7890ce640 1 --2- 172.21.15.84:0/749367694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc770004320 0x7fc7700047e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc77400aa70 tx=0x7fc7700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c545c9b3bc8714a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.860+0000 7fc7888cd640 1 -- 172.21.15.84:0/749367694 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774004220 con 0x7fc770004320 2023-10-16T09:50:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.860+0000 7fc7888cd640 1 -- 172.21.15.84:0/749367694 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc774004380 con 0x7fc770004320 2023-10-16T09:50:21.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.860+0000 7fc7888cd640 1 -- 172.21.15.84:0/749367694 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774003830 con 0x7fc770004320 2023-10-16T09:50:21.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.860+0000 7fc78bb33640 1 -- 172.21.15.84:0/749367694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc770004320 msgr2=0x7fc7700047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:21.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.860+0000 7fc78bb33640 1 --2- 172.21.15.84:0/749367694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc770004320 0x7fc7700047e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc77400aa70 tx=0x7fc7700057f0 comp rx=0 tx=0).stop 2023-10-16T09:50:21.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.860+0000 7fc78a0d0640 1 -- 172.21.15.84:0/749367694 reap_dead start 2023-10-16T09:50:21.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.861+0000 7fc78bb33640 1 -- 172.21.15.84:0/749367694 shutdown_connections 2023-10-16T09:50:21.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.861+0000 7fc78bb33640 1 -- 172.21.15.84:0/749367694 >> 172.21.15.84:0/749367694 conn(0x7fc7840ce9b0 msgr2=0x7fc7840cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:21.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.861+0000 7fc78bb33640 1 -- 172.21.15.84:0/749367694 shutdown_connections 2023-10-16T09:50:21.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.861+0000 7fc78bb33640 1 -- 172.21.15.84:0/749367694 wait complete. 2023-10-16T09:50:21.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.862+0000 7fc78bb33640 1 Processor -- start 2023-10-16T09:50:21.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.863+0000 7fc78bb33640 1 -- start start 2023-10-16T09:50:21.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.863+0000 7fc78bb33640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc770004320 0x7fc7841517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:21.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.863+0000 7fc78bb33640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc784150d30 0x7fc7840c1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:21.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.863+0000 7fc78bb33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7840c24e0 0x7fc784069da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:21.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.864+0000 7fc78bb33640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840c33a0 con 0x7fc770004320 2023-10-16T09:50:21.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.864+0000 7fc78bb33640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840c2de0 con 0x7fc7840c24e0 2023-10-16T09:50:21.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.864+0000 7fc78bb33640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840c30c0 con 0x7fc784150d30 2023-10-16T09:50:21.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.864+0000 7fc7898cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc770004320 0x7fc7841517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:21.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.864+0000 7fc7898cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc770004320 0x7fc7841517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37716/0 (socket says 172.21.15.84:37716) 2023-10-16T09:50:21.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.864+0000 7fc7898cf640 1 -- 172.21.15.84:0/1459357851 learned_addr learned my addr 172.21.15.84:0/1459357851 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:21.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.864+0000 7fc78a0d0640 1 --2- 172.21.15.84:0/1459357851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7840c24e0 0x7fc784069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:21.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.864+0000 7fc7890ce640 1 --2- 172.21.15.84:0/1459357851 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc784150d30 0x7fc7840c1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:21.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.864+0000 7fc7898cf640 1 -- 172.21.15.84:0/1459357851 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc784150d30 msgr2=0x7fc7840c1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:21.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.864+0000 7fc7898cf640 1 --2- 172.21.15.84:0/1459357851 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc784150d30 0x7fc7840c1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:21.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.864+0000 7fc7898cf640 1 -- 172.21.15.84:0/1459357851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7840c24e0 msgr2=0x7fc784069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:21.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.864+0000 7fc7898cf640 1 --2- 172.21.15.84:0/1459357851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7840c24e0 0x7fc784069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:21.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.864+0000 7fc7898cf640 1 -- 172.21.15.84:0/1459357851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7840c3da0 con 0x7fc770004320 2023-10-16T09:50:21.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.865+0000 7fc7898cf640 1 --2- 172.21.15.84:0/1459357851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc770004320 0x7fc7841517b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc78000e780 tx=0x7fc78000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:21.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.865+0000 7fc77a7fc640 1 -- 172.21.15.84:0/1459357851 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78000ee90 con 0x7fc770004320 2023-10-16T09:50:21.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.865+0000 7fc77a7fc640 1 -- 172.21.15.84:0/1459357851 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78000c870 con 0x7fc770004320 2023-10-16T09:50:21.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.865+0000 7fc78bb33640 1 -- 172.21.15.84:0/1459357851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7840c4080 con 0x7fc770004320 2023-10-16T09:50:21.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.865+0000 7fc77a7fc640 1 -- 172.21.15.84:0/1459357851 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78000cc20 con 0x7fc770004320 2023-10-16T09:50:21.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.865+0000 7fc78bb33640 1 -- 172.21.15.84:0/1459357851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7840c2960 con 0x7fc770004320 2023-10-16T09:50:21.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.866+0000 7fc78bb33640 1 -- 172.21.15.84:0/1459357851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7840c2b50 con 0x7fc770004320 2023-10-16T09:50:21.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.867+0000 7fc77a7fc640 1 -- 172.21.15.84:0/1459357851 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc78001d460 con 0x7fc770004320 2023-10-16T09:50:21.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.868+0000 7fc77a7fc640 1 --2- 172.21.15.84:0/1459357851 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc76806ae70 0x7fc76806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:21.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.868+0000 7fc77a7fc640 1 -- 172.21.15.84:0/1459357851 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc780089580 con 0x7fc770004320 2023-10-16T09:50:21.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.868+0000 7fc7890ce640 1 --2- 172.21.15.84:0/1459357851 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc76806ae70 0x7fc76806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:21.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.869+0000 7fc7890ce640 1 --2- 172.21.15.84:0/1459357851 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc76806ae70 0x7fc76806d330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc774035000 tx=0x7fc774039040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:21.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:21.872+0000 7fc77a7fc640 1 -- 172.21.15.84:0/1459357851 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc780054590 con 0x7fc770004320 2023-10-16T09:50:22.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.122+0000 7fc78bb33640 1 -- 172.21.15.84:0/1459357851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fc7840cda60 con 0x7fc770004320 2023-10-16T09:50:22.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.678+0000 7fc77a7fc640 1 -- 172.21.15.84:0/1459357851 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) v1 ==== 92+0+0 (secure 0 0 0) 0x7fc780057bb0 con 0x7fc770004320 2023-10-16T09:50:22.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.683+0000 7fc78bb33640 1 -- 172.21.15.84:0/1459357851 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc76806ae70 msgr2=0x7fc76806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:22.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.683+0000 7fc78bb33640 1 --2- 172.21.15.84:0/1459357851 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc76806ae70 0x7fc76806d330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc774035000 tx=0x7fc774039040 comp rx=0 tx=0).stop 2023-10-16T09:50:22.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.683+0000 7fc78bb33640 1 -- 172.21.15.84:0/1459357851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc770004320 msgr2=0x7fc7841517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:22.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.683+0000 7fc78bb33640 1 --2- 172.21.15.84:0/1459357851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc770004320 0x7fc7841517b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc78000e780 tx=0x7fc78000ec40 comp rx=0 tx=0).stop 2023-10-16T09:50:22.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.684+0000 7fc78bb33640 1 -- 172.21.15.84:0/1459357851 shutdown_connections 2023-10-16T09:50:22.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.684+0000 7fc78bb33640 1 --2- 172.21.15.84:0/1459357851 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7fc76806ae70 0x7fc76806d330 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:22.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.684+0000 7fc78bb33640 1 --2- 172.21.15.84:0/1459357851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7840c24e0 0x7fc784069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:22.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.684+0000 7fc78bb33640 1 --2- 172.21.15.84:0/1459357851 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc784150d30 0x7fc7840c1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:22.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.684+0000 7fc78bb33640 1 --2- 172.21.15.84:0/1459357851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc770004320 0x7fc7841517b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:22.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.684+0000 7fc78bb33640 1 -- 172.21.15.84:0/1459357851 >> 172.21.15.84:0/1459357851 conn(0x7fc7840ce9b0 msgr2=0x7fc7840ae410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:22.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.684+0000 7fc78bb33640 1 -- 172.21.15.84:0/1459357851 shutdown_connections 2023-10-16T09:50:22.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.684+0000 7fc78bb33640 1 -- 172.21.15.84:0/1459357851 wait complete. 2023-10-16T09:50:22.712 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T09:50:22.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.908+0000 7f1aaab16640 1 Processor -- start 2023-10-16T09:50:22.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.909+0000 7f1aaab16640 1 -- start start 2023-10-16T09:50:22.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.910+0000 7f1aaab16640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1aa415cd70 0x7f1aa40d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:22.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.910+0000 7f1aaab16640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa40b8480 0x7f1aa40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:22.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.910+0000 7f1aaab16640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa415d150 con 0x7f1aa40d0be0 2023-10-16T09:50:22.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.910+0000 7f1aaab16640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa40d1180 con 0x7f1aa415cd70 2023-10-16T09:50:22.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.910+0000 7f1aaab16640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40d1460 con 0x7f1aa40b8480 2023-10-16T09:50:22.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.910+0000 7f1aa9b14640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1aa415cd70 0x7f1aa40d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:22.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.910+0000 7f1aaa315640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1aa40d0be0 0x7f1aa40b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:42008/0 (socket says 172.21.15.84:42008) 2023-10-16T09:50:22.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.910+0000 7f1aaa315640 1 -- 172.21.15.84:0/1277900634 learned_addr learned my addr 172.21.15.84:0/1277900634 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:22.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.910+0000 7f1aa9313640 1 --2- 172.21.15.84:0/1277900634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa40b8480 0x7f1aa40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:22.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.911+0000 7f1aa9b14640 1 -- 172.21.15.84:0/1277900634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa40b8480 msgr2=0x7f1aa40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:22.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.911+0000 7f1aa9b14640 1 --2- 172.21.15.84:0/1277900634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa40b8480 0x7f1aa40cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:22.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.911+0000 7f1aa9b14640 1 -- 172.21.15.84:0/1277900634 >> v1:172.21.15.157:6789/0 conn(0x7f1aa40d0be0 legacy=0x7f1aa40b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:22.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.911+0000 7f1aa9b14640 1 -- 172.21.15.84:0/1277900634 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa40d1740 con 0x7f1aa415cd70 2023-10-16T09:50:22.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.912+0000 7f1aa9b14640 1 --2- 172.21.15.84:0/1277900634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa415cd70 0x7f1aa40d2220 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00097a0 tx=0x7f1aa002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=328dec7e3d0bcf39 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:22.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.913+0000 7f1aa8b12640 1 -- 172.21.15.84:0/1277900634 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0031830 con 0x7f1aa415cd70 2023-10-16T09:50:22.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.913+0000 7f1aa8b12640 1 -- 172.21.15.84:0/1277900634 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa0031990 con 0x7f1aa415cd70 2023-10-16T09:50:22.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.913+0000 7f1aa8b12640 1 -- 172.21.15.84:0/1277900634 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0031d20 con 0x7f1aa415cd70 2023-10-16T09:50:22.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.914+0000 7f1aaab16640 1 -- 172.21.15.84:0/1277900634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa415cd70 msgr2=0x7f1aa40d2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:22.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.914+0000 7f1aaab16640 1 --2- 172.21.15.84:0/1277900634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa415cd70 0x7f1aa40d2220 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00097a0 tx=0x7f1aa002b8d0 comp rx=0 tx=0).stop 2023-10-16T09:50:22.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.914+0000 7f1aaab16640 1 -- 172.21.15.84:0/1277900634 shutdown_connections 2023-10-16T09:50:22.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.914+0000 7f1aaab16640 1 --2- 172.21.15.84:0/1277900634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa40b8480 0x7f1aa40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:22.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.914+0000 7f1aaab16640 1 --2- 172.21.15.84:0/1277900634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa415cd70 0x7f1aa40d2220 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:22.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.914+0000 7f1aaab16640 1 -- 172.21.15.84:0/1277900634 >> 172.21.15.84:0/1277900634 conn(0x7f1aa40cf570 msgr2=0x7f1aa40cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:22.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.914+0000 7f1aaab16640 1 -- 172.21.15.84:0/1277900634 shutdown_connections 2023-10-16T09:50:22.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.914+0000 7f1aaab16640 1 -- 172.21.15.84:0/1277900634 wait complete. 2023-10-16T09:50:22.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.917+0000 7f1aaab16640 1 Processor -- start 2023-10-16T09:50:22.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.917+0000 7f1aaab16640 1 -- start start 2023-10-16T09:50:22.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.918+0000 7f1aaab16640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1aa40b8480 0x7f1aa4148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:22.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.918+0000 7f1aaab16640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa40d0be0 0x7f1aa41f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:22.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.918+0000 7f1aaab16640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa415cd70 0x7f1aa41f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:22.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.918+0000 7f1aaab16640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa4148950 con 0x7f1aa40b8480 2023-10-16T09:50:22.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.918+0000 7f1aaab16640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40cc630 con 0x7f1aa415cd70 2023-10-16T09:50:22.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.918+0000 7f1aaab16640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40cba20 con 0x7f1aa40d0be0 2023-10-16T09:50:22.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.919+0000 7f1aa9b14640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1aa40b8480 0x7f1aa4148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:22.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.919+0000 7f1aa9b14640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1aa40b8480 0x7f1aa4148410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37750/0 (socket says 172.21.15.84:37750) 2023-10-16T09:50:22.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.919+0000 7f1aa9b14640 1 -- 172.21.15.84:0/2142265227 learned_addr learned my addr 172.21.15.84:0/2142265227 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:22.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.919+0000 7f1aaa315640 1 --2- 172.21.15.84:0/2142265227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa415cd70 0x7f1aa41f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:22.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.919+0000 7f1aa9313640 1 --2- 172.21.15.84:0/2142265227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa40d0be0 0x7f1aa41f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:22.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.919+0000 7f1aa9b14640 1 -- 172.21.15.84:0/2142265227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa40d0be0 msgr2=0x7f1aa41f55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:22.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.919+0000 7f1aa9b14640 1 --2- 172.21.15.84:0/2142265227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa40d0be0 0x7f1aa41f55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:22.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.919+0000 7f1aa9b14640 1 -- 172.21.15.84:0/2142265227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa415cd70 msgr2=0x7f1aa41f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:22.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.919+0000 7f1aa9b14640 1 --2- 172.21.15.84:0/2142265227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa415cd70 0x7f1aa41f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:22.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.919+0000 7f1aa9b14640 1 -- 172.21.15.84:0/2142265227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa0009400 con 0x7f1aa40b8480 2023-10-16T09:50:22.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.919+0000 7f1aa9313640 1 --2- 172.21.15.84:0/2142265227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa40d0be0 0x7f1aa41f55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:50:22.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.920+0000 7f1aa9b14640 1 --2- 172.21.15.84:0/2142265227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1aa40b8480 0x7f1aa4148410 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00098c0 tx=0x7f1aa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:22.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.920+0000 7f1a9a7fc640 1 -- 172.21.15.84:0/2142265227 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa003a6f0 con 0x7f1aa40b8480 2023-10-16T09:50:22.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.920+0000 7f1a9a7fc640 1 -- 172.21.15.84:0/2142265227 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa003a870 con 0x7f1aa40b8480 2023-10-16T09:50:22.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.920+0000 7f1aaab16640 1 -- 172.21.15.84:0/2142265227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa40cbd00 con 0x7f1aa40b8480 2023-10-16T09:50:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.920+0000 7f1a9a7fc640 1 -- 172.21.15.84:0/2142265227 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa003ac20 con 0x7f1aa40b8480 2023-10-16T09:50:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.920+0000 7f1aaab16640 1 -- 172.21.15.84:0/2142265227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa40cc2c0 con 0x7f1aa40b8480 2023-10-16T09:50:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.921+0000 7f1aaab16640 1 -- 172.21.15.84:0/2142265227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa404e9b0 con 0x7f1aa40b8480 2023-10-16T09:50:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.922+0000 7f1a9a7fc640 1 -- 172.21.15.84:0/2142265227 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 6) v1 ==== 89246+0+0 (secure 0 0 0) 0x7f1aa004c3b0 con 0x7f1aa40b8480 2023-10-16T09:50:22.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.923+0000 7f1a9a7fc640 1 --2- 172.21.15.84:0/2142265227 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f1a8806aef0 0x7f1a8806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:22.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.923+0000 7f1a9a7fc640 1 -- 172.21.15.84:0/2142265227 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1aa00ab210 con 0x7f1aa40b8480 2023-10-16T09:50:22.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.923+0000 7f1aa9313640 1 -- 172.21.15.84:0/2142265227 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f1a8806aef0 msgr2=0x7f1a8806d3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3010246666 2023-10-16T09:50:22.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.923+0000 7f1aa9313640 1 --2- 172.21.15.84:0/2142265227 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f1a8806aef0 0x7f1a8806d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:50:22.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:22.928+0000 7f1a9a7fc640 1 -- 172.21.15.84:0/2142265227 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1aa00762d0 con 0x7f1aa40b8480 2023-10-16T09:50:23.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.111+0000 7f1aaab16640 1 -- 172.21.15.84:0/2142265227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f1aa40cc770 con 0x7f1aa40b8480 2023-10-16T09:50:23.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.119+0000 7f1a9a7fc640 1 -- 172.21.15.84:0/2142265227 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1aa00798f0 con 0x7f1aa40b8480 2023-10-16T09:50:23.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.121+0000 7f1aaab16640 1 -- 172.21.15.84:0/2142265227 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f1a8806aef0 msgr2=0x7f1a8806d3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:50:23.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.121+0000 7f1aaab16640 1 --2- 172.21.15.84:0/2142265227 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f1a8806aef0 0x7f1a8806d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.121+0000 7f1aaab16640 1 -- 172.21.15.84:0/2142265227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1aa40b8480 msgr2=0x7f1aa4148410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:23.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.121+0000 7f1aaab16640 1 --2- 172.21.15.84:0/2142265227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1aa40b8480 0x7f1aa4148410 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00098c0 tx=0x7f1aa00046c0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.122+0000 7f1aaab16640 1 -- 172.21.15.84:0/2142265227 shutdown_connections 2023-10-16T09:50:23.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.122+0000 7f1aaab16640 1 --2- 172.21.15.84:0/2142265227 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f1a8806aef0 0x7f1a8806d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.122+0000 7f1aaab16640 1 --2- 172.21.15.84:0/2142265227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa415cd70 0x7f1aa41f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.122+0000 7f1aaab16640 1 --2- 172.21.15.84:0/2142265227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa40d0be0 0x7f1aa41f55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.122+0000 7f1aaab16640 1 --2- 172.21.15.84:0/2142265227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1aa40b8480 0x7f1aa4148410 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.122+0000 7f1aaab16640 1 -- 172.21.15.84:0/2142265227 >> 172.21.15.84:0/2142265227 conn(0x7f1aa40cf570 msgr2=0x7f1aa40d2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:23.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.122+0000 7f1aaab16640 1 -- 172.21.15.84:0/2142265227 shutdown_connections 2023-10-16T09:50:23.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.122+0000 7f1aaab16640 1 -- 172.21.15.84:0/2142265227 wait complete. 2023-10-16T09:50:23.140 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T09:50:23.201 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:23.200+0000 7f0681212040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T09:50:23.225 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:23.223+0000 7f5e5203a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T09:50:23.255 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:23.254+0000 7f0681212040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T09:50:23.282 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:23.280+0000 7f5e5203a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T09:50:23.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.336+0000 7f9a98f0a640 1 Processor -- start 2023-10-16T09:50:23.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.336+0000 7f9a98f0a640 1 -- start start 2023-10-16T09:50:23.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.337+0000 7f9a98f0a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415cd70 0x7f9a940d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:23.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.337+0000 7f9a98f0a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9a940cded0 0x7f9a940b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:23.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.337+0000 7f9a98f0a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a9415d150 con 0x7f9a940b8480 2023-10-16T09:50:23.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.337+0000 7f9a98f0a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a940d1180 con 0x7f9a940cded0 2023-10-16T09:50:23.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.337+0000 7f9a98f0a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940d1460 con 0x7f9a9415cd70 2023-10-16T09:50:23.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.338+0000 7f9a93fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9a940cded0 0x7f9a940b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:23.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.338+0000 7f9a93fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9a940cded0 0x7f9a940b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37754/0 (socket says 172.21.15.84:37754) 2023-10-16T09:50:23.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.338+0000 7f9a92ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9a940b8480 0x7f9a940cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33644/0 (socket says 172.21.15.84:33644) 2023-10-16T09:50:23.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.338+0000 7f9a93fff640 1 -- 172.21.15.84:0/2987341504 learned_addr learned my addr 172.21.15.84:0/2987341504 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:23.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.338+0000 7f9a937fe640 1 --2- 172.21.15.84:0/2987341504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415cd70 0x7f9a940d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:23.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.338+0000 7f9a93fff640 1 -- 172.21.15.84:0/2987341504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415cd70 msgr2=0x7f9a940d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:23.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.338+0000 7f9a93fff640 1 --2- 172.21.15.84:0/2987341504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415cd70 0x7f9a940d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.338+0000 7f9a93fff640 1 -- 172.21.15.84:0/2987341504 >> v1:172.21.15.84:6789/0 conn(0x7f9a940b8480 legacy=0x7f9a940cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:23.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.338+0000 7f9a927fc640 1 -- 172.21.15.84:0/2987341504 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1609755749 0 0) 0x7f9a9415d150 con 0x7f9a940b8480 2023-10-16T09:50:23.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.338+0000 7f9a93fff640 1 -- 172.21.15.84:0/2987341504 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a940d1740 con 0x7f9a940cded0 2023-10-16T09:50:23.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.339+0000 7f9a93fff640 1 --2- 172.21.15.84:0/2987341504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a940cded0 0x7f9a940b5c90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009e50 tx=0x7f9a8c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959f480641e88a66 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:23.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.340+0000 7f9a927fc640 1 -- 172.21.15.84:0/2987341504 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c02cb00 con 0x7f9a940cded0 2023-10-16T09:50:23.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.340+0000 7f9a927fc640 1 -- 172.21.15.84:0/2987341504 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c02cc60 con 0x7f9a940cded0 2023-10-16T09:50:23.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.340+0000 7f9a927fc640 1 -- 172.21.15.84:0/2987341504 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c0343f0 con 0x7f9a940cded0 2023-10-16T09:50:23.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.340+0000 7f9a98f0a640 1 -- 172.21.15.84:0/2987341504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a940cded0 msgr2=0x7f9a940b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:23.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.340+0000 7f9a98f0a640 1 --2- 172.21.15.84:0/2987341504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a940cded0 0x7f9a940b5c90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009e50 tx=0x7f9a8c02bf60 comp rx=0 tx=0).stop 2023-10-16T09:50:23.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.340+0000 7f9a98f0a640 1 -- 172.21.15.84:0/2987341504 shutdown_connections 2023-10-16T09:50:23.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.340+0000 7f9a98f0a640 1 --2- 172.21.15.84:0/2987341504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a940cded0 0x7f9a940b5c90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.340+0000 7f9a98f0a640 1 --2- 172.21.15.84:0/2987341504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415cd70 0x7f9a940d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.340+0000 7f9a98f0a640 1 -- 172.21.15.84:0/2987341504 >> 172.21.15.84:0/2987341504 conn(0x7f9a940cf570 msgr2=0x7f9a940cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:23.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.340+0000 7f9a98f0a640 1 -- 172.21.15.84:0/2987341504 shutdown_connections 2023-10-16T09:50:23.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.340+0000 7f9a98f0a640 1 -- 172.21.15.84:0/2987341504 wait complete. 2023-10-16T09:50:23.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.342+0000 7f9a98f0a640 1 Processor -- start 2023-10-16T09:50:23.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.342+0000 7f9a98f0a640 1 -- start start 2023-10-16T09:50:23.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.343+0000 7f9a98f0a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a940b8480 0x7f9a9415f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:23.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.343+0000 7f9a98f0a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a940cded0 0x7f9a9415fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:23.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.343+0000 7f9a98f0a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415cd70 0x7f9a941e5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:23.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.343+0000 7f9a98f0a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a941428b0 con 0x7f9a940b8480 2023-10-16T09:50:23.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.343+0000 7f9a98f0a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940c9f30 con 0x7f9a940cded0 2023-10-16T09:50:23.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.343+0000 7f9a98f0a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940ca210 con 0x7f9a9415cd70 2023-10-16T09:50:23.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.343+0000 7f9a937fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a940b8480 0x7f9a9415f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:23.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.343+0000 7f9a937fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a940b8480 0x7f9a9415f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37764/0 (socket says 172.21.15.84:37764) 2023-10-16T09:50:23.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.343+0000 7f9a937fe640 1 -- 172.21.15.84:0/649474895 learned_addr learned my addr 172.21.15.84:0/649474895 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:23.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.344+0000 7f9a93fff640 1 --2- 172.21.15.84:0/649474895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415cd70 0x7f9a941e5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:23.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.344+0000 7f9a92ffd640 1 --2- 172.21.15.84:0/649474895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a940cded0 0x7f9a9415fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:23.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.344+0000 7f9a937fe640 1 -- 172.21.15.84:0/649474895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415cd70 msgr2=0x7f9a941e5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:23.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.344+0000 7f9a937fe640 1 --2- 172.21.15.84:0/649474895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415cd70 0x7f9a941e5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.344+0000 7f9a937fe640 1 -- 172.21.15.84:0/649474895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a940cded0 msgr2=0x7f9a9415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:23.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.344+0000 7f9a937fe640 1 --2- 172.21.15.84:0/649474895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a940cded0 0x7f9a9415fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.344+0000 7f9a937fe640 1 -- 172.21.15.84:0/649474895 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8c009b30 con 0x7f9a940b8480 2023-10-16T09:50:23.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.344+0000 7f9a93fff640 1 --2- 172.21.15.84:0/649474895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415cd70 0x7f9a941e5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:23.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.344+0000 7f9a937fe640 1 --2- 172.21.15.84:0/649474895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a940b8480 0x7f9a9415f800 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00d800 tx=0x7f9a7c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:23.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.344+0000 7f9a8bfff640 1 -- 172.21.15.84:0/649474895 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c00b6c0 con 0x7f9a940b8480 2023-10-16T09:50:23.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.344+0000 7f9a8bfff640 1 -- 172.21.15.84:0/649474895 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c00b820 con 0x7f9a940b8480 2023-10-16T09:50:23.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.344+0000 7f9a98f0a640 1 -- 172.21.15.84:0/649474895 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a94142b00 con 0x7f9a940b8480 2023-10-16T09:50:23.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.345+0000 7f9a8bfff640 1 -- 172.21.15.84:0/649474895 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c00bbb0 con 0x7f9a940b8480 2023-10-16T09:50:23.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.345+0000 7f9a98f0a640 1 -- 172.21.15.84:0/649474895 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a94160280 con 0x7f9a940b8480 2023-10-16T09:50:23.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.346+0000 7f9a98f0a640 1 -- 172.21.15.84:0/649474895 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a940ca430 con 0x7f9a940b8480 2023-10-16T09:50:23.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.347+0000 7f9a8bfff640 1 -- 172.21.15.84:0/649474895 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 6) v1 ==== 89246+0+0 (secure 0 0 0) 0x7f9a7c00bd10 con 0x7f9a940b8480 2023-10-16T09:50:23.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.348+0000 7f9a8bfff640 1 --2- 172.21.15.84:0/649474895 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9a7406aef0 0x7f9a7406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:23.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.348+0000 7f9a8bfff640 1 -- 172.21.15.84:0/649474895 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a7c089c40 con 0x7f9a940b8480 2023-10-16T09:50:23.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.348+0000 7f9a92ffd640 1 -- 172.21.15.84:0/649474895 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9a7406aef0 msgr2=0x7f9a7406d3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3010246666 2023-10-16T09:50:23.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.348+0000 7f9a92ffd640 1 --2- 172.21.15.84:0/649474895 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9a7406aef0 0x7f9a7406d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:50:23.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.353+0000 7f9a8bfff640 1 -- 172.21.15.84:0/649474895 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9a7c054c60 con 0x7f9a940b8480 2023-10-16T09:50:23.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.533+0000 7f9a98f0a640 1 -- 172.21.15.84:0/649474895 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f9a9404e9b0 con 0x7f9a940b8480 2023-10-16T09:50:23.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.540+0000 7f9a8bfff640 1 -- 172.21.15.84:0/649474895 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9a7c058280 con 0x7f9a940b8480 2023-10-16T09:50:23.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.543+0000 7f9a98f0a640 1 -- 172.21.15.84:0/649474895 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9a7406aef0 msgr2=0x7f9a7406d3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:50:23.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.543+0000 7f9a98f0a640 1 --2- 172.21.15.84:0/649474895 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9a7406aef0 0x7f9a7406d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.543+0000 7f9a98f0a640 1 -- 172.21.15.84:0/649474895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a940b8480 msgr2=0x7f9a9415f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:23.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.543+0000 7f9a98f0a640 1 --2- 172.21.15.84:0/649474895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a940b8480 0x7f9a9415f800 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00d800 tx=0x7f9a7c00dcc0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.543+0000 7f9a98f0a640 1 -- 172.21.15.84:0/649474895 shutdown_connections 2023-10-16T09:50:23.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.543+0000 7f9a98f0a640 1 --2- 172.21.15.84:0/649474895 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f9a7406aef0 0x7f9a7406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.543+0000 7f9a98f0a640 1 --2- 172.21.15.84:0/649474895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415cd70 0x7f9a941e5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.543+0000 7f9a98f0a640 1 --2- 172.21.15.84:0/649474895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a940cded0 0x7f9a9415fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.543+0000 7f9a98f0a640 1 --2- 172.21.15.84:0/649474895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a940b8480 0x7f9a9415f800 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.543+0000 7f9a98f0a640 1 -- 172.21.15.84:0/649474895 >> 172.21.15.84:0/649474895 conn(0x7f9a940cf570 msgr2=0x7f9a9415bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:23.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.543+0000 7f9a98f0a640 1 -- 172.21.15.84:0/649474895 shutdown_connections 2023-10-16T09:50:23.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.543+0000 7f9a98f0a640 1 -- 172.21.15.84:0/649474895 wait complete. 2023-10-16T09:50:23.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:23.563 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-16T09:50:23.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:23.621 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-10-16T09:50:23.671 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir0 1m 2023-10-16T09:50:23.713 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:23.712+0000 7f0681212040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T09:50:23.752 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:23.751+0000 7f5e5203a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T09:50:23.766 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:23.764+0000 7f0681212040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T09:50:23.807 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:23.806+0000 7f5e5203a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T09:50:23.817 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:23.815+0000 7f0681212040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T09:50:23.860 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:23.858+0000 7f5e5203a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T09:50:23.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.868+0000 7f0f34f78640 1 Processor -- start 2023-10-16T09:50:23.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.869+0000 7f0f34f78640 1 -- start start 2023-10-16T09:50:23.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.870+0000 7f0f34f78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f3015d030 0x7f0f300d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:23.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.870+0000 7f0f34f78640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0f3015d410 0x7f0f30161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:23.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.870+0000 7f0f34f78640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0f30161990 0x7f0f300cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:23.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.870+0000 7f0f34f78640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f300d2b50 con 0x7f0f30161990 2023-10-16T09:50:23.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.870+0000 7f0f34f78640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f300b5d90 con 0x7f0f3015d410 2023-10-16T09:50:23.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.870+0000 7f0f34f78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f30160fc0 con 0x7f0f3015d030 2023-10-16T09:50:23.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.870+0000 7f0f2ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0f30161990 0x7f0f300cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:23.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.870+0000 7f0f2ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0f30161990 0x7f0f300cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37776/0 (socket says 172.21.15.84:37776) 2023-10-16T09:50:23.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.870+0000 7f0f2ed9d640 1 -- 172.21.15.84:0/3086623112 learned_addr learned my addr 172.21.15.84:0/3086623112 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:23.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.871+0000 7f0f27fff640 1 --2- 172.21.15.84:0/3086623112 >> v2:172.21.15.157:3300/0 conn(0x7f0f3015d410 0x7f0f30161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:23.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.871+0000 7f0f2e59c640 1 --2- 172.21.15.84:0/3086623112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f3015d030 0x7f0f300d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:23.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.871+0000 7f0f2ed9d640 1 -- 172.21.15.84:0/3086623112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f3015d030 msgr2=0x7f0f300d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:23.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.871+0000 7f0f2ed9d640 1 --2- 172.21.15.84:0/3086623112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f3015d030 0x7f0f300d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.871+0000 7f0f2ed9d640 1 -- 172.21.15.84:0/3086623112 >> v2:172.21.15.157:3300/0 conn(0x7f0f3015d410 msgr2=0x7f0f30161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:23.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.871+0000 7f0f2ed9d640 1 --2- 172.21.15.84:0/3086623112 >> v2:172.21.15.157:3300/0 conn(0x7f0f3015d410 0x7f0f30161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.871+0000 7f0f2ed9d640 1 -- 172.21.15.84:0/3086623112 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f300d25d0 con 0x7f0f30161990 2023-10-16T09:50:23.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.871+0000 7f0f27fff640 1 --2- 172.21.15.84:0/3086623112 >> v2:172.21.15.157:3300/0 conn(0x7f0f3015d410 0x7f0f30161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:23.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.872+0000 7f0f2ed9d640 1 --2- 172.21.15.84:0/3086623112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f30161990 0x7f0f300cd260 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0f30073760 tx=0x7f0f2802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b80825db5de3439 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:23.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.873+0000 7f0f2dd9b640 1 -- 172.21.15.84:0/3086623112 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2802cb00 con 0x7f0f30161990 2023-10-16T09:50:23.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.873+0000 7f0f2dd9b640 1 -- 172.21.15.84:0/3086623112 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2802cc60 con 0x7f0f30161990 2023-10-16T09:50:23.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.873+0000 7f0f2dd9b640 1 -- 172.21.15.84:0/3086623112 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f280343f0 con 0x7f0f30161990 2023-10-16T09:50:23.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.873+0000 7f0f34f78640 1 -- 172.21.15.84:0/3086623112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f30161990 msgr2=0x7f0f300cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:23.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.873+0000 7f0f34f78640 1 --2- 172.21.15.84:0/3086623112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f30161990 0x7f0f300cd260 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0f30073760 tx=0x7f0f2802bf60 comp rx=0 tx=0).stop 2023-10-16T09:50:23.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.874+0000 7f0f34f78640 1 -- 172.21.15.84:0/3086623112 shutdown_connections 2023-10-16T09:50:23.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.874+0000 7f0f34f78640 1 --2- 172.21.15.84:0/3086623112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f30161990 0x7f0f300cd260 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.874+0000 7f0f34f78640 1 --2- 172.21.15.84:0/3086623112 >> v2:172.21.15.157:3300/0 conn(0x7f0f3015d410 0x7f0f30161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.874+0000 7f0f34f78640 1 --2- 172.21.15.84:0/3086623112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f3015d030 0x7f0f300d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.874+0000 7f0f34f78640 1 -- 172.21.15.84:0/3086623112 >> 172.21.15.84:0/3086623112 conn(0x7f0f300cf720 msgr2=0x7f0f300cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:23.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.874+0000 7f0f34f78640 1 -- 172.21.15.84:0/3086623112 shutdown_connections 2023-10-16T09:50:23.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.874+0000 7f0f34f78640 1 -- 172.21.15.84:0/3086623112 wait complete. 2023-10-16T09:50:23.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.875+0000 7f0f34f78640 1 Processor -- start 2023-10-16T09:50:23.880 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:23.876+0000 7f0681212040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T09:50:23.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.876+0000 7f0f34f78640 1 -- start start 2023-10-16T09:50:23.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.876+0000 7f0f34f78640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f3015d030 0x7f0f301f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:23.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.876+0000 7f0f34f78640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f3015d410 0x7f0f300cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:23.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.876+0000 7f0f34f78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f30161990 0x7f0f3015e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:23.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.876+0000 7f0f34f78640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3014dc30 con 0x7f0f3015d410 2023-10-16T09:50:23.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.876+0000 7f0f34f78640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f300cb4c0 con 0x7f0f3015d030 2023-10-16T09:50:23.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.876+0000 7f0f34f78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f300cb7a0 con 0x7f0f30161990 2023-10-16T09:50:23.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.877+0000 7f0f27fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f3015d410 0x7f0f300cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:23.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.877+0000 7f0f27fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f3015d410 0x7f0f300cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37780/0 (socket says 172.21.15.84:37780) 2023-10-16T09:50:23.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.877+0000 7f0f27fff640 1 -- 172.21.15.84:0/1510360250 learned_addr learned my addr 172.21.15.84:0/1510360250 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:23.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.877+0000 7f0f2e59c640 1 --2- 172.21.15.84:0/1510360250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f3015d030 0x7f0f301f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:23.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.877+0000 7f0f2ed9d640 1 --2- 172.21.15.84:0/1510360250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f30161990 0x7f0f3015e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:23.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.877+0000 7f0f27fff640 1 -- 172.21.15.84:0/1510360250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f30161990 msgr2=0x7f0f3015e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:23.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.877+0000 7f0f27fff640 1 --2- 172.21.15.84:0/1510360250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f30161990 0x7f0f3015e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.877+0000 7f0f27fff640 1 -- 172.21.15.84:0/1510360250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f3015d030 msgr2=0x7f0f301f24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:23.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.877+0000 7f0f27fff640 1 --2- 172.21.15.84:0/1510360250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f3015d030 0x7f0f301f24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:23.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.877+0000 7f0f27fff640 1 -- 172.21.15.84:0/1510360250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f28009b30 con 0x7f0f3015d410 2023-10-16T09:50:23.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.877+0000 7f0f27fff640 1 --2- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f3015d410 0x7f0f300cab50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800c7f0 tx=0x7f0f1800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:23.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.878+0000 7f0f26ffd640 1 -- 172.21.15.84:0/1510360250 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f18007820 con 0x7f0f3015d410 2023-10-16T09:50:23.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.878+0000 7f0f26ffd640 1 -- 172.21.15.84:0/1510360250 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f18007980 con 0x7f0f3015d410 2023-10-16T09:50:23.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.878+0000 7f0f26ffd640 1 -- 172.21.15.84:0/1510360250 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f18007cf0 con 0x7f0f3015d410 2023-10-16T09:50:23.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.878+0000 7f0f34f78640 1 -- 172.21.15.84:0/1510360250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f300cba80 con 0x7f0f3015d410 2023-10-16T09:50:23.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.878+0000 7f0f34f78640 1 -- 172.21.15.84:0/1510360250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f300cc040 con 0x7f0f3015d410 2023-10-16T09:50:23.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.879+0000 7f0f34f78640 1 -- 172.21.15.84:0/1510360250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f300cc200 con 0x7f0f3015d410 2023-10-16T09:50:23.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.880+0000 7f0f26ffd640 1 -- 172.21.15.84:0/1510360250 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 6) v1 ==== 89246+0+0 (secure 0 0 0) 0x7f0f180113f0 con 0x7f0f3015d410 2023-10-16T09:50:23.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.881+0000 7f0f26ffd640 1 --2- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f1006aec0 0x7f0f1006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:23.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.881+0000 7f0f2e59c640 1 -- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f1006aec0 msgr2=0x7f0f1006d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3010246666 2023-10-16T09:50:23.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.881+0000 7f0f26ffd640 1 -- 172.21.15.84:0/1510360250 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f18089a20 con 0x7f0f3015d410 2023-10-16T09:50:23.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.881+0000 7f0f2e59c640 1 --2- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f1006aec0 0x7f0f1006d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:50:23.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:23.886+0000 7f0f26ffd640 1 -- 172.21.15.84:0/1510360250 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0f18054a30 con 0x7f0f3015d410 2023-10-16T09:50:23.925 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:23.924+0000 7f5e5203a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T09:50:23.930 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:23.927+0000 7f0681212040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T09:50:23.975 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:23.974+0000 7f5e5203a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T09:50:24.040 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:24.038+0000 7f0681212040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T09:50:24.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:24.082+0000 7f0f2e59c640 1 -- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f1006aec0 msgr2=0x7f0f1006d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3010246666 2023-10-16T09:50:24.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:24.082+0000 7f0f2e59c640 1 --2- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f1006aec0 0x7f0f1006d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:50:24.088 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:24.087+0000 7f5e5203a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T09:50:24.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:24.088+0000 7f0f34f78640 1 -- 172.21.15.84:0/1510360250 --> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f300d2870 con 0x7f0f1006aec0 2023-10-16T09:50:24.102 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:24.100+0000 7f0681212040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T09:50:24.152 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:24.150+0000 7f0681212040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T09:50:24.154 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:24.152+0000 7f5e5203a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T09:50:24.204 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:24.202+0000 7f5e5203a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T09:50:24.210 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:24.208+0000 7f0681212040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T09:50:24.260 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:24.259+0000 7f0681212040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T09:50:24.261 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:24.259+0000 7f5e5203a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T09:50:24.311 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:24.310+0000 7f5e5203a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T09:50:24.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:24.483+0000 7f0f2e59c640 1 -- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f1006aec0 msgr2=0x7f0f1006d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3010246666 2023-10-16T09:50:24.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:24.483+0000 7f0f2e59c640 1 --2- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f1006aec0 0x7f0f1006d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T09:50:24.645 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:24.643+0000 7f0681212040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T09:50:24.697 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:24.696+0000 7f5e5203a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T09:50:24.805 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:24.803+0000 7f0681212040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T09:50:24.857 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:24.855+0000 7f5e5203a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T09:50:24.914 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:24.912+0000 7f0681212040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T09:50:24.964 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:24.962+0000 7f0681212040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T09:50:24.965 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:24.963+0000 7f5e5203a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T09:50:25.016 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:25.014+0000 7f5e5203a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T09:50:25.072 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:25.070+0000 7f0681212040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:50:25.124 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:25.122+0000 7f5e5203a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:50:25.179 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:25.177+0000 7f0681212040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T09:50:25.232 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:25.230+0000 7f5e5203a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T09:50:25.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:25.284+0000 7f0f2e59c640 1 -- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f1006aec0 msgr2=0x7f0f1006d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3010246666 2023-10-16T09:50:25.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:25.284+0000 7f0f2e59c640 1 --2- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f1006aec0 0x7f0f1006d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T09:50:25.337 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:25.335+0000 7f0681212040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:50:25.385 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:25.384+0000 7f0681212040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T09:50:25.392 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:25.390+0000 7f5e5203a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:50:25.441 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:25.439+0000 7f5e5203a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T09:50:25.504 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:25.502+0000 7f0681212040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T09:50:25.560 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:25.558+0000 7f5e5203a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T09:50:26.315 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:26.313+0000 7f0681212040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T09:50:26.377 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:26.375+0000 7f5e5203a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T09:50:26.610 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:26.609+0000 7f0681212040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T09:50:26.667 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:26.665+0000 7f0681212040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T09:50:26.671 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:26.669+0000 7f5e5203a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T09:50:26.728 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:26.726+0000 7f5e5203a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T09:50:26.781 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T09:50:26.780+0000 7f0681212040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T09:50:26.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:26.850+0000 7f0f26ffd640 1 -- 172.21.15.84:0/1510360250 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 7) v1 ==== 89246+0+0 (secure 0 0 0) 0x7f0f18054e00 con 0x7f0f3015d410 2023-10-16T09:50:26.879 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:26.878+0000 7f5e5203a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T09:50:26.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:26.886+0000 7f0f2e59c640 1 -- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f1006aec0 msgr2=0x7f0f1006d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3010246666 2023-10-16T09:50:26.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:26.886+0000 7f0f2e59c640 1 --2- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f1006aec0 0x7f0f1006d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T09:50:26.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:26.899+0000 7f0f26ffd640 1 -- 172.21.15.84:0/1510360250 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 8) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f0f18060560 con 0x7f0f3015d410 2023-10-16T09:50:26.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:26.899+0000 7f0f26ffd640 1 -- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f1006aec0 msgr2=0x7f0f1006d380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:50:26.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:26.899+0000 7f0f26ffd640 1 --2- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:6832/3010246666,v1:172.21.15.84:6833/3010246666] conn(0x7f0f1006aec0 0x7f0f1006d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:27.229 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:27.227+0000 7f5dd132b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:50:27.230 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:27.227+0000 7f5dd132b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:50:27.230 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:27.227+0000 7f5dd132b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:50:27.230 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:27.227+0000 7f5dd132b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:50:27.230 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:27.227+0000 7f5dd132b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:50:27.230 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:27.227+0000 7f5dceae6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:50:27.231 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:27.227+0000 7f5dceae6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:50:27.231 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:27.227+0000 7f5dceae6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:50:27.231 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:27.227+0000 7f5dceae6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:50:27.231 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:50:27.227+0000 7f5dceae6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:50:27.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:27.911+0000 7f0f26ffd640 1 -- 172.21.15.84:0/1510360250 <== mon.0 v2:172.21.15.84:3300/0 9 ==== mgrmap(e 9) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f0f180556d0 con 0x7f0f3015d410 2023-10-16T09:50:27.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:27.912+0000 7f0f26ffd640 1 --2- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f0f100276c0 0x7f0f10027aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:27.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:27.912+0000 7f0f26ffd640 1 -- 172.21.15.84:0/1510360250 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f10027fe0 con 0x7f0f100276c0 2023-10-16T09:50:27.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:27.916+0000 7f0f2e59c640 1 --2- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f0f100276c0 0x7f0f10027aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:27.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:27.921+0000 7f0f2e59c640 1 --2- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f0f100276c0 0x7f0f10027aa0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0f20007040 tx=0x7f0f20008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:28.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.128+0000 7f0f26ffd640 1 -- 172.21.15.84:0/1510360250 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f0f10027fe0 con 0x7f0f100276c0 2023-10-16T09:50:28.131 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path /snap_test_dir1/dir0 2023-10-16T09:50:28.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.132+0000 7f0f34f78640 1 -- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f0f100276c0 msgr2=0x7f0f10027aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:28.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.132+0000 7f0f34f78640 1 --2- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f0f100276c0 0x7f0f10027aa0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0f20007040 tx=0x7f0f20008040 comp rx=0 tx=0).stop 2023-10-16T09:50:28.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.132+0000 7f0f34f78640 1 -- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f3015d410 msgr2=0x7f0f300cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:28.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.132+0000 7f0f34f78640 1 --2- 172.21.15.84:0/1510360250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f3015d410 0x7f0f300cab50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800c7f0 tx=0x7f0f1800ccb0 comp rx=0 tx=0).stop 2023-10-16T09:50:28.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.132+0000 7f0f2ed9d640 1 -- 172.21.15.84:0/1510360250 reap_dead start 2023-10-16T09:50:28.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.132+0000 7f0f34f78640 1 -- 172.21.15.84:0/1510360250 shutdown_connections 2023-10-16T09:50:28.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.132+0000 7f0f34f78640 1 -- 172.21.15.84:0/1510360250 >> 172.21.15.84:0/1510360250 conn(0x7f0f300cf720 msgr2=0x7f0f300b5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:28.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.132+0000 7f0f34f78640 1 -- 172.21.15.84:0/1510360250 shutdown_connections 2023-10-16T09:50:28.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.132+0000 7f0f34f78640 1 -- 172.21.15.84:0/1510360250 wait complete. 2023-10-16T09:50:28.154 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-10-16T09:50:28.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:28.155 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-10-16T09:50:28.209 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir1 1m 2023-10-16T09:50:28.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.419+0000 7fd3940c2640 1 Processor -- start 2023-10-16T09:50:28.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.419+0000 7fd3940c2640 1 -- start start 2023-10-16T09:50:28.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.421+0000 7fd3940c2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd38c159e90 0x7fd38c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:28.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.421+0000 7fd3940c2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd38c06c140 0x7fd38c06c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:28.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.421+0000 7fd3940c2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c06cad0 con 0x7fd38c159ab0 2023-10-16T09:50:28.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.421+0000 7fd3940c2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c15d890 con 0x7fd38c06c140 2023-10-16T09:50:28.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.421+0000 7fd3940c2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c070360 con 0x7fd38c159e90 2023-10-16T09:50:28.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.421+0000 7fd391e5e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd38c159ab0 0x7fd38c06fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37890/0 (socket says 172.21.15.84:37890) 2023-10-16T09:50:28.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.421+0000 7fd391e5e640 1 -- 172.21.15.84:0/3065715649 learned_addr learned my addr 172.21.15.84:0/3065715649 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:28.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.421+0000 7fd39265f640 1 --2- 172.21.15.84:0/3065715649 >> v2:172.21.15.84:3300/0 conn(0x7fd38c06c140 0x7fd38c06c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:28.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.421+0000 7fd39165d640 1 --2- 172.21.15.84:0/3065715649 >> v2:172.21.15.157:3300/0 conn(0x7fd38c159e90 0x7fd38c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:28.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.421+0000 7fd39265f640 1 -- 172.21.15.84:0/3065715649 >> v2:172.21.15.157:3300/0 conn(0x7fd38c159e90 msgr2=0x7fd38c0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:28.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.421+0000 7fd39265f640 1 --2- 172.21.15.84:0/3065715649 >> v2:172.21.15.157:3300/0 conn(0x7fd38c159e90 0x7fd38c0ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:28.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.421+0000 7fd39265f640 1 -- 172.21.15.84:0/3065715649 >> v1:172.21.15.84:6789/0 conn(0x7fd38c159ab0 legacy=0x7fd38c06fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:28.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.421+0000 7fd39265f640 1 -- 172.21.15.84:0/3065715649 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c0cd1f0 con 0x7fd38c06c140 2023-10-16T09:50:28.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.423+0000 7fd39265f640 1 --2- 172.21.15.84:0/3065715649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd38c06c140 0x7fd38c06c590 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd38c06d900 tx=0x7fd38402bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca553ec21bf168f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:28.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.423+0000 7fd390e5c640 1 -- 172.21.15.84:0/3065715649 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38402cb50 con 0x7fd38c06c140 2023-10-16T09:50:28.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.423+0000 7fd390e5c640 1 -- 172.21.15.84:0/3065715649 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38402ccb0 con 0x7fd38c06c140 2023-10-16T09:50:28.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.423+0000 7fd390e5c640 1 -- 172.21.15.84:0/3065715649 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384034660 con 0x7fd38c06c140 2023-10-16T09:50:28.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.423+0000 7fd3940c2640 1 -- 172.21.15.84:0/3065715649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd38c06c140 msgr2=0x7fd38c06c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:28.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.423+0000 7fd3940c2640 1 --2- 172.21.15.84:0/3065715649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd38c06c140 0x7fd38c06c590 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd38c06d900 tx=0x7fd38402bda0 comp rx=0 tx=0).stop 2023-10-16T09:50:28.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.424+0000 7fd3940c2640 1 -- 172.21.15.84:0/3065715649 shutdown_connections 2023-10-16T09:50:28.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.424+0000 7fd3940c2640 1 --2- 172.21.15.84:0/3065715649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd38c06c140 0x7fd38c06c590 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:28.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.424+0000 7fd3940c2640 1 --2- 172.21.15.84:0/3065715649 >> v2:172.21.15.157:3300/0 conn(0x7fd38c159e90 0x7fd38c0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:28.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.424+0000 7fd3940c2640 1 -- 172.21.15.84:0/3065715649 >> 172.21.15.84:0/3065715649 conn(0x7fd38c0ce9b0 msgr2=0x7fd38c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:28.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.424+0000 7fd3940c2640 1 -- 172.21.15.84:0/3065715649 shutdown_connections 2023-10-16T09:50:28.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.424+0000 7fd3940c2640 1 -- 172.21.15.84:0/3065715649 wait complete. 2023-10-16T09:50:28.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.425+0000 7fd3940c2640 1 Processor -- start 2023-10-16T09:50:28.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.426+0000 7fd3940c2640 1 -- start start 2023-10-16T09:50:28.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.427+0000 7fd3940c2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd38c06c140 0x7fd38c1f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:28.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.427+0000 7fd3940c2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd38c159ab0 0x7fd38c1f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:28.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.427+0000 7fd3940c2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c159e90 0x7fd38c0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:28.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.427+0000 7fd3940c2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c1e29f0 con 0x7fd38c06c140 2023-10-16T09:50:28.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.427+0000 7fd3940c2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c1f3940 con 0x7fd38c159e90 2023-10-16T09:50:28.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.427+0000 7fd3940c2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0c1f50 con 0x7fd38c159ab0 2023-10-16T09:50:28.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.427+0000 7fd391e5e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd38c06c140 0x7fd38c1f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:28.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.427+0000 7fd391e5e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd38c06c140 0x7fd38c1f29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40554/0 (socket says 172.21.15.84:40554) 2023-10-16T09:50:28.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.427+0000 7fd391e5e640 1 -- 172.21.15.84:0/879054936 learned_addr learned my addr 172.21.15.84:0/879054936 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:28.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.427+0000 7fd391e5e640 1 -- 172.21.15.84:0/879054936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd38c159ab0 msgr2=0x7fd38c1f6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:28.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.427+0000 7fd391e5e640 1 --2- 172.21.15.84:0/879054936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd38c159ab0 0x7fd38c1f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:28.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.427+0000 7fd391e5e640 1 -- 172.21.15.84:0/879054936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c159e90 msgr2=0x7fd38c0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:28.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.427+0000 7fd391e5e640 1 --2- 172.21.15.84:0/879054936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c159e90 0x7fd38c0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:28.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.427+0000 7fd391e5e640 1 -- 172.21.15.84:0/879054936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd384009a70 con 0x7fd38c06c140 2023-10-16T09:50:28.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.428+0000 7fd391e5e640 1 --2- 172.21.15.84:0/879054936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd38c06c140 0x7fd38c1f29c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd38800c4d0 tx=0x7fd38800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:28.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.428+0000 7fd3827fc640 1 -- 172.21.15.84:0/879054936 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388005ce0 con 0x7fd38c06c140 2023-10-16T09:50:28.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.428+0000 7fd3827fc640 1 -- 172.21.15.84:0/879054936 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd388005e40 con 0x7fd38c06c140 2023-10-16T09:50:28.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.428+0000 7fd3827fc640 1 -- 172.21.15.84:0/879054936 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388007270 con 0x7fd38c06c140 2023-10-16T09:50:28.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.428+0000 7fd3940c2640 1 -- 172.21.15.84:0/879054936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38c1f30a0 con 0x7fd38c06c140 2023-10-16T09:50:28.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.428+0000 7fd3940c2640 1 -- 172.21.15.84:0/879054936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c1f3660 con 0x7fd38c06c140 2023-10-16T09:50:28.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.429+0000 7fd3940c2640 1 -- 172.21.15.84:0/879054936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c04e160 con 0x7fd38c06c140 2023-10-16T09:50:28.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.430+0000 7fd3827fc640 1 -- 172.21.15.84:0/879054936 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 9) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fd3880073d0 con 0x7fd38c06c140 2023-10-16T09:50:28.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.431+0000 7fd3827fc640 1 --2- 172.21.15.84:0/879054936 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd37006ad10 0x7fd37006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:28.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.431+0000 7fd3827fc640 1 -- 172.21.15.84:0/879054936 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fd3880885b0 con 0x7fd38c06c140 2023-10-16T09:50:28.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.431+0000 7fd39165d640 1 --2- 172.21.15.84:0/879054936 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd37006ad10 0x7fd37006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:28.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.432+0000 7fd39165d640 1 --2- 172.21.15.84:0/879054936 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd37006ad10 0x7fd37006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd37c006fd0 tx=0x7fd37c008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:28.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.436+0000 7fd3827fc640 1 -- 172.21.15.84:0/879054936 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd388053950 con 0x7fd38c06c140 2023-10-16T09:50:28.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.655+0000 7fd3940c2640 1 -- 172.21.15.84:0/879054936 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd38c0cd7b0 con 0x7fd37006ad10 2023-10-16T09:50:28.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.833+0000 7fd3827fc640 1 -- 172.21.15.84:0/879054936 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fd37c00ae20 con 0x7fd37006ad10 2023-10-16T09:50:28.835 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path /snap_test_dir1/dir1 2023-10-16T09:50:28.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.836+0000 7fd3940c2640 1 -- 172.21.15.84:0/879054936 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd37006ad10 msgr2=0x7fd37006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:28.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.836+0000 7fd3940c2640 1 --2- 172.21.15.84:0/879054936 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd37006ad10 0x7fd37006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd37c006fd0 tx=0x7fd37c008040 comp rx=0 tx=0).stop 2023-10-16T09:50:28.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.836+0000 7fd3940c2640 1 -- 172.21.15.84:0/879054936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd38c06c140 msgr2=0x7fd38c1f29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:28.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.836+0000 7fd3940c2640 1 --2- 172.21.15.84:0/879054936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd38c06c140 0x7fd38c1f29c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd38800c4d0 tx=0x7fd38800c990 comp rx=0 tx=0).stop 2023-10-16T09:50:28.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.836+0000 7fd3940c2640 1 -- 172.21.15.84:0/879054936 shutdown_connections 2023-10-16T09:50:28.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.836+0000 7fd3940c2640 1 --2- 172.21.15.84:0/879054936 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd37006ad10 0x7fd37006d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:28.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.836+0000 7fd3940c2640 1 --2- 172.21.15.84:0/879054936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c159e90 0x7fd38c0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:28.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.836+0000 7fd3940c2640 1 --2- 172.21.15.84:0/879054936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd38c159ab0 0x7fd38c1f6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:28.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.836+0000 7fd3940c2640 1 --2- 172.21.15.84:0/879054936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd38c06c140 0x7fd38c1f29c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:28.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.836+0000 7fd3940c2640 1 -- 172.21.15.84:0/879054936 >> 172.21.15.84:0/879054936 conn(0x7fd38c0ce9b0 msgr2=0x7fd38c0b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:28.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.836+0000 7fd3940c2640 1 -- 172.21.15.84:0/879054936 shutdown_connections 2023-10-16T09:50:28.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:28.836+0000 7fd3940c2640 1 -- 172.21.15.84:0/879054936 wait complete. 2023-10-16T09:50:28.857 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-10-16T09:50:28.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:28.858 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-10-16T09:50:28.921 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir2 1m 2023-10-16T09:50:29.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.122+0000 7fb7b3733640 1 Processor -- start 2023-10-16T09:50:29.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.122+0000 7fb7b3733640 1 -- start start 2023-10-16T09:50:29.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.123+0000 7fb7b3733640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb7ac15d330 0x7fb7ac1618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:29.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.123+0000 7fb7b3733640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7ac0661c0 0x7fb7ac0ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:29.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.123+0000 7fb7b3733640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7ac0d0890 con 0x7fb7ac06c7d0 2023-10-16T09:50:29.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.123+0000 7fb7b3733640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7ac06b1e0 con 0x7fb7ac15d330 2023-10-16T09:50:29.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.123+0000 7fb7b3733640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac0d05b0 con 0x7fb7ac0661c0 2023-10-16T09:50:29.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.124+0000 7fb7b14cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb7ac15d330 0x7fb7ac1618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:29.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.124+0000 7fb7b14cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb7ac15d330 0x7fb7ac1618b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40578/0 (socket says 172.21.15.84:40578) 2023-10-16T09:50:29.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.124+0000 7fb7b14cf640 1 -- 172.21.15.84:0/3236796200 learned_addr learned my addr 172.21.15.84:0/3236796200 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:29.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.124+0000 7fb7b1cd0640 1 --2- 172.21.15.84:0/3236796200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7ac0661c0 0x7fb7ac0ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:29.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.124+0000 7fb7b14cf640 1 -- 172.21.15.84:0/3236796200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7ac0661c0 msgr2=0x7fb7ac0ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:29.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.124+0000 7fb7b14cf640 1 --2- 172.21.15.84:0/3236796200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7ac0661c0 0x7fb7ac0ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:29.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.124+0000 7fb7b14cf640 1 -- 172.21.15.84:0/3236796200 >> v1:172.21.15.157:6789/0 conn(0x7fb7ac06c7d0 legacy=0x7fb7ac065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:29.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.124+0000 7fb7b14cf640 1 -- 172.21.15.84:0/3236796200 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ac160dd0 con 0x7fb7ac15d330 2023-10-16T09:50:29.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.125+0000 7fb7b14cf640 1 --2- 172.21.15.84:0/3236796200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7ac15d330 0x7fb7ac1618b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009e50 tx=0x7fb79c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88defa55aa907527 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:29.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.126+0000 7fb7abfff640 1 -- 172.21.15.84:0/3236796200 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c02c9e0 con 0x7fb7ac15d330 2023-10-16T09:50:29.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.126+0000 7fb7abfff640 1 -- 172.21.15.84:0/3236796200 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79c02cb40 con 0x7fb7ac15d330 2023-10-16T09:50:29.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.126+0000 7fb7abfff640 1 -- 172.21.15.84:0/3236796200 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c0343f0 con 0x7fb7ac15d330 2023-10-16T09:50:29.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.127+0000 7fb7b3733640 1 -- 172.21.15.84:0/3236796200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7ac15d330 msgr2=0x7fb7ac1618b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:29.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.127+0000 7fb7b3733640 1 --2- 172.21.15.84:0/3236796200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7ac15d330 0x7fb7ac1618b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009e50 tx=0x7fb79c02be40 comp rx=0 tx=0).stop 2023-10-16T09:50:29.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.127+0000 7fb7b3733640 1 -- 172.21.15.84:0/3236796200 shutdown_connections 2023-10-16T09:50:29.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.127+0000 7fb7b3733640 1 --2- 172.21.15.84:0/3236796200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7ac0661c0 0x7fb7ac0ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:29.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.127+0000 7fb7b3733640 1 --2- 172.21.15.84:0/3236796200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7ac15d330 0x7fb7ac1618b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:29.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.127+0000 7fb7b3733640 1 -- 172.21.15.84:0/3236796200 >> 172.21.15.84:0/3236796200 conn(0x7fb7ac0d0ab0 msgr2=0x7fb7ac0d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:29.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.127+0000 7fb7b3733640 1 -- 172.21.15.84:0/3236796200 shutdown_connections 2023-10-16T09:50:29.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.127+0000 7fb7b3733640 1 -- 172.21.15.84:0/3236796200 wait complete. 2023-10-16T09:50:29.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.129+0000 7fb7b3733640 1 Processor -- start 2023-10-16T09:50:29.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.129+0000 7fb7b3733640 1 -- start start 2023-10-16T09:50:29.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.131+0000 7fb7b3733640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7ac0661c0 0x7fb7ac151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:29.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.131+0000 7fb7b3733640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7ac06c7d0 0x7fb7ac0c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:29.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.131+0000 7fb7b3733640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7ac15d330 0x7fb7ac0afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:29.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.131+0000 7fb7b3733640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac0c73d0 con 0x7fb7ac15d330 2023-10-16T09:50:29.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.131+0000 7fb7b3733640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac0c67c0 con 0x7fb7ac06c7d0 2023-10-16T09:50:29.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.131+0000 7fb7b3733640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac0c6aa0 con 0x7fb7ac0661c0 2023-10-16T09:50:29.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.131+0000 7fb7b1cd0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7ac15d330 0x7fb7ac0afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:29.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.131+0000 7fb7b1cd0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7ac15d330 0x7fb7ac0afc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40586/0 (socket says 172.21.15.84:40586) 2023-10-16T09:50:29.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.131+0000 7fb7b1cd0640 1 -- 172.21.15.84:0/529525187 learned_addr learned my addr 172.21.15.84:0/529525187 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:29.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.131+0000 7fb7b14cf640 1 --2- 172.21.15.84:0/529525187 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7ac0661c0 0x7fb7ac151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:29.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.131+0000 7fb7b0cce640 1 --2- 172.21.15.84:0/529525187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7ac06c7d0 0x7fb7ac0c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:29.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.132+0000 7fb7b1cd0640 1 -- 172.21.15.84:0/529525187 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7ac0661c0 msgr2=0x7fb7ac151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:29.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.132+0000 7fb7b1cd0640 1 --2- 172.21.15.84:0/529525187 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7ac0661c0 0x7fb7ac151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:29.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.132+0000 7fb7b1cd0640 1 -- 172.21.15.84:0/529525187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7ac06c7d0 msgr2=0x7fb7ac0c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:29.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.132+0000 7fb7b1cd0640 1 --2- 172.21.15.84:0/529525187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7ac06c7d0 0x7fb7ac0c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:29.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.132+0000 7fb7b1cd0640 1 -- 172.21.15.84:0/529525187 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79c009b30 con 0x7fb7ac15d330 2023-10-16T09:50:29.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.132+0000 7fb7b14cf640 1 --2- 172.21.15.84:0/529525187 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7ac0661c0 0x7fb7ac151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:50:29.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.132+0000 7fb7b1cd0640 1 --2- 172.21.15.84:0/529525187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7ac15d330 0x7fb7ac0afc70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400d5a0 tx=0x7fb7a400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:29.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.132+0000 7fb7a9ffb640 1 -- 172.21.15.84:0/529525187 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a40125f0 con 0x7fb7ac15d330 2023-10-16T09:50:29.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.132+0000 7fb7a9ffb640 1 -- 172.21.15.84:0/529525187 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a4012770 con 0x7fb7ac15d330 2023-10-16T09:50:29.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.132+0000 7fb7b3733640 1 -- 172.21.15.84:0/529525187 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7ac0c6d80 con 0x7fb7ac15d330 2023-10-16T09:50:29.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.132+0000 7fb7a9ffb640 1 -- 172.21.15.84:0/529525187 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4012b20 con 0x7fb7ac15d330 2023-10-16T09:50:29.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.132+0000 7fb7b3733640 1 -- 172.21.15.84:0/529525187 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7ac152430 con 0x7fb7ac15d330 2023-10-16T09:50:29.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.134+0000 7fb7b3733640 1 -- 172.21.15.84:0/529525187 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7ac04e160 con 0x7fb7ac15d330 2023-10-16T09:50:29.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.135+0000 7fb7a9ffb640 1 -- 172.21.15.84:0/529525187 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 10) v1 ==== 89349+0+0 (secure 0 0 0) 0x7fb7a4012c80 con 0x7fb7ac15d330 2023-10-16T09:50:29.139 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:50:29.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.135+0000 7fb7a9ffb640 1 --2- 172.21.15.84:0/529525187 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fb79806afe0 0x7fb79806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:29.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.136+0000 7fb7a9ffb640 1 -- 172.21.15.84:0/529525187 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fb7a4089cd0 con 0x7fb7ac15d330 2023-10-16T09:50:29.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.136+0000 7fb7b14cf640 1 --2- 172.21.15.84:0/529525187 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fb79806afe0 0x7fb79806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:29.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.137+0000 7fb7b14cf640 1 --2- 172.21.15.84:0/529525187 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fb79806afe0 0x7fb79806d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb79c035000 tx=0x7fb79c030000 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:29.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.140+0000 7fb7a9ffb640 1 -- 172.21.15.84:0/529525187 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb7a4054b80 con 0x7fb7ac15d330 2023-10-16T09:50:29.145 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:50:29.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.358+0000 7fb7b3733640 1 -- 172.21.15.84:0/529525187 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir2", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7ac1610b0 con 0x7fb79806afe0 2023-10-16T09:50:29.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.541+0000 7fb7a9ffb640 1 -- 172.21.15.84:0/529525187 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fb79c0440c0 con 0x7fb79806afe0 2023-10-16T09:50:29.544 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path /snap_test_dir1/dir2 2023-10-16T09:50:29.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.545+0000 7fb7b3733640 1 -- 172.21.15.84:0/529525187 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fb79806afe0 msgr2=0x7fb79806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:29.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.545+0000 7fb7b3733640 1 --2- 172.21.15.84:0/529525187 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fb79806afe0 0x7fb79806d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb79c035000 tx=0x7fb79c030000 comp rx=0 tx=0).stop 2023-10-16T09:50:29.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.545+0000 7fb7b3733640 1 -- 172.21.15.84:0/529525187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7ac15d330 msgr2=0x7fb7ac0afc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:29.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.545+0000 7fb7b3733640 1 --2- 172.21.15.84:0/529525187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7ac15d330 0x7fb7ac0afc70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400d5a0 tx=0x7fb7a400da60 comp rx=0 tx=0).stop 2023-10-16T09:50:29.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.545+0000 7fb7b3733640 1 -- 172.21.15.84:0/529525187 shutdown_connections 2023-10-16T09:50:29.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.545+0000 7fb7b3733640 1 --2- 172.21.15.84:0/529525187 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fb79806afe0 0x7fb79806d4a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:29.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.545+0000 7fb7b3733640 1 --2- 172.21.15.84:0/529525187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7ac15d330 0x7fb7ac0afc70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:29.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.545+0000 7fb7b3733640 1 --2- 172.21.15.84:0/529525187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7ac06c7d0 0x7fb7ac0c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:29.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.545+0000 7fb7b3733640 1 --2- 172.21.15.84:0/529525187 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7ac0661c0 0x7fb7ac151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:29.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.545+0000 7fb7b3733640 1 -- 172.21.15.84:0/529525187 >> 172.21.15.84:0/529525187 conn(0x7fb7ac0d0ab0 msgr2=0x7fb7ac0b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:29.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.545+0000 7fb7b3733640 1 -- 172.21.15.84:0/529525187 shutdown_connections 2023-10-16T09:50:29.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.545+0000 7fb7b3733640 1 -- 172.21.15.84:0/529525187 wait complete. 2023-10-16T09:50:29.567 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-10-16T09:50:29.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:29.568 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-10-16T09:50:29.624 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir3 1m 2023-10-16T09:50:29.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.831+0000 7f7eaa26f640 1 Processor -- start 2023-10-16T09:50:29.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.831+0000 7f7eaa26f640 1 -- start start 2023-10-16T09:50:29.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.833+0000 7f7eaa26f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ea415ad70 0x7f7ea4069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:29.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.833+0000 7f7eaa26f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea40cb030 con 0x7f7ea40c9ca0 2023-10-16T09:50:29.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.833+0000 7f7eaa26f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea40c91c0 con 0x7f7ea40caac0 2023-10-16T09:50:29.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.833+0000 7f7eaa26f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40c94a0 con 0x7f7ea415ad70 2023-10-16T09:50:29.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.833+0000 7f7e9bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7ea40c9ca0 0x7f7ea40ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37892/0 (socket says 172.21.15.84:37892) 2023-10-16T09:50:29.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.833+0000 7f7e9bfff640 1 -- 172.21.15.84:0/195350130 learned_addr learned my addr 172.21.15.84:0/195350130 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:29.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.834+0000 7f7ea3fff640 1 --2- 172.21.15.84:0/195350130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ea415ad70 0x7f7ea4069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:29.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.834+0000 7f7ea37fe640 1 -- 172.21.15.84:0/195350130 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655561873 0 0) 0x7f7ea40cb030 con 0x7f7ea40c9ca0 2023-10-16T09:50:29.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.834+0000 7f7ea37fe640 1 -- 172.21.15.84:0/195350130 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e90003520 con 0x7f7ea40c9ca0 2023-10-16T09:50:29.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.834+0000 7f7ea37fe640 1 -- 172.21.15.84:0/195350130 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2301226904 0 0) 0x7f7e90003520 con 0x7f7ea40c9ca0 2023-10-16T09:50:29.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.834+0000 7f7ea37fe640 1 -- 172.21.15.84:0/195350130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ea415ad70 msgr2=0x7f7ea4069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:29.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.834+0000 7f7ea37fe640 1 --2- 172.21.15.84:0/195350130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ea415ad70 0x7f7ea4069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:29.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.834+0000 7f7ea37fe640 1 -- 172.21.15.84:0/195350130 >> v1:172.21.15.157:6789/0 conn(0x7f7ea40caac0 legacy=0x7f7ea40ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:29.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.834+0000 7f7ea37fe640 1 -- 172.21.15.84:0/195350130 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea40c9780 con 0x7f7ea40c9ca0 2023-10-16T09:50:29.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.834+0000 7f7ea37fe640 1 -- 172.21.15.84:0/195350130 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f7e8c001670 con 0x7f7ea40c9ca0 2023-10-16T09:50:29.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.834+0000 7f7ea37fe640 1 -- 172.21.15.84:0/195350130 >> v1:172.21.15.84:6789/0 conn(0x7f7ea40c9ca0 legacy=0x7f7ea40ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:29.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.834+0000 7f7ea37fe640 1 --2- 172.21.15.84:0/195350130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ea415ad70 0x7f7e900039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:29.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.834+0000 7f7ea37fe640 1 --2- 172.21.15.84:0/195350130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e90003f70 0x7f7e900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:29.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.834+0000 7f7ea37fe640 1 --2- 172.21.15.84:0/195350130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e90004d00 0x7f7e90005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:29.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.834+0000 7f7ea37fe640 1 -- 172.21.15.84:0/195350130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40c94a0 con 0x7f7e90003f70 2023-10-16T09:50:29.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.834+0000 7f7ea37fe640 1 -- 172.21.15.84:0/195350130 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9000bac0 con 0x7f7e90004d00 2023-10-16T09:50:29.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.834+0000 7f7ea37fe640 1 -- 172.21.15.84:0/195350130 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9000bc80 con 0x7f7ea415ad70 2023-10-16T09:50:29.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.834+0000 7f7e9bfff640 1 --2- 172.21.15.84:0/195350130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e90003f70 0x7f7e900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:29.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.835+0000 7f7ea3fff640 1 --2- 172.21.15.84:0/195350130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ea415ad70 0x7f7e900039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:29.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.835+0000 7f7e9bfff640 1 -- 172.21.15.84:0/195350130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ea415ad70 msgr2=0x7f7e900039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:29.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.835+0000 7f7ea880c640 1 --2- 172.21.15.84:0/195350130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e90004d00 0x7f7e90005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:29.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.835+0000 7f7e9bfff640 1 --2- 172.21.15.84:0/195350130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ea415ad70 0x7f7e900039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:29.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.835+0000 7f7e9bfff640 1 -- 172.21.15.84:0/195350130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e90004d00 msgr2=0x7f7e90005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:29.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.835+0000 7f7e9bfff640 1 --2- 172.21.15.84:0/195350130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e90004d00 0x7f7e90005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:29.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.835+0000 7f7e9bfff640 1 -- 172.21.15.84:0/195350130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e90003520 con 0x7f7e90003f70 2023-10-16T09:50:29.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.836+0000 7f7e9bfff640 1 --2- 172.21.15.84:0/195350130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e90003f70 0x7f7e900047c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00c0e0 tx=0x7f7e900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232c743bea54b313 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:29.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.836+0000 7f7ea37fe640 1 -- 172.21.15.84:0/195350130 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c005ad0 con 0x7f7e90003f70 2023-10-16T09:50:29.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.837+0000 7f7ea37fe640 1 -- 172.21.15.84:0/195350130 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8c005c30 con 0x7f7e90003f70 2023-10-16T09:50:29.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.837+0000 7f7ea37fe640 1 -- 172.21.15.84:0/195350130 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0072b0 con 0x7f7e90003f70 2023-10-16T09:50:29.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.837+0000 7f7eaa26f640 1 -- 172.21.15.84:0/195350130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e90003f70 msgr2=0x7f7e900047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:29.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.837+0000 7f7eaa26f640 1 --2- 172.21.15.84:0/195350130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e90003f70 0x7f7e900047c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00c0e0 tx=0x7f7e900057f0 comp rx=0 tx=0).stop 2023-10-16T09:50:29.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.837+0000 7f7ea880c640 1 -- 172.21.15.84:0/195350130 reap_dead start 2023-10-16T09:50:29.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.837+0000 7f7eaa26f640 1 -- 172.21.15.84:0/195350130 shutdown_connections 2023-10-16T09:50:29.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.837+0000 7f7eaa26f640 1 -- 172.21.15.84:0/195350130 >> 172.21.15.84:0/195350130 conn(0x7f7ea40ce9b0 msgr2=0x7f7ea40cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:29.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.837+0000 7f7eaa26f640 1 -- 172.21.15.84:0/195350130 shutdown_connections 2023-10-16T09:50:29.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.837+0000 7f7eaa26f640 1 -- 172.21.15.84:0/195350130 wait complete. 2023-10-16T09:50:29.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.839+0000 7f7eaa26f640 1 Processor -- start 2023-10-16T09:50:29.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.839+0000 7f7eaa26f640 1 -- start start 2023-10-16T09:50:29.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.840+0000 7f7eaa26f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e90003f70 0x7f7ea40c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:29.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.840+0000 7f7eaa26f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ea40cd580 0x7f7ea41594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:29.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.840+0000 7f7eaa26f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ea415d3b0 0x7f7ea415d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:29.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.840+0000 7f7eaa26f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea415dda0 con 0x7f7e90003f70 2023-10-16T09:50:29.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.840+0000 7f7eaa26f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40ae430 con 0x7f7ea40cd580 2023-10-16T09:50:29.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.840+0000 7f7eaa26f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea41599f0 con 0x7f7ea415d3b0 2023-10-16T09:50:29.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.840+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e90003f70 0x7f7ea40c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:29.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.840+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e90003f70 0x7f7ea40c24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40612/0 (socket says 172.21.15.84:40612) 2023-10-16T09:50:29.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.840+0000 7f7ea3fff640 1 -- 172.21.15.84:0/2403128109 learned_addr learned my addr 172.21.15.84:0/2403128109 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:29.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.841+0000 7f7ea3fff640 1 -- 172.21.15.84:0/2403128109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ea415d3b0 msgr2=0x7f7ea415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:29.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.841+0000 7f7ea3fff640 1 --2- 172.21.15.84:0/2403128109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ea415d3b0 0x7f7ea415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:29.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.841+0000 7f7ea3fff640 1 -- 172.21.15.84:0/2403128109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ea40cd580 msgr2=0x7f7ea41594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:29.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.841+0000 7f7ea3fff640 1 --2- 172.21.15.84:0/2403128109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ea40cd580 0x7f7ea41594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:29.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.841+0000 7f7ea3fff640 1 -- 172.21.15.84:0/2403128109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea40c3740 con 0x7f7e90003f70 2023-10-16T09:50:29.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.841+0000 7f7ea3fff640 1 --2- 172.21.15.84:0/2403128109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e90003f70 0x7f7ea40c24c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7e94010860 tx=0x7f7e94010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:29.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.841+0000 7f7ea17fa640 1 -- 172.21.15.84:0/2403128109 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94009610 con 0x7f7e90003f70 2023-10-16T09:50:29.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.841+0000 7f7ea17fa640 1 -- 172.21.15.84:0/2403128109 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9400eb00 con 0x7f7e90003f70 2023-10-16T09:50:29.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.841+0000 7f7ea17fa640 1 -- 172.21.15.84:0/2403128109 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400ee90 con 0x7f7e90003f70 2023-10-16T09:50:29.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.841+0000 7f7eaa26f640 1 -- 172.21.15.84:0/2403128109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea40c2bc0 con 0x7f7e90003f70 2023-10-16T09:50:29.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.841+0000 7f7eaa26f640 1 -- 172.21.15.84:0/2403128109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea40c3180 con 0x7f7e90003f70 2023-10-16T09:50:29.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.843+0000 7f7eaa26f640 1 -- 172.21.15.84:0/2403128109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea404e160 con 0x7f7e90003f70 2023-10-16T09:50:29.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.843+0000 7f7ea17fa640 1 -- 172.21.15.84:0/2403128109 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 10) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f7e9401c400 con 0x7f7e90003f70 2023-10-16T09:50:29.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.844+0000 7f7ea17fa640 1 --2- 172.21.15.84:0/2403128109 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f7e8406afe0 0x7f7e8406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:29.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.844+0000 7f7ea17fa640 1 -- 172.21.15.84:0/2403128109 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f7e94089e20 con 0x7f7e90003f70 2023-10-16T09:50:29.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.845+0000 7f7e9bfff640 1 --2- 172.21.15.84:0/2403128109 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f7e8406afe0 0x7f7e8406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:29.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.845+0000 7f7e9bfff640 1 --2- 172.21.15.84:0/2403128109 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f7e8406afe0 0x7f7e8406d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c034000 tx=0x7f7e8c007000 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:29.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:29.849+0000 7f7ea17fa640 1 -- 172.21.15.84:0/2403128109 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7e94054cb0 con 0x7f7e90003f70 2023-10-16T09:50:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.066+0000 7f7eaa26f640 1 -- 172.21.15.84:0/2403128109 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir3", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ea40adbe0 con 0x7f7e8406afe0 2023-10-16T09:50:30.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.241+0000 7f7ea17fa640 1 -- 172.21.15.84:0/2403128109 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f7e8c0014f0 con 0x7f7e8406afe0 2023-10-16T09:50:30.243 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path /snap_test_dir1/dir3 2023-10-16T09:50:30.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.244+0000 7f7eaa26f640 1 -- 172.21.15.84:0/2403128109 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f7e8406afe0 msgr2=0x7f7e8406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:30.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.244+0000 7f7eaa26f640 1 --2- 172.21.15.84:0/2403128109 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f7e8406afe0 0x7f7e8406d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c034000 tx=0x7f7e8c007000 comp rx=0 tx=0).stop 2023-10-16T09:50:30.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.244+0000 7f7eaa26f640 1 -- 172.21.15.84:0/2403128109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e90003f70 msgr2=0x7f7ea40c24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:30.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.244+0000 7f7eaa26f640 1 --2- 172.21.15.84:0/2403128109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e90003f70 0x7f7ea40c24c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7e94010860 tx=0x7f7e94010d20 comp rx=0 tx=0).stop 2023-10-16T09:50:30.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.245+0000 7f7eaa26f640 1 -- 172.21.15.84:0/2403128109 shutdown_connections 2023-10-16T09:50:30.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.245+0000 7f7eaa26f640 1 --2- 172.21.15.84:0/2403128109 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f7e8406afe0 0x7f7e8406d4a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:30.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.245+0000 7f7eaa26f640 1 --2- 172.21.15.84:0/2403128109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ea415d3b0 0x7f7ea415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:30.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.245+0000 7f7eaa26f640 1 --2- 172.21.15.84:0/2403128109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ea40cd580 0x7f7ea41594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:30.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.245+0000 7f7eaa26f640 1 --2- 172.21.15.84:0/2403128109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e90003f70 0x7f7ea40c24c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:30.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.245+0000 7f7eaa26f640 1 -- 172.21.15.84:0/2403128109 >> 172.21.15.84:0/2403128109 conn(0x7f7ea40ce9b0 msgr2=0x7f7ea4070d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:30.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.245+0000 7f7eaa26f640 1 -- 172.21.15.84:0/2403128109 shutdown_connections 2023-10-16T09:50:30.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.245+0000 7f7eaa26f640 1 -- 172.21.15.84:0/2403128109 wait complete. 2023-10-16T09:50:30.266 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-10-16T09:50:30.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:30.266 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-10-16T09:50:30.325 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir4 1m 2023-10-16T09:50:30.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.539+0000 7f29c7790640 1 Processor -- start 2023-10-16T09:50:30.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.539+0000 7f29c7790640 1 -- start start 2023-10-16T09:50:30.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.541+0000 7f29c7790640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29c00cf5f0 0x7f29c006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:30.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.541+0000 7f29c7790640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c0161740 con 0x7f29c0070c10 2023-10-16T09:50:30.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.541+0000 7f29c7790640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c006b0f0 con 0x7f29c0070830 2023-10-16T09:50:30.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.541+0000 7f29c7790640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c01619b0 con 0x7f29c00cf5f0 2023-10-16T09:50:30.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.541+0000 7f29c5d2d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f29c0070c10 0x7f29c00cd690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37900/0 (socket says 172.21.15.84:37900) 2023-10-16T09:50:30.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.541+0000 7f29c5d2d640 1 -- 172.21.15.84:0/2760393311 learned_addr learned my addr 172.21.15.84:0/2760393311 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:30.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.542+0000 7f29c4d2b640 1 --2- 172.21.15.84:0/2760393311 >> v2:172.21.15.157:3300/0 conn(0x7f29c00cf5f0 0x7f29c006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:30.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.542+0000 7f29bffff640 1 -- 172.21.15.84:0/2760393311 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 963327137 0 0) 0x7f29c0161740 con 0x7f29c0070c10 2023-10-16T09:50:30.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.542+0000 7f29bffff640 1 -- 172.21.15.84:0/2760393311 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29b0003520 con 0x7f29c0070c10 2023-10-16T09:50:30.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.542+0000 7f29bffff640 1 -- 172.21.15.84:0/2760393311 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1292035130 0 0) 0x7f29b0003520 con 0x7f29c0070c10 2023-10-16T09:50:30.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.542+0000 7f29bffff640 1 -- 172.21.15.84:0/2760393311 >> v2:172.21.15.157:3300/0 conn(0x7f29c00cf5f0 msgr2=0x7f29c006d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:30.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.542+0000 7f29bffff640 1 --2- 172.21.15.84:0/2760393311 >> v2:172.21.15.157:3300/0 conn(0x7f29c00cf5f0 0x7f29c006d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:30.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.542+0000 7f29bffff640 1 -- 172.21.15.84:0/2760393311 >> v1:172.21.15.157:6789/0 conn(0x7f29c0070830 legacy=0x7f29c006c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:30.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.542+0000 7f29bffff640 1 -- 172.21.15.84:0/2760393311 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c006b3d0 con 0x7f29c0070c10 2023-10-16T09:50:30.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.542+0000 7f29bffff640 1 -- 172.21.15.84:0/2760393311 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f29b8001670 con 0x7f29c0070c10 2023-10-16T09:50:30.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.542+0000 7f29c4d2b640 1 --2- 172.21.15.84:0/2760393311 >> v2:172.21.15.157:3300/0 conn(0x7f29c00cf5f0 0x7f29c006d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:50:30.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.542+0000 7f29bffff640 1 -- 172.21.15.84:0/2760393311 >> v1:172.21.15.84:6789/0 conn(0x7f29c0070c10 legacy=0x7f29c00cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:30.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.542+0000 7f29bffff640 1 --2- 172.21.15.84:0/2760393311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00039c0 0x7f29b0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:30.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.542+0000 7f29bffff640 1 --2- 172.21.15.84:0/2760393311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29b00047c0 0x7f29b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:30.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.543+0000 7f29bffff640 1 --2- 172.21.15.84:0/2760393311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29b0005380 0x7f29b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:30.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.543+0000 7f29bffff640 1 -- 172.21.15.84:0/2760393311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c01619b0 con 0x7f29b00039c0 2023-10-16T09:50:30.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.543+0000 7f29bffff640 1 -- 172.21.15.84:0/2760393311 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b000c260 con 0x7f29b00047c0 2023-10-16T09:50:30.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.543+0000 7f29bffff640 1 -- 172.21.15.84:0/2760393311 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b000c420 con 0x7f29b0005380 2023-10-16T09:50:30.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.543+0000 7f29c552c640 1 --2- 172.21.15.84:0/2760393311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00039c0 0x7f29b0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:30.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.543+0000 7f29c4d2b640 1 --2- 172.21.15.84:0/2760393311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29b00047c0 0x7f29b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:30.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.543+0000 7f29c552c640 1 -- 172.21.15.84:0/2760393311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29b0005380 msgr2=0x7f29b000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:30.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.543+0000 7f29c552c640 1 --2- 172.21.15.84:0/2760393311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29b0005380 0x7f29b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:30.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.543+0000 7f29c552c640 1 -- 172.21.15.84:0/2760393311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29b00047c0 msgr2=0x7f29b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:30.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.543+0000 7f29c552c640 1 --2- 172.21.15.84:0/2760393311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29b00047c0 0x7f29b0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:30.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.543+0000 7f29c552c640 1 -- 172.21.15.84:0/2760393311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b0003520 con 0x7f29b00039c0 2023-10-16T09:50:30.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.543+0000 7f29c5d2d640 1 -- 172.21.15.84:0/2760393311 reap_dead start 2023-10-16T09:50:30.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.544+0000 7f29c552c640 1 --2- 172.21.15.84:0/2760393311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00039c0 0x7f29b0003de0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f29b4001e50 tx=0x7f29b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a32b78cc6ae8c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:30.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.544+0000 7f29bffff640 1 -- 172.21.15.84:0/2760393311 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4003b70 con 0x7f29b00039c0 2023-10-16T09:50:30.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.545+0000 7f29bffff640 1 -- 172.21.15.84:0/2760393311 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b4003cd0 con 0x7f29b00039c0 2023-10-16T09:50:30.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.545+0000 7f29bffff640 1 -- 172.21.15.84:0/2760393311 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4005310 con 0x7f29b00039c0 2023-10-16T09:50:30.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.545+0000 7f29c7790640 1 -- 172.21.15.84:0/2760393311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00039c0 msgr2=0x7f29b0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:30.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.545+0000 7f29c7790640 1 --2- 172.21.15.84:0/2760393311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00039c0 0x7f29b0003de0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f29b4001e50 tx=0x7f29b00057f0 comp rx=0 tx=0).stop 2023-10-16T09:50:30.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.545+0000 7f29c7790640 1 -- 172.21.15.84:0/2760393311 shutdown_connections 2023-10-16T09:50:30.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.545+0000 7f29c7790640 1 --2- 172.21.15.84:0/2760393311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00039c0 0x7f29b0003de0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:30.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.545+0000 7f29c7790640 1 -- 172.21.15.84:0/2760393311 >> 172.21.15.84:0/2760393311 conn(0x7f29c00cfac0 msgr2=0x7f29c00cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:30.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.545+0000 7f29c7790640 1 -- 172.21.15.84:0/2760393311 shutdown_connections 2023-10-16T09:50:30.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.545+0000 7f29c7790640 1 -- 172.21.15.84:0/2760393311 wait complete. 2023-10-16T09:50:30.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.547+0000 7f29c7790640 1 Processor -- start 2023-10-16T09:50:30.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.547+0000 7f29c7790640 1 -- start start 2023-10-16T09:50:30.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.548+0000 7f29c7790640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00039c0 0x7f29c00b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:30.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.548+0000 7f29c7790640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c00b4740 0x7f29c01f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:30.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.548+0000 7f29c7790640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c01f2c80 0x7f29c01f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:30.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.548+0000 7f29c7790640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c01f3640 con 0x7f29b00039c0 2023-10-16T09:50:30.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.548+0000 7f29c7790640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00b0570 con 0x7f29c00b4740 2023-10-16T09:50:30.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.548+0000 7f29c7790640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00b5710 con 0x7f29c01f2c80 2023-10-16T09:50:30.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.548+0000 7f29c552c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00039c0 0x7f29c00b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.548+0000 7f29c552c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00039c0 0x7f29c00b4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40628/0 (socket says 172.21.15.84:40628) 2023-10-16T09:50:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.548+0000 7f29c552c640 1 -- 172.21.15.84:0/3461875193 learned_addr learned my addr 172.21.15.84:0/3461875193 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.549+0000 7f29c4d2b640 1 --2- 172.21.15.84:0/3461875193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c00b4740 0x7f29c01f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.549+0000 7f29c552c640 1 -- 172.21.15.84:0/3461875193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c01f2c80 msgr2=0x7f29c01f3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.549+0000 7f29c552c640 1 --2- 172.21.15.84:0/3461875193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c01f2c80 0x7f29c01f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:30.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.549+0000 7f29c552c640 1 -- 172.21.15.84:0/3461875193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c00b4740 msgr2=0x7f29c01f2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:30.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.549+0000 7f29c552c640 1 --2- 172.21.15.84:0/3461875193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c00b4740 0x7f29c01f2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:30.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.549+0000 7f29c552c640 1 -- 172.21.15.84:0/3461875193 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c00b2910 con 0x7f29b00039c0 2023-10-16T09:50:30.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.549+0000 7f29c4d2b640 1 --2- 172.21.15.84:0/3461875193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c00b4740 0x7f29c01f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:30.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.549+0000 7f29c552c640 1 --2- 172.21.15.84:0/3461875193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00039c0 0x7f29c00b4200 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f29b402bf70 tx=0x7f29b40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:30.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.549+0000 7f29bdffb640 1 -- 172.21.15.84:0/3461875193 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b403e070 con 0x7f29b00039c0 2023-10-16T09:50:30.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.550+0000 7f29bdffb640 1 -- 172.21.15.84:0/3461875193 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b4006b50 con 0x7f29b00039c0 2023-10-16T09:50:30.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.550+0000 7f29bdffb640 1 -- 172.21.15.84:0/3461875193 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4007020 con 0x7f29b00039c0 2023-10-16T09:50:30.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.550+0000 7f29c7790640 1 -- 172.21.15.84:0/3461875193 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c00b4bc0 con 0x7f29b00039c0 2023-10-16T09:50:30.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.550+0000 7f29c7790640 1 -- 172.21.15.84:0/3461875193 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c00b5150 con 0x7f29b00039c0 2023-10-16T09:50:30.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.551+0000 7f29c7790640 1 -- 172.21.15.84:0/3461875193 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c004e160 con 0x7f29b00039c0 2023-10-16T09:50:30.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.552+0000 7f29bdffb640 1 -- 172.21.15.84:0/3461875193 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 10) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f29b4007180 con 0x7f29b00039c0 2023-10-16T09:50:30.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.552+0000 7f29bdffb640 1 --2- 172.21.15.84:0/3461875193 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f29a406afe0 0x7f29a406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:30.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.553+0000 7f29bdffb640 1 -- 172.21.15.84:0/3461875193 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f29b40aafe0 con 0x7f29b00039c0 2023-10-16T09:50:30.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.553+0000 7f29c4d2b640 1 --2- 172.21.15.84:0/3461875193 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f29a406afe0 0x7f29a406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:30.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.554+0000 7f29c4d2b640 1 --2- 172.21.15.84:0/3461875193 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f29a406afe0 0x7f29a406d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f29c00cc290 tx=0x7f29ac008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:30.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.557+0000 7f29bdffb640 1 -- 172.21.15.84:0/3461875193 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f29b4075da0 con 0x7f29b00039c0 2023-10-16T09:50:30.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.771+0000 7f29c7790640 1 -- 172.21.15.84:0/3461875193 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir4", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f29c00cf060 con 0x7f29a406afe0 2023-10-16T09:50:30.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.943+0000 7f29bdffb640 1 -- 172.21.15.84:0/3461875193 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f29ac00c040 con 0x7f29a406afe0 2023-10-16T09:50:30.947 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path /snap_test_dir1/dir4 2023-10-16T09:50:30.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.950+0000 7f29bdffb640 1 -- 172.21.15.84:0/3461875193 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f29b4076170 con 0x7f29b00039c0 2023-10-16T09:50:30.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.950+0000 7f29c7790640 1 -- 172.21.15.84:0/3461875193 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f29a406afe0 msgr2=0x7f29a406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:30.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.950+0000 7f29c7790640 1 --2- 172.21.15.84:0/3461875193 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f29a406afe0 0x7f29a406d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f29c00cc290 tx=0x7f29ac008040 comp rx=0 tx=0).stop 2023-10-16T09:50:30.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.950+0000 7f29c7790640 1 -- 172.21.15.84:0/3461875193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00039c0 msgr2=0x7f29c00b4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:30.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.950+0000 7f29c7790640 1 --2- 172.21.15.84:0/3461875193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00039c0 0x7f29c00b4200 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f29b402bf70 tx=0x7f29b40014a0 comp rx=0 tx=0).stop 2023-10-16T09:50:30.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.951+0000 7f29c7790640 1 -- 172.21.15.84:0/3461875193 shutdown_connections 2023-10-16T09:50:30.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.951+0000 7f29c7790640 1 --2- 172.21.15.84:0/3461875193 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f29a406afe0 0x7f29a406d4a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:30.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.951+0000 7f29c7790640 1 --2- 172.21.15.84:0/3461875193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29c01f2c80 0x7f29c01f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:30.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.951+0000 7f29c7790640 1 --2- 172.21.15.84:0/3461875193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c00b4740 0x7f29c01f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:30.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.951+0000 7f29c7790640 1 --2- 172.21.15.84:0/3461875193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00039c0 0x7f29c00b4200 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:30.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.951+0000 7f29c7790640 1 -- 172.21.15.84:0/3461875193 >> 172.21.15.84:0/3461875193 conn(0x7f29c00cfac0 msgr2=0x7f29c0160be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:30.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.951+0000 7f29c7790640 1 -- 172.21.15.84:0/3461875193 shutdown_connections 2023-10-16T09:50:30.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:30.951+0000 7f29c7790640 1 -- 172.21.15.84:0/3461875193 wait complete. 2023-10-16T09:50:30.972 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-10-16T09:50:30.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:30.973 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-10-16T09:50:31.022 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir5 1m 2023-10-16T09:50:31.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.222+0000 7f4ac86cf640 1 Processor -- start 2023-10-16T09:50:31.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.223+0000 7f4ac86cf640 1 -- start start 2023-10-16T09:50:31.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.224+0000 7f4ac86cf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ac015d150 0x7f4ac00ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:31.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.224+0000 7f4ac86cf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac0160b90 0x7f4ac0160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:31.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.224+0000 7f4ac86cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4ac0069da0 0x7f4ac006a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:31.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.224+0000 7f4ac86cf640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac015d530 con 0x7f4ac0069da0 2023-10-16T09:50:31.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.224+0000 7f4ac86cf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac006a6f0 con 0x7f4ac015d150 2023-10-16T09:50:31.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.224+0000 7f4ac86cf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac00ac890 con 0x7f4ac0160b90 2023-10-16T09:50:31.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.225+0000 7f4ac6c6c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4ac0069da0 0x7f4ac006a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:31.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.225+0000 7f4ac6c6c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4ac0069da0 0x7f4ac006a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40638/0 (socket says 172.21.15.84:40638) 2023-10-16T09:50:31.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.225+0000 7f4ac6c6c640 1 -- 172.21.15.84:0/2112564462 learned_addr learned my addr 172.21.15.84:0/2112564462 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:31.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.225+0000 7f4ac646b640 1 --2- 172.21.15.84:0/2112564462 >> v2:172.21.15.157:3300/0 conn(0x7f4ac015d150 0x7f4ac00ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:31.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.225+0000 7f4ac5c6a640 1 --2- 172.21.15.84:0/2112564462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac0160b90 0x7f4ac0160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:31.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.225+0000 7f4ac6c6c640 1 -- 172.21.15.84:0/2112564462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac0160b90 msgr2=0x7f4ac0160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:31.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.225+0000 7f4ac6c6c640 1 --2- 172.21.15.84:0/2112564462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac0160b90 0x7f4ac0160fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.225+0000 7f4ac6c6c640 1 -- 172.21.15.84:0/2112564462 >> v2:172.21.15.157:3300/0 conn(0x7f4ac015d150 msgr2=0x7f4ac00ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:31.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.225+0000 7f4ac6c6c640 1 --2- 172.21.15.84:0/2112564462 >> v2:172.21.15.157:3300/0 conn(0x7f4ac015d150 0x7f4ac00ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.225+0000 7f4ac6c6c640 1 -- 172.21.15.84:0/2112564462 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac0161650 con 0x7f4ac0069da0 2023-10-16T09:50:31.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.225+0000 7f4ac646b640 1 --2- 172.21.15.84:0/2112564462 >> v2:172.21.15.157:3300/0 conn(0x7f4ac015d150 0x7f4ac00ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:50:31.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.227+0000 7f4ac6c6c640 1 --2- 172.21.15.84:0/2112564462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ac0069da0 0x7f4ac006a1b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00b15a0 tx=0x7f4ab802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f77c46fc24fc08 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:31.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.227+0000 7f4ac5469640 1 -- 172.21.15.84:0/2112564462 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab802ca70 con 0x7f4ac0069da0 2023-10-16T09:50:31.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.227+0000 7f4ac5469640 1 -- 172.21.15.84:0/2112564462 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab802cbd0 con 0x7f4ac0069da0 2023-10-16T09:50:31.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.227+0000 7f4ac5469640 1 -- 172.21.15.84:0/2112564462 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab80343f0 con 0x7f4ac0069da0 2023-10-16T09:50:31.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.228+0000 7f4ac86cf640 1 -- 172.21.15.84:0/2112564462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ac0069da0 msgr2=0x7f4ac006a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:31.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.228+0000 7f4ac86cf640 1 --2- 172.21.15.84:0/2112564462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ac0069da0 0x7f4ac006a1b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00b15a0 tx=0x7f4ab802bed0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.228+0000 7f4ac86cf640 1 -- 172.21.15.84:0/2112564462 shutdown_connections 2023-10-16T09:50:31.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.228+0000 7f4ac86cf640 1 --2- 172.21.15.84:0/2112564462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ac0069da0 0x7f4ac006a1b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.228+0000 7f4ac86cf640 1 --2- 172.21.15.84:0/2112564462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac0160b90 0x7f4ac0160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.228+0000 7f4ac86cf640 1 --2- 172.21.15.84:0/2112564462 >> v2:172.21.15.157:3300/0 conn(0x7f4ac015d150 0x7f4ac00ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.228+0000 7f4ac86cf640 1 -- 172.21.15.84:0/2112564462 >> 172.21.15.84:0/2112564462 conn(0x7f4ac00ce9b0 msgr2=0x7f4ac00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:31.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.228+0000 7f4ac86cf640 1 -- 172.21.15.84:0/2112564462 shutdown_connections 2023-10-16T09:50:31.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.228+0000 7f4ac86cf640 1 -- 172.21.15.84:0/2112564462 wait complete. 2023-10-16T09:50:31.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.230+0000 7f4ac86cf640 1 Processor -- start 2023-10-16T09:50:31.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.230+0000 7f4ac86cf640 1 -- start start 2023-10-16T09:50:31.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.231+0000 7f4ac86cf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0069da0 0x7f4ac00b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:31.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.231+0000 7f4ac86cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ac015d150 0x7f4ac00af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:31.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.232+0000 7f4ac86cf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac0160b90 0x7f4ac00b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:31.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.232+0000 7f4ac86cf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac00b1000 con 0x7f4ac015d150 2023-10-16T09:50:31.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.232+0000 7f4ac86cf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac00b0a40 con 0x7f4ac0069da0 2023-10-16T09:50:31.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.232+0000 7f4ac86cf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac00b0d20 con 0x7f4ac0160b90 2023-10-16T09:50:31.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.232+0000 7f4ac5c6a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ac015d150 0x7f4ac00af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:31.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.232+0000 7f4ac5c6a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ac015d150 0x7f4ac00af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40644/0 (socket says 172.21.15.84:40644) 2023-10-16T09:50:31.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.232+0000 7f4ac5c6a640 1 -- 172.21.15.84:0/1359802108 learned_addr learned my addr 172.21.15.84:0/1359802108 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:31.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.232+0000 7f4ac646b640 1 --2- 172.21.15.84:0/1359802108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0069da0 0x7f4ac00b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:31.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.232+0000 7f4ac5c6a640 1 -- 172.21.15.84:0/1359802108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac0160b90 msgr2=0x7f4ac00b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:31.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.232+0000 7f4ac5c6a640 1 --2- 172.21.15.84:0/1359802108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac0160b90 0x7f4ac00b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.232+0000 7f4ac5c6a640 1 -- 172.21.15.84:0/1359802108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0069da0 msgr2=0x7f4ac00b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:31.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.232+0000 7f4ac5c6a640 1 --2- 172.21.15.84:0/1359802108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0069da0 0x7f4ac00b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.232+0000 7f4ac5c6a640 1 -- 172.21.15.84:0/1359802108 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab8009b30 con 0x7f4ac015d150 2023-10-16T09:50:31.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.233+0000 7f4ac5c6a640 1 --2- 172.21.15.84:0/1359802108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ac015d150 0x7f4ac00af230 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000d800 tx=0x7f4ab000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:31.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.233+0000 7f4ab6ffd640 1 -- 172.21.15.84:0/1359802108 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab000b6c0 con 0x7f4ac015d150 2023-10-16T09:50:31.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.233+0000 7f4ab6ffd640 1 -- 172.21.15.84:0/1359802108 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab000b820 con 0x7f4ac015d150 2023-10-16T09:50:31.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.233+0000 7f4ab6ffd640 1 -- 172.21.15.84:0/1359802108 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab000bbb0 con 0x7f4ac015d150 2023-10-16T09:50:31.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.233+0000 7f4ac86cf640 1 -- 172.21.15.84:0/1359802108 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac00c6c00 con 0x7f4ac015d150 2023-10-16T09:50:31.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.233+0000 7f4ac86cf640 1 -- 172.21.15.84:0/1359802108 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac00b43c0 con 0x7f4ac015d150 2023-10-16T09:50:31.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.234+0000 7f4ac86cf640 1 -- 172.21.15.84:0/1359802108 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac004e160 con 0x7f4ac015d150 2023-10-16T09:50:31.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.235+0000 7f4ab6ffd640 1 -- 172.21.15.84:0/1359802108 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f4ab000bd10 con 0x7f4ac015d150 2023-10-16T09:50:31.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.236+0000 7f4ab6ffd640 1 --2- 172.21.15.84:0/1359802108 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f4aa406b070 0x7f4aa406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:31.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.236+0000 7f4ab6ffd640 1 -- 172.21.15.84:0/1359802108 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f4ab0089fa0 con 0x7f4ac015d150 2023-10-16T09:50:31.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.236+0000 7f4ac646b640 1 --2- 172.21.15.84:0/1359802108 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f4aa406b070 0x7f4aa406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:31.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.237+0000 7f4ac646b640 1 --2- 172.21.15.84:0/1359802108 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f4aa406b070 0x7f4aa406d530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4abc006fd0 tx=0x7f4abc008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:31.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.241+0000 7f4ab6ffd640 1 -- 172.21.15.84:0/1359802108 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4ab00bcfe0 con 0x7f4ac015d150 2023-10-16T09:50:31.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.452+0000 7f4ac86cf640 1 -- 172.21.15.84:0/1359802108 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir5", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ac00b26d0 con 0x7f4aa406b070 2023-10-16T09:50:31.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.636+0000 7f4ab6ffd640 1 -- 172.21.15.84:0/1359802108 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f4ac00b26d0 con 0x7f4aa406b070 2023-10-16T09:50:31.638 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path /snap_test_dir1/dir5 2023-10-16T09:50:31.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.639+0000 7f4ac86cf640 1 -- 172.21.15.84:0/1359802108 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f4aa406b070 msgr2=0x7f4aa406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:31.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.639+0000 7f4ac86cf640 1 --2- 172.21.15.84:0/1359802108 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f4aa406b070 0x7f4aa406d530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4abc006fd0 tx=0x7f4abc008040 comp rx=0 tx=0).stop 2023-10-16T09:50:31.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.639+0000 7f4ac86cf640 1 -- 172.21.15.84:0/1359802108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ac015d150 msgr2=0x7f4ac00af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:31.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.639+0000 7f4ac86cf640 1 --2- 172.21.15.84:0/1359802108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ac015d150 0x7f4ac00af230 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000d800 tx=0x7f4ab000dcc0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.639+0000 7f4ac86cf640 1 -- 172.21.15.84:0/1359802108 shutdown_connections 2023-10-16T09:50:31.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.639+0000 7f4ac86cf640 1 --2- 172.21.15.84:0/1359802108 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f4aa406b070 0x7f4aa406d530 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.639+0000 7f4ac86cf640 1 --2- 172.21.15.84:0/1359802108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ac0160b90 0x7f4ac00b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.639+0000 7f4ac86cf640 1 --2- 172.21.15.84:0/1359802108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ac015d150 0x7f4ac00af230 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.639+0000 7f4ac86cf640 1 --2- 172.21.15.84:0/1359802108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ac0069da0 0x7f4ac00b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.639+0000 7f4ac86cf640 1 -- 172.21.15.84:0/1359802108 >> 172.21.15.84:0/1359802108 conn(0x7f4ac00ce9b0 msgr2=0x7f4ac00c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:31.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.640+0000 7f4ac86cf640 1 -- 172.21.15.84:0/1359802108 shutdown_connections 2023-10-16T09:50:31.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.640+0000 7f4ac86cf640 1 -- 172.21.15.84:0/1359802108 wait complete. 2023-10-16T09:50:31.662 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-10-16T09:50:31.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:31.662 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-10-16T09:50:31.712 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir6 1m 2023-10-16T09:50:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.909+0000 7f386ed3e640 1 Processor -- start 2023-10-16T09:50:31.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.910+0000 7f386ed3e640 1 -- start start 2023-10-16T09:50:31.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.911+0000 7f386ed3e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3868159ab0 0x7f38680ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:31.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.911+0000 7f386ed3e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3868070360 con 0x7f386806c140 2023-10-16T09:50:31.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.911+0000 7f386ed3e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38680cd1f0 con 0x7f3868159e90 2023-10-16T09:50:31.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.911+0000 7f386ed3e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f386815d890 con 0x7f3868159ab0 2023-10-16T09:50:31.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.912+0000 7f386d2db640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f386806c140 0x7f386815dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37914/0 (socket says 172.21.15.84:37914) 2023-10-16T09:50:31.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.912+0000 7f386d2db640 1 -- 172.21.15.84:0/904459273 learned_addr learned my addr 172.21.15.84:0/904459273 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:31.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.912+0000 7f386cada640 1 --2- 172.21.15.84:0/904459273 >> v2:172.21.15.157:3300/0 conn(0x7f3868159ab0 0x7f38680ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:31.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.912+0000 7f38677fe640 1 -- 172.21.15.84:0/904459273 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839749162 0 0) 0x7f3868070360 con 0x7f386806c140 2023-10-16T09:50:31.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.912+0000 7f38677fe640 1 -- 172.21.15.84:0/904459273 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3858003520 con 0x7f386806c140 2023-10-16T09:50:31.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.912+0000 7f38677fe640 1 -- 172.21.15.84:0/904459273 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 255814626 0 0) 0x7f3858003520 con 0x7f386806c140 2023-10-16T09:50:31.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.912+0000 7f38677fe640 1 -- 172.21.15.84:0/904459273 >> v2:172.21.15.157:3300/0 conn(0x7f3868159ab0 msgr2=0x7f38680ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:31.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.912+0000 7f38677fe640 1 --2- 172.21.15.84:0/904459273 >> v2:172.21.15.157:3300/0 conn(0x7f3868159ab0 0x7f38680ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.912+0000 7f38677fe640 1 -- 172.21.15.84:0/904459273 >> v1:172.21.15.157:6789/0 conn(0x7f3868159e90 legacy=0x7f386806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:31.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.912+0000 7f38677fe640 1 -- 172.21.15.84:0/904459273 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38680cd4d0 con 0x7f386806c140 2023-10-16T09:50:31.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.912+0000 7f38677fe640 1 -- 172.21.15.84:0/904459273 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f38600019f0 con 0x7f386806c140 2023-10-16T09:50:31.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.913+0000 7f38677fe640 1 -- 172.21.15.84:0/904459273 >> v1:172.21.15.84:6789/0 conn(0x7f386806c140 legacy=0x7f386815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:31.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.913+0000 7f38677fe640 1 --2- 172.21.15.84:0/904459273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3858003970 0x7f3858003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:31.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.913+0000 7f38677fe640 1 --2- 172.21.15.84:0/904459273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38580047c0 0x7f3858004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:31.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.913+0000 7f38677fe640 1 --2- 172.21.15.84:0/904459273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3858005380 0x7f385800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:31.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.913+0000 7f38677fe640 1 -- 172.21.15.84:0/904459273 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386815d890 con 0x7f3858005380 2023-10-16T09:50:31.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.913+0000 7f38677fe640 1 -- 172.21.15.84:0/904459273 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385800c210 con 0x7f38580047c0 2023-10-16T09:50:31.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.913+0000 7f38677fe640 1 -- 172.21.15.84:0/904459273 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385800c3d0 con 0x7f3858003970 2023-10-16T09:50:31.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.913+0000 7f386d2db640 1 --2- 172.21.15.84:0/904459273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3858005380 0x7f385800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:31.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.913+0000 7f386cada640 1 --2- 172.21.15.84:0/904459273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3858003970 0x7f3858003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:31.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.913+0000 7f3867fff640 1 --2- 172.21.15.84:0/904459273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38580047c0 0x7f3858004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:31.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.913+0000 7f386d2db640 1 -- 172.21.15.84:0/904459273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3858003970 msgr2=0x7f3858003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:31.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.913+0000 7f386d2db640 1 --2- 172.21.15.84:0/904459273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3858003970 0x7f3858003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.913+0000 7f386d2db640 1 -- 172.21.15.84:0/904459273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38580047c0 msgr2=0x7f3858004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:31.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.913+0000 7f386d2db640 1 --2- 172.21.15.84:0/904459273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38580047c0 0x7f3858004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.913+0000 7f386d2db640 1 -- 172.21.15.84:0/904459273 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3858003520 con 0x7f3858005380 2023-10-16T09:50:31.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.913+0000 7f3867fff640 1 --2- 172.21.15.84:0/904459273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38580047c0 0x7f3858004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:31.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.914+0000 7f386d2db640 1 -- 172.21.15.84:0/904459273 reap_dead start 2023-10-16T09:50:31.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.914+0000 7f386d2db640 1 --2- 172.21.15.84:0/904459273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3858005380 0x7f385800b980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f386000a830 tx=0x7f38580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9726bd2583ff43ed server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:31.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.915+0000 7f38677fe640 1 -- 172.21.15.84:0/904459273 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600072d0 con 0x7f3858005380 2023-10-16T09:50:31.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.915+0000 7f38677fe640 1 -- 172.21.15.84:0/904459273 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386002f070 con 0x7f3858005380 2023-10-16T09:50:31.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.915+0000 7f38677fe640 1 -- 172.21.15.84:0/904459273 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860007690 con 0x7f3858005380 2023-10-16T09:50:31.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.915+0000 7f386ed3e640 1 -- 172.21.15.84:0/904459273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3858005380 msgr2=0x7f385800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:31.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.915+0000 7f386ed3e640 1 --2- 172.21.15.84:0/904459273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3858005380 0x7f385800b980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f386000a830 tx=0x7f38580057f0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.915+0000 7f386ed3e640 1 -- 172.21.15.84:0/904459273 shutdown_connections 2023-10-16T09:50:31.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.915+0000 7f386ed3e640 1 --2- 172.21.15.84:0/904459273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3858005380 0x7f385800b980 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.915+0000 7f386ed3e640 1 -- 172.21.15.84:0/904459273 >> 172.21.15.84:0/904459273 conn(0x7f38680ce9b0 msgr2=0x7f38680cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:31.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.915+0000 7f386ed3e640 1 -- 172.21.15.84:0/904459273 shutdown_connections 2023-10-16T09:50:31.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.915+0000 7f386ed3e640 1 -- 172.21.15.84:0/904459273 wait complete. 2023-10-16T09:50:31.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.917+0000 7f386ed3e640 1 Processor -- start 2023-10-16T09:50:31.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.917+0000 7f386ed3e640 1 -- start start 2023-10-16T09:50:31.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.918+0000 7f386ed3e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3858005380 0x7f3868152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:31.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.918+0000 7f386ed3e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38681e2d80 0x7f38681e3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:31.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.918+0000 7f386ed3e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38681e3740 0x7f38680c3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:31.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.918+0000 7f386ed3e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38681e4830 con 0x7f3858005380 2023-10-16T09:50:31.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.918+0000 7f386ed3e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38680c34a0 con 0x7f38681e2d80 2023-10-16T09:50:31.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.918+0000 7f386ed3e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38681e3bc0 con 0x7f38681e3740 2023-10-16T09:50:31.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.918+0000 7f386cada640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3858005380 0x7f3868152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:31.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.918+0000 7f386cada640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3858005380 0x7f3868152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40658/0 (socket says 172.21.15.84:40658) 2023-10-16T09:50:31.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.918+0000 7f386cada640 1 -- 172.21.15.84:0/3359912828 learned_addr learned my addr 172.21.15.84:0/3359912828 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:31.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.919+0000 7f386cada640 1 -- 172.21.15.84:0/3359912828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38681e3740 msgr2=0x7f38680c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:31.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.919+0000 7f386cada640 1 --2- 172.21.15.84:0/3359912828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38681e3740 0x7f38680c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.919+0000 7f386cada640 1 -- 172.21.15.84:0/3359912828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38681e2d80 msgr2=0x7f38681e3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:31.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.919+0000 7f386cada640 1 --2- 172.21.15.84:0/3359912828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38681e2d80 0x7f38681e3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:31.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.919+0000 7f386cada640 1 -- 172.21.15.84:0/3359912828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38680c2890 con 0x7f3858005380 2023-10-16T09:50:31.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.919+0000 7f386cada640 1 --2- 172.21.15.84:0/3359912828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3858005380 0x7f3868152c50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f385c00f5e0 tx=0x7f385c00faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:31.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.919+0000 7f38657fa640 1 -- 172.21.15.84:0/3359912828 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c013810 con 0x7f3858005380 2023-10-16T09:50:31.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.919+0000 7f38657fa640 1 -- 172.21.15.84:0/3359912828 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c013970 con 0x7f3858005380 2023-10-16T09:50:31.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.919+0000 7f38657fa640 1 -- 172.21.15.84:0/3359912828 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c013ce0 con 0x7f3858005380 2023-10-16T09:50:31.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.919+0000 7f386ed3e640 1 -- 172.21.15.84:0/3359912828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38680c2b70 con 0x7f3858005380 2023-10-16T09:50:31.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.920+0000 7f386ed3e640 1 -- 172.21.15.84:0/3359912828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38680c3130 con 0x7f3858005380 2023-10-16T09:50:31.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.921+0000 7f386ed3e640 1 -- 172.21.15.84:0/3359912828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f386804e160 con 0x7f3858005380 2023-10-16T09:50:31.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.922+0000 7f38657fa640 1 -- 172.21.15.84:0/3359912828 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f385c01f460 con 0x7f3858005380 2023-10-16T09:50:31.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.923+0000 7f38657fa640 1 --2- 172.21.15.84:0/3359912828 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f384c06b070 0x7f384c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:31.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.923+0000 7f38657fa640 1 -- 172.21.15.84:0/3359912828 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f385c089ef0 con 0x7f3858005380 2023-10-16T09:50:31.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.923+0000 7f3867fff640 1 --2- 172.21.15.84:0/3359912828 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f384c06b070 0x7f384c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:31.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.924+0000 7f3867fff640 1 --2- 172.21.15.84:0/3359912828 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f384c06b070 0x7f384c06d530 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f38681e4a90 tx=0x7f3854009040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:31.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:31.927+0000 7f38657fa640 1 -- 172.21.15.84:0/3359912828 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f385c054b80 con 0x7f3858005380 2023-10-16T09:50:32.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.145+0000 7f386ed3e640 1 -- 172.21.15.84:0/3359912828 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir6", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f38680cda90 con 0x7f384c06b070 2023-10-16T09:50:32.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.316+0000 7f38657fa640 1 -- 172.21.15.84:0/3359912828 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f3854001330 con 0x7f384c06b070 2023-10-16T09:50:32.318 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path /snap_test_dir1/dir6 2023-10-16T09:50:32.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.319+0000 7f386ed3e640 1 -- 172.21.15.84:0/3359912828 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f384c06b070 msgr2=0x7f384c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:32.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.319+0000 7f386ed3e640 1 --2- 172.21.15.84:0/3359912828 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f384c06b070 0x7f384c06d530 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f38681e4a90 tx=0x7f3854009040 comp rx=0 tx=0).stop 2023-10-16T09:50:32.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.319+0000 7f386ed3e640 1 -- 172.21.15.84:0/3359912828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3858005380 msgr2=0x7f3868152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:32.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.319+0000 7f386ed3e640 1 --2- 172.21.15.84:0/3359912828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3858005380 0x7f3868152c50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f385c00f5e0 tx=0x7f385c00faa0 comp rx=0 tx=0).stop 2023-10-16T09:50:32.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.319+0000 7f386ed3e640 1 -- 172.21.15.84:0/3359912828 shutdown_connections 2023-10-16T09:50:32.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.319+0000 7f386ed3e640 1 --2- 172.21.15.84:0/3359912828 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f384c06b070 0x7f384c06d530 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:32.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.319+0000 7f386ed3e640 1 --2- 172.21.15.84:0/3359912828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38681e3740 0x7f38680c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:32.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.319+0000 7f386ed3e640 1 --2- 172.21.15.84:0/3359912828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38681e2d80 0x7f38681e3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:32.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.319+0000 7f386ed3e640 1 --2- 172.21.15.84:0/3359912828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3858005380 0x7f3868152c50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:32.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.319+0000 7f386ed3e640 1 -- 172.21.15.84:0/3359912828 >> 172.21.15.84:0/3359912828 conn(0x7f38680ce9b0 msgr2=0x7f386806b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:32.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.319+0000 7f386ed3e640 1 -- 172.21.15.84:0/3359912828 shutdown_connections 2023-10-16T09:50:32.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.320+0000 7f386ed3e640 1 -- 172.21.15.84:0/3359912828 wait complete. 2023-10-16T09:50:32.341 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-10-16T09:50:32.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:32.342 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-10-16T09:50:32.395 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir7 1m 2023-10-16T09:50:32.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.606+0000 7f0f2b59e640 1 Processor -- start 2023-10-16T09:50:32.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.606+0000 7f0f2b59e640 1 -- start start 2023-10-16T09:50:32.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.607+0000 7f0f2b59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0f2c070b60 0x7f0f2c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:32.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.607+0000 7f0f2b59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0f2c06c100 0x7f0f2c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:32.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.607+0000 7f0f2b59e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f2c15aef0 con 0x7f0f2c159950 2023-10-16T09:50:32.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.607+0000 7f0f2b59e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f2c15e490 con 0x7f0f2c06c100 2023-10-16T09:50:32.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.607+0000 7f0f2b59e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f2c15d910 con 0x7f0f2c070b60 2023-10-16T09:50:32.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.607+0000 7f0f2a59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0f2c159950 0x7f0f2c070450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37928/0 (socket says 172.21.15.84:37928) 2023-10-16T09:50:32.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.607+0000 7f0f2a59c640 1 -- 172.21.15.84:0/3016962470 learned_addr learned my addr 172.21.15.84:0/3016962470 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:32.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.607+0000 7f0f2ad9d640 1 --2- 172.21.15.84:0/3016962470 >> v2:172.21.15.84:3300/0 conn(0x7f0f2c06c100 0x7f0f2c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:32.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.608+0000 7f0f29d9b640 1 --2- 172.21.15.84:0/3016962470 >> v2:172.21.15.157:3300/0 conn(0x7f0f2c070b60 0x7f0f2c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:32.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.608+0000 7f0f2959a640 1 -- 172.21.15.84:0/3016962470 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 733757688 0 0) 0x7f0f14001110 con 0x7f0f2c159950 2023-10-16T09:50:32.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.608+0000 7f0f2959a640 1 -- 172.21.15.84:0/3016962470 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f18003520 con 0x7f0f2c159950 2023-10-16T09:50:32.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.608+0000 7f0f2ad9d640 1 -- 172.21.15.84:0/3016962470 >> v2:172.21.15.157:3300/0 conn(0x7f0f2c070b60 msgr2=0x7f0f2c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:32.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.608+0000 7f0f2ad9d640 1 --2- 172.21.15.84:0/3016962470 >> v2:172.21.15.157:3300/0 conn(0x7f0f2c070b60 0x7f0f2c15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:32.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.608+0000 7f0f2ad9d640 1 -- 172.21.15.84:0/3016962470 >> v1:172.21.15.84:6789/0 conn(0x7f0f2c159950 legacy=0x7f0f2c070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:32.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.608+0000 7f0f2ad9d640 1 -- 172.21.15.84:0/3016962470 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c15dbf0 con 0x7f0f2c06c100 2023-10-16T09:50:32.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.609+0000 7f0f2ad9d640 1 --2- 172.21.15.84:0/3016962470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c06c100 0x7f0f2c06c510 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0716a0 tx=0x7f0f2402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83369bc4dfb743f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:32.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.610+0000 7f0f2959a640 1 -- 172.21.15.84:0/3016962470 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2402e070 con 0x7f0f2c06c100 2023-10-16T09:50:32.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.610+0000 7f0f2959a640 1 -- 172.21.15.84:0/3016962470 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f240315a0 con 0x7f0f2c06c100 2023-10-16T09:50:32.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.610+0000 7f0f2959a640 1 -- 172.21.15.84:0/3016962470 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24031930 con 0x7f0f2c06c100 2023-10-16T09:50:32.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.610+0000 7f0f2b59e640 1 -- 172.21.15.84:0/3016962470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c06c100 msgr2=0x7f0f2c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:32.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.610+0000 7f0f2b59e640 1 --2- 172.21.15.84:0/3016962470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c06c100 0x7f0f2c06c510 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0716a0 tx=0x7f0f2402b840 comp rx=0 tx=0).stop 2023-10-16T09:50:32.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.610+0000 7f0f2b59e640 1 -- 172.21.15.84:0/3016962470 shutdown_connections 2023-10-16T09:50:32.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.610+0000 7f0f2b59e640 1 --2- 172.21.15.84:0/3016962470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c06c100 0x7f0f2c06c510 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:32.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.610+0000 7f0f2b59e640 1 --2- 172.21.15.84:0/3016962470 >> v2:172.21.15.157:3300/0 conn(0x7f0f2c070b60 0x7f0f2c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:32.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.610+0000 7f0f2b59e640 1 -- 172.21.15.84:0/3016962470 >> 172.21.15.84:0/3016962470 conn(0x7f0f2c0ce6a0 msgr2=0x7f0f2c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:32.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.610+0000 7f0f2b59e640 1 -- 172.21.15.84:0/3016962470 shutdown_connections 2023-10-16T09:50:32.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.610+0000 7f0f2b59e640 1 -- 172.21.15.84:0/3016962470 wait complete. 2023-10-16T09:50:32.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.612+0000 7f0f2b59e640 1 Processor -- start 2023-10-16T09:50:32.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.612+0000 7f0f2b59e640 1 -- start start 2023-10-16T09:50:32.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.613+0000 7f0f2b59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f2c06c100 0x7f0f2c0af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:32.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.613+0000 7f0f2b59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f2c070b60 0x7f0f2c0c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:32.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.613+0000 7f0f2b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c159950 0x7f0f2c0c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:32.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.613+0000 7f0f2b59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c14c900 con 0x7f0f2c159950 2023-10-16T09:50:32.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.613+0000 7f0f2b59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c0c8640 con 0x7f0f2c070b60 2023-10-16T09:50:32.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.613+0000 7f0f2b59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c14c2e0 con 0x7f0f2c06c100 2023-10-16T09:50:32.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.613+0000 7f0f2ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c159950 0x7f0f2c0c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:32.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.613+0000 7f0f2ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c159950 0x7f0f2c0c5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40676/0 (socket says 172.21.15.84:40676) 2023-10-16T09:50:32.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.613+0000 7f0f2ad9d640 1 -- 172.21.15.84:0/690396747 learned_addr learned my addr 172.21.15.84:0/690396747 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:32.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.614+0000 7f0f29d9b640 1 --2- 172.21.15.84:0/690396747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f2c070b60 0x7f0f2c0c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:32.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.614+0000 7f0f2ad9d640 1 -- 172.21.15.84:0/690396747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f2c06c100 msgr2=0x7f0f2c0af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:32.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.614+0000 7f0f2a59c640 1 --2- 172.21.15.84:0/690396747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f2c06c100 0x7f0f2c0af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:32.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.614+0000 7f0f2ad9d640 1 --2- 172.21.15.84:0/690396747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f2c06c100 0x7f0f2c0af1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:32.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.614+0000 7f0f2ad9d640 1 -- 172.21.15.84:0/690396747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f2c070b60 msgr2=0x7f0f2c0c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:32.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.614+0000 7f0f2ad9d640 1 --2- 172.21.15.84:0/690396747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f2c070b60 0x7f0f2c0c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:32.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.614+0000 7f0f2ad9d640 1 -- 172.21.15.84:0/690396747 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f24009400 con 0x7f0f2c159950 2023-10-16T09:50:32.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.614+0000 7f0f29d9b640 1 --2- 172.21.15.84:0/690396747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f2c070b60 0x7f0f2c0c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:32.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.614+0000 7f0f2ad9d640 1 --2- 172.21.15.84:0/690396747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c159950 0x7f0f2c0c5d30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009890 tx=0x7f0f240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:32.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.614+0000 7f0f22ffd640 1 -- 172.21.15.84:0/690396747 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24032840 con 0x7f0f2c159950 2023-10-16T09:50:32.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.615+0000 7f0f22ffd640 1 -- 172.21.15.84:0/690396747 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f240329a0 con 0x7f0f2c159950 2023-10-16T09:50:32.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.615+0000 7f0f2b59e640 1 -- 172.21.15.84:0/690396747 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2c0c8080 con 0x7f0f2c159950 2023-10-16T09:50:32.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.615+0000 7f0f22ffd640 1 -- 172.21.15.84:0/690396747 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24032d30 con 0x7f0f2c159950 2023-10-16T09:50:32.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.615+0000 7f0f2b59e640 1 -- 172.21.15.84:0/690396747 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2c14c520 con 0x7f0f2c159950 2023-10-16T09:50:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.616+0000 7f0f2b59e640 1 -- 172.21.15.84:0/690396747 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2c14ca40 con 0x7f0f2c159950 2023-10-16T09:50:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.617+0000 7f0f22ffd640 1 -- 172.21.15.84:0/690396747 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f0f2404e430 con 0x7f0f2c159950 2023-10-16T09:50:32.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.618+0000 7f0f22ffd640 1 --2- 172.21.15.84:0/690396747 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f0f0c06b070 0x7f0f0c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:32.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.618+0000 7f0f22ffd640 1 -- 172.21.15.84:0/690396747 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f0f240abb50 con 0x7f0f2c159950 2023-10-16T09:50:32.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.618+0000 7f0f2a59c640 1 --2- 172.21.15.84:0/690396747 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f0f0c06b070 0x7f0f0c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:32.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.619+0000 7f0f2a59c640 1 --2- 172.21.15.84:0/690396747 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f0f0c06b070 0x7f0f0c06d530 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0f14001370 tx=0x7f0f14009040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:32.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.622+0000 7f0f22ffd640 1 -- 172.21.15.84:0/690396747 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0f2c14ca40 con 0x7f0f2c159950 2023-10-16T09:50:32.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.834+0000 7f0f2b59e640 1 -- 172.21.15.84:0/690396747 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir7", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f2c15e1b0 con 0x7f0f0c06b070 2023-10-16T09:50:32.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.993+0000 7f0f22ffd640 1 -- 172.21.15.84:0/690396747 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f0f14005890 con 0x7f0f0c06b070 2023-10-16T09:50:32.995 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path /snap_test_dir1/dir7 2023-10-16T09:50:32.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.996+0000 7f0f2b59e640 1 -- 172.21.15.84:0/690396747 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f0f0c06b070 msgr2=0x7f0f0c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:32.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.996+0000 7f0f2b59e640 1 --2- 172.21.15.84:0/690396747 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f0f0c06b070 0x7f0f0c06d530 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0f14001370 tx=0x7f0f14009040 comp rx=0 tx=0).stop 2023-10-16T09:50:32.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.996+0000 7f0f2b59e640 1 -- 172.21.15.84:0/690396747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c159950 msgr2=0x7f0f2c0c5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:32.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.996+0000 7f0f2b59e640 1 --2- 172.21.15.84:0/690396747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c159950 0x7f0f2c0c5d30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009890 tx=0x7f0f240046c0 comp rx=0 tx=0).stop 2023-10-16T09:50:32.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.997+0000 7f0f2b59e640 1 -- 172.21.15.84:0/690396747 shutdown_connections 2023-10-16T09:50:33.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.997+0000 7f0f2b59e640 1 --2- 172.21.15.84:0/690396747 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f0f0c06b070 0x7f0f0c06d530 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:33.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.997+0000 7f0f2b59e640 1 --2- 172.21.15.84:0/690396747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c159950 0x7f0f2c0c5d30 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:33.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.997+0000 7f0f2b59e640 1 --2- 172.21.15.84:0/690396747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f2c070b60 0x7f0f2c0c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:33.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.997+0000 7f0f2b59e640 1 --2- 172.21.15.84:0/690396747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f2c06c100 0x7f0f2c0af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:33.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.997+0000 7f0f2b59e640 1 -- 172.21.15.84:0/690396747 >> 172.21.15.84:0/690396747 conn(0x7f0f2c0ce6a0 msgr2=0x7f0f2c070450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:33.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.997+0000 7f0f2b59e640 1 -- 172.21.15.84:0/690396747 shutdown_connections 2023-10-16T09:50:33.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:32.997+0000 7f0f2b59e640 1 -- 172.21.15.84:0/690396747 wait complete. 2023-10-16T09:50:33.019 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-10-16T09:50:33.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:33.020 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-10-16T09:50:33.071 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir8 1m 2023-10-16T09:50:33.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.330+0000 7ff72ffb0640 1 Processor -- start 2023-10-16T09:50:33.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.330+0000 7ff72ffb0640 1 -- start start 2023-10-16T09:50:33.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.332+0000 7ff72ffb0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff72815ad70 0x7ff728069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:33.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.332+0000 7ff72ffb0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff7280caac0 0x7ff72806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:33.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.332+0000 7ff72ffb0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7280cb030 con 0x7ff7280c9ca0 2023-10-16T09:50:33.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.332+0000 7ff72ffb0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7280c91c0 con 0x7ff7280caac0 2023-10-16T09:50:33.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.332+0000 7ff72ffb0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7280c94a0 con 0x7ff72815ad70 2023-10-16T09:50:33.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.332+0000 7ff72e54d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff7280caac0 0x7ff72806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:33.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.332+0000 7ff72e54d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff7280caac0 0x7ff72806a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40678/0 (socket says 172.21.15.84:40678) 2023-10-16T09:50:33.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.332+0000 7ff72e54d640 1 -- 172.21.15.84:0/1922692266 learned_addr learned my addr 172.21.15.84:0/1922692266 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:33.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.332+0000 7ff72dd4c640 1 --2- 172.21.15.84:0/1922692266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff72815ad70 0x7ff728069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:33.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.333+0000 7ff72e54d640 1 -- 172.21.15.84:0/1922692266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff72815ad70 msgr2=0x7ff728069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:33.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.333+0000 7ff72e54d640 1 --2- 172.21.15.84:0/1922692266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff72815ad70 0x7ff728069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:33.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.333+0000 7ff72e54d640 1 -- 172.21.15.84:0/1922692266 >> v1:172.21.15.157:6789/0 conn(0x7ff7280c9ca0 legacy=0x7ff7280ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:33.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.333+0000 7ff72e54d640 1 -- 172.21.15.84:0/1922692266 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7280c9780 con 0x7ff7280caac0 2023-10-16T09:50:33.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.334+0000 7ff72e54d640 1 --2- 172.21.15.84:0/1922692266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7280caac0 0x7ff72806a2e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff720009e50 tx=0x7ff72002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88a4414d8d1dbad server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:33.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.334+0000 7ff72cd4a640 1 -- 172.21.15.84:0/1922692266 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72002ca80 con 0x7ff7280caac0 2023-10-16T09:50:33.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.334+0000 7ff72cd4a640 1 -- 172.21.15.84:0/1922692266 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72002cbe0 con 0x7ff7280caac0 2023-10-16T09:50:33.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.334+0000 7ff72cd4a640 1 -- 172.21.15.84:0/1922692266 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7200343f0 con 0x7ff7280caac0 2023-10-16T09:50:33.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.335+0000 7ff72ffb0640 1 -- 172.21.15.84:0/1922692266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7280caac0 msgr2=0x7ff72806a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:33.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.335+0000 7ff72ffb0640 1 --2- 172.21.15.84:0/1922692266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7280caac0 0x7ff72806a2e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff720009e50 tx=0x7ff72002bee0 comp rx=0 tx=0).stop 2023-10-16T09:50:33.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.335+0000 7ff72ffb0640 1 -- 172.21.15.84:0/1922692266 shutdown_connections 2023-10-16T09:50:33.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.335+0000 7ff72ffb0640 1 --2- 172.21.15.84:0/1922692266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7280caac0 0x7ff72806a2e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:33.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.335+0000 7ff72ffb0640 1 --2- 172.21.15.84:0/1922692266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff72815ad70 0x7ff728069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:33.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.335+0000 7ff72ffb0640 1 -- 172.21.15.84:0/1922692266 >> 172.21.15.84:0/1922692266 conn(0x7ff7280ce9b0 msgr2=0x7ff7280cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:33.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.335+0000 7ff72ffb0640 1 -- 172.21.15.84:0/1922692266 shutdown_connections 2023-10-16T09:50:33.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.335+0000 7ff72ffb0640 1 -- 172.21.15.84:0/1922692266 wait complete. 2023-10-16T09:50:33.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.337+0000 7ff72ffb0640 1 Processor -- start 2023-10-16T09:50:33.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.337+0000 7ff72ffb0640 1 -- start start 2023-10-16T09:50:33.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.338+0000 7ff72ffb0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7280c9ca0 0x7ff7280c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:33.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.338+0000 7ff72ffb0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7280caac0 0x7ff728155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:33.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.338+0000 7ff72ffb0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff72815ad70 0x7ff72815dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:33.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.338+0000 7ff72ffb0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72815bc10 con 0x7ff72815ad70 2023-10-16T09:50:33.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.338+0000 7ff72ffb0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72815c670 con 0x7ff7280c9ca0 2023-10-16T09:50:33.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.338+0000 7ff72ffb0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72815b930 con 0x7ff7280caac0 2023-10-16T09:50:33.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.338+0000 7ff72e54d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff72815ad70 0x7ff72815dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:33.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.338+0000 7ff72e54d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff72815ad70 0x7ff72815dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40684/0 (socket says 172.21.15.84:40684) 2023-10-16T09:50:33.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.338+0000 7ff72e54d640 1 -- 172.21.15.84:0/4207038698 learned_addr learned my addr 172.21.15.84:0/4207038698 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:33.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.339+0000 7ff72e54d640 1 -- 172.21.15.84:0/4207038698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7280caac0 msgr2=0x7ff728155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:33.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.339+0000 7ff72e54d640 1 --2- 172.21.15.84:0/4207038698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7280caac0 0x7ff728155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:33.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.339+0000 7ff72e54d640 1 -- 172.21.15.84:0/4207038698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7280c9ca0 msgr2=0x7ff7280c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:33.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.339+0000 7ff72e54d640 1 --2- 172.21.15.84:0/4207038698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7280c9ca0 0x7ff7280c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:33.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.339+0000 7ff72e54d640 1 -- 172.21.15.84:0/4207038698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff720009b30 con 0x7ff72815ad70 2023-10-16T09:50:33.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.339+0000 7ff72e54d640 1 --2- 172.21.15.84:0/4207038698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff72815ad70 0x7ff72815dc40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff720009e50 tx=0x7ff720004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:33.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.339+0000 7ff71a7fc640 1 -- 172.21.15.84:0/4207038698 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720009510 con 0x7ff72815ad70 2023-10-16T09:50:33.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.339+0000 7ff71a7fc640 1 -- 172.21.15.84:0/4207038698 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72003e450 con 0x7ff72815ad70 2023-10-16T09:50:33.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.339+0000 7ff71a7fc640 1 -- 172.21.15.84:0/4207038698 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72003e800 con 0x7ff72815ad70 2023-10-16T09:50:33.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.339+0000 7ff72ffb0640 1 -- 172.21.15.84:0/4207038698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72815bfb0 con 0x7ff72815ad70 2023-10-16T09:50:33.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.339+0000 7ff72ffb0640 1 -- 172.21.15.84:0/4207038698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff728155580 con 0x7ff72815ad70 2023-10-16T09:50:33.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.340+0000 7ff72ffb0640 1 -- 172.21.15.84:0/4207038698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72815bde0 con 0x7ff72815ad70 2023-10-16T09:50:33.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.341+0000 7ff71a7fc640 1 -- 172.21.15.84:0/4207038698 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7ff72002cd50 con 0x7ff72815ad70 2023-10-16T09:50:33.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.343+0000 7ff71a7fc640 1 --2- 172.21.15.84:0/4207038698 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff70c06b070 0x7ff70c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:33.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.343+0000 7ff71a7fc640 1 -- 172.21.15.84:0/4207038698 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff7200ac6a0 con 0x7ff72815ad70 2023-10-16T09:50:33.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.343+0000 7ff72dd4c640 1 --2- 172.21.15.84:0/4207038698 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff70c06b070 0x7ff70c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:33.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.344+0000 7ff72dd4c640 1 --2- 172.21.15.84:0/4207038698 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff70c06b070 0x7ff70c06d530 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff724006fd0 tx=0x7ff724008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:33.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.347+0000 7ff71a7fc640 1 -- 172.21.15.84:0/4207038698 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff720077460 con 0x7ff72815ad70 2023-10-16T09:50:33.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.565+0000 7ff72ffb0640 1 -- 172.21.15.84:0/4207038698 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir8", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7280ad4d0 con 0x7ff70c06b070 2023-10-16T09:50:33.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.756+0000 7ff71a7fc640 1 -- 172.21.15.84:0/4207038698 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ff72400be20 con 0x7ff70c06b070 2023-10-16T09:50:33.758 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path /snap_test_dir1/dir8 2023-10-16T09:50:33.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.759+0000 7ff72ffb0640 1 -- 172.21.15.84:0/4207038698 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff70c06b070 msgr2=0x7ff70c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:33.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.759+0000 7ff72ffb0640 1 --2- 172.21.15.84:0/4207038698 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff70c06b070 0x7ff70c06d530 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff724006fd0 tx=0x7ff724008040 comp rx=0 tx=0).stop 2023-10-16T09:50:33.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.759+0000 7ff72ffb0640 1 -- 172.21.15.84:0/4207038698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff72815ad70 msgr2=0x7ff72815dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:33.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.759+0000 7ff72ffb0640 1 --2- 172.21.15.84:0/4207038698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff72815ad70 0x7ff72815dc40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff720009e50 tx=0x7ff720004a40 comp rx=0 tx=0).stop 2023-10-16T09:50:33.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.760+0000 7ff72ffb0640 1 -- 172.21.15.84:0/4207038698 shutdown_connections 2023-10-16T09:50:33.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.760+0000 7ff72ffb0640 1 --2- 172.21.15.84:0/4207038698 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff70c06b070 0x7ff70c06d530 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:33.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.760+0000 7ff72ffb0640 1 --2- 172.21.15.84:0/4207038698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff72815ad70 0x7ff72815dc40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:33.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.760+0000 7ff72ffb0640 1 --2- 172.21.15.84:0/4207038698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7280caac0 0x7ff728155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:33.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.760+0000 7ff72ffb0640 1 --2- 172.21.15.84:0/4207038698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7280c9ca0 0x7ff7280c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:33.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.760+0000 7ff72ffb0640 1 -- 172.21.15.84:0/4207038698 >> 172.21.15.84:0/4207038698 conn(0x7ff7280ce9b0 msgr2=0x7ff72806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:33.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.760+0000 7ff72ffb0640 1 -- 172.21.15.84:0/4207038698 shutdown_connections 2023-10-16T09:50:33.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:33.760+0000 7ff72ffb0640 1 -- 172.21.15.84:0/4207038698 wait complete. 2023-10-16T09:50:33.781 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-10-16T09:50:33.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:50:33.782 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-10-16T09:50:33.833 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir9 1m 2023-10-16T09:50:34.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.046+0000 7f89e50df640 1 Processor -- start 2023-10-16T09:50:34.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.046+0000 7f89e50df640 1 -- start start 2023-10-16T09:50:34.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.047+0000 7f89e50df640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f89e006c040 0x7f89e006c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:34.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.047+0000 7f89e50df640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f89e00cda80 0x7f89e015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:34.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.047+0000 7f89e50df640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e006c960 con 0x7f89e00ad970 2023-10-16T09:50:34.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.047+0000 7f89e50df640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e015b9d0 con 0x7f89e00cda80 2023-10-16T09:50:34.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.047+0000 7f89e50df640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e015bcb0 con 0x7f89e006c040 2023-10-16T09:50:34.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.048+0000 7f89e48de640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f89e00cda80 0x7f89e015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:34.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.048+0000 7f89df7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f89e00ad970 0x7f89e015e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37944/0 (socket says 172.21.15.84:37944) 2023-10-16T09:50:34.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.048+0000 7f89df7fe640 1 -- 172.21.15.84:0/1368760493 learned_addr learned my addr 172.21.15.84:0/1368760493 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:34.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.048+0000 7f89dffff640 1 --2- 172.21.15.84:0/1368760493 >> v2:172.21.15.157:3300/0 conn(0x7f89e006c040 0x7f89e006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:34.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.048+0000 7f89e48de640 1 -- 172.21.15.84:0/1368760493 >> v2:172.21.15.157:3300/0 conn(0x7f89e006c040 msgr2=0x7f89e006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:34.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.048+0000 7f89e48de640 1 --2- 172.21.15.84:0/1368760493 >> v2:172.21.15.157:3300/0 conn(0x7f89e006c040 0x7f89e006c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:34.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.048+0000 7f89e48de640 1 -- 172.21.15.84:0/1368760493 >> v1:172.21.15.84:6789/0 conn(0x7f89e00ad970 legacy=0x7f89e015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:34.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.048+0000 7f89e48de640 1 -- 172.21.15.84:0/1368760493 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e015bf90 con 0x7f89e00cda80 2023-10-16T09:50:34.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.048+0000 7f89deffd640 1 -- 172.21.15.84:0/1368760493 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169189248 0 0) 0x7f89e006c960 con 0x7f89e00ad970 2023-10-16T09:50:34.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.050+0000 7f89e48de640 1 --2- 172.21.15.84:0/1368760493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e00cda80 0x7f89e015ed90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f89e00cf350 tx=0x7f89d802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8492d707767d5233 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:34.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.050+0000 7f89deffd640 1 -- 172.21.15.84:0/1368760493 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d802cc80 con 0x7f89e00cda80 2023-10-16T09:50:34.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.050+0000 7f89deffd640 1 -- 172.21.15.84:0/1368760493 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d802cde0 con 0x7f89e00cda80 2023-10-16T09:50:34.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.050+0000 7f89deffd640 1 -- 172.21.15.84:0/1368760493 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8034660 con 0x7f89e00cda80 2023-10-16T09:50:34.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.050+0000 7f89e50df640 1 -- 172.21.15.84:0/1368760493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e00cda80 msgr2=0x7f89e015ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:34.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.050+0000 7f89e50df640 1 --2- 172.21.15.84:0/1368760493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e00cda80 0x7f89e015ed90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f89e00cf350 tx=0x7f89d802bed0 comp rx=0 tx=0).stop 2023-10-16T09:50:34.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.051+0000 7f89e50df640 1 -- 172.21.15.84:0/1368760493 shutdown_connections 2023-10-16T09:50:34.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.051+0000 7f89e50df640 1 --2- 172.21.15.84:0/1368760493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e00cda80 0x7f89e015ed90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:34.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.051+0000 7f89e50df640 1 --2- 172.21.15.84:0/1368760493 >> v2:172.21.15.157:3300/0 conn(0x7f89e006c040 0x7f89e006c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:34.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.051+0000 7f89e50df640 1 -- 172.21.15.84:0/1368760493 >> 172.21.15.84:0/1368760493 conn(0x7f89e00ce6a0 msgr2=0x7f89e00ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:34.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.051+0000 7f89e50df640 1 -- 172.21.15.84:0/1368760493 shutdown_connections 2023-10-16T09:50:34.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.051+0000 7f89e50df640 1 -- 172.21.15.84:0/1368760493 wait complete. 2023-10-16T09:50:34.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.052+0000 7f89e50df640 1 Processor -- start 2023-10-16T09:50:34.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.053+0000 7f89e50df640 1 -- start start 2023-10-16T09:50:34.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.053+0000 7f89e50df640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89e006c040 0x7f89e00c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:34.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.053+0000 7f89e50df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89e00ad970 0x7f89e00b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:34.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.053+0000 7f89e50df640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e00cda80 0x7f89e00acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:34.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.053+0000 7f89e50df640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e00c6a70 con 0x7f89e00cda80 2023-10-16T09:50:34.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.053+0000 7f89e50df640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e00c65e0 con 0x7f89e00ad970 2023-10-16T09:50:34.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.053+0000 7f89e50df640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e00c5cb0 con 0x7f89e006c040 2023-10-16T09:50:34.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.054+0000 7f89e48de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e00cda80 0x7f89e00acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:34.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.054+0000 7f89e48de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e00cda80 0x7f89e00acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40708/0 (socket says 172.21.15.84:40708) 2023-10-16T09:50:34.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.054+0000 7f89e48de640 1 -- 172.21.15.84:0/2264052407 learned_addr learned my addr 172.21.15.84:0/2264052407 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:34.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.054+0000 7f89df7fe640 1 --2- 172.21.15.84:0/2264052407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89e00ad970 0x7f89e00b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:34.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.054+0000 7f89dffff640 1 --2- 172.21.15.84:0/2264052407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89e006c040 0x7f89e00c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:34.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.054+0000 7f89e48de640 1 -- 172.21.15.84:0/2264052407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89e006c040 msgr2=0x7f89e00c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:34.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.054+0000 7f89e48de640 1 --2- 172.21.15.84:0/2264052407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89e006c040 0x7f89e00c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:34.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.054+0000 7f89e48de640 1 -- 172.21.15.84:0/2264052407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89e00ad970 msgr2=0x7f89e00b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:34.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.054+0000 7f89e48de640 1 --2- 172.21.15.84:0/2264052407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89e00ad970 0x7f89e00b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:34.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.054+0000 7f89e48de640 1 -- 172.21.15.84:0/2264052407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d8009a70 con 0x7f89e00cda80 2023-10-16T09:50:34.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.054+0000 7f89df7fe640 1 --2- 172.21.15.84:0/2264052407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89e00ad970 0x7f89e00b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:34.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.055+0000 7f89e48de640 1 --2- 172.21.15.84:0/2264052407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e00cda80 0x7f89e00acb50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009e80 tx=0x7f89d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:34.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.055+0000 7f89dcff9640 1 -- 172.21.15.84:0/2264052407 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d803e610 con 0x7f89e00cda80 2023-10-16T09:50:34.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.055+0000 7f89dcff9640 1 -- 172.21.15.84:0/2264052407 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d803e770 con 0x7f89e00cda80 2023-10-16T09:50:34.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.055+0000 7f89dcff9640 1 -- 172.21.15.84:0/2264052407 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d803eb60 con 0x7f89e00cda80 2023-10-16T09:50:34.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.055+0000 7f89e50df640 1 -- 172.21.15.84:0/2264052407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e00c5f90 con 0x7f89e00cda80 2023-10-16T09:50:34.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.055+0000 7f89e50df640 1 -- 172.21.15.84:0/2264052407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e00c8540 con 0x7f89e00cda80 2023-10-16T09:50:34.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.056+0000 7f89e50df640 1 -- 172.21.15.84:0/2264052407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e006b330 con 0x7f89e00cda80 2023-10-16T09:50:34.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.057+0000 7f89dcff9640 1 -- 172.21.15.84:0/2264052407 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f89d803b050 con 0x7f89e00cda80 2023-10-16T09:50:34.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.058+0000 7f89dcff9640 1 --2- 172.21.15.84:0/2264052407 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f89c006b0c0 0x7f89c006d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.058+0000 7f89dcff9640 1 -- 172.21.15.84:0/2264052407 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f89d80ac350 con 0x7f89e00cda80 2023-10-16T09:50:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.058+0000 7f89dffff640 1 --2- 172.21.15.84:0/2264052407 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f89c006b0c0 0x7f89c006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.059+0000 7f89dffff640 1 --2- 172.21.15.84:0/2264052407 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f89c006b0c0 0x7f89c006d580 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f89d0006fd0 tx=0x7f89d0008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.062+0000 7f89dcff9640 1 -- 172.21.15.84:0/2264052407 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f89d8077060 con 0x7f89e00cda80 2023-10-16T09:50:34.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.278+0000 7f89e50df640 1 -- 172.21.15.84:0/2264052407 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir9", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f89e00c5e20 con 0x7f89c006b0c0 2023-10-16T09:50:34.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.459+0000 7f89dcff9640 1 -- 172.21.15.84:0/2264052407 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f89e00c5e20 con 0x7f89c006b0c0 2023-10-16T09:50:34.462 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path /snap_test_dir1/dir9 2023-10-16T09:50:34.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.463+0000 7f89e50df640 1 -- 172.21.15.84:0/2264052407 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f89c006b0c0 msgr2=0x7f89c006d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:34.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.463+0000 7f89e50df640 1 --2- 172.21.15.84:0/2264052407 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f89c006b0c0 0x7f89c006d580 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f89d0006fd0 tx=0x7f89d0008040 comp rx=0 tx=0).stop 2023-10-16T09:50:34.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.463+0000 7f89e50df640 1 -- 172.21.15.84:0/2264052407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e00cda80 msgr2=0x7f89e00acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:34.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.463+0000 7f89e50df640 1 --2- 172.21.15.84:0/2264052407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e00cda80 0x7f89e00acb50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009e80 tx=0x7f89d8004a40 comp rx=0 tx=0).stop 2023-10-16T09:50:34.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.463+0000 7f89e50df640 1 -- 172.21.15.84:0/2264052407 shutdown_connections 2023-10-16T09:50:34.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.463+0000 7f89e50df640 1 --2- 172.21.15.84:0/2264052407 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f89c006b0c0 0x7f89c006d580 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:34.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.463+0000 7f89e50df640 1 --2- 172.21.15.84:0/2264052407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e00cda80 0x7f89e00acb50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:34.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.463+0000 7f89e50df640 1 --2- 172.21.15.84:0/2264052407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89e00ad970 0x7f89e00b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:34.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.463+0000 7f89e50df640 1 --2- 172.21.15.84:0/2264052407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89e006c040 0x7f89e00c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:34.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.463+0000 7f89e50df640 1 -- 172.21.15.84:0/2264052407 >> 172.21.15.84:0/2264052407 conn(0x7f89e00ce6a0 msgr2=0x7f89e015da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:34.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.463+0000 7f89e50df640 1 -- 172.21.15.84:0/2264052407 shutdown_connections 2023-10-16T09:50:34.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.463+0000 7f89e50df640 1 -- 172.21.15.84:0/2264052407 wait complete. 2023-10-16T09:50:34.486 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-10-16T09:50:34.487 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir0 1m 2023-10-16T09:50:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.701+0000 7f18a911d640 1 Processor -- start 2023-10-16T09:50:34.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.701+0000 7f18a911d640 1 -- start start 2023-10-16T09:50:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.703+0000 7f18a911d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18a406b660 0x7f18a406ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.703+0000 7f18a911d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18a41428c0 0x7f18a40c9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:34.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.703+0000 7f18a911d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a40cc6b0 con 0x7f18a40c9690 2023-10-16T09:50:34.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.703+0000 7f18a911d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a40c9b20 con 0x7f18a41428c0 2023-10-16T09:50:34.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.703+0000 7f18a911d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a41599b0 con 0x7f18a406b660 2023-10-16T09:50:34.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.703+0000 7f189bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18a41428c0 0x7f18a40c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:34.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.703+0000 7f189bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18a41428c0 0x7f18a40c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40720/0 (socket says 172.21.15.84:40720) 2023-10-16T09:50:34.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.703+0000 7f189bfff640 1 -- 172.21.15.84:0/2383707183 learned_addr learned my addr 172.21.15.84:0/2383707183 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:34.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.704+0000 7f18a2d9d640 1 --2- 172.21.15.84:0/2383707183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18a406b660 0x7f18a406ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:34.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.704+0000 7f189bfff640 1 -- 172.21.15.84:0/2383707183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18a406b660 msgr2=0x7f18a406ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:34.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.704+0000 7f189bfff640 1 --2- 172.21.15.84:0/2383707183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18a406b660 0x7f18a406ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:34.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.704+0000 7f189bfff640 1 -- 172.21.15.84:0/2383707183 >> v1:172.21.15.157:6789/0 conn(0x7f18a40c9690 legacy=0x7f18a415d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:34.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.704+0000 7f189bfff640 1 -- 172.21.15.84:0/2383707183 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a406c290 con 0x7f18a41428c0 2023-10-16T09:50:34.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.705+0000 7f189bfff640 1 --2- 172.21.15.84:0/2383707183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18a41428c0 0x7f18a40c9150 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f188c009870 tx=0x7f188c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39dd4f8479aa8874 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:34.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.706+0000 7f18a259c640 1 -- 172.21.15.84:0/2383707183 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c030840 con 0x7f18a41428c0 2023-10-16T09:50:34.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.706+0000 7f18a259c640 1 -- 172.21.15.84:0/2383707183 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c0309a0 con 0x7f18a41428c0 2023-10-16T09:50:34.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.706+0000 7f18a259c640 1 -- 172.21.15.84:0/2383707183 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c030d30 con 0x7f18a41428c0 2023-10-16T09:50:34.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.706+0000 7f18a911d640 1 -- 172.21.15.84:0/2383707183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18a41428c0 msgr2=0x7f18a40c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:34.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.706+0000 7f18a911d640 1 --2- 172.21.15.84:0/2383707183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18a41428c0 0x7f18a40c9150 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f188c009870 tx=0x7f188c02b980 comp rx=0 tx=0).stop 2023-10-16T09:50:34.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.707+0000 7f18a911d640 1 -- 172.21.15.84:0/2383707183 shutdown_connections 2023-10-16T09:50:34.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.707+0000 7f18a911d640 1 --2- 172.21.15.84:0/2383707183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18a41428c0 0x7f18a40c9150 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:34.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.707+0000 7f18a911d640 1 --2- 172.21.15.84:0/2383707183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18a406b660 0x7f18a406ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:34.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.707+0000 7f18a911d640 1 -- 172.21.15.84:0/2383707183 >> 172.21.15.84:0/2383707183 conn(0x7f18a40ce9b0 msgr2=0x7f18a40cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:34.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.707+0000 7f18a911d640 1 -- 172.21.15.84:0/2383707183 shutdown_connections 2023-10-16T09:50:34.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.707+0000 7f18a911d640 1 -- 172.21.15.84:0/2383707183 wait complete. 2023-10-16T09:50:34.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.708+0000 7f18a911d640 1 Processor -- start 2023-10-16T09:50:34.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.709+0000 7f18a911d640 1 -- start start 2023-10-16T09:50:34.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.709+0000 7f18a911d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18a406b660 0x7f18a41518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:34.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.709+0000 7f18a911d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18a40c9690 0x7f18a40c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:34.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.709+0000 7f18a911d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18a41428c0 0x7f18a40c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:34.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.709+0000 7f18a911d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a40c28c0 con 0x7f18a406b660 2023-10-16T09:50:34.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.709+0000 7f18a911d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a40c13f0 con 0x7f18a40c9690 2023-10-16T09:50:34.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.709+0000 7f18a911d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a40c7e90 con 0x7f18a41428c0 2023-10-16T09:50:34.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.710+0000 7f18a2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18a406b660 0x7f18a41518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:34.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.710+0000 7f18a2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18a406b660 0x7f18a41518e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40726/0 (socket says 172.21.15.84:40726) 2023-10-16T09:50:34.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.710+0000 7f18a2d9d640 1 -- 172.21.15.84:0/3199429984 learned_addr learned my addr 172.21.15.84:0/3199429984 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:34.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.710+0000 7f189bfff640 1 --2- 172.21.15.84:0/3199429984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18a40c9690 0x7f18a40c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:34.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.710+0000 7f18a359e640 1 --2- 172.21.15.84:0/3199429984 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18a41428c0 0x7f18a40c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:34.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.710+0000 7f18a2d9d640 1 -- 172.21.15.84:0/3199429984 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18a41428c0 msgr2=0x7f18a40c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:34.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.710+0000 7f18a2d9d640 1 --2- 172.21.15.84:0/3199429984 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18a41428c0 0x7f18a40c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:34.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.710+0000 7f18a2d9d640 1 -- 172.21.15.84:0/3199429984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18a40c9690 msgr2=0x7f18a40c44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:34.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.710+0000 7f18a2d9d640 1 --2- 172.21.15.84:0/3199429984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18a40c9690 0x7f18a40c44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:34.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.710+0000 7f18a2d9d640 1 -- 172.21.15.84:0/3199429984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c0094d0 con 0x7f18a406b660 2023-10-16T09:50:34.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.711+0000 7f18a2d9d640 1 --2- 172.21.15.84:0/3199429984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18a406b660 0x7f18a41518e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f189400d5d0 tx=0x7f189400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:34.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.711+0000 7f189b7fe640 1 -- 172.21.15.84:0/3199429984 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18940125f0 con 0x7f18a406b660 2023-10-16T09:50:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.711+0000 7f189b7fe640 1 -- 172.21.15.84:0/3199429984 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1894012770 con 0x7f18a406b660 2023-10-16T09:50:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.711+0000 7f18a911d640 1 -- 172.21.15.84:0/3199429984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a40c8170 con 0x7f18a406b660 2023-10-16T09:50:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.711+0000 7f189b7fe640 1 -- 172.21.15.84:0/3199429984 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1894012b20 con 0x7f18a406b660 2023-10-16T09:50:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.711+0000 7f18a911d640 1 -- 172.21.15.84:0/3199429984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a40c24e0 con 0x7f18a406b660 2023-10-16T09:50:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.712+0000 7f18a911d640 1 -- 172.21.15.84:0/3199429984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a404e160 con 0x7f18a406b660 2023-10-16T09:50:34.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.713+0000 7f189b7fe640 1 -- 172.21.15.84:0/3199429984 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f1894012c80 con 0x7f18a406b660 2023-10-16T09:50:34.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.714+0000 7f189b7fe640 1 --2- 172.21.15.84:0/3199429984 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f188406b0c0 0x7f188406d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:34.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.714+0000 7f189b7fe640 1 -- 172.21.15.84:0/3199429984 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f1894089f70 con 0x7f18a406b660 2023-10-16T09:50:34.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.715+0000 7f189bfff640 1 --2- 172.21.15.84:0/3199429984 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f188406b0c0 0x7f188406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:34.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.715+0000 7f189bfff640 1 --2- 172.21.15.84:0/3199429984 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f188406b0c0 0x7f188406d580 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f188c036000 tx=0x7f188c008030 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.719+0000 7f189b7fe640 1 -- 172.21.15.84:0/3199429984 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1894054d10 con 0x7f18a406b660 2023-10-16T09:50:34.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:34.935+0000 7f18a911d640 1 -- 172.21.15.84:0/3199429984 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f18a406c850 con 0x7f188406b0c0 2023-10-16T09:50:35.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.079+0000 7f189b7fe640 1 -- 172.21.15.84:0/3199429984 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f188c030b90 con 0x7f188406b0c0 2023-10-16T09:50:35.082 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-10-16T09:50:35.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.084+0000 7f18a911d640 1 -- 172.21.15.84:0/3199429984 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f188406b0c0 msgr2=0x7f188406d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:35.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.084+0000 7f18a911d640 1 --2- 172.21.15.84:0/3199429984 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f188406b0c0 0x7f188406d580 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f188c036000 tx=0x7f188c008030 comp rx=0 tx=0).stop 2023-10-16T09:50:35.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.084+0000 7f18a911d640 1 -- 172.21.15.84:0/3199429984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18a406b660 msgr2=0x7f18a41518e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:35.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.084+0000 7f18a911d640 1 --2- 172.21.15.84:0/3199429984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18a406b660 0x7f18a41518e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f189400d5d0 tx=0x7f189400da90 comp rx=0 tx=0).stop 2023-10-16T09:50:35.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.084+0000 7f18a911d640 1 -- 172.21.15.84:0/3199429984 shutdown_connections 2023-10-16T09:50:35.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.084+0000 7f18a911d640 1 --2- 172.21.15.84:0/3199429984 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f188406b0c0 0x7f188406d580 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.084+0000 7f18a911d640 1 --2- 172.21.15.84:0/3199429984 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18a41428c0 0x7f18a40c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.084+0000 7f18a911d640 1 --2- 172.21.15.84:0/3199429984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18a40c9690 0x7f18a40c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.084+0000 7f18a911d640 1 --2- 172.21.15.84:0/3199429984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18a406b660 0x7f18a41518e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.084+0000 7f18a911d640 1 -- 172.21.15.84:0/3199429984 >> 172.21.15.84:0/3199429984 conn(0x7f18a40ce9b0 msgr2=0x7f18a4070770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:35.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.084+0000 7f18a911d640 1 -- 172.21.15.84:0/3199429984 shutdown_connections 2023-10-16T09:50:35.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.084+0000 7f18a911d640 1 -- 172.21.15.84:0/3199429984 wait complete. 2023-10-16T09:50:35.108 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-10-16T09:50:35.109 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir1 1m 2023-10-16T09:50:35.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.326+0000 7f8808268640 1 Processor -- start 2023-10-16T09:50:35.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.327+0000 7f8808268640 1 -- start start 2023-10-16T09:50:35.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.328+0000 7f8808268640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8800159ab0 0x7f88000ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:35.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.328+0000 7f8808268640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f880006c140 0x7f880006c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:35.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.328+0000 7f8808268640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f880006cad0 con 0x7f8800159e90 2023-10-16T09:50:35.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.328+0000 7f8808268640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f880015d890 con 0x7f8800159ab0 2023-10-16T09:50:35.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.328+0000 7f8808268640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8800070360 con 0x7f880006c140 2023-10-16T09:50:35.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.328+0000 7f8805803640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8800159e90 0x7f880006fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37952/0 (socket says 172.21.15.84:37952) 2023-10-16T09:50:35.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.328+0000 7f8805803640 1 -- 172.21.15.84:0/1719869311 learned_addr learned my addr 172.21.15.84:0/1719869311 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:35.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.328+0000 7f8806004640 1 --2- 172.21.15.84:0/1719869311 >> v2:172.21.15.84:3300/0 conn(0x7f8800159ab0 0x7f88000ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:35.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.329+0000 7f8806805640 1 --2- 172.21.15.84:0/1719869311 >> v2:172.21.15.157:3300/0 conn(0x7f880006c140 0x7f880006c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:35.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.329+0000 7f8805002640 1 -- 172.21.15.84:0/1719869311 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599270483 0 0) 0x7f87f4001110 con 0x7f8800159e90 2023-10-16T09:50:35.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.329+0000 7f8805002640 1 -- 172.21.15.84:0/1719869311 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87e8003520 con 0x7f8800159e90 2023-10-16T09:50:35.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.329+0000 7f8806004640 1 -- 172.21.15.84:0/1719869311 >> v2:172.21.15.157:3300/0 conn(0x7f880006c140 msgr2=0x7f880006c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:35.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.329+0000 7f8806004640 1 --2- 172.21.15.84:0/1719869311 >> v2:172.21.15.157:3300/0 conn(0x7f880006c140 0x7f880006c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.329+0000 7f8806004640 1 -- 172.21.15.84:0/1719869311 >> v1:172.21.15.84:6789/0 conn(0x7f8800159e90 legacy=0x7f880006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:35.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.329+0000 7f8806004640 1 -- 172.21.15.84:0/1719869311 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88000cd1f0 con 0x7f8800159ab0 2023-10-16T09:50:35.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.330+0000 7f8806004640 1 --2- 172.21.15.84:0/1719869311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8800159ab0 0x7f88000ce370 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f88000adcd0 tx=0x7f87f002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cda439ea7b35f37 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:35.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.331+0000 7f8805002640 1 -- 172.21.15.84:0/1719869311 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f002cb60 con 0x7f8800159ab0 2023-10-16T09:50:35.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.331+0000 7f8805002640 1 -- 172.21.15.84:0/1719869311 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f002ccc0 con 0x7f8800159ab0 2023-10-16T09:50:35.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.331+0000 7f8805002640 1 -- 172.21.15.84:0/1719869311 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0034660 con 0x7f8800159ab0 2023-10-16T09:50:35.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.331+0000 7f8808268640 1 -- 172.21.15.84:0/1719869311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8800159ab0 msgr2=0x7f88000ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:35.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.331+0000 7f8808268640 1 --2- 172.21.15.84:0/1719869311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8800159ab0 0x7f88000ce370 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f88000adcd0 tx=0x7f87f002bdb0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.332+0000 7f8808268640 1 -- 172.21.15.84:0/1719869311 shutdown_connections 2023-10-16T09:50:35.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.332+0000 7f8808268640 1 --2- 172.21.15.84:0/1719869311 >> v2:172.21.15.157:3300/0 conn(0x7f880006c140 0x7f880006c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.332+0000 7f8808268640 1 --2- 172.21.15.84:0/1719869311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8800159ab0 0x7f88000ce370 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.332+0000 7f8808268640 1 -- 172.21.15.84:0/1719869311 >> 172.21.15.84:0/1719869311 conn(0x7f88000ce9b0 msgr2=0x7f88000cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:35.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.332+0000 7f8808268640 1 -- 172.21.15.84:0/1719869311 shutdown_connections 2023-10-16T09:50:35.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.332+0000 7f8808268640 1 -- 172.21.15.84:0/1719869311 wait complete. 2023-10-16T09:50:35.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.333+0000 7f8808268640 1 Processor -- start 2023-10-16T09:50:35.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.334+0000 7f8808268640 1 -- start start 2023-10-16T09:50:35.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.334+0000 7f8808268640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880006c140 0x7f8800153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:35.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.334+0000 7f8808268640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8800159ab0 0x7f88000b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:35.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.335+0000 7f8808268640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8800159e90 0x7f88000b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:35.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.335+0000 7f8808268640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880015ca60 con 0x7f8800159e90 2023-10-16T09:50:35.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.335+0000 7f8808268640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88001512b0 con 0x7f8800159ab0 2023-10-16T09:50:35.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.335+0000 7f8808268640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880015ccb0 con 0x7f880006c140 2023-10-16T09:50:35.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.335+0000 7f8806805640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8800159e90 0x7f88000b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:35.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.335+0000 7f8806805640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8800159e90 0x7f88000b09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40738/0 (socket says 172.21.15.84:40738) 2023-10-16T09:50:35.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.335+0000 7f8806805640 1 -- 172.21.15.84:0/2776761412 learned_addr learned my addr 172.21.15.84:0/2776761412 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:35.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.335+0000 7f8806004640 1 --2- 172.21.15.84:0/2776761412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880006c140 0x7f8800153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:35.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.335+0000 7f8805803640 1 --2- 172.21.15.84:0/2776761412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8800159ab0 0x7f88000b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:35.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.335+0000 7f8806805640 1 -- 172.21.15.84:0/2776761412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880006c140 msgr2=0x7f8800153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:35.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.335+0000 7f8806805640 1 --2- 172.21.15.84:0/2776761412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880006c140 0x7f8800153140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.335+0000 7f8806805640 1 -- 172.21.15.84:0/2776761412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8800159ab0 msgr2=0x7f88000b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:35.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.335+0000 7f8806805640 1 --2- 172.21.15.84:0/2776761412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8800159ab0 0x7f88000b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.335+0000 7f8806805640 1 -- 172.21.15.84:0/2776761412 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f0009a70 con 0x7f8800159e90 2023-10-16T09:50:35.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.335+0000 7f8806004640 1 --2- 172.21.15.84:0/2776761412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880006c140 0x7f8800153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:35.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.335+0000 7f8805803640 1 --2- 172.21.15.84:0/2776761412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8800159ab0 0x7f88000b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:35.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.336+0000 7f8806805640 1 --2- 172.21.15.84:0/2776761412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8800159e90 0x7f88000b09c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f87f800d600 tx=0x7f87f800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:35.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.336+0000 7f87feffd640 1 -- 172.21.15.84:0/2776761412 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f80125f0 con 0x7f8800159e90 2023-10-16T09:50:35.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.336+0000 7f87feffd640 1 -- 172.21.15.84:0/2776761412 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f8012770 con 0x7f8800159e90 2023-10-16T09:50:35.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.336+0000 7f8808268640 1 -- 172.21.15.84:0/2776761412 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880015cf90 con 0x7f8800159e90 2023-10-16T09:50:35.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.336+0000 7f87feffd640 1 -- 172.21.15.84:0/2776761412 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8012b20 con 0x7f8800159e90 2023-10-16T09:50:35.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.336+0000 7f8808268640 1 -- 172.21.15.84:0/2776761412 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8800151a20 con 0x7f8800159e90 2023-10-16T09:50:35.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.337+0000 7f8808268640 1 -- 172.21.15.84:0/2776761412 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f880015d1d0 con 0x7f8800159e90 2023-10-16T09:50:35.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.338+0000 7f87feffd640 1 -- 172.21.15.84:0/2776761412 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f87f8012c80 con 0x7f8800159e90 2023-10-16T09:50:35.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.339+0000 7f87feffd640 1 --2- 172.21.15.84:0/2776761412 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f87ec06b0a0 0x7f87ec06d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:35.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.339+0000 7f87feffd640 1 -- 172.21.15.84:0/2776761412 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f87f8089f40 con 0x7f8800159e90 2023-10-16T09:50:35.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.339+0000 7f8806004640 1 --2- 172.21.15.84:0/2776761412 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f87ec06b0a0 0x7f87ec06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:35.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.340+0000 7f8806004640 1 --2- 172.21.15.84:0/2776761412 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f87ec06b0a0 0x7f87ec06d560 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f87f0036000 tx=0x7f87f0030000 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:35.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.343+0000 7f87feffd640 1 -- 172.21.15.84:0/2776761412 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f87f8054c80 con 0x7f8800159e90 2023-10-16T09:50:35.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.557+0000 7f8808268640 1 -- 172.21.15.84:0/2776761412 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f88000cd7b0 con 0x7f87ec06b0a0 2023-10-16T09:50:35.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.690+0000 7f87feffd640 1 -- 172.21.15.84:0/2776761412 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f87f002cb10 con 0x7f87ec06b0a0 2023-10-16T09:50:35.692 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-10-16T09:50:35.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.693+0000 7f8808268640 1 -- 172.21.15.84:0/2776761412 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f87ec06b0a0 msgr2=0x7f87ec06d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:35.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.693+0000 7f8808268640 1 --2- 172.21.15.84:0/2776761412 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f87ec06b0a0 0x7f87ec06d560 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f87f0036000 tx=0x7f87f0030000 comp rx=0 tx=0).stop 2023-10-16T09:50:35.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.693+0000 7f8808268640 1 -- 172.21.15.84:0/2776761412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8800159e90 msgr2=0x7f88000b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:35.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.693+0000 7f8808268640 1 --2- 172.21.15.84:0/2776761412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8800159e90 0x7f88000b09c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f87f800d600 tx=0x7f87f800dac0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.693+0000 7f8808268640 1 -- 172.21.15.84:0/2776761412 shutdown_connections 2023-10-16T09:50:35.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.693+0000 7f8808268640 1 --2- 172.21.15.84:0/2776761412 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f87ec06b0a0 0x7f87ec06d560 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.693+0000 7f8808268640 1 --2- 172.21.15.84:0/2776761412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8800159e90 0x7f88000b09c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.693+0000 7f8808268640 1 --2- 172.21.15.84:0/2776761412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8800159ab0 0x7f88000b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.693+0000 7f8808268640 1 --2- 172.21.15.84:0/2776761412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880006c140 0x7f8800153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.693+0000 7f8808268640 1 -- 172.21.15.84:0/2776761412 >> 172.21.15.84:0/2776761412 conn(0x7f88000ce9b0 msgr2=0x7f88000ae480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:35.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.693+0000 7f8808268640 1 -- 172.21.15.84:0/2776761412 shutdown_connections 2023-10-16T09:50:35.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.693+0000 7f8808268640 1 -- 172.21.15.84:0/2776761412 wait complete. 2023-10-16T09:50:35.716 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-10-16T09:50:35.716 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir2 1m 2023-10-16T09:50:35.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.928+0000 7efe4d4b4640 1 Processor -- start 2023-10-16T09:50:35.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.928+0000 7efe4d4b4640 1 -- start start 2023-10-16T09:50:35.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.929+0000 7efe4d4b4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efe480addb0 0x7efe480caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:35.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.929+0000 7efe4d4b4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe480c9000 0x7efe48069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:35.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.929+0000 7efe4d4b4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe480cb140 con 0x7efe4806a2e0 2023-10-16T09:50:35.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.929+0000 7efe4d4b4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe48161a50 con 0x7efe480addb0 2023-10-16T09:50:35.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.929+0000 7efe4d4b4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4806a750 con 0x7efe480c9000 2023-10-16T09:50:35.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.930+0000 7efe46ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efe480addb0 0x7efe480caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:35.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.930+0000 7efe467fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe480c9000 0x7efe48069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:35.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.930+0000 7efe46ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efe480addb0 0x7efe480caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:54384/0 (socket says 172.21.15.84:54384) 2023-10-16T09:50:35.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.930+0000 7efe467fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe480c9000 0x7efe48069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:52594/0 (socket says 172.21.15.84:52594) 2023-10-16T09:50:35.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.930+0000 7efe46ffd640 1 -- 172.21.15.84:0/2975946099 learned_addr learned my addr 172.21.15.84:0/2975946099 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:35.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.930+0000 7efe477fe640 1 --1- 172.21.15.84:0/2975946099 >> v1:172.21.15.157:6789/0 conn(0x7efe4806a2e0 0x7efe480cc580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:38042/0 (socket says 172.21.15.84:38042) 2023-10-16T09:50:35.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.931+0000 7efe46ffd640 1 -- 172.21.15.84:0/2975946099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe480c9000 msgr2=0x7efe48069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:35.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.931+0000 7efe46ffd640 1 --2- 172.21.15.84:0/2975946099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe480c9000 0x7efe48069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.931+0000 7efe46ffd640 1 -- 172.21.15.84:0/2975946099 >> v1:172.21.15.157:6789/0 conn(0x7efe4806a2e0 legacy=0x7efe480cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:35.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.931+0000 7efe46ffd640 1 -- 172.21.15.84:0/2975946099 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4806b030 con 0x7efe480addb0 2023-10-16T09:50:35.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.931+0000 7efe45ffb640 1 -- 172.21.15.84:0/2975946099 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197772884 0 0) 0x7efe480cb140 con 0x7efe4806a2e0 2023-10-16T09:50:35.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.931+0000 7efe467fc640 1 --2- 172.21.15.84:0/2975946099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe480c9000 0x7efe48069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:50:35.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.932+0000 7efe46ffd640 1 --2- 172.21.15.84:0/2975946099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe480addb0 0x7efe480caa80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efe300098a0 tx=0x7efe3002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4622872f7129b0c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:35.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.933+0000 7efe45ffb640 1 -- 172.21.15.84:0/2975946099 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30030840 con 0x7efe480addb0 2023-10-16T09:50:35.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.933+0000 7efe45ffb640 1 -- 172.21.15.84:0/2975946099 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe300309a0 con 0x7efe480addb0 2023-10-16T09:50:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.933+0000 7efe45ffb640 1 -- 172.21.15.84:0/2975946099 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30030d30 con 0x7efe480addb0 2023-10-16T09:50:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.934+0000 7efe4d4b4640 1 -- 172.21.15.84:0/2975946099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe480addb0 msgr2=0x7efe480caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.934+0000 7efe4d4b4640 1 --2- 172.21.15.84:0/2975946099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe480addb0 0x7efe480caa80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efe300098a0 tx=0x7efe3002b980 comp rx=0 tx=0).stop 2023-10-16T09:50:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.934+0000 7efe4d4b4640 1 -- 172.21.15.84:0/2975946099 shutdown_connections 2023-10-16T09:50:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.934+0000 7efe4d4b4640 1 --2- 172.21.15.84:0/2975946099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe480c9000 0x7efe48069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.934+0000 7efe4d4b4640 1 --2- 172.21.15.84:0/2975946099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe480addb0 0x7efe480caa80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.934+0000 7efe4d4b4640 1 -- 172.21.15.84:0/2975946099 >> 172.21.15.84:0/2975946099 conn(0x7efe480ce9b0 msgr2=0x7efe480cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:35.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.934+0000 7efe4d4b4640 1 -- 172.21.15.84:0/2975946099 shutdown_connections 2023-10-16T09:50:35.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.934+0000 7efe4d4b4640 1 -- 172.21.15.84:0/2975946099 wait complete. 2023-10-16T09:50:35.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.936+0000 7efe4d4b4640 1 Processor -- start 2023-10-16T09:50:35.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.936+0000 7efe4d4b4640 1 -- start start 2023-10-16T09:50:35.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.937+0000 7efe4d4b4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe4806a2e0 0x7efe480ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:35.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.937+0000 7efe4d4b4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe480addb0 0x7efe481f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:35.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.937+0000 7efe4d4b4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe480c9000 0x7efe480b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:35.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.937+0000 7efe4d4b4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe481f7260 con 0x7efe480c9000 2023-10-16T09:50:35.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.937+0000 7efe4d4b4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe481646a0 con 0x7efe4806a2e0 2023-10-16T09:50:35.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.937+0000 7efe4d4b4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe48164980 con 0x7efe480addb0 2023-10-16T09:50:35.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.938+0000 7efe477fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe480c9000 0x7efe480b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:35.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.938+0000 7efe477fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe480c9000 0x7efe480b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43656/0 (socket says 172.21.15.84:43656) 2023-10-16T09:50:35.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.938+0000 7efe477fe640 1 -- 172.21.15.84:0/3512998542 learned_addr learned my addr 172.21.15.84:0/3512998542 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:35.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.938+0000 7efe46ffd640 1 --2- 172.21.15.84:0/3512998542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe4806a2e0 0x7efe480ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:35.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.938+0000 7efe467fc640 1 --2- 172.21.15.84:0/3512998542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe480addb0 0x7efe481f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:35.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.938+0000 7efe477fe640 1 -- 172.21.15.84:0/3512998542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe480addb0 msgr2=0x7efe481f7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:35.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.938+0000 7efe477fe640 1 --2- 172.21.15.84:0/3512998542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe480addb0 0x7efe481f7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.938+0000 7efe477fe640 1 -- 172.21.15.84:0/3512998542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe4806a2e0 msgr2=0x7efe480ae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:35.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.938+0000 7efe477fe640 1 --2- 172.21.15.84:0/3512998542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe4806a2e0 0x7efe480ae400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:35.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.938+0000 7efe477fe640 1 -- 172.21.15.84:0/3512998542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe38009a70 con 0x7efe480c9000 2023-10-16T09:50:35.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.938+0000 7efe477fe640 1 --2- 172.21.15.84:0/3512998542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe480c9000 0x7efe480b4110 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7efe4000c450 tx=0x7efe4000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.939+0000 7efe3f7fe640 1 -- 172.21.15.84:0/3512998542 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe400074e0 con 0x7efe480c9000 2023-10-16T09:50:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.939+0000 7efe3f7fe640 1 -- 172.21.15.84:0/3512998542 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe40007660 con 0x7efe480c9000 2023-10-16T09:50:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.939+0000 7efe3f7fe640 1 -- 172.21.15.84:0/3512998542 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe40004a20 con 0x7efe480c9000 2023-10-16T09:50:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.939+0000 7efe4d4b4640 1 -- 172.21.15.84:0/3512998542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe300094d0 con 0x7efe480c9000 2023-10-16T09:50:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.939+0000 7efe4d4b4640 1 -- 172.21.15.84:0/3512998542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe481f7520 con 0x7efe480c9000 2023-10-16T09:50:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.940+0000 7efe4d4b4640 1 -- 172.21.15.84:0/3512998542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe48164ea0 con 0x7efe480c9000 2023-10-16T09:50:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.941+0000 7efe3f7fe640 1 -- 172.21.15.84:0/3512998542 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7efe40004b80 con 0x7efe480c9000 2023-10-16T09:50:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.942+0000 7efe3f7fe640 1 --2- 172.21.15.84:0/3512998542 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efe2806b070 0x7efe2806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.942+0000 7efe3f7fe640 1 -- 172.21.15.84:0/3512998542 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7efe40088c80 con 0x7efe480c9000 2023-10-16T09:50:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.942+0000 7efe46ffd640 1 --2- 172.21.15.84:0/3512998542 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efe2806b070 0x7efe2806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:35.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.943+0000 7efe46ffd640 1 --2- 172.21.15.84:0/3512998542 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efe2806b070 0x7efe2806d530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efe30036000 tx=0x7efe30008030 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:35.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:35.947+0000 7efe3f7fe640 1 -- 172.21.15.84:0/3512998542 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efe400539d0 con 0x7efe480c9000 2023-10-16T09:50:36.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.163+0000 7efe4d4b4640 1 -- 172.21.15.84:0/3512998542 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7efe4806b5f0 con 0x7efe2806b070 2023-10-16T09:50:36.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.306+0000 7efe3f7fe640 1 -- 172.21.15.84:0/3512998542 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7efe30030b90 con 0x7efe2806b070 2023-10-16T09:50:36.308 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-10-16T09:50:36.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.309+0000 7efe4d4b4640 1 -- 172.21.15.84:0/3512998542 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efe2806b070 msgr2=0x7efe2806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:36.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.309+0000 7efe4d4b4640 1 --2- 172.21.15.84:0/3512998542 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efe2806b070 0x7efe2806d530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efe30036000 tx=0x7efe30008030 comp rx=0 tx=0).stop 2023-10-16T09:50:36.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.309+0000 7efe4d4b4640 1 -- 172.21.15.84:0/3512998542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe480c9000 msgr2=0x7efe480b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:36.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.309+0000 7efe4d4b4640 1 --2- 172.21.15.84:0/3512998542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe480c9000 0x7efe480b4110 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7efe4000c450 tx=0x7efe4000c910 comp rx=0 tx=0).stop 2023-10-16T09:50:36.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.309+0000 7efe4d4b4640 1 -- 172.21.15.84:0/3512998542 shutdown_connections 2023-10-16T09:50:36.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.309+0000 7efe4d4b4640 1 --2- 172.21.15.84:0/3512998542 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efe2806b070 0x7efe2806d530 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:36.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.309+0000 7efe4d4b4640 1 --2- 172.21.15.84:0/3512998542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe480c9000 0x7efe480b4110 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:36.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.309+0000 7efe4d4b4640 1 --2- 172.21.15.84:0/3512998542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe480addb0 0x7efe481f7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:36.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.309+0000 7efe4d4b4640 1 --2- 172.21.15.84:0/3512998542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe4806a2e0 0x7efe480ae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:36.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.309+0000 7efe4d4b4640 1 -- 172.21.15.84:0/3512998542 >> 172.21.15.84:0/3512998542 conn(0x7efe480ce9b0 msgr2=0x7efe480b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:36.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.309+0000 7efe4d4b4640 1 -- 172.21.15.84:0/3512998542 shutdown_connections 2023-10-16T09:50:36.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.309+0000 7efe4d4b4640 1 -- 172.21.15.84:0/3512998542 wait complete. 2023-10-16T09:50:36.332 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-10-16T09:50:36.332 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir3 1m 2023-10-16T09:50:36.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.542+0000 7fec7f6a2640 1 Processor -- start 2023-10-16T09:50:36.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.542+0000 7fec7f6a2640 1 -- start start 2023-10-16T09:50:36.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.544+0000 7fec7f6a2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec7815ad70 0x7fec78069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:36.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.544+0000 7fec7f6a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec780c9ca0 0x7fec7806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:36.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.544+0000 7fec7f6a2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec780cb030 con 0x7fec780ca3b0 2023-10-16T09:50:36.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.544+0000 7fec7f6a2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec780c91c0 con 0x7fec7815ad70 2023-10-16T09:50:36.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.544+0000 7fec7f6a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec780c94a0 con 0x7fec780c9ca0 2023-10-16T09:50:36.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.544+0000 7fec7d43e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec7815ad70 0x7fec78069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:36.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.544+0000 7fec7d43e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec7815ad70 0x7fec78069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43658/0 (socket says 172.21.15.84:43658) 2023-10-16T09:50:36.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.544+0000 7fec7d43e640 1 -- 172.21.15.84:0/3446031184 learned_addr learned my addr 172.21.15.84:0/3446031184 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:36.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.545+0000 7fec7d43e640 1 -- 172.21.15.84:0/3446031184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec780c9ca0 msgr2=0x7fec7806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:36.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.545+0000 7fec7d43e640 1 --2- 172.21.15.84:0/3446031184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec780c9ca0 0x7fec7806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:36.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.545+0000 7fec7d43e640 1 -- 172.21.15.84:0/3446031184 >> v1:172.21.15.157:6789/0 conn(0x7fec780ca3b0 legacy=0x7fec780ca840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T09:50:36.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.545+0000 7fec7d43e640 1 -- 172.21.15.84:0/3446031184 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec780c9780 con 0x7fec7815ad70 2023-10-16T09:50:36.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.546+0000 7fec7d43e640 1 --2- 172.21.15.84:0/3446031184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec7815ad70 0x7fec78069da0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fec74009870 tx=0x7fec7402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c37455d708b379 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:36.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.546+0000 7fec6ffff640 1 -- 172.21.15.84:0/3446031184 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74031640 con 0x7fec7815ad70 2023-10-16T09:50:36.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.546+0000 7fec6ffff640 1 -- 172.21.15.84:0/3446031184 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec740317a0 con 0x7fec7815ad70 2023-10-16T09:50:36.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.546+0000 7fec6ffff640 1 -- 172.21.15.84:0/3446031184 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74031b30 con 0x7fec7815ad70 2023-10-16T09:50:36.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.547+0000 7fec7f6a2640 1 -- 172.21.15.84:0/3446031184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec7815ad70 msgr2=0x7fec78069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:36.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.547+0000 7fec7f6a2640 1 --2- 172.21.15.84:0/3446031184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec7815ad70 0x7fec78069da0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fec74009870 tx=0x7fec7402b8f0 comp rx=0 tx=0).stop 2023-10-16T09:50:36.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.547+0000 7fec7f6a2640 1 -- 172.21.15.84:0/3446031184 shutdown_connections 2023-10-16T09:50:36.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.547+0000 7fec7f6a2640 1 --2- 172.21.15.84:0/3446031184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec780c9ca0 0x7fec7806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:36.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.547+0000 7fec7f6a2640 1 --2- 172.21.15.84:0/3446031184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec7815ad70 0x7fec78069da0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:36.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.547+0000 7fec7f6a2640 1 -- 172.21.15.84:0/3446031184 >> 172.21.15.84:0/3446031184 conn(0x7fec780ce9b0 msgr2=0x7fec780cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:36.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.547+0000 7fec7f6a2640 1 -- 172.21.15.84:0/3446031184 shutdown_connections 2023-10-16T09:50:36.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.547+0000 7fec7f6a2640 1 -- 172.21.15.84:0/3446031184 wait complete. 2023-10-16T09:50:36.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.549+0000 7fec7f6a2640 1 Processor -- start 2023-10-16T09:50:36.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.549+0000 7fec7f6a2640 1 -- start start 2023-10-16T09:50:36.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.550+0000 7fec7f6a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec780c9ca0 0x7fec780b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:36.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.550+0000 7fec7f6a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec780ca3b0 0x7fec780c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:36.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.550+0000 7fec7f6a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec7815ad70 0x7fec780c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:36.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.550+0000 7fec7f6a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec780c8310 con 0x7fec780c9ca0 2023-10-16T09:50:36.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.550+0000 7fec7f6a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7815a140 con 0x7fec7815ad70 2023-10-16T09:50:36.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.550+0000 7fec7f6a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec780c7690 con 0x7fec780ca3b0 2023-10-16T09:50:36.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.550+0000 7fec7d43e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec780c9ca0 0x7fec780b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:36.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.550+0000 7fec7d43e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec780c9ca0 0x7fec780b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43666/0 (socket says 172.21.15.84:43666) 2023-10-16T09:50:36.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.550+0000 7fec7d43e640 1 -- 172.21.15.84:0/2272103290 learned_addr learned my addr 172.21.15.84:0/2272103290 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:36.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.551+0000 7fec7d43e640 1 -- 172.21.15.84:0/2272103290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec780ca3b0 msgr2=0x7fec780c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:36.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.551+0000 7fec7d43e640 1 --2- 172.21.15.84:0/2272103290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec780ca3b0 0x7fec780c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:36.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.551+0000 7fec7d43e640 1 -- 172.21.15.84:0/2272103290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec7815ad70 msgr2=0x7fec780c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:36.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.551+0000 7fec7d43e640 1 --2- 172.21.15.84:0/2272103290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec7815ad70 0x7fec780c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:36.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.551+0000 7fec7d43e640 1 -- 172.21.15.84:0/2272103290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec740094d0 con 0x7fec780c9ca0 2023-10-16T09:50:36.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.551+0000 7fec7d43e640 1 --2- 172.21.15.84:0/2272103290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec780c9ca0 0x7fec780b3230 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fec74009870 tx=0x7fec740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:36.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.551+0000 7fec6dffb640 1 -- 172.21.15.84:0/2272103290 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7403ac60 con 0x7fec780c9ca0 2023-10-16T09:50:36.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.551+0000 7fec6dffb640 1 -- 172.21.15.84:0/2272103290 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7403adc0 con 0x7fec780c9ca0 2023-10-16T09:50:36.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.551+0000 7fec7f6a2640 1 -- 172.21.15.84:0/2272103290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec780c7970 con 0x7fec780c9ca0 2023-10-16T09:50:36.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.551+0000 7fec6dffb640 1 -- 172.21.15.84:0/2272103290 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74044610 con 0x7fec780c9ca0 2023-10-16T09:50:36.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.551+0000 7fec7f6a2640 1 -- 172.21.15.84:0/2272103290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec780c7f30 con 0x7fec780c9ca0 2023-10-16T09:50:36.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.553+0000 7fec7f6a2640 1 -- 172.21.15.84:0/2272103290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7804e160 con 0x7fec780c9ca0 2023-10-16T09:50:36.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.554+0000 7fec6dffb640 1 -- 172.21.15.84:0/2272103290 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fec7402f050 con 0x7fec780c9ca0 2023-10-16T09:50:36.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.555+0000 7fec6dffb640 1 --2- 172.21.15.84:0/2272103290 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fec5c06b070 0x7fec5c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:36.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.555+0000 7fec6dffb640 1 -- 172.21.15.84:0/2272103290 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fec740abaf0 con 0x7fec780c9ca0 2023-10-16T09:50:36.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.555+0000 7fec7cc3d640 1 --2- 172.21.15.84:0/2272103290 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fec5c06b070 0x7fec5c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:36.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.556+0000 7fec7cc3d640 1 --2- 172.21.15.84:0/2272103290 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fec5c06b070 0x7fec5c06d530 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fec78153460 tx=0x7fec68006c90 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:36.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.560+0000 7fec6dffb640 1 -- 172.21.15.84:0/2272103290 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fec740768c0 con 0x7fec780c9ca0 2023-10-16T09:50:36.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.781+0000 7fec7f6a2640 1 -- 172.21.15.84:0/2272103290 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fec780c8580 con 0x7fec5c06b070 2023-10-16T09:50:36.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.911+0000 7fec6dffb640 1 -- 172.21.15.84:0/2272103290 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fec780c8580 con 0x7fec5c06b070 2023-10-16T09:50:36.913 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-10-16T09:50:36.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.914+0000 7fec7f6a2640 1 -- 172.21.15.84:0/2272103290 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fec5c06b070 msgr2=0x7fec5c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:36.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.914+0000 7fec7f6a2640 1 --2- 172.21.15.84:0/2272103290 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fec5c06b070 0x7fec5c06d530 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fec78153460 tx=0x7fec68006c90 comp rx=0 tx=0).stop 2023-10-16T09:50:36.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.914+0000 7fec7f6a2640 1 -- 172.21.15.84:0/2272103290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec780c9ca0 msgr2=0x7fec780b3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:36.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.914+0000 7fec7f6a2640 1 --2- 172.21.15.84:0/2272103290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec780c9ca0 0x7fec780b3230 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fec74009870 tx=0x7fec740046c0 comp rx=0 tx=0).stop 2023-10-16T09:50:36.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.915+0000 7fec7f6a2640 1 -- 172.21.15.84:0/2272103290 shutdown_connections 2023-10-16T09:50:36.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.915+0000 7fec7f6a2640 1 --2- 172.21.15.84:0/2272103290 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fec5c06b070 0x7fec5c06d530 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:36.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.915+0000 7fec7f6a2640 1 --2- 172.21.15.84:0/2272103290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec7815ad70 0x7fec780c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:36.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.915+0000 7fec7f6a2640 1 --2- 172.21.15.84:0/2272103290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec780ca3b0 0x7fec780c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:36.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.915+0000 7fec7f6a2640 1 --2- 172.21.15.84:0/2272103290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec780c9ca0 0x7fec780b3230 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:36.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.915+0000 7fec7f6a2640 1 -- 172.21.15.84:0/2272103290 >> 172.21.15.84:0/2272103290 conn(0x7fec780ce9b0 msgr2=0x7fec7815b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:36.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.915+0000 7fec7f6a2640 1 -- 172.21.15.84:0/2272103290 shutdown_connections 2023-10-16T09:50:36.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:36.915+0000 7fec7f6a2640 1 -- 172.21.15.84:0/2272103290 wait complete. 2023-10-16T09:50:36.937 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-10-16T09:50:36.937 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir4 1m 2023-10-16T09:50:37.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.142+0000 7f9c626af640 1 Processor -- start 2023-10-16T09:50:37.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.142+0000 7f9c626af640 1 -- start start 2023-10-16T09:50:37.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.144+0000 7f9c626af640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c5c15d150 0x7f9c5c0ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:37.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.144+0000 7f9c626af640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9c5c160b90 0x7f9c5c160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:37.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.144+0000 7f9c626af640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c5c0ac890 con 0x7f9c5c069da0 2023-10-16T09:50:37.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.144+0000 7f9c626af640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c5c15d530 con 0x7f9c5c160b90 2023-10-16T09:50:37.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.144+0000 7f9c626af640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c161650 con 0x7f9c5c15d150 2023-10-16T09:50:37.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.144+0000 7f9c53fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9c5c160b90 0x7f9c5c160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:37.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.144+0000 7f9c53fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9c5c160b90 0x7f9c5c160fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43674/0 (socket says 172.21.15.84:43674) 2023-10-16T09:50:37.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.144+0000 7f9c53fff640 1 -- 172.21.15.84:0/3499737381 learned_addr learned my addr 172.21.15.84:0/3499737381 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:37.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.145+0000 7f9c5bfff640 1 --2- 172.21.15.84:0/3499737381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c5c15d150 0x7f9c5c0ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:37.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.145+0000 7f9c53fff640 1 -- 172.21.15.84:0/3499737381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c5c15d150 msgr2=0x7f9c5c0ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:37.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.145+0000 7f9c53fff640 1 --2- 172.21.15.84:0/3499737381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c5c15d150 0x7f9c5c0ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:37.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.145+0000 7f9c53fff640 1 -- 172.21.15.84:0/3499737381 >> v1:172.21.15.157:6789/0 conn(0x7f9c5c069da0 legacy=0x7f9c5c06a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:37.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.145+0000 7f9c53fff640 1 -- 172.21.15.84:0/3499737381 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c161930 con 0x7f9c5c160b90 2023-10-16T09:50:37.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.145+0000 7f9c5bfff640 1 --2- 172.21.15.84:0/3499737381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c5c15d150 0x7f9c5c0ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:37.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.146+0000 7f9c53fff640 1 --2- 172.21.15.84:0/3499737381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c5c160b90 0x7f9c5c160fb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009870 tx=0x7f9c4402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b64c1a9ba53daf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:37.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.146+0000 7f9c5b7fe640 1 -- 172.21.15.84:0/3499737381 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c44031640 con 0x7f9c5c160b90 2023-10-16T09:50:37.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.147+0000 7f9c5b7fe640 1 -- 172.21.15.84:0/3499737381 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c440317a0 con 0x7f9c5c160b90 2023-10-16T09:50:37.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.147+0000 7f9c5b7fe640 1 -- 172.21.15.84:0/3499737381 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c44031b30 con 0x7f9c5c160b90 2023-10-16T09:50:37.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.147+0000 7f9c626af640 1 -- 172.21.15.84:0/3499737381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c5c160b90 msgr2=0x7f9c5c160fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:37.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.147+0000 7f9c626af640 1 --2- 172.21.15.84:0/3499737381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c5c160b90 0x7f9c5c160fb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009870 tx=0x7f9c4402b8f0 comp rx=0 tx=0).stop 2023-10-16T09:50:37.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.147+0000 7f9c626af640 1 -- 172.21.15.84:0/3499737381 shutdown_connections 2023-10-16T09:50:37.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.147+0000 7f9c626af640 1 --2- 172.21.15.84:0/3499737381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c5c160b90 0x7f9c5c160fb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:37.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.147+0000 7f9c626af640 1 --2- 172.21.15.84:0/3499737381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c5c15d150 0x7f9c5c0ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:37.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.147+0000 7f9c626af640 1 -- 172.21.15.84:0/3499737381 >> 172.21.15.84:0/3499737381 conn(0x7f9c5c0ce9b0 msgr2=0x7f9c5c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:37.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.147+0000 7f9c626af640 1 -- 172.21.15.84:0/3499737381 shutdown_connections 2023-10-16T09:50:37.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.148+0000 7f9c626af640 1 -- 172.21.15.84:0/3499737381 wait complete. 2023-10-16T09:50:37.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.149+0000 7f9c626af640 1 Processor -- start 2023-10-16T09:50:37.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.149+0000 7f9c626af640 1 -- start start 2023-10-16T09:50:37.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.150+0000 7f9c626af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c5c069da0 0x7f9c5c14ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:37.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.150+0000 7f9c626af640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c5c15d150 0x7f9c5c0afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:37.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.150+0000 7f9c626af640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c5c160b90 0x7f9c5c0b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:37.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.150+0000 7f9c626af640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c15e3b0 con 0x7f9c5c160b90 2023-10-16T09:50:37.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.150+0000 7f9c626af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c15e690 con 0x7f9c5c069da0 2023-10-16T09:50:37.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.150+0000 7f9c626af640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c15e970 con 0x7f9c5c15d150 2023-10-16T09:50:37.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.151+0000 7f9c60c4c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c5c160b90 0x7f9c5c0b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:37.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.151+0000 7f9c60c4c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c5c160b90 0x7f9c5c0b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43678/0 (socket says 172.21.15.84:43678) 2023-10-16T09:50:37.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.151+0000 7f9c60c4c640 1 -- 172.21.15.84:0/326335409 learned_addr learned my addr 172.21.15.84:0/326335409 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:37.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.151+0000 7f9c53fff640 1 --2- 172.21.15.84:0/326335409 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c5c15d150 0x7f9c5c0afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:37.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.151+0000 7f9c5bfff640 1 --2- 172.21.15.84:0/326335409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c5c069da0 0x7f9c5c14ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:37.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.151+0000 7f9c60c4c640 1 -- 172.21.15.84:0/326335409 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c5c15d150 msgr2=0x7f9c5c0afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:37.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.151+0000 7f9c60c4c640 1 --2- 172.21.15.84:0/326335409 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c5c15d150 0x7f9c5c0afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:37.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.151+0000 7f9c60c4c640 1 -- 172.21.15.84:0/326335409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c5c069da0 msgr2=0x7f9c5c14ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:37.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.151+0000 7f9c60c4c640 1 --2- 172.21.15.84:0/326335409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c5c069da0 0x7f9c5c14ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:37.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.151+0000 7f9c60c4c640 1 -- 172.21.15.84:0/326335409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c440094d0 con 0x7f9c5c160b90 2023-10-16T09:50:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.151+0000 7f9c53fff640 1 --2- 172.21.15.84:0/326335409 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c5c15d150 0x7f9c5c0afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.151+0000 7f9c5bfff640 1 --2- 172.21.15.84:0/326335409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c5c069da0 0x7f9c5c14ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.151+0000 7f9c60c4c640 1 --2- 172.21.15.84:0/326335409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c5c160b90 0x7f9c5c0b0170 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400c780 tx=0x7f9c5400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.152+0000 7f9c597fa640 1 -- 172.21.15.84:0/326335409 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c54004b60 con 0x7f9c5c160b90 2023-10-16T09:50:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.152+0000 7f9c597fa640 1 -- 172.21.15.84:0/326335409 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5400ce90 con 0x7f9c5c160b90 2023-10-16T09:50:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.152+0000 7f9c597fa640 1 -- 172.21.15.84:0/326335409 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5400a850 con 0x7f9c5c160b90 2023-10-16T09:50:37.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.152+0000 7f9c626af640 1 -- 172.21.15.84:0/326335409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c15ec50 con 0x7f9c5c160b90 2023-10-16T09:50:37.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.152+0000 7f9c626af640 1 -- 172.21.15.84:0/326335409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c15e080 con 0x7f9c5c160b90 2023-10-16T09:50:37.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.153+0000 7f9c626af640 1 -- 172.21.15.84:0/326335409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c15e4f0 con 0x7f9c5c160b90 2023-10-16T09:50:37.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.154+0000 7f9c597fa640 1 -- 172.21.15.84:0/326335409 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f9c5400a9b0 con 0x7f9c5c160b90 2023-10-16T09:50:37.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.155+0000 7f9c597fa640 1 --2- 172.21.15.84:0/326335409 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f9c3c06b070 0x7f9c3c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:37.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.155+0000 7f9c597fa640 1 -- 172.21.15.84:0/326335409 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f9c54089c50 con 0x7f9c5c160b90 2023-10-16T09:50:37.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.156+0000 7f9c5bfff640 1 --2- 172.21.15.84:0/326335409 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f9c3c06b070 0x7f9c3c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:37.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.156+0000 7f9c5bfff640 1 --2- 172.21.15.84:0/326335409 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f9c3c06b070 0x7f9c3c06d530 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0b4900 tx=0x7f9c4c008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:37.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.160+0000 7f9c597fa640 1 -- 172.21.15.84:0/326335409 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9c5c15e4f0 con 0x7f9c5c160b90 2023-10-16T09:50:37.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.375+0000 7f9c626af640 1 -- 172.21.15.84:0/326335409 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c5c0ace30 con 0x7f9c3c06b070 2023-10-16T09:50:37.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.524+0000 7f9c597fa640 1 -- 172.21.15.84:0/326335409 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f9c4c00c040 con 0x7f9c3c06b070 2023-10-16T09:50:37.527 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-10-16T09:50:37.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.528+0000 7f9c626af640 1 -- 172.21.15.84:0/326335409 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f9c3c06b070 msgr2=0x7f9c3c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:37.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.528+0000 7f9c626af640 1 --2- 172.21.15.84:0/326335409 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f9c3c06b070 0x7f9c3c06d530 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0b4900 tx=0x7f9c4c008040 comp rx=0 tx=0).stop 2023-10-16T09:50:37.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.528+0000 7f9c626af640 1 -- 172.21.15.84:0/326335409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c5c160b90 msgr2=0x7f9c5c0b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:37.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.528+0000 7f9c626af640 1 --2- 172.21.15.84:0/326335409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c5c160b90 0x7f9c5c0b0170 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400c780 tx=0x7f9c5400cc40 comp rx=0 tx=0).stop 2023-10-16T09:50:37.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.528+0000 7f9c626af640 1 -- 172.21.15.84:0/326335409 shutdown_connections 2023-10-16T09:50:37.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.528+0000 7f9c626af640 1 --2- 172.21.15.84:0/326335409 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f9c3c06b070 0x7f9c3c06d530 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:37.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.528+0000 7f9c626af640 1 --2- 172.21.15.84:0/326335409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c5c160b90 0x7f9c5c0b0170 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:37.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.528+0000 7f9c626af640 1 --2- 172.21.15.84:0/326335409 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c5c15d150 0x7f9c5c0afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:37.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.528+0000 7f9c626af640 1 --2- 172.21.15.84:0/326335409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c5c069da0 0x7f9c5c14ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:37.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.528+0000 7f9c626af640 1 -- 172.21.15.84:0/326335409 >> 172.21.15.84:0/326335409 conn(0x7f9c5c0ce9b0 msgr2=0x7f9c5c0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:37.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.528+0000 7f9c626af640 1 -- 172.21.15.84:0/326335409 shutdown_connections 2023-10-16T09:50:37.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.528+0000 7f9c626af640 1 -- 172.21.15.84:0/326335409 wait complete. 2023-10-16T09:50:37.549 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-10-16T09:50:37.550 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir5 1m 2023-10-16T09:50:37.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.757+0000 7f727bb97640 1 Processor -- start 2023-10-16T09:50:37.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.757+0000 7f727bb97640 1 -- start start 2023-10-16T09:50:37.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.759+0000 7f727bb97640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7274159ab0 0x7f72740ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:37.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.759+0000 7f727bb97640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7274159e90 0x7f727406fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:37.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.759+0000 7f727bb97640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72740cd1f0 con 0x7f727406c140 2023-10-16T09:50:37.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.759+0000 7f727bb97640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f727415d890 con 0x7f7274159e90 2023-10-16T09:50:37.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.759+0000 7f727bb97640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72740cd4d0 con 0x7f7274159ab0 2023-10-16T09:50:37.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.760+0000 7f7279132640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7274159e90 0x7f727406fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:37.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.760+0000 7f7279933640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7274159ab0 0x7f72740ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:37.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.760+0000 7f7279933640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7274159ab0 0x7f72740ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:52656/0 (socket says 172.21.15.84:52656) 2023-10-16T09:50:37.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.760+0000 7f7279132640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7274159e90 0x7f727406fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:54412/0 (socket says 172.21.15.84:54412) 2023-10-16T09:50:37.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.760+0000 7f7279933640 1 -- 172.21.15.84:0/339498722 learned_addr learned my addr 172.21.15.84:0/339498722 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:37.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.760+0000 7f7279933640 1 -- 172.21.15.84:0/339498722 >> v2:172.21.15.157:3300/0 conn(0x7f7274159e90 msgr2=0x7f727406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:37.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.760+0000 7f7279933640 1 --2- 172.21.15.84:0/339498722 >> v2:172.21.15.157:3300/0 conn(0x7f7274159e90 0x7f727406fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:37.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.760+0000 7f7279933640 1 -- 172.21.15.84:0/339498722 >> v1:172.21.15.157:6789/0 conn(0x7f727406c140 legacy=0x7f727415dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:37.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.760+0000 7f7279933640 1 -- 172.21.15.84:0/339498722 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72740cd7b0 con 0x7f7274159ab0 2023-10-16T09:50:37.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.762+0000 7f7279933640 1 --2- 172.21.15.84:0/339498722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7274159ab0 0x7f72740ce370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f72700098a0 tx=0x7f727002b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f27c79a2ca3d5cfc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:37.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.762+0000 7f7278931640 1 -- 172.21.15.84:0/339498722 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7270032640 con 0x7f7274159ab0 2023-10-16T09:50:37.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.762+0000 7f7278931640 1 -- 172.21.15.84:0/339498722 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727002e070 con 0x7f7274159ab0 2023-10-16T09:50:37.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.762+0000 7f7278931640 1 -- 172.21.15.84:0/339498722 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72700329c0 con 0x7f7274159ab0 2023-10-16T09:50:37.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.763+0000 7f727bb97640 1 -- 172.21.15.84:0/339498722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7274159ab0 msgr2=0x7f72740ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:37.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.763+0000 7f727bb97640 1 --2- 172.21.15.84:0/339498722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7274159ab0 0x7f72740ce370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f72700098a0 tx=0x7f727002b850 comp rx=0 tx=0).stop 2023-10-16T09:50:37.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.763+0000 7f727bb97640 1 -- 172.21.15.84:0/339498722 shutdown_connections 2023-10-16T09:50:37.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.763+0000 7f727bb97640 1 --2- 172.21.15.84:0/339498722 >> v2:172.21.15.157:3300/0 conn(0x7f7274159e90 0x7f727406fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:37.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.763+0000 7f727bb97640 1 --2- 172.21.15.84:0/339498722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7274159ab0 0x7f72740ce370 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:37.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.763+0000 7f727bb97640 1 -- 172.21.15.84:0/339498722 >> 172.21.15.84:0/339498722 conn(0x7f72740ce9b0 msgr2=0x7f72740cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:37.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.763+0000 7f727bb97640 1 -- 172.21.15.84:0/339498722 shutdown_connections 2023-10-16T09:50:37.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.763+0000 7f727bb97640 1 -- 172.21.15.84:0/339498722 wait complete. 2023-10-16T09:50:37.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.765+0000 7f727bb97640 1 Processor -- start 2023-10-16T09:50:37.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.765+0000 7f727bb97640 1 -- start start 2023-10-16T09:50:37.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.767+0000 7f727bb97640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727406c140 0x7f7274150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:37.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.767+0000 7f727bb97640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7274159ab0 0x7f7274151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:37.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.767+0000 7f727bb97640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7274159e90 0x7f72740c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:37.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.767+0000 7f727bb97640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72741596a0 con 0x7f7274159ab0 2023-10-16T09:50:37.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.767+0000 7f727bb97640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72741593c0 con 0x7f7274159e90 2023-10-16T09:50:37.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.767+0000 7f727bb97640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72740c4a00 con 0x7f727406c140 2023-10-16T09:50:37.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.767+0000 7f7279132640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7274159ab0 0x7f7274151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:37.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.767+0000 7f7279132640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7274159ab0 0x7f7274151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43690/0 (socket says 172.21.15.84:43690) 2023-10-16T09:50:37.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.767+0000 7f7279132640 1 -- 172.21.15.84:0/2789426550 learned_addr learned my addr 172.21.15.84:0/2789426550 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:37.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.767+0000 7f727a134640 1 --2- 172.21.15.84:0/2789426550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7274159e90 0x7f72740c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:37.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.767+0000 7f7279933640 1 --2- 172.21.15.84:0/2789426550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727406c140 0x7f7274150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:37.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.767+0000 7f7279132640 1 -- 172.21.15.84:0/2789426550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727406c140 msgr2=0x7f7274150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:37.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.767+0000 7f7279132640 1 --2- 172.21.15.84:0/2789426550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727406c140 0x7f7274150b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:37.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.767+0000 7f7279132640 1 -- 172.21.15.84:0/2789426550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7274159e90 msgr2=0x7f72740c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:37.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.767+0000 7f7279132640 1 --2- 172.21.15.84:0/2789426550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7274159e90 0x7f72740c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:37.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.767+0000 7f7279132640 1 -- 172.21.15.84:0/2789426550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72700094d0 con 0x7f7274159ab0 2023-10-16T09:50:37.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.767+0000 7f727a134640 1 --2- 172.21.15.84:0/2789426550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7274159e90 0x7f72740c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:37.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.767+0000 7f7279933640 1 --2- 172.21.15.84:0/2789426550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727406c140 0x7f7274150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:37.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.768+0000 7f7279132640 1 --2- 172.21.15.84:0/2789426550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7274159ab0 0x7f7274151c30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f726400e9b0 tx=0x7f726400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:37.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.768+0000 7f726a7fc640 1 -- 172.21.15.84:0/2789426550 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7264009670 con 0x7f7274159ab0 2023-10-16T09:50:37.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.768+0000 7f726a7fc640 1 -- 172.21.15.84:0/2789426550 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726400c9a0 con 0x7f7274159ab0 2023-10-16T09:50:37.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.768+0000 7f726a7fc640 1 -- 172.21.15.84:0/2789426550 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726400cd50 con 0x7f7274159ab0 2023-10-16T09:50:37.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.768+0000 7f727bb97640 1 -- 172.21.15.84:0/2789426550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72740c4ce0 con 0x7f7274159ab0 2023-10-16T09:50:37.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.768+0000 7f727bb97640 1 -- 172.21.15.84:0/2789426550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7274151080 con 0x7f7274159ab0 2023-10-16T09:50:37.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.769+0000 7f727bb97640 1 -- 172.21.15.84:0/2789426550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72740c4860 con 0x7f7274159ab0 2023-10-16T09:50:37.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.770+0000 7f726a7fc640 1 -- 172.21.15.84:0/2789426550 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f7264013470 con 0x7f7274159ab0 2023-10-16T09:50:37.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.771+0000 7f726a7fc640 1 --2- 172.21.15.84:0/2789426550 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f725806b070 0x7f725806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:37.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.771+0000 7f726a7fc640 1 -- 172.21.15.84:0/2789426550 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f7264089d40 con 0x7f7274159ab0 2023-10-16T09:50:37.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.772+0000 7f7279933640 1 --2- 172.21.15.84:0/2789426550 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f725806b070 0x7f725806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:37.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.773+0000 7f7279933640 1 --2- 172.21.15.84:0/2789426550 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f725806b070 0x7f725806d530 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7270036000 tx=0x7f727003d040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:37.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.779+0000 7f726a7fc640 1 -- 172.21.15.84:0/2789426550 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f72740c4860 con 0x7f7274159ab0 2023-10-16T09:50:37.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:37.997+0000 7f727bb97640 1 -- 172.21.15.84:0/2789426550 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f72740adf70 con 0x7f725806b070 2023-10-16T09:50:38.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.146+0000 7f726a7fc640 1 -- 172.21.15.84:0/2789426550 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f72700328a0 con 0x7f725806b070 2023-10-16T09:50:38.149 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-10-16T09:50:38.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.150+0000 7f727bb97640 1 -- 172.21.15.84:0/2789426550 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f725806b070 msgr2=0x7f725806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:38.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.150+0000 7f727bb97640 1 --2- 172.21.15.84:0/2789426550 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f725806b070 0x7f725806d530 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7270036000 tx=0x7f727003d040 comp rx=0 tx=0).stop 2023-10-16T09:50:38.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.150+0000 7f727bb97640 1 -- 172.21.15.84:0/2789426550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7274159ab0 msgr2=0x7f7274151c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:38.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.150+0000 7f727bb97640 1 --2- 172.21.15.84:0/2789426550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7274159ab0 0x7f7274151c30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f726400e9b0 tx=0x7f726400ee70 comp rx=0 tx=0).stop 2023-10-16T09:50:38.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.150+0000 7f727bb97640 1 -- 172.21.15.84:0/2789426550 shutdown_connections 2023-10-16T09:50:38.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.150+0000 7f727bb97640 1 --2- 172.21.15.84:0/2789426550 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f725806b070 0x7f725806d530 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.150+0000 7f727bb97640 1 --2- 172.21.15.84:0/2789426550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7274159e90 0x7f72740c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.150+0000 7f727bb97640 1 --2- 172.21.15.84:0/2789426550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7274159ab0 0x7f7274151c30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.150+0000 7f727bb97640 1 --2- 172.21.15.84:0/2789426550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727406c140 0x7f7274150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.150+0000 7f727bb97640 1 -- 172.21.15.84:0/2789426550 >> 172.21.15.84:0/2789426550 conn(0x7f72740ce9b0 msgr2=0x7f72740b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:38.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.150+0000 7f727bb97640 1 -- 172.21.15.84:0/2789426550 shutdown_connections 2023-10-16T09:50:38.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.151+0000 7f727bb97640 1 -- 172.21.15.84:0/2789426550 wait complete. 2023-10-16T09:50:38.173 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-10-16T09:50:38.173 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir6 1m 2023-10-16T09:50:38.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.381+0000 7ff55e9a9640 1 Processor -- start 2023-10-16T09:50:38.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.382+0000 7ff55e9a9640 1 -- start start 2023-10-16T09:50:38.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.383+0000 7ff55e9a9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff55815ad70 0x7ff558069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:38.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.383+0000 7ff55e9a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff5580caac0 0x7ff55806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:38.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.383+0000 7ff55e9a9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5580cb030 con 0x7ff5580c9ca0 2023-10-16T09:50:38.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.383+0000 7ff55e9a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5580c91c0 con 0x7ff5580caac0 2023-10-16T09:50:38.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.383+0000 7ff55e9a9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5580c94a0 con 0x7ff55815ad70 2023-10-16T09:50:38.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.383+0000 7ff55cf46640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff5580caac0 0x7ff55806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:38.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.383+0000 7ff54ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff5580c9ca0 0x7ff5580ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32786/0 (socket says 172.21.15.84:32786) 2023-10-16T09:50:38.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.383+0000 7ff54ffff640 1 -- 172.21.15.84:0/680931341 learned_addr learned my addr 172.21.15.84:0/680931341 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:38.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.384+0000 7ff557fff640 1 --2- 172.21.15.84:0/680931341 >> v2:172.21.15.157:3300/0 conn(0x7ff55815ad70 0x7ff558069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:38.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.384+0000 7ff55cf46640 1 -- 172.21.15.84:0/680931341 >> v2:172.21.15.157:3300/0 conn(0x7ff55815ad70 msgr2=0x7ff558069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:38.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.384+0000 7ff55cf46640 1 --2- 172.21.15.84:0/680931341 >> v2:172.21.15.157:3300/0 conn(0x7ff55815ad70 0x7ff558069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.384+0000 7ff55cf46640 1 -- 172.21.15.84:0/680931341 >> v1:172.21.15.84:6789/0 conn(0x7ff5580c9ca0 legacy=0x7ff5580ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:38.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.384+0000 7ff55cf46640 1 -- 172.21.15.84:0/680931341 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5580c9780 con 0x7ff5580caac0 2023-10-16T09:50:38.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.384+0000 7ff5577fe640 1 -- 172.21.15.84:0/680931341 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771049710 0 0) 0x7ff5580cb030 con 0x7ff5580c9ca0 2023-10-16T09:50:38.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.385+0000 7ff55cf46640 1 --2- 172.21.15.84:0/680931341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5580caac0 0x7ff55806a2e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff5580cf780 tx=0x7ff55002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fefeb5d0103aea server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:38.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.385+0000 7ff5577fe640 1 -- 172.21.15.84:0/680931341 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55002cc80 con 0x7ff5580caac0 2023-10-16T09:50:38.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.385+0000 7ff5577fe640 1 -- 172.21.15.84:0/680931341 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55002cde0 con 0x7ff5580caac0 2023-10-16T09:50:38.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.385+0000 7ff5577fe640 1 -- 172.21.15.84:0/680931341 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff550034660 con 0x7ff5580caac0 2023-10-16T09:50:38.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.386+0000 7ff55e9a9640 1 -- 172.21.15.84:0/680931341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5580caac0 msgr2=0x7ff55806a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:38.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.386+0000 7ff55e9a9640 1 --2- 172.21.15.84:0/680931341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5580caac0 0x7ff55806a2e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff5580cf780 tx=0x7ff55002bed0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.386+0000 7ff55e9a9640 1 -- 172.21.15.84:0/680931341 shutdown_connections 2023-10-16T09:50:38.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.386+0000 7ff55e9a9640 1 --2- 172.21.15.84:0/680931341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5580caac0 0x7ff55806a2e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.386+0000 7ff55e9a9640 1 --2- 172.21.15.84:0/680931341 >> v2:172.21.15.157:3300/0 conn(0x7ff55815ad70 0x7ff558069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.386+0000 7ff55e9a9640 1 -- 172.21.15.84:0/680931341 >> 172.21.15.84:0/680931341 conn(0x7ff5580ce9b0 msgr2=0x7ff5580cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:38.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.386+0000 7ff55e9a9640 1 -- 172.21.15.84:0/680931341 shutdown_connections 2023-10-16T09:50:38.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.386+0000 7ff55e9a9640 1 -- 172.21.15.84:0/680931341 wait complete. 2023-10-16T09:50:38.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.388+0000 7ff55e9a9640 1 Processor -- start 2023-10-16T09:50:38.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.388+0000 7ff55e9a9640 1 -- start start 2023-10-16T09:50:38.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.389+0000 7ff55e9a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5580c9ca0 0x7ff5580b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:38.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.389+0000 7ff55e9a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5580caac0 0x7ff5580b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:38.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.389+0000 7ff55e9a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff55815ad70 0x7ff5580b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:38.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.389+0000 7ff55e9a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5580c5b20 con 0x7ff5580caac0 2023-10-16T09:50:38.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.389+0000 7ff55e9a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5580c5d90 con 0x7ff5580c9ca0 2023-10-16T09:50:38.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.389+0000 7ff55e9a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5580c6070 con 0x7ff55815ad70 2023-10-16T09:50:38.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.389+0000 7ff54ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5580caac0 0x7ff5580b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:38.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.389+0000 7ff54ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5580caac0 0x7ff5580b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43718/0 (socket says 172.21.15.84:43718) 2023-10-16T09:50:38.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.389+0000 7ff54ffff640 1 -- 172.21.15.84:0/3287742560 learned_addr learned my addr 172.21.15.84:0/3287742560 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:38.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.389+0000 7ff557fff640 1 --2- 172.21.15.84:0/3287742560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5580c9ca0 0x7ff5580b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:38.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.389+0000 7ff54ffff640 1 -- 172.21.15.84:0/3287742560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff55815ad70 msgr2=0x7ff5580b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:38.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.389+0000 7ff54ffff640 1 --2- 172.21.15.84:0/3287742560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff55815ad70 0x7ff5580b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.389+0000 7ff54ffff640 1 -- 172.21.15.84:0/3287742560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5580c9ca0 msgr2=0x7ff5580b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:38.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.389+0000 7ff54ffff640 1 --2- 172.21.15.84:0/3287742560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5580c9ca0 0x7ff5580b24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.390+0000 7ff54ffff640 1 -- 172.21.15.84:0/3287742560 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff550009a70 con 0x7ff5580caac0 2023-10-16T09:50:38.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.390+0000 7ff54ffff640 1 --2- 172.21.15.84:0/3287742560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5580caac0 0x7ff5580b3230 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff54000c4d0 tx=0x7ff54000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:38.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.390+0000 7ff5557fa640 1 -- 172.21.15.84:0/3287742560 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5400074e0 con 0x7ff5580caac0 2023-10-16T09:50:38.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.390+0000 7ff5557fa640 1 -- 172.21.15.84:0/3287742560 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff540007660 con 0x7ff5580caac0 2023-10-16T09:50:38.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.390+0000 7ff5557fa640 1 -- 172.21.15.84:0/3287742560 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff540004a20 con 0x7ff5580caac0 2023-10-16T09:50:38.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.390+0000 7ff55e9a9640 1 -- 172.21.15.84:0/3287742560 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5580c6350 con 0x7ff5580caac0 2023-10-16T09:50:38.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.390+0000 7ff55e9a9640 1 -- 172.21.15.84:0/3287742560 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5580b1760 con 0x7ff5580caac0 2023-10-16T09:50:38.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.391+0000 7ff55e9a9640 1 -- 172.21.15.84:0/3287742560 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5580c6540 con 0x7ff5580caac0 2023-10-16T09:50:38.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.392+0000 7ff5557fa640 1 -- 172.21.15.84:0/3287742560 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7ff540004b80 con 0x7ff5580caac0 2023-10-16T09:50:38.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.393+0000 7ff5557fa640 1 --2- 172.21.15.84:0/3287742560 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff53806b070 0x7ff53806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:38.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.393+0000 7ff5557fa640 1 -- 172.21.15.84:0/3287742560 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff540088c40 con 0x7ff5580caac0 2023-10-16T09:50:38.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.394+0000 7ff557fff640 1 --2- 172.21.15.84:0/3287742560 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff53806b070 0x7ff53806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:38.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.394+0000 7ff557fff640 1 --2- 172.21.15.84:0/3287742560 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff53806b070 0x7ff53806d530 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff548006fd0 tx=0x7ff548008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:38.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.398+0000 7ff5557fa640 1 -- 172.21.15.84:0/3287742560 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff540053960 con 0x7ff5580caac0 2023-10-16T09:50:38.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.611+0000 7ff55e9a9640 1 -- 172.21.15.84:0/3287742560 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5580c5f00 con 0x7ff53806b070 2023-10-16T09:50:38.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.745+0000 7ff5557fa640 1 -- 172.21.15.84:0/3287742560 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ff5580c5f00 con 0x7ff53806b070 2023-10-16T09:50:38.748 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-10-16T09:50:38.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.748+0000 7ff55e9a9640 1 -- 172.21.15.84:0/3287742560 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff53806b070 msgr2=0x7ff53806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:38.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.748+0000 7ff55e9a9640 1 --2- 172.21.15.84:0/3287742560 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff53806b070 0x7ff53806d530 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff548006fd0 tx=0x7ff548008040 comp rx=0 tx=0).stop 2023-10-16T09:50:38.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.749+0000 7ff55e9a9640 1 -- 172.21.15.84:0/3287742560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5580caac0 msgr2=0x7ff5580b3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:38.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.749+0000 7ff55e9a9640 1 --2- 172.21.15.84:0/3287742560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5580caac0 0x7ff5580b3230 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff54000c4d0 tx=0x7ff54000c990 comp rx=0 tx=0).stop 2023-10-16T09:50:38.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.749+0000 7ff55e9a9640 1 -- 172.21.15.84:0/3287742560 shutdown_connections 2023-10-16T09:50:38.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.749+0000 7ff55e9a9640 1 --2- 172.21.15.84:0/3287742560 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff53806b070 0x7ff53806d530 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.749+0000 7ff55e9a9640 1 --2- 172.21.15.84:0/3287742560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff55815ad70 0x7ff5580b3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.749+0000 7ff55e9a9640 1 --2- 172.21.15.84:0/3287742560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5580caac0 0x7ff5580b3230 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.749+0000 7ff55e9a9640 1 --2- 172.21.15.84:0/3287742560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5580c9ca0 0x7ff5580b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.749+0000 7ff55e9a9640 1 -- 172.21.15.84:0/3287742560 >> 172.21.15.84:0/3287742560 conn(0x7ff5580ce9b0 msgr2=0x7ff5580cd270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:38.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.749+0000 7ff55e9a9640 1 -- 172.21.15.84:0/3287742560 shutdown_connections 2023-10-16T09:50:38.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.749+0000 7ff55e9a9640 1 -- 172.21.15.84:0/3287742560 wait complete. 2023-10-16T09:50:38.769 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-10-16T09:50:38.770 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir7 1m 2023-10-16T09:50:38.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.977+0000 7ff090032640 1 Processor -- start 2023-10-16T09:50:38.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.977+0000 7ff090032640 1 -- start start 2023-10-16T09:50:38.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.979+0000 7ff090032640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff08815ad70 0x7ff088069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:38.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.979+0000 7ff090032640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff0880caac0 0x7ff08806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:38.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.979+0000 7ff090032640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0880cb030 con 0x7ff0880c9ca0 2023-10-16T09:50:38.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.979+0000 7ff090032640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0880c91c0 con 0x7ff08815ad70 2023-10-16T09:50:38.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.979+0000 7ff090032640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0880c94a0 con 0x7ff0880caac0 2023-10-16T09:50:38.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.979+0000 7ff08ddce640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff08815ad70 0x7ff088069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:38.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.979+0000 7ff08ddce640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff08815ad70 0x7ff088069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43728/0 (socket says 172.21.15.84:43728) 2023-10-16T09:50:38.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.979+0000 7ff08ddce640 1 -- 172.21.15.84:0/1897014767 learned_addr learned my addr 172.21.15.84:0/1897014767 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:38.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.980+0000 7ff08ddce640 1 -- 172.21.15.84:0/1897014767 >> v2:172.21.15.157:3300/0 conn(0x7ff0880caac0 msgr2=0x7ff08806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:38.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.980+0000 7ff08ddce640 1 --2- 172.21.15.84:0/1897014767 >> v2:172.21.15.157:3300/0 conn(0x7ff0880caac0 0x7ff08806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.980+0000 7ff08ddce640 1 -- 172.21.15.84:0/1897014767 >> v1:172.21.15.157:6789/0 conn(0x7ff0880c9ca0 legacy=0x7ff0880ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:38.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.980+0000 7ff08ddce640 1 -- 172.21.15.84:0/1897014767 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0880c9780 con 0x7ff08815ad70 2023-10-16T09:50:38.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.981+0000 7ff08ddce640 1 --2- 172.21.15.84:0/1897014767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff08815ad70 0x7ff088069da0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff08806d090 tx=0x7ff08402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2831ca491c7867 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:38.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.981+0000 7ff08cdcc640 1 -- 172.21.15.84:0/1897014767 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff084031870 con 0x7ff08815ad70 2023-10-16T09:50:38.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.981+0000 7ff08cdcc640 1 -- 172.21.15.84:0/1897014767 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0840319d0 con 0x7ff08815ad70 2023-10-16T09:50:38.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.981+0000 7ff08cdcc640 1 -- 172.21.15.84:0/1897014767 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff084031d60 con 0x7ff08815ad70 2023-10-16T09:50:38.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.982+0000 7ff090032640 1 -- 172.21.15.84:0/1897014767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff08815ad70 msgr2=0x7ff088069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:38.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.982+0000 7ff090032640 1 --2- 172.21.15.84:0/1897014767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff08815ad70 0x7ff088069da0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff08806d090 tx=0x7ff08402b910 comp rx=0 tx=0).stop 2023-10-16T09:50:38.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.982+0000 7ff090032640 1 -- 172.21.15.84:0/1897014767 shutdown_connections 2023-10-16T09:50:38.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.982+0000 7ff090032640 1 --2- 172.21.15.84:0/1897014767 >> v2:172.21.15.157:3300/0 conn(0x7ff0880caac0 0x7ff08806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.982+0000 7ff090032640 1 --2- 172.21.15.84:0/1897014767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff08815ad70 0x7ff088069da0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.982+0000 7ff090032640 1 -- 172.21.15.84:0/1897014767 >> 172.21.15.84:0/1897014767 conn(0x7ff0880ce9b0 msgr2=0x7ff0880cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:38.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.982+0000 7ff090032640 1 -- 172.21.15.84:0/1897014767 shutdown_connections 2023-10-16T09:50:38.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.982+0000 7ff090032640 1 -- 172.21.15.84:0/1897014767 wait complete. 2023-10-16T09:50:38.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.984+0000 7ff090032640 1 Processor -- start 2023-10-16T09:50:38.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.984+0000 7ff090032640 1 -- start start 2023-10-16T09:50:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.985+0000 7ff090032640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0880c9ca0 0x7ff0881f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.985+0000 7ff090032640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0880caac0 0x7ff0880c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.985+0000 7ff090032640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff08815ad70 0x7ff0880c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.985+0000 7ff090032640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0881f32c0 con 0x7ff0880caac0 2023-10-16T09:50:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.985+0000 7ff090032640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0881f6840 con 0x7ff0880c9ca0 2023-10-16T09:50:38.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.985+0000 7ff090032640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0881f2740 con 0x7ff08815ad70 2023-10-16T09:50:38.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.985+0000 7ff08d5cd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0880caac0 0x7ff0880c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:38.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.985+0000 7ff08d5cd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0880caac0 0x7ff0880c7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43740/0 (socket says 172.21.15.84:43740) 2023-10-16T09:50:38.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.985+0000 7ff08d5cd640 1 -- 172.21.15.84:0/2063634990 learned_addr learned my addr 172.21.15.84:0/2063634990 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:38.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.985+0000 7ff08d5cd640 1 -- 172.21.15.84:0/2063634990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff08815ad70 msgr2=0x7ff0880c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:38.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.985+0000 7ff08d5cd640 1 --2- 172.21.15.84:0/2063634990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff08815ad70 0x7ff0880c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.985+0000 7ff08ddce640 1 --2- 172.21.15.84:0/2063634990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0880c9ca0 0x7ff0881f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:38.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.985+0000 7ff08d5cd640 1 -- 172.21.15.84:0/2063634990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0880c9ca0 msgr2=0x7ff0881f6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:38.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.985+0000 7ff08d5cd640 1 --2- 172.21.15.84:0/2063634990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0880c9ca0 0x7ff0881f6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:38.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.985+0000 7ff08d5cd640 1 -- 172.21.15.84:0/2063634990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0840094d0 con 0x7ff0880caac0 2023-10-16T09:50:38.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.986+0000 7ff08d5cd640 1 --2- 172.21.15.84:0/2063634990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0880caac0 0x7ff0880c7850 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff07800c410 tx=0x7ff07800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:38.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.986+0000 7ff07e7fc640 1 -- 172.21.15.84:0/2063634990 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0780035f0 con 0x7ff0880caac0 2023-10-16T09:50:38.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.986+0000 7ff07e7fc640 1 -- 172.21.15.84:0/2063634990 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff078003750 con 0x7ff0880caac0 2023-10-16T09:50:38.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.986+0000 7ff07e7fc640 1 -- 172.21.15.84:0/2063634990 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078003b40 con 0x7ff0880caac0 2023-10-16T09:50:38.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.986+0000 7ff090032640 1 -- 172.21.15.84:0/2063634990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0881f2a20 con 0x7ff0880caac0 2023-10-16T09:50:38.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.986+0000 7ff090032640 1 -- 172.21.15.84:0/2063634990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0881f2fe0 con 0x7ff0880caac0 2023-10-16T09:50:38.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.987+0000 7ff090032640 1 -- 172.21.15.84:0/2063634990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0880c5b10 con 0x7ff0880caac0 2023-10-16T09:50:38.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.988+0000 7ff07e7fc640 1 -- 172.21.15.84:0/2063634990 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7ff078003ca0 con 0x7ff0880caac0 2023-10-16T09:50:38.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.990+0000 7ff07e7fc640 1 --2- 172.21.15.84:0/2063634990 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff06c06b070 0x7ff06c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:38.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.990+0000 7ff07e7fc640 1 -- 172.21.15.84:0/2063634990 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff078088cb0 con 0x7ff0880caac0 2023-10-16T09:50:38.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.990+0000 7ff08ddce640 1 --2- 172.21.15.84:0/2063634990 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff06c06b070 0x7ff06c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:38.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.991+0000 7ff08ddce640 1 --2- 172.21.15.84:0/2063634990 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff06c06b070 0x7ff06c06d530 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff084036000 tx=0x7ff08403c040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:38.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:38.994+0000 7ff07e7fc640 1 -- 172.21.15.84:0/2063634990 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff0880c5b10 con 0x7ff0880caac0 2023-10-16T09:50:39.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.214+0000 7ff090032640 1 -- 172.21.15.84:0/2063634990 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0880ad4d0 con 0x7ff06c06b070 2023-10-16T09:50:39.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.360+0000 7ff07e7fc640 1 -- 172.21.15.84:0/2063634990 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ff084031cc0 con 0x7ff06c06b070 2023-10-16T09:50:39.363 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-10-16T09:50:39.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.364+0000 7ff090032640 1 -- 172.21.15.84:0/2063634990 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff06c06b070 msgr2=0x7ff06c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:39.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.364+0000 7ff090032640 1 --2- 172.21.15.84:0/2063634990 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff06c06b070 0x7ff06c06d530 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff084036000 tx=0x7ff08403c040 comp rx=0 tx=0).stop 2023-10-16T09:50:39.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.364+0000 7ff090032640 1 -- 172.21.15.84:0/2063634990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0880caac0 msgr2=0x7ff0880c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:39.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.364+0000 7ff090032640 1 --2- 172.21.15.84:0/2063634990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0880caac0 0x7ff0880c7850 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff07800c410 tx=0x7ff07800c8d0 comp rx=0 tx=0).stop 2023-10-16T09:50:39.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.364+0000 7ff090032640 1 -- 172.21.15.84:0/2063634990 shutdown_connections 2023-10-16T09:50:39.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.364+0000 7ff090032640 1 --2- 172.21.15.84:0/2063634990 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff06c06b070 0x7ff06c06d530 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:39.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.364+0000 7ff090032640 1 --2- 172.21.15.84:0/2063634990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff08815ad70 0x7ff0880c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:39.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.364+0000 7ff090032640 1 --2- 172.21.15.84:0/2063634990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0880caac0 0x7ff0880c7850 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:39.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.364+0000 7ff090032640 1 --2- 172.21.15.84:0/2063634990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0880c9ca0 0x7ff0881f6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:39.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.364+0000 7ff090032640 1 -- 172.21.15.84:0/2063634990 >> 172.21.15.84:0/2063634990 conn(0x7ff0880ce9b0 msgr2=0x7ff0880ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:39.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.364+0000 7ff090032640 1 -- 172.21.15.84:0/2063634990 shutdown_connections 2023-10-16T09:50:39.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.364+0000 7ff090032640 1 -- 172.21.15.84:0/2063634990 wait complete. 2023-10-16T09:50:39.386 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-10-16T09:50:39.387 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir8 1m 2023-10-16T09:50:39.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.595+0000 7f5610d41640 1 Processor -- start 2023-10-16T09:50:39.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.596+0000 7f5610d41640 1 -- start start 2023-10-16T09:50:39.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.597+0000 7f5610d41640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f560c160ad0 0x7f560c0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:39.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.597+0000 7f5610d41640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f560c0ce350 0x7f560c160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:39.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.597+0000 7f5610d41640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f560c0ae260 0x7f560c0ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:39.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.597+0000 7f5610d41640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560c0ce800 con 0x7f560c0ce350 2023-10-16T09:50:39.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.597+0000 7f5610d41640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560c0cdec0 con 0x7f560c160ad0 2023-10-16T09:50:39.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.597+0000 7f5610d41640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0acdd0 con 0x7f560c0ae260 2023-10-16T09:50:39.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.597+0000 7f5609d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f560c0ce350 0x7f560c160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:39.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.598+0000 7f5609d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f560c0ce350 0x7f560c160eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43750/0 (socket says 172.21.15.84:43750) 2023-10-16T09:50:39.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.598+0000 7f5609d9b640 1 -- 172.21.15.84:0/15367978 learned_addr learned my addr 172.21.15.84:0/15367978 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:39.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.598+0000 7f560a59c640 1 --2- 172.21.15.84:0/15367978 >> v2:172.21.15.157:3300/0 conn(0x7f560c160ad0 0x7f560c0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:39.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.598+0000 7f560ad9d640 1 --2- 172.21.15.84:0/15367978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f560c0ae260 0x7f560c0ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:39.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.598+0000 7f5609d9b640 1 -- 172.21.15.84:0/15367978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f560c0ae260 msgr2=0x7f560c0ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:39.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.598+0000 7f5609d9b640 1 --2- 172.21.15.84:0/15367978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f560c0ae260 0x7f560c0ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:39.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.598+0000 7f5609d9b640 1 -- 172.21.15.84:0/15367978 >> v2:172.21.15.157:3300/0 conn(0x7f560c160ad0 msgr2=0x7f560c0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:39.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.598+0000 7f5609d9b640 1 --2- 172.21.15.84:0/15367978 >> v2:172.21.15.157:3300/0 conn(0x7f560c160ad0 0x7f560c0ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:39.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.598+0000 7f5609d9b640 1 -- 172.21.15.84:0/15367978 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c1613f0 con 0x7f560c0ce350 2023-10-16T09:50:39.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.600+0000 7f5609d9b640 1 --2- 172.21.15.84:0/15367978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f560c0ce350 0x7f560c160eb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f560c0721f0 tx=0x7f55f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5bbac36f162a417 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:39.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.600+0000 7f560959a640 1 -- 172.21.15.84:0/15367978 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f4031870 con 0x7f560c0ce350 2023-10-16T09:50:39.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.600+0000 7f560959a640 1 -- 172.21.15.84:0/15367978 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f40319d0 con 0x7f560c0ce350 2023-10-16T09:50:39.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.600+0000 7f560959a640 1 -- 172.21.15.84:0/15367978 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f4031d60 con 0x7f560c0ce350 2023-10-16T09:50:39.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.601+0000 7f5610d41640 1 -- 172.21.15.84:0/15367978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f560c0ce350 msgr2=0x7f560c160eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:39.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.601+0000 7f5610d41640 1 --2- 172.21.15.84:0/15367978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f560c0ce350 0x7f560c160eb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f560c0721f0 tx=0x7f55f402b910 comp rx=0 tx=0).stop 2023-10-16T09:50:39.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.601+0000 7f5610d41640 1 -- 172.21.15.84:0/15367978 shutdown_connections 2023-10-16T09:50:39.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.601+0000 7f5610d41640 1 --2- 172.21.15.84:0/15367978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f560c0ce350 0x7f560c160eb0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:39.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.601+0000 7f5610d41640 1 --2- 172.21.15.84:0/15367978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f560c0ae260 0x7f560c0ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:39.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.601+0000 7f5610d41640 1 --2- 172.21.15.84:0/15367978 >> v2:172.21.15.157:3300/0 conn(0x7f560c160ad0 0x7f560c0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:39.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.601+0000 7f5610d41640 1 -- 172.21.15.84:0/15367978 >> 172.21.15.84:0/15367978 conn(0x7f560c06bc10 msgr2=0x7f560c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:39.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.601+0000 7f5610d41640 1 -- 172.21.15.84:0/15367978 shutdown_connections 2023-10-16T09:50:39.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.601+0000 7f5610d41640 1 -- 172.21.15.84:0/15367978 wait complete. 2023-10-16T09:50:39.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.602+0000 7f5610d41640 1 Processor -- start 2023-10-16T09:50:39.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.603+0000 7f5610d41640 1 -- start start 2023-10-16T09:50:39.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.603+0000 7f5610d41640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f560c0ae260 0x7f560c158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:39.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.604+0000 7f5610d41640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f560c0ce350 0x7f560c15db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:39.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.604+0000 7f5610d41640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f560c160ad0 0x7f560c0c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:39.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.604+0000 7f5610d41640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c15e590 con 0x7f560c160ad0 2023-10-16T09:50:39.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.604+0000 7f5610d41640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c15e870 con 0x7f560c0ae260 2023-10-16T09:50:39.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.604+0000 7f5610d41640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c15eb50 con 0x7f560c0ce350 2023-10-16T09:50:39.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.604+0000 7f560ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f560c160ad0 0x7f560c0c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:39.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.604+0000 7f560ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f560c160ad0 0x7f560c0c5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43760/0 (socket says 172.21.15.84:43760) 2023-10-16T09:50:39.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.604+0000 7f560ad9d640 1 -- 172.21.15.84:0/312044407 learned_addr learned my addr 172.21.15.84:0/312044407 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:39.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.604+0000 7f560a59c640 1 --2- 172.21.15.84:0/312044407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f560c0ae260 0x7f560c158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:39.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.604+0000 7f5609d9b640 1 --2- 172.21.15.84:0/312044407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f560c0ce350 0x7f560c15db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:39.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.604+0000 7f560ad9d640 1 -- 172.21.15.84:0/312044407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f560c0ce350 msgr2=0x7f560c15db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:39.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.604+0000 7f560ad9d640 1 --2- 172.21.15.84:0/312044407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f560c0ce350 0x7f560c15db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:39.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.604+0000 7f560ad9d640 1 -- 172.21.15.84:0/312044407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f560c0ae260 msgr2=0x7f560c158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:39.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.604+0000 7f560ad9d640 1 --2- 172.21.15.84:0/312044407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f560c0ae260 0x7f560c158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:39.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.604+0000 7f560ad9d640 1 -- 172.21.15.84:0/312044407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f40094d0 con 0x7f560c160ad0 2023-10-16T09:50:39.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.604+0000 7f5609d9b640 1 --2- 172.21.15.84:0/312044407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f560c0ce350 0x7f560c15db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:39.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.605+0000 7f560ad9d640 1 --2- 172.21.15.84:0/312044407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f560c160ad0 0x7f560c0c5050 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f560400d500 tx=0x7f560400d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:39.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.605+0000 7f5602ffd640 1 -- 172.21.15.84:0/312044407 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604014620 con 0x7f560c160ad0 2023-10-16T09:50:39.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.605+0000 7f5602ffd640 1 -- 172.21.15.84:0/312044407 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5604014780 con 0x7f560c160ad0 2023-10-16T09:50:39.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.605+0000 7f5610d41640 1 -- 172.21.15.84:0/312044407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560c15ee30 con 0x7f560c160ad0 2023-10-16T09:50:39.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.605+0000 7f5602ffd640 1 -- 172.21.15.84:0/312044407 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604014b30 con 0x7f560c160ad0 2023-10-16T09:50:39.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.605+0000 7f5610d41640 1 -- 172.21.15.84:0/312044407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560c158ad0 con 0x7f560c160ad0 2023-10-16T09:50:39.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.606+0000 7f5610d41640 1 -- 172.21.15.84:0/312044407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560c04e160 con 0x7f560c160ad0 2023-10-16T09:50:39.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.607+0000 7f5602ffd640 1 -- 172.21.15.84:0/312044407 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f5604014c90 con 0x7f560c160ad0 2023-10-16T09:50:39.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.608+0000 7f5602ffd640 1 --2- 172.21.15.84:0/312044407 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f55ec06b070 0x7f55ec06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:39.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.608+0000 7f5602ffd640 1 -- 172.21.15.84:0/312044407 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f560408a110 con 0x7f560c160ad0 2023-10-16T09:50:39.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.609+0000 7f560a59c640 1 --2- 172.21.15.84:0/312044407 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f55ec06b070 0x7f55ec06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:39.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.609+0000 7f560a59c640 1 --2- 172.21.15.84:0/312044407 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f55ec06b070 0x7f55ec06d530 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f560c160870 tx=0x7f55fc00b040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:39.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.613+0000 7f5602ffd640 1 -- 172.21.15.84:0/312044407 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5604054db0 con 0x7f560c160ad0 2023-10-16T09:50:39.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.830+0000 7f5610d41640 1 -- 172.21.15.84:0/312044407 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f560c0cea50 con 0x7f55ec06b070 2023-10-16T09:50:39.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.967+0000 7f5602ffd640 1 -- 172.21.15.84:0/312044407 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f55fc009c10 con 0x7f55ec06b070 2023-10-16T09:50:39.969 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-10-16T09:50:39.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.970+0000 7f5610d41640 1 -- 172.21.15.84:0/312044407 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f55ec06b070 msgr2=0x7f55ec06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:39.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.970+0000 7f5610d41640 1 --2- 172.21.15.84:0/312044407 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f55ec06b070 0x7f55ec06d530 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f560c160870 tx=0x7f55fc00b040 comp rx=0 tx=0).stop 2023-10-16T09:50:39.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.970+0000 7f5610d41640 1 -- 172.21.15.84:0/312044407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f560c160ad0 msgr2=0x7f560c0c5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:39.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.970+0000 7f5610d41640 1 --2- 172.21.15.84:0/312044407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f560c160ad0 0x7f560c0c5050 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f560400d500 tx=0x7f560400d9c0 comp rx=0 tx=0).stop 2023-10-16T09:50:39.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.971+0000 7f5610d41640 1 -- 172.21.15.84:0/312044407 shutdown_connections 2023-10-16T09:50:39.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.971+0000 7f5610d41640 1 --2- 172.21.15.84:0/312044407 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f55ec06b070 0x7f55ec06d530 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:39.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.971+0000 7f5610d41640 1 --2- 172.21.15.84:0/312044407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f560c160ad0 0x7f560c0c5050 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:39.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.971+0000 7f5610d41640 1 --2- 172.21.15.84:0/312044407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f560c0ce350 0x7f560c15db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:39.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.971+0000 7f5610d41640 1 --2- 172.21.15.84:0/312044407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f560c0ae260 0x7f560c158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:39.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.971+0000 7f5610d41640 1 -- 172.21.15.84:0/312044407 >> 172.21.15.84:0/312044407 conn(0x7f560c06bc10 msgr2=0x7f560c0ccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:39.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.971+0000 7f5610d41640 1 -- 172.21.15.84:0/312044407 shutdown_connections 2023-10-16T09:50:39.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:39.971+0000 7f5610d41640 1 -- 172.21.15.84:0/312044407 wait complete. 2023-10-16T09:50:39.993 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-10-16T09:50:39.994 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir9 1m 2023-10-16T09:50:40.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.205+0000 7f053b97c640 1 Processor -- start 2023-10-16T09:50:40.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.205+0000 7f053b97c640 1 -- start start 2023-10-16T09:50:40.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.207+0000 7f053b97c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f053406a4b0 0x7f05340ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:40.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.207+0000 7f053b97c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05340c9ca0 0x7f05340ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:40.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.207+0000 7f053b97c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05340cb030 con 0x7f053415ad70 2023-10-16T09:50:40.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.207+0000 7f053b97c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05340cae30 con 0x7f053406a4b0 2023-10-16T09:50:40.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.207+0000 7f053b97c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05340c91c0 con 0x7f05340c9ca0 2023-10-16T09:50:40.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.207+0000 7f0538f17640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f053406a4b0 0x7f05340ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:40.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.207+0000 7f0539718640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f053415ad70 0x7f0534069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32790/0 (socket says 172.21.15.84:32790) 2023-10-16T09:50:40.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.207+0000 7f0539718640 1 -- 172.21.15.84:0/3287850029 learned_addr learned my addr 172.21.15.84:0/3287850029 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T09:50:40.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.207+0000 7f0538f17640 1 -- 172.21.15.84:0/3287850029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05340c9ca0 msgr2=0x7f05340ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:50:40.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.207+0000 7f0539f19640 1 --2- 172.21.15.84:0/3287850029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05340c9ca0 0x7f05340ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:40.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.207+0000 7f0538f17640 1 --2- 172.21.15.84:0/3287850029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05340c9ca0 0x7f05340ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:40.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.208+0000 7f0538f17640 1 -- 172.21.15.84:0/3287850029 >> v1:172.21.15.84:6789/0 conn(0x7f053415ad70 legacy=0x7f0534069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:40.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.207+0000 7f052bfff640 1 -- 172.21.15.84:0/3287850029 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655524759 0 0) 0x7f05340cb030 con 0x7f053415ad70 2023-10-16T09:50:40.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.208+0000 7f0538f17640 1 -- 172.21.15.84:0/3287850029 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05340c94a0 con 0x7f053406a4b0 2023-10-16T09:50:40.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.209+0000 7f0538f17640 1 --2- 172.21.15.84:0/3287850029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f053406a4b0 0x7f05340ca3b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f05240097a0 tx=0x7f052402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349dd8c3fb8f37e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:40.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.209+0000 7f052bfff640 1 -- 172.21.15.84:0/3287850029 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052402d040 con 0x7f053406a4b0 2023-10-16T09:50:40.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.209+0000 7f052bfff640 1 -- 172.21.15.84:0/3287850029 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0524031810 con 0x7f053406a4b0 2023-10-16T09:50:40.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.209+0000 7f052bfff640 1 -- 172.21.15.84:0/3287850029 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524031ba0 con 0x7f053406a4b0 2023-10-16T09:50:40.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.210+0000 7f053b97c640 1 -- 172.21.15.84:0/3287850029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f053406a4b0 msgr2=0x7f05340ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:40.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.210+0000 7f053b97c640 1 --2- 172.21.15.84:0/3287850029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f053406a4b0 0x7f05340ca3b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f05240097a0 tx=0x7f052402b8b0 comp rx=0 tx=0).stop 2023-10-16T09:50:40.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.210+0000 7f053b97c640 1 -- 172.21.15.84:0/3287850029 shutdown_connections 2023-10-16T09:50:40.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.210+0000 7f053b97c640 1 --2- 172.21.15.84:0/3287850029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05340c9ca0 0x7f05340ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:40.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.210+0000 7f053b97c640 1 --2- 172.21.15.84:0/3287850029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f053406a4b0 0x7f05340ca3b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:40.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.210+0000 7f053b97c640 1 -- 172.21.15.84:0/3287850029 >> 172.21.15.84:0/3287850029 conn(0x7f05340ce9b0 msgr2=0x7f05340cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:40.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.210+0000 7f053b97c640 1 -- 172.21.15.84:0/3287850029 shutdown_connections 2023-10-16T09:50:40.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.210+0000 7f053b97c640 1 -- 172.21.15.84:0/3287850029 wait complete. 2023-10-16T09:50:40.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.212+0000 7f053b97c640 1 Processor -- start 2023-10-16T09:50:40.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.212+0000 7f053b97c640 1 -- start start 2023-10-16T09:50:40.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.213+0000 7f053b97c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f053406a4b0 0x7f05340a9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:40.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.213+0000 7f053b97c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05340c9ca0 0x7f05340c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:40.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.213+0000 7f053b97c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f053415ad70 0x7f053415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:40.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.213+0000 7f053b97c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05340c6cd0 con 0x7f053415ad70 2023-10-16T09:50:40.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.213+0000 7f053b97c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05340c6360 con 0x7f053406a4b0 2023-10-16T09:50:40.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.213+0000 7f053b97c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05340c6640 con 0x7f05340c9ca0 2023-10-16T09:50:40.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.213+0000 7f0539f19640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f053415ad70 0x7f053415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:40.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.213+0000 7f0539f19640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f053415ad70 0x7f053415d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43788/0 (socket says 172.21.15.84:43788) 2023-10-16T09:50:40.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.213+0000 7f0539f19640 1 -- 172.21.15.84:0/2533678084 learned_addr learned my addr 172.21.15.84:0/2533678084 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T09:50:40.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.213+0000 7f0539718640 1 --2- 172.21.15.84:0/2533678084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f053406a4b0 0x7f05340a9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:40.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.213+0000 7f0538f17640 1 --2- 172.21.15.84:0/2533678084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05340c9ca0 0x7f05340c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:40.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.213+0000 7f0539f19640 1 -- 172.21.15.84:0/2533678084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05340c9ca0 msgr2=0x7f05340c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:40.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.213+0000 7f0539f19640 1 --2- 172.21.15.84:0/2533678084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05340c9ca0 0x7f05340c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:40.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.213+0000 7f0539f19640 1 -- 172.21.15.84:0/2533678084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f053406a4b0 msgr2=0x7f05340a9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:40.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.213+0000 7f0539f19640 1 --2- 172.21.15.84:0/2533678084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f053406a4b0 0x7f05340a9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:40.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.213+0000 7f0539f19640 1 -- 172.21.15.84:0/2533678084 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0524009400 con 0x7f053415ad70 2023-10-16T09:50:40.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.213+0000 7f0538f17640 1 --2- 172.21.15.84:0/2533678084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05340c9ca0 0x7f05340c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:50:40.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.214+0000 7f0539f19640 1 --2- 172.21.15.84:0/2533678084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f053415ad70 0x7f053415d8b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f052c00cd70 tx=0x7f052c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:40.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.214+0000 7f0529ffb640 1 -- 172.21.15.84:0/2533678084 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c003b90 con 0x7f053415ad70 2023-10-16T09:50:40.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.214+0000 7f0529ffb640 1 -- 172.21.15.84:0/2533678084 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c003cf0 con 0x7f053415ad70 2023-10-16T09:50:40.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.214+0000 7f0529ffb640 1 -- 172.21.15.84:0/2533678084 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c004240 con 0x7f053415ad70 2023-10-16T09:50:40.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.214+0000 7f053b97c640 1 -- 172.21.15.84:0/2533678084 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05340c6920 con 0x7f053415ad70 2023-10-16T09:50:40.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.214+0000 7f053b97c640 1 -- 172.21.15.84:0/2533678084 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05340c5fc0 con 0x7f053415ad70 2023-10-16T09:50:40.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.215+0000 7f053b97c640 1 -- 172.21.15.84:0/2533678084 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05340c7120 con 0x7f053415ad70 2023-10-16T09:50:40.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.216+0000 7f0529ffb640 1 -- 172.21.15.84:0/2533678084 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f052c0043a0 con 0x7f053415ad70 2023-10-16T09:50:40.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.217+0000 7f0529ffb640 1 --2- 172.21.15.84:0/2533678084 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f051806b070 0x7f051806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:50:40.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.217+0000 7f0529ffb640 1 -- 172.21.15.84:0/2533678084 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f052c012070 con 0x7f053415ad70 2023-10-16T09:50:40.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.218+0000 7f0539718640 1 --2- 172.21.15.84:0/2533678084 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f051806b070 0x7f051806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:50:40.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.218+0000 7f0539718640 1 --2- 172.21.15.84:0/2533678084 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f051806b070 0x7f051806d530 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0530000f60 tx=0x7f0530007c00 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:50:40.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.222+0000 7f0529ffb640 1 -- 172.21.15.84:0/2533678084 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f052c0551f0 con 0x7f053415ad70 2023-10-16T09:50:40.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.437+0000 7f053b97c640 1 -- 172.21.15.84:0/2533678084 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f05340c9780 con 0x7f051806b070 2023-10-16T09:50:40.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.573+0000 7f0529ffb640 1 -- 172.21.15.84:0/2533678084 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f0530007270 con 0x7f051806b070 2023-10-16T09:50:40.575 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-10-16T09:50:40.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.576+0000 7f053b97c640 1 -- 172.21.15.84:0/2533678084 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f051806b070 msgr2=0x7f051806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:40.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.576+0000 7f053b97c640 1 --2- 172.21.15.84:0/2533678084 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f051806b070 0x7f051806d530 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0530000f60 tx=0x7f0530007c00 comp rx=0 tx=0).stop 2023-10-16T09:50:40.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.576+0000 7f053b97c640 1 -- 172.21.15.84:0/2533678084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f053415ad70 msgr2=0x7f053415d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:50:40.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.576+0000 7f053b97c640 1 --2- 172.21.15.84:0/2533678084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f053415ad70 0x7f053415d8b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f052c00cd70 tx=0x7f052c00a660 comp rx=0 tx=0).stop 2023-10-16T09:50:40.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.576+0000 7f053b97c640 1 -- 172.21.15.84:0/2533678084 shutdown_connections 2023-10-16T09:50:40.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.576+0000 7f053b97c640 1 --2- 172.21.15.84:0/2533678084 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f051806b070 0x7f051806d530 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:40.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.576+0000 7f053b97c640 1 --2- 172.21.15.84:0/2533678084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f053415ad70 0x7f053415d8b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:40.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.576+0000 7f053b97c640 1 --2- 172.21.15.84:0/2533678084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05340c9ca0 0x7f05340c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:40.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.576+0000 7f053b97c640 1 --2- 172.21.15.84:0/2533678084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f053406a4b0 0x7f05340a9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:50:40.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.576+0000 7f053b97c640 1 -- 172.21.15.84:0/2533678084 >> 172.21.15.84:0/2533678084 conn(0x7f05340ce9b0 msgr2=0x7f05340afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:50:40.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.576+0000 7f053b97c640 1 -- 172.21.15.84:0/2533678084 shutdown_connections 2023-10-16T09:50:40.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T09:50:40.576+0000 7f053b97c640 1 -- 172.21.15.84:0/2533678084 wait complete. 2023-10-16T09:50:40.599 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-10-16T09:50:59.285 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:50:59.295 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:51:00.104 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:51:00.102+0000 7f5db8239640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:51:00.104 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:51:00.102+0000 7f5db8239640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:51:00.105 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:51:00.102+0000 7f5db8239640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:51:00.105 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:51:00.102+0000 7f5db8239640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:51:00.105 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:51:00.102+0000 7f5db8239640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:51:29.449 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:51:29.459 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:51:59.601 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:51:59.611 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:52:29.750 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:52:29.759 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:52:59.888 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:52:59.897 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:53:01.070 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:53:01.069+0000 7f5db3a30640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:53:01.071 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:53:01.069+0000 7f5db3a30640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:53:01.071 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:53:01.069+0000 7f5db3a30640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:53:01.071 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:53:01.069+0000 7f5db3a30640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:53:01.071 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:53:01.069+0000 7f5db3a30640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:53:30.028 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:53:30.037 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:54:00.174 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:54:00.185 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:54:30.320 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:54:30.330 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:55:00.466 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:55:00.477 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:55:30.621 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:55:30.631 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:56:00.259 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:56:00.258+0000 7f5dc12cb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:56:00.259 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:56:00.258+0000 7f5dc12cb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:56:00.260 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:56:00.258+0000 7f5dc12cb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:56:00.260 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:56:00.258+0000 7f5dc12cb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:56:00.260 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T09:56:00.258+0000 7f5dc12cb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:56:00.766 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:56:00.774 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:56:30.937 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:56:30.947 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:57:01.077 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:57:01.087 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:57:31.351 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:57:31.361 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:58:01.503 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:58:01.513 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:58:31.650 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:58:31.661 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:59:01.815 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:59:01.825 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:59:31.960 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:59:31.973 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:00:02.113 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:00:02.124 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:00:32.286 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:00:32.296 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:01:02.464 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:01:02.474 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:01:21.600 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir0 1m 2023-10-16T10:01:21.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.828+0000 7ff123085640 1 Processor -- start 2023-10-16T10:01:21.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.828+0000 7ff123085640 1 -- start start 2023-10-16T10:01:21.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.829+0000 7ff123085640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff11c06c040 0x7ff11c06c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:21.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.830+0000 7ff123085640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11c0ad970 0x7ff11c15e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:21.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.830+0000 7ff123085640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff11c06c960 con 0x7ff11c15ebc0 2023-10-16T10:01:21.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.830+0000 7ff123085640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff11c15b9d0 con 0x7ff11c06c040 2023-10-16T10:01:21.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.830+0000 7ff123085640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c15bcb0 con 0x7ff11c0ad970 2023-10-16T10:01:21.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.830+0000 7ff122083640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff11c06c040 0x7ff11c06c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:21.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.830+0000 7ff122083640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff11c06c040 0x7ff11c06c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55120/0 (socket says 172.21.15.84:55120) 2023-10-16T10:01:21.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.830+0000 7ff122083640 1 -- 172.21.15.84:0/1816434723 learned_addr learned my addr 172.21.15.84:0/1816434723 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:21.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.830+0000 7ff122083640 1 -- 172.21.15.84:0/1816434723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11c0ad970 msgr2=0x7ff11c15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:21.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.830+0000 7ff122083640 1 --2- 172.21.15.84:0/1816434723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11c0ad970 0x7ff11c15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:21.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.830+0000 7ff122083640 1 -- 172.21.15.84:0/1816434723 >> v1:172.21.15.157:6789/0 conn(0x7ff11c15ebc0 legacy=0x7ff11c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:21.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.830+0000 7ff122083640 1 -- 172.21.15.84:0/1816434723 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c15bf90 con 0x7ff11c06c040 2023-10-16T10:01:21.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.831+0000 7ff122083640 1 --2- 172.21.15.84:0/1816434723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c06c040 0x7ff11c06c420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff1180098a0 tx=0x7ff11802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc8c18d17141e14 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:21.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.832+0000 7ff121081640 1 -- 172.21.15.84:0/1816434723 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118031640 con 0x7ff11c06c040 2023-10-16T10:01:21.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.832+0000 7ff121081640 1 -- 172.21.15.84:0/1816434723 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1180317a0 con 0x7ff11c06c040 2023-10-16T10:01:21.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.832+0000 7ff121081640 1 -- 172.21.15.84:0/1816434723 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118031b30 con 0x7ff11c06c040 2023-10-16T10:01:21.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.833+0000 7ff123085640 1 -- 172.21.15.84:0/1816434723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c06c040 msgr2=0x7ff11c06c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:21.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.833+0000 7ff123085640 1 --2- 172.21.15.84:0/1816434723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c06c040 0x7ff11c06c420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff1180098a0 tx=0x7ff11802b8f0 comp rx=0 tx=0).stop 2023-10-16T10:01:21.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.833+0000 7ff123085640 1 -- 172.21.15.84:0/1816434723 shutdown_connections 2023-10-16T10:01:21.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.833+0000 7ff123085640 1 --2- 172.21.15.84:0/1816434723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11c0ad970 0x7ff11c15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:21.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.833+0000 7ff123085640 1 --2- 172.21.15.84:0/1816434723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c06c040 0x7ff11c06c420 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:21.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.833+0000 7ff123085640 1 -- 172.21.15.84:0/1816434723 >> 172.21.15.84:0/1816434723 conn(0x7ff11c0ce6a0 msgr2=0x7ff11c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:21.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.833+0000 7ff123085640 1 -- 172.21.15.84:0/1816434723 shutdown_connections 2023-10-16T10:01:21.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.833+0000 7ff123085640 1 -- 172.21.15.84:0/1816434723 wait complete. 2023-10-16T10:01:21.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.835+0000 7ff123085640 1 Processor -- start 2023-10-16T10:01:21.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.835+0000 7ff123085640 1 -- start start 2023-10-16T10:01:21.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.836+0000 7ff123085640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c06c040 0x7ff11c0c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:21.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.837+0000 7ff123085640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11c0ad970 0x7ff11c0c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:21.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.837+0000 7ff123085640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff11c15ebc0 0x7ff11c0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:21.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.837+0000 7ff123085640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c0c5ed0 con 0x7ff11c06c040 2023-10-16T10:01:21.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.837+0000 7ff123085640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c0c72b0 con 0x7ff11c15ebc0 2023-10-16T10:01:21.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.837+0000 7ff123085640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c0c74c0 con 0x7ff11c0ad970 2023-10-16T10:01:21.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.837+0000 7ff122083640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c06c040 0x7ff11c0c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:21.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.837+0000 7ff122083640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c06c040 0x7ff11c0c6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55124/0 (socket says 172.21.15.84:55124) 2023-10-16T10:01:21.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.837+0000 7ff122083640 1 -- 172.21.15.84:0/3928657436 learned_addr learned my addr 172.21.15.84:0/3928657436 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:21.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.837+0000 7ff122884640 1 --2- 172.21.15.84:0/3928657436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff11c15ebc0 0x7ff11c0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:21.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.837+0000 7ff121882640 1 --2- 172.21.15.84:0/3928657436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11c0ad970 0x7ff11c0c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:21.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.837+0000 7ff122083640 1 -- 172.21.15.84:0/3928657436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11c0ad970 msgr2=0x7ff11c0c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:21.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.837+0000 7ff122083640 1 --2- 172.21.15.84:0/3928657436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11c0ad970 0x7ff11c0c62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:21.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.837+0000 7ff122083640 1 -- 172.21.15.84:0/3928657436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff11c15ebc0 msgr2=0x7ff11c0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:21.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.837+0000 7ff122083640 1 --2- 172.21.15.84:0/3928657436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff11c15ebc0 0x7ff11c0ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:21.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.837+0000 7ff122083640 1 -- 172.21.15.84:0/3928657436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1180094d0 con 0x7ff11c06c040 2023-10-16T10:01:21.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.837+0000 7ff122884640 1 --2- 172.21.15.84:0/3928657436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff11c15ebc0 0x7ff11c0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:21.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.837+0000 7ff121882640 1 --2- 172.21.15.84:0/3928657436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11c0ad970 0x7ff11c0c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:21.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.838+0000 7ff122083640 1 --2- 172.21.15.84:0/3928657436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c06c040 0x7ff11c0c6d70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff1180098a0 tx=0x7ff1180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:21.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.838+0000 7ff112ffd640 1 -- 172.21.15.84:0/3928657436 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11803ac60 con 0x7ff11c06c040 2023-10-16T10:01:21.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.838+0000 7ff112ffd640 1 -- 172.21.15.84:0/3928657436 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11803ade0 con 0x7ff11c06c040 2023-10-16T10:01:21.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.838+0000 7ff123085640 1 -- 172.21.15.84:0/3928657436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11c0c77a0 con 0x7ff11c06c040 2023-10-16T10:01:21.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.838+0000 7ff112ffd640 1 -- 172.21.15.84:0/3928657436 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118044630 con 0x7ff11c06c040 2023-10-16T10:01:21.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.838+0000 7ff123085640 1 -- 172.21.15.84:0/3928657436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11c0c5af0 con 0x7ff11c06c040 2023-10-16T10:01:21.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.840+0000 7ff123085640 1 -- 172.21.15.84:0/3928657436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11c06b330 con 0x7ff11c06c040 2023-10-16T10:01:21.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.840+0000 7ff112ffd640 1 -- 172.21.15.84:0/3928657436 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7ff11802f050 con 0x7ff11c06c040 2023-10-16T10:01:21.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.841+0000 7ff112ffd640 1 --2- 172.21.15.84:0/3928657436 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff10006b070 0x7ff10006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:21.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.841+0000 7ff112ffd640 1 -- 172.21.15.84:0/3928657436 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff1180abcd0 con 0x7ff11c06c040 2023-10-16T10:01:21.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.841+0000 7ff121882640 1 --2- 172.21.15.84:0/3928657436 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff10006b070 0x7ff10006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:21.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.842+0000 7ff121882640 1 --2- 172.21.15.84:0/3928657436 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff10006b070 0x7ff10006d530 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff11c1f6fc0 tx=0x7ff10c008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:21.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:21.847+0000 7ff112ffd640 1 -- 172.21.15.84:0/3928657436 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff118076a10 con 0x7ff11c06c040 2023-10-16T10:01:22.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.066+0000 7ff123085640 1 -- 172.21.15.84:0/3928657436 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff11c0cc550 con 0x7ff10006b070 2023-10-16T10:01:22.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.256+0000 7ff112ffd640 1 -- 172.21.15.84:0/3928657436 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff10c00adc0 con 0x7ff10006b070 2023-10-16T10:01:22.259 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-10-16T10:01:22.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.260+0000 7ff123085640 1 -- 172.21.15.84:0/3928657436 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff10006b070 msgr2=0x7ff10006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:22.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.260+0000 7ff123085640 1 --2- 172.21.15.84:0/3928657436 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff10006b070 0x7ff10006d530 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff11c1f6fc0 tx=0x7ff10c008040 comp rx=0 tx=0).stop 2023-10-16T10:01:22.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.260+0000 7ff123085640 1 -- 172.21.15.84:0/3928657436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c06c040 msgr2=0x7ff11c0c6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:22.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.260+0000 7ff123085640 1 --2- 172.21.15.84:0/3928657436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c06c040 0x7ff11c0c6d70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff1180098a0 tx=0x7ff1180046c0 comp rx=0 tx=0).stop 2023-10-16T10:01:22.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.260+0000 7ff123085640 1 -- 172.21.15.84:0/3928657436 shutdown_connections 2023-10-16T10:01:22.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.260+0000 7ff123085640 1 --2- 172.21.15.84:0/3928657436 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff10006b070 0x7ff10006d530 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:22.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.260+0000 7ff123085640 1 --2- 172.21.15.84:0/3928657436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff11c15ebc0 0x7ff11c0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:22.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.260+0000 7ff123085640 1 --2- 172.21.15.84:0/3928657436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11c0ad970 0x7ff11c0c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:22.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.260+0000 7ff123085640 1 --2- 172.21.15.84:0/3928657436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11c06c040 0x7ff11c0c6d70 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:22.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.260+0000 7ff123085640 1 -- 172.21.15.84:0/3928657436 >> 172.21.15.84:0/3928657436 conn(0x7ff11c0ce6a0 msgr2=0x7ff11c15dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:22.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.261+0000 7ff123085640 1 -- 172.21.15.84:0/3928657436 shutdown_connections 2023-10-16T10:01:22.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.261+0000 7ff123085640 1 -- 172.21.15.84:0/3928657436 wait complete. 2023-10-16T10:01:22.282 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-10-16T10:01:22.283 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir1 1m 2023-10-16T10:01:22.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.500+0000 7facd651d640 1 Processor -- start 2023-10-16T10:01:22.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.500+0000 7facd651d640 1 -- start start 2023-10-16T10:01:22.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.502+0000 7facd651d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7facd015ad70 0x7facd0069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:22.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.502+0000 7facd651d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7facd00caac0 0x7facd006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:22.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.502+0000 7facd651d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facd00cb030 con 0x7facd00c9ca0 2023-10-16T10:01:22.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.502+0000 7facd651d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facd00c91c0 con 0x7facd00caac0 2023-10-16T10:01:22.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.502+0000 7facd651d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facd00c94a0 con 0x7facd015ad70 2023-10-16T10:01:22.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.502+0000 7facd4aba640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7facd00caac0 0x7facd006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:22.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.502+0000 7facc7fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7facd00c9ca0 0x7facd00ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32846/0 (socket says 172.21.15.84:32846) 2023-10-16T10:01:22.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.502+0000 7facd4aba640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7facd00caac0 0x7facd006a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55126/0 (socket says 172.21.15.84:55126) 2023-10-16T10:01:22.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.502+0000 7facc7fff640 1 -- 172.21.15.84:0/3711888232 learned_addr learned my addr 172.21.15.84:0/3711888232 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:22.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.502+0000 7faccffff640 1 --2- 172.21.15.84:0/3711888232 >> v2:172.21.15.157:3300/0 conn(0x7facd015ad70 0x7facd0069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:22.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.502+0000 7facd4aba640 1 -- 172.21.15.84:0/3711888232 >> v2:172.21.15.157:3300/0 conn(0x7facd015ad70 msgr2=0x7facd0069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:22.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.502+0000 7facd4aba640 1 --2- 172.21.15.84:0/3711888232 >> v2:172.21.15.157:3300/0 conn(0x7facd015ad70 0x7facd0069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:22.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.502+0000 7facd4aba640 1 -- 172.21.15.84:0/3711888232 >> v1:172.21.15.84:6789/0 conn(0x7facd00c9ca0 legacy=0x7facd00ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:22.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.503+0000 7facd4aba640 1 -- 172.21.15.84:0/3711888232 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd00c9780 con 0x7facd00caac0 2023-10-16T10:01:22.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.503+0000 7faccffff640 1 --2- 172.21.15.84:0/3711888232 >> v2:172.21.15.157:3300/0 conn(0x7facd015ad70 0x7facd0069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:22.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.504+0000 7facd4aba640 1 --2- 172.21.15.84:0/3711888232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facd00caac0 0x7facd006a2e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7facd00cf780 tx=0x7facc802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af79deed165078b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:22.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.504+0000 7faccf7fe640 1 -- 172.21.15.84:0/3711888232 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc802cc70 con 0x7facd00caac0 2023-10-16T10:01:22.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.504+0000 7faccf7fe640 1 -- 172.21.15.84:0/3711888232 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc802cdd0 con 0x7facd00caac0 2023-10-16T10:01:22.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.504+0000 7faccf7fe640 1 -- 172.21.15.84:0/3711888232 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc8034660 con 0x7facd00caac0 2023-10-16T10:01:22.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.504+0000 7facd651d640 1 -- 172.21.15.84:0/3711888232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facd00caac0 msgr2=0x7facd006a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:22.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.504+0000 7facd651d640 1 --2- 172.21.15.84:0/3711888232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facd00caac0 0x7facd006a2e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7facd00cf780 tx=0x7facc802bec0 comp rx=0 tx=0).stop 2023-10-16T10:01:22.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.505+0000 7facd651d640 1 -- 172.21.15.84:0/3711888232 shutdown_connections 2023-10-16T10:01:22.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.505+0000 7facd651d640 1 --2- 172.21.15.84:0/3711888232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facd00caac0 0x7facd006a2e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:22.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.505+0000 7facd651d640 1 --2- 172.21.15.84:0/3711888232 >> v2:172.21.15.157:3300/0 conn(0x7facd015ad70 0x7facd0069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:22.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.505+0000 7facd651d640 1 -- 172.21.15.84:0/3711888232 >> 172.21.15.84:0/3711888232 conn(0x7facd00ce9b0 msgr2=0x7facd00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:22.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.505+0000 7facd651d640 1 -- 172.21.15.84:0/3711888232 shutdown_connections 2023-10-16T10:01:22.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.505+0000 7facd651d640 1 -- 172.21.15.84:0/3711888232 wait complete. 2023-10-16T10:01:22.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.506+0000 7facd651d640 1 Processor -- start 2023-10-16T10:01:22.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.507+0000 7facd651d640 1 -- start start 2023-10-16T10:01:22.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.508+0000 7facd651d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facd00c9ca0 0x7facd00b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:22.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.508+0000 7facd651d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facd00caac0 0x7facd01507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:22.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.508+0000 7facd651d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facd015ad70 0x7facd00cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:22.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.508+0000 7facd651d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd00c5a80 con 0x7facd00caac0 2023-10-16T10:01:22.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.508+0000 7facd651d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd00b39c0 con 0x7facd015ad70 2023-10-16T10:01:22.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.508+0000 7facd651d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd015b450 con 0x7facd00c9ca0 2023-10-16T10:01:22.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.508+0000 7facc7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facd00caac0 0x7facd01507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:22.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.508+0000 7facc7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facd00caac0 0x7facd01507b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55130/0 (socket says 172.21.15.84:55130) 2023-10-16T10:01:22.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.508+0000 7facc7fff640 1 -- 172.21.15.84:0/2535434525 learned_addr learned my addr 172.21.15.84:0/2535434525 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:22.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.508+0000 7facd4aba640 1 --2- 172.21.15.84:0/2535434525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facd015ad70 0x7facd00cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:22.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.508+0000 7faccffff640 1 --2- 172.21.15.84:0/2535434525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facd00c9ca0 0x7facd00b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:22.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.508+0000 7facc7fff640 1 -- 172.21.15.84:0/2535434525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facd00c9ca0 msgr2=0x7facd00b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:22.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.508+0000 7facc7fff640 1 --2- 172.21.15.84:0/2535434525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facd00c9ca0 0x7facd00b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:22.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.508+0000 7facc7fff640 1 -- 172.21.15.84:0/2535434525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facd015ad70 msgr2=0x7facd00cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:22.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.508+0000 7facc7fff640 1 --2- 172.21.15.84:0/2535434525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facd015ad70 0x7facd00cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:22.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.508+0000 7facc7fff640 1 -- 172.21.15.84:0/2535434525 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc8009a70 con 0x7facd00caac0 2023-10-16T10:01:22.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.509+0000 7facc7fff640 1 --2- 172.21.15.84:0/2535434525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facd00caac0 0x7facd01507b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7facb800c4a0 tx=0x7facb800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:22.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.509+0000 7faccd7fa640 1 -- 172.21.15.84:0/2535434525 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8005ce0 con 0x7facd00caac0 2023-10-16T10:01:22.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.509+0000 7faccd7fa640 1 -- 172.21.15.84:0/2535434525 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb8005e40 con 0x7facd00caac0 2023-10-16T10:01:22.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.509+0000 7facd651d640 1 -- 172.21.15.84:0/2535434525 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd015b730 con 0x7facd00caac0 2023-10-16T10:01:22.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.509+0000 7faccd7fa640 1 -- 172.21.15.84:0/2535434525 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8007270 con 0x7facd00caac0 2023-10-16T10:01:22.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.509+0000 7facd651d640 1 -- 172.21.15.84:0/2535434525 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd015bcf0 con 0x7facd00caac0 2023-10-16T10:01:22.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.510+0000 7facd651d640 1 -- 172.21.15.84:0/2535434525 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd004e160 con 0x7facd00caac0 2023-10-16T10:01:22.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.511+0000 7faccd7fa640 1 -- 172.21.15.84:0/2535434525 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7facb80073d0 con 0x7facd00caac0 2023-10-16T10:01:22.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.512+0000 7faccd7fa640 1 --2- 172.21.15.84:0/2535434525 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7facb006b070 0x7facb006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:22.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.512+0000 7faccd7fa640 1 -- 172.21.15.84:0/2535434525 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7facb8088f30 con 0x7facd00caac0 2023-10-16T10:01:22.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.512+0000 7faccffff640 1 --2- 172.21.15.84:0/2535434525 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7facb006b070 0x7facb006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:22.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.513+0000 7faccffff640 1 --2- 172.21.15.84:0/2535434525 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7facb006b070 0x7facb006d530 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7facc000b840 tx=0x7facc0008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:22.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.516+0000 7faccd7fa640 1 -- 172.21.15.84:0/2535434525 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7facb8053c70 con 0x7facd00caac0 2023-10-16T10:01:22.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.731+0000 7facd651d640 1 -- 172.21.15.84:0/2535434525 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7facd00ad4d0 con 0x7facb006b070 2023-10-16T10:01:22.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.916+0000 7faccd7fa640 1 -- 172.21.15.84:0/2535434525 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7facc000c040 con 0x7facb006b070 2023-10-16T10:01:22.918 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-10-16T10:01:22.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.920+0000 7facd651d640 1 -- 172.21.15.84:0/2535434525 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7facb006b070 msgr2=0x7facb006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:22.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.920+0000 7facd651d640 1 --2- 172.21.15.84:0/2535434525 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7facb006b070 0x7facb006d530 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7facc000b840 tx=0x7facc0008040 comp rx=0 tx=0).stop 2023-10-16T10:01:22.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.920+0000 7facd651d640 1 -- 172.21.15.84:0/2535434525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facd00caac0 msgr2=0x7facd01507b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:22.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.920+0000 7facd651d640 1 --2- 172.21.15.84:0/2535434525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facd00caac0 0x7facd01507b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7facb800c4a0 tx=0x7facb800c960 comp rx=0 tx=0).stop 2023-10-16T10:01:22.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.920+0000 7facd651d640 1 -- 172.21.15.84:0/2535434525 shutdown_connections 2023-10-16T10:01:22.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.920+0000 7facd651d640 1 --2- 172.21.15.84:0/2535434525 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7facb006b070 0x7facb006d530 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.920+0000 7facd651d640 1 --2- 172.21.15.84:0/2535434525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facd015ad70 0x7facd00cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.920+0000 7facd651d640 1 --2- 172.21.15.84:0/2535434525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facd00caac0 0x7facd01507b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.920+0000 7facd651d640 1 --2- 172.21.15.84:0/2535434525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facd00c9ca0 0x7facd00b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.920+0000 7facd651d640 1 -- 172.21.15.84:0/2535434525 >> 172.21.15.84:0/2535434525 conn(0x7facd00ce9b0 msgr2=0x7facd00acb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.920+0000 7facd651d640 1 -- 172.21.15.84:0/2535434525 shutdown_connections 2023-10-16T10:01:22.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:22.920+0000 7facd651d640 1 -- 172.21.15.84:0/2535434525 wait complete. 2023-10-16T10:01:22.942 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-10-16T10:01:22.943 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir2 1m 2023-10-16T10:01:23.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.151+0000 7f4ceec34640 1 Processor -- start 2023-10-16T10:01:23.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.151+0000 7f4ceec34640 1 -- start start 2023-10-16T10:01:23.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.152+0000 7f4ceec34640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ce806a4b0 0x7f4ce80ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:23.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.152+0000 7f4ceec34640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4ce80c9ca0 0x7f4ce80ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:23.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.152+0000 7f4ceec34640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce80cb030 con 0x7f4ce815ad70 2023-10-16T10:01:23.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.152+0000 7f4ceec34640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce80cae30 con 0x7f4ce80c9ca0 2023-10-16T10:01:23.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.152+0000 7f4ceec34640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce80c91c0 con 0x7f4ce806a4b0 2023-10-16T10:01:23.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.153+0000 7f4cec9d0640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4ce815ad70 0x7f4ce8069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32852/0 (socket says 172.21.15.84:32852) 2023-10-16T10:01:23.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.153+0000 7f4cec9d0640 1 -- 172.21.15.84:0/4194578455 learned_addr learned my addr 172.21.15.84:0/4194578455 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:23.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.153+0000 7f4ced1d1640 1 --2- 172.21.15.84:0/4194578455 >> v2:172.21.15.84:3300/0 conn(0x7f4ce80c9ca0 0x7f4ce80ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:23.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.153+0000 7f4ced1d1640 1 -- 172.21.15.84:0/4194578455 >> v2:172.21.15.157:3300/0 conn(0x7f4ce806a4b0 msgr2=0x7f4ce80ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:23.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.153+0000 7f4ced1d1640 1 --2- 172.21.15.84:0/4194578455 >> v2:172.21.15.157:3300/0 conn(0x7f4ce806a4b0 0x7f4ce80ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:23.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.153+0000 7f4ced1d1640 1 -- 172.21.15.84:0/4194578455 >> v1:172.21.15.84:6789/0 conn(0x7f4ce815ad70 legacy=0x7f4ce8069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:23.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.153+0000 7f4ced1d1640 1 -- 172.21.15.84:0/4194578455 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce80c94a0 con 0x7f4ce80c9ca0 2023-10-16T10:01:23.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.154+0000 7f4ced1d1640 1 --2- 172.21.15.84:0/4194578455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ce80c9ca0 0x7f4ce80ca8f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80ce870 tx=0x7f4ce002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67e769b7742a3d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:23.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.155+0000 7f4ce77fe640 1 -- 172.21.15.84:0/4194578455 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0031830 con 0x7f4ce80c9ca0 2023-10-16T10:01:23.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.155+0000 7f4ce77fe640 1 -- 172.21.15.84:0/4194578455 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce0031990 con 0x7f4ce80c9ca0 2023-10-16T10:01:23.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.155+0000 7f4ce77fe640 1 -- 172.21.15.84:0/4194578455 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0031d20 con 0x7f4ce80c9ca0 2023-10-16T10:01:23.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.155+0000 7f4ceec34640 1 -- 172.21.15.84:0/4194578455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ce80c9ca0 msgr2=0x7f4ce80ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:23.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.155+0000 7f4ceec34640 1 --2- 172.21.15.84:0/4194578455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ce80c9ca0 0x7f4ce80ca8f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80ce870 tx=0x7f4ce002b8d0 comp rx=0 tx=0).stop 2023-10-16T10:01:23.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.155+0000 7f4ceec34640 1 -- 172.21.15.84:0/4194578455 shutdown_connections 2023-10-16T10:01:23.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.155+0000 7f4ceec34640 1 --2- 172.21.15.84:0/4194578455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ce80c9ca0 0x7f4ce80ca8f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:23.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.155+0000 7f4ceec34640 1 --2- 172.21.15.84:0/4194578455 >> v2:172.21.15.157:3300/0 conn(0x7f4ce806a4b0 0x7f4ce80ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:23.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.155+0000 7f4ceec34640 1 -- 172.21.15.84:0/4194578455 >> 172.21.15.84:0/4194578455 conn(0x7f4ce80ce9b0 msgr2=0x7f4ce80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:23.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.156+0000 7f4ceec34640 1 -- 172.21.15.84:0/4194578455 shutdown_connections 2023-10-16T10:01:23.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.156+0000 7f4ceec34640 1 -- 172.21.15.84:0/4194578455 wait complete. 2023-10-16T10:01:23.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.157+0000 7f4ceec34640 1 Processor -- start 2023-10-16T10:01:23.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.158+0000 7f4ceec34640 1 -- start start 2023-10-16T10:01:23.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.158+0000 7f4ceec34640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ce806a4b0 0x7f4ce80a8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:23.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.158+0000 7f4ceec34640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ce80c9ca0 0x7f4ce80c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:23.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.158+0000 7f4ceec34640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce815ad70 0x7f4ce80a9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:23.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.158+0000 7f4ceec34640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce80c60d0 con 0x7f4ce806a4b0 2023-10-16T10:01:23.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.158+0000 7f4ceec34640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce80a9650 con 0x7f4ce815ad70 2023-10-16T10:01:23.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.158+0000 7f4ceec34640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce80a98f0 con 0x7f4ce80c9ca0 2023-10-16T10:01:23.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.159+0000 7f4cec9d0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ce806a4b0 0x7f4ce80a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:23.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.159+0000 7f4cec9d0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ce806a4b0 0x7f4ce80a8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55156/0 (socket says 172.21.15.84:55156) 2023-10-16T10:01:23.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.159+0000 7f4cec9d0640 1 -- 172.21.15.84:0/1446077758 learned_addr learned my addr 172.21.15.84:0/1446077758 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:23.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.159+0000 7f4ced1d1640 1 --2- 172.21.15.84:0/1446077758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce815ad70 0x7f4ce80a9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:23.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.159+0000 7f4ce7fff640 1 --2- 172.21.15.84:0/1446077758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ce80c9ca0 0x7f4ce80c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:23.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.159+0000 7f4cec9d0640 1 -- 172.21.15.84:0/1446077758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ce80c9ca0 msgr2=0x7f4ce80c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:23.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.159+0000 7f4cec9d0640 1 --2- 172.21.15.84:0/1446077758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ce80c9ca0 0x7f4ce80c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:23.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.159+0000 7f4cec9d0640 1 -- 172.21.15.84:0/1446077758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce815ad70 msgr2=0x7f4ce80a9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:23.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.159+0000 7f4cec9d0640 1 --2- 172.21.15.84:0/1446077758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce815ad70 0x7f4ce80a9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:23.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.159+0000 7f4cec9d0640 1 -- 172.21.15.84:0/1446077758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce0009400 con 0x7f4ce806a4b0 2023-10-16T10:01:23.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.160+0000 7f4cec9d0640 1 --2- 172.21.15.84:0/1446077758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ce806a4b0 0x7f4ce80a8680 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800cbc0 tx=0x7f4cd800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:23.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.160+0000 7f4ce57fa640 1 -- 172.21.15.84:0/1446077758 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd800aca0 con 0x7f4ce806a4b0 2023-10-16T10:01:23.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.160+0000 7f4ce57fa640 1 -- 172.21.15.84:0/1446077758 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd800ae00 con 0x7f4ce806a4b0 2023-10-16T10:01:23.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.160+0000 7f4ceec34640 1 -- 172.21.15.84:0/1446077758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce80a9bd0 con 0x7f4ce806a4b0 2023-10-16T10:01:23.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.160+0000 7f4ce57fa640 1 -- 172.21.15.84:0/1446077758 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8012600 con 0x7f4ce806a4b0 2023-10-16T10:01:23.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.160+0000 7f4ceec34640 1 -- 172.21.15.84:0/1446077758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce80c5d60 con 0x7f4ce806a4b0 2023-10-16T10:01:23.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.161+0000 7f4ceec34640 1 -- 172.21.15.84:0/1446077758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce804e160 con 0x7f4ce806a4b0 2023-10-16T10:01:23.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.162+0000 7f4ce57fa640 1 -- 172.21.15.84:0/1446077758 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f4cd8007070 con 0x7f4ce806a4b0 2023-10-16T10:01:23.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.163+0000 7f4ce57fa640 1 --2- 172.21.15.84:0/1446077758 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f4cc806b0a0 0x7f4cc806d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:23.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.163+0000 7f4ce57fa640 1 -- 172.21.15.84:0/1446077758 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f4cd8089850 con 0x7f4ce806a4b0 2023-10-16T10:01:23.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.163+0000 7f4ce7fff640 1 --2- 172.21.15.84:0/1446077758 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f4cc806b0a0 0x7f4cc806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:23.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.164+0000 7f4ce7fff640 1 --2- 172.21.15.84:0/1446077758 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f4cc806b0a0 0x7f4cc806d560 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80b3520 tx=0x7f4cd0008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:23.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.168+0000 7f4ce57fa640 1 -- 172.21.15.84:0/1446077758 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4cd8054620 con 0x7f4ce806a4b0 2023-10-16T10:01:23.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.383+0000 7f4ceec34640 1 -- 172.21.15.84:0/1446077758 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ce80c9780 con 0x7f4cc806b0a0 2023-10-16T10:01:23.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.574+0000 7f4ce57fa640 1 -- 172.21.15.84:0/1446077758 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f4cd0009e40 con 0x7f4cc806b0a0 2023-10-16T10:01:23.576 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-10-16T10:01:23.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.577+0000 7f4ceec34640 1 -- 172.21.15.84:0/1446077758 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f4cc806b0a0 msgr2=0x7f4cc806d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:23.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.577+0000 7f4ceec34640 1 --2- 172.21.15.84:0/1446077758 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f4cc806b0a0 0x7f4cc806d560 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80b3520 tx=0x7f4cd0008040 comp rx=0 tx=0).stop 2023-10-16T10:01:23.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.577+0000 7f4ceec34640 1 -- 172.21.15.84:0/1446077758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ce806a4b0 msgr2=0x7f4ce80a8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:23.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.577+0000 7f4ceec34640 1 --2- 172.21.15.84:0/1446077758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ce806a4b0 0x7f4ce80a8680 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800cbc0 tx=0x7f4cd800a060 comp rx=0 tx=0).stop 2023-10-16T10:01:23.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.578+0000 7f4ceec34640 1 -- 172.21.15.84:0/1446077758 shutdown_connections 2023-10-16T10:01:23.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.578+0000 7f4ceec34640 1 --2- 172.21.15.84:0/1446077758 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f4cc806b0a0 0x7f4cc806d560 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:23.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.578+0000 7f4ceec34640 1 --2- 172.21.15.84:0/1446077758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce815ad70 0x7f4ce80a9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:23.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.578+0000 7f4ceec34640 1 --2- 172.21.15.84:0/1446077758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ce80c9ca0 0x7f4ce80c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:23.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.578+0000 7f4ceec34640 1 --2- 172.21.15.84:0/1446077758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ce806a4b0 0x7f4ce80a8680 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:23.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.578+0000 7f4ceec34640 1 -- 172.21.15.84:0/1446077758 >> 172.21.15.84:0/1446077758 conn(0x7f4ce80ce9b0 msgr2=0x7f4ce80b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:23.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.578+0000 7f4ceec34640 1 -- 172.21.15.84:0/1446077758 shutdown_connections 2023-10-16T10:01:23.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.578+0000 7f4ceec34640 1 -- 172.21.15.84:0/1446077758 wait complete. 2023-10-16T10:01:23.599 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-10-16T10:01:23.599 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir3 1m 2023-10-16T10:01:23.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.815+0000 7f6df33cc640 1 Processor -- start 2023-10-16T10:01:23.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.815+0000 7f6df33cc640 1 -- start start 2023-10-16T10:01:23.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.817+0000 7f6df33cc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dec159110 0x7f6dec15cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:23.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.817+0000 7f6df33cc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6dec0ade50 0x7f6dec069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:23.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.817+0000 7f6df33cc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6dec06a2e0 0x7f6dec06b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:23.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.817+0000 7f6df33cc640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dec06a770 con 0x7f6dec0ade50 2023-10-16T10:01:23.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.817+0000 7f6df33cc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dec1594f0 con 0x7f6dec06a2e0 2023-10-16T10:01:23.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.817+0000 7f6df33cc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec15d130 con 0x7f6dec159110 2023-10-16T10:01:23.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.817+0000 7f6df0967640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6dec0ade50 0x7f6dec069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:23.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.817+0000 7f6df0967640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6dec0ade50 0x7f6dec069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55160/0 (socket says 172.21.15.84:55160) 2023-10-16T10:01:23.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.817+0000 7f6df0967640 1 -- 172.21.15.84:0/1966098591 learned_addr learned my addr 172.21.15.84:0/1966098591 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:23.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.818+0000 7f6df1969640 1 --2- 172.21.15.84:0/1966098591 >> v2:172.21.15.157:3300/0 conn(0x7f6dec06a2e0 0x7f6dec06b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:23.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.818+0000 7f6df1168640 1 --2- 172.21.15.84:0/1966098591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dec159110 0x7f6dec15cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:23.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.818+0000 7f6df0967640 1 -- 172.21.15.84:0/1966098591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dec159110 msgr2=0x7f6dec15cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:23.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.818+0000 7f6df0967640 1 --2- 172.21.15.84:0/1966098591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dec159110 0x7f6dec15cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:23.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.818+0000 7f6df0967640 1 -- 172.21.15.84:0/1966098591 >> v2:172.21.15.157:3300/0 conn(0x7f6dec06a2e0 msgr2=0x7f6dec06b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:23.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.818+0000 7f6df0967640 1 --2- 172.21.15.84:0/1966098591 >> v2:172.21.15.157:3300/0 conn(0x7f6dec06a2e0 0x7f6dec06b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:23.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.818+0000 7f6df0967640 1 -- 172.21.15.84:0/1966098591 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec15d400 con 0x7f6dec0ade50 2023-10-16T10:01:23.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.818+0000 7f6df1969640 1 --2- 172.21.15.84:0/1966098591 >> v2:172.21.15.157:3300/0 conn(0x7f6dec06a2e0 0x7f6dec06b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:23.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.819+0000 7f6df0967640 1 --2- 172.21.15.84:0/1966098591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dec0ade50 0x7f6dec069da0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6dec15cb30 tx=0x7f6ddc02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5eff2c840d4f02 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:23.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.820+0000 7f6de3fff640 1 -- 172.21.15.84:0/1966098591 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc031880 con 0x7f6dec0ade50 2023-10-16T10:01:23.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.820+0000 7f6de3fff640 1 -- 172.21.15.84:0/1966098591 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc0319e0 con 0x7f6dec0ade50 2023-10-16T10:01:23.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.820+0000 7f6de3fff640 1 -- 172.21.15.84:0/1966098591 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc031d70 con 0x7f6dec0ade50 2023-10-16T10:01:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.820+0000 7f6df33cc640 1 -- 172.21.15.84:0/1966098591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dec0ade50 msgr2=0x7f6dec069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.820+0000 7f6df33cc640 1 --2- 172.21.15.84:0/1966098591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dec0ade50 0x7f6dec069da0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6dec15cb30 tx=0x7f6ddc02b920 comp rx=0 tx=0).stop 2023-10-16T10:01:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.820+0000 7f6df33cc640 1 -- 172.21.15.84:0/1966098591 shutdown_connections 2023-10-16T10:01:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.820+0000 7f6df33cc640 1 --2- 172.21.15.84:0/1966098591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dec0ade50 0x7f6dec069da0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.820+0000 7f6df33cc640 1 --2- 172.21.15.84:0/1966098591 >> v2:172.21.15.157:3300/0 conn(0x7f6dec06a2e0 0x7f6dec06b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.820+0000 7f6df33cc640 1 --2- 172.21.15.84:0/1966098591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dec159110 0x7f6dec15cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.820+0000 7f6df33cc640 1 -- 172.21.15.84:0/1966098591 >> 172.21.15.84:0/1966098591 conn(0x7f6dec0ce9b0 msgr2=0x7f6dec0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.820+0000 7f6df33cc640 1 -- 172.21.15.84:0/1966098591 shutdown_connections 2023-10-16T10:01:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.820+0000 7f6df33cc640 1 -- 172.21.15.84:0/1966098591 wait complete. 2023-10-16T10:01:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.822+0000 7f6df33cc640 1 Processor -- start 2023-10-16T10:01:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.822+0000 7f6df33cc640 1 -- start start 2023-10-16T10:01:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.823+0000 7f6df33cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dec06a2e0 0x7f6dec0cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.823+0000 7f6df33cc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dec0ade50 0x7f6dec0c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.823+0000 7f6df33cc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dec159110 0x7f6dec0c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:23.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.823+0000 7f6df33cc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec0c9a70 con 0x7f6dec159110 2023-10-16T10:01:23.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.823+0000 7f6df33cc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec0c5e30 con 0x7f6dec06a2e0 2023-10-16T10:01:23.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.823+0000 7f6df33cc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec0c8df0 con 0x7f6dec0ade50 2023-10-16T10:01:23.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.823+0000 7f6df1969640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dec159110 0x7f6dec0c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:23.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.824+0000 7f6df1969640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dec159110 0x7f6dec0c86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55162/0 (socket says 172.21.15.84:55162) 2023-10-16T10:01:23.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.824+0000 7f6df1969640 1 -- 172.21.15.84:0/3883821276 learned_addr learned my addr 172.21.15.84:0/3883821276 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:23.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.824+0000 7f6df1168640 1 --2- 172.21.15.84:0/3883821276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dec06a2e0 0x7f6dec0cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:23.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.824+0000 7f6df1969640 1 -- 172.21.15.84:0/3883821276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dec0ade50 msgr2=0x7f6dec0c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:23.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.824+0000 7f6df1969640 1 --2- 172.21.15.84:0/3883821276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dec0ade50 0x7f6dec0c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:23.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.824+0000 7f6df1969640 1 -- 172.21.15.84:0/3883821276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dec06a2e0 msgr2=0x7f6dec0cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:23.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.824+0000 7f6df1969640 1 --2- 172.21.15.84:0/3883821276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dec06a2e0 0x7f6dec0cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:23.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.824+0000 7f6df1969640 1 -- 172.21.15.84:0/3883821276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc0094d0 con 0x7f6dec159110 2023-10-16T10:01:23.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.824+0000 7f6df1168640 1 --2- 172.21.15.84:0/3883821276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dec06a2e0 0x7f6dec0cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:23.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.824+0000 7f6df1969640 1 --2- 172.21.15.84:0/3883821276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dec159110 0x7f6dec0c86f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6de400cd10 tx=0x7f6de400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:23.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.824+0000 7f6de1ffb640 1 -- 172.21.15.84:0/3883821276 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de400f680 con 0x7f6dec159110 2023-10-16T10:01:23.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.825+0000 7f6de1ffb640 1 -- 172.21.15.84:0/3883821276 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de400f800 con 0x7f6dec159110 2023-10-16T10:01:23.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.825+0000 7f6df33cc640 1 -- 172.21.15.84:0/3883821276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dec0c90d0 con 0x7f6dec159110 2023-10-16T10:01:23.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.825+0000 7f6de1ffb640 1 -- 172.21.15.84:0/3883821276 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de400fbb0 con 0x7f6dec159110 2023-10-16T10:01:23.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.825+0000 7f6df33cc640 1 -- 172.21.15.84:0/3883821276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dec0c9690 con 0x7f6dec159110 2023-10-16T10:01:23.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.826+0000 7f6df33cc640 1 -- 172.21.15.84:0/3883821276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dec0c9bb0 con 0x7f6dec159110 2023-10-16T10:01:23.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.827+0000 7f6de1ffb640 1 -- 172.21.15.84:0/3883821276 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f6de400b040 con 0x7f6dec159110 2023-10-16T10:01:23.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.828+0000 7f6de1ffb640 1 --2- 172.21.15.84:0/3883821276 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f6dd006b070 0x7f6dd006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:23.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.828+0000 7f6de1ffb640 1 -- 172.21.15.84:0/3883821276 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f6de408a490 con 0x7f6dec159110 2023-10-16T10:01:23.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.828+0000 7f6df1168640 1 --2- 172.21.15.84:0/3883821276 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f6dd006b070 0x7f6dd006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:23.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.829+0000 7f6df1168640 1 --2- 172.21.15.84:0/3883821276 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f6dd006b070 0x7f6dd006d530 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6de8008890 tx=0x7f6de8008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:23.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:23.832+0000 7f6de1ffb640 1 -- 172.21.15.84:0/3883821276 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6de4055140 con 0x7f6dec159110 2023-10-16T10:01:24.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.052+0000 7f6df33cc640 1 -- 172.21.15.84:0/3883821276 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f6dec15d6e0 con 0x7f6dd006b070 2023-10-16T10:01:24.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.248+0000 7f6de1ffb640 1 -- 172.21.15.84:0/3883821276 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f6de800c040 con 0x7f6dd006b070 2023-10-16T10:01:24.250 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-10-16T10:01:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.251+0000 7f6df33cc640 1 -- 172.21.15.84:0/3883821276 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f6dd006b070 msgr2=0x7f6dd006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.251+0000 7f6df33cc640 1 --2- 172.21.15.84:0/3883821276 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f6dd006b070 0x7f6dd006d530 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6de8008890 tx=0x7f6de8008040 comp rx=0 tx=0).stop 2023-10-16T10:01:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.251+0000 7f6df33cc640 1 -- 172.21.15.84:0/3883821276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dec159110 msgr2=0x7f6dec0c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:24.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.251+0000 7f6df33cc640 1 --2- 172.21.15.84:0/3883821276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dec159110 0x7f6dec0c86f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6de400cd10 tx=0x7f6de400a650 comp rx=0 tx=0).stop 2023-10-16T10:01:24.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.251+0000 7f6df33cc640 1 -- 172.21.15.84:0/3883821276 shutdown_connections 2023-10-16T10:01:24.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.251+0000 7f6df33cc640 1 --2- 172.21.15.84:0/3883821276 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f6dd006b070 0x7f6dd006d530 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:24.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.251+0000 7f6df33cc640 1 --2- 172.21.15.84:0/3883821276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dec159110 0x7f6dec0c86f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:24.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.251+0000 7f6df33cc640 1 --2- 172.21.15.84:0/3883821276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dec0ade50 0x7f6dec0c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:24.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.251+0000 7f6df33cc640 1 --2- 172.21.15.84:0/3883821276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dec06a2e0 0x7f6dec0cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:24.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.251+0000 7f6df33cc640 1 -- 172.21.15.84:0/3883821276 >> 172.21.15.84:0/3883821276 conn(0x7f6dec0ce9b0 msgr2=0x7f6dec0c52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:24.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.252+0000 7f6df33cc640 1 -- 172.21.15.84:0/3883821276 shutdown_connections 2023-10-16T10:01:24.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.252+0000 7f6df33cc640 1 -- 172.21.15.84:0/3883821276 wait complete. 2023-10-16T10:01:24.272 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-10-16T10:01:24.272 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir4 1m 2023-10-16T10:01:24.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.481+0000 7f1403ebb640 1 Processor -- start 2023-10-16T10:01:24.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.481+0000 7f1403ebb640 1 -- start start 2023-10-16T10:01:24.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.482+0000 7f1403ebb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f13fc159ab0 0x7f13fc0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:24.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.483+0000 7f1403ebb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13fc070360 con 0x7f13fc06c140 2023-10-16T10:01:24.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.483+0000 7f1403ebb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13fc0cd1f0 con 0x7f13fc159e90 2023-10-16T10:01:24.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.483+0000 7f1403ebb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13fc15d890 con 0x7f13fc159ab0 2023-10-16T10:01:24.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.483+0000 7f1402458640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f13fc06c140 0x7f13fc15dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32866/0 (socket says 172.21.15.84:32866) 2023-10-16T10:01:24.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.483+0000 7f1402458640 1 -- 172.21.15.84:0/3625491742 learned_addr learned my addr 172.21.15.84:0/3625491742 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:24.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.483+0000 7f1401c57640 1 --2- 172.21.15.84:0/3625491742 >> v2:172.21.15.157:3300/0 conn(0x7f13fc159ab0 0x7f13fc0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:24.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.484+0000 7f1400c55640 1 -- 172.21.15.84:0/3625491742 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2684693153 0 0) 0x7f13fc070360 con 0x7f13fc06c140 2023-10-16T10:01:24.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.484+0000 7f1400c55640 1 -- 172.21.15.84:0/3625491742 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13e8003520 con 0x7f13fc06c140 2023-10-16T10:01:24.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.484+0000 7f1400c55640 1 -- 172.21.15.84:0/3625491742 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1779631798 0 0) 0x7f13e8003520 con 0x7f13fc06c140 2023-10-16T10:01:24.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.484+0000 7f1400c55640 1 -- 172.21.15.84:0/3625491742 >> v2:172.21.15.157:3300/0 conn(0x7f13fc159ab0 msgr2=0x7f13fc0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:24.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.484+0000 7f1400c55640 1 --2- 172.21.15.84:0/3625491742 >> v2:172.21.15.157:3300/0 conn(0x7f13fc159ab0 0x7f13fc0ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:24.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.484+0000 7f1400c55640 1 -- 172.21.15.84:0/3625491742 >> v1:172.21.15.157:6789/0 conn(0x7f13fc159e90 legacy=0x7f13fc06fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:24.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.484+0000 7f1400c55640 1 -- 172.21.15.84:0/3625491742 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc0cd4d0 con 0x7f13fc06c140 2023-10-16T10:01:24.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.484+0000 7f1400c55640 1 -- 172.21.15.84:0/3625491742 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f13f40019f0 con 0x7f13fc06c140 2023-10-16T10:01:24.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.484+0000 7f1400c55640 1 -- 172.21.15.84:0/3625491742 >> v1:172.21.15.84:6789/0 conn(0x7f13fc06c140 legacy=0x7f13fc15dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:24.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.484+0000 7f1400c55640 1 --2- 172.21.15.84:0/3625491742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13e8003970 0x7f13e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:24.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.484+0000 7f1400c55640 1 --2- 172.21.15.84:0/3625491742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e80047c0 0x7f13e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:24.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.484+0000 7f1400c55640 1 --2- 172.21.15.84:0/3625491742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e80052a0 0x7f13e800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:24.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.484+0000 7f1400c55640 1 -- 172.21.15.84:0/3625491742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc15d890 con 0x7f13e80047c0 2023-10-16T10:01:24.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.484+0000 7f1400c55640 1 -- 172.21.15.84:0/3625491742 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e800c170 con 0x7f13e80052a0 2023-10-16T10:01:24.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.484+0000 7f1400c55640 1 -- 172.21.15.84:0/3625491742 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e800c330 con 0x7f13e8003970 2023-10-16T10:01:24.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.485+0000 7f1401456640 1 --2- 172.21.15.84:0/3625491742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e80047c0 0x7f13e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:24.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.485+0000 7f1401c57640 1 --2- 172.21.15.84:0/3625491742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13e8003970 0x7f13e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:24.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.485+0000 7f1401456640 1 -- 172.21.15.84:0/3625491742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13e8003970 msgr2=0x7f13e8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:24.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.485+0000 7f1402458640 1 --2- 172.21.15.84:0/3625491742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e80052a0 0x7f13e800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:24.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.485+0000 7f1401456640 1 --2- 172.21.15.84:0/3625491742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13e8003970 0x7f13e8003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:24.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.485+0000 7f1401456640 1 -- 172.21.15.84:0/3625491742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e80052a0 msgr2=0x7f13e800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:24.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.485+0000 7f1401456640 1 --2- 172.21.15.84:0/3625491742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e80052a0 0x7f13e800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:24.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.485+0000 7f1401456640 1 -- 172.21.15.84:0/3625491742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e8003520 con 0x7f13e80047c0 2023-10-16T10:01:24.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.485+0000 7f1402458640 1 -- 172.21.15.84:0/3625491742 reap_dead start 2023-10-16T10:01:24.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.486+0000 7f1401456640 1 --2- 172.21.15.84:0/3625491742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e80047c0 0x7f13e8004c40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f13f8001720 tx=0x7f13e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587c1653134c4e5a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:24.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.486+0000 7f1400c55640 1 -- 172.21.15.84:0/3625491742 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f80052d0 con 0x7f13e80047c0 2023-10-16T10:01:24.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.486+0000 7f1400c55640 1 -- 172.21.15.84:0/3625491742 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f802e050 con 0x7f13e80047c0 2023-10-16T10:01:24.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.486+0000 7f1400c55640 1 -- 172.21.15.84:0/3625491742 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8003830 con 0x7f13e80047c0 2023-10-16T10:01:24.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.487+0000 7f1403ebb640 1 -- 172.21.15.84:0/3625491742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e80047c0 msgr2=0x7f13e8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:24.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.487+0000 7f1403ebb640 1 --2- 172.21.15.84:0/3625491742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e80047c0 0x7f13e8004c40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f13f8001720 tx=0x7f13e80057f0 comp rx=0 tx=0).stop 2023-10-16T10:01:24.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.487+0000 7f1403ebb640 1 -- 172.21.15.84:0/3625491742 shutdown_connections 2023-10-16T10:01:24.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.487+0000 7f1403ebb640 1 --2- 172.21.15.84:0/3625491742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e80047c0 0x7f13e8004c40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:24.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.487+0000 7f1403ebb640 1 -- 172.21.15.84:0/3625491742 >> 172.21.15.84:0/3625491742 conn(0x7f13fc0ce9b0 msgr2=0x7f13fc0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:24.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.487+0000 7f1403ebb640 1 -- 172.21.15.84:0/3625491742 shutdown_connections 2023-10-16T10:01:24.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.487+0000 7f1403ebb640 1 -- 172.21.15.84:0/3625491742 wait complete. 2023-10-16T10:01:24.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.489+0000 7f1403ebb640 1 Processor -- start 2023-10-16T10:01:24.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.489+0000 7f1403ebb640 1 -- start start 2023-10-16T10:01:24.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.490+0000 7f1403ebb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e80047c0 0x7f13fc0c2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:24.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.490+0000 7f1403ebb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13fc1f33d0 0x7f13fc06c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:24.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.490+0000 7f1403ebb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13fc0c7d60 0x7f13fc0c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:24.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.490+0000 7f1403ebb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc1f38e0 con 0x7f13e80047c0 2023-10-16T10:01:24.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.490+0000 7f1403ebb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc0c2fd0 con 0x7f13fc0c7d60 2023-10-16T10:01:24.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.490+0000 7f1403ebb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc0c32b0 con 0x7f13fc1f33d0 2023-10-16T10:01:24.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.490+0000 7f1401c57640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e80047c0 0x7f13fc0c2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:24.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.490+0000 7f1401c57640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e80047c0 0x7f13fc0c2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55194/0 (socket says 172.21.15.84:55194) 2023-10-16T10:01:24.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.490+0000 7f1401c57640 1 -- 172.21.15.84:0/2971763170 learned_addr learned my addr 172.21.15.84:0/2971763170 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:24.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.491+0000 7f1401c57640 1 -- 172.21.15.84:0/2971763170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13fc1f33d0 msgr2=0x7f13fc06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:24.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.491+0000 7f1401c57640 1 --2- 172.21.15.84:0/2971763170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13fc1f33d0 0x7f13fc06c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:24.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.491+0000 7f1401c57640 1 -- 172.21.15.84:0/2971763170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13fc0c7d60 msgr2=0x7f13fc0c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:24.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.491+0000 7f1401c57640 1 --2- 172.21.15.84:0/2971763170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13fc0c7d60 0x7f13fc0c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:24.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.491+0000 7f1401c57640 1 -- 172.21.15.84:0/2971763170 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc0c3590 con 0x7f13e80047c0 2023-10-16T10:01:24.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.491+0000 7f1401c57640 1 --2- 172.21.15.84:0/2971763170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e80047c0 0x7f13fc0c2780 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f13f00108e0 tx=0x7f13f0010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:24.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.491+0000 7f13ee7fc640 1 -- 172.21.15.84:0/2971763170 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f000eb60 con 0x7f13e80047c0 2023-10-16T10:01:24.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.491+0000 7f13ee7fc640 1 -- 172.21.15.84:0/2971763170 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f000ecc0 con 0x7f13e80047c0 2023-10-16T10:01:24.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.491+0000 7f1403ebb640 1 -- 172.21.15.84:0/2971763170 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13fc0c3870 con 0x7f13e80047c0 2023-10-16T10:01:24.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.491+0000 7f13ee7fc640 1 -- 172.21.15.84:0/2971763170 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f001c400 con 0x7f13e80047c0 2023-10-16T10:01:24.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.491+0000 7f1403ebb640 1 -- 172.21.15.84:0/2971763170 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13fc0c8700 con 0x7f13e80047c0 2023-10-16T10:01:24.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.493+0000 7f1403ebb640 1 -- 172.21.15.84:0/2971763170 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13fc0c3ab0 con 0x7f13e80047c0 2023-10-16T10:01:24.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.494+0000 7f13ee7fc640 1 -- 172.21.15.84:0/2971763170 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f13f001c560 con 0x7f13e80047c0 2023-10-16T10:01:24.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.496+0000 7f13ee7fc640 1 --2- 172.21.15.84:0/2971763170 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f13e006b070 0x7f13e006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:24.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.496+0000 7f13ee7fc640 1 -- 172.21.15.84:0/2971763170 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f13f008a190 con 0x7f13e80047c0 2023-10-16T10:01:24.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.496+0000 7f1401456640 1 --2- 172.21.15.84:0/2971763170 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f13e006b070 0x7f13e006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:24.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.497+0000 7f1401456640 1 --2- 172.21.15.84:0/2971763170 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f13e006b070 0x7f13e006d530 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f13f8033000 tx=0x7f13f802f000 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:24.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.500+0000 7f13ee7fc640 1 -- 172.21.15.84:0/2971763170 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f13f0054eb0 con 0x7f13e80047c0 2023-10-16T10:01:24.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.724+0000 7f1403ebb640 1 -- 172.21.15.84:0/2971763170 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f13fc0cda90 con 0x7f13e006b070 2023-10-16T10:01:24.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.932+0000 7f13ee7fc640 1 -- 172.21.15.84:0/2971763170 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f13f80052d0 con 0x7f13e006b070 2023-10-16T10:01:24.935 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-10-16T10:01:24.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.936+0000 7f1403ebb640 1 -- 172.21.15.84:0/2971763170 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f13e006b070 msgr2=0x7f13e006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:24.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.936+0000 7f1403ebb640 1 --2- 172.21.15.84:0/2971763170 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f13e006b070 0x7f13e006d530 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f13f8033000 tx=0x7f13f802f000 comp rx=0 tx=0).stop 2023-10-16T10:01:24.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.936+0000 7f1403ebb640 1 -- 172.21.15.84:0/2971763170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e80047c0 msgr2=0x7f13fc0c2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:24.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.936+0000 7f1403ebb640 1 --2- 172.21.15.84:0/2971763170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e80047c0 0x7f13fc0c2780 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f13f00108e0 tx=0x7f13f0010da0 comp rx=0 tx=0).stop 2023-10-16T10:01:24.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.936+0000 7f1403ebb640 1 -- 172.21.15.84:0/2971763170 shutdown_connections 2023-10-16T10:01:24.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.936+0000 7f1403ebb640 1 --2- 172.21.15.84:0/2971763170 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f13e006b070 0x7f13e006d530 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:24.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.936+0000 7f1403ebb640 1 --2- 172.21.15.84:0/2971763170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13fc0c7d60 0x7f13fc0c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:24.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.936+0000 7f1403ebb640 1 --2- 172.21.15.84:0/2971763170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13fc1f33d0 0x7f13fc06c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:24.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.936+0000 7f1403ebb640 1 --2- 172.21.15.84:0/2971763170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e80047c0 0x7f13fc0c2780 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:24.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.936+0000 7f1403ebb640 1 -- 172.21.15.84:0/2971763170 >> 172.21.15.84:0/2971763170 conn(0x7f13fc0ce9b0 msgr2=0x7f13fc0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:24.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.936+0000 7f1403ebb640 1 -- 172.21.15.84:0/2971763170 shutdown_connections 2023-10-16T10:01:24.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:24.936+0000 7f1403ebb640 1 -- 172.21.15.84:0/2971763170 wait complete. 2023-10-16T10:01:24.958 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-10-16T10:01:24.958 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir5 1m 2023-10-16T10:01:25.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.166+0000 7fc47515a640 1 Processor -- start 2023-10-16T10:01:25.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.167+0000 7fc47515a640 1 -- start start 2023-10-16T10:01:25.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.168+0000 7fc47515a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc47006c850 0x7fc4700ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:25.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.168+0000 7fc47515a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc470070360 con 0x7fc470159ab0 2023-10-16T10:01:25.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.168+0000 7fc47515a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4700cd1f0 con 0x7fc470159e90 2023-10-16T10:01:25.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.168+0000 7fc47515a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc47015d890 con 0x7fc47006c850 2023-10-16T10:01:25.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.168+0000 7fc46ed9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc470159ab0 0x7fc47006fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32874/0 (socket says 172.21.15.84:32874) 2023-10-16T10:01:25.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.168+0000 7fc46ed9d640 1 -- 172.21.15.84:0/4111324257 learned_addr learned my addr 172.21.15.84:0/4111324257 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:25.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.169+0000 7fc46f59e640 1 --2- 172.21.15.84:0/4111324257 >> v2:172.21.15.157:3300/0 conn(0x7fc47006c850 0x7fc4700ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:25.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.169+0000 7fc46e59c640 1 -- 172.21.15.84:0/4111324257 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787309158 0 0) 0x7fc470070360 con 0x7fc470159ab0 2023-10-16T10:01:25.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.169+0000 7fc46e59c640 1 -- 172.21.15.84:0/4111324257 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc45c003520 con 0x7fc470159ab0 2023-10-16T10:01:25.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.169+0000 7fc46e59c640 1 -- 172.21.15.84:0/4111324257 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1252624217 0 0) 0x7fc45c003520 con 0x7fc470159ab0 2023-10-16T10:01:25.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.169+0000 7fc46e59c640 1 -- 172.21.15.84:0/4111324257 >> v2:172.21.15.157:3300/0 conn(0x7fc47006c850 msgr2=0x7fc4700ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:25.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.169+0000 7fc46e59c640 1 --2- 172.21.15.84:0/4111324257 >> v2:172.21.15.157:3300/0 conn(0x7fc47006c850 0x7fc4700ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:25.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.169+0000 7fc46e59c640 1 -- 172.21.15.84:0/4111324257 >> v1:172.21.15.157:6789/0 conn(0x7fc470159e90 legacy=0x7fc47006c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:25.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.169+0000 7fc46e59c640 1 -- 172.21.15.84:0/4111324257 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4700cd4d0 con 0x7fc470159ab0 2023-10-16T10:01:25.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.169+0000 7fc46e59c640 1 -- 172.21.15.84:0/4111324257 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc460001690 con 0x7fc470159ab0 2023-10-16T10:01:25.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.169+0000 7fc46e59c640 1 -- 172.21.15.84:0/4111324257 >> v1:172.21.15.84:6789/0 conn(0x7fc470159ab0 legacy=0x7fc47006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:25.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.169+0000 7fc46e59c640 1 --2- 172.21.15.84:0/4111324257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc45c003970 0x7fc45c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:25.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.169+0000 7fc46e59c640 1 --2- 172.21.15.84:0/4111324257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c0047c0 0x7fc45c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:25.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.169+0000 7fc46e59c640 1 --2- 172.21.15.84:0/4111324257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc45c005380 0x7fc45c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:25.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.169+0000 7fc46e59c640 1 -- 172.21.15.84:0/4111324257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47015d890 con 0x7fc45c005380 2023-10-16T10:01:25.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.169+0000 7fc46e59c640 1 -- 172.21.15.84:0/4111324257 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c00c210 con 0x7fc45c0047c0 2023-10-16T10:01:25.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.169+0000 7fc46e59c640 1 -- 172.21.15.84:0/4111324257 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c00e5e0 con 0x7fc45c003970 2023-10-16T10:01:25.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.170+0000 7fc46f59e640 1 --2- 172.21.15.84:0/4111324257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc45c005380 0x7fc45c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:25.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.170+0000 7fc467fff640 1 --2- 172.21.15.84:0/4111324257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c0047c0 0x7fc45c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:25.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.170+0000 7fc46ed9d640 1 --2- 172.21.15.84:0/4111324257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc45c003970 0x7fc45c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:25.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.170+0000 7fc46f59e640 1 -- 172.21.15.84:0/4111324257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc45c003970 msgr2=0x7fc45c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:25.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.170+0000 7fc46f59e640 1 --2- 172.21.15.84:0/4111324257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc45c003970 0x7fc45c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:25.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.170+0000 7fc46f59e640 1 -- 172.21.15.84:0/4111324257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c0047c0 msgr2=0x7fc45c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:25.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.170+0000 7fc46f59e640 1 --2- 172.21.15.84:0/4111324257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c0047c0 0x7fc45c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:25.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.170+0000 7fc46f59e640 1 -- 172.21.15.84:0/4111324257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c003520 con 0x7fc45c005380 2023-10-16T10:01:25.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.170+0000 7fc467fff640 1 --2- 172.21.15.84:0/4111324257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c0047c0 0x7fc45c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:25.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.170+0000 7fc46ed9d640 1 --2- 172.21.15.84:0/4111324257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc45c003970 0x7fc45c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:25.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.171+0000 7fc46f59e640 1 -- 172.21.15.84:0/4111324257 reap_dead start 2023-10-16T10:01:25.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.171+0000 7fc46f59e640 1 --2- 172.21.15.84:0/4111324257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc45c005380 0x7fc45c00b980 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc468007330 tx=0x7fc45c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c1f82d0a137e4d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:25.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.172+0000 7fc46e59c640 1 -- 172.21.15.84:0/4111324257 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46802ccd0 con 0x7fc45c005380 2023-10-16T10:01:25.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.172+0000 7fc46e59c640 1 -- 172.21.15.84:0/4111324257 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46802ce30 con 0x7fc45c005380 2023-10-16T10:01:25.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.172+0000 7fc46e59c640 1 -- 172.21.15.84:0/4111324257 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468034680 con 0x7fc45c005380 2023-10-16T10:01:25.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.172+0000 7fc47515a640 1 -- 172.21.15.84:0/4111324257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc45c005380 msgr2=0x7fc45c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:25.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.172+0000 7fc47515a640 1 --2- 172.21.15.84:0/4111324257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc45c005380 0x7fc45c00b980 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc468007330 tx=0x7fc45c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:01:25.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.172+0000 7fc47515a640 1 -- 172.21.15.84:0/4111324257 shutdown_connections 2023-10-16T10:01:25.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.172+0000 7fc47515a640 1 --2- 172.21.15.84:0/4111324257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc45c005380 0x7fc45c00b980 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:25.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.172+0000 7fc47515a640 1 -- 172.21.15.84:0/4111324257 >> 172.21.15.84:0/4111324257 conn(0x7fc4700ce9b0 msgr2=0x7fc4700cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:25.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.172+0000 7fc47515a640 1 -- 172.21.15.84:0/4111324257 shutdown_connections 2023-10-16T10:01:25.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.172+0000 7fc47515a640 1 -- 172.21.15.84:0/4111324257 wait complete. 2023-10-16T10:01:25.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.174+0000 7fc47515a640 1 Processor -- start 2023-10-16T10:01:25.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.174+0000 7fc47515a640 1 -- start start 2023-10-16T10:01:25.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.175+0000 7fc47515a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c005380 0x7fc4700acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:25.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.175+0000 7fc47515a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc470158b60 0x7fc470155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:25.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.175+0000 7fc47515a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc470158fe0 0x7fc470155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:25.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.175+0000 7fc47515a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc470156800 con 0x7fc470158b60 2023-10-16T10:01:25.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.175+0000 7fc47515a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4700c3350 con 0x7fc45c005380 2023-10-16T10:01:25.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.175+0000 7fc47515a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47015c720 con 0x7fc470158fe0 2023-10-16T10:01:25.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.175+0000 7fc467fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc470158b60 0x7fc470155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:25.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.175+0000 7fc467fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc470158b60 0x7fc470155890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55206/0 (socket says 172.21.15.84:55206) 2023-10-16T10:01:25.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.175+0000 7fc467fff640 1 -- 172.21.15.84:0/3847166976 learned_addr learned my addr 172.21.15.84:0/3847166976 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:25.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.176+0000 7fc467fff640 1 -- 172.21.15.84:0/3847166976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc470158fe0 msgr2=0x7fc470155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:25.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.176+0000 7fc467fff640 1 --2- 172.21.15.84:0/3847166976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc470158fe0 0x7fc470155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:25.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.176+0000 7fc467fff640 1 -- 172.21.15.84:0/3847166976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c005380 msgr2=0x7fc4700acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:25.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.176+0000 7fc467fff640 1 --2- 172.21.15.84:0/3847166976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c005380 0x7fc4700acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:25.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.176+0000 7fc467fff640 1 -- 172.21.15.84:0/3847166976 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47015ca00 con 0x7fc470158b60 2023-10-16T10:01:25.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.176+0000 7fc467fff640 1 --2- 172.21.15.84:0/3847166976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc470158b60 0x7fc470155890 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc45800d770 tx=0x7fc45800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:25.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.176+0000 7fc4677fe640 1 -- 172.21.15.84:0/3847166976 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc458005270 con 0x7fc470158b60 2023-10-16T10:01:25.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.176+0000 7fc4677fe640 1 -- 172.21.15.84:0/3847166976 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4580053d0 con 0x7fc470158b60 2023-10-16T10:01:25.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.176+0000 7fc4677fe640 1 -- 172.21.15.84:0/3847166976 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45800b8f0 con 0x7fc470158b60 2023-10-16T10:01:25.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.176+0000 7fc47515a640 1 -- 172.21.15.84:0/3847166976 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47015cce0 con 0x7fc470158b60 2023-10-16T10:01:25.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.177+0000 7fc47515a640 1 -- 172.21.15.84:0/3847166976 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4700ad280 con 0x7fc470158b60 2023-10-16T10:01:25.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.178+0000 7fc47515a640 1 -- 172.21.15.84:0/3847166976 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47004e160 con 0x7fc470158b60 2023-10-16T10:01:25.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.179+0000 7fc4677fe640 1 -- 172.21.15.84:0/3847166976 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fc45800ba50 con 0x7fc470158b60 2023-10-16T10:01:25.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.180+0000 7fc4677fe640 1 --2- 172.21.15.84:0/3847166976 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fc45006b070 0x7fc45006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:25.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.180+0000 7fc4677fe640 1 -- 172.21.15.84:0/3847166976 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc458089b50 con 0x7fc470158b60 2023-10-16T10:01:25.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.180+0000 7fc46ed9d640 1 --2- 172.21.15.84:0/3847166976 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fc45006b070 0x7fc45006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:25.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.181+0000 7fc46ed9d640 1 --2- 172.21.15.84:0/3847166976 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fc45006b070 0x7fc45006d530 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc460001eb0 tx=0x7fc460009040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:25.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.184+0000 7fc4677fe640 1 -- 172.21.15.84:0/3847166976 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc458054760 con 0x7fc470158b60 2023-10-16T10:01:25.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.400+0000 7fc47515a640 1 -- 172.21.15.84:0/3847166976 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4700cda90 con 0x7fc45006b070 2023-10-16T10:01:25.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.607+0000 7fc4677fe640 1 -- 172.21.15.84:0/3847166976 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc460001570 con 0x7fc45006b070 2023-10-16T10:01:25.609 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-10-16T10:01:25.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.610+0000 7fc47515a640 1 -- 172.21.15.84:0/3847166976 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fc45006b070 msgr2=0x7fc45006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:25.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.610+0000 7fc47515a640 1 --2- 172.21.15.84:0/3847166976 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fc45006b070 0x7fc45006d530 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc460001eb0 tx=0x7fc460009040 comp rx=0 tx=0).stop 2023-10-16T10:01:25.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.610+0000 7fc47515a640 1 -- 172.21.15.84:0/3847166976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc470158b60 msgr2=0x7fc470155890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:25.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.610+0000 7fc47515a640 1 --2- 172.21.15.84:0/3847166976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc470158b60 0x7fc470155890 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc45800d770 tx=0x7fc45800dc30 comp rx=0 tx=0).stop 2023-10-16T10:01:25.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.610+0000 7fc47515a640 1 -- 172.21.15.84:0/3847166976 shutdown_connections 2023-10-16T10:01:25.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.610+0000 7fc47515a640 1 --2- 172.21.15.84:0/3847166976 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fc45006b070 0x7fc45006d530 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:25.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.610+0000 7fc47515a640 1 --2- 172.21.15.84:0/3847166976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc470158fe0 0x7fc470155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:25.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.610+0000 7fc47515a640 1 --2- 172.21.15.84:0/3847166976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc470158b60 0x7fc470155890 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:25.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.610+0000 7fc47515a640 1 --2- 172.21.15.84:0/3847166976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c005380 0x7fc4700acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:25.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.610+0000 7fc47515a640 1 -- 172.21.15.84:0/3847166976 >> 172.21.15.84:0/3847166976 conn(0x7fc4700ce9b0 msgr2=0x7fc470156f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:25.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.610+0000 7fc47515a640 1 -- 172.21.15.84:0/3847166976 shutdown_connections 2023-10-16T10:01:25.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.610+0000 7fc47515a640 1 -- 172.21.15.84:0/3847166976 wait complete. 2023-10-16T10:01:25.630 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-10-16T10:01:25.631 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir6 1m 2023-10-16T10:01:25.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.843+0000 7ff70a901640 1 Processor -- start 2023-10-16T10:01:25.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.843+0000 7ff70a901640 1 -- start start 2023-10-16T10:01:25.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.845+0000 7ff70a901640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff70406a4b0 0x7ff7040ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:25.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.845+0000 7ff70a901640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7040cb030 con 0x7ff7040c9ca0 2023-10-16T10:01:25.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.845+0000 7ff70a901640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7040c91c0 con 0x7ff70415ad70 2023-10-16T10:01:25.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.845+0000 7ff70a901640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7040c94a0 con 0x7ff70406a4b0 2023-10-16T10:01:25.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.845+0000 7ff708e9e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff7040c9ca0 0x7ff7040ad4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39856/0 (socket says 172.21.15.84:39856) 2023-10-16T10:01:25.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.845+0000 7ff708e9e640 1 -- 172.21.15.84:0/3918507702 learned_addr learned my addr 172.21.15.84:0/3918507702 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:25.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.845+0000 7ff7037fe640 1 --2- 172.21.15.84:0/3918507702 >> v2:172.21.15.84:3300/0 conn(0x7ff70406a4b0 0x7ff7040ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:25.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.846+0000 7ff702ffd640 1 -- 172.21.15.84:0/3918507702 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092074443 0 0) 0x7ff7040cb030 con 0x7ff7040c9ca0 2023-10-16T10:01:25.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.846+0000 7ff702ffd640 1 -- 172.21.15.84:0/3918507702 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6ec003520 con 0x7ff7040c9ca0 2023-10-16T10:01:25.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.846+0000 7ff7037fe640 1 -- 172.21.15.84:0/3918507702 >> v1:172.21.15.157:6789/0 conn(0x7ff70415ad70 legacy=0x7ff704069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:25.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.846+0000 7ff7037fe640 1 -- 172.21.15.84:0/3918507702 >> v1:172.21.15.84:6789/0 conn(0x7ff7040c9ca0 legacy=0x7ff7040ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:25.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.846+0000 7ff7037fe640 1 -- 172.21.15.84:0/3918507702 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7040c9780 con 0x7ff70406a4b0 2023-10-16T10:01:25.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.847+0000 7ff7037fe640 1 --2- 172.21.15.84:0/3918507702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff70406a4b0 0x7ff7040ca3b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80097a0 tx=0x7ff6f802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73369d13c02c6235 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:25.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.847+0000 7ff702ffd640 1 -- 172.21.15.84:0/3918507702 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8030610 con 0x7ff70406a4b0 2023-10-16T10:01:25.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.847+0000 7ff702ffd640 1 -- 172.21.15.84:0/3918507702 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f8030770 con 0x7ff70406a4b0 2023-10-16T10:01:25.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.847+0000 7ff702ffd640 1 -- 172.21.15.84:0/3918507702 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8030b00 con 0x7ff70406a4b0 2023-10-16T10:01:25.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.848+0000 7ff70a901640 1 -- 172.21.15.84:0/3918507702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff70406a4b0 msgr2=0x7ff7040ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:25.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.848+0000 7ff70a901640 1 --2- 172.21.15.84:0/3918507702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff70406a4b0 0x7ff7040ca3b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80097a0 tx=0x7ff6f802b960 comp rx=0 tx=0).stop 2023-10-16T10:01:25.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.848+0000 7ff70a901640 1 -- 172.21.15.84:0/3918507702 shutdown_connections 2023-10-16T10:01:25.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.848+0000 7ff70a901640 1 --2- 172.21.15.84:0/3918507702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff70406a4b0 0x7ff7040ca3b0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:25.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.848+0000 7ff70a901640 1 -- 172.21.15.84:0/3918507702 >> 172.21.15.84:0/3918507702 conn(0x7ff7040ce9b0 msgr2=0x7ff7040cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:25.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.848+0000 7ff70a901640 1 -- 172.21.15.84:0/3918507702 shutdown_connections 2023-10-16T10:01:25.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.848+0000 7ff70a901640 1 -- 172.21.15.84:0/3918507702 wait complete. 2023-10-16T10:01:25.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.850+0000 7ff70a901640 1 Processor -- start 2023-10-16T10:01:25.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.850+0000 7ff70a901640 1 -- start start 2023-10-16T10:01:25.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.851+0000 7ff70a901640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70406a4b0 0x7ff7040c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:25.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.851+0000 7ff70a901640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7040c9ca0 0x7ff7040b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:25.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.851+0000 7ff70a901640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff70415ad70 0x7ff7040b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:25.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.851+0000 7ff70a901640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70414c0b0 con 0x7ff7040c9ca0 2023-10-16T10:01:25.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.851+0000 7ff70a901640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70414b1c0 con 0x7ff70415ad70 2023-10-16T10:01:25.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.851+0000 7ff70a901640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7040c54c0 con 0x7ff70406a4b0 2023-10-16T10:01:25.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.851+0000 7ff7037fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7040c9ca0 0x7ff7040b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:25.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.851+0000 7ff7037fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7040c9ca0 0x7ff7040b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56938/0 (socket says 172.21.15.84:56938) 2023-10-16T10:01:25.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.851+0000 7ff7037fe640 1 -- 172.21.15.84:0/3540281072 learned_addr learned my addr 172.21.15.84:0/3540281072 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:25.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.851+0000 7ff708e9e640 1 --2- 172.21.15.84:0/3540281072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff70415ad70 0x7ff7040b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.852+0000 7ff7037fe640 1 -- 172.21.15.84:0/3540281072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70406a4b0 msgr2=0x7ff7040c4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.852+0000 7ff7037fe640 1 --2- 172.21.15.84:0/3540281072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70406a4b0 0x7ff7040c4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.852+0000 7ff7037fe640 1 -- 172.21.15.84:0/3540281072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff70415ad70 msgr2=0x7ff7040b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.852+0000 7ff7037fe640 1 --2- 172.21.15.84:0/3540281072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff70415ad70 0x7ff7040b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.852+0000 7ff7037fe640 1 -- 172.21.15.84:0/3540281072 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f8009400 con 0x7ff7040c9ca0 2023-10-16T10:01:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.852+0000 7ff7037fe640 1 --2- 172.21.15.84:0/3540281072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7040c9ca0 0x7ff7040b2ef0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8039700 tx=0x7ff6f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.852+0000 7ff700ff9640 1 -- 172.21.15.84:0/3540281072 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f803e040 con 0x7ff7040c9ca0 2023-10-16T10:01:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.852+0000 7ff700ff9640 1 -- 172.21.15.84:0/3540281072 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f8039d10 con 0x7ff7040c9ca0 2023-10-16T10:01:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.852+0000 7ff700ff9640 1 -- 172.21.15.84:0/3540281072 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8043680 con 0x7ff7040c9ca0 2023-10-16T10:01:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.852+0000 7ff70a901640 1 -- 172.21.15.84:0/3540281072 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70414b430 con 0x7ff7040c9ca0 2023-10-16T10:01:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.852+0000 7ff70a901640 1 -- 172.21.15.84:0/3540281072 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70414b9f0 con 0x7ff7040c9ca0 2023-10-16T10:01:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.854+0000 7ff70a901640 1 -- 172.21.15.84:0/3540281072 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff70404e160 con 0x7ff7040c9ca0 2023-10-16T10:01:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.855+0000 7ff700ff9640 1 -- 172.21.15.84:0/3540281072 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7ff6f80423e0 con 0x7ff7040c9ca0 2023-10-16T10:01:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.856+0000 7ff700ff9640 1 --2- 172.21.15.84:0/3540281072 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff6f006b070 0x7ff6f006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.856+0000 7ff700ff9640 1 -- 172.21.15.84:0/3540281072 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff6f80abb30 con 0x7ff7040c9ca0 2023-10-16T10:01:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.856+0000 7ff703fff640 1 --2- 172.21.15.84:0/3540281072 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff6f006b070 0x7ff6f006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.857+0000 7ff703fff640 1 --2- 172.21.15.84:0/3540281072 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff6f006b070 0x7ff6f006d530 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4000f60 tx=0x7ff6f40079b0 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:25.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:25.861+0000 7ff700ff9640 1 -- 172.21.15.84:0/3540281072 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff6f8076750 con 0x7ff7040c9ca0 2023-10-16T10:01:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.077+0000 7ff70a901640 1 -- 172.21.15.84:0/3540281072 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7040ca8f0 con 0x7ff6f006b070 2023-10-16T10:01:26.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.278+0000 7ff700ff9640 1 -- 172.21.15.84:0/3540281072 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff6f4001210 con 0x7ff6f006b070 2023-10-16T10:01:26.280 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-10-16T10:01:26.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.281+0000 7ff70a901640 1 -- 172.21.15.84:0/3540281072 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff6f006b070 msgr2=0x7ff6f006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:26.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.281+0000 7ff70a901640 1 --2- 172.21.15.84:0/3540281072 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff6f006b070 0x7ff6f006d530 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4000f60 tx=0x7ff6f40079b0 comp rx=0 tx=0).stop 2023-10-16T10:01:26.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.281+0000 7ff70a901640 1 -- 172.21.15.84:0/3540281072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7040c9ca0 msgr2=0x7ff7040b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:26.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.281+0000 7ff70a901640 1 --2- 172.21.15.84:0/3540281072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7040c9ca0 0x7ff7040b2ef0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8039700 tx=0x7ff6f8004730 comp rx=0 tx=0).stop 2023-10-16T10:01:26.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.281+0000 7ff70a901640 1 -- 172.21.15.84:0/3540281072 shutdown_connections 2023-10-16T10:01:26.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.281+0000 7ff70a901640 1 --2- 172.21.15.84:0/3540281072 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff6f006b070 0x7ff6f006d530 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:26.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.281+0000 7ff70a901640 1 --2- 172.21.15.84:0/3540281072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff70415ad70 0x7ff7040b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:26.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.281+0000 7ff70a901640 1 --2- 172.21.15.84:0/3540281072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7040c9ca0 0x7ff7040b2ef0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:26.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.281+0000 7ff70a901640 1 --2- 172.21.15.84:0/3540281072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70406a4b0 0x7ff7040c4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:26.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.281+0000 7ff70a901640 1 -- 172.21.15.84:0/3540281072 >> 172.21.15.84:0/3540281072 conn(0x7ff7040ce9b0 msgr2=0x7ff704069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:26.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.282+0000 7ff70a901640 1 -- 172.21.15.84:0/3540281072 shutdown_connections 2023-10-16T10:01:26.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.282+0000 7ff70a901640 1 -- 172.21.15.84:0/3540281072 wait complete. 2023-10-16T10:01:26.303 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-10-16T10:01:26.304 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir7 1m 2023-10-16T10:01:26.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.518+0000 7f620e415640 1 Processor -- start 2023-10-16T10:01:26.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.518+0000 7f620e415640 1 -- start start 2023-10-16T10:01:26.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.520+0000 7f620e415640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f620806a4b0 0x7f62080c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:26.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.520+0000 7f620e415640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62080ac890 con 0x7f62080ad350 2023-10-16T10:01:26.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.520+0000 7f620e415640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62080ad7a0 con 0x7f620815d150 2023-10-16T10:01:26.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.520+0000 7f620e415640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620815d530 con 0x7f620806a4b0 2023-10-16T10:01:26.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.520+0000 7f62077fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f62080ad350 0x7f6208069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39860/0 (socket says 172.21.15.84:39860) 2023-10-16T10:01:26.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.520+0000 7f62077fe640 1 -- 172.21.15.84:0/3661463722 learned_addr learned my addr 172.21.15.84:0/3661463722 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:26.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.520+0000 7f620c9b2640 1 --2- 172.21.15.84:0/3661463722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f620806a4b0 0x7f62080c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:26.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.520+0000 7f6206ffd640 1 -- 172.21.15.84:0/3661463722 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960661508 0 0) 0x7f62080ac890 con 0x7f62080ad350 2023-10-16T10:01:26.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.520+0000 7f6206ffd640 1 -- 172.21.15.84:0/3661463722 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61f0003520 con 0x7f62080ad350 2023-10-16T10:01:26.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.521+0000 7f6206ffd640 1 -- 172.21.15.84:0/3661463722 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2309697253 0 0) 0x7f61f0003520 con 0x7f62080ad350 2023-10-16T10:01:26.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.521+0000 7f6206ffd640 1 -- 172.21.15.84:0/3661463722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f620806a4b0 msgr2=0x7f62080c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:26.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.521+0000 7f6206ffd640 1 --2- 172.21.15.84:0/3661463722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f620806a4b0 0x7f62080c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:26.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.521+0000 7f6206ffd640 1 -- 172.21.15.84:0/3661463722 >> v1:172.21.15.157:6789/0 conn(0x7f620815d150 legacy=0x7f6208160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:26.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.521+0000 7f6206ffd640 1 -- 172.21.15.84:0/3661463722 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6208161650 con 0x7f62080ad350 2023-10-16T10:01:26.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.521+0000 7f6206ffd640 1 -- 172.21.15.84:0/3661463722 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f61fc001670 con 0x7f62080ad350 2023-10-16T10:01:26.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.521+0000 7f6206ffd640 1 -- 172.21.15.84:0/3661463722 >> v1:172.21.15.84:6789/0 conn(0x7f62080ad350 legacy=0x7f6208069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:26.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.521+0000 7f6206ffd640 1 --2- 172.21.15.84:0/3661463722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f620806a4b0 0x7f61f00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:26.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.521+0000 7f6206ffd640 1 --2- 172.21.15.84:0/3661463722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61f0003f20 0x7f61f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:26.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.521+0000 7f6206ffd640 1 --2- 172.21.15.84:0/3661463722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61f0004d00 0x7f61f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:26.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.521+0000 7f6206ffd640 1 -- 172.21.15.84:0/3661463722 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620815d530 con 0x7f61f0003f20 2023-10-16T10:01:26.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.521+0000 7f6206ffd640 1 -- 172.21.15.84:0/3661463722 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f000ba60 con 0x7f61f0004d00 2023-10-16T10:01:26.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.521+0000 7f6206ffd640 1 -- 172.21.15.84:0/3661463722 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f000bc20 con 0x7f620806a4b0 2023-10-16T10:01:26.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.521+0000 7f62077fe640 1 --2- 172.21.15.84:0/3661463722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61f0003f20 0x7f61f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:26.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.522+0000 7f62077fe640 1 -- 172.21.15.84:0/3661463722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f620806a4b0 msgr2=0x7f61f00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:26.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.522+0000 7f62077fe640 1 --2- 172.21.15.84:0/3661463722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f620806a4b0 0x7f61f00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:26.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.522+0000 7f62077fe640 1 -- 172.21.15.84:0/3661463722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61f0004d00 msgr2=0x7f61f0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:26.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.522+0000 7f62077fe640 1 --2- 172.21.15.84:0/3661463722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61f0004d00 0x7f61f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:26.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.522+0000 7f62077fe640 1 -- 172.21.15.84:0/3661463722 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f0003520 con 0x7f61f0003f20 2023-10-16T10:01:26.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.523+0000 7f62077fe640 1 --2- 172.21.15.84:0/3661463722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61f0003f20 0x7f61f00047c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00c240 tx=0x7f61f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2271578d74911c62 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:26.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.523+0000 7f6206ffd640 1 -- 172.21.15.84:0/3661463722 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc005ad0 con 0x7f61f0003f20 2023-10-16T10:01:26.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.523+0000 7f6206ffd640 1 -- 172.21.15.84:0/3661463722 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61fc005c30 con 0x7f61f0003f20 2023-10-16T10:01:26.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.523+0000 7f6206ffd640 1 -- 172.21.15.84:0/3661463722 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc02caf0 con 0x7f61f0003f20 2023-10-16T10:01:26.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.523+0000 7f620e415640 1 -- 172.21.15.84:0/3661463722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61f0003f20 msgr2=0x7f61f00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:26.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.523+0000 7f620e415640 1 --2- 172.21.15.84:0/3661463722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61f0003f20 0x7f61f00047c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00c240 tx=0x7f61f00057f0 comp rx=0 tx=0).stop 2023-10-16T10:01:26.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.523+0000 7f620c9b2640 1 -- 172.21.15.84:0/3661463722 reap_dead start 2023-10-16T10:01:26.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.523+0000 7f620e415640 1 -- 172.21.15.84:0/3661463722 shutdown_connections 2023-10-16T10:01:26.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.523+0000 7f620e415640 1 -- 172.21.15.84:0/3661463722 >> 172.21.15.84:0/3661463722 conn(0x7f62080ce9b0 msgr2=0x7f62080cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:26.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.524+0000 7f620e415640 1 -- 172.21.15.84:0/3661463722 shutdown_connections 2023-10-16T10:01:26.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.524+0000 7f620e415640 1 -- 172.21.15.84:0/3661463722 wait complete. 2023-10-16T10:01:26.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.525+0000 7f620e415640 1 Processor -- start 2023-10-16T10:01:26.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.525+0000 7f620e415640 1 -- start start 2023-10-16T10:01:26.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.526+0000 7f620e415640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61f0003f20 0x7f62080c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:26.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.526+0000 7f620e415640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62081f2480 0x7f62080c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:26.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.526+0000 7f620e415640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62080c6c10 0x7f62080b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:26.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.526+0000 7f620e415640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62080c7d90 con 0x7f62080c6c10 2023-10-16T10:01:26.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.526+0000 7f620e415640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62081f35d0 con 0x7f61f0003f20 2023-10-16T10:01:26.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.526+0000 7f620e415640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62081f2970 con 0x7f62081f2480 2023-10-16T10:01:26.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.527+0000 7f620c9b2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62080c6c10 0x7f62080b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:26.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.527+0000 7f620c9b2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62080c6c10 0x7f62080b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56950/0 (socket says 172.21.15.84:56950) 2023-10-16T10:01:26.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.527+0000 7f620c9b2640 1 -- 172.21.15.84:0/215649469 learned_addr learned my addr 172.21.15.84:0/215649469 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:26.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.527+0000 7f620c9b2640 1 -- 172.21.15.84:0/215649469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62081f2480 msgr2=0x7f62080c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:26.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.527+0000 7f620c9b2640 1 --2- 172.21.15.84:0/215649469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62081f2480 0x7f62080c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:26.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.527+0000 7f620c9b2640 1 -- 172.21.15.84:0/215649469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61f0003f20 msgr2=0x7f62080c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:26.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.527+0000 7f620c9b2640 1 --2- 172.21.15.84:0/215649469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61f0003f20 0x7f62080c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:26.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.527+0000 7f620c9b2640 1 -- 172.21.15.84:0/215649469 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62081f2c50 con 0x7f62080c6c10 2023-10-16T10:01:26.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.527+0000 7f620c9b2640 1 --2- 172.21.15.84:0/215649469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62080c6c10 0x7f62080b3000 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f62080c8f90 tx=0x7f620000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:26.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.528+0000 7f6204ff9640 1 -- 172.21.15.84:0/215649469 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6200010040 con 0x7f62080c6c10 2023-10-16T10:01:26.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.528+0000 7f6204ff9640 1 -- 172.21.15.84:0/215649469 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620000bb10 con 0x7f62080c6c10 2023-10-16T10:01:26.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.528+0000 7f620e415640 1 -- 172.21.15.84:0/215649469 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62081f2f30 con 0x7f62080c6c10 2023-10-16T10:01:26.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.528+0000 7f6204ff9640 1 -- 172.21.15.84:0/215649469 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6200014400 con 0x7f62080c6c10 2023-10-16T10:01:26.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.528+0000 7f620e415640 1 -- 172.21.15.84:0/215649469 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62080b3540 con 0x7f62080c6c10 2023-10-16T10:01:26.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.529+0000 7f620e415640 1 -- 172.21.15.84:0/215649469 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620804e160 con 0x7f62080c6c10 2023-10-16T10:01:26.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.530+0000 7f6204ff9640 1 -- 172.21.15.84:0/215649469 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f6200014560 con 0x7f62080c6c10 2023-10-16T10:01:26.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.531+0000 7f6204ff9640 1 --2- 172.21.15.84:0/215649469 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f61f406b070 0x7f61f406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:26.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.531+0000 7f6204ff9640 1 -- 172.21.15.84:0/215649469 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f6200089f60 con 0x7f62080c6c10 2023-10-16T10:01:26.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.531+0000 7f6207fff640 1 --2- 172.21.15.84:0/215649469 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f61f406b070 0x7f61f406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:26.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.532+0000 7f6207fff640 1 --2- 172.21.15.84:0/215649469 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f61f406b070 0x7f61f406d530 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f61f8001590 tx=0x7f61f8007990 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:26.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.535+0000 7f6204ff9640 1 -- 172.21.15.84:0/215649469 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6200054c10 con 0x7f62080c6c10 2023-10-16T10:01:26.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.750+0000 7f620e415640 1 -- 172.21.15.84:0/215649469 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f62080acb50 con 0x7f61f406b070 2023-10-16T10:01:26.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.944+0000 7f6204ff9640 1 -- 172.21.15.84:0/215649469 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f61f8005080 con 0x7f61f406b070 2023-10-16T10:01:26.946 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-10-16T10:01:26.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.947+0000 7f620e415640 1 -- 172.21.15.84:0/215649469 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f61f406b070 msgr2=0x7f61f406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:26.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.947+0000 7f620e415640 1 --2- 172.21.15.84:0/215649469 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f61f406b070 0x7f61f406d530 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f61f8001590 tx=0x7f61f8007990 comp rx=0 tx=0).stop 2023-10-16T10:01:26.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.947+0000 7f620e415640 1 -- 172.21.15.84:0/215649469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62080c6c10 msgr2=0x7f62080b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:26.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.947+0000 7f620e415640 1 --2- 172.21.15.84:0/215649469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62080c6c10 0x7f62080b3000 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f62080c8f90 tx=0x7f620000b480 comp rx=0 tx=0).stop 2023-10-16T10:01:26.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.947+0000 7f620e415640 1 -- 172.21.15.84:0/215649469 shutdown_connections 2023-10-16T10:01:26.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.947+0000 7f620e415640 1 --2- 172.21.15.84:0/215649469 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f61f406b070 0x7f61f406d530 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:26.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.947+0000 7f620e415640 1 --2- 172.21.15.84:0/215649469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62080c6c10 0x7f62080b3000 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:26.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.947+0000 7f620e415640 1 --2- 172.21.15.84:0/215649469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62081f2480 0x7f62080c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:26.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.947+0000 7f620e415640 1 --2- 172.21.15.84:0/215649469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61f0003f20 0x7f62080c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:26.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.947+0000 7f620e415640 1 -- 172.21.15.84:0/215649469 >> 172.21.15.84:0/215649469 conn(0x7f62080ce9b0 msgr2=0x7f620815b700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:26.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.948+0000 7f620e415640 1 -- 172.21.15.84:0/215649469 shutdown_connections 2023-10-16T10:01:26.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:26.948+0000 7f620e415640 1 -- 172.21.15.84:0/215649469 wait complete. 2023-10-16T10:01:26.970 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-10-16T10:01:26.971 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir8 1m 2023-10-16T10:01:27.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.186+0000 7f67a4d9a640 1 Processor -- start 2023-10-16T10:01:27.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.186+0000 7f67a4d9a640 1 -- start start 2023-10-16T10:01:27.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.187+0000 7f67a4d9a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f67a0159950 0x7f67a015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:27.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.187+0000 7f67a4d9a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f67a006c810 0x7f67a00708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:27.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.187+0000 7f67a4d9a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a015aef0 con 0x7f67a0070450 2023-10-16T10:01:27.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.187+0000 7f67a4d9a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a0070e20 con 0x7f67a006c810 2023-10-16T10:01:27.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.187+0000 7f67a4d9a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a015e490 con 0x7f67a0159950 2023-10-16T10:01:27.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.188+0000 7f679ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f67a006c810 0x7f67a00708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:27.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.188+0000 7f679ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f67a006c810 0x7f67a00708e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56954/0 (socket says 172.21.15.84:56954) 2023-10-16T10:01:27.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.188+0000 7f679ffff640 1 -- 172.21.15.84:0/413669020 learned_addr learned my addr 172.21.15.84:0/413669020 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:27.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.188+0000 7f679f7fe640 1 --2- 172.21.15.84:0/413669020 >> v2:172.21.15.157:3300/0 conn(0x7f67a0159950 0x7f67a015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:27.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.188+0000 7f679ffff640 1 -- 172.21.15.84:0/413669020 >> v2:172.21.15.157:3300/0 conn(0x7f67a0159950 msgr2=0x7f67a015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:27.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.188+0000 7f679ffff640 1 --2- 172.21.15.84:0/413669020 >> v2:172.21.15.157:3300/0 conn(0x7f67a0159950 0x7f67a015a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:27.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.188+0000 7f679ffff640 1 -- 172.21.15.84:0/413669020 >> v1:172.21.15.157:6789/0 conn(0x7f67a0070450 legacy=0x7f67a006c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:27.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.188+0000 7f679ffff640 1 -- 172.21.15.84:0/413669020 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a015d910 con 0x7f67a006c810 2023-10-16T10:01:27.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.189+0000 7f679ffff640 1 --2- 172.21.15.84:0/413669020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67a006c810 0x7f67a00708e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f67a00ca950 tx=0x7f679802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64670453bd3d358 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:27.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.190+0000 7f679effd640 1 -- 172.21.15.84:0/413669020 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679802c9f0 con 0x7f67a006c810 2023-10-16T10:01:27.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.190+0000 7f679effd640 1 -- 172.21.15.84:0/413669020 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679802cb50 con 0x7f67a006c810 2023-10-16T10:01:27.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.190+0000 7f679effd640 1 -- 172.21.15.84:0/413669020 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67980343f0 con 0x7f67a006c810 2023-10-16T10:01:27.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.190+0000 7f67a4d9a640 1 -- 172.21.15.84:0/413669020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67a006c810 msgr2=0x7f67a00708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:27.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.190+0000 7f67a4d9a640 1 --2- 172.21.15.84:0/413669020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67a006c810 0x7f67a00708e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f67a00ca950 tx=0x7f679802be50 comp rx=0 tx=0).stop 2023-10-16T10:01:27.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.190+0000 7f67a4d9a640 1 -- 172.21.15.84:0/413669020 shutdown_connections 2023-10-16T10:01:27.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.190+0000 7f67a4d9a640 1 --2- 172.21.15.84:0/413669020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67a006c810 0x7f67a00708e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:27.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.190+0000 7f67a4d9a640 1 --2- 172.21.15.84:0/413669020 >> v2:172.21.15.157:3300/0 conn(0x7f67a0159950 0x7f67a015a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:27.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.190+0000 7f67a4d9a640 1 -- 172.21.15.84:0/413669020 >> 172.21.15.84:0/413669020 conn(0x7f67a00ce6a0 msgr2=0x7f67a00ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:27.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.190+0000 7f67a4d9a640 1 -- 172.21.15.84:0/413669020 shutdown_connections 2023-10-16T10:01:27.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.190+0000 7f67a4d9a640 1 -- 172.21.15.84:0/413669020 wait complete. 2023-10-16T10:01:27.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.192+0000 7f67a4d9a640 1 Processor -- start 2023-10-16T10:01:27.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.192+0000 7f67a4d9a640 1 -- start start 2023-10-16T10:01:27.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.193+0000 7f67a4d9a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67a006c810 0x7f67a01504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:27.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.193+0000 7f67a4d9a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67a0070450 0x7f67a015b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:27.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.193+0000 7f67a4d9a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a0159950 0x7f67a00ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:27.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.193+0000 7f67a4d9a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a00c6fc0 con 0x7f67a0070450 2023-10-16T10:01:27.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.193+0000 7f67a4d9a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a015b5d0 con 0x7f67a0159950 2023-10-16T10:01:27.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.193+0000 7f67a4d9a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a015b870 con 0x7f67a006c810 2023-10-16T10:01:27.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.193+0000 7f6797fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67a0070450 0x7f67a015b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:27.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.193+0000 7f6797fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67a0070450 0x7f67a015b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56964/0 (socket says 172.21.15.84:56964) 2023-10-16T10:01:27.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.193+0000 7f6797fff640 1 -- 172.21.15.84:0/286390196 learned_addr learned my addr 172.21.15.84:0/286390196 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:27.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.194+0000 7f679ffff640 1 --2- 172.21.15.84:0/286390196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a0159950 0x7f67a00ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:27.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.194+0000 7f6797fff640 1 -- 172.21.15.84:0/286390196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67a006c810 msgr2=0x7f67a01504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:27.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.194+0000 7f6797fff640 1 --2- 172.21.15.84:0/286390196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67a006c810 0x7f67a01504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:27.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.194+0000 7f6797fff640 1 -- 172.21.15.84:0/286390196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a0159950 msgr2=0x7f67a00ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:27.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.194+0000 7f6797fff640 1 --2- 172.21.15.84:0/286390196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a0159950 0x7f67a00ae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:27.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.194+0000 7f6797fff640 1 -- 172.21.15.84:0/286390196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6798009b30 con 0x7f67a0070450 2023-10-16T10:01:27.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.194+0000 7f679ffff640 1 --2- 172.21.15.84:0/286390196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a0159950 0x7f67a00ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:27.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.194+0000 7f6797fff640 1 --2- 172.21.15.84:0/286390196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67a0070450 0x7f67a015b090 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f678800c430 tx=0x7f678800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:27.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.194+0000 7f679cff9640 1 -- 172.21.15.84:0/286390196 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6788011790 con 0x7f67a0070450 2023-10-16T10:01:27.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.195+0000 7f679cff9640 1 -- 172.21.15.84:0/286390196 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67880118f0 con 0x7f67a0070450 2023-10-16T10:01:27.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.195+0000 7f679cff9640 1 -- 172.21.15.84:0/286390196 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6788011ce0 con 0x7f67a0070450 2023-10-16T10:01:27.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.195+0000 7f67a4d9a640 1 -- 172.21.15.84:0/286390196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a015bb50 con 0x7f67a0070450 2023-10-16T10:01:27.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.195+0000 7f67a4d9a640 1 -- 172.21.15.84:0/286390196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a015c110 con 0x7f67a0070450 2023-10-16T10:01:27.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.196+0000 7f67a4d9a640 1 -- 172.21.15.84:0/286390196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a00c7190 con 0x7f67a0070450 2023-10-16T10:01:27.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.197+0000 7f679cff9640 1 -- 172.21.15.84:0/286390196 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f678801c460 con 0x7f67a0070450 2023-10-16T10:01:27.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.198+0000 7f679cff9640 1 --2- 172.21.15.84:0/286390196 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f678006b070 0x7f678006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:27.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.198+0000 7f679cff9640 1 -- 172.21.15.84:0/286390196 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f6788089a70 con 0x7f67a0070450 2023-10-16T10:01:27.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.198+0000 7f679f7fe640 1 --2- 172.21.15.84:0/286390196 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f678006b070 0x7f678006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:27.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.199+0000 7f679f7fe640 1 --2- 172.21.15.84:0/286390196 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f678006b070 0x7f678006d530 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6790006fd0 tx=0x7f6790008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:27.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.202+0000 7f679cff9640 1 -- 172.21.15.84:0/286390196 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f67a00c7190 con 0x7f67a0070450 2023-10-16T10:01:27.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.423+0000 7f67a4d9a640 1 -- 172.21.15.84:0/286390196 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f67a015dbf0 con 0x7f678006b070 2023-10-16T10:01:27.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.624+0000 7f679cff9640 1 -- 172.21.15.84:0/286390196 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f679000be20 con 0x7f678006b070 2023-10-16T10:01:27.626 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-10-16T10:01:27.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.628+0000 7f67a4d9a640 1 -- 172.21.15.84:0/286390196 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f678006b070 msgr2=0x7f678006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:27.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.628+0000 7f67a4d9a640 1 --2- 172.21.15.84:0/286390196 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f678006b070 0x7f678006d530 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6790006fd0 tx=0x7f6790008040 comp rx=0 tx=0).stop 2023-10-16T10:01:27.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.628+0000 7f67a4d9a640 1 -- 172.21.15.84:0/286390196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67a0070450 msgr2=0x7f67a015b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:27.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.628+0000 7f67a4d9a640 1 --2- 172.21.15.84:0/286390196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67a0070450 0x7f67a015b090 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f678800c430 tx=0x7f678800c8f0 comp rx=0 tx=0).stop 2023-10-16T10:01:27.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.628+0000 7f67a4d9a640 1 -- 172.21.15.84:0/286390196 shutdown_connections 2023-10-16T10:01:27.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.628+0000 7f67a4d9a640 1 --2- 172.21.15.84:0/286390196 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f678006b070 0x7f678006d530 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:27.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.628+0000 7f67a4d9a640 1 --2- 172.21.15.84:0/286390196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a0159950 0x7f67a00ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:27.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.628+0000 7f67a4d9a640 1 --2- 172.21.15.84:0/286390196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67a0070450 0x7f67a015b090 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:27.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.628+0000 7f67a4d9a640 1 --2- 172.21.15.84:0/286390196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67a006c810 0x7f67a01504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:27.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.628+0000 7f67a4d9a640 1 -- 172.21.15.84:0/286390196 >> 172.21.15.84:0/286390196 conn(0x7f67a00ce6a0 msgr2=0x7f67a00c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:27.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.628+0000 7f67a4d9a640 1 -- 172.21.15.84:0/286390196 shutdown_connections 2023-10-16T10:01:27.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.628+0000 7f67a4d9a640 1 -- 172.21.15.84:0/286390196 wait complete. 2023-10-16T10:01:27.649 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-10-16T10:01:27.649 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir9 1m 2023-10-16T10:01:27.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.866+0000 7fec1eae0640 1 Processor -- start 2023-10-16T10:01:27.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.866+0000 7fec1eae0640 1 -- start start 2023-10-16T10:01:27.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.868+0000 7fec1eae0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec1815ad70 0x7fec18069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:27.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.868+0000 7fec1eae0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec180caac0 0x7fec1806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:27.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.868+0000 7fec1eae0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec180cb030 con 0x7fec180c9ca0 2023-10-16T10:01:27.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.868+0000 7fec1eae0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec180c91c0 con 0x7fec1815ad70 2023-10-16T10:01:27.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.868+0000 7fec1eae0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec180c94a0 con 0x7fec180caac0 2023-10-16T10:01:27.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.868+0000 7fec1c87c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec1815ad70 0x7fec18069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:27.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.868+0000 7fec1c87c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec1815ad70 0x7fec18069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56966/0 (socket says 172.21.15.84:56966) 2023-10-16T10:01:27.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.868+0000 7fec1c87c640 1 -- 172.21.15.84:0/1714788596 learned_addr learned my addr 172.21.15.84:0/1714788596 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:27.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.868+0000 7fec0ffff640 1 --1- 172.21.15.84:0/1714788596 >> v1:172.21.15.84:6789/0 conn(0x7fec180c9ca0 0x7fec180ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39870/0 (socket says 172.21.15.84:39870) 2023-10-16T10:01:27.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.869+0000 7fec1c87c640 1 -- 172.21.15.84:0/1714788596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec180caac0 msgr2=0x7fec1806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:27.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.869+0000 7fec1c87c640 1 --2- 172.21.15.84:0/1714788596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec180caac0 0x7fec1806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:27.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.869+0000 7fec1c87c640 1 -- 172.21.15.84:0/1714788596 >> v1:172.21.15.84:6789/0 conn(0x7fec180c9ca0 legacy=0x7fec180ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:27.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.869+0000 7fec1c87c640 1 -- 172.21.15.84:0/1714788596 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec180c9780 con 0x7fec1815ad70 2023-10-16T10:01:27.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.870+0000 7fec1c87c640 1 --2- 172.21.15.84:0/1714788596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec1815ad70 0x7fec18069da0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fec14009e50 tx=0x7fec1402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5d92755effdcd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:27.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.870+0000 7fec0f7fe640 1 -- 172.21.15.84:0/1714788596 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1402cb00 con 0x7fec1815ad70 2023-10-16T10:01:27.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.870+0000 7fec0f7fe640 1 -- 172.21.15.84:0/1714788596 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1402cc60 con 0x7fec1815ad70 2023-10-16T10:01:27.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.870+0000 7fec0f7fe640 1 -- 172.21.15.84:0/1714788596 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec140343f0 con 0x7fec1815ad70 2023-10-16T10:01:27.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.871+0000 7fec1eae0640 1 -- 172.21.15.84:0/1714788596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec1815ad70 msgr2=0x7fec18069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:27.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.871+0000 7fec1eae0640 1 --2- 172.21.15.84:0/1714788596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec1815ad70 0x7fec18069da0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fec14009e50 tx=0x7fec1402bf60 comp rx=0 tx=0).stop 2023-10-16T10:01:27.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.871+0000 7fec1eae0640 1 -- 172.21.15.84:0/1714788596 shutdown_connections 2023-10-16T10:01:27.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.871+0000 7fec1eae0640 1 --2- 172.21.15.84:0/1714788596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec180caac0 0x7fec1806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:27.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.871+0000 7fec1eae0640 1 --2- 172.21.15.84:0/1714788596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec1815ad70 0x7fec18069da0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:27.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.871+0000 7fec1eae0640 1 -- 172.21.15.84:0/1714788596 >> 172.21.15.84:0/1714788596 conn(0x7fec180ce9b0 msgr2=0x7fec180cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:27.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.871+0000 7fec1eae0640 1 -- 172.21.15.84:0/1714788596 shutdown_connections 2023-10-16T10:01:27.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.871+0000 7fec1eae0640 1 -- 172.21.15.84:0/1714788596 wait complete. 2023-10-16T10:01:27.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.873+0000 7fec1eae0640 1 Processor -- start 2023-10-16T10:01:27.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.873+0000 7fec1eae0640 1 -- start start 2023-10-16T10:01:27.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.874+0000 7fec1eae0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec180c9ca0 0x7fec1815b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:27.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.874+0000 7fec1eae0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec180caac0 0x7fec18154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:27.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.874+0000 7fec1eae0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec1815ad70 0x7fec1815d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:27.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.874+0000 7fec1eae0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1815e310 con 0x7fec180c9ca0 2023-10-16T10:01:27.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.874+0000 7fec1eae0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec180c3790 con 0x7fec180caac0 2023-10-16T10:01:27.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.874+0000 7fec1eae0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec180c3a70 con 0x7fec1815ad70 2023-10-16T10:01:27.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.874+0000 7fec1c87c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec180c9ca0 0x7fec1815b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:27.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.874+0000 7fec1c87c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec180c9ca0 0x7fec1815b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56974/0 (socket says 172.21.15.84:56974) 2023-10-16T10:01:27.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.874+0000 7fec1c87c640 1 -- 172.21.15.84:0/2765339199 learned_addr learned my addr 172.21.15.84:0/2765339199 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:27.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.875+0000 7fec1c87c640 1 -- 172.21.15.84:0/2765339199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec1815ad70 msgr2=0x7fec1815d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:27.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.875+0000 7fec0ffff640 1 --2- 172.21.15.84:0/2765339199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec180caac0 0x7fec18154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:27.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.875+0000 7fec1c87c640 1 --2- 172.21.15.84:0/2765339199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec1815ad70 0x7fec1815d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:27.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.875+0000 7fec1c87c640 1 -- 172.21.15.84:0/2765339199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec180caac0 msgr2=0x7fec18154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:27.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.875+0000 7fec1c87c640 1 --2- 172.21.15.84:0/2765339199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec180caac0 0x7fec18154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:27.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.875+0000 7fec1c87c640 1 -- 172.21.15.84:0/2765339199 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec14009b30 con 0x7fec180c9ca0 2023-10-16T10:01:27.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.875+0000 7fec1c87c640 1 --2- 172.21.15.84:0/2765339199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec180c9ca0 0x7fec1815b450 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fec14009e50 tx=0x7fec14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:27.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.875+0000 7fec0d7fa640 1 -- 172.21.15.84:0/2765339199 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1403e450 con 0x7fec180c9ca0 2023-10-16T10:01:27.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.875+0000 7fec0d7fa640 1 -- 172.21.15.84:0/2765339199 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1403e5f0 con 0x7fec180c9ca0 2023-10-16T10:01:27.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.875+0000 7fec0d7fa640 1 -- 172.21.15.84:0/2765339199 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1403e9a0 con 0x7fec180c9ca0 2023-10-16T10:01:27.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.875+0000 7fec1eae0640 1 -- 172.21.15.84:0/2765339199 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec180c3d50 con 0x7fec180c9ca0 2023-10-16T10:01:27.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.876+0000 7fec1eae0640 1 -- 172.21.15.84:0/2765339199 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec180c6f50 con 0x7fec180c9ca0 2023-10-16T10:01:27.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.877+0000 7fec1eae0640 1 -- 172.21.15.84:0/2765339199 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec180c4270 con 0x7fec180c9ca0 2023-10-16T10:01:27.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.878+0000 7fec0d7fa640 1 -- 172.21.15.84:0/2765339199 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fec1403eb00 con 0x7fec180c9ca0 2023-10-16T10:01:27.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.879+0000 7fec0d7fa640 1 --2- 172.21.15.84:0/2765339199 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fec0406b0a0 0x7fec0406d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:27.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.879+0000 7fec0d7fa640 1 -- 172.21.15.84:0/2765339199 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fec140ac680 con 0x7fec180c9ca0 2023-10-16T10:01:27.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.879+0000 7fec0ffff640 1 --2- 172.21.15.84:0/2765339199 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fec0406b0a0 0x7fec0406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:27.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.879+0000 7fec0ffff640 1 --2- 172.21.15.84:0/2765339199 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fec0406b0a0 0x7fec0406d560 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fec08007fd0 tx=0x7fec08009040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:27.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:27.883+0000 7fec0d7fa640 1 -- 172.21.15.84:0/2765339199 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fec180c4270 con 0x7fec180c9ca0 2023-10-16T10:01:28.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.104+0000 7fec1eae0640 1 -- 172.21.15.84:0/2765339199 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fec180c3900 con 0x7fec0406b0a0 2023-10-16T10:01:28.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.294+0000 7fec0d7fa640 1 -- 172.21.15.84:0/2765339199 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fec180c3900 con 0x7fec0406b0a0 2023-10-16T10:01:28.296 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-10-16T10:01:28.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.297+0000 7fec1eae0640 1 -- 172.21.15.84:0/2765339199 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fec0406b0a0 msgr2=0x7fec0406d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:28.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.297+0000 7fec1eae0640 1 --2- 172.21.15.84:0/2765339199 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fec0406b0a0 0x7fec0406d560 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fec08007fd0 tx=0x7fec08009040 comp rx=0 tx=0).stop 2023-10-16T10:01:28.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.297+0000 7fec1eae0640 1 -- 172.21.15.84:0/2765339199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec180c9ca0 msgr2=0x7fec1815b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:28.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.297+0000 7fec1eae0640 1 --2- 172.21.15.84:0/2765339199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec180c9ca0 0x7fec1815b450 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fec14009e50 tx=0x7fec14004a40 comp rx=0 tx=0).stop 2023-10-16T10:01:28.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.297+0000 7fec1eae0640 1 -- 172.21.15.84:0/2765339199 shutdown_connections 2023-10-16T10:01:28.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.297+0000 7fec1eae0640 1 --2- 172.21.15.84:0/2765339199 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fec0406b0a0 0x7fec0406d560 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:28.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.297+0000 7fec1eae0640 1 --2- 172.21.15.84:0/2765339199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec1815ad70 0x7fec1815d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:28.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.297+0000 7fec1eae0640 1 --2- 172.21.15.84:0/2765339199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec180caac0 0x7fec18154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:28.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.297+0000 7fec1eae0640 1 --2- 172.21.15.84:0/2765339199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec180c9ca0 0x7fec1815b450 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:28.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.297+0000 7fec1eae0640 1 -- 172.21.15.84:0/2765339199 >> 172.21.15.84:0/2765339199 conn(0x7fec180ce9b0 msgr2=0x7fec180ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:28.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.297+0000 7fec1eae0640 1 -- 172.21.15.84:0/2765339199 shutdown_connections 2023-10-16T10:01:28.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.297+0000 7fec1eae0640 1 -- 172.21.15.84:0/2765339199 wait complete. 2023-10-16T10:01:28.317 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-10-16T10:01:28.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:28.319 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-10-16T10:01:28.375 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T10:01:28.376 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T10:01:28.376 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T10:01:28.376 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:28.377 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T10:01:28.377 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:28.377 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_00_UTC 2023-10-16T10:01:28.378 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T10:01:28.378 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_01_UTC 2023-10-16T10:01:28.379 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_01_UTC 2023-10-16T10:01:28.379 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:28.381 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_51_00_UTC', 'scheduled-2023-10-16-09_52_00_UTC', 'scheduled-2023-10-16-09_53_00_UTC', 'scheduled-2023-10-16-09_54_00_UTC', 'scheduled-2023-10-16-09_55_00_UTC', 'scheduled-2023-10-16-09_56_00_UTC', 'scheduled-2023-10-16-09_57_00_UTC', 'scheduled-2023-10-16-09_58_00_UTC', 'scheduled-2023-10-16-09_59_01_UTC', 'scheduled-2023-10-16-10_00_01_UTC', 'scheduled-2023-10-16-10_01_00_UTC'] 2023-10-16T10:01:28.381 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir0 2023-10-16T10:01:28.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.599+0000 7f83e11f4640 1 Processor -- start 2023-10-16T10:01:28.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.599+0000 7f83e11f4640 1 -- start start 2023-10-16T10:01:28.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.600+0000 7f83e11f4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83dc15cd70 0x7f83dc0d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:28.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.601+0000 7f83e11f4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83dc0cded0 0x7f83dc0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:28.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.601+0000 7f83e11f4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83dc15d150 con 0x7f83dc0b8480 2023-10-16T10:01:28.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.601+0000 7f83e11f4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83dc0d1180 con 0x7f83dc15cd70 2023-10-16T10:01:28.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.601+0000 7f83e11f4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc0d1460 con 0x7f83dc0cded0 2023-10-16T10:01:28.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.601+0000 7f83dbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83dc15cd70 0x7f83dc0d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:28.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.601+0000 7f83dbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83dc15cd70 0x7f83dc0d2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56976/0 (socket says 172.21.15.84:56976) 2023-10-16T10:01:28.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.601+0000 7f83dbfff640 1 -- 172.21.15.84:0/3115408974 learned_addr learned my addr 172.21.15.84:0/3115408974 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:28.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.601+0000 7f83e09f3640 1 --2- 172.21.15.84:0/3115408974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83dc0cded0 0x7f83dc0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:28.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.602+0000 7f83dbfff640 1 -- 172.21.15.84:0/3115408974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83dc0cded0 msgr2=0x7f83dc0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:28.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.602+0000 7f83dbfff640 1 --2- 172.21.15.84:0/3115408974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83dc0cded0 0x7f83dc0b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:28.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.602+0000 7f83dbfff640 1 -- 172.21.15.84:0/3115408974 >> v1:172.21.15.157:6789/0 conn(0x7f83dc0b8480 legacy=0x7f83dc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:28.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.602+0000 7f83dbfff640 1 -- 172.21.15.84:0/3115408974 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc0d1740 con 0x7f83dc15cd70 2023-10-16T10:01:28.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.603+0000 7f83dbfff640 1 --2- 172.21.15.84:0/3115408974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83dc15cd70 0x7f83dc0d2220 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0b56a0 tx=0x7f83cc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4860dee2fe0fe775 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:28.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.603+0000 7f83daffd640 1 -- 172.21.15.84:0/3115408974 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc031830 con 0x7f83dc15cd70 2023-10-16T10:01:28.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.603+0000 7f83daffd640 1 -- 172.21.15.84:0/3115408974 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83cc031990 con 0x7f83dc15cd70 2023-10-16T10:01:28.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.603+0000 7f83daffd640 1 -- 172.21.15.84:0/3115408974 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc031d20 con 0x7f83dc15cd70 2023-10-16T10:01:28.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.604+0000 7f83e11f4640 1 -- 172.21.15.84:0/3115408974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83dc15cd70 msgr2=0x7f83dc0d2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:28.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.604+0000 7f83e11f4640 1 --2- 172.21.15.84:0/3115408974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83dc15cd70 0x7f83dc0d2220 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0b56a0 tx=0x7f83cc02b8d0 comp rx=0 tx=0).stop 2023-10-16T10:01:28.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.604+0000 7f83e11f4640 1 -- 172.21.15.84:0/3115408974 shutdown_connections 2023-10-16T10:01:28.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.604+0000 7f83e11f4640 1 --2- 172.21.15.84:0/3115408974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83dc0cded0 0x7f83dc0b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:28.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.604+0000 7f83e11f4640 1 --2- 172.21.15.84:0/3115408974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83dc15cd70 0x7f83dc0d2220 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:28.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.604+0000 7f83e11f4640 1 -- 172.21.15.84:0/3115408974 >> 172.21.15.84:0/3115408974 conn(0x7f83dc0cf570 msgr2=0x7f83dc0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:28.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.604+0000 7f83e11f4640 1 -- 172.21.15.84:0/3115408974 shutdown_connections 2023-10-16T10:01:28.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.604+0000 7f83e11f4640 1 -- 172.21.15.84:0/3115408974 wait complete. 2023-10-16T10:01:28.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.606+0000 7f83e11f4640 1 Processor -- start 2023-10-16T10:01:28.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.606+0000 7f83e11f4640 1 -- start start 2023-10-16T10:01:28.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.607+0000 7f83e11f4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83dc0b8480 0x7f83dc14aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:28.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.607+0000 7f83e11f4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83dc0cded0 0x7f83dc0cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:28.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.607+0000 7f83e11f4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83dc15cd70 0x7f83dc0d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:28.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.607+0000 7f83e11f4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc14b1e0 con 0x7f83dc0cded0 2023-10-16T10:01:28.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.607+0000 7f83e11f4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc0ac560 con 0x7f83dc0b8480 2023-10-16T10:01:28.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.607+0000 7f83e11f4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc0cc470 con 0x7f83dc15cd70 2023-10-16T10:01:28.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.607+0000 7f83db7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83dc0cded0 0x7f83dc0cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:28.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.607+0000 7f83db7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83dc0cded0 0x7f83dc0cbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56990/0 (socket says 172.21.15.84:56990) 2023-10-16T10:01:28.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.607+0000 7f83db7fe640 1 -- 172.21.15.84:0/2651042935 learned_addr learned my addr 172.21.15.84:0/2651042935 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:28.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.607+0000 7f83dbfff640 1 --2- 172.21.15.84:0/2651042935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83dc0b8480 0x7f83dc14aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:28.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.607+0000 7f83db7fe640 1 -- 172.21.15.84:0/2651042935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83dc15cd70 msgr2=0x7f83dc0d2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:28.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.607+0000 7f83db7fe640 1 --2- 172.21.15.84:0/2651042935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83dc15cd70 0x7f83dc0d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:28.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.607+0000 7f83db7fe640 1 -- 172.21.15.84:0/2651042935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83dc0b8480 msgr2=0x7f83dc14aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:28.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.607+0000 7f83db7fe640 1 --2- 172.21.15.84:0/2651042935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83dc0b8480 0x7f83dc14aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:28.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.607+0000 7f83db7fe640 1 -- 172.21.15.84:0/2651042935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83cc0094d0 con 0x7f83dc0cded0 2023-10-16T10:01:28.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.608+0000 7f83db7fe640 1 --2- 172.21.15.84:0/2651042935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83dc0cded0 0x7f83dc0cbc40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f83d000c460 tx=0x7f83d000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:28.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.608+0000 7f83d8ff9640 1 -- 172.21.15.84:0/2651042935 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d0012830 con 0x7f83dc0cded0 2023-10-16T10:01:28.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.608+0000 7f83d8ff9640 1 -- 172.21.15.84:0/2651042935 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d00129d0 con 0x7f83dc0cded0 2023-10-16T10:01:28.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.608+0000 7f83e11f4640 1 -- 172.21.15.84:0/2651042935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83dc0cc750 con 0x7f83dc0cded0 2023-10-16T10:01:28.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.608+0000 7f83d8ff9640 1 -- 172.21.15.84:0/2651042935 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d0012d80 con 0x7f83dc0cded0 2023-10-16T10:01:28.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.608+0000 7f83e11f4640 1 -- 172.21.15.84:0/2651042935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83dc0ccd10 con 0x7f83dc0cded0 2023-10-16T10:01:28.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.609+0000 7f83e11f4640 1 -- 172.21.15.84:0/2651042935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83dc14b320 con 0x7f83dc0cded0 2023-10-16T10:01:28.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.610+0000 7f83d8ff9640 1 -- 172.21.15.84:0/2651042935 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f83d001a3b0 con 0x7f83dc0cded0 2023-10-16T10:01:28.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.611+0000 7f83d8ff9640 1 --2- 172.21.15.84:0/2651042935 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f83c806b070 0x7f83c806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:28.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.612+0000 7f83d8ff9640 1 -- 172.21.15.84:0/2651042935 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f83d0089800 con 0x7f83dc0cded0 2023-10-16T10:01:28.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.612+0000 7f83dbfff640 1 --2- 172.21.15.84:0/2651042935 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f83c806b070 0x7f83c806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:28.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.612+0000 7f83dbfff640 1 --2- 172.21.15.84:0/2651042935 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f83c806b070 0x7f83c806d530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f83cc036000 tx=0x7f83cc03c040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:28.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.616+0000 7f83d8ff9640 1 -- 172.21.15.84:0/2651042935 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f83dc14b320 con 0x7f83dc0cded0 2023-10-16T10:01:28.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.828+0000 7f83e11f4640 1 -- 172.21.15.84:0/2651042935 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f83dc0d1d00 con 0x7f83c806b070 2023-10-16T10:01:28.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.846+0000 7f83d8ff9640 1 -- 172.21.15.84:0/2651042935 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f83cc031b80 con 0x7f83c806b070 2023-10-16T10:01:28.847 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:01:28.848 INFO:teuthology.orchestra.run.smithi084.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:27", "first": "2023-10-16T09:51:00", "last": "2023-10-16T10:01:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:28.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.848+0000 7f83e11f4640 1 -- 172.21.15.84:0/2651042935 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f83c806b070 msgr2=0x7f83c806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:28.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.848+0000 7f83e11f4640 1 --2- 172.21.15.84:0/2651042935 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f83c806b070 0x7f83c806d530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f83cc036000 tx=0x7f83cc03c040 comp rx=0 tx=0).stop 2023-10-16T10:01:28.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.848+0000 7f83e11f4640 1 -- 172.21.15.84:0/2651042935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83dc0cded0 msgr2=0x7f83dc0cbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:28.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.848+0000 7f83e11f4640 1 --2- 172.21.15.84:0/2651042935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83dc0cded0 0x7f83dc0cbc40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f83d000c460 tx=0x7f83d000c920 comp rx=0 tx=0).stop 2023-10-16T10:01:28.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.849+0000 7f83e11f4640 1 -- 172.21.15.84:0/2651042935 shutdown_connections 2023-10-16T10:01:28.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.849+0000 7f83e11f4640 1 --2- 172.21.15.84:0/2651042935 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f83c806b070 0x7f83c806d530 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:28.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.849+0000 7f83e11f4640 1 --2- 172.21.15.84:0/2651042935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83dc15cd70 0x7f83dc0d2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:28.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.849+0000 7f83e11f4640 1 --2- 172.21.15.84:0/2651042935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83dc0cded0 0x7f83dc0cbc40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:28.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.849+0000 7f83e11f4640 1 --2- 172.21.15.84:0/2651042935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83dc0b8480 0x7f83dc14aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:28.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.849+0000 7f83e11f4640 1 -- 172.21.15.84:0/2651042935 >> 172.21.15.84:0/2651042935 conn(0x7f83dc0cf570 msgr2=0x7f83dc0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:28.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.849+0000 7f83e11f4640 1 -- 172.21.15.84:0/2651042935 shutdown_connections 2023-10-16T10:01:28.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:28.849+0000 7f83e11f4640 1 -- 172.21.15.84:0/2651042935 wait complete. 2023-10-16T10:01:28.870 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:27", "first": "2023-10-16T09:51:00", "last": "2023-10-16T10:01:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:28.870 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:50:27', 'first': '2023-10-16T09:51:00', 'last': '2023-10-16T10:01:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T10:01:28.871 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T10:01:28.871 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T10:01:28.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:28.872 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-10-16T10:01:28.925 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_01_UTC 2023-10-16T10:01:28.925 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T10:01:28.925 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T10:01:28.926 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:28.926 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T10:01:28.926 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:28.926 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_01_UTC 2023-10-16T10:01:28.927 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_01_UTC 2023-10-16T10:01:28.927 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_00_UTC 2023-10-16T10:01:28.927 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_00_UTC 2023-10-16T10:01:28.927 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:28.929 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_51_01_UTC', 'scheduled-2023-10-16-09_52_00_UTC', 'scheduled-2023-10-16-09_53_00_UTC', 'scheduled-2023-10-16-09_54_00_UTC', 'scheduled-2023-10-16-09_55_00_UTC', 'scheduled-2023-10-16-09_56_00_UTC', 'scheduled-2023-10-16-09_57_01_UTC', 'scheduled-2023-10-16-09_58_01_UTC', 'scheduled-2023-10-16-09_59_00_UTC', 'scheduled-2023-10-16-10_00_00_UTC', 'scheduled-2023-10-16-10_01_00_UTC'] 2023-10-16T10:01:28.929 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir1 2023-10-16T10:01:29.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.142+0000 7f43c260e640 1 Processor -- start 2023-10-16T10:01:29.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.142+0000 7f43c260e640 1 -- start start 2023-10-16T10:01:29.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.143+0000 7f43c260e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f43bc15ad70 0x7f43bc069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:29.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.143+0000 7f43c260e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f43bc0c9ca0 0x7f43bc06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:29.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.143+0000 7f43c260e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43bc0ca3b0 0x7f43bc0ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:29.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.143+0000 7f43c260e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43bc0cad80 con 0x7f43bc15ad70 2023-10-16T10:01:29.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.143+0000 7f43c260e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43bc0cb030 con 0x7f43bc0c9ca0 2023-10-16T10:01:29.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.143+0000 7f43c260e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc0c91c0 con 0x7f43bc0ca3b0 2023-10-16T10:01:29.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.144+0000 7f43bbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f43bc15ad70 0x7f43bc069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:29.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.144+0000 7f43bbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f43bc15ad70 0x7f43bc069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56998/0 (socket says 172.21.15.84:56998) 2023-10-16T10:01:29.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.144+0000 7f43bbfff640 1 -- 172.21.15.84:0/446542479 learned_addr learned my addr 172.21.15.84:0/446542479 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:29.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.144+0000 7f43bbfff640 1 -- 172.21.15.84:0/446542479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43bc0ca3b0 msgr2=0x7f43bc0ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:29.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.144+0000 7f43bbfff640 1 --2- 172.21.15.84:0/446542479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43bc0ca3b0 0x7f43bc0ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.144+0000 7f43b3fff640 1 --2- 172.21.15.84:0/446542479 >> v2:172.21.15.157:3300/0 conn(0x7f43bc0c9ca0 0x7f43bc06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:29.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.144+0000 7f43bbfff640 1 -- 172.21.15.84:0/446542479 >> v2:172.21.15.157:3300/0 conn(0x7f43bc0c9ca0 msgr2=0x7f43bc06a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:29.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.144+0000 7f43bbfff640 1 --2- 172.21.15.84:0/446542479 >> v2:172.21.15.157:3300/0 conn(0x7f43bc0c9ca0 0x7f43bc06a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.144+0000 7f43bbfff640 1 -- 172.21.15.84:0/446542479 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43bc0c94a0 con 0x7f43bc15ad70 2023-10-16T10:01:29.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.145+0000 7f43bbfff640 1 --2- 172.21.15.84:0/446542479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc15ad70 0x7f43bc069da0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0ce870 tx=0x7f43ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8399a0a6a79ff629 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:29.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.145+0000 7f43bb7fe640 1 -- 172.21.15.84:0/446542479 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac031870 con 0x7f43bc15ad70 2023-10-16T10:01:29.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.146+0000 7f43bb7fe640 1 -- 172.21.15.84:0/446542479 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ac0319d0 con 0x7f43bc15ad70 2023-10-16T10:01:29.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.146+0000 7f43bb7fe640 1 -- 172.21.15.84:0/446542479 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac031d60 con 0x7f43bc15ad70 2023-10-16T10:01:29.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.146+0000 7f43c260e640 1 -- 172.21.15.84:0/446542479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc15ad70 msgr2=0x7f43bc069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:29.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.146+0000 7f43c260e640 1 --2- 172.21.15.84:0/446542479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc15ad70 0x7f43bc069da0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0ce870 tx=0x7f43ac02b910 comp rx=0 tx=0).stop 2023-10-16T10:01:29.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.146+0000 7f43c260e640 1 -- 172.21.15.84:0/446542479 shutdown_connections 2023-10-16T10:01:29.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.146+0000 7f43c260e640 1 --2- 172.21.15.84:0/446542479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43bc0ca3b0 0x7f43bc0ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.146+0000 7f43c260e640 1 --2- 172.21.15.84:0/446542479 >> v2:172.21.15.157:3300/0 conn(0x7f43bc0c9ca0 0x7f43bc06a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.146+0000 7f43c260e640 1 --2- 172.21.15.84:0/446542479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc15ad70 0x7f43bc069da0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.146+0000 7f43c260e640 1 -- 172.21.15.84:0/446542479 >> 172.21.15.84:0/446542479 conn(0x7f43bc0ce9b0 msgr2=0x7f43bc0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:29.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.146+0000 7f43c260e640 1 -- 172.21.15.84:0/446542479 shutdown_connections 2023-10-16T10:01:29.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.146+0000 7f43c260e640 1 -- 172.21.15.84:0/446542479 wait complete. 2023-10-16T10:01:29.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.148+0000 7f43c260e640 1 Processor -- start 2023-10-16T10:01:29.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.148+0000 7f43c260e640 1 -- start start 2023-10-16T10:01:29.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.149+0000 7f43c260e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43bc0c9ca0 0x7f43bc0a8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:29.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.149+0000 7f43c260e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc0ca3b0 0x7f43bc0aa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:29.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.149+0000 7f43c260e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43bc15ad70 0x7f43bc0c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:29.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.149+0000 7f43c260e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc0a9870 con 0x7f43bc0ca3b0 2023-10-16T10:01:29.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.149+0000 7f43c260e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc0a89c0 con 0x7f43bc0c9ca0 2023-10-16T10:01:29.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.149+0000 7f43c260e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc0a8c60 con 0x7f43bc15ad70 2023-10-16T10:01:29.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.150+0000 7f43b3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc0ca3b0 0x7f43bc0aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:29.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.150+0000 7f43b3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc0ca3b0 0x7f43bc0aa510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57008/0 (socket says 172.21.15.84:57008) 2023-10-16T10:01:29.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.150+0000 7f43b3fff640 1 -- 172.21.15.84:0/1923489672 learned_addr learned my addr 172.21.15.84:0/1923489672 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:29.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.150+0000 7f43bbfff640 1 --2- 172.21.15.84:0/1923489672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43bc0c9ca0 0x7f43bc0a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:29.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.150+0000 7f43b3fff640 1 -- 172.21.15.84:0/1923489672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43bc15ad70 msgr2=0x7f43bc0c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:29.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.150+0000 7f43c0bab640 1 --2- 172.21.15.84:0/1923489672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43bc15ad70 0x7f43bc0c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:29.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.150+0000 7f43b3fff640 1 --2- 172.21.15.84:0/1923489672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43bc15ad70 0x7f43bc0c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.150+0000 7f43b3fff640 1 -- 172.21.15.84:0/1923489672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43bc0c9ca0 msgr2=0x7f43bc0a8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:29.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.150+0000 7f43b3fff640 1 --2- 172.21.15.84:0/1923489672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43bc0c9ca0 0x7f43bc0a8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.150+0000 7f43b3fff640 1 -- 172.21.15.84:0/1923489672 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac0094d0 con 0x7f43bc0ca3b0 2023-10-16T10:01:29.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.150+0000 7f43b3fff640 1 --2- 172.21.15.84:0/1923489672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc0ca3b0 0x7f43bc0aa510 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f43a400c7f0 tx=0x7f43a400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:29.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.151+0000 7f43b97fa640 1 -- 172.21.15.84:0/1923489672 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4004220 con 0x7f43bc0ca3b0 2023-10-16T10:01:29.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.151+0000 7f43b97fa640 1 -- 172.21.15.84:0/1923489672 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a40043a0 con 0x7f43bc0ca3b0 2023-10-16T10:01:29.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.151+0000 7f43b97fa640 1 -- 172.21.15.84:0/1923489672 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4007860 con 0x7f43bc0ca3b0 2023-10-16T10:01:29.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.151+0000 7f43c260e640 1 -- 172.21.15.84:0/1923489672 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc0a8f40 con 0x7f43bc0ca3b0 2023-10-16T10:01:29.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.151+0000 7f43c260e640 1 -- 172.21.15.84:0/1923489672 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43bc0a9500 con 0x7f43bc0ca3b0 2023-10-16T10:01:29.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.152+0000 7f43c260e640 1 -- 172.21.15.84:0/1923489672 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43bc04e160 con 0x7f43bc0ca3b0 2023-10-16T10:01:29.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.153+0000 7f43b97fa640 1 -- 172.21.15.84:0/1923489672 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f43a4007bd0 con 0x7f43bc0ca3b0 2023-10-16T10:01:29.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.154+0000 7f43b97fa640 1 --2- 172.21.15.84:0/1923489672 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f439c06b070 0x7f439c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:29.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.154+0000 7f43b97fa640 1 -- 172.21.15.84:0/1923489672 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f43a4089b80 con 0x7f43bc0ca3b0 2023-10-16T10:01:29.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.154+0000 7f43bbfff640 1 --2- 172.21.15.84:0/1923489672 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f439c06b070 0x7f439c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:29.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.155+0000 7f43bbfff640 1 --2- 172.21.15.84:0/1923489672 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f439c06b070 0x7f439c06d530 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f43ac036000 tx=0x7f43ac03c040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:29.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.158+0000 7f43b97fa640 1 -- 172.21.15.84:0/1923489672 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f43a40547a0 con 0x7f43bc0ca3b0 2023-10-16T10:01:29.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.373+0000 7f43c260e640 1 -- 172.21.15.84:0/1923489672 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43bc0c9780 con 0x7f439c06b070 2023-10-16T10:01:29.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.391+0000 7f43b97fa640 1 -- 172.21.15.84:0/1923489672 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f43ac031810 con 0x7f439c06b070 2023-10-16T10:01:29.393 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:01:29.394 INFO:teuthology.orchestra.run.smithi084.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:28", "first": "2023-10-16T09:51:01", "last": "2023-10-16T10:01:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:29.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.394+0000 7f43c260e640 1 -- 172.21.15.84:0/1923489672 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f439c06b070 msgr2=0x7f439c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:29.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.394+0000 7f43c260e640 1 --2- 172.21.15.84:0/1923489672 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f439c06b070 0x7f439c06d530 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f43ac036000 tx=0x7f43ac03c040 comp rx=0 tx=0).stop 2023-10-16T10:01:29.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.394+0000 7f43c260e640 1 -- 172.21.15.84:0/1923489672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc0ca3b0 msgr2=0x7f43bc0aa510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:29.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.394+0000 7f43c260e640 1 --2- 172.21.15.84:0/1923489672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc0ca3b0 0x7f43bc0aa510 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f43a400c7f0 tx=0x7f43a400ccb0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.394+0000 7f43c260e640 1 -- 172.21.15.84:0/1923489672 shutdown_connections 2023-10-16T10:01:29.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.394+0000 7f43c260e640 1 --2- 172.21.15.84:0/1923489672 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f439c06b070 0x7f439c06d530 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.394+0000 7f43c260e640 1 --2- 172.21.15.84:0/1923489672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43bc15ad70 0x7f43bc0c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.394+0000 7f43c260e640 1 --2- 172.21.15.84:0/1923489672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc0ca3b0 0x7f43bc0aa510 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.394+0000 7f43c260e640 1 --2- 172.21.15.84:0/1923489672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43bc0c9ca0 0x7f43bc0a8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.394+0000 7f43c260e640 1 -- 172.21.15.84:0/1923489672 >> 172.21.15.84:0/1923489672 conn(0x7f43bc0ce9b0 msgr2=0x7f43bc06b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:29.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.395+0000 7f43c260e640 1 -- 172.21.15.84:0/1923489672 shutdown_connections 2023-10-16T10:01:29.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.395+0000 7f43c260e640 1 -- 172.21.15.84:0/1923489672 wait complete. 2023-10-16T10:01:29.413 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:28", "first": "2023-10-16T09:51:01", "last": "2023-10-16T10:01:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:29.414 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:50:28', 'first': '2023-10-16T09:51:01', 'last': '2023-10-16T10:01:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T10:01:29.414 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T10:01:29.414 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T10:01:29.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:29.415 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-10-16T10:01:29.467 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T10:01:29.468 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_01_UTC 2023-10-16T10:01:29.468 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T10:01:29.468 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:29.468 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T10:01:29.469 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:29.469 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_01_UTC 2023-10-16T10:01:29.469 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_01_UTC 2023-10-16T10:01:29.469 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_01_UTC 2023-10-16T10:01:29.469 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_00_UTC 2023-10-16T10:01:29.470 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:29.471 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_51_00_UTC', 'scheduled-2023-10-16-09_52_01_UTC', 'scheduled-2023-10-16-09_53_00_UTC', 'scheduled-2023-10-16-09_54_00_UTC', 'scheduled-2023-10-16-09_55_00_UTC', 'scheduled-2023-10-16-09_56_00_UTC', 'scheduled-2023-10-16-09_57_01_UTC', 'scheduled-2023-10-16-09_58_01_UTC', 'scheduled-2023-10-16-09_59_01_UTC', 'scheduled-2023-10-16-10_00_00_UTC', 'scheduled-2023-10-16-10_01_00_UTC'] 2023-10-16T10:01:29.471 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir2 2023-10-16T10:01:29.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.687+0000 7f9b5e07c640 1 Processor -- start 2023-10-16T10:01:29.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.688+0000 7f9b5e07c640 1 -- start start 2023-10-16T10:01:29.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.689+0000 7f9b5e07c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b5815ad70 0x7f9b58069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:29.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.689+0000 7f9b5e07c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9b580caac0 0x7f9b5806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:29.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.689+0000 7f9b5e07c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b580cb030 con 0x7f9b580c9ca0 2023-10-16T10:01:29.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.689+0000 7f9b5e07c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b580c91c0 con 0x7f9b580caac0 2023-10-16T10:01:29.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.689+0000 7f9b5e07c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580c94a0 con 0x7f9b5815ad70 2023-10-16T10:01:29.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.689+0000 7f9b56ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9b580c9ca0 0x7f9b580ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39884/0 (socket says 172.21.15.84:39884) 2023-10-16T10:01:29.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.690+0000 7f9b56ffd640 1 -- 172.21.15.84:0/1888215789 learned_addr learned my addr 172.21.15.84:0/1888215789 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:29.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.689+0000 7f9b57fff640 1 --2- 172.21.15.84:0/1888215789 >> v2:172.21.15.84:3300/0 conn(0x7f9b580caac0 0x7f9b5806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:29.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.690+0000 7f9b577fe640 1 --2- 172.21.15.84:0/1888215789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b5815ad70 0x7f9b58069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:29.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.690+0000 7f9b567fc640 1 -- 172.21.15.84:0/1888215789 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807208208 0 0) 0x7f9b580cb030 con 0x7f9b580c9ca0 2023-10-16T10:01:29.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.690+0000 7f9b567fc640 1 -- 172.21.15.84:0/1888215789 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b44003520 con 0x7f9b580c9ca0 2023-10-16T10:01:29.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.690+0000 7f9b57fff640 1 -- 172.21.15.84:0/1888215789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b5815ad70 msgr2=0x7f9b58069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:29.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.690+0000 7f9b57fff640 1 --2- 172.21.15.84:0/1888215789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b5815ad70 0x7f9b58069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.690+0000 7f9b57fff640 1 -- 172.21.15.84:0/1888215789 >> v1:172.21.15.84:6789/0 conn(0x7f9b580c9ca0 legacy=0x7f9b580ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:29.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.690+0000 7f9b57fff640 1 -- 172.21.15.84:0/1888215789 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b580c9780 con 0x7f9b580caac0 2023-10-16T10:01:29.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.691+0000 7f9b57fff640 1 --2- 172.21.15.84:0/1888215789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b580caac0 0x7f9b5806a2e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9b500097a0 tx=0x7f9b5002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd48abef8723436 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:29.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.692+0000 7f9b567fc640 1 -- 172.21.15.84:0/1888215789 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5002d040 con 0x7f9b580caac0 2023-10-16T10:01:29.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.692+0000 7f9b567fc640 1 -- 172.21.15.84:0/1888215789 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b50031810 con 0x7f9b580caac0 2023-10-16T10:01:29.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.692+0000 7f9b567fc640 1 -- 172.21.15.84:0/1888215789 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50031b80 con 0x7f9b580caac0 2023-10-16T10:01:29.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.692+0000 7f9b5e07c640 1 -- 172.21.15.84:0/1888215789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b580caac0 msgr2=0x7f9b5806a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:29.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.692+0000 7f9b5e07c640 1 --2- 172.21.15.84:0/1888215789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b580caac0 0x7f9b5806a2e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9b500097a0 tx=0x7f9b5002b8b0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.692+0000 7f9b5e07c640 1 -- 172.21.15.84:0/1888215789 shutdown_connections 2023-10-16T10:01:29.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.692+0000 7f9b5e07c640 1 --2- 172.21.15.84:0/1888215789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b580caac0 0x7f9b5806a2e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.692+0000 7f9b5e07c640 1 --2- 172.21.15.84:0/1888215789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b5815ad70 0x7f9b58069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.692+0000 7f9b5e07c640 1 -- 172.21.15.84:0/1888215789 >> 172.21.15.84:0/1888215789 conn(0x7f9b580ce9b0 msgr2=0x7f9b580cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:29.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.692+0000 7f9b5e07c640 1 -- 172.21.15.84:0/1888215789 shutdown_connections 2023-10-16T10:01:29.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.692+0000 7f9b5e07c640 1 -- 172.21.15.84:0/1888215789 wait complete. 2023-10-16T10:01:29.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.694+0000 7f9b5e07c640 1 Processor -- start 2023-10-16T10:01:29.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.694+0000 7f9b5e07c640 1 -- start start 2023-10-16T10:01:29.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.695+0000 7f9b5e07c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b580c9ca0 0x7f9b5815b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:29.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.695+0000 7f9b5e07c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b580caac0 0x7f9b580c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:29.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.695+0000 7f9b5e07c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b5815ad70 0x7f9b5815dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:29.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.695+0000 7f9b5e07c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580c7780 con 0x7f9b580caac0 2023-10-16T10:01:29.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.695+0000 7f9b5e07c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580c6b70 con 0x7f9b5815ad70 2023-10-16T10:01:29.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.695+0000 7f9b5e07c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580c6e50 con 0x7f9b580c9ca0 2023-10-16T10:01:29.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.695+0000 7f9b56ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b580caac0 0x7f9b580c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:29.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.695+0000 7f9b56ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b580caac0 0x7f9b580c4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57016/0 (socket says 172.21.15.84:57016) 2023-10-16T10:01:29.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.695+0000 7f9b56ffd640 1 -- 172.21.15.84:0/498960227 learned_addr learned my addr 172.21.15.84:0/498960227 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:29.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.696+0000 7f9b56ffd640 1 -- 172.21.15.84:0/498960227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b580c9ca0 msgr2=0x7f9b5815b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:29.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.696+0000 7f9b56ffd640 1 --2- 172.21.15.84:0/498960227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b580c9ca0 0x7f9b5815b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.696+0000 7f9b56ffd640 1 -- 172.21.15.84:0/498960227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b5815ad70 msgr2=0x7f9b5815dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:29.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.696+0000 7f9b56ffd640 1 --2- 172.21.15.84:0/498960227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b5815ad70 0x7f9b5815dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.696+0000 7f9b56ffd640 1 -- 172.21.15.84:0/498960227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b50009400 con 0x7f9b580caac0 2023-10-16T10:01:29.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.696+0000 7f9b56ffd640 1 --2- 172.21.15.84:0/498960227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b580caac0 0x7f9b580c4ca0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000c430 tx=0x7f9b4000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:29.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.696+0000 7f9b4ffff640 1 -- 172.21.15.84:0/498960227 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40005ce0 con 0x7f9b580caac0 2023-10-16T10:01:29.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.696+0000 7f9b4ffff640 1 -- 172.21.15.84:0/498960227 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b40005e40 con 0x7f9b580caac0 2023-10-16T10:01:29.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.696+0000 7f9b4ffff640 1 -- 172.21.15.84:0/498960227 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b400072f0 con 0x7f9b580caac0 2023-10-16T10:01:29.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.696+0000 7f9b5e07c640 1 -- 172.21.15.84:0/498960227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b580c7130 con 0x7f9b580caac0 2023-10-16T10:01:29.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.697+0000 7f9b5e07c640 1 -- 172.21.15.84:0/498960227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b580c51e0 con 0x7f9b580caac0 2023-10-16T10:01:29.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.698+0000 7f9b5e07c640 1 -- 172.21.15.84:0/498960227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5804e160 con 0x7f9b580caac0 2023-10-16T10:01:29.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.699+0000 7f9b4ffff640 1 -- 172.21.15.84:0/498960227 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f9b40007450 con 0x7f9b580caac0 2023-10-16T10:01:29.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.700+0000 7f9b4ffff640 1 --2- 172.21.15.84:0/498960227 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f9b3806b070 0x7f9b3806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:29.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.700+0000 7f9b4ffff640 1 -- 172.21.15.84:0/498960227 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f9b40089130 con 0x7f9b580caac0 2023-10-16T10:01:29.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.700+0000 7f9b577fe640 1 --2- 172.21.15.84:0/498960227 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f9b3806b070 0x7f9b3806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:29.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.701+0000 7f9b577fe640 1 --2- 172.21.15.84:0/498960227 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f9b3806b070 0x7f9b3806d530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9b480070b0 tx=0x7f9b48007040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:29.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.705+0000 7f9b4ffff640 1 -- 172.21.15.84:0/498960227 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9b40053e60 con 0x7f9b580caac0 2023-10-16T10:01:29.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.920+0000 7f9b5e07c640 1 -- 172.21.15.84:0/498960227 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9b580ad4d0 con 0x7f9b3806b070 2023-10-16T10:01:29.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.935+0000 7f9b4ffff640 1 -- 172.21.15.84:0/498960227 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f9b48009b10 con 0x7f9b3806b070 2023-10-16T10:01:29.937 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:01:29.937 INFO:teuthology.orchestra.run.smithi084.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:29", "first": "2023-10-16T09:51:00", "last": "2023-10-16T10:01:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:29.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.938+0000 7f9b5e07c640 1 -- 172.21.15.84:0/498960227 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f9b3806b070 msgr2=0x7f9b3806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:29.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.938+0000 7f9b5e07c640 1 --2- 172.21.15.84:0/498960227 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f9b3806b070 0x7f9b3806d530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9b480070b0 tx=0x7f9b48007040 comp rx=0 tx=0).stop 2023-10-16T10:01:29.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.938+0000 7f9b5e07c640 1 -- 172.21.15.84:0/498960227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b580caac0 msgr2=0x7f9b580c4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:29.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.938+0000 7f9b5e07c640 1 --2- 172.21.15.84:0/498960227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b580caac0 0x7f9b580c4ca0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000c430 tx=0x7f9b4000c8f0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.938+0000 7f9b5e07c640 1 -- 172.21.15.84:0/498960227 shutdown_connections 2023-10-16T10:01:29.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.938+0000 7f9b5e07c640 1 --2- 172.21.15.84:0/498960227 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f9b3806b070 0x7f9b3806d530 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.938+0000 7f9b5e07c640 1 --2- 172.21.15.84:0/498960227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b5815ad70 0x7f9b5815dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.938+0000 7f9b5e07c640 1 --2- 172.21.15.84:0/498960227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b580caac0 0x7f9b580c4ca0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.938+0000 7f9b5e07c640 1 --2- 172.21.15.84:0/498960227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b580c9ca0 0x7f9b5815b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:29.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.938+0000 7f9b5e07c640 1 -- 172.21.15.84:0/498960227 >> 172.21.15.84:0/498960227 conn(0x7f9b580ce9b0 msgr2=0x7f9b580afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:29.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.938+0000 7f9b5e07c640 1 -- 172.21.15.84:0/498960227 shutdown_connections 2023-10-16T10:01:29.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:29.938+0000 7f9b5e07c640 1 -- 172.21.15.84:0/498960227 wait complete. 2023-10-16T10:01:29.957 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:29", "first": "2023-10-16T09:51:00", "last": "2023-10-16T10:01:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:29.958 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:50:29', 'first': '2023-10-16T09:51:00', 'last': '2023-10-16T10:01:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T10:01:29.958 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T10:01:29.959 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T10:01:29.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:29.959 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-10-16T10:01:30.013 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_01_UTC 2023-10-16T10:01:30.013 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_01_UTC 2023-10-16T10:01:30.013 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_01_UTC 2023-10-16T10:01:30.013 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_01_UTC 2023-10-16T10:01:30.014 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_01_UTC 2023-10-16T10:01:30.014 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:30.014 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_00_UTC 2023-10-16T10:01:30.014 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_01_UTC 2023-10-16T10:01:30.015 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_01_UTC 2023-10-16T10:01:30.015 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_01_UTC 2023-10-16T10:01:30.015 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_01_UTC 2023-10-16T10:01:30.016 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_51_01_UTC', 'scheduled-2023-10-16-09_52_01_UTC', 'scheduled-2023-10-16-09_53_01_UTC', 'scheduled-2023-10-16-09_54_01_UTC', 'scheduled-2023-10-16-09_55_01_UTC', 'scheduled-2023-10-16-09_56_00_UTC', 'scheduled-2023-10-16-09_57_00_UTC', 'scheduled-2023-10-16-09_58_01_UTC', 'scheduled-2023-10-16-09_59_01_UTC', 'scheduled-2023-10-16-10_00_01_UTC', 'scheduled-2023-10-16-10_01_01_UTC'] 2023-10-16T10:01:30.017 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir3 2023-10-16T10:01:30.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.232+0000 7fe5f70f2640 1 Processor -- start 2023-10-16T10:01:30.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.233+0000 7fe5f70f2640 1 -- start start 2023-10-16T10:01:30.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.234+0000 7fe5f70f2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe5f015ad70 0x7fe5f0069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:30.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.234+0000 7fe5f70f2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f00c9ca0 0x7fe5f006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:30.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.234+0000 7fe5f70f2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe5f00ca3b0 0x7fe5f00ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:30.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.234+0000 7fe5f70f2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f00cad80 con 0x7fe5f015ad70 2023-10-16T10:01:30.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.234+0000 7fe5f70f2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f00cb030 con 0x7fe5f00ca3b0 2023-10-16T10:01:30.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.234+0000 7fe5f70f2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f00c91c0 con 0x7fe5f00c9ca0 2023-10-16T10:01:30.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.234+0000 7fe5f4e8e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe5f015ad70 0x7fe5f0069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:30.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.235+0000 7fe5f4e8e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe5f015ad70 0x7fe5f0069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57020/0 (socket says 172.21.15.84:57020) 2023-10-16T10:01:30.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.235+0000 7fe5f4e8e640 1 -- 172.21.15.84:0/4078675134 learned_addr learned my addr 172.21.15.84:0/4078675134 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:30.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.235+0000 7fe5f568f640 1 --2- 172.21.15.84:0/4078675134 >> v2:172.21.15.157:3300/0 conn(0x7fe5f00ca3b0 0x7fe5f00ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:30.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.235+0000 7fe5effff640 1 --2- 172.21.15.84:0/4078675134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f00c9ca0 0x7fe5f006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:30.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.235+0000 7fe5f4e8e640 1 -- 172.21.15.84:0/4078675134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f00c9ca0 msgr2=0x7fe5f006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:30.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.235+0000 7fe5f4e8e640 1 --2- 172.21.15.84:0/4078675134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f00c9ca0 0x7fe5f006a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:30.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.235+0000 7fe5f4e8e640 1 -- 172.21.15.84:0/4078675134 >> v2:172.21.15.157:3300/0 conn(0x7fe5f00ca3b0 msgr2=0x7fe5f00ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:30.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.235+0000 7fe5f4e8e640 1 --2- 172.21.15.84:0/4078675134 >> v2:172.21.15.157:3300/0 conn(0x7fe5f00ca3b0 0x7fe5f00ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:30.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.235+0000 7fe5f4e8e640 1 -- 172.21.15.84:0/4078675134 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f00c94a0 con 0x7fe5f015ad70 2023-10-16T10:01:30.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.236+0000 7fe5f4e8e640 1 --2- 172.21.15.84:0/4078675134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f015ad70 0x7fe5f0069da0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00cf820 tx=0x7fe5e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd42bfaf1d7f0de server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:30.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.237+0000 7fe5ef7fe640 1 -- 172.21.15.84:0/4078675134 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4031870 con 0x7fe5f015ad70 2023-10-16T10:01:30.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.237+0000 7fe5ef7fe640 1 -- 172.21.15.84:0/4078675134 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e40319d0 con 0x7fe5f015ad70 2023-10-16T10:01:30.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.237+0000 7fe5ef7fe640 1 -- 172.21.15.84:0/4078675134 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4031d60 con 0x7fe5f015ad70 2023-10-16T10:01:30.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.237+0000 7fe5f70f2640 1 -- 172.21.15.84:0/4078675134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f015ad70 msgr2=0x7fe5f0069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:30.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.237+0000 7fe5f70f2640 1 --2- 172.21.15.84:0/4078675134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f015ad70 0x7fe5f0069da0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00cf820 tx=0x7fe5e402b910 comp rx=0 tx=0).stop 2023-10-16T10:01:30.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.237+0000 7fe5f70f2640 1 -- 172.21.15.84:0/4078675134 shutdown_connections 2023-10-16T10:01:30.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.237+0000 7fe5f70f2640 1 --2- 172.21.15.84:0/4078675134 >> v2:172.21.15.157:3300/0 conn(0x7fe5f00ca3b0 0x7fe5f00ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:30.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.237+0000 7fe5f70f2640 1 --2- 172.21.15.84:0/4078675134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f00c9ca0 0x7fe5f006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:30.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.237+0000 7fe5f70f2640 1 --2- 172.21.15.84:0/4078675134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f015ad70 0x7fe5f0069da0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:30.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.237+0000 7fe5f70f2640 1 -- 172.21.15.84:0/4078675134 >> 172.21.15.84:0/4078675134 conn(0x7fe5f00ce9b0 msgr2=0x7fe5f00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:30.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.237+0000 7fe5f70f2640 1 -- 172.21.15.84:0/4078675134 shutdown_connections 2023-10-16T10:01:30.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.237+0000 7fe5f70f2640 1 -- 172.21.15.84:0/4078675134 wait complete. 2023-10-16T10:01:30.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.239+0000 7fe5f70f2640 1 Processor -- start 2023-10-16T10:01:30.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.239+0000 7fe5f70f2640 1 -- start start 2023-10-16T10:01:30.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.240+0000 7fe5f70f2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f00c9ca0 0x7fe5f014f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:30.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.240+0000 7fe5f70f2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f00ca3b0 0x7fe5f00a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:30.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.240+0000 7fe5f70f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f015ad70 0x7fe5f00c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:30.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.240+0000 7fe5f70f2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f00af560 con 0x7fe5f00c9ca0 2023-10-16T10:01:30.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.240+0000 7fe5f70f2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f00b35a0 con 0x7fe5f015ad70 2023-10-16T10:01:30.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.240+0000 7fe5f70f2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f00b2990 con 0x7fe5f00ca3b0 2023-10-16T10:01:30.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.240+0000 7fe5f4e8e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f00c9ca0 0x7fe5f014f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:30.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.240+0000 7fe5f4e8e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f00c9ca0 0x7fe5f014f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57036/0 (socket says 172.21.15.84:57036) 2023-10-16T10:01:30.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.240+0000 7fe5f4e8e640 1 -- 172.21.15.84:0/3195722726 learned_addr learned my addr 172.21.15.84:0/3195722726 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:30.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.241+0000 7fe5f568f640 1 --2- 172.21.15.84:0/3195722726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f015ad70 0x7fe5f00c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:30.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.241+0000 7fe5effff640 1 --2- 172.21.15.84:0/3195722726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f00ca3b0 0x7fe5f00a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:30.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.241+0000 7fe5f4e8e640 1 -- 172.21.15.84:0/3195722726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f00ca3b0 msgr2=0x7fe5f00a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:30.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.241+0000 7fe5f4e8e640 1 --2- 172.21.15.84:0/3195722726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f00ca3b0 0x7fe5f00a8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:30.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.241+0000 7fe5f4e8e640 1 -- 172.21.15.84:0/3195722726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f015ad70 msgr2=0x7fe5f00c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:30.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.241+0000 7fe5f4e8e640 1 --2- 172.21.15.84:0/3195722726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f015ad70 0x7fe5f00c7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:30.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.241+0000 7fe5f4e8e640 1 -- 172.21.15.84:0/3195722726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e40094d0 con 0x7fe5f00c9ca0 2023-10-16T10:01:30.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.241+0000 7fe5effff640 1 --2- 172.21.15.84:0/3195722726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f00ca3b0 0x7fe5f00a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:30.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.241+0000 7fe5f4e8e640 1 --2- 172.21.15.84:0/3195722726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f00c9ca0 0x7fe5f014f980 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4000c00 tx=0x7fe5e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:30.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.241+0000 7fe5ed7fa640 1 -- 172.21.15.84:0/3195722726 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e403abe0 con 0x7fe5f00c9ca0 2023-10-16T10:01:30.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.242+0000 7fe5ed7fa640 1 -- 172.21.15.84:0/3195722726 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e403ad40 con 0x7fe5f00c9ca0 2023-10-16T10:01:30.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.242+0000 7fe5f70f2640 1 -- 172.21.15.84:0/3195722726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f00b2c70 con 0x7fe5f00c9ca0 2023-10-16T10:01:30.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.242+0000 7fe5ed7fa640 1 -- 172.21.15.84:0/3195722726 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4044610 con 0x7fe5f00c9ca0 2023-10-16T10:01:30.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.242+0000 7fe5f70f2640 1 -- 172.21.15.84:0/3195722726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f00b3230 con 0x7fe5f00c9ca0 2023-10-16T10:01:30.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.243+0000 7fe5f70f2640 1 -- 172.21.15.84:0/3195722726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f004e160 con 0x7fe5f00c9ca0 2023-10-16T10:01:30.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.244+0000 7fe5ed7fa640 1 -- 172.21.15.84:0/3195722726 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fe5e404d450 con 0x7fe5f00c9ca0 2023-10-16T10:01:30.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.245+0000 7fe5ed7fa640 1 --2- 172.21.15.84:0/3195722726 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fe5d406b070 0x7fe5d406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:30.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.245+0000 7fe5ed7fa640 1 -- 172.21.15.84:0/3195722726 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fe5e402d040 con 0x7fe5f00c9ca0 2023-10-16T10:01:30.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.245+0000 7fe5effff640 1 --2- 172.21.15.84:0/3195722726 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fe5d406b070 0x7fe5d406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:30.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.246+0000 7fe5effff640 1 --2- 172.21.15.84:0/3195722726 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fe5d406b070 0x7fe5d406d530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00c8a20 tx=0x7fe5dc008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:30.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.249+0000 7fe5ed7fa640 1 -- 172.21.15.84:0/3195722726 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe5e4076db0 con 0x7fe5f00c9ca0 2023-10-16T10:01:30.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.461+0000 7fe5f70f2640 1 -- 172.21.15.84:0/3195722726 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5f00c9780 con 0x7fe5d406b070 2023-10-16T10:01:30.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.474+0000 7fe5ed7fa640 1 -- 172.21.15.84:0/3195722726 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fe5dc009550 con 0x7fe5d406b070 2023-10-16T10:01:30.477 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:01:30.477 INFO:teuthology.orchestra.run.smithi084.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:30", "first": "2023-10-16T09:51:01", "last": "2023-10-16T10:01:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:30.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.477+0000 7fe5f70f2640 1 -- 172.21.15.84:0/3195722726 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fe5d406b070 msgr2=0x7fe5d406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:30.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.477+0000 7fe5f70f2640 1 --2- 172.21.15.84:0/3195722726 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fe5d406b070 0x7fe5d406d530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00c8a20 tx=0x7fe5dc008040 comp rx=0 tx=0).stop 2023-10-16T10:01:30.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.477+0000 7fe5f70f2640 1 -- 172.21.15.84:0/3195722726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f00c9ca0 msgr2=0x7fe5f014f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:30.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.477+0000 7fe5f70f2640 1 --2- 172.21.15.84:0/3195722726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f00c9ca0 0x7fe5f014f980 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4000c00 tx=0x7fe5e40046c0 comp rx=0 tx=0).stop 2023-10-16T10:01:30.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.477+0000 7fe5f70f2640 1 -- 172.21.15.84:0/3195722726 shutdown_connections 2023-10-16T10:01:30.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.477+0000 7fe5f70f2640 1 --2- 172.21.15.84:0/3195722726 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fe5d406b070 0x7fe5d406d530 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:30.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.477+0000 7fe5f70f2640 1 --2- 172.21.15.84:0/3195722726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f015ad70 0x7fe5f00c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:30.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.477+0000 7fe5f70f2640 1 --2- 172.21.15.84:0/3195722726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5f00ca3b0 0x7fe5f00a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:30.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.477+0000 7fe5f70f2640 1 --2- 172.21.15.84:0/3195722726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f00c9ca0 0x7fe5f014f980 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:30.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.477+0000 7fe5f70f2640 1 -- 172.21.15.84:0/3195722726 >> 172.21.15.84:0/3195722726 conn(0x7fe5f00ce9b0 msgr2=0x7fe5f00ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:30.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.477+0000 7fe5f70f2640 1 -- 172.21.15.84:0/3195722726 shutdown_connections 2023-10-16T10:01:30.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.477+0000 7fe5f70f2640 1 -- 172.21.15.84:0/3195722726 wait complete. 2023-10-16T10:01:30.498 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:30", "first": "2023-10-16T09:51:01", "last": "2023-10-16T10:01:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:30.499 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:50:30', 'first': '2023-10-16T09:51:01', 'last': '2023-10-16T10:01:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T10:01:30.499 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T10:01:30.499 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T10:01:30.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:30.500 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-10-16T10:01:30.554 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T10:01:30.555 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T10:01:30.555 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T10:01:30.555 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:30.556 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T10:01:30.556 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_01_UTC 2023-10-16T10:01:30.557 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_00_UTC 2023-10-16T10:01:30.557 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T10:01:30.557 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_00_UTC 2023-10-16T10:01:30.558 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_00_UTC 2023-10-16T10:01:30.558 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:30.559 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_51_00_UTC', 'scheduled-2023-10-16-09_52_00_UTC', 'scheduled-2023-10-16-09_53_00_UTC', 'scheduled-2023-10-16-09_54_00_UTC', 'scheduled-2023-10-16-09_55_00_UTC', 'scheduled-2023-10-16-09_56_01_UTC', 'scheduled-2023-10-16-09_57_00_UTC', 'scheduled-2023-10-16-09_58_00_UTC', 'scheduled-2023-10-16-09_59_00_UTC', 'scheduled-2023-10-16-10_00_00_UTC', 'scheduled-2023-10-16-10_01_00_UTC'] 2023-10-16T10:01:30.560 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir4 2023-10-16T10:01:30.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.770+0000 7efc2b71d640 1 Processor -- start 2023-10-16T10:01:30.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.771+0000 7efc2b71d640 1 -- start start 2023-10-16T10:01:30.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.772+0000 7efc2b71d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efc240addb0 0x7efc240caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:30.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.772+0000 7efc2b71d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc2406a4b0 0x7efc241650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:30.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.772+0000 7efc2b71d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc240cb140 con 0x7efc240c9000 2023-10-16T10:01:30.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.772+0000 7efc2b71d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc24161a50 con 0x7efc240addb0 2023-10-16T10:01:30.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.772+0000 7efc2b71d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2406b030 con 0x7efc2406a4b0 2023-10-16T10:01:30.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.772+0000 7efc294b9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efc240addb0 0x7efc240caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:30.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.772+0000 7efc294b9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efc240addb0 0x7efc240caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57038/0 (socket says 172.21.15.84:57038) 2023-10-16T10:01:30.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.772+0000 7efc294b9640 1 -- 172.21.15.84:0/2884287100 learned_addr learned my addr 172.21.15.84:0/2884287100 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:30.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.773+0000 7efc294b9640 1 -- 172.21.15.84:0/2884287100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc2406a4b0 msgr2=0x7efc241650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:30.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.773+0000 7efc294b9640 1 --2- 172.21.15.84:0/2884287100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc2406a4b0 0x7efc241650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:30.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.773+0000 7efc294b9640 1 -- 172.21.15.84:0/2884287100 >> v1:172.21.15.157:6789/0 conn(0x7efc240c9000 legacy=0x7efc24069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:30.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.773+0000 7efc294b9640 1 -- 172.21.15.84:0/2884287100 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2406b310 con 0x7efc240addb0 2023-10-16T10:01:30.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.774+0000 7efc294b9640 1 --2- 172.21.15.84:0/2884287100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc240addb0 0x7efc240caa80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efc20009870 tx=0x7efc2002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d33679e9ee3651a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:30.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.774+0000 7efc1bfff640 1 -- 172.21.15.84:0/2884287100 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc20031640 con 0x7efc240addb0 2023-10-16T10:01:30.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.774+0000 7efc1bfff640 1 -- 172.21.15.84:0/2884287100 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc200317a0 con 0x7efc240addb0 2023-10-16T10:01:30.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.774+0000 7efc1bfff640 1 -- 172.21.15.84:0/2884287100 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc20031b30 con 0x7efc240addb0 2023-10-16T10:01:30.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.775+0000 7efc2b71d640 1 -- 172.21.15.84:0/2884287100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc240addb0 msgr2=0x7efc240caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:30.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.775+0000 7efc2b71d640 1 --2- 172.21.15.84:0/2884287100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc240addb0 0x7efc240caa80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efc20009870 tx=0x7efc2002b8f0 comp rx=0 tx=0).stop 2023-10-16T10:01:30.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.775+0000 7efc2b71d640 1 -- 172.21.15.84:0/2884287100 shutdown_connections 2023-10-16T10:01:30.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.775+0000 7efc2b71d640 1 --2- 172.21.15.84:0/2884287100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc2406a4b0 0x7efc241650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:30.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.775+0000 7efc2b71d640 1 --2- 172.21.15.84:0/2884287100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc240addb0 0x7efc240caa80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:30.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.775+0000 7efc2b71d640 1 -- 172.21.15.84:0/2884287100 >> 172.21.15.84:0/2884287100 conn(0x7efc240ce9b0 msgr2=0x7efc240cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:30.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.775+0000 7efc2b71d640 1 -- 172.21.15.84:0/2884287100 shutdown_connections 2023-10-16T10:01:30.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.775+0000 7efc2b71d640 1 -- 172.21.15.84:0/2884287100 wait complete. 2023-10-16T10:01:30.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.777+0000 7efc2b71d640 1 Processor -- start 2023-10-16T10:01:30.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.777+0000 7efc2b71d640 1 -- start start 2023-10-16T10:01:30.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.778+0000 7efc2b71d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc2406a4b0 0x7efc2415e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:30.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.778+0000 7efc2b71d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc240addb0 0x7efc240c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:30.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.778+0000 7efc2b71d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc240c9000 0x7efc240b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:30.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.778+0000 7efc2b71d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc240c2a00 con 0x7efc2406a4b0 2023-10-16T10:01:30.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.778+0000 7efc2b71d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc240c14d0 con 0x7efc240addb0 2023-10-16T10:01:30.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.778+0000 7efc2b71d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2415f420 con 0x7efc240c9000 2023-10-16T10:01:30.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.778+0000 7efc294b9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc2406a4b0 0x7efc2415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:30.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.778+0000 7efc294b9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc2406a4b0 0x7efc2415e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57050/0 (socket says 172.21.15.84:57050) 2023-10-16T10:01:30.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.778+0000 7efc294b9640 1 -- 172.21.15.84:0/3094924247 learned_addr learned my addr 172.21.15.84:0/3094924247 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.778+0000 7efc294b9640 1 -- 172.21.15.84:0/3094924247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc240c9000 msgr2=0x7efc240b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.778+0000 7efc294b9640 1 --2- 172.21.15.84:0/3094924247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc240c9000 0x7efc240b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.778+0000 7efc294b9640 1 -- 172.21.15.84:0/3094924247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc240addb0 msgr2=0x7efc240c24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.778+0000 7efc294b9640 1 --2- 172.21.15.84:0/3094924247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc240addb0 0x7efc240c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.778+0000 7efc294b9640 1 -- 172.21.15.84:0/3094924247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc200094d0 con 0x7efc2406a4b0 2023-10-16T10:01:30.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.779+0000 7efc294b9640 1 --2- 172.21.15.84:0/3094924247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc2406a4b0 0x7efc2415e720 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efc20009870 tx=0x7efc200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:30.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.779+0000 7efc19ffb640 1 -- 172.21.15.84:0/3094924247 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2003ac60 con 0x7efc2406a4b0 2023-10-16T10:01:30.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.779+0000 7efc19ffb640 1 -- 172.21.15.84:0/3094924247 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2003adc0 con 0x7efc2406a4b0 2023-10-16T10:01:30.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.779+0000 7efc2b71d640 1 -- 172.21.15.84:0/3094924247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2415f140 con 0x7efc2406a4b0 2023-10-16T10:01:30.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.779+0000 7efc19ffb640 1 -- 172.21.15.84:0/3094924247 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc20044610 con 0x7efc2406a4b0 2023-10-16T10:01:30.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.779+0000 7efc2b71d640 1 -- 172.21.15.84:0/3094924247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc240c31e0 con 0x7efc2406a4b0 2023-10-16T10:01:30.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.780+0000 7efc2b71d640 1 -- 172.21.15.84:0/3094924247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc240c2d60 con 0x7efc2406a4b0 2023-10-16T10:01:30.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.781+0000 7efc19ffb640 1 -- 172.21.15.84:0/3094924247 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7efc2002f050 con 0x7efc2406a4b0 2023-10-16T10:01:30.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.782+0000 7efc19ffb640 1 --2- 172.21.15.84:0/3094924247 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efc0806b070 0x7efc0806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:30.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.782+0000 7efc19ffb640 1 -- 172.21.15.84:0/3094924247 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7efc200abc30 con 0x7efc2406a4b0 2023-10-16T10:01:30.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.782+0000 7efc28cb8640 1 --2- 172.21.15.84:0/3094924247 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efc0806b070 0x7efc0806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:30.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.783+0000 7efc28cb8640 1 --2- 172.21.15.84:0/3094924247 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efc0806b070 0x7efc0806d530 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efc14006fd0 tx=0x7efc14006bc0 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:30.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.786+0000 7efc19ffb640 1 -- 172.21.15.84:0/3094924247 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efc240c2d60 con 0x7efc2406a4b0 2023-10-16T10:01:30.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:30.987+0000 7efc2b71d640 1 -- 172.21.15.84:0/3094924247 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc2406b8d0 con 0x7efc0806b070 2023-10-16T10:01:31.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.000+0000 7efc19ffb640 1 -- 172.21.15.84:0/3094924247 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7efc14009280 con 0x7efc0806b070 2023-10-16T10:01:31.002 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:01:31.002 INFO:teuthology.orchestra.run.smithi084.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:30", "first": "2023-10-16T09:51:00", "last": "2023-10-16T10:01:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:31.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.002+0000 7efc2b71d640 1 -- 172.21.15.84:0/3094924247 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efc0806b070 msgr2=0x7efc0806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:31.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.002+0000 7efc2b71d640 1 --2- 172.21.15.84:0/3094924247 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efc0806b070 0x7efc0806d530 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efc14006fd0 tx=0x7efc14006bc0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.003+0000 7efc2b71d640 1 -- 172.21.15.84:0/3094924247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc2406a4b0 msgr2=0x7efc2415e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:31.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.003+0000 7efc2b71d640 1 --2- 172.21.15.84:0/3094924247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc2406a4b0 0x7efc2415e720 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efc20009870 tx=0x7efc200046c0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.003+0000 7efc2b71d640 1 -- 172.21.15.84:0/3094924247 shutdown_connections 2023-10-16T10:01:31.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.003+0000 7efc2b71d640 1 --2- 172.21.15.84:0/3094924247 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efc0806b070 0x7efc0806d530 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.003+0000 7efc2b71d640 1 --2- 172.21.15.84:0/3094924247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc240c9000 0x7efc240b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.003+0000 7efc2b71d640 1 --2- 172.21.15.84:0/3094924247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc240addb0 0x7efc240c24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.003+0000 7efc2b71d640 1 --2- 172.21.15.84:0/3094924247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc2406a4b0 0x7efc2415e720 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.003+0000 7efc2b71d640 1 -- 172.21.15.84:0/3094924247 >> 172.21.15.84:0/3094924247 conn(0x7efc240ce9b0 msgr2=0x7efc240b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:31.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.003+0000 7efc2b71d640 1 -- 172.21.15.84:0/3094924247 shutdown_connections 2023-10-16T10:01:31.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.003+0000 7efc2b71d640 1 -- 172.21.15.84:0/3094924247 wait complete. 2023-10-16T10:01:31.021 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:30", "first": "2023-10-16T09:51:00", "last": "2023-10-16T10:01:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:31.021 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:50:30', 'first': '2023-10-16T09:51:00', 'last': '2023-10-16T10:01:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T10:01:31.021 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T10:01:31.022 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T10:01:31.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:31.023 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-10-16T10:01:31.074 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T10:01:31.075 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T10:01:31.075 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_01_UTC 2023-10-16T10:01:31.075 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:31.075 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T10:01:31.076 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_01_UTC 2023-10-16T10:01:31.076 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_01_UTC 2023-10-16T10:01:31.076 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_01_UTC 2023-10-16T10:01:31.077 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_01_UTC 2023-10-16T10:01:31.077 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_01_UTC 2023-10-16T10:01:31.077 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_01_UTC 2023-10-16T10:01:31.079 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_51_00_UTC', 'scheduled-2023-10-16-09_52_00_UTC', 'scheduled-2023-10-16-09_53_01_UTC', 'scheduled-2023-10-16-09_54_00_UTC', 'scheduled-2023-10-16-09_55_00_UTC', 'scheduled-2023-10-16-09_56_01_UTC', 'scheduled-2023-10-16-09_57_01_UTC', 'scheduled-2023-10-16-09_58_01_UTC', 'scheduled-2023-10-16-09_59_01_UTC', 'scheduled-2023-10-16-10_00_01_UTC', 'scheduled-2023-10-16-10_01_01_UTC'] 2023-10-16T10:01:31.080 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir5 2023-10-16T10:01:31.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.282+0000 7f4e12494640 1 Processor -- start 2023-10-16T10:01:31.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.282+0000 7f4e12494640 1 -- start start 2023-10-16T10:01:31.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.284+0000 7f4e12494640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4e0c159ab0 0x7f4e0c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:31.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.284+0000 7f4e12494640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e0c159e90 0x7f4e0c06fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:31.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.284+0000 7f4e12494640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e0c0cd1f0 con 0x7f4e0c06c140 2023-10-16T10:01:31.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.284+0000 7f4e12494640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e0c15d890 con 0x7f4e0c159ab0 2023-10-16T10:01:31.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.284+0000 7f4e12494640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c0cd4d0 con 0x7f4e0c159e90 2023-10-16T10:01:31.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.285+0000 7f4e037fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e0c159e90 0x7f4e0c06fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:31.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.285+0000 7f4e0bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4e0c159ab0 0x7f4e0c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:31.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.285+0000 7f4e037fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e0c159e90 0x7f4e0c06fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:36754/0 (socket says 172.21.15.84:36754) 2023-10-16T10:01:31.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.285+0000 7f4e037fe640 1 -- 172.21.15.84:0/730223040 learned_addr learned my addr 172.21.15.84:0/730223040 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:31.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.285+0000 7f4e037fe640 1 -- 172.21.15.84:0/730223040 >> v2:172.21.15.157:3300/0 conn(0x7f4e0c159ab0 msgr2=0x7f4e0c0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:31.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.285+0000 7f4e037fe640 1 --2- 172.21.15.84:0/730223040 >> v2:172.21.15.157:3300/0 conn(0x7f4e0c159ab0 0x7f4e0c0ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.285+0000 7f4e037fe640 1 -- 172.21.15.84:0/730223040 >> v1:172.21.15.157:6789/0 conn(0x7f4e0c06c140 legacy=0x7f4e0c15dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:31.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.285+0000 7f4e037fe640 1 -- 172.21.15.84:0/730223040 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c0cd7b0 con 0x7f4e0c159e90 2023-10-16T10:01:31.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.285+0000 7f4e0bfff640 1 --2- 172.21.15.84:0/730223040 >> v2:172.21.15.157:3300/0 conn(0x7f4e0c159ab0 0x7f4e0c0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:01:31.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.287+0000 7f4e037fe640 1 --2- 172.21.15.84:0/730223040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e0c159e90 0x7f4e0c06fc50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4df40098a0 tx=0x7f4df402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=787376bce724cd6e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:31.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.287+0000 7f4e0b7fe640 1 -- 172.21.15.84:0/730223040 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df40317c0 con 0x7f4e0c159e90 2023-10-16T10:01:31.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.288+0000 7f4e0b7fe640 1 -- 172.21.15.84:0/730223040 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df4031920 con 0x7f4e0c159e90 2023-10-16T10:01:31.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.288+0000 7f4e0b7fe640 1 -- 172.21.15.84:0/730223040 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4031cb0 con 0x7f4e0c159e90 2023-10-16T10:01:31.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.288+0000 7f4e12494640 1 -- 172.21.15.84:0/730223040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e0c159e90 msgr2=0x7f4e0c06fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:31.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.288+0000 7f4e12494640 1 --2- 172.21.15.84:0/730223040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e0c159e90 0x7f4e0c06fc50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4df40098a0 tx=0x7f4df402b860 comp rx=0 tx=0).stop 2023-10-16T10:01:31.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.288+0000 7f4e12494640 1 -- 172.21.15.84:0/730223040 shutdown_connections 2023-10-16T10:01:31.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.288+0000 7f4e12494640 1 --2- 172.21.15.84:0/730223040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e0c159e90 0x7f4e0c06fc50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.288+0000 7f4e12494640 1 --2- 172.21.15.84:0/730223040 >> v2:172.21.15.157:3300/0 conn(0x7f4e0c159ab0 0x7f4e0c0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.288+0000 7f4e12494640 1 -- 172.21.15.84:0/730223040 >> 172.21.15.84:0/730223040 conn(0x7f4e0c0ce9b0 msgr2=0x7f4e0c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:31.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.288+0000 7f4e12494640 1 -- 172.21.15.84:0/730223040 shutdown_connections 2023-10-16T10:01:31.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.288+0000 7f4e12494640 1 -- 172.21.15.84:0/730223040 wait complete. 2023-10-16T10:01:31.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.290+0000 7f4e12494640 1 Processor -- start 2023-10-16T10:01:31.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.290+0000 7f4e12494640 1 -- start start 2023-10-16T10:01:31.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.291+0000 7f4e12494640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e0c06c140 0x7f4e0c0c4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:31.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.291+0000 7f4e12494640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e0c159ab0 0x7f4e0c14ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:31.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.291+0000 7f4e12494640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e0c159e90 0x7f4e0c0c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:31.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.291+0000 7f4e12494640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c0c3180 con 0x7f4e0c06c140 2023-10-16T10:01:31.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.291+0000 7f4e12494640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c0c3460 con 0x7f4e0c159ab0 2023-10-16T10:01:31.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.291+0000 7f4e12494640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c0c3740 con 0x7f4e0c159e90 2023-10-16T10:01:31.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.291+0000 7f4e0bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e0c06c140 0x7f4e0c0c4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:31.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.291+0000 7f4e0bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e0c06c140 0x7f4e0c0c4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57052/0 (socket says 172.21.15.84:57052) 2023-10-16T10:01:31.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.292+0000 7f4e0bfff640 1 -- 172.21.15.84:0/1793067520 learned_addr learned my addr 172.21.15.84:0/1793067520 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:31.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.292+0000 7f4e037fe640 1 --2- 172.21.15.84:0/1793067520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e0c159ab0 0x7f4e0c14ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:31.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.292+0000 7f4e0bfff640 1 -- 172.21.15.84:0/1793067520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e0c159e90 msgr2=0x7f4e0c0c24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:31.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.292+0000 7f4e10a31640 1 --2- 172.21.15.84:0/1793067520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e0c159e90 0x7f4e0c0c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:31.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.292+0000 7f4e0bfff640 1 --2- 172.21.15.84:0/1793067520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e0c159e90 0x7f4e0c0c24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.292+0000 7f4e0bfff640 1 -- 172.21.15.84:0/1793067520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e0c159ab0 msgr2=0x7f4e0c14ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:31.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.292+0000 7f4e0bfff640 1 --2- 172.21.15.84:0/1793067520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e0c159ab0 0x7f4e0c14ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.292+0000 7f4e0bfff640 1 -- 172.21.15.84:0/1793067520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc009a70 con 0x7f4e0c06c140 2023-10-16T10:01:31.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.292+0000 7f4e037fe640 1 --2- 172.21.15.84:0/1793067520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e0c159ab0 0x7f4e0c14ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:31.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.292+0000 7f4e10a31640 1 --2- 172.21.15.84:0/1793067520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e0c159e90 0x7f4e0c0c24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:31.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.292+0000 7f4e0bfff640 1 --2- 172.21.15.84:0/1793067520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e0c06c140 0x7f4e0c0c4220 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00efc0 tx=0x7f4dfc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:31.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.292+0000 7f4e097fa640 1 -- 172.21.15.84:0/1793067520 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc014680 con 0x7f4e0c06c140 2023-10-16T10:01:31.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.293+0000 7f4e097fa640 1 -- 172.21.15.84:0/1793067520 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dfc00f040 con 0x7f4e0c06c140 2023-10-16T10:01:31.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.293+0000 7f4e12494640 1 -- 172.21.15.84:0/1793067520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df40094d0 con 0x7f4e0c06c140 2023-10-16T10:01:31.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.293+0000 7f4e097fa640 1 -- 172.21.15.84:0/1793067520 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc014a00 con 0x7f4e0c06c140 2023-10-16T10:01:31.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.293+0000 7f4e12494640 1 -- 172.21.15.84:0/1793067520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0c0c3d00 con 0x7f4e0c06c140 2023-10-16T10:01:31.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.294+0000 7f4e12494640 1 -- 172.21.15.84:0/1793067520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e0c04e160 con 0x7f4e0c06c140 2023-10-16T10:01:31.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.295+0000 7f4e097fa640 1 -- 172.21.15.84:0/1793067520 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f4dfc014b60 con 0x7f4e0c06c140 2023-10-16T10:01:31.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.296+0000 7f4e097fa640 1 --2- 172.21.15.84:0/1793067520 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f4dec06b070 0x7f4dec06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:31.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.296+0000 7f4e097fa640 1 -- 172.21.15.84:0/1793067520 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f4dfc08ab70 con 0x7f4e0c06c140 2023-10-16T10:01:31.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.296+0000 7f4e037fe640 1 --2- 172.21.15.84:0/1793067520 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f4dec06b070 0x7f4dec06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:31.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.297+0000 7f4e037fe640 1 --2- 172.21.15.84:0/1793067520 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f4dec06b070 0x7f4dec06d530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4df4036000 tx=0x7f4df403d040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:31.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.300+0000 7f4e097fa640 1 -- 172.21.15.84:0/1793067520 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4dfc055790 con 0x7f4e0c06c140 2023-10-16T10:01:31.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.511+0000 7f4e12494640 1 -- 172.21.15.84:0/1793067520 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4e0c0adf70 con 0x7f4dec06b070 2023-10-16T10:01:31.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.528+0000 7f4e097fa640 1 -- 172.21.15.84:0/1793067520 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f4df4031b90 con 0x7f4dec06b070 2023-10-16T10:01:31.530 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:01:31.530 INFO:teuthology.orchestra.run.smithi084.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:31", "first": "2023-10-16T09:51:00", "last": "2023-10-16T10:01:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:31.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.530+0000 7f4e12494640 1 -- 172.21.15.84:0/1793067520 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f4dec06b070 msgr2=0x7f4dec06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:31.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.530+0000 7f4e12494640 1 --2- 172.21.15.84:0/1793067520 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f4dec06b070 0x7f4dec06d530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4df4036000 tx=0x7f4df403d040 comp rx=0 tx=0).stop 2023-10-16T10:01:31.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.530+0000 7f4e12494640 1 -- 172.21.15.84:0/1793067520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e0c06c140 msgr2=0x7f4e0c0c4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:31.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.530+0000 7f4e12494640 1 --2- 172.21.15.84:0/1793067520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e0c06c140 0x7f4e0c0c4220 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00efc0 tx=0x7f4dfc00c490 comp rx=0 tx=0).stop 2023-10-16T10:01:31.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.531+0000 7f4e12494640 1 -- 172.21.15.84:0/1793067520 shutdown_connections 2023-10-16T10:01:31.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.531+0000 7f4e12494640 1 --2- 172.21.15.84:0/1793067520 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f4dec06b070 0x7f4dec06d530 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.531+0000 7f4e12494640 1 --2- 172.21.15.84:0/1793067520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e0c159e90 0x7f4e0c0c24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.531+0000 7f4e12494640 1 --2- 172.21.15.84:0/1793067520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e0c159ab0 0x7f4e0c14ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.531+0000 7f4e12494640 1 --2- 172.21.15.84:0/1793067520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e0c06c140 0x7f4e0c0c4220 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.531+0000 7f4e12494640 1 -- 172.21.15.84:0/1793067520 >> 172.21.15.84:0/1793067520 conn(0x7f4e0c0ce9b0 msgr2=0x7f4e0c070480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:31.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.531+0000 7f4e12494640 1 -- 172.21.15.84:0/1793067520 shutdown_connections 2023-10-16T10:01:31.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.531+0000 7f4e12494640 1 -- 172.21.15.84:0/1793067520 wait complete. 2023-10-16T10:01:31.551 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:31", "first": "2023-10-16T09:51:00", "last": "2023-10-16T10:01:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:31.551 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:50:31', 'first': '2023-10-16T09:51:00', 'last': '2023-10-16T10:01:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T10:01:31.552 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T10:01:31.552 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T10:01:31.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:31.553 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-10-16T10:01:31.607 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_01_UTC 2023-10-16T10:01:31.607 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_01_UTC 2023-10-16T10:01:31.607 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T10:01:31.608 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:31.608 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_01_UTC 2023-10-16T10:01:31.608 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:31.608 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_00_UTC 2023-10-16T10:01:31.609 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T10:01:31.609 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_00_UTC 2023-10-16T10:01:31.609 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_00_UTC 2023-10-16T10:01:31.609 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:31.610 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_51_01_UTC', 'scheduled-2023-10-16-09_52_01_UTC', 'scheduled-2023-10-16-09_53_00_UTC', 'scheduled-2023-10-16-09_54_00_UTC', 'scheduled-2023-10-16-09_55_01_UTC', 'scheduled-2023-10-16-09_56_00_UTC', 'scheduled-2023-10-16-09_57_00_UTC', 'scheduled-2023-10-16-09_58_00_UTC', 'scheduled-2023-10-16-09_59_00_UTC', 'scheduled-2023-10-16-10_00_00_UTC', 'scheduled-2023-10-16-10_01_00_UTC'] 2023-10-16T10:01:31.611 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir6 2023-10-16T10:01:31.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.816+0000 7f6f8b516640 1 Processor -- start 2023-10-16T10:01:31.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.816+0000 7f6f8b516640 1 -- start start 2023-10-16T10:01:31.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.818+0000 7f6f8b516640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6f840caa80 0x7f6f84069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:31.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.818+0000 7f6f8b516640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f840caef0 con 0x7f6f840addb0 2023-10-16T10:01:31.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.818+0000 7f6f8b516640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f8406a6f0 con 0x7f6f8406a2e0 2023-10-16T10:01:31.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.818+0000 7f6f8b516640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f84161a50 con 0x7f6f840caa80 2023-10-16T10:01:31.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.818+0000 7f6f892b2640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6f840addb0 0x7f6f840c9000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39890/0 (socket says 172.21.15.84:39890) 2023-10-16T10:01:31.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.818+0000 7f6f892b2640 1 -- 172.21.15.84:0/3547173465 learned_addr learned my addr 172.21.15.84:0/3547173465 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:31.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.819+0000 7f6f88ab1640 1 --2- 172.21.15.84:0/3547173465 >> v2:172.21.15.157:3300/0 conn(0x7f6f840caa80 0x7f6f84069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:31.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.819+0000 7f6f7bfff640 1 -- 172.21.15.84:0/3547173465 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708783319 0 0) 0x7f6f840caef0 con 0x7f6f840addb0 2023-10-16T10:01:31.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.819+0000 7f6f7bfff640 1 -- 172.21.15.84:0/3547173465 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f70003520 con 0x7f6f840addb0 2023-10-16T10:01:31.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.819+0000 7f6f7bfff640 1 -- 172.21.15.84:0/3547173465 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2258501960 0 0) 0x7f6f70003520 con 0x7f6f840addb0 2023-10-16T10:01:31.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.819+0000 7f6f7bfff640 1 -- 172.21.15.84:0/3547173465 >> v2:172.21.15.157:3300/0 conn(0x7f6f840caa80 msgr2=0x7f6f84069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:31.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.819+0000 7f6f7bfff640 1 --2- 172.21.15.84:0/3547173465 >> v2:172.21.15.157:3300/0 conn(0x7f6f840caa80 0x7f6f84069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.819+0000 7f6f7bfff640 1 -- 172.21.15.84:0/3547173465 >> v1:172.21.15.157:6789/0 conn(0x7f6f8406a2e0 legacy=0x7f6f841650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:31.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.819+0000 7f6f7bfff640 1 -- 172.21.15.84:0/3547173465 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f840cb140 con 0x7f6f840addb0 2023-10-16T10:01:31.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.819+0000 7f6f7bfff640 1 -- 172.21.15.84:0/3547173465 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f6f80001670 con 0x7f6f840addb0 2023-10-16T10:01:31.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.819+0000 7f6f7bfff640 1 -- 172.21.15.84:0/3547173465 >> v1:172.21.15.84:6789/0 conn(0x7f6f840addb0 legacy=0x7f6f840c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:31.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.819+0000 7f6f7bfff640 1 --2- 172.21.15.84:0/3547173465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f70003970 0x7f6f70003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:31.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.819+0000 7f6f7bfff640 1 --2- 172.21.15.84:0/3547173465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f700047c0 0x7f6f70004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:31.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.819+0000 7f6f7bfff640 1 --2- 172.21.15.84:0/3547173465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f70005180 0x7f6f7000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:31.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.819+0000 7f6f7bfff640 1 -- 172.21.15.84:0/3547173465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f84161a50 con 0x7f6f70005180 2023-10-16T10:01:31.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.819+0000 7f6f7bfff640 1 -- 172.21.15.84:0/3547173465 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7000c020 con 0x7f6f70003970 2023-10-16T10:01:31.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.819+0000 7f6f7bfff640 1 -- 172.21.15.84:0/3547173465 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7000c1e0 con 0x7f6f700047c0 2023-10-16T10:01:31.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.820+0000 7f6f89ab3640 1 --2- 172.21.15.84:0/3547173465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f70005180 0x7f6f7000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:31.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.820+0000 7f6f892b2640 1 --2- 172.21.15.84:0/3547173465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f70003970 0x7f6f70003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:31.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.820+0000 7f6f88ab1640 1 --2- 172.21.15.84:0/3547173465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f700047c0 0x7f6f70004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:31.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.820+0000 7f6f89ab3640 1 -- 172.21.15.84:0/3547173465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f700047c0 msgr2=0x7f6f70004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:31.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.820+0000 7f6f89ab3640 1 --2- 172.21.15.84:0/3547173465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f700047c0 0x7f6f70004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.820+0000 7f6f89ab3640 1 -- 172.21.15.84:0/3547173465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f70003970 msgr2=0x7f6f70003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:31.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.820+0000 7f6f89ab3640 1 --2- 172.21.15.84:0/3547173465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f70003970 0x7f6f70003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.820+0000 7f6f89ab3640 1 -- 172.21.15.84:0/3547173465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f70003520 con 0x7f6f70005180 2023-10-16T10:01:31.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.820+0000 7f6f88ab1640 1 --2- 172.21.15.84:0/3547173465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f700047c0 0x7f6f70004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:31.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.820+0000 7f6f892b2640 1 --2- 172.21.15.84:0/3547173465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f70003970 0x7f6f70003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:31.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.821+0000 7f6f89ab3640 1 -- 172.21.15.84:0/3547173465 reap_dead start 2023-10-16T10:01:31.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.821+0000 7f6f89ab3640 1 --2- 172.21.15.84:0/3547173465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f70005180 0x7f6f7000b980 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c001e50 tx=0x7f6f700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbeac2a393794f6b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:31.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.821+0000 7f6f7bfff640 1 -- 172.21.15.84:0/3547173465 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c003b70 con 0x7f6f70005180 2023-10-16T10:01:31.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.821+0000 7f6f7bfff640 1 -- 172.21.15.84:0/3547173465 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7c003d10 con 0x7f6f70005180 2023-10-16T10:01:31.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.821+0000 7f6f7bfff640 1 -- 172.21.15.84:0/3547173465 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c005310 con 0x7f6f70005180 2023-10-16T10:01:31.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.822+0000 7f6f8b516640 1 -- 172.21.15.84:0/3547173465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f70005180 msgr2=0x7f6f7000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:31.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.822+0000 7f6f8b516640 1 --2- 172.21.15.84:0/3547173465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f70005180 0x7f6f7000b980 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c001e50 tx=0x7f6f700057f0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.822+0000 7f6f8b516640 1 -- 172.21.15.84:0/3547173465 shutdown_connections 2023-10-16T10:01:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.822+0000 7f6f8b516640 1 --2- 172.21.15.84:0/3547173465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f70005180 0x7f6f7000b980 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.822+0000 7f6f8b516640 1 -- 172.21.15.84:0/3547173465 >> 172.21.15.84:0/3547173465 conn(0x7f6f840ce9b0 msgr2=0x7f6f840cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.822+0000 7f6f8b516640 1 -- 172.21.15.84:0/3547173465 shutdown_connections 2023-10-16T10:01:31.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.822+0000 7f6f8b516640 1 -- 172.21.15.84:0/3547173465 wait complete. 2023-10-16T10:01:31.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.824+0000 7f6f8b516640 1 Processor -- start 2023-10-16T10:01:31.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.824+0000 7f6f8b516640 1 -- start start 2023-10-16T10:01:31.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.825+0000 7f6f8b516640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f70005180 0x7f6f84164780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:31.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.825+0000 7f6f8b516640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f840c6320 0x7f6f840c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:31.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.825+0000 7f6f8b516640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f840c6780 0x7f6f840c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:31.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.825+0000 7f6f8b516640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f840c88e0 con 0x7f6f70005180 2023-10-16T10:01:31.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.825+0000 7f6f8b516640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f840aef10 con 0x7f6f840c6320 2023-10-16T10:01:31.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.825+0000 7f6f8b516640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f840af1f0 con 0x7f6f840c6780 2023-10-16T10:01:31.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.825+0000 7f6f892b2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f70005180 0x7f6f84164780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:31.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.825+0000 7f6f892b2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f70005180 0x7f6f84164780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57068/0 (socket says 172.21.15.84:57068) 2023-10-16T10:01:31.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.825+0000 7f6f892b2640 1 -- 172.21.15.84:0/53399840 learned_addr learned my addr 172.21.15.84:0/53399840 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:31.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.825+0000 7f6f892b2640 1 -- 172.21.15.84:0/53399840 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f840c6780 msgr2=0x7f6f840c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:31.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.825+0000 7f6f892b2640 1 --2- 172.21.15.84:0/53399840 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f840c6780 0x7f6f840c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.825+0000 7f6f892b2640 1 -- 172.21.15.84:0/53399840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f840c6320 msgr2=0x7f6f840c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:31.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.825+0000 7f6f892b2640 1 --2- 172.21.15.84:0/53399840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f840c6320 0x7f6f840c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:31.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.825+0000 7f6f892b2640 1 -- 172.21.15.84:0/53399840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f840af4d0 con 0x7f6f70005180 2023-10-16T10:01:31.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.826+0000 7f6f892b2640 1 --2- 172.21.15.84:0/53399840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f70005180 0x7f6f84164780 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6f80001200 tx=0x7f6f8000dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:31.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.826+0000 7f6f79ffb640 1 -- 172.21.15.84:0/53399840 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f800074e0 con 0x7f6f70005180 2023-10-16T10:01:31.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.826+0000 7f6f79ffb640 1 -- 172.21.15.84:0/53399840 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f80007640 con 0x7f6f70005180 2023-10-16T10:01:31.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.826+0000 7f6f79ffb640 1 -- 172.21.15.84:0/53399840 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f800049e0 con 0x7f6f70005180 2023-10-16T10:01:31.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.826+0000 7f6f8b516640 1 -- 172.21.15.84:0/53399840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f840af7b0 con 0x7f6f70005180 2023-10-16T10:01:31.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.826+0000 7f6f8b516640 1 -- 172.21.15.84:0/53399840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f840aebf0 con 0x7f6f70005180 2023-10-16T10:01:31.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.827+0000 7f6f8b516640 1 -- 172.21.15.84:0/53399840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f8404e160 con 0x7f6f70005180 2023-10-16T10:01:31.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.828+0000 7f6f79ffb640 1 -- 172.21.15.84:0/53399840 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f6f80004b40 con 0x7f6f70005180 2023-10-16T10:01:31.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.829+0000 7f6f79ffb640 1 --2- 172.21.15.84:0/53399840 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f6f6806b0a0 0x7f6f6806d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:31.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.829+0000 7f6f79ffb640 1 -- 172.21.15.84:0/53399840 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f6f800894b0 con 0x7f6f70005180 2023-10-16T10:01:31.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.829+0000 7f6f88ab1640 1 --2- 172.21.15.84:0/53399840 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f6f6806b0a0 0x7f6f6806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:31.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.830+0000 7f6f88ab1640 1 --2- 172.21.15.84:0/53399840 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f6f6806b0a0 0x7f6f6806d560 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6f8406f9c0 tx=0x7f6f74008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:31.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:31.833+0000 7f6f79ffb640 1 -- 172.21.15.84:0/53399840 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6f800540d0 con 0x7f6f70005180 2023-10-16T10:01:32.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.036+0000 7f6f8b516640 1 -- 172.21.15.84:0/53399840 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f8406b310 con 0x7f6f6806b0a0 2023-10-16T10:01:32.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.049+0000 7f6f79ffb640 1 -- 172.21.15.84:0/53399840 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f6f74009650 con 0x7f6f6806b0a0 2023-10-16T10:01:32.051 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:01:32.051 INFO:teuthology.orchestra.run.smithi084.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:32", "first": "2023-10-16T09:51:01", "last": "2023-10-16T10:01:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:32.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.052+0000 7f6f8b516640 1 -- 172.21.15.84:0/53399840 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f6f6806b0a0 msgr2=0x7f6f6806d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:32.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.052+0000 7f6f8b516640 1 --2- 172.21.15.84:0/53399840 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f6f6806b0a0 0x7f6f6806d560 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6f8406f9c0 tx=0x7f6f74008040 comp rx=0 tx=0).stop 2023-10-16T10:01:32.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.052+0000 7f6f8b516640 1 -- 172.21.15.84:0/53399840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f70005180 msgr2=0x7f6f84164780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:32.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.052+0000 7f6f8b516640 1 --2- 172.21.15.84:0/53399840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f70005180 0x7f6f84164780 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6f80001200 tx=0x7f6f8000dc70 comp rx=0 tx=0).stop 2023-10-16T10:01:32.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.052+0000 7f6f8b516640 1 -- 172.21.15.84:0/53399840 shutdown_connections 2023-10-16T10:01:32.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.052+0000 7f6f8b516640 1 --2- 172.21.15.84:0/53399840 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f6f6806b0a0 0x7f6f6806d560 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.052+0000 7f6f8b516640 1 --2- 172.21.15.84:0/53399840 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f840c6780 0x7f6f840c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.052+0000 7f6f8b516640 1 --2- 172.21.15.84:0/53399840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f840c6320 0x7f6f840c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.052+0000 7f6f8b516640 1 --2- 172.21.15.84:0/53399840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f70005180 0x7f6f84164780 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.052+0000 7f6f8b516640 1 -- 172.21.15.84:0/53399840 >> 172.21.15.84:0/53399840 conn(0x7f6f840ce9b0 msgr2=0x7f6f840b0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:32.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.052+0000 7f6f8b516640 1 -- 172.21.15.84:0/53399840 shutdown_connections 2023-10-16T10:01:32.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.052+0000 7f6f8b516640 1 -- 172.21.15.84:0/53399840 wait complete. 2023-10-16T10:01:32.072 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:32", "first": "2023-10-16T09:51:01", "last": "2023-10-16T10:01:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:32.073 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:50:32', 'first': '2023-10-16T09:51:01', 'last': '2023-10-16T10:01:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T10:01:32.073 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T10:01:32.073 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T10:01:32.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:32.074 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-10-16T10:01:32.127 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T10:01:32.128 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_01_UTC 2023-10-16T10:01:32.128 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T10:01:32.128 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:32.128 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_01_UTC 2023-10-16T10:01:32.129 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_01_UTC 2023-10-16T10:01:32.129 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_01_UTC 2023-10-16T10:01:32.129 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T10:01:32.129 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_00_UTC 2023-10-16T10:01:32.129 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_00_UTC 2023-10-16T10:01:32.129 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_01_UTC 2023-10-16T10:01:32.131 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_51_00_UTC', 'scheduled-2023-10-16-09_52_01_UTC', 'scheduled-2023-10-16-09_53_00_UTC', 'scheduled-2023-10-16-09_54_00_UTC', 'scheduled-2023-10-16-09_55_01_UTC', 'scheduled-2023-10-16-09_56_01_UTC', 'scheduled-2023-10-16-09_57_01_UTC', 'scheduled-2023-10-16-09_58_00_UTC', 'scheduled-2023-10-16-09_59_00_UTC', 'scheduled-2023-10-16-10_00_00_UTC', 'scheduled-2023-10-16-10_01_01_UTC'] 2023-10-16T10:01:32.131 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir7 2023-10-16T10:01:32.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.345+0000 7f1493051640 1 Processor -- start 2023-10-16T10:01:32.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.345+0000 7f1493051640 1 -- start start 2023-10-16T10:01:32.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.347+0000 7f1493051640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f148c06b660 0x7f148c06ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:32.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.347+0000 7f1493051640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f148c1428c0 0x7f148c0c9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:32.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.347+0000 7f1493051640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f148c0c9690 0x7f148c15d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:32.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.347+0000 7f1493051640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148c0c9b20 con 0x7f148c1428c0 2023-10-16T10:01:32.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.347+0000 7f1493051640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148c0cc6b0 con 0x7f148c06b660 2023-10-16T10:01:32.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.347+0000 7f1493051640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148c1599b0 con 0x7f148c0c9690 2023-10-16T10:01:32.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.347+0000 7f148bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f148c1428c0 0x7f148c0c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:32.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.347+0000 7f148bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f148c1428c0 0x7f148c0c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57072/0 (socket says 172.21.15.84:57072) 2023-10-16T10:01:32.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.347+0000 7f148bfff640 1 -- 172.21.15.84:0/1030622677 learned_addr learned my addr 172.21.15.84:0/1030622677 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:32.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.348+0000 7f148bfff640 1 -- 172.21.15.84:0/1030622677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f148c0c9690 msgr2=0x7f148c15d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:32.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.348+0000 7f148bfff640 1 --2- 172.21.15.84:0/1030622677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f148c0c9690 0x7f148c15d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.348+0000 7f148bfff640 1 -- 172.21.15.84:0/1030622677 >> v2:172.21.15.157:3300/0 conn(0x7f148c06b660 msgr2=0x7f148c06ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:32.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.348+0000 7f148bfff640 1 --2- 172.21.15.84:0/1030622677 >> v2:172.21.15.157:3300/0 conn(0x7f148c06b660 0x7f148c06ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.348+0000 7f148bfff640 1 -- 172.21.15.84:0/1030622677 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c06c290 con 0x7f148c1428c0 2023-10-16T10:01:32.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.349+0000 7f148bfff640 1 --2- 172.21.15.84:0/1030622677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f148c1428c0 0x7f148c0c9150 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f148c0c9120 tx=0x7f147802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4405a3868c771db server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:32.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.349+0000 7f148b7fe640 1 -- 172.21.15.84:0/1030622677 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478031870 con 0x7f148c1428c0 2023-10-16T10:01:32.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.349+0000 7f148b7fe640 1 -- 172.21.15.84:0/1030622677 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14780319d0 con 0x7f148c1428c0 2023-10-16T10:01:32.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.349+0000 7f148b7fe640 1 -- 172.21.15.84:0/1030622677 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478031d60 con 0x7f148c1428c0 2023-10-16T10:01:32.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.349+0000 7f1493051640 1 -- 172.21.15.84:0/1030622677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f148c1428c0 msgr2=0x7f148c0c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:32.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.349+0000 7f1493051640 1 --2- 172.21.15.84:0/1030622677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f148c1428c0 0x7f148c0c9150 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f148c0c9120 tx=0x7f147802b910 comp rx=0 tx=0).stop 2023-10-16T10:01:32.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.350+0000 7f1493051640 1 -- 172.21.15.84:0/1030622677 shutdown_connections 2023-10-16T10:01:32.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.350+0000 7f1493051640 1 --2- 172.21.15.84:0/1030622677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f148c1428c0 0x7f148c0c9150 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.350+0000 7f1493051640 1 --2- 172.21.15.84:0/1030622677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f148c0c9690 0x7f148c15d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.350+0000 7f1493051640 1 --2- 172.21.15.84:0/1030622677 >> v2:172.21.15.157:3300/0 conn(0x7f148c06b660 0x7f148c06ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.350+0000 7f1493051640 1 -- 172.21.15.84:0/1030622677 >> 172.21.15.84:0/1030622677 conn(0x7f148c0ce9b0 msgr2=0x7f148c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:32.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.350+0000 7f1493051640 1 -- 172.21.15.84:0/1030622677 shutdown_connections 2023-10-16T10:01:32.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.350+0000 7f1493051640 1 -- 172.21.15.84:0/1030622677 wait complete. 2023-10-16T10:01:32.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.352+0000 7f1493051640 1 Processor -- start 2023-10-16T10:01:32.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.352+0000 7f1493051640 1 -- start start 2023-10-16T10:01:32.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.353+0000 7f1493051640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f148c06b660 0x7f148c0c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:32.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.353+0000 7f1493051640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f148c0c9690 0x7f148c0c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:32.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.353+0000 7f1493051640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f148c1428c0 0x7f148c0c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:32.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.353+0000 7f1493051640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c0c3730 con 0x7f148c06b660 2023-10-16T10:01:32.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.353+0000 7f1493051640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c0c7990 con 0x7f148c0c9690 2023-10-16T10:01:32.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.353+0000 7f1493051640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148c0c6d80 con 0x7f148c1428c0 2023-10-16T10:01:32.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.353+0000 7f1490ded640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f148c06b660 0x7f148c0c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:32.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.353+0000 7f1490ded640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f148c06b660 0x7f148c0c67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57086/0 (socket says 172.21.15.84:57086) 2023-10-16T10:01:32.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.353+0000 7f1490ded640 1 -- 172.21.15.84:0/2872513107 learned_addr learned my addr 172.21.15.84:0/2872513107 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:32.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.353+0000 7f1490ded640 1 -- 172.21.15.84:0/2872513107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f148c1428c0 msgr2=0x7f148c0c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:32.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.353+0000 7f1490ded640 1 --2- 172.21.15.84:0/2872513107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f148c1428c0 0x7f148c0c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.353+0000 7f1490ded640 1 -- 172.21.15.84:0/2872513107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f148c0c9690 msgr2=0x7f148c0c31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:32.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.353+0000 7f148bfff640 1 --2- 172.21.15.84:0/2872513107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f148c0c9690 0x7f148c0c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:32.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.353+0000 7f1490ded640 1 --2- 172.21.15.84:0/2872513107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f148c0c9690 0x7f148c0c31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.353+0000 7f1490ded640 1 -- 172.21.15.84:0/2872513107 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14780094d0 con 0x7f148c06b660 2023-10-16T10:01:32.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.354+0000 7f1490ded640 1 --2- 172.21.15.84:0/2872513107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f148c06b660 0x7f148c0c67a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f148000b410 tx=0x7f148000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:32.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.354+0000 7f14897fa640 1 -- 172.21.15.84:0/2872513107 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480011830 con 0x7f148c06b660 2023-10-16T10:01:32.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.354+0000 7f14897fa640 1 -- 172.21.15.84:0/2872513107 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14800119b0 con 0x7f148c06b660 2023-10-16T10:01:32.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.354+0000 7f14897fa640 1 -- 172.21.15.84:0/2872513107 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480011d60 con 0x7f148c06b660 2023-10-16T10:01:32.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.354+0000 7f1493051640 1 -- 172.21.15.84:0/2872513107 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148c0c7060 con 0x7f148c06b660 2023-10-16T10:01:32.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.354+0000 7f1493051640 1 -- 172.21.15.84:0/2872513107 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148c0c7620 con 0x7f148c06b660 2023-10-16T10:01:32.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.355+0000 7f1493051640 1 -- 172.21.15.84:0/2872513107 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148c04e160 con 0x7f148c06b660 2023-10-16T10:01:32.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.356+0000 7f14897fa640 1 -- 172.21.15.84:0/2872513107 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f148001a400 con 0x7f148c06b660 2023-10-16T10:01:32.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.357+0000 7f14897fa640 1 --2- 172.21.15.84:0/2872513107 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f147006b070 0x7f147006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:32.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.357+0000 7f14897fa640 1 -- 172.21.15.84:0/2872513107 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f1480089c20 con 0x7f148c06b660 2023-10-16T10:01:32.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.357+0000 7f148bfff640 1 --2- 172.21.15.84:0/2872513107 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f147006b070 0x7f147006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:32.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.358+0000 7f148bfff640 1 --2- 172.21.15.84:0/2872513107 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f147006b070 0x7f147006d530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1478036000 tx=0x7f1478008030 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:32.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.361+0000 7f14897fa640 1 -- 172.21.15.84:0/2872513107 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1480054950 con 0x7f148c06b660 2023-10-16T10:01:32.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.572+0000 7f1493051640 1 -- 172.21.15.84:0/2872513107 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f148c06c850 con 0x7f147006b070 2023-10-16T10:01:32.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.588+0000 7f14897fa640 1 -- 172.21.15.84:0/2872513107 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f1478031c90 con 0x7f147006b070 2023-10-16T10:01:32.590 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:01:32.590 INFO:teuthology.orchestra.run.smithi084.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:32", "first": "2023-10-16T09:51:00", "last": "2023-10-16T10:01:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:32.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.590+0000 7f1493051640 1 -- 172.21.15.84:0/2872513107 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f147006b070 msgr2=0x7f147006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:32.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.590+0000 7f1493051640 1 --2- 172.21.15.84:0/2872513107 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f147006b070 0x7f147006d530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1478036000 tx=0x7f1478008030 comp rx=0 tx=0).stop 2023-10-16T10:01:32.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.590+0000 7f1493051640 1 -- 172.21.15.84:0/2872513107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f148c06b660 msgr2=0x7f148c0c67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:32.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.590+0000 7f1493051640 1 --2- 172.21.15.84:0/2872513107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f148c06b660 0x7f148c0c67a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f148000b410 tx=0x7f148000b8d0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.590+0000 7f1493051640 1 -- 172.21.15.84:0/2872513107 shutdown_connections 2023-10-16T10:01:32.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.590+0000 7f1493051640 1 --2- 172.21.15.84:0/2872513107 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f147006b070 0x7f147006d530 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.590+0000 7f1493051640 1 --2- 172.21.15.84:0/2872513107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f148c1428c0 0x7f148c0c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.590+0000 7f1493051640 1 --2- 172.21.15.84:0/2872513107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f148c0c9690 0x7f148c0c31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.590+0000 7f1493051640 1 --2- 172.21.15.84:0/2872513107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f148c06b660 0x7f148c0c67a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.590+0000 7f1493051640 1 -- 172.21.15.84:0/2872513107 >> 172.21.15.84:0/2872513107 conn(0x7f148c0ce9b0 msgr2=0x7f148c069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:32.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.591+0000 7f1493051640 1 -- 172.21.15.84:0/2872513107 shutdown_connections 2023-10-16T10:01:32.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.591+0000 7f1493051640 1 -- 172.21.15.84:0/2872513107 wait complete. 2023-10-16T10:01:32.611 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:01:32.612 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:32", "first": "2023-10-16T09:51:00", "last": "2023-10-16T10:01:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:32.613 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:50:32', 'first': '2023-10-16T09:51:00', 'last': '2023-10-16T10:01:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T10:01:32.613 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T10:01:32.613 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T10:01:32.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:32.614 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-10-16T10:01:32.660 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:01:32.708 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_01_UTC 2023-10-16T10:01:32.708 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_01_UTC 2023-10-16T10:01:32.708 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_01_UTC 2023-10-16T10:01:32.709 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_01_UTC 2023-10-16T10:01:32.709 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_01_UTC 2023-10-16T10:01:32.709 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:32.709 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_00_UTC 2023-10-16T10:01:32.710 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T10:01:32.710 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_00_UTC 2023-10-16T10:01:32.710 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_01_UTC 2023-10-16T10:01:32.710 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:32.711 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_51_01_UTC', 'scheduled-2023-10-16-09_52_01_UTC', 'scheduled-2023-10-16-09_53_01_UTC', 'scheduled-2023-10-16-09_54_01_UTC', 'scheduled-2023-10-16-09_55_01_UTC', 'scheduled-2023-10-16-09_56_00_UTC', 'scheduled-2023-10-16-09_57_00_UTC', 'scheduled-2023-10-16-09_58_00_UTC', 'scheduled-2023-10-16-09_59_00_UTC', 'scheduled-2023-10-16-10_00_01_UTC', 'scheduled-2023-10-16-10_01_00_UTC'] 2023-10-16T10:01:32.712 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir8 2023-10-16T10:01:32.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.972+0000 7fe61aeaf640 1 Processor -- start 2023-10-16T10:01:32.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.972+0000 7fe61aeaf640 1 -- start start 2023-10-16T10:01:32.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.974+0000 7fe61aeaf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe614159e90 0x7fe6140ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:32.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.974+0000 7fe61aeaf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe61406c140 0x7fe61406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:32.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.974+0000 7fe61aeaf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe614070360 con 0x7fe614159ab0 2023-10-16T10:01:32.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.974+0000 7fe61aeaf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe61406cb10 con 0x7fe61406c140 2023-10-16T10:01:32.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.974+0000 7fe61aeaf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61415d890 con 0x7fe614159e90 2023-10-16T10:01:32.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.975+0000 7fe613fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe614159e90 0x7fe6140ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:32.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.975+0000 7fe618c4b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe614159ab0 0x7fe61406fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:43390/0 (socket says 172.21.15.84:43390) 2023-10-16T10:01:32.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.975+0000 7fe618c4b640 1 -- 172.21.15.84:0/2441723941 learned_addr learned my addr 172.21.15.84:0/2441723941 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:32.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.975+0000 7fe61944c640 1 --2- 172.21.15.84:0/2441723941 >> v2:172.21.15.157:3300/0 conn(0x7fe61406c140 0x7fe61406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:32.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.976+0000 7fe6137fe640 1 -- 172.21.15.84:0/2441723941 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390142582 0 0) 0x7fe614070360 con 0x7fe614159ab0 2023-10-16T10:01:32.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.976+0000 7fe6137fe640 1 -- 172.21.15.84:0/2441723941 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe604003520 con 0x7fe614159ab0 2023-10-16T10:01:32.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.976+0000 7fe61944c640 1 -- 172.21.15.84:0/2441723941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe614159e90 msgr2=0x7fe6140ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:32.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.976+0000 7fe61944c640 1 --2- 172.21.15.84:0/2441723941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe614159e90 0x7fe6140ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.976+0000 7fe61944c640 1 -- 172.21.15.84:0/2441723941 >> v1:172.21.15.157:6789/0 conn(0x7fe614159ab0 legacy=0x7fe61406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:32.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.976+0000 7fe61944c640 1 -- 172.21.15.84:0/2441723941 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6140cd1f0 con 0x7fe61406c140 2023-10-16T10:01:32.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.977+0000 7fe61944c640 1 --2- 172.21.15.84:0/2441723941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe61406c140 0x7fe61406c5d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009dc0 tx=0x7fe60c02bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae71ada698bb25c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:32.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.978+0000 7fe6137fe640 1 -- 172.21.15.84:0/2441723941 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c02cb10 con 0x7fe61406c140 2023-10-16T10:01:32.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.978+0000 7fe6137fe640 1 -- 172.21.15.84:0/2441723941 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60c02cc70 con 0x7fe61406c140 2023-10-16T10:01:32.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.978+0000 7fe6137fe640 1 -- 172.21.15.84:0/2441723941 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c034440 con 0x7fe61406c140 2023-10-16T10:01:32.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.978+0000 7fe61aeaf640 1 -- 172.21.15.84:0/2441723941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe61406c140 msgr2=0x7fe61406c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:32.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.978+0000 7fe61aeaf640 1 --2- 172.21.15.84:0/2441723941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe61406c140 0x7fe61406c5d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009dc0 tx=0x7fe60c02bd60 comp rx=0 tx=0).stop 2023-10-16T10:01:32.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.978+0000 7fe61aeaf640 1 -- 172.21.15.84:0/2441723941 shutdown_connections 2023-10-16T10:01:32.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.978+0000 7fe61aeaf640 1 --2- 172.21.15.84:0/2441723941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe61406c140 0x7fe61406c5d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.978+0000 7fe61aeaf640 1 --2- 172.21.15.84:0/2441723941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe614159e90 0x7fe6140ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.978+0000 7fe61aeaf640 1 -- 172.21.15.84:0/2441723941 >> 172.21.15.84:0/2441723941 conn(0x7fe6140ce9b0 msgr2=0x7fe6140cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:32.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.979+0000 7fe61aeaf640 1 -- 172.21.15.84:0/2441723941 shutdown_connections 2023-10-16T10:01:32.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.979+0000 7fe61aeaf640 1 -- 172.21.15.84:0/2441723941 wait complete. 2023-10-16T10:01:32.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.980+0000 7fe61aeaf640 1 Processor -- start 2023-10-16T10:01:32.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.980+0000 7fe61aeaf640 1 -- start start 2023-10-16T10:01:32.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.981+0000 7fe61aeaf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe61406c140 0x7fe614158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:32.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.981+0000 7fe61aeaf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe614159ab0 0x7fe6140c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:32.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.981+0000 7fe61aeaf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe614159e90 0x7fe6140acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:32.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.981+0000 7fe61aeaf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140c6a90 con 0x7fe614159e90 2023-10-16T10:01:32.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.981+0000 7fe61aeaf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140c7a90 con 0x7fe61406c140 2023-10-16T10:01:32.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.981+0000 7fe61aeaf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140c8610 con 0x7fe614159ab0 2023-10-16T10:01:32.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.982+0000 7fe61944c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe614159e90 0x7fe6140acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:32.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.982+0000 7fe61944c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe614159e90 0x7fe6140acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57092/0 (socket says 172.21.15.84:57092) 2023-10-16T10:01:32.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.982+0000 7fe61944c640 1 -- 172.21.15.84:0/3901014578 learned_addr learned my addr 172.21.15.84:0/3901014578 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:32.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.982+0000 7fe618c4b640 1 --2- 172.21.15.84:0/3901014578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe61406c140 0x7fe614158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:32.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.982+0000 7fe613fff640 1 --2- 172.21.15.84:0/3901014578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe614159ab0 0x7fe6140c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:32.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.982+0000 7fe61944c640 1 -- 172.21.15.84:0/3901014578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe614159ab0 msgr2=0x7fe6140c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:32.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.982+0000 7fe61944c640 1 --2- 172.21.15.84:0/3901014578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe614159ab0 0x7fe6140c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.982+0000 7fe61944c640 1 -- 172.21.15.84:0/3901014578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe61406c140 msgr2=0x7fe614158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:32.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.982+0000 7fe61944c640 1 --2- 172.21.15.84:0/3901014578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe61406c140 0x7fe614158b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:32.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.982+0000 7fe61944c640 1 -- 172.21.15.84:0/3901014578 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60c009a70 con 0x7fe614159e90 2023-10-16T10:01:32.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.982+0000 7fe61944c640 1 --2- 172.21.15.84:0/3901014578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe614159e90 0x7fe6140acb00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe60c037040 tx=0x7fe60c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:32.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.983+0000 7fe6117fa640 1 -- 172.21.15.84:0/3901014578 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c040650 con 0x7fe614159e90 2023-10-16T10:01:32.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.983+0000 7fe6117fa640 1 -- 172.21.15.84:0/3901014578 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60c0407b0 con 0x7fe614159e90 2023-10-16T10:01:32.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.983+0000 7fe6117fa640 1 -- 172.21.15.84:0/3901014578 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c040b60 con 0x7fe614159e90 2023-10-16T10:01:32.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.983+0000 7fe61aeaf640 1 -- 172.21.15.84:0/3901014578 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6140c7ce0 con 0x7fe614159e90 2023-10-16T10:01:32.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.983+0000 7fe61aeaf640 1 -- 172.21.15.84:0/3901014578 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6140c82a0 con 0x7fe614159e90 2023-10-16T10:01:32.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.984+0000 7fe61aeaf640 1 -- 172.21.15.84:0/3901014578 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6140cd790 con 0x7fe614159e90 2023-10-16T10:01:32.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.985+0000 7fe6117fa640 1 -- 172.21.15.84:0/3901014578 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fe60c040cc0 con 0x7fe614159e90 2023-10-16T10:01:32.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.986+0000 7fe6117fa640 1 --2- 172.21.15.84:0/3901014578 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fe5f806b070 0x7fe5f806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:32.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.986+0000 7fe6117fa640 1 -- 172.21.15.84:0/3901014578 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fe60c0ac7d0 con 0x7fe614159e90 2023-10-16T10:01:32.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.986+0000 7fe618c4b640 1 --2- 172.21.15.84:0/3901014578 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fe5f806b070 0x7fe5f806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:32.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.987+0000 7fe618c4b640 1 --2- 172.21.15.84:0/3901014578 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fe5f806b070 0x7fe5f806d530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe6080011e0 tx=0x7fe608009040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:32.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:32.990+0000 7fe6117fa640 1 -- 172.21.15.84:0/3901014578 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe6140cd790 con 0x7fe614159e90 2023-10-16T10:01:33.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.204+0000 7fe61aeaf640 1 -- 172.21.15.84:0/3901014578 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6140cd4d0 con 0x7fe5f806b070 2023-10-16T10:01:33.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.219+0000 7fe6117fa640 1 -- 172.21.15.84:0/3901014578 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fe6080012d0 con 0x7fe5f806b070 2023-10-16T10:01:33.221 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:01:33.222 INFO:teuthology.orchestra.run.smithi084.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:33", "first": "2023-10-16T09:51:01", "last": "2023-10-16T10:01:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:33.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.222+0000 7fe61aeaf640 1 -- 172.21.15.84:0/3901014578 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fe5f806b070 msgr2=0x7fe5f806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:33.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.222+0000 7fe61aeaf640 1 --2- 172.21.15.84:0/3901014578 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fe5f806b070 0x7fe5f806d530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe6080011e0 tx=0x7fe608009040 comp rx=0 tx=0).stop 2023-10-16T10:01:33.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.222+0000 7fe61aeaf640 1 -- 172.21.15.84:0/3901014578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe614159e90 msgr2=0x7fe6140acb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:33.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.222+0000 7fe61aeaf640 1 --2- 172.21.15.84:0/3901014578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe614159e90 0x7fe6140acb00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe60c037040 tx=0x7fe60c004a40 comp rx=0 tx=0).stop 2023-10-16T10:01:33.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.222+0000 7fe61aeaf640 1 -- 172.21.15.84:0/3901014578 shutdown_connections 2023-10-16T10:01:33.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.222+0000 7fe61aeaf640 1 --2- 172.21.15.84:0/3901014578 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fe5f806b070 0x7fe5f806d530 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:33.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.222+0000 7fe61aeaf640 1 --2- 172.21.15.84:0/3901014578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe614159e90 0x7fe6140acb00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:33.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.222+0000 7fe61aeaf640 1 --2- 172.21.15.84:0/3901014578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe614159ab0 0x7fe6140c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:33.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.222+0000 7fe61aeaf640 1 --2- 172.21.15.84:0/3901014578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe61406c140 0x7fe614158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:33.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.222+0000 7fe61aeaf640 1 -- 172.21.15.84:0/3901014578 >> 172.21.15.84:0/3901014578 conn(0x7fe6140ce9b0 msgr2=0x7fe614070d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:33.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.222+0000 7fe61aeaf640 1 -- 172.21.15.84:0/3901014578 shutdown_connections 2023-10-16T10:01:33.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.222+0000 7fe61aeaf640 1 -- 172.21.15.84:0/3901014578 wait complete. 2023-10-16T10:01:33.243 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:33", "first": "2023-10-16T09:51:01", "last": "2023-10-16T10:01:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:33.243 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:50:33', 'first': '2023-10-16T09:51:01', 'last': '2023-10-16T10:01:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T10:01:33.244 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T10:01:33.244 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T10:01:33.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:33.245 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-10-16T10:01:33.299 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T10:01:33.300 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T10:01:33.300 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_01_UTC 2023-10-16T10:01:33.300 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_01_UTC 2023-10-16T10:01:33.300 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_01_UTC 2023-10-16T10:01:33.301 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:33.301 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_00_UTC 2023-10-16T10:01:33.301 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T10:01:33.301 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_00_UTC 2023-10-16T10:01:33.301 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_00_UTC 2023-10-16T10:01:33.302 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:33.303 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_51_00_UTC', 'scheduled-2023-10-16-09_52_00_UTC', 'scheduled-2023-10-16-09_53_01_UTC', 'scheduled-2023-10-16-09_54_01_UTC', 'scheduled-2023-10-16-09_55_01_UTC', 'scheduled-2023-10-16-09_56_00_UTC', 'scheduled-2023-10-16-09_57_00_UTC', 'scheduled-2023-10-16-09_58_00_UTC', 'scheduled-2023-10-16-09_59_00_UTC', 'scheduled-2023-10-16-10_00_00_UTC', 'scheduled-2023-10-16-10_01_00_UTC'] 2023-10-16T10:01:33.303 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir9 2023-10-16T10:01:33.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.520+0000 7f1387bcc640 1 Processor -- start 2023-10-16T10:01:33.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.520+0000 7f1387bcc640 1 -- start start 2023-10-16T10:01:33.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.522+0000 7f1387bcc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13800aee90 0x7f13800cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:33.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.522+0000 7f1387bcc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13800cf550 con 0x7f1380069da0 2023-10-16T10:01:33.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.522+0000 7f1387bcc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f138006c710 con 0x7f13800cb9c0 2023-10-16T10:01:33.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.522+0000 7f1387bcc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13800af2f0 con 0x7f13800aee90 2023-10-16T10:01:33.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.522+0000 7f1386169640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1380069da0 0x7f13800cbec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39906/0 (socket says 172.21.15.84:39906) 2023-10-16T10:01:33.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.522+0000 7f1386169640 1 -- 172.21.15.84:0/2058944090 learned_addr learned my addr 172.21.15.84:0/2058944090 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:33.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.523+0000 7f1385167640 1 --2- 172.21.15.84:0/2058944090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13800aee90 0x7f13800cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:33.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.523+0000 7f1384966640 1 -- 172.21.15.84:0/2058944090 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307056547 0 0) 0x7f13800cf550 con 0x7f1380069da0 2023-10-16T10:01:33.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.523+0000 7f1384966640 1 -- 172.21.15.84:0/2058944090 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f136c003520 con 0x7f1380069da0 2023-10-16T10:01:33.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.523+0000 7f1384966640 1 -- 172.21.15.84:0/2058944090 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4037727143 0 0) 0x7f136c003520 con 0x7f1380069da0 2023-10-16T10:01:33.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.523+0000 7f1384966640 1 -- 172.21.15.84:0/2058944090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13800aee90 msgr2=0x7f13800cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:33.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.523+0000 7f1384966640 1 --2- 172.21.15.84:0/2058944090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13800aee90 0x7f13800cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:33.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.523+0000 7f1384966640 1 -- 172.21.15.84:0/2058944090 >> v1:172.21.15.157:6789/0 conn(0x7f13800cb9c0 legacy=0x7f138006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:33.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.523+0000 7f1384966640 1 -- 172.21.15.84:0/2058944090 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138006c9f0 con 0x7f1380069da0 2023-10-16T10:01:33.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.523+0000 7f1384966640 1 -- 172.21.15.84:0/2058944090 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f13780019f0 con 0x7f1380069da0 2023-10-16T10:01:33.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.523+0000 7f1384966640 1 -- 172.21.15.84:0/2058944090 >> v1:172.21.15.84:6789/0 conn(0x7f1380069da0 legacy=0x7f13800cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:33.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.523+0000 7f1384966640 1 --2- 172.21.15.84:0/2058944090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13800aee90 0x7f136c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:33.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.523+0000 7f1384966640 1 --2- 172.21.15.84:0/2058944090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f136c003f20 0x7f136c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:33.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.523+0000 7f1384966640 1 --2- 172.21.15.84:0/2058944090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f136c004d00 0x7f136c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:33.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.523+0000 7f1384966640 1 -- 172.21.15.84:0/2058944090 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13800af2f0 con 0x7f136c004d00 2023-10-16T10:01:33.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.523+0000 7f1384966640 1 -- 172.21.15.84:0/2058944090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c00bb10 con 0x7f136c003f20 2023-10-16T10:01:33.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.523+0000 7f1384966640 1 -- 172.21.15.84:0/2058944090 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f136c00bcd0 con 0x7f13800aee90 2023-10-16T10:01:33.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.524+0000 7f1386169640 1 --2- 172.21.15.84:0/2058944090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f136c004d00 0x7f136c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:33.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.524+0000 7f1385167640 1 --2- 172.21.15.84:0/2058944090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f136c003f20 0x7f136c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:33.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.524+0000 7f1386169640 1 -- 172.21.15.84:0/2058944090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13800aee90 msgr2=0x7f136c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:33.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.524+0000 7f1386169640 1 --2- 172.21.15.84:0/2058944090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13800aee90 0x7f136c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:33.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.524+0000 7f1386169640 1 -- 172.21.15.84:0/2058944090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f136c003f20 msgr2=0x7f136c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:33.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.524+0000 7f1386169640 1 --2- 172.21.15.84:0/2058944090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f136c003f20 0x7f136c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:33.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.524+0000 7f1386169640 1 -- 172.21.15.84:0/2058944090 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c003520 con 0x7f136c004d00 2023-10-16T10:01:33.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.524+0000 7f1385167640 1 --2- 172.21.15.84:0/2058944090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f136c003f20 0x7f136c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:33.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.525+0000 7f1386169640 1 --2- 172.21.15.84:0/2058944090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f136c004d00 0x7f136c005180 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f137800a830 tx=0x7f136c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bdf9d6d9c34410c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:33.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.525+0000 7f1384966640 1 -- 172.21.15.84:0/2058944090 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13780072d0 con 0x7f136c004d00 2023-10-16T10:01:33.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.526+0000 7f1384966640 1 -- 172.21.15.84:0/2058944090 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137802f070 con 0x7f136c004d00 2023-10-16T10:01:33.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.526+0000 7f1384966640 1 -- 172.21.15.84:0/2058944090 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378007690 con 0x7f136c004d00 2023-10-16T10:01:33.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.526+0000 7f1387bcc640 1 -- 172.21.15.84:0/2058944090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f136c004d00 msgr2=0x7f136c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:33.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.526+0000 7f1387bcc640 1 --2- 172.21.15.84:0/2058944090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f136c004d00 0x7f136c005180 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f137800a830 tx=0x7f136c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:01:33.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.526+0000 7f1386169640 1 -- 172.21.15.84:0/2058944090 reap_dead start 2023-10-16T10:01:33.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.526+0000 7f1387bcc640 1 -- 172.21.15.84:0/2058944090 shutdown_connections 2023-10-16T10:01:33.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.526+0000 7f1387bcc640 1 -- 172.21.15.84:0/2058944090 >> 172.21.15.84:0/2058944090 conn(0x7f13800b14c0 msgr2=0x7f13800b18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:33.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.526+0000 7f1387bcc640 1 -- 172.21.15.84:0/2058944090 shutdown_connections 2023-10-16T10:01:33.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.526+0000 7f1387bcc640 1 -- 172.21.15.84:0/2058944090 wait complete. 2023-10-16T10:01:33.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.528+0000 7f1387bcc640 1 Processor -- start 2023-10-16T10:01:33.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.528+0000 7f1387bcc640 1 -- start start 2023-10-16T10:01:33.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.529+0000 7f1387bcc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f136c004d00 0x7f138015e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:33.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.529+0000 7f1387bcc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13800b2fa0 0x7f13800b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:33.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.529+0000 7f1387bcc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13800b3c40 0x7f13800b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:33.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.529+0000 7f1387bcc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13800b39f0 con 0x7f13800b3c40 2023-10-16T10:01:33.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.529+0000 7f1387bcc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13800c9be0 con 0x7f13800b2fa0 2023-10-16T10:01:33.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.529+0000 7f1387bcc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13800c9030 con 0x7f136c004d00 2023-10-16T10:01:33.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.529+0000 7f1386169640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13800b3c40 0x7f13800b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:33.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.530+0000 7f1386169640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13800b3c40 0x7f13800b40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57112/0 (socket says 172.21.15.84:57112) 2023-10-16T10:01:33.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.530+0000 7f1386169640 1 -- 172.21.15.84:0/2040015567 learned_addr learned my addr 172.21.15.84:0/2040015567 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:33.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.530+0000 7f1385968640 1 --2- 172.21.15.84:0/2040015567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f136c004d00 0x7f138015e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:33.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.530+0000 7f1385167640 1 --2- 172.21.15.84:0/2040015567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13800b2fa0 0x7f13800b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:33.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.530+0000 7f1386169640 1 -- 172.21.15.84:0/2040015567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f136c004d00 msgr2=0x7f138015e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:33.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.530+0000 7f1386169640 1 --2- 172.21.15.84:0/2040015567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f136c004d00 0x7f138015e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:33.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.530+0000 7f1386169640 1 -- 172.21.15.84:0/2040015567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13800b2fa0 msgr2=0x7f13800b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:33.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.530+0000 7f1386169640 1 --2- 172.21.15.84:0/2040015567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13800b2fa0 0x7f13800b3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:33.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.530+0000 7f1386169640 1 -- 172.21.15.84:0/2040015567 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13800c92b0 con 0x7f13800b3c40 2023-10-16T10:01:33.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.530+0000 7f1385167640 1 --2- 172.21.15.84:0/2040015567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13800b2fa0 0x7f13800b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:33.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.530+0000 7f1386169640 1 --2- 172.21.15.84:0/2040015567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13800b3c40 0x7f13800b40c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f137802cc40 tx=0x7f1378001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:33.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.531+0000 7f13767fc640 1 -- 172.21.15.84:0/2040015567 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378001250 con 0x7f13800b3c40 2023-10-16T10:01:33.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.531+0000 7f13767fc640 1 -- 172.21.15.84:0/2040015567 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1378040480 con 0x7f13800b3c40 2023-10-16T10:01:33.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.531+0000 7f13767fc640 1 -- 172.21.15.84:0/2040015567 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378040830 con 0x7f13800b3c40 2023-10-16T10:01:33.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.531+0000 7f1387bcc640 1 -- 172.21.15.84:0/2040015567 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13800c9590 con 0x7f13800b3c40 2023-10-16T10:01:33.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.531+0000 7f1387bcc640 1 -- 172.21.15.84:0/2040015567 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13800c7410 con 0x7f13800b3c40 2023-10-16T10:01:33.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.532+0000 7f1387bcc640 1 -- 172.21.15.84:0/2040015567 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13800c75a0 con 0x7f13800b3c40 2023-10-16T10:01:33.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.533+0000 7f13767fc640 1 -- 172.21.15.84:0/2040015567 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f1378031030 con 0x7f13800b3c40 2023-10-16T10:01:33.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.534+0000 7f13767fc640 1 --2- 172.21.15.84:0/2040015567 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f136406b0a0 0x7f136406d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:33.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.534+0000 7f13767fc640 1 -- 172.21.15.84:0/2040015567 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f13780ab460 con 0x7f13800b3c40 2023-10-16T10:01:33.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.534+0000 7f1385968640 1 --2- 172.21.15.84:0/2040015567 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f136406b0a0 0x7f136406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:33.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.535+0000 7f1385968640 1 --2- 172.21.15.84:0/2040015567 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f136406b0a0 0x7f136406d560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f137c001680 tx=0x7f137c009040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:33.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.538+0000 7f13767fc640 1 -- 172.21.15.84:0/2040015567 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1378075fe0 con 0x7f13800b3c40 2023-10-16T10:01:33.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.749+0000 7f1387bcc640 1 -- 172.21.15.84:0/2040015567 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f138006ccd0 con 0x7f136406b0a0 2023-10-16T10:01:33.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.765+0000 7f13767fc640 1 -- 172.21.15.84:0/2040015567 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f137c001340 con 0x7f136406b0a0 2023-10-16T10:01:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:01:33.767 INFO:teuthology.orchestra.run.smithi084.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:34", "first": "2023-10-16T09:51:00", "last": "2023-10-16T10:01:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:33.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.768+0000 7f1387bcc640 1 -- 172.21.15.84:0/2040015567 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f136406b0a0 msgr2=0x7f136406d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:33.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.768+0000 7f1387bcc640 1 --2- 172.21.15.84:0/2040015567 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f136406b0a0 0x7f136406d560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f137c001680 tx=0x7f137c009040 comp rx=0 tx=0).stop 2023-10-16T10:01:33.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.768+0000 7f1387bcc640 1 -- 172.21.15.84:0/2040015567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13800b3c40 msgr2=0x7f13800b40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:33.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.768+0000 7f1387bcc640 1 --2- 172.21.15.84:0/2040015567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13800b3c40 0x7f13800b40c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f137802cc40 tx=0x7f1378001d80 comp rx=0 tx=0).stop 2023-10-16T10:01:33.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.768+0000 7f1387bcc640 1 -- 172.21.15.84:0/2040015567 shutdown_connections 2023-10-16T10:01:33.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.768+0000 7f1387bcc640 1 --2- 172.21.15.84:0/2040015567 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f136406b0a0 0x7f136406d560 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:33.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.768+0000 7f1387bcc640 1 --2- 172.21.15.84:0/2040015567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13800b3c40 0x7f13800b40c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:33.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.768+0000 7f1387bcc640 1 --2- 172.21.15.84:0/2040015567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13800b2fa0 0x7f13800b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:33.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.768+0000 7f1387bcc640 1 --2- 172.21.15.84:0/2040015567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f136c004d00 0x7f138015e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:33.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.768+0000 7f1387bcc640 1 -- 172.21.15.84:0/2040015567 >> 172.21.15.84:0/2040015567 conn(0x7f13800b14c0 msgr2=0x7f13800cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:33.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.768+0000 7f1387bcc640 1 -- 172.21.15.84:0/2040015567 shutdown_connections 2023-10-16T10:01:33.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:33.768+0000 7f1387bcc640 1 -- 172.21.15.84:0/2040015567 wait complete. 2023-10-16T10:01:33.789 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:50:34", "first": "2023-10-16T09:51:00", "last": "2023-10-16T10:01:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T10:01:33.789 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:50:34', 'first': '2023-10-16T09:51:00', 'last': '2023-10-16T10:01:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T10:01:33.789 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T10:01:33.790 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T10:01:33.790 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir0 1m 2023-10-16T10:01:34.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.005+0000 7f3299f89640 1 Processor -- start 2023-10-16T10:01:34.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.005+0000 7f3299f89640 1 -- start start 2023-10-16T10:01:34.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.007+0000 7f3299f89640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f329415d150 0x7f32940ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:34.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.007+0000 7f3299f89640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f329406a4b0 0x7f3294160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:34.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.007+0000 7f3299f89640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32940ac890 con 0x7f3294160b90 2023-10-16T10:01:34.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.007+0000 7f3299f89640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f329415d530 con 0x7f329406a4b0 2023-10-16T10:01:34.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.007+0000 7f3299f89640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3294161650 con 0x7f329415d150 2023-10-16T10:01:34.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.007+0000 7f3292ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3294160b90 0x7f3294069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39920/0 (socket says 172.21.15.84:39920) 2023-10-16T10:01:34.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.007+0000 7f3292ffd640 1 -- 172.21.15.84:0/3757796080 learned_addr learned my addr 172.21.15.84:0/3757796080 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:34.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.008+0000 7f32937fe640 1 --2- 172.21.15.84:0/3757796080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f329415d150 0x7f32940ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:34.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.008+0000 7f3293fff640 1 --2- 172.21.15.84:0/3757796080 >> v2:172.21.15.157:3300/0 conn(0x7f329406a4b0 0x7f3294160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:34.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.008+0000 7f32927fc640 1 -- 172.21.15.84:0/3757796080 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 509496798 0 0) 0x7f32940ac890 con 0x7f3294160b90 2023-10-16T10:01:34.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.008+0000 7f32927fc640 1 -- 172.21.15.84:0/3757796080 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f327c003520 con 0x7f3294160b90 2023-10-16T10:01:34.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.008+0000 7f32927fc640 1 -- 172.21.15.84:0/3757796080 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1241176667 0 0) 0x7f327c003520 con 0x7f3294160b90 2023-10-16T10:01:34.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.008+0000 7f32927fc640 1 -- 172.21.15.84:0/3757796080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f329415d150 msgr2=0x7f32940ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:34.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.008+0000 7f32927fc640 1 --2- 172.21.15.84:0/3757796080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f329415d150 0x7f32940ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:34.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.008+0000 7f32927fc640 1 -- 172.21.15.84:0/3757796080 >> v2:172.21.15.157:3300/0 conn(0x7f329406a4b0 msgr2=0x7f3294160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:34.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.008+0000 7f32927fc640 1 --2- 172.21.15.84:0/3757796080 >> v2:172.21.15.157:3300/0 conn(0x7f329406a4b0 0x7f3294160fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:34.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.008+0000 7f32927fc640 1 -- 172.21.15.84:0/3757796080 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3294161930 con 0x7f3294160b90 2023-10-16T10:01:34.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.008+0000 7f32927fc640 1 -- 172.21.15.84:0/3757796080 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f3288001670 con 0x7f3294160b90 2023-10-16T10:01:34.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.008+0000 7f32927fc640 1 -- 172.21.15.84:0/3757796080 >> v1:172.21.15.84:6789/0 conn(0x7f3294160b90 legacy=0x7f3294069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:34.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.008+0000 7f32927fc640 1 --2- 172.21.15.84:0/3757796080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f329415d150 0x7f327c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:34.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.008+0000 7f32927fc640 1 --2- 172.21.15.84:0/3757796080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f327c003f40 0x7f327c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:34.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.008+0000 7f32927fc640 1 --2- 172.21.15.84:0/3757796080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f327c004d00 0x7f327c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:34.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.008+0000 7f32927fc640 1 -- 172.21.15.84:0/3757796080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329415d530 con 0x7f327c004d00 2023-10-16T10:01:34.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.008+0000 7f32927fc640 1 -- 172.21.15.84:0/3757796080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3294161650 con 0x7f327c003f40 2023-10-16T10:01:34.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.008+0000 7f32927fc640 1 -- 172.21.15.84:0/3757796080 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327c00ba30 con 0x7f329415d150 2023-10-16T10:01:34.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.009+0000 7f3293fff640 1 --2- 172.21.15.84:0/3757796080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f327c004d00 0x7f327c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:34.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.009+0000 7f3292ffd640 1 --2- 172.21.15.84:0/3757796080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f327c003f40 0x7f327c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:34.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.009+0000 7f3293fff640 1 -- 172.21.15.84:0/3757796080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f329415d150 msgr2=0x7f327c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:34.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.009+0000 7f32937fe640 1 --2- 172.21.15.84:0/3757796080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f329415d150 0x7f327c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:34.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.009+0000 7f3293fff640 1 --2- 172.21.15.84:0/3757796080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f329415d150 0x7f327c003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:34.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.009+0000 7f3293fff640 1 -- 172.21.15.84:0/3757796080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f327c003f40 msgr2=0x7f327c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:34.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.009+0000 7f3293fff640 1 --2- 172.21.15.84:0/3757796080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f327c003f40 0x7f327c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:34.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.009+0000 7f3293fff640 1 -- 172.21.15.84:0/3757796080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c003520 con 0x7f327c004d00 2023-10-16T10:01:34.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.009+0000 7f3292ffd640 1 --2- 172.21.15.84:0/3757796080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f327c003f40 0x7f327c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:34.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.010+0000 7f3293fff640 1 --2- 172.21.15.84:0/3757796080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f327c004d00 0x7f327c005180 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f328c007360 tx=0x7f327c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa30055ae4da128 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:34.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.010+0000 7f32927fc640 1 -- 172.21.15.84:0/3757796080 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c02ccf0 con 0x7f327c004d00 2023-10-16T10:01:34.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.010+0000 7f32927fc640 1 -- 172.21.15.84:0/3757796080 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328c02ce50 con 0x7f327c004d00 2023-10-16T10:01:34.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.010+0000 7f32927fc640 1 -- 172.21.15.84:0/3757796080 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c034660 con 0x7f327c004d00 2023-10-16T10:01:34.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.011+0000 7f3299f89640 1 -- 172.21.15.84:0/3757796080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f327c004d00 msgr2=0x7f327c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:34.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.011+0000 7f3299f89640 1 --2- 172.21.15.84:0/3757796080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f327c004d00 0x7f327c005180 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f328c007360 tx=0x7f327c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:01:34.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.011+0000 7f3293fff640 1 -- 172.21.15.84:0/3757796080 reap_dead start 2023-10-16T10:01:34.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.011+0000 7f3299f89640 1 -- 172.21.15.84:0/3757796080 shutdown_connections 2023-10-16T10:01:34.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.011+0000 7f3299f89640 1 -- 172.21.15.84:0/3757796080 >> 172.21.15.84:0/3757796080 conn(0x7f32940ce9b0 msgr2=0x7f32940cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:34.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.011+0000 7f3299f89640 1 -- 172.21.15.84:0/3757796080 shutdown_connections 2023-10-16T10:01:34.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.011+0000 7f3299f89640 1 -- 172.21.15.84:0/3757796080 wait complete. 2023-10-16T10:01:34.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.013+0000 7f3299f89640 1 Processor -- start 2023-10-16T10:01:34.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.013+0000 7f3299f89640 1 -- start start 2023-10-16T10:01:34.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.014+0000 7f3299f89640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f327c004d00 0x7f32940c2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:34.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.014+0000 7f3299f89640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32941f1bf0 0x7f32941f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:34.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.014+0000 7f3299f89640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3294156c60 0x7f32941570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:34.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.014+0000 7f3299f89640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32940ae570 con 0x7f327c004d00 2023-10-16T10:01:34.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.014+0000 7f3299f89640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3294155ee0 con 0x7f3294156c60 2023-10-16T10:01:34.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.014+0000 7f3299f89640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3294156180 con 0x7f32941f1bf0 2023-10-16T10:01:34.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.014+0000 7f32937fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f327c004d00 0x7f32940c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:34.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.014+0000 7f32937fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f327c004d00 0x7f32940c2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57120/0 (socket says 172.21.15.84:57120) 2023-10-16T10:01:34.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.014+0000 7f32937fe640 1 -- 172.21.15.84:0/939637976 learned_addr learned my addr 172.21.15.84:0/939637976 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:34.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.015+0000 7f32937fe640 1 -- 172.21.15.84:0/939637976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32941f1bf0 msgr2=0x7f32941f2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:34.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.015+0000 7f32937fe640 1 --2- 172.21.15.84:0/939637976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32941f1bf0 0x7f32941f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:34.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.015+0000 7f32937fe640 1 -- 172.21.15.84:0/939637976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3294156c60 msgr2=0x7f32941570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:34.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.015+0000 7f3293fff640 1 --2- 172.21.15.84:0/939637976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3294156c60 0x7f32941570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:34.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.015+0000 7f32937fe640 1 --2- 172.21.15.84:0/939637976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3294156c60 0x7f32941570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:34.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.015+0000 7f32937fe640 1 -- 172.21.15.84:0/939637976 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3294156460 con 0x7f327c004d00 2023-10-16T10:01:34.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.015+0000 7f32937fe640 1 --2- 172.21.15.84:0/939637976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f327c004d00 0x7f32940c2600 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3284010860 tx=0x7f3284010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:34.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.015+0000 7f327bfff640 1 -- 172.21.15.84:0/939637976 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284009610 con 0x7f327c004d00 2023-10-16T10:01:34.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.015+0000 7f327bfff640 1 -- 172.21.15.84:0/939637976 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328400eb60 con 0x7f327c004d00 2023-10-16T10:01:34.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.015+0000 7f327bfff640 1 -- 172.21.15.84:0/939637976 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328401c400 con 0x7f327c004d00 2023-10-16T10:01:34.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.015+0000 7f3299f89640 1 -- 172.21.15.84:0/939637976 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3294156740 con 0x7f327c004d00 2023-10-16T10:01:34.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.015+0000 7f3299f89640 1 -- 172.21.15.84:0/939637976 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32940c2b40 con 0x7f327c004d00 2023-10-16T10:01:34.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.016+0000 7f3299f89640 1 -- 172.21.15.84:0/939637976 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329404e160 con 0x7f327c004d00 2023-10-16T10:01:34.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.017+0000 7f327bfff640 1 -- 172.21.15.84:0/939637976 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f328401c560 con 0x7f327c004d00 2023-10-16T10:01:34.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.018+0000 7f327bfff640 1 --2- 172.21.15.84:0/939637976 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f328006b070 0x7f328006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:34.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.018+0000 7f327bfff640 1 -- 172.21.15.84:0/939637976 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f328408a090 con 0x7f327c004d00 2023-10-16T10:01:34.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.019+0000 7f3292ffd640 1 --2- 172.21.15.84:0/939637976 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f328006b070 0x7f328006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:34.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.019+0000 7f3292ffd640 1 --2- 172.21.15.84:0/939637976 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f328006b070 0x7f328006d530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f32880017a0 tx=0x7f3288009040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:34.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.023+0000 7f327bfff640 1 -- 172.21.15.84:0/939637976 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3284054dd0 con 0x7f327c004d00 2023-10-16T10:01:34.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.237+0000 7f3299f89640 1 -- 172.21.15.84:0/939637976 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f32940ace30 con 0x7f328006b070 2023-10-16T10:01:34.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.457+0000 7f327bfff640 1 -- 172.21.15.84:0/939637976 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f32880047e0 con 0x7f328006b070 2023-10-16T10:01:34.460 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-10-16T10:01:34.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.461+0000 7f3299f89640 1 -- 172.21.15.84:0/939637976 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f328006b070 msgr2=0x7f328006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:34.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.461+0000 7f3299f89640 1 --2- 172.21.15.84:0/939637976 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f328006b070 0x7f328006d530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f32880017a0 tx=0x7f3288009040 comp rx=0 tx=0).stop 2023-10-16T10:01:34.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.461+0000 7f3299f89640 1 -- 172.21.15.84:0/939637976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f327c004d00 msgr2=0x7f32940c2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:34.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.461+0000 7f3299f89640 1 --2- 172.21.15.84:0/939637976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f327c004d00 0x7f32940c2600 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3284010860 tx=0x7f3284010d20 comp rx=0 tx=0).stop 2023-10-16T10:01:34.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.461+0000 7f3299f89640 1 -- 172.21.15.84:0/939637976 shutdown_connections 2023-10-16T10:01:34.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.461+0000 7f3299f89640 1 --2- 172.21.15.84:0/939637976 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f328006b070 0x7f328006d530 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:34.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.461+0000 7f3299f89640 1 --2- 172.21.15.84:0/939637976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3294156c60 0x7f32941570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:34.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.461+0000 7f3299f89640 1 --2- 172.21.15.84:0/939637976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32941f1bf0 0x7f32941f2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:34.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.461+0000 7f3299f89640 1 --2- 172.21.15.84:0/939637976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f327c004d00 0x7f32940c2600 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:34.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.461+0000 7f3299f89640 1 -- 172.21.15.84:0/939637976 >> 172.21.15.84:0/939637976 conn(0x7f32940ce9b0 msgr2=0x7f32940b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:34.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.461+0000 7f3299f89640 1 -- 172.21.15.84:0/939637976 shutdown_connections 2023-10-16T10:01:34.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:34.461+0000 7f3299f89640 1 -- 172.21.15.84:0/939637976 wait complete. 2023-10-16T10:01:34.484 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-10-16T10:01:34.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:34.485 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-10-16T10:01:34.541 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T10:01:34.541 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T10:01:34.542 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T10:01:34.542 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:34.542 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T10:01:34.542 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:34.543 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_00_UTC 2023-10-16T10:01:34.543 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T10:01:34.543 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_01_UTC 2023-10-16T10:01:34.543 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_01_UTC 2023-10-16T10:01:34.544 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:34.545 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_51_00_UTC 2023-10-16T10:01:34.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:34.547 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_51_00_UTC) 2023-10-16T10:01:34.617 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_52_00_UTC 2023-10-16T10:01:34.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:34.618 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_52_00_UTC) 2023-10-16T10:01:34.689 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_53_00_UTC 2023-10-16T10:01:34.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:34.690 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_53_00_UTC) 2023-10-16T10:01:34.761 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:34.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:34.762 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_54_00_UTC) 2023-10-16T10:01:34.835 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_55_00_UTC 2023-10-16T10:01:34.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:34.836 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_55_00_UTC) 2023-10-16T10:01:34.901 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:34.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:34.902 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_56_00_UTC) 2023-10-16T10:01:34.973 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_57_00_UTC 2023-10-16T10:01:34.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:34.975 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_57_00_UTC) 2023-10-16T10:01:35.049 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_58_00_UTC 2023-10-16T10:01:35.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:35.050 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_58_00_UTC) 2023-10-16T10:01:35.125 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_59_01_UTC 2023-10-16T10:01:35.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:35.126 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_59_01_UTC) 2023-10-16T10:01:35.201 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-10_00_01_UTC 2023-10-16T10:01:35.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:35.202 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-10_00_01_UTC) 2023-10-16T10:01:35.276 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:35.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:35.277 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-10_01_00_UTC) 2023-10-16T10:01:35.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:35.351 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-10-16T10:01:35.415 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir1 1m 2023-10-16T10:01:35.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.639+0000 7ff7794a9640 1 Processor -- start 2023-10-16T10:01:35.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.639+0000 7ff7794a9640 1 -- start start 2023-10-16T10:01:35.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.640+0000 7ff7794a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77406b8c0 0x7ff77406bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:35.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.640+0000 7ff7794a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff77406b1a0 0x7ff7740ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:35.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.640+0000 7ff7794a9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7740cd390 con 0x7ff77415d130 2023-10-16T10:01:35.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.640+0000 7ff7794a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77406c1e0 con 0x7ff77406b1a0 2023-10-16T10:01:35.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.640+0000 7ff7794a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740cc2f0 con 0x7ff77406b8c0 2023-10-16T10:01:35.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.641+0000 7ff7737fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff77406b1a0 0x7ff7740ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:35.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.641+0000 7ff7727fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff77415d130 0x7ff7740cbaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35502/0 (socket says 172.21.15.84:35502) 2023-10-16T10:01:35.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.641+0000 7ff7727fc640 1 -- 172.21.15.84:0/3650009415 learned_addr learned my addr 172.21.15.84:0/3650009415 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:35.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.641+0000 7ff7737fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff77406b1a0 0x7ff7740ad6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48758/0 (socket says 172.21.15.84:48758) 2023-10-16T10:01:35.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.641+0000 7ff772ffd640 1 --2- 172.21.15.84:0/3650009415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77406b8c0 0x7ff77406bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:35.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.641+0000 7ff7737fe640 1 -- 172.21.15.84:0/3650009415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77406b8c0 msgr2=0x7ff77406bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:35.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.641+0000 7ff7737fe640 1 --2- 172.21.15.84:0/3650009415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77406b8c0 0x7ff77406bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:35.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.641+0000 7ff7737fe640 1 -- 172.21.15.84:0/3650009415 >> v1:172.21.15.84:6789/0 conn(0x7ff77415d130 legacy=0x7ff7740cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:35.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.641+0000 7ff7737fe640 1 -- 172.21.15.84:0/3650009415 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7740cc5d0 con 0x7ff77406b1a0 2023-10-16T10:01:35.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.641+0000 7ff772ffd640 1 --2- 172.21.15.84:0/3650009415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77406b8c0 0x7ff77406bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:35.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.641+0000 7ff771ffb640 1 -- 172.21.15.84:0/3650009415 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 320866096 0 0) 0x7ff7740cd390 con 0x7ff77415d130 2023-10-16T10:01:35.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.642+0000 7ff7737fe640 1 --2- 172.21.15.84:0/3650009415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff77406b1a0 0x7ff7740ad6c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0097a0 tx=0x7ff76c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84491259a1b93aa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:35.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.642+0000 7ff771ffb640 1 -- 172.21.15.84:0/3650009415 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c02d040 con 0x7ff77406b1a0 2023-10-16T10:01:35.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.642+0000 7ff771ffb640 1 -- 172.21.15.84:0/3650009415 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c031810 con 0x7ff77406b1a0 2023-10-16T10:01:35.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.642+0000 7ff771ffb640 1 -- 172.21.15.84:0/3650009415 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c031ba0 con 0x7ff77406b1a0 2023-10-16T10:01:35.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.643+0000 7ff7794a9640 1 -- 172.21.15.84:0/3650009415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff77406b1a0 msgr2=0x7ff7740ad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:35.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.643+0000 7ff7794a9640 1 --2- 172.21.15.84:0/3650009415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff77406b1a0 0x7ff7740ad6c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0097a0 tx=0x7ff76c02b8b0 comp rx=0 tx=0).stop 2023-10-16T10:01:35.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.643+0000 7ff7794a9640 1 -- 172.21.15.84:0/3650009415 shutdown_connections 2023-10-16T10:01:35.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.643+0000 7ff7794a9640 1 --2- 172.21.15.84:0/3650009415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff77406b1a0 0x7ff7740ad6c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:35.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.643+0000 7ff7794a9640 1 --2- 172.21.15.84:0/3650009415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77406b8c0 0x7ff77406bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:35.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.643+0000 7ff7794a9640 1 -- 172.21.15.84:0/3650009415 >> 172.21.15.84:0/3650009415 conn(0x7ff7740d0ad0 msgr2=0x7ff7740d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:35.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.643+0000 7ff7794a9640 1 -- 172.21.15.84:0/3650009415 shutdown_connections 2023-10-16T10:01:35.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.643+0000 7ff7794a9640 1 -- 172.21.15.84:0/3650009415 wait complete. 2023-10-16T10:01:35.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.645+0000 7ff7794a9640 1 Processor -- start 2023-10-16T10:01:35.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.645+0000 7ff7794a9640 1 -- start start 2023-10-16T10:01:35.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.646+0000 7ff7794a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77406b1a0 0x7ff7741f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:35.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.646+0000 7ff7794a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff77406b8c0 0x7ff774159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:35.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.646+0000 7ff7794a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff77415d130 0x7ff7741f37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:35.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.646+0000 7ff7794a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7741596c0 con 0x7ff77406b8c0 2023-10-16T10:01:35.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.646+0000 7ff7794a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7741f3d20 con 0x7ff77415d130 2023-10-16T10:01:35.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.646+0000 7ff7794a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7741f4b60 con 0x7ff77406b1a0 2023-10-16T10:01:35.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.646+0000 7ff7727fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff77406b8c0 0x7ff774159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:35.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.646+0000 7ff7727fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff77406b8c0 0x7ff774159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48762/0 (socket says 172.21.15.84:48762) 2023-10-16T10:01:35.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.646+0000 7ff7727fc640 1 -- 172.21.15.84:0/3751100944 learned_addr learned my addr 172.21.15.84:0/3751100944 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:35.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.647+0000 7ff7727fc640 1 -- 172.21.15.84:0/3751100944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77406b1a0 msgr2=0x7ff7741f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:35.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.647+0000 7ff7727fc640 1 --2- 172.21.15.84:0/3751100944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77406b1a0 0x7ff7741f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:35.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.647+0000 7ff7727fc640 1 -- 172.21.15.84:0/3751100944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff77415d130 msgr2=0x7ff7741f37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:35.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.647+0000 7ff7727fc640 1 --2- 172.21.15.84:0/3751100944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff77415d130 0x7ff7741f37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:35.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.647+0000 7ff7727fc640 1 -- 172.21.15.84:0/3751100944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c009400 con 0x7ff77406b8c0 2023-10-16T10:01:35.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.647+0000 7ff7727fc640 1 --2- 172.21.15.84:0/3751100944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff77406b8c0 0x7ff774159180 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00c4b0 tx=0x7ff75c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:35.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.647+0000 7ff76b7fe640 1 -- 172.21.15.84:0/3751100944 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c0074e0 con 0x7ff77406b8c0 2023-10-16T10:01:35.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.647+0000 7ff76b7fe640 1 -- 172.21.15.84:0/3751100944 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c007660 con 0x7ff77406b8c0 2023-10-16T10:01:35.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.647+0000 7ff7794a9640 1 -- 172.21.15.84:0/3751100944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7741f3f50 con 0x7ff77406b8c0 2023-10-16T10:01:35.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.647+0000 7ff76b7fe640 1 -- 172.21.15.84:0/3751100944 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c004a20 con 0x7ff77406b8c0 2023-10-16T10:01:35.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.647+0000 7ff7794a9640 1 -- 172.21.15.84:0/3751100944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7741f4510 con 0x7ff77406b8c0 2023-10-16T10:01:35.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.649+0000 7ff7794a9640 1 -- 172.21.15.84:0/3751100944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff77404e160 con 0x7ff77406b8c0 2023-10-16T10:01:35.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.650+0000 7ff76b7fe640 1 -- 172.21.15.84:0/3751100944 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7ff75c004b80 con 0x7ff77406b8c0 2023-10-16T10:01:35.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.650+0000 7ff76b7fe640 1 --2- 172.21.15.84:0/3751100944 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff75406b070 0x7ff75406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:35.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.651+0000 7ff76b7fe640 1 -- 172.21.15.84:0/3751100944 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff75c088c50 con 0x7ff77406b8c0 2023-10-16T10:01:35.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.651+0000 7ff772ffd640 1 --2- 172.21.15.84:0/3751100944 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff75406b070 0x7ff75406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:35.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.652+0000 7ff772ffd640 1 --2- 172.21.15.84:0/3751100944 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff75406b070 0x7ff75406d530 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff764008ef0 tx=0x7ff764008420 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:35.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.655+0000 7ff76b7fe640 1 -- 172.21.15.84:0/3751100944 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff75c08e020 con 0x7ff77406b8c0 2023-10-16T10:01:35.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:35.868+0000 7ff7794a9640 1 -- 172.21.15.84:0/3751100944 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7741f4cd0 con 0x7ff75406b070 2023-10-16T10:01:36.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:36.103+0000 7ff76b7fe640 1 -- 172.21.15.84:0/3751100944 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7ff7741f4cd0 con 0x7ff75406b070 2023-10-16T10:01:36.105 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path /snap_test_dir1/dir1 2023-10-16T10:01:36.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:36.106+0000 7ff7794a9640 1 -- 172.21.15.84:0/3751100944 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff75406b070 msgr2=0x7ff75406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:36.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:36.106+0000 7ff7794a9640 1 --2- 172.21.15.84:0/3751100944 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff75406b070 0x7ff75406d530 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff764008ef0 tx=0x7ff764008420 comp rx=0 tx=0).stop 2023-10-16T10:01:36.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:36.106+0000 7ff7794a9640 1 -- 172.21.15.84:0/3751100944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff77406b8c0 msgr2=0x7ff774159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:36.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:36.106+0000 7ff7794a9640 1 --2- 172.21.15.84:0/3751100944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff77406b8c0 0x7ff774159180 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00c4b0 tx=0x7ff75c00c970 comp rx=0 tx=0).stop 2023-10-16T10:01:36.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:36.106+0000 7ff7794a9640 1 -- 172.21.15.84:0/3751100944 shutdown_connections 2023-10-16T10:01:36.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:36.106+0000 7ff7794a9640 1 --2- 172.21.15.84:0/3751100944 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff75406b070 0x7ff75406d530 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:36.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:36.106+0000 7ff7794a9640 1 --2- 172.21.15.84:0/3751100944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff77415d130 0x7ff7741f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:36.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:36.106+0000 7ff7794a9640 1 --2- 172.21.15.84:0/3751100944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff77406b8c0 0x7ff774159180 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:36.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:36.106+0000 7ff7794a9640 1 --2- 172.21.15.84:0/3751100944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77406b1a0 0x7ff7741f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:36.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:36.106+0000 7ff7794a9640 1 -- 172.21.15.84:0/3751100944 >> 172.21.15.84:0/3751100944 conn(0x7ff7740d0ad0 msgr2=0x7ff7740ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:36.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:36.107+0000 7ff7794a9640 1 -- 172.21.15.84:0/3751100944 shutdown_connections 2023-10-16T10:01:36.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:36.107+0000 7ff7794a9640 1 -- 172.21.15.84:0/3751100944 wait complete. 2023-10-16T10:01:36.128 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2023-10-16T10:01:36.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:36.129 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-10-16T10:01:36.185 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_01_UTC 2023-10-16T10:01:36.185 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T10:01:36.186 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T10:01:36.186 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:36.186 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T10:01:36.186 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:36.187 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_01_UTC 2023-10-16T10:01:36.187 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_01_UTC 2023-10-16T10:01:36.187 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_00_UTC 2023-10-16T10:01:36.187 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_00_UTC 2023-10-16T10:01:36.187 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:36.188 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_51_01_UTC 2023-10-16T10:01:36.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:36.189 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_51_01_UTC) 2023-10-16T10:01:36.261 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_52_00_UTC 2023-10-16T10:01:36.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:36.261 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_52_00_UTC) 2023-10-16T10:01:36.333 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_53_00_UTC 2023-10-16T10:01:36.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:36.334 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_53_00_UTC) 2023-10-16T10:01:36.406 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:36.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:36.407 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_54_00_UTC) 2023-10-16T10:01:36.482 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_55_00_UTC 2023-10-16T10:01:36.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:36.483 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_55_00_UTC) 2023-10-16T10:01:36.553 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:36.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:36.555 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_56_00_UTC) 2023-10-16T10:01:36.625 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_57_01_UTC 2023-10-16T10:01:36.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:36.626 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_57_01_UTC) 2023-10-16T10:01:36.699 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_58_01_UTC 2023-10-16T10:01:36.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:36.700 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_58_01_UTC) 2023-10-16T10:01:36.770 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_59_00_UTC 2023-10-16T10:01:36.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:36.771 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_59_00_UTC) 2023-10-16T10:01:36.842 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-10_00_00_UTC 2023-10-16T10:01:36.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:36.843 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-10_00_00_UTC) 2023-10-16T10:01:36.916 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:36.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:36.918 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-10_01_00_UTC) 2023-10-16T10:01:36.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:36.997 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2023-10-16T10:01:37.061 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir2 1m 2023-10-16T10:01:37.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.286+0000 7f905d4e7640 1 Processor -- start 2023-10-16T10:01:37.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.286+0000 7f905d4e7640 1 -- start start 2023-10-16T10:01:37.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.287+0000 7f905d4e7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9058159ab0 0x7f90580ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:37.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.288+0000 7f905d4e7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9058070360 con 0x7f9058159e90 2023-10-16T10:01:37.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.288+0000 7f905d4e7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90580cd1f0 con 0x7f905806c140 2023-10-16T10:01:37.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.288+0000 7f905d4e7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905815d890 con 0x7f9058159ab0 2023-10-16T10:01:37.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.288+0000 7f9056ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9058159ab0 0x7f90580ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:37.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.288+0000 7f9056ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9058159ab0 0x7f90580ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48776/0 (socket says 172.21.15.84:48776) 2023-10-16T10:01:37.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.288+0000 7f904ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9058159e90 0x7f905806fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35518/0 (socket says 172.21.15.84:35518) 2023-10-16T10:01:37.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.288+0000 7f9056ffd640 1 -- 172.21.15.84:0/3109688292 learned_addr learned my addr 172.21.15.84:0/3109688292 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:37.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.288+0000 7f9056ffd640 1 -- 172.21.15.84:0/3109688292 >> v1:172.21.15.157:6789/0 conn(0x7f905806c140 legacy=0x7f905815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:37.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.288+0000 7f9056ffd640 1 -- 172.21.15.84:0/3109688292 >> v1:172.21.15.84:6789/0 conn(0x7f9058159e90 legacy=0x7f905806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:37.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.288+0000 7f9056ffd640 1 -- 172.21.15.84:0/3109688292 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90580cd4d0 con 0x7f9058159ab0 2023-10-16T10:01:37.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.288+0000 7f90567fc640 1 -- 172.21.15.84:0/3109688292 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925822184 0 0) 0x7f9058070360 con 0x7f9058159e90 2023-10-16T10:01:37.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.289+0000 7f9056ffd640 1 --2- 172.21.15.84:0/3109688292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9058159ab0 0x7f90580ce370 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9048009870 tx=0x7f904802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37b47a4f74ff095 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:37.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.290+0000 7f90567fc640 1 -- 172.21.15.84:0/3109688292 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90480317d0 con 0x7f9058159ab0 2023-10-16T10:01:37.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.290+0000 7f90567fc640 1 -- 172.21.15.84:0/3109688292 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9048031930 con 0x7f9058159ab0 2023-10-16T10:01:37.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.290+0000 7f90567fc640 1 -- 172.21.15.84:0/3109688292 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048031cc0 con 0x7f9058159ab0 2023-10-16T10:01:37.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.290+0000 7f905d4e7640 1 -- 172.21.15.84:0/3109688292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9058159ab0 msgr2=0x7f90580ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:37.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.290+0000 7f905d4e7640 1 --2- 172.21.15.84:0/3109688292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9058159ab0 0x7f90580ce370 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9048009870 tx=0x7f904802b870 comp rx=0 tx=0).stop 2023-10-16T10:01:37.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.290+0000 7f905d4e7640 1 -- 172.21.15.84:0/3109688292 shutdown_connections 2023-10-16T10:01:37.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.290+0000 7f905d4e7640 1 --2- 172.21.15.84:0/3109688292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9058159ab0 0x7f90580ce370 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:37.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.290+0000 7f905d4e7640 1 -- 172.21.15.84:0/3109688292 >> 172.21.15.84:0/3109688292 conn(0x7f90580ce9b0 msgr2=0x7f90580cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:37.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.290+0000 7f905d4e7640 1 -- 172.21.15.84:0/3109688292 shutdown_connections 2023-10-16T10:01:37.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.290+0000 7f905d4e7640 1 -- 172.21.15.84:0/3109688292 wait complete. 2023-10-16T10:01:37.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.292+0000 7f905d4e7640 1 Processor -- start 2023-10-16T10:01:37.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.292+0000 7f905d4e7640 1 -- start start 2023-10-16T10:01:37.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.293+0000 7f905d4e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f905806c140 0x7f905814a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:37.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.293+0000 7f905d4e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9058159ab0 0x7f90581493e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:37.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.293+0000 7f905d4e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9058159e90 0x7f90580c0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:37.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.293+0000 7f905d4e7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90580bfb50 con 0x7f905806c140 2023-10-16T10:01:37.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.293+0000 7f905d4e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90580c03b0 con 0x7f9058159e90 2023-10-16T10:01:37.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.293+0000 7f905d4e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9058149920 con 0x7f9058159ab0 2023-10-16T10:01:37.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.293+0000 7f9056ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f905806c140 0x7f905814a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:37.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.294+0000 7f9056ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f905806c140 0x7f905814a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48792/0 (socket says 172.21.15.84:48792) 2023-10-16T10:01:37.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.294+0000 7f9056ffd640 1 -- 172.21.15.84:0/2773629589 learned_addr learned my addr 172.21.15.84:0/2773629589 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:37.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.294+0000 7f904ffff640 1 --2- 172.21.15.84:0/2773629589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9058159ab0 0x7f90581493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:37.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.294+0000 7f9056ffd640 1 -- 172.21.15.84:0/2773629589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9058159ab0 msgr2=0x7f90581493e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:37.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.294+0000 7f9056ffd640 1 --2- 172.21.15.84:0/2773629589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9058159ab0 0x7f90581493e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:37.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.294+0000 7f9056ffd640 1 -- 172.21.15.84:0/2773629589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9058159e90 msgr2=0x7f90580c0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:37.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.294+0000 7f9056ffd640 1 --2- 172.21.15.84:0/2773629589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9058159e90 0x7f90580c0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:37.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.294+0000 7f9056ffd640 1 -- 172.21.15.84:0/2773629589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90480094d0 con 0x7f905806c140 2023-10-16T10:01:37.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.294+0000 7f9056ffd640 1 --2- 172.21.15.84:0/2773629589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f905806c140 0x7f905814a710 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9048000c00 tx=0x7f90480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:37.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.295+0000 7f904f7fe640 1 -- 172.21.15.84:0/2773629589 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90480328f0 con 0x7f905806c140 2023-10-16T10:01:37.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.295+0000 7f904f7fe640 1 -- 172.21.15.84:0/2773629589 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9048032a70 con 0x7f905806c140 2023-10-16T10:01:37.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.295+0000 7f905d4e7640 1 -- 172.21.15.84:0/2773629589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905814ae10 con 0x7f905806c140 2023-10-16T10:01:37.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.295+0000 7f904f7fe640 1 -- 172.21.15.84:0/2773629589 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048032e20 con 0x7f905806c140 2023-10-16T10:01:37.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.295+0000 7f905d4e7640 1 -- 172.21.15.84:0/2773629589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90580c0040 con 0x7f905806c140 2023-10-16T10:01:37.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.296+0000 7f905d4e7640 1 -- 172.21.15.84:0/2773629589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905804e160 con 0x7f905806c140 2023-10-16T10:01:37.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.297+0000 7f904f7fe640 1 -- 172.21.15.84:0/2773629589 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f9048030070 con 0x7f905806c140 2023-10-16T10:01:37.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.298+0000 7f904f7fe640 1 --2- 172.21.15.84:0/2773629589 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f903806b070 0x7f903806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:37.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.298+0000 7f904f7fe640 1 -- 172.21.15.84:0/2773629589 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f90480ab7a0 con 0x7f905806c140 2023-10-16T10:01:37.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.298+0000 7f904ffff640 1 --2- 172.21.15.84:0/2773629589 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f903806b070 0x7f903806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:37.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.299+0000 7f904ffff640 1 --2- 172.21.15.84:0/2773629589 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f903806b070 0x7f903806d530 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f905814a240 tx=0x7f9040009040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:37.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.302+0000 7f904f7fe640 1 -- 172.21.15.84:0/2773629589 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9048076500 con 0x7f905806c140 2023-10-16T10:01:37.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.518+0000 7f905d4e7640 1 -- 172.21.15.84:0/2773629589 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f90580cda90 con 0x7f903806b070 2023-10-16T10:01:37.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.745+0000 7f904f7fe640 1 -- 172.21.15.84:0/2773629589 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f9040001350 con 0x7f903806b070 2023-10-16T10:01:37.746 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path /snap_test_dir1/dir2 2023-10-16T10:01:37.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.748+0000 7f905d4e7640 1 -- 172.21.15.84:0/2773629589 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f903806b070 msgr2=0x7f903806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:37.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.748+0000 7f905d4e7640 1 --2- 172.21.15.84:0/2773629589 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f903806b070 0x7f903806d530 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f905814a240 tx=0x7f9040009040 comp rx=0 tx=0).stop 2023-10-16T10:01:37.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.748+0000 7f905d4e7640 1 -- 172.21.15.84:0/2773629589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f905806c140 msgr2=0x7f905814a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:37.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.748+0000 7f905d4e7640 1 --2- 172.21.15.84:0/2773629589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f905806c140 0x7f905814a710 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9048000c00 tx=0x7f90480046c0 comp rx=0 tx=0).stop 2023-10-16T10:01:37.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.748+0000 7f905d4e7640 1 -- 172.21.15.84:0/2773629589 shutdown_connections 2023-10-16T10:01:37.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.748+0000 7f905d4e7640 1 --2- 172.21.15.84:0/2773629589 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f903806b070 0x7f903806d530 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:37.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.748+0000 7f905d4e7640 1 --2- 172.21.15.84:0/2773629589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9058159e90 0x7f90580c0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:37.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.748+0000 7f905d4e7640 1 --2- 172.21.15.84:0/2773629589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9058159ab0 0x7f90581493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:37.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.748+0000 7f905d4e7640 1 --2- 172.21.15.84:0/2773629589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f905806c140 0x7f905814a710 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:37.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.748+0000 7f905d4e7640 1 -- 172.21.15.84:0/2773629589 >> 172.21.15.84:0/2773629589 conn(0x7f90580ce9b0 msgr2=0x7f905806b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:37.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.749+0000 7f905d4e7640 1 -- 172.21.15.84:0/2773629589 shutdown_connections 2023-10-16T10:01:37.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:37.749+0000 7f905d4e7640 1 -- 172.21.15.84:0/2773629589 wait complete. 2023-10-16T10:01:37.770 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2023-10-16T10:01:37.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:37.771 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-10-16T10:01:37.828 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T10:01:37.828 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_01_UTC 2023-10-16T10:01:37.828 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T10:01:37.829 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:37.829 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T10:01:37.829 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:37.829 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_01_UTC 2023-10-16T10:01:37.830 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_01_UTC 2023-10-16T10:01:37.830 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_01_UTC 2023-10-16T10:01:37.830 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_00_UTC 2023-10-16T10:01:37.830 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:37.831 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_51_00_UTC 2023-10-16T10:01:37.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:37.832 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_51_00_UTC) 2023-10-16T10:01:37.905 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_52_01_UTC 2023-10-16T10:01:37.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:37.906 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_52_01_UTC) 2023-10-16T10:01:37.980 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_53_00_UTC 2023-10-16T10:01:37.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:37.980 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_53_00_UTC) 2023-10-16T10:01:38.055 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:38.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:38.056 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_54_00_UTC) 2023-10-16T10:01:38.129 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_55_00_UTC 2023-10-16T10:01:38.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:38.130 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_55_00_UTC) 2023-10-16T10:01:38.200 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:38.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:38.201 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_56_00_UTC) 2023-10-16T10:01:38.275 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_57_01_UTC 2023-10-16T10:01:38.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:38.276 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_57_01_UTC) 2023-10-16T10:01:38.349 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_58_01_UTC 2023-10-16T10:01:38.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:38.350 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_58_01_UTC) 2023-10-16T10:01:38.425 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_59_01_UTC 2023-10-16T10:01:38.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:38.426 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_59_01_UTC) 2023-10-16T10:01:38.497 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-10_00_00_UTC 2023-10-16T10:01:38.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:38.499 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-10_00_00_UTC) 2023-10-16T10:01:38.570 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:38.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:38.571 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-10_01_00_UTC) 2023-10-16T10:01:38.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:38.645 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2023-10-16T10:01:38.708 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir3 1m 2023-10-16T10:01:38.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.924+0000 7ff73b52e640 1 Processor -- start 2023-10-16T10:01:38.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.924+0000 7ff73b52e640 1 -- start start 2023-10-16T10:01:38.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.926+0000 7ff73b52e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff73415cd70 0x7ff7340d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:38.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.926+0000 7ff73b52e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73415d150 con 0x7ff7340b8480 2023-10-16T10:01:38.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.926+0000 7ff73b52e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7340d1180 con 0x7ff7340cded0 2023-10-16T10:01:38.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.926+0000 7ff73b52e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7340d1460 con 0x7ff73415cd70 2023-10-16T10:01:38.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.926+0000 7ff739d2b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff7340b8480 0x7ff7340cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35534/0 (socket says 172.21.15.84:35534) 2023-10-16T10:01:38.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.926+0000 7ff739d2b640 1 -- 172.21.15.84:0/2855978931 learned_addr learned my addr 172.21.15.84:0/2855978931 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:38.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.927+0000 7ff73a52c640 1 --2- 172.21.15.84:0/2855978931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff73415cd70 0x7ff7340d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:38.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.927+0000 7ff73952a640 1 -- 172.21.15.84:0/2855978931 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823861906 0 0) 0x7ff73415d150 con 0x7ff7340b8480 2023-10-16T10:01:38.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.927+0000 7ff73952a640 1 -- 172.21.15.84:0/2855978931 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff720003520 con 0x7ff7340b8480 2023-10-16T10:01:38.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.927+0000 7ff73952a640 1 -- 172.21.15.84:0/2855978931 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1723566026 0 0) 0x7ff720003520 con 0x7ff7340b8480 2023-10-16T10:01:38.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.927+0000 7ff73952a640 1 -- 172.21.15.84:0/2855978931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff73415cd70 msgr2=0x7ff7340d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:38.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.927+0000 7ff73952a640 1 --2- 172.21.15.84:0/2855978931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff73415cd70 0x7ff7340d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:38.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.927+0000 7ff73952a640 1 -- 172.21.15.84:0/2855978931 >> v1:172.21.15.157:6789/0 conn(0x7ff7340cded0 legacy=0x7ff7340b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:38.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.927+0000 7ff73952a640 1 -- 172.21.15.84:0/2855978931 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7340d1740 con 0x7ff7340b8480 2023-10-16T10:01:38.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.927+0000 7ff73952a640 1 -- 172.21.15.84:0/2855978931 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ff724001670 con 0x7ff7340b8480 2023-10-16T10:01:38.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.927+0000 7ff73952a640 1 -- 172.21.15.84:0/2855978931 >> v1:172.21.15.84:6789/0 conn(0x7ff7340b8480 legacy=0x7ff7340cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:38.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.927+0000 7ff73952a640 1 --2- 172.21.15.84:0/2855978931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7200039a0 0x7ff720003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:38.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.927+0000 7ff73952a640 1 --2- 172.21.15.84:0/2855978931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff73415cd70 0x7ff7200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:38.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.927+0000 7ff73952a640 1 --2- 172.21.15.84:0/2855978931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff720004d00 0x7ff720005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:38.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.927+0000 7ff73952a640 1 -- 172.21.15.84:0/2855978931 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7340d1460 con 0x7ff720004d00 2023-10-16T10:01:38.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.927+0000 7ff73952a640 1 -- 172.21.15.84:0/2855978931 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72000bab0 con 0x7ff7200039a0 2023-10-16T10:01:38.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.927+0000 7ff73952a640 1 -- 172.21.15.84:0/2855978931 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72000bc70 con 0x7ff73415cd70 2023-10-16T10:01:38.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.928+0000 7ff73ad2d640 1 --2- 172.21.15.84:0/2855978931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff720004d00 0x7ff720005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:38.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.928+0000 7ff739d2b640 1 --2- 172.21.15.84:0/2855978931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff73415cd70 0x7ff7200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:38.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.928+0000 7ff73a52c640 1 --2- 172.21.15.84:0/2855978931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7200039a0 0x7ff720003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:38.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.928+0000 7ff73ad2d640 1 -- 172.21.15.84:0/2855978931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff73415cd70 msgr2=0x7ff7200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:38.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.928+0000 7ff73ad2d640 1 --2- 172.21.15.84:0/2855978931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff73415cd70 0x7ff7200047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:38.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.928+0000 7ff73ad2d640 1 -- 172.21.15.84:0/2855978931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7200039a0 msgr2=0x7ff720003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:38.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.928+0000 7ff73ad2d640 1 --2- 172.21.15.84:0/2855978931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7200039a0 0x7ff720003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:38.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.928+0000 7ff73ad2d640 1 -- 172.21.15.84:0/2855978931 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff720003520 con 0x7ff720004d00 2023-10-16T10:01:38.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.929+0000 7ff73ad2d640 1 --2- 172.21.15.84:0/2855978931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff720004d00 0x7ff720005180 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff72c001e80 tx=0x7ff7200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea97bea176a462c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:38.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.929+0000 7ff73952a640 1 -- 172.21.15.84:0/2855978931 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c003b90 con 0x7ff720004d00 2023-10-16T10:01:38.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.929+0000 7ff73952a640 1 -- 172.21.15.84:0/2855978931 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72c003cf0 con 0x7ff720004d00 2023-10-16T10:01:38.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.929+0000 7ff73952a640 1 -- 172.21.15.84:0/2855978931 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c005310 con 0x7ff720004d00 2023-10-16T10:01:38.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.930+0000 7ff73b52e640 1 -- 172.21.15.84:0/2855978931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff720004d00 msgr2=0x7ff720005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:38.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.930+0000 7ff73b52e640 1 --2- 172.21.15.84:0/2855978931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff720004d00 0x7ff720005180 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff72c001e80 tx=0x7ff7200057f0 comp rx=0 tx=0).stop 2023-10-16T10:01:38.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.930+0000 7ff73ad2d640 1 -- 172.21.15.84:0/2855978931 reap_dead start 2023-10-16T10:01:38.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.930+0000 7ff73b52e640 1 -- 172.21.15.84:0/2855978931 shutdown_connections 2023-10-16T10:01:38.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.930+0000 7ff73b52e640 1 -- 172.21.15.84:0/2855978931 >> 172.21.15.84:0/2855978931 conn(0x7ff7340cf570 msgr2=0x7ff7340cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:38.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.930+0000 7ff73b52e640 1 -- 172.21.15.84:0/2855978931 shutdown_connections 2023-10-16T10:01:38.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.930+0000 7ff73b52e640 1 -- 172.21.15.84:0/2855978931 wait complete. 2023-10-16T10:01:38.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.932+0000 7ff73b52e640 1 Processor -- start 2023-10-16T10:01:38.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.932+0000 7ff73b52e640 1 -- start start 2023-10-16T10:01:38.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.933+0000 7ff73b52e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff720004d00 0x7ff7341f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:38.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.933+0000 7ff73b52e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7340c9bc0 0x7ff7340cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:38.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.933+0000 7ff73b52e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7341585c0 0x7ff734158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:38.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.933+0000 7ff73b52e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7340caeb0 con 0x7ff7340c9bc0 2023-10-16T10:01:38.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.933+0000 7ff73b52e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7340ca020 con 0x7ff720004d00 2023-10-16T10:01:38.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.933+0000 7ff73b52e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7340ca230 con 0x7ff7341585c0 2023-10-16T10:01:38.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.933+0000 7ff739d2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7340c9bc0 0x7ff7340cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:38.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.934+0000 7ff739d2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7340c9bc0 0x7ff7340cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48814/0 (socket says 172.21.15.84:48814) 2023-10-16T10:01:38.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.934+0000 7ff739d2b640 1 -- 172.21.15.84:0/3832030712 learned_addr learned my addr 172.21.15.84:0/3832030712 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:38.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.934+0000 7ff73a52c640 1 --2- 172.21.15.84:0/3832030712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff720004d00 0x7ff7341f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:38.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.934+0000 7ff73ad2d640 1 --2- 172.21.15.84:0/3832030712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7341585c0 0x7ff734158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:38.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.934+0000 7ff739d2b640 1 -- 172.21.15.84:0/3832030712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7341585c0 msgr2=0x7ff734158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:38.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.934+0000 7ff739d2b640 1 --2- 172.21.15.84:0/3832030712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7341585c0 0x7ff734158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:38.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.934+0000 7ff739d2b640 1 -- 172.21.15.84:0/3832030712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff720004d00 msgr2=0x7ff7341f22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:38.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.934+0000 7ff739d2b640 1 --2- 172.21.15.84:0/3832030712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff720004d00 0x7ff7341f22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:38.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.934+0000 7ff739d2b640 1 -- 172.21.15.84:0/3832030712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7340ca510 con 0x7ff7340c9bc0 2023-10-16T10:01:38.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.934+0000 7ff73a52c640 1 --2- 172.21.15.84:0/3832030712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff720004d00 0x7ff7341f22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:38.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.934+0000 7ff739d2b640 1 --2- 172.21.15.84:0/3832030712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7340c9bc0 0x7ff7340cc1e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff72400e780 tx=0x7ff72400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:38.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.936+0000 7ff72affd640 1 -- 172.21.15.84:0/3832030712 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72400c6c0 con 0x7ff7340c9bc0 2023-10-16T10:01:38.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.936+0000 7ff72affd640 1 -- 172.21.15.84:0/3832030712 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72400c840 con 0x7ff7340c9bc0 2023-10-16T10:01:38.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.936+0000 7ff72affd640 1 -- 172.21.15.84:0/3832030712 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72400cbf0 con 0x7ff7340c9bc0 2023-10-16T10:01:38.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.936+0000 7ff73b52e640 1 -- 172.21.15.84:0/3832030712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7340ca7f0 con 0x7ff7340c9bc0 2023-10-16T10:01:38.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.937+0000 7ff73b52e640 1 -- 172.21.15.84:0/3832030712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff734155f70 con 0x7ff7340c9bc0 2023-10-16T10:01:38.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.938+0000 7ff73b52e640 1 -- 172.21.15.84:0/3832030712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7340caff0 con 0x7ff7340c9bc0 2023-10-16T10:01:38.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.939+0000 7ff72affd640 1 -- 172.21.15.84:0/3832030712 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7ff72400cd50 con 0x7ff7340c9bc0 2023-10-16T10:01:38.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.939+0000 7ff72affd640 1 --2- 172.21.15.84:0/3832030712 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff71806b070 0x7ff71806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:38.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.940+0000 7ff72affd640 1 -- 172.21.15.84:0/3832030712 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff724089ac0 con 0x7ff7340c9bc0 2023-10-16T10:01:38.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.940+0000 7ff73a52c640 1 --2- 172.21.15.84:0/3832030712 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff71806b070 0x7ff71806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:38.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.940+0000 7ff73a52c640 1 --2- 172.21.15.84:0/3832030712 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff71806b070 0x7ff71806d530 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff730006fd0 tx=0x7ff730008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:38.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:38.944+0000 7ff72affd640 1 -- 172.21.15.84:0/3832030712 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff7340caff0 con 0x7ff7340c9bc0 2023-10-16T10:01:39.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:39.160+0000 7ff73b52e640 1 -- 172.21.15.84:0/3832030712 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7340d1d00 con 0x7ff71806b070 2023-10-16T10:01:39.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:39.400+0000 7ff72affd640 1 -- 172.21.15.84:0/3832030712 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7ff73000be90 con 0x7ff71806b070 2023-10-16T10:01:39.401 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path /snap_test_dir1/dir3 2023-10-16T10:01:39.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:39.403+0000 7ff73b52e640 1 -- 172.21.15.84:0/3832030712 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff71806b070 msgr2=0x7ff71806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:39.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:39.403+0000 7ff73b52e640 1 --2- 172.21.15.84:0/3832030712 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff71806b070 0x7ff71806d530 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff730006fd0 tx=0x7ff730008040 comp rx=0 tx=0).stop 2023-10-16T10:01:39.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:39.403+0000 7ff73b52e640 1 -- 172.21.15.84:0/3832030712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7340c9bc0 msgr2=0x7ff7340cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:39.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:39.403+0000 7ff73b52e640 1 --2- 172.21.15.84:0/3832030712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7340c9bc0 0x7ff7340cc1e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff72400e780 tx=0x7ff72400ec40 comp rx=0 tx=0).stop 2023-10-16T10:01:39.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:39.403+0000 7ff73b52e640 1 -- 172.21.15.84:0/3832030712 shutdown_connections 2023-10-16T10:01:39.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:39.403+0000 7ff73b52e640 1 --2- 172.21.15.84:0/3832030712 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7ff71806b070 0x7ff71806d530 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:39.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:39.403+0000 7ff73b52e640 1 --2- 172.21.15.84:0/3832030712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7341585c0 0x7ff734158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:39.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:39.403+0000 7ff73b52e640 1 --2- 172.21.15.84:0/3832030712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7340c9bc0 0x7ff7340cc1e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:39.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:39.403+0000 7ff73b52e640 1 --2- 172.21.15.84:0/3832030712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff720004d00 0x7ff7341f22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:39.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:39.403+0000 7ff73b52e640 1 -- 172.21.15.84:0/3832030712 >> 172.21.15.84:0/3832030712 conn(0x7ff7340cf570 msgr2=0x7ff73415bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:39.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:39.403+0000 7ff73b52e640 1 -- 172.21.15.84:0/3832030712 shutdown_connections 2023-10-16T10:01:39.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:39.403+0000 7ff73b52e640 1 -- 172.21.15.84:0/3832030712 wait complete. 2023-10-16T10:01:39.423 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2023-10-16T10:01:39.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:39.424 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-10-16T10:01:39.477 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_01_UTC 2023-10-16T10:01:39.478 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_01_UTC 2023-10-16T10:01:39.478 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_01_UTC 2023-10-16T10:01:39.478 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_01_UTC 2023-10-16T10:01:39.478 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_01_UTC 2023-10-16T10:01:39.479 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:39.479 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_00_UTC 2023-10-16T10:01:39.479 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_01_UTC 2023-10-16T10:01:39.479 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_01_UTC 2023-10-16T10:01:39.479 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_01_UTC 2023-10-16T10:01:39.479 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_01_UTC 2023-10-16T10:01:39.480 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_51_01_UTC 2023-10-16T10:01:39.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:39.481 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_51_01_UTC) 2023-10-16T10:01:39.545 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_52_01_UTC 2023-10-16T10:01:39.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:39.546 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_52_01_UTC) 2023-10-16T10:01:39.611 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_53_01_UTC 2023-10-16T10:01:39.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:39.612 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_53_01_UTC) 2023-10-16T10:01:39.684 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_54_01_UTC 2023-10-16T10:01:39.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:39.685 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_54_01_UTC) 2023-10-16T10:01:39.758 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_55_01_UTC 2023-10-16T10:01:39.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:39.759 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_55_01_UTC) 2023-10-16T10:01:39.829 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:39.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:39.830 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_56_00_UTC) 2023-10-16T10:01:39.899 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_57_00_UTC 2023-10-16T10:01:39.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:39.900 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_57_00_UTC) 2023-10-16T10:01:40.012 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_58_01_UTC 2023-10-16T10:01:40.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:40.013 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_58_01_UTC) 2023-10-16T10:01:40.117 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_59_01_UTC 2023-10-16T10:01:40.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:40.118 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_59_01_UTC) 2023-10-16T10:01:40.215 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-10_00_01_UTC 2023-10-16T10:01:40.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:40.216 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-10_00_01_UTC) 2023-10-16T10:01:40.305 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-10_01_01_UTC 2023-10-16T10:01:40.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:40.306 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-10_01_01_UTC) 2023-10-16T10:01:40.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:40.381 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2023-10-16T10:01:40.424 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir4 1m 2023-10-16T10:01:40.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.666+0000 7fd16e5a9640 1 Processor -- start 2023-10-16T10:01:40.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.667+0000 7fd16e5a9640 1 -- start start 2023-10-16T10:01:40.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.668+0000 7fd16e5a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd16815ad70 0x7fd168069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:40.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.668+0000 7fd16e5a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd1680c9ca0 0x7fd16806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:40.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.668+0000 7fd16e5a9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd1680ca3b0 0x7fd1680ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:40.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.668+0000 7fd16e5a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1680cad80 con 0x7fd1680c9ca0 2023-10-16T10:01:40.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.668+0000 7fd16e5a9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1680cb030 con 0x7fd1680ca3b0 2023-10-16T10:01:40.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.668+0000 7fd16e5a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1680c91c0 con 0x7fd16815ad70 2023-10-16T10:01:40.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.668+0000 7fd15ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd1680c9ca0 0x7fd16806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:40.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.668+0000 7fd15ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd1680c9ca0 0x7fd16806a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48824/0 (socket says 172.21.15.84:48824) 2023-10-16T10:01:40.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.669+0000 7fd15ffff640 1 -- 172.21.15.84:0/730842611 learned_addr learned my addr 172.21.15.84:0/730842611 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:40.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.669+0000 7fd16cb46640 1 --2- 172.21.15.84:0/730842611 >> v2:172.21.15.157:3300/0 conn(0x7fd1680ca3b0 0x7fd1680ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:40.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.669+0000 7fd15ffff640 1 -- 172.21.15.84:0/730842611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd16815ad70 msgr2=0x7fd168069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:40.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.669+0000 7fd15ffff640 1 --2- 172.21.15.84:0/730842611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd16815ad70 0x7fd168069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:40.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.669+0000 7fd15ffff640 1 -- 172.21.15.84:0/730842611 >> v2:172.21.15.157:3300/0 conn(0x7fd1680ca3b0 msgr2=0x7fd1680ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:40.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.669+0000 7fd15ffff640 1 --2- 172.21.15.84:0/730842611 >> v2:172.21.15.157:3300/0 conn(0x7fd1680ca3b0 0x7fd1680ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:40.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.669+0000 7fd15ffff640 1 -- 172.21.15.84:0/730842611 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1680c94a0 con 0x7fd1680c9ca0 2023-10-16T10:01:40.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.670+0000 7fd15ffff640 1 --2- 172.21.15.84:0/730842611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1680c9ca0 0x7fd16806a2e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd1680cf820 tx=0x7fd15002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8671fedf329c552 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:40.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.670+0000 7fd1677fe640 1 -- 172.21.15.84:0/730842611 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd150031870 con 0x7fd1680c9ca0 2023-10-16T10:01:40.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.670+0000 7fd1677fe640 1 -- 172.21.15.84:0/730842611 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1500319d0 con 0x7fd1680c9ca0 2023-10-16T10:01:40.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.670+0000 7fd1677fe640 1 -- 172.21.15.84:0/730842611 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd150031d60 con 0x7fd1680c9ca0 2023-10-16T10:01:40.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.671+0000 7fd16e5a9640 1 -- 172.21.15.84:0/730842611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1680c9ca0 msgr2=0x7fd16806a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:40.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.671+0000 7fd16e5a9640 1 --2- 172.21.15.84:0/730842611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1680c9ca0 0x7fd16806a2e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd1680cf820 tx=0x7fd15002b910 comp rx=0 tx=0).stop 2023-10-16T10:01:40.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.671+0000 7fd16e5a9640 1 -- 172.21.15.84:0/730842611 shutdown_connections 2023-10-16T10:01:40.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.671+0000 7fd16e5a9640 1 --2- 172.21.15.84:0/730842611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1680c9ca0 0x7fd16806a2e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:40.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.671+0000 7fd16e5a9640 1 --2- 172.21.15.84:0/730842611 >> v2:172.21.15.157:3300/0 conn(0x7fd1680ca3b0 0x7fd1680ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:40.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.671+0000 7fd16e5a9640 1 --2- 172.21.15.84:0/730842611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd16815ad70 0x7fd168069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:40.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.671+0000 7fd16e5a9640 1 -- 172.21.15.84:0/730842611 >> 172.21.15.84:0/730842611 conn(0x7fd1680ce9b0 msgr2=0x7fd1680cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:40.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.671+0000 7fd16e5a9640 1 -- 172.21.15.84:0/730842611 shutdown_connections 2023-10-16T10:01:40.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.671+0000 7fd16e5a9640 1 -- 172.21.15.84:0/730842611 wait complete. 2023-10-16T10:01:40.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.673+0000 7fd16e5a9640 1 Processor -- start 2023-10-16T10:01:40.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.673+0000 7fd16e5a9640 1 -- start start 2023-10-16T10:01:40.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.674+0000 7fd16e5a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1680c9ca0 0x7fd16814b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:40.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.674+0000 7fd16e5a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1680ca3b0 0x7fd16815c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:40.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.674+0000 7fd16e5a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd16815ad70 0x7fd16815cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:40.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.674+0000 7fd16e5a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1680b1dc0 con 0x7fd16815ad70 2023-10-16T10:01:40.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.674+0000 7fd16e5a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1680b11b0 con 0x7fd1680c9ca0 2023-10-16T10:01:40.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.674+0000 7fd16e5a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1680b1490 con 0x7fd1680ca3b0 2023-10-16T10:01:40.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.674+0000 7fd16cb46640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd16815ad70 0x7fd16815cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:40.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.674+0000 7fd16cb46640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd16815ad70 0x7fd16815cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48832/0 (socket says 172.21.15.84:48832) 2023-10-16T10:01:40.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.674+0000 7fd16cb46640 1 -- 172.21.15.84:0/2249067556 learned_addr learned my addr 172.21.15.84:0/2249067556 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:40.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.675+0000 7fd167fff640 1 --2- 172.21.15.84:0/2249067556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1680c9ca0 0x7fd16814b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:40.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.675+0000 7fd16cb46640 1 -- 172.21.15.84:0/2249067556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1680ca3b0 msgr2=0x7fd16815c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:40.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.675+0000 7fd15ffff640 1 --2- 172.21.15.84:0/2249067556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1680ca3b0 0x7fd16815c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:40.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.675+0000 7fd16cb46640 1 --2- 172.21.15.84:0/2249067556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1680ca3b0 0x7fd16815c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:40.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.675+0000 7fd16cb46640 1 -- 172.21.15.84:0/2249067556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1680c9ca0 msgr2=0x7fd16814b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:40.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.675+0000 7fd16cb46640 1 --2- 172.21.15.84:0/2249067556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1680c9ca0 0x7fd16814b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:40.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.675+0000 7fd16cb46640 1 -- 172.21.15.84:0/2249067556 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1500094d0 con 0x7fd16815ad70 2023-10-16T10:01:40.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.675+0000 7fd167fff640 1 --2- 172.21.15.84:0/2249067556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1680c9ca0 0x7fd16814b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:40.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.675+0000 7fd16cb46640 1 --2- 172.21.15.84:0/2249067556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd16815ad70 0x7fd16815cb70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd16000cd70 tx=0x7fd16000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:40.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.675+0000 7fd1657fa640 1 -- 172.21.15.84:0/2249067556 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160003b90 con 0x7fd16815ad70 2023-10-16T10:01:40.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.675+0000 7fd1657fa640 1 -- 172.21.15.84:0/2249067556 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd160003cf0 con 0x7fd16815ad70 2023-10-16T10:01:40.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.676+0000 7fd16e5a9640 1 -- 172.21.15.84:0/2249067556 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1680b1770 con 0x7fd16815ad70 2023-10-16T10:01:40.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.676+0000 7fd1657fa640 1 -- 172.21.15.84:0/2249067556 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160004240 con 0x7fd16815ad70 2023-10-16T10:01:40.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.676+0000 7fd16e5a9640 1 -- 172.21.15.84:0/2249067556 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd16814b550 con 0x7fd16815ad70 2023-10-16T10:01:40.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.677+0000 7fd16e5a9640 1 -- 172.21.15.84:0/2249067556 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1680b1f30 con 0x7fd16815ad70 2023-10-16T10:01:40.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.678+0000 7fd1657fa640 1 -- 172.21.15.84:0/2249067556 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fd1600043a0 con 0x7fd16815ad70 2023-10-16T10:01:40.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.679+0000 7fd1657fa640 1 --2- 172.21.15.84:0/2249067556 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd14806b070 0x7fd14806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:40.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.679+0000 7fd1657fa640 1 -- 172.21.15.84:0/2249067556 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fd160012070 con 0x7fd16815ad70 2023-10-16T10:01:40.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.679+0000 7fd167fff640 1 --2- 172.21.15.84:0/2249067556 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd14806b070 0x7fd14806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:40.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.680+0000 7fd167fff640 1 --2- 172.21.15.84:0/2249067556 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd14806b070 0x7fd14806d530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd168158740 tx=0x7fd158008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:40.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.683+0000 7fd1657fa640 1 -- 172.21.15.84:0/2249067556 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd1600551c0 con 0x7fd16815ad70 2023-10-16T10:01:40.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:40.900+0000 7fd16e5a9640 1 -- 172.21.15.84:0/2249067556 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1680c9780 con 0x7fd14806b070 2023-10-16T10:01:41.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:41.078+0000 7fd1657fa640 1 -- 172.21.15.84:0/2249067556 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fd15800a460 con 0x7fd14806b070 2023-10-16T10:01:41.079 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path /snap_test_dir1/dir4 2023-10-16T10:01:41.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:41.080+0000 7fd16e5a9640 1 -- 172.21.15.84:0/2249067556 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd14806b070 msgr2=0x7fd14806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:41.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:41.080+0000 7fd16e5a9640 1 --2- 172.21.15.84:0/2249067556 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd14806b070 0x7fd14806d530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd168158740 tx=0x7fd158008040 comp rx=0 tx=0).stop 2023-10-16T10:01:41.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:41.080+0000 7fd16e5a9640 1 -- 172.21.15.84:0/2249067556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd16815ad70 msgr2=0x7fd16815cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:41.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:41.080+0000 7fd16e5a9640 1 --2- 172.21.15.84:0/2249067556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd16815ad70 0x7fd16815cb70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd16000cd70 tx=0x7fd16000a660 comp rx=0 tx=0).stop 2023-10-16T10:01:41.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:41.081+0000 7fd16e5a9640 1 -- 172.21.15.84:0/2249067556 shutdown_connections 2023-10-16T10:01:41.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:41.081+0000 7fd16e5a9640 1 --2- 172.21.15.84:0/2249067556 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd14806b070 0x7fd14806d530 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:41.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:41.081+0000 7fd16e5a9640 1 --2- 172.21.15.84:0/2249067556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd16815ad70 0x7fd16815cb70 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:41.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:41.081+0000 7fd16e5a9640 1 --2- 172.21.15.84:0/2249067556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1680ca3b0 0x7fd16815c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:41.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:41.081+0000 7fd16e5a9640 1 --2- 172.21.15.84:0/2249067556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1680c9ca0 0x7fd16814b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:41.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:41.081+0000 7fd16e5a9640 1 -- 172.21.15.84:0/2249067556 >> 172.21.15.84:0/2249067556 conn(0x7fd1680ce9b0 msgr2=0x7fd1680afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:41.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:41.081+0000 7fd16e5a9640 1 -- 172.21.15.84:0/2249067556 shutdown_connections 2023-10-16T10:01:41.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:41.081+0000 7fd16e5a9640 1 -- 172.21.15.84:0/2249067556 wait complete. 2023-10-16T10:01:41.098 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2023-10-16T10:01:41.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:41.099 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-10-16T10:01:41.146 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T10:01:41.147 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T10:01:41.147 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T10:01:41.147 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:41.147 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T10:01:41.147 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_01_UTC 2023-10-16T10:01:41.148 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_00_UTC 2023-10-16T10:01:41.148 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T10:01:41.148 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_00_UTC 2023-10-16T10:01:41.148 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_00_UTC 2023-10-16T10:01:41.148 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:41.149 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_51_00_UTC 2023-10-16T10:01:41.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:41.150 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_51_00_UTC) 2023-10-16T10:01:41.270 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_52_00_UTC 2023-10-16T10:01:41.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:41.271 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_52_00_UTC) 2023-10-16T10:01:41.323 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_53_00_UTC 2023-10-16T10:01:41.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:41.324 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_53_00_UTC) 2023-10-16T10:01:41.380 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:41.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:41.381 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_54_00_UTC) 2023-10-16T10:01:41.444 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_55_00_UTC 2023-10-16T10:01:41.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:41.445 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_55_00_UTC) 2023-10-16T10:01:41.510 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_56_01_UTC 2023-10-16T10:01:41.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:41.512 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_56_01_UTC) 2023-10-16T10:01:41.590 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_57_00_UTC 2023-10-16T10:01:41.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:41.591 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_57_00_UTC) 2023-10-16T10:01:41.657 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_58_00_UTC 2023-10-16T10:01:41.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:41.659 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_58_00_UTC) 2023-10-16T10:01:41.723 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_59_00_UTC 2023-10-16T10:01:41.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:41.724 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_59_00_UTC) 2023-10-16T10:01:41.792 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-10_00_00_UTC 2023-10-16T10:01:41.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:41.793 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-10_00_00_UTC) 2023-10-16T10:01:41.864 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:41.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:41.866 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-10_01_00_UTC) 2023-10-16T10:01:41.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:41.937 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2023-10-16T10:01:42.000 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir5 1m 2023-10-16T10:01:42.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.211+0000 7fc1de52c640 1 Processor -- start 2023-10-16T10:01:42.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.211+0000 7fc1de52c640 1 -- start start 2023-10-16T10:01:42.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.212+0000 7fc1de52c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc1d815ad70 0x7fc1d8069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:42.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.212+0000 7fc1de52c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1d80caac0 0x7fc1d806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:42.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.212+0000 7fc1de52c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d80cb030 con 0x7fc1d80c9ca0 2023-10-16T10:01:42.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.212+0000 7fc1de52c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1d80c91c0 con 0x7fc1d815ad70 2023-10-16T10:01:42.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.212+0000 7fc1de52c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d80c94a0 con 0x7fc1d80caac0 2023-10-16T10:01:42.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.213+0000 7fc1d77fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc1d80c9ca0 0x7fc1d80ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35538/0 (socket says 172.21.15.84:35538) 2023-10-16T10:01:42.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.213+0000 7fc1d77fe640 1 -- 172.21.15.84:0/404672277 learned_addr learned my addr 172.21.15.84:0/404672277 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:42.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.213+0000 7fc1d7fff640 1 --2- 172.21.15.84:0/404672277 >> v2:172.21.15.157:3300/0 conn(0x7fc1d815ad70 0x7fc1d8069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:42.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.213+0000 7fc1dcac9640 1 --2- 172.21.15.84:0/404672277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1d80caac0 0x7fc1d806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:42.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.213+0000 7fc1d6ffd640 1 -- 172.21.15.84:0/404672277 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2176572719 0 0) 0x7fc1d80cb030 con 0x7fc1d80c9ca0 2023-10-16T10:01:42.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.213+0000 7fc1d6ffd640 1 -- 172.21.15.84:0/404672277 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1c4003520 con 0x7fc1d80c9ca0 2023-10-16T10:01:42.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.213+0000 7fc1d6ffd640 1 -- 172.21.15.84:0/404672277 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2833806633 0 0) 0x7fc1c4003520 con 0x7fc1d80c9ca0 2023-10-16T10:01:42.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.213+0000 7fc1d6ffd640 1 -- 172.21.15.84:0/404672277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1d80caac0 msgr2=0x7fc1d806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:42.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.213+0000 7fc1d6ffd640 1 --2- 172.21.15.84:0/404672277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1d80caac0 0x7fc1d806a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:42.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.213+0000 7fc1d6ffd640 1 -- 172.21.15.84:0/404672277 >> v2:172.21.15.157:3300/0 conn(0x7fc1d815ad70 msgr2=0x7fc1d8069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:42.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.213+0000 7fc1d6ffd640 1 --2- 172.21.15.84:0/404672277 >> v2:172.21.15.157:3300/0 conn(0x7fc1d815ad70 0x7fc1d8069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:42.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.213+0000 7fc1d6ffd640 1 -- 172.21.15.84:0/404672277 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d80c9780 con 0x7fc1d80c9ca0 2023-10-16T10:01:42.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.213+0000 7fc1d6ffd640 1 -- 172.21.15.84:0/404672277 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc1c0001670 con 0x7fc1d80c9ca0 2023-10-16T10:01:42.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.213+0000 7fc1d6ffd640 1 -- 172.21.15.84:0/404672277 >> v1:172.21.15.84:6789/0 conn(0x7fc1d80c9ca0 legacy=0x7fc1d80ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:42.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.214+0000 7fc1d6ffd640 1 --2- 172.21.15.84:0/404672277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1c4003970 0x7fc1c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:42.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.214+0000 7fc1d6ffd640 1 --2- 172.21.15.84:0/404672277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c40047c0 0x7fc1c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:42.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.214+0000 7fc1d6ffd640 1 --2- 172.21.15.84:0/404672277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1c4005180 0x7fc1c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:42.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.214+0000 7fc1d6ffd640 1 -- 172.21.15.84:0/404672277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d80c91c0 con 0x7fc1c4005180 2023-10-16T10:01:42.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.214+0000 7fc1d6ffd640 1 -- 172.21.15.84:0/404672277 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d80c94a0 con 0x7fc1c4003970 2023-10-16T10:01:42.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.214+0000 7fc1d6ffd640 1 -- 172.21.15.84:0/404672277 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c400bfc0 con 0x7fc1c40047c0 2023-10-16T10:01:42.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.214+0000 7fc1dcac9640 1 --2- 172.21.15.84:0/404672277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1d80caac0 0x7fc1d806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:01:42.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.214+0000 7fc1d7fff640 1 --2- 172.21.15.84:0/404672277 >> v2:172.21.15.157:3300/0 conn(0x7fc1d815ad70 0x7fc1d8069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:01:42.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.214+0000 7fc1dcac9640 1 --2- 172.21.15.84:0/404672277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1c4005180 0x7fc1c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:42.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.214+0000 7fc1d77fe640 1 --2- 172.21.15.84:0/404672277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c40047c0 0x7fc1c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:42.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.214+0000 7fc1dcac9640 1 -- 172.21.15.84:0/404672277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c40047c0 msgr2=0x7fc1c4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:42.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.214+0000 7fc1dcac9640 1 --2- 172.21.15.84:0/404672277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c40047c0 0x7fc1c4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:42.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.214+0000 7fc1d7fff640 1 --2- 172.21.15.84:0/404672277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1c4003970 0x7fc1c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:42.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.214+0000 7fc1dcac9640 1 -- 172.21.15.84:0/404672277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1c4003970 msgr2=0x7fc1c4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:42.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.214+0000 7fc1dcac9640 1 --2- 172.21.15.84:0/404672277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1c4003970 0x7fc1c4003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:42.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.214+0000 7fc1dcac9640 1 -- 172.21.15.84:0/404672277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c4003520 con 0x7fc1c4005180 2023-10-16T10:01:42.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.215+0000 7fc1dcac9640 1 --2- 172.21.15.84:0/404672277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1c4005180 0x7fc1c400b980 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009f30 tx=0x7fc1c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42301601bd2cba5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:42.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.216+0000 7fc1d6ffd640 1 -- 172.21.15.84:0/404672277 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d002cc70 con 0x7fc1c4005180 2023-10-16T10:01:42.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.216+0000 7fc1d6ffd640 1 -- 172.21.15.84:0/404672277 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d002cdd0 con 0x7fc1c4005180 2023-10-16T10:01:42.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.216+0000 7fc1d6ffd640 1 -- 172.21.15.84:0/404672277 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0003bb0 con 0x7fc1c4005180 2023-10-16T10:01:42.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.216+0000 7fc1de52c640 1 -- 172.21.15.84:0/404672277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1c4005180 msgr2=0x7fc1c400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:42.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.216+0000 7fc1de52c640 1 --2- 172.21.15.84:0/404672277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1c4005180 0x7fc1c400b980 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009f30 tx=0x7fc1c40057f0 comp rx=0 tx=0).stop 2023-10-16T10:01:42.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.216+0000 7fc1dcac9640 1 -- 172.21.15.84:0/404672277 reap_dead start 2023-10-16T10:01:42.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.216+0000 7fc1de52c640 1 -- 172.21.15.84:0/404672277 shutdown_connections 2023-10-16T10:01:42.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.216+0000 7fc1de52c640 1 -- 172.21.15.84:0/404672277 >> 172.21.15.84:0/404672277 conn(0x7fc1d80ce9b0 msgr2=0x7fc1d80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:42.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.216+0000 7fc1de52c640 1 -- 172.21.15.84:0/404672277 shutdown_connections 2023-10-16T10:01:42.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.216+0000 7fc1de52c640 1 -- 172.21.15.84:0/404672277 wait complete. 2023-10-16T10:01:42.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.218+0000 7fc1de52c640 1 Processor -- start 2023-10-16T10:01:42.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.218+0000 7fc1de52c640 1 -- start start 2023-10-16T10:01:42.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.219+0000 7fc1de52c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c4005180 0x7fc1d80c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:42.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.219+0000 7fc1de52c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1d80b2240 0x7fc1d814e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:42.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.219+0000 7fc1de52c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1d80c4180 0x7fc1d80c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:42.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.219+0000 7fc1de52c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d80c4b70 con 0x7fc1d80c4180 2023-10-16T10:01:42.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.219+0000 7fc1de52c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d80c3ce0 con 0x7fc1d80b2240 2023-10-16T10:01:42.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.219+0000 7fc1de52c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d80c6f70 con 0x7fc1c4005180 2023-10-16T10:01:42.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.220+0000 7fc1dcac9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1d80c4180 0x7fc1d80c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:42.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.220+0000 7fc1dcac9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1d80c4180 0x7fc1d80c4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48856/0 (socket says 172.21.15.84:48856) 2023-10-16T10:01:42.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.220+0000 7fc1dcac9640 1 -- 172.21.15.84:0/1322508010 learned_addr learned my addr 172.21.15.84:0/1322508010 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:42.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.220+0000 7fc1dcac9640 1 -- 172.21.15.84:0/1322508010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c4005180 msgr2=0x7fc1d80c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:42.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.220+0000 7fc1d77fe640 1 --2- 172.21.15.84:0/1322508010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1d80b2240 0x7fc1d814e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:42.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.220+0000 7fc1dcac9640 1 --2- 172.21.15.84:0/1322508010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c4005180 0x7fc1d80c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:42.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.220+0000 7fc1dcac9640 1 -- 172.21.15.84:0/1322508010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1d80b2240 msgr2=0x7fc1d814e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:42.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.220+0000 7fc1dcac9640 1 --2- 172.21.15.84:0/1322508010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1d80b2240 0x7fc1d814e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:42.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.220+0000 7fc1dcac9640 1 -- 172.21.15.84:0/1322508010 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d80c3720 con 0x7fc1d80c4180 2023-10-16T10:01:42.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.220+0000 7fc1dcac9640 1 --2- 172.21.15.84:0/1322508010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1d80c4180 0x7fc1d80c4630 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc1d002c6e0 tx=0x7fc1d0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:42.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.221+0000 7fc1d4ff9640 1 -- 172.21.15.84:0/1322508010 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d003c070 con 0x7fc1d80c4180 2023-10-16T10:01:42.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.221+0000 7fc1d4ff9640 1 -- 172.21.15.84:0/1322508010 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d0003e30 con 0x7fc1d80c4180 2023-10-16T10:01:42.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.221+0000 7fc1d4ff9640 1 -- 172.21.15.84:0/1322508010 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00044f0 con 0x7fc1d80c4180 2023-10-16T10:01:42.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.221+0000 7fc1de52c640 1 -- 172.21.15.84:0/1322508010 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d80c3a00 con 0x7fc1d80c4180 2023-10-16T10:01:42.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.221+0000 7fc1de52c640 1 -- 172.21.15.84:0/1322508010 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d814b860 con 0x7fc1d80c4180 2023-10-16T10:01:42.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.222+0000 7fc1de52c640 1 -- 172.21.15.84:0/1322508010 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1d804e160 con 0x7fc1d80c4180 2023-10-16T10:01:42.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.223+0000 7fc1d4ff9640 1 -- 172.21.15.84:0/1322508010 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fc1d0004650 con 0x7fc1d80c4180 2023-10-16T10:01:42.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.224+0000 7fc1d4ff9640 1 --2- 172.21.15.84:0/1322508010 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fc1b806b070 0x7fc1b806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:42.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.224+0000 7fc1d4ff9640 1 -- 172.21.15.84:0/1322508010 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6631+0+0 (secure 0 0 0) 0x7fc1d00aaf70 con 0x7fc1d80c4180 2023-10-16T10:01:42.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.224+0000 7fc1d7fff640 1 --2- 172.21.15.84:0/1322508010 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fc1b806b070 0x7fc1b806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:42.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.225+0000 7fc1d7fff640 1 --2- 172.21.15.84:0/1322508010 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fc1b806b070 0x7fc1b806d530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8006e80 tx=0x7fc1c8006df0 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:42.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.229+0000 7fc1d4ff9640 1 -- 172.21.15.84:0/1322508010 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc1d00759f0 con 0x7fc1d80c4180 2023-10-16T10:01:42.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.443+0000 7fc1de52c640 1 -- 172.21.15.84:0/1322508010 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1d806a2e0 con 0x7fc1b806b070 2023-10-16T10:01:42.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.678+0000 7fc1d4ff9640 1 -- 172.21.15.84:0/1322508010 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fc1c8009e80 con 0x7fc1b806b070 2023-10-16T10:01:42.680 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path /snap_test_dir1/dir5 2023-10-16T10:01:42.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.682+0000 7fc1de52c640 1 -- 172.21.15.84:0/1322508010 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fc1b806b070 msgr2=0x7fc1b806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:42.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.682+0000 7fc1de52c640 1 --2- 172.21.15.84:0/1322508010 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fc1b806b070 0x7fc1b806d530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8006e80 tx=0x7fc1c8006df0 comp rx=0 tx=0).stop 2023-10-16T10:01:42.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.682+0000 7fc1de52c640 1 -- 172.21.15.84:0/1322508010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1d80c4180 msgr2=0x7fc1d80c4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:42.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.682+0000 7fc1de52c640 1 --2- 172.21.15.84:0/1322508010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1d80c4180 0x7fc1d80c4630 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc1d002c6e0 tx=0x7fc1d0004ab0 comp rx=0 tx=0).stop 2023-10-16T10:01:42.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.682+0000 7fc1de52c640 1 -- 172.21.15.84:0/1322508010 shutdown_connections 2023-10-16T10:01:42.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.682+0000 7fc1de52c640 1 --2- 172.21.15.84:0/1322508010 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fc1b806b070 0x7fc1b806d530 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:42.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.682+0000 7fc1de52c640 1 --2- 172.21.15.84:0/1322508010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1d80c4180 0x7fc1d80c4630 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:42.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.682+0000 7fc1de52c640 1 --2- 172.21.15.84:0/1322508010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1d80b2240 0x7fc1d814e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:42.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.682+0000 7fc1de52c640 1 --2- 172.21.15.84:0/1322508010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1c4005180 0x7fc1d80c2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:42.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.682+0000 7fc1de52c640 1 -- 172.21.15.84:0/1322508010 >> 172.21.15.84:0/1322508010 conn(0x7fc1d80ce9b0 msgr2=0x7fc1d80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:42.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.682+0000 7fc1de52c640 1 -- 172.21.15.84:0/1322508010 shutdown_connections 2023-10-16T10:01:42.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:42.682+0000 7fc1de52c640 1 -- 172.21.15.84:0/1322508010 wait complete. 2023-10-16T10:01:42.701 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2023-10-16T10:01:42.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:42.702 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-10-16T10:01:42.754 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T10:01:42.754 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T10:01:42.754 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_01_UTC 2023-10-16T10:01:42.754 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:42.755 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T10:01:42.755 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_01_UTC 2023-10-16T10:01:42.755 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_01_UTC 2023-10-16T10:01:42.755 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_01_UTC 2023-10-16T10:01:42.755 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_01_UTC 2023-10-16T10:01:42.756 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_01_UTC 2023-10-16T10:01:42.756 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_01_UTC 2023-10-16T10:01:42.757 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_51_00_UTC 2023-10-16T10:01:42.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:42.758 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_51_00_UTC) 2023-10-16T10:01:42.832 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_52_00_UTC 2023-10-16T10:01:42.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:42.833 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_52_00_UTC) 2023-10-16T10:01:42.905 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_53_01_UTC 2023-10-16T10:01:42.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:42.906 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_53_01_UTC) 2023-10-16T10:01:42.999 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:42.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:43.000 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_54_00_UTC) 2023-10-16T10:01:43.075 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_55_00_UTC 2023-10-16T10:01:43.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:43.076 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_55_00_UTC) 2023-10-16T10:01:43.126 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_56_01_UTC 2023-10-16T10:01:43.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:43.127 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_56_01_UTC) 2023-10-16T10:01:43.186 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_57_01_UTC 2023-10-16T10:01:43.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:43.187 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_57_01_UTC) 2023-10-16T10:01:43.239 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_58_01_UTC 2023-10-16T10:01:43.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:43.240 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_58_01_UTC) 2023-10-16T10:01:43.293 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_59_01_UTC 2023-10-16T10:01:43.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:43.294 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_59_01_UTC) 2023-10-16T10:01:43.352 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-10_00_01_UTC 2023-10-16T10:01:43.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:43.352 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-10_00_01_UTC) 2023-10-16T10:01:43.415 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-10_01_01_UTC 2023-10-16T10:01:43.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:43.416 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-10_01_01_UTC) 2023-10-16T10:01:43.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:43.497 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2023-10-16T10:01:43.543 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir6 1m 2023-10-16T10:01:43.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.797+0000 7fd574e64640 1 Processor -- start 2023-10-16T10:01:43.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.797+0000 7fd574e64640 1 -- start start 2023-10-16T10:01:43.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.798+0000 7fd574e64640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd570159e90 0x7fd5700ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:43.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.798+0000 7fd574e64640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd57006c140 0x7fd57006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:43.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.798+0000 7fd574e64640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd570070360 con 0x7fd570159ab0 2023-10-16T10:01:43.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.798+0000 7fd574e64640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd57006cb10 con 0x7fd57006c140 2023-10-16T10:01:43.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.798+0000 7fd574e64640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57015d890 con 0x7fd570159e90 2023-10-16T10:01:43.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.799+0000 7fd56ed9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd57006c140 0x7fd57006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:43.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.799+0000 7fd56dd9b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd570159e90 0x7fd5700ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:43.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.799+0000 7fd56ed9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd57006c140 0x7fd57006c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:51424/0 (socket says 172.21.15.84:51424) 2023-10-16T10:01:43.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.799+0000 7fd56ed9d640 1 -- 172.21.15.84:0/3653401879 learned_addr learned my addr 172.21.15.84:0/3653401879 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:43.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.799+0000 7fd56e59c640 1 --1- 172.21.15.84:0/3653401879 >> v1:172.21.15.157:6789/0 conn(0x7fd570159ab0 0x7fd57006fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:36260/0 (socket says 172.21.15.84:36260) 2023-10-16T10:01:43.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.800+0000 7fd56ed9d640 1 -- 172.21.15.84:0/3653401879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd570159e90 msgr2=0x7fd5700ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:43.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.800+0000 7fd56ed9d640 1 --2- 172.21.15.84:0/3653401879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd570159e90 0x7fd5700ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:43.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.800+0000 7fd56ed9d640 1 -- 172.21.15.84:0/3653401879 >> v1:172.21.15.157:6789/0 conn(0x7fd570159ab0 legacy=0x7fd57006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:43.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.800+0000 7fd56ed9d640 1 -- 172.21.15.84:0/3653401879 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5700cd1f0 con 0x7fd57006c140 2023-10-16T10:01:43.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.801+0000 7fd56ed9d640 1 --2- 172.21.15.84:0/3653401879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd57006c140 0x7fd57006c5d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd568009e80 tx=0x7fd56802be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9782f335ce86ef2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:43.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.802+0000 7fd56d59a640 1 -- 172.21.15.84:0/3653401879 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56802c9c0 con 0x7fd57006c140 2023-10-16T10:01:43.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.802+0000 7fd56d59a640 1 -- 172.21.15.84:0/3653401879 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56802cb20 con 0x7fd57006c140 2023-10-16T10:01:43.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.802+0000 7fd56d59a640 1 -- 172.21.15.84:0/3653401879 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5680343f0 con 0x7fd57006c140 2023-10-16T10:01:43.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.802+0000 7fd574e64640 1 -- 172.21.15.84:0/3653401879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd57006c140 msgr2=0x7fd57006c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:43.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.803+0000 7fd574e64640 1 --2- 172.21.15.84:0/3653401879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd57006c140 0x7fd57006c5d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd568009e80 tx=0x7fd56802be20 comp rx=0 tx=0).stop 2023-10-16T10:01:43.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.803+0000 7fd574e64640 1 -- 172.21.15.84:0/3653401879 shutdown_connections 2023-10-16T10:01:43.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.803+0000 7fd574e64640 1 --2- 172.21.15.84:0/3653401879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd57006c140 0x7fd57006c5d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:43.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.803+0000 7fd574e64640 1 --2- 172.21.15.84:0/3653401879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd570159e90 0x7fd5700ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:43.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.803+0000 7fd574e64640 1 -- 172.21.15.84:0/3653401879 >> 172.21.15.84:0/3653401879 conn(0x7fd5700ce9b0 msgr2=0x7fd5700cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:43.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.803+0000 7fd574e64640 1 -- 172.21.15.84:0/3653401879 shutdown_connections 2023-10-16T10:01:43.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.803+0000 7fd574e64640 1 -- 172.21.15.84:0/3653401879 wait complete. 2023-10-16T10:01:43.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.805+0000 7fd574e64640 1 Processor -- start 2023-10-16T10:01:43.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.805+0000 7fd574e64640 1 -- start start 2023-10-16T10:01:43.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.806+0000 7fd574e64640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd57006c140 0x7fd5701e52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:43.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.806+0000 7fd574e64640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd570159ab0 0x7fd5700c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:43.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.806+0000 7fd574e64640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd570159e90 0x7fd5700c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:43.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.806+0000 7fd574e64640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5701e5830 con 0x7fd570159e90 2023-10-16T10:01:43.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.806+0000 7fd574e64640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700c8460 con 0x7fd570159ab0 2023-10-16T10:01:43.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.806+0000 7fd574e64640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700c8e30 con 0x7fd57006c140 2023-10-16T10:01:43.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.806+0000 7fd56ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd570159e90 0x7fd5700c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:43.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.806+0000 7fd56ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd570159e90 0x7fd5700c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48864/0 (socket says 172.21.15.84:48864) 2023-10-16T10:01:43.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.806+0000 7fd56ed9d640 1 -- 172.21.15.84:0/1289558663 learned_addr learned my addr 172.21.15.84:0/1289558663 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:43.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.807+0000 7fd56e59c640 1 --2- 172.21.15.84:0/1289558663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd57006c140 0x7fd5701e52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:43.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.807+0000 7fd56dd9b640 1 --2- 172.21.15.84:0/1289558663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd570159ab0 0x7fd5700c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:43.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.807+0000 7fd56ed9d640 1 -- 172.21.15.84:0/1289558663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd57006c140 msgr2=0x7fd5701e52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:43.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.807+0000 7fd56ed9d640 1 --2- 172.21.15.84:0/1289558663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd57006c140 0x7fd5701e52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:43.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.807+0000 7fd56ed9d640 1 -- 172.21.15.84:0/1289558663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd570159ab0 msgr2=0x7fd5700c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:43.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.807+0000 7fd56ed9d640 1 --2- 172.21.15.84:0/1289558663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd570159ab0 0x7fd5700c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:43.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.807+0000 7fd56ed9d640 1 -- 172.21.15.84:0/1289558663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd568009b30 con 0x7fd570159e90 2023-10-16T10:01:43.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.807+0000 7fd56dd9b640 1 --2- 172.21.15.84:0/1289558663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd570159ab0 0x7fd5700c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:43.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.807+0000 7fd56ed9d640 1 --2- 172.21.15.84:0/1289558663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd570159e90 0x7fd5700c6d00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd568036040 tx=0x7fd568004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:43.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.807+0000 7fd55effd640 1 -- 172.21.15.84:0/1289558663 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568009510 con 0x7fd570159e90 2023-10-16T10:01:43.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.807+0000 7fd55effd640 1 -- 172.21.15.84:0/1289558663 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56803e450 con 0x7fd570159e90 2023-10-16T10:01:43.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.808+0000 7fd55effd640 1 -- 172.21.15.84:0/1289558663 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56803e7e0 con 0x7fd570159e90 2023-10-16T10:01:43.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.808+0000 7fd574e64640 1 -- 172.21.15.84:0/1289558663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5700c7850 con 0x7fd570159e90 2023-10-16T10:01:43.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.808+0000 7fd574e64640 1 -- 172.21.15.84:0/1289558663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5700c7e10 con 0x7fd570159e90 2023-10-16T10:01:43.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.809+0000 7fd574e64640 1 -- 172.21.15.84:0/1289558663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57006f6a0 con 0x7fd570159e90 2023-10-16T10:01:43.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.810+0000 7fd55effd640 1 -- 172.21.15.84:0/1289558663 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fd56803e940 con 0x7fd570159e90 2023-10-16T10:01:43.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.810+0000 7fd55effd640 1 --2- 172.21.15.84:0/1289558663 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd55006b070 0x7fd55006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:43.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.811+0000 7fd55effd640 1 -- 172.21.15.84:0/1289558663 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6727+0+0 (secure 0 0 0) 0x7fd5680acaa0 con 0x7fd570159e90 2023-10-16T10:01:43.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.811+0000 7fd56e59c640 1 --2- 172.21.15.84:0/1289558663 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd55006b070 0x7fd55006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:43.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.811+0000 7fd56e59c640 1 --2- 172.21.15.84:0/1289558663 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd55006b070 0x7fd55006d530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd560000f60 tx=0x7fd560007c80 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:43.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:43.815+0000 7fd55effd640 1 -- 172.21.15.84:0/1289558663 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd5680774d0 con 0x7fd570159e90 2023-10-16T10:01:44.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:44.048+0000 7fd574e64640 1 -- 172.21.15.84:0/1289558663 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5700cd7b0 con 0x7fd55006b070 2023-10-16T10:01:44.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:44.264+0000 7fd55effd640 1 -- 172.21.15.84:0/1289558663 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fd560005a50 con 0x7fd55006b070 2023-10-16T10:01:44.265 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path /snap_test_dir1/dir6 2023-10-16T10:01:44.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:44.267+0000 7fd574e64640 1 -- 172.21.15.84:0/1289558663 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd55006b070 msgr2=0x7fd55006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:44.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:44.267+0000 7fd574e64640 1 --2- 172.21.15.84:0/1289558663 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd55006b070 0x7fd55006d530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd560000f60 tx=0x7fd560007c80 comp rx=0 tx=0).stop 2023-10-16T10:01:44.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:44.268+0000 7fd574e64640 1 -- 172.21.15.84:0/1289558663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd570159e90 msgr2=0x7fd5700c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:44.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:44.268+0000 7fd574e64640 1 --2- 172.21.15.84:0/1289558663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd570159e90 0x7fd5700c6d00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd568036040 tx=0x7fd568004a40 comp rx=0 tx=0).stop 2023-10-16T10:01:44.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:44.268+0000 7fd574e64640 1 -- 172.21.15.84:0/1289558663 shutdown_connections 2023-10-16T10:01:44.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:44.268+0000 7fd574e64640 1 --2- 172.21.15.84:0/1289558663 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd55006b070 0x7fd55006d530 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:44.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:44.268+0000 7fd574e64640 1 --2- 172.21.15.84:0/1289558663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd570159e90 0x7fd5700c6d00 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:44.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:44.268+0000 7fd574e64640 1 --2- 172.21.15.84:0/1289558663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd570159ab0 0x7fd5700c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:44.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:44.268+0000 7fd574e64640 1 --2- 172.21.15.84:0/1289558663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd57006c140 0x7fd5701e52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:44.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:44.268+0000 7fd574e64640 1 -- 172.21.15.84:0/1289558663 >> 172.21.15.84:0/1289558663 conn(0x7fd5700ce9b0 msgr2=0x7fd5700ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:44.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:44.268+0000 7fd574e64640 1 -- 172.21.15.84:0/1289558663 shutdown_connections 2023-10-16T10:01:44.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:44.268+0000 7fd574e64640 1 -- 172.21.15.84:0/1289558663 wait complete. 2023-10-16T10:01:44.290 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2023-10-16T10:01:44.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:44.291 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-10-16T10:01:44.346 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_01_UTC 2023-10-16T10:01:44.346 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_01_UTC 2023-10-16T10:01:44.347 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T10:01:44.347 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:44.347 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_01_UTC 2023-10-16T10:01:44.347 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:44.347 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_00_UTC 2023-10-16T10:01:44.348 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T10:01:44.348 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_00_UTC 2023-10-16T10:01:44.348 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_00_UTC 2023-10-16T10:01:44.348 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:44.349 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_51_01_UTC 2023-10-16T10:01:44.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:44.350 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_51_01_UTC) 2023-10-16T10:01:44.415 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_52_01_UTC 2023-10-16T10:01:44.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:44.416 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_52_01_UTC) 2023-10-16T10:01:44.487 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_53_00_UTC 2023-10-16T10:01:44.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:44.488 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_53_00_UTC) 2023-10-16T10:01:44.559 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:44.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:44.560 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_54_00_UTC) 2023-10-16T10:01:44.631 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_55_01_UTC 2023-10-16T10:01:44.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:44.631 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_55_01_UTC) 2023-10-16T10:01:44.709 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:44.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:44.710 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_56_00_UTC) 2023-10-16T10:01:44.780 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_57_00_UTC 2023-10-16T10:01:44.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:44.781 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_57_00_UTC) 2023-10-16T10:01:44.848 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_58_00_UTC 2023-10-16T10:01:44.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:44.849 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_58_00_UTC) 2023-10-16T10:01:44.920 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_59_00_UTC 2023-10-16T10:01:44.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:44.921 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_59_00_UTC) 2023-10-16T10:01:45.021 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-10_00_00_UTC 2023-10-16T10:01:45.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:45.022 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-10_00_00_UTC) 2023-10-16T10:01:45.077 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:45.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:45.078 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-10_01_00_UTC) 2023-10-16T10:01:45.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:45.129 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2023-10-16T10:01:45.180 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir7 1m 2023-10-16T10:01:45.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.379+0000 7fd86e9fc640 1 Processor -- start 2023-10-16T10:01:45.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.380+0000 7fd86e9fc640 1 -- start start 2023-10-16T10:01:45.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.381+0000 7fd86e9fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86806a4b0 0x7fd8680ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:45.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.381+0000 7fd86e9fc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8680cb030 con 0x7fd86815ad70 2023-10-16T10:01:45.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.381+0000 7fd86e9fc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8680c91c0 con 0x7fd8680c9ca0 2023-10-16T10:01:45.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.381+0000 7fd86e9fc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680c94a0 con 0x7fd86806a4b0 2023-10-16T10:01:45.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.381+0000 7fd867fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd86815ad70 0x7fd868069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35546/0 (socket says 172.21.15.84:35546) 2023-10-16T10:01:45.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.381+0000 7fd867fff640 1 -- 172.21.15.84:0/1719772649 learned_addr learned my addr 172.21.15.84:0/1719772649 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:45.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.382+0000 7fd85ffff640 1 --2- 172.21.15.84:0/1719772649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86806a4b0 0x7fd8680ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:45.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.382+0000 7fd8677fe640 1 -- 172.21.15.84:0/1719772649 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961235378 0 0) 0x7fd8680cb030 con 0x7fd86815ad70 2023-10-16T10:01:45.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.382+0000 7fd8677fe640 1 -- 172.21.15.84:0/1719772649 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd854003520 con 0x7fd86815ad70 2023-10-16T10:01:45.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.382+0000 7fd8677fe640 1 -- 172.21.15.84:0/1719772649 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1185682468 0 0) 0x7fd854003520 con 0x7fd86815ad70 2023-10-16T10:01:45.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.382+0000 7fd8677fe640 1 -- 172.21.15.84:0/1719772649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86806a4b0 msgr2=0x7fd8680ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:45.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.382+0000 7fd8677fe640 1 --2- 172.21.15.84:0/1719772649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86806a4b0 0x7fd8680ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:45.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.382+0000 7fd8677fe640 1 -- 172.21.15.84:0/1719772649 >> v1:172.21.15.157:6789/0 conn(0x7fd8680c9ca0 legacy=0x7fd8680ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:45.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.382+0000 7fd8677fe640 1 -- 172.21.15.84:0/1719772649 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8680c9780 con 0x7fd86815ad70 2023-10-16T10:01:45.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.382+0000 7fd8677fe640 1 -- 172.21.15.84:0/1719772649 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd858001670 con 0x7fd86815ad70 2023-10-16T10:01:45.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.382+0000 7fd8677fe640 1 -- 172.21.15.84:0/1719772649 >> v1:172.21.15.84:6789/0 conn(0x7fd86815ad70 legacy=0x7fd868069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:45.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.382+0000 7fd8677fe640 1 --2- 172.21.15.84:0/1719772649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8540039a0 0x7fd854003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:45.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.382+0000 7fd8677fe640 1 --2- 172.21.15.84:0/1719772649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8540047c0 0x7fd854004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:45.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.382+0000 7fd8677fe640 1 --2- 172.21.15.84:0/1719772649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86806a4b0 0x7fd854005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:45.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.382+0000 7fd8677fe640 1 -- 172.21.15.84:0/1719772649 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680c94a0 con 0x7fd8540047c0 2023-10-16T10:01:45.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.382+0000 7fd8677fe640 1 -- 172.21.15.84:0/1719772649 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85400ba60 con 0x7fd8540039a0 2023-10-16T10:01:45.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.383+0000 7fd8677fe640 1 -- 172.21.15.84:0/1719772649 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85400bc20 con 0x7fd86806a4b0 2023-10-16T10:01:45.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.383+0000 7fd85ffff640 1 --2- 172.21.15.84:0/1719772649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8540047c0 0x7fd854004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:45.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.383+0000 7fd867fff640 1 --2- 172.21.15.84:0/1719772649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8540039a0 0x7fd854003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:45.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.383+0000 7fd85ffff640 1 -- 172.21.15.84:0/1719772649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86806a4b0 msgr2=0x7fd854005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:45.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.383+0000 7fd86cf99640 1 --2- 172.21.15.84:0/1719772649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86806a4b0 0x7fd854005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:45.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.383+0000 7fd85ffff640 1 --2- 172.21.15.84:0/1719772649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd86806a4b0 0x7fd854005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:45.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.383+0000 7fd85ffff640 1 -- 172.21.15.84:0/1719772649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8540039a0 msgr2=0x7fd854003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:45.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.383+0000 7fd85ffff640 1 --2- 172.21.15.84:0/1719772649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8540039a0 0x7fd854003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:45.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.383+0000 7fd85ffff640 1 -- 172.21.15.84:0/1719772649 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd854003520 con 0x7fd8540047c0 2023-10-16T10:01:45.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.383+0000 7fd867fff640 1 --2- 172.21.15.84:0/1719772649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8540039a0 0x7fd854003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:45.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.384+0000 7fd85ffff640 1 --2- 172.21.15.84:0/1719772649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8540047c0 0x7fd854004c40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd850009a90 tx=0x7fd8540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4185405ba074b6c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:45.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.385+0000 7fd8677fe640 1 -- 172.21.15.84:0/1719772649 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85002c680 con 0x7fd8540047c0 2023-10-16T10:01:45.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.385+0000 7fd8677fe640 1 -- 172.21.15.84:0/1719772649 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85002c7e0 con 0x7fd8540047c0 2023-10-16T10:01:45.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.385+0000 7fd8677fe640 1 -- 172.21.15.84:0/1719772649 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85002cb70 con 0x7fd8540047c0 2023-10-16T10:01:45.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.385+0000 7fd86e9fc640 1 -- 172.21.15.84:0/1719772649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8540047c0 msgr2=0x7fd854004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:45.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.385+0000 7fd86e9fc640 1 --2- 172.21.15.84:0/1719772649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8540047c0 0x7fd854004c40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd850009a90 tx=0x7fd8540057f0 comp rx=0 tx=0).stop 2023-10-16T10:01:45.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.385+0000 7fd86cf99640 1 -- 172.21.15.84:0/1719772649 reap_dead start 2023-10-16T10:01:45.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.385+0000 7fd86e9fc640 1 -- 172.21.15.84:0/1719772649 shutdown_connections 2023-10-16T10:01:45.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.385+0000 7fd86e9fc640 1 -- 172.21.15.84:0/1719772649 >> 172.21.15.84:0/1719772649 conn(0x7fd8680ce9b0 msgr2=0x7fd8680cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:45.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.385+0000 7fd86e9fc640 1 -- 172.21.15.84:0/1719772649 shutdown_connections 2023-10-16T10:01:45.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.385+0000 7fd86e9fc640 1 -- 172.21.15.84:0/1719772649 wait complete. 2023-10-16T10:01:45.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.387+0000 7fd86e9fc640 1 Processor -- start 2023-10-16T10:01:45.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.387+0000 7fd86e9fc640 1 -- start start 2023-10-16T10:01:45.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.388+0000 7fd86e9fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8540047c0 0x7fd8680ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:45.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.388+0000 7fd86e9fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd868158080 0x7fd8680c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:45.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.388+0000 7fd86e9fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8681f2ac0 0x7fd8681f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:45.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.388+0000 7fd86e9fc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680adff0 con 0x7fd8540047c0 2023-10-16T10:01:45.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.388+0000 7fd86e9fc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8681f4740 con 0x7fd8681f2ac0 2023-10-16T10:01:45.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.388+0000 7fd86e9fc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8681f3b70 con 0x7fd868158080 2023-10-16T10:01:45.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.388+0000 7fd867fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8540047c0 0x7fd8680ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:45.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.388+0000 7fd867fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8540047c0 0x7fd8680ad1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48870/0 (socket says 172.21.15.84:48870) 2023-10-16T10:01:45.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.388+0000 7fd867fff640 1 -- 172.21.15.84:0/463364606 learned_addr learned my addr 172.21.15.84:0/463364606 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:45.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.389+0000 7fd867fff640 1 -- 172.21.15.84:0/463364606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd868158080 msgr2=0x7fd8680c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:45.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.389+0000 7fd867fff640 1 --2- 172.21.15.84:0/463364606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd868158080 0x7fd8680c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:45.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.389+0000 7fd867fff640 1 -- 172.21.15.84:0/463364606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8681f2ac0 msgr2=0x7fd8681f2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:45.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.389+0000 7fd867fff640 1 --2- 172.21.15.84:0/463364606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8681f2ac0 0x7fd8681f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:45.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.389+0000 7fd867fff640 1 -- 172.21.15.84:0/463364606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8681f3e10 con 0x7fd8540047c0 2023-10-16T10:01:45.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.389+0000 7fd867fff640 1 --2- 172.21.15.84:0/463364606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8540047c0 0x7fd8680ad1c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd85800d860 tx=0x7fd85800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:45.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.389+0000 7fd8657fa640 1 -- 172.21.15.84:0/463364606 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8580074e0 con 0x7fd8540047c0 2023-10-16T10:01:45.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.389+0000 7fd8657fa640 1 -- 172.21.15.84:0/463364606 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd858007640 con 0x7fd8540047c0 2023-10-16T10:01:45.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.389+0000 7fd8657fa640 1 -- 172.21.15.84:0/463364606 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8580049e0 con 0x7fd8540047c0 2023-10-16T10:01:45.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.389+0000 7fd86e9fc640 1 -- 172.21.15.84:0/463364606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8681f40f0 con 0x7fd8540047c0 2023-10-16T10:01:45.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.390+0000 7fd86e9fc640 1 -- 172.21.15.84:0/463364606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8680ad700 con 0x7fd8540047c0 2023-10-16T10:01:45.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.391+0000 7fd86e9fc640 1 -- 172.21.15.84:0/463364606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd86804e160 con 0x7fd8540047c0 2023-10-16T10:01:45.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.392+0000 7fd8657fa640 1 -- 172.21.15.84:0/463364606 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fd858004b40 con 0x7fd8540047c0 2023-10-16T10:01:45.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.392+0000 7fd8657fa640 1 --2- 172.21.15.84:0/463364606 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd84806b070 0x7fd84806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:45.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.393+0000 7fd8657fa640 1 -- 172.21.15.84:0/463364606 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd858089960 con 0x7fd8540047c0 2023-10-16T10:01:45.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.393+0000 7fd85ffff640 1 --2- 172.21.15.84:0/463364606 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd84806b070 0x7fd84806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:45.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.394+0000 7fd85ffff640 1 --2- 172.21.15.84:0/463364606 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd84806b070 0x7fd84806d530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd850036000 tx=0x7fd85002e000 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:45.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.401+0000 7fd8657fa640 1 -- 172.21.15.84:0/463364606 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd858054220 con 0x7fd8540047c0 2023-10-16T10:01:45.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.638+0000 7fd86e9fc640 1 -- 172.21.15.84:0/463364606 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8680adbe0 con 0x7fd84806b070 2023-10-16T10:01:45.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.858+0000 7fd8657fa640 1 -- 172.21.15.84:0/463364606 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fd8500040c0 con 0x7fd84806b070 2023-10-16T10:01:45.863 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path /snap_test_dir1/dir7 2023-10-16T10:01:45.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.863+0000 7fd86e9fc640 1 -- 172.21.15.84:0/463364606 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd84806b070 msgr2=0x7fd84806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:45.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.863+0000 7fd86e9fc640 1 --2- 172.21.15.84:0/463364606 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd84806b070 0x7fd84806d530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd850036000 tx=0x7fd85002e000 comp rx=0 tx=0).stop 2023-10-16T10:01:45.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.863+0000 7fd86e9fc640 1 -- 172.21.15.84:0/463364606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8540047c0 msgr2=0x7fd8680ad1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:45.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.863+0000 7fd86e9fc640 1 --2- 172.21.15.84:0/463364606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8540047c0 0x7fd8680ad1c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd85800d860 tx=0x7fd85800dd20 comp rx=0 tx=0).stop 2023-10-16T10:01:45.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.863+0000 7fd86e9fc640 1 -- 172.21.15.84:0/463364606 shutdown_connections 2023-10-16T10:01:45.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.863+0000 7fd86e9fc640 1 --2- 172.21.15.84:0/463364606 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd84806b070 0x7fd84806d530 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:45.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.863+0000 7fd86e9fc640 1 --2- 172.21.15.84:0/463364606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8681f2ac0 0x7fd8681f2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:45.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.863+0000 7fd86e9fc640 1 --2- 172.21.15.84:0/463364606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd868158080 0x7fd8680c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:45.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.863+0000 7fd86e9fc640 1 --2- 172.21.15.84:0/463364606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8540047c0 0x7fd8680ad1c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:45.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.863+0000 7fd86e9fc640 1 -- 172.21.15.84:0/463364606 >> 172.21.15.84:0/463364606 conn(0x7fd8680ce9b0 msgr2=0x7fd868069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:45.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.863+0000 7fd86e9fc640 1 -- 172.21.15.84:0/463364606 shutdown_connections 2023-10-16T10:01:45.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:45.863+0000 7fd86e9fc640 1 -- 172.21.15.84:0/463364606 wait complete. 2023-10-16T10:01:45.888 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2023-10-16T10:01:45.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:45.889 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-10-16T10:01:45.944 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T10:01:45.944 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_01_UTC 2023-10-16T10:01:45.945 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T10:01:45.945 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:45.945 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_01_UTC 2023-10-16T10:01:45.945 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_01_UTC 2023-10-16T10:01:45.945 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_01_UTC 2023-10-16T10:01:45.946 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T10:01:45.946 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_00_UTC 2023-10-16T10:01:45.946 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_00_UTC 2023-10-16T10:01:45.946 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_01_UTC 2023-10-16T10:01:45.948 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_51_00_UTC 2023-10-16T10:01:45.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:45.949 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_51_00_UTC) 2023-10-16T10:01:46.045 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_52_01_UTC 2023-10-16T10:01:46.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:46.046 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_52_01_UTC) 2023-10-16T10:01:46.098 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_53_00_UTC 2023-10-16T10:01:46.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:46.099 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_53_00_UTC) 2023-10-16T10:01:46.153 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_54_00_UTC 2023-10-16T10:01:46.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:46.154 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_54_00_UTC) 2023-10-16T10:01:46.212 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_55_01_UTC 2023-10-16T10:01:46.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:46.213 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_55_01_UTC) 2023-10-16T10:01:46.293 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_56_01_UTC 2023-10-16T10:01:46.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:46.294 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_56_01_UTC) 2023-10-16T10:01:46.348 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_57_01_UTC 2023-10-16T10:01:46.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:46.349 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_57_01_UTC) 2023-10-16T10:01:46.395 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_58_00_UTC 2023-10-16T10:01:46.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:46.396 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_58_00_UTC) 2023-10-16T10:01:46.451 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_59_00_UTC 2023-10-16T10:01:46.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:46.452 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_59_00_UTC) 2023-10-16T10:01:46.514 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-10_00_00_UTC 2023-10-16T10:01:46.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:46.515 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-10_00_00_UTC) 2023-10-16T10:01:46.579 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-10_01_01_UTC 2023-10-16T10:01:46.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:46.580 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-10_01_01_UTC) 2023-10-16T10:01:46.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:46.647 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2023-10-16T10:01:46.712 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir8 1m 2023-10-16T10:01:46.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.929+0000 7f507e97c640 1 Processor -- start 2023-10-16T10:01:46.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.929+0000 7f507e97c640 1 -- start start 2023-10-16T10:01:46.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.930+0000 7f507e97c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5078159ab0 0x7f50780ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:46.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.930+0000 7f507e97c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5078159e90 0x7f507806fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:46.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.930+0000 7f507e97c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f507806c140 0x7f507806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:46.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.930+0000 7f507e97c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507806cb10 con 0x7f507806c140 2023-10-16T10:01:46.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.931+0000 7f507e97c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507815d890 con 0x7f5078159e90 2023-10-16T10:01:46.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.931+0000 7f507e97c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50780cd1f0 con 0x7f5078159ab0 2023-10-16T10:01:46.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.931+0000 7f507cf19640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f507806c140 0x7f507806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:46.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.931+0000 7f507cf19640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f507806c140 0x7f507806c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38456/0 (socket says 172.21.15.84:38456) 2023-10-16T10:01:46.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.931+0000 7f507cf19640 1 -- 172.21.15.84:0/3166050878 learned_addr learned my addr 172.21.15.84:0/3166050878 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:46.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.931+0000 7f5077fff640 1 --2- 172.21.15.84:0/3166050878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5078159ab0 0x7f50780ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:46.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.931+0000 7f50777fe640 1 --2- 172.21.15.84:0/3166050878 >> v2:172.21.15.157:3300/0 conn(0x7f5078159e90 0x7f507806fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:46.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.932+0000 7f507cf19640 1 -- 172.21.15.84:0/3166050878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5078159ab0 msgr2=0x7f50780ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:46.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.932+0000 7f507cf19640 1 --2- 172.21.15.84:0/3166050878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5078159ab0 0x7f50780ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:46.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.932+0000 7f507cf19640 1 -- 172.21.15.84:0/3166050878 >> v2:172.21.15.157:3300/0 conn(0x7f5078159e90 msgr2=0x7f507806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:46.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.932+0000 7f507cf19640 1 --2- 172.21.15.84:0/3166050878 >> v2:172.21.15.157:3300/0 conn(0x7f5078159e90 0x7f507806fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:46.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.932+0000 7f507cf19640 1 -- 172.21.15.84:0/3166050878 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50780cd4d0 con 0x7f507806c140 2023-10-16T10:01:46.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.933+0000 7f507cf19640 1 --2- 172.21.15.84:0/3166050878 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507806c140 0x7f507806c5d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5070009870 tx=0x7f507002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2da51136ac1647 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:46.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.934+0000 7f5076ffd640 1 -- 172.21.15.84:0/3166050878 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070031740 con 0x7f507806c140 2023-10-16T10:01:46.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.934+0000 7f5076ffd640 1 -- 172.21.15.84:0/3166050878 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50700318a0 con 0x7f507806c140 2023-10-16T10:01:46.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.934+0000 7f5076ffd640 1 -- 172.21.15.84:0/3166050878 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070031c30 con 0x7f507806c140 2023-10-16T10:01:46.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.934+0000 7f507e97c640 1 -- 172.21.15.84:0/3166050878 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507806c140 msgr2=0x7f507806c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:46.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.934+0000 7f507e97c640 1 --2- 172.21.15.84:0/3166050878 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507806c140 0x7f507806c5d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5070009870 tx=0x7f507002b7e0 comp rx=0 tx=0).stop 2023-10-16T10:01:46.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.934+0000 7f507e97c640 1 -- 172.21.15.84:0/3166050878 shutdown_connections 2023-10-16T10:01:46.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.934+0000 7f507e97c640 1 --2- 172.21.15.84:0/3166050878 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507806c140 0x7f507806c5d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:46.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.934+0000 7f507e97c640 1 --2- 172.21.15.84:0/3166050878 >> v2:172.21.15.157:3300/0 conn(0x7f5078159e90 0x7f507806fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:46.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.934+0000 7f507e97c640 1 --2- 172.21.15.84:0/3166050878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5078159ab0 0x7f50780ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:46.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.934+0000 7f507e97c640 1 -- 172.21.15.84:0/3166050878 >> 172.21.15.84:0/3166050878 conn(0x7f50780ce9b0 msgr2=0x7f50780cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:46.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.934+0000 7f507e97c640 1 -- 172.21.15.84:0/3166050878 shutdown_connections 2023-10-16T10:01:46.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.934+0000 7f507e97c640 1 -- 172.21.15.84:0/3166050878 wait complete. 2023-10-16T10:01:46.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.936+0000 7f507e97c640 1 Processor -- start 2023-10-16T10:01:46.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.936+0000 7f507e97c640 1 -- start start 2023-10-16T10:01:46.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.937+0000 7f507e97c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507806c140 0x7f50780afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:46.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.937+0000 7f507e97c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5078159ab0 0x7f50781e5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:46.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.937+0000 7f507e97c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5078159e90 0x7f50780c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:46.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.937+0000 7f507e97c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50781e5ec0 con 0x7f507806c140 2023-10-16T10:01:46.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.937+0000 7f507e97c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50780c6d20 con 0x7f5078159e90 2023-10-16T10:01:46.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.937+0000 7f507e97c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50780c7000 con 0x7f5078159ab0 2023-10-16T10:01:46.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.937+0000 7f5077fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507806c140 0x7f50780afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:46.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.938+0000 7f5077fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507806c140 0x7f50780afae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38460/0 (socket says 172.21.15.84:38460) 2023-10-16T10:01:46.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.938+0000 7f5077fff640 1 -- 172.21.15.84:0/3200270060 learned_addr learned my addr 172.21.15.84:0/3200270060 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:46.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.938+0000 7f5077fff640 1 -- 172.21.15.84:0/3200270060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5078159ab0 msgr2=0x7f50781e5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:46.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.938+0000 7f507cf19640 1 --2- 172.21.15.84:0/3200270060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5078159e90 0x7f50780c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:46.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.938+0000 7f5077fff640 1 --2- 172.21.15.84:0/3200270060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5078159ab0 0x7f50781e5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:46.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.938+0000 7f5077fff640 1 -- 172.21.15.84:0/3200270060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5078159e90 msgr2=0x7f50780c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:46.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.938+0000 7f5077fff640 1 --2- 172.21.15.84:0/3200270060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5078159e90 0x7f50780c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:46.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.938+0000 7f5077fff640 1 -- 172.21.15.84:0/3200270060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50700094d0 con 0x7f507806c140 2023-10-16T10:01:46.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.938+0000 7f5077fff640 1 --2- 172.21.15.84:0/3200270060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507806c140 0x7f50780afae0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f506800e890 tx=0x7f506800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:46.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.939+0000 7f5074ff9640 1 -- 172.21.15.84:0/3200270060 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506800c940 con 0x7f507806c140 2023-10-16T10:01:46.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.939+0000 7f5074ff9640 1 -- 172.21.15.84:0/3200270060 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506800caa0 con 0x7f507806c140 2023-10-16T10:01:46.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.939+0000 7f5074ff9640 1 -- 172.21.15.84:0/3200270060 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506800ce30 con 0x7f507806c140 2023-10-16T10:01:46.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.939+0000 7f507e97c640 1 -- 172.21.15.84:0/3200270060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50780c72e0 con 0x7f507806c140 2023-10-16T10:01:46.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.939+0000 7f507e97c640 1 -- 172.21.15.84:0/3200270060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5078157420 con 0x7f507806c140 2023-10-16T10:01:46.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.940+0000 7f507e97c640 1 -- 172.21.15.84:0/3200270060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5078071350 con 0x7f507806c140 2023-10-16T10:01:46.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.941+0000 7f5074ff9640 1 -- 172.21.15.84:0/3200270060 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f5068013470 con 0x7f507806c140 2023-10-16T10:01:46.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.942+0000 7f5074ff9640 1 --2- 172.21.15.84:0/3200270060 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f505806b070 0x7f505806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:46.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.942+0000 7f5074ff9640 1 -- 172.21.15.84:0/3200270060 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6583+0+0 (secure 0 0 0) 0x7f5068089f30 con 0x7f507806c140 2023-10-16T10:01:46.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.942+0000 7f50777fe640 1 --2- 172.21.15.84:0/3200270060 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f505806b070 0x7f505806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:46.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.942+0000 7f50777fe640 1 --2- 172.21.15.84:0/3200270060 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f505806b070 0x7f505806d530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5078157140 tx=0x7f506000b040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:46.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:46.946+0000 7f5074ff9640 1 -- 172.21.15.84:0/3200270060 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f50680549e0 con 0x7f507806c140 2023-10-16T10:01:47.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:47.179+0000 7f507e97c640 1 -- 172.21.15.84:0/3200270060 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f50780c6e90 con 0x7f505806b070 2023-10-16T10:01:47.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:47.385+0000 7f5074ff9640 1 -- 172.21.15.84:0/3200270060 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f50780c6e90 con 0x7f505806b070 2023-10-16T10:01:47.387 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path /snap_test_dir1/dir8 2023-10-16T10:01:47.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:47.388+0000 7f507e97c640 1 -- 172.21.15.84:0/3200270060 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f505806b070 msgr2=0x7f505806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:47.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:47.388+0000 7f507e97c640 1 --2- 172.21.15.84:0/3200270060 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f505806b070 0x7f505806d530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5078157140 tx=0x7f506000b040 comp rx=0 tx=0).stop 2023-10-16T10:01:47.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:47.388+0000 7f507e97c640 1 -- 172.21.15.84:0/3200270060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507806c140 msgr2=0x7f50780afae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:47.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:47.388+0000 7f507e97c640 1 --2- 172.21.15.84:0/3200270060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507806c140 0x7f50780afae0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f506800e890 tx=0x7f506800ed50 comp rx=0 tx=0).stop 2023-10-16T10:01:47.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:47.389+0000 7f507e97c640 1 -- 172.21.15.84:0/3200270060 shutdown_connections 2023-10-16T10:01:47.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:47.389+0000 7f507e97c640 1 --2- 172.21.15.84:0/3200270060 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f505806b070 0x7f505806d530 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:47.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:47.389+0000 7f507e97c640 1 --2- 172.21.15.84:0/3200270060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5078159e90 0x7f50780c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:47.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:47.389+0000 7f507e97c640 1 --2- 172.21.15.84:0/3200270060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5078159ab0 0x7f50781e5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:47.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:47.389+0000 7f507e97c640 1 --2- 172.21.15.84:0/3200270060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507806c140 0x7f50780afae0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:47.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:47.389+0000 7f507e97c640 1 -- 172.21.15.84:0/3200270060 >> 172.21.15.84:0/3200270060 conn(0x7f50780ce9b0 msgr2=0x7f5078159010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:47.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:47.389+0000 7f507e97c640 1 -- 172.21.15.84:0/3200270060 shutdown_connections 2023-10-16T10:01:47.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:47.389+0000 7f507e97c640 1 -- 172.21.15.84:0/3200270060 wait complete. 2023-10-16T10:01:47.409 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2023-10-16T10:01:47.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:47.410 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-10-16T10:01:47.464 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_01_UTC 2023-10-16T10:01:47.464 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_01_UTC 2023-10-16T10:01:47.464 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_01_UTC 2023-10-16T10:01:47.464 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_01_UTC 2023-10-16T10:01:47.465 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_01_UTC 2023-10-16T10:01:47.465 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:47.465 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_00_UTC 2023-10-16T10:01:47.465 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T10:01:47.465 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_00_UTC 2023-10-16T10:01:47.465 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_01_UTC 2023-10-16T10:01:47.466 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:47.467 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_51_01_UTC 2023-10-16T10:01:47.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:47.467 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_51_01_UTC) 2023-10-16T10:01:47.539 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_52_01_UTC 2023-10-16T10:01:47.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:47.540 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_52_01_UTC) 2023-10-16T10:01:47.617 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_53_01_UTC 2023-10-16T10:01:47.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:47.618 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_53_01_UTC) 2023-10-16T10:01:47.692 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_54_01_UTC 2023-10-16T10:01:47.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:47.693 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_54_01_UTC) 2023-10-16T10:01:47.766 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_55_01_UTC 2023-10-16T10:01:47.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:47.767 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_55_01_UTC) 2023-10-16T10:01:47.916 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:47.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:47.917 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_56_00_UTC) 2023-10-16T10:01:47.981 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_57_00_UTC 2023-10-16T10:01:47.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:47.982 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_57_00_UTC) 2023-10-16T10:01:48.050 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_58_00_UTC 2023-10-16T10:01:48.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:48.051 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_58_00_UTC) 2023-10-16T10:01:48.100 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_59_00_UTC 2023-10-16T10:01:48.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:48.101 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_59_00_UTC) 2023-10-16T10:01:48.157 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-10_00_01_UTC 2023-10-16T10:01:48.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:48.158 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-10_00_01_UTC) 2023-10-16T10:01:48.220 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:48.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:48.221 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-10_01_00_UTC) 2023-10-16T10:01:48.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:48.295 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2023-10-16T10:01:48.368 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir9 1m 2023-10-16T10:01:48.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.584+0000 7f40f8383640 1 Processor -- start 2023-10-16T10:01:48.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.584+0000 7f40f8383640 1 -- start start 2023-10-16T10:01:48.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.586+0000 7f40f8383640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f40f006b660 0x7f40f006ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:48.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.586+0000 7f40f8383640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f40f00c9860 0x7f40f00ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:48.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.586+0000 7f40f8383640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f00cc6b0 con 0x7f40f01428c0 2023-10-16T10:01:48.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.586+0000 7f40f8383640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f01599b0 con 0x7f40f00c9860 2023-10-16T10:01:48.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.586+0000 7f40f8383640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f006c290 con 0x7f40f006b660 2023-10-16T10:01:48.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.586+0000 7f40f591e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f40f01428c0 0x7f40f00c9150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34302/0 (socket says 172.21.15.84:34302) 2023-10-16T10:01:48.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.586+0000 7f40f591e640 1 -- 172.21.15.84:0/3463449205 learned_addr learned my addr 172.21.15.84:0/3463449205 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:48.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.586+0000 7f40f6920640 1 --2- 172.21.15.84:0/3463449205 >> v2:172.21.15.84:3300/0 conn(0x7f40f00c9860 0x7f40f00ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:48.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.586+0000 7f40f611f640 1 --2- 172.21.15.84:0/3463449205 >> v2:172.21.15.157:3300/0 conn(0x7f40f006b660 0x7f40f006ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:48.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.586+0000 7f40f511d640 1 -- 172.21.15.84:0/3463449205 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 164721965 0 0) 0x7f40f00cc6b0 con 0x7f40f01428c0 2023-10-16T10:01:48.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.587+0000 7f40f511d640 1 -- 172.21.15.84:0/3463449205 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40dc003520 con 0x7f40f01428c0 2023-10-16T10:01:48.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.587+0000 7f40f6920640 1 -- 172.21.15.84:0/3463449205 >> v2:172.21.15.157:3300/0 conn(0x7f40f006b660 msgr2=0x7f40f006ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:48.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.587+0000 7f40f6920640 1 --2- 172.21.15.84:0/3463449205 >> v2:172.21.15.157:3300/0 conn(0x7f40f006b660 0x7f40f006ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:48.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.587+0000 7f40f6920640 1 -- 172.21.15.84:0/3463449205 >> v1:172.21.15.84:6789/0 conn(0x7f40f01428c0 legacy=0x7f40f00c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:48.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.587+0000 7f40f6920640 1 -- 172.21.15.84:0/3463449205 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f006c570 con 0x7f40f00c9860 2023-10-16T10:01:48.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.588+0000 7f40f6920640 1 --2- 172.21.15.84:0/3463449205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f00c9860 0x7f40f00ccb40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f40f00c9120 tx=0x7f40e802cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42977faec6227ed8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:48.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.588+0000 7f40f511d640 1 -- 172.21.15.84:0/3463449205 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e802db70 con 0x7f40f00c9860 2023-10-16T10:01:48.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.588+0000 7f40f511d640 1 -- 172.21.15.84:0/3463449205 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e802dcd0 con 0x7f40f00c9860 2023-10-16T10:01:48.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.588+0000 7f40f511d640 1 -- 172.21.15.84:0/3463449205 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8008660 con 0x7f40f00c9860 2023-10-16T10:01:48.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.589+0000 7f40f8383640 1 -- 172.21.15.84:0/3463449205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f00c9860 msgr2=0x7f40f00ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:48.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.589+0000 7f40f8383640 1 --2- 172.21.15.84:0/3463449205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f00c9860 0x7f40f00ccb40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f40f00c9120 tx=0x7f40e802cfd0 comp rx=0 tx=0).stop 2023-10-16T10:01:48.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.589+0000 7f40f8383640 1 -- 172.21.15.84:0/3463449205 shutdown_connections 2023-10-16T10:01:48.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.589+0000 7f40f8383640 1 --2- 172.21.15.84:0/3463449205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f00c9860 0x7f40f00ccb40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:48.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.589+0000 7f40f8383640 1 --2- 172.21.15.84:0/3463449205 >> v2:172.21.15.157:3300/0 conn(0x7f40f006b660 0x7f40f006ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:48.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.589+0000 7f40f8383640 1 -- 172.21.15.84:0/3463449205 >> 172.21.15.84:0/3463449205 conn(0x7f40f00ce9b0 msgr2=0x7f40f00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:48.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.589+0000 7f40f8383640 1 -- 172.21.15.84:0/3463449205 shutdown_connections 2023-10-16T10:01:48.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.589+0000 7f40f8383640 1 -- 172.21.15.84:0/3463449205 wait complete. 2023-10-16T10:01:48.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.591+0000 7f40f8383640 1 Processor -- start 2023-10-16T10:01:48.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.591+0000 7f40f8383640 1 -- start start 2023-10-16T10:01:48.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.592+0000 7f40f8383640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40f006b660 0x7f40f00c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:48.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.592+0000 7f40f8383640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f00c9860 0x7f40f0152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:48.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.592+0000 7f40f8383640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40f01428c0 0x7f40f0157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:48.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.592+0000 7f40f8383640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f0152730 con 0x7f40f00c9860 2023-10-16T10:01:48.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.592+0000 7f40f8383640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f0151e00 con 0x7f40f006b660 2023-10-16T10:01:48.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.592+0000 7f40f8383640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f01520e0 con 0x7f40f01428c0 2023-10-16T10:01:48.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.592+0000 7f40f591e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f00c9860 0x7f40f0152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:48.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.592+0000 7f40f591e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f00c9860 0x7f40f0152bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38474/0 (socket says 172.21.15.84:38474) 2023-10-16T10:01:48.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.592+0000 7f40f591e640 1 -- 172.21.15.84:0/3301866432 learned_addr learned my addr 172.21.15.84:0/3301866432 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:48.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.592+0000 7f40f591e640 1 -- 172.21.15.84:0/3301866432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40f01428c0 msgr2=0x7f40f0157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:48.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.592+0000 7f40f591e640 1 --2- 172.21.15.84:0/3301866432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40f01428c0 0x7f40f0157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:48.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.592+0000 7f40f591e640 1 -- 172.21.15.84:0/3301866432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40f006b660 msgr2=0x7f40f00c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:48.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.592+0000 7f40f591e640 1 --2- 172.21.15.84:0/3301866432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40f006b660 0x7f40f00c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:48.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.592+0000 7f40f591e640 1 -- 172.21.15.84:0/3301866432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e800ab80 con 0x7f40f00c9860 2023-10-16T10:01:48.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.593+0000 7f40f591e640 1 --2- 172.21.15.84:0/3301866432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f00c9860 0x7f40f0152bc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f40e000c500 tx=0x7f40e000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:48.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.593+0000 7f40e6ffd640 1 -- 172.21.15.84:0/3301866432 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0005ce0 con 0x7f40f00c9860 2023-10-16T10:01:48.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.593+0000 7f40e6ffd640 1 -- 172.21.15.84:0/3301866432 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e0005e40 con 0x7f40f00c9860 2023-10-16T10:01:48.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.593+0000 7f40e6ffd640 1 -- 172.21.15.84:0/3301866432 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0007330 con 0x7f40f00c9860 2023-10-16T10:01:48.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.593+0000 7f40f8383640 1 -- 172.21.15.84:0/3301866432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f01523c0 con 0x7f40f00c9860 2023-10-16T10:01:48.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.593+0000 7f40f8383640 1 -- 172.21.15.84:0/3301866432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f0153100 con 0x7f40f00c9860 2023-10-16T10:01:48.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.594+0000 7f40f8383640 1 -- 172.21.15.84:0/3301866432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f004e160 con 0x7f40f00c9860 2023-10-16T10:01:48.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.595+0000 7f40e6ffd640 1 -- 172.21.15.84:0/3301866432 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f40e0007550 con 0x7f40f00c9860 2023-10-16T10:01:48.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.596+0000 7f40e6ffd640 1 --2- 172.21.15.84:0/3301866432 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f40d406b070 0x7f40d406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:48.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.596+0000 7f40e6ffd640 1 -- 172.21.15.84:0/3301866432 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 7203+0+0 (secure 0 0 0) 0x7f40e0089600 con 0x7f40f00c9860 2023-10-16T10:01:48.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.596+0000 7f40f611f640 1 --2- 172.21.15.84:0/3301866432 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f40d406b070 0x7f40d406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:48.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.597+0000 7f40f611f640 1 --2- 172.21.15.84:0/3301866432 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f40d406b070 0x7f40d406d530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f40ec006fd0 tx=0x7f40ec008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:48.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.600+0000 7f40e6ffd640 1 -- 172.21.15.84:0/3301866432 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f40e0053f00 con 0x7f40f00c9860 2023-10-16T10:01:48.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:48.818+0000 7f40f8383640 1 -- 172.21.15.84:0/3301866432 --> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f40f006cb30 con 0x7f40d406b070 2023-10-16T10:01:49.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:49.028+0000 7f40e6ffd640 1 -- 172.21.15.84:0/3301866432 <== mgr.5322 v2:172.21.15.84:6832/4265272960 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f40ec00be20 con 0x7f40d406b070 2023-10-16T10:01:49.030 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path /snap_test_dir1/dir9 2023-10-16T10:01:49.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:49.031+0000 7f40f8383640 1 -- 172.21.15.84:0/3301866432 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f40d406b070 msgr2=0x7f40d406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:49.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:49.031+0000 7f40f8383640 1 --2- 172.21.15.84:0/3301866432 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f40d406b070 0x7f40d406d530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f40ec006fd0 tx=0x7f40ec008040 comp rx=0 tx=0).stop 2023-10-16T10:01:49.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:49.031+0000 7f40f8383640 1 -- 172.21.15.84:0/3301866432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f00c9860 msgr2=0x7f40f0152bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:49.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:49.031+0000 7f40f8383640 1 --2- 172.21.15.84:0/3301866432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f00c9860 0x7f40f0152bc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f40e000c500 tx=0x7f40e000c9c0 comp rx=0 tx=0).stop 2023-10-16T10:01:49.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:49.031+0000 7f40f8383640 1 -- 172.21.15.84:0/3301866432 shutdown_connections 2023-10-16T10:01:49.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:49.031+0000 7f40f8383640 1 --2- 172.21.15.84:0/3301866432 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f40d406b070 0x7f40d406d530 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:49.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:49.031+0000 7f40f8383640 1 --2- 172.21.15.84:0/3301866432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40f01428c0 0x7f40f0157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:49.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:49.031+0000 7f40f8383640 1 --2- 172.21.15.84:0/3301866432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f00c9860 0x7f40f0152bc0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:49.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:49.031+0000 7f40f8383640 1 --2- 172.21.15.84:0/3301866432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40f006b660 0x7f40f00c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:49.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:49.031+0000 7f40f8383640 1 -- 172.21.15.84:0/3301866432 >> 172.21.15.84:0/3301866432 conn(0x7f40f00ce9b0 msgr2=0x7f40f00c5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:49.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:49.032+0000 7f40f8383640 1 -- 172.21.15.84:0/3301866432 shutdown_connections 2023-10-16T10:01:49.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:49.032+0000 7f40f8383640 1 -- 172.21.15.84:0/3301866432 wait complete. 2023-10-16T10:01:49.053 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2023-10-16T10:01:49.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:49.054 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-10-16T10:01:49.106 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T10:01:49.107 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T10:01:49.107 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_53_01_UTC 2023-10-16T10:01:49.107 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_54_01_UTC 2023-10-16T10:01:49.107 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_55_01_UTC 2023-10-16T10:01:49.108 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:49.108 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_57_00_UTC 2023-10-16T10:01:49.108 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T10:01:49.108 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-09_59_00_UTC 2023-10-16T10:01:49.108 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_00_00_UTC 2023-10-16T10:01:49.108 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:49.109 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_51_00_UTC 2023-10-16T10:01:49.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:49.110 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_51_00_UTC) 2023-10-16T10:01:49.190 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_52_00_UTC 2023-10-16T10:01:49.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:49.191 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_52_00_UTC) 2023-10-16T10:01:49.266 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_53_01_UTC 2023-10-16T10:01:49.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:49.267 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_53_01_UTC) 2023-10-16T10:01:49.344 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_54_01_UTC 2023-10-16T10:01:49.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:49.346 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_54_01_UTC) 2023-10-16T10:01:49.421 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_55_01_UTC 2023-10-16T10:01:49.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:49.422 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_55_01_UTC) 2023-10-16T10:01:49.492 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_56_00_UTC 2023-10-16T10:01:49.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:49.493 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_56_00_UTC) 2023-10-16T10:01:49.566 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_57_00_UTC 2023-10-16T10:01:49.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:49.567 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_57_00_UTC) 2023-10-16T10:01:49.643 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_58_00_UTC 2023-10-16T10:01:49.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:49.644 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_58_00_UTC) 2023-10-16T10:01:49.718 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_59_00_UTC 2023-10-16T10:01:49.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:49.720 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_59_00_UTC) 2023-10-16T10:01:49.797 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-10_00_00_UTC 2023-10-16T10:01:49.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:49.798 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-10_00_00_UTC) 2023-10-16T10:01:49.902 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-10_01_00_UTC 2023-10-16T10:01:49.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:49.903 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-10_01_00_UTC) 2023-10-16T10:01:49.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:49.956 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2023-10-16T10:01:50.006 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T10:01:50.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.210+0000 7fbfb0434640 1 Processor -- start 2023-10-16T10:01:50.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.211+0000 7fbfb0434640 1 -- start start 2023-10-16T10:01:50.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.212+0000 7fbfb0434640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbfa8159ab0 0x7fbfa80ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.212+0000 7fbfb0434640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbfa8159e90 0x7fbfa806fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.212+0000 7fbfb0434640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfa806c140 0x7fbfa806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.212+0000 7fbfb0434640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa806cb10 con 0x7fbfa8159ab0 2023-10-16T10:01:50.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.212+0000 7fbfb0434640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa815d890 con 0x7fbfa8159e90 2023-10-16T10:01:50.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.212+0000 7fbfb0434640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa80cd1f0 con 0x7fbfa806c140 2023-10-16T10:01:50.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.212+0000 7fbfae1d0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbfa8159ab0 0x7fbfa80ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:50.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.212+0000 7fbfae1d0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbfa8159ab0 0x7fbfa80ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38488/0 (socket says 172.21.15.84:38488) 2023-10-16T10:01:50.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.212+0000 7fbfae1d0640 1 -- 172.21.15.84:0/1485892056 learned_addr learned my addr 172.21.15.84:0/1485892056 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:50.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.213+0000 7fbfad9cf640 1 --2- 172.21.15.84:0/1485892056 >> v2:172.21.15.157:3300/0 conn(0x7fbfa8159e90 0x7fbfa806fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:50.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.213+0000 7fbfae9d1640 1 --2- 172.21.15.84:0/1485892056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfa806c140 0x7fbfa806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:50.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.213+0000 7fbfae1d0640 1 -- 172.21.15.84:0/1485892056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfa806c140 msgr2=0x7fbfa806c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:50.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.213+0000 7fbfae1d0640 1 --2- 172.21.15.84:0/1485892056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfa806c140 0x7fbfa806c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.213+0000 7fbfae1d0640 1 -- 172.21.15.84:0/1485892056 >> v2:172.21.15.157:3300/0 conn(0x7fbfa8159e90 msgr2=0x7fbfa806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:50.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.213+0000 7fbfae1d0640 1 --2- 172.21.15.84:0/1485892056 >> v2:172.21.15.157:3300/0 conn(0x7fbfa8159e90 0x7fbfa806fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.213+0000 7fbfae1d0640 1 -- 172.21.15.84:0/1485892056 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa80cd4d0 con 0x7fbfa8159ab0 2023-10-16T10:01:50.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.213+0000 7fbfad9cf640 1 --2- 172.21.15.84:0/1485892056 >> v2:172.21.15.157:3300/0 conn(0x7fbfa8159e90 0x7fbfa806fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:50.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.213+0000 7fbfae9d1640 1 --2- 172.21.15.84:0/1485892056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfa806c140 0x7fbfa806c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:50.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.214+0000 7fbfae1d0640 1 --2- 172.21.15.84:0/1485892056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa8159ab0 0x7fbfa80ce370 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbfa806d900 tx=0x7fbfa402dfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba2420064aea9d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:50.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.215+0000 7fbfad1ce640 1 -- 172.21.15.84:0/1485892056 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa402ed60 con 0x7fbfa8159ab0 2023-10-16T10:01:50.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.215+0000 7fbfad1ce640 1 -- 172.21.15.84:0/1485892056 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa4036440 con 0x7fbfa8159ab0 2023-10-16T10:01:50.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.215+0000 7fbfad1ce640 1 -- 172.21.15.84:0/1485892056 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa40367d0 con 0x7fbfa8159ab0 2023-10-16T10:01:50.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.215+0000 7fbfb0434640 1 -- 172.21.15.84:0/1485892056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa8159ab0 msgr2=0x7fbfa80ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:50.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.215+0000 7fbfb0434640 1 --2- 172.21.15.84:0/1485892056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa8159ab0 0x7fbfa80ce370 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbfa806d900 tx=0x7fbfa402dfb0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.215+0000 7fbfb0434640 1 -- 172.21.15.84:0/1485892056 shutdown_connections 2023-10-16T10:01:50.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.215+0000 7fbfb0434640 1 --2- 172.21.15.84:0/1485892056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfa806c140 0x7fbfa806c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.215+0000 7fbfb0434640 1 --2- 172.21.15.84:0/1485892056 >> v2:172.21.15.157:3300/0 conn(0x7fbfa8159e90 0x7fbfa806fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.215+0000 7fbfb0434640 1 --2- 172.21.15.84:0/1485892056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa8159ab0 0x7fbfa80ce370 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.215+0000 7fbfb0434640 1 -- 172.21.15.84:0/1485892056 >> 172.21.15.84:0/1485892056 conn(0x7fbfa80ce9b0 msgr2=0x7fbfa80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:50.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.216+0000 7fbfb0434640 1 -- 172.21.15.84:0/1485892056 shutdown_connections 2023-10-16T10:01:50.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.216+0000 7fbfb0434640 1 -- 172.21.15.84:0/1485892056 wait complete. 2023-10-16T10:01:50.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.218+0000 7fbfb0434640 1 Processor -- start 2023-10-16T10:01:50.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.218+0000 7fbfb0434640 1 -- start start 2023-10-16T10:01:50.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.219+0000 7fbfb0434640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa806c140 0x7fbfa8154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.219+0000 7fbfb0434640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa8159ab0 0x7fbfa80b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.219+0000 7fbfb0434640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfa8159e90 0x7fbfa80b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.219+0000 7fbfb0434640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa80b30d0 con 0x7fbfa806c140 2023-10-16T10:01:50.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.219+0000 7fbfb0434640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa80c3f90 con 0x7fbfa8159ab0 2023-10-16T10:01:50.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.219+0000 7fbfb0434640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa8155510 con 0x7fbfa8159e90 2023-10-16T10:01:50.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.219+0000 7fbfae1d0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa806c140 0x7fbfa8154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:50.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.219+0000 7fbfae1d0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa806c140 0x7fbfa8154fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38502/0 (socket says 172.21.15.84:38502) 2023-10-16T10:01:50.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.219+0000 7fbfae1d0640 1 -- 172.21.15.84:0/2549682472 learned_addr learned my addr 172.21.15.84:0/2549682472 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:50.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.220+0000 7fbfae1d0640 1 -- 172.21.15.84:0/2549682472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfa8159e90 msgr2=0x7fbfa80b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:50.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.220+0000 7fbfae1d0640 1 --2- 172.21.15.84:0/2549682472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfa8159e90 0x7fbfa80b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.220+0000 7fbfae1d0640 1 -- 172.21.15.84:0/2549682472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa8159ab0 msgr2=0x7fbfa80b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:50.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.220+0000 7fbfad9cf640 1 --2- 172.21.15.84:0/2549682472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa8159ab0 0x7fbfa80b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:50.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.220+0000 7fbfae1d0640 1 --2- 172.21.15.84:0/2549682472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa8159ab0 0x7fbfa80b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.220+0000 7fbfae1d0640 1 -- 172.21.15.84:0/2549682472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa400b770 con 0x7fbfa806c140 2023-10-16T10:01:50.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.220+0000 7fbfad9cf640 1 --2- 172.21.15.84:0/2549682472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa8159ab0 0x7fbfa80b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:50.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.220+0000 7fbfae1d0640 1 --2- 172.21.15.84:0/2549682472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa806c140 0x7fbfa8154fd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbfa402dff0 tx=0x7fbfa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:50.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.220+0000 7fbf9effd640 1 -- 172.21.15.84:0/2549682472 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa403f750 con 0x7fbfa806c140 2023-10-16T10:01:50.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.220+0000 7fbf9effd640 1 -- 172.21.15.84:0/2549682472 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa403f8b0 con 0x7fbfa806c140 2023-10-16T10:01:50.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.220+0000 7fbfb0434640 1 -- 172.21.15.84:0/2549682472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa80c3660 con 0x7fbfa806c140 2023-10-16T10:01:50.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.220+0000 7fbf9effd640 1 -- 172.21.15.84:0/2549682472 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa403fc60 con 0x7fbfa806c140 2023-10-16T10:01:50.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.221+0000 7fbfb0434640 1 -- 172.21.15.84:0/2549682472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa80c3c20 con 0x7fbfa806c140 2023-10-16T10:01:50.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.222+0000 7fbfb0434640 1 -- 172.21.15.84:0/2549682472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa80adf50 con 0x7fbfa806c140 2023-10-16T10:01:50.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.223+0000 7fbf9effd640 1 -- 172.21.15.84:0/2549682472 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fbfa40403e0 con 0x7fbfa806c140 2023-10-16T10:01:50.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.224+0000 7fbf9effd640 1 --2- 172.21.15.84:0/2549682472 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fbf8c06b070 0x7fbf8c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.224+0000 7fbf9effd640 1 -- 172.21.15.84:0/2549682472 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbfa40ac0b0 con 0x7fbfa806c140 2023-10-16T10:01:50.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.224+0000 7fbfad9cf640 1 --2- 172.21.15.84:0/2549682472 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fbf8c06b070 0x7fbf8c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:50.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.225+0000 7fbfad9cf640 1 --2- 172.21.15.84:0/2549682472 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fbf8c06b070 0x7fbf8c06d530 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbf98006fd0 tx=0x7fbf98008040 comp rx=0 tx=0).ready entity=mgr.5322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:50.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.229+0000 7fbf9effd640 1 -- 172.21.15.84:0/2549682472 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbfa80adf50 con 0x7fbfa806c140 2023-10-16T10:01:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.493+0000 7fbfb0434640 1 -- 172.21.15.84:0/2549682472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fbfa80adf50 con 0x7fbfa806c140 2023-10-16T10:01:50.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.865+0000 7fbf9effd640 1 -- 172.21.15.84:0/2549682472 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v17) v1 ==== 93+0+0 (secure 0 0 0) 0x7fbfa80adf50 con 0x7fbfa806c140 2023-10-16T10:01:50.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.872+0000 7fbfb0434640 1 -- 172.21.15.84:0/2549682472 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fbf8c06b070 msgr2=0x7fbf8c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:50.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.872+0000 7fbfb0434640 1 --2- 172.21.15.84:0/2549682472 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fbf8c06b070 0x7fbf8c06d530 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbf98006fd0 tx=0x7fbf98008040 comp rx=0 tx=0).stop 2023-10-16T10:01:50.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.872+0000 7fbfb0434640 1 -- 172.21.15.84:0/2549682472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa806c140 msgr2=0x7fbfa8154fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:50.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.872+0000 7fbfb0434640 1 --2- 172.21.15.84:0/2549682472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa806c140 0x7fbfa8154fd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbfa402dff0 tx=0x7fbfa40046c0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.873+0000 7fbfb0434640 1 -- 172.21.15.84:0/2549682472 shutdown_connections 2023-10-16T10:01:50.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.873+0000 7fbfb0434640 1 --2- 172.21.15.84:0/2549682472 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fbf8c06b070 0x7fbf8c06d530 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbf98006fd0 tx=0x7fbf98008040 comp rx=0 tx=0).stop 2023-10-16T10:01:50.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.873+0000 7fbfb0434640 1 --2- 172.21.15.84:0/2549682472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfa8159e90 0x7fbfa80b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.873+0000 7fbfb0434640 1 --2- 172.21.15.84:0/2549682472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa8159ab0 0x7fbfa80b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.873+0000 7fbfb0434640 1 --2- 172.21.15.84:0/2549682472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa806c140 0x7fbfa8154fd0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.873+0000 7fbfb0434640 1 -- 172.21.15.84:0/2549682472 >> 172.21.15.84:0/2549682472 conn(0x7fbfa80ce9b0 msgr2=0x7fbfa81582b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:50.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.874+0000 7fbfb0434640 1 -- 172.21.15.84:0/2549682472 shutdown_connections 2023-10-16T10:01:50.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:50.875+0000 7fbfb0434640 1 -- 172.21.15.84:0/2549682472 wait complete. 2023-10-16T10:01:50.918 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:01:50.919 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:01:51.017 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:01:51.120 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:01:51.120 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:01:51.121 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:01:51.168 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:01:51.169 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:01:51.263 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:01:51.459 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:51.458+0000 7fc76d413040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:01:51.473 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:51.472+0000 7f4a9d82b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:01:51.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.502+0000 7f82d1d5e640 1 Processor -- start 2023-10-16T10:01:51.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.503+0000 7f82d1d5e640 1 -- start start 2023-10-16T10:01:51.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.504+0000 7f82d1d5e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f82cc15cd70 0x7f82cc0d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.504+0000 7f82d1d5e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f82cc0b8480 0x7f82cc0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.504+0000 7f82d1d5e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82cc15d150 con 0x7f82cc0d0be0 2023-10-16T10:01:51.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.504+0000 7f82d1d5e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82cc0d1180 con 0x7f82cc0b8480 2023-10-16T10:01:51.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.504+0000 7f82d1d5e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82cc0d1460 con 0x7f82cc15cd70 2023-10-16T10:01:51.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.504+0000 7f82d155d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f82cc0d0be0 0x7f82cc0b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34322/0 (socket says 172.21.15.84:34322) 2023-10-16T10:01:51.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.504+0000 7f82d155d640 1 -- 172.21.15.84:0/1282266045 learned_addr learned my addr 172.21.15.84:0/1282266045 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:51.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.504+0000 7f82cbfff640 1 --2- 172.21.15.84:0/1282266045 >> v2:172.21.15.84:3300/0 conn(0x7f82cc0b8480 0x7f82cc0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.505+0000 7f82d0d5c640 1 --2- 172.21.15.84:0/1282266045 >> v2:172.21.15.157:3300/0 conn(0x7f82cc15cd70 0x7f82cc0d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.505+0000 7f82cb7fe640 1 -- 172.21.15.84:0/1282266045 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439712054 0 0) 0x7f82cc15d150 con 0x7f82cc0d0be0 2023-10-16T10:01:51.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.505+0000 7f82cb7fe640 1 -- 172.21.15.84:0/1282266045 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82bc003520 con 0x7f82cc0d0be0 2023-10-16T10:01:51.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.505+0000 7f82cbfff640 1 -- 172.21.15.84:0/1282266045 >> v2:172.21.15.157:3300/0 conn(0x7f82cc15cd70 msgr2=0x7f82cc0d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:51.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.505+0000 7f82cbfff640 1 --2- 172.21.15.84:0/1282266045 >> v2:172.21.15.157:3300/0 conn(0x7f82cc15cd70 0x7f82cc0d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.505+0000 7f82cbfff640 1 -- 172.21.15.84:0/1282266045 >> v1:172.21.15.84:6789/0 conn(0x7f82cc0d0be0 legacy=0x7f82cc0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.505+0000 7f82cbfff640 1 -- 172.21.15.84:0/1282266045 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc0d1740 con 0x7f82cc0b8480 2023-10-16T10:01:51.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.506+0000 7f82cbfff640 1 --2- 172.21.15.84:0/1282266045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82cc0b8480 0x7f82cc0cd7c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0b56a0 tx=0x7f82b802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44dbf3b17549930a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:51.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.507+0000 7f82cb7fe640 1 -- 172.21.15.84:0/1282266045 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b802e070 con 0x7f82cc0b8480 2023-10-16T10:01:51.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.507+0000 7f82cb7fe640 1 -- 172.21.15.84:0/1282266045 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b80315a0 con 0x7f82cc0b8480 2023-10-16T10:01:51.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.507+0000 7f82cb7fe640 1 -- 172.21.15.84:0/1282266045 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8031930 con 0x7f82cc0b8480 2023-10-16T10:01:51.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.507+0000 7f82d1d5e640 1 -- 172.21.15.84:0/1282266045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82cc0b8480 msgr2=0x7f82cc0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.507+0000 7f82d1d5e640 1 --2- 172.21.15.84:0/1282266045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82cc0b8480 0x7f82cc0cd7c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0b56a0 tx=0x7f82b802b840 comp rx=0 tx=0).stop 2023-10-16T10:01:51.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.508+0000 7f82d1d5e640 1 -- 172.21.15.84:0/1282266045 shutdown_connections 2023-10-16T10:01:51.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.508+0000 7f82d1d5e640 1 --2- 172.21.15.84:0/1282266045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82cc0b8480 0x7f82cc0cd7c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.508+0000 7f82d1d5e640 1 --2- 172.21.15.84:0/1282266045 >> v2:172.21.15.157:3300/0 conn(0x7f82cc15cd70 0x7f82cc0d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.508+0000 7f82d1d5e640 1 -- 172.21.15.84:0/1282266045 >> 172.21.15.84:0/1282266045 conn(0x7f82cc0cf570 msgr2=0x7f82cc0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:51.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.508+0000 7f82d1d5e640 1 -- 172.21.15.84:0/1282266045 shutdown_connections 2023-10-16T10:01:51.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.508+0000 7f82d1d5e640 1 -- 172.21.15.84:0/1282266045 wait complete. 2023-10-16T10:01:51.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.510+0000 7f82d1d5e640 1 Processor -- start 2023-10-16T10:01:51.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.510+0000 7f82d1d5e640 1 -- start start 2023-10-16T10:01:51.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.511+0000 7f82d1d5e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82cc0b8480 0x7f82cc0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.511+0000 7f82d1d5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82cc0d0be0 0x7f82cc0b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.511+0000 7f82d1d5e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82cc15cd70 0x7f82cc1f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.511+0000 7f82d1d5e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc1e7010 con 0x7f82cc15cd70 2023-10-16T10:01:51.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.511+0000 7f82d1d5e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc1e6770 con 0x7f82cc0d0be0 2023-10-16T10:01:51.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.511+0000 7f82d1d5e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc1e6a50 con 0x7f82cc0b8480 2023-10-16T10:01:51.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.511+0000 7f82d155d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82cc15cd70 0x7f82cc1f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.511+0000 7f82d155d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82cc15cd70 0x7f82cc1f59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38528/0 (socket says 172.21.15.84:38528) 2023-10-16T10:01:51.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.511+0000 7f82d155d640 1 -- 172.21.15.84:0/2730585750 learned_addr learned my addr 172.21.15.84:0/2730585750 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:51.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.511+0000 7f82cbfff640 1 --2- 172.21.15.84:0/2730585750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82cc0d0be0 0x7f82cc0b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.512+0000 7f82d0d5c640 1 --2- 172.21.15.84:0/2730585750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82cc0b8480 0x7f82cc0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.512+0000 7f82d155d640 1 -- 172.21.15.84:0/2730585750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82cc0b8480 msgr2=0x7f82cc0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.512+0000 7f82d155d640 1 --2- 172.21.15.84:0/2730585750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82cc0b8480 0x7f82cc0b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.512+0000 7f82d155d640 1 -- 172.21.15.84:0/2730585750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82cc0d0be0 msgr2=0x7f82cc0b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.512+0000 7f82d155d640 1 --2- 172.21.15.84:0/2730585750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82cc0d0be0 0x7f82cc0b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.512+0000 7f82d155d640 1 -- 172.21.15.84:0/2730585750 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b8009400 con 0x7f82cc15cd70 2023-10-16T10:01:51.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.512+0000 7f82d0d5c640 1 --2- 172.21.15.84:0/2730585750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82cc0b8480 0x7f82cc0b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:51.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.512+0000 7f82d155d640 1 --2- 172.21.15.84:0/2730585750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82cc15cd70 0x7f82cc1f59e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f82c400caa0 tx=0x7f82c400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:51.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.512+0000 7f82c97fa640 1 -- 172.21.15.84:0/2730585750 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c400ac00 con 0x7f82cc15cd70 2023-10-16T10:01:51.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.512+0000 7f82c97fa640 1 -- 172.21.15.84:0/2730585750 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c400ad60 con 0x7f82cc15cd70 2023-10-16T10:01:51.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.512+0000 7f82d1d5e640 1 -- 172.21.15.84:0/2730585750 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82cc1e6d30 con 0x7f82cc15cd70 2023-10-16T10:01:51.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.512+0000 7f82c97fa640 1 -- 172.21.15.84:0/2730585750 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4012620 con 0x7f82cc15cd70 2023-10-16T10:01:51.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.512+0000 7f82d1d5e640 1 -- 172.21.15.84:0/2730585750 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82cc1e62b0 con 0x7f82cc15cd70 2023-10-16T10:01:51.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.513+0000 7f82d1d5e640 1 -- 172.21.15.84:0/2730585750 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82cc14fd40 con 0x7f82cc15cd70 2023-10-16T10:01:51.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.515+0000 7f82c97fa640 1 -- 172.21.15.84:0/2730585750 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f82c4007070 con 0x7f82cc15cd70 2023-10-16T10:01:51.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.515+0000 7f82c97fa640 1 --2- 172.21.15.84:0/2730585750 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f82b006b020 0x7f82b006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.516+0000 7f82c97fa640 1 -- 172.21.15.84:0/2730585750 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6695+0+0 (secure 0 0 0) 0x7f82c4089c80 con 0x7f82cc15cd70 2023-10-16T10:01:51.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.516+0000 7f82d0d5c640 1 -- 172.21.15.84:0/2730585750 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f82b006b020 msgr2=0x7f82b006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4265272960 2023-10-16T10:01:51.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.516+0000 7f82d0d5c640 1 --2- 172.21.15.84:0/2730585750 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f82b006b020 0x7f82b006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:51.522 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:51.516+0000 7fc76d413040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:01:51.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.520+0000 7f82c97fa640 1 -- 172.21.15.84:0/2730585750 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f82c4054760 con 0x7f82cc15cd70 2023-10-16T10:01:51.532 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:51.531+0000 7f4a9d82b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:01:51.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.717+0000 7f82d0d5c640 1 -- 172.21.15.84:0/2730585750 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f82b006b020 msgr2=0x7f82b006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4265272960 2023-10-16T10:01:51.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.717+0000 7f82d0d5c640 1 --2- 172.21.15.84:0/2730585750 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f82b006b020 0x7f82b006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:51.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.733+0000 7f82d1d5e640 1 -- 172.21.15.84:0/2730585750 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82cc1e6500 con 0x7f82cc15cd70 2023-10-16T10:01:51.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.735+0000 7f82c97fa640 1 -- 172.21.15.84:0/2730585750 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+18045 (secure 0 0 0) 0x7f82c4057d80 con 0x7f82cc15cd70 2023-10-16T10:01:51.737 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:01:51.737 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":47,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:01:50.837632+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:50:10.297512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-16T09:50:11.394493+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":218,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1628587200":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/3968394661":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/1060139177":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/982275204":"2023-10-17T09:50:26.881911+0000","172.21.15.84:6833/3010246666":"2023-10-17T09:50:26.881911+0000","172.21.15.84:6832/3010246666":"2023-10-17T09:50:26.881911+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":3,"length":1},{"begin":5,"length":3},{"begin":9,"length":1},{"begin":11,"length":2},{"begin":14,"length":2},{"begin":17,"length":1},{"begin":20,"length":4},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":2},{"begin":33,"length":3},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":44,"length":2},{"begin":47,"length":3},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":56,"length":1},{"begin":58,"length":6},{"begin":65,"length":1},{"begin":67,"length":2},{"begin":71,"length":3},{"begin":75,"length":4},{"begin":82,"length":1},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":94,"length":3},{"begin":98,"length":1},{"begin":100,"length":1},{"begin":102,"length":2},{"begin":105,"length":1},{"begin":109,"length":3},{"begin":155,"length":64}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":7,"length":1},{"begin":14,"length":1},{"begin":28,"length":1},{"begin":39,"length":1},{"begin":48,"length":1},{"begin":58,"length":1},{"begin":67,"length":1},{"begin":76,"length":1},{"begin":103,"length":1},{"begin":210,"length":9}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:01:51.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.737+0000 7f82d1d5e640 1 -- 172.21.15.84:0/2730585750 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f82b006b020 msgr2=0x7f82b006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:51.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.737+0000 7f82d1d5e640 1 --2- 172.21.15.84:0/2730585750 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f82b006b020 0x7f82b006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.737+0000 7f82d1d5e640 1 -- 172.21.15.84:0/2730585750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82cc15cd70 msgr2=0x7f82cc1f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.737+0000 7f82d1d5e640 1 --2- 172.21.15.84:0/2730585750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82cc15cd70 0x7f82cc1f59e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f82c400caa0 tx=0x7f82c400cf60 comp rx=0 tx=0).stop 2023-10-16T10:01:51.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.737+0000 7f82d1d5e640 1 -- 172.21.15.84:0/2730585750 shutdown_connections 2023-10-16T10:01:51.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.737+0000 7f82d1d5e640 1 --2- 172.21.15.84:0/2730585750 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f82b006b020 0x7f82b006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.737+0000 7f82d1d5e640 1 --2- 172.21.15.84:0/2730585750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82cc15cd70 0x7f82cc1f59e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.737+0000 7f82d1d5e640 1 --2- 172.21.15.84:0/2730585750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82cc0d0be0 0x7f82cc0b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.737+0000 7f82d1d5e640 1 --2- 172.21.15.84:0/2730585750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82cc0b8480 0x7f82cc0b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.737+0000 7f82d1d5e640 1 -- 172.21.15.84:0/2730585750 >> 172.21.15.84:0/2730585750 conn(0x7f82cc0cf570 msgr2=0x7f82cc0d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:51.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.737+0000 7f82d1d5e640 1 -- 172.21.15.84:0/2730585750 shutdown_connections 2023-10-16T10:01:51.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.737+0000 7f82d1d5e640 1 -- 172.21.15.84:0/2730585750 wait complete. 2023-10-16T10:01:51.756 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:50:10.297512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:50:11.394493+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 218, 'snap_epoch': 47, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:01:51.757 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:01:51.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.962+0000 7f723266f640 1 Processor -- start 2023-10-16T10:01:51.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.962+0000 7f723266f640 1 -- start start 2023-10-16T10:01:51.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.963+0000 7f723266f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f722c06b8c0 0x7f722c06bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.963+0000 7f723266f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f722c15d130 0x7f722c0cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.963+0000 7f723266f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f722c0cd390 con 0x7f722c06b1a0 2023-10-16T10:01:51.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.964+0000 7f723266f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f722c06c1e0 con 0x7f722c06b8c0 2023-10-16T10:01:51.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.964+0000 7f723266f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f722c0cc2f0 con 0x7f722c15d130 2023-10-16T10:01:51.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.964+0000 7f722bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f722c06b8c0 0x7f722c06bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.964+0000 7f722bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f722c06b8c0 0x7f722c06bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38540/0 (socket says 172.21.15.84:38540) 2023-10-16T10:01:51.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.964+0000 7f722bfff640 1 -- 172.21.15.84:0/4120575713 learned_addr learned my addr 172.21.15.84:0/4120575713 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:51.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.964+0000 7f7223fff640 1 --2- 172.21.15.84:0/4120575713 >> v2:172.21.15.157:3300/0 conn(0x7f722c15d130 0x7f722c0cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.964+0000 7f722bfff640 1 -- 172.21.15.84:0/4120575713 >> v2:172.21.15.157:3300/0 conn(0x7f722c15d130 msgr2=0x7f722c0cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:51.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.964+0000 7f722bfff640 1 --2- 172.21.15.84:0/4120575713 >> v2:172.21.15.157:3300/0 conn(0x7f722c15d130 0x7f722c0cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.964+0000 7f722bfff640 1 -- 172.21.15.84:0/4120575713 >> v1:172.21.15.157:6789/0 conn(0x7f722c06b1a0 legacy=0x7f722c0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:51.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.964+0000 7f722bfff640 1 -- 172.21.15.84:0/4120575713 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c0cc5d0 con 0x7f722c06b8c0 2023-10-16T10:01:51.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.964+0000 7f7223fff640 1 --2- 172.21.15.84:0/4120575713 >> v2:172.21.15.157:3300/0 conn(0x7f722c15d130 0x7f722c0cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:51.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.965+0000 7f722bfff640 1 --2- 172.21.15.84:0/4120575713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f722c06b8c0 0x7f722c06bca0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f722c073d50 tx=0x7f721c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2a7a76eaf62138 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:51.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.966+0000 7f722b7fe640 1 -- 172.21.15.84:0/4120575713 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c031860 con 0x7f722c06b8c0 2023-10-16T10:01:51.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.966+0000 7f722b7fe640 1 -- 172.21.15.84:0/4120575713 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c0319c0 con 0x7f722c06b8c0 2023-10-16T10:01:51.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.966+0000 7f722b7fe640 1 -- 172.21.15.84:0/4120575713 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c031d50 con 0x7f722c06b8c0 2023-10-16T10:01:51.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.966+0000 7f723266f640 1 -- 172.21.15.84:0/4120575713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f722c06b8c0 msgr2=0x7f722c06bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.966+0000 7f723266f640 1 --2- 172.21.15.84:0/4120575713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f722c06b8c0 0x7f722c06bca0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f722c073d50 tx=0x7f721c02b900 comp rx=0 tx=0).stop 2023-10-16T10:01:51.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.967+0000 7f723266f640 1 -- 172.21.15.84:0/4120575713 shutdown_connections 2023-10-16T10:01:51.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.967+0000 7f723266f640 1 --2- 172.21.15.84:0/4120575713 >> v2:172.21.15.157:3300/0 conn(0x7f722c15d130 0x7f722c0cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.967+0000 7f723266f640 1 --2- 172.21.15.84:0/4120575713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f722c06b8c0 0x7f722c06bca0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.967+0000 7f723266f640 1 -- 172.21.15.84:0/4120575713 >> 172.21.15.84:0/4120575713 conn(0x7f722c0d0ad0 msgr2=0x7f722c0d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:51.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.967+0000 7f723266f640 1 -- 172.21.15.84:0/4120575713 shutdown_connections 2023-10-16T10:01:51.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.967+0000 7f723266f640 1 -- 172.21.15.84:0/4120575713 wait complete. 2023-10-16T10:01:51.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.968+0000 7f723266f640 1 Processor -- start 2023-10-16T10:01:51.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.969+0000 7f723266f640 1 -- start start 2023-10-16T10:01:51.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.970+0000 7f723266f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f722c06b1a0 0x7f722c09a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.970+0000 7f723266f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f722c06b8c0 0x7f722c1503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.970+0000 7f723266f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f722c15d130 0x7f722c14fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.970+0000 7f723266f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c150900 con 0x7f722c06b1a0 2023-10-16T10:01:51.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.970+0000 7f723266f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c09def0 con 0x7f722c06b8c0 2023-10-16T10:01:51.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.970+0000 7f723266f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722c1501a0 con 0x7f722c15d130 2023-10-16T10:01:51.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.970+0000 7f722bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f722c06b1a0 0x7f722c09a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.970+0000 7f722bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f722c06b1a0 0x7f722c09a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38556/0 (socket says 172.21.15.84:38556) 2023-10-16T10:01:51.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.970+0000 7f722bfff640 1 -- 172.21.15.84:0/2540752121 learned_addr learned my addr 172.21.15.84:0/2540752121 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:51.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.970+0000 7f7230c0c640 1 --2- 172.21.15.84:0/2540752121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f722c15d130 0x7f722c14fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.970+0000 7f722bfff640 1 -- 172.21.15.84:0/2540752121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f722c15d130 msgr2=0x7f722c14fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.970+0000 7f722bfff640 1 --2- 172.21.15.84:0/2540752121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f722c15d130 0x7f722c14fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.970+0000 7f722bfff640 1 -- 172.21.15.84:0/2540752121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f722c06b8c0 msgr2=0x7f722c1503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.970+0000 7f722bfff640 1 --2- 172.21.15.84:0/2540752121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f722c06b8c0 0x7f722c1503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.970+0000 7f722bfff640 1 -- 172.21.15.84:0/2540752121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c009470 con 0x7f722c06b1a0 2023-10-16T10:01:51.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.970+0000 7f7230c0c640 1 --2- 172.21.15.84:0/2540752121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f722c15d130 0x7f722c14fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:51.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.971+0000 7f722bfff640 1 --2- 172.21.15.84:0/2540752121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f722c06b1a0 0x7f722c09a7d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f721c000c00 tx=0x7f721c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:51.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.971+0000 7f72297fa640 1 -- 172.21.15.84:0/2540752121 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c031830 con 0x7f722c06b1a0 2023-10-16T10:01:51.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.971+0000 7f72297fa640 1 -- 172.21.15.84:0/2540752121 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c0443f0 con 0x7f722c06b1a0 2023-10-16T10:01:51.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.971+0000 7f723266f640 1 -- 172.21.15.84:0/2540752121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f722c09b1e0 con 0x7f722c06b1a0 2023-10-16T10:01:51.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.971+0000 7f72297fa640 1 -- 172.21.15.84:0/2540752121 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c0447c0 con 0x7f722c06b1a0 2023-10-16T10:01:51.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.971+0000 7f723266f640 1 -- 172.21.15.84:0/2540752121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722c09e190 con 0x7f722c06b1a0 2023-10-16T10:01:51.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.972+0000 7f723266f640 1 -- 172.21.15.84:0/2540752121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f722c09e3b0 con 0x7f722c06b1a0 2023-10-16T10:01:51.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.973+0000 7f72297fa640 1 -- 172.21.15.84:0/2540752121 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f721c04d450 con 0x7f722c06b1a0 2023-10-16T10:01:51.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.974+0000 7f72297fa640 1 --2- 172.21.15.84:0/2540752121 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f720c06b020 0x7f720c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.974+0000 7f72297fa640 1 -- 172.21.15.84:0/2540752121 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6583+0+0 (secure 0 0 0) 0x7f721c0ac6a0 con 0x7f722c06b1a0 2023-10-16T10:01:51.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.974+0000 7f7223fff640 1 -- 172.21.15.84:0/2540752121 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f720c06b020 msgr2=0x7f720c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4265272960 2023-10-16T10:01:51.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.974+0000 7f7223fff640 1 --2- 172.21.15.84:0/2540752121 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f720c06b020 0x7f720c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:51.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:51.980+0000 7f72297fa640 1 -- 172.21.15.84:0/2540752121 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f721c076ea0 con 0x7f722c06b1a0 2023-10-16T10:01:52.015 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:52.013+0000 7f4a9d82b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:01:52.018 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:52.017+0000 7fc76d413040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:01:52.068 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:52.066+0000 7f4a9d82b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:01:52.072 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:52.071+0000 7fc76d413040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:01:52.118 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:52.117+0000 7f4a9d82b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:01:52.124 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:52.123+0000 7fc76d413040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:01:52.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.175+0000 7f7223fff640 1 -- 172.21.15.84:0/2540752121 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f720c06b020 msgr2=0x7f720c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4265272960 2023-10-16T10:01:52.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.175+0000 7f7223fff640 1 --2- 172.21.15.84:0/2540752121 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f720c06b020 0x7f720c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:52.178 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:52.177+0000 7f4a9d82b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:01:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.181+0000 7f723266f640 1 -- 172.21.15.84:0/2540752121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f722c04e160 con 0x7f722c06b1a0 2023-10-16T10:01:52.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.181+0000 7f72297fa640 1 -- 172.21.15.84:0/2540752121 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f721c07a4c0 con 0x7f722c06b1a0 2023-10-16T10:01:52.183 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:01:52.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.184+0000 7f723266f640 1 -- 172.21.15.84:0/2540752121 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f720c06b020 msgr2=0x7f720c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:52.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.184+0000 7f723266f640 1 --2- 172.21.15.84:0/2540752121 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f720c06b020 0x7f720c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.184+0000 7f723266f640 1 -- 172.21.15.84:0/2540752121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f722c06b1a0 msgr2=0x7f722c09a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.184+0000 7f723266f640 1 --2- 172.21.15.84:0/2540752121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f722c06b1a0 0x7f722c09a7d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f721c000c00 tx=0x7f721c0046c0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.184+0000 7f723266f640 1 -- 172.21.15.84:0/2540752121 shutdown_connections 2023-10-16T10:01:52.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.184+0000 7f723266f640 1 --2- 172.21.15.84:0/2540752121 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f720c06b020 0x7f720c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.184+0000 7f723266f640 1 --2- 172.21.15.84:0/2540752121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f722c15d130 0x7f722c14fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.184+0000 7f723266f640 1 --2- 172.21.15.84:0/2540752121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f722c06b8c0 0x7f722c1503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.184+0000 7f723266f640 1 --2- 172.21.15.84:0/2540752121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f722c06b1a0 0x7f722c09a7d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.184+0000 7f723266f640 1 -- 172.21.15.84:0/2540752121 >> 172.21.15.84:0/2540752121 conn(0x7f722c0d0ad0 msgr2=0x7f722c06ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:52.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.184+0000 7f723266f640 1 -- 172.21.15.84:0/2540752121 shutdown_connections 2023-10-16T10:01:52.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.184+0000 7f723266f640 1 -- 172.21.15.84:0/2540752121 wait complete. 2023-10-16T10:01:52.188 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:52.186+0000 7fc76d413040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:01:52.204 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:01:52.227 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:52.226+0000 7f4a9d82b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:01:52.240 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:52.239+0000 7fc76d413040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:01:52.339 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:52.338+0000 7f4a9d82b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:01:52.356 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:52.355+0000 7fc76d413040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:01:52.404 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:52.403+0000 7f4a9d82b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:01:52.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.406+0000 7efd3bb41640 1 Processor -- start 2023-10-16T10:01:52.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.406+0000 7efd3bb41640 1 -- start start 2023-10-16T10:01:52.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.408+0000 7efd3bb41640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efd34160ad0 0x7efd340ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.408+0000 7efd3bb41640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efd340ce350 0x7efd34160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.408+0000 7efd3bb41640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd340ce800 con 0x7efd340ae260 2023-10-16T10:01:52.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.408+0000 7efd3bb41640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd340cdec0 con 0x7efd340ce350 2023-10-16T10:01:52.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.408+0000 7efd3bb41640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd340acdd0 con 0x7efd34160ad0 2023-10-16T10:01:52.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.408+0000 7efd390dc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efd340ce350 0x7efd34160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.408+0000 7efd390dc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efd340ce350 0x7efd34160eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38570/0 (socket says 172.21.15.84:38570) 2023-10-16T10:01:52.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.408+0000 7efd390dc640 1 -- 172.21.15.84:0/160784967 learned_addr learned my addr 172.21.15.84:0/160784967 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:52.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.408+0000 7efd398dd640 1 --2- 172.21.15.84:0/160784967 >> v2:172.21.15.157:3300/0 conn(0x7efd34160ad0 0x7efd340ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.409+0000 7efd390dc640 1 -- 172.21.15.84:0/160784967 >> v2:172.21.15.157:3300/0 conn(0x7efd34160ad0 msgr2=0x7efd340ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:52.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.409+0000 7efd390dc640 1 --2- 172.21.15.84:0/160784967 >> v2:172.21.15.157:3300/0 conn(0x7efd34160ad0 0x7efd340ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.409+0000 7efd390dc640 1 -- 172.21.15.84:0/160784967 >> v1:172.21.15.157:6789/0 conn(0x7efd340ae260 legacy=0x7efd340ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:52.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.409+0000 7efd390dc640 1 -- 172.21.15.84:0/160784967 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd341613f0 con 0x7efd340ce350 2023-10-16T10:01:52.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.409+0000 7efd398dd640 1 --2- 172.21.15.84:0/160784967 >> v2:172.21.15.157:3300/0 conn(0x7efd34160ad0 0x7efd340ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:52.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.410+0000 7efd390dc640 1 --2- 172.21.15.84:0/160784967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd340ce350 0x7efd34160eb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efd340721f0 tx=0x7efd2402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628fa8e400f52b40 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:52.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.410+0000 7efd388db640 1 -- 172.21.15.84:0/160784967 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd24031870 con 0x7efd340ce350 2023-10-16T10:01:52.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.410+0000 7efd388db640 1 -- 172.21.15.84:0/160784967 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd240319d0 con 0x7efd340ce350 2023-10-16T10:01:52.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.410+0000 7efd388db640 1 -- 172.21.15.84:0/160784967 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd24031d60 con 0x7efd340ce350 2023-10-16T10:01:52.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.411+0000 7efd3bb41640 1 -- 172.21.15.84:0/160784967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd340ce350 msgr2=0x7efd34160eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.411+0000 7efd3bb41640 1 --2- 172.21.15.84:0/160784967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd340ce350 0x7efd34160eb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efd340721f0 tx=0x7efd2402b910 comp rx=0 tx=0).stop 2023-10-16T10:01:52.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.411+0000 7efd3bb41640 1 -- 172.21.15.84:0/160784967 shutdown_connections 2023-10-16T10:01:52.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.411+0000 7efd3bb41640 1 --2- 172.21.15.84:0/160784967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd340ce350 0x7efd34160eb0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.411+0000 7efd3bb41640 1 --2- 172.21.15.84:0/160784967 >> v2:172.21.15.157:3300/0 conn(0x7efd34160ad0 0x7efd340ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.411+0000 7efd3bb41640 1 -- 172.21.15.84:0/160784967 >> 172.21.15.84:0/160784967 conn(0x7efd3406bc10 msgr2=0x7efd3406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:52.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.411+0000 7efd3bb41640 1 -- 172.21.15.84:0/160784967 shutdown_connections 2023-10-16T10:01:52.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.411+0000 7efd3bb41640 1 -- 172.21.15.84:0/160784967 wait complete. 2023-10-16T10:01:52.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.413+0000 7efd3bb41640 1 Processor -- start 2023-10-16T10:01:52.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.413+0000 7efd3bb41640 1 -- start start 2023-10-16T10:01:52.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.414+0000 7efd3bb41640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd340ae260 0x7efd3415d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.414+0000 7efd3bb41640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd340ce350 0x7efd3415f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.414+0000 7efd3bb41640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd34160ad0 0x7efd3415f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.414+0000 7efd3bb41640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd34070610 con 0x7efd340ae260 2023-10-16T10:01:52.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.414+0000 7efd3bb41640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd340b4a80 con 0x7efd34160ad0 2023-10-16T10:01:52.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.414+0000 7efd3bb41640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd340708a0 con 0x7efd340ce350 2023-10-16T10:01:52.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.414+0000 7efd398dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd340ae260 0x7efd3415d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.414+0000 7efd398dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd340ae260 0x7efd3415d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38586/0 (socket says 172.21.15.84:38586) 2023-10-16T10:01:52.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.414+0000 7efd398dd640 1 -- 172.21.15.84:0/2706947919 learned_addr learned my addr 172.21.15.84:0/2706947919 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:52.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.414+0000 7efd390dc640 1 --2- 172.21.15.84:0/2706947919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd340ce350 0x7efd3415f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.414+0000 7efd398dd640 1 -- 172.21.15.84:0/2706947919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd340ce350 msgr2=0x7efd3415f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.414+0000 7efd398dd640 1 --2- 172.21.15.84:0/2706947919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd340ce350 0x7efd3415f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.414+0000 7efd398dd640 1 -- 172.21.15.84:0/2706947919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd34160ad0 msgr2=0x7efd3415f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.414+0000 7efd3a0de640 1 --2- 172.21.15.84:0/2706947919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd34160ad0 0x7efd3415f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.414+0000 7efd398dd640 1 --2- 172.21.15.84:0/2706947919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd34160ad0 0x7efd3415f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.414+0000 7efd398dd640 1 -- 172.21.15.84:0/2706947919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd240094d0 con 0x7efd340ae260 2023-10-16T10:01:52.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.415+0000 7efd398dd640 1 --2- 172.21.15.84:0/2706947919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd340ae260 0x7efd3415d2e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c7f0 tx=0x7efd3000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:52.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.415+0000 7efd2a7fc640 1 -- 172.21.15.84:0/2706947919 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30007a30 con 0x7efd340ae260 2023-10-16T10:01:52.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.415+0000 7efd2a7fc640 1 -- 172.21.15.84:0/2706947919 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd30007b90 con 0x7efd340ae260 2023-10-16T10:01:52.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.415+0000 7efd2a7fc640 1 -- 172.21.15.84:0/2706947919 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3000ce40 con 0x7efd340ae260 2023-10-16T10:01:52.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.415+0000 7efd3bb41640 1 -- 172.21.15.84:0/2706947919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd34070b80 con 0x7efd340ae260 2023-10-16T10:01:52.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.415+0000 7efd3bb41640 1 -- 172.21.15.84:0/2706947919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd34071140 con 0x7efd340ae260 2023-10-16T10:01:52.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.416+0000 7efd3bb41640 1 -- 172.21.15.84:0/2706947919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3404e160 con 0x7efd340ae260 2023-10-16T10:01:52.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.417+0000 7efd2a7fc640 1 -- 172.21.15.84:0/2706947919 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) v1 ==== 89400+0+0 (secure 0 0 0) 0x7efd300113f0 con 0x7efd340ae260 2023-10-16T10:01:52.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.418+0000 7efd2a7fc640 1 --2- 172.21.15.84:0/2706947919 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efd1806b020 0x7efd1806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.418+0000 7efd2a7fc640 1 -- 172.21.15.84:0/2706947919 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6583+0+0 (secure 0 0 0) 0x7efd30089fc0 con 0x7efd340ae260 2023-10-16T10:01:52.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.418+0000 7efd390dc640 1 -- 172.21.15.84:0/2706947919 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efd1806b020 msgr2=0x7efd1806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4265272960 2023-10-16T10:01:52.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.418+0000 7efd390dc640 1 --2- 172.21.15.84:0/2706947919 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efd1806b020 0x7efd1806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:52.427 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:52.426+0000 7fc76d413040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:01:52.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.427+0000 7efd2a7fc640 1 -- 172.21.15.84:0/2706947919 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efd30054960 con 0x7efd340ae260 2023-10-16T10:01:52.455 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:52.453+0000 7f4a9d82b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:01:52.478 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:52.476+0000 7fc76d413040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:01:52.511 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:52.509+0000 7f4a9d82b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:01:52.536 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:52.534+0000 7fc76d413040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:01:52.562 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:52.559+0000 7f4a9d82b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:01:52.587 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:52.585+0000 7fc76d413040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:01:52.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.618+0000 7efd3bb41640 1 -- 172.21.15.84:0/2706947919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efd34065d60 con 0x7efd340ae260 2023-10-16T10:01:52.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.619+0000 7efd2a7fc640 1 -- 172.21.15.84:0/2706947919 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) v1 ==== 104+0+11 (secure 0 0 0) 0x7efd30057f80 con 0x7efd340ae260 2023-10-16T10:01:52.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.619+0000 7efd390dc640 1 -- 172.21.15.84:0/2706947919 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efd1806b020 msgr2=0x7efd1806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4265272960 2023-10-16T10:01:52.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.619+0000 7efd390dc640 1 --2- 172.21.15.84:0/2706947919 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efd1806b020 0x7efd1806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:52.622 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T10:01:52.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.621+0000 7efd3bb41640 1 -- 172.21.15.84:0/2706947919 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efd1806b020 msgr2=0x7efd1806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:52.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.621+0000 7efd3bb41640 1 --2- 172.21.15.84:0/2706947919 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efd1806b020 0x7efd1806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.621+0000 7efd3bb41640 1 -- 172.21.15.84:0/2706947919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd340ae260 msgr2=0x7efd3415d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.621+0000 7efd3bb41640 1 --2- 172.21.15.84:0/2706947919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd340ae260 0x7efd3415d2e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c7f0 tx=0x7efd3000cbf0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.621+0000 7efd3bb41640 1 -- 172.21.15.84:0/2706947919 shutdown_connections 2023-10-16T10:01:52.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.621+0000 7efd3bb41640 1 --2- 172.21.15.84:0/2706947919 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7efd1806b020 0x7efd1806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.621+0000 7efd3bb41640 1 --2- 172.21.15.84:0/2706947919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd34160ad0 0x7efd3415f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.621+0000 7efd3bb41640 1 --2- 172.21.15.84:0/2706947919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd340ce350 0x7efd3415f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.621+0000 7efd3bb41640 1 --2- 172.21.15.84:0/2706947919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd340ae260 0x7efd3415d2e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.621+0000 7efd3bb41640 1 -- 172.21.15.84:0/2706947919 >> 172.21.15.84:0/2706947919 conn(0x7efd3406bc10 msgr2=0x7efd340ccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:52.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.622+0000 7efd3bb41640 1 -- 172.21.15.84:0/2706947919 shutdown_connections 2023-10-16T10:01:52.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.622+0000 7efd3bb41640 1 -- 172.21.15.84:0/2706947919 wait complete. 2023-10-16T10:01:52.641 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:01:52.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.899+0000 7f1a06fce640 1 Processor -- start 2023-10-16T10:01:52.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.899+0000 7f1a06fce640 1 -- start start 2023-10-16T10:01:52.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.900+0000 7f1a06fce640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1a0015d330 0x7f1a001618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.900+0000 7f1a06fce640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0006c7d0 0x7f1a00065ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.900+0000 7f1a06fce640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a000d0890 con 0x7f1a0006cc80 2023-10-16T10:01:52.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.900+0000 7f1a06fce640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0006b1e0 con 0x7f1a0015d330 2023-10-16T10:01:52.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.900+0000 7f1a06fce640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000d05b0 con 0x7f1a0006c7d0 2023-10-16T10:01:52.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.901+0000 7f1a0556b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1a0006cc80 0x7f1a000ce9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34330/0 (socket says 172.21.15.84:34330) 2023-10-16T10:01:52.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.901+0000 7f1a0556b640 1 -- 172.21.15.84:0/3796964326 learned_addr learned my addr 172.21.15.84:0/3796964326 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:52.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.901+0000 7f19fffff640 1 --2- 172.21.15.84:0/3796964326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0006c7d0 0x7f1a00065ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.901+0000 7f1a04d6a640 1 --2- 172.21.15.84:0/3796964326 >> v2:172.21.15.157:3300/0 conn(0x7f1a0015d330 0x7f1a001618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.901+0000 7f19ff7fe640 1 -- 172.21.15.84:0/3796964326 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355672998 0 0) 0x7f1a000d0890 con 0x7f1a0006cc80 2023-10-16T10:01:52.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.901+0000 7f19ff7fe640 1 -- 172.21.15.84:0/3796964326 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19f0003520 con 0x7f1a0006cc80 2023-10-16T10:01:52.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.901+0000 7f19ff7fe640 1 -- 172.21.15.84:0/3796964326 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4293846604 0 0) 0x7f19f0003520 con 0x7f1a0006cc80 2023-10-16T10:01:52.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.901+0000 7f19ff7fe640 1 -- 172.21.15.84:0/3796964326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0006c7d0 msgr2=0x7f1a00065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:52.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.901+0000 7f19ff7fe640 1 --2- 172.21.15.84:0/3796964326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0006c7d0 0x7f1a00065ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.902+0000 7f19ff7fe640 1 -- 172.21.15.84:0/3796964326 >> v2:172.21.15.157:3300/0 conn(0x7f1a0015d330 msgr2=0x7f1a001618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:52.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.902+0000 7f19ff7fe640 1 --2- 172.21.15.84:0/3796964326 >> v2:172.21.15.157:3300/0 conn(0x7f1a0015d330 0x7f1a001618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.902+0000 7f19ff7fe640 1 -- 172.21.15.84:0/3796964326 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a00160dd0 con 0x7f1a0006cc80 2023-10-16T10:01:52.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.902+0000 7f19ff7fe640 1 -- 172.21.15.84:0/3796964326 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f19f80019f0 con 0x7f1a0006cc80 2023-10-16T10:01:52.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.902+0000 7f19ff7fe640 1 -- 172.21.15.84:0/3796964326 >> v1:172.21.15.84:6789/0 conn(0x7f1a0006cc80 legacy=0x7f1a000ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.902+0000 7f19ff7fe640 1 --2- 172.21.15.84:0/3796964326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0006c7d0 0x7f19f0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.902+0000 7f19ff7fe640 1 --2- 172.21.15.84:0/3796964326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0003f40 0x7f19f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.902+0000 7f19ff7fe640 1 --2- 172.21.15.84:0/3796964326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0004d00 0x7f19f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.902+0000 7f19ff7fe640 1 -- 172.21.15.84:0/3796964326 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0006b1e0 con 0x7f19f0003f40 2023-10-16T10:01:52.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.902+0000 7f19ff7fe640 1 -- 172.21.15.84:0/3796964326 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000d05b0 con 0x7f19f0004d00 2023-10-16T10:01:52.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.902+0000 7f19ff7fe640 1 -- 172.21.15.84:0/3796964326 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f000ba30 con 0x7f1a0006c7d0 2023-10-16T10:01:52.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.902+0000 7f19fffff640 1 --2- 172.21.15.84:0/3796964326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0003f40 0x7f19f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.902+0000 7f19fffff640 1 -- 172.21.15.84:0/3796964326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0006c7d0 msgr2=0x7f19f0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:52.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.902+0000 7f1a04d6a640 1 --2- 172.21.15.84:0/3796964326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0006c7d0 0x7f19f0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.902+0000 7f19fffff640 1 --2- 172.21.15.84:0/3796964326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a0006c7d0 0x7f19f0003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.902+0000 7f19fffff640 1 -- 172.21.15.84:0/3796964326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0004d00 msgr2=0x7f19f0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:52.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.902+0000 7f19fffff640 1 --2- 172.21.15.84:0/3796964326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0004d00 0x7f19f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.902+0000 7f19fffff640 1 -- 172.21.15.84:0/3796964326 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f0003520 con 0x7f19f0003f40 2023-10-16T10:01:52.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.903+0000 7f19fffff640 1 --2- 172.21.15.84:0/3796964326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0003f40 0x7f19f00047c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009ac0 tx=0x7f19f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f92f64a74850bba server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:52.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.904+0000 7f19ff7fe640 1 -- 172.21.15.84:0/3796964326 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec02c6b0 con 0x7f19f0003f40 2023-10-16T10:01:52.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.904+0000 7f19ff7fe640 1 -- 172.21.15.84:0/3796964326 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ec02c810 con 0x7f19f0003f40 2023-10-16T10:01:52.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.904+0000 7f19ff7fe640 1 -- 172.21.15.84:0/3796964326 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec02cba0 con 0x7f19f0003f40 2023-10-16T10:01:52.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.904+0000 7f1a06fce640 1 -- 172.21.15.84:0/3796964326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0003f40 msgr2=0x7f19f00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.904+0000 7f1a06fce640 1 --2- 172.21.15.84:0/3796964326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0003f40 0x7f19f00047c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009ac0 tx=0x7f19f00057f0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.904+0000 7f1a0556b640 1 -- 172.21.15.84:0/3796964326 reap_dead start 2023-10-16T10:01:52.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.904+0000 7f1a06fce640 1 -- 172.21.15.84:0/3796964326 shutdown_connections 2023-10-16T10:01:52.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.905+0000 7f1a06fce640 1 -- 172.21.15.84:0/3796964326 >> 172.21.15.84:0/3796964326 conn(0x7f1a000d0ab0 msgr2=0x7f1a000d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:52.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.905+0000 7f1a06fce640 1 -- 172.21.15.84:0/3796964326 shutdown_connections 2023-10-16T10:01:52.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.905+0000 7f1a06fce640 1 -- 172.21.15.84:0/3796964326 wait complete. 2023-10-16T10:01:52.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.906+0000 7f1a06fce640 1 Processor -- start 2023-10-16T10:01:52.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.906+0000 7f1a06fce640 1 -- start start 2023-10-16T10:01:52.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.907+0000 7f1a06fce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0003f40 0x7f1a000b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.907+0000 7f1a06fce640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a001e67d0 0x7f1a001546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.907+0000 7f1a06fce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a00154e10 0x7f1a000b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.907+0000 7f1a06fce640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000b1240 con 0x7f1a00154e10 2023-10-16T10:01:52.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.907+0000 7f1a06fce640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000b2a00 con 0x7f19f0003f40 2023-10-16T10:01:52.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.907+0000 7f1a06fce640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a00154be0 con 0x7f1a001e67d0 2023-10-16T10:01:52.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.908+0000 7f1a0556b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a00154e10 0x7f1a000b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.908+0000 7f1a0556b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a00154e10 0x7f1a000b1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38606/0 (socket says 172.21.15.84:38606) 2023-10-16T10:01:52.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.908+0000 7f1a0556b640 1 -- 172.21.15.84:0/2908173802 learned_addr learned my addr 172.21.15.84:0/2908173802 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:52.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.908+0000 7f19fffff640 1 --2- 172.21.15.84:0/2908173802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a001e67d0 0x7f1a001546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.908+0000 7f1a04d6a640 1 --2- 172.21.15.84:0/2908173802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0003f40 0x7f1a000b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.908+0000 7f1a0556b640 1 -- 172.21.15.84:0/2908173802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a001e67d0 msgr2=0x7f1a001546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.908+0000 7f1a0556b640 1 --2- 172.21.15.84:0/2908173802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a001e67d0 0x7f1a001546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.908+0000 7f1a0556b640 1 -- 172.21.15.84:0/2908173802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0003f40 msgr2=0x7f1a000b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.908+0000 7f1a0556b640 1 --2- 172.21.15.84:0/2908173802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0003f40 0x7f1a000b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.908+0000 7f1a0556b640 1 -- 172.21.15.84:0/2908173802 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a000b2f80 con 0x7f1a00154e10 2023-10-16T10:01:52.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.908+0000 7f1a0556b640 1 --2- 172.21.15.84:0/2908173802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a00154e10 0x7f1a000b1e30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1a000cb000 tx=0x7f19f800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:52.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.909+0000 7f19fd7fa640 1 -- 172.21.15.84:0/2908173802 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f800a610 con 0x7f1a00154e10 2023-10-16T10:01:52.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.909+0000 7f19fd7fa640 1 -- 172.21.15.84:0/2908173802 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f800a770 con 0x7f1a00154e10 2023-10-16T10:01:52.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.909+0000 7f19fd7fa640 1 -- 172.21.15.84:0/2908173802 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f800ab20 con 0x7f1a00154e10 2023-10-16T10:01:52.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.909+0000 7f1a06fce640 1 -- 172.21.15.84:0/2908173802 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a000b2ca0 con 0x7f1a00154e10 2023-10-16T10:01:52.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.909+0000 7f1a06fce640 1 -- 172.21.15.84:0/2908173802 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a001e94b0 con 0x7f1a00154e10 2023-10-16T10:01:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.910+0000 7f1a06fce640 1 -- 172.21.15.84:0/2908173802 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a000b1380 con 0x7f1a00154e10 2023-10-16T10:01:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.911+0000 7f19fd7fa640 1 -- 172.21.15.84:0/2908173802 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f19f800ac80 con 0x7f1a00154e10 2023-10-16T10:01:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.912+0000 7f19fd7fa640 1 --2- 172.21.15.84:0/2908173802 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f19e406b050 0x7f19e406d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.912+0000 7f19fd7fa640 1 -- 172.21.15.84:0/2908173802 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6791+0+0 (secure 0 0 0) 0x7f19f8089960 con 0x7f1a00154e10 2023-10-16T10:01:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.912+0000 7f1a04d6a640 1 -- 172.21.15.84:0/2908173802 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f19e406b050 msgr2=0x7f19e406d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4265272960 2023-10-16T10:01:52.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.912+0000 7f1a04d6a640 1 --2- 172.21.15.84:0/2908173802 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f19e406b050 0x7f19e406d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:52.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:52.916+0000 7f19fd7fa640 1 -- 172.21.15.84:0/2908173802 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1a000b1380 con 0x7f1a00154e10 2023-10-16T10:01:52.949 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:52.948+0000 7f4a9d82b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:01:52.991 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:52.989+0000 7fc76d413040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:01:53.108 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:53.107+0000 7f4a9d82b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:01:53.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.110+0000 7f1a06fce640 1 -- 172.21.15.84:0/2908173802 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1a000b1380 con 0x7f1a00154e10 2023-10-16T10:01:53.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.111+0000 7f19fd7fa640 1 -- 172.21.15.84:0/2908173802 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1a000b1380 con 0x7f1a00154e10 2023-10-16T10:01:53.112 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T10:01:53.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.113+0000 7f1a04d6a640 1 -- 172.21.15.84:0/2908173802 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f19e406b050 msgr2=0x7f19e406d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4265272960 2023-10-16T10:01:53.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.113+0000 7f1a04d6a640 1 --2- 172.21.15.84:0/2908173802 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f19e406b050 0x7f19e406d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:53.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.113+0000 7f1a06fce640 1 -- 172.21.15.84:0/2908173802 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f19e406b050 msgr2=0x7f19e406d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:53.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.113+0000 7f1a06fce640 1 --2- 172.21.15.84:0/2908173802 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f19e406b050 0x7f19e406d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.113+0000 7f1a06fce640 1 -- 172.21.15.84:0/2908173802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a00154e10 msgr2=0x7f1a000b1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:53.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.113+0000 7f1a06fce640 1 --2- 172.21.15.84:0/2908173802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a00154e10 0x7f1a000b1e30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1a000cb000 tx=0x7f19f800cf00 comp rx=0 tx=0).stop 2023-10-16T10:01:53.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.113+0000 7f1a06fce640 1 -- 172.21.15.84:0/2908173802 shutdown_connections 2023-10-16T10:01:53.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.113+0000 7f1a06fce640 1 --2- 172.21.15.84:0/2908173802 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f19e406b050 0x7f19e406d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.113+0000 7f1a06fce640 1 --2- 172.21.15.84:0/2908173802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a00154e10 0x7f1a000b1e30 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.113+0000 7f1a06fce640 1 --2- 172.21.15.84:0/2908173802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a001e67d0 0x7f1a001546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.113+0000 7f1a06fce640 1 --2- 172.21.15.84:0/2908173802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0003f40 0x7f1a000b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.113+0000 7f1a06fce640 1 -- 172.21.15.84:0/2908173802 >> 172.21.15.84:0/2908173802 conn(0x7f1a000d0ab0 msgr2=0x7f1a0015e310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:53.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.114+0000 7f1a06fce640 1 -- 172.21.15.84:0/2908173802 shutdown_connections 2023-10-16T10:01:53.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.114+0000 7f1a06fce640 1 -- 172.21.15.84:0/2908173802 wait complete. 2023-10-16T10:01:53.133 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T10:01:53.159 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:53.158+0000 7fc76d413040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:01:53.221 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:53.219+0000 7f4a9d82b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:01:53.271 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:53.269+0000 7f4a9d82b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:01:53.275 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:53.274+0000 7fc76d413040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:01:53.328 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:53.326+0000 7fc76d413040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:01:53.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.339+0000 7f9341c00640 1 Processor -- start 2023-10-16T10:01:53.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.339+0000 7f9341c00640 1 -- start start 2023-10-16T10:01:53.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.341+0000 7f9341c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f933c15d030 0x7f933c0d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.341+0000 7f9341c00640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f933c0cd260 0x7f933c0cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.341+0000 7f9341c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933c0b5d90 con 0x7f933c15d410 2023-10-16T10:01:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.341+0000 7f9341c00640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933c0d2b50 con 0x7f933c0cd260 2023-10-16T10:01:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.341+0000 7f9341c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933c0cdbf0 con 0x7f933c15d030 2023-10-16T10:01:53.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.342+0000 7f933bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f933c0cd260 0x7f933c0cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:53.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.342+0000 7f933bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f933c0cd260 0x7f933c0cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38612/0 (socket says 172.21.15.84:38612) 2023-10-16T10:01:53.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.342+0000 7f933bfff640 1 -- 172.21.15.84:0/61924595 learned_addr learned my addr 172.21.15.84:0/61924595 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:53.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.342+0000 7f933bfff640 1 -- 172.21.15.84:0/61924595 >> v2:172.21.15.157:3300/0 conn(0x7f933c15d030 msgr2=0x7f933c0d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:53.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.342+0000 7f933bfff640 1 --2- 172.21.15.84:0/61924595 >> v2:172.21.15.157:3300/0 conn(0x7f933c15d030 0x7f933c0d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.342+0000 7f933bfff640 1 -- 172.21.15.84:0/61924595 >> v1:172.21.15.157:6789/0 conn(0x7f933c15d410 legacy=0x7f933c161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:53.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.342+0000 7f933bfff640 1 -- 172.21.15.84:0/61924595 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c160fc0 con 0x7f933c0cd260 2023-10-16T10:01:53.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.343+0000 7f933bfff640 1 --2- 172.21.15.84:0/61924595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f933c0cd260 0x7f933c0cd6b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f933c06cd10 tx=0x7f933402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2301cef80baa5e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:53.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.344+0000 7f933affd640 1 -- 172.21.15.84:0/61924595 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933402cb20 con 0x7f933c0cd260 2023-10-16T10:01:53.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.344+0000 7f933affd640 1 -- 172.21.15.84:0/61924595 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933402cc80 con 0x7f933c0cd260 2023-10-16T10:01:53.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.344+0000 7f933affd640 1 -- 172.21.15.84:0/61924595 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93340343f0 con 0x7f933c0cd260 2023-10-16T10:01:53.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.344+0000 7f9341c00640 1 -- 172.21.15.84:0/61924595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f933c0cd260 msgr2=0x7f933c0cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:53.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.344+0000 7f9341c00640 1 --2- 172.21.15.84:0/61924595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f933c0cd260 0x7f933c0cd6b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f933c06cd10 tx=0x7f933402bf80 comp rx=0 tx=0).stop 2023-10-16T10:01:53.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.344+0000 7f9341c00640 1 -- 172.21.15.84:0/61924595 shutdown_connections 2023-10-16T10:01:53.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.344+0000 7f9341c00640 1 --2- 172.21.15.84:0/61924595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f933c0cd260 0x7f933c0cd6b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.345+0000 7f9341c00640 1 --2- 172.21.15.84:0/61924595 >> v2:172.21.15.157:3300/0 conn(0x7f933c15d030 0x7f933c0d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.345+0000 7f9341c00640 1 -- 172.21.15.84:0/61924595 >> 172.21.15.84:0/61924595 conn(0x7f933c0cf720 msgr2=0x7f933c0cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:53.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.345+0000 7f9341c00640 1 -- 172.21.15.84:0/61924595 shutdown_connections 2023-10-16T10:01:53.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.345+0000 7f9341c00640 1 -- 172.21.15.84:0/61924595 wait complete. 2023-10-16T10:01:53.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.346+0000 7f9341c00640 1 Processor -- start 2023-10-16T10:01:53.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.346+0000 7f9341c00640 1 -- start start 2023-10-16T10:01:53.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.347+0000 7f9341c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933c0cd260 0x7f933c0c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:53.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.347+0000 7f9341c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f933c15d030 0x7f933c1e6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:53.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.347+0000 7f9341c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f933c15d410 0x7f933c0c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:53.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.347+0000 7f9341c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0ccba0 con 0x7f933c15d030 2023-10-16T10:01:53.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.347+0000 7f9341c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c1e7c60 con 0x7f933c0cd260 2023-10-16T10:01:53.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.347+0000 7f9341c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933c1e73c0 con 0x7f933c15d410 2023-10-16T10:01:53.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.348+0000 7f9333fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f933c15d030 0x7f933c1e6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:53.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.348+0000 7f9333fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f933c15d030 0x7f933c1e6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38620/0 (socket says 172.21.15.84:38620) 2023-10-16T10:01:53.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.348+0000 7f9333fff640 1 -- 172.21.15.84:0/4248361389 learned_addr learned my addr 172.21.15.84:0/4248361389 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:53.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.348+0000 7f933bfff640 1 --2- 172.21.15.84:0/4248361389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f933c15d410 0x7f933c0c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:53.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.348+0000 7f933b7fe640 1 --2- 172.21.15.84:0/4248361389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933c0cd260 0x7f933c0c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:53.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.348+0000 7f9333fff640 1 -- 172.21.15.84:0/4248361389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f933c15d410 msgr2=0x7f933c0c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:53.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.348+0000 7f9333fff640 1 --2- 172.21.15.84:0/4248361389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f933c15d410 0x7f933c0c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.348+0000 7f9333fff640 1 -- 172.21.15.84:0/4248361389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933c0cd260 msgr2=0x7f933c0c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:53.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.348+0000 7f9333fff640 1 --2- 172.21.15.84:0/4248361389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933c0cd260 0x7f933c0c6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.348+0000 7f9333fff640 1 -- 172.21.15.84:0/4248361389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9334009b30 con 0x7f933c15d030 2023-10-16T10:01:53.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.348+0000 7f933bfff640 1 --2- 172.21.15.84:0/4248361389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f933c15d410 0x7f933c0c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:53.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.348+0000 7f933b7fe640 1 --2- 172.21.15.84:0/4248361389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933c0cd260 0x7f933c0c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:53.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.348+0000 7f9333fff640 1 --2- 172.21.15.84:0/4248361389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f933c15d030 0x7f933c1e6b70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f932400b3c0 tx=0x7f932400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:53.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.349+0000 7f9338ff9640 1 -- 172.21.15.84:0/4248361389 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324011830 con 0x7f933c15d030 2023-10-16T10:01:53.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.349+0000 7f9338ff9640 1 -- 172.21.15.84:0/4248361389 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9324011990 con 0x7f933c15d030 2023-10-16T10:01:53.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.349+0000 7f9341c00640 1 -- 172.21.15.84:0/4248361389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c1e76a0 con 0x7f933c15d030 2023-10-16T10:01:53.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.349+0000 7f9338ff9640 1 -- 172.21.15.84:0/4248361389 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324011d80 con 0x7f933c15d030 2023-10-16T10:01:53.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.349+0000 7f9341c00640 1 -- 172.21.15.84:0/4248361389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933c15d9c0 con 0x7f933c15d030 2023-10-16T10:01:53.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.350+0000 7f9341c00640 1 -- 172.21.15.84:0/4248361389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933c04e160 con 0x7f933c15d030 2023-10-16T10:01:53.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.351+0000 7f9338ff9640 1 -- 172.21.15.84:0/4248361389 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f932401b460 con 0x7f933c15d030 2023-10-16T10:01:53.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.352+0000 7f9338ff9640 1 --2- 172.21.15.84:0/4248361389 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f931c06b050 0x7f931c06d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:53.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.352+0000 7f933b7fe640 1 -- 172.21.15.84:0/4248361389 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f931c06b050 msgr2=0x7f931c06d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4265272960 2023-10-16T10:01:53.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.353+0000 7f9338ff9640 1 -- 172.21.15.84:0/4248361389 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6791+0+0 (secure 0 0 0) 0x7f93240898d0 con 0x7f933c15d030 2023-10-16T10:01:53.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.353+0000 7f933b7fe640 1 --2- 172.21.15.84:0/4248361389 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f931c06b050 0x7f931c06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:53.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.356+0000 7f9338ff9640 1 -- 172.21.15.84:0/4248361389 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f93240542b0 con 0x7f933c15d030 2023-10-16T10:01:53.383 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:53.382+0000 7f4a9d82b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:01:53.445 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:53.443+0000 7fc76d413040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:01:53.495 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:53.493+0000 7f4a9d82b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:01:53.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.554+0000 7f933b7fe640 1 -- 172.21.15.84:0/4248361389 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f931c06b050 msgr2=0x7f931c06d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4265272960 2023-10-16T10:01:53.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.554+0000 7f933b7fe640 1 --2- 172.21.15.84:0/4248361389 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f931c06b050 0x7f931c06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:53.556 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:53.555+0000 7fc76d413040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:01:53.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.564+0000 7f9341c00640 1 -- 172.21.15.84:0/4248361389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f933c0d0a00 con 0x7f933c15d030 2023-10-16T10:01:53.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.565+0000 7f9338ff9640 1 -- 172.21.15.84:0/4248361389 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v49) v1 ==== 80+0+353 (secure 0 0 0) 0x7f93240578d0 con 0x7f933c15d030 2023-10-16T10:01:53.567 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1628587200 2023-10-17T09:50:26.881911+0000 2023-10-16T10:01:53.567 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3968394661 2023-10-17T09:50:26.881911+0000 2023-10-16T10:01:53.567 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1060139177 2023-10-17T09:50:26.881911+0000 2023-10-16T10:01:53.567 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/982275204 2023-10-17T09:50:26.881911+0000 2023-10-16T10:01:53.567 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6833/3010246666 2023-10-17T09:50:26.881911+0000 2023-10-16T10:01:53.568 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6832/3010246666 2023-10-17T09:50:26.881911+0000 2023-10-16T10:01:53.568 INFO:teuthology.orchestra.run.smithi084.stderr:listed 6 entries 2023-10-16T10:01:53.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.567+0000 7f9341c00640 1 -- 172.21.15.84:0/4248361389 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f931c06b050 msgr2=0x7f931c06d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:53.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.567+0000 7f9341c00640 1 --2- 172.21.15.84:0/4248361389 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f931c06b050 0x7f931c06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.567+0000 7f9341c00640 1 -- 172.21.15.84:0/4248361389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f933c15d030 msgr2=0x7f933c1e6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:53.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.567+0000 7f9341c00640 1 --2- 172.21.15.84:0/4248361389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f933c15d030 0x7f933c1e6b70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f932400b3c0 tx=0x7f932400b880 comp rx=0 tx=0).stop 2023-10-16T10:01:53.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.568+0000 7f9341c00640 1 -- 172.21.15.84:0/4248361389 shutdown_connections 2023-10-16T10:01:53.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.568+0000 7f9341c00640 1 --2- 172.21.15.84:0/4248361389 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f931c06b050 0x7f931c06d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.568+0000 7f9341c00640 1 --2- 172.21.15.84:0/4248361389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f933c15d410 0x7f933c0c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.568+0000 7f9341c00640 1 --2- 172.21.15.84:0/4248361389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f933c15d030 0x7f933c1e6b70 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.568+0000 7f9341c00640 1 --2- 172.21.15.84:0/4248361389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933c0cd260 0x7f933c0c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.568+0000 7f9341c00640 1 -- 172.21.15.84:0/4248361389 >> 172.21.15.84:0/4248361389 conn(0x7f933c0cf720 msgr2=0x7f933c0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:53.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.568+0000 7f9341c00640 1 -- 172.21.15.84:0/4248361389 shutdown_connections 2023-10-16T10:01:53.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:53.568+0000 7f9341c00640 1 -- 172.21.15.84:0/4248361389 wait complete. 2023-10-16T10:01:53.586 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T10:01:53.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:53.587 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T10:01:53.652 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:53.651+0000 7f4a9d82b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:01:53.701 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:53.700+0000 7f4a9d82b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:01:53.722 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:53.720+0000 7fc76d413040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:01:53.739 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T10:01:53.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:53.740 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T10:01:53.774 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:53.773+0000 7fc76d413040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:01:53.786 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:01:53.787 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:01:53.826 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:53.824+0000 7f4a9d82b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:01:53.842 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:01:53.843 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:01:53.843 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:01:53.896 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:01:53.897 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:01:53.897 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:01:53.899 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:53.898+0000 7fc76d413040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:01:53.948 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:01:53.949 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:01:54.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.157+0000 7fd73afd5640 1 Processor -- start 2023-10-16T10:01:54.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.157+0000 7fd73afd5640 1 -- start start 2023-10-16T10:01:54.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.159+0000 7fd73afd5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd73415aaa0 0x7fd73415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.159+0000 7fd73afd5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd73415ed60 con 0x7fd73415ae80 2023-10-16T10:01:54.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.159+0000 7fd73afd5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7340707e0 con 0x7fd7340ce390 2023-10-16T10:01:54.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.159+0000 7fd73afd5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd73406b810 con 0x7fd73415aaa0 2023-10-16T10:01:54.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.159+0000 7fd733fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd73415ae80 0x7fd7340cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34342/0 (socket says 172.21.15.84:34342) 2023-10-16T10:01:54.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.159+0000 7fd733fff640 1 -- 172.21.15.84:0/3677700846 learned_addr learned my addr 172.21.15.84:0/3677700846 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:54.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.159+0000 7fd738d71640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd73415aaa0 0x7fd73415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.160+0000 7fd7337fe640 1 -- 172.21.15.84:0/3677700846 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784755835 0 0) 0x7fd73415ed60 con 0x7fd73415ae80 2023-10-16T10:01:54.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.160+0000 7fd738d71640 1 -- 172.21.15.84:0/3677700846 >> v1:172.21.15.157:6789/0 conn(0x7fd7340ce390 legacy=0x7fd734070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:54.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.160+0000 7fd738d71640 1 -- 172.21.15.84:0/3677700846 >> v1:172.21.15.84:6789/0 conn(0x7fd73415ae80 legacy=0x7fd7340cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.160+0000 7fd738d71640 1 -- 172.21.15.84:0/3677700846 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73415ef90 con 0x7fd73415aaa0 2023-10-16T10:01:54.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.161+0000 7fd738d71640 1 --2- 172.21.15.84:0/3677700846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd73415aaa0 0x7fd73415e820 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd7280097a0 tx=0x7fd72802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c072d768e5701399 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:54.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.161+0000 7fd7337fe640 1 -- 172.21.15.84:0/3677700846 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728031830 con 0x7fd73415aaa0 2023-10-16T10:01:54.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.161+0000 7fd7337fe640 1 -- 172.21.15.84:0/3677700846 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd728031990 con 0x7fd73415aaa0 2023-10-16T10:01:54.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.161+0000 7fd7337fe640 1 -- 172.21.15.84:0/3677700846 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728031d20 con 0x7fd73415aaa0 2023-10-16T10:01:54.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.162+0000 7fd73afd5640 1 -- 172.21.15.84:0/3677700846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd73415aaa0 msgr2=0x7fd73415e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.162+0000 7fd73afd5640 1 --2- 172.21.15.84:0/3677700846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd73415aaa0 0x7fd73415e820 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd7280097a0 tx=0x7fd72802b8d0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.162+0000 7fd73afd5640 1 -- 172.21.15.84:0/3677700846 shutdown_connections 2023-10-16T10:01:54.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.162+0000 7fd73afd5640 1 --2- 172.21.15.84:0/3677700846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd73415aaa0 0x7fd73415e820 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.162+0000 7fd73afd5640 1 -- 172.21.15.84:0/3677700846 >> 172.21.15.84:0/3677700846 conn(0x7fd7340cb410 msgr2=0x7fd7340cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:54.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.162+0000 7fd73afd5640 1 -- 172.21.15.84:0/3677700846 shutdown_connections 2023-10-16T10:01:54.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.162+0000 7fd73afd5640 1 -- 172.21.15.84:0/3677700846 wait complete. 2023-10-16T10:01:54.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.164+0000 7fd73afd5640 1 Processor -- start 2023-10-16T10:01:54.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.164+0000 7fd73afd5640 1 -- start start 2023-10-16T10:01:54.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.165+0000 7fd73afd5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7340ce390 0x7fd7340af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.165+0000 7fd73afd5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd73415aaa0 0x7fd7340af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.165+0000 7fd73afd5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd73415ae80 0x7fd7341f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.165+0000 7fd73afd5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7341f1e40 con 0x7fd73415aaa0 2023-10-16T10:01:54.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.165+0000 7fd73afd5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7340c3180 con 0x7fd7340ce390 2023-10-16T10:01:54.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.165+0000 7fd73afd5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7341f17f0 con 0x7fd73415ae80 2023-10-16T10:01:54.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.165+0000 7fd733fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd73415aaa0 0x7fd7340af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.165+0000 7fd733fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd73415aaa0 0x7fd7340af7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38630/0 (socket says 172.21.15.84:38630) 2023-10-16T10:01:54.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.165+0000 7fd733fff640 1 -- 172.21.15.84:0/3032161692 learned_addr learned my addr 172.21.15.84:0/3032161692 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:54.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.165+0000 7fd739572640 1 --2- 172.21.15.84:0/3032161692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd73415ae80 0x7fd7341f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.165+0000 7fd738d71640 1 --2- 172.21.15.84:0/3032161692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7340ce390 0x7fd7340af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.165+0000 7fd733fff640 1 -- 172.21.15.84:0/3032161692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd73415ae80 msgr2=0x7fd7341f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.165+0000 7fd733fff640 1 --2- 172.21.15.84:0/3032161692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd73415ae80 0x7fd7341f22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.165+0000 7fd733fff640 1 -- 172.21.15.84:0/3032161692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7340ce390 msgr2=0x7fd7340af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.165+0000 7fd733fff640 1 --2- 172.21.15.84:0/3032161692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7340ce390 0x7fd7340af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.165+0000 7fd733fff640 1 -- 172.21.15.84:0/3032161692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd728009400 con 0x7fd73415aaa0 2023-10-16T10:01:54.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.165+0000 7fd738d71640 1 --2- 172.21.15.84:0/3032161692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7340ce390 0x7fd7340af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:54.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.166+0000 7fd733fff640 1 --2- 172.21.15.84:0/3032161692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd73415aaa0 0x7fd7340af7c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd72000c410 tx=0x7fd72000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:54.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.166+0000 7fd7317fa640 1 -- 172.21.15.84:0/3032161692 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7200074e0 con 0x7fd73415aaa0 2023-10-16T10:01:54.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.166+0000 7fd7317fa640 1 -- 172.21.15.84:0/3032161692 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd720007660 con 0x7fd73415aaa0 2023-10-16T10:01:54.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.166+0000 7fd73afd5640 1 -- 172.21.15.84:0/3032161692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7341f1ad0 con 0x7fd73415aaa0 2023-10-16T10:01:54.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.166+0000 7fd7317fa640 1 -- 172.21.15.84:0/3032161692 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720004a20 con 0x7fd73415aaa0 2023-10-16T10:01:54.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.166+0000 7fd73afd5640 1 -- 172.21.15.84:0/3032161692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73415dfa0 con 0x7fd73415aaa0 2023-10-16T10:01:54.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.167+0000 7fd73afd5640 1 -- 172.21.15.84:0/3032161692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7341f1fb0 con 0x7fd73415aaa0 2023-10-16T10:01:54.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.168+0000 7fd7317fa640 1 -- 172.21.15.84:0/3032161692 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) v1 ==== 89400+0+0 (secure 0 0 0) 0x7fd720004b80 con 0x7fd73415aaa0 2023-10-16T10:01:54.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.169+0000 7fd7317fa640 1 --2- 172.21.15.84:0/3032161692 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd71806b020 0x7fd71806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.169+0000 7fd7317fa640 1 -- 172.21.15.84:0/3032161692 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6791+0+0 (secure 0 0 0) 0x7fd720089ce0 con 0x7fd73415aaa0 2023-10-16T10:01:54.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.169+0000 7fd738d71640 1 -- 172.21.15.84:0/3032161692 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd71806b020 msgr2=0x7fd71806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4265272960 2023-10-16T10:01:54.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.169+0000 7fd738d71640 1 --2- 172.21.15.84:0/3032161692 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd71806b020 0x7fd71806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:54.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.173+0000 7fd7317fa640 1 -- 172.21.15.84:0/3032161692 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd7200545b0 con 0x7fd73415aaa0 2023-10-16T10:01:54.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.370+0000 7fd738d71640 1 -- 172.21.15.84:0/3032161692 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd71806b020 msgr2=0x7fd71806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4265272960 2023-10-16T10:01:54.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.370+0000 7fd738d71640 1 --2- 172.21.15.84:0/3032161692 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd71806b020 0x7fd71806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:54.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.404+0000 7fd73afd5640 1 -- 172.21.15.84:0/3032161692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd73404e160 con 0x7fd73415aaa0 2023-10-16T10:01:54.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.405+0000 7fd7317fa640 1 -- 172.21.15.84:0/3032161692 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fd720057bd0 con 0x7fd73415aaa0 2023-10-16T10:01:54.406 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:01:54.407 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3662158203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3662158203},{"type":"v1","addr":"172.21.15.157:6835","nonce":3662158203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5091,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2502120537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2502120537},{"type":"v1","addr":"172.21.15.84:6836","nonce":2502120537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2715651376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2715651376},{"type":"v1","addr":"172.21.15.84:6837","nonce":2715651376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:50:12.380180+0000","modified":"2023-10-16T09:50:15.438597+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6833/62459899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":62459899},{"type":"v1","addr":"172.21.15.157:6833","nonce":62459899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T10:01:54.407 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 17 2023-10-16T10:01:54.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.408+0000 7fd73afd5640 1 -- 172.21.15.84:0/3032161692 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd71806b020 msgr2=0x7fd71806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:54.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.408+0000 7fd73afd5640 1 --2- 172.21.15.84:0/3032161692 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd71806b020 0x7fd71806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.408+0000 7fd73afd5640 1 -- 172.21.15.84:0/3032161692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd73415aaa0 msgr2=0x7fd7340af7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.408+0000 7fd73afd5640 1 --2- 172.21.15.84:0/3032161692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd73415aaa0 0x7fd7340af7c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd72000c410 tx=0x7fd72000c8d0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.408+0000 7fd73afd5640 1 -- 172.21.15.84:0/3032161692 shutdown_connections 2023-10-16T10:01:54.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.408+0000 7fd73afd5640 1 --2- 172.21.15.84:0/3032161692 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd71806b020 0x7fd71806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.408+0000 7fd73afd5640 1 --2- 172.21.15.84:0/3032161692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd73415ae80 0x7fd7341f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.408+0000 7fd73afd5640 1 --2- 172.21.15.84:0/3032161692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd73415aaa0 0x7fd7340af7c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.408+0000 7fd73afd5640 1 --2- 172.21.15.84:0/3032161692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7340ce390 0x7fd7340af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.408+0000 7fd73afd5640 1 -- 172.21.15.84:0/3032161692 >> 172.21.15.84:0/3032161692 conn(0x7fd7340cb410 msgr2=0x7fd73406f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:54.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.408+0000 7fd73afd5640 1 -- 172.21.15.84:0/3032161692 shutdown_connections 2023-10-16T10:01:54.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.408+0000 7fd73afd5640 1 -- 172.21.15.84:0/3032161692 wait complete. 2023-10-16T10:01:54.428 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:01:54.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.620+0000 7f7da8a13640 1 Processor -- start 2023-10-16T10:01:54.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.620+0000 7f7da8a13640 1 -- start start 2023-10-16T10:01:54.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.621+0000 7f7da8a13640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7da40c9060 0x7f7da406f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.621+0000 7f7da8a13640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da40c9440 0x7f7da406f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.621+0000 7f7da8a13640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da40ca560 con 0x7f7da406c9e0 2023-10-16T10:01:54.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.621+0000 7f7da8a13640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da40c9950 con 0x7f7da40c9060 2023-10-16T10:01:54.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.621+0000 7f7da8a13640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da40c9c30 con 0x7f7da40c9440 2023-10-16T10:01:54.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.622+0000 7f7d9bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7da406c9e0 0x7f7da4164d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34352/0 (socket says 172.21.15.84:34352) 2023-10-16T10:01:54.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.622+0000 7f7d9bfff640 1 -- 172.21.15.84:0/242739313 learned_addr learned my addr 172.21.15.84:0/242739313 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:54.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.622+0000 7f7da3fff640 1 --2- 172.21.15.84:0/242739313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da40c9440 0x7f7da406f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.622+0000 7f7da37fe640 1 --2- 172.21.15.84:0/242739313 >> v2:172.21.15.157:3300/0 conn(0x7f7da40c9060 0x7f7da406f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.622+0000 7f7da2ffd640 1 -- 172.21.15.84:0/242739313 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160220600 0 0) 0x7f7da40ca560 con 0x7f7da406c9e0 2023-10-16T10:01:54.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.622+0000 7f7da2ffd640 1 -- 172.21.15.84:0/242739313 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d90003520 con 0x7f7da406c9e0 2023-10-16T10:01:54.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.622+0000 7f7da3fff640 1 -- 172.21.15.84:0/242739313 >> v2:172.21.15.157:3300/0 conn(0x7f7da40c9060 msgr2=0x7f7da406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:54.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.622+0000 7f7da3fff640 1 --2- 172.21.15.84:0/242739313 >> v2:172.21.15.157:3300/0 conn(0x7f7da40c9060 0x7f7da406f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.622+0000 7f7da3fff640 1 -- 172.21.15.84:0/242739313 >> v1:172.21.15.84:6789/0 conn(0x7f7da406c9e0 legacy=0x7f7da4164d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.622+0000 7f7da3fff640 1 -- 172.21.15.84:0/242739313 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da40c9f10 con 0x7f7da40c9440 2023-10-16T10:01:54.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.622+0000 7f7da37fe640 1 --2- 172.21.15.84:0/242739313 >> v2:172.21.15.157:3300/0 conn(0x7f7da40c9060 0x7f7da406f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:01:54.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.623+0000 7f7da3fff640 1 --2- 172.21.15.84:0/242739313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da40c9440 0x7f7da406f980 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009d90 tx=0x7f7d9c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a0cfbb52c910c87 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:54.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.624+0000 7f7da2ffd640 1 -- 172.21.15.84:0/242739313 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c02cc50 con 0x7f7da40c9440 2023-10-16T10:01:54.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.624+0000 7f7da2ffd640 1 -- 172.21.15.84:0/242739313 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c02cdb0 con 0x7f7da40c9440 2023-10-16T10:01:54.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.624+0000 7f7da2ffd640 1 -- 172.21.15.84:0/242739313 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c034640 con 0x7f7da40c9440 2023-10-16T10:01:54.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.624+0000 7f7da8a13640 1 -- 172.21.15.84:0/242739313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da40c9440 msgr2=0x7f7da406f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.624+0000 7f7da8a13640 1 --2- 172.21.15.84:0/242739313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da40c9440 0x7f7da406f980 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009d90 tx=0x7f7d9c02bea0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.624+0000 7f7da8a13640 1 -- 172.21.15.84:0/242739313 shutdown_connections 2023-10-16T10:01:54.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.624+0000 7f7da8a13640 1 --2- 172.21.15.84:0/242739313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da40c9440 0x7f7da406f980 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.624+0000 7f7da8a13640 1 --2- 172.21.15.84:0/242739313 >> v2:172.21.15.157:3300/0 conn(0x7f7da40c9060 0x7f7da406f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.624+0000 7f7da8a13640 1 -- 172.21.15.84:0/242739313 >> 172.21.15.84:0/242739313 conn(0x7f7da40ce6a0 msgr2=0x7f7da40ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:54.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.625+0000 7f7da8a13640 1 -- 172.21.15.84:0/242739313 shutdown_connections 2023-10-16T10:01:54.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.625+0000 7f7da8a13640 1 -- 172.21.15.84:0/242739313 wait complete. 2023-10-16T10:01:54.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.626+0000 7f7da8a13640 1 Processor -- start 2023-10-16T10:01:54.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.626+0000 7f7da8a13640 1 -- start start 2023-10-16T10:01:54.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.627+0000 7f7da8a13640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da406c9e0 0x7f7da4156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.627+0000 7f7da8a13640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da40c9060 0x7f7da40c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.627+0000 7f7da8a13640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7da40c9440 0x7f7da40c4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.627+0000 7f7da8a13640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da41f98a0 con 0x7f7da40c9440 2023-10-16T10:01:54.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.627+0000 7f7da8a13640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da41621e0 con 0x7f7da40c9060 2023-10-16T10:01:54.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.627+0000 7f7da8a13640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da41624c0 con 0x7f7da406c9e0 2023-10-16T10:01:54.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.628+0000 7f7da3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7da40c9440 0x7f7da40c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.628+0000 7f7da3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7da40c9440 0x7f7da40c4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38644/0 (socket says 172.21.15.84:38644) 2023-10-16T10:01:54.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.628+0000 7f7da3fff640 1 -- 172.21.15.84:0/2786478952 learned_addr learned my addr 172.21.15.84:0/2786478952 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:54.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.628+0000 7f7da37fe640 1 --2- 172.21.15.84:0/2786478952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da406c9e0 0x7f7da4156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.628+0000 7f7d9bfff640 1 --2- 172.21.15.84:0/2786478952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da40c9060 0x7f7da40c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.628+0000 7f7da3fff640 1 -- 172.21.15.84:0/2786478952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da406c9e0 msgr2=0x7f7da4156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.628+0000 7f7da3fff640 1 --2- 172.21.15.84:0/2786478952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da406c9e0 0x7f7da4156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.628+0000 7f7da3fff640 1 -- 172.21.15.84:0/2786478952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da40c9060 msgr2=0x7f7da40c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.628+0000 7f7da3fff640 1 --2- 172.21.15.84:0/2786478952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da40c9060 0x7f7da40c6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.628+0000 7f7da3fff640 1 -- 172.21.15.84:0/2786478952 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c009a70 con 0x7f7da40c9440 2023-10-16T10:01:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.628+0000 7f7d9bfff640 1 --2- 172.21.15.84:0/2786478952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da40c9060 0x7f7da40c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.629+0000 7f7da3fff640 1 --2- 172.21.15.84:0/2786478952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7da40c9440 0x7f7da40c4260 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009eb0 tx=0x7f7d9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:54.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.629+0000 7f7da0ff9640 1 -- 172.21.15.84:0/2786478952 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c03e610 con 0x7f7da40c9440 2023-10-16T10:01:54.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.629+0000 7f7da0ff9640 1 -- 172.21.15.84:0/2786478952 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c03e770 con 0x7f7da40c9440 2023-10-16T10:01:54.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.629+0000 7f7da0ff9640 1 -- 172.21.15.84:0/2786478952 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c03eb60 con 0x7f7da40c9440 2023-10-16T10:01:54.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.629+0000 7f7da8a13640 1 -- 172.21.15.84:0/2786478952 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da41627a0 con 0x7f7da40c9440 2023-10-16T10:01:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.629+0000 7f7da8a13640 1 -- 172.21.15.84:0/2786478952 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da4162d60 con 0x7f7da40c9440 2023-10-16T10:01:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.630+0000 7f7da8a13640 1 -- 172.21.15.84:0/2786478952 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da404e9b0 con 0x7f7da40c9440 2023-10-16T10:01:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.631+0000 7f7da0ff9640 1 -- 172.21.15.84:0/2786478952 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f7d9c03b050 con 0x7f7da40c9440 2023-10-16T10:01:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.632+0000 7f7da0ff9640 1 --2- 172.21.15.84:0/2786478952 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f7d8406b020 0x7f7d8406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.632+0000 7f7da0ff9640 1 -- 172.21.15.84:0/2786478952 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6791+0+0 (secure 0 0 0) 0x7f7d9c0abed0 con 0x7f7da40c9440 2023-10-16T10:01:54.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.632+0000 7f7da37fe640 1 -- 172.21.15.84:0/2786478952 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f7d8406b020 msgr2=0x7f7d8406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4265272960 2023-10-16T10:01:54.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.632+0000 7f7da37fe640 1 --2- 172.21.15.84:0/2786478952 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f7d8406b020 0x7f7d8406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:54.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.636+0000 7f7da0ff9640 1 -- 172.21.15.84:0/2786478952 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7d9c076970 con 0x7f7da40c9440 2023-10-16T10:01:54.662 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:54.660+0000 7f4a9d82b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:01:54.722 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:54.720+0000 7fc76d413040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:01:54.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.828+0000 7f7da8a13640 1 -- 172.21.15.84:0/2786478952 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7da40b4620 con 0x7f7da40c9440 2023-10-16T10:01:54.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.830+0000 7f7da0ff9640 1 -- 172.21.15.84:0/2786478952 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18190 (secure 0 0 0) 0x7f7d9c039090 con 0x7f7da40c9440 2023-10-16T10:01:54.832 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:01:54.832 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":49,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:01:52.823862+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:50:10.297512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-16T09:50:11.394493+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":48,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1628587200":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/3968394661":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/1060139177":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/982275204":"2023-10-17T09:50:26.881911+0000","172.21.15.84:6833/3010246666":"2023-10-17T09:50:26.881911+0000","172.21.15.84:6832/3010246666":"2023-10-17T09:50:26.881911+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":7,"length":1},{"begin":11,"length":1},{"begin":14,"length":1},{"begin":20,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":48,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":72,"length":1},{"begin":76,"length":1},{"begin":84,"length":2},{"begin":92,"length":1},{"begin":100,"length":1},{"begin":103,"length":1},{"begin":200,"length":21}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":3,"length":1},{"begin":5,"length":2},{"begin":9,"length":1},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":21,"length":3},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":33,"length":3},{"begin":37,"length":1},{"begin":44,"length":2},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":56,"length":1},{"begin":59,"length":5},{"begin":68,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":77,"length":2},{"begin":82,"length":1},{"begin":86,"length":3},{"begin":94,"length":3},{"begin":98,"length":1},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":109,"length":3},{"begin":155,"length":45}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:01:54.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.832+0000 7f7da8a13640 1 -- 172.21.15.84:0/2786478952 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f7d8406b020 msgr2=0x7f7d8406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:54.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.832+0000 7f7da8a13640 1 --2- 172.21.15.84:0/2786478952 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f7d8406b020 0x7f7d8406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.832+0000 7f7da8a13640 1 -- 172.21.15.84:0/2786478952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7da40c9440 msgr2=0x7f7da40c4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.832+0000 7f7da8a13640 1 --2- 172.21.15.84:0/2786478952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7da40c9440 0x7f7da40c4260 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009eb0 tx=0x7f7d9c004a40 comp rx=0 tx=0).stop 2023-10-16T10:01:54.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.832+0000 7f7da8a13640 1 -- 172.21.15.84:0/2786478952 shutdown_connections 2023-10-16T10:01:54.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.832+0000 7f7da8a13640 1 --2- 172.21.15.84:0/2786478952 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7f7d8406b020 0x7f7d8406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.832+0000 7f7da8a13640 1 --2- 172.21.15.84:0/2786478952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7da40c9440 0x7f7da40c4260 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.832+0000 7f7da8a13640 1 --2- 172.21.15.84:0/2786478952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da40c9060 0x7f7da40c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.832+0000 7f7da8a13640 1 --2- 172.21.15.84:0/2786478952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da406c9e0 0x7f7da4156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.832+0000 7f7da8a13640 1 -- 172.21.15.84:0/2786478952 >> 172.21.15.84:0/2786478952 conn(0x7f7da40ce6a0 msgr2=0x7f7da406a750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:54.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.833+0000 7f7da8a13640 1 -- 172.21.15.84:0/2786478952 shutdown_connections 2023-10-16T10:01:54.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:54.833+0000 7f7da8a13640 1 -- 172.21.15.84:0/2786478952 wait complete. 2023-10-16T10:01:54.851 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:50:10.297512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:50:11.394493+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 220, 'snap_epoch': 48, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:01:54.852 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:01:54.961 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:54.959+0000 7f4a9d82b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:01:55.018 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:55.017+0000 7f4a9d82b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:01:55.036 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:55.035+0000 7fc76d413040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:01:55.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.046+0000 7fd588a94640 1 Processor -- start 2023-10-16T10:01:55.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.046+0000 7fd588a94640 1 -- start start 2023-10-16T10:01:55.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.047+0000 7fd588a94640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd584159ab0 0x7fd5840ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.047+0000 7fd588a94640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd58406c140 0x7fd58406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.047+0000 7fd588a94640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd584070360 con 0x7fd584159e90 2023-10-16T10:01:55.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.048+0000 7fd588a94640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd58406cb10 con 0x7fd584159ab0 2023-10-16T10:01:55.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.048+0000 7fd588a94640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58415d890 con 0x7fd58406c140 2023-10-16T10:01:55.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.048+0000 7fd57bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd584159e90 0x7fd58406fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34362/0 (socket says 172.21.15.84:34362) 2023-10-16T10:01:55.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.048+0000 7fd57bfff640 1 -- 172.21.15.84:0/642286238 learned_addr learned my addr 172.21.15.84:0/642286238 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:55.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.048+0000 7fd58259c640 1 --2- 172.21.15.84:0/642286238 >> v2:172.21.15.84:3300/0 conn(0x7fd584159ab0 0x7fd5840ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.048+0000 7fd582d9d640 1 --2- 172.21.15.84:0/642286238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd58406c140 0x7fd58406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.048+0000 7fd58259c640 1 -- 172.21.15.84:0/642286238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd58406c140 msgr2=0x7fd58406c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:55.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.048+0000 7fd58259c640 1 --2- 172.21.15.84:0/642286238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd58406c140 0x7fd58406c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.048+0000 7fd58259c640 1 -- 172.21.15.84:0/642286238 >> v1:172.21.15.84:6789/0 conn(0x7fd584159e90 legacy=0x7fd58406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.048+0000 7fd58259c640 1 -- 172.21.15.84:0/642286238 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5840cd1f0 con 0x7fd584159ab0 2023-10-16T10:01:55.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.049+0000 7fd58259c640 1 --2- 172.21.15.84:0/642286238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd584159ab0 0x7fd5840ce370 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd5740097a0 tx=0x7fd57402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1af1b307797c6b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:55.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.050+0000 7fd581d9b640 1 -- 172.21.15.84:0/642286238 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57402f050 con 0x7fd584159ab0 2023-10-16T10:01:55.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.050+0000 7fd581d9b640 1 -- 172.21.15.84:0/642286238 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd574031700 con 0x7fd584159ab0 2023-10-16T10:01:55.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.050+0000 7fd581d9b640 1 -- 172.21.15.84:0/642286238 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574031a90 con 0x7fd584159ab0 2023-10-16T10:01:55.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.050+0000 7fd588a94640 1 -- 172.21.15.84:0/642286238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd584159ab0 msgr2=0x7fd5840ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.050+0000 7fd588a94640 1 --2- 172.21.15.84:0/642286238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd584159ab0 0x7fd5840ce370 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd5740097a0 tx=0x7fd57402b7a0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.050+0000 7fd588a94640 1 -- 172.21.15.84:0/642286238 shutdown_connections 2023-10-16T10:01:55.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.050+0000 7fd588a94640 1 --2- 172.21.15.84:0/642286238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd58406c140 0x7fd58406c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.050+0000 7fd588a94640 1 --2- 172.21.15.84:0/642286238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd584159ab0 0x7fd5840ce370 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.050+0000 7fd588a94640 1 -- 172.21.15.84:0/642286238 >> 172.21.15.84:0/642286238 conn(0x7fd5840ce9b0 msgr2=0x7fd5840cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:55.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.050+0000 7fd588a94640 1 -- 172.21.15.84:0/642286238 shutdown_connections 2023-10-16T10:01:55.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.050+0000 7fd588a94640 1 -- 172.21.15.84:0/642286238 wait complete. 2023-10-16T10:01:55.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.052+0000 7fd588a94640 1 Processor -- start 2023-10-16T10:01:55.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.052+0000 7fd588a94640 1 -- start start 2023-10-16T10:01:55.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.053+0000 7fd588a94640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd58406c140 0x7fd58415a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.053+0000 7fd588a94640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd584159ab0 0x7fd58414ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.053+0000 7fd588a94640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd584159e90 0x7fd58415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.053+0000 7fd588a94640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58415d320 con 0x7fd58406c140 2023-10-16T10:01:55.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.053+0000 7fd588a94640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5840c4810 con 0x7fd584159ab0 2023-10-16T10:01:55.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.053+0000 7fd588a94640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58414e410 con 0x7fd584159e90 2023-10-16T10:01:55.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.053+0000 7fd58259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd58406c140 0x7fd58415a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.053+0000 7fd58259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd58406c140 0x7fd58415a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38650/0 (socket says 172.21.15.84:38650) 2023-10-16T10:01:55.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.053+0000 7fd58259c640 1 -- 172.21.15.84:0/535523533 learned_addr learned my addr 172.21.15.84:0/535523533 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:55.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.054+0000 7fd58259c640 1 -- 172.21.15.84:0/535523533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd584159e90 msgr2=0x7fd58415b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.054+0000 7fd58259c640 1 --2- 172.21.15.84:0/535523533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd584159e90 0x7fd58415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.054+0000 7fd58259c640 1 -- 172.21.15.84:0/535523533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd584159ab0 msgr2=0x7fd58414ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.054+0000 7fd58259c640 1 --2- 172.21.15.84:0/535523533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd584159ab0 0x7fd58414ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.054+0000 7fd58259c640 1 -- 172.21.15.84:0/535523533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd574009400 con 0x7fd58406c140 2023-10-16T10:01:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.054+0000 7fd58259c640 1 --2- 172.21.15.84:0/535523533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd58406c140 0x7fd58415a630 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd574036040 tx=0x7fd5740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.054+0000 7fd57affd640 1 -- 172.21.15.84:0/535523533 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574033a30 con 0x7fd58406c140 2023-10-16T10:01:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.054+0000 7fd57affd640 1 -- 172.21.15.84:0/535523533 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd574033bb0 con 0x7fd58406c140 2023-10-16T10:01:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.054+0000 7fd57affd640 1 -- 172.21.15.84:0/535523533 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57403c400 con 0x7fd58406c140 2023-10-16T10:01:55.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.054+0000 7fd588a94640 1 -- 172.21.15.84:0/535523533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5840c5610 con 0x7fd58406c140 2023-10-16T10:01:55.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.054+0000 7fd588a94640 1 -- 172.21.15.84:0/535523533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5840c5330 con 0x7fd58406c140 2023-10-16T10:01:55.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.056+0000 7fd588a94640 1 -- 172.21.15.84:0/535523533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58404e160 con 0x7fd58406c140 2023-10-16T10:01:55.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.056+0000 7fd57affd640 1 -- 172.21.15.84:0/535523533 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) v1 ==== 89400+0+0 (secure 0 0 0) 0x7fd57403a040 con 0x7fd58406c140 2023-10-16T10:01:55.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.057+0000 7fd57affd640 1 --2- 172.21.15.84:0/535523533 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd56406b020 0x7fd56406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.057+0000 7fd57affd640 1 -- 172.21.15.84:0/535523533 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6375+0+0 (secure 0 0 0) 0x7fd5740aaf60 con 0x7fd58406c140 2023-10-16T10:01:55.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.057+0000 7fd57bfff640 1 -- 172.21.15.84:0/535523533 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd56406b020 msgr2=0x7fd56406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4265272960 2023-10-16T10:01:55.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.057+0000 7fd57bfff640 1 --2- 172.21.15.84:0/535523533 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd56406b020 0x7fd56406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:55.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.062+0000 7fd57affd640 1 -- 172.21.15.84:0/535523533 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd574075b00 con 0x7fd58406c140 2023-10-16T10:01:55.097 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:55.096+0000 7fc76d413040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:01:55.142 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:01:55.140+0000 7f4a9d82b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:01:55.221 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:55.220+0000 7fc76d413040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:01:55.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.241+0000 7fd57affd640 1 -- 172.21.15.84:0/535523533 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 18) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fd574075ed0 con 0x7fd58406c140 2023-10-16T10:01:55.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.241+0000 7fd57affd640 1 -- 172.21.15.84:0/535523533 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd56406b020 msgr2=0x7fd56406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:55.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.241+0000 7fd57affd640 1 --2- 172.21.15.84:0/535523533 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd56406b020 0x7fd56406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.279+0000 7fd588a94640 1 -- 172.21.15.84:0/535523533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd5840ccb60 con 0x7fd58406c140 2023-10-16T10:01:55.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.279+0000 7fd57affd640 1 -- 172.21.15.84:0/535523533 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd57407d3a0 con 0x7fd58406c140 2023-10-16T10:01:55.281 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:01:55.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.282+0000 7fd588a94640 1 -- 172.21.15.84:0/535523533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd58406c140 msgr2=0x7fd58415a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.282+0000 7fd588a94640 1 --2- 172.21.15.84:0/535523533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd58406c140 0x7fd58415a630 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd574036040 tx=0x7fd5740046c0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.282+0000 7fd588a94640 1 -- 172.21.15.84:0/535523533 shutdown_connections 2023-10-16T10:01:55.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.282+0000 7fd588a94640 1 --2- 172.21.15.84:0/535523533 >> [v2:172.21.15.84:6832/4265272960,v1:172.21.15.84:6833/4265272960] conn(0x7fd56406b020 0x7fd56406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.282+0000 7fd588a94640 1 --2- 172.21.15.84:0/535523533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd584159e90 0x7fd58415b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.282+0000 7fd588a94640 1 --2- 172.21.15.84:0/535523533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd584159ab0 0x7fd58414ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.282+0000 7fd588a94640 1 --2- 172.21.15.84:0/535523533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd58406c140 0x7fd58415a630 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.282+0000 7fd588a94640 1 -- 172.21.15.84:0/535523533 >> 172.21.15.84:0/535523533 conn(0x7fd5840ce9b0 msgr2=0x7fd5840b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:55.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.282+0000 7fd588a94640 1 -- 172.21.15.84:0/535523533 shutdown_connections 2023-10-16T10:01:55.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.282+0000 7fd588a94640 1 -- 172.21.15.84:0/535523533 wait complete. 2023-10-16T10:01:55.302 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:01:55.304 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:55.302+0000 7fc6f2448640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:55.304 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:55.302+0000 7fc6f2448640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:55.304 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:55.302+0000 7fc6f2448640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:55.304 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:55.302+0000 7fc6f2448640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:55.304 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:55.302+0000 7fc6f2448640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:55.305 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:55.303+0000 7fc6eec41640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:55.306 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:55.303+0000 7fc6eec41640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:55.306 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:55.303+0000 7fc6eec41640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:55.306 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:55.303+0000 7fc6eec41640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:55.306 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:01:55.303+0000 7fc6eec41640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:55.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.485+0000 7f1066b67640 1 Processor -- start 2023-10-16T10:01:55.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.485+0000 7f1066b67640 1 -- start start 2023-10-16T10:01:55.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.486+0000 7f1066b67640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f10600ac890 0x7f10600accc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.486+0000 7f1066b67640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1060160a90 con 0x7f106015dde0 2023-10-16T10:01:55.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.486+0000 7f1066b67640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10600ad200 con 0x7f106015c9f0 2023-10-16T10:01:55.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.486+0000 7f1066b67640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10600cdec0 con 0x7f10600ac890 2023-10-16T10:01:55.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.487+0000 7f1065364640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f106015dde0 0x7f10600709e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34374/0 (socket says 172.21.15.84:34374) 2023-10-16T10:01:55.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.487+0000 7f1065364640 1 -- 172.21.15.84:0/254136476 learned_addr learned my addr 172.21.15.84:0/254136476 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:55.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.487+0000 7f1066366640 1 --2- 172.21.15.84:0/254136476 >> v2:172.21.15.157:3300/0 conn(0x7f10600ac890 0x7f10600accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.487+0000 7f1064b63640 1 -- 172.21.15.84:0/254136476 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 168390201 0 0) 0x7f1060160a90 con 0x7f106015dde0 2023-10-16T10:01:55.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.487+0000 7f1064b63640 1 -- 172.21.15.84:0/254136476 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f104c003520 con 0x7f106015dde0 2023-10-16T10:01:55.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1064b63640 1 -- 172.21.15.84:0/254136476 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1837732182 0 0) 0x7f104c003520 con 0x7f106015dde0 2023-10-16T10:01:55.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1064b63640 1 -- 172.21.15.84:0/254136476 >> v2:172.21.15.157:3300/0 conn(0x7f10600ac890 msgr2=0x7f10600accc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:55.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1064b63640 1 --2- 172.21.15.84:0/254136476 >> v2:172.21.15.157:3300/0 conn(0x7f10600ac890 0x7f10600accc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1064b63640 1 -- 172.21.15.84:0/254136476 >> v1:172.21.15.157:6789/0 conn(0x7f106015c9f0 legacy=0x7f10600c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1064b63640 1 -- 172.21.15.84:0/254136476 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10600cd900 con 0x7f106015dde0 2023-10-16T10:01:55.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1064b63640 1 -- 172.21.15.84:0/254136476 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f105c001670 con 0x7f106015dde0 2023-10-16T10:01:55.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1064b63640 1 -- 172.21.15.84:0/254136476 >> v1:172.21.15.84:6789/0 conn(0x7f106015dde0 legacy=0x7f10600709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1064b63640 1 --2- 172.21.15.84:0/254136476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f104c003970 0x7f104c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1064b63640 1 --2- 172.21.15.84:0/254136476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104c0047c0 0x7f104c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1064b63640 1 --2- 172.21.15.84:0/254136476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f104c005180 0x7f104c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1064b63640 1 -- 172.21.15.84:0/254136476 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c00c050 con 0x7f104c005180 2023-10-16T10:01:55.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1064b63640 1 -- 172.21.15.84:0/254136476 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c00c190 con 0x7f104c0047c0 2023-10-16T10:01:55.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1064b63640 1 -- 172.21.15.84:0/254136476 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104c00c350 con 0x7f104c003970 2023-10-16T10:01:55.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1066366640 1 --2- 172.21.15.84:0/254136476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f104c005180 0x7f104c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1066366640 1 -- 172.21.15.84:0/254136476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f104c003970 msgr2=0x7f104c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:55.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1066366640 1 --2- 172.21.15.84:0/254136476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f104c003970 0x7f104c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1065364640 1 --2- 172.21.15.84:0/254136476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104c0047c0 0x7f104c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1066366640 1 -- 172.21.15.84:0/254136476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104c0047c0 msgr2=0x7f104c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:55.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.488+0000 7f1066366640 1 --2- 172.21.15.84:0/254136476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104c0047c0 0x7f104c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.489+0000 7f1066366640 1 -- 172.21.15.84:0/254136476 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c003520 con 0x7f104c005180 2023-10-16T10:01:55.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.489+0000 7f1066366640 1 -- 172.21.15.84:0/254136476 reap_dead start 2023-10-16T10:01:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.489+0000 7f1066366640 1 --2- 172.21.15.84:0/254136476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f104c005180 0x7f104c00b980 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1058009fc0 tx=0x7f104c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2724062cbfcadf9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.490+0000 7f1064b63640 1 -- 172.21.15.84:0/254136476 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105802cc60 con 0x7f104c005180 2023-10-16T10:01:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.490+0000 7f1064b63640 1 -- 172.21.15.84:0/254136476 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105800f040 con 0x7f104c005180 2023-10-16T10:01:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.490+0000 7f1064b63640 1 -- 172.21.15.84:0/254136476 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058003990 con 0x7f104c005180 2023-10-16T10:01:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.490+0000 7f1066b67640 1 -- 172.21.15.84:0/254136476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f104c005180 msgr2=0x7f104c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.490+0000 7f1066b67640 1 --2- 172.21.15.84:0/254136476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f104c005180 0x7f104c00b980 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1058009fc0 tx=0x7f104c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.490+0000 7f1066b67640 1 -- 172.21.15.84:0/254136476 shutdown_connections 2023-10-16T10:01:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.490+0000 7f1066b67640 1 --2- 172.21.15.84:0/254136476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f104c005180 0x7f104c00b980 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.490+0000 7f1066b67640 1 -- 172.21.15.84:0/254136476 >> 172.21.15.84:0/254136476 conn(0x7f10600ce6a0 msgr2=0x7f10600ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.490+0000 7f1066b67640 1 -- 172.21.15.84:0/254136476 shutdown_connections 2023-10-16T10:01:55.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.490+0000 7f1066b67640 1 -- 172.21.15.84:0/254136476 wait complete. 2023-10-16T10:01:55.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.492+0000 7f1066b67640 1 Processor -- start 2023-10-16T10:01:55.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.492+0000 7f1066b67640 1 -- start start 2023-10-16T10:01:55.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.493+0000 7f1066b67640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104c005180 0x7f1060153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.493+0000 7f1066b67640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106015b040 0x7f106015b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.493+0000 7f1066b67640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f106006b710 0x7f106006bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.493+0000 7f1066b67640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106015ba00 con 0x7f106006b710 2023-10-16T10:01:55.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.493+0000 7f1066b67640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10600c97a0 con 0x7f104c005180 2023-10-16T10:01:55.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.493+0000 7f1066b67640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1060153ad0 con 0x7f106015b040 2023-10-16T10:01:55.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.493+0000 7f1066366640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f106006b710 0x7f106006bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.493+0000 7f1066366640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f106006b710 0x7f106006bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38776/0 (socket says 172.21.15.84:38776) 2023-10-16T10:01:55.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.493+0000 7f1066366640 1 -- 172.21.15.84:0/158666849 learned_addr learned my addr 172.21.15.84:0/158666849 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:55.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.494+0000 7f1066366640 1 -- 172.21.15.84:0/158666849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106015b040 msgr2=0x7f106015b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.494+0000 7f1065364640 1 --2- 172.21.15.84:0/158666849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106015b040 0x7f106015b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.494+0000 7f1065b65640 1 --2- 172.21.15.84:0/158666849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104c005180 0x7f1060153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.494+0000 7f1066366640 1 --2- 172.21.15.84:0/158666849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106015b040 0x7f106015b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.494+0000 7f1066366640 1 -- 172.21.15.84:0/158666849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104c005180 msgr2=0x7f1060153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.494+0000 7f1066366640 1 --2- 172.21.15.84:0/158666849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104c005180 0x7f1060153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.494+0000 7f1066366640 1 -- 172.21.15.84:0/158666849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106006c480 con 0x7f106006b710 2023-10-16T10:01:55.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.494+0000 7f1066366640 1 --2- 172.21.15.84:0/158666849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f106006b710 0x7f106006bb90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f105800e200 tx=0x7f1058009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:55.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.494+0000 7f10527fc640 1 -- 172.21.15.84:0/158666849 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058035df0 con 0x7f106006b710 2023-10-16T10:01:55.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.494+0000 7f10527fc640 1 -- 172.21.15.84:0/158666849 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1058003990 con 0x7f106006b710 2023-10-16T10:01:55.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.494+0000 7f10527fc640 1 -- 172.21.15.84:0/158666849 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058003d40 con 0x7f106006b710 2023-10-16T10:01:55.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.494+0000 7f1066b67640 1 -- 172.21.15.84:0/158666849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10600c8e70 con 0x7f106006b710 2023-10-16T10:01:55.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.495+0000 7f1066b67640 1 -- 172.21.15.84:0/158666849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10600c9430 con 0x7f106006b710 2023-10-16T10:01:55.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.496+0000 7f1066b67640 1 -- 172.21.15.84:0/158666849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106004e9b0 con 0x7f106006b710 2023-10-16T10:01:55.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.497+0000 7f10527fc640 1 -- 172.21.15.84:0/158666849 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f1058004030 con 0x7f106006b710 2023-10-16T10:01:55.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.498+0000 7f10527fc640 1 -- 172.21.15.84:0/158666849 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f10580ac1a0 con 0x7f106006b710 2023-10-16T10:01:55.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.502+0000 7f10527fc640 1 -- 172.21.15.84:0/158666849 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f10580773d0 con 0x7f106006b710 2023-10-16T10:01:55.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.680+0000 7f1066b67640 1 -- 172.21.15.84:0/158666849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f10600ad860 con 0x7f106006b710 2023-10-16T10:01:55.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.680+0000 7f10527fc640 1 -- 172.21.15.84:0/158666849 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7f105807a9f0 con 0x7f106006b710 2023-10-16T10:01:55.682 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T10:01:55.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.683+0000 7f1066b67640 1 -- 172.21.15.84:0/158666849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f106006b710 msgr2=0x7f106006bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.683+0000 7f1066b67640 1 --2- 172.21.15.84:0/158666849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f106006b710 0x7f106006bb90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f105800e200 tx=0x7f1058009750 comp rx=0 tx=0).stop 2023-10-16T10:01:55.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.683+0000 7f1066b67640 1 -- 172.21.15.84:0/158666849 shutdown_connections 2023-10-16T10:01:55.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.683+0000 7f1066b67640 1 --2- 172.21.15.84:0/158666849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f106006b710 0x7f106006bb90 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.683+0000 7f1066b67640 1 --2- 172.21.15.84:0/158666849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106015b040 0x7f106015b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.683+0000 7f1066b67640 1 --2- 172.21.15.84:0/158666849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f104c005180 0x7f1060153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.683+0000 7f1066b67640 1 -- 172.21.15.84:0/158666849 >> 172.21.15.84:0/158666849 conn(0x7f10600ce6a0 msgr2=0x7f106015a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:55.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.683+0000 7f1066b67640 1 -- 172.21.15.84:0/158666849 shutdown_connections 2023-10-16T10:01:55.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.683+0000 7f1066b67640 1 -- 172.21.15.84:0/158666849 wait complete. 2023-10-16T10:01:55.702 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:01:55.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.909+0000 7f6a437a9640 1 Processor -- start 2023-10-16T10:01:55.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.909+0000 7f6a437a9640 1 -- start start 2023-10-16T10:01:55.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.910+0000 7f6a437a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6a3c159ab0 0x7f6a3c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.911+0000 7f6a437a9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3c070360 con 0x7f6a3c159e90 2023-10-16T10:01:55.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.911+0000 7f6a437a9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3c0cd1f0 con 0x7f6a3c06c140 2023-10-16T10:01:55.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.911+0000 7f6a437a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a3c15d890 con 0x7f6a3c159ab0 2023-10-16T10:01:55.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.911+0000 7f6a41545640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6a3c159ab0 0x7f6a3c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.911+0000 7f6a41545640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6a3c159ab0 0x7f6a3c0ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43506/0 (socket says 172.21.15.84:43506) 2023-10-16T10:01:55.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.911+0000 7f6a41545640 1 -- 172.21.15.84:0/3823527581 learned_addr learned my addr 172.21.15.84:0/3823527581 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:55.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.911+0000 7f6a41545640 1 -- 172.21.15.84:0/3823527581 >> v1:172.21.15.157:6789/0 conn(0x7f6a3c06c140 legacy=0x7f6a3c15dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:55.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.911+0000 7f6a41545640 1 -- 172.21.15.84:0/3823527581 >> v1:172.21.15.84:6789/0 conn(0x7f6a3c159e90 legacy=0x7f6a3c06fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.911+0000 7f6a33fff640 1 -- 172.21.15.84:0/3823527581 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3724964294 0 0) 0x7f6a3c070360 con 0x7f6a3c159e90 2023-10-16T10:01:55.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.911+0000 7f6a41545640 1 -- 172.21.15.84:0/3823527581 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c0cd4d0 con 0x7f6a3c159ab0 2023-10-16T10:01:55.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.912+0000 7f6a41545640 1 --2- 172.21.15.84:0/3823527581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a3c159ab0 0x7f6a3c0ce370 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009870 tx=0x7f6a3802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef1aa7f9c1c30fd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:55.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.913+0000 7f6a33fff640 1 -- 172.21.15.84:0/3823527581 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a380317e0 con 0x7f6a3c159ab0 2023-10-16T10:01:55.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.913+0000 7f6a33fff640 1 -- 172.21.15.84:0/3823527581 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a38031940 con 0x7f6a3c159ab0 2023-10-16T10:01:55.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.913+0000 7f6a33fff640 1 -- 172.21.15.84:0/3823527581 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38031cd0 con 0x7f6a3c159ab0 2023-10-16T10:01:55.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.913+0000 7f6a437a9640 1 -- 172.21.15.84:0/3823527581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a3c159ab0 msgr2=0x7f6a3c0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.913+0000 7f6a437a9640 1 --2- 172.21.15.84:0/3823527581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a3c159ab0 0x7f6a3c0ce370 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009870 tx=0x7f6a3802b880 comp rx=0 tx=0).stop 2023-10-16T10:01:55.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.913+0000 7f6a437a9640 1 -- 172.21.15.84:0/3823527581 shutdown_connections 2023-10-16T10:01:55.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.913+0000 7f6a437a9640 1 --2- 172.21.15.84:0/3823527581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a3c159ab0 0x7f6a3c0ce370 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.913+0000 7f6a437a9640 1 -- 172.21.15.84:0/3823527581 >> 172.21.15.84:0/3823527581 conn(0x7f6a3c0ce9b0 msgr2=0x7f6a3c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:55.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.913+0000 7f6a437a9640 1 -- 172.21.15.84:0/3823527581 shutdown_connections 2023-10-16T10:01:55.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.914+0000 7f6a437a9640 1 -- 172.21.15.84:0/3823527581 wait complete. 2023-10-16T10:01:55.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.915+0000 7f6a437a9640 1 Processor -- start 2023-10-16T10:01:55.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.915+0000 7f6a437a9640 1 -- start start 2023-10-16T10:01:55.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.916+0000 7f6a437a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a3c06c140 0x7f6a3c0c2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.916+0000 7f6a437a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a3c159ab0 0x7f6a3c153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.916+0000 7f6a437a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a3c159e90 0x7f6a3c154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.916+0000 7f6a437a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c1f3570 con 0x7f6a3c159ab0 2023-10-16T10:01:55.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.916+0000 7f6a437a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c1f23a0 con 0x7f6a3c06c140 2023-10-16T10:01:55.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.916+0000 7f6a437a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c14a8e0 con 0x7f6a3c159e90 2023-10-16T10:01:55.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.917+0000 7f6a40d44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a3c159ab0 0x7f6a3c153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.917+0000 7f6a40d44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a3c159ab0 0x7f6a3c153d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43510/0 (socket says 172.21.15.84:43510) 2023-10-16T10:01:55.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.917+0000 7f6a40d44640 1 -- 172.21.15.84:0/3641782535 learned_addr learned my addr 172.21.15.84:0/3641782535 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:55.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.917+0000 7f6a41d46640 1 --2- 172.21.15.84:0/3641782535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a3c159e90 0x7f6a3c154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.917+0000 7f6a41545640 1 --2- 172.21.15.84:0/3641782535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a3c06c140 0x7f6a3c0c2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.917+0000 7f6a40d44640 1 -- 172.21.15.84:0/3641782535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a3c159e90 msgr2=0x7f6a3c154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.917+0000 7f6a40d44640 1 --2- 172.21.15.84:0/3641782535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a3c159e90 0x7f6a3c154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.917+0000 7f6a40d44640 1 -- 172.21.15.84:0/3641782535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a3c06c140 msgr2=0x7f6a3c0c2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.917+0000 7f6a40d44640 1 --2- 172.21.15.84:0/3641782535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a3c06c140 0x7f6a3c0c2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.917+0000 7f6a40d44640 1 -- 172.21.15.84:0/3641782535 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a380094d0 con 0x7f6a3c159ab0 2023-10-16T10:01:55.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.918+0000 7f6a40d44640 1 --2- 172.21.15.84:0/3641782535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a3c159ab0 0x7f6a3c153d00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00c4b0 tx=0x7f6a2c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:55.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.918+0000 7f6a31ffb640 1 -- 172.21.15.84:0/3641782535 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c005480 con 0x7f6a3c159ab0 2023-10-16T10:01:55.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.918+0000 7f6a31ffb640 1 -- 172.21.15.84:0/3641782535 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c0035f0 con 0x7f6a3c159ab0 2023-10-16T10:01:55.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.918+0000 7f6a31ffb640 1 -- 172.21.15.84:0/3641782535 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0039a0 con 0x7f6a3c159ab0 2023-10-16T10:01:55.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.918+0000 7f6a437a9640 1 -- 172.21.15.84:0/3641782535 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3c1f2ec0 con 0x7f6a3c159ab0 2023-10-16T10:01:55.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.918+0000 7f6a437a9640 1 -- 172.21.15.84:0/3641782535 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3c0c5300 con 0x7f6a3c159ab0 2023-10-16T10:01:55.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.919+0000 7f6a437a9640 1 -- 172.21.15.84:0/3641782535 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3c1f3050 con 0x7f6a3c159ab0 2023-10-16T10:01:55.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.920+0000 7f6a31ffb640 1 -- 172.21.15.84:0/3641782535 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f6a2c003b00 con 0x7f6a3c159ab0 2023-10-16T10:01:55.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.921+0000 7f6a31ffb640 1 -- 172.21.15.84:0/3641782535 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f6a2c088b20 con 0x7f6a3c159ab0 2023-10-16T10:01:55.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:55.925+0000 7f6a31ffb640 1 -- 172.21.15.84:0/3641782535 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6a2c053b40 con 0x7f6a3c159ab0 2023-10-16T10:01:56.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.119+0000 7f6a437a9640 1 -- 172.21.15.84:0/3641782535 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6a3c04e160 con 0x7f6a3c159ab0 2023-10-16T10:01:56.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.120+0000 7f6a31ffb640 1 -- 172.21.15.84:0/3641782535 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6a2c017020 con 0x7f6a3c159ab0 2023-10-16T10:01:56.121 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T10:01:56.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.122+0000 7f6a437a9640 1 -- 172.21.15.84:0/3641782535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a3c159ab0 msgr2=0x7f6a3c153d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.122+0000 7f6a437a9640 1 --2- 172.21.15.84:0/3641782535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a3c159ab0 0x7f6a3c153d00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00c4b0 tx=0x7f6a2c00c970 comp rx=0 tx=0).stop 2023-10-16T10:01:56.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.122+0000 7f6a437a9640 1 -- 172.21.15.84:0/3641782535 shutdown_connections 2023-10-16T10:01:56.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.122+0000 7f6a437a9640 1 --2- 172.21.15.84:0/3641782535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a3c159e90 0x7f6a3c154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.122+0000 7f6a437a9640 1 --2- 172.21.15.84:0/3641782535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a3c159ab0 0x7f6a3c153d00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.122+0000 7f6a437a9640 1 --2- 172.21.15.84:0/3641782535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a3c06c140 0x7f6a3c0c2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.122+0000 7f6a437a9640 1 -- 172.21.15.84:0/3641782535 >> 172.21.15.84:0/3641782535 conn(0x7f6a3c0ce9b0 msgr2=0x7f6a3c0b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:56.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.123+0000 7f6a437a9640 1 -- 172.21.15.84:0/3641782535 shutdown_connections 2023-10-16T10:01:56.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.123+0000 7f6a437a9640 1 -- 172.21.15.84:0/3641782535 wait complete. 2023-10-16T10:01:56.142 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:01:56.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.353+0000 7fbb313f4640 1 Processor -- start 2023-10-16T10:01:56.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.353+0000 7fbb313f4640 1 -- start start 2023-10-16T10:01:56.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.354+0000 7fbb313f4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb2c06d220 0x7fbb2c0c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.354+0000 7fbb313f4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbb2c065ab0 0x7fbb2c0cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.354+0000 7fbb313f4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2c0acd70 con 0x7fbb2c06d600 2023-10-16T10:01:56.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.354+0000 7fbb313f4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb2c0ac4a0 con 0x7fbb2c065ab0 2023-10-16T10:01:56.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.354+0000 7fbb313f4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c065f40 con 0x7fbb2c06d220 2023-10-16T10:01:56.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.355+0000 7fbb2b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbb2c065ab0 0x7fbb2c0cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.355+0000 7fbb2b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbb2c065ab0 0x7fbb2c0cdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43520/0 (socket says 172.21.15.84:43520) 2023-10-16T10:01:56.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.355+0000 7fbb2b7fe640 1 -- 172.21.15.84:0/189538609 learned_addr learned my addr 172.21.15.84:0/189538609 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:56.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.355+0000 7fbb2b7fe640 1 -- 172.21.15.84:0/189538609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb2c06d220 msgr2=0x7fbb2c0c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:56.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.355+0000 7fbb2b7fe640 1 --2- 172.21.15.84:0/189538609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb2c06d220 0x7fbb2c0c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.355+0000 7fbb2b7fe640 1 -- 172.21.15.84:0/189538609 >> v1:172.21.15.157:6789/0 conn(0x7fbb2c06d600 legacy=0x7fbb2c0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:56.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.355+0000 7fbb2b7fe640 1 -- 172.21.15.84:0/189538609 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c0ac720 con 0x7fbb2c065ab0 2023-10-16T10:01:56.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.356+0000 7fbb2b7fe640 1 --2- 172.21.15.84:0/189538609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb2c065ab0 0x7fbb2c0cdb70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009870 tx=0x7fbb1402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9586785d08a4a0b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:56.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.356+0000 7fbb2affd640 1 -- 172.21.15.84:0/189538609 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14031640 con 0x7fbb2c065ab0 2023-10-16T10:01:56.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.356+0000 7fbb2affd640 1 -- 172.21.15.84:0/189538609 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb140317a0 con 0x7fbb2c065ab0 2023-10-16T10:01:56.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.356+0000 7fbb2affd640 1 -- 172.21.15.84:0/189538609 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14031b30 con 0x7fbb2c065ab0 2023-10-16T10:01:56.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.357+0000 7fbb313f4640 1 -- 172.21.15.84:0/189538609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb2c065ab0 msgr2=0x7fbb2c0cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.357+0000 7fbb313f4640 1 --2- 172.21.15.84:0/189538609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb2c065ab0 0x7fbb2c0cdb70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009870 tx=0x7fbb1402b8f0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.357+0000 7fbb313f4640 1 -- 172.21.15.84:0/189538609 shutdown_connections 2023-10-16T10:01:56.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.357+0000 7fbb313f4640 1 --2- 172.21.15.84:0/189538609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb2c065ab0 0x7fbb2c0cdb70 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.357+0000 7fbb313f4640 1 --2- 172.21.15.84:0/189538609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb2c06d220 0x7fbb2c0c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.357+0000 7fbb313f4640 1 -- 172.21.15.84:0/189538609 >> 172.21.15.84:0/189538609 conn(0x7fbb2c0b0a80 msgr2=0x7fbb2c0b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:56.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.357+0000 7fbb313f4640 1 -- 172.21.15.84:0/189538609 shutdown_connections 2023-10-16T10:01:56.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.357+0000 7fbb313f4640 1 -- 172.21.15.84:0/189538609 wait complete. 2023-10-16T10:01:56.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.359+0000 7fbb313f4640 1 Processor -- start 2023-10-16T10:01:56.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.359+0000 7fbb313f4640 1 -- start start 2023-10-16T10:01:56.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.360+0000 7fbb313f4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb2c06d220 0x7fbb2c152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.360+0000 7fbb313f4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb2c06d600 0x7fbb2c152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.360+0000 7fbb313f4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb2c0b04f0 0x7fbb2c1f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.360+0000 7fbb313f4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0c7970 con 0x7fbb2c0b04f0 2023-10-16T10:01:56.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.360+0000 7fbb313f4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c152aa0 con 0x7fbb2c06d600 2023-10-16T10:01:56.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.360+0000 7fbb313f4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0c73b0 con 0x7fbb2c06d220 2023-10-16T10:01:56.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.360+0000 7fbb30bf3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb2c0b04f0 0x7fbb2c1f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.360+0000 7fbb30bf3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb2c0b04f0 0x7fbb2c1f5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43536/0 (socket says 172.21.15.84:43536) 2023-10-16T10:01:56.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.360+0000 7fbb30bf3640 1 -- 172.21.15.84:0/2026831184 learned_addr learned my addr 172.21.15.84:0/2026831184 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:56.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.360+0000 7fbb2bfff640 1 --2- 172.21.15.84:0/2026831184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb2c06d220 0x7fbb2c152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.360+0000 7fbb30bf3640 1 -- 172.21.15.84:0/2026831184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb2c06d220 msgr2=0x7fbb2c152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.360+0000 7fbb30bf3640 1 --2- 172.21.15.84:0/2026831184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb2c06d220 0x7fbb2c152020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.360+0000 7fbb30bf3640 1 -- 172.21.15.84:0/2026831184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb2c06d600 msgr2=0x7fbb2c152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.360+0000 7fbb2b7fe640 1 --2- 172.21.15.84:0/2026831184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb2c06d600 0x7fbb2c152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.360+0000 7fbb30bf3640 1 --2- 172.21.15.84:0/2026831184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb2c06d600 0x7fbb2c152560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.360+0000 7fbb30bf3640 1 -- 172.21.15.84:0/2026831184 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb140094d0 con 0x7fbb2c0b04f0 2023-10-16T10:01:56.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.361+0000 7fbb30bf3640 1 --2- 172.21.15.84:0/2026831184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb2c0b04f0 0x7fbb2c1f5a60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400c390 tx=0x7fbb2400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:56.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.361+0000 7fbb28ff9640 1 -- 172.21.15.84:0/2026831184 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb240035f0 con 0x7fbb2c0b04f0 2023-10-16T10:01:56.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.361+0000 7fbb28ff9640 1 -- 172.21.15.84:0/2026831184 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb24003750 con 0x7fbb2c0b04f0 2023-10-16T10:01:56.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.361+0000 7fbb313f4640 1 -- 172.21.15.84:0/2026831184 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2c0c7690 con 0x7fbb2c0b04f0 2023-10-16T10:01:56.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.361+0000 7fbb28ff9640 1 -- 172.21.15.84:0/2026831184 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24003b00 con 0x7fbb2c0b04f0 2023-10-16T10:01:56.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.361+0000 7fbb313f4640 1 -- 172.21.15.84:0/2026831184 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2c0b2030 con 0x7fbb2c0b04f0 2023-10-16T10:01:56.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.362+0000 7fbb313f4640 1 -- 172.21.15.84:0/2026831184 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb2c0b21f0 con 0x7fbb2c0b04f0 2023-10-16T10:01:56.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.363+0000 7fbb28ff9640 1 -- 172.21.15.84:0/2026831184 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fbb24003c60 con 0x7fbb2c0b04f0 2023-10-16T10:01:56.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.364+0000 7fbb28ff9640 1 --2- 172.21.15.84:0/2026831184 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fbb0c06acc0 0x7fbb0c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.364+0000 7fbb28ff9640 1 -- 172.21.15.84:0/2026831184 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6718+0+0 (secure 0 0 0) 0x7fbb24088be0 con 0x7fbb2c0b04f0 2023-10-16T10:01:56.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.364+0000 7fbb2bfff640 1 --2- 172.21.15.84:0/2026831184 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fbb0c06acc0 0x7fbb0c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.365+0000 7fbb2bfff640 1 --2- 172.21.15.84:0/2026831184 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fbb0c06acc0 0x7fbb0c06d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c007dc0 tx=0x7fbb1c0072f0 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:56.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.369+0000 7fbb28ff9640 1 -- 172.21.15.84:0/2026831184 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbb24053be0 con 0x7fbb2c0b04f0 2023-10-16T10:01:56.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.579+0000 7fbb313f4640 1 -- 172.21.15.84:0/2026831184 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb2c04e9b0 con 0x7fbb2c0b04f0 2023-10-16T10:01:56.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.580+0000 7fbb28ff9640 1 -- 172.21.15.84:0/2026831184 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fbb24017020 con 0x7fbb2c0b04f0 2023-10-16T10:01:56.582 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:01:56.582 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3662158203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3662158203},{"type":"v1","addr":"172.21.15.157:6835","nonce":3662158203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5091,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2502120537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2502120537},{"type":"v1","addr":"172.21.15.84:6836","nonce":2502120537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2715651376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2715651376},{"type":"v1","addr":"172.21.15.84:6837","nonce":2715651376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:50:12.380180+0000","modified":"2023-10-16T09:50:15.438597+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6833/62459899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":62459899},{"type":"v1","addr":"172.21.15.157:6833","nonce":62459899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T10:01:56.583 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 17 2023-10-16T10:01:56.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.583+0000 7fbb313f4640 1 -- 172.21.15.84:0/2026831184 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fbb0c06acc0 msgr2=0x7fbb0c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.583+0000 7fbb313f4640 1 --2- 172.21.15.84:0/2026831184 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fbb0c06acc0 0x7fbb0c06d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c007dc0 tx=0x7fbb1c0072f0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.583+0000 7fbb313f4640 1 -- 172.21.15.84:0/2026831184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb2c0b04f0 msgr2=0x7fbb2c1f5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.583+0000 7fbb313f4640 1 --2- 172.21.15.84:0/2026831184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb2c0b04f0 0x7fbb2c1f5a60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400c390 tx=0x7fbb2400c850 comp rx=0 tx=0).stop 2023-10-16T10:01:56.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.583+0000 7fbb313f4640 1 -- 172.21.15.84:0/2026831184 shutdown_connections 2023-10-16T10:01:56.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.583+0000 7fbb313f4640 1 --2- 172.21.15.84:0/2026831184 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fbb0c06acc0 0x7fbb0c06d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.583+0000 7fbb313f4640 1 --2- 172.21.15.84:0/2026831184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb2c0b04f0 0x7fbb2c1f5a60 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.583+0000 7fbb313f4640 1 --2- 172.21.15.84:0/2026831184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb2c06d600 0x7fbb2c152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.583+0000 7fbb313f4640 1 --2- 172.21.15.84:0/2026831184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb2c06d220 0x7fbb2c152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.583+0000 7fbb313f4640 1 -- 172.21.15.84:0/2026831184 >> 172.21.15.84:0/2026831184 conn(0x7fbb2c0b0a80 msgr2=0x7fbb2c0c9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:56.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.583+0000 7fbb313f4640 1 -- 172.21.15.84:0/2026831184 shutdown_connections 2023-10-16T10:01:56.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.583+0000 7fbb313f4640 1 -- 172.21.15.84:0/2026831184 wait complete. 2023-10-16T10:01:56.601 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:01:56.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.801+0000 7f0ade415640 1 Processor -- start 2023-10-16T10:01:56.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.802+0000 7f0ade415640 1 -- start start 2023-10-16T10:01:56.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.803+0000 7f0ade415640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0ad806b4c0 0x7f0ad815be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.803+0000 7f0ade415640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0ad806fdb0 0x7f0ad8070200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.803+0000 7f0ade415640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad815c3a0 con 0x7f0ad815adf0 2023-10-16T10:01:56.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.803+0000 7f0ade415640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad806dc00 con 0x7f0ad806b4c0 2023-10-16T10:01:56.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.803+0000 7f0ade415640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad806cf80 con 0x7f0ad806fdb0 2023-10-16T10:01:56.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.803+0000 7f0ad77fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0ad806b4c0 0x7f0ad815be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.803+0000 7f0ad77fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0ad806b4c0 0x7f0ad815be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43548/0 (socket says 172.21.15.84:43548) 2023-10-16T10:01:56.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.803+0000 7f0ad77fe640 1 -- 172.21.15.84:0/3834478904 learned_addr learned my addr 172.21.15.84:0/3834478904 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:56.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.804+0000 7f0adc9b2640 1 --2- 172.21.15.84:0/3834478904 >> v2:172.21.15.157:3300/0 conn(0x7f0ad806fdb0 0x7f0ad8070200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.804+0000 7f0ad77fe640 1 -- 172.21.15.84:0/3834478904 >> v2:172.21.15.157:3300/0 conn(0x7f0ad806fdb0 msgr2=0x7f0ad8070200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:56.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.804+0000 7f0ad77fe640 1 --2- 172.21.15.84:0/3834478904 >> v2:172.21.15.157:3300/0 conn(0x7f0ad806fdb0 0x7f0ad8070200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.804+0000 7f0ad77fe640 1 -- 172.21.15.84:0/3834478904 >> v1:172.21.15.157:6789/0 conn(0x7f0ad815adf0 legacy=0x7f0ad806adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:56.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.804+0000 7f0ad77fe640 1 -- 172.21.15.84:0/3834478904 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad806d260 con 0x7f0ad806b4c0 2023-10-16T10:01:56.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.805+0000 7f0ad77fe640 1 --2- 172.21.15.84:0/3834478904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad806b4c0 0x7f0ad815be60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0ad809cf40 tx=0x7f0ac002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d292f69212ab64b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:56.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.805+0000 7f0ad6ffd640 1 -- 172.21.15.84:0/3834478904 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0031870 con 0x7f0ad806b4c0 2023-10-16T10:01:56.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.805+0000 7f0ad6ffd640 1 -- 172.21.15.84:0/3834478904 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac00319d0 con 0x7f0ad806b4c0 2023-10-16T10:01:56.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.806+0000 7f0ad6ffd640 1 -- 172.21.15.84:0/3834478904 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0031d60 con 0x7f0ad806b4c0 2023-10-16T10:01:56.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.806+0000 7f0ade415640 1 -- 172.21.15.84:0/3834478904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad806b4c0 msgr2=0x7f0ad815be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.806+0000 7f0ade415640 1 --2- 172.21.15.84:0/3834478904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad806b4c0 0x7f0ad815be60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0ad809cf40 tx=0x7f0ac002b910 comp rx=0 tx=0).stop 2023-10-16T10:01:56.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.806+0000 7f0ade415640 1 -- 172.21.15.84:0/3834478904 shutdown_connections 2023-10-16T10:01:56.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.806+0000 7f0ade415640 1 --2- 172.21.15.84:0/3834478904 >> v2:172.21.15.157:3300/0 conn(0x7f0ad806fdb0 0x7f0ad8070200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.806+0000 7f0ade415640 1 --2- 172.21.15.84:0/3834478904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad806b4c0 0x7f0ad815be60 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.806+0000 7f0ade415640 1 -- 172.21.15.84:0/3834478904 >> 172.21.15.84:0/3834478904 conn(0x7f0ad80b0d30 msgr2=0x7f0ad80b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:56.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.806+0000 7f0ade415640 1 -- 172.21.15.84:0/3834478904 shutdown_connections 2023-10-16T10:01:56.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.806+0000 7f0ade415640 1 -- 172.21.15.84:0/3834478904 wait complete. 2023-10-16T10:01:56.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.808+0000 7f0ade415640 1 Processor -- start 2023-10-16T10:01:56.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.808+0000 7f0ade415640 1 -- start start 2023-10-16T10:01:56.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.809+0000 7f0ade415640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ad806b4c0 0x7f0ad81f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.809+0000 7f0ade415640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ad806fdb0 0x7f0ad81f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.809+0000 7f0ade415640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad815adf0 0x7f0ad80b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.809+0000 7f0ade415640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad81599b0 con 0x7f0ad815adf0 2023-10-16T10:01:56.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.809+0000 7f0ade415640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad80b4ab0 con 0x7f0ad806fdb0 2023-10-16T10:01:56.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.809+0000 7f0ade415640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad80b4d90 con 0x7f0ad806b4c0 2023-10-16T10:01:56.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.809+0000 7f0adc9b2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad815adf0 0x7f0ad80b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.809+0000 7f0adc9b2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad815adf0 0x7f0ad80b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43560/0 (socket says 172.21.15.84:43560) 2023-10-16T10:01:56.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.809+0000 7f0adc9b2640 1 -- 172.21.15.84:0/848941823 learned_addr learned my addr 172.21.15.84:0/848941823 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:56.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.810+0000 7f0adc9b2640 1 -- 172.21.15.84:0/848941823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ad806b4c0 msgr2=0x7f0ad81f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.810+0000 7f0adc9b2640 1 --2- 172.21.15.84:0/848941823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ad806b4c0 0x7f0ad81f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.810+0000 7f0adc9b2640 1 -- 172.21.15.84:0/848941823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ad806fdb0 msgr2=0x7f0ad81f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.810+0000 7f0adc9b2640 1 --2- 172.21.15.84:0/848941823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ad806fdb0 0x7f0ad81f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.810+0000 7f0adc9b2640 1 -- 172.21.15.84:0/848941823 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac00094d0 con 0x7f0ad815adf0 2023-10-16T10:01:56.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.810+0000 7f0adc9b2640 1 --2- 172.21.15.84:0/848941823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad815adf0 0x7f0ad80b3d40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000d800 tx=0x7f0ad000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:56.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.810+0000 7f0ad4ff9640 1 -- 172.21.15.84:0/848941823 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad000b6c0 con 0x7f0ad815adf0 2023-10-16T10:01:56.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.810+0000 7f0ad4ff9640 1 -- 172.21.15.84:0/848941823 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad000b840 con 0x7f0ad815adf0 2023-10-16T10:01:56.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.810+0000 7f0ad4ff9640 1 -- 172.21.15.84:0/848941823 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad000bbf0 con 0x7f0ad815adf0 2023-10-16T10:01:56.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.810+0000 7f0ade415640 1 -- 172.21.15.84:0/848941823 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad80c3e40 con 0x7f0ad815adf0 2023-10-16T10:01:56.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.810+0000 7f0ade415640 1 -- 172.21.15.84:0/848941823 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad80c4400 con 0x7f0ad815adf0 2023-10-16T10:01:56.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.812+0000 7f0ade415640 1 -- 172.21.15.84:0/848941823 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad804e160 con 0x7f0ad815adf0 2023-10-16T10:01:56.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.812+0000 7f0ad4ff9640 1 -- 172.21.15.84:0/848941823 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f0ad000bd50 con 0x7f0ad815adf0 2023-10-16T10:01:56.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.813+0000 7f0ad4ff9640 1 --2- 172.21.15.84:0/848941823 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f0ab806acc0 0x7f0ab806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.813+0000 7f0ad4ff9640 1 -- 172.21.15.84:0/848941823 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f0ad0089c80 con 0x7f0ad815adf0 2023-10-16T10:01:56.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.814+0000 7f0ad7fff640 1 --2- 172.21.15.84:0/848941823 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f0ab806acc0 0x7f0ab806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.814+0000 7f0ad7fff640 1 --2- 172.21.15.84:0/848941823 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f0ab806acc0 0x7f0ab806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0ac800ab90 tx=0x7f0ac8009230 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:56.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:56.817+0000 7f0ad4ff9640 1 -- 172.21.15.84:0/848941823 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0ad0054bb0 con 0x7f0ad815adf0 2023-10-16T10:01:57.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.013+0000 7f0ade415640 1 -- 172.21.15.84:0/848941823 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ad8066000 con 0x7f0ad815adf0 2023-10-16T10:01:57.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.015+0000 7f0ad4ff9640 1 -- 172.21.15.84:0/848941823 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18060 (secure 0 0 0) 0x7f0ad00581d0 con 0x7f0ad815adf0 2023-10-16T10:01:57.017 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:01:57.017 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":51,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:01:55.223630+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:50:10.297512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-16T09:50:11.394493+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":50,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2429213939":"2023-10-17T10:01:55.223514+0000","172.21.15.84:6833/3010246666":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/1060139177":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/3968394661":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/2493862491":"2023-10-17T10:01:55.223514+0000","172.21.15.84:6832/3010246666":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/1628587200":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/3763261264":"2023-10-17T10:01:55.223514+0000","172.21.15.84:0/848414450":"2023-10-17T10:01:55.223514+0000","172.21.15.84:6832/4265272960":"2023-10-17T10:01:55.223514+0000","172.21.15.84:0/3285111878":"2023-10-17T10:01:55.223514+0000","172.21.15.84:0/3734084949":"2023-10-17T10:01:55.223514+0000","172.21.15.84:0/982275204":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/1706229169":"2023-10-17T10:01:55.223514+0000","172.21.15.84:6833/4265272960":"2023-10-17T10:01:55.223514+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":7,"length":1},{"begin":11,"length":1},{"begin":14,"length":1},{"begin":20,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":48,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":72,"length":1},{"begin":76,"length":1},{"begin":84,"length":2},{"begin":92,"length":1},{"begin":100,"length":1},{"begin":103,"length":1},{"begin":106,"length":1},{"begin":200,"length":22}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:01:57.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.017+0000 7f0ade415640 1 -- 172.21.15.84:0/848941823 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f0ab806acc0 msgr2=0x7f0ab806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.017+0000 7f0ade415640 1 --2- 172.21.15.84:0/848941823 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f0ab806acc0 0x7f0ab806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0ac800ab90 tx=0x7f0ac8009230 comp rx=0 tx=0).stop 2023-10-16T10:01:57.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.017+0000 7f0ade415640 1 -- 172.21.15.84:0/848941823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad815adf0 msgr2=0x7f0ad80b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.017+0000 7f0ade415640 1 --2- 172.21.15.84:0/848941823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad815adf0 0x7f0ad80b3d40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000d800 tx=0x7f0ad000dcc0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.017+0000 7f0ade415640 1 -- 172.21.15.84:0/848941823 shutdown_connections 2023-10-16T10:01:57.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.018+0000 7f0ade415640 1 --2- 172.21.15.84:0/848941823 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f0ab806acc0 0x7f0ab806d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.018+0000 7f0ade415640 1 --2- 172.21.15.84:0/848941823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ad815adf0 0x7f0ad80b3d40 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.018+0000 7f0ade415640 1 --2- 172.21.15.84:0/848941823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ad806fdb0 0x7f0ad81f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.018+0000 7f0ade415640 1 --2- 172.21.15.84:0/848941823 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ad806b4c0 0x7f0ad81f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.018+0000 7f0ade415640 1 -- 172.21.15.84:0/848941823 >> 172.21.15.84:0/848941823 conn(0x7f0ad80b0d30 msgr2=0x7f0ad815c540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:57.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.018+0000 7f0ade415640 1 -- 172.21.15.84:0/848941823 shutdown_connections 2023-10-16T10:01:57.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.018+0000 7f0ade415640 1 -- 172.21.15.84:0/848941823 wait complete. 2023-10-16T10:01:57.039 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T10:01:57.039 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:01:57.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.248+0000 7fb9bf800640 1 Processor -- start 2023-10-16T10:01:57.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.248+0000 7fb9bf800640 1 -- start start 2023-10-16T10:01:57.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.250+0000 7fb9bf800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9b80afef0 0x7fb9b80cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.250+0000 7fb9bf800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9b806aeb0 0x7fb9b806b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.250+0000 7fb9bf800640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb9b80b24a0 0x7fb9b80b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.250+0000 7fb9bf800640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b806b870 con 0x7fb9b80b24a0 2023-10-16T10:01:57.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.250+0000 7fb9bf800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b80b2e70 con 0x7fb9b80afef0 2023-10-16T10:01:57.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.250+0000 7fb9bf800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b80d0180 con 0x7fb9b806aeb0 2023-10-16T10:01:57.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.250+0000 7fb9bdd9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb9b80b24a0 0x7fb9b80b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.250+0000 7fb9bdd9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb9b80b24a0 0x7fb9b80b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43574/0 (socket says 172.21.15.84:43574) 2023-10-16T10:01:57.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.250+0000 7fb9bdd9d640 1 -- 172.21.15.84:0/1508107581 learned_addr learned my addr 172.21.15.84:0/1508107581 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:57.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.250+0000 7fb9bd59c640 1 --2- 172.21.15.84:0/1508107581 >> v2:172.21.15.157:3300/0 conn(0x7fb9b80afef0 0x7fb9b80cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.250+0000 7fb9bdd9d640 1 -- 172.21.15.84:0/1508107581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9b806aeb0 msgr2=0x7fb9b806b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:57.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.250+0000 7fb9bcd9b640 1 --2- 172.21.15.84:0/1508107581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9b806aeb0 0x7fb9b806b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.250+0000 7fb9bdd9d640 1 --2- 172.21.15.84:0/1508107581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9b806aeb0 0x7fb9b806b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.250+0000 7fb9bdd9d640 1 -- 172.21.15.84:0/1508107581 >> v2:172.21.15.157:3300/0 conn(0x7fb9b80afef0 msgr2=0x7fb9b80cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:57.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.250+0000 7fb9bdd9d640 1 --2- 172.21.15.84:0/1508107581 >> v2:172.21.15.157:3300/0 conn(0x7fb9b80afef0 0x7fb9b80cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.250+0000 7fb9bdd9d640 1 -- 172.21.15.84:0/1508107581 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b80cfea0 con 0x7fb9b80b24a0 2023-10-16T10:01:57.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.252+0000 7fb9bdd9d640 1 --2- 172.21.15.84:0/1508107581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b80b24a0 0x7fb9b80b2930 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8071b90 tx=0x7fb9b002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bbdc7a3423d7aa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:57.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.252+0000 7fb9affff640 1 -- 172.21.15.84:0/1508107581 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b002c950 con 0x7fb9b80b24a0 2023-10-16T10:01:57.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.252+0000 7fb9affff640 1 -- 172.21.15.84:0/1508107581 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b002cab0 con 0x7fb9b80b24a0 2023-10-16T10:01:57.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.252+0000 7fb9affff640 1 -- 172.21.15.84:0/1508107581 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b002ce40 con 0x7fb9b80b24a0 2023-10-16T10:01:57.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.252+0000 7fb9bf800640 1 -- 172.21.15.84:0/1508107581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b80b24a0 msgr2=0x7fb9b80b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.252+0000 7fb9bf800640 1 --2- 172.21.15.84:0/1508107581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b80b24a0 0x7fb9b80b2930 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8071b90 tx=0x7fb9b002bdb0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.253+0000 7fb9bf800640 1 -- 172.21.15.84:0/1508107581 shutdown_connections 2023-10-16T10:01:57.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.253+0000 7fb9bf800640 1 --2- 172.21.15.84:0/1508107581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b80b24a0 0x7fb9b80b2930 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.253+0000 7fb9bf800640 1 --2- 172.21.15.84:0/1508107581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9b806aeb0 0x7fb9b806b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.253+0000 7fb9bf800640 1 --2- 172.21.15.84:0/1508107581 >> v2:172.21.15.157:3300/0 conn(0x7fb9b80afef0 0x7fb9b80cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.253+0000 7fb9bf800640 1 -- 172.21.15.84:0/1508107581 >> 172.21.15.84:0/1508107581 conn(0x7fb9b80d0ad0 msgr2=0x7fb9b80d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:57.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.253+0000 7fb9bf800640 1 -- 172.21.15.84:0/1508107581 shutdown_connections 2023-10-16T10:01:57.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.253+0000 7fb9bf800640 1 -- 172.21.15.84:0/1508107581 wait complete. 2023-10-16T10:01:57.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.254+0000 7fb9bf800640 1 Processor -- start 2023-10-16T10:01:57.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.255+0000 7fb9bf800640 1 -- start start 2023-10-16T10:01:57.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.255+0000 7fb9bf800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9b806aeb0 0x7fb9b8155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.256+0000 7fb9bf800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9b80afef0 0x7fb9b80ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.256+0000 7fb9bf800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b80b24a0 0x7fb9b815e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.256+0000 7fb9bf800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b80b3cc0 con 0x7fb9b80b24a0 2023-10-16T10:01:57.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.256+0000 7fb9bf800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b80b30b0 con 0x7fb9b806aeb0 2023-10-16T10:01:57.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.256+0000 7fb9bf800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b80b3390 con 0x7fb9b80afef0 2023-10-16T10:01:57.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.256+0000 7fb9bdd9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b80b24a0 0x7fb9b815e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.256+0000 7fb9bdd9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b80b24a0 0x7fb9b815e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43576/0 (socket says 172.21.15.84:43576) 2023-10-16T10:01:57.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.256+0000 7fb9bdd9d640 1 -- 172.21.15.84:0/2437536805 learned_addr learned my addr 172.21.15.84:0/2437536805 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:57.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.256+0000 7fb9bd59c640 1 --2- 172.21.15.84:0/2437536805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9b806aeb0 0x7fb9b8155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.256+0000 7fb9bdd9d640 1 -- 172.21.15.84:0/2437536805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9b80afef0 msgr2=0x7fb9b80ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.256+0000 7fb9bdd9d640 1 --2- 172.21.15.84:0/2437536805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9b80afef0 0x7fb9b80ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.256+0000 7fb9bdd9d640 1 -- 172.21.15.84:0/2437536805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9b806aeb0 msgr2=0x7fb9b8155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.256+0000 7fb9bdd9d640 1 --2- 172.21.15.84:0/2437536805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9b806aeb0 0x7fb9b8155620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.256+0000 7fb9bdd9d640 1 -- 172.21.15.84:0/2437536805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b0009b30 con 0x7fb9b80b24a0 2023-10-16T10:01:57.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.257+0000 7fb9bdd9d640 1 --2- 172.21.15.84:0/2437536805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b80b24a0 0x7fb9b815e580 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0036040 tx=0x7fb9b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:57.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.257+0000 7fb9adffb640 1 -- 172.21.15.84:0/2437536805 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0009510 con 0x7fb9b80b24a0 2023-10-16T10:01:57.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.257+0000 7fb9adffb640 1 -- 172.21.15.84:0/2437536805 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b003e450 con 0x7fb9b80b24a0 2023-10-16T10:01:57.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.257+0000 7fb9bf800640 1 -- 172.21.15.84:0/2437536805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b80b3670 con 0x7fb9b80b24a0 2023-10-16T10:01:57.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.257+0000 7fb9adffb640 1 -- 172.21.15.84:0/2437536805 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b003e800 con 0x7fb9b80b24a0 2023-10-16T10:01:57.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.257+0000 7fb9bf800640 1 -- 172.21.15.84:0/2437536805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b8155b60 con 0x7fb9b80b24a0 2023-10-16T10:01:57.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.258+0000 7fb9bf800640 1 -- 172.21.15.84:0/2437536805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b80b2180 con 0x7fb9b80b24a0 2023-10-16T10:01:57.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.259+0000 7fb9adffb640 1 -- 172.21.15.84:0/2437536805 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fb9b003e960 con 0x7fb9b80b24a0 2023-10-16T10:01:57.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.261+0000 7fb9adffb640 1 --2- 172.21.15.84:0/2437536805 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fb99c06acf0 0x7fb99c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.261+0000 7fb9adffb640 1 -- 172.21.15.84:0/2437536805 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6718+0+0 (secure 0 0 0) 0x7fb9b00ac440 con 0x7fb9b80b24a0 2023-10-16T10:01:57.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.261+0000 7fb9bd59c640 1 --2- 172.21.15.84:0/2437536805 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fb99c06acf0 0x7fb99c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.262+0000 7fb9bd59c640 1 --2- 172.21.15.84:0/2437536805 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fb99c06acf0 0x7fb99c06d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400b7a0 tx=0x7fb9b4008040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:57.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.265+0000 7fb9adffb640 1 -- 172.21.15.84:0/2437536805 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb9b00774f0 con 0x7fb9b80b24a0 2023-10-16T10:01:57.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.508+0000 7fb9bf800640 1 -- 172.21.15.84:0/2437536805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9b804e160 con 0x7fb9b80b24a0 2023-10-16T10:01:57.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.509+0000 7fb9adffb640 1 -- 172.21.15.84:0/2437536805 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fb9b007ab10 con 0x7fb9b80b24a0 2023-10-16T10:01:57.511 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:01:57.511 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3662158203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3662158203},{"type":"v1","addr":"172.21.15.157:6835","nonce":3662158203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5091,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2502120537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2502120537},{"type":"v1","addr":"172.21.15.84:6836","nonce":2502120537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2715651376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2715651376},{"type":"v1","addr":"172.21.15.84:6837","nonce":2715651376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:50:12.380180+0000","modified":"2023-10-16T09:50:15.438597+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6833/62459899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":62459899},{"type":"v1","addr":"172.21.15.157:6833","nonce":62459899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T10:01:57.511 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 17 2023-10-16T10:01:57.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.512+0000 7fb9bf800640 1 -- 172.21.15.84:0/2437536805 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fb99c06acf0 msgr2=0x7fb99c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.512+0000 7fb9bf800640 1 --2- 172.21.15.84:0/2437536805 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fb99c06acf0 0x7fb99c06d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400b7a0 tx=0x7fb9b4008040 comp rx=0 tx=0).stop 2023-10-16T10:01:57.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.512+0000 7fb9bf800640 1 -- 172.21.15.84:0/2437536805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b80b24a0 msgr2=0x7fb9b815e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.512+0000 7fb9bf800640 1 --2- 172.21.15.84:0/2437536805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b80b24a0 0x7fb9b815e580 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0036040 tx=0x7fb9b0004a40 comp rx=0 tx=0).stop 2023-10-16T10:01:57.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.512+0000 7fb9bf800640 1 -- 172.21.15.84:0/2437536805 shutdown_connections 2023-10-16T10:01:57.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.512+0000 7fb9bf800640 1 --2- 172.21.15.84:0/2437536805 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fb99c06acf0 0x7fb99c06d1b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.512+0000 7fb9bf800640 1 --2- 172.21.15.84:0/2437536805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b80b24a0 0x7fb9b815e580 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.512+0000 7fb9bf800640 1 --2- 172.21.15.84:0/2437536805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9b80afef0 0x7fb9b80ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.512+0000 7fb9bf800640 1 --2- 172.21.15.84:0/2437536805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9b806aeb0 0x7fb9b8155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.512+0000 7fb9bf800640 1 -- 172.21.15.84:0/2437536805 >> 172.21.15.84:0/2437536805 conn(0x7fb9b80d0ad0 msgr2=0x7fb9b815fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:57.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.512+0000 7fb9bf800640 1 -- 172.21.15.84:0/2437536805 shutdown_connections 2023-10-16T10:01:57.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.512+0000 7fb9bf800640 1 -- 172.21.15.84:0/2437536805 wait complete. 2023-10-16T10:01:57.531 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:01:57.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.741+0000 7fcf52f1d640 1 Processor -- start 2023-10-16T10:01:57.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.742+0000 7fcf52f1d640 1 -- start start 2023-10-16T10:01:57.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.743+0000 7fcf52f1d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c1593b0 0x7fcf4c06f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.743+0000 7fcf52f1d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf4c159790 con 0x7fcf4c0cc920 2023-10-16T10:01:57.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.743+0000 7fcf52f1d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf4c0c9b20 con 0x7fcf4c0caa10 2023-10-16T10:01:57.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.743+0000 7fcf52f1d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c0c9e00 con 0x7fcf4c1593b0 2023-10-16T10:01:57.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.744+0000 7fcf514ba640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fcf4c0cc920 0x7fcf4c0ccdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:50276/0 (socket says 172.21.15.84:50276) 2023-10-16T10:01:57.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.744+0000 7fcf514ba640 1 -- 172.21.15.84:0/198507869 learned_addr learned my addr 172.21.15.84:0/198507869 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:57.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.744+0000 7fcf50cb9640 1 --2- 172.21.15.84:0/198507869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c1593b0 0x7fcf4c06f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.744+0000 7fcf4b7fe640 1 -- 172.21.15.84:0/198507869 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165377291 0 0) 0x7fcf4c159790 con 0x7fcf4c0cc920 2023-10-16T10:01:57.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.744+0000 7fcf4b7fe640 1 -- 172.21.15.84:0/198507869 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf3c003440 con 0x7fcf4c0cc920 2023-10-16T10:01:57.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.744+0000 7fcf4b7fe640 1 -- 172.21.15.84:0/198507869 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2250260207 0 0) 0x7fcf3c003440 con 0x7fcf4c0cc920 2023-10-16T10:01:57.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.744+0000 7fcf4b7fe640 1 -- 172.21.15.84:0/198507869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c1593b0 msgr2=0x7fcf4c06f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:57.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.744+0000 7fcf4b7fe640 1 --2- 172.21.15.84:0/198507869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c1593b0 0x7fcf4c06f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.744+0000 7fcf4b7fe640 1 -- 172.21.15.84:0/198507869 >> v1:172.21.15.157:6789/0 conn(0x7fcf4c0caa10 legacy=0x7fcf4c06fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.744+0000 7fcf4b7fe640 1 -- 172.21.15.84:0/198507869 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c0ca0e0 con 0x7fcf4c0cc920 2023-10-16T10:01:57.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.744+0000 7fcf4b7fe640 1 -- 172.21.15.84:0/198507869 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fcf440019f0 con 0x7fcf4c0cc920 2023-10-16T10:01:57.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.744+0000 7fcf4b7fe640 1 -- 172.21.15.84:0/198507869 >> v1:172.21.15.84:6789/0 conn(0x7fcf4c0cc920 legacy=0x7fcf4c0ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.745+0000 7fcf4b7fe640 1 --2- 172.21.15.84:0/198507869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf3c003a90 0x7fcf3c003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.745+0000 7fcf4b7fe640 1 --2- 172.21.15.84:0/198507869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c1593b0 0x7fcf3c00a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.745+0000 7fcf4b7fe640 1 --2- 172.21.15.84:0/198507869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3c00b050 0x7fcf3c00d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.745+0000 7fcf4b7fe640 1 -- 172.21.15.84:0/198507869 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c0c9e00 con 0x7fcf3c003a90 2023-10-16T10:01:57.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.745+0000 7fcf4b7fe640 1 -- 172.21.15.84:0/198507869 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c00dd60 con 0x7fcf3c00b050 2023-10-16T10:01:57.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.745+0000 7fcf4b7fe640 1 -- 172.21.15.84:0/198507869 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c00df20 con 0x7fcf4c1593b0 2023-10-16T10:01:57.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.745+0000 7fcf50cb9640 1 --2- 172.21.15.84:0/198507869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf3c003a90 0x7fcf3c003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.745+0000 7fcf50cb9640 1 -- 172.21.15.84:0/198507869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c1593b0 msgr2=0x7fcf3c00a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:57.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.745+0000 7fcf50cb9640 1 --2- 172.21.15.84:0/198507869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c1593b0 0x7fcf3c00a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.745+0000 7fcf50cb9640 1 -- 172.21.15.84:0/198507869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3c00b050 msgr2=0x7fcf3c00d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:57.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.745+0000 7fcf514ba640 1 --2- 172.21.15.84:0/198507869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3c00b050 0x7fcf3c00d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.745+0000 7fcf50cb9640 1 --2- 172.21.15.84:0/198507869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3c00b050 0x7fcf3c00d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.745+0000 7fcf50cb9640 1 -- 172.21.15.84:0/198507869 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c003440 con 0x7fcf3c003a90 2023-10-16T10:01:57.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.746+0000 7fcf50cb9640 1 --2- 172.21.15.84:0/198507869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf3c003a90 0x7fcf3c003eb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009ac0 tx=0x7fcf3c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1529bff9fb4ec3a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:57.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.747+0000 7fcf4b7fe640 1 -- 172.21.15.84:0/198507869 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4002c710 con 0x7fcf3c003a90 2023-10-16T10:01:57.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.747+0000 7fcf4b7fe640 1 -- 172.21.15.84:0/198507869 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4002c870 con 0x7fcf3c003a90 2023-10-16T10:01:57.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.747+0000 7fcf4b7fe640 1 -- 172.21.15.84:0/198507869 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4002cc00 con 0x7fcf3c003a90 2023-10-16T10:01:57.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.747+0000 7fcf52f1d640 1 -- 172.21.15.84:0/198507869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf3c003a90 msgr2=0x7fcf3c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.747+0000 7fcf52f1d640 1 --2- 172.21.15.84:0/198507869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf3c003a90 0x7fcf3c003eb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009ac0 tx=0x7fcf3c0036f0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.747+0000 7fcf514ba640 1 -- 172.21.15.84:0/198507869 reap_dead start 2023-10-16T10:01:57.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.747+0000 7fcf52f1d640 1 -- 172.21.15.84:0/198507869 shutdown_connections 2023-10-16T10:01:57.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.747+0000 7fcf52f1d640 1 -- 172.21.15.84:0/198507869 >> 172.21.15.84:0/198507869 conn(0x7fcf4c0cb410 msgr2=0x7fcf4c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:57.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.747+0000 7fcf52f1d640 1 -- 172.21.15.84:0/198507869 shutdown_connections 2023-10-16T10:01:57.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.747+0000 7fcf52f1d640 1 -- 172.21.15.84:0/198507869 wait complete. 2023-10-16T10:01:57.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.749+0000 7fcf52f1d640 1 Processor -- start 2023-10-16T10:01:57.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.749+0000 7fcf52f1d640 1 -- start start 2023-10-16T10:01:57.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.750+0000 7fcf52f1d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3c003a90 0x7fcf4c0c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.750+0000 7fcf52f1d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf4c0b1530 0x7fcf4c1f1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.750+0000 7fcf52f1d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c1e2cf0 0x7fcf4c1e3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.750+0000 7fcf52f1d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c1e36b0 con 0x7fcf4c0b1530 2023-10-16T10:01:57.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.750+0000 7fcf52f1d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c0b4390 con 0x7fcf3c003a90 2023-10-16T10:01:57.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.750+0000 7fcf52f1d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c1f1a40 con 0x7fcf4c1e2cf0 2023-10-16T10:01:57.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.750+0000 7fcf4bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf4c0b1530 0x7fcf4c1f1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.750+0000 7fcf4bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf4c0b1530 0x7fcf4c1f1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43608/0 (socket says 172.21.15.84:43608) 2023-10-16T10:01:57.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.750+0000 7fcf4bfff640 1 -- 172.21.15.84:0/3346615176 learned_addr learned my addr 172.21.15.84:0/3346615176 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:57.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.751+0000 7fcf50cb9640 1 --2- 172.21.15.84:0/3346615176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3c003a90 0x7fcf4c0c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.751+0000 7fcf4bfff640 1 -- 172.21.15.84:0/3346615176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c1e2cf0 msgr2=0x7fcf4c1e3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.751+0000 7fcf4bfff640 1 --2- 172.21.15.84:0/3346615176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c1e2cf0 0x7fcf4c1e3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.751+0000 7fcf4bfff640 1 -- 172.21.15.84:0/3346615176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3c003a90 msgr2=0x7fcf4c0c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.751+0000 7fcf4bfff640 1 --2- 172.21.15.84:0/3346615176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3c003a90 0x7fcf4c0c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.751+0000 7fcf4bfff640 1 -- 172.21.15.84:0/3346615176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c0b19b0 con 0x7fcf4c0b1530 2023-10-16T10:01:57.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.751+0000 7fcf50cb9640 1 --2- 172.21.15.84:0/3346615176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3c003a90 0x7fcf4c0c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:57.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.751+0000 7fcf4bfff640 1 --2- 172.21.15.84:0/3346615176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf4c0b1530 0x7fcf4c1f1500 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800c500 tx=0x7fcf3800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:57.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.751+0000 7fcf497fa640 1 -- 172.21.15.84:0/3346615176 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf380052d0 con 0x7fcf4c0b1530 2023-10-16T10:01:57.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.752+0000 7fcf497fa640 1 -- 172.21.15.84:0/3346615176 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf38005430 con 0x7fcf4c0b1530 2023-10-16T10:01:57.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.752+0000 7fcf497fa640 1 -- 172.21.15.84:0/3346615176 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38012850 con 0x7fcf4c0b1530 2023-10-16T10:01:57.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.752+0000 7fcf52f1d640 1 -- 172.21.15.84:0/3346615176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf4c0b3af0 con 0x7fcf4c0b1530 2023-10-16T10:01:57.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.752+0000 7fcf52f1d640 1 -- 172.21.15.84:0/3346615176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf4c0b40b0 con 0x7fcf4c0b1530 2023-10-16T10:01:57.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.753+0000 7fcf52f1d640 1 -- 172.21.15.84:0/3346615176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf4c04e160 con 0x7fcf4c0b1530 2023-10-16T10:01:57.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.754+0000 7fcf497fa640 1 -- 172.21.15.84:0/3346615176 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fcf380129b0 con 0x7fcf4c0b1530 2023-10-16T10:01:57.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.755+0000 7fcf497fa640 1 --2- 172.21.15.84:0/3346615176 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fcf3006acc0 0x7fcf3006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.755+0000 7fcf497fa640 1 -- 172.21.15.84:0/3346615176 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6718+0+0 (secure 0 0 0) 0x7fcf380891d0 con 0x7fcf4c0b1530 2023-10-16T10:01:57.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.755+0000 7fcf50cb9640 1 --2- 172.21.15.84:0/3346615176 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fcf3006acc0 0x7fcf3006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.756+0000 7fcf50cb9640 1 --2- 172.21.15.84:0/3346615176 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fcf3006acc0 0x7fcf3006d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcf40036000 tx=0x7fcf40030000 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:57.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.759+0000 7fcf497fa640 1 -- 172.21.15.84:0/3346615176 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcf380541d0 con 0x7fcf4c0b1530 2023-10-16T10:01:57.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:57.912+0000 7fcf497fa640 1 -- 172.21.15.84:0/3346615176 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 20) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fcf380545a0 con 0x7fcf4c0b1530 2023-10-16T10:01:58.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.007+0000 7fcf52f1d640 1 -- 172.21.15.84:0/3346615176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf4c158e40 con 0x7fcf4c0b1530 2023-10-16T10:01:58.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.009+0000 7fcf497fa640 1 -- 172.21.15.84:0/3346615176 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fcf38071490 con 0x7fcf4c0b1530 2023-10-16T10:01:58.011 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:01:58.011 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3662158203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3662158203},{"type":"v1","addr":"172.21.15.157:6835","nonce":3662158203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5091,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2502120537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2502120537},{"type":"v1","addr":"172.21.15.84:6836","nonce":2502120537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2715651376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2715651376},{"type":"v1","addr":"172.21.15.84:6837","nonce":2715651376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:50:12.380180+0000","modified":"2023-10-16T09:50:15.438597+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6833/62459899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":62459899},{"type":"v1","addr":"172.21.15.157:6833","nonce":62459899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T10:01:58.011 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 17 2023-10-16T10:01:58.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.012+0000 7fcf52f1d640 1 -- 172.21.15.84:0/3346615176 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fcf3006acc0 msgr2=0x7fcf3006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.012+0000 7fcf52f1d640 1 --2- 172.21.15.84:0/3346615176 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fcf3006acc0 0x7fcf3006d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcf40036000 tx=0x7fcf40030000 comp rx=0 tx=0).stop 2023-10-16T10:01:58.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.012+0000 7fcf52f1d640 1 -- 172.21.15.84:0/3346615176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf4c0b1530 msgr2=0x7fcf4c1f1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.012+0000 7fcf52f1d640 1 --2- 172.21.15.84:0/3346615176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf4c0b1530 0x7fcf4c1f1500 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800c500 tx=0x7fcf3800c9c0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.012+0000 7fcf52f1d640 1 -- 172.21.15.84:0/3346615176 shutdown_connections 2023-10-16T10:01:58.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.012+0000 7fcf52f1d640 1 --2- 172.21.15.84:0/3346615176 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fcf3006acc0 0x7fcf3006d180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.012+0000 7fcf52f1d640 1 --2- 172.21.15.84:0/3346615176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf4c1e2cf0 0x7fcf4c1e3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.012+0000 7fcf52f1d640 1 --2- 172.21.15.84:0/3346615176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf4c0b1530 0x7fcf4c1f1500 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.012+0000 7fcf52f1d640 1 --2- 172.21.15.84:0/3346615176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3c003a90 0x7fcf4c0c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.012+0000 7fcf52f1d640 1 -- 172.21.15.84:0/3346615176 >> 172.21.15.84:0/3346615176 conn(0x7fcf4c0cb410 msgr2=0x7fcf4c0c9260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:58.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.012+0000 7fcf52f1d640 1 -- 172.21.15.84:0/3346615176 shutdown_connections 2023-10-16T10:01:58.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.012+0000 7fcf52f1d640 1 -- 172.21.15.84:0/3346615176 wait complete. 2023-10-16T10:01:58.033 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:01:58.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.236+0000 7fc68a901640 1 Processor -- start 2023-10-16T10:01:58.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.236+0000 7fc68a901640 1 -- start start 2023-10-16T10:01:58.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.238+0000 7fc68a901640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc68415ae80 0x7fc68415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.238+0000 7fc68a901640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc68415ed60 con 0x7fc6840ce390 2023-10-16T10:01:58.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.238+0000 7fc68a901640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6840707e0 con 0x7fc68415aaa0 2023-10-16T10:01:58.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.238+0000 7fc68a901640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68406b810 con 0x7fc68415ae80 2023-10-16T10:01:58.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.238+0000 7fc688e9e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc6840ce390 0x7fc684070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:50290/0 (socket says 172.21.15.84:50290) 2023-10-16T10:01:58.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.238+0000 7fc688e9e640 1 -- 172.21.15.84:0/3084577240 learned_addr learned my addr 172.21.15.84:0/3084577240 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:58.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.238+0000 7fc67bfff640 1 --2- 172.21.15.84:0/3084577240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc68415ae80 0x7fc68415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.239+0000 7fc6837fe640 1 -- 172.21.15.84:0/3084577240 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362283289 0 0) 0x7fc68415ed60 con 0x7fc6840ce390 2023-10-16T10:01:58.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.239+0000 7fc6837fe640 1 -- 172.21.15.84:0/3084577240 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc670003520 con 0x7fc6840ce390 2023-10-16T10:01:58.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.239+0000 7fc6837fe640 1 -- 172.21.15.84:0/3084577240 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2893419543 0 0) 0x7fc670003520 con 0x7fc6840ce390 2023-10-16T10:01:58.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.239+0000 7fc6837fe640 1 -- 172.21.15.84:0/3084577240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc68415ae80 msgr2=0x7fc68415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:58.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.239+0000 7fc6837fe640 1 --2- 172.21.15.84:0/3084577240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc68415ae80 0x7fc68415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.239+0000 7fc6837fe640 1 -- 172.21.15.84:0/3084577240 >> v1:172.21.15.157:6789/0 conn(0x7fc68415aaa0 legacy=0x7fc6840cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.239+0000 7fc6837fe640 1 -- 172.21.15.84:0/3084577240 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68415ef90 con 0x7fc6840ce390 2023-10-16T10:01:58.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.239+0000 7fc6837fe640 1 -- 172.21.15.84:0/3084577240 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc67c0019f0 con 0x7fc6840ce390 2023-10-16T10:01:58.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.239+0000 7fc6837fe640 1 -- 172.21.15.84:0/3084577240 >> v1:172.21.15.84:6789/0 conn(0x7fc6840ce390 legacy=0x7fc684070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.239+0000 7fc6837fe640 1 --2- 172.21.15.84:0/3084577240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6700039a0 0x7fc670003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.239+0000 7fc6837fe640 1 --2- 172.21.15.84:0/3084577240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc68415ae80 0x7fc6700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.239+0000 7fc6837fe640 1 --2- 172.21.15.84:0/3084577240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc670004d00 0x7fc670005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.239+0000 7fc6837fe640 1 -- 172.21.15.84:0/3084577240 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68406b810 con 0x7fc6700039a0 2023-10-16T10:01:58.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.239+0000 7fc6837fe640 1 -- 172.21.15.84:0/3084577240 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67000ba90 con 0x7fc670004d00 2023-10-16T10:01:58.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.239+0000 7fc6837fe640 1 -- 172.21.15.84:0/3084577240 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67000bc50 con 0x7fc68415ae80 2023-10-16T10:01:58.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.239+0000 7fc683fff640 1 --2- 172.21.15.84:0/3084577240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6700039a0 0x7fc670003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.240+0000 7fc67bfff640 1 --2- 172.21.15.84:0/3084577240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc68415ae80 0x7fc6700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.240+0000 7fc688e9e640 1 --2- 172.21.15.84:0/3084577240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc670004d00 0x7fc670005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.240+0000 7fc683fff640 1 -- 172.21.15.84:0/3084577240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc68415ae80 msgr2=0x7fc6700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:58.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.240+0000 7fc683fff640 1 --2- 172.21.15.84:0/3084577240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc68415ae80 0x7fc6700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.240+0000 7fc683fff640 1 -- 172.21.15.84:0/3084577240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc670004d00 msgr2=0x7fc670005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:58.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.240+0000 7fc683fff640 1 --2- 172.21.15.84:0/3084577240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc670004d00 0x7fc670005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.240+0000 7fc683fff640 1 -- 172.21.15.84:0/3084577240 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc670003520 con 0x7fc6700039a0 2023-10-16T10:01:58.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.240+0000 7fc67bfff640 1 --2- 172.21.15.84:0/3084577240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc68415ae80 0x7fc6700047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:58.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.240+0000 7fc688e9e640 1 --2- 172.21.15.84:0/3084577240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc670004d00 0x7fc670005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:58.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.241+0000 7fc683fff640 1 --2- 172.21.15.84:0/3084577240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6700039a0 0x7fc670003dc0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc674001720 tx=0x7fc6700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd744c7fc6df4c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:58.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.242+0000 7fc6837fe640 1 -- 172.21.15.84:0/3084577240 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6740052d0 con 0x7fc6700039a0 2023-10-16T10:01:58.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.242+0000 7fc6837fe640 1 -- 172.21.15.84:0/3084577240 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67402e050 con 0x7fc6700039a0 2023-10-16T10:01:58.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.242+0000 7fc6837fe640 1 -- 172.21.15.84:0/3084577240 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6740035f0 con 0x7fc6700039a0 2023-10-16T10:01:58.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.242+0000 7fc68a901640 1 -- 172.21.15.84:0/3084577240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6700039a0 msgr2=0x7fc670003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.242+0000 7fc68a901640 1 --2- 172.21.15.84:0/3084577240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6700039a0 0x7fc670003dc0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc674001720 tx=0x7fc6700057f0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.242+0000 7fc688e9e640 1 -- 172.21.15.84:0/3084577240 reap_dead start 2023-10-16T10:01:58.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.242+0000 7fc68a901640 1 -- 172.21.15.84:0/3084577240 shutdown_connections 2023-10-16T10:01:58.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.242+0000 7fc68a901640 1 -- 172.21.15.84:0/3084577240 >> 172.21.15.84:0/3084577240 conn(0x7fc6840cb410 msgr2=0x7fc6840cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:58.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.242+0000 7fc68a901640 1 -- 172.21.15.84:0/3084577240 shutdown_connections 2023-10-16T10:01:58.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.242+0000 7fc68a901640 1 -- 172.21.15.84:0/3084577240 wait complete. 2023-10-16T10:01:58.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.244+0000 7fc68a901640 1 Processor -- start 2023-10-16T10:01:58.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.245+0000 7fc68a901640 1 -- start start 2023-10-16T10:01:58.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.245+0000 7fc68a901640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6700039a0 0x7fc6840b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.245+0000 7fc68a901640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc684155aa0 0x7fc6840c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.246+0000 7fc68a901640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6840c4bd0 0x7fc68415c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.246+0000 7fc68a901640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840c5050 con 0x7fc6840c4bd0 2023-10-16T10:01:58.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.246+0000 7fc68a901640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840b3400 con 0x7fc6700039a0 2023-10-16T10:01:58.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.246+0000 7fc68a901640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68415b380 con 0x7fc684155aa0 2023-10-16T10:01:58.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.246+0000 7fc688e9e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6840c4bd0 0x7fc68415c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.246+0000 7fc688e9e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6840c4bd0 0x7fc68415c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43626/0 (socket says 172.21.15.84:43626) 2023-10-16T10:01:58.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.246+0000 7fc688e9e640 1 -- 172.21.15.84:0/2669848417 learned_addr learned my addr 172.21.15.84:0/2669848417 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:58.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.246+0000 7fc683fff640 1 --2- 172.21.15.84:0/2669848417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6700039a0 0x7fc6840b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.246+0000 7fc67bfff640 1 --2- 172.21.15.84:0/2669848417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc684155aa0 0x7fc6840c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.246+0000 7fc688e9e640 1 -- 172.21.15.84:0/2669848417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc684155aa0 msgr2=0x7fc6840c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.246+0000 7fc688e9e640 1 --2- 172.21.15.84:0/2669848417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc684155aa0 0x7fc6840c4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.246+0000 7fc688e9e640 1 -- 172.21.15.84:0/2669848417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6700039a0 msgr2=0x7fc6840b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.246+0000 7fc688e9e640 1 --2- 172.21.15.84:0/2669848417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6700039a0 0x7fc6840b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.246+0000 7fc688e9e640 1 -- 172.21.15.84:0/2669848417 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68415b660 con 0x7fc6840c4bd0 2023-10-16T10:01:58.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.246+0000 7fc683fff640 1 --2- 172.21.15.84:0/2669848417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6700039a0 0x7fc6840b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:58.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.247+0000 7fc688e9e640 1 --2- 172.21.15.84:0/2669848417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6840c4bd0 0x7fc68415c420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc6840ce2e0 tx=0x7fc67c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:58.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.247+0000 7fc6817fa640 1 -- 172.21.15.84:0/2669848417 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c004d60 con 0x7fc6840c4bd0 2023-10-16T10:01:58.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.247+0000 7fc6817fa640 1 -- 172.21.15.84:0/2669848417 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67c00bb80 con 0x7fc6840c4bd0 2023-10-16T10:01:58.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.247+0000 7fc68a901640 1 -- 172.21.15.84:0/2669848417 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68415b940 con 0x7fc6840c4bd0 2023-10-16T10:01:58.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.247+0000 7fc6817fa640 1 -- 172.21.15.84:0/2669848417 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c007110 con 0x7fc6840c4bd0 2023-10-16T10:01:58.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.247+0000 7fc68a901640 1 -- 172.21.15.84:0/2669848417 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68415bf00 con 0x7fc6840c4bd0 2023-10-16T10:01:58.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.248+0000 7fc68a901640 1 -- 172.21.15.84:0/2669848417 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc68404e160 con 0x7fc6840c4bd0 2023-10-16T10:01:58.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.249+0000 7fc6817fa640 1 -- 172.21.15.84:0/2669848417 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fc67c007270 con 0x7fc6840c4bd0 2023-10-16T10:01:58.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.250+0000 7fc6817fa640 1 --2- 172.21.15.84:0/2669848417 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fc66406af00 0x7fc66406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.250+0000 7fc6817fa640 1 -- 172.21.15.84:0/2669848417 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6718+0+0 (secure 0 0 0) 0x7fc67c089e50 con 0x7fc6840c4bd0 2023-10-16T10:01:58.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.250+0000 7fc683fff640 1 --2- 172.21.15.84:0/2669848417 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fc66406af00 0x7fc66406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.251+0000 7fc683fff640 1 --2- 172.21.15.84:0/2669848417 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fc66406af00 0x7fc66406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc674033000 tx=0x7fc674009030 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:58.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.255+0000 7fc6817fa640 1 -- 172.21.15.84:0/2669848417 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc67c054aa0 con 0x7fc6840c4bd0 2023-10-16T10:01:58.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.447+0000 7fc68a901640 1 -- 172.21.15.84:0/2669848417 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc68415e350 con 0x7fc6840c4bd0 2023-10-16T10:01:58.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.450+0000 7fc6817fa640 1 -- 172.21.15.84:0/2669848417 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+18060 (secure 0 0 0) 0x7fc67c0580c0 con 0x7fc6840c4bd0 2023-10-16T10:01:58.452 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:01:58.453 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":52,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:01:57.828586+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:50:10.297512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-16T09:50:11.394493+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":50,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2429213939":"2023-10-17T10:01:55.223514+0000","172.21.15.84:6833/3010246666":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/1060139177":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/3968394661":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/2493862491":"2023-10-17T10:01:55.223514+0000","172.21.15.84:6832/3010246666":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/1628587200":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/3763261264":"2023-10-17T10:01:55.223514+0000","172.21.15.84:0/848414450":"2023-10-17T10:01:55.223514+0000","172.21.15.84:6832/4265272960":"2023-10-17T10:01:55.223514+0000","172.21.15.84:0/3285111878":"2023-10-17T10:01:55.223514+0000","172.21.15.84:0/3734084949":"2023-10-17T10:01:55.223514+0000","172.21.15.84:0/982275204":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/1706229169":"2023-10-17T10:01:55.223514+0000","172.21.15.84:6833/4265272960":"2023-10-17T10:01:55.223514+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":7,"length":1},{"begin":11,"length":1},{"begin":14,"length":1},{"begin":20,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":48,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":65,"length":1},{"begin":67,"length":1},{"begin":72,"length":1},{"begin":76,"length":1},{"begin":84,"length":2},{"begin":92,"length":1},{"begin":100,"length":1},{"begin":103,"length":1},{"begin":106,"length":1},{"begin":200,"length":22}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:01:58.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.453+0000 7fc68a901640 1 -- 172.21.15.84:0/2669848417 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fc66406af00 msgr2=0x7fc66406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.453+0000 7fc68a901640 1 --2- 172.21.15.84:0/2669848417 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fc66406af00 0x7fc66406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc674033000 tx=0x7fc674009030 comp rx=0 tx=0).stop 2023-10-16T10:01:58.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.453+0000 7fc68a901640 1 -- 172.21.15.84:0/2669848417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6840c4bd0 msgr2=0x7fc68415c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.453+0000 7fc68a901640 1 --2- 172.21.15.84:0/2669848417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6840c4bd0 0x7fc68415c420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc6840ce2e0 tx=0x7fc67c00b4a0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.453+0000 7fc68a901640 1 -- 172.21.15.84:0/2669848417 shutdown_connections 2023-10-16T10:01:58.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.453+0000 7fc68a901640 1 --2- 172.21.15.84:0/2669848417 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fc66406af00 0x7fc66406d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.453+0000 7fc68a901640 1 --2- 172.21.15.84:0/2669848417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6840c4bd0 0x7fc68415c420 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.453+0000 7fc68a901640 1 --2- 172.21.15.84:0/2669848417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc684155aa0 0x7fc6840c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.453+0000 7fc68a901640 1 --2- 172.21.15.84:0/2669848417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6700039a0 0x7fc6840b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.453+0000 7fc68a901640 1 -- 172.21.15.84:0/2669848417 >> 172.21.15.84:0/2669848417 conn(0x7fc6840cb410 msgr2=0x7fc68406f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:58.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.453+0000 7fc68a901640 1 -- 172.21.15.84:0/2669848417 shutdown_connections 2023-10-16T10:01:58.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.454+0000 7fc68a901640 1 -- 172.21.15.84:0/2669848417 wait complete. 2023-10-16T10:01:58.474 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T10:01:58.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.679+0000 7f24aa42e640 1 Processor -- start 2023-10-16T10:01:58.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.679+0000 7f24aa42e640 1 -- start start 2023-10-16T10:01:58.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.681+0000 7f24aa42e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24a406c040 0x7f24a406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.681+0000 7f24aa42e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f24a40cda80 0x7f24a415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.681+0000 7f24aa42e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a406c960 con 0x7f24a40ad970 2023-10-16T10:01:58.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.681+0000 7f24aa42e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a415b9d0 con 0x7f24a40cda80 2023-10-16T10:01:58.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.681+0000 7f24aa42e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a415bcb0 con 0x7f24a406c040 2023-10-16T10:01:58.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.681+0000 7f24a8c2b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f24a40ad970 0x7f24a415e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:50302/0 (socket says 172.21.15.84:50302) 2023-10-16T10:01:58.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.681+0000 7f24a8c2b640 1 -- 172.21.15.84:0/3602830737 learned_addr learned my addr 172.21.15.84:0/3602830737 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:01:58.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.681+0000 7f24a942c640 1 --2- 172.21.15.84:0/3602830737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24a406c040 0x7f24a406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.681+0000 7f249bfff640 1 -- 172.21.15.84:0/3602830737 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 337948560 0 0) 0x7f24a406c960 con 0x7f24a40ad970 2023-10-16T10:01:58.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.682+0000 7f249bfff640 1 -- 172.21.15.84:0/3602830737 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2490003520 con 0x7f24a40ad970 2023-10-16T10:01:58.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.682+0000 7f24a9c2d640 1 --2- 172.21.15.84:0/3602830737 >> v2:172.21.15.157:3300/0 conn(0x7f24a40cda80 0x7f24a415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.682+0000 7f249bfff640 1 -- 172.21.15.84:0/3602830737 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3511294910 0 0) 0x7f2490003520 con 0x7f24a40ad970 2023-10-16T10:01:58.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.682+0000 7f249bfff640 1 -- 172.21.15.84:0/3602830737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24a406c040 msgr2=0x7f24a406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:58.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.682+0000 7f249bfff640 1 --2- 172.21.15.84:0/3602830737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24a406c040 0x7f24a406c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.682+0000 7f249bfff640 1 -- 172.21.15.84:0/3602830737 >> v2:172.21.15.157:3300/0 conn(0x7f24a40cda80 msgr2=0x7f24a415ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:58.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.682+0000 7f249bfff640 1 --2- 172.21.15.84:0/3602830737 >> v2:172.21.15.157:3300/0 conn(0x7f24a40cda80 0x7f24a415ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.682+0000 7f249bfff640 1 -- 172.21.15.84:0/3602830737 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a415bf90 con 0x7f24a40ad970 2023-10-16T10:01:58.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.682+0000 7f249bfff640 1 -- 172.21.15.84:0/3602830737 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f2494001670 con 0x7f24a40ad970 2023-10-16T10:01:58.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.682+0000 7f249bfff640 1 -- 172.21.15.84:0/3602830737 >> v1:172.21.15.84:6789/0 conn(0x7f24a40ad970 legacy=0x7f24a415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.682+0000 7f249bfff640 1 --2- 172.21.15.84:0/3602830737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2490003970 0x7f2490003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.682+0000 7f249bfff640 1 --2- 172.21.15.84:0/3602830737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24a406c040 0x7f24900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.682+0000 7f249bfff640 1 --2- 172.21.15.84:0/3602830737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2490004d00 0x7f2490005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.682+0000 7f249bfff640 1 -- 172.21.15.84:0/3602830737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a415b9d0 con 0x7f2490003970 2023-10-16T10:01:58.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.682+0000 7f249bfff640 1 -- 172.21.15.84:0/3602830737 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a415bcb0 con 0x7f2490004d00 2023-10-16T10:01:58.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.682+0000 7f249bfff640 1 -- 172.21.15.84:0/3602830737 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249000bab0 con 0x7f24a406c040 2023-10-16T10:01:58.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.682+0000 7f24a9c2d640 1 --2- 172.21.15.84:0/3602830737 >> v2:172.21.15.157:3300/0 conn(0x7f24a40cda80 0x7f24a415ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:58.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.682+0000 7f24a942c640 1 --2- 172.21.15.84:0/3602830737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2490003970 0x7f2490003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.683+0000 7f24a8c2b640 1 --2- 172.21.15.84:0/3602830737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24a406c040 0x7f24900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.683+0000 7f24a942c640 1 -- 172.21.15.84:0/3602830737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24a406c040 msgr2=0x7f24900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.683+0000 7f24a942c640 1 --2- 172.21.15.84:0/3602830737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24a406c040 0x7f24900047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.683+0000 7f24a942c640 1 -- 172.21.15.84:0/3602830737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2490004d00 msgr2=0x7f2490005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.683+0000 7f24a942c640 1 --2- 172.21.15.84:0/3602830737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2490004d00 0x7f2490005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.683+0000 7f24a942c640 1 -- 172.21.15.84:0/3602830737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2490003520 con 0x7f2490003970 2023-10-16T10:01:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.684+0000 7f24a942c640 1 --2- 172.21.15.84:0/3602830737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2490003970 0x7f2490003d90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009a90 tx=0x7f24900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc073a13ccd277f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.684+0000 7f249bfff640 1 -- 172.21.15.84:0/3602830737 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a002c680 con 0x7f2490003970 2023-10-16T10:01:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.684+0000 7f249bfff640 1 -- 172.21.15.84:0/3602830737 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a002c7e0 con 0x7f2490003970 2023-10-16T10:01:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.684+0000 7f249bfff640 1 -- 172.21.15.84:0/3602830737 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a002cb70 con 0x7f2490003970 2023-10-16T10:01:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.685+0000 7f24aa42e640 1 -- 172.21.15.84:0/3602830737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2490003970 msgr2=0x7f2490003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.685+0000 7f24aa42e640 1 --2- 172.21.15.84:0/3602830737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2490003970 0x7f2490003d90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009a90 tx=0x7f24900057f0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.685+0000 7f24a9c2d640 1 -- 172.21.15.84:0/3602830737 reap_dead start 2023-10-16T10:01:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.685+0000 7f24aa42e640 1 -- 172.21.15.84:0/3602830737 shutdown_connections 2023-10-16T10:01:58.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.685+0000 7f24aa42e640 1 -- 172.21.15.84:0/3602830737 >> 172.21.15.84:0/3602830737 conn(0x7f24a40ce6a0 msgr2=0x7f24a40ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:58.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.685+0000 7f24aa42e640 1 -- 172.21.15.84:0/3602830737 shutdown_connections 2023-10-16T10:01:58.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.685+0000 7f24aa42e640 1 -- 172.21.15.84:0/3602830737 wait complete. 2023-10-16T10:01:58.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.687+0000 7f24aa42e640 1 Processor -- start 2023-10-16T10:01:58.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.687+0000 7f24aa42e640 1 -- start start 2023-10-16T10:01:58.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.688+0000 7f24aa42e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2490003970 0x7f24a41eff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.688+0000 7f24aa42e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24a40ae030 0x7f24a40b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.688+0000 7f24aa42e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24a4145670 0x7f24a40b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.688+0000 7f24aa42e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a40ae5a0 con 0x7f24a40ae030 2023-10-16T10:01:58.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.688+0000 7f24aa42e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a41f4970 con 0x7f24a4145670 2023-10-16T10:01:58.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.688+0000 7f24aa42e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a41f3bb0 con 0x7f2490003970 2023-10-16T10:01:58.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.688+0000 7f24a8c2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24a40ae030 0x7f24a40b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.688+0000 7f24a8c2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24a40ae030 0x7f24a40b2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43646/0 (socket says 172.21.15.84:43646) 2023-10-16T10:01:58.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.688+0000 7f24a8c2b640 1 -- 172.21.15.84:0/2864406840 learned_addr learned my addr 172.21.15.84:0/2864406840 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:58.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.689+0000 7f24a8c2b640 1 -- 172.21.15.84:0/2864406840 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2490003970 msgr2=0x7f24a41eff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.689+0000 7f24a8c2b640 1 --2- 172.21.15.84:0/2864406840 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2490003970 0x7f24a41eff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.689+0000 7f24a8c2b640 1 -- 172.21.15.84:0/2864406840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24a4145670 msgr2=0x7f24a40b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.689+0000 7f24a8c2b640 1 --2- 172.21.15.84:0/2864406840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24a4145670 0x7f24a40b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.689+0000 7f24a8c2b640 1 -- 172.21.15.84:0/2864406840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a41f3e90 con 0x7f24a40ae030 2023-10-16T10:01:58.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.689+0000 7f24a8c2b640 1 --2- 172.21.15.84:0/2864406840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24a40ae030 0x7f24a40b2800 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f249400d750 tx=0x7f249400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:58.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.689+0000 7f2499ffb640 1 -- 172.21.15.84:0/2864406840 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24940074e0 con 0x7f24a40ae030 2023-10-16T10:01:58.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.689+0000 7f2499ffb640 1 -- 172.21.15.84:0/2864406840 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2494007640 con 0x7f24a40ae030 2023-10-16T10:01:58.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.689+0000 7f2499ffb640 1 -- 172.21.15.84:0/2864406840 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494004a00 con 0x7f24a40ae030 2023-10-16T10:01:58.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.689+0000 7f24aa42e640 1 -- 172.21.15.84:0/2864406840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a41f4170 con 0x7f24a40ae030 2023-10-16T10:01:58.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.689+0000 7f24aa42e640 1 -- 172.21.15.84:0/2864406840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a4145da0 con 0x7f24a40ae030 2023-10-16T10:01:58.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.690+0000 7f24aa42e640 1 -- 172.21.15.84:0/2864406840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a406b330 con 0x7f24a40ae030 2023-10-16T10:01:58.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.691+0000 7f2499ffb640 1 -- 172.21.15.84:0/2864406840 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f2494004b60 con 0x7f24a40ae030 2023-10-16T10:01:58.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.693+0000 7f2499ffb640 1 --2- 172.21.15.84:0/2864406840 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f248806af30 0x7f248806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.693+0000 7f2499ffb640 1 -- 172.21.15.84:0/2864406840 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f2494089450 con 0x7f24a40ae030 2023-10-16T10:01:58.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.693+0000 7f24a942c640 1 --2- 172.21.15.84:0/2864406840 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f248806af30 0x7f248806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.694+0000 7f24a942c640 1 --2- 172.21.15.84:0/2864406840 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f248806af30 0x7f248806d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f24a0036000 tx=0x7f24a002e000 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:58.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.697+0000 7f2499ffb640 1 -- 172.21.15.84:0/2864406840 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2494054010 con 0x7f24a40ae030 2023-10-16T10:01:58.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.928+0000 7f24aa42e640 1 -- 172.21.15.84:0/2864406840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f24a40c80c0 con 0x7f24a40ae030 2023-10-16T10:01:58.931 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-16T10:01:58.929+0000 7fca0c7ff640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T10:01:58.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.933+0000 7f2499ffb640 1 -- 172.21.15.84:0/2864406840 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f249400de60 con 0x7f24a40ae030 2023-10-16T10:01:58.934 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T10:01:58.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.936+0000 7f24aa42e640 1 -- 172.21.15.84:0/2864406840 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f248806af30 msgr2=0x7f248806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.936+0000 7f24aa42e640 1 --2- 172.21.15.84:0/2864406840 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f248806af30 0x7f248806d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f24a0036000 tx=0x7f24a002e000 comp rx=0 tx=0).stop 2023-10-16T10:01:58.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.936+0000 7f24aa42e640 1 -- 172.21.15.84:0/2864406840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24a40ae030 msgr2=0x7f24a40b2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.936+0000 7f24aa42e640 1 --2- 172.21.15.84:0/2864406840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24a40ae030 0x7f24a40b2800 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f249400d750 tx=0x7f249400dc10 comp rx=0 tx=0).stop 2023-10-16T10:01:58.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.936+0000 7f24aa42e640 1 -- 172.21.15.84:0/2864406840 shutdown_connections 2023-10-16T10:01:58.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.936+0000 7f24aa42e640 1 --2- 172.21.15.84:0/2864406840 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f248806af30 0x7f248806d3f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.936+0000 7f24aa42e640 1 --2- 172.21.15.84:0/2864406840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24a4145670 0x7f24a40b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.936+0000 7f24aa42e640 1 --2- 172.21.15.84:0/2864406840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24a40ae030 0x7f24a40b2800 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.936+0000 7f24aa42e640 1 --2- 172.21.15.84:0/2864406840 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2490003970 0x7f24a41eff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.936+0000 7f24aa42e640 1 -- 172.21.15.84:0/2864406840 >> 172.21.15.84:0/2864406840 conn(0x7f24a40ce6a0 msgr2=0x7f24a415d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:58.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.936+0000 7f24aa42e640 1 -- 172.21.15.84:0/2864406840 shutdown_connections 2023-10-16T10:01:58.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:58.936+0000 7f24aa42e640 1 -- 172.21.15.84:0/2864406840 wait complete. 2023-10-16T10:01:58.957 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T10:01:59.053 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T10:01:59.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.170+0000 7ff6a9085640 1 Processor -- start 2023-10-16T10:01:59.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.170+0000 7ff6a9085640 1 -- start start 2023-10-16T10:01:59.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.172+0000 7ff6a9085640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff6a415cd70 0x7ff6a40d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:59.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.172+0000 7ff6a9085640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff6a40b8480 0x7ff6a40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:59.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.172+0000 7ff6a9085640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a40d0be0 0x7ff6a40cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:59.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.172+0000 7ff6a9085640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a415d150 con 0x7ff6a40b8480 2023-10-16T10:01:59.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.172+0000 7ff6a9085640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a40d1180 con 0x7ff6a415cd70 2023-10-16T10:01:59.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.172+0000 7ff6a9085640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a40d1460 con 0x7ff6a40d0be0 2023-10-16T10:01:59.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.172+0000 7ff6a37fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff6a40b8480 0x7ff6a40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:59.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.172+0000 7ff6a37fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff6a40b8480 0x7ff6a40cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43654/0 (socket says 172.21.15.84:43654) 2023-10-16T10:01:59.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.172+0000 7ff6a37fe640 1 -- 172.21.15.84:0/695101078 learned_addr learned my addr 172.21.15.84:0/695101078 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:59.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.172+0000 7ff6a37fe640 1 -- 172.21.15.84:0/695101078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a40d0be0 msgr2=0x7ff6a40cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:59.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.172+0000 7ff6a37fe640 1 --2- 172.21.15.84:0/695101078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a40d0be0 0x7ff6a40cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.172+0000 7ff6a37fe640 1 -- 172.21.15.84:0/695101078 >> v2:172.21.15.157:3300/0 conn(0x7ff6a415cd70 msgr2=0x7ff6a40d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:59.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.173+0000 7ff6a37fe640 1 --2- 172.21.15.84:0/695101078 >> v2:172.21.15.157:3300/0 conn(0x7ff6a415cd70 0x7ff6a40d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.173+0000 7ff6a37fe640 1 -- 172.21.15.84:0/695101078 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a40d1740 con 0x7ff6a40b8480 2023-10-16T10:01:59.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.174+0000 7ff6a37fe640 1 --2- 172.21.15.84:0/695101078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a40b8480 0x7ff6a40cd7c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff6a40b56a0 tx=0x7ff68c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ff058fc393179a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:59.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.174+0000 7ff6a2ffd640 1 -- 172.21.15.84:0/695101078 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c031870 con 0x7ff6a40b8480 2023-10-16T10:01:59.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.174+0000 7ff6a2ffd640 1 -- 172.21.15.84:0/695101078 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68c0319d0 con 0x7ff6a40b8480 2023-10-16T10:01:59.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.174+0000 7ff6a2ffd640 1 -- 172.21.15.84:0/695101078 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c031d60 con 0x7ff6a40b8480 2023-10-16T10:01:59.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.174+0000 7ff6a9085640 1 -- 172.21.15.84:0/695101078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a40b8480 msgr2=0x7ff6a40cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:59.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.174+0000 7ff6a9085640 1 --2- 172.21.15.84:0/695101078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a40b8480 0x7ff6a40cd7c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff6a40b56a0 tx=0x7ff68c02b910 comp rx=0 tx=0).stop 2023-10-16T10:01:59.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.175+0000 7ff6a9085640 1 -- 172.21.15.84:0/695101078 shutdown_connections 2023-10-16T10:01:59.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.175+0000 7ff6a9085640 1 --2- 172.21.15.84:0/695101078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a40b8480 0x7ff6a40cd7c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.175+0000 7ff6a9085640 1 --2- 172.21.15.84:0/695101078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a40d0be0 0x7ff6a40cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.175+0000 7ff6a9085640 1 --2- 172.21.15.84:0/695101078 >> v2:172.21.15.157:3300/0 conn(0x7ff6a415cd70 0x7ff6a40d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.175+0000 7ff6a9085640 1 -- 172.21.15.84:0/695101078 >> 172.21.15.84:0/695101078 conn(0x7ff6a40cf570 msgr2=0x7ff6a40cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.175+0000 7ff6a9085640 1 -- 172.21.15.84:0/695101078 shutdown_connections 2023-10-16T10:01:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.175+0000 7ff6a9085640 1 -- 172.21.15.84:0/695101078 wait complete. 2023-10-16T10:01:59.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.176+0000 7ff6a9085640 1 Processor -- start 2023-10-16T10:01:59.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.177+0000 7ff6a9085640 1 -- start start 2023-10-16T10:01:59.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.177+0000 7ff6a9085640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6a40b8480 0x7ff6a4152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:59.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.177+0000 7ff6a9085640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a40d0be0 0x7ff6a40c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:59.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.177+0000 7ff6a9085640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a415cd70 0x7ff6a40ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:59.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.177+0000 7ff6a9085640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a4153310 con 0x7ff6a415cd70 2023-10-16T10:01:59.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.177+0000 7ff6a9085640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a415e020 con 0x7ff6a40b8480 2023-10-16T10:01:59.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.177+0000 7ff6a9085640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a415e2c0 con 0x7ff6a40d0be0 2023-10-16T10:01:59.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.178+0000 7ff6a8884640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a415cd70 0x7ff6a40ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:59.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.178+0000 7ff6a8884640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a415cd70 0x7ff6a40ca290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43658/0 (socket says 172.21.15.84:43658) 2023-10-16T10:01:59.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.178+0000 7ff6a8884640 1 -- 172.21.15.84:0/1430707492 learned_addr learned my addr 172.21.15.84:0/1430707492 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:01:59.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.178+0000 7ff6a3fff640 1 --2- 172.21.15.84:0/1430707492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6a40b8480 0x7ff6a4152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:59.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.178+0000 7ff6a37fe640 1 --2- 172.21.15.84:0/1430707492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a40d0be0 0x7ff6a40c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:59.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.178+0000 7ff6a8884640 1 -- 172.21.15.84:0/1430707492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a40d0be0 msgr2=0x7ff6a40c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:59.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.178+0000 7ff6a8884640 1 --2- 172.21.15.84:0/1430707492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a40d0be0 0x7ff6a40c9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.178+0000 7ff6a8884640 1 -- 172.21.15.84:0/1430707492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6a40b8480 msgr2=0x7ff6a4152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:59.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.178+0000 7ff6a8884640 1 --2- 172.21.15.84:0/1430707492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6a40b8480 0x7ff6a4152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.178+0000 7ff6a8884640 1 -- 172.21.15.84:0/1430707492 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c0094d0 con 0x7ff6a415cd70 2023-10-16T10:01:59.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.178+0000 7ff6a8884640 1 --2- 172.21.15.84:0/1430707492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a415cd70 0x7ff6a40ca290 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00b3e0 tx=0x7ff69c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:59.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.179+0000 7ff6a0ff9640 1 -- 172.21.15.84:0/1430707492 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c011830 con 0x7ff6a415cd70 2023-10-16T10:01:59.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.179+0000 7ff6a0ff9640 1 -- 172.21.15.84:0/1430707492 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c011990 con 0x7ff6a415cd70 2023-10-16T10:01:59.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.179+0000 7ff6a0ff9640 1 -- 172.21.15.84:0/1430707492 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c011d20 con 0x7ff6a415cd70 2023-10-16T10:01:59.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.179+0000 7ff6a9085640 1 -- 172.21.15.84:0/1430707492 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a415e880 con 0x7ff6a415cd70 2023-10-16T10:01:59.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.179+0000 7ff6a9085640 1 -- 172.21.15.84:0/1430707492 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a415eda0 con 0x7ff6a415cd70 2023-10-16T10:01:59.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.180+0000 7ff6a9085640 1 -- 172.21.15.84:0/1430707492 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a415ea70 con 0x7ff6a415cd70 2023-10-16T10:01:59.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.181+0000 7ff6a0ff9640 1 -- 172.21.15.84:0/1430707492 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) v1 ==== 89275+0+0 (secure 0 0 0) 0x7ff69c01a400 con 0x7ff6a415cd70 2023-10-16T10:01:59.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.182+0000 7ff6a0ff9640 1 --2- 172.21.15.84:0/1430707492 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7ff68406af00 0x7ff68406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:59.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.182+0000 7ff6a0ff9640 1 -- 172.21.15.84:0/1430707492 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6440+0+0 (secure 0 0 0) 0x7ff69c089b40 con 0x7ff6a415cd70 2023-10-16T10:01:59.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.182+0000 7ff6a3fff640 1 --2- 172.21.15.84:0/1430707492 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7ff68406af00 0x7ff68406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:59.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.183+0000 7ff6a3fff640 1 --2- 172.21.15.84:0/1430707492 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7ff68406af00 0x7ff68406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff694009f20 tx=0x7ff6940094f0 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:59.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.186+0000 7ff6a0ff9640 1 -- 172.21.15.84:0/1430707492 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff69c054820 con 0x7ff6a415cd70 2023-10-16T10:01:59.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.411+0000 7ff6a9085640 1 -- 172.21.15.84:0/1430707492 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff6a404e9b0 con 0x7ff6a415cd70 2023-10-16T10:01:59.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.946+0000 7ff6a0ff9640 1 -- 172.21.15.84:0/1430707492 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff69c057e40 con 0x7ff6a415cd70 2023-10-16T10:01:59.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.950+0000 7ff6a9085640 1 -- 172.21.15.84:0/1430707492 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7ff68406af00 msgr2=0x7ff68406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:59.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.950+0000 7ff6a9085640 1 --2- 172.21.15.84:0/1430707492 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7ff68406af00 0x7ff68406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff694009f20 tx=0x7ff6940094f0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.950+0000 7ff6a9085640 1 -- 172.21.15.84:0/1430707492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a415cd70 msgr2=0x7ff6a40ca290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:59.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.950+0000 7ff6a9085640 1 --2- 172.21.15.84:0/1430707492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a415cd70 0x7ff6a40ca290 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00b3e0 tx=0x7ff69c009f90 comp rx=0 tx=0).stop 2023-10-16T10:01:59.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.950+0000 7ff6a9085640 1 -- 172.21.15.84:0/1430707492 shutdown_connections 2023-10-16T10:01:59.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.950+0000 7ff6a9085640 1 --2- 172.21.15.84:0/1430707492 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7ff68406af00 0x7ff68406d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.950+0000 7ff6a9085640 1 --2- 172.21.15.84:0/1430707492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a415cd70 0x7ff6a40ca290 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.950+0000 7ff6a9085640 1 --2- 172.21.15.84:0/1430707492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a40d0be0 0x7ff6a40c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.950+0000 7ff6a9085640 1 --2- 172.21.15.84:0/1430707492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6a40b8480 0x7ff6a4152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.950+0000 7ff6a9085640 1 -- 172.21.15.84:0/1430707492 >> 172.21.15.84:0/1430707492 conn(0x7ff6a40cf570 msgr2=0x7ff6a40d2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:59.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.951+0000 7ff6a9085640 1 -- 172.21.15.84:0/1430707492 shutdown_connections 2023-10-16T10:01:59.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:01:59.952+0000 7ff6a9085640 1 -- 172.21.15.84:0/1430707492 wait complete. 2023-10-16T10:01:59.977 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T10:02:00.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.189+0000 7fa25b8b4640 1 Processor -- start 2023-10-16T10:02:00.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.190+0000 7fa25b8b4640 1 -- start start 2023-10-16T10:02:00.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.191+0000 7fa25b8b4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa25415ac10 0x7fa25406c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:00.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.191+0000 7fa25b8b4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa2540cac20 0x7fa2540b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:00.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.191+0000 7fa25b8b4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2540b0d00 con 0x7fa2540b0360 2023-10-16T10:02:00.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.191+0000 7fa25b8b4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa25415f800 con 0x7fa2540cac20 2023-10-16T10:02:00.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.191+0000 7fa25b8b4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25415b310 con 0x7fa25415ac10 2023-10-16T10:02:00.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.191+0000 7fa258e4f640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa2540b0360 0x7fa2540ca510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:50318/0 (socket says 172.21.15.84:50318) 2023-10-16T10:02:00.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.192+0000 7fa258e4f640 1 -- 172.21.15.84:0/2879531167 learned_addr learned my addr 172.21.15.84:0/2879531167 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:00.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.191+0000 7fa259e51640 1 --2- 172.21.15.84:0/2879531167 >> v2:172.21.15.84:3300/0 conn(0x7fa2540cac20 0x7fa2540b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:00.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.192+0000 7fa259650640 1 --2- 172.21.15.84:0/2879531167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa25415ac10 0x7fa25406c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:00.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.192+0000 7fa24bfff640 1 -- 172.21.15.84:0/2879531167 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914051385 0 0) 0x7fa2540b0d00 con 0x7fa2540b0360 2023-10-16T10:02:00.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.192+0000 7fa24bfff640 1 -- 172.21.15.84:0/2879531167 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa240003520 con 0x7fa2540b0360 2023-10-16T10:02:00.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.192+0000 7fa259e51640 1 -- 172.21.15.84:0/2879531167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa25415ac10 msgr2=0x7fa25406c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:00.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.192+0000 7fa259e51640 1 --2- 172.21.15.84:0/2879531167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa25415ac10 0x7fa25406c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.192+0000 7fa259e51640 1 -- 172.21.15.84:0/2879531167 >> v1:172.21.15.84:6789/0 conn(0x7fa2540b0360 legacy=0x7fa2540ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:00.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.192+0000 7fa259e51640 1 -- 172.21.15.84:0/2879531167 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25406f640 con 0x7fa2540cac20 2023-10-16T10:02:00.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.194+0000 7fa259e51640 1 --2- 172.21.15.84:0/2879531167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2540cac20 0x7fa2540b07c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009d90 tx=0x7fa24c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86d90e160d316a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:00.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.194+0000 7fa24bfff640 1 -- 172.21.15.84:0/2879531167 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c02cbc0 con 0x7fa2540cac20 2023-10-16T10:02:00.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.194+0000 7fa24bfff640 1 -- 172.21.15.84:0/2879531167 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24c02cd20 con 0x7fa2540cac20 2023-10-16T10:02:00.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.194+0000 7fa24bfff640 1 -- 172.21.15.84:0/2879531167 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c034660 con 0x7fa2540cac20 2023-10-16T10:02:00.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.194+0000 7fa25b8b4640 1 -- 172.21.15.84:0/2879531167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2540cac20 msgr2=0x7fa2540b07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:00.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.194+0000 7fa25b8b4640 1 --2- 172.21.15.84:0/2879531167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2540cac20 0x7fa2540b07c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009d90 tx=0x7fa24c02be10 comp rx=0 tx=0).stop 2023-10-16T10:02:00.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.194+0000 7fa25b8b4640 1 -- 172.21.15.84:0/2879531167 shutdown_connections 2023-10-16T10:02:00.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.194+0000 7fa25b8b4640 1 --2- 172.21.15.84:0/2879531167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2540cac20 0x7fa2540b07c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.194+0000 7fa25b8b4640 1 --2- 172.21.15.84:0/2879531167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa25415ac10 0x7fa25406c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.194+0000 7fa25b8b4640 1 -- 172.21.15.84:0/2879531167 >> 172.21.15.84:0/2879531167 conn(0x7fa2540cb360 msgr2=0x7fa2540cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:00.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.195+0000 7fa25b8b4640 1 -- 172.21.15.84:0/2879531167 shutdown_connections 2023-10-16T10:02:00.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.195+0000 7fa25b8b4640 1 -- 172.21.15.84:0/2879531167 wait complete. 2023-10-16T10:02:00.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.196+0000 7fa25b8b4640 1 Processor -- start 2023-10-16T10:02:00.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.196+0000 7fa25b8b4640 1 -- start start 2023-10-16T10:02:00.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.197+0000 7fa25b8b4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2540b0360 0x7fa2540b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:00.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.197+0000 7fa25b8b4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2540cac20 0x7fa2540c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:00.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.197+0000 7fa25b8b4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa25415ac10 0x7fa254157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:00.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.197+0000 7fa25b8b4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2540b1ae0 con 0x7fa2540cac20 2023-10-16T10:02:00.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.197+0000 7fa25b8b4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2540b1dc0 con 0x7fa25415ac10 2023-10-16T10:02:00.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.197+0000 7fa25b8b4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2540b20a0 con 0x7fa2540b0360 2023-10-16T10:02:00.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.198+0000 7fa258e4f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2540cac20 0x7fa2540c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:00.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.198+0000 7fa258e4f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2540cac20 0x7fa2540c6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43688/0 (socket says 172.21.15.84:43688) 2023-10-16T10:02:00.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.198+0000 7fa258e4f640 1 -- 172.21.15.84:0/1359787764 learned_addr learned my addr 172.21.15.84:0/1359787764 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:00.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.198+0000 7fa259650640 1 --2- 172.21.15.84:0/1359787764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2540b0360 0x7fa2540b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:00.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.198+0000 7fa259e51640 1 --2- 172.21.15.84:0/1359787764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa25415ac10 0x7fa254157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:00.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.198+0000 7fa258e4f640 1 -- 172.21.15.84:0/1359787764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2540b0360 msgr2=0x7fa2540b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:00.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.198+0000 7fa258e4f640 1 --2- 172.21.15.84:0/1359787764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2540b0360 0x7fa2540b1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.198+0000 7fa258e4f640 1 -- 172.21.15.84:0/1359787764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa25415ac10 msgr2=0x7fa254157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:00.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.198+0000 7fa258e4f640 1 --2- 172.21.15.84:0/1359787764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa25415ac10 0x7fa254157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.198+0000 7fa258e4f640 1 -- 172.21.15.84:0/1359787764 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24c009a70 con 0x7fa2540cac20 2023-10-16T10:02:00.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.198+0000 7fa258e4f640 1 --2- 172.21.15.84:0/1359787764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2540cac20 0x7fa2540c6a80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa24400c410 tx=0x7fa24400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:00.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.199+0000 7fa249ffb640 1 -- 172.21.15.84:0/1359787764 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa244005ce0 con 0x7fa2540cac20 2023-10-16T10:02:00.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.199+0000 7fa249ffb640 1 -- 172.21.15.84:0/1359787764 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa244005e40 con 0x7fa2540cac20 2023-10-16T10:02:00.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.199+0000 7fa249ffb640 1 -- 172.21.15.84:0/1359787764 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2440072f0 con 0x7fa2540cac20 2023-10-16T10:02:00.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.199+0000 7fa25b8b4640 1 -- 172.21.15.84:0/1359787764 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2540b2380 con 0x7fa2540cac20 2023-10-16T10:02:00.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.199+0000 7fa25b8b4640 1 -- 172.21.15.84:0/1359787764 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2540ce410 con 0x7fa2540cac20 2023-10-16T10:02:00.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.200+0000 7fa25b8b4640 1 -- 172.21.15.84:0/1359787764 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25404e160 con 0x7fa2540cac20 2023-10-16T10:02:00.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.201+0000 7fa249ffb640 1 -- 172.21.15.84:0/1359787764 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa244007510 con 0x7fa2540cac20 2023-10-16T10:02:00.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.202+0000 7fa249ffb640 1 --2- 172.21.15.84:0/1359787764 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fa23806af90 0x7fa23806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:00.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.202+0000 7fa249ffb640 1 -- 172.21.15.84:0/1359787764 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fa244089100 con 0x7fa2540cac20 2023-10-16T10:02:00.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.202+0000 7fa259650640 1 --2- 172.21.15.84:0/1359787764 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fa23806af90 0x7fa23806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:00.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.202+0000 7fa259650640 1 --2- 172.21.15.84:0/1359787764 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fa23806af90 0x7fa23806d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa250006fd0 tx=0x7fa250008040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:00.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.206+0000 7fa249ffb640 1 -- 172.21.15.84:0/1359787764 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa244053e70 con 0x7fa2540cac20 2023-10-16T10:02:00.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.397+0000 7fa25b8b4640 1 -- 172.21.15.84:0/1359787764 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa25406c2e0 con 0x7fa2540cac20 2023-10-16T10:02:00.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.950+0000 7fa249ffb640 1 -- 172.21.15.84:0/1359787764 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v54) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa244057490 con 0x7fa2540cac20 2023-10-16T10:02:00.952 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-16T10:02:00.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.953+0000 7fa25b8b4640 1 -- 172.21.15.84:0/1359787764 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fa23806af90 msgr2=0x7fa23806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:00.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.953+0000 7fa25b8b4640 1 --2- 172.21.15.84:0/1359787764 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fa23806af90 0x7fa23806d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa250006fd0 tx=0x7fa250008040 comp rx=0 tx=0).stop 2023-10-16T10:02:00.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.953+0000 7fa25b8b4640 1 -- 172.21.15.84:0/1359787764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2540cac20 msgr2=0x7fa2540c6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:00.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.953+0000 7fa25b8b4640 1 --2- 172.21.15.84:0/1359787764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2540cac20 0x7fa2540c6a80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa24400c410 tx=0x7fa24400c8d0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.954+0000 7fa25b8b4640 1 -- 172.21.15.84:0/1359787764 shutdown_connections 2023-10-16T10:02:00.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.954+0000 7fa25b8b4640 1 --2- 172.21.15.84:0/1359787764 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fa23806af90 0x7fa23806d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.954+0000 7fa25b8b4640 1 --2- 172.21.15.84:0/1359787764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa25415ac10 0x7fa254157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.954+0000 7fa25b8b4640 1 --2- 172.21.15.84:0/1359787764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2540cac20 0x7fa2540c6a80 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.954+0000 7fa25b8b4640 1 --2- 172.21.15.84:0/1359787764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2540b0360 0x7fa2540b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.954+0000 7fa25b8b4640 1 -- 172.21.15.84:0/1359787764 >> 172.21.15.84:0/1359787764 conn(0x7fa2540cb360 msgr2=0x7fa25406c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:00.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.954+0000 7fa25b8b4640 1 -- 172.21.15.84:0/1359787764 shutdown_connections 2023-10-16T10:02:00.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:00.955+0000 7fa25b8b4640 1 -- 172.21.15.84:0/1359787764 wait complete. 2023-10-16T10:02:00.977 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T10:02:01.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.185+0000 7f6649b47640 1 Processor -- start 2023-10-16T10:02:01.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.185+0000 7f6649b47640 1 -- start start 2023-10-16T10:02:01.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.186+0000 7f6649b47640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f664406afc0 0x7f66441592e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.186+0000 7f6649b47640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66440c8f30 0x7f66440c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.186+0000 7f6649b47640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66440b0ae0 con 0x7f664406bea0 2023-10-16T10:02:01.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.186+0000 7f6649b47640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66440c9900 con 0x7f664406afc0 2023-10-16T10:02:01.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.186+0000 7f6649b47640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664406c6c0 con 0x7f66440c8f30 2023-10-16T10:02:01.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.187+0000 7f6642ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f664406bea0 0x7f66440b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:50334/0 (socket says 172.21.15.84:50334) 2023-10-16T10:02:01.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.187+0000 7f6642ffd640 1 -- 172.21.15.84:0/3718696594 learned_addr learned my addr 172.21.15.84:0/3718696594 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:01.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.187+0000 7f66437fe640 1 --2- 172.21.15.84:0/3718696594 >> v2:172.21.15.157:3300/0 conn(0x7f664406afc0 0x7f66441592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.187+0000 7f6643fff640 1 --2- 172.21.15.84:0/3718696594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66440c8f30 0x7f66440c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.187+0000 7f66427fc640 1 -- 172.21.15.84:0/3718696594 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946597696 0 0) 0x7f66440b0ae0 con 0x7f664406bea0 2023-10-16T10:02:01.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.187+0000 7f66427fc640 1 -- 172.21.15.84:0/3718696594 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6630003520 con 0x7f664406bea0 2023-10-16T10:02:01.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.188+0000 7f66427fc640 1 -- 172.21.15.84:0/3718696594 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2455706114 0 0) 0x7f6630003520 con 0x7f664406bea0 2023-10-16T10:02:01.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.188+0000 7f66427fc640 1 -- 172.21.15.84:0/3718696594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66440c8f30 msgr2=0x7f66440c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:01.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.188+0000 7f66427fc640 1 --2- 172.21.15.84:0/3718696594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66440c8f30 0x7f66440c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.188+0000 7f66427fc640 1 -- 172.21.15.84:0/3718696594 >> v2:172.21.15.157:3300/0 conn(0x7f664406afc0 msgr2=0x7f66441592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:01.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.188+0000 7f66427fc640 1 --2- 172.21.15.84:0/3718696594 >> v2:172.21.15.157:3300/0 conn(0x7f664406afc0 0x7f66441592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.188+0000 7f66427fc640 1 -- 172.21.15.84:0/3718696594 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66440cc860 con 0x7f664406bea0 2023-10-16T10:02:01.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.188+0000 7f66427fc640 1 -- 172.21.15.84:0/3718696594 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f6634001670 con 0x7f664406bea0 2023-10-16T10:02:01.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.188+0000 7f66427fc640 1 -- 172.21.15.84:0/3718696594 >> v1:172.21.15.84:6789/0 conn(0x7f664406bea0 legacy=0x7f66440b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.188+0000 7f66427fc640 1 --2- 172.21.15.84:0/3718696594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66440c8f30 0x7f66300039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.188+0000 7f66427fc640 1 --2- 172.21.15.84:0/3718696594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6630003f60 0x7f66300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.188+0000 7f66427fc640 1 --2- 172.21.15.84:0/3718696594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6630004d00 0x7f6630005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.188+0000 7f66427fc640 1 -- 172.21.15.84:0/3718696594 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66440c9900 con 0x7f6630004d00 2023-10-16T10:02:01.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.188+0000 7f66427fc640 1 -- 172.21.15.84:0/3718696594 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664406c6c0 con 0x7f6630003f60 2023-10-16T10:02:01.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.188+0000 7f66427fc640 1 -- 172.21.15.84:0/3718696594 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663000b9b0 con 0x7f66440c8f30 2023-10-16T10:02:01.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.188+0000 7f6643fff640 1 --2- 172.21.15.84:0/3718696594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6630004d00 0x7f6630005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.188+0000 7f6642ffd640 1 --2- 172.21.15.84:0/3718696594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6630003f60 0x7f66300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.188+0000 7f66437fe640 1 --2- 172.21.15.84:0/3718696594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66440c8f30 0x7f66300039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.188+0000 7f6643fff640 1 -- 172.21.15.84:0/3718696594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66440c8f30 msgr2=0x7f66300039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:01.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.189+0000 7f6643fff640 1 --2- 172.21.15.84:0/3718696594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66440c8f30 0x7f66300039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.189+0000 7f6643fff640 1 -- 172.21.15.84:0/3718696594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6630003f60 msgr2=0x7f66300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:01.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.189+0000 7f6643fff640 1 --2- 172.21.15.84:0/3718696594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6630003f60 0x7f66300047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.189+0000 7f6643fff640 1 -- 172.21.15.84:0/3718696594 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6630003520 con 0x7f6630004d00 2023-10-16T10:02:01.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.190+0000 7f6643fff640 1 --2- 172.21.15.84:0/3718696594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6630004d00 0x7f6630005180 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f663c009fc0 tx=0x7f66300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becfb7c9731a0da9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:01.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.190+0000 7f66427fc640 1 -- 172.21.15.84:0/3718696594 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c02cc20 con 0x7f6630004d00 2023-10-16T10:02:01.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.190+0000 7f66427fc640 1 -- 172.21.15.84:0/3718696594 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663c00f040 con 0x7f6630004d00 2023-10-16T10:02:01.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.190+0000 7f66427fc640 1 -- 172.21.15.84:0/3718696594 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c034440 con 0x7f6630004d00 2023-10-16T10:02:01.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.190+0000 7f6649b47640 1 -- 172.21.15.84:0/3718696594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6630004d00 msgr2=0x7f6630005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.190+0000 7f6649b47640 1 --2- 172.21.15.84:0/3718696594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6630004d00 0x7f6630005180 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f663c009fc0 tx=0x7f66300057f0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.190+0000 7f6643fff640 1 -- 172.21.15.84:0/3718696594 reap_dead start 2023-10-16T10:02:01.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.190+0000 7f6649b47640 1 -- 172.21.15.84:0/3718696594 shutdown_connections 2023-10-16T10:02:01.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.190+0000 7f6649b47640 1 -- 172.21.15.84:0/3718696594 >> 172.21.15.84:0/3718696594 conn(0x7f66440cb360 msgr2=0x7f66440cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:01.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.191+0000 7f6649b47640 1 -- 172.21.15.84:0/3718696594 shutdown_connections 2023-10-16T10:02:01.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.191+0000 7f6649b47640 1 -- 172.21.15.84:0/3718696594 wait complete. 2023-10-16T10:02:01.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.192+0000 7f6649b47640 1 Processor -- start 2023-10-16T10:02:01.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.192+0000 7f6649b47640 1 -- start start 2023-10-16T10:02:01.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.193+0000 7f6649b47640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6630004d00 0x7f66440c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.193+0000 7f6649b47640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66441406e0 0x7f66440c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.193+0000 7f6649b47640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6644140b60 0x7f66440c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.193+0000 7f6649b47640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66440c6a00 con 0x7f6630004d00 2023-10-16T10:02:01.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.193+0000 7f6649b47640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66440c6c50 con 0x7f66441406e0 2023-10-16T10:02:01.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.193+0000 7f6649b47640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66440c6f30 con 0x7f6644140b60 2023-10-16T10:02:01.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.194+0000 7f66437fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6630004d00 0x7f66440c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.194+0000 7f66437fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6630004d00 0x7f66440c4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43716/0 (socket says 172.21.15.84:43716) 2023-10-16T10:02:01.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.194+0000 7f66437fe640 1 -- 172.21.15.84:0/1548041158 learned_addr learned my addr 172.21.15.84:0/1548041158 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:01.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.194+0000 7f6642ffd640 1 --2- 172.21.15.84:0/1548041158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66441406e0 0x7f66440c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.194+0000 7f6643fff640 1 --2- 172.21.15.84:0/1548041158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6644140b60 0x7f66440c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.194+0000 7f66437fe640 1 -- 172.21.15.84:0/1548041158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6644140b60 msgr2=0x7f66440c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.194+0000 7f66437fe640 1 --2- 172.21.15.84:0/1548041158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6644140b60 0x7f66440c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.194+0000 7f66437fe640 1 -- 172.21.15.84:0/1548041158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66441406e0 msgr2=0x7f66440c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.194+0000 7f66437fe640 1 --2- 172.21.15.84:0/1548041158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66441406e0 0x7f66440c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.194+0000 7f66437fe640 1 -- 172.21.15.84:0/1548041158 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66440c7210 con 0x7f6630004d00 2023-10-16T10:02:01.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.194+0000 7f6643fff640 1 --2- 172.21.15.84:0/1548041158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6644140b60 0x7f66440c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:01.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.194+0000 7f66437fe640 1 --2- 172.21.15.84:0/1548041158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6630004d00 0x7f66440c4fa0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f662c0108e0 tx=0x7f662c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:01.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.195+0000 7f663bfff640 1 -- 172.21.15.84:0/1548041158 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c00eb60 con 0x7f6630004d00 2023-10-16T10:02:01.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.195+0000 7f663bfff640 1 -- 172.21.15.84:0/1548041158 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662c00ecc0 con 0x7f6630004d00 2023-10-16T10:02:01.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.195+0000 7f663bfff640 1 -- 172.21.15.84:0/1548041158 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c01c400 con 0x7f6630004d00 2023-10-16T10:02:01.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.195+0000 7f6649b47640 1 -- 172.21.15.84:0/1548041158 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66440c74f0 con 0x7f6630004d00 2023-10-16T10:02:01.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.195+0000 7f6649b47640 1 -- 172.21.15.84:0/1548041158 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6644155850 con 0x7f6630004d00 2023-10-16T10:02:01.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.196+0000 7f6649b47640 1 -- 172.21.15.84:0/1548041158 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f664404e160 con 0x7f6630004d00 2023-10-16T10:02:01.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.197+0000 7f663bfff640 1 -- 172.21.15.84:0/1548041158 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f662c01c560 con 0x7f6630004d00 2023-10-16T10:02:01.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.198+0000 7f663bfff640 1 --2- 172.21.15.84:0/1548041158 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f662406afc0 0x7f662406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.198+0000 7f663bfff640 1 -- 172.21.15.84:0/1548041158 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6002+0+0 (secure 0 0 0) 0x7f662c089f90 con 0x7f6630004d00 2023-10-16T10:02:01.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.198+0000 7f6642ffd640 1 --2- 172.21.15.84:0/1548041158 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f662406afc0 0x7f662406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.199+0000 7f6642ffd640 1 --2- 172.21.15.84:0/1548041158 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f662406afc0 0x7f662406d480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f66441405b0 tx=0x7f6634009040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:01.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.202+0000 7f663bfff640 1 -- 172.21.15.84:0/1548041158 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f662c054ea0 con 0x7f6630004d00 2023-10-16T10:02:01.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.407+0000 7f6649b47640 1 -- 172.21.15.84:0/1548041158 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f66440cb0c0 con 0x7f6630004d00 2023-10-16T10:02:01.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.953+0000 7f663bfff640 1 -- 172.21.15.84:0/1548041158 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v55) v1 ==== 169+0+0 (secure 0 0 0) 0x7f662c0584c0 con 0x7f6630004d00 2023-10-16T10:02:01.955 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-16T10:02:01.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.957+0000 7f6649b47640 1 -- 172.21.15.84:0/1548041158 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f662406afc0 msgr2=0x7f662406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.957+0000 7f6649b47640 1 --2- 172.21.15.84:0/1548041158 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f662406afc0 0x7f662406d480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f66441405b0 tx=0x7f6634009040 comp rx=0 tx=0).stop 2023-10-16T10:02:01.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.957+0000 7f6649b47640 1 -- 172.21.15.84:0/1548041158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6630004d00 msgr2=0x7f66440c4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.957+0000 7f6649b47640 1 --2- 172.21.15.84:0/1548041158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6630004d00 0x7f66440c4fa0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f662c0108e0 tx=0x7f662c010da0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.958+0000 7f6649b47640 1 -- 172.21.15.84:0/1548041158 shutdown_connections 2023-10-16T10:02:01.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.958+0000 7f6649b47640 1 --2- 172.21.15.84:0/1548041158 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f662406afc0 0x7f662406d480 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.958+0000 7f6649b47640 1 --2- 172.21.15.84:0/1548041158 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6644140b60 0x7f66440c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.958+0000 7f6649b47640 1 --2- 172.21.15.84:0/1548041158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66441406e0 0x7f66440c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.958+0000 7f6649b47640 1 --2- 172.21.15.84:0/1548041158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6630004d00 0x7f66440c4fa0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.958+0000 7f6649b47640 1 -- 172.21.15.84:0/1548041158 >> 172.21.15.84:0/1548041158 conn(0x7f66440cb360 msgr2=0x7f66440ae870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:01.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.961+0000 7f6649b47640 1 -- 172.21.15.84:0/1548041158 shutdown_connections 2023-10-16T10:02:01.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:01.961+0000 7f6649b47640 1 -- 172.21.15.84:0/1548041158 wait complete. 2023-10-16T10:02:01.987 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T10:02:02.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.176+0000 7f611565d640 1 Processor -- start 2023-10-16T10:02:02.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.176+0000 7f611565d640 1 -- start start 2023-10-16T10:02:02.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.177+0000 7f611565d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f611015ad70 0x7f6110069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.178+0000 7f611565d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61100caac0 0x7f611006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.178+0000 7f611565d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61100cb030 con 0x7f61100c9ca0 2023-10-16T10:02:02.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.178+0000 7f611565d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61100c91c0 con 0x7f611015ad70 2023-10-16T10:02:02.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.178+0000 7f611565d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61100c94a0 con 0x7f61100caac0 2023-10-16T10:02:02.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.178+0000 7f610effd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f611015ad70 0x7f6110069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.178+0000 7f610e7fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f61100c9ca0 0x7f61100ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:50336/0 (socket says 172.21.15.84:50336) 2023-10-16T10:02:02.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.178+0000 7f610e7fc640 1 -- 172.21.15.84:0/4106397449 learned_addr learned my addr 172.21.15.84:0/4106397449 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:02.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.178+0000 7f610f7fe640 1 --2- 172.21.15.84:0/4106397449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61100caac0 0x7f611006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.178+0000 7f610effd640 1 -- 172.21.15.84:0/4106397449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61100caac0 msgr2=0x7f611006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:02.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.178+0000 7f610effd640 1 --2- 172.21.15.84:0/4106397449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61100caac0 0x7f611006a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.178+0000 7f610effd640 1 -- 172.21.15.84:0/4106397449 >> v1:172.21.15.84:6789/0 conn(0x7f61100c9ca0 legacy=0x7f61100ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.178+0000 7f610effd640 1 -- 172.21.15.84:0/4106397449 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61100c9780 con 0x7f611015ad70 2023-10-16T10:02:02.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.178+0000 7f610dffb640 1 -- 172.21.15.84:0/4106397449 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204067933 0 0) 0x7f61100cb030 con 0x7f61100c9ca0 2023-10-16T10:02:02.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.178+0000 7f610f7fe640 1 --2- 172.21.15.84:0/4106397449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61100caac0 0x7f611006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:02.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.180+0000 7f610effd640 1 --2- 172.21.15.84:0/4106397449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f611015ad70 0x7f6110069da0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f61000097a0 tx=0x7f610002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbc626efed5ae6d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:02.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.180+0000 7f610dffb640 1 -- 172.21.15.84:0/4106397449 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610002d040 con 0x7f611015ad70 2023-10-16T10:02:02.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.180+0000 7f610dffb640 1 -- 172.21.15.84:0/4106397449 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6100031810 con 0x7f611015ad70 2023-10-16T10:02:02.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.180+0000 7f610dffb640 1 -- 172.21.15.84:0/4106397449 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6100031ba0 con 0x7f611015ad70 2023-10-16T10:02:02.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.180+0000 7f611565d640 1 -- 172.21.15.84:0/4106397449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f611015ad70 msgr2=0x7f6110069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.180+0000 7f611565d640 1 --2- 172.21.15.84:0/4106397449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f611015ad70 0x7f6110069da0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f61000097a0 tx=0x7f610002b8b0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.180+0000 7f611565d640 1 -- 172.21.15.84:0/4106397449 shutdown_connections 2023-10-16T10:02:02.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.180+0000 7f611565d640 1 --2- 172.21.15.84:0/4106397449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61100caac0 0x7f611006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.180+0000 7f611565d640 1 --2- 172.21.15.84:0/4106397449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f611015ad70 0x7f6110069da0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.181+0000 7f611565d640 1 -- 172.21.15.84:0/4106397449 >> 172.21.15.84:0/4106397449 conn(0x7f61100ce9b0 msgr2=0x7f61100cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:02.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.181+0000 7f611565d640 1 -- 172.21.15.84:0/4106397449 shutdown_connections 2023-10-16T10:02:02.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.181+0000 7f611565d640 1 -- 172.21.15.84:0/4106397449 wait complete. 2023-10-16T10:02:02.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.182+0000 7f611565d640 1 Processor -- start 2023-10-16T10:02:02.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.182+0000 7f611565d640 1 -- start start 2023-10-16T10:02:02.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.183+0000 7f611565d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61100c9ca0 0x7f61101f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.183+0000 7f611565d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61100caac0 0x7f61100ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.183+0000 7f611565d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f611015ad70 0x7f61100acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.183+0000 7f611565d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61100c74d0 con 0x7f611015ad70 2023-10-16T10:02:02.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.183+0000 7f611565d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61100c6b70 con 0x7f61100c9ca0 2023-10-16T10:02:02.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.183+0000 7f611565d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61100c6e10 con 0x7f61100caac0 2023-10-16T10:02:02.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.184+0000 7f610f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f611015ad70 0x7f61100acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.184+0000 7f610f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f611015ad70 0x7f61100acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43738/0 (socket says 172.21.15.84:43738) 2023-10-16T10:02:02.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.184+0000 7f610f7fe640 1 -- 172.21.15.84:0/383170553 learned_addr learned my addr 172.21.15.84:0/383170553 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:02.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.184+0000 7f610e7fc640 1 --2- 172.21.15.84:0/383170553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61100caac0 0x7f61100ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.184+0000 7f610f7fe640 1 -- 172.21.15.84:0/383170553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61100caac0 msgr2=0x7f61100ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.184+0000 7f610f7fe640 1 --2- 172.21.15.84:0/383170553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61100caac0 0x7f61100ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.184+0000 7f610f7fe640 1 -- 172.21.15.84:0/383170553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61100c9ca0 msgr2=0x7f61101f1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.184+0000 7f610f7fe640 1 --2- 172.21.15.84:0/383170553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61100c9ca0 0x7f61101f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.184+0000 7f610f7fe640 1 -- 172.21.15.84:0/383170553 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6100009400 con 0x7f611015ad70 2023-10-16T10:02:02.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.184+0000 7f610e7fc640 1 --2- 172.21.15.84:0/383170553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61100caac0 0x7f61100ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:02.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.184+0000 7f610f7fe640 1 --2- 172.21.15.84:0/383170553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f611015ad70 0x7f61100acdd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f610800cef0 tx=0x7f610800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:02.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.185+0000 7f60f77fe640 1 -- 172.21.15.84:0/383170553 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610800d840 con 0x7f611015ad70 2023-10-16T10:02:02.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.185+0000 7f60f77fe640 1 -- 172.21.15.84:0/383170553 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f610800d9c0 con 0x7f611015ad70 2023-10-16T10:02:02.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.185+0000 7f611565d640 1 -- 172.21.15.84:0/383170553 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61100c70f0 con 0x7f611015ad70 2023-10-16T10:02:02.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.185+0000 7f60f77fe640 1 -- 172.21.15.84:0/383170553 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610800dd70 con 0x7f611015ad70 2023-10-16T10:02:02.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.185+0000 7f611565d640 1 -- 172.21.15.84:0/383170553 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61100c78f0 con 0x7f611015ad70 2023-10-16T10:02:02.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.186+0000 7f611565d640 1 -- 172.21.15.84:0/383170553 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61100c7610 con 0x7f611015ad70 2023-10-16T10:02:02.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.187+0000 7f60f77fe640 1 -- 172.21.15.84:0/383170553 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f610801e3b0 con 0x7f611015ad70 2023-10-16T10:02:02.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.188+0000 7f60f77fe640 1 --2- 172.21.15.84:0/383170553 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f60fc06af90 0x7f60fc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.188+0000 7f60f77fe640 1 -- 172.21.15.84:0/383170553 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5588+0+0 (secure 0 0 0) 0x7f610808a130 con 0x7f611015ad70 2023-10-16T10:02:02.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.188+0000 7f610effd640 1 --2- 172.21.15.84:0/383170553 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f60fc06af90 0x7f60fc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.189+0000 7f610effd640 1 --2- 172.21.15.84:0/383170553 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f60fc06af90 0x7f60fc06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6100036000 tx=0x7f6100030000 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:02.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.192+0000 7f60f77fe640 1 -- 172.21.15.84:0/383170553 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f61100c7610 con 0x7f611015ad70 2023-10-16T10:02:02.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.384+0000 7f611565d640 1 -- 172.21.15.84:0/383170553 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f61100c7610 con 0x7f611015ad70 2023-10-16T10:02:02.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.390+0000 7f60f77fe640 1 -- 172.21.15.84:0/383170553 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f61100c7610 con 0x7f611015ad70 2023-10-16T10:02:02.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.393+0000 7f611565d640 1 -- 172.21.15.84:0/383170553 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f60fc06af90 msgr2=0x7f60fc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.393+0000 7f611565d640 1 --2- 172.21.15.84:0/383170553 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f60fc06af90 0x7f60fc06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6100036000 tx=0x7f6100030000 comp rx=0 tx=0).stop 2023-10-16T10:02:02.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.393+0000 7f611565d640 1 -- 172.21.15.84:0/383170553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f611015ad70 msgr2=0x7f61100acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.393+0000 7f611565d640 1 --2- 172.21.15.84:0/383170553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f611015ad70 0x7f61100acdd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f610800cef0 tx=0x7f610800a710 comp rx=0 tx=0).stop 2023-10-16T10:02:02.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.393+0000 7f611565d640 1 -- 172.21.15.84:0/383170553 shutdown_connections 2023-10-16T10:02:02.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.393+0000 7f611565d640 1 --2- 172.21.15.84:0/383170553 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f60fc06af90 0x7f60fc06d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.393+0000 7f611565d640 1 --2- 172.21.15.84:0/383170553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f611015ad70 0x7f61100acdd0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.393+0000 7f611565d640 1 --2- 172.21.15.84:0/383170553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61100caac0 0x7f61100ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.393+0000 7f611565d640 1 --2- 172.21.15.84:0/383170553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61100c9ca0 0x7f61101f1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.393+0000 7f611565d640 1 -- 172.21.15.84:0/383170553 >> 172.21.15.84:0/383170553 conn(0x7f61100ce9b0 msgr2=0x7f6110070d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:02.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.393+0000 7f611565d640 1 -- 172.21.15.84:0/383170553 shutdown_connections 2023-10-16T10:02:02.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.393+0000 7f611565d640 1 -- 172.21.15.84:0/383170553 wait complete. 2023-10-16T10:02:02.412 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T10:02:02.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.620+0000 7f4eaf289640 1 Processor -- start 2023-10-16T10:02:02.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.621+0000 7f4eaf289640 1 -- start start 2023-10-16T10:02:02.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.622+0000 7f4eaf289640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ea815d150 0x7f4ea80ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.622+0000 7f4eaf289640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea80ac890 con 0x7f4ea8160b90 2023-10-16T10:02:02.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.622+0000 7f4eaf289640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea8161650 con 0x7f4ea806a4b0 2023-10-16T10:02:02.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.622+0000 7f4eaf289640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea815d530 con 0x7f4ea815d150 2023-10-16T10:02:02.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.622+0000 7f4eac824640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4ea8160b90 0x7f4ea8069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:50344/0 (socket says 172.21.15.84:50344) 2023-10-16T10:02:02.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.623+0000 7f4eac824640 1 -- 172.21.15.84:0/1761850493 learned_addr learned my addr 172.21.15.84:0/1761850493 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:02.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.623+0000 7f4ea7fff640 1 -- 172.21.15.84:0/1761850493 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2982761420 0 0) 0x7f4ea80ac890 con 0x7f4ea8160b90 2023-10-16T10:02:02.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.623+0000 7f4ea7fff640 1 -- 172.21.15.84:0/1761850493 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e90003520 con 0x7f4ea8160b90 2023-10-16T10:02:02.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.623+0000 7f4ead025640 1 --2- 172.21.15.84:0/1761850493 >> v2:172.21.15.157:3300/0 conn(0x7f4ea815d150 0x7f4ea80ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.623+0000 7f4ea7fff640 1 -- 172.21.15.84:0/1761850493 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2969651064 0 0) 0x7f4e90003520 con 0x7f4ea8160b90 2023-10-16T10:02:02.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.623+0000 7f4ea7fff640 1 -- 172.21.15.84:0/1761850493 >> v2:172.21.15.157:3300/0 conn(0x7f4ea815d150 msgr2=0x7f4ea80ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:02.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.623+0000 7f4ea7fff640 1 --2- 172.21.15.84:0/1761850493 >> v2:172.21.15.157:3300/0 conn(0x7f4ea815d150 0x7f4ea80ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.623+0000 7f4ea7fff640 1 -- 172.21.15.84:0/1761850493 >> v1:172.21.15.157:6789/0 conn(0x7f4ea806a4b0 legacy=0x7f4ea80c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:02.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.623+0000 7f4ea7fff640 1 -- 172.21.15.84:0/1761850493 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea8161930 con 0x7f4ea8160b90 2023-10-16T10:02:02.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.623+0000 7f4ea7fff640 1 -- 172.21.15.84:0/1761850493 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f4e9c001670 con 0x7f4ea8160b90 2023-10-16T10:02:02.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.623+0000 7f4ea7fff640 1 -- 172.21.15.84:0/1761850493 >> v1:172.21.15.84:6789/0 conn(0x7f4ea8160b90 legacy=0x7f4ea8069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.623+0000 7f4ea7fff640 1 --2- 172.21.15.84:0/1761850493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e90003970 0x7f4e90003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.624+0000 7f4ea7fff640 1 --2- 172.21.15.84:0/1761850493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e900047c0 0x7f4e90004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.624+0000 7f4ea7fff640 1 --2- 172.21.15.84:0/1761850493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e90005180 0x7f4e9000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.624+0000 7f4ea7fff640 1 -- 172.21.15.84:0/1761850493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea815d530 con 0x7f4e900047c0 2023-10-16T10:02:02.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.624+0000 7f4ea7fff640 1 -- 172.21.15.84:0/1761850493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9000c0b0 con 0x7f4e90005180 2023-10-16T10:02:02.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.624+0000 7f4ea7fff640 1 -- 172.21.15.84:0/1761850493 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9000c270 con 0x7f4e90003970 2023-10-16T10:02:02.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.624+0000 7f4eac824640 1 --2- 172.21.15.84:0/1761850493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e900047c0 0x7f4e90004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.624+0000 7f4ead826640 1 --2- 172.21.15.84:0/1761850493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e90005180 0x7f4e9000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.624+0000 7f4ead025640 1 --2- 172.21.15.84:0/1761850493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e90003970 0x7f4e90003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.624+0000 7f4eac824640 1 -- 172.21.15.84:0/1761850493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e90003970 msgr2=0x7f4e90003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:02.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.624+0000 7f4eac824640 1 --2- 172.21.15.84:0/1761850493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e90003970 0x7f4e90003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.624+0000 7f4eac824640 1 -- 172.21.15.84:0/1761850493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e90005180 msgr2=0x7f4e9000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:02.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.624+0000 7f4eac824640 1 --2- 172.21.15.84:0/1761850493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e90005180 0x7f4e9000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.624+0000 7f4eac824640 1 -- 172.21.15.84:0/1761850493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e90003520 con 0x7f4e900047c0 2023-10-16T10:02:02.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.624+0000 7f4ead826640 1 -- 172.21.15.84:0/1761850493 reap_dead start 2023-10-16T10:02:02.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.625+0000 7f4eac824640 1 --2- 172.21.15.84:0/1761850493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e900047c0 0x7f4e90004c40 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c001eb0 tx=0x7f4e900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2644af7a35dba57 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:02.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.625+0000 7f4ea7fff640 1 -- 172.21.15.84:0/1761850493 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c005ad0 con 0x7f4e900047c0 2023-10-16T10:02:02.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.625+0000 7f4ea7fff640 1 -- 172.21.15.84:0/1761850493 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c005c50 con 0x7f4e900047c0 2023-10-16T10:02:02.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.625+0000 7f4ea7fff640 1 -- 172.21.15.84:0/1761850493 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c02c890 con 0x7f4e900047c0 2023-10-16T10:02:02.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.626+0000 7f4eaf289640 1 -- 172.21.15.84:0/1761850493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e900047c0 msgr2=0x7f4e90004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.626+0000 7f4eaf289640 1 --2- 172.21.15.84:0/1761850493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e900047c0 0x7f4e90004c40 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c001eb0 tx=0x7f4e900057f0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.626+0000 7f4eaf289640 1 -- 172.21.15.84:0/1761850493 shutdown_connections 2023-10-16T10:02:02.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.626+0000 7f4eaf289640 1 --2- 172.21.15.84:0/1761850493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e900047c0 0x7f4e90004c40 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.626+0000 7f4eaf289640 1 -- 172.21.15.84:0/1761850493 >> 172.21.15.84:0/1761850493 conn(0x7f4ea80ce9b0 msgr2=0x7f4ea80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:02.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.626+0000 7f4eaf289640 1 -- 172.21.15.84:0/1761850493 shutdown_connections 2023-10-16T10:02:02.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.626+0000 7f4eaf289640 1 -- 172.21.15.84:0/1761850493 wait complete. 2023-10-16T10:02:02.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.628+0000 7f4eaf289640 1 Processor -- start 2023-10-16T10:02:02.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.628+0000 7f4eaf289640 1 -- start start 2023-10-16T10:02:02.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.629+0000 7f4eaf289640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e900047c0 0x7f4ea80c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.629+0000 7f4eaf289640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ea80c6490 0x7f4ea81f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.629+0000 7f4eaf289640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ea81445c0 0x7f4ea80c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.629+0000 7f4eaf289640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea81474f0 con 0x7f4e900047c0 2023-10-16T10:02:02.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.629+0000 7f4eaf289640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea80c6970 con 0x7f4ea81445c0 2023-10-16T10:02:02.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.629+0000 7f4eaf289640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea80c6c50 con 0x7f4ea80c6490 2023-10-16T10:02:02.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.629+0000 7f4ead025640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e900047c0 0x7f4ea80c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.629+0000 7f4ead025640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e900047c0 0x7f4ea80c4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43754/0 (socket says 172.21.15.84:43754) 2023-10-16T10:02:02.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.629+0000 7f4ead025640 1 -- 172.21.15.84:0/1861202423 learned_addr learned my addr 172.21.15.84:0/1861202423 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:02.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.629+0000 7f4ead025640 1 -- 172.21.15.84:0/1861202423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ea80c6490 msgr2=0x7f4ea81f1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.629+0000 7f4ead025640 1 --2- 172.21.15.84:0/1861202423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ea80c6490 0x7f4ea81f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.629+0000 7f4ead025640 1 -- 172.21.15.84:0/1861202423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ea81445c0 msgr2=0x7f4ea80c24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.629+0000 7f4ead025640 1 --2- 172.21.15.84:0/1861202423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ea81445c0 0x7f4ea80c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.629+0000 7f4ead025640 1 -- 172.21.15.84:0/1861202423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea80c6f30 con 0x7f4e900047c0 2023-10-16T10:02:02.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.630+0000 7f4ead025640 1 --2- 172.21.15.84:0/1861202423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e900047c0 0x7f4ea80c4c80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4e98007e10 tx=0x7f4e9800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:02.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.630+0000 7f4ea5ffb640 1 -- 172.21.15.84:0/1861202423 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e980145f0 con 0x7f4e900047c0 2023-10-16T10:02:02.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.630+0000 7f4ea5ffb640 1 -- 172.21.15.84:0/1861202423 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e98014750 con 0x7f4e900047c0 2023-10-16T10:02:02.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.630+0000 7f4ea5ffb640 1 -- 172.21.15.84:0/1861202423 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98014ae0 con 0x7f4e900047c0 2023-10-16T10:02:02.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.630+0000 7f4eaf289640 1 -- 172.21.15.84:0/1861202423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea80c7210 con 0x7f4e900047c0 2023-10-16T10:02:02.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.630+0000 7f4eaf289640 1 -- 172.21.15.84:0/1861202423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea80c51c0 con 0x7f4e900047c0 2023-10-16T10:02:02.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.631+0000 7f4eaf289640 1 -- 172.21.15.84:0/1861202423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea804e160 con 0x7f4e900047c0 2023-10-16T10:02:02.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.632+0000 7f4ea5ffb640 1 -- 172.21.15.84:0/1861202423 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4e98014c40 con 0x7f4e900047c0 2023-10-16T10:02:02.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.633+0000 7f4ea5ffb640 1 --2- 172.21.15.84:0/1861202423 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f4e9406afc0 0x7f4e9406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.633+0000 7f4ea5ffb640 1 -- 172.21.15.84:0/1861202423 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5588+0+0 (secure 0 0 0) 0x7f4e9808a580 con 0x7f4e900047c0 2023-10-16T10:02:02.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.633+0000 7f4eac824640 1 --2- 172.21.15.84:0/1861202423 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f4e9406afc0 0x7f4e9406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.634+0000 7f4eac824640 1 --2- 172.21.15.84:0/1861202423 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f4e9406afc0 0x7f4e9406d480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c034000 tx=0x7f4e9c039040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:02.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.637+0000 7f4ea5ffb640 1 -- 172.21.15.84:0/1861202423 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4e980553e0 con 0x7f4e900047c0 2023-10-16T10:02:02.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.814+0000 7f4eaf289640 1 -- 172.21.15.84:0/1861202423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f4ea80ade10 con 0x7f4e900047c0 2023-10-16T10:02:02.817 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:02:02.826 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:02:02.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.819+0000 7f4ea5ffb640 1 -- 172.21.15.84:0/1861202423 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v14) v1 ==== 120+0+0 (secure 0 0 0) 0x7f4e9800fea0 con 0x7f4e900047c0 2023-10-16T10:02:02.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.822+0000 7f4eaf289640 1 -- 172.21.15.84:0/1861202423 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f4e9406afc0 msgr2=0x7f4e9406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.822+0000 7f4eaf289640 1 --2- 172.21.15.84:0/1861202423 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f4e9406afc0 0x7f4e9406d480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c034000 tx=0x7f4e9c039040 comp rx=0 tx=0).stop 2023-10-16T10:02:02.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.822+0000 7f4eaf289640 1 -- 172.21.15.84:0/1861202423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e900047c0 msgr2=0x7f4ea80c4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.822+0000 7f4eaf289640 1 --2- 172.21.15.84:0/1861202423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e900047c0 0x7f4ea80c4c80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4e98007e10 tx=0x7f4e9800f520 comp rx=0 tx=0).stop 2023-10-16T10:02:02.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.823+0000 7f4eaf289640 1 -- 172.21.15.84:0/1861202423 shutdown_connections 2023-10-16T10:02:02.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.823+0000 7f4eaf289640 1 --2- 172.21.15.84:0/1861202423 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f4e9406afc0 0x7f4e9406d480 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.823+0000 7f4eaf289640 1 --2- 172.21.15.84:0/1861202423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ea81445c0 0x7f4ea80c24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.823+0000 7f4eaf289640 1 --2- 172.21.15.84:0/1861202423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ea80c6490 0x7f4ea81f1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.823+0000 7f4eaf289640 1 --2- 172.21.15.84:0/1861202423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e900047c0 0x7f4ea80c4c80 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.823+0000 7f4eaf289640 1 -- 172.21.15.84:0/1861202423 >> 172.21.15.84:0/1861202423 conn(0x7f4ea80ce9b0 msgr2=0x7f4ea815b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:02.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.823+0000 7f4eaf289640 1 -- 172.21.15.84:0/1861202423 shutdown_connections 2023-10-16T10:02:02.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:02.823+0000 7f4eaf289640 1 -- 172.21.15.84:0/1861202423 wait complete. 2023-10-16T10:02:02.842 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T10:02:03.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.064+0000 7fb3ef59e640 1 Processor -- start 2023-10-16T10:02:03.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.065+0000 7fb3ef59e640 1 -- start start 2023-10-16T10:02:03.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.066+0000 7fb3ef59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f00adb30 0x7fb3f00cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.066+0000 7fb3ef59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb3f00b0c90 0x7fb3f00c9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.066+0000 7fb3ef59e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f006f3a0 con 0x7fb3f006c630 2023-10-16T10:02:03.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.066+0000 7fb3ef59e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f00c8e70 con 0x7fb3f00b0c90 2023-10-16T10:02:03.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.066+0000 7fb3ef59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00c9150 con 0x7fb3f00adb30 2023-10-16T10:02:03.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.066+0000 7fb3ee59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb3f006c630 0x7fb3f00cd260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:50360/0 (socket says 172.21.15.84:50360) 2023-10-16T10:02:03.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.067+0000 7fb3ee59c640 1 -- 172.21.15.84:0/1040436348 learned_addr learned my addr 172.21.15.84:0/1040436348 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:03.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.067+0000 7fb3ed59a640 1 -- 172.21.15.84:0/1040436348 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 717764672 0 0) 0x7fb3f006f3a0 con 0x7fb3f006c630 2023-10-16T10:02:03.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.067+0000 7fb3ed59a640 1 -- 172.21.15.84:0/1040436348 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3d8003520 con 0x7fb3f006c630 2023-10-16T10:02:03.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.067+0000 7fb3edd9b640 1 --2- 172.21.15.84:0/1040436348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f00adb30 0x7fb3f00cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.067+0000 7fb3eed9d640 1 --2- 172.21.15.84:0/1040436348 >> v2:172.21.15.157:3300/0 conn(0x7fb3f00b0c90 0x7fb3f00c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.067+0000 7fb3ed59a640 1 -- 172.21.15.84:0/1040436348 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 142678737 0 0) 0x7fb3d8003520 con 0x7fb3f006c630 2023-10-16T10:02:03.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.067+0000 7fb3ed59a640 1 -- 172.21.15.84:0/1040436348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f00adb30 msgr2=0x7fb3f00cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:03.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.067+0000 7fb3ed59a640 1 --2- 172.21.15.84:0/1040436348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f00adb30 0x7fb3f00cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.067+0000 7fb3ed59a640 1 -- 172.21.15.84:0/1040436348 >> v2:172.21.15.157:3300/0 conn(0x7fb3f00b0c90 msgr2=0x7fb3f00c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:03.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.067+0000 7fb3ed59a640 1 --2- 172.21.15.84:0/1040436348 >> v2:172.21.15.157:3300/0 conn(0x7fb3f00b0c90 0x7fb3f00c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.067+0000 7fb3ed59a640 1 -- 172.21.15.84:0/1040436348 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f00c9430 con 0x7fb3f006c630 2023-10-16T10:02:03.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.067+0000 7fb3ed59a640 1 -- 172.21.15.84:0/1040436348 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fb3e0001690 con 0x7fb3f006c630 2023-10-16T10:02:03.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.067+0000 7fb3edd9b640 1 --2- 172.21.15.84:0/1040436348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f00adb30 0x7fb3f00cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:02:03.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.067+0000 7fb3ed59a640 1 -- 172.21.15.84:0/1040436348 >> v1:172.21.15.84:6789/0 conn(0x7fb3f006c630 legacy=0x7fb3f00cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.068+0000 7fb3ed59a640 1 --2- 172.21.15.84:0/1040436348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f00adb30 0x7fb3d8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.068+0000 7fb3ed59a640 1 --2- 172.21.15.84:0/1040436348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3d8003f60 0x7fb3d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.068+0000 7fb3ed59a640 1 --2- 172.21.15.84:0/1040436348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3d8004d00 0x7fb3d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.068+0000 7fb3ed59a640 1 -- 172.21.15.84:0/1040436348 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00c8e70 con 0x7fb3d8003f60 2023-10-16T10:02:03.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.068+0000 7fb3ed59a640 1 -- 172.21.15.84:0/1040436348 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00c9150 con 0x7fb3d8004d00 2023-10-16T10:02:03.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.068+0000 7fb3ed59a640 1 -- 172.21.15.84:0/1040436348 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d800bae0 con 0x7fb3f00adb30 2023-10-16T10:02:03.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.068+0000 7fb3edd9b640 1 --2- 172.21.15.84:0/1040436348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3d8003f60 0x7fb3d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.068+0000 7fb3edd9b640 1 -- 172.21.15.84:0/1040436348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f00adb30 msgr2=0x7fb3d8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:03.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.068+0000 7fb3edd9b640 1 --2- 172.21.15.84:0/1040436348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3f00adb30 0x7fb3d8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.068+0000 7fb3edd9b640 1 -- 172.21.15.84:0/1040436348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3d8004d00 msgr2=0x7fb3d8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:03.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.068+0000 7fb3edd9b640 1 --2- 172.21.15.84:0/1040436348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3d8004d00 0x7fb3d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.068+0000 7fb3edd9b640 1 -- 172.21.15.84:0/1040436348 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d8003520 con 0x7fb3d8003f60 2023-10-16T10:02:03.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.069+0000 7fb3edd9b640 1 --2- 172.21.15.84:0/1040436348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3d8003f60 0x7fb3d80047c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40098a0 tx=0x7fb3d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aae504ac97aa83c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:03.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.069+0000 7fb3ed59a640 1 -- 172.21.15.84:0/1040436348 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4004230 con 0x7fb3d8003f60 2023-10-16T10:02:03.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.070+0000 7fb3ed59a640 1 -- 172.21.15.84:0/1040436348 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e4004390 con 0x7fb3d8003f60 2023-10-16T10:02:03.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.070+0000 7fb3ed59a640 1 -- 172.21.15.84:0/1040436348 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4003830 con 0x7fb3d8003f60 2023-10-16T10:02:03.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.070+0000 7fb3ef59e640 1 -- 172.21.15.84:0/1040436348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3d8003f60 msgr2=0x7fb3d80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.070+0000 7fb3ef59e640 1 --2- 172.21.15.84:0/1040436348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3d8003f60 0x7fb3d80047c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40098a0 tx=0x7fb3d80057f0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.070+0000 7fb3eed9d640 1 -- 172.21.15.84:0/1040436348 reap_dead start 2023-10-16T10:02:03.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.070+0000 7fb3ef59e640 1 -- 172.21.15.84:0/1040436348 shutdown_connections 2023-10-16T10:02:03.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.070+0000 7fb3ef59e640 1 -- 172.21.15.84:0/1040436348 >> 172.21.15.84:0/1040436348 conn(0x7fb3f00b0170 msgr2=0x7fb3f00b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:03.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.070+0000 7fb3ef59e640 1 -- 172.21.15.84:0/1040436348 shutdown_connections 2023-10-16T10:02:03.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.070+0000 7fb3ef59e640 1 -- 172.21.15.84:0/1040436348 wait complete. 2023-10-16T10:02:03.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.072+0000 7fb3ef59e640 1 Processor -- start 2023-10-16T10:02:03.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.072+0000 7fb3ef59e640 1 -- start start 2023-10-16T10:02:03.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.073+0000 7fb3ef59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3d8003f60 0x7fb3f01626d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.073+0000 7fb3ef59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f0162e90 0x7fb3f015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.073+0000 7fb3ef59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f00b28a0 0x7fb3f00b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.073+0000 7fb3ef59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00b3260 con 0x7fb3f00b28a0 2023-10-16T10:02:03.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.073+0000 7fb3ef59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f015e250 con 0x7fb3f0162e90 2023-10-16T10:02:03.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.073+0000 7fb3ef59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f0163630 con 0x7fb3d8003f60 2023-10-16T10:02:03.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.073+0000 7fb3eed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f00b28a0 0x7fb3f00b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.073+0000 7fb3eed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f00b28a0 0x7fb3f00b2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43776/0 (socket says 172.21.15.84:43776) 2023-10-16T10:02:03.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.073+0000 7fb3eed9d640 1 -- 172.21.15.84:0/806356934 learned_addr learned my addr 172.21.15.84:0/806356934 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:03.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.074+0000 7fb3edd9b640 1 --2- 172.21.15.84:0/806356934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f0162e90 0x7fb3f015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.074+0000 7fb3ee59c640 1 --2- 172.21.15.84:0/806356934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3d8003f60 0x7fb3f01626d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.074+0000 7fb3eed9d640 1 -- 172.21.15.84:0/806356934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3d8003f60 msgr2=0x7fb3f01626d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.074+0000 7fb3eed9d640 1 --2- 172.21.15.84:0/806356934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3d8003f60 0x7fb3f01626d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.074+0000 7fb3eed9d640 1 -- 172.21.15.84:0/806356934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f0162e90 msgr2=0x7fb3f015dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.074+0000 7fb3eed9d640 1 --2- 172.21.15.84:0/806356934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f0162e90 0x7fb3f015dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.074+0000 7fb3eed9d640 1 -- 172.21.15.84:0/806356934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f0163910 con 0x7fb3f00b28a0 2023-10-16T10:02:03.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.074+0000 7fb3edd9b640 1 --2- 172.21.15.84:0/806356934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f0162e90 0x7fb3f015dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:03.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.074+0000 7fb3eed9d640 1 --2- 172.21.15.84:0/806356934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f00b28a0 0x7fb3f00b2d20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00b1070 tx=0x7fb3e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:03.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.074+0000 7fb3deffd640 1 -- 172.21.15.84:0/806356934 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e800ed90 con 0x7fb3f00b28a0 2023-10-16T10:02:03.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.074+0000 7fb3deffd640 1 -- 172.21.15.84:0/806356934 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e8014420 con 0x7fb3f00b28a0 2023-10-16T10:02:03.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.074+0000 7fb3ef59e640 1 -- 172.21.15.84:0/806356934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f0163bf0 con 0x7fb3f00b28a0 2023-10-16T10:02:03.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.074+0000 7fb3deffd640 1 -- 172.21.15.84:0/806356934 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80147d0 con 0x7fb3f00b28a0 2023-10-16T10:02:03.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.074+0000 7fb3ef59e640 1 -- 172.21.15.84:0/806356934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f01641b0 con 0x7fb3f00b28a0 2023-10-16T10:02:03.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.075+0000 7fb3ef59e640 1 -- 172.21.15.84:0/806356934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f004e9b0 con 0x7fb3f00b28a0 2023-10-16T10:02:03.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.076+0000 7fb3deffd640 1 -- 172.21.15.84:0/806356934 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb3e8014930 con 0x7fb3f00b28a0 2023-10-16T10:02:03.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.078+0000 7fb3deffd640 1 --2- 172.21.15.84:0/806356934 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fb3d006af90 0x7fb3d006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.078+0000 7fb3deffd640 1 -- 172.21.15.84:0/806356934 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5588+0+0 (secure 0 0 0) 0x7fb3e8089a90 con 0x7fb3f00b28a0 2023-10-16T10:02:03.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.078+0000 7fb3ee59c640 1 --2- 172.21.15.84:0/806356934 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fb3d006af90 0x7fb3d006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.079+0000 7fb3ee59c640 1 --2- 172.21.15.84:0/806356934 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fb3d006af90 0x7fb3d006d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0001eb0 tx=0x7fb3e0009040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:03.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.082+0000 7fb3deffd640 1 -- 172.21.15.84:0/806356934 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb3e8054a90 con 0x7fb3f00b28a0 2023-10-16T10:02:03.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.260+0000 7fb3ef59e640 1 -- 172.21.15.84:0/806356934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fb3f006d320 con 0x7fb3f00b28a0 2023-10-16T10:02:03.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.267+0000 7fb3deffd640 1 -- 172.21.15.84:0/806356934 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v15) v1 ==== 115+0+0 (secure 0 0 0) 0x7fb3e80580b0 con 0x7fb3f00b28a0 2023-10-16T10:02:03.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.270+0000 7fb3ef59e640 1 -- 172.21.15.84:0/806356934 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fb3d006af90 msgr2=0x7fb3d006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.270+0000 7fb3ef59e640 1 --2- 172.21.15.84:0/806356934 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fb3d006af90 0x7fb3d006d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0001eb0 tx=0x7fb3e0009040 comp rx=0 tx=0).stop 2023-10-16T10:02:03.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.270+0000 7fb3ef59e640 1 -- 172.21.15.84:0/806356934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f00b28a0 msgr2=0x7fb3f00b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.270+0000 7fb3ef59e640 1 --2- 172.21.15.84:0/806356934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f00b28a0 0x7fb3f00b2d20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00b1070 tx=0x7fb3e800e4b0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.270+0000 7fb3ef59e640 1 -- 172.21.15.84:0/806356934 shutdown_connections 2023-10-16T10:02:03.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.270+0000 7fb3ef59e640 1 --2- 172.21.15.84:0/806356934 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fb3d006af90 0x7fb3d006d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.270+0000 7fb3ef59e640 1 --2- 172.21.15.84:0/806356934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f00b28a0 0x7fb3f00b2d20 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.270+0000 7fb3ef59e640 1 --2- 172.21.15.84:0/806356934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f0162e90 0x7fb3f015dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.270+0000 7fb3ef59e640 1 --2- 172.21.15.84:0/806356934 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3d8003f60 0x7fb3f01626d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.270+0000 7fb3ef59e640 1 -- 172.21.15.84:0/806356934 >> 172.21.15.84:0/806356934 conn(0x7fb3f00b0170 msgr2=0x7fb3f00cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:03.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.270+0000 7fb3ef59e640 1 -- 172.21.15.84:0/806356934 shutdown_connections 2023-10-16T10:02:03.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.270+0000 7fb3ef59e640 1 -- 172.21.15.84:0/806356934 wait complete. 2023-10-16T10:02:03.290 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-10-16T10:02:03.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.488+0000 7fd7d14d8640 1 Processor -- start 2023-10-16T10:02:03.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.489+0000 7fd7d14d8640 1 -- start start 2023-10-16T10:02:03.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.490+0000 7fd7d14d8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7cc0cdef0 0x7fd7cc1609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.490+0000 7fd7d14d8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd7cc0b84a0 0x7fd7cc0d12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.490+0000 7fd7d14d8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7cc0d0060 con 0x7fd7cc15cdf0 2023-10-16T10:02:03.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.490+0000 7fd7d14d8640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7cc161970 con 0x7fd7cc0b84a0 2023-10-16T10:02:03.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.490+0000 7fd7d14d8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7cc0d0e60 con 0x7fd7cc0cdef0 2023-10-16T10:02:03.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.490+0000 7fd7d0cd7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd7cc0b84a0 0x7fd7cc0d12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.490+0000 7fd7cbfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd7cc15cdf0 0x7fd7cc0cd7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:50374/0 (socket says 172.21.15.84:50374) 2023-10-16T10:02:03.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.490+0000 7fd7cbfff640 1 -- 172.21.15.84:0/2136850355 learned_addr learned my addr 172.21.15.84:0/2136850355 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:03.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.490+0000 7fd7d0cd7640 1 -- 172.21.15.84:0/2136850355 >> v2:172.21.15.157:3300/0 conn(0x7fd7cc0cdef0 msgr2=0x7fd7cc1609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:03.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.490+0000 7fd7d0cd7640 1 --2- 172.21.15.84:0/2136850355 >> v2:172.21.15.157:3300/0 conn(0x7fd7cc0cdef0 0x7fd7cc1609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.490+0000 7fd7d0cd7640 1 -- 172.21.15.84:0/2136850355 >> v1:172.21.15.84:6789/0 conn(0x7fd7cc15cdf0 legacy=0x7fd7cc0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.491+0000 7fd7d0cd7640 1 -- 172.21.15.84:0/2136850355 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc1610d0 con 0x7fd7cc0b84a0 2023-10-16T10:02:03.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.492+0000 7fd7d0cd7640 1 --2- 172.21.15.84:0/2136850355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7cc0b84a0 0x7fd7cc0d12f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0d05f0 tx=0x7fd7c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8489b47ac9eada server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:03.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.492+0000 7fd7caffd640 1 -- 172.21.15.84:0/2136850355 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4031830 con 0x7fd7cc0b84a0 2023-10-16T10:02:03.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.492+0000 7fd7caffd640 1 -- 172.21.15.84:0/2136850355 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c4031990 con 0x7fd7cc0b84a0 2023-10-16T10:02:03.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.492+0000 7fd7caffd640 1 -- 172.21.15.84:0/2136850355 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4031d20 con 0x7fd7cc0b84a0 2023-10-16T10:02:03.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.492+0000 7fd7d14d8640 1 -- 172.21.15.84:0/2136850355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7cc0b84a0 msgr2=0x7fd7cc0d12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.492+0000 7fd7d14d8640 1 --2- 172.21.15.84:0/2136850355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7cc0b84a0 0x7fd7cc0d12f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0d05f0 tx=0x7fd7c402b8d0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.492+0000 7fd7d14d8640 1 -- 172.21.15.84:0/2136850355 shutdown_connections 2023-10-16T10:02:03.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.492+0000 7fd7d14d8640 1 --2- 172.21.15.84:0/2136850355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7cc0b84a0 0x7fd7cc0d12f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.492+0000 7fd7d14d8640 1 --2- 172.21.15.84:0/2136850355 >> v2:172.21.15.157:3300/0 conn(0x7fd7cc0cdef0 0x7fd7cc1609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.492+0000 7fd7d14d8640 1 -- 172.21.15.84:0/2136850355 >> 172.21.15.84:0/2136850355 conn(0x7fd7cc06bc60 msgr2=0x7fd7cc06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:03.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.493+0000 7fd7d14d8640 1 -- 172.21.15.84:0/2136850355 shutdown_connections 2023-10-16T10:02:03.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.493+0000 7fd7d14d8640 1 -- 172.21.15.84:0/2136850355 wait complete. 2023-10-16T10:02:03.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.494+0000 7fd7d14d8640 1 Processor -- start 2023-10-16T10:02:03.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.494+0000 7fd7d14d8640 1 -- start start 2023-10-16T10:02:03.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.495+0000 7fd7d14d8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7cc0b84a0 0x7fd7cc145c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.495+0000 7fd7d14d8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7cc0cdef0 0x7fd7cc152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.495+0000 7fd7d14d8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7cc15cdf0 0x7fd7cc1528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.495+0000 7fd7d14d8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc06c570 con 0x7fd7cc0b84a0 2023-10-16T10:02:03.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.495+0000 7fd7d14d8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc151ca0 con 0x7fd7cc0cdef0 2023-10-16T10:02:03.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.495+0000 7fd7d14d8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc06c290 con 0x7fd7cc15cdf0 2023-10-16T10:02:03.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.495+0000 7fd7cbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7cc0b84a0 0x7fd7cc145c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.496+0000 7fd7cbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7cc0b84a0 0x7fd7cc145c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43802/0 (socket says 172.21.15.84:43802) 2023-10-16T10:02:03.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.496+0000 7fd7cbfff640 1 -- 172.21.15.84:0/3041465868 learned_addr learned my addr 172.21.15.84:0/3041465868 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:03.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.496+0000 7fd7cbfff640 1 -- 172.21.15.84:0/3041465868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7cc15cdf0 msgr2=0x7fd7cc1528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.496+0000 7fd7cbfff640 1 --2- 172.21.15.84:0/3041465868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7cc15cdf0 0x7fd7cc1528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.496+0000 7fd7cbfff640 1 -- 172.21.15.84:0/3041465868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7cc0cdef0 msgr2=0x7fd7cc152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.496+0000 7fd7cb7fe640 1 --2- 172.21.15.84:0/3041465868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7cc0cdef0 0x7fd7cc152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.496+0000 7fd7cbfff640 1 --2- 172.21.15.84:0/3041465868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7cc0cdef0 0x7fd7cc152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.496+0000 7fd7cbfff640 1 -- 172.21.15.84:0/3041465868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c4009400 con 0x7fd7cc0b84a0 2023-10-16T10:02:03.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.496+0000 7fd7cb7fe640 1 --2- 172.21.15.84:0/3041465868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7cc0cdef0 0x7fd7cc152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:03.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.496+0000 7fd7cbfff640 1 --2- 172.21.15.84:0/3041465868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7cc0b84a0 0x7fd7cc145c30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400c480 tx=0x7fd7b400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:03.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.496+0000 7fd7c8ff9640 1 -- 172.21.15.84:0/3041465868 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4005ce0 con 0x7fd7cc0b84a0 2023-10-16T10:02:03.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.497+0000 7fd7c8ff9640 1 -- 172.21.15.84:0/3041465868 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b4005e40 con 0x7fd7cc0b84a0 2023-10-16T10:02:03.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.497+0000 7fd7c8ff9640 1 -- 172.21.15.84:0/3041465868 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b40072f0 con 0x7fd7cc0b84a0 2023-10-16T10:02:03.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.497+0000 7fd7d14d8640 1 -- 172.21.15.84:0/3041465868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7cc145430 con 0x7fd7cc0b84a0 2023-10-16T10:02:03.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.497+0000 7fd7d14d8640 1 -- 172.21.15.84:0/3041465868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc146170 con 0x7fd7cc0b84a0 2023-10-16T10:02:03.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.498+0000 7fd7d14d8640 1 -- 172.21.15.84:0/3041465868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7cc145950 con 0x7fd7cc0b84a0 2023-10-16T10:02:03.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.499+0000 7fd7c8ff9640 1 -- 172.21.15.84:0/3041465868 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd7b4007510 con 0x7fd7cc0b84a0 2023-10-16T10:02:03.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.499+0000 7fd7c8ff9640 1 --2- 172.21.15.84:0/3041465868 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd7ac06af90 0x7fd7ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.500+0000 7fd7c8ff9640 1 -- 172.21.15.84:0/3041465868 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5588+0+0 (secure 0 0 0) 0x7fd7b4088bf0 con 0x7fd7cc0b84a0 2023-10-16T10:02:03.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.500+0000 7fd7cb7fe640 1 --2- 172.21.15.84:0/3041465868 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd7ac06af90 0x7fd7ac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.500+0000 7fd7cb7fe640 1 --2- 172.21.15.84:0/3041465868 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd7ac06af90 0x7fd7ac06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0b8070 tx=0x7fd7bc0073c0 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:03.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.504+0000 7fd7c8ff9640 1 -- 172.21.15.84:0/3041465868 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd7cc145950 con 0x7fd7cc0b84a0 2023-10-16T10:02:03.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.783+0000 7fd7d14d8640 1 -- 172.21.15.84:0/3041465868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7fd7cc145950 con 0x7fd7cc0b84a0 2023-10-16T10:02:03.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.980+0000 7fd7c8ff9640 1 -- 172.21.15.84:0/3041465868 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v467) v1 ==== 158+0+0 (secure 0 0 0) 0x7fd7cc145950 con 0x7fd7cc0b84a0 2023-10-16T10:02:03.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.983+0000 7fd7d14d8640 1 -- 172.21.15.84:0/3041465868 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd7ac06af90 msgr2=0x7fd7ac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.983+0000 7fd7d14d8640 1 --2- 172.21.15.84:0/3041465868 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd7ac06af90 0x7fd7ac06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0b8070 tx=0x7fd7bc0073c0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.983+0000 7fd7d14d8640 1 -- 172.21.15.84:0/3041465868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7cc0b84a0 msgr2=0x7fd7cc145c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.983+0000 7fd7d14d8640 1 --2- 172.21.15.84:0/3041465868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7cc0b84a0 0x7fd7cc145c30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400c480 tx=0x7fd7b400c940 comp rx=0 tx=0).stop 2023-10-16T10:02:03.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.984+0000 7fd7d14d8640 1 -- 172.21.15.84:0/3041465868 shutdown_connections 2023-10-16T10:02:03.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.984+0000 7fd7d14d8640 1 --2- 172.21.15.84:0/3041465868 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd7ac06af90 0x7fd7ac06d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.984+0000 7fd7d14d8640 1 --2- 172.21.15.84:0/3041465868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7cc15cdf0 0x7fd7cc1528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.984+0000 7fd7d14d8640 1 --2- 172.21.15.84:0/3041465868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7cc0cdef0 0x7fd7cc152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.984+0000 7fd7d14d8640 1 --2- 172.21.15.84:0/3041465868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7cc0b84a0 0x7fd7cc145c30 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.984+0000 7fd7d14d8640 1 -- 172.21.15.84:0/3041465868 >> 172.21.15.84:0/3041465868 conn(0x7fd7cc06bc60 msgr2=0x7fd7cc0b5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:03.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.984+0000 7fd7d14d8640 1 -- 172.21.15.84:0/3041465868 shutdown_connections 2023-10-16T10:02:03.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:03.984+0000 7fd7d14d8640 1 -- 172.21.15.84:0/3041465868 wait complete. 2023-10-16T10:02:04.005 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2023-10-16T10:02:04.006 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2023-10-16T10:02:04.006 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T10:02:04.007 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:02:04.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.212+0000 7f1e4eb89640 1 Processor -- start 2023-10-16T10:02:04.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.212+0000 7f1e4eb89640 1 -- start start 2023-10-16T10:02:04.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.214+0000 7f1e4eb89640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e48159f00 0x7f1e480b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.214+0000 7f1e4eb89640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1e4815d880 0x7f1e480b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.214+0000 7f1e4eb89640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e4806f7f0 con 0x7f1e48159b20 2023-10-16T10:02:04.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.214+0000 7f1e4eb89640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4806c5e0 con 0x7f1e4815d880 2023-10-16T10:02:04.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.214+0000 7f1e4eb89640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4806c8c0 con 0x7f1e48159f00 2023-10-16T10:02:04.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.214+0000 7f1e4d126640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1e4815d880 0x7f1e480b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.214+0000 7f1e4d126640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1e4815d880 0x7f1e480b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43804/0 (socket says 172.21.15.84:43804) 2023-10-16T10:02:04.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.214+0000 7f1e4d126640 1 -- 172.21.15.84:0/2253459015 learned_addr learned my addr 172.21.15.84:0/2253459015 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:04.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.214+0000 7f1e47fff640 1 --2- 172.21.15.84:0/2253459015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e48159f00 0x7f1e480b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.215+0000 7f1e4d126640 1 -- 172.21.15.84:0/2253459015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e48159f00 msgr2=0x7f1e480b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:04.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.215+0000 7f1e4d126640 1 --2- 172.21.15.84:0/2253459015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e48159f00 0x7f1e480b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.215+0000 7f1e4d126640 1 -- 172.21.15.84:0/2253459015 >> v1:172.21.15.157:6789/0 conn(0x7f1e48159b20 legacy=0x7f1e4806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:04.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.215+0000 7f1e4d126640 1 -- 172.21.15.84:0/2253459015 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4806cba0 con 0x7f1e4815d880 2023-10-16T10:02:04.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.216+0000 7f1e4d126640 1 --2- 172.21.15.84:0/2253459015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4815d880 0x7f1e480b0950 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009e50 tx=0x7f1e4002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0cead98bea33f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:04.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.216+0000 7f1e477fe640 1 -- 172.21.15.84:0/2253459015 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4002ca70 con 0x7f1e4815d880 2023-10-16T10:02:04.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.216+0000 7f1e477fe640 1 -- 172.21.15.84:0/2253459015 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4002cbd0 con 0x7f1e4815d880 2023-10-16T10:02:04.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.216+0000 7f1e477fe640 1 -- 172.21.15.84:0/2253459015 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e400343f0 con 0x7f1e4815d880 2023-10-16T10:02:04.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.217+0000 7f1e4eb89640 1 -- 172.21.15.84:0/2253459015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4815d880 msgr2=0x7f1e480b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:04.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.217+0000 7f1e4eb89640 1 --2- 172.21.15.84:0/2253459015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4815d880 0x7f1e480b0950 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009e50 tx=0x7f1e4002bed0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.217+0000 7f1e4eb89640 1 -- 172.21.15.84:0/2253459015 shutdown_connections 2023-10-16T10:02:04.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.217+0000 7f1e4eb89640 1 --2- 172.21.15.84:0/2253459015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4815d880 0x7f1e480b0950 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.217+0000 7f1e4eb89640 1 --2- 172.21.15.84:0/2253459015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e48159f00 0x7f1e480b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.217+0000 7f1e4eb89640 1 -- 172.21.15.84:0/2253459015 >> 172.21.15.84:0/2253459015 conn(0x7f1e480cb410 msgr2=0x7f1e480cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:04.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.217+0000 7f1e4eb89640 1 -- 172.21.15.84:0/2253459015 shutdown_connections 2023-10-16T10:02:04.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.217+0000 7f1e4eb89640 1 -- 172.21.15.84:0/2253459015 wait complete. 2023-10-16T10:02:04.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.219+0000 7f1e4eb89640 1 Processor -- start 2023-10-16T10:02:04.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.219+0000 7f1e4eb89640 1 -- start start 2023-10-16T10:02:04.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.220+0000 7f1e4eb89640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e48159b20 0x7f1e480acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.220+0000 7f1e4eb89640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e48159f00 0x7f1e4814a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.220+0000 7f1e4eb89640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e4815d880 0x7f1e4814af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.220+0000 7f1e4eb89640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e480c7e90 con 0x7f1e48159f00 2023-10-16T10:02:04.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.220+0000 7f1e4eb89640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e480c2bc0 con 0x7f1e4815d880 2023-10-16T10:02:04.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.220+0000 7f1e4eb89640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e480c7210 con 0x7f1e48159b20 2023-10-16T10:02:04.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.220+0000 7f1e47fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e48159f00 0x7f1e4814a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.220+0000 7f1e47fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e48159f00 0x7f1e4814a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43810/0 (socket says 172.21.15.84:43810) 2023-10-16T10:02:04.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.220+0000 7f1e47fff640 1 -- 172.21.15.84:0/1606956404 learned_addr learned my addr 172.21.15.84:0/1606956404 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:04.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.221+0000 7f1e4c925640 1 --2- 172.21.15.84:0/1606956404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e48159b20 0x7f1e480acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.221+0000 7f1e47fff640 1 -- 172.21.15.84:0/1606956404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e48159b20 msgr2=0x7f1e480acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:04.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.221+0000 7f1e4d126640 1 --2- 172.21.15.84:0/1606956404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e4815d880 0x7f1e4814af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.221+0000 7f1e47fff640 1 --2- 172.21.15.84:0/1606956404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e48159b20 0x7f1e480acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.221+0000 7f1e47fff640 1 -- 172.21.15.84:0/1606956404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e4815d880 msgr2=0x7f1e4814af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:04.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.221+0000 7f1e47fff640 1 --2- 172.21.15.84:0/1606956404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e4815d880 0x7f1e4814af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.221+0000 7f1e47fff640 1 -- 172.21.15.84:0/1606956404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e40009b30 con 0x7f1e48159f00 2023-10-16T10:02:04.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.221+0000 7f1e47fff640 1 --2- 172.21.15.84:0/1606956404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e48159f00 0x7f1e4814a9e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400c7f0 tx=0x7f1e3400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:04.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.221+0000 7f1e457fa640 1 -- 172.21.15.84:0/1606956404 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e34004220 con 0x7f1e48159f00 2023-10-16T10:02:04.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.222+0000 7f1e457fa640 1 -- 172.21.15.84:0/1606956404 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e34004380 con 0x7f1e48159f00 2023-10-16T10:02:04.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.222+0000 7f1e457fa640 1 -- 172.21.15.84:0/1606956404 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e34007840 con 0x7f1e48159f00 2023-10-16T10:02:04.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.222+0000 7f1e4eb89640 1 -- 172.21.15.84:0/1606956404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e480c74f0 con 0x7f1e48159f00 2023-10-16T10:02:04.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.222+0000 7f1e4eb89640 1 -- 172.21.15.84:0/1606956404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e480c7ab0 con 0x7f1e48159f00 2023-10-16T10:02:04.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.223+0000 7f1e4eb89640 1 -- 172.21.15.84:0/1606956404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4806c270 con 0x7f1e48159f00 2023-10-16T10:02:04.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.224+0000 7f1e457fa640 1 -- 172.21.15.84:0/1606956404 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1e34007bb0 con 0x7f1e48159f00 2023-10-16T10:02:04.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.225+0000 7f1e457fa640 1 --2- 172.21.15.84:0/1606956404 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f1e2c06af90 0x7f1e2c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.225+0000 7f1e457fa640 1 -- 172.21.15.84:0/1606956404 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5588+0+0 (secure 0 0 0) 0x7f1e34089800 con 0x7f1e48159f00 2023-10-16T10:02:04.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.225+0000 7f1e4c925640 1 --2- 172.21.15.84:0/1606956404 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f1e2c06af90 0x7f1e2c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.226+0000 7f1e4c925640 1 --2- 172.21.15.84:0/1606956404 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f1e2c06af90 0x7f1e2c06d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00a750 tx=0x7f1e3c008040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:04.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.229+0000 7f1e457fa640 1 -- 172.21.15.84:0/1606956404 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1e34054670 con 0x7f1e48159f00 2023-10-16T10:02:04.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.421+0000 7f1e4eb89640 1 -- 172.21.15.84:0/1606956404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e4804e160 con 0x7f1e48159f00 2023-10-16T10:02:04.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.423+0000 7f1e457fa640 1 -- 172.21.15.84:0/1606956404 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+13900 (secure 0 0 0) 0x7f1e3401e070 con 0x7f1e48159f00 2023-10-16T10:02:04.424 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:02:04.425 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":55,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:02:01.950867+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6833/62459899":"2023-10-17T10:01:58.930460+0000","172.21.15.157:6832/62459899":"2023-10-17T10:01:58.930460+0000","172.21.15.84:0/2429213939":"2023-10-17T10:01:55.223514+0000","172.21.15.84:6833/3010246666":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/1060139177":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/3968394661":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/2493862491":"2023-10-17T10:01:55.223514+0000","172.21.15.84:6832/3010246666":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/1628587200":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/3763261264":"2023-10-17T10:01:55.223514+0000","172.21.15.84:0/848414450":"2023-10-17T10:01:55.223514+0000","172.21.15.84:6832/4265272960":"2023-10-17T10:01:55.223514+0000","172.21.15.84:0/3285111878":"2023-10-17T10:01:55.223514+0000","172.21.15.84:0/3734084949":"2023-10-17T10:01:55.223514+0000","172.21.15.84:0/982275204":"2023-10-17T09:50:26.881911+0000","172.21.15.84:0/1706229169":"2023-10-17T10:01:55.223514+0000","172.21.15.84:6833/4265272960":"2023-10-17T10:01:55.223514+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:02:04.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.425+0000 7f1e4eb89640 1 -- 172.21.15.84:0/1606956404 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f1e2c06af90 msgr2=0x7f1e2c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:04.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.425+0000 7f1e4eb89640 1 --2- 172.21.15.84:0/1606956404 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f1e2c06af90 0x7f1e2c06d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00a750 tx=0x7f1e3c008040 comp rx=0 tx=0).stop 2023-10-16T10:02:04.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.425+0000 7f1e4eb89640 1 -- 172.21.15.84:0/1606956404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e48159f00 msgr2=0x7f1e4814a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:04.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.425+0000 7f1e4eb89640 1 --2- 172.21.15.84:0/1606956404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e48159f00 0x7f1e4814a9e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400c7f0 tx=0x7f1e3400ccb0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.426+0000 7f1e4eb89640 1 -- 172.21.15.84:0/1606956404 shutdown_connections 2023-10-16T10:02:04.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.426+0000 7f1e4eb89640 1 --2- 172.21.15.84:0/1606956404 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f1e2c06af90 0x7f1e2c06d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.426+0000 7f1e4eb89640 1 --2- 172.21.15.84:0/1606956404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e4815d880 0x7f1e4814af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.426+0000 7f1e4eb89640 1 --2- 172.21.15.84:0/1606956404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e48159f00 0x7f1e4814a9e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.426+0000 7f1e4eb89640 1 --2- 172.21.15.84:0/1606956404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e48159b20 0x7f1e480acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.426+0000 7f1e4eb89640 1 -- 172.21.15.84:0/1606956404 >> 172.21.15.84:0/1606956404 conn(0x7f1e480cb410 msgr2=0x7f1e480b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:04.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.426+0000 7f1e4eb89640 1 -- 172.21.15.84:0/1606956404 shutdown_connections 2023-10-16T10:02:04.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.426+0000 7f1e4eb89640 1 -- 172.21.15.84:0/1606956404 wait complete. 2023-10-16T10:02:04.446 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:02:04.447 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:02:04.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.663+0000 7ff0e4ddc640 1 Processor -- start 2023-10-16T10:02:04.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.664+0000 7ff0e4ddc640 1 -- start start 2023-10-16T10:02:04.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.665+0000 7ff0e4ddc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0e015af80 0x7ff0e00cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.665+0000 7ff0e4ddc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff0e0065ab0 0x7ff0e0065f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.665+0000 7ff0e4ddc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e0066450 con 0x7ff0e006faf0 2023-10-16T10:02:04.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.665+0000 7ff0e4ddc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e00d0930 con 0x7ff0e0065ab0 2023-10-16T10:02:04.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.665+0000 7ff0e4ddc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e015eff0 con 0x7ff0e015af80 2023-10-16T10:02:04.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.666+0000 7ff0ddd9b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff0e006faf0 0x7ff0e015f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:50378/0 (socket says 172.21.15.84:50378) 2023-10-16T10:02:04.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.666+0000 7ff0ddd9b640 1 -- 172.21.15.84:0/1908484616 learned_addr learned my addr 172.21.15.84:0/1908484616 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:04.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.666+0000 7ff0de59c640 1 --2- 172.21.15.84:0/1908484616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0e015af80 0x7ff0e00cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.666+0000 7ff0ded9d640 1 --2- 172.21.15.84:0/1908484616 >> v2:172.21.15.157:3300/0 conn(0x7ff0e0065ab0 0x7ff0e0065f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.666+0000 7ff0dd59a640 1 -- 172.21.15.84:0/1908484616 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283848309 0 0) 0x7ff0e0066450 con 0x7ff0e006faf0 2023-10-16T10:02:04.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.666+0000 7ff0dd59a640 1 -- 172.21.15.84:0/1908484616 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0cc003520 con 0x7ff0e006faf0 2023-10-16T10:02:04.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.666+0000 7ff0dd59a640 1 -- 172.21.15.84:0/1908484616 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1921010267 0 0) 0x7ff0cc003520 con 0x7ff0e006faf0 2023-10-16T10:02:04.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.666+0000 7ff0dd59a640 1 -- 172.21.15.84:0/1908484616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0e015af80 msgr2=0x7ff0e00cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:04.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.666+0000 7ff0dd59a640 1 --2- 172.21.15.84:0/1908484616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0e015af80 0x7ff0e00cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.666+0000 7ff0dd59a640 1 -- 172.21.15.84:0/1908484616 >> v2:172.21.15.157:3300/0 conn(0x7ff0e0065ab0 msgr2=0x7ff0e0065f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:04.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.666+0000 7ff0dd59a640 1 --2- 172.21.15.84:0/1908484616 >> v2:172.21.15.157:3300/0 conn(0x7ff0e0065ab0 0x7ff0e0065f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.666+0000 7ff0dd59a640 1 -- 172.21.15.84:0/1908484616 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e015ecc0 con 0x7ff0e006faf0 2023-10-16T10:02:04.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.666+0000 7ff0dd59a640 1 -- 172.21.15.84:0/1908484616 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ff0c8001670 con 0x7ff0e006faf0 2023-10-16T10:02:04.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.666+0000 7ff0dd59a640 1 -- 172.21.15.84:0/1908484616 >> v1:172.21.15.84:6789/0 conn(0x7ff0e006faf0 legacy=0x7ff0e015f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:04.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.667+0000 7ff0dd59a640 1 --2- 172.21.15.84:0/1908484616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0e015af80 0x7ff0cc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.667+0000 7ff0dd59a640 1 --2- 172.21.15.84:0/1908484616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0cc003ef0 0x7ff0cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.667+0000 7ff0dd59a640 1 --2- 172.21.15.84:0/1908484616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0cc004d00 0x7ff0cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.667+0000 7ff0dd59a640 1 -- 172.21.15.84:0/1908484616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00d0930 con 0x7ff0cc003ef0 2023-10-16T10:02:04.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.667+0000 7ff0dd59a640 1 -- 172.21.15.84:0/1908484616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e015eff0 con 0x7ff0cc004d00 2023-10-16T10:02:04.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.667+0000 7ff0dd59a640 1 -- 172.21.15.84:0/1908484616 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc00b9d0 con 0x7ff0e015af80 2023-10-16T10:02:04.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.667+0000 7ff0ddd9b640 1 --2- 172.21.15.84:0/1908484616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0cc003ef0 0x7ff0cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.667+0000 7ff0de59c640 1 --2- 172.21.15.84:0/1908484616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0e015af80 0x7ff0cc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.667+0000 7ff0ddd9b640 1 -- 172.21.15.84:0/1908484616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0e015af80 msgr2=0x7ff0cc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:04.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.667+0000 7ff0ddd9b640 1 --2- 172.21.15.84:0/1908484616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0e015af80 0x7ff0cc003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.667+0000 7ff0ded9d640 1 --2- 172.21.15.84:0/1908484616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0cc004d00 0x7ff0cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.667+0000 7ff0ddd9b640 1 -- 172.21.15.84:0/1908484616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0cc004d00 msgr2=0x7ff0cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:04.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.667+0000 7ff0ddd9b640 1 --2- 172.21.15.84:0/1908484616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0cc004d00 0x7ff0cc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.667+0000 7ff0ddd9b640 1 -- 172.21.15.84:0/1908484616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc003520 con 0x7ff0cc003ef0 2023-10-16T10:02:04.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.668+0000 7ff0ddd9b640 1 --2- 172.21.15.84:0/1908484616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0cc003ef0 0x7ff0cc0047c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800c270 tx=0x7ff0cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485f32b7a279e060 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:04.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.669+0000 7ff0dd59a640 1 -- 172.21.15.84:0/1908484616 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c802cae0 con 0x7ff0cc003ef0 2023-10-16T10:02:04.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.669+0000 7ff0dd59a640 1 -- 172.21.15.84:0/1908484616 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c802cc40 con 0x7ff0cc003ef0 2023-10-16T10:02:04.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.669+0000 7ff0dd59a640 1 -- 172.21.15.84:0/1908484616 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8005af0 con 0x7ff0cc003ef0 2023-10-16T10:02:04.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.669+0000 7ff0e4ddc640 1 -- 172.21.15.84:0/1908484616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0cc003ef0 msgr2=0x7ff0cc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:04.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.669+0000 7ff0e4ddc640 1 --2- 172.21.15.84:0/1908484616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0cc003ef0 0x7ff0cc0047c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800c270 tx=0x7ff0cc0057f0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.669+0000 7ff0ded9d640 1 -- 172.21.15.84:0/1908484616 reap_dead start 2023-10-16T10:02:04.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.669+0000 7ff0e4ddc640 1 -- 172.21.15.84:0/1908484616 shutdown_connections 2023-10-16T10:02:04.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.669+0000 7ff0e4ddc640 1 -- 172.21.15.84:0/1908484616 >> 172.21.15.84:0/1908484616 conn(0x7ff0e00d0ab0 msgr2=0x7ff0e00d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:04.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.670+0000 7ff0e4ddc640 1 -- 172.21.15.84:0/1908484616 shutdown_connections 2023-10-16T10:02:04.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.670+0000 7ff0e4ddc640 1 -- 172.21.15.84:0/1908484616 wait complete. 2023-10-16T10:02:04.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.671+0000 7ff0e4ddc640 1 Processor -- start 2023-10-16T10:02:04.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.671+0000 7ff0e4ddc640 1 -- start start 2023-10-16T10:02:04.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.672+0000 7ff0e4ddc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0cc003ef0 0x7ff0e00b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.672+0000 7ff0e4ddc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0e0158010 0x7ff0e00b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.672+0000 7ff0e4ddc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0e015d510 0x7ff0e015d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.672+0000 7ff0e4ddc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e015ded0 con 0x7ff0e0158010 2023-10-16T10:02:04.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.672+0000 7ff0e4ddc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00b0d50 con 0x7ff0e015d510 2023-10-16T10:02:04.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.672+0000 7ff0e4ddc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00b1030 con 0x7ff0cc003ef0 2023-10-16T10:02:04.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.673+0000 7ff0ddd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0e0158010 0x7ff0e00b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.673+0000 7ff0ddd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0e0158010 0x7ff0e00b6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43834/0 (socket says 172.21.15.84:43834) 2023-10-16T10:02:04.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.673+0000 7ff0ddd9b640 1 -- 172.21.15.84:0/3528986369 learned_addr learned my addr 172.21.15.84:0/3528986369 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:04.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.673+0000 7ff0de59c640 1 --2- 172.21.15.84:0/3528986369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0cc003ef0 0x7ff0e00b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.673+0000 7ff0ded9d640 1 --2- 172.21.15.84:0/3528986369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0e015d510 0x7ff0e015d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.673+0000 7ff0ddd9b640 1 -- 172.21.15.84:0/3528986369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0cc003ef0 msgr2=0x7ff0e00b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:04.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.673+0000 7ff0ddd9b640 1 --2- 172.21.15.84:0/3528986369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0cc003ef0 0x7ff0e00b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.673+0000 7ff0ddd9b640 1 -- 172.21.15.84:0/3528986369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0e015d510 msgr2=0x7ff0e015d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:04.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.673+0000 7ff0ddd9b640 1 --2- 172.21.15.84:0/3528986369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0e015d510 0x7ff0e015d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.673+0000 7ff0ddd9b640 1 -- 172.21.15.84:0/3528986369 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e00b1310 con 0x7ff0e0158010 2023-10-16T10:02:04.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.674+0000 7ff0ddd9b640 1 --2- 172.21.15.84:0/3528986369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0e0158010 0x7ff0e00b6420 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800c210 tx=0x7ff0c8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:04.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.674+0000 7ff0d6ffd640 1 -- 172.21.15.84:0/3528986369 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8039030 con 0x7ff0e0158010 2023-10-16T10:02:04.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.674+0000 7ff0d6ffd640 1 -- 172.21.15.84:0/3528986369 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c8004bf0 con 0x7ff0e0158010 2023-10-16T10:02:04.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.674+0000 7ff0d6ffd640 1 -- 172.21.15.84:0/3528986369 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c803c040 con 0x7ff0e0158010 2023-10-16T10:02:04.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.674+0000 7ff0e4ddc640 1 -- 172.21.15.84:0/3528986369 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e00b15f0 con 0x7ff0e0158010 2023-10-16T10:02:04.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.674+0000 7ff0e4ddc640 1 -- 172.21.15.84:0/3528986369 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e00b4af0 con 0x7ff0e0158010 2023-10-16T10:02:04.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.675+0000 7ff0e4ddc640 1 -- 172.21.15.84:0/3528986369 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e0072e50 con 0x7ff0e0158010 2023-10-16T10:02:04.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.676+0000 7ff0d6ffd640 1 -- 172.21.15.84:0/3528986369 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff0c8032070 con 0x7ff0e0158010 2023-10-16T10:02:04.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.677+0000 7ff0d6ffd640 1 --2- 172.21.15.84:0/3528986369 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7ff0c006af90 0x7ff0c006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.678+0000 7ff0d6ffd640 1 -- 172.21.15.84:0/3528986369 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5588+0+0 (secure 0 0 0) 0x7ff0c80aaa40 con 0x7ff0e0158010 2023-10-16T10:02:04.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.678+0000 7ff0de59c640 1 --2- 172.21.15.84:0/3528986369 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7ff0c006af90 0x7ff0c006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.678+0000 7ff0de59c640 1 --2- 172.21.15.84:0/3528986369 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7ff0c006af90 0x7ff0c006d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0003bc0 tx=0x7ff0d000b040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:04.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.683+0000 7ff0d6ffd640 1 -- 172.21.15.84:0/3528986369 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff0c80ace60 con 0x7ff0e0158010 2023-10-16T10:02:04.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.877+0000 7ff0e4ddc640 1 -- 172.21.15.84:0/3528986369 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0e00b4cb0 con 0x7ff0e0158010 2023-10-16T10:02:04.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.878+0000 7ff0d6ffd640 1 -- 172.21.15.84:0/3528986369 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff0c8075a50 con 0x7ff0e0158010 2023-10-16T10:02:04.880 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:02:04.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.880+0000 7ff0e4ddc640 1 -- 172.21.15.84:0/3528986369 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7ff0c006af90 msgr2=0x7ff0c006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:04.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.881+0000 7ff0e4ddc640 1 --2- 172.21.15.84:0/3528986369 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7ff0c006af90 0x7ff0c006d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0003bc0 tx=0x7ff0d000b040 comp rx=0 tx=0).stop 2023-10-16T10:02:04.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.881+0000 7ff0e4ddc640 1 -- 172.21.15.84:0/3528986369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0e0158010 msgr2=0x7ff0e00b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:04.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.881+0000 7ff0e4ddc640 1 --2- 172.21.15.84:0/3528986369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0e0158010 0x7ff0e00b6420 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800c210 tx=0x7ff0c8001a00 comp rx=0 tx=0).stop 2023-10-16T10:02:04.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.881+0000 7ff0e4ddc640 1 -- 172.21.15.84:0/3528986369 shutdown_connections 2023-10-16T10:02:04.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.881+0000 7ff0e4ddc640 1 --2- 172.21.15.84:0/3528986369 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7ff0c006af90 0x7ff0c006d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.881+0000 7ff0e4ddc640 1 --2- 172.21.15.84:0/3528986369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0e015d510 0x7ff0e015d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.881+0000 7ff0e4ddc640 1 --2- 172.21.15.84:0/3528986369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0e0158010 0x7ff0e00b6420 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.881+0000 7ff0e4ddc640 1 --2- 172.21.15.84:0/3528986369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0cc003ef0 0x7ff0e00b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.881+0000 7ff0e4ddc640 1 -- 172.21.15.84:0/3528986369 >> 172.21.15.84:0/3528986369 conn(0x7ff0e00d0ab0 msgr2=0x7ff0e0158a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:04.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.881+0000 7ff0e4ddc640 1 -- 172.21.15.84:0/3528986369 shutdown_connections 2023-10-16T10:02:04.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:04.881+0000 7ff0e4ddc640 1 -- 172.21.15.84:0/3528986369 wait complete. 2023-10-16T10:02:04.902 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-10-16T10:02:05.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.114+0000 7f59a5da9640 1 Processor -- start 2023-10-16T10:02:05.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.115+0000 7f59a5da9640 1 -- start start 2023-10-16T10:02:05.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.116+0000 7f59a5da9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f59a00ce390 0x7f59a015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:05.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.116+0000 7f59a5da9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59a015ed60 con 0x7f59a015ae80 2023-10-16T10:02:05.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.116+0000 7f59a5da9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59a00707e0 con 0x7f59a015aaa0 2023-10-16T10:02:05.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.116+0000 7f59a5da9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59a006b810 con 0x7f59a00ce390 2023-10-16T10:02:05.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.116+0000 7f599effd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f59a015ae80 0x7f59a0070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:50380/0 (socket says 172.21.15.84:50380) 2023-10-16T10:02:05.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.116+0000 7f599effd640 1 -- 172.21.15.84:0/1999855878 learned_addr learned my addr 172.21.15.84:0/1999855878 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:05.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.117+0000 7f599ffff640 1 --2- 172.21.15.84:0/1999855878 >> v2:172.21.15.84:3300/0 conn(0x7f59a00ce390 0x7f59a015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:05.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.117+0000 7f599ffff640 1 -- 172.21.15.84:0/1999855878 >> v1:172.21.15.157:6789/0 conn(0x7f59a015aaa0 legacy=0x7f59a00cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:05.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.117+0000 7f599ffff640 1 -- 172.21.15.84:0/1999855878 >> v1:172.21.15.84:6789/0 conn(0x7f59a015ae80 legacy=0x7f59a0070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:05.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.117+0000 7f599ffff640 1 -- 172.21.15.84:0/1999855878 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a015ef90 con 0x7f59a00ce390 2023-10-16T10:02:05.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.118+0000 7f599ffff640 1 --2- 172.21.15.84:0/1999855878 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59a00ce390 0x7f59a015e820 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f59980097a0 tx=0x7f599802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e17f6385617fe3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:05.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.118+0000 7f599e7fc640 1 -- 172.21.15.84:0/1999855878 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998031830 con 0x7f59a00ce390 2023-10-16T10:02:05.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.118+0000 7f599e7fc640 1 -- 172.21.15.84:0/1999855878 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5998031990 con 0x7f59a00ce390 2023-10-16T10:02:05.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.118+0000 7f599e7fc640 1 -- 172.21.15.84:0/1999855878 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998031d20 con 0x7f59a00ce390 2023-10-16T10:02:05.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.119+0000 7f59a5da9640 1 -- 172.21.15.84:0/1999855878 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59a00ce390 msgr2=0x7f59a015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:05.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.119+0000 7f59a5da9640 1 --2- 172.21.15.84:0/1999855878 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59a00ce390 0x7f59a015e820 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f59980097a0 tx=0x7f599802b8d0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.119+0000 7f59a5da9640 1 -- 172.21.15.84:0/1999855878 shutdown_connections 2023-10-16T10:02:05.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.119+0000 7f59a5da9640 1 --2- 172.21.15.84:0/1999855878 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59a00ce390 0x7f59a015e820 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.119+0000 7f59a5da9640 1 -- 172.21.15.84:0/1999855878 >> 172.21.15.84:0/1999855878 conn(0x7f59a00cb410 msgr2=0x7f59a00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:05.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.119+0000 7f59a5da9640 1 -- 172.21.15.84:0/1999855878 shutdown_connections 2023-10-16T10:02:05.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.119+0000 7f59a5da9640 1 -- 172.21.15.84:0/1999855878 wait complete. 2023-10-16T10:02:05.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.122+0000 7f59a5da9640 1 Processor -- start 2023-10-16T10:02:05.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.122+0000 7f59a5da9640 1 -- start start 2023-10-16T10:02:05.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.123+0000 7f59a5da9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59a00ce390 0x7f59a00c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:05.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.123+0000 7f59a5da9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59a015aaa0 0x7f59a00c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:05.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.123+0000 7f59a5da9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59a015ae80 0x7f59a0158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:05.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.123+0000 7f59a5da9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a00aef20 con 0x7f59a015aaa0 2023-10-16T10:02:05.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.123+0000 7f59a5da9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a00af780 con 0x7f59a00ce390 2023-10-16T10:02:05.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.123+0000 7f59a5da9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a00afa60 con 0x7f59a015ae80 2023-10-16T10:02:05.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.123+0000 7f599effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59a015aaa0 0x7f59a00c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:05.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.123+0000 7f599effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59a015aaa0 0x7f59a00c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43838/0 (socket says 172.21.15.84:43838) 2023-10-16T10:02:05.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.123+0000 7f599effd640 1 -- 172.21.15.84:0/1515567754 learned_addr learned my addr 172.21.15.84:0/1515567754 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:05.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.124+0000 7f599f7fe640 1 --2- 172.21.15.84:0/1515567754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59a00ce390 0x7f59a00c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:05.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.124+0000 7f599ffff640 1 --2- 172.21.15.84:0/1515567754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59a015ae80 0x7f59a0158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:05.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.124+0000 7f599effd640 1 -- 172.21.15.84:0/1515567754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59a015ae80 msgr2=0x7f59a0158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:05.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.124+0000 7f599effd640 1 --2- 172.21.15.84:0/1515567754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59a015ae80 0x7f59a0158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.124+0000 7f599effd640 1 -- 172.21.15.84:0/1515567754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59a00ce390 msgr2=0x7f59a00c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:05.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.124+0000 7f599effd640 1 --2- 172.21.15.84:0/1515567754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59a00ce390 0x7f59a00c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.124+0000 7f599effd640 1 -- 172.21.15.84:0/1515567754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5998009400 con 0x7f59a015aaa0 2023-10-16T10:02:05.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.124+0000 7f599effd640 1 --2- 172.21.15.84:0/1515567754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59a015aaa0 0x7f59a00c87c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f599000cb10 tx=0x7f599000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:05.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.124+0000 7f5997fff640 1 -- 172.21.15.84:0/1515567754 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599000ac70 con 0x7f59a015aaa0 2023-10-16T10:02:05.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.124+0000 7f5997fff640 1 -- 172.21.15.84:0/1515567754 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599000add0 con 0x7f59a015aaa0 2023-10-16T10:02:05.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.124+0000 7f5997fff640 1 -- 172.21.15.84:0/1515567754 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5990012600 con 0x7f59a015aaa0 2023-10-16T10:02:05.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.125+0000 7f59a5da9640 1 -- 172.21.15.84:0/1515567754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a00afd40 con 0x7f59a015aaa0 2023-10-16T10:02:05.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.125+0000 7f59a5da9640 1 -- 172.21.15.84:0/1515567754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a0159480 con 0x7f59a015aaa0 2023-10-16T10:02:05.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.126+0000 7f59a5da9640 1 -- 172.21.15.84:0/1515567754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59a015f530 con 0x7f59a015aaa0 2023-10-16T10:02:05.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.127+0000 7f5997fff640 1 -- 172.21.15.84:0/1515567754 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5990007070 con 0x7f59a015aaa0 2023-10-16T10:02:05.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.128+0000 7f5997fff640 1 --2- 172.21.15.84:0/1515567754 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f598006af90 0x7f598006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:05.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.128+0000 7f5997fff640 1 -- 172.21.15.84:0/1515567754 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5588+0+0 (secure 0 0 0) 0x7f5990089530 con 0x7f59a015aaa0 2023-10-16T10:02:05.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.128+0000 7f599f7fe640 1 --2- 172.21.15.84:0/1515567754 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f598006af90 0x7f598006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:05.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.129+0000 7f599f7fe640 1 --2- 172.21.15.84:0/1515567754 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f598006af90 0x7f598006d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f59a015b520 tx=0x7f5988009040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:05.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.134+0000 7f5997fff640 1 -- 172.21.15.84:0/1515567754 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f59a015f530 con 0x7f59a015aaa0 2023-10-16T10:02:05.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.383+0000 7f59a5da9640 1 -- 172.21.15.84:0/1515567754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f59a015f530 con 0x7f59a015aaa0 2023-10-16T10:02:05.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.994+0000 7f5997fff640 1 -- 172.21.15.84:0/1515567754 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v469) v1 ==== 157+0+0 (secure 0 0 0) 0x7f59a015f530 con 0x7f59a015aaa0 2023-10-16T10:02:05.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.997+0000 7f59a5da9640 1 -- 172.21.15.84:0/1515567754 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f598006af90 msgr2=0x7f598006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:05.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.997+0000 7f59a5da9640 1 --2- 172.21.15.84:0/1515567754 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f598006af90 0x7f598006d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f59a015b520 tx=0x7f5988009040 comp rx=0 tx=0).stop 2023-10-16T10:02:05.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.997+0000 7f59a5da9640 1 -- 172.21.15.84:0/1515567754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59a015aaa0 msgr2=0x7f59a00c87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:06.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.997+0000 7f59a5da9640 1 --2- 172.21.15.84:0/1515567754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59a015aaa0 0x7f59a00c87c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f599000cb10 tx=0x7f599000a060 comp rx=0 tx=0).stop 2023-10-16T10:02:06.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.997+0000 7f59a5da9640 1 -- 172.21.15.84:0/1515567754 shutdown_connections 2023-10-16T10:02:06.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.997+0000 7f59a5da9640 1 --2- 172.21.15.84:0/1515567754 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f598006af90 0x7f598006d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.998+0000 7f59a5da9640 1 --2- 172.21.15.84:0/1515567754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59a015ae80 0x7f59a0158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.998+0000 7f59a5da9640 1 --2- 172.21.15.84:0/1515567754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59a015aaa0 0x7f59a00c87c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.998+0000 7f59a5da9640 1 --2- 172.21.15.84:0/1515567754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59a00ce390 0x7f59a00c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.998+0000 7f59a5da9640 1 -- 172.21.15.84:0/1515567754 >> 172.21.15.84:0/1515567754 conn(0x7f59a00cb410 msgr2=0x7f59a006f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:06.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.998+0000 7f59a5da9640 1 -- 172.21.15.84:0/1515567754 shutdown_connections 2023-10-16T10:02:06.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:05.998+0000 7f59a5da9640 1 -- 172.21.15.84:0/1515567754 wait complete. 2023-10-16T10:02:06.020 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T10:02:06.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.236+0000 7f199e55e640 1 Processor -- start 2023-10-16T10:02:06.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.237+0000 7f199e55e640 1 -- start start 2023-10-16T10:02:06.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.238+0000 7f199e55e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f199815cd70 0x7f19980d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:06.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.238+0000 7f199e55e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f19980b8480 0x7f19980cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:06.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.238+0000 7f199e55e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f19980d0be0 0x7f19980cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:06.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.238+0000 7f199e55e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f199815d150 con 0x7f19980d0be0 2023-10-16T10:02:06.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.238+0000 7f199e55e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19980d1180 con 0x7f19980b8480 2023-10-16T10:02:06.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.238+0000 7f199e55e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19980d1460 con 0x7f199815cd70 2023-10-16T10:02:06.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.238+0000 7f199dd5d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f19980d0be0 0x7f19980cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:06.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.238+0000 7f199dd5d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f19980d0be0 0x7f19980cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33140/0 (socket says 172.21.15.84:33140) 2023-10-16T10:02:06.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.238+0000 7f199dd5d640 1 -- 172.21.15.84:0/992270518 learned_addr learned my addr 172.21.15.84:0/992270518 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:06.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.239+0000 7f199dd5d640 1 -- 172.21.15.84:0/992270518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f199815cd70 msgr2=0x7f19980d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:06.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.239+0000 7f199dd5d640 1 --2- 172.21.15.84:0/992270518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f199815cd70 0x7f19980d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.239+0000 7f199cd5b640 1 --2- 172.21.15.84:0/992270518 >> v2:172.21.15.157:3300/0 conn(0x7f19980b8480 0x7f19980cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:06.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.239+0000 7f199dd5d640 1 -- 172.21.15.84:0/992270518 >> v2:172.21.15.157:3300/0 conn(0x7f19980b8480 msgr2=0x7f19980cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:06.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.239+0000 7f199dd5d640 1 --2- 172.21.15.84:0/992270518 >> v2:172.21.15.157:3300/0 conn(0x7f19980b8480 0x7f19980cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.239+0000 7f199dd5d640 1 -- 172.21.15.84:0/992270518 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19980d1740 con 0x7f19980d0be0 2023-10-16T10:02:06.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.239+0000 7f199cd5b640 1 --2- 172.21.15.84:0/992270518 >> v2:172.21.15.157:3300/0 conn(0x7f19980b8480 0x7f19980cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:06.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.240+0000 7f199dd5d640 1 --2- 172.21.15.84:0/992270518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19980d0be0 0x7f19980cdd00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1998066ab0 tx=0x7f199002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375502127ea2cce7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:06.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.240+0000 7f1997fff640 1 -- 172.21.15.84:0/992270518 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990031870 con 0x7f19980d0be0 2023-10-16T10:02:06.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.240+0000 7f1997fff640 1 -- 172.21.15.84:0/992270518 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19900319d0 con 0x7f19980d0be0 2023-10-16T10:02:06.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.241+0000 7f1997fff640 1 -- 172.21.15.84:0/992270518 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990031d60 con 0x7f19980d0be0 2023-10-16T10:02:06.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.241+0000 7f199e55e640 1 -- 172.21.15.84:0/992270518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19980d0be0 msgr2=0x7f19980cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:06.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.241+0000 7f199e55e640 1 --2- 172.21.15.84:0/992270518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19980d0be0 0x7f19980cdd00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1998066ab0 tx=0x7f199002b910 comp rx=0 tx=0).stop 2023-10-16T10:02:06.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.241+0000 7f199e55e640 1 -- 172.21.15.84:0/992270518 shutdown_connections 2023-10-16T10:02:06.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.241+0000 7f199e55e640 1 --2- 172.21.15.84:0/992270518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19980d0be0 0x7f19980cdd00 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.241+0000 7f199e55e640 1 --2- 172.21.15.84:0/992270518 >> v2:172.21.15.157:3300/0 conn(0x7f19980b8480 0x7f19980cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.241+0000 7f199e55e640 1 --2- 172.21.15.84:0/992270518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f199815cd70 0x7f19980d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.241+0000 7f199e55e640 1 -- 172.21.15.84:0/992270518 >> 172.21.15.84:0/992270518 conn(0x7f19980cf570 msgr2=0x7f19980cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:06.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.241+0000 7f199e55e640 1 -- 172.21.15.84:0/992270518 shutdown_connections 2023-10-16T10:02:06.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.241+0000 7f199e55e640 1 -- 172.21.15.84:0/992270518 wait complete. 2023-10-16T10:02:06.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.243+0000 7f199e55e640 1 Processor -- start 2023-10-16T10:02:06.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.244+0000 7f199e55e640 1 -- start start 2023-10-16T10:02:06.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.245+0000 7f199e55e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19980b8480 0x7f19980c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:06.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.245+0000 7f199e55e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19980d0be0 0x7f1998152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:06.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.245+0000 7f199e55e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f199815cd70 0x7f19981f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:06.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.245+0000 7f199e55e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199815dd70 con 0x7f19980b8480 2023-10-16T10:02:06.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.245+0000 7f199e55e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199815d6d0 con 0x7f19980d0be0 2023-10-16T10:02:06.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.245+0000 7f199e55e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19981559e0 con 0x7f199815cd70 2023-10-16T10:02:06.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.245+0000 7f199d55c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19980b8480 0x7f19980c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:06.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.245+0000 7f199d55c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19980b8480 0x7f19980c74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33144/0 (socket says 172.21.15.84:33144) 2023-10-16T10:02:06.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.245+0000 7f199d55c640 1 -- 172.21.15.84:0/4180746665 learned_addr learned my addr 172.21.15.84:0/4180746665 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:06.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.245+0000 7f199d55c640 1 -- 172.21.15.84:0/4180746665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f199815cd70 msgr2=0x7f19981f1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:06.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.245+0000 7f199d55c640 1 --2- 172.21.15.84:0/4180746665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f199815cd70 0x7f19981f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.245+0000 7f199d55c640 1 -- 172.21.15.84:0/4180746665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19980d0be0 msgr2=0x7f1998152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:06.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.245+0000 7f199d55c640 1 --2- 172.21.15.84:0/4180746665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19980d0be0 0x7f1998152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.245+0000 7f199d55c640 1 -- 172.21.15.84:0/4180746665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19900094d0 con 0x7f19980b8480 2023-10-16T10:02:06.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.246+0000 7f199d55c640 1 --2- 172.21.15.84:0/4180746665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19980b8480 0x7f19980c74a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f198800ba50 tx=0x7f198800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:06.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.246+0000 7f1995ffb640 1 -- 172.21.15.84:0/4180746665 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198800cbe0 con 0x7f19980b8480 2023-10-16T10:02:06.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.246+0000 7f1995ffb640 1 -- 172.21.15.84:0/4180746665 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198800cd60 con 0x7f19980b8480 2023-10-16T10:02:06.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.246+0000 7f199e55e640 1 -- 172.21.15.84:0/4180746665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1998155cc0 con 0x7f19980b8480 2023-10-16T10:02:06.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.246+0000 7f1995ffb640 1 -- 172.21.15.84:0/4180746665 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1988014680 con 0x7f19980b8480 2023-10-16T10:02:06.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.246+0000 7f199e55e640 1 -- 172.21.15.84:0/4180746665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1998152470 con 0x7f19980b8480 2023-10-16T10:02:06.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.247+0000 7f199e55e640 1 -- 172.21.15.84:0/4180746665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199804e9b0 con 0x7f19980b8480 2023-10-16T10:02:06.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.248+0000 7f1995ffb640 1 -- 172.21.15.84:0/4180746665 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f198801d3b0 con 0x7f19980b8480 2023-10-16T10:02:06.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.249+0000 7f1995ffb640 1 --2- 172.21.15.84:0/4180746665 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f198406afe0 0x7f198406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:06.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.249+0000 7f1995ffb640 1 -- 172.21.15.84:0/4180746665 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5588+0+0 (secure 0 0 0) 0x7f1988089660 con 0x7f19980b8480 2023-10-16T10:02:06.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.249+0000 7f199cd5b640 1 --2- 172.21.15.84:0/4180746665 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f198406afe0 0x7f198406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:06.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.250+0000 7f199cd5b640 1 --2- 172.21.15.84:0/4180746665 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f198406afe0 0x7f198406d4a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f198c006fd0 tx=0x7f198c008040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:06.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.254+0000 7f1995ffb640 1 -- 172.21.15.84:0/4180746665 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1988054760 con 0x7f19980b8480 2023-10-16T10:02:06.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.445+0000 7f199e55e640 1 -- 172.21.15.84:0/4180746665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f19980cdcc0 con 0x7f19980b8480 2023-10-16T10:02:06.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.450+0000 7f1995ffb640 1 -- 172.21.15.84:0/4180746665 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1988057d80 con 0x7f19980b8480 2023-10-16T10:02:06.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.453+0000 7f199e55e640 1 -- 172.21.15.84:0/4180746665 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f198406afe0 msgr2=0x7f198406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:06.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.453+0000 7f199e55e640 1 --2- 172.21.15.84:0/4180746665 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f198406afe0 0x7f198406d4a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f198c006fd0 tx=0x7f198c008040 comp rx=0 tx=0).stop 2023-10-16T10:02:06.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.453+0000 7f199e55e640 1 -- 172.21.15.84:0/4180746665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19980b8480 msgr2=0x7f19980c74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:06.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.453+0000 7f199e55e640 1 --2- 172.21.15.84:0/4180746665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19980b8480 0x7f19980c74a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f198800ba50 tx=0x7f198800bf10 comp rx=0 tx=0).stop 2023-10-16T10:02:06.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.453+0000 7f199e55e640 1 -- 172.21.15.84:0/4180746665 shutdown_connections 2023-10-16T10:02:06.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.453+0000 7f199e55e640 1 --2- 172.21.15.84:0/4180746665 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f198406afe0 0x7f198406d4a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.453+0000 7f199e55e640 1 --2- 172.21.15.84:0/4180746665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f199815cd70 0x7f19981f1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.453+0000 7f199e55e640 1 --2- 172.21.15.84:0/4180746665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19980d0be0 0x7f1998152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.453+0000 7f199e55e640 1 --2- 172.21.15.84:0/4180746665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19980b8480 0x7f19980c74a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.453+0000 7f199e55e640 1 -- 172.21.15.84:0/4180746665 >> 172.21.15.84:0/4180746665 conn(0x7f19980cf570 msgr2=0x7f19980b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:06.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.453+0000 7f199e55e640 1 -- 172.21.15.84:0/4180746665 shutdown_connections 2023-10-16T10:02:06.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:06.453+0000 7f199e55e640 1 -- 172.21.15.84:0/4180746665 wait complete. 2023-10-16T10:02:06.472 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:02:06.472 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:02:06.569 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:02:06.696 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:02:06.696 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:02:06.697 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:02:06.754 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:02:06.755 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:02:06.755 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:02:06.815 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:02:06.815 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:02:06.816 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:02:06.876 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:02:06.876 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:02:06.877 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:02:06.934 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:02:06.934 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T10:02:06.935 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:02:07.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.154+0000 7f979837d640 1 Processor -- start 2023-10-16T10:02:07.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.154+0000 7f979837d640 1 -- start start 2023-10-16T10:02:07.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.156+0000 7f979837d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f979015aaa0 0x7f979015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.156+0000 7f979837d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f979015ae80 0x7f97900cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.156+0000 7f979837d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f979015ed60 con 0x7f97900ccaa0 2023-10-16T10:02:07.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.156+0000 7f979837d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979006b810 con 0x7f979015ae80 2023-10-16T10:02:07.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.156+0000 7f979837d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97900707e0 con 0x7f979015aaa0 2023-10-16T10:02:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.156+0000 7f9795918640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f979015ae80 0x7f97900cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.156+0000 7f9795918640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f979015ae80 0x7f97900cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33152/0 (socket says 172.21.15.84:33152) 2023-10-16T10:02:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.156+0000 7f9795918640 1 -- 172.21.15.84:0/2444174461 learned_addr learned my addr 172.21.15.84:0/2444174461 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.157+0000 7f9796119640 1 --2- 172.21.15.84:0/2444174461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f979015aaa0 0x7f979015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.157+0000 7f9795918640 1 -- 172.21.15.84:0/2444174461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f979015aaa0 msgr2=0x7f979015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:07.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.157+0000 7f9795918640 1 --2- 172.21.15.84:0/2444174461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f979015aaa0 0x7f979015e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.157+0000 7f9795918640 1 -- 172.21.15.84:0/2444174461 >> v1:172.21.15.157:6789/0 conn(0x7f97900ccaa0 legacy=0x7f9790070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:07.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.157+0000 7f9795918640 1 -- 172.21.15.84:0/2444174461 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979015ef90 con 0x7f979015ae80 2023-10-16T10:02:07.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.158+0000 7f9795918640 1 --2- 172.21.15.84:0/2444174461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979015ae80 0x7f97900cc560 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f978c009870 tx=0x7f978c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b686319c6e303527 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:07.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.158+0000 7f9795117640 1 -- 172.21.15.84:0/2444174461 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c031640 con 0x7f979015ae80 2023-10-16T10:02:07.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.158+0000 7f9795117640 1 -- 172.21.15.84:0/2444174461 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c0317a0 con 0x7f979015ae80 2023-10-16T10:02:07.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.158+0000 7f9795117640 1 -- 172.21.15.84:0/2444174461 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c031b30 con 0x7f979015ae80 2023-10-16T10:02:07.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.159+0000 7f979837d640 1 -- 172.21.15.84:0/2444174461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979015ae80 msgr2=0x7f97900cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.159+0000 7f979837d640 1 --2- 172.21.15.84:0/2444174461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979015ae80 0x7f97900cc560 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f978c009870 tx=0x7f978c02b8f0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.159+0000 7f979837d640 1 -- 172.21.15.84:0/2444174461 shutdown_connections 2023-10-16T10:02:07.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.159+0000 7f979837d640 1 --2- 172.21.15.84:0/2444174461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979015ae80 0x7f97900cc560 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.159+0000 7f979837d640 1 --2- 172.21.15.84:0/2444174461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f979015aaa0 0x7f979015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.159+0000 7f979837d640 1 -- 172.21.15.84:0/2444174461 >> 172.21.15.84:0/2444174461 conn(0x7f97900cb410 msgr2=0x7f97900cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:07.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.159+0000 7f979837d640 1 -- 172.21.15.84:0/2444174461 shutdown_connections 2023-10-16T10:02:07.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.159+0000 7f979837d640 1 -- 172.21.15.84:0/2444174461 wait complete. 2023-10-16T10:02:07.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.160+0000 7f979837d640 1 Processor -- start 2023-10-16T10:02:07.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.161+0000 7f979837d640 1 -- start start 2023-10-16T10:02:07.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.161+0000 7f979837d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97900ccaa0 0x7f97901f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.161+0000 7f979837d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979015aaa0 0x7f97901591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.161+0000 7f979837d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f979015ae80 0x7f97900b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.161+0000 7f979837d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9790159cd0 con 0x7f97900ccaa0 2023-10-16T10:02:07.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.161+0000 7f979837d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97901f7120 con 0x7f979015aaa0 2023-10-16T10:02:07.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.162+0000 7f979837d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9790159730 con 0x7f979015ae80 2023-10-16T10:02:07.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.162+0000 7f9796119640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97900ccaa0 0x7f97901f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.162+0000 7f9796119640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97900ccaa0 0x7f97901f6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33154/0 (socket says 172.21.15.84:33154) 2023-10-16T10:02:07.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.162+0000 7f9796119640 1 -- 172.21.15.84:0/1751357573 learned_addr learned my addr 172.21.15.84:0/1751357573 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:07.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.162+0000 7f9796119640 1 -- 172.21.15.84:0/1751357573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f979015ae80 msgr2=0x7f97900b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.162+0000 7f9796119640 1 --2- 172.21.15.84:0/1751357573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f979015ae80 0x7f97900b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.162+0000 7f9796119640 1 -- 172.21.15.84:0/1751357573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979015aaa0 msgr2=0x7f97901591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.162+0000 7f9796119640 1 --2- 172.21.15.84:0/1751357573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979015aaa0 0x7f97901591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.162+0000 7f9796119640 1 -- 172.21.15.84:0/1751357573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c0094d0 con 0x7f97900ccaa0 2023-10-16T10:02:07.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.162+0000 7f9796119640 1 --2- 172.21.15.84:0/1751357573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97900ccaa0 0x7f97901f6be0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f978800ce10 tx=0x7f978800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:07.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.163+0000 7f9786ffd640 1 -- 172.21.15.84:0/1751357573 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788003b90 con 0x7f97900ccaa0 2023-10-16T10:02:07.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.163+0000 7f9786ffd640 1 -- 172.21.15.84:0/1751357573 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9788003cf0 con 0x7f97900ccaa0 2023-10-16T10:02:07.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.163+0000 7f9786ffd640 1 -- 172.21.15.84:0/1751357573 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788004240 con 0x7f97900ccaa0 2023-10-16T10:02:07.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.163+0000 7f979837d640 1 -- 172.21.15.84:0/1751357573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97900c5fd0 con 0x7f97900ccaa0 2023-10-16T10:02:07.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.163+0000 7f979837d640 1 -- 172.21.15.84:0/1751357573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97900c6590 con 0x7f97900ccaa0 2023-10-16T10:02:07.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.164+0000 7f979837d640 1 -- 172.21.15.84:0/1751357573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979004e160 con 0x7f97900ccaa0 2023-10-16T10:02:07.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.165+0000 7f9786ffd640 1 -- 172.21.15.84:0/1751357573 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f97880043a0 con 0x7f97900ccaa0 2023-10-16T10:02:07.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.166+0000 7f9786ffd640 1 --2- 172.21.15.84:0/1751357573 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f977c06af90 0x7f977c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.166+0000 7f9786ffd640 1 -- 172.21.15.84:0/1751357573 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5588+0+0 (secure 0 0 0) 0x7f9788012070 con 0x7f97900ccaa0 2023-10-16T10:02:07.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.166+0000 7f9795918640 1 --2- 172.21.15.84:0/1751357573 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f977c06af90 0x7f977c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.167+0000 7f9795918640 1 --2- 172.21.15.84:0/1751357573 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f977c06af90 0x7f977c06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f978c036000 tx=0x7f978c030000 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:07.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.170+0000 7f9786ffd640 1 -- 172.21.15.84:0/1751357573 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9788055150 con 0x7f97900ccaa0 2023-10-16T10:02:07.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.399+0000 7f979837d640 1 -- 172.21.15.84:0/1751357573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f979015e350 con 0x7f97900ccaa0 2023-10-16T10:02:07.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.400+0000 7f9786ffd640 1 -- 172.21.15.84:0/1751357573 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f9788003e60 con 0x7f97900ccaa0 2023-10-16T10:02:07.402 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:02:07.402 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3662158203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3662158203},{"type":"v1","addr":"172.21.15.157:6835","nonce":3662158203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1898363339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1898363339},{"type":"v1","addr":"172.21.15.157:6833","nonce":1898363339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5091,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2502120537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2502120537},{"type":"v1","addr":"172.21.15.84:6836","nonce":2502120537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2715651376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2715651376},{"type":"v1","addr":"172.21.15.84:6837","nonce":2715651376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[]} 2023-10-16T10:02:07.402 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 19 2023-10-16T10:02:07.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.402+0000 7f979837d640 1 -- 172.21.15.84:0/1751357573 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f977c06af90 msgr2=0x7f977c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.402+0000 7f979837d640 1 --2- 172.21.15.84:0/1751357573 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f977c06af90 0x7f977c06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f978c036000 tx=0x7f978c030000 comp rx=0 tx=0).stop 2023-10-16T10:02:07.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.402+0000 7f979837d640 1 -- 172.21.15.84:0/1751357573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97900ccaa0 msgr2=0x7f97901f6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.402+0000 7f979837d640 1 --2- 172.21.15.84:0/1751357573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97900ccaa0 0x7f97901f6be0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f978800ce10 tx=0x7f978800a660 comp rx=0 tx=0).stop 2023-10-16T10:02:07.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.403+0000 7f979837d640 1 -- 172.21.15.84:0/1751357573 shutdown_connections 2023-10-16T10:02:07.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.403+0000 7f979837d640 1 --2- 172.21.15.84:0/1751357573 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f977c06af90 0x7f977c06d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.403+0000 7f979837d640 1 --2- 172.21.15.84:0/1751357573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f979015ae80 0x7f97900b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.403+0000 7f979837d640 1 --2- 172.21.15.84:0/1751357573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979015aaa0 0x7f97901591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.403+0000 7f979837d640 1 --2- 172.21.15.84:0/1751357573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97900ccaa0 0x7f97901f6be0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.403+0000 7f979837d640 1 -- 172.21.15.84:0/1751357573 >> 172.21.15.84:0/1751357573 conn(0x7f97900cb410 msgr2=0x7f97900b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:07.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.403+0000 7f979837d640 1 -- 172.21.15.84:0/1751357573 shutdown_connections 2023-10-16T10:02:07.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.403+0000 7f979837d640 1 -- 172.21.15.84:0/1751357573 wait complete. 2023-10-16T10:02:07.423 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T10:02:07.423 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T10:02:07.424 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T10:02:07.424 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:02:07.424 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T10:02:07.424 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T10:02:07.425 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T10:02:07.425 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:02:07.425 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T10:02:07.426 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T10:02:07.426 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T10:02:07.426 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:02:07.426 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T10:02:07.427 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T10:02:07.427 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T10:02:07.427 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:02:07.442 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T10:02:07.442 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T10:02:07.446 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T10:02:07.446 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T10:02:07.448 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T10:02:07.449 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T10:02:07.455 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T10:02:07.455 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T10:02:07.494 INFO:tasks.ceph.mds.c:Started 2023-10-16T10:02:07.495 DEBUG:teuthology.parallel:result is None 2023-10-16T10:02:07.502 INFO:tasks.ceph.mds.a:Started 2023-10-16T10:02:07.502 DEBUG:teuthology.parallel:result is None 2023-10-16T10:02:07.503 INFO:tasks.ceph.mds.d:Started 2023-10-16T10:02:07.504 DEBUG:teuthology.parallel:result is None 2023-10-16T10:02:07.511 INFO:tasks.ceph.mds.b:Started 2023-10-16T10:02:07.512 DEBUG:teuthology.parallel:result is None 2023-10-16T10:02:07.512 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T10:02:07.658 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T10:02:07.659 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T10:02:07.666 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-10-16T10:02:07.676 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T10:02:07.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.703+0000 7ff8abfff640 1 Processor -- start 2023-10-16T10:02:07.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.703+0000 7ff8abfff640 1 -- start start 2023-10-16T10:02:07.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.704+0000 7ff8abfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff8ac0cded0 0x7ff8ac0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.705+0000 7ff8abfff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8ac15d150 con 0x7ff8ac0b8480 2023-10-16T10:02:07.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.705+0000 7ff8abfff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8ac0d1180 con 0x7ff8ac15cd70 2023-10-16T10:02:07.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.705+0000 7ff8abfff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8ac0d1460 con 0x7ff8ac0cded0 2023-10-16T10:02:07.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.705+0000 7ff8aa7fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff8ac0b8480 0x7ff8ac0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34840/0 (socket says 172.21.15.84:34840) 2023-10-16T10:02:07.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.705+0000 7ff8aa7fc640 1 -- 172.21.15.84:0/1940843462 learned_addr learned my addr 172.21.15.84:0/1940843462 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:07.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.705+0000 7ff8ab7fe640 1 --2- 172.21.15.84:0/1940843462 >> v2:172.21.15.157:3300/0 conn(0x7ff8ac0cded0 0x7ff8ac0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.705+0000 7ff8a9ffb640 1 -- 172.21.15.84:0/1940843462 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285024902 0 0) 0x7ff8ac15d150 con 0x7ff8ac0b8480 2023-10-16T10:02:07.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.705+0000 7ff8a9ffb640 1 -- 172.21.15.84:0/1940843462 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff894003520 con 0x7ff8ac0b8480 2023-10-16T10:02:07.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.705+0000 7ff8ab7fe640 1 -- 172.21.15.84:0/1940843462 >> v1:172.21.15.157:6789/0 conn(0x7ff8ac15cd70 legacy=0x7ff8ac0d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.705+0000 7ff8ab7fe640 1 -- 172.21.15.84:0/1940843462 >> v1:172.21.15.84:6789/0 conn(0x7ff8ac0b8480 legacy=0x7ff8ac0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.706+0000 7ff8ab7fe640 1 -- 172.21.15.84:0/1940843462 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac0d1740 con 0x7ff8ac0cded0 2023-10-16T10:02:07.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.707+0000 7ff8ab7fe640 1 --2- 172.21.15.84:0/1940843462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8ac0cded0 0x7ff8ac0b5c90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4009d90 tx=0x7ff8a402bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31b4e325ee86d145 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:07.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.707+0000 7ff8a9ffb640 1 -- 172.21.15.84:0/1940843462 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a402ccf0 con 0x7ff8ac0cded0 2023-10-16T10:02:07.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.707+0000 7ff8a9ffb640 1 -- 172.21.15.84:0/1940843462 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a402ce50 con 0x7ff8ac0cded0 2023-10-16T10:02:07.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.707+0000 7ff8a9ffb640 1 -- 172.21.15.84:0/1940843462 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4034660 con 0x7ff8ac0cded0 2023-10-16T10:02:07.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.707+0000 7ff8abfff640 1 -- 172.21.15.84:0/1940843462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8ac0cded0 msgr2=0x7ff8ac0b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.707+0000 7ff8abfff640 1 --2- 172.21.15.84:0/1940843462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8ac0cded0 0x7ff8ac0b5c90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4009d90 tx=0x7ff8a402bf40 comp rx=0 tx=0).stop 2023-10-16T10:02:07.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.708+0000 7ff8abfff640 1 -- 172.21.15.84:0/1940843462 shutdown_connections 2023-10-16T10:02:07.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.708+0000 7ff8abfff640 1 --2- 172.21.15.84:0/1940843462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8ac0cded0 0x7ff8ac0b5c90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.708+0000 7ff8abfff640 1 -- 172.21.15.84:0/1940843462 >> 172.21.15.84:0/1940843462 conn(0x7ff8ac0cf570 msgr2=0x7ff8ac0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:07.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.708+0000 7ff8abfff640 1 -- 172.21.15.84:0/1940843462 shutdown_connections 2023-10-16T10:02:07.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.708+0000 7ff8abfff640 1 -- 172.21.15.84:0/1940843462 wait complete. 2023-10-16T10:02:07.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.709+0000 7ff8abfff640 1 Processor -- start 2023-10-16T10:02:07.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.709+0000 7ff8abfff640 1 -- start start 2023-10-16T10:02:07.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.710+0000 7ff8abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8ac0b8480 0x7ff8ac1564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.710+0000 7ff8abfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8ac0cded0 0x7ff8ac158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.710+0000 7ff8abfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8ac15cd70 0x7ff8ac1f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.710+0000 7ff8abfff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac156a00 con 0x7ff8ac0b8480 2023-10-16T10:02:07.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.710+0000 7ff8abfff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac158e50 con 0x7ff8ac15cd70 2023-10-16T10:02:07.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.710+0000 7ff8abfff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac159080 con 0x7ff8ac0cded0 2023-10-16T10:02:07.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.711+0000 7ff8aaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8ac0b8480 0x7ff8ac1564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.711+0000 7ff8aaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8ac0b8480 0x7ff8ac1564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33172/0 (socket says 172.21.15.84:33172) 2023-10-16T10:02:07.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.711+0000 7ff8aaffd640 1 -- 172.21.15.84:0/1983035973 learned_addr learned my addr 172.21.15.84:0/1983035973 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:07.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.711+0000 7ff8aa7fc640 1 --2- 172.21.15.84:0/1983035973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8ac0cded0 0x7ff8ac158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.711+0000 7ff8aaffd640 1 -- 172.21.15.84:0/1983035973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8ac0cded0 msgr2=0x7ff8ac158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.711+0000 7ff8aaffd640 1 --2- 172.21.15.84:0/1983035973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8ac0cded0 0x7ff8ac158910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.711+0000 7ff8aaffd640 1 -- 172.21.15.84:0/1983035973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8ac15cd70 msgr2=0x7ff8ac1f24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.711+0000 7ff8ab7fe640 1 --2- 172.21.15.84:0/1983035973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8ac15cd70 0x7ff8ac1f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.711+0000 7ff8aaffd640 1 --2- 172.21.15.84:0/1983035973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8ac15cd70 0x7ff8ac1f24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.711+0000 7ff8aaffd640 1 -- 172.21.15.84:0/1983035973 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a4009a70 con 0x7ff8ac0b8480 2023-10-16T10:02:07.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.711+0000 7ff8ab7fe640 1 --2- 172.21.15.84:0/1983035973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8ac15cd70 0x7ff8ac1f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:07.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.711+0000 7ff8aaffd640 1 --2- 172.21.15.84:0/1983035973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8ac0b8480 0x7ff8ac1564c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00c400 tx=0x7ff89c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:07.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.712+0000 7ff8937fe640 1 -- 172.21.15.84:0/1983035973 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c0052d0 con 0x7ff8ac0b8480 2023-10-16T10:02:07.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.712+0000 7ff8937fe640 1 -- 172.21.15.84:0/1983035973 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89c005430 con 0x7ff8ac0b8480 2023-10-16T10:02:07.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.712+0000 7ff8abfff640 1 -- 172.21.15.84:0/1983035973 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ac159360 con 0x7ff8ac0b8480 2023-10-16T10:02:07.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.712+0000 7ff8937fe640 1 -- 172.21.15.84:0/1983035973 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c003830 con 0x7ff8ac0b8480 2023-10-16T10:02:07.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.712+0000 7ff8abfff640 1 -- 172.21.15.84:0/1983035973 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ac15e810 con 0x7ff8ac0b8480 2023-10-16T10:02:07.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.713+0000 7ff8abfff640 1 -- 172.21.15.84:0/1983035973 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8ac15e390 con 0x7ff8ac0b8480 2023-10-16T10:02:07.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.714+0000 7ff8937fe640 1 -- 172.21.15.84:0/1983035973 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff89c003990 con 0x7ff8ac0b8480 2023-10-16T10:02:07.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.715+0000 7ff8937fe640 1 --2- 172.21.15.84:0/1983035973 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7ff89806af90 0x7ff89806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.716+0000 7ff8937fe640 1 -- 172.21.15.84:0/1983035973 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5588+0+0 (secure 0 0 0) 0x7ff89c088bf0 con 0x7ff8ac0b8480 2023-10-16T10:02:07.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.716+0000 7ff8aa7fc640 1 --2- 172.21.15.84:0/1983035973 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7ff89806af90 0x7ff89806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.716+0000 7ff8aa7fc640 1 --2- 172.21.15.84:0/1983035973 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7ff89806af90 0x7ff89806d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00011e0 tx=0x7ff8a0009040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:07.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.720+0000 7ff8937fe640 1 -- 172.21.15.84:0/1983035973 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff8ac15e390 con 0x7ff8ac0b8480 2023-10-16T10:02:07.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:07.929+0000 7ff8abfff640 1 -- 172.21.15.84:0/1983035973 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff8ac15e390 con 0x7ff8ac0b8480 2023-10-16T10:02:08.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:08.868+0000 7ff8937fe640 1 -- 172.21.15.84:0/1983035973 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v60) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff8ac15e390 con 0x7ff8ac0b8480 2023-10-16T10:02:08.871 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-10-16T10:02:08.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:08.873+0000 7ff8abfff640 1 -- 172.21.15.84:0/1983035973 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7ff89806af90 msgr2=0x7ff89806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:08.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:08.873+0000 7ff8abfff640 1 --2- 172.21.15.84:0/1983035973 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7ff89806af90 0x7ff89806d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00011e0 tx=0x7ff8a0009040 comp rx=0 tx=0).stop 2023-10-16T10:02:08.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:08.873+0000 7ff8abfff640 1 -- 172.21.15.84:0/1983035973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8ac0b8480 msgr2=0x7ff8ac1564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:08.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:08.873+0000 7ff8abfff640 1 --2- 172.21.15.84:0/1983035973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8ac0b8480 0x7ff8ac1564c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00c400 tx=0x7ff89c00c8c0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:08.873+0000 7ff8abfff640 1 -- 172.21.15.84:0/1983035973 shutdown_connections 2023-10-16T10:02:08.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:08.873+0000 7ff8abfff640 1 --2- 172.21.15.84:0/1983035973 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7ff89806af90 0x7ff89806d450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:08.873+0000 7ff8abfff640 1 --2- 172.21.15.84:0/1983035973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8ac15cd70 0x7ff8ac1f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:08.873+0000 7ff8abfff640 1 --2- 172.21.15.84:0/1983035973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8ac0cded0 0x7ff8ac158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:08.873+0000 7ff8abfff640 1 --2- 172.21.15.84:0/1983035973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8ac0b8480 0x7ff8ac1564c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:08.873+0000 7ff8abfff640 1 -- 172.21.15.84:0/1983035973 >> 172.21.15.84:0/1983035973 conn(0x7ff8ac0cf570 msgr2=0x7ff8ac0b7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:08.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:08.873+0000 7ff8abfff640 1 -- 172.21.15.84:0/1983035973 shutdown_connections 2023-10-16T10:02:08.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:08.873+0000 7ff8abfff640 1 -- 172.21.15.84:0/1983035973 wait complete. 2023-10-16T10:02:08.895 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T10:02:09.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.113+0000 7f4a6f59e640 1 Processor -- start 2023-10-16T10:02:09.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.113+0000 7f4a6f59e640 1 -- start start 2023-10-16T10:02:09.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.114+0000 7f4a6f59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4a70159950 0x7f4a7015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.114+0000 7f4a6f59e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a7015aef0 con 0x7f4a70070450 2023-10-16T10:02:09.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.114+0000 7f4a6f59e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a7015d910 con 0x7f4a7006c810 2023-10-16T10:02:09.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.115+0000 7f4a6f59e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a7015e490 con 0x7f4a70159950 2023-10-16T10:02:09.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.115+0000 7f4a67fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4a70070450 0x7f4a7006c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34842/0 (socket says 172.21.15.84:34842) 2023-10-16T10:02:09.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.115+0000 7f4a67fff640 1 -- 172.21.15.84:0/3753746565 learned_addr learned my addr 172.21.15.84:0/3753746565 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:09.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.115+0000 7f4a6dd9b640 1 -- 172.21.15.84:0/3753746565 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3900863393 0 0) 0x7f4a58001110 con 0x7f4a70070450 2023-10-16T10:02:09.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.115+0000 7f4a6e59c640 1 --2- 172.21.15.84:0/3753746565 >> v2:172.21.15.157:3300/0 conn(0x7f4a70159950 0x7f4a7015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.115+0000 7f4a6dd9b640 1 -- 172.21.15.84:0/3753746565 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a5c003520 con 0x7f4a70070450 2023-10-16T10:02:09.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.115+0000 7f4a6dd9b640 1 -- 172.21.15.84:0/3753746565 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 276207627 0 0) 0x7f4a5c003520 con 0x7f4a70070450 2023-10-16T10:02:09.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.115+0000 7f4a6dd9b640 1 -- 172.21.15.84:0/3753746565 >> v2:172.21.15.157:3300/0 conn(0x7f4a70159950 msgr2=0x7f4a7015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:09.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.115+0000 7f4a6dd9b640 1 --2- 172.21.15.84:0/3753746565 >> v2:172.21.15.157:3300/0 conn(0x7f4a70159950 0x7f4a7015a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.115+0000 7f4a6dd9b640 1 -- 172.21.15.84:0/3753746565 >> v1:172.21.15.157:6789/0 conn(0x7f4a7006c810 legacy=0x7f4a700c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:09.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.115+0000 7f4a6dd9b640 1 -- 172.21.15.84:0/3753746565 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7015dbf0 con 0x7f4a70070450 2023-10-16T10:02:09.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.116+0000 7f4a6dd9b640 1 -- 172.21.15.84:0/3753746565 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f4a58001800 con 0x7f4a70070450 2023-10-16T10:02:09.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.116+0000 7f4a6dd9b640 1 -- 172.21.15.84:0/3753746565 >> v1:172.21.15.84:6789/0 conn(0x7f4a70070450 legacy=0x7f4a7006c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.116+0000 7f4a6dd9b640 1 --2- 172.21.15.84:0/3753746565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a5c0039c0 0x7f4a5c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.116+0000 7f4a6dd9b640 1 --2- 172.21.15.84:0/3753746565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a5c0047c0 0x7f4a5c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.116+0000 7f4a6dd9b640 1 --2- 172.21.15.84:0/3753746565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a5c005250 0x7f4a5c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.116+0000 7f4a6dd9b640 1 -- 172.21.15.84:0/3753746565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7015e490 con 0x7f4a5c0039c0 2023-10-16T10:02:09.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.116+0000 7f4a6dd9b640 1 -- 172.21.15.84:0/3753746565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c00c1a0 con 0x7f4a5c005250 2023-10-16T10:02:09.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.116+0000 7f4a6dd9b640 1 -- 172.21.15.84:0/3753746565 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c00c360 con 0x7f4a5c0047c0 2023-10-16T10:02:09.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.116+0000 7f4a6e59c640 1 --2- 172.21.15.84:0/3753746565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a5c0039c0 0x7f4a5c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.116+0000 7f4a6ed9d640 1 --2- 172.21.15.84:0/3753746565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a5c005250 0x7f4a5c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.116+0000 7f4a6e59c640 1 -- 172.21.15.84:0/3753746565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a5c0047c0 msgr2=0x7f4a5c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:09.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.116+0000 7f4a6e59c640 1 --2- 172.21.15.84:0/3753746565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a5c0047c0 0x7f4a5c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.116+0000 7f4a6e59c640 1 -- 172.21.15.84:0/3753746565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a5c005250 msgr2=0x7f4a5c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:09.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.116+0000 7f4a6e59c640 1 --2- 172.21.15.84:0/3753746565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a5c005250 0x7f4a5c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.116+0000 7f4a6e59c640 1 -- 172.21.15.84:0/3753746565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5c003520 con 0x7f4a5c0039c0 2023-10-16T10:02:09.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.116+0000 7f4a6ed9d640 1 --2- 172.21.15.84:0/3753746565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a5c005250 0x7f4a5c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:09.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.116+0000 7f4a6ed9d640 1 -- 172.21.15.84:0/3753746565 reap_dead start 2023-10-16T10:02:09.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.117+0000 7f4a6e59c640 1 --2- 172.21.15.84:0/3753746565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a5c0039c0 0x7f4a5c003dc0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000ba30 tx=0x7f4a5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502c9f2e025be622 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:09.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.118+0000 7f4a6dd9b640 1 -- 172.21.15.84:0/3753746565 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a60032630 con 0x7f4a5c0039c0 2023-10-16T10:02:09.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.118+0000 7f4a6dd9b640 1 -- 172.21.15.84:0/3753746565 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a60032790 con 0x7f4a5c0039c0 2023-10-16T10:02:09.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.118+0000 7f4a6dd9b640 1 -- 172.21.15.84:0/3753746565 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a60032b20 con 0x7f4a5c0039c0 2023-10-16T10:02:09.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.118+0000 7f4a6f59e640 1 -- 172.21.15.84:0/3753746565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a5c0039c0 msgr2=0x7f4a5c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.118+0000 7f4a6f59e640 1 --2- 172.21.15.84:0/3753746565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a5c0039c0 0x7f4a5c003dc0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000ba30 tx=0x7f4a5c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.118+0000 7f4a6f59e640 1 -- 172.21.15.84:0/3753746565 shutdown_connections 2023-10-16T10:02:09.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.118+0000 7f4a6f59e640 1 --2- 172.21.15.84:0/3753746565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a5c0039c0 0x7f4a5c003dc0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.118+0000 7f4a6f59e640 1 -- 172.21.15.84:0/3753746565 >> 172.21.15.84:0/3753746565 conn(0x7f4a700ce6a0 msgr2=0x7f4a700ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:09.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.118+0000 7f4a6f59e640 1 -- 172.21.15.84:0/3753746565 shutdown_connections 2023-10-16T10:02:09.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.119+0000 7f4a6f59e640 1 -- 172.21.15.84:0/3753746565 wait complete. 2023-10-16T10:02:09.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.120+0000 7f4a6f59e640 1 Processor -- start 2023-10-16T10:02:09.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.120+0000 7f4a6f59e640 1 -- start start 2023-10-16T10:02:09.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.121+0000 7f4a6f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a5c0039c0 0x7f4a700c2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.121+0000 7f4a6f59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a700b3e70 0x7f4a700b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.121+0000 7f4a6f59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a700acac0 0x7f4a700acf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.121+0000 7f4a6f59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a700ad480 con 0x7f4a5c0039c0 2023-10-16T10:02:09.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.121+0000 7f4a6f59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a700c31a0 con 0x7f4a700acac0 2023-10-16T10:02:09.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.121+0000 7f4a6f59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a701f6160 con 0x7f4a700b3e70 2023-10-16T10:02:09.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.122+0000 7f4a6e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a5c0039c0 0x7f4a700c2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.122+0000 7f4a6e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a5c0039c0 0x7f4a700c2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33188/0 (socket says 172.21.15.84:33188) 2023-10-16T10:02:09.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.122+0000 7f4a6e59c640 1 -- 172.21.15.84:0/1327274544 learned_addr learned my addr 172.21.15.84:0/1327274544 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:09.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.122+0000 7f4a6e59c640 1 -- 172.21.15.84:0/1327274544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a700b3e70 msgr2=0x7f4a700b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.122+0000 7f4a6e59c640 1 --2- 172.21.15.84:0/1327274544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a700b3e70 0x7f4a700b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.122+0000 7f4a6e59c640 1 -- 172.21.15.84:0/1327274544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a700acac0 msgr2=0x7f4a700acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.122+0000 7f4a6ed9d640 1 --2- 172.21.15.84:0/1327274544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a700acac0 0x7f4a700acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.122+0000 7f4a6e59c640 1 --2- 172.21.15.84:0/1327274544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a700acac0 0x7f4a700acf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.122+0000 7f4a6e59c640 1 -- 172.21.15.84:0/1327274544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a701f6440 con 0x7f4a5c0039c0 2023-10-16T10:02:09.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.122+0000 7f4a6e59c640 1 --2- 172.21.15.84:0/1327274544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a5c0039c0 0x7f4a700c2c60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4a6002beb0 tx=0x7f4a60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:09.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.123+0000 7f4a66ffd640 1 -- 172.21.15.84:0/1327274544 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6003ba70 con 0x7f4a5c0039c0 2023-10-16T10:02:09.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.123+0000 7f4a66ffd640 1 -- 172.21.15.84:0/1327274544 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6003bbd0 con 0x7f4a5c0039c0 2023-10-16T10:02:09.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.123+0000 7f4a66ffd640 1 -- 172.21.15.84:0/1327274544 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a60030400 con 0x7f4a5c0039c0 2023-10-16T10:02:09.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.123+0000 7f4a6f59e640 1 -- 172.21.15.84:0/1327274544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a701f6720 con 0x7f4a5c0039c0 2023-10-16T10:02:09.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.123+0000 7f4a6f59e640 1 -- 172.21.15.84:0/1327274544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a700b42f0 con 0x7f4a5c0039c0 2023-10-16T10:02:09.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.124+0000 7f4a6f59e640 1 -- 172.21.15.84:0/1327274544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a7004e9b0 con 0x7f4a5c0039c0 2023-10-16T10:02:09.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.125+0000 7f4a66ffd640 1 -- 172.21.15.84:0/1327274544 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4a60030560 con 0x7f4a5c0039c0 2023-10-16T10:02:09.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.126+0000 7f4a66ffd640 1 --2- 172.21.15.84:0/1327274544 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f4a5006afb0 0x7f4a5006d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.126+0000 7f4a66ffd640 1 -- 172.21.15.84:0/1327274544 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4a600abc70 con 0x7f4a5c0039c0 2023-10-16T10:02:09.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.126+0000 7f4a67fff640 1 --2- 172.21.15.84:0/1327274544 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f4a5006afb0 0x7f4a5006d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.127+0000 7f4a67fff640 1 --2- 172.21.15.84:0/1327274544 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f4a5006afb0 0x7f4a5006d470 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4a58001930 tx=0x7f4a580015c0 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:09.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.130+0000 7f4a66ffd640 1 -- 172.21.15.84:0/1327274544 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4a60076db0 con 0x7f4a5c0039c0 2023-10-16T10:02:09.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.392+0000 7f4a6f59e640 1 -- 172.21.15.84:0/1327274544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4a700c9330 con 0x7f4a5c0039c0 2023-10-16T10:02:09.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.393+0000 7f4a66ffd640 1 -- 172.21.15.84:0/1327274544 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4a60036020 con 0x7f4a5c0039c0 2023-10-16T10:02:09.395 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:02:09.395 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T10:02:09.396 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-10-16T10:02:09.396 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.396 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-10-16T10:02:09.396 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l6dhjKRAA0pMvY6AK6qtFXDprNhwsqg==", 2023-10-16T10:02:09.396 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.397 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:02:09.397 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:02:09.397 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:02:09.397 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:02:09.397 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.398 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:02:09.398 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.398 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-10-16T10:02:09.398 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lFpMxEBAA0TLPRt5q3FwwBvMG019cjQ==", 2023-10-16T10:02:09.398 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.399 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:02:09.399 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:02:09.399 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:02:09.399 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:02:09.399 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.400 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:02:09.400 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.400 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-10-16T10:02:09.400 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l73RBOhAAOMEPxfUAMHhn5Blyvd5oMw==", 2023-10-16T10:02:09.400 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.400 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:02:09.401 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:02:09.401 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:02:09.401 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:02:09.401 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.401 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:02:09.401 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.402 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-10-16T10:02:09.402 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lcTIRIhAA0+FWm4b4yw77LD3+WKbk7w==", 2023-10-16T10:02:09.404 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.404 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:02:09.404 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:02:09.405 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:02:09.405 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:02:09.405 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.405 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:02:09.405 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.406 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-10-16T10:02:09.406 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDYBi1l1zBLERAA3YLW3xIYQoRVw7KSkwN3Og==", 2023-10-16T10:02:09.406 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.406 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:02:09.407 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:02:09.407 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:02:09.407 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.407 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:02:09.407 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.408 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-10-16T10:02:09.408 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDbBi1lFUUxKRAAQbus8SmYprJRmWTjoCznNA==", 2023-10-16T10:02:09.408 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.408 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:02:09.408 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:02:09.408 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:02:09.409 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.409 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:02:09.409 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.409 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-10-16T10:02:09.409 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDeBi1l+9lBMxAArl6sb/h15uBG/a9lVH40Hg==", 2023-10-16T10:02:09.410 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.410 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:02:09.410 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:02:09.410 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:02:09.410 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.411 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:02:09.411 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.411 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-10-16T10:02:09.411 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDiBi1l2rnEERAA3oRD71xa0fEOynM8zKuBEg==", 2023-10-16T10:02:09.411 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.411 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:02:09.412 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:02:09.412 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:02:09.412 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.412 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:02:09.412 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.413 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-10-16T10:02:09.413 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDnBi1lR7oLChAAZto8fDovDbpOd3Fcnc7C6w==", 2023-10-16T10:02:09.413 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.413 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:02:09.413 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:02:09.413 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:02:09.414 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.414 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:02:09.414 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.414 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-10-16T10:02:09.414 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDqBi1lmjg/JBAAlINKRGBc0/UWJJHMVFw57w==", 2023-10-16T10:02:09.415 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.415 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:02:09.415 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:02:09.415 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:02:09.415 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.416 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:02:09.416 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.416 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-10-16T10:02:09.416 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDuBi1lu0kMARAAtuSeM8F3yr8CVrc3p1biaQ==", 2023-10-16T10:02:09.416 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.417 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:02:09.417 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:02:09.417 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:02:09.417 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.417 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:02:09.417 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.418 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-10-16T10:02:09.418 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDxBi1lf6mNGBAAHGbqzJ0fmDxQIsasboHMfg==", 2023-10-16T10:02:09.418 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.418 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:02:09.418 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:02:09.419 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:02:09.419 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.419 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:02:09.419 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.419 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-10-16T10:02:09.420 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ==", 2023-10-16T10:02:09.420 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.420 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:02:09.420 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow r", 2023-10-16T10:02:09.420 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T10:02:09.421 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.421 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:02:09.421 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.421 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-10-16T10:02:09.421 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1lHU+4BxAAd8ET8hQ8Y+EUwdTJr9/+Ig==", 2023-10-16T10:02:09.422 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.422 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:02:09.422 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:02:09.422 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:02:09.422 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:02:09.423 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.423 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:02:09.423 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.423 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-10-16T10:02:09.423 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lA1BOORAAFNyfqgEF34MVR9YX3KBKkQ==", 2023-10-16T10:02:09.423 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.424 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:02:09.424 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:02:09.424 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:02:09.424 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:02:09.424 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.425 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:02:09.425 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.425 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-10-16T10:02:09.425 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1l848oERAANMfrZz9JrlgNVh7ZFbtnaA==", 2023-10-16T10:02:09.425 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.426 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:02:09.426 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:02:09.426 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:02:09.426 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:02:09.426 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.427 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:02:09.427 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.427 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-10-16T10:02:09.427 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDRBi1lTAxTGRAAMptxnxy3Qit/cTCXie6CrQ==", 2023-10-16T10:02:09.427 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.428 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:02:09.428 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:02:09.428 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:02:09.428 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:02:09.428 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.428 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:02:09.429 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.429 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-10-16T10:02:09.429 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lPuJxHRAABAnGPVCrkq0JR4Y6iFGZlw==", 2023-10-16T10:02:09.429 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.429 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:02:09.430 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T10:02:09.430 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:02:09.430 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.430 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:02:09.430 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:09.431 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-10-16T10:02:09.431 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lLxnKEBAA+a/pOHUp7Sg3m4weAust5w==", 2023-10-16T10:02:09.431 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:02:09.431 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:02:09.431 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T10:02:09.432 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:02:09.432 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.432 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:09.432 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-10-16T10:02:09.432 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T10:02:09.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.396+0000 7f4a6f59e640 1 -- 172.21.15.84:0/1327274544 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f4a5006afb0 msgr2=0x7f4a5006d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.396+0000 7f4a6f59e640 1 --2- 172.21.15.84:0/1327274544 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f4a5006afb0 0x7f4a5006d470 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4a58001930 tx=0x7f4a580015c0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.396+0000 7f4a6f59e640 1 -- 172.21.15.84:0/1327274544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a5c0039c0 msgr2=0x7f4a700c2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.396+0000 7f4a6f59e640 1 --2- 172.21.15.84:0/1327274544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a5c0039c0 0x7f4a700c2c60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4a6002beb0 tx=0x7f4a60004730 comp rx=0 tx=0).stop 2023-10-16T10:02:09.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.396+0000 7f4a6f59e640 1 -- 172.21.15.84:0/1327274544 shutdown_connections 2023-10-16T10:02:09.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.396+0000 7f4a6f59e640 1 --2- 172.21.15.84:0/1327274544 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f4a5006afb0 0x7f4a5006d470 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.396+0000 7f4a6f59e640 1 --2- 172.21.15.84:0/1327274544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a700acac0 0x7f4a700acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.396+0000 7f4a6f59e640 1 --2- 172.21.15.84:0/1327274544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a700b3e70 0x7f4a700b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.396+0000 7f4a6f59e640 1 --2- 172.21.15.84:0/1327274544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a5c0039c0 0x7f4a700c2c60 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.396+0000 7f4a6f59e640 1 -- 172.21.15.84:0/1327274544 >> 172.21.15.84:0/1327274544 conn(0x7f4a700ce6a0 msgr2=0x7f4a700ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:09.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.396+0000 7f4a6f59e640 1 -- 172.21.15.84:0/1327274544 shutdown_connections 2023-10-16T10:02:09.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.396+0000 7f4a6f59e640 1 -- 172.21.15.84:0/1327274544 wait complete. 2023-10-16T10:02:09.437 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:02:09.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.654+0000 7fabf2ebb640 1 Processor -- start 2023-10-16T10:02:09.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.654+0000 7fabf2ebb640 1 -- start start 2023-10-16T10:02:09.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.655+0000 7fabf2ebb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabec06fc60 0x7fabec165250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.655+0000 7fabf2ebb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabec0c9b20 con 0x7fabec0caa10 2023-10-16T10:02:09.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.655+0000 7fabf2ebb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabec0c9e00 con 0x7fabec1593b0 2023-10-16T10:02:09.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.655+0000 7fabf2ebb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec159840 con 0x7fabec06fc60 2023-10-16T10:02:09.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.656+0000 7fabf0c57640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fabec0caa10 0x7fabec0b0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34850/0 (socket says 172.21.15.84:34850) 2023-10-16T10:02:09.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.656+0000 7fabf0c57640 1 -- 172.21.15.84:0/3671938092 learned_addr learned my addr 172.21.15.84:0/3671938092 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:09.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.656+0000 7fabeb7fe640 1 -- 172.21.15.84:0/3671938092 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1703684265 0 0) 0x7fabec0c9b20 con 0x7fabec0caa10 2023-10-16T10:02:09.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.656+0000 7fabeb7fe640 1 -- 172.21.15.84:0/3671938092 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabdc005730 con 0x7fabec0caa10 2023-10-16T10:02:09.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.656+0000 7fabf1458640 1 --2- 172.21.15.84:0/3671938092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabec06fc60 0x7fabec165250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.656+0000 7fabeb7fe640 1 -- 172.21.15.84:0/3671938092 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1263198149 0 0) 0x7fabdc005730 con 0x7fabec0caa10 2023-10-16T10:02:09.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.656+0000 7fabeb7fe640 1 -- 172.21.15.84:0/3671938092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabec06fc60 msgr2=0x7fabec165250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:09.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.656+0000 7fabeb7fe640 1 --2- 172.21.15.84:0/3671938092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabec06fc60 0x7fabec165250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.656+0000 7fabeb7fe640 1 -- 172.21.15.84:0/3671938092 >> v1:172.21.15.157:6789/0 conn(0x7fabec1593b0 legacy=0x7fabec06f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.657+0000 7fabeb7fe640 1 -- 172.21.15.84:0/3671938092 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec0ca0e0 con 0x7fabec0caa10 2023-10-16T10:02:09.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.657+0000 7fabeb7fe640 1 -- 172.21.15.84:0/3671938092 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fabe0001670 con 0x7fabec0caa10 2023-10-16T10:02:09.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.657+0000 7fabeb7fe640 1 -- 172.21.15.84:0/3671938092 >> v1:172.21.15.84:6789/0 conn(0x7fabec0caa10 legacy=0x7fabec0b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.657+0000 7fabeb7fe640 1 --2- 172.21.15.84:0/3671938092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc005bb0 0x7fabdc005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.657+0000 7fabeb7fe640 1 --2- 172.21.15.84:0/3671938092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabdc0069d0 0x7fabdc006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.657+0000 7fabeb7fe640 1 --2- 172.21.15.84:0/3671938092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabec06fc60 0x7fabdc00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.657+0000 7fabeb7fe640 1 -- 172.21.15.84:0/3671938092 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc0077a0 con 0x7fabdc005bb0 2023-10-16T10:02:09.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.657+0000 7fabeb7fe640 1 -- 172.21.15.84:0/3671938092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc00e0d0 con 0x7fabdc0069d0 2023-10-16T10:02:09.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.657+0000 7fabeb7fe640 1 -- 172.21.15.84:0/3671938092 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc00e210 con 0x7fabec06fc60 2023-10-16T10:02:09.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.657+0000 7fabf0c57640 1 --2- 172.21.15.84:0/3671938092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc005bb0 0x7fabdc005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.657+0000 7fabf1458640 1 --2- 172.21.15.84:0/3671938092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabec06fc60 0x7fabdc00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.657+0000 7fabebfff640 1 --2- 172.21.15.84:0/3671938092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabdc0069d0 0x7fabdc006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.657+0000 7fabf0c57640 1 -- 172.21.15.84:0/3671938092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabec06fc60 msgr2=0x7fabdc00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:09.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.657+0000 7fabf0c57640 1 --2- 172.21.15.84:0/3671938092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabec06fc60 0x7fabdc00db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.657+0000 7fabf0c57640 1 -- 172.21.15.84:0/3671938092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabdc0069d0 msgr2=0x7fabdc006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:09.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.657+0000 7fabf0c57640 1 --2- 172.21.15.84:0/3671938092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabdc0069d0 0x7fabdc006e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.657+0000 7fabf0c57640 1 -- 172.21.15.84:0/3671938092 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc005730 con 0x7fabdc005bb0 2023-10-16T10:02:09.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.659+0000 7fabf0c57640 1 --2- 172.21.15.84:0/3671938092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc005bb0 0x7fabdc005fd0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fabe000c190 tx=0x7fabdc007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6283cfa9ed5f70ca server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:09.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.659+0000 7fabeb7fe640 1 -- 172.21.15.84:0/3671938092 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe0005ad0 con 0x7fabdc005bb0 2023-10-16T10:02:09.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.659+0000 7fabeb7fe640 1 -- 172.21.15.84:0/3671938092 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe0005c50 con 0x7fabdc005bb0 2023-10-16T10:02:09.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.659+0000 7fabeb7fe640 1 -- 172.21.15.84:0/3671938092 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe002c810 con 0x7fabdc005bb0 2023-10-16T10:02:09.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.660+0000 7fabf2ebb640 1 -- 172.21.15.84:0/3671938092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc005bb0 msgr2=0x7fabdc005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.660+0000 7fabf2ebb640 1 --2- 172.21.15.84:0/3671938092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc005bb0 0x7fabdc005fd0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fabe000c190 tx=0x7fabdc007a00 comp rx=0 tx=0).stop 2023-10-16T10:02:09.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.660+0000 7fabf1458640 1 -- 172.21.15.84:0/3671938092 reap_dead start 2023-10-16T10:02:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.660+0000 7fabf2ebb640 1 -- 172.21.15.84:0/3671938092 shutdown_connections 2023-10-16T10:02:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.660+0000 7fabf2ebb640 1 -- 172.21.15.84:0/3671938092 >> 172.21.15.84:0/3671938092 conn(0x7fabec0cb410 msgr2=0x7fabec0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.660+0000 7fabf2ebb640 1 -- 172.21.15.84:0/3671938092 shutdown_connections 2023-10-16T10:02:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.660+0000 7fabf2ebb640 1 -- 172.21.15.84:0/3671938092 wait complete. 2023-10-16T10:02:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.662+0000 7fabf2ebb640 1 Processor -- start 2023-10-16T10:02:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.662+0000 7fabf2ebb640 1 -- start start 2023-10-16T10:02:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.663+0000 7fabf2ebb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc005bb0 0x7fabec0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.663+0000 7fabf2ebb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabec153930 0x7fabec153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.663+0000 7fabf2ebb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabec0c7ca0 0x7fabec0c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.663+0000 7fabf2ebb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec1542f0 con 0x7fabdc005bb0 2023-10-16T10:02:09.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.663+0000 7fabf2ebb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0b2a00 con 0x7fabec153930 2023-10-16T10:02:09.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.663+0000 7fabf2ebb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0b3a10 con 0x7fabec0c7ca0 2023-10-16T10:02:09.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.663+0000 7fabf0c57640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc005bb0 0x7fabec0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.663+0000 7fabf0c57640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc005bb0 0x7fabec0affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33202/0 (socket says 172.21.15.84:33202) 2023-10-16T10:02:09.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.663+0000 7fabf0c57640 1 -- 172.21.15.84:0/2235601312 learned_addr learned my addr 172.21.15.84:0/2235601312 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:09.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.663+0000 7fabebfff640 1 --2- 172.21.15.84:0/2235601312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabec153930 0x7fabec153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.663+0000 7fabf1458640 1 --2- 172.21.15.84:0/2235601312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabec0c7ca0 0x7fabec0c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.663+0000 7fabf0c57640 1 -- 172.21.15.84:0/2235601312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabec0c7ca0 msgr2=0x7fabec0c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.663+0000 7fabf0c57640 1 --2- 172.21.15.84:0/2235601312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabec0c7ca0 0x7fabec0c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.663+0000 7fabf0c57640 1 -- 172.21.15.84:0/2235601312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabec153930 msgr2=0x7fabec153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.663+0000 7fabf0c57640 1 --2- 172.21.15.84:0/2235601312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabec153930 0x7fabec153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.663+0000 7fabf0c57640 1 -- 172.21.15.84:0/2235601312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec0b2c50 con 0x7fabdc005bb0 2023-10-16T10:02:09.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.664+0000 7fabf0c57640 1 --2- 172.21.15.84:0/2235601312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc005bb0 0x7fabec0affb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fabe002c3b0 tx=0x7fabe0001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:09.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.664+0000 7fabe97fa640 1 -- 172.21.15.84:0/2235601312 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe0039070 con 0x7fabdc005bb0 2023-10-16T10:02:09.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.664+0000 7fabe97fa640 1 -- 172.21.15.84:0/2235601312 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe002cc60 con 0x7fabdc005bb0 2023-10-16T10:02:09.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.664+0000 7fabe97fa640 1 -- 172.21.15.84:0/2235601312 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe003c040 con 0x7fabdc005bb0 2023-10-16T10:02:09.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.664+0000 7fabf2ebb640 1 -- 172.21.15.84:0/2235601312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabec0b2f30 con 0x7fabdc005bb0 2023-10-16T10:02:09.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.664+0000 7fabf2ebb640 1 -- 172.21.15.84:0/2235601312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabec0b34f0 con 0x7fabdc005bb0 2023-10-16T10:02:09.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.665+0000 7fabf2ebb640 1 -- 172.21.15.84:0/2235601312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabec0b0b90 con 0x7fabdc005bb0 2023-10-16T10:02:09.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.666+0000 7fabe97fa640 1 -- 172.21.15.84:0/2235601312 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fabe0032070 con 0x7fabdc005bb0 2023-10-16T10:02:09.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.667+0000 7fabe97fa640 1 --2- 172.21.15.84:0/2235601312 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fabd006afc0 0x7fabd006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.667+0000 7fabe97fa640 1 -- 172.21.15.84:0/2235601312 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fabe00aa800 con 0x7fabdc005bb0 2023-10-16T10:02:09.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.667+0000 7fabebfff640 1 --2- 172.21.15.84:0/2235601312 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fabd006afc0 0x7fabd006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.668+0000 7fabebfff640 1 --2- 172.21.15.84:0/2235601312 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fabd006afc0 0x7fabd006d480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fabec0afce0 tx=0x7fabd8009040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:09.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.671+0000 7fabe97fa640 1 -- 172.21.15.84:0/2235601312 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fabe00759c0 con 0x7fabdc005bb0 2023-10-16T10:02:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.863+0000 7fabf2ebb640 1 -- 172.21.15.84:0/2235601312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabec04e160 con 0x7fabdc005bb0 2023-10-16T10:02:09.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.865+0000 7fabe97fa640 1 -- 172.21.15.84:0/2235601312 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+12833 (secure 0 0 0) 0x7fabe0078fe0 con 0x7fabdc005bb0 2023-10-16T10:02:09.867 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:02:09.867 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":60,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:02:08.866139+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:02:09.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.868+0000 7fabf2ebb640 1 -- 172.21.15.84:0/2235601312 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fabd006afc0 msgr2=0x7fabd006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.868+0000 7fabf2ebb640 1 --2- 172.21.15.84:0/2235601312 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fabd006afc0 0x7fabd006d480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fabec0afce0 tx=0x7fabd8009040 comp rx=0 tx=0).stop 2023-10-16T10:02:09.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.868+0000 7fabf2ebb640 1 -- 172.21.15.84:0/2235601312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc005bb0 msgr2=0x7fabec0affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.868+0000 7fabf2ebb640 1 --2- 172.21.15.84:0/2235601312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc005bb0 0x7fabec0affb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fabe002c3b0 tx=0x7fabe0001930 comp rx=0 tx=0).stop 2023-10-16T10:02:09.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.868+0000 7fabf2ebb640 1 -- 172.21.15.84:0/2235601312 shutdown_connections 2023-10-16T10:02:09.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.868+0000 7fabf2ebb640 1 --2- 172.21.15.84:0/2235601312 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fabd006afc0 0x7fabd006d480 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.868+0000 7fabf2ebb640 1 --2- 172.21.15.84:0/2235601312 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabec0c7ca0 0x7fabec0c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.868+0000 7fabf2ebb640 1 --2- 172.21.15.84:0/2235601312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabec153930 0x7fabec153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.868+0000 7fabf2ebb640 1 --2- 172.21.15.84:0/2235601312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc005bb0 0x7fabec0affb0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.868+0000 7fabf2ebb640 1 -- 172.21.15.84:0/2235601312 >> 172.21.15.84:0/2235601312 conn(0x7fabec0cb410 msgr2=0x7fabec06c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:09.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.868+0000 7fabf2ebb640 1 -- 172.21.15.84:0/2235601312 shutdown_connections 2023-10-16T10:02:09.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:09.869+0000 7fabf2ebb640 1 -- 172.21.15.84:0/2235601312 wait complete. 2023-10-16T10:02:09.889 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:02:09.890 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:02:10.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.110+0000 7fb477ac9640 1 Processor -- start 2023-10-16T10:02:10.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.110+0000 7fb477ac9640 1 -- start start 2023-10-16T10:02:10.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.112+0000 7fb477ac9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb47006a4b0 0x7fb47016c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.112+0000 7fb477ac9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4700caef0 con 0x7fb4700caa80 2023-10-16T10:02:10.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.112+0000 7fb477ac9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4700cb140 con 0x7fb4700addb0 2023-10-16T10:02:10.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.112+0000 7fb477ac9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb470161a50 con 0x7fb47006a4b0 2023-10-16T10:02:10.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.112+0000 7fb475064640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb4700caa80 0x7fb470069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34858/0 (socket says 172.21.15.84:34858) 2023-10-16T10:02:10.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.112+0000 7fb475064640 1 -- 172.21.15.84:0/2336092396 learned_addr learned my addr 172.21.15.84:0/2336092396 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:10.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.112+0000 7fb476066640 1 --2- 172.21.15.84:0/2336092396 >> v2:172.21.15.84:3300/0 conn(0x7fb47006a4b0 0x7fb47016c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:10.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.112+0000 7fb476066640 1 -- 172.21.15.84:0/2336092396 >> v1:172.21.15.157:6789/0 conn(0x7fb4700addb0 legacy=0x7fb4700c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:10.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.112+0000 7fb476066640 1 -- 172.21.15.84:0/2336092396 >> v1:172.21.15.84:6789/0 conn(0x7fb4700caa80 legacy=0x7fb470069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.112+0000 7fb476066640 1 -- 172.21.15.84:0/2336092396 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47006b030 con 0x7fb47006a4b0 2023-10-16T10:02:10.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.113+0000 7fb476066640 1 --2- 172.21.15.84:0/2336092396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb47006a4b0 0x7fb47016c400 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb468009d90 tx=0x7fb46802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698e2d8ac468be0d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:10.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.114+0000 7fb474863640 1 -- 172.21.15.84:0/2336092396 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46802cc60 con 0x7fb47006a4b0 2023-10-16T10:02:10.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.114+0000 7fb474863640 1 -- 172.21.15.84:0/2336092396 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46802cdc0 con 0x7fb47006a4b0 2023-10-16T10:02:10.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.114+0000 7fb474863640 1 -- 172.21.15.84:0/2336092396 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468034660 con 0x7fb47006a4b0 2023-10-16T10:02:10.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.114+0000 7fb477ac9640 1 -- 172.21.15.84:0/2336092396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb47006a4b0 msgr2=0x7fb47016c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.114+0000 7fb477ac9640 1 --2- 172.21.15.84:0/2336092396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb47006a4b0 0x7fb47016c400 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb468009d90 tx=0x7fb46802beb0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.115+0000 7fb477ac9640 1 -- 172.21.15.84:0/2336092396 shutdown_connections 2023-10-16T10:02:10.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.115+0000 7fb477ac9640 1 --2- 172.21.15.84:0/2336092396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb47006a4b0 0x7fb47016c400 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.115+0000 7fb477ac9640 1 -- 172.21.15.84:0/2336092396 >> 172.21.15.84:0/2336092396 conn(0x7fb4700ce9b0 msgr2=0x7fb4700cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:10.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.115+0000 7fb477ac9640 1 -- 172.21.15.84:0/2336092396 shutdown_connections 2023-10-16T10:02:10.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.115+0000 7fb477ac9640 1 -- 172.21.15.84:0/2336092396 wait complete. 2023-10-16T10:02:10.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.116+0000 7fb477ac9640 1 Processor -- start 2023-10-16T10:02:10.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.116+0000 7fb477ac9640 1 -- start start 2023-10-16T10:02:10.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.117+0000 7fb477ac9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb47006a4b0 0x7fb470155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.117+0000 7fb477ac9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4700addb0 0x7fb4701ece00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.117+0000 7fb477ac9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4700caa80 0x7fb4701ed340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.117+0000 7fb477ac9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4701ec9a0 con 0x7fb4700addb0 2023-10-16T10:02:10.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.117+0000 7fb477ac9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4701ebe20 con 0x7fb47006a4b0 2023-10-16T10:02:10.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.117+0000 7fb477ac9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4701ec100 con 0x7fb4700caa80 2023-10-16T10:02:10.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.118+0000 7fb475064640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4700addb0 0x7fb4701ece00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:10.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.118+0000 7fb475064640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4700addb0 0x7fb4701ece00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33212/0 (socket says 172.21.15.84:33212) 2023-10-16T10:02:10.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.118+0000 7fb475064640 1 -- 172.21.15.84:0/1490157299 learned_addr learned my addr 172.21.15.84:0/1490157299 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:10.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.118+0000 7fb475865640 1 --2- 172.21.15.84:0/1490157299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb47006a4b0 0x7fb470155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:10.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.118+0000 7fb476066640 1 --2- 172.21.15.84:0/1490157299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4700caa80 0x7fb4701ed340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:10.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.118+0000 7fb475064640 1 -- 172.21.15.84:0/1490157299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4700caa80 msgr2=0x7fb4701ed340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.118+0000 7fb475064640 1 --2- 172.21.15.84:0/1490157299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4700caa80 0x7fb4701ed340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.118+0000 7fb475064640 1 -- 172.21.15.84:0/1490157299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb47006a4b0 msgr2=0x7fb470155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.118+0000 7fb475064640 1 --2- 172.21.15.84:0/1490157299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb47006a4b0 0x7fb470155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.118+0000 7fb475064640 1 -- 172.21.15.84:0/1490157299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb468009a70 con 0x7fb4700addb0 2023-10-16T10:02:10.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.118+0000 7fb475865640 1 --2- 172.21.15.84:0/1490157299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb47006a4b0 0x7fb470155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:10.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.118+0000 7fb475064640 1 --2- 172.21.15.84:0/1490157299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4700addb0 0x7fb4701ece00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb46000c480 tx=0x7fb46000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:10.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.119+0000 7fb4667fc640 1 -- 172.21.15.84:0/1490157299 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460005ce0 con 0x7fb4700addb0 2023-10-16T10:02:10.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.119+0000 7fb4667fc640 1 -- 172.21.15.84:0/1490157299 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb460005e40 con 0x7fb4700addb0 2023-10-16T10:02:10.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.119+0000 7fb477ac9640 1 -- 172.21.15.84:0/1490157299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4701ec3e0 con 0x7fb4700addb0 2023-10-16T10:02:10.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.119+0000 7fb4667fc640 1 -- 172.21.15.84:0/1490157299 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4600072f0 con 0x7fb4700addb0 2023-10-16T10:02:10.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.119+0000 7fb477ac9640 1 -- 172.21.15.84:0/1490157299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4701eb0b0 con 0x7fb4700addb0 2023-10-16T10:02:10.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.120+0000 7fb477ac9640 1 -- 172.21.15.84:0/1490157299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb47004e160 con 0x7fb4700addb0 2023-10-16T10:02:10.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.121+0000 7fb4667fc640 1 -- 172.21.15.84:0/1490157299 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb460007510 con 0x7fb4700addb0 2023-10-16T10:02:10.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.123+0000 7fb4667fc640 1 --2- 172.21.15.84:0/1490157299 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fb45406af90 0x7fb45406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.123+0000 7fb4667fc640 1 -- 172.21.15.84:0/1490157299 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb460088910 con 0x7fb4700addb0 2023-10-16T10:02:10.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.123+0000 7fb475865640 1 --2- 172.21.15.84:0/1490157299 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fb45406af90 0x7fb45406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:10.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.124+0000 7fb475865640 1 --2- 172.21.15.84:0/1490157299 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fb45406af90 0x7fb45406d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb46c003eb0 tx=0x7fb46c009040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:10.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.127+0000 7fb4667fc640 1 -- 172.21.15.84:0/1490157299 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb460053ca0 con 0x7fb4700addb0 2023-10-16T10:02:10.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.320+0000 7fb477ac9640 1 -- 172.21.15.84:0/1490157299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb47006b5c0 con 0x7fb4700addb0 2023-10-16T10:02:10.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.321+0000 7fb4667fc640 1 -- 172.21.15.84:0/1490157299 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb460017020 con 0x7fb4700addb0 2023-10-16T10:02:10.322 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:02:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.324+0000 7fb477ac9640 1 -- 172.21.15.84:0/1490157299 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fb45406af90 msgr2=0x7fb45406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.324+0000 7fb477ac9640 1 --2- 172.21.15.84:0/1490157299 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fb45406af90 0x7fb45406d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb46c003eb0 tx=0x7fb46c009040 comp rx=0 tx=0).stop 2023-10-16T10:02:10.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.324+0000 7fb477ac9640 1 -- 172.21.15.84:0/1490157299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4700addb0 msgr2=0x7fb4701ece00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.324+0000 7fb477ac9640 1 --2- 172.21.15.84:0/1490157299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4700addb0 0x7fb4701ece00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb46000c480 tx=0x7fb46000c940 comp rx=0 tx=0).stop 2023-10-16T10:02:10.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.324+0000 7fb477ac9640 1 -- 172.21.15.84:0/1490157299 shutdown_connections 2023-10-16T10:02:10.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.324+0000 7fb477ac9640 1 --2- 172.21.15.84:0/1490157299 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fb45406af90 0x7fb45406d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.324+0000 7fb477ac9640 1 --2- 172.21.15.84:0/1490157299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4700caa80 0x7fb4701ed340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.324+0000 7fb477ac9640 1 --2- 172.21.15.84:0/1490157299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4700addb0 0x7fb4701ece00 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.324+0000 7fb477ac9640 1 --2- 172.21.15.84:0/1490157299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb47006a4b0 0x7fb470155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.324+0000 7fb477ac9640 1 -- 172.21.15.84:0/1490157299 >> 172.21.15.84:0/1490157299 conn(0x7fb4700ce9b0 msgr2=0x7fb470160ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:10.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.324+0000 7fb477ac9640 1 -- 172.21.15.84:0/1490157299 shutdown_connections 2023-10-16T10:02:10.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.324+0000 7fb477ac9640 1 -- 172.21.15.84:0/1490157299 wait complete. 2023-10-16T10:02:10.344 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T10:02:10.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.547+0000 7f5e62fc5640 1 Processor -- start 2023-10-16T10:02:10.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.547+0000 7f5e62fc5640 1 -- start start 2023-10-16T10:02:10.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.549+0000 7f5e62fc5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e5c0ce390 0x7f5e5c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.549+0000 7f5e62fc5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e5c15ed60 con 0x7f5e5c15ae80 2023-10-16T10:02:10.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.549+0000 7f5e62fc5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e5c0707e0 con 0x7f5e5c15aaa0 2023-10-16T10:02:10.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.549+0000 7f5e62fc5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c06b810 con 0x7f5e5c0ce390 2023-10-16T10:02:10.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.549+0000 7f5e5bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f5e5c15ae80 0x7f5e5c070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34870/0 (socket says 172.21.15.84:34870) 2023-10-16T10:02:10.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.549+0000 7f5e5bfff640 1 -- 172.21.15.84:0/395586620 learned_addr learned my addr 172.21.15.84:0/395586620 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:10.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.550+0000 7f5e61562640 1 --2- 172.21.15.84:0/395586620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e5c0ce390 0x7f5e5c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:10.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.550+0000 7f5e5b7fe640 1 -- 172.21.15.84:0/395586620 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219901925 0 0) 0x7f5e5c15ed60 con 0x7f5e5c15ae80 2023-10-16T10:02:10.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.550+0000 7f5e5b7fe640 1 -- 172.21.15.84:0/395586620 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e4c003520 con 0x7f5e5c15ae80 2023-10-16T10:02:10.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.550+0000 7f5e5b7fe640 1 -- 172.21.15.84:0/395586620 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2282327710 0 0) 0x7f5e4c003520 con 0x7f5e5c15ae80 2023-10-16T10:02:10.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.550+0000 7f5e5b7fe640 1 -- 172.21.15.84:0/395586620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e5c0ce390 msgr2=0x7f5e5c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:10.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.550+0000 7f5e5b7fe640 1 --2- 172.21.15.84:0/395586620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e5c0ce390 0x7f5e5c15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.550+0000 7f5e5b7fe640 1 -- 172.21.15.84:0/395586620 >> v1:172.21.15.157:6789/0 conn(0x7f5e5c15aaa0 legacy=0x7f5e5c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.550+0000 7f5e5b7fe640 1 -- 172.21.15.84:0/395586620 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c15ef90 con 0x7f5e5c15ae80 2023-10-16T10:02:10.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.550+0000 7f5e5b7fe640 1 -- 172.21.15.84:0/395586620 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f5e48001690 con 0x7f5e5c15ae80 2023-10-16T10:02:10.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.550+0000 7f5e5b7fe640 1 -- 172.21.15.84:0/395586620 >> v1:172.21.15.84:6789/0 conn(0x7f5e5c15ae80 legacy=0x7f5e5c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.550+0000 7f5e5b7fe640 1 --2- 172.21.15.84:0/395586620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e4c0039a0 0x7f5e4c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.550+0000 7f5e5b7fe640 1 --2- 172.21.15.84:0/395586620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e5c0ce390 0x7f5e4c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.550+0000 7f5e5b7fe640 1 --2- 172.21.15.84:0/395586620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e4c004d00 0x7f5e4c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.550+0000 7f5e5b7fe640 1 -- 172.21.15.84:0/395586620 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c06b810 con 0x7f5e4c004d00 2023-10-16T10:02:10.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.550+0000 7f5e5b7fe640 1 -- 172.21.15.84:0/395586620 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c00ba30 con 0x7f5e4c0039a0 2023-10-16T10:02:10.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.550+0000 7f5e5b7fe640 1 -- 172.21.15.84:0/395586620 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c00bbf0 con 0x7f5e5c0ce390 2023-10-16T10:02:10.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.550+0000 7f5e61562640 1 --2- 172.21.15.84:0/395586620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e4c004d00 0x7f5e4c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:10.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.551+0000 7f5e61562640 1 -- 172.21.15.84:0/395586620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e5c0ce390 msgr2=0x7f5e4c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:10.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.551+0000 7f5e61562640 1 --2- 172.21.15.84:0/395586620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e5c0ce390 0x7f5e4c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.551+0000 7f5e61562640 1 -- 172.21.15.84:0/395586620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e4c0039a0 msgr2=0x7f5e4c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:10.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.551+0000 7f5e61562640 1 --2- 172.21.15.84:0/395586620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e4c0039a0 0x7f5e4c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.551+0000 7f5e61562640 1 -- 172.21.15.84:0/395586620 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c003520 con 0x7f5e4c004d00 2023-10-16T10:02:10.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.552+0000 7f5e61562640 1 --2- 172.21.15.84:0/395586620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e4c004d00 0x7f5e4c005180 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009f90 tx=0x7f5e4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96710a88d5138c76 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:10.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.552+0000 7f5e5b7fe640 1 -- 172.21.15.84:0/395586620 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5402cc20 con 0x7f5e4c004d00 2023-10-16T10:02:10.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.552+0000 7f5e5b7fe640 1 -- 172.21.15.84:0/395586620 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5400f040 con 0x7f5e4c004d00 2023-10-16T10:02:10.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.552+0000 7f5e5b7fe640 1 -- 172.21.15.84:0/395586620 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54034440 con 0x7f5e4c004d00 2023-10-16T10:02:10.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.553+0000 7f5e62fc5640 1 -- 172.21.15.84:0/395586620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e4c004d00 msgr2=0x7f5e4c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.553+0000 7f5e62fc5640 1 --2- 172.21.15.84:0/395586620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e4c004d00 0x7f5e4c005180 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009f90 tx=0x7f5e4c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.553+0000 7f5e61562640 1 -- 172.21.15.84:0/395586620 reap_dead start 2023-10-16T10:02:10.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.553+0000 7f5e62fc5640 1 -- 172.21.15.84:0/395586620 shutdown_connections 2023-10-16T10:02:10.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.553+0000 7f5e62fc5640 1 -- 172.21.15.84:0/395586620 >> 172.21.15.84:0/395586620 conn(0x7f5e5c0cb410 msgr2=0x7f5e5c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:10.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.553+0000 7f5e62fc5640 1 -- 172.21.15.84:0/395586620 shutdown_connections 2023-10-16T10:02:10.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.553+0000 7f5e62fc5640 1 -- 172.21.15.84:0/395586620 wait complete. 2023-10-16T10:02:10.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.555+0000 7f5e62fc5640 1 Processor -- start 2023-10-16T10:02:10.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.555+0000 7f5e62fc5640 1 -- start start 2023-10-16T10:02:10.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.556+0000 7f5e62fc5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e4c004d00 0x7f5e5c15dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.556+0000 7f5e62fc5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e5c1e45b0 0x7f5e5c14b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.556+0000 7f5e62fc5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e5c1f5910 0x7f5e5c1f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.556+0000 7f5e62fc5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c1f6300 con 0x7f5e5c1f5910 2023-10-16T10:02:10.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.556+0000 7f5e62fc5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c15e2c0 con 0x7f5e5c1e45b0 2023-10-16T10:02:10.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.556+0000 7f5e62fc5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c14be40 con 0x7f5e4c004d00 2023-10-16T10:02:10.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.556+0000 7f5e61562640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e5c1f5910 0x7f5e5c1f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:10.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.556+0000 7f5e61562640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e5c1f5910 0x7f5e5c1f5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33240/0 (socket says 172.21.15.84:33240) 2023-10-16T10:02:10.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.556+0000 7f5e61562640 1 -- 172.21.15.84:0/1418834448 learned_addr learned my addr 172.21.15.84:0/1418834448 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:10.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.556+0000 7f5e61562640 1 -- 172.21.15.84:0/1418834448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e4c004d00 msgr2=0x7f5e5c15dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.556+0000 7f5e61562640 1 --2- 172.21.15.84:0/1418834448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e4c004d00 0x7f5e5c15dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.556+0000 7f5e61562640 1 -- 172.21.15.84:0/1418834448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e5c1e45b0 msgr2=0x7f5e5c14b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.556+0000 7f5e61562640 1 --2- 172.21.15.84:0/1418834448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e5c1e45b0 0x7f5e5c14b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.556+0000 7f5e61562640 1 -- 172.21.15.84:0/1418834448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c14c320 con 0x7f5e5c1f5910 2023-10-16T10:02:10.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.557+0000 7f5e61562640 1 --2- 172.21.15.84:0/1418834448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e5c1f5910 0x7f5e5c1f5dc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009f90 tx=0x7f5e54009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:10.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.557+0000 7f5e597fa640 1 -- 172.21.15.84:0/1418834448 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5402cea0 con 0x7f5e5c1f5910 2023-10-16T10:02:10.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.557+0000 7f5e597fa640 1 -- 172.21.15.84:0/1418834448 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e54034c90 con 0x7f5e5c1f5910 2023-10-16T10:02:10.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.557+0000 7f5e62fc5640 1 -- 172.21.15.84:0/1418834448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5c15b5e0 con 0x7f5e5c1f5910 2023-10-16T10:02:10.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.557+0000 7f5e597fa640 1 -- 172.21.15.84:0/1418834448 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54003990 con 0x7f5e5c1f5910 2023-10-16T10:02:10.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.557+0000 7f5e62fc5640 1 -- 172.21.15.84:0/1418834448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5c15bb60 con 0x7f5e5c1f5910 2023-10-16T10:02:10.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.558+0000 7f5e62fc5640 1 -- 172.21.15.84:0/1418834448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5c04e160 con 0x7f5e5c1f5910 2023-10-16T10:02:10.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.559+0000 7f5e597fa640 1 -- 172.21.15.84:0/1418834448 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5e54047460 con 0x7f5e5c1f5910 2023-10-16T10:02:10.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.560+0000 7f5e597fa640 1 --2- 172.21.15.84:0/1418834448 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f5e4006afc0 0x7f5e4006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.560+0000 7f5e597fa640 1 -- 172.21.15.84:0/1418834448 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5e540ab360 con 0x7f5e5c1f5910 2023-10-16T10:02:10.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.560+0000 7f5e60d61640 1 --2- 172.21.15.84:0/1418834448 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f5e4006afc0 0x7f5e4006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:10.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.561+0000 7f5e60d61640 1 --2- 172.21.15.84:0/1418834448 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f5e4006afc0 0x7f5e4006d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5e50001590 tx=0x7f5e50007990 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:10.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.564+0000 7f5e597fa640 1 -- 172.21.15.84:0/1418834448 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5e54076640 con 0x7f5e5c1f5910 2023-10-16T10:02:10.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.760+0000 7f5e62fc5640 1 -- 172.21.15.84:0/1418834448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5e5c0cdc60 con 0x7f5e5c1f5910 2023-10-16T10:02:10.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.761+0000 7f5e597fa640 1 -- 172.21.15.84:0/1418834448 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v60) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5e54079c60 con 0x7f5e5c1f5910 2023-10-16T10:02:10.763 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:02:10.763 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T10:02:10.763 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:10.763 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-16T10:02:10.763 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-16T10:02:10.764 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:10.764 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T10:02:10.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.764+0000 7f5e62fc5640 1 -- 172.21.15.84:0/1418834448 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f5e4006afc0 msgr2=0x7f5e4006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.764+0000 7f5e62fc5640 1 --2- 172.21.15.84:0/1418834448 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f5e4006afc0 0x7f5e4006d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5e50001590 tx=0x7f5e50007990 comp rx=0 tx=0).stop 2023-10-16T10:02:10.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.764+0000 7f5e62fc5640 1 -- 172.21.15.84:0/1418834448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e5c1f5910 msgr2=0x7f5e5c1f5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.764+0000 7f5e62fc5640 1 --2- 172.21.15.84:0/1418834448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e5c1f5910 0x7f5e5c1f5dc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009f90 tx=0x7f5e54009750 comp rx=0 tx=0).stop 2023-10-16T10:02:10.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.764+0000 7f5e62fc5640 1 -- 172.21.15.84:0/1418834448 shutdown_connections 2023-10-16T10:02:10.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.764+0000 7f5e62fc5640 1 --2- 172.21.15.84:0/1418834448 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f5e4006afc0 0x7f5e4006d480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.764+0000 7f5e62fc5640 1 --2- 172.21.15.84:0/1418834448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e5c1f5910 0x7f5e5c1f5dc0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.764+0000 7f5e62fc5640 1 --2- 172.21.15.84:0/1418834448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e5c1e45b0 0x7f5e5c14b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.764+0000 7f5e62fc5640 1 --2- 172.21.15.84:0/1418834448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e4c004d00 0x7f5e5c15dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.764+0000 7f5e62fc5640 1 -- 172.21.15.84:0/1418834448 >> 172.21.15.84:0/1418834448 conn(0x7f5e5c0cb410 msgr2=0x7f5e5c06f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:10.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.764+0000 7f5e62fc5640 1 -- 172.21.15.84:0/1418834448 shutdown_connections 2023-10-16T10:02:10.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.764+0000 7f5e62fc5640 1 -- 172.21.15.84:0/1418834448 wait complete. 2023-10-16T10:02:10.783 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T10:02:10.783 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T10:02:11.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.998+0000 7f3ebed6c640 1 Processor -- start 2023-10-16T10:02:11.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:10.998+0000 7f3ebed6c640 1 -- start start 2023-10-16T10:02:11.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.000+0000 7f3ebed6c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3eb80c8fa0 0x7f3eb8165aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.000+0000 7f3ebed6c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb8165610 con 0x7f3eb80c1d30 2023-10-16T10:02:11.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.000+0000 7f3ebed6c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb80ce370 con 0x7f3eb80b0360 2023-10-16T10:02:11.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.000+0000 7f3ebed6c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb80ce650 con 0x7f3eb80c8fa0 2023-10-16T10:02:11.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.000+0000 7f3ebcb08640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3eb80c1d30 0x7f3eb80cda90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34878/0 (socket says 172.21.15.84:34878) 2023-10-16T10:02:11.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.000+0000 7f3ebcb08640 1 -- 172.21.15.84:0/848492917 learned_addr learned my addr 172.21.15.84:0/848492917 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:11.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.001+0000 7f3eaffff640 1 --2- 172.21.15.84:0/848492917 >> v2:172.21.15.157:3300/0 conn(0x7f3eb80c8fa0 0x7f3eb8165aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:11.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.001+0000 7f3eaf7fe640 1 -- 172.21.15.84:0/848492917 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3837560197 0 0) 0x7f3eb8165610 con 0x7f3eb80c1d30 2023-10-16T10:02:11.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.001+0000 7f3eaf7fe640 1 -- 172.21.15.84:0/848492917 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ea0003520 con 0x7f3eb80c1d30 2023-10-16T10:02:11.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.001+0000 7f3eaf7fe640 1 -- 172.21.15.84:0/848492917 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 684672479 0 0) 0x7f3ea0003520 con 0x7f3eb80c1d30 2023-10-16T10:02:11.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.001+0000 7f3eaf7fe640 1 -- 172.21.15.84:0/848492917 >> v2:172.21.15.157:3300/0 conn(0x7f3eb80c8fa0 msgr2=0x7f3eb8165aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:11.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.001+0000 7f3eaf7fe640 1 --2- 172.21.15.84:0/848492917 >> v2:172.21.15.157:3300/0 conn(0x7f3eb80c8fa0 0x7f3eb8165aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.001+0000 7f3eaf7fe640 1 -- 172.21.15.84:0/848492917 >> v1:172.21.15.157:6789/0 conn(0x7f3eb80b0360 legacy=0x7f3eb80b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:11.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.001+0000 7f3eaf7fe640 1 -- 172.21.15.84:0/848492917 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb8164fc0 con 0x7f3eb80c1d30 2023-10-16T10:02:11.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.001+0000 7f3eaf7fe640 1 -- 172.21.15.84:0/848492917 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f3eb4001670 con 0x7f3eb80c1d30 2023-10-16T10:02:11.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.001+0000 7f3eaf7fe640 1 -- 172.21.15.84:0/848492917 >> v1:172.21.15.84:6789/0 conn(0x7f3eb80c1d30 legacy=0x7f3eb80cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:11.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.001+0000 7f3eaf7fe640 1 --2- 172.21.15.84:0/848492917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea00039e0 0x7f3ea0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.001+0000 7f3eaf7fe640 1 --2- 172.21.15.84:0/848492917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea00047c0 0x7f3ea0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.001+0000 7f3eaf7fe640 1 --2- 172.21.15.84:0/848492917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea0005180 0x7f3ea000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.001+0000 7f3eaf7fe640 1 -- 172.21.15.84:0/848492917 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb80ce650 con 0x7f3ea0005180 2023-10-16T10:02:11.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.001+0000 7f3eaf7fe640 1 -- 172.21.15.84:0/848492917 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea000c0b0 con 0x7f3ea00047c0 2023-10-16T10:02:11.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.001+0000 7f3eaf7fe640 1 -- 172.21.15.84:0/848492917 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea000c270 con 0x7f3ea00039e0 2023-10-16T10:02:11.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.002+0000 7f3ebd309640 1 --2- 172.21.15.84:0/848492917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea0005180 0x7f3ea000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:11.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.002+0000 7f3eaffff640 1 --2- 172.21.15.84:0/848492917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea00047c0 0x7f3ea0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:11.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.002+0000 7f3ebcb08640 1 --2- 172.21.15.84:0/848492917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea00039e0 0x7f3ea0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:11.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.002+0000 7f3ebd309640 1 -- 172.21.15.84:0/848492917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea00039e0 msgr2=0x7f3ea0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:11.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.002+0000 7f3ebd309640 1 --2- 172.21.15.84:0/848492917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea00039e0 0x7f3ea0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.002+0000 7f3ebd309640 1 -- 172.21.15.84:0/848492917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea00047c0 msgr2=0x7f3ea0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:11.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.002+0000 7f3ebd309640 1 --2- 172.21.15.84:0/848492917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea00047c0 0x7f3ea0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.002+0000 7f3ebd309640 1 -- 172.21.15.84:0/848492917 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea0003520 con 0x7f3ea0005180 2023-10-16T10:02:11.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.002+0000 7f3ebcb08640 1 --2- 172.21.15.84:0/848492917 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea00039e0 0x7f3ea0003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:11.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.003+0000 7f3ebd309640 1 -- 172.21.15.84:0/848492917 reap_dead start 2023-10-16T10:02:11.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.003+0000 7f3ebd309640 1 --2- 172.21.15.84:0/848492917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea0005180 0x7f3ea000b980 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0001e50 tx=0x7f3ea00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e396cd55f62cf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:11.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.003+0000 7f3eaf7fe640 1 -- 172.21.15.84:0/848492917 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0003b70 con 0x7f3ea0005180 2023-10-16T10:02:11.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.003+0000 7f3eaf7fe640 1 -- 172.21.15.84:0/848492917 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb0003d10 con 0x7f3ea0005180 2023-10-16T10:02:11.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.003+0000 7f3eaf7fe640 1 -- 172.21.15.84:0/848492917 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0005310 con 0x7f3ea0005180 2023-10-16T10:02:11.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.004+0000 7f3ebed6c640 1 -- 172.21.15.84:0/848492917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea0005180 msgr2=0x7f3ea000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:11.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.004+0000 7f3ebed6c640 1 --2- 172.21.15.84:0/848492917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea0005180 0x7f3ea000b980 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0001e50 tx=0x7f3ea00057f0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.004+0000 7f3ebed6c640 1 -- 172.21.15.84:0/848492917 shutdown_connections 2023-10-16T10:02:11.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.004+0000 7f3ebed6c640 1 --2- 172.21.15.84:0/848492917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea0005180 0x7f3ea000b980 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.004+0000 7f3ebed6c640 1 -- 172.21.15.84:0/848492917 >> 172.21.15.84:0/848492917 conn(0x7f3eb80cb360 msgr2=0x7f3eb80cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:11.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.004+0000 7f3ebed6c640 1 -- 172.21.15.84:0/848492917 shutdown_connections 2023-10-16T10:02:11.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.004+0000 7f3ebed6c640 1 -- 172.21.15.84:0/848492917 wait complete. 2023-10-16T10:02:11.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.006+0000 7f3ebed6c640 1 Processor -- start 2023-10-16T10:02:11.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.006+0000 7f3ebed6c640 1 -- start start 2023-10-16T10:02:11.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.007+0000 7f3ebed6c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea0005180 0x7f3eb8155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.007+0000 7f3ebed6c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eb80703e0 0x7f3eb8070860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.007+0000 7f3ebed6c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3eb80b2530 0x7f3eb80b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.007+0000 7f3ebed6c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb80b3d90 con 0x7f3eb80703e0 2023-10-16T10:02:11.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.007+0000 7f3ebed6c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb80b2f20 con 0x7f3ea0005180 2023-10-16T10:02:11.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.007+0000 7f3ebed6c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb8070da0 con 0x7f3eb80b2530 2023-10-16T10:02:11.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.007+0000 7f3eaffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eb80703e0 0x7f3eb8070860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:11.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.007+0000 7f3eaffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eb80703e0 0x7f3eb8070860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33244/0 (socket says 172.21.15.84:33244) 2023-10-16T10:02:11.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.007+0000 7f3eaffff640 1 -- 172.21.15.84:0/1763332671 learned_addr learned my addr 172.21.15.84:0/1763332671 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:11.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.008+0000 7f3eaffff640 1 -- 172.21.15.84:0/1763332671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3eb80b2530 msgr2=0x7f3eb80b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:11.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.008+0000 7f3eaffff640 1 --2- 172.21.15.84:0/1763332671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3eb80b2530 0x7f3eb80b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.008+0000 7f3eaffff640 1 -- 172.21.15.84:0/1763332671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea0005180 msgr2=0x7f3eb8155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:11.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.008+0000 7f3eaffff640 1 --2- 172.21.15.84:0/1763332671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea0005180 0x7f3eb8155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.008+0000 7f3eaffff640 1 -- 172.21.15.84:0/1763332671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb8164860 con 0x7f3eb80703e0 2023-10-16T10:02:11.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.008+0000 7f3eaffff640 1 --2- 172.21.15.84:0/1763332671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eb80703e0 0x7f3eb8070860 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8011570 tx=0x7f3ea8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:11.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.008+0000 7f3ead7fa640 1 -- 172.21.15.84:0/1763332671 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8015930 con 0x7f3eb80703e0 2023-10-16T10:02:11.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.008+0000 7f3ead7fa640 1 -- 172.21.15.84:0/1763332671 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea8015a90 con 0x7f3eb80703e0 2023-10-16T10:02:11.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.008+0000 7f3ead7fa640 1 -- 172.21.15.84:0/1763332671 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8015e60 con 0x7f3eb80703e0 2023-10-16T10:02:11.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.008+0000 7f3ebed6c640 1 -- 172.21.15.84:0/1763332671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb80b3110 con 0x7f3eb80703e0 2023-10-16T10:02:11.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.008+0000 7f3ebed6c640 1 -- 172.21.15.84:0/1763332671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb80b36d0 con 0x7f3eb80703e0 2023-10-16T10:02:11.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.009+0000 7f3ebed6c640 1 -- 172.21.15.84:0/1763332671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb804e160 con 0x7f3eb80703e0 2023-10-16T10:02:11.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.010+0000 7f3ead7fa640 1 -- 172.21.15.84:0/1763332671 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3ea800f460 con 0x7f3eb80703e0 2023-10-16T10:02:11.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.012+0000 7f3ead7fa640 1 --2- 172.21.15.84:0/1763332671 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f3ea406af90 0x7f3ea406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.012+0000 7f3ead7fa640 1 -- 172.21.15.84:0/1763332671 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3ea8089c70 con 0x7f3eb80703e0 2023-10-16T10:02:11.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.012+0000 7f3ebcb08640 1 --2- 172.21.15.84:0/1763332671 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f3ea406af90 0x7f3ea406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:11.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.013+0000 7f3ebcb08640 1 --2- 172.21.15.84:0/1763332671 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f3ea406af90 0x7f3ea406d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4000f60 tx=0x7f3eb4009040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:11.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.016+0000 7f3ead7fa640 1 -- 172.21.15.84:0/1763332671 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3ea8054f50 con 0x7f3eb80703e0 2023-10-16T10:02:11.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:11.209+0000 7f3ebed6c640 1 -- 172.21.15.84:0/1763332671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3eb80b4860 con 0x7f3eb80703e0 2023-10-16T10:02:12.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.036+0000 7f3ead7fa640 1 -- 172.21.15.84:0/1763332671 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v61) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3ea8058570 con 0x7f3eb80703e0 2023-10-16T10:02:12.037 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-16T10:02:12.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.040+0000 7f3ebed6c640 1 -- 172.21.15.84:0/1763332671 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f3ea406af90 msgr2=0x7f3ea406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:12.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.040+0000 7f3ebed6c640 1 --2- 172.21.15.84:0/1763332671 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f3ea406af90 0x7f3ea406d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4000f60 tx=0x7f3eb4009040 comp rx=0 tx=0).stop 2023-10-16T10:02:12.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.040+0000 7f3ebed6c640 1 -- 172.21.15.84:0/1763332671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eb80703e0 msgr2=0x7f3eb8070860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:12.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.040+0000 7f3ebed6c640 1 --2- 172.21.15.84:0/1763332671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eb80703e0 0x7f3eb8070860 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8011570 tx=0x7f3ea8011a30 comp rx=0 tx=0).stop 2023-10-16T10:02:12.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.041+0000 7f3ebed6c640 1 -- 172.21.15.84:0/1763332671 shutdown_connections 2023-10-16T10:02:12.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.041+0000 7f3ebed6c640 1 --2- 172.21.15.84:0/1763332671 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f3ea406af90 0x7f3ea406d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:12.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.041+0000 7f3ebed6c640 1 --2- 172.21.15.84:0/1763332671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3eb80b2530 0x7f3eb80b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:12.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.041+0000 7f3ebed6c640 1 --2- 172.21.15.84:0/1763332671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eb80703e0 0x7f3eb8070860 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:12.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.041+0000 7f3ebed6c640 1 --2- 172.21.15.84:0/1763332671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea0005180 0x7f3eb8155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:12.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.041+0000 7f3ebed6c640 1 -- 172.21.15.84:0/1763332671 >> 172.21.15.84:0/1763332671 conn(0x7f3eb80cb360 msgr2=0x7f3eb8160e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:12.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.042+0000 7f3ebed6c640 1 -- 172.21.15.84:0/1763332671 shutdown_connections 2023-10-16T10:02:12.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.042+0000 7f3ebed6c640 1 -- 172.21.15.84:0/1763332671 wait complete. 2023-10-16T10:02:12.069 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T10:02:12.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.280+0000 7fd215a09640 1 Processor -- start 2023-10-16T10:02:12.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.280+0000 7fd215a09640 1 -- start start 2023-10-16T10:02:12.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.282+0000 7fd215a09640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd210159f00 0x7fd2100b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:12.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.282+0000 7fd215a09640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21006f890 con 0x7fd210159b20 2023-10-16T10:02:12.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.282+0000 7fd215a09640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21006c650 con 0x7fd21015d8c0 2023-10-16T10:02:12.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.282+0000 7fd215a09640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21006c930 con 0x7fd210159f00 2023-10-16T10:02:12.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.282+0000 7fd207fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd210159f00 0x7fd2100b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:12.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.282+0000 7fd207fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd210159f00 0x7fd2100b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33256/0 (socket says 172.21.15.84:33256) 2023-10-16T10:02:12.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.282+0000 7fd207fff640 1 -- 172.21.15.84:0/335358003 learned_addr learned my addr 172.21.15.84:0/335358003 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:12.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.283+0000 7fd207fff640 1 -- 172.21.15.84:0/335358003 >> v1:172.21.15.157:6789/0 conn(0x7fd21015d8c0 legacy=0x7fd21006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:12.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.283+0000 7fd207fff640 1 -- 172.21.15.84:0/335358003 >> v1:172.21.15.84:6789/0 conn(0x7fd210159b20 legacy=0x7fd21006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:12.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.283+0000 7fd207fff640 1 -- 172.21.15.84:0/335358003 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21006cc10 con 0x7fd210159f00 2023-10-16T10:02:12.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.284+0000 7fd207fff640 1 --2- 172.21.15.84:0/335358003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd210159f00 0x7fd2100b0410 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009870 tx=0x7fd1f802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c561097eb5ebd504 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:12.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.284+0000 7fd20e7fc640 1 -- 172.21.15.84:0/335358003 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8030860 con 0x7fd210159f00 2023-10-16T10:02:12.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.284+0000 7fd20e7fc640 1 -- 172.21.15.84:0/335358003 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f80309c0 con 0x7fd210159f00 2023-10-16T10:02:12.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.284+0000 7fd20e7fc640 1 -- 172.21.15.84:0/335358003 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8030d50 con 0x7fd210159f00 2023-10-16T10:02:12.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.285+0000 7fd215a09640 1 -- 172.21.15.84:0/335358003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd210159f00 msgr2=0x7fd2100b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:12.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.285+0000 7fd215a09640 1 --2- 172.21.15.84:0/335358003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd210159f00 0x7fd2100b0410 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009870 tx=0x7fd1f802b9a0 comp rx=0 tx=0).stop 2023-10-16T10:02:12.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.285+0000 7fd215a09640 1 -- 172.21.15.84:0/335358003 shutdown_connections 2023-10-16T10:02:12.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.285+0000 7fd215a09640 1 --2- 172.21.15.84:0/335358003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd210159f00 0x7fd2100b0410 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:12.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.285+0000 7fd215a09640 1 -- 172.21.15.84:0/335358003 >> 172.21.15.84:0/335358003 conn(0x7fd2100cb410 msgr2=0x7fd2100cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:12.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.285+0000 7fd215a09640 1 -- 172.21.15.84:0/335358003 shutdown_connections 2023-10-16T10:02:12.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.285+0000 7fd215a09640 1 -- 172.21.15.84:0/335358003 wait complete. 2023-10-16T10:02:12.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.287+0000 7fd215a09640 1 Processor -- start 2023-10-16T10:02:12.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.287+0000 7fd215a09640 1 -- start start 2023-10-16T10:02:12.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.288+0000 7fd215a09640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd210159b20 0x7fd2101f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:12.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.288+0000 7fd215a09640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd210159f00 0x7fd2101e7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:12.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.288+0000 7fd215a09640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21015d8c0 0x7fd2100c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:12.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.288+0000 7fd215a09640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21015c1c0 con 0x7fd210159f00 2023-10-16T10:02:12.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.288+0000 7fd215a09640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21015c4a0 con 0x7fd21015d8c0 2023-10-16T10:02:12.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.288+0000 7fd215a09640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21015c780 con 0x7fd210159b20 2023-10-16T10:02:12.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.288+0000 7fd207fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd210159f00 0x7fd2101e7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:12.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.288+0000 7fd207fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd210159f00 0x7fd2101e7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33260/0 (socket says 172.21.15.84:33260) 2023-10-16T10:02:12.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.288+0000 7fd207fff640 1 -- 172.21.15.84:0/2902205391 learned_addr learned my addr 172.21.15.84:0/2902205391 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:12.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.289+0000 7fd20f7fe640 1 --2- 172.21.15.84:0/2902205391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21015d8c0 0x7fd2100c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:12.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.289+0000 7fd20effd640 1 --2- 172.21.15.84:0/2902205391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd210159b20 0x7fd2101f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:12.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.289+0000 7fd207fff640 1 -- 172.21.15.84:0/2902205391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd210159b20 msgr2=0x7fd2101f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:12.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.289+0000 7fd207fff640 1 --2- 172.21.15.84:0/2902205391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd210159b20 0x7fd2101f4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:12.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.289+0000 7fd207fff640 1 -- 172.21.15.84:0/2902205391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21015d8c0 msgr2=0x7fd2100c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:12.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.289+0000 7fd207fff640 1 --2- 172.21.15.84:0/2902205391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21015d8c0 0x7fd2100c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:12.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.289+0000 7fd207fff640 1 -- 172.21.15.84:0/2902205391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f80094d0 con 0x7fd210159f00 2023-10-16T10:02:12.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.289+0000 7fd207fff640 1 --2- 172.21.15.84:0/2902205391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd210159f00 0x7fd2101e7070 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009870 tx=0x7fd1f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:12.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.290+0000 7fd20cff9640 1 -- 172.21.15.84:0/2902205391 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8030810 con 0x7fd210159f00 2023-10-16T10:02:12.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.290+0000 7fd20cff9640 1 -- 172.21.15.84:0/2902205391 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f8043400 con 0x7fd210159f00 2023-10-16T10:02:12.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.290+0000 7fd20cff9640 1 -- 172.21.15.84:0/2902205391 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f80437b0 con 0x7fd210159f00 2023-10-16T10:02:12.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.290+0000 7fd215a09640 1 -- 172.21.15.84:0/2902205391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21015ca60 con 0x7fd210159f00 2023-10-16T10:02:12.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.290+0000 7fd215a09640 1 -- 172.21.15.84:0/2902205391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd210158fc0 con 0x7fd210159f00 2023-10-16T10:02:12.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.291+0000 7fd215a09640 1 -- 172.21.15.84:0/2902205391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21015cf80 con 0x7fd210159f00 2023-10-16T10:02:12.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.293+0000 7fd20cff9640 1 -- 172.21.15.84:0/2902205391 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd1f8043910 con 0x7fd210159f00 2023-10-16T10:02:12.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.294+0000 7fd20cff9640 1 --2- 172.21.15.84:0/2902205391 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd1f006af90 0x7fd1f006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:12.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.294+0000 7fd20cff9640 1 -- 172.21.15.84:0/2902205391 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fd1f803e070 con 0x7fd210159f00 2023-10-16T10:02:12.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.294+0000 7fd20effd640 1 --2- 172.21.15.84:0/2902205391 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd1f006af90 0x7fd1f006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:12.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.295+0000 7fd20effd640 1 --2- 172.21.15.84:0/2902205391 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd1f006af90 0x7fd1f006d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd200000f60 tx=0x7fd200009040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:12.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.298+0000 7fd20cff9640 1 -- 172.21.15.84:0/2902205391 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd1f8078480 con 0x7fd210159f00 2023-10-16T10:02:12.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:12.509+0000 7fd215a09640 1 -- 172.21.15.84:0/2902205391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd21015cf80 con 0x7fd210159f00 2023-10-16T10:02:13.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.041+0000 7fd20cff9640 1 -- 172.21.15.84:0/2902205391 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v62) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd21015cf80 con 0x7fd210159f00 2023-10-16T10:02:13.044 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-16T10:02:13.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.045+0000 7fd215a09640 1 -- 172.21.15.84:0/2902205391 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd1f006af90 msgr2=0x7fd1f006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:13.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.045+0000 7fd215a09640 1 --2- 172.21.15.84:0/2902205391 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd1f006af90 0x7fd1f006d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd200000f60 tx=0x7fd200009040 comp rx=0 tx=0).stop 2023-10-16T10:02:13.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.045+0000 7fd215a09640 1 -- 172.21.15.84:0/2902205391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd210159f00 msgr2=0x7fd2101e7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:13.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.045+0000 7fd215a09640 1 --2- 172.21.15.84:0/2902205391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd210159f00 0x7fd2101e7070 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009870 tx=0x7fd1f8004730 comp rx=0 tx=0).stop 2023-10-16T10:02:13.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.046+0000 7fd215a09640 1 -- 172.21.15.84:0/2902205391 shutdown_connections 2023-10-16T10:02:13.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.046+0000 7fd215a09640 1 --2- 172.21.15.84:0/2902205391 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd1f006af90 0x7fd1f006d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.046+0000 7fd215a09640 1 --2- 172.21.15.84:0/2902205391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21015d8c0 0x7fd2100c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.046+0000 7fd215a09640 1 --2- 172.21.15.84:0/2902205391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd210159f00 0x7fd2101e7070 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.046+0000 7fd215a09640 1 --2- 172.21.15.84:0/2902205391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd210159b20 0x7fd2101f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.046+0000 7fd215a09640 1 -- 172.21.15.84:0/2902205391 >> 172.21.15.84:0/2902205391 conn(0x7fd2100cb410 msgr2=0x7fd21006fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:13.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.046+0000 7fd215a09640 1 -- 172.21.15.84:0/2902205391 shutdown_connections 2023-10-16T10:02:13.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.047+0000 7fd215a09640 1 -- 172.21.15.84:0/2902205391 wait complete. 2023-10-16T10:02:13.082 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T10:02:13.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.268+0000 7fbf2acd1640 1 Processor -- start 2023-10-16T10:02:13.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.268+0000 7fbf2acd1640 1 -- start start 2023-10-16T10:02:13.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.269+0000 7fbf2acd1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf2415dde0 0x7fbf240709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:13.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.269+0000 7fbf2acd1640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf24160a90 con 0x7fbf2415c9f0 2023-10-16T10:02:13.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.270+0000 7fbf2acd1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf240cd900 con 0x7fbf24070f20 2023-10-16T10:02:13.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.270+0000 7fbf2acd1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240cdec0 con 0x7fbf2415dde0 2023-10-16T10:02:13.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.270+0000 7fbf29ccf640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbf2415c9f0 0x7fbf240c8cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34894/0 (socket says 172.21.15.84:34894) 2023-10-16T10:02:13.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.270+0000 7fbf29ccf640 1 -- 172.21.15.84:0/4234794379 learned_addr learned my addr 172.21.15.84:0/4234794379 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:13.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.270+0000 7fbf294ce640 1 --2- 172.21.15.84:0/4234794379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf2415dde0 0x7fbf240709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:13.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.270+0000 7fbf28ccd640 1 -- 172.21.15.84:0/4234794379 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039887912 0 0) 0x7fbf24160a90 con 0x7fbf2415c9f0 2023-10-16T10:02:13.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.270+0000 7fbf28ccd640 1 -- 172.21.15.84:0/4234794379 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf10003520 con 0x7fbf2415c9f0 2023-10-16T10:02:13.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.271+0000 7fbf28ccd640 1 -- 172.21.15.84:0/4234794379 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4013686323 0 0) 0x7fbf10003520 con 0x7fbf2415c9f0 2023-10-16T10:02:13.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.271+0000 7fbf28ccd640 1 -- 172.21.15.84:0/4234794379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf2415dde0 msgr2=0x7fbf240709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:13.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.271+0000 7fbf28ccd640 1 --2- 172.21.15.84:0/4234794379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf2415dde0 0x7fbf240709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.271+0000 7fbf28ccd640 1 -- 172.21.15.84:0/4234794379 >> v1:172.21.15.157:6789/0 conn(0x7fbf24070f20 legacy=0x7fbf240ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:13.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.271+0000 7fbf28ccd640 1 -- 172.21.15.84:0/4234794379 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf240cdbe0 con 0x7fbf2415c9f0 2023-10-16T10:02:13.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.271+0000 7fbf28ccd640 1 -- 172.21.15.84:0/4234794379 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fbf20001670 con 0x7fbf2415c9f0 2023-10-16T10:02:13.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.271+0000 7fbf28ccd640 1 -- 172.21.15.84:0/4234794379 >> v1:172.21.15.84:6789/0 conn(0x7fbf2415c9f0 legacy=0x7fbf240c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:13.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.271+0000 7fbf28ccd640 1 --2- 172.21.15.84:0/4234794379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf2415dde0 0x7fbf100039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:13.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.271+0000 7fbf28ccd640 1 --2- 172.21.15.84:0/4234794379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf10003f20 0x7fbf100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:13.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.271+0000 7fbf28ccd640 1 --2- 172.21.15.84:0/4234794379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf10004d00 0x7fbf10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:13.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.271+0000 7fbf28ccd640 1 -- 172.21.15.84:0/4234794379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1000ba30 con 0x7fbf10003f20 2023-10-16T10:02:13.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.271+0000 7fbf28ccd640 1 -- 172.21.15.84:0/4234794379 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1000bba0 con 0x7fbf10004d00 2023-10-16T10:02:13.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.271+0000 7fbf28ccd640 1 -- 172.21.15.84:0/4234794379 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1000bd60 con 0x7fbf2415dde0 2023-10-16T10:02:13.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.271+0000 7fbf294ce640 1 --2- 172.21.15.84:0/4234794379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf10003f20 0x7fbf100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:13.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.271+0000 7fbf29ccf640 1 --2- 172.21.15.84:0/4234794379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf2415dde0 0x7fbf100039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:13.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.272+0000 7fbf2a4d0640 1 --2- 172.21.15.84:0/4234794379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf10004d00 0x7fbf10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:13.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.272+0000 7fbf294ce640 1 -- 172.21.15.84:0/4234794379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf2415dde0 msgr2=0x7fbf100039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:13.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.272+0000 7fbf294ce640 1 --2- 172.21.15.84:0/4234794379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf2415dde0 0x7fbf100039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.272+0000 7fbf294ce640 1 -- 172.21.15.84:0/4234794379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf10004d00 msgr2=0x7fbf10005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:13.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.272+0000 7fbf294ce640 1 --2- 172.21.15.84:0/4234794379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf10004d00 0x7fbf10005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.272+0000 7fbf294ce640 1 -- 172.21.15.84:0/4234794379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10003520 con 0x7fbf10003f20 2023-10-16T10:02:13.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.272+0000 7fbf29ccf640 1 --2- 172.21.15.84:0/4234794379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf2415dde0 0x7fbf100039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:13.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.273+0000 7fbf294ce640 1 --2- 172.21.15.84:0/4234794379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf10003f20 0x7fbf100047c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009a90 tx=0x7fbf100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3621c5b0eded72fd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:13.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.273+0000 7fbf28ccd640 1 -- 172.21.15.84:0/4234794379 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1402c680 con 0x7fbf10003f20 2023-10-16T10:02:13.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.273+0000 7fbf28ccd640 1 -- 172.21.15.84:0/4234794379 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1402c7e0 con 0x7fbf10003f20 2023-10-16T10:02:13.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.273+0000 7fbf28ccd640 1 -- 172.21.15.84:0/4234794379 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1402cb70 con 0x7fbf10003f20 2023-10-16T10:02:13.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.274+0000 7fbf2acd1640 1 -- 172.21.15.84:0/4234794379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf10003f20 msgr2=0x7fbf100047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:13.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.274+0000 7fbf2acd1640 1 --2- 172.21.15.84:0/4234794379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf10003f20 0x7fbf100047c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009a90 tx=0x7fbf100057f0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.274+0000 7fbf2a4d0640 1 -- 172.21.15.84:0/4234794379 reap_dead start 2023-10-16T10:02:13.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.274+0000 7fbf2acd1640 1 -- 172.21.15.84:0/4234794379 shutdown_connections 2023-10-16T10:02:13.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.274+0000 7fbf2acd1640 1 -- 172.21.15.84:0/4234794379 >> 172.21.15.84:0/4234794379 conn(0x7fbf240ce6a0 msgr2=0x7fbf240ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:13.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.274+0000 7fbf2acd1640 1 -- 172.21.15.84:0/4234794379 shutdown_connections 2023-10-16T10:02:13.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.274+0000 7fbf2acd1640 1 -- 172.21.15.84:0/4234794379 wait complete. 2023-10-16T10:02:13.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.276+0000 7fbf2acd1640 1 Processor -- start 2023-10-16T10:02:13.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.276+0000 7fbf2acd1640 1 -- start start 2023-10-16T10:02:13.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.277+0000 7fbf2acd1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf10003f20 0x7fbf24150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:13.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.277+0000 7fbf2acd1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf2406aa30 0x7fbf2406af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:13.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.277+0000 7fbf2acd1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf240c46d0 0x7fbf240c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:13.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.277+0000 7fbf2acd1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf241511a0 con 0x7fbf10003f20 2023-10-16T10:02:13.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.277+0000 7fbf2acd1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240c5090 con 0x7fbf2406aa30 2023-10-16T10:02:13.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.277+0000 7fbf2acd1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2406c2a0 con 0x7fbf240c46d0 2023-10-16T10:02:13.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.277+0000 7fbf29ccf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf10003f20 0x7fbf24150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:13.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.277+0000 7fbf29ccf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf10003f20 0x7fbf24150c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33274/0 (socket says 172.21.15.84:33274) 2023-10-16T10:02:13.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.277+0000 7fbf29ccf640 1 -- 172.21.15.84:0/4047520055 learned_addr learned my addr 172.21.15.84:0/4047520055 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:13.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.277+0000 7fbf2a4d0640 1 --2- 172.21.15.84:0/4047520055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf240c46d0 0x7fbf240c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:13.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.277+0000 7fbf294ce640 1 --2- 172.21.15.84:0/4047520055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf2406aa30 0x7fbf2406af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:13.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.277+0000 7fbf29ccf640 1 -- 172.21.15.84:0/4047520055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf240c46d0 msgr2=0x7fbf240c4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:13.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.277+0000 7fbf29ccf640 1 --2- 172.21.15.84:0/4047520055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf240c46d0 0x7fbf240c4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.277+0000 7fbf29ccf640 1 -- 172.21.15.84:0/4047520055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf2406aa30 msgr2=0x7fbf2406af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:13.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.277+0000 7fbf29ccf640 1 --2- 172.21.15.84:0/4047520055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf2406aa30 0x7fbf2406af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.277+0000 7fbf29ccf640 1 -- 172.21.15.84:0/4047520055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2406b450 con 0x7fbf10003f20 2023-10-16T10:02:13.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.277+0000 7fbf2a4d0640 1 --2- 172.21.15.84:0/4047520055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf240c46d0 0x7fbf240c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:13.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.278+0000 7fbf29ccf640 1 --2- 172.21.15.84:0/4047520055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf10003f20 0x7fbf24150c60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000d8b0 tx=0x7fbf2000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:13.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.278+0000 7fbf1a7fc640 1 -- 172.21.15.84:0/4047520055 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf200074e0 con 0x7fbf10003f20 2023-10-16T10:02:13.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.278+0000 7fbf1a7fc640 1 -- 172.21.15.84:0/4047520055 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf20007640 con 0x7fbf10003f20 2023-10-16T10:02:13.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.278+0000 7fbf1a7fc640 1 -- 172.21.15.84:0/4047520055 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf20004a00 con 0x7fbf10003f20 2023-10-16T10:02:13.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.278+0000 7fbf2acd1640 1 -- 172.21.15.84:0/4047520055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf2406b720 con 0x7fbf10003f20 2023-10-16T10:02:13.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.278+0000 7fbf2acd1640 1 -- 172.21.15.84:0/4047520055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2406bce0 con 0x7fbf10003f20 2023-10-16T10:02:13.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.279+0000 7fbf2acd1640 1 -- 172.21.15.84:0/4047520055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf24160b80 con 0x7fbf10003f20 2023-10-16T10:02:13.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.280+0000 7fbf1a7fc640 1 -- 172.21.15.84:0/4047520055 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbf20004b60 con 0x7fbf10003f20 2023-10-16T10:02:13.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.281+0000 7fbf1a7fc640 1 --2- 172.21.15.84:0/4047520055 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fbf0806afc0 0x7fbf0806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:13.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.281+0000 7fbf1a7fc640 1 -- 172.21.15.84:0/4047520055 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fbf20089240 con 0x7fbf10003f20 2023-10-16T10:02:13.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.282+0000 7fbf294ce640 1 --2- 172.21.15.84:0/4047520055 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fbf0806afc0 0x7fbf0806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:13.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.282+0000 7fbf294ce640 1 --2- 172.21.15.84:0/4047520055 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fbf0806afc0 0x7fbf0806d480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbf14036000 tx=0x7fbf14030000 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:13.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.287+0000 7fbf1a7fc640 1 -- 172.21.15.84:0/4047520055 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbf20057c70 con 0x7fbf10003f20 2023-10-16T10:02:13.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:13.512+0000 7fbf2acd1640 1 -- 172.21.15.84:0/4047520055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbf2404e9b0 con 0x7fbf10003f20 2023-10-16T10:02:14.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.058+0000 7fbf1a7fc640 1 -- 172.21.15.84:0/4047520055 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7fbf20057800 con 0x7fbf10003f20 2023-10-16T10:02:14.060 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 6 and data pool 7 2023-10-16T10:02:14.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.062+0000 7fbf2acd1640 1 -- 172.21.15.84:0/4047520055 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fbf0806afc0 msgr2=0x7fbf0806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.062+0000 7fbf2acd1640 1 --2- 172.21.15.84:0/4047520055 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fbf0806afc0 0x7fbf0806d480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbf14036000 tx=0x7fbf14030000 comp rx=0 tx=0).stop 2023-10-16T10:02:14.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.062+0000 7fbf2acd1640 1 -- 172.21.15.84:0/4047520055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf10003f20 msgr2=0x7fbf24150c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.062+0000 7fbf2acd1640 1 --2- 172.21.15.84:0/4047520055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf10003f20 0x7fbf24150c60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000d8b0 tx=0x7fbf2000dd70 comp rx=0 tx=0).stop 2023-10-16T10:02:14.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.063+0000 7fbf2acd1640 1 -- 172.21.15.84:0/4047520055 shutdown_connections 2023-10-16T10:02:14.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.063+0000 7fbf2acd1640 1 --2- 172.21.15.84:0/4047520055 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fbf0806afc0 0x7fbf0806d480 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.063+0000 7fbf2acd1640 1 --2- 172.21.15.84:0/4047520055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf240c46d0 0x7fbf240c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.063+0000 7fbf2acd1640 1 --2- 172.21.15.84:0/4047520055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf2406aa30 0x7fbf2406af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.063+0000 7fbf2acd1640 1 --2- 172.21.15.84:0/4047520055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf10003f20 0x7fbf24150c60 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.063+0000 7fbf2acd1640 1 -- 172.21.15.84:0/4047520055 >> 172.21.15.84:0/4047520055 conn(0x7fbf240ce6a0 msgr2=0x7fbf240adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:14.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.063+0000 7fbf2acd1640 1 -- 172.21.15.84:0/4047520055 shutdown_connections 2023-10-16T10:02:14.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.063+0000 7fbf2acd1640 1 -- 172.21.15.84:0/4047520055 wait complete. 2023-10-16T10:02:14.097 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:02:14.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.283+0000 7fece39ce640 1 Processor -- start 2023-10-16T10:02:14.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.283+0000 7fece39ce640 1 -- start start 2023-10-16T10:02:14.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.284+0000 7fece39ce640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fecdc15aaa0 0x7fecdc15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.284+0000 7fece39ce640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecdc15ae80 0x7fecdc0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.284+0000 7fece39ce640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecdc15ed60 con 0x7fecdc0ccaa0 2023-10-16T10:02:14.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.284+0000 7fece39ce640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecdc06b810 con 0x7fecdc15aaa0 2023-10-16T10:02:14.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.284+0000 7fece39ce640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0707e0 con 0x7fecdc15ae80 2023-10-16T10:02:14.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.285+0000 7fece176a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fecdc15aaa0 0x7fecdc15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.285+0000 7fece0f69640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecdc15ae80 0x7fecdc0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.285+0000 7fece176a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fecdc15aaa0 0x7fecdc15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:59382/0 (socket says 172.21.15.84:59382) 2023-10-16T10:02:14.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.285+0000 7fece176a640 1 -- 172.21.15.84:0/3291124652 learned_addr learned my addr 172.21.15.84:0/3291124652 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:14.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.286+0000 7fece176a640 1 -- 172.21.15.84:0/3291124652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecdc15ae80 msgr2=0x7fecdc0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:14.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.286+0000 7fece176a640 1 --2- 172.21.15.84:0/3291124652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecdc15ae80 0x7fecdc0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.286+0000 7fece176a640 1 -- 172.21.15.84:0/3291124652 >> v1:172.21.15.157:6789/0 conn(0x7fecdc0ccaa0 legacy=0x7fecdc070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.286+0000 7fece176a640 1 -- 172.21.15.84:0/3291124652 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc15ef90 con 0x7fecdc15aaa0 2023-10-16T10:02:14.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.287+0000 7fece176a640 1 --2- 172.21.15.84:0/3291124652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecdc15aaa0 0x7fecdc15e820 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fecd8009870 tx=0x7fecd802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7099cb5ebf882235 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:14.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.288+0000 7fecd3fff640 1 -- 172.21.15.84:0/3291124652 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd8030840 con 0x7fecdc15aaa0 2023-10-16T10:02:14.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.288+0000 7fecd3fff640 1 -- 172.21.15.84:0/3291124652 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd80309a0 con 0x7fecdc15aaa0 2023-10-16T10:02:14.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.288+0000 7fecd3fff640 1 -- 172.21.15.84:0/3291124652 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd8030d30 con 0x7fecdc15aaa0 2023-10-16T10:02:14.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.289+0000 7fece39ce640 1 -- 172.21.15.84:0/3291124652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecdc15aaa0 msgr2=0x7fecdc15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.289+0000 7fece39ce640 1 --2- 172.21.15.84:0/3291124652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecdc15aaa0 0x7fecdc15e820 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fecd8009870 tx=0x7fecd802b980 comp rx=0 tx=0).stop 2023-10-16T10:02:14.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.289+0000 7fece39ce640 1 -- 172.21.15.84:0/3291124652 shutdown_connections 2023-10-16T10:02:14.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.289+0000 7fece39ce640 1 --2- 172.21.15.84:0/3291124652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecdc15ae80 0x7fecdc0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.289+0000 7fece39ce640 1 --2- 172.21.15.84:0/3291124652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecdc15aaa0 0x7fecdc15e820 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.289+0000 7fece39ce640 1 -- 172.21.15.84:0/3291124652 >> 172.21.15.84:0/3291124652 conn(0x7fecdc0cb410 msgr2=0x7fecdc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:14.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.289+0000 7fece39ce640 1 -- 172.21.15.84:0/3291124652 shutdown_connections 2023-10-16T10:02:14.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.289+0000 7fece39ce640 1 -- 172.21.15.84:0/3291124652 wait complete. 2023-10-16T10:02:14.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.291+0000 7fece39ce640 1 Processor -- start 2023-10-16T10:02:14.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.291+0000 7fece39ce640 1 -- start start 2023-10-16T10:02:14.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.292+0000 7fece39ce640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecdc0ccaa0 0x7fecdc159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.292+0000 7fece39ce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecdc15aaa0 0x7fecdc1574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.292+0000 7fece39ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecdc15ae80 0x7fecdc155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.292+0000 7fece39ce640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0c4e80 con 0x7fecdc15ae80 2023-10-16T10:02:14.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.292+0000 7fece39ce640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0c4300 con 0x7fecdc15aaa0 2023-10-16T10:02:14.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.292+0000 7fece39ce640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0c45e0 con 0x7fecdc0ccaa0 2023-10-16T10:02:14.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.292+0000 7fece1f6b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecdc15ae80 0x7fecdc155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.292+0000 7fece1f6b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecdc15ae80 0x7fecdc155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33290/0 (socket says 172.21.15.84:33290) 2023-10-16T10:02:14.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.292+0000 7fece1f6b640 1 -- 172.21.15.84:0/3274579014 learned_addr learned my addr 172.21.15.84:0/3274579014 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:14.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.293+0000 7fece176a640 1 --2- 172.21.15.84:0/3274579014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecdc0ccaa0 0x7fecdc159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.293+0000 7fece0f69640 1 --2- 172.21.15.84:0/3274579014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecdc15aaa0 0x7fecdc1574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.293+0000 7fece1f6b640 1 -- 172.21.15.84:0/3274579014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecdc0ccaa0 msgr2=0x7fecdc159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.293+0000 7fece1f6b640 1 --2- 172.21.15.84:0/3274579014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecdc0ccaa0 0x7fecdc159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.293+0000 7fece1f6b640 1 -- 172.21.15.84:0/3274579014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecdc15aaa0 msgr2=0x7fecdc1574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.293+0000 7fece1f6b640 1 --2- 172.21.15.84:0/3274579014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecdc15aaa0 0x7fecdc1574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.293+0000 7fece1f6b640 1 -- 172.21.15.84:0/3274579014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd80094d0 con 0x7fecdc15ae80 2023-10-16T10:02:14.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.293+0000 7fece0f69640 1 --2- 172.21.15.84:0/3274579014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecdc15aaa0 0x7fecdc1574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:14.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.293+0000 7fece1f6b640 1 --2- 172.21.15.84:0/3274579014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecdc15ae80 0x7fecdc155b20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fecd400c6b0 tx=0x7fecd400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:14.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.293+0000 7fecd1ffb640 1 -- 172.21.15.84:0/3274579014 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4003b70 con 0x7fecdc15ae80 2023-10-16T10:02:14.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.294+0000 7fecd1ffb640 1 -- 172.21.15.84:0/3274579014 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd4003cf0 con 0x7fecdc15ae80 2023-10-16T10:02:14.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.294+0000 7fece39ce640 1 -- 172.21.15.84:0/3274579014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecdc0c48c0 con 0x7fecdc15ae80 2023-10-16T10:02:14.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.294+0000 7fecd1ffb640 1 -- 172.21.15.84:0/3274579014 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4003ea0 con 0x7fecdc15ae80 2023-10-16T10:02:14.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.294+0000 7fece39ce640 1 -- 172.21.15.84:0/3274579014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecdc0c3f00 con 0x7fecdc15ae80 2023-10-16T10:02:14.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.295+0000 7fece39ce640 1 -- 172.21.15.84:0/3274579014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecdc15bf20 con 0x7fecdc15ae80 2023-10-16T10:02:14.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.296+0000 7fecd1ffb640 1 -- 172.21.15.84:0/3274579014 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fecd4005310 con 0x7fecdc15ae80 2023-10-16T10:02:14.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.297+0000 7fecd1ffb640 1 --2- 172.21.15.84:0/3274579014 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fecc006af90 0x7fecc006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.297+0000 7fecd1ffb640 1 -- 172.21.15.84:0/3274579014 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fecd4089540 con 0x7fecdc15ae80 2023-10-16T10:02:14.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.297+0000 7fece176a640 1 --2- 172.21.15.84:0/3274579014 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fecc006af90 0x7fecc006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.298+0000 7fece176a640 1 --2- 172.21.15.84:0/3274579014 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fecc006af90 0x7fecc006d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fecd8036000 tx=0x7fecd8008030 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:14.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.301+0000 7fecd1ffb640 1 -- 172.21.15.84:0/3274579014 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fecdc15bf20 con 0x7fecdc15ae80 2023-10-16T10:02:14.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.479+0000 7fece39ce640 1 -- 172.21.15.84:0/3274579014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fecdc15bf20 con 0x7fecdc15ae80 2023-10-16T10:02:14.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.482+0000 7fecd1ffb640 1 -- 172.21.15.84:0/3274579014 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16502 (secure 0 0 0) 0x7fecdc15bf20 con 0x7fecdc15ae80 2023-10-16T10:02:14.484 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:02:14.485 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":63,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:02:13.513714+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:02:11.266402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-16T10:02:12.562100+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:02:14.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.485+0000 7fece39ce640 1 -- 172.21.15.84:0/3274579014 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fecc006af90 msgr2=0x7fecc006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.485+0000 7fece39ce640 1 --2- 172.21.15.84:0/3274579014 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fecc006af90 0x7fecc006d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fecd8036000 tx=0x7fecd8008030 comp rx=0 tx=0).stop 2023-10-16T10:02:14.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.485+0000 7fece39ce640 1 -- 172.21.15.84:0/3274579014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecdc15ae80 msgr2=0x7fecdc155b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.485+0000 7fece39ce640 1 --2- 172.21.15.84:0/3274579014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecdc15ae80 0x7fecdc155b20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fecd400c6b0 tx=0x7fecd400cb70 comp rx=0 tx=0).stop 2023-10-16T10:02:14.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.485+0000 7fece39ce640 1 -- 172.21.15.84:0/3274579014 shutdown_connections 2023-10-16T10:02:14.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.485+0000 7fece39ce640 1 --2- 172.21.15.84:0/3274579014 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fecc006af90 0x7fecc006d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.485+0000 7fece39ce640 1 --2- 172.21.15.84:0/3274579014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecdc15ae80 0x7fecdc155b20 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.485+0000 7fece39ce640 1 --2- 172.21.15.84:0/3274579014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecdc15aaa0 0x7fecdc1574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.485+0000 7fece39ce640 1 --2- 172.21.15.84:0/3274579014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecdc0ccaa0 0x7fecdc159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.485+0000 7fece39ce640 1 -- 172.21.15.84:0/3274579014 >> 172.21.15.84:0/3274579014 conn(0x7fecdc0cb410 msgr2=0x7fecdc0ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:14.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.486+0000 7fece39ce640 1 -- 172.21.15.84:0/3274579014 shutdown_connections 2023-10-16T10:02:14.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.486+0000 7fece39ce640 1 -- 172.21.15.84:0/3274579014 wait complete. 2023-10-16T10:02:14.504 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:02:14.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.706+0000 7f89ab015640 1 Processor -- start 2023-10-16T10:02:14.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.706+0000 7f89ab015640 1 -- start start 2023-10-16T10:02:14.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.708+0000 7f89ab015640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a40cd930 0x7f89a4164240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.708+0000 7f89ab015640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a4159790 con 0x7f89a40caa10 2023-10-16T10:02:14.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.708+0000 7f89ab015640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a40c9b20 con 0x7f89a41593b0 2023-10-16T10:02:14.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.708+0000 7f89ab015640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a4070370 con 0x7f89a40cd930 2023-10-16T10:02:14.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.708+0000 7f89a3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f89a40caa10 0x7f89a406fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34898/0 (socket says 172.21.15.84:34898) 2023-10-16T10:02:14.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.708+0000 7f89a3fff640 1 -- 172.21.15.84:0/347632940 learned_addr learned my addr 172.21.15.84:0/347632940 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:14.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.708+0000 7f89a95b2640 1 --2- 172.21.15.84:0/347632940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a40cd930 0x7f89a4164240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.709+0000 7f89a37fe640 1 -- 172.21.15.84:0/347632940 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1024747956 0 0) 0x7f89a4159790 con 0x7f89a40caa10 2023-10-16T10:02:14.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.709+0000 7f89a37fe640 1 -- 172.21.15.84:0/347632940 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8994003440 con 0x7f89a40caa10 2023-10-16T10:02:14.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.709+0000 7f89a37fe640 1 -- 172.21.15.84:0/347632940 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 185873893 0 0) 0x7f8994003440 con 0x7f89a40caa10 2023-10-16T10:02:14.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.709+0000 7f89a37fe640 1 -- 172.21.15.84:0/347632940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a40cd930 msgr2=0x7f89a4164240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:14.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.709+0000 7f89a37fe640 1 --2- 172.21.15.84:0/347632940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a40cd930 0x7f89a4164240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.709+0000 7f89a37fe640 1 -- 172.21.15.84:0/347632940 >> v1:172.21.15.157:6789/0 conn(0x7f89a41593b0 legacy=0x7f89a406f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.709+0000 7f89a37fe640 1 -- 172.21.15.84:0/347632940 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a40c9e00 con 0x7f89a40caa10 2023-10-16T10:02:14.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.709+0000 7f89a37fe640 1 -- 172.21.15.84:0/347632940 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f8990001670 con 0x7f89a40caa10 2023-10-16T10:02:14.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.709+0000 7f89a37fe640 1 -- 172.21.15.84:0/347632940 >> v1:172.21.15.84:6789/0 conn(0x7f89a40caa10 legacy=0x7f89a406fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.709+0000 7f89a37fe640 1 --2- 172.21.15.84:0/347632940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a40cd930 0x7f8994003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.709+0000 7f89a37fe640 1 --2- 172.21.15.84:0/347632940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8994004060 0x7f899400a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.709+0000 7f89a37fe640 1 --2- 172.21.15.84:0/347632940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f899400b090 0x7f899400d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.709+0000 7f89a37fe640 1 -- 172.21.15.84:0/347632940 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a4070370 con 0x7f899400b090 2023-10-16T10:02:14.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.709+0000 7f89a37fe640 1 -- 172.21.15.84:0/347632940 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899400de00 con 0x7f8994004060 2023-10-16T10:02:14.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.709+0000 7f89a37fe640 1 -- 172.21.15.84:0/347632940 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899400dfc0 con 0x7f89a40cd930 2023-10-16T10:02:14.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.709+0000 7f89a95b2640 1 --2- 172.21.15.84:0/347632940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f899400b090 0x7f899400d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.710+0000 7f89a3fff640 1 --2- 172.21.15.84:0/347632940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8994004060 0x7f899400a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.710+0000 7f89a8db1640 1 --2- 172.21.15.84:0/347632940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a40cd930 0x7f8994003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.710+0000 7f89a95b2640 1 -- 172.21.15.84:0/347632940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a40cd930 msgr2=0x7f8994003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:14.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.710+0000 7f89a95b2640 1 --2- 172.21.15.84:0/347632940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a40cd930 0x7f8994003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.710+0000 7f89a95b2640 1 -- 172.21.15.84:0/347632940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8994004060 msgr2=0x7f899400a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:14.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.710+0000 7f89a95b2640 1 --2- 172.21.15.84:0/347632940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8994004060 0x7f899400a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.710+0000 7f89a95b2640 1 -- 172.21.15.84:0/347632940 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8994003440 con 0x7f899400b090 2023-10-16T10:02:14.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.710+0000 7f89a8db1640 1 --2- 172.21.15.84:0/347632940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a40cd930 0x7f8994003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:14.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.710+0000 7f89a3fff640 1 --2- 172.21.15.84:0/347632940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8994004060 0x7f899400a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:14.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.711+0000 7f89a95b2640 1 --2- 172.21.15.84:0/347632940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f899400b090 0x7f899400d520 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f899c009f90 tx=0x7f89940036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3348e2589ee56efc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:14.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.711+0000 7f89a37fe640 1 -- 172.21.15.84:0/347632940 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c02cbd0 con 0x7f899400b090 2023-10-16T10:02:14.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.711+0000 7f89a37fe640 1 -- 172.21.15.84:0/347632940 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899c00f040 con 0x7f899400b090 2023-10-16T10:02:14.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.711+0000 7f89a37fe640 1 -- 172.21.15.84:0/347632940 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c034440 con 0x7f899400b090 2023-10-16T10:02:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.712+0000 7f89ab015640 1 -- 172.21.15.84:0/347632940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f899400b090 msgr2=0x7f899400d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.712+0000 7f89ab015640 1 --2- 172.21.15.84:0/347632940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f899400b090 0x7f899400d520 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f899c009f90 tx=0x7f89940036f0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.712+0000 7f89a95b2640 1 -- 172.21.15.84:0/347632940 reap_dead start 2023-10-16T10:02:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.712+0000 7f89ab015640 1 -- 172.21.15.84:0/347632940 shutdown_connections 2023-10-16T10:02:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.712+0000 7f89ab015640 1 -- 172.21.15.84:0/347632940 >> 172.21.15.84:0/347632940 conn(0x7f89a40cb410 msgr2=0x7f89a40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.712+0000 7f89ab015640 1 -- 172.21.15.84:0/347632940 shutdown_connections 2023-10-16T10:02:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.712+0000 7f89ab015640 1 -- 172.21.15.84:0/347632940 wait complete. 2023-10-16T10:02:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.714+0000 7f89ab015640 1 Processor -- start 2023-10-16T10:02:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.714+0000 7f89ab015640 1 -- start start 2023-10-16T10:02:14.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.715+0000 7f89ab015640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f899400b090 0x7f89a41f2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.715+0000 7f89ab015640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a414db30 0x7f89a41ef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.715+0000 7f89ab015640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89a40c8270 0x7f89a40ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.715+0000 7f89ab015640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a406d2c0 con 0x7f89a40c8270 2023-10-16T10:02:14.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.715+0000 7f89ab015640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a406d4d0 con 0x7f899400b090 2023-10-16T10:02:14.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.715+0000 7f89ab015640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a406d7b0 con 0x7f89a414db30 2023-10-16T10:02:14.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.715+0000 7f89a95b2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89a40c8270 0x7f89a40ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.715+0000 7f89a95b2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89a40c8270 0x7f89a40ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33308/0 (socket says 172.21.15.84:33308) 2023-10-16T10:02:14.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.715+0000 7f89a95b2640 1 -- 172.21.15.84:0/2363680967 learned_addr learned my addr 172.21.15.84:0/2363680967 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:14.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.716+0000 7f89a95b2640 1 -- 172.21.15.84:0/2363680967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a414db30 msgr2=0x7f89a41ef1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.716+0000 7f89a95b2640 1 --2- 172.21.15.84:0/2363680967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a414db30 0x7f89a41ef1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.716+0000 7f89a95b2640 1 -- 172.21.15.84:0/2363680967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f899400b090 msgr2=0x7f89a41f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.716+0000 7f89a95b2640 1 --2- 172.21.15.84:0/2363680967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f899400b090 0x7f89a41f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.716+0000 7f89a95b2640 1 -- 172.21.15.84:0/2363680967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a415abf0 con 0x7f89a40c8270 2023-10-16T10:02:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.716+0000 7f89a95b2640 1 --2- 172.21.15.84:0/2363680967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89a40c8270 0x7f89a40ce920 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f899c037040 tx=0x7f899c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.716+0000 7f89a17fa640 1 -- 172.21.15.84:0/2363680967 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c003b90 con 0x7f89a40c8270 2023-10-16T10:02:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.716+0000 7f89a17fa640 1 -- 172.21.15.84:0/2363680967 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899c003cf0 con 0x7f89a40c8270 2023-10-16T10:02:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.716+0000 7f89ab015640 1 -- 172.21.15.84:0/2363680967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a415aed0 con 0x7f89a40c8270 2023-10-16T10:02:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.716+0000 7f89a17fa640 1 -- 172.21.15.84:0/2363680967 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c034a60 con 0x7f89a40c8270 2023-10-16T10:02:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.716+0000 7f89ab015640 1 -- 172.21.15.84:0/2363680967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a41ef6e0 con 0x7f89a40c8270 2023-10-16T10:02:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.718+0000 7f89ab015640 1 -- 172.21.15.84:0/2363680967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a404e160 con 0x7f89a40c8270 2023-10-16T10:02:14.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.718+0000 7f89a17fa640 1 -- 172.21.15.84:0/2363680967 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f899c034bc0 con 0x7f89a40c8270 2023-10-16T10:02:14.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.719+0000 7f89a17fa640 1 --2- 172.21.15.84:0/2363680967 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f898806afc0 0x7f898806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.719+0000 7f89a17fa640 1 -- 172.21.15.84:0/2363680967 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f899c0aaf50 con 0x7f89a40c8270 2023-10-16T10:02:14.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.720+0000 7f89a8db1640 1 --2- 172.21.15.84:0/2363680967 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f898806afc0 0x7f898806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.720+0000 7f89a8db1640 1 --2- 172.21.15.84:0/2363680967 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f898806afc0 0x7f898806d480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8998001670 tx=0x7f8998009040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:14.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.724+0000 7f89a17fa640 1 -- 172.21.15.84:0/2363680967 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f899c076030 con 0x7f89a40c8270 2023-10-16T10:02:14.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.920+0000 7f89ab015640 1 -- 172.21.15.84:0/2363680967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89a40b4910 con 0x7f89a40c8270 2023-10-16T10:02:14.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.922+0000 7f89a17fa640 1 -- 172.21.15.84:0/2363680967 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16502 (secure 0 0 0) 0x7f899c039090 con 0x7f89a40c8270 2023-10-16T10:02:14.924 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:02:14.925 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":63,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:02:13.513714+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:02:11.266402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-16T10:02:12.562100+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:02:14.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.925+0000 7f89ab015640 1 -- 172.21.15.84:0/2363680967 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f898806afc0 msgr2=0x7f898806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.925+0000 7f89ab015640 1 --2- 172.21.15.84:0/2363680967 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f898806afc0 0x7f898806d480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8998001670 tx=0x7f8998009040 comp rx=0 tx=0).stop 2023-10-16T10:02:14.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.925+0000 7f89ab015640 1 -- 172.21.15.84:0/2363680967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89a40c8270 msgr2=0x7f89a40ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.925+0000 7f89ab015640 1 --2- 172.21.15.84:0/2363680967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89a40c8270 0x7f89a40ce920 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f899c037040 tx=0x7f899c009750 comp rx=0 tx=0).stop 2023-10-16T10:02:14.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.925+0000 7f89ab015640 1 -- 172.21.15.84:0/2363680967 shutdown_connections 2023-10-16T10:02:14.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.925+0000 7f89ab015640 1 --2- 172.21.15.84:0/2363680967 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f898806afc0 0x7f898806d480 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.925+0000 7f89ab015640 1 --2- 172.21.15.84:0/2363680967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89a40c8270 0x7f89a40ce920 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.925+0000 7f89ab015640 1 --2- 172.21.15.84:0/2363680967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a414db30 0x7f89a41ef1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.925+0000 7f89ab015640 1 --2- 172.21.15.84:0/2363680967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f899400b090 0x7f89a41f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.925+0000 7f89ab015640 1 -- 172.21.15.84:0/2363680967 >> 172.21.15.84:0/2363680967 conn(0x7f89a40cb410 msgr2=0x7f89a40afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:14.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.925+0000 7f89ab015640 1 -- 172.21.15.84:0/2363680967 shutdown_connections 2023-10-16T10:02:14.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:14.925+0000 7f89ab015640 1 -- 172.21.15.84:0/2363680967 wait complete. 2023-10-16T10:02:14.946 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T10:02:15.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.154+0000 7f05e7615640 1 Processor -- start 2023-10-16T10:02:15.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.154+0000 7f05e7615640 1 -- start start 2023-10-16T10:02:15.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.155+0000 7f05e7615640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e0159ab0 0x7f05e00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.155+0000 7f05e7615640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e0070360 con 0x7f05e0159e90 2023-10-16T10:02:15.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.155+0000 7f05e7615640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e00cd1f0 con 0x7f05e006c140 2023-10-16T10:02:15.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.155+0000 7f05e7615640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e015d890 con 0x7f05e0159ab0 2023-10-16T10:02:15.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.156+0000 7f05e4bb0640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f05e0159e90 0x7f05e006fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34902/0 (socket says 172.21.15.84:34902) 2023-10-16T10:02:15.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.156+0000 7f05e4bb0640 1 -- 172.21.15.84:0/2823371444 learned_addr learned my addr 172.21.15.84:0/2823371444 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:15.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.156+0000 7f05d7fff640 1 -- 172.21.15.84:0/2823371444 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132532200 0 0) 0x7f05e0070360 con 0x7f05e0159e90 2023-10-16T10:02:15.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.156+0000 7f05d7fff640 1 -- 172.21.15.84:0/2823371444 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05cc003520 con 0x7f05e0159e90 2023-10-16T10:02:15.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.156+0000 7f05e53b1640 1 --2- 172.21.15.84:0/2823371444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e0159ab0 0x7f05e00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.156+0000 7f05d7fff640 1 -- 172.21.15.84:0/2823371444 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 737945716 0 0) 0x7f05cc003520 con 0x7f05e0159e90 2023-10-16T10:02:15.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.156+0000 7f05d7fff640 1 -- 172.21.15.84:0/2823371444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e0159ab0 msgr2=0x7f05e00ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:15.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.156+0000 7f05d7fff640 1 --2- 172.21.15.84:0/2823371444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e0159ab0 0x7f05e00ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.156+0000 7f05d7fff640 1 -- 172.21.15.84:0/2823371444 >> v1:172.21.15.157:6789/0 conn(0x7f05e006c140 legacy=0x7f05e015dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.156+0000 7f05d7fff640 1 -- 172.21.15.84:0/2823371444 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e00cd4d0 con 0x7f05e0159e90 2023-10-16T10:02:15.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.156+0000 7f05d7fff640 1 -- 172.21.15.84:0/2823371444 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f05d0001670 con 0x7f05e0159e90 2023-10-16T10:02:15.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.156+0000 7f05d7fff640 1 -- 172.21.15.84:0/2823371444 >> v1:172.21.15.84:6789/0 conn(0x7f05e0159e90 legacy=0x7f05e006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.156+0000 7f05d7fff640 1 --2- 172.21.15.84:0/2823371444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05cc0039a0 0x7f05cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.156+0000 7f05d7fff640 1 --2- 172.21.15.84:0/2823371444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e0159ab0 0x7f05cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.156+0000 7f05d7fff640 1 --2- 172.21.15.84:0/2823371444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05cc004d50 0x7f05cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.156+0000 7f05d7fff640 1 -- 172.21.15.84:0/2823371444 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e015d890 con 0x7f05cc0039a0 2023-10-16T10:02:15.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.157+0000 7f05d7fff640 1 -- 172.21.15.84:0/2823371444 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc00bc00 con 0x7f05cc004d50 2023-10-16T10:02:15.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.157+0000 7f05d7fff640 1 -- 172.21.15.84:0/2823371444 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc00dfd0 con 0x7f05e0159ab0 2023-10-16T10:02:15.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.157+0000 7f05e53b1640 1 --2- 172.21.15.84:0/2823371444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05cc0039a0 0x7f05cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.157+0000 7f05e4bb0640 1 --2- 172.21.15.84:0/2823371444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e0159ab0 0x7f05cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.157+0000 7f05e5bb2640 1 --2- 172.21.15.84:0/2823371444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05cc004d50 0x7f05cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.157+0000 7f05e53b1640 1 -- 172.21.15.84:0/2823371444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e0159ab0 msgr2=0x7f05cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:15.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.157+0000 7f05e53b1640 1 --2- 172.21.15.84:0/2823371444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e0159ab0 0x7f05cc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.157+0000 7f05e53b1640 1 -- 172.21.15.84:0/2823371444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05cc004d50 msgr2=0x7f05cc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:15.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.157+0000 7f05e53b1640 1 --2- 172.21.15.84:0/2823371444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05cc004d50 0x7f05cc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.157+0000 7f05e53b1640 1 -- 172.21.15.84:0/2823371444 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05cc003520 con 0x7f05cc0039a0 2023-10-16T10:02:15.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.157+0000 7f05e5bb2640 1 --2- 172.21.15.84:0/2823371444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05cc004d50 0x7f05cc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:15.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.157+0000 7f05e4bb0640 1 --2- 172.21.15.84:0/2823371444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e0159ab0 0x7f05cc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:15.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.158+0000 7f05e53b1640 1 --2- 172.21.15.84:0/2823371444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05cc0039a0 0x7f05cc003dc0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00ba00 tx=0x7f05cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d14df779663602c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:15.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.159+0000 7f05d7fff640 1 -- 172.21.15.84:0/2823371444 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc033680 con 0x7f05cc0039a0 2023-10-16T10:02:15.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.159+0000 7f05d7fff640 1 -- 172.21.15.84:0/2823371444 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05dc0337e0 con 0x7f05cc0039a0 2023-10-16T10:02:15.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.159+0000 7f05d7fff640 1 -- 172.21.15.84:0/2823371444 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc033b70 con 0x7f05cc0039a0 2023-10-16T10:02:15.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.159+0000 7f05e7615640 1 -- 172.21.15.84:0/2823371444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05cc0039a0 msgr2=0x7f05cc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.159+0000 7f05e7615640 1 --2- 172.21.15.84:0/2823371444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05cc0039a0 0x7f05cc003dc0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00ba00 tx=0x7f05cc0057f0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.159+0000 7f05e5bb2640 1 -- 172.21.15.84:0/2823371444 reap_dead start 2023-10-16T10:02:15.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.159+0000 7f05e7615640 1 -- 172.21.15.84:0/2823371444 shutdown_connections 2023-10-16T10:02:15.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.159+0000 7f05e7615640 1 -- 172.21.15.84:0/2823371444 >> 172.21.15.84:0/2823371444 conn(0x7f05e00ce9b0 msgr2=0x7f05e00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:15.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.159+0000 7f05e7615640 1 -- 172.21.15.84:0/2823371444 shutdown_connections 2023-10-16T10:02:15.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.159+0000 7f05e7615640 1 -- 172.21.15.84:0/2823371444 wait complete. 2023-10-16T10:02:15.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.161+0000 7f05e7615640 1 Processor -- start 2023-10-16T10:02:15.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.161+0000 7f05e7615640 1 -- start start 2023-10-16T10:02:15.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.162+0000 7f05e7615640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05cc0039a0 0x7f05e015b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.162+0000 7f05e7615640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e014cf30 0x7f05e015ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.162+0000 7f05e7615640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e006c520 0x7f05e006c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.162+0000 7f05e7615640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e006cee0 con 0x7f05cc0039a0 2023-10-16T10:02:15.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.162+0000 7f05e7615640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e0152480 con 0x7f05e014cf30 2023-10-16T10:02:15.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.162+0000 7f05e7615640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e015bce0 con 0x7f05e006c520 2023-10-16T10:02:15.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.162+0000 7f05e53b1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05cc0039a0 0x7f05e015b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.162+0000 7f05e53b1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05cc0039a0 0x7f05e015b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33324/0 (socket says 172.21.15.84:33324) 2023-10-16T10:02:15.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.162+0000 7f05e53b1640 1 -- 172.21.15.84:0/710041754 learned_addr learned my addr 172.21.15.84:0/710041754 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:15.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.163+0000 7f05e53b1640 1 -- 172.21.15.84:0/710041754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e006c520 msgr2=0x7f05e006c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.163+0000 7f05e53b1640 1 --2- 172.21.15.84:0/710041754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e006c520 0x7f05e006c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.163+0000 7f05e53b1640 1 -- 172.21.15.84:0/710041754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e014cf30 msgr2=0x7f05e015ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.163+0000 7f05e53b1640 1 --2- 172.21.15.84:0/710041754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e014cf30 0x7f05e015ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.163+0000 7f05e53b1640 1 -- 172.21.15.84:0/710041754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e015bfc0 con 0x7f05cc0039a0 2023-10-16T10:02:15.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.163+0000 7f05e53b1640 1 --2- 172.21.15.84:0/710041754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05cc0039a0 0x7f05e015b700 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00ba00 tx=0x7f05dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:15.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.163+0000 7f05d5ffb640 1 -- 172.21.15.84:0/710041754 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc004220 con 0x7f05cc0039a0 2023-10-16T10:02:15.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.163+0000 7f05d5ffb640 1 -- 172.21.15.84:0/710041754 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05dc040070 con 0x7f05cc0039a0 2023-10-16T10:02:15.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.163+0000 7f05d5ffb640 1 -- 172.21.15.84:0/710041754 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc033e20 con 0x7f05cc0039a0 2023-10-16T10:02:15.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.163+0000 7f05e7615640 1 -- 172.21.15.84:0/710041754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e015c2a0 con 0x7f05cc0039a0 2023-10-16T10:02:15.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.163+0000 7f05e7615640 1 -- 172.21.15.84:0/710041754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e014d3b0 con 0x7f05cc0039a0 2023-10-16T10:02:15.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.164+0000 7f05e7615640 1 -- 172.21.15.84:0/710041754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e01525c0 con 0x7f05cc0039a0 2023-10-16T10:02:15.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.165+0000 7f05d5ffb640 1 -- 172.21.15.84:0/710041754 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f05dc03c600 con 0x7f05cc0039a0 2023-10-16T10:02:15.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.166+0000 7f05d5ffb640 1 --2- 172.21.15.84:0/710041754 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f05c406af90 0x7f05c406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.166+0000 7f05d5ffb640 1 -- 172.21.15.84:0/710041754 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05dc0ab9a0 con 0x7f05cc0039a0 2023-10-16T10:02:15.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.167+0000 7f05e4bb0640 1 --2- 172.21.15.84:0/710041754 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f05c406af90 0x7f05c406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.167+0000 7f05e4bb0640 1 --2- 172.21.15.84:0/710041754 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f05c406af90 0x7f05c406d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f05d0000ff0 tx=0x7f05d0009040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:15.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.170+0000 7f05d5ffb640 1 -- 172.21.15.84:0/710041754 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f05dc07a3c0 con 0x7f05cc0039a0 2023-10-16T10:02:15.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:15.400+0000 7f05e7615640 1 -- 172.21.15.84:0/710041754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f05e004e160 con 0x7f05cc0039a0 2023-10-16T10:02:16.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.103+0000 7f05d5ffb640 1 -- 172.21.15.84:0/710041754 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f05dc0420b0 con 0x7f05cc0039a0 2023-10-16T10:02:16.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.107+0000 7f05e7615640 1 -- 172.21.15.84:0/710041754 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f05c406af90 msgr2=0x7f05c406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.107+0000 7f05e7615640 1 --2- 172.21.15.84:0/710041754 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f05c406af90 0x7f05c406d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f05d0000ff0 tx=0x7f05d0009040 comp rx=0 tx=0).stop 2023-10-16T10:02:16.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.107+0000 7f05e7615640 1 -- 172.21.15.84:0/710041754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05cc0039a0 msgr2=0x7f05e015b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.107+0000 7f05e7615640 1 --2- 172.21.15.84:0/710041754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05cc0039a0 0x7f05e015b700 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00ba00 tx=0x7f05dc004730 comp rx=0 tx=0).stop 2023-10-16T10:02:16.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.108+0000 7f05e7615640 1 -- 172.21.15.84:0/710041754 shutdown_connections 2023-10-16T10:02:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.108+0000 7f05e7615640 1 --2- 172.21.15.84:0/710041754 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f05c406af90 0x7f05c406d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.108+0000 7f05e7615640 1 --2- 172.21.15.84:0/710041754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05e006c520 0x7f05e006c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.108+0000 7f05e7615640 1 --2- 172.21.15.84:0/710041754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e014cf30 0x7f05e015ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.108+0000 7f05e7615640 1 --2- 172.21.15.84:0/710041754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05cc0039a0 0x7f05e015b700 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.108+0000 7f05e7615640 1 -- 172.21.15.84:0/710041754 >> 172.21.15.84:0/710041754 conn(0x7f05e00ce9b0 msgr2=0x7f05e0158c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:16.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.108+0000 7f05e7615640 1 -- 172.21.15.84:0/710041754 shutdown_connections 2023-10-16T10:02:16.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.108+0000 7f05e7615640 1 -- 172.21.15.84:0/710041754 wait complete. 2023-10-16T10:02:16.131 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T10:02:16.132 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T10:02:16.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.352+0000 7fe7ff00e640 1 Processor -- start 2023-10-16T10:02:16.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.352+0000 7fe7ff00e640 1 -- start start 2023-10-16T10:02:16.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.354+0000 7fe7ff00e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe7f81618b0 0x7fe7f8065ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.354+0000 7fe7ff00e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f806cee0 con 0x7fe7f815d330 2023-10-16T10:02:16.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.354+0000 7fe7ff00e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f80663d0 con 0x7fe7f8065ff0 2023-10-16T10:02:16.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.354+0000 7fe7ff00e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f806b1e0 con 0x7fe7f81618b0 2023-10-16T10:02:16.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.354+0000 7fe7fcdaa640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe7f815d330 0x7fe7f806c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34202/0 (socket says 172.21.15.84:34202) 2023-10-16T10:02:16.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.354+0000 7fe7fcdaa640 1 -- 172.21.15.84:0/3561515028 learned_addr learned my addr 172.21.15.84:0/3561515028 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:16.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.355+0000 7fe7f77fe640 1 -- 172.21.15.84:0/3561515028 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1445318952 0 0) 0x7fe7f806cee0 con 0x7fe7f815d330 2023-10-16T10:02:16.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.355+0000 7fe7f77fe640 1 -- 172.21.15.84:0/3561515028 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7d8003520 con 0x7fe7f815d330 2023-10-16T10:02:16.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.355+0000 7fe7f7fff640 1 --2- 172.21.15.84:0/3561515028 >> v2:172.21.15.157:3300/0 conn(0x7fe7f81618b0 0x7fe7f8065ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.355+0000 7fe7f77fe640 1 -- 172.21.15.84:0/3561515028 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2747867440 0 0) 0x7fe7d8003520 con 0x7fe7f815d330 2023-10-16T10:02:16.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.355+0000 7fe7f77fe640 1 -- 172.21.15.84:0/3561515028 >> v2:172.21.15.157:3300/0 conn(0x7fe7f81618b0 msgr2=0x7fe7f8065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:16.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.355+0000 7fe7f77fe640 1 --2- 172.21.15.84:0/3561515028 >> v2:172.21.15.157:3300/0 conn(0x7fe7f81618b0 0x7fe7f8065ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.355+0000 7fe7f77fe640 1 -- 172.21.15.84:0/3561515028 >> v1:172.21.15.157:6789/0 conn(0x7fe7f8065ff0 legacy=0x7fe7f80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:16.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.355+0000 7fe7f77fe640 1 -- 172.21.15.84:0/3561515028 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f80d0890 con 0x7fe7f815d330 2023-10-16T10:02:16.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.355+0000 7fe7f77fe640 1 -- 172.21.15.84:0/3561515028 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fe7e8001670 con 0x7fe7f815d330 2023-10-16T10:02:16.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.355+0000 7fe7f77fe640 1 -- 172.21.15.84:0/3561515028 >> v1:172.21.15.84:6789/0 conn(0x7fe7f815d330 legacy=0x7fe7f806c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.355+0000 7fe7f77fe640 1 --2- 172.21.15.84:0/3561515028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d80039c0 0x7fe7d8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.355+0000 7fe7f77fe640 1 --2- 172.21.15.84:0/3561515028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7d80047c0 0x7fe7d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.355+0000 7fe7f77fe640 1 --2- 172.21.15.84:0/3561515028 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7d8005250 0x7fe7d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.355+0000 7fe7f77fe640 1 -- 172.21.15.84:0/3561515028 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f806b1e0 con 0x7fe7d80047c0 2023-10-16T10:02:16.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.355+0000 7fe7f77fe640 1 -- 172.21.15.84:0/3561515028 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d800c1a0 con 0x7fe7d80039c0 2023-10-16T10:02:16.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.355+0000 7fe7f77fe640 1 -- 172.21.15.84:0/3561515028 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d800c360 con 0x7fe7d8005250 2023-10-16T10:02:16.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.356+0000 7fe7f7fff640 1 --2- 172.21.15.84:0/3561515028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7d80047c0 0x7fe7d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.356+0000 7fe7fcdaa640 1 --2- 172.21.15.84:0/3561515028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d80039c0 0x7fe7d8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.356+0000 7fe7f7fff640 1 -- 172.21.15.84:0/3561515028 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7d8005250 msgr2=0x7fe7d800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:16.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.356+0000 7fe7f7fff640 1 --2- 172.21.15.84:0/3561515028 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7d8005250 0x7fe7d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.356+0000 7fe7f7fff640 1 -- 172.21.15.84:0/3561515028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d80039c0 msgr2=0x7fe7d8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:16.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.356+0000 7fe7f7fff640 1 --2- 172.21.15.84:0/3561515028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d80039c0 0x7fe7d8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.356+0000 7fe7f7fff640 1 -- 172.21.15.84:0/3561515028 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d8003520 con 0x7fe7d80047c0 2023-10-16T10:02:16.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.356+0000 7fe7fd5ab640 1 -- 172.21.15.84:0/3561515028 reap_dead start 2023-10-16T10:02:16.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.357+0000 7fe7f7fff640 1 --2- 172.21.15.84:0/3561515028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7d80047c0 0x7fe7d8004c40 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0007360 tx=0x7fe7d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258bb1755630d4a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:16.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.357+0000 7fe7f77fe640 1 -- 172.21.15.84:0/3561515028 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e002ccd0 con 0x7fe7d80047c0 2023-10-16T10:02:16.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.357+0000 7fe7f77fe640 1 -- 172.21.15.84:0/3561515028 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e002ce30 con 0x7fe7d80047c0 2023-10-16T10:02:16.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.357+0000 7fe7f77fe640 1 -- 172.21.15.84:0/3561515028 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0034660 con 0x7fe7d80047c0 2023-10-16T10:02:16.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.358+0000 7fe7ff00e640 1 -- 172.21.15.84:0/3561515028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7d80047c0 msgr2=0x7fe7d8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.358+0000 7fe7ff00e640 1 --2- 172.21.15.84:0/3561515028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7d80047c0 0x7fe7d8004c40 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0007360 tx=0x7fe7d80057f0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.358+0000 7fe7ff00e640 1 -- 172.21.15.84:0/3561515028 shutdown_connections 2023-10-16T10:02:16.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.358+0000 7fe7ff00e640 1 --2- 172.21.15.84:0/3561515028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7d80047c0 0x7fe7d8004c40 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.358+0000 7fe7ff00e640 1 -- 172.21.15.84:0/3561515028 >> 172.21.15.84:0/3561515028 conn(0x7fe7f80d0ab0 msgr2=0x7fe7f80d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:16.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.358+0000 7fe7ff00e640 1 -- 172.21.15.84:0/3561515028 shutdown_connections 2023-10-16T10:02:16.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.358+0000 7fe7ff00e640 1 -- 172.21.15.84:0/3561515028 wait complete. 2023-10-16T10:02:16.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.360+0000 7fe7ff00e640 1 Processor -- start 2023-10-16T10:02:16.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.360+0000 7fe7ff00e640 1 -- start start 2023-10-16T10:02:16.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.361+0000 7fe7ff00e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7d80047c0 0x7fe7f80aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.361+0000 7fe7ff00e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f81f6ed0 0x7fe7f815d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.361+0000 7fe7ff00e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f815dc50 0x7fe7f8159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.361+0000 7fe7ff00e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f815e560 con 0x7fe7f815dc50 2023-10-16T10:02:16.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.361+0000 7fe7ff00e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f80cbbb0 con 0x7fe7f81f6ed0 2023-10-16T10:02:16.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.361+0000 7fe7ff00e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f815e280 con 0x7fe7d80047c0 2023-10-16T10:02:16.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.361+0000 7fe7fd5ab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f815dc50 0x7fe7f8159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.361+0000 7fe7fd5ab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f815dc50 0x7fe7f8159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44636/0 (socket says 172.21.15.84:44636) 2023-10-16T10:02:16.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.361+0000 7fe7fd5ab640 1 -- 172.21.15.84:0/1933407345 learned_addr learned my addr 172.21.15.84:0/1933407345 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:16.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.362+0000 7fe7fd5ab640 1 -- 172.21.15.84:0/1933407345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7d80047c0 msgr2=0x7fe7f80aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.362+0000 7fe7fcdaa640 1 --2- 172.21.15.84:0/1933407345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7d80047c0 0x7fe7f80aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.362+0000 7fe7fd5ab640 1 --2- 172.21.15.84:0/1933407345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7d80047c0 0x7fe7f80aefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.362+0000 7fe7fd5ab640 1 -- 172.21.15.84:0/1933407345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f81f6ed0 msgr2=0x7fe7f815d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.362+0000 7fe7fd5ab640 1 --2- 172.21.15.84:0/1933407345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f81f6ed0 0x7fe7f815d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.362+0000 7fe7fd5ab640 1 -- 172.21.15.84:0/1933407345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f81f63f0 con 0x7fe7f815dc50 2023-10-16T10:02:16.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.362+0000 7fe7fd5ab640 1 --2- 172.21.15.84:0/1933407345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f815dc50 0x7fe7f8159140 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80afa70 tx=0x7fe7f000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:16.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.362+0000 7fe7f57fa640 1 -- 172.21.15.84:0/1933407345 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f00074e0 con 0x7fe7f815dc50 2023-10-16T10:02:16.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.362+0000 7fe7f57fa640 1 -- 172.21.15.84:0/1933407345 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f00044a0 con 0x7fe7f815dc50 2023-10-16T10:02:16.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.362+0000 7fe7f57fa640 1 -- 172.21.15.84:0/1933407345 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0005a00 con 0x7fe7f815dc50 2023-10-16T10:02:16.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.362+0000 7fe7ff00e640 1 -- 172.21.15.84:0/1933407345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f81f66d0 con 0x7fe7f815dc50 2023-10-16T10:02:16.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.363+0000 7fe7ff00e640 1 -- 172.21.15.84:0/1933407345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f80af500 con 0x7fe7f815dc50 2023-10-16T10:02:16.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.364+0000 7fe7ff00e640 1 -- 172.21.15.84:0/1933407345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f81f6250 con 0x7fe7f815dc50 2023-10-16T10:02:16.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.364+0000 7fe7f57fa640 1 -- 172.21.15.84:0/1933407345 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe7f0005b60 con 0x7fe7f815dc50 2023-10-16T10:02:16.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.365+0000 7fe7f57fa640 1 --2- 172.21.15.84:0/1933407345 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fe7dc06af90 0x7fe7dc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.365+0000 7fe7f57fa640 1 -- 172.21.15.84:0/1933407345 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7f0088900 con 0x7fe7f815dc50 2023-10-16T10:02:16.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.366+0000 7fe7fcdaa640 1 --2- 172.21.15.84:0/1933407345 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fe7dc06af90 0x7fe7dc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.366+0000 7fe7fcdaa640 1 --2- 172.21.15.84:0/1933407345 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fe7dc06af90 0x7fe7dc06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8001eb0 tx=0x7fe7e8000f70 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:16.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.370+0000 7fe7f57fa640 1 -- 172.21.15.84:0/1933407345 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe7f81f6250 con 0x7fe7f815dc50 2023-10-16T10:02:16.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:16.596+0000 7fe7ff00e640 1 -- 172.21.15.84:0/1933407345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe7f81f6250 con 0x7fe7f815dc50 2023-10-16T10:02:17.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.106+0000 7fe7f57fa640 1 -- 172.21.15.84:0/1933407345 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe7f81f6250 con 0x7fe7f815dc50 2023-10-16T10:02:17.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.110+0000 7fe7ff00e640 1 -- 172.21.15.84:0/1933407345 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fe7dc06af90 msgr2=0x7fe7dc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:17.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.110+0000 7fe7ff00e640 1 --2- 172.21.15.84:0/1933407345 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fe7dc06af90 0x7fe7dc06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8001eb0 tx=0x7fe7e8000f70 comp rx=0 tx=0).stop 2023-10-16T10:02:17.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.110+0000 7fe7ff00e640 1 -- 172.21.15.84:0/1933407345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f815dc50 msgr2=0x7fe7f8159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:17.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.110+0000 7fe7ff00e640 1 --2- 172.21.15.84:0/1933407345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f815dc50 0x7fe7f8159140 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80afa70 tx=0x7fe7f000c820 comp rx=0 tx=0).stop 2023-10-16T10:02:17.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.110+0000 7fe7ff00e640 1 -- 172.21.15.84:0/1933407345 shutdown_connections 2023-10-16T10:02:17.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.110+0000 7fe7ff00e640 1 --2- 172.21.15.84:0/1933407345 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fe7dc06af90 0x7fe7dc06d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.110+0000 7fe7ff00e640 1 --2- 172.21.15.84:0/1933407345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f815dc50 0x7fe7f8159140 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.110+0000 7fe7ff00e640 1 --2- 172.21.15.84:0/1933407345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f81f6ed0 0x7fe7f815d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.110+0000 7fe7ff00e640 1 --2- 172.21.15.84:0/1933407345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7d80047c0 0x7fe7f80aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.110+0000 7fe7ff00e640 1 -- 172.21.15.84:0/1933407345 >> 172.21.15.84:0/1933407345 conn(0x7fe7f80d0ab0 msgr2=0x7fe7f80cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:17.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.110+0000 7fe7ff00e640 1 -- 172.21.15.84:0/1933407345 shutdown_connections 2023-10-16T10:02:17.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.111+0000 7fe7ff00e640 1 -- 172.21.15.84:0/1933407345 wait complete. 2023-10-16T10:02:17.133 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T10:02:17.134 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:02:17.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.331+0000 7fca48947640 1 Processor -- start 2023-10-16T10:02:17.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.332+0000 7fca48947640 1 -- start start 2023-10-16T10:02:17.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.333+0000 7fca48947640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fca44159950 0x7fca4415a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.333+0000 7fca48947640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca44070450 0x7fca440708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.333+0000 7fca48947640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca44070df0 con 0x7fca4406c100 2023-10-16T10:02:17.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.333+0000 7fca48947640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca4415e490 con 0x7fca44159950 2023-10-16T10:02:17.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.333+0000 7fca48947640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4415aef0 con 0x7fca44070450 2023-10-16T10:02:17.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.333+0000 7fca437fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fca44159950 0x7fca4415a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:17.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.333+0000 7fca437fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fca44159950 0x7fca4415a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44638/0 (socket says 172.21.15.84:44638) 2023-10-16T10:02:17.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.333+0000 7fca437fe640 1 -- 172.21.15.84:0/4199152385 learned_addr learned my addr 172.21.15.84:0/4199152385 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:17.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.334+0000 7fca3bfff640 1 --2- 172.21.15.84:0/4199152385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca44070450 0x7fca440708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:17.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.334+0000 7fca437fe640 1 -- 172.21.15.84:0/4199152385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca44070450 msgr2=0x7fca440708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:17.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.334+0000 7fca437fe640 1 --2- 172.21.15.84:0/4199152385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca44070450 0x7fca440708b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.334+0000 7fca437fe640 1 -- 172.21.15.84:0/4199152385 >> v1:172.21.15.157:6789/0 conn(0x7fca4406c100 legacy=0x7fca4406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:17.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.334+0000 7fca437fe640 1 -- 172.21.15.84:0/4199152385 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4406cc80 con 0x7fca44159950 2023-10-16T10:02:17.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.335+0000 7fca437fe640 1 --2- 172.21.15.84:0/4199152385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca44159950 0x7fca4415a9b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fca34009870 tx=0x7fca3402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68dd340281900408 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:17.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.335+0000 7fca42ffd640 1 -- 172.21.15.84:0/4199152385 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca340317c0 con 0x7fca44159950 2023-10-16T10:02:17.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.335+0000 7fca42ffd640 1 -- 172.21.15.84:0/4199152385 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca34031920 con 0x7fca44159950 2023-10-16T10:02:17.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.335+0000 7fca42ffd640 1 -- 172.21.15.84:0/4199152385 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34031cb0 con 0x7fca44159950 2023-10-16T10:02:17.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.336+0000 7fca48947640 1 -- 172.21.15.84:0/4199152385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca44159950 msgr2=0x7fca4415a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:17.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.336+0000 7fca48947640 1 --2- 172.21.15.84:0/4199152385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca44159950 0x7fca4415a9b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fca34009870 tx=0x7fca3402b860 comp rx=0 tx=0).stop 2023-10-16T10:02:17.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.336+0000 7fca48947640 1 -- 172.21.15.84:0/4199152385 shutdown_connections 2023-10-16T10:02:17.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.336+0000 7fca48947640 1 --2- 172.21.15.84:0/4199152385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca44070450 0x7fca440708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.336+0000 7fca48947640 1 --2- 172.21.15.84:0/4199152385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca44159950 0x7fca4415a9b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.336+0000 7fca48947640 1 -- 172.21.15.84:0/4199152385 >> 172.21.15.84:0/4199152385 conn(0x7fca440ce6a0 msgr2=0x7fca440ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:17.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.336+0000 7fca48947640 1 -- 172.21.15.84:0/4199152385 shutdown_connections 2023-10-16T10:02:17.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.336+0000 7fca48947640 1 -- 172.21.15.84:0/4199152385 wait complete. 2023-10-16T10:02:17.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.337+0000 7fca48947640 1 Processor -- start 2023-10-16T10:02:17.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.338+0000 7fca48947640 1 -- start start 2023-10-16T10:02:17.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.338+0000 7fca48947640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca4406c100 0x7fca441f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.338+0000 7fca48947640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca44070450 0x7fca440c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.338+0000 7fca48947640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca44159950 0x7fca440b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.338+0000 7fca48947640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca440c83b0 con 0x7fca4406c100 2023-10-16T10:02:17.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.338+0000 7fca48947640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca440c7500 con 0x7fca44159950 2023-10-16T10:02:17.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.338+0000 7fca48947640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca441f0610 con 0x7fca44070450 2023-10-16T10:02:17.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.339+0000 7fca437fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca4406c100 0x7fca441f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:17.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.339+0000 7fca437fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca4406c100 0x7fca441f00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44648/0 (socket says 172.21.15.84:44648) 2023-10-16T10:02:17.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.339+0000 7fca437fe640 1 -- 172.21.15.84:0/3178033951 learned_addr learned my addr 172.21.15.84:0/3178033951 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:17.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.339+0000 7fca43fff640 1 --2- 172.21.15.84:0/3178033951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca44159950 0x7fca440b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:17.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.339+0000 7fca3bfff640 1 --2- 172.21.15.84:0/3178033951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca44070450 0x7fca440c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:17.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.339+0000 7fca437fe640 1 -- 172.21.15.84:0/3178033951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca44070450 msgr2=0x7fca440c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:17.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.339+0000 7fca437fe640 1 --2- 172.21.15.84:0/3178033951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca44070450 0x7fca440c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.339+0000 7fca437fe640 1 -- 172.21.15.84:0/3178033951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca44159950 msgr2=0x7fca440b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:17.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.339+0000 7fca437fe640 1 --2- 172.21.15.84:0/3178033951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca44159950 0x7fca440b0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.339+0000 7fca437fe640 1 -- 172.21.15.84:0/3178033951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca340094d0 con 0x7fca4406c100 2023-10-16T10:02:17.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.339+0000 7fca43fff640 1 --2- 172.21.15.84:0/3178033951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca44159950 0x7fca440b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:17.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.339+0000 7fca437fe640 1 --2- 172.21.15.84:0/3178033951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca4406c100 0x7fca441f00d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fca3402b8a0 tx=0x7fca340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:17.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.340+0000 7fca40ff9640 1 -- 172.21.15.84:0/3178033951 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca340329a0 con 0x7fca4406c100 2023-10-16T10:02:17.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.340+0000 7fca40ff9640 1 -- 172.21.15.84:0/3178033951 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca34032b40 con 0x7fca4406c100 2023-10-16T10:02:17.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.340+0000 7fca40ff9640 1 -- 172.21.15.84:0/3178033951 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca340453f0 con 0x7fca4406c100 2023-10-16T10:02:17.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.340+0000 7fca48947640 1 -- 172.21.15.84:0/3178033951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca440c77a0 con 0x7fca4406c100 2023-10-16T10:02:17.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.340+0000 7fca48947640 1 -- 172.21.15.84:0/3178033951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca440c7d60 con 0x7fca4406c100 2023-10-16T10:02:17.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.341+0000 7fca48947640 1 -- 172.21.15.84:0/3178033951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4404e9b0 con 0x7fca4406c100 2023-10-16T10:02:17.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.342+0000 7fca40ff9640 1 -- 172.21.15.84:0/3178033951 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fca3404e450 con 0x7fca4406c100 2023-10-16T10:02:17.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.343+0000 7fca40ff9640 1 --2- 172.21.15.84:0/3178033951 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fca2406af90 0x7fca2406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.343+0000 7fca40ff9640 1 -- 172.21.15.84:0/3178033951 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca340abc50 con 0x7fca4406c100 2023-10-16T10:02:17.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.343+0000 7fca3bfff640 1 --2- 172.21.15.84:0/3178033951 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fca2406af90 0x7fca2406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:17.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.344+0000 7fca3bfff640 1 --2- 172.21.15.84:0/3178033951 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fca2406af90 0x7fca2406d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fca2c006fd0 tx=0x7fca2c008040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:17.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.347+0000 7fca40ff9640 1 -- 172.21.15.84:0/3178033951 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fca34076c70 con 0x7fca4406c100 2023-10-16T10:02:17.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.560+0000 7fca48947640 1 -- 172.21.15.84:0/3178033951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca4415dea0 con 0x7fca4406c100 2023-10-16T10:02:17.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.561+0000 7fca40ff9640 1 -- 172.21.15.84:0/3178033951 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fca3407a290 con 0x7fca4406c100 2023-10-16T10:02:17.563 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:02:17.563 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3040499154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3040499154},{"type":"v1","addr":"172.21.15.157:6835","nonce":3040499154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2727537766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2727537766},{"type":"v1","addr":"172.21.15.157:6833","nonce":2727537766}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/49292950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":49292950},{"type":"v1","addr":"172.21.15.84:6836","nonce":49292950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:02:13.514054+0000","modified":"2023-10-16T10:02:16.598055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/3839442164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3839442164},{"type":"v1","addr":"172.21.15.84:6837","nonce":3839442164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T10:02:17.564 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 25 2023-10-16T10:02:17.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.564+0000 7fca48947640 1 -- 172.21.15.84:0/3178033951 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fca2406af90 msgr2=0x7fca2406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:17.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.564+0000 7fca48947640 1 --2- 172.21.15.84:0/3178033951 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fca2406af90 0x7fca2406d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fca2c006fd0 tx=0x7fca2c008040 comp rx=0 tx=0).stop 2023-10-16T10:02:17.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.564+0000 7fca48947640 1 -- 172.21.15.84:0/3178033951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca4406c100 msgr2=0x7fca441f00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:17.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.564+0000 7fca48947640 1 --2- 172.21.15.84:0/3178033951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca4406c100 0x7fca441f00d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fca3402b8a0 tx=0x7fca340046c0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.564+0000 7fca48947640 1 -- 172.21.15.84:0/3178033951 shutdown_connections 2023-10-16T10:02:17.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.564+0000 7fca48947640 1 --2- 172.21.15.84:0/3178033951 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fca2406af90 0x7fca2406d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.564+0000 7fca48947640 1 --2- 172.21.15.84:0/3178033951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca44159950 0x7fca440b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.564+0000 7fca48947640 1 --2- 172.21.15.84:0/3178033951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca44070450 0x7fca440c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.564+0000 7fca48947640 1 --2- 172.21.15.84:0/3178033951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca4406c100 0x7fca441f00d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.564+0000 7fca48947640 1 -- 172.21.15.84:0/3178033951 >> 172.21.15.84:0/3178033951 conn(0x7fca440ce6a0 msgr2=0x7fca440ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:17.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.564+0000 7fca48947640 1 -- 172.21.15.84:0/3178033951 shutdown_connections 2023-10-16T10:02:17.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.564+0000 7fca48947640 1 -- 172.21.15.84:0/3178033951 wait complete. 2023-10-16T10:02:17.583 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:02:17.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.787+0000 7f1844683640 1 Processor -- start 2023-10-16T10:02:17.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.788+0000 7f1844683640 1 -- start start 2023-10-16T10:02:17.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.789+0000 7f1844683640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f183c0b0b20 0x7f183c1615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.789+0000 7f1844683640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f183c15d3f0 0x7f183c15d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.789+0000 7f1844683640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f183c15e3e0 con 0x7f183c0cd0c0 2023-10-16T10:02:17.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.789+0000 7f1844683640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f183c0cd4a0 con 0x7f183c15d3f0 2023-10-16T10:02:17.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.789+0000 7f1844683640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f183c15e100 con 0x7f183c0b0b20 2023-10-16T10:02:17.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.789+0000 7f184241f640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f183c0cd0c0 0x7f183c0b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34206/0 (socket says 172.21.15.84:34206) 2023-10-16T10:02:17.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.790+0000 7f184241f640 1 -- 172.21.15.84:0/1127046390 learned_addr learned my addr 172.21.15.84:0/1127046390 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:17.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.790+0000 7f1842c20640 1 --2- 172.21.15.84:0/1127046390 >> v2:172.21.15.84:3300/0 conn(0x7f183c15d3f0 0x7f183c15d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:17.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.790+0000 7f1841c1e640 1 --2- 172.21.15.84:0/1127046390 >> v2:172.21.15.157:3300/0 conn(0x7f183c0b0b20 0x7f183c1615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:17.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.790+0000 7f1842c20640 1 -- 172.21.15.84:0/1127046390 >> v2:172.21.15.157:3300/0 conn(0x7f183c0b0b20 msgr2=0x7f183c1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:17.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.790+0000 7f1842c20640 1 --2- 172.21.15.84:0/1127046390 >> v2:172.21.15.157:3300/0 conn(0x7f183c0b0b20 0x7f183c1615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.790+0000 7f1842c20640 1 -- 172.21.15.84:0/1127046390 >> v1:172.21.15.84:6789/0 conn(0x7f183c0cd0c0 legacy=0x7f183c0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:17.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.790+0000 7f1842c20640 1 -- 172.21.15.84:0/1127046390 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183c160b10 con 0x7f183c15d3f0 2023-10-16T10:02:17.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.791+0000 7f1842c20640 1 --2- 172.21.15.84:0/1127046390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f183c15d3f0 0x7f183c15d820 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f183c15afc0 tx=0x7f183402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e89c767d4e87f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:17.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.791+0000 7f184141d640 1 -- 172.21.15.84:0/1127046390 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183402cbf0 con 0x7f183c15d3f0 2023-10-16T10:02:17.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.792+0000 7f184141d640 1 -- 172.21.15.84:0/1127046390 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183402cd50 con 0x7f183c15d3f0 2023-10-16T10:02:17.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.792+0000 7f184141d640 1 -- 172.21.15.84:0/1127046390 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1834034660 con 0x7f183c15d3f0 2023-10-16T10:02:17.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.792+0000 7f1844683640 1 -- 172.21.15.84:0/1127046390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f183c15d3f0 msgr2=0x7f183c15d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:17.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.792+0000 7f1844683640 1 --2- 172.21.15.84:0/1127046390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f183c15d3f0 0x7f183c15d820 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f183c15afc0 tx=0x7f183402be40 comp rx=0 tx=0).stop 2023-10-16T10:02:17.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.792+0000 7f1844683640 1 -- 172.21.15.84:0/1127046390 shutdown_connections 2023-10-16T10:02:17.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.792+0000 7f1844683640 1 --2- 172.21.15.84:0/1127046390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f183c15d3f0 0x7f183c15d820 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.792+0000 7f1844683640 1 --2- 172.21.15.84:0/1127046390 >> v2:172.21.15.157:3300/0 conn(0x7f183c0b0b20 0x7f183c1615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.792+0000 7f1844683640 1 -- 172.21.15.84:0/1127046390 >> 172.21.15.84:0/1127046390 conn(0x7f183c0cb410 msgr2=0x7f183c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.792+0000 7f1844683640 1 -- 172.21.15.84:0/1127046390 shutdown_connections 2023-10-16T10:02:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.792+0000 7f1844683640 1 -- 172.21.15.84:0/1127046390 wait complete. 2023-10-16T10:02:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.794+0000 7f1844683640 1 Processor -- start 2023-10-16T10:02:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.794+0000 7f1844683640 1 -- start start 2023-10-16T10:02:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.795+0000 7f1844683640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f183c0b0b20 0x7f183c15fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.795+0000 7f1844683640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f183c0cd0c0 0x7f183c156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.795+0000 7f1844683640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f183c15d3f0 0x7f183c156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.795+0000 7f1844683640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183c0c3320 con 0x7f183c0b0b20 2023-10-16T10:02:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.795+0000 7f1844683640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183c0c8850 con 0x7f183c0cd0c0 2023-10-16T10:02:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.795+0000 7f1844683640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183c0c2560 con 0x7f183c15d3f0 2023-10-16T10:02:17.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.795+0000 7f184241f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f183c0b0b20 0x7f183c15fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:17.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.795+0000 7f184241f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f183c0b0b20 0x7f183c15fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44660/0 (socket says 172.21.15.84:44660) 2023-10-16T10:02:17.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.795+0000 7f184241f640 1 -- 172.21.15.84:0/2434147875 learned_addr learned my addr 172.21.15.84:0/2434147875 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:17.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.796+0000 7f184241f640 1 -- 172.21.15.84:0/2434147875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f183c15d3f0 msgr2=0x7f183c156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:17.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.796+0000 7f184241f640 1 --2- 172.21.15.84:0/2434147875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f183c15d3f0 0x7f183c156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.796+0000 7f184241f640 1 -- 172.21.15.84:0/2434147875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f183c0cd0c0 msgr2=0x7f183c156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:17.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.796+0000 7f184241f640 1 --2- 172.21.15.84:0/2434147875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f183c0cd0c0 0x7f183c156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.796+0000 7f184241f640 1 -- 172.21.15.84:0/2434147875 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1834009a70 con 0x7f183c0b0b20 2023-10-16T10:02:17.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.796+0000 7f184241f640 1 --2- 172.21.15.84:0/2434147875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f183c0b0b20 0x7f183c15fea0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f183800c3c0 tx=0x7f183800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:17.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.796+0000 7f182effd640 1 -- 172.21.15.84:0/2434147875 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838005ce0 con 0x7f183c0b0b20 2023-10-16T10:02:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.796+0000 7f182effd640 1 -- 172.21.15.84:0/2434147875 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1838005e40 con 0x7f183c0b0b20 2023-10-16T10:02:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.796+0000 7f182effd640 1 -- 172.21.15.84:0/2434147875 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18380072d0 con 0x7f183c0b0b20 2023-10-16T10:02:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.796+0000 7f1844683640 1 -- 172.21.15.84:0/2434147875 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183c0c2840 con 0x7f183c0b0b20 2023-10-16T10:02:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.796+0000 7f1844683640 1 -- 172.21.15.84:0/2434147875 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f183c0c2e00 con 0x7f183c0b0b20 2023-10-16T10:02:17.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.797+0000 7f1844683640 1 -- 172.21.15.84:0/2434147875 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f183c04e160 con 0x7f183c0b0b20 2023-10-16T10:02:17.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.798+0000 7f182effd640 1 -- 172.21.15.84:0/2434147875 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1838007430 con 0x7f183c0b0b20 2023-10-16T10:02:17.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.799+0000 7f182effd640 1 --2- 172.21.15.84:0/2434147875 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f182006afc0 0x7f182006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.799+0000 7f182effd640 1 -- 172.21.15.84:0/2434147875 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1838088cd0 con 0x7f183c0b0b20 2023-10-16T10:02:17.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.800+0000 7f1841c1e640 1 --2- 172.21.15.84:0/2434147875 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f182006afc0 0x7f182006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:17.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.800+0000 7f1841c1e640 1 --2- 172.21.15.84:0/2434147875 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f182006afc0 0x7f182006d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f183c1609b0 tx=0x7f1830008040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:17.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.803+0000 7f182effd640 1 -- 172.21.15.84:0/2434147875 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1838053c60 con 0x7f183c0b0b20 2023-10-16T10:02:17.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.995+0000 7f1844683640 1 -- 172.21.15.84:0/2434147875 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f183c0cf750 con 0x7f183c0b0b20 2023-10-16T10:02:17.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:17.998+0000 7f182effd640 1 -- 172.21.15.84:0/2434147875 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16489 (secure 0 0 0) 0x7f1838017020 con 0x7f183c0b0b20 2023-10-16T10:02:18.000 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:02:18.000 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":64,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:02:15.073908+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:02:11.266402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-16T10:02:12.562100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:02:18.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.001+0000 7f1844683640 1 -- 172.21.15.84:0/2434147875 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f182006afc0 msgr2=0x7f182006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.001+0000 7f1844683640 1 --2- 172.21.15.84:0/2434147875 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f182006afc0 0x7f182006d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f183c1609b0 tx=0x7f1830008040 comp rx=0 tx=0).stop 2023-10-16T10:02:18.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.001+0000 7f1844683640 1 -- 172.21.15.84:0/2434147875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f183c0b0b20 msgr2=0x7f183c15fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.001+0000 7f1844683640 1 --2- 172.21.15.84:0/2434147875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f183c0b0b20 0x7f183c15fea0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f183800c3c0 tx=0x7f183800c880 comp rx=0 tx=0).stop 2023-10-16T10:02:18.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.001+0000 7f1844683640 1 -- 172.21.15.84:0/2434147875 shutdown_connections 2023-10-16T10:02:18.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.001+0000 7f1844683640 1 --2- 172.21.15.84:0/2434147875 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f182006afc0 0x7f182006d480 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.001+0000 7f1844683640 1 --2- 172.21.15.84:0/2434147875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f183c15d3f0 0x7f183c156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.001+0000 7f1844683640 1 --2- 172.21.15.84:0/2434147875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f183c0cd0c0 0x7f183c156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.001+0000 7f1844683640 1 --2- 172.21.15.84:0/2434147875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f183c0b0b20 0x7f183c15fea0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.001+0000 7f1844683640 1 -- 172.21.15.84:0/2434147875 >> 172.21.15.84:0/2434147875 conn(0x7f183c0cb410 msgr2=0x7f183c0ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:18.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.001+0000 7f1844683640 1 -- 172.21.15.84:0/2434147875 shutdown_connections 2023-10-16T10:02:18.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.001+0000 7f1844683640 1 -- 172.21.15.84:0/2434147875 wait complete. 2023-10-16T10:02:18.021 INFO:tasks.ceph.mds.a.smithi084.stderr:2023-10-16T10:02:18.019+0000 7f5267eae640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:02:18.023 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T10:02:18.023 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:02:18.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.236+0000 7fe9c657d640 1 Processor -- start 2023-10-16T10:02:18.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.236+0000 7fe9c657d640 1 -- start start 2023-10-16T10:02:18.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.238+0000 7fe9c657d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe9c0159f00 0x7fe9c00b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:18.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.238+0000 7fe9c657d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c006f7f0 con 0x7fe9c015d880 2023-10-16T10:02:18.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.238+0000 7fe9c657d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c006c5e0 con 0x7fe9c0159b20 2023-10-16T10:02:18.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.238+0000 7fe9c657d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c006c8c0 con 0x7fe9c0159f00 2023-10-16T10:02:18.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.238+0000 7fe9bf7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe9c0159f00 0x7fe9c00b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:18.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.238+0000 7fe9bf7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe9c0159f00 0x7fe9c00b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44666/0 (socket says 172.21.15.84:44666) 2023-10-16T10:02:18.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.238+0000 7fe9bf7fe640 1 -- 172.21.15.84:0/2601833720 learned_addr learned my addr 172.21.15.84:0/2601833720 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:18.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.239+0000 7fe9bf7fe640 1 -- 172.21.15.84:0/2601833720 >> v1:172.21.15.157:6789/0 conn(0x7fe9c0159b20 legacy=0x7fe9c006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:18.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.239+0000 7fe9bf7fe640 1 -- 172.21.15.84:0/2601833720 >> v1:172.21.15.84:6789/0 conn(0x7fe9c015d880 legacy=0x7fe9c006fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.239+0000 7fe9bf7fe640 1 -- 172.21.15.84:0/2601833720 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c006cba0 con 0x7fe9c0159f00 2023-10-16T10:02:18.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.240+0000 7fe9bf7fe640 1 --2- 172.21.15.84:0/2601833720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9c0159f00 0x7fe9c00b0410 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009870 tx=0x7fe9b402dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e85dd3dd311101 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:18.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.240+0000 7fe9beffd640 1 -- 172.21.15.84:0/2601833720 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b402e960 con 0x7fe9c0159f00 2023-10-16T10:02:18.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.240+0000 7fe9beffd640 1 -- 172.21.15.84:0/2601833720 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b402eac0 con 0x7fe9c0159f00 2023-10-16T10:02:18.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.240+0000 7fe9beffd640 1 -- 172.21.15.84:0/2601833720 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b402ee50 con 0x7fe9c0159f00 2023-10-16T10:02:18.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.241+0000 7fe9c657d640 1 -- 172.21.15.84:0/2601833720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9c0159f00 msgr2=0x7fe9c00b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.241+0000 7fe9c657d640 1 --2- 172.21.15.84:0/2601833720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9c0159f00 0x7fe9c00b0410 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009870 tx=0x7fe9b402dbb0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.241+0000 7fe9c657d640 1 -- 172.21.15.84:0/2601833720 shutdown_connections 2023-10-16T10:02:18.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.241+0000 7fe9c657d640 1 --2- 172.21.15.84:0/2601833720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9c0159f00 0x7fe9c00b0410 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.241+0000 7fe9c657d640 1 -- 172.21.15.84:0/2601833720 >> 172.21.15.84:0/2601833720 conn(0x7fe9c00cb410 msgr2=0x7fe9c00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:18.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.241+0000 7fe9c657d640 1 -- 172.21.15.84:0/2601833720 shutdown_connections 2023-10-16T10:02:18.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.241+0000 7fe9c657d640 1 -- 172.21.15.84:0/2601833720 wait complete. 2023-10-16T10:02:18.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.242+0000 7fe9c657d640 1 Processor -- start 2023-10-16T10:02:18.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.243+0000 7fe9c657d640 1 -- start start 2023-10-16T10:02:18.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.243+0000 7fe9c657d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9c0159b20 0x7fe9c01519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:18.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.243+0000 7fe9c657d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9c0159f00 0x7fe9c015a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:18.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.243+0000 7fe9c657d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9c015d880 0x7fe9c00cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:18.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.243+0000 7fe9c657d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00b27b0 con 0x7fe9c015d880 2023-10-16T10:02:18.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.243+0000 7fe9c657d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00b32c0 con 0x7fe9c0159f00 2023-10-16T10:02:18.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.243+0000 7fe9c657d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00b2990 con 0x7fe9c0159b20 2023-10-16T10:02:18.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.244+0000 7fe9c4b1a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9c015d880 0x7fe9c00cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:18.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.244+0000 7fe9c4b1a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9c015d880 0x7fe9c00cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44672/0 (socket says 172.21.15.84:44672) 2023-10-16T10:02:18.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.244+0000 7fe9c4b1a640 1 -- 172.21.15.84:0/1035079487 learned_addr learned my addr 172.21.15.84:0/1035079487 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:18.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.244+0000 7fe9bf7fe640 1 --2- 172.21.15.84:0/1035079487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9c0159f00 0x7fe9c015a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:18.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.244+0000 7fe9bffff640 1 --2- 172.21.15.84:0/1035079487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9c0159b20 0x7fe9c01519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:18.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.244+0000 7fe9c4b1a640 1 -- 172.21.15.84:0/1035079487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9c0159b20 msgr2=0x7fe9c01519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.244+0000 7fe9c4b1a640 1 --2- 172.21.15.84:0/1035079487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9c0159b20 0x7fe9c01519f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.244+0000 7fe9c4b1a640 1 -- 172.21.15.84:0/1035079487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9c0159f00 msgr2=0x7fe9c015a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.244+0000 7fe9c4b1a640 1 --2- 172.21.15.84:0/1035079487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9c0159f00 0x7fe9c015a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.244+0000 7fe9c4b1a640 1 -- 172.21.15.84:0/1035079487 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b40094d0 con 0x7fe9c015d880 2023-10-16T10:02:18.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.244+0000 7fe9bf7fe640 1 --2- 172.21.15.84:0/1035079487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9c0159f00 0x7fe9c015a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:18.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.245+0000 7fe9c4b1a640 1 --2- 172.21.15.84:0/1035079487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9c015d880 0x7fe9c00cc960 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800c7a0 tx=0x7fe9b800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:18.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.245+0000 7fe9bcff9640 1 -- 172.21.15.84:0/1035079487 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8003b70 con 0x7fe9c015d880 2023-10-16T10:02:18.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.245+0000 7fe9bcff9640 1 -- 172.21.15.84:0/1035079487 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b8003cf0 con 0x7fe9c015d880 2023-10-16T10:02:18.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.245+0000 7fe9bcff9640 1 -- 172.21.15.84:0/1035079487 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8003ea0 con 0x7fe9c015d880 2023-10-16T10:02:18.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.245+0000 7fe9c657d640 1 -- 172.21.15.84:0/1035079487 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c00b2c70 con 0x7fe9c015d880 2023-10-16T10:02:18.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.245+0000 7fe9c657d640 1 -- 172.21.15.84:0/1035079487 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c015a820 con 0x7fe9c015d880 2023-10-16T10:02:18.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.246+0000 7fe9c657d640 1 -- 172.21.15.84:0/1035079487 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c004e160 con 0x7fe9c015d880 2023-10-16T10:02:18.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.247+0000 7fe9bcff9640 1 -- 172.21.15.84:0/1035079487 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe9b800aa90 con 0x7fe9c015d880 2023-10-16T10:02:18.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.248+0000 7fe9bcff9640 1 --2- 172.21.15.84:0/1035079487 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fe9ac06af90 0x7fe9ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:18.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.248+0000 7fe9bcff9640 1 -- 172.21.15.84:0/1035079487 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9b8089690 con 0x7fe9c015d880 2023-10-16T10:02:18.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.248+0000 7fe9bffff640 1 --2- 172.21.15.84:0/1035079487 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fe9ac06af90 0x7fe9ac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:18.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.249+0000 7fe9bffff640 1 --2- 172.21.15.84:0/1035079487 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fe9ac06af90 0x7fe9ac06d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000a750 tx=0x7fe9b0008040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:18.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.252+0000 7fe9bcff9640 1 -- 172.21.15.84:0/1035079487 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe9b8054590 con 0x7fe9c015d880 2023-10-16T10:02:18.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.430+0000 7fe9c657d640 1 -- 172.21.15.84:0/1035079487 --> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe9c00b2b00 con 0x7fe9ac06af90 2023-10-16T10:02:18.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.446+0000 7fe9bcff9640 1 -- 172.21.15.84:0/1035079487 <== mgr.5742 v2:172.21.15.84:6832/585186562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286501 (secure 0 0 0) 0x7fe9c00b2b00 con 0x7fe9ac06af90 2023-10-16T10:02:18.500 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:02:18.502 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T10:02:18.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.453+0000 7fe9c657d640 1 -- 172.21.15.84:0/1035079487 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fe9ac06af90 msgr2=0x7fe9ac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.453+0000 7fe9c657d640 1 --2- 172.21.15.84:0/1035079487 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fe9ac06af90 0x7fe9ac06d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000a750 tx=0x7fe9b0008040 comp rx=0 tx=0).stop 2023-10-16T10:02:18.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.453+0000 7fe9c657d640 1 -- 172.21.15.84:0/1035079487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9c015d880 msgr2=0x7fe9c00cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.453+0000 7fe9c657d640 1 --2- 172.21.15.84:0/1035079487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9c015d880 0x7fe9c00cc960 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800c7a0 tx=0x7fe9b800cc60 comp rx=0 tx=0).stop 2023-10-16T10:02:18.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.453+0000 7fe9c657d640 1 -- 172.21.15.84:0/1035079487 shutdown_connections 2023-10-16T10:02:18.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.453+0000 7fe9c657d640 1 --2- 172.21.15.84:0/1035079487 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fe9ac06af90 0x7fe9ac06d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.453+0000 7fe9c657d640 1 --2- 172.21.15.84:0/1035079487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9c015d880 0x7fe9c00cc960 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.453+0000 7fe9c657d640 1 --2- 172.21.15.84:0/1035079487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9c0159f00 0x7fe9c015a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.453+0000 7fe9c657d640 1 --2- 172.21.15.84:0/1035079487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9c0159b20 0x7fe9c01519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.453+0000 7fe9c657d640 1 -- 172.21.15.84:0/1035079487 >> 172.21.15.84:0/1035079487 conn(0x7fe9c00cb410 msgr2=0x7fe9c00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:18.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.453+0000 7fe9c657d640 1 -- 172.21.15.84:0/1035079487 shutdown_connections 2023-10-16T10:02:18.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.453+0000 7fe9c657d640 1 -- 172.21.15.84:0/1035079487 wait complete. 2023-10-16T10:02:18.507 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-16T10:02:17.259209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":144,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":330132,"kb_used_data":7128,"kb_used_omap":12,"kb_used_meta":322867,"kb_avail":754644588,"statfs":{"total":773094113280,"available":772756058112,"internally_reserved":0,"allocated":7299072,"data_stored":4021516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":330616377},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.221005"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853975+0000","last_change":"2023-10-16T10:02:14.088132+0000","last_active":"2023-10-16T10:02:15.853975+0000","last_peered":"2023-10-16T10:02:15.853975+0000","last_clean":"2023-10-16T10:02:15.853975+0000","last_became_active":"2023-10-16T10:02:14.087338+0000","last_became_peered":"2023-10-16T10:02:14.087338+0000","last_unstale":"2023-10-16T10:02:15.853975+0000","last_undegraded":"2023-10-16T10:02:15.853975+0000","last_fullsized":"2023-10-16T10:02:15.853975+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:09.201694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853369+0000","last_change":"2023-10-16T10:02:14.108341+0000","last_active":"2023-10-16T10:02:15.853369+0000","last_peered":"2023-10-16T10:02:15.853369+0000","last_clean":"2023-10-16T10:02:15.853369+0000","last_became_active":"2023-10-16T10:02:14.107965+0000","last_became_peered":"2023-10-16T10:02:14.107965+0000","last_unstale":"2023-10-16T10:02:15.853369+0000","last_undegraded":"2023-10-16T10:02:15.853369+0000","last_fullsized":"2023-10-16T10:02:15.853369+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:26:37.089713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083425+0000","last_change":"2023-10-16T10:02:14.098377+0000","last_active":"2023-10-16T10:02:15.083425+0000","last_peered":"2023-10-16T10:02:15.083425+0000","last_clean":"2023-10-16T10:02:15.083425+0000","last_became_active":"2023-10-16T10:02:14.098126+0000","last_became_peered":"2023-10-16T10:02:14.098126+0000","last_unstale":"2023-10-16T10:02:15.083425+0000","last_undegraded":"2023-10-16T10:02:15.083425+0000","last_fullsized":"2023-10-16T10:02:15.083425+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:48:52.298460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083982+0000","last_change":"2023-10-16T10:02:14.113409+0000","last_active":"2023-10-16T10:02:15.083982+0000","last_peered":"2023-10-16T10:02:15.083982+0000","last_clean":"2023-10-16T10:02:15.083982+0000","last_became_active":"2023-10-16T10:02:14.112394+0000","last_became_peered":"2023-10-16T10:02:14.112394+0000","last_unstale":"2023-10-16T10:02:15.083982+0000","last_undegraded":"2023-10-16T10:02:15.083982+0000","last_fullsized":"2023-10-16T10:02:15.083982+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:36.785811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.852855+0000","last_change":"2023-10-16T10:02:14.108359+0000","last_active":"2023-10-16T10:02:15.852855+0000","last_peered":"2023-10-16T10:02:15.852855+0000","last_clean":"2023-10-16T10:02:15.852855+0000","last_became_active":"2023-10-16T10:02:14.107877+0000","last_became_peered":"2023-10-16T10:02:14.107877+0000","last_unstale":"2023-10-16T10:02:15.852855+0000","last_undegraded":"2023-10-16T10:02:15.852855+0000","last_fullsized":"2023-10-16T10:02:15.852855+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:39:57.827606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.330891+0000","last_change":"2023-10-16T10:02:14.098885+0000","last_active":"2023-10-16T10:02:15.330891+0000","last_peered":"2023-10-16T10:02:15.330891+0000","last_clean":"2023-10-16T10:02:15.330891+0000","last_became_active":"2023-10-16T10:02:14.096342+0000","last_became_peered":"2023-10-16T10:02:14.096342+0000","last_unstale":"2023-10-16T10:02:15.330891+0000","last_undegraded":"2023-10-16T10:02:15.330891+0000","last_fullsized":"2023-10-16T10:02:15.330891+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:32.569778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.854141+0000","last_change":"2023-10-16T10:02:14.096334+0000","last_active":"2023-10-16T10:02:15.854141+0000","last_peered":"2023-10-16T10:02:15.854141+0000","last_clean":"2023-10-16T10:02:15.854141+0000","last_became_active":"2023-10-16T10:02:14.095991+0000","last_became_peered":"2023-10-16T10:02:14.095991+0000","last_unstale":"2023-10-16T10:02:15.854141+0000","last_undegraded":"2023-10-16T10:02:15.854141+0000","last_fullsized":"2023-10-16T10:02:15.854141+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:48:28.768978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.082434+0000","last_change":"2023-10-16T10:02:14.099198+0000","last_active":"2023-10-16T10:02:15.082434+0000","last_peered":"2023-10-16T10:02:15.082434+0000","last_clean":"2023-10-16T10:02:15.082434+0000","last_became_active":"2023-10-16T10:02:14.096136+0000","last_became_peered":"2023-10-16T10:02:14.096136+0000","last_unstale":"2023-10-16T10:02:15.082434+0000","last_undegraded":"2023-10-16T10:02:15.082434+0000","last_fullsized":"2023-10-16T10:02:15.082434+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:53.047567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083761+0000","last_change":"2023-10-16T10:02:14.092815+0000","last_active":"2023-10-16T10:02:15.083761+0000","last_peered":"2023-10-16T10:02:15.083761+0000","last_clean":"2023-10-16T10:02:15.083761+0000","last_became_active":"2023-10-16T10:02:14.092495+0000","last_became_peered":"2023-10-16T10:02:14.092495+0000","last_unstale":"2023-10-16T10:02:15.083761+0000","last_undegraded":"2023-10-16T10:02:15.083761+0000","last_fullsized":"2023-10-16T10:02:15.083761+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:09:06.890750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084403+0000","last_change":"2023-10-16T10:02:14.099018+0000","last_active":"2023-10-16T10:02:15.084403+0000","last_peered":"2023-10-16T10:02:15.084403+0000","last_clean":"2023-10-16T10:02:15.084403+0000","last_became_active":"2023-10-16T10:02:14.098706+0000","last_became_peered":"2023-10-16T10:02:14.098706+0000","last_unstale":"2023-10-16T10:02:15.084403+0000","last_undegraded":"2023-10-16T10:02:15.084403+0000","last_fullsized":"2023-10-16T10:02:15.084403+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:41:24.644785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.082679+0000","last_change":"2023-10-16T10:02:14.103119+0000","last_active":"2023-10-16T10:02:15.082679+0000","last_peered":"2023-10-16T10:02:15.082679+0000","last_clean":"2023-10-16T10:02:15.082679+0000","last_became_active":"2023-10-16T10:02:14.102839+0000","last_became_peered":"2023-10-16T10:02:14.102839+0000","last_unstale":"2023-10-16T10:02:15.082679+0000","last_undegraded":"2023-10-16T10:02:15.082679+0000","last_fullsized":"2023-10-16T10:02:15.082679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:36:22.346800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853099+0000","last_change":"2023-10-16T10:02:14.101568+0000","last_active":"2023-10-16T10:02:15.853099+0000","last_peered":"2023-10-16T10:02:15.853099+0000","last_clean":"2023-10-16T10:02:15.853099+0000","last_became_active":"2023-10-16T10:02:14.101293+0000","last_became_peered":"2023-10-16T10:02:14.101293+0000","last_unstale":"2023-10-16T10:02:15.853099+0000","last_undegraded":"2023-10-16T10:02:15.853099+0000","last_fullsized":"2023-10-16T10:02:15.853099+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:00.096453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.082740+0000","last_change":"2023-10-16T10:02:14.105019+0000","last_active":"2023-10-16T10:02:15.082740+0000","last_peered":"2023-10-16T10:02:15.082740+0000","last_clean":"2023-10-16T10:02:15.082740+0000","last_became_active":"2023-10-16T10:02:14.104553+0000","last_became_peered":"2023-10-16T10:02:14.104553+0000","last_unstale":"2023-10-16T10:02:15.082740+0000","last_undegraded":"2023-10-16T10:02:15.082740+0000","last_fullsized":"2023-10-16T10:02:15.082740+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:03.008219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083140+0000","last_change":"2023-10-16T10:02:14.092002+0000","last_active":"2023-10-16T10:02:15.083140+0000","last_peered":"2023-10-16T10:02:15.083140+0000","last_clean":"2023-10-16T10:02:15.083140+0000","last_became_active":"2023-10-16T10:02:14.089336+0000","last_became_peered":"2023-10-16T10:02:14.089336+0000","last_unstale":"2023-10-16T10:02:15.083140+0000","last_undegraded":"2023-10-16T10:02:15.083140+0000","last_fullsized":"2023-10-16T10:02:15.083140+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:48:41.792471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.330939+0000","last_change":"2023-10-16T10:02:14.095813+0000","last_active":"2023-10-16T10:02:15.330939+0000","last_peered":"2023-10-16T10:02:15.330939+0000","last_clean":"2023-10-16T10:02:15.330939+0000","last_became_active":"2023-10-16T10:02:14.089993+0000","last_became_peered":"2023-10-16T10:02:14.089993+0000","last_unstale":"2023-10-16T10:02:15.330939+0000","last_undegraded":"2023-10-16T10:02:15.330939+0000","last_fullsized":"2023-10-16T10:02:15.330939+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:17.666241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.851765+0000","last_change":"2023-10-16T10:02:14.100364+0000","last_active":"2023-10-16T10:02:15.851765+0000","last_peered":"2023-10-16T10:02:15.851765+0000","last_clean":"2023-10-16T10:02:15.851765+0000","last_became_active":"2023-10-16T10:02:14.099931+0000","last_became_peered":"2023-10-16T10:02:14.099931+0000","last_unstale":"2023-10-16T10:02:15.851765+0000","last_undegraded":"2023-10-16T10:02:15.851765+0000","last_fullsized":"2023-10-16T10:02:15.851765+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:36:22.346800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084001+0000","last_change":"2023-10-16T10:02:14.099188+0000","last_active":"2023-10-16T10:02:15.084001+0000","last_peered":"2023-10-16T10:02:15.084001+0000","last_clean":"2023-10-16T10:02:15.084001+0000","last_became_active":"2023-10-16T10:02:14.098896+0000","last_became_peered":"2023-10-16T10:02:14.098896+0000","last_unstale":"2023-10-16T10:02:15.084001+0000","last_undegraded":"2023-10-16T10:02:15.084001+0000","last_fullsized":"2023-10-16T10:02:15.084001+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:11:56.063588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084056+0000","last_change":"2023-10-16T10:02:14.107323+0000","last_active":"2023-10-16T10:02:15.084056+0000","last_peered":"2023-10-16T10:02:15.084056+0000","last_clean":"2023-10-16T10:02:15.084056+0000","last_became_active":"2023-10-16T10:02:14.106677+0000","last_became_peered":"2023-10-16T10:02:14.106677+0000","last_unstale":"2023-10-16T10:02:15.084056+0000","last_undegraded":"2023-10-16T10:02:15.084056+0000","last_fullsized":"2023-10-16T10:02:15.084056+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:28:24.465488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.852945+0000","last_change":"2023-10-16T10:02:14.098999+0000","last_active":"2023-10-16T10:02:15.852945+0000","last_peered":"2023-10-16T10:02:15.852945+0000","last_clean":"2023-10-16T10:02:15.852945+0000","last_became_active":"2023-10-16T10:02:14.098692+0000","last_became_peered":"2023-10-16T10:02:14.098692+0000","last_unstale":"2023-10-16T10:02:15.852945+0000","last_undegraded":"2023-10-16T10:02:15.852945+0000","last_fullsized":"2023-10-16T10:02:15.852945+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:09:06.890750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083285+0000","last_change":"2023-10-16T10:02:14.097706+0000","last_active":"2023-10-16T10:02:15.083285+0000","last_peered":"2023-10-16T10:02:15.083285+0000","last_clean":"2023-10-16T10:02:15.083285+0000","last_became_active":"2023-10-16T10:02:14.097437+0000","last_became_peered":"2023-10-16T10:02:14.097437+0000","last_unstale":"2023-10-16T10:02:15.083285+0000","last_undegraded":"2023-10-16T10:02:15.083285+0000","last_fullsized":"2023-10-16T10:02:15.083285+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:35.414153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331294+0000","last_change":"2023-10-16T10:02:14.108510+0000","last_active":"2023-10-16T10:02:15.331294+0000","last_peered":"2023-10-16T10:02:15.331294+0000","last_clean":"2023-10-16T10:02:15.331294+0000","last_became_active":"2023-10-16T10:02:14.108291+0000","last_became_peered":"2023-10-16T10:02:14.108291+0000","last_unstale":"2023-10-16T10:02:15.331294+0000","last_undegraded":"2023-10-16T10:02:15.331294+0000","last_fullsized":"2023-10-16T10:02:15.331294+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:29:29.947212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083369+0000","last_change":"2023-10-16T10:02:14.099008+0000","last_active":"2023-10-16T10:02:15.083369+0000","last_peered":"2023-10-16T10:02:15.083369+0000","last_clean":"2023-10-16T10:02:15.083369+0000","last_became_active":"2023-10-16T10:02:14.098623+0000","last_became_peered":"2023-10-16T10:02:14.098623+0000","last_unstale":"2023-10-16T10:02:15.083369+0000","last_undegraded":"2023-10-16T10:02:15.083369+0000","last_fullsized":"2023-10-16T10:02:15.083369+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:20.173125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.852589+0000","last_change":"2023-10-16T10:02:14.089129+0000","last_active":"2023-10-16T10:02:15.852589+0000","last_peered":"2023-10-16T10:02:15.852589+0000","last_clean":"2023-10-16T10:02:15.852589+0000","last_became_active":"2023-10-16T10:02:14.088869+0000","last_became_peered":"2023-10-16T10:02:14.088869+0000","last_unstale":"2023-10-16T10:02:15.852589+0000","last_undegraded":"2023-10-16T10:02:15.852589+0000","last_fullsized":"2023-10-16T10:02:15.852589+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:48:41.792471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.851817+0000","last_change":"2023-10-16T10:02:14.087824+0000","last_active":"2023-10-16T10:02:15.851817+0000","last_peered":"2023-10-16T10:02:15.851817+0000","last_clean":"2023-10-16T10:02:15.851817+0000","last_became_active":"2023-10-16T10:02:14.087620+0000","last_became_peered":"2023-10-16T10:02:14.087620+0000","last_unstale":"2023-10-16T10:02:15.851817+0000","last_undegraded":"2023-10-16T10:02:15.851817+0000","last_fullsized":"2023-10-16T10:02:15.851817+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:21.047154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853347+0000","last_change":"2023-10-16T10:02:14.089519+0000","last_active":"2023-10-16T10:02:15.853347+0000","last_peered":"2023-10-16T10:02:15.853347+0000","last_clean":"2023-10-16T10:02:15.853347+0000","last_became_active":"2023-10-16T10:02:14.089201+0000","last_became_peered":"2023-10-16T10:02:14.089201+0000","last_unstale":"2023-10-16T10:02:15.853347+0000","last_undegraded":"2023-10-16T10:02:15.853347+0000","last_fullsized":"2023-10-16T10:02:15.853347+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:49.923362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084248+0000","last_change":"2023-10-16T10:02:14.103218+0000","last_active":"2023-10-16T10:02:15.084248+0000","last_peered":"2023-10-16T10:02:15.084248+0000","last_clean":"2023-10-16T10:02:15.084248+0000","last_became_active":"2023-10-16T10:02:14.102937+0000","last_became_peered":"2023-10-16T10:02:14.102937+0000","last_unstale":"2023-10-16T10:02:15.084248+0000","last_undegraded":"2023-10-16T10:02:15.084248+0000","last_fullsized":"2023-10-16T10:02:15.084248+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:26.757675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083881+0000","last_change":"2023-10-16T10:02:14.101055+0000","last_active":"2023-10-16T10:02:15.083881+0000","last_peered":"2023-10-16T10:02:15.083881+0000","last_clean":"2023-10-16T10:02:15.083881+0000","last_became_active":"2023-10-16T10:02:14.100575+0000","last_became_peered":"2023-10-16T10:02:14.100575+0000","last_unstale":"2023-10-16T10:02:15.083881+0000","last_undegraded":"2023-10-16T10:02:15.083881+0000","last_fullsized":"2023-10-16T10:02:15.083881+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:37:37.802382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853158+0000","last_change":"2023-10-16T10:02:14.106849+0000","last_active":"2023-10-16T10:02:15.853158+0000","last_peered":"2023-10-16T10:02:15.853158+0000","last_clean":"2023-10-16T10:02:15.853158+0000","last_became_active":"2023-10-16T10:02:14.104132+0000","last_became_peered":"2023-10-16T10:02:14.104132+0000","last_unstale":"2023-10-16T10:02:15.853158+0000","last_undegraded":"2023-10-16T10:02:15.853158+0000","last_fullsized":"2023-10-16T10:02:15.853158+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:47:46.869784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853168+0000","last_change":"2023-10-16T10:02:14.088497+0000","last_active":"2023-10-16T10:02:15.853168+0000","last_peered":"2023-10-16T10:02:15.853168+0000","last_clean":"2023-10-16T10:02:15.853168+0000","last_became_active":"2023-10-16T10:02:14.085556+0000","last_became_peered":"2023-10-16T10:02:14.085556+0000","last_unstale":"2023-10-16T10:02:15.853168+0000","last_undegraded":"2023-10-16T10:02:15.853168+0000","last_fullsized":"2023-10-16T10:02:15.853168+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:36.785811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083254+0000","last_change":"2023-10-16T10:02:14.089445+0000","last_active":"2023-10-16T10:02:15.083254+0000","last_peered":"2023-10-16T10:02:15.083254+0000","last_clean":"2023-10-16T10:02:15.083254+0000","last_became_active":"2023-10-16T10:02:14.089148+0000","last_became_peered":"2023-10-16T10:02:14.089148+0000","last_unstale":"2023-10-16T10:02:15.083254+0000","last_undegraded":"2023-10-16T10:02:15.083254+0000","last_fullsized":"2023-10-16T10:02:15.083254+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:47:04.918722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331082+0000","last_change":"2023-10-16T10:02:14.102795+0000","last_active":"2023-10-16T10:02:15.331082+0000","last_peered":"2023-10-16T10:02:15.331082+0000","last_clean":"2023-10-16T10:02:15.331082+0000","last_became_active":"2023-10-16T10:02:14.102551+0000","last_became_peered":"2023-10-16T10:02:14.102551+0000","last_unstale":"2023-10-16T10:02:15.331082+0000","last_undegraded":"2023-10-16T10:02:15.331082+0000","last_fullsized":"2023-10-16T10:02:15.331082+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:10.287376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.082427+0000","last_change":"2023-10-16T10:02:14.080355+0000","last_active":"2023-10-16T10:02:15.082427+0000","last_peered":"2023-10-16T10:02:15.082427+0000","last_clean":"2023-10-16T10:02:15.082427+0000","last_became_active":"2023-10-16T10:02:14.080047+0000","last_became_peered":"2023-10-16T10:02:14.080047+0000","last_unstale":"2023-10-16T10:02:15.082427+0000","last_undegraded":"2023-10-16T10:02:15.082427+0000","last_fullsized":"2023-10-16T10:02:15.082427+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:49:01.278823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.850837+0000","last_change":"2023-10-16T10:02:14.110370+0000","last_active":"2023-10-16T10:02:15.850837+0000","last_peered":"2023-10-16T10:02:15.850837+0000","last_clean":"2023-10-16T10:02:15.850837+0000","last_became_active":"2023-10-16T10:02:14.110076+0000","last_became_peered":"2023-10-16T10:02:14.110076+0000","last_unstale":"2023-10-16T10:02:15.850837+0000","last_undegraded":"2023-10-16T10:02:15.850837+0000","last_fullsized":"2023-10-16T10:02:15.850837+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:48.244293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'4","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331875+0000","last_change":"2023-10-16T10:02:13.081072+0000","last_active":"2023-10-16T10:02:15.331875+0000","last_peered":"2023-10-16T10:02:15.331875+0000","last_clean":"2023-10-16T10:02:15.331875+0000","last_became_active":"2023-10-16T10:02:13.080304+0000","last_became_peered":"2023-10-16T10:02:13.080304+0000","last_unstale":"2023-10-16T10:02:15.331875+0000","last_undegraded":"2023-10-16T10:02:15.331875+0000","last_fullsized":"2023-10-16T10:02:15.331875+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:15.580804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853673+0000","last_change":"2023-10-16T10:02:14.087498+0000","last_active":"2023-10-16T10:02:15.853673+0000","last_peered":"2023-10-16T10:02:15.853673+0000","last_clean":"2023-10-16T10:02:15.853673+0000","last_became_active":"2023-10-16T10:02:14.087213+0000","last_became_peered":"2023-10-16T10:02:14.087213+0000","last_unstale":"2023-10-16T10:02:15.853673+0000","last_undegraded":"2023-10-16T10:02:15.853673+0000","last_fullsized":"2023-10-16T10:02:15.853673+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:11:13.894974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"63'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084241+0000","last_change":"2023-10-16T10:02:13.084564+0000","last_active":"2023-10-16T10:02:15.084241+0000","last_peered":"2023-10-16T10:02:15.084241+0000","last_clean":"2023-10-16T10:02:15.084241+0000","last_became_active":"2023-10-16T10:02:13.084232+0000","last_became_peered":"2023-10-16T10:02:13.084232+0000","last_unstale":"2023-10-16T10:02:15.084241+0000","last_undegraded":"2023-10-16T10:02:15.084241+0000","last_fullsized":"2023-10-16T10:02:15.084241+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:07:37.011371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853466+0000","last_change":"2023-10-16T10:02:14.111281+0000","last_active":"2023-10-16T10:02:15.853466+0000","last_peered":"2023-10-16T10:02:15.853466+0000","last_clean":"2023-10-16T10:02:15.853466+0000","last_became_active":"2023-10-16T10:02:14.110934+0000","last_became_peered":"2023-10-16T10:02:14.110934+0000","last_unstale":"2023-10-16T10:02:15.853466+0000","last_undegraded":"2023-10-16T10:02:15.853466+0000","last_fullsized":"2023-10-16T10:02:15.853466+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:54.881219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"63'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.850917+0000","last_change":"2023-10-16T10:02:13.073719+0000","last_active":"2023-10-16T10:02:15.850917+0000","last_peered":"2023-10-16T10:02:15.850917+0000","last_clean":"2023-10-16T10:02:15.850917+0000","last_became_active":"2023-10-16T10:02:13.073439+0000","last_became_peered":"2023-10-16T10:02:13.073439+0000","last_unstale":"2023-10-16T10:02:15.850917+0000","last_undegraded":"2023-10-16T10:02:15.850917+0000","last_fullsized":"2023-10-16T10:02:15.850917+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:26:15.458200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.851011+0000","last_change":"2023-10-16T10:02:14.100418+0000","last_active":"2023-10-16T10:02:15.851011+0000","last_peered":"2023-10-16T10:02:15.851011+0000","last_clean":"2023-10-16T10:02:15.851011+0000","last_became_active":"2023-10-16T10:02:14.096948+0000","last_became_peered":"2023-10-16T10:02:14.096948+0000","last_unstale":"2023-10-16T10:02:15.851011+0000","last_undegraded":"2023-10-16T10:02:15.851011+0000","last_fullsized":"2023-10-16T10:02:15.851011+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:02:52.172622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"63'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853797+0000","last_change":"2023-10-16T10:02:13.080182+0000","last_active":"2023-10-16T10:02:15.853797+0000","last_peered":"2023-10-16T10:02:15.853797+0000","last_clean":"2023-10-16T10:02:15.853797+0000","last_became_active":"2023-10-16T10:02:13.079751+0000","last_became_peered":"2023-10-16T10:02:13.079751+0000","last_unstale":"2023-10-16T10:02:15.853797+0000","last_undegraded":"2023-10-16T10:02:15.853797+0000","last_fullsized":"2023-10-16T10:02:15.853797+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:03:18.530382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083710+0000","last_change":"2023-10-16T10:02:14.089957+0000","last_active":"2023-10-16T10:02:15.083710+0000","last_peered":"2023-10-16T10:02:15.083710+0000","last_clean":"2023-10-16T10:02:15.083710+0000","last_became_active":"2023-10-16T10:02:14.089685+0000","last_became_peered":"2023-10-16T10:02:14.089685+0000","last_unstale":"2023-10-16T10:02:15.083710+0000","last_undegraded":"2023-10-16T10:02:15.083710+0000","last_fullsized":"2023-10-16T10:02:15.083710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:07.821140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"63'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331541+0000","last_change":"2023-10-16T10:02:13.078415+0000","last_active":"2023-10-16T10:02:15.331541+0000","last_peered":"2023-10-16T10:02:15.331541+0000","last_clean":"2023-10-16T10:02:15.331541+0000","last_became_active":"2023-10-16T10:02:13.078170+0000","last_became_peered":"2023-10-16T10:02:13.078170+0000","last_unstale":"2023-10-16T10:02:15.331541+0000","last_undegraded":"2023-10-16T10:02:15.331541+0000","last_fullsized":"2023-10-16T10:02:15.331541+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:09:37.828921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083510+0000","last_change":"2023-10-16T10:02:14.100325+0000","last_active":"2023-10-16T10:02:15.083510+0000","last_peered":"2023-10-16T10:02:15.083510+0000","last_clean":"2023-10-16T10:02:15.083510+0000","last_became_active":"2023-10-16T10:02:14.097653+0000","last_became_peered":"2023-10-16T10:02:14.097653+0000","last_unstale":"2023-10-16T10:02:15.083510+0000","last_undegraded":"2023-10-16T10:02:15.083510+0000","last_fullsized":"2023-10-16T10:02:15.083510+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:12:17.504297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"63'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083730+0000","last_change":"2023-10-16T10:02:13.076015+0000","last_active":"2023-10-16T10:02:15.083730+0000","last_peered":"2023-10-16T10:02:15.083730+0000","last_clean":"2023-10-16T10:02:15.083730+0000","last_became_active":"2023-10-16T10:02:13.075315+0000","last_became_peered":"2023-10-16T10:02:13.075315+0000","last_unstale":"2023-10-16T10:02:15.083730+0000","last_undegraded":"2023-10-16T10:02:15.083730+0000","last_fullsized":"2023-10-16T10:02:15.083730+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:15:48.884075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.082560+0000","last_change":"2023-10-16T10:02:14.096515+0000","last_active":"2023-10-16T10:02:15.082560+0000","last_peered":"2023-10-16T10:02:15.082560+0000","last_clean":"2023-10-16T10:02:15.082560+0000","last_became_active":"2023-10-16T10:02:14.095812+0000","last_became_peered":"2023-10-16T10:02:14.095812+0000","last_unstale":"2023-10-16T10:02:15.082560+0000","last_undegraded":"2023-10-16T10:02:15.082560+0000","last_fullsized":"2023-10-16T10:02:15.082560+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:18:15.353674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.852754+0000","last_change":"2023-10-16T10:02:13.069389+0000","last_active":"2023-10-16T10:02:15.852754+0000","last_peered":"2023-10-16T10:02:15.852754+0000","last_clean":"2023-10-16T10:02:15.852754+0000","last_became_active":"2023-10-16T10:02:13.069063+0000","last_became_peered":"2023-10-16T10:02:13.069063+0000","last_unstale":"2023-10-16T10:02:15.852754+0000","last_undegraded":"2023-10-16T10:02:15.852754+0000","last_fullsized":"2023-10-16T10:02:15.852754+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:50.758738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.851602+0000","last_change":"2023-10-16T10:02:14.090663+0000","last_active":"2023-10-16T10:02:15.851602+0000","last_peered":"2023-10-16T10:02:15.851602+0000","last_clean":"2023-10-16T10:02:15.851602+0000","last_became_active":"2023-10-16T10:02:14.090440+0000","last_became_peered":"2023-10-16T10:02:14.090440+0000","last_unstale":"2023-10-16T10:02:15.851602+0000","last_undegraded":"2023-10-16T10:02:15.851602+0000","last_fullsized":"2023-10-16T10:02:15.851602+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:18:15.353674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.082650+0000","last_change":"2023-10-16T10:02:13.068689+0000","last_active":"2023-10-16T10:02:15.082650+0000","last_peered":"2023-10-16T10:02:15.082650+0000","last_clean":"2023-10-16T10:02:15.082650+0000","last_became_active":"2023-10-16T10:02:13.068426+0000","last_became_peered":"2023-10-16T10:02:13.068426+0000","last_unstale":"2023-10-16T10:02:15.082650+0000","last_undegraded":"2023-10-16T10:02:15.082650+0000","last_fullsized":"2023-10-16T10:02:15.082650+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:42:38.072387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084206+0000","last_change":"2023-10-16T10:02:14.099445+0000","last_active":"2023-10-16T10:02:15.084206+0000","last_peered":"2023-10-16T10:02:15.084206+0000","last_clean":"2023-10-16T10:02:15.084206+0000","last_became_active":"2023-10-16T10:02:14.099142+0000","last_became_peered":"2023-10-16T10:02:14.099142+0000","last_unstale":"2023-10-16T10:02:15.084206+0000","last_undegraded":"2023-10-16T10:02:15.084206+0000","last_fullsized":"2023-10-16T10:02:15.084206+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:27.774246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084258+0000","last_change":"2023-10-16T10:02:13.073414+0000","last_active":"2023-10-16T10:02:15.084258+0000","last_peered":"2023-10-16T10:02:15.084258+0000","last_clean":"2023-10-16T10:02:15.084258+0000","last_became_active":"2023-10-16T10:02:13.073130+0000","last_became_peered":"2023-10-16T10:02:13.073130+0000","last_unstale":"2023-10-16T10:02:15.084258+0000","last_undegraded":"2023-10-16T10:02:15.084258+0000","last_fullsized":"2023-10-16T10:02:15.084258+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:47.231504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.854266+0000","last_change":"2023-10-16T10:02:14.098714+0000","last_active":"2023-10-16T10:02:15.854266+0000","last_peered":"2023-10-16T10:02:15.854266+0000","last_clean":"2023-10-16T10:02:15.854266+0000","last_became_active":"2023-10-16T10:02:14.097848+0000","last_became_peered":"2023-10-16T10:02:14.097848+0000","last_unstale":"2023-10-16T10:02:15.854266+0000","last_undegraded":"2023-10-16T10:02:15.854266+0000","last_fullsized":"2023-10-16T10:02:15.854266+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:17:48.268251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084539+0000","last_change":"2023-10-16T10:02:13.075792+0000","last_active":"2023-10-16T10:02:15.084539+0000","last_peered":"2023-10-16T10:02:15.084539+0000","last_clean":"2023-10-16T10:02:15.084539+0000","last_became_active":"2023-10-16T10:02:13.075465+0000","last_became_peered":"2023-10-16T10:02:13.075465+0000","last_unstale":"2023-10-16T10:02:15.084539+0000","last_undegraded":"2023-10-16T10:02:15.084539+0000","last_fullsized":"2023-10-16T10:02:15.084539+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:52:57.656343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331508+0000","last_change":"2023-10-16T10:02:14.096959+0000","last_active":"2023-10-16T10:02:15.331508+0000","last_peered":"2023-10-16T10:02:15.331508+0000","last_clean":"2023-10-16T10:02:15.331508+0000","last_became_active":"2023-10-16T10:02:14.091205+0000","last_became_peered":"2023-10-16T10:02:14.091205+0000","last_unstale":"2023-10-16T10:02:15.331508+0000","last_undegraded":"2023-10-16T10:02:15.331508+0000","last_fullsized":"2023-10-16T10:02:15.331508+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:15:39.423661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"63'3","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.854511+0000","last_change":"2023-10-16T10:02:13.086073+0000","last_active":"2023-10-16T10:02:15.854511+0000","last_peered":"2023-10-16T10:02:15.854511+0000","last_clean":"2023-10-16T10:02:15.854511+0000","last_became_active":"2023-10-16T10:02:13.085249+0000","last_became_peered":"2023-10-16T10:02:13.085249+0000","last_unstale":"2023-10-16T10:02:15.854511+0000","last_undegraded":"2023-10-16T10:02:15.854511+0000","last_fullsized":"2023-10-16T10:02:15.854511+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:09:35.302158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853998+0000","last_change":"2023-10-16T10:02:14.102405+0000","last_active":"2023-10-16T10:02:15.853998+0000","last_peered":"2023-10-16T10:02:15.853998+0000","last_clean":"2023-10-16T10:02:15.853998+0000","last_became_active":"2023-10-16T10:02:14.102026+0000","last_became_peered":"2023-10-16T10:02:14.102026+0000","last_unstale":"2023-10-16T10:02:15.853998+0000","last_undegraded":"2023-10-16T10:02:15.853998+0000","last_fullsized":"2023-10-16T10:02:15.853998+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:53:24.214913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"63'6","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083320+0000","last_change":"2023-10-16T10:02:13.072831+0000","last_active":"2023-10-16T10:02:15.083320+0000","last_peered":"2023-10-16T10:02:15.083320+0000","last_clean":"2023-10-16T10:02:15.083320+0000","last_became_active":"2023-10-16T10:02:13.072394+0000","last_became_peered":"2023-10-16T10:02:13.072394+0000","last_unstale":"2023-10-16T10:02:15.083320+0000","last_undegraded":"2023-10-16T10:02:15.083320+0000","last_fullsized":"2023-10-16T10:02:15.083320+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:57.765713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.852559+0000","last_change":"2023-10-16T10:02:14.105873+0000","last_active":"2023-10-16T10:02:15.852559+0000","last_peered":"2023-10-16T10:02:15.852559+0000","last_clean":"2023-10-16T10:02:15.852559+0000","last_became_active":"2023-10-16T10:02:14.103749+0000","last_became_peered":"2023-10-16T10:02:14.103749+0000","last_unstale":"2023-10-16T10:02:15.852559+0000","last_undegraded":"2023-10-16T10:02:15.852559+0000","last_fullsized":"2023-10-16T10:02:15.852559+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:12:17.504297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"63'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331811+0000","last_change":"2023-10-16T10:02:13.069939+0000","last_active":"2023-10-16T10:02:15.331811+0000","last_peered":"2023-10-16T10:02:15.331811+0000","last_clean":"2023-10-16T10:02:15.331811+0000","last_became_active":"2023-10-16T10:02:13.069646+0000","last_became_peered":"2023-10-16T10:02:13.069646+0000","last_unstale":"2023-10-16T10:02:15.331811+0000","last_undegraded":"2023-10-16T10:02:15.331811+0000","last_fullsized":"2023-10-16T10:02:15.331811+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:12:25.454689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083630+0000","last_change":"2023-10-16T10:02:14.093892+0000","last_active":"2023-10-16T10:02:15.083630+0000","last_peered":"2023-10-16T10:02:15.083630+0000","last_clean":"2023-10-16T10:02:15.083630+0000","last_became_active":"2023-10-16T10:02:14.093513+0000","last_became_peered":"2023-10-16T10:02:14.093513+0000","last_unstale":"2023-10-16T10:02:15.083630+0000","last_undegraded":"2023-10-16T10:02:15.083630+0000","last_fullsized":"2023-10-16T10:02:15.083630+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:48:47.171684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.854081+0000","last_change":"2023-10-16T10:02:13.077582+0000","last_active":"2023-10-16T10:02:15.854081+0000","last_peered":"2023-10-16T10:02:15.854081+0000","last_clean":"2023-10-16T10:02:15.854081+0000","last_became_active":"2023-10-16T10:02:13.077313+0000","last_became_peered":"2023-10-16T10:02:13.077313+0000","last_unstale":"2023-10-16T10:02:15.854081+0000","last_undegraded":"2023-10-16T10:02:15.854081+0000","last_fullsized":"2023-10-16T10:02:15.854081+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:14.171959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.330879+0000","last_change":"2023-10-16T10:02:14.090936+0000","last_active":"2023-10-16T10:02:15.330879+0000","last_peered":"2023-10-16T10:02:15.330879+0000","last_clean":"2023-10-16T10:02:15.330879+0000","last_became_active":"2023-10-16T10:02:14.088143+0000","last_became_peered":"2023-10-16T10:02:14.088143+0000","last_unstale":"2023-10-16T10:02:15.330879+0000","last_undegraded":"2023-10-16T10:02:15.330879+0000","last_fullsized":"2023-10-16T10:02:15.330879+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:03:19.543170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331182+0000","last_change":"2023-10-16T10:02:13.070253+0000","last_active":"2023-10-16T10:02:15.331182+0000","last_peered":"2023-10-16T10:02:15.331182+0000","last_clean":"2023-10-16T10:02:15.331182+0000","last_became_active":"2023-10-16T10:02:13.069873+0000","last_became_peered":"2023-10-16T10:02:13.069873+0000","last_unstale":"2023-10-16T10:02:15.331182+0000","last_undegraded":"2023-10-16T10:02:15.331182+0000","last_fullsized":"2023-10-16T10:02:15.331182+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:02:00.842932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.854233+0000","last_change":"2023-10-16T10:02:14.092483+0000","last_active":"2023-10-16T10:02:15.854233+0000","last_peered":"2023-10-16T10:02:15.854233+0000","last_clean":"2023-10-16T10:02:15.854233+0000","last_became_active":"2023-10-16T10:02:14.092178+0000","last_became_peered":"2023-10-16T10:02:14.092178+0000","last_unstale":"2023-10-16T10:02:15.854233+0000","last_undegraded":"2023-10-16T10:02:15.854233+0000","last_fullsized":"2023-10-16T10:02:15.854233+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:18:43.035144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"63'8","reported_seq":20,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.082745+0000","last_change":"2023-10-16T10:02:13.076002+0000","last_active":"2023-10-16T10:02:15.082745+0000","last_peered":"2023-10-16T10:02:15.082745+0000","last_clean":"2023-10-16T10:02:15.082745+0000","last_became_active":"2023-10-16T10:02:13.074652+0000","last_became_peered":"2023-10-16T10:02:13.074652+0000","last_unstale":"2023-10-16T10:02:15.082745+0000","last_undegraded":"2023-10-16T10:02:15.082745+0000","last_fullsized":"2023-10-16T10:02:15.082745+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:39.135820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084043+0000","last_change":"2023-10-16T10:02:14.107028+0000","last_active":"2023-10-16T10:02:15.084043+0000","last_peered":"2023-10-16T10:02:15.084043+0000","last_clean":"2023-10-16T10:02:15.084043+0000","last_became_active":"2023-10-16T10:02:14.106696+0000","last_became_peered":"2023-10-16T10:02:14.106696+0000","last_unstale":"2023-10-16T10:02:15.084043+0000","last_undegraded":"2023-10-16T10:02:15.084043+0000","last_fullsized":"2023-10-16T10:02:15.084043+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:36:50.232508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"63'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084093+0000","last_change":"2023-10-16T10:02:13.066391+0000","last_active":"2023-10-16T10:02:15.084093+0000","last_peered":"2023-10-16T10:02:15.084093+0000","last_clean":"2023-10-16T10:02:15.084093+0000","last_became_active":"2023-10-16T10:02:13.064926+0000","last_became_peered":"2023-10-16T10:02:13.064926+0000","last_unstale":"2023-10-16T10:02:15.084093+0000","last_undegraded":"2023-10-16T10:02:15.084093+0000","last_fullsized":"2023-10-16T10:02:15.084093+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:17:49.516687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084078+0000","last_change":"2023-10-16T10:02:14.111895+0000","last_active":"2023-10-16T10:02:15.084078+0000","last_peered":"2023-10-16T10:02:15.084078+0000","last_clean":"2023-10-16T10:02:15.084078+0000","last_became_active":"2023-10-16T10:02:14.111617+0000","last_became_peered":"2023-10-16T10:02:14.111617+0000","last_unstale":"2023-10-16T10:02:15.084078+0000","last_undegraded":"2023-10-16T10:02:15.084078+0000","last_fullsized":"2023-10-16T10:02:15.084078+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:49.923362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"63'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084408+0000","last_change":"2023-10-16T10:02:13.076569+0000","last_active":"2023-10-16T10:02:15.084408+0000","last_peered":"2023-10-16T10:02:15.084408+0000","last_clean":"2023-10-16T10:02:15.084408+0000","last_became_active":"2023-10-16T10:02:13.076291+0000","last_became_peered":"2023-10-16T10:02:13.076291+0000","last_unstale":"2023-10-16T10:02:15.084408+0000","last_undegraded":"2023-10-16T10:02:15.084408+0000","last_fullsized":"2023-10-16T10:02:15.084408+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:13.792570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.852885+0000","last_change":"2023-10-16T10:02:14.105827+0000","last_active":"2023-10-16T10:02:15.852885+0000","last_peered":"2023-10-16T10:02:15.852885+0000","last_clean":"2023-10-16T10:02:15.852885+0000","last_became_active":"2023-10-16T10:02:14.105408+0000","last_became_peered":"2023-10-16T10:02:14.105408+0000","last_unstale":"2023-10-16T10:02:15.852885+0000","last_undegraded":"2023-10-16T10:02:15.852885+0000","last_fullsized":"2023-10-16T10:02:15.852885+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:48:47.171684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"63'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853850+0000","last_change":"2023-10-16T10:02:13.069640+0000","last_active":"2023-10-16T10:02:15.853850+0000","last_peered":"2023-10-16T10:02:15.853850+0000","last_clean":"2023-10-16T10:02:15.853850+0000","last_became_active":"2023-10-16T10:02:13.069302+0000","last_became_peered":"2023-10-16T10:02:13.069302+0000","last_unstale":"2023-10-16T10:02:15.853850+0000","last_undegraded":"2023-10-16T10:02:15.853850+0000","last_fullsized":"2023-10-16T10:02:15.853850+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:00:23.785232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.852426+0000","last_change":"2023-10-16T10:02:14.105594+0000","last_active":"2023-10-16T10:02:15.852426+0000","last_peered":"2023-10-16T10:02:15.852426+0000","last_clean":"2023-10-16T10:02:15.852426+0000","last_became_active":"2023-10-16T10:02:14.103393+0000","last_became_peered":"2023-10-16T10:02:14.103393+0000","last_unstale":"2023-10-16T10:02:15.852426+0000","last_undegraded":"2023-10-16T10:02:15.852426+0000","last_fullsized":"2023-10-16T10:02:15.852426+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:20.173125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331644+0000","last_change":"2023-10-16T10:02:13.082207+0000","last_active":"2023-10-16T10:02:15.331644+0000","last_peered":"2023-10-16T10:02:15.331644+0000","last_clean":"2023-10-16T10:02:15.331644+0000","last_became_active":"2023-10-16T10:02:13.081937+0000","last_became_peered":"2023-10-16T10:02:13.081937+0000","last_unstale":"2023-10-16T10:02:15.331644+0000","last_undegraded":"2023-10-16T10:02:15.331644+0000","last_fullsized":"2023-10-16T10:02:15.331644+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:12:25.373505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331132+0000","last_change":"2023-10-16T10:02:14.097379+0000","last_active":"2023-10-16T10:02:15.331132+0000","last_peered":"2023-10-16T10:02:15.331132+0000","last_clean":"2023-10-16T10:02:15.331132+0000","last_became_active":"2023-10-16T10:02:14.091543+0000","last_became_peered":"2023-10-16T10:02:14.091543+0000","last_unstale":"2023-10-16T10:02:15.331132+0000","last_undegraded":"2023-10-16T10:02:15.331132+0000","last_fullsized":"2023-10-16T10:02:15.331132+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:56.401413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"63'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331860+0000","last_change":"2023-10-16T10:02:13.074721+0000","last_active":"2023-10-16T10:02:15.331860+0000","last_peered":"2023-10-16T10:02:15.331860+0000","last_clean":"2023-10-16T10:02:15.331860+0000","last_became_active":"2023-10-16T10:02:13.072975+0000","last_became_peered":"2023-10-16T10:02:13.072975+0000","last_unstale":"2023-10-16T10:02:15.331860+0000","last_undegraded":"2023-10-16T10:02:15.331860+0000","last_fullsized":"2023-10-16T10:02:15.331860+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:36.966524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.850378+0000","last_change":"2023-10-16T10:02:14.101271+0000","last_active":"2023-10-16T10:02:15.850378+0000","last_peered":"2023-10-16T10:02:15.850378+0000","last_clean":"2023-10-16T10:02:15.850378+0000","last_became_active":"2023-10-16T10:02:14.101024+0000","last_became_peered":"2023-10-16T10:02:14.101024+0000","last_unstale":"2023-10-16T10:02:15.850378+0000","last_undegraded":"2023-10-16T10:02:15.850378+0000","last_fullsized":"2023-10-16T10:02:15.850378+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:45:26.800557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.852499+0000","last_change":"2023-10-16T10:02:13.082558+0000","last_active":"2023-10-16T10:02:15.852499+0000","last_peered":"2023-10-16T10:02:15.852499+0000","last_clean":"2023-10-16T10:02:15.852499+0000","last_became_active":"2023-10-16T10:02:13.082244+0000","last_became_peered":"2023-10-16T10:02:13.082244+0000","last_unstale":"2023-10-16T10:02:15.852499+0000","last_undegraded":"2023-10-16T10:02:15.852499+0000","last_fullsized":"2023-10-16T10:02:15.852499+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:19:10.604917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.082454+0000","last_change":"2023-10-16T10:02:14.086143+0000","last_active":"2023-10-16T10:02:15.082454+0000","last_peered":"2023-10-16T10:02:15.082454+0000","last_clean":"2023-10-16T10:02:15.082454+0000","last_became_active":"2023-10-16T10:02:14.085853+0000","last_became_peered":"2023-10-16T10:02:14.085853+0000","last_unstale":"2023-10-16T10:02:15.082454+0000","last_undegraded":"2023-10-16T10:02:15.082454+0000","last_fullsized":"2023-10-16T10:02:15.082454+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:56.378112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083186+0000","last_change":"2023-10-16T10:02:13.079440+0000","last_active":"2023-10-16T10:02:15.083186+0000","last_peered":"2023-10-16T10:02:15.083186+0000","last_clean":"2023-10-16T10:02:15.083186+0000","last_became_active":"2023-10-16T10:02:13.078808+0000","last_became_peered":"2023-10-16T10:02:13.078808+0000","last_unstale":"2023-10-16T10:02:15.083186+0000","last_undegraded":"2023-10-16T10:02:15.083186+0000","last_fullsized":"2023-10-16T10:02:15.083186+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:53:51.639925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083099+0000","last_change":"2023-10-16T10:02:14.098831+0000","last_active":"2023-10-16T10:02:15.083099+0000","last_peered":"2023-10-16T10:02:15.083099+0000","last_clean":"2023-10-16T10:02:15.083099+0000","last_became_active":"2023-10-16T10:02:14.094053+0000","last_became_peered":"2023-10-16T10:02:14.094053+0000","last_unstale":"2023-10-16T10:02:15.083099+0000","last_undegraded":"2023-10-16T10:02:15.083099+0000","last_fullsized":"2023-10-16T10:02:15.083099+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:21.047154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.851076+0000","last_change":"2023-10-16T10:02:13.079573+0000","last_active":"2023-10-16T10:02:15.851076+0000","last_peered":"2023-10-16T10:02:15.851076+0000","last_clean":"2023-10-16T10:02:15.851076+0000","last_became_active":"2023-10-16T10:02:13.079008+0000","last_became_peered":"2023-10-16T10:02:13.079008+0000","last_unstale":"2023-10-16T10:02:15.851076+0000","last_undegraded":"2023-10-16T10:02:15.851076+0000","last_fullsized":"2023-10-16T10:02:15.851076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:36.966524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853982+0000","last_change":"2023-10-16T10:02:14.099831+0000","last_active":"2023-10-16T10:02:15.853982+0000","last_peered":"2023-10-16T10:02:15.853982+0000","last_clean":"2023-10-16T10:02:15.853982+0000","last_became_active":"2023-10-16T10:02:14.098418+0000","last_became_peered":"2023-10-16T10:02:14.098418+0000","last_unstale":"2023-10-16T10:02:15.853982+0000","last_undegraded":"2023-10-16T10:02:15.853982+0000","last_fullsized":"2023-10-16T10:02:15.853982+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:41:24.644785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331646+0000","last_change":"2023-10-16T10:02:13.082763+0000","last_active":"2023-10-16T10:02:15.331646+0000","last_peered":"2023-10-16T10:02:15.331646+0000","last_clean":"2023-10-16T10:02:15.331646+0000","last_became_active":"2023-10-16T10:02:13.082494+0000","last_became_peered":"2023-10-16T10:02:13.082494+0000","last_unstale":"2023-10-16T10:02:15.331646+0000","last_undegraded":"2023-10-16T10:02:15.331646+0000","last_fullsized":"2023-10-16T10:02:15.331646+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:02:32.975638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084106+0000","last_change":"2023-10-16T10:02:14.095123+0000","last_active":"2023-10-16T10:02:15.084106+0000","last_peered":"2023-10-16T10:02:15.084106+0000","last_clean":"2023-10-16T10:02:15.084106+0000","last_became_active":"2023-10-16T10:02:14.094853+0000","last_became_peered":"2023-10-16T10:02:14.094853+0000","last_unstale":"2023-10-16T10:02:15.084106+0000","last_undegraded":"2023-10-16T10:02:15.084106+0000","last_fullsized":"2023-10-16T10:02:15.084106+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:11:13.894974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.854619+0000","last_change":"2023-10-16T10:02:13.086282+0000","last_active":"2023-10-16T10:02:15.854619+0000","last_peered":"2023-10-16T10:02:15.854619+0000","last_clean":"2023-10-16T10:02:15.854619+0000","last_became_active":"2023-10-16T10:02:13.085939+0000","last_became_peered":"2023-10-16T10:02:13.085939+0000","last_unstale":"2023-10-16T10:02:15.854619+0000","last_undegraded":"2023-10-16T10:02:15.854619+0000","last_fullsized":"2023-10-16T10:02:15.854619+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:09.274588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.854377+0000","last_change":"2023-10-16T10:02:14.097225+0000","last_active":"2023-10-16T10:02:15.854377+0000","last_peered":"2023-10-16T10:02:15.854377+0000","last_clean":"2023-10-16T10:02:15.854377+0000","last_became_active":"2023-10-16T10:02:14.095692+0000","last_became_peered":"2023-10-16T10:02:14.095692+0000","last_unstale":"2023-10-16T10:02:15.854377+0000","last_undegraded":"2023-10-16T10:02:15.854377+0000","last_fullsized":"2023-10-16T10:02:15.854377+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:32:10.613842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.850122+0000","last_change":"2023-10-16T10:02:13.073665+0000","last_active":"2023-10-16T10:02:15.850122+0000","last_peered":"2023-10-16T10:02:15.850122+0000","last_clean":"2023-10-16T10:02:15.850122+0000","last_became_active":"2023-10-16T10:02:13.073426+0000","last_became_peered":"2023-10-16T10:02:13.073426+0000","last_unstale":"2023-10-16T10:02:15.850122+0000","last_undegraded":"2023-10-16T10:02:15.850122+0000","last_fullsized":"2023-10-16T10:02:15.850122+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:47:57.299735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331033+0000","last_change":"2023-10-16T10:02:14.099071+0000","last_active":"2023-10-16T10:02:15.331033+0000","last_peered":"2023-10-16T10:02:15.331033+0000","last_clean":"2023-10-16T10:02:15.331033+0000","last_became_active":"2023-10-16T10:02:14.098635+0000","last_became_peered":"2023-10-16T10:02:14.098635+0000","last_unstale":"2023-10-16T10:02:15.331033+0000","last_undegraded":"2023-10-16T10:02:15.331033+0000","last_fullsized":"2023-10-16T10:02:15.331033+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:09:36.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.855048+0000","last_change":"2023-10-16T10:02:13.084852+0000","last_active":"2023-10-16T10:02:15.855048+0000","last_peered":"2023-10-16T10:02:15.855048+0000","last_clean":"2023-10-16T10:02:15.855048+0000","last_became_active":"2023-10-16T10:02:13.084521+0000","last_became_peered":"2023-10-16T10:02:13.084521+0000","last_unstale":"2023-10-16T10:02:15.855048+0000","last_undegraded":"2023-10-16T10:02:15.855048+0000","last_fullsized":"2023-10-16T10:02:15.855048+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:02:32.975638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331469+0000","last_change":"2023-10-16T10:02:14.095368+0000","last_active":"2023-10-16T10:02:15.331469+0000","last_peered":"2023-10-16T10:02:15.331469+0000","last_clean":"2023-10-16T10:02:15.331469+0000","last_became_active":"2023-10-16T10:02:14.095095+0000","last_became_peered":"2023-10-16T10:02:14.095095+0000","last_unstale":"2023-10-16T10:02:15.331469+0000","last_undegraded":"2023-10-16T10:02:15.331469+0000","last_fullsized":"2023-10-16T10:02:15.331469+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:32:27.250633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083939+0000","last_change":"2023-10-16T10:02:13.071942+0000","last_active":"2023-10-16T10:02:15.083939+0000","last_peered":"2023-10-16T10:02:15.083939+0000","last_clean":"2023-10-16T10:02:15.083939+0000","last_became_active":"2023-10-16T10:02:13.071360+0000","last_became_peered":"2023-10-16T10:02:13.071360+0000","last_unstale":"2023-10-16T10:02:15.083939+0000","last_undegraded":"2023-10-16T10:02:15.083939+0000","last_fullsized":"2023-10-16T10:02:15.083939+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:16.310641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.854243+0000","last_change":"2023-10-16T10:02:14.105043+0000","last_active":"2023-10-16T10:02:15.854243+0000","last_peered":"2023-10-16T10:02:15.854243+0000","last_clean":"2023-10-16T10:02:15.854243+0000","last_became_active":"2023-10-16T10:02:14.104762+0000","last_became_peered":"2023-10-16T10:02:14.104762+0000","last_unstale":"2023-10-16T10:02:15.854243+0000","last_undegraded":"2023-10-16T10:02:15.854243+0000","last_fullsized":"2023-10-16T10:02:15.854243+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:14.957159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.850105+0000","last_change":"2023-10-16T10:02:13.080847+0000","last_active":"2023-10-16T10:02:15.850105+0000","last_peered":"2023-10-16T10:02:15.850105+0000","last_clean":"2023-10-16T10:02:15.850105+0000","last_became_active":"2023-10-16T10:02:13.080561+0000","last_became_peered":"2023-10-16T10:02:13.080561+0000","last_unstale":"2023-10-16T10:02:15.850105+0000","last_undegraded":"2023-10-16T10:02:15.850105+0000","last_fullsized":"2023-10-16T10:02:15.850105+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:31:13.381358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083753+0000","last_change":"2023-10-16T10:02:14.093397+0000","last_active":"2023-10-16T10:02:15.083753+0000","last_peered":"2023-10-16T10:02:15.083753+0000","last_clean":"2023-10-16T10:02:15.083753+0000","last_became_active":"2023-10-16T10:02:14.093140+0000","last_became_peered":"2023-10-16T10:02:14.093140+0000","last_unstale":"2023-10-16T10:02:15.083753+0000","last_undegraded":"2023-10-16T10:02:15.083753+0000","last_fullsized":"2023-10-16T10:02:15.083753+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:41.123730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.082828+0000","last_change":"2023-10-16T10:02:13.067760+0000","last_active":"2023-10-16T10:02:15.082828+0000","last_peered":"2023-10-16T10:02:15.082828+0000","last_clean":"2023-10-16T10:02:15.082828+0000","last_became_active":"2023-10-16T10:02:13.067505+0000","last_became_peered":"2023-10-16T10:02:13.067505+0000","last_unstale":"2023-10-16T10:02:15.082828+0000","last_undegraded":"2023-10-16T10:02:15.082828+0000","last_fullsized":"2023-10-16T10:02:15.082828+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:06:46.286774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084737+0000","last_change":"2023-10-16T10:02:14.093589+0000","last_active":"2023-10-16T10:02:15.084737+0000","last_peered":"2023-10-16T10:02:15.084737+0000","last_clean":"2023-10-16T10:02:15.084737+0000","last_became_active":"2023-10-16T10:02:14.093264+0000","last_became_peered":"2023-10-16T10:02:14.093264+0000","last_unstale":"2023-10-16T10:02:15.084737+0000","last_undegraded":"2023-10-16T10:02:15.084737+0000","last_fullsized":"2023-10-16T10:02:15.084737+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:47.585878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"63'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.855042+0000","last_change":"2023-10-16T10:02:13.068582+0000","last_active":"2023-10-16T10:02:15.855042+0000","last_peered":"2023-10-16T10:02:15.855042+0000","last_clean":"2023-10-16T10:02:15.855042+0000","last_became_active":"2023-10-16T10:02:13.068269+0000","last_became_peered":"2023-10-16T10:02:13.068269+0000","last_unstale":"2023-10-16T10:02:15.855042+0000","last_undegraded":"2023-10-16T10:02:15.855042+0000","last_fullsized":"2023-10-16T10:02:15.855042+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:28.986842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"51'139","reported_seq":288,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-16T10:02:14.070889+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T10:02:14.070889+0000","last_peered":"2023-10-16T10:02:14.070889+0000","last_clean":"2023-10-16T10:02:14.070889+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T10:02:14.070889+0000","last_undegraded":"2023-10-16T10:02:14.070889+0000","last_fullsized":"2023-10-16T10:02:14.070889+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575010,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36656,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335184,"statfs":{"total":96636764160,"available":96599228416,"internally_reserved":0,"allocated":1347584,"data_stored":941803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.1890000000000001,"15min":1.153},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.67200000000000004},"max":{"1min":1.853,"5min":1.925,"15min":1.998},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.2190000000000001,"15min":1.238},"min":{"1min":0.97699999999999998,"5min":0.84299999999999997,"15min":0.68100000000000005},"max":{"1min":1.75,"5min":2.016,"15min":3.694},"last":0.90000000000000002}]},{"osd":1,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2609999999999999,"15min":1.2250000000000001},"min":{"1min":0.94899999999999995,"5min":0.876,"15min":0.747},"max":{"1min":1.534,"5min":1.766,"15min":1.766},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2569999999999999,"15min":1.212},"min":{"1min":0.91400000000000003,"5min":0.79000000000000004,"15min":0.69399999999999995},"max":{"1min":1.595,"5min":1.704,"15min":1.7410000000000001},"last":1.512}]},{"osd":2,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2989999999999999,"15min":1.2689999999999999},"min":{"1min":0.83599999999999997,"5min":0.81100000000000005,"15min":0.63},"max":{"1min":1.607,"5min":1.8740000000000001,"15min":1.885},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.2989999999999999,"15min":1.325},"min":{"1min":1.0620000000000001,"5min":0.89500000000000002,"15min":0.70799999999999996},"max":{"1min":1.6799999999999999,"5min":1.8620000000000001,"15min":3.7759999999999998},"last":1.407}]},{"osd":3,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3580000000000001,"15min":1.3480000000000001},"min":{"1min":1.0289999999999999,"5min":0.85099999999999998,"15min":0.82799999999999996},"max":{"1min":1.726,"5min":1.996,"15min":1.996},"last":1.577},{"interface":"front","average":{"1min":1.397,"5min":1.409,"15min":1.3899999999999999},"min":{"1min":1.069,"5min":0.75900000000000001,"15min":0.66900000000000004},"max":{"1min":1.837,"5min":2.004,"15min":2.141},"last":1.361}]},{"osd":4,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.369,"15min":1.3959999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0310000000000001},"last":1.45},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3160000000000001,"15min":1.339},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.84199999999999997},"max":{"1min":1.613,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.2869999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.5309999999999999,"15min":1.544},"min":{"1min":0.93200000000000005,"5min":0.91100000000000003,"15min":0.872},"max":{"1min":2.121,"5min":2.165,"15min":2.4510000000000001},"last":1.609},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4379999999999999,"15min":1.472},"min":{"1min":1.2290000000000001,"5min":0.99099999999999999,"15min":0.94099999999999995},"max":{"1min":1.7350000000000001,"5min":1.9990000000000001,"15min":2.3210000000000002},"last":1.5389999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3899999999999999,"15min":1.454},"min":{"1min":0.94499999999999995,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.6859999999999999,"5min":1.946,"15min":2.6259999999999999},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.621,"5min":1.6299999999999999,"15min":1.6439999999999999},"min":{"1min":1.032,"5min":1.032,"15min":0.98799999999999999},"max":{"1min":2.3799999999999999,"5min":2.3799999999999999,"15min":2.5670000000000002},"last":1.73}]}]},{"osd":6,"up_from":13,"seq":55834575010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36092,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335748,"statfs":{"total":96636764160,"available":96599805952,"internally_reserved":0,"allocated":770048,"data_stored":358317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.1160000000000001,"15min":1.1539999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.667,"5min":1.6910000000000001,"15min":2.04},"last":1.389},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.21,"15min":1.242},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.624},"max":{"1min":1.847,"5min":1.847,"15min":2.1000000000000001},"last":1.55}]},{"osd":1,"last update":"Mon Oct 16 10:02:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.161,"15min":1.1890000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.423,"5min":1.6919999999999999,"15min":1.9259999999999999},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.288,"15min":1.3089999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.734,"5min":1.734,"15min":1.873},"last":1.45}]},{"osd":2,"last update":"Mon Oct 16 10:02:12 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.55,"5min":2.036,"15min":2.036},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2849999999999999,"15min":1.339},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":2.1720000000000002},"last":1.514}]},{"osd":3,"last update":"Mon Oct 16 10:02:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2190000000000001,"15min":1.258},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.8239999999999998,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.335},{"interface":"front","average":{"1min":1.179,"5min":1.23,"15min":1.248},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.619,"5min":1.825,"15min":2.0640000000000001},"last":1.2969999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:02:12 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.23,"15min":1.3440000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.4540000000000002},"last":1.236},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4430000000000001,"15min":1.49},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.6880000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.1739999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:02:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.337,"15min":1.361},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.3090000000000002,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.454,"5min":1.492,"15min":1.5529999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.5289999999999999},"last":1.5800000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:02:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.4910000000000001,"15min":1.532},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":3.5739999999999998,"5min":3.5739999999999998,"15min":3.5739999999999998},"last":1.6399999999999999},{"interface":"front","average":{"1min":1.327,"5min":1.4750000000000001,"15min":1.5389999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.9790000000000001,"5min":2.4249999999999998,"15min":3.5419999999999998},"last":1.488}]}]},{"osd":5,"up_from":13,"seq":55834575010,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36084,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":753664,"data_stored":351415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.2390000000000001,"15min":1.236},"min":{"1min":0.81499999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.4770000000000001,"5min":1.675,"15min":2.1019999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":1.329,"5min":1.3320000000000001,"15min":1.3169999999999999},"min":{"1min":1.0289999999999999,"5min":0.89800000000000002,"15min":0.77200000000000002},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.915},"last":0.84199999999999997}]},{"osd":1,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.1919999999999999,"15min":1.149},"min":{"1min":0.83999999999999997,"5min":0.72799999999999998,"15min":0.68799999999999994},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":2.4569999999999999},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.23,"5min":1.1799999999999999,"15min":1.1879999999999999},"min":{"1min":0.95299999999999996,"5min":0.82499999999999996,"15min":0.78600000000000003},"max":{"1min":1.5700000000000001,"5min":1.6970000000000001,"15min":2.2410000000000001},"last":0.91400000000000003}]},{"osd":2,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.163,"15min":1.1619999999999999},"min":{"1min":0.871,"5min":0.80900000000000005,"15min":0.71299999999999997},"max":{"1min":1.639,"5min":1.639,"15min":1.77},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.264,"15min":1.2669999999999999},"min":{"1min":0.94699999999999995,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.6990000000000001,"5min":1.714,"15min":1.7270000000000001},"last":1.4950000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.319,"15min":1.3049999999999999},"min":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.73799999999999999},"max":{"1min":1.917,"5min":1.97,"15min":1.97},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.403,"5min":1.389,"15min":1.3939999999999999},"min":{"1min":1.095,"5min":0.874,"15min":0.874},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":2.1160000000000001},"last":0.97599999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2729999999999999,"15min":1.3560000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.52100000000000002},"max":{"1min":1.6359999999999999,"5min":1.7989999999999999,"15min":2.355},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.478,"5min":1.5189999999999999,"15min":1.5569999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.63600000000000001},"max":{"1min":2.0819999999999999,"5min":2.0920000000000001,"15min":2.4079999999999999},"last":1.4419999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3640000000000001,"15min":1.4179999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.55500000000000005},"max":{"1min":1.847,"5min":1.847,"15min":2.3580000000000001},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.403,"15min":1.478},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.58399999999999996},"max":{"1min":1.7150000000000001,"5min":1.889,"15min":2.4159999999999999},"last":1.359}]},{"osd":7,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.506,"15min":1.5820000000000001},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.78700000000000003},"max":{"1min":1.9970000000000001,"5min":2.1859999999999999,"15min":2.4809999999999999},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.587,"15min":1.6479999999999999},"min":{"1min":1.0640000000000001,"5min":0.99199999999999999,"15min":0.75600000000000001},"max":{"1min":1.917,"5min":1.9179999999999999,"15min":2.931},"last":1.5720000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575011,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62468,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309372,"statfs":{"total":96636764160,"available":96572796928,"internally_reserved":0,"allocated":774144,"data_stored":359257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.171,"15min":1.1930000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.76,"5min":1.76,"15min":4.2640000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.246,"15min":1.228},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.64600000000000002},"max":{"1min":4.915,"5min":4.915,"15min":4.915},"last":11.974}]},{"osd":1,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.2849999999999999,"15min":1.248},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.66000000000000003},"max":{"1min":4.758,"5min":4.758,"15min":4.758},"last":9.9939999999999998},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.232,"15min":1.2310000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.624},"max":{"1min":1.6060000000000001,"5min":1.712,"15min":2.0259999999999998},"last":5.9420000000000002}]},{"osd":2,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.238,"15min":1.25},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.528,"5min":1.756,"15min":2.3130000000000002},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.387,"15min":1.3779999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.70799999999999996},"max":{"1min":5.0549999999999997,"5min":5.0549999999999997,"15min":5.0549999999999997},"last":3.673}]},{"osd":3,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.323},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.6800000000000002,"5min":2.6800000000000002,"15min":3.262},"last":6.8079999999999998},{"interface":"front","average":{"1min":1.292,"5min":1.387,"15min":1.429},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.7729999999999999,"5min":1.923,"15min":3.3050000000000002},"last":1.4450000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.4370000000000001,"15min":1.415},"min":{"1min":0.871,"5min":0.752,"15min":0.752},"max":{"1min":5.8940000000000001,"5min":5.8940000000000001,"15min":5.8940000000000001},"last":8.702},{"interface":"front","average":{"1min":1.474,"5min":1.4019999999999999,"15min":1.4179999999999999},"min":{"1min":1,"5min":0.91600000000000004,"15min":0.69099999999999995},"max":{"1min":3.0430000000000001,"5min":3.0430000000000001,"15min":3.669},"last":1.1000000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3959999999999999,"15min":1.4390000000000001},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.56599999999999995},"max":{"1min":2.5,"5min":2.5,"15min":3.8090000000000002},"last":8.0579999999999998},{"interface":"front","average":{"1min":1.6899999999999999,"5min":1.569,"15min":1.5600000000000001},"min":{"1min":1.0029999999999999,"5min":0.91500000000000004,"15min":0.70299999999999996},"max":{"1min":6.4089999999999998,"5min":6.4089999999999998,"15min":6.4089999999999998},"last":4.625}]},{"osd":7,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.506,"15min":1.5289999999999999},"min":{"1min":1.0960000000000001,"5min":0.98499999999999999,"15min":0.60999999999999999},"max":{"1min":2.9399999999999999,"5min":2.9399999999999999,"15min":3.5150000000000001},"last":2.4060000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5860000000000001,"15min":1.625},"min":{"1min":1.2170000000000001,"5min":0.98399999999999999,"15min":0.89600000000000002},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":3.9849999999999999},"last":1.48}]}]},{"osd":3,"up_from":13,"seq":55834575010,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36092,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335748,"statfs":{"total":96636764160,"available":96599805952,"internally_reserved":0,"allocated":761856,"data_stored":351983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.88500000000000001,"15min":0.86499999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.40699999999999997},"max":{"1min":1.208,"5min":1.3360000000000001,"15min":1.462},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.006,"5min":1.0629999999999999,"15min":1.008},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.29999999999999999},"max":{"1min":1.385,"5min":1.708,"15min":1.708},"last":1.002}]},{"osd":1,"last update":"Mon Oct 16 10:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.97799999999999998,"15min":0.94599999999999995},"min":{"1min":0.73899999999999999,"5min":0.66400000000000003,"15min":0.46999999999999997},"max":{"1min":1.2909999999999999,"5min":1.458,"15min":1.7150000000000001},"last":1.105},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.105,"15min":1.0429999999999999},"min":{"1min":0.83199999999999996,"5min":0.76700000000000002,"15min":0.39600000000000002},"max":{"1min":1.603,"5min":1.643,"15min":1.643},"last":1.139}]},{"osd":2,"last update":"Mon Oct 16 10:01:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1120000000000001,"15min":1.077},"min":{"1min":0.89600000000000002,"5min":0.81999999999999995,"15min":0.439},"max":{"1min":1.4039999999999999,"5min":1.4590000000000001,"15min":1.8640000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2150000000000001,"15min":1.1579999999999999},"min":{"1min":0.93700000000000006,"5min":0.79000000000000004,"15min":0.51900000000000002},"max":{"1min":1.7170000000000001,"5min":1.776,"15min":2.028},"last":1.546}]},{"osd":4,"last update":"Mon Oct 16 10:01:19 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.3029999999999999,"15min":1.2949999999999999},"min":{"1min":1.0029999999999999,"5min":0.96499999999999997,"15min":0.755},"max":{"1min":1.619,"5min":1.649,"15min":2.1619999999999999},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.337,"15min":1.3320000000000001},"min":{"1min":1.097,"5min":1.042,"15min":0.73599999999999999},"max":{"1min":1.5720000000000001,"5min":1.766,"15min":2.532},"last":1.804}]},{"osd":5,"last update":"Mon Oct 16 10:01:19 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.355,"15min":1.339},"min":{"1min":1.042,"5min":1.042,"15min":0.76900000000000002},"max":{"1min":1.6120000000000001,"5min":1.7989999999999999,"15min":2.0019999999999998},"last":1.333},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3600000000000001,"15min":1.3700000000000001},"min":{"1min":1.03,"5min":1.016,"15min":0.76700000000000002},"max":{"1min":1.573,"5min":1.855,"15min":2.0419999999999998},"last":1.5740000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:01:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.468,"15min":1.456},"min":{"1min":1.103,"5min":0.98699999999999999,"15min":0.81599999999999995},"max":{"1min":1.7110000000000001,"5min":2.0819999999999999,"15min":2.3029999999999999},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4710000000000001,"15min":1.472},"min":{"1min":1.179,"5min":1.1060000000000001,"15min":0.80300000000000005},"max":{"1min":1.681,"5min":1.976,"15min":2.1240000000000001},"last":1.4710000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:01:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.45,"15min":1.4910000000000001},"min":{"1min":1.137,"5min":1.0569999999999999,"15min":0.75},"max":{"1min":1.6830000000000001,"5min":1.9430000000000001,"15min":2.403},"last":1.992},{"interface":"front","average":{"1min":1.542,"5min":1.579,"15min":1.6040000000000001},"min":{"1min":1.3560000000000001,"5min":1.282,"15min":0.92400000000000004},"max":{"1min":1.7529999999999999,"5min":2.157,"15min":2.6099999999999999},"last":2.2429999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575009,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40268,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331572,"statfs":{"total":96636764160,"available":96595529728,"internally_reserved":0,"allocated":778240,"data_stored":359279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86799999999999999,"15min":0.85099999999999998},"min":{"1min":0.69199999999999995,"5min":0.55500000000000005,"15min":0.28999999999999998},"max":{"1min":1.5469999999999999,"5min":1.6679999999999999,"15min":2.8690000000000002},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.115,"15min":1.02},"min":{"1min":0.78800000000000003,"5min":0.41199999999999998,"15min":0.28899999999999998},"max":{"1min":1.244,"5min":1.629,"15min":1.841},"last":0.93700000000000006}]},{"osd":1,"last update":"Mon Oct 16 10:01:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.087,"15min":1.034},"min":{"1min":0.81699999999999995,"5min":0.628,"15min":0.42899999999999999},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":2.6059999999999999},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.1519999999999999,"15min":1.115},"min":{"1min":0.86499999999999999,"5min":0.70899999999999996,"15min":0.40699999999999997},"max":{"1min":1.5669999999999999,"5min":1.712,"15min":2.6499999999999999},"last":1.3160000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:01:24 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0629999999999999,"15min":0.98099999999999998},"min":{"1min":0.80000000000000004,"5min":0.67100000000000004,"15min":0.38},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":2.5339999999999998},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2030000000000001,"15min":1.1919999999999999},"min":{"1min":0.92300000000000004,"5min":0.69099999999999995,"15min":0.48499999999999999},"max":{"1min":1.7150000000000001,"5min":1.7569999999999999,"15min":2.9700000000000002},"last":1.3899999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:01:24 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.371,"15min":1.3540000000000001},"min":{"1min":1.0860000000000001,"5min":0.84899999999999998,"15min":0.65000000000000002},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":2.2309999999999999},"last":1.546},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.3480000000000001,"15min":1.2929999999999999},"min":{"1min":1.1850000000000001,"5min":0.90200000000000002,"15min":0.77400000000000002},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":2.2000000000000002},"last":1.028}]},{"osd":5,"last update":"Mon Oct 16 10:01:24 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.403,"15min":1.3899999999999999},"min":{"1min":1.0349999999999999,"5min":0.81999999999999995,"15min":0.63300000000000001},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.7770000000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4419999999999999,"15min":1.4390000000000001},"min":{"1min":1.238,"5min":0.76600000000000001,"15min":0.68300000000000005},"max":{"1min":1.956,"5min":1.956,"15min":2.952},"last":1.6419999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:01:24 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.3799999999999999,"15min":1.3180000000000001},"min":{"1min":1.036,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.2469999999999999},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.454,"15min":1.431},"min":{"1min":1.2190000000000001,"5min":0.878,"15min":0.70499999999999996},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":2.8180000000000001},"last":1.577}]},{"osd":7,"last update":"Mon Oct 16 10:01:24 2023","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.5169999999999999,"15min":1.502},"min":{"1min":1.1379999999999999,"5min":0.96899999999999997,"15min":0.72999999999999998},"max":{"1min":2.0310000000000001,"5min":2.262,"15min":3.746},"last":1.754},{"interface":"front","average":{"1min":1.516,"5min":1.48,"15min":1.4279999999999999},"min":{"1min":1.226,"5min":1.091,"15min":0.79900000000000004},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3769999999999998},"last":1.151}]}]},{"osd":1,"up_from":13,"seq":55834575009,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49960,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321880,"statfs":{"total":96636764160,"available":96585605120,"internally_reserved":0,"allocated":753664,"data_stored":351415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.86699999999999999,"15min":0.88300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.137,"5min":1.5609999999999999,"15min":2.2010000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.98299999999999998,"15min":0.99299999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.45300000000000001},"max":{"1min":1.242,"5min":1.665,"15min":2.1619999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.999,"15min":1.0049999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.42299999999999999},"max":{"1min":1.526,"5min":1.526,"15min":2.3849999999999998},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.2430000000000001,"15min":1.238},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.50800000000000001},"max":{"1min":1.6419999999999999,"5min":1.667,"15min":2.2959999999999998},"last":1.4490000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.145,"15min":1.1459999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.532,"5min":1.665,"15min":2.254},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.89400000000000002,"5min":1.0980000000000001,"15min":1.1040000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.36099999999999999},"max":{"1min":1.3380000000000001,"5min":1.8280000000000001,"15min":2.8109999999999999},"last":1.1830000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.22,"15min":1.254},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7010000000000001,"5min":1.7769999999999999,"15min":2.4540000000000002},"last":1.105},{"interface":"front","average":{"1min":1.123,"5min":1.343,"15min":1.4059999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.624},"max":{"1min":1.4890000000000001,"5min":1.7929999999999999,"15min":2.851},"last":1.365}]},{"osd":5,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.335,"15min":1.3580000000000001},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.78300000000000003},"max":{"1min":1.5780000000000001,"5min":1.8240000000000001,"15min":2.7120000000000002},"last":1.131},{"interface":"front","average":{"1min":1.159,"5min":1.3280000000000001,"15min":1.3460000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.5740000000000001,"5min":2.0470000000000002,"15min":2.512},"last":1.397}]},{"osd":6,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.413,"15min":1.4279999999999999},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.72099999999999997},"max":{"1min":1.744,"5min":1.8979999999999999,"15min":2.75},"last":1.226},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.3939999999999999,"15min":1.419},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.748},"max":{"1min":1.591,"5min":2.1680000000000001,"15min":2.669},"last":1.24}]},{"osd":7,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.4710000000000001,"15min":1.5149999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.7370000000000001,"5min":1.885,"15min":2.8799999999999999},"last":1.486},{"interface":"front","average":{"1min":1.365,"5min":1.504,"15min":1.5189999999999999},"min":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.86699999999999999},"max":{"1min":1.8380000000000001,"5min":1.9470000000000001,"15min":3.1400000000000001},"last":1.3100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575010,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1359872,"data_stored":948047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83099999999999996,"15min":0.84199999999999997},"min":{"1min":0.68000000000000005,"5min":0.61299999999999999,"15min":0.376},"max":{"1min":0.97699999999999998,"5min":1.546,"15min":1.6419999999999999},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.046,"5min":1.0740000000000001,"15min":1.0109999999999999},"min":{"1min":0.72399999999999998,"5min":0.69099999999999995,"15min":0.41799999999999998},"max":{"1min":1.3819999999999999,"5min":1.5369999999999999,"15min":1.8500000000000001},"last":0.88100000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.026,"15min":1.0309999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.38900000000000001},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":2.2010000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.093,"5min":1.0880000000000001,"15min":1.0780000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.46600000000000003},"max":{"1min":1.5620000000000001,"5min":1.726,"15min":1.8959999999999999},"last":0.97499999999999998}]},{"osd":3,"last update":"Mon Oct 16 10:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.0569999999999999,"15min":1.01},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.534,"5min":1.534,"15min":1.6140000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.2110000000000001,"15min":1.181},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.47599999999999998},"max":{"1min":1.5189999999999999,"5min":1.7589999999999999,"15min":2.6110000000000002},"last":1.1459999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3180000000000001},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.77800000000000002},"max":{"1min":1.669,"5min":1.946,"15min":1.9930000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3069999999999999,"15min":1.28},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.5660000000000001,"5min":1.6040000000000001,"15min":2.0499999999999998},"last":1.0269999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3240000000000001,"15min":1.3140000000000001},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.77500000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":2.8159999999999998},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.359,"5min":1.3959999999999999,"15min":1.4079999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.7270000000000001,"5min":2.069,"15min":2.9009999999999998},"last":1.3979999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.409,"15min":1.4219999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.714,"5min":2.145,"15min":2.7160000000000002},"last":1.345},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.4570000000000001,"15min":1.4119999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.84999999999999998},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":2.056},"last":1.099}]},{"osd":7,"last update":"Mon Oct 16 10:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":1.492,"15min":1.452},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.81499999999999995},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.466,"15min":1.421},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.0390000000000001,"5min":2.157,"15min":2.157},"last":1.0529999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:02:18.513 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:02:18.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.737+0000 7f7b2f209640 1 Processor -- start 2023-10-16T10:02:18.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.737+0000 7f7b2f209640 1 -- start start 2023-10-16T10:02:18.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.738+0000 7f7b2f209640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b2815ae80 0x7f7b2815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:18.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.739+0000 7f7b2f209640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7b280ce390 0x7f7b28070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:18.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.739+0000 7f7b2f209640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b2815ed60 con 0x7f7b2815aaa0 2023-10-16T10:02:18.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.739+0000 7f7b2f209640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b2806b810 con 0x7f7b280ce390 2023-10-16T10:02:18.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.739+0000 7f7b2f209640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b280707e0 con 0x7f7b2815ae80 2023-10-16T10:02:18.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.739+0000 7f7b2d7a6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7b280ce390 0x7f7b28070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:18.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.739+0000 7f7b2d7a6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7b280ce390 0x7f7b28070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44682/0 (socket says 172.21.15.84:44682) 2023-10-16T10:02:18.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.739+0000 7f7b2d7a6640 1 -- 172.21.15.84:0/3076395018 learned_addr learned my addr 172.21.15.84:0/3076395018 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:18.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.739+0000 7f7b2d7a6640 1 -- 172.21.15.84:0/3076395018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b2815ae80 msgr2=0x7f7b2815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:18.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.739+0000 7f7b27fff640 1 --2- 172.21.15.84:0/3076395018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b2815ae80 0x7f7b2815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:18.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.739+0000 7f7b2d7a6640 1 --2- 172.21.15.84:0/3076395018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b2815ae80 0x7f7b2815e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.739+0000 7f7b2d7a6640 1 -- 172.21.15.84:0/3076395018 >> v1:172.21.15.157:6789/0 conn(0x7f7b2815aaa0 legacy=0x7f7b280cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:18.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.739+0000 7f7b2d7a6640 1 -- 172.21.15.84:0/3076395018 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2815ef90 con 0x7f7b280ce390 2023-10-16T10:02:18.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.740+0000 7f7b2d7a6640 1 --2- 172.21.15.84:0/3076395018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b280ce390 0x7f7b28070f50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009e50 tx=0x7f7b2002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab38176eb2370e42 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:18.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.741+0000 7f7b277fe640 1 -- 172.21.15.84:0/3076395018 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2002ca70 con 0x7f7b280ce390 2023-10-16T10:02:18.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.741+0000 7f7b277fe640 1 -- 172.21.15.84:0/3076395018 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2002cbd0 con 0x7f7b280ce390 2023-10-16T10:02:18.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.741+0000 7f7b277fe640 1 -- 172.21.15.84:0/3076395018 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b200343f0 con 0x7f7b280ce390 2023-10-16T10:02:18.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.741+0000 7f7b2f209640 1 -- 172.21.15.84:0/3076395018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b280ce390 msgr2=0x7f7b28070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.741+0000 7f7b2f209640 1 --2- 172.21.15.84:0/3076395018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b280ce390 0x7f7b28070f50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009e50 tx=0x7f7b2002bed0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.741+0000 7f7b2f209640 1 -- 172.21.15.84:0/3076395018 shutdown_connections 2023-10-16T10:02:18.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.741+0000 7f7b2f209640 1 --2- 172.21.15.84:0/3076395018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b280ce390 0x7f7b28070f50 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.741+0000 7f7b2f209640 1 --2- 172.21.15.84:0/3076395018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b2815ae80 0x7f7b2815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.741+0000 7f7b2f209640 1 -- 172.21.15.84:0/3076395018 >> 172.21.15.84:0/3076395018 conn(0x7f7b280cb410 msgr2=0x7f7b280cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:18.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.742+0000 7f7b2f209640 1 -- 172.21.15.84:0/3076395018 shutdown_connections 2023-10-16T10:02:18.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.742+0000 7f7b2f209640 1 -- 172.21.15.84:0/3076395018 wait complete. 2023-10-16T10:02:18.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.744+0000 7f7b2f209640 1 Processor -- start 2023-10-16T10:02:18.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.744+0000 7f7b2f209640 1 -- start start 2023-10-16T10:02:18.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.745+0000 7f7b2f209640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b280ce390 0x7f7b2815dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:18.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.745+0000 7f7b2f209640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b2815aaa0 0x7f7b280c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:18.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.745+0000 7f7b2f209640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b2815ae80 0x7f7b280c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:18.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.745+0000 7f7b2f209640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b28159a10 con 0x7f7b280ce390 2023-10-16T10:02:18.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.745+0000 7f7b2f209640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b280c6520 con 0x7f7b2815aaa0 2023-10-16T10:02:18.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.745+0000 7f7b2f209640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2815b480 con 0x7f7b2815ae80 2023-10-16T10:02:18.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.745+0000 7f7b2cfa5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b280ce390 0x7f7b2815dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:18.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.745+0000 7f7b2cfa5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b280ce390 0x7f7b2815dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44684/0 (socket says 172.21.15.84:44684) 2023-10-16T10:02:18.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.745+0000 7f7b2cfa5640 1 -- 172.21.15.84:0/3190833831 learned_addr learned my addr 172.21.15.84:0/3190833831 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:18.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.746+0000 7f7b27fff640 1 --2- 172.21.15.84:0/3190833831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b2815aaa0 0x7f7b280c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:18.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.746+0000 7f7b2cfa5640 1 -- 172.21.15.84:0/3190833831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b2815ae80 msgr2=0x7f7b280c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.746+0000 7f7b2d7a6640 1 --2- 172.21.15.84:0/3190833831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b2815ae80 0x7f7b280c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:18.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.746+0000 7f7b2cfa5640 1 --2- 172.21.15.84:0/3190833831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b2815ae80 0x7f7b280c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.746+0000 7f7b2cfa5640 1 -- 172.21.15.84:0/3190833831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b2815aaa0 msgr2=0x7f7b280c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.746+0000 7f7b2cfa5640 1 --2- 172.21.15.84:0/3190833831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b2815aaa0 0x7f7b280c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.746+0000 7f7b2cfa5640 1 -- 172.21.15.84:0/3190833831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b20009b30 con 0x7f7b280ce390 2023-10-16T10:02:18.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.746+0000 7f7b27fff640 1 --2- 172.21.15.84:0/3190833831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b2815aaa0 0x7f7b280c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:18.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.746+0000 7f7b2cfa5640 1 --2- 172.21.15.84:0/3190833831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b280ce390 0x7f7b2815dbb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00b3e0 tx=0x7f7b1c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:18.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.746+0000 7f7b257fa640 1 -- 172.21.15.84:0/3190833831 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c011830 con 0x7f7b280ce390 2023-10-16T10:02:18.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.746+0000 7f7b257fa640 1 -- 172.21.15.84:0/3190833831 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1c0119b0 con 0x7f7b280ce390 2023-10-16T10:02:18.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.746+0000 7f7b257fa640 1 -- 172.21.15.84:0/3190833831 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c011d60 con 0x7f7b280ce390 2023-10-16T10:02:18.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.746+0000 7f7b2f209640 1 -- 172.21.15.84:0/3190833831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2815b760 con 0x7f7b280ce390 2023-10-16T10:02:18.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.747+0000 7f7b2f209640 1 -- 172.21.15.84:0/3190833831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b2815bd20 con 0x7f7b280ce390 2023-10-16T10:02:18.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.748+0000 7f7b2f209640 1 -- 172.21.15.84:0/3190833831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b2815f530 con 0x7f7b280ce390 2023-10-16T10:02:18.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.749+0000 7f7b257fa640 1 -- 172.21.15.84:0/3190833831 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7b1c01a400 con 0x7f7b280ce390 2023-10-16T10:02:18.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.750+0000 7f7b257fa640 1 --2- 172.21.15.84:0/3190833831 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f7b0c06af90 0x7f7b0c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:18.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.750+0000 7f7b257fa640 1 -- 172.21.15.84:0/3190833831 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b1c0898e0 con 0x7f7b280ce390 2023-10-16T10:02:18.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.750+0000 7f7b27fff640 1 --2- 172.21.15.84:0/3190833831 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f7b0c06af90 0x7f7b0c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:18.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.751+0000 7f7b27fff640 1 --2- 172.21.15.84:0/3190833831 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f7b0c06af90 0x7f7b0c06d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7b280c54e0 tx=0x7f7b14008040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:18.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.755+0000 7f7b257fa640 1 -- 172.21.15.84:0/3190833831 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7b2815f530 con 0x7f7b280ce390 2023-10-16T10:02:18.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.942+0000 7f7b2f209640 1 -- 172.21.15.84:0/3190833831 --> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7b2815f270 con 0x7f7b0c06af90 2023-10-16T10:02:18.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.964+0000 7f7b257fa640 1 -- 172.21.15.84:0/3190833831 <== mgr.5742 v2:172.21.15.84:6832/585186562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286501 (secure 0 0 0) 0x7f7b1400c040 con 0x7f7b0c06af90 2023-10-16T10:02:18.967 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:02:18.971 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T10:02:18.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.971+0000 7f7b2f209640 1 -- 172.21.15.84:0/3190833831 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f7b0c06af90 msgr2=0x7f7b0c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.971+0000 7f7b2f209640 1 --2- 172.21.15.84:0/3190833831 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f7b0c06af90 0x7f7b0c06d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7b280c54e0 tx=0x7f7b14008040 comp rx=0 tx=0).stop 2023-10-16T10:02:18.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.971+0000 7f7b2f209640 1 -- 172.21.15.84:0/3190833831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b280ce390 msgr2=0x7f7b2815dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.971+0000 7f7b2f209640 1 --2- 172.21.15.84:0/3190833831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b280ce390 0x7f7b2815dbb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00b3e0 tx=0x7f7b1c00b8a0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.971+0000 7f7b2f209640 1 -- 172.21.15.84:0/3190833831 shutdown_connections 2023-10-16T10:02:18.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.971+0000 7f7b2f209640 1 --2- 172.21.15.84:0/3190833831 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f7b0c06af90 0x7f7b0c06d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.971+0000 7f7b2f209640 1 --2- 172.21.15.84:0/3190833831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b2815ae80 0x7f7b280c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.971+0000 7f7b2f209640 1 --2- 172.21.15.84:0/3190833831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b2815aaa0 0x7f7b280c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.971+0000 7f7b2f209640 1 --2- 172.21.15.84:0/3190833831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b280ce390 0x7f7b2815dbb0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.971+0000 7f7b2f209640 1 -- 172.21.15.84:0/3190833831 >> 172.21.15.84:0/3190833831 conn(0x7f7b280cb410 msgr2=0x7f7b280b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:18.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.971+0000 7f7b2f209640 1 -- 172.21.15.84:0/3190833831 shutdown_connections 2023-10-16T10:02:18.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:18.972+0000 7f7b2f209640 1 -- 172.21.15.84:0/3190833831 wait complete. 2023-10-16T10:02:18.993 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-16T10:02:17.259209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":144,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":330132,"kb_used_data":7128,"kb_used_omap":12,"kb_used_meta":322867,"kb_avail":754644588,"statfs":{"total":773094113280,"available":772756058112,"internally_reserved":0,"allocated":7299072,"data_stored":4021516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":330616377},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.221005"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853975+0000","last_change":"2023-10-16T10:02:14.088132+0000","last_active":"2023-10-16T10:02:15.853975+0000","last_peered":"2023-10-16T10:02:15.853975+0000","last_clean":"2023-10-16T10:02:15.853975+0000","last_became_active":"2023-10-16T10:02:14.087338+0000","last_became_peered":"2023-10-16T10:02:14.087338+0000","last_unstale":"2023-10-16T10:02:15.853975+0000","last_undegraded":"2023-10-16T10:02:15.853975+0000","last_fullsized":"2023-10-16T10:02:15.853975+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:09.201694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853369+0000","last_change":"2023-10-16T10:02:14.108341+0000","last_active":"2023-10-16T10:02:15.853369+0000","last_peered":"2023-10-16T10:02:15.853369+0000","last_clean":"2023-10-16T10:02:15.853369+0000","last_became_active":"2023-10-16T10:02:14.107965+0000","last_became_peered":"2023-10-16T10:02:14.107965+0000","last_unstale":"2023-10-16T10:02:15.853369+0000","last_undegraded":"2023-10-16T10:02:15.853369+0000","last_fullsized":"2023-10-16T10:02:15.853369+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:26:37.089713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083425+0000","last_change":"2023-10-16T10:02:14.098377+0000","last_active":"2023-10-16T10:02:15.083425+0000","last_peered":"2023-10-16T10:02:15.083425+0000","last_clean":"2023-10-16T10:02:15.083425+0000","last_became_active":"2023-10-16T10:02:14.098126+0000","last_became_peered":"2023-10-16T10:02:14.098126+0000","last_unstale":"2023-10-16T10:02:15.083425+0000","last_undegraded":"2023-10-16T10:02:15.083425+0000","last_fullsized":"2023-10-16T10:02:15.083425+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:48:52.298460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083982+0000","last_change":"2023-10-16T10:02:14.113409+0000","last_active":"2023-10-16T10:02:15.083982+0000","last_peered":"2023-10-16T10:02:15.083982+0000","last_clean":"2023-10-16T10:02:15.083982+0000","last_became_active":"2023-10-16T10:02:14.112394+0000","last_became_peered":"2023-10-16T10:02:14.112394+0000","last_unstale":"2023-10-16T10:02:15.083982+0000","last_undegraded":"2023-10-16T10:02:15.083982+0000","last_fullsized":"2023-10-16T10:02:15.083982+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:36.785811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.852855+0000","last_change":"2023-10-16T10:02:14.108359+0000","last_active":"2023-10-16T10:02:15.852855+0000","last_peered":"2023-10-16T10:02:15.852855+0000","last_clean":"2023-10-16T10:02:15.852855+0000","last_became_active":"2023-10-16T10:02:14.107877+0000","last_became_peered":"2023-10-16T10:02:14.107877+0000","last_unstale":"2023-10-16T10:02:15.852855+0000","last_undegraded":"2023-10-16T10:02:15.852855+0000","last_fullsized":"2023-10-16T10:02:15.852855+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:39:57.827606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.330891+0000","last_change":"2023-10-16T10:02:14.098885+0000","last_active":"2023-10-16T10:02:15.330891+0000","last_peered":"2023-10-16T10:02:15.330891+0000","last_clean":"2023-10-16T10:02:15.330891+0000","last_became_active":"2023-10-16T10:02:14.096342+0000","last_became_peered":"2023-10-16T10:02:14.096342+0000","last_unstale":"2023-10-16T10:02:15.330891+0000","last_undegraded":"2023-10-16T10:02:15.330891+0000","last_fullsized":"2023-10-16T10:02:15.330891+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:32.569778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.854141+0000","last_change":"2023-10-16T10:02:14.096334+0000","last_active":"2023-10-16T10:02:15.854141+0000","last_peered":"2023-10-16T10:02:15.854141+0000","last_clean":"2023-10-16T10:02:15.854141+0000","last_became_active":"2023-10-16T10:02:14.095991+0000","last_became_peered":"2023-10-16T10:02:14.095991+0000","last_unstale":"2023-10-16T10:02:15.854141+0000","last_undegraded":"2023-10-16T10:02:15.854141+0000","last_fullsized":"2023-10-16T10:02:15.854141+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:48:28.768978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.082434+0000","last_change":"2023-10-16T10:02:14.099198+0000","last_active":"2023-10-16T10:02:15.082434+0000","last_peered":"2023-10-16T10:02:15.082434+0000","last_clean":"2023-10-16T10:02:15.082434+0000","last_became_active":"2023-10-16T10:02:14.096136+0000","last_became_peered":"2023-10-16T10:02:14.096136+0000","last_unstale":"2023-10-16T10:02:15.082434+0000","last_undegraded":"2023-10-16T10:02:15.082434+0000","last_fullsized":"2023-10-16T10:02:15.082434+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:53.047567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083761+0000","last_change":"2023-10-16T10:02:14.092815+0000","last_active":"2023-10-16T10:02:15.083761+0000","last_peered":"2023-10-16T10:02:15.083761+0000","last_clean":"2023-10-16T10:02:15.083761+0000","last_became_active":"2023-10-16T10:02:14.092495+0000","last_became_peered":"2023-10-16T10:02:14.092495+0000","last_unstale":"2023-10-16T10:02:15.083761+0000","last_undegraded":"2023-10-16T10:02:15.083761+0000","last_fullsized":"2023-10-16T10:02:15.083761+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:09:06.890750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084403+0000","last_change":"2023-10-16T10:02:14.099018+0000","last_active":"2023-10-16T10:02:15.084403+0000","last_peered":"2023-10-16T10:02:15.084403+0000","last_clean":"2023-10-16T10:02:15.084403+0000","last_became_active":"2023-10-16T10:02:14.098706+0000","last_became_peered":"2023-10-16T10:02:14.098706+0000","last_unstale":"2023-10-16T10:02:15.084403+0000","last_undegraded":"2023-10-16T10:02:15.084403+0000","last_fullsized":"2023-10-16T10:02:15.084403+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:41:24.644785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.082679+0000","last_change":"2023-10-16T10:02:14.103119+0000","last_active":"2023-10-16T10:02:15.082679+0000","last_peered":"2023-10-16T10:02:15.082679+0000","last_clean":"2023-10-16T10:02:15.082679+0000","last_became_active":"2023-10-16T10:02:14.102839+0000","last_became_peered":"2023-10-16T10:02:14.102839+0000","last_unstale":"2023-10-16T10:02:15.082679+0000","last_undegraded":"2023-10-16T10:02:15.082679+0000","last_fullsized":"2023-10-16T10:02:15.082679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:36:22.346800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853099+0000","last_change":"2023-10-16T10:02:14.101568+0000","last_active":"2023-10-16T10:02:15.853099+0000","last_peered":"2023-10-16T10:02:15.853099+0000","last_clean":"2023-10-16T10:02:15.853099+0000","last_became_active":"2023-10-16T10:02:14.101293+0000","last_became_peered":"2023-10-16T10:02:14.101293+0000","last_unstale":"2023-10-16T10:02:15.853099+0000","last_undegraded":"2023-10-16T10:02:15.853099+0000","last_fullsized":"2023-10-16T10:02:15.853099+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:00.096453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.082740+0000","last_change":"2023-10-16T10:02:14.105019+0000","last_active":"2023-10-16T10:02:15.082740+0000","last_peered":"2023-10-16T10:02:15.082740+0000","last_clean":"2023-10-16T10:02:15.082740+0000","last_became_active":"2023-10-16T10:02:14.104553+0000","last_became_peered":"2023-10-16T10:02:14.104553+0000","last_unstale":"2023-10-16T10:02:15.082740+0000","last_undegraded":"2023-10-16T10:02:15.082740+0000","last_fullsized":"2023-10-16T10:02:15.082740+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:03.008219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083140+0000","last_change":"2023-10-16T10:02:14.092002+0000","last_active":"2023-10-16T10:02:15.083140+0000","last_peered":"2023-10-16T10:02:15.083140+0000","last_clean":"2023-10-16T10:02:15.083140+0000","last_became_active":"2023-10-16T10:02:14.089336+0000","last_became_peered":"2023-10-16T10:02:14.089336+0000","last_unstale":"2023-10-16T10:02:15.083140+0000","last_undegraded":"2023-10-16T10:02:15.083140+0000","last_fullsized":"2023-10-16T10:02:15.083140+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:48:41.792471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.330939+0000","last_change":"2023-10-16T10:02:14.095813+0000","last_active":"2023-10-16T10:02:15.330939+0000","last_peered":"2023-10-16T10:02:15.330939+0000","last_clean":"2023-10-16T10:02:15.330939+0000","last_became_active":"2023-10-16T10:02:14.089993+0000","last_became_peered":"2023-10-16T10:02:14.089993+0000","last_unstale":"2023-10-16T10:02:15.330939+0000","last_undegraded":"2023-10-16T10:02:15.330939+0000","last_fullsized":"2023-10-16T10:02:15.330939+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:17.666241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.851765+0000","last_change":"2023-10-16T10:02:14.100364+0000","last_active":"2023-10-16T10:02:15.851765+0000","last_peered":"2023-10-16T10:02:15.851765+0000","last_clean":"2023-10-16T10:02:15.851765+0000","last_became_active":"2023-10-16T10:02:14.099931+0000","last_became_peered":"2023-10-16T10:02:14.099931+0000","last_unstale":"2023-10-16T10:02:15.851765+0000","last_undegraded":"2023-10-16T10:02:15.851765+0000","last_fullsized":"2023-10-16T10:02:15.851765+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:36:22.346800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084001+0000","last_change":"2023-10-16T10:02:14.099188+0000","last_active":"2023-10-16T10:02:15.084001+0000","last_peered":"2023-10-16T10:02:15.084001+0000","last_clean":"2023-10-16T10:02:15.084001+0000","last_became_active":"2023-10-16T10:02:14.098896+0000","last_became_peered":"2023-10-16T10:02:14.098896+0000","last_unstale":"2023-10-16T10:02:15.084001+0000","last_undegraded":"2023-10-16T10:02:15.084001+0000","last_fullsized":"2023-10-16T10:02:15.084001+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:11:56.063588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084056+0000","last_change":"2023-10-16T10:02:14.107323+0000","last_active":"2023-10-16T10:02:15.084056+0000","last_peered":"2023-10-16T10:02:15.084056+0000","last_clean":"2023-10-16T10:02:15.084056+0000","last_became_active":"2023-10-16T10:02:14.106677+0000","last_became_peered":"2023-10-16T10:02:14.106677+0000","last_unstale":"2023-10-16T10:02:15.084056+0000","last_undegraded":"2023-10-16T10:02:15.084056+0000","last_fullsized":"2023-10-16T10:02:15.084056+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:28:24.465488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.852945+0000","last_change":"2023-10-16T10:02:14.098999+0000","last_active":"2023-10-16T10:02:15.852945+0000","last_peered":"2023-10-16T10:02:15.852945+0000","last_clean":"2023-10-16T10:02:15.852945+0000","last_became_active":"2023-10-16T10:02:14.098692+0000","last_became_peered":"2023-10-16T10:02:14.098692+0000","last_unstale":"2023-10-16T10:02:15.852945+0000","last_undegraded":"2023-10-16T10:02:15.852945+0000","last_fullsized":"2023-10-16T10:02:15.852945+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:09:06.890750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083285+0000","last_change":"2023-10-16T10:02:14.097706+0000","last_active":"2023-10-16T10:02:15.083285+0000","last_peered":"2023-10-16T10:02:15.083285+0000","last_clean":"2023-10-16T10:02:15.083285+0000","last_became_active":"2023-10-16T10:02:14.097437+0000","last_became_peered":"2023-10-16T10:02:14.097437+0000","last_unstale":"2023-10-16T10:02:15.083285+0000","last_undegraded":"2023-10-16T10:02:15.083285+0000","last_fullsized":"2023-10-16T10:02:15.083285+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:35.414153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331294+0000","last_change":"2023-10-16T10:02:14.108510+0000","last_active":"2023-10-16T10:02:15.331294+0000","last_peered":"2023-10-16T10:02:15.331294+0000","last_clean":"2023-10-16T10:02:15.331294+0000","last_became_active":"2023-10-16T10:02:14.108291+0000","last_became_peered":"2023-10-16T10:02:14.108291+0000","last_unstale":"2023-10-16T10:02:15.331294+0000","last_undegraded":"2023-10-16T10:02:15.331294+0000","last_fullsized":"2023-10-16T10:02:15.331294+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:29:29.947212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083369+0000","last_change":"2023-10-16T10:02:14.099008+0000","last_active":"2023-10-16T10:02:15.083369+0000","last_peered":"2023-10-16T10:02:15.083369+0000","last_clean":"2023-10-16T10:02:15.083369+0000","last_became_active":"2023-10-16T10:02:14.098623+0000","last_became_peered":"2023-10-16T10:02:14.098623+0000","last_unstale":"2023-10-16T10:02:15.083369+0000","last_undegraded":"2023-10-16T10:02:15.083369+0000","last_fullsized":"2023-10-16T10:02:15.083369+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:20.173125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.852589+0000","last_change":"2023-10-16T10:02:14.089129+0000","last_active":"2023-10-16T10:02:15.852589+0000","last_peered":"2023-10-16T10:02:15.852589+0000","last_clean":"2023-10-16T10:02:15.852589+0000","last_became_active":"2023-10-16T10:02:14.088869+0000","last_became_peered":"2023-10-16T10:02:14.088869+0000","last_unstale":"2023-10-16T10:02:15.852589+0000","last_undegraded":"2023-10-16T10:02:15.852589+0000","last_fullsized":"2023-10-16T10:02:15.852589+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:48:41.792471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.851817+0000","last_change":"2023-10-16T10:02:14.087824+0000","last_active":"2023-10-16T10:02:15.851817+0000","last_peered":"2023-10-16T10:02:15.851817+0000","last_clean":"2023-10-16T10:02:15.851817+0000","last_became_active":"2023-10-16T10:02:14.087620+0000","last_became_peered":"2023-10-16T10:02:14.087620+0000","last_unstale":"2023-10-16T10:02:15.851817+0000","last_undegraded":"2023-10-16T10:02:15.851817+0000","last_fullsized":"2023-10-16T10:02:15.851817+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:21.047154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853347+0000","last_change":"2023-10-16T10:02:14.089519+0000","last_active":"2023-10-16T10:02:15.853347+0000","last_peered":"2023-10-16T10:02:15.853347+0000","last_clean":"2023-10-16T10:02:15.853347+0000","last_became_active":"2023-10-16T10:02:14.089201+0000","last_became_peered":"2023-10-16T10:02:14.089201+0000","last_unstale":"2023-10-16T10:02:15.853347+0000","last_undegraded":"2023-10-16T10:02:15.853347+0000","last_fullsized":"2023-10-16T10:02:15.853347+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:49.923362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084248+0000","last_change":"2023-10-16T10:02:14.103218+0000","last_active":"2023-10-16T10:02:15.084248+0000","last_peered":"2023-10-16T10:02:15.084248+0000","last_clean":"2023-10-16T10:02:15.084248+0000","last_became_active":"2023-10-16T10:02:14.102937+0000","last_became_peered":"2023-10-16T10:02:14.102937+0000","last_unstale":"2023-10-16T10:02:15.084248+0000","last_undegraded":"2023-10-16T10:02:15.084248+0000","last_fullsized":"2023-10-16T10:02:15.084248+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:26.757675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083881+0000","last_change":"2023-10-16T10:02:14.101055+0000","last_active":"2023-10-16T10:02:15.083881+0000","last_peered":"2023-10-16T10:02:15.083881+0000","last_clean":"2023-10-16T10:02:15.083881+0000","last_became_active":"2023-10-16T10:02:14.100575+0000","last_became_peered":"2023-10-16T10:02:14.100575+0000","last_unstale":"2023-10-16T10:02:15.083881+0000","last_undegraded":"2023-10-16T10:02:15.083881+0000","last_fullsized":"2023-10-16T10:02:15.083881+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:37:37.802382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853158+0000","last_change":"2023-10-16T10:02:14.106849+0000","last_active":"2023-10-16T10:02:15.853158+0000","last_peered":"2023-10-16T10:02:15.853158+0000","last_clean":"2023-10-16T10:02:15.853158+0000","last_became_active":"2023-10-16T10:02:14.104132+0000","last_became_peered":"2023-10-16T10:02:14.104132+0000","last_unstale":"2023-10-16T10:02:15.853158+0000","last_undegraded":"2023-10-16T10:02:15.853158+0000","last_fullsized":"2023-10-16T10:02:15.853158+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:47:46.869784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853168+0000","last_change":"2023-10-16T10:02:14.088497+0000","last_active":"2023-10-16T10:02:15.853168+0000","last_peered":"2023-10-16T10:02:15.853168+0000","last_clean":"2023-10-16T10:02:15.853168+0000","last_became_active":"2023-10-16T10:02:14.085556+0000","last_became_peered":"2023-10-16T10:02:14.085556+0000","last_unstale":"2023-10-16T10:02:15.853168+0000","last_undegraded":"2023-10-16T10:02:15.853168+0000","last_fullsized":"2023-10-16T10:02:15.853168+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:36.785811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083254+0000","last_change":"2023-10-16T10:02:14.089445+0000","last_active":"2023-10-16T10:02:15.083254+0000","last_peered":"2023-10-16T10:02:15.083254+0000","last_clean":"2023-10-16T10:02:15.083254+0000","last_became_active":"2023-10-16T10:02:14.089148+0000","last_became_peered":"2023-10-16T10:02:14.089148+0000","last_unstale":"2023-10-16T10:02:15.083254+0000","last_undegraded":"2023-10-16T10:02:15.083254+0000","last_fullsized":"2023-10-16T10:02:15.083254+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:47:04.918722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331082+0000","last_change":"2023-10-16T10:02:14.102795+0000","last_active":"2023-10-16T10:02:15.331082+0000","last_peered":"2023-10-16T10:02:15.331082+0000","last_clean":"2023-10-16T10:02:15.331082+0000","last_became_active":"2023-10-16T10:02:14.102551+0000","last_became_peered":"2023-10-16T10:02:14.102551+0000","last_unstale":"2023-10-16T10:02:15.331082+0000","last_undegraded":"2023-10-16T10:02:15.331082+0000","last_fullsized":"2023-10-16T10:02:15.331082+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:10.287376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.082427+0000","last_change":"2023-10-16T10:02:14.080355+0000","last_active":"2023-10-16T10:02:15.082427+0000","last_peered":"2023-10-16T10:02:15.082427+0000","last_clean":"2023-10-16T10:02:15.082427+0000","last_became_active":"2023-10-16T10:02:14.080047+0000","last_became_peered":"2023-10-16T10:02:14.080047+0000","last_unstale":"2023-10-16T10:02:15.082427+0000","last_undegraded":"2023-10-16T10:02:15.082427+0000","last_fullsized":"2023-10-16T10:02:15.082427+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:49:01.278823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.850837+0000","last_change":"2023-10-16T10:02:14.110370+0000","last_active":"2023-10-16T10:02:15.850837+0000","last_peered":"2023-10-16T10:02:15.850837+0000","last_clean":"2023-10-16T10:02:15.850837+0000","last_became_active":"2023-10-16T10:02:14.110076+0000","last_became_peered":"2023-10-16T10:02:14.110076+0000","last_unstale":"2023-10-16T10:02:15.850837+0000","last_undegraded":"2023-10-16T10:02:15.850837+0000","last_fullsized":"2023-10-16T10:02:15.850837+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:48.244293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'4","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331875+0000","last_change":"2023-10-16T10:02:13.081072+0000","last_active":"2023-10-16T10:02:15.331875+0000","last_peered":"2023-10-16T10:02:15.331875+0000","last_clean":"2023-10-16T10:02:15.331875+0000","last_became_active":"2023-10-16T10:02:13.080304+0000","last_became_peered":"2023-10-16T10:02:13.080304+0000","last_unstale":"2023-10-16T10:02:15.331875+0000","last_undegraded":"2023-10-16T10:02:15.331875+0000","last_fullsized":"2023-10-16T10:02:15.331875+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:15.580804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853673+0000","last_change":"2023-10-16T10:02:14.087498+0000","last_active":"2023-10-16T10:02:15.853673+0000","last_peered":"2023-10-16T10:02:15.853673+0000","last_clean":"2023-10-16T10:02:15.853673+0000","last_became_active":"2023-10-16T10:02:14.087213+0000","last_became_peered":"2023-10-16T10:02:14.087213+0000","last_unstale":"2023-10-16T10:02:15.853673+0000","last_undegraded":"2023-10-16T10:02:15.853673+0000","last_fullsized":"2023-10-16T10:02:15.853673+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:11:13.894974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"63'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084241+0000","last_change":"2023-10-16T10:02:13.084564+0000","last_active":"2023-10-16T10:02:15.084241+0000","last_peered":"2023-10-16T10:02:15.084241+0000","last_clean":"2023-10-16T10:02:15.084241+0000","last_became_active":"2023-10-16T10:02:13.084232+0000","last_became_peered":"2023-10-16T10:02:13.084232+0000","last_unstale":"2023-10-16T10:02:15.084241+0000","last_undegraded":"2023-10-16T10:02:15.084241+0000","last_fullsized":"2023-10-16T10:02:15.084241+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:07:37.011371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853466+0000","last_change":"2023-10-16T10:02:14.111281+0000","last_active":"2023-10-16T10:02:15.853466+0000","last_peered":"2023-10-16T10:02:15.853466+0000","last_clean":"2023-10-16T10:02:15.853466+0000","last_became_active":"2023-10-16T10:02:14.110934+0000","last_became_peered":"2023-10-16T10:02:14.110934+0000","last_unstale":"2023-10-16T10:02:15.853466+0000","last_undegraded":"2023-10-16T10:02:15.853466+0000","last_fullsized":"2023-10-16T10:02:15.853466+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:54.881219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"63'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.850917+0000","last_change":"2023-10-16T10:02:13.073719+0000","last_active":"2023-10-16T10:02:15.850917+0000","last_peered":"2023-10-16T10:02:15.850917+0000","last_clean":"2023-10-16T10:02:15.850917+0000","last_became_active":"2023-10-16T10:02:13.073439+0000","last_became_peered":"2023-10-16T10:02:13.073439+0000","last_unstale":"2023-10-16T10:02:15.850917+0000","last_undegraded":"2023-10-16T10:02:15.850917+0000","last_fullsized":"2023-10-16T10:02:15.850917+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:26:15.458200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.851011+0000","last_change":"2023-10-16T10:02:14.100418+0000","last_active":"2023-10-16T10:02:15.851011+0000","last_peered":"2023-10-16T10:02:15.851011+0000","last_clean":"2023-10-16T10:02:15.851011+0000","last_became_active":"2023-10-16T10:02:14.096948+0000","last_became_peered":"2023-10-16T10:02:14.096948+0000","last_unstale":"2023-10-16T10:02:15.851011+0000","last_undegraded":"2023-10-16T10:02:15.851011+0000","last_fullsized":"2023-10-16T10:02:15.851011+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:02:52.172622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"63'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853797+0000","last_change":"2023-10-16T10:02:13.080182+0000","last_active":"2023-10-16T10:02:15.853797+0000","last_peered":"2023-10-16T10:02:15.853797+0000","last_clean":"2023-10-16T10:02:15.853797+0000","last_became_active":"2023-10-16T10:02:13.079751+0000","last_became_peered":"2023-10-16T10:02:13.079751+0000","last_unstale":"2023-10-16T10:02:15.853797+0000","last_undegraded":"2023-10-16T10:02:15.853797+0000","last_fullsized":"2023-10-16T10:02:15.853797+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:03:18.530382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083710+0000","last_change":"2023-10-16T10:02:14.089957+0000","last_active":"2023-10-16T10:02:15.083710+0000","last_peered":"2023-10-16T10:02:15.083710+0000","last_clean":"2023-10-16T10:02:15.083710+0000","last_became_active":"2023-10-16T10:02:14.089685+0000","last_became_peered":"2023-10-16T10:02:14.089685+0000","last_unstale":"2023-10-16T10:02:15.083710+0000","last_undegraded":"2023-10-16T10:02:15.083710+0000","last_fullsized":"2023-10-16T10:02:15.083710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:07.821140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"63'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331541+0000","last_change":"2023-10-16T10:02:13.078415+0000","last_active":"2023-10-16T10:02:15.331541+0000","last_peered":"2023-10-16T10:02:15.331541+0000","last_clean":"2023-10-16T10:02:15.331541+0000","last_became_active":"2023-10-16T10:02:13.078170+0000","last_became_peered":"2023-10-16T10:02:13.078170+0000","last_unstale":"2023-10-16T10:02:15.331541+0000","last_undegraded":"2023-10-16T10:02:15.331541+0000","last_fullsized":"2023-10-16T10:02:15.331541+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:09:37.828921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083510+0000","last_change":"2023-10-16T10:02:14.100325+0000","last_active":"2023-10-16T10:02:15.083510+0000","last_peered":"2023-10-16T10:02:15.083510+0000","last_clean":"2023-10-16T10:02:15.083510+0000","last_became_active":"2023-10-16T10:02:14.097653+0000","last_became_peered":"2023-10-16T10:02:14.097653+0000","last_unstale":"2023-10-16T10:02:15.083510+0000","last_undegraded":"2023-10-16T10:02:15.083510+0000","last_fullsized":"2023-10-16T10:02:15.083510+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:12:17.504297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"63'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083730+0000","last_change":"2023-10-16T10:02:13.076015+0000","last_active":"2023-10-16T10:02:15.083730+0000","last_peered":"2023-10-16T10:02:15.083730+0000","last_clean":"2023-10-16T10:02:15.083730+0000","last_became_active":"2023-10-16T10:02:13.075315+0000","last_became_peered":"2023-10-16T10:02:13.075315+0000","last_unstale":"2023-10-16T10:02:15.083730+0000","last_undegraded":"2023-10-16T10:02:15.083730+0000","last_fullsized":"2023-10-16T10:02:15.083730+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:15:48.884075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.082560+0000","last_change":"2023-10-16T10:02:14.096515+0000","last_active":"2023-10-16T10:02:15.082560+0000","last_peered":"2023-10-16T10:02:15.082560+0000","last_clean":"2023-10-16T10:02:15.082560+0000","last_became_active":"2023-10-16T10:02:14.095812+0000","last_became_peered":"2023-10-16T10:02:14.095812+0000","last_unstale":"2023-10-16T10:02:15.082560+0000","last_undegraded":"2023-10-16T10:02:15.082560+0000","last_fullsized":"2023-10-16T10:02:15.082560+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:18:15.353674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.852754+0000","last_change":"2023-10-16T10:02:13.069389+0000","last_active":"2023-10-16T10:02:15.852754+0000","last_peered":"2023-10-16T10:02:15.852754+0000","last_clean":"2023-10-16T10:02:15.852754+0000","last_became_active":"2023-10-16T10:02:13.069063+0000","last_became_peered":"2023-10-16T10:02:13.069063+0000","last_unstale":"2023-10-16T10:02:15.852754+0000","last_undegraded":"2023-10-16T10:02:15.852754+0000","last_fullsized":"2023-10-16T10:02:15.852754+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:50.758738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.851602+0000","last_change":"2023-10-16T10:02:14.090663+0000","last_active":"2023-10-16T10:02:15.851602+0000","last_peered":"2023-10-16T10:02:15.851602+0000","last_clean":"2023-10-16T10:02:15.851602+0000","last_became_active":"2023-10-16T10:02:14.090440+0000","last_became_peered":"2023-10-16T10:02:14.090440+0000","last_unstale":"2023-10-16T10:02:15.851602+0000","last_undegraded":"2023-10-16T10:02:15.851602+0000","last_fullsized":"2023-10-16T10:02:15.851602+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:18:15.353674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.082650+0000","last_change":"2023-10-16T10:02:13.068689+0000","last_active":"2023-10-16T10:02:15.082650+0000","last_peered":"2023-10-16T10:02:15.082650+0000","last_clean":"2023-10-16T10:02:15.082650+0000","last_became_active":"2023-10-16T10:02:13.068426+0000","last_became_peered":"2023-10-16T10:02:13.068426+0000","last_unstale":"2023-10-16T10:02:15.082650+0000","last_undegraded":"2023-10-16T10:02:15.082650+0000","last_fullsized":"2023-10-16T10:02:15.082650+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:42:38.072387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084206+0000","last_change":"2023-10-16T10:02:14.099445+0000","last_active":"2023-10-16T10:02:15.084206+0000","last_peered":"2023-10-16T10:02:15.084206+0000","last_clean":"2023-10-16T10:02:15.084206+0000","last_became_active":"2023-10-16T10:02:14.099142+0000","last_became_peered":"2023-10-16T10:02:14.099142+0000","last_unstale":"2023-10-16T10:02:15.084206+0000","last_undegraded":"2023-10-16T10:02:15.084206+0000","last_fullsized":"2023-10-16T10:02:15.084206+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:27.774246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084258+0000","last_change":"2023-10-16T10:02:13.073414+0000","last_active":"2023-10-16T10:02:15.084258+0000","last_peered":"2023-10-16T10:02:15.084258+0000","last_clean":"2023-10-16T10:02:15.084258+0000","last_became_active":"2023-10-16T10:02:13.073130+0000","last_became_peered":"2023-10-16T10:02:13.073130+0000","last_unstale":"2023-10-16T10:02:15.084258+0000","last_undegraded":"2023-10-16T10:02:15.084258+0000","last_fullsized":"2023-10-16T10:02:15.084258+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:47.231504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.854266+0000","last_change":"2023-10-16T10:02:14.098714+0000","last_active":"2023-10-16T10:02:15.854266+0000","last_peered":"2023-10-16T10:02:15.854266+0000","last_clean":"2023-10-16T10:02:15.854266+0000","last_became_active":"2023-10-16T10:02:14.097848+0000","last_became_peered":"2023-10-16T10:02:14.097848+0000","last_unstale":"2023-10-16T10:02:15.854266+0000","last_undegraded":"2023-10-16T10:02:15.854266+0000","last_fullsized":"2023-10-16T10:02:15.854266+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:17:48.268251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084539+0000","last_change":"2023-10-16T10:02:13.075792+0000","last_active":"2023-10-16T10:02:15.084539+0000","last_peered":"2023-10-16T10:02:15.084539+0000","last_clean":"2023-10-16T10:02:15.084539+0000","last_became_active":"2023-10-16T10:02:13.075465+0000","last_became_peered":"2023-10-16T10:02:13.075465+0000","last_unstale":"2023-10-16T10:02:15.084539+0000","last_undegraded":"2023-10-16T10:02:15.084539+0000","last_fullsized":"2023-10-16T10:02:15.084539+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:52:57.656343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331508+0000","last_change":"2023-10-16T10:02:14.096959+0000","last_active":"2023-10-16T10:02:15.331508+0000","last_peered":"2023-10-16T10:02:15.331508+0000","last_clean":"2023-10-16T10:02:15.331508+0000","last_became_active":"2023-10-16T10:02:14.091205+0000","last_became_peered":"2023-10-16T10:02:14.091205+0000","last_unstale":"2023-10-16T10:02:15.331508+0000","last_undegraded":"2023-10-16T10:02:15.331508+0000","last_fullsized":"2023-10-16T10:02:15.331508+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:15:39.423661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"63'3","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.854511+0000","last_change":"2023-10-16T10:02:13.086073+0000","last_active":"2023-10-16T10:02:15.854511+0000","last_peered":"2023-10-16T10:02:15.854511+0000","last_clean":"2023-10-16T10:02:15.854511+0000","last_became_active":"2023-10-16T10:02:13.085249+0000","last_became_peered":"2023-10-16T10:02:13.085249+0000","last_unstale":"2023-10-16T10:02:15.854511+0000","last_undegraded":"2023-10-16T10:02:15.854511+0000","last_fullsized":"2023-10-16T10:02:15.854511+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:09:35.302158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853998+0000","last_change":"2023-10-16T10:02:14.102405+0000","last_active":"2023-10-16T10:02:15.853998+0000","last_peered":"2023-10-16T10:02:15.853998+0000","last_clean":"2023-10-16T10:02:15.853998+0000","last_became_active":"2023-10-16T10:02:14.102026+0000","last_became_peered":"2023-10-16T10:02:14.102026+0000","last_unstale":"2023-10-16T10:02:15.853998+0000","last_undegraded":"2023-10-16T10:02:15.853998+0000","last_fullsized":"2023-10-16T10:02:15.853998+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:53:24.214913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"63'6","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083320+0000","last_change":"2023-10-16T10:02:13.072831+0000","last_active":"2023-10-16T10:02:15.083320+0000","last_peered":"2023-10-16T10:02:15.083320+0000","last_clean":"2023-10-16T10:02:15.083320+0000","last_became_active":"2023-10-16T10:02:13.072394+0000","last_became_peered":"2023-10-16T10:02:13.072394+0000","last_unstale":"2023-10-16T10:02:15.083320+0000","last_undegraded":"2023-10-16T10:02:15.083320+0000","last_fullsized":"2023-10-16T10:02:15.083320+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:57.765713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.852559+0000","last_change":"2023-10-16T10:02:14.105873+0000","last_active":"2023-10-16T10:02:15.852559+0000","last_peered":"2023-10-16T10:02:15.852559+0000","last_clean":"2023-10-16T10:02:15.852559+0000","last_became_active":"2023-10-16T10:02:14.103749+0000","last_became_peered":"2023-10-16T10:02:14.103749+0000","last_unstale":"2023-10-16T10:02:15.852559+0000","last_undegraded":"2023-10-16T10:02:15.852559+0000","last_fullsized":"2023-10-16T10:02:15.852559+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:12:17.504297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"63'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331811+0000","last_change":"2023-10-16T10:02:13.069939+0000","last_active":"2023-10-16T10:02:15.331811+0000","last_peered":"2023-10-16T10:02:15.331811+0000","last_clean":"2023-10-16T10:02:15.331811+0000","last_became_active":"2023-10-16T10:02:13.069646+0000","last_became_peered":"2023-10-16T10:02:13.069646+0000","last_unstale":"2023-10-16T10:02:15.331811+0000","last_undegraded":"2023-10-16T10:02:15.331811+0000","last_fullsized":"2023-10-16T10:02:15.331811+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:12:25.454689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083630+0000","last_change":"2023-10-16T10:02:14.093892+0000","last_active":"2023-10-16T10:02:15.083630+0000","last_peered":"2023-10-16T10:02:15.083630+0000","last_clean":"2023-10-16T10:02:15.083630+0000","last_became_active":"2023-10-16T10:02:14.093513+0000","last_became_peered":"2023-10-16T10:02:14.093513+0000","last_unstale":"2023-10-16T10:02:15.083630+0000","last_undegraded":"2023-10-16T10:02:15.083630+0000","last_fullsized":"2023-10-16T10:02:15.083630+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:48:47.171684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.854081+0000","last_change":"2023-10-16T10:02:13.077582+0000","last_active":"2023-10-16T10:02:15.854081+0000","last_peered":"2023-10-16T10:02:15.854081+0000","last_clean":"2023-10-16T10:02:15.854081+0000","last_became_active":"2023-10-16T10:02:13.077313+0000","last_became_peered":"2023-10-16T10:02:13.077313+0000","last_unstale":"2023-10-16T10:02:15.854081+0000","last_undegraded":"2023-10-16T10:02:15.854081+0000","last_fullsized":"2023-10-16T10:02:15.854081+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:14.171959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.330879+0000","last_change":"2023-10-16T10:02:14.090936+0000","last_active":"2023-10-16T10:02:15.330879+0000","last_peered":"2023-10-16T10:02:15.330879+0000","last_clean":"2023-10-16T10:02:15.330879+0000","last_became_active":"2023-10-16T10:02:14.088143+0000","last_became_peered":"2023-10-16T10:02:14.088143+0000","last_unstale":"2023-10-16T10:02:15.330879+0000","last_undegraded":"2023-10-16T10:02:15.330879+0000","last_fullsized":"2023-10-16T10:02:15.330879+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:03:19.543170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331182+0000","last_change":"2023-10-16T10:02:13.070253+0000","last_active":"2023-10-16T10:02:15.331182+0000","last_peered":"2023-10-16T10:02:15.331182+0000","last_clean":"2023-10-16T10:02:15.331182+0000","last_became_active":"2023-10-16T10:02:13.069873+0000","last_became_peered":"2023-10-16T10:02:13.069873+0000","last_unstale":"2023-10-16T10:02:15.331182+0000","last_undegraded":"2023-10-16T10:02:15.331182+0000","last_fullsized":"2023-10-16T10:02:15.331182+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:02:00.842932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.854233+0000","last_change":"2023-10-16T10:02:14.092483+0000","last_active":"2023-10-16T10:02:15.854233+0000","last_peered":"2023-10-16T10:02:15.854233+0000","last_clean":"2023-10-16T10:02:15.854233+0000","last_became_active":"2023-10-16T10:02:14.092178+0000","last_became_peered":"2023-10-16T10:02:14.092178+0000","last_unstale":"2023-10-16T10:02:15.854233+0000","last_undegraded":"2023-10-16T10:02:15.854233+0000","last_fullsized":"2023-10-16T10:02:15.854233+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:18:43.035144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"63'8","reported_seq":20,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.082745+0000","last_change":"2023-10-16T10:02:13.076002+0000","last_active":"2023-10-16T10:02:15.082745+0000","last_peered":"2023-10-16T10:02:15.082745+0000","last_clean":"2023-10-16T10:02:15.082745+0000","last_became_active":"2023-10-16T10:02:13.074652+0000","last_became_peered":"2023-10-16T10:02:13.074652+0000","last_unstale":"2023-10-16T10:02:15.082745+0000","last_undegraded":"2023-10-16T10:02:15.082745+0000","last_fullsized":"2023-10-16T10:02:15.082745+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:39.135820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084043+0000","last_change":"2023-10-16T10:02:14.107028+0000","last_active":"2023-10-16T10:02:15.084043+0000","last_peered":"2023-10-16T10:02:15.084043+0000","last_clean":"2023-10-16T10:02:15.084043+0000","last_became_active":"2023-10-16T10:02:14.106696+0000","last_became_peered":"2023-10-16T10:02:14.106696+0000","last_unstale":"2023-10-16T10:02:15.084043+0000","last_undegraded":"2023-10-16T10:02:15.084043+0000","last_fullsized":"2023-10-16T10:02:15.084043+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:36:50.232508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"63'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084093+0000","last_change":"2023-10-16T10:02:13.066391+0000","last_active":"2023-10-16T10:02:15.084093+0000","last_peered":"2023-10-16T10:02:15.084093+0000","last_clean":"2023-10-16T10:02:15.084093+0000","last_became_active":"2023-10-16T10:02:13.064926+0000","last_became_peered":"2023-10-16T10:02:13.064926+0000","last_unstale":"2023-10-16T10:02:15.084093+0000","last_undegraded":"2023-10-16T10:02:15.084093+0000","last_fullsized":"2023-10-16T10:02:15.084093+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:17:49.516687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084078+0000","last_change":"2023-10-16T10:02:14.111895+0000","last_active":"2023-10-16T10:02:15.084078+0000","last_peered":"2023-10-16T10:02:15.084078+0000","last_clean":"2023-10-16T10:02:15.084078+0000","last_became_active":"2023-10-16T10:02:14.111617+0000","last_became_peered":"2023-10-16T10:02:14.111617+0000","last_unstale":"2023-10-16T10:02:15.084078+0000","last_undegraded":"2023-10-16T10:02:15.084078+0000","last_fullsized":"2023-10-16T10:02:15.084078+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:49.923362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"63'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084408+0000","last_change":"2023-10-16T10:02:13.076569+0000","last_active":"2023-10-16T10:02:15.084408+0000","last_peered":"2023-10-16T10:02:15.084408+0000","last_clean":"2023-10-16T10:02:15.084408+0000","last_became_active":"2023-10-16T10:02:13.076291+0000","last_became_peered":"2023-10-16T10:02:13.076291+0000","last_unstale":"2023-10-16T10:02:15.084408+0000","last_undegraded":"2023-10-16T10:02:15.084408+0000","last_fullsized":"2023-10-16T10:02:15.084408+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:13.792570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.852885+0000","last_change":"2023-10-16T10:02:14.105827+0000","last_active":"2023-10-16T10:02:15.852885+0000","last_peered":"2023-10-16T10:02:15.852885+0000","last_clean":"2023-10-16T10:02:15.852885+0000","last_became_active":"2023-10-16T10:02:14.105408+0000","last_became_peered":"2023-10-16T10:02:14.105408+0000","last_unstale":"2023-10-16T10:02:15.852885+0000","last_undegraded":"2023-10-16T10:02:15.852885+0000","last_fullsized":"2023-10-16T10:02:15.852885+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:48:47.171684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"63'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853850+0000","last_change":"2023-10-16T10:02:13.069640+0000","last_active":"2023-10-16T10:02:15.853850+0000","last_peered":"2023-10-16T10:02:15.853850+0000","last_clean":"2023-10-16T10:02:15.853850+0000","last_became_active":"2023-10-16T10:02:13.069302+0000","last_became_peered":"2023-10-16T10:02:13.069302+0000","last_unstale":"2023-10-16T10:02:15.853850+0000","last_undegraded":"2023-10-16T10:02:15.853850+0000","last_fullsized":"2023-10-16T10:02:15.853850+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:00:23.785232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.852426+0000","last_change":"2023-10-16T10:02:14.105594+0000","last_active":"2023-10-16T10:02:15.852426+0000","last_peered":"2023-10-16T10:02:15.852426+0000","last_clean":"2023-10-16T10:02:15.852426+0000","last_became_active":"2023-10-16T10:02:14.103393+0000","last_became_peered":"2023-10-16T10:02:14.103393+0000","last_unstale":"2023-10-16T10:02:15.852426+0000","last_undegraded":"2023-10-16T10:02:15.852426+0000","last_fullsized":"2023-10-16T10:02:15.852426+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:20.173125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331644+0000","last_change":"2023-10-16T10:02:13.082207+0000","last_active":"2023-10-16T10:02:15.331644+0000","last_peered":"2023-10-16T10:02:15.331644+0000","last_clean":"2023-10-16T10:02:15.331644+0000","last_became_active":"2023-10-16T10:02:13.081937+0000","last_became_peered":"2023-10-16T10:02:13.081937+0000","last_unstale":"2023-10-16T10:02:15.331644+0000","last_undegraded":"2023-10-16T10:02:15.331644+0000","last_fullsized":"2023-10-16T10:02:15.331644+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:12:25.373505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331132+0000","last_change":"2023-10-16T10:02:14.097379+0000","last_active":"2023-10-16T10:02:15.331132+0000","last_peered":"2023-10-16T10:02:15.331132+0000","last_clean":"2023-10-16T10:02:15.331132+0000","last_became_active":"2023-10-16T10:02:14.091543+0000","last_became_peered":"2023-10-16T10:02:14.091543+0000","last_unstale":"2023-10-16T10:02:15.331132+0000","last_undegraded":"2023-10-16T10:02:15.331132+0000","last_fullsized":"2023-10-16T10:02:15.331132+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:56.401413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"63'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331860+0000","last_change":"2023-10-16T10:02:13.074721+0000","last_active":"2023-10-16T10:02:15.331860+0000","last_peered":"2023-10-16T10:02:15.331860+0000","last_clean":"2023-10-16T10:02:15.331860+0000","last_became_active":"2023-10-16T10:02:13.072975+0000","last_became_peered":"2023-10-16T10:02:13.072975+0000","last_unstale":"2023-10-16T10:02:15.331860+0000","last_undegraded":"2023-10-16T10:02:15.331860+0000","last_fullsized":"2023-10-16T10:02:15.331860+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:36.966524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.850378+0000","last_change":"2023-10-16T10:02:14.101271+0000","last_active":"2023-10-16T10:02:15.850378+0000","last_peered":"2023-10-16T10:02:15.850378+0000","last_clean":"2023-10-16T10:02:15.850378+0000","last_became_active":"2023-10-16T10:02:14.101024+0000","last_became_peered":"2023-10-16T10:02:14.101024+0000","last_unstale":"2023-10-16T10:02:15.850378+0000","last_undegraded":"2023-10-16T10:02:15.850378+0000","last_fullsized":"2023-10-16T10:02:15.850378+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:45:26.800557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.852499+0000","last_change":"2023-10-16T10:02:13.082558+0000","last_active":"2023-10-16T10:02:15.852499+0000","last_peered":"2023-10-16T10:02:15.852499+0000","last_clean":"2023-10-16T10:02:15.852499+0000","last_became_active":"2023-10-16T10:02:13.082244+0000","last_became_peered":"2023-10-16T10:02:13.082244+0000","last_unstale":"2023-10-16T10:02:15.852499+0000","last_undegraded":"2023-10-16T10:02:15.852499+0000","last_fullsized":"2023-10-16T10:02:15.852499+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:19:10.604917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.082454+0000","last_change":"2023-10-16T10:02:14.086143+0000","last_active":"2023-10-16T10:02:15.082454+0000","last_peered":"2023-10-16T10:02:15.082454+0000","last_clean":"2023-10-16T10:02:15.082454+0000","last_became_active":"2023-10-16T10:02:14.085853+0000","last_became_peered":"2023-10-16T10:02:14.085853+0000","last_unstale":"2023-10-16T10:02:15.082454+0000","last_undegraded":"2023-10-16T10:02:15.082454+0000","last_fullsized":"2023-10-16T10:02:15.082454+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:56.378112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083186+0000","last_change":"2023-10-16T10:02:13.079440+0000","last_active":"2023-10-16T10:02:15.083186+0000","last_peered":"2023-10-16T10:02:15.083186+0000","last_clean":"2023-10-16T10:02:15.083186+0000","last_became_active":"2023-10-16T10:02:13.078808+0000","last_became_peered":"2023-10-16T10:02:13.078808+0000","last_unstale":"2023-10-16T10:02:15.083186+0000","last_undegraded":"2023-10-16T10:02:15.083186+0000","last_fullsized":"2023-10-16T10:02:15.083186+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:53:51.639925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083099+0000","last_change":"2023-10-16T10:02:14.098831+0000","last_active":"2023-10-16T10:02:15.083099+0000","last_peered":"2023-10-16T10:02:15.083099+0000","last_clean":"2023-10-16T10:02:15.083099+0000","last_became_active":"2023-10-16T10:02:14.094053+0000","last_became_peered":"2023-10-16T10:02:14.094053+0000","last_unstale":"2023-10-16T10:02:15.083099+0000","last_undegraded":"2023-10-16T10:02:15.083099+0000","last_fullsized":"2023-10-16T10:02:15.083099+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:21.047154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.851076+0000","last_change":"2023-10-16T10:02:13.079573+0000","last_active":"2023-10-16T10:02:15.851076+0000","last_peered":"2023-10-16T10:02:15.851076+0000","last_clean":"2023-10-16T10:02:15.851076+0000","last_became_active":"2023-10-16T10:02:13.079008+0000","last_became_peered":"2023-10-16T10:02:13.079008+0000","last_unstale":"2023-10-16T10:02:15.851076+0000","last_undegraded":"2023-10-16T10:02:15.851076+0000","last_fullsized":"2023-10-16T10:02:15.851076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:36.966524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.853982+0000","last_change":"2023-10-16T10:02:14.099831+0000","last_active":"2023-10-16T10:02:15.853982+0000","last_peered":"2023-10-16T10:02:15.853982+0000","last_clean":"2023-10-16T10:02:15.853982+0000","last_became_active":"2023-10-16T10:02:14.098418+0000","last_became_peered":"2023-10-16T10:02:14.098418+0000","last_unstale":"2023-10-16T10:02:15.853982+0000","last_undegraded":"2023-10-16T10:02:15.853982+0000","last_fullsized":"2023-10-16T10:02:15.853982+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:41:24.644785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331646+0000","last_change":"2023-10-16T10:02:13.082763+0000","last_active":"2023-10-16T10:02:15.331646+0000","last_peered":"2023-10-16T10:02:15.331646+0000","last_clean":"2023-10-16T10:02:15.331646+0000","last_became_active":"2023-10-16T10:02:13.082494+0000","last_became_peered":"2023-10-16T10:02:13.082494+0000","last_unstale":"2023-10-16T10:02:15.331646+0000","last_undegraded":"2023-10-16T10:02:15.331646+0000","last_fullsized":"2023-10-16T10:02:15.331646+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:02:32.975638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084106+0000","last_change":"2023-10-16T10:02:14.095123+0000","last_active":"2023-10-16T10:02:15.084106+0000","last_peered":"2023-10-16T10:02:15.084106+0000","last_clean":"2023-10-16T10:02:15.084106+0000","last_became_active":"2023-10-16T10:02:14.094853+0000","last_became_peered":"2023-10-16T10:02:14.094853+0000","last_unstale":"2023-10-16T10:02:15.084106+0000","last_undegraded":"2023-10-16T10:02:15.084106+0000","last_fullsized":"2023-10-16T10:02:15.084106+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:11:13.894974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.854619+0000","last_change":"2023-10-16T10:02:13.086282+0000","last_active":"2023-10-16T10:02:15.854619+0000","last_peered":"2023-10-16T10:02:15.854619+0000","last_clean":"2023-10-16T10:02:15.854619+0000","last_became_active":"2023-10-16T10:02:13.085939+0000","last_became_peered":"2023-10-16T10:02:13.085939+0000","last_unstale":"2023-10-16T10:02:15.854619+0000","last_undegraded":"2023-10-16T10:02:15.854619+0000","last_fullsized":"2023-10-16T10:02:15.854619+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:09.274588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.854377+0000","last_change":"2023-10-16T10:02:14.097225+0000","last_active":"2023-10-16T10:02:15.854377+0000","last_peered":"2023-10-16T10:02:15.854377+0000","last_clean":"2023-10-16T10:02:15.854377+0000","last_became_active":"2023-10-16T10:02:14.095692+0000","last_became_peered":"2023-10-16T10:02:14.095692+0000","last_unstale":"2023-10-16T10:02:15.854377+0000","last_undegraded":"2023-10-16T10:02:15.854377+0000","last_fullsized":"2023-10-16T10:02:15.854377+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:32:10.613842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.850122+0000","last_change":"2023-10-16T10:02:13.073665+0000","last_active":"2023-10-16T10:02:15.850122+0000","last_peered":"2023-10-16T10:02:15.850122+0000","last_clean":"2023-10-16T10:02:15.850122+0000","last_became_active":"2023-10-16T10:02:13.073426+0000","last_became_peered":"2023-10-16T10:02:13.073426+0000","last_unstale":"2023-10-16T10:02:15.850122+0000","last_undegraded":"2023-10-16T10:02:15.850122+0000","last_fullsized":"2023-10-16T10:02:15.850122+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:47:57.299735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331033+0000","last_change":"2023-10-16T10:02:14.099071+0000","last_active":"2023-10-16T10:02:15.331033+0000","last_peered":"2023-10-16T10:02:15.331033+0000","last_clean":"2023-10-16T10:02:15.331033+0000","last_became_active":"2023-10-16T10:02:14.098635+0000","last_became_peered":"2023-10-16T10:02:14.098635+0000","last_unstale":"2023-10-16T10:02:15.331033+0000","last_undegraded":"2023-10-16T10:02:15.331033+0000","last_fullsized":"2023-10-16T10:02:15.331033+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:09:36.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.855048+0000","last_change":"2023-10-16T10:02:13.084852+0000","last_active":"2023-10-16T10:02:15.855048+0000","last_peered":"2023-10-16T10:02:15.855048+0000","last_clean":"2023-10-16T10:02:15.855048+0000","last_became_active":"2023-10-16T10:02:13.084521+0000","last_became_peered":"2023-10-16T10:02:13.084521+0000","last_unstale":"2023-10-16T10:02:15.855048+0000","last_undegraded":"2023-10-16T10:02:15.855048+0000","last_fullsized":"2023-10-16T10:02:15.855048+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:02:32.975638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.331469+0000","last_change":"2023-10-16T10:02:14.095368+0000","last_active":"2023-10-16T10:02:15.331469+0000","last_peered":"2023-10-16T10:02:15.331469+0000","last_clean":"2023-10-16T10:02:15.331469+0000","last_became_active":"2023-10-16T10:02:14.095095+0000","last_became_peered":"2023-10-16T10:02:14.095095+0000","last_unstale":"2023-10-16T10:02:15.331469+0000","last_undegraded":"2023-10-16T10:02:15.331469+0000","last_fullsized":"2023-10-16T10:02:15.331469+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:32:27.250633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083939+0000","last_change":"2023-10-16T10:02:13.071942+0000","last_active":"2023-10-16T10:02:15.083939+0000","last_peered":"2023-10-16T10:02:15.083939+0000","last_clean":"2023-10-16T10:02:15.083939+0000","last_became_active":"2023-10-16T10:02:13.071360+0000","last_became_peered":"2023-10-16T10:02:13.071360+0000","last_unstale":"2023-10-16T10:02:15.083939+0000","last_undegraded":"2023-10-16T10:02:15.083939+0000","last_fullsized":"2023-10-16T10:02:15.083939+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:16.310641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.854243+0000","last_change":"2023-10-16T10:02:14.105043+0000","last_active":"2023-10-16T10:02:15.854243+0000","last_peered":"2023-10-16T10:02:15.854243+0000","last_clean":"2023-10-16T10:02:15.854243+0000","last_became_active":"2023-10-16T10:02:14.104762+0000","last_became_peered":"2023-10-16T10:02:14.104762+0000","last_unstale":"2023-10-16T10:02:15.854243+0000","last_undegraded":"2023-10-16T10:02:15.854243+0000","last_fullsized":"2023-10-16T10:02:15.854243+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:14.957159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.850105+0000","last_change":"2023-10-16T10:02:13.080847+0000","last_active":"2023-10-16T10:02:15.850105+0000","last_peered":"2023-10-16T10:02:15.850105+0000","last_clean":"2023-10-16T10:02:15.850105+0000","last_became_active":"2023-10-16T10:02:13.080561+0000","last_became_peered":"2023-10-16T10:02:13.080561+0000","last_unstale":"2023-10-16T10:02:15.850105+0000","last_undegraded":"2023-10-16T10:02:15.850105+0000","last_fullsized":"2023-10-16T10:02:15.850105+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:31:13.381358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.083753+0000","last_change":"2023-10-16T10:02:14.093397+0000","last_active":"2023-10-16T10:02:15.083753+0000","last_peered":"2023-10-16T10:02:15.083753+0000","last_clean":"2023-10-16T10:02:15.083753+0000","last_became_active":"2023-10-16T10:02:14.093140+0000","last_became_peered":"2023-10-16T10:02:14.093140+0000","last_unstale":"2023-10-16T10:02:15.083753+0000","last_undegraded":"2023-10-16T10:02:15.083753+0000","last_fullsized":"2023-10-16T10:02:15.083753+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:41.123730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.082828+0000","last_change":"2023-10-16T10:02:13.067760+0000","last_active":"2023-10-16T10:02:15.082828+0000","last_peered":"2023-10-16T10:02:15.082828+0000","last_clean":"2023-10-16T10:02:15.082828+0000","last_became_active":"2023-10-16T10:02:13.067505+0000","last_became_peered":"2023-10-16T10:02:13.067505+0000","last_unstale":"2023-10-16T10:02:15.082828+0000","last_undegraded":"2023-10-16T10:02:15.082828+0000","last_fullsized":"2023-10-16T10:02:15.082828+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:06:46.286774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.084737+0000","last_change":"2023-10-16T10:02:14.093589+0000","last_active":"2023-10-16T10:02:15.084737+0000","last_peered":"2023-10-16T10:02:15.084737+0000","last_clean":"2023-10-16T10:02:15.084737+0000","last_became_active":"2023-10-16T10:02:14.093264+0000","last_became_peered":"2023-10-16T10:02:14.093264+0000","last_unstale":"2023-10-16T10:02:15.084737+0000","last_undegraded":"2023-10-16T10:02:15.084737+0000","last_fullsized":"2023-10-16T10:02:15.084737+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:13.036354+0000","last_clean_scrub_stamp":"2023-10-16T10:02:13.036354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:47.585878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"63'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-16T10:02:15.855042+0000","last_change":"2023-10-16T10:02:13.068582+0000","last_active":"2023-10-16T10:02:15.855042+0000","last_peered":"2023-10-16T10:02:15.855042+0000","last_clean":"2023-10-16T10:02:15.855042+0000","last_became_active":"2023-10-16T10:02:13.068269+0000","last_became_peered":"2023-10-16T10:02:13.068269+0000","last_unstale":"2023-10-16T10:02:15.855042+0000","last_undegraded":"2023-10-16T10:02:15.855042+0000","last_fullsized":"2023-10-16T10:02:15.855042+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:12.023566+0000","last_clean_scrub_stamp":"2023-10-16T10:02:12.023566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:28.986842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"51'139","reported_seq":288,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-16T10:02:14.070889+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T10:02:14.070889+0000","last_peered":"2023-10-16T10:02:14.070889+0000","last_clean":"2023-10-16T10:02:14.070889+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T10:02:14.070889+0000","last_undegraded":"2023-10-16T10:02:14.070889+0000","last_fullsized":"2023-10-16T10:02:14.070889+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575010,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36656,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335184,"statfs":{"total":96636764160,"available":96599228416,"internally_reserved":0,"allocated":1347584,"data_stored":941803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.1890000000000001,"15min":1.153},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.67200000000000004},"max":{"1min":1.853,"5min":1.925,"15min":1.998},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.2190000000000001,"15min":1.238},"min":{"1min":0.97699999999999998,"5min":0.84299999999999997,"15min":0.68100000000000005},"max":{"1min":1.75,"5min":2.016,"15min":3.694},"last":0.90000000000000002}]},{"osd":1,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2609999999999999,"15min":1.2250000000000001},"min":{"1min":0.94899999999999995,"5min":0.876,"15min":0.747},"max":{"1min":1.534,"5min":1.766,"15min":1.766},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2569999999999999,"15min":1.212},"min":{"1min":0.91400000000000003,"5min":0.79000000000000004,"15min":0.69399999999999995},"max":{"1min":1.595,"5min":1.704,"15min":1.7410000000000001},"last":1.512}]},{"osd":2,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2989999999999999,"15min":1.2689999999999999},"min":{"1min":0.83599999999999997,"5min":0.81100000000000005,"15min":0.63},"max":{"1min":1.607,"5min":1.8740000000000001,"15min":1.885},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.2989999999999999,"15min":1.325},"min":{"1min":1.0620000000000001,"5min":0.89500000000000002,"15min":0.70799999999999996},"max":{"1min":1.6799999999999999,"5min":1.8620000000000001,"15min":3.7759999999999998},"last":1.407}]},{"osd":3,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3580000000000001,"15min":1.3480000000000001},"min":{"1min":1.0289999999999999,"5min":0.85099999999999998,"15min":0.82799999999999996},"max":{"1min":1.726,"5min":1.996,"15min":1.996},"last":1.577},{"interface":"front","average":{"1min":1.397,"5min":1.409,"15min":1.3899999999999999},"min":{"1min":1.069,"5min":0.75900000000000001,"15min":0.66900000000000004},"max":{"1min":1.837,"5min":2.004,"15min":2.141},"last":1.361}]},{"osd":4,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.369,"15min":1.3959999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0310000000000001},"last":1.45},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3160000000000001,"15min":1.339},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.84199999999999997},"max":{"1min":1.613,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.2869999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.5309999999999999,"15min":1.544},"min":{"1min":0.93200000000000005,"5min":0.91100000000000003,"15min":0.872},"max":{"1min":2.121,"5min":2.165,"15min":2.4510000000000001},"last":1.609},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4379999999999999,"15min":1.472},"min":{"1min":1.2290000000000001,"5min":0.99099999999999999,"15min":0.94099999999999995},"max":{"1min":1.7350000000000001,"5min":1.9990000000000001,"15min":2.3210000000000002},"last":1.5389999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3899999999999999,"15min":1.454},"min":{"1min":0.94499999999999995,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.6859999999999999,"5min":1.946,"15min":2.6259999999999999},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.621,"5min":1.6299999999999999,"15min":1.6439999999999999},"min":{"1min":1.032,"5min":1.032,"15min":0.98799999999999999},"max":{"1min":2.3799999999999999,"5min":2.3799999999999999,"15min":2.5670000000000002},"last":1.73}]}]},{"osd":6,"up_from":13,"seq":55834575010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36092,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335748,"statfs":{"total":96636764160,"available":96599805952,"internally_reserved":0,"allocated":770048,"data_stored":358317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.1160000000000001,"15min":1.1539999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.667,"5min":1.6910000000000001,"15min":2.04},"last":1.389},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.21,"15min":1.242},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.624},"max":{"1min":1.847,"5min":1.847,"15min":2.1000000000000001},"last":1.55}]},{"osd":1,"last update":"Mon Oct 16 10:02:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.161,"15min":1.1890000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.423,"5min":1.6919999999999999,"15min":1.9259999999999999},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.288,"15min":1.3089999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.734,"5min":1.734,"15min":1.873},"last":1.45}]},{"osd":2,"last update":"Mon Oct 16 10:02:12 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.55,"5min":2.036,"15min":2.036},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2849999999999999,"15min":1.339},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":2.1720000000000002},"last":1.514}]},{"osd":3,"last update":"Mon Oct 16 10:02:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2190000000000001,"15min":1.258},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.8239999999999998,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.335},{"interface":"front","average":{"1min":1.179,"5min":1.23,"15min":1.248},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.619,"5min":1.825,"15min":2.0640000000000001},"last":1.2969999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:02:12 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.23,"15min":1.3440000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.4540000000000002},"last":1.236},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4430000000000001,"15min":1.49},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.6880000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.1739999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:02:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.337,"15min":1.361},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.3090000000000002,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.454,"5min":1.492,"15min":1.5529999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.5289999999999999},"last":1.5800000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:02:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.4910000000000001,"15min":1.532},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":3.5739999999999998,"5min":3.5739999999999998,"15min":3.5739999999999998},"last":1.6399999999999999},{"interface":"front","average":{"1min":1.327,"5min":1.4750000000000001,"15min":1.5389999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.9790000000000001,"5min":2.4249999999999998,"15min":3.5419999999999998},"last":1.488}]}]},{"osd":5,"up_from":13,"seq":55834575010,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36084,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":753664,"data_stored":351415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.2390000000000001,"15min":1.236},"min":{"1min":0.81499999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.4770000000000001,"5min":1.675,"15min":2.1019999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":1.329,"5min":1.3320000000000001,"15min":1.3169999999999999},"min":{"1min":1.0289999999999999,"5min":0.89800000000000002,"15min":0.77200000000000002},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.915},"last":0.84199999999999997}]},{"osd":1,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.1919999999999999,"15min":1.149},"min":{"1min":0.83999999999999997,"5min":0.72799999999999998,"15min":0.68799999999999994},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":2.4569999999999999},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.23,"5min":1.1799999999999999,"15min":1.1879999999999999},"min":{"1min":0.95299999999999996,"5min":0.82499999999999996,"15min":0.78600000000000003},"max":{"1min":1.5700000000000001,"5min":1.6970000000000001,"15min":2.2410000000000001},"last":0.91400000000000003}]},{"osd":2,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.163,"15min":1.1619999999999999},"min":{"1min":0.871,"5min":0.80900000000000005,"15min":0.71299999999999997},"max":{"1min":1.639,"5min":1.639,"15min":1.77},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.264,"15min":1.2669999999999999},"min":{"1min":0.94699999999999995,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.6990000000000001,"5min":1.714,"15min":1.7270000000000001},"last":1.4950000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.319,"15min":1.3049999999999999},"min":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.73799999999999999},"max":{"1min":1.917,"5min":1.97,"15min":1.97},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.403,"5min":1.389,"15min":1.3939999999999999},"min":{"1min":1.095,"5min":0.874,"15min":0.874},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":2.1160000000000001},"last":0.97599999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2729999999999999,"15min":1.3560000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.52100000000000002},"max":{"1min":1.6359999999999999,"5min":1.7989999999999999,"15min":2.355},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.478,"5min":1.5189999999999999,"15min":1.5569999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.63600000000000001},"max":{"1min":2.0819999999999999,"5min":2.0920000000000001,"15min":2.4079999999999999},"last":1.4419999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3640000000000001,"15min":1.4179999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.55500000000000005},"max":{"1min":1.847,"5min":1.847,"15min":2.3580000000000001},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.403,"15min":1.478},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.58399999999999996},"max":{"1min":1.7150000000000001,"5min":1.889,"15min":2.4159999999999999},"last":1.359}]},{"osd":7,"last update":"Mon Oct 16 10:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.506,"15min":1.5820000000000001},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.78700000000000003},"max":{"1min":1.9970000000000001,"5min":2.1859999999999999,"15min":2.4809999999999999},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.587,"15min":1.6479999999999999},"min":{"1min":1.0640000000000001,"5min":0.99199999999999999,"15min":0.75600000000000001},"max":{"1min":1.917,"5min":1.9179999999999999,"15min":2.931},"last":1.5720000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575011,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62468,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309372,"statfs":{"total":96636764160,"available":96572796928,"internally_reserved":0,"allocated":774144,"data_stored":359257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.171,"15min":1.1930000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.76,"5min":1.76,"15min":4.2640000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.246,"15min":1.228},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.64600000000000002},"max":{"1min":4.915,"5min":4.915,"15min":4.915},"last":11.974}]},{"osd":1,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.2849999999999999,"15min":1.248},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.66000000000000003},"max":{"1min":4.758,"5min":4.758,"15min":4.758},"last":9.9939999999999998},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.232,"15min":1.2310000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.624},"max":{"1min":1.6060000000000001,"5min":1.712,"15min":2.0259999999999998},"last":5.9420000000000002}]},{"osd":2,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.238,"15min":1.25},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.528,"5min":1.756,"15min":2.3130000000000002},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.387,"15min":1.3779999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.70799999999999996},"max":{"1min":5.0549999999999997,"5min":5.0549999999999997,"15min":5.0549999999999997},"last":3.673}]},{"osd":3,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.323},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.6800000000000002,"5min":2.6800000000000002,"15min":3.262},"last":6.8079999999999998},{"interface":"front","average":{"1min":1.292,"5min":1.387,"15min":1.429},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.7729999999999999,"5min":1.923,"15min":3.3050000000000002},"last":1.4450000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.4370000000000001,"15min":1.415},"min":{"1min":0.871,"5min":0.752,"15min":0.752},"max":{"1min":5.8940000000000001,"5min":5.8940000000000001,"15min":5.8940000000000001},"last":8.702},{"interface":"front","average":{"1min":1.474,"5min":1.4019999999999999,"15min":1.4179999999999999},"min":{"1min":1,"5min":0.91600000000000004,"15min":0.69099999999999995},"max":{"1min":3.0430000000000001,"5min":3.0430000000000001,"15min":3.669},"last":1.1000000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3959999999999999,"15min":1.4390000000000001},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.56599999999999995},"max":{"1min":2.5,"5min":2.5,"15min":3.8090000000000002},"last":8.0579999999999998},{"interface":"front","average":{"1min":1.6899999999999999,"5min":1.569,"15min":1.5600000000000001},"min":{"1min":1.0029999999999999,"5min":0.91500000000000004,"15min":0.70299999999999996},"max":{"1min":6.4089999999999998,"5min":6.4089999999999998,"15min":6.4089999999999998},"last":4.625}]},{"osd":7,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.506,"15min":1.5289999999999999},"min":{"1min":1.0960000000000001,"5min":0.98499999999999999,"15min":0.60999999999999999},"max":{"1min":2.9399999999999999,"5min":2.9399999999999999,"15min":3.5150000000000001},"last":2.4060000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5860000000000001,"15min":1.625},"min":{"1min":1.2170000000000001,"5min":0.98399999999999999,"15min":0.89600000000000002},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":3.9849999999999999},"last":1.48}]}]},{"osd":3,"up_from":13,"seq":55834575010,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36092,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335748,"statfs":{"total":96636764160,"available":96599805952,"internally_reserved":0,"allocated":761856,"data_stored":351983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.88500000000000001,"15min":0.86499999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.40699999999999997},"max":{"1min":1.208,"5min":1.3360000000000001,"15min":1.462},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.006,"5min":1.0629999999999999,"15min":1.008},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.29999999999999999},"max":{"1min":1.385,"5min":1.708,"15min":1.708},"last":1.002}]},{"osd":1,"last update":"Mon Oct 16 10:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.97799999999999998,"15min":0.94599999999999995},"min":{"1min":0.73899999999999999,"5min":0.66400000000000003,"15min":0.46999999999999997},"max":{"1min":1.2909999999999999,"5min":1.458,"15min":1.7150000000000001},"last":1.105},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.105,"15min":1.0429999999999999},"min":{"1min":0.83199999999999996,"5min":0.76700000000000002,"15min":0.39600000000000002},"max":{"1min":1.603,"5min":1.643,"15min":1.643},"last":1.139}]},{"osd":2,"last update":"Mon Oct 16 10:01:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1120000000000001,"15min":1.077},"min":{"1min":0.89600000000000002,"5min":0.81999999999999995,"15min":0.439},"max":{"1min":1.4039999999999999,"5min":1.4590000000000001,"15min":1.8640000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2150000000000001,"15min":1.1579999999999999},"min":{"1min":0.93700000000000006,"5min":0.79000000000000004,"15min":0.51900000000000002},"max":{"1min":1.7170000000000001,"5min":1.776,"15min":2.028},"last":1.546}]},{"osd":4,"last update":"Mon Oct 16 10:01:19 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.3029999999999999,"15min":1.2949999999999999},"min":{"1min":1.0029999999999999,"5min":0.96499999999999997,"15min":0.755},"max":{"1min":1.619,"5min":1.649,"15min":2.1619999999999999},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.337,"15min":1.3320000000000001},"min":{"1min":1.097,"5min":1.042,"15min":0.73599999999999999},"max":{"1min":1.5720000000000001,"5min":1.766,"15min":2.532},"last":1.804}]},{"osd":5,"last update":"Mon Oct 16 10:01:19 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.355,"15min":1.339},"min":{"1min":1.042,"5min":1.042,"15min":0.76900000000000002},"max":{"1min":1.6120000000000001,"5min":1.7989999999999999,"15min":2.0019999999999998},"last":1.333},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3600000000000001,"15min":1.3700000000000001},"min":{"1min":1.03,"5min":1.016,"15min":0.76700000000000002},"max":{"1min":1.573,"5min":1.855,"15min":2.0419999999999998},"last":1.5740000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:01:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.468,"15min":1.456},"min":{"1min":1.103,"5min":0.98699999999999999,"15min":0.81599999999999995},"max":{"1min":1.7110000000000001,"5min":2.0819999999999999,"15min":2.3029999999999999},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4710000000000001,"15min":1.472},"min":{"1min":1.179,"5min":1.1060000000000001,"15min":0.80300000000000005},"max":{"1min":1.681,"5min":1.976,"15min":2.1240000000000001},"last":1.4710000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:01:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.45,"15min":1.4910000000000001},"min":{"1min":1.137,"5min":1.0569999999999999,"15min":0.75},"max":{"1min":1.6830000000000001,"5min":1.9430000000000001,"15min":2.403},"last":1.992},{"interface":"front","average":{"1min":1.542,"5min":1.579,"15min":1.6040000000000001},"min":{"1min":1.3560000000000001,"5min":1.282,"15min":0.92400000000000004},"max":{"1min":1.7529999999999999,"5min":2.157,"15min":2.6099999999999999},"last":2.2429999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575009,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40268,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331572,"statfs":{"total":96636764160,"available":96595529728,"internally_reserved":0,"allocated":778240,"data_stored":359279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86799999999999999,"15min":0.85099999999999998},"min":{"1min":0.69199999999999995,"5min":0.55500000000000005,"15min":0.28999999999999998},"max":{"1min":1.5469999999999999,"5min":1.6679999999999999,"15min":2.8690000000000002},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.115,"15min":1.02},"min":{"1min":0.78800000000000003,"5min":0.41199999999999998,"15min":0.28899999999999998},"max":{"1min":1.244,"5min":1.629,"15min":1.841},"last":0.93700000000000006}]},{"osd":1,"last update":"Mon Oct 16 10:01:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.087,"15min":1.034},"min":{"1min":0.81699999999999995,"5min":0.628,"15min":0.42899999999999999},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":2.6059999999999999},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.1519999999999999,"15min":1.115},"min":{"1min":0.86499999999999999,"5min":0.70899999999999996,"15min":0.40699999999999997},"max":{"1min":1.5669999999999999,"5min":1.712,"15min":2.6499999999999999},"last":1.3160000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:01:24 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0629999999999999,"15min":0.98099999999999998},"min":{"1min":0.80000000000000004,"5min":0.67100000000000004,"15min":0.38},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":2.5339999999999998},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2030000000000001,"15min":1.1919999999999999},"min":{"1min":0.92300000000000004,"5min":0.69099999999999995,"15min":0.48499999999999999},"max":{"1min":1.7150000000000001,"5min":1.7569999999999999,"15min":2.9700000000000002},"last":1.3899999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:01:24 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.371,"15min":1.3540000000000001},"min":{"1min":1.0860000000000001,"5min":0.84899999999999998,"15min":0.65000000000000002},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":2.2309999999999999},"last":1.546},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.3480000000000001,"15min":1.2929999999999999},"min":{"1min":1.1850000000000001,"5min":0.90200000000000002,"15min":0.77400000000000002},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":2.2000000000000002},"last":1.028}]},{"osd":5,"last update":"Mon Oct 16 10:01:24 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.403,"15min":1.3899999999999999},"min":{"1min":1.0349999999999999,"5min":0.81999999999999995,"15min":0.63300000000000001},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.7770000000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4419999999999999,"15min":1.4390000000000001},"min":{"1min":1.238,"5min":0.76600000000000001,"15min":0.68300000000000005},"max":{"1min":1.956,"5min":1.956,"15min":2.952},"last":1.6419999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:01:24 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.3799999999999999,"15min":1.3180000000000001},"min":{"1min":1.036,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.2469999999999999},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.454,"15min":1.431},"min":{"1min":1.2190000000000001,"5min":0.878,"15min":0.70499999999999996},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":2.8180000000000001},"last":1.577}]},{"osd":7,"last update":"Mon Oct 16 10:01:24 2023","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.5169999999999999,"15min":1.502},"min":{"1min":1.1379999999999999,"5min":0.96899999999999997,"15min":0.72999999999999998},"max":{"1min":2.0310000000000001,"5min":2.262,"15min":3.746},"last":1.754},{"interface":"front","average":{"1min":1.516,"5min":1.48,"15min":1.4279999999999999},"min":{"1min":1.226,"5min":1.091,"15min":0.79900000000000004},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3769999999999998},"last":1.151}]}]},{"osd":1,"up_from":13,"seq":55834575009,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49960,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321880,"statfs":{"total":96636764160,"available":96585605120,"internally_reserved":0,"allocated":753664,"data_stored":351415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.86699999999999999,"15min":0.88300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.137,"5min":1.5609999999999999,"15min":2.2010000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.98299999999999998,"15min":0.99299999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.45300000000000001},"max":{"1min":1.242,"5min":1.665,"15min":2.1619999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.999,"15min":1.0049999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.42299999999999999},"max":{"1min":1.526,"5min":1.526,"15min":2.3849999999999998},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.2430000000000001,"15min":1.238},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.50800000000000001},"max":{"1min":1.6419999999999999,"5min":1.667,"15min":2.2959999999999998},"last":1.4490000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.145,"15min":1.1459999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.532,"5min":1.665,"15min":2.254},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.89400000000000002,"5min":1.0980000000000001,"15min":1.1040000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.36099999999999999},"max":{"1min":1.3380000000000001,"5min":1.8280000000000001,"15min":2.8109999999999999},"last":1.1830000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.22,"15min":1.254},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7010000000000001,"5min":1.7769999999999999,"15min":2.4540000000000002},"last":1.105},{"interface":"front","average":{"1min":1.123,"5min":1.343,"15min":1.4059999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.624},"max":{"1min":1.4890000000000001,"5min":1.7929999999999999,"15min":2.851},"last":1.365}]},{"osd":5,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.335,"15min":1.3580000000000001},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.78300000000000003},"max":{"1min":1.5780000000000001,"5min":1.8240000000000001,"15min":2.7120000000000002},"last":1.131},{"interface":"front","average":{"1min":1.159,"5min":1.3280000000000001,"15min":1.3460000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.5740000000000001,"5min":2.0470000000000002,"15min":2.512},"last":1.397}]},{"osd":6,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.413,"15min":1.4279999999999999},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.72099999999999997},"max":{"1min":1.744,"5min":1.8979999999999999,"15min":2.75},"last":1.226},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.3939999999999999,"15min":1.419},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.748},"max":{"1min":1.591,"5min":2.1680000000000001,"15min":2.669},"last":1.24}]},{"osd":7,"last update":"Mon Oct 16 10:02:13 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.4710000000000001,"15min":1.5149999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.7370000000000001,"5min":1.885,"15min":2.8799999999999999},"last":1.486},{"interface":"front","average":{"1min":1.365,"5min":1.504,"15min":1.5189999999999999},"min":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.86699999999999999},"max":{"1min":1.8380000000000001,"5min":1.9470000000000001,"15min":3.1400000000000001},"last":1.3100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575010,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1359872,"data_stored":948047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83099999999999996,"15min":0.84199999999999997},"min":{"1min":0.68000000000000005,"5min":0.61299999999999999,"15min":0.376},"max":{"1min":0.97699999999999998,"5min":1.546,"15min":1.6419999999999999},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.046,"5min":1.0740000000000001,"15min":1.0109999999999999},"min":{"1min":0.72399999999999998,"5min":0.69099999999999995,"15min":0.41799999999999998},"max":{"1min":1.3819999999999999,"5min":1.5369999999999999,"15min":1.8500000000000001},"last":0.88100000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.026,"15min":1.0309999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.38900000000000001},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":2.2010000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.093,"5min":1.0880000000000001,"15min":1.0780000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.46600000000000003},"max":{"1min":1.5620000000000001,"5min":1.726,"15min":1.8959999999999999},"last":0.97499999999999998}]},{"osd":3,"last update":"Mon Oct 16 10:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.0569999999999999,"15min":1.01},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.534,"5min":1.534,"15min":1.6140000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.2110000000000001,"15min":1.181},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.47599999999999998},"max":{"1min":1.5189999999999999,"5min":1.7589999999999999,"15min":2.6110000000000002},"last":1.1459999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3180000000000001},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.77800000000000002},"max":{"1min":1.669,"5min":1.946,"15min":1.9930000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3069999999999999,"15min":1.28},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.5660000000000001,"5min":1.6040000000000001,"15min":2.0499999999999998},"last":1.0269999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3240000000000001,"15min":1.3140000000000001},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.77500000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":2.8159999999999998},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.359,"5min":1.3959999999999999,"15min":1.4079999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.7270000000000001,"5min":2.069,"15min":2.9009999999999998},"last":1.3979999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.409,"15min":1.4219999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.714,"5min":2.145,"15min":2.7160000000000002},"last":1.345},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.4570000000000001,"15min":1.4119999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.84999999999999998},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":2.056},"last":1.099}]},{"osd":7,"last update":"Mon Oct 16 10:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":1.492,"15min":1.452},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.81499999999999995},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.466,"15min":1.421},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.0390000000000001,"5min":2.157,"15min":2.157},"last":1.0529999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:02:18.999 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T10:02:19.000 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T10:02:19.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.261+0000 7fe8df163640 1 Processor -- start 2023-10-16T10:02:19.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.261+0000 7fe8df163640 1 -- start start 2023-10-16T10:02:19.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.262+0000 7fe8df163640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8d8159ac0 0x7fe8d815e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.263+0000 7fe8df163640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe8d80704e0 0x7fe8d8070970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.263+0000 7fe8df163640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d815e5c0 con 0x7fe8d80cc270 2023-10-16T10:02:19.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.263+0000 7fe8df163640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d8070eb0 con 0x7fe8d80704e0 2023-10-16T10:02:19.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.263+0000 7fe8df163640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d806b810 con 0x7fe8d8159ac0 2023-10-16T10:02:19.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.263+0000 7fe8dd960640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe8d80cc270 0x7fe8d80c8cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34218/0 (socket says 172.21.15.84:34218) 2023-10-16T10:02:19.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.263+0000 7fe8dd960640 1 -- 172.21.15.84:0/323479882 learned_addr learned my addr 172.21.15.84:0/323479882 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:19.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.263+0000 7fe8de161640 1 --2- 172.21.15.84:0/323479882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8d8159ac0 0x7fe8d815e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.263+0000 7fe8de962640 1 --2- 172.21.15.84:0/323479882 >> v2:172.21.15.157:3300/0 conn(0x7fe8d80704e0 0x7fe8d8070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.263+0000 7fe8dd15f640 1 -- 172.21.15.84:0/323479882 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3280496900 0 0) 0x7fe8c8001110 con 0x7fe8d80cc270 2023-10-16T10:02:19.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.263+0000 7fe8dd15f640 1 -- 172.21.15.84:0/323479882 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8c4003520 con 0x7fe8d80cc270 2023-10-16T10:02:19.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.264+0000 7fe8dd15f640 1 -- 172.21.15.84:0/323479882 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1711317207 0 0) 0x7fe8c4003520 con 0x7fe8d80cc270 2023-10-16T10:02:19.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.264+0000 7fe8dd15f640 1 -- 172.21.15.84:0/323479882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8d8159ac0 msgr2=0x7fe8d815e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:19.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.264+0000 7fe8dd15f640 1 --2- 172.21.15.84:0/323479882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8d8159ac0 0x7fe8d815e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.264+0000 7fe8dd15f640 1 -- 172.21.15.84:0/323479882 >> v2:172.21.15.157:3300/0 conn(0x7fe8d80704e0 msgr2=0x7fe8d8070970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:19.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.264+0000 7fe8dd15f640 1 --2- 172.21.15.84:0/323479882 >> v2:172.21.15.157:3300/0 conn(0x7fe8d80704e0 0x7fe8d8070970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.264+0000 7fe8dd15f640 1 -- 172.21.15.84:0/323479882 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d80cc810 con 0x7fe8d80cc270 2023-10-16T10:02:19.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.264+0000 7fe8dd15f640 1 -- 172.21.15.84:0/323479882 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fe8c8001800 con 0x7fe8d80cc270 2023-10-16T10:02:19.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.264+0000 7fe8dd15f640 1 -- 172.21.15.84:0/323479882 >> v1:172.21.15.84:6789/0 conn(0x7fe8d80cc270 legacy=0x7fe8d80c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.264+0000 7fe8dd15f640 1 --2- 172.21.15.84:0/323479882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8d8159ac0 0x7fe8c4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.264+0000 7fe8dd15f640 1 --2- 172.21.15.84:0/323479882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8c4003f40 0x7fe8c40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.264+0000 7fe8dd15f640 1 --2- 172.21.15.84:0/323479882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8c4004d00 0x7fe8c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.264+0000 7fe8dd15f640 1 -- 172.21.15.84:0/323479882 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d8070eb0 con 0x7fe8c4004d00 2023-10-16T10:02:19.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.264+0000 7fe8dd15f640 1 -- 172.21.15.84:0/323479882 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d806b810 con 0x7fe8c4003f40 2023-10-16T10:02:19.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.264+0000 7fe8dd15f640 1 -- 172.21.15.84:0/323479882 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c400b980 con 0x7fe8d8159ac0 2023-10-16T10:02:19.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.264+0000 7fe8de962640 1 --2- 172.21.15.84:0/323479882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8c4004d00 0x7fe8c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.265+0000 7fe8de161640 1 --2- 172.21.15.84:0/323479882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8d8159ac0 0x7fe8c4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.265+0000 7fe8dd960640 1 --2- 172.21.15.84:0/323479882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8c4003f40 0x7fe8c40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.265+0000 7fe8de962640 1 -- 172.21.15.84:0/323479882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8d8159ac0 msgr2=0x7fe8c4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:19.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.265+0000 7fe8de962640 1 --2- 172.21.15.84:0/323479882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8d8159ac0 0x7fe8c4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.265+0000 7fe8de962640 1 -- 172.21.15.84:0/323479882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8c4003f40 msgr2=0x7fe8c40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:19.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.265+0000 7fe8de962640 1 --2- 172.21.15.84:0/323479882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8c4003f40 0x7fe8c40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.265+0000 7fe8de962640 1 -- 172.21.15.84:0/323479882 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c4003520 con 0x7fe8c4004d00 2023-10-16T10:02:19.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.266+0000 7fe8de962640 1 --2- 172.21.15.84:0/323479882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8c4004d00 0x7fe8c4005180 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0010380 tx=0x7fe8c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f1971ea02ae83a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:19.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.266+0000 7fe8dd15f640 1 -- 172.21.15.84:0/323479882 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d002ee30 con 0x7fe8c4004d00 2023-10-16T10:02:19.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.266+0000 7fe8dd15f640 1 -- 172.21.15.84:0/323479882 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d0011040 con 0x7fe8c4004d00 2023-10-16T10:02:19.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.266+0000 7fe8dd15f640 1 -- 172.21.15.84:0/323479882 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d000b2f0 con 0x7fe8c4004d00 2023-10-16T10:02:19.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.267+0000 7fe8df163640 1 -- 172.21.15.84:0/323479882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8c4004d00 msgr2=0x7fe8c4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.267+0000 7fe8df163640 1 --2- 172.21.15.84:0/323479882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8c4004d00 0x7fe8c4005180 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0010380 tx=0x7fe8c40057f0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.267+0000 7fe8de962640 1 -- 172.21.15.84:0/323479882 reap_dead start 2023-10-16T10:02:19.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.267+0000 7fe8df163640 1 -- 172.21.15.84:0/323479882 shutdown_connections 2023-10-16T10:02:19.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.267+0000 7fe8df163640 1 -- 172.21.15.84:0/323479882 >> 172.21.15.84:0/323479882 conn(0x7fe8d80ce6a0 msgr2=0x7fe8d80ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:19.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.267+0000 7fe8df163640 1 -- 172.21.15.84:0/323479882 shutdown_connections 2023-10-16T10:02:19.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.267+0000 7fe8df163640 1 -- 172.21.15.84:0/323479882 wait complete. 2023-10-16T10:02:19.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.269+0000 7fe8df163640 1 Processor -- start 2023-10-16T10:02:19.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.269+0000 7fe8df163640 1 -- start start 2023-10-16T10:02:19.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.270+0000 7fe8df163640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8c4004d00 0x7fe8d8147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.270+0000 7fe8df163640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8d81e4fc0 0x7fe8d81e35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.270+0000 7fe8df163640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8d8155a20 0x7fe8d81564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.270+0000 7fe8df163640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d81e3f20 con 0x7fe8c4004d00 2023-10-16T10:02:19.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.270+0000 7fe8df163640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d81e4200 con 0x7fe8d81e4fc0 2023-10-16T10:02:19.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.270+0000 7fe8df163640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d81e44e0 con 0x7fe8d8155a20 2023-10-16T10:02:19.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.271+0000 7fe8de161640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8c4004d00 0x7fe8d8147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.271+0000 7fe8de161640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8c4004d00 0x7fe8d8147ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44702/0 (socket says 172.21.15.84:44702) 2023-10-16T10:02:19.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.271+0000 7fe8de161640 1 -- 172.21.15.84:0/3616780623 learned_addr learned my addr 172.21.15.84:0/3616780623 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:19.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.271+0000 7fe8de962640 1 --2- 172.21.15.84:0/3616780623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8d8155a20 0x7fe8d81564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.271+0000 7fe8dd960640 1 --2- 172.21.15.84:0/3616780623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8d81e4fc0 0x7fe8d81e35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.271+0000 7fe8de161640 1 -- 172.21.15.84:0/3616780623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8d8155a20 msgr2=0x7fe8d81564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.271+0000 7fe8de161640 1 --2- 172.21.15.84:0/3616780623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8d8155a20 0x7fe8d81564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.271+0000 7fe8de161640 1 -- 172.21.15.84:0/3616780623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8d81e4fc0 msgr2=0x7fe8d81e35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.271+0000 7fe8de161640 1 --2- 172.21.15.84:0/3616780623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8d81e4fc0 0x7fe8d81e35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.271+0000 7fe8de161640 1 -- 172.21.15.84:0/3616780623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d81e47c0 con 0x7fe8c4004d00 2023-10-16T10:02:19.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.272+0000 7fe8de161640 1 --2- 172.21.15.84:0/3616780623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8c4004d00 0x7fe8d8147ad0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4011600 tx=0x7fe8d4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:19.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.272+0000 7fe8ceffd640 1 -- 172.21.15.84:0/3616780623 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400f920 con 0x7fe8c4004d00 2023-10-16T10:02:19.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.272+0000 7fe8ceffd640 1 -- 172.21.15.84:0/3616780623 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d400fa80 con 0x7fe8c4004d00 2023-10-16T10:02:19.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.272+0000 7fe8ceffd640 1 -- 172.21.15.84:0/3616780623 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400fdf0 con 0x7fe8c4004d00 2023-10-16T10:02:19.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.272+0000 7fe8df163640 1 -- 172.21.15.84:0/3616780623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d81e4aa0 con 0x7fe8c4004d00 2023-10-16T10:02:19.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.272+0000 7fe8df163640 1 -- 172.21.15.84:0/3616780623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d81e3cd0 con 0x7fe8c4004d00 2023-10-16T10:02:19.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.273+0000 7fe8df163640 1 -- 172.21.15.84:0/3616780623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d81e4c30 con 0x7fe8c4004d00 2023-10-16T10:02:19.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.274+0000 7fe8ceffd640 1 -- 172.21.15.84:0/3616780623 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe8d4015460 con 0x7fe8c4004d00 2023-10-16T10:02:19.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.275+0000 7fe8ceffd640 1 --2- 172.21.15.84:0/3616780623 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fe8bc06af90 0x7fe8bc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.275+0000 7fe8ceffd640 1 -- 172.21.15.84:0/3616780623 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8d4089ce0 con 0x7fe8c4004d00 2023-10-16T10:02:19.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.276+0000 7fe8dd960640 1 --2- 172.21.15.84:0/3616780623 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fe8bc06af90 0x7fe8bc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.276+0000 7fe8dd960640 1 --2- 172.21.15.84:0/3616780623 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fe8bc06af90 0x7fe8bc06d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80019b0 tx=0x7fe8c8009040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:19.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.280+0000 7fe8ceffd640 1 -- 172.21.15.84:0/3616780623 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe8d4054be0 con 0x7fe8c4004d00 2023-10-16T10:02:19.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.546+0000 7fe8df163640 1 -- 172.21.15.84:0/3616780623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe8d804e9b0 con 0x7fe8c4004d00 2023-10-16T10:02:19.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.551+0000 7fe8ceffd640 1 -- 172.21.15.84:0/3616780623 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe8d40220e0 con 0x7fe8c4004d00 2023-10-16T10:02:19.552 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2023-10-16T10:02:19.553 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ== 2023-10-16T10:02:19.553 INFO:teuthology.orchestra.run.smithi084.stdout: caps mds = "allow" 2023-10-16T10:02:19.553 INFO:teuthology.orchestra.run.smithi084.stdout: caps mon = "allow r" 2023-10-16T10:02:19.553 INFO:teuthology.orchestra.run.smithi084.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T10:02:19.553 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-16T10:02:19.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.554+0000 7fe8df163640 1 -- 172.21.15.84:0/3616780623 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fe8bc06af90 msgr2=0x7fe8bc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.554+0000 7fe8df163640 1 --2- 172.21.15.84:0/3616780623 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fe8bc06af90 0x7fe8bc06d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80019b0 tx=0x7fe8c8009040 comp rx=0 tx=0).stop 2023-10-16T10:02:19.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.554+0000 7fe8df163640 1 -- 172.21.15.84:0/3616780623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8c4004d00 msgr2=0x7fe8d8147ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.554+0000 7fe8df163640 1 --2- 172.21.15.84:0/3616780623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8c4004d00 0x7fe8d8147ad0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4011600 tx=0x7fe8d4011ac0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.554+0000 7fe8df163640 1 -- 172.21.15.84:0/3616780623 shutdown_connections 2023-10-16T10:02:19.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.554+0000 7fe8df163640 1 --2- 172.21.15.84:0/3616780623 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fe8bc06af90 0x7fe8bc06d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.554+0000 7fe8df163640 1 --2- 172.21.15.84:0/3616780623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8d8155a20 0x7fe8d81564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.554+0000 7fe8df163640 1 --2- 172.21.15.84:0/3616780623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8d81e4fc0 0x7fe8d81e35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.554+0000 7fe8df163640 1 --2- 172.21.15.84:0/3616780623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8c4004d00 0x7fe8d8147ad0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.554+0000 7fe8df163640 1 -- 172.21.15.84:0/3616780623 >> 172.21.15.84:0/3616780623 conn(0x7fe8d80ce6a0 msgr2=0x7fe8d815c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:19.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.554+0000 7fe8df163640 1 -- 172.21.15.84:0/3616780623 shutdown_connections 2023-10-16T10:02:19.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.554+0000 7fe8df163640 1 -- 172.21.15.84:0/3616780623 wait complete. 2023-10-16T10:02:19.574 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:02:19.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.787+0000 7f0af9333640 1 Processor -- start 2023-10-16T10:02:19.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.787+0000 7f0af9333640 1 -- start start 2023-10-16T10:02:19.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.789+0000 7f0af9333640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0af40cad80 0x7f0af40cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.789+0000 7f0af9333640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af40c93e0 con 0x7f0af41647e0 2023-10-16T10:02:19.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.789+0000 7f0af9333640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af40c8eb0 con 0x7f0af4164bc0 2023-10-16T10:02:19.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.789+0000 7f0af9333640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af40ca010 con 0x7f0af40cad80 2023-10-16T10:02:19.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.789+0000 7f0af2ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0af41647e0 0x7f0af40ad4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34234/0 (socket says 172.21.15.84:34234) 2023-10-16T10:02:19.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.789+0000 7f0af2ffd640 1 -- 172.21.15.84:0/3275096211 learned_addr learned my addr 172.21.15.84:0/3275096211 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:19.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.789+0000 7f0af37fe640 1 --2- 172.21.15.84:0/3275096211 >> v2:172.21.15.84:3300/0 conn(0x7f0af40cad80 0x7f0af40cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.789+0000 7f0af27fc640 1 -- 172.21.15.84:0/3275096211 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3748856254 0 0) 0x7f0af40c93e0 con 0x7f0af41647e0 2023-10-16T10:02:19.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.789+0000 7f0af27fc640 1 -- 172.21.15.84:0/3275096211 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ae0003520 con 0x7f0af41647e0 2023-10-16T10:02:19.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.790+0000 7f0af37fe640 1 -- 172.21.15.84:0/3275096211 >> v1:172.21.15.157:6789/0 conn(0x7f0af4164bc0 legacy=0x7f0af406f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.790+0000 7f0af37fe640 1 -- 172.21.15.84:0/3275096211 >> v1:172.21.15.84:6789/0 conn(0x7f0af41647e0 legacy=0x7f0af40ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.790+0000 7f0af27fc640 1 -- 172.21.15.84:0/3275096211 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3595494447 0 0) 0x7f0ae0003520 con 0x7f0af41647e0 2023-10-16T10:02:19.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.790+0000 7f0af37fe640 1 -- 172.21.15.84:0/3275096211 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af406cc70 con 0x7f0af40cad80 2023-10-16T10:02:19.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.791+0000 7f0af37fe640 1 --2- 172.21.15.84:0/3275096211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0af40cad80 0x7f0af40cf380 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0097a0 tx=0x7f0aec02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f16dfd7b164e16 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:19.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.791+0000 7f0af27fc640 1 -- 172.21.15.84:0/3275096211 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec02f050 con 0x7f0af40cad80 2023-10-16T10:02:19.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.791+0000 7f0af27fc640 1 -- 172.21.15.84:0/3275096211 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec031500 con 0x7f0af40cad80 2023-10-16T10:02:19.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.792+0000 7f0af27fc640 1 -- 172.21.15.84:0/3275096211 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4001670 con 0x7f0af40cad80 2023-10-16T10:02:19.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.792+0000 7f0af9333640 1 -- 172.21.15.84:0/3275096211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0af40cad80 msgr2=0x7f0af40cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.792+0000 7f0af9333640 1 --2- 172.21.15.84:0/3275096211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0af40cad80 0x7f0af40cf380 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0097a0 tx=0x7f0aec02b820 comp rx=0 tx=0).stop 2023-10-16T10:02:19.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.792+0000 7f0af9333640 1 -- 172.21.15.84:0/3275096211 shutdown_connections 2023-10-16T10:02:19.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.792+0000 7f0af9333640 1 --2- 172.21.15.84:0/3275096211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0af40cad80 0x7f0af40cf380 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.792+0000 7f0af9333640 1 -- 172.21.15.84:0/3275096211 >> 172.21.15.84:0/3275096211 conn(0x7f0af40ce860 msgr2=0x7f0af40cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:19.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.792+0000 7f0af9333640 1 -- 172.21.15.84:0/3275096211 shutdown_connections 2023-10-16T10:02:19.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.792+0000 7f0af9333640 1 -- 172.21.15.84:0/3275096211 wait complete. 2023-10-16T10:02:19.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.794+0000 7f0af9333640 1 Processor -- start 2023-10-16T10:02:19.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.794+0000 7f0af9333640 1 -- start start 2023-10-16T10:02:19.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.795+0000 7f0af9333640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0af40cad80 0x7f0af4159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.795+0000 7f0af9333640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0af41647e0 0x7f0af41ecc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.795+0000 7f0af9333640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0af4164bc0 0x7f0af41633c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.795+0000 7f0af9333640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af41ec210 con 0x7f0af4164bc0 2023-10-16T10:02:19.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.795+0000 7f0af9333640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af41ec7d0 con 0x7f0af40cad80 2023-10-16T10:02:19.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.795+0000 7f0af9333640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af415aee0 con 0x7f0af41647e0 2023-10-16T10:02:19.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.795+0000 7f0af37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0af4164bc0 0x7f0af41633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.795+0000 7f0af37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0af4164bc0 0x7f0af41633c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44720/0 (socket says 172.21.15.84:44720) 2023-10-16T10:02:19.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.795+0000 7f0af37fe640 1 -- 172.21.15.84:0/3147290966 learned_addr learned my addr 172.21.15.84:0/3147290966 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:19.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.796+0000 7f0af37fe640 1 -- 172.21.15.84:0/3147290966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0af41647e0 msgr2=0x7f0af41ecc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.796+0000 7f0af37fe640 1 --2- 172.21.15.84:0/3147290966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0af41647e0 0x7f0af41ecc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.796+0000 7f0af37fe640 1 -- 172.21.15.84:0/3147290966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0af40cad80 msgr2=0x7f0af4159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.796+0000 7f0af37fe640 1 --2- 172.21.15.84:0/3147290966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0af40cad80 0x7f0af4159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.796+0000 7f0af37fe640 1 -- 172.21.15.84:0/3147290966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aec009400 con 0x7f0af4164bc0 2023-10-16T10:02:19.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.796+0000 7f0af37fe640 1 --2- 172.21.15.84:0/3147290966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0af4164bc0 0x7f0af41633c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0097a0 tx=0x7f0aec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:19.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.796+0000 7f0aebfff640 1 -- 172.21.15.84:0/3147290966 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec03d940 con 0x7f0af4164bc0 2023-10-16T10:02:19.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.796+0000 7f0aebfff640 1 -- 172.21.15.84:0/3147290966 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec03daa0 con 0x7f0af4164bc0 2023-10-16T10:02:19.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.796+0000 7f0af9333640 1 -- 172.21.15.84:0/3147290966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af41eb9b0 con 0x7f0af4164bc0 2023-10-16T10:02:19.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.796+0000 7f0aebfff640 1 -- 172.21.15.84:0/3147290966 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec03de10 con 0x7f0af4164bc0 2023-10-16T10:02:19.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.797+0000 7f0af9333640 1 -- 172.21.15.84:0/3147290966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af41ec460 con 0x7f0af4164bc0 2023-10-16T10:02:19.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.798+0000 7f0af9333640 1 -- 172.21.15.84:0/3147290966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af40c9be0 con 0x7f0af4164bc0 2023-10-16T10:02:19.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.798+0000 7f0aebfff640 1 -- 172.21.15.84:0/3147290966 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0aec03a040 con 0x7f0af4164bc0 2023-10-16T10:02:19.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.799+0000 7f0aebfff640 1 --2- 172.21.15.84:0/3147290966 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f0ad406afc0 0x7f0ad406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.799+0000 7f0aebfff640 1 -- 172.21.15.84:0/3147290966 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0aec0ab410 con 0x7f0af4164bc0 2023-10-16T10:02:19.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.800+0000 7f0af2ffd640 1 --2- 172.21.15.84:0/3147290966 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f0ad406afc0 0x7f0ad406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.800+0000 7f0af2ffd640 1 --2- 172.21.15.84:0/3147290966 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f0ad406afc0 0x7f0ad406d480 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4001e80 tx=0x7f0ae40010a0 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:19.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:19.804+0000 7f0aebfff640 1 -- 172.21.15.84:0/3147290966 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0aec076330 con 0x7f0af4164bc0 2023-10-16T10:02:20.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.036+0000 7f0af9333640 1 -- 172.21.15.84:0/3147290966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0af41ec960 con 0x7f0af4164bc0 2023-10-16T10:02:20.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.037+0000 7f0aebfff640 1 -- 172.21.15.84:0/3147290966 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f0aec079950 con 0x7f0af4164bc0 2023-10-16T10:02:20.039 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:02:20.039 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3040499154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3040499154},{"type":"v1","addr":"172.21.15.157:6835","nonce":3040499154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2727537766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2727537766},{"type":"v1","addr":"172.21.15.157:6833","nonce":2727537766}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/49292950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":49292950},{"type":"v1","addr":"172.21.15.84:6836","nonce":49292950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:02:13.514054+0000","modified":"2023-10-16T10:02:16.598055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/3839442164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3839442164},{"type":"v1","addr":"172.21.15.84:6837","nonce":3839442164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T10:02:20.039 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 25 2023-10-16T10:02:20.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.039+0000 7f0af9333640 1 -- 172.21.15.84:0/3147290966 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f0ad406afc0 msgr2=0x7f0ad406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:20.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.039+0000 7f0af9333640 1 --2- 172.21.15.84:0/3147290966 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f0ad406afc0 0x7f0ad406d480 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4001e80 tx=0x7f0ae40010a0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.039+0000 7f0af9333640 1 -- 172.21.15.84:0/3147290966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0af4164bc0 msgr2=0x7f0af41633c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:20.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.039+0000 7f0af9333640 1 --2- 172.21.15.84:0/3147290966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0af4164bc0 0x7f0af41633c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0097a0 tx=0x7f0aec0046c0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.040+0000 7f0af9333640 1 -- 172.21.15.84:0/3147290966 shutdown_connections 2023-10-16T10:02:20.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.040+0000 7f0af9333640 1 --2- 172.21.15.84:0/3147290966 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f0ad406afc0 0x7f0ad406d480 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.040+0000 7f0af9333640 1 --2- 172.21.15.84:0/3147290966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0af4164bc0 0x7f0af41633c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.040+0000 7f0af9333640 1 --2- 172.21.15.84:0/3147290966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0af41647e0 0x7f0af41ecc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.040+0000 7f0af9333640 1 --2- 172.21.15.84:0/3147290966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0af40cad80 0x7f0af4159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.040+0000 7f0af9333640 1 -- 172.21.15.84:0/3147290966 >> 172.21.15.84:0/3147290966 conn(0x7f0af40ce860 msgr2=0x7f0af40afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:20.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.040+0000 7f0af9333640 1 -- 172.21.15.84:0/3147290966 shutdown_connections 2023-10-16T10:02:20.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.040+0000 7f0af9333640 1 -- 172.21.15.84:0/3147290966 wait complete. 2023-10-16T10:02:20.463 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T10:02:13.514054+0000', 'modified': '2023-10-16T10:02:16.598055+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5991}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5991': {'gid': 5991, 'name': 'a', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.84:6837/3839442164', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6835', 'nonce': 3839442164}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 3839442164}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T10:02:20.464 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T10:02:20.464 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-16T10:02:20.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.688+0000 7f1d2cb27640 1 Processor -- start 2023-10-16T10:02:20.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.688+0000 7f1d2cb27640 1 -- start start 2023-10-16T10:02:20.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.689+0000 7f1d2cb27640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d28070b60 0x7f1d2815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.689+0000 7f1d2cb27640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1d2806c100 0x7f1d2806c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.690+0000 7f1d2cb27640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2815aef0 con 0x7f1d28159950 2023-10-16T10:02:20.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.690+0000 7f1d2cb27640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d2815e490 con 0x7f1d2806c100 2023-10-16T10:02:20.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.690+0000 7f1d2cb27640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2815d910 con 0x7f1d28070b60 2023-10-16T10:02:20.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.690+0000 7f1d277fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1d28159950 0x7f1d28070450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34250/0 (socket says 172.21.15.84:34250) 2023-10-16T10:02:20.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.690+0000 7f1d277fe640 1 -- 172.21.15.84:0/3421591021 learned_addr learned my addr 172.21.15.84:0/3421591021 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:20.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.690+0000 7f1d27fff640 1 --2- 172.21.15.84:0/3421591021 >> v2:172.21.15.84:3300/0 conn(0x7f1d2806c100 0x7f1d2806c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.690+0000 7f1d27fff640 1 -- 172.21.15.84:0/3421591021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d28070b60 msgr2=0x7f1d2815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:20.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.690+0000 7f1d1effd640 1 --2- 172.21.15.84:0/3421591021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d28070b60 0x7f1d2815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.690+0000 7f1d27fff640 1 --2- 172.21.15.84:0/3421591021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d28070b60 0x7f1d2815a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.690+0000 7f1d27fff640 1 -- 172.21.15.84:0/3421591021 >> v1:172.21.15.84:6789/0 conn(0x7f1d28159950 legacy=0x7f1d28070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:20.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.690+0000 7f1d27fff640 1 -- 172.21.15.84:0/3421591021 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2815dbf0 con 0x7f1d2806c100 2023-10-16T10:02:20.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.690+0000 7f1d26ffd640 1 -- 172.21.15.84:0/3421591021 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 552392463 0 0) 0x7f1d18001110 con 0x7f1d28159950 2023-10-16T10:02:20.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.690+0000 7f1d1effd640 1 --2- 172.21.15.84:0/3421591021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d28070b60 0x7f1d2815a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:20.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.691+0000 7f1d27fff640 1 --2- 172.21.15.84:0/3421591021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d2806c100 0x7f1d2806c550 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f1d200097a0 tx=0x7f1d2002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1c484219274936 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:20.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.692+0000 7f1d26ffd640 1 -- 172.21.15.84:0/3421591021 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d20031780 con 0x7f1d2806c100 2023-10-16T10:02:20.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.692+0000 7f1d26ffd640 1 -- 172.21.15.84:0/3421591021 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d200318e0 con 0x7f1d2806c100 2023-10-16T10:02:20.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.692+0000 7f1d26ffd640 1 -- 172.21.15.84:0/3421591021 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d20031c70 con 0x7f1d2806c100 2023-10-16T10:02:20.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.692+0000 7f1d2cb27640 1 -- 172.21.15.84:0/3421591021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d2806c100 msgr2=0x7f1d2806c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:20.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.692+0000 7f1d2cb27640 1 --2- 172.21.15.84:0/3421591021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d2806c100 0x7f1d2806c550 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f1d200097a0 tx=0x7f1d2002b820 comp rx=0 tx=0).stop 2023-10-16T10:02:20.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.692+0000 7f1d2cb27640 1 -- 172.21.15.84:0/3421591021 shutdown_connections 2023-10-16T10:02:20.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.692+0000 7f1d2cb27640 1 --2- 172.21.15.84:0/3421591021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d2806c100 0x7f1d2806c550 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.693+0000 7f1d2cb27640 1 --2- 172.21.15.84:0/3421591021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d28070b60 0x7f1d2815a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.693+0000 7f1d2cb27640 1 -- 172.21.15.84:0/3421591021 >> 172.21.15.84:0/3421591021 conn(0x7f1d280ce6a0 msgr2=0x7f1d280ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:20.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.693+0000 7f1d2cb27640 1 -- 172.21.15.84:0/3421591021 shutdown_connections 2023-10-16T10:02:20.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.693+0000 7f1d2cb27640 1 -- 172.21.15.84:0/3421591021 wait complete. 2023-10-16T10:02:20.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.694+0000 7f1d2cb27640 1 Processor -- start 2023-10-16T10:02:20.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.695+0000 7f1d2cb27640 1 -- start start 2023-10-16T10:02:20.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.695+0000 7f1d2cb27640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d2806c100 0x7f1d280c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.695+0000 7f1d2cb27640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d28070b60 0x7f1d281f3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.695+0000 7f1d2cb27640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d28159950 0x7f1d28147980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.696+0000 7f1d2cb27640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d281f37a0 con 0x7f1d28070b60 2023-10-16T10:02:20.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.696+0000 7f1d2cb27640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d280c3eb0 con 0x7f1d28159950 2023-10-16T10:02:20.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.696+0000 7f1d2cb27640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d280c4150 con 0x7f1d2806c100 2023-10-16T10:02:20.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.696+0000 7f1d1effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d28070b60 0x7f1d281f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.696+0000 7f1d1effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d28070b60 0x7f1d281f3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44734/0 (socket says 172.21.15.84:44734) 2023-10-16T10:02:20.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.696+0000 7f1d1effd640 1 -- 172.21.15.84:0/2044434232 learned_addr learned my addr 172.21.15.84:0/2044434232 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:20.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.696+0000 7f1d277fe640 1 --2- 172.21.15.84:0/2044434232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d2806c100 0x7f1d280c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.696+0000 7f1d1effd640 1 -- 172.21.15.84:0/2044434232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d2806c100 msgr2=0x7f1d280c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:20.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.696+0000 7f1d1effd640 1 --2- 172.21.15.84:0/2044434232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d2806c100 0x7f1d280c21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.696+0000 7f1d1effd640 1 -- 172.21.15.84:0/2044434232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d28159950 msgr2=0x7f1d28147980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:20.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.696+0000 7f1d1effd640 1 --2- 172.21.15.84:0/2044434232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d28159950 0x7f1d28147980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.696+0000 7f1d1effd640 1 -- 172.21.15.84:0/2044434232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d20009400 con 0x7f1d28070b60 2023-10-16T10:02:20.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.696+0000 7f1d277fe640 1 --2- 172.21.15.84:0/2044434232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d2806c100 0x7f1d280c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:20.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.696+0000 7f1d1effd640 1 --2- 172.21.15.84:0/2044434232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d28070b60 0x7f1d281f3260 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000cf60 tx=0x7f1d1000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:20.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.697+0000 7f1d24ff9640 1 -- 172.21.15.84:0/2044434232 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1000d840 con 0x7f1d28070b60 2023-10-16T10:02:20.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.697+0000 7f1d24ff9640 1 -- 172.21.15.84:0/2044434232 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1000d9a0 con 0x7f1d28070b60 2023-10-16T10:02:20.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.697+0000 7f1d24ff9640 1 -- 172.21.15.84:0/2044434232 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1000dd30 con 0x7f1d28070b60 2023-10-16T10:02:20.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.697+0000 7f1d2cb27640 1 -- 172.21.15.84:0/2044434232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d280c4430 con 0x7f1d28070b60 2023-10-16T10:02:20.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.697+0000 7f1d2cb27640 1 -- 172.21.15.84:0/2044434232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d280c2790 con 0x7f1d28070b60 2023-10-16T10:02:20.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.700+0000 7f1d24ff9640 1 -- 172.21.15.84:0/2044434232 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1d10015440 con 0x7f1d28070b60 2023-10-16T10:02:20.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.701+0000 7f1d24ff9640 1 --2- 172.21.15.84:0/2044434232 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f1d0806afe0 0x7f1d0806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.701+0000 7f1d24ff9640 1 -- 172.21.15.84:0/2044434232 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d1008a1e0 con 0x7f1d28070b60 2023-10-16T10:02:20.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.701+0000 7f1d277fe640 1 --2- 172.21.15.84:0/2044434232 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f1d0806afe0 0x7f1d0806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.702+0000 7f1d309fb740 1 Processor -- start 2023-10-16T10:02:20.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.702+0000 7f1d309fb740 1 -- start start 2023-10-16T10:02:20.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.702+0000 7f1d277fe640 1 --2- 172.21.15.84:0/2044434232 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f1d0806afe0 0x7f1d0806d4a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1d18000f60 tx=0x7f1d18009040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:20.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.703+0000 7f1d309fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c6170 0x55cfbf899070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.703+0000 7f1d309fb740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8c6550 0x55cfbf8c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.703+0000 7f1d309fb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8995b0 0x55cfbf8c9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.703+0000 7f1d309fb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbf7b97a0 con 0x55cfbf8995b0 2023-10-16T10:02:20.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.703+0000 7f1d309fb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbf80b740 con 0x55cfbf8c6170 2023-10-16T10:02:20.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.703+0000 7f1d309fb740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfbf808e30 con 0x55cfbf8c6550 2023-10-16T10:02:20.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.703+0000 7f1d27fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8995b0 0x55cfbf8c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.703+0000 7f1d27fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8995b0 0x55cfbf8c9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44750/0 (socket says 172.21.15.84:44750) 2023-10-16T10:02:20.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.703+0000 7f1d27fff640 1 -- 172.21.15.84:0/1298544616 learned_addr learned my addr 172.21.15.84:0/1298544616 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:20.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.703+0000 7f1d1effd640 1 --2- 172.21.15.84:0/1298544616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8c6550 0x55cfbf8c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.703+0000 7f1d277fe640 1 --2- 172.21.15.84:0/1298544616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c6170 0x55cfbf899070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.703+0000 7f1d27fff640 1 -- 172.21.15.84:0/1298544616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8c6550 msgr2=0x55cfbf8c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:20.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.703+0000 7f1d27fff640 1 --2- 172.21.15.84:0/1298544616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8c6550 0x55cfbf8c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.704+0000 7f1d27fff640 1 -- 172.21.15.84:0/1298544616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c6170 msgr2=0x55cfbf899070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:20.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.704+0000 7f1d27fff640 1 --2- 172.21.15.84:0/1298544616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c6170 0x55cfbf899070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.704+0000 7f1d27fff640 1 -- 172.21.15.84:0/1298544616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfbf7c8910 con 0x55cfbf8995b0 2023-10-16T10:02:20.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.704+0000 7f1d27fff640 1 --2- 172.21.15.84:0/1298544616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8995b0 0x55cfbf8c9440 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1d20034040 tx=0x7f1d2002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9e5acd843fd312 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:20.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.704+0000 7f1d1d7fa640 1 -- 172.21.15.84:0/1298544616 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d20031c50 con 0x55cfbf8995b0 2023-10-16T10:02:20.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.704+0000 7f1d1d7fa640 1 -- 172.21.15.84:0/1298544616 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d20031dd0 con 0x55cfbf8995b0 2023-10-16T10:02:20.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.704+0000 7f1d1d7fa640 1 -- 172.21.15.84:0/1298544616 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d20004270 con 0x55cfbf8995b0 2023-10-16T10:02:20.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.705+0000 7f1d309fb740 1 -- 172.21.15.84:0/1298544616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8995b0 msgr2=0x55cfbf8c9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:20.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.705+0000 7f1d309fb740 1 --2- 172.21.15.84:0/1298544616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8995b0 0x55cfbf8c9440 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1d20034040 tx=0x7f1d2002e000 comp rx=0 tx=0).stop 2023-10-16T10:02:20.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.705+0000 7f1d309fb740 1 -- 172.21.15.84:0/1298544616 shutdown_connections 2023-10-16T10:02:20.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.705+0000 7f1d309fb740 1 --2- 172.21.15.84:0/1298544616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8995b0 0x55cfbf8c9440 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.705+0000 7f1d309fb740 1 --2- 172.21.15.84:0/1298544616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8c6550 0x55cfbf8c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.705+0000 7f1d309fb740 1 --2- 172.21.15.84:0/1298544616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c6170 0x55cfbf899070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.705+0000 7f1d309fb740 1 -- 172.21.15.84:0/1298544616 >> 172.21.15.84:0/1298544616 conn(0x55cfbf89b380 msgr2=0x55cfbf8e3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:20.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.705+0000 7f1d309fb740 1 -- 172.21.15.84:0/1298544616 shutdown_connections 2023-10-16T10:02:20.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.705+0000 7f1d309fb740 1 -- 172.21.15.84:0/1298544616 wait complete. 2023-10-16T10:02:20.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.706+0000 7f1d309fb740 1 -- start start 2023-10-16T10:02:20.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.706+0000 7f1d309fb740 1 Processor -- start 2023-10-16T10:02:20.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.707+0000 7f1d309fb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6170 0x55cfbf8c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.707+0000 7f1d309fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c6550 0x55cfbf8dcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.707+0000 7f1d309fb740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8ccb00 0x55cfbf8dd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.707+0000 7f1d309fb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbf804720 con 0x55cfbf8c6170 2023-10-16T10:02:20.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.707+0000 7f1d309fb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbf7a2970 con 0x55cfbf8c6550 2023-10-16T10:02:20.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.707+0000 7f1d309fb740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfbf7a2610 con 0x55cfbf8ccb00 2023-10-16T10:02:20.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.707+0000 7f1d309fb740 10 client.0 _ll_register_callbacks cb 0x55cfbf7c49d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:02:20.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.707+0000 7f1d277fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6170 0x55cfbf8c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.707+0000 7f1d277fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6170 0x55cfbf8c41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44756/0 (socket says 172.21.15.84:44756) 2023-10-16T10:02:20.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.707+0000 7f1d277fe640 1 -- 172.21.15.84:0/2961017436 learned_addr learned my addr 172.21.15.84:0/2961017436 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:20.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.708+0000 7f1d27fff640 1 --2- 172.21.15.84:0/2961017436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8ccb00 0x55cfbf8dd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.708+0000 7f1d1effd640 1 --2- 172.21.15.84:0/2961017436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c6550 0x55cfbf8dcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.708+0000 7f1d277fe640 1 -- 172.21.15.84:0/2961017436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8ccb00 msgr2=0x55cfbf8dd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:20.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.708+0000 7f1d277fe640 1 --2- 172.21.15.84:0/2961017436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8ccb00 0x55cfbf8dd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.708+0000 7f1d277fe640 1 -- 172.21.15.84:0/2961017436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c6550 msgr2=0x55cfbf8dcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:20.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.708+0000 7f1d277fe640 1 --2- 172.21.15.84:0/2961017436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c6550 0x55cfbf8dcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.708+0000 7f1d277fe640 1 -- 172.21.15.84:0/2961017436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d2815dbf0 con 0x55cfbf8c6170 2023-10-16T10:02:20.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.708+0000 7f1d1effd640 1 --2- 172.21.15.84:0/2961017436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c6550 0x55cfbf8dcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:20.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.708+0000 7f1d277fe640 1 --2- 172.21.15.84:0/2961017436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6170 0x55cfbf8c41b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1d18005aa0 tx=0x7f1d18000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fc29056de267ca server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:20.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.708+0000 7f1d067fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T10:02:20.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.708+0000 7f1d067fc640 1 -- 172.21.15.84:0/2961017436 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d18021630 con 0x55cfbf8c6170 2023-10-16T10:02:20.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.708+0000 7f1d067fc640 1 -- 172.21.15.84:0/2961017436 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d18020a80 con 0x55cfbf8c6170 2023-10-16T10:02:20.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.708+0000 7f1d309fb740 1 -- 172.21.15.84:0/2961017436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cfbf82b160 con 0x55cfbf8c6170 2023-10-16T10:02:20.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.709+0000 7f1d067fc640 1 -- 172.21.15.84:0/2961017436 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d18020c40 con 0x55cfbf8c6170 2023-10-16T10:02:20.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.709+0000 7f1d309fb740 10 client.6129 fetch_fsmap learned FSMap version 25 2023-10-16T10:02:20.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.709+0000 7f1d309fb740 1 -- 172.21.15.84:0/2961017436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55cfbf802da0 con 0x55cfbf8c6170 2023-10-16T10:02:20.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.709+0000 7f1d067fc640 1 -- 172.21.15.84:0/2961017436 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cfbf82b160 con 0x55cfbf8c6170 2023-10-16T10:02:20.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.709+0000 7f1d067fc640 1 -- 172.21.15.84:0/2961017436 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1d18020420 con 0x55cfbf8c6170 2023-10-16T10:02:20.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.709+0000 7f1d309fb740 10 client.6129 fetch_fsmap finished waiting for FSMap version 25 2023-10-16T10:02:20.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.709+0000 7f1d309fb740 10 client.6129 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T10:02:20.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.709+0000 7f1d309fb740 20 client.6129 populate_metadata read hostname 'smithi084' 2023-10-16T10:02:20.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.709+0000 7f1d309fb740 1 --2- 172.21.15.84:0/2961017436 >> [v2:172.21.15.84:6835/3839442164,v1:172.21.15.84:6837/3839442164] conn(0x55cfbf8f4ea0 0x55cfbf901510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.709+0000 7f1d309fb740 4 client.6129 mds_command: new command op to 5991 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T10:02:20.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.709+0000 7f1d309fb740 1 -- 172.21.15.84:0/2961017436 --> [v2:172.21.15.84:6835/3839442164,v1:172.21.15.84:6837/3839442164] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfbf7d6cc0 con 0x55cfbf8f4ea0 2023-10-16T10:02:20.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.709+0000 7f1d1effd640 1 --2- 172.21.15.84:0/2961017436 >> [v2:172.21.15.84:6835/3839442164,v1:172.21.15.84:6837/3839442164] conn(0x55cfbf8f4ea0 0x55cfbf901510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.710+0000 7f1d1effd640 1 --2- 172.21.15.84:0/2961017436 >> [v2:172.21.15.84:6835/3839442164,v1:172.21.15.84:6837/3839442164] conn(0x55cfbf8f4ea0 0x55cfbf901510 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9532d7462506bb server_cookie=9c365ecd7a3cfbae in_seq=0 out_seq=0 2023-10-16T10:02:20.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.710+0000 7f1d067fc640 10 client.6129 ms_handle_connect on v2:172.21.15.84:6835/3839442164 2023-10-16T10:02:20.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.711+0000 7f1d067fc640 1 -- 172.21.15.84:0/2961017436 <== mds.0 v2:172.21.15.84:6835/3839442164 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cfbf7d6cc0 con 0x55cfbf8f4ea0 2023-10-16T10:02:20.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.711+0000 7f1d067fc640 10 client.6129 handle_command_reply: tid=0 2023-10-16T10:02:20.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.711+0000 7f1d309fb740 1 client.6129 shutdown 2023-10-16T10:02:20.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.711+0000 7f1d309fb740 1 -- 172.21.15.84:0/2961017436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6170 msgr2=0x55cfbf8c41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:20.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.711+0000 7f1d309fb740 1 --2- 172.21.15.84:0/2961017436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6170 0x55cfbf8c41b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1d18005aa0 tx=0x7f1d18000ed0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.711+0000 7f1d309fb740 1 -- 172.21.15.84:0/2961017436 shutdown_connections 2023-10-16T10:02:20.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.711+0000 7f1d309fb740 1 --2- 172.21.15.84:0/2961017436 >> [v2:172.21.15.84:6835/3839442164,v1:172.21.15.84:6837/3839442164] conn(0x55cfbf8f4ea0 0x55cfbf901510 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.711+0000 7f1d309fb740 1 --2- 172.21.15.84:0/2961017436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8ccb00 0x55cfbf8dd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.711+0000 7f1d309fb740 1 --2- 172.21.15.84:0/2961017436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c6550 0x55cfbf8dcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.711+0000 7f1d309fb740 1 --2- 172.21.15.84:0/2961017436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6170 0x55cfbf8c41b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.711+0000 7f1d067fc640 0 client.6129 ms_handle_reset on v2:172.21.15.84:6835/3839442164 2023-10-16T10:02:20.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.711+0000 7f1d309fb740 1 -- 172.21.15.84:0/2961017436 >> 172.21.15.84:0/2961017436 conn(0x55cfbf8995b0 msgr2=0x55cfbf8c8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:20.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.711+0000 7f1d309fb740 1 -- 172.21.15.84:0/2961017436 shutdown_connections 2023-10-16T10:02:20.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.711+0000 7f1d309fb740 1 -- 172.21.15.84:0/2961017436 wait complete. 2023-10-16T10:02:20.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.712+0000 7f1d309fb740 20 client.6129 trim_cache size 0 max 16384 2023-10-16T10:02:20.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.728+0000 7f1d309fb740 1 Processor -- start 2023-10-16T10:02:20.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.728+0000 7f1d309fb740 1 -- start start 2023-10-16T10:02:20.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.729+0000 7f1d309fb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6170 0x55cfbf8f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.729+0000 7f1d309fb740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8c6550 0x55cfbf905530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.729+0000 7f1d309fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c8940 0x55cfbf8cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.729+0000 7f1d309fb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbf568f20 con 0x55cfbf8c6170 2023-10-16T10:02:20.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.729+0000 7f1d309fb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbf8ddab0 con 0x55cfbf8c8940 2023-10-16T10:02:20.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.729+0000 7f1d309fb740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfbf905f00 con 0x55cfbf8c6550 2023-10-16T10:02:20.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.729+0000 7f1d277fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6170 0x55cfbf8f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.729+0000 7f1d277fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6170 0x55cfbf8f69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44770/0 (socket says 172.21.15.84:44770) 2023-10-16T10:02:20.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.729+0000 7f1d277fe640 1 -- 172.21.15.84:0/1240416450 learned_addr learned my addr 172.21.15.84:0/1240416450 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:20.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.730+0000 7f1d277fe640 1 -- 172.21.15.84:0/1240416450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8c6550 msgr2=0x55cfbf905530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:20.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.730+0000 7f1d277fe640 1 --2- 172.21.15.84:0/1240416450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8c6550 0x55cfbf905530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.730+0000 7f1d277fe640 1 -- 172.21.15.84:0/1240416450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c8940 msgr2=0x55cfbf8cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:20.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.730+0000 7f1d277fe640 1 --2- 172.21.15.84:0/1240416450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c8940 0x55cfbf8cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.730+0000 7f1d277fe640 1 -- 172.21.15.84:0/1240416450 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfbf7a1b80 con 0x55cfbf8c6170 2023-10-16T10:02:20.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.730+0000 7f1d277fe640 1 --2- 172.21.15.84:0/1240416450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6170 0x55cfbf8f69d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800d620 tx=0x7f1d1801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a820b0d5d3f317dc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:20.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.730+0000 7f1d1dffb640 1 -- 172.21.15.84:0/1240416450 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1800dba0 con 0x55cfbf8c6170 2023-10-16T10:02:20.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.730+0000 7f1d1dffb640 1 -- 172.21.15.84:0/1240416450 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1801a050 con 0x55cfbf8c6170 2023-10-16T10:02:20.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.730+0000 7f1d1dffb640 1 -- 172.21.15.84:0/1240416450 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d18020b70 con 0x55cfbf8c6170 2023-10-16T10:02:20.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.731+0000 7f1d309fb740 1 -- 172.21.15.84:0/1240416450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6170 msgr2=0x55cfbf8f69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:20.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.731+0000 7f1d309fb740 1 --2- 172.21.15.84:0/1240416450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6170 0x55cfbf8f69d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800d620 tx=0x7f1d1801d000 comp rx=0 tx=0).stop 2023-10-16T10:02:20.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.731+0000 7f1d309fb740 1 -- 172.21.15.84:0/1240416450 shutdown_connections 2023-10-16T10:02:20.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.731+0000 7f1d309fb740 1 --2- 172.21.15.84:0/1240416450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c8940 0x55cfbf8cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.731+0000 7f1d309fb740 1 --2- 172.21.15.84:0/1240416450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8c6550 0x55cfbf905530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.731+0000 7f1d309fb740 1 --2- 172.21.15.84:0/1240416450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6170 0x55cfbf8f69d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.731+0000 7f1d309fb740 1 -- 172.21.15.84:0/1240416450 >> 172.21.15.84:0/1240416450 conn(0x55cfbf8995b0 msgr2=0x55cfbf8e7410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:20.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.731+0000 7f1d309fb740 1 -- 172.21.15.84:0/1240416450 shutdown_connections 2023-10-16T10:02:20.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.731+0000 7f1d309fb740 1 -- 172.21.15.84:0/1240416450 wait complete. 2023-10-16T10:02:20.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.732+0000 7f1d309fb740 1 -- start start 2023-10-16T10:02:20.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.732+0000 7f1d309fb740 1 Processor -- start 2023-10-16T10:02:20.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.733+0000 7f1d309fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c6170 0x55cfbf8e5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.733+0000 7f1d309fb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6550 0x55cfbf8dc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.733+0000 7f1d309fb740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8c8940 0x55cfbf94cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.733+0000 7f1d309fb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbf94d880 con 0x55cfbf8c6550 2023-10-16T10:02:20.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.733+0000 7f1d309fb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbf8c47b0 con 0x55cfbf8c6170 2023-10-16T10:02:20.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.733+0000 7f1d309fb740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfbf8c4ce0 con 0x55cfbf8c8940 2023-10-16T10:02:20.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.733+0000 7f1d309fb740 10 client.0 _ll_register_callbacks cb 0x7f1d18001330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:02:20.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.733+0000 7f1d1effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6550 0x55cfbf8dc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.733+0000 7f1d1effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6550 0x55cfbf8dc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44784/0 (socket says 172.21.15.84:44784) 2023-10-16T10:02:20.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.733+0000 7f1d1effd640 1 -- 172.21.15.84:0/1290986088 learned_addr learned my addr 172.21.15.84:0/1290986088 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:20.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.733+0000 7f1d277fe640 1 --2- 172.21.15.84:0/1290986088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c6170 0x55cfbf8e5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.733+0000 7f1d1effd640 1 -- 172.21.15.84:0/1290986088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8c8940 msgr2=0x55cfbf94cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:20.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.733+0000 7f1d27fff640 1 --2- 172.21.15.84:0/1290986088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8c8940 0x55cfbf94cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.733+0000 7f1d1effd640 1 --2- 172.21.15.84:0/1290986088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8c8940 0x55cfbf94cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.733+0000 7f1d1effd640 1 -- 172.21.15.84:0/1290986088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c6170 msgr2=0x55cfbf8e5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:20.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.733+0000 7f1d1effd640 1 --2- 172.21.15.84:0/1290986088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c6170 0x55cfbf8e5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.734+0000 7f1d1effd640 1 -- 172.21.15.84:0/1290986088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfbf802da0 con 0x55cfbf8c6550 2023-10-16T10:02:20.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.734+0000 7f1d1effd640 1 --2- 172.21.15.84:0/1290986088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6550 0x55cfbf8dc610 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1d10058e60 tx=0x7f1d1005d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe198ad4858a5a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:20.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.734+0000 7f1d067fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T10:02:20.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.734+0000 7f1d067fc640 1 -- 172.21.15.84:0/1290986088 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d100555e0 con 0x55cfbf8c6550 2023-10-16T10:02:20.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.734+0000 7f1d067fc640 1 -- 172.21.15.84:0/1290986088 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d10055740 con 0x55cfbf8c6550 2023-10-16T10:02:20.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.734+0000 7f1d309fb740 1 -- 172.21.15.84:0/1290986088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d20031dd0 con 0x55cfbf8c6550 2023-10-16T10:02:20.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.734+0000 7f1d067fc640 1 -- 172.21.15.84:0/1290986088 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d100886e0 con 0x55cfbf8c6550 2023-10-16T10:02:20.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.734+0000 7f1d309fb740 10 client.6135 fetch_fsmap learned FSMap version 25 2023-10-16T10:02:20.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.734+0000 7f1d309fb740 1 -- 172.21.15.84:0/1290986088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55cfbf7c7170 con 0x55cfbf8c6550 2023-10-16T10:02:20.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.734+0000 7f1d067fc640 1 -- 172.21.15.84:0/1290986088 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d20031dd0 con 0x55cfbf8c6550 2023-10-16T10:02:20.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.735+0000 7f1d067fc640 1 -- 172.21.15.84:0/1290986088 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1d10008040 con 0x55cfbf8c6550 2023-10-16T10:02:20.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.735+0000 7f1d309fb740 10 client.6135 fetch_fsmap finished waiting for FSMap version 25 2023-10-16T10:02:20.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.735+0000 7f1d309fb740 10 client.6135 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T10:02:20.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.735+0000 7f1d309fb740 20 client.6135 populate_metadata read hostname 'smithi084' 2023-10-16T10:02:20.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.735+0000 7f1d309fb740 1 --2- 172.21.15.84:0/1290986088 >> [v2:172.21.15.84:6835/3839442164,v1:172.21.15.84:6837/3839442164] conn(0x55cfbf8cc020 0x55cfbf8cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.735+0000 7f1d309fb740 4 client.6135 mds_command: new command op to 5991 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T10:02:20.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.735+0000 7f1d309fb740 1 -- 172.21.15.84:0/1290986088 --> [v2:172.21.15.84:6835/3839442164,v1:172.21.15.84:6837/3839442164] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cfbf7fd700 con 0x55cfbf8cc020 2023-10-16T10:02:20.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.735+0000 7f1d277fe640 1 --2- 172.21.15.84:0/1290986088 >> [v2:172.21.15.84:6835/3839442164,v1:172.21.15.84:6837/3839442164] conn(0x55cfbf8cc020 0x55cfbf8cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.735+0000 7f1d067fc640 10 client.6135 ms_handle_connect on v2:172.21.15.84:6835/3839442164 2023-10-16T10:02:20.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.735+0000 7f1d277fe640 1 --2- 172.21.15.84:0/1290986088 >> [v2:172.21.15.84:6835/3839442164,v1:172.21.15.84:6837/3839442164] conn(0x55cfbf8cc020 0x55cfbf8cc400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f50c50c022e564fe server_cookie=f3abaf374a4f629 in_seq=0 out_seq=0 2023-10-16T10:02:20.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.736+0000 7f1d067fc640 1 -- 172.21.15.84:0/1290986088 <== mds.0 v2:172.21.15.84:6835/3839442164 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55cfbf7fd700 con 0x55cfbf8cc020 2023-10-16T10:02:20.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.736+0000 7f1d067fc640 10 client.6135 handle_command_reply: tid=0 2023-10-16T10:02:20.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.736+0000 7f1d309fb740 1 client.6135 shutdown 2023-10-16T10:02:20.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.736+0000 7f1d309fb740 1 -- 172.21.15.84:0/1290986088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6550 msgr2=0x55cfbf8dc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:20.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.736+0000 7f1d309fb740 1 --2- 172.21.15.84:0/1290986088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6550 0x55cfbf8dc610 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1d10058e60 tx=0x7f1d1005d030 comp rx=0 tx=0).stop 2023-10-16T10:02:20.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.736+0000 7f1d309fb740 1 -- 172.21.15.84:0/1290986088 shutdown_connections 2023-10-16T10:02:20.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.736+0000 7f1d309fb740 1 --2- 172.21.15.84:0/1290986088 >> [v2:172.21.15.84:6835/3839442164,v1:172.21.15.84:6837/3839442164] conn(0x55cfbf8cc020 0x55cfbf8cc400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.736+0000 7f1d309fb740 1 --2- 172.21.15.84:0/1290986088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55cfbf8c8940 0x55cfbf94cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.736+0000 7f1d309fb740 1 --2- 172.21.15.84:0/1290986088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfbf8c6550 0x55cfbf8dc610 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.736+0000 7f1d309fb740 1 --2- 172.21.15.84:0/1290986088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbf8c6170 0x55cfbf8e5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.736+0000 7f1d067fc640 0 client.6135 ms_handle_reset on v2:172.21.15.84:6835/3839442164 2023-10-16T10:02:20.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.736+0000 7f1d309fb740 1 -- 172.21.15.84:0/1290986088 >> 172.21.15.84:0/1290986088 conn(0x55cfbf8995b0 msgr2=0x55cfbf8f69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:20.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.736+0000 7f1d309fb740 1 -- 172.21.15.84:0/1290986088 shutdown_connections 2023-10-16T10:02:20.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.736+0000 7f1d309fb740 1 -- 172.21.15.84:0/1290986088 wait complete. 2023-10-16T10:02:20.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.736+0000 7f1d309fb740 20 client.6135 trim_cache size 0 max 16384 2023-10-16T10:02:20.751 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T10:02:20.751 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "63fb4d68-f881-4694-b880-d10369105c0e", 2023-10-16T10:02:20.752 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-16T10:02:20.752 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 5991, 2023-10-16T10:02:20.752 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-16T10:02:20.752 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-16T10:02:20.752 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-16T10:02:20.752 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 6.6757411209999997, 2023-10-16T10:02:20.753 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 25, 2023-10-16T10:02:20.753 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 64, 2023-10-16T10:02:20.753 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 64, 2023-10-16T10:02:20.753 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 13.073809158 2023-10-16T10:02:20.753 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T10:02:20.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.737+0000 7f1d2cb27640 1 -- 172.21.15.84:0/2044434232 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f1d0806afe0 msgr2=0x7f1d0806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:20.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.737+0000 7f1d2cb27640 1 --2- 172.21.15.84:0/2044434232 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f1d0806afe0 0x7f1d0806d4a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1d18000f60 tx=0x7f1d18009040 comp rx=0 tx=0).stop 2023-10-16T10:02:20.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.737+0000 7f1d2cb27640 1 -- 172.21.15.84:0/2044434232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d28070b60 msgr2=0x7f1d281f3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:20.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.737+0000 7f1d2cb27640 1 --2- 172.21.15.84:0/2044434232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d28070b60 0x7f1d281f3260 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000cf60 tx=0x7f1d1000cf90 comp rx=0 tx=0).stop 2023-10-16T10:02:20.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.737+0000 7f1d2cb27640 1 -- 172.21.15.84:0/2044434232 shutdown_connections 2023-10-16T10:02:20.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.737+0000 7f1d2cb27640 1 --2- 172.21.15.84:0/2044434232 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f1d0806afe0 0x7f1d0806d4a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.737+0000 7f1d2cb27640 1 --2- 172.21.15.84:0/2044434232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d28159950 0x7f1d28147980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.737+0000 7f1d2cb27640 1 --2- 172.21.15.84:0/2044434232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d28070b60 0x7f1d281f3260 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.737+0000 7f1d2cb27640 1 --2- 172.21.15.84:0/2044434232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d2806c100 0x7f1d280c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.737+0000 7f1d2cb27640 1 -- 172.21.15.84:0/2044434232 >> 172.21.15.84:0/2044434232 conn(0x7f1d280ce6a0 msgr2=0x7f1d28070450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:20.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.737+0000 7f1d2cb27640 1 -- 172.21.15.84:0/2044434232 shutdown_connections 2023-10-16T10:02:20.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.737+0000 7f1d2cb27640 1 -- 172.21.15.84:0/2044434232 wait complete. 2023-10-16T10:02:20.757 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T10:02:20.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.970+0000 7f7cfbfff640 1 Processor -- start 2023-10-16T10:02:20.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.971+0000 7f7cfbfff640 1 -- start start 2023-10-16T10:02:20.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.972+0000 7f7cfbfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7cfc15d100 0x7f7cfc0b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.972+0000 7f7cfbfff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc0b8430 con 0x7f7cfc0b1810 2023-10-16T10:02:20.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.972+0000 7f7cfbfff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc0d2160 con 0x7f7cfc15cd20 2023-10-16T10:02:20.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.973+0000 7f7cfbfff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cfc0b8680 con 0x7f7cfc15d100 2023-10-16T10:02:20.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.973+0000 7f7cfb7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7cfc0b1810 0x7f7cfc0d06e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34254/0 (socket says 172.21.15.84:34254) 2023-10-16T10:02:20.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.973+0000 7f7cfb7fe640 1 -- 172.21.15.84:0/3414049864 learned_addr learned my addr 172.21.15.84:0/3414049864 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:20.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.973+0000 7f7cf3fff640 1 --2- 172.21.15.84:0/3414049864 >> v2:172.21.15.157:3300/0 conn(0x7f7cfc15d100 0x7f7cfc0b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.973+0000 7f7cfa7fc640 1 -- 172.21.15.84:0/3414049864 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4064622278 0 0) 0x7f7cfc0b8430 con 0x7f7cfc0b1810 2023-10-16T10:02:20.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.973+0000 7f7cfa7fc640 1 -- 172.21.15.84:0/3414049864 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ce8003520 con 0x7f7cfc0b1810 2023-10-16T10:02:20.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.974+0000 7f7cfa7fc640 1 -- 172.21.15.84:0/3414049864 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4197897148 0 0) 0x7f7ce8003520 con 0x7f7cfc0b1810 2023-10-16T10:02:20.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.974+0000 7f7cfa7fc640 1 -- 172.21.15.84:0/3414049864 >> v2:172.21.15.157:3300/0 conn(0x7f7cfc15d100 msgr2=0x7f7cfc0b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:20.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.974+0000 7f7cfa7fc640 1 --2- 172.21.15.84:0/3414049864 >> v2:172.21.15.157:3300/0 conn(0x7f7cfc15d100 0x7f7cfc0b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.974+0000 7f7cfa7fc640 1 -- 172.21.15.84:0/3414049864 >> v1:172.21.15.157:6789/0 conn(0x7f7cfc15cd20 legacy=0x7f7cfc0cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:20.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.974+0000 7f7cfa7fc640 1 -- 172.21.15.84:0/3414049864 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc0b8960 con 0x7f7cfc0b1810 2023-10-16T10:02:20.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.974+0000 7f7cfa7fc640 1 -- 172.21.15.84:0/3414049864 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f7cf40019f0 con 0x7f7cfc0b1810 2023-10-16T10:02:20.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.974+0000 7f7cfa7fc640 1 -- 172.21.15.84:0/3414049864 >> v1:172.21.15.84:6789/0 conn(0x7f7cfc0b1810 legacy=0x7f7cfc0d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:20.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.974+0000 7f7cfa7fc640 1 --2- 172.21.15.84:0/3414049864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ce80039c0 0x7f7ce8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.974+0000 7f7cfa7fc640 1 --2- 172.21.15.84:0/3414049864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce80047c0 0x7f7ce8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.974+0000 7f7cfa7fc640 1 --2- 172.21.15.84:0/3414049864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ce8005180 0x7f7ce800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.974+0000 7f7cfa7fc640 1 -- 172.21.15.84:0/3414049864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0b8680 con 0x7f7ce80039c0 2023-10-16T10:02:20.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.974+0000 7f7cfa7fc640 1 -- 172.21.15.84:0/3414049864 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce800c060 con 0x7f7ce80047c0 2023-10-16T10:02:20.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.974+0000 7f7cfa7fc640 1 -- 172.21.15.84:0/3414049864 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce800c220 con 0x7f7ce8005180 2023-10-16T10:02:20.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.974+0000 7f7cfaffd640 1 --2- 172.21.15.84:0/3414049864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ce80039c0 0x7f7ce8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.975+0000 7f7cfb7fe640 1 --2- 172.21.15.84:0/3414049864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ce8005180 0x7f7ce800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.975+0000 7f7cf3fff640 1 --2- 172.21.15.84:0/3414049864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce80047c0 0x7f7ce8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.975+0000 7f7cfaffd640 1 -- 172.21.15.84:0/3414049864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ce8005180 msgr2=0x7f7ce800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:20.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.975+0000 7f7cfaffd640 1 --2- 172.21.15.84:0/3414049864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ce8005180 0x7f7ce800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.975+0000 7f7cfaffd640 1 -- 172.21.15.84:0/3414049864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce80047c0 msgr2=0x7f7ce8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:20.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.975+0000 7f7cfaffd640 1 --2- 172.21.15.84:0/3414049864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce80047c0 0x7f7ce8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.975+0000 7f7cfaffd640 1 -- 172.21.15.84:0/3414049864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce8003520 con 0x7f7ce80039c0 2023-10-16T10:02:20.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.975+0000 7f7cfb7fe640 1 --2- 172.21.15.84:0/3414049864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ce8005180 0x7f7ce800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:20.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.975+0000 7f7cfb7fe640 1 -- 172.21.15.84:0/3414049864 reap_dead start 2023-10-16T10:02:20.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.976+0000 7f7cfaffd640 1 --2- 172.21.15.84:0/3414049864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ce80039c0 0x7f7ce8003de0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7cec001750 tx=0x7f7ce80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a60c3a4a4165968 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:20.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.976+0000 7f7cfa7fc640 1 -- 172.21.15.84:0/3414049864 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec0052d0 con 0x7f7ce80039c0 2023-10-16T10:02:20.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.976+0000 7f7cfa7fc640 1 -- 172.21.15.84:0/3414049864 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cec02e050 con 0x7f7ce80039c0 2023-10-16T10:02:20.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.976+0000 7f7cfa7fc640 1 -- 172.21.15.84:0/3414049864 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec003830 con 0x7f7ce80039c0 2023-10-16T10:02:20.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.977+0000 7f7cfbfff640 1 -- 172.21.15.84:0/3414049864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ce80039c0 msgr2=0x7f7ce8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:20.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.977+0000 7f7cfbfff640 1 --2- 172.21.15.84:0/3414049864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ce80039c0 0x7f7ce8003de0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7cec001750 tx=0x7f7ce80057f0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.977+0000 7f7cfbfff640 1 -- 172.21.15.84:0/3414049864 shutdown_connections 2023-10-16T10:02:20.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.977+0000 7f7cfbfff640 1 --2- 172.21.15.84:0/3414049864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ce80039c0 0x7f7ce8003de0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.977+0000 7f7cfbfff640 1 -- 172.21.15.84:0/3414049864 >> 172.21.15.84:0/3414049864 conn(0x7f7cfc0cf590 msgr2=0x7f7cfc0cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:20.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.977+0000 7f7cfbfff640 1 -- 172.21.15.84:0/3414049864 shutdown_connections 2023-10-16T10:02:20.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.977+0000 7f7cfbfff640 1 -- 172.21.15.84:0/3414049864 wait complete. 2023-10-16T10:02:20.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.979+0000 7f7cfbfff640 1 Processor -- start 2023-10-16T10:02:20.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.979+0000 7f7cfbfff640 1 -- start start 2023-10-16T10:02:20.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.980+0000 7f7cfbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce80039c0 0x7f7cfc0c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.980+0000 7f7cfbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cfc14efe0 0x7f7cfc15d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.980+0000 7f7cfbfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cfc15dca0 0x7f7cfc0c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.980+0000 7f7cfbfff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc15e120 con 0x7f7cfc14efe0 2023-10-16T10:02:20.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.980+0000 7f7cfbfff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0b2250 con 0x7f7ce80039c0 2023-10-16T10:02:20.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.980+0000 7f7cfbfff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0c7110 con 0x7f7cfc15dca0 2023-10-16T10:02:20.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.980+0000 7f7cf3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cfc14efe0 0x7f7cfc15d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.980+0000 7f7cf3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cfc14efe0 0x7f7cfc15d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44800/0 (socket says 172.21.15.84:44800) 2023-10-16T10:02:20.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.980+0000 7f7cf3fff640 1 -- 172.21.15.84:0/2980587512 learned_addr learned my addr 172.21.15.84:0/2980587512 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:20.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.980+0000 7f7cfaffd640 1 --2- 172.21.15.84:0/2980587512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce80039c0 0x7f7cfc0c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.980+0000 7f7cfb7fe640 1 --2- 172.21.15.84:0/2980587512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cfc15dca0 0x7f7cfc0c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.980+0000 7f7cf3fff640 1 -- 172.21.15.84:0/2980587512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cfc15dca0 msgr2=0x7f7cfc0c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:20.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.980+0000 7f7cf3fff640 1 --2- 172.21.15.84:0/2980587512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cfc15dca0 0x7f7cfc0c7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.980+0000 7f7cf3fff640 1 -- 172.21.15.84:0/2980587512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce80039c0 msgr2=0x7f7cfc0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:20.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.980+0000 7f7cf3fff640 1 --2- 172.21.15.84:0/2980587512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce80039c0 0x7f7cfc0c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:20.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.980+0000 7f7cf3fff640 1 -- 172.21.15.84:0/2980587512 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc0cc120 con 0x7f7cfc14efe0 2023-10-16T10:02:20.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.981+0000 7f7cf3fff640 1 --2- 172.21.15.84:0/2980587512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cfc14efe0 0x7f7cfc15d760 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4010910 tx=0x7f7ce4010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.981+0000 7f7cf37fe640 1 -- 172.21.15.84:0/2980587512 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4009690 con 0x7f7cfc14efe0 2023-10-16T10:02:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.981+0000 7f7cf37fe640 1 -- 172.21.15.84:0/2980587512 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce400ec20 con 0x7f7cfc14efe0 2023-10-16T10:02:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.981+0000 7f7cf37fe640 1 -- 172.21.15.84:0/2980587512 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce401c400 con 0x7f7cfc14efe0 2023-10-16T10:02:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.981+0000 7f7cfbfff640 1 -- 172.21.15.84:0/2980587512 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc0cc400 con 0x7f7cfc14efe0 2023-10-16T10:02:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.982+0000 7f7cfbfff640 1 -- 172.21.15.84:0/2980587512 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc0c7640 con 0x7f7cfc14efe0 2023-10-16T10:02:20.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.983+0000 7f7cfbfff640 1 -- 172.21.15.84:0/2980587512 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc04e9b0 con 0x7f7cfc14efe0 2023-10-16T10:02:20.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.984+0000 7f7cf37fe640 1 -- 172.21.15.84:0/2980587512 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7ce401c560 con 0x7f7cfc14efe0 2023-10-16T10:02:20.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.985+0000 7f7cf37fe640 1 --2- 172.21.15.84:0/2980587512 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f7cdc06b010 0x7f7cdc06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:20.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.985+0000 7f7cf37fe640 1 -- 172.21.15.84:0/2980587512 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ce4089c70 con 0x7f7cfc14efe0 2023-10-16T10:02:20.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.985+0000 7f7cfaffd640 1 --2- 172.21.15.84:0/2980587512 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f7cdc06b010 0x7f7cdc06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:20.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.985+0000 7f7cfaffd640 1 --2- 172.21.15.84:0/2980587512 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f7cdc06b010 0x7f7cdc06d4d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7cec033000 tx=0x7f7cec009030 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:20.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:20.989+0000 7f7cf37fe640 1 -- 172.21.15.84:0/2980587512 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7ce4054d30 con 0x7f7cfc14efe0 2023-10-16T10:02:21.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:21.223+0000 7f7cfbfff640 1 -- 172.21.15.84:0/2980587512 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7cfc0c60b0 con 0x7f7cfc14efe0 2023-10-16T10:02:21.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:21.223+0000 7f7cf37fe640 1 -- 172.21.15.84:0/2980587512 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7ce4089070 con 0x7f7cfc14efe0 2023-10-16T10:02:21.225 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T10:02:21.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:21.226+0000 7f7cfbfff640 1 -- 172.21.15.84:0/2980587512 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f7cdc06b010 msgr2=0x7f7cdc06d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:21.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:21.226+0000 7f7cfbfff640 1 --2- 172.21.15.84:0/2980587512 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f7cdc06b010 0x7f7cdc06d4d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7cec033000 tx=0x7f7cec009030 comp rx=0 tx=0).stop 2023-10-16T10:02:21.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:21.226+0000 7f7cfbfff640 1 -- 172.21.15.84:0/2980587512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cfc14efe0 msgr2=0x7f7cfc15d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:21.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:21.226+0000 7f7cfbfff640 1 --2- 172.21.15.84:0/2980587512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cfc14efe0 0x7f7cfc15d760 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4010910 tx=0x7f7ce4010dd0 comp rx=0 tx=0).stop 2023-10-16T10:02:21.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:21.226+0000 7f7cfbfff640 1 -- 172.21.15.84:0/2980587512 shutdown_connections 2023-10-16T10:02:21.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:21.226+0000 7f7cfbfff640 1 --2- 172.21.15.84:0/2980587512 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f7cdc06b010 0x7f7cdc06d4d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:21.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:21.226+0000 7f7cfbfff640 1 --2- 172.21.15.84:0/2980587512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cfc15dca0 0x7f7cfc0c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:21.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:21.226+0000 7f7cfbfff640 1 --2- 172.21.15.84:0/2980587512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cfc14efe0 0x7f7cfc15d760 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:21.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:21.226+0000 7f7cfbfff640 1 --2- 172.21.15.84:0/2980587512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce80039c0 0x7f7cfc0c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:21.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:21.226+0000 7f7cfbfff640 1 -- 172.21.15.84:0/2980587512 >> 172.21.15.84:0/2980587512 conn(0x7f7cfc0cf590 msgr2=0x7f7cfc0b6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:21.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:21.226+0000 7f7cfbfff640 1 -- 172.21.15.84:0/2980587512 shutdown_connections 2023-10-16T10:02:21.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:21.227+0000 7f7cfbfff640 1 -- 172.21.15.84:0/2980587512 wait complete. 2023-10-16T10:02:21.246 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T10:02:21.246 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T10:02:21.247 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-16T10:02:21.247 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T10:02:21.247 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T10:02:21.248 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T10:02:21.248 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T10:02:21.248 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T10:02:21.249 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:02:21.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:21.250 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-16T10:02:21.300 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T10:02:21.300 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T10:02:21.300 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T10:02:21.301 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:02:21.301 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T10:02:21.301 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:02:21.301 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:02:21.301 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:02:21.302 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-16T10:02:21.302 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:02:21.302 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:02:21.302 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-16T10:02:21.302 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:02:21.303 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:02:21.303 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T10:02:21.303 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:02:21.303 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T10:02:21.303 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 41610sec preferred_lft 41610sec 2023-10-16T10:02:21.304 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-16T10:02:21.304 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:02:21.304 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T10:02:21.304 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:02:21.304 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T10:02:21.305 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:02:21.305 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::20a8:eaff:fe9d:5cea/64 scope link 2023-10-16T10:02:21.305 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:02:21.305 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:02:21.305 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T10:02:21.306 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1854:13ff:fe47:be92/64 scope link 2023-10-16T10:02:21.306 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:02:21.306 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:02:21.306 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether ee:d5:41:a3:a5:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T10:02:21.306 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ecd5:41ff:fea3:a570/64 scope link 2023-10-16T10:02:21.307 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:02:21.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:21.308 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T10:02:21.398 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-16T10:02:21.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:21.403 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-16T10:02:21.458 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-16T10:02:21.458 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T10:02:21.459 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T10:02:21.459 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T10:02:21.459 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T10:02:21.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:21.460 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T10:02:21.460 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T10:02:21.461 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T10:02:21.461 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T10:02:21.461 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T10:02:21.462 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T10:02:21.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:21.647 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T10:02:21.698 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T10:02:21.699 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T10:02:21.699 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:02:21.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:21.700 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-16T10:02:21.794 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:02:21.795 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T10:02:21.852 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:02:21.853 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:02:21.854 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:02:21.907 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-16T10:02:21.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:21.908 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:02:22.053 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:02:22.053 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T10:02:22.084 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:02:22.084 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T10:02:22.084 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T10:02:22.084 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with new device syntax: 0@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T10:02:22.085 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T10:02:22.085 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-16T10:02:22.086 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2023-10-16T10:02:22.086 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T10:02:22.086 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T10:02:22.086 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T10:02:22.089 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T10:02:22.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:22.090 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:02:22.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:22.171 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T10:02:22.234 INFO:teuthology.orchestra.run.smithi084.stdout:client6144 2023-10-16T10:02:22.234 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:02:22.235 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:02:22.284 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T10:02:22.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.547+0000 7f4fae521640 1 Processor -- start 2023-10-16T10:02:22.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.547+0000 7f4fae521640 1 -- start start 2023-10-16T10:02:22.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.549+0000 7f4fae521640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4fa8070b60 0x7f4fa815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:22.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.549+0000 7f4fae521640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4fa806c100 0x7f4fa806c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:22.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.549+0000 7f4fae521640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa815aef0 con 0x7f4fa8159950 2023-10-16T10:02:22.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.549+0000 7f4fae521640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa815e490 con 0x7f4fa8070b60 2023-10-16T10:02:22.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.549+0000 7f4fae521640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa815d910 con 0x7f4fa806c100 2023-10-16T10:02:22.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.549+0000 7f4facd1e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4fa8070b60 0x7f4fa815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:22.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.549+0000 7f4facd1e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4fa8070b60 0x7f4fa815a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44808/0 (socket says 172.21.15.84:44808) 2023-10-16T10:02:22.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.549+0000 7f4facd1e640 1 -- 172.21.15.84:0/396529728 learned_addr learned my addr 172.21.15.84:0/396529728 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:22.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.549+0000 7f4fadd20640 1 --2- 172.21.15.84:0/396529728 >> v2:172.21.15.157:3300/0 conn(0x7f4fa806c100 0x7f4fa806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:22.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.550+0000 7f4facd1e640 1 -- 172.21.15.84:0/396529728 >> v2:172.21.15.157:3300/0 conn(0x7f4fa806c100 msgr2=0x7f4fa806c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:22.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.550+0000 7f4facd1e640 1 --2- 172.21.15.84:0/396529728 >> v2:172.21.15.157:3300/0 conn(0x7f4fa806c100 0x7f4fa806c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:22.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.550+0000 7f4facd1e640 1 -- 172.21.15.84:0/396529728 >> v1:172.21.15.157:6789/0 conn(0x7f4fa8159950 legacy=0x7f4fa8070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:22.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.550+0000 7f4facd1e640 1 -- 172.21.15.84:0/396529728 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa815dbf0 con 0x7f4fa8070b60 2023-10-16T10:02:22.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.551+0000 7f4facd1e640 1 --2- 172.21.15.84:0/396529728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fa8070b60 0x7f4fa815a9b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80ade70 tx=0x7f4f9802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de15630323dcddce server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:22.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.551+0000 7f4f9ffff640 1 -- 172.21.15.84:0/396529728 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f980317e0 con 0x7f4fa8070b60 2023-10-16T10:02:22.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.551+0000 7f4f9ffff640 1 -- 172.21.15.84:0/396529728 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f98031940 con 0x7f4fa8070b60 2023-10-16T10:02:22.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.551+0000 7f4f9ffff640 1 -- 172.21.15.84:0/396529728 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f98031cd0 con 0x7f4fa8070b60 2023-10-16T10:02:22.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.552+0000 7f4fae521640 1 -- 172.21.15.84:0/396529728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fa8070b60 msgr2=0x7f4fa815a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:22.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.552+0000 7f4fae521640 1 --2- 172.21.15.84:0/396529728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fa8070b60 0x7f4fa815a9b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80ade70 tx=0x7f4f9802b880 comp rx=0 tx=0).stop 2023-10-16T10:02:22.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.552+0000 7f4fae521640 1 -- 172.21.15.84:0/396529728 shutdown_connections 2023-10-16T10:02:22.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.552+0000 7f4fae521640 1 --2- 172.21.15.84:0/396529728 >> v2:172.21.15.157:3300/0 conn(0x7f4fa806c100 0x7f4fa806c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:22.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.552+0000 7f4fae521640 1 --2- 172.21.15.84:0/396529728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fa8070b60 0x7f4fa815a9b0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:22.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.552+0000 7f4fae521640 1 -- 172.21.15.84:0/396529728 >> 172.21.15.84:0/396529728 conn(0x7f4fa80ce6a0 msgr2=0x7f4fa80ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:22.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.552+0000 7f4fae521640 1 -- 172.21.15.84:0/396529728 shutdown_connections 2023-10-16T10:02:22.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.552+0000 7f4fae521640 1 -- 172.21.15.84:0/396529728 wait complete. 2023-10-16T10:02:22.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.554+0000 7f4fae521640 1 Processor -- start 2023-10-16T10:02:22.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.554+0000 7f4fae521640 1 -- start start 2023-10-16T10:02:22.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.556+0000 7f4fae521640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fa806c100 0x7f4fa8153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:22.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.556+0000 7f4fae521640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fa8070b60 0x7f4fa8151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:22.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.556+0000 7f4fae521640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fa8159950 0x7f4fa80c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:22.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.556+0000 7f4fae521640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa8153e20 con 0x7f4fa8159950 2023-10-16T10:02:22.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.556+0000 7f4fae521640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa8154100 con 0x7f4fa8070b60 2023-10-16T10:02:22.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.556+0000 7f4fae521640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa81543e0 con 0x7f4fa806c100 2023-10-16T10:02:22.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.556+0000 7f4fadd20640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fa8159950 0x7f4fa80c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:22.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.556+0000 7f4fadd20640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fa8159950 0x7f4fa80c4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44824/0 (socket says 172.21.15.84:44824) 2023-10-16T10:02:22.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.556+0000 7f4fadd20640 1 -- 172.21.15.84:0/2902790071 learned_addr learned my addr 172.21.15.84:0/2902790071 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:22.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.556+0000 7f4fad51f640 1 --2- 172.21.15.84:0/2902790071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fa806c100 0x7f4fa8153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:22.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.556+0000 7f4facd1e640 1 --2- 172.21.15.84:0/2902790071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fa8070b60 0x7f4fa8151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:22.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.556+0000 7f4fadd20640 1 -- 172.21.15.84:0/2902790071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fa806c100 msgr2=0x7f4fa8153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:22.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.556+0000 7f4fadd20640 1 --2- 172.21.15.84:0/2902790071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fa806c100 0x7f4fa8153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:22.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.556+0000 7f4fadd20640 1 -- 172.21.15.84:0/2902790071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fa8070b60 msgr2=0x7f4fa8151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:22.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.556+0000 7f4fadd20640 1 --2- 172.21.15.84:0/2902790071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fa8070b60 0x7f4fa8151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:22.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.556+0000 7f4fadd20640 1 -- 172.21.15.84:0/2902790071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f980094d0 con 0x7f4fa8159950 2023-10-16T10:02:22.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.557+0000 7f4fadd20640 1 --2- 172.21.15.84:0/2902790071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fa8159950 0x7f4fa80c4ef0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0007e40 tx=0x7f4fa000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:22.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.557+0000 7f4f9dffb640 1 -- 172.21.15.84:0/2902790071 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0014620 con 0x7f4fa8159950 2023-10-16T10:02:22.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.557+0000 7f4f9dffb640 1 -- 172.21.15.84:0/2902790071 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa000ee90 con 0x7f4fa8159950 2023-10-16T10:02:22.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.557+0000 7f4fae521640 1 -- 172.21.15.84:0/2902790071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa81546c0 con 0x7f4fa8159950 2023-10-16T10:02:22.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.557+0000 7f4f9dffb640 1 -- 172.21.15.84:0/2902790071 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa00149e0 con 0x7f4fa8159950 2023-10-16T10:02:22.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.557+0000 7f4fae521640 1 -- 172.21.15.84:0/2902790071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa80c5430 con 0x7f4fa8159950 2023-10-16T10:02:22.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.558+0000 7f4fae521640 1 -- 172.21.15.84:0/2902790071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa804e9b0 con 0x7f4fa8159950 2023-10-16T10:02:22.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.559+0000 7f4f9dffb640 1 -- 172.21.15.84:0/2902790071 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4fa0014b40 con 0x7f4fa8159950 2023-10-16T10:02:22.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.560+0000 7f4f9dffb640 1 --2- 172.21.15.84:0/2902790071 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f4f8c06af90 0x7f4f8c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:22.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.560+0000 7f4f9dffb640 1 -- 172.21.15.84:0/2902790071 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fa008a1c0 con 0x7f4fa8159950 2023-10-16T10:02:22.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.560+0000 7f4fad51f640 1 --2- 172.21.15.84:0/2902790071 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f4f8c06af90 0x7f4f8c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:22.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.561+0000 7f4fad51f640 1 --2- 172.21.15.84:0/2902790071 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f4f8c06af90 0x7f4f8c06d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4006fd0 tx=0x7f4fa4008040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:22.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.564+0000 7f4f9dffb640 1 -- 172.21.15.84:0/2902790071 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4fa0055150 con 0x7f4fa8159950 2023-10-16T10:02:22.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.781+0000 7f4fae521640 1 -- 172.21.15.84:0/2902790071 --> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fa815e1b0 con 0x7f4f8c06af90 2023-10-16T10:02:22.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.784+0000 7f4f9dffb640 1 -- 172.21.15.84:0/2902790071 <== mgr.5742 v2:172.21.15.84:6832/585186562 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f4fa400bda0 con 0x7f4f8c06af90 2023-10-16T10:02:22.786 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T10:02:22.786 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:02:22.786 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephfs" 2023-10-16T10:02:22.787 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:02:22.787 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T10:02:22.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.787+0000 7f4fae521640 1 -- 172.21.15.84:0/2902790071 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f4f8c06af90 msgr2=0x7f4f8c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:22.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.787+0000 7f4fae521640 1 --2- 172.21.15.84:0/2902790071 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f4f8c06af90 0x7f4f8c06d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4006fd0 tx=0x7f4fa4008040 comp rx=0 tx=0).stop 2023-10-16T10:02:22.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.787+0000 7f4fae521640 1 -- 172.21.15.84:0/2902790071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fa8159950 msgr2=0x7f4fa80c4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:22.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.787+0000 7f4fae521640 1 --2- 172.21.15.84:0/2902790071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fa8159950 0x7f4fa80c4ef0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0007e40 tx=0x7f4fa000e4b0 comp rx=0 tx=0).stop 2023-10-16T10:02:22.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.787+0000 7f4fae521640 1 -- 172.21.15.84:0/2902790071 shutdown_connections 2023-10-16T10:02:22.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.787+0000 7f4fae521640 1 --2- 172.21.15.84:0/2902790071 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f4f8c06af90 0x7f4f8c06d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:22.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.787+0000 7f4fae521640 1 --2- 172.21.15.84:0/2902790071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fa8159950 0x7f4fa80c4ef0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:22.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.787+0000 7f4fae521640 1 --2- 172.21.15.84:0/2902790071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fa8070b60 0x7f4fa8151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:22.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.787+0000 7f4fae521640 1 --2- 172.21.15.84:0/2902790071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fa806c100 0x7f4fa8153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:22.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.787+0000 7f4fae521640 1 -- 172.21.15.84:0/2902790071 >> 172.21.15.84:0/2902790071 conn(0x7f4fa80ce6a0 msgr2=0x7f4fa80c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:22.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.787+0000 7f4fae521640 1 -- 172.21.15.84:0/2902790071 shutdown_connections 2023-10-16T10:02:22.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:22.787+0000 7f4fae521640 1 -- 172.21.15.84:0/2902790071 wait complete. 2023-10-16T10:02:22.807 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T10:02:23.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.014+0000 7f88844bb640 1 Processor -- start 2023-10-16T10:02:23.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.014+0000 7f88844bb640 1 -- start start 2023-10-16T10:02:23.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.016+0000 7f88844bb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f887c0caa80 0x7f887c069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:23.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.016+0000 7f88844bb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f887c06a2e0 0x7f887c1650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:23.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.016+0000 7f88844bb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887c0caef0 con 0x7f887c0addb0 2023-10-16T10:02:23.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.016+0000 7f88844bb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887c161a50 con 0x7f887c06a2e0 2023-10-16T10:02:23.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.016+0000 7f88844bb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887c06a6f0 con 0x7f887c0caa80 2023-10-16T10:02:23.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.016+0000 7f8882a58640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f887c06a2e0 0x7f887c1650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:23.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.016+0000 7f8882a58640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f887c06a2e0 0x7f887c1650d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44830/0 (socket says 172.21.15.84:44830) 2023-10-16T10:02:23.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.016+0000 7f8882a58640 1 -- 172.21.15.84:0/2430824699 learned_addr learned my addr 172.21.15.84:0/2430824699 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:23.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.017+0000 7f8881a56640 1 --2- 172.21.15.84:0/2430824699 >> v2:172.21.15.157:3300/0 conn(0x7f887c0caa80 0x7f887c069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:23.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.017+0000 7f8882a58640 1 -- 172.21.15.84:0/2430824699 >> v2:172.21.15.157:3300/0 conn(0x7f887c0caa80 msgr2=0x7f887c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:23.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.017+0000 7f8882a58640 1 --2- 172.21.15.84:0/2430824699 >> v2:172.21.15.157:3300/0 conn(0x7f887c0caa80 0x7f887c069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:23.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.017+0000 7f8882a58640 1 -- 172.21.15.84:0/2430824699 >> v1:172.21.15.157:6789/0 conn(0x7f887c0addb0 legacy=0x7f887c0c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:23.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.017+0000 7f8882a58640 1 -- 172.21.15.84:0/2430824699 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c0cb140 con 0x7f887c06a2e0 2023-10-16T10:02:23.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.017+0000 7f8881a56640 1 --2- 172.21.15.84:0/2430824699 >> v2:172.21.15.157:3300/0 conn(0x7f887c0caa80 0x7f887c069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:23.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.018+0000 7f8882a58640 1 --2- 172.21.15.84:0/2430824699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f887c06a2e0 0x7f887c1650d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f887c0cdbe0 tx=0x7f887402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2409f1b1858dcd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:23.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.018+0000 7f8881255640 1 -- 172.21.15.84:0/2430824699 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874031870 con 0x7f887c06a2e0 2023-10-16T10:02:23.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.018+0000 7f8881255640 1 -- 172.21.15.84:0/2430824699 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88740319d0 con 0x7f887c06a2e0 2023-10-16T10:02:23.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.018+0000 7f8881255640 1 -- 172.21.15.84:0/2430824699 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874031d60 con 0x7f887c06a2e0 2023-10-16T10:02:23.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.019+0000 7f88844bb640 1 -- 172.21.15.84:0/2430824699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f887c06a2e0 msgr2=0x7f887c1650d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:23.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.019+0000 7f88844bb640 1 --2- 172.21.15.84:0/2430824699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f887c06a2e0 0x7f887c1650d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f887c0cdbe0 tx=0x7f887402b910 comp rx=0 tx=0).stop 2023-10-16T10:02:23.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.019+0000 7f88844bb640 1 -- 172.21.15.84:0/2430824699 shutdown_connections 2023-10-16T10:02:23.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.019+0000 7f88844bb640 1 --2- 172.21.15.84:0/2430824699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f887c06a2e0 0x7f887c1650d0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:23.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.019+0000 7f88844bb640 1 --2- 172.21.15.84:0/2430824699 >> v2:172.21.15.157:3300/0 conn(0x7f887c0caa80 0x7f887c069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:23.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.019+0000 7f88844bb640 1 -- 172.21.15.84:0/2430824699 >> 172.21.15.84:0/2430824699 conn(0x7f887c0ce9b0 msgr2=0x7f887c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:23.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.019+0000 7f88844bb640 1 -- 172.21.15.84:0/2430824699 shutdown_connections 2023-10-16T10:02:23.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.019+0000 7f88844bb640 1 -- 172.21.15.84:0/2430824699 wait complete. 2023-10-16T10:02:23.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.021+0000 7f88844bb640 1 Processor -- start 2023-10-16T10:02:23.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.021+0000 7f88844bb640 1 -- start start 2023-10-16T10:02:23.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.022+0000 7f88844bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f887c06a2e0 0x7f887c0b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:23.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.022+0000 7f88844bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f887c0addb0 0x7f887c0b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:23.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.022+0000 7f88844bb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f887c0caa80 0x7f887c0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:23.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.022+0000 7f88844bb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c1f9b90 con 0x7f887c0caa80 2023-10-16T10:02:23.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.022+0000 7f88844bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c1fa9b0 con 0x7f887c0addb0 2023-10-16T10:02:23.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.022+0000 7f88844bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c1f9da0 con 0x7f887c06a2e0 2023-10-16T10:02:23.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.022+0000 7f8882a58640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f887c0caa80 0x7f887c0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:23.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.022+0000 7f8882a58640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f887c0caa80 0x7f887c0b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44832/0 (socket says 172.21.15.84:44832) 2023-10-16T10:02:23.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.022+0000 7f8882a58640 1 -- 172.21.15.84:0/2907494722 learned_addr learned my addr 172.21.15.84:0/2907494722 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:23.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.023+0000 7f8881a56640 1 --2- 172.21.15.84:0/2907494722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f887c0addb0 0x7f887c0b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:23.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.023+0000 7f8882257640 1 --2- 172.21.15.84:0/2907494722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f887c06a2e0 0x7f887c0b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:23.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.023+0000 7f8882a58640 1 -- 172.21.15.84:0/2907494722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f887c06a2e0 msgr2=0x7f887c0b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:23.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.023+0000 7f8882a58640 1 --2- 172.21.15.84:0/2907494722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f887c06a2e0 0x7f887c0b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:23.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.023+0000 7f8882a58640 1 -- 172.21.15.84:0/2907494722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f887c0addb0 msgr2=0x7f887c0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:23.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.023+0000 7f8882a58640 1 --2- 172.21.15.84:0/2907494722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f887c0addb0 0x7f887c0b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:23.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.023+0000 7f8882a58640 1 -- 172.21.15.84:0/2907494722 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88740094d0 con 0x7f887c0caa80 2023-10-16T10:02:23.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.023+0000 7f8882257640 1 --2- 172.21.15.84:0/2907494722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f887c06a2e0 0x7f887c0b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:23.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.023+0000 7f8882a58640 1 --2- 172.21.15.84:0/2907494722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f887c0caa80 0x7f887c0b4040 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8874009960 tx=0x7f88740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:23.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.024+0000 7f8872ffd640 1 -- 172.21.15.84:0/2907494722 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88740443f0 con 0x7f887c0caa80 2023-10-16T10:02:23.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.024+0000 7f8872ffd640 1 -- 172.21.15.84:0/2907494722 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8874044570 con 0x7f887c0caa80 2023-10-16T10:02:23.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.024+0000 7f8872ffd640 1 -- 172.21.15.84:0/2907494722 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874044920 con 0x7f887c0caa80 2023-10-16T10:02:23.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.024+0000 7f88844bb640 1 -- 172.21.15.84:0/2907494722 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887c1fa080 con 0x7f887c0caa80 2023-10-16T10:02:23.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.024+0000 7f88844bb640 1 -- 172.21.15.84:0/2907494722 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887c1fa640 con 0x7f887c0caa80 2023-10-16T10:02:23.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.025+0000 7f88844bb640 1 -- 172.21.15.84:0/2907494722 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887c04e160 con 0x7f887c0caa80 2023-10-16T10:02:23.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.026+0000 7f8872ffd640 1 -- 172.21.15.84:0/2907494722 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8874044a80 con 0x7f887c0caa80 2023-10-16T10:02:23.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.027+0000 7f8872ffd640 1 --2- 172.21.15.84:0/2907494722 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f886006af90 0x7f886006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:23.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.028+0000 7f8872ffd640 1 -- 172.21.15.84:0/2907494722 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f887402d040 con 0x7f887c0caa80 2023-10-16T10:02:23.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.028+0000 7f8882257640 1 --2- 172.21.15.84:0/2907494722 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f886006af90 0x7f886006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:23.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.028+0000 7f8882257640 1 --2- 172.21.15.84:0/2907494722 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f886006af90 0x7f886006d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8878001dc0 tx=0x7f8878009040 comp rx=0 tx=0).ready entity=mgr.5742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:23.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.032+0000 7f8872ffd640 1 -- 172.21.15.84:0/2907494722 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8874076f60 con 0x7f887c0caa80 2023-10-16T10:02:23.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.275+0000 7f88844bb640 1 -- 172.21.15.84:0/2907494722 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f887c0c7450 con 0x7f887c0caa80 2023-10-16T10:02:23.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.857+0000 7f8872ffd640 1 -- 172.21.15.84:0/2907494722 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v22) v1 ==== 92+0+0 (secure 0 0 0) 0x7f887407a580 con 0x7f887c0caa80 2023-10-16T10:02:23.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.860+0000 7f88844bb640 1 -- 172.21.15.84:0/2907494722 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f886006af90 msgr2=0x7f886006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:23.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.860+0000 7f88844bb640 1 --2- 172.21.15.84:0/2907494722 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f886006af90 0x7f886006d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8878001dc0 tx=0x7f8878009040 comp rx=0 tx=0).stop 2023-10-16T10:02:23.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.860+0000 7f88844bb640 1 -- 172.21.15.84:0/2907494722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f887c0caa80 msgr2=0x7f887c0b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:23.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.860+0000 7f88844bb640 1 --2- 172.21.15.84:0/2907494722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f887c0caa80 0x7f887c0b4040 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8874009960 tx=0x7f88740046c0 comp rx=0 tx=0).stop 2023-10-16T10:02:23.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.861+0000 7f88844bb640 1 -- 172.21.15.84:0/2907494722 shutdown_connections 2023-10-16T10:02:23.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.861+0000 7f88844bb640 1 --2- 172.21.15.84:0/2907494722 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f886006af90 0x7f886006d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:23.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.861+0000 7f88844bb640 1 --2- 172.21.15.84:0/2907494722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f887c0caa80 0x7f887c0b4040 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:23.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.861+0000 7f88844bb640 1 --2- 172.21.15.84:0/2907494722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f887c0addb0 0x7f887c0b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:23.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.861+0000 7f88844bb640 1 --2- 172.21.15.84:0/2907494722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f887c06a2e0 0x7f887c0b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:23.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.861+0000 7f88844bb640 1 -- 172.21.15.84:0/2907494722 >> 172.21.15.84:0/2907494722 conn(0x7f887c0ce9b0 msgr2=0x7f887c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:23.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.861+0000 7f88844bb640 1 -- 172.21.15.84:0/2907494722 shutdown_connections 2023-10-16T10:02:23.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:23.862+0000 7f88844bb640 1 -- 172.21.15.84:0/2907494722 wait complete. 2023-10-16T10:02:23.896 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T10:02:24.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.097+0000 7fc11dc7c640 1 Processor -- start 2023-10-16T10:02:24.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.098+0000 7fc11dc7c640 1 -- start start 2023-10-16T10:02:24.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.099+0000 7fc11dc7c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc118160ad0 0x7fc1180ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:24.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.099+0000 7fc11dc7c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1180ce350 0x7fc118160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:24.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.099+0000 7fc11dc7c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc1180ae260 0x7fc1180ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:24.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.099+0000 7fc11dc7c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1180ce800 con 0x7fc1180ae260 2023-10-16T10:02:24.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.099+0000 7fc11dc7c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1180cdec0 con 0x7fc118160ad0 2023-10-16T10:02:24.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.099+0000 7fc11dc7c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1180acdd0 con 0x7fc1180ce350 2023-10-16T10:02:24.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.100+0000 7fc117fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc1180ae260 0x7fc1180ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:24.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.100+0000 7fc117fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc1180ae260 0x7fc1180ccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44852/0 (socket says 172.21.15.84:44852) 2023-10-16T10:02:24.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.100+0000 7fc117fff640 1 -- 172.21.15.84:0/2626368428 learned_addr learned my addr 172.21.15.84:0/2626368428 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:24.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.100+0000 7fc116ffd640 1 --2- 172.21.15.84:0/2626368428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1180ce350 0x7fc118160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:24.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.100+0000 7fc1177fe640 1 --2- 172.21.15.84:0/2626368428 >> v2:172.21.15.157:3300/0 conn(0x7fc118160ad0 0x7fc1180ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:24.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.100+0000 7fc117fff640 1 -- 172.21.15.84:0/2626368428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1180ce350 msgr2=0x7fc118160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:24.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.100+0000 7fc117fff640 1 --2- 172.21.15.84:0/2626368428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1180ce350 0x7fc118160eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.100+0000 7fc117fff640 1 -- 172.21.15.84:0/2626368428 >> v2:172.21.15.157:3300/0 conn(0x7fc118160ad0 msgr2=0x7fc1180ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:24.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.100+0000 7fc117fff640 1 --2- 172.21.15.84:0/2626368428 >> v2:172.21.15.157:3300/0 conn(0x7fc118160ad0 0x7fc1180ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.100+0000 7fc117fff640 1 -- 172.21.15.84:0/2626368428 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1181613f0 con 0x7fc1180ae260 2023-10-16T10:02:24.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.102+0000 7fc117fff640 1 --2- 172.21.15.84:0/2626368428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1180ae260 0x7fc1180ccd80 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc1180721f0 tx=0x7fc11002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4038f378350e1129 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:24.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.102+0000 7fc1167fc640 1 -- 172.21.15.84:0/2626368428 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11002ca70 con 0x7fc1180ae260 2023-10-16T10:02:24.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.102+0000 7fc1167fc640 1 -- 172.21.15.84:0/2626368428 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11002cbd0 con 0x7fc1180ae260 2023-10-16T10:02:24.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.102+0000 7fc1167fc640 1 -- 172.21.15.84:0/2626368428 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1100343f0 con 0x7fc1180ae260 2023-10-16T10:02:24.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.103+0000 7fc11dc7c640 1 -- 172.21.15.84:0/2626368428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1180ae260 msgr2=0x7fc1180ccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:24.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.103+0000 7fc11dc7c640 1 --2- 172.21.15.84:0/2626368428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1180ae260 0x7fc1180ccd80 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc1180721f0 tx=0x7fc11002bed0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.103+0000 7fc11dc7c640 1 -- 172.21.15.84:0/2626368428 shutdown_connections 2023-10-16T10:02:24.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.103+0000 7fc11dc7c640 1 --2- 172.21.15.84:0/2626368428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1180ae260 0x7fc1180ccd80 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.103+0000 7fc11dc7c640 1 --2- 172.21.15.84:0/2626368428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1180ce350 0x7fc118160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.103+0000 7fc11dc7c640 1 --2- 172.21.15.84:0/2626368428 >> v2:172.21.15.157:3300/0 conn(0x7fc118160ad0 0x7fc1180ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.103+0000 7fc11dc7c640 1 -- 172.21.15.84:0/2626368428 >> 172.21.15.84:0/2626368428 conn(0x7fc11806bc10 msgr2=0x7fc11806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:24.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.103+0000 7fc11dc7c640 1 -- 172.21.15.84:0/2626368428 shutdown_connections 2023-10-16T10:02:24.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.104+0000 7fc11dc7c640 1 -- 172.21.15.84:0/2626368428 wait complete. 2023-10-16T10:02:24.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.106+0000 7fc11dc7c640 1 Processor -- start 2023-10-16T10:02:24.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.106+0000 7fc11dc7c640 1 -- start start 2023-10-16T10:02:24.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.107+0000 7fc11dc7c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1180ae260 0x7fc1180b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:24.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.107+0000 7fc11dc7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1180ce350 0x7fc1180c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:24.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.107+0000 7fc11dc7c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc118160ad0 0x7fc1181585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:24.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.107+0000 7fc11dc7c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1180b46f0 con 0x7fc1180ae260 2023-10-16T10:02:24.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.107+0000 7fc11dc7c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1180b3ae0 con 0x7fc1180ce350 2023-10-16T10:02:24.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.107+0000 7fc11dc7c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1180b3dc0 con 0x7fc118160ad0 2023-10-16T10:02:24.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.107+0000 7fc1177fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1180ae260 0x7fc1180b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:24.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.107+0000 7fc1177fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1180ae260 0x7fc1180b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44864/0 (socket says 172.21.15.84:44864) 2023-10-16T10:02:24.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.107+0000 7fc1177fe640 1 -- 172.21.15.84:0/3668269290 learned_addr learned my addr 172.21.15.84:0/3668269290 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:24.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.108+0000 7fc116ffd640 1 --2- 172.21.15.84:0/3668269290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1180ce350 0x7fc1180c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:24.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.108+0000 7fc117fff640 1 --2- 172.21.15.84:0/3668269290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc118160ad0 0x7fc1181585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:24.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.108+0000 7fc1177fe640 1 -- 172.21.15.84:0/3668269290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc118160ad0 msgr2=0x7fc1181585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:24.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.108+0000 7fc1177fe640 1 --2- 172.21.15.84:0/3668269290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc118160ad0 0x7fc1181585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.108+0000 7fc1177fe640 1 -- 172.21.15.84:0/3668269290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1180ce350 msgr2=0x7fc1180c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:24.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.108+0000 7fc1177fe640 1 --2- 172.21.15.84:0/3668269290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1180ce350 0x7fc1180c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.108+0000 7fc1177fe640 1 -- 172.21.15.84:0/3668269290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc110009b30 con 0x7fc1180ae260 2023-10-16T10:02:24.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.108+0000 7fc117fff640 1 --2- 172.21.15.84:0/3668269290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc118160ad0 0x7fc1181585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:24.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.108+0000 7fc1177fe640 1 --2- 172.21.15.84:0/3668269290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1180ae260 0x7fc1180b34b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc10800d800 tx=0x7fc10800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:24.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.108+0000 7fc0fffff640 1 -- 172.21.15.84:0/3668269290 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10800b6c0 con 0x7fc1180ae260 2023-10-16T10:02:24.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.109+0000 7fc0fffff640 1 -- 172.21.15.84:0/3668269290 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10800b820 con 0x7fc1180ae260 2023-10-16T10:02:24.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.109+0000 7fc0fffff640 1 -- 172.21.15.84:0/3668269290 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10800bbb0 con 0x7fc1180ae260 2023-10-16T10:02:24.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.109+0000 7fc11dc7c640 1 -- 172.21.15.84:0/3668269290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1180b40a0 con 0x7fc1180ae260 2023-10-16T10:02:24.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.109+0000 7fc11dc7c640 1 -- 172.21.15.84:0/3668269290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1180c7c20 con 0x7fc1180ae260 2023-10-16T10:02:24.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.110+0000 7fc11dc7c640 1 -- 172.21.15.84:0/3668269290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11804e160 con 0x7fc1180ae260 2023-10-16T10:02:24.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.111+0000 7fc0fffff640 1 -- 172.21.15.84:0/3668269290 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fc10800bd10 con 0x7fc1180ae260 2023-10-16T10:02:24.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.112+0000 7fc0fffff640 1 --2- 172.21.15.84:0/3668269290 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fc10406b010 0x7fc10406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:24.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.112+0000 7fc0fffff640 1 -- 172.21.15.84:0/3668269290 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc108089d80 con 0x7fc1180ae260 2023-10-16T10:02:24.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.112+0000 7fc116ffd640 1 -- 172.21.15.84:0/3668269290 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fc10406b010 msgr2=0x7fc10406d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/585186562 2023-10-16T10:02:24.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.112+0000 7fc116ffd640 1 --2- 172.21.15.84:0/3668269290 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fc10406b010 0x7fc10406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:02:24.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.117+0000 7fc0fffff640 1 -- 172.21.15.84:0/3668269290 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc108054ca0 con 0x7fc1180ae260 2023-10-16T10:02:24.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.309+0000 7fc11dc7c640 1 -- 172.21.15.84:0/3668269290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fc1180cdb70 con 0x7fc1180ae260 2023-10-16T10:02:24.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.313+0000 7fc116ffd640 1 -- 172.21.15.84:0/3668269290 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fc10406b010 msgr2=0x7fc10406d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/585186562 2023-10-16T10:02:24.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.313+0000 7fc116ffd640 1 --2- 172.21.15.84:0/3668269290 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fc10406b010 0x7fc10406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:02:24.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.315+0000 7fc0fffff640 1 -- 172.21.15.84:0/3668269290 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v17)=0 v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc1080582c0 con 0x7fc1180ae260 2023-10-16T10:02:24.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.317+0000 7fc11dc7c640 1 -- 172.21.15.84:0/3668269290 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fc10406b010 msgr2=0x7fc10406d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:02:24.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.317+0000 7fc11dc7c640 1 --2- 172.21.15.84:0/3668269290 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fc10406b010 0x7fc10406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.317+0000 7fc11dc7c640 1 -- 172.21.15.84:0/3668269290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1180ae260 msgr2=0x7fc1180b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:24.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.317+0000 7fc11dc7c640 1 --2- 172.21.15.84:0/3668269290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1180ae260 0x7fc1180b34b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc10800d800 tx=0x7fc10800dcc0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.317+0000 7fc11dc7c640 1 -- 172.21.15.84:0/3668269290 shutdown_connections 2023-10-16T10:02:24.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.317+0000 7fc11dc7c640 1 --2- 172.21.15.84:0/3668269290 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fc10406b010 0x7fc10406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.317+0000 7fc11dc7c640 1 --2- 172.21.15.84:0/3668269290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc118160ad0 0x7fc1181585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.317+0000 7fc11dc7c640 1 --2- 172.21.15.84:0/3668269290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1180ce350 0x7fc1180c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.317+0000 7fc11dc7c640 1 --2- 172.21.15.84:0/3668269290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1180ae260 0x7fc1180b34b0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.317+0000 7fc11dc7c640 1 -- 172.21.15.84:0/3668269290 >> 172.21.15.84:0/3668269290 conn(0x7fc11806bc10 msgr2=0x7fc1180b01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:24.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.317+0000 7fc11dc7c640 1 -- 172.21.15.84:0/3668269290 shutdown_connections 2023-10-16T10:02:24.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.318+0000 7fc11dc7c640 1 -- 172.21.15.84:0/3668269290 wait complete. 2023-10-16T10:02:24.337 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T10:02:24.390 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:24.389+0000 7f9b24c4c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:02:24.422 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:24.421+0000 7eff6b213040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:02:24.447 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:24.445+0000 7f9b24c4c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:02:24.481 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:24.479+0000 7eff6b213040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:02:24.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.543+0000 7f85c1e7c640 1 Processor -- start 2023-10-16T10:02:24.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.543+0000 7f85c1e7c640 1 -- start start 2023-10-16T10:02:24.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.545+0000 7f85c1e7c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85bc15cd70 0x7f85bc0d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:24.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.545+0000 7f85c1e7c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f85bc0cded0 0x7f85bc0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:24.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.545+0000 7f85c1e7c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85bc15d150 con 0x7f85bc0b8480 2023-10-16T10:02:24.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.545+0000 7f85c1e7c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85bc0d1180 con 0x7f85bc0cded0 2023-10-16T10:02:24.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.545+0000 7f85c1e7c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0d1460 con 0x7f85bc15cd70 2023-10-16T10:02:24.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.545+0000 7f85bbfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f85bc0b8480 0x7f85bc0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34284/0 (socket says 172.21.15.84:34284) 2023-10-16T10:02:24.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.545+0000 7f85bbfff640 1 -- 172.21.15.84:0/2458057663 learned_addr learned my addr 172.21.15.84:0/2458057663 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:24.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.545+0000 7f85c0e7a640 1 --2- 172.21.15.84:0/2458057663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85bc15cd70 0x7f85bc0d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:24.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.545+0000 7f85c167b640 1 --2- 172.21.15.84:0/2458057663 >> v2:172.21.15.157:3300/0 conn(0x7f85bc0cded0 0x7f85bc0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:24.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.546+0000 7f85bb7fe640 1 -- 172.21.15.84:0/2458057663 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3840979522 0 0) 0x7f85bc15d150 con 0x7f85bc0b8480 2023-10-16T10:02:24.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.546+0000 7f85bb7fe640 1 -- 172.21.15.84:0/2458057663 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85ac003520 con 0x7f85bc0b8480 2023-10-16T10:02:24.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.546+0000 7f85bb7fe640 1 -- 172.21.15.84:0/2458057663 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 443398762 0 0) 0x7f85ac003520 con 0x7f85bc0b8480 2023-10-16T10:02:24.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.546+0000 7f85bb7fe640 1 -- 172.21.15.84:0/2458057663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85bc15cd70 msgr2=0x7f85bc0d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:24.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.546+0000 7f85bb7fe640 1 --2- 172.21.15.84:0/2458057663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85bc15cd70 0x7f85bc0d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.546+0000 7f85bb7fe640 1 -- 172.21.15.84:0/2458057663 >> v2:172.21.15.157:3300/0 conn(0x7f85bc0cded0 msgr2=0x7f85bc0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:24.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.546+0000 7f85bb7fe640 1 --2- 172.21.15.84:0/2458057663 >> v2:172.21.15.157:3300/0 conn(0x7f85bc0cded0 0x7f85bc0b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.546+0000 7f85bb7fe640 1 -- 172.21.15.84:0/2458057663 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc0d1740 con 0x7f85bc0b8480 2023-10-16T10:02:24.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.546+0000 7f85bb7fe640 1 -- 172.21.15.84:0/2458057663 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f85a8001670 con 0x7f85bc0b8480 2023-10-16T10:02:24.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.546+0000 7f85c0e7a640 1 --2- 172.21.15.84:0/2458057663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85bc15cd70 0x7f85bc0d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:02:24.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.546+0000 7f85bb7fe640 1 -- 172.21.15.84:0/2458057663 >> v1:172.21.15.84:6789/0 conn(0x7f85bc0b8480 legacy=0x7f85bc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:24.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.546+0000 7f85bb7fe640 1 --2- 172.21.15.84:0/2458057663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ac003970 0x7f85ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:24.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.546+0000 7f85bb7fe640 1 --2- 172.21.15.84:0/2458057663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85ac0047c0 0x7f85ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:24.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.546+0000 7f85bb7fe640 1 --2- 172.21.15.84:0/2458057663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85bc15cd70 0x7f85ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:24.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.546+0000 7f85bb7fe640 1 -- 172.21.15.84:0/2458057663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0d1180 con 0x7f85ac0047c0 2023-10-16T10:02:24.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.546+0000 7f85bb7fe640 1 -- 172.21.15.84:0/2458057663 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0d1460 con 0x7f85ac003970 2023-10-16T10:02:24.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.546+0000 7f85bb7fe640 1 -- 172.21.15.84:0/2458057663 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac00b980 con 0x7f85bc15cd70 2023-10-16T10:02:24.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.546+0000 7f85bbfff640 1 --2- 172.21.15.84:0/2458057663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85ac0047c0 0x7f85ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:24.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.547+0000 7f85bbfff640 1 -- 172.21.15.84:0/2458057663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85bc15cd70 msgr2=0x7f85ac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:24.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.547+0000 7f85c167b640 1 --2- 172.21.15.84:0/2458057663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85bc15cd70 0x7f85ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:24.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.547+0000 7f85c0e7a640 1 --2- 172.21.15.84:0/2458057663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ac003970 0x7f85ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:24.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.547+0000 7f85bbfff640 1 --2- 172.21.15.84:0/2458057663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85bc15cd70 0x7f85ac005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.547+0000 7f85bbfff640 1 -- 172.21.15.84:0/2458057663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ac003970 msgr2=0x7f85ac003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:24.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.547+0000 7f85bbfff640 1 --2- 172.21.15.84:0/2458057663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ac003970 0x7f85ac003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.547+0000 7f85bbfff640 1 -- 172.21.15.84:0/2458057663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b0009400 con 0x7f85ac0047c0 2023-10-16T10:02:24.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.548+0000 7f85bbfff640 1 --2- 172.21.15.84:0/2458057663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85ac0047c0 0x7f85ac004c40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f85a800c240 tx=0x7f85ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0d8335860df4b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:24.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.548+0000 7f85bb7fe640 1 -- 172.21.15.84:0/2458057663 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a802cb00 con 0x7f85ac0047c0 2023-10-16T10:02:24.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.548+0000 7f85bb7fe640 1 -- 172.21.15.84:0/2458057663 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a802cc60 con 0x7f85ac0047c0 2023-10-16T10:02:24.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.548+0000 7f85bb7fe640 1 -- 172.21.15.84:0/2458057663 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8005af0 con 0x7f85ac0047c0 2023-10-16T10:02:24.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.549+0000 7f85c1e7c640 1 -- 172.21.15.84:0/2458057663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85ac0047c0 msgr2=0x7f85ac004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:24.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.549+0000 7f85c1e7c640 1 --2- 172.21.15.84:0/2458057663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85ac0047c0 0x7f85ac004c40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f85a800c240 tx=0x7f85ac0057f0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.549+0000 7f85c167b640 1 -- 172.21.15.84:0/2458057663 reap_dead start 2023-10-16T10:02:24.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.549+0000 7f85c1e7c640 1 -- 172.21.15.84:0/2458057663 shutdown_connections 2023-10-16T10:02:24.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.549+0000 7f85c1e7c640 1 -- 172.21.15.84:0/2458057663 >> 172.21.15.84:0/2458057663 conn(0x7f85bc0cf570 msgr2=0x7f85bc0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:24.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.549+0000 7f85c1e7c640 1 -- 172.21.15.84:0/2458057663 shutdown_connections 2023-10-16T10:02:24.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.549+0000 7f85c1e7c640 1 -- 172.21.15.84:0/2458057663 wait complete. 2023-10-16T10:02:24.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.551+0000 7f85c1e7c640 1 Processor -- start 2023-10-16T10:02:24.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.551+0000 7f85c1e7c640 1 -- start start 2023-10-16T10:02:24.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.552+0000 7f85c1e7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ac0047c0 0x7f85bc0a9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:24.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.552+0000 7f85c1e7c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85bc15eee0 0x7f85bc0b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:24.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.552+0000 7f85c1e7c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85bc0b2790 0x7f85bc0b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:24.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.552+0000 7f85c1e7c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0b3150 con 0x7f85bc0b2790 2023-10-16T10:02:24.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.552+0000 7f85c1e7c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0aa750 con 0x7f85ac0047c0 2023-10-16T10:02:24.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.552+0000 7f85c1e7c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0aaa30 con 0x7f85bc15eee0 2023-10-16T10:02:24.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.552+0000 7f85c167b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85bc0b2790 0x7f85bc0b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:24.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.552+0000 7f85c167b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85bc0b2790 0x7f85bc0b2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44878/0 (socket says 172.21.15.84:44878) 2023-10-16T10:02:24.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.552+0000 7f85c167b640 1 -- 172.21.15.84:0/1866232407 learned_addr learned my addr 172.21.15.84:0/1866232407 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:24.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.553+0000 7f85c0e7a640 1 --2- 172.21.15.84:0/1866232407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ac0047c0 0x7f85bc0a9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:24.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.553+0000 7f85bbfff640 1 --2- 172.21.15.84:0/1866232407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85bc15eee0 0x7f85bc0b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:24.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.553+0000 7f85c167b640 1 -- 172.21.15.84:0/1866232407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85bc15eee0 msgr2=0x7f85bc0b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:24.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.553+0000 7f85c167b640 1 --2- 172.21.15.84:0/1866232407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85bc15eee0 0x7f85bc0b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.553+0000 7f85c167b640 1 -- 172.21.15.84:0/1866232407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ac0047c0 msgr2=0x7f85bc0a9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:24.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.553+0000 7f85c167b640 1 --2- 172.21.15.84:0/1866232407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ac0047c0 0x7f85bc0a9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.553+0000 7f85c167b640 1 -- 172.21.15.84:0/1866232407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc0aad10 con 0x7f85bc0b2790 2023-10-16T10:02:24.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.553+0000 7f85c167b640 1 --2- 172.21.15.84:0/1866232407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85bc0b2790 0x7f85bc0b2c10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0b56a0 tx=0x7f85b400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:24.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.553+0000 7f85b97fa640 1 -- 172.21.15.84:0/1866232407 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4003990 con 0x7f85bc0b2790 2023-10-16T10:02:24.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.553+0000 7f85b97fa640 1 -- 172.21.15.84:0/1866232407 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b4011040 con 0x7f85bc0b2790 2023-10-16T10:02:24.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.553+0000 7f85c1e7c640 1 -- 172.21.15.84:0/1866232407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85bc0aaff0 con 0x7f85bc0b2790 2023-10-16T10:02:24.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.553+0000 7f85b97fa640 1 -- 172.21.15.84:0/1866232407 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4003d50 con 0x7f85bc0b2790 2023-10-16T10:02:24.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.554+0000 7f85c1e7c640 1 -- 172.21.15.84:0/1866232407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85bc0ab2d0 con 0x7f85bc0b2790 2023-10-16T10:02:24.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.555+0000 7f85c1e7c640 1 -- 172.21.15.84:0/1866232407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85bc1f20d0 con 0x7f85bc0b2790 2023-10-16T10:02:24.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.555+0000 7f85b97fa640 1 -- 172.21.15.84:0/1866232407 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f85b4003eb0 con 0x7f85bc0b2790 2023-10-16T10:02:24.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.556+0000 7f85b97fa640 1 --2- 172.21.15.84:0/1866232407 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f85a006b010 0x7f85a006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:24.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.556+0000 7f85b97fa640 1 -- 172.21.15.84:0/1866232407 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85b4089de0 con 0x7f85bc0b2790 2023-10-16T10:02:24.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.556+0000 7f85c0e7a640 1 -- 172.21.15.84:0/1866232407 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f85a006b010 msgr2=0x7f85a006d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/585186562 2023-10-16T10:02:24.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.557+0000 7f85c0e7a640 1 --2- 172.21.15.84:0/1866232407 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f85a006b010 0x7f85a006d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:02:24.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.561+0000 7f85b97fa640 1 -- 172.21.15.84:0/1866232407 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f85bc1f20d0 con 0x7f85bc0b2790 2023-10-16T10:02:24.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.753+0000 7f85c1e7c640 1 -- 172.21.15.84:0/1866232407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f85bc1f20d0 con 0x7f85bc0b2790 2023-10-16T10:02:24.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.758+0000 7f85c0e7a640 1 -- 172.21.15.84:0/1866232407 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f85a006b010 msgr2=0x7f85a006d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/585186562 2023-10-16T10:02:24.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.758+0000 7f85c0e7a640 1 --2- 172.21.15.84:0/1866232407 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f85a006b010 0x7f85a006d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:02:24.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.759+0000 7f85b97fa640 1 -- 172.21.15.84:0/1866232407 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v18)=0 v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7f85bc1f20d0 con 0x7f85bc0b2790 2023-10-16T10:02:24.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.761+0000 7f85c1e7c640 1 -- 172.21.15.84:0/1866232407 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f85a006b010 msgr2=0x7f85a006d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:02:24.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.761+0000 7f85c1e7c640 1 --2- 172.21.15.84:0/1866232407 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f85a006b010 0x7f85a006d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.761+0000 7f85c1e7c640 1 -- 172.21.15.84:0/1866232407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85bc0b2790 msgr2=0x7f85bc0b2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:24.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.761+0000 7f85c1e7c640 1 --2- 172.21.15.84:0/1866232407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85bc0b2790 0x7f85bc0b2c10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0b56a0 tx=0x7f85b400f4a0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.762+0000 7f85c1e7c640 1 -- 172.21.15.84:0/1866232407 shutdown_connections 2023-10-16T10:02:24.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.762+0000 7f85c1e7c640 1 --2- 172.21.15.84:0/1866232407 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7f85a006b010 0x7f85a006d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.762+0000 7f85c1e7c640 1 --2- 172.21.15.84:0/1866232407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85bc0b2790 0x7f85bc0b2c10 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.762+0000 7f85c1e7c640 1 --2- 172.21.15.84:0/1866232407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85bc15eee0 0x7f85bc0b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.762+0000 7f85c1e7c640 1 --2- 172.21.15.84:0/1866232407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ac0047c0 0x7f85bc0a9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.762+0000 7f85c1e7c640 1 -- 172.21.15.84:0/1866232407 >> 172.21.15.84:0/1866232407 conn(0x7f85bc0cf570 msgr2=0x7f85bc0b5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:24.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.762+0000 7f85c1e7c640 1 -- 172.21.15.84:0/1866232407 shutdown_connections 2023-10-16T10:02:24.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:24.762+0000 7f85c1e7c640 1 -- 172.21.15.84:0/1866232407 wait complete. 2023-10-16T10:02:24.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:24.782 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-16T10:02:24.839 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-16T10:02:24.909 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:24.907+0000 7f9b24c4c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:02:24.932 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:24.930+0000 7eff6b213040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:02:24.963 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:24.961+0000 7f9b24c4c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:02:24.984 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:24.982+0000 7eff6b213040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:02:25.015 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:25.014+0000 7f9b24c4c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:02:25.034 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:25.033+0000 7eff6b213040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:02:25.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.039+0000 7fd5e1d2c640 1 Processor -- start 2023-10-16T10:02:25.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.040+0000 7fd5e1d2c640 1 -- start start 2023-10-16T10:02:25.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.041+0000 7fd5e1d2c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd5dc0caf80 0x7fd5dc0ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:25.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.041+0000 7fd5e1d2c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd5dc0b4e50 0x7fd5dc069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:25.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.041+0000 7fd5e1d2c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5dc0c9930 con 0x7fd5dc065ab0 2023-10-16T10:02:25.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.041+0000 7fd5e1d2c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5dc15b1e0 con 0x7fd5dc0b4e50 2023-10-16T10:02:25.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.041+0000 7fd5e1d2c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5dc0c9c10 con 0x7fd5dc0caf80 2023-10-16T10:02:25.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.041+0000 7fd5dbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd5dc0b4e50 0x7fd5dc069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:25.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.042+0000 7fd5dbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd5dc0b4e50 0x7fd5dc069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44894/0 (socket says 172.21.15.84:44894) 2023-10-16T10:02:25.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.042+0000 7fd5d3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd5dc065ab0 0x7fd5dc0cb360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34300/0 (socket says 172.21.15.84:34300) 2023-10-16T10:02:25.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.042+0000 7fd5dbfff640 1 -- 172.21.15.84:0/1953734162 learned_addr learned my addr 172.21.15.84:0/1953734162 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:25.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.042+0000 7fd5db7fe640 1 --2- 172.21.15.84:0/1953734162 >> v2:172.21.15.157:3300/0 conn(0x7fd5dc0caf80 0x7fd5dc0ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:25.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.042+0000 7fd5dbfff640 1 -- 172.21.15.84:0/1953734162 >> v2:172.21.15.157:3300/0 conn(0x7fd5dc0caf80 msgr2=0x7fd5dc0ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:25.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.042+0000 7fd5dbfff640 1 --2- 172.21.15.84:0/1953734162 >> v2:172.21.15.157:3300/0 conn(0x7fd5dc0caf80 0x7fd5dc0ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:25.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.042+0000 7fd5dbfff640 1 -- 172.21.15.84:0/1953734162 >> v1:172.21.15.84:6789/0 conn(0x7fd5dc065ab0 legacy=0x7fd5dc0cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:25.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.042+0000 7fd5dbfff640 1 -- 172.21.15.84:0/1953734162 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5dc0c9ef0 con 0x7fd5dc0b4e50 2023-10-16T10:02:25.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.044+0000 7fd5dbfff640 1 --2- 172.21.15.84:0/1953734162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5dc0b4e50 0x7fd5dc069da0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc06e5e0 tx=0x7fd5d402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3f2967f07386ec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:25.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.044+0000 7fd5daffd640 1 -- 172.21.15.84:0/1953734162 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d402ca80 con 0x7fd5dc0b4e50 2023-10-16T10:02:25.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.044+0000 7fd5daffd640 1 -- 172.21.15.84:0/1953734162 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d402cbe0 con 0x7fd5dc0b4e50 2023-10-16T10:02:25.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.044+0000 7fd5daffd640 1 -- 172.21.15.84:0/1953734162 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40343f0 con 0x7fd5dc0b4e50 2023-10-16T10:02:25.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.045+0000 7fd5e1d2c640 1 -- 172.21.15.84:0/1953734162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5dc0b4e50 msgr2=0x7fd5dc069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:25.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.045+0000 7fd5e1d2c640 1 --2- 172.21.15.84:0/1953734162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5dc0b4e50 0x7fd5dc069da0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc06e5e0 tx=0x7fd5d402bee0 comp rx=0 tx=0).stop 2023-10-16T10:02:25.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.045+0000 7fd5e1d2c640 1 -- 172.21.15.84:0/1953734162 shutdown_connections 2023-10-16T10:02:25.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.045+0000 7fd5e1d2c640 1 --2- 172.21.15.84:0/1953734162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5dc0b4e50 0x7fd5dc069da0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:25.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.045+0000 7fd5e1d2c640 1 --2- 172.21.15.84:0/1953734162 >> v2:172.21.15.157:3300/0 conn(0x7fd5dc0caf80 0x7fd5dc0ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:25.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.045+0000 7fd5e1d2c640 1 -- 172.21.15.84:0/1953734162 >> 172.21.15.84:0/1953734162 conn(0x7fd5dc06bc10 msgr2=0x7fd5dc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:25.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.045+0000 7fd5e1d2c640 1 -- 172.21.15.84:0/1953734162 shutdown_connections 2023-10-16T10:02:25.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.045+0000 7fd5e1d2c640 1 -- 172.21.15.84:0/1953734162 wait complete. 2023-10-16T10:02:25.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.047+0000 7fd5e1d2c640 1 Processor -- start 2023-10-16T10:02:25.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.048+0000 7fd5e1d2c640 1 -- start start 2023-10-16T10:02:25.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.049+0000 7fd5e1d2c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5dc065ab0 0x7fd5dc14ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:25.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.049+0000 7fd5e1d2c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5dc0b4e50 0x7fd5dc0cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:25.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.049+0000 7fd5e1d2c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5dc0caf80 0x7fd5dc1f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:25.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.049+0000 7fd5e1d2c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc0c9500 con 0x7fd5dc065ab0 2023-10-16T10:02:25.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.049+0000 7fd5e1d2c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc0ad010 con 0x7fd5dc0b4e50 2023-10-16T10:02:25.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.049+0000 7fd5e1d2c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc14f7b0 con 0x7fd5dc0caf80 2023-10-16T10:02:25.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.049+0000 7fd5db7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5dc065ab0 0x7fd5dc14ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:25.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.049+0000 7fd5db7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5dc065ab0 0x7fd5dc14ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44904/0 (socket says 172.21.15.84:44904) 2023-10-16T10:02:25.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.049+0000 7fd5db7fe640 1 -- 172.21.15.84:0/1347274145 learned_addr learned my addr 172.21.15.84:0/1347274145 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:25.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.050+0000 7fd5d3fff640 1 --2- 172.21.15.84:0/1347274145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5dc0b4e50 0x7fd5dc0cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:25.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.050+0000 7fd5dbfff640 1 --2- 172.21.15.84:0/1347274145 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5dc0caf80 0x7fd5dc1f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:25.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.050+0000 7fd5db7fe640 1 -- 172.21.15.84:0/1347274145 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5dc0caf80 msgr2=0x7fd5dc1f1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:25.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.050+0000 7fd5db7fe640 1 --2- 172.21.15.84:0/1347274145 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5dc0caf80 0x7fd5dc1f1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:25.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.050+0000 7fd5db7fe640 1 -- 172.21.15.84:0/1347274145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5dc0b4e50 msgr2=0x7fd5dc0cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:25.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.050+0000 7fd5db7fe640 1 --2- 172.21.15.84:0/1347274145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5dc0b4e50 0x7fd5dc0cd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:25.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.050+0000 7fd5db7fe640 1 -- 172.21.15.84:0/1347274145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d4009b30 con 0x7fd5dc065ab0 2023-10-16T10:02:25.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.050+0000 7fd5d3fff640 1 --2- 172.21.15.84:0/1347274145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5dc0b4e50 0x7fd5dc0cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:25.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.050+0000 7fd5db7fe640 1 --2- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5dc065ab0 0x7fd5dc14ee40 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00d830 tx=0x7fd5cc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:25.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.050+0000 7fd5d8ff9640 1 -- 172.21.15.84:0/1347274145 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc00b6c0 con 0x7fd5dc065ab0 2023-10-16T10:02:25.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.050+0000 7fd5d8ff9640 1 -- 172.21.15.84:0/1347274145 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5cc00b820 con 0x7fd5dc065ab0 2023-10-16T10:02:25.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.050+0000 7fd5d8ff9640 1 -- 172.21.15.84:0/1347274145 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc00bbb0 con 0x7fd5dc065ab0 2023-10-16T10:02:25.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.050+0000 7fd5e1d2c640 1 -- 172.21.15.84:0/1347274145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5dc14f4d0 con 0x7fd5dc065ab0 2023-10-16T10:02:25.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.051+0000 7fd5e1d2c640 1 -- 172.21.15.84:0/1347274145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5dc0acca0 con 0x7fd5dc065ab0 2023-10-16T10:02:25.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.052+0000 7fd5e1d2c640 1 -- 172.21.15.84:0/1347274145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5dc0ace30 con 0x7fd5dc065ab0 2023-10-16T10:02:25.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.053+0000 7fd5d8ff9640 1 -- 172.21.15.84:0/1347274145 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fd5cc01e460 con 0x7fd5dc065ab0 2023-10-16T10:02:25.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.054+0000 7fd5d8ff9640 1 --2- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd5bc06afe0 0x7fd5bc06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:25.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.054+0000 7fd5d3fff640 1 -- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd5bc06afe0 msgr2=0x7fd5bc06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/585186562 2023-10-16T10:02:25.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.054+0000 7fd5d8ff9640 1 -- 172.21.15.84:0/1347274145 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5cc089f80 con 0x7fd5dc065ab0 2023-10-16T10:02:25.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.054+0000 7fd5d3fff640 1 --2- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd5bc06afe0 0x7fd5bc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:02:25.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.058+0000 7fd5d8ff9640 1 -- 172.21.15.84:0/1347274145 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd5cc054f10 con 0x7fd5dc065ab0 2023-10-16T10:02:25.084 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:25.082+0000 7f9b24c4c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:02:25.096 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:25.095+0000 7eff6b213040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:02:25.135 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:25.133+0000 7f9b24c4c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:02:25.145 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:25.144+0000 7eff6b213040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:02:25.250 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:25.249+0000 7f9b24c4c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:02:25.256 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:25.255+0000 7eff6b213040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:02:25.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.255+0000 7fd5d3fff640 1 -- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd5bc06afe0 msgr2=0x7fd5bc06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/585186562 2023-10-16T10:02:25.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.255+0000 7fd5d3fff640 1 --2- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd5bc06afe0 0x7fd5bc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:02:25.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.274+0000 7fd5e1d2c640 1 -- 172.21.15.84:0/1347274145 --> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5dc0ca4b0 con 0x7fd5bc06afe0 2023-10-16T10:02:25.313 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:25.312+0000 7f9b24c4c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:02:25.318 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:25.316+0000 7eff6b213040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:02:25.363 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:25.361+0000 7f9b24c4c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:02:25.368 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:25.367+0000 7eff6b213040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:02:25.419 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:25.418+0000 7f9b24c4c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:02:25.424 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:25.423+0000 7eff6b213040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:02:25.469 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:25.467+0000 7f9b24c4c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:02:25.474 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:25.473+0000 7eff6b213040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:02:25.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.656+0000 7fd5d3fff640 1 -- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd5bc06afe0 msgr2=0x7fd5bc06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/585186562 2023-10-16T10:02:25.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:25.656+0000 7fd5d3fff640 1 --2- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd5bc06afe0 0x7fd5bc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T10:02:25.845 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:25.844+0000 7f9b24c4c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:02:25.853 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:25.852+0000 7eff6b213040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:02:26.003 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:26.002+0000 7f9b24c4c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:02:26.012 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:26.011+0000 7eff6b213040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:02:26.111 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:26.110+0000 7f9b24c4c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:02:26.121 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:26.118+0000 7eff6b213040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:02:26.161 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:26.160+0000 7f9b24c4c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:02:26.169 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:26.168+0000 7eff6b213040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:02:26.268 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:26.267+0000 7f9b24c4c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:02:26.276 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:26.274+0000 7eff6b213040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:02:26.373 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:26.372+0000 7f9b24c4c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:02:26.382 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:26.381+0000 7eff6b213040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:02:26.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:26.457+0000 7fd5d3fff640 1 -- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd5bc06afe0 msgr2=0x7fd5bc06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/585186562 2023-10-16T10:02:26.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:26.457+0000 7fd5d3fff640 1 --2- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd5bc06afe0 0x7fd5bc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T10:02:26.529 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:26.528+0000 7f9b24c4c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:02:26.539 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:26.538+0000 7eff6b213040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:02:26.578 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:26.576+0000 7f9b24c4c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:02:26.588 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:26.587+0000 7eff6b213040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:02:26.696 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:26.695+0000 7f9b24c4c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:02:26.706 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:26.705+0000 7eff6b213040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:02:27.503 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:27.502+0000 7f9b24c4c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:02:27.525 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:27.524+0000 7eff6b213040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:02:27.797 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:27.795+0000 7f9b24c4c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:02:27.819 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:27.818+0000 7eff6b213040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:02:27.855 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:27.854+0000 7f9b24c4c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:02:27.878 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:27.877+0000 7eff6b213040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:02:27.967 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:27.965+0000 7f9b24c4c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:02:27.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:27.983+0000 7fd5d8ff9640 1 -- 172.21.15.84:0/1347274145 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 23) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fd5cc0552e0 con 0x7fd5dc065ab0 2023-10-16T10:02:27.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:27.983+0000 7fd5d8ff9640 1 -- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd5bc06afe0 msgr2=0x7fd5bc06d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:02:27.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:27.983+0000 7fd5d8ff9640 1 --2- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:6832/585186562,v1:172.21.15.84:6833/585186562] conn(0x7fd5bc06afe0 0x7fd5bc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:27.998 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:02:27.997+0000 7eff6b213040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:02:28.290 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:28.289+0000 7f9aa477f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:28.291 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:28.289+0000 7f9aa477f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:28.291 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:28.289+0000 7f9aa477f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:28.291 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:28.289+0000 7f9aa477f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:28.291 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:28.289+0000 7f9aa477f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:28.292 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:28.289+0000 7f9aa0737640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:28.292 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:28.289+0000 7f9aa0737640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:28.292 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:28.289+0000 7f9aa0737640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:28.293 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:28.289+0000 7f9aa0737640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:28.293 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:02:28.289+0000 7f9aa0737640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:28.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:28.995+0000 7fd5d8ff9640 1 -- 172.21.15.84:0/1347274145 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 24) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fd5cc0706f0 con 0x7fd5dc065ab0 2023-10-16T10:02:28.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:28.996+0000 7fd5d8ff9640 1 --2- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd5bc0358e0 0x7fd5bc074ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:28.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:28.996+0000 7fd5d8ff9640 1 -- 172.21.15.84:0/1347274145 --> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5bc035cc0 con 0x7fd5bc0358e0 2023-10-16T10:02:29.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.001+0000 7fd5d3fff640 1 --2- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd5bc0358e0 0x7fd5bc074ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:29.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.006+0000 7fd5d3fff640 1 --2- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd5bc0358e0 0x7fd5bc074ea0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0c68c0 tx=0x7fd5c4007d70 comp rx=0 tx=0).ready entity=mgr.6165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:29.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.189+0000 7fd5d8ff9640 1 -- 172.21.15.84:0/1347274145 <== mgr.6165 v2:172.21.15.84:6832/633652932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd5bc035cc0 con 0x7fd5bc0358e0 2023-10-16T10:02:29.191 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path snap_test_dir1 2023-10-16T10:02:29.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.192+0000 7fd5e1d2c640 1 -- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd5bc0358e0 msgr2=0x7fd5bc074ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:29.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.193+0000 7fd5e1d2c640 1 --2- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd5bc0358e0 0x7fd5bc074ea0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0c68c0 tx=0x7fd5c4007d70 comp rx=0 tx=0).stop 2023-10-16T10:02:29.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.193+0000 7fd5e1d2c640 1 -- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5dc065ab0 msgr2=0x7fd5dc14ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:29.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.193+0000 7fd5e1d2c640 1 --2- 172.21.15.84:0/1347274145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5dc065ab0 0x7fd5dc14ee40 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00d830 tx=0x7fd5cc00dcf0 comp rx=0 tx=0).stop 2023-10-16T10:02:29.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.193+0000 7fd5dbfff640 1 -- 172.21.15.84:0/1347274145 reap_dead start 2023-10-16T10:02:29.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.193+0000 7fd5e1d2c640 1 -- 172.21.15.84:0/1347274145 shutdown_connections 2023-10-16T10:02:29.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.193+0000 7fd5e1d2c640 1 -- 172.21.15.84:0/1347274145 >> 172.21.15.84:0/1347274145 conn(0x7fd5dc06bc10 msgr2=0x7fd5dc069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:29.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.193+0000 7fd5e1d2c640 1 -- 172.21.15.84:0/1347274145 shutdown_connections 2023-10-16T10:02:29.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.193+0000 7fd5e1d2c640 1 -- 172.21.15.84:0/1347274145 wait complete. 2023-10-16T10:02:29.215 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-16T10:02:29.216 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 2m 2023-10-16T10:02:29.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.423+0000 7f984f4e0640 1 Processor -- start 2023-10-16T10:02:29.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.423+0000 7f984f4e0640 1 -- start start 2023-10-16T10:02:29.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.425+0000 7f984f4e0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f984806a4b0 0x7f98480ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:29.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.425+0000 7f984f4e0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f98480c9ca0 0x7f98480ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:29.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.425+0000 7f984f4e0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98480cb030 con 0x7f984815ad70 2023-10-16T10:02:29.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.425+0000 7f984f4e0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98480cae30 con 0x7f98480c9ca0 2023-10-16T10:02:29.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.425+0000 7f984f4e0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98480c91c0 con 0x7f984806a4b0 2023-10-16T10:02:29.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.425+0000 7f984da7d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f98480c9ca0 0x7f98480ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:29.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.425+0000 7f984da7d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f98480c9ca0 0x7f98480ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55934/0 (socket says 172.21.15.84:55934) 2023-10-16T10:02:29.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.425+0000 7f984da7d640 1 -- 172.21.15.84:0/1133831713 learned_addr learned my addr 172.21.15.84:0/1133831713 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:29.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.425+0000 7f984ca7b640 1 --2- 172.21.15.84:0/1133831713 >> v2:172.21.15.157:3300/0 conn(0x7f984806a4b0 0x7f98480ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:29.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.426+0000 7f984da7d640 1 -- 172.21.15.84:0/1133831713 >> v2:172.21.15.157:3300/0 conn(0x7f984806a4b0 msgr2=0x7f98480ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:29.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.426+0000 7f984da7d640 1 --2- 172.21.15.84:0/1133831713 >> v2:172.21.15.157:3300/0 conn(0x7f984806a4b0 0x7f98480ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:29.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.426+0000 7f984da7d640 1 -- 172.21.15.84:0/1133831713 >> v1:172.21.15.157:6789/0 conn(0x7f984815ad70 legacy=0x7f9848069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:29.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.426+0000 7f984da7d640 1 -- 172.21.15.84:0/1133831713 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98480c94a0 con 0x7f98480c9ca0 2023-10-16T10:02:29.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.427+0000 7f984da7d640 1 --2- 172.21.15.84:0/1133831713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98480c9ca0 0x7f98480ca8f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f98480ce870 tx=0x7f984002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27bd086937f5e94 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:29.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.427+0000 7f983bfff640 1 -- 172.21.15.84:0/1133831713 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984002ca80 con 0x7f98480c9ca0 2023-10-16T10:02:29.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.427+0000 7f983bfff640 1 -- 172.21.15.84:0/1133831713 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984002cbe0 con 0x7f98480c9ca0 2023-10-16T10:02:29.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.427+0000 7f983bfff640 1 -- 172.21.15.84:0/1133831713 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98400343f0 con 0x7f98480c9ca0 2023-10-16T10:02:29.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.428+0000 7f984f4e0640 1 -- 172.21.15.84:0/1133831713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98480c9ca0 msgr2=0x7f98480ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:29.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.428+0000 7f984f4e0640 1 --2- 172.21.15.84:0/1133831713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98480c9ca0 0x7f98480ca8f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f98480ce870 tx=0x7f984002bee0 comp rx=0 tx=0).stop 2023-10-16T10:02:29.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.428+0000 7f984f4e0640 1 -- 172.21.15.84:0/1133831713 shutdown_connections 2023-10-16T10:02:29.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.428+0000 7f984f4e0640 1 --2- 172.21.15.84:0/1133831713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98480c9ca0 0x7f98480ca8f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:29.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.428+0000 7f984f4e0640 1 --2- 172.21.15.84:0/1133831713 >> v2:172.21.15.157:3300/0 conn(0x7f984806a4b0 0x7f98480ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:29.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.428+0000 7f984f4e0640 1 -- 172.21.15.84:0/1133831713 >> 172.21.15.84:0/1133831713 conn(0x7f98480ce9b0 msgr2=0x7f98480cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:29.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.428+0000 7f984f4e0640 1 -- 172.21.15.84:0/1133831713 shutdown_connections 2023-10-16T10:02:29.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.428+0000 7f984f4e0640 1 -- 172.21.15.84:0/1133831713 wait complete. 2023-10-16T10:02:29.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.430+0000 7f984f4e0640 1 Processor -- start 2023-10-16T10:02:29.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.430+0000 7f984f4e0640 1 -- start start 2023-10-16T10:02:29.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.431+0000 7f984f4e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f984806a4b0 0x7f9848150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:29.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.431+0000 7f984f4e0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98480c9ca0 0x7f98480c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:29.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.431+0000 7f984f4e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f984815ad70 0x7f98480acf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:29.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.431+0000 7f984f4e0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98480cd230 con 0x7f984815ad70 2023-10-16T10:02:29.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.431+0000 7f984f4e0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98481513b0 con 0x7f98480c9ca0 2023-10-16T10:02:29.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.431+0000 7f984f4e0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98480cc620 con 0x7f984806a4b0 2023-10-16T10:02:29.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.431+0000 7f984da7d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f984815ad70 0x7f98480acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:29.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.431+0000 7f984da7d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f984815ad70 0x7f98480acf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55950/0 (socket says 172.21.15.84:55950) 2023-10-16T10:02:29.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.431+0000 7f984da7d640 1 -- 172.21.15.84:0/2277849061 learned_addr learned my addr 172.21.15.84:0/2277849061 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:29.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.431+0000 7f984d27c640 1 --2- 172.21.15.84:0/2277849061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f984806a4b0 0x7f9848150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:29.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.431+0000 7f984ca7b640 1 --2- 172.21.15.84:0/2277849061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98480c9ca0 0x7f98480c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:29.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.431+0000 7f984da7d640 1 -- 172.21.15.84:0/2277849061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f984806a4b0 msgr2=0x7f9848150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:29.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.431+0000 7f984da7d640 1 --2- 172.21.15.84:0/2277849061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f984806a4b0 0x7f9848150e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:29.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.431+0000 7f984da7d640 1 -- 172.21.15.84:0/2277849061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98480c9ca0 msgr2=0x7f98480c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:29.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.431+0000 7f984da7d640 1 --2- 172.21.15.84:0/2277849061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98480c9ca0 0x7f98480c4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:29.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.431+0000 7f984da7d640 1 -- 172.21.15.84:0/2277849061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9840009b30 con 0x7f984815ad70 2023-10-16T10:02:29.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.432+0000 7f984da7d640 1 --2- 172.21.15.84:0/2277849061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f984815ad70 0x7f98480acf10 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9840036040 tx=0x7f9840004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:29.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.432+0000 7f9839ffb640 1 -- 172.21.15.84:0/2277849061 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840009510 con 0x7f984815ad70 2023-10-16T10:02:29.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.432+0000 7f9839ffb640 1 -- 172.21.15.84:0/2277849061 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984003e450 con 0x7f984815ad70 2023-10-16T10:02:29.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.432+0000 7f984f4e0640 1 -- 172.21.15.84:0/2277849061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98480cc900 con 0x7f984815ad70 2023-10-16T10:02:29.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.432+0000 7f9839ffb640 1 -- 172.21.15.84:0/2277849061 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984003e7e0 con 0x7f984815ad70 2023-10-16T10:02:29.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.432+0000 7f984f4e0640 1 -- 172.21.15.84:0/2277849061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98480ccec0 con 0x7f984815ad70 2023-10-16T10:02:29.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.433+0000 7f984f4e0640 1 -- 172.21.15.84:0/2277849061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98480cd3e0 con 0x7f984815ad70 2023-10-16T10:02:29.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.434+0000 7f9839ffb640 1 -- 172.21.15.84:0/2277849061 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f984002cd50 con 0x7f984815ad70 2023-10-16T10:02:29.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.435+0000 7f9839ffb640 1 --2- 172.21.15.84:0/2277849061 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f982c06ad40 0x7f982c06d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:29.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.435+0000 7f9839ffb640 1 -- 172.21.15.84:0/2277849061 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f98400abcf0 con 0x7f984815ad70 2023-10-16T10:02:29.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.436+0000 7f984d27c640 1 --2- 172.21.15.84:0/2277849061 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f982c06ad40 0x7f982c06d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:29.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.436+0000 7f984d27c640 1 --2- 172.21.15.84:0/2277849061 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f982c06ad40 0x7f982c06d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f984400a730 tx=0x7f9844008040 comp rx=0 tx=0).ready entity=mgr.6165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:29.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.440+0000 7f9839ffb640 1 -- 172.21.15.84:0/2277849061 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f98480cd3e0 con 0x7f984815ad70 2023-10-16T10:02:29.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.653+0000 7f984f4e0640 1 -- 172.21.15.84:0/2277849061 --> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "2m", "target": ["mon-mgr", ""]}) v1 -- 0x7f98480c9780 con 0x7f982c06ad40 2023-10-16T10:02:29.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.812+0000 7f9839ffb640 1 -- 172.21.15.84:0/2277849061 <== mgr.6165 v2:172.21.15.84:6832/633652932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f984400a380 con 0x7f982c06ad40 2023-10-16T10:02:29.814 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path snap_test_dir1 2023-10-16T10:02:29.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.815+0000 7f984f4e0640 1 -- 172.21.15.84:0/2277849061 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f982c06ad40 msgr2=0x7f982c06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:29.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.815+0000 7f984f4e0640 1 --2- 172.21.15.84:0/2277849061 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f982c06ad40 0x7f982c06d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f984400a730 tx=0x7f9844008040 comp rx=0 tx=0).stop 2023-10-16T10:02:29.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.815+0000 7f984f4e0640 1 -- 172.21.15.84:0/2277849061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f984815ad70 msgr2=0x7f98480acf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:29.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.815+0000 7f984f4e0640 1 --2- 172.21.15.84:0/2277849061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f984815ad70 0x7f98480acf10 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9840036040 tx=0x7f9840004a40 comp rx=0 tx=0).stop 2023-10-16T10:02:29.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.815+0000 7f984f4e0640 1 -- 172.21.15.84:0/2277849061 shutdown_connections 2023-10-16T10:02:29.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.815+0000 7f984f4e0640 1 --2- 172.21.15.84:0/2277849061 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f982c06ad40 0x7f982c06d200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:29.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.815+0000 7f984f4e0640 1 --2- 172.21.15.84:0/2277849061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f984815ad70 0x7f98480acf10 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:29.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.815+0000 7f984f4e0640 1 --2- 172.21.15.84:0/2277849061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98480c9ca0 0x7f98480c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:29.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.815+0000 7f984f4e0640 1 --2- 172.21.15.84:0/2277849061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f984806a4b0 0x7f9848150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:29.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.815+0000 7f984f4e0640 1 -- 172.21.15.84:0/2277849061 >> 172.21.15.84:0/2277849061 conn(0x7f98480ce9b0 msgr2=0x7f984806b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:29.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.815+0000 7f984f4e0640 1 -- 172.21.15.84:0/2277849061 shutdown_connections 2023-10-16T10:02:29.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:29.815+0000 7f984f4e0640 1 -- 172.21.15.84:0/2277849061 wait complete. 2023-10-16T10:02:29.839 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-16T10:02:29.840 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-16-10_03_00 in ~31s... 2023-10-16T10:02:29.840 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-16-10_04_00 in ~91s... 2023-10-16T10:02:29.840 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m', '2m'], expected_retention: [] 2023-10-16T10:02:29.841 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-16T10:02:30.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.059+0000 7f573ec02640 1 Processor -- start 2023-10-16T10:02:30.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.059+0000 7f573ec02640 1 -- start start 2023-10-16T10:02:30.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.060+0000 7f573ec02640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5738159950 0x7f573815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:30.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.061+0000 7f573ec02640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573815aef0 con 0x7f573806c810 2023-10-16T10:02:30.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.061+0000 7f573ec02640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573806cc80 con 0x7f5738070450 2023-10-16T10:02:30.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.061+0000 7f573ec02640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573815e490 con 0x7f5738159950 2023-10-16T10:02:30.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.061+0000 7f573e401640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f573806c810 0x7f57380c9090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:45420/0 (socket says 172.21.15.84:45420) 2023-10-16T10:02:30.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.061+0000 7f573e401640 1 -- 172.21.15.84:0/1839307114 learned_addr learned my addr 172.21.15.84:0/1839307114 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:02:30.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.061+0000 7f573dc00640 1 --2- 172.21.15.84:0/1839307114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5738159950 0x7f573815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:30.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.061+0000 7f573cbfe640 1 -- 172.21.15.84:0/1839307114 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4012199034 0 0) 0x7f5730001490 con 0x7f573806c810 2023-10-16T10:02:30.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.061+0000 7f573cbfe640 1 -- 172.21.15.84:0/1839307114 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5724003520 con 0x7f573806c810 2023-10-16T10:02:30.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.062+0000 7f573cbfe640 1 -- 172.21.15.84:0/1839307114 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2429990304 0 0) 0x7f5724003520 con 0x7f573806c810 2023-10-16T10:02:30.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.062+0000 7f573cbfe640 1 -- 172.21.15.84:0/1839307114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5738159950 msgr2=0x7f573815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:30.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.062+0000 7f573cbfe640 1 --2- 172.21.15.84:0/1839307114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5738159950 0x7f573815a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:30.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.062+0000 7f573cbfe640 1 -- 172.21.15.84:0/1839307114 >> v1:172.21.15.157:6789/0 conn(0x7f5738070450 legacy=0x7f573806c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:30.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.062+0000 7f573cbfe640 1 -- 172.21.15.84:0/1839307114 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573815d910 con 0x7f573806c810 2023-10-16T10:02:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.062+0000 7f573cbfe640 1 -- 172.21.15.84:0/1839307114 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f5730001b80 con 0x7f573806c810 2023-10-16T10:02:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.062+0000 7f573cbfe640 1 -- 172.21.15.84:0/1839307114 >> v1:172.21.15.84:6789/0 conn(0x7f573806c810 legacy=0x7f57380c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.062+0000 7f573cbfe640 1 --2- 172.21.15.84:0/1839307114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57240039a0 0x7f5724003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.062+0000 7f573cbfe640 1 --2- 172.21.15.84:0/1839307114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57240047c0 0x7f5724004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:30.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.062+0000 7f573cbfe640 1 --2- 172.21.15.84:0/1839307114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57240052a0 0x7f572400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:30.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.062+0000 7f573cbfe640 1 -- 172.21.15.84:0/1839307114 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573815e490 con 0x7f57240047c0 2023-10-16T10:02:30.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.062+0000 7f573cbfe640 1 -- 172.21.15.84:0/1839307114 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572400c0d0 con 0x7f57240039a0 2023-10-16T10:02:30.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.062+0000 7f573cbfe640 1 -- 172.21.15.84:0/1839307114 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572400c290 con 0x7f57240052a0 2023-10-16T10:02:30.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.062+0000 7f573dc00640 1 --2- 172.21.15.84:0/1839307114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5738159950 0x7f573815a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:02:30.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.062+0000 7f573d3ff640 1 --2- 172.21.15.84:0/1839307114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57240047c0 0x7f5724004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:30.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.063+0000 7f573d3ff640 1 -- 172.21.15.84:0/1839307114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57240052a0 msgr2=0x7f572400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:30.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.063+0000 7f573d3ff640 1 --2- 172.21.15.84:0/1839307114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57240052a0 0x7f572400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:30.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.063+0000 7f573d3ff640 1 -- 172.21.15.84:0/1839307114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57240039a0 msgr2=0x7f5724003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:30.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.063+0000 7f573d3ff640 1 --2- 172.21.15.84:0/1839307114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57240039a0 0x7f5724003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:30.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.063+0000 7f573d3ff640 1 -- 172.21.15.84:0/1839307114 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5724003520 con 0x7f57240047c0 2023-10-16T10:02:30.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.064+0000 7f573d3ff640 1 --2- 172.21.15.84:0/1839307114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57240047c0 0x7f5724004c40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5728001330 tx=0x7f57240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc5e60bffee5c50 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:30.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.064+0000 7f573cbfe640 1 -- 172.21.15.84:0/1839307114 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57280052d0 con 0x7f57240047c0 2023-10-16T10:02:30.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.064+0000 7f573cbfe640 1 -- 172.21.15.84:0/1839307114 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572802e050 con 0x7f57240047c0 2023-10-16T10:02:30.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.064+0000 7f573cbfe640 1 -- 172.21.15.84:0/1839307114 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57280035f0 con 0x7f57240047c0 2023-10-16T10:02:30.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.065+0000 7f573ec02640 1 -- 172.21.15.84:0/1839307114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57240047c0 msgr2=0x7f5724004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:30.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.065+0000 7f573ec02640 1 --2- 172.21.15.84:0/1839307114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57240047c0 0x7f5724004c40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5728001330 tx=0x7f57240057f0 comp rx=0 tx=0).stop 2023-10-16T10:02:30.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.065+0000 7f573e401640 1 -- 172.21.15.84:0/1839307114 reap_dead start 2023-10-16T10:02:30.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.065+0000 7f573ec02640 1 -- 172.21.15.84:0/1839307114 shutdown_connections 2023-10-16T10:02:30.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.065+0000 7f573ec02640 1 -- 172.21.15.84:0/1839307114 >> 172.21.15.84:0/1839307114 conn(0x7f57380ce6a0 msgr2=0x7f57380ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:30.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.065+0000 7f573ec02640 1 -- 172.21.15.84:0/1839307114 shutdown_connections 2023-10-16T10:02:30.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.065+0000 7f573ec02640 1 -- 172.21.15.84:0/1839307114 wait complete. 2023-10-16T10:02:30.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.067+0000 7f573ec02640 1 Processor -- start 2023-10-16T10:02:30.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.067+0000 7f573ec02640 1 -- start start 2023-10-16T10:02:30.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.068+0000 7f573ec02640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57240047c0 0x7f573814df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:30.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.068+0000 7f573ec02640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57380c21d0 0x7f57380c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:30.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.068+0000 7f573ec02640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57380ccb00 0x7f57380ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:30.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.068+0000 7f573ec02640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57380cd4c0 con 0x7f57380c21d0 2023-10-16T10:02:30.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.068+0000 7f573ec02640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573814e480 con 0x7f57380ccb00 2023-10-16T10:02:30.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.068+0000 7f573ec02640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573814ef90 con 0x7f57240047c0 2023-10-16T10:02:30.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.068+0000 7f573d3ff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57380c21d0 0x7f57380c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:30.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.068+0000 7f573d3ff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57380c21d0 0x7f57380c44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55974/0 (socket says 172.21.15.84:55974) 2023-10-16T10:02:30.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.068+0000 7f573d3ff640 1 -- 172.21.15.84:0/4200380901 learned_addr learned my addr 172.21.15.84:0/4200380901 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:02:30.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.068+0000 7f573dc00640 1 --2- 172.21.15.84:0/4200380901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57240047c0 0x7f573814df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:30.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.069+0000 7f573d3ff640 1 -- 172.21.15.84:0/4200380901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57240047c0 msgr2=0x7f573814df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:30.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.069+0000 7f573d3ff640 1 --2- 172.21.15.84:0/4200380901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57240047c0 0x7f573814df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:30.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.069+0000 7f573d3ff640 1 -- 172.21.15.84:0/4200380901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57380ccb00 msgr2=0x7f57380ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:30.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.069+0000 7f573d3ff640 1 --2- 172.21.15.84:0/4200380901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57380ccb00 0x7f57380ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:30.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.069+0000 7f573d3ff640 1 -- 172.21.15.84:0/4200380901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57380c4a10 con 0x7f57380c21d0 2023-10-16T10:02:30.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.069+0000 7f573d3ff640 1 --2- 172.21.15.84:0/4200380901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57380c21d0 0x7f57380c44d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f572802bd00 tx=0x7f5728001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:30.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.069+0000 7f572e7fc640 1 -- 172.21.15.84:0/4200380901 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57280076c0 con 0x7f57380c21d0 2023-10-16T10:02:30.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.069+0000 7f572e7fc640 1 -- 172.21.15.84:0/4200380901 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5728007a30 con 0x7f57380c21d0 2023-10-16T10:02:30.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.069+0000 7f572e7fc640 1 -- 172.21.15.84:0/4200380901 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728006c00 con 0x7f57380c21d0 2023-10-16T10:02:30.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.069+0000 7f573ec02640 1 -- 172.21.15.84:0/4200380901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573814e680 con 0x7f57380c21d0 2023-10-16T10:02:30.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.069+0000 7f573ec02640 1 -- 172.21.15.84:0/4200380901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573814e940 con 0x7f57380c21d0 2023-10-16T10:02:30.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.071+0000 7f573ec02640 1 -- 172.21.15.84:0/4200380901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f573815a500 con 0x7f57380c21d0 2023-10-16T10:02:30.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.072+0000 7f572e7fc640 1 -- 172.21.15.84:0/4200380901 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f57280035f0 con 0x7f57380c21d0 2023-10-16T10:02:30.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.072+0000 7f572e7fc640 1 --2- 172.21.15.84:0/4200380901 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f571c06afe0 0x7f571c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:30.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.073+0000 7f572e7fc640 1 -- 172.21.15.84:0/4200380901 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f57280ab750 con 0x7f57380c21d0 2023-10-16T10:02:30.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.073+0000 7f573dc00640 1 --2- 172.21.15.84:0/4200380901 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f571c06afe0 0x7f571c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:30.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.074+0000 7f573dc00640 1 --2- 172.21.15.84:0/4200380901 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f571c06afe0 0x7f571c06d4a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5734006a90 tx=0x7f5734008040 comp rx=0 tx=0).ready entity=mgr.6165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:30.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.077+0000 7f572e7fc640 1 -- 172.21.15.84:0/4200380901 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5728076480 con 0x7f57380c21d0 2023-10-16T10:02:30.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.303+0000 7f573ec02640 1 -- 172.21.15.84:0/4200380901 --> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f573815ded0 con 0x7f571c06afe0 2023-10-16T10:02:30.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.316+0000 7f572e7fc640 1 -- 172.21.15.84:0/4200380901 <== mgr.6165 v2:172.21.15.84:6832/633652932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7f573400be30 con 0x7f571c06afe0 2023-10-16T10:02:30.318 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:02:30.318 INFO:teuthology.orchestra.run.smithi084.stdout:{"path": "snap_test_dir1", "schedule": ["1m", "2m"], "retention": [{}, {}]} 2023-10-16T10:02:30.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.319+0000 7f573ec02640 1 -- 172.21.15.84:0/4200380901 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f571c06afe0 msgr2=0x7f571c06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:30.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.319+0000 7f573ec02640 1 --2- 172.21.15.84:0/4200380901 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f571c06afe0 0x7f571c06d4a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5734006a90 tx=0x7f5734008040 comp rx=0 tx=0).stop 2023-10-16T10:02:30.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.319+0000 7f573ec02640 1 -- 172.21.15.84:0/4200380901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57380c21d0 msgr2=0x7f57380c44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:30.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.319+0000 7f573ec02640 1 --2- 172.21.15.84:0/4200380901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57380c21d0 0x7f57380c44d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f572802bd00 tx=0x7f5728001120 comp rx=0 tx=0).stop 2023-10-16T10:02:30.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.319+0000 7f573ec02640 1 -- 172.21.15.84:0/4200380901 shutdown_connections 2023-10-16T10:02:30.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.319+0000 7f573ec02640 1 --2- 172.21.15.84:0/4200380901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57380ccb00 0x7f57380ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:30.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.319+0000 7f573ec02640 1 --2- 172.21.15.84:0/4200380901 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f571c06afe0 0x7f571c06d4a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:30.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.319+0000 7f573ec02640 1 --2- 172.21.15.84:0/4200380901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57380c21d0 0x7f57380c44d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:30.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.319+0000 7f573ec02640 1 --2- 172.21.15.84:0/4200380901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57240047c0 0x7f573814df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:30.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.319+0000 7f573ec02640 1 -- 172.21.15.84:0/4200380901 >> 172.21.15.84:0/4200380901 conn(0x7f57380ce6a0 msgr2=0x7f5738070830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:30.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.319+0000 7f573ec02640 1 -- 172.21.15.84:0/4200380901 shutdown_connections 2023-10-16T10:02:30.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:02:30.319+0000 7f573ec02640 1 -- 172.21.15.84:0/4200380901 wait complete. 2023-10-16T10:02:30.339 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m", "2m"], "retention": [{}, {}]} 2023-10-16T10:02:30.340 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m', '2m'], 'retention': [{}, {}]} 2023-10-16T10:02:30.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:30.340 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:30.397 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:30.397 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:30.398 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:31.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:31.400 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:31.462 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:31.463 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:31.463 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:32.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:32.465 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:32.527 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:32.528 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:32.528 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:32.963 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:02:32.972 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:02:33.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:33.530 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:33.594 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:33.594 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:33.594 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:34.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:34.596 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:34.660 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:34.660 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:34.660 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:35.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:35.662 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:35.727 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:35.727 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:35.727 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:36.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:36.729 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:36.791 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:36.791 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:36.792 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:37.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:37.794 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:37.854 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:37.854 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:37.855 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:38.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:38.856 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:38.918 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:38.918 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:38.918 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:39.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:39.920 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:39.984 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:39.984 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:39.985 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:40.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:40.986 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:41.050 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:41.050 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:41.050 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:42.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:42.053 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:42.115 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:42.115 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:42.116 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:43.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:43.117 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:43.180 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:43.180 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:43.181 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:44.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:44.183 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:44.245 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:44.246 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:44.246 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:45.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:45.247 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:45.312 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:45.313 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:45.313 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:46.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:46.315 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:46.377 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:46.378 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:46.378 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:47.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:47.379 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:47.443 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:47.444 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:47.444 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:48.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:48.446 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:48.510 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:48.510 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:48.510 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:49.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:49.512 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:49.580 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:49.581 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:49.581 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:50.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:50.583 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:50.643 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:50.644 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:50.644 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:51.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:51.646 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:51.708 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:51.708 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:51.709 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:52.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:52.711 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:52.772 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:52.773 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:52.773 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:53.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:53.774 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:53.837 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:53.837 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:53.837 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:54.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:54.839 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:54.908 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:54.908 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:54.909 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:55.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:55.910 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:55.973 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:55.974 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:55.974 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:56.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:56.976 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:57.040 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:57.041 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:57.041 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:58.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:58.043 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:58.104 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:58.105 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:58.105 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:59.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:59.107 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:59.173 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:59.173 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:59.174 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:00.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:00.175 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:00.240 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:03:00.240 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:00.240 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:00.837 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:03:00.835+0000 7f9a9371d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:00.837 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:03:00.835+0000 7f9a9371d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:00.838 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:03:00.835+0000 7f9a9371d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:00.838 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:03:00.835+0000 7f9a9371d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:00.838 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:03:00.835+0000 7f9a9371d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:01.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:01.242 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:01.305 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:01.307 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:01.307 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:01.308 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:01.308 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-16-10_03_00_UTC'] 2023-10-16T10:03:01.308 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697450549.8399897, now = 1697450581.308723, timo = 31 2023-10-16T10:03:02.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:02.310 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:02.370 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:02.372 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:02.372 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:02.372 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:03.110 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:03:03.120 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:03:03.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:03.374 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:03.438 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:03.439 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:03.440 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:03.440 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:04.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:04.442 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:04.503 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:04.504 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:04.505 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:04.505 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:05.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:05.507 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:05.567 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:05.568 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:05.569 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:05.569 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:06.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:06.571 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:06.635 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:06.637 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:06.638 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:06.638 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:07.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:07.640 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:07.702 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:07.703 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:07.704 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:07.704 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:08.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:08.706 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:08.768 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:08.769 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:08.770 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:08.770 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:09.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:09.772 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:09.834 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:09.836 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:09.836 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:09.836 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:10.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:10.839 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:10.900 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:10.901 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:10.901 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:10.902 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:11.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:11.903 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:11.966 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:11.968 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:11.968 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:11.968 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:12.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:12.971 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:13.032 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:13.034 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:13.034 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:13.034 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:14.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:14.036 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:14.096 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:14.097 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:14.098 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:14.098 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:15.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:15.100 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:15.161 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:15.163 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:15.163 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:15.163 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:16.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:16.164 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:16.225 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:16.226 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:16.227 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:16.227 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:17.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:17.229 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:17.290 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:17.291 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:17.292 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:17.292 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:18.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:18.294 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:18.356 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:18.358 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:18.358 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:18.358 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:19.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:19.360 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:19.419 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:19.420 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:19.420 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:19.420 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:20.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:20.426 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:20.489 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:20.490 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:20.491 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:20.491 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:21.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:21.492 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:21.554 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:21.555 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:21.555 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:21.556 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:22.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:22.557 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:22.619 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:22.621 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:22.621 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:22.621 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:23.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:23.623 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:23.688 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:23.689 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:23.689 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:23.690 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:24.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:24.692 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:24.753 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:24.755 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:24.755 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:24.756 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:25.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:25.758 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:25.820 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:25.821 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:25.822 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:25.822 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:26.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:26.824 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:26.886 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:26.887 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:26.887 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:26.888 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:27.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:27.889 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:27.949 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:27.950 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:27.951 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:27.951 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:28.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:28.953 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:29.010 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:29.011 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:29.012 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:29.012 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:30.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:30.014 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:30.074 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:30.076 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:30.076 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:30.076 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:31.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:31.078 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:31.139 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:31.141 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:31.141 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:31.142 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:32.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:32.143 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:32.205 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:32.206 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:32.206 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:32.207 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:33.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:33.208 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:33.256 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:03:33.265 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:03:33.273 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:33.274 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:33.274 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:33.275 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:34.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:34.276 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:34.336 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:34.337 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:34.338 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:34.338 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:35.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:35.340 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:35.404 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:35.405 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:35.405 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:35.406 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:36.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:36.407 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:36.470 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:36.471 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:36.472 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:36.472 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:37.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:37.474 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:37.536 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:37.537 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:37.538 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:37.538 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:38.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:38.539 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:38.601 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:38.602 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:38.603 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:38.603 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:39.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:39.605 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:39.665 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:39.667 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:39.667 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:39.667 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:40.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:40.670 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:40.740 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:40.741 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:40.741 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:40.742 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:41.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:41.744 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:41.805 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:41.806 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:41.806 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:41.807 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:42.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:42.809 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:42.870 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:42.872 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:42.872 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:42.873 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:43.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:43.874 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:43.936 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:43.937 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:43.937 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:43.938 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:44.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:44.940 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:45.004 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:45.005 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:45.006 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:45.006 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:46.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:46.008 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:46.072 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:46.073 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:46.074 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:46.074 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:47.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:47.076 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:47.137 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:47.138 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:47.138 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:47.139 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:48.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:48.141 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:48.203 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:48.205 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:48.205 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:48.206 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:49.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:49.207 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:49.270 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:49.271 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:49.271 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:49.272 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:50.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:50.274 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:50.335 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:50.336 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:50.337 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:50.337 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:51.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:51.338 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:51.400 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:51.402 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:51.402 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:51.403 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:52.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:52.404 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:52.467 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:52.468 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:52.469 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:52.469 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:53.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:53.471 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:53.533 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:53.534 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:53.534 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:53.535 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:54.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:54.536 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:54.599 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:54.601 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:54.601 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:54.601 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:55.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:55.603 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:55.665 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:55.666 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:55.667 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:55.667 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:56.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:56.669 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:56.732 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:56.733 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:56.734 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:56.734 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:57.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:57.736 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:57.798 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:57.799 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:57.800 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:57.800 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:58.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:58.802 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:58.864 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:58.865 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:58.866 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:58.866 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:59.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:59.867 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:59.929 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:59.930 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:59.930 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:03:59.931 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:04:00.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:00.933 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:04:00.991 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:04:00.991 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_04_00_UTC 2023-10-16T10:04:00.992 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC', 'scheduled-2023-10-16-10_04_00_UTC'} 2023-10-16T10:04:00.992 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-16-10_04_00_UTC'} 2023-10-16T10:04:00.993 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:04:00.993 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-16-10_04_00_UTC'] 2023-10-16T10:04:00.993 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697450549.8399897, now = 1697450640.9937844, timo = 91 2023-10-16T10:04:01.994 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-16T10:04:02.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.218+0000 7f2e4e472640 1 Processor -- start 2023-10-16T10:04:02.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.219+0000 7f2e4e472640 1 -- start start 2023-10-16T10:04:02.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.220+0000 7f2e4e472640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2e480ad970 0x7f2e480cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:02.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.220+0000 7f2e4e472640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e480ccfb0 0x7f2e480cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:02.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.220+0000 7f2e4e472640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4815b9d0 con 0x7f2e4806c040 2023-10-16T10:04:02.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.220+0000 7f2e4e472640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e4815bcb0 con 0x7f2e480ad970 2023-10-16T10:04:02.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.220+0000 7f2e4e472640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4815bf90 con 0x7f2e480ccfb0 2023-10-16T10:04:02.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.220+0000 7f2e4cc6f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2e480ad970 0x7f2e480cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:02.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.220+0000 7f2e4d470640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2e4806c040 0x7f2e4806c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:50484/0 (socket says 172.21.15.84:50484) 2023-10-16T10:04:02.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.221+0000 7f2e4d470640 1 -- 172.21.15.84:0/3992395951 learned_addr learned my addr 172.21.15.84:0/3992395951 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:02.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.221+0000 7f2e4cc6f640 1 -- 172.21.15.84:0/3992395951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e480ccfb0 msgr2=0x7f2e480cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:02.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.221+0000 7f2e4dc71640 1 --2- 172.21.15.84:0/3992395951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e480ccfb0 0x7f2e480cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:02.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.221+0000 7f2e4cc6f640 1 --2- 172.21.15.84:0/3992395951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e480ccfb0 0x7f2e480cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.221+0000 7f2e4cc6f640 1 -- 172.21.15.84:0/3992395951 >> v1:172.21.15.84:6789/0 conn(0x7f2e4806c040 legacy=0x7f2e4806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:02.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.221+0000 7f2e3ffff640 1 -- 172.21.15.84:0/3992395951 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746095061 0 0) 0x7f2e4815b9d0 con 0x7f2e4806c040 2023-10-16T10:04:02.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.221+0000 7f2e4cc6f640 1 -- 172.21.15.84:0/3992395951 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e480cc270 con 0x7f2e480ad970 2023-10-16T10:04:02.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.221+0000 7f2e4dc71640 1 --2- 172.21.15.84:0/3992395951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e480ccfb0 0x7f2e480cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:02.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.222+0000 7f2e4cc6f640 1 --2- 172.21.15.84:0/3992395951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e480ad970 0x7f2e480cca70 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2e380097a0 tx=0x7f2e3802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489b53a3f8b46a7b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:02.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.222+0000 7f2e3ffff640 1 -- 172.21.15.84:0/3992395951 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3802d040 con 0x7f2e480ad970 2023-10-16T10:04:02.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.223+0000 7f2e3ffff640 1 -- 172.21.15.84:0/3992395951 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e38031810 con 0x7f2e480ad970 2023-10-16T10:04:02.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.223+0000 7f2e3ffff640 1 -- 172.21.15.84:0/3992395951 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38031ba0 con 0x7f2e480ad970 2023-10-16T10:04:02.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.223+0000 7f2e4e472640 1 -- 172.21.15.84:0/3992395951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e480ad970 msgr2=0x7f2e480cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:02.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.223+0000 7f2e4e472640 1 --2- 172.21.15.84:0/3992395951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e480ad970 0x7f2e480cca70 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2e380097a0 tx=0x7f2e3802b8b0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.223+0000 7f2e4e472640 1 -- 172.21.15.84:0/3992395951 shutdown_connections 2023-10-16T10:04:02.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.223+0000 7f2e4e472640 1 --2- 172.21.15.84:0/3992395951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e480ccfb0 0x7f2e480cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.223+0000 7f2e4e472640 1 --2- 172.21.15.84:0/3992395951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e480ad970 0x7f2e480cca70 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.223+0000 7f2e4e472640 1 -- 172.21.15.84:0/3992395951 >> 172.21.15.84:0/3992395951 conn(0x7f2e480ce6a0 msgr2=0x7f2e480ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:02.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.223+0000 7f2e4e472640 1 -- 172.21.15.84:0/3992395951 shutdown_connections 2023-10-16T10:04:02.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.223+0000 7f2e4e472640 1 -- 172.21.15.84:0/3992395951 wait complete. 2023-10-16T10:04:02.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.225+0000 7f2e4e472640 1 Processor -- start 2023-10-16T10:04:02.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.225+0000 7f2e4e472640 1 -- start start 2023-10-16T10:04:02.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.226+0000 7f2e4e472640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e4806c040 0x7f2e48156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:02.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.226+0000 7f2e4e472640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e480ad970 0x7f2e480aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:02.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.226+0000 7f2e4e472640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e480ccfb0 0x7f2e480af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:02.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.226+0000 7f2e4e472640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e480c21b0 con 0x7f2e480ad970 2023-10-16T10:04:02.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.226+0000 7f2e4e472640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e480afb90 con 0x7f2e4806c040 2023-10-16T10:04:02.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.226+0000 7f2e4e472640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e480afe70 con 0x7f2e480ccfb0 2023-10-16T10:04:02.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.226+0000 7f2e4cc6f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e480ad970 0x7f2e480aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:02.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.226+0000 7f2e4cc6f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e480ad970 0x7f2e480aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46956/0 (socket says 172.21.15.84:46956) 2023-10-16T10:04:02.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.226+0000 7f2e4cc6f640 1 -- 172.21.15.84:0/3379801130 learned_addr learned my addr 172.21.15.84:0/3379801130 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:02.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.227+0000 7f2e4cc6f640 1 -- 172.21.15.84:0/3379801130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e480ccfb0 msgr2=0x7f2e480af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:02.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.227+0000 7f2e4dc71640 1 --2- 172.21.15.84:0/3379801130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e480ccfb0 0x7f2e480af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:02.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.227+0000 7f2e4cc6f640 1 --2- 172.21.15.84:0/3379801130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e480ccfb0 0x7f2e480af510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.227+0000 7f2e4d470640 1 --2- 172.21.15.84:0/3379801130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e4806c040 0x7f2e48156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:02.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.227+0000 7f2e4cc6f640 1 -- 172.21.15.84:0/3379801130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e4806c040 msgr2=0x7f2e48156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:02.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.227+0000 7f2e4cc6f640 1 --2- 172.21.15.84:0/3379801130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e4806c040 0x7f2e48156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.227+0000 7f2e4cc6f640 1 -- 172.21.15.84:0/3379801130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e38009400 con 0x7f2e480ad970 2023-10-16T10:04:02.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.227+0000 7f2e4cc6f640 1 --2- 172.21.15.84:0/3379801130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e480ad970 0x7f2e480aefd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2e380096f0 tx=0x7f2e380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:02.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.228+0000 7f2e3dffb640 1 -- 172.21.15.84:0/3379801130 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3803ad60 con 0x7f2e480ad970 2023-10-16T10:04:02.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.228+0000 7f2e3dffb640 1 -- 172.21.15.84:0/3379801130 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e38044400 con 0x7f2e480ad970 2023-10-16T10:04:02.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.228+0000 7f2e3dffb640 1 -- 172.21.15.84:0/3379801130 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380447b0 con 0x7f2e480ad970 2023-10-16T10:04:02.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.228+0000 7f2e4e472640 1 -- 172.21.15.84:0/3379801130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e480b0150 con 0x7f2e480ad970 2023-10-16T10:04:02.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.228+0000 7f2e4e472640 1 -- 172.21.15.84:0/3379801130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e481571a0 con 0x7f2e480ad970 2023-10-16T10:04:02.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.229+0000 7f2e4e472640 1 -- 172.21.15.84:0/3379801130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4806b330 con 0x7f2e480ad970 2023-10-16T10:04:02.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.230+0000 7f2e3dffb640 1 -- 172.21.15.84:0/3379801130 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 28) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f2e3804d440 con 0x7f2e480ad970 2023-10-16T10:04:02.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.230+0000 7f2e3dffb640 1 --2- 172.21.15.84:0/3379801130 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f2e2c06afe0 0x7f2e2c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:02.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.231+0000 7f2e3dffb640 1 -- 172.21.15.84:0/3379801130 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6111+0+0 (secure 0 0 0) 0x7f2e380abee0 con 0x7f2e480ad970 2023-10-16T10:04:02.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.231+0000 7f2e4d470640 1 --2- 172.21.15.84:0/3379801130 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f2e2c06afe0 0x7f2e2c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:02.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.231+0000 7f2e4d470640 1 --2- 172.21.15.84:0/3379801130 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f2e2c06afe0 0x7f2e2c06d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2e44000f60 tx=0x7f2e44009040 comp rx=0 tx=0).ready entity=mgr.6165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:02.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.235+0000 7f2e3dffb640 1 -- 172.21.15.84:0/3379801130 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2e38076bd0 con 0x7f2e480ad970 2023-10-16T10:04:02.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.447+0000 7f2e4e472640 1 -- 172.21.15.84:0/3379801130 --> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e480cc550 con 0x7f2e2c06afe0 2023-10-16T10:04:02.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.629+0000 7f2e3dffb640 1 -- 172.21.15.84:0/3379801130 <== mgr.6165 v2:172.21.15.84:6832/633652932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f2e44001c70 con 0x7f2e2c06afe0 2023-10-16T10:04:02.631 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path snap_test_dir1 2023-10-16T10:04:02.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.632+0000 7f2e4e472640 1 -- 172.21.15.84:0/3379801130 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f2e2c06afe0 msgr2=0x7f2e2c06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:02.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.632+0000 7f2e4e472640 1 --2- 172.21.15.84:0/3379801130 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f2e2c06afe0 0x7f2e2c06d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2e44000f60 tx=0x7f2e44009040 comp rx=0 tx=0).stop 2023-10-16T10:04:02.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.632+0000 7f2e4e472640 1 -- 172.21.15.84:0/3379801130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e480ad970 msgr2=0x7f2e480aefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:02.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.632+0000 7f2e4e472640 1 --2- 172.21.15.84:0/3379801130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e480ad970 0x7f2e480aefd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2e380096f0 tx=0x7f2e380046c0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.632+0000 7f2e4e472640 1 -- 172.21.15.84:0/3379801130 shutdown_connections 2023-10-16T10:04:02.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.632+0000 7f2e4e472640 1 --2- 172.21.15.84:0/3379801130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e480ccfb0 0x7f2e480af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.632+0000 7f2e4e472640 1 --2- 172.21.15.84:0/3379801130 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f2e2c06afe0 0x7f2e2c06d4a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.632+0000 7f2e4e472640 1 --2- 172.21.15.84:0/3379801130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e480ad970 0x7f2e480aefd0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.632+0000 7f2e4e472640 1 --2- 172.21.15.84:0/3379801130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e4806c040 0x7f2e48156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.632+0000 7f2e4e472640 1 -- 172.21.15.84:0/3379801130 >> 172.21.15.84:0/3379801130 conn(0x7f2e480ce6a0 msgr2=0x7f2e480ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:02.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.633+0000 7f2e4e472640 1 -- 172.21.15.84:0/3379801130 shutdown_connections 2023-10-16T10:04:02.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:02.633+0000 7f2e4e472640 1 -- 172.21.15.84:0/3379801130 wait complete. 2023-10-16T10:04:02.654 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-16T10:04:02.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:02.655 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:04:02.709 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:04:02.710 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_04_00_UTC 2023-10-16T10:04:02.711 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:04:02.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:02.712 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_03_00_UTC) 2023-10-16T10:04:02.775 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_04_00_UTC 2023-10-16T10:04:02.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:02.777 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_04_00_UTC) 2023-10-16T10:04:02.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:02.841 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-16T10:04:02.902 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T10:04:03.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.121+0000 7f09c49a9640 1 Processor -- start 2023-10-16T10:04:03.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.121+0000 7f09c49a9640 1 -- start start 2023-10-16T10:04:03.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.122+0000 7f09c49a9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f09c0159e90 0x7f09c00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:03.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.122+0000 7f09c49a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f09c006c140 0x7f09c006c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:03.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.123+0000 7f09c49a9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c006cad0 con 0x7f09c0159ab0 2023-10-16T10:04:03.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.123+0000 7f09c49a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c015d890 con 0x7f09c006c140 2023-10-16T10:04:03.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.123+0000 7f09c49a9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c0070360 con 0x7f09c0159e90 2023-10-16T10:04:03.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.123+0000 7f09be59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f09c0159ab0 0x7f09c006fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:50486/0 (socket says 172.21.15.84:50486) 2023-10-16T10:04:03.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.123+0000 7f09be59c640 1 -- 172.21.15.84:0/1961444631 learned_addr learned my addr 172.21.15.84:0/1961444631 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:03.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.123+0000 7f09bed9d640 1 --2- 172.21.15.84:0/1961444631 >> v2:172.21.15.84:3300/0 conn(0x7f09c006c140 0x7f09c006c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:03.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.123+0000 7f09bdd9b640 1 -- 172.21.15.84:0/1961444631 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070332464 0 0) 0x7f09b0001110 con 0x7f09c0159ab0 2023-10-16T10:04:03.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.123+0000 7f09bdd9b640 1 -- 172.21.15.84:0/1961444631 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09ac003520 con 0x7f09c0159ab0 2023-10-16T10:04:03.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.123+0000 7f09bed9d640 1 -- 172.21.15.84:0/1961444631 >> v2:172.21.15.157:3300/0 conn(0x7f09c0159e90 msgr2=0x7f09c00ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:03.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.123+0000 7f09bed9d640 1 --2- 172.21.15.84:0/1961444631 >> v2:172.21.15.157:3300/0 conn(0x7f09c0159e90 0x7f09c00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.123+0000 7f09bed9d640 1 -- 172.21.15.84:0/1961444631 >> v1:172.21.15.84:6789/0 conn(0x7f09c0159ab0 legacy=0x7f09c006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:03.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.123+0000 7f09bed9d640 1 -- 172.21.15.84:0/1961444631 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c00cd1f0 con 0x7f09c006c140 2023-10-16T10:04:03.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.125+0000 7f09bed9d640 1 --2- 172.21.15.84:0/1961444631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c006c140 0x7f09c006c590 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f09c006d900 tx=0x7f09b802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cc7bac4e656074 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:03.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.125+0000 7f09bdd9b640 1 -- 172.21.15.84:0/1961444631 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8031710 con 0x7f09c006c140 2023-10-16T10:04:03.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.125+0000 7f09bdd9b640 1 -- 172.21.15.84:0/1961444631 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b802f050 con 0x7f09c006c140 2023-10-16T10:04:03.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.125+0000 7f09bdd9b640 1 -- 172.21.15.84:0/1961444631 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8031a90 con 0x7f09c006c140 2023-10-16T10:04:03.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.125+0000 7f09c49a9640 1 -- 172.21.15.84:0/1961444631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c006c140 msgr2=0x7f09c006c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:03.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.125+0000 7f09c49a9640 1 --2- 172.21.15.84:0/1961444631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c006c140 0x7f09c006c590 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f09c006d900 tx=0x7f09b802b7b0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.125+0000 7f09c49a9640 1 -- 172.21.15.84:0/1961444631 shutdown_connections 2023-10-16T10:04:03.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.125+0000 7f09c49a9640 1 --2- 172.21.15.84:0/1961444631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c006c140 0x7f09c006c590 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.125+0000 7f09c49a9640 1 --2- 172.21.15.84:0/1961444631 >> v2:172.21.15.157:3300/0 conn(0x7f09c0159e90 0x7f09c00ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.126+0000 7f09c49a9640 1 -- 172.21.15.84:0/1961444631 >> 172.21.15.84:0/1961444631 conn(0x7f09c00ce9b0 msgr2=0x7f09c00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:03.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.126+0000 7f09c49a9640 1 -- 172.21.15.84:0/1961444631 shutdown_connections 2023-10-16T10:04:03.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.126+0000 7f09c49a9640 1 -- 172.21.15.84:0/1961444631 wait complete. 2023-10-16T10:04:03.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.127+0000 7f09c49a9640 1 Processor -- start 2023-10-16T10:04:03.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.127+0000 7f09c49a9640 1 -- start start 2023-10-16T10:04:03.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.128+0000 7f09c49a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c006c140 0x7f09c0158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:03.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.128+0000 7f09c49a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09c0159ab0 0x7f09c015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:03.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.128+0000 7f09c49a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c0159e90 0x7f09c0070760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:03.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.128+0000 7f09c49a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c0159190 con 0x7f09c0159e90 2023-10-16T10:04:03.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.128+0000 7f09c49a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c01e3730 con 0x7f09c006c140 2023-10-16T10:04:03.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.128+0000 7f09c49a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c01e2b20 con 0x7f09c0159ab0 2023-10-16T10:04:03.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.129+0000 7f09bed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c0159e90 0x7f09c0070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:03.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.129+0000 7f09bed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c0159e90 0x7f09c0070760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46984/0 (socket says 172.21.15.84:46984) 2023-10-16T10:04:03.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.129+0000 7f09bed9d640 1 -- 172.21.15.84:0/183042507 learned_addr learned my addr 172.21.15.84:0/183042507 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:03.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.129+0000 7f09b5d9b640 1 --2- 172.21.15.84:0/183042507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09c0159ab0 0x7f09c015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:03.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.129+0000 7f09be59c640 1 --2- 172.21.15.84:0/183042507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c006c140 0x7f09c0158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:03.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.129+0000 7f09bed9d640 1 -- 172.21.15.84:0/183042507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09c0159ab0 msgr2=0x7f09c015cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:03.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.129+0000 7f09bed9d640 1 --2- 172.21.15.84:0/183042507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09c0159ab0 0x7f09c015cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.129+0000 7f09bed9d640 1 -- 172.21.15.84:0/183042507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c006c140 msgr2=0x7f09c0158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:03.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.129+0000 7f09bed9d640 1 --2- 172.21.15.84:0/183042507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c006c140 0x7f09c0158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.129+0000 7f09bed9d640 1 -- 172.21.15.84:0/183042507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b8009400 con 0x7f09c0159e90 2023-10-16T10:04:03.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.129+0000 7f09be59c640 1 --2- 172.21.15.84:0/183042507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c006c140 0x7f09c0158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:03.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.130+0000 7f09bed9d640 1 --2- 172.21.15.84:0/183042507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c0159e90 0x7f09c0070760 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f09b802b7f0 tx=0x7f09b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:03.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.130+0000 7f09b77fe640 1 -- 172.21.15.84:0/183042507 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b803c810 con 0x7f09c0159e90 2023-10-16T10:04:03.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.130+0000 7f09b77fe640 1 -- 172.21.15.84:0/183042507 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b803c970 con 0x7f09c0159e90 2023-10-16T10:04:03.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.130+0000 7f09c49a9640 1 -- 172.21.15.84:0/183042507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c01e2e00 con 0x7f09c0159e90 2023-10-16T10:04:03.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.130+0000 7f09b77fe640 1 -- 172.21.15.84:0/183042507 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b803cd20 con 0x7f09c0159e90 2023-10-16T10:04:03.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.130+0000 7f09c49a9640 1 -- 172.21.15.84:0/183042507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c01e33c0 con 0x7f09c0159e90 2023-10-16T10:04:03.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.131+0000 7f09c49a9640 1 -- 172.21.15.84:0/183042507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c01e38c0 con 0x7f09c0159e90 2023-10-16T10:04:03.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.132+0000 7f09b77fe640 1 -- 172.21.15.84:0/183042507 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 28) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f09b803a040 con 0x7f09c0159e90 2023-10-16T10:04:03.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.133+0000 7f09b77fe640 1 --2- 172.21.15.84:0/183042507 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f09a006afe0 0x7f09a006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:03.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.133+0000 7f09b77fe640 1 -- 172.21.15.84:0/183042507 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6111+0+0 (secure 0 0 0) 0x7f09b80ab2b0 con 0x7f09c0159e90 2023-10-16T10:04:03.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.133+0000 7f09be59c640 1 --2- 172.21.15.84:0/183042507 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f09a006afe0 0x7f09a006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:03.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.134+0000 7f09be59c640 1 --2- 172.21.15.84:0/183042507 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f09a006afe0 0x7f09a006d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f09b0001370 tx=0x7f09b0009040 comp rx=0 tx=0).ready entity=mgr.6165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:03.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.138+0000 7f09b77fe640 1 -- 172.21.15.84:0/183042507 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f09b80760c0 con 0x7f09c0159e90 2023-10-16T10:04:03.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.384+0000 7f09c49a9640 1 -- 172.21.15.84:0/183042507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f09c004e160 con 0x7f09c0159e90 2023-10-16T10:04:03.470 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:04:03.479 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:04:03.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.923+0000 7f09b77fe640 1 -- 172.21.15.84:0/183042507 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v29) v1 ==== 93+0+0 (secure 0 0 0) 0x7f09b80796e0 con 0x7f09c0159e90 2023-10-16T10:04:03.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.931+0000 7f09c49a9640 1 -- 172.21.15.84:0/183042507 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f09a006afe0 msgr2=0x7f09a006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:03.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.931+0000 7f09c49a9640 1 --2- 172.21.15.84:0/183042507 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f09a006afe0 0x7f09a006d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f09b0001370 tx=0x7f09b0009040 comp rx=0 tx=0).stop 2023-10-16T10:04:03.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.931+0000 7f09c49a9640 1 -- 172.21.15.84:0/183042507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c0159e90 msgr2=0x7f09c0070760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:03.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.931+0000 7f09c49a9640 1 --2- 172.21.15.84:0/183042507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c0159e90 0x7f09c0070760 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f09b802b7f0 tx=0x7f09b80046c0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.932+0000 7f09c49a9640 1 -- 172.21.15.84:0/183042507 shutdown_connections 2023-10-16T10:04:03.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.932+0000 7f09c49a9640 1 --2- 172.21.15.84:0/183042507 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f09a006afe0 0x7f09a006d4a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.932+0000 7f09c49a9640 1 --2- 172.21.15.84:0/183042507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c0159e90 0x7f09c0070760 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.932+0000 7f09c49a9640 1 --2- 172.21.15.84:0/183042507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09c0159ab0 0x7f09c015cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.932+0000 7f09c49a9640 1 --2- 172.21.15.84:0/183042507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c006c140 0x7f09c0158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.932+0000 7f09c49a9640 1 -- 172.21.15.84:0/183042507 >> 172.21.15.84:0/183042507 conn(0x7f09c00ce9b0 msgr2=0x7f09c00aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:03.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.933+0000 7f09c49a9640 1 -- 172.21.15.84:0/183042507 shutdown_connections 2023-10-16T10:04:03.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:03.934+0000 7f09c49a9640 1 -- 172.21.15.84:0/183042507 wait complete. 2023-10-16T10:04:03.953 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:04:03.954 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:04:04.048 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:04:04.147 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:04:04.148 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:04:04.148 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:04:04.201 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:04:04.202 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:04:04.291 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:04:04.447 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:04.445+0000 7fd18f213040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:04:04.491 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:04.489+0000 7ff35a63a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:04:04.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.490+0000 7f398ddff640 1 Processor -- start 2023-10-16T10:04:04.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.490+0000 7f398ddff640 1 -- start start 2023-10-16T10:04:04.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.492+0000 7f398ddff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f398815cd70 0x7f39880d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.492+0000 7f398ddff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f39880cded0 0x7f39880b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.492+0000 7f398ddff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398815d150 con 0x7f39880b8480 2023-10-16T10:04:04.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.492+0000 7f398ddff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39880d1180 con 0x7f39880cded0 2023-10-16T10:04:04.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.492+0000 7f398ddff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39880d1460 con 0x7f398815cd70 2023-10-16T10:04:04.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.492+0000 7f3987fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f39880b8480 0x7f39880cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:50498/0 (socket says 172.21.15.84:50498) 2023-10-16T10:04:04.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.492+0000 7f3987fff640 1 -- 172.21.15.84:0/3125220847 learned_addr learned my addr 172.21.15.84:0/3125220847 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:04.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.493+0000 7f398cdfd640 1 --2- 172.21.15.84:0/3125220847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f398815cd70 0x7f39880d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.493+0000 7f398d5fe640 1 --2- 172.21.15.84:0/3125220847 >> v2:172.21.15.157:3300/0 conn(0x7f39880cded0 0x7f39880b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.493+0000 7f39877fe640 1 -- 172.21.15.84:0/3125220847 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1530835047 0 0) 0x7f398815d150 con 0x7f39880b8480 2023-10-16T10:04:04.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.493+0000 7f39877fe640 1 -- 172.21.15.84:0/3125220847 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3978003520 con 0x7f39880b8480 2023-10-16T10:04:04.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.493+0000 7f39877fe640 1 -- 172.21.15.84:0/3125220847 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3279594984 0 0) 0x7f3978003520 con 0x7f39880b8480 2023-10-16T10:04:04.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.493+0000 7f39877fe640 1 -- 172.21.15.84:0/3125220847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f398815cd70 msgr2=0x7f39880d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:04.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.493+0000 7f39877fe640 1 --2- 172.21.15.84:0/3125220847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f398815cd70 0x7f39880d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.493+0000 7f39877fe640 1 -- 172.21.15.84:0/3125220847 >> v2:172.21.15.157:3300/0 conn(0x7f39880cded0 msgr2=0x7f39880b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:04.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.493+0000 7f39877fe640 1 --2- 172.21.15.84:0/3125220847 >> v2:172.21.15.157:3300/0 conn(0x7f39880cded0 0x7f39880b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.493+0000 7f39877fe640 1 -- 172.21.15.84:0/3125220847 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39880d1740 con 0x7f39880b8480 2023-10-16T10:04:04.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.493+0000 7f39877fe640 1 -- 172.21.15.84:0/3125220847 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f3974001670 con 0x7f39880b8480 2023-10-16T10:04:04.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.493+0000 7f39877fe640 1 -- 172.21.15.84:0/3125220847 >> v1:172.21.15.84:6789/0 conn(0x7f39880b8480 legacy=0x7f39880cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:04.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.493+0000 7f39877fe640 1 --2- 172.21.15.84:0/3125220847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3978003970 0x7f3978003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.493+0000 7f39877fe640 1 --2- 172.21.15.84:0/3125220847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f398815cd70 0x7f39780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.493+0000 7f39877fe640 1 --2- 172.21.15.84:0/3125220847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3978004d00 0x7f3978005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.493+0000 7f39877fe640 1 -- 172.21.15.84:0/3125220847 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39880d1180 con 0x7f3978004d00 2023-10-16T10:04:04.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.493+0000 7f39877fe640 1 -- 172.21.15.84:0/3125220847 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39880d1460 con 0x7f3978003970 2023-10-16T10:04:04.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.493+0000 7f39877fe640 1 -- 172.21.15.84:0/3125220847 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397800b980 con 0x7f398815cd70 2023-10-16T10:04:04.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.494+0000 7f398d5fe640 1 --2- 172.21.15.84:0/3125220847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3978004d00 0x7f3978005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.494+0000 7f3987fff640 1 --2- 172.21.15.84:0/3125220847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f398815cd70 0x7f39780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.494+0000 7f398cdfd640 1 --2- 172.21.15.84:0/3125220847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3978003970 0x7f3978003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.494+0000 7f398d5fe640 1 -- 172.21.15.84:0/3125220847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f398815cd70 msgr2=0x7f39780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:04.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.494+0000 7f398d5fe640 1 --2- 172.21.15.84:0/3125220847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f398815cd70 0x7f39780047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.494+0000 7f398d5fe640 1 -- 172.21.15.84:0/3125220847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3978003970 msgr2=0x7f3978003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:04.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.494+0000 7f398d5fe640 1 --2- 172.21.15.84:0/3125220847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3978003970 0x7f3978003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.494+0000 7f398d5fe640 1 -- 172.21.15.84:0/3125220847 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3978003520 con 0x7f3978004d00 2023-10-16T10:04:04.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.495+0000 7f398d5fe640 1 --2- 172.21.15.84:0/3125220847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3978004d00 0x7f3978005180 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3980009fc0 tx=0x7f39780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a726f7cb4490bd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:04.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.495+0000 7f39877fe640 1 -- 172.21.15.84:0/3125220847 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398002cc60 con 0x7f3978004d00 2023-10-16T10:04:04.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.495+0000 7f39877fe640 1 -- 172.21.15.84:0/3125220847 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398000f040 con 0x7f3978004d00 2023-10-16T10:04:04.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.495+0000 7f39877fe640 1 -- 172.21.15.84:0/3125220847 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3980034440 con 0x7f3978004d00 2023-10-16T10:04:04.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.496+0000 7f398ddff640 1 -- 172.21.15.84:0/3125220847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3978004d00 msgr2=0x7f3978005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:04.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.496+0000 7f398ddff640 1 --2- 172.21.15.84:0/3125220847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3978004d00 0x7f3978005180 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3980009fc0 tx=0x7f39780057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.496+0000 7f398d5fe640 1 -- 172.21.15.84:0/3125220847 reap_dead start 2023-10-16T10:04:04.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.496+0000 7f398ddff640 1 -- 172.21.15.84:0/3125220847 shutdown_connections 2023-10-16T10:04:04.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.496+0000 7f398ddff640 1 -- 172.21.15.84:0/3125220847 >> 172.21.15.84:0/3125220847 conn(0x7f39880cf570 msgr2=0x7f39880cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:04.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.496+0000 7f398ddff640 1 -- 172.21.15.84:0/3125220847 shutdown_connections 2023-10-16T10:04:04.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.496+0000 7f398ddff640 1 -- 172.21.15.84:0/3125220847 wait complete. 2023-10-16T10:04:04.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.498+0000 7f398ddff640 1 Processor -- start 2023-10-16T10:04:04.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.498+0000 7f398ddff640 1 -- start start 2023-10-16T10:04:04.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.499+0000 7f398ddff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3978004d00 0x7f39880ac490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.499+0000 7f398ddff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f398814ba10 0x7f39881f2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.499+0000 7f398ddff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39881f25d0 0x7f398815d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.499+0000 7f398ddff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398814b580 con 0x7f398814ba10 2023-10-16T10:04:04.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.499+0000 7f398ddff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39881f3030 con 0x7f3978004d00 2023-10-16T10:04:04.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.499+0000 7f398ddff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39881f2a70 con 0x7f39881f25d0 2023-10-16T10:04:04.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.499+0000 7f3987fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f398814ba10 0x7f39881f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.499+0000 7f3987fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f398814ba10 0x7f39881f2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47022/0 (socket says 172.21.15.84:47022) 2023-10-16T10:04:04.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.499+0000 7f3987fff640 1 -- 172.21.15.84:0/3734114121 learned_addr learned my addr 172.21.15.84:0/3734114121 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:04.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.500+0000 7f398d5fe640 1 --2- 172.21.15.84:0/3734114121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39881f25d0 0x7f398815d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.500+0000 7f398cdfd640 1 --2- 172.21.15.84:0/3734114121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3978004d00 0x7f39880ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.500+0000 7f3987fff640 1 -- 172.21.15.84:0/3734114121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39881f25d0 msgr2=0x7f398815d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:04.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.500+0000 7f3987fff640 1 --2- 172.21.15.84:0/3734114121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39881f25d0 0x7f398815d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.500+0000 7f3987fff640 1 -- 172.21.15.84:0/3734114121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3978004d00 msgr2=0x7f39880ac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:04.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.500+0000 7f3987fff640 1 --2- 172.21.15.84:0/3734114121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3978004d00 0x7f39880ac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.500+0000 7f3987fff640 1 -- 172.21.15.84:0/3734114121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39881f2d50 con 0x7f398814ba10 2023-10-16T10:04:04.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.500+0000 7f398cdfd640 1 --2- 172.21.15.84:0/3734114121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3978004d00 0x7f39880ac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:04.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.500+0000 7f3987fff640 1 --2- 172.21.15.84:0/3734114121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f398814ba10 0x7f39881f2090 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3974007310 tx=0x7f39740077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:04.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.500+0000 7f39857fa640 1 -- 172.21.15.84:0/3734114121 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397400f9a0 con 0x7f398814ba10 2023-10-16T10:04:04.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.500+0000 7f39857fa640 1 -- 172.21.15.84:0/3734114121 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397400fb00 con 0x7f398814ba10 2023-10-16T10:04:04.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.500+0000 7f39857fa640 1 -- 172.21.15.84:0/3734114121 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397400feb0 con 0x7f398814ba10 2023-10-16T10:04:04.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.500+0000 7f398ddff640 1 -- 172.21.15.84:0/3734114121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398814b210 con 0x7f398814ba10 2023-10-16T10:04:04.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.501+0000 7f398ddff640 1 -- 172.21.15.84:0/3734114121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398814c070 con 0x7f398814ba10 2023-10-16T10:04:04.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.502+0000 7f398ddff640 1 -- 172.21.15.84:0/3734114121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398814b730 con 0x7f398814ba10 2023-10-16T10:04:04.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.503+0000 7f39857fa640 1 -- 172.21.15.84:0/3734114121 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 29) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f397401d460 con 0x7f398814ba10 2023-10-16T10:04:04.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.504+0000 7f39857fa640 1 --2- 172.21.15.84:0/3734114121 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f396c06af90 0x7f396c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.515 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:04.504+0000 7fd18f213040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:04:04.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.504+0000 7f39857fa640 1 -- 172.21.15.84:0/3734114121 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6111+0+0 (secure 0 0 0) 0x7f3974089a70 con 0x7f398814ba10 2023-10-16T10:04:04.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.504+0000 7f398cdfd640 1 -- 172.21.15.84:0/3734114121 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f396c06af90 msgr2=0x7f396c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/633652932 2023-10-16T10:04:04.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.504+0000 7f398cdfd640 1 --2- 172.21.15.84:0/3734114121 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f396c06af90 0x7f396c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:04.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.508+0000 7f39857fa640 1 -- 172.21.15.84:0/3734114121 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f398814b730 con 0x7f398814ba10 2023-10-16T10:04:04.546 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:04.545+0000 7ff35a63a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:04:04.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.702+0000 7f398ddff640 1 -- 172.21.15.84:0/3734114121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f398814c070 con 0x7f398814ba10 2023-10-16T10:04:04.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.705+0000 7f398cdfd640 1 -- 172.21.15.84:0/3734114121 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f396c06af90 msgr2=0x7f396c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/633652932 2023-10-16T10:04:04.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.705+0000 7f398cdfd640 1 --2- 172.21.15.84:0/3734114121 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f396c06af90 0x7f396c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:04:04.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.705+0000 7f39857fa640 1 -- 172.21.15.84:0/3734114121 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+17138 (secure 0 0 0) 0x7f398814c070 con 0x7f398814ba10 2023-10-16T10:04:04.707 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:04.708 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":67,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:03:29.904431+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:02:11.266402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-16T10:02:12.562100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.13","mappings":[{"from":0,"to":5}]},{"pgid":"7.2d","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2816959180":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/4150953381":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/555434621":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/3440750653":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/3654971153":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/621840141":"2023-10-17T10:02:27.969914+0000","172.21.15.84:6833/585186562":"2023-10-17T10:02:27.969914+0000","172.21.15.84:6832/585186562":"2023-10-17T10:02:27.969914+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:04:04.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.707+0000 7f398ddff640 1 -- 172.21.15.84:0/3734114121 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f396c06af90 msgr2=0x7f396c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:04.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.707+0000 7f398ddff640 1 --2- 172.21.15.84:0/3734114121 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f396c06af90 0x7f396c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.707+0000 7f398ddff640 1 -- 172.21.15.84:0/3734114121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f398814ba10 msgr2=0x7f39881f2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:04.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.707+0000 7f398ddff640 1 --2- 172.21.15.84:0/3734114121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f398814ba10 0x7f39881f2090 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3974007310 tx=0x7f39740077d0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.708+0000 7f398ddff640 1 -- 172.21.15.84:0/3734114121 shutdown_connections 2023-10-16T10:04:04.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.708+0000 7f398ddff640 1 --2- 172.21.15.84:0/3734114121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39881f25d0 0x7f398815d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.708+0000 7f398ddff640 1 --2- 172.21.15.84:0/3734114121 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f396c06af90 0x7f396c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.708+0000 7f398ddff640 1 --2- 172.21.15.84:0/3734114121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f398814ba10 0x7f39881f2090 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.708+0000 7f398ddff640 1 --2- 172.21.15.84:0/3734114121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3978004d00 0x7f39880ac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.708+0000 7f398ddff640 1 -- 172.21.15.84:0/3734114121 >> 172.21.15.84:0/3734114121 conn(0x7f39880cf570 msgr2=0x7f39880cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:04.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.708+0000 7f398ddff640 1 -- 172.21.15.84:0/3734114121 shutdown_connections 2023-10-16T10:04:04.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.708+0000 7f398ddff640 1 -- 172.21.15.84:0/3734114121 wait complete. 2023-10-16T10:04:04.728 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:02:11.266402+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:02:12.562100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:04:04.729 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:04:04.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.941+0000 7f50f475a640 1 Processor -- start 2023-10-16T10:04:04.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.941+0000 7f50f475a640 1 -- start start 2023-10-16T10:04:04.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.943+0000 7f50f475a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f50ec15cd70 0x7f50ec0d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.943+0000 7f50f475a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50ec0cded0 0x7f50ec0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.943+0000 7f50f475a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ec15d150 con 0x7f50ec0b8480 2023-10-16T10:04:04.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.943+0000 7f50f475a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50ec0d1180 con 0x7f50ec15cd70 2023-10-16T10:04:04.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.943+0000 7f50f475a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec0d1460 con 0x7f50ec0cded0 2023-10-16T10:04:04.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.943+0000 7f50f24f6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f50ec15cd70 0x7f50ec0d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.943+0000 7f50f1cf5640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f50ec0b8480 0x7f50ec0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:50500/0 (socket says 172.21.15.84:50500) 2023-10-16T10:04:04.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.943+0000 7f50f1cf5640 1 -- 172.21.15.84:0/2348327623 learned_addr learned my addr 172.21.15.84:0/2348327623 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:04.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.944+0000 7f50f2cf7640 1 --2- 172.21.15.84:0/2348327623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50ec0cded0 0x7f50ec0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.944+0000 7f50f24f6640 1 -- 172.21.15.84:0/2348327623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50ec0cded0 msgr2=0x7f50ec0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:04.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.944+0000 7f50f24f6640 1 --2- 172.21.15.84:0/2348327623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50ec0cded0 0x7f50ec0b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.944+0000 7f50f24f6640 1 -- 172.21.15.84:0/2348327623 >> v1:172.21.15.84:6789/0 conn(0x7f50ec0b8480 legacy=0x7f50ec0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:04.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.944+0000 7f50f24f6640 1 -- 172.21.15.84:0/2348327623 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec0d1740 con 0x7f50ec15cd70 2023-10-16T10:04:04.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.945+0000 7f50f24f6640 1 --2- 172.21.15.84:0/2348327623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50ec15cd70 0x7f50ec0d2220 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f50e80097a0 tx=0x7f50e802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3605e20aa089c14 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:04.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.945+0000 7f50f14f4640 1 -- 172.21.15.84:0/2348327623 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e802d040 con 0x7f50ec15cd70 2023-10-16T10:04:04.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.945+0000 7f50f14f4640 1 -- 172.21.15.84:0/2348327623 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e8031810 con 0x7f50ec15cd70 2023-10-16T10:04:04.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.945+0000 7f50f14f4640 1 -- 172.21.15.84:0/2348327623 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e8031ba0 con 0x7f50ec15cd70 2023-10-16T10:04:04.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.946+0000 7f50f475a640 1 -- 172.21.15.84:0/2348327623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50ec15cd70 msgr2=0x7f50ec0d2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:04.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.946+0000 7f50f475a640 1 --2- 172.21.15.84:0/2348327623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50ec15cd70 0x7f50ec0d2220 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f50e80097a0 tx=0x7f50e802b8b0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.946+0000 7f50f475a640 1 -- 172.21.15.84:0/2348327623 shutdown_connections 2023-10-16T10:04:04.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.946+0000 7f50f475a640 1 --2- 172.21.15.84:0/2348327623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50ec0cded0 0x7f50ec0b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.946+0000 7f50f475a640 1 --2- 172.21.15.84:0/2348327623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50ec15cd70 0x7f50ec0d2220 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.946+0000 7f50f475a640 1 -- 172.21.15.84:0/2348327623 >> 172.21.15.84:0/2348327623 conn(0x7f50ec0cf570 msgr2=0x7f50ec0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:04.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.946+0000 7f50f475a640 1 -- 172.21.15.84:0/2348327623 shutdown_connections 2023-10-16T10:04:04.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.946+0000 7f50f475a640 1 -- 172.21.15.84:0/2348327623 wait complete. 2023-10-16T10:04:04.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.948+0000 7f50f475a640 1 Processor -- start 2023-10-16T10:04:04.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.948+0000 7f50f475a640 1 -- start start 2023-10-16T10:04:04.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.949+0000 7f50f475a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50ec0b8480 0x7f50ec0b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.949+0000 7f50f475a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50ec0cded0 0x7f50ec1583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.949+0000 7f50f475a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ec15cd70 0x7f50ec158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.949+0000 7f50f475a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec1597e0 con 0x7f50ec0b8480 2023-10-16T10:04:04.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.949+0000 7f50f475a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec0b4080 con 0x7f50ec15cd70 2023-10-16T10:04:04.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.949+0000 7f50f475a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec15d4e0 con 0x7f50ec0cded0 2023-10-16T10:04:04.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.949+0000 7f50f24f6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50ec0b8480 0x7f50ec0b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.949+0000 7f50f24f6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50ec0b8480 0x7f50ec0b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47040/0 (socket says 172.21.15.84:47040) 2023-10-16T10:04:04.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.949+0000 7f50f24f6640 1 -- 172.21.15.84:0/3522642629 learned_addr learned my addr 172.21.15.84:0/3522642629 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:04.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.949+0000 7f50f1cf5640 1 --2- 172.21.15.84:0/3522642629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50ec0cded0 0x7f50ec1583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.950+0000 7f50f2cf7640 1 --2- 172.21.15.84:0/3522642629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ec15cd70 0x7f50ec158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.950+0000 7f50f24f6640 1 -- 172.21.15.84:0/3522642629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50ec0cded0 msgr2=0x7f50ec1583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:04.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.950+0000 7f50f24f6640 1 --2- 172.21.15.84:0/3522642629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50ec0cded0 0x7f50ec1583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.950+0000 7f50f24f6640 1 -- 172.21.15.84:0/3522642629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ec15cd70 msgr2=0x7f50ec158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:04.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.950+0000 7f50f24f6640 1 --2- 172.21.15.84:0/3522642629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ec15cd70 0x7f50ec158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.950+0000 7f50f24f6640 1 -- 172.21.15.84:0/3522642629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e8009400 con 0x7f50ec0b8480 2023-10-16T10:04:04.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.950+0000 7f50f24f6640 1 --2- 172.21.15.84:0/3522642629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50ec0b8480 0x7f50ec0b3b40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f50e8000c00 tx=0x7f50e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:04.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.950+0000 7f50e2ffd640 1 -- 172.21.15.84:0/3522642629 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e803ad60 con 0x7f50ec0b8480 2023-10-16T10:04:04.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.950+0000 7f50e2ffd640 1 -- 172.21.15.84:0/3522642629 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e80443f0 con 0x7f50ec0b8480 2023-10-16T10:04:04.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.950+0000 7f50f475a640 1 -- 172.21.15.84:0/3522642629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ec158f40 con 0x7f50ec0b8480 2023-10-16T10:04:04.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.950+0000 7f50e2ffd640 1 -- 172.21.15.84:0/3522642629 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e80447a0 con 0x7f50ec0b8480 2023-10-16T10:04:04.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.950+0000 7f50f475a640 1 -- 172.21.15.84:0/3522642629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ec159500 con 0x7f50ec0b8480 2023-10-16T10:04:04.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.951+0000 7f50f475a640 1 -- 172.21.15.84:0/3522642629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ec15d340 con 0x7f50ec0b8480 2023-10-16T10:04:04.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.953+0000 7f50e2ffd640 1 -- 172.21.15.84:0/3522642629 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 29) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f50e804d450 con 0x7f50ec0b8480 2023-10-16T10:04:04.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.954+0000 7f50e2ffd640 1 --2- 172.21.15.84:0/3522642629 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f50d006af90 0x7f50d006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.954+0000 7f50e2ffd640 1 -- 172.21.15.84:0/3522642629 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6111+0+0 (secure 0 0 0) 0x7f50e80abd60 con 0x7f50ec0b8480 2023-10-16T10:04:04.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.954+0000 7f50f1cf5640 1 -- 172.21.15.84:0/3522642629 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f50d006af90 msgr2=0x7f50d006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/633652932 2023-10-16T10:04:04.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.954+0000 7f50f1cf5640 1 --2- 172.21.15.84:0/3522642629 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f50d006af90 0x7f50d006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:04.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:04.959+0000 7f50e2ffd640 1 -- 172.21.15.84:0/3522642629 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f50ec15d340 con 0x7f50ec0b8480 2023-10-16T10:04:04.979 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:04.977+0000 7fd18f213040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:04:04.991 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:04.990+0000 7ff35a63a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:04:05.032 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:05.031+0000 7fd18f213040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:04:05.043 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:05.042+0000 7ff35a63a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:04:05.086 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:05.085+0000 7fd18f213040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:04:05.097 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:05.095+0000 7ff35a63a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:04:05.148 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:05.146+0000 7fd18f213040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:04:05.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.155+0000 7f50f475a640 1 -- 172.21.15.84:0/3522642629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50ec15d340 con 0x7f50ec0b8480 2023-10-16T10:04:05.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.155+0000 7f50f1cf5640 1 -- 172.21.15.84:0/3522642629 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f50d006af90 msgr2=0x7f50d006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/633652932 2023-10-16T10:04:05.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.155+0000 7f50f1cf5640 1 --2- 172.21.15.84:0/3522642629 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f50d006af90 0x7f50d006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:04:05.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.155+0000 7f50e2ffd640 1 -- 172.21.15.84:0/3522642629 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7f50ec15d340 con 0x7f50ec0b8480 2023-10-16T10:04:05.158 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:04:05.158 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:05.157+0000 7ff35a63a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:04:05.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.158+0000 7f50f475a640 1 -- 172.21.15.84:0/3522642629 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f50d006af90 msgr2=0x7f50d006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:05.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.158+0000 7f50f475a640 1 --2- 172.21.15.84:0/3522642629 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f50d006af90 0x7f50d006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.158+0000 7f50f475a640 1 -- 172.21.15.84:0/3522642629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50ec0b8480 msgr2=0x7f50ec0b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.158+0000 7f50f475a640 1 --2- 172.21.15.84:0/3522642629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50ec0b8480 0x7f50ec0b3b40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f50e8000c00 tx=0x7f50e80046c0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.158+0000 7f50f475a640 1 -- 172.21.15.84:0/3522642629 shutdown_connections 2023-10-16T10:04:05.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.158+0000 7f50f475a640 1 --2- 172.21.15.84:0/3522642629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ec15cd70 0x7f50ec158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.158+0000 7f50f475a640 1 --2- 172.21.15.84:0/3522642629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50ec0cded0 0x7f50ec1583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.158+0000 7f50f475a640 1 --2- 172.21.15.84:0/3522642629 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f50d006af90 0x7f50d006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.158+0000 7f50f475a640 1 --2- 172.21.15.84:0/3522642629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50ec0b8480 0x7f50ec0b3b40 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.158+0000 7f50f475a640 1 -- 172.21.15.84:0/3522642629 >> 172.21.15.84:0/3522642629 conn(0x7f50ec0cf570 msgr2=0x7f50ec15bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:05.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.158+0000 7f50f475a640 1 -- 172.21.15.84:0/3522642629 shutdown_connections 2023-10-16T10:04:05.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.158+0000 7f50f475a640 1 -- 172.21.15.84:0/3522642629 wait complete. 2023-10-16T10:04:05.178 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:04:05.202 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:05.201+0000 7fd18f213040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:04:05.207 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:05.206+0000 7ff35a63a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:04:05.318 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:05.317+0000 7ff35a63a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:04:05.319 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:05.318+0000 7fd18f213040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:04:05.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.379+0000 7f4f2ef35640 1 Processor -- start 2023-10-16T10:04:05.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.379+0000 7f4f2ef35640 1 -- start start 2023-10-16T10:04:05.381 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:05.379+0000 7ff35a63a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:04:05.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.380+0000 7f4f2ef35640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4f280ca0a0 0x7f4f2806c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:05.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.380+0000 7f4f2ef35640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f2815d910 con 0x7f4f28070b60 2023-10-16T10:04:05.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.380+0000 7f4f2ef35640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f2815dbf0 con 0x7f4f28159950 2023-10-16T10:04:05.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.380+0000 7f4f2ef35640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f2815e490 con 0x7f4f280ca0a0 2023-10-16T10:04:05.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.381+0000 7f4f2e734640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4f280ca0a0 0x7f4f2806c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:05.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.381+0000 7f4f2e734640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4f280ca0a0 0x7f4f2806c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47052/0 (socket says 172.21.15.84:47052) 2023-10-16T10:04:05.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.381+0000 7f4f2e734640 1 -- 172.21.15.84:0/524024526 learned_addr learned my addr 172.21.15.84:0/524024526 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:05.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.381+0000 7f4f2e734640 1 -- 172.21.15.84:0/524024526 >> v1:172.21.15.157:6789/0 conn(0x7f4f28159950 legacy=0x7f4f28070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:05.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.381+0000 7f4f2e734640 1 -- 172.21.15.84:0/524024526 >> v1:172.21.15.84:6789/0 conn(0x7f4f28070b60 legacy=0x7f4f2806c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.381+0000 7f4f2e734640 1 -- 172.21.15.84:0/524024526 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2815ded0 con 0x7f4f280ca0a0 2023-10-16T10:04:05.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.382+0000 7f4f2e734640 1 --2- 172.21.15.84:0/524024526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f280ca0a0 0x7f4f2806c810 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f4f20009e50 tx=0x7f4f2002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9afd612a783bc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:05.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.382+0000 7f4f2cf31640 1 -- 172.21.15.84:0/524024526 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2002ca80 con 0x7f4f280ca0a0 2023-10-16T10:04:05.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.382+0000 7f4f2cf31640 1 -- 172.21.15.84:0/524024526 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2002cbe0 con 0x7f4f280ca0a0 2023-10-16T10:04:05.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.382+0000 7f4f2cf31640 1 -- 172.21.15.84:0/524024526 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f200343f0 con 0x7f4f280ca0a0 2023-10-16T10:04:05.386 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:05.383+0000 7fd18f213040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:04:05.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.383+0000 7f4f2ef35640 1 -- 172.21.15.84:0/524024526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f280ca0a0 msgr2=0x7f4f2806c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.383+0000 7f4f2ef35640 1 --2- 172.21.15.84:0/524024526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f280ca0a0 0x7f4f2806c810 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f4f20009e50 tx=0x7f4f2002bee0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.383+0000 7f4f2ef35640 1 -- 172.21.15.84:0/524024526 shutdown_connections 2023-10-16T10:04:05.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.383+0000 7f4f2ef35640 1 --2- 172.21.15.84:0/524024526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f280ca0a0 0x7f4f2806c810 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.383+0000 7f4f2ef35640 1 -- 172.21.15.84:0/524024526 >> 172.21.15.84:0/524024526 conn(0x7f4f280ce6a0 msgr2=0x7f4f280ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:05.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.383+0000 7f4f2ef35640 1 -- 172.21.15.84:0/524024526 shutdown_connections 2023-10-16T10:04:05.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.383+0000 7f4f2ef35640 1 -- 172.21.15.84:0/524024526 wait complete. 2023-10-16T10:04:05.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.385+0000 7f4f2ef35640 1 Processor -- start 2023-10-16T10:04:05.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.385+0000 7f4f2ef35640 1 -- start start 2023-10-16T10:04:05.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.386+0000 7f4f2ef35640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f28070b60 0x7f4f28155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:05.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.386+0000 7f4f2ef35640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f280ca0a0 0x7f4f280b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:05.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.386+0000 7f4f2ef35640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f28159950 0x7f4f280af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:05.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.386+0000 7f4f2ef35640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f280c26c0 con 0x7f4f28159950 2023-10-16T10:04:05.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.386+0000 7f4f2ef35640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f280b37c0 con 0x7f4f280ca0a0 2023-10-16T10:04:05.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.386+0000 7f4f2ef35640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f280b3aa0 con 0x7f4f28070b60 2023-10-16T10:04:05.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.386+0000 7f4f2e734640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f28159950 0x7f4f280af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:05.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.386+0000 7f4f2e734640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f28159950 0x7f4f280af1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47054/0 (socket says 172.21.15.84:47054) 2023-10-16T10:04:05.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.386+0000 7f4f2e734640 1 -- 172.21.15.84:0/2256413147 learned_addr learned my addr 172.21.15.84:0/2256413147 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:05.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.386+0000 7f4f2df33640 1 --2- 172.21.15.84:0/2256413147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f28070b60 0x7f4f28155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:05.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.387+0000 7f4f2d732640 1 --2- 172.21.15.84:0/2256413147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f280ca0a0 0x7f4f280b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:05.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.387+0000 7f4f2e734640 1 -- 172.21.15.84:0/2256413147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f28070b60 msgr2=0x7f4f28155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.387+0000 7f4f2e734640 1 --2- 172.21.15.84:0/2256413147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f28070b60 0x7f4f28155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.387+0000 7f4f2e734640 1 -- 172.21.15.84:0/2256413147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f280ca0a0 msgr2=0x7f4f280b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.387+0000 7f4f2e734640 1 --2- 172.21.15.84:0/2256413147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f280ca0a0 0x7f4f280b2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.387+0000 7f4f2e734640 1 -- 172.21.15.84:0/2256413147 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f20009b30 con 0x7f4f28159950 2023-10-16T10:04:05.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.387+0000 7f4f2df33640 1 --2- 172.21.15.84:0/2256413147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f28070b60 0x7f4f28155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:05.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.387+0000 7f4f2e734640 1 --2- 172.21.15.84:0/2256413147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f28159950 0x7f4f280af1f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f4f20036040 tx=0x7f4f20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:05.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.387+0000 7f4f1e7fc640 1 -- 172.21.15.84:0/2256413147 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2003e450 con 0x7f4f28159950 2023-10-16T10:04:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.387+0000 7f4f1e7fc640 1 -- 172.21.15.84:0/2256413147 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2003e5d0 con 0x7f4f28159950 2023-10-16T10:04:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.388+0000 7f4f1e7fc640 1 -- 172.21.15.84:0/2256413147 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2003e980 con 0x7f4f28159950 2023-10-16T10:04:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.388+0000 7f4f2ef35640 1 -- 172.21.15.84:0/2256413147 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f280b3d80 con 0x7f4f28159950 2023-10-16T10:04:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.388+0000 7f4f2ef35640 1 -- 172.21.15.84:0/2256413147 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f280accc0 con 0x7f4f28159950 2023-10-16T10:04:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.389+0000 7f4f2ef35640 1 -- 172.21.15.84:0/2256413147 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f2804e9b0 con 0x7f4f28159950 2023-10-16T10:04:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.390+0000 7f4f1e7fc640 1 -- 172.21.15.84:0/2256413147 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 29) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f4f2002cd50 con 0x7f4f28159950 2023-10-16T10:04:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.391+0000 7f4f1e7fc640 1 --2- 172.21.15.84:0/2256413147 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f4f0c06af90 0x7f4f0c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.391+0000 7f4f1e7fc640 1 -- 172.21.15.84:0/2256413147 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6111+0+0 (secure 0 0 0) 0x7f4f200ac520 con 0x7f4f28159950 2023-10-16T10:04:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.391+0000 7f4f2df33640 1 -- 172.21.15.84:0/2256413147 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f4f0c06af90 msgr2=0x7f4f0c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/633652932 2023-10-16T10:04:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.391+0000 7f4f2df33640 1 --2- 172.21.15.84:0/2256413147 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f4f0c06af90 0x7f4f0c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:05.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.396+0000 7f4f1e7fc640 1 -- 172.21.15.84:0/2256413147 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4f20077500 con 0x7f4f28159950 2023-10-16T10:04:05.432 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:05.431+0000 7ff35a63a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:04:05.438 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:05.437+0000 7fd18f213040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:04:05.489 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:05.487+0000 7ff35a63a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:04:05.497 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:05.496+0000 7fd18f213040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:04:05.539 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:05.538+0000 7ff35a63a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:04:05.549 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:05.548+0000 7fd18f213040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:04:05.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.589+0000 7f4f2ef35640 1 -- 172.21.15.84:0/2256413147 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4f280b4620 con 0x7f4f28159950 2023-10-16T10:04:05.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.590+0000 7f4f1e7fc640 1 -- 172.21.15.84:0/2256413147 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4f2007ab20 con 0x7f4f28159950 2023-10-16T10:04:05.592 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T10:04:05.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.592+0000 7f4f2df33640 1 -- 172.21.15.84:0/2256413147 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f4f0c06af90 msgr2=0x7f4f0c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/633652932 2023-10-16T10:04:05.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.592+0000 7f4f2df33640 1 --2- 172.21.15.84:0/2256413147 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f4f0c06af90 0x7f4f0c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:04:05.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.593+0000 7f4f2ef35640 1 -- 172.21.15.84:0/2256413147 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f4f0c06af90 msgr2=0x7f4f0c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:05.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.593+0000 7f4f2ef35640 1 --2- 172.21.15.84:0/2256413147 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f4f0c06af90 0x7f4f0c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.593+0000 7f4f2ef35640 1 -- 172.21.15.84:0/2256413147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f28159950 msgr2=0x7f4f280af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.593+0000 7f4f2ef35640 1 --2- 172.21.15.84:0/2256413147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f28159950 0x7f4f280af1f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f4f20036040 tx=0x7f4f20004a40 comp rx=0 tx=0).stop 2023-10-16T10:04:05.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.593+0000 7f4f2ef35640 1 -- 172.21.15.84:0/2256413147 shutdown_connections 2023-10-16T10:04:05.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.593+0000 7f4f2ef35640 1 --2- 172.21.15.84:0/2256413147 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f4f0c06af90 0x7f4f0c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.593+0000 7f4f2ef35640 1 --2- 172.21.15.84:0/2256413147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f28159950 0x7f4f280af1f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.593+0000 7f4f2ef35640 1 --2- 172.21.15.84:0/2256413147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f280ca0a0 0x7f4f280b2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.593+0000 7f4f2ef35640 1 --2- 172.21.15.84:0/2256413147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f28070b60 0x7f4f28155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.593+0000 7f4f2ef35640 1 -- 172.21.15.84:0/2256413147 >> 172.21.15.84:0/2256413147 conn(0x7f4f280ce6a0 msgr2=0x7f4f2806f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:05.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.593+0000 7f4f2ef35640 1 -- 172.21.15.84:0/2256413147 shutdown_connections 2023-10-16T10:04:05.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.593+0000 7f4f2ef35640 1 -- 172.21.15.84:0/2256413147 wait complete. 2023-10-16T10:04:05.613 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:04:05.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.868+0000 7f3ff5800640 1 Processor -- start 2023-10-16T10:04:05.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.868+0000 7f3ff5800640 1 -- start start 2023-10-16T10:04:05.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.870+0000 7f3ff5800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3ff015d410 0x7f3ff00d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:05.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.870+0000 7f3ff5800640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff00b5d90 con 0x7f3ff015d030 2023-10-16T10:04:05.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.870+0000 7f3ff5800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff0160fc0 con 0x7f3ff00cd260 2023-10-16T10:04:05.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.870+0000 7f3ff5800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff00d2b50 con 0x7f3ff015d410 2023-10-16T10:04:05.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.870+0000 7f3feeffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3ff015d030 0x7f3ff0161450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55630/0 (socket says 172.21.15.84:55630) 2023-10-16T10:04:05.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.870+0000 7f3feeffd640 1 -- 172.21.15.84:0/1486333686 learned_addr learned my addr 172.21.15.84:0/1486333686 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:05.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.870+0000 7f3fee7fc640 1 --2- 172.21.15.84:0/1486333686 >> v2:172.21.15.157:3300/0 conn(0x7f3ff015d410 0x7f3ff00d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:05.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.870+0000 7f3fedffb640 1 -- 172.21.15.84:0/1486333686 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2841397884 0 0) 0x7f3ff00b5d90 con 0x7f3ff015d030 2023-10-16T10:04:05.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.870+0000 7f3fedffb640 1 -- 172.21.15.84:0/1486333686 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fd8003520 con 0x7f3ff015d030 2023-10-16T10:04:05.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.871+0000 7f3fedffb640 1 -- 172.21.15.84:0/1486333686 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1757494331 0 0) 0x7f3fd8003520 con 0x7f3ff015d030 2023-10-16T10:04:05.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.871+0000 7f3fedffb640 1 -- 172.21.15.84:0/1486333686 >> v2:172.21.15.157:3300/0 conn(0x7f3ff015d410 msgr2=0x7f3ff00d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:05.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.871+0000 7f3fedffb640 1 --2- 172.21.15.84:0/1486333686 >> v2:172.21.15.157:3300/0 conn(0x7f3ff015d410 0x7f3ff00d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.871+0000 7f3fedffb640 1 -- 172.21.15.84:0/1486333686 >> v1:172.21.15.157:6789/0 conn(0x7f3ff00cd260 legacy=0x7f3ff00b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.871+0000 7f3fedffb640 1 -- 172.21.15.84:0/1486333686 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff00d25d0 con 0x7f3ff015d030 2023-10-16T10:04:05.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.871+0000 7f3fedffb640 1 -- 172.21.15.84:0/1486333686 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f3fe0001670 con 0x7f3ff015d030 2023-10-16T10:04:05.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.871+0000 7f3fee7fc640 1 --2- 172.21.15.84:0/1486333686 >> v2:172.21.15.157:3300/0 conn(0x7f3ff015d410 0x7f3ff00d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:05.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.871+0000 7f3fedffb640 1 -- 172.21.15.84:0/1486333686 >> v1:172.21.15.84:6789/0 conn(0x7f3ff015d030 legacy=0x7f3ff0161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.871+0000 7f3fedffb640 1 --2- 172.21.15.84:0/1486333686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fd8003970 0x7f3fd8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:05.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.871+0000 7f3fedffb640 1 --2- 172.21.15.84:0/1486333686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fd80047c0 0x7f3fd8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:05.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.871+0000 7f3fedffb640 1 --2- 172.21.15.84:0/1486333686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd8005180 0x7f3fd800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:05.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.871+0000 7f3fedffb640 1 -- 172.21.15.84:0/1486333686 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd800c080 con 0x7f3fd8003970 2023-10-16T10:04:05.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.871+0000 7f3fedffb640 1 -- 172.21.15.84:0/1486333686 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd800c1f0 con 0x7f3fd8005180 2023-10-16T10:04:05.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.871+0000 7f3fedffb640 1 -- 172.21.15.84:0/1486333686 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd800c3b0 con 0x7f3fd80047c0 2023-10-16T10:04:05.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.871+0000 7f3feeffd640 1 --2- 172.21.15.84:0/1486333686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fd8003970 0x7f3fd8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:05.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.871+0000 7f3fee7fc640 1 --2- 172.21.15.84:0/1486333686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fd80047c0 0x7f3fd8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:05.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.872+0000 7f3feeffd640 1 -- 172.21.15.84:0/1486333686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fd80047c0 msgr2=0x7f3fd8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:05.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.872+0000 7f3feeffd640 1 --2- 172.21.15.84:0/1486333686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fd80047c0 0x7f3fd8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.872+0000 7f3feeffd640 1 -- 172.21.15.84:0/1486333686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd8005180 msgr2=0x7f3fd800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:05.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.872+0000 7f3fef7fe640 1 --2- 172.21.15.84:0/1486333686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd8005180 0x7f3fd800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:05.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.872+0000 7f3feeffd640 1 --2- 172.21.15.84:0/1486333686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd8005180 0x7f3fd800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.872+0000 7f3feeffd640 1 -- 172.21.15.84:0/1486333686 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd8003520 con 0x7f3fd8003970 2023-10-16T10:04:05.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.872+0000 7f3fef7fe640 1 --2- 172.21.15.84:0/1486333686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd8005180 0x7f3fd800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:05.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.872+0000 7f3fef7fe640 1 -- 172.21.15.84:0/1486333686 reap_dead start 2023-10-16T10:04:05.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.873+0000 7f3feeffd640 1 --2- 172.21.15.84:0/1486333686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fd8003970 0x7f3fd8003d90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000c190 tx=0x7f3fd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd702893c73256c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:05.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.873+0000 7f3fedffb640 1 -- 172.21.15.84:0/1486333686 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0005ad0 con 0x7f3fd8003970 2023-10-16T10:04:05.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.873+0000 7f3fedffb640 1 -- 172.21.15.84:0/1486333686 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe0005c30 con 0x7f3fd8003970 2023-10-16T10:04:05.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.873+0000 7f3fedffb640 1 -- 172.21.15.84:0/1486333686 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe002cae0 con 0x7f3fd8003970 2023-10-16T10:04:05.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.874+0000 7f3ff5800640 1 -- 172.21.15.84:0/1486333686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fd8003970 msgr2=0x7f3fd8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.874+0000 7f3ff5800640 1 --2- 172.21.15.84:0/1486333686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fd8003970 0x7f3fd8003d90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000c190 tx=0x7f3fd80057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.874+0000 7f3ff5800640 1 -- 172.21.15.84:0/1486333686 shutdown_connections 2023-10-16T10:04:05.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.874+0000 7f3ff5800640 1 --2- 172.21.15.84:0/1486333686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fd8003970 0x7f3fd8003d90 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.874+0000 7f3ff5800640 1 -- 172.21.15.84:0/1486333686 >> 172.21.15.84:0/1486333686 conn(0x7f3ff00cf720 msgr2=0x7f3ff00cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:05.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.874+0000 7f3ff5800640 1 -- 172.21.15.84:0/1486333686 shutdown_connections 2023-10-16T10:04:05.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.874+0000 7f3ff5800640 1 -- 172.21.15.84:0/1486333686 wait complete. 2023-10-16T10:04:05.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.875+0000 7f3ff5800640 1 Processor -- start 2023-10-16T10:04:05.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.876+0000 7f3ff5800640 1 -- start start 2023-10-16T10:04:05.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.876+0000 7f3ff5800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd8003970 0x7f3ff00c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:05.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.876+0000 7f3ff5800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ff00c9130 0x7f3ff014d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:05.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.876+0000 7f3ff5800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ff015d890 0x7f3ff00b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:05.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.877+0000 7f3ff5800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff00c6e40 con 0x7f3ff015d890 2023-10-16T10:04:05.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.877+0000 7f3ff5800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff014f010 con 0x7f3fd8003970 2023-10-16T10:04:05.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.877+0000 7f3ff5800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff014f2f0 con 0x7f3ff00c9130 2023-10-16T10:04:05.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.877+0000 7f3fef7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ff015d890 0x7f3ff00b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:05.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.877+0000 7f3fef7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ff015d890 0x7f3ff00b2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37430/0 (socket says 172.21.15.84:37430) 2023-10-16T10:04:05.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.877+0000 7f3fef7fe640 1 -- 172.21.15.84:0/2800587673 learned_addr learned my addr 172.21.15.84:0/2800587673 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:05.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.877+0000 7f3fee7fc640 1 --2- 172.21.15.84:0/2800587673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ff00c9130 0x7f3ff014d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:05.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.877+0000 7f3feeffd640 1 --2- 172.21.15.84:0/2800587673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd8003970 0x7f3ff00c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:05.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.877+0000 7f3fef7fe640 1 -- 172.21.15.84:0/2800587673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ff00c9130 msgr2=0x7f3ff014d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.877+0000 7f3fef7fe640 1 --2- 172.21.15.84:0/2800587673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ff00c9130 0x7f3ff014d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.877+0000 7f3fef7fe640 1 -- 172.21.15.84:0/2800587673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd8003970 msgr2=0x7f3ff00c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.877+0000 7f3fef7fe640 1 --2- 172.21.15.84:0/2800587673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd8003970 0x7f3ff00c6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.877+0000 7f3fef7fe640 1 -- 172.21.15.84:0/2800587673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff014f5d0 con 0x7f3ff015d890 2023-10-16T10:04:05.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.877+0000 7f3fef7fe640 1 --2- 172.21.15.84:0/2800587673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ff015d890 0x7f3ff00b2510 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0073730 tx=0x7f3fe800d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:05.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.878+0000 7f3fd77fe640 1 -- 172.21.15.84:0/2800587673 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8003d80 con 0x7f3ff015d890 2023-10-16T10:04:05.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.878+0000 7f3fd77fe640 1 -- 172.21.15.84:0/2800587673 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe8005070 con 0x7f3ff015d890 2023-10-16T10:04:05.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.878+0000 7f3fd77fe640 1 -- 172.21.15.84:0/2800587673 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8005420 con 0x7f3ff015d890 2023-10-16T10:04:05.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.878+0000 7f3ff5800640 1 -- 172.21.15.84:0/2800587673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff014f8b0 con 0x7f3ff015d890 2023-10-16T10:04:05.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.878+0000 7f3ff5800640 1 -- 172.21.15.84:0/2800587673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff014d7f0 con 0x7f3ff015d890 2023-10-16T10:04:05.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.879+0000 7f3ff5800640 1 -- 172.21.15.84:0/2800587673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff004e160 con 0x7f3ff015d890 2023-10-16T10:04:05.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.880+0000 7f3fd77fe640 1 -- 172.21.15.84:0/2800587673 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 29) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f3fe8005580 con 0x7f3ff015d890 2023-10-16T10:04:05.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.881+0000 7f3fd77fe640 1 --2- 172.21.15.84:0/2800587673 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f3fdc06af90 0x7f3fdc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:05.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.881+0000 7f3fd77fe640 1 -- 172.21.15.84:0/2800587673 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6111+0+0 (secure 0 0 0) 0x7f3fe8089330 con 0x7f3ff015d890 2023-10-16T10:04:05.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.881+0000 7f3feeffd640 1 -- 172.21.15.84:0/2800587673 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f3fdc06af90 msgr2=0x7f3fdc06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/633652932 2023-10-16T10:04:05.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.881+0000 7f3feeffd640 1 --2- 172.21.15.84:0/2800587673 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f3fdc06af90 0x7f3fdc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:05.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:05.885+0000 7f3fd77fe640 1 -- 172.21.15.84:0/2800587673 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3fe8054020 con 0x7f3ff015d890 2023-10-16T10:04:05.920 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:05.919+0000 7ff35a63a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:04:05.952 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:05.951+0000 7fd18f213040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:04:06.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.077+0000 7f3ff5800640 1 -- 172.21.15.84:0/2800587673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ff00c9aa0 con 0x7f3ff015d890 2023-10-16T10:04:06.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.078+0000 7f3fd77fe640 1 -- 172.21.15.84:0/2800587673 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3fe8005830 con 0x7f3ff015d890 2023-10-16T10:04:06.080 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T10:04:06.081 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:06.080+0000 7ff35a63a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:04:06.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.080+0000 7f3ff5800640 1 -- 172.21.15.84:0/2800587673 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f3fdc06af90 msgr2=0x7f3fdc06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:06.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.080+0000 7f3ff5800640 1 --2- 172.21.15.84:0/2800587673 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f3fdc06af90 0x7f3fdc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.080+0000 7f3ff5800640 1 -- 172.21.15.84:0/2800587673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ff015d890 msgr2=0x7f3ff00b2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.080+0000 7f3ff5800640 1 --2- 172.21.15.84:0/2800587673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ff015d890 0x7f3ff00b2510 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0073730 tx=0x7f3fe800d970 comp rx=0 tx=0).stop 2023-10-16T10:04:06.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.081+0000 7f3ff5800640 1 -- 172.21.15.84:0/2800587673 shutdown_connections 2023-10-16T10:04:06.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.081+0000 7f3ff5800640 1 --2- 172.21.15.84:0/2800587673 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f3fdc06af90 0x7f3fdc06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.081+0000 7f3ff5800640 1 --2- 172.21.15.84:0/2800587673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ff015d890 0x7f3ff00b2510 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.081+0000 7f3ff5800640 1 --2- 172.21.15.84:0/2800587673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ff00c9130 0x7f3ff014d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.081+0000 7f3ff5800640 1 --2- 172.21.15.84:0/2800587673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd8003970 0x7f3ff00c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.081+0000 7f3ff5800640 1 -- 172.21.15.84:0/2800587673 >> 172.21.15.84:0/2800587673 conn(0x7f3ff00cf720 msgr2=0x7f3ff00d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:06.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.081+0000 7f3ff5800640 1 -- 172.21.15.84:0/2800587673 shutdown_connections 2023-10-16T10:04:06.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.081+0000 7f3ff5800640 1 -- 172.21.15.84:0/2800587673 wait complete. 2023-10-16T10:04:06.101 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T10:04:06.119 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:06.118+0000 7fd18f213040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:04:06.196 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:06.194+0000 7ff35a63a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:04:06.235 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:06.234+0000 7fd18f213040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:04:06.246 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:06.244+0000 7ff35a63a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:04:06.287 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:06.286+0000 7fd18f213040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:04:06.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.301+0000 7fd1d4583640 1 Processor -- start 2023-10-16T10:04:06.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.301+0000 7fd1d4583640 1 -- start start 2023-10-16T10:04:06.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.303+0000 7fd1d4583640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd1cc0caac0 0x7fd1cc0ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.303+0000 7fd1d4583640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1cc0cb030 con 0x7fd1cc15ad70 2023-10-16T10:04:06.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.303+0000 7fd1d4583640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1cc0c91c0 con 0x7fd1cc06a4b0 2023-10-16T10:04:06.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.303+0000 7fd1d4583640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1cc0c94a0 con 0x7fd1cc0caac0 2023-10-16T10:04:06.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.303+0000 7fd1d231f640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd1cc15ad70 0x7fd1cc069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55638/0 (socket says 172.21.15.84:55638) 2023-10-16T10:04:06.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.303+0000 7fd1d231f640 1 -- 172.21.15.84:0/538675011 learned_addr learned my addr 172.21.15.84:0/538675011 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:06.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.304+0000 7fd1d2b20640 1 --2- 172.21.15.84:0/538675011 >> v2:172.21.15.157:3300/0 conn(0x7fd1cc0caac0 0x7fd1cc0ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.304+0000 7fd1d131d640 1 -- 172.21.15.84:0/538675011 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489961349 0 0) 0x7fd1cc0cb030 con 0x7fd1cc15ad70 2023-10-16T10:04:06.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.304+0000 7fd1d131d640 1 -- 172.21.15.84:0/538675011 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1b8003520 con 0x7fd1cc15ad70 2023-10-16T10:04:06.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.304+0000 7fd1d131d640 1 -- 172.21.15.84:0/538675011 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2077722951 0 0) 0x7fd1b8003520 con 0x7fd1cc15ad70 2023-10-16T10:04:06.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.304+0000 7fd1d131d640 1 -- 172.21.15.84:0/538675011 >> v2:172.21.15.157:3300/0 conn(0x7fd1cc0caac0 msgr2=0x7fd1cc0ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:06.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.304+0000 7fd1d131d640 1 --2- 172.21.15.84:0/538675011 >> v2:172.21.15.157:3300/0 conn(0x7fd1cc0caac0 0x7fd1cc0ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.304+0000 7fd1d131d640 1 -- 172.21.15.84:0/538675011 >> v1:172.21.15.157:6789/0 conn(0x7fd1cc06a4b0 legacy=0x7fd1cc0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.304+0000 7fd1d131d640 1 -- 172.21.15.84:0/538675011 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc0c9780 con 0x7fd1cc15ad70 2023-10-16T10:04:06.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.304+0000 7fd1d131d640 1 -- 172.21.15.84:0/538675011 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd1c8001670 con 0x7fd1cc15ad70 2023-10-16T10:04:06.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.304+0000 7fd1d131d640 1 -- 172.21.15.84:0/538675011 >> v1:172.21.15.84:6789/0 conn(0x7fd1cc15ad70 legacy=0x7fd1cc069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.304+0000 7fd1d131d640 1 --2- 172.21.15.84:0/538675011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b8003970 0x7fd1b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.304+0000 7fd1d131d640 1 --2- 172.21.15.84:0/538675011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1b80047c0 0x7fd1b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.304+0000 7fd1d131d640 1 --2- 172.21.15.84:0/538675011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1b8005180 0x7fd1b800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.304+0000 7fd1d131d640 1 -- 172.21.15.84:0/538675011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0c94a0 con 0x7fd1b8005180 2023-10-16T10:04:06.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.304+0000 7fd1d131d640 1 -- 172.21.15.84:0/538675011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b800c020 con 0x7fd1b80047c0 2023-10-16T10:04:06.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.304+0000 7fd1d131d640 1 -- 172.21.15.84:0/538675011 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b800c1e0 con 0x7fd1b8003970 2023-10-16T10:04:06.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.305+0000 7fd1d2b20640 1 --2- 172.21.15.84:0/538675011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1b8005180 0x7fd1b800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.305+0000 7fd1d1b1e640 1 --2- 172.21.15.84:0/538675011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1b80047c0 0x7fd1b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.305+0000 7fd1d231f640 1 --2- 172.21.15.84:0/538675011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b8003970 0x7fd1b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.305+0000 7fd1d2b20640 1 -- 172.21.15.84:0/538675011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b8003970 msgr2=0x7fd1b8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:06.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.305+0000 7fd1d2b20640 1 --2- 172.21.15.84:0/538675011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b8003970 0x7fd1b8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.305+0000 7fd1d2b20640 1 -- 172.21.15.84:0/538675011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1b80047c0 msgr2=0x7fd1b8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:06.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.305+0000 7fd1d2b20640 1 --2- 172.21.15.84:0/538675011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1b80047c0 0x7fd1b8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.305+0000 7fd1d2b20640 1 -- 172.21.15.84:0/538675011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b8003520 con 0x7fd1b8005180 2023-10-16T10:04:06.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.306+0000 7fd1d2b20640 1 -- 172.21.15.84:0/538675011 reap_dead start 2023-10-16T10:04:06.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.307+0000 7fd1d2b20640 1 --2- 172.21.15.84:0/538675011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1b8005180 0x7fd1b800b980 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4009fc0 tx=0x7fd1b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdc73d52501358e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:06.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.307+0000 7fd1d131d640 1 -- 172.21.15.84:0/538675011 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c402cc60 con 0x7fd1b8005180 2023-10-16T10:04:06.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.307+0000 7fd1d131d640 1 -- 172.21.15.84:0/538675011 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c400f040 con 0x7fd1b8005180 2023-10-16T10:04:06.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.307+0000 7fd1d131d640 1 -- 172.21.15.84:0/538675011 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4003990 con 0x7fd1b8005180 2023-10-16T10:04:06.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.308+0000 7fd1d4583640 1 -- 172.21.15.84:0/538675011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1b8005180 msgr2=0x7fd1b800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.308+0000 7fd1d4583640 1 --2- 172.21.15.84:0/538675011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1b8005180 0x7fd1b800b980 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4009fc0 tx=0x7fd1b80057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.308+0000 7fd1d4583640 1 -- 172.21.15.84:0/538675011 shutdown_connections 2023-10-16T10:04:06.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.308+0000 7fd1d4583640 1 --2- 172.21.15.84:0/538675011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1b8005180 0x7fd1b800b980 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.308+0000 7fd1d4583640 1 -- 172.21.15.84:0/538675011 >> 172.21.15.84:0/538675011 conn(0x7fd1cc0ce9b0 msgr2=0x7fd1cc0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:06.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.308+0000 7fd1d4583640 1 -- 172.21.15.84:0/538675011 shutdown_connections 2023-10-16T10:04:06.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.308+0000 7fd1d4583640 1 -- 172.21.15.84:0/538675011 wait complete. 2023-10-16T10:04:06.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.309+0000 7fd1d4583640 1 Processor -- start 2023-10-16T10:04:06.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.310+0000 7fd1d4583640 1 -- start start 2023-10-16T10:04:06.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.311+0000 7fd1d4583640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b8005180 0x7fd1cc151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.311+0000 7fd1d4583640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1cc0b2d40 0x7fd1cc0cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.311+0000 7fd1d4583640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1cc0b31c0 0x7fd1cc0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.311+0000 7fd1d4583640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0ce140 con 0x7fd1cc0b31c0 2023-10-16T10:04:06.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.311+0000 7fd1d4583640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0b2260 con 0x7fd1cc0b2d40 2023-10-16T10:04:06.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.311+0000 7fd1d4583640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0b1650 con 0x7fd1b8005180 2023-10-16T10:04:06.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.311+0000 7fd1d2b20640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1cc0b31c0 0x7fd1cc0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.311+0000 7fd1d2b20640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1cc0b31c0 0x7fd1cc0cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37448/0 (socket says 172.21.15.84:37448) 2023-10-16T10:04:06.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.311+0000 7fd1d2b20640 1 -- 172.21.15.84:0/31564889 learned_addr learned my addr 172.21.15.84:0/31564889 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:06.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.311+0000 7fd1d231f640 1 --2- 172.21.15.84:0/31564889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b8005180 0x7fd1cc151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.311+0000 7fd1d1b1e640 1 --2- 172.21.15.84:0/31564889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1cc0b2d40 0x7fd1cc0cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.311+0000 7fd1d2b20640 1 -- 172.21.15.84:0/31564889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b8005180 msgr2=0x7fd1cc151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.311+0000 7fd1d2b20640 1 --2- 172.21.15.84:0/31564889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b8005180 0x7fd1cc151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.311+0000 7fd1d2b20640 1 -- 172.21.15.84:0/31564889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1cc0b2d40 msgr2=0x7fd1cc0cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.311+0000 7fd1d2b20640 1 --2- 172.21.15.84:0/31564889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1cc0b2d40 0x7fd1cc0cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.311+0000 7fd1d2b20640 1 -- 172.21.15.84:0/31564889 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc0b1930 con 0x7fd1cc0b31c0 2023-10-16T10:04:06.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.311+0000 7fd1d1b1e640 1 --2- 172.21.15.84:0/31564889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1cc0b2d40 0x7fd1cc0cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:06.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.312+0000 7fd1d2b20640 1 --2- 172.21.15.84:0/31564889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1cc0b31c0 0x7fd1cc0cdc00 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd1c402c6d0 tx=0x7fd1c4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:06.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.312+0000 7fd1c2ffd640 1 -- 172.21.15.84:0/31564889 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4035e40 con 0x7fd1cc0b31c0 2023-10-16T10:04:06.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.312+0000 7fd1c2ffd640 1 -- 172.21.15.84:0/31564889 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c4003990 con 0x7fd1cc0b31c0 2023-10-16T10:04:06.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.312+0000 7fd1c2ffd640 1 -- 172.21.15.84:0/31564889 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4003d40 con 0x7fd1cc0b31c0 2023-10-16T10:04:06.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.312+0000 7fd1d4583640 1 -- 172.21.15.84:0/31564889 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1cc0b1c10 con 0x7fd1cc0b31c0 2023-10-16T10:04:06.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.312+0000 7fd1d4583640 1 -- 172.21.15.84:0/31564889 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1cc159240 con 0x7fd1cc0b31c0 2023-10-16T10:04:06.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.313+0000 7fd1d4583640 1 -- 172.21.15.84:0/31564889 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1cc04e160 con 0x7fd1cc0b31c0 2023-10-16T10:04:06.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.314+0000 7fd1c2ffd640 1 -- 172.21.15.84:0/31564889 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 29) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fd1c4004030 con 0x7fd1cc0b31c0 2023-10-16T10:04:06.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.315+0000 7fd1c2ffd640 1 --2- 172.21.15.84:0/31564889 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd1b006afc0 0x7fd1b006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.315+0000 7fd1c2ffd640 1 -- 172.21.15.84:0/31564889 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6111+0+0 (secure 0 0 0) 0x7fd1c40ac600 con 0x7fd1cc0b31c0 2023-10-16T10:04:06.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.315+0000 7fd1d231f640 1 -- 172.21.15.84:0/31564889 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd1b006afc0 msgr2=0x7fd1b006d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/633652932 2023-10-16T10:04:06.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.315+0000 7fd1d231f640 1 --2- 172.21.15.84:0/31564889 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd1b006afc0 0x7fd1b006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:06.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.323+0000 7fd1c2ffd640 1 -- 172.21.15.84:0/31564889 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd1c4077420 con 0x7fd1cc0b31c0 2023-10-16T10:04:06.355 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:06.354+0000 7ff35a63a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:04:06.406 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:06.405+0000 7fd18f213040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:04:06.462 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:06.461+0000 7ff35a63a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:04:06.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.516+0000 7fd1d231f640 1 -- 172.21.15.84:0/31564889 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd1b006afc0 msgr2=0x7fd1b006d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/633652932 2023-10-16T10:04:06.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.516+0000 7fd1d231f640 1 --2- 172.21.15.84:0/31564889 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd1b006afc0 0x7fd1b006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:04:06.519 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:06.517+0000 7fd18f213040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:04:06.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.530+0000 7fd1d4583640 1 -- 172.21.15.84:0/31564889 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd1cc069ea0 con 0x7fd1cc0b31c0 2023-10-16T10:04:06.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.530+0000 7fd1c2ffd640 1 -- 172.21.15.84:0/31564889 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v67) v1 ==== 80+0+466 (secure 0 0 0) 0x7fd1c407aa40 con 0x7fd1cc0b31c0 2023-10-16T10:04:06.532 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2816959180 2023-10-17T10:02:27.969914+0000 2023-10-16T10:04:06.533 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/4150953381 2023-10-17T10:02:27.969914+0000 2023-10-16T10:04:06.533 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/555434621 2023-10-17T10:02:27.969914+0000 2023-10-16T10:04:06.533 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3440750653 2023-10-17T10:02:27.969914+0000 2023-10-16T10:04:06.533 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3654971153 2023-10-17T10:02:27.969914+0000 2023-10-16T10:04:06.533 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/621840141 2023-10-17T10:02:27.969914+0000 2023-10-16T10:04:06.534 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6833/585186562 2023-10-17T10:02:27.969914+0000 2023-10-16T10:04:06.534 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6832/585186562 2023-10-17T10:02:27.969914+0000 2023-10-16T10:04:06.535 INFO:teuthology.orchestra.run.smithi084.stderr:listed 8 entries 2023-10-16T10:04:06.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.533+0000 7fd1d4583640 1 -- 172.21.15.84:0/31564889 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd1b006afc0 msgr2=0x7fd1b006d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:06.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.533+0000 7fd1d4583640 1 --2- 172.21.15.84:0/31564889 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd1b006afc0 0x7fd1b006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.533+0000 7fd1d4583640 1 -- 172.21.15.84:0/31564889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1cc0b31c0 msgr2=0x7fd1cc0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.533+0000 7fd1d4583640 1 --2- 172.21.15.84:0/31564889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1cc0b31c0 0x7fd1cc0cdc00 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd1c402c6d0 tx=0x7fd1c4009750 comp rx=0 tx=0).stop 2023-10-16T10:04:06.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.533+0000 7fd1d4583640 1 -- 172.21.15.84:0/31564889 shutdown_connections 2023-10-16T10:04:06.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.533+0000 7fd1d4583640 1 --2- 172.21.15.84:0/31564889 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd1b006afc0 0x7fd1b006d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.533+0000 7fd1d4583640 1 --2- 172.21.15.84:0/31564889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1cc0b31c0 0x7fd1cc0cdc00 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.533+0000 7fd1d4583640 1 --2- 172.21.15.84:0/31564889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1cc0b2d40 0x7fd1cc0cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.533+0000 7fd1d4583640 1 --2- 172.21.15.84:0/31564889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1b8005180 0x7fd1cc151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.533+0000 7fd1d4583640 1 -- 172.21.15.84:0/31564889 >> 172.21.15.84:0/31564889 conn(0x7fd1cc0ce9b0 msgr2=0x7fd1cc15bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:06.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.533+0000 7fd1d4583640 1 -- 172.21.15.84:0/31564889 shutdown_connections 2023-10-16T10:04:06.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:06.533+0000 7fd1d4583640 1 -- 172.21.15.84:0/31564889 wait complete. 2023-10-16T10:04:06.552 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T10:04:06.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:06.553 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T10:04:06.618 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:06.616+0000 7ff35a63a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:04:06.669 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:06.667+0000 7ff35a63a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:04:06.686 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:06.684+0000 7fd18f213040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:04:06.702 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T10:04:06.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:06.703 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T10:04:06.742 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:06.741+0000 7fd18f213040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:04:06.754 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:04:06.755 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:04:06.791 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:06.789+0000 7ff35a63a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:04:06.815 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:04:06.816 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:04:06.816 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:04:06.865 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:06.864+0000 7fd18f213040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:04:06.872 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:04:06.873 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:04:06.873 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:04:06.930 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:04:06.931 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:04:07.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.142+0000 7fef6155a640 1 Processor -- start 2023-10-16T10:04:07.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.142+0000 7fef6155a640 1 -- start start 2023-10-16T10:04:07.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.144+0000 7fef6155a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fef5c15aaa0 0x7fef5c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.144+0000 7fef6155a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef5c0ce390 0x7fef5c070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.144+0000 7fef6155a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef5c15ed60 con 0x7fef5c15ae80 2023-10-16T10:04:07.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.144+0000 7fef6155a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef5c06b810 con 0x7fef5c15aaa0 2023-10-16T10:04:07.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.144+0000 7fef6155a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c0707e0 con 0x7fef5c0ce390 2023-10-16T10:04:07.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.144+0000 7fef5affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fef5c15aaa0 0x7fef5c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.144+0000 7fef5affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fef5c15aaa0 0x7fef5c15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37464/0 (socket says 172.21.15.84:37464) 2023-10-16T10:04:07.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.144+0000 7fef5affd640 1 -- 172.21.15.84:0/835609135 learned_addr learned my addr 172.21.15.84:0/835609135 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:07.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.144+0000 7fef5affd640 1 -- 172.21.15.84:0/835609135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef5c0ce390 msgr2=0x7fef5c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:07.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.144+0000 7fef5affd640 1 --2- 172.21.15.84:0/835609135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef5c0ce390 0x7fef5c070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.144+0000 7fef5affd640 1 -- 172.21.15.84:0/835609135 >> v1:172.21.15.157:6789/0 conn(0x7fef5c15ae80 legacy=0x7fef5c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:07.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.144+0000 7fef5affd640 1 -- 172.21.15.84:0/835609135 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5c15ef90 con 0x7fef5c15aaa0 2023-10-16T10:04:07.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.145+0000 7fef5affd640 1 --2- 172.21.15.84:0/835609135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5c15aaa0 0x7fef5c15e820 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fef5c0cd990 tx=0x7fef4c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae27cee2b699974 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:07.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.146+0000 7fef59ffb640 1 -- 172.21.15.84:0/835609135 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c031640 con 0x7fef5c15aaa0 2023-10-16T10:04:07.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.146+0000 7fef59ffb640 1 -- 172.21.15.84:0/835609135 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4c0317a0 con 0x7fef5c15aaa0 2023-10-16T10:04:07.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.146+0000 7fef59ffb640 1 -- 172.21.15.84:0/835609135 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c031b30 con 0x7fef5c15aaa0 2023-10-16T10:04:07.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.147+0000 7fef6155a640 1 -- 172.21.15.84:0/835609135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5c15aaa0 msgr2=0x7fef5c15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.147+0000 7fef6155a640 1 --2- 172.21.15.84:0/835609135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5c15aaa0 0x7fef5c15e820 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fef5c0cd990 tx=0x7fef4c02b8f0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.147+0000 7fef6155a640 1 -- 172.21.15.84:0/835609135 shutdown_connections 2023-10-16T10:04:07.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.147+0000 7fef6155a640 1 --2- 172.21.15.84:0/835609135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef5c0ce390 0x7fef5c070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.147+0000 7fef6155a640 1 --2- 172.21.15.84:0/835609135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5c15aaa0 0x7fef5c15e820 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.147+0000 7fef6155a640 1 -- 172.21.15.84:0/835609135 >> 172.21.15.84:0/835609135 conn(0x7fef5c0cb410 msgr2=0x7fef5c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:07.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.147+0000 7fef6155a640 1 -- 172.21.15.84:0/835609135 shutdown_connections 2023-10-16T10:04:07.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.147+0000 7fef6155a640 1 -- 172.21.15.84:0/835609135 wait complete. 2023-10-16T10:04:07.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.149+0000 7fef6155a640 1 Processor -- start 2023-10-16T10:04:07.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.150+0000 7fef6155a640 1 -- start start 2023-10-16T10:04:07.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.151+0000 7fef6155a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef5c0ce390 0x7fef5c1581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.151+0000 7fef6155a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef5c15aaa0 0x7fef5c1e45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.151+0000 7fef6155a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5c15ae80 0x7fef5c1e4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.151+0000 7fef6155a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c15c020 con 0x7fef5c15ae80 2023-10-16T10:04:07.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.151+0000 7fef6155a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c15c300 con 0x7fef5c15aaa0 2023-10-16T10:04:07.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.151+0000 7fef6155a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c15c5e0 con 0x7fef5c0ce390 2023-10-16T10:04:07.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.152+0000 7fef5b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5c15ae80 0x7fef5c1e4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.152+0000 7fef5b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5c15ae80 0x7fef5c1e4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37466/0 (socket says 172.21.15.84:37466) 2023-10-16T10:04:07.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.152+0000 7fef5b7fe640 1 -- 172.21.15.84:0/1986814716 learned_addr learned my addr 172.21.15.84:0/1986814716 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:07.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.152+0000 7fef5a7fc640 1 --2- 172.21.15.84:0/1986814716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef5c15aaa0 0x7fef5c1e45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.152+0000 7fef5affd640 1 --2- 172.21.15.84:0/1986814716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef5c0ce390 0x7fef5c1581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.152+0000 7fef5b7fe640 1 -- 172.21.15.84:0/1986814716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef5c0ce390 msgr2=0x7fef5c1581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.152+0000 7fef5b7fe640 1 --2- 172.21.15.84:0/1986814716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef5c0ce390 0x7fef5c1581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.152+0000 7fef5b7fe640 1 -- 172.21.15.84:0/1986814716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef5c15aaa0 msgr2=0x7fef5c1e45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.152+0000 7fef5b7fe640 1 --2- 172.21.15.84:0/1986814716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef5c15aaa0 0x7fef5c1e45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.152+0000 7fef5b7fe640 1 -- 172.21.15.84:0/1986814716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c0094d0 con 0x7fef5c15ae80 2023-10-16T10:04:07.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.152+0000 7fef5b7fe640 1 --2- 172.21.15.84:0/1986814716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5c15ae80 0x7fef5c1e4af0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fef5400b970 tx=0x7fef5400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.153+0000 7fef437fe640 1 -- 172.21.15.84:0/1986814716 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5400cb40 con 0x7fef5c15ae80 2023-10-16T10:04:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.153+0000 7fef437fe640 1 -- 172.21.15.84:0/1986814716 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5400ccc0 con 0x7fef5c15ae80 2023-10-16T10:04:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.153+0000 7fef6155a640 1 -- 172.21.15.84:0/1986814716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef5c15c8c0 con 0x7fef5c15ae80 2023-10-16T10:04:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.153+0000 7fef437fe640 1 -- 172.21.15.84:0/1986814716 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef54014680 con 0x7fef5c15ae80 2023-10-16T10:04:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.153+0000 7fef6155a640 1 -- 172.21.15.84:0/1986814716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef5c15ce80 con 0x7fef5c15ae80 2023-10-16T10:04:07.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.154+0000 7fef6155a640 1 -- 172.21.15.84:0/1986814716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef5c04e160 con 0x7fef5c15ae80 2023-10-16T10:04:07.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.155+0000 7fef437fe640 1 -- 172.21.15.84:0/1986814716 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 29) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fef5401d3b0 con 0x7fef5c15ae80 2023-10-16T10:04:07.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.156+0000 7fef437fe640 1 --2- 172.21.15.84:0/1986814716 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fef4806afc0 0x7fef4806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.156+0000 7fef437fe640 1 -- 172.21.15.84:0/1986814716 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6111+0+0 (secure 0 0 0) 0x7fef540898a0 con 0x7fef5c15ae80 2023-10-16T10:04:07.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.156+0000 7fef5affd640 1 -- 172.21.15.84:0/1986814716 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fef4806afc0 msgr2=0x7fef4806d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/633652932 2023-10-16T10:04:07.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.156+0000 7fef5affd640 1 --2- 172.21.15.84:0/1986814716 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fef4806afc0 0x7fef4806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:07.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.160+0000 7fef437fe640 1 -- 172.21.15.84:0/1986814716 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fef540546a0 con 0x7fef5c15ae80 2023-10-16T10:04:07.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.357+0000 7fef5affd640 1 -- 172.21.15.84:0/1986814716 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fef4806afc0 msgr2=0x7fef4806d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/633652932 2023-10-16T10:04:07.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.357+0000 7fef5affd640 1 --2- 172.21.15.84:0/1986814716 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fef4806afc0 0x7fef4806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:04:07.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.388+0000 7fef6155a640 1 -- 172.21.15.84:0/1986814716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef5c0c8a00 con 0x7fef5c15ae80 2023-10-16T10:04:07.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.389+0000 7fef437fe640 1 -- 172.21.15.84:0/1986814716 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fef54019090 con 0x7fef5c15ae80 2023-10-16T10:04:07.391 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:07.391 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3040499154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3040499154},{"type":"v1","addr":"172.21.15.157:6835","nonce":3040499154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2727537766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2727537766},{"type":"v1","addr":"172.21.15.157:6833","nonce":2727537766}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/49292950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":49292950},{"type":"v1","addr":"172.21.15.84:6836","nonce":49292950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:02:13.514054+0000","modified":"2023-10-16T10:02:16.598055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/3839442164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3839442164},{"type":"v1","addr":"172.21.15.84:6837","nonce":3839442164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T10:04:07.392 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 25 2023-10-16T10:04:07.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.392+0000 7fef6155a640 1 -- 172.21.15.84:0/1986814716 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fef4806afc0 msgr2=0x7fef4806d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:07.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.392+0000 7fef6155a640 1 --2- 172.21.15.84:0/1986814716 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fef4806afc0 0x7fef4806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.392+0000 7fef6155a640 1 -- 172.21.15.84:0/1986814716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5c15ae80 msgr2=0x7fef5c1e4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.392+0000 7fef6155a640 1 --2- 172.21.15.84:0/1986814716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5c15ae80 0x7fef5c1e4af0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fef5400b970 tx=0x7fef5400be30 comp rx=0 tx=0).stop 2023-10-16T10:04:07.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.392+0000 7fef6155a640 1 -- 172.21.15.84:0/1986814716 shutdown_connections 2023-10-16T10:04:07.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.392+0000 7fef6155a640 1 --2- 172.21.15.84:0/1986814716 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fef4806afc0 0x7fef4806d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.392+0000 7fef6155a640 1 --2- 172.21.15.84:0/1986814716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5c15ae80 0x7fef5c1e4af0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.392+0000 7fef6155a640 1 --2- 172.21.15.84:0/1986814716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef5c15aaa0 0x7fef5c1e45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.392+0000 7fef6155a640 1 --2- 172.21.15.84:0/1986814716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef5c0ce390 0x7fef5c1581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.392+0000 7fef6155a640 1 -- 172.21.15.84:0/1986814716 >> 172.21.15.84:0/1986814716 conn(0x7fef5c0cb410 msgr2=0x7fef5c0b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:07.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.392+0000 7fef6155a640 1 -- 172.21.15.84:0/1986814716 shutdown_connections 2023-10-16T10:04:07.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.392+0000 7fef6155a640 1 -- 172.21.15.84:0/1986814716 wait complete. 2023-10-16T10:04:07.412 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:04:07.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.596+0000 7fd58eca2640 1 Processor -- start 2023-10-16T10:04:07.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.596+0000 7fd58eca2640 1 -- start start 2023-10-16T10:04:07.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.597+0000 7fd58eca2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd588070a60 0x7fd5880aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.598+0000 7fd58eca2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd58806e8d0 con 0x7fd5880705d0 2023-10-16T10:04:07.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.598+0000 7fd58eca2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd58806d830 con 0x7fd588159d10 2023-10-16T10:04:07.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.598+0000 7fd58eca2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd58806e4b0 con 0x7fd588070a60 2023-10-16T10:04:07.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.598+0000 7fd587fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd5880705d0 0x7fd5880ae5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55648/0 (socket says 172.21.15.84:55648) 2023-10-16T10:04:07.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.598+0000 7fd587fff640 1 -- 172.21.15.84:0/1968081756 learned_addr learned my addr 172.21.15.84:0/1968081756 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:07.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.598+0000 7fd58d23f640 1 --2- 172.21.15.84:0/1968081756 >> v2:172.21.15.84:3300/0 conn(0x7fd588070a60 0x7fd5880aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.598+0000 7fd58d23f640 1 -- 172.21.15.84:0/1968081756 >> v1:172.21.15.157:6789/0 conn(0x7fd588159d10 legacy=0x7fd5880ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.598+0000 7fd58d23f640 1 -- 172.21.15.84:0/1968081756 >> v1:172.21.15.84:6789/0 conn(0x7fd5880705d0 legacy=0x7fd5880ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.598+0000 7fd58d23f640 1 -- 172.21.15.84:0/1968081756 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58806db10 con 0x7fd588070a60 2023-10-16T10:04:07.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.600+0000 7fd58d23f640 1 --2- 172.21.15.84:0/1968081756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd588070a60 0x7fd5880aecf0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd580009d90 tx=0x7fd58002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84033e4bd1d37aa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:07.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.600+0000 7fd5877fe640 1 -- 172.21.15.84:0/1968081756 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58002ee70 con 0x7fd588070a60 2023-10-16T10:04:07.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.600+0000 7fd5877fe640 1 -- 172.21.15.84:0/1968081756 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd580036440 con 0x7fd588070a60 2023-10-16T10:04:07.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.600+0000 7fd5877fe640 1 -- 172.21.15.84:0/1968081756 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5800367d0 con 0x7fd588070a60 2023-10-16T10:04:07.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.600+0000 7fd58eca2640 1 -- 172.21.15.84:0/1968081756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd588070a60 msgr2=0x7fd5880aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.600+0000 7fd58eca2640 1 --2- 172.21.15.84:0/1968081756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd588070a60 0x7fd5880aecf0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd580009d90 tx=0x7fd58002e0c0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.600+0000 7fd58eca2640 1 -- 172.21.15.84:0/1968081756 shutdown_connections 2023-10-16T10:04:07.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.600+0000 7fd58eca2640 1 --2- 172.21.15.84:0/1968081756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd588070a60 0x7fd5880aecf0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.601+0000 7fd58eca2640 1 -- 172.21.15.84:0/1968081756 >> 172.21.15.84:0/1968081756 conn(0x7fd5880b1560 msgr2=0x7fd5880b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:07.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.601+0000 7fd58eca2640 1 -- 172.21.15.84:0/1968081756 shutdown_connections 2023-10-16T10:04:07.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.601+0000 7fd58eca2640 1 -- 172.21.15.84:0/1968081756 wait complete. 2023-10-16T10:04:07.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.602+0000 7fd58eca2640 1 Processor -- start 2023-10-16T10:04:07.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.602+0000 7fd58eca2640 1 -- start start 2023-10-16T10:04:07.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.603+0000 7fd58eca2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5880705d0 0x7fd5880c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.603+0000 7fd58eca2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd588070a60 0x7fd5880c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.603+0000 7fd58eca2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd588159d10 0x7fd5881536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.603+0000 7fd58eca2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd588156b00 con 0x7fd588159d10 2023-10-16T10:04:07.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.603+0000 7fd58eca2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5880c8a90 con 0x7fd5880705d0 2023-10-16T10:04:07.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.603+0000 7fd58eca2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5880c8d10 con 0x7fd588070a60 2023-10-16T10:04:07.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.604+0000 7fd58d23f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd588159d10 0x7fd5881536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.604+0000 7fd58d23f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd588159d10 0x7fd5881536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37478/0 (socket says 172.21.15.84:37478) 2023-10-16T10:04:07.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.604+0000 7fd58d23f640 1 -- 172.21.15.84:0/1405590005 learned_addr learned my addr 172.21.15.84:0/1405590005 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:07.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.604+0000 7fd58d23f640 1 -- 172.21.15.84:0/1405590005 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd588070a60 msgr2=0x7fd5880c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.604+0000 7fd58d23f640 1 --2- 172.21.15.84:0/1405590005 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd588070a60 0x7fd5880c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.604+0000 7fd58d23f640 1 -- 172.21.15.84:0/1405590005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5880705d0 msgr2=0x7fd5880c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.604+0000 7fd58ca3e640 1 --2- 172.21.15.84:0/1405590005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5880705d0 0x7fd5880c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.604+0000 7fd58d23f640 1 --2- 172.21.15.84:0/1405590005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5880705d0 0x7fd5880c8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.604+0000 7fd58d23f640 1 -- 172.21.15.84:0/1405590005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd580009a70 con 0x7fd588159d10 2023-10-16T10:04:07.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.604+0000 7fd58d23f640 1 --2- 172.21.15.84:0/1405590005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd588159d10 0x7fd5881536f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd580009d90 tx=0x7fd580004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:07.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.605+0000 7fd5857fa640 1 -- 172.21.15.84:0/1405590005 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58003f850 con 0x7fd588159d10 2023-10-16T10:04:07.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.605+0000 7fd5857fa640 1 -- 172.21.15.84:0/1405590005 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58003f9d0 con 0x7fd588159d10 2023-10-16T10:04:07.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.605+0000 7fd5857fa640 1 -- 172.21.15.84:0/1405590005 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58003fd80 con 0x7fd588159d10 2023-10-16T10:04:07.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.605+0000 7fd58eca2640 1 -- 172.21.15.84:0/1405590005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5880c8ff0 con 0x7fd588159d10 2023-10-16T10:04:07.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.605+0000 7fd58eca2640 1 -- 172.21.15.84:0/1405590005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5880c4160 con 0x7fd588159d10 2023-10-16T10:04:07.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.606+0000 7fd58eca2640 1 -- 172.21.15.84:0/1405590005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5880c94b0 con 0x7fd588159d10 2023-10-16T10:04:07.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.607+0000 7fd5857fa640 1 -- 172.21.15.84:0/1405590005 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 29) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fd58003d070 con 0x7fd588159d10 2023-10-16T10:04:07.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.608+0000 7fd5857fa640 1 --2- 172.21.15.84:0/1405590005 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd56c06af90 0x7fd56c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.608+0000 7fd58ca3e640 1 -- 172.21.15.84:0/1405590005 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd56c06af90 msgr2=0x7fd56c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/633652932 2023-10-16T10:04:07.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.608+0000 7fd5857fa640 1 -- 172.21.15.84:0/1405590005 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6111+0+0 (secure 0 0 0) 0x7fd5800ac340 con 0x7fd588159d10 2023-10-16T10:04:07.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.608+0000 7fd58ca3e640 1 --2- 172.21.15.84:0/1405590005 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd56c06af90 0x7fd56c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:07.617 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:07.613+0000 7ff35a63a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:04:07.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.612+0000 7fd5857fa640 1 -- 172.21.15.84:0/1405590005 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd5800770b0 con 0x7fd588159d10 2023-10-16T10:04:07.712 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:07.710+0000 7fd18f213040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:04:07.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.809+0000 7fd58ca3e640 1 -- 172.21.15.84:0/1405590005 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd56c06af90 msgr2=0x7fd56c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/633652932 2023-10-16T10:04:07.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.809+0000 7fd58ca3e640 1 --2- 172.21.15.84:0/1405590005 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd56c06af90 0x7fd56c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:04:07.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.821+0000 7fd58eca2640 1 -- 172.21.15.84:0/1405590005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5880714d0 con 0x7fd588159d10 2023-10-16T10:04:07.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.822+0000 7fd5857fa640 1 -- 172.21.15.84:0/1405590005 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+17138 (secure 0 0 0) 0x7fd58007a6d0 con 0x7fd588159d10 2023-10-16T10:04:07.824 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:07.824 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":67,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:03:29.904431+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:02:11.266402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-16T10:02:12.562100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.13","mappings":[{"from":0,"to":5}]},{"pgid":"7.2d","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2816959180":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/4150953381":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/555434621":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/3440750653":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/3654971153":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/621840141":"2023-10-17T10:02:27.969914+0000","172.21.15.84:6833/585186562":"2023-10-17T10:02:27.969914+0000","172.21.15.84:6832/585186562":"2023-10-17T10:02:27.969914+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:04:07.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.825+0000 7fd58eca2640 1 -- 172.21.15.84:0/1405590005 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd56c06af90 msgr2=0x7fd56c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:07.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.825+0000 7fd58eca2640 1 --2- 172.21.15.84:0/1405590005 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd56c06af90 0x7fd56c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.825+0000 7fd58eca2640 1 -- 172.21.15.84:0/1405590005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd588159d10 msgr2=0x7fd5881536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.825+0000 7fd58eca2640 1 --2- 172.21.15.84:0/1405590005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd588159d10 0x7fd5881536f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd580009d90 tx=0x7fd580004a40 comp rx=0 tx=0).stop 2023-10-16T10:04:07.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.825+0000 7fd58eca2640 1 -- 172.21.15.84:0/1405590005 shutdown_connections 2023-10-16T10:04:07.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.825+0000 7fd58eca2640 1 --2- 172.21.15.84:0/1405590005 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7fd56c06af90 0x7fd56c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.825+0000 7fd58eca2640 1 --2- 172.21.15.84:0/1405590005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd588159d10 0x7fd5881536f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.825+0000 7fd58eca2640 1 --2- 172.21.15.84:0/1405590005 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd588070a60 0x7fd5880c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.825+0000 7fd58eca2640 1 --2- 172.21.15.84:0/1405590005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5880705d0 0x7fd5880c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.825+0000 7fd58eca2640 1 -- 172.21.15.84:0/1405590005 >> 172.21.15.84:0/1405590005 conn(0x7fd5880b1560 msgr2=0x7fd58806b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:07.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.825+0000 7fd58eca2640 1 -- 172.21.15.84:0/1405590005 shutdown_connections 2023-10-16T10:04:07.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:07.825+0000 7fd58eca2640 1 -- 172.21.15.84:0/1405590005 wait complete. 2023-10-16T10:04:07.844 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:02:11.266402+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:02:12.562100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:04:07.845 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:04:07.913 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:07.912+0000 7ff35a63a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:04:07.983 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:07.981+0000 7ff35a63a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:04:08.033 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:08.031+0000 7fd18f213040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:04:08.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.055+0000 7f2ce1c3f640 1 Processor -- start 2023-10-16T10:04:08.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.055+0000 7f2ce1c3f640 1 -- start start 2023-10-16T10:04:08.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.056+0000 7f2ce1c3f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2cdc0cf710 0x7f2cdc0b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.056+0000 7f2ce1c3f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2cdc0cfaf0 0x7f2cdc0b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.056+0000 7f2ce1c3f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cdc0d3980 con 0x7f2cdc0b38e0 2023-10-16T10:04:08.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.056+0000 7f2ce1c3f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cdc0d3c60 con 0x7f2cdc0cfaf0 2023-10-16T10:04:08.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.056+0000 7f2ce1c3f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cdc0ce630 con 0x7f2cdc0cf710 2023-10-16T10:04:08.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.057+0000 7f2cd3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2cdc0cfaf0 0x7f2cdc0b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.057+0000 7f2cd3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2cdc0cfaf0 0x7f2cdc0b3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37482/0 (socket says 172.21.15.84:37482) 2023-10-16T10:04:08.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.057+0000 7f2cd3fff640 1 -- 172.21.15.84:0/3447674609 learned_addr learned my addr 172.21.15.84:0/3447674609 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:08.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.057+0000 7f2cdbfff640 1 --1- 172.21.15.84:0/3447674609 >> v1:172.21.15.84:6789/0 conn(0x7f2cdc0b38e0 0x7f2cdc0d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55664/0 (socket says 172.21.15.84:55664) 2023-10-16T10:04:08.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.057+0000 7f2cdb7fe640 1 --2- 172.21.15.84:0/3447674609 >> v2:172.21.15.157:3300/0 conn(0x7f2cdc0cf710 0x7f2cdc0b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.057+0000 7f2cd3fff640 1 -- 172.21.15.84:0/3447674609 >> v2:172.21.15.157:3300/0 conn(0x7f2cdc0cf710 msgr2=0x7f2cdc0b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.057+0000 7f2cd3fff640 1 --2- 172.21.15.84:0/3447674609 >> v2:172.21.15.157:3300/0 conn(0x7f2cdc0cf710 0x7f2cdc0b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.057+0000 7f2cd3fff640 1 -- 172.21.15.84:0/3447674609 >> v1:172.21.15.84:6789/0 conn(0x7f2cdc0b38e0 legacy=0x7f2cdc0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.057+0000 7f2cd3fff640 1 -- 172.21.15.84:0/3447674609 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc0ce910 con 0x7f2cdc0cfaf0 2023-10-16T10:04:08.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.059+0000 7f2cd3fff640 1 --2- 172.21.15.84:0/3447674609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2cdc0cfaf0 0x7f2cdc0b3250 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0d2c20 tx=0x7f2cc402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdafbfe0c4017065 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:08.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.059+0000 7f2cdaffd640 1 -- 172.21.15.84:0/3447674609 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4031870 con 0x7f2cdc0cfaf0 2023-10-16T10:04:08.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.059+0000 7f2cdaffd640 1 -- 172.21.15.84:0/3447674609 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc40319d0 con 0x7f2cdc0cfaf0 2023-10-16T10:04:08.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.059+0000 7f2cdaffd640 1 -- 172.21.15.84:0/3447674609 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4031d60 con 0x7f2cdc0cfaf0 2023-10-16T10:04:08.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.060+0000 7f2ce1c3f640 1 -- 172.21.15.84:0/3447674609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2cdc0cfaf0 msgr2=0x7f2cdc0b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.060+0000 7f2ce1c3f640 1 --2- 172.21.15.84:0/3447674609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2cdc0cfaf0 0x7f2cdc0b3250 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0d2c20 tx=0x7f2cc402b910 comp rx=0 tx=0).stop 2023-10-16T10:04:08.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.060+0000 7f2ce1c3f640 1 -- 172.21.15.84:0/3447674609 shutdown_connections 2023-10-16T10:04:08.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.060+0000 7f2ce1c3f640 1 --2- 172.21.15.84:0/3447674609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2cdc0cfaf0 0x7f2cdc0b3250 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.060+0000 7f2ce1c3f640 1 --2- 172.21.15.84:0/3447674609 >> v2:172.21.15.157:3300/0 conn(0x7f2cdc0cf710 0x7f2cdc0b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.060+0000 7f2ce1c3f640 1 -- 172.21.15.84:0/3447674609 >> 172.21.15.84:0/3447674609 conn(0x7f2cdc0d2cc0 msgr2=0x7f2cdc0d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:08.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.060+0000 7f2ce1c3f640 1 -- 172.21.15.84:0/3447674609 shutdown_connections 2023-10-16T10:04:08.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.060+0000 7f2ce1c3f640 1 -- 172.21.15.84:0/3447674609 wait complete. 2023-10-16T10:04:08.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.062+0000 7f2ce1c3f640 1 Processor -- start 2023-10-16T10:04:08.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.062+0000 7f2ce1c3f640 1 -- start start 2023-10-16T10:04:08.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.063+0000 7f2ce1c3f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cdc0b38e0 0x7f2cdc1f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.063+0000 7f2ce1c3f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2cdc0cf710 0x7f2cdc155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.063+0000 7f2ce1c3f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cdc0cfaf0 0x7f2cdc0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.063+0000 7f2ce1c3f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc0afc40 con 0x7f2cdc0cf710 2023-10-16T10:04:08.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.063+0000 7f2ce1c3f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc1f6a80 con 0x7f2cdc0b38e0 2023-10-16T10:04:08.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.063+0000 7f2ce1c3f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc0c6e40 con 0x7f2cdc0cfaf0 2023-10-16T10:04:08.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.064+0000 7f2cd3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2cdc0cf710 0x7f2cdc155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.064+0000 7f2cd3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2cdc0cf710 0x7f2cdc155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37484/0 (socket says 172.21.15.84:37484) 2023-10-16T10:04:08.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.064+0000 7f2cd3fff640 1 -- 172.21.15.84:0/3152168733 learned_addr learned my addr 172.21.15.84:0/3152168733 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:08.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.064+0000 7f2cdb7fe640 1 --2- 172.21.15.84:0/3152168733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cdc0b38e0 0x7f2cdc1f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.064+0000 7f2cd3fff640 1 -- 172.21.15.84:0/3152168733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cdc0cfaf0 msgr2=0x7f2cdc0b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.064+0000 7f2cd3fff640 1 --2- 172.21.15.84:0/3152168733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cdc0cfaf0 0x7f2cdc0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.064+0000 7f2cd3fff640 1 -- 172.21.15.84:0/3152168733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cdc0b38e0 msgr2=0x7f2cdc1f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.064+0000 7f2cd3fff640 1 --2- 172.21.15.84:0/3152168733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cdc0b38e0 0x7f2cdc1f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.064+0000 7f2cd3fff640 1 -- 172.21.15.84:0/3152168733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc40094d0 con 0x7f2cdc0cf710 2023-10-16T10:04:08.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.064+0000 7f2cd3fff640 1 --2- 172.21.15.84:0/3152168733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2cdc0cf710 0x7f2cdc155f60 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2cc402b950 tx=0x7f2cc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:08.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.065+0000 7f2cd8ff9640 1 -- 172.21.15.84:0/3152168733 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc403ad30 con 0x7f2cdc0cf710 2023-10-16T10:04:08.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.065+0000 7f2cd8ff9640 1 -- 172.21.15.84:0/3152168733 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc40443f0 con 0x7f2cdc0cf710 2023-10-16T10:04:08.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.065+0000 7f2cd8ff9640 1 -- 172.21.15.84:0/3152168733 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40447a0 con 0x7f2cdc0cf710 2023-10-16T10:04:08.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.065+0000 7f2ce1c3f640 1 -- 172.21.15.84:0/3152168733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cdc0b15b0 con 0x7f2cdc0cf710 2023-10-16T10:04:08.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.065+0000 7f2ce1c3f640 1 -- 172.21.15.84:0/3152168733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cdc0b1b70 con 0x7f2cdc0cf710 2023-10-16T10:04:08.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.066+0000 7f2ce1c3f640 1 -- 172.21.15.84:0/3152168733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cdc0b1d60 con 0x7f2cdc0cf710 2023-10-16T10:04:08.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.067+0000 7f2cd8ff9640 1 -- 172.21.15.84:0/3152168733 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 29) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f2cc404d450 con 0x7f2cdc0cf710 2023-10-16T10:04:08.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.068+0000 7f2cd8ff9640 1 --2- 172.21.15.84:0/3152168733 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f2cbc06b010 0x7f2cbc06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.068+0000 7f2cd8ff9640 1 -- 172.21.15.84:0/3152168733 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6167+0+0 (secure 0 0 0) 0x7f2cc402d040 con 0x7f2cdc0cf710 2023-10-16T10:04:08.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.068+0000 7f2cdb7fe640 1 -- 172.21.15.84:0/3152168733 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f2cbc06b010 msgr2=0x7f2cbc06d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/633652932 2023-10-16T10:04:08.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.068+0000 7f2cdb7fe640 1 --2- 172.21.15.84:0/3152168733 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f2cbc06b010 0x7f2cbc06d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:08.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.072+0000 7f2cd8ff9640 1 -- 172.21.15.84:0/3152168733 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2cc4076c60 con 0x7f2cdc0cf710 2023-10-16T10:04:08.094 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:08.093+0000 7fd18f213040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:04:08.100 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:08.099+0000 7ff35a63a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:04:08.227 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:08.226+0000 7fd18f213040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:04:08.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.236+0000 7f2cd8ff9640 1 -- 172.21.15.84:0/3152168733 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 30) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f2cc4077030 con 0x7f2cdc0cf710 2023-10-16T10:04:08.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.237+0000 7f2cd8ff9640 1 -- 172.21.15.84:0/3152168733 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f2cbc06b010 msgr2=0x7f2cbc06d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:08.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.237+0000 7f2cd8ff9640 1 --2- 172.21.15.84:0/3152168733 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f2cbc06b010 0x7f2cbc06d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.290+0000 7f2ce1c3f640 1 -- 172.21.15.84:0/3152168733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2cdc04e160 con 0x7f2cdc0cf710 2023-10-16T10:04:08.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.291+0000 7f2cd8ff9640 1 -- 172.21.15.84:0/3152168733 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2cc4090150 con 0x7f2cdc0cf710 2023-10-16T10:04:08.293 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:04:08.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.294+0000 7f2ce1c3f640 1 -- 172.21.15.84:0/3152168733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2cdc0cf710 msgr2=0x7f2cdc155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.294+0000 7f2ce1c3f640 1 --2- 172.21.15.84:0/3152168733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2cdc0cf710 0x7f2cdc155f60 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2cc402b950 tx=0x7f2cc40046c0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.294+0000 7f2ce1c3f640 1 -- 172.21.15.84:0/3152168733 shutdown_connections 2023-10-16T10:04:08.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.294+0000 7f2ce1c3f640 1 --2- 172.21.15.84:0/3152168733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cdc0cfaf0 0x7f2cdc0b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.294+0000 7f2ce1c3f640 1 --2- 172.21.15.84:0/3152168733 >> [v2:172.21.15.84:6832/633652932,v1:172.21.15.84:6833/633652932] conn(0x7f2cbc06b010 0x7f2cbc06d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.294+0000 7f2ce1c3f640 1 --2- 172.21.15.84:0/3152168733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2cdc0cf710 0x7f2cdc155f60 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.294+0000 7f2ce1c3f640 1 --2- 172.21.15.84:0/3152168733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cdc0b38e0 0x7f2cdc1f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.294+0000 7f2ce1c3f640 1 -- 172.21.15.84:0/3152168733 >> 172.21.15.84:0/3152168733 conn(0x7f2cdc0d2cc0 msgr2=0x7f2cdc0d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:08.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.294+0000 7f2ce1c3f640 1 -- 172.21.15.84:0/3152168733 shutdown_connections 2023-10-16T10:04:08.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.294+0000 7f2ce1c3f640 1 -- 172.21.15.84:0/3152168733 wait complete. 2023-10-16T10:04:08.303 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:08.302+0000 7fd111242640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:08.304 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:08.302+0000 7fd111242640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:08.304 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:08.302+0000 7fd111242640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:08.304 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:08.302+0000 7fd111242640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:08.304 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:08.302+0000 7fd111242640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:08.305 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:08.302+0000 7fd112a45640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:08.305 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:08.302+0000 7fd112a45640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:08.305 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:08.302+0000 7fd112a45640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:08.305 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:08.302+0000 7fd112a45640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:08.305 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:08.302+0000 7fd112a45640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:08.315 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:04:08.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.491+0000 7f0892389640 1 Processor -- start 2023-10-16T10:04:08.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.491+0000 7f0892389640 1 -- start start 2023-10-16T10:04:08.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.493+0000 7f0892389640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f088c06b8c0 0x7f088c06bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.493+0000 7f0892389640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f088c15d130 0x7f088c0cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.493+0000 7f0892389640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088c0cd390 con 0x7f088c06b1a0 2023-10-16T10:04:08.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.493+0000 7f0892389640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088c06c1e0 con 0x7f088c15d130 2023-10-16T10:04:08.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.493+0000 7f0892389640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088c0cc2f0 con 0x7f088c06b8c0 2023-10-16T10:04:08.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.493+0000 7f088b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f088c15d130 0x7f088c0cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.493+0000 7f088b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f088c15d130 0x7f088c0cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37606/0 (socket says 172.21.15.84:37606) 2023-10-16T10:04:08.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.493+0000 7f088b7fe640 1 -- 172.21.15.84:0/1340513426 learned_addr learned my addr 172.21.15.84:0/1340513426 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:08.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.493+0000 7f088bfff640 1 --2- 172.21.15.84:0/1340513426 >> v2:172.21.15.157:3300/0 conn(0x7f088c06b8c0 0x7f088c06bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.493+0000 7f088b7fe640 1 -- 172.21.15.84:0/1340513426 >> v2:172.21.15.157:3300/0 conn(0x7f088c06b8c0 msgr2=0x7f088c06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.494+0000 7f088b7fe640 1 --2- 172.21.15.84:0/1340513426 >> v2:172.21.15.157:3300/0 conn(0x7f088c06b8c0 0x7f088c06bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.494+0000 7f088b7fe640 1 -- 172.21.15.84:0/1340513426 >> v1:172.21.15.157:6789/0 conn(0x7f088c06b1a0 legacy=0x7f088c0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.494+0000 7f088b7fe640 1 -- 172.21.15.84:0/1340513426 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c0cc5d0 con 0x7f088c15d130 2023-10-16T10:04:08.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.495+0000 7f088b7fe640 1 --2- 172.21.15.84:0/1340513426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f088c15d130 0x7f088c0cbaa0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f088c06ad90 tx=0x7f088002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2a5c657d8a7895 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:08.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.495+0000 7f088affd640 1 -- 172.21.15.84:0/1340513426 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880031870 con 0x7f088c15d130 2023-10-16T10:04:08.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.495+0000 7f088affd640 1 -- 172.21.15.84:0/1340513426 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08800319d0 con 0x7f088c15d130 2023-10-16T10:04:08.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.495+0000 7f088affd640 1 -- 172.21.15.84:0/1340513426 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880031d60 con 0x7f088c15d130 2023-10-16T10:04:08.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.496+0000 7f0892389640 1 -- 172.21.15.84:0/1340513426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f088c15d130 msgr2=0x7f088c0cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.496+0000 7f0892389640 1 --2- 172.21.15.84:0/1340513426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f088c15d130 0x7f088c0cbaa0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f088c06ad90 tx=0x7f088002b910 comp rx=0 tx=0).stop 2023-10-16T10:04:08.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.496+0000 7f0892389640 1 -- 172.21.15.84:0/1340513426 shutdown_connections 2023-10-16T10:04:08.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.496+0000 7f0892389640 1 --2- 172.21.15.84:0/1340513426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f088c15d130 0x7f088c0cbaa0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.496+0000 7f0892389640 1 --2- 172.21.15.84:0/1340513426 >> v2:172.21.15.157:3300/0 conn(0x7f088c06b8c0 0x7f088c06bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.496+0000 7f0892389640 1 -- 172.21.15.84:0/1340513426 >> 172.21.15.84:0/1340513426 conn(0x7f088c0d0ad0 msgr2=0x7f088c0d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:08.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.496+0000 7f0892389640 1 -- 172.21.15.84:0/1340513426 shutdown_connections 2023-10-16T10:04:08.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.496+0000 7f0892389640 1 -- 172.21.15.84:0/1340513426 wait complete. 2023-10-16T10:04:08.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.498+0000 7f0892389640 1 Processor -- start 2023-10-16T10:04:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.498+0000 7f0892389640 1 -- start start 2023-10-16T10:04:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.499+0000 7f0892389640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088c06b1a0 0x7f088c158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.499+0000 7f0892389640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f088c06b8c0 0x7f088c15f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.499+0000 7f0892389640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f088c15d130 0x7f088c156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.499+0000 7f0892389640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c1589b0 con 0x7f088c06b8c0 2023-10-16T10:04:08.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.499+0000 7f0892389640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c157da0 con 0x7f088c06b1a0 2023-10-16T10:04:08.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.499+0000 7f0892389640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088c158080 con 0x7f088c15d130 2023-10-16T10:04:08.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.500+0000 7f088b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f088c06b8c0 0x7f088c15f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.500+0000 7f088b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f088c06b8c0 0x7f088c15f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37608/0 (socket says 172.21.15.84:37608) 2023-10-16T10:04:08.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.500+0000 7f088b7fe640 1 -- 172.21.15.84:0/2966686858 learned_addr learned my addr 172.21.15.84:0/2966686858 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:08.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.500+0000 7f088b7fe640 1 -- 172.21.15.84:0/2966686858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f088c15d130 msgr2=0x7f088c156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.500+0000 7f088b7fe640 1 --2- 172.21.15.84:0/2966686858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f088c15d130 0x7f088c156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.500+0000 7f088b7fe640 1 -- 172.21.15.84:0/2966686858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088c06b1a0 msgr2=0x7f088c158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.500+0000 7f088bfff640 1 --2- 172.21.15.84:0/2966686858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088c06b1a0 0x7f088c158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.500+0000 7f088b7fe640 1 --2- 172.21.15.84:0/2966686858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088c06b1a0 0x7f088c158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.500+0000 7f088b7fe640 1 -- 172.21.15.84:0/2966686858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08800094d0 con 0x7f088c06b8c0 2023-10-16T10:04:08.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.501+0000 7f088b7fe640 1 --2- 172.21.15.84:0/2966686858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f088c06b8c0 0x7f088c15f7c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f08800090b0 tx=0x7f08800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:08.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.501+0000 7f0888ff9640 1 -- 172.21.15.84:0/2966686858 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088003adf0 con 0x7f088c06b8c0 2023-10-16T10:04:08.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.501+0000 7f0888ff9640 1 -- 172.21.15.84:0/2966686858 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08800443f0 con 0x7f088c06b8c0 2023-10-16T10:04:08.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.501+0000 7f0892389640 1 -- 172.21.15.84:0/2966686858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088c158360 con 0x7f088c06b8c0 2023-10-16T10:04:08.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.501+0000 7f0888ff9640 1 -- 172.21.15.84:0/2966686858 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08800447a0 con 0x7f088c06b8c0 2023-10-16T10:04:08.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.501+0000 7f0892389640 1 -- 172.21.15.84:0/2966686858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088c15fd00 con 0x7f088c06b8c0 2023-10-16T10:04:08.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.502+0000 7f0892389640 1 -- 172.21.15.84:0/2966686858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088c158b20 con 0x7f088c06b8c0 2023-10-16T10:04:08.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.503+0000 7f0888ff9640 1 -- 172.21.15.84:0/2966686858 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 30) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f088004d450 con 0x7f088c06b8c0 2023-10-16T10:04:08.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.504+0000 7f0888ff9640 1 -- 172.21.15.84:0/2966686858 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f088002d040 con 0x7f088c06b8c0 2023-10-16T10:04:08.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.508+0000 7f0888ff9640 1 -- 172.21.15.84:0/2966686858 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0880076b50 con 0x7f088c06b8c0 2023-10-16T10:04:08.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.699+0000 7f0892389640 1 -- 172.21.15.84:0/2966686858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f088c15fef0 con 0x7f088c06b8c0 2023-10-16T10:04:08.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.699+0000 7f0888ff9640 1 -- 172.21.15.84:0/2966686858 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0880035020 con 0x7f088c06b8c0 2023-10-16T10:04:08.701 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T10:04:08.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.702+0000 7f0892389640 1 -- 172.21.15.84:0/2966686858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f088c06b8c0 msgr2=0x7f088c15f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.702+0000 7f0892389640 1 --2- 172.21.15.84:0/2966686858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f088c06b8c0 0x7f088c15f7c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f08800090b0 tx=0x7f08800046c0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.702+0000 7f0892389640 1 -- 172.21.15.84:0/2966686858 shutdown_connections 2023-10-16T10:04:08.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.702+0000 7f0892389640 1 --2- 172.21.15.84:0/2966686858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f088c15d130 0x7f088c156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.702+0000 7f0892389640 1 --2- 172.21.15.84:0/2966686858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f088c06b8c0 0x7f088c15f7c0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.702+0000 7f0892389640 1 --2- 172.21.15.84:0/2966686858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088c06b1a0 0x7f088c158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.702+0000 7f0892389640 1 -- 172.21.15.84:0/2966686858 >> 172.21.15.84:0/2966686858 conn(0x7f088c0d0ad0 msgr2=0x7f088c0ae040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:08.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.702+0000 7f0892389640 1 -- 172.21.15.84:0/2966686858 shutdown_connections 2023-10-16T10:04:08.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.702+0000 7f0892389640 1 -- 172.21.15.84:0/2966686858 wait complete. 2023-10-16T10:04:08.721 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:04:08.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.928+0000 7fa34ccd4640 1 Processor -- start 2023-10-16T10:04:08.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.928+0000 7fa34ccd4640 1 -- start start 2023-10-16T10:04:08.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.930+0000 7fa34ccd4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa34806b660 0x7fa34806ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.930+0000 7fa34ccd4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa3480c9860 0x7fa3480ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.930+0000 7fa34ccd4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3480cc6b0 con 0x7fa3481428c0 2023-10-16T10:04:08.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.930+0000 7fa34ccd4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3481599b0 con 0x7fa3480c9860 2023-10-16T10:04:08.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.930+0000 7fa34ccd4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34806c290 con 0x7fa34806b660 2023-10-16T10:04:08.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.930+0000 7fa33dd9b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa3481428c0 0x7fa3480c9150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55674/0 (socket says 172.21.15.84:55674) 2023-10-16T10:04:08.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.930+0000 7fa33dd9b640 1 -- 172.21.15.84:0/329217463 learned_addr learned my addr 172.21.15.84:0/329217463 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:08.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.930+0000 7fa345d9b640 1 -- 172.21.15.84:0/329217463 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610141616 0 0) 0x7fa3480cc6b0 con 0x7fa3481428c0 2023-10-16T10:04:08.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa345d9b640 1 -- 172.21.15.84:0/329217463 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa334003520 con 0x7fa3481428c0 2023-10-16T10:04:08.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa34659c640 1 --2- 172.21.15.84:0/329217463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa34806b660 0x7fa34806ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa346d9d640 1 --2- 172.21.15.84:0/329217463 >> v2:172.21.15.157:3300/0 conn(0x7fa3480c9860 0x7fa3480ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa345d9b640 1 -- 172.21.15.84:0/329217463 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 539134727 0 0) 0x7fa334003520 con 0x7fa3481428c0 2023-10-16T10:04:08.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa345d9b640 1 -- 172.21.15.84:0/329217463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa34806b660 msgr2=0x7fa34806ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa345d9b640 1 --2- 172.21.15.84:0/329217463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa34806b660 0x7fa34806ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa345d9b640 1 -- 172.21.15.84:0/329217463 >> v2:172.21.15.157:3300/0 conn(0x7fa3480c9860 msgr2=0x7fa3480ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa345d9b640 1 --2- 172.21.15.84:0/329217463 >> v2:172.21.15.157:3300/0 conn(0x7fa3480c9860 0x7fa3480ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa345d9b640 1 -- 172.21.15.84:0/329217463 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa34806c570 con 0x7fa3481428c0 2023-10-16T10:04:08.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa345d9b640 1 -- 172.21.15.84:0/329217463 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fa330001670 con 0x7fa3481428c0 2023-10-16T10:04:08.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa345d9b640 1 -- 172.21.15.84:0/329217463 >> v1:172.21.15.84:6789/0 conn(0x7fa3481428c0 legacy=0x7fa3480c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa345d9b640 1 --2- 172.21.15.84:0/329217463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa334003970 0x7fa334003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa345d9b640 1 --2- 172.21.15.84:0/329217463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3340047c0 0x7fa334004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa345d9b640 1 --2- 172.21.15.84:0/329217463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa34806b660 0x7fa3340051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa345d9b640 1 -- 172.21.15.84:0/329217463 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3481599b0 con 0x7fa3340047c0 2023-10-16T10:04:08.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa345d9b640 1 -- 172.21.15.84:0/329217463 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34806c290 con 0x7fa334003970 2023-10-16T10:04:08.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa345d9b640 1 -- 172.21.15.84:0/329217463 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33400bb60 con 0x7fa34806b660 2023-10-16T10:04:08.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa346d9d640 1 --2- 172.21.15.84:0/329217463 >> v2:172.21.15.157:3300/0 conn(0x7fa3480c9860 0x7fa3480ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:08.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.931+0000 7fa33dd9b640 1 --2- 172.21.15.84:0/329217463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3340047c0 0x7fa334004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.932+0000 7fa34659c640 1 --2- 172.21.15.84:0/329217463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa334003970 0x7fa334003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.932+0000 7fa33dd9b640 1 -- 172.21.15.84:0/329217463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa34806b660 msgr2=0x7fa3340051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.932+0000 7fa33dd9b640 1 --2- 172.21.15.84:0/329217463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa34806b660 0x7fa3340051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.932+0000 7fa33dd9b640 1 -- 172.21.15.84:0/329217463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa334003970 msgr2=0x7fa334003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.932+0000 7fa33dd9b640 1 --2- 172.21.15.84:0/329217463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa334003970 0x7fa334003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.932+0000 7fa33dd9b640 1 -- 172.21.15.84:0/329217463 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa334003520 con 0x7fa3340047c0 2023-10-16T10:04:08.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.933+0000 7fa33dd9b640 1 --2- 172.21.15.84:0/329217463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3340047c0 0x7fa334004c40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa33000c190 tx=0x7fa3340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09b53fa68e1e632 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:08.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.933+0000 7fa345d9b640 1 -- 172.21.15.84:0/329217463 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa330005ad0 con 0x7fa3340047c0 2023-10-16T10:04:08.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.933+0000 7fa345d9b640 1 -- 172.21.15.84:0/329217463 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa330005c30 con 0x7fa3340047c0 2023-10-16T10:04:08.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.933+0000 7fa345d9b640 1 -- 172.21.15.84:0/329217463 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33002c810 con 0x7fa3340047c0 2023-10-16T10:04:08.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.933+0000 7fa34ccd4640 1 -- 172.21.15.84:0/329217463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3340047c0 msgr2=0x7fa334004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.933+0000 7fa34ccd4640 1 --2- 172.21.15.84:0/329217463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3340047c0 0x7fa334004c40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa33000c190 tx=0x7fa3340057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.933+0000 7fa346d9d640 1 -- 172.21.15.84:0/329217463 reap_dead start 2023-10-16T10:04:08.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.933+0000 7fa34ccd4640 1 -- 172.21.15.84:0/329217463 shutdown_connections 2023-10-16T10:04:08.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.933+0000 7fa34ccd4640 1 -- 172.21.15.84:0/329217463 >> 172.21.15.84:0/329217463 conn(0x7fa3480ce9b0 msgr2=0x7fa3480cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:08.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.934+0000 7fa34ccd4640 1 -- 172.21.15.84:0/329217463 shutdown_connections 2023-10-16T10:04:08.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.934+0000 7fa34ccd4640 1 -- 172.21.15.84:0/329217463 wait complete. 2023-10-16T10:04:08.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.935+0000 7fa34ccd4640 1 Processor -- start 2023-10-16T10:04:08.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.935+0000 7fa34ccd4640 1 -- start start 2023-10-16T10:04:08.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.936+0000 7fa34ccd4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3340047c0 0x7fa348141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.936+0000 7fa34ccd4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3481421c0 0x7fa3480bfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.936+0000 7fa34ccd4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa348141c40 0x7fa3480bff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.936+0000 7fa34ccd4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa348150970 con 0x7fa348141c40 2023-10-16T10:04:08.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.936+0000 7fa34ccd4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3480c51f0 con 0x7fa3340047c0 2023-10-16T10:04:08.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.936+0000 7fa34ccd4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3481500d0 con 0x7fa3481421c0 2023-10-16T10:04:08.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.937+0000 7fa346d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa348141c40 0x7fa3480bff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.937+0000 7fa346d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa348141c40 0x7fa3480bff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37626/0 (socket says 172.21.15.84:37626) 2023-10-16T10:04:08.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.937+0000 7fa346d9d640 1 -- 172.21.15.84:0/3406685975 learned_addr learned my addr 172.21.15.84:0/3406685975 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:08.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.937+0000 7fa33dd9b640 1 --2- 172.21.15.84:0/3406685975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3481421c0 0x7fa3480bfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.937+0000 7fa346d9d640 1 -- 172.21.15.84:0/3406685975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3481421c0 msgr2=0x7fa3480bfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.937+0000 7fa34659c640 1 --2- 172.21.15.84:0/3406685975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3340047c0 0x7fa348141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.937+0000 7fa346d9d640 1 --2- 172.21.15.84:0/3406685975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3481421c0 0x7fa3480bfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.937+0000 7fa346d9d640 1 -- 172.21.15.84:0/3406685975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3340047c0 msgr2=0x7fa348141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.937+0000 7fa346d9d640 1 --2- 172.21.15.84:0/3406685975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3340047c0 0x7fa348141700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.937+0000 7fa346d9d640 1 -- 172.21.15.84:0/3406685975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3481503b0 con 0x7fa348141c40 2023-10-16T10:04:08.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.937+0000 7fa346d9d640 1 --2- 172.21.15.84:0/3406685975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa348141c40 0x7fa3480bff70 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa3480ae3a0 tx=0x7fa34000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:08.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.938+0000 7fa33f7fe640 1 -- 172.21.15.84:0/3406685975 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa340015440 con 0x7fa348141c40 2023-10-16T10:04:08.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.938+0000 7fa33f7fe640 1 -- 172.21.15.84:0/3406685975 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3400155c0 con 0x7fa348141c40 2023-10-16T10:04:08.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.938+0000 7fa33f7fe640 1 -- 172.21.15.84:0/3406685975 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa340015970 con 0x7fa348141c40 2023-10-16T10:04:08.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.938+0000 7fa34ccd4640 1 -- 172.21.15.84:0/3406685975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa348150690 con 0x7fa348141c40 2023-10-16T10:04:08.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.938+0000 7fa34ccd4640 1 -- 172.21.15.84:0/3406685975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa34814fc60 con 0x7fa348141c40 2023-10-16T10:04:08.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.939+0000 7fa34ccd4640 1 -- 172.21.15.84:0/3406685975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa348151f30 con 0x7fa348141c40 2023-10-16T10:04:08.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.940+0000 7fa33f7fe640 1 -- 172.21.15.84:0/3406685975 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 30) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fa34000d430 con 0x7fa348141c40 2023-10-16T10:04:08.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.941+0000 7fa33f7fe640 1 -- 172.21.15.84:0/3406685975 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fa340089a50 con 0x7fa348141c40 2023-10-16T10:04:08.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:08.946+0000 7fa33f7fe640 1 -- 172.21.15.84:0/3406685975 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa348151f30 con 0x7fa348141c40 2023-10-16T10:04:09.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.138+0000 7fa34ccd4640 1 -- 172.21.15.84:0/3406685975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa348152310 con 0x7fa348141c40 2023-10-16T10:04:09.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.139+0000 7fa33f7fe640 1 -- 172.21.15.84:0/3406685975 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa348152310 con 0x7fa348141c40 2023-10-16T10:04:09.141 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T10:04:09.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.141+0000 7fa34ccd4640 1 -- 172.21.15.84:0/3406685975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa348141c40 msgr2=0x7fa3480bff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:09.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.141+0000 7fa34ccd4640 1 --2- 172.21.15.84:0/3406685975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa348141c40 0x7fa3480bff70 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa3480ae3a0 tx=0x7fa34000a710 comp rx=0 tx=0).stop 2023-10-16T10:04:09.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.142+0000 7fa34ccd4640 1 -- 172.21.15.84:0/3406685975 shutdown_connections 2023-10-16T10:04:09.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.142+0000 7fa34ccd4640 1 --2- 172.21.15.84:0/3406685975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa348141c40 0x7fa3480bff70 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.142+0000 7fa34ccd4640 1 --2- 172.21.15.84:0/3406685975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3481421c0 0x7fa3480bfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.142+0000 7fa34ccd4640 1 --2- 172.21.15.84:0/3406685975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3340047c0 0x7fa348141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.142+0000 7fa34ccd4640 1 -- 172.21.15.84:0/3406685975 >> 172.21.15.84:0/3406685975 conn(0x7fa3480ce9b0 msgr2=0x7fa348069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:09.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.142+0000 7fa34ccd4640 1 -- 172.21.15.84:0/3406685975 shutdown_connections 2023-10-16T10:04:09.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.142+0000 7fa34ccd4640 1 -- 172.21.15.84:0/3406685975 wait complete. 2023-10-16T10:04:09.162 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:04:09.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.375+0000 7f3265c2d640 1 Processor -- start 2023-10-16T10:04:09.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.375+0000 7f3265c2d640 1 -- start start 2023-10-16T10:04:09.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.376+0000 7f3265c2d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f326015aaa0 0x7f326015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.376+0000 7f3265c2d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f326015ae80 0x7f32600cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.377+0000 7f3265c2d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f32600ccaa0 0x7f3260070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.377+0000 7f3265c2d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326006b810 con 0x7f326015ae80 2023-10-16T10:04:09.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.377+0000 7f3265c2d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326015ed60 con 0x7f32600ccaa0 2023-10-16T10:04:09.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.377+0000 7f3265c2d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32600707e0 con 0x7f326015aaa0 2023-10-16T10:04:09.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.377+0000 7f3257fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f326015ae80 0x7f32600cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.377+0000 7f3257fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f326015ae80 0x7f32600cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37638/0 (socket says 172.21.15.84:37638) 2023-10-16T10:04:09.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.377+0000 7f3257fff640 1 -- 172.21.15.84:0/3935951067 learned_addr learned my addr 172.21.15.84:0/3935951067 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:09.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.377+0000 7f325ffff640 1 --2- 172.21.15.84:0/3935951067 >> v2:172.21.15.157:3300/0 conn(0x7f32600ccaa0 0x7f3260070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.377+0000 7f3257fff640 1 -- 172.21.15.84:0/3935951067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f326015aaa0 msgr2=0x7f326015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:09.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.377+0000 7f3257fff640 1 --2- 172.21.15.84:0/3935951067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f326015aaa0 0x7f326015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.377+0000 7f3257fff640 1 -- 172.21.15.84:0/3935951067 >> v2:172.21.15.157:3300/0 conn(0x7f32600ccaa0 msgr2=0x7f3260070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:09.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.377+0000 7f3257fff640 1 --2- 172.21.15.84:0/3935951067 >> v2:172.21.15.157:3300/0 conn(0x7f32600ccaa0 0x7f3260070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.377+0000 7f3257fff640 1 -- 172.21.15.84:0/3935951067 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326015ef90 con 0x7f326015ae80 2023-10-16T10:04:09.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.378+0000 7f3257fff640 1 --2- 172.21.15.84:0/3935951067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326015ae80 0x7f32600cc560 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f32600ae3c0 tx=0x7f324802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05deffb4cf8776c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:09.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.379+0000 7f325effd640 1 -- 172.21.15.84:0/3935951067 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248031860 con 0x7f326015ae80 2023-10-16T10:04:09.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.379+0000 7f325effd640 1 -- 172.21.15.84:0/3935951067 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32480319c0 con 0x7f326015ae80 2023-10-16T10:04:09.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.379+0000 7f325effd640 1 -- 172.21.15.84:0/3935951067 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248031d50 con 0x7f326015ae80 2023-10-16T10:04:09.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.379+0000 7f3265c2d640 1 -- 172.21.15.84:0/3935951067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326015ae80 msgr2=0x7f32600cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:09.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.379+0000 7f3265c2d640 1 --2- 172.21.15.84:0/3935951067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326015ae80 0x7f32600cc560 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f32600ae3c0 tx=0x7f324802b900 comp rx=0 tx=0).stop 2023-10-16T10:04:09.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.379+0000 7f3265c2d640 1 -- 172.21.15.84:0/3935951067 shutdown_connections 2023-10-16T10:04:09.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.379+0000 7f3265c2d640 1 --2- 172.21.15.84:0/3935951067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326015ae80 0x7f32600cc560 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.379+0000 7f3265c2d640 1 --2- 172.21.15.84:0/3935951067 >> v2:172.21.15.157:3300/0 conn(0x7f32600ccaa0 0x7f3260070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.379+0000 7f3265c2d640 1 --2- 172.21.15.84:0/3935951067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f326015aaa0 0x7f326015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.379+0000 7f3265c2d640 1 -- 172.21.15.84:0/3935951067 >> 172.21.15.84:0/3935951067 conn(0x7f32600cb410 msgr2=0x7f32600cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:09.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.379+0000 7f3265c2d640 1 -- 172.21.15.84:0/3935951067 shutdown_connections 2023-10-16T10:04:09.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.380+0000 7f3265c2d640 1 -- 172.21.15.84:0/3935951067 wait complete. 2023-10-16T10:04:09.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.381+0000 7f3265c2d640 1 Processor -- start 2023-10-16T10:04:09.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.381+0000 7f3265c2d640 1 -- start start 2023-10-16T10:04:09.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.382+0000 7f3265c2d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32600ccaa0 0x7f32601e45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.382+0000 7f3265c2d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f326015aaa0 0x7f3260158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.382+0000 7f3265c2d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326015ae80 0x7f32600c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.382+0000 7f3265c2d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32601593f0 con 0x7f326015ae80 2023-10-16T10:04:09.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.382+0000 7f3265c2d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326015cd10 con 0x7f32600ccaa0 2023-10-16T10:04:09.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.382+0000 7f3265c2d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32601f2940 con 0x7f326015aaa0 2023-10-16T10:04:09.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.383+0000 7f325ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326015ae80 0x7f32600c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.383+0000 7f325ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326015ae80 0x7f32600c4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37642/0 (socket says 172.21.15.84:37642) 2023-10-16T10:04:09.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.383+0000 7f325ffff640 1 -- 172.21.15.84:0/3368041609 learned_addr learned my addr 172.21.15.84:0/3368041609 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:09.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.383+0000 7f325f7fe640 1 --2- 172.21.15.84:0/3368041609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32600ccaa0 0x7f32601e45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.383+0000 7f3257fff640 1 --2- 172.21.15.84:0/3368041609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f326015aaa0 0x7f3260158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.383+0000 7f325ffff640 1 -- 172.21.15.84:0/3368041609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f326015aaa0 msgr2=0x7f3260158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:09.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.383+0000 7f325ffff640 1 --2- 172.21.15.84:0/3368041609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f326015aaa0 0x7f3260158eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.383+0000 7f325ffff640 1 -- 172.21.15.84:0/3368041609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32600ccaa0 msgr2=0x7f32601e45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:09.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.383+0000 7f325ffff640 1 --2- 172.21.15.84:0/3368041609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32600ccaa0 0x7f32601e45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.383+0000 7f325ffff640 1 -- 172.21.15.84:0/3368041609 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32480094d0 con 0x7f326015ae80 2023-10-16T10:04:09.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.383+0000 7f325ffff640 1 --2- 172.21.15.84:0/3368041609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326015ae80 0x7f32600c4690 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f325800cc30 tx=0x7f325800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:09.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.384+0000 7f325cff9640 1 -- 172.21.15.84:0/3368041609 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325800ce40 con 0x7f326015ae80 2023-10-16T10:04:09.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.384+0000 7f325cff9640 1 -- 172.21.15.84:0/3368041609 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3258003990 con 0x7f326015ae80 2023-10-16T10:04:09.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.384+0000 7f325cff9640 1 -- 172.21.15.84:0/3368041609 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258003d20 con 0x7f326015ae80 2023-10-16T10:04:09.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.384+0000 7f3265c2d640 1 -- 172.21.15.84:0/3368041609 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32600c21f0 con 0x7f326015ae80 2023-10-16T10:04:09.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.384+0000 7f3265c2d640 1 -- 172.21.15.84:0/3368041609 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32601e52a0 con 0x7f326015ae80 2023-10-16T10:04:09.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.385+0000 7f3265c2d640 1 -- 172.21.15.84:0/3368041609 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326015cef0 con 0x7f326015ae80 2023-10-16T10:04:09.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.386+0000 7f325cff9640 1 -- 172.21.15.84:0/3368041609 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 31) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f3258003e80 con 0x7f326015ae80 2023-10-16T10:04:09.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.387+0000 7f325cff9640 1 --2- 172.21.15.84:0/3368041609 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f324006acc0 0x7f324006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.387+0000 7f325cff9640 1 -- 172.21.15.84:0/3368041609 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f32580899e0 con 0x7f326015ae80 2023-10-16T10:04:09.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.387+0000 7f325f7fe640 1 --2- 172.21.15.84:0/3368041609 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f324006acc0 0x7f324006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.388+0000 7f325f7fe640 1 --2- 172.21.15.84:0/3368041609 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f324006acc0 0x7f324006d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f32600b38f0 tx=0x7f3250008040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:09.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.391+0000 7f325cff9640 1 -- 172.21.15.84:0/3368041609 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f326015cef0 con 0x7f326015ae80 2023-10-16T10:04:09.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.621+0000 7f3265c2d640 1 -- 172.21.15.84:0/3368041609 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f326015cef0 con 0x7f326015ae80 2023-10-16T10:04:09.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.622+0000 7f325cff9640 1 -- 172.21.15.84:0/3368041609 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f326015cef0 con 0x7f326015ae80 2023-10-16T10:04:09.624 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:09.624 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3040499154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3040499154},{"type":"v1","addr":"172.21.15.157:6835","nonce":3040499154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2727537766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2727537766},{"type":"v1","addr":"172.21.15.157:6833","nonce":2727537766}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/49292950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":49292950},{"type":"v1","addr":"172.21.15.84:6836","nonce":49292950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:02:13.514054+0000","modified":"2023-10-16T10:02:16.598055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/3839442164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3839442164},{"type":"v1","addr":"172.21.15.84:6837","nonce":3839442164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T10:04:09.624 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 25 2023-10-16T10:04:09.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.624+0000 7f3265c2d640 1 -- 172.21.15.84:0/3368041609 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f324006acc0 msgr2=0x7f324006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:09.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.624+0000 7f3265c2d640 1 --2- 172.21.15.84:0/3368041609 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f324006acc0 0x7f324006d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f32600b38f0 tx=0x7f3250008040 comp rx=0 tx=0).stop 2023-10-16T10:04:09.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.624+0000 7f3265c2d640 1 -- 172.21.15.84:0/3368041609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326015ae80 msgr2=0x7f32600c4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:09.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.624+0000 7f3265c2d640 1 --2- 172.21.15.84:0/3368041609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326015ae80 0x7f32600c4690 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f325800cc30 tx=0x7f325800a5c0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.624+0000 7f3265c2d640 1 -- 172.21.15.84:0/3368041609 shutdown_connections 2023-10-16T10:04:09.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.624+0000 7f3265c2d640 1 --2- 172.21.15.84:0/3368041609 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f324006acc0 0x7f324006d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.624+0000 7f3265c2d640 1 --2- 172.21.15.84:0/3368041609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326015ae80 0x7f32600c4690 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.624+0000 7f3265c2d640 1 --2- 172.21.15.84:0/3368041609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f326015aaa0 0x7f3260158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.624+0000 7f3265c2d640 1 --2- 172.21.15.84:0/3368041609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32600ccaa0 0x7f32601e45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.624+0000 7f3265c2d640 1 -- 172.21.15.84:0/3368041609 >> 172.21.15.84:0/3368041609 conn(0x7f32600cb410 msgr2=0x7f32600ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:09.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.625+0000 7f3265c2d640 1 -- 172.21.15.84:0/3368041609 shutdown_connections 2023-10-16T10:04:09.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.625+0000 7f3265c2d640 1 -- 172.21.15.84:0/3368041609 wait complete. 2023-10-16T10:04:09.644 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:04:09.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.858+0000 7f3742ba9640 1 Processor -- start 2023-10-16T10:04:09.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.858+0000 7f3742ba9640 1 -- start start 2023-10-16T10:04:09.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.859+0000 7f3742ba9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f373c159f00 0x7f373c0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.860+0000 7f3742ba9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f373c15d880 0x7f373c0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.860+0000 7f3742ba9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f373c06f7f0 con 0x7f373c159b20 2023-10-16T10:04:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.860+0000 7f3742ba9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373c06c5e0 con 0x7f373c15d880 2023-10-16T10:04:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.860+0000 7f3742ba9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373c06c8c0 con 0x7f373c159f00 2023-10-16T10:04:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.860+0000 7f3741146640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f373c15d880 0x7f373c0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.860+0000 7f373bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f373c159f00 0x7f373c0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.860+0000 7f3741146640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f373c15d880 0x7f373c0b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:32930/0 (socket says 172.21.15.84:32930) 2023-10-16T10:04:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.860+0000 7f3741146640 1 -- 172.21.15.84:0/3216996629 learned_addr learned my addr 172.21.15.84:0/3216996629 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.861+0000 7f373bfff640 1 -- 172.21.15.84:0/3216996629 >> v2:172.21.15.157:3300/0 conn(0x7f373c15d880 msgr2=0x7f373c0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.861+0000 7f373bfff640 1 --2- 172.21.15.84:0/3216996629 >> v2:172.21.15.157:3300/0 conn(0x7f373c15d880 0x7f373c0b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.861+0000 7f373bfff640 1 -- 172.21.15.84:0/3216996629 >> v1:172.21.15.157:6789/0 conn(0x7f373c159b20 legacy=0x7f373c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.861+0000 7f373bfff640 1 -- 172.21.15.84:0/3216996629 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373c06cba0 con 0x7f373c159f00 2023-10-16T10:04:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.861+0000 7f3741146640 1 --2- 172.21.15.84:0/3216996629 >> v2:172.21.15.157:3300/0 conn(0x7f373c15d880 0x7f373c0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:04:09.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.863+0000 7f373bfff640 1 --2- 172.21.15.84:0/3216996629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f373c159f00 0x7f373c0b0410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f37280097a0 tx=0x7f372802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e81c96f7b990c25 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:09.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.863+0000 7f373b7fe640 1 -- 172.21.15.84:0/3216996629 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372802d040 con 0x7f373c159f00 2023-10-16T10:04:09.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.863+0000 7f373b7fe640 1 -- 172.21.15.84:0/3216996629 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3728031810 con 0x7f373c159f00 2023-10-16T10:04:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.863+0000 7f373b7fe640 1 -- 172.21.15.84:0/3216996629 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3728031ba0 con 0x7f373c159f00 2023-10-16T10:04:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.864+0000 7f3742ba9640 1 -- 172.21.15.84:0/3216996629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f373c159f00 msgr2=0x7f373c0b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.864+0000 7f3742ba9640 1 --2- 172.21.15.84:0/3216996629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f373c159f00 0x7f373c0b0410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f37280097a0 tx=0x7f372802b8b0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.864+0000 7f3742ba9640 1 -- 172.21.15.84:0/3216996629 shutdown_connections 2023-10-16T10:04:09.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.864+0000 7f3742ba9640 1 --2- 172.21.15.84:0/3216996629 >> v2:172.21.15.157:3300/0 conn(0x7f373c15d880 0x7f373c0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.864+0000 7f3742ba9640 1 --2- 172.21.15.84:0/3216996629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f373c159f00 0x7f373c0b0410 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.864+0000 7f3742ba9640 1 -- 172.21.15.84:0/3216996629 >> 172.21.15.84:0/3216996629 conn(0x7f373c0cb410 msgr2=0x7f373c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:09.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.864+0000 7f3742ba9640 1 -- 172.21.15.84:0/3216996629 shutdown_connections 2023-10-16T10:04:09.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.864+0000 7f3742ba9640 1 -- 172.21.15.84:0/3216996629 wait complete. 2023-10-16T10:04:09.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.866+0000 7f3742ba9640 1 Processor -- start 2023-10-16T10:04:09.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.866+0000 7f3742ba9640 1 -- start start 2023-10-16T10:04:09.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.868+0000 7f3742ba9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f373c159b20 0x7f373c1f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.868+0000 7f3742ba9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f373c159f00 0x7f373c0c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.868+0000 7f3742ba9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f373c15d880 0x7f373c0c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.868+0000 7f3742ba9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c1f3050 con 0x7f373c159b20 2023-10-16T10:04:09.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.868+0000 7f3742ba9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c1526e0 con 0x7f373c159f00 2023-10-16T10:04:09.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.868+0000 7f3742ba9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373c0c70c0 con 0x7f373c15d880 2023-10-16T10:04:09.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.868+0000 7f3740945640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f373c159b20 0x7f373c1f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.868+0000 7f3740945640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f373c159b20 0x7f373c1f2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37644/0 (socket says 172.21.15.84:37644) 2023-10-16T10:04:09.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.868+0000 7f3740945640 1 -- 172.21.15.84:0/2863582438 learned_addr learned my addr 172.21.15.84:0/2863582438 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:09.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.868+0000 7f373bfff640 1 --2- 172.21.15.84:0/2863582438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f373c159f00 0x7f373c0c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.868+0000 7f3741146640 1 --2- 172.21.15.84:0/2863582438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f373c15d880 0x7f373c0c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.868+0000 7f3740945640 1 -- 172.21.15.84:0/2863582438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f373c15d880 msgr2=0x7f373c0c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:09.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.868+0000 7f3740945640 1 --2- 172.21.15.84:0/2863582438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f373c15d880 0x7f373c0c5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.868+0000 7f3740945640 1 -- 172.21.15.84:0/2863582438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f373c159f00 msgr2=0x7f373c0c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:09.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.868+0000 7f3740945640 1 --2- 172.21.15.84:0/2863582438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f373c159f00 0x7f373c0c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.868+0000 7f3740945640 1 -- 172.21.15.84:0/2863582438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3734009b30 con 0x7f373c159b20 2023-10-16T10:04:09.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.869+0000 7f3740945640 1 --2- 172.21.15.84:0/2863582438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f373c159b20 0x7f373c1f2b10 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f373000c430 tx=0x7f373000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:09.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.869+0000 7f37397fa640 1 -- 172.21.15.84:0/2863582438 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37300052d0 con 0x7f373c159b20 2023-10-16T10:04:09.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.869+0000 7f37397fa640 1 -- 172.21.15.84:0/2863582438 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3730005430 con 0x7f373c159b20 2023-10-16T10:04:09.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.869+0000 7f37397fa640 1 -- 172.21.15.84:0/2863582438 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730003830 con 0x7f373c159b20 2023-10-16T10:04:09.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.869+0000 7f3742ba9640 1 -- 172.21.15.84:0/2863582438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3728009400 con 0x7f373c159b20 2023-10-16T10:04:09.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.869+0000 7f3742ba9640 1 -- 172.21.15.84:0/2863582438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373c0c6790 con 0x7f373c159b20 2023-10-16T10:04:09.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.870+0000 7f3742ba9640 1 -- 172.21.15.84:0/2863582438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f373c04e160 con 0x7f373c159b20 2023-10-16T10:04:09.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.871+0000 7f37397fa640 1 -- 172.21.15.84:0/2863582438 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 31) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f3730003990 con 0x7f373c159b20 2023-10-16T10:04:09.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.872+0000 7f37397fa640 1 --2- 172.21.15.84:0/2863582438 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f372006acc0 0x7f372006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.872+0000 7f37397fa640 1 -- 172.21.15.84:0/2863582438 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f3730088aa0 con 0x7f373c159b20 2023-10-16T10:04:09.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.872+0000 7f373bfff640 1 --2- 172.21.15.84:0/2863582438 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f372006acc0 0x7f372006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.873+0000 7f373bfff640 1 --2- 172.21.15.84:0/2863582438 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f372006acc0 0x7f372006d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3728036000 tx=0x7f372803c040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:09.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:09.877+0000 7f37397fa640 1 -- 172.21.15.84:0/2863582438 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3730053b80 con 0x7f373c159b20 2023-10-16T10:04:10.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.074+0000 7f3742ba9640 1 -- 172.21.15.84:0/2863582438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f373c0b10b0 con 0x7f373c159b20 2023-10-16T10:04:10.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.077+0000 7f37397fa640 1 -- 172.21.15.84:0/2863582438 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+17701 (secure 0 0 0) 0x7f3730017090 con 0x7f373c159b20 2023-10-16T10:04:10.079 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:10.079 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":69,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:04:08.229141+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:02:11.266402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-16T10:02:12.562100+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":68,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.13","mappings":[{"from":0,"to":5}]},{"pgid":"7.2d","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6832/633652932":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/724237746":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/3806126461":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/3926486342":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/3656680984":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/2816959180":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/4150953381":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/351541199":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/555434621":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/3440750653":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/3623389468":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/3654971153":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/621840141":"2023-10-17T10:02:27.969914+0000","172.21.15.84:6833/585186562":"2023-10-17T10:02:27.969914+0000","172.21.15.84:6833/633652932":"2023-10-17T10:04:08.229109+0000","172.21.15.84:6832/585186562":"2023-10-17T10:02:27.969914+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:04:10.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.079+0000 7f3742ba9640 1 -- 172.21.15.84:0/2863582438 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f372006acc0 msgr2=0x7f372006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.079+0000 7f3742ba9640 1 --2- 172.21.15.84:0/2863582438 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f372006acc0 0x7f372006d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3728036000 tx=0x7f372803c040 comp rx=0 tx=0).stop 2023-10-16T10:04:10.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.079+0000 7f3742ba9640 1 -- 172.21.15.84:0/2863582438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f373c159b20 msgr2=0x7f373c1f2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.079+0000 7f3742ba9640 1 --2- 172.21.15.84:0/2863582438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f373c159b20 0x7f373c1f2b10 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f373000c430 tx=0x7f373000c8f0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.079+0000 7f3742ba9640 1 -- 172.21.15.84:0/2863582438 shutdown_connections 2023-10-16T10:04:10.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.079+0000 7f3742ba9640 1 --2- 172.21.15.84:0/2863582438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f373c15d880 0x7f373c0c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.079+0000 7f3742ba9640 1 --2- 172.21.15.84:0/2863582438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f373c159f00 0x7f373c0c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.079+0000 7f3742ba9640 1 --2- 172.21.15.84:0/2863582438 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f372006acc0 0x7f372006d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.079+0000 7f3742ba9640 1 --2- 172.21.15.84:0/2863582438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f373c159b20 0x7f373c1f2b10 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.079+0000 7f3742ba9640 1 -- 172.21.15.84:0/2863582438 >> 172.21.15.84:0/2863582438 conn(0x7f373c0cb410 msgr2=0x7f373c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:10.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.080+0000 7f3742ba9640 1 -- 172.21.15.84:0/2863582438 shutdown_connections 2023-10-16T10:04:10.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.080+0000 7f3742ba9640 1 -- 172.21.15.84:0/2863582438 wait complete. 2023-10-16T10:04:10.101 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T10:04:10.101 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:04:10.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.319+0000 7f6168df4640 1 Processor -- start 2023-10-16T10:04:10.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.319+0000 7f6168df4640 1 -- start start 2023-10-16T10:04:10.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.320+0000 7f6168df4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f61640ad970 0x7f61640cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.320+0000 7f6168df4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f616415b9d0 con 0x7f616406c040 2023-10-16T10:04:10.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.320+0000 7f6168df4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f616415bcb0 con 0x7f61640ccfb0 2023-10-16T10:04:10.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.320+0000 7f6168df4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f616415bf90 con 0x7f61640ad970 2023-10-16T10:04:10.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.321+0000 7f6162ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f61640ad970 0x7f61640cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.321+0000 7f6162ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f61640ad970 0x7f61640cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37658/0 (socket says 172.21.15.84:37658) 2023-10-16T10:04:10.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.321+0000 7f6162ffd640 1 -- 172.21.15.84:0/1711499333 learned_addr learned my addr 172.21.15.84:0/1711499333 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:10.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.321+0000 7f6162ffd640 1 -- 172.21.15.84:0/1711499333 >> v1:172.21.15.157:6789/0 conn(0x7f61640ccfb0 legacy=0x7f61640cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:10.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.321+0000 7f6162ffd640 1 -- 172.21.15.84:0/1711499333 >> v1:172.21.15.84:6789/0 conn(0x7f616406c040 legacy=0x7f616406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.321+0000 7f6162ffd640 1 -- 172.21.15.84:0/1711499333 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61640cc270 con 0x7f61640ad970 2023-10-16T10:04:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.322+0000 7f6162ffd640 1 --2- 172.21.15.84:0/1711499333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61640ad970 0x7f61640cca70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f61580098a0 tx=0x7f615802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e16b665b1ee282 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.323+0000 7f61627fc640 1 -- 172.21.15.84:0/1711499333 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158030840 con 0x7f61640ad970 2023-10-16T10:04:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.323+0000 7f61627fc640 1 -- 172.21.15.84:0/1711499333 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61580309a0 con 0x7f61640ad970 2023-10-16T10:04:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.323+0000 7f61627fc640 1 -- 172.21.15.84:0/1711499333 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158030d30 con 0x7f61640ad970 2023-10-16T10:04:10.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.323+0000 7f6168df4640 1 -- 172.21.15.84:0/1711499333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61640ad970 msgr2=0x7f61640cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.323+0000 7f6168df4640 1 --2- 172.21.15.84:0/1711499333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61640ad970 0x7f61640cca70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f61580098a0 tx=0x7f615802b980 comp rx=0 tx=0).stop 2023-10-16T10:04:10.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.323+0000 7f6168df4640 1 -- 172.21.15.84:0/1711499333 shutdown_connections 2023-10-16T10:04:10.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.323+0000 7f6168df4640 1 --2- 172.21.15.84:0/1711499333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61640ad970 0x7f61640cca70 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.323+0000 7f6168df4640 1 -- 172.21.15.84:0/1711499333 >> 172.21.15.84:0/1711499333 conn(0x7f61640ce6a0 msgr2=0x7f61640ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:10.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.324+0000 7f6168df4640 1 -- 172.21.15.84:0/1711499333 shutdown_connections 2023-10-16T10:04:10.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.324+0000 7f6168df4640 1 -- 172.21.15.84:0/1711499333 wait complete. 2023-10-16T10:04:10.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.325+0000 7f6168df4640 1 Processor -- start 2023-10-16T10:04:10.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.325+0000 7f6168df4640 1 -- start start 2023-10-16T10:04:10.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.326+0000 7f6168df4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f616406c040 0x7f616406f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.326+0000 7f6168df4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61640ad970 0x7f616415ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.326+0000 7f6168df4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61640ccfb0 0x7f616415b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.326+0000 7f6168df4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61641f09f0 con 0x7f616406c040 2023-10-16T10:04:10.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.326+0000 7f6168df4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61640c5010 con 0x7f61640ccfb0 2023-10-16T10:04:10.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.326+0000 7f6168df4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61640b2f20 con 0x7f61640ad970 2023-10-16T10:04:10.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.327+0000 7f61637fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f616406c040 0x7f616406f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.327+0000 7f61637fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f616406c040 0x7f616406f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37674/0 (socket says 172.21.15.84:37674) 2023-10-16T10:04:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.327+0000 7f61637fe640 1 -- 172.21.15.84:0/889954872 learned_addr learned my addr 172.21.15.84:0/889954872 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.327+0000 7f6163fff640 1 --2- 172.21.15.84:0/889954872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61640ccfb0 0x7f616415b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.327+0000 7f61637fe640 1 -- 172.21.15.84:0/889954872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61640ad970 msgr2=0x7f616415ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.327+0000 7f61637fe640 1 --2- 172.21.15.84:0/889954872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61640ad970 0x7f616415ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.327+0000 7f61637fe640 1 -- 172.21.15.84:0/889954872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61640ccfb0 msgr2=0x7f616415b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.327+0000 7f61637fe640 1 --2- 172.21.15.84:0/889954872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61640ccfb0 0x7f616415b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.327+0000 7f61637fe640 1 -- 172.21.15.84:0/889954872 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61580094d0 con 0x7f616406c040 2023-10-16T10:04:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.327+0000 7f6163fff640 1 --2- 172.21.15.84:0/889954872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61640ccfb0 0x7f616415b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.327+0000 7f61637fe640 1 --2- 172.21.15.84:0/889954872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f616406c040 0x7f616406f970 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f615400c400 tx=0x7f615400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.328+0000 7f614bfff640 1 -- 172.21.15.84:0/889954872 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61540052d0 con 0x7f616406c040 2023-10-16T10:04:10.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.328+0000 7f614bfff640 1 -- 172.21.15.84:0/889954872 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6154005430 con 0x7f616406c040 2023-10-16T10:04:10.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.328+0000 7f614bfff640 1 -- 172.21.15.84:0/889954872 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154003830 con 0x7f616406c040 2023-10-16T10:04:10.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.328+0000 7f6168df4640 1 -- 172.21.15.84:0/889954872 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61640b3200 con 0x7f616406c040 2023-10-16T10:04:10.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.328+0000 7f6168df4640 1 -- 172.21.15.84:0/889954872 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61641f0260 con 0x7f616406c040 2023-10-16T10:04:10.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.329+0000 7f6168df4640 1 -- 172.21.15.84:0/889954872 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616404e9b0 con 0x7f616406c040 2023-10-16T10:04:10.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.330+0000 7f614bfff640 1 -- 172.21.15.84:0/889954872 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 31) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f6154003990 con 0x7f616406c040 2023-10-16T10:04:10.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.331+0000 7f614bfff640 1 --2- 172.21.15.84:0/889954872 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f615006acc0 0x7f615006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.331+0000 7f614bfff640 1 -- 172.21.15.84:0/889954872 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f6154088b90 con 0x7f616406c040 2023-10-16T10:04:10.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.331+0000 7f6162ffd640 1 --2- 172.21.15.84:0/889954872 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f615006acc0 0x7f615006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.332+0000 7f6162ffd640 1 --2- 172.21.15.84:0/889954872 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f615006acc0 0x7f615006d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6158036000 tx=0x7f615803b040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:10.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.336+0000 7f614bfff640 1 -- 172.21.15.84:0/889954872 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6154053ba0 con 0x7f616406c040 2023-10-16T10:04:10.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.575+0000 7f6168df4640 1 -- 172.21.15.84:0/889954872 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f616406b2b0 con 0x7f616406c040 2023-10-16T10:04:10.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.576+0000 7f614bfff640 1 -- 172.21.15.84:0/889954872 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f6154017090 con 0x7f616406c040 2023-10-16T10:04:10.578 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:10.578 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3040499154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3040499154},{"type":"v1","addr":"172.21.15.157:6835","nonce":3040499154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2727537766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2727537766},{"type":"v1","addr":"172.21.15.157:6833","nonce":2727537766}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/49292950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":49292950},{"type":"v1","addr":"172.21.15.84:6836","nonce":49292950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:02:13.514054+0000","modified":"2023-10-16T10:02:16.598055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/3839442164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3839442164},{"type":"v1","addr":"172.21.15.84:6837","nonce":3839442164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T10:04:10.579 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 25 2023-10-16T10:04:10.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.579+0000 7f6168df4640 1 -- 172.21.15.84:0/889954872 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f615006acc0 msgr2=0x7f615006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.579+0000 7f6168df4640 1 --2- 172.21.15.84:0/889954872 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f615006acc0 0x7f615006d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6158036000 tx=0x7f615803b040 comp rx=0 tx=0).stop 2023-10-16T10:04:10.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.579+0000 7f6168df4640 1 -- 172.21.15.84:0/889954872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f616406c040 msgr2=0x7f616406f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.579+0000 7f6168df4640 1 --2- 172.21.15.84:0/889954872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f616406c040 0x7f616406f970 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f615400c400 tx=0x7f615400c8c0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.579+0000 7f6168df4640 1 -- 172.21.15.84:0/889954872 shutdown_connections 2023-10-16T10:04:10.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.579+0000 7f6168df4640 1 --2- 172.21.15.84:0/889954872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61640ccfb0 0x7f616415b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.579+0000 7f6168df4640 1 --2- 172.21.15.84:0/889954872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61640ad970 0x7f616415ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.579+0000 7f6168df4640 1 --2- 172.21.15.84:0/889954872 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f615006acc0 0x7f615006d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.579+0000 7f6168df4640 1 --2- 172.21.15.84:0/889954872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f616406c040 0x7f616406f970 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.579+0000 7f6168df4640 1 -- 172.21.15.84:0/889954872 >> 172.21.15.84:0/889954872 conn(0x7f61640ce6a0 msgr2=0x7f616415c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:10.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.580+0000 7f6168df4640 1 -- 172.21.15.84:0/889954872 shutdown_connections 2023-10-16T10:04:10.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.580+0000 7f6168df4640 1 -- 172.21.15.84:0/889954872 wait complete. 2023-10-16T10:04:10.599 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:04:10.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.795+0000 7f016f683640 1 Processor -- start 2023-10-16T10:04:10.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.795+0000 7f016f683640 1 -- start start 2023-10-16T10:04:10.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.796+0000 7f016f683640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f01680cd0c0 0x7f01681615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.796+0000 7f016f683640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f016815e3e0 con 0x7f01680b0410 2023-10-16T10:04:10.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.796+0000 7f016f683640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01680cd4a0 con 0x7f016815db00 2023-10-16T10:04:10.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.796+0000 7f016f683640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f016815e100 con 0x7f01680cd0c0 2023-10-16T10:04:10.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.797+0000 7f016d41f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f01680cd0c0 0x7f01681615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.797+0000 7f016cc1e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f01680b0410 0x7f016815d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55686/0 (socket says 172.21.15.84:55686) 2023-10-16T10:04:10.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.797+0000 7f016cc1e640 1 -- 172.21.15.84:0/2591138930 learned_addr learned my addr 172.21.15.84:0/2591138930 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:10.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.797+0000 7f016d41f640 1 -- 172.21.15.84:0/2591138930 >> v1:172.21.15.157:6789/0 conn(0x7f016815db00 legacy=0x7f016806f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:10.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.797+0000 7f016d41f640 1 -- 172.21.15.84:0/2591138930 >> v1:172.21.15.84:6789/0 conn(0x7f01680b0410 legacy=0x7f016815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.797+0000 7f016d41f640 1 -- 172.21.15.84:0/2591138930 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0168160b10 con 0x7f01680cd0c0 2023-10-16T10:04:10.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.798+0000 7f016d41f640 1 --2- 172.21.15.84:0/2591138930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01680cd0c0 0x7f01681615f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f01640097a0 tx=0x7f016402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4658ae51b6d766d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:10.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.798+0000 7f015ffff640 1 -- 172.21.15.84:0/2591138930 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016402e070 con 0x7f01680cd0c0 2023-10-16T10:04:10.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.799+0000 7f015ffff640 1 -- 172.21.15.84:0/2591138930 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01640315a0 con 0x7f01680cd0c0 2023-10-16T10:04:10.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.799+0000 7f015ffff640 1 -- 172.21.15.84:0/2591138930 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164031930 con 0x7f01680cd0c0 2023-10-16T10:04:10.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.799+0000 7f016f683640 1 -- 172.21.15.84:0/2591138930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01680cd0c0 msgr2=0x7f01681615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.799+0000 7f016f683640 1 --2- 172.21.15.84:0/2591138930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01680cd0c0 0x7f01681615f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f01640097a0 tx=0x7f016402b840 comp rx=0 tx=0).stop 2023-10-16T10:04:10.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.799+0000 7f016f683640 1 -- 172.21.15.84:0/2591138930 shutdown_connections 2023-10-16T10:04:10.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.799+0000 7f016f683640 1 --2- 172.21.15.84:0/2591138930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01680cd0c0 0x7f01681615f0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.799+0000 7f016f683640 1 -- 172.21.15.84:0/2591138930 >> 172.21.15.84:0/2591138930 conn(0x7f01680cb410 msgr2=0x7f01680cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:10.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.799+0000 7f016f683640 1 -- 172.21.15.84:0/2591138930 shutdown_connections 2023-10-16T10:04:10.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.799+0000 7f016f683640 1 -- 172.21.15.84:0/2591138930 wait complete. 2023-10-16T10:04:10.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.801+0000 7f016f683640 1 Processor -- start 2023-10-16T10:04:10.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.801+0000 7f016f683640 1 -- start start 2023-10-16T10:04:10.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.802+0000 7f016f683640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01680b0410 0x7f01680b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.802+0000 7f016f683640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01680cd0c0 0x7f0168159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.802+0000 7f016f683640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f016815db00 0x7f0168156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.802+0000 7f016f683640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016815a5f0 con 0x7f01680b0410 2023-10-16T10:04:10.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.802+0000 7f016f683640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016815a310 con 0x7f01680cd0c0 2023-10-16T10:04:10.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.802+0000 7f016f683640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01680c2660 con 0x7f016815db00 2023-10-16T10:04:10.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.802+0000 7f016d41f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01680b0410 0x7f01680b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.802+0000 7f016d41f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01680b0410 0x7f01680b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37698/0 (socket says 172.21.15.84:37698) 2023-10-16T10:04:10.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.803+0000 7f016d41f640 1 -- 172.21.15.84:0/1070929549 learned_addr learned my addr 172.21.15.84:0/1070929549 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:10.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.803+0000 7f016cc1e640 1 --2- 172.21.15.84:0/1070929549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01680cd0c0 0x7f0168159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.803+0000 7f016d41f640 1 -- 172.21.15.84:0/1070929549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f016815db00 msgr2=0x7f0168156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.803+0000 7f016d41f640 1 --2- 172.21.15.84:0/1070929549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f016815db00 0x7f0168156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.803+0000 7f016d41f640 1 -- 172.21.15.84:0/1070929549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01680cd0c0 msgr2=0x7f0168159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.803+0000 7f016d41f640 1 --2- 172.21.15.84:0/1070929549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01680cd0c0 0x7f0168159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.803+0000 7f016d41f640 1 -- 172.21.15.84:0/1070929549 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0164009400 con 0x7f01680b0410 2023-10-16T10:04:10.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.803+0000 7f016cc1e640 1 --2- 172.21.15.84:0/1070929549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01680cd0c0 0x7f0168159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:10.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.803+0000 7f016d41f640 1 --2- 172.21.15.84:0/1070929549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01680b0410 0x7f01680b1590 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f0164034040 tx=0x7f01640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:10.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.804+0000 7f015dffb640 1 -- 172.21.15.84:0/1070929549 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164032900 con 0x7f01680b0410 2023-10-16T10:04:10.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.804+0000 7f015dffb640 1 -- 172.21.15.84:0/1070929549 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0164032a80 con 0x7f01680b0410 2023-10-16T10:04:10.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.804+0000 7f015dffb640 1 -- 172.21.15.84:0/1070929549 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164032e30 con 0x7f01680b0410 2023-10-16T10:04:10.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.804+0000 7f016f683640 1 -- 172.21.15.84:0/1070929549 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01680c2940 con 0x7f01680b0410 2023-10-16T10:04:10.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.804+0000 7f016f683640 1 -- 172.21.15.84:0/1070929549 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0168159fb0 con 0x7f01680b0410 2023-10-16T10:04:10.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.805+0000 7f016f683640 1 -- 172.21.15.84:0/1070929549 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01680c24c0 con 0x7f01680b0410 2023-10-16T10:04:10.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.806+0000 7f015dffb640 1 -- 172.21.15.84:0/1070929549 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 31) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f016403b3f0 con 0x7f01680b0410 2023-10-16T10:04:10.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.807+0000 7f015dffb640 1 --2- 172.21.15.84:0/1070929549 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f014c06acc0 0x7f014c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.807+0000 7f015dffb640 1 -- 172.21.15.84:0/1070929549 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f01640ab6d0 con 0x7f01680b0410 2023-10-16T10:04:10.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.807+0000 7f016cc1e640 1 --2- 172.21.15.84:0/1070929549 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f014c06acc0 0x7f014c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.808+0000 7f016cc1e640 1 --2- 172.21.15.84:0/1070929549 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f014c06acc0 0x7f014c06d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f01680708d0 tx=0x7f0158009040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:10.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.811+0000 7f015dffb640 1 -- 172.21.15.84:0/1070929549 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f01680c24c0 con 0x7f01680b0410 2023-10-16T10:04:10.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:10.986+0000 7f015dffb640 1 -- 172.21.15.84:0/1070929549 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 32) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f016403bd30 con 0x7f01680b0410 2023-10-16T10:04:11.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.048+0000 7f016f683640 1 -- 172.21.15.84:0/1070929549 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01680c24c0 con 0x7f01680b0410 2023-10-16T10:04:11.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.049+0000 7f015dffb640 1 -- 172.21.15.84:0/1070929549 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f01680c24c0 con 0x7f01680b0410 2023-10-16T10:04:11.051 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:11.051 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3040499154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3040499154},{"type":"v1","addr":"172.21.15.157:6835","nonce":3040499154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2727537766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2727537766},{"type":"v1","addr":"172.21.15.157:6833","nonce":2727537766}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/49292950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":49292950},{"type":"v1","addr":"172.21.15.84:6836","nonce":49292950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:02:13.514054+0000","modified":"2023-10-16T10:02:16.598055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/3839442164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3839442164},{"type":"v1","addr":"172.21.15.84:6837","nonce":3839442164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T10:04:11.051 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 25 2023-10-16T10:04:11.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.052+0000 7f016f683640 1 -- 172.21.15.84:0/1070929549 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f014c06acc0 msgr2=0x7f014c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.052+0000 7f016f683640 1 --2- 172.21.15.84:0/1070929549 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f014c06acc0 0x7f014c06d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f01680708d0 tx=0x7f0158009040 comp rx=0 tx=0).stop 2023-10-16T10:04:11.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.052+0000 7f016f683640 1 -- 172.21.15.84:0/1070929549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01680b0410 msgr2=0x7f01680b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.052+0000 7f016f683640 1 --2- 172.21.15.84:0/1070929549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01680b0410 0x7f01680b1590 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f0164034040 tx=0x7f01640046c0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.052+0000 7f016f683640 1 -- 172.21.15.84:0/1070929549 shutdown_connections 2023-10-16T10:04:11.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.052+0000 7f016f683640 1 --2- 172.21.15.84:0/1070929549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f016815db00 0x7f0168156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.052+0000 7f016f683640 1 --2- 172.21.15.84:0/1070929549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01680cd0c0 0x7f0168159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.052+0000 7f016f683640 1 --2- 172.21.15.84:0/1070929549 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f014c06acc0 0x7f014c06d180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.052+0000 7f016f683640 1 --2- 172.21.15.84:0/1070929549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01680b0410 0x7f01680b1590 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.052+0000 7f016f683640 1 -- 172.21.15.84:0/1070929549 >> 172.21.15.84:0/1070929549 conn(0x7f01680cb410 msgr2=0x7f01680c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:11.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.052+0000 7f016f683640 1 -- 172.21.15.84:0/1070929549 shutdown_connections 2023-10-16T10:04:11.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.052+0000 7f016f683640 1 -- 172.21.15.84:0/1070929549 wait complete. 2023-10-16T10:04:11.072 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:04:11.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.277+0000 7f9f5f49f640 1 Processor -- start 2023-10-16T10:04:11.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.277+0000 7f9f5f49f640 1 -- start start 2023-10-16T10:04:11.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.278+0000 7f9f5f49f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9f5815b320 0x7f9f580cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.278+0000 7f9f5f49f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5815b9d0 con 0x7f9f5806c040 2023-10-16T10:04:11.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.279+0000 7f9f5f49f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5815bcb0 con 0x7f9f580ad970 2023-10-16T10:04:11.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.279+0000 7f9f5f49f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f5815b7d0 con 0x7f9f5815b320 2023-10-16T10:04:11.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.279+0000 7f9f5e49d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9f5806c040 0x7f9f5806c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55690/0 (socket says 172.21.15.84:55690) 2023-10-16T10:04:11.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.279+0000 7f9f5e49d640 1 -- 172.21.15.84:0/2815867458 learned_addr learned my addr 172.21.15.84:0/2815867458 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:11.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.279+0000 7f9f5ec9e640 1 --2- 172.21.15.84:0/2815867458 >> v2:172.21.15.84:3300/0 conn(0x7f9f5815b320 0x7f9f580cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.279+0000 7f9f5ec9e640 1 -- 172.21.15.84:0/2815867458 >> v1:172.21.15.157:6789/0 conn(0x7f9f580ad970 legacy=0x7f9f580cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:11.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.279+0000 7f9f5ec9e640 1 -- 172.21.15.84:0/2815867458 >> v1:172.21.15.84:6789/0 conn(0x7f9f5806c040 legacy=0x7f9f5806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.279+0000 7f9f5ec9e640 1 -- 172.21.15.84:0/2815867458 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5815bf90 con 0x7f9f5815b320 2023-10-16T10:04:11.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.280+0000 7f9f5ec9e640 1 --2- 172.21.15.84:0/2815867458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f5815b320 0x7f9f580cd180 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009dc0 tx=0x7f9f5002be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8611c44958966d2a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:11.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.281+0000 7f9f5d49b640 1 -- 172.21.15.84:0/2815867458 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5002cc40 con 0x7f9f5815b320 2023-10-16T10:04:11.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.281+0000 7f9f5d49b640 1 -- 172.21.15.84:0/2815867458 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5002cda0 con 0x7f9f5815b320 2023-10-16T10:04:11.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.281+0000 7f9f5d49b640 1 -- 172.21.15.84:0/2815867458 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50034660 con 0x7f9f5815b320 2023-10-16T10:04:11.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.281+0000 7f9f5f49f640 1 -- 172.21.15.84:0/2815867458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f5815b320 msgr2=0x7f9f580cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.281+0000 7f9f5f49f640 1 --2- 172.21.15.84:0/2815867458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f5815b320 0x7f9f580cd180 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009dc0 tx=0x7f9f5002be90 comp rx=0 tx=0).stop 2023-10-16T10:04:11.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.281+0000 7f9f5f49f640 1 -- 172.21.15.84:0/2815867458 shutdown_connections 2023-10-16T10:04:11.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.281+0000 7f9f5f49f640 1 --2- 172.21.15.84:0/2815867458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f5815b320 0x7f9f580cd180 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.281+0000 7f9f5f49f640 1 -- 172.21.15.84:0/2815867458 >> 172.21.15.84:0/2815867458 conn(0x7f9f580ce6a0 msgr2=0x7f9f580ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:11.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.281+0000 7f9f5f49f640 1 -- 172.21.15.84:0/2815867458 shutdown_connections 2023-10-16T10:04:11.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.281+0000 7f9f5f49f640 1 -- 172.21.15.84:0/2815867458 wait complete. 2023-10-16T10:04:11.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.283+0000 7f9f5f49f640 1 Processor -- start 2023-10-16T10:04:11.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.283+0000 7f9f5f49f640 1 -- start start 2023-10-16T10:04:11.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.284+0000 7f9f5f49f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f5806c040 0x7f9f581f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.284+0000 7f9f5f49f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f580ad970 0x7f9f5814ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.284+0000 7f9f5f49f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f5815b320 0x7f9f580c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.284+0000 7f9f5f49f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f580c9c40 con 0x7f9f5815b320 2023-10-16T10:04:11.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.284+0000 7f9f5f49f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581f2d20 con 0x7f9f580ad970 2023-10-16T10:04:11.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.284+0000 7f9f5f49f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581f3000 con 0x7f9f5806c040 2023-10-16T10:04:11.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.284+0000 7f9f5ec9e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f5815b320 0x7f9f580c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.284+0000 7f9f5ec9e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f5815b320 0x7f9f580c9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37708/0 (socket says 172.21.15.84:37708) 2023-10-16T10:04:11.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.284+0000 7f9f5ec9e640 1 -- 172.21.15.84:0/2897789853 learned_addr learned my addr 172.21.15.84:0/2897789853 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:11.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.285+0000 7f9f5e49d640 1 --2- 172.21.15.84:0/2897789853 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f5806c040 0x7f9f581f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.285+0000 7f9f5dc9c640 1 --2- 172.21.15.84:0/2897789853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f580ad970 0x7f9f5814ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.285+0000 7f9f5ec9e640 1 -- 172.21.15.84:0/2897789853 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f5806c040 msgr2=0x7f9f581f26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.285+0000 7f9f5ec9e640 1 --2- 172.21.15.84:0/2897789853 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f5806c040 0x7f9f581f26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.285+0000 7f9f5ec9e640 1 -- 172.21.15.84:0/2897789853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f580ad970 msgr2=0x7f9f5814ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.285+0000 7f9f5ec9e640 1 --2- 172.21.15.84:0/2897789853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f580ad970 0x7f9f5814ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.285+0000 7f9f5ec9e640 1 -- 172.21.15.84:0/2897789853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f50009a70 con 0x7f9f5815b320 2023-10-16T10:04:11.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.285+0000 7f9f5ec9e640 1 --2- 172.21.15.84:0/2897789853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f5815b320 0x7f9f580c9700 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9f50037040 tx=0x7f9f50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:11.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.286+0000 7f9f4effd640 1 -- 172.21.15.84:0/2897789853 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5003e610 con 0x7f9f5815b320 2023-10-16T10:04:11.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.286+0000 7f9f4effd640 1 -- 172.21.15.84:0/2897789853 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5003e770 con 0x7f9f5815b320 2023-10-16T10:04:11.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.286+0000 7f9f4effd640 1 -- 172.21.15.84:0/2897789853 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5003eb00 con 0x7f9f5815b320 2023-10-16T10:04:11.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.286+0000 7f9f5f49f640 1 -- 172.21.15.84:0/2897789853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f581f32e0 con 0x7f9f5815b320 2023-10-16T10:04:11.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.286+0000 7f9f5f49f640 1 -- 172.21.15.84:0/2897789853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f581f35c0 con 0x7f9f5815b320 2023-10-16T10:04:11.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.287+0000 7f9f5f49f640 1 -- 172.21.15.84:0/2897789853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5806b330 con 0x7f9f5815b320 2023-10-16T10:04:11.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.288+0000 7f9f4effd640 1 -- 172.21.15.84:0/2897789853 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 32) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f9f5003b050 con 0x7f9f5815b320 2023-10-16T10:04:11.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.289+0000 7f9f4effd640 1 --2- 172.21.15.84:0/2897789853 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f9f3c06af30 0x7f9f3c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.289+0000 7f9f4effd640 1 -- 172.21.15.84:0/2897789853 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f9f500ac0f0 con 0x7f9f5815b320 2023-10-16T10:04:11.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.289+0000 7f9f5e49d640 1 --2- 172.21.15.84:0/2897789853 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f9f3c06af30 0x7f9f3c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.290+0000 7f9f5e49d640 1 --2- 172.21.15.84:0/2897789853 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f9f3c06af30 0x7f9f3c06d3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9f54005880 tx=0x7f9f54009040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:11.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.293+0000 7f9f4effd640 1 -- 172.21.15.84:0/2897789853 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9f50076e30 con 0x7f9f5815b320 2023-10-16T10:04:11.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.485+0000 7f9f5f49f640 1 -- 172.21.15.84:0/2897789853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f580ce120 con 0x7f9f5815b320 2023-10-16T10:04:11.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.487+0000 7f9f4effd640 1 -- 172.21.15.84:0/2897789853 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+17701 (secure 0 0 0) 0x7f9f50039090 con 0x7f9f5815b320 2023-10-16T10:04:11.489 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:11.489 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":69,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:04:08.229141+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:02:11.266402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-16T10:02:12.562100+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":68,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.13","mappings":[{"from":0,"to":5}]},{"pgid":"7.2d","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6832/633652932":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/724237746":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/3806126461":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/3926486342":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/3656680984":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/2816959180":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/4150953381":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/351541199":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/555434621":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/3440750653":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/3623389468":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/3654971153":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/621840141":"2023-10-17T10:02:27.969914+0000","172.21.15.84:6833/585186562":"2023-10-17T10:02:27.969914+0000","172.21.15.84:6833/633652932":"2023-10-17T10:04:08.229109+0000","172.21.15.84:6832/585186562":"2023-10-17T10:02:27.969914+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:04:11.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.490+0000 7f9f5f49f640 1 -- 172.21.15.84:0/2897789853 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f9f3c06af30 msgr2=0x7f9f3c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.490+0000 7f9f5f49f640 1 --2- 172.21.15.84:0/2897789853 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f9f3c06af30 0x7f9f3c06d3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9f54005880 tx=0x7f9f54009040 comp rx=0 tx=0).stop 2023-10-16T10:04:11.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.490+0000 7f9f5f49f640 1 -- 172.21.15.84:0/2897789853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f5815b320 msgr2=0x7f9f580c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.490+0000 7f9f5f49f640 1 --2- 172.21.15.84:0/2897789853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f5815b320 0x7f9f580c9700 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9f50037040 tx=0x7f9f50004a40 comp rx=0 tx=0).stop 2023-10-16T10:04:11.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.490+0000 7f9f5f49f640 1 -- 172.21.15.84:0/2897789853 shutdown_connections 2023-10-16T10:04:11.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.490+0000 7f9f5f49f640 1 --2- 172.21.15.84:0/2897789853 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f9f3c06af30 0x7f9f3c06d3f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.490+0000 7f9f5f49f640 1 --2- 172.21.15.84:0/2897789853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f5815b320 0x7f9f580c9700 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.490+0000 7f9f5f49f640 1 --2- 172.21.15.84:0/2897789853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f580ad970 0x7f9f5814ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.490+0000 7f9f5f49f640 1 --2- 172.21.15.84:0/2897789853 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f5806c040 0x7f9f581f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.490+0000 7f9f5f49f640 1 -- 172.21.15.84:0/2897789853 >> 172.21.15.84:0/2897789853 conn(0x7f9f580ce6a0 msgr2=0x7f9f5815db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:11.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.490+0000 7f9f5f49f640 1 -- 172.21.15.84:0/2897789853 shutdown_connections 2023-10-16T10:04:11.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.490+0000 7f9f5f49f640 1 -- 172.21.15.84:0/2897789853 wait complete. 2023-10-16T10:04:11.511 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T10:04:11.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.725+0000 7efe15059640 1 Processor -- start 2023-10-16T10:04:11.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.725+0000 7efe15059640 1 -- start start 2023-10-16T10:04:11.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.727+0000 7efe15059640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe1006c040 0x7efe1006c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.727+0000 7efe15059640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe1015f4a0 con 0x7efe100cda80 2023-10-16T10:04:11.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.727+0000 7efe15059640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe1006c960 con 0x7efe100ad970 2023-10-16T10:04:11.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.727+0000 7efe15059640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1015b9d0 con 0x7efe1006c040 2023-10-16T10:04:11.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.727+0000 7efe14858640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7efe100cda80 0x7efe1015ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55706/0 (socket says 172.21.15.84:55706) 2023-10-16T10:04:11.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.727+0000 7efe14858640 1 -- 172.21.15.84:0/3347624243 learned_addr learned my addr 172.21.15.84:0/3347624243 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:11.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.728+0000 7efe0ffff640 1 --2- 172.21.15.84:0/3347624243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe1006c040 0x7efe1006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.728+0000 7efe0f7fe640 1 -- 172.21.15.84:0/3347624243 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036588675 0 0) 0x7efe1015f4a0 con 0x7efe100cda80 2023-10-16T10:04:11.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.728+0000 7efe0f7fe640 1 -- 172.21.15.84:0/3347624243 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdfc003520 con 0x7efe100cda80 2023-10-16T10:04:11.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.728+0000 7efe0f7fe640 1 -- 172.21.15.84:0/3347624243 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4134964768 0 0) 0x7efdfc003520 con 0x7efe100cda80 2023-10-16T10:04:11.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.728+0000 7efe0f7fe640 1 -- 172.21.15.84:0/3347624243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe1006c040 msgr2=0x7efe1006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:11.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.728+0000 7efe0f7fe640 1 --2- 172.21.15.84:0/3347624243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe1006c040 0x7efe1006c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.728+0000 7efe0f7fe640 1 -- 172.21.15.84:0/3347624243 >> v1:172.21.15.157:6789/0 conn(0x7efe100ad970 legacy=0x7efe1015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.728+0000 7efe0f7fe640 1 -- 172.21.15.84:0/3347624243 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1015bcb0 con 0x7efe100cda80 2023-10-16T10:04:11.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.728+0000 7efe0f7fe640 1 -- 172.21.15.84:0/3347624243 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7efe080019f0 con 0x7efe100cda80 2023-10-16T10:04:11.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.728+0000 7efe0f7fe640 1 -- 172.21.15.84:0/3347624243 >> v1:172.21.15.84:6789/0 conn(0x7efe100cda80 legacy=0x7efe1015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.728+0000 7efe0f7fe640 1 --2- 172.21.15.84:0/3347624243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdfc0039a0 0x7efdfc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.728+0000 7efe0f7fe640 1 --2- 172.21.15.84:0/3347624243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe1006c040 0x7efdfc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.728+0000 7efe0f7fe640 1 --2- 172.21.15.84:0/3347624243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdfc004d00 0x7efdfc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.728+0000 7efe0f7fe640 1 -- 172.21.15.84:0/3347624243 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1015b9d0 con 0x7efdfc0039a0 2023-10-16T10:04:11.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.728+0000 7efe0f7fe640 1 -- 172.21.15.84:0/3347624243 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc00ba30 con 0x7efdfc004d00 2023-10-16T10:04:11.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.728+0000 7efe0f7fe640 1 -- 172.21.15.84:0/3347624243 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc00bbf0 con 0x7efe1006c040 2023-10-16T10:04:11.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.729+0000 7efe0ffff640 1 --2- 172.21.15.84:0/3347624243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdfc0039a0 0x7efdfc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.729+0000 7efe07fff640 1 --2- 172.21.15.84:0/3347624243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe1006c040 0x7efdfc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.729+0000 7efe14858640 1 --2- 172.21.15.84:0/3347624243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdfc004d00 0x7efdfc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.729+0000 7efe0ffff640 1 -- 172.21.15.84:0/3347624243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe1006c040 msgr2=0x7efdfc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:11.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.729+0000 7efe0ffff640 1 --2- 172.21.15.84:0/3347624243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe1006c040 0x7efdfc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.729+0000 7efe0ffff640 1 -- 172.21.15.84:0/3347624243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdfc004d00 msgr2=0x7efdfc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:11.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.729+0000 7efe0ffff640 1 --2- 172.21.15.84:0/3347624243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdfc004d00 0x7efdfc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.729+0000 7efe0ffff640 1 -- 172.21.15.84:0/3347624243 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc003520 con 0x7efdfc0039a0 2023-10-16T10:04:11.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.730+0000 7efe0ffff640 1 --2- 172.21.15.84:0/3347624243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdfc0039a0 0x7efdfc003dc0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7efe00009a90 tx=0x7efdfc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf2e0d37d42ee9c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:11.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.730+0000 7efe0f7fe640 1 -- 172.21.15.84:0/3347624243 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0002c680 con 0x7efdfc0039a0 2023-10-16T10:04:11.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.730+0000 7efe0f7fe640 1 -- 172.21.15.84:0/3347624243 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0002c7e0 con 0x7efdfc0039a0 2023-10-16T10:04:11.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.730+0000 7efe0f7fe640 1 -- 172.21.15.84:0/3347624243 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0002cb70 con 0x7efdfc0039a0 2023-10-16T10:04:11.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.731+0000 7efe15059640 1 -- 172.21.15.84:0/3347624243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdfc0039a0 msgr2=0x7efdfc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.731+0000 7efe15059640 1 --2- 172.21.15.84:0/3347624243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdfc0039a0 0x7efdfc003dc0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7efe00009a90 tx=0x7efdfc0057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.731+0000 7efe14858640 1 -- 172.21.15.84:0/3347624243 reap_dead start 2023-10-16T10:04:11.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.731+0000 7efe15059640 1 -- 172.21.15.84:0/3347624243 shutdown_connections 2023-10-16T10:04:11.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.731+0000 7efe15059640 1 -- 172.21.15.84:0/3347624243 >> 172.21.15.84:0/3347624243 conn(0x7efe100ce6a0 msgr2=0x7efe100ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:11.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.731+0000 7efe15059640 1 -- 172.21.15.84:0/3347624243 shutdown_connections 2023-10-16T10:04:11.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.731+0000 7efe15059640 1 -- 172.21.15.84:0/3347624243 wait complete. 2023-10-16T10:04:11.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.733+0000 7efe15059640 1 Processor -- start 2023-10-16T10:04:11.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.733+0000 7efe15059640 1 -- start start 2023-10-16T10:04:11.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.734+0000 7efe15059640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdfc0039a0 0x7efe100c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.734+0000 7efe15059640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe100b2b50 0x7efe100acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.734+0000 7efe15059640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe101f1bb0 0x7efe101f2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.734+0000 7efe15059640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe100c9b40 con 0x7efe101f1bb0 2023-10-16T10:04:11.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.734+0000 7efe15059640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe101f2570 con 0x7efe100b2b50 2023-10-16T10:04:11.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.734+0000 7efe15059640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe100c9de0 con 0x7efdfc0039a0 2023-10-16T10:04:11.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.734+0000 7efe14858640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe101f1bb0 0x7efe101f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.735+0000 7efe14858640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe101f1bb0 0x7efe101f2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37722/0 (socket says 172.21.15.84:37722) 2023-10-16T10:04:11.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.735+0000 7efe14858640 1 -- 172.21.15.84:0/2960033355 learned_addr learned my addr 172.21.15.84:0/2960033355 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:11.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.735+0000 7efe0ffff640 1 --2- 172.21.15.84:0/2960033355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdfc0039a0 0x7efe100c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.735+0000 7efe07fff640 1 --2- 172.21.15.84:0/2960033355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe100b2b50 0x7efe100acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.735+0000 7efe14858640 1 -- 172.21.15.84:0/2960033355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdfc0039a0 msgr2=0x7efe100c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.735+0000 7efe14858640 1 --2- 172.21.15.84:0/2960033355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdfc0039a0 0x7efe100c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.735+0000 7efe14858640 1 -- 172.21.15.84:0/2960033355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe100b2b50 msgr2=0x7efe100acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.735+0000 7efe14858640 1 --2- 172.21.15.84:0/2960033355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe100b2b50 0x7efe100acb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.735+0000 7efe14858640 1 -- 172.21.15.84:0/2960033355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe100ca0c0 con 0x7efe101f1bb0 2023-10-16T10:04:11.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.735+0000 7efe07fff640 1 --2- 172.21.15.84:0/2960033355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe100b2b50 0x7efe100acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:11.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.735+0000 7efe14858640 1 --2- 172.21.15.84:0/2960033355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe101f1bb0 0x7efe101f2030 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7efe100b11c0 tx=0x7efe0800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:11.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.736+0000 7efe0d7fa640 1 -- 172.21.15.84:0/2960033355 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0800ec10 con 0x7efe101f1bb0 2023-10-16T10:04:11.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.736+0000 7efe0d7fa640 1 -- 172.21.15.84:0/2960033355 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0800ed70 con 0x7efe101f1bb0 2023-10-16T10:04:11.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.736+0000 7efe0d7fa640 1 -- 172.21.15.84:0/2960033355 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0800eee0 con 0x7efe101f1bb0 2023-10-16T10:04:11.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.736+0000 7efe15059640 1 -- 172.21.15.84:0/2960033355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe100ca3a0 con 0x7efe101f1bb0 2023-10-16T10:04:11.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.736+0000 7efe15059640 1 -- 172.21.15.84:0/2960033355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe100caba0 con 0x7efe101f1bb0 2023-10-16T10:04:11.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.737+0000 7efe15059640 1 -- 172.21.15.84:0/2960033355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe1006b330 con 0x7efe101f1bb0 2023-10-16T10:04:11.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.738+0000 7efe0d7fa640 1 -- 172.21.15.84:0/2960033355 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 32) v1 ==== 89275+0+0 (secure 0 0 0) 0x7efe08013640 con 0x7efe101f1bb0 2023-10-16T10:04:11.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.739+0000 7efe0d7fa640 1 --2- 172.21.15.84:0/2960033355 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7efdf006af00 0x7efdf006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.739+0000 7efe0d7fa640 1 -- 172.21.15.84:0/2960033355 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6483+0+0 (secure 0 0 0) 0x7efe08089f10 con 0x7efe101f1bb0 2023-10-16T10:04:11.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.739+0000 7efe0ffff640 1 --2- 172.21.15.84:0/2960033355 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7efdf006af00 0x7efdf006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.740+0000 7efe0ffff640 1 --2- 172.21.15.84:0/2960033355 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7efdf006af00 0x7efdf006d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efe00036000 tx=0x7efe00030000 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:11.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.743+0000 7efe0d7fa640 1 -- 172.21.15.84:0/2960033355 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efe08058650 con 0x7efe101f1bb0 2023-10-16T10:04:11.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.958+0000 7efe15059640 1 -- 172.21.15.84:0/2960033355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efe1004e9b0 con 0x7efe101f1bb0 2023-10-16T10:04:11.961 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-16T10:04:11.960+0000 7fca0c7ff640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T10:04:11.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.963+0000 7efe0d7fa640 1 -- 172.21.15.84:0/2960033355 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7efe080581e0 con 0x7efe101f1bb0 2023-10-16T10:04:11.965 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T10:04:11.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.966+0000 7efe15059640 1 -- 172.21.15.84:0/2960033355 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7efdf006af00 msgr2=0x7efdf006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.966+0000 7efe15059640 1 --2- 172.21.15.84:0/2960033355 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7efdf006af00 0x7efdf006d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efe00036000 tx=0x7efe00030000 comp rx=0 tx=0).stop 2023-10-16T10:04:11.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.967+0000 7efe15059640 1 -- 172.21.15.84:0/2960033355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe101f1bb0 msgr2=0x7efe101f2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.967+0000 7efe15059640 1 --2- 172.21.15.84:0/2960033355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe101f1bb0 0x7efe101f2030 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7efe100b11c0 tx=0x7efe0800e530 comp rx=0 tx=0).stop 2023-10-16T10:04:11.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.967+0000 7efe15059640 1 -- 172.21.15.84:0/2960033355 shutdown_connections 2023-10-16T10:04:11.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.967+0000 7efe15059640 1 --2- 172.21.15.84:0/2960033355 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7efdf006af00 0x7efdf006d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.967+0000 7efe15059640 1 --2- 172.21.15.84:0/2960033355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe101f1bb0 0x7efe101f2030 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.967+0000 7efe15059640 1 --2- 172.21.15.84:0/2960033355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe100b2b50 0x7efe100acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.967+0000 7efe15059640 1 --2- 172.21.15.84:0/2960033355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdfc0039a0 0x7efe100c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.967+0000 7efe15059640 1 -- 172.21.15.84:0/2960033355 >> 172.21.15.84:0/2960033355 conn(0x7efe100ce6a0 msgr2=0x7efe1015c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:11.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.967+0000 7efe15059640 1 -- 172.21.15.84:0/2960033355 shutdown_connections 2023-10-16T10:04:11.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:11.967+0000 7efe15059640 1 -- 172.21.15.84:0/2960033355 wait complete. 2023-10-16T10:04:11.989 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T10:04:12.042 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T10:04:12.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.176+0000 7fe147dd5640 1 Processor -- start 2023-10-16T10:04:12.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.176+0000 7fe147dd5640 1 -- start start 2023-10-16T10:04:12.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.178+0000 7fe147dd5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe140159ab0 0x7fe1400ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:12.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.178+0000 7fe147dd5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe140159e90 0x7fe14006fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:12.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.178+0000 7fe147dd5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1400cd1f0 con 0x7fe14006c140 2023-10-16T10:04:12.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.178+0000 7fe147dd5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe14015d890 con 0x7fe140159e90 2023-10-16T10:04:12.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.178+0000 7fe147dd5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1400cd4d0 con 0x7fe140159ab0 2023-10-16T10:04:12.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.178+0000 7fe146372640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe14006c140 0x7fe14015dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55724/0 (socket says 172.21.15.84:55724) 2023-10-16T10:04:12.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.178+0000 7fe146372640 1 -- 172.21.15.84:0/2237937591 learned_addr learned my addr 172.21.15.84:0/2237937591 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:12.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.178+0000 7fe145370640 1 --2- 172.21.15.84:0/2237937591 >> v2:172.21.15.84:3300/0 conn(0x7fe140159e90 0x7fe14006fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:12.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.178+0000 7fe145b71640 1 --2- 172.21.15.84:0/2237937591 >> v2:172.21.15.157:3300/0 conn(0x7fe140159ab0 0x7fe1400ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:12.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.178+0000 7fe144b6f640 1 -- 172.21.15.84:0/2237937591 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2870610192 0 0) 0x7fe1400cd1f0 con 0x7fe14006c140 2023-10-16T10:04:12.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.179+0000 7fe144b6f640 1 -- 172.21.15.84:0/2237937591 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe12c003520 con 0x7fe14006c140 2023-10-16T10:04:12.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.179+0000 7fe145370640 1 -- 172.21.15.84:0/2237937591 >> v2:172.21.15.157:3300/0 conn(0x7fe140159ab0 msgr2=0x7fe1400ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:12.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.179+0000 7fe145370640 1 --2- 172.21.15.84:0/2237937591 >> v2:172.21.15.157:3300/0 conn(0x7fe140159ab0 0x7fe1400ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:12.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.179+0000 7fe145370640 1 -- 172.21.15.84:0/2237937591 >> v1:172.21.15.84:6789/0 conn(0x7fe14006c140 legacy=0x7fe14015dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:12.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.179+0000 7fe145370640 1 -- 172.21.15.84:0/2237937591 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1400cd7b0 con 0x7fe140159e90 2023-10-16T10:04:12.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.180+0000 7fe145370640 1 --2- 172.21.15.84:0/2237937591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe140159e90 0x7fe14006fc50 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe1400cfa00 tx=0x7fe13002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f1c1fc1a27b59b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:12.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.180+0000 7fe144b6f640 1 -- 172.21.15.84:0/2237937591 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130031710 con 0x7fe140159e90 2023-10-16T10:04:12.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.180+0000 7fe144b6f640 1 -- 172.21.15.84:0/2237937591 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13002f050 con 0x7fe140159e90 2023-10-16T10:04:12.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.180+0000 7fe144b6f640 1 -- 172.21.15.84:0/2237937591 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130031a90 con 0x7fe140159e90 2023-10-16T10:04:12.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.180+0000 7fe147dd5640 1 -- 172.21.15.84:0/2237937591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe140159e90 msgr2=0x7fe14006fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:12.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.180+0000 7fe147dd5640 1 --2- 172.21.15.84:0/2237937591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe140159e90 0x7fe14006fc50 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe1400cfa00 tx=0x7fe13002b7b0 comp rx=0 tx=0).stop 2023-10-16T10:04:12.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.181+0000 7fe147dd5640 1 -- 172.21.15.84:0/2237937591 shutdown_connections 2023-10-16T10:04:12.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.181+0000 7fe147dd5640 1 --2- 172.21.15.84:0/2237937591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe140159e90 0x7fe14006fc50 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:12.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.181+0000 7fe147dd5640 1 --2- 172.21.15.84:0/2237937591 >> v2:172.21.15.157:3300/0 conn(0x7fe140159ab0 0x7fe1400ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:12.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.181+0000 7fe147dd5640 1 -- 172.21.15.84:0/2237937591 >> 172.21.15.84:0/2237937591 conn(0x7fe1400ce9b0 msgr2=0x7fe1400cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:12.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.181+0000 7fe147dd5640 1 -- 172.21.15.84:0/2237937591 shutdown_connections 2023-10-16T10:04:12.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.181+0000 7fe147dd5640 1 -- 172.21.15.84:0/2237937591 wait complete. 2023-10-16T10:04:12.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.182+0000 7fe147dd5640 1 Processor -- start 2023-10-16T10:04:12.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.182+0000 7fe147dd5640 1 -- start start 2023-10-16T10:04:12.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.183+0000 7fe147dd5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe14006c140 0x7fe14014fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:12.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.183+0000 7fe147dd5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe140159ab0 0x7fe1400c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:12.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.183+0000 7fe147dd5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe140159e90 0x7fe1400ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:12.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.183+0000 7fe147dd5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1401e29f0 con 0x7fe140159e90 2023-10-16T10:04:12.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.183+0000 7fe147dd5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1400c84f0 con 0x7fe140159ab0 2023-10-16T10:04:12.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.183+0000 7fe147dd5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1400c87d0 con 0x7fe14006c140 2023-10-16T10:04:12.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.184+0000 7fe146372640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe140159e90 0x7fe1400ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:12.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.184+0000 7fe146372640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe140159e90 0x7fe1400ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37766/0 (socket says 172.21.15.84:37766) 2023-10-16T10:04:12.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.184+0000 7fe146372640 1 -- 172.21.15.84:0/4289545868 learned_addr learned my addr 172.21.15.84:0/4289545868 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:12.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.184+0000 7fe146372640 1 -- 172.21.15.84:0/4289545868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe14006c140 msgr2=0x7fe14014fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:12.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.184+0000 7fe146372640 1 --2- 172.21.15.84:0/4289545868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe14006c140 0x7fe14014fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:12.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.184+0000 7fe146372640 1 -- 172.21.15.84:0/4289545868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe140159ab0 msgr2=0x7fe1400c24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:12.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.184+0000 7fe146372640 1 --2- 172.21.15.84:0/4289545868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe140159ab0 0x7fe1400c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:12.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.184+0000 7fe146372640 1 -- 172.21.15.84:0/4289545868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe130009400 con 0x7fe140159e90 2023-10-16T10:04:12.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.184+0000 7fe146372640 1 --2- 172.21.15.84:0/4289545868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe140159e90 0x7fe1400ac890 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe13800cb40 tx=0x7fe13800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:12.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.185+0000 7fe1367fc640 1 -- 172.21.15.84:0/4289545868 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13800aca0 con 0x7fe140159e90 2023-10-16T10:04:12.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.185+0000 7fe1367fc640 1 -- 172.21.15.84:0/4289545868 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13800ae00 con 0x7fe140159e90 2023-10-16T10:04:12.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.185+0000 7fe1367fc640 1 -- 172.21.15.84:0/4289545868 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138012620 con 0x7fe140159e90 2023-10-16T10:04:12.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.185+0000 7fe147dd5640 1 -- 172.21.15.84:0/4289545868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1401e2c90 con 0x7fe140159e90 2023-10-16T10:04:12.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.185+0000 7fe147dd5640 1 -- 172.21.15.84:0/4289545868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe140150270 con 0x7fe140159e90 2023-10-16T10:04:12.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.186+0000 7fe147dd5640 1 -- 172.21.15.84:0/4289545868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1401e31b0 con 0x7fe140159e90 2023-10-16T10:04:12.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.187+0000 7fe1367fc640 1 -- 172.21.15.84:0/4289545868 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 32) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fe138007070 con 0x7fe140159e90 2023-10-16T10:04:12.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.188+0000 7fe1367fc640 1 --2- 172.21.15.84:0/4289545868 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe12406af00 0x7fe12406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:12.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.188+0000 7fe1367fc640 1 -- 172.21.15.84:0/4289545868 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6569+0+0 (secure 0 0 0) 0x7fe138089a20 con 0x7fe140159e90 2023-10-16T10:04:12.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.188+0000 7fe145b71640 1 --2- 172.21.15.84:0/4289545868 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe12406af00 0x7fe12406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:12.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.189+0000 7fe145b71640 1 --2- 172.21.15.84:0/4289545868 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe12406af00 0x7fe12406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0098f0 tx=0x7fe13c008040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:12.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.192+0000 7fe1367fc640 1 -- 172.21.15.84:0/4289545868 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe1401e31b0 con 0x7fe140159e90 2023-10-16T10:04:12.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.423+0000 7fe147dd5640 1 -- 172.21.15.84:0/4289545868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe1401e31b0 con 0x7fe140159e90 2023-10-16T10:04:12.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.981+0000 7fe1367fc640 1 -- 172.21.15.84:0/4289545868 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe1401e31b0 con 0x7fe140159e90 2023-10-16T10:04:12.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.984+0000 7fe147dd5640 1 -- 172.21.15.84:0/4289545868 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe12406af00 msgr2=0x7fe12406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:12.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.984+0000 7fe147dd5640 1 --2- 172.21.15.84:0/4289545868 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe12406af00 0x7fe12406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0098f0 tx=0x7fe13c008040 comp rx=0 tx=0).stop 2023-10-16T10:04:12.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.984+0000 7fe147dd5640 1 -- 172.21.15.84:0/4289545868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe140159e90 msgr2=0x7fe1400ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:12.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.984+0000 7fe147dd5640 1 --2- 172.21.15.84:0/4289545868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe140159e90 0x7fe1400ac890 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe13800cb40 tx=0x7fe13800a060 comp rx=0 tx=0).stop 2023-10-16T10:04:12.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.984+0000 7fe147dd5640 1 -- 172.21.15.84:0/4289545868 shutdown_connections 2023-10-16T10:04:12.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.984+0000 7fe147dd5640 1 --2- 172.21.15.84:0/4289545868 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe12406af00 0x7fe12406d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:12.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.984+0000 7fe147dd5640 1 --2- 172.21.15.84:0/4289545868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe140159e90 0x7fe1400ac890 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:12.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.984+0000 7fe147dd5640 1 --2- 172.21.15.84:0/4289545868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe140159ab0 0x7fe1400c24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:12.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.984+0000 7fe147dd5640 1 --2- 172.21.15.84:0/4289545868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe14006c140 0x7fe14014fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:12.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.984+0000 7fe147dd5640 1 -- 172.21.15.84:0/4289545868 >> 172.21.15.84:0/4289545868 conn(0x7fe1400ce9b0 msgr2=0x7fe14006b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:12.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.985+0000 7fe147dd5640 1 -- 172.21.15.84:0/4289545868 shutdown_connections 2023-10-16T10:04:12.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:12.985+0000 7fe147dd5640 1 -- 172.21.15.84:0/4289545868 wait complete. 2023-10-16T10:04:13.013 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T10:04:13.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.223+0000 7fad7db6c640 1 Processor -- start 2023-10-16T10:04:13.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.223+0000 7fad7db6c640 1 -- start start 2023-10-16T10:04:13.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.224+0000 7fad7db6c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad780b0360 0x7fad780b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:13.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.225+0000 7fad7db6c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad780b0d00 con 0x7fad7815ac10 2023-10-16T10:04:13.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.225+0000 7fad7db6c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad7815b310 con 0x7fad780ca510 2023-10-16T10:04:13.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.225+0000 7fad7db6c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7815f800 con 0x7fad780b0360 2023-10-16T10:04:13.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.225+0000 7fad777fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fad7815ac10 0x7fad7806c730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55726/0 (socket says 172.21.15.84:55726) 2023-10-16T10:04:13.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.225+0000 7fad777fe640 1 -- 172.21.15.84:0/898588581 learned_addr learned my addr 172.21.15.84:0/898588581 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:13.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.225+0000 7fad6ffff640 1 --2- 172.21.15.84:0/898588581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad780b0360 0x7fad780b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:13.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.225+0000 7fad76ffd640 1 -- 172.21.15.84:0/898588581 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3279965550 0 0) 0x7fad780b0d00 con 0x7fad7815ac10 2023-10-16T10:04:13.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.225+0000 7fad76ffd640 1 -- 172.21.15.84:0/898588581 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad64003520 con 0x7fad7815ac10 2023-10-16T10:04:13.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.226+0000 7fad76ffd640 1 -- 172.21.15.84:0/898588581 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1240737812 0 0) 0x7fad64003520 con 0x7fad7815ac10 2023-10-16T10:04:13.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.226+0000 7fad76ffd640 1 -- 172.21.15.84:0/898588581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad780b0360 msgr2=0x7fad780b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:13.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.226+0000 7fad76ffd640 1 --2- 172.21.15.84:0/898588581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad780b0360 0x7fad780b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:13.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.226+0000 7fad76ffd640 1 -- 172.21.15.84:0/898588581 >> v1:172.21.15.157:6789/0 conn(0x7fad780ca510 legacy=0x7fad780ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:13.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.226+0000 7fad76ffd640 1 -- 172.21.15.84:0/898588581 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7806f640 con 0x7fad7815ac10 2023-10-16T10:04:13.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.226+0000 7fad76ffd640 1 -- 172.21.15.84:0/898588581 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fad68001670 con 0x7fad7815ac10 2023-10-16T10:04:13.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.226+0000 7fad76ffd640 1 -- 172.21.15.84:0/898588581 >> v1:172.21.15.84:6789/0 conn(0x7fad7815ac10 legacy=0x7fad7806c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:13.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.226+0000 7fad76ffd640 1 --2- 172.21.15.84:0/898588581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad64003a10 0x7fad64003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:13.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.226+0000 7fad76ffd640 1 --2- 172.21.15.84:0/898588581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad640047c0 0x7fad64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:13.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.226+0000 7fad76ffd640 1 --2- 172.21.15.84:0/898588581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad780b0360 0x7fad6400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:13.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.226+0000 7fad76ffd640 1 -- 172.21.15.84:0/898588581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7815f800 con 0x7fad640047c0 2023-10-16T10:04:13.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.226+0000 7fad76ffd640 1 -- 172.21.15.84:0/898588581 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad640054e0 con 0x7fad64003a10 2023-10-16T10:04:13.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.226+0000 7fad76ffd640 1 -- 172.21.15.84:0/898588581 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6400bec0 con 0x7fad780b0360 2023-10-16T10:04:13.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.226+0000 7fad6ffff640 1 --2- 172.21.15.84:0/898588581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad640047c0 0x7fad64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:13.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.226+0000 7fad777fe640 1 --2- 172.21.15.84:0/898588581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad64003a10 0x7fad64003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:13.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.226+0000 7fad77fff640 1 --2- 172.21.15.84:0/898588581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad780b0360 0x7fad6400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:13.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.227+0000 7fad6ffff640 1 -- 172.21.15.84:0/898588581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad780b0360 msgr2=0x7fad6400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:13.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.227+0000 7fad6ffff640 1 --2- 172.21.15.84:0/898588581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad780b0360 0x7fad6400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:13.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.227+0000 7fad6ffff640 1 -- 172.21.15.84:0/898588581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad64003a10 msgr2=0x7fad64003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:13.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.227+0000 7fad6ffff640 1 --2- 172.21.15.84:0/898588581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad64003a10 0x7fad64003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:13.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.227+0000 7fad6ffff640 1 -- 172.21.15.84:0/898588581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad64003520 con 0x7fad640047c0 2023-10-16T10:04:13.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.227+0000 7fad77fff640 1 --2- 172.21.15.84:0/898588581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad780b0360 0x7fad6400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:13.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.228+0000 7fad6ffff640 1 --2- 172.21.15.84:0/898588581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad640047c0 0x7fad64004c40 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fad6000ba30 tx=0x7fad640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df79aba5dc089cd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:13.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.228+0000 7fad76ffd640 1 -- 172.21.15.84:0/898588581 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60032640 con 0x7fad640047c0 2023-10-16T10:04:13.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.228+0000 7fad76ffd640 1 -- 172.21.15.84:0/898588581 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad600327a0 con 0x7fad640047c0 2023-10-16T10:04:13.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.228+0000 7fad76ffd640 1 -- 172.21.15.84:0/898588581 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60032b30 con 0x7fad640047c0 2023-10-16T10:04:13.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.229+0000 7fad7db6c640 1 -- 172.21.15.84:0/898588581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad640047c0 msgr2=0x7fad64004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:13.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.229+0000 7fad7db6c640 1 --2- 172.21.15.84:0/898588581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad640047c0 0x7fad64004c40 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fad6000ba30 tx=0x7fad640057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:13.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.229+0000 7fad77fff640 1 -- 172.21.15.84:0/898588581 reap_dead start 2023-10-16T10:04:13.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.229+0000 7fad7db6c640 1 -- 172.21.15.84:0/898588581 shutdown_connections 2023-10-16T10:04:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.229+0000 7fad7db6c640 1 -- 172.21.15.84:0/898588581 >> 172.21.15.84:0/898588581 conn(0x7fad780cb360 msgr2=0x7fad780cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.229+0000 7fad7db6c640 1 -- 172.21.15.84:0/898588581 shutdown_connections 2023-10-16T10:04:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.229+0000 7fad7db6c640 1 -- 172.21.15.84:0/898588581 wait complete. 2023-10-16T10:04:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.230+0000 7fad7db6c640 1 Processor -- start 2023-10-16T10:04:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.231+0000 7fad7db6c640 1 -- start start 2023-10-16T10:04:13.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.231+0000 7fad7db6c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad640047c0 0x7fad781f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:13.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.232+0000 7fad7db6c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad781ee5c0 0x7fad78157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:13.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.232+0000 7fad7db6c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad78157860 0x7fad78070d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:13.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.232+0000 7fad7db6c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad781f2ab0 con 0x7fad78157860 2023-10-16T10:04:13.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.232+0000 7fad7db6c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad78156e90 con 0x7fad781ee5c0 2023-10-16T10:04:13.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.232+0000 7fad7db6c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad781eedf0 con 0x7fad640047c0 2023-10-16T10:04:13.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.232+0000 7fad77fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad78157860 0x7fad78070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:13.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.232+0000 7fad77fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad78157860 0x7fad78070d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37784/0 (socket says 172.21.15.84:37784) 2023-10-16T10:04:13.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.232+0000 7fad77fff640 1 -- 172.21.15.84:0/1098818831 learned_addr learned my addr 172.21.15.84:0/1098818831 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:13.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.232+0000 7fad777fe640 1 --2- 172.21.15.84:0/1098818831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad640047c0 0x7fad781f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:13.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.232+0000 7fad6ffff640 1 --2- 172.21.15.84:0/1098818831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad781ee5c0 0x7fad78157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:13.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.232+0000 7fad77fff640 1 -- 172.21.15.84:0/1098818831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad640047c0 msgr2=0x7fad781f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:13.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.232+0000 7fad77fff640 1 --2- 172.21.15.84:0/1098818831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad640047c0 0x7fad781f2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:13.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.232+0000 7fad77fff640 1 -- 172.21.15.84:0/1098818831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad781ee5c0 msgr2=0x7fad78157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:13.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.232+0000 7fad77fff640 1 --2- 172.21.15.84:0/1098818831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad781ee5c0 0x7fad78157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:13.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.232+0000 7fad77fff640 1 -- 172.21.15.84:0/1098818831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad781ef0d0 con 0x7fad78157860 2023-10-16T10:04:13.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.232+0000 7fad6ffff640 1 --2- 172.21.15.84:0/1098818831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad781ee5c0 0x7fad78157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:13.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.233+0000 7fad77fff640 1 --2- 172.21.15.84:0/1098818831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad78157860 0x7fad78070d50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fad78070570 tx=0x7fad7000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:13.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.233+0000 7fad74ff9640 1 -- 172.21.15.84:0/1098818831 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70006d50 con 0x7fad78157860 2023-10-16T10:04:13.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.233+0000 7fad74ff9640 1 -- 172.21.15.84:0/1098818831 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad70004910 con 0x7fad78157860 2023-10-16T10:04:13.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.233+0000 7fad74ff9640 1 -- 172.21.15.84:0/1098818831 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70004cc0 con 0x7fad78157860 2023-10-16T10:04:13.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.233+0000 7fad7db6c640 1 -- 172.21.15.84:0/1098818831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad781ef3b0 con 0x7fad78157860 2023-10-16T10:04:13.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.233+0000 7fad7db6c640 1 -- 172.21.15.84:0/1098818831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad781eea40 con 0x7fad78157860 2023-10-16T10:04:13.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.234+0000 7fad7db6c640 1 -- 172.21.15.84:0/1098818831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad78156c60 con 0x7fad78157860 2023-10-16T10:04:13.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.235+0000 7fad74ff9640 1 -- 172.21.15.84:0/1098818831 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fad70005030 con 0x7fad78157860 2023-10-16T10:04:13.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.236+0000 7fad74ff9640 1 --2- 172.21.15.84:0/1098818831 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fad5806af90 0x7fad5806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:13.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.236+0000 7fad74ff9640 1 -- 172.21.15.84:0/1098818831 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6569+0+0 (secure 0 0 0) 0x7fad70089d00 con 0x7fad78157860 2023-10-16T10:04:13.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.237+0000 7fad777fe640 1 --2- 172.21.15.84:0/1098818831 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fad5806af90 0x7fad5806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:13.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.237+0000 7fad777fe640 1 --2- 172.21.15.84:0/1098818831 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fad5806af90 0x7fad5806d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fad68005990 tx=0x7fad68009040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:13.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.241+0000 7fad74ff9640 1 -- 172.21.15.84:0/1098818831 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fad78156c60 con 0x7fad78157860 2023-10-16T10:04:13.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:13.433+0000 7fad7db6c640 1 -- 172.21.15.84:0/1098818831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fad78156c60 con 0x7fad78157860 2023-10-16T10:04:14.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.003+0000 7fad74ff9640 1 -- 172.21.15.84:0/1098818831 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v72) v1 ==== 181+0+0 (secure 0 0 0) 0x7fad78156c60 con 0x7fad78157860 2023-10-16T10:04:14.005 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-16T10:04:14.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.012+0000 7fad7db6c640 1 -- 172.21.15.84:0/1098818831 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fad5806af90 msgr2=0x7fad5806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:14.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.012+0000 7fad7db6c640 1 --2- 172.21.15.84:0/1098818831 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fad5806af90 0x7fad5806d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fad68005990 tx=0x7fad68009040 comp rx=0 tx=0).stop 2023-10-16T10:04:14.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.012+0000 7fad7db6c640 1 -- 172.21.15.84:0/1098818831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad78157860 msgr2=0x7fad78070d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:14.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.012+0000 7fad7db6c640 1 --2- 172.21.15.84:0/1098818831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad78157860 0x7fad78070d50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fad78070570 tx=0x7fad7000dc00 comp rx=0 tx=0).stop 2023-10-16T10:04:14.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.016+0000 7fad7db6c640 1 -- 172.21.15.84:0/1098818831 shutdown_connections 2023-10-16T10:04:14.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.016+0000 7fad7db6c640 1 --2- 172.21.15.84:0/1098818831 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fad5806af90 0x7fad5806d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:14.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.016+0000 7fad7db6c640 1 --2- 172.21.15.84:0/1098818831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad78157860 0x7fad78070d50 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:14.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.016+0000 7fad7db6c640 1 --2- 172.21.15.84:0/1098818831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad781ee5c0 0x7fad78157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:14.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.016+0000 7fad7db6c640 1 --2- 172.21.15.84:0/1098818831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad640047c0 0x7fad781f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:14.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.016+0000 7fad7db6c640 1 -- 172.21.15.84:0/1098818831 >> 172.21.15.84:0/1098818831 conn(0x7fad780cb360 msgr2=0x7fad78072f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:14.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.018+0000 7fad7db6c640 1 -- 172.21.15.84:0/1098818831 shutdown_connections 2023-10-16T10:04:14.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.018+0000 7fad7db6c640 1 -- 172.21.15.84:0/1098818831 wait complete. 2023-10-16T10:04:14.041 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T10:04:14.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.241+0000 7f56fd9b0640 1 Processor -- start 2023-10-16T10:04:14.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.242+0000 7f56fd9b0640 1 -- start start 2023-10-16T10:04:14.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.243+0000 7f56fd9b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56f80b8a10 0x7f56f80d2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:14.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.243+0000 7f56fd9b0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f56f80b4750 0x7f56f80b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:14.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.243+0000 7f56fd9b0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f80d1420 con 0x7f56f815cf50 2023-10-16T10:04:14.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.243+0000 7f56fd9b0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f80b5120 con 0x7f56f80b4750 2023-10-16T10:04:14.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.243+0000 7f56fd9b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f815d330 con 0x7f56f80b8a10 2023-10-16T10:04:14.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.243+0000 7f56f6ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f56f815cf50 0x7f56f80d0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55736/0 (socket says 172.21.15.84:55736) 2023-10-16T10:04:14.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.243+0000 7f56f6ffd640 1 -- 172.21.15.84:0/3395432055 learned_addr learned my addr 172.21.15.84:0/3395432055 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:14.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.243+0000 7f56f77fe640 1 --2- 172.21.15.84:0/3395432055 >> v2:172.21.15.84:3300/0 conn(0x7f56f80b4750 0x7f56f80b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:14.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.244+0000 7f56f5ffb640 1 -- 172.21.15.84:0/3395432055 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831303901 0 0) 0x7f56f80d1420 con 0x7f56f815cf50 2023-10-16T10:04:14.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.244+0000 7f56f67fc640 1 --2- 172.21.15.84:0/3395432055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56f80b8a10 0x7f56f80d2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:14.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.244+0000 7f56f5ffb640 1 -- 172.21.15.84:0/3395432055 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56e4003520 con 0x7f56f815cf50 2023-10-16T10:04:14.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.244+0000 7f56f77fe640 1 -- 172.21.15.84:0/3395432055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56f80b8a10 msgr2=0x7f56f80d2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:14.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.244+0000 7f56f77fe640 1 --2- 172.21.15.84:0/3395432055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56f80b8a10 0x7f56f80d2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:14.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.244+0000 7f56f5ffb640 1 -- 172.21.15.84:0/3395432055 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1551305590 0 0) 0x7f56e4003520 con 0x7f56f815cf50 2023-10-16T10:04:14.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.244+0000 7f56f77fe640 1 -- 172.21.15.84:0/3395432055 >> v1:172.21.15.84:6789/0 conn(0x7f56f815cf50 legacy=0x7f56f80d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:14.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.244+0000 7f56f77fe640 1 -- 172.21.15.84:0/3395432055 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f80d1680 con 0x7f56f80b4750 2023-10-16T10:04:14.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.245+0000 7f56f77fe640 1 --2- 172.21.15.84:0/3395432055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56f80b4750 0x7f56f80b4be0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f56f00097a0 tx=0x7f56f002b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b00b1f6e94af8e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:14.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.245+0000 7f56f5ffb640 1 -- 172.21.15.84:0/3395432055 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e80019f0 con 0x7f56f80b4750 2023-10-16T10:04:14.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.245+0000 7f56f5ffb640 1 -- 172.21.15.84:0/3395432055 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f002e050 con 0x7f56f80b4750 2023-10-16T10:04:14.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.245+0000 7f56f5ffb640 1 -- 172.21.15.84:0/3395432055 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f0030710 con 0x7f56f80b4750 2023-10-16T10:04:14.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.246+0000 7f56fd9b0640 1 -- 172.21.15.84:0/3395432055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56f80b4750 msgr2=0x7f56f80b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:14.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.246+0000 7f56fd9b0640 1 --2- 172.21.15.84:0/3395432055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56f80b4750 0x7f56f80b4be0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f56f00097a0 tx=0x7f56f002b790 comp rx=0 tx=0).stop 2023-10-16T10:04:14.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.246+0000 7f56fd9b0640 1 -- 172.21.15.84:0/3395432055 shutdown_connections 2023-10-16T10:04:14.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.246+0000 7f56fd9b0640 1 --2- 172.21.15.84:0/3395432055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56f80b4750 0x7f56f80b4be0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:14.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.246+0000 7f56fd9b0640 1 --2- 172.21.15.84:0/3395432055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56f80b8a10 0x7f56f80d2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:14.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.246+0000 7f56fd9b0640 1 -- 172.21.15.84:0/3395432055 >> 172.21.15.84:0/3395432055 conn(0x7f56f80cf750 msgr2=0x7f56f80cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:14.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.246+0000 7f56fd9b0640 1 -- 172.21.15.84:0/3395432055 shutdown_connections 2023-10-16T10:04:14.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.246+0000 7f56fd9b0640 1 -- 172.21.15.84:0/3395432055 wait complete. 2023-10-16T10:04:14.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.248+0000 7f56fd9b0640 1 Processor -- start 2023-10-16T10:04:14.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.248+0000 7f56fd9b0640 1 -- start start 2023-10-16T10:04:14.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.249+0000 7f56fd9b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56f80b4750 0x7f56f81f2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:14.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.249+0000 7f56fd9b0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56f80b8a10 0x7f56f81e6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:14.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.249+0000 7f56fd9b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56f815cf50 0x7f56f80cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:14.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.249+0000 7f56fd9b0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f8156d20 con 0x7f56f80b8a10 2023-10-16T10:04:14.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.249+0000 7f56fd9b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f81f25d0 con 0x7f56f80b4750 2023-10-16T10:04:14.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.249+0000 7f56fd9b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f8154f50 con 0x7f56f815cf50 2023-10-16T10:04:14.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.249+0000 7f56f67fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56f80b8a10 0x7f56f81e6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:14.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.249+0000 7f56f67fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56f80b8a10 0x7f56f81e6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37810/0 (socket says 172.21.15.84:37810) 2023-10-16T10:04:14.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.249+0000 7f56f67fc640 1 -- 172.21.15.84:0/3003058620 learned_addr learned my addr 172.21.15.84:0/3003058620 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:14.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.250+0000 7f56f77fe640 1 --2- 172.21.15.84:0/3003058620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56f815cf50 0x7f56f80cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:14.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.250+0000 7f56f67fc640 1 -- 172.21.15.84:0/3003058620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56f815cf50 msgr2=0x7f56f80cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:14.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.250+0000 7f56f6ffd640 1 --2- 172.21.15.84:0/3003058620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56f80b4750 0x7f56f81f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:14.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.250+0000 7f56f67fc640 1 --2- 172.21.15.84:0/3003058620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56f815cf50 0x7f56f80cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:14.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.250+0000 7f56f67fc640 1 -- 172.21.15.84:0/3003058620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56f80b4750 msgr2=0x7f56f81f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:14.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.250+0000 7f56f67fc640 1 --2- 172.21.15.84:0/3003058620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56f80b4750 0x7f56f81f2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:14.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.250+0000 7f56f67fc640 1 -- 172.21.15.84:0/3003058620 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f0009400 con 0x7f56f80b8a10 2023-10-16T10:04:14.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.250+0000 7f56f67fc640 1 --2- 172.21.15.84:0/3003058620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56f80b8a10 0x7f56f81e6840 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f56e000d830 tx=0x7f56e000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:14.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.250+0000 7f56effff640 1 -- 172.21.15.84:0/3003058620 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e000b6c0 con 0x7f56f80b8a10 2023-10-16T10:04:14.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.250+0000 7f56effff640 1 -- 172.21.15.84:0/3003058620 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e000b820 con 0x7f56f80b8a10 2023-10-16T10:04:14.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.250+0000 7f56effff640 1 -- 172.21.15.84:0/3003058620 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e000bbb0 con 0x7f56f80b8a10 2023-10-16T10:04:14.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.250+0000 7f56fd9b0640 1 -- 172.21.15.84:0/3003058620 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f8155230 con 0x7f56f80b8a10 2023-10-16T10:04:14.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.251+0000 7f56fd9b0640 1 -- 172.21.15.84:0/3003058620 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f81557f0 con 0x7f56f80b8a10 2023-10-16T10:04:14.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.251+0000 7f56fd9b0640 1 -- 172.21.15.84:0/3003058620 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f8072350 con 0x7f56f80b8a10 2023-10-16T10:04:14.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.253+0000 7f56effff640 1 -- 172.21.15.84:0/3003058620 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f56e001e460 con 0x7f56f80b8a10 2023-10-16T10:04:14.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.254+0000 7f56effff640 1 --2- 172.21.15.84:0/3003058620 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f56d806af90 0x7f56d806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:14.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.254+0000 7f56effff640 1 -- 172.21.15.84:0/3003058620 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6103+0+0 (secure 0 0 0) 0x7f56e008abc0 con 0x7f56f80b8a10 2023-10-16T10:04:14.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.255+0000 7f56f6ffd640 1 --2- 172.21.15.84:0/3003058620 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f56d806af90 0x7f56d806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:14.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.255+0000 7f56f6ffd640 1 --2- 172.21.15.84:0/3003058620 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f56d806af90 0x7f56d806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f56e8001560 tx=0x7f56e8009040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:14.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.259+0000 7f56effff640 1 -- 172.21.15.84:0/3003058620 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f56e00559b0 con 0x7f56f80b8a10 2023-10-16T10:04:14.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:14.452+0000 7f56fd9b0640 1 -- 172.21.15.84:0/3003058620 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f56f804e160 con 0x7f56f80b8a10 2023-10-16T10:04:15.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.014+0000 7f56effff640 1 -- 172.21.15.84:0/3003058620 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v73) v1 ==== 169+0+0 (secure 0 0 0) 0x7f56e0058fd0 con 0x7f56f80b8a10 2023-10-16T10:04:15.016 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-16T10:04:15.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.017+0000 7f56fd9b0640 1 -- 172.21.15.84:0/3003058620 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f56d806af90 msgr2=0x7f56d806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:15.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.017+0000 7f56fd9b0640 1 --2- 172.21.15.84:0/3003058620 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f56d806af90 0x7f56d806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f56e8001560 tx=0x7f56e8009040 comp rx=0 tx=0).stop 2023-10-16T10:04:15.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.017+0000 7f56fd9b0640 1 -- 172.21.15.84:0/3003058620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56f80b8a10 msgr2=0x7f56f81e6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:15.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.017+0000 7f56fd9b0640 1 --2- 172.21.15.84:0/3003058620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56f80b8a10 0x7f56f81e6840 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f56e000d830 tx=0x7f56e000dcf0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.018+0000 7f56fd9b0640 1 -- 172.21.15.84:0/3003058620 shutdown_connections 2023-10-16T10:04:15.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.018+0000 7f56fd9b0640 1 --2- 172.21.15.84:0/3003058620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56f815cf50 0x7f56f80cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.018+0000 7f56fd9b0640 1 --2- 172.21.15.84:0/3003058620 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f56d806af90 0x7f56d806d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.018+0000 7f56fd9b0640 1 --2- 172.21.15.84:0/3003058620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56f80b8a10 0x7f56f81e6840 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.018+0000 7f56fd9b0640 1 --2- 172.21.15.84:0/3003058620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56f80b4750 0x7f56f81f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.018+0000 7f56fd9b0640 1 -- 172.21.15.84:0/3003058620 >> 172.21.15.84:0/3003058620 conn(0x7f56f80cf750 msgr2=0x7f56f815c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:15.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.018+0000 7f56fd9b0640 1 -- 172.21.15.84:0/3003058620 shutdown_connections 2023-10-16T10:04:15.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.018+0000 7f56fd9b0640 1 -- 172.21.15.84:0/3003058620 wait complete. 2023-10-16T10:04:15.044 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T10:04:15.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.252+0000 7efe01690640 1 Processor -- start 2023-10-16T10:04:15.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.252+0000 7efe01690640 1 -- start start 2023-10-16T10:04:15.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.253+0000 7efe01690640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efdfc15ad70 0x7efdfc069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:15.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.254+0000 7efe01690640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdfc0caac0 0x7efdfc06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:15.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.254+0000 7efe01690640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdfc0cb030 con 0x7efdfc0c9ca0 2023-10-16T10:04:15.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.254+0000 7efe01690640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdfc0c91c0 con 0x7efdfc15ad70 2023-10-16T10:04:15.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.254+0000 7efe01690640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0c94a0 con 0x7efdfc0caac0 2023-10-16T10:04:15.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.254+0000 7efdfaffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efdfc15ad70 0x7efdfc069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:15.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.254+0000 7efdfaffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efdfc15ad70 0x7efdfc069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37816/0 (socket says 172.21.15.84:37816) 2023-10-16T10:04:15.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.254+0000 7efdfaffd640 1 -- 172.21.15.84:0/2508709365 learned_addr learned my addr 172.21.15.84:0/2508709365 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:15.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.254+0000 7efdfb7fe640 1 --2- 172.21.15.84:0/2508709365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdfc0caac0 0x7efdfc06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:15.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.255+0000 7efdfaffd640 1 -- 172.21.15.84:0/2508709365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdfc0caac0 msgr2=0x7efdfc06a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:15.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.255+0000 7efdfaffd640 1 --2- 172.21.15.84:0/2508709365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdfc0caac0 0x7efdfc06a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.255+0000 7efdfaffd640 1 -- 172.21.15.84:0/2508709365 >> v1:172.21.15.157:6789/0 conn(0x7efdfc0c9ca0 legacy=0x7efdfc0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:15.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.255+0000 7efdfaffd640 1 -- 172.21.15.84:0/2508709365 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc0c9780 con 0x7efdfc15ad70 2023-10-16T10:04:15.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.256+0000 7efdfaffd640 1 --2- 172.21.15.84:0/2508709365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdfc15ad70 0x7efdfc069da0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7efdec009870 tx=0x7efdec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e79d9f929c7236 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:15.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.256+0000 7efdfa7fc640 1 -- 172.21.15.84:0/2508709365 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec031640 con 0x7efdfc15ad70 2023-10-16T10:04:15.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.256+0000 7efdfa7fc640 1 -- 172.21.15.84:0/2508709365 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec0317a0 con 0x7efdfc15ad70 2023-10-16T10:04:15.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.256+0000 7efdfa7fc640 1 -- 172.21.15.84:0/2508709365 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec031b30 con 0x7efdfc15ad70 2023-10-16T10:04:15.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.257+0000 7efe01690640 1 -- 172.21.15.84:0/2508709365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdfc15ad70 msgr2=0x7efdfc069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:15.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.257+0000 7efe01690640 1 --2- 172.21.15.84:0/2508709365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdfc15ad70 0x7efdfc069da0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7efdec009870 tx=0x7efdec02b8f0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.257+0000 7efe01690640 1 -- 172.21.15.84:0/2508709365 shutdown_connections 2023-10-16T10:04:15.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.257+0000 7efe01690640 1 --2- 172.21.15.84:0/2508709365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdfc0caac0 0x7efdfc06a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.257+0000 7efe01690640 1 --2- 172.21.15.84:0/2508709365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdfc15ad70 0x7efdfc069da0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.257+0000 7efe01690640 1 -- 172.21.15.84:0/2508709365 >> 172.21.15.84:0/2508709365 conn(0x7efdfc0ce9b0 msgr2=0x7efdfc0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:15.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.257+0000 7efe01690640 1 -- 172.21.15.84:0/2508709365 shutdown_connections 2023-10-16T10:04:15.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.257+0000 7efe01690640 1 -- 172.21.15.84:0/2508709365 wait complete. 2023-10-16T10:04:15.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.259+0000 7efe01690640 1 Processor -- start 2023-10-16T10:04:15.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.259+0000 7efe01690640 1 -- start start 2023-10-16T10:04:15.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.260+0000 7efe01690640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdfc0c9ca0 0x7efdfc1580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:15.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.260+0000 7efe01690640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdfc0caac0 0x7efdfc155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:15.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.260+0000 7efe01690640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdfc15ad70 0x7efdfc159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:15.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.260+0000 7efe01690640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc1f1670 con 0x7efdfc15ad70 2023-10-16T10:04:15.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.260+0000 7efe01690640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc1f1910 con 0x7efdfc0caac0 2023-10-16T10:04:15.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.260+0000 7efe01690640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc1f1bf0 con 0x7efdfc0c9ca0 2023-10-16T10:04:15.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.260+0000 7efdfb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdfc15ad70 0x7efdfc159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:15.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.260+0000 7efdfb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdfc15ad70 0x7efdfc159060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37830/0 (socket says 172.21.15.84:37830) 2023-10-16T10:04:15.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.260+0000 7efdfb7fe640 1 -- 172.21.15.84:0/3855920316 learned_addr learned my addr 172.21.15.84:0/3855920316 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:15.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.261+0000 7efdfaffd640 1 --2- 172.21.15.84:0/3855920316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdfc0c9ca0 0x7efdfc1580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:15.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.261+0000 7efdf27fc640 1 --2- 172.21.15.84:0/3855920316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdfc0caac0 0x7efdfc155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:15.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.261+0000 7efdfb7fe640 1 -- 172.21.15.84:0/3855920316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdfc0c9ca0 msgr2=0x7efdfc1580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:15.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.261+0000 7efdfb7fe640 1 --2- 172.21.15.84:0/3855920316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdfc0c9ca0 0x7efdfc1580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.261+0000 7efdfb7fe640 1 -- 172.21.15.84:0/3855920316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdfc0caac0 msgr2=0x7efdfc155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:15.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.261+0000 7efdfb7fe640 1 --2- 172.21.15.84:0/3855920316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdfc0caac0 0x7efdfc155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.261+0000 7efdfb7fe640 1 -- 172.21.15.84:0/3855920316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec0094d0 con 0x7efdfc15ad70 2023-10-16T10:04:15.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.261+0000 7efdf27fc640 1 --2- 172.21.15.84:0/3855920316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdfc0caac0 0x7efdfc155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:15.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.261+0000 7efdfb7fe640 1 --2- 172.21.15.84:0/3855920316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdfc15ad70 0x7efdfc159060 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7efdf4007e40 tx=0x7efdf400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:15.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.261+0000 7efdf3fff640 1 -- 172.21.15.84:0/3855920316 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4014620 con 0x7efdfc15ad70 2023-10-16T10:04:15.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.261+0000 7efdf3fff640 1 -- 172.21.15.84:0/3855920316 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf400ee90 con 0x7efdfc15ad70 2023-10-16T10:04:15.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.262+0000 7efdf3fff640 1 -- 172.21.15.84:0/3855920316 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf40149e0 con 0x7efdfc15ad70 2023-10-16T10:04:15.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.262+0000 7efe01690640 1 -- 172.21.15.84:0/3855920316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdfc1f1ed0 con 0x7efdfc15ad70 2023-10-16T10:04:15.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.262+0000 7efe01690640 1 -- 172.21.15.84:0/3855920316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc0afc10 con 0x7efdfc15ad70 2023-10-16T10:04:15.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.263+0000 7efe01690640 1 -- 172.21.15.84:0/3855920316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdfc1f23f0 con 0x7efdfc15ad70 2023-10-16T10:04:15.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.264+0000 7efdf3fff640 1 -- 172.21.15.84:0/3855920316 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efdf4014b40 con 0x7efdfc15ad70 2023-10-16T10:04:15.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.265+0000 7efdf3fff640 1 --2- 172.21.15.84:0/3855920316 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7efde006af90 0x7efde006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:15.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.266+0000 7efdf3fff640 1 -- 172.21.15.84:0/3855920316 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5631+0+0 (secure 0 0 0) 0x7efdf408a110 con 0x7efdfc15ad70 2023-10-16T10:04:15.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.266+0000 7efdfaffd640 1 --2- 172.21.15.84:0/3855920316 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7efde006af90 0x7efde006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:15.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.267+0000 7efdfaffd640 1 --2- 172.21.15.84:0/3855920316 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7efde006af90 0x7efde006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efdec036000 tx=0x7efdec03c040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:15.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.280+0000 7efdf3fff640 1 -- 172.21.15.84:0/3855920316 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efdfc1f23f0 con 0x7efdfc15ad70 2023-10-16T10:04:15.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.473+0000 7efe01690640 1 -- 172.21.15.84:0/3855920316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7efdfc1f23f0 con 0x7efdfc15ad70 2023-10-16T10:04:15.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.480+0000 7efdf3fff640 1 -- 172.21.15.84:0/3855920316 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7efdfc1f23f0 con 0x7efdfc15ad70 2023-10-16T10:04:15.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.483+0000 7efe01690640 1 -- 172.21.15.84:0/3855920316 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7efde006af90 msgr2=0x7efde006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:15.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.483+0000 7efe01690640 1 --2- 172.21.15.84:0/3855920316 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7efde006af90 0x7efde006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efdec036000 tx=0x7efdec03c040 comp rx=0 tx=0).stop 2023-10-16T10:04:15.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.483+0000 7efe01690640 1 -- 172.21.15.84:0/3855920316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdfc15ad70 msgr2=0x7efdfc159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:15.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.483+0000 7efe01690640 1 --2- 172.21.15.84:0/3855920316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdfc15ad70 0x7efdfc159060 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7efdf4007e40 tx=0x7efdf400e4b0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.483+0000 7efe01690640 1 -- 172.21.15.84:0/3855920316 shutdown_connections 2023-10-16T10:04:15.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.483+0000 7efe01690640 1 --2- 172.21.15.84:0/3855920316 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7efde006af90 0x7efde006d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.483+0000 7efe01690640 1 --2- 172.21.15.84:0/3855920316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdfc15ad70 0x7efdfc159060 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.483+0000 7efe01690640 1 --2- 172.21.15.84:0/3855920316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdfc0caac0 0x7efdfc155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.483+0000 7efe01690640 1 --2- 172.21.15.84:0/3855920316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdfc0c9ca0 0x7efdfc1580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.483+0000 7efe01690640 1 -- 172.21.15.84:0/3855920316 >> 172.21.15.84:0/3855920316 conn(0x7efdfc0ce9b0 msgr2=0x7efdfc0b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:15.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.484+0000 7efe01690640 1 -- 172.21.15.84:0/3855920316 shutdown_connections 2023-10-16T10:04:15.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.484+0000 7efe01690640 1 -- 172.21.15.84:0/3855920316 wait complete. 2023-10-16T10:04:15.504 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T10:04:15.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.716+0000 7f4b03ccf640 1 Processor -- start 2023-10-16T10:04:15.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.717+0000 7f4b03ccf640 1 -- start start 2023-10-16T10:04:15.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.718+0000 7f4b03ccf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4afc06a4b0 0x7f4afc0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:15.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.718+0000 7f4b03ccf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4afc0c9ca0 0x7f4afc0ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:15.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.718+0000 7f4b03ccf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4afc0cb030 con 0x7f4afc15ad70 2023-10-16T10:04:15.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.718+0000 7f4b03ccf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4afc0cae30 con 0x7f4afc06a4b0 2023-10-16T10:04:15.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.718+0000 7f4b03ccf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0c91c0 con 0x7f4afc0c9ca0 2023-10-16T10:04:15.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.718+0000 7f4b01a6b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4afc15ad70 0x7f4afc069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38538/0 (socket says 172.21.15.84:38538) 2023-10-16T10:04:15.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.718+0000 7f4b01a6b640 1 -- 172.21.15.84:0/2240760242 learned_addr learned my addr 172.21.15.84:0/2240760242 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:15.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.719+0000 7f4b0126a640 1 --2- 172.21.15.84:0/2240760242 >> v2:172.21.15.157:3300/0 conn(0x7f4afc06a4b0 0x7f4afc0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:15.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.719+0000 7f4b0226c640 1 --2- 172.21.15.84:0/2240760242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4afc0c9ca0 0x7f4afc0ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:15.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.719+0000 7f4b00a69640 1 -- 172.21.15.84:0/2240760242 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127583982 0 0) 0x7f4afc0cb030 con 0x7f4afc15ad70 2023-10-16T10:04:15.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.719+0000 7f4b00a69640 1 -- 172.21.15.84:0/2240760242 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ae8003520 con 0x7f4afc15ad70 2023-10-16T10:04:15.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.719+0000 7f4b00a69640 1 -- 172.21.15.84:0/2240760242 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2896748782 0 0) 0x7f4ae8003520 con 0x7f4afc15ad70 2023-10-16T10:04:15.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.719+0000 7f4b00a69640 1 -- 172.21.15.84:0/2240760242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4afc0c9ca0 msgr2=0x7f4afc0ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:15.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.719+0000 7f4b00a69640 1 --2- 172.21.15.84:0/2240760242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4afc0c9ca0 0x7f4afc0ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.719+0000 7f4b00a69640 1 -- 172.21.15.84:0/2240760242 >> v2:172.21.15.157:3300/0 conn(0x7f4afc06a4b0 msgr2=0x7f4afc0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:15.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.719+0000 7f4b00a69640 1 --2- 172.21.15.84:0/2240760242 >> v2:172.21.15.157:3300/0 conn(0x7f4afc06a4b0 0x7f4afc0ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.719+0000 7f4b00a69640 1 -- 172.21.15.84:0/2240760242 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc0c94a0 con 0x7f4afc15ad70 2023-10-16T10:04:15.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.719+0000 7f4b00a69640 1 -- 172.21.15.84:0/2240760242 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f4af8001670 con 0x7f4afc15ad70 2023-10-16T10:04:15.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.719+0000 7f4b00a69640 1 -- 172.21.15.84:0/2240760242 >> v1:172.21.15.84:6789/0 conn(0x7f4afc15ad70 legacy=0x7f4afc069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:15.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.719+0000 7f4b00a69640 1 --2- 172.21.15.84:0/2240760242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4afc0c9ca0 0x7f4ae8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:15.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.719+0000 7f4b00a69640 1 --2- 172.21.15.84:0/2240760242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ae8003f40 0x7f4ae80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:15.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.720+0000 7f4b00a69640 1 --2- 172.21.15.84:0/2240760242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ae8004d00 0x7f4ae8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:15.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.720+0000 7f4b00a69640 1 -- 172.21.15.84:0/2240760242 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0cae30 con 0x7f4ae8003f40 2023-10-16T10:04:15.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.720+0000 7f4b00a69640 1 -- 172.21.15.84:0/2240760242 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0c91c0 con 0x7f4ae8004d00 2023-10-16T10:04:15.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.720+0000 7f4b00a69640 1 -- 172.21.15.84:0/2240760242 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae800b980 con 0x7f4afc0c9ca0 2023-10-16T10:04:15.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.720+0000 7f4b0126a640 1 --2- 172.21.15.84:0/2240760242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ae8003f40 0x7f4ae80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:15.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.720+0000 7f4b01a6b640 1 --2- 172.21.15.84:0/2240760242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4afc0c9ca0 0x7f4ae8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:15.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.720+0000 7f4b0126a640 1 -- 172.21.15.84:0/2240760242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4afc0c9ca0 msgr2=0x7f4ae8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:15.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.720+0000 7f4b0226c640 1 --2- 172.21.15.84:0/2240760242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ae8004d00 0x7f4ae8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:15.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.720+0000 7f4b0126a640 1 --2- 172.21.15.84:0/2240760242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4afc0c9ca0 0x7f4ae8003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.720+0000 7f4b0126a640 1 -- 172.21.15.84:0/2240760242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ae8004d00 msgr2=0x7f4ae8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.720+0000 7f4b0126a640 1 --2- 172.21.15.84:0/2240760242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ae8004d00 0x7f4ae8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.720+0000 7f4b0126a640 1 -- 172.21.15.84:0/2240760242 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae8003520 con 0x7f4ae8003f40 2023-10-16T10:04:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.721+0000 7f4b0126a640 1 --2- 172.21.15.84:0/2240760242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ae8003f40 0x7f4ae80047c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009af0 tx=0x7f4ae80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab2cc7ab6526998 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.722+0000 7f4b00a69640 1 -- 172.21.15.84:0/2240760242 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec02c710 con 0x7f4ae8003f40 2023-10-16T10:04:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.722+0000 7f4b00a69640 1 -- 172.21.15.84:0/2240760242 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aec02c870 con 0x7f4ae8003f40 2023-10-16T10:04:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.722+0000 7f4b00a69640 1 -- 172.21.15.84:0/2240760242 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec02cc00 con 0x7f4ae8003f40 2023-10-16T10:04:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.722+0000 7f4b03ccf640 1 -- 172.21.15.84:0/2240760242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ae8003f40 msgr2=0x7f4ae80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.722+0000 7f4b03ccf640 1 --2- 172.21.15.84:0/2240760242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ae8003f40 0x7f4ae80047c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009af0 tx=0x7f4ae80057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.722+0000 7f4b0226c640 1 -- 172.21.15.84:0/2240760242 reap_dead start 2023-10-16T10:04:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.722+0000 7f4b03ccf640 1 -- 172.21.15.84:0/2240760242 shutdown_connections 2023-10-16T10:04:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.722+0000 7f4b03ccf640 1 -- 172.21.15.84:0/2240760242 >> 172.21.15.84:0/2240760242 conn(0x7f4afc0ce9b0 msgr2=0x7f4afc0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:15.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.722+0000 7f4b03ccf640 1 -- 172.21.15.84:0/2240760242 shutdown_connections 2023-10-16T10:04:15.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.722+0000 7f4b03ccf640 1 -- 172.21.15.84:0/2240760242 wait complete. 2023-10-16T10:04:15.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.724+0000 7f4b03ccf640 1 Processor -- start 2023-10-16T10:04:15.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.724+0000 7f4b03ccf640 1 -- start start 2023-10-16T10:04:15.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.725+0000 7f4b03ccf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ae8003f40 0x7f4afc15a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:15.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.725+0000 7f4b03ccf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4afc0c4550 0x7f4afc0c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:15.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.725+0000 7f4b03ccf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4afc0c2720 0x7f4afc0c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:15.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.725+0000 7f4b03ccf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0c5db0 con 0x7f4ae8003f40 2023-10-16T10:04:15.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.725+0000 7f4b03ccf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0c30e0 con 0x7f4afc0c2720 2023-10-16T10:04:15.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.725+0000 7f4b03ccf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0aeb40 con 0x7f4afc0c4550 2023-10-16T10:04:15.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.726+0000 7f4b01a6b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ae8003f40 0x7f4afc15a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:15.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.726+0000 7f4b01a6b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ae8003f40 0x7f4afc15a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50824/0 (socket says 172.21.15.84:50824) 2023-10-16T10:04:15.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.726+0000 7f4b01a6b640 1 -- 172.21.15.84:0/706507888 learned_addr learned my addr 172.21.15.84:0/706507888 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:15.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.726+0000 7f4b01a6b640 1 -- 172.21.15.84:0/706507888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4afc0c4550 msgr2=0x7f4afc0c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:15.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.726+0000 7f4b01a6b640 1 --2- 172.21.15.84:0/706507888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4afc0c4550 0x7f4afc0c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.726+0000 7f4b01a6b640 1 -- 172.21.15.84:0/706507888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4afc0c2720 msgr2=0x7f4afc0c2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:15.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.726+0000 7f4b01a6b640 1 --2- 172.21.15.84:0/706507888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4afc0c2720 0x7f4afc0c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.726+0000 7f4b01a6b640 1 -- 172.21.15.84:0/706507888 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc0aee20 con 0x7f4ae8003f40 2023-10-16T10:04:15.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.726+0000 7f4b01a6b640 1 --2- 172.21.15.84:0/706507888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ae8003f40 0x7f4afc15a190 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f4af800d7a0 tx=0x7f4af800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:15.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.727+0000 7f4af27fc640 1 -- 172.21.15.84:0/706507888 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af80074e0 con 0x7f4ae8003f40 2023-10-16T10:04:15.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.727+0000 7f4af27fc640 1 -- 172.21.15.84:0/706507888 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af8007640 con 0x7f4ae8003f40 2023-10-16T10:04:15.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.727+0000 7f4af27fc640 1 -- 172.21.15.84:0/706507888 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8004a00 con 0x7f4ae8003f40 2023-10-16T10:04:15.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.727+0000 7f4b03ccf640 1 -- 172.21.15.84:0/706507888 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4afc0af100 con 0x7f4ae8003f40 2023-10-16T10:04:15.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.727+0000 7f4b03ccf640 1 -- 172.21.15.84:0/706507888 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4afc0c5190 con 0x7f4ae8003f40 2023-10-16T10:04:15.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.728+0000 7f4b03ccf640 1 -- 172.21.15.84:0/706507888 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4afc0af290 con 0x7f4ae8003f40 2023-10-16T10:04:15.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.731+0000 7f4af27fc640 1 -- 172.21.15.84:0/706507888 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4af8004b60 con 0x7f4ae8003f40 2023-10-16T10:04:15.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.732+0000 7f4af27fc640 1 --2- 172.21.15.84:0/706507888 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f4ae006af90 0x7f4ae006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:15.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.732+0000 7f4af27fc640 1 -- 172.21.15.84:0/706507888 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f4af8089010 con 0x7f4ae8003f40 2023-10-16T10:04:15.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.732+0000 7f4b0126a640 1 --2- 172.21.15.84:0/706507888 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f4ae006af90 0x7f4ae006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:15.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.733+0000 7f4b0126a640 1 --2- 172.21.15.84:0/706507888 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f4ae006af90 0x7f4ae006d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4aec036000 tx=0x7f4aec02e000 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:15.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.737+0000 7f4af27fc640 1 -- 172.21.15.84:0/706507888 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4af8054040 con 0x7f4ae8003f40 2023-10-16T10:04:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.931+0000 7f4b03ccf640 1 -- 172.21.15.84:0/706507888 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f4afc04e160 con 0x7f4ae8003f40 2023-10-16T10:04:15.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.938+0000 7f4af27fc640 1 -- 172.21.15.84:0/706507888 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v20) v1 ==== 120+0+0 (secure 0 0 0) 0x7f4af8057660 con 0x7f4ae8003f40 2023-10-16T10:04:15.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.941+0000 7f4b03ccf640 1 -- 172.21.15.84:0/706507888 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f4ae006af90 msgr2=0x7f4ae006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:15.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.941+0000 7f4b03ccf640 1 --2- 172.21.15.84:0/706507888 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f4ae006af90 0x7f4ae006d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4aec036000 tx=0x7f4aec02e000 comp rx=0 tx=0).stop 2023-10-16T10:04:15.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.941+0000 7f4b03ccf640 1 -- 172.21.15.84:0/706507888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ae8003f40 msgr2=0x7f4afc15a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:15.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.941+0000 7f4b03ccf640 1 --2- 172.21.15.84:0/706507888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ae8003f40 0x7f4afc15a190 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f4af800d7a0 tx=0x7f4af800dc60 comp rx=0 tx=0).stop 2023-10-16T10:04:15.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.941+0000 7f4b03ccf640 1 -- 172.21.15.84:0/706507888 shutdown_connections 2023-10-16T10:04:15.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.941+0000 7f4b03ccf640 1 --2- 172.21.15.84:0/706507888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4afc0c2720 0x7f4afc0c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.941+0000 7f4b03ccf640 1 --2- 172.21.15.84:0/706507888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4afc0c4550 0x7f4afc0c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.941+0000 7f4b03ccf640 1 --2- 172.21.15.84:0/706507888 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f4ae006af90 0x7f4ae006d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.941+0000 7f4b03ccf640 1 --2- 172.21.15.84:0/706507888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ae8003f40 0x7f4afc15a190 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.941+0000 7f4b03ccf640 1 -- 172.21.15.84:0/706507888 >> 172.21.15.84:0/706507888 conn(0x7f4afc0ce9b0 msgr2=0x7f4afc15d510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:15.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.941+0000 7f4b03ccf640 1 -- 172.21.15.84:0/706507888 shutdown_connections 2023-10-16T10:04:15.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:15.941+0000 7f4b03ccf640 1 -- 172.21.15.84:0/706507888 wait complete. 2023-10-16T10:04:15.963 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T10:04:16.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.167+0000 7f30a4d01640 1 Processor -- start 2023-10-16T10:04:16.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.167+0000 7f30a4d01640 1 -- start start 2023-10-16T10:04:16.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.168+0000 7f30a4d01640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f30a015aaa0 0x7f30a015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:16.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.168+0000 7f30a4d01640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30a015ae80 0x7f30a00cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:16.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.168+0000 7f30a4d01640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a00707e0 con 0x7f30a00ccaa0 2023-10-16T10:04:16.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.168+0000 7f30a4d01640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a006b7a0 con 0x7f30a015aaa0 2023-10-16T10:04:16.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.168+0000 7f30a4d01640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a015ef20 con 0x7f30a015ae80 2023-10-16T10:04:16.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.169+0000 7f3095d9b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30a015ae80 0x7f30a00cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:16.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.169+0000 7f309e59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f30a015aaa0 0x7f30a015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:16.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.169+0000 7f309ed9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f30a00ccaa0 0x7f30a0070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:52748/0 (socket says 172.21.15.84:52748) 2023-10-16T10:04:16.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.169+0000 7f3095d9b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30a015ae80 0x7f30a00cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:36170/0 (socket says 172.21.15.84:36170) 2023-10-16T10:04:16.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.169+0000 7f309ed9d640 1 -- 172.21.15.84:0/2449089590 learned_addr learned my addr 172.21.15.84:0/2449089590 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:16.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.170+0000 7f309e59c640 1 -- 172.21.15.84:0/2449089590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30a015ae80 msgr2=0x7f30a00cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:16.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.170+0000 7f309e59c640 1 --2- 172.21.15.84:0/2449089590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30a015ae80 0x7f30a00cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:16.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.170+0000 7f309e59c640 1 -- 172.21.15.84:0/2449089590 >> v1:172.21.15.157:6789/0 conn(0x7f30a00ccaa0 legacy=0x7f30a0070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:16.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.170+0000 7f309e59c640 1 -- 172.21.15.84:0/2449089590 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a015f200 con 0x7f30a015aaa0 2023-10-16T10:04:16.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.170+0000 7f3095d9b640 1 --2- 172.21.15.84:0/2449089590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30a015ae80 0x7f30a00cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:04:16.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.170+0000 7f309dd9b640 1 -- 172.21.15.84:0/2449089590 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1046634194 0 0) 0x7f30a00707e0 con 0x7f30a00ccaa0 2023-10-16T10:04:16.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.172+0000 7f309e59c640 1 --2- 172.21.15.84:0/2449089590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30a015aaa0 0x7f30a015e820 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f30900097a0 tx=0x7f309002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b137c63b449282ac server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:16.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.173+0000 7f309dd9b640 1 -- 172.21.15.84:0/2449089590 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309002e070 con 0x7f30a015aaa0 2023-10-16T10:04:16.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.173+0000 7f309dd9b640 1 -- 172.21.15.84:0/2449089590 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30900315a0 con 0x7f30a015aaa0 2023-10-16T10:04:16.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.173+0000 7f309dd9b640 1 -- 172.21.15.84:0/2449089590 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090031930 con 0x7f30a015aaa0 2023-10-16T10:04:16.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.174+0000 7f30a4d01640 1 -- 172.21.15.84:0/2449089590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30a015aaa0 msgr2=0x7f30a015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:16.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.174+0000 7f30a4d01640 1 --2- 172.21.15.84:0/2449089590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30a015aaa0 0x7f30a015e820 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f30900097a0 tx=0x7f309002b840 comp rx=0 tx=0).stop 2023-10-16T10:04:16.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.174+0000 7f30a4d01640 1 -- 172.21.15.84:0/2449089590 shutdown_connections 2023-10-16T10:04:16.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.174+0000 7f30a4d01640 1 --2- 172.21.15.84:0/2449089590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30a015ae80 0x7f30a00cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:16.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.174+0000 7f30a4d01640 1 --2- 172.21.15.84:0/2449089590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30a015aaa0 0x7f30a015e820 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:16.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.174+0000 7f30a4d01640 1 -- 172.21.15.84:0/2449089590 >> 172.21.15.84:0/2449089590 conn(0x7f30a00cb410 msgr2=0x7f30a00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:16.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.174+0000 7f30a4d01640 1 -- 172.21.15.84:0/2449089590 shutdown_connections 2023-10-16T10:04:16.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.174+0000 7f30a4d01640 1 -- 172.21.15.84:0/2449089590 wait complete. 2023-10-16T10:04:16.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.176+0000 7f30a4d01640 1 Processor -- start 2023-10-16T10:04:16.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.176+0000 7f30a4d01640 1 -- start start 2023-10-16T10:04:16.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.178+0000 7f30a4d01640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30a00ccaa0 0x7f30a014ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:16.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.178+0000 7f30a4d01640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30a015aaa0 0x7f30a00af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:16.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.178+0000 7f30a4d01640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30a015ae80 0x7f30a00af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:16.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.178+0000 7f30a4d01640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a00c6ac0 con 0x7f30a00ccaa0 2023-10-16T10:04:16.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.178+0000 7f30a4d01640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a00c6da0 con 0x7f30a015aaa0 2023-10-16T10:04:16.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.178+0000 7f30a4d01640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a00c7080 con 0x7f30a015ae80 2023-10-16T10:04:16.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.178+0000 7f309e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30a00ccaa0 0x7f30a014ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:16.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.178+0000 7f309e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30a00ccaa0 0x7f30a014ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50838/0 (socket says 172.21.15.84:50838) 2023-10-16T10:04:16.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.178+0000 7f309e59c640 1 -- 172.21.15.84:0/960385988 learned_addr learned my addr 172.21.15.84:0/960385988 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:16.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.178+0000 7f3095d9b640 1 --2- 172.21.15.84:0/960385988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30a015aaa0 0x7f30a00af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:16.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.178+0000 7f309ed9d640 1 --2- 172.21.15.84:0/960385988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30a015ae80 0x7f30a00af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:16.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.178+0000 7f309e59c640 1 -- 172.21.15.84:0/960385988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30a015ae80 msgr2=0x7f30a00af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:16.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.178+0000 7f309e59c640 1 --2- 172.21.15.84:0/960385988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30a015ae80 0x7f30a00af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:16.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.178+0000 7f309e59c640 1 -- 172.21.15.84:0/960385988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30a015aaa0 msgr2=0x7f30a00af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:16.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.178+0000 7f309e59c640 1 --2- 172.21.15.84:0/960385988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30a015aaa0 0x7f30a00af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:16.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.179+0000 7f309e59c640 1 -- 172.21.15.84:0/960385988 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3088009400 con 0x7f30a00ccaa0 2023-10-16T10:04:16.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.179+0000 7f309e59c640 1 --2- 172.21.15.84:0/960385988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30a00ccaa0 0x7f30a014ce80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3090034040 tx=0x7f30900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:16.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.179+0000 7f30977fe640 1 -- 172.21.15.84:0/960385988 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090032840 con 0x7f30a00ccaa0 2023-10-16T10:04:16.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.179+0000 7f30977fe640 1 -- 172.21.15.84:0/960385988 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3090031f00 con 0x7f30a00ccaa0 2023-10-16T10:04:16.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.179+0000 7f30977fe640 1 -- 172.21.15.84:0/960385988 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090032be0 con 0x7f30a00ccaa0 2023-10-16T10:04:16.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.179+0000 7f30a4d01640 1 -- 172.21.15.84:0/960385988 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3090009400 con 0x7f30a00ccaa0 2023-10-16T10:04:16.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.179+0000 7f30a4d01640 1 -- 172.21.15.84:0/960385988 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a00c7640 con 0x7f30a00ccaa0 2023-10-16T10:04:16.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.181+0000 7f30a4d01640 1 -- 172.21.15.84:0/960385988 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a004e160 con 0x7f30a00ccaa0 2023-10-16T10:04:16.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.182+0000 7f30977fe640 1 -- 172.21.15.84:0/960385988 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3090032d40 con 0x7f30a00ccaa0 2023-10-16T10:04:16.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.182+0000 7f30977fe640 1 --2- 172.21.15.84:0/960385988 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f308006af90 0x7f308006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:16.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.183+0000 7f30977fe640 1 -- 172.21.15.84:0/960385988 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f30900ab840 con 0x7f30a00ccaa0 2023-10-16T10:04:16.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.183+0000 7f3095d9b640 1 --2- 172.21.15.84:0/960385988 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f308006af90 0x7f308006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:16.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.183+0000 7f3095d9b640 1 --2- 172.21.15.84:0/960385988 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f308006af90 0x7f308006d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f30a00c2300 tx=0x7f3088008040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:16.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.187+0000 7f30977fe640 1 -- 172.21.15.84:0/960385988 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3090076730 con 0x7f30a00ccaa0 2023-10-16T10:04:16.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.380+0000 7f30a4d01640 1 -- 172.21.15.84:0/960385988 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f30a00cdc60 con 0x7f30a00ccaa0 2023-10-16T10:04:16.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.387+0000 7f30977fe640 1 -- 172.21.15.84:0/960385988 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7f3090079d50 con 0x7f30a00ccaa0 2023-10-16T10:04:16.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.390+0000 7f30a4d01640 1 -- 172.21.15.84:0/960385988 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f308006af90 msgr2=0x7f308006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:16.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.390+0000 7f30a4d01640 1 --2- 172.21.15.84:0/960385988 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f308006af90 0x7f308006d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f30a00c2300 tx=0x7f3088008040 comp rx=0 tx=0).stop 2023-10-16T10:04:16.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.390+0000 7f30a4d01640 1 -- 172.21.15.84:0/960385988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30a00ccaa0 msgr2=0x7f30a014ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:16.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.390+0000 7f30a4d01640 1 --2- 172.21.15.84:0/960385988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30a00ccaa0 0x7f30a014ce80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3090034040 tx=0x7f30900046c0 comp rx=0 tx=0).stop 2023-10-16T10:04:16.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.390+0000 7f30a4d01640 1 -- 172.21.15.84:0/960385988 shutdown_connections 2023-10-16T10:04:16.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.390+0000 7f30a4d01640 1 --2- 172.21.15.84:0/960385988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30a015ae80 0x7f30a00af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:16.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.390+0000 7f30a4d01640 1 --2- 172.21.15.84:0/960385988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30a015aaa0 0x7f30a00af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:16.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.390+0000 7f30a4d01640 1 --2- 172.21.15.84:0/960385988 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f308006af90 0x7f308006d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:16.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.390+0000 7f30a4d01640 1 --2- 172.21.15.84:0/960385988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30a00ccaa0 0x7f30a014ce80 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:16.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.390+0000 7f30a4d01640 1 -- 172.21.15.84:0/960385988 >> 172.21.15.84:0/960385988 conn(0x7f30a00cb410 msgr2=0x7f30a00b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:16.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.391+0000 7f30a4d01640 1 -- 172.21.15.84:0/960385988 shutdown_connections 2023-10-16T10:04:16.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.391+0000 7f30a4d01640 1 -- 172.21.15.84:0/960385988 wait complete. 2023-10-16T10:04:16.412 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-10-16T10:04:16.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.612+0000 7fc6c6de2640 1 Processor -- start 2023-10-16T10:04:16.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.612+0000 7fc6c6de2640 1 -- start start 2023-10-16T10:04:16.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.614+0000 7fc6c6de2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6c00cd810 0x7fc6c006fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:16.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.614+0000 7fc6c6de2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c00cdca0 con 0x7fc6c01599c0 2023-10-16T10:04:16.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.614+0000 7fc6c6de2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c006bab0 con 0x7fc6c00cd380 2023-10-16T10:04:16.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.614+0000 7fc6c6de2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c006f300 con 0x7fc6c00cd810 2023-10-16T10:04:16.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.614+0000 7fc6c5de0640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc6c01599c0 0x7fc6c00ca1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38540/0 (socket says 172.21.15.84:38540) 2023-10-16T10:04:16.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.614+0000 7fc6c5de0640 1 -- 172.21.15.84:0/1657990623 learned_addr learned my addr 172.21.15.84:0/1657990623 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:16.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.614+0000 7fc6c4dde640 1 -- 172.21.15.84:0/1657990623 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 631357654 0 0) 0x7fc6c00cdca0 con 0x7fc6c01599c0 2023-10-16T10:04:16.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.614+0000 7fc6c65e1640 1 --2- 172.21.15.84:0/1657990623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6c00cd810 0x7fc6c006fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:16.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.614+0000 7fc6c4dde640 1 -- 172.21.15.84:0/1657990623 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6ac003520 con 0x7fc6c01599c0 2023-10-16T10:04:16.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.615+0000 7fc6c4dde640 1 -- 172.21.15.84:0/1657990623 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 781066099 0 0) 0x7fc6ac003520 con 0x7fc6c01599c0 2023-10-16T10:04:16.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.615+0000 7fc6c4dde640 1 -- 172.21.15.84:0/1657990623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6c00cd810 msgr2=0x7fc6c006fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:16.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.615+0000 7fc6c4dde640 1 --2- 172.21.15.84:0/1657990623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6c00cd810 0x7fc6c006fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:16.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.615+0000 7fc6c4dde640 1 -- 172.21.15.84:0/1657990623 >> v1:172.21.15.157:6789/0 conn(0x7fc6c00cd380 legacy=0x7fc6c006f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:16.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.615+0000 7fc6c4dde640 1 -- 172.21.15.84:0/1657990623 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c006bd90 con 0x7fc6c01599c0 2023-10-16T10:04:16.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.615+0000 7fc6c4dde640 1 -- 172.21.15.84:0/1657990623 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc6bc001670 con 0x7fc6c01599c0 2023-10-16T10:04:16.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.615+0000 7fc6c4dde640 1 -- 172.21.15.84:0/1657990623 >> v1:172.21.15.84:6789/0 conn(0x7fc6c01599c0 legacy=0x7fc6c00ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:16.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.615+0000 7fc6c4dde640 1 --2- 172.21.15.84:0/1657990623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ac0039a0 0x7fc6ac003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:16.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.615+0000 7fc6c4dde640 1 --2- 172.21.15.84:0/1657990623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6ac0047c0 0x7fc6ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:16.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.615+0000 7fc6c4dde640 1 --2- 172.21.15.84:0/1657990623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6c00cd810 0x7fc6ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:16.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.615+0000 7fc6c4dde640 1 -- 172.21.15.84:0/1657990623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c006f300 con 0x7fc6ac0047c0 2023-10-16T10:04:16.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.615+0000 7fc6c4dde640 1 -- 172.21.15.84:0/1657990623 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac0054e0 con 0x7fc6ac0039a0 2023-10-16T10:04:16.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.615+0000 7fc6c4dde640 1 -- 172.21.15.84:0/1657990623 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac00bec0 con 0x7fc6c00cd810 2023-10-16T10:04:16.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.615+0000 7fc6c55df640 1 --2- 172.21.15.84:0/1657990623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6ac0047c0 0x7fc6ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:16.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.615+0000 7fc6c5de0640 1 --2- 172.21.15.84:0/1657990623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ac0039a0 0x7fc6ac003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:16.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.616+0000 7fc6c65e1640 1 --2- 172.21.15.84:0/1657990623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6c00cd810 0x7fc6ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:16.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.616+0000 7fc6c55df640 1 -- 172.21.15.84:0/1657990623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6c00cd810 msgr2=0x7fc6ac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:16.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.616+0000 7fc6c55df640 1 --2- 172.21.15.84:0/1657990623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6c00cd810 0x7fc6ac00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:16.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.616+0000 7fc6c55df640 1 -- 172.21.15.84:0/1657990623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ac0039a0 msgr2=0x7fc6ac003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:16.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.616+0000 7fc6c55df640 1 --2- 172.21.15.84:0/1657990623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ac0039a0 0x7fc6ac003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:16.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.616+0000 7fc6c55df640 1 -- 172.21.15.84:0/1657990623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac003520 con 0x7fc6ac0047c0 2023-10-16T10:04:16.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.617+0000 7fc6c55df640 1 --2- 172.21.15.84:0/1657990623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6ac0047c0 0x7fc6ac004c40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0001750 tx=0x7fc6ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f6de59392851b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:16.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.617+0000 7fc6c4dde640 1 -- 172.21.15.84:0/1657990623 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b00052d0 con 0x7fc6ac0047c0 2023-10-16T10:04:16.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.617+0000 7fc6c4dde640 1 -- 172.21.15.84:0/1657990623 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b002e050 con 0x7fc6ac0047c0 2023-10-16T10:04:16.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.617+0000 7fc6c4dde640 1 -- 172.21.15.84:0/1657990623 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b00035f0 con 0x7fc6ac0047c0 2023-10-16T10:04:16.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.618+0000 7fc6c6de2640 1 -- 172.21.15.84:0/1657990623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6ac0047c0 msgr2=0x7fc6ac004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:16.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.618+0000 7fc6c6de2640 1 --2- 172.21.15.84:0/1657990623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6ac0047c0 0x7fc6ac004c40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0001750 tx=0x7fc6ac0057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:16.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.618+0000 7fc6c65e1640 1 -- 172.21.15.84:0/1657990623 reap_dead start 2023-10-16T10:04:16.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.618+0000 7fc6c6de2640 1 -- 172.21.15.84:0/1657990623 shutdown_connections 2023-10-16T10:04:16.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.618+0000 7fc6c6de2640 1 -- 172.21.15.84:0/1657990623 >> 172.21.15.84:0/1657990623 conn(0x7fc6c00b0a90 msgr2=0x7fc6c00ce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:16.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.618+0000 7fc6c6de2640 1 -- 172.21.15.84:0/1657990623 shutdown_connections 2023-10-16T10:04:16.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.618+0000 7fc6c6de2640 1 -- 172.21.15.84:0/1657990623 wait complete. 2023-10-16T10:04:16.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.620+0000 7fc6c6de2640 1 Processor -- start 2023-10-16T10:04:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.620+0000 7fc6c6de2640 1 -- start start 2023-10-16T10:04:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.620+0000 7fc6c6de2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6ac0047c0 0x7fc6c0153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.621+0000 7fc6c6de2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c0157960 0x7fc6c0157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.621+0000 7fc6c6de2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6c015ae60 0x7fc6c015b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.621+0000 7fc6c6de2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c01542b0 con 0x7fc6ac0047c0 2023-10-16T10:04:16.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.621+0000 7fc6c6de2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c015b820 con 0x7fc6c0157960 2023-10-16T10:04:16.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.621+0000 7fc6c6de2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c0154440 con 0x7fc6c015ae60 2023-10-16T10:04:16.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.621+0000 7fc6c5de0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6ac0047c0 0x7fc6c0153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:16.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.621+0000 7fc6c5de0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6ac0047c0 0x7fc6c0153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50856/0 (socket says 172.21.15.84:50856) 2023-10-16T10:04:16.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.621+0000 7fc6c5de0640 1 -- 172.21.15.84:0/1238239772 learned_addr learned my addr 172.21.15.84:0/1238239772 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:16.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.621+0000 7fc6c65e1640 1 --2- 172.21.15.84:0/1238239772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6c015ae60 0x7fc6c015b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:16.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.621+0000 7fc6c5de0640 1 -- 172.21.15.84:0/1238239772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6c015ae60 msgr2=0x7fc6c015b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:16.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.621+0000 7fc6c5de0640 1 --2- 172.21.15.84:0/1238239772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6c015ae60 0x7fc6c015b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:16.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.621+0000 7fc6c55df640 1 --2- 172.21.15.84:0/1238239772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c0157960 0x7fc6c0157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:16.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.621+0000 7fc6c5de0640 1 -- 172.21.15.84:0/1238239772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c0157960 msgr2=0x7fc6c0157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:16.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.621+0000 7fc6c5de0640 1 --2- 172.21.15.84:0/1238239772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c0157960 0x7fc6c0157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:16.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.621+0000 7fc6c5de0640 1 -- 172.21.15.84:0/1238239772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c0158d00 con 0x7fc6ac0047c0 2023-10-16T10:04:16.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.622+0000 7fc6c5de0640 1 --2- 172.21.15.84:0/1238239772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6ac0047c0 0x7fc6c0153d70 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc00e760 tx=0x7fc6bc00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:16.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.622+0000 7fc6b67fc640 1 -- 172.21.15.84:0/1238239772 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc00ee70 con 0x7fc6ac0047c0 2023-10-16T10:04:16.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.622+0000 7fc6b67fc640 1 -- 172.21.15.84:0/1238239772 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6bc00c870 con 0x7fc6ac0047c0 2023-10-16T10:04:16.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.622+0000 7fc6b67fc640 1 -- 172.21.15.84:0/1238239772 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc00cc20 con 0x7fc6ac0047c0 2023-10-16T10:04:16.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.622+0000 7fc6c6de2640 1 -- 172.21.15.84:0/1238239772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c0158460 con 0x7fc6ac0047c0 2023-10-16T10:04:16.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.622+0000 7fc6c6de2640 1 -- 172.21.15.84:0/1238239772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c0158a20 con 0x7fc6ac0047c0 2023-10-16T10:04:16.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.623+0000 7fc6c6de2640 1 -- 172.21.15.84:0/1238239772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c004e9b0 con 0x7fc6ac0047c0 2023-10-16T10:04:16.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.624+0000 7fc6b67fc640 1 -- 172.21.15.84:0/1238239772 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc6bc01d460 con 0x7fc6ac0047c0 2023-10-16T10:04:16.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.625+0000 7fc6b67fc640 1 --2- 172.21.15.84:0/1238239772 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fc6a406afc0 0x7fc6a406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:16.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.625+0000 7fc6b67fc640 1 -- 172.21.15.84:0/1238239772 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5631+0+0 (secure 0 0 0) 0x7fc6bc089710 con 0x7fc6ac0047c0 2023-10-16T10:04:16.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.625+0000 7fc6c55df640 1 --2- 172.21.15.84:0/1238239772 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fc6a406afc0 0x7fc6a406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:16.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.626+0000 7fc6c55df640 1 --2- 172.21.15.84:0/1238239772 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fc6a406afc0 0x7fc6a406d480 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0033000 tx=0x7fc6b0009030 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:16.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.630+0000 7fc6b67fc640 1 -- 172.21.15.84:0/1238239772 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc6bc058180 con 0x7fc6ac0047c0 2023-10-16T10:04:16.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:16.891+0000 7fc6c6de2640 1 -- 172.21.15.84:0/1238239772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7fc6c0071000 con 0x7fc6ac0047c0 2023-10-16T10:04:17.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.028+0000 7fc6b67fc640 1 -- 172.21.15.84:0/1238239772 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v551) v1 ==== 154+0+0 (secure 0 0 0) 0x7fc6bc057d10 con 0x7fc6ac0047c0 2023-10-16T10:04:17.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.031+0000 7fc6c6de2640 1 -- 172.21.15.84:0/1238239772 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fc6a406afc0 msgr2=0x7fc6a406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.031+0000 7fc6c6de2640 1 --2- 172.21.15.84:0/1238239772 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fc6a406afc0 0x7fc6a406d480 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0033000 tx=0x7fc6b0009030 comp rx=0 tx=0).stop 2023-10-16T10:04:17.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.031+0000 7fc6c6de2640 1 -- 172.21.15.84:0/1238239772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6ac0047c0 msgr2=0x7fc6c0153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.031+0000 7fc6c6de2640 1 --2- 172.21.15.84:0/1238239772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6ac0047c0 0x7fc6c0153d70 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc00e760 tx=0x7fc6bc00ec20 comp rx=0 tx=0).stop 2023-10-16T10:04:17.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.031+0000 7fc6c6de2640 1 -- 172.21.15.84:0/1238239772 shutdown_connections 2023-10-16T10:04:17.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.031+0000 7fc6c6de2640 1 --2- 172.21.15.84:0/1238239772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6c015ae60 0x7fc6c015b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.031+0000 7fc6c6de2640 1 --2- 172.21.15.84:0/1238239772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c0157960 0x7fc6c0157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.031+0000 7fc6c6de2640 1 --2- 172.21.15.84:0/1238239772 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fc6a406afc0 0x7fc6a406d480 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.031+0000 7fc6c6de2640 1 --2- 172.21.15.84:0/1238239772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6ac0047c0 0x7fc6c0153d70 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.032+0000 7fc6c6de2640 1 -- 172.21.15.84:0/1238239772 >> 172.21.15.84:0/1238239772 conn(0x7fc6c00b0a90 msgr2=0x7fc6c006f750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:17.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.032+0000 7fc6c6de2640 1 -- 172.21.15.84:0/1238239772 shutdown_connections 2023-10-16T10:04:17.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.032+0000 7fc6c6de2640 1 -- 172.21.15.84:0/1238239772 wait complete. 2023-10-16T10:04:17.051 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2023-10-16T10:04:17.052 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2023-10-16T10:04:17.052 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T10:04:17.053 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:04:17.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.258+0000 7fbefe341640 1 Processor -- start 2023-10-16T10:04:17.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.258+0000 7fbefe341640 1 -- start start 2023-10-16T10:04:17.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.259+0000 7fbefe341640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbef8159b20 0x7fbef806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.259+0000 7fbefe341640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef806f7f0 con 0x7fbef80b0b20 2023-10-16T10:04:17.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.259+0000 7fbefe341640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef806c5e0 con 0x7fbef8159f00 2023-10-16T10:04:17.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.259+0000 7fbefe341640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef806c8c0 con 0x7fbef8159b20 2023-10-16T10:04:17.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.260+0000 7fbefc8de640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbef80b0b20 0x7fbef806fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38542/0 (socket says 172.21.15.84:38542) 2023-10-16T10:04:17.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.260+0000 7fbefc8de640 1 -- 172.21.15.84:0/3446079206 learned_addr learned my addr 172.21.15.84:0/3446079206 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:17.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.260+0000 7fbef7fff640 1 --2- 172.21.15.84:0/3446079206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbef8159b20 0x7fbef806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.260+0000 7fbef6ffd640 1 -- 172.21.15.84:0/3446079206 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562337155 0 0) 0x7fbef806f7f0 con 0x7fbef80b0b20 2023-10-16T10:04:17.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.260+0000 7fbef6ffd640 1 -- 172.21.15.84:0/3446079206 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbee0003520 con 0x7fbef80b0b20 2023-10-16T10:04:17.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.260+0000 7fbef6ffd640 1 -- 172.21.15.84:0/3446079206 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3961890291 0 0) 0x7fbee0003520 con 0x7fbef80b0b20 2023-10-16T10:04:17.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.260+0000 7fbef6ffd640 1 -- 172.21.15.84:0/3446079206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbef8159b20 msgr2=0x7fbef806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:17.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbef6ffd640 1 --2- 172.21.15.84:0/3446079206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbef8159b20 0x7fbef806bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbef6ffd640 1 -- 172.21.15.84:0/3446079206 >> v1:172.21.15.157:6789/0 conn(0x7fbef8159f00 legacy=0x7fbef80b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbef6ffd640 1 -- 172.21.15.84:0/3446079206 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef806cba0 con 0x7fbef80b0b20 2023-10-16T10:04:17.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbef6ffd640 1 -- 172.21.15.84:0/3446079206 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fbef00019f0 con 0x7fbef80b0b20 2023-10-16T10:04:17.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbef6ffd640 1 -- 172.21.15.84:0/3446079206 >> v1:172.21.15.84:6789/0 conn(0x7fbef80b0b20 legacy=0x7fbef806fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbef6ffd640 1 --2- 172.21.15.84:0/3446079206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00039a0 0x7fbee0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbef6ffd640 1 --2- 172.21.15.84:0/3446079206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee00047c0 0x7fbee0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbef6ffd640 1 --2- 172.21.15.84:0/3446079206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbef8159b20 0x7fbee0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbef6ffd640 1 -- 172.21.15.84:0/3446079206 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef806c8c0 con 0x7fbee00039a0 2023-10-16T10:04:17.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbef6ffd640 1 -- 172.21.15.84:0/3446079206 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee000ba30 con 0x7fbee00047c0 2023-10-16T10:04:17.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbef6ffd640 1 -- 172.21.15.84:0/3446079206 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee000de00 con 0x7fbef8159b20 2023-10-16T10:04:17.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbef7fff640 1 --2- 172.21.15.84:0/3446079206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00039a0 0x7fbee0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbefc8de640 1 --2- 172.21.15.84:0/3446079206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbef8159b20 0x7fbee0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbef77fe640 1 --2- 172.21.15.84:0/3446079206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee00047c0 0x7fbee0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbef7fff640 1 -- 172.21.15.84:0/3446079206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbef8159b20 msgr2=0x7fbee0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:17.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbef7fff640 1 --2- 172.21.15.84:0/3446079206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbef8159b20 0x7fbee0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbef7fff640 1 -- 172.21.15.84:0/3446079206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee00047c0 msgr2=0x7fbee0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:17.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbef7fff640 1 --2- 172.21.15.84:0/3446079206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee00047c0 0x7fbee0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.261+0000 7fbef7fff640 1 -- 172.21.15.84:0/3446079206 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee0003520 con 0x7fbee00039a0 2023-10-16T10:04:17.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.262+0000 7fbef7fff640 1 --2- 172.21.15.84:0/3446079206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00039a0 0x7fbee0003dc0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009a90 tx=0x7fbee00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c169245ab6c863 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:17.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.263+0000 7fbef6ffd640 1 -- 172.21.15.84:0/3446079206 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee802c680 con 0x7fbee00039a0 2023-10-16T10:04:17.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.263+0000 7fbef6ffd640 1 -- 172.21.15.84:0/3446079206 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee802c7e0 con 0x7fbee00039a0 2023-10-16T10:04:17.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.263+0000 7fbef6ffd640 1 -- 172.21.15.84:0/3446079206 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee802cb70 con 0x7fbee00039a0 2023-10-16T10:04:17.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.263+0000 7fbefe341640 1 -- 172.21.15.84:0/3446079206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00039a0 msgr2=0x7fbee0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.263+0000 7fbefe341640 1 --2- 172.21.15.84:0/3446079206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00039a0 0x7fbee0003dc0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009a90 tx=0x7fbee00057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.263+0000 7fbefc8de640 1 -- 172.21.15.84:0/3446079206 reap_dead start 2023-10-16T10:04:17.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.263+0000 7fbefe341640 1 -- 172.21.15.84:0/3446079206 shutdown_connections 2023-10-16T10:04:17.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.263+0000 7fbefe341640 1 -- 172.21.15.84:0/3446079206 >> 172.21.15.84:0/3446079206 conn(0x7fbef80cb410 msgr2=0x7fbef80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:17.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.263+0000 7fbefe341640 1 -- 172.21.15.84:0/3446079206 shutdown_connections 2023-10-16T10:04:17.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.263+0000 7fbefe341640 1 -- 172.21.15.84:0/3446079206 wait complete. 2023-10-16T10:04:17.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.265+0000 7fbefe341640 1 Processor -- start 2023-10-16T10:04:17.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.265+0000 7fbefe341640 1 -- start start 2023-10-16T10:04:17.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.266+0000 7fbefe341640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00039a0 0x7fbef815b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.266+0000 7fbefe341640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbef80c6cf0 0x7fbef80c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.266+0000 7fbefe341640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbef814da10 0x7fbef814de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.266+0000 7fbefe341640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef80c8430 con 0x7fbee00039a0 2023-10-16T10:04:17.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.266+0000 7fbefe341640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef814e3d0 con 0x7fbef80c6cf0 2023-10-16T10:04:17.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.266+0000 7fbefe341640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef80c77b0 con 0x7fbef814da10 2023-10-16T10:04:17.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.266+0000 7fbef7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00039a0 0x7fbef815b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.266+0000 7fbef7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00039a0 0x7fbef815b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50882/0 (socket says 172.21.15.84:50882) 2023-10-16T10:04:17.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.266+0000 7fbef7fff640 1 -- 172.21.15.84:0/961161581 learned_addr learned my addr 172.21.15.84:0/961161581 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:17.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.267+0000 7fbef77fe640 1 --2- 172.21.15.84:0/961161581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbef80c6cf0 0x7fbef80c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.267+0000 7fbef7fff640 1 -- 172.21.15.84:0/961161581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbef814da10 msgr2=0x7fbef814de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.267+0000 7fbef7fff640 1 --2- 172.21.15.84:0/961161581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbef814da10 0x7fbef814de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.267+0000 7fbef7fff640 1 -- 172.21.15.84:0/961161581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbef80c6cf0 msgr2=0x7fbef80c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.267+0000 7fbef7fff640 1 --2- 172.21.15.84:0/961161581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbef80c6cf0 0x7fbef80c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.267+0000 7fbef7fff640 1 -- 172.21.15.84:0/961161581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef80c7a90 con 0x7fbee00039a0 2023-10-16T10:04:17.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.267+0000 7fbef7fff640 1 --2- 172.21.15.84:0/961161581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00039a0 0x7fbef815b120 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbee802c220 tx=0x7fbee8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:17.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.267+0000 7fbef4ff9640 1 -- 172.21.15.84:0/961161581 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8003810 con 0x7fbee00039a0 2023-10-16T10:04:17.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.267+0000 7fbef4ff9640 1 -- 172.21.15.84:0/961161581 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee8003970 con 0x7fbee00039a0 2023-10-16T10:04:17.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.267+0000 7fbefe341640 1 -- 172.21.15.84:0/961161581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef80c7d70 con 0x7fbee00039a0 2023-10-16T10:04:17.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.267+0000 7fbef4ff9640 1 -- 172.21.15.84:0/961161581 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8003ce0 con 0x7fbee00039a0 2023-10-16T10:04:17.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.267+0000 7fbefe341640 1 -- 172.21.15.84:0/961161581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef815a5a0 con 0x7fbee00039a0 2023-10-16T10:04:17.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.268+0000 7fbefe341640 1 -- 172.21.15.84:0/961161581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef804e160 con 0x7fbee00039a0 2023-10-16T10:04:17.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.270+0000 7fbef4ff9640 1 -- 172.21.15.84:0/961161581 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbee8034440 con 0x7fbee00039a0 2023-10-16T10:04:17.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.270+0000 7fbef4ff9640 1 --2- 172.21.15.84:0/961161581 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fbee406af90 0x7fbee406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.271+0000 7fbef4ff9640 1 -- 172.21.15.84:0/961161581 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5631+0+0 (secure 0 0 0) 0x7fbee80ab6a0 con 0x7fbee00039a0 2023-10-16T10:04:17.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.271+0000 7fbef77fe640 1 --2- 172.21.15.84:0/961161581 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fbee406af90 0x7fbee406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.271+0000 7fbef77fe640 1 --2- 172.21.15.84:0/961161581 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fbee406af90 0x7fbee406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbef80b48c0 tx=0x7fbeec009040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:17.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.275+0000 7fbef4ff9640 1 -- 172.21.15.84:0/961161581 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbee8076730 con 0x7fbee00039a0 2023-10-16T10:04:17.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.465+0000 7fbefe341640 1 -- 172.21.15.84:0/961161581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbef80b10b0 con 0x7fbee00039a0 2023-10-16T10:04:17.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.468+0000 7fbef4ff9640 1 -- 172.21.15.84:0/961161581 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+13958 (secure 0 0 0) 0x7fbee80044e0 con 0x7fbee00039a0 2023-10-16T10:04:17.469 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:17.470 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":73,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:04:15.005194+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6837/3839442164":"2023-10-17T10:04:11.960899+0000","172.21.15.84:6835/3839442164":"2023-10-17T10:04:11.960899+0000","172.21.15.84:6832/633652932":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/724237746":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/3806126461":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/3926486342":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/3656680984":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/2816959180":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/4150953381":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/351541199":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/555434621":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/3440750653":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/3623389468":"2023-10-17T10:04:08.229109+0000","172.21.15.84:0/3654971153":"2023-10-17T10:02:27.969914+0000","172.21.15.84:0/621840141":"2023-10-17T10:02:27.969914+0000","172.21.15.84:6833/585186562":"2023-10-17T10:02:27.969914+0000","172.21.15.84:6833/633652932":"2023-10-17T10:04:08.229109+0000","172.21.15.84:6832/585186562":"2023-10-17T10:02:27.969914+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:04:17.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.470+0000 7fbefe341640 1 -- 172.21.15.84:0/961161581 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fbee406af90 msgr2=0x7fbee406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.470+0000 7fbefe341640 1 --2- 172.21.15.84:0/961161581 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fbee406af90 0x7fbee406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbef80b48c0 tx=0x7fbeec009040 comp rx=0 tx=0).stop 2023-10-16T10:04:17.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.470+0000 7fbefe341640 1 -- 172.21.15.84:0/961161581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00039a0 msgr2=0x7fbef815b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.470+0000 7fbefe341640 1 --2- 172.21.15.84:0/961161581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00039a0 0x7fbef815b120 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbee802c220 tx=0x7fbee8004730 comp rx=0 tx=0).stop 2023-10-16T10:04:17.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.471+0000 7fbefe341640 1 -- 172.21.15.84:0/961161581 shutdown_connections 2023-10-16T10:04:17.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.471+0000 7fbefe341640 1 --2- 172.21.15.84:0/961161581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbef814da10 0x7fbef814de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.471+0000 7fbefe341640 1 --2- 172.21.15.84:0/961161581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbef80c6cf0 0x7fbef80c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.471+0000 7fbefe341640 1 --2- 172.21.15.84:0/961161581 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fbee406af90 0x7fbee406d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.471+0000 7fbefe341640 1 --2- 172.21.15.84:0/961161581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00039a0 0x7fbef815b120 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.471+0000 7fbefe341640 1 -- 172.21.15.84:0/961161581 >> 172.21.15.84:0/961161581 conn(0x7fbef80cb410 msgr2=0x7fbef815d910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.471+0000 7fbefe341640 1 -- 172.21.15.84:0/961161581 shutdown_connections 2023-10-16T10:04:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.471+0000 7fbefe341640 1 -- 172.21.15.84:0/961161581 wait complete. 2023-10-16T10:04:17.490 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:04:17.491 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:04:17.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.702+0000 7f2591244640 1 Processor -- start 2023-10-16T10:04:17.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.702+0000 7f2591244640 1 -- start start 2023-10-16T10:04:17.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.703+0000 7f2591244640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f258c06a4b0 0x7f258c0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.704+0000 7f2591244640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f258c0c9ca0 0x7f258c0ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.704+0000 7f2591244640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258c0cb030 con 0x7f258c15ad70 2023-10-16T10:04:17.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.704+0000 7f2591244640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258c0cae30 con 0x7f258c06a4b0 2023-10-16T10:04:17.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.704+0000 7f2591244640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258c0c91c0 con 0x7f258c0c9ca0 2023-10-16T10:04:17.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.704+0000 7f258ad9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f258c15ad70 0x7f258c069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38548/0 (socket says 172.21.15.84:38548) 2023-10-16T10:04:17.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.704+0000 7f258ad9d640 1 -- 172.21.15.84:0/4220156106 learned_addr learned my addr 172.21.15.84:0/4220156106 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:17.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.704+0000 7f258a59c640 1 --2- 172.21.15.84:0/4220156106 >> v2:172.21.15.84:3300/0 conn(0x7f258c06a4b0 0x7f258c0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.704+0000 7f258b59e640 1 --2- 172.21.15.84:0/4220156106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f258c0c9ca0 0x7f258c0ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.704+0000 7f2589d9b640 1 -- 172.21.15.84:0/4220156106 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958924914 0 0) 0x7f258c0cb030 con 0x7f258c15ad70 2023-10-16T10:04:17.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.704+0000 7f2589d9b640 1 -- 172.21.15.84:0/4220156106 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2574003520 con 0x7f258c15ad70 2023-10-16T10:04:17.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.704+0000 7f258a59c640 1 -- 172.21.15.84:0/4220156106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f258c0c9ca0 msgr2=0x7f258c0ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:17.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.704+0000 7f258a59c640 1 --2- 172.21.15.84:0/4220156106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f258c0c9ca0 0x7f258c0ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.704+0000 7f258a59c640 1 -- 172.21.15.84:0/4220156106 >> v1:172.21.15.84:6789/0 conn(0x7f258c15ad70 legacy=0x7f258c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.704+0000 7f258a59c640 1 -- 172.21.15.84:0/4220156106 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c0c94a0 con 0x7f258c06a4b0 2023-10-16T10:04:17.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.705+0000 7f258a59c640 1 --2- 172.21.15.84:0/4220156106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f258c06a4b0 0x7f258c0ca3b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f25800097a0 tx=0x7f258002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7af121faeec9ba server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:17.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.706+0000 7f2589d9b640 1 -- 172.21.15.84:0/4220156106 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258002d040 con 0x7f258c06a4b0 2023-10-16T10:04:17.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.706+0000 7f2589d9b640 1 -- 172.21.15.84:0/4220156106 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2580031810 con 0x7f258c06a4b0 2023-10-16T10:04:17.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.706+0000 7f2589d9b640 1 -- 172.21.15.84:0/4220156106 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2580031ba0 con 0x7f258c06a4b0 2023-10-16T10:04:17.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.706+0000 7f2591244640 1 -- 172.21.15.84:0/4220156106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f258c06a4b0 msgr2=0x7f258c0ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.706+0000 7f2591244640 1 --2- 172.21.15.84:0/4220156106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f258c06a4b0 0x7f258c0ca3b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f25800097a0 tx=0x7f258002b8b0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.706+0000 7f2591244640 1 -- 172.21.15.84:0/4220156106 shutdown_connections 2023-10-16T10:04:17.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.706+0000 7f2591244640 1 --2- 172.21.15.84:0/4220156106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f258c0c9ca0 0x7f258c0ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.706+0000 7f2591244640 1 --2- 172.21.15.84:0/4220156106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f258c06a4b0 0x7f258c0ca3b0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.706+0000 7f2591244640 1 -- 172.21.15.84:0/4220156106 >> 172.21.15.84:0/4220156106 conn(0x7f258c0ce9b0 msgr2=0x7f258c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:17.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.706+0000 7f2591244640 1 -- 172.21.15.84:0/4220156106 shutdown_connections 2023-10-16T10:04:17.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.706+0000 7f2591244640 1 -- 172.21.15.84:0/4220156106 wait complete. 2023-10-16T10:04:17.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.708+0000 7f2591244640 1 Processor -- start 2023-10-16T10:04:17.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.708+0000 7f2591244640 1 -- start start 2023-10-16T10:04:17.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.709+0000 7f2591244640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f258c06a4b0 0x7f258c0c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.709+0000 7f2591244640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f258c0c9ca0 0x7f258c0c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.709+0000 7f2591244640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f258c15ad70 0x7f258c15cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.709+0000 7f2591244640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c14a510 con 0x7f258c06a4b0 2023-10-16T10:04:17.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.709+0000 7f2591244640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c149c00 con 0x7f258c15ad70 2023-10-16T10:04:17.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.709+0000 7f2591244640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258c149ec0 con 0x7f258c0c9ca0 2023-10-16T10:04:17.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.709+0000 7f258ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f258c06a4b0 0x7f258c0c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.709+0000 7f258ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f258c06a4b0 0x7f258c0c8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50910/0 (socket says 172.21.15.84:50910) 2023-10-16T10:04:17.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.709+0000 7f258ad9d640 1 -- 172.21.15.84:0/354432543 learned_addr learned my addr 172.21.15.84:0/354432543 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:17.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.709+0000 7f258ad9d640 1 -- 172.21.15.84:0/354432543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f258c0c9ca0 msgr2=0x7f258c0c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.709+0000 7f258ad9d640 1 --2- 172.21.15.84:0/354432543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f258c0c9ca0 0x7f258c0c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.709+0000 7f258ad9d640 1 -- 172.21.15.84:0/354432543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f258c15ad70 msgr2=0x7f258c15cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.709+0000 7f258ad9d640 1 --2- 172.21.15.84:0/354432543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f258c15ad70 0x7f258c15cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.709+0000 7f258ad9d640 1 -- 172.21.15.84:0/354432543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2580009400 con 0x7f258c06a4b0 2023-10-16T10:04:17.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.710+0000 7f258ad9d640 1 --2- 172.21.15.84:0/354432543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f258c06a4b0 0x7f258c0c8c30 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f257c00c400 tx=0x7f257c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:17.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.710+0000 7f25737fe640 1 -- 172.21.15.84:0/354432543 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c005ce0 con 0x7f258c06a4b0 2023-10-16T10:04:17.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.710+0000 7f25737fe640 1 -- 172.21.15.84:0/354432543 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c005e40 con 0x7f258c06a4b0 2023-10-16T10:04:17.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.710+0000 7f2591244640 1 -- 172.21.15.84:0/354432543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258c14a1a0 con 0x7f258c06a4b0 2023-10-16T10:04:17.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.710+0000 7f25737fe640 1 -- 172.21.15.84:0/354432543 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c007310 con 0x7f258c06a4b0 2023-10-16T10:04:17.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.710+0000 7f2591244640 1 -- 172.21.15.84:0/354432543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258c0c7120 con 0x7f258c06a4b0 2023-10-16T10:04:17.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.711+0000 7f2591244640 1 -- 172.21.15.84:0/354432543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f258c04e160 con 0x7f258c06a4b0 2023-10-16T10:04:17.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.712+0000 7f25737fe640 1 -- 172.21.15.84:0/354432543 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f257c007470 con 0x7f258c06a4b0 2023-10-16T10:04:17.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.713+0000 7f25737fe640 1 --2- 172.21.15.84:0/354432543 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f257806af90 0x7f257806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.713+0000 7f25737fe640 1 -- 172.21.15.84:0/354432543 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f257c088c60 con 0x7f258c06a4b0 2023-10-16T10:04:17.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.713+0000 7f258a59c640 1 --2- 172.21.15.84:0/354432543 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f257806af90 0x7f257806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.714+0000 7f258a59c640 1 --2- 172.21.15.84:0/354432543 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f257806af90 0x7f257806d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2580036000 tx=0x7f2580030000 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:17.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.717+0000 7f25737fe640 1 -- 172.21.15.84:0/354432543 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f257c053bb0 con 0x7f258c06a4b0 2023-10-16T10:04:17.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.910+0000 7f2591244640 1 -- 172.21.15.84:0/354432543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f258c0c7120 con 0x7f258c06a4b0 2023-10-16T10:04:17.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.911+0000 7f25737fe640 1 -- 172.21.15.84:0/354432543 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f258c0c7120 con 0x7f258c06a4b0 2023-10-16T10:04:17.912 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:04:17.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.913+0000 7f2591244640 1 -- 172.21.15.84:0/354432543 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f257806af90 msgr2=0x7f257806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.913+0000 7f2591244640 1 --2- 172.21.15.84:0/354432543 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f257806af90 0x7f257806d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2580036000 tx=0x7f2580030000 comp rx=0 tx=0).stop 2023-10-16T10:04:17.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.913+0000 7f2591244640 1 -- 172.21.15.84:0/354432543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f258c06a4b0 msgr2=0x7f258c0c8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.913+0000 7f2591244640 1 --2- 172.21.15.84:0/354432543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f258c06a4b0 0x7f258c0c8c30 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f257c00c400 tx=0x7f257c00c8c0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.913+0000 7f2591244640 1 -- 172.21.15.84:0/354432543 shutdown_connections 2023-10-16T10:04:17.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.913+0000 7f2591244640 1 --2- 172.21.15.84:0/354432543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f258c15ad70 0x7f258c15cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.913+0000 7f2591244640 1 --2- 172.21.15.84:0/354432543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f258c0c9ca0 0x7f258c0c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.913+0000 7f2591244640 1 --2- 172.21.15.84:0/354432543 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f257806af90 0x7f257806d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.913+0000 7f2591244640 1 --2- 172.21.15.84:0/354432543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f258c06a4b0 0x7f258c0c8c30 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.913+0000 7f2591244640 1 -- 172.21.15.84:0/354432543 >> 172.21.15.84:0/354432543 conn(0x7f258c0ce9b0 msgr2=0x7f258c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:17.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.913+0000 7f2591244640 1 -- 172.21.15.84:0/354432543 shutdown_connections 2023-10-16T10:04:17.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:17.914+0000 7f2591244640 1 -- 172.21.15.84:0/354432543 wait complete. 2023-10-16T10:04:17.932 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-10-16T10:04:18.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.141+0000 7f65fee15640 1 Processor -- start 2023-10-16T10:04:18.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.142+0000 7f65fee15640 1 -- start start 2023-10-16T10:04:18.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.143+0000 7f65fee15640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f65f80cd0c0 0x7f65f81615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:18.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.143+0000 7f65fee15640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f815db00 0x7f65f80b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:18.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.143+0000 7f65fee15640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f815e3e0 con 0x7f65f80b0410 2023-10-16T10:04:18.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.143+0000 7f65fee15640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f80b0de0 con 0x7f65f80cd0c0 2023-10-16T10:04:18.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.143+0000 7f65fee15640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80cd4a0 con 0x7f65f815db00 2023-10-16T10:04:18.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.143+0000 7f65fcbb1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f65f80cd0c0 0x7f65f81615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:18.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.143+0000 7f65f7fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f65f80b0410 0x7f65f815d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38556/0 (socket says 172.21.15.84:38556) 2023-10-16T10:04:18.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.143+0000 7f65f7fff640 1 -- 172.21.15.84:0/955886190 learned_addr learned my addr 172.21.15.84:0/955886190 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:18.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.144+0000 7f65fd3b2640 1 --2- 172.21.15.84:0/955886190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f815db00 0x7f65f80b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:18.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.144+0000 7f65fcbb1640 1 -- 172.21.15.84:0/955886190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f815db00 msgr2=0x7f65f80b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:18.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.144+0000 7f65fcbb1640 1 --2- 172.21.15.84:0/955886190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f815db00 0x7f65f80b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:18.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.144+0000 7f65fcbb1640 1 -- 172.21.15.84:0/955886190 >> v1:172.21.15.84:6789/0 conn(0x7f65f80b0410 legacy=0x7f65f815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:18.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.144+0000 7f65fcbb1640 1 -- 172.21.15.84:0/955886190 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f815e100 con 0x7f65f80cd0c0 2023-10-16T10:04:18.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.144+0000 7f65f77fe640 1 -- 172.21.15.84:0/955886190 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273628772 0 0) 0x7f65f815e3e0 con 0x7f65f80b0410 2023-10-16T10:04:18.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.144+0000 7f65fd3b2640 1 --2- 172.21.15.84:0/955886190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f815db00 0x7f65f80b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:18.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.145+0000 7f65fcbb1640 1 --2- 172.21.15.84:0/955886190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f80cd0c0 0x7f65f81615f0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009d90 tx=0x7f65e802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72152da5d0b403b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:18.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.145+0000 7f65f77fe640 1 -- 172.21.15.84:0/955886190 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e802cbd0 con 0x7f65f80cd0c0 2023-10-16T10:04:18.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.145+0000 7f65f77fe640 1 -- 172.21.15.84:0/955886190 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e802cd30 con 0x7f65f80cd0c0 2023-10-16T10:04:18.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.146+0000 7f65f77fe640 1 -- 172.21.15.84:0/955886190 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e802cea0 con 0x7f65f80cd0c0 2023-10-16T10:04:18.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.146+0000 7f65fee15640 1 -- 172.21.15.84:0/955886190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f80cd0c0 msgr2=0x7f65f81615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:18.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.146+0000 7f65fee15640 1 --2- 172.21.15.84:0/955886190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f80cd0c0 0x7f65f81615f0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009d90 tx=0x7f65e802be20 comp rx=0 tx=0).stop 2023-10-16T10:04:18.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.146+0000 7f65fee15640 1 -- 172.21.15.84:0/955886190 shutdown_connections 2023-10-16T10:04:18.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.146+0000 7f65fee15640 1 --2- 172.21.15.84:0/955886190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f815db00 0x7f65f80b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:18.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.146+0000 7f65fee15640 1 --2- 172.21.15.84:0/955886190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f80cd0c0 0x7f65f81615f0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:18.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.146+0000 7f65fee15640 1 -- 172.21.15.84:0/955886190 >> 172.21.15.84:0/955886190 conn(0x7f65f80cb410 msgr2=0x7f65f80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:18.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.146+0000 7f65fee15640 1 -- 172.21.15.84:0/955886190 shutdown_connections 2023-10-16T10:04:18.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.146+0000 7f65fee15640 1 -- 172.21.15.84:0/955886190 wait complete. 2023-10-16T10:04:18.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.148+0000 7f65fee15640 1 Processor -- start 2023-10-16T10:04:18.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.148+0000 7f65fee15640 1 -- start start 2023-10-16T10:04:18.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.149+0000 7f65fee15640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f80b0410 0x7f65f81f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:18.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.149+0000 7f65fee15640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f80cd0c0 0x7f65f815f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:18.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.149+0000 7f65fee15640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f815db00 0x7f65f80b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:18.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.149+0000 7f65fee15640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f815e6c0 con 0x7f65f815db00 2023-10-16T10:04:18.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.149+0000 7f65fee15640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f815e9a0 con 0x7f65f80cd0c0 2023-10-16T10:04:18.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.149+0000 7f65fee15640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f815ec80 con 0x7f65f80b0410 2023-10-16T10:04:18.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.149+0000 7f65fd3b2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f815db00 0x7f65f80b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:18.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.149+0000 7f65fd3b2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f815db00 0x7f65f80b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50934/0 (socket says 172.21.15.84:50934) 2023-10-16T10:04:18.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.149+0000 7f65fd3b2640 1 -- 172.21.15.84:0/2288191308 learned_addr learned my addr 172.21.15.84:0/2288191308 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:18.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.150+0000 7f65f7fff640 1 --2- 172.21.15.84:0/2288191308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f80cd0c0 0x7f65f815f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:18.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.150+0000 7f65fcbb1640 1 --2- 172.21.15.84:0/2288191308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f80b0410 0x7f65f81f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:18.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.150+0000 7f65fd3b2640 1 -- 172.21.15.84:0/2288191308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f80b0410 msgr2=0x7f65f81f5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:18.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.150+0000 7f65fd3b2640 1 --2- 172.21.15.84:0/2288191308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f80b0410 0x7f65f81f5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:18.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.150+0000 7f65fd3b2640 1 -- 172.21.15.84:0/2288191308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f80cd0c0 msgr2=0x7f65f815f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:18.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.150+0000 7f65fd3b2640 1 --2- 172.21.15.84:0/2288191308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f80cd0c0 0x7f65f815f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:18.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.150+0000 7f65fd3b2640 1 -- 172.21.15.84:0/2288191308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e8009a70 con 0x7f65f815db00 2023-10-16T10:04:18.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.150+0000 7f65fd3b2640 1 --2- 172.21.15.84:0/2288191308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f815db00 0x7f65f80b1590 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f65f000c7f0 tx=0x7f65f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:18.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.150+0000 7f65f57fa640 1 -- 172.21.15.84:0/2288191308 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f0007820 con 0x7f65f815db00 2023-10-16T10:04:18.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.150+0000 7f65f57fa640 1 -- 172.21.15.84:0/2288191308 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f0007980 con 0x7f65f815db00 2023-10-16T10:04:18.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.150+0000 7f65fee15640 1 -- 172.21.15.84:0/2288191308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f815ef60 con 0x7f65f815db00 2023-10-16T10:04:18.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.151+0000 7f65f57fa640 1 -- 172.21.15.84:0/2288191308 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f0007d70 con 0x7f65f815db00 2023-10-16T10:04:18.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.151+0000 7f65fee15640 1 -- 172.21.15.84:0/2288191308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f81f61e0 con 0x7f65f815db00 2023-10-16T10:04:18.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.152+0000 7f65fee15640 1 -- 172.21.15.84:0/2288191308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f815f480 con 0x7f65f815db00 2023-10-16T10:04:18.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.153+0000 7f65f57fa640 1 -- 172.21.15.84:0/2288191308 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f65f00113f0 con 0x7f65f815db00 2023-10-16T10:04:18.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.154+0000 7f65f57fa640 1 --2- 172.21.15.84:0/2288191308 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f65d806af90 0x7f65d806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:18.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.154+0000 7f65f57fa640 1 -- 172.21.15.84:0/2288191308 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f65f0089ba0 con 0x7f65f815db00 2023-10-16T10:04:18.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.154+0000 7f65fcbb1640 1 --2- 172.21.15.84:0/2288191308 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f65d806af90 0x7f65d806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:18.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.155+0000 7f65fcbb1640 1 --2- 172.21.15.84:0/2288191308 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f65d806af90 0x7f65d806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f65e8036000 tx=0x7f65e8030000 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:18.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.159+0000 7f65f57fa640 1 -- 172.21.15.84:0/2288191308 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f65f815f480 con 0x7f65f815db00 2023-10-16T10:04:18.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:18.422+0000 7f65fee15640 1 -- 172.21.15.84:0/2288191308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f65f815f480 con 0x7f65f815db00 2023-10-16T10:04:19.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.043+0000 7f65f57fa640 1 -- 172.21.15.84:0/2288191308 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v553) v1 ==== 159+0+0 (secure 0 0 0) 0x7f65f815f480 con 0x7f65f815db00 2023-10-16T10:04:19.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.047+0000 7f65fee15640 1 -- 172.21.15.84:0/2288191308 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f65d806af90 msgr2=0x7f65d806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:19.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.047+0000 7f65fee15640 1 --2- 172.21.15.84:0/2288191308 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f65d806af90 0x7f65d806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f65e8036000 tx=0x7f65e8030000 comp rx=0 tx=0).stop 2023-10-16T10:04:19.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.047+0000 7f65fee15640 1 -- 172.21.15.84:0/2288191308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f815db00 msgr2=0x7f65f80b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:19.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.047+0000 7f65fee15640 1 --2- 172.21.15.84:0/2288191308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f815db00 0x7f65f80b1590 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f65f000c7f0 tx=0x7f65f000ccb0 comp rx=0 tx=0).stop 2023-10-16T10:04:19.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.047+0000 7f65fee15640 1 -- 172.21.15.84:0/2288191308 shutdown_connections 2023-10-16T10:04:19.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.047+0000 7f65fee15640 1 --2- 172.21.15.84:0/2288191308 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f65d806af90 0x7f65d806d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:19.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.047+0000 7f65fee15640 1 --2- 172.21.15.84:0/2288191308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f815db00 0x7f65f80b1590 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:19.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.047+0000 7f65fee15640 1 --2- 172.21.15.84:0/2288191308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f80cd0c0 0x7f65f815f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:19.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.047+0000 7f65fee15640 1 --2- 172.21.15.84:0/2288191308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f80b0410 0x7f65f81f5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:19.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.047+0000 7f65fee15640 1 -- 172.21.15.84:0/2288191308 >> 172.21.15.84:0/2288191308 conn(0x7f65f80cb410 msgr2=0x7f65f815b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:19.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.048+0000 7f65fee15640 1 -- 172.21.15.84:0/2288191308 shutdown_connections 2023-10-16T10:04:19.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.048+0000 7f65fee15640 1 -- 172.21.15.84:0/2288191308 wait complete. 2023-10-16T10:04:19.070 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T10:04:19.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.286+0000 7f01ba4a2640 1 Processor -- start 2023-10-16T10:04:19.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.286+0000 7f01ba4a2640 1 -- start start 2023-10-16T10:04:19.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.288+0000 7f01ba4a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01b415ad70 0x7f01b4069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:19.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.288+0000 7f01ba4a2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f01b40caac0 0x7f01b406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:19.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.288+0000 7f01ba4a2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b40cb030 con 0x7f01b40c9ca0 2023-10-16T10:04:19.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.288+0000 7f01ba4a2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b40c91c0 con 0x7f01b40caac0 2023-10-16T10:04:19.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.288+0000 7f01ba4a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40c94a0 con 0x7f01b415ad70 2023-10-16T10:04:19.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.288+0000 7f01abfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f01b40c9ca0 0x7f01b40ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38572/0 (socket says 172.21.15.84:38572) 2023-10-16T10:04:19.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.288+0000 7f01abfff640 1 -- 172.21.15.84:0/4084296718 learned_addr learned my addr 172.21.15.84:0/4084296718 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:19.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.288+0000 7f01b3fff640 1 --2- 172.21.15.84:0/4084296718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01b415ad70 0x7f01b4069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:19.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.288+0000 7f01b8a3f640 1 --2- 172.21.15.84:0/4084296718 >> v2:172.21.15.157:3300/0 conn(0x7f01b40caac0 0x7f01b406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:19.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.288+0000 7f01b37fe640 1 -- 172.21.15.84:0/4084296718 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671586399 0 0) 0x7f01b40cb030 con 0x7f01b40c9ca0 2023-10-16T10:04:19.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.289+0000 7f01b37fe640 1 -- 172.21.15.84:0/4084296718 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01a0003520 con 0x7f01b40c9ca0 2023-10-16T10:04:19.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.289+0000 7f01b37fe640 1 -- 172.21.15.84:0/4084296718 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 870909995 0 0) 0x7f01a0003520 con 0x7f01b40c9ca0 2023-10-16T10:04:19.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.289+0000 7f01b37fe640 1 -- 172.21.15.84:0/4084296718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01b415ad70 msgr2=0x7f01b4069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:19.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.289+0000 7f01b37fe640 1 --2- 172.21.15.84:0/4084296718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01b415ad70 0x7f01b4069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:19.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.289+0000 7f01b37fe640 1 -- 172.21.15.84:0/4084296718 >> v2:172.21.15.157:3300/0 conn(0x7f01b40caac0 msgr2=0x7f01b406a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:19.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.289+0000 7f01b37fe640 1 --2- 172.21.15.84:0/4084296718 >> v2:172.21.15.157:3300/0 conn(0x7f01b40caac0 0x7f01b406a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:19.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.289+0000 7f01b37fe640 1 -- 172.21.15.84:0/4084296718 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b40c9780 con 0x7f01b40c9ca0 2023-10-16T10:04:19.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.289+0000 7f01b37fe640 1 -- 172.21.15.84:0/4084296718 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f019c001670 con 0x7f01b40c9ca0 2023-10-16T10:04:19.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.289+0000 7f01b3fff640 1 --2- 172.21.15.84:0/4084296718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01b415ad70 0x7f01b4069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:19.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.289+0000 7f01b37fe640 1 -- 172.21.15.84:0/4084296718 >> v1:172.21.15.84:6789/0 conn(0x7f01b40c9ca0 legacy=0x7f01b40ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:19.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.289+0000 7f01b37fe640 1 --2- 172.21.15.84:0/4084296718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01a0003970 0x7f01a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:19.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.289+0000 7f01b37fe640 1 --2- 172.21.15.84:0/4084296718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01b415ad70 0x7f01a00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:19.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.289+0000 7f01b37fe640 1 --2- 172.21.15.84:0/4084296718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a0004d00 0x7f01a0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:19.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.289+0000 7f01b37fe640 1 -- 172.21.15.84:0/4084296718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40c91c0 con 0x7f01a0003970 2023-10-16T10:04:19.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.289+0000 7f01b37fe640 1 -- 172.21.15.84:0/4084296718 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40c94a0 con 0x7f01a0004d00 2023-10-16T10:04:19.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.289+0000 7f01b37fe640 1 -- 172.21.15.84:0/4084296718 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a000ba00 con 0x7f01b415ad70 2023-10-16T10:04:19.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.289+0000 7f01b3fff640 1 --2- 172.21.15.84:0/4084296718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01a0003970 0x7f01a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:19.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.290+0000 7f01abfff640 1 --2- 172.21.15.84:0/4084296718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01b415ad70 0x7f01a00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:19.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.290+0000 7f01b8a3f640 1 --2- 172.21.15.84:0/4084296718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a0004d00 0x7f01a0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:19.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.290+0000 7f01b3fff640 1 -- 172.21.15.84:0/4084296718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01b415ad70 msgr2=0x7f01a00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:19.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.290+0000 7f01b3fff640 1 --2- 172.21.15.84:0/4084296718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01b415ad70 0x7f01a00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:19.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.290+0000 7f01b3fff640 1 -- 172.21.15.84:0/4084296718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a0004d00 msgr2=0x7f01a0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:19.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.290+0000 7f01b3fff640 1 --2- 172.21.15.84:0/4084296718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a0004d00 0x7f01a0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:19.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.290+0000 7f01b3fff640 1 -- 172.21.15.84:0/4084296718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a0003520 con 0x7f01a0003970 2023-10-16T10:04:19.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.290+0000 7f01b8a3f640 1 --2- 172.21.15.84:0/4084296718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a0004d00 0x7f01a0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:19.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.291+0000 7f01b3fff640 1 --2- 172.21.15.84:0/4084296718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01a0003970 0x7f01a0003d90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f01a40098a0 tx=0x7f01a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46b026cc933092 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:19.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.291+0000 7f01b37fe640 1 -- 172.21.15.84:0/4084296718 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4004230 con 0x7f01a0003970 2023-10-16T10:04:19.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.291+0000 7f01b37fe640 1 -- 172.21.15.84:0/4084296718 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a4004390 con 0x7f01a0003970 2023-10-16T10:04:19.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.291+0000 7f01b37fe640 1 -- 172.21.15.84:0/4084296718 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4003830 con 0x7f01a0003970 2023-10-16T10:04:19.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.292+0000 7f01ba4a2640 1 -- 172.21.15.84:0/4084296718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01a0003970 msgr2=0x7f01a0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:19.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.292+0000 7f01ba4a2640 1 --2- 172.21.15.84:0/4084296718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01a0003970 0x7f01a0003d90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f01a40098a0 tx=0x7f01a00057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:19.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.292+0000 7f01b8a3f640 1 -- 172.21.15.84:0/4084296718 reap_dead start 2023-10-16T10:04:19.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.292+0000 7f01ba4a2640 1 -- 172.21.15.84:0/4084296718 shutdown_connections 2023-10-16T10:04:19.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.292+0000 7f01ba4a2640 1 -- 172.21.15.84:0/4084296718 >> 172.21.15.84:0/4084296718 conn(0x7f01b40ce9b0 msgr2=0x7f01b40cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:19.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.292+0000 7f01ba4a2640 1 -- 172.21.15.84:0/4084296718 shutdown_connections 2023-10-16T10:04:19.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.292+0000 7f01ba4a2640 1 -- 172.21.15.84:0/4084296718 wait complete. 2023-10-16T10:04:19.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.294+0000 7f01ba4a2640 1 Processor -- start 2023-10-16T10:04:19.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.294+0000 7f01ba4a2640 1 -- start start 2023-10-16T10:04:19.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.295+0000 7f01ba4a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a0003970 0x7f01b415b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:19.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.295+0000 7f01ba4a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01b40b36f0 0x7f01b415cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:19.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.295+0000 7f01ba4a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01b40b3b70 0x7f01b415bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:19.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.295+0000 7f01ba4a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40c5450 con 0x7f01b40b3b70 2023-10-16T10:04:19.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.295+0000 7f01ba4a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40c4840 con 0x7f01a0003970 2023-10-16T10:04:19.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.295+0000 7f01ba4a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40c4b20 con 0x7f01b40b36f0 2023-10-16T10:04:19.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.295+0000 7f01b8a3f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01b40b3b70 0x7f01b415bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:19.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.295+0000 7f01b8a3f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01b40b3b70 0x7f01b415bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50944/0 (socket says 172.21.15.84:50944) 2023-10-16T10:04:19.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.295+0000 7f01b8a3f640 1 -- 172.21.15.84:0/3220504323 learned_addr learned my addr 172.21.15.84:0/3220504323 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:19.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.295+0000 7f01abfff640 1 --2- 172.21.15.84:0/3220504323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01b40b36f0 0x7f01b415cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:19.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.295+0000 7f01b8a3f640 1 -- 172.21.15.84:0/3220504323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01b40b36f0 msgr2=0x7f01b415cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:19.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.295+0000 7f01b8a3f640 1 --2- 172.21.15.84:0/3220504323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01b40b36f0 0x7f01b415cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:19.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.295+0000 7f01b8a3f640 1 -- 172.21.15.84:0/3220504323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a0003970 msgr2=0x7f01b415b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:19.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.295+0000 7f01b8a3f640 1 --2- 172.21.15.84:0/3220504323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a0003970 0x7f01b415b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:19.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.295+0000 7f01b8a3f640 1 -- 172.21.15.84:0/3220504323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b40c4e00 con 0x7f01b40b3b70 2023-10-16T10:04:19.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.295+0000 7f01abfff640 1 --2- 172.21.15.84:0/3220504323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01b40b36f0 0x7f01b415cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:19.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.296+0000 7f01b8a3f640 1 --2- 172.21.15.84:0/3220504323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01b40b3b70 0x7f01b415bce0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f01b40cf780 tx=0x7f01ac009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:19.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.296+0000 7f01b17fa640 1 -- 172.21.15.84:0/3220504323 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac007cf0 con 0x7f01b40b3b70 2023-10-16T10:04:19.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.296+0000 7f01b17fa640 1 -- 172.21.15.84:0/3220504323 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac007e70 con 0x7f01b40b3b70 2023-10-16T10:04:19.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.296+0000 7f01b17fa640 1 -- 172.21.15.84:0/3220504323 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac014620 con 0x7f01b40b3b70 2023-10-16T10:04:19.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.296+0000 7f01ba4a2640 1 -- 172.21.15.84:0/3220504323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b40c50e0 con 0x7f01b40b3b70 2023-10-16T10:04:19.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.296+0000 7f01ba4a2640 1 -- 172.21.15.84:0/3220504323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b415d450 con 0x7f01b40b3b70 2023-10-16T10:04:19.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.297+0000 7f01ba4a2640 1 -- 172.21.15.84:0/3220504323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b404e160 con 0x7f01b40b3b70 2023-10-16T10:04:19.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.298+0000 7f01b17fa640 1 -- 172.21.15.84:0/3220504323 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f01ac01d3b0 con 0x7f01b40b3b70 2023-10-16T10:04:19.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.299+0000 7f01b17fa640 1 --2- 172.21.15.84:0/3220504323 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f019406af90 0x7f019406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:19.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.299+0000 7f01b17fa640 1 -- 172.21.15.84:0/3220504323 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f01ac089400 con 0x7f01b40b3b70 2023-10-16T10:04:19.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.299+0000 7f01b3fff640 1 --2- 172.21.15.84:0/3220504323 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f019406af90 0x7f019406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:19.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.300+0000 7f01b3fff640 1 --2- 172.21.15.84:0/3220504323 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f019406af90 0x7f019406d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f01a4035000 tx=0x7f01a4030000 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:19.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.304+0000 7f01b17fa640 1 -- 172.21.15.84:0/3220504323 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f01ac0543f0 con 0x7f01b40b3b70 2023-10-16T10:04:19.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.500+0000 7f01ba4a2640 1 -- 172.21.15.84:0/3220504323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f01b40cef60 con 0x7f01b40b3b70 2023-10-16T10:04:19.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.508+0000 7f01b17fa640 1 -- 172.21.15.84:0/3220504323 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f01ac057a10 con 0x7f01b40b3b70 2023-10-16T10:04:19.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.510+0000 7f01ba4a2640 1 -- 172.21.15.84:0/3220504323 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f019406af90 msgr2=0x7f019406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:19.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.510+0000 7f01ba4a2640 1 --2- 172.21.15.84:0/3220504323 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f019406af90 0x7f019406d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f01a4035000 tx=0x7f01a4030000 comp rx=0 tx=0).stop 2023-10-16T10:04:19.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.510+0000 7f01ba4a2640 1 -- 172.21.15.84:0/3220504323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01b40b3b70 msgr2=0x7f01b415bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:19.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.510+0000 7f01ba4a2640 1 --2- 172.21.15.84:0/3220504323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01b40b3b70 0x7f01b415bce0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f01b40cf780 tx=0x7f01ac009d40 comp rx=0 tx=0).stop 2023-10-16T10:04:19.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.510+0000 7f01ba4a2640 1 -- 172.21.15.84:0/3220504323 shutdown_connections 2023-10-16T10:04:19.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.510+0000 7f01ba4a2640 1 --2- 172.21.15.84:0/3220504323 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f019406af90 0x7f019406d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:19.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.510+0000 7f01ba4a2640 1 --2- 172.21.15.84:0/3220504323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01b40b3b70 0x7f01b415bce0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:19.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.510+0000 7f01ba4a2640 1 --2- 172.21.15.84:0/3220504323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01b40b36f0 0x7f01b415cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:19.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.510+0000 7f01ba4a2640 1 --2- 172.21.15.84:0/3220504323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a0003970 0x7f01b415b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:19.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.510+0000 7f01ba4a2640 1 -- 172.21.15.84:0/3220504323 >> 172.21.15.84:0/3220504323 conn(0x7f01b40ce9b0 msgr2=0x7f01b40b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:19.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.510+0000 7f01ba4a2640 1 -- 172.21.15.84:0/3220504323 shutdown_connections 2023-10-16T10:04:19.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:19.511+0000 7f01ba4a2640 1 -- 172.21.15.84:0/3220504323 wait complete. 2023-10-16T10:04:19.531 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:04:19.532 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:04:19.654 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:04:19.778 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:04:19.779 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:04:19.779 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:04:19.839 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:04:19.840 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:04:19.840 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:04:19.899 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:04:19.900 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:04:19.900 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:04:19.958 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:04:19.958 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:04:19.958 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:04:20.019 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:04:20.019 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T10:04:20.020 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:04:20.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.248+0000 7f20d7289640 1 Processor -- start 2023-10-16T10:04:20.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.248+0000 7f20d7289640 1 -- start start 2023-10-16T10:04:20.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.249+0000 7f20d7289640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f20d0159b20 0x7f20d006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:20.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.249+0000 7f20d7289640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d006f7f0 con 0x7f20d00b0b20 2023-10-16T10:04:20.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.249+0000 7f20d7289640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d006c5e0 con 0x7f20d0159f00 2023-10-16T10:04:20.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.249+0000 7f20d7289640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d006c8c0 con 0x7f20d0159b20 2023-10-16T10:04:20.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.250+0000 7f20d5025640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f20d0159b20 0x7f20d006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:20.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.250+0000 7f20d5025640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f20d0159b20 0x7f20d006bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50960/0 (socket says 172.21.15.84:50960) 2023-10-16T10:04:20.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.250+0000 7f20d5025640 1 -- 172.21.15.84:0/2022557159 learned_addr learned my addr 172.21.15.84:0/2022557159 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:20.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.250+0000 7f20d5025640 1 -- 172.21.15.84:0/2022557159 >> v1:172.21.15.157:6789/0 conn(0x7f20d0159f00 legacy=0x7f20d00b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:20.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.250+0000 7f20d5025640 1 -- 172.21.15.84:0/2022557159 >> v1:172.21.15.84:6789/0 conn(0x7f20d00b0b20 legacy=0x7f20d006fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.250+0000 7f20d5025640 1 -- 172.21.15.84:0/2022557159 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d006cba0 con 0x7f20d0159b20 2023-10-16T10:04:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.251+0000 7f20d5025640 1 --2- 172.21.15.84:0/2022557159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20d0159b20 0x7f20d006bbe0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00bb10 tx=0x7f20cc02e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a149f542f7e49f5a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.251+0000 7f20c7fff640 1 -- 172.21.15.84:0/2022557159 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc036440 con 0x7f20d0159b20 2023-10-16T10:04:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.251+0000 7f20c7fff640 1 -- 172.21.15.84:0/2022557159 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20cc0365a0 con 0x7f20d0159b20 2023-10-16T10:04:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.251+0000 7f20c7fff640 1 -- 172.21.15.84:0/2022557159 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc036930 con 0x7f20d0159b20 2023-10-16T10:04:20.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.252+0000 7f20d7289640 1 -- 172.21.15.84:0/2022557159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20d0159b20 msgr2=0x7f20d006bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:20.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.252+0000 7f20d7289640 1 --2- 172.21.15.84:0/2022557159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20d0159b20 0x7f20d006bbe0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00bb10 tx=0x7f20cc02e170 comp rx=0 tx=0).stop 2023-10-16T10:04:20.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.252+0000 7f20d7289640 1 -- 172.21.15.84:0/2022557159 shutdown_connections 2023-10-16T10:04:20.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.252+0000 7f20d7289640 1 --2- 172.21.15.84:0/2022557159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20d0159b20 0x7f20d006bbe0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:20.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.252+0000 7f20d7289640 1 -- 172.21.15.84:0/2022557159 >> 172.21.15.84:0/2022557159 conn(0x7f20d00cb410 msgr2=0x7f20d00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:20.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.252+0000 7f20d7289640 1 -- 172.21.15.84:0/2022557159 shutdown_connections 2023-10-16T10:04:20.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.252+0000 7f20d7289640 1 -- 172.21.15.84:0/2022557159 wait complete. 2023-10-16T10:04:20.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.254+0000 7f20d7289640 1 Processor -- start 2023-10-16T10:04:20.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.254+0000 7f20d7289640 1 -- start start 2023-10-16T10:04:20.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.255+0000 7f20d7289640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20d00b0b20 0x7f20d00c15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:20.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.255+0000 7f20d7289640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20d0159b20 0x7f20d0144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:20.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.255+0000 7f20d7289640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d0159f00 0x7f20d015c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:20.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.255+0000 7f20d7289640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d0145260 con 0x7f20d00b0b20 2023-10-16T10:04:20.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.255+0000 7f20d7289640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d0142ea0 con 0x7f20d0159f00 2023-10-16T10:04:20.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.255+0000 7f20d7289640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d0145490 con 0x7f20d0159b20 2023-10-16T10:04:20.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.255+0000 7f20d5025640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20d00b0b20 0x7f20d00c15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:20.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.255+0000 7f20d5025640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20d00b0b20 0x7f20d00c15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50974/0 (socket says 172.21.15.84:50974) 2023-10-16T10:04:20.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.255+0000 7f20d5025640 1 -- 172.21.15.84:0/2637137476 learned_addr learned my addr 172.21.15.84:0/2637137476 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:20.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.256+0000 7f20d5025640 1 -- 172.21.15.84:0/2637137476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20d0159b20 msgr2=0x7f20d0144d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:20.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.256+0000 7f20d5025640 1 --2- 172.21.15.84:0/2637137476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20d0159b20 0x7f20d0144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:20.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.256+0000 7f20d5025640 1 -- 172.21.15.84:0/2637137476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d0159f00 msgr2=0x7f20d015c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:20.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.256+0000 7f20d5025640 1 --2- 172.21.15.84:0/2637137476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d0159f00 0x7f20d015c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:20.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.256+0000 7f20d5025640 1 -- 172.21.15.84:0/2637137476 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc00b770 con 0x7f20d00b0b20 2023-10-16T10:04:20.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.256+0000 7f20d5025640 1 --2- 172.21.15.84:0/2637137476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20d00b0b20 0x7f20d00c15e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00b2c0 tx=0x7f20cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:20.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.256+0000 7f20c5ffb640 1 -- 172.21.15.84:0/2637137476 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc034b00 con 0x7f20d00b0b20 2023-10-16T10:04:20.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.256+0000 7f20c5ffb640 1 -- 172.21.15.84:0/2637137476 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20cc034c60 con 0x7f20d00b0b20 2023-10-16T10:04:20.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.256+0000 7f20d7289640 1 -- 172.21.15.84:0/2637137476 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d0145770 con 0x7f20d00b0b20 2023-10-16T10:04:20.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.256+0000 7f20c5ffb640 1 -- 172.21.15.84:0/2637137476 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc03f400 con 0x7f20d00b0b20 2023-10-16T10:04:20.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.256+0000 7f20d7289640 1 -- 172.21.15.84:0/2637137476 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d0153d70 con 0x7f20d00b0b20 2023-10-16T10:04:20.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.257+0000 7f20d7289640 1 -- 172.21.15.84:0/2637137476 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d0145c90 con 0x7f20d00b0b20 2023-10-16T10:04:20.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.258+0000 7f20c5ffb640 1 -- 172.21.15.84:0/2637137476 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f20cc03d040 con 0x7f20d00b0b20 2023-10-16T10:04:20.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.259+0000 7f20c5ffb640 1 --2- 172.21.15.84:0/2637137476 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f20b406af90 0x7f20b406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:20.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.259+0000 7f20c5ffb640 1 -- 172.21.15.84:0/2637137476 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f20cc0ae3c0 con 0x7f20d00b0b20 2023-10-16T10:04:20.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.260+0000 7f20d4824640 1 --2- 172.21.15.84:0/2637137476 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f20b406af90 0x7f20b406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:20.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.260+0000 7f20d4824640 1 --2- 172.21.15.84:0/2637137476 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f20b406af90 0x7f20b406d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f20d01f20d0 tx=0x7f20c0007990 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:20.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.264+0000 7f20c5ffb640 1 -- 172.21.15.84:0/2637137476 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f20cc0793a0 con 0x7f20d00b0b20 2023-10-16T10:04:20.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.498+0000 7f20d7289640 1 -- 172.21.15.84:0/2637137476 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20d004e160 con 0x7f20d00b0b20 2023-10-16T10:04:20.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.499+0000 7f20c5ffb640 1 -- 172.21.15.84:0/2637137476 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f20cc050e90 con 0x7f20d00b0b20 2023-10-16T10:04:20.501 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:20.501 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3040499154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3040499154},{"type":"v1","addr":"172.21.15.157:6835","nonce":3040499154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2727537766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2727537766},{"type":"v1","addr":"172.21.15.157:6833","nonce":2727537766}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/49292950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":49292950},{"type":"v1","addr":"172.21.15.84:6836","nonce":49292950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6441,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/1311852739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":1311852739},{"type":"v1","addr":"172.21.15.84:6837","nonce":1311852739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[]} 2023-10-16T10:04:20.502 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 27 2023-10-16T10:04:20.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.502+0000 7f20d7289640 1 -- 172.21.15.84:0/2637137476 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f20b406af90 msgr2=0x7f20b406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:20.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.502+0000 7f20d7289640 1 --2- 172.21.15.84:0/2637137476 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f20b406af90 0x7f20b406d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f20d01f20d0 tx=0x7f20c0007990 comp rx=0 tx=0).stop 2023-10-16T10:04:20.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.502+0000 7f20d7289640 1 -- 172.21.15.84:0/2637137476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20d00b0b20 msgr2=0x7f20d00c15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:20.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.502+0000 7f20d7289640 1 --2- 172.21.15.84:0/2637137476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20d00b0b20 0x7f20d00c15e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00b2c0 tx=0x7f20cc0046c0 comp rx=0 tx=0).stop 2023-10-16T10:04:20.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.502+0000 7f20d7289640 1 -- 172.21.15.84:0/2637137476 shutdown_connections 2023-10-16T10:04:20.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.502+0000 7f20d7289640 1 --2- 172.21.15.84:0/2637137476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d0159f00 0x7f20d015c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:20.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.502+0000 7f20d7289640 1 --2- 172.21.15.84:0/2637137476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20d0159b20 0x7f20d0144d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:20.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.502+0000 7f20d7289640 1 --2- 172.21.15.84:0/2637137476 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f20b406af90 0x7f20b406d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:20.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.502+0000 7f20d7289640 1 --2- 172.21.15.84:0/2637137476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20d00b0b20 0x7f20d00c15e0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:20.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.502+0000 7f20d7289640 1 -- 172.21.15.84:0/2637137476 >> 172.21.15.84:0/2637137476 conn(0x7f20d00cb410 msgr2=0x7f20d00cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:20.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.502+0000 7f20d7289640 1 -- 172.21.15.84:0/2637137476 shutdown_connections 2023-10-16T10:04:20.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:20.502+0000 7f20d7289640 1 -- 172.21.15.84:0/2637137476 wait complete. 2023-10-16T10:04:20.522 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T10:04:20.522 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T10:04:20.523 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T10:04:20.523 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:04:20.523 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T10:04:20.524 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T10:04:20.524 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T10:04:20.524 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:04:20.524 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T10:04:20.525 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T10:04:20.525 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T10:04:20.525 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:04:20.526 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T10:04:20.526 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T10:04:20.526 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T10:04:20.527 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:04:20.539 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T10:04:20.539 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T10:04:20.541 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T10:04:20.541 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T10:04:20.548 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T10:04:20.549 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T10:04:20.589 INFO:tasks.ceph.mds.c:Started 2023-10-16T10:04:20.590 DEBUG:teuthology.parallel:result is None 2023-10-16T10:04:20.597 INFO:tasks.ceph.mds.a:Started 2023-10-16T10:04:20.597 DEBUG:teuthology.parallel:result is None 2023-10-16T10:04:20.598 INFO:tasks.ceph.mds.d:Started 2023-10-16T10:04:20.598 DEBUG:teuthology.parallel:result is None 2023-10-16T10:04:20.752 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T10:04:20.753 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T10:04:20.783 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-10-16T10:04:22.485 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-16T10:04:26.548 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T10:04:26.549 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T10:04:26.558 INFO:tasks.ceph.mds.b:Started 2023-10-16T10:04:26.559 DEBUG:teuthology.parallel:result is None 2023-10-16T10:04:26.559 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T10:04:26.748 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T10:04:26.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.779+0000 7f33ef4e6640 1 Processor -- start 2023-10-16T10:04:26.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.779+0000 7f33ef4e6640 1 -- start start 2023-10-16T10:04:26.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.780+0000 7f33ef4e6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f33e806c040 0x7f33e806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:26.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.780+0000 7f33ef4e6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f33e80cda80 0x7f33e815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:26.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.780+0000 7f33ef4e6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e806c960 con 0x7f33e80ad970 2023-10-16T10:04:26.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.780+0000 7f33ef4e6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e815b9d0 con 0x7f33e80cda80 2023-10-16T10:04:26.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.780+0000 7f33ef4e6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e815bcb0 con 0x7f33e806c040 2023-10-16T10:04:26.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.781+0000 7f33ee4e4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f33e806c040 0x7f33e806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:26.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.781+0000 7f33ee4e4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f33e806c040 0x7f33e806c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:32860/0 (socket says 172.21.15.84:32860) 2023-10-16T10:04:26.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.781+0000 7f33ee4e4640 1 -- 172.21.15.84:0/201631674 learned_addr learned my addr 172.21.15.84:0/201631674 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:26.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.781+0000 7f33eece5640 1 --2- 172.21.15.84:0/201631674 >> v2:172.21.15.84:3300/0 conn(0x7f33e80cda80 0x7f33e815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:26.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.781+0000 7f33ee4e4640 1 -- 172.21.15.84:0/201631674 >> v2:172.21.15.84:3300/0 conn(0x7f33e80cda80 msgr2=0x7f33e815ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:26.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.781+0000 7f33ee4e4640 1 --2- 172.21.15.84:0/201631674 >> v2:172.21.15.84:3300/0 conn(0x7f33e80cda80 0x7f33e815ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:26.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.781+0000 7f33ee4e4640 1 -- 172.21.15.84:0/201631674 >> v1:172.21.15.84:6789/0 conn(0x7f33e80ad970 legacy=0x7f33e815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:26.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.781+0000 7f33ee4e4640 1 -- 172.21.15.84:0/201631674 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e815bf90 con 0x7f33e806c040 2023-10-16T10:04:26.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.783+0000 7f33ee4e4640 1 --2- 172.21.15.84:0/201631674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e806c040 0x7f33e806c420 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f33e80cf350 tx=0x7f33e402b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e215253dac2ef651 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:26.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.783+0000 7f33ed4e2640 1 -- 172.21.15.84:0/201631674 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4030860 con 0x7f33e806c040 2023-10-16T10:04:26.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.783+0000 7f33ed4e2640 1 -- 172.21.15.84:0/201631674 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e40309c0 con 0x7f33e806c040 2023-10-16T10:04:26.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.783+0000 7f33ed4e2640 1 -- 172.21.15.84:0/201631674 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4030d50 con 0x7f33e806c040 2023-10-16T10:04:26.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.784+0000 7f33ef4e6640 1 -- 172.21.15.84:0/201631674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e806c040 msgr2=0x7f33e806c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:26.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.784+0000 7f33ef4e6640 1 --2- 172.21.15.84:0/201631674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e806c040 0x7f33e806c420 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f33e80cf350 tx=0x7f33e402b9a0 comp rx=0 tx=0).stop 2023-10-16T10:04:26.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.784+0000 7f33ef4e6640 1 -- 172.21.15.84:0/201631674 shutdown_connections 2023-10-16T10:04:26.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.784+0000 7f33ef4e6640 1 --2- 172.21.15.84:0/201631674 >> v2:172.21.15.84:3300/0 conn(0x7f33e80cda80 0x7f33e815ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:26.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.784+0000 7f33ef4e6640 1 --2- 172.21.15.84:0/201631674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e806c040 0x7f33e806c420 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:26.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.784+0000 7f33ef4e6640 1 -- 172.21.15.84:0/201631674 >> 172.21.15.84:0/201631674 conn(0x7f33e80ce6a0 msgr2=0x7f33e80ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:26.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.784+0000 7f33ef4e6640 1 -- 172.21.15.84:0/201631674 shutdown_connections 2023-10-16T10:04:26.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.784+0000 7f33ef4e6640 1 -- 172.21.15.84:0/201631674 wait complete. 2023-10-16T10:04:26.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.786+0000 7f33ef4e6640 1 Processor -- start 2023-10-16T10:04:26.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.786+0000 7f33ef4e6640 1 -- start start 2023-10-16T10:04:26.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.787+0000 7f33ef4e6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33e806c040 0x7f33e80c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:26.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.787+0000 7f33ef4e6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e80ad970 0x7f33e80acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:26.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.787+0000 7f33ef4e6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33e80cda80 0x7f33e8149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:26.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.787+0000 7f33ef4e6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e80c8620 con 0x7f33e806c040 2023-10-16T10:04:26.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.787+0000 7f33ef4e6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e80af1b0 con 0x7f33e80ad970 2023-10-16T10:04:26.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.787+0000 7f33ef4e6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e80ae630 con 0x7f33e80cda80 2023-10-16T10:04:26.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.788+0000 7f33ee4e4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33e806c040 0x7f33e80c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:26.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.788+0000 7f33ee4e4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33e806c040 0x7f33e80c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41052/0 (socket says 172.21.15.84:41052) 2023-10-16T10:04:26.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.788+0000 7f33ee4e4640 1 -- 172.21.15.84:0/3143619650 learned_addr learned my addr 172.21.15.84:0/3143619650 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:26.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.788+0000 7f33edce3640 1 --2- 172.21.15.84:0/3143619650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e80ad970 0x7f33e80acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:26.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.788+0000 7f33eece5640 1 --2- 172.21.15.84:0/3143619650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33e80cda80 0x7f33e8149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:26.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.788+0000 7f33ee4e4640 1 -- 172.21.15.84:0/3143619650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33e80cda80 msgr2=0x7f33e8149630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:26.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.788+0000 7f33ee4e4640 1 --2- 172.21.15.84:0/3143619650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33e80cda80 0x7f33e8149630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:26.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.788+0000 7f33ee4e4640 1 -- 172.21.15.84:0/3143619650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e80ad970 msgr2=0x7f33e80acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:26.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.788+0000 7f33ee4e4640 1 --2- 172.21.15.84:0/3143619650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e80ad970 0x7f33e80acb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:26.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.788+0000 7f33ee4e4640 1 -- 172.21.15.84:0/3143619650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e40094d0 con 0x7f33e806c040 2023-10-16T10:04:26.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.788+0000 7f33edce3640 1 --2- 172.21.15.84:0/3143619650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e80ad970 0x7f33e80acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:26.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.789+0000 7f33ee4e4640 1 --2- 172.21.15.84:0/3143619650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33e806c040 0x7f33e80c80e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f33e4000c00 tx=0x7f33e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:26.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.789+0000 7f33deffd640 1 -- 172.21.15.84:0/3143619650 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4039c90 con 0x7f33e806c040 2023-10-16T10:04:26.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.789+0000 7f33deffd640 1 -- 172.21.15.84:0/3143619650 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e4039df0 con 0x7f33e806c040 2023-10-16T10:04:26.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.789+0000 7f33deffd640 1 -- 172.21.15.84:0/3143619650 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4043640 con 0x7f33e806c040 2023-10-16T10:04:26.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.789+0000 7f33ef4e6640 1 -- 172.21.15.84:0/3143619650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e80ae910 con 0x7f33e806c040 2023-10-16T10:04:26.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.789+0000 7f33ef4e6640 1 -- 172.21.15.84:0/3143619650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e80aeed0 con 0x7f33e806c040 2023-10-16T10:04:26.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.790+0000 7f33ef4e6640 1 -- 172.21.15.84:0/3143619650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e806b330 con 0x7f33e806c040 2023-10-16T10:04:26.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.791+0000 7f33deffd640 1 -- 172.21.15.84:0/3143619650 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f33e40437a0 con 0x7f33e806c040 2023-10-16T10:04:26.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.792+0000 7f33deffd640 1 --2- 172.21.15.84:0/3143619650 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f33cc06af90 0x7f33cc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:26.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.792+0000 7f33deffd640 1 -- 172.21.15.84:0/3143619650 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f33e403e070 con 0x7f33e806c040 2023-10-16T10:04:26.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.793+0000 7f33edce3640 1 --2- 172.21.15.84:0/3143619650 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f33cc06af90 0x7f33cc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:26.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.793+0000 7f33edce3640 1 --2- 172.21.15.84:0/3143619650 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f33cc06af90 0x7f33cc06d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f33d8007fd0 tx=0x7f33d8009040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:26.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:26.799+0000 7f33deffd640 1 -- 172.21.15.84:0/3143619650 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f33e4077880 con 0x7f33e806c040 2023-10-16T10:04:27.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:27.003+0000 7f33ef4e6640 1 -- 172.21.15.84:0/3143619650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f33e804e9b0 con 0x7f33e806c040 2023-10-16T10:04:28.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.322+0000 7f33deffd640 1 -- 172.21.15.84:0/3143619650 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v78) v1 ==== 97+0+0 (secure 0 0 0) 0x7f33e407aea0 con 0x7f33e806c040 2023-10-16T10:04:28.323 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-10-16T10:04:28.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.326+0000 7f33ef4e6640 1 -- 172.21.15.84:0/3143619650 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f33cc06af90 msgr2=0x7f33cc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:28.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.326+0000 7f33ef4e6640 1 --2- 172.21.15.84:0/3143619650 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f33cc06af90 0x7f33cc06d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f33d8007fd0 tx=0x7f33d8009040 comp rx=0 tx=0).stop 2023-10-16T10:04:28.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.326+0000 7f33ef4e6640 1 -- 172.21.15.84:0/3143619650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33e806c040 msgr2=0x7f33e80c80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:28.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.326+0000 7f33ef4e6640 1 --2- 172.21.15.84:0/3143619650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33e806c040 0x7f33e80c80e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f33e4000c00 tx=0x7f33e4004730 comp rx=0 tx=0).stop 2023-10-16T10:04:28.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.326+0000 7f33ef4e6640 1 -- 172.21.15.84:0/3143619650 shutdown_connections 2023-10-16T10:04:28.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.326+0000 7f33ef4e6640 1 --2- 172.21.15.84:0/3143619650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33e80cda80 0x7f33e8149630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:28.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.326+0000 7f33ef4e6640 1 --2- 172.21.15.84:0/3143619650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e80ad970 0x7f33e80acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:28.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.326+0000 7f33ef4e6640 1 --2- 172.21.15.84:0/3143619650 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f33cc06af90 0x7f33cc06d450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:28.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.326+0000 7f33ef4e6640 1 --2- 172.21.15.84:0/3143619650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33e806c040 0x7f33e80c80e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:28.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.326+0000 7f33ef4e6640 1 -- 172.21.15.84:0/3143619650 >> 172.21.15.84:0/3143619650 conn(0x7f33e80ce6a0 msgr2=0x7f33e806f700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:28.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.326+0000 7f33ef4e6640 1 -- 172.21.15.84:0/3143619650 shutdown_connections 2023-10-16T10:04:28.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.326+0000 7f33ef4e6640 1 -- 172.21.15.84:0/3143619650 wait complete. 2023-10-16T10:04:28.351 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T10:04:28.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.569+0000 7fdf4f090640 1 Processor -- start 2023-10-16T10:04:28.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.569+0000 7fdf4f090640 1 -- start start 2023-10-16T10:04:28.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.571+0000 7fdf4f090640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf4815ae80 0x7fdf4815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:28.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.571+0000 7fdf4f090640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdf480ce390 0x7fdf48070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:28.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.571+0000 7fdf4f090640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf4815ed60 con 0x7fdf4815aaa0 2023-10-16T10:04:28.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.571+0000 7fdf4f090640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf4806b810 con 0x7fdf480ce390 2023-10-16T10:04:28.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.571+0000 7fdf4f090640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf480707e0 con 0x7fdf4815ae80 2023-10-16T10:04:28.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.571+0000 7fdf4d62d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdf480ce390 0x7fdf48070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:28.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.572+0000 7fdf4d62d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdf480ce390 0x7fdf48070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41058/0 (socket says 172.21.15.84:41058) 2023-10-16T10:04:28.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.572+0000 7fdf4d62d640 1 -- 172.21.15.84:0/1180975127 learned_addr learned my addr 172.21.15.84:0/1180975127 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:28.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.572+0000 7fdf47fff640 1 --2- 172.21.15.84:0/1180975127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf4815ae80 0x7fdf4815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:28.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.572+0000 7fdf4d62d640 1 -- 172.21.15.84:0/1180975127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf4815ae80 msgr2=0x7fdf4815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:28.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.572+0000 7fdf4d62d640 1 --2- 172.21.15.84:0/1180975127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf4815ae80 0x7fdf4815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:28.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.572+0000 7fdf4d62d640 1 -- 172.21.15.84:0/1180975127 >> v1:172.21.15.157:6789/0 conn(0x7fdf4815aaa0 legacy=0x7fdf480cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:28.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.572+0000 7fdf4d62d640 1 -- 172.21.15.84:0/1180975127 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4815ef90 con 0x7fdf480ce390 2023-10-16T10:04:28.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.573+0000 7fdf4d62d640 1 --2- 172.21.15.84:0/1180975127 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf480ce390 0x7fdf48070f50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009e50 tx=0x7fdf4002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e24c0f7e5610d09 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:28.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.573+0000 7fdf477fe640 1 -- 172.21.15.84:0/1180975127 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4002cb00 con 0x7fdf480ce390 2023-10-16T10:04:28.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.573+0000 7fdf477fe640 1 -- 172.21.15.84:0/1180975127 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4002cc60 con 0x7fdf480ce390 2023-10-16T10:04:28.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.573+0000 7fdf477fe640 1 -- 172.21.15.84:0/1180975127 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf400343f0 con 0x7fdf480ce390 2023-10-16T10:04:28.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.574+0000 7fdf4f090640 1 -- 172.21.15.84:0/1180975127 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf480ce390 msgr2=0x7fdf48070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:28.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.574+0000 7fdf4f090640 1 --2- 172.21.15.84:0/1180975127 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf480ce390 0x7fdf48070f50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009e50 tx=0x7fdf4002bf60 comp rx=0 tx=0).stop 2023-10-16T10:04:28.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.574+0000 7fdf4f090640 1 -- 172.21.15.84:0/1180975127 shutdown_connections 2023-10-16T10:04:28.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.574+0000 7fdf4f090640 1 --2- 172.21.15.84:0/1180975127 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf480ce390 0x7fdf48070f50 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:28.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.574+0000 7fdf4f090640 1 --2- 172.21.15.84:0/1180975127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf4815ae80 0x7fdf4815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:28.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.574+0000 7fdf4f090640 1 -- 172.21.15.84:0/1180975127 >> 172.21.15.84:0/1180975127 conn(0x7fdf480cb410 msgr2=0x7fdf480cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:28.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.574+0000 7fdf4f090640 1 -- 172.21.15.84:0/1180975127 shutdown_connections 2023-10-16T10:04:28.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.574+0000 7fdf4f090640 1 -- 172.21.15.84:0/1180975127 wait complete. 2023-10-16T10:04:28.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.576+0000 7fdf4f090640 1 Processor -- start 2023-10-16T10:04:28.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.576+0000 7fdf4f090640 1 -- start start 2023-10-16T10:04:28.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.577+0000 7fdf4f090640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf480ce390 0x7fdf480c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:28.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.577+0000 7fdf4f090640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf4815aaa0 0x7fdf4814ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:28.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.577+0000 7fdf4f090640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf4815ae80 0x7fdf480b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:28.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.577+0000 7fdf4f090640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf480c5320 con 0x7fdf480ce390 2023-10-16T10:04:28.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.577+0000 7fdf4f090640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf480c2b30 con 0x7fdf4815aaa0 2023-10-16T10:04:28.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.577+0000 7fdf4f090640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf480c2e10 con 0x7fdf4815ae80 2023-10-16T10:04:28.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.577+0000 7fdf4ce2c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf480ce390 0x7fdf480c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:28.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.577+0000 7fdf4ce2c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf480ce390 0x7fdf480c24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41064/0 (socket says 172.21.15.84:41064) 2023-10-16T10:04:28.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.577+0000 7fdf4ce2c640 1 -- 172.21.15.84:0/111493119 learned_addr learned my addr 172.21.15.84:0/111493119 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:28.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.578+0000 7fdf47fff640 1 --2- 172.21.15.84:0/111493119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf4815aaa0 0x7fdf4814ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:28.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.578+0000 7fdf4d62d640 1 --2- 172.21.15.84:0/111493119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf4815ae80 0x7fdf480b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:28.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.578+0000 7fdf4ce2c640 1 -- 172.21.15.84:0/111493119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf4815ae80 msgr2=0x7fdf480b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:28.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.578+0000 7fdf4ce2c640 1 --2- 172.21.15.84:0/111493119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf4815ae80 0x7fdf480b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:28.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.578+0000 7fdf4ce2c640 1 -- 172.21.15.84:0/111493119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf4815aaa0 msgr2=0x7fdf4814ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:28.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.578+0000 7fdf4ce2c640 1 --2- 172.21.15.84:0/111493119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf4815aaa0 0x7fdf4814ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:28.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.578+0000 7fdf4ce2c640 1 -- 172.21.15.84:0/111493119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf40009b30 con 0x7fdf480ce390 2023-10-16T10:04:28.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.578+0000 7fdf47fff640 1 --2- 172.21.15.84:0/111493119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf4815aaa0 0x7fdf4814ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:28.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.578+0000 7fdf4ce2c640 1 --2- 172.21.15.84:0/111493119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf480ce390 0x7fdf480c24a0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00c450 tx=0x7fdf3c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:28.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.578+0000 7fdf457fa640 1 -- 172.21.15.84:0/111493119 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0052d0 con 0x7fdf480ce390 2023-10-16T10:04:28.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.578+0000 7fdf457fa640 1 -- 172.21.15.84:0/111493119 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c005430 con 0x7fdf480ce390 2023-10-16T10:04:28.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.578+0000 7fdf457fa640 1 -- 172.21.15.84:0/111493119 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c012870 con 0x7fdf480ce390 2023-10-16T10:04:28.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.578+0000 7fdf4f090640 1 -- 172.21.15.84:0/111493119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf48156860 con 0x7fdf480ce390 2023-10-16T10:04:28.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.579+0000 7fdf4f090640 1 -- 172.21.15.84:0/111493119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4814f240 con 0x7fdf480ce390 2023-10-16T10:04:28.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.580+0000 7fdf4f090640 1 -- 172.21.15.84:0/111493119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf48156d80 con 0x7fdf480ce390 2023-10-16T10:04:28.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.581+0000 7fdf457fa640 1 -- 172.21.15.84:0/111493119 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdf3c0129d0 con 0x7fdf480ce390 2023-10-16T10:04:28.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.581+0000 7fdf457fa640 1 --2- 172.21.15.84:0/111493119 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fdf2c06af90 0x7fdf2c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:28.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.582+0000 7fdf457fa640 1 -- 172.21.15.84:0/111493119 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdf3c088f80 con 0x7fdf480ce390 2023-10-16T10:04:28.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.582+0000 7fdf47fff640 1 --2- 172.21.15.84:0/111493119 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fdf2c06af90 0x7fdf2c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:28.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.582+0000 7fdf47fff640 1 --2- 172.21.15.84:0/111493119 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fdf2c06af90 0x7fdf2c06d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdf480c8390 tx=0x7fdf34008040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:28.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.586+0000 7fdf457fa640 1 -- 172.21.15.84:0/111493119 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdf3c0bbfe0 con 0x7fdf480ce390 2023-10-16T10:04:28.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.848+0000 7fdf4f090640 1 -- 172.21.15.84:0/111493119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fdf4804e160 con 0x7fdf480ce390 2023-10-16T10:04:28.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.849+0000 7fdf457fa640 1 -- 172.21.15.84:0/111493119 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fdf48156d80 con 0x7fdf480ce390 2023-10-16T10:04:28.849 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:28.850 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T10:04:28.850 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-10-16T10:04:28.850 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.850 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-10-16T10:04:28.851 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l6dhjKRAA0pMvY6AK6qtFXDprNhwsqg==", 2023-10-16T10:04:28.851 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.851 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:04:28.851 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:04:28.851 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:04:28.852 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:04:28.852 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.852 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:04:28.852 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.852 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-10-16T10:04:28.853 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lFpMxEBAA0TLPRt5q3FwwBvMG019cjQ==", 2023-10-16T10:04:28.853 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.853 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:04:28.853 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:04:28.853 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:04:28.854 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:04:28.854 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.854 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:04:28.854 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.854 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-10-16T10:04:28.855 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l73RBOhAAOMEPxfUAMHhn5Blyvd5oMw==", 2023-10-16T10:04:28.855 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.855 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:04:28.855 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:04:28.855 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:04:28.856 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:04:28.856 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.856 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:04:28.856 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.856 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-10-16T10:04:28.857 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lcTIRIhAA0+FWm4b4yw77LD3+WKbk7w==", 2023-10-16T10:04:28.857 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.857 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:04:28.857 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:04:28.857 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:04:28.858 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:04:28.858 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.858 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:04:28.858 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.858 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-10-16T10:04:28.859 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDYBi1l1zBLERAA3YLW3xIYQoRVw7KSkwN3Og==", 2023-10-16T10:04:28.859 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.859 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:04:28.859 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:04:28.859 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:04:28.860 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.860 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:04:28.860 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.860 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-10-16T10:04:28.860 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDbBi1lFUUxKRAAQbus8SmYprJRmWTjoCznNA==", 2023-10-16T10:04:28.861 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.861 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:04:28.861 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:04:28.861 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:04:28.861 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.861 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:04:28.862 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.862 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-10-16T10:04:28.862 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDeBi1l+9lBMxAArl6sb/h15uBG/a9lVH40Hg==", 2023-10-16T10:04:28.862 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.862 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:04:28.863 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:04:28.863 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:04:28.863 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.863 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:04:28.863 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.864 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-10-16T10:04:28.864 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDiBi1l2rnEERAA3oRD71xa0fEOynM8zKuBEg==", 2023-10-16T10:04:28.864 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.864 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:04:28.864 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:04:28.865 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:04:28.865 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.865 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:04:28.865 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.865 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-10-16T10:04:28.866 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDnBi1lR7oLChAAZto8fDovDbpOd3Fcnc7C6w==", 2023-10-16T10:04:28.866 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.866 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:04:28.866 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:04:28.866 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:04:28.867 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.867 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:04:28.867 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.867 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-10-16T10:04:28.867 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDqBi1lmjg/JBAAlINKRGBc0/UWJJHMVFw57w==", 2023-10-16T10:04:28.868 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.868 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:04:28.868 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:04:28.868 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:04:28.868 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.869 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:04:28.869 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.869 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-10-16T10:04:28.869 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDuBi1lu0kMARAAtuSeM8F3yr8CVrc3p1biaQ==", 2023-10-16T10:04:28.869 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.869 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:04:28.870 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:04:28.870 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:04:28.870 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.870 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:04:28.870 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.871 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-10-16T10:04:28.871 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDxBi1lf6mNGBAAHGbqzJ0fmDxQIsasboHMfg==", 2023-10-16T10:04:28.871 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.871 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:04:28.871 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:04:28.872 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:04:28.872 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.872 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:04:28.872 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.872 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-10-16T10:04:28.873 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ==", 2023-10-16T10:04:28.873 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.873 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:04:28.873 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow r", 2023-10-16T10:04:28.873 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T10:04:28.874 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.874 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:04:28.874 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.874 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-10-16T10:04:28.874 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1lHU+4BxAAd8ET8hQ8Y+EUwdTJr9/+Ig==", 2023-10-16T10:04:28.875 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.875 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:04:28.875 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:04:28.875 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:04:28.875 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:04:28.875 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.876 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:04:28.876 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.876 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-10-16T10:04:28.876 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lA1BOORAAFNyfqgEF34MVR9YX3KBKkQ==", 2023-10-16T10:04:28.876 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.877 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:04:28.877 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:04:28.877 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:04:28.877 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:04:28.877 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.878 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:04:28.878 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.878 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-10-16T10:04:28.878 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1l848oERAANMfrZz9JrlgNVh7ZFbtnaA==", 2023-10-16T10:04:28.878 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.878 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:04:28.879 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:04:28.879 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:04:28.879 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:04:28.879 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.879 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:04:28.880 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.880 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-10-16T10:04:28.880 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDRBi1lTAxTGRAAMptxnxy3Qit/cTCXie6CrQ==", 2023-10-16T10:04:28.880 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.880 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:04:28.881 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:04:28.881 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:04:28.881 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:04:28.881 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.881 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:04:28.881 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.882 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-10-16T10:04:28.882 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lPuJxHRAABAnGPVCrkq0JR4Y6iFGZlw==", 2023-10-16T10:04:28.882 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.882 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:04:28.882 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T10:04:28.883 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:04:28.883 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.883 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:04:28.883 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:28.883 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-10-16T10:04:28.884 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lLxnKEBAA+a/pOHUp7Sg3m4weAust5w==", 2023-10-16T10:04:28.884 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:04:28.884 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:04:28.884 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T10:04:28.884 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:04:28.885 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.885 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:28.885 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-10-16T10:04:28.885 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T10:04:28.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.851+0000 7fdf4f090640 1 -- 172.21.15.84:0/111493119 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fdf2c06af90 msgr2=0x7fdf2c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:28.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.851+0000 7fdf4f090640 1 --2- 172.21.15.84:0/111493119 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fdf2c06af90 0x7fdf2c06d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdf480c8390 tx=0x7fdf34008040 comp rx=0 tx=0).stop 2023-10-16T10:04:28.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.851+0000 7fdf4f090640 1 -- 172.21.15.84:0/111493119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf480ce390 msgr2=0x7fdf480c24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:28.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.851+0000 7fdf4f090640 1 --2- 172.21.15.84:0/111493119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf480ce390 0x7fdf480c24a0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00c450 tx=0x7fdf3c00c910 comp rx=0 tx=0).stop 2023-10-16T10:04:28.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.851+0000 7fdf4f090640 1 -- 172.21.15.84:0/111493119 shutdown_connections 2023-10-16T10:04:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.851+0000 7fdf4f090640 1 --2- 172.21.15.84:0/111493119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf4815ae80 0x7fdf480b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.851+0000 7fdf4f090640 1 --2- 172.21.15.84:0/111493119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf4815aaa0 0x7fdf4814ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.851+0000 7fdf4f090640 1 --2- 172.21.15.84:0/111493119 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fdf2c06af90 0x7fdf2c06d450 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.851+0000 7fdf4f090640 1 --2- 172.21.15.84:0/111493119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf480ce390 0x7fdf480c24a0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.851+0000 7fdf4f090640 1 -- 172.21.15.84:0/111493119 >> 172.21.15.84:0/111493119 conn(0x7fdf480cb410 msgr2=0x7fdf480b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.852+0000 7fdf4f090640 1 -- 172.21.15.84:0/111493119 shutdown_connections 2023-10-16T10:04:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:28.852+0000 7fdf4f090640 1 -- 172.21.15.84:0/111493119 wait complete. 2023-10-16T10:04:28.889 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:04:29.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.111+0000 7fbbe338a640 1 Processor -- start 2023-10-16T10:04:29.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.111+0000 7fbbe338a640 1 -- start start 2023-10-16T10:04:29.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.112+0000 7fbbe338a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbdc159b20 0x7fbbdc06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:29.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.112+0000 7fbbe338a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbbdc159f00 0x7fbbdc0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:29.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.112+0000 7fbbe338a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbdc06f7f0 con 0x7fbbdc06c120 2023-10-16T10:04:29.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.112+0000 7fbbe338a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbdc06c5e0 con 0x7fbbdc159f00 2023-10-16T10:04:29.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.112+0000 7fbbe338a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc06c8c0 con 0x7fbbdc159b20 2023-10-16T10:04:29.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.113+0000 7fbbe1126640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbdc159b20 0x7fbbdc06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:29.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.113+0000 7fbbe1126640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbdc159b20 0x7fbbdc06bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:50476/0 (socket says 172.21.15.84:50476) 2023-10-16T10:04:29.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.113+0000 7fbbe1126640 1 -- 172.21.15.84:0/1367427143 learned_addr learned my addr 172.21.15.84:0/1367427143 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:29.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.113+0000 7fbbe0925640 1 --2- 172.21.15.84:0/1367427143 >> v2:172.21.15.157:3300/0 conn(0x7fbbdc159f00 0x7fbbdc0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:29.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.114+0000 7fbbe1126640 1 -- 172.21.15.84:0/1367427143 >> v2:172.21.15.157:3300/0 conn(0x7fbbdc159f00 msgr2=0x7fbbdc0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:29.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.114+0000 7fbbe1126640 1 --2- 172.21.15.84:0/1367427143 >> v2:172.21.15.157:3300/0 conn(0x7fbbdc159f00 0x7fbbdc0b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:29.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.114+0000 7fbbe1126640 1 -- 172.21.15.84:0/1367427143 >> v1:172.21.15.157:6789/0 conn(0x7fbbdc06c120 legacy=0x7fbbdc0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:29.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.114+0000 7fbbe1126640 1 -- 172.21.15.84:0/1367427143 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbdc06cba0 con 0x7fbbdc159b20 2023-10-16T10:04:29.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.115+0000 7fbbe1126640 1 --2- 172.21.15.84:0/1367427143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbdc159b20 0x7fbbdc06bbe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009e50 tx=0x7fbbd802e170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd6e115954b543fe server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:29.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.116+0000 7fbbd3fff640 1 -- 172.21.15.84:0/1367427143 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd802ed10 con 0x7fbbdc159b20 2023-10-16T10:04:29.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.116+0000 7fbbd3fff640 1 -- 172.21.15.84:0/1367427143 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd802ee70 con 0x7fbbdc159b20 2023-10-16T10:04:29.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.116+0000 7fbbd3fff640 1 -- 172.21.15.84:0/1367427143 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd80365f0 con 0x7fbbdc159b20 2023-10-16T10:04:29.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.117+0000 7fbbe338a640 1 -- 172.21.15.84:0/1367427143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbdc159b20 msgr2=0x7fbbdc06bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:29.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.117+0000 7fbbe338a640 1 --2- 172.21.15.84:0/1367427143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbdc159b20 0x7fbbdc06bbe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009e50 tx=0x7fbbd802e170 comp rx=0 tx=0).stop 2023-10-16T10:04:29.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.117+0000 7fbbe338a640 1 -- 172.21.15.84:0/1367427143 shutdown_connections 2023-10-16T10:04:29.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.117+0000 7fbbe338a640 1 --2- 172.21.15.84:0/1367427143 >> v2:172.21.15.157:3300/0 conn(0x7fbbdc159f00 0x7fbbdc0b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:29.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.117+0000 7fbbe338a640 1 --2- 172.21.15.84:0/1367427143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbdc159b20 0x7fbbdc06bbe0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:29.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.117+0000 7fbbe338a640 1 -- 172.21.15.84:0/1367427143 >> 172.21.15.84:0/1367427143 conn(0x7fbbdc0cb410 msgr2=0x7fbbdc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:29.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.117+0000 7fbbe338a640 1 -- 172.21.15.84:0/1367427143 shutdown_connections 2023-10-16T10:04:29.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.117+0000 7fbbe338a640 1 -- 172.21.15.84:0/1367427143 wait complete. 2023-10-16T10:04:29.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.119+0000 7fbbe338a640 1 Processor -- start 2023-10-16T10:04:29.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.119+0000 7fbbe338a640 1 -- start start 2023-10-16T10:04:29.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.120+0000 7fbbe338a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbdc06c120 0x7fbbdc1583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:29.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.120+0000 7fbbe338a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbdc159b20 0x7fbbdc0b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:29.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.120+0000 7fbbe338a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbdc159f00 0x7fbbdc0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:29.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.120+0000 7fbbe338a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc1588e0 con 0x7fbbdc159f00 2023-10-16T10:04:29.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.120+0000 7fbbe338a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc0b3f10 con 0x7fbbdc06c120 2023-10-16T10:04:29.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.121+0000 7fbbe338a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc15a4b0 con 0x7fbbdc159b20 2023-10-16T10:04:29.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.121+0000 7fbbe1927640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbdc159f00 0x7fbbdc0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:29.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.121+0000 7fbbe1927640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbdc159f00 0x7fbbdc0ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41070/0 (socket says 172.21.15.84:41070) 2023-10-16T10:04:29.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.121+0000 7fbbe1927640 1 -- 172.21.15.84:0/1167415355 learned_addr learned my addr 172.21.15.84:0/1167415355 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:29.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.121+0000 7fbbe1126640 1 --2- 172.21.15.84:0/1167415355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbdc06c120 0x7fbbdc1583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:29.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.121+0000 7fbbe0925640 1 --2- 172.21.15.84:0/1167415355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbdc159b20 0x7fbbdc0b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:29.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.121+0000 7fbbe1927640 1 -- 172.21.15.84:0/1167415355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbdc159b20 msgr2=0x7fbbdc0b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:29.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.121+0000 7fbbe1927640 1 --2- 172.21.15.84:0/1167415355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbdc159b20 0x7fbbdc0b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:29.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.121+0000 7fbbe1927640 1 -- 172.21.15.84:0/1167415355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbdc06c120 msgr2=0x7fbbdc1583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:29.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.121+0000 7fbbe1927640 1 --2- 172.21.15.84:0/1167415355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbdc06c120 0x7fbbdc1583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:29.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.121+0000 7fbbe1927640 1 -- 172.21.15.84:0/1167415355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd8009b30 con 0x7fbbdc159f00 2023-10-16T10:04:29.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.122+0000 7fbbe1927640 1 --2- 172.21.15.84:0/1167415355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbdc159f00 0x7fbbdc0ccc40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400c3c0 tx=0x7fbbd400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:29.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.122+0000 7fbbd1ffb640 1 -- 172.21.15.84:0/1167415355 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4005220 con 0x7fbbdc159f00 2023-10-16T10:04:29.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.122+0000 7fbbd1ffb640 1 -- 172.21.15.84:0/1167415355 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd4005380 con 0x7fbbdc159f00 2023-10-16T10:04:29.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.122+0000 7fbbe338a640 1 -- 172.21.15.84:0/1167415355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbdc15a790 con 0x7fbbdc159f00 2023-10-16T10:04:29.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.122+0000 7fbbd1ffb640 1 -- 172.21.15.84:0/1167415355 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4007270 con 0x7fbbdc159f00 2023-10-16T10:04:29.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.122+0000 7fbbe338a640 1 -- 172.21.15.84:0/1167415355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbdc0c5b80 con 0x7fbbdc159f00 2023-10-16T10:04:29.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.123+0000 7fbbe338a640 1 -- 172.21.15.84:0/1167415355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbdc15a310 con 0x7fbbdc159f00 2023-10-16T10:04:29.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.124+0000 7fbbd1ffb640 1 -- 172.21.15.84:0/1167415355 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbbd40073d0 con 0x7fbbdc159f00 2023-10-16T10:04:29.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.125+0000 7fbbd1ffb640 1 --2- 172.21.15.84:0/1167415355 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fbbc006af90 0x7fbbc006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:29.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.125+0000 7fbbd1ffb640 1 -- 172.21.15.84:0/1167415355 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbbd4088800 con 0x7fbbdc159f00 2023-10-16T10:04:29.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.125+0000 7fbbe1126640 1 --2- 172.21.15.84:0/1167415355 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fbbc006af90 0x7fbbc006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:29.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.126+0000 7fbbe1126640 1 --2- 172.21.15.84:0/1167415355 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fbbc006af90 0x7fbbc006d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8037000 tx=0x7fbbd8032000 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:29.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.129+0000 7fbbd1ffb640 1 -- 172.21.15.84:0/1167415355 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbbdc15a310 con 0x7fbbdc159f00 2023-10-16T10:04:29.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-16T10:04:29.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.325+0000 7fbbe338a640 1 -- 172.21.15.84:0/1167415355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbdc15acb0 con 0x7fbbdc159f00 2023-10-16T10:04:29.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.327+0000 7fbbd1ffb640 1 -- 172.21.15.84:0/1167415355 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+12833 (secure 0 0 0) 0x7fbbdc15acb0 con 0x7fbbdc159f00 2023-10-16T10:04:29.328 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:29.328 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":78,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:04:28.318751+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:04:29.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.329+0000 7fbbe338a640 1 -- 172.21.15.84:0/1167415355 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fbbc006af90 msgr2=0x7fbbc006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:29.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.329+0000 7fbbe338a640 1 --2- 172.21.15.84:0/1167415355 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fbbc006af90 0x7fbbc006d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8037000 tx=0x7fbbd8032000 comp rx=0 tx=0).stop 2023-10-16T10:04:29.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.329+0000 7fbbe338a640 1 -- 172.21.15.84:0/1167415355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbdc159f00 msgr2=0x7fbbdc0ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:29.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.329+0000 7fbbe338a640 1 --2- 172.21.15.84:0/1167415355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbdc159f00 0x7fbbdc0ccc40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400c3c0 tx=0x7fbbd400c880 comp rx=0 tx=0).stop 2023-10-16T10:04:29.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.329+0000 7fbbe338a640 1 -- 172.21.15.84:0/1167415355 shutdown_connections 2023-10-16T10:04:29.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.329+0000 7fbbe338a640 1 --2- 172.21.15.84:0/1167415355 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fbbc006af90 0x7fbbc006d450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:29.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.330+0000 7fbbe338a640 1 --2- 172.21.15.84:0/1167415355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbdc159f00 0x7fbbdc0ccc40 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:29.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.330+0000 7fbbe338a640 1 --2- 172.21.15.84:0/1167415355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbdc159b20 0x7fbbdc0b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:29.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.330+0000 7fbbe338a640 1 --2- 172.21.15.84:0/1167415355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbdc06c120 0x7fbbdc1583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:29.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.330+0000 7fbbe338a640 1 -- 172.21.15.84:0/1167415355 >> 172.21.15.84:0/1167415355 conn(0x7fbbdc0cb410 msgr2=0x7fbbdc0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:29.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.330+0000 7fbbe338a640 1 -- 172.21.15.84:0/1167415355 shutdown_connections 2023-10-16T10:04:29.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.330+0000 7fbbe338a640 1 -- 172.21.15.84:0/1167415355 wait complete. 2023-10-16T10:04:29.351 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:04:29.352 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:04:29.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.569+0000 7f463b87c640 1 Processor -- start 2023-10-16T10:04:29.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.569+0000 7f463b87c640 1 -- start start 2023-10-16T10:04:29.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.571+0000 7f463b87c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4634160ad0 0x7f46340ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:29.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.571+0000 7f463b87c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f46340ae260 0x7f463406a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:29.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.571+0000 7f463b87c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46340ce800 con 0x7f46340ce350 2023-10-16T10:04:29.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.571+0000 7f463b87c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46340cdec0 con 0x7f46340ae260 2023-10-16T10:04:29.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.571+0000 7f463b87c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46340acdd0 con 0x7f4634160ad0 2023-10-16T10:04:29.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.571+0000 7f4639e19640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f46340ae260 0x7f463406a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:29.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.571+0000 7f4639e19640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f46340ae260 0x7f463406a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41074/0 (socket says 172.21.15.84:41074) 2023-10-16T10:04:29.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.571+0000 7f4639e19640 1 -- 172.21.15.84:0/1183169301 learned_addr learned my addr 172.21.15.84:0/1183169301 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:29.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.571+0000 7f4638e17640 1 --1- 172.21.15.84:0/1183169301 >> v1:172.21.15.84:6789/0 conn(0x7f46340ce350 0x7f4634160eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40484/0 (socket says 172.21.15.84:40484) 2023-10-16T10:04:29.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.572+0000 7f4639618640 1 --2- 172.21.15.84:0/1183169301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4634160ad0 0x7f46340ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:29.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.572+0000 7f4639e19640 1 -- 172.21.15.84:0/1183169301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4634160ad0 msgr2=0x7f46340ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:29.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.572+0000 7f4639e19640 1 --2- 172.21.15.84:0/1183169301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4634160ad0 0x7f46340ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:29.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.572+0000 7f4639e19640 1 -- 172.21.15.84:0/1183169301 >> v1:172.21.15.84:6789/0 conn(0x7f46340ce350 legacy=0x7f4634160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:29.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.572+0000 7f4639e19640 1 -- 172.21.15.84:0/1183169301 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46340ae740 con 0x7f46340ae260 2023-10-16T10:04:29.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.573+0000 7f4639e19640 1 --2- 172.21.15.84:0/1183169301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46340ae260 0x7f463406a780 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f462c009e50 tx=0x7f462c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafc60d1d28291c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:29.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.573+0000 7f4627fff640 1 -- 172.21.15.84:0/1183169301 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c02cb00 con 0x7f46340ae260 2023-10-16T10:04:29.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.573+0000 7f4627fff640 1 -- 172.21.15.84:0/1183169301 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c02cc60 con 0x7f46340ae260 2023-10-16T10:04:29.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.573+0000 7f4627fff640 1 -- 172.21.15.84:0/1183169301 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c0343f0 con 0x7f46340ae260 2023-10-16T10:04:29.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.574+0000 7f463b87c640 1 -- 172.21.15.84:0/1183169301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46340ae260 msgr2=0x7f463406a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:29.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.574+0000 7f463b87c640 1 --2- 172.21.15.84:0/1183169301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46340ae260 0x7f463406a780 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f462c009e50 tx=0x7f462c02bf60 comp rx=0 tx=0).stop 2023-10-16T10:04:29.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.574+0000 7f463b87c640 1 -- 172.21.15.84:0/1183169301 shutdown_connections 2023-10-16T10:04:29.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.574+0000 7f463b87c640 1 --2- 172.21.15.84:0/1183169301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46340ae260 0x7f463406a780 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:29.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.574+0000 7f463b87c640 1 --2- 172.21.15.84:0/1183169301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4634160ad0 0x7f46340ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:29.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.574+0000 7f463b87c640 1 -- 172.21.15.84:0/1183169301 >> 172.21.15.84:0/1183169301 conn(0x7f463406bc10 msgr2=0x7f463406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:29.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.574+0000 7f463b87c640 1 -- 172.21.15.84:0/1183169301 shutdown_connections 2023-10-16T10:04:29.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.574+0000 7f463b87c640 1 -- 172.21.15.84:0/1183169301 wait complete. 2023-10-16T10:04:29.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.576+0000 7f463b87c640 1 Processor -- start 2023-10-16T10:04:29.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.576+0000 7f463b87c640 1 -- start start 2023-10-16T10:04:29.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.577+0000 7f463b87c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46340ae260 0x7f46340c3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:29.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.577+0000 7f463b87c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46340ce350 0x7f463415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:29.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.577+0000 7f463b87c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4634160ad0 0x7f463406ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:29.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.577+0000 7f463b87c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46340b4e70 con 0x7f4634160ad0 2023-10-16T10:04:29.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.577+0000 7f463b87c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46340b4010 con 0x7f46340ae260 2023-10-16T10:04:29.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.577+0000 7f463b87c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46340c3b70 con 0x7f46340ce350 2023-10-16T10:04:29.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.577+0000 7f4639e19640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4634160ad0 0x7f463406ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:29.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.577+0000 7f4639e19640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4634160ad0 0x7f463406ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41080/0 (socket says 172.21.15.84:41080) 2023-10-16T10:04:29.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.577+0000 7f4639e19640 1 -- 172.21.15.84:0/2823732130 learned_addr learned my addr 172.21.15.84:0/2823732130 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:29.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.578+0000 7f4638e17640 1 --2- 172.21.15.84:0/2823732130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46340ce350 0x7f463415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.578+0000 7f4639e19640 1 -- 172.21.15.84:0/2823732130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46340ce350 msgr2=0x7f463415f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.578+0000 7f4639618640 1 --2- 172.21.15.84:0/2823732130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46340ae260 0x7f46340c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.578+0000 7f4639e19640 1 --2- 172.21.15.84:0/2823732130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46340ce350 0x7f463415f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.578+0000 7f4639e19640 1 -- 172.21.15.84:0/2823732130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46340ae260 msgr2=0x7f46340c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.578+0000 7f4639e19640 1 --2- 172.21.15.84:0/2823732130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46340ae260 0x7f46340c3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.578+0000 7f4639e19640 1 -- 172.21.15.84:0/2823732130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462c009b30 con 0x7f4634160ad0 2023-10-16T10:04:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.578+0000 7f4638e17640 1 --2- 172.21.15.84:0/2823732130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46340ce350 0x7f463415f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.578+0000 7f4639e19640 1 --2- 172.21.15.84:0/2823732130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4634160ad0 0x7f463406ac00 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f462c009e50 tx=0x7f462c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.578+0000 7f4625ffb640 1 -- 172.21.15.84:0/2823732130 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c009510 con 0x7f4634160ad0 2023-10-16T10:04:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.578+0000 7f4625ffb640 1 -- 172.21.15.84:0/2823732130 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c03e450 con 0x7f4634160ad0 2023-10-16T10:04:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.578+0000 7f463b87c640 1 -- 172.21.15.84:0/2823732130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463415f900 con 0x7f4634160ad0 2023-10-16T10:04:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.579+0000 7f4625ffb640 1 -- 172.21.15.84:0/2823732130 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c03e800 con 0x7f4634160ad0 2023-10-16T10:04:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.579+0000 7f463b87c640 1 -- 172.21.15.84:0/2823732130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46340b4540 con 0x7f4634160ad0 2023-10-16T10:04:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.580+0000 7f463b87c640 1 -- 172.21.15.84:0/2823732130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46340cea30 con 0x7f4634160ad0 2023-10-16T10:04:29.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.581+0000 7f4625ffb640 1 -- 172.21.15.84:0/2823732130 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f462c03e960 con 0x7f4634160ad0 2023-10-16T10:04:29.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.581+0000 7f4625ffb640 1 --2- 172.21.15.84:0/2823732130 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f461806af90 0x7f461806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:29.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.582+0000 7f4625ffb640 1 -- 172.21.15.84:0/2823732130 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f462c0ac2d0 con 0x7f4634160ad0 2023-10-16T10:04:29.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.582+0000 7f4639618640 1 --2- 172.21.15.84:0/2823732130 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f461806af90 0x7f461806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:29.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.582+0000 7f4639618640 1 --2- 172.21.15.84:0/2823732130 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f461806af90 0x7f461806d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f463000b7c0 tx=0x7f4630008040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:29.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.586+0000 7f4625ffb640 1 -- 172.21.15.84:0/2823732130 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f46340cea30 con 0x7f4634160ad0 2023-10-16T10:04:29.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.778+0000 7f463b87c640 1 -- 172.21.15.84:0/2823732130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f46341f5f30 con 0x7f4634160ad0 2023-10-16T10:04:29.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.779+0000 7f4625ffb640 1 -- 172.21.15.84:0/2823732130 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f46341f5f30 con 0x7f4634160ad0 2023-10-16T10:04:29.779 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:04:29.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.781+0000 7f463b87c640 1 -- 172.21.15.84:0/2823732130 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f461806af90 msgr2=0x7f461806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:29.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.781+0000 7f463b87c640 1 --2- 172.21.15.84:0/2823732130 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f461806af90 0x7f461806d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f463000b7c0 tx=0x7f4630008040 comp rx=0 tx=0).stop 2023-10-16T10:04:29.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.781+0000 7f463b87c640 1 -- 172.21.15.84:0/2823732130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4634160ad0 msgr2=0x7f463406ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:29.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.781+0000 7f463b87c640 1 --2- 172.21.15.84:0/2823732130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4634160ad0 0x7f463406ac00 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f462c009e50 tx=0x7f462c004a40 comp rx=0 tx=0).stop 2023-10-16T10:04:29.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.781+0000 7f463b87c640 1 -- 172.21.15.84:0/2823732130 shutdown_connections 2023-10-16T10:04:29.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.781+0000 7f463b87c640 1 --2- 172.21.15.84:0/2823732130 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f461806af90 0x7f461806d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:29.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.781+0000 7f463b87c640 1 --2- 172.21.15.84:0/2823732130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4634160ad0 0x7f463406ac00 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:29.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.781+0000 7f463b87c640 1 --2- 172.21.15.84:0/2823732130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46340ce350 0x7f463415f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:29.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.781+0000 7f463b87c640 1 --2- 172.21.15.84:0/2823732130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46340ae260 0x7f46340c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:29.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.781+0000 7f463b87c640 1 -- 172.21.15.84:0/2823732130 >> 172.21.15.84:0/2823732130 conn(0x7f463406bc10 msgr2=0x7f46340ca8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:29.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.782+0000 7f463b87c640 1 -- 172.21.15.84:0/2823732130 shutdown_connections 2023-10-16T10:04:29.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:29.782+0000 7f463b87c640 1 -- 172.21.15.84:0/2823732130 wait complete. 2023-10-16T10:04:29.801 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T10:04:30.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.018+0000 7fbb47fff640 1 Processor -- start 2023-10-16T10:04:30.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.018+0000 7fbb47fff640 1 -- start start 2023-10-16T10:04:30.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.020+0000 7fbb47fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbb4806c040 0x7fbb4806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:30.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.020+0000 7fbb47fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb480cda80 0x7fbb4815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:30.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.020+0000 7fbb47fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4806c960 con 0x7fbb480ad970 2023-10-16T10:04:30.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.020+0000 7fbb47fff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb4815b9d0 con 0x7fbb4806c040 2023-10-16T10:04:30.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.020+0000 7fbb47fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4815bcb0 con 0x7fbb480cda80 2023-10-16T10:04:30.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.020+0000 7fbb3ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbb480ad970 0x7fbb4815e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40492/0 (socket says 172.21.15.84:40492) 2023-10-16T10:04:30.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.020+0000 7fbb46ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbb4806c040 0x7fbb4806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:30.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.020+0000 7fbb3ffff640 1 -- 172.21.15.84:0/1046113468 learned_addr learned my addr 172.21.15.84:0/1046113468 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:30.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.020+0000 7fbb477fe640 1 --2- 172.21.15.84:0/1046113468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb480cda80 0x7fbb4815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:30.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.020+0000 7fbb46ffd640 1 -- 172.21.15.84:0/1046113468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb480cda80 msgr2=0x7fbb4815ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:30.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.020+0000 7fbb46ffd640 1 --2- 172.21.15.84:0/1046113468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb480cda80 0x7fbb4815ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:30.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.020+0000 7fbb46ffd640 1 -- 172.21.15.84:0/1046113468 >> v1:172.21.15.84:6789/0 conn(0x7fbb480ad970 legacy=0x7fbb4815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:30.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.020+0000 7fbb46ffd640 1 -- 172.21.15.84:0/1046113468 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4815bf90 con 0x7fbb4806c040 2023-10-16T10:04:30.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.020+0000 7fbb467fc640 1 -- 172.21.15.84:0/1046113468 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503035737 0 0) 0x7fbb4806c960 con 0x7fbb480ad970 2023-10-16T10:04:30.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.020+0000 7fbb477fe640 1 --2- 172.21.15.84:0/1046113468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb480cda80 0x7fbb4815ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:30.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.022+0000 7fbb46ffd640 1 --2- 172.21.15.84:0/1046113468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb4806c040 0x7fbb4806c420 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fbb380097a0 tx=0x7fbb3802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c2b723fa0311a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:30.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.022+0000 7fbb467fc640 1 -- 172.21.15.84:0/1046113468 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3802d040 con 0x7fbb4806c040 2023-10-16T10:04:30.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.022+0000 7fbb467fc640 1 -- 172.21.15.84:0/1046113468 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb38031810 con 0x7fbb4806c040 2023-10-16T10:04:30.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.022+0000 7fbb467fc640 1 -- 172.21.15.84:0/1046113468 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb38031ba0 con 0x7fbb4806c040 2023-10-16T10:04:30.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.022+0000 7fbb47fff640 1 -- 172.21.15.84:0/1046113468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb4806c040 msgr2=0x7fbb4806c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:30.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.022+0000 7fbb47fff640 1 --2- 172.21.15.84:0/1046113468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb4806c040 0x7fbb4806c420 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fbb380097a0 tx=0x7fbb3802b8b0 comp rx=0 tx=0).stop 2023-10-16T10:04:30.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.022+0000 7fbb47fff640 1 -- 172.21.15.84:0/1046113468 shutdown_connections 2023-10-16T10:04:30.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.022+0000 7fbb47fff640 1 --2- 172.21.15.84:0/1046113468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb480cda80 0x7fbb4815ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:30.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.022+0000 7fbb47fff640 1 --2- 172.21.15.84:0/1046113468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb4806c040 0x7fbb4806c420 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:30.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.022+0000 7fbb47fff640 1 -- 172.21.15.84:0/1046113468 >> 172.21.15.84:0/1046113468 conn(0x7fbb480ce6a0 msgr2=0x7fbb480ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:30.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.023+0000 7fbb47fff640 1 -- 172.21.15.84:0/1046113468 shutdown_connections 2023-10-16T10:04:30.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.023+0000 7fbb47fff640 1 -- 172.21.15.84:0/1046113468 wait complete. 2023-10-16T10:04:30.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.024+0000 7fbb47fff640 1 Processor -- start 2023-10-16T10:04:30.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.024+0000 7fbb47fff640 1 -- start start 2023-10-16T10:04:30.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.025+0000 7fbb47fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb4806c040 0x7fbb48159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:30.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.025+0000 7fbb47fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb480ad970 0x7fbb480c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:30.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.025+0000 7fbb47fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb480cda80 0x7fbb4815d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:30.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.025+0000 7fbb47fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb480c8290 con 0x7fbb480cda80 2023-10-16T10:04:30.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.025+0000 7fbb47fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb480c7680 con 0x7fbb4806c040 2023-10-16T10:04:30.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.025+0000 7fbb47fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb480c7960 con 0x7fbb480ad970 2023-10-16T10:04:30.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.026+0000 7fbb477fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb480cda80 0x7fbb4815d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:30.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.026+0000 7fbb477fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb480cda80 0x7fbb4815d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41104/0 (socket says 172.21.15.84:41104) 2023-10-16T10:04:30.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.026+0000 7fbb477fe640 1 -- 172.21.15.84:0/3373951400 learned_addr learned my addr 172.21.15.84:0/3373951400 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:30.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.026+0000 7fbb46ffd640 1 --2- 172.21.15.84:0/3373951400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb4806c040 0x7fbb48159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:30.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.026+0000 7fbb477fe640 1 -- 172.21.15.84:0/3373951400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb480ad970 msgr2=0x7fbb480c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:30.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.026+0000 7fbb477fe640 1 --2- 172.21.15.84:0/3373951400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb480ad970 0x7fbb480c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:30.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.026+0000 7fbb477fe640 1 -- 172.21.15.84:0/3373951400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb4806c040 msgr2=0x7fbb48159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:30.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.026+0000 7fbb477fe640 1 --2- 172.21.15.84:0/3373951400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb4806c040 0x7fbb48159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:30.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.026+0000 7fbb477fe640 1 -- 172.21.15.84:0/3373951400 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb38009400 con 0x7fbb480cda80 2023-10-16T10:04:30.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.026+0000 7fbb46ffd640 1 --2- 172.21.15.84:0/3373951400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb4806c040 0x7fbb48159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:30.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.026+0000 7fbb477fe640 1 --2- 172.21.15.84:0/3373951400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb480cda80 0x7fbb4815d350 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000cef0 tx=0x7fbb4000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:30.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.026+0000 7fbb44ff9640 1 -- 172.21.15.84:0/3373951400 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4000d840 con 0x7fbb480cda80 2023-10-16T10:04:30.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.027+0000 7fbb44ff9640 1 -- 172.21.15.84:0/3373951400 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4000d9c0 con 0x7fbb480cda80 2023-10-16T10:04:30.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.027+0000 7fbb44ff9640 1 -- 172.21.15.84:0/3373951400 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4000dd70 con 0x7fbb480cda80 2023-10-16T10:04:30.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.027+0000 7fbb47fff640 1 -- 172.21.15.84:0/3373951400 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb480c7c40 con 0x7fbb480cda80 2023-10-16T10:04:30.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.027+0000 7fbb47fff640 1 -- 172.21.15.84:0/3373951400 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4815a310 con 0x7fbb480cda80 2023-10-16T10:04:30.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.028+0000 7fbb47fff640 1 -- 172.21.15.84:0/3373951400 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb4806b330 con 0x7fbb480cda80 2023-10-16T10:04:30.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.029+0000 7fbb44ff9640 1 -- 172.21.15.84:0/3373951400 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbb4001e3b0 con 0x7fbb480cda80 2023-10-16T10:04:30.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.030+0000 7fbb44ff9640 1 --2- 172.21.15.84:0/3373951400 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fbb2c06af90 0x7fbb2c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:30.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.030+0000 7fbb44ff9640 1 -- 172.21.15.84:0/3373951400 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbb40089f70 con 0x7fbb480cda80 2023-10-16T10:04:30.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.030+0000 7fbb46ffd640 1 --2- 172.21.15.84:0/3373951400 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fbb2c06af90 0x7fbb2c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:30.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.030+0000 7fbb46ffd640 1 --2- 172.21.15.84:0/3373951400 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fbb2c06af90 0x7fbb2c06d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbb38036000 tx=0x7fbb3803c040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:30.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.034+0000 7fbb44ff9640 1 -- 172.21.15.84:0/3373951400 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbb400bcfe0 con 0x7fbb480cda80 2023-10-16T10:04:30.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.227+0000 7fbb47fff640 1 -- 172.21.15.84:0/3373951400 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbb4804e9b0 con 0x7fbb480cda80 2023-10-16T10:04:30.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.228+0000 7fbb44ff9640 1 -- 172.21.15.84:0/3373951400 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbb4001a070 con 0x7fbb480cda80 2023-10-16T10:04:30.228 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:30.229 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T10:04:30.229 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:30.229 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-16T10:04:30.229 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-16T10:04:30.230 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:30.230 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T10:04:30.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.230+0000 7fbb47fff640 1 -- 172.21.15.84:0/3373951400 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fbb2c06af90 msgr2=0x7fbb2c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:30.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.230+0000 7fbb47fff640 1 --2- 172.21.15.84:0/3373951400 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fbb2c06af90 0x7fbb2c06d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbb38036000 tx=0x7fbb3803c040 comp rx=0 tx=0).stop 2023-10-16T10:04:30.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.230+0000 7fbb47fff640 1 -- 172.21.15.84:0/3373951400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb480cda80 msgr2=0x7fbb4815d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:30.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.230+0000 7fbb47fff640 1 --2- 172.21.15.84:0/3373951400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb480cda80 0x7fbb4815d350 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000cef0 tx=0x7fbb4000a710 comp rx=0 tx=0).stop 2023-10-16T10:04:30.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.231+0000 7fbb47fff640 1 -- 172.21.15.84:0/3373951400 shutdown_connections 2023-10-16T10:04:30.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.231+0000 7fbb47fff640 1 --2- 172.21.15.84:0/3373951400 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fbb2c06af90 0x7fbb2c06d450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:30.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.231+0000 7fbb47fff640 1 --2- 172.21.15.84:0/3373951400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb480cda80 0x7fbb4815d350 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:30.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.231+0000 7fbb47fff640 1 --2- 172.21.15.84:0/3373951400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb480ad970 0x7fbb480c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:30.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.231+0000 7fbb47fff640 1 --2- 172.21.15.84:0/3373951400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb4806c040 0x7fbb48159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:30.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.231+0000 7fbb47fff640 1 -- 172.21.15.84:0/3373951400 >> 172.21.15.84:0/3373951400 conn(0x7fbb480ce6a0 msgr2=0x7fbb4806f700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:30.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.231+0000 7fbb47fff640 1 -- 172.21.15.84:0/3373951400 shutdown_connections 2023-10-16T10:04:30.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.231+0000 7fbb47fff640 1 -- 172.21.15.84:0/3373951400 wait complete. 2023-10-16T10:04:30.250 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T10:04:30.251 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T10:04:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.462+0000 7fde8fa00640 1 Processor -- start 2023-10-16T10:04:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.463+0000 7fde8fa00640 1 -- start start 2023-10-16T10:04:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.464+0000 7fde8fa00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fde8815cf60 0x7fde8806f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.464+0000 7fde8fa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8806f9d0 0x7fde880c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.464+0000 7fde8fa00640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde880c9620 con 0x7fde880c9ff0 2023-10-16T10:04:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.464+0000 7fde8fa00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8815d340 con 0x7fde8815cf60 2023-10-16T10:04:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.464+0000 7fde8fa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde880cc860 con 0x7fde8806f9d0 2023-10-16T10:04:30.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.465+0000 7fde8df9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fde880c9ff0 0x7fde880b0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40500/0 (socket says 172.21.15.84:40500) 2023-10-16T10:04:30.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.465+0000 7fde8df9d640 1 -- 172.21.15.84:0/4213951530 learned_addr learned my addr 172.21.15.84:0/4213951530 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:30.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.465+0000 7fde8cf9b640 1 --2- 172.21.15.84:0/4213951530 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8806f9d0 0x7fde880c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:30.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.465+0000 7fde8d79c640 1 --2- 172.21.15.84:0/4213951530 >> v2:172.21.15.157:3300/0 conn(0x7fde8815cf60 0x7fde8806f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:30.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.465+0000 7fde7ffff640 1 -- 172.21.15.84:0/4213951530 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 172001863 0 0) 0x7fde880c9620 con 0x7fde880c9ff0 2023-10-16T10:04:30.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.465+0000 7fde7ffff640 1 -- 172.21.15.84:0/4213951530 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde74003520 con 0x7fde880c9ff0 2023-10-16T10:04:30.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde7ffff640 1 -- 172.21.15.84:0/4213951530 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2476913777 0 0) 0x7fde74003520 con 0x7fde880c9ff0 2023-10-16T10:04:30.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde7ffff640 1 -- 172.21.15.84:0/4213951530 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8806f9d0 msgr2=0x7fde880c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:30.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde7ffff640 1 --2- 172.21.15.84:0/4213951530 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8806f9d0 0x7fde880c9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:30.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde7ffff640 1 -- 172.21.15.84:0/4213951530 >> v2:172.21.15.157:3300/0 conn(0x7fde8815cf60 msgr2=0x7fde8806f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:30.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde7ffff640 1 --2- 172.21.15.84:0/4213951530 >> v2:172.21.15.157:3300/0 conn(0x7fde8815cf60 0x7fde8806f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:30.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde7ffff640 1 -- 172.21.15.84:0/4213951530 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde880ad4a0 con 0x7fde880c9ff0 2023-10-16T10:04:30.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde7ffff640 1 -- 172.21.15.84:0/4213951530 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fde800019f0 con 0x7fde880c9ff0 2023-10-16T10:04:30.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde7ffff640 1 -- 172.21.15.84:0/4213951530 >> v1:172.21.15.84:6789/0 conn(0x7fde880c9ff0 legacy=0x7fde880b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:30.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde7ffff640 1 --2- 172.21.15.84:0/4213951530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde740039e0 0x7fde74003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:30.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde7ffff640 1 --2- 172.21.15.84:0/4213951530 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8806f9d0 0x7fde740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:30.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde7ffff640 1 --2- 172.21.15.84:0/4213951530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde74004d00 0x7fde74005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:30.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde7ffff640 1 -- 172.21.15.84:0/4213951530 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde880cc860 con 0x7fde740039e0 2023-10-16T10:04:30.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde7ffff640 1 -- 172.21.15.84:0/4213951530 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7400b980 con 0x7fde74004d00 2023-10-16T10:04:30.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde7ffff640 1 -- 172.21.15.84:0/4213951530 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde7400baf0 con 0x7fde8806f9d0 2023-10-16T10:04:30.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde8d79c640 1 --2- 172.21.15.84:0/4213951530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde740039e0 0x7fde74003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:30.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde8cf9b640 1 --2- 172.21.15.84:0/4213951530 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8806f9d0 0x7fde740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:30.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde8d79c640 1 -- 172.21.15.84:0/4213951530 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8806f9d0 msgr2=0x7fde740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:30.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde8d79c640 1 --2- 172.21.15.84:0/4213951530 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8806f9d0 0x7fde740047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:30.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde8df9d640 1 --2- 172.21.15.84:0/4213951530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde74004d00 0x7fde74005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:30.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.466+0000 7fde8d79c640 1 -- 172.21.15.84:0/4213951530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde74004d00 msgr2=0x7fde74005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:30.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.467+0000 7fde8d79c640 1 --2- 172.21.15.84:0/4213951530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde74004d00 0x7fde74005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:30.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.467+0000 7fde8d79c640 1 -- 172.21.15.84:0/4213951530 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde74003520 con 0x7fde740039e0 2023-10-16T10:04:30.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.468+0000 7fde8d79c640 1 --2- 172.21.15.84:0/4213951530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde740039e0 0x7fde74003e00 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fde84009ac0 tx=0x7fde740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3aad762fb9d44d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:30.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.468+0000 7fde7ffff640 1 -- 172.21.15.84:0/4213951530 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8402c710 con 0x7fde740039e0 2023-10-16T10:04:30.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.468+0000 7fde7ffff640 1 -- 172.21.15.84:0/4213951530 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8402c870 con 0x7fde740039e0 2023-10-16T10:04:30.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.468+0000 7fde7ffff640 1 -- 172.21.15.84:0/4213951530 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8402cc00 con 0x7fde740039e0 2023-10-16T10:04:30.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.469+0000 7fde8fa00640 1 -- 172.21.15.84:0/4213951530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde740039e0 msgr2=0x7fde74003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:30.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.469+0000 7fde8fa00640 1 --2- 172.21.15.84:0/4213951530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde740039e0 0x7fde74003e00 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fde84009ac0 tx=0x7fde740057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:30.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.469+0000 7fde8df9d640 1 -- 172.21.15.84:0/4213951530 reap_dead start 2023-10-16T10:04:30.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.469+0000 7fde8fa00640 1 -- 172.21.15.84:0/4213951530 shutdown_connections 2023-10-16T10:04:30.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.469+0000 7fde8fa00640 1 -- 172.21.15.84:0/4213951530 >> 172.21.15.84:0/4213951530 conn(0x7fde880cb360 msgr2=0x7fde880cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:30.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.469+0000 7fde8fa00640 1 -- 172.21.15.84:0/4213951530 shutdown_connections 2023-10-16T10:04:30.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.469+0000 7fde8fa00640 1 -- 172.21.15.84:0/4213951530 wait complete. 2023-10-16T10:04:30.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.470+0000 7fde8fa00640 1 Processor -- start 2023-10-16T10:04:30.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.471+0000 7fde8fa00640 1 -- start start 2023-10-16T10:04:30.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.472+0000 7fde8fa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde740039e0 0x7fde880b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:30.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.472+0000 7fde8fa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde88070590 0x7fde88157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:30.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.472+0000 7fde8fa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde880709f0 0x7fde88157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:30.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.472+0000 7fde8fa00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde880b1370 con 0x7fde880709f0 2023-10-16T10:04:30.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.472+0000 7fde8fa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde880b15c0 con 0x7fde740039e0 2023-10-16T10:04:30.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.472+0000 7fde8fa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde880b18a0 con 0x7fde88070590 2023-10-16T10:04:30.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.472+0000 7fde8df9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde880709f0 0x7fde88157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:30.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.472+0000 7fde8df9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde880709f0 0x7fde88157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41122/0 (socket says 172.21.15.84:41122) 2023-10-16T10:04:30.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.472+0000 7fde8df9d640 1 -- 172.21.15.84:0/538301079 learned_addr learned my addr 172.21.15.84:0/538301079 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:30.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.472+0000 7fde8d79c640 1 --2- 172.21.15.84:0/538301079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde740039e0 0x7fde880b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:30.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.472+0000 7fde8cf9b640 1 --2- 172.21.15.84:0/538301079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde88070590 0x7fde88157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:30.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.472+0000 7fde8df9d640 1 -- 172.21.15.84:0/538301079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde88070590 msgr2=0x7fde88157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:30.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.472+0000 7fde8df9d640 1 --2- 172.21.15.84:0/538301079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde88070590 0x7fde88157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:30.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.472+0000 7fde8df9d640 1 -- 172.21.15.84:0/538301079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde740039e0 msgr2=0x7fde880b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:30.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.472+0000 7fde8df9d640 1 --2- 172.21.15.84:0/538301079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde740039e0 0x7fde880b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:30.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.472+0000 7fde8df9d640 1 -- 172.21.15.84:0/538301079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde880b1b80 con 0x7fde880709f0 2023-10-16T10:04:30.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.473+0000 7fde8df9d640 1 --2- 172.21.15.84:0/538301079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde880709f0 0x7fde88157f70 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fde880b1340 tx=0x7fde8000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:30.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.473+0000 7fde7dffb640 1 -- 172.21.15.84:0/538301079 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde80004d60 con 0x7fde880709f0 2023-10-16T10:04:30.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.473+0000 7fde7dffb640 1 -- 172.21.15.84:0/538301079 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8000bb80 con 0x7fde880709f0 2023-10-16T10:04:30.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.473+0000 7fde8fa00640 1 -- 172.21.15.84:0/538301079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde880b1e60 con 0x7fde880709f0 2023-10-16T10:04:30.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.473+0000 7fde7dffb640 1 -- 172.21.15.84:0/538301079 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde80007110 con 0x7fde880709f0 2023-10-16T10:04:30.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.473+0000 7fde8fa00640 1 -- 172.21.15.84:0/538301079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde880b2cb0 con 0x7fde880709f0 2023-10-16T10:04:30.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.474+0000 7fde8fa00640 1 -- 172.21.15.84:0/538301079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde880b2050 con 0x7fde880709f0 2023-10-16T10:04:30.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.475+0000 7fde7dffb640 1 -- 172.21.15.84:0/538301079 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fde80007270 con 0x7fde880709f0 2023-10-16T10:04:30.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.476+0000 7fde7dffb640 1 --2- 172.21.15.84:0/538301079 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fde6c06afc0 0x7fde6c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:30.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.476+0000 7fde7dffb640 1 -- 172.21.15.84:0/538301079 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fde80089920 con 0x7fde880709f0 2023-10-16T10:04:30.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.477+0000 7fde8d79c640 1 --2- 172.21.15.84:0/538301079 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fde6c06afc0 0x7fde6c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:30.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.477+0000 7fde8d79c640 1 --2- 172.21.15.84:0/538301079 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fde6c06afc0 0x7fde6c06d480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fde84036000 tx=0x7fde84030000 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:30.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.481+0000 7fde7dffb640 1 -- 172.21.15.84:0/538301079 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fde80054af0 con 0x7fde880709f0 2023-10-16T10:04:30.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:30.672+0000 7fde8fa00640 1 -- 172.21.15.84:0/538301079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fde8804e160 con 0x7fde880709f0 2023-10-16T10:04:31.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.362+0000 7fde7dffb640 1 -- 172.21.15.84:0/538301079 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) v1 ==== 138+0+0 (secure 0 0 0) 0x7fde80058110 con 0x7fde880709f0 2023-10-16T10:04:31.363 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-16T10:04:31.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.366+0000 7fde8fa00640 1 -- 172.21.15.84:0/538301079 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fde6c06afc0 msgr2=0x7fde6c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:31.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.366+0000 7fde8fa00640 1 --2- 172.21.15.84:0/538301079 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fde6c06afc0 0x7fde6c06d480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fde84036000 tx=0x7fde84030000 comp rx=0 tx=0).stop 2023-10-16T10:04:31.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.366+0000 7fde8fa00640 1 -- 172.21.15.84:0/538301079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde880709f0 msgr2=0x7fde88157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:31.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.366+0000 7fde8fa00640 1 --2- 172.21.15.84:0/538301079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde880709f0 0x7fde88157f70 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fde880b1340 tx=0x7fde8000b4a0 comp rx=0 tx=0).stop 2023-10-16T10:04:31.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.367+0000 7fde8fa00640 1 -- 172.21.15.84:0/538301079 shutdown_connections 2023-10-16T10:04:31.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.367+0000 7fde8fa00640 1 --2- 172.21.15.84:0/538301079 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fde6c06afc0 0x7fde6c06d480 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:31.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.367+0000 7fde8fa00640 1 --2- 172.21.15.84:0/538301079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde880709f0 0x7fde88157f70 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:31.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.367+0000 7fde8fa00640 1 --2- 172.21.15.84:0/538301079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde88070590 0x7fde88157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:31.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.367+0000 7fde8fa00640 1 --2- 172.21.15.84:0/538301079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde740039e0 0x7fde880b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:31.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.367+0000 7fde8fa00640 1 -- 172.21.15.84:0/538301079 >> 172.21.15.84:0/538301079 conn(0x7fde880cb360 msgr2=0x7fde880ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:31.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.367+0000 7fde8fa00640 1 -- 172.21.15.84:0/538301079 shutdown_connections 2023-10-16T10:04:31.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.367+0000 7fde8fa00640 1 -- 172.21.15.84:0/538301079 wait complete. 2023-10-16T10:04:31.390 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T10:04:31.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.601+0000 7f7e67f8a640 1 Processor -- start 2023-10-16T10:04:31.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.601+0000 7f7e67f8a640 1 -- start start 2023-10-16T10:04:31.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.602+0000 7f7e67f8a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7e6015aaa0 0x7f7e6015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:31.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.602+0000 7f7e67f8a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e600707e0 con 0x7f7e6015ae80 2023-10-16T10:04:31.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.602+0000 7f7e67f8a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6006b7a0 con 0x7f7e600ce390 2023-10-16T10:04:31.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.602+0000 7f7e67f8a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e6015ef20 con 0x7f7e6015aaa0 2023-10-16T10:04:31.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.603+0000 7f7e65d26640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7e6015aaa0 0x7f7e6015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:31.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.603+0000 7f7e65525640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7e6015ae80 0x7f7e600cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40516/0 (socket says 172.21.15.84:40516) 2023-10-16T10:04:31.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.603+0000 7f7e65525640 1 -- 172.21.15.84:0/1613800582 learned_addr learned my addr 172.21.15.84:0/1613800582 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:31.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.603+0000 7f7e65d26640 1 -- 172.21.15.84:0/1613800582 >> v1:172.21.15.157:6789/0 conn(0x7f7e600ce390 legacy=0x7f7e60070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:31.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.603+0000 7f7e65d26640 1 -- 172.21.15.84:0/1613800582 >> v1:172.21.15.84:6789/0 conn(0x7f7e6015ae80 legacy=0x7f7e600cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:31.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.603+0000 7f7e65d26640 1 -- 172.21.15.84:0/1613800582 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6015f200 con 0x7f7e6015aaa0 2023-10-16T10:04:31.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.604+0000 7f7e65d26640 1 --2- 172.21.15.84:0/1613800582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e6015aaa0 0x7f7e6015e820 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0097a0 tx=0x7f7e5c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797c20a8a299b0b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:31.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.604+0000 7f7e64d24640 1 -- 172.21.15.84:0/1613800582 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c02e070 con 0x7f7e6015aaa0 2023-10-16T10:04:31.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.605+0000 7f7e64d24640 1 -- 172.21.15.84:0/1613800582 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c0315a0 con 0x7f7e6015aaa0 2023-10-16T10:04:31.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.605+0000 7f7e64d24640 1 -- 172.21.15.84:0/1613800582 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c031930 con 0x7f7e6015aaa0 2023-10-16T10:04:31.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.605+0000 7f7e67f8a640 1 -- 172.21.15.84:0/1613800582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e6015aaa0 msgr2=0x7f7e6015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:31.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.605+0000 7f7e67f8a640 1 --2- 172.21.15.84:0/1613800582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e6015aaa0 0x7f7e6015e820 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0097a0 tx=0x7f7e5c02b840 comp rx=0 tx=0).stop 2023-10-16T10:04:31.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.605+0000 7f7e67f8a640 1 -- 172.21.15.84:0/1613800582 shutdown_connections 2023-10-16T10:04:31.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.605+0000 7f7e67f8a640 1 --2- 172.21.15.84:0/1613800582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e6015aaa0 0x7f7e6015e820 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:31.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.605+0000 7f7e67f8a640 1 -- 172.21.15.84:0/1613800582 >> 172.21.15.84:0/1613800582 conn(0x7f7e600cb410 msgr2=0x7f7e600cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:31.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.605+0000 7f7e67f8a640 1 -- 172.21.15.84:0/1613800582 shutdown_connections 2023-10-16T10:04:31.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.606+0000 7f7e67f8a640 1 -- 172.21.15.84:0/1613800582 wait complete. 2023-10-16T10:04:31.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.608+0000 7f7e67f8a640 1 Processor -- start 2023-10-16T10:04:31.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.608+0000 7f7e67f8a640 1 -- start start 2023-10-16T10:04:31.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.609+0000 7f7e67f8a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e600ce390 0x7f7e601f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:31.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.609+0000 7f7e67f8a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e6015aaa0 0x7f7e600c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:31.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.609+0000 7f7e67f8a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e6015ae80 0x7f7e600ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:31.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.609+0000 7f7e67f8a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e601f3910 con 0x7f7e600ce390 2023-10-16T10:04:31.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.609+0000 7f7e67f8a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6015b4e0 con 0x7f7e6015ae80 2023-10-16T10:04:31.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.609+0000 7f7e67f8a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e601f3070 con 0x7f7e6015aaa0 2023-10-16T10:04:31.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.609+0000 7f7e65d26640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e600ce390 0x7f7e601f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:31.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.609+0000 7f7e65d26640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e600ce390 0x7f7e601f29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41134/0 (socket says 172.21.15.84:41134) 2023-10-16T10:04:31.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.609+0000 7f7e65d26640 1 -- 172.21.15.84:0/401601366 learned_addr learned my addr 172.21.15.84:0/401601366 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:31.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.610+0000 7f7e65d26640 1 -- 172.21.15.84:0/401601366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e6015aaa0 msgr2=0x7f7e600c3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:31.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.610+0000 7f7e65d26640 1 --2- 172.21.15.84:0/401601366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e6015aaa0 0x7f7e600c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:31.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.610+0000 7f7e65d26640 1 -- 172.21.15.84:0/401601366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e6015ae80 msgr2=0x7f7e600ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:31.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.610+0000 7f7e65d26640 1 --2- 172.21.15.84:0/401601366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e6015ae80 0x7f7e600ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:31.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.610+0000 7f7e65d26640 1 -- 172.21.15.84:0/401601366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c009400 con 0x7f7e600ce390 2023-10-16T10:04:31.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.610+0000 7f7e65d26640 1 --2- 172.21.15.84:0/401601366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e600ce390 0x7f7e601f29e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c034040 tx=0x7f7e5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:31.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.610+0000 7f7e567fc640 1 -- 172.21.15.84:0/401601366 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c032900 con 0x7f7e600ce390 2023-10-16T10:04:31.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.610+0000 7f7e567fc640 1 -- 172.21.15.84:0/401601366 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c032a80 con 0x7f7e600ce390 2023-10-16T10:04:31.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.610+0000 7f7e67f8a640 1 -- 172.21.15.84:0/401601366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e601f3350 con 0x7f7e600ce390 2023-10-16T10:04:31.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.610+0000 7f7e567fc640 1 -- 172.21.15.84:0/401601366 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c032e30 con 0x7f7e600ce390 2023-10-16T10:04:31.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.610+0000 7f7e67f8a640 1 -- 172.21.15.84:0/401601366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e600c42e0 con 0x7f7e600ce390 2023-10-16T10:04:31.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.612+0000 7f7e67f8a640 1 -- 172.21.15.84:0/401601366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e600c44a0 con 0x7f7e600ce390 2023-10-16T10:04:31.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.613+0000 7f7e567fc640 1 -- 172.21.15.84:0/401601366 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7e5c04e470 con 0x7f7e600ce390 2023-10-16T10:04:31.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.614+0000 7f7e567fc640 1 --2- 172.21.15.84:0/401601366 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f7e4406afc0 0x7f7e4406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:31.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.614+0000 7f7e567fc640 1 -- 172.21.15.84:0/401601366 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f7e5c0ab770 con 0x7f7e600ce390 2023-10-16T10:04:31.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.615+0000 7f7e65525640 1 --2- 172.21.15.84:0/401601366 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f7e4406afc0 0x7f7e4406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:31.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.616+0000 7f7e65525640 1 --2- 172.21.15.84:0/401601366 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f7e4406afc0 0x7f7e4406d480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7e60156f20 tx=0x7f7e50007a40 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.618+0000 7f7e567fc640 1 -- 172.21.15.84:0/401601366 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7e5c0767c0 con 0x7f7e600ce390 2023-10-16T10:04:31.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:31.811+0000 7f7e67f8a640 1 -- 172.21.15.84:0/401601366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7e6004e160 con 0x7f7e600ce390 2023-10-16T10:04:32.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.378+0000 7f7e567fc640 1 -- 172.21.15.84:0/401601366 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7e5c079de0 con 0x7f7e600ce390 2023-10-16T10:04:32.379 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-16T10:04:32.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.382+0000 7f7e67f8a640 1 -- 172.21.15.84:0/401601366 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f7e4406afc0 msgr2=0x7f7e4406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:32.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.382+0000 7f7e67f8a640 1 --2- 172.21.15.84:0/401601366 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f7e4406afc0 0x7f7e4406d480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7e60156f20 tx=0x7f7e50007a40 comp rx=0 tx=0).stop 2023-10-16T10:04:32.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.382+0000 7f7e67f8a640 1 -- 172.21.15.84:0/401601366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e600ce390 msgr2=0x7f7e601f29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:32.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.382+0000 7f7e67f8a640 1 --2- 172.21.15.84:0/401601366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e600ce390 0x7f7e601f29e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c034040 tx=0x7f7e5c0046c0 comp rx=0 tx=0).stop 2023-10-16T10:04:32.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.382+0000 7f7e67f8a640 1 -- 172.21.15.84:0/401601366 shutdown_connections 2023-10-16T10:04:32.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.382+0000 7f7e67f8a640 1 --2- 172.21.15.84:0/401601366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e6015ae80 0x7f7e600ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:32.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.382+0000 7f7e67f8a640 1 --2- 172.21.15.84:0/401601366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e6015aaa0 0x7f7e600c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:32.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.382+0000 7f7e67f8a640 1 --2- 172.21.15.84:0/401601366 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f7e4406afc0 0x7f7e4406d480 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7e60156f20 tx=0x7f7e50007a40 comp rx=0 tx=0).stop 2023-10-16T10:04:32.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.382+0000 7f7e67f8a640 1 --2- 172.21.15.84:0/401601366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e600ce390 0x7f7e601f29e0 secure :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c034040 tx=0x7f7e5c0046c0 comp rx=0 tx=0).stop 2023-10-16T10:04:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.382+0000 7f7e67f8a640 1 -- 172.21.15.84:0/401601366 >> 172.21.15.84:0/401601366 conn(0x7f7e600cb410 msgr2=0x7f7e600b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.384+0000 7f7e67f8a640 1 -- 172.21.15.84:0/401601366 shutdown_connections 2023-10-16T10:04:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.384+0000 7f7e67f8a640 1 -- 172.21.15.84:0/401601366 wait complete. 2023-10-16T10:04:32.418 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T10:04:32.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.603+0000 7ffae359e640 1 Processor -- start 2023-10-16T10:04:32.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.603+0000 7ffae359e640 1 -- start start 2023-10-16T10:04:32.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.605+0000 7ffae359e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ffae40ca0a0 0x7ffae406c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:32.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.605+0000 7ffae359e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae415d910 con 0x7ffae4070b60 2023-10-16T10:04:32.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.605+0000 7ffae359e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae415dbf0 con 0x7ffae4159950 2023-10-16T10:04:32.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.605+0000 7ffae359e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae415e490 con 0x7ffae40ca0a0 2023-10-16T10:04:32.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.605+0000 7ffae2d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ffae40ca0a0 0x7ffae406c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:32.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.605+0000 7ffad9d9b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ffae4070b60 0x7ffae406c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40526/0 (socket says 172.21.15.84:40526) 2023-10-16T10:04:32.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.605+0000 7ffad9d9b640 1 -- 172.21.15.84:0/4052488983 learned_addr learned my addr 172.21.15.84:0/4052488983 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:32.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.606+0000 7ffae2d9d640 1 -- 172.21.15.84:0/4052488983 >> v1:172.21.15.157:6789/0 conn(0x7ffae4159950 legacy=0x7ffae4070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:32.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.606+0000 7ffae2d9d640 1 -- 172.21.15.84:0/4052488983 >> v1:172.21.15.84:6789/0 conn(0x7ffae4070b60 legacy=0x7ffae406c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:32.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.606+0000 7ffae2d9d640 1 -- 172.21.15.84:0/4052488983 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae415ded0 con 0x7ffae40ca0a0 2023-10-16T10:04:32.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.607+0000 7ffae2d9d640 1 --2- 172.21.15.84:0/4052488983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae40ca0a0 0x7ffae406c810 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ffadc009d90 tx=0x7ffadc02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b034c2a0eedb6779 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:32.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.607+0000 7ffae1d9b640 1 -- 172.21.15.84:0/4052488983 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc02cbd0 con 0x7ffae40ca0a0 2023-10-16T10:04:32.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.607+0000 7ffae1d9b640 1 -- 172.21.15.84:0/4052488983 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffadc02cd30 con 0x7ffae40ca0a0 2023-10-16T10:04:32.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.607+0000 7ffae1d9b640 1 -- 172.21.15.84:0/4052488983 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc02cea0 con 0x7ffae40ca0a0 2023-10-16T10:04:32.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.608+0000 7ffae359e640 1 -- 172.21.15.84:0/4052488983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae40ca0a0 msgr2=0x7ffae406c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:32.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.608+0000 7ffae359e640 1 --2- 172.21.15.84:0/4052488983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae40ca0a0 0x7ffae406c810 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ffadc009d90 tx=0x7ffadc02be20 comp rx=0 tx=0).stop 2023-10-16T10:04:32.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.608+0000 7ffae359e640 1 -- 172.21.15.84:0/4052488983 shutdown_connections 2023-10-16T10:04:32.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.608+0000 7ffae359e640 1 --2- 172.21.15.84:0/4052488983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae40ca0a0 0x7ffae406c810 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:32.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.608+0000 7ffae359e640 1 -- 172.21.15.84:0/4052488983 >> 172.21.15.84:0/4052488983 conn(0x7ffae40ce6a0 msgr2=0x7ffae40ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:32.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.608+0000 7ffae359e640 1 -- 172.21.15.84:0/4052488983 shutdown_connections 2023-10-16T10:04:32.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.608+0000 7ffae359e640 1 -- 172.21.15.84:0/4052488983 wait complete. 2023-10-16T10:04:32.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.610+0000 7ffae359e640 1 Processor -- start 2023-10-16T10:04:32.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.610+0000 7ffae359e640 1 -- start start 2023-10-16T10:04:32.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.611+0000 7ffae359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae4070b60 0x7ffae41576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:32.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.611+0000 7ffae359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae40ca0a0 0x7ffae4151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:32.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.611+0000 7ffae359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae4159950 0x7ffae406c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:32.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.611+0000 7ffae359e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae40b3460 con 0x7ffae4159950 2023-10-16T10:04:32.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.611+0000 7ffae359e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae4157c10 con 0x7ffae4070b60 2023-10-16T10:04:32.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.611+0000 7ffae359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae40b2520 con 0x7ffae40ca0a0 2023-10-16T10:04:32.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.611+0000 7ffae2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae4159950 0x7ffae406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:32.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.611+0000 7ffae2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae4159950 0x7ffae406c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41146/0 (socket says 172.21.15.84:41146) 2023-10-16T10:04:32.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.611+0000 7ffae2d9d640 1 -- 172.21.15.84:0/3011416920 learned_addr learned my addr 172.21.15.84:0/3011416920 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:32.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.611+0000 7ffae259c640 1 --2- 172.21.15.84:0/3011416920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae4070b60 0x7ffae41576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:32.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.611+0000 7ffae2d9d640 1 -- 172.21.15.84:0/3011416920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae40ca0a0 msgr2=0x7ffae4151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:32.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.611+0000 7ffae2d9d640 1 --2- 172.21.15.84:0/3011416920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae40ca0a0 0x7ffae4151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:32.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.611+0000 7ffae2d9d640 1 -- 172.21.15.84:0/3011416920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae4070b60 msgr2=0x7ffae41576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:32.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.611+0000 7ffae2d9d640 1 --2- 172.21.15.84:0/3011416920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae4070b60 0x7ffae41576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:32.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.611+0000 7ffae2d9d640 1 -- 172.21.15.84:0/3011416920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc009a70 con 0x7ffae4159950 2023-10-16T10:04:32.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.611+0000 7ffae259c640 1 --2- 172.21.15.84:0/3011416920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae4070b60 0x7ffae41576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.612+0000 7ffae2d9d640 1 --2- 172.21.15.84:0/3011416920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae4159950 0x7ffae406c090 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ffadc009d90 tx=0x7ffadc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.612+0000 7ffadb7fe640 1 -- 172.21.15.84:0/3011416920 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc02cea0 con 0x7ffae4159950 2023-10-16T10:04:32.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.612+0000 7ffadb7fe640 1 -- 172.21.15.84:0/3011416920 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffadc040450 con 0x7ffae4159950 2023-10-16T10:04:32.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.612+0000 7ffadb7fe640 1 -- 172.21.15.84:0/3011416920 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc0407e0 con 0x7ffae4159950 2023-10-16T10:04:32.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.612+0000 7ffae359e640 1 -- 172.21.15.84:0/3011416920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae40b27e0 con 0x7ffae4159950 2023-10-16T10:04:32.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.612+0000 7ffae359e640 1 -- 172.21.15.84:0/3011416920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae40b2da0 con 0x7ffae4159950 2023-10-16T10:04:32.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.613+0000 7ffae359e640 1 -- 172.21.15.84:0/3011416920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae40b35a0 con 0x7ffae4159950 2023-10-16T10:04:32.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.615+0000 7ffadb7fe640 1 -- 172.21.15.84:0/3011416920 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ffadc040940 con 0x7ffae4159950 2023-10-16T10:04:32.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.615+0000 7ffadb7fe640 1 --2- 172.21.15.84:0/3011416920 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7ffac406afc0 0x7ffac406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:32.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.616+0000 7ffadb7fe640 1 -- 172.21.15.84:0/3011416920 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5609+0+0 (secure 0 0 0) 0x7ffadc0abb70 con 0x7ffae4159950 2023-10-16T10:04:32.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.616+0000 7ffae259c640 1 --2- 172.21.15.84:0/3011416920 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7ffac406afc0 0x7ffac406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:32.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.616+0000 7ffae259c640 1 --2- 172.21.15.84:0/3011416920 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7ffac406afc0 0x7ffac406d480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffae40c52d0 tx=0x7ffad4008040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:32.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.620+0000 7ffadb7fe640 1 -- 172.21.15.84:0/3011416920 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffae40b35a0 con 0x7ffae4159950 2023-10-16T10:04:32.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:32.847+0000 7ffae359e640 1 -- 172.21.15.84:0/3011416920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ffae40b35a0 con 0x7ffae4159950 2023-10-16T10:04:33.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.392+0000 7ffadb7fe640 1 -- 172.21.15.84:0/3011416920 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v31) v1 ==== 172+0+0 (secure 0 0 0) 0x7ffae40b35a0 con 0x7ffae4159950 2023-10-16T10:04:33.394 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 8 and data pool 9 2023-10-16T10:04:33.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.396+0000 7ffae359e640 1 -- 172.21.15.84:0/3011416920 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7ffac406afc0 msgr2=0x7ffac406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:33.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.396+0000 7ffae359e640 1 --2- 172.21.15.84:0/3011416920 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7ffac406afc0 0x7ffac406d480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffae40c52d0 tx=0x7ffad4008040 comp rx=0 tx=0).stop 2023-10-16T10:04:33.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.396+0000 7ffae359e640 1 -- 172.21.15.84:0/3011416920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae4159950 msgr2=0x7ffae406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:33.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.396+0000 7ffae359e640 1 --2- 172.21.15.84:0/3011416920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae4159950 0x7ffae406c090 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ffadc009d90 tx=0x7ffadc004a40 comp rx=0 tx=0).stop 2023-10-16T10:04:33.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.397+0000 7ffae359e640 1 -- 172.21.15.84:0/3011416920 shutdown_connections 2023-10-16T10:04:33.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.397+0000 7ffae359e640 1 --2- 172.21.15.84:0/3011416920 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7ffac406afc0 0x7ffac406d480 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:33.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.397+0000 7ffae359e640 1 --2- 172.21.15.84:0/3011416920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae4159950 0x7ffae406c090 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:33.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.397+0000 7ffae359e640 1 --2- 172.21.15.84:0/3011416920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae40ca0a0 0x7ffae4151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:33.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.397+0000 7ffae359e640 1 --2- 172.21.15.84:0/3011416920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae4070b60 0x7ffae41576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:33.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.397+0000 7ffae359e640 1 -- 172.21.15.84:0/3011416920 >> 172.21.15.84:0/3011416920 conn(0x7ffae40ce6a0 msgr2=0x7ffae4070450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:33.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.397+0000 7ffae359e640 1 -- 172.21.15.84:0/3011416920 shutdown_connections 2023-10-16T10:04:33.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.397+0000 7ffae359e640 1 -- 172.21.15.84:0/3011416920 wait complete. 2023-10-16T10:04:33.423 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:04:33.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.607+0000 7f1e1610f640 1 Processor -- start 2023-10-16T10:04:33.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.607+0000 7f1e1610f640 1 -- start start 2023-10-16T10:04:33.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.608+0000 7f1e1610f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e10159f00 0x7f1e100b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:33.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.608+0000 7f1e1610f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1e1015d880 0x7f1e100b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:33.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.608+0000 7f1e1610f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1006f7f0 con 0x7f1e10159b20 2023-10-16T10:04:33.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.608+0000 7f1e1610f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1006c5e0 con 0x7f1e1015d880 2023-10-16T10:04:33.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.608+0000 7f1e1610f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1006c8c0 con 0x7f1e10159f00 2023-10-16T10:04:33.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.609+0000 7f1e07fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e10159f00 0x7f1e100b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:33.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.609+0000 7f1e07fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e10159f00 0x7f1e100b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:50570/0 (socket says 172.21.15.84:50570) 2023-10-16T10:04:33.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.609+0000 7f1e07fff640 1 -- 172.21.15.84:0/2513932545 learned_addr learned my addr 172.21.15.84:0/2513932545 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:33.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.609+0000 7f1e0ffff640 1 --2- 172.21.15.84:0/2513932545 >> v2:172.21.15.157:3300/0 conn(0x7f1e1015d880 0x7f1e100b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:33.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.610+0000 7f1e07fff640 1 -- 172.21.15.84:0/2513932545 >> v2:172.21.15.157:3300/0 conn(0x7f1e1015d880 msgr2=0x7f1e100b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.610+0000 7f1e07fff640 1 --2- 172.21.15.84:0/2513932545 >> v2:172.21.15.157:3300/0 conn(0x7f1e1015d880 0x7f1e100b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.610+0000 7f1e07fff640 1 -- 172.21.15.84:0/2513932545 >> v1:172.21.15.157:6789/0 conn(0x7f1e10159b20 legacy=0x7f1e1006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.610+0000 7f1e07fff640 1 -- 172.21.15.84:0/2513932545 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1006cba0 con 0x7f1e10159f00 2023-10-16T10:04:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.611+0000 7f1e07fff640 1 --2- 172.21.15.84:0/2513932545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e10159f00 0x7f1e100b0410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009870 tx=0x7f1df802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fabeefddaff3c383 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:33.615 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:04:33.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.612+0000 7f1e0effd640 1 -- 172.21.15.84:0/2513932545 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8030840 con 0x7f1e10159f00 2023-10-16T10:04:33.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.612+0000 7f1e0effd640 1 -- 172.21.15.84:0/2513932545 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df80309a0 con 0x7f1e10159f00 2023-10-16T10:04:33.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.612+0000 7f1e0effd640 1 -- 172.21.15.84:0/2513932545 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8030d30 con 0x7f1e10159f00 2023-10-16T10:04:33.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.613+0000 7f1e1610f640 1 -- 172.21.15.84:0/2513932545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e10159f00 msgr2=0x7f1e100b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:33.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.613+0000 7f1e1610f640 1 --2- 172.21.15.84:0/2513932545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e10159f00 0x7f1e100b0410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009870 tx=0x7f1df802b980 comp rx=0 tx=0).stop 2023-10-16T10:04:33.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.613+0000 7f1e1610f640 1 -- 172.21.15.84:0/2513932545 shutdown_connections 2023-10-16T10:04:33.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.613+0000 7f1e1610f640 1 --2- 172.21.15.84:0/2513932545 >> v2:172.21.15.157:3300/0 conn(0x7f1e1015d880 0x7f1e100b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:33.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.613+0000 7f1e1610f640 1 --2- 172.21.15.84:0/2513932545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e10159f00 0x7f1e100b0410 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:33.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.613+0000 7f1e1610f640 1 -- 172.21.15.84:0/2513932545 >> 172.21.15.84:0/2513932545 conn(0x7f1e100cb410 msgr2=0x7f1e100cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:33.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.613+0000 7f1e1610f640 1 -- 172.21.15.84:0/2513932545 shutdown_connections 2023-10-16T10:04:33.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.613+0000 7f1e1610f640 1 -- 172.21.15.84:0/2513932545 wait complete. 2023-10-16T10:04:33.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.615+0000 7f1e1610f640 1 Processor -- start 2023-10-16T10:04:33.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.615+0000 7f1e1610f640 1 -- start start 2023-10-16T10:04:33.620 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:04:33.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.616+0000 7f1e1610f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e10159b20 0x7f1e101500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:33.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.616+0000 7f1e1610f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e10159f00 0x7f1e100aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:33.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.616+0000 7f1e1610f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e1015d880 0x7f1e10151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:33.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.616+0000 7f1e1610f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1015add0 con 0x7f1e1015d880 2023-10-16T10:04:33.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.616+0000 7f1e1610f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e100b0080 con 0x7f1e10159b20 2023-10-16T10:04:33.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.616+0000 7f1e1610f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e10152000 con 0x7f1e10159f00 2023-10-16T10:04:33.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.616+0000 7f1e0ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e1015d880 0x7f1e10151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:33.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.616+0000 7f1e0ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e1015d880 0x7f1e10151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41148/0 (socket says 172.21.15.84:41148) 2023-10-16T10:04:33.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.616+0000 7f1e0ffff640 1 -- 172.21.15.84:0/804208796 learned_addr learned my addr 172.21.15.84:0/804208796 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:33.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.617+0000 7f1e07fff640 1 --2- 172.21.15.84:0/804208796 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e10159f00 0x7f1e100aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:33.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.617+0000 7f1e0ffff640 1 -- 172.21.15.84:0/804208796 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e10159f00 msgr2=0x7f1e100aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:33.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.617+0000 7f1e0ffff640 1 --2- 172.21.15.84:0/804208796 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e10159f00 0x7f1e100aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:33.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.617+0000 7f1e0ffff640 1 -- 172.21.15.84:0/804208796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e10159b20 msgr2=0x7f1e101500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:33.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.617+0000 7f1e0f7fe640 1 --2- 172.21.15.84:0/804208796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e10159b20 0x7f1e101500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:33.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.617+0000 7f1e0ffff640 1 --2- 172.21.15.84:0/804208796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e10159b20 0x7f1e101500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:33.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.617+0000 7f1e0ffff640 1 -- 172.21.15.84:0/804208796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df80094d0 con 0x7f1e1015d880 2023-10-16T10:04:33.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.617+0000 7f1e0f7fe640 1 --2- 172.21.15.84:0/804208796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e10159b20 0x7f1e101500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:33.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.617+0000 7f1e0ffff640 1 --2- 172.21.15.84:0/804208796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e1015d880 0x7f1e10151ac0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009d30 tx=0x7f1e0800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:33.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.617+0000 7f1e0cff9640 1 -- 172.21.15.84:0/804208796 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08009ba0 con 0x7f1e1015d880 2023-10-16T10:04:33.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.617+0000 7f1e0cff9640 1 -- 172.21.15.84:0/804208796 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0800f040 con 0x7f1e1015d880 2023-10-16T10:04:33.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.617+0000 7f1e0cff9640 1 -- 172.21.15.84:0/804208796 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e080146c0 con 0x7f1e1015d880 2023-10-16T10:04:33.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.617+0000 7f1e1610f640 1 -- 172.21.15.84:0/804208796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e100af470 con 0x7f1e1015d880 2023-10-16T10:04:33.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.618+0000 7f1e1610f640 1 -- 172.21.15.84:0/804208796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e100afa30 con 0x7f1e1015d880 2023-10-16T10:04:33.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.619+0000 7f1e1610f640 1 -- 172.21.15.84:0/804208796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1006c270 con 0x7f1e1015d880 2023-10-16T10:04:33.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.620+0000 7f1e0cff9640 1 -- 172.21.15.84:0/804208796 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1e08014820 con 0x7f1e1015d880 2023-10-16T10:04:33.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.621+0000 7f1e0cff9640 1 --2- 172.21.15.84:0/804208796 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f1df006af90 0x7f1df006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:33.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.621+0000 7f1e0cff9640 1 -- 172.21.15.84:0/804208796 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e0808a400 con 0x7f1e1015d880 2023-10-16T10:04:33.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.621+0000 7f1e0f7fe640 1 --2- 172.21.15.84:0/804208796 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f1df006af90 0x7f1df006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:33.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.622+0000 7f1e0f7fe640 1 --2- 172.21.15.84:0/804208796 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f1df006af90 0x7f1df006d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1e00000f60 tx=0x7f1e00009040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:33.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.625+0000 7f1e0cff9640 1 -- 172.21.15.84:0/804208796 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1e08055290 con 0x7f1e1015d880 2023-10-16T10:04:33.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.818+0000 7f1e1610f640 1 -- 172.21.15.84:0/804208796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e1004e160 con 0x7f1e1015d880 2023-10-16T10:04:33.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.820+0000 7f1e0cff9640 1 -- 172.21.15.84:0/804208796 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16554 (secure 0 0 0) 0x7f1e080588b0 con 0x7f1e1015d880 2023-10-16T10:04:33.822 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:33.822 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":81,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:04:32.848646+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:04:30.728624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-16T10:04:31.866029+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:04:33.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.822+0000 7f1e1610f640 1 -- 172.21.15.84:0/804208796 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f1df006af90 msgr2=0x7f1df006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:33.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.822+0000 7f1e1610f640 1 --2- 172.21.15.84:0/804208796 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f1df006af90 0x7f1df006d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1e00000f60 tx=0x7f1e00009040 comp rx=0 tx=0).stop 2023-10-16T10:04:33.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.823+0000 7f1e1610f640 1 -- 172.21.15.84:0/804208796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e1015d880 msgr2=0x7f1e10151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:33.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.823+0000 7f1e1610f640 1 --2- 172.21.15.84:0/804208796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e1015d880 0x7f1e10151ac0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009d30 tx=0x7f1e0800c490 comp rx=0 tx=0).stop 2023-10-16T10:04:33.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.823+0000 7f1e1610f640 1 -- 172.21.15.84:0/804208796 shutdown_connections 2023-10-16T10:04:33.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.823+0000 7f1e1610f640 1 --2- 172.21.15.84:0/804208796 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f1df006af90 0x7f1df006d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:33.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.823+0000 7f1e1610f640 1 --2- 172.21.15.84:0/804208796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e1015d880 0x7f1e10151ac0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:33.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.823+0000 7f1e1610f640 1 --2- 172.21.15.84:0/804208796 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e10159f00 0x7f1e100aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:33.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.823+0000 7f1e1610f640 1 --2- 172.21.15.84:0/804208796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e10159b20 0x7f1e101500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:33.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.823+0000 7f1e1610f640 1 -- 172.21.15.84:0/804208796 >> 172.21.15.84:0/804208796 conn(0x7f1e100cb410 msgr2=0x7f1e101566a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:33.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.823+0000 7f1e1610f640 1 -- 172.21.15.84:0/804208796 shutdown_connections 2023-10-16T10:04:33.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:33.823+0000 7f1e1610f640 1 -- 172.21.15.84:0/804208796 wait complete. 2023-10-16T10:04:33.843 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:04:34.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.049+0000 7f51e3797640 1 Processor -- start 2023-10-16T10:04:34.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.049+0000 7f51e3797640 1 -- start start 2023-10-16T10:04:34.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.051+0000 7f51e3797640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f51dc0b0b20 0x7f51dc1615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:34.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.051+0000 7f51e3797640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f51dc15d3f0 0x7f51dc15d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:34.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.051+0000 7f51e3797640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51dc15e3e0 con 0x7f51dc0cd0c0 2023-10-16T10:04:34.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.051+0000 7f51e3797640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc0cd4a0 con 0x7f51dc15d3f0 2023-10-16T10:04:34.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.051+0000 7f51e3797640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc15e100 con 0x7f51dc0b0b20 2023-10-16T10:04:34.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.051+0000 7f51e1533640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f51dc0cd0c0 0x7f51dc0b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40528/0 (socket says 172.21.15.84:40528) 2023-10-16T10:04:34.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.051+0000 7f51e1d34640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f51dc15d3f0 0x7f51dc15d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:34.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.051+0000 7f51e1533640 1 -- 172.21.15.84:0/306045933 learned_addr learned my addr 172.21.15.84:0/306045933 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:34.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.051+0000 7f51e0d32640 1 --2- 172.21.15.84:0/306045933 >> v2:172.21.15.157:3300/0 conn(0x7f51dc0b0b20 0x7f51dc1615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:34.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.052+0000 7f51e1d34640 1 -- 172.21.15.84:0/306045933 >> v2:172.21.15.157:3300/0 conn(0x7f51dc0b0b20 msgr2=0x7f51dc1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:34.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.052+0000 7f51e1d34640 1 --2- 172.21.15.84:0/306045933 >> v2:172.21.15.157:3300/0 conn(0x7f51dc0b0b20 0x7f51dc1615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:34.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.052+0000 7f51e1d34640 1 -- 172.21.15.84:0/306045933 >> v1:172.21.15.84:6789/0 conn(0x7f51dc0cd0c0 legacy=0x7f51dc0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:34.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.052+0000 7f51d3fff640 1 -- 172.21.15.84:0/306045933 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297281317 0 0) 0x7f51dc15e3e0 con 0x7f51dc0cd0c0 2023-10-16T10:04:34.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.052+0000 7f51e1d34640 1 -- 172.21.15.84:0/306045933 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc160b10 con 0x7f51dc15d3f0 2023-10-16T10:04:34.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.053+0000 7f51e1d34640 1 --2- 172.21.15.84:0/306045933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51dc15d3f0 0x7f51dc15d820 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f51dc15afc0 tx=0x7f51d402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65185bddb00e9678 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:34.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.053+0000 7f51d3fff640 1 -- 172.21.15.84:0/306045933 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d402cbf0 con 0x7f51dc15d3f0 2023-10-16T10:04:34.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.053+0000 7f51d3fff640 1 -- 172.21.15.84:0/306045933 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d402cd50 con 0x7f51dc15d3f0 2023-10-16T10:04:34.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.053+0000 7f51d3fff640 1 -- 172.21.15.84:0/306045933 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4034660 con 0x7f51dc15d3f0 2023-10-16T10:04:34.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.054+0000 7f51e3797640 1 -- 172.21.15.84:0/306045933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51dc15d3f0 msgr2=0x7f51dc15d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:34.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.054+0000 7f51e3797640 1 --2- 172.21.15.84:0/306045933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51dc15d3f0 0x7f51dc15d820 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f51dc15afc0 tx=0x7f51d402be40 comp rx=0 tx=0).stop 2023-10-16T10:04:34.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.054+0000 7f51e3797640 1 -- 172.21.15.84:0/306045933 shutdown_connections 2023-10-16T10:04:34.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.054+0000 7f51e3797640 1 --2- 172.21.15.84:0/306045933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51dc15d3f0 0x7f51dc15d820 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:34.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.054+0000 7f51e3797640 1 --2- 172.21.15.84:0/306045933 >> v2:172.21.15.157:3300/0 conn(0x7f51dc0b0b20 0x7f51dc1615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:34.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.054+0000 7f51e3797640 1 -- 172.21.15.84:0/306045933 >> 172.21.15.84:0/306045933 conn(0x7f51dc0cb410 msgr2=0x7f51dc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:34.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.054+0000 7f51e3797640 1 -- 172.21.15.84:0/306045933 shutdown_connections 2023-10-16T10:04:34.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.054+0000 7f51e3797640 1 -- 172.21.15.84:0/306045933 wait complete. 2023-10-16T10:04:34.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.056+0000 7f51e3797640 1 Processor -- start 2023-10-16T10:04:34.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.056+0000 7f51e3797640 1 -- start start 2023-10-16T10:04:34.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.057+0000 7f51e3797640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51dc0b0b20 0x7f51dc0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:34.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.057+0000 7f51e3797640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51dc0cd0c0 0x7f51dc1f3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:34.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.057+0000 7f51e3797640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51dc15d3f0 0x7f51dc1f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:34.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.057+0000 7f51e3797640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0c7bb0 con 0x7f51dc0b0b20 2023-10-16T10:04:34.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.057+0000 7f51e3797640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0c3d60 con 0x7f51dc15d3f0 2023-10-16T10:04:34.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.057+0000 7f51e3797640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0c4040 con 0x7f51dc0cd0c0 2023-10-16T10:04:34.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.057+0000 7f51e1533640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51dc0b0b20 0x7f51dc0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:34.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.057+0000 7f51e1533640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51dc0b0b20 0x7f51dc0c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41166/0 (socket says 172.21.15.84:41166) 2023-10-16T10:04:34.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.058+0000 7f51e1533640 1 -- 172.21.15.84:0/2694858624 learned_addr learned my addr 172.21.15.84:0/2694858624 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:34.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.058+0000 7f51e1d34640 1 --2- 172.21.15.84:0/2694858624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51dc15d3f0 0x7f51dc1f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:34.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.058+0000 7f51e1533640 1 -- 172.21.15.84:0/2694858624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51dc0cd0c0 msgr2=0x7f51dc1f3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:34.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.058+0000 7f51e1533640 1 --2- 172.21.15.84:0/2694858624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51dc0cd0c0 0x7f51dc1f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:34.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.058+0000 7f51e1533640 1 -- 172.21.15.84:0/2694858624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51dc15d3f0 msgr2=0x7f51dc1f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.058+0000 7f51e1533640 1 --2- 172.21.15.84:0/2694858624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51dc15d3f0 0x7f51dc1f1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.058+0000 7f51e1533640 1 -- 172.21.15.84:0/2694858624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d4009a70 con 0x7f51dc0b0b20 2023-10-16T10:04:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.058+0000 7f51e1533640 1 --2- 172.21.15.84:0/2694858624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51dc0b0b20 0x7f51dc0c6470 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f51d800c430 tx=0x7f51d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.058+0000 7f51d1ffb640 1 -- 172.21.15.84:0/2694858624 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d80074e0 con 0x7f51dc0b0b20 2023-10-16T10:04:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.058+0000 7f51d1ffb640 1 -- 172.21.15.84:0/2694858624 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d8007660 con 0x7f51dc0b0b20 2023-10-16T10:04:34.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.058+0000 7f51d1ffb640 1 -- 172.21.15.84:0/2694858624 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8004a20 con 0x7f51dc0b0b20 2023-10-16T10:04:34.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.059+0000 7f51e3797640 1 -- 172.21.15.84:0/2694858624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51dc0c4320 con 0x7f51dc0b0b20 2023-10-16T10:04:34.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.059+0000 7f51e3797640 1 -- 172.21.15.84:0/2694858624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51dc1f3c60 con 0x7f51dc0b0b20 2023-10-16T10:04:34.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.060+0000 7f51e3797640 1 -- 172.21.15.84:0/2694858624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51dc1f3e50 con 0x7f51dc0b0b20 2023-10-16T10:04:34.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.061+0000 7f51d1ffb640 1 -- 172.21.15.84:0/2694858624 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f51d8004b80 con 0x7f51dc0b0b20 2023-10-16T10:04:34.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.062+0000 7f51d1ffb640 1 --2- 172.21.15.84:0/2694858624 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f51c006af90 0x7f51c006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:34.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.062+0000 7f51d1ffb640 1 -- 172.21.15.84:0/2694858624 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51d8089590 con 0x7f51dc0b0b20 2023-10-16T10:04:34.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.062+0000 7f51e0d32640 1 --2- 172.21.15.84:0/2694858624 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f51c006af90 0x7f51c006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:34.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.063+0000 7f51e0d32640 1 --2- 172.21.15.84:0/2694858624 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f51c006af90 0x7f51c006d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51cc006fd0 tx=0x7f51cc008040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:34.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.066+0000 7f51d1ffb640 1 -- 172.21.15.84:0/2694858624 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f51d80545c0 con 0x7f51dc0b0b20 2023-10-16T10:04:34.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.264+0000 7f51e3797640 1 -- 172.21.15.84:0/2694858624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51dc04e160 con 0x7f51dc0b0b20 2023-10-16T10:04:34.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.267+0000 7f51d1ffb640 1 -- 172.21.15.84:0/2694858624 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f51d8057be0 con 0x7f51dc0b0b20 2023-10-16T10:04:34.269 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:34.269 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":82,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:04:33.896255+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:04:30.728624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-16T10:04:31.866029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:04:34.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.269+0000 7f51e3797640 1 -- 172.21.15.84:0/2694858624 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f51c006af90 msgr2=0x7f51c006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:34.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.269+0000 7f51e3797640 1 --2- 172.21.15.84:0/2694858624 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f51c006af90 0x7f51c006d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51cc006fd0 tx=0x7f51cc008040 comp rx=0 tx=0).stop 2023-10-16T10:04:34.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.269+0000 7f51e3797640 1 -- 172.21.15.84:0/2694858624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51dc0b0b20 msgr2=0x7f51dc0c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:34.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.269+0000 7f51e3797640 1 --2- 172.21.15.84:0/2694858624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51dc0b0b20 0x7f51dc0c6470 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f51d800c430 tx=0x7f51d800c8f0 comp rx=0 tx=0).stop 2023-10-16T10:04:34.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.269+0000 7f51e3797640 1 -- 172.21.15.84:0/2694858624 shutdown_connections 2023-10-16T10:04:34.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.269+0000 7f51e3797640 1 --2- 172.21.15.84:0/2694858624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51dc15d3f0 0x7f51dc1f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:34.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.269+0000 7f51e3797640 1 --2- 172.21.15.84:0/2694858624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51dc0cd0c0 0x7f51dc1f3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:34.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.269+0000 7f51e3797640 1 --2- 172.21.15.84:0/2694858624 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f51c006af90 0x7f51c006d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:34.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.269+0000 7f51e3797640 1 --2- 172.21.15.84:0/2694858624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51dc0b0b20 0x7f51dc0c6470 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:34.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.269+0000 7f51e3797640 1 -- 172.21.15.84:0/2694858624 >> 172.21.15.84:0/2694858624 conn(0x7f51dc0cb410 msgr2=0x7f51dc0ca9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:34.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.270+0000 7f51e3797640 1 -- 172.21.15.84:0/2694858624 shutdown_connections 2023-10-16T10:04:34.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.270+0000 7f51e3797640 1 -- 172.21.15.84:0/2694858624 wait complete. 2023-10-16T10:04:34.290 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T10:04:34.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.496+0000 7f9982ba9640 1 Processor -- start 2023-10-16T10:04:34.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.497+0000 7f9982ba9640 1 -- start start 2023-10-16T10:04:34.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.498+0000 7f9982ba9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f997c06a4b0 0x7f997c0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:34.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.498+0000 7f9982ba9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997c0cb030 con 0x7f997c15ad70 2023-10-16T10:04:34.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.498+0000 7f9982ba9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997c0c91c0 con 0x7f997c0c9ca0 2023-10-16T10:04:34.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.498+0000 7f9982ba9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997c0c94a0 con 0x7f997c06a4b0 2023-10-16T10:04:34.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.498+0000 7f9980945640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f997c15ad70 0x7f997c069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40536/0 (socket says 172.21.15.84:40536) 2023-10-16T10:04:34.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.498+0000 7f997bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f997c06a4b0 0x7f997c0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:34.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.498+0000 7f9980945640 1 -- 172.21.15.84:0/3172955312 learned_addr learned my addr 172.21.15.84:0/3172955312 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:34.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.499+0000 7f997bfff640 1 -- 172.21.15.84:0/3172955312 >> v1:172.21.15.157:6789/0 conn(0x7f997c0c9ca0 legacy=0x7f997c0ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:34.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.499+0000 7f997bfff640 1 -- 172.21.15.84:0/3172955312 >> v1:172.21.15.84:6789/0 conn(0x7f997c15ad70 legacy=0x7f997c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:34.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.499+0000 7f997bfff640 1 -- 172.21.15.84:0/3172955312 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c0c9780 con 0x7f997c06a4b0 2023-10-16T10:04:34.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.499+0000 7f997b7fe640 1 -- 172.21.15.84:0/3172955312 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683671728 0 0) 0x7f997c0cb030 con 0x7f997c15ad70 2023-10-16T10:04:34.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.500+0000 7f997bfff640 1 --2- 172.21.15.84:0/3172955312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c06a4b0 0x7f997c0ca3b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f99640097a0 tx=0x7f996402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6676ec9a79a2caf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:34.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.500+0000 7f997b7fe640 1 -- 172.21.15.84:0/3172955312 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9964031830 con 0x7f997c06a4b0 2023-10-16T10:04:34.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.500+0000 7f997b7fe640 1 -- 172.21.15.84:0/3172955312 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9964031990 con 0x7f997c06a4b0 2023-10-16T10:04:34.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.500+0000 7f997b7fe640 1 -- 172.21.15.84:0/3172955312 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9964031d20 con 0x7f997c06a4b0 2023-10-16T10:04:34.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.501+0000 7f9982ba9640 1 -- 172.21.15.84:0/3172955312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c06a4b0 msgr2=0x7f997c0ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:34.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.501+0000 7f9982ba9640 1 --2- 172.21.15.84:0/3172955312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c06a4b0 0x7f997c0ca3b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f99640097a0 tx=0x7f996402b8d0 comp rx=0 tx=0).stop 2023-10-16T10:04:34.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.501+0000 7f9982ba9640 1 -- 172.21.15.84:0/3172955312 shutdown_connections 2023-10-16T10:04:34.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.501+0000 7f9982ba9640 1 --2- 172.21.15.84:0/3172955312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c06a4b0 0x7f997c0ca3b0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:34.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.501+0000 7f9982ba9640 1 -- 172.21.15.84:0/3172955312 >> 172.21.15.84:0/3172955312 conn(0x7f997c0ce9b0 msgr2=0x7f997c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:34.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.501+0000 7f9982ba9640 1 -- 172.21.15.84:0/3172955312 shutdown_connections 2023-10-16T10:04:34.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.501+0000 7f9982ba9640 1 -- 172.21.15.84:0/3172955312 wait complete. 2023-10-16T10:04:34.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.503+0000 7f9982ba9640 1 Processor -- start 2023-10-16T10:04:34.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.503+0000 7f9982ba9640 1 -- start start 2023-10-16T10:04:34.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.504+0000 7f9982ba9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997c06a4b0 0x7f997c1f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:34.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.504+0000 7f9982ba9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c0c9ca0 0x7f997c0c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:34.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.504+0000 7f9982ba9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c15ad70 0x7f997c158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:34.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.504+0000 7f9982ba9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c1f4410 con 0x7f997c0c9ca0 2023-10-16T10:04:34.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.504+0000 7f9982ba9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c159d40 con 0x7f997c15ad70 2023-10-16T10:04:34.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.504+0000 7f9982ba9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997c1f0fe0 con 0x7f997c06a4b0 2023-10-16T10:04:34.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.504+0000 7f997bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c0c9ca0 0x7f997c0c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:34.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.504+0000 7f997bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c0c9ca0 0x7f997c0c4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41180/0 (socket says 172.21.15.84:41180) 2023-10-16T10:04:34.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.504+0000 7f997bfff640 1 -- 172.21.15.84:0/3221488216 learned_addr learned my addr 172.21.15.84:0/3221488216 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:34.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.505+0000 7f9980945640 1 --2- 172.21.15.84:0/3221488216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997c06a4b0 0x7f997c1f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:34.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.505+0000 7f9981146640 1 --2- 172.21.15.84:0/3221488216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c15ad70 0x7f997c158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:34.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.505+0000 7f997bfff640 1 -- 172.21.15.84:0/3221488216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997c06a4b0 msgr2=0x7f997c1f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:34.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.505+0000 7f997bfff640 1 --2- 172.21.15.84:0/3221488216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997c06a4b0 0x7f997c1f3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:34.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.505+0000 7f997bfff640 1 -- 172.21.15.84:0/3221488216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c15ad70 msgr2=0x7f997c158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:34.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.505+0000 7f997bfff640 1 --2- 172.21.15.84:0/3221488216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c15ad70 0x7f997c158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:34.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.505+0000 7f997bfff640 1 -- 172.21.15.84:0/3221488216 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9964009400 con 0x7f997c0c9ca0 2023-10-16T10:04:34.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.505+0000 7f9981146640 1 --2- 172.21.15.84:0/3221488216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c15ad70 0x7f997c158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:34.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.505+0000 7f997bfff640 1 --2- 172.21.15.84:0/3221488216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c0c9ca0 0x7f997c0c4900 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f996402bee0 tx=0x7f99640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:34.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.506+0000 7f99797fa640 1 -- 172.21.15.84:0/3221488216 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996403ba70 con 0x7f997c0c9ca0 2023-10-16T10:04:34.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.506+0000 7f99797fa640 1 -- 172.21.15.84:0/3221488216 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996403bbd0 con 0x7f997c0c9ca0 2023-10-16T10:04:34.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.506+0000 7f99797fa640 1 -- 172.21.15.84:0/3221488216 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99640453f0 con 0x7f997c0c9ca0 2023-10-16T10:04:34.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.506+0000 7f9982ba9640 1 -- 172.21.15.84:0/3221488216 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997c0a93d0 con 0x7f997c0c9ca0 2023-10-16T10:04:34.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.506+0000 7f9982ba9640 1 -- 172.21.15.84:0/3221488216 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997c1f1530 con 0x7f997c0c9ca0 2023-10-16T10:04:34.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.507+0000 7f9982ba9640 1 -- 172.21.15.84:0/3221488216 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997c04e160 con 0x7f997c0c9ca0 2023-10-16T10:04:34.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.508+0000 7f99797fa640 1 -- 172.21.15.84:0/3221488216 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9964045550 con 0x7f997c0c9ca0 2023-10-16T10:04:34.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.509+0000 7f99797fa640 1 --2- 172.21.15.84:0/3221488216 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f995c06af90 0x7f995c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:34.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.509+0000 7f99797fa640 1 -- 172.21.15.84:0/3221488216 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99640abcc0 con 0x7f997c0c9ca0 2023-10-16T10:04:34.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.510+0000 7f9980945640 1 --2- 172.21.15.84:0/3221488216 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f995c06af90 0x7f995c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:34.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.510+0000 7f9980945640 1 --2- 172.21.15.84:0/3221488216 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f995c06af90 0x7f995c06d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f996c007ed0 tx=0x7f996c007e60 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:34.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.514+0000 7f99797fa640 1 -- 172.21.15.84:0/3221488216 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9964076c70 con 0x7f997c0c9ca0 2023-10-16T10:04:34.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:34.746+0000 7f9982ba9640 1 -- 172.21.15.84:0/3221488216 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f997c0c8f00 con 0x7f997c0c9ca0 2023-10-16T10:04:35.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.406+0000 7f99797fa640 1 -- 172.21.15.84:0/3221488216 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7f996407a290 con 0x7f997c0c9ca0 2023-10-16T10:04:35.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.410+0000 7f9982ba9640 1 -- 172.21.15.84:0/3221488216 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f995c06af90 msgr2=0x7f995c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:35.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.410+0000 7f9982ba9640 1 --2- 172.21.15.84:0/3221488216 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f995c06af90 0x7f995c06d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f996c007ed0 tx=0x7f996c007e60 comp rx=0 tx=0).stop 2023-10-16T10:04:35.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.410+0000 7f9982ba9640 1 -- 172.21.15.84:0/3221488216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c0c9ca0 msgr2=0x7f997c0c4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:35.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.410+0000 7f9982ba9640 1 --2- 172.21.15.84:0/3221488216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c0c9ca0 0x7f997c0c4900 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f996402bee0 tx=0x7f99640046c0 comp rx=0 tx=0).stop 2023-10-16T10:04:35.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.410+0000 7f9982ba9640 1 -- 172.21.15.84:0/3221488216 shutdown_connections 2023-10-16T10:04:35.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.410+0000 7f9982ba9640 1 --2- 172.21.15.84:0/3221488216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c15ad70 0x7f997c158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:35.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.410+0000 7f9982ba9640 1 --2- 172.21.15.84:0/3221488216 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f995c06af90 0x7f995c06d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:35.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.410+0000 7f9982ba9640 1 --2- 172.21.15.84:0/3221488216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c0c9ca0 0x7f997c0c4900 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:35.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.410+0000 7f9982ba9640 1 --2- 172.21.15.84:0/3221488216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997c06a4b0 0x7f997c1f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:35.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.410+0000 7f9982ba9640 1 -- 172.21.15.84:0/3221488216 >> 172.21.15.84:0/3221488216 conn(0x7f997c0ce9b0 msgr2=0x7f997c0afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:35.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.411+0000 7f9982ba9640 1 -- 172.21.15.84:0/3221488216 shutdown_connections 2023-10-16T10:04:35.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.411+0000 7f9982ba9640 1 -- 172.21.15.84:0/3221488216 wait complete. 2023-10-16T10:04:35.435 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T10:04:35.436 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T10:04:35.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.644+0000 7f72a687c640 1 Processor -- start 2023-10-16T10:04:35.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.644+0000 7f72a687c640 1 -- start start 2023-10-16T10:04:35.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.646+0000 7f72a687c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f72a015ad70 0x7f72a0069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:35.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.646+0000 7f72a687c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a00cb030 con 0x7f72a00c9ca0 2023-10-16T10:04:35.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.646+0000 7f72a687c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a00c91c0 con 0x7f72a00caac0 2023-10-16T10:04:35.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.646+0000 7f72a687c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a00c94a0 con 0x7f72a015ad70 2023-10-16T10:04:35.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.646+0000 7f7297fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f72a00c9ca0 0x7f72a00ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39730/0 (socket says 172.21.15.84:39730) 2023-10-16T10:04:35.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.646+0000 7f7297fff640 1 -- 172.21.15.84:0/4127335291 learned_addr learned my addr 172.21.15.84:0/4127335291 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:35.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.647+0000 7f729ffff640 1 --2- 172.21.15.84:0/4127335291 >> v2:172.21.15.157:3300/0 conn(0x7f72a015ad70 0x7f72a0069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:35.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.647+0000 7f729f7fe640 1 -- 172.21.15.84:0/4127335291 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369295563 0 0) 0x7f72a00cb030 con 0x7f72a00c9ca0 2023-10-16T10:04:35.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.647+0000 7f729f7fe640 1 -- 172.21.15.84:0/4127335291 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f728c003520 con 0x7f72a00c9ca0 2023-10-16T10:04:35.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.647+0000 7f729f7fe640 1 -- 172.21.15.84:0/4127335291 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 967018629 0 0) 0x7f728c003520 con 0x7f72a00c9ca0 2023-10-16T10:04:35.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.647+0000 7f729f7fe640 1 -- 172.21.15.84:0/4127335291 >> v2:172.21.15.157:3300/0 conn(0x7f72a015ad70 msgr2=0x7f72a0069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:35.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.647+0000 7f729f7fe640 1 --2- 172.21.15.84:0/4127335291 >> v2:172.21.15.157:3300/0 conn(0x7f72a015ad70 0x7f72a0069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:35.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.647+0000 7f729f7fe640 1 -- 172.21.15.84:0/4127335291 >> v1:172.21.15.157:6789/0 conn(0x7f72a00caac0 legacy=0x7f72a00ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:35.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.647+0000 7f729f7fe640 1 -- 172.21.15.84:0/4127335291 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a00c9780 con 0x7f72a00c9ca0 2023-10-16T10:04:35.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.647+0000 7f729f7fe640 1 -- 172.21.15.84:0/4127335291 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f7288001670 con 0x7f72a00c9ca0 2023-10-16T10:04:35.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.647+0000 7f729f7fe640 1 -- 172.21.15.84:0/4127335291 >> v1:172.21.15.84:6789/0 conn(0x7f72a00c9ca0 legacy=0x7f72a00ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:35.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.647+0000 7f729f7fe640 1 --2- 172.21.15.84:0/4127335291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f728c0039c0 0x7f728c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:35.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.647+0000 7f729f7fe640 1 --2- 172.21.15.84:0/4127335291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f728c0047c0 0x7f728c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:35.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.647+0000 7f729f7fe640 1 --2- 172.21.15.84:0/4127335291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f728c005180 0x7f728c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:35.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.647+0000 7f729f7fe640 1 -- 172.21.15.84:0/4127335291 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a00c94a0 con 0x7f728c0047c0 2023-10-16T10:04:35.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.647+0000 7f729f7fe640 1 -- 172.21.15.84:0/4127335291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c00c080 con 0x7f728c005180 2023-10-16T10:04:35.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.647+0000 7f729f7fe640 1 -- 172.21.15.84:0/4127335291 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728c00c240 con 0x7f728c0039c0 2023-10-16T10:04:35.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.647+0000 7f7297fff640 1 --2- 172.21.15.84:0/4127335291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f728c0047c0 0x7f728c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:35.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.648+0000 7f7297fff640 1 -- 172.21.15.84:0/4127335291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f728c0039c0 msgr2=0x7f728c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:35.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.648+0000 7f7297fff640 1 --2- 172.21.15.84:0/4127335291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f728c0039c0 0x7f728c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:35.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.648+0000 7f7297fff640 1 -- 172.21.15.84:0/4127335291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f728c005180 msgr2=0x7f728c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:35.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.648+0000 7f7297fff640 1 --2- 172.21.15.84:0/4127335291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f728c005180 0x7f728c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:35.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.648+0000 7f7297fff640 1 -- 172.21.15.84:0/4127335291 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c003520 con 0x7f728c0047c0 2023-10-16T10:04:35.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.648+0000 7f72a4e19640 1 -- 172.21.15.84:0/4127335291 reap_dead start 2023-10-16T10:04:35.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.649+0000 7f7297fff640 1 --2- 172.21.15.84:0/4127335291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f728c0047c0 0x7f728c004c40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f728800c0e0 tx=0x7f728c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f6ed274cbc7a78 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:35.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.649+0000 7f729f7fe640 1 -- 172.21.15.84:0/4127335291 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7288005ad0 con 0x7f728c0047c0 2023-10-16T10:04:35.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.649+0000 7f729f7fe640 1 -- 172.21.15.84:0/4127335291 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7288005c30 con 0x7f728c0047c0 2023-10-16T10:04:35.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.649+0000 7f729f7fe640 1 -- 172.21.15.84:0/4127335291 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728802c7d0 con 0x7f728c0047c0 2023-10-16T10:04:35.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.650+0000 7f72a687c640 1 -- 172.21.15.84:0/4127335291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f728c0047c0 msgr2=0x7f728c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:35.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.650+0000 7f72a687c640 1 --2- 172.21.15.84:0/4127335291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f728c0047c0 0x7f728c004c40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f728800c0e0 tx=0x7f728c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:35.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.650+0000 7f72a687c640 1 -- 172.21.15.84:0/4127335291 shutdown_connections 2023-10-16T10:04:35.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.650+0000 7f72a687c640 1 --2- 172.21.15.84:0/4127335291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f728c0047c0 0x7f728c004c40 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:35.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.650+0000 7f72a687c640 1 -- 172.21.15.84:0/4127335291 >> 172.21.15.84:0/4127335291 conn(0x7f72a00ce9b0 msgr2=0x7f72a00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:35.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.650+0000 7f72a687c640 1 -- 172.21.15.84:0/4127335291 shutdown_connections 2023-10-16T10:04:35.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.650+0000 7f72a687c640 1 -- 172.21.15.84:0/4127335291 wait complete. 2023-10-16T10:04:35.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.652+0000 7f72a687c640 1 Processor -- start 2023-10-16T10:04:35.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.652+0000 7f72a687c640 1 -- start start 2023-10-16T10:04:35.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.653+0000 7f72a687c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f728c0047c0 0x7f72a00ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:35.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.653+0000 7f72a687c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a00acd60 0x7f72a015c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:35.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.653+0000 7f72a687c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72a015a190 0x7f72a015cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:35.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.653+0000 7f72a687c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a00ce120 con 0x7f728c0047c0 2023-10-16T10:04:35.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.653+0000 7f72a687c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a00addc0 con 0x7f72a00acd60 2023-10-16T10:04:35.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.653+0000 7f72a687c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a00ad240 con 0x7f72a015a190 2023-10-16T10:04:35.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.653+0000 7f729ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f728c0047c0 0x7f72a00ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:35.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.653+0000 7f729ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f728c0047c0 0x7f72a00ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43768/0 (socket says 172.21.15.84:43768) 2023-10-16T10:04:35.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.653+0000 7f729ffff640 1 -- 172.21.15.84:0/2257709005 learned_addr learned my addr 172.21.15.84:0/2257709005 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:35.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.654+0000 7f729ffff640 1 -- 172.21.15.84:0/2257709005 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72a015a190 msgr2=0x7f72a015cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:35.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.654+0000 7f729ffff640 1 --2- 172.21.15.84:0/2257709005 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72a015a190 0x7f72a015cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:35.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.654+0000 7f729ffff640 1 -- 172.21.15.84:0/2257709005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a00acd60 msgr2=0x7f72a015c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:35.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.654+0000 7f729ffff640 1 --2- 172.21.15.84:0/2257709005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a00acd60 0x7f72a015c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:35.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.654+0000 7f729ffff640 1 -- 172.21.15.84:0/2257709005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a00ad520 con 0x7f728c0047c0 2023-10-16T10:04:35.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.654+0000 7f729ffff640 1 --2- 172.21.15.84:0/2257709005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f728c0047c0 0x7f72a00ccf60 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f729000e7a0 tx=0x7f729000eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:35.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.654+0000 7f729d7fa640 1 -- 172.21.15.84:0/2257709005 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729000c6d0 con 0x7f728c0047c0 2023-10-16T10:04:35.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.654+0000 7f729d7fa640 1 -- 172.21.15.84:0/2257709005 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729000c850 con 0x7f728c0047c0 2023-10-16T10:04:35.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.654+0000 7f729d7fa640 1 -- 172.21.15.84:0/2257709005 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729000cc00 con 0x7f728c0047c0 2023-10-16T10:04:35.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.654+0000 7f72a687c640 1 -- 172.21.15.84:0/2257709005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a00ad800 con 0x7f728c0047c0 2023-10-16T10:04:35.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.654+0000 7f72a687c640 1 -- 172.21.15.84:0/2257709005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a00cd4a0 con 0x7f728c0047c0 2023-10-16T10:04:35.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.656+0000 7f72a687c640 1 -- 172.21.15.84:0/2257709005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a004e160 con 0x7f728c0047c0 2023-10-16T10:04:35.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.656+0000 7f729d7fa640 1 -- 172.21.15.84:0/2257709005 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7290013470 con 0x7f728c0047c0 2023-10-16T10:04:35.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.657+0000 7f729d7fa640 1 --2- 172.21.15.84:0/2257709005 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f728006afc0 0x7f728006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:35.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.657+0000 7f729d7fa640 1 -- 172.21.15.84:0/2257709005 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7290089740 con 0x7f728c0047c0 2023-10-16T10:04:35.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.658+0000 7f7297fff640 1 --2- 172.21.15.84:0/2257709005 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f728006afc0 0x7f728006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:35.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.658+0000 7f7297fff640 1 --2- 172.21.15.84:0/2257709005 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f728006afc0 0x7f728006d480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7288034000 tx=0x7f7288007000 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:35.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.662+0000 7f729d7fa640 1 -- 172.21.15.84:0/2257709005 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f72900546e0 con 0x7f728c0047c0 2023-10-16T10:04:35.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:35.896+0000 7f72a687c640 1 -- 172.21.15.84:0/2257709005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f72a00ae170 con 0x7f728c0047c0 2023-10-16T10:04:36.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.419+0000 7f729d7fa640 1 -- 172.21.15.84:0/2257709005 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7290057d00 con 0x7f728c0047c0 2023-10-16T10:04:36.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.423+0000 7f72a687c640 1 -- 172.21.15.84:0/2257709005 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f728006afc0 msgr2=0x7f728006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:36.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.423+0000 7f72a687c640 1 --2- 172.21.15.84:0/2257709005 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f728006afc0 0x7f728006d480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7288034000 tx=0x7f7288007000 comp rx=0 tx=0).stop 2023-10-16T10:04:36.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.423+0000 7f72a687c640 1 -- 172.21.15.84:0/2257709005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f728c0047c0 msgr2=0x7f72a00ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:36.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.423+0000 7f72a687c640 1 --2- 172.21.15.84:0/2257709005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f728c0047c0 0x7f72a00ccf60 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f729000e7a0 tx=0x7f729000eba0 comp rx=0 tx=0).stop 2023-10-16T10:04:36.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.423+0000 7f72a687c640 1 -- 172.21.15.84:0/2257709005 shutdown_connections 2023-10-16T10:04:36.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.423+0000 7f72a687c640 1 --2- 172.21.15.84:0/2257709005 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72a015a190 0x7f72a015cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:36.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.423+0000 7f72a687c640 1 --2- 172.21.15.84:0/2257709005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a00acd60 0x7f72a015c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:36.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.423+0000 7f72a687c640 1 --2- 172.21.15.84:0/2257709005 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f728006afc0 0x7f728006d480 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:36.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.423+0000 7f72a687c640 1 --2- 172.21.15.84:0/2257709005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f728c0047c0 0x7f72a00ccf60 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:36.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.423+0000 7f72a687c640 1 -- 172.21.15.84:0/2257709005 >> 172.21.15.84:0/2257709005 conn(0x7f72a00ce9b0 msgr2=0x7f72a006b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:36.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.423+0000 7f72a687c640 1 -- 172.21.15.84:0/2257709005 shutdown_connections 2023-10-16T10:04:36.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.423+0000 7f72a687c640 1 -- 172.21.15.84:0/2257709005 wait complete. 2023-10-16T10:04:36.447 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T10:04:36.447 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:04:36.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.660+0000 7fe6460a2640 1 Processor -- start 2023-10-16T10:04:36.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.660+0000 7fe6460a2640 1 -- start start 2023-10-16T10:04:36.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.661+0000 7fe6460a2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe6400cac50 0x7fe6400cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:36.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.661+0000 7fe6460a2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe64006b410 0x7fe64006b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:36.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.661+0000 7fe6460a2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe64015e080 con 0x7fe6400adb30 2023-10-16T10:04:36.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.661+0000 7fe6460a2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe64015e5b0 con 0x7fe64006b410 2023-10-16T10:04:36.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.661+0000 7fe6460a2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe64006bdc0 con 0x7fe6400cac50 2023-10-16T10:04:36.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.662+0000 7fe63ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe64006b410 0x7fe64006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:36.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.662+0000 7fe63ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe64006b410 0x7fe64006b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43776/0 (socket says 172.21.15.84:43776) 2023-10-16T10:04:36.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.662+0000 7fe63ffff640 1 -- 172.21.15.84:0/4255060564 learned_addr learned my addr 172.21.15.84:0/4255060564 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:36.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.662+0000 7fe63effd640 1 --2- 172.21.15.84:0/4255060564 >> v2:172.21.15.157:3300/0 conn(0x7fe6400cac50 0x7fe6400cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:36.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.662+0000 7fe63ffff640 1 -- 172.21.15.84:0/4255060564 >> v2:172.21.15.157:3300/0 conn(0x7fe6400cac50 msgr2=0x7fe6400cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:36.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.662+0000 7fe63ffff640 1 --2- 172.21.15.84:0/4255060564 >> v2:172.21.15.157:3300/0 conn(0x7fe6400cac50 0x7fe6400cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:36.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.662+0000 7fe63ffff640 1 -- 172.21.15.84:0/4255060564 >> v1:172.21.15.157:6789/0 conn(0x7fe6400adb30 legacy=0x7fe6400c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:36.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.662+0000 7fe63ffff640 1 -- 172.21.15.84:0/4255060564 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6400c8ef0 con 0x7fe64006b410 2023-10-16T10:04:36.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.662+0000 7fe63effd640 1 --2- 172.21.15.84:0/4255060564 >> v2:172.21.15.157:3300/0 conn(0x7fe6400cac50 0x7fe6400cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:36.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.663+0000 7fe63ffff640 1 --2- 172.21.15.84:0/4255060564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe64006b410 0x7fe64006b880 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fe640072560 tx=0x7fe63802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea839cfd23649ad2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:36.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.664+0000 7fe63e7fc640 1 -- 172.21.15.84:0/4255060564 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638031870 con 0x7fe64006b410 2023-10-16T10:04:36.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.664+0000 7fe63e7fc640 1 -- 172.21.15.84:0/4255060564 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6380319d0 con 0x7fe64006b410 2023-10-16T10:04:36.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.664+0000 7fe63e7fc640 1 -- 172.21.15.84:0/4255060564 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638031d60 con 0x7fe64006b410 2023-10-16T10:04:36.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.665+0000 7fe6460a2640 1 -- 172.21.15.84:0/4255060564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe64006b410 msgr2=0x7fe64006b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:36.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.665+0000 7fe6460a2640 1 --2- 172.21.15.84:0/4255060564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe64006b410 0x7fe64006b880 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fe640072560 tx=0x7fe63802b910 comp rx=0 tx=0).stop 2023-10-16T10:04:36.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.665+0000 7fe6460a2640 1 -- 172.21.15.84:0/4255060564 shutdown_connections 2023-10-16T10:04:36.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.665+0000 7fe6460a2640 1 --2- 172.21.15.84:0/4255060564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe64006b410 0x7fe64006b880 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:36.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.665+0000 7fe6460a2640 1 --2- 172.21.15.84:0/4255060564 >> v2:172.21.15.157:3300/0 conn(0x7fe6400cac50 0x7fe6400cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:36.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.665+0000 7fe6460a2640 1 -- 172.21.15.84:0/4255060564 >> 172.21.15.84:0/4255060564 conn(0x7fe6400ce860 msgr2=0x7fe6400cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:36.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.665+0000 7fe6460a2640 1 -- 172.21.15.84:0/4255060564 shutdown_connections 2023-10-16T10:04:36.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.665+0000 7fe6460a2640 1 -- 172.21.15.84:0/4255060564 wait complete. 2023-10-16T10:04:36.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.667+0000 7fe6460a2640 1 Processor -- start 2023-10-16T10:04:36.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.667+0000 7fe6460a2640 1 -- start start 2023-10-16T10:04:36.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.668+0000 7fe6460a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe64006b410 0x7fe64015aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:36.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.668+0000 7fe6460a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6400adb30 0x7fe6400b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:36.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.668+0000 7fe6460a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6400cac50 0x7fe6400aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:36.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.668+0000 7fe6460a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6401f37f0 con 0x7fe64006b410 2023-10-16T10:04:36.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.668+0000 7fe6460a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6401f3510 con 0x7fe6400cac50 2023-10-16T10:04:36.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.668+0000 7fe6460a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6400b1cc0 con 0x7fe6400adb30 2023-10-16T10:04:36.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.668+0000 7fe63f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe64006b410 0x7fe64015aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:36.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.668+0000 7fe63f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe64006b410 0x7fe64015aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43780/0 (socket says 172.21.15.84:43780) 2023-10-16T10:04:36.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.668+0000 7fe63f7fe640 1 -- 172.21.15.84:0/330248281 learned_addr learned my addr 172.21.15.84:0/330248281 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:36.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.668+0000 7fe63f7fe640 1 -- 172.21.15.84:0/330248281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6400adb30 msgr2=0x7fe6400b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:36.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.668+0000 7fe63effd640 1 --2- 172.21.15.84:0/330248281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6400adb30 0x7fe6400b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:36.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.668+0000 7fe63ffff640 1 --2- 172.21.15.84:0/330248281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6400cac50 0x7fe6400aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:36.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.668+0000 7fe63f7fe640 1 --2- 172.21.15.84:0/330248281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6400adb30 0x7fe6400b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:36.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.668+0000 7fe63f7fe640 1 -- 172.21.15.84:0/330248281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6400cac50 msgr2=0x7fe6400aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:36.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.668+0000 7fe63f7fe640 1 --2- 172.21.15.84:0/330248281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6400cac50 0x7fe6400aefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:36.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.668+0000 7fe63f7fe640 1 -- 172.21.15.84:0/330248281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6380094d0 con 0x7fe64006b410 2023-10-16T10:04:36.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.669+0000 7fe63f7fe640 1 --2- 172.21.15.84:0/330248281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe64006b410 0x7fe64015aae0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe63000c7c0 tx=0x7fe63000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:36.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.669+0000 7fe627fff640 1 -- 172.21.15.84:0/330248281 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63000aa50 con 0x7fe64006b410 2023-10-16T10:04:36.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.669+0000 7fe627fff640 1 -- 172.21.15.84:0/330248281 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63000abb0 con 0x7fe64006b410 2023-10-16T10:04:36.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.669+0000 7fe627fff640 1 -- 172.21.15.84:0/330248281 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe630012420 con 0x7fe64006b410 2023-10-16T10:04:36.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.669+0000 7fe6460a2640 1 -- 172.21.15.84:0/330248281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6400b1fa0 con 0x7fe64006b410 2023-10-16T10:04:36.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.669+0000 7fe6460a2640 1 -- 172.21.15.84:0/330248281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6400af4f0 con 0x7fe64006b410 2023-10-16T10:04:36.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.670+0000 7fe6460a2640 1 -- 172.21.15.84:0/330248281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6400b1b20 con 0x7fe64006b410 2023-10-16T10:04:36.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.672+0000 7fe627fff640 1 -- 172.21.15.84:0/330248281 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe630012580 con 0x7fe64006b410 2023-10-16T10:04:36.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.672+0000 7fe627fff640 1 --2- 172.21.15.84:0/330248281 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe62c06af90 0x7fe62c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:36.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.673+0000 7fe627fff640 1 -- 172.21.15.84:0/330248281 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe630089710 con 0x7fe64006b410 2023-10-16T10:04:36.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.673+0000 7fe63effd640 1 --2- 172.21.15.84:0/330248281 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe62c06af90 0x7fe62c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:36.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.673+0000 7fe63effd640 1 --2- 172.21.15.84:0/330248281 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe62c06af90 0x7fe62c06d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe6400c9860 tx=0x7fe634008040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:36.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.677+0000 7fe627fff640 1 -- 172.21.15.84:0/330248281 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe6400b1b20 con 0x7fe64006b410 2023-10-16T10:04:36.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.902+0000 7fe6460a2640 1 -- 172.21.15.84:0/330248281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6400b1b20 con 0x7fe64006b410 2023-10-16T10:04:36.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.904+0000 7fe627fff640 1 -- 172.21.15.84:0/330248281 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fe6400b1b20 con 0x7fe64006b410 2023-10-16T10:04:36.905 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:36.906 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5066,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1223610299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1223610299},{"type":"v1","addr":"172.21.15.157:6833","nonce":1223610299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3636443426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3636443426},{"type":"v1","addr":"172.21.15.157:6835","nonce":3636443426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6537,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/3575464874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3575464874},{"type":"v1","addr":"172.21.15.84:6836","nonce":3575464874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:04:32.849497+0000","modified":"2023-10-16T10:04:35.898561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.84:6837/2669751864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2669751864},{"type":"v1","addr":"172.21.15.84:6837","nonce":2669751864}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-16T10:04:36.906 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 35 2023-10-16T10:04:36.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.906+0000 7fe6460a2640 1 -- 172.21.15.84:0/330248281 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe62c06af90 msgr2=0x7fe62c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:36.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.906+0000 7fe6460a2640 1 --2- 172.21.15.84:0/330248281 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe62c06af90 0x7fe62c06d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe6400c9860 tx=0x7fe634008040 comp rx=0 tx=0).stop 2023-10-16T10:04:36.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.906+0000 7fe6460a2640 1 -- 172.21.15.84:0/330248281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe64006b410 msgr2=0x7fe64015aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:36.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.906+0000 7fe6460a2640 1 --2- 172.21.15.84:0/330248281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe64006b410 0x7fe64015aae0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe63000c7c0 tx=0x7fe63000cc80 comp rx=0 tx=0).stop 2023-10-16T10:04:36.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.906+0000 7fe6460a2640 1 -- 172.21.15.84:0/330248281 shutdown_connections 2023-10-16T10:04:36.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.906+0000 7fe6460a2640 1 --2- 172.21.15.84:0/330248281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6400cac50 0x7fe6400aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:36.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.906+0000 7fe6460a2640 1 --2- 172.21.15.84:0/330248281 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6400adb30 0x7fe6400b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:36.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.906+0000 7fe6460a2640 1 --2- 172.21.15.84:0/330248281 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe62c06af90 0x7fe62c06d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:36.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.906+0000 7fe6460a2640 1 --2- 172.21.15.84:0/330248281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe64006b410 0x7fe64015aae0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:36.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.906+0000 7fe6460a2640 1 -- 172.21.15.84:0/330248281 >> 172.21.15.84:0/330248281 conn(0x7fe6400ce860 msgr2=0x7fe640160b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:36.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.906+0000 7fe6460a2640 1 -- 172.21.15.84:0/330248281 shutdown_connections 2023-10-16T10:04:36.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:36.906+0000 7fe6460a2640 1 -- 172.21.15.84:0/330248281 wait complete. 2023-10-16T10:04:36.924 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:04:37.088 INFO:tasks.ceph.mds.c.smithi084.stderr:2023-10-16T10:04:37.086+0000 7f47ff7bc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:04:37.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.133+0000 7f925dfa9640 1 Processor -- start 2023-10-16T10:04:37.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.134+0000 7f925dfa9640 1 -- start start 2023-10-16T10:04:37.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.135+0000 7f925dfa9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f92580cd0c0 0x7f92581615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:37.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.135+0000 7f925dfa9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f925815db00 0x7f92580b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:37.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.135+0000 7f925dfa9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925815e3e0 con 0x7f92580b0410 2023-10-16T10:04:37.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.135+0000 7f925dfa9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92580b0de0 con 0x7f92580cd0c0 2023-10-16T10:04:37.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.135+0000 7f925dfa9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92580cd4a0 con 0x7f925815db00 2023-10-16T10:04:37.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.135+0000 7f92577fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f92580cd0c0 0x7f92581615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:37.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.136+0000 7f92577fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f92580cd0c0 0x7f92581615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43788/0 (socket says 172.21.15.84:43788) 2023-10-16T10:04:37.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.136+0000 7f92577fe640 1 -- 172.21.15.84:0/4222485425 learned_addr learned my addr 172.21.15.84:0/4222485425 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:37.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.136+0000 7f9257fff640 1 --2- 172.21.15.84:0/4222485425 >> v2:172.21.15.157:3300/0 conn(0x7f925815db00 0x7f92580b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:37.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.136+0000 7f92577fe640 1 -- 172.21.15.84:0/4222485425 >> v2:172.21.15.157:3300/0 conn(0x7f925815db00 msgr2=0x7f92580b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:37.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.136+0000 7f92577fe640 1 --2- 172.21.15.84:0/4222485425 >> v2:172.21.15.157:3300/0 conn(0x7f925815db00 0x7f92580b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.136+0000 7f92577fe640 1 -- 172.21.15.84:0/4222485425 >> v1:172.21.15.157:6789/0 conn(0x7f92580b0410 legacy=0x7f925815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:37.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.136+0000 7f92577fe640 1 -- 172.21.15.84:0/4222485425 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925815e100 con 0x7f92580cd0c0 2023-10-16T10:04:37.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.136+0000 7f9257fff640 1 --2- 172.21.15.84:0/4222485425 >> v2:172.21.15.157:3300/0 conn(0x7f925815db00 0x7f92580b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:37.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.138+0000 7f92577fe640 1 --2- 172.21.15.84:0/4222485425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92580cd0c0 0x7f92581615f0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f925815afc0 tx=0x7f924802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73866abd2d7b5060 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:37.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.138+0000 7f92567fc640 1 -- 172.21.15.84:0/4222485425 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92480317e0 con 0x7f92580cd0c0 2023-10-16T10:04:37.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.138+0000 7f92567fc640 1 -- 172.21.15.84:0/4222485425 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9248031940 con 0x7f92580cd0c0 2023-10-16T10:04:37.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.138+0000 7f92567fc640 1 -- 172.21.15.84:0/4222485425 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9248031cd0 con 0x7f92580cd0c0 2023-10-16T10:04:37.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.139+0000 7f925dfa9640 1 -- 172.21.15.84:0/4222485425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92580cd0c0 msgr2=0x7f92581615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:37.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.139+0000 7f925dfa9640 1 --2- 172.21.15.84:0/4222485425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92580cd0c0 0x7f92581615f0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f925815afc0 tx=0x7f924802b880 comp rx=0 tx=0).stop 2023-10-16T10:04:37.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.139+0000 7f925dfa9640 1 -- 172.21.15.84:0/4222485425 shutdown_connections 2023-10-16T10:04:37.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.139+0000 7f925dfa9640 1 --2- 172.21.15.84:0/4222485425 >> v2:172.21.15.157:3300/0 conn(0x7f925815db00 0x7f92580b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.139+0000 7f925dfa9640 1 --2- 172.21.15.84:0/4222485425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92580cd0c0 0x7f92581615f0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.139+0000 7f925dfa9640 1 -- 172.21.15.84:0/4222485425 >> 172.21.15.84:0/4222485425 conn(0x7f92580cb410 msgr2=0x7f92580cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:37.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.139+0000 7f925dfa9640 1 -- 172.21.15.84:0/4222485425 shutdown_connections 2023-10-16T10:04:37.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.139+0000 7f925dfa9640 1 -- 172.21.15.84:0/4222485425 wait complete. 2023-10-16T10:04:37.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.141+0000 7f925dfa9640 1 Processor -- start 2023-10-16T10:04:37.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.141+0000 7f925dfa9640 1 -- start start 2023-10-16T10:04:37.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.142+0000 7f925dfa9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92580b0410 0x7f9258153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:37.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.142+0000 7f925dfa9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92580cd0c0 0x7f925815ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:37.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.142+0000 7f925dfa9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925815db00 0x7f925815b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:37.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.142+0000 7f925dfa9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9258153540 con 0x7f92580cd0c0 2023-10-16T10:04:37.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.142+0000 7f925dfa9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925815fd40 con 0x7f925815db00 2023-10-16T10:04:37.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.142+0000 7f925dfa9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9258155a30 con 0x7f92580b0410 2023-10-16T10:04:37.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.142+0000 7f9256ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92580cd0c0 0x7f925815ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:37.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.142+0000 7f9256ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92580cd0c0 0x7f925815ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43794/0 (socket says 172.21.15.84:43794) 2023-10-16T10:04:37.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.142+0000 7f9256ffd640 1 -- 172.21.15.84:0/4291822356 learned_addr learned my addr 172.21.15.84:0/4291822356 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:37.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.143+0000 7f9257fff640 1 --2- 172.21.15.84:0/4291822356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925815db00 0x7f925815b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:37.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.143+0000 7f92577fe640 1 --2- 172.21.15.84:0/4291822356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92580b0410 0x7f9258153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:37.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.143+0000 7f9256ffd640 1 -- 172.21.15.84:0/4291822356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92580b0410 msgr2=0x7f9258153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:37.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.143+0000 7f9256ffd640 1 --2- 172.21.15.84:0/4291822356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92580b0410 0x7f9258153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.143+0000 7f9256ffd640 1 -- 172.21.15.84:0/4291822356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925815db00 msgr2=0x7f925815b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:37.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.143+0000 7f9256ffd640 1 --2- 172.21.15.84:0/4291822356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925815db00 0x7f925815b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.143+0000 7f9256ffd640 1 -- 172.21.15.84:0/4291822356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92480094d0 con 0x7f92580cd0c0 2023-10-16T10:04:37.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.143+0000 7f9257fff640 1 --2- 172.21.15.84:0/4291822356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925815db00 0x7f925815b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:37.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.143+0000 7f92577fe640 1 --2- 172.21.15.84:0/4291822356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92580b0410 0x7f9258153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:37.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.143+0000 7f9256ffd640 1 --2- 172.21.15.84:0/4291822356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92580cd0c0 0x7f925815ef80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f924c00b3f0 tx=0x7f924c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:37.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.143+0000 7f923ffff640 1 -- 172.21.15.84:0/4291822356 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c011830 con 0x7f92580cd0c0 2023-10-16T10:04:37.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.143+0000 7f923ffff640 1 -- 172.21.15.84:0/4291822356 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924c011990 con 0x7f92580cd0c0 2023-10-16T10:04:37.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.143+0000 7f923ffff640 1 -- 172.21.15.84:0/4291822356 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c011d40 con 0x7f92580cd0c0 2023-10-16T10:04:37.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.143+0000 7f925dfa9640 1 -- 172.21.15.84:0/4291822356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9258154990 con 0x7f92580cd0c0 2023-10-16T10:04:37.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.144+0000 7f925dfa9640 1 -- 172.21.15.84:0/4291822356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9258154f50 con 0x7f92580cd0c0 2023-10-16T10:04:37.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.145+0000 7f925dfa9640 1 -- 172.21.15.84:0/4291822356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f925804e160 con 0x7f92580cd0c0 2023-10-16T10:04:37.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.146+0000 7f923ffff640 1 -- 172.21.15.84:0/4291822356 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f924c01a400 con 0x7f92580cd0c0 2023-10-16T10:04:37.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.146+0000 7f923ffff640 1 --2- 172.21.15.84:0/4291822356 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f924406af90 0x7f924406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:37.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.147+0000 7f923ffff640 1 -- 172.21.15.84:0/4291822356 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f924c089650 con 0x7f92580cd0c0 2023-10-16T10:04:37.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.147+0000 7f92577fe640 1 --2- 172.21.15.84:0/4291822356 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f924406af90 0x7f924406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:37.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.147+0000 7f92577fe640 1 --2- 172.21.15.84:0/4291822356 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f924406af90 0x7f924406d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9248036000 tx=0x7f924803d040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.157+0000 7f923ffff640 1 -- 172.21.15.84:0/4291822356 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f924c0545e0 con 0x7f92580cd0c0 2023-10-16T10:04:37.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.347+0000 7f925dfa9640 1 -- 172.21.15.84:0/4291822356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92580b4910 con 0x7f92580cd0c0 2023-10-16T10:04:37.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.349+0000 7f923ffff640 1 -- 172.21.15.84:0/4291822356 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f924c057c00 con 0x7f92580cd0c0 2023-10-16T10:04:37.351 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:37.351 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":82,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:04:33.896255+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:04:30.728624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-16T10:04:31.866029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:04:37.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.351+0000 7f925dfa9640 1 -- 172.21.15.84:0/4291822356 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f924406af90 msgr2=0x7f924406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:37.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.351+0000 7f925dfa9640 1 --2- 172.21.15.84:0/4291822356 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f924406af90 0x7f924406d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9248036000 tx=0x7f924803d040 comp rx=0 tx=0).stop 2023-10-16T10:04:37.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.351+0000 7f925dfa9640 1 -- 172.21.15.84:0/4291822356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92580cd0c0 msgr2=0x7f925815ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:37.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.351+0000 7f925dfa9640 1 --2- 172.21.15.84:0/4291822356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92580cd0c0 0x7f925815ef80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f924c00b3f0 tx=0x7f924c00b8b0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.351+0000 7f925dfa9640 1 -- 172.21.15.84:0/4291822356 shutdown_connections 2023-10-16T10:04:37.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.351+0000 7f925dfa9640 1 --2- 172.21.15.84:0/4291822356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925815db00 0x7f925815b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.351+0000 7f925dfa9640 1 --2- 172.21.15.84:0/4291822356 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f924406af90 0x7f924406d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.351+0000 7f925dfa9640 1 --2- 172.21.15.84:0/4291822356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92580cd0c0 0x7f925815ef80 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.351+0000 7f925dfa9640 1 --2- 172.21.15.84:0/4291822356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92580b0410 0x7f9258153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.351+0000 7f925dfa9640 1 -- 172.21.15.84:0/4291822356 >> 172.21.15.84:0/4291822356 conn(0x7f92580cb410 msgr2=0x7f925815c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:37.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.352+0000 7f925dfa9640 1 -- 172.21.15.84:0/4291822356 shutdown_connections 2023-10-16T10:04:37.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.352+0000 7f925dfa9640 1 -- 172.21.15.84:0/4291822356 wait complete. 2023-10-16T10:04:37.371 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T10:04:37.372 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:04:37.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.579+0000 7f548311d640 1 Processor -- start 2023-10-16T10:04:37.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.579+0000 7f548311d640 1 -- start start 2023-10-16T10:04:37.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.581+0000 7f548311d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f547c15aaa0 0x7f547c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:37.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.581+0000 7f548311d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f547c0ce390 0x7f547c070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:37.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.581+0000 7f548311d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f547c15ed60 con 0x7f547c15ae80 2023-10-16T10:04:37.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.581+0000 7f548311d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f547c06b810 con 0x7f547c0ce390 2023-10-16T10:04:37.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.581+0000 7f548311d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547c0707e0 con 0x7f547c15aaa0 2023-10-16T10:04:37.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.581+0000 7f547bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f547c15ae80 0x7f547c0cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39740/0 (socket says 172.21.15.84:39740) 2023-10-16T10:04:37.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.581+0000 7f547bfff640 1 -- 172.21.15.84:0/2626205447 learned_addr learned my addr 172.21.15.84:0/2626205447 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:37.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.582+0000 7f5480eb9640 1 --2- 172.21.15.84:0/2626205447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f547c15aaa0 0x7f547c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:37.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.582+0000 7f54816ba640 1 --2- 172.21.15.84:0/2626205447 >> v2:172.21.15.157:3300/0 conn(0x7f547c0ce390 0x7f547c070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:37.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.582+0000 7f547b7fe640 1 -- 172.21.15.84:0/2626205447 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627623838 0 0) 0x7f547c15ed60 con 0x7f547c15ae80 2023-10-16T10:04:37.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.582+0000 7f547b7fe640 1 -- 172.21.15.84:0/2626205447 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f546c003520 con 0x7f547c15ae80 2023-10-16T10:04:37.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.582+0000 7f547b7fe640 1 -- 172.21.15.84:0/2626205447 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3859091023 0 0) 0x7f546c003520 con 0x7f547c15ae80 2023-10-16T10:04:37.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.582+0000 7f547b7fe640 1 -- 172.21.15.84:0/2626205447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f547c15aaa0 msgr2=0x7f547c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:37.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.582+0000 7f547b7fe640 1 --2- 172.21.15.84:0/2626205447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f547c15aaa0 0x7f547c15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.582+0000 7f547b7fe640 1 -- 172.21.15.84:0/2626205447 >> v2:172.21.15.157:3300/0 conn(0x7f547c0ce390 msgr2=0x7f547c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:37.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.582+0000 7f547b7fe640 1 --2- 172.21.15.84:0/2626205447 >> v2:172.21.15.157:3300/0 conn(0x7f547c0ce390 0x7f547c070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.582+0000 7f547b7fe640 1 -- 172.21.15.84:0/2626205447 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c15ef90 con 0x7f547c15ae80 2023-10-16T10:04:37.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.582+0000 7f547b7fe640 1 -- 172.21.15.84:0/2626205447 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f5468001670 con 0x7f547c15ae80 2023-10-16T10:04:37.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.582+0000 7f547b7fe640 1 -- 172.21.15.84:0/2626205447 >> v1:172.21.15.84:6789/0 conn(0x7f547c15ae80 legacy=0x7f547c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:37.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.582+0000 7f547b7fe640 1 --2- 172.21.15.84:0/2626205447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f547c15aaa0 0x7f546c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:37.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.582+0000 7f547b7fe640 1 --2- 172.21.15.84:0/2626205447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f546c003ef0 0x7f546c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:37.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.582+0000 7f547b7fe640 1 --2- 172.21.15.84:0/2626205447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f546c004d00 0x7f546c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:37.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.583+0000 7f547b7fe640 1 -- 172.21.15.84:0/2626205447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c06b810 con 0x7f546c004d00 2023-10-16T10:04:37.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.583+0000 7f547b7fe640 1 -- 172.21.15.84:0/2626205447 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c0707e0 con 0x7f546c003ef0 2023-10-16T10:04:37.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.583+0000 7f547b7fe640 1 -- 172.21.15.84:0/2626205447 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546c00b980 con 0x7f547c15aaa0 2023-10-16T10:04:37.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.583+0000 7f54816ba640 1 --2- 172.21.15.84:0/2626205447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f546c004d00 0x7f546c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:37.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.583+0000 7f5480eb9640 1 --2- 172.21.15.84:0/2626205447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f547c15aaa0 0x7f546c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:37.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.583+0000 7f54816ba640 1 -- 172.21.15.84:0/2626205447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f547c15aaa0 msgr2=0x7f546c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:37.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.583+0000 7f54816ba640 1 --2- 172.21.15.84:0/2626205447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f547c15aaa0 0x7f546c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.583+0000 7f54816ba640 1 -- 172.21.15.84:0/2626205447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f546c003ef0 msgr2=0x7f546c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:37.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.583+0000 7f547bfff640 1 --2- 172.21.15.84:0/2626205447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f546c003ef0 0x7f546c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:37.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.583+0000 7f54816ba640 1 --2- 172.21.15.84:0/2626205447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f546c003ef0 0x7f546c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.583+0000 7f54816ba640 1 -- 172.21.15.84:0/2626205447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546c003520 con 0x7f546c004d00 2023-10-16T10:04:37.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.583+0000 7f547bfff640 1 --2- 172.21.15.84:0/2626205447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f546c003ef0 0x7f546c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:37.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.585+0000 7f54816ba640 1 --2- 172.21.15.84:0/2626205447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f546c004d00 0x7f546c005180 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f5474009fc0 tx=0x7f546c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6672651ca6d08111 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:37.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.585+0000 7f547b7fe640 1 -- 172.21.15.84:0/2626205447 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547402cc40 con 0x7f546c004d00 2023-10-16T10:04:37.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.585+0000 7f547b7fe640 1 -- 172.21.15.84:0/2626205447 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547400f040 con 0x7f546c004d00 2023-10-16T10:04:37.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.585+0000 7f547b7fe640 1 -- 172.21.15.84:0/2626205447 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474034440 con 0x7f546c004d00 2023-10-16T10:04:37.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.585+0000 7f548311d640 1 -- 172.21.15.84:0/2626205447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f546c004d00 msgr2=0x7f546c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:37.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.585+0000 7f548311d640 1 --2- 172.21.15.84:0/2626205447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f546c004d00 0x7f546c005180 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f5474009fc0 tx=0x7f546c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.585+0000 7f54816ba640 1 -- 172.21.15.84:0/2626205447 reap_dead start 2023-10-16T10:04:37.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.585+0000 7f548311d640 1 -- 172.21.15.84:0/2626205447 shutdown_connections 2023-10-16T10:04:37.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.585+0000 7f548311d640 1 -- 172.21.15.84:0/2626205447 >> 172.21.15.84:0/2626205447 conn(0x7f547c0cb410 msgr2=0x7f547c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:37.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.586+0000 7f548311d640 1 -- 172.21.15.84:0/2626205447 shutdown_connections 2023-10-16T10:04:37.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.586+0000 7f548311d640 1 -- 172.21.15.84:0/2626205447 wait complete. 2023-10-16T10:04:37.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.587+0000 7f548311d640 1 Processor -- start 2023-10-16T10:04:37.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.588+0000 7f548311d640 1 -- start start 2023-10-16T10:04:37.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.588+0000 7f548311d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f546c004d00 0x7f547c1f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:37.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.588+0000 7f548311d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f547c151840 0x7f547c0b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:37.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.589+0000 7f548311d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f547c151cc0 0x7f547c0b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:37.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.589+0000 7f548311d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c152830 con 0x7f547c151840 2023-10-16T10:04:37.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.589+0000 7f548311d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c15b8a0 con 0x7f546c004d00 2023-10-16T10:04:37.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.589+0000 7f548311d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547c1503d0 con 0x7f547c151cc0 2023-10-16T10:04:37.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.589+0000 7f547bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f547c151840 0x7f547c0b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:37.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.589+0000 7f547bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f547c151840 0x7f547c0b2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43814/0 (socket says 172.21.15.84:43814) 2023-10-16T10:04:37.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.589+0000 7f547bfff640 1 -- 172.21.15.84:0/3573348995 learned_addr learned my addr 172.21.15.84:0/3573348995 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:37.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.589+0000 7f5480eb9640 1 --2- 172.21.15.84:0/3573348995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f546c004d00 0x7f547c1f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:37.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.589+0000 7f54816ba640 1 --2- 172.21.15.84:0/3573348995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f547c151cc0 0x7f547c0b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:37.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.589+0000 7f547bfff640 1 -- 172.21.15.84:0/3573348995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f547c151cc0 msgr2=0x7f547c0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:37.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.589+0000 7f547bfff640 1 --2- 172.21.15.84:0/3573348995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f547c151cc0 0x7f547c0b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.589+0000 7f547bfff640 1 -- 172.21.15.84:0/3573348995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f546c004d00 msgr2=0x7f547c1f3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:37.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.589+0000 7f547bfff640 1 --2- 172.21.15.84:0/3573348995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f546c004d00 0x7f547c1f3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.589+0000 7f547bfff640 1 -- 172.21.15.84:0/3573348995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c152270 con 0x7f547c151840 2023-10-16T10:04:37.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.590+0000 7f547bfff640 1 --2- 172.21.15.84:0/3573348995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f547c151840 0x7f547c0b2b40 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f546800dae0 tx=0x7f5468007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:37.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.590+0000 7f54797fa640 1 -- 172.21.15.84:0/3573348995 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54680074e0 con 0x7f547c151840 2023-10-16T10:04:37.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.590+0000 7f54797fa640 1 -- 172.21.15.84:0/3573348995 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5468007640 con 0x7f547c151840 2023-10-16T10:04:37.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.590+0000 7f548311d640 1 -- 172.21.15.84:0/3573348995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547c152550 con 0x7f547c151840 2023-10-16T10:04:37.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.590+0000 7f54797fa640 1 -- 172.21.15.84:0/3573348995 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54680049e0 con 0x7f547c151840 2023-10-16T10:04:37.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.590+0000 7f548311d640 1 -- 172.21.15.84:0/3573348995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547c1f3dc0 con 0x7f547c151840 2023-10-16T10:04:37.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.591+0000 7f548311d640 1 -- 172.21.15.84:0/3573348995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547c15b2e0 con 0x7f547c151840 2023-10-16T10:04:37.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.592+0000 7f54797fa640 1 -- 172.21.15.84:0/3573348995 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5468004b40 con 0x7f547c151840 2023-10-16T10:04:37.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.594+0000 7f54797fa640 1 --2- 172.21.15.84:0/3573348995 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f546006afc0 0x7f546006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:37.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.594+0000 7f54797fa640 1 -- 172.21.15.84:0/3573348995 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54680891c0 con 0x7f547c151840 2023-10-16T10:04:37.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.594+0000 7f5480eb9640 1 --2- 172.21.15.84:0/3573348995 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f546006afc0 0x7f546006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:37.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.594+0000 7f5480eb9640 1 --2- 172.21.15.84:0/3573348995 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f546006afc0 0x7f546006d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5470009a10 tx=0x7f5470008040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:37.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.598+0000 7f54797fa640 1 -- 172.21.15.84:0/3573348995 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f54680540e0 con 0x7f547c151840 2023-10-16T10:04:37.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.785+0000 7f548311d640 1 -- 172.21.15.84:0/3573348995 --> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f547c15f550 con 0x7f546006afc0 2023-10-16T10:04:37.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.797+0000 7f54797fa640 1 -- 172.21.15.84:0/3573348995 <== mgr.6279 v2:172.21.15.84:6832/2091947430 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286760 (secure 0 0 0) 0x7f5470013a10 con 0x7f546006afc0 2023-10-16T10:04:37.800 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:37.804 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T10:04:37.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.804+0000 7f548311d640 1 -- 172.21.15.84:0/3573348995 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f546006afc0 msgr2=0x7f546006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:37.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.804+0000 7f548311d640 1 --2- 172.21.15.84:0/3573348995 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f546006afc0 0x7f546006d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5470009a10 tx=0x7f5470008040 comp rx=0 tx=0).stop 2023-10-16T10:04:37.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.804+0000 7f548311d640 1 -- 172.21.15.84:0/3573348995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f547c151840 msgr2=0x7f547c0b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:37.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.804+0000 7f548311d640 1 --2- 172.21.15.84:0/3573348995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f547c151840 0x7f547c0b2b40 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f546800dae0 tx=0x7f5468007e30 comp rx=0 tx=0).stop 2023-10-16T10:04:37.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.804+0000 7f548311d640 1 -- 172.21.15.84:0/3573348995 shutdown_connections 2023-10-16T10:04:37.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.804+0000 7f548311d640 1 --2- 172.21.15.84:0/3573348995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f547c151cc0 0x7f547c0b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.804+0000 7f548311d640 1 --2- 172.21.15.84:0/3573348995 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f546006afc0 0x7f546006d480 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.804+0000 7f548311d640 1 --2- 172.21.15.84:0/3573348995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f547c151840 0x7f547c0b2b40 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.804+0000 7f548311d640 1 --2- 172.21.15.84:0/3573348995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f546c004d00 0x7f547c1f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:37.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.804+0000 7f548311d640 1 -- 172.21.15.84:0/3573348995 >> 172.21.15.84:0/3573348995 conn(0x7f547c0cb410 msgr2=0x7f547c06ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:37.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.804+0000 7f548311d640 1 -- 172.21.15.84:0/3573348995 shutdown_connections 2023-10-16T10:04:37.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:37.804+0000 7f548311d640 1 -- 172.21.15.84:0/3573348995 wait complete. 2023-10-16T10:04:37.824 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-16T10:04:36.257969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":315256,"kb_used_data":8900,"kb_used_omap":12,"kb_used_meta":306227,"kb_avail":754659464,"statfs":{"total":773094113280,"available":772771291136,"internally_reserved":0,"allocated":9113600,"data_stored":4929552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":313577017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.893831"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.902751+0000","last_change":"2023-10-16T10:04:32.926079+0000","last_active":"2023-10-16T10:04:33.902751+0000","last_peered":"2023-10-16T10:04:33.902751+0000","last_clean":"2023-10-16T10:04:33.902751+0000","last_became_active":"2023-10-16T10:04:32.925814+0000","last_became_peered":"2023-10-16T10:04:32.925814+0000","last_unstale":"2023-10-16T10:04:33.902751+0000","last_undegraded":"2023-10-16T10:04:33.902751+0000","last_fullsized":"2023-10-16T10:04:33.902751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:00:01.019028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.935265+0000","last_change":"2023-10-16T10:04:32.932684+0000","last_active":"2023-10-16T10:04:32.935265+0000","last_peered":"2023-10-16T10:04:32.935265+0000","last_clean":"2023-10-16T10:04:32.935265+0000","last_became_active":"2023-10-16T10:04:32.932408+0000","last_became_peered":"2023-10-16T10:04:32.932408+0000","last_unstale":"2023-10-16T10:04:32.935265+0000","last_undegraded":"2023-10-16T10:04:32.935265+0000","last_fullsized":"2023-10-16T10:04:32.935265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:35:58.720210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941388+0000","last_change":"2023-10-16T10:04:32.938354+0000","last_active":"2023-10-16T10:04:32.941388+0000","last_peered":"2023-10-16T10:04:32.941388+0000","last_clean":"2023-10-16T10:04:32.941388+0000","last_became_active":"2023-10-16T10:04:32.938028+0000","last_became_peered":"2023-10-16T10:04:32.938028+0000","last_unstale":"2023-10-16T10:04:32.941388+0000","last_undegraded":"2023-10-16T10:04:32.941388+0000","last_fullsized":"2023-10-16T10:04:32.941388+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:50.758989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.931637+0000","last_change":"2023-10-16T10:04:32.930020+0000","last_active":"2023-10-16T10:04:32.931637+0000","last_peered":"2023-10-16T10:04:32.931637+0000","last_clean":"2023-10-16T10:04:32.931637+0000","last_became_active":"2023-10-16T10:04:32.929644+0000","last_became_peered":"2023-10-16T10:04:32.929644+0000","last_unstale":"2023-10-16T10:04:32.931637+0000","last_undegraded":"2023-10-16T10:04:32.931637+0000","last_fullsized":"2023-10-16T10:04:32.931637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:07.694250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.902509+0000","last_change":"2023-10-16T10:04:32.922017+0000","last_active":"2023-10-16T10:04:33.902509+0000","last_peered":"2023-10-16T10:04:33.902509+0000","last_clean":"2023-10-16T10:04:33.902509+0000","last_became_active":"2023-10-16T10:04:32.921680+0000","last_became_peered":"2023-10-16T10:04:32.921680+0000","last_unstale":"2023-10-16T10:04:33.902509+0000","last_undegraded":"2023-10-16T10:04:33.902509+0000","last_fullsized":"2023-10-16T10:04:33.902509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:23:07.310751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.923277+0000","last_change":"2023-10-16T10:04:32.914554+0000","last_active":"2023-10-16T10:04:32.923277+0000","last_peered":"2023-10-16T10:04:32.923277+0000","last_clean":"2023-10-16T10:04:32.923277+0000","last_became_active":"2023-10-16T10:04:32.914057+0000","last_became_peered":"2023-10-16T10:04:32.914057+0000","last_unstale":"2023-10-16T10:04:32.923277+0000","last_undegraded":"2023-10-16T10:04:32.923277+0000","last_fullsized":"2023-10-16T10:04:32.923277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:59:49.327036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.905455+0000","last_change":"2023-10-16T10:04:32.930264+0000","last_active":"2023-10-16T10:04:33.905455+0000","last_peered":"2023-10-16T10:04:33.905455+0000","last_clean":"2023-10-16T10:04:33.905455+0000","last_became_active":"2023-10-16T10:04:32.929196+0000","last_became_peered":"2023-10-16T10:04:32.929196+0000","last_unstale":"2023-10-16T10:04:33.905455+0000","last_undegraded":"2023-10-16T10:04:33.905455+0000","last_fullsized":"2023-10-16T10:04:33.905455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:19.713522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.942248+0000","last_change":"2023-10-16T10:04:32.936863+0000","last_active":"2023-10-16T10:04:32.942248+0000","last_peered":"2023-10-16T10:04:32.942248+0000","last_clean":"2023-10-16T10:04:32.942248+0000","last_became_active":"2023-10-16T10:04:32.936506+0000","last_became_peered":"2023-10-16T10:04:32.936506+0000","last_unstale":"2023-10-16T10:04:32.942248+0000","last_undegraded":"2023-10-16T10:04:32.942248+0000","last_fullsized":"2023-10-16T10:04:32.942248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:19:32.252156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.908082+0000","last_change":"2023-10-16T10:04:32.928380+0000","last_active":"2023-10-16T10:04:33.908082+0000","last_peered":"2023-10-16T10:04:33.908082+0000","last_clean":"2023-10-16T10:04:33.908082+0000","last_became_active":"2023-10-16T10:04:32.926155+0000","last_became_peered":"2023-10-16T10:04:32.926155+0000","last_unstale":"2023-10-16T10:04:33.908082+0000","last_undegraded":"2023-10-16T10:04:33.908082+0000","last_fullsized":"2023-10-16T10:04:33.908082+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:35.530504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.226758+0000","last_change":"2023-10-16T10:04:32.920526+0000","last_active":"2023-10-16T10:04:34.226758+0000","last_peered":"2023-10-16T10:04:34.226758+0000","last_clean":"2023-10-16T10:04:34.226758+0000","last_became_active":"2023-10-16T10:04:32.920007+0000","last_became_peered":"2023-10-16T10:04:32.920007+0000","last_unstale":"2023-10-16T10:04:34.226758+0000","last_undegraded":"2023-10-16T10:04:34.226758+0000","last_fullsized":"2023-10-16T10:04:34.226758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:07.595656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.227009+0000","last_change":"2023-10-16T10:04:32.927399+0000","last_active":"2023-10-16T10:04:34.227009+0000","last_peered":"2023-10-16T10:04:34.227009+0000","last_clean":"2023-10-16T10:04:34.227009+0000","last_became_active":"2023-10-16T10:04:32.927124+0000","last_became_peered":"2023-10-16T10:04:32.927124+0000","last_unstale":"2023-10-16T10:04:34.227009+0000","last_undegraded":"2023-10-16T10:04:34.227009+0000","last_fullsized":"2023-10-16T10:04:34.227009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:29.941247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.929331+0000","last_change":"2023-10-16T10:04:32.925077+0000","last_active":"2023-10-16T10:04:32.929331+0000","last_peered":"2023-10-16T10:04:32.929331+0000","last_clean":"2023-10-16T10:04:32.929331+0000","last_became_active":"2023-10-16T10:04:32.924807+0000","last_became_peered":"2023-10-16T10:04:32.924807+0000","last_unstale":"2023-10-16T10:04:32.929331+0000","last_undegraded":"2023-10-16T10:04:32.929331+0000","last_fullsized":"2023-10-16T10:04:32.929331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:57.061875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.907486+0000","last_change":"2023-10-16T10:04:32.921077+0000","last_active":"2023-10-16T10:04:33.907486+0000","last_peered":"2023-10-16T10:04:33.907486+0000","last_clean":"2023-10-16T10:04:33.907486+0000","last_became_active":"2023-10-16T10:04:32.920772+0000","last_became_peered":"2023-10-16T10:04:32.920772+0000","last_unstale":"2023-10-16T10:04:33.907486+0000","last_undegraded":"2023-10-16T10:04:33.907486+0000","last_fullsized":"2023-10-16T10:04:33.907486+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:25.996045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.906831+0000","last_change":"2023-10-16T10:04:32.923805+0000","last_active":"2023-10-16T10:04:33.906831+0000","last_peered":"2023-10-16T10:04:33.906831+0000","last_clean":"2023-10-16T10:04:33.906831+0000","last_became_active":"2023-10-16T10:04:32.923435+0000","last_became_peered":"2023-10-16T10:04:32.923435+0000","last_unstale":"2023-10-16T10:04:33.906831+0000","last_undegraded":"2023-10-16T10:04:33.906831+0000","last_fullsized":"2023-10-16T10:04:33.906831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:20.246873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.906133+0000","last_change":"2023-10-16T10:04:32.923191+0000","last_active":"2023-10-16T10:04:33.906133+0000","last_peered":"2023-10-16T10:04:33.906133+0000","last_clean":"2023-10-16T10:04:33.906133+0000","last_became_active":"2023-10-16T10:04:32.922896+0000","last_became_peered":"2023-10-16T10:04:32.922896+0000","last_unstale":"2023-10-16T10:04:33.906133+0000","last_undegraded":"2023-10-16T10:04:33.906133+0000","last_fullsized":"2023-10-16T10:04:33.906133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:02:38.279482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.926536+0000","last_change":"2023-10-16T10:04:32.922411+0000","last_active":"2023-10-16T10:04:32.926536+0000","last_peered":"2023-10-16T10:04:32.926536+0000","last_clean":"2023-10-16T10:04:32.926536+0000","last_became_active":"2023-10-16T10:04:32.921967+0000","last_became_peered":"2023-10-16T10:04:32.921967+0000","last_unstale":"2023-10-16T10:04:32.926536+0000","last_undegraded":"2023-10-16T10:04:32.926536+0000","last_fullsized":"2023-10-16T10:04:32.926536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:10:49.135556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.935605+0000","last_change":"2023-10-16T10:04:32.934143+0000","last_active":"2023-10-16T10:04:32.935605+0000","last_peered":"2023-10-16T10:04:32.935605+0000","last_clean":"2023-10-16T10:04:32.935605+0000","last_became_active":"2023-10-16T10:04:32.933738+0000","last_became_peered":"2023-10-16T10:04:32.933738+0000","last_unstale":"2023-10-16T10:04:32.935605+0000","last_undegraded":"2023-10-16T10:04:32.935605+0000","last_fullsized":"2023-10-16T10:04:32.935605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:01:58.526456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941444+0000","last_change":"2023-10-16T10:04:32.938667+0000","last_active":"2023-10-16T10:04:32.941444+0000","last_peered":"2023-10-16T10:04:32.941444+0000","last_clean":"2023-10-16T10:04:32.941444+0000","last_became_active":"2023-10-16T10:04:32.938104+0000","last_became_peered":"2023-10-16T10:04:32.938104+0000","last_unstale":"2023-10-16T10:04:32.941444+0000","last_undegraded":"2023-10-16T10:04:32.941444+0000","last_fullsized":"2023-10-16T10:04:32.941444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:43.061747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.902318+0000","last_change":"2023-10-16T10:04:32.923865+0000","last_active":"2023-10-16T10:04:33.902318+0000","last_peered":"2023-10-16T10:04:33.902318+0000","last_clean":"2023-10-16T10:04:33.902318+0000","last_became_active":"2023-10-16T10:04:32.923152+0000","last_became_peered":"2023-10-16T10:04:32.923152+0000","last_unstale":"2023-10-16T10:04:33.902318+0000","last_undegraded":"2023-10-16T10:04:33.902318+0000","last_fullsized":"2023-10-16T10:04:33.902318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:15:34.338966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.935632+0000","last_change":"2023-10-16T10:04:32.933489+0000","last_active":"2023-10-16T10:04:32.935632+0000","last_peered":"2023-10-16T10:04:32.935632+0000","last_clean":"2023-10-16T10:04:32.935632+0000","last_became_active":"2023-10-16T10:04:32.930967+0000","last_became_peered":"2023-10-16T10:04:32.930967+0000","last_unstale":"2023-10-16T10:04:32.935632+0000","last_undegraded":"2023-10-16T10:04:32.935632+0000","last_fullsized":"2023-10-16T10:04:32.935632+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:56.950292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941923+0000","last_change":"2023-10-16T10:04:32.941047+0000","last_active":"2023-10-16T10:04:32.941923+0000","last_peered":"2023-10-16T10:04:32.941923+0000","last_clean":"2023-10-16T10:04:32.941923+0000","last_became_active":"2023-10-16T10:04:32.939094+0000","last_became_peered":"2023-10-16T10:04:32.939094+0000","last_unstale":"2023-10-16T10:04:32.941923+0000","last_undegraded":"2023-10-16T10:04:32.941923+0000","last_fullsized":"2023-10-16T10:04:32.941923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:28:14.674334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.934786+0000","last_change":"2023-10-16T10:04:32.930517+0000","last_active":"2023-10-16T10:04:32.934786+0000","last_peered":"2023-10-16T10:04:32.934786+0000","last_clean":"2023-10-16T10:04:32.934786+0000","last_became_active":"2023-10-16T10:04:32.930282+0000","last_became_peered":"2023-10-16T10:04:32.930282+0000","last_unstale":"2023-10-16T10:04:32.934786+0000","last_undegraded":"2023-10-16T10:04:32.934786+0000","last_fullsized":"2023-10-16T10:04:32.934786+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:59:06.817328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941412+0000","last_change":"2023-10-16T10:04:32.938433+0000","last_active":"2023-10-16T10:04:32.941412+0000","last_peered":"2023-10-16T10:04:32.941412+0000","last_clean":"2023-10-16T10:04:32.941412+0000","last_became_active":"2023-10-16T10:04:32.937023+0000","last_became_peered":"2023-10-16T10:04:32.937023+0000","last_unstale":"2023-10-16T10:04:32.941412+0000","last_undegraded":"2023-10-16T10:04:32.941412+0000","last_fullsized":"2023-10-16T10:04:32.941412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:59.516633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.906260+0000","last_change":"2023-10-16T10:04:32.917152+0000","last_active":"2023-10-16T10:04:33.906260+0000","last_peered":"2023-10-16T10:04:33.906260+0000","last_clean":"2023-10-16T10:04:33.906260+0000","last_became_active":"2023-10-16T10:04:32.916260+0000","last_became_peered":"2023-10-16T10:04:32.916260+0000","last_unstale":"2023-10-16T10:04:33.906260+0000","last_undegraded":"2023-10-16T10:04:33.906260+0000","last_fullsized":"2023-10-16T10:04:33.906260+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:22:10.081358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.902812+0000","last_change":"2023-10-16T10:04:32.927618+0000","last_active":"2023-10-16T10:04:33.902812+0000","last_peered":"2023-10-16T10:04:33.902812+0000","last_clean":"2023-10-16T10:04:33.902812+0000","last_became_active":"2023-10-16T10:04:32.927364+0000","last_became_peered":"2023-10-16T10:04:32.927364+0000","last_unstale":"2023-10-16T10:04:33.902812+0000","last_undegraded":"2023-10-16T10:04:33.902812+0000","last_fullsized":"2023-10-16T10:04:33.902812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:11:58.169115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.929466+0000","last_change":"2023-10-16T10:04:32.925291+0000","last_active":"2023-10-16T10:04:32.929466+0000","last_peered":"2023-10-16T10:04:32.929466+0000","last_clean":"2023-10-16T10:04:32.929466+0000","last_became_active":"2023-10-16T10:04:32.923656+0000","last_became_peered":"2023-10-16T10:04:32.923656+0000","last_unstale":"2023-10-16T10:04:32.929466+0000","last_undegraded":"2023-10-16T10:04:32.929466+0000","last_fullsized":"2023-10-16T10:04:32.929466+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:56:24.735587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.933646+0000","last_change":"2023-10-16T10:04:32.931384+0000","last_active":"2023-10-16T10:04:32.933646+0000","last_peered":"2023-10-16T10:04:32.933646+0000","last_clean":"2023-10-16T10:04:32.933646+0000","last_became_active":"2023-10-16T10:04:32.931034+0000","last_became_peered":"2023-10-16T10:04:32.931034+0000","last_unstale":"2023-10-16T10:04:32.933646+0000","last_undegraded":"2023-10-16T10:04:32.933646+0000","last_fullsized":"2023-10-16T10:04:32.933646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:11:15.386027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.902426+0000","last_change":"2023-10-16T10:04:32.924415+0000","last_active":"2023-10-16T10:04:33.902426+0000","last_peered":"2023-10-16T10:04:33.902426+0000","last_clean":"2023-10-16T10:04:33.902426+0000","last_became_active":"2023-10-16T10:04:32.924131+0000","last_became_peered":"2023-10-16T10:04:32.924131+0000","last_unstale":"2023-10-16T10:04:33.902426+0000","last_undegraded":"2023-10-16T10:04:33.902426+0000","last_fullsized":"2023-10-16T10:04:33.902426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:48:04.380170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.936250+0000","last_change":"2023-10-16T10:04:32.932935+0000","last_active":"2023-10-16T10:04:32.936250+0000","last_peered":"2023-10-16T10:04:32.936250+0000","last_clean":"2023-10-16T10:04:32.936250+0000","last_became_active":"2023-10-16T10:04:32.932667+0000","last_became_peered":"2023-10-16T10:04:32.932667+0000","last_unstale":"2023-10-16T10:04:32.936250+0000","last_undegraded":"2023-10-16T10:04:32.936250+0000","last_fullsized":"2023-10-16T10:04:32.936250+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:55:38.814648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941500+0000","last_change":"2023-10-16T10:04:32.939264+0000","last_active":"2023-10-16T10:04:32.941500+0000","last_peered":"2023-10-16T10:04:32.941500+0000","last_clean":"2023-10-16T10:04:32.941500+0000","last_became_active":"2023-10-16T10:04:32.938951+0000","last_became_peered":"2023-10-16T10:04:32.938951+0000","last_unstale":"2023-10-16T10:04:32.941500+0000","last_undegraded":"2023-10-16T10:04:32.941500+0000","last_fullsized":"2023-10-16T10:04:32.941500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:05.860336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.902127+0000","last_change":"2023-10-16T10:04:32.918179+0000","last_active":"2023-10-16T10:04:33.902127+0000","last_peered":"2023-10-16T10:04:33.902127+0000","last_clean":"2023-10-16T10:04:33.902127+0000","last_became_active":"2023-10-16T10:04:32.917748+0000","last_became_peered":"2023-10-16T10:04:32.917748+0000","last_unstale":"2023-10-16T10:04:33.902127+0000","last_undegraded":"2023-10-16T10:04:33.902127+0000","last_fullsized":"2023-10-16T10:04:33.902127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:41:02.640911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.928626+0000","last_change":"2023-10-16T10:04:32.925916+0000","last_active":"2023-10-16T10:04:32.928626+0000","last_peered":"2023-10-16T10:04:32.928626+0000","last_clean":"2023-10-16T10:04:32.928626+0000","last_became_active":"2023-10-16T10:04:32.925628+0000","last_became_peered":"2023-10-16T10:04:32.925628+0000","last_unstale":"2023-10-16T10:04:32.928626+0000","last_undegraded":"2023-10-16T10:04:32.928626+0000","last_fullsized":"2023-10-16T10:04:32.928626+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:34.845253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.928642+0000","last_change":"2023-10-16T10:04:32.925657+0000","last_active":"2023-10-16T10:04:32.928642+0000","last_peered":"2023-10-16T10:04:32.928642+0000","last_clean":"2023-10-16T10:04:32.928642+0000","last_became_active":"2023-10-16T10:04:32.925339+0000","last_became_peered":"2023-10-16T10:04:32.925339+0000","last_unstale":"2023-10-16T10:04:32.928642+0000","last_undegraded":"2023-10-16T10:04:32.928642+0000","last_fullsized":"2023-10-16T10:04:32.928642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:17.155011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"81'4","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.415175+0000","last_change":"2023-10-16T10:04:32.417835+0000","last_active":"2023-10-16T10:04:33.415175+0000","last_peered":"2023-10-16T10:04:33.415175+0000","last_clean":"2023-10-16T10:04:33.415175+0000","last_became_active":"2023-10-16T10:04:32.417569+0000","last_became_peered":"2023-10-16T10:04:32.417569+0000","last_unstale":"2023-10-16T10:04:33.415175+0000","last_undegraded":"2023-10-16T10:04:33.415175+0000","last_fullsized":"2023-10-16T10:04:33.415175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:33.267723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.227014+0000","last_change":"2023-10-16T10:04:32.928052+0000","last_active":"2023-10-16T10:04:34.227014+0000","last_peered":"2023-10-16T10:04:34.227014+0000","last_clean":"2023-10-16T10:04:34.227014+0000","last_became_active":"2023-10-16T10:04:32.927718+0000","last_became_peered":"2023-10-16T10:04:32.927718+0000","last_unstale":"2023-10-16T10:04:34.227014+0000","last_undegraded":"2023-10-16T10:04:34.227014+0000","last_fullsized":"2023-10-16T10:04:34.227014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:05:04.283334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"81'1","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.415765+0000","last_change":"2023-10-16T10:04:32.414330+0000","last_active":"2023-10-16T10:04:33.415765+0000","last_peered":"2023-10-16T10:04:33.415765+0000","last_clean":"2023-10-16T10:04:33.415765+0000","last_became_active":"2023-10-16T10:04:32.413240+0000","last_became_peered":"2023-10-16T10:04:32.413240+0000","last_unstale":"2023-10-16T10:04:33.415765+0000","last_undegraded":"2023-10-16T10:04:33.415765+0000","last_fullsized":"2023-10-16T10:04:33.415765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:50.101733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.924402+0000","last_change":"2023-10-16T10:04:32.917028+0000","last_active":"2023-10-16T10:04:32.924402+0000","last_peered":"2023-10-16T10:04:32.924402+0000","last_clean":"2023-10-16T10:04:32.924402+0000","last_became_active":"2023-10-16T10:04:32.916590+0000","last_became_peered":"2023-10-16T10:04:32.916590+0000","last_unstale":"2023-10-16T10:04:32.924402+0000","last_undegraded":"2023-10-16T10:04:32.924402+0000","last_fullsized":"2023-10-16T10:04:32.924402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:02:44.125425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"81'2","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.409078+0000","last_change":"2023-10-16T10:04:32.410241+0000","last_active":"2023-10-16T10:04:33.409078+0000","last_peered":"2023-10-16T10:04:33.409078+0000","last_clean":"2023-10-16T10:04:33.409078+0000","last_became_active":"2023-10-16T10:04:32.409955+0000","last_became_peered":"2023-10-16T10:04:32.409955+0000","last_unstale":"2023-10-16T10:04:33.409078+0000","last_undegraded":"2023-10-16T10:04:33.409078+0000","last_fullsized":"2023-10-16T10:04:33.409078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:05.180348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.928672+0000","last_change":"2023-10-16T10:04:32.926355+0000","last_active":"2023-10-16T10:04:32.928672+0000","last_peered":"2023-10-16T10:04:32.928672+0000","last_clean":"2023-10-16T10:04:32.928672+0000","last_became_active":"2023-10-16T10:04:32.926095+0000","last_became_peered":"2023-10-16T10:04:32.926095+0000","last_unstale":"2023-10-16T10:04:32.928672+0000","last_undegraded":"2023-10-16T10:04:32.928672+0000","last_fullsized":"2023-10-16T10:04:32.928672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:25.996045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"81'2","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.416756+0000","last_change":"2023-10-16T10:04:32.419982+0000","last_active":"2023-10-16T10:04:33.416756+0000","last_peered":"2023-10-16T10:04:33.416756+0000","last_clean":"2023-10-16T10:04:33.416756+0000","last_became_active":"2023-10-16T10:04:32.419520+0000","last_became_peered":"2023-10-16T10:04:32.419520+0000","last_unstale":"2023-10-16T10:04:33.416756+0000","last_undegraded":"2023-10-16T10:04:33.416756+0000","last_fullsized":"2023-10-16T10:04:33.416756+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:16.068838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.934806+0000","last_change":"2023-10-16T10:04:32.931577+0000","last_active":"2023-10-16T10:04:32.934806+0000","last_peered":"2023-10-16T10:04:32.934806+0000","last_clean":"2023-10-16T10:04:32.934806+0000","last_became_active":"2023-10-16T10:04:32.930615+0000","last_became_peered":"2023-10-16T10:04:32.930615+0000","last_unstale":"2023-10-16T10:04:32.934806+0000","last_undegraded":"2023-10-16T10:04:32.934806+0000","last_fullsized":"2023-10-16T10:04:32.934806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:46:00.280779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"81'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.907933+0000","last_change":"2023-10-16T10:04:32.404293+0000","last_active":"2023-10-16T10:04:33.907933+0000","last_peered":"2023-10-16T10:04:33.907933+0000","last_clean":"2023-10-16T10:04:33.907933+0000","last_became_active":"2023-10-16T10:04:32.403501+0000","last_became_peered":"2023-10-16T10:04:32.403501+0000","last_unstale":"2023-10-16T10:04:33.907933+0000","last_undegraded":"2023-10-16T10:04:33.907933+0000","last_fullsized":"2023-10-16T10:04:33.907933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:12.189885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.936232+0000","last_change":"2023-10-16T10:04:32.931169+0000","last_active":"2023-10-16T10:04:32.936232+0000","last_peered":"2023-10-16T10:04:32.936232+0000","last_clean":"2023-10-16T10:04:32.936232+0000","last_became_active":"2023-10-16T10:04:32.930358+0000","last_became_peered":"2023-10-16T10:04:32.930358+0000","last_unstale":"2023-10-16T10:04:32.936232+0000","last_undegraded":"2023-10-16T10:04:32.936232+0000","last_fullsized":"2023-10-16T10:04:32.936232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:11:42.522908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"81'1","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.411412+0000","last_change":"2023-10-16T10:04:32.409472+0000","last_active":"2023-10-16T10:04:33.411412+0000","last_peered":"2023-10-16T10:04:33.411412+0000","last_clean":"2023-10-16T10:04:33.411412+0000","last_became_active":"2023-10-16T10:04:32.408232+0000","last_became_peered":"2023-10-16T10:04:32.408232+0000","last_unstale":"2023-10-16T10:04:33.411412+0000","last_undegraded":"2023-10-16T10:04:33.411412+0000","last_fullsized":"2023-10-16T10:04:33.411412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:25:18.407018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.906736+0000","last_change":"2023-10-16T10:04:32.927591+0000","last_active":"2023-10-16T10:04:33.906736+0000","last_peered":"2023-10-16T10:04:33.906736+0000","last_clean":"2023-10-16T10:04:33.906736+0000","last_became_active":"2023-10-16T10:04:32.927312+0000","last_became_peered":"2023-10-16T10:04:32.927312+0000","last_unstale":"2023-10-16T10:04:33.906736+0000","last_undegraded":"2023-10-16T10:04:33.906736+0000","last_fullsized":"2023-10-16T10:04:33.906736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:11.098931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.918197+0000","last_change":"2023-10-16T10:04:32.415160+0000","last_active":"2023-10-16T10:04:32.918197+0000","last_peered":"2023-10-16T10:04:32.918197+0000","last_clean":"2023-10-16T10:04:32.918197+0000","last_became_active":"2023-10-16T10:04:32.414824+0000","last_became_peered":"2023-10-16T10:04:32.414824+0000","last_unstale":"2023-10-16T10:04:32.918197+0000","last_undegraded":"2023-10-16T10:04:32.918197+0000","last_fullsized":"2023-10-16T10:04:32.918197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:25:04.823560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.935629+0000","last_change":"2023-10-16T10:04:32.933809+0000","last_active":"2023-10-16T10:04:32.935629+0000","last_peered":"2023-10-16T10:04:32.935629+0000","last_clean":"2023-10-16T10:04:32.935629+0000","last_became_active":"2023-10-16T10:04:32.933568+0000","last_became_peered":"2023-10-16T10:04:32.933568+0000","last_unstale":"2023-10-16T10:04:32.935629+0000","last_undegraded":"2023-10-16T10:04:32.935629+0000","last_fullsized":"2023-10-16T10:04:32.935629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:43.061747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.905372+0000","last_change":"2023-10-16T10:04:32.409147+0000","last_active":"2023-10-16T10:04:33.905372+0000","last_peered":"2023-10-16T10:04:33.905372+0000","last_clean":"2023-10-16T10:04:33.905372+0000","last_became_active":"2023-10-16T10:04:32.408629+0000","last_became_peered":"2023-10-16T10:04:32.408629+0000","last_unstale":"2023-10-16T10:04:33.905372+0000","last_undegraded":"2023-10-16T10:04:33.905372+0000","last_fullsized":"2023-10-16T10:04:33.905372+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:18.591659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.908053+0000","last_change":"2023-10-16T10:04:32.925973+0000","last_active":"2023-10-16T10:04:33.908053+0000","last_peered":"2023-10-16T10:04:33.908053+0000","last_clean":"2023-10-16T10:04:33.908053+0000","last_became_active":"2023-10-16T10:04:32.924473+0000","last_became_peered":"2023-10-16T10:04:32.924473+0000","last_unstale":"2023-10-16T10:04:33.908053+0000","last_undegraded":"2023-10-16T10:04:33.908053+0000","last_fullsized":"2023-10-16T10:04:33.908053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:05:42.992114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.933536+0000","last_change":"2023-10-16T10:04:32.418245+0000","last_active":"2023-10-16T10:04:32.933536+0000","last_peered":"2023-10-16T10:04:32.933536+0000","last_clean":"2023-10-16T10:04:32.933536+0000","last_became_active":"2023-10-16T10:04:32.417836+0000","last_became_peered":"2023-10-16T10:04:32.417836+0000","last_unstale":"2023-10-16T10:04:32.933536+0000","last_undegraded":"2023-10-16T10:04:32.933536+0000","last_fullsized":"2023-10-16T10:04:32.933536+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:03:58.243837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.943375+0000","last_change":"2023-10-16T10:04:32.942699+0000","last_active":"2023-10-16T10:04:32.943375+0000","last_peered":"2023-10-16T10:04:32.943375+0000","last_clean":"2023-10-16T10:04:32.943375+0000","last_became_active":"2023-10-16T10:04:32.942439+0000","last_became_peered":"2023-10-16T10:04:32.942439+0000","last_unstale":"2023-10-16T10:04:32.943375+0000","last_undegraded":"2023-10-16T10:04:32.943375+0000","last_fullsized":"2023-10-16T10:04:32.943375+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:59:06.817328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.925205+0000","last_change":"2023-10-16T10:04:32.411805+0000","last_active":"2023-10-16T10:04:32.925205+0000","last_peered":"2023-10-16T10:04:32.925205+0000","last_clean":"2023-10-16T10:04:32.925205+0000","last_became_active":"2023-10-16T10:04:32.410870+0000","last_became_peered":"2023-10-16T10:04:32.410870+0000","last_unstale":"2023-10-16T10:04:32.925205+0000","last_undegraded":"2023-10-16T10:04:32.925205+0000","last_fullsized":"2023-10-16T10:04:32.925205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:05:21.781941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.226433+0000","last_change":"2023-10-16T10:04:32.923001+0000","last_active":"2023-10-16T10:04:34.226433+0000","last_peered":"2023-10-16T10:04:34.226433+0000","last_clean":"2023-10-16T10:04:34.226433+0000","last_became_active":"2023-10-16T10:04:32.922735+0000","last_became_peered":"2023-10-16T10:04:32.922735+0000","last_unstale":"2023-10-16T10:04:34.226433+0000","last_undegraded":"2023-10-16T10:04:34.226433+0000","last_fullsized":"2023-10-16T10:04:34.226433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:21:02.362549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"81'3","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.415190+0000","last_change":"2023-10-16T10:04:32.418402+0000","last_active":"2023-10-16T10:04:33.415190+0000","last_peered":"2023-10-16T10:04:33.415190+0000","last_clean":"2023-10-16T10:04:33.415190+0000","last_became_active":"2023-10-16T10:04:32.417878+0000","last_became_peered":"2023-10-16T10:04:32.417878+0000","last_unstale":"2023-10-16T10:04:33.415190+0000","last_undegraded":"2023-10-16T10:04:33.415190+0000","last_fullsized":"2023-10-16T10:04:33.415190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:19:31.235315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941214+0000","last_change":"2023-10-16T10:04:32.936505+0000","last_active":"2023-10-16T10:04:32.941214+0000","last_peered":"2023-10-16T10:04:32.941214+0000","last_clean":"2023-10-16T10:04:32.941214+0000","last_became_active":"2023-10-16T10:04:32.936228+0000","last_became_peered":"2023-10-16T10:04:32.936228+0000","last_unstale":"2023-10-16T10:04:32.941214+0000","last_undegraded":"2023-10-16T10:04:32.941214+0000","last_fullsized":"2023-10-16T10:04:32.941214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:35:58.720210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"81'6","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.908172+0000","last_change":"2023-10-16T10:04:32.411084+0000","last_active":"2023-10-16T10:04:33.908172+0000","last_peered":"2023-10-16T10:04:33.908172+0000","last_clean":"2023-10-16T10:04:33.908172+0000","last_became_active":"2023-10-16T10:04:32.407041+0000","last_became_peered":"2023-10-16T10:04:32.407041+0000","last_unstale":"2023-10-16T10:04:33.908172+0000","last_undegraded":"2023-10-16T10:04:33.908172+0000","last_fullsized":"2023-10-16T10:04:33.908172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:33:12.451753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.930036+0000","last_change":"2023-10-16T10:04:32.926391+0000","last_active":"2023-10-16T10:04:32.930036+0000","last_peered":"2023-10-16T10:04:32.930036+0000","last_clean":"2023-10-16T10:04:32.930036+0000","last_became_active":"2023-10-16T10:04:32.925637+0000","last_became_peered":"2023-10-16T10:04:32.925637+0000","last_unstale":"2023-10-16T10:04:32.930036+0000","last_undegraded":"2023-10-16T10:04:32.930036+0000","last_fullsized":"2023-10-16T10:04:32.930036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:17:08.069358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"81'2","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.407868+0000","last_change":"2023-10-16T10:04:32.410283+0000","last_active":"2023-10-16T10:04:33.407868+0000","last_peered":"2023-10-16T10:04:33.407868+0000","last_clean":"2023-10-16T10:04:33.407868+0000","last_became_active":"2023-10-16T10:04:32.409976+0000","last_became_peered":"2023-10-16T10:04:32.409976+0000","last_unstale":"2023-10-16T10:04:33.407868+0000","last_undegraded":"2023-10-16T10:04:33.407868+0000","last_fullsized":"2023-10-16T10:04:33.407868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:57:16.836538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941493+0000","last_change":"2023-10-16T10:04:32.938998+0000","last_active":"2023-10-16T10:04:32.941493+0000","last_peered":"2023-10-16T10:04:32.941493+0000","last_clean":"2023-10-16T10:04:32.941493+0000","last_became_active":"2023-10-16T10:04:32.938683+0000","last_became_peered":"2023-10-16T10:04:32.938683+0000","last_unstale":"2023-10-16T10:04:32.941493+0000","last_undegraded":"2023-10-16T10:04:32.941493+0000","last_fullsized":"2023-10-16T10:04:32.941493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:11:42.522908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.906205+0000","last_change":"2023-10-16T10:04:32.411769+0000","last_active":"2023-10-16T10:04:33.906205+0000","last_peered":"2023-10-16T10:04:33.906205+0000","last_clean":"2023-10-16T10:04:33.906205+0000","last_became_active":"2023-10-16T10:04:32.411479+0000","last_became_peered":"2023-10-16T10:04:32.411479+0000","last_unstale":"2023-10-16T10:04:33.906205+0000","last_undegraded":"2023-10-16T10:04:33.906205+0000","last_fullsized":"2023-10-16T10:04:33.906205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:33.828412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.905287+0000","last_change":"2023-10-16T10:04:32.929535+0000","last_active":"2023-10-16T10:04:33.905287+0000","last_peered":"2023-10-16T10:04:33.905287+0000","last_clean":"2023-10-16T10:04:33.905287+0000","last_became_active":"2023-10-16T10:04:32.928852+0000","last_became_peered":"2023-10-16T10:04:32.928852+0000","last_unstale":"2023-10-16T10:04:33.905287+0000","last_undegraded":"2023-10-16T10:04:33.905287+0000","last_fullsized":"2023-10-16T10:04:33.905287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:46:15.616966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.918283+0000","last_change":"2023-10-16T10:04:32.413859+0000","last_active":"2023-10-16T10:04:32.918283+0000","last_peered":"2023-10-16T10:04:32.918283+0000","last_clean":"2023-10-16T10:04:32.918283+0000","last_became_active":"2023-10-16T10:04:32.413589+0000","last_became_peered":"2023-10-16T10:04:32.413589+0000","last_unstale":"2023-10-16T10:04:32.918283+0000","last_undegraded":"2023-10-16T10:04:32.918283+0000","last_fullsized":"2023-10-16T10:04:32.918283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:18.489976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941324+0000","last_change":"2023-10-16T10:04:32.937634+0000","last_active":"2023-10-16T10:04:32.941324+0000","last_peered":"2023-10-16T10:04:32.941324+0000","last_clean":"2023-10-16T10:04:32.941324+0000","last_became_active":"2023-10-16T10:04:32.936778+0000","last_became_peered":"2023-10-16T10:04:32.936778+0000","last_unstale":"2023-10-16T10:04:32.941324+0000","last_undegraded":"2023-10-16T10:04:32.941324+0000","last_fullsized":"2023-10-16T10:04:32.941324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:55:38.814648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"81'8","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.418933+0000","last_change":"2023-10-16T10:04:32.418238+0000","last_active":"2023-10-16T10:04:33.418933+0000","last_peered":"2023-10-16T10:04:33.418933+0000","last_clean":"2023-10-16T10:04:33.418933+0000","last_became_active":"2023-10-16T10:04:32.417949+0000","last_became_peered":"2023-10-16T10:04:32.417949+0000","last_unstale":"2023-10-16T10:04:33.418933+0000","last_undegraded":"2023-10-16T10:04:33.418933+0000","last_fullsized":"2023-10-16T10:04:33.418933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:44.953428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941524+0000","last_change":"2023-10-16T10:04:32.939737+0000","last_active":"2023-10-16T10:04:32.941524+0000","last_peered":"2023-10-16T10:04:32.941524+0000","last_clean":"2023-10-16T10:04:32.941524+0000","last_became_active":"2023-10-16T10:04:32.939017+0000","last_became_peered":"2023-10-16T10:04:32.939017+0000","last_unstale":"2023-10-16T10:04:32.941524+0000","last_undegraded":"2023-10-16T10:04:32.941524+0000","last_fullsized":"2023-10-16T10:04:32.941524+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:46:00.280779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"81'1","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.407782+0000","last_change":"2023-10-16T10:04:32.412234+0000","last_active":"2023-10-16T10:04:33.407782+0000","last_peered":"2023-10-16T10:04:33.407782+0000","last_clean":"2023-10-16T10:04:33.407782+0000","last_became_active":"2023-10-16T10:04:32.411416+0000","last_became_peered":"2023-10-16T10:04:32.411416+0000","last_unstale":"2023-10-16T10:04:33.407782+0000","last_undegraded":"2023-10-16T10:04:33.407782+0000","last_fullsized":"2023-10-16T10:04:33.407782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:44:48.305279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.929424+0000","last_change":"2023-10-16T10:04:32.925656+0000","last_active":"2023-10-16T10:04:32.929424+0000","last_peered":"2023-10-16T10:04:32.929424+0000","last_clean":"2023-10-16T10:04:32.929424+0000","last_became_active":"2023-10-16T10:04:32.925377+0000","last_became_peered":"2023-10-16T10:04:32.925377+0000","last_unstale":"2023-10-16T10:04:32.929424+0000","last_undegraded":"2023-10-16T10:04:32.929424+0000","last_fullsized":"2023-10-16T10:04:32.929424+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:30:33.124497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"81'2","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.409509+0000","last_change":"2023-10-16T10:04:32.407759+0000","last_active":"2023-10-16T10:04:33.409509+0000","last_peered":"2023-10-16T10:04:33.409509+0000","last_clean":"2023-10-16T10:04:33.409509+0000","last_became_active":"2023-10-16T10:04:32.407431+0000","last_became_peered":"2023-10-16T10:04:32.407431+0000","last_unstale":"2023-10-16T10:04:33.409509+0000","last_undegraded":"2023-10-16T10:04:33.409509+0000","last_fullsized":"2023-10-16T10:04:33.409509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:28:55.400277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.931274+0000","last_change":"2023-10-16T10:04:32.928011+0000","last_active":"2023-10-16T10:04:32.931274+0000","last_peered":"2023-10-16T10:04:32.931274+0000","last_clean":"2023-10-16T10:04:32.931274+0000","last_became_active":"2023-10-16T10:04:32.927717+0000","last_became_peered":"2023-10-16T10:04:32.927717+0000","last_unstale":"2023-10-16T10:04:32.931274+0000","last_undegraded":"2023-10-16T10:04:32.931274+0000","last_fullsized":"2023-10-16T10:04:32.931274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:50.758989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"81'1","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.416739+0000","last_change":"2023-10-16T10:04:32.417048+0000","last_active":"2023-10-16T10:04:33.416739+0000","last_peered":"2023-10-16T10:04:33.416739+0000","last_clean":"2023-10-16T10:04:33.416739+0000","last_became_active":"2023-10-16T10:04:32.416257+0000","last_became_peered":"2023-10-16T10:04:32.416257+0000","last_unstale":"2023-10-16T10:04:33.416739+0000","last_undegraded":"2023-10-16T10:04:33.416739+0000","last_fullsized":"2023-10-16T10:04:33.416739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:09:24.405679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.943382+0000","last_change":"2023-10-16T10:04:32.942704+0000","last_active":"2023-10-16T10:04:32.943382+0000","last_peered":"2023-10-16T10:04:32.943382+0000","last_clean":"2023-10-16T10:04:32.943382+0000","last_became_active":"2023-10-16T10:04:32.942449+0000","last_became_peered":"2023-10-16T10:04:32.942449+0000","last_unstale":"2023-10-16T10:04:32.943382+0000","last_undegraded":"2023-10-16T10:04:32.943382+0000","last_fullsized":"2023-10-16T10:04:32.943382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:01:58.526456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.907757+0000","last_change":"2023-10-16T10:04:32.409334+0000","last_active":"2023-10-16T10:04:32.907757+0000","last_peered":"2023-10-16T10:04:32.907757+0000","last_clean":"2023-10-16T10:04:32.907757+0000","last_became_active":"2023-10-16T10:04:32.409063+0000","last_became_peered":"2023-10-16T10:04:32.409063+0000","last_unstale":"2023-10-16T10:04:32.907757+0000","last_undegraded":"2023-10-16T10:04:32.907757+0000","last_fullsized":"2023-10-16T10:04:32.907757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:05:13.298000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.930129+0000","last_change":"2023-10-16T10:04:32.926560+0000","last_active":"2023-10-16T10:04:32.930129+0000","last_peered":"2023-10-16T10:04:32.930129+0000","last_clean":"2023-10-16T10:04:32.930129+0000","last_became_active":"2023-10-16T10:04:32.926069+0000","last_became_peered":"2023-10-16T10:04:32.926069+0000","last_unstale":"2023-10-16T10:04:32.930129+0000","last_undegraded":"2023-10-16T10:04:32.930129+0000","last_fullsized":"2023-10-16T10:04:32.930129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:06:33.097982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"81'2","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.411018+0000","last_change":"2023-10-16T10:04:32.405741+0000","last_active":"2023-10-16T10:04:33.411018+0000","last_peered":"2023-10-16T10:04:33.411018+0000","last_clean":"2023-10-16T10:04:33.411018+0000","last_became_active":"2023-10-16T10:04:32.405464+0000","last_became_peered":"2023-10-16T10:04:32.405464+0000","last_unstale":"2023-10-16T10:04:33.411018+0000","last_undegraded":"2023-10-16T10:04:33.411018+0000","last_fullsized":"2023-10-16T10:04:33.411018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:23:17.597276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.926585+0000","last_change":"2023-10-16T10:04:32.921140+0000","last_active":"2023-10-16T10:04:32.926585+0000","last_peered":"2023-10-16T10:04:32.926585+0000","last_clean":"2023-10-16T10:04:32.926585+0000","last_became_active":"2023-10-16T10:04:32.919995+0000","last_became_peered":"2023-10-16T10:04:32.919995+0000","last_unstale":"2023-10-16T10:04:32.926585+0000","last_undegraded":"2023-10-16T10:04:32.926585+0000","last_fullsized":"2023-10-16T10:04:32.926585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:30.945110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.226305+0000","last_change":"2023-10-16T10:04:32.397651+0000","last_active":"2023-10-16T10:04:34.226305+0000","last_peered":"2023-10-16T10:04:34.226305+0000","last_clean":"2023-10-16T10:04:34.226305+0000","last_became_active":"2023-10-16T10:04:32.397339+0000","last_became_peered":"2023-10-16T10:04:32.397339+0000","last_unstale":"2023-10-16T10:04:34.226305+0000","last_undegraded":"2023-10-16T10:04:34.226305+0000","last_fullsized":"2023-10-16T10:04:34.226305+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:45.561197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.928725+0000","last_change":"2023-10-16T10:04:32.926226+0000","last_active":"2023-10-16T10:04:32.928725+0000","last_peered":"2023-10-16T10:04:32.928725+0000","last_clean":"2023-10-16T10:04:32.928725+0000","last_became_active":"2023-10-16T10:04:32.925300+0000","last_became_peered":"2023-10-16T10:04:32.925300+0000","last_unstale":"2023-10-16T10:04:32.928725+0000","last_undegraded":"2023-10-16T10:04:32.928725+0000","last_fullsized":"2023-10-16T10:04:32.928725+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:22:10.081358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.918683+0000","last_change":"2023-10-16T10:04:32.415558+0000","last_active":"2023-10-16T10:04:32.918683+0000","last_peered":"2023-10-16T10:04:32.918683+0000","last_clean":"2023-10-16T10:04:32.918683+0000","last_became_active":"2023-10-16T10:04:32.415270+0000","last_became_peered":"2023-10-16T10:04:32.415270+0000","last_unstale":"2023-10-16T10:04:32.918683+0000","last_undegraded":"2023-10-16T10:04:32.918683+0000","last_fullsized":"2023-10-16T10:04:32.918683+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:03.418601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.930580+0000","last_change":"2023-10-16T10:04:32.928711+0000","last_active":"2023-10-16T10:04:32.930580+0000","last_peered":"2023-10-16T10:04:32.930580+0000","last_clean":"2023-10-16T10:04:32.930580+0000","last_became_active":"2023-10-16T10:04:32.928158+0000","last_became_peered":"2023-10-16T10:04:32.928158+0000","last_unstale":"2023-10-16T10:04:32.930580+0000","last_undegraded":"2023-10-16T10:04:32.930580+0000","last_fullsized":"2023-10-16T10:04:32.930580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:20.342702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"81'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.227090+0000","last_change":"2023-10-16T10:04:32.407006+0000","last_active":"2023-10-16T10:04:34.227090+0000","last_peered":"2023-10-16T10:04:34.227090+0000","last_clean":"2023-10-16T10:04:34.227090+0000","last_became_active":"2023-10-16T10:04:32.406703+0000","last_became_peered":"2023-10-16T10:04:32.406703+0000","last_unstale":"2023-10-16T10:04:34.227090+0000","last_undegraded":"2023-10-16T10:04:34.227090+0000","last_fullsized":"2023-10-16T10:04:34.227090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:00.019443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.930212+0000","last_change":"2023-10-16T10:04:32.927241+0000","last_active":"2023-10-16T10:04:32.930212+0000","last_peered":"2023-10-16T10:04:32.930212+0000","last_clean":"2023-10-16T10:04:32.930212+0000","last_became_active":"2023-10-16T10:04:32.925980+0000","last_became_peered":"2023-10-16T10:04:32.925980+0000","last_unstale":"2023-10-16T10:04:32.930212+0000","last_undegraded":"2023-10-16T10:04:32.930212+0000","last_fullsized":"2023-10-16T10:04:32.930212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:45:36.242711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.911438+0000","last_change":"2023-10-16T10:04:32.409196+0000","last_active":"2023-10-16T10:04:32.911438+0000","last_peered":"2023-10-16T10:04:32.911438+0000","last_clean":"2023-10-16T10:04:32.911438+0000","last_became_active":"2023-10-16T10:04:32.408877+0000","last_became_peered":"2023-10-16T10:04:32.408877+0000","last_unstale":"2023-10-16T10:04:32.911438+0000","last_undegraded":"2023-10-16T10:04:32.911438+0000","last_fullsized":"2023-10-16T10:04:32.911438+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:48:13.191783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.226361+0000","last_change":"2023-10-16T10:04:32.925826+0000","last_active":"2023-10-16T10:04:34.226361+0000","last_peered":"2023-10-16T10:04:34.226361+0000","last_clean":"2023-10-16T10:04:34.226361+0000","last_became_active":"2023-10-16T10:04:32.925552+0000","last_became_peered":"2023-10-16T10:04:32.925552+0000","last_unstale":"2023-10-16T10:04:34.226361+0000","last_undegraded":"2023-10-16T10:04:34.226361+0000","last_fullsized":"2023-10-16T10:04:34.226361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:48.096383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.907166+0000","last_change":"2023-10-16T10:04:32.406428+0000","last_active":"2023-10-16T10:04:33.907166+0000","last_peered":"2023-10-16T10:04:33.907166+0000","last_clean":"2023-10-16T10:04:33.907166+0000","last_became_active":"2023-10-16T10:04:32.405741+0000","last_became_peered":"2023-10-16T10:04:32.405741+0000","last_unstale":"2023-10-16T10:04:33.907166+0000","last_undegraded":"2023-10-16T10:04:33.907166+0000","last_fullsized":"2023-10-16T10:04:33.907166+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:57:26.493635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.928691+0000","last_change":"2023-10-16T10:04:32.926992+0000","last_active":"2023-10-16T10:04:32.928691+0000","last_peered":"2023-10-16T10:04:32.928691+0000","last_clean":"2023-10-16T10:04:32.928691+0000","last_became_active":"2023-10-16T10:04:32.923695+0000","last_became_peered":"2023-10-16T10:04:32.923695+0000","last_unstale":"2023-10-16T10:04:32.928691+0000","last_undegraded":"2023-10-16T10:04:32.928691+0000","last_fullsized":"2023-10-16T10:04:32.928691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:11.098931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.918554+0000","last_change":"2023-10-16T10:04:32.417336+0000","last_active":"2023-10-16T10:04:32.918554+0000","last_peered":"2023-10-16T10:04:32.918554+0000","last_clean":"2023-10-16T10:04:32.918554+0000","last_became_active":"2023-10-16T10:04:32.416809+0000","last_became_peered":"2023-10-16T10:04:32.416809+0000","last_unstale":"2023-10-16T10:04:32.918554+0000","last_undegraded":"2023-10-16T10:04:32.918554+0000","last_fullsized":"2023-10-16T10:04:32.918554+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:48.076788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.934822+0000","last_change":"2023-10-16T10:04:32.932133+0000","last_active":"2023-10-16T10:04:32.934822+0000","last_peered":"2023-10-16T10:04:32.934822+0000","last_clean":"2023-10-16T10:04:32.934822+0000","last_became_active":"2023-10-16T10:04:32.931892+0000","last_became_peered":"2023-10-16T10:04:32.931892+0000","last_unstale":"2023-10-16T10:04:32.934822+0000","last_undegraded":"2023-10-16T10:04:32.934822+0000","last_fullsized":"2023-10-16T10:04:32.934822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:28:14.674334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.226207+0000","last_change":"2023-10-16T10:04:32.418863+0000","last_active":"2023-10-16T10:04:34.226207+0000","last_peered":"2023-10-16T10:04:34.226207+0000","last_clean":"2023-10-16T10:04:34.226207+0000","last_became_active":"2023-10-16T10:04:32.418528+0000","last_became_peered":"2023-10-16T10:04:32.418528+0000","last_unstale":"2023-10-16T10:04:34.226207+0000","last_undegraded":"2023-10-16T10:04:34.226207+0000","last_fullsized":"2023-10-16T10:04:34.226207+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:17:57.734225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941904+0000","last_change":"2023-10-16T10:04:32.940664+0000","last_active":"2023-10-16T10:04:32.941904+0000","last_peered":"2023-10-16T10:04:32.941904+0000","last_clean":"2023-10-16T10:04:32.941904+0000","last_became_active":"2023-10-16T10:04:32.940069+0000","last_became_peered":"2023-10-16T10:04:32.940069+0000","last_unstale":"2023-10-16T10:04:32.941904+0000","last_undegraded":"2023-10-16T10:04:32.941904+0000","last_fullsized":"2023-10-16T10:04:32.941904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:56.950292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.907062+0000","last_change":"2023-10-16T10:04:32.404763+0000","last_active":"2023-10-16T10:04:33.907062+0000","last_peered":"2023-10-16T10:04:33.907062+0000","last_clean":"2023-10-16T10:04:33.907062+0000","last_became_active":"2023-10-16T10:04:32.404413+0000","last_became_peered":"2023-10-16T10:04:32.404413+0000","last_unstale":"2023-10-16T10:04:33.907062+0000","last_undegraded":"2023-10-16T10:04:33.907062+0000","last_fullsized":"2023-10-16T10:04:33.907062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:44:56.777165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.226098+0000","last_change":"2023-10-16T10:04:32.924270+0000","last_active":"2023-10-16T10:04:34.226098+0000","last_peered":"2023-10-16T10:04:34.226098+0000","last_clean":"2023-10-16T10:04:34.226098+0000","last_became_active":"2023-10-16T10:04:32.923978+0000","last_became_peered":"2023-10-16T10:04:32.923978+0000","last_unstale":"2023-10-16T10:04:34.226098+0000","last_undegraded":"2023-10-16T10:04:34.226098+0000","last_fullsized":"2023-10-16T10:04:34.226098+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:43.542318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.225973+0000","last_change":"2023-10-16T10:04:32.403952+0000","last_active":"2023-10-16T10:04:34.225973+0000","last_peered":"2023-10-16T10:04:34.225973+0000","last_clean":"2023-10-16T10:04:34.225973+0000","last_became_active":"2023-10-16T10:04:32.403632+0000","last_became_peered":"2023-10-16T10:04:32.403632+0000","last_unstale":"2023-10-16T10:04:34.225973+0000","last_undegraded":"2023-10-16T10:04:34.225973+0000","last_fullsized":"2023-10-16T10:04:34.225973+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:11:14.711977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.935647+0000","last_change":"2023-10-16T10:04:32.934407+0000","last_active":"2023-10-16T10:04:32.935647+0000","last_peered":"2023-10-16T10:04:32.935647+0000","last_clean":"2023-10-16T10:04:32.935647+0000","last_became_active":"2023-10-16T10:04:32.933719+0000","last_became_peered":"2023-10-16T10:04:32.933719+0000","last_unstale":"2023-10-16T10:04:32.935647+0000","last_undegraded":"2023-10-16T10:04:32.935647+0000","last_fullsized":"2023-10-16T10:04:32.935647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:59.516633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.920722+0000","last_change":"2023-10-16T10:04:32.416437+0000","last_active":"2023-10-16T10:04:32.920722+0000","last_peered":"2023-10-16T10:04:32.920722+0000","last_clean":"2023-10-16T10:04:32.920722+0000","last_became_active":"2023-10-16T10:04:32.416133+0000","last_became_peered":"2023-10-16T10:04:32.416133+0000","last_unstale":"2023-10-16T10:04:32.920722+0000","last_undegraded":"2023-10-16T10:04:32.920722+0000","last_fullsized":"2023-10-16T10:04:32.920722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:46.222780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.931341+0000","last_change":"2023-10-16T10:04:32.929966+0000","last_active":"2023-10-16T10:04:32.931341+0000","last_peered":"2023-10-16T10:04:32.931341+0000","last_clean":"2023-10-16T10:04:32.931341+0000","last_became_active":"2023-10-16T10:04:32.929716+0000","last_became_peered":"2023-10-16T10:04:32.929716+0000","last_unstale":"2023-10-16T10:04:32.931341+0000","last_undegraded":"2023-10-16T10:04:32.931341+0000","last_fullsized":"2023-10-16T10:04:32.931341+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:03:59.260678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.906655+0000","last_change":"2023-10-16T10:04:32.419617+0000","last_active":"2023-10-16T10:04:33.906655+0000","last_peered":"2023-10-16T10:04:33.906655+0000","last_clean":"2023-10-16T10:04:33.906655+0000","last_became_active":"2023-10-16T10:04:32.419285+0000","last_became_peered":"2023-10-16T10:04:32.419285+0000","last_unstale":"2023-10-16T10:04:33.906655+0000","last_undegraded":"2023-10-16T10:04:33.906655+0000","last_fullsized":"2023-10-16T10:04:33.906655+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:07:20.064189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"69'161","reported_seq":383,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.408331+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T10:04:32.408331+0000","last_peered":"2023-10-16T10:04:32.408331+0000","last_clean":"2023-10-16T10:04:32.408331+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T10:04:32.408331+0000","last_undegraded":"2023-10-16T10:04:32.408331+0000","last_fullsized":"2023-10-16T10:04:32.408331+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36880,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334960,"statfs":{"total":96636764160,"available":96598999040,"internally_reserved":0,"allocated":1572864,"data_stored":1056025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1639999999999999,"15min":1.167},"min":{"1min":0.65000000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.6599999999999999,"5min":1.853,"15min":1.998},"last":1.52},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1719999999999999,"15min":1.208},"min":{"1min":0.58099999999999996,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.5629999999999999,"5min":1.75,"15min":3.694},"last":1.0109999999999999}]},{"osd":1,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.222,"15min":1.2290000000000001},"min":{"1min":0.63300000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.579,"5min":1.923,"15min":1.923},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.234,"15min":1.22},"min":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5309999999999999,"5min":1.704,"15min":1.7410000000000001},"last":0.96899999999999997}]},{"osd":2,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2,"15min":1.2490000000000001},"min":{"1min":0.89000000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5820000000000001,"5min":1.837,"15min":1.885},"last":1.609},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.284,"15min":1.306},"min":{"1min":0.81499999999999995,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.7430000000000001,"5min":1.8620000000000001,"15min":3.7759999999999998},"last":1.375}]},{"osd":3,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.3120000000000001,"15min":1.3380000000000001},"min":{"1min":0.93500000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5389999999999999,"5min":1.996,"15min":1.996},"last":1.661},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.3600000000000001,"15min":1.3859999999999999},"min":{"1min":1.0509999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6659999999999999,"5min":1.8759999999999999,"15min":2.141},"last":1.2669999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.367,"15min":1.3759999999999999},"min":{"1min":1.004,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7829999999999999,"5min":2.028,"15min":2.0310000000000001},"last":1.137},{"interface":"front","average":{"1min":1.274,"5min":1.3120000000000001,"15min":1.3200000000000001},"min":{"1min":0.79200000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.55,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.571}]},{"osd":5,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4690000000000001,"15min":1.5209999999999999},"min":{"1min":1.111,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.77,"5min":2.1320000000000001,"15min":2.4510000000000001},"last":1.7},{"interface":"front","average":{"1min":1.403,"5min":1.4379999999999999,"15min":1.4410000000000001},"min":{"1min":0.92200000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6899999999999999,"5min":2.2170000000000001,"15min":2.3210000000000002},"last":1.444}]},{"osd":6,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.413,"15min":1.4159999999999999},"min":{"1min":1.054,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.659,"5min":1.946,"15min":2.6259999999999999},"last":1.218},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.6040000000000001,"15min":1.623},"min":{"1min":1.21,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.986,"5min":2.3799999999999999,"15min":2.5670000000000002},"last":1.758}]}]},{"osd":6,"up_from":13,"seq":55834575036,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":991232,"data_stored":471957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.127,"15min":1.147},"min":{"1min":0.75900000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4790000000000001,"5min":1.6910000000000001,"15min":2.04},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.236,"15min":1.246},"min":{"1min":0.98599999999999999,"5min":0.65900000000000003,"15min":0.624},"max":{"1min":1.641,"5min":1.847,"15min":2.1000000000000001},"last":1.2529999999999999}]},{"osd":1,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1240000000000001,"15min":1.1779999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.9259999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.274,"15min":1.3009999999999999},"min":{"1min":0.91900000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.395}]},{"osd":2,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.169,"15min":1.2010000000000001},"min":{"1min":0.745,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":2.036},"last":1.375},{"interface":"front","average":{"1min":1.238,"5min":1.288,"15min":1.3200000000000001},"min":{"1min":0.79900000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.925,"5min":1.925,"15min":2.1720000000000002},"last":1.335}]},{"osd":3,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.228,"15min":1.2390000000000001},"min":{"1min":0.69599999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.742,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1859999999999999,"15min":1.234},"min":{"1min":0.83399999999999996,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6020000000000001,"5min":1.6359999999999999,"15min":2.0640000000000001},"last":1.296}]},{"osd":4,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.196,"15min":1.2849999999999999},"min":{"1min":0.77800000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.794,"5min":2.1629999999999998,"15min":2.4540000000000002},"last":1.508},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4299999999999999,"15min":1.45},"min":{"1min":0.77700000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.901,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.4750000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.276,"15min":1.323},"min":{"1min":0.80400000000000005,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.603,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.544,"5min":1.482,"15min":1.5229999999999999},"min":{"1min":1.3089999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.865,"5min":2.3849999999999998,"15min":2.5289999999999999},"last":1.72}]},{"osd":7,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.456,"15min":1.4890000000000001},"min":{"1min":0.871,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.843,"5min":3.5739999999999998,"15min":3.5739999999999998},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.431,"15min":1.5},"min":{"1min":0.92600000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8340000000000001,"5min":2.4249999999999998,"15min":3.5419999999999998},"last":1.4350000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335540,"statfs":{"total":96636764160,"available":96599592960,"internally_reserved":0,"allocated":987136,"data_stored":466309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.1399999999999999,"15min":1.214},"min":{"1min":0.63100000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.486,"5min":1.675,"15min":2.1019999999999999},"last":2.3959999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.2789999999999999,"15min":1.3029999999999999},"min":{"1min":0.72399999999999998,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.724,"5min":1.8600000000000001,"15min":1.915},"last":2.4510000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.135,"15min":1.149},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.629,"5min":1.7729999999999999,"15min":2.4569999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.179,"5min":1.1279999999999999,"15min":1.1559999999999999},"min":{"1min":0.84299999999999997,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.526,"5min":1.6970000000000001,"15min":2.2410000000000001},"last":1.2410000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.157,"15min":1.163},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71299999999999997},"max":{"1min":1.675,"5min":1.7470000000000001,"15min":1.77},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.228,"15min":1.246},"min":{"1min":0.83499999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.5780000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.083}]},{"osd":3,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.288,"15min":1.2869999999999999},"min":{"1min":0.81699999999999995,"5min":0.76000000000000001,"15min":0.73799999999999999},"max":{"1min":1.8839999999999999,"5min":1.97,"15min":1.97},"last":1.167},{"interface":"front","average":{"1min":1.298,"5min":1.355,"15min":1.3660000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.623,"5min":1.7989999999999999,"15min":2.1160000000000001},"last":1.4850000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.2529999999999999,"15min":1.2949999999999999},"min":{"1min":0.79700000000000004,"5min":0.77100000000000002,"15min":0.52100000000000002},"max":{"1min":1.569,"5min":1.7989999999999999,"15min":2.355},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4990000000000001,"15min":1.5229999999999999},"min":{"1min":0.91700000000000004,"5min":0.89500000000000002,"15min":0.63600000000000001},"max":{"1min":1.8069999999999999,"5min":2.0920000000000001,"15min":2.4079999999999999},"last":1.548}]},{"osd":6,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.3740000000000001,"15min":1.379},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.55500000000000005},"max":{"1min":1.726,"5min":1.847,"15min":2.3580000000000001},"last":1.141},{"interface":"front","average":{"1min":1.375,"5min":1.3839999999999999,"15min":1.4099999999999999},"min":{"1min":1.151,"5min":0.83299999999999996,"15min":0.58399999999999996},"max":{"1min":1.6120000000000001,"5min":1.889,"15min":2.4159999999999999},"last":1.0389999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.478,"15min":1.5109999999999999},"min":{"1min":1.0669999999999999,"5min":0.88600000000000001,"15min":0.78700000000000003},"max":{"1min":1.913,"5min":2.1989999999999998,"15min":2.4809999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.575,"15min":1.5960000000000001},"min":{"1min":1.2509999999999999,"5min":0.99199999999999999,"15min":0.75600000000000001},"max":{"1min":1.851,"5min":1.9430000000000001,"15min":2.931},"last":1.3029999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46056,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325784,"statfs":{"total":96636764160,"available":96589602816,"internally_reserved":0,"allocated":1003520,"data_stored":473569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.107,"15min":1.163},"min":{"1min":0.74199999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.554,"5min":1.76,"15min":4.2640000000000002},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.3460000000000001,"15min":1.252},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.756,"5min":11.974,"15min":11.974},"last":1.571}]},{"osd":1,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.3580000000000001,"15min":1.2809999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.7549999999999999,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":1.8939999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.2130000000000001,"15min":1.24},"min":{"1min":0.67700000000000005,"5min":0.63600000000000001,"15min":0.624},"max":{"1min":1.5309999999999999,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":0.98699999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.254,"15min":1.2490000000000001},"min":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.75,"5min":1.756,"15min":2.3130000000000002},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.377,"15min":1.3799999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.615,"5min":5.0549999999999997,"15min":5.0549999999999997},"last":1.415}]},{"osd":3,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3160000000000001,"15min":1.3129999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.6739999999999999,"5min":6.8079999999999998,"15min":6.8079999999999998},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.3759999999999999,"15min":1.395},"min":{"1min":0.93100000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.885,"5min":1.923,"15min":3.3050000000000002},"last":1.6930000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.498,"15min":1.423},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9590000000000001,"5min":8.702,"15min":8.702},"last":1.7869999999999999},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.69099999999999995},"max":{"1min":1.754,"5min":3.0430000000000001,"15min":3.669},"last":1.2789999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.4810000000000001,"15min":1.4299999999999999},"min":{"1min":1.0049999999999999,"5min":0.80700000000000005,"15min":0.56599999999999995},"max":{"1min":1.9850000000000001,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":1.8600000000000001},{"interface":"front","average":{"1min":1.534,"5min":1.605,"15min":1.5509999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.70299999999999996},"max":{"1min":2.1179999999999999,"5min":6.4089999999999998,"15min":6.4089999999999998},"last":2.0459999999999998}]},{"osd":7,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.508,"15min":1.5029999999999999},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.60999999999999999},"max":{"1min":2.177,"5min":2.9399999999999999,"15min":3.5150000000000001},"last":1.629},{"interface":"front","average":{"1min":1.631,"5min":1.585,"15min":1.5860000000000001},"min":{"1min":1.1220000000000001,"5min":0.98399999999999999,"15min":0.89600000000000002},"max":{"1min":2.0539999999999998,"5min":2.6120000000000001,"15min":3.9849999999999999},"last":2.0030000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575036,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36304,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335536,"statfs":{"total":96636764160,"available":96599588864,"internally_reserved":0,"allocated":987136,"data_stored":465713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.83099999999999996,"15min":0.85999999999999999},"min":{"1min":0.60799999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4339999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.95599999999999996,"15min":1.004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.29999999999999999},"max":{"1min":1.385,"5min":1.708,"15min":1.708},"last":1.093}]},{"osd":1,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.91200000000000003,"15min":0.94299999999999995},"min":{"1min":0.42799999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.2210000000000001,"5min":1.49,"15min":1.7150000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.006,"15min":1.0509999999999999},"min":{"1min":0.52900000000000003,"5min":0.42099999999999999,"15min":0.39600000000000002},"max":{"1min":1.4079999999999999,"5min":1.603,"15min":1.643},"last":1.0509999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":1.0149999999999999,"15min":1.0580000000000001},"min":{"1min":0.46500000000000002,"5min":0.44800000000000001,"15min":0.439},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.8640000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.141,"15min":1.163},"min":{"1min":0.55100000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.534,"5min":1.776,"15min":2.028},"last":1.2250000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.24,"15min":1.276},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.5740000000000001,"5min":1.8069999999999999,"15min":2.1619999999999999},"last":1.369},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.296,"15min":1.3220000000000001},"min":{"1min":0.75600000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.6779999999999999,"5min":1.8500000000000001,"15min":2.532},"last":1.5109999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.278,"15min":1.3169999999999999},"min":{"1min":0.77800000000000002,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.5629999999999999,"5min":1.7350000000000001,"15min":2.0019999999999998},"last":1.663},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3029999999999999,"15min":1.3460000000000001},"min":{"1min":0.91200000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.5589999999999999,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":1.8080000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.375,"15min":1.429},"min":{"1min":0.72099999999999997,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6799999999999999,"5min":1.891,"15min":2.3029999999999999},"last":2.1080000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3999999999999999,"15min":1.4410000000000001},"min":{"1min":0.88300000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.619,"5min":1.976,"15min":2.1240000000000001},"last":1.998}]},{"osd":7,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3959999999999999,"15min":1.4490000000000001},"min":{"1min":0.73199999999999998,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7450000000000001,"5min":2.032,"15min":2.403},"last":1.8560000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.504,"15min":1.5640000000000001},"min":{"1min":0.68600000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8320000000000001,"5min":2.2429999999999999,"15min":2.6099999999999999},"last":2.0720000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575035,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40476,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331364,"statfs":{"total":96636764160,"available":96595316736,"internally_reserved":0,"allocated":999424,"data_stored":472013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.875,"15min":0.86699999999999999},"min":{"1min":0.499,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.506,"5min":1.5469999999999999,"15min":2.8690000000000002},"last":1.0580000000000001},{"interface":"front","average":{"1min":1,"5min":0.98099999999999998,"15min":1.028},"min":{"1min":0.52100000000000002,"5min":0.34999999999999998,"15min":0.28899999999999998},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.72499999999999998}]},{"osd":1,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.044,"15min":1.036},"min":{"1min":0.56999999999999995,"5min":0.52900000000000003,"15min":0.42899999999999999},"max":{"1min":1.5329999999999999,"5min":1.8080000000000001,"15min":2.6059999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0629999999999999,"15min":1.1140000000000001},"min":{"1min":0.622,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4770000000000001,"5min":1.792,"15min":2.6499999999999999},"last":1}]},{"osd":3,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":1.006,"15min":1.0089999999999999},"min":{"1min":0.55400000000000005,"5min":0.41399999999999998,"15min":0.38},"max":{"1min":1.3300000000000001,"5min":1.776,"15min":2.5339999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.226,"5min":1.169,"15min":1.1970000000000001},"min":{"1min":0.59799999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.7969999999999999,"5min":1.8,"15min":2.9700000000000002},"last":0.86899999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.3420000000000001,"15min":1.3480000000000001},"min":{"1min":1.083,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":2.2309999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3240000000000001,"15min":1.302},"min":{"1min":0.97899999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.2000000000000002},"last":1.151}]},{"osd":5,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3600000000000001,"15min":1.373},"min":{"1min":1.0249999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.9810000000000001,"5min":2.1640000000000001,"15min":2.7770000000000001},"last":1.26},{"interface":"front","average":{"1min":1.462,"5min":1.391,"15min":1.415},"min":{"1min":0.72699999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.885,"5min":1.956,"15min":2.952},"last":1.621}]},{"osd":6,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3400000000000001,"15min":1.3360000000000001},"min":{"1min":0.89100000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.831,"5min":2.1970000000000001,"15min":2.2469999999999999},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.4330000000000001,"15min":1.4410000000000001},"min":{"1min":1.1180000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.8140000000000001,"5min":2.044,"15min":2.8180000000000001},"last":1.353}]},{"osd":7,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.4630000000000001,"15min":1.4850000000000001},"min":{"1min":1.0109999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.012,"5min":2.0739999999999998,"15min":3.746},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.411,"15min":1.431},"min":{"1min":1.085,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.9279999999999999,"5min":2.3109999999999999,"15min":2.3769999999999998},"last":1.1850000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575035,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50196,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321644,"statfs":{"total":96636764160,"available":96585363456,"internally_reserved":0,"allocated":983040,"data_stored":465747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.83399999999999996,"15min":0.88},"min":{"1min":0.67400000000000004,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2909999999999999,"5min":1.363,"15min":2.2010000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.087,"5min":0.99099999999999999,"15min":0.998},"min":{"1min":0.66100000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.6200000000000001,"5min":1.798,"15min":2.1619999999999999},"last":1.242}]},{"osd":2,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.98199999999999998,"15min":0.995},"min":{"1min":0.60399999999999998,"5min":0.45300000000000001,"15min":0.42299999999999999},"max":{"1min":1.3700000000000001,"5min":1.526,"15min":2.3849999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2430000000000001},"min":{"1min":0.70199999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5669999999999999,"5min":1.667,"15min":2.2959999999999998},"last":1.181}]},{"osd":3,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.113,"15min":1.143},"min":{"1min":0.70199999999999996,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":2.254},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.0660000000000001,"15min":1.1060000000000001},"min":{"1min":0.83299999999999996,"5min":0.40699999999999997,"15min":0.36099999999999999},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":2.8109999999999999},"last":1.1180000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.206,"15min":1.2450000000000001},"min":{"1min":0.84099999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5189999999999999,"5min":1.7010000000000001,"15min":2.4540000000000002},"last":1.046},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3069999999999999,"15min":1.3799999999999999},"min":{"1min":1.0309999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5389999999999999,"5min":1.7789999999999999,"15min":2.851},"last":1.29}]},{"osd":5,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3129999999999999,"15min":1.345},"min":{"1min":1.155,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5820000000000001,"5min":1.7889999999999999,"15min":2.7120000000000002},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.298,"15min":1.335},"min":{"1min":1.034,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.7290000000000001,"5min":1.7869999999999999,"15min":2.512},"last":1.415}]},{"osd":6,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.385,"15min":1.4099999999999999},"min":{"1min":1.0760000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6830000000000001,"5min":1.7649999999999999,"15min":2.75},"last":1.593},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.3620000000000001,"15min":1.401},"min":{"1min":1.014,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.8029999999999999,"5min":1.851,"15min":2.669},"last":1.6870000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4339999999999999,"15min":1.494},"min":{"1min":1.1619999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.784,"5min":1.8799999999999999,"15min":2.8799999999999999},"last":1.377},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.4910000000000001,"15min":1.506},"min":{"1min":1.2589999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":3.1400000000000001},"last":1.8280000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32732,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339108,"statfs":{"total":96636764160,"available":96603246592,"internally_reserved":0,"allocated":1589248,"data_stored":1058219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.86899999999999999,"15min":0.85699999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.585,"5min":2.766,"15min":2.766},"last":1.6950000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.998,"15min":1.03},"min":{"1min":0.44900000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.5720000000000001,"5min":3.96,"15min":3.96},"last":0.85799999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0660000000000001,"15min":1.0429999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.38900000000000001},"max":{"1min":1.526,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":1.7889999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.107,"15min":1.089},"min":{"1min":0.50700000000000001,"5min":0.48999999999999999,"15min":0.46600000000000003},"max":{"1min":1.722,"5min":4.4290000000000003,"15min":4.4290000000000003},"last":1.0880000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.054,"15min":1.032},"min":{"1min":0.495,"5min":0.38,"15min":0.38},"max":{"1min":1.498,"5min":4.7380000000000004,"15min":4.7380000000000004},"last":1.024},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.24,"15min":1.202},"min":{"1min":0.47099999999999997,"5min":0.44,"15min":0.44},"max":{"1min":2.1859999999999999,"5min":5.1500000000000004,"15min":5.1500000000000004},"last":1.8759999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.341,"15min":1.3340000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.859,"5min":4.1970000000000001,"15min":4.1970000000000001},"last":1.2},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.3049999999999999,"15min":1.2909999999999999},"min":{"1min":0.77900000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.623,"5min":4.9630000000000001,"15min":4.9630000000000001},"last":1.1459999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3280000000000001,"15min":1.3220000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6799999999999999,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.4239999999999999,"15min":1.407},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.1269999999999998,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":1.9510000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.464,"15min":1.4390000000000001},"min":{"1min":0.98099999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0739999999999998,"5min":3.8250000000000002,"15min":3.8250000000000002},"last":2},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.444,"15min":1.4259999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.84999999999999998},"max":{"1min":2.02,"5min":4.0979999999999999,"15min":4.0979999999999999},"last":1.746}]},{"osd":7,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.49,"15min":1.4750000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7929999999999999,"5min":4.3369999999999997,"15min":4.3369999999999997},"last":1.8380000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4610000000000001,"15min":1.4359999999999999},"min":{"1min":0.83599999999999997,"5min":0.755,"15min":0.755},"max":{"1min":1.8169999999999999,"5min":5.0570000000000004,"15min":5.0570000000000004},"last":1.371}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:04:37.830 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:04:38.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.083+0000 7f4f6bc44640 1 Processor -- start 2023-10-16T10:04:38.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.083+0000 7f4f6bc44640 1 -- start start 2023-10-16T10:04:38.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.085+0000 7f4f6bc44640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f640cd0c0 0x7f4f641615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:38.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.085+0000 7f4f6bc44640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4f6415db00 0x7f4f640b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:38.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.085+0000 7f4f6bc44640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f640b0db0 con 0x7f4f640b0410 2023-10-16T10:04:38.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.085+0000 7f4f6bc44640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6415e3e0 con 0x7f4f6415db00 2023-10-16T10:04:38.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.085+0000 7f4f6bc44640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640cd4a0 con 0x7f4f640cd0c0 2023-10-16T10:04:38.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.086+0000 7f4f6a1e1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4f6415db00 0x7f4f640b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:38.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.086+0000 7f4f699e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f640cd0c0 0x7f4f641615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:38.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.086+0000 7f4f691df640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4f640b0410 0x7f4f6415d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:40030/0 (socket says 172.21.15.84:40030) 2023-10-16T10:04:38.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.086+0000 7f4f691df640 1 -- 172.21.15.84:0/2675653252 learned_addr learned my addr 172.21.15.84:0/2675653252 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:38.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.087+0000 7f4f699e0640 1 -- 172.21.15.84:0/2675653252 >> v2:172.21.15.157:3300/0 conn(0x7f4f6415db00 msgr2=0x7f4f640b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:38.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.087+0000 7f4f699e0640 1 --2- 172.21.15.84:0/2675653252 >> v2:172.21.15.157:3300/0 conn(0x7f4f6415db00 0x7f4f640b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.087+0000 7f4f699e0640 1 -- 172.21.15.84:0/2675653252 >> v1:172.21.15.157:6789/0 conn(0x7f4f640b0410 legacy=0x7f4f6415d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:38.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.087+0000 7f4f699e0640 1 -- 172.21.15.84:0/2675653252 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6415e100 con 0x7f4f640cd0c0 2023-10-16T10:04:38.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.087+0000 7f4f6a1e1640 1 --2- 172.21.15.84:0/2675653252 >> v2:172.21.15.157:3300/0 conn(0x7f4f6415db00 0x7f4f640b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:04:38.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.088+0000 7f4f699e0640 1 --2- 172.21.15.84:0/2675653252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f640cd0c0 0x7f4f641615f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4f600097a0 tx=0x7f4f6002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=928df30500663051 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:38.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.089+0000 7f4f689de640 1 -- 172.21.15.84:0/2675653252 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f60031780 con 0x7f4f640cd0c0 2023-10-16T10:04:38.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.089+0000 7f4f689de640 1 -- 172.21.15.84:0/2675653252 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f600318e0 con 0x7f4f640cd0c0 2023-10-16T10:04:38.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.089+0000 7f4f689de640 1 -- 172.21.15.84:0/2675653252 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f60031c70 con 0x7f4f640cd0c0 2023-10-16T10:04:38.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.089+0000 7f4f6bc44640 1 -- 172.21.15.84:0/2675653252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f640cd0c0 msgr2=0x7f4f641615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:38.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.089+0000 7f4f6bc44640 1 --2- 172.21.15.84:0/2675653252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f640cd0c0 0x7f4f641615f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4f600097a0 tx=0x7f4f6002b820 comp rx=0 tx=0).stop 2023-10-16T10:04:38.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.089+0000 7f4f6bc44640 1 -- 172.21.15.84:0/2675653252 shutdown_connections 2023-10-16T10:04:38.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.089+0000 7f4f6bc44640 1 --2- 172.21.15.84:0/2675653252 >> v2:172.21.15.157:3300/0 conn(0x7f4f6415db00 0x7f4f640b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.089+0000 7f4f6bc44640 1 --2- 172.21.15.84:0/2675653252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f640cd0c0 0x7f4f641615f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.089+0000 7f4f6bc44640 1 -- 172.21.15.84:0/2675653252 >> 172.21.15.84:0/2675653252 conn(0x7f4f640cb410 msgr2=0x7f4f640cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:38.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.090+0000 7f4f6bc44640 1 -- 172.21.15.84:0/2675653252 shutdown_connections 2023-10-16T10:04:38.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.090+0000 7f4f6bc44640 1 -- 172.21.15.84:0/2675653252 wait complete. 2023-10-16T10:04:38.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.091+0000 7f4f6bc44640 1 Processor -- start 2023-10-16T10:04:38.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.092+0000 7f4f6bc44640 1 -- start start 2023-10-16T10:04:38.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.092+0000 7f4f6bc44640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f640b0410 0x7f4f640c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:38.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.093+0000 7f4f6bc44640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f640cd0c0 0x7f4f640b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:38.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.093+0000 7f4f6bc44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f6415db00 0x7f4f64156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:38.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.093+0000 7f4f6bc44640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640c6cd0 con 0x7f4f6415db00 2023-10-16T10:04:38.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.093+0000 7f4f6bc44640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f64157150 con 0x7f4f640cd0c0 2023-10-16T10:04:38.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.093+0000 7f4f6bc44640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640c6f20 con 0x7f4f640b0410 2023-10-16T10:04:38.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.093+0000 7f4f6a1e1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f6415db00 0x7f4f64156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:38.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.093+0000 7f4f6a1e1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f6415db00 0x7f4f64156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43830/0 (socket says 172.21.15.84:43830) 2023-10-16T10:04:38.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.093+0000 7f4f6a1e1640 1 -- 172.21.15.84:0/2155204309 learned_addr learned my addr 172.21.15.84:0/2155204309 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:38.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.093+0000 7f4f699e0640 1 --2- 172.21.15.84:0/2155204309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f640b0410 0x7f4f640c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:38.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.093+0000 7f4f691df640 1 --2- 172.21.15.84:0/2155204309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f640cd0c0 0x7f4f640b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:38.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.093+0000 7f4f6a1e1640 1 -- 172.21.15.84:0/2155204309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f640b0410 msgr2=0x7f4f640c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:38.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.093+0000 7f4f6a1e1640 1 --2- 172.21.15.84:0/2155204309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f640b0410 0x7f4f640c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.093+0000 7f4f6a1e1640 1 -- 172.21.15.84:0/2155204309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f640cd0c0 msgr2=0x7f4f640b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:38.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.093+0000 7f4f6a1e1640 1 --2- 172.21.15.84:0/2155204309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f640cd0c0 0x7f4f640b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.093+0000 7f4f6a1e1640 1 -- 172.21.15.84:0/2155204309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5c009a70 con 0x7f4f6415db00 2023-10-16T10:04:38.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.094+0000 7f4f6a1e1640 1 --2- 172.21.15.84:0/2155204309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f6415db00 0x7f4f64156c10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00efc0 tx=0x7f4f5c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:38.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.094+0000 7f4f5a7fc640 1 -- 172.21.15.84:0/2155204309 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c009d70 con 0x7f4f6415db00 2023-10-16T10:04:38.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.094+0000 7f4f5a7fc640 1 -- 172.21.15.84:0/2155204309 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5c00f040 con 0x7f4f6415db00 2023-10-16T10:04:38.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.094+0000 7f4f6bc44640 1 -- 172.21.15.84:0/2155204309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f60009400 con 0x7f4f6415db00 2023-10-16T10:04:38.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.094+0000 7f4f5a7fc640 1 -- 172.21.15.84:0/2155204309 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c003990 con 0x7f4f6415db00 2023-10-16T10:04:38.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.094+0000 7f4f6bc44640 1 -- 172.21.15.84:0/2155204309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f640c74e0 con 0x7f4f6415db00 2023-10-16T10:04:38.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.095+0000 7f4f6bc44640 1 -- 172.21.15.84:0/2155204309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6404e160 con 0x7f4f6415db00 2023-10-16T10:04:38.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.096+0000 7f4f5a7fc640 1 -- 172.21.15.84:0/2155204309 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4f5c014480 con 0x7f4f6415db00 2023-10-16T10:04:38.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.097+0000 7f4f5a7fc640 1 --2- 172.21.15.84:0/2155204309 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f4f4806af90 0x7f4f4806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:38.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.097+0000 7f4f5a7fc640 1 -- 172.21.15.84:0/2155204309 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f5c08a500 con 0x7f4f6415db00 2023-10-16T10:04:38.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.097+0000 7f4f699e0640 1 --2- 172.21.15.84:0/2155204309 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f4f4806af90 0x7f4f4806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:38.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.098+0000 7f4f699e0640 1 --2- 172.21.15.84:0/2155204309 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f4f4806af90 0x7f4f4806d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4f60036000 tx=0x7f4f6003d040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:38.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.102+0000 7f4f5a7fc640 1 -- 172.21.15.84:0/2155204309 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4f5c055380 con 0x7f4f6415db00 2023-10-16T10:04:38.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.295+0000 7f4f6bc44640 1 -- 172.21.15.84:0/2155204309 --> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4f64160df0 con 0x7f4f4806af90 2023-10-16T10:04:38.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.304+0000 7f4f5a7fc640 1 -- 172.21.15.84:0/2155204309 <== mgr.6279 v2:172.21.15.84:6832/2091947430 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286760 (secure 0 0 0) 0x7f4f60031a50 con 0x7f4f4806af90 2023-10-16T10:04:38.309 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:38.311 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T10:04:38.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.310+0000 7f4f6bc44640 1 -- 172.21.15.84:0/2155204309 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f4f4806af90 msgr2=0x7f4f4806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:38.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.310+0000 7f4f6bc44640 1 --2- 172.21.15.84:0/2155204309 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f4f4806af90 0x7f4f4806d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4f60036000 tx=0x7f4f6003d040 comp rx=0 tx=0).stop 2023-10-16T10:04:38.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.310+0000 7f4f6bc44640 1 -- 172.21.15.84:0/2155204309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f6415db00 msgr2=0x7f4f64156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:38.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.310+0000 7f4f6bc44640 1 --2- 172.21.15.84:0/2155204309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f6415db00 0x7f4f64156c10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00efc0 tx=0x7f4f5c009f90 comp rx=0 tx=0).stop 2023-10-16T10:04:38.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.310+0000 7f4f6bc44640 1 -- 172.21.15.84:0/2155204309 shutdown_connections 2023-10-16T10:04:38.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.310+0000 7f4f6bc44640 1 --2- 172.21.15.84:0/2155204309 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f4f4806af90 0x7f4f4806d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.310+0000 7f4f6bc44640 1 --2- 172.21.15.84:0/2155204309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f6415db00 0x7f4f64156c10 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.310+0000 7f4f6bc44640 1 --2- 172.21.15.84:0/2155204309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f640cd0c0 0x7f4f640b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.310+0000 7f4f6bc44640 1 --2- 172.21.15.84:0/2155204309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f640b0410 0x7f4f640c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.310+0000 7f4f6bc44640 1 -- 172.21.15.84:0/2155204309 >> 172.21.15.84:0/2155204309 conn(0x7f4f640cb410 msgr2=0x7f4f640c9840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:38.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.310+0000 7f4f6bc44640 1 -- 172.21.15.84:0/2155204309 shutdown_connections 2023-10-16T10:04:38.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.310+0000 7f4f6bc44640 1 -- 172.21.15.84:0/2155204309 wait complete. 2023-10-16T10:04:38.329 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-10-16T10:04:38.258842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":315256,"kb_used_data":8900,"kb_used_omap":12,"kb_used_meta":306227,"kb_avail":754659464,"statfs":{"total":773094113280,"available":772771291136,"internally_reserved":0,"allocated":9113600,"data_stored":4929552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":313577017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002242"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.902751+0000","last_change":"2023-10-16T10:04:32.926079+0000","last_active":"2023-10-16T10:04:33.902751+0000","last_peered":"2023-10-16T10:04:33.902751+0000","last_clean":"2023-10-16T10:04:33.902751+0000","last_became_active":"2023-10-16T10:04:32.925814+0000","last_became_peered":"2023-10-16T10:04:32.925814+0000","last_unstale":"2023-10-16T10:04:33.902751+0000","last_undegraded":"2023-10-16T10:04:33.902751+0000","last_fullsized":"2023-10-16T10:04:33.902751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:00:01.019028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.935265+0000","last_change":"2023-10-16T10:04:32.932684+0000","last_active":"2023-10-16T10:04:32.935265+0000","last_peered":"2023-10-16T10:04:32.935265+0000","last_clean":"2023-10-16T10:04:32.935265+0000","last_became_active":"2023-10-16T10:04:32.932408+0000","last_became_peered":"2023-10-16T10:04:32.932408+0000","last_unstale":"2023-10-16T10:04:32.935265+0000","last_undegraded":"2023-10-16T10:04:32.935265+0000","last_fullsized":"2023-10-16T10:04:32.935265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:35:58.720210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941388+0000","last_change":"2023-10-16T10:04:32.938354+0000","last_active":"2023-10-16T10:04:32.941388+0000","last_peered":"2023-10-16T10:04:32.941388+0000","last_clean":"2023-10-16T10:04:32.941388+0000","last_became_active":"2023-10-16T10:04:32.938028+0000","last_became_peered":"2023-10-16T10:04:32.938028+0000","last_unstale":"2023-10-16T10:04:32.941388+0000","last_undegraded":"2023-10-16T10:04:32.941388+0000","last_fullsized":"2023-10-16T10:04:32.941388+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:50.758989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.931637+0000","last_change":"2023-10-16T10:04:32.930020+0000","last_active":"2023-10-16T10:04:32.931637+0000","last_peered":"2023-10-16T10:04:32.931637+0000","last_clean":"2023-10-16T10:04:32.931637+0000","last_became_active":"2023-10-16T10:04:32.929644+0000","last_became_peered":"2023-10-16T10:04:32.929644+0000","last_unstale":"2023-10-16T10:04:32.931637+0000","last_undegraded":"2023-10-16T10:04:32.931637+0000","last_fullsized":"2023-10-16T10:04:32.931637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:07.694250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.902509+0000","last_change":"2023-10-16T10:04:32.922017+0000","last_active":"2023-10-16T10:04:33.902509+0000","last_peered":"2023-10-16T10:04:33.902509+0000","last_clean":"2023-10-16T10:04:33.902509+0000","last_became_active":"2023-10-16T10:04:32.921680+0000","last_became_peered":"2023-10-16T10:04:32.921680+0000","last_unstale":"2023-10-16T10:04:33.902509+0000","last_undegraded":"2023-10-16T10:04:33.902509+0000","last_fullsized":"2023-10-16T10:04:33.902509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:23:07.310751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.923277+0000","last_change":"2023-10-16T10:04:32.914554+0000","last_active":"2023-10-16T10:04:32.923277+0000","last_peered":"2023-10-16T10:04:32.923277+0000","last_clean":"2023-10-16T10:04:32.923277+0000","last_became_active":"2023-10-16T10:04:32.914057+0000","last_became_peered":"2023-10-16T10:04:32.914057+0000","last_unstale":"2023-10-16T10:04:32.923277+0000","last_undegraded":"2023-10-16T10:04:32.923277+0000","last_fullsized":"2023-10-16T10:04:32.923277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:59:49.327036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.905455+0000","last_change":"2023-10-16T10:04:32.930264+0000","last_active":"2023-10-16T10:04:33.905455+0000","last_peered":"2023-10-16T10:04:33.905455+0000","last_clean":"2023-10-16T10:04:33.905455+0000","last_became_active":"2023-10-16T10:04:32.929196+0000","last_became_peered":"2023-10-16T10:04:32.929196+0000","last_unstale":"2023-10-16T10:04:33.905455+0000","last_undegraded":"2023-10-16T10:04:33.905455+0000","last_fullsized":"2023-10-16T10:04:33.905455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:19.713522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.942248+0000","last_change":"2023-10-16T10:04:32.936863+0000","last_active":"2023-10-16T10:04:32.942248+0000","last_peered":"2023-10-16T10:04:32.942248+0000","last_clean":"2023-10-16T10:04:32.942248+0000","last_became_active":"2023-10-16T10:04:32.936506+0000","last_became_peered":"2023-10-16T10:04:32.936506+0000","last_unstale":"2023-10-16T10:04:32.942248+0000","last_undegraded":"2023-10-16T10:04:32.942248+0000","last_fullsized":"2023-10-16T10:04:32.942248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:19:32.252156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.908082+0000","last_change":"2023-10-16T10:04:32.928380+0000","last_active":"2023-10-16T10:04:33.908082+0000","last_peered":"2023-10-16T10:04:33.908082+0000","last_clean":"2023-10-16T10:04:33.908082+0000","last_became_active":"2023-10-16T10:04:32.926155+0000","last_became_peered":"2023-10-16T10:04:32.926155+0000","last_unstale":"2023-10-16T10:04:33.908082+0000","last_undegraded":"2023-10-16T10:04:33.908082+0000","last_fullsized":"2023-10-16T10:04:33.908082+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:35.530504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.226758+0000","last_change":"2023-10-16T10:04:32.920526+0000","last_active":"2023-10-16T10:04:34.226758+0000","last_peered":"2023-10-16T10:04:34.226758+0000","last_clean":"2023-10-16T10:04:34.226758+0000","last_became_active":"2023-10-16T10:04:32.920007+0000","last_became_peered":"2023-10-16T10:04:32.920007+0000","last_unstale":"2023-10-16T10:04:34.226758+0000","last_undegraded":"2023-10-16T10:04:34.226758+0000","last_fullsized":"2023-10-16T10:04:34.226758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:07.595656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.227009+0000","last_change":"2023-10-16T10:04:32.927399+0000","last_active":"2023-10-16T10:04:34.227009+0000","last_peered":"2023-10-16T10:04:34.227009+0000","last_clean":"2023-10-16T10:04:34.227009+0000","last_became_active":"2023-10-16T10:04:32.927124+0000","last_became_peered":"2023-10-16T10:04:32.927124+0000","last_unstale":"2023-10-16T10:04:34.227009+0000","last_undegraded":"2023-10-16T10:04:34.227009+0000","last_fullsized":"2023-10-16T10:04:34.227009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:29.941247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.929331+0000","last_change":"2023-10-16T10:04:32.925077+0000","last_active":"2023-10-16T10:04:32.929331+0000","last_peered":"2023-10-16T10:04:32.929331+0000","last_clean":"2023-10-16T10:04:32.929331+0000","last_became_active":"2023-10-16T10:04:32.924807+0000","last_became_peered":"2023-10-16T10:04:32.924807+0000","last_unstale":"2023-10-16T10:04:32.929331+0000","last_undegraded":"2023-10-16T10:04:32.929331+0000","last_fullsized":"2023-10-16T10:04:32.929331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:57.061875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.907486+0000","last_change":"2023-10-16T10:04:32.921077+0000","last_active":"2023-10-16T10:04:33.907486+0000","last_peered":"2023-10-16T10:04:33.907486+0000","last_clean":"2023-10-16T10:04:33.907486+0000","last_became_active":"2023-10-16T10:04:32.920772+0000","last_became_peered":"2023-10-16T10:04:32.920772+0000","last_unstale":"2023-10-16T10:04:33.907486+0000","last_undegraded":"2023-10-16T10:04:33.907486+0000","last_fullsized":"2023-10-16T10:04:33.907486+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:25.996045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.906831+0000","last_change":"2023-10-16T10:04:32.923805+0000","last_active":"2023-10-16T10:04:33.906831+0000","last_peered":"2023-10-16T10:04:33.906831+0000","last_clean":"2023-10-16T10:04:33.906831+0000","last_became_active":"2023-10-16T10:04:32.923435+0000","last_became_peered":"2023-10-16T10:04:32.923435+0000","last_unstale":"2023-10-16T10:04:33.906831+0000","last_undegraded":"2023-10-16T10:04:33.906831+0000","last_fullsized":"2023-10-16T10:04:33.906831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:20.246873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.906133+0000","last_change":"2023-10-16T10:04:32.923191+0000","last_active":"2023-10-16T10:04:33.906133+0000","last_peered":"2023-10-16T10:04:33.906133+0000","last_clean":"2023-10-16T10:04:33.906133+0000","last_became_active":"2023-10-16T10:04:32.922896+0000","last_became_peered":"2023-10-16T10:04:32.922896+0000","last_unstale":"2023-10-16T10:04:33.906133+0000","last_undegraded":"2023-10-16T10:04:33.906133+0000","last_fullsized":"2023-10-16T10:04:33.906133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:02:38.279482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.926536+0000","last_change":"2023-10-16T10:04:32.922411+0000","last_active":"2023-10-16T10:04:32.926536+0000","last_peered":"2023-10-16T10:04:32.926536+0000","last_clean":"2023-10-16T10:04:32.926536+0000","last_became_active":"2023-10-16T10:04:32.921967+0000","last_became_peered":"2023-10-16T10:04:32.921967+0000","last_unstale":"2023-10-16T10:04:32.926536+0000","last_undegraded":"2023-10-16T10:04:32.926536+0000","last_fullsized":"2023-10-16T10:04:32.926536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:10:49.135556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.935605+0000","last_change":"2023-10-16T10:04:32.934143+0000","last_active":"2023-10-16T10:04:32.935605+0000","last_peered":"2023-10-16T10:04:32.935605+0000","last_clean":"2023-10-16T10:04:32.935605+0000","last_became_active":"2023-10-16T10:04:32.933738+0000","last_became_peered":"2023-10-16T10:04:32.933738+0000","last_unstale":"2023-10-16T10:04:32.935605+0000","last_undegraded":"2023-10-16T10:04:32.935605+0000","last_fullsized":"2023-10-16T10:04:32.935605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:01:58.526456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941444+0000","last_change":"2023-10-16T10:04:32.938667+0000","last_active":"2023-10-16T10:04:32.941444+0000","last_peered":"2023-10-16T10:04:32.941444+0000","last_clean":"2023-10-16T10:04:32.941444+0000","last_became_active":"2023-10-16T10:04:32.938104+0000","last_became_peered":"2023-10-16T10:04:32.938104+0000","last_unstale":"2023-10-16T10:04:32.941444+0000","last_undegraded":"2023-10-16T10:04:32.941444+0000","last_fullsized":"2023-10-16T10:04:32.941444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:43.061747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.902318+0000","last_change":"2023-10-16T10:04:32.923865+0000","last_active":"2023-10-16T10:04:33.902318+0000","last_peered":"2023-10-16T10:04:33.902318+0000","last_clean":"2023-10-16T10:04:33.902318+0000","last_became_active":"2023-10-16T10:04:32.923152+0000","last_became_peered":"2023-10-16T10:04:32.923152+0000","last_unstale":"2023-10-16T10:04:33.902318+0000","last_undegraded":"2023-10-16T10:04:33.902318+0000","last_fullsized":"2023-10-16T10:04:33.902318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:15:34.338966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.935632+0000","last_change":"2023-10-16T10:04:32.933489+0000","last_active":"2023-10-16T10:04:32.935632+0000","last_peered":"2023-10-16T10:04:32.935632+0000","last_clean":"2023-10-16T10:04:32.935632+0000","last_became_active":"2023-10-16T10:04:32.930967+0000","last_became_peered":"2023-10-16T10:04:32.930967+0000","last_unstale":"2023-10-16T10:04:32.935632+0000","last_undegraded":"2023-10-16T10:04:32.935632+0000","last_fullsized":"2023-10-16T10:04:32.935632+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:56.950292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941923+0000","last_change":"2023-10-16T10:04:32.941047+0000","last_active":"2023-10-16T10:04:32.941923+0000","last_peered":"2023-10-16T10:04:32.941923+0000","last_clean":"2023-10-16T10:04:32.941923+0000","last_became_active":"2023-10-16T10:04:32.939094+0000","last_became_peered":"2023-10-16T10:04:32.939094+0000","last_unstale":"2023-10-16T10:04:32.941923+0000","last_undegraded":"2023-10-16T10:04:32.941923+0000","last_fullsized":"2023-10-16T10:04:32.941923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:28:14.674334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.934786+0000","last_change":"2023-10-16T10:04:32.930517+0000","last_active":"2023-10-16T10:04:32.934786+0000","last_peered":"2023-10-16T10:04:32.934786+0000","last_clean":"2023-10-16T10:04:32.934786+0000","last_became_active":"2023-10-16T10:04:32.930282+0000","last_became_peered":"2023-10-16T10:04:32.930282+0000","last_unstale":"2023-10-16T10:04:32.934786+0000","last_undegraded":"2023-10-16T10:04:32.934786+0000","last_fullsized":"2023-10-16T10:04:32.934786+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:59:06.817328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941412+0000","last_change":"2023-10-16T10:04:32.938433+0000","last_active":"2023-10-16T10:04:32.941412+0000","last_peered":"2023-10-16T10:04:32.941412+0000","last_clean":"2023-10-16T10:04:32.941412+0000","last_became_active":"2023-10-16T10:04:32.937023+0000","last_became_peered":"2023-10-16T10:04:32.937023+0000","last_unstale":"2023-10-16T10:04:32.941412+0000","last_undegraded":"2023-10-16T10:04:32.941412+0000","last_fullsized":"2023-10-16T10:04:32.941412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:59.516633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.906260+0000","last_change":"2023-10-16T10:04:32.917152+0000","last_active":"2023-10-16T10:04:33.906260+0000","last_peered":"2023-10-16T10:04:33.906260+0000","last_clean":"2023-10-16T10:04:33.906260+0000","last_became_active":"2023-10-16T10:04:32.916260+0000","last_became_peered":"2023-10-16T10:04:32.916260+0000","last_unstale":"2023-10-16T10:04:33.906260+0000","last_undegraded":"2023-10-16T10:04:33.906260+0000","last_fullsized":"2023-10-16T10:04:33.906260+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:22:10.081358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.902812+0000","last_change":"2023-10-16T10:04:32.927618+0000","last_active":"2023-10-16T10:04:33.902812+0000","last_peered":"2023-10-16T10:04:33.902812+0000","last_clean":"2023-10-16T10:04:33.902812+0000","last_became_active":"2023-10-16T10:04:32.927364+0000","last_became_peered":"2023-10-16T10:04:32.927364+0000","last_unstale":"2023-10-16T10:04:33.902812+0000","last_undegraded":"2023-10-16T10:04:33.902812+0000","last_fullsized":"2023-10-16T10:04:33.902812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:11:58.169115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.929466+0000","last_change":"2023-10-16T10:04:32.925291+0000","last_active":"2023-10-16T10:04:32.929466+0000","last_peered":"2023-10-16T10:04:32.929466+0000","last_clean":"2023-10-16T10:04:32.929466+0000","last_became_active":"2023-10-16T10:04:32.923656+0000","last_became_peered":"2023-10-16T10:04:32.923656+0000","last_unstale":"2023-10-16T10:04:32.929466+0000","last_undegraded":"2023-10-16T10:04:32.929466+0000","last_fullsized":"2023-10-16T10:04:32.929466+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:56:24.735587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.933646+0000","last_change":"2023-10-16T10:04:32.931384+0000","last_active":"2023-10-16T10:04:32.933646+0000","last_peered":"2023-10-16T10:04:32.933646+0000","last_clean":"2023-10-16T10:04:32.933646+0000","last_became_active":"2023-10-16T10:04:32.931034+0000","last_became_peered":"2023-10-16T10:04:32.931034+0000","last_unstale":"2023-10-16T10:04:32.933646+0000","last_undegraded":"2023-10-16T10:04:32.933646+0000","last_fullsized":"2023-10-16T10:04:32.933646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:11:15.386027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.902426+0000","last_change":"2023-10-16T10:04:32.924415+0000","last_active":"2023-10-16T10:04:33.902426+0000","last_peered":"2023-10-16T10:04:33.902426+0000","last_clean":"2023-10-16T10:04:33.902426+0000","last_became_active":"2023-10-16T10:04:32.924131+0000","last_became_peered":"2023-10-16T10:04:32.924131+0000","last_unstale":"2023-10-16T10:04:33.902426+0000","last_undegraded":"2023-10-16T10:04:33.902426+0000","last_fullsized":"2023-10-16T10:04:33.902426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:48:04.380170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.936250+0000","last_change":"2023-10-16T10:04:32.932935+0000","last_active":"2023-10-16T10:04:32.936250+0000","last_peered":"2023-10-16T10:04:32.936250+0000","last_clean":"2023-10-16T10:04:32.936250+0000","last_became_active":"2023-10-16T10:04:32.932667+0000","last_became_peered":"2023-10-16T10:04:32.932667+0000","last_unstale":"2023-10-16T10:04:32.936250+0000","last_undegraded":"2023-10-16T10:04:32.936250+0000","last_fullsized":"2023-10-16T10:04:32.936250+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:55:38.814648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941500+0000","last_change":"2023-10-16T10:04:32.939264+0000","last_active":"2023-10-16T10:04:32.941500+0000","last_peered":"2023-10-16T10:04:32.941500+0000","last_clean":"2023-10-16T10:04:32.941500+0000","last_became_active":"2023-10-16T10:04:32.938951+0000","last_became_peered":"2023-10-16T10:04:32.938951+0000","last_unstale":"2023-10-16T10:04:32.941500+0000","last_undegraded":"2023-10-16T10:04:32.941500+0000","last_fullsized":"2023-10-16T10:04:32.941500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:05.860336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.902127+0000","last_change":"2023-10-16T10:04:32.918179+0000","last_active":"2023-10-16T10:04:33.902127+0000","last_peered":"2023-10-16T10:04:33.902127+0000","last_clean":"2023-10-16T10:04:33.902127+0000","last_became_active":"2023-10-16T10:04:32.917748+0000","last_became_peered":"2023-10-16T10:04:32.917748+0000","last_unstale":"2023-10-16T10:04:33.902127+0000","last_undegraded":"2023-10-16T10:04:33.902127+0000","last_fullsized":"2023-10-16T10:04:33.902127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:41:02.640911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.928626+0000","last_change":"2023-10-16T10:04:32.925916+0000","last_active":"2023-10-16T10:04:32.928626+0000","last_peered":"2023-10-16T10:04:32.928626+0000","last_clean":"2023-10-16T10:04:32.928626+0000","last_became_active":"2023-10-16T10:04:32.925628+0000","last_became_peered":"2023-10-16T10:04:32.925628+0000","last_unstale":"2023-10-16T10:04:32.928626+0000","last_undegraded":"2023-10-16T10:04:32.928626+0000","last_fullsized":"2023-10-16T10:04:32.928626+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:34.845253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.928642+0000","last_change":"2023-10-16T10:04:32.925657+0000","last_active":"2023-10-16T10:04:32.928642+0000","last_peered":"2023-10-16T10:04:32.928642+0000","last_clean":"2023-10-16T10:04:32.928642+0000","last_became_active":"2023-10-16T10:04:32.925339+0000","last_became_peered":"2023-10-16T10:04:32.925339+0000","last_unstale":"2023-10-16T10:04:32.928642+0000","last_undegraded":"2023-10-16T10:04:32.928642+0000","last_fullsized":"2023-10-16T10:04:32.928642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:17.155011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"81'4","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.415175+0000","last_change":"2023-10-16T10:04:32.417835+0000","last_active":"2023-10-16T10:04:33.415175+0000","last_peered":"2023-10-16T10:04:33.415175+0000","last_clean":"2023-10-16T10:04:33.415175+0000","last_became_active":"2023-10-16T10:04:32.417569+0000","last_became_peered":"2023-10-16T10:04:32.417569+0000","last_unstale":"2023-10-16T10:04:33.415175+0000","last_undegraded":"2023-10-16T10:04:33.415175+0000","last_fullsized":"2023-10-16T10:04:33.415175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:33.267723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.227014+0000","last_change":"2023-10-16T10:04:32.928052+0000","last_active":"2023-10-16T10:04:34.227014+0000","last_peered":"2023-10-16T10:04:34.227014+0000","last_clean":"2023-10-16T10:04:34.227014+0000","last_became_active":"2023-10-16T10:04:32.927718+0000","last_became_peered":"2023-10-16T10:04:32.927718+0000","last_unstale":"2023-10-16T10:04:34.227014+0000","last_undegraded":"2023-10-16T10:04:34.227014+0000","last_fullsized":"2023-10-16T10:04:34.227014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:05:04.283334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"81'1","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.415765+0000","last_change":"2023-10-16T10:04:32.414330+0000","last_active":"2023-10-16T10:04:33.415765+0000","last_peered":"2023-10-16T10:04:33.415765+0000","last_clean":"2023-10-16T10:04:33.415765+0000","last_became_active":"2023-10-16T10:04:32.413240+0000","last_became_peered":"2023-10-16T10:04:32.413240+0000","last_unstale":"2023-10-16T10:04:33.415765+0000","last_undegraded":"2023-10-16T10:04:33.415765+0000","last_fullsized":"2023-10-16T10:04:33.415765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:50.101733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.924402+0000","last_change":"2023-10-16T10:04:32.917028+0000","last_active":"2023-10-16T10:04:32.924402+0000","last_peered":"2023-10-16T10:04:32.924402+0000","last_clean":"2023-10-16T10:04:32.924402+0000","last_became_active":"2023-10-16T10:04:32.916590+0000","last_became_peered":"2023-10-16T10:04:32.916590+0000","last_unstale":"2023-10-16T10:04:32.924402+0000","last_undegraded":"2023-10-16T10:04:32.924402+0000","last_fullsized":"2023-10-16T10:04:32.924402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:02:44.125425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"81'2","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.409078+0000","last_change":"2023-10-16T10:04:32.410241+0000","last_active":"2023-10-16T10:04:33.409078+0000","last_peered":"2023-10-16T10:04:33.409078+0000","last_clean":"2023-10-16T10:04:33.409078+0000","last_became_active":"2023-10-16T10:04:32.409955+0000","last_became_peered":"2023-10-16T10:04:32.409955+0000","last_unstale":"2023-10-16T10:04:33.409078+0000","last_undegraded":"2023-10-16T10:04:33.409078+0000","last_fullsized":"2023-10-16T10:04:33.409078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:05.180348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.928672+0000","last_change":"2023-10-16T10:04:32.926355+0000","last_active":"2023-10-16T10:04:32.928672+0000","last_peered":"2023-10-16T10:04:32.928672+0000","last_clean":"2023-10-16T10:04:32.928672+0000","last_became_active":"2023-10-16T10:04:32.926095+0000","last_became_peered":"2023-10-16T10:04:32.926095+0000","last_unstale":"2023-10-16T10:04:32.928672+0000","last_undegraded":"2023-10-16T10:04:32.928672+0000","last_fullsized":"2023-10-16T10:04:32.928672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:25.996045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"81'2","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.416756+0000","last_change":"2023-10-16T10:04:32.419982+0000","last_active":"2023-10-16T10:04:33.416756+0000","last_peered":"2023-10-16T10:04:33.416756+0000","last_clean":"2023-10-16T10:04:33.416756+0000","last_became_active":"2023-10-16T10:04:32.419520+0000","last_became_peered":"2023-10-16T10:04:32.419520+0000","last_unstale":"2023-10-16T10:04:33.416756+0000","last_undegraded":"2023-10-16T10:04:33.416756+0000","last_fullsized":"2023-10-16T10:04:33.416756+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:16.068838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.934806+0000","last_change":"2023-10-16T10:04:32.931577+0000","last_active":"2023-10-16T10:04:32.934806+0000","last_peered":"2023-10-16T10:04:32.934806+0000","last_clean":"2023-10-16T10:04:32.934806+0000","last_became_active":"2023-10-16T10:04:32.930615+0000","last_became_peered":"2023-10-16T10:04:32.930615+0000","last_unstale":"2023-10-16T10:04:32.934806+0000","last_undegraded":"2023-10-16T10:04:32.934806+0000","last_fullsized":"2023-10-16T10:04:32.934806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:46:00.280779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"81'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.907933+0000","last_change":"2023-10-16T10:04:32.404293+0000","last_active":"2023-10-16T10:04:33.907933+0000","last_peered":"2023-10-16T10:04:33.907933+0000","last_clean":"2023-10-16T10:04:33.907933+0000","last_became_active":"2023-10-16T10:04:32.403501+0000","last_became_peered":"2023-10-16T10:04:32.403501+0000","last_unstale":"2023-10-16T10:04:33.907933+0000","last_undegraded":"2023-10-16T10:04:33.907933+0000","last_fullsized":"2023-10-16T10:04:33.907933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:12.189885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.936232+0000","last_change":"2023-10-16T10:04:32.931169+0000","last_active":"2023-10-16T10:04:32.936232+0000","last_peered":"2023-10-16T10:04:32.936232+0000","last_clean":"2023-10-16T10:04:32.936232+0000","last_became_active":"2023-10-16T10:04:32.930358+0000","last_became_peered":"2023-10-16T10:04:32.930358+0000","last_unstale":"2023-10-16T10:04:32.936232+0000","last_undegraded":"2023-10-16T10:04:32.936232+0000","last_fullsized":"2023-10-16T10:04:32.936232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:11:42.522908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"81'1","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.411412+0000","last_change":"2023-10-16T10:04:32.409472+0000","last_active":"2023-10-16T10:04:33.411412+0000","last_peered":"2023-10-16T10:04:33.411412+0000","last_clean":"2023-10-16T10:04:33.411412+0000","last_became_active":"2023-10-16T10:04:32.408232+0000","last_became_peered":"2023-10-16T10:04:32.408232+0000","last_unstale":"2023-10-16T10:04:33.411412+0000","last_undegraded":"2023-10-16T10:04:33.411412+0000","last_fullsized":"2023-10-16T10:04:33.411412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:25:18.407018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.906736+0000","last_change":"2023-10-16T10:04:32.927591+0000","last_active":"2023-10-16T10:04:33.906736+0000","last_peered":"2023-10-16T10:04:33.906736+0000","last_clean":"2023-10-16T10:04:33.906736+0000","last_became_active":"2023-10-16T10:04:32.927312+0000","last_became_peered":"2023-10-16T10:04:32.927312+0000","last_unstale":"2023-10-16T10:04:33.906736+0000","last_undegraded":"2023-10-16T10:04:33.906736+0000","last_fullsized":"2023-10-16T10:04:33.906736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:11.098931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.918197+0000","last_change":"2023-10-16T10:04:32.415160+0000","last_active":"2023-10-16T10:04:32.918197+0000","last_peered":"2023-10-16T10:04:32.918197+0000","last_clean":"2023-10-16T10:04:32.918197+0000","last_became_active":"2023-10-16T10:04:32.414824+0000","last_became_peered":"2023-10-16T10:04:32.414824+0000","last_unstale":"2023-10-16T10:04:32.918197+0000","last_undegraded":"2023-10-16T10:04:32.918197+0000","last_fullsized":"2023-10-16T10:04:32.918197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:25:04.823560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.935629+0000","last_change":"2023-10-16T10:04:32.933809+0000","last_active":"2023-10-16T10:04:32.935629+0000","last_peered":"2023-10-16T10:04:32.935629+0000","last_clean":"2023-10-16T10:04:32.935629+0000","last_became_active":"2023-10-16T10:04:32.933568+0000","last_became_peered":"2023-10-16T10:04:32.933568+0000","last_unstale":"2023-10-16T10:04:32.935629+0000","last_undegraded":"2023-10-16T10:04:32.935629+0000","last_fullsized":"2023-10-16T10:04:32.935629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:43.061747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.905372+0000","last_change":"2023-10-16T10:04:32.409147+0000","last_active":"2023-10-16T10:04:33.905372+0000","last_peered":"2023-10-16T10:04:33.905372+0000","last_clean":"2023-10-16T10:04:33.905372+0000","last_became_active":"2023-10-16T10:04:32.408629+0000","last_became_peered":"2023-10-16T10:04:32.408629+0000","last_unstale":"2023-10-16T10:04:33.905372+0000","last_undegraded":"2023-10-16T10:04:33.905372+0000","last_fullsized":"2023-10-16T10:04:33.905372+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:18.591659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.908053+0000","last_change":"2023-10-16T10:04:32.925973+0000","last_active":"2023-10-16T10:04:33.908053+0000","last_peered":"2023-10-16T10:04:33.908053+0000","last_clean":"2023-10-16T10:04:33.908053+0000","last_became_active":"2023-10-16T10:04:32.924473+0000","last_became_peered":"2023-10-16T10:04:32.924473+0000","last_unstale":"2023-10-16T10:04:33.908053+0000","last_undegraded":"2023-10-16T10:04:33.908053+0000","last_fullsized":"2023-10-16T10:04:33.908053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:05:42.992114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.933536+0000","last_change":"2023-10-16T10:04:32.418245+0000","last_active":"2023-10-16T10:04:32.933536+0000","last_peered":"2023-10-16T10:04:32.933536+0000","last_clean":"2023-10-16T10:04:32.933536+0000","last_became_active":"2023-10-16T10:04:32.417836+0000","last_became_peered":"2023-10-16T10:04:32.417836+0000","last_unstale":"2023-10-16T10:04:32.933536+0000","last_undegraded":"2023-10-16T10:04:32.933536+0000","last_fullsized":"2023-10-16T10:04:32.933536+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:03:58.243837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.943375+0000","last_change":"2023-10-16T10:04:32.942699+0000","last_active":"2023-10-16T10:04:32.943375+0000","last_peered":"2023-10-16T10:04:32.943375+0000","last_clean":"2023-10-16T10:04:32.943375+0000","last_became_active":"2023-10-16T10:04:32.942439+0000","last_became_peered":"2023-10-16T10:04:32.942439+0000","last_unstale":"2023-10-16T10:04:32.943375+0000","last_undegraded":"2023-10-16T10:04:32.943375+0000","last_fullsized":"2023-10-16T10:04:32.943375+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:59:06.817328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.925205+0000","last_change":"2023-10-16T10:04:32.411805+0000","last_active":"2023-10-16T10:04:32.925205+0000","last_peered":"2023-10-16T10:04:32.925205+0000","last_clean":"2023-10-16T10:04:32.925205+0000","last_became_active":"2023-10-16T10:04:32.410870+0000","last_became_peered":"2023-10-16T10:04:32.410870+0000","last_unstale":"2023-10-16T10:04:32.925205+0000","last_undegraded":"2023-10-16T10:04:32.925205+0000","last_fullsized":"2023-10-16T10:04:32.925205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:05:21.781941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.226433+0000","last_change":"2023-10-16T10:04:32.923001+0000","last_active":"2023-10-16T10:04:34.226433+0000","last_peered":"2023-10-16T10:04:34.226433+0000","last_clean":"2023-10-16T10:04:34.226433+0000","last_became_active":"2023-10-16T10:04:32.922735+0000","last_became_peered":"2023-10-16T10:04:32.922735+0000","last_unstale":"2023-10-16T10:04:34.226433+0000","last_undegraded":"2023-10-16T10:04:34.226433+0000","last_fullsized":"2023-10-16T10:04:34.226433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:21:02.362549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"81'3","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.415190+0000","last_change":"2023-10-16T10:04:32.418402+0000","last_active":"2023-10-16T10:04:33.415190+0000","last_peered":"2023-10-16T10:04:33.415190+0000","last_clean":"2023-10-16T10:04:33.415190+0000","last_became_active":"2023-10-16T10:04:32.417878+0000","last_became_peered":"2023-10-16T10:04:32.417878+0000","last_unstale":"2023-10-16T10:04:33.415190+0000","last_undegraded":"2023-10-16T10:04:33.415190+0000","last_fullsized":"2023-10-16T10:04:33.415190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:19:31.235315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941214+0000","last_change":"2023-10-16T10:04:32.936505+0000","last_active":"2023-10-16T10:04:32.941214+0000","last_peered":"2023-10-16T10:04:32.941214+0000","last_clean":"2023-10-16T10:04:32.941214+0000","last_became_active":"2023-10-16T10:04:32.936228+0000","last_became_peered":"2023-10-16T10:04:32.936228+0000","last_unstale":"2023-10-16T10:04:32.941214+0000","last_undegraded":"2023-10-16T10:04:32.941214+0000","last_fullsized":"2023-10-16T10:04:32.941214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:35:58.720210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"81'6","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.908172+0000","last_change":"2023-10-16T10:04:32.411084+0000","last_active":"2023-10-16T10:04:33.908172+0000","last_peered":"2023-10-16T10:04:33.908172+0000","last_clean":"2023-10-16T10:04:33.908172+0000","last_became_active":"2023-10-16T10:04:32.407041+0000","last_became_peered":"2023-10-16T10:04:32.407041+0000","last_unstale":"2023-10-16T10:04:33.908172+0000","last_undegraded":"2023-10-16T10:04:33.908172+0000","last_fullsized":"2023-10-16T10:04:33.908172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:33:12.451753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.930036+0000","last_change":"2023-10-16T10:04:32.926391+0000","last_active":"2023-10-16T10:04:32.930036+0000","last_peered":"2023-10-16T10:04:32.930036+0000","last_clean":"2023-10-16T10:04:32.930036+0000","last_became_active":"2023-10-16T10:04:32.925637+0000","last_became_peered":"2023-10-16T10:04:32.925637+0000","last_unstale":"2023-10-16T10:04:32.930036+0000","last_undegraded":"2023-10-16T10:04:32.930036+0000","last_fullsized":"2023-10-16T10:04:32.930036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:17:08.069358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"81'2","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.407868+0000","last_change":"2023-10-16T10:04:32.410283+0000","last_active":"2023-10-16T10:04:33.407868+0000","last_peered":"2023-10-16T10:04:33.407868+0000","last_clean":"2023-10-16T10:04:33.407868+0000","last_became_active":"2023-10-16T10:04:32.409976+0000","last_became_peered":"2023-10-16T10:04:32.409976+0000","last_unstale":"2023-10-16T10:04:33.407868+0000","last_undegraded":"2023-10-16T10:04:33.407868+0000","last_fullsized":"2023-10-16T10:04:33.407868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:57:16.836538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941493+0000","last_change":"2023-10-16T10:04:32.938998+0000","last_active":"2023-10-16T10:04:32.941493+0000","last_peered":"2023-10-16T10:04:32.941493+0000","last_clean":"2023-10-16T10:04:32.941493+0000","last_became_active":"2023-10-16T10:04:32.938683+0000","last_became_peered":"2023-10-16T10:04:32.938683+0000","last_unstale":"2023-10-16T10:04:32.941493+0000","last_undegraded":"2023-10-16T10:04:32.941493+0000","last_fullsized":"2023-10-16T10:04:32.941493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:11:42.522908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.906205+0000","last_change":"2023-10-16T10:04:32.411769+0000","last_active":"2023-10-16T10:04:33.906205+0000","last_peered":"2023-10-16T10:04:33.906205+0000","last_clean":"2023-10-16T10:04:33.906205+0000","last_became_active":"2023-10-16T10:04:32.411479+0000","last_became_peered":"2023-10-16T10:04:32.411479+0000","last_unstale":"2023-10-16T10:04:33.906205+0000","last_undegraded":"2023-10-16T10:04:33.906205+0000","last_fullsized":"2023-10-16T10:04:33.906205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:33.828412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.905287+0000","last_change":"2023-10-16T10:04:32.929535+0000","last_active":"2023-10-16T10:04:33.905287+0000","last_peered":"2023-10-16T10:04:33.905287+0000","last_clean":"2023-10-16T10:04:33.905287+0000","last_became_active":"2023-10-16T10:04:32.928852+0000","last_became_peered":"2023-10-16T10:04:32.928852+0000","last_unstale":"2023-10-16T10:04:33.905287+0000","last_undegraded":"2023-10-16T10:04:33.905287+0000","last_fullsized":"2023-10-16T10:04:33.905287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:46:15.616966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.918283+0000","last_change":"2023-10-16T10:04:32.413859+0000","last_active":"2023-10-16T10:04:32.918283+0000","last_peered":"2023-10-16T10:04:32.918283+0000","last_clean":"2023-10-16T10:04:32.918283+0000","last_became_active":"2023-10-16T10:04:32.413589+0000","last_became_peered":"2023-10-16T10:04:32.413589+0000","last_unstale":"2023-10-16T10:04:32.918283+0000","last_undegraded":"2023-10-16T10:04:32.918283+0000","last_fullsized":"2023-10-16T10:04:32.918283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:18.489976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941324+0000","last_change":"2023-10-16T10:04:32.937634+0000","last_active":"2023-10-16T10:04:32.941324+0000","last_peered":"2023-10-16T10:04:32.941324+0000","last_clean":"2023-10-16T10:04:32.941324+0000","last_became_active":"2023-10-16T10:04:32.936778+0000","last_became_peered":"2023-10-16T10:04:32.936778+0000","last_unstale":"2023-10-16T10:04:32.941324+0000","last_undegraded":"2023-10-16T10:04:32.941324+0000","last_fullsized":"2023-10-16T10:04:32.941324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:55:38.814648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"81'8","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.418933+0000","last_change":"2023-10-16T10:04:32.418238+0000","last_active":"2023-10-16T10:04:33.418933+0000","last_peered":"2023-10-16T10:04:33.418933+0000","last_clean":"2023-10-16T10:04:33.418933+0000","last_became_active":"2023-10-16T10:04:32.417949+0000","last_became_peered":"2023-10-16T10:04:32.417949+0000","last_unstale":"2023-10-16T10:04:33.418933+0000","last_undegraded":"2023-10-16T10:04:33.418933+0000","last_fullsized":"2023-10-16T10:04:33.418933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:44.953428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941524+0000","last_change":"2023-10-16T10:04:32.939737+0000","last_active":"2023-10-16T10:04:32.941524+0000","last_peered":"2023-10-16T10:04:32.941524+0000","last_clean":"2023-10-16T10:04:32.941524+0000","last_became_active":"2023-10-16T10:04:32.939017+0000","last_became_peered":"2023-10-16T10:04:32.939017+0000","last_unstale":"2023-10-16T10:04:32.941524+0000","last_undegraded":"2023-10-16T10:04:32.941524+0000","last_fullsized":"2023-10-16T10:04:32.941524+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:46:00.280779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"81'1","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.407782+0000","last_change":"2023-10-16T10:04:32.412234+0000","last_active":"2023-10-16T10:04:33.407782+0000","last_peered":"2023-10-16T10:04:33.407782+0000","last_clean":"2023-10-16T10:04:33.407782+0000","last_became_active":"2023-10-16T10:04:32.411416+0000","last_became_peered":"2023-10-16T10:04:32.411416+0000","last_unstale":"2023-10-16T10:04:33.407782+0000","last_undegraded":"2023-10-16T10:04:33.407782+0000","last_fullsized":"2023-10-16T10:04:33.407782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:44:48.305279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.929424+0000","last_change":"2023-10-16T10:04:32.925656+0000","last_active":"2023-10-16T10:04:32.929424+0000","last_peered":"2023-10-16T10:04:32.929424+0000","last_clean":"2023-10-16T10:04:32.929424+0000","last_became_active":"2023-10-16T10:04:32.925377+0000","last_became_peered":"2023-10-16T10:04:32.925377+0000","last_unstale":"2023-10-16T10:04:32.929424+0000","last_undegraded":"2023-10-16T10:04:32.929424+0000","last_fullsized":"2023-10-16T10:04:32.929424+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:30:33.124497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"81'2","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.409509+0000","last_change":"2023-10-16T10:04:32.407759+0000","last_active":"2023-10-16T10:04:33.409509+0000","last_peered":"2023-10-16T10:04:33.409509+0000","last_clean":"2023-10-16T10:04:33.409509+0000","last_became_active":"2023-10-16T10:04:32.407431+0000","last_became_peered":"2023-10-16T10:04:32.407431+0000","last_unstale":"2023-10-16T10:04:33.409509+0000","last_undegraded":"2023-10-16T10:04:33.409509+0000","last_fullsized":"2023-10-16T10:04:33.409509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:28:55.400277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.931274+0000","last_change":"2023-10-16T10:04:32.928011+0000","last_active":"2023-10-16T10:04:32.931274+0000","last_peered":"2023-10-16T10:04:32.931274+0000","last_clean":"2023-10-16T10:04:32.931274+0000","last_became_active":"2023-10-16T10:04:32.927717+0000","last_became_peered":"2023-10-16T10:04:32.927717+0000","last_unstale":"2023-10-16T10:04:32.931274+0000","last_undegraded":"2023-10-16T10:04:32.931274+0000","last_fullsized":"2023-10-16T10:04:32.931274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:50.758989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"81'1","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.416739+0000","last_change":"2023-10-16T10:04:32.417048+0000","last_active":"2023-10-16T10:04:33.416739+0000","last_peered":"2023-10-16T10:04:33.416739+0000","last_clean":"2023-10-16T10:04:33.416739+0000","last_became_active":"2023-10-16T10:04:32.416257+0000","last_became_peered":"2023-10-16T10:04:32.416257+0000","last_unstale":"2023-10-16T10:04:33.416739+0000","last_undegraded":"2023-10-16T10:04:33.416739+0000","last_fullsized":"2023-10-16T10:04:33.416739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:09:24.405679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.943382+0000","last_change":"2023-10-16T10:04:32.942704+0000","last_active":"2023-10-16T10:04:32.943382+0000","last_peered":"2023-10-16T10:04:32.943382+0000","last_clean":"2023-10-16T10:04:32.943382+0000","last_became_active":"2023-10-16T10:04:32.942449+0000","last_became_peered":"2023-10-16T10:04:32.942449+0000","last_unstale":"2023-10-16T10:04:32.943382+0000","last_undegraded":"2023-10-16T10:04:32.943382+0000","last_fullsized":"2023-10-16T10:04:32.943382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:01:58.526456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.907757+0000","last_change":"2023-10-16T10:04:32.409334+0000","last_active":"2023-10-16T10:04:32.907757+0000","last_peered":"2023-10-16T10:04:32.907757+0000","last_clean":"2023-10-16T10:04:32.907757+0000","last_became_active":"2023-10-16T10:04:32.409063+0000","last_became_peered":"2023-10-16T10:04:32.409063+0000","last_unstale":"2023-10-16T10:04:32.907757+0000","last_undegraded":"2023-10-16T10:04:32.907757+0000","last_fullsized":"2023-10-16T10:04:32.907757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:05:13.298000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.930129+0000","last_change":"2023-10-16T10:04:32.926560+0000","last_active":"2023-10-16T10:04:32.930129+0000","last_peered":"2023-10-16T10:04:32.930129+0000","last_clean":"2023-10-16T10:04:32.930129+0000","last_became_active":"2023-10-16T10:04:32.926069+0000","last_became_peered":"2023-10-16T10:04:32.926069+0000","last_unstale":"2023-10-16T10:04:32.930129+0000","last_undegraded":"2023-10-16T10:04:32.930129+0000","last_fullsized":"2023-10-16T10:04:32.930129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:06:33.097982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"81'2","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.411018+0000","last_change":"2023-10-16T10:04:32.405741+0000","last_active":"2023-10-16T10:04:33.411018+0000","last_peered":"2023-10-16T10:04:33.411018+0000","last_clean":"2023-10-16T10:04:33.411018+0000","last_became_active":"2023-10-16T10:04:32.405464+0000","last_became_peered":"2023-10-16T10:04:32.405464+0000","last_unstale":"2023-10-16T10:04:33.411018+0000","last_undegraded":"2023-10-16T10:04:33.411018+0000","last_fullsized":"2023-10-16T10:04:33.411018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:23:17.597276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.926585+0000","last_change":"2023-10-16T10:04:32.921140+0000","last_active":"2023-10-16T10:04:32.926585+0000","last_peered":"2023-10-16T10:04:32.926585+0000","last_clean":"2023-10-16T10:04:32.926585+0000","last_became_active":"2023-10-16T10:04:32.919995+0000","last_became_peered":"2023-10-16T10:04:32.919995+0000","last_unstale":"2023-10-16T10:04:32.926585+0000","last_undegraded":"2023-10-16T10:04:32.926585+0000","last_fullsized":"2023-10-16T10:04:32.926585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:30.945110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.226305+0000","last_change":"2023-10-16T10:04:32.397651+0000","last_active":"2023-10-16T10:04:34.226305+0000","last_peered":"2023-10-16T10:04:34.226305+0000","last_clean":"2023-10-16T10:04:34.226305+0000","last_became_active":"2023-10-16T10:04:32.397339+0000","last_became_peered":"2023-10-16T10:04:32.397339+0000","last_unstale":"2023-10-16T10:04:34.226305+0000","last_undegraded":"2023-10-16T10:04:34.226305+0000","last_fullsized":"2023-10-16T10:04:34.226305+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:45.561197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.928725+0000","last_change":"2023-10-16T10:04:32.926226+0000","last_active":"2023-10-16T10:04:32.928725+0000","last_peered":"2023-10-16T10:04:32.928725+0000","last_clean":"2023-10-16T10:04:32.928725+0000","last_became_active":"2023-10-16T10:04:32.925300+0000","last_became_peered":"2023-10-16T10:04:32.925300+0000","last_unstale":"2023-10-16T10:04:32.928725+0000","last_undegraded":"2023-10-16T10:04:32.928725+0000","last_fullsized":"2023-10-16T10:04:32.928725+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:22:10.081358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.918683+0000","last_change":"2023-10-16T10:04:32.415558+0000","last_active":"2023-10-16T10:04:32.918683+0000","last_peered":"2023-10-16T10:04:32.918683+0000","last_clean":"2023-10-16T10:04:32.918683+0000","last_became_active":"2023-10-16T10:04:32.415270+0000","last_became_peered":"2023-10-16T10:04:32.415270+0000","last_unstale":"2023-10-16T10:04:32.918683+0000","last_undegraded":"2023-10-16T10:04:32.918683+0000","last_fullsized":"2023-10-16T10:04:32.918683+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:03.418601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.930580+0000","last_change":"2023-10-16T10:04:32.928711+0000","last_active":"2023-10-16T10:04:32.930580+0000","last_peered":"2023-10-16T10:04:32.930580+0000","last_clean":"2023-10-16T10:04:32.930580+0000","last_became_active":"2023-10-16T10:04:32.928158+0000","last_became_peered":"2023-10-16T10:04:32.928158+0000","last_unstale":"2023-10-16T10:04:32.930580+0000","last_undegraded":"2023-10-16T10:04:32.930580+0000","last_fullsized":"2023-10-16T10:04:32.930580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:20.342702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"81'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.227090+0000","last_change":"2023-10-16T10:04:32.407006+0000","last_active":"2023-10-16T10:04:34.227090+0000","last_peered":"2023-10-16T10:04:34.227090+0000","last_clean":"2023-10-16T10:04:34.227090+0000","last_became_active":"2023-10-16T10:04:32.406703+0000","last_became_peered":"2023-10-16T10:04:32.406703+0000","last_unstale":"2023-10-16T10:04:34.227090+0000","last_undegraded":"2023-10-16T10:04:34.227090+0000","last_fullsized":"2023-10-16T10:04:34.227090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:00.019443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.930212+0000","last_change":"2023-10-16T10:04:32.927241+0000","last_active":"2023-10-16T10:04:32.930212+0000","last_peered":"2023-10-16T10:04:32.930212+0000","last_clean":"2023-10-16T10:04:32.930212+0000","last_became_active":"2023-10-16T10:04:32.925980+0000","last_became_peered":"2023-10-16T10:04:32.925980+0000","last_unstale":"2023-10-16T10:04:32.930212+0000","last_undegraded":"2023-10-16T10:04:32.930212+0000","last_fullsized":"2023-10-16T10:04:32.930212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:45:36.242711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.911438+0000","last_change":"2023-10-16T10:04:32.409196+0000","last_active":"2023-10-16T10:04:32.911438+0000","last_peered":"2023-10-16T10:04:32.911438+0000","last_clean":"2023-10-16T10:04:32.911438+0000","last_became_active":"2023-10-16T10:04:32.408877+0000","last_became_peered":"2023-10-16T10:04:32.408877+0000","last_unstale":"2023-10-16T10:04:32.911438+0000","last_undegraded":"2023-10-16T10:04:32.911438+0000","last_fullsized":"2023-10-16T10:04:32.911438+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:48:13.191783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.226361+0000","last_change":"2023-10-16T10:04:32.925826+0000","last_active":"2023-10-16T10:04:34.226361+0000","last_peered":"2023-10-16T10:04:34.226361+0000","last_clean":"2023-10-16T10:04:34.226361+0000","last_became_active":"2023-10-16T10:04:32.925552+0000","last_became_peered":"2023-10-16T10:04:32.925552+0000","last_unstale":"2023-10-16T10:04:34.226361+0000","last_undegraded":"2023-10-16T10:04:34.226361+0000","last_fullsized":"2023-10-16T10:04:34.226361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:48.096383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.907166+0000","last_change":"2023-10-16T10:04:32.406428+0000","last_active":"2023-10-16T10:04:33.907166+0000","last_peered":"2023-10-16T10:04:33.907166+0000","last_clean":"2023-10-16T10:04:33.907166+0000","last_became_active":"2023-10-16T10:04:32.405741+0000","last_became_peered":"2023-10-16T10:04:32.405741+0000","last_unstale":"2023-10-16T10:04:33.907166+0000","last_undegraded":"2023-10-16T10:04:33.907166+0000","last_fullsized":"2023-10-16T10:04:33.907166+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:57:26.493635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.928691+0000","last_change":"2023-10-16T10:04:32.926992+0000","last_active":"2023-10-16T10:04:32.928691+0000","last_peered":"2023-10-16T10:04:32.928691+0000","last_clean":"2023-10-16T10:04:32.928691+0000","last_became_active":"2023-10-16T10:04:32.923695+0000","last_became_peered":"2023-10-16T10:04:32.923695+0000","last_unstale":"2023-10-16T10:04:32.928691+0000","last_undegraded":"2023-10-16T10:04:32.928691+0000","last_fullsized":"2023-10-16T10:04:32.928691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:11.098931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.918554+0000","last_change":"2023-10-16T10:04:32.417336+0000","last_active":"2023-10-16T10:04:32.918554+0000","last_peered":"2023-10-16T10:04:32.918554+0000","last_clean":"2023-10-16T10:04:32.918554+0000","last_became_active":"2023-10-16T10:04:32.416809+0000","last_became_peered":"2023-10-16T10:04:32.416809+0000","last_unstale":"2023-10-16T10:04:32.918554+0000","last_undegraded":"2023-10-16T10:04:32.918554+0000","last_fullsized":"2023-10-16T10:04:32.918554+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:48.076788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.934822+0000","last_change":"2023-10-16T10:04:32.932133+0000","last_active":"2023-10-16T10:04:32.934822+0000","last_peered":"2023-10-16T10:04:32.934822+0000","last_clean":"2023-10-16T10:04:32.934822+0000","last_became_active":"2023-10-16T10:04:32.931892+0000","last_became_peered":"2023-10-16T10:04:32.931892+0000","last_unstale":"2023-10-16T10:04:32.934822+0000","last_undegraded":"2023-10-16T10:04:32.934822+0000","last_fullsized":"2023-10-16T10:04:32.934822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:28:14.674334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.226207+0000","last_change":"2023-10-16T10:04:32.418863+0000","last_active":"2023-10-16T10:04:34.226207+0000","last_peered":"2023-10-16T10:04:34.226207+0000","last_clean":"2023-10-16T10:04:34.226207+0000","last_became_active":"2023-10-16T10:04:32.418528+0000","last_became_peered":"2023-10-16T10:04:32.418528+0000","last_unstale":"2023-10-16T10:04:34.226207+0000","last_undegraded":"2023-10-16T10:04:34.226207+0000","last_fullsized":"2023-10-16T10:04:34.226207+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:17:57.734225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.941904+0000","last_change":"2023-10-16T10:04:32.940664+0000","last_active":"2023-10-16T10:04:32.941904+0000","last_peered":"2023-10-16T10:04:32.941904+0000","last_clean":"2023-10-16T10:04:32.941904+0000","last_became_active":"2023-10-16T10:04:32.940069+0000","last_became_peered":"2023-10-16T10:04:32.940069+0000","last_unstale":"2023-10-16T10:04:32.941904+0000","last_undegraded":"2023-10-16T10:04:32.941904+0000","last_fullsized":"2023-10-16T10:04:32.941904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:56.950292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.907062+0000","last_change":"2023-10-16T10:04:32.404763+0000","last_active":"2023-10-16T10:04:33.907062+0000","last_peered":"2023-10-16T10:04:33.907062+0000","last_clean":"2023-10-16T10:04:33.907062+0000","last_became_active":"2023-10-16T10:04:32.404413+0000","last_became_peered":"2023-10-16T10:04:32.404413+0000","last_unstale":"2023-10-16T10:04:33.907062+0000","last_undegraded":"2023-10-16T10:04:33.907062+0000","last_fullsized":"2023-10-16T10:04:33.907062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:44:56.777165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.226098+0000","last_change":"2023-10-16T10:04:32.924270+0000","last_active":"2023-10-16T10:04:34.226098+0000","last_peered":"2023-10-16T10:04:34.226098+0000","last_clean":"2023-10-16T10:04:34.226098+0000","last_became_active":"2023-10-16T10:04:32.923978+0000","last_became_peered":"2023-10-16T10:04:32.923978+0000","last_unstale":"2023-10-16T10:04:34.226098+0000","last_undegraded":"2023-10-16T10:04:34.226098+0000","last_fullsized":"2023-10-16T10:04:34.226098+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:43.542318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:34.225973+0000","last_change":"2023-10-16T10:04:32.403952+0000","last_active":"2023-10-16T10:04:34.225973+0000","last_peered":"2023-10-16T10:04:34.225973+0000","last_clean":"2023-10-16T10:04:34.225973+0000","last_became_active":"2023-10-16T10:04:32.403632+0000","last_became_peered":"2023-10-16T10:04:32.403632+0000","last_unstale":"2023-10-16T10:04:34.225973+0000","last_undegraded":"2023-10-16T10:04:34.225973+0000","last_fullsized":"2023-10-16T10:04:34.225973+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:11:14.711977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.935647+0000","last_change":"2023-10-16T10:04:32.934407+0000","last_active":"2023-10-16T10:04:32.935647+0000","last_peered":"2023-10-16T10:04:32.935647+0000","last_clean":"2023-10-16T10:04:32.935647+0000","last_became_active":"2023-10-16T10:04:32.933719+0000","last_became_peered":"2023-10-16T10:04:32.933719+0000","last_unstale":"2023-10-16T10:04:32.935647+0000","last_undegraded":"2023-10-16T10:04:32.935647+0000","last_fullsized":"2023-10-16T10:04:32.935647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:59.516633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.920722+0000","last_change":"2023-10-16T10:04:32.416437+0000","last_active":"2023-10-16T10:04:32.920722+0000","last_peered":"2023-10-16T10:04:32.920722+0000","last_clean":"2023-10-16T10:04:32.920722+0000","last_became_active":"2023-10-16T10:04:32.416133+0000","last_became_peered":"2023-10-16T10:04:32.416133+0000","last_unstale":"2023-10-16T10:04:32.920722+0000","last_undegraded":"2023-10-16T10:04:32.920722+0000","last_fullsized":"2023-10-16T10:04:32.920722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:46.222780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.931341+0000","last_change":"2023-10-16T10:04:32.929966+0000","last_active":"2023-10-16T10:04:32.931341+0000","last_peered":"2023-10-16T10:04:32.931341+0000","last_clean":"2023-10-16T10:04:32.931341+0000","last_became_active":"2023-10-16T10:04:32.929716+0000","last_became_peered":"2023-10-16T10:04:32.929716+0000","last_unstale":"2023-10-16T10:04:32.931341+0000","last_undegraded":"2023-10-16T10:04:32.931341+0000","last_fullsized":"2023-10-16T10:04:32.931341+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:32.363759+0000","last_clean_scrub_stamp":"2023-10-16T10:04:32.363759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:03:59.260678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-16T10:04:33.906655+0000","last_change":"2023-10-16T10:04:32.419617+0000","last_active":"2023-10-16T10:04:33.906655+0000","last_peered":"2023-10-16T10:04:33.906655+0000","last_clean":"2023-10-16T10:04:33.906655+0000","last_became_active":"2023-10-16T10:04:32.419285+0000","last_became_peered":"2023-10-16T10:04:32.419285+0000","last_unstale":"2023-10-16T10:04:33.906655+0000","last_undegraded":"2023-10-16T10:04:33.906655+0000","last_fullsized":"2023-10-16T10:04:33.906655+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:31.346918+0000","last_clean_scrub_stamp":"2023-10-16T10:04:31.346918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:07:20.064189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"69'161","reported_seq":383,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-10-16T10:04:32.408331+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T10:04:32.408331+0000","last_peered":"2023-10-16T10:04:32.408331+0000","last_clean":"2023-10-16T10:04:32.408331+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T10:04:32.408331+0000","last_undegraded":"2023-10-16T10:04:32.408331+0000","last_fullsized":"2023-10-16T10:04:32.408331+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36880,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334960,"statfs":{"total":96636764160,"available":96598999040,"internally_reserved":0,"allocated":1572864,"data_stored":1056025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1639999999999999,"15min":1.167},"min":{"1min":0.65000000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.6599999999999999,"5min":1.853,"15min":1.998},"last":1.52},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1719999999999999,"15min":1.208},"min":{"1min":0.58099999999999996,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.5629999999999999,"5min":1.75,"15min":3.694},"last":1.0109999999999999}]},{"osd":1,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.222,"15min":1.2290000000000001},"min":{"1min":0.63300000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.579,"5min":1.923,"15min":1.923},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.234,"15min":1.22},"min":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5309999999999999,"5min":1.704,"15min":1.7410000000000001},"last":0.96899999999999997}]},{"osd":2,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2,"15min":1.2490000000000001},"min":{"1min":0.89000000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5820000000000001,"5min":1.837,"15min":1.885},"last":1.609},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.284,"15min":1.306},"min":{"1min":0.81499999999999995,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.7430000000000001,"5min":1.8620000000000001,"15min":3.7759999999999998},"last":1.375}]},{"osd":3,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.3120000000000001,"15min":1.3380000000000001},"min":{"1min":0.93500000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5389999999999999,"5min":1.996,"15min":1.996},"last":1.661},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.3600000000000001,"15min":1.3859999999999999},"min":{"1min":1.0509999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6659999999999999,"5min":1.8759999999999999,"15min":2.141},"last":1.2669999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.367,"15min":1.3759999999999999},"min":{"1min":1.004,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7829999999999999,"5min":2.028,"15min":2.0310000000000001},"last":1.137},{"interface":"front","average":{"1min":1.274,"5min":1.3120000000000001,"15min":1.3200000000000001},"min":{"1min":0.79200000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.55,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.571}]},{"osd":5,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4690000000000001,"15min":1.5209999999999999},"min":{"1min":1.111,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.77,"5min":2.1320000000000001,"15min":2.4510000000000001},"last":1.7},{"interface":"front","average":{"1min":1.403,"5min":1.4379999999999999,"15min":1.4410000000000001},"min":{"1min":0.92200000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6899999999999999,"5min":2.2170000000000001,"15min":2.3210000000000002},"last":1.444}]},{"osd":6,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.413,"15min":1.4159999999999999},"min":{"1min":1.054,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.659,"5min":1.946,"15min":2.6259999999999999},"last":1.218},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.6040000000000001,"15min":1.623},"min":{"1min":1.21,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.986,"5min":2.3799999999999999,"15min":2.5670000000000002},"last":1.758}]}]},{"osd":6,"up_from":13,"seq":55834575036,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":991232,"data_stored":471957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.127,"15min":1.147},"min":{"1min":0.75900000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4790000000000001,"5min":1.6910000000000001,"15min":2.04},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.236,"15min":1.246},"min":{"1min":0.98599999999999999,"5min":0.65900000000000003,"15min":0.624},"max":{"1min":1.641,"5min":1.847,"15min":2.1000000000000001},"last":1.2529999999999999}]},{"osd":1,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1240000000000001,"15min":1.1779999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.9259999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.274,"15min":1.3009999999999999},"min":{"1min":0.91900000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.395}]},{"osd":2,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.169,"15min":1.2010000000000001},"min":{"1min":0.745,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":2.036},"last":1.375},{"interface":"front","average":{"1min":1.238,"5min":1.288,"15min":1.3200000000000001},"min":{"1min":0.79900000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.925,"5min":1.925,"15min":2.1720000000000002},"last":1.335}]},{"osd":3,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.228,"15min":1.2390000000000001},"min":{"1min":0.69599999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.742,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1859999999999999,"15min":1.234},"min":{"1min":0.83399999999999996,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6020000000000001,"5min":1.6359999999999999,"15min":2.0640000000000001},"last":1.296}]},{"osd":4,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.196,"15min":1.2849999999999999},"min":{"1min":0.77800000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.794,"5min":2.1629999999999998,"15min":2.4540000000000002},"last":1.508},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4299999999999999,"15min":1.45},"min":{"1min":0.77700000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.901,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.4750000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.276,"15min":1.323},"min":{"1min":0.80400000000000005,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.603,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.544,"5min":1.482,"15min":1.5229999999999999},"min":{"1min":1.3089999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.865,"5min":2.3849999999999998,"15min":2.5289999999999999},"last":1.72}]},{"osd":7,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.456,"15min":1.4890000000000001},"min":{"1min":0.871,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.843,"5min":3.5739999999999998,"15min":3.5739999999999998},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.431,"15min":1.5},"min":{"1min":0.92600000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8340000000000001,"5min":2.4249999999999998,"15min":3.5419999999999998},"last":1.4350000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335540,"statfs":{"total":96636764160,"available":96599592960,"internally_reserved":0,"allocated":987136,"data_stored":466309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.1399999999999999,"15min":1.214},"min":{"1min":0.63100000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.486,"5min":1.675,"15min":2.1019999999999999},"last":2.3959999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.2789999999999999,"15min":1.3029999999999999},"min":{"1min":0.72399999999999998,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.724,"5min":1.8600000000000001,"15min":1.915},"last":2.4510000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.135,"15min":1.149},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.629,"5min":1.7729999999999999,"15min":2.4569999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.179,"5min":1.1279999999999999,"15min":1.1559999999999999},"min":{"1min":0.84299999999999997,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.526,"5min":1.6970000000000001,"15min":2.2410000000000001},"last":1.2410000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.157,"15min":1.163},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71299999999999997},"max":{"1min":1.675,"5min":1.7470000000000001,"15min":1.77},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.228,"15min":1.246},"min":{"1min":0.83499999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.5780000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.083}]},{"osd":3,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.288,"15min":1.2869999999999999},"min":{"1min":0.81699999999999995,"5min":0.76000000000000001,"15min":0.73799999999999999},"max":{"1min":1.8839999999999999,"5min":1.97,"15min":1.97},"last":1.167},{"interface":"front","average":{"1min":1.298,"5min":1.355,"15min":1.3660000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.623,"5min":1.7989999999999999,"15min":2.1160000000000001},"last":1.4850000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.2529999999999999,"15min":1.2949999999999999},"min":{"1min":0.79700000000000004,"5min":0.77100000000000002,"15min":0.52100000000000002},"max":{"1min":1.569,"5min":1.7989999999999999,"15min":2.355},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4990000000000001,"15min":1.5229999999999999},"min":{"1min":0.91700000000000004,"5min":0.89500000000000002,"15min":0.63600000000000001},"max":{"1min":1.8069999999999999,"5min":2.0920000000000001,"15min":2.4079999999999999},"last":1.548}]},{"osd":6,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.3740000000000001,"15min":1.379},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.55500000000000005},"max":{"1min":1.726,"5min":1.847,"15min":2.3580000000000001},"last":1.141},{"interface":"front","average":{"1min":1.375,"5min":1.3839999999999999,"15min":1.4099999999999999},"min":{"1min":1.151,"5min":0.83299999999999996,"15min":0.58399999999999996},"max":{"1min":1.6120000000000001,"5min":1.889,"15min":2.4159999999999999},"last":1.0389999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.478,"15min":1.5109999999999999},"min":{"1min":1.0669999999999999,"5min":0.88600000000000001,"15min":0.78700000000000003},"max":{"1min":1.913,"5min":2.1989999999999998,"15min":2.4809999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.575,"15min":1.5960000000000001},"min":{"1min":1.2509999999999999,"5min":0.99199999999999999,"15min":0.75600000000000001},"max":{"1min":1.851,"5min":1.9430000000000001,"15min":2.931},"last":1.3029999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46056,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325784,"statfs":{"total":96636764160,"available":96589602816,"internally_reserved":0,"allocated":1003520,"data_stored":473569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.107,"15min":1.163},"min":{"1min":0.74199999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.554,"5min":1.76,"15min":4.2640000000000002},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.3460000000000001,"15min":1.252},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.756,"5min":11.974,"15min":11.974},"last":1.571}]},{"osd":1,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.3580000000000001,"15min":1.2809999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.7549999999999999,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":1.8939999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.2130000000000001,"15min":1.24},"min":{"1min":0.67700000000000005,"5min":0.63600000000000001,"15min":0.624},"max":{"1min":1.5309999999999999,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":0.98699999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.254,"15min":1.2490000000000001},"min":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.75,"5min":1.756,"15min":2.3130000000000002},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.377,"15min":1.3799999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.615,"5min":5.0549999999999997,"15min":5.0549999999999997},"last":1.415}]},{"osd":3,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3160000000000001,"15min":1.3129999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.6739999999999999,"5min":6.8079999999999998,"15min":6.8079999999999998},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.3759999999999999,"15min":1.395},"min":{"1min":0.93100000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.885,"5min":1.923,"15min":3.3050000000000002},"last":1.6930000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.498,"15min":1.423},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9590000000000001,"5min":8.702,"15min":8.702},"last":1.7869999999999999},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.69099999999999995},"max":{"1min":1.754,"5min":3.0430000000000001,"15min":3.669},"last":1.2789999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.4810000000000001,"15min":1.4299999999999999},"min":{"1min":1.0049999999999999,"5min":0.80700000000000005,"15min":0.56599999999999995},"max":{"1min":1.9850000000000001,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":1.8600000000000001},{"interface":"front","average":{"1min":1.534,"5min":1.605,"15min":1.5509999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.70299999999999996},"max":{"1min":2.1179999999999999,"5min":6.4089999999999998,"15min":6.4089999999999998},"last":2.0459999999999998}]},{"osd":7,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.508,"15min":1.5029999999999999},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.60999999999999999},"max":{"1min":2.177,"5min":2.9399999999999999,"15min":3.5150000000000001},"last":1.629},{"interface":"front","average":{"1min":1.631,"5min":1.585,"15min":1.5860000000000001},"min":{"1min":1.1220000000000001,"5min":0.98399999999999999,"15min":0.89600000000000002},"max":{"1min":2.0539999999999998,"5min":2.6120000000000001,"15min":3.9849999999999999},"last":2.0030000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575036,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36304,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335536,"statfs":{"total":96636764160,"available":96599588864,"internally_reserved":0,"allocated":987136,"data_stored":465713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.83099999999999996,"15min":0.85999999999999999},"min":{"1min":0.60799999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4339999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.95599999999999996,"15min":1.004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.29999999999999999},"max":{"1min":1.385,"5min":1.708,"15min":1.708},"last":1.093}]},{"osd":1,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.91200000000000003,"15min":0.94299999999999995},"min":{"1min":0.42799999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.2210000000000001,"5min":1.49,"15min":1.7150000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.006,"15min":1.0509999999999999},"min":{"1min":0.52900000000000003,"5min":0.42099999999999999,"15min":0.39600000000000002},"max":{"1min":1.4079999999999999,"5min":1.603,"15min":1.643},"last":1.0509999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":1.0149999999999999,"15min":1.0580000000000001},"min":{"1min":0.46500000000000002,"5min":0.44800000000000001,"15min":0.439},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.8640000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.141,"15min":1.163},"min":{"1min":0.55100000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.534,"5min":1.776,"15min":2.028},"last":1.2250000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.24,"15min":1.276},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.5740000000000001,"5min":1.8069999999999999,"15min":2.1619999999999999},"last":1.369},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.296,"15min":1.3220000000000001},"min":{"1min":0.75600000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.6779999999999999,"5min":1.8500000000000001,"15min":2.532},"last":1.5109999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.278,"15min":1.3169999999999999},"min":{"1min":0.77800000000000002,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.5629999999999999,"5min":1.7350000000000001,"15min":2.0019999999999998},"last":1.663},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3029999999999999,"15min":1.3460000000000001},"min":{"1min":0.91200000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.5589999999999999,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":1.8080000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.375,"15min":1.429},"min":{"1min":0.72099999999999997,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6799999999999999,"5min":1.891,"15min":2.3029999999999999},"last":2.1080000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3999999999999999,"15min":1.4410000000000001},"min":{"1min":0.88300000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.619,"5min":1.976,"15min":2.1240000000000001},"last":1.998}]},{"osd":7,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3959999999999999,"15min":1.4490000000000001},"min":{"1min":0.73199999999999998,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7450000000000001,"5min":2.032,"15min":2.403},"last":1.8560000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.504,"15min":1.5640000000000001},"min":{"1min":0.68600000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8320000000000001,"5min":2.2429999999999999,"15min":2.6099999999999999},"last":2.0720000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575035,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40476,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331364,"statfs":{"total":96636764160,"available":96595316736,"internally_reserved":0,"allocated":999424,"data_stored":472013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.875,"15min":0.86699999999999999},"min":{"1min":0.499,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.506,"5min":1.5469999999999999,"15min":2.8690000000000002},"last":1.0580000000000001},{"interface":"front","average":{"1min":1,"5min":0.98099999999999998,"15min":1.028},"min":{"1min":0.52100000000000002,"5min":0.34999999999999998,"15min":0.28899999999999998},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.72499999999999998}]},{"osd":1,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.044,"15min":1.036},"min":{"1min":0.56999999999999995,"5min":0.52900000000000003,"15min":0.42899999999999999},"max":{"1min":1.5329999999999999,"5min":1.8080000000000001,"15min":2.6059999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0629999999999999,"15min":1.1140000000000001},"min":{"1min":0.622,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4770000000000001,"5min":1.792,"15min":2.6499999999999999},"last":1}]},{"osd":3,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":1.006,"15min":1.0089999999999999},"min":{"1min":0.55400000000000005,"5min":0.41399999999999998,"15min":0.38},"max":{"1min":1.3300000000000001,"5min":1.776,"15min":2.5339999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.226,"5min":1.169,"15min":1.1970000000000001},"min":{"1min":0.59799999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.7969999999999999,"5min":1.8,"15min":2.9700000000000002},"last":0.86899999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.3420000000000001,"15min":1.3480000000000001},"min":{"1min":1.083,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":2.2309999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3240000000000001,"15min":1.302},"min":{"1min":0.97899999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.2000000000000002},"last":1.151}]},{"osd":5,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3600000000000001,"15min":1.373},"min":{"1min":1.0249999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.9810000000000001,"5min":2.1640000000000001,"15min":2.7770000000000001},"last":1.26},{"interface":"front","average":{"1min":1.462,"5min":1.391,"15min":1.415},"min":{"1min":0.72699999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.885,"5min":1.956,"15min":2.952},"last":1.621}]},{"osd":6,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3400000000000001,"15min":1.3360000000000001},"min":{"1min":0.89100000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.831,"5min":2.1970000000000001,"15min":2.2469999999999999},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.4330000000000001,"15min":1.4410000000000001},"min":{"1min":1.1180000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.8140000000000001,"5min":2.044,"15min":2.8180000000000001},"last":1.353}]},{"osd":7,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.4630000000000001,"15min":1.4850000000000001},"min":{"1min":1.0109999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.012,"5min":2.0739999999999998,"15min":3.746},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.411,"15min":1.431},"min":{"1min":1.085,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.9279999999999999,"5min":2.3109999999999999,"15min":2.3769999999999998},"last":1.1850000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575035,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50196,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321644,"statfs":{"total":96636764160,"available":96585363456,"internally_reserved":0,"allocated":983040,"data_stored":465747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.83399999999999996,"15min":0.88},"min":{"1min":0.67400000000000004,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2909999999999999,"5min":1.363,"15min":2.2010000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.087,"5min":0.99099999999999999,"15min":0.998},"min":{"1min":0.66100000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.6200000000000001,"5min":1.798,"15min":2.1619999999999999},"last":1.242}]},{"osd":2,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.98199999999999998,"15min":0.995},"min":{"1min":0.60399999999999998,"5min":0.45300000000000001,"15min":0.42299999999999999},"max":{"1min":1.3700000000000001,"5min":1.526,"15min":2.3849999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2430000000000001},"min":{"1min":0.70199999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5669999999999999,"5min":1.667,"15min":2.2959999999999998},"last":1.181}]},{"osd":3,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.113,"15min":1.143},"min":{"1min":0.70199999999999996,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":2.254},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.0660000000000001,"15min":1.1060000000000001},"min":{"1min":0.83299999999999996,"5min":0.40699999999999997,"15min":0.36099999999999999},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":2.8109999999999999},"last":1.1180000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.206,"15min":1.2450000000000001},"min":{"1min":0.84099999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5189999999999999,"5min":1.7010000000000001,"15min":2.4540000000000002},"last":1.046},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3069999999999999,"15min":1.3799999999999999},"min":{"1min":1.0309999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5389999999999999,"5min":1.7789999999999999,"15min":2.851},"last":1.29}]},{"osd":5,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3129999999999999,"15min":1.345},"min":{"1min":1.155,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5820000000000001,"5min":1.7889999999999999,"15min":2.7120000000000002},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.298,"15min":1.335},"min":{"1min":1.034,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.7290000000000001,"5min":1.7869999999999999,"15min":2.512},"last":1.415}]},{"osd":6,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.385,"15min":1.4099999999999999},"min":{"1min":1.0760000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6830000000000001,"5min":1.7649999999999999,"15min":2.75},"last":1.593},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.3620000000000001,"15min":1.401},"min":{"1min":1.014,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.8029999999999999,"5min":1.851,"15min":2.669},"last":1.6870000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4339999999999999,"15min":1.494},"min":{"1min":1.1619999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.784,"5min":1.8799999999999999,"15min":2.8799999999999999},"last":1.377},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.4910000000000001,"15min":1.506},"min":{"1min":1.2589999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":3.1400000000000001},"last":1.8280000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32732,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339108,"statfs":{"total":96636764160,"available":96603246592,"internally_reserved":0,"allocated":1589248,"data_stored":1058219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.86899999999999999,"15min":0.85699999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.585,"5min":2.766,"15min":2.766},"last":1.6950000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.998,"15min":1.03},"min":{"1min":0.44900000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.5720000000000001,"5min":3.96,"15min":3.96},"last":0.85799999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0660000000000001,"15min":1.0429999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.38900000000000001},"max":{"1min":1.526,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":1.7889999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.107,"15min":1.089},"min":{"1min":0.50700000000000001,"5min":0.48999999999999999,"15min":0.46600000000000003},"max":{"1min":1.722,"5min":4.4290000000000003,"15min":4.4290000000000003},"last":1.0880000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.054,"15min":1.032},"min":{"1min":0.495,"5min":0.38,"15min":0.38},"max":{"1min":1.498,"5min":4.7380000000000004,"15min":4.7380000000000004},"last":1.024},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.24,"15min":1.202},"min":{"1min":0.47099999999999997,"5min":0.44,"15min":0.44},"max":{"1min":2.1859999999999999,"5min":5.1500000000000004,"15min":5.1500000000000004},"last":1.8759999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.341,"15min":1.3340000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.859,"5min":4.1970000000000001,"15min":4.1970000000000001},"last":1.2},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.3049999999999999,"15min":1.2909999999999999},"min":{"1min":0.77900000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.623,"5min":4.9630000000000001,"15min":4.9630000000000001},"last":1.1459999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3280000000000001,"15min":1.3220000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6799999999999999,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.4239999999999999,"15min":1.407},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.1269999999999998,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":1.9510000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.464,"15min":1.4390000000000001},"min":{"1min":0.98099999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0739999999999998,"5min":3.8250000000000002,"15min":3.8250000000000002},"last":2},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.444,"15min":1.4259999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.84999999999999998},"max":{"1min":2.02,"5min":4.0979999999999999,"15min":4.0979999999999999},"last":1.746}]},{"osd":7,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.49,"15min":1.4750000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7929999999999999,"5min":4.3369999999999997,"15min":4.3369999999999997},"last":1.8380000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4610000000000001,"15min":1.4359999999999999},"min":{"1min":0.83599999999999997,"5min":0.755,"15min":0.755},"max":{"1min":1.8169999999999999,"5min":5.0570000000000004,"15min":5.0570000000000004},"last":1.371}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:04:38.334 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T10:04:38.335 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T10:04:38.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.597+0000 7f527cffb640 1 Processor -- start 2023-10-16T10:04:38.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.597+0000 7f527cffb640 1 -- start start 2023-10-16T10:04:38.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.598+0000 7f527cffb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f52780b3070 0x7f52780cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:38.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.598+0000 7f527cffb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52780b3940 con 0x7f52780d1b10 2023-10-16T10:04:38.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.598+0000 7f527cffb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52780b45e0 con 0x7f52780b3e70 2023-10-16T10:04:38.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.598+0000 7f527cffb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52780b3b90 con 0x7f52780b3070 2023-10-16T10:04:38.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.598+0000 7f5276d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f52780d1b10 0x7f52780c6f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39754/0 (socket says 172.21.15.84:39754) 2023-10-16T10:04:38.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.598+0000 7f5276d9d640 1 -- 172.21.15.84:0/3198506387 learned_addr learned my addr 172.21.15.84:0/3198506387 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:38.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.599+0000 7f527559a640 1 -- 172.21.15.84:0/3198506387 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2295028662 0 0) 0x7f52780b3940 con 0x7f52780d1b10 2023-10-16T10:04:38.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.599+0000 7f527559a640 1 -- 172.21.15.84:0/3198506387 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5264003520 con 0x7f52780d1b10 2023-10-16T10:04:38.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.599+0000 7f527659c640 1 --2- 172.21.15.84:0/3198506387 >> v2:172.21.15.157:3300/0 conn(0x7f52780b3070 0x7f52780cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:38.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.599+0000 7f527559a640 1 -- 172.21.15.84:0/3198506387 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1307330296 0 0) 0x7f5264003520 con 0x7f52780d1b10 2023-10-16T10:04:38.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.599+0000 7f527559a640 1 -- 172.21.15.84:0/3198506387 >> v2:172.21.15.157:3300/0 conn(0x7f52780b3070 msgr2=0x7f52780cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:38.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.599+0000 7f527559a640 1 --2- 172.21.15.84:0/3198506387 >> v2:172.21.15.157:3300/0 conn(0x7f52780b3070 0x7f52780cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.599+0000 7f527559a640 1 -- 172.21.15.84:0/3198506387 >> v1:172.21.15.157:6789/0 conn(0x7f52780b3e70 legacy=0x7f52780c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:38.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.599+0000 7f527559a640 1 -- 172.21.15.84:0/3198506387 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52780d3840 con 0x7f52780d1b10 2023-10-16T10:04:38.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.599+0000 7f527559a640 1 -- 172.21.15.84:0/3198506387 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f5270001670 con 0x7f52780d1b10 2023-10-16T10:04:38.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.599+0000 7f527559a640 1 -- 172.21.15.84:0/3198506387 >> v1:172.21.15.84:6789/0 conn(0x7f52780d1b10 legacy=0x7f52780c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:38.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.599+0000 7f527559a640 1 --2- 172.21.15.84:0/3198506387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5264003970 0x7f5264003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:38.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.599+0000 7f527559a640 1 --2- 172.21.15.84:0/3198506387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52640047c0 0x7f5264004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:38.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.599+0000 7f527559a640 1 --2- 172.21.15.84:0/3198506387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5264005180 0x7f526400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:38.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.599+0000 7f527559a640 1 -- 172.21.15.84:0/3198506387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52780b3b90 con 0x7f5264003970 2023-10-16T10:04:38.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.599+0000 7f527559a640 1 -- 172.21.15.84:0/3198506387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526400c0d0 con 0x7f5264005180 2023-10-16T10:04:38.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.599+0000 7f527559a640 1 -- 172.21.15.84:0/3198506387 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526400c290 con 0x7f52640047c0 2023-10-16T10:04:38.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.600+0000 7f527659c640 1 --2- 172.21.15.84:0/3198506387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5264003970 0x7f5264003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:38.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.600+0000 7f527659c640 1 -- 172.21.15.84:0/3198506387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52640047c0 msgr2=0x7f5264004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:38.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.600+0000 7f527659c640 1 --2- 172.21.15.84:0/3198506387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52640047c0 0x7f5264004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.600+0000 7f527659c640 1 -- 172.21.15.84:0/3198506387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5264005180 msgr2=0x7f526400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:38.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.600+0000 7f527659c640 1 --2- 172.21.15.84:0/3198506387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5264005180 0x7f526400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.600+0000 7f527659c640 1 -- 172.21.15.84:0/3198506387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5264003520 con 0x7f5264003970 2023-10-16T10:04:38.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.600+0000 7f5276d9d640 1 -- 172.21.15.84:0/3198506387 reap_dead start 2023-10-16T10:04:38.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.601+0000 7f527659c640 1 --2- 172.21.15.84:0/3198506387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5264003970 0x7f5264003d90 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f526800ba10 tx=0x7f52640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e0d809298ecfa3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:38.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.601+0000 7f527559a640 1 -- 172.21.15.84:0/3198506387 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268005b90 con 0x7f5264003970 2023-10-16T10:04:38.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.601+0000 7f527559a640 1 -- 172.21.15.84:0/3198506387 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5268005cf0 con 0x7f5264003970 2023-10-16T10:04:38.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.601+0000 7f527559a640 1 -- 172.21.15.84:0/3198506387 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268006270 con 0x7f5264003970 2023-10-16T10:04:38.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.602+0000 7f527cffb640 1 -- 172.21.15.84:0/3198506387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5264003970 msgr2=0x7f5264003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:38.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.602+0000 7f527cffb640 1 --2- 172.21.15.84:0/3198506387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5264003970 0x7f5264003d90 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f526800ba10 tx=0x7f52640057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.602+0000 7f527cffb640 1 -- 172.21.15.84:0/3198506387 shutdown_connections 2023-10-16T10:04:38.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.602+0000 7f527cffb640 1 --2- 172.21.15.84:0/3198506387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5264003970 0x7f5264003d90 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.602+0000 7f527cffb640 1 -- 172.21.15.84:0/3198506387 >> 172.21.15.84:0/3198506387 conn(0x7f52780d2d20 msgr2=0x7f52780d3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:38.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.602+0000 7f527cffb640 1 -- 172.21.15.84:0/3198506387 shutdown_connections 2023-10-16T10:04:38.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.602+0000 7f527cffb640 1 -- 172.21.15.84:0/3198506387 wait complete. 2023-10-16T10:04:38.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.604+0000 7f527cffb640 1 Processor -- start 2023-10-16T10:04:38.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.604+0000 7f527cffb640 1 -- start start 2023-10-16T10:04:38.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.605+0000 7f527cffb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5264003970 0x7f5278154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:38.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.605+0000 7f527cffb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f527815c8a0 0x7f527815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:38.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.605+0000 7f527cffb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5278158430 0x7f52781588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:38.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.605+0000 7f527cffb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527815d260 con 0x7f527815c8a0 2023-10-16T10:04:38.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.605+0000 7f527cffb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52780b6ac0 con 0x7f5278158430 2023-10-16T10:04:38.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.605+0000 7f527cffb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52781549a0 con 0x7f5264003970 2023-10-16T10:04:38.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.605+0000 7f5275d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f527815c8a0 0x7f527815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:38.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.605+0000 7f5275d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f527815c8a0 0x7f527815cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43852/0 (socket says 172.21.15.84:43852) 2023-10-16T10:04:38.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.605+0000 7f5275d9b640 1 -- 172.21.15.84:0/2679671271 learned_addr learned my addr 172.21.15.84:0/2679671271 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:38.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.606+0000 7f527659c640 1 --2- 172.21.15.84:0/2679671271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5264003970 0x7f5278154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:38.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.606+0000 7f5276d9d640 1 --2- 172.21.15.84:0/2679671271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5278158430 0x7f52781588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:38.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.606+0000 7f5275d9b640 1 -- 172.21.15.84:0/2679671271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5264003970 msgr2=0x7f5278154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:38.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.606+0000 7f5275d9b640 1 --2- 172.21.15.84:0/2679671271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5264003970 0x7f5278154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.606+0000 7f5275d9b640 1 -- 172.21.15.84:0/2679671271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5278158430 msgr2=0x7f52781588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:38.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.606+0000 7f5275d9b640 1 --2- 172.21.15.84:0/2679671271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5278158430 0x7f52781588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.606+0000 7f5275d9b640 1 -- 172.21.15.84:0/2679671271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52780b6d10 con 0x7f527815c8a0 2023-10-16T10:04:38.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.606+0000 7f5275d9b640 1 --2- 172.21.15.84:0/2679671271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f527815c8a0 0x7f527815cd20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f526000c740 tx=0x7f526000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:38.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.606+0000 7f526effd640 1 -- 172.21.15.84:0/2679671271 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260006d50 con 0x7f527815c8a0 2023-10-16T10:04:38.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.606+0000 7f526effd640 1 -- 172.21.15.84:0/2679671271 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526000ce50 con 0x7f527815c8a0 2023-10-16T10:04:38.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.607+0000 7f526effd640 1 -- 172.21.15.84:0/2679671271 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260003bb0 con 0x7f527815c8a0 2023-10-16T10:04:38.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.607+0000 7f527cffb640 1 -- 172.21.15.84:0/2679671271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52780b6ff0 con 0x7f527815c8a0 2023-10-16T10:04:38.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.607+0000 7f527cffb640 1 -- 172.21.15.84:0/2679671271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52780b75b0 con 0x7f527815c8a0 2023-10-16T10:04:38.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.608+0000 7f527cffb640 1 -- 172.21.15.84:0/2679671271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52780b7740 con 0x7f527815c8a0 2023-10-16T10:04:38.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.609+0000 7f526effd640 1 -- 172.21.15.84:0/2679671271 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5260003d10 con 0x7f527815c8a0 2023-10-16T10:04:38.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.610+0000 7f526effd640 1 --2- 172.21.15.84:0/2679671271 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f525806af90 0x7f525806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:38.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.610+0000 7f526effd640 1 -- 172.21.15.84:0/2679671271 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5260089290 con 0x7f527815c8a0 2023-10-16T10:04:38.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.610+0000 7f527659c640 1 --2- 172.21.15.84:0/2679671271 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f525806af90 0x7f525806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:38.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.611+0000 7f527659c640 1 --2- 172.21.15.84:0/2679671271 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f525806af90 0x7f525806d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5268036000 tx=0x7f5268030000 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:38.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.614+0000 7f526effd640 1 -- 172.21.15.84:0/2679671271 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f52600542c0 con 0x7f527815c8a0 2023-10-16T10:04:38.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.875+0000 7f527cffb640 1 -- 172.21.15.84:0/2679671271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f527804e160 con 0x7f527815c8a0 2023-10-16T10:04:38.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.880+0000 7f526effd640 1 -- 172.21.15.84:0/2679671271 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f52600578e0 con 0x7f527815c8a0 2023-10-16T10:04:38.882 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2023-10-16T10:04:38.882 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ== 2023-10-16T10:04:38.882 INFO:teuthology.orchestra.run.smithi084.stdout: caps mds = "allow" 2023-10-16T10:04:38.883 INFO:teuthology.orchestra.run.smithi084.stdout: caps mon = "allow r" 2023-10-16T10:04:38.883 INFO:teuthology.orchestra.run.smithi084.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T10:04:38.883 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-16T10:04:38.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.883+0000 7f527cffb640 1 -- 172.21.15.84:0/2679671271 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f525806af90 msgr2=0x7f525806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:38.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.883+0000 7f527cffb640 1 --2- 172.21.15.84:0/2679671271 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f525806af90 0x7f525806d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5268036000 tx=0x7f5268030000 comp rx=0 tx=0).stop 2023-10-16T10:04:38.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.883+0000 7f527cffb640 1 -- 172.21.15.84:0/2679671271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f527815c8a0 msgr2=0x7f527815cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:38.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.883+0000 7f527cffb640 1 --2- 172.21.15.84:0/2679671271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f527815c8a0 0x7f527815cd20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f526000c740 tx=0x7f526000cc00 comp rx=0 tx=0).stop 2023-10-16T10:04:38.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.883+0000 7f527cffb640 1 -- 172.21.15.84:0/2679671271 shutdown_connections 2023-10-16T10:04:38.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.883+0000 7f527cffb640 1 --2- 172.21.15.84:0/2679671271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5278158430 0x7f52781588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.883+0000 7f527cffb640 1 --2- 172.21.15.84:0/2679671271 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f525806af90 0x7f525806d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.883+0000 7f527cffb640 1 --2- 172.21.15.84:0/2679671271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f527815c8a0 0x7f527815cd20 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.883+0000 7f527cffb640 1 --2- 172.21.15.84:0/2679671271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5264003970 0x7f5278154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:38.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.883+0000 7f527cffb640 1 -- 172.21.15.84:0/2679671271 >> 172.21.15.84:0/2679671271 conn(0x7f52780d2d20 msgr2=0x7f527806a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:38.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.883+0000 7f527cffb640 1 -- 172.21.15.84:0/2679671271 shutdown_connections 2023-10-16T10:04:38.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:38.883+0000 7f527cffb640 1 -- 172.21.15.84:0/2679671271 wait complete. 2023-10-16T10:04:38.902 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:04:39.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.112+0000 7f04225b3640 1 Processor -- start 2023-10-16T10:04:39.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.112+0000 7f04225b3640 1 -- start start 2023-10-16T10:04:39.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.114+0000 7f04225b3640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f041c15c9f0 0x7f041c0c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:39.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.114+0000 7f04225b3640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f041c160a90 con 0x7f041c0c9230 2023-10-16T10:04:39.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.114+0000 7f04225b3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f041c0cd900 con 0x7f041c15dde0 2023-10-16T10:04:39.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.114+0000 7f04225b3640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041c0cdec0 con 0x7f041c15c9f0 2023-10-16T10:04:39.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.114+0000 7f0420db0640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f041c0c9230 0x7f041c0709e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39770/0 (socket says 172.21.15.84:39770) 2023-10-16T10:04:39.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.114+0000 7f04215b1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f041c15c9f0 0x7f041c0c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:39.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.114+0000 7f0420db0640 1 -- 172.21.15.84:0/388857720 learned_addr learned my addr 172.21.15.84:0/388857720 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:39.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.115+0000 7f04215b1640 1 -- 172.21.15.84:0/388857720 >> v1:172.21.15.157:6789/0 conn(0x7f041c15dde0 legacy=0x7f041c0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:39.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.115+0000 7f04215b1640 1 -- 172.21.15.84:0/388857720 >> v1:172.21.15.84:6789/0 conn(0x7f041c0c9230 legacy=0x7f041c0709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:39.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.115+0000 7f04215b1640 1 -- 172.21.15.84:0/388857720 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c0cdbe0 con 0x7f041c15c9f0 2023-10-16T10:04:39.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.116+0000 7f04215b1640 1 --2- 172.21.15.84:0/388857720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c15c9f0 0x7f041c0c8cf0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f04180097a0 tx=0x7f041802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169ff74f65cb8457 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:39.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.116+0000 7f0413fff640 1 -- 172.21.15.84:0/388857720 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418031830 con 0x7f041c15c9f0 2023-10-16T10:04:39.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.116+0000 7f0413fff640 1 -- 172.21.15.84:0/388857720 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0418031990 con 0x7f041c15c9f0 2023-10-16T10:04:39.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.116+0000 7f0413fff640 1 -- 172.21.15.84:0/388857720 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418031d20 con 0x7f041c15c9f0 2023-10-16T10:04:39.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.117+0000 7f04225b3640 1 -- 172.21.15.84:0/388857720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c15c9f0 msgr2=0x7f041c0c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:39.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.117+0000 7f04225b3640 1 --2- 172.21.15.84:0/388857720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c15c9f0 0x7f041c0c8cf0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f04180097a0 tx=0x7f041802b8d0 comp rx=0 tx=0).stop 2023-10-16T10:04:39.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.117+0000 7f04225b3640 1 -- 172.21.15.84:0/388857720 shutdown_connections 2023-10-16T10:04:39.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.117+0000 7f04225b3640 1 --2- 172.21.15.84:0/388857720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c15c9f0 0x7f041c0c8cf0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:39.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.117+0000 7f04225b3640 1 -- 172.21.15.84:0/388857720 >> 172.21.15.84:0/388857720 conn(0x7f041c0ce6a0 msgr2=0x7f041c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:39.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.117+0000 7f04225b3640 1 -- 172.21.15.84:0/388857720 shutdown_connections 2023-10-16T10:04:39.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.117+0000 7f04225b3640 1 -- 172.21.15.84:0/388857720 wait complete. 2023-10-16T10:04:39.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.119+0000 7f04225b3640 1 Processor -- start 2023-10-16T10:04:39.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.119+0000 7f04225b3640 1 -- start start 2023-10-16T10:04:39.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.121+0000 7f04225b3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f041c0c9230 0x7f041c153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:39.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.121+0000 7f04225b3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c15c9f0 0x7f041c157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:39.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.121+0000 7f04225b3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c15dde0 0x7f041c157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:39.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.121+0000 7f04225b3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c0c2af0 con 0x7f041c15dde0 2023-10-16T10:04:39.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.121+0000 7f04225b3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c1f2930 con 0x7f041c15c9f0 2023-10-16T10:04:39.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.121+0000 7f04225b3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c0c24a0 con 0x7f041c0c9230 2023-10-16T10:04:39.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.121+0000 7f0421db2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c15dde0 0x7f041c157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:39.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.121+0000 7f0421db2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c15dde0 0x7f041c157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43884/0 (socket says 172.21.15.84:43884) 2023-10-16T10:04:39.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.121+0000 7f0421db2640 1 -- 172.21.15.84:0/638694656 learned_addr learned my addr 172.21.15.84:0/638694656 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:39.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.121+0000 7f04215b1640 1 --2- 172.21.15.84:0/638694656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f041c0c9230 0x7f041c153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:39.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.121+0000 7f0420db0640 1 --2- 172.21.15.84:0/638694656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c15c9f0 0x7f041c157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:39.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.121+0000 7f0421db2640 1 -- 172.21.15.84:0/638694656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f041c0c9230 msgr2=0x7f041c153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:39.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.121+0000 7f0421db2640 1 --2- 172.21.15.84:0/638694656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f041c0c9230 0x7f041c153720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:39.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.121+0000 7f0421db2640 1 -- 172.21.15.84:0/638694656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c15c9f0 msgr2=0x7f041c157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:39.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.121+0000 7f0421db2640 1 --2- 172.21.15.84:0/638694656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c15c9f0 0x7f041c157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:39.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.121+0000 7f0421db2640 1 -- 172.21.15.84:0/638694656 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0418009400 con 0x7f041c15dde0 2023-10-16T10:04:39.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.121+0000 7f0420db0640 1 --2- 172.21.15.84:0/638694656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c15c9f0 0x7f041c157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:39.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.122+0000 7f0421db2640 1 --2- 172.21.15.84:0/638694656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c15dde0 0x7f041c157f90 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f041400b960 tx=0x7f041400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:39.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.122+0000 7f0411ffb640 1 -- 172.21.15.84:0/638694656 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041400cb20 con 0x7f041c15dde0 2023-10-16T10:04:39.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.122+0000 7f0411ffb640 1 -- 172.21.15.84:0/638694656 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041400cc80 con 0x7f041c15dde0 2023-10-16T10:04:39.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.122+0000 7f0411ffb640 1 -- 172.21.15.84:0/638694656 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0414014440 con 0x7f041c15dde0 2023-10-16T10:04:39.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.122+0000 7f04225b3640 1 -- 172.21.15.84:0/638694656 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041c1541e0 con 0x7f041c15dde0 2023-10-16T10:04:39.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.122+0000 7f04225b3640 1 -- 172.21.15.84:0/638694656 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c153c60 con 0x7f041c15dde0 2023-10-16T10:04:39.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.124+0000 7f04225b3640 1 -- 172.21.15.84:0/638694656 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041c0c2c30 con 0x7f041c15dde0 2023-10-16T10:04:39.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.124+0000 7f0411ffb640 1 -- 172.21.15.84:0/638694656 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f041401d3b0 con 0x7f041c15dde0 2023-10-16T10:04:39.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.126+0000 7f0411ffb640 1 --2- 172.21.15.84:0/638694656 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f040006af90 0x7f040006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:39.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.126+0000 7f0411ffb640 1 -- 172.21.15.84:0/638694656 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04140896d0 con 0x7f041c15dde0 2023-10-16T10:04:39.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.126+0000 7f04215b1640 1 --2- 172.21.15.84:0/638694656 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f040006af90 0x7f040006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:39.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.127+0000 7f04215b1640 1 --2- 172.21.15.84:0/638694656 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f040006af90 0x7f040006d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0418036000 tx=0x7f0418008030 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:39.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.130+0000 7f0411ffb640 1 -- 172.21.15.84:0/638694656 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f041c0c2c30 con 0x7f041c15dde0 2023-10-16T10:04:39.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.359+0000 7f04225b3640 1 -- 172.21.15.84:0/638694656 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f041c0c2c30 con 0x7f041c15dde0 2023-10-16T10:04:39.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.360+0000 7f0411ffb640 1 -- 172.21.15.84:0/638694656 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f041c0c2c30 con 0x7f041c15dde0 2023-10-16T10:04:39.362 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:39.362 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5066,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1223610299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1223610299},{"type":"v1","addr":"172.21.15.157:6833","nonce":1223610299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3636443426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3636443426},{"type":"v1","addr":"172.21.15.157:6835","nonce":3636443426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6537,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/3575464874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3575464874},{"type":"v1","addr":"172.21.15.84:6836","nonce":3575464874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:04:32.849497+0000","modified":"2023-10-16T10:04:35.898561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.84:6837/2669751864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2669751864},{"type":"v1","addr":"172.21.15.84:6837","nonce":2669751864}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-16T10:04:39.362 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 35 2023-10-16T10:04:39.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.362+0000 7f04225b3640 1 -- 172.21.15.84:0/638694656 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f040006af90 msgr2=0x7f040006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:39.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.362+0000 7f04225b3640 1 --2- 172.21.15.84:0/638694656 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f040006af90 0x7f040006d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0418036000 tx=0x7f0418008030 comp rx=0 tx=0).stop 2023-10-16T10:04:39.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.362+0000 7f04225b3640 1 -- 172.21.15.84:0/638694656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c15dde0 msgr2=0x7f041c157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:39.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.362+0000 7f04225b3640 1 --2- 172.21.15.84:0/638694656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c15dde0 0x7f041c157f90 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f041400b960 tx=0x7f041400be20 comp rx=0 tx=0).stop 2023-10-16T10:04:39.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.363+0000 7f04225b3640 1 -- 172.21.15.84:0/638694656 shutdown_connections 2023-10-16T10:04:39.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.363+0000 7f04225b3640 1 --2- 172.21.15.84:0/638694656 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f040006af90 0x7f040006d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:39.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.363+0000 7f04225b3640 1 --2- 172.21.15.84:0/638694656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c15dde0 0x7f041c157f90 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:39.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.363+0000 7f04225b3640 1 --2- 172.21.15.84:0/638694656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c15c9f0 0x7f041c157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:39.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.363+0000 7f04225b3640 1 --2- 172.21.15.84:0/638694656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f041c0c9230 0x7f041c153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:39.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.363+0000 7f04225b3640 1 -- 172.21.15.84:0/638694656 >> 172.21.15.84:0/638694656 conn(0x7f041c0ce6a0 msgr2=0x7f041c0ae100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:39.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.363+0000 7f04225b3640 1 -- 172.21.15.84:0/638694656 shutdown_connections 2023-10-16T10:04:39.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:39.363+0000 7f04225b3640 1 -- 172.21.15.84:0/638694656 wait complete. 2023-10-16T10:04:39.784 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T10:04:32.849497+0000', 'modified': '2023-10-16T10:04:35.898561+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6540}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6540': {'gid': 6540, 'name': 'c', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.84:6837/2669751864', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6835', 'nonce': 2669751864}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 2669751864}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T10:04:39.785 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T10:04:39.785 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-16T10:04:40.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.003+0000 7f9e5e305640 1 Processor -- start 2023-10-16T10:04:40.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.004+0000 7f9e5e305640 1 -- start start 2023-10-16T10:04:40.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.005+0000 7f9e5e305640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9e5806b660 0x7f9e5806ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.005+0000 7f9e5e305640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e580c9860 0x7f9e580ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.005+0000 7f9e5e305640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e580cc6b0 con 0x7f9e581428c0 2023-10-16T10:04:40.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.005+0000 7f9e5e305640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e581599b0 con 0x7f9e5806b660 2023-10-16T10:04:40.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.005+0000 7f9e5e305640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5806c290 con 0x7f9e580c9860 2023-10-16T10:04:40.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.006+0000 7f9e57fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9e5806b660 0x7f9e5806ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.006+0000 7f9e57fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9e5806b660 0x7f9e5806ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43900/0 (socket says 172.21.15.84:43900) 2023-10-16T10:04:40.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.006+0000 7f9e57fff640 1 -- 172.21.15.84:0/491084872 learned_addr learned my addr 172.21.15.84:0/491084872 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:40.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.006+0000 7f9e5c8a2640 1 --2- 172.21.15.84:0/491084872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e580c9860 0x7f9e580ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.006+0000 7f9e57fff640 1 -- 172.21.15.84:0/491084872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e580c9860 msgr2=0x7f9e580ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:40.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.006+0000 7f9e57fff640 1 --2- 172.21.15.84:0/491084872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e580c9860 0x7f9e580ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.006+0000 7f9e57fff640 1 -- 172.21.15.84:0/491084872 >> v1:172.21.15.157:6789/0 conn(0x7f9e581428c0 legacy=0x7f9e580c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:40.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.006+0000 7f9e57fff640 1 -- 172.21.15.84:0/491084872 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5806c570 con 0x7f9e5806b660 2023-10-16T10:04:40.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.006+0000 7f9e5c8a2640 1 --2- 172.21.15.84:0/491084872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e580c9860 0x7f9e580ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:40.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.008+0000 7f9e57fff640 1 --2- 172.21.15.84:0/491084872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e5806b660 0x7f9e5806ba40 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009870 tx=0x7f9e4002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4495cf817c7e8ce2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:40.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.008+0000 7f9e56ffd640 1 -- 172.21.15.84:0/491084872 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40030840 con 0x7f9e5806b660 2023-10-16T10:04:40.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.008+0000 7f9e56ffd640 1 -- 172.21.15.84:0/491084872 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e400309a0 con 0x7f9e5806b660 2023-10-16T10:04:40.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.008+0000 7f9e56ffd640 1 -- 172.21.15.84:0/491084872 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40030d30 con 0x7f9e5806b660 2023-10-16T10:04:40.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.009+0000 7f9e5e305640 1 -- 172.21.15.84:0/491084872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e5806b660 msgr2=0x7f9e5806ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:40.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.009+0000 7f9e5e305640 1 --2- 172.21.15.84:0/491084872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e5806b660 0x7f9e5806ba40 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009870 tx=0x7f9e4002b980 comp rx=0 tx=0).stop 2023-10-16T10:04:40.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.009+0000 7f9e5e305640 1 -- 172.21.15.84:0/491084872 shutdown_connections 2023-10-16T10:04:40.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.009+0000 7f9e5e305640 1 --2- 172.21.15.84:0/491084872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e580c9860 0x7f9e580ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.009+0000 7f9e5e305640 1 --2- 172.21.15.84:0/491084872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e5806b660 0x7f9e5806ba40 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.009+0000 7f9e5e305640 1 -- 172.21.15.84:0/491084872 >> 172.21.15.84:0/491084872 conn(0x7f9e580ce9b0 msgr2=0x7f9e580cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:40.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.009+0000 7f9e5e305640 1 -- 172.21.15.84:0/491084872 shutdown_connections 2023-10-16T10:04:40.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.009+0000 7f9e5e305640 1 -- 172.21.15.84:0/491084872 wait complete. 2023-10-16T10:04:40.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.010+0000 7f9e5e305640 1 Processor -- start 2023-10-16T10:04:40.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.011+0000 7f9e5e305640 1 -- start start 2023-10-16T10:04:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.011+0000 7f9e5e305640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e5806b660 0x7f9e58141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.011+0000 7f9e5e305640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e580c9860 0x7f9e5815bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.011+0000 7f9e5e305640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e581428c0 0x7f9e5815c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.011+0000 7f9e5e305640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e580b1860 con 0x7f9e581428c0 2023-10-16T10:04:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.011+0000 7f9e5e305640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e580b14c0 con 0x7f9e5806b660 2023-10-16T10:04:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.011+0000 7f9e5e305640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e580c40d0 con 0x7f9e580c9860 2023-10-16T10:04:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.012+0000 7f9e5c8a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e581428c0 0x7f9e5815c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.012+0000 7f9e5c8a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e581428c0 0x7f9e5815c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43912/0 (socket says 172.21.15.84:43912) 2023-10-16T10:04:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.012+0000 7f9e5c8a2640 1 -- 172.21.15.84:0/905069893 learned_addr learned my addr 172.21.15.84:0/905069893 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:40.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.012+0000 7f9e5c8a2640 1 -- 172.21.15.84:0/905069893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e580c9860 msgr2=0x7f9e5815bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:40.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.012+0000 7f9e577fe640 1 --2- 172.21.15.84:0/905069893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e580c9860 0x7f9e5815bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.012+0000 7f9e57fff640 1 --2- 172.21.15.84:0/905069893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e5806b660 0x7f9e58141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.012+0000 7f9e5c8a2640 1 --2- 172.21.15.84:0/905069893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e580c9860 0x7f9e5815bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.012+0000 7f9e5c8a2640 1 -- 172.21.15.84:0/905069893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e5806b660 msgr2=0x7f9e58141df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:40.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.012+0000 7f9e5c8a2640 1 --2- 172.21.15.84:0/905069893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e5806b660 0x7f9e58141df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.012+0000 7f9e5c8a2640 1 -- 172.21.15.84:0/905069893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e400094d0 con 0x7f9e581428c0 2023-10-16T10:04:40.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.012+0000 7f9e5c8a2640 1 --2- 172.21.15.84:0/905069893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e581428c0 0x7f9e5815c240 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000e5b0 tx=0x7f9e5000ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:40.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.013+0000 7f9e54ff9640 1 -- 172.21.15.84:0/905069893 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50004230 con 0x7f9e581428c0 2023-10-16T10:04:40.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.013+0000 7f9e54ff9640 1 -- 172.21.15.84:0/905069893 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e50004390 con 0x7f9e581428c0 2023-10-16T10:04:40.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.013+0000 7f9e54ff9640 1 -- 172.21.15.84:0/905069893 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50003810 con 0x7f9e581428c0 2023-10-16T10:04:40.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.013+0000 7f9e5e305640 1 -- 172.21.15.84:0/905069893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e580c43b0 con 0x7f9e581428c0 2023-10-16T10:04:40.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.013+0000 7f9e5e305640 1 -- 172.21.15.84:0/905069893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e58142330 con 0x7f9e581428c0 2023-10-16T10:04:40.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.015+0000 7f9e54ff9640 1 -- 172.21.15.84:0/905069893 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9e50003970 con 0x7f9e581428c0 2023-10-16T10:04:40.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.016+0000 7f9e54ff9640 1 --2- 172.21.15.84:0/905069893 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f9e3806af90 0x7f9e3806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.016+0000 7f9e54ff9640 1 -- 172.21.15.84:0/905069893 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e50089400 con 0x7f9e581428c0 2023-10-16T10:04:40.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.016+0000 7f9e57fff640 1 --2- 172.21.15.84:0/905069893 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f9e3806af90 0x7f9e3806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.017+0000 7f9e60f75740 1 Processor -- start 2023-10-16T10:04:40.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.017+0000 7f9e60f75740 1 -- start start 2023-10-16T10:04:40.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.017+0000 7f9e57fff640 1 --2- 172.21.15.84:0/905069893 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f9e3806af90 0x7f9e3806d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9e40036000 tx=0x7f9e4003b040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:40.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.018+0000 7f9e60f75740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942b95ef80 0x55942b931e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.018+0000 7f9e60f75740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95f360 0x55942b961d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.018+0000 7f9e60f75740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b9323c0 0x55942b962250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.018+0000 7f9e60f75740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942b8527a0 con 0x55942b95f360 2023-10-16T10:04:40.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.018+0000 7f9e60f75740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942b8a4740 con 0x55942b95ef80 2023-10-16T10:04:40.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.018+0000 7f9e60f75740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55942b8a1e30 con 0x55942b9323c0 2023-10-16T10:04:40.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.018+0000 7f9e577fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95f360 0x55942b961d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.018+0000 7f9e577fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95f360 0x55942b961d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43928/0 (socket says 172.21.15.84:43928) 2023-10-16T10:04:40.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.018+0000 7f9e577fe640 1 -- 172.21.15.84:0/827158362 learned_addr learned my addr 172.21.15.84:0/827158362 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:40.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.018+0000 7f9e577fe640 1 -- 172.21.15.84:0/827158362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b9323c0 msgr2=0x55942b962250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:40.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.018+0000 7f9e577fe640 1 --2- 172.21.15.84:0/827158362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b9323c0 0x55942b962250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.018+0000 7f9e577fe640 1 -- 172.21.15.84:0/827158362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942b95ef80 msgr2=0x55942b931e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:40.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.018+0000 7f9e577fe640 1 --2- 172.21.15.84:0/827158362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942b95ef80 0x55942b931e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.018+0000 7f9e577fe640 1 -- 172.21.15.84:0/827158362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55942b861910 con 0x55942b95f360 2023-10-16T10:04:40.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.019+0000 7f9e577fe640 1 --2- 172.21.15.84:0/827158362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95f360 0x55942b961d10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9e48001c20 tx=0x7f9e480020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495c9164132984c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:40.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.019+0000 7f9e4dffb640 1 -- 172.21.15.84:0/827158362 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4800dc60 con 0x55942b95f360 2023-10-16T10:04:40.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.019+0000 7f9e4dffb640 1 -- 172.21.15.84:0/827158362 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4800ddc0 con 0x55942b95f360 2023-10-16T10:04:40.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.019+0000 7f9e4dffb640 1 -- 172.21.15.84:0/827158362 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e480125f0 con 0x55942b95f360 2023-10-16T10:04:40.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.019+0000 7f9e60f75740 1 -- 172.21.15.84:0/827158362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95f360 msgr2=0x55942b961d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:40.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.019+0000 7f9e60f75740 1 --2- 172.21.15.84:0/827158362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95f360 0x55942b961d10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9e48001c20 tx=0x7f9e480020e0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.019+0000 7f9e60f75740 1 -- 172.21.15.84:0/827158362 shutdown_connections 2023-10-16T10:04:40.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.019+0000 7f9e60f75740 1 --2- 172.21.15.84:0/827158362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b9323c0 0x55942b962250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.019+0000 7f9e60f75740 1 --2- 172.21.15.84:0/827158362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95f360 0x55942b961d10 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.019+0000 7f9e60f75740 1 --2- 172.21.15.84:0/827158362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942b95ef80 0x55942b931e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.019+0000 7f9e60f75740 1 -- 172.21.15.84:0/827158362 >> 172.21.15.84:0/827158362 conn(0x55942b934190 msgr2=0x55942b97cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:40.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.020+0000 7f9e60f75740 1 -- 172.21.15.84:0/827158362 shutdown_connections 2023-10-16T10:04:40.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.020+0000 7f9e60f75740 1 -- 172.21.15.84:0/827158362 wait complete. 2023-10-16T10:04:40.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.020+0000 7f9e60f75740 1 -- start start 2023-10-16T10:04:40.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.020+0000 7f9e60f75740 1 Processor -- start 2023-10-16T10:04:40.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.021+0000 7f9e60f75740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95ef80 0x55942b977740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.021+0000 7f9e60f75740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b95f360 0x55942b9757d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.021+0000 7f9e60f75740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942b975d10 0x55942b94fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.021+0000 7f9e60f75740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942b89d720 con 0x55942b95ef80 2023-10-16T10:04:40.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.021+0000 7f9e60f75740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942b83b970 con 0x55942b975d10 2023-10-16T10:04:40.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.021+0000 7f9e60f75740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55942b83b610 con 0x55942b95f360 2023-10-16T10:04:40.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.021+0000 7f9e60f75740 10 client.0 _ll_register_callbacks cb 0x55942b85d9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:04:40.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.022+0000 7f9e57fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95ef80 0x55942b977740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.022+0000 7f9e57fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95ef80 0x55942b977740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43932/0 (socket says 172.21.15.84:43932) 2023-10-16T10:04:40.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.022+0000 7f9e57fff640 1 -- 172.21.15.84:0/148762014 learned_addr learned my addr 172.21.15.84:0/148762014 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:40.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.022+0000 7f9e577fe640 1 --2- 172.21.15.84:0/148762014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b95f360 0x55942b9757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.022+0000 7f9e57fff640 1 -- 172.21.15.84:0/148762014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b95f360 msgr2=0x55942b9757d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:40.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.022+0000 7f9e57fff640 1 --2- 172.21.15.84:0/148762014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b95f360 0x55942b9757d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.022+0000 7f9e57fff640 1 -- 172.21.15.84:0/148762014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942b975d10 msgr2=0x55942b94fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:40.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.022+0000 7f9e57fff640 1 --2- 172.21.15.84:0/148762014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942b975d10 0x55942b94fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.022+0000 7f9e57fff640 1 -- 172.21.15.84:0/148762014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e580cc6b0 con 0x55942b95ef80 2023-10-16T10:04:40.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.022+0000 7f9e577fe640 1 --2- 172.21.15.84:0/148762014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b95f360 0x55942b9757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:40.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.022+0000 7f9e57fff640 1 --2- 172.21.15.84:0/148762014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95ef80 0x55942b977740 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9e4002b9c0 tx=0x7f9e400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2064c0deee3f18 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:40.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.022+0000 7f9e36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T10:04:40.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.023+0000 7f9e36ffd640 1 -- 172.21.15.84:0/148762014 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40049070 con 0x55942b95ef80 2023-10-16T10:04:40.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.023+0000 7f9e36ffd640 1 -- 172.21.15.84:0/148762014 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e4003e070 con 0x55942b95ef80 2023-10-16T10:04:40.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.023+0000 7f9e60f75740 1 -- 172.21.15.84:0/148762014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55942b8c4160 con 0x55942b95ef80 2023-10-16T10:04:40.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.023+0000 7f9e36ffd640 1 -- 172.21.15.84:0/148762014 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e40044590 con 0x55942b95ef80 2023-10-16T10:04:40.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.023+0000 7f9e60f75740 10 client.6684 fetch_fsmap learned FSMap version 35 2023-10-16T10:04:40.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.023+0000 7f9e60f75740 1 -- 172.21.15.84:0/148762014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55942b89bda0 con 0x55942b95ef80 2023-10-16T10:04:40.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.023+0000 7f9e36ffd640 1 -- 172.21.15.84:0/148762014 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55942b8c4160 con 0x55942b95ef80 2023-10-16T10:04:40.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.023+0000 7f9e36ffd640 1 -- 172.21.15.84:0/148762014 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9e400446f0 con 0x55942b95ef80 2023-10-16T10:04:40.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.023+0000 7f9e60f75740 10 client.6684 fetch_fsmap finished waiting for FSMap version 35 2023-10-16T10:04:40.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.023+0000 7f9e60f75740 10 client.6684 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T10:04:40.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.023+0000 7f9e60f75740 20 client.6684 populate_metadata read hostname 'smithi084' 2023-10-16T10:04:40.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.023+0000 7f9e60f75740 1 --2- 172.21.15.84:0/148762014 >> [v2:172.21.15.84:6835/2669751864,v1:172.21.15.84:6837/2669751864] conn(0x55942b990d80 0x55942b99a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.023+0000 7f9e60f75740 4 client.6684 mds_command: new command op to 6540 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T10:04:40.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.023+0000 7f9e60f75740 1 -- 172.21.15.84:0/148762014 --> [v2:172.21.15.84:6835/2669751864,v1:172.21.15.84:6837/2669751864] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55942b86fcc0 con 0x55942b990d80 2023-10-16T10:04:40.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.024+0000 7f9e577fe640 1 --2- 172.21.15.84:0/148762014 >> [v2:172.21.15.84:6835/2669751864,v1:172.21.15.84:6837/2669751864] conn(0x55942b990d80 0x55942b99a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.024+0000 7f9e577fe640 1 --2- 172.21.15.84:0/148762014 >> [v2:172.21.15.84:6835/2669751864,v1:172.21.15.84:6837/2669751864] conn(0x55942b990d80 0x55942b99a5b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c08d6ba086f793e5 server_cookie=fd6d25f99bffbe5b in_seq=0 out_seq=0 2023-10-16T10:04:40.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.024+0000 7f9e36ffd640 10 client.6684 ms_handle_connect on v2:172.21.15.84:6835/2669751864 2023-10-16T10:04:40.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.025+0000 7f9e36ffd640 1 -- 172.21.15.84:0/148762014 <== mds.0 v2:172.21.15.84:6835/2669751864 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55942b86fcc0 con 0x55942b990d80 2023-10-16T10:04:40.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.025+0000 7f9e36ffd640 10 client.6684 handle_command_reply: tid=0 2023-10-16T10:04:40.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.025+0000 7f9e60f75740 1 client.6684 shutdown 2023-10-16T10:04:40.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.025+0000 7f9e60f75740 1 -- 172.21.15.84:0/148762014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95ef80 msgr2=0x55942b977740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:40.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.025+0000 7f9e60f75740 1 --2- 172.21.15.84:0/148762014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95ef80 0x55942b977740 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9e4002b9c0 tx=0x7f9e400046c0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.025+0000 7f9e60f75740 1 -- 172.21.15.84:0/148762014 shutdown_connections 2023-10-16T10:04:40.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.025+0000 7f9e60f75740 1 --2- 172.21.15.84:0/148762014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942b975d10 0x55942b94fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.025+0000 7f9e60f75740 1 --2- 172.21.15.84:0/148762014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b95f360 0x55942b9757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.025+0000 7f9e60f75740 1 --2- 172.21.15.84:0/148762014 >> [v2:172.21.15.84:6835/2669751864,v1:172.21.15.84:6837/2669751864] conn(0x55942b990d80 0x55942b99a5b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.025+0000 7f9e60f75740 1 --2- 172.21.15.84:0/148762014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95ef80 0x55942b977740 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.025+0000 7f9e36ffd640 0 client.6684 ms_handle_reset on v2:172.21.15.84:6835/2669751864 2023-10-16T10:04:40.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.025+0000 7f9e60f75740 1 -- 172.21.15.84:0/148762014 >> 172.21.15.84:0/148762014 conn(0x55942b9323c0 msgr2=0x55942b935e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:40.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.025+0000 7f9e60f75740 1 -- 172.21.15.84:0/148762014 shutdown_connections 2023-10-16T10:04:40.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.026+0000 7f9e60f75740 1 -- 172.21.15.84:0/148762014 wait complete. 2023-10-16T10:04:40.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.026+0000 7f9e60f75740 20 client.6684 trim_cache size 0 max 16384 2023-10-16T10:04:40.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.042+0000 7f9e60f75740 1 Processor -- start 2023-10-16T10:04:40.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.042+0000 7f9e60f75740 1 -- start start 2023-10-16T10:04:40.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.043+0000 7f9e60f75740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b95ef80 0x55942b98f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.043+0000 7f9e60f75740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95f360 0x55942b9e7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.043+0000 7f9e60f75740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942b975d10 0x55942b99e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.043+0000 7f9e60f75740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942b601f20 con 0x55942b95f360 2023-10-16T10:04:40.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.043+0000 7f9e60f75740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942b935ac0 con 0x55942b975d10 2023-10-16T10:04:40.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.043+0000 7f9e60f75740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55942b97d790 con 0x55942b95ef80 2023-10-16T10:04:40.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.043+0000 7f9e577fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95f360 0x55942b9e7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.043+0000 7f9e577fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95f360 0x55942b9e7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43940/0 (socket says 172.21.15.84:43940) 2023-10-16T10:04:40.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.043+0000 7f9e577fe640 1 -- 172.21.15.84:0/1542761153 learned_addr learned my addr 172.21.15.84:0/1542761153 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:40.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.043+0000 7f9e577fe640 1 -- 172.21.15.84:0/1542761153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b95ef80 msgr2=0x55942b98f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:40.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.043+0000 7f9e577fe640 1 --2- 172.21.15.84:0/1542761153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b95ef80 0x55942b98f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.043+0000 7f9e577fe640 1 -- 172.21.15.84:0/1542761153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942b975d10 msgr2=0x55942b99e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:40.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.043+0000 7f9e577fe640 1 --2- 172.21.15.84:0/1542761153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942b975d10 0x55942b99e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.043+0000 7f9e577fe640 1 -- 172.21.15.84:0/1542761153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55942b83ab80 con 0x55942b95f360 2023-10-16T10:04:40.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.044+0000 7f9e577fe640 1 --2- 172.21.15.84:0/1542761153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95f360 0x55942b9e7f30 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9e48015040 tx=0x7f9e4800b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdcb00c9be50d2f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:40.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.044+0000 7f9e4e7fc640 1 -- 172.21.15.84:0/1542761153 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e48025530 con 0x55942b95f360 2023-10-16T10:04:40.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.044+0000 7f9e4e7fc640 1 -- 172.21.15.84:0/1542761153 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e48006b50 con 0x55942b95f360 2023-10-16T10:04:40.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.044+0000 7f9e4e7fc640 1 -- 172.21.15.84:0/1542761153 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e48003970 con 0x55942b95f360 2023-10-16T10:04:40.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.045+0000 7f9e60f75740 1 -- 172.21.15.84:0/1542761153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95f360 msgr2=0x55942b9e7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:40.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.045+0000 7f9e60f75740 1 --2- 172.21.15.84:0/1542761153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95f360 0x55942b9e7f30 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9e48015040 tx=0x7f9e4800b000 comp rx=0 tx=0).stop 2023-10-16T10:04:40.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.045+0000 7f9e60f75740 1 -- 172.21.15.84:0/1542761153 shutdown_connections 2023-10-16T10:04:40.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.045+0000 7f9e60f75740 1 --2- 172.21.15.84:0/1542761153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942b975d10 0x55942b99e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.045+0000 7f9e60f75740 1 --2- 172.21.15.84:0/1542761153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b95f360 0x55942b9e7f30 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.045+0000 7f9e60f75740 1 --2- 172.21.15.84:0/1542761153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b95ef80 0x55942b98f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.045+0000 7f9e60f75740 1 -- 172.21.15.84:0/1542761153 >> 172.21.15.84:0/1542761153 conn(0x55942b9323c0 msgr2=0x55942b980220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:40.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.045+0000 7f9e60f75740 1 -- 172.21.15.84:0/1542761153 shutdown_connections 2023-10-16T10:04:40.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.045+0000 7f9e60f75740 1 -- 172.21.15.84:0/1542761153 wait complete. 2023-10-16T10:04:40.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.045+0000 7f9e60f75740 1 -- start start 2023-10-16T10:04:40.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.046+0000 7f9e60f75740 1 Processor -- start 2023-10-16T10:04:40.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.046+0000 7f9e60f75740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b95ef80 0x55942b9e69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.046+0000 7f9e60f75740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942b95f360 0x55942b99e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.047+0000 7f9e60f75740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b975d10 0x55942b99eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.047+0000 7f9e60f75740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942b937320 con 0x55942b975d10 2023-10-16T10:04:40.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.047+0000 7f9e60f75740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942b98fa70 con 0x55942b95f360 2023-10-16T10:04:40.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.047+0000 7f9e60f75740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55942b993dd0 con 0x55942b95ef80 2023-10-16T10:04:40.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.047+0000 7f9e60f75740 10 client.0 _ll_register_callbacks cb 0x7f9e400096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:04:40.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.047+0000 7f9e5c8a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b975d10 0x55942b99eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.047+0000 7f9e5c8a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b975d10 0x55942b99eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43944/0 (socket says 172.21.15.84:43944) 2023-10-16T10:04:40.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.047+0000 7f9e5c8a2640 1 -- 172.21.15.84:0/3374416889 learned_addr learned my addr 172.21.15.84:0/3374416889 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:40.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.047+0000 7f9e5c8a2640 1 -- 172.21.15.84:0/3374416889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b95ef80 msgr2=0x55942b9e69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:40.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.047+0000 7f9e57fff640 1 --2- 172.21.15.84:0/3374416889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b95ef80 0x55942b9e69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.047+0000 7f9e5c8a2640 1 --2- 172.21.15.84:0/3374416889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b95ef80 0x55942b9e69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.047+0000 7f9e5c8a2640 1 -- 172.21.15.84:0/3374416889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942b95f360 msgr2=0x55942b99e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:40.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.047+0000 7f9e577fe640 1 --2- 172.21.15.84:0/3374416889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942b95f360 0x55942b99e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.047+0000 7f9e5c8a2640 1 --2- 172.21.15.84:0/3374416889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942b95f360 0x55942b99e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.047+0000 7f9e5c8a2640 1 -- 172.21.15.84:0/3374416889 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55942b861910 con 0x55942b975d10 2023-10-16T10:04:40.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.047+0000 7f9e5c8a2640 1 --2- 172.21.15.84:0/3374416889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b975d10 0x55942b99eb10 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f9e50057c40 tx=0x7f9e50058780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7056b6e7d8ab528a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:40.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.047+0000 7f9e36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T10:04:40.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.048+0000 7f9e36ffd640 1 -- 172.21.15.84:0/3374416889 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50054650 con 0x55942b975d10 2023-10-16T10:04:40.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.048+0000 7f9e36ffd640 1 -- 172.21.15.84:0/3374416889 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e500547b0 con 0x55942b975d10 2023-10-16T10:04:40.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.048+0000 7f9e60f75740 1 -- 172.21.15.84:0/3374416889 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55942b9792e0 con 0x55942b975d10 2023-10-16T10:04:40.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.048+0000 7f9e36ffd640 1 -- 172.21.15.84:0/3374416889 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e50057430 con 0x55942b975d10 2023-10-16T10:04:40.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.048+0000 7f9e60f75740 10 client.6690 fetch_fsmap learned FSMap version 35 2023-10-16T10:04:40.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.048+0000 7f9e60f75740 1 -- 172.21.15.84:0/3374416889 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55942b860170 con 0x55942b975d10 2023-10-16T10:04:40.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.048+0000 7f9e36ffd640 1 -- 172.21.15.84:0/3374416889 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55942b9792e0 con 0x55942b975d10 2023-10-16T10:04:40.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.048+0000 7f9e36ffd640 1 -- 172.21.15.84:0/3374416889 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9e5005fdb0 con 0x55942b975d10 2023-10-16T10:04:40.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.048+0000 7f9e60f75740 10 client.6690 fetch_fsmap finished waiting for FSMap version 35 2023-10-16T10:04:40.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.048+0000 7f9e60f75740 10 client.6690 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T10:04:40.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.048+0000 7f9e60f75740 20 client.6690 populate_metadata read hostname 'smithi084' 2023-10-16T10:04:40.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.048+0000 7f9e60f75740 1 --2- 172.21.15.84:0/3374416889 >> [v2:172.21.15.84:6835/2669751864,v1:172.21.15.84:6837/2669751864] conn(0x55942b9944c0 0x55942b9948a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.048+0000 7f9e60f75740 4 client.6690 mds_command: new command op to 6540 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T10:04:40.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.048+0000 7f9e60f75740 1 -- 172.21.15.84:0/3374416889 --> [v2:172.21.15.84:6835/2669751864,v1:172.21.15.84:6837/2669751864] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f9e4800ddc0 con 0x55942b9944c0 2023-10-16T10:04:40.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.048+0000 7f9e57fff640 1 --2- 172.21.15.84:0/3374416889 >> [v2:172.21.15.84:6835/2669751864,v1:172.21.15.84:6837/2669751864] conn(0x55942b9944c0 0x55942b9948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.049+0000 7f9e57fff640 1 --2- 172.21.15.84:0/3374416889 >> [v2:172.21.15.84:6835/2669751864,v1:172.21.15.84:6837/2669751864] conn(0x55942b9944c0 0x55942b9948a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=743bf0744561620 server_cookie=e02ee4524c67adcb in_seq=0 out_seq=0 2023-10-16T10:04:40.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.049+0000 7f9e36ffd640 10 client.6690 ms_handle_connect on v2:172.21.15.84:6835/2669751864 2023-10-16T10:04:40.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.049+0000 7f9e36ffd640 1 -- 172.21.15.84:0/3374416889 <== mds.0 v2:172.21.15.84:6835/2669751864 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x7f9e4800ddc0 con 0x55942b9944c0 2023-10-16T10:04:40.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.049+0000 7f9e36ffd640 10 client.6690 handle_command_reply: tid=0 2023-10-16T10:04:40.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.049+0000 7f9e60f75740 1 client.6690 shutdown 2023-10-16T10:04:40.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.049+0000 7f9e60f75740 1 -- 172.21.15.84:0/3374416889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b975d10 msgr2=0x55942b99eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:40.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.049+0000 7f9e60f75740 1 --2- 172.21.15.84:0/3374416889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b975d10 0x55942b99eb10 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f9e50057c40 tx=0x7f9e50058780 comp rx=0 tx=0).stop 2023-10-16T10:04:40.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.049+0000 7f9e60f75740 1 -- 172.21.15.84:0/3374416889 shutdown_connections 2023-10-16T10:04:40.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.049+0000 7f9e60f75740 1 --2- 172.21.15.84:0/3374416889 >> [v2:172.21.15.84:6835/2669751864,v1:172.21.15.84:6837/2669751864] conn(0x55942b9944c0 0x55942b9948a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.049+0000 7f9e60f75740 1 --2- 172.21.15.84:0/3374416889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55942b975d10 0x55942b99eb10 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.049+0000 7f9e60f75740 1 --2- 172.21.15.84:0/3374416889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55942b95f360 0x55942b99e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.049+0000 7f9e60f75740 1 --2- 172.21.15.84:0/3374416889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55942b95ef80 0x55942b9e69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.049+0000 7f9e36ffd640 0 client.6690 ms_handle_reset on v2:172.21.15.84:6835/2669751864 2023-10-16T10:04:40.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.049+0000 7f9e60f75740 1 -- 172.21.15.84:0/3374416889 >> 172.21.15.84:0/3374416889 conn(0x55942b9323c0 msgr2=0x55942b980220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:40.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.050+0000 7f9e60f75740 1 -- 172.21.15.84:0/3374416889 shutdown_connections 2023-10-16T10:04:40.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.050+0000 7f9e60f75740 1 -- 172.21.15.84:0/3374416889 wait complete. 2023-10-16T10:04:40.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.050+0000 7f9e60f75740 20 client.6690 trim_cache size 0 max 16384 2023-10-16T10:04:40.066 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T10:04:40.066 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "63fb4d68-f881-4694-b880-d10369105c0e", 2023-10-16T10:04:40.066 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-16T10:04:40.066 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 6540, 2023-10-16T10:04:40.067 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-16T10:04:40.067 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-16T10:04:40.067 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-16T10:04:40.067 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 6.6562161790000003, 2023-10-16T10:04:40.067 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 35, 2023-10-16T10:04:40.068 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 82, 2023-10-16T10:04:40.068 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 82, 2023-10-16T10:04:40.068 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 19.294222342000001 2023-10-16T10:04:40.068 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T10:04:40.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.050+0000 7f9e5e305640 1 -- 172.21.15.84:0/905069893 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f9e3806af90 msgr2=0x7f9e3806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:40.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.050+0000 7f9e5e305640 1 --2- 172.21.15.84:0/905069893 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f9e3806af90 0x7f9e3806d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9e40036000 tx=0x7f9e4003b040 comp rx=0 tx=0).stop 2023-10-16T10:04:40.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.050+0000 7f9e5e305640 1 -- 172.21.15.84:0/905069893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e581428c0 msgr2=0x7f9e5815c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:40.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.050+0000 7f9e5e305640 1 --2- 172.21.15.84:0/905069893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e581428c0 0x7f9e5815c240 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000e5b0 tx=0x7f9e5000ea70 comp rx=0 tx=0).stop 2023-10-16T10:04:40.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.050+0000 7f9e5e305640 1 -- 172.21.15.84:0/905069893 shutdown_connections 2023-10-16T10:04:40.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.050+0000 7f9e5e305640 1 --2- 172.21.15.84:0/905069893 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f9e3806af90 0x7f9e3806d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.050+0000 7f9e5e305640 1 --2- 172.21.15.84:0/905069893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e581428c0 0x7f9e5815c240 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.050+0000 7f9e5e305640 1 --2- 172.21.15.84:0/905069893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9e580c9860 0x7f9e5815bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.050+0000 7f9e5e305640 1 --2- 172.21.15.84:0/905069893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e5806b660 0x7f9e58141df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.050+0000 7f9e5e305640 1 -- 172.21.15.84:0/905069893 >> 172.21.15.84:0/905069893 conn(0x7f9e580ce9b0 msgr2=0x7f9e5806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:40.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.050+0000 7f9e5e305640 1 -- 172.21.15.84:0/905069893 shutdown_connections 2023-10-16T10:04:40.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.051+0000 7f9e5e305640 1 -- 172.21.15.84:0/905069893 wait complete. 2023-10-16T10:04:40.072 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T10:04:40.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.270+0000 7ff8a79ee640 1 Processor -- start 2023-10-16T10:04:40.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.271+0000 7ff8a79ee640 1 -- start start 2023-10-16T10:04:40.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.272+0000 7ff8a79ee640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a00b0b20 0x7ff8a01615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.272+0000 7ff8a79ee640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff8a015d3f0 0x7ff8a015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.272+0000 7ff8a79ee640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a015e3e0 con 0x7ff8a00cd0c0 2023-10-16T10:04:40.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.272+0000 7ff8a79ee640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a00cd4a0 con 0x7ff8a015d3f0 2023-10-16T10:04:40.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.272+0000 7ff8a79ee640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a015e100 con 0x7ff8a00b0b20 2023-10-16T10:04:40.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.272+0000 7ff8a578a640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff8a00cd0c0 0x7ff8a00b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39780/0 (socket says 172.21.15.84:39780) 2023-10-16T10:04:40.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.273+0000 7ff8a578a640 1 -- 172.21.15.84:0/2907320293 learned_addr learned my addr 172.21.15.84:0/2907320293 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:40.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.273+0000 7ff8a4f89640 1 --2- 172.21.15.84:0/2907320293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a00b0b20 0x7ff8a01615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.273+0000 7ff8a5f8b640 1 --2- 172.21.15.84:0/2907320293 >> v2:172.21.15.157:3300/0 conn(0x7ff8a015d3f0 0x7ff8a015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.273+0000 7ff893fff640 1 -- 172.21.15.84:0/2907320293 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787139187 0 0) 0x7ff8a015e3e0 con 0x7ff8a00cd0c0 2023-10-16T10:04:40.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.273+0000 7ff893fff640 1 -- 172.21.15.84:0/2907320293 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff88c003520 con 0x7ff8a00cd0c0 2023-10-16T10:04:40.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.273+0000 7ff893fff640 1 -- 172.21.15.84:0/2907320293 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3308020769 0 0) 0x7ff88c003520 con 0x7ff8a00cd0c0 2023-10-16T10:04:40.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.273+0000 7ff893fff640 1 -- 172.21.15.84:0/2907320293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a00b0b20 msgr2=0x7ff8a01615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:40.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.273+0000 7ff893fff640 1 --2- 172.21.15.84:0/2907320293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a00b0b20 0x7ff8a01615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.273+0000 7ff893fff640 1 -- 172.21.15.84:0/2907320293 >> v2:172.21.15.157:3300/0 conn(0x7ff8a015d3f0 msgr2=0x7ff8a015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:40.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.273+0000 7ff893fff640 1 --2- 172.21.15.84:0/2907320293 >> v2:172.21.15.157:3300/0 conn(0x7ff8a015d3f0 0x7ff8a015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.273+0000 7ff893fff640 1 -- 172.21.15.84:0/2907320293 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a0160b10 con 0x7ff8a00cd0c0 2023-10-16T10:04:40.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.273+0000 7ff893fff640 1 -- 172.21.15.84:0/2907320293 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ff89c001670 con 0x7ff8a00cd0c0 2023-10-16T10:04:40.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.273+0000 7ff893fff640 1 -- 172.21.15.84:0/2907320293 >> v1:172.21.15.84:6789/0 conn(0x7ff8a00cd0c0 legacy=0x7ff8a00b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:40.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.274+0000 7ff893fff640 1 --2- 172.21.15.84:0/2907320293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a00b0b20 0x7ff88c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.274+0000 7ff893fff640 1 --2- 172.21.15.84:0/2907320293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff88c003f40 0x7ff88c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.274+0000 7ff893fff640 1 --2- 172.21.15.84:0/2907320293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff88c004d00 0x7ff88c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.274+0000 7ff893fff640 1 -- 172.21.15.84:0/2907320293 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a00cd4a0 con 0x7ff88c004d00 2023-10-16T10:04:40.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.274+0000 7ff893fff640 1 -- 172.21.15.84:0/2907320293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a015e100 con 0x7ff88c003f40 2023-10-16T10:04:40.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.274+0000 7ff893fff640 1 -- 172.21.15.84:0/2907320293 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c00bae0 con 0x7ff8a00b0b20 2023-10-16T10:04:40.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.274+0000 7ff8a5f8b640 1 --2- 172.21.15.84:0/2907320293 >> v2:172.21.15.157:3300/0 conn(0x7ff8a015d3f0 0x7ff8a015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:40.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.274+0000 7ff8a5f8b640 1 --2- 172.21.15.84:0/2907320293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff88c004d00 0x7ff88c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.274+0000 7ff8a578a640 1 --2- 172.21.15.84:0/2907320293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a00b0b20 0x7ff88c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.274+0000 7ff8a5f8b640 1 -- 172.21.15.84:0/2907320293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a00b0b20 msgr2=0x7ff88c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:40.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.274+0000 7ff8a4f89640 1 --2- 172.21.15.84:0/2907320293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff88c003f40 0x7ff88c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.274+0000 7ff8a5f8b640 1 --2- 172.21.15.84:0/2907320293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a00b0b20 0x7ff88c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.274+0000 7ff8a5f8b640 1 -- 172.21.15.84:0/2907320293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff88c003f40 msgr2=0x7ff88c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:40.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.274+0000 7ff8a5f8b640 1 --2- 172.21.15.84:0/2907320293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff88c003f40 0x7ff88c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.274+0000 7ff8a5f8b640 1 -- 172.21.15.84:0/2907320293 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c003520 con 0x7ff88c004d00 2023-10-16T10:04:40.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.274+0000 7ff8a578a640 1 --2- 172.21.15.84:0/2907320293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a00b0b20 0x7ff88c003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:40.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.274+0000 7ff8a4f89640 1 --2- 172.21.15.84:0/2907320293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff88c003f40 0x7ff88c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:40.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.275+0000 7ff8a5f8b640 1 --2- 172.21.15.84:0/2907320293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff88c004d00 0x7ff88c005180 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff8980073e0 tx=0x7ff88c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83d307b9182005f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:40.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.276+0000 7ff893fff640 1 -- 172.21.15.84:0/2907320293 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89802cdc0 con 0x7ff88c004d00 2023-10-16T10:04:40.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.276+0000 7ff893fff640 1 -- 172.21.15.84:0/2907320293 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff898034440 con 0x7ff88c004d00 2023-10-16T10:04:40.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.276+0000 7ff893fff640 1 -- 172.21.15.84:0/2907320293 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8980347b0 con 0x7ff88c004d00 2023-10-16T10:04:40.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.276+0000 7ff8a79ee640 1 -- 172.21.15.84:0/2907320293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff88c004d00 msgr2=0x7ff88c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:40.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.276+0000 7ff8a79ee640 1 --2- 172.21.15.84:0/2907320293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff88c004d00 0x7ff88c005180 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff8980073e0 tx=0x7ff88c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.276+0000 7ff8a5f8b640 1 -- 172.21.15.84:0/2907320293 reap_dead start 2023-10-16T10:04:40.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.276+0000 7ff8a79ee640 1 -- 172.21.15.84:0/2907320293 shutdown_connections 2023-10-16T10:04:40.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.276+0000 7ff8a79ee640 1 -- 172.21.15.84:0/2907320293 >> 172.21.15.84:0/2907320293 conn(0x7ff8a00cb410 msgr2=0x7ff8a00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:40.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.276+0000 7ff8a79ee640 1 -- 172.21.15.84:0/2907320293 shutdown_connections 2023-10-16T10:04:40.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.276+0000 7ff8a79ee640 1 -- 172.21.15.84:0/2907320293 wait complete. 2023-10-16T10:04:40.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.278+0000 7ff8a79ee640 1 Processor -- start 2023-10-16T10:04:40.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.278+0000 7ff8a79ee640 1 -- start start 2023-10-16T10:04:40.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.279+0000 7ff8a79ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff88c004d00 0x7ff8a00c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.279+0000 7ff8a79ee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a00c4980 0x7ff8a006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.279+0000 7ff8a79ee640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a00b25a0 0x7ff8a00b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.279+0000 7ff8a79ee640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a00c2bc0 con 0x7ff88c004d00 2023-10-16T10:04:40.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.279+0000 7ff8a79ee640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a00c2ea0 con 0x7ff8a00c4980 2023-10-16T10:04:40.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.279+0000 7ff8a79ee640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a00c3180 con 0x7ff8a00b25a0 2023-10-16T10:04:40.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.279+0000 7ff8a578a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff88c004d00 0x7ff8a00c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.280+0000 7ff8a578a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff88c004d00 0x7ff8a00c24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43970/0 (socket says 172.21.15.84:43970) 2023-10-16T10:04:40.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.280+0000 7ff8a578a640 1 -- 172.21.15.84:0/917375277 learned_addr learned my addr 172.21.15.84:0/917375277 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:40.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.280+0000 7ff8a4f89640 1 --2- 172.21.15.84:0/917375277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a00c4980 0x7ff8a006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.280+0000 7ff8a5f8b640 1 --2- 172.21.15.84:0/917375277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a00b25a0 0x7ff8a00b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.280+0000 7ff8a578a640 1 -- 172.21.15.84:0/917375277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a00b25a0 msgr2=0x7ff8a00b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:40.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.280+0000 7ff8a578a640 1 --2- 172.21.15.84:0/917375277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a00b25a0 0x7ff8a00b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.280+0000 7ff8a578a640 1 -- 172.21.15.84:0/917375277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a00c4980 msgr2=0x7ff8a006bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:40.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.280+0000 7ff8a578a640 1 --2- 172.21.15.84:0/917375277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a00c4980 0x7ff8a006bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.280+0000 7ff8a578a640 1 -- 172.21.15.84:0/917375277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a00c3460 con 0x7ff88c004d00 2023-10-16T10:04:40.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.280+0000 7ff8a5f8b640 1 --2- 172.21.15.84:0/917375277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a00b25a0 0x7ff8a00b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:40.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.280+0000 7ff8a4f89640 1 --2- 172.21.15.84:0/917375277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a00c4980 0x7ff8a006bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:40.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.280+0000 7ff8a578a640 1 --2- 172.21.15.84:0/917375277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff88c004d00 0x7ff8a00c24c0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00d8e0 tx=0x7ff89c00dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:40.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.281+0000 7ff891ffb640 1 -- 172.21.15.84:0/917375277 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c0074e0 con 0x7ff88c004d00 2023-10-16T10:04:40.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.281+0000 7ff891ffb640 1 -- 172.21.15.84:0/917375277 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89c007640 con 0x7ff88c004d00 2023-10-16T10:04:40.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.281+0000 7ff891ffb640 1 -- 172.21.15.84:0/917375277 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c0049e0 con 0x7ff88c004d00 2023-10-16T10:04:40.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.281+0000 7ff8a79ee640 1 -- 172.21.15.84:0/917375277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a00c3740 con 0x7ff88c004d00 2023-10-16T10:04:40.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.281+0000 7ff8a79ee640 1 -- 172.21.15.84:0/917375277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a006c250 con 0x7ff88c004d00 2023-10-16T10:04:40.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.282+0000 7ff8a79ee640 1 -- 172.21.15.84:0/917375277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a004e160 con 0x7ff88c004d00 2023-10-16T10:04:40.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.283+0000 7ff891ffb640 1 -- 172.21.15.84:0/917375277 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff89c004b40 con 0x7ff88c004d00 2023-10-16T10:04:40.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.284+0000 7ff891ffb640 1 --2- 172.21.15.84:0/917375277 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7ff88406af90 0x7ff88406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:40.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.284+0000 7ff891ffb640 1 -- 172.21.15.84:0/917375277 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff89c089330 con 0x7ff88c004d00 2023-10-16T10:04:40.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.284+0000 7ff8a4f89640 1 --2- 172.21.15.84:0/917375277 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7ff88406af90 0x7ff88406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:40.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.285+0000 7ff8a4f89640 1 --2- 172.21.15.84:0/917375277 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7ff88406af90 0x7ff88406d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff8a015cae0 tx=0x7ff894008040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:40.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.288+0000 7ff891ffb640 1 -- 172.21.15.84:0/917375277 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff89c054120 con 0x7ff88c004d00 2023-10-16T10:04:40.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.504+0000 7ff8a79ee640 1 -- 172.21.15.84:0/917375277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff8a0158000 con 0x7ff88c004d00 2023-10-16T10:04:40.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.504+0000 7ff891ffb640 1 -- 172.21.15.84:0/917375277 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff89c057740 con 0x7ff88c004d00 2023-10-16T10:04:40.506 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T10:04:40.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.507+0000 7ff8a79ee640 1 -- 172.21.15.84:0/917375277 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7ff88406af90 msgr2=0x7ff88406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:40.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.507+0000 7ff8a79ee640 1 --2- 172.21.15.84:0/917375277 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7ff88406af90 0x7ff88406d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff8a015cae0 tx=0x7ff894008040 comp rx=0 tx=0).stop 2023-10-16T10:04:40.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.507+0000 7ff8a79ee640 1 -- 172.21.15.84:0/917375277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff88c004d00 msgr2=0x7ff8a00c24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:40.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.507+0000 7ff8a79ee640 1 --2- 172.21.15.84:0/917375277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff88c004d00 0x7ff8a00c24c0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00d8e0 tx=0x7ff89c00dda0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.507+0000 7ff8a79ee640 1 -- 172.21.15.84:0/917375277 shutdown_connections 2023-10-16T10:04:40.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.507+0000 7ff8a79ee640 1 --2- 172.21.15.84:0/917375277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8a00b25a0 0x7ff8a00b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.507+0000 7ff8a79ee640 1 --2- 172.21.15.84:0/917375277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a00c4980 0x7ff8a006bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.507+0000 7ff8a79ee640 1 --2- 172.21.15.84:0/917375277 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7ff88406af90 0x7ff88406d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.507+0000 7ff8a79ee640 1 --2- 172.21.15.84:0/917375277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff88c004d00 0x7ff8a00c24c0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:40.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.507+0000 7ff8a79ee640 1 -- 172.21.15.84:0/917375277 >> 172.21.15.84:0/917375277 conn(0x7ff8a00cb410 msgr2=0x7ff8a00af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:40.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.507+0000 7ff8a79ee640 1 -- 172.21.15.84:0/917375277 shutdown_connections 2023-10-16T10:04:40.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:40.508+0000 7ff8a79ee640 1 -- 172.21.15.84:0/917375277 wait complete. 2023-10-16T10:04:40.526 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T10:04:40.526 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T10:04:40.527 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-16T10:04:40.527 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T10:04:40.527 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T10:04:40.528 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T10:04:40.528 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T10:04:40.528 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T10:04:40.528 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:04:40.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:40.529 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-16T10:04:40.580 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T10:04:40.581 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T10:04:40.581 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T10:04:40.581 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:04:40.581 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T10:04:40.581 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:04:40.582 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:04:40.582 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:04:40.582 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-16T10:04:40.582 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:04:40.582 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:04:40.583 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-16T10:04:40.583 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:04:40.583 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:04:40.583 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T10:04:40.583 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:04:40.583 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T10:04:40.584 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 41470sec preferred_lft 41470sec 2023-10-16T10:04:40.584 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-16T10:04:40.584 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:04:40.584 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T10:04:40.584 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:04:40.585 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T10:04:40.585 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:04:40.585 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::20a8:eaff:fe9d:5cea/64 scope link 2023-10-16T10:04:40.585 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:04:40.585 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:04:40.585 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T10:04:40.586 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1854:13ff:fe47:be92/64 scope link 2023-10-16T10:04:40.586 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:04:40.586 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:04:40.586 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether ee:d5:41:a3:a5:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T10:04:40.587 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ecd5:41ff:fea3:a570/64 scope link 2023-10-16T10:04:40.587 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:04:40.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:40.588 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T10:04:40.717 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-16T10:04:40.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:40.721 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-16T10:04:40.781 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-16T10:04:40.782 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T10:04:40.782 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T10:04:40.782 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T10:04:40.782 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T10:04:40.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:40.784 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T10:04:40.785 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T10:04:40.785 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T10:04:40.785 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T10:04:40.786 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T10:04:40.786 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T10:04:40.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:40.978 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T10:04:41.029 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T10:04:41.029 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T10:04:41.030 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:04:41.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:41.031 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-16T10:04:41.123 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:04:41.124 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T10:04:41.180 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:04:41.180 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:04:41.181 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:04:41.235 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-16T10:04:41.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:41.236 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:04:41.377 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:04:41.378 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T10:04:41.411 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:04:41.411 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T10:04:41.411 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T10:04:41.411 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with new device syntax: 0@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T10:04:41.412 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T10:04:41.413 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-16T10:04:41.413 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2023-10-16T10:04:41.413 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T10:04:41.414 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T10:04:41.414 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T10:04:41.417 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T10:04:41.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:41.418 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:04:41.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:41.493 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T10:04:41.555 INFO:teuthology.orchestra.run.smithi084.stdout:client5380 2023-10-16T10:04:41.556 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:04:41.556 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:04:41.602 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T10:04:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.861+0000 7f3d9b708640 1 Processor -- start 2023-10-16T10:04:41.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.861+0000 7f3d9b708640 1 -- start start 2023-10-16T10:04:41.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.863+0000 7f3d9b708640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d940caac0 0x7f3d940ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:41.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.863+0000 7f3d9b708640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d940cb030 con 0x7f3d9415ad70 2023-10-16T10:04:41.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.863+0000 7f3d9b708640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d940c91c0 con 0x7f3d9406a4b0 2023-10-16T10:04:41.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.863+0000 7f3d9b708640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d940c94a0 con 0x7f3d940caac0 2023-10-16T10:04:41.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.863+0000 7f3d994a4640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3d9415ad70 0x7f3d94069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39796/0 (socket says 172.21.15.84:39796) 2023-10-16T10:04:41.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.863+0000 7f3d994a4640 1 -- 172.21.15.84:0/4117071171 learned_addr learned my addr 172.21.15.84:0/4117071171 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:41.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.863+0000 7f3d99ca5640 1 --2- 172.21.15.84:0/4117071171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d940caac0 0x7f3d940ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:41.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.863+0000 7f3d8bfff640 1 -- 172.21.15.84:0/4117071171 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042466865 0 0) 0x7f3d940cb030 con 0x7f3d9415ad70 2023-10-16T10:04:41.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.864+0000 7f3d8bfff640 1 -- 172.21.15.84:0/4117071171 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d80003520 con 0x7f3d9415ad70 2023-10-16T10:04:41.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.864+0000 7f3d8bfff640 1 -- 172.21.15.84:0/4117071171 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3061338702 0 0) 0x7f3d80003520 con 0x7f3d9415ad70 2023-10-16T10:04:41.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.864+0000 7f3d8bfff640 1 -- 172.21.15.84:0/4117071171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d940caac0 msgr2=0x7f3d940ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:41.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.864+0000 7f3d8bfff640 1 --2- 172.21.15.84:0/4117071171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d940caac0 0x7f3d940ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:41.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.864+0000 7f3d8bfff640 1 -- 172.21.15.84:0/4117071171 >> v1:172.21.15.157:6789/0 conn(0x7f3d9406a4b0 legacy=0x7f3d940ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:41.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.864+0000 7f3d8bfff640 1 -- 172.21.15.84:0/4117071171 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d940c9780 con 0x7f3d9415ad70 2023-10-16T10:04:41.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.864+0000 7f3d8bfff640 1 -- 172.21.15.84:0/4117071171 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f3d90001670 con 0x7f3d9415ad70 2023-10-16T10:04:41.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.864+0000 7f3d8bfff640 1 -- 172.21.15.84:0/4117071171 >> v1:172.21.15.84:6789/0 conn(0x7f3d9415ad70 legacy=0x7f3d94069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:41.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.864+0000 7f3d8bfff640 1 --2- 172.21.15.84:0/4117071171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d940caac0 0x7f3d800039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:41.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.864+0000 7f3d8bfff640 1 --2- 172.21.15.84:0/4117071171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d80004020 0x7f3d800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:41.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.864+0000 7f3d8bfff640 1 --2- 172.21.15.84:0/4117071171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d80004d00 0x7f3d80005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:41.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.864+0000 7f3d8bfff640 1 -- 172.21.15.84:0/4117071171 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d940c94a0 con 0x7f3d80004020 2023-10-16T10:04:41.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.864+0000 7f3d8bfff640 1 -- 172.21.15.84:0/4117071171 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8000bbc0 con 0x7f3d80004d00 2023-10-16T10:04:41.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.864+0000 7f3d8bfff640 1 -- 172.21.15.84:0/4117071171 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8000bd80 con 0x7f3d940caac0 2023-10-16T10:04:41.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.864+0000 7f3d98ca3640 1 --2- 172.21.15.84:0/4117071171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d80004020 0x7f3d800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:41.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.865+0000 7f3d98ca3640 1 -- 172.21.15.84:0/4117071171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d940caac0 msgr2=0x7f3d800039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:41.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.865+0000 7f3d994a4640 1 --2- 172.21.15.84:0/4117071171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d940caac0 0x7f3d800039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:41.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.865+0000 7f3d99ca5640 1 --2- 172.21.15.84:0/4117071171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d80004d00 0x7f3d80005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:41.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.865+0000 7f3d98ca3640 1 --2- 172.21.15.84:0/4117071171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d940caac0 0x7f3d800039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:41.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.865+0000 7f3d98ca3640 1 -- 172.21.15.84:0/4117071171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d80004d00 msgr2=0x7f3d80005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:41.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.865+0000 7f3d98ca3640 1 --2- 172.21.15.84:0/4117071171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d80004d00 0x7f3d80005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:41.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.865+0000 7f3d98ca3640 1 -- 172.21.15.84:0/4117071171 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d80003520 con 0x7f3d80004020 2023-10-16T10:04:41.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.866+0000 7f3d98ca3640 1 --2- 172.21.15.84:0/4117071171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d80004020 0x7f3d800047c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3d84001750 tx=0x7f3d800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df8b452ae2b5bdc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:41.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.866+0000 7f3d8bfff640 1 -- 172.21.15.84:0/4117071171 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d840052d0 con 0x7f3d80004020 2023-10-16T10:04:41.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.866+0000 7f3d8bfff640 1 -- 172.21.15.84:0/4117071171 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8402e050 con 0x7f3d80004020 2023-10-16T10:04:41.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.866+0000 7f3d8bfff640 1 -- 172.21.15.84:0/4117071171 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84003830 con 0x7f3d80004020 2023-10-16T10:04:41.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.867+0000 7f3d9b708640 1 -- 172.21.15.84:0/4117071171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d80004020 msgr2=0x7f3d800047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:41.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.867+0000 7f3d9b708640 1 --2- 172.21.15.84:0/4117071171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d80004020 0x7f3d800047c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3d84001750 tx=0x7f3d800057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:41.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.867+0000 7f3d99ca5640 1 -- 172.21.15.84:0/4117071171 reap_dead start 2023-10-16T10:04:41.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.867+0000 7f3d9b708640 1 -- 172.21.15.84:0/4117071171 shutdown_connections 2023-10-16T10:04:41.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.867+0000 7f3d9b708640 1 -- 172.21.15.84:0/4117071171 >> 172.21.15.84:0/4117071171 conn(0x7f3d940ce9b0 msgr2=0x7f3d940cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:41.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.867+0000 7f3d9b708640 1 -- 172.21.15.84:0/4117071171 shutdown_connections 2023-10-16T10:04:41.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.867+0000 7f3d9b708640 1 -- 172.21.15.84:0/4117071171 wait complete. 2023-10-16T10:04:41.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.869+0000 7f3d9b708640 1 Processor -- start 2023-10-16T10:04:41.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.869+0000 7f3d9b708640 1 -- start start 2023-10-16T10:04:41.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.870+0000 7f3d9b708640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d80004020 0x7f3d940c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:41.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.870+0000 7f3d9b708640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d940cc580 0x7f3d94154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:41.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.870+0000 7f3d9b708640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d94154d60 0x7f3d9415cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:41.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.870+0000 7f3d9b708640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d940cca80 con 0x7f3d940cc580 2023-10-16T10:04:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.870+0000 7f3d9b708640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d940ccd60 con 0x7f3d80004020 2023-10-16T10:04:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.870+0000 7f3d9b708640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d940cd040 con 0x7f3d94154d60 2023-10-16T10:04:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.870+0000 7f3d98ca3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d940cc580 0x7f3d94154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.870+0000 7f3d98ca3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d940cc580 0x7f3d94154820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43986/0 (socket says 172.21.15.84:43986) 2023-10-16T10:04:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.870+0000 7f3d98ca3640 1 -- 172.21.15.84:0/1328208255 learned_addr learned my addr 172.21.15.84:0/1328208255 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.870+0000 7f3d994a4640 1 --2- 172.21.15.84:0/1328208255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d80004020 0x7f3d940c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.870+0000 7f3d99ca5640 1 --2- 172.21.15.84:0/1328208255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d94154d60 0x7f3d9415cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.870+0000 7f3d98ca3640 1 -- 172.21.15.84:0/1328208255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d94154d60 msgr2=0x7f3d9415cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.870+0000 7f3d98ca3640 1 --2- 172.21.15.84:0/1328208255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d94154d60 0x7f3d9415cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.870+0000 7f3d98ca3640 1 -- 172.21.15.84:0/1328208255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d80004020 msgr2=0x7f3d940c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.870+0000 7f3d98ca3640 1 --2- 172.21.15.84:0/1328208255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d80004020 0x7f3d940c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.870+0000 7f3d98ca3640 1 -- 172.21.15.84:0/1328208255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d940cd320 con 0x7f3d940cc580 2023-10-16T10:04:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.871+0000 7f3d98ca3640 1 --2- 172.21.15.84:0/1328208255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d940cc580 0x7f3d94154820 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f3d84008fd0 tx=0x7f3d84001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.871+0000 7f3d89ffb640 1 -- 172.21.15.84:0/1328208255 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d840076c0 con 0x7f3d940cc580 2023-10-16T10:04:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.871+0000 7f3d89ffb640 1 -- 172.21.15.84:0/1328208255 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d84007a30 con 0x7f3d940cc580 2023-10-16T10:04:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.871+0000 7f3d89ffb640 1 -- 172.21.15.84:0/1328208255 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84006c00 con 0x7f3d940cc580 2023-10-16T10:04:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.871+0000 7f3d9b708640 1 -- 172.21.15.84:0/1328208255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d940cd600 con 0x7f3d940cc580 2023-10-16T10:04:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.871+0000 7f3d9b708640 1 -- 172.21.15.84:0/1328208255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9415dab0 con 0x7f3d940cc580 2023-10-16T10:04:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.872+0000 7f3d9b708640 1 -- 172.21.15.84:0/1328208255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d940cd790 con 0x7f3d940cc580 2023-10-16T10:04:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.873+0000 7f3d89ffb640 1 -- 172.21.15.84:0/1328208255 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3d840035f0 con 0x7f3d940cc580 2023-10-16T10:04:41.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.874+0000 7f3d89ffb640 1 --2- 172.21.15.84:0/1328208255 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f3d7806af90 0x7f3d7806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:41.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.874+0000 7f3d89ffb640 1 -- 172.21.15.84:0/1328208255 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d840ab9a0 con 0x7f3d940cc580 2023-10-16T10:04:41.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.874+0000 7f3d994a4640 1 --2- 172.21.15.84:0/1328208255 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f3d7806af90 0x7f3d7806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:41.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.875+0000 7f3d994a4640 1 --2- 172.21.15.84:0/1328208255 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f3d7806af90 0x7f3d7806d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d90000ff0 tx=0x7f3d90009040 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:41.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:41.878+0000 7f3d89ffb640 1 -- 172.21.15.84:0/1328208255 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3d84076790 con 0x7f3d940cc580 2023-10-16T10:04:42.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.099+0000 7f3d9b708640 1 -- 172.21.15.84:0/1328208255 --> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d940cced0 con 0x7f3d7806af90 2023-10-16T10:04:42.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.101+0000 7f3d89ffb640 1 -- 172.21.15.84:0/1328208255 <== mgr.6279 v2:172.21.15.84:6832/2091947430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3d940cced0 con 0x7f3d7806af90 2023-10-16T10:04:42.103 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T10:04:42.103 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:04:42.103 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephfs" 2023-10-16T10:04:42.104 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:04:42.104 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T10:04:42.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.104+0000 7f3d9b708640 1 -- 172.21.15.84:0/1328208255 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f3d7806af90 msgr2=0x7f3d7806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:42.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.104+0000 7f3d9b708640 1 --2- 172.21.15.84:0/1328208255 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f3d7806af90 0x7f3d7806d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d90000ff0 tx=0x7f3d90009040 comp rx=0 tx=0).stop 2023-10-16T10:04:42.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.104+0000 7f3d9b708640 1 -- 172.21.15.84:0/1328208255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d940cc580 msgr2=0x7f3d94154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:42.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.104+0000 7f3d9b708640 1 --2- 172.21.15.84:0/1328208255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d940cc580 0x7f3d94154820 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f3d84008fd0 tx=0x7f3d84001120 comp rx=0 tx=0).stop 2023-10-16T10:04:42.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.104+0000 7f3d9b708640 1 -- 172.21.15.84:0/1328208255 shutdown_connections 2023-10-16T10:04:42.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.104+0000 7f3d9b708640 1 --2- 172.21.15.84:0/1328208255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d94154d60 0x7f3d9415cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:42.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.104+0000 7f3d9b708640 1 --2- 172.21.15.84:0/1328208255 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f3d7806af90 0x7f3d7806d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:42.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.104+0000 7f3d9b708640 1 --2- 172.21.15.84:0/1328208255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d940cc580 0x7f3d94154820 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:42.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.104+0000 7f3d9b708640 1 --2- 172.21.15.84:0/1328208255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d80004020 0x7f3d940c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:42.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.104+0000 7f3d9b708640 1 -- 172.21.15.84:0/1328208255 >> 172.21.15.84:0/1328208255 conn(0x7f3d940ce9b0 msgr2=0x7f3d94159fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:42.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.105+0000 7f3d9b708640 1 -- 172.21.15.84:0/1328208255 shutdown_connections 2023-10-16T10:04:42.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.105+0000 7f3d9b708640 1 -- 172.21.15.84:0/1328208255 wait complete. 2023-10-16T10:04:42.125 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T10:04:42.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.341+0000 7f616fc44640 1 Processor -- start 2023-10-16T10:04:42.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.341+0000 7f616fc44640 1 -- start start 2023-10-16T10:04:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.343+0000 7f616fc44640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f616815ad70 0x7f6168069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.343+0000 7f616fc44640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61680cb030 con 0x7f61680caac0 2023-10-16T10:04:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.343+0000 7f616fc44640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61680c91c0 con 0x7f61680c9ca0 2023-10-16T10:04:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.343+0000 7f616fc44640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61680c94a0 con 0x7f616815ad70 2023-10-16T10:04:42.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.343+0000 7f616d9e0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f616815ad70 0x7f6168069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:42.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.343+0000 7f616e1e1640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f61680caac0 0x7f61680ad4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39806/0 (socket says 172.21.15.84:39806) 2023-10-16T10:04:42.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.343+0000 7f616e1e1640 1 -- 172.21.15.84:0/2985721203 learned_addr learned my addr 172.21.15.84:0/2985721203 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:42.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.344+0000 7f616d9e0640 1 -- 172.21.15.84:0/2985721203 >> v1:172.21.15.157:6789/0 conn(0x7f61680c9ca0 legacy=0x7f61680ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:42.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.344+0000 7f616d9e0640 1 -- 172.21.15.84:0/2985721203 >> v1:172.21.15.84:6789/0 conn(0x7f61680caac0 legacy=0x7f61680ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:42.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.344+0000 7f616d9e0640 1 -- 172.21.15.84:0/2985721203 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61680c9780 con 0x7f616815ad70 2023-10-16T10:04:42.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.345+0000 7f616d9e0640 1 --2- 172.21.15.84:0/2985721203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f616815ad70 0x7f6168069da0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f61640097a0 tx=0x7f616402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc44e9ac4f96f8f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:42.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.345+0000 7f616c9de640 1 -- 172.21.15.84:0/2985721203 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164031830 con 0x7f616815ad70 2023-10-16T10:04:42.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.345+0000 7f616c9de640 1 -- 172.21.15.84:0/2985721203 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6164031990 con 0x7f616815ad70 2023-10-16T10:04:42.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.345+0000 7f616c9de640 1 -- 172.21.15.84:0/2985721203 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164031d20 con 0x7f616815ad70 2023-10-16T10:04:42.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.346+0000 7f616fc44640 1 -- 172.21.15.84:0/2985721203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f616815ad70 msgr2=0x7f6168069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:42.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.346+0000 7f616fc44640 1 --2- 172.21.15.84:0/2985721203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f616815ad70 0x7f6168069da0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f61640097a0 tx=0x7f616402b8d0 comp rx=0 tx=0).stop 2023-10-16T10:04:42.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.346+0000 7f616fc44640 1 -- 172.21.15.84:0/2985721203 shutdown_connections 2023-10-16T10:04:42.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.346+0000 7f616fc44640 1 --2- 172.21.15.84:0/2985721203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f616815ad70 0x7f6168069da0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:42.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.346+0000 7f616fc44640 1 -- 172.21.15.84:0/2985721203 >> 172.21.15.84:0/2985721203 conn(0x7f61680ce9b0 msgr2=0x7f61680cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:42.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.346+0000 7f616fc44640 1 -- 172.21.15.84:0/2985721203 shutdown_connections 2023-10-16T10:04:42.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.346+0000 7f616fc44640 1 -- 172.21.15.84:0/2985721203 wait complete. 2023-10-16T10:04:42.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.348+0000 7f616fc44640 1 Processor -- start 2023-10-16T10:04:42.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.348+0000 7f616fc44640 1 -- start start 2023-10-16T10:04:42.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.349+0000 7f616fc44640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61680c9ca0 0x7f61680aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:42.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.349+0000 7f616fc44640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61680caac0 0x7f61680c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:42.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.349+0000 7f616fc44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61680c7010 0x7f61680ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:42.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.349+0000 7f616fc44640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61680c81a0 con 0x7f61680c7010 2023-10-16T10:04:42.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.349+0000 7f616fc44640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61680c7520 con 0x7f61680c9ca0 2023-10-16T10:04:42.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.349+0000 7f616fc44640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61680c7800 con 0x7f61680caac0 2023-10-16T10:04:42.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.349+0000 7f616e1e1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61680c7010 0x7f61680ab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:42.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.349+0000 7f616e1e1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61680c7010 0x7f61680ab4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44008/0 (socket says 172.21.15.84:44008) 2023-10-16T10:04:42.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.349+0000 7f616e1e1640 1 -- 172.21.15.84:0/1143566474 learned_addr learned my addr 172.21.15.84:0/1143566474 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:42.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.349+0000 7f616d9e0640 1 --2- 172.21.15.84:0/1143566474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61680c9ca0 0x7f61680aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:42.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.349+0000 7f616e1e1640 1 -- 172.21.15.84:0/1143566474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61680caac0 msgr2=0x7f61680c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:42.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.349+0000 7f616e1e1640 1 --2- 172.21.15.84:0/1143566474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61680caac0 0x7f61680c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:42.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.349+0000 7f616e1e1640 1 -- 172.21.15.84:0/1143566474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61680c9ca0 msgr2=0x7f61680aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:42.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.349+0000 7f616e1e1640 1 --2- 172.21.15.84:0/1143566474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61680c9ca0 0x7f61680aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:42.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.349+0000 7f616e1e1640 1 -- 172.21.15.84:0/1143566474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6164009400 con 0x7f61680c7010 2023-10-16T10:04:42.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.350+0000 7f616d9e0640 1 --2- 172.21.15.84:0/1143566474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61680c9ca0 0x7f61680aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:42.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.350+0000 7f616e1e1640 1 --2- 172.21.15.84:0/1143566474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61680c7010 0x7f61680ab4d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f616000ca80 tx=0x7f616000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:42.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.350+0000 7f615e7fc640 1 -- 172.21.15.84:0/1143566474 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616000ac30 con 0x7f61680c7010 2023-10-16T10:04:42.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.350+0000 7f615e7fc640 1 -- 172.21.15.84:0/1143566474 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616000ad90 con 0x7f61680c7010 2023-10-16T10:04:42.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.350+0000 7f615e7fc640 1 -- 172.21.15.84:0/1143566474 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6160012600 con 0x7f61680c7010 2023-10-16T10:04:42.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.350+0000 7f616fc44640 1 -- 172.21.15.84:0/1143566474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61680c7ae0 con 0x7f61680c7010 2023-10-16T10:04:42.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.350+0000 7f616fc44640 1 -- 172.21.15.84:0/1143566474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61680af140 con 0x7f61680c7010 2023-10-16T10:04:42.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.352+0000 7f616fc44640 1 -- 172.21.15.84:0/1143566474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61680c82e0 con 0x7f61680c7010 2023-10-16T10:04:42.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.352+0000 7f615e7fc640 1 -- 172.21.15.84:0/1143566474 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 33) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6160007070 con 0x7f61680c7010 2023-10-16T10:04:42.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.353+0000 7f615e7fc640 1 --2- 172.21.15.84:0/1143566474 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f614c06af90 0x7f614c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:42.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.353+0000 7f615e7fc640 1 -- 172.21.15.84:0/1143566474 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6160089470 con 0x7f61680c7010 2023-10-16T10:04:42.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.354+0000 7f616d9e0640 1 --2- 172.21.15.84:0/1143566474 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f614c06af90 0x7f614c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:42.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.354+0000 7f616d9e0640 1 --2- 172.21.15.84:0/1143566474 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f614c06af90 0x7f614c06d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6164036000 tx=0x7f6164008030 comp rx=0 tx=0).ready entity=mgr.6279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:42.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.358+0000 7f615e7fc640 1 -- 172.21.15.84:0/1143566474 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f61680c82e0 con 0x7f61680c7010 2023-10-16T10:04:42.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.596+0000 7f616fc44640 1 -- 172.21.15.84:0/1143566474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f61680c82e0 con 0x7f61680c7010 2023-10-16T10:04:42.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.936+0000 7f615e7fc640 1 -- 172.21.15.84:0/1143566474 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v34) v1 ==== 92+0+0 (secure 0 0 0) 0x7f61680c82e0 con 0x7f61680c7010 2023-10-16T10:04:42.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.939+0000 7f616fc44640 1 -- 172.21.15.84:0/1143566474 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f614c06af90 msgr2=0x7f614c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:42.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.939+0000 7f616fc44640 1 --2- 172.21.15.84:0/1143566474 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f614c06af90 0x7f614c06d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6164036000 tx=0x7f6164008030 comp rx=0 tx=0).stop 2023-10-16T10:04:42.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.939+0000 7f616fc44640 1 -- 172.21.15.84:0/1143566474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61680c7010 msgr2=0x7f61680ab4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:42.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.939+0000 7f616fc44640 1 --2- 172.21.15.84:0/1143566474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61680c7010 0x7f61680ab4d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f616000ca80 tx=0x7f616000cf40 comp rx=0 tx=0).stop 2023-10-16T10:04:42.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.940+0000 7f616fc44640 1 -- 172.21.15.84:0/1143566474 shutdown_connections 2023-10-16T10:04:42.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.940+0000 7f616fc44640 1 --2- 172.21.15.84:0/1143566474 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7f614c06af90 0x7f614c06d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:42.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.940+0000 7f616fc44640 1 --2- 172.21.15.84:0/1143566474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61680c7010 0x7f61680ab4d0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:42.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.940+0000 7f616fc44640 1 --2- 172.21.15.84:0/1143566474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61680caac0 0x7f61680c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:42.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.940+0000 7f616fc44640 1 --2- 172.21.15.84:0/1143566474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61680c9ca0 0x7f61680aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:42.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.940+0000 7f616fc44640 1 -- 172.21.15.84:0/1143566474 >> 172.21.15.84:0/1143566474 conn(0x7f61680ce9b0 msgr2=0x7f616806b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:42.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.941+0000 7f616fc44640 1 -- 172.21.15.84:0/1143566474 shutdown_connections 2023-10-16T10:04:42.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:42.941+0000 7f616fc44640 1 -- 172.21.15.84:0/1143566474 wait complete. 2023-10-16T10:04:42.966 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T10:04:43.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.162+0000 7fedfa8d7640 1 Processor -- start 2023-10-16T10:04:43.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.162+0000 7fedfa8d7640 1 -- start start 2023-10-16T10:04:43.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.163+0000 7fedfa8d7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fedf415d030 0x7fedf40d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:43.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.164+0000 7fedfa8d7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf40b5d90 con 0x7fedf415d410 2023-10-16T10:04:43.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.164+0000 7fedfa8d7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf4160fc0 con 0x7fedf40cd260 2023-10-16T10:04:43.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.164+0000 7fedfa8d7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf40d2b50 con 0x7fedf415d030 2023-10-16T10:04:43.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.164+0000 7fedf37fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fedf415d410 0x7fedf4161450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39822/0 (socket says 172.21.15.84:39822) 2023-10-16T10:04:43.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.164+0000 7fedf37fe640 1 -- 172.21.15.84:0/1608844096 learned_addr learned my addr 172.21.15.84:0/1608844096 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:43.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.164+0000 7fedf3fff640 1 --2- 172.21.15.84:0/1608844096 >> v2:172.21.15.157:3300/0 conn(0x7fedf415d030 0x7fedf40d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:43.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.164+0000 7fedf2ffd640 1 -- 172.21.15.84:0/1608844096 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1403839786 0 0) 0x7fedf40b5d90 con 0x7fedf415d410 2023-10-16T10:04:43.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.164+0000 7fedf2ffd640 1 -- 172.21.15.84:0/1608844096 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feddc003520 con 0x7fedf415d410 2023-10-16T10:04:43.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.165+0000 7fedf2ffd640 1 -- 172.21.15.84:0/1608844096 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1398921987 0 0) 0x7feddc003520 con 0x7fedf415d410 2023-10-16T10:04:43.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.165+0000 7fedf2ffd640 1 -- 172.21.15.84:0/1608844096 >> v2:172.21.15.157:3300/0 conn(0x7fedf415d030 msgr2=0x7fedf40d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:43.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.165+0000 7fedf2ffd640 1 --2- 172.21.15.84:0/1608844096 >> v2:172.21.15.157:3300/0 conn(0x7fedf415d030 0x7fedf40d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.165+0000 7fedf2ffd640 1 -- 172.21.15.84:0/1608844096 >> v1:172.21.15.157:6789/0 conn(0x7fedf40cd260 legacy=0x7fedf40b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:43.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.165+0000 7fedf2ffd640 1 -- 172.21.15.84:0/1608844096 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf40d25d0 con 0x7fedf415d410 2023-10-16T10:04:43.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.165+0000 7fedf2ffd640 1 -- 172.21.15.84:0/1608844096 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fede8001670 con 0x7fedf415d410 2023-10-16T10:04:43.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.165+0000 7fedf2ffd640 1 -- 172.21.15.84:0/1608844096 >> v1:172.21.15.84:6789/0 conn(0x7fedf415d410 legacy=0x7fedf4161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:43.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.165+0000 7fedf2ffd640 1 --2- 172.21.15.84:0/1608844096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feddc0039c0 0x7feddc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:43.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.165+0000 7fedf2ffd640 1 --2- 172.21.15.84:0/1608844096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feddc0047c0 0x7feddc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:43.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.165+0000 7fedf2ffd640 1 --2- 172.21.15.84:0/1608844096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feddc005180 0x7feddc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:43.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.165+0000 7fedf2ffd640 1 -- 172.21.15.84:0/1608844096 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc00c030 con 0x7feddc0047c0 2023-10-16T10:04:43.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.165+0000 7fedf2ffd640 1 -- 172.21.15.84:0/1608844096 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc00c1a0 con 0x7feddc005180 2023-10-16T10:04:43.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.165+0000 7fedf2ffd640 1 -- 172.21.15.84:0/1608844096 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feddc00c360 con 0x7feddc0039c0 2023-10-16T10:04:43.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.165+0000 7fedf37fe640 1 --2- 172.21.15.84:0/1608844096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feddc0047c0 0x7feddc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:43.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.166+0000 7fedf3fff640 1 --2- 172.21.15.84:0/1608844096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feddc0039c0 0x7feddc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:43.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.166+0000 7fedf8e74640 1 --2- 172.21.15.84:0/1608844096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feddc005180 0x7feddc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:43.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.166+0000 7fedf37fe640 1 -- 172.21.15.84:0/1608844096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feddc0039c0 msgr2=0x7feddc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:43.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.166+0000 7fedf37fe640 1 --2- 172.21.15.84:0/1608844096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feddc0039c0 0x7feddc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.166+0000 7fedf37fe640 1 -- 172.21.15.84:0/1608844096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feddc005180 msgr2=0x7feddc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:43.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.166+0000 7fedf37fe640 1 --2- 172.21.15.84:0/1608844096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feddc005180 0x7feddc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.166+0000 7fedf37fe640 1 -- 172.21.15.84:0/1608844096 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc003520 con 0x7feddc0047c0 2023-10-16T10:04:43.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.166+0000 7fedf8e74640 1 --2- 172.21.15.84:0/1608844096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feddc005180 0x7feddc00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:43.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.166+0000 7fedf8e74640 1 -- 172.21.15.84:0/1608844096 reap_dead start 2023-10-16T10:04:43.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.167+0000 7fedf37fe640 1 --2- 172.21.15.84:0/1608844096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feddc0047c0 0x7feddc004c40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fede8007fb0 tx=0x7feddc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e35e45dcf707dd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:43.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.167+0000 7fedf2ffd640 1 -- 172.21.15.84:0/1608844096 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede8005ad0 con 0x7feddc0047c0 2023-10-16T10:04:43.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.167+0000 7fedf2ffd640 1 -- 172.21.15.84:0/1608844096 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede8005c30 con 0x7feddc0047c0 2023-10-16T10:04:43.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.167+0000 7fedf2ffd640 1 -- 172.21.15.84:0/1608844096 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede802c7c0 con 0x7feddc0047c0 2023-10-16T10:04:43.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.168+0000 7fedfa8d7640 1 -- 172.21.15.84:0/1608844096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feddc0047c0 msgr2=0x7feddc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:43.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.168+0000 7fedfa8d7640 1 --2- 172.21.15.84:0/1608844096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feddc0047c0 0x7feddc004c40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fede8007fb0 tx=0x7feddc0057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.168+0000 7fedfa8d7640 1 -- 172.21.15.84:0/1608844096 shutdown_connections 2023-10-16T10:04:43.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.168+0000 7fedfa8d7640 1 --2- 172.21.15.84:0/1608844096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feddc0047c0 0x7feddc004c40 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.168+0000 7fedfa8d7640 1 -- 172.21.15.84:0/1608844096 >> 172.21.15.84:0/1608844096 conn(0x7fedf40cf720 msgr2=0x7fedf40cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:43.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.168+0000 7fedfa8d7640 1 -- 172.21.15.84:0/1608844096 shutdown_connections 2023-10-16T10:04:43.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.168+0000 7fedfa8d7640 1 -- 172.21.15.84:0/1608844096 wait complete. 2023-10-16T10:04:43.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.171+0000 7fedfa8d7640 1 Processor -- start 2023-10-16T10:04:43.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.171+0000 7fedfa8d7640 1 -- start start 2023-10-16T10:04:43.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.172+0000 7fedfa8d7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feddc0047c0 0x7fedf40b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:43.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.172+0000 7fedfa8d7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedf40b4560 0x7fedf414c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:43.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.173+0000 7fedfa8d7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fedf4065ab0 0x7fedf4065f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:43.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.173+0000 7fedfa8d7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf40c6c50 con 0x7feddc0047c0 2023-10-16T10:04:43.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.173+0000 7fedfa8d7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf40c8f20 con 0x7fedf40b4560 2023-10-16T10:04:43.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.173+0000 7fedfa8d7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf4066470 con 0x7fedf4065ab0 2023-10-16T10:04:43.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.173+0000 7fedf3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feddc0047c0 0x7fedf40b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:43.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.173+0000 7fedf3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feddc0047c0 0x7fedf40b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44046/0 (socket says 172.21.15.84:44046) 2023-10-16T10:04:43.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.173+0000 7fedf3fff640 1 -- 172.21.15.84:0/494822575 learned_addr learned my addr 172.21.15.84:0/494822575 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:43.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.173+0000 7fedf8e74640 1 --2- 172.21.15.84:0/494822575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fedf4065ab0 0x7fedf4065f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:43.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.173+0000 7fedf37fe640 1 --2- 172.21.15.84:0/494822575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedf40b4560 0x7fedf414c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:43.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.173+0000 7fedf3fff640 1 -- 172.21.15.84:0/494822575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fedf4065ab0 msgr2=0x7fedf4065f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:43.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.173+0000 7fedf3fff640 1 --2- 172.21.15.84:0/494822575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fedf4065ab0 0x7fedf4065f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.173+0000 7fedf3fff640 1 -- 172.21.15.84:0/494822575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedf40b4560 msgr2=0x7fedf414c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:43.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.173+0000 7fedf3fff640 1 --2- 172.21.15.84:0/494822575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedf40b4560 0x7fedf414c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.173+0000 7fedf3fff640 1 -- 172.21.15.84:0/494822575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf40c96d0 con 0x7feddc0047c0 2023-10-16T10:04:43.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.173+0000 7fedf37fe640 1 --2- 172.21.15.84:0/494822575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedf40b4560 0x7fedf414c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:43.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.173+0000 7fedf3fff640 1 --2- 172.21.15.84:0/494822575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feddc0047c0 0x7fedf40b4c60 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fede40116d0 tx=0x7fede4011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:43.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.174+0000 7fedf0ff9640 1 -- 172.21.15.84:0/494822575 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede400f9c0 con 0x7feddc0047c0 2023-10-16T10:04:43.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.174+0000 7fedf0ff9640 1 -- 172.21.15.84:0/494822575 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede400fb20 con 0x7feddc0047c0 2023-10-16T10:04:43.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.174+0000 7fedf0ff9640 1 -- 172.21.15.84:0/494822575 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede400fe90 con 0x7feddc0047c0 2023-10-16T10:04:43.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.174+0000 7fedfa8d7640 1 -- 172.21.15.84:0/494822575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf40b49e0 con 0x7feddc0047c0 2023-10-16T10:04:43.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.174+0000 7fedfa8d7640 1 -- 172.21.15.84:0/494822575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf40c93f0 con 0x7feddc0047c0 2023-10-16T10:04:43.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.175+0000 7fedfa8d7640 1 -- 172.21.15.84:0/494822575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf40c6ec0 con 0x7feddc0047c0 2023-10-16T10:04:43.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.176+0000 7fedf0ff9640 1 -- 172.21.15.84:0/494822575 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 34) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fede4015460 con 0x7feddc0047c0 2023-10-16T10:04:43.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.177+0000 7fedf0ff9640 1 --2- 172.21.15.84:0/494822575 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fede006afe0 0x7fede006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:43.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.178+0000 7fedf37fe640 1 -- 172.21.15.84:0/494822575 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fede006afe0 msgr2=0x7fede006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2091947430 2023-10-16T10:04:43.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.178+0000 7fedf0ff9640 1 -- 172.21.15.84:0/494822575 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fede4089d60 con 0x7feddc0047c0 2023-10-16T10:04:43.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.178+0000 7fedf37fe640 1 --2- 172.21.15.84:0/494822575 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fede006afe0 0x7fede006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:43.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.181+0000 7fedf0ff9640 1 -- 172.21.15.84:0/494822575 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fede4054cf0 con 0x7feddc0047c0 2023-10-16T10:04:43.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.379+0000 7fedf37fe640 1 -- 172.21.15.84:0/494822575 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fede006afe0 msgr2=0x7fede006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2091947430 2023-10-16T10:04:43.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.379+0000 7fedf37fe640 1 --2- 172.21.15.84:0/494822575 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fede006afe0 0x7fede006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:04:43.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.389+0000 7fedfa8d7640 1 -- 172.21.15.84:0/494822575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fedf40b2300 con 0x7feddc0047c0 2023-10-16T10:04:43.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.393+0000 7fedf0ff9640 1 -- 172.21.15.84:0/494822575 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7fede40220d0 con 0x7feddc0047c0 2023-10-16T10:04:43.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.397+0000 7fedfa8d7640 1 -- 172.21.15.84:0/494822575 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fede006afe0 msgr2=0x7fede006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:43.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.397+0000 7fedfa8d7640 1 --2- 172.21.15.84:0/494822575 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fede006afe0 0x7fede006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.397+0000 7fedfa8d7640 1 -- 172.21.15.84:0/494822575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feddc0047c0 msgr2=0x7fedf40b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:43.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.397+0000 7fedfa8d7640 1 --2- 172.21.15.84:0/494822575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feddc0047c0 0x7fedf40b4c60 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fede40116d0 tx=0x7fede4011b90 comp rx=0 tx=0).stop 2023-10-16T10:04:43.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.397+0000 7fedfa8d7640 1 -- 172.21.15.84:0/494822575 shutdown_connections 2023-10-16T10:04:43.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.397+0000 7fedfa8d7640 1 --2- 172.21.15.84:0/494822575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fedf4065ab0 0x7fedf4065f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.397+0000 7fedfa8d7640 1 --2- 172.21.15.84:0/494822575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedf40b4560 0x7fedf414c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.397+0000 7fedfa8d7640 1 --2- 172.21.15.84:0/494822575 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fede006afe0 0x7fede006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.397+0000 7fedfa8d7640 1 --2- 172.21.15.84:0/494822575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feddc0047c0 0x7fedf40b4c60 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.397+0000 7fedfa8d7640 1 -- 172.21.15.84:0/494822575 >> 172.21.15.84:0/494822575 conn(0x7fedf40cf720 msgr2=0x7fedf415c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:43.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.397+0000 7fedfa8d7640 1 -- 172.21.15.84:0/494822575 shutdown_connections 2023-10-16T10:04:43.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.397+0000 7fedfa8d7640 1 -- 172.21.15.84:0/494822575 wait complete. 2023-10-16T10:04:43.424 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T10:04:43.494 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:43.492+0000 7f2095c2e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:04:43.509 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:43.507+0000 7f6326413040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:04:43.548 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:43.546+0000 7f2095c2e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:04:43.565 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:43.564+0000 7f6326413040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:04:43.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.624+0000 7ff3748ba640 1 Processor -- start 2023-10-16T10:04:43.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.624+0000 7ff3748ba640 1 -- start start 2023-10-16T10:04:43.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.626+0000 7ff3748ba640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff37015cd70 0x7ff3700d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:43.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.626+0000 7ff3748ba640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff3700b8480 0x7ff3700cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:43.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.626+0000 7ff3748ba640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff37015d150 con 0x7ff3700d0be0 2023-10-16T10:04:43.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.626+0000 7ff3748ba640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3700d1180 con 0x7ff3700b8480 2023-10-16T10:04:43.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.626+0000 7ff3748ba640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3700d1460 con 0x7ff37015cd70 2023-10-16T10:04:43.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.626+0000 7ff367fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff3700b8480 0x7ff3700cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:43.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.626+0000 7ff367fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff3700b8480 0x7ff3700cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44054/0 (socket says 172.21.15.84:44054) 2023-10-16T10:04:43.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.626+0000 7ff367fff640 1 -- 172.21.15.84:0/2935001146 learned_addr learned my addr 172.21.15.84:0/2935001146 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:43.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.626+0000 7ff36ed9d640 1 --2- 172.21.15.84:0/2935001146 >> v2:172.21.15.157:3300/0 conn(0x7ff37015cd70 0x7ff3700d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:43.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.626+0000 7ff367fff640 1 -- 172.21.15.84:0/2935001146 >> v2:172.21.15.157:3300/0 conn(0x7ff37015cd70 msgr2=0x7ff3700d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:43.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.626+0000 7ff367fff640 1 --2- 172.21.15.84:0/2935001146 >> v2:172.21.15.157:3300/0 conn(0x7ff37015cd70 0x7ff3700d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.626+0000 7ff367fff640 1 -- 172.21.15.84:0/2935001146 >> v1:172.21.15.157:6789/0 conn(0x7ff3700d0be0 legacy=0x7ff3700b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:43.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.626+0000 7ff367fff640 1 -- 172.21.15.84:0/2935001146 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3700d1740 con 0x7ff3700b8480 2023-10-16T10:04:43.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.626+0000 7ff36ed9d640 1 --2- 172.21.15.84:0/2935001146 >> v2:172.21.15.157:3300/0 conn(0x7ff37015cd70 0x7ff3700d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:43.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.627+0000 7ff367fff640 1 --2- 172.21.15.84:0/2935001146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3700b8480 0x7ff3700cd7c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff3700b56a0 tx=0x7ff35802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3018835a724dfc4b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:43.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.628+0000 7ff36e59c640 1 -- 172.21.15.84:0/2935001146 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3580317e0 con 0x7ff3700b8480 2023-10-16T10:04:43.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.628+0000 7ff36e59c640 1 -- 172.21.15.84:0/2935001146 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff358031940 con 0x7ff3700b8480 2023-10-16T10:04:43.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.628+0000 7ff36e59c640 1 -- 172.21.15.84:0/2935001146 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff358031cd0 con 0x7ff3700b8480 2023-10-16T10:04:43.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.628+0000 7ff3748ba640 1 -- 172.21.15.84:0/2935001146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3700b8480 msgr2=0x7ff3700cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:43.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.628+0000 7ff3748ba640 1 --2- 172.21.15.84:0/2935001146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3700b8480 0x7ff3700cd7c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff3700b56a0 tx=0x7ff35802b880 comp rx=0 tx=0).stop 2023-10-16T10:04:43.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.629+0000 7ff3748ba640 1 -- 172.21.15.84:0/2935001146 shutdown_connections 2023-10-16T10:04:43.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.629+0000 7ff3748ba640 1 --2- 172.21.15.84:0/2935001146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3700b8480 0x7ff3700cd7c0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.629+0000 7ff3748ba640 1 --2- 172.21.15.84:0/2935001146 >> v2:172.21.15.157:3300/0 conn(0x7ff37015cd70 0x7ff3700d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.629+0000 7ff3748ba640 1 -- 172.21.15.84:0/2935001146 >> 172.21.15.84:0/2935001146 conn(0x7ff3700cf570 msgr2=0x7ff3700cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:43.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.629+0000 7ff3748ba640 1 -- 172.21.15.84:0/2935001146 shutdown_connections 2023-10-16T10:04:43.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.629+0000 7ff3748ba640 1 -- 172.21.15.84:0/2935001146 wait complete. 2023-10-16T10:04:43.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.631+0000 7ff3748ba640 1 Processor -- start 2023-10-16T10:04:43.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.631+0000 7ff3748ba640 1 -- start start 2023-10-16T10:04:43.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.633+0000 7ff3748ba640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3700b8480 0x7ff3701f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:43.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.633+0000 7ff3748ba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3700d0be0 0x7ff3700a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:43.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.633+0000 7ff3748ba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff37015cd70 0x7ff3700a9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:43.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.633+0000 7ff3748ba640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff370149cb0 con 0x7ff37015cd70 2023-10-16T10:04:43.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.633+0000 7ff3748ba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3700cc560 con 0x7ff3700d0be0 2023-10-16T10:04:43.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.633+0000 7ff3748ba640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3701f22d0 con 0x7ff3700b8480 2023-10-16T10:04:43.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.633+0000 7ff36f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff37015cd70 0x7ff3700a9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:43.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.633+0000 7ff36f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff37015cd70 0x7ff3700a9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44064/0 (socket says 172.21.15.84:44064) 2023-10-16T10:04:43.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.633+0000 7ff36f59e640 1 -- 172.21.15.84:0/117920719 learned_addr learned my addr 172.21.15.84:0/117920719 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:43.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.634+0000 7ff36ed9d640 1 --2- 172.21.15.84:0/117920719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3700b8480 0x7ff3701f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:43.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.634+0000 7ff367fff640 1 --2- 172.21.15.84:0/117920719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3700d0be0 0x7ff3700a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:43.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.634+0000 7ff36f59e640 1 -- 172.21.15.84:0/117920719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3700b8480 msgr2=0x7ff3701f1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:43.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.634+0000 7ff36f59e640 1 --2- 172.21.15.84:0/117920719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3700b8480 0x7ff3701f1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.634+0000 7ff36f59e640 1 -- 172.21.15.84:0/117920719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3700d0be0 msgr2=0x7ff3700a94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:43.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.634+0000 7ff36f59e640 1 --2- 172.21.15.84:0/117920719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3700d0be0 0x7ff3700a94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.634+0000 7ff36f59e640 1 -- 172.21.15.84:0/117920719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3580094d0 con 0x7ff37015cd70 2023-10-16T10:04:43.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.634+0000 7ff36ed9d640 1 --2- 172.21.15.84:0/117920719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3700b8480 0x7ff3701f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:43.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.634+0000 7ff36f59e640 1 --2- 172.21.15.84:0/117920719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff37015cd70 0x7ff3700a9a30 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff36800c780 tx=0x7ff36800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:43.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.634+0000 7ff3677fe640 1 -- 172.21.15.84:0/117920719 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff368003d80 con 0x7ff37015cd70 2023-10-16T10:04:43.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.634+0000 7ff3677fe640 1 -- 172.21.15.84:0/117920719 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36800ce30 con 0x7ff37015cd70 2023-10-16T10:04:43.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.634+0000 7ff3748ba640 1 -- 172.21.15.84:0/117920719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3700cbcc0 con 0x7ff37015cd70 2023-10-16T10:04:43.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.634+0000 7ff3677fe640 1 -- 172.21.15.84:0/117920719 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff368005290 con 0x7ff37015cd70 2023-10-16T10:04:43.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.634+0000 7ff3748ba640 1 -- 172.21.15.84:0/117920719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3700cc280 con 0x7ff37015cd70 2023-10-16T10:04:43.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.636+0000 7ff3748ba640 1 -- 172.21.15.84:0/117920719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff370149e20 con 0x7ff37015cd70 2023-10-16T10:04:43.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.637+0000 7ff3677fe640 1 -- 172.21.15.84:0/117920719 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 34) v1 ==== 89348+0+0 (secure 0 0 0) 0x7ff3680053f0 con 0x7ff37015cd70 2023-10-16T10:04:43.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.638+0000 7ff3677fe640 1 --2- 172.21.15.84:0/117920719 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7ff35006afe0 0x7ff35006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:43.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.638+0000 7ff3677fe640 1 -- 172.21.15.84:0/117920719 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff368088f80 con 0x7ff37015cd70 2023-10-16T10:04:43.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.638+0000 7ff36ed9d640 1 -- 172.21.15.84:0/117920719 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7ff35006afe0 msgr2=0x7ff35006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2091947430 2023-10-16T10:04:43.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.638+0000 7ff36ed9d640 1 --2- 172.21.15.84:0/117920719 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7ff35006afe0 0x7ff35006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:43.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.641+0000 7ff3677fe640 1 -- 172.21.15.84:0/117920719 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff368053f10 con 0x7ff37015cd70 2023-10-16T10:04:43.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.821+0000 7ff3748ba640 1 -- 172.21.15.84:0/117920719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7ff37004e9b0 con 0x7ff37015cd70 2023-10-16T10:04:43.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.826+0000 7ff3677fe640 1 -- 172.21.15.84:0/117920719 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff368017070 con 0x7ff37015cd70 2023-10-16T10:04:43.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.829+0000 7ff3748ba640 1 -- 172.21.15.84:0/117920719 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7ff35006afe0 msgr2=0x7ff35006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:43.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.829+0000 7ff3748ba640 1 --2- 172.21.15.84:0/117920719 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7ff35006afe0 0x7ff35006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.829+0000 7ff3748ba640 1 -- 172.21.15.84:0/117920719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff37015cd70 msgr2=0x7ff3700a9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:43.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.829+0000 7ff3748ba640 1 --2- 172.21.15.84:0/117920719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff37015cd70 0x7ff3700a9a30 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff36800c780 tx=0x7ff36800cc40 comp rx=0 tx=0).stop 2023-10-16T10:04:43.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.829+0000 7ff3748ba640 1 -- 172.21.15.84:0/117920719 shutdown_connections 2023-10-16T10:04:43.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.829+0000 7ff3748ba640 1 --2- 172.21.15.84:0/117920719 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7ff35006afe0 0x7ff35006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.829+0000 7ff3748ba640 1 --2- 172.21.15.84:0/117920719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff37015cd70 0x7ff3700a9a30 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.829+0000 7ff3748ba640 1 --2- 172.21.15.84:0/117920719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3700d0be0 0x7ff3700a94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.829+0000 7ff3748ba640 1 --2- 172.21.15.84:0/117920719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3700b8480 0x7ff3701f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:43.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.829+0000 7ff3748ba640 1 -- 172.21.15.84:0/117920719 >> 172.21.15.84:0/117920719 conn(0x7ff3700cf570 msgr2=0x7ff3700d2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:43.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.829+0000 7ff3748ba640 1 -- 172.21.15.84:0/117920719 shutdown_connections 2023-10-16T10:04:43.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:43.829+0000 7ff3748ba640 1 -- 172.21.15.84:0/117920719 wait complete. 2023-10-16T10:04:43.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:43.847 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-16T10:04:43.900 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-10-16T10:04:44.004 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:44.002+0000 7f2095c2e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:04:44.034 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:44.032+0000 7f6326413040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:04:44.056 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:44.054+0000 7f2095c2e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:04:44.087 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:44.086+0000 7f6326413040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:04:44.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.094+0000 7fe795d33640 1 Processor -- start 2023-10-16T10:04:44.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.094+0000 7fe795d33640 1 -- start start 2023-10-16T10:04:44.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.095+0000 7fe795d33640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe790065ab0 0x7fe790161100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:44.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.096+0000 7fe795d33640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7900ac890 con 0x7fe790160530 2023-10-16T10:04:44.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.096+0000 7fe795d33640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7900ae800 con 0x7fe790069fa0 2023-10-16T10:04:44.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.096+0000 7fe795d33640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7900ae1f0 con 0x7fe790065ab0 2023-10-16T10:04:44.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.096+0000 7fe78f7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe790160530 0x7fe7901609f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39830/0 (socket says 172.21.15.84:39830) 2023-10-16T10:04:44.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.096+0000 7fe78f7fe640 1 -- 172.21.15.84:0/4150868929 learned_addr learned my addr 172.21.15.84:0/4150868929 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:44.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.096+0000 7fe787fff640 1 --2- 172.21.15.84:0/4150868929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe790065ab0 0x7fe790161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:44.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.096+0000 7fe78effd640 1 -- 172.21.15.84:0/4150868929 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697084580 0 0) 0x7fe7900ac890 con 0x7fe790160530 2023-10-16T10:04:44.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.096+0000 7fe78effd640 1 -- 172.21.15.84:0/4150868929 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe77c003520 con 0x7fe790160530 2023-10-16T10:04:44.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.097+0000 7fe787fff640 1 -- 172.21.15.84:0/4150868929 >> v1:172.21.15.157:6789/0 conn(0x7fe790069fa0 legacy=0x7fe79006a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:44.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.097+0000 7fe787fff640 1 -- 172.21.15.84:0/4150868929 >> v1:172.21.15.84:6789/0 conn(0x7fe790160530 legacy=0x7fe7901609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:44.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.097+0000 7fe787fff640 1 -- 172.21.15.84:0/4150868929 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7900acae0 con 0x7fe790065ab0 2023-10-16T10:04:44.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.098+0000 7fe787fff640 1 --2- 172.21.15.84:0/4150868929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe790065ab0 0x7fe790161100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe7780097d0 tx=0x7fe77802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6faa994113be4a93 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:44.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.098+0000 7fe78effd640 1 -- 172.21.15.84:0/4150868929 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778031800 con 0x7fe790065ab0 2023-10-16T10:04:44.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.098+0000 7fe78effd640 1 -- 172.21.15.84:0/4150868929 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe778031960 con 0x7fe790065ab0 2023-10-16T10:04:44.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.098+0000 7fe78effd640 1 -- 172.21.15.84:0/4150868929 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778031cd0 con 0x7fe790065ab0 2023-10-16T10:04:44.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.099+0000 7fe795d33640 1 -- 172.21.15.84:0/4150868929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe790065ab0 msgr2=0x7fe790161100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:44.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.099+0000 7fe795d33640 1 --2- 172.21.15.84:0/4150868929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe790065ab0 0x7fe790161100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe7780097d0 tx=0x7fe77802b8a0 comp rx=0 tx=0).stop 2023-10-16T10:04:44.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.099+0000 7fe795d33640 1 -- 172.21.15.84:0/4150868929 shutdown_connections 2023-10-16T10:04:44.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.099+0000 7fe795d33640 1 --2- 172.21.15.84:0/4150868929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe790065ab0 0x7fe790161100 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:44.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.099+0000 7fe795d33640 1 -- 172.21.15.84:0/4150868929 >> 172.21.15.84:0/4150868929 conn(0x7fe79006bc10 msgr2=0x7fe79006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:44.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.099+0000 7fe795d33640 1 -- 172.21.15.84:0/4150868929 shutdown_connections 2023-10-16T10:04:44.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.099+0000 7fe795d33640 1 -- 172.21.15.84:0/4150868929 wait complete. 2023-10-16T10:04:44.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.101+0000 7fe795d33640 1 Processor -- start 2023-10-16T10:04:44.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.101+0000 7fe795d33640 1 -- start start 2023-10-16T10:04:44.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.103+0000 7fe795d33640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe790065ab0 0x7fe7900aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:44.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.103+0000 7fe795d33640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe790069fa0 0x7fe79014c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:44.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.103+0000 7fe795d33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe790160530 0x7fe79014c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:44.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.103+0000 7fe795d33640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7900c2ce0 con 0x7fe790069fa0 2023-10-16T10:04:44.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.103+0000 7fe795d33640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79014cd00 con 0x7fe790160530 2023-10-16T10:04:44.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.103+0000 7fe795d33640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79014cf70 con 0x7fe790065ab0 2023-10-16T10:04:44.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.103+0000 7fe787fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe790069fa0 0x7fe79014c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:44.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.103+0000 7fe787fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe790069fa0 0x7fe79014c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44080/0 (socket says 172.21.15.84:44080) 2023-10-16T10:04:44.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.103+0000 7fe787fff640 1 -- 172.21.15.84:0/2619031404 learned_addr learned my addr 172.21.15.84:0/2619031404 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:44.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.103+0000 7fe78f7fe640 1 --2- 172.21.15.84:0/2619031404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe790065ab0 0x7fe7900aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:44.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.103+0000 7fe78ffff640 1 --2- 172.21.15.84:0/2619031404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe790160530 0x7fe79014c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:44.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.103+0000 7fe787fff640 1 -- 172.21.15.84:0/2619031404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe790065ab0 msgr2=0x7fe7900aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:44.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.103+0000 7fe787fff640 1 --2- 172.21.15.84:0/2619031404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe790065ab0 0x7fe7900aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:44.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.103+0000 7fe787fff640 1 -- 172.21.15.84:0/2619031404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe790160530 msgr2=0x7fe79014c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:44.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.103+0000 7fe787fff640 1 --2- 172.21.15.84:0/2619031404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe790160530 0x7fe79014c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:44.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.104+0000 7fe787fff640 1 -- 172.21.15.84:0/2619031404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe778009400 con 0x7fe790069fa0 2023-10-16T10:04:44.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.104+0000 7fe787fff640 1 --2- 172.21.15.84:0/2619031404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe790069fa0 0x7fe79014c280 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe7780097d0 tx=0x7fe7780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:44.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.104+0000 7fe78cff9640 1 -- 172.21.15.84:0/2619031404 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778003810 con 0x7fe790069fa0 2023-10-16T10:04:44.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.104+0000 7fe78cff9640 1 -- 172.21.15.84:0/2619031404 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe778003970 con 0x7fe790069fa0 2023-10-16T10:04:44.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.104+0000 7fe795d33640 1 -- 172.21.15.84:0/2619031404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79014a390 con 0x7fe790069fa0 2023-10-16T10:04:44.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.104+0000 7fe78cff9640 1 -- 172.21.15.84:0/2619031404 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778003ce0 con 0x7fe790069fa0 2023-10-16T10:04:44.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.104+0000 7fe795d33640 1 -- 172.21.15.84:0/2619031404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79014a950 con 0x7fe790069fa0 2023-10-16T10:04:44.111 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:44.106+0000 7f2095c2e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:04:44.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.106+0000 7fe795d33640 1 -- 172.21.15.84:0/2619031404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79014ab40 con 0x7fe790069fa0 2023-10-16T10:04:44.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.106+0000 7fe78cff9640 1 -- 172.21.15.84:0/2619031404 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 34) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fe77803a450 con 0x7fe790069fa0 2023-10-16T10:04:44.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.108+0000 7fe78cff9640 1 --2- 172.21.15.84:0/2619031404 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe77006afe0 0x7fe77006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:44.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.108+0000 7fe78cff9640 1 -- 172.21.15.84:0/2619031404 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe77802e070 con 0x7fe790069fa0 2023-10-16T10:04:44.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.108+0000 7fe78f7fe640 1 -- 172.21.15.84:0/2619031404 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe77006afe0 msgr2=0x7fe77006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2091947430 2023-10-16T10:04:44.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.108+0000 7fe78f7fe640 1 --2- 172.21.15.84:0/2619031404 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe77006afe0 0x7fe77006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:44.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.112+0000 7fe78cff9640 1 -- 172.21.15.84:0/2619031404 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe778076830 con 0x7fe790069fa0 2023-10-16T10:04:44.140 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:44.139+0000 7f6326413040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:04:44.169 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:44.168+0000 7f2095c2e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:04:44.202 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:44.201+0000 7f6326413040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:04:44.218 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:44.217+0000 7f2095c2e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:04:44.252 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:44.250+0000 7f6326413040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:04:44.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.309+0000 7fe78f7fe640 1 -- 172.21.15.84:0/2619031404 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe77006afe0 msgr2=0x7fe77006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2091947430 2023-10-16T10:04:44.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.309+0000 7fe78f7fe640 1 --2- 172.21.15.84:0/2619031404 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe77006afe0 0x7fe77006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:04:44.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.326+0000 7fe795d33640 1 -- 172.21.15.84:0/2619031404 --> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7900ae490 con 0x7fe77006afe0 2023-10-16T10:04:44.329 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:44.328+0000 7f2095c2e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:04:44.363 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:44.361+0000 7f6326413040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:04:44.393 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:44.392+0000 7f2095c2e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:04:44.427 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:44.425+0000 7f6326413040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:04:44.443 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:44.442+0000 7f2095c2e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:04:44.476 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:44.475+0000 7f6326413040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:04:44.500 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:44.498+0000 7f2095c2e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:04:44.532 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:44.530+0000 7f6326413040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:04:44.549 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:44.548+0000 7f2095c2e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:04:44.581 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:44.580+0000 7f6326413040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:04:44.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.710+0000 7fe78f7fe640 1 -- 172.21.15.84:0/2619031404 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe77006afe0 msgr2=0x7fe77006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2091947430 2023-10-16T10:04:44.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:44.710+0000 7fe78f7fe640 1 --2- 172.21.15.84:0/2619031404 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe77006afe0 0x7fe77006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T10:04:44.928 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:44.926+0000 7f2095c2e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:04:44.957 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:44.956+0000 7f6326413040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:04:45.087 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:45.086+0000 7f2095c2e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:04:45.116 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:45.115+0000 7f6326413040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:04:45.196 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:45.194+0000 7f2095c2e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:04:45.224 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:45.222+0000 7f6326413040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:04:45.246 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:45.245+0000 7f2095c2e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:04:45.274 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:45.272+0000 7f6326413040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:04:45.353 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:45.351+0000 7f2095c2e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:04:45.379 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:45.378+0000 7f6326413040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:04:45.459 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:45.457+0000 7f2095c2e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:04:45.485 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:45.484+0000 7f6326413040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:04:45.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:45.511+0000 7fe78f7fe640 1 -- 172.21.15.84:0/2619031404 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe77006afe0 msgr2=0x7fe77006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2091947430 2023-10-16T10:04:45.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:45.511+0000 7fe78f7fe640 1 --2- 172.21.15.84:0/2619031404 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe77006afe0 0x7fe77006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T10:04:45.614 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:45.613+0000 7f2095c2e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:04:45.641 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:45.639+0000 7f6326413040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:04:45.663 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:45.662+0000 7f2095c2e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:04:45.691 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:45.689+0000 7f6326413040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:04:45.784 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:45.782+0000 7f2095c2e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:04:45.811 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:45.809+0000 7f6326413040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:04:46.607 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:46.605+0000 7f2095c2e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:04:46.653 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:46.651+0000 7f6326413040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:04:46.901 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:46.899+0000 7f2095c2e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:04:46.947 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:46.946+0000 7f6326413040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:04:46.958 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:46.956+0000 7f2095c2e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:04:47.005 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:47.004+0000 7f6326413040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:04:47.073 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:47.072+0000 7f2095c2e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:04:47.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:47.113+0000 7fe78f7fe640 1 -- 172.21.15.84:0/2619031404 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe77006afe0 msgr2=0x7fe77006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2091947430 2023-10-16T10:04:47.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:47.113+0000 7fe78f7fe640 1 --2- 172.21.15.84:0/2619031404 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe77006afe0 0x7fe77006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T10:04:47.123 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:47.121+0000 7f6326413040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:04:47.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:47.133+0000 7fe78cff9640 1 -- 172.21.15.84:0/2619031404 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 35) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fe778076c00 con 0x7fe790069fa0 2023-10-16T10:04:47.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:47.134+0000 7fe78cff9640 1 -- 172.21.15.84:0/2619031404 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe77006afe0 msgr2=0x7fe77006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:47.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:47.134+0000 7fe78cff9640 1 --2- 172.21.15.84:0/2619031404 >> [v2:172.21.15.84:6832/2091947430,v1:172.21.15.84:6833/2091947430] conn(0x7fe77006afe0 0x7fe77006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:47.465 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:47.464+0000 7f62a2b41640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:47.466 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:47.464+0000 7f62a2b41640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:47.466 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:47.464+0000 7f62a2b41640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:47.466 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:47.464+0000 7f62a2b41640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:47.467 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:47.464+0000 7f62a2b41640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:47.467 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:47.465+0000 7f62a5b87640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:47.467 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:47.465+0000 7f62a5b87640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:47.467 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:47.465+0000 7f62a5b87640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:47.468 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:47.465+0000 7f62a5b87640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:47.468 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:47.465+0000 7f62a5b87640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:48.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.172+0000 7fe78cff9640 1 -- 172.21.15.84:0/2619031404 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 36) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fe7780782a0 con 0x7fe790069fa0 2023-10-16T10:04:48.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.173+0000 7fe78cff9640 1 --2- 172.21.15.84:0/2619031404 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7fe7700355a0 0x7fe770035980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:48.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.174+0000 7fe78cff9640 1 -- 172.21.15.84:0/2619031404 --> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe770035ec0 con 0x7fe7700355a0 2023-10-16T10:04:48.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.179+0000 7fe78f7fe640 1 --2- 172.21.15.84:0/2619031404 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7fe7700355a0 0x7fe770035980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:48.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.194+0000 7fe78f7fe640 1 --2- 172.21.15.84:0/2619031404 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7fe7700355a0 0x7fe770035980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe7800011e0 tx=0x7fe780007d30 comp rx=0 tx=0).ready entity=mgr.6717 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:48.219 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:48.217+0000 7f62acb95640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-16T10:04:48.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.218+0000 7fe78cff9640 1 -- 172.21.15.84:0/2619031404 <== mgr.6717 v2:172.21.15.84:6832/2103121481 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7fe770035ec0 con 0x7fe7700355a0 2023-10-16T10:04:48.219 INFO:teuthology.orchestra.run.smithi084.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-16T10:04:48.241 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T10:04:48.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:48.243 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-16T10:04:48.303 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T10:04:48.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.523+0000 7f43b6a34640 1 Processor -- start 2023-10-16T10:04:48.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.523+0000 7f43b6a34640 1 -- start start 2023-10-16T10:04:48.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.524+0000 7f43b6a34640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43b0159e90 0x7f43b00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:48.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.524+0000 7f43b6a34640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f43b006c140 0x7f43b006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:48.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.524+0000 7f43b6a34640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b0070360 con 0x7f43b0159ab0 2023-10-16T10:04:48.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.524+0000 7f43b6a34640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b006cb10 con 0x7f43b006c140 2023-10-16T10:04:48.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.524+0000 7f43b6a34640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b015d890 con 0x7f43b0159e90 2023-10-16T10:04:48.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.525+0000 7f43b4fd1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f43b006c140 0x7f43b006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:48.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.525+0000 7f43af7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43b0159e90 0x7f43b00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:48.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.525+0000 7f43af7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43b0159e90 0x7f43b00ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:44144/0 (socket says 172.21.15.84:44144) 2023-10-16T10:04:48.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.525+0000 7f43af7fe640 1 -- 172.21.15.84:0/1118086396 learned_addr learned my addr 172.21.15.84:0/1118086396 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:48.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.526+0000 7f43af7fe640 1 -- 172.21.15.84:0/1118086396 >> v2:172.21.15.157:3300/0 conn(0x7f43b006c140 msgr2=0x7f43b006c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:48.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.526+0000 7f43af7fe640 1 --2- 172.21.15.84:0/1118086396 >> v2:172.21.15.157:3300/0 conn(0x7f43b006c140 0x7f43b006c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:48.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.526+0000 7f43af7fe640 1 -- 172.21.15.84:0/1118086396 >> v1:172.21.15.157:6789/0 conn(0x7f43b0159ab0 legacy=0x7f43b006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:48.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.526+0000 7f43af7fe640 1 -- 172.21.15.84:0/1118086396 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b00cd1f0 con 0x7f43b0159e90 2023-10-16T10:04:48.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.527+0000 7f43af7fe640 1 --2- 172.21.15.84:0/1118086396 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43b0159e90 0x7f43b00ce370 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f43a400bb40 tx=0x7f43a402e030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efa8b04618fa8e33 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:48.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.528+0000 7f43aeffd640 1 -- 172.21.15.84:0/1118086396 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a402ede0 con 0x7f43b0159e90 2023-10-16T10:04:48.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.528+0000 7f43aeffd640 1 -- 172.21.15.84:0/1118086396 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a4036440 con 0x7f43b0159e90 2023-10-16T10:04:48.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.528+0000 7f43aeffd640 1 -- 172.21.15.84:0/1118086396 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a40367d0 con 0x7f43b0159e90 2023-10-16T10:04:48.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.528+0000 7f43b6a34640 1 -- 172.21.15.84:0/1118086396 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43b0159e90 msgr2=0x7f43b00ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:48.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.528+0000 7f43b6a34640 1 --2- 172.21.15.84:0/1118086396 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43b0159e90 0x7f43b00ce370 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f43a400bb40 tx=0x7f43a402e030 comp rx=0 tx=0).stop 2023-10-16T10:04:48.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.528+0000 7f43b6a34640 1 -- 172.21.15.84:0/1118086396 shutdown_connections 2023-10-16T10:04:48.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.528+0000 7f43b6a34640 1 --2- 172.21.15.84:0/1118086396 >> v2:172.21.15.157:3300/0 conn(0x7f43b006c140 0x7f43b006c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:48.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.528+0000 7f43b6a34640 1 --2- 172.21.15.84:0/1118086396 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43b0159e90 0x7f43b00ce370 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:48.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.528+0000 7f43b6a34640 1 -- 172.21.15.84:0/1118086396 >> 172.21.15.84:0/1118086396 conn(0x7f43b00ce9b0 msgr2=0x7f43b00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:48.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.528+0000 7f43b6a34640 1 -- 172.21.15.84:0/1118086396 shutdown_connections 2023-10-16T10:04:48.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.528+0000 7f43b6a34640 1 -- 172.21.15.84:0/1118086396 wait complete. 2023-10-16T10:04:48.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.530+0000 7f43b6a34640 1 Processor -- start 2023-10-16T10:04:48.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.530+0000 7f43b6a34640 1 -- start start 2023-10-16T10:04:48.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.531+0000 7f43b6a34640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43b006c140 0x7f43b015ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:48.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.531+0000 7f43b6a34640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43b0159ab0 0x7f43b0157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:48.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.531+0000 7f43b6a34640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43b0159e90 0x7f43b015c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:48.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.531+0000 7f43b6a34640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b01576a0 con 0x7f43b0159ab0 2023-10-16T10:04:48.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.531+0000 7f43b6a34640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b015c910 con 0x7f43b0159e90 2023-10-16T10:04:48.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.531+0000 7f43b6a34640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00c2be0 con 0x7f43b006c140 2023-10-16T10:04:48.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.532+0000 7f43af7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43b0159ab0 0x7f43b0157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:48.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.532+0000 7f43af7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43b0159ab0 0x7f43b0157160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59572/0 (socket says 172.21.15.84:59572) 2023-10-16T10:04:48.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.532+0000 7f43af7fe640 1 -- 172.21.15.84:0/3727727080 learned_addr learned my addr 172.21.15.84:0/3727727080 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:48.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.532+0000 7f43b4fd1640 1 --2- 172.21.15.84:0/3727727080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43b0159e90 0x7f43b015c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:48.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.532+0000 7f43af7fe640 1 -- 172.21.15.84:0/3727727080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43b006c140 msgr2=0x7f43b015ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:48.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.532+0000 7f43af7fe640 1 --2- 172.21.15.84:0/3727727080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43b006c140 0x7f43b015ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:48.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.532+0000 7f43af7fe640 1 -- 172.21.15.84:0/3727727080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43b0159e90 msgr2=0x7f43b015c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:48.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.532+0000 7f43af7fe640 1 --2- 172.21.15.84:0/3727727080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43b0159e90 0x7f43b015c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:48.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.532+0000 7f43af7fe640 1 -- 172.21.15.84:0/3727727080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a400b770 con 0x7f43b0159ab0 2023-10-16T10:04:48.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.532+0000 7f43af7fe640 1 --2- 172.21.15.84:0/3727727080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43b0159ab0 0x7f43b0157160 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f43a400b2c0 tx=0x7f43a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:48.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.533+0000 7f43acff9640 1 -- 172.21.15.84:0/3727727080 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4034750 con 0x7f43b0159ab0 2023-10-16T10:04:48.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.533+0000 7f43acff9640 1 -- 172.21.15.84:0/3727727080 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a40348b0 con 0x7f43b0159ab0 2023-10-16T10:04:48.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.533+0000 7f43acff9640 1 -- 172.21.15.84:0/3727727080 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4034c40 con 0x7f43b0159ab0 2023-10-16T10:04:48.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.533+0000 7f43b6a34640 1 -- 172.21.15.84:0/3727727080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b015b420 con 0x7f43b0159ab0 2023-10-16T10:04:48.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.533+0000 7f43b6a34640 1 -- 172.21.15.84:0/3727727080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b015b9e0 con 0x7f43b0159ab0 2023-10-16T10:04:48.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.534+0000 7f43b6a34640 1 -- 172.21.15.84:0/3727727080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b004e160 con 0x7f43b0159ab0 2023-10-16T10:04:48.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.535+0000 7f43acff9640 1 -- 172.21.15.84:0/3727727080 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 36) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f43a4049360 con 0x7f43b0159ab0 2023-10-16T10:04:48.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.536+0000 7f43acff9640 1 --2- 172.21.15.84:0/3727727080 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f439c06ad10 0x7f439c06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:48.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.536+0000 7f43acff9640 1 -- 172.21.15.84:0/3727727080 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f43a40ab4f0 con 0x7f43b0159ab0 2023-10-16T10:04:48.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.536+0000 7f43affff640 1 --2- 172.21.15.84:0/3727727080 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f439c06ad10 0x7f439c06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:48.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.537+0000 7f43affff640 1 --2- 172.21.15.84:0/3727727080 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f439c06ad10 0x7f439c06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f43a0000f60 tx=0x7f43a00079b0 comp rx=0 tx=0).ready entity=mgr.6717 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:48.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.541+0000 7f43acff9640 1 -- 172.21.15.84:0/3727727080 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f43a4076780 con 0x7f43b0159ab0 2023-10-16T10:04:48.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:48.781+0000 7f43b6a34640 1 -- 172.21.15.84:0/3727727080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f43b015d590 con 0x7f43b0159ab0 2023-10-16T10:04:49.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.172+0000 7f43acff9640 1 -- 172.21.15.84:0/3727727080 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v37) v1 ==== 93+0+0 (secure 0 0 0) 0x7f43a403b020 con 0x7f43b0159ab0 2023-10-16T10:04:49.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.175+0000 7f43b6a34640 1 -- 172.21.15.84:0/3727727080 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f439c06ad10 msgr2=0x7f439c06d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:49.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.175+0000 7f43b6a34640 1 --2- 172.21.15.84:0/3727727080 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f439c06ad10 0x7f439c06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f43a0000f60 tx=0x7f43a00079b0 comp rx=0 tx=0).stop 2023-10-16T10:04:49.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.176+0000 7f43b6a34640 1 -- 172.21.15.84:0/3727727080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43b0159ab0 msgr2=0x7f43b0157160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:49.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.176+0000 7f43b6a34640 1 --2- 172.21.15.84:0/3727727080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43b0159ab0 0x7f43b0157160 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f43a400b2c0 tx=0x7f43a40046c0 comp rx=0 tx=0).stop 2023-10-16T10:04:49.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.176+0000 7f43b6a34640 1 -- 172.21.15.84:0/3727727080 shutdown_connections 2023-10-16T10:04:49.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.176+0000 7f43b6a34640 1 --2- 172.21.15.84:0/3727727080 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f439c06ad10 0x7f439c06d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:49.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.176+0000 7f43b6a34640 1 --2- 172.21.15.84:0/3727727080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43b0159e90 0x7f43b015c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:49.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.176+0000 7f43b6a34640 1 --2- 172.21.15.84:0/3727727080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43b0159ab0 0x7f43b0157160 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:49.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.176+0000 7f43b6a34640 1 --2- 172.21.15.84:0/3727727080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43b006c140 0x7f43b015ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:49.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.176+0000 7f43b6a34640 1 -- 172.21.15.84:0/3727727080 >> 172.21.15.84:0/3727727080 conn(0x7f43b00ce9b0 msgr2=0x7f43b0158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:49.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.177+0000 7f43b6a34640 1 -- 172.21.15.84:0/3727727080 shutdown_connections 2023-10-16T10:04:49.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.177+0000 7f43b6a34640 1 -- 172.21.15.84:0/3727727080 wait complete. 2023-10-16T10:04:49.202 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:04:49.202 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:04:49.309 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:04:49.404 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:04:49.405 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:04:49.405 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:04:49.457 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:04:49.457 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:04:49.551 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:04:49.703 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:49.701+0000 7fe455e32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:04:49.722 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:49.721+0000 7f95c9812040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:04:49.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.748+0000 7f6e9f6a2640 1 Processor -- start 2023-10-16T10:04:49.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.748+0000 7f6e9f6a2640 1 -- start start 2023-10-16T10:04:49.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.750+0000 7f6e9f6a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e9815aaa0 0x7f6e9815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:49.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.750+0000 7f6e9f6a2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6e9815ae80 0x7f6e980cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:49.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.750+0000 7f6e9f6a2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9815ed60 con 0x7f6e980ccaa0 2023-10-16T10:04:49.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.750+0000 7f6e9f6a2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e9806b810 con 0x7f6e9815ae80 2023-10-16T10:04:49.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.750+0000 7f6e9f6a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980707e0 con 0x7f6e9815aaa0 2023-10-16T10:04:49.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.751+0000 7f6e9cc3d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6e9815ae80 0x7f6e980cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:49.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.751+0000 7f6e9d43e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e9815aaa0 0x7f6e9815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:49.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.751+0000 7f6e9dc3f640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6e980ccaa0 0x7f6e98070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:43020/0 (socket says 172.21.15.84:43020) 2023-10-16T10:04:49.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.751+0000 7f6e9dc3f640 1 -- 172.21.15.84:0/4155008297 learned_addr learned my addr 172.21.15.84:0/4155008297 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:49.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.751+0000 7f6e9cc3d640 1 -- 172.21.15.84:0/4155008297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e9815aaa0 msgr2=0x7f6e9815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:49.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.751+0000 7f6e9cc3d640 1 --2- 172.21.15.84:0/4155008297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e9815aaa0 0x7f6e9815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:49.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.751+0000 7f6e9cc3d640 1 -- 172.21.15.84:0/4155008297 >> v1:172.21.15.157:6789/0 conn(0x7f6e980ccaa0 legacy=0x7f6e98070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:49.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.751+0000 7f6e9cc3d640 1 -- 172.21.15.84:0/4155008297 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9815ef90 con 0x7f6e9815ae80 2023-10-16T10:04:49.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.751+0000 7f6e9d43e640 1 --2- 172.21.15.84:0/4155008297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e9815aaa0 0x7f6e9815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:04:49.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.753+0000 7f6e9cc3d640 1 --2- 172.21.15.84:0/4155008297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e9815ae80 0x7f6e980cc560 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6e880097a0 tx=0x7f6e8802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d444c3988bfafdce server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:49.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.753+0000 7f6e8ffff640 1 -- 172.21.15.84:0/4155008297 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8802d040 con 0x7f6e9815ae80 2023-10-16T10:04:49.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.753+0000 7f6e8ffff640 1 -- 172.21.15.84:0/4155008297 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e88031810 con 0x7f6e9815ae80 2023-10-16T10:04:49.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.753+0000 7f6e8ffff640 1 -- 172.21.15.84:0/4155008297 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e88031ba0 con 0x7f6e9815ae80 2023-10-16T10:04:49.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.754+0000 7f6e9f6a2640 1 -- 172.21.15.84:0/4155008297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e9815ae80 msgr2=0x7f6e980cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:49.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.754+0000 7f6e9f6a2640 1 --2- 172.21.15.84:0/4155008297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e9815ae80 0x7f6e980cc560 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6e880097a0 tx=0x7f6e8802b8b0 comp rx=0 tx=0).stop 2023-10-16T10:04:49.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.754+0000 7f6e9f6a2640 1 -- 172.21.15.84:0/4155008297 shutdown_connections 2023-10-16T10:04:49.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.754+0000 7f6e9f6a2640 1 --2- 172.21.15.84:0/4155008297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e9815ae80 0x7f6e980cc560 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:49.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.754+0000 7f6e9f6a2640 1 --2- 172.21.15.84:0/4155008297 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e9815aaa0 0x7f6e9815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:49.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.754+0000 7f6e9f6a2640 1 -- 172.21.15.84:0/4155008297 >> 172.21.15.84:0/4155008297 conn(0x7f6e980cb410 msgr2=0x7f6e980cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:49.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.754+0000 7f6e9f6a2640 1 -- 172.21.15.84:0/4155008297 shutdown_connections 2023-10-16T10:04:49.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.754+0000 7f6e9f6a2640 1 -- 172.21.15.84:0/4155008297 wait complete. 2023-10-16T10:04:49.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.756+0000 7f6e9f6a2640 1 Processor -- start 2023-10-16T10:04:49.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.756+0000 7f6e9f6a2640 1 -- start start 2023-10-16T10:04:49.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.757+0000 7f6e9f6a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e980ccaa0 0x7f6e9814be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:49.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.757+0000 7f6e9f6a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e9815aaa0 0x7f6e9815b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:49.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.757+0000 7f6e9f6a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e9815ae80 0x7f6e980c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:49.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.757+0000 7f6e9f6a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9814c3d0 con 0x7f6e9815aaa0 2023-10-16T10:04:49.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.757+0000 7f6e9f6a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9815bb00 con 0x7f6e980ccaa0 2023-10-16T10:04:49.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.757+0000 7f6e9f6a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980c4810 con 0x7f6e9815ae80 2023-10-16T10:04:49.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.757+0000 7f6e9cc3d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e9815aaa0 0x7f6e9815b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:49.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.757+0000 7f6e9cc3d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e9815aaa0 0x7f6e9815b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59590/0 (socket says 172.21.15.84:59590) 2023-10-16T10:04:49.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.757+0000 7f6e9cc3d640 1 -- 172.21.15.84:0/4150825475 learned_addr learned my addr 172.21.15.84:0/4150825475 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:49.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.757+0000 7f6e9d43e640 1 --2- 172.21.15.84:0/4150825475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e980ccaa0 0x7f6e9814be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:49.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.758+0000 7f6e9dc3f640 1 --2- 172.21.15.84:0/4150825475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e9815ae80 0x7f6e980c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:49.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.758+0000 7f6e9cc3d640 1 -- 172.21.15.84:0/4150825475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e9815ae80 msgr2=0x7f6e980c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:49.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.758+0000 7f6e9cc3d640 1 --2- 172.21.15.84:0/4150825475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e9815ae80 0x7f6e980c4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:49.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.758+0000 7f6e9cc3d640 1 -- 172.21.15.84:0/4150825475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e980ccaa0 msgr2=0x7f6e9814be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:49.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.758+0000 7f6e9cc3d640 1 --2- 172.21.15.84:0/4150825475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e980ccaa0 0x7f6e9814be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:49.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.758+0000 7f6e9cc3d640 1 -- 172.21.15.84:0/4150825475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e94009400 con 0x7f6e9815aaa0 2023-10-16T10:04:49.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.758+0000 7f6e9cc3d640 1 --2- 172.21.15.84:0/4150825475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e9815aaa0 0x7f6e9815b320 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f6e8802bec0 tx=0x7f6e880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:49.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.758+0000 7f6e8dffb640 1 -- 172.21.15.84:0/4150825475 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8803ad60 con 0x7f6e9815aaa0 2023-10-16T10:04:49.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.758+0000 7f6e8dffb640 1 -- 172.21.15.84:0/4150825475 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e880423f0 con 0x7f6e9815aaa0 2023-10-16T10:04:49.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.758+0000 7f6e8dffb640 1 -- 172.21.15.84:0/4150825475 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e880427a0 con 0x7f6e9815aaa0 2023-10-16T10:04:49.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.758+0000 7f6e9f6a2640 1 -- 172.21.15.84:0/4150825475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e88009400 con 0x7f6e9815aaa0 2023-10-16T10:04:49.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.758+0000 7f6e9f6a2640 1 -- 172.21.15.84:0/4150825475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e980c3ee0 con 0x7f6e9815aaa0 2023-10-16T10:04:49.765 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:49.759+0000 7fe455e32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:04:49.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.760+0000 7f6e9f6a2640 1 -- 172.21.15.84:0/4150825475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9804e160 con 0x7f6e9815aaa0 2023-10-16T10:04:49.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.760+0000 7f6e8dffb640 1 -- 172.21.15.84:0/4150825475 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 37) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f6e8804b3b0 con 0x7f6e9815aaa0 2023-10-16T10:04:49.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.761+0000 7f6e8dffb640 1 --2- 172.21.15.84:0/4150825475 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f6e7c06af90 0x7f6e7c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:49.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.761+0000 7f6e8dffb640 1 -- 172.21.15.84:0/4150825475 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f6e880abb60 con 0x7f6e9815aaa0 2023-10-16T10:04:49.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.761+0000 7f6e9d43e640 1 -- 172.21.15.84:0/4150825475 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f6e7c06af90 msgr2=0x7f6e7c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2103121481 2023-10-16T10:04:49.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.761+0000 7f6e9d43e640 1 --2- 172.21.15.84:0/4150825475 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f6e7c06af90 0x7f6e7c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:49.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.766+0000 7f6e8dffb640 1 -- 172.21.15.84:0/4150825475 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6e88076880 con 0x7f6e9815aaa0 2023-10-16T10:04:49.782 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:49.781+0000 7f95c9812040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:04:49.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.958+0000 7f6e9f6a2640 1 -- 172.21.15.84:0/4150825475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e980cdc60 con 0x7f6e9815aaa0 2023-10-16T10:04:49.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.960+0000 7f6e8dffb640 1 -- 172.21.15.84:0/4150825475 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17042 (secure 0 0 0) 0x7f6e88035090 con 0x7f6e9815aaa0 2023-10-16T10:04:49.962 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:49.962 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":83,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:04:47.124638+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:04:30.728624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-16T10:04:31.866029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/3177129458":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/1145018607":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/1499920052":"2023-10-17T10:04:47.124608+0000","172.21.15.84:6833/2091947430":"2023-10-17T10:04:47.124608+0000","172.21.15.84:6832/2091947430":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/2509405053":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/3991569032":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/3599228496":"2023-10-17T10:04:47.124608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:04:49.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.962+0000 7f6e9d43e640 1 -- 172.21.15.84:0/4150825475 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f6e7c06af90 msgr2=0x7f6e7c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2103121481 2023-10-16T10:04:49.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.962+0000 7f6e9d43e640 1 --2- 172.21.15.84:0/4150825475 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f6e7c06af90 0x7f6e7c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:04:49.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.962+0000 7f6e9f6a2640 1 -- 172.21.15.84:0/4150825475 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f6e7c06af90 msgr2=0x7f6e7c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:49.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.962+0000 7f6e9f6a2640 1 --2- 172.21.15.84:0/4150825475 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f6e7c06af90 0x7f6e7c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:49.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.962+0000 7f6e9f6a2640 1 -- 172.21.15.84:0/4150825475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e9815aaa0 msgr2=0x7f6e9815b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:49.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.962+0000 7f6e9f6a2640 1 --2- 172.21.15.84:0/4150825475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e9815aaa0 0x7f6e9815b320 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f6e8802bec0 tx=0x7f6e880046c0 comp rx=0 tx=0).stop 2023-10-16T10:04:49.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.963+0000 7f6e9f6a2640 1 -- 172.21.15.84:0/4150825475 shutdown_connections 2023-10-16T10:04:49.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.963+0000 7f6e9f6a2640 1 --2- 172.21.15.84:0/4150825475 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f6e7c06af90 0x7f6e7c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:49.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.963+0000 7f6e9f6a2640 1 --2- 172.21.15.84:0/4150825475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e9815ae80 0x7f6e980c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:49.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.963+0000 7f6e9f6a2640 1 --2- 172.21.15.84:0/4150825475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e9815aaa0 0x7f6e9815b320 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:49.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.963+0000 7f6e9f6a2640 1 --2- 172.21.15.84:0/4150825475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e980ccaa0 0x7f6e9814be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:49.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.963+0000 7f6e9f6a2640 1 -- 172.21.15.84:0/4150825475 >> 172.21.15.84:0/4150825475 conn(0x7f6e980cb410 msgr2=0x7f6e9806cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:49.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.963+0000 7f6e9f6a2640 1 -- 172.21.15.84:0/4150825475 shutdown_connections 2023-10-16T10:04:49.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:49.963+0000 7f6e9f6a2640 1 -- 172.21.15.84:0/4150825475 wait complete. 2023-10-16T10:04:49.983 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:04:30.728624+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:04:31.866029+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:04:49.984 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:04:50.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.185+0000 7f0f2af1d640 1 Processor -- start 2023-10-16T10:04:50.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.186+0000 7f0f2af1d640 1 -- start start 2023-10-16T10:04:50.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.187+0000 7f0f2af1d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0f2415af80 0x7f0f240cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:50.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.187+0000 7f0f2af1d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0f2406faf0 0x7f0f2415f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:50.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.187+0000 7f0f2af1d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f2415eff0 con 0x7f0f24065ab0 2023-10-16T10:04:50.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.187+0000 7f0f2af1d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f240d0930 con 0x7f0f2406faf0 2023-10-16T10:04:50.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.187+0000 7f0f2af1d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f2415ecc0 con 0x7f0f2415af80 2023-10-16T10:04:50.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.188+0000 7f0f23fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0f2406faf0 0x7f0f2415f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:50.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.188+0000 7f0f23fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0f2406faf0 0x7f0f2415f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59596/0 (socket says 172.21.15.84:59596) 2023-10-16T10:04:50.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.188+0000 7f0f23fff640 1 -- 172.21.15.84:0/3353008713 learned_addr learned my addr 172.21.15.84:0/3353008713 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:50.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.188+0000 7f0f28cb9640 1 --2- 172.21.15.84:0/3353008713 >> v2:172.21.15.157:3300/0 conn(0x7f0f2415af80 0x7f0f240cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:50.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.188+0000 7f0f23fff640 1 -- 172.21.15.84:0/3353008713 >> v2:172.21.15.157:3300/0 conn(0x7f0f2415af80 msgr2=0x7f0f240cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:50.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.188+0000 7f0f23fff640 1 --2- 172.21.15.84:0/3353008713 >> v2:172.21.15.157:3300/0 conn(0x7f0f2415af80 0x7f0f240cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.188+0000 7f0f23fff640 1 -- 172.21.15.84:0/3353008713 >> v1:172.21.15.157:6789/0 conn(0x7f0f24065ab0 legacy=0x7f0f240ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:50.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.188+0000 7f0f23fff640 1 -- 172.21.15.84:0/3353008713 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2415b360 con 0x7f0f2406faf0 2023-10-16T10:04:50.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.189+0000 7f0f23fff640 1 --2- 172.21.15.84:0/3353008713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2406faf0 0x7f0f2415f480 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f0f240c7aa0 tx=0x7f0f1002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c4c01145298406 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:50.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.192+0000 7f0f237fe640 1 -- 172.21.15.84:0/3353008713 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10031870 con 0x7f0f2406faf0 2023-10-16T10:04:50.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.192+0000 7f0f237fe640 1 -- 172.21.15.84:0/3353008713 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f100319d0 con 0x7f0f2406faf0 2023-10-16T10:04:50.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.192+0000 7f0f237fe640 1 -- 172.21.15.84:0/3353008713 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10031d60 con 0x7f0f2406faf0 2023-10-16T10:04:50.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.192+0000 7f0f2af1d640 1 -- 172.21.15.84:0/3353008713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2406faf0 msgr2=0x7f0f2415f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:50.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.192+0000 7f0f2af1d640 1 --2- 172.21.15.84:0/3353008713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2406faf0 0x7f0f2415f480 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f0f240c7aa0 tx=0x7f0f1002b910 comp rx=0 tx=0).stop 2023-10-16T10:04:50.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.193+0000 7f0f2af1d640 1 -- 172.21.15.84:0/3353008713 shutdown_connections 2023-10-16T10:04:50.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.193+0000 7f0f2af1d640 1 --2- 172.21.15.84:0/3353008713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2406faf0 0x7f0f2415f480 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.193+0000 7f0f2af1d640 1 --2- 172.21.15.84:0/3353008713 >> v2:172.21.15.157:3300/0 conn(0x7f0f2415af80 0x7f0f240cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.193+0000 7f0f2af1d640 1 -- 172.21.15.84:0/3353008713 >> 172.21.15.84:0/3353008713 conn(0x7f0f240d0ab0 msgr2=0x7f0f240d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:50.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.193+0000 7f0f2af1d640 1 -- 172.21.15.84:0/3353008713 shutdown_connections 2023-10-16T10:04:50.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.193+0000 7f0f2af1d640 1 -- 172.21.15.84:0/3353008713 wait complete. 2023-10-16T10:04:50.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.194+0000 7f0f2af1d640 1 Processor -- start 2023-10-16T10:04:50.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.195+0000 7f0f2af1d640 1 -- start start 2023-10-16T10:04:50.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.196+0000 7f0f2af1d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f24065ab0 0x7f0f241500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:50.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.196+0000 7f0f2af1d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f2406faf0 0x7f0f2406c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:50.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.196+0000 7f0f2af1d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2415af80 0x7f0f240ae440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:50.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.196+0000 7f0f2af1d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2406aeb0 con 0x7f0f2415af80 2023-10-16T10:04:50.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.196+0000 7f0f2af1d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2406bcd0 con 0x7f0f24065ab0 2023-10-16T10:04:50.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.196+0000 7f0f2af1d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f241505f0 con 0x7f0f2406faf0 2023-10-16T10:04:50.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.196+0000 7f0f294ba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2415af80 0x7f0f240ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:50.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.196+0000 7f0f294ba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2415af80 0x7f0f240ae440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59600/0 (socket says 172.21.15.84:59600) 2023-10-16T10:04:50.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.196+0000 7f0f294ba640 1 -- 172.21.15.84:0/2905685985 learned_addr learned my addr 172.21.15.84:0/2905685985 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:50.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.196+0000 7f0f23fff640 1 --2- 172.21.15.84:0/2905685985 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f2406faf0 0x7f0f2406c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:50.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.196+0000 7f0f28cb9640 1 --2- 172.21.15.84:0/2905685985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f24065ab0 0x7f0f241500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:50.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.196+0000 7f0f294ba640 1 -- 172.21.15.84:0/2905685985 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f2406faf0 msgr2=0x7f0f2406c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:50.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.196+0000 7f0f294ba640 1 --2- 172.21.15.84:0/2905685985 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f2406faf0 0x7f0f2406c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.196+0000 7f0f294ba640 1 -- 172.21.15.84:0/2905685985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f24065ab0 msgr2=0x7f0f241500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:50.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.196+0000 7f0f294ba640 1 --2- 172.21.15.84:0/2905685985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f24065ab0 0x7f0f241500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.196+0000 7f0f294ba640 1 -- 172.21.15.84:0/2905685985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f100094d0 con 0x7f0f2415af80 2023-10-16T10:04:50.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.197+0000 7f0f28cb9640 1 --2- 172.21.15.84:0/2905685985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f24065ab0 0x7f0f241500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:50.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.197+0000 7f0f294ba640 1 --2- 172.21.15.84:0/2905685985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2415af80 0x7f0f240ae440 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c00b970 tx=0x7f0f1c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:50.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.197+0000 7f0f217fa640 1 -- 172.21.15.84:0/2905685985 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c00cb40 con 0x7f0f2415af80 2023-10-16T10:04:50.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.197+0000 7f0f217fa640 1 -- 172.21.15.84:0/2905685985 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1c00cca0 con 0x7f0f2415af80 2023-10-16T10:04:50.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.197+0000 7f0f2af1d640 1 -- 172.21.15.84:0/2905685985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2406b0c0 con 0x7f0f2415af80 2023-10-16T10:04:50.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.197+0000 7f0f217fa640 1 -- 172.21.15.84:0/2905685985 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c014680 con 0x7f0f2415af80 2023-10-16T10:04:50.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.197+0000 7f0f2af1d640 1 -- 172.21.15.84:0/2905685985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2406b680 con 0x7f0f2415af80 2023-10-16T10:04:50.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.198+0000 7f0f2af1d640 1 -- 172.21.15.84:0/2905685985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2406be40 con 0x7f0f2415af80 2023-10-16T10:04:50.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.200+0000 7f0f217fa640 1 -- 172.21.15.84:0/2905685985 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 37) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f0f1c01d3b0 con 0x7f0f2415af80 2023-10-16T10:04:50.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.201+0000 7f0f217fa640 1 --2- 172.21.15.84:0/2905685985 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f0f0806af90 0x7f0f0806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:50.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.201+0000 7f0f217fa640 1 -- 172.21.15.84:0/2905685985 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f0f1c089730 con 0x7f0f2415af80 2023-10-16T10:04:50.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.201+0000 7f0f28cb9640 1 -- 172.21.15.84:0/2905685985 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f0f0806af90 msgr2=0x7f0f0806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2103121481 2023-10-16T10:04:50.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.201+0000 7f0f28cb9640 1 --2- 172.21.15.84:0/2905685985 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f0f0806af90 0x7f0f0806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:50.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.206+0000 7f0f217fa640 1 -- 172.21.15.84:0/2905685985 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0f1c054570 con 0x7f0f2415af80 2023-10-16T10:04:50.233 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:50.232+0000 7f95c9812040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:04:50.233 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:50.232+0000 7fe455e32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:04:50.285 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:50.284+0000 7f95c9812040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:04:50.287 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:50.286+0000 7fe455e32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:04:50.335 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:50.335+0000 7f95c9812040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:04:50.339 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:50.338+0000 7fe455e32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:04:50.396 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:50.396+0000 7f95c9812040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:04:50.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.397+0000 7f0f2af1d640 1 -- 172.21.15.84:0/2905685985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0f2404e160 con 0x7f0f2415af80 2023-10-16T10:04:50.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.398+0000 7f0f217fa640 1 -- 172.21.15.84:0/2905685985 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0f1c057b90 con 0x7f0f2415af80 2023-10-16T10:04:50.399 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:04:50.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.400+0000 7f0f2af1d640 1 -- 172.21.15.84:0/2905685985 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f0f0806af90 msgr2=0x7f0f0806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:50.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.400+0000 7f0f2af1d640 1 --2- 172.21.15.84:0/2905685985 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f0f0806af90 0x7f0f0806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.400+0000 7f0f2af1d640 1 -- 172.21.15.84:0/2905685985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2415af80 msgr2=0x7f0f240ae440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:50.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.400+0000 7f0f2af1d640 1 --2- 172.21.15.84:0/2905685985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2415af80 0x7f0f240ae440 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c00b970 tx=0x7f0f1c00be30 comp rx=0 tx=0).stop 2023-10-16T10:04:50.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.400+0000 7f0f2af1d640 1 -- 172.21.15.84:0/2905685985 shutdown_connections 2023-10-16T10:04:50.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.400+0000 7f0f2af1d640 1 --2- 172.21.15.84:0/2905685985 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f0f0806af90 0x7f0f0806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.400+0000 7f0f2af1d640 1 --2- 172.21.15.84:0/2905685985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2415af80 0x7f0f240ae440 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.400+0000 7f0f2af1d640 1 --2- 172.21.15.84:0/2905685985 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f2406faf0 0x7f0f2406c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.400+0000 7f0f2af1d640 1 --2- 172.21.15.84:0/2905685985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f24065ab0 0x7f0f241500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.400+0000 7f0f2af1d640 1 -- 172.21.15.84:0/2905685985 >> 172.21.15.84:0/2905685985 conn(0x7f0f240d0ab0 msgr2=0x7f0f2415a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:50.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.400+0000 7f0f2af1d640 1 -- 172.21.15.84:0/2905685985 shutdown_connections 2023-10-16T10:04:50.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.400+0000 7f0f2af1d640 1 -- 172.21.15.84:0/2905685985 wait complete. 2023-10-16T10:04:50.404 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:50.401+0000 7fe455e32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:04:50.419 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:04:50.445 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:50.445+0000 7f95c9812040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:04:50.454 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:50.453+0000 7fe455e32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:04:50.555 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:50.555+0000 7f95c9812040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:04:50.572 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:50.570+0000 7fe455e32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:04:50.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.610+0000 7f3a8d035640 1 Processor -- start 2023-10-16T10:04:50.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.611+0000 7f3a8d035640 1 -- start start 2023-10-16T10:04:50.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.612+0000 7f3a8d035640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8815cd70 0x7f3a880d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:50.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.613+0000 7f3a8d035640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a880cded0 0x7f3a880b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:50.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.613+0000 7f3a8d035640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a8815d150 con 0x7f3a880b8480 2023-10-16T10:04:50.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.613+0000 7f3a8d035640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a880d1180 con 0x7f3a880cded0 2023-10-16T10:04:50.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.613+0000 7f3a8d035640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a880d1460 con 0x7f3a8815cd70 2023-10-16T10:04:50.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.613+0000 7f3a8c834640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a880cded0 0x7f3a880b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:50.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.613+0000 7f3a87fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8815cd70 0x7f3a880d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:50.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.613+0000 7f3a877fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3a880b8480 0x7f3a880cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:43038/0 (socket says 172.21.15.84:43038) 2023-10-16T10:04:50.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.613+0000 7f3a877fe640 1 -- 172.21.15.84:0/1550365969 learned_addr learned my addr 172.21.15.84:0/1550365969 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:50.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.614+0000 7f3a8c834640 1 -- 172.21.15.84:0/1550365969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8815cd70 msgr2=0x7f3a880d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:50.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.614+0000 7f3a8c834640 1 --2- 172.21.15.84:0/1550365969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8815cd70 0x7f3a880d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.614+0000 7f3a8c834640 1 -- 172.21.15.84:0/1550365969 >> v1:172.21.15.157:6789/0 conn(0x7f3a880b8480 legacy=0x7f3a880cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:50.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.614+0000 7f3a8c834640 1 -- 172.21.15.84:0/1550365969 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a880d1740 con 0x7f3a880cded0 2023-10-16T10:04:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.615+0000 7f3a8c834640 1 --2- 172.21.15.84:0/1550365969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880cded0 0x7f3a880b5c90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3a80009d90 tx=0x7f3a8002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=871f5702189782b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.616+0000 7f3a86ffd640 1 -- 172.21.15.84:0/1550365969 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8002cc50 con 0x7f3a880cded0 2023-10-16T10:04:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.616+0000 7f3a86ffd640 1 -- 172.21.15.84:0/1550365969 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8002cdb0 con 0x7f3a880cded0 2023-10-16T10:04:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.616+0000 7f3a86ffd640 1 -- 172.21.15.84:0/1550365969 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80034660 con 0x7f3a880cded0 2023-10-16T10:04:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.616+0000 7f3a8d035640 1 -- 172.21.15.84:0/1550365969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880cded0 msgr2=0x7f3a880b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:50.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.616+0000 7f3a8d035640 1 --2- 172.21.15.84:0/1550365969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880cded0 0x7f3a880b5c90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3a80009d90 tx=0x7f3a8002bea0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.617+0000 7f3a8d035640 1 -- 172.21.15.84:0/1550365969 shutdown_connections 2023-10-16T10:04:50.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.617+0000 7f3a8d035640 1 --2- 172.21.15.84:0/1550365969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880cded0 0x7f3a880b5c90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.617+0000 7f3a8d035640 1 --2- 172.21.15.84:0/1550365969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8815cd70 0x7f3a880d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.617+0000 7f3a8d035640 1 -- 172.21.15.84:0/1550365969 >> 172.21.15.84:0/1550365969 conn(0x7f3a880cf570 msgr2=0x7f3a880cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:50.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.617+0000 7f3a8d035640 1 -- 172.21.15.84:0/1550365969 shutdown_connections 2023-10-16T10:04:50.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.617+0000 7f3a8d035640 1 -- 172.21.15.84:0/1550365969 wait complete. 2023-10-16T10:04:50.622 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:50.618+0000 7f95c9812040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:04:50.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.619+0000 7f3a8d035640 1 Processor -- start 2023-10-16T10:04:50.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.620+0000 7f3a8d035640 1 -- start start 2023-10-16T10:04:50.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.621+0000 7f3a8d035640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a880b8480 0x7f3a881f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:50.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.621+0000 7f3a8d035640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880cded0 0x7f3a881f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:50.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.621+0000 7f3a8d035640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8815cd70 0x7f3a880c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:50.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.621+0000 7f3a8d035640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a881f5a30 con 0x7f3a880b8480 2023-10-16T10:04:50.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.621+0000 7f3a8d035640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a881f5ff0 con 0x7f3a880cded0 2023-10-16T10:04:50.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.621+0000 7f3a8d035640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a880c9000 con 0x7f3a8815cd70 2023-10-16T10:04:50.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.621+0000 7f3a87fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a880b8480 0x7f3a881f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:50.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.621+0000 7f3a87fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a880b8480 0x7f3a881f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59608/0 (socket says 172.21.15.84:59608) 2023-10-16T10:04:50.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.621+0000 7f3a87fff640 1 -- 172.21.15.84:0/32747520 learned_addr learned my addr 172.21.15.84:0/32747520 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:50.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.622+0000 7f3a877fe640 1 --2- 172.21.15.84:0/32747520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880cded0 0x7f3a881f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:50.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.622+0000 7f3a8c834640 1 --2- 172.21.15.84:0/32747520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8815cd70 0x7f3a880c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:50.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.622+0000 7f3a87fff640 1 -- 172.21.15.84:0/32747520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8815cd70 msgr2=0x7f3a880c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:50.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.622+0000 7f3a87fff640 1 --2- 172.21.15.84:0/32747520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8815cd70 0x7f3a880c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.622+0000 7f3a87fff640 1 -- 172.21.15.84:0/32747520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880cded0 msgr2=0x7f3a881f1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:50.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.622+0000 7f3a87fff640 1 --2- 172.21.15.84:0/32747520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880cded0 0x7f3a881f1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.622+0000 7f3a87fff640 1 -- 172.21.15.84:0/32747520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a80009a70 con 0x7f3a880b8480 2023-10-16T10:04:50.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.622+0000 7f3a87fff640 1 --2- 172.21.15.84:0/32747520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a880b8480 0x7f3a881f6480 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800e8a0 tx=0x7f3a7800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:50.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.622+0000 7f3a84ff9640 1 -- 172.21.15.84:0/32747520 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a780094f0 con 0x7f3a880b8480 2023-10-16T10:04:50.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.623+0000 7f3a84ff9640 1 -- 172.21.15.84:0/32747520 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7800c940 con 0x7f3a880b8480 2023-10-16T10:04:50.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.623+0000 7f3a84ff9640 1 -- 172.21.15.84:0/32747520 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7800ccf0 con 0x7f3a880b8480 2023-10-16T10:04:50.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.623+0000 7f3a8d035640 1 -- 172.21.15.84:0/32747520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a880c92e0 con 0x7f3a880b8480 2023-10-16T10:04:50.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.623+0000 7f3a8d035640 1 -- 172.21.15.84:0/32747520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a881f2230 con 0x7f3a880b8480 2023-10-16T10:04:50.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.624+0000 7f3a8d035640 1 -- 172.21.15.84:0/32747520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a8804e9b0 con 0x7f3a880b8480 2023-10-16T10:04:50.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.625+0000 7f3a84ff9640 1 -- 172.21.15.84:0/32747520 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 37) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f3a78013470 con 0x7f3a880b8480 2023-10-16T10:04:50.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.626+0000 7f3a84ff9640 1 --2- 172.21.15.84:0/32747520 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f3a7406af90 0x7f3a7406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:50.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.626+0000 7f3a877fe640 1 -- 172.21.15.84:0/32747520 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f3a7406af90 msgr2=0x7f3a7406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2103121481 2023-10-16T10:04:50.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.626+0000 7f3a84ff9640 1 -- 172.21.15.84:0/32747520 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f3a78089aa0 con 0x7f3a880b8480 2023-10-16T10:04:50.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.626+0000 7f3a877fe640 1 --2- 172.21.15.84:0/32747520 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f3a7406af90 0x7f3a7406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:50.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.630+0000 7f3a84ff9640 1 -- 172.21.15.84:0/32747520 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3a78054850 con 0x7f3a880b8480 2023-10-16T10:04:50.641 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:50.640+0000 7fe455e32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:04:50.670 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:50.670+0000 7f95c9812040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:04:50.694 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:50.692+0000 7fe455e32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:04:50.727 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:50.726+0000 7f95c9812040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:04:50.753 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:50.751+0000 7fe455e32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:04:50.778 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:50.778+0000 7f95c9812040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:04:50.806 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:50.805+0000 7fe455e32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:04:50.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.824+0000 7f3a8d035640 1 -- 172.21.15.84:0/32747520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a880b1980 con 0x7f3a880b8480 2023-10-16T10:04:50.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.825+0000 7f3a84ff9640 1 -- 172.21.15.84:0/32747520 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3a78057e70 con 0x7f3a880b8480 2023-10-16T10:04:50.827 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T10:04:50.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.827+0000 7f3a877fe640 1 -- 172.21.15.84:0/32747520 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f3a7406af90 msgr2=0x7f3a7406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2103121481 2023-10-16T10:04:50.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.827+0000 7f3a877fe640 1 --2- 172.21.15.84:0/32747520 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f3a7406af90 0x7f3a7406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:04:50.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.828+0000 7f3a8d035640 1 -- 172.21.15.84:0/32747520 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f3a7406af90 msgr2=0x7f3a7406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:50.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.828+0000 7f3a8d035640 1 --2- 172.21.15.84:0/32747520 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f3a7406af90 0x7f3a7406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.828+0000 7f3a8d035640 1 -- 172.21.15.84:0/32747520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a880b8480 msgr2=0x7f3a881f6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:50.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.828+0000 7f3a8d035640 1 --2- 172.21.15.84:0/32747520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a880b8480 0x7f3a881f6480 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800e8a0 tx=0x7f3a7800ed60 comp rx=0 tx=0).stop 2023-10-16T10:04:50.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.828+0000 7f3a8d035640 1 -- 172.21.15.84:0/32747520 shutdown_connections 2023-10-16T10:04:50.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.828+0000 7f3a8d035640 1 --2- 172.21.15.84:0/32747520 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f3a7406af90 0x7f3a7406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.828+0000 7f3a8d035640 1 --2- 172.21.15.84:0/32747520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8815cd70 0x7f3a880c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.828+0000 7f3a8d035640 1 --2- 172.21.15.84:0/32747520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880cded0 0x7f3a881f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.828+0000 7f3a8d035640 1 --2- 172.21.15.84:0/32747520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a880b8480 0x7f3a881f6480 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:50.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.828+0000 7f3a8d035640 1 -- 172.21.15.84:0/32747520 >> 172.21.15.84:0/32747520 conn(0x7f3a880cf570 msgr2=0x7f3a880b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:50.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.828+0000 7f3a8d035640 1 -- 172.21.15.84:0/32747520 shutdown_connections 2023-10-16T10:04:50.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:50.828+0000 7f3a8d035640 1 -- 172.21.15.84:0/32747520 wait complete. 2023-10-16T10:04:50.846 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:04:51.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.094+0000 7f5824e39640 1 Processor -- start 2023-10-16T10:04:51.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.094+0000 7f5824e39640 1 -- start start 2023-10-16T10:04:51.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.096+0000 7f5824e39640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58200c9800 0x7f582015d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:51.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.096+0000 7f5824e39640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f582006af40 0x7f582015dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:51.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.096+0000 7f5824e39640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f582015a120 con 0x7f5820159d40 2023-10-16T10:04:51.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.096+0000 7f5824e39640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f582015e3b0 con 0x7f582006af40 2023-10-16T10:04:51.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.096+0000 7f5824e39640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582015e690 con 0x7f58200c9800 2023-10-16T10:04:51.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.096+0000 7f581ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f582006af40 0x7f582015dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:51.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.096+0000 7f581e59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f5820159d40 0x7f5820071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48796/0 (socket says 172.21.15.84:48796) 2023-10-16T10:04:51.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.096+0000 7f581e59c640 1 -- 172.21.15.84:0/1259815018 learned_addr learned my addr 172.21.15.84:0/1259815018 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:51.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.097+0000 7f581d59a640 1 -- 172.21.15.84:0/1259815018 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3149221128 0 0) 0x7f582015a120 con 0x7f5820159d40 2023-10-16T10:04:51.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.097+0000 7f581dd9b640 1 --2- 172.21.15.84:0/1259815018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58200c9800 0x7f582015d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:51.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.097+0000 7f581d59a640 1 -- 172.21.15.84:0/1259815018 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5808003520 con 0x7f5820159d40 2023-10-16T10:04:51.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.097+0000 7f581ed9d640 1 -- 172.21.15.84:0/1259815018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58200c9800 msgr2=0x7f582015d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:51.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.097+0000 7f581ed9d640 1 --2- 172.21.15.84:0/1259815018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58200c9800 0x7f582015d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.097+0000 7f581ed9d640 1 -- 172.21.15.84:0/1259815018 >> v1:172.21.15.84:6789/0 conn(0x7f5820159d40 legacy=0x7f5820071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:51.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.097+0000 7f581ed9d640 1 -- 172.21.15.84:0/1259815018 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5820070f60 con 0x7f582006af40 2023-10-16T10:04:51.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.098+0000 7f581ed9d640 1 --2- 172.21.15.84:0/1259815018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f582006af40 0x7f582015dd20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f5818009d90 tx=0x7f581802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c180f4bfec8814d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:51.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.099+0000 7f581d59a640 1 -- 172.21.15.84:0/1259815018 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581802cc50 con 0x7f582006af40 2023-10-16T10:04:51.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.099+0000 7f581d59a640 1 -- 172.21.15.84:0/1259815018 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581802cdb0 con 0x7f582006af40 2023-10-16T10:04:51.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.099+0000 7f581d59a640 1 -- 172.21.15.84:0/1259815018 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818034660 con 0x7f582006af40 2023-10-16T10:04:51.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.099+0000 7f5824e39640 1 -- 172.21.15.84:0/1259815018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f582006af40 msgr2=0x7f582015dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:51.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.099+0000 7f5824e39640 1 --2- 172.21.15.84:0/1259815018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f582006af40 0x7f582015dd20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f5818009d90 tx=0x7f581802bea0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.099+0000 7f5824e39640 1 -- 172.21.15.84:0/1259815018 shutdown_connections 2023-10-16T10:04:51.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.099+0000 7f5824e39640 1 --2- 172.21.15.84:0/1259815018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f582006af40 0x7f582015dd20 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.099+0000 7f5824e39640 1 --2- 172.21.15.84:0/1259815018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58200c9800 0x7f582015d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.099+0000 7f5824e39640 1 -- 172.21.15.84:0/1259815018 >> 172.21.15.84:0/1259815018 conn(0x7f582006bc10 msgr2=0x7f582006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:51.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.099+0000 7f5824e39640 1 -- 172.21.15.84:0/1259815018 shutdown_connections 2023-10-16T10:04:51.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.099+0000 7f5824e39640 1 -- 172.21.15.84:0/1259815018 wait complete. 2023-10-16T10:04:51.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.101+0000 7f5824e39640 1 Processor -- start 2023-10-16T10:04:51.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.101+0000 7f5824e39640 1 -- start start 2023-10-16T10:04:51.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.102+0000 7f5824e39640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f582006af40 0x7f582015c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:51.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.102+0000 7f5824e39640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200c9800 0x7f582014d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:51.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.102+0000 7f5824e39640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5820159d40 0x7f58200afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:51.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.102+0000 7f5824e39640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582015cc60 con 0x7f582006af40 2023-10-16T10:04:51.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.102+0000 7f5824e39640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582015a700 con 0x7f58200c9800 2023-10-16T10:04:51.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.102+0000 7f5824e39640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582015a9e0 con 0x7f5820159d40 2023-10-16T10:04:51.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.103+0000 7f581e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f582006af40 0x7f582015c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:51.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.103+0000 7f581e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f582006af40 0x7f582015c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59630/0 (socket says 172.21.15.84:59630) 2023-10-16T10:04:51.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.103+0000 7f581e59c640 1 -- 172.21.15.84:0/532472338 learned_addr learned my addr 172.21.15.84:0/532472338 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:51.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.103+0000 7f581dd9b640 1 --2- 172.21.15.84:0/532472338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200c9800 0x7f582014d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:51.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.103+0000 7f581ed9d640 1 --2- 172.21.15.84:0/532472338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5820159d40 0x7f58200afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:51.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.103+0000 7f581e59c640 1 -- 172.21.15.84:0/532472338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5820159d40 msgr2=0x7f58200afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:51.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.103+0000 7f581e59c640 1 --2- 172.21.15.84:0/532472338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5820159d40 0x7f58200afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.103+0000 7f581e59c640 1 -- 172.21.15.84:0/532472338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200c9800 msgr2=0x7f582014d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:51.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.103+0000 7f581e59c640 1 --2- 172.21.15.84:0/532472338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200c9800 0x7f582014d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.103+0000 7f581e59c640 1 -- 172.21.15.84:0/532472338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5818009a70 con 0x7f582006af40 2023-10-16T10:04:51.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.103+0000 7f581dd9b640 1 --2- 172.21.15.84:0/532472338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200c9800 0x7f582014d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:51.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.103+0000 7f581e59c640 1 --2- 172.21.15.84:0/532472338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f582006af40 0x7f582015c720 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f581000c500 tx=0x7f581000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:51.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.104+0000 7f580effd640 1 -- 172.21.15.84:0/532472338 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810005ce0 con 0x7f582006af40 2023-10-16T10:04:51.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.104+0000 7f580effd640 1 -- 172.21.15.84:0/532472338 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5810005e40 con 0x7f582006af40 2023-10-16T10:04:51.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.104+0000 7f580effd640 1 -- 172.21.15.84:0/532472338 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810007330 con 0x7f582006af40 2023-10-16T10:04:51.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.104+0000 7f5824e39640 1 -- 172.21.15.84:0/532472338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582015acc0 con 0x7f582006af40 2023-10-16T10:04:51.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.104+0000 7f5824e39640 1 -- 172.21.15.84:0/532472338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582015b280 con 0x7f582006af40 2023-10-16T10:04:51.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.105+0000 7f5824e39640 1 -- 172.21.15.84:0/532472338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582004e160 con 0x7f582006af40 2023-10-16T10:04:51.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.106+0000 7f580effd640 1 -- 172.21.15.84:0/532472338 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 37) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f5810007490 con 0x7f582006af40 2023-10-16T10:04:51.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.107+0000 7f580effd640 1 --2- 172.21.15.84:0/532472338 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f580006afc0 0x7f580006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:51.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.107+0000 7f580effd640 1 -- 172.21.15.84:0/532472338 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f5810088f90 con 0x7f582006af40 2023-10-16T10:04:51.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.107+0000 7f581dd9b640 1 -- 172.21.15.84:0/532472338 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f580006afc0 msgr2=0x7f580006d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2103121481 2023-10-16T10:04:51.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.107+0000 7f581dd9b640 1 --2- 172.21.15.84:0/532472338 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f580006afc0 0x7f580006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:51.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.112+0000 7f580effd640 1 -- 172.21.15.84:0/532472338 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5810053e80 con 0x7f582006af40 2023-10-16T10:04:51.166 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:51.166+0000 7f95c9812040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:04:51.208 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:51.206+0000 7fe455e32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:04:51.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.305+0000 7f5824e39640 1 -- 172.21.15.84:0/532472338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f582006cab0 con 0x7f582006af40 2023-10-16T10:04:51.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.306+0000 7f580effd640 1 -- 172.21.15.84:0/532472338 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5810017020 con 0x7f582006af40 2023-10-16T10:04:51.308 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T10:04:51.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.308+0000 7f581dd9b640 1 -- 172.21.15.84:0/532472338 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f580006afc0 msgr2=0x7f580006d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2103121481 2023-10-16T10:04:51.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.308+0000 7f581dd9b640 1 --2- 172.21.15.84:0/532472338 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f580006afc0 0x7f580006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:04:51.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.308+0000 7f5824e39640 1 -- 172.21.15.84:0/532472338 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f580006afc0 msgr2=0x7f580006d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:51.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.308+0000 7f5824e39640 1 --2- 172.21.15.84:0/532472338 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f580006afc0 0x7f580006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.309+0000 7f5824e39640 1 -- 172.21.15.84:0/532472338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f582006af40 msgr2=0x7f582015c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:51.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.309+0000 7f5824e39640 1 --2- 172.21.15.84:0/532472338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f582006af40 0x7f582015c720 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f581000c500 tx=0x7f581000c9c0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.309+0000 7f5824e39640 1 -- 172.21.15.84:0/532472338 shutdown_connections 2023-10-16T10:04:51.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.309+0000 7f5824e39640 1 --2- 172.21.15.84:0/532472338 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f580006afc0 0x7f580006d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.309+0000 7f5824e39640 1 --2- 172.21.15.84:0/532472338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5820159d40 0x7f58200afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.309+0000 7f5824e39640 1 --2- 172.21.15.84:0/532472338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200c9800 0x7f582014d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.309+0000 7f5824e39640 1 --2- 172.21.15.84:0/532472338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f582006af40 0x7f582015c720 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.309+0000 7f5824e39640 1 -- 172.21.15.84:0/532472338 >> 172.21.15.84:0/532472338 conn(0x7f582006bc10 msgr2=0x7f5820069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:51.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.309+0000 7f5824e39640 1 -- 172.21.15.84:0/532472338 shutdown_connections 2023-10-16T10:04:51.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.309+0000 7f5824e39640 1 -- 172.21.15.84:0/532472338 wait complete. 2023-10-16T10:04:51.326 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:51.325+0000 7f95c9812040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:04:51.329 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T10:04:51.374 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:51.373+0000 7fe455e32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:04:51.435 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:51.434+0000 7f95c9812040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:04:51.485 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:51.485+0000 7f95c9812040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:04:51.486 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:51.484+0000 7fe455e32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:04:51.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.534+0000 7f32490ad640 1 Processor -- start 2023-10-16T10:04:51.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.534+0000 7f32490ad640 1 -- start start 2023-10-16T10:04:51.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.536+0000 7f32490ad640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32440ad7a0 0x7f3244070240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:51.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.536+0000 7f32490ad640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f32440adc00 0x7f3244070780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:51.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.536+0000 7f32490ad640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32440ae1a0 con 0x7f3244160c80 2023-10-16T10:04:51.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.536+0000 7f32490ad640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32440ae480 con 0x7f32440adc00 2023-10-16T10:04:51.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.536+0000 7f32490ad640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32440ae760 con 0x7f32440ad7a0 2023-10-16T10:04:51.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.536+0000 7f3243fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3244160c80 0x7f3244161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48806/0 (socket says 172.21.15.84:48806) 2023-10-16T10:04:51.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.536+0000 7f3243fff640 1 -- 172.21.15.84:0/3970522197 learned_addr learned my addr 172.21.15.84:0/3970522197 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:51.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.536+0000 7f32488ac640 1 --2- 172.21.15.84:0/3970522197 >> v2:172.21.15.157:3300/0 conn(0x7f32440adc00 0x7f3244070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:51.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.536+0000 7f32437fe640 1 --2- 172.21.15.84:0/3970522197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32440ad7a0 0x7f3244070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:51.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.536+0000 7f3242ffd640 1 -- 172.21.15.84:0/3970522197 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3861958519 0 0) 0x7f32440ae1a0 con 0x7f3244160c80 2023-10-16T10:04:51.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.537+0000 7f3242ffd640 1 -- 172.21.15.84:0/3970522197 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f322c003520 con 0x7f3244160c80 2023-10-16T10:04:51.540 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:51.537+0000 7fe455e32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:04:51.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.537+0000 7f3242ffd640 1 -- 172.21.15.84:0/3970522197 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1754167831 0 0) 0x7f322c003520 con 0x7f3244160c80 2023-10-16T10:04:51.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.537+0000 7f3242ffd640 1 -- 172.21.15.84:0/3970522197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32440ad7a0 msgr2=0x7f3244070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:51.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.537+0000 7f3242ffd640 1 --2- 172.21.15.84:0/3970522197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32440ad7a0 0x7f3244070240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.537+0000 7f3242ffd640 1 -- 172.21.15.84:0/3970522197 >> v2:172.21.15.157:3300/0 conn(0x7f32440adc00 msgr2=0x7f3244070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:51.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.537+0000 7f3242ffd640 1 --2- 172.21.15.84:0/3970522197 >> v2:172.21.15.157:3300/0 conn(0x7f32440adc00 0x7f3244070780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.537+0000 7f3242ffd640 1 -- 172.21.15.84:0/3970522197 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32440aea40 con 0x7f3244160c80 2023-10-16T10:04:51.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.537+0000 7f3242ffd640 1 -- 172.21.15.84:0/3970522197 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f3234001670 con 0x7f3244160c80 2023-10-16T10:04:51.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.537+0000 7f3242ffd640 1 -- 172.21.15.84:0/3970522197 >> v1:172.21.15.84:6789/0 conn(0x7f3244160c80 legacy=0x7f3244161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:51.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.537+0000 7f3242ffd640 1 --2- 172.21.15.84:0/3970522197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32440ad7a0 0x7f322c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:51.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.537+0000 7f3242ffd640 1 --2- 172.21.15.84:0/3970522197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f322c003f40 0x7f322c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:51.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.537+0000 7f3242ffd640 1 --2- 172.21.15.84:0/3970522197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f322c004d00 0x7f322c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:51.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.537+0000 7f3242ffd640 1 -- 172.21.15.84:0/3970522197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32440ae480 con 0x7f322c004d00 2023-10-16T10:04:51.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.537+0000 7f3242ffd640 1 -- 172.21.15.84:0/3970522197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32440ae760 con 0x7f322c003f40 2023-10-16T10:04:51.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.537+0000 7f3242ffd640 1 -- 172.21.15.84:0/3970522197 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c00b980 con 0x7f32440ad7a0 2023-10-16T10:04:51.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.537+0000 7f32488ac640 1 --2- 172.21.15.84:0/3970522197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f322c004d00 0x7f322c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:51.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.538+0000 7f3243fff640 1 --2- 172.21.15.84:0/3970522197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32440ad7a0 0x7f322c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:51.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.538+0000 7f32437fe640 1 --2- 172.21.15.84:0/3970522197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f322c003f40 0x7f322c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:51.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.538+0000 7f32488ac640 1 -- 172.21.15.84:0/3970522197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32440ad7a0 msgr2=0x7f322c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:51.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.538+0000 7f32488ac640 1 --2- 172.21.15.84:0/3970522197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32440ad7a0 0x7f322c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.538+0000 7f32488ac640 1 -- 172.21.15.84:0/3970522197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f322c003f40 msgr2=0x7f322c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:51.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.538+0000 7f32488ac640 1 --2- 172.21.15.84:0/3970522197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f322c003f40 0x7f322c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.538+0000 7f32488ac640 1 -- 172.21.15.84:0/3970522197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c003520 con 0x7f322c004d00 2023-10-16T10:04:51.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.540+0000 7f32488ac640 1 --2- 172.21.15.84:0/3970522197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f322c004d00 0x7f322c005180 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f323c00e170 tx=0x7f322c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6e7a476a834531 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:51.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.540+0000 7f3242ffd640 1 -- 172.21.15.84:0/3970522197 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c02cc20 con 0x7f322c004d00 2023-10-16T10:04:51.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.540+0000 7f3242ffd640 1 -- 172.21.15.84:0/3970522197 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c00f040 con 0x7f322c004d00 2023-10-16T10:04:51.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.540+0000 7f3242ffd640 1 -- 172.21.15.84:0/3970522197 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c034440 con 0x7f322c004d00 2023-10-16T10:04:51.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.541+0000 7f32490ad640 1 -- 172.21.15.84:0/3970522197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f322c004d00 msgr2=0x7f322c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:51.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.541+0000 7f32490ad640 1 --2- 172.21.15.84:0/3970522197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f322c004d00 0x7f322c005180 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f323c00e170 tx=0x7f322c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.541+0000 7f32488ac640 1 -- 172.21.15.84:0/3970522197 reap_dead start 2023-10-16T10:04:51.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.541+0000 7f32490ad640 1 -- 172.21.15.84:0/3970522197 shutdown_connections 2023-10-16T10:04:51.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.541+0000 7f32490ad640 1 -- 172.21.15.84:0/3970522197 >> 172.21.15.84:0/3970522197 conn(0x7f32440cc230 msgr2=0x7f32440cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:51.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.541+0000 7f32490ad640 1 -- 172.21.15.84:0/3970522197 shutdown_connections 2023-10-16T10:04:51.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.541+0000 7f32490ad640 1 -- 172.21.15.84:0/3970522197 wait complete. 2023-10-16T10:04:51.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.542+0000 7f32490ad640 1 Processor -- start 2023-10-16T10:04:51.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.543+0000 7f32490ad640 1 -- start start 2023-10-16T10:04:51.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.543+0000 7f32490ad640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f322c004d00 0x7f32440c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:51.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.544+0000 7f32490ad640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f324406bd20 0x7f324406c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:51.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.544+0000 7f32490ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32441e7900 0x7f32441e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:51.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.544+0000 7f32490ad640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32441e82c0 con 0x7f324406bd20 2023-10-16T10:04:51.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.544+0000 7f32490ad640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324406c800 con 0x7f32441e7900 2023-10-16T10:04:51.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.544+0000 7f32490ad640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324406cae0 con 0x7f322c004d00 2023-10-16T10:04:51.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.544+0000 7f32437fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f324406bd20 0x7f324406c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:51.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.544+0000 7f32437fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f324406bd20 0x7f324406c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59658/0 (socket says 172.21.15.84:59658) 2023-10-16T10:04:51.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.544+0000 7f32437fe640 1 -- 172.21.15.84:0/1460074629 learned_addr learned my addr 172.21.15.84:0/1460074629 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:51.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.544+0000 7f3243fff640 1 --2- 172.21.15.84:0/1460074629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f322c004d00 0x7f32440c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:51.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.544+0000 7f32488ac640 1 --2- 172.21.15.84:0/1460074629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32441e7900 0x7f32441e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:51.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.544+0000 7f32437fe640 1 -- 172.21.15.84:0/1460074629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f322c004d00 msgr2=0x7f32440c26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:51.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.544+0000 7f32437fe640 1 --2- 172.21.15.84:0/1460074629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f322c004d00 0x7f32440c26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.544+0000 7f32437fe640 1 -- 172.21.15.84:0/1460074629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32441e7900 msgr2=0x7f32441e7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:51.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.544+0000 7f32437fe640 1 --2- 172.21.15.84:0/1460074629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32441e7900 0x7f32441e7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.544+0000 7f32437fe640 1 -- 172.21.15.84:0/1460074629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324406cdc0 con 0x7f324406bd20 2023-10-16T10:04:51.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.544+0000 7f32488ac640 1 --2- 172.21.15.84:0/1460074629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32441e7900 0x7f32441e7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:51.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.545+0000 7f32437fe640 1 --2- 172.21.15.84:0/1460074629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f324406bd20 0x7f324406c1a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f3238011540 tx=0x7f3238011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:51.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.545+0000 7f3240ff9640 1 -- 172.21.15.84:0/1460074629 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238015930 con 0x7f324406bd20 2023-10-16T10:04:51.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.545+0000 7f3240ff9640 1 -- 172.21.15.84:0/1460074629 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3238015a90 con 0x7f324406bd20 2023-10-16T10:04:51.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.545+0000 7f3240ff9640 1 -- 172.21.15.84:0/1460074629 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238015e60 con 0x7f324406bd20 2023-10-16T10:04:51.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.545+0000 7f32490ad640 1 -- 172.21.15.84:0/1460074629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324406d0a0 con 0x7f324406bd20 2023-10-16T10:04:51.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.545+0000 7f32490ad640 1 -- 172.21.15.84:0/1460074629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32440c2bf0 con 0x7f324406bd20 2023-10-16T10:04:51.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.546+0000 7f32490ad640 1 -- 172.21.15.84:0/1460074629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324404e9b0 con 0x7f324406bd20 2023-10-16T10:04:51.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.547+0000 7f3240ff9640 1 -- 172.21.15.84:0/1460074629 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 37) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f323800f460 con 0x7f324406bd20 2023-10-16T10:04:51.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.548+0000 7f3240ff9640 1 --2- 172.21.15.84:0/1460074629 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f323006afc0 0x7f323006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:51.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.548+0000 7f3240ff9640 1 -- 172.21.15.84:0/1460074629 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f323808a010 con 0x7f324406bd20 2023-10-16T10:04:51.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.548+0000 7f3243fff640 1 -- 172.21.15.84:0/1460074629 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f323006afc0 msgr2=0x7f323006d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2103121481 2023-10-16T10:04:51.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.548+0000 7f3243fff640 1 --2- 172.21.15.84:0/1460074629 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f323006afc0 0x7f323006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:51.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.556+0000 7f3240ff9640 1 -- 172.21.15.84:0/1460074629 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3238054e50 con 0x7f324406bd20 2023-10-16T10:04:51.596 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:51.595+0000 7f95c9812040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:04:51.655 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:51.653+0000 7fe455e32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:04:51.703 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:51.702+0000 7f95c9812040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:04:51.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.749+0000 7f3243fff640 1 -- 172.21.15.84:0/1460074629 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f323006afc0 msgr2=0x7f323006d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2103121481 2023-10-16T10:04:51.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.749+0000 7f3243fff640 1 --2- 172.21.15.84:0/1460074629 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f323006afc0 0x7f323006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:04:51.763 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:51.761+0000 7fe455e32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:04:51.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.762+0000 7f32490ad640 1 -- 172.21.15.84:0/1460074629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f32440b5730 con 0x7f324406bd20 2023-10-16T10:04:51.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.763+0000 7f3240ff9640 1 -- 172.21.15.84:0/1460074629 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v83) v1 ==== 80+0+470 (secure 0 0 0) 0x7f3238058470 con 0x7f324406bd20 2023-10-16T10:04:51.764 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3177129458 2023-10-17T10:04:47.124608+0000 2023-10-16T10:04:51.765 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1145018607 2023-10-17T10:04:47.124608+0000 2023-10-16T10:04:51.765 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1499920052 2023-10-17T10:04:47.124608+0000 2023-10-16T10:04:51.765 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6833/2091947430 2023-10-17T10:04:47.124608+0000 2023-10-16T10:04:51.765 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6832/2091947430 2023-10-17T10:04:47.124608+0000 2023-10-16T10:04:51.765 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2509405053 2023-10-17T10:04:47.124608+0000 2023-10-16T10:04:51.766 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3991569032 2023-10-17T10:04:47.124608+0000 2023-10-16T10:04:51.766 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3599228496 2023-10-17T10:04:47.124608+0000 2023-10-16T10:04:51.766 INFO:teuthology.orchestra.run.smithi084.stderr:listed 8 entries 2023-10-16T10:04:51.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.765+0000 7f32490ad640 1 -- 172.21.15.84:0/1460074629 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f323006afc0 msgr2=0x7f323006d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:51.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.765+0000 7f32490ad640 1 --2- 172.21.15.84:0/1460074629 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f323006afc0 0x7f323006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.765+0000 7f32490ad640 1 -- 172.21.15.84:0/1460074629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f324406bd20 msgr2=0x7f324406c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:51.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.765+0000 7f32490ad640 1 --2- 172.21.15.84:0/1460074629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f324406bd20 0x7f324406c1a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f3238011540 tx=0x7f3238011a00 comp rx=0 tx=0).stop 2023-10-16T10:04:51.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.765+0000 7f32490ad640 1 -- 172.21.15.84:0/1460074629 shutdown_connections 2023-10-16T10:04:51.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.765+0000 7f32490ad640 1 --2- 172.21.15.84:0/1460074629 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f323006afc0 0x7f323006d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.765+0000 7f32490ad640 1 --2- 172.21.15.84:0/1460074629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32441e7900 0x7f32441e7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.765+0000 7f32490ad640 1 --2- 172.21.15.84:0/1460074629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f324406bd20 0x7f324406c1a0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.765+0000 7f32490ad640 1 --2- 172.21.15.84:0/1460074629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f322c004d00 0x7f32440c26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:51.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.765+0000 7f32490ad640 1 -- 172.21.15.84:0/1460074629 >> 172.21.15.84:0/1460074629 conn(0x7f32440cc230 msgr2=0x7f32440ce850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:51.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.766+0000 7f32490ad640 1 -- 172.21.15.84:0/1460074629 shutdown_connections 2023-10-16T10:04:51.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:51.766+0000 7f32490ad640 1 -- 172.21.15.84:0/1460074629 wait complete. 2023-10-16T10:04:51.785 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T10:04:51.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:51.786 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T10:04:51.859 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:51.858+0000 7f95c9812040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:04:51.908 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:51.908+0000 7f95c9812040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:04:51.929 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T10:04:51.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:51.929 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T10:04:51.930 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:51.928+0000 7fe455e32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:04:51.974 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:04:51.975 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:04:51.982 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:51.981+0000 7fe455e32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:04:52.030 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:04:52.030 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:04:52.030 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:04:52.033 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:52.032+0000 7f95c9812040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:04:52.083 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:04:52.084 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:04:52.084 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:04:52.107 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:52.106+0000 7fe455e32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:04:52.137 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:04:52.137 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:04:52.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.348+0000 7f2634a6f640 1 Processor -- start 2023-10-16T10:04:52.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.348+0000 7f2634a6f640 1 -- start start 2023-10-16T10:04:52.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.350+0000 7f2634a6f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26300adb50 0x7f263006f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:52.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.350+0000 7f2634a6f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f26300b0b20 0x7f2630164f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:52.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.350+0000 7f2634a6f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2630071450 con 0x7f2630070fd0 2023-10-16T10:04:52.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.350+0000 7f2634a6f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26300cc600 con 0x7f26300b0b20 2023-10-16T10:04:52.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.350+0000 7f2634a6f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26300cc8e0 con 0x7f26300adb50 2023-10-16T10:04:52.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.350+0000 7f262ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f26300b0b20 0x7f2630164f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:52.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.350+0000 7f262ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f26300b0b20 0x7f2630164f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59664/0 (socket says 172.21.15.84:59664) 2023-10-16T10:04:52.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.350+0000 7f262ed9d640 1 -- 172.21.15.84:0/1085955803 learned_addr learned my addr 172.21.15.84:0/1085955803 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:52.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.350+0000 7f262e59c640 1 --2- 172.21.15.84:0/1085955803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26300adb50 0x7f263006f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:52.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.351+0000 7f262ed9d640 1 -- 172.21.15.84:0/1085955803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26300adb50 msgr2=0x7f263006f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:52.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.351+0000 7f262ed9d640 1 --2- 172.21.15.84:0/1085955803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26300adb50 0x7f263006f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:52.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.351+0000 7f262ed9d640 1 -- 172.21.15.84:0/1085955803 >> v1:172.21.15.157:6789/0 conn(0x7f2630070fd0 legacy=0x7f26300b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:52.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.351+0000 7f262ed9d640 1 -- 172.21.15.84:0/1085955803 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26300ccbc0 con 0x7f26300b0b20 2023-10-16T10:04:52.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.352+0000 7f262ed9d640 1 --2- 172.21.15.84:0/1085955803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26300b0b20 0x7f2630164f60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2628009e50 tx=0x7f262802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52eb3a13d6b8616f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:52.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.353+0000 7f262dd9b640 1 -- 172.21.15.84:0/1085955803 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262802cb00 con 0x7f26300b0b20 2023-10-16T10:04:52.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.353+0000 7f262dd9b640 1 -- 172.21.15.84:0/1085955803 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262802cc60 con 0x7f26300b0b20 2023-10-16T10:04:52.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.353+0000 7f262dd9b640 1 -- 172.21.15.84:0/1085955803 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26280343f0 con 0x7f26300b0b20 2023-10-16T10:04:52.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.353+0000 7f2634a6f640 1 -- 172.21.15.84:0/1085955803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26300b0b20 msgr2=0x7f2630164f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:52.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.353+0000 7f2634a6f640 1 --2- 172.21.15.84:0/1085955803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26300b0b20 0x7f2630164f60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2628009e50 tx=0x7f262802bf60 comp rx=0 tx=0).stop 2023-10-16T10:04:52.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.353+0000 7f2634a6f640 1 -- 172.21.15.84:0/1085955803 shutdown_connections 2023-10-16T10:04:52.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.353+0000 7f2634a6f640 1 --2- 172.21.15.84:0/1085955803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26300b0b20 0x7f2630164f60 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:52.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.353+0000 7f2634a6f640 1 --2- 172.21.15.84:0/1085955803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26300adb50 0x7f263006f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:52.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.353+0000 7f2634a6f640 1 -- 172.21.15.84:0/1085955803 >> 172.21.15.84:0/1085955803 conn(0x7f26300cb410 msgr2=0x7f26300cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:52.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.354+0000 7f2634a6f640 1 -- 172.21.15.84:0/1085955803 shutdown_connections 2023-10-16T10:04:52.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.354+0000 7f2634a6f640 1 -- 172.21.15.84:0/1085955803 wait complete. 2023-10-16T10:04:52.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.355+0000 7f2634a6f640 1 Processor -- start 2023-10-16T10:04:52.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.355+0000 7f2634a6f640 1 -- start start 2023-10-16T10:04:52.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.356+0000 7f2634a6f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2630070fd0 0x7f2630164510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:52.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.356+0000 7f2634a6f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26300adb50 0x7f26300c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:52.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.356+0000 7f2634a6f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26300b0b20 0x7f26300b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:52.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.356+0000 7f2634a6f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26300c5790 con 0x7f2630070fd0 2023-10-16T10:04:52.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.356+0000 7f2634a6f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26300b04a0 con 0x7f26300adb50 2023-10-16T10:04:52.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.356+0000 7f2634a6f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26300b0780 con 0x7f26300b0b20 2023-10-16T10:04:52.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.356+0000 7f262e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2630070fd0 0x7f2630164510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:52.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.357+0000 7f262e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2630070fd0 0x7f2630164510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59678/0 (socket says 172.21.15.84:59678) 2023-10-16T10:04:52.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.357+0000 7f262e59c640 1 -- 172.21.15.84:0/4089174782 learned_addr learned my addr 172.21.15.84:0/4089174782 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:52.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.357+0000 7f2627fff640 1 --2- 172.21.15.84:0/4089174782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26300adb50 0x7f26300c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:52.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.357+0000 7f262ed9d640 1 --2- 172.21.15.84:0/4089174782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26300b0b20 0x7f26300b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:52.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.357+0000 7f262e59c640 1 -- 172.21.15.84:0/4089174782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26300b0b20 msgr2=0x7f26300b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:52.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.357+0000 7f262e59c640 1 --2- 172.21.15.84:0/4089174782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26300b0b20 0x7f26300b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:52.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.357+0000 7f262e59c640 1 -- 172.21.15.84:0/4089174782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26300adb50 msgr2=0x7f26300c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:52.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.357+0000 7f262e59c640 1 --2- 172.21.15.84:0/4089174782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26300adb50 0x7f26300c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:52.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.357+0000 7f262e59c640 1 -- 172.21.15.84:0/4089174782 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2628009b30 con 0x7f2630070fd0 2023-10-16T10:04:52.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.357+0000 7f262e59c640 1 --2- 172.21.15.84:0/4089174782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2630070fd0 0x7f2630164510 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f262000e9b0 tx=0x7f262000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:52.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.358+0000 7f2626ffd640 1 -- 172.21.15.84:0/4089174782 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2620009670 con 0x7f2630070fd0 2023-10-16T10:04:52.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.358+0000 7f2626ffd640 1 -- 172.21.15.84:0/4089174782 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262000c9a0 con 0x7f2630070fd0 2023-10-16T10:04:52.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.358+0000 7f2626ffd640 1 -- 172.21.15.84:0/4089174782 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262000cd50 con 0x7f2630070fd0 2023-10-16T10:04:52.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.358+0000 7f2634a6f640 1 -- 172.21.15.84:0/4089174782 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2630163d10 con 0x7f2630070fd0 2023-10-16T10:04:52.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.358+0000 7f2634a6f640 1 -- 172.21.15.84:0/4089174782 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26300b0130 con 0x7f2630070fd0 2023-10-16T10:04:52.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.359+0000 7f2634a6f640 1 -- 172.21.15.84:0/4089174782 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26300c20d0 con 0x7f2630070fd0 2023-10-16T10:04:52.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.360+0000 7f2626ffd640 1 -- 172.21.15.84:0/4089174782 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 37) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f2620013470 con 0x7f2630070fd0 2023-10-16T10:04:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.361+0000 7f2626ffd640 1 --2- 172.21.15.84:0/4089174782 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f261006af90 0x7f261006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.361+0000 7f2626ffd640 1 -- 172.21.15.84:0/4089174782 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f2620089a60 con 0x7f2630070fd0 2023-10-16T10:04:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.361+0000 7f2627fff640 1 -- 172.21.15.84:0/4089174782 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f261006af90 msgr2=0x7f261006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2103121481 2023-10-16T10:04:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.361+0000 7f2627fff640 1 --2- 172.21.15.84:0/4089174782 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f261006af90 0x7f261006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.365+0000 7f2626ffd640 1 -- 172.21.15.84:0/4089174782 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2620054940 con 0x7f2630070fd0 2023-10-16T10:04:52.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.562+0000 7f2627fff640 1 -- 172.21.15.84:0/4089174782 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f261006af90 msgr2=0x7f261006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2103121481 2023-10-16T10:04:52.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.562+0000 7f2627fff640 1 --2- 172.21.15.84:0/4089174782 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f261006af90 0x7f261006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:04:52.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.604+0000 7f2634a6f640 1 -- 172.21.15.84:0/4089174782 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f263004e160 con 0x7f2630070fd0 2023-10-16T10:04:52.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.605+0000 7f2626ffd640 1 -- 172.21.15.84:0/4089174782 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f2620017090 con 0x7f2630070fd0 2023-10-16T10:04:52.607 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:52.607 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5066,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1223610299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1223610299},{"type":"v1","addr":"172.21.15.157:6833","nonce":1223610299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3636443426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3636443426},{"type":"v1","addr":"172.21.15.157:6835","nonce":3636443426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6537,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/3575464874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3575464874},{"type":"v1","addr":"172.21.15.84:6836","nonce":3575464874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:04:32.849497+0000","modified":"2023-10-16T10:04:35.898561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.84:6837/2669751864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2669751864},{"type":"v1","addr":"172.21.15.84:6837","nonce":2669751864}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-16T10:04:52.608 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 35 2023-10-16T10:04:52.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.608+0000 7f2634a6f640 1 -- 172.21.15.84:0/4089174782 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f261006af90 msgr2=0x7f261006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:52.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.608+0000 7f2634a6f640 1 --2- 172.21.15.84:0/4089174782 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f261006af90 0x7f261006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:52.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.608+0000 7f2634a6f640 1 -- 172.21.15.84:0/4089174782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2630070fd0 msgr2=0x7f2630164510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:52.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.608+0000 7f2634a6f640 1 --2- 172.21.15.84:0/4089174782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2630070fd0 0x7f2630164510 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f262000e9b0 tx=0x7f262000ee70 comp rx=0 tx=0).stop 2023-10-16T10:04:52.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.608+0000 7f2634a6f640 1 -- 172.21.15.84:0/4089174782 shutdown_connections 2023-10-16T10:04:52.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.608+0000 7f2634a6f640 1 --2- 172.21.15.84:0/4089174782 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f261006af90 0x7f261006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:52.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.608+0000 7f2634a6f640 1 --2- 172.21.15.84:0/4089174782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26300b0b20 0x7f26300b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:52.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.608+0000 7f2634a6f640 1 --2- 172.21.15.84:0/4089174782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26300adb50 0x7f26300c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:52.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.608+0000 7f2634a6f640 1 --2- 172.21.15.84:0/4089174782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2630070fd0 0x7f2630164510 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:52.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.608+0000 7f2634a6f640 1 -- 172.21.15.84:0/4089174782 >> 172.21.15.84:0/4089174782 conn(0x7f26300cb410 msgr2=0x7f263015f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:52.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.608+0000 7f2634a6f640 1 -- 172.21.15.84:0/4089174782 shutdown_connections 2023-10-16T10:04:52.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.608+0000 7f2634a6f640 1 -- 172.21.15.84:0/4089174782 wait complete. 2023-10-16T10:04:52.626 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:04:52.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.811+0000 7f0b510a2640 1 Processor -- start 2023-10-16T10:04:52.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.811+0000 7f0b510a2640 1 -- start start 2023-10-16T10:04:52.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.813+0000 7f0b510a2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0b4c15aaa0 0x7f0b4c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:52.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.813+0000 7f0b510a2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0b4c15ae80 0x7f0b4c0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:52.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.813+0000 7f0b510a2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b4c15ed60 con 0x7f0b4c0ccaa0 2023-10-16T10:04:52.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.813+0000 7f0b510a2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b4c06b810 con 0x7f0b4c15aaa0 2023-10-16T10:04:52.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.813+0000 7f0b510a2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b4c0707e0 con 0x7f0b4c15ae80 2023-10-16T10:04:52.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.813+0000 7f0b4ad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0b4c15aaa0 0x7f0b4c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:52.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.813+0000 7f0b4ad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0b4c15aaa0 0x7f0b4c15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59688/0 (socket says 172.21.15.84:59688) 2023-10-16T10:04:52.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.813+0000 7f0b4ad9d640 1 -- 172.21.15.84:0/2455044989 learned_addr learned my addr 172.21.15.84:0/2455044989 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:52.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.813+0000 7f0b43fff640 1 --2- 172.21.15.84:0/2455044989 >> v2:172.21.15.157:3300/0 conn(0x7f0b4c15ae80 0x7f0b4c0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:52.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.814+0000 7f0b4ad9d640 1 -- 172.21.15.84:0/2455044989 >> v2:172.21.15.157:3300/0 conn(0x7f0b4c15ae80 msgr2=0x7f0b4c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:52.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.814+0000 7f0b4ad9d640 1 --2- 172.21.15.84:0/2455044989 >> v2:172.21.15.157:3300/0 conn(0x7f0b4c15ae80 0x7f0b4c0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:52.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.814+0000 7f0b4ad9d640 1 -- 172.21.15.84:0/2455044989 >> v1:172.21.15.157:6789/0 conn(0x7f0b4c0ccaa0 legacy=0x7f0b4c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:52.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.814+0000 7f0b4ad9d640 1 -- 172.21.15.84:0/2455044989 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c15ef90 con 0x7f0b4c15aaa0 2023-10-16T10:04:52.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.814+0000 7f0b43fff640 1 --2- 172.21.15.84:0/2455044989 >> v2:172.21.15.157:3300/0 conn(0x7f0b4c15ae80 0x7f0b4c0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:52.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.815+0000 7f0b4ad9d640 1 --2- 172.21.15.84:0/2455044989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b4c15aaa0 0x7f0b4c15e820 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c0c59c0 tx=0x7f0b3c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e2f6986fcdc32e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:52.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.815+0000 7f0b4a59c640 1 -- 172.21.15.84:0/2455044989 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c030860 con 0x7f0b4c15aaa0 2023-10-16T10:04:52.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.815+0000 7f0b4a59c640 1 -- 172.21.15.84:0/2455044989 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3c0309c0 con 0x7f0b4c15aaa0 2023-10-16T10:04:52.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.815+0000 7f0b4a59c640 1 -- 172.21.15.84:0/2455044989 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c030d50 con 0x7f0b4c15aaa0 2023-10-16T10:04:52.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.816+0000 7f0b510a2640 1 -- 172.21.15.84:0/2455044989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b4c15aaa0 msgr2=0x7f0b4c15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:52.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.816+0000 7f0b510a2640 1 --2- 172.21.15.84:0/2455044989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b4c15aaa0 0x7f0b4c15e820 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c0c59c0 tx=0x7f0b3c02b9a0 comp rx=0 tx=0).stop 2023-10-16T10:04:52.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.816+0000 7f0b510a2640 1 -- 172.21.15.84:0/2455044989 shutdown_connections 2023-10-16T10:04:52.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.816+0000 7f0b510a2640 1 --2- 172.21.15.84:0/2455044989 >> v2:172.21.15.157:3300/0 conn(0x7f0b4c15ae80 0x7f0b4c0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:52.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.816+0000 7f0b510a2640 1 --2- 172.21.15.84:0/2455044989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b4c15aaa0 0x7f0b4c15e820 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:52.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.816+0000 7f0b510a2640 1 -- 172.21.15.84:0/2455044989 >> 172.21.15.84:0/2455044989 conn(0x7f0b4c0cb410 msgr2=0x7f0b4c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:52.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.816+0000 7f0b510a2640 1 -- 172.21.15.84:0/2455044989 shutdown_connections 2023-10-16T10:04:52.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.816+0000 7f0b510a2640 1 -- 172.21.15.84:0/2455044989 wait complete. 2023-10-16T10:04:52.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.818+0000 7f0b510a2640 1 Processor -- start 2023-10-16T10:04:52.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.818+0000 7f0b510a2640 1 -- start start 2023-10-16T10:04:52.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.819+0000 7f0b510a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b4c0ccaa0 0x7f0b4c0c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:52.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.819+0000 7f0b510a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b4c15aaa0 0x7f0b4c146cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:52.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.819+0000 7f0b510a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b4c15ae80 0x7f0b4c0c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:52.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.819+0000 7f0b510a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0c80e0 con 0x7f0b4c15aaa0 2023-10-16T10:04:52.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.819+0000 7f0b510a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0c74d0 con 0x7f0b4c0ccaa0 2023-10-16T10:04:52.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.819+0000 7f0b510a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0c77b0 con 0x7f0b4c15ae80 2023-10-16T10:04:52.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.819+0000 7f0b43fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b4c15aaa0 0x7f0b4c146cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:52.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.819+0000 7f0b43fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b4c15aaa0 0x7f0b4c146cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59696/0 (socket says 172.21.15.84:59696) 2023-10-16T10:04:52.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.819+0000 7f0b43fff640 1 -- 172.21.15.84:0/1118003211 learned_addr learned my addr 172.21.15.84:0/1118003211 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:52.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.819+0000 7f0b4ad9d640 1 --2- 172.21.15.84:0/1118003211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b4c0ccaa0 0x7f0b4c0c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:52.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.819+0000 7f0b4b59e640 1 --2- 172.21.15.84:0/1118003211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b4c15ae80 0x7f0b4c0c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:52.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.819+0000 7f0b43fff640 1 -- 172.21.15.84:0/1118003211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b4c15ae80 msgr2=0x7f0b4c0c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:52.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.819+0000 7f0b43fff640 1 --2- 172.21.15.84:0/1118003211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b4c15ae80 0x7f0b4c0c67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:52.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.819+0000 7f0b43fff640 1 -- 172.21.15.84:0/1118003211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b4c0ccaa0 msgr2=0x7f0b4c0c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:52.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.819+0000 7f0b43fff640 1 --2- 172.21.15.84:0/1118003211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b4c0ccaa0 0x7f0b4c0c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:52.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.819+0000 7f0b43fff640 1 -- 172.21.15.84:0/1118003211 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c0094d0 con 0x7f0b4c15aaa0 2023-10-16T10:04:52.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.819+0000 7f0b4b59e640 1 --2- 172.21.15.84:0/1118003211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b4c15ae80 0x7f0b4c0c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:52.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.819+0000 7f0b4ad9d640 1 --2- 172.21.15.84:0/1118003211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b4c0ccaa0 0x7f0b4c0c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:52.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.820+0000 7f0b43fff640 1 --2- 172.21.15.84:0/1118003211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b4c15aaa0 0x7f0b4c146cb0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400d800 tx=0x7f0b3400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:52.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.820+0000 7f0b437fe640 1 -- 172.21.15.84:0/1118003211 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34004030 con 0x7f0b4c15aaa0 2023-10-16T10:04:52.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.820+0000 7f0b437fe640 1 -- 172.21.15.84:0/1118003211 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b34004190 con 0x7f0b4c15aaa0 2023-10-16T10:04:52.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.820+0000 7f0b437fe640 1 -- 172.21.15.84:0/1118003211 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34004520 con 0x7f0b4c15aaa0 2023-10-16T10:04:52.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.820+0000 7f0b510a2640 1 -- 172.21.15.84:0/1118003211 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4c0c7a90 con 0x7f0b4c15aaa0 2023-10-16T10:04:52.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.820+0000 7f0b510a2640 1 -- 172.21.15.84:0/1118003211 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4c1471f0 con 0x7f0b4c15aaa0 2023-10-16T10:04:52.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.821+0000 7f0b510a2640 1 -- 172.21.15.84:0/1118003211 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4c0c8290 con 0x7f0b4c15aaa0 2023-10-16T10:04:52.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.823+0000 7f0b437fe640 1 -- 172.21.15.84:0/1118003211 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 37) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f0b340096a0 con 0x7f0b4c15aaa0 2023-10-16T10:04:52.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.824+0000 7f0b437fe640 1 --2- 172.21.15.84:0/1118003211 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f0b2c06afc0 0x7f0b2c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:52.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.824+0000 7f0b437fe640 1 -- 172.21.15.84:0/1118003211 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f0b34089920 con 0x7f0b4c15aaa0 2023-10-16T10:04:52.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.824+0000 7f0b4ad9d640 1 -- 172.21.15.84:0/1118003211 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f0b2c06afc0 msgr2=0x7f0b2c06d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2103121481 2023-10-16T10:04:52.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.824+0000 7f0b4ad9d640 1 --2- 172.21.15.84:0/1118003211 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f0b2c06afc0 0x7f0b2c06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:52.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:52.829+0000 7f0b437fe640 1 -- 172.21.15.84:0/1118003211 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0b4c0c8290 con 0x7f0b4c15aaa0 2023-10-16T10:04:52.854 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:52.853+0000 7f95c9812040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:04:52.942 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:52.941+0000 7fe455e32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:04:53.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.023+0000 7f0b510a2640 1 -- 172.21.15.84:0/1118003211 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b4c04e160 con 0x7f0b4c15aaa0 2023-10-16T10:04:53.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.025+0000 7f0b4ad9d640 1 -- 172.21.15.84:0/1118003211 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f0b2c06afc0 msgr2=0x7f0b2c06d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2103121481 2023-10-16T10:04:53.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.025+0000 7f0b4ad9d640 1 --2- 172.21.15.84:0/1118003211 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f0b2c06afc0 0x7f0b2c06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:04:53.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.026+0000 7f0b437fe640 1 -- 172.21.15.84:0/1118003211 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17042 (secure 0 0 0) 0x7f0b34058280 con 0x7f0b4c15aaa0 2023-10-16T10:04:53.045 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:53.045 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":83,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:04:47.124638+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:04:30.728624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-16T10:04:31.866029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/3177129458":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/1145018607":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/1499920052":"2023-10-17T10:04:47.124608+0000","172.21.15.84:6833/2091947430":"2023-10-17T10:04:47.124608+0000","172.21.15.84:6832/2091947430":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/2509405053":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/3991569032":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/3599228496":"2023-10-17T10:04:47.124608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:04:53.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.028+0000 7f0b510a2640 1 -- 172.21.15.84:0/1118003211 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f0b2c06afc0 msgr2=0x7f0b2c06d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:53.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.028+0000 7f0b510a2640 1 --2- 172.21.15.84:0/1118003211 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f0b2c06afc0 0x7f0b2c06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.028+0000 7f0b510a2640 1 -- 172.21.15.84:0/1118003211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b4c15aaa0 msgr2=0x7f0b4c146cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:53.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.028+0000 7f0b510a2640 1 --2- 172.21.15.84:0/1118003211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b4c15aaa0 0x7f0b4c146cb0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400d800 tx=0x7f0b3400dcc0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.028+0000 7f0b510a2640 1 -- 172.21.15.84:0/1118003211 shutdown_connections 2023-10-16T10:04:53.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.028+0000 7f0b510a2640 1 --2- 172.21.15.84:0/1118003211 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7f0b2c06afc0 0x7f0b2c06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.028+0000 7f0b510a2640 1 --2- 172.21.15.84:0/1118003211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b4c15ae80 0x7f0b4c0c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.028+0000 7f0b510a2640 1 --2- 172.21.15.84:0/1118003211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b4c15aaa0 0x7f0b4c146cb0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.028+0000 7f0b510a2640 1 --2- 172.21.15.84:0/1118003211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b4c0ccaa0 0x7f0b4c0c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.028+0000 7f0b510a2640 1 -- 172.21.15.84:0/1118003211 >> 172.21.15.84:0/1118003211 conn(0x7f0b4c0cb410 msgr2=0x7f0b4c0ae230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:53.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.029+0000 7f0b510a2640 1 -- 172.21.15.84:0/1118003211 shutdown_connections 2023-10-16T10:04:53.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.029+0000 7f0b510a2640 1 -- 172.21.15.84:0/1118003211 wait complete. 2023-10-16T10:04:53.050 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:04:30.728624+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:04:31.866029+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:04:53.050 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:04:53.150 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:53.150+0000 7f95c9812040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:04:53.209 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:53.208+0000 7f95c9812040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:04:53.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.250+0000 7fa4cb061640 1 Processor -- start 2023-10-16T10:04:53.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.250+0000 7fa4cb061640 1 -- start start 2023-10-16T10:04:53.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.251+0000 7fa4cb061640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c415af80 0x7fa4c40cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:53.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.252+0000 7fa4cb061640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa4c4065ab0 0x7fa4c4065f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:53.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.252+0000 7fa4cb061640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c4066450 con 0x7fa4c406faf0 2023-10-16T10:04:53.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.252+0000 7fa4cb061640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c40d0930 con 0x7fa4c4065ab0 2023-10-16T10:04:53.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.252+0000 7fa4cb061640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c415eff0 con 0x7fa4c415af80 2023-10-16T10:04:53.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.252+0000 7fa4c95fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa4c4065ab0 0x7fa4c4065f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:53.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.252+0000 7fa4c95fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa4c4065ab0 0x7fa4c4065f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59698/0 (socket says 172.21.15.84:59698) 2023-10-16T10:04:53.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.252+0000 7fa4c95fe640 1 -- 172.21.15.84:0/2093415365 learned_addr learned my addr 172.21.15.84:0/2093415365 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:53.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.252+0000 7fa4c8dfd640 1 --2- 172.21.15.84:0/2093415365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c415af80 0x7fa4c40cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:53.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.252+0000 7fa4c95fe640 1 -- 172.21.15.84:0/2093415365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c415af80 msgr2=0x7fa4c40cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:53.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.252+0000 7fa4c95fe640 1 --2- 172.21.15.84:0/2093415365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c415af80 0x7fa4c40cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.253+0000 7fa4c95fe640 1 -- 172.21.15.84:0/2093415365 >> v1:172.21.15.157:6789/0 conn(0x7fa4c406faf0 legacy=0x7fa4c415f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:53.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.253+0000 7fa4c95fe640 1 -- 172.21.15.84:0/2093415365 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c415ecc0 con 0x7fa4c4065ab0 2023-10-16T10:04:53.257 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:53.253+0000 7fe455e32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:04:53.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.254+0000 7fa4c95fe640 1 --2- 172.21.15.84:0/2093415365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4c4065ab0 0x7fa4c4065f10 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009e50 tx=0x7fa4bc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b12929e34a3145 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:53.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.254+0000 7fa4bb7fe640 1 -- 172.21.15.84:0/2093415365 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc02c9e0 con 0x7fa4c4065ab0 2023-10-16T10:04:53.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.254+0000 7fa4bb7fe640 1 -- 172.21.15.84:0/2093415365 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4bc02cb40 con 0x7fa4c4065ab0 2023-10-16T10:04:53.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.254+0000 7fa4bb7fe640 1 -- 172.21.15.84:0/2093415365 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc0343f0 con 0x7fa4c4065ab0 2023-10-16T10:04:53.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.254+0000 7fa4cb061640 1 -- 172.21.15.84:0/2093415365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4c4065ab0 msgr2=0x7fa4c4065f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:53.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.254+0000 7fa4cb061640 1 --2- 172.21.15.84:0/2093415365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4c4065ab0 0x7fa4c4065f10 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009e50 tx=0x7fa4bc02be40 comp rx=0 tx=0).stop 2023-10-16T10:04:53.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.255+0000 7fa4cb061640 1 -- 172.21.15.84:0/2093415365 shutdown_connections 2023-10-16T10:04:53.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.255+0000 7fa4cb061640 1 --2- 172.21.15.84:0/2093415365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4c4065ab0 0x7fa4c4065f10 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.255+0000 7fa4cb061640 1 --2- 172.21.15.84:0/2093415365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c415af80 0x7fa4c40cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.255+0000 7fa4cb061640 1 -- 172.21.15.84:0/2093415365 >> 172.21.15.84:0/2093415365 conn(0x7fa4c40d0ab0 msgr2=0x7fa4c40d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:53.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.255+0000 7fa4cb061640 1 -- 172.21.15.84:0/2093415365 shutdown_connections 2023-10-16T10:04:53.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.255+0000 7fa4cb061640 1 -- 172.21.15.84:0/2093415365 wait complete. 2023-10-16T10:04:53.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.256+0000 7fa4cb061640 1 Processor -- start 2023-10-16T10:04:53.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.257+0000 7fa4cb061640 1 -- start start 2023-10-16T10:04:53.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.257+0000 7fa4cb061640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4c4065ab0 0x7fa4c406c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:53.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.257+0000 7fa4cb061640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c406faf0 0x7fa4c40ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:53.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.258+0000 7fa4cb061640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c415af80 0x7fa4c40ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:53.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.258+0000 7fa4cb061640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c406cf30 con 0x7fa4c4065ab0 2023-10-16T10:04:53.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.258+0000 7fa4cb061640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c41f6e90 con 0x7fa4c406faf0 2023-10-16T10:04:53.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.258+0000 7fa4cb061640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c41f2b30 con 0x7fa4c415af80 2023-10-16T10:04:53.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.258+0000 7fa4c8dfd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4c4065ab0 0x7fa4c406c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:53.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.258+0000 7fa4c8dfd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4c4065ab0 0x7fa4c406c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59708/0 (socket says 172.21.15.84:59708) 2023-10-16T10:04:53.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.258+0000 7fa4c8dfd640 1 -- 172.21.15.84:0/3823344625 learned_addr learned my addr 172.21.15.84:0/3823344625 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:53.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.258+0000 7fa4bbfff640 1 --2- 172.21.15.84:0/3823344625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c406faf0 0x7fa4c40ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:53.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.258+0000 7fa4c95fe640 1 --2- 172.21.15.84:0/3823344625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c415af80 0x7fa4c40ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:53.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.258+0000 7fa4c8dfd640 1 -- 172.21.15.84:0/3823344625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c415af80 msgr2=0x7fa4c40ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:53.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.258+0000 7fa4c8dfd640 1 --2- 172.21.15.84:0/3823344625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c415af80 0x7fa4c40ae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.258+0000 7fa4c8dfd640 1 -- 172.21.15.84:0/3823344625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c406faf0 msgr2=0x7fa4c40ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:53.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.258+0000 7fa4c8dfd640 1 --2- 172.21.15.84:0/3823344625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c406faf0 0x7fa4c40ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.258+0000 7fa4c8dfd640 1 -- 172.21.15.84:0/3823344625 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4bc009b30 con 0x7fa4c4065ab0 2023-10-16T10:04:53.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.258+0000 7fa4bbfff640 1 --2- 172.21.15.84:0/3823344625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c406faf0 0x7fa4c40ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:53.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.259+0000 7fa4c8dfd640 1 --2- 172.21.15.84:0/3823344625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4c4065ab0 0x7fa4c406c9f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000d800 tx=0x7fa4c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:53.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.259+0000 7fa4b97fa640 1 -- 172.21.15.84:0/3823344625 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c000b6c0 con 0x7fa4c4065ab0 2023-10-16T10:04:53.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.259+0000 7fa4b97fa640 1 -- 172.21.15.84:0/3823344625 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c000b820 con 0x7fa4c4065ab0 2023-10-16T10:04:53.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.259+0000 7fa4b97fa640 1 -- 172.21.15.84:0/3823344625 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c000bbb0 con 0x7fa4c4065ab0 2023-10-16T10:04:53.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.259+0000 7fa4cb061640 1 -- 172.21.15.84:0/3823344625 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c41f2e10 con 0x7fa4c4065ab0 2023-10-16T10:04:53.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.259+0000 7fa4cb061640 1 -- 172.21.15.84:0/3823344625 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c41f33d0 con 0x7fa4c4065ab0 2023-10-16T10:04:53.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.260+0000 7fa4cb061640 1 -- 172.21.15.84:0/3823344625 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c406fef0 con 0x7fa4c4065ab0 2023-10-16T10:04:53.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.261+0000 7fa4b97fa640 1 -- 172.21.15.84:0/3823344625 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 37) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fa4c000bd10 con 0x7fa4c4065ab0 2023-10-16T10:04:53.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.262+0000 7fa4b97fa640 1 --2- 172.21.15.84:0/3823344625 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7fa4b006af90 0x7fa4b006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:53.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.262+0000 7fa4b97fa640 1 -- 172.21.15.84:0/3823344625 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fa4c0089e90 con 0x7fa4c4065ab0 2023-10-16T10:04:53.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.262+0000 7fa4bbfff640 1 -- 172.21.15.84:0/3823344625 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7fa4b006af90 msgr2=0x7fa4b006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2103121481 2023-10-16T10:04:53.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.262+0000 7fa4bbfff640 1 --2- 172.21.15.84:0/3823344625 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7fa4b006af90 0x7fa4b006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:53.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.266+0000 7fa4b97fa640 1 -- 172.21.15.84:0/3823344625 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa4c0054cd0 con 0x7fa4c4065ab0 2023-10-16T10:04:53.315 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:53.313+0000 7fe455e32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:04:53.324 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:04:53.324+0000 7f95c9812040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:04:53.433 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:53.432+0000 7fe455e32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:04:53.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.453+0000 7fa4b97fa640 1 -- 172.21.15.84:0/3823344625 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 38) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fa4c0088250 con 0x7fa4c4065ab0 2023-10-16T10:04:53.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.454+0000 7fa4b97fa640 1 -- 172.21.15.84:0/3823344625 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7fa4b006af90 msgr2=0x7fa4b006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:53.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.454+0000 7fa4b97fa640 1 --2- 172.21.15.84:0/3823344625 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7fa4b006af90 0x7fa4b006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.468+0000 7fa4cb061640 1 -- 172.21.15.84:0/3823344625 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa4c404e160 con 0x7fa4c4065ab0 2023-10-16T10:04:53.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.469+0000 7fa4b97fa640 1 -- 172.21.15.84:0/3823344625 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa4c0073ae0 con 0x7fa4c4065ab0 2023-10-16T10:04:53.471 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:04:53.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.471+0000 7fa4cb061640 1 -- 172.21.15.84:0/3823344625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4c4065ab0 msgr2=0x7fa4c406c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:53.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.471+0000 7fa4cb061640 1 --2- 172.21.15.84:0/3823344625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4c4065ab0 0x7fa4c406c9f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000d800 tx=0x7fa4c000dcc0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.472+0000 7fa4cb061640 1 -- 172.21.15.84:0/3823344625 shutdown_connections 2023-10-16T10:04:53.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.472+0000 7fa4cb061640 1 --2- 172.21.15.84:0/3823344625 >> [v2:172.21.15.84:6832/2103121481,v1:172.21.15.84:6833/2103121481] conn(0x7fa4b006af90 0x7fa4b006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.472+0000 7fa4cb061640 1 --2- 172.21.15.84:0/3823344625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4c415af80 0x7fa4c40ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.472+0000 7fa4cb061640 1 --2- 172.21.15.84:0/3823344625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c406faf0 0x7fa4c40ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.472+0000 7fa4cb061640 1 --2- 172.21.15.84:0/3823344625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4c4065ab0 0x7fa4c406c9f0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.472+0000 7fa4cb061640 1 -- 172.21.15.84:0/3823344625 >> 172.21.15.84:0/3823344625 conn(0x7fa4c40d0ab0 msgr2=0x7fa4c4158ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:53.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.472+0000 7fa4cb061640 1 -- 172.21.15.84:0/3823344625 shutdown_connections 2023-10-16T10:04:53.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.472+0000 7fa4cb061640 1 -- 172.21.15.84:0/3823344625 wait complete. 2023-10-16T10:04:53.495 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:04:53.528 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:53.526+0000 7fe3d622b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:53.529 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:53.527+0000 7fe3d622b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:53.529 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:53.527+0000 7fe3d622b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:53.530 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:53.527+0000 7fe3d622b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:53.530 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:53.527+0000 7fe3d622b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:53.530 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:53.528+0000 7fe3d9231640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:53.530 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:53.528+0000 7fe3d9231640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:53.530 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:53.528+0000 7fe3d9231640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:53.531 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:53.528+0000 7fe3d9231640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:53.531 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:04:53.528+0000 7fe3d9231640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.672+0000 7f118a50a640 1 Processor -- start 2023-10-16T10:04:53.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.672+0000 7f118a50a640 1 -- start start 2023-10-16T10:04:53.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.673+0000 7f118a50a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f118415c9f0 0x7f11840c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:53.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.673+0000 7f118a50a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f118415dde0 0x7f11840ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:53.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.673+0000 7f118a50a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1184160a90 con 0x7f11840c9230 2023-10-16T10:04:53.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.673+0000 7f118a50a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11840cdec0 con 0x7f118415dde0 2023-10-16T10:04:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.673+0000 7f118a50a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11840cd900 con 0x7f118415c9f0 2023-10-16T10:04:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.674+0000 7f1189d09640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f118415dde0 0x7f11840ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.674+0000 7f1189d09640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f118415dde0 0x7f11840ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59798/0 (socket says 172.21.15.84:59798) 2023-10-16T10:04:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.674+0000 7f1189d09640 1 -- 172.21.15.84:0/988363783 learned_addr learned my addr 172.21.15.84:0/988363783 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.674+0000 7f1189d09640 1 -- 172.21.15.84:0/988363783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f118415c9f0 msgr2=0x7f11840c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.674+0000 7f1189508640 1 --2- 172.21.15.84:0/988363783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f118415c9f0 0x7f11840c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.674+0000 7f1189d09640 1 --2- 172.21.15.84:0/988363783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f118415c9f0 0x7f11840c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.674+0000 7f1189d09640 1 -- 172.21.15.84:0/988363783 >> v1:172.21.15.84:6789/0 conn(0x7f11840c9230 legacy=0x7f11840709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:53.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.674+0000 7f1189d09640 1 -- 172.21.15.84:0/988363783 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11840cdbe0 con 0x7f118415dde0 2023-10-16T10:04:53.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.674+0000 7f117bfff640 1 -- 172.21.15.84:0/988363783 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590865833 0 0) 0x7f1184160a90 con 0x7f11840c9230 2023-10-16T10:04:53.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.674+0000 7f1189508640 1 --2- 172.21.15.84:0/988363783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f118415c9f0 0x7f11840c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:53.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.675+0000 7f1189d09640 1 --2- 172.21.15.84:0/988363783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f118415dde0 0x7f11840ac890 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f117c009e50 tx=0x7f117c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c999d06d7c21a76f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:53.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.675+0000 7f117bfff640 1 -- 172.21.15.84:0/988363783 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c02cb00 con 0x7f118415dde0 2023-10-16T10:04:53.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.675+0000 7f117bfff640 1 -- 172.21.15.84:0/988363783 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c02cc60 con 0x7f118415dde0 2023-10-16T10:04:53.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.675+0000 7f117bfff640 1 -- 172.21.15.84:0/988363783 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c0343f0 con 0x7f118415dde0 2023-10-16T10:04:53.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.676+0000 7f118a50a640 1 -- 172.21.15.84:0/988363783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f118415dde0 msgr2=0x7f11840ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:53.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.676+0000 7f118a50a640 1 --2- 172.21.15.84:0/988363783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f118415dde0 0x7f11840ac890 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f117c009e50 tx=0x7f117c02bf60 comp rx=0 tx=0).stop 2023-10-16T10:04:53.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.676+0000 7f118a50a640 1 -- 172.21.15.84:0/988363783 shutdown_connections 2023-10-16T10:04:53.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.676+0000 7f118a50a640 1 --2- 172.21.15.84:0/988363783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f118415dde0 0x7f11840ac890 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.676+0000 7f118a50a640 1 --2- 172.21.15.84:0/988363783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f118415c9f0 0x7f11840c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.676+0000 7f118a50a640 1 -- 172.21.15.84:0/988363783 >> 172.21.15.84:0/988363783 conn(0x7f11840ce6a0 msgr2=0x7f11840ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:53.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.676+0000 7f118a50a640 1 -- 172.21.15.84:0/988363783 shutdown_connections 2023-10-16T10:04:53.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.676+0000 7f118a50a640 1 -- 172.21.15.84:0/988363783 wait complete. 2023-10-16T10:04:53.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.678+0000 7f118a50a640 1 Processor -- start 2023-10-16T10:04:53.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.678+0000 7f118a50a640 1 -- start start 2023-10-16T10:04:53.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.679+0000 7f118a50a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11840c9230 0x7f11840c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:53.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.679+0000 7f118a50a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f118415c9f0 0x7f11840c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:53.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.679+0000 7f118a50a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f118415dde0 0x7f11840c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:53.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.679+0000 7f118a50a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118406adf0 con 0x7f11840c9230 2023-10-16T10:04:53.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.679+0000 7f118a50a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11840c80b0 con 0x7f118415dde0 2023-10-16T10:04:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.679+0000 7f118a50a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11840c8380 con 0x7f118415c9f0 2023-10-16T10:04:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.679+0000 7f1189508640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11840c9230 0x7f11840c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.679+0000 7f1189508640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11840c9230 0x7f11840c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59806/0 (socket says 172.21.15.84:59806) 2023-10-16T10:04:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.679+0000 7f1189508640 1 -- 172.21.15.84:0/3357192361 learned_addr learned my addr 172.21.15.84:0/3357192361 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.680+0000 7f1189508640 1 -- 172.21.15.84:0/3357192361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f118415c9f0 msgr2=0x7f11840c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:53.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.680+0000 7f1189508640 1 --2- 172.21.15.84:0/3357192361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f118415c9f0 0x7f11840c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.680+0000 7f1189508640 1 -- 172.21.15.84:0/3357192361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f118415dde0 msgr2=0x7f11840c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.680+0000 7f1189508640 1 --2- 172.21.15.84:0/3357192361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f118415dde0 0x7f11840c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.680+0000 7f1189508640 1 -- 172.21.15.84:0/3357192361 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c009b30 con 0x7f11840c9230 2023-10-16T10:04:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.680+0000 7f1189508640 1 --2- 172.21.15.84:0/3357192361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11840c9230 0x7f11840c57e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f118000c7f0 tx=0x7f118000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:53.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.680+0000 7f1179ffb640 1 -- 172.21.15.84:0/3357192361 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180007a30 con 0x7f11840c9230 2023-10-16T10:04:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.680+0000 7f1179ffb640 1 -- 172.21.15.84:0/3357192361 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1180007b90 con 0x7f11840c9230 2023-10-16T10:04:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.680+0000 7f1179ffb640 1 -- 172.21.15.84:0/3357192361 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118000ce40 con 0x7f11840c9230 2023-10-16T10:04:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.680+0000 7f118a50a640 1 -- 172.21.15.84:0/3357192361 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11840c8660 con 0x7f11840c9230 2023-10-16T10:04:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.680+0000 7f118a50a640 1 -- 172.21.15.84:0/3357192361 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118406b0b0 con 0x7f11840c9230 2023-10-16T10:04:53.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.681+0000 7f118a50a640 1 -- 172.21.15.84:0/3357192361 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118404e9b0 con 0x7f11840c9230 2023-10-16T10:04:53.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.682+0000 7f1179ffb640 1 -- 172.21.15.84:0/3357192361 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 38) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f11800113f0 con 0x7f11840c9230 2023-10-16T10:04:53.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.683+0000 7f1179ffb640 1 -- 172.21.15.84:0/3357192361 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f11800895d0 con 0x7f11840c9230 2023-10-16T10:04:53.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.687+0000 7f1179ffb640 1 -- 172.21.15.84:0/3357192361 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1180054730 con 0x7f11840c9230 2023-10-16T10:04:53.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.865+0000 7f118a50a640 1 -- 172.21.15.84:0/3357192361 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f11840adac0 con 0x7f11840c9230 2023-10-16T10:04:53.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.866+0000 7f1179ffb640 1 -- 172.21.15.84:0/3357192361 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1180057d50 con 0x7f11840c9230 2023-10-16T10:04:53.867 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T10:04:53.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.869+0000 7f118a50a640 1 -- 172.21.15.84:0/3357192361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11840c9230 msgr2=0x7f11840c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:53.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.869+0000 7f118a50a640 1 --2- 172.21.15.84:0/3357192361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11840c9230 0x7f11840c57e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f118000c7f0 tx=0x7f118000cbf0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.869+0000 7f118a50a640 1 -- 172.21.15.84:0/3357192361 shutdown_connections 2023-10-16T10:04:53.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.869+0000 7f118a50a640 1 --2- 172.21.15.84:0/3357192361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f118415dde0 0x7f11840c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.869+0000 7f118a50a640 1 --2- 172.21.15.84:0/3357192361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f118415c9f0 0x7f11840c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.869+0000 7f118a50a640 1 --2- 172.21.15.84:0/3357192361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11840c9230 0x7f11840c57e0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:53.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.869+0000 7f118a50a640 1 -- 172.21.15.84:0/3357192361 >> 172.21.15.84:0/3357192361 conn(0x7f11840ce6a0 msgr2=0x7f11840c1430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:53.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.869+0000 7f118a50a640 1 -- 172.21.15.84:0/3357192361 shutdown_connections 2023-10-16T10:04:53.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:53.869+0000 7f118a50a640 1 -- 172.21.15.84:0/3357192361 wait complete. 2023-10-16T10:04:53.888 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:04:54.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.147+0000 7fe88b59e640 1 Processor -- start 2023-10-16T10:04:54.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.147+0000 7fe88b59e640 1 -- start start 2023-10-16T10:04:54.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.149+0000 7fe88b59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88c0ad970 0x7fe88c0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:54.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.149+0000 7fe88b59e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88c15b9d0 con 0x7fe88c06c040 2023-10-16T10:04:54.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.149+0000 7fe88b59e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88c15bcb0 con 0x7fe88c0ccfb0 2023-10-16T10:04:54.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.149+0000 7fe88b59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c15bf90 con 0x7fe88c0ad970 2023-10-16T10:04:54.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.149+0000 7fe88a59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe88c06c040 0x7fe88c06c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48836/0 (socket says 172.21.15.84:48836) 2023-10-16T10:04:54.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.149+0000 7fe88a59c640 1 -- 172.21.15.84:0/1995033792 learned_addr learned my addr 172.21.15.84:0/1995033792 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:54.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.149+0000 7fe889d9b640 1 -- 172.21.15.84:0/1995033792 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353992421 0 0) 0x7fe88c15b9d0 con 0x7fe88c06c040 2023-10-16T10:04:54.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.149+0000 7fe889d9b640 1 -- 172.21.15.84:0/1995033792 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe878003520 con 0x7fe88c06c040 2023-10-16T10:04:54.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.149+0000 7fe881d9b640 1 --2- 172.21.15.84:0/1995033792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88c0ad970 0x7fe88c0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:54.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.150+0000 7fe889d9b640 1 -- 172.21.15.84:0/1995033792 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1405848786 0 0) 0x7fe878003520 con 0x7fe88c06c040 2023-10-16T10:04:54.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.150+0000 7fe889d9b640 1 -- 172.21.15.84:0/1995033792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88c0ad970 msgr2=0x7fe88c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:54.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.150+0000 7fe889d9b640 1 --2- 172.21.15.84:0/1995033792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88c0ad970 0x7fe88c0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.150+0000 7fe889d9b640 1 -- 172.21.15.84:0/1995033792 >> v1:172.21.15.157:6789/0 conn(0x7fe88c0ccfb0 legacy=0x7fe88c0cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:54.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.150+0000 7fe889d9b640 1 -- 172.21.15.84:0/1995033792 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c0cc270 con 0x7fe88c06c040 2023-10-16T10:04:54.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.150+0000 7fe889d9b640 1 -- 172.21.15.84:0/1995033792 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fe87c001670 con 0x7fe88c06c040 2023-10-16T10:04:54.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.150+0000 7fe889d9b640 1 -- 172.21.15.84:0/1995033792 >> v1:172.21.15.84:6789/0 conn(0x7fe88c06c040 legacy=0x7fe88c06c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:54.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.150+0000 7fe889d9b640 1 --2- 172.21.15.84:0/1995033792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88c0ad970 0x7fe8780039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:54.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.150+0000 7fe889d9b640 1 --2- 172.21.15.84:0/1995033792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe878004000 0x7fe8780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:54.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.150+0000 7fe889d9b640 1 --2- 172.21.15.84:0/1995033792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe878004d00 0x7fe878005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:54.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.150+0000 7fe889d9b640 1 -- 172.21.15.84:0/1995033792 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c15bf90 con 0x7fe878004000 2023-10-16T10:04:54.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.150+0000 7fe889d9b640 1 -- 172.21.15.84:0/1995033792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87800bb70 con 0x7fe878004d00 2023-10-16T10:04:54.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.150+0000 7fe889d9b640 1 -- 172.21.15.84:0/1995033792 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87800bd30 con 0x7fe88c0ad970 2023-10-16T10:04:54.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.150+0000 7fe881d9b640 1 --2- 172.21.15.84:0/1995033792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe878004000 0x7fe8780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:54.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.151+0000 7fe88a59c640 1 --2- 172.21.15.84:0/1995033792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88c0ad970 0x7fe8780039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:54.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.151+0000 7fe881d9b640 1 -- 172.21.15.84:0/1995033792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88c0ad970 msgr2=0x7fe8780039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:54.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.151+0000 7fe881d9b640 1 --2- 172.21.15.84:0/1995033792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88c0ad970 0x7fe8780039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.151+0000 7fe88ad9d640 1 --2- 172.21.15.84:0/1995033792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe878004d00 0x7fe878005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:54.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.151+0000 7fe881d9b640 1 -- 172.21.15.84:0/1995033792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe878004d00 msgr2=0x7fe878005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:54.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.151+0000 7fe881d9b640 1 --2- 172.21.15.84:0/1995033792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe878004d00 0x7fe878005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.151+0000 7fe881d9b640 1 -- 172.21.15.84:0/1995033792 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe878003520 con 0x7fe878004000 2023-10-16T10:04:54.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.151+0000 7fe88a59c640 1 --2- 172.21.15.84:0/1995033792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88c0ad970 0x7fe8780039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:54.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.151+0000 7fe88ad9d640 1 --2- 172.21.15.84:0/1995033792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe878004d00 0x7fe878005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:54.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.152+0000 7fe881d9b640 1 --2- 172.21.15.84:0/1995033792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe878004000 0x7fe8780047c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe87400ba30 tx=0x7fe8780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc313457942dff7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:54.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.152+0000 7fe889d9b640 1 -- 172.21.15.84:0/1995033792 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874032630 con 0x7fe878004000 2023-10-16T10:04:54.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.152+0000 7fe889d9b640 1 -- 172.21.15.84:0/1995033792 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe874032790 con 0x7fe878004000 2023-10-16T10:04:54.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.152+0000 7fe889d9b640 1 -- 172.21.15.84:0/1995033792 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874032b20 con 0x7fe878004000 2023-10-16T10:04:54.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.153+0000 7fe88b59e640 1 -- 172.21.15.84:0/1995033792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe878004000 msgr2=0x7fe8780047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:54.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.153+0000 7fe88b59e640 1 --2- 172.21.15.84:0/1995033792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe878004000 0x7fe8780047c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe87400ba30 tx=0x7fe8780057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.153+0000 7fe88ad9d640 1 -- 172.21.15.84:0/1995033792 reap_dead start 2023-10-16T10:04:54.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.153+0000 7fe88b59e640 1 -- 172.21.15.84:0/1995033792 shutdown_connections 2023-10-16T10:04:54.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.153+0000 7fe88b59e640 1 -- 172.21.15.84:0/1995033792 >> 172.21.15.84:0/1995033792 conn(0x7fe88c0ce6a0 msgr2=0x7fe88c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:54.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.153+0000 7fe88b59e640 1 -- 172.21.15.84:0/1995033792 shutdown_connections 2023-10-16T10:04:54.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.153+0000 7fe88b59e640 1 -- 172.21.15.84:0/1995033792 wait complete. 2023-10-16T10:04:54.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.155+0000 7fe88b59e640 1 Processor -- start 2023-10-16T10:04:54.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.155+0000 7fe88b59e640 1 -- start start 2023-10-16T10:04:54.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.155+0000 7fe88b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe878004000 0x7fe88c0c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:54.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.156+0000 7fe88b59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe88c0c77e0 0x7fe88c0c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:54.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.156+0000 7fe88b59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88c06f700 0x7fe88c06fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:54.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.156+0000 7fe88b59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c0700c0 con 0x7fe878004000 2023-10-16T10:04:54.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.156+0000 7fe88b59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c1f0600 con 0x7fe88c0c77e0 2023-10-16T10:04:54.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.156+0000 7fe88b59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c1f08e0 con 0x7fe88c06f700 2023-10-16T10:04:54.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.156+0000 7fe88a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe878004000 0x7fe88c0c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:54.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.156+0000 7fe88a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe878004000 0x7fe88c0c6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59824/0 (socket says 172.21.15.84:59824) 2023-10-16T10:04:54.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.156+0000 7fe88a59c640 1 -- 172.21.15.84:0/3517762215 learned_addr learned my addr 172.21.15.84:0/3517762215 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:54.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.156+0000 7fe88ad9d640 1 --2- 172.21.15.84:0/3517762215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88c06f700 0x7fe88c06fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:54.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.156+0000 7fe881d9b640 1 --2- 172.21.15.84:0/3517762215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe88c0c77e0 0x7fe88c0c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:54.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.156+0000 7fe88a59c640 1 -- 172.21.15.84:0/3517762215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88c06f700 msgr2=0x7fe88c06fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:54.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.156+0000 7fe88a59c640 1 --2- 172.21.15.84:0/3517762215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88c06f700 0x7fe88c06fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.156+0000 7fe88a59c640 1 -- 172.21.15.84:0/3517762215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe88c0c77e0 msgr2=0x7fe88c0c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:54.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.156+0000 7fe88a59c640 1 --2- 172.21.15.84:0/3517762215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe88c0c77e0 0x7fe88c0c7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.156+0000 7fe88a59c640 1 -- 172.21.15.84:0/3517762215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c0cdcd0 con 0x7fe878004000 2023-10-16T10:04:54.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.157+0000 7fe88a59c640 1 --2- 172.21.15.84:0/3517762215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe878004000 0x7fe88c0c6fc0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00d800 tx=0x7fe87c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:54.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.157+0000 7fe8837fe640 1 -- 172.21.15.84:0/3517762215 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c0074e0 con 0x7fe878004000 2023-10-16T10:04:54.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.157+0000 7fe8837fe640 1 -- 172.21.15.84:0/3517762215 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87c007640 con 0x7fe878004000 2023-10-16T10:04:54.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.157+0000 7fe8837fe640 1 -- 172.21.15.84:0/3517762215 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c004a00 con 0x7fe878004000 2023-10-16T10:04:54.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.157+0000 7fe88b59e640 1 -- 172.21.15.84:0/3517762215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe88c1f0bc0 con 0x7fe878004000 2023-10-16T10:04:54.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.157+0000 7fe88b59e640 1 -- 172.21.15.84:0/3517762215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88c1f1210 con 0x7fe878004000 2023-10-16T10:04:54.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.158+0000 7fe88b59e640 1 -- 172.21.15.84:0/3517762215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe88c06b330 con 0x7fe878004000 2023-10-16T10:04:54.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.159+0000 7fe8837fe640 1 -- 172.21.15.84:0/3517762215 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 38) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe87c004b60 con 0x7fe878004000 2023-10-16T10:04:54.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.160+0000 7fe8837fe640 1 -- 172.21.15.84:0/3517762215 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6397+0+0 (secure 0 0 0) 0x7fe87c088c90 con 0x7fe878004000 2023-10-16T10:04:54.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.164+0000 7fe8837fe640 1 -- 172.21.15.84:0/3517762215 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe87c053f30 con 0x7fe878004000 2023-10-16T10:04:54.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.355+0000 7fe88b59e640 1 -- 172.21.15.84:0/3517762215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe88c04e9b0 con 0x7fe878004000 2023-10-16T10:04:54.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.356+0000 7fe8837fe640 1 -- 172.21.15.84:0/3517762215 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe87c019020 con 0x7fe878004000 2023-10-16T10:04:54.358 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T10:04:54.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.359+0000 7fe88b59e640 1 -- 172.21.15.84:0/3517762215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe878004000 msgr2=0x7fe88c0c6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:54.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.359+0000 7fe88b59e640 1 --2- 172.21.15.84:0/3517762215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe878004000 0x7fe88c0c6fc0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00d800 tx=0x7fe87c00dcc0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.359+0000 7fe88b59e640 1 -- 172.21.15.84:0/3517762215 shutdown_connections 2023-10-16T10:04:54.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.359+0000 7fe88b59e640 1 --2- 172.21.15.84:0/3517762215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe88c06f700 0x7fe88c06fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.359+0000 7fe88b59e640 1 --2- 172.21.15.84:0/3517762215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe88c0c77e0 0x7fe88c0c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.359+0000 7fe88b59e640 1 --2- 172.21.15.84:0/3517762215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe878004000 0x7fe88c0c6fc0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.359+0000 7fe88b59e640 1 -- 172.21.15.84:0/3517762215 >> 172.21.15.84:0/3517762215 conn(0x7fe88c0ce6a0 msgr2=0x7fe88c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:54.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.359+0000 7fe88b59e640 1 -- 172.21.15.84:0/3517762215 shutdown_connections 2023-10-16T10:04:54.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.359+0000 7fe88b59e640 1 -- 172.21.15.84:0/3517762215 wait complete. 2023-10-16T10:04:54.380 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:04:54.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.591+0000 7f479e23f640 1 Processor -- start 2023-10-16T10:04:54.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.591+0000 7f479e23f640 1 -- start start 2023-10-16T10:04:54.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.592+0000 7f479e23f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f47980cfdd0 0x7f47980b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:54.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.592+0000 7f479e23f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47980b09c0 0x7f479815d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:54.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.593+0000 7f479e23f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47980cf940 con 0x7f479806b8c0 2023-10-16T10:04:54.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.593+0000 7f479e23f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47980ce990 con 0x7f47980cfdd0 2023-10-16T10:04:54.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.593+0000 7f479e23f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479815e290 con 0x7f47980b09c0 2023-10-16T10:04:54.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.593+0000 7f47977fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f479806b8c0 0x7f479806bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48838/0 (socket says 172.21.15.84:48838) 2023-10-16T10:04:54.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.593+0000 7f47977fe640 1 -- 172.21.15.84:0/1528070689 learned_addr learned my addr 172.21.15.84:0/1528070689 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:54.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.593+0000 7f4796ffd640 1 -- 172.21.15.84:0/1528070689 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2092064456 0 0) 0x7f47980cf940 con 0x7f479806b8c0 2023-10-16T10:04:54.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.593+0000 7f4796ffd640 1 -- 172.21.15.84:0/1528070689 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4784003520 con 0x7f479806b8c0 2023-10-16T10:04:54.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.593+0000 7f478ffff640 1 --2- 172.21.15.84:0/1528070689 >> v2:172.21.15.157:3300/0 conn(0x7f47980cfdd0 0x7f47980b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:54.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.593+0000 7f4797fff640 1 --2- 172.21.15.84:0/1528070689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47980b09c0 0x7f479815d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:54.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f4796ffd640 1 -- 172.21.15.84:0/1528070689 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1618445505 0 0) 0x7f4784003520 con 0x7f479806b8c0 2023-10-16T10:04:54.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f4796ffd640 1 -- 172.21.15.84:0/1528070689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47980b09c0 msgr2=0x7f479815d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:54.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f4796ffd640 1 --2- 172.21.15.84:0/1528070689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47980b09c0 0x7f479815d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f4796ffd640 1 -- 172.21.15.84:0/1528070689 >> v2:172.21.15.157:3300/0 conn(0x7f47980cfdd0 msgr2=0x7f47980b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:54.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f4796ffd640 1 --2- 172.21.15.84:0/1528070689 >> v2:172.21.15.157:3300/0 conn(0x7f47980cfdd0 0x7f47980b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f4796ffd640 1 -- 172.21.15.84:0/1528070689 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47980cf010 con 0x7f479806b8c0 2023-10-16T10:04:54.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f4796ffd640 1 -- 172.21.15.84:0/1528070689 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f47880017f0 con 0x7f479806b8c0 2023-10-16T10:04:54.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f4796ffd640 1 -- 172.21.15.84:0/1528070689 >> v1:172.21.15.84:6789/0 conn(0x7f479806b8c0 legacy=0x7f479806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:54.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f4796ffd640 1 --2- 172.21.15.84:0/1528070689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4784003970 0x7f4784003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:54.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f4796ffd640 1 --2- 172.21.15.84:0/1528070689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47980b09c0 0x7f47840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:54.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f4796ffd640 1 --2- 172.21.15.84:0/1528070689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4784004d00 0x7f4784005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:54.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f4796ffd640 1 -- 172.21.15.84:0/1528070689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479815e290 con 0x7f4784003970 2023-10-16T10:04:54.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f4796ffd640 1 -- 172.21.15.84:0/1528070689 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478400ba30 con 0x7f4784004d00 2023-10-16T10:04:54.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f4796ffd640 1 -- 172.21.15.84:0/1528070689 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478400bbf0 con 0x7f47980b09c0 2023-10-16T10:04:54.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f47977fe640 1 --2- 172.21.15.84:0/1528070689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4784003970 0x7f4784003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:54.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f478ffff640 1 --2- 172.21.15.84:0/1528070689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47980b09c0 0x7f47840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:54.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f47977fe640 1 -- 172.21.15.84:0/1528070689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47980b09c0 msgr2=0x7f47840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:54.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f4797fff640 1 --2- 172.21.15.84:0/1528070689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4784004d00 0x7f4784005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:54.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f47977fe640 1 --2- 172.21.15.84:0/1528070689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47980b09c0 0x7f47840047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f47977fe640 1 -- 172.21.15.84:0/1528070689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4784004d00 msgr2=0x7f4784005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:54.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f47977fe640 1 --2- 172.21.15.84:0/1528070689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4784004d00 0x7f4784005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.594+0000 7f47977fe640 1 -- 172.21.15.84:0/1528070689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4784003520 con 0x7f4784003970 2023-10-16T10:04:54.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.596+0000 7f47977fe640 1 --2- 172.21.15.84:0/1528070689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4784003970 0x7f4784003d90 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f478800c0e0 tx=0x7f47840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbac7c6d038b92cc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:54.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.596+0000 7f4796ffd640 1 -- 172.21.15.84:0/1528070689 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47880072f0 con 0x7f4784003970 2023-10-16T10:04:54.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.596+0000 7f4796ffd640 1 -- 172.21.15.84:0/1528070689 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4788007450 con 0x7f4784003970 2023-10-16T10:04:54.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.596+0000 7f4796ffd640 1 -- 172.21.15.84:0/1528070689 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478802c810 con 0x7f4784003970 2023-10-16T10:04:54.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.596+0000 7f479e23f640 1 -- 172.21.15.84:0/1528070689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4784003970 msgr2=0x7f4784003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:54.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.596+0000 7f479e23f640 1 --2- 172.21.15.84:0/1528070689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4784003970 0x7f4784003d90 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f478800c0e0 tx=0x7f47840057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.596+0000 7f4797fff640 1 -- 172.21.15.84:0/1528070689 reap_dead start 2023-10-16T10:04:54.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.596+0000 7f479e23f640 1 -- 172.21.15.84:0/1528070689 shutdown_connections 2023-10-16T10:04:54.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.596+0000 7f479e23f640 1 -- 172.21.15.84:0/1528070689 >> 172.21.15.84:0/1528070689 conn(0x7f4798099ea0 msgr2=0x7f47980d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:54.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.596+0000 7f479e23f640 1 -- 172.21.15.84:0/1528070689 shutdown_connections 2023-10-16T10:04:54.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.596+0000 7f479e23f640 1 -- 172.21.15.84:0/1528070689 wait complete. 2023-10-16T10:04:54.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.598+0000 7f479e23f640 1 Processor -- start 2023-10-16T10:04:54.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.598+0000 7f479e23f640 1 -- start start 2023-10-16T10:04:54.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.599+0000 7f479e23f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4784003970 0x7f479814d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:54.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.599+0000 7f479e23f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47980af900 0x7f47980ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:54.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.599+0000 7f479e23f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47980c6eb0 0x7f479815e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:54.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.599+0000 7f479e23f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479814d550 con 0x7f47980af900 2023-10-16T10:04:54.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.599+0000 7f479e23f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47980ae630 con 0x7f47980c6eb0 2023-10-16T10:04:54.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.599+0000 7f479e23f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47980ae860 con 0x7f4784003970 2023-10-16T10:04:54.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.599+0000 7f478ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47980af900 0x7f47980ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:54.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.599+0000 7f478ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47980af900 0x7f47980ae0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59848/0 (socket says 172.21.15.84:59848) 2023-10-16T10:04:54.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.599+0000 7f478ffff640 1 -- 172.21.15.84:0/2622867013 learned_addr learned my addr 172.21.15.84:0/2622867013 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:54.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.600+0000 7f47977fe640 1 --2- 172.21.15.84:0/2622867013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4784003970 0x7f479814d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:54.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.600+0000 7f4797fff640 1 --2- 172.21.15.84:0/2622867013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47980c6eb0 0x7f479815e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:54.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.600+0000 7f478ffff640 1 -- 172.21.15.84:0/2622867013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4784003970 msgr2=0x7f479814d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:54.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.600+0000 7f478ffff640 1 --2- 172.21.15.84:0/2622867013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4784003970 0x7f479814d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.600+0000 7f478ffff640 1 -- 172.21.15.84:0/2622867013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47980c6eb0 msgr2=0x7f479815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:54.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.600+0000 7f478ffff640 1 --2- 172.21.15.84:0/2622867013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47980c6eb0 0x7f479815e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.600+0000 7f478ffff640 1 -- 172.21.15.84:0/2622867013 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47980aeb40 con 0x7f47980af900 2023-10-16T10:04:54.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.600+0000 7f478ffff640 1 --2- 172.21.15.84:0/2622867013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47980af900 0x7f47980ae0f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f478000e810 tx=0x7f478000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:54.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.600+0000 7f4794ff9640 1 -- 172.21.15.84:0/2622867013 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478000c6d0 con 0x7f47980af900 2023-10-16T10:04:54.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.600+0000 7f4794ff9640 1 -- 172.21.15.84:0/2622867013 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478000c830 con 0x7f47980af900 2023-10-16T10:04:54.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.600+0000 7f479e23f640 1 -- 172.21.15.84:0/2622867013 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47980aee20 con 0x7f47980af900 2023-10-16T10:04:54.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.600+0000 7f4794ff9640 1 -- 172.21.15.84:0/2622867013 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478000cba0 con 0x7f47980af900 2023-10-16T10:04:54.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.601+0000 7f479e23f640 1 -- 172.21.15.84:0/2622867013 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47980af3e0 con 0x7f47980af900 2023-10-16T10:04:54.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.601+0000 7f479e23f640 1 -- 172.21.15.84:0/2622867013 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f479804e160 con 0x7f47980af900 2023-10-16T10:04:54.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.602+0000 7f4794ff9640 1 -- 172.21.15.84:0/2622867013 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 39) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f478000cd00 con 0x7f47980af900 2023-10-16T10:04:54.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.603+0000 7f4794ff9640 1 --2- 172.21.15.84:0/2622867013 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f477806ad10 0x7f477806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:54.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.603+0000 7f4794ff9640 1 -- 172.21.15.84:0/2622867013 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f4780089290 con 0x7f47980af900 2023-10-16T10:04:54.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.604+0000 7f47977fe640 1 --2- 172.21.15.84:0/2622867013 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f477806ad10 0x7f477806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:54.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.604+0000 7f47977fe640 1 --2- 172.21.15.84:0/2622867013 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f477806ad10 0x7f477806d1d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4788034000 tx=0x7f4788007000 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:54.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.607+0000 7f4794ff9640 1 -- 172.21.15.84:0/2622867013 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4780054380 con 0x7f47980af900 2023-10-16T10:04:54.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.820+0000 7f479e23f640 1 -- 172.21.15.84:0/2622867013 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47980ce240 con 0x7f47980af900 2023-10-16T10:04:54.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.821+0000 7f4794ff9640 1 -- 172.21.15.84:0/2622867013 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f47800579a0 con 0x7f47980af900 2023-10-16T10:04:54.823 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:54.823 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5066,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1223610299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1223610299},{"type":"v1","addr":"172.21.15.157:6833","nonce":1223610299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3636443426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3636443426},{"type":"v1","addr":"172.21.15.157:6835","nonce":3636443426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6537,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/3575464874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3575464874},{"type":"v1","addr":"172.21.15.84:6836","nonce":3575464874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:04:32.849497+0000","modified":"2023-10-16T10:04:35.898561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.84:6837/2669751864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2669751864},{"type":"v1","addr":"172.21.15.84:6837","nonce":2669751864}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-16T10:04:54.824 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 35 2023-10-16T10:04:54.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.823+0000 7f479e23f640 1 -- 172.21.15.84:0/2622867013 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f477806ad10 msgr2=0x7f477806d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:54.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.823+0000 7f479e23f640 1 --2- 172.21.15.84:0/2622867013 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f477806ad10 0x7f477806d1d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4788034000 tx=0x7f4788007000 comp rx=0 tx=0).stop 2023-10-16T10:04:54.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.824+0000 7f479e23f640 1 -- 172.21.15.84:0/2622867013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47980af900 msgr2=0x7f47980ae0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:54.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.824+0000 7f479e23f640 1 --2- 172.21.15.84:0/2622867013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47980af900 0x7f47980ae0f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f478000e810 tx=0x7f478000ecd0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.824+0000 7f479e23f640 1 -- 172.21.15.84:0/2622867013 shutdown_connections 2023-10-16T10:04:54.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.824+0000 7f479e23f640 1 --2- 172.21.15.84:0/2622867013 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f477806ad10 0x7f477806d1d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.824+0000 7f479e23f640 1 --2- 172.21.15.84:0/2622867013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47980c6eb0 0x7f479815e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.824+0000 7f479e23f640 1 --2- 172.21.15.84:0/2622867013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47980af900 0x7f47980ae0f0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.824+0000 7f479e23f640 1 --2- 172.21.15.84:0/2622867013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4784003970 0x7f479814d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:54.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.824+0000 7f479e23f640 1 -- 172.21.15.84:0/2622867013 >> 172.21.15.84:0/2622867013 conn(0x7f4798099ea0 msgr2=0x7f479815c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:54.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.824+0000 7f479e23f640 1 -- 172.21.15.84:0/2622867013 shutdown_connections 2023-10-16T10:04:54.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:54.824+0000 7f479e23f640 1 -- 172.21.15.84:0/2622867013 wait complete. 2023-10-16T10:04:54.842 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:04:55.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.048+0000 7f84f7861640 1 Processor -- start 2023-10-16T10:04:55.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.048+0000 7f84f7861640 1 -- start start 2023-10-16T10:04:55.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.050+0000 7f84f7861640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f84f015ae80 0x7f84f015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.050+0000 7f84f7861640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f84f00ce390 0x7f84f0070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.050+0000 7f84f7861640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f015ed60 con 0x7f84f015aaa0 2023-10-16T10:04:55.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.050+0000 7f84f7861640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f006b810 con 0x7f84f00ce390 2023-10-16T10:04:55.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.050+0000 7f84f7861640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f00707e0 con 0x7f84f015ae80 2023-10-16T10:04:55.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.050+0000 7f84f5dfe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f84f00ce390 0x7f84f0070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:55.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.050+0000 7f84f5dfe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f84f00ce390 0x7f84f0070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59862/0 (socket says 172.21.15.84:59862) 2023-10-16T10:04:55.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.050+0000 7f84f5dfe640 1 -- 172.21.15.84:0/4203478449 learned_addr learned my addr 172.21.15.84:0/4203478449 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:55.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.051+0000 7f84f4dfc640 1 --2- 172.21.15.84:0/4203478449 >> v2:172.21.15.157:3300/0 conn(0x7f84f015ae80 0x7f84f015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:55.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.051+0000 7f84f5dfe640 1 -- 172.21.15.84:0/4203478449 >> v2:172.21.15.157:3300/0 conn(0x7f84f015ae80 msgr2=0x7f84f015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:55.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.051+0000 7f84f5dfe640 1 --2- 172.21.15.84:0/4203478449 >> v2:172.21.15.157:3300/0 conn(0x7f84f015ae80 0x7f84f015e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.051+0000 7f84f5dfe640 1 -- 172.21.15.84:0/4203478449 >> v1:172.21.15.157:6789/0 conn(0x7f84f015aaa0 legacy=0x7f84f00cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:55.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.051+0000 7f84f5dfe640 1 -- 172.21.15.84:0/4203478449 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f015ef90 con 0x7f84f00ce390 2023-10-16T10:04:55.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.052+0000 7f84f5dfe640 1 --2- 172.21.15.84:0/4203478449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84f00ce390 0x7f84f0070f50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f84f00c59c0 tx=0x7f84e802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee91e96da0ad011 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:55.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.052+0000 7f84e7fff640 1 -- 172.21.15.84:0/4203478449 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e802ca80 con 0x7f84f00ce390 2023-10-16T10:04:55.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.052+0000 7f84e7fff640 1 -- 172.21.15.84:0/4203478449 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e802cbe0 con 0x7f84f00ce390 2023-10-16T10:04:55.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.052+0000 7f84e7fff640 1 -- 172.21.15.84:0/4203478449 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e80343f0 con 0x7f84f00ce390 2023-10-16T10:04:55.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.053+0000 7f84f7861640 1 -- 172.21.15.84:0/4203478449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84f00ce390 msgr2=0x7f84f0070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:55.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.053+0000 7f84f7861640 1 --2- 172.21.15.84:0/4203478449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84f00ce390 0x7f84f0070f50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f84f00c59c0 tx=0x7f84e802bee0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.053+0000 7f84f7861640 1 -- 172.21.15.84:0/4203478449 shutdown_connections 2023-10-16T10:04:55.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.053+0000 7f84f7861640 1 --2- 172.21.15.84:0/4203478449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84f00ce390 0x7f84f0070f50 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.053+0000 7f84f7861640 1 --2- 172.21.15.84:0/4203478449 >> v2:172.21.15.157:3300/0 conn(0x7f84f015ae80 0x7f84f015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.053+0000 7f84f7861640 1 -- 172.21.15.84:0/4203478449 >> 172.21.15.84:0/4203478449 conn(0x7f84f00cb410 msgr2=0x7f84f00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:55.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.053+0000 7f84f7861640 1 -- 172.21.15.84:0/4203478449 shutdown_connections 2023-10-16T10:04:55.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.053+0000 7f84f7861640 1 -- 172.21.15.84:0/4203478449 wait complete. 2023-10-16T10:04:55.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.055+0000 7f84f7861640 1 Processor -- start 2023-10-16T10:04:55.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.055+0000 7f84f7861640 1 -- start start 2023-10-16T10:04:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.056+0000 7f84f7861640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84f00ce390 0x7f84f00b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.056+0000 7f84f7861640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84f015aaa0 0x7f84f01593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.056+0000 7f84f7861640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84f015ae80 0x7f84f0155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.056+0000 7f84f7861640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f01f5fe0 con 0x7f84f00ce390 2023-10-16T10:04:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.056+0000 7f84f7861640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f01f62c0 con 0x7f84f015aaa0 2023-10-16T10:04:55.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.056+0000 7f84f7861640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f01f65a0 con 0x7f84f015ae80 2023-10-16T10:04:55.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.056+0000 7f84f55fd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84f00ce390 0x7f84f00b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:55.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.056+0000 7f84f55fd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84f00ce390 0x7f84f00b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59876/0 (socket says 172.21.15.84:59876) 2023-10-16T10:04:55.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.056+0000 7f84f55fd640 1 -- 172.21.15.84:0/3873655254 learned_addr learned my addr 172.21.15.84:0/3873655254 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:55.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.056+0000 7f84f4dfc640 1 --2- 172.21.15.84:0/3873655254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84f015aaa0 0x7f84f01593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:55.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.056+0000 7f84f55fd640 1 -- 172.21.15.84:0/3873655254 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84f015ae80 msgr2=0x7f84f0155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:55.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.056+0000 7f84f5dfe640 1 --2- 172.21.15.84:0/3873655254 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84f015ae80 0x7f84f0155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:55.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.056+0000 7f84f55fd640 1 --2- 172.21.15.84:0/3873655254 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84f015ae80 0x7f84f0155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.056+0000 7f84f55fd640 1 -- 172.21.15.84:0/3873655254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84f015aaa0 msgr2=0x7f84f01593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:55.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.056+0000 7f84f55fd640 1 --2- 172.21.15.84:0/3873655254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84f015aaa0 0x7f84f01593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.056+0000 7f84f55fd640 1 -- 172.21.15.84:0/3873655254 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e8009b30 con 0x7f84f00ce390 2023-10-16T10:04:55.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.056+0000 7f84f4dfc640 1 --2- 172.21.15.84:0/3873655254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84f015aaa0 0x7f84f01593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:55.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.057+0000 7f84f55fd640 1 --2- 172.21.15.84:0/3873655254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84f00ce390 0x7f84f00b1720 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00c430 tx=0x7f84ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:55.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.057+0000 7f84e5ffb640 1 -- 172.21.15.84:0/3873655254 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec011790 con 0x7f84f00ce390 2023-10-16T10:04:55.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.057+0000 7f84e5ffb640 1 -- 172.21.15.84:0/3873655254 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ec0118f0 con 0x7f84f00ce390 2023-10-16T10:04:55.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.057+0000 7f84e5ffb640 1 -- 172.21.15.84:0/3873655254 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec011ce0 con 0x7f84f00ce390 2023-10-16T10:04:55.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.057+0000 7f84f7861640 1 -- 172.21.15.84:0/3873655254 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f01f6880 con 0x7f84f00ce390 2023-10-16T10:04:55.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.057+0000 7f84f7861640 1 -- 172.21.15.84:0/3873655254 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f01f5c90 con 0x7f84f00ce390 2023-10-16T10:04:55.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.058+0000 7f84f7861640 1 -- 172.21.15.84:0/3873655254 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f004e160 con 0x7f84f00ce390 2023-10-16T10:04:55.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.059+0000 7f84e5ffb640 1 -- 172.21.15.84:0/3873655254 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 39) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f84ec01c460 con 0x7f84f00ce390 2023-10-16T10:04:55.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.060+0000 7f84e5ffb640 1 --2- 172.21.15.84:0/3873655254 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f84d406acc0 0x7f84d406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.060+0000 7f84e5ffb640 1 -- 172.21.15.84:0/3873655254 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f84ec089510 con 0x7f84f00ce390 2023-10-16T10:04:55.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.060+0000 7f84f4dfc640 1 --2- 172.21.15.84:0/3873655254 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f84d406acc0 0x7f84d406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:55.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.061+0000 7f84f4dfc640 1 --2- 172.21.15.84:0/3873655254 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f84d406acc0 0x7f84d406d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f84e0006fd0 tx=0x7f84e0008040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:55.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.064+0000 7f84e5ffb640 1 -- 172.21.15.84:0/3873655254 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f84ec054580 con 0x7f84f00ce390 2023-10-16T10:04:55.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.255+0000 7f84f7861640 1 -- 172.21.15.84:0/3873655254 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84f0157c90 con 0x7f84f00ce390 2023-10-16T10:04:55.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.257+0000 7f84e5ffb640 1 -- 172.21.15.84:0/3873655254 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+17543 (secure 0 0 0) 0x7f84ec057ba0 con 0x7f84f00ce390 2023-10-16T10:04:55.259 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:55.259 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":84,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:04:53.436179+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:04:30.728624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-16T10:04:31.866029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6832/2103121481":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/3340998984":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/2855693098":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/1664791074":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/2565413151":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/890477594":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/3177129458":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/1145018607":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/1499920052":"2023-10-17T10:04:47.124608+0000","172.21.15.84:6833/2103121481":"2023-10-17T10:04:53.436086+0000","172.21.15.84:6833/2091947430":"2023-10-17T10:04:47.124608+0000","172.21.15.84:6832/2091947430":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/1204215627":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/2509405053":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/3991569032":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/3599228496":"2023-10-17T10:04:47.124608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:04:55.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.260+0000 7f84f7861640 1 -- 172.21.15.84:0/3873655254 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f84d406acc0 msgr2=0x7f84d406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:55.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.260+0000 7f84f7861640 1 --2- 172.21.15.84:0/3873655254 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f84d406acc0 0x7f84d406d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f84e0006fd0 tx=0x7f84e0008040 comp rx=0 tx=0).stop 2023-10-16T10:04:55.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.260+0000 7f84f7861640 1 -- 172.21.15.84:0/3873655254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84f00ce390 msgr2=0x7f84f00b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:55.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.260+0000 7f84f7861640 1 --2- 172.21.15.84:0/3873655254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84f00ce390 0x7f84f00b1720 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00c430 tx=0x7f84ec00c8f0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.260+0000 7f84f7861640 1 -- 172.21.15.84:0/3873655254 shutdown_connections 2023-10-16T10:04:55.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.260+0000 7f84f7861640 1 --2- 172.21.15.84:0/3873655254 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f84d406acc0 0x7f84d406d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.260+0000 7f84f7861640 1 --2- 172.21.15.84:0/3873655254 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84f015ae80 0x7f84f0155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.260+0000 7f84f7861640 1 --2- 172.21.15.84:0/3873655254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84f015aaa0 0x7f84f01593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.260+0000 7f84f7861640 1 --2- 172.21.15.84:0/3873655254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84f00ce390 0x7f84f00b1720 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.260+0000 7f84f7861640 1 -- 172.21.15.84:0/3873655254 >> 172.21.15.84:0/3873655254 conn(0x7f84f00cb410 msgr2=0x7f84f006fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:55.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.260+0000 7f84f7861640 1 -- 172.21.15.84:0/3873655254 shutdown_connections 2023-10-16T10:04:55.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.260+0000 7f84f7861640 1 -- 172.21.15.84:0/3873655254 wait complete. 2023-10-16T10:04:55.280 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T10:04:55.280 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:04:55.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.480+0000 7fbc5f2e7640 1 Processor -- start 2023-10-16T10:04:55.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.481+0000 7fbc5f2e7640 1 -- start start 2023-10-16T10:04:55.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.482+0000 7fbc5f2e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbc5815ae80 0x7fbc5815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.482+0000 7fbc5f2e7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc5815ed60 con 0x7fbc580ce390 2023-10-16T10:04:55.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.482+0000 7fbc5f2e7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc580707e0 con 0x7fbc5815aaa0 2023-10-16T10:04:55.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.482+0000 7fbc5f2e7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc5806b810 con 0x7fbc5815ae80 2023-10-16T10:04:55.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.482+0000 7fbc5d884640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbc580ce390 0x7fbc58070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48844/0 (socket says 172.21.15.84:48844) 2023-10-16T10:04:55.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.482+0000 7fbc5d884640 1 -- 172.21.15.84:0/1712485850 learned_addr learned my addr 172.21.15.84:0/1712485850 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:55.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.483+0000 7fbc5c882640 1 --2- 172.21.15.84:0/1712485850 >> v2:172.21.15.157:3300/0 conn(0x7fbc5815ae80 0x7fbc5815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:55.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.483+0000 7fbc4bfff640 1 -- 172.21.15.84:0/1712485850 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1479663022 0 0) 0x7fbc5815ed60 con 0x7fbc580ce390 2023-10-16T10:04:55.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.483+0000 7fbc4bfff640 1 -- 172.21.15.84:0/1712485850 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc44003520 con 0x7fbc580ce390 2023-10-16T10:04:55.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.483+0000 7fbc4bfff640 1 -- 172.21.15.84:0/1712485850 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1224678352 0 0) 0x7fbc44003520 con 0x7fbc580ce390 2023-10-16T10:04:55.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.483+0000 7fbc4bfff640 1 -- 172.21.15.84:0/1712485850 >> v2:172.21.15.157:3300/0 conn(0x7fbc5815ae80 msgr2=0x7fbc5815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:55.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.483+0000 7fbc4bfff640 1 --2- 172.21.15.84:0/1712485850 >> v2:172.21.15.157:3300/0 conn(0x7fbc5815ae80 0x7fbc5815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.483+0000 7fbc4bfff640 1 -- 172.21.15.84:0/1712485850 >> v1:172.21.15.157:6789/0 conn(0x7fbc5815aaa0 legacy=0x7fbc580cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:55.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.483+0000 7fbc4bfff640 1 -- 172.21.15.84:0/1712485850 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5815ef90 con 0x7fbc580ce390 2023-10-16T10:04:55.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.483+0000 7fbc4bfff640 1 -- 172.21.15.84:0/1712485850 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fbc500019f0 con 0x7fbc580ce390 2023-10-16T10:04:55.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.483+0000 7fbc4bfff640 1 -- 172.21.15.84:0/1712485850 >> v1:172.21.15.84:6789/0 conn(0x7fbc580ce390 legacy=0x7fbc58070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:55.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.483+0000 7fbc4bfff640 1 --2- 172.21.15.84:0/1712485850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc44003970 0x7fbc44003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.483+0000 7fbc4bfff640 1 --2- 172.21.15.84:0/1712485850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc440047c0 0x7fbc44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.484+0000 7fbc4bfff640 1 --2- 172.21.15.84:0/1712485850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc44005180 0x7fbc4400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.484+0000 7fbc4bfff640 1 -- 172.21.15.84:0/1712485850 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5806b810 con 0x7fbc44005180 2023-10-16T10:04:55.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.484+0000 7fbc4bfff640 1 -- 172.21.15.84:0/1712485850 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4400c020 con 0x7fbc44003970 2023-10-16T10:04:55.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.484+0000 7fbc4bfff640 1 -- 172.21.15.84:0/1712485850 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4400c1e0 con 0x7fbc440047c0 2023-10-16T10:04:55.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.484+0000 7fbc5d884640 1 --2- 172.21.15.84:0/1712485850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc44005180 0x7fbc4400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:55.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.484+0000 7fbc5d083640 1 --2- 172.21.15.84:0/1712485850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc44003970 0x7fbc44003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:55.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.484+0000 7fbc5d884640 1 -- 172.21.15.84:0/1712485850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc440047c0 msgr2=0x7fbc44004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:55.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.484+0000 7fbc5c882640 1 --2- 172.21.15.84:0/1712485850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc440047c0 0x7fbc44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:55.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.484+0000 7fbc5d884640 1 --2- 172.21.15.84:0/1712485850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc440047c0 0x7fbc44004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.484+0000 7fbc5d884640 1 -- 172.21.15.84:0/1712485850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc44003970 msgr2=0x7fbc44003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:55.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.484+0000 7fbc5d884640 1 --2- 172.21.15.84:0/1712485850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc44003970 0x7fbc44003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.484+0000 7fbc5d884640 1 -- 172.21.15.84:0/1712485850 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc44003520 con 0x7fbc44005180 2023-10-16T10:04:55.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.485+0000 7fbc5d884640 1 -- 172.21.15.84:0/1712485850 reap_dead start 2023-10-16T10:04:55.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.486+0000 7fbc5d884640 1 --2- 172.21.15.84:0/1712485850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc44005180 0x7fbc4400b980 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000a820 tx=0x7fbc440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70df2e3ad52d4a14 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:55.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.486+0000 7fbc4bfff640 1 -- 172.21.15.84:0/1712485850 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc500072d0 con 0x7fbc44005180 2023-10-16T10:04:55.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.486+0000 7fbc4bfff640 1 -- 172.21.15.84:0/1712485850 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5002f070 con 0x7fbc44005180 2023-10-16T10:04:55.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.486+0000 7fbc4bfff640 1 -- 172.21.15.84:0/1712485850 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50007690 con 0x7fbc44005180 2023-10-16T10:04:55.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.486+0000 7fbc5f2e7640 1 -- 172.21.15.84:0/1712485850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc44005180 msgr2=0x7fbc4400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:55.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.486+0000 7fbc5f2e7640 1 --2- 172.21.15.84:0/1712485850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc44005180 0x7fbc4400b980 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000a820 tx=0x7fbc440057f0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.487+0000 7fbc5f2e7640 1 -- 172.21.15.84:0/1712485850 shutdown_connections 2023-10-16T10:04:55.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.487+0000 7fbc5f2e7640 1 --2- 172.21.15.84:0/1712485850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc44005180 0x7fbc4400b980 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.487+0000 7fbc5f2e7640 1 -- 172.21.15.84:0/1712485850 >> 172.21.15.84:0/1712485850 conn(0x7fbc580cb410 msgr2=0x7fbc580cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:55.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.487+0000 7fbc5f2e7640 1 -- 172.21.15.84:0/1712485850 shutdown_connections 2023-10-16T10:04:55.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.487+0000 7fbc5f2e7640 1 -- 172.21.15.84:0/1712485850 wait complete. 2023-10-16T10:04:55.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.488+0000 7fbc5f2e7640 1 Processor -- start 2023-10-16T10:04:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.489+0000 7fbc5f2e7640 1 -- start start 2023-10-16T10:04:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.489+0000 7fbc5f2e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc44005180 0x7fbc580c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.489+0000 7fbc5f2e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc58144e40 0x7fbc580c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.489+0000 7fbc5f2e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc580c4750 0x7fbc58146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.489+0000 7fbc5f2e7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc580ae230 con 0x7fbc580c4750 2023-10-16T10:04:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.489+0000 7fbc5f2e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc580c86f0 con 0x7fbc58144e40 2023-10-16T10:04:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.489+0000 7fbc5f2e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc580ae480 con 0x7fbc44005180 2023-10-16T10:04:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.490+0000 7fbc5d884640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc580c4750 0x7fbc58146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.490+0000 7fbc5d884640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc580c4750 0x7fbc58146eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59894/0 (socket says 172.21.15.84:59894) 2023-10-16T10:04:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.490+0000 7fbc5d884640 1 -- 172.21.15.84:0/4005509761 learned_addr learned my addr 172.21.15.84:0/4005509761 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:55.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.490+0000 7fbc5d083640 1 --2- 172.21.15.84:0/4005509761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc44005180 0x7fbc580c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:55.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.490+0000 7fbc5d884640 1 -- 172.21.15.84:0/4005509761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc44005180 msgr2=0x7fbc580c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:55.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.490+0000 7fbc5d884640 1 --2- 172.21.15.84:0/4005509761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc44005180 0x7fbc580c8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.490+0000 7fbc5d884640 1 -- 172.21.15.84:0/4005509761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc58144e40 msgr2=0x7fbc580c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:55.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.490+0000 7fbc5d884640 1 --2- 172.21.15.84:0/4005509761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc58144e40 0x7fbc580c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.490+0000 7fbc5d884640 1 -- 172.21.15.84:0/4005509761 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc580ae760 con 0x7fbc580c4750 2023-10-16T10:04:55.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.490+0000 7fbc5d884640 1 --2- 172.21.15.84:0/4005509761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc580c4750 0x7fbc58146eb0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fbc50000f80 tx=0x7fbc5815f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:55.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.491+0000 7fbc49ffb640 1 -- 172.21.15.84:0/4005509761 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50044070 con 0x7fbc580c4750 2023-10-16T10:04:55.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.491+0000 7fbc49ffb640 1 -- 172.21.15.84:0/4005509761 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc50001250 con 0x7fbc580c4750 2023-10-16T10:04:55.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.491+0000 7fbc5f2e7640 1 -- 172.21.15.84:0/4005509761 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc580aea40 con 0x7fbc580c4750 2023-10-16T10:04:55.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.491+0000 7fbc49ffb640 1 -- 172.21.15.84:0/4005509761 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc500074e0 con 0x7fbc580c4750 2023-10-16T10:04:55.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.491+0000 7fbc5f2e7640 1 -- 172.21.15.84:0/4005509761 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc580c4bd0 con 0x7fbc580c4750 2023-10-16T10:04:55.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.492+0000 7fbc5f2e7640 1 -- 172.21.15.84:0/4005509761 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5804e160 con 0x7fbc580c4750 2023-10-16T10:04:55.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.493+0000 7fbc49ffb640 1 -- 172.21.15.84:0/4005509761 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 39) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fbc50031030 con 0x7fbc580c4750 2023-10-16T10:04:55.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.494+0000 7fbc49ffb640 1 --2- 172.21.15.84:0/4005509761 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fbc3c06acc0 0x7fbc3c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.494+0000 7fbc49ffb640 1 -- 172.21.15.84:0/4005509761 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6397+0+0 (secure 0 0 0) 0x7fbc500ab130 con 0x7fbc580c4750 2023-10-16T10:04:55.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.494+0000 7fbc5d083640 1 --2- 172.21.15.84:0/4005509761 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fbc3c06acc0 0x7fbc3c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:55.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.495+0000 7fbc5d083640 1 --2- 172.21.15.84:0/4005509761 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fbc3c06acc0 0x7fbc3c06d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbc54007fd0 tx=0x7fbc54009040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:55.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.498+0000 7fbc49ffb640 1 -- 172.21.15.84:0/4005509761 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbc500761a0 con 0x7fbc580c4750 2023-10-16T10:04:55.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.717+0000 7fbc5f2e7640 1 -- 172.21.15.84:0/4005509761 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc5804e160 con 0x7fbc580c4750 2023-10-16T10:04:55.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.717+0000 7fbc49ffb640 1 -- 172.21.15.84:0/4005509761 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fbc500797c0 con 0x7fbc580c4750 2023-10-16T10:04:55.719 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:55.720 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5066,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1223610299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1223610299},{"type":"v1","addr":"172.21.15.157:6833","nonce":1223610299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3636443426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3636443426},{"type":"v1","addr":"172.21.15.157:6835","nonce":3636443426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6537,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/3575464874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3575464874},{"type":"v1","addr":"172.21.15.84:6836","nonce":3575464874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:04:32.849497+0000","modified":"2023-10-16T10:04:35.898561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.84:6837/2669751864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2669751864},{"type":"v1","addr":"172.21.15.84:6837","nonce":2669751864}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-16T10:04:55.720 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 35 2023-10-16T10:04:55.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.720+0000 7fbc5f2e7640 1 -- 172.21.15.84:0/4005509761 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fbc3c06acc0 msgr2=0x7fbc3c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:55.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.720+0000 7fbc5f2e7640 1 --2- 172.21.15.84:0/4005509761 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fbc3c06acc0 0x7fbc3c06d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbc54007fd0 tx=0x7fbc54009040 comp rx=0 tx=0).stop 2023-10-16T10:04:55.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.720+0000 7fbc5f2e7640 1 -- 172.21.15.84:0/4005509761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc580c4750 msgr2=0x7fbc58146eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:55.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.720+0000 7fbc5f2e7640 1 --2- 172.21.15.84:0/4005509761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc580c4750 0x7fbc58146eb0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fbc50000f80 tx=0x7fbc5815f830 comp rx=0 tx=0).stop 2023-10-16T10:04:55.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.720+0000 7fbc5f2e7640 1 -- 172.21.15.84:0/4005509761 shutdown_connections 2023-10-16T10:04:55.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.720+0000 7fbc5f2e7640 1 --2- 172.21.15.84:0/4005509761 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fbc3c06acc0 0x7fbc3c06d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.720+0000 7fbc5f2e7640 1 --2- 172.21.15.84:0/4005509761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc580c4750 0x7fbc58146eb0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.720+0000 7fbc5f2e7640 1 --2- 172.21.15.84:0/4005509761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc58144e40 0x7fbc580c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.720+0000 7fbc5f2e7640 1 --2- 172.21.15.84:0/4005509761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc44005180 0x7fbc580c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.720+0000 7fbc5f2e7640 1 -- 172.21.15.84:0/4005509761 >> 172.21.15.84:0/4005509761 conn(0x7fbc580cb410 msgr2=0x7fbc5806cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:55.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.720+0000 7fbc5f2e7640 1 -- 172.21.15.84:0/4005509761 shutdown_connections 2023-10-16T10:04:55.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.720+0000 7fbc5f2e7640 1 -- 172.21.15.84:0/4005509761 wait complete. 2023-10-16T10:04:55.739 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:04:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.941+0000 7f935394e640 1 Processor -- start 2023-10-16T10:04:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.941+0000 7f935394e640 1 -- start start 2023-10-16T10:04:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.943+0000 7f935394e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f934c15aaa0 0x7f934c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.943+0000 7f935394e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f934c15ae80 0x7f934c0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.943+0000 7f935394e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f934c0ccaa0 0x7f934c070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.943+0000 7f935394e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f934c06b810 con 0x7f934c15ae80 2023-10-16T10:04:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.943+0000 7f935394e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f934c15ed60 con 0x7f934c0ccaa0 2023-10-16T10:04:55.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.943+0000 7f935394e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934c0707e0 con 0x7f934c15aaa0 2023-10-16T10:04:55.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.943+0000 7f9350ee9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f934c15ae80 0x7f934c0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:55.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.943+0000 7f9350ee9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f934c15ae80 0x7f934c0cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50168/0 (socket says 172.21.15.84:50168) 2023-10-16T10:04:55.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.943+0000 7f9350ee9640 1 -- 172.21.15.84:0/3979281740 learned_addr learned my addr 172.21.15.84:0/3979281740 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:55.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.944+0000 7f93516ea640 1 --2- 172.21.15.84:0/3979281740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f934c15aaa0 0x7f934c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:55.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.944+0000 7f9350ee9640 1 -- 172.21.15.84:0/3979281740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f934c15aaa0 msgr2=0x7f934c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:55.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.944+0000 7f9350ee9640 1 --2- 172.21.15.84:0/3979281740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f934c15aaa0 0x7f934c15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.944+0000 7f9350ee9640 1 -- 172.21.15.84:0/3979281740 >> v2:172.21.15.157:3300/0 conn(0x7f934c0ccaa0 msgr2=0x7f934c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:55.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.944+0000 7f9350ee9640 1 --2- 172.21.15.84:0/3979281740 >> v2:172.21.15.157:3300/0 conn(0x7f934c0ccaa0 0x7f934c070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.944+0000 7f9350ee9640 1 -- 172.21.15.84:0/3979281740 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c15ef90 con 0x7f934c15ae80 2023-10-16T10:04:55.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.945+0000 7f9350ee9640 1 --2- 172.21.15.84:0/3979281740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f934c15ae80 0x7f934c0cc560 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f934c0ae3c0 tx=0x7f933802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1fca34e3aa45f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:55.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.945+0000 7f934bfff640 1 -- 172.21.15.84:0/3979281740 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338031870 con 0x7f934c15ae80 2023-10-16T10:04:55.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.945+0000 7f934bfff640 1 -- 172.21.15.84:0/3979281740 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93380319d0 con 0x7f934c15ae80 2023-10-16T10:04:55.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.945+0000 7f934bfff640 1 -- 172.21.15.84:0/3979281740 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338031d60 con 0x7f934c15ae80 2023-10-16T10:04:55.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.946+0000 7f935394e640 1 -- 172.21.15.84:0/3979281740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f934c15ae80 msgr2=0x7f934c0cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:55.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.946+0000 7f935394e640 1 --2- 172.21.15.84:0/3979281740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f934c15ae80 0x7f934c0cc560 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f934c0ae3c0 tx=0x7f933802b910 comp rx=0 tx=0).stop 2023-10-16T10:04:55.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.946+0000 7f935394e640 1 -- 172.21.15.84:0/3979281740 shutdown_connections 2023-10-16T10:04:55.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.946+0000 7f935394e640 1 --2- 172.21.15.84:0/3979281740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f934c15ae80 0x7f934c0cc560 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.946+0000 7f935394e640 1 --2- 172.21.15.84:0/3979281740 >> v2:172.21.15.157:3300/0 conn(0x7f934c0ccaa0 0x7f934c070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.946+0000 7f935394e640 1 --2- 172.21.15.84:0/3979281740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f934c15aaa0 0x7f934c15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.946+0000 7f935394e640 1 -- 172.21.15.84:0/3979281740 >> 172.21.15.84:0/3979281740 conn(0x7f934c0cb410 msgr2=0x7f934c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:55.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.946+0000 7f935394e640 1 -- 172.21.15.84:0/3979281740 shutdown_connections 2023-10-16T10:04:55.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.946+0000 7f935394e640 1 -- 172.21.15.84:0/3979281740 wait complete. 2023-10-16T10:04:55.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.948+0000 7f935394e640 1 Processor -- start 2023-10-16T10:04:55.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.948+0000 7f935394e640 1 -- start start 2023-10-16T10:04:55.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.949+0000 7f935394e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f934c0ccaa0 0x7f934c14cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.949+0000 7f935394e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f934c15aaa0 0x7f934c0c33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.949+0000 7f935394e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f934c15ae80 0x7f934c0b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.949+0000 7f935394e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c14d2c0 con 0x7f934c0ccaa0 2023-10-16T10:04:55.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.949+0000 7f935394e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c0c4da0 con 0x7f934c15aaa0 2023-10-16T10:04:55.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.949+0000 7f935394e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934c0c2f20 con 0x7f934c15ae80 2023-10-16T10:04:55.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.949+0000 7f93516ea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f934c0ccaa0 0x7f934c14cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:55.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.949+0000 7f93516ea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f934c0ccaa0 0x7f934c14cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50174/0 (socket says 172.21.15.84:50174) 2023-10-16T10:04:55.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.949+0000 7f93516ea640 1 -- 172.21.15.84:0/4226838778 learned_addr learned my addr 172.21.15.84:0/4226838778 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:55.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.950+0000 7f93516ea640 1 -- 172.21.15.84:0/4226838778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f934c15ae80 msgr2=0x7f934c0b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:55.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.950+0000 7f93516ea640 1 --2- 172.21.15.84:0/4226838778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f934c15ae80 0x7f934c0b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.950+0000 7f93516ea640 1 -- 172.21.15.84:0/4226838778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f934c15aaa0 msgr2=0x7f934c0c33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:55.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.950+0000 7f93516ea640 1 --2- 172.21.15.84:0/4226838778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f934c15aaa0 0x7f934c0c33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:55.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.950+0000 7f93516ea640 1 -- 172.21.15.84:0/4226838778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93380094d0 con 0x7f934c0ccaa0 2023-10-16T10:04:55.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.950+0000 7f93516ea640 1 --2- 172.21.15.84:0/4226838778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f934c0ccaa0 0x7f934c14cd80 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f9340007e40 tx=0x7f934000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:55.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.950+0000 7f9349ffb640 1 -- 172.21.15.84:0/4226838778 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340014620 con 0x7f934c0ccaa0 2023-10-16T10:04:55.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.950+0000 7f9349ffb640 1 -- 172.21.15.84:0/4226838778 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934000ee90 con 0x7f934c0ccaa0 2023-10-16T10:04:55.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.950+0000 7f935394e640 1 -- 172.21.15.84:0/4226838778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934c0c38f0 con 0x7f934c0ccaa0 2023-10-16T10:04:55.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.950+0000 7f9349ffb640 1 -- 172.21.15.84:0/4226838778 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93400149c0 con 0x7f934c0ccaa0 2023-10-16T10:04:55.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.950+0000 7f935394e640 1 -- 172.21.15.84:0/4226838778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934c0c28d0 con 0x7f934c0ccaa0 2023-10-16T10:04:55.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.951+0000 7f935394e640 1 -- 172.21.15.84:0/4226838778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934c04e160 con 0x7f934c0ccaa0 2023-10-16T10:04:55.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.952+0000 7f9349ffb640 1 -- 172.21.15.84:0/4226838778 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 39) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f9340014b20 con 0x7f934c0ccaa0 2023-10-16T10:04:55.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.953+0000 7f9349ffb640 1 --2- 172.21.15.84:0/4226838778 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f933006acc0 0x7f933006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:55.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.953+0000 7f9349ffb640 1 -- 172.21.15.84:0/4226838778 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f9340089eb0 con 0x7f934c0ccaa0 2023-10-16T10:04:55.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.953+0000 7f9350ee9640 1 --2- 172.21.15.84:0/4226838778 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f933006acc0 0x7f933006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:55.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.954+0000 7f9350ee9640 1 --2- 172.21.15.84:0/4226838778 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f933006acc0 0x7f933006d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9338036000 tx=0x7f9338030000 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:55.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:55.957+0000 7f9349ffb640 1 -- 172.21.15.84:0/4226838778 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9340054ff0 con 0x7f934c0ccaa0 2023-10-16T10:04:56.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.185+0000 7f935394e640 1 -- 172.21.15.84:0/4226838778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f934c15e350 con 0x7f934c0ccaa0 2023-10-16T10:04:56.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.187+0000 7f9349ffb640 1 -- 172.21.15.84:0/4226838778 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f9340058610 con 0x7f934c0ccaa0 2023-10-16T10:04:56.188 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:56.189 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5066,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1223610299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1223610299},{"type":"v1","addr":"172.21.15.157:6833","nonce":1223610299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3636443426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3636443426},{"type":"v1","addr":"172.21.15.157:6835","nonce":3636443426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6537,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/3575464874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3575464874},{"type":"v1","addr":"172.21.15.84:6836","nonce":3575464874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:04:32.849497+0000","modified":"2023-10-16T10:04:35.898561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.84:6837/2669751864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2669751864},{"type":"v1","addr":"172.21.15.84:6837","nonce":2669751864}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-16T10:04:56.189 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 35 2023-10-16T10:04:56.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.189+0000 7f935394e640 1 -- 172.21.15.84:0/4226838778 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f933006acc0 msgr2=0x7f933006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:56.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.189+0000 7f935394e640 1 --2- 172.21.15.84:0/4226838778 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f933006acc0 0x7f933006d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9338036000 tx=0x7f9338030000 comp rx=0 tx=0).stop 2023-10-16T10:04:56.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.189+0000 7f935394e640 1 -- 172.21.15.84:0/4226838778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f934c0ccaa0 msgr2=0x7f934c14cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:56.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.189+0000 7f935394e640 1 --2- 172.21.15.84:0/4226838778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f934c0ccaa0 0x7f934c14cd80 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f9340007e40 tx=0x7f934000e4b0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.190+0000 7f935394e640 1 -- 172.21.15.84:0/4226838778 shutdown_connections 2023-10-16T10:04:56.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.190+0000 7f935394e640 1 --2- 172.21.15.84:0/4226838778 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f933006acc0 0x7f933006d180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.190+0000 7f935394e640 1 --2- 172.21.15.84:0/4226838778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f934c15ae80 0x7f934c0b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.190+0000 7f935394e640 1 --2- 172.21.15.84:0/4226838778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f934c15aaa0 0x7f934c0c33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.190+0000 7f935394e640 1 --2- 172.21.15.84:0/4226838778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f934c0ccaa0 0x7f934c14cd80 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.190+0000 7f935394e640 1 -- 172.21.15.84:0/4226838778 >> 172.21.15.84:0/4226838778 conn(0x7f934c0cb410 msgr2=0x7f934c0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:56.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.190+0000 7f935394e640 1 -- 172.21.15.84:0/4226838778 shutdown_connections 2023-10-16T10:04:56.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.190+0000 7f935394e640 1 -- 172.21.15.84:0/4226838778 wait complete. 2023-10-16T10:04:56.210 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:04:56.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.414+0000 7f4849613640 1 Processor -- start 2023-10-16T10:04:56.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.414+0000 7f4849613640 1 -- start start 2023-10-16T10:04:56.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.415+0000 7f4849613640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4844070b60 0x7f484415a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:56.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.415+0000 7f4849613640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f484406c100 0x7f484406c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:56.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.415+0000 7f4849613640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f484415aef0 con 0x7f4844159950 2023-10-16T10:04:56.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.415+0000 7f4849613640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484415e490 con 0x7f4844070b60 2023-10-16T10:04:56.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.415+0000 7f4849613640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484415d910 con 0x7f484406c100 2023-10-16T10:04:56.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.416+0000 7f483bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4844070b60 0x7f484415a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:56.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.416+0000 7f483bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4844070b60 0x7f484415a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50190/0 (socket says 172.21.15.84:50190) 2023-10-16T10:04:56.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.416+0000 7f483bfff640 1 -- 172.21.15.84:0/2200639097 learned_addr learned my addr 172.21.15.84:0/2200639097 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:56.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.416+0000 7f4848e12640 1 --2- 172.21.15.84:0/2200639097 >> v2:172.21.15.157:3300/0 conn(0x7f484406c100 0x7f484406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:56.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.416+0000 7f483bfff640 1 -- 172.21.15.84:0/2200639097 >> v2:172.21.15.157:3300/0 conn(0x7f484406c100 msgr2=0x7f484406c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:56.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.416+0000 7f483bfff640 1 --2- 172.21.15.84:0/2200639097 >> v2:172.21.15.157:3300/0 conn(0x7f484406c100 0x7f484406c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.416+0000 7f483bfff640 1 -- 172.21.15.84:0/2200639097 >> v1:172.21.15.157:6789/0 conn(0x7f4844159950 legacy=0x7f4844070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:56.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.416+0000 7f483bfff640 1 -- 172.21.15.84:0/2200639097 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484415dbf0 con 0x7f4844070b60 2023-10-16T10:04:56.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.417+0000 7f483bfff640 1 --2- 172.21.15.84:0/2200639097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4844070b60 0x7f484415a9b0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f48440ade70 tx=0x7f482c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6c0335dcdee499 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:56.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.418+0000 7f48437fe640 1 -- 172.21.15.84:0/2200639097 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c0317e0 con 0x7f4844070b60 2023-10-16T10:04:56.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.418+0000 7f48437fe640 1 -- 172.21.15.84:0/2200639097 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c031940 con 0x7f4844070b60 2023-10-16T10:04:56.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.418+0000 7f48437fe640 1 -- 172.21.15.84:0/2200639097 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c031cd0 con 0x7f4844070b60 2023-10-16T10:04:56.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.418+0000 7f4849613640 1 -- 172.21.15.84:0/2200639097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4844070b60 msgr2=0x7f484415a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:56.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.418+0000 7f4849613640 1 --2- 172.21.15.84:0/2200639097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4844070b60 0x7f484415a9b0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f48440ade70 tx=0x7f482c02b880 comp rx=0 tx=0).stop 2023-10-16T10:04:56.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.418+0000 7f4849613640 1 -- 172.21.15.84:0/2200639097 shutdown_connections 2023-10-16T10:04:56.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.418+0000 7f4849613640 1 --2- 172.21.15.84:0/2200639097 >> v2:172.21.15.157:3300/0 conn(0x7f484406c100 0x7f484406c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.418+0000 7f4849613640 1 --2- 172.21.15.84:0/2200639097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4844070b60 0x7f484415a9b0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.418+0000 7f4849613640 1 -- 172.21.15.84:0/2200639097 >> 172.21.15.84:0/2200639097 conn(0x7f48440ce6a0 msgr2=0x7f48440ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:56.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.419+0000 7f4849613640 1 -- 172.21.15.84:0/2200639097 shutdown_connections 2023-10-16T10:04:56.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.419+0000 7f4849613640 1 -- 172.21.15.84:0/2200639097 wait complete. 2023-10-16T10:04:56.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.420+0000 7f4849613640 1 Processor -- start 2023-10-16T10:04:56.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.420+0000 7f4849613640 1 -- start start 2023-10-16T10:04:56.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.421+0000 7f4849613640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f484406c100 0x7f484415c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:56.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.421+0000 7f4849613640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4844070b60 0x7f48441521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:56.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.421+0000 7f4849613640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4844159950 0x7f4844149d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:56.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.421+0000 7f4849613640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484415c9b0 con 0x7f484406c100 2023-10-16T10:04:56.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.421+0000 7f4849613640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48441526f0 con 0x7f4844070b60 2023-10-16T10:04:56.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.421+0000 7f4849613640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f484414c0f0 con 0x7f4844159950 2023-10-16T10:04:56.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.421+0000 7f4843fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f484406c100 0x7f484415c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:56.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.422+0000 7f4843fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f484406c100 0x7f484415c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50200/0 (socket says 172.21.15.84:50200) 2023-10-16T10:04:56.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.422+0000 7f4843fff640 1 -- 172.21.15.84:0/837604633 learned_addr learned my addr 172.21.15.84:0/837604633 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:56.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.422+0000 7f483bfff640 1 --2- 172.21.15.84:0/837604633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4844070b60 0x7f48441521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:56.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.422+0000 7f4848e12640 1 --2- 172.21.15.84:0/837604633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4844159950 0x7f4844149d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:56.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.422+0000 7f4843fff640 1 -- 172.21.15.84:0/837604633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4844159950 msgr2=0x7f4844149d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:56.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.422+0000 7f4843fff640 1 --2- 172.21.15.84:0/837604633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4844159950 0x7f4844149d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.422+0000 7f4843fff640 1 -- 172.21.15.84:0/837604633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4844070b60 msgr2=0x7f48441521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:56.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.422+0000 7f4843fff640 1 --2- 172.21.15.84:0/837604633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4844070b60 0x7f48441521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.422+0000 7f4843fff640 1 -- 172.21.15.84:0/837604633 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c0094d0 con 0x7f484406c100 2023-10-16T10:04:56.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.422+0000 7f4843fff640 1 --2- 172.21.15.84:0/837604633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f484406c100 0x7f484415c470 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f483400b3e0 tx=0x7f483400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:56.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.423+0000 7f48417fa640 1 -- 172.21.15.84:0/837604633 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834011830 con 0x7f484406c100 2023-10-16T10:04:56.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.423+0000 7f48417fa640 1 -- 172.21.15.84:0/837604633 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48340119b0 con 0x7f484406c100 2023-10-16T10:04:56.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.423+0000 7f48417fa640 1 -- 172.21.15.84:0/837604633 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834011d60 con 0x7f484406c100 2023-10-16T10:04:56.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.423+0000 7f4849613640 1 -- 172.21.15.84:0/837604633 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f484414c3d0 con 0x7f484406c100 2023-10-16T10:04:56.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.423+0000 7f4849613640 1 -- 172.21.15.84:0/837604633 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f484414a2c0 con 0x7f484406c100 2023-10-16T10:04:56.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.424+0000 7f4849613640 1 -- 172.21.15.84:0/837604633 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484404e9b0 con 0x7f484406c100 2023-10-16T10:04:56.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.425+0000 7f48417fa640 1 -- 172.21.15.84:0/837604633 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 40) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f483401a400 con 0x7f484406c100 2023-10-16T10:04:56.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.427+0000 7f48417fa640 1 --2- 172.21.15.84:0/837604633 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f482406af00 0x7f482406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:56.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.427+0000 7f48417fa640 1 -- 172.21.15.84:0/837604633 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f4834089ba0 con 0x7f484406c100 2023-10-16T10:04:56.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.427+0000 7f483bfff640 1 --2- 172.21.15.84:0/837604633 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f482406af00 0x7f482406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:56.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.428+0000 7f483bfff640 1 --2- 172.21.15.84:0/837604633 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f482406af00 0x7f482406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f482c036000 tx=0x7f482c03d040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:56.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.431+0000 7f48417fa640 1 -- 172.21.15.84:0/837604633 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f48340548b0 con 0x7f484406c100 2023-10-16T10:04:56.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.625+0000 7f4849613640 1 -- 172.21.15.84:0/837604633 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48440b4620 con 0x7f484406c100 2023-10-16T10:04:56.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.626+0000 7f48417fa640 1 -- 172.21.15.84:0/837604633 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+17543 (secure 0 0 0) 0x7f4834015090 con 0x7f484406c100 2023-10-16T10:04:56.628 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:04:56.629 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":84,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:04:53.436179+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:04:30.728624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-16T10:04:31.866029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6832/2103121481":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/3340998984":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/2855693098":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/1664791074":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/2565413151":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/890477594":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/3177129458":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/1145018607":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/1499920052":"2023-10-17T10:04:47.124608+0000","172.21.15.84:6833/2103121481":"2023-10-17T10:04:53.436086+0000","172.21.15.84:6833/2091947430":"2023-10-17T10:04:47.124608+0000","172.21.15.84:6832/2091947430":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/1204215627":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/2509405053":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/3991569032":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/3599228496":"2023-10-17T10:04:47.124608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:04:56.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.629+0000 7f4849613640 1 -- 172.21.15.84:0/837604633 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f482406af00 msgr2=0x7f482406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:56.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.629+0000 7f4849613640 1 --2- 172.21.15.84:0/837604633 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f482406af00 0x7f482406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f482c036000 tx=0x7f482c03d040 comp rx=0 tx=0).stop 2023-10-16T10:04:56.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.629+0000 7f4849613640 1 -- 172.21.15.84:0/837604633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f484406c100 msgr2=0x7f484415c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:56.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.629+0000 7f4849613640 1 --2- 172.21.15.84:0/837604633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f484406c100 0x7f484415c470 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f483400b3e0 tx=0x7f483400b8a0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.629+0000 7f4849613640 1 -- 172.21.15.84:0/837604633 shutdown_connections 2023-10-16T10:04:56.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.629+0000 7f4849613640 1 --2- 172.21.15.84:0/837604633 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f482406af00 0x7f482406d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.629+0000 7f4849613640 1 --2- 172.21.15.84:0/837604633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4844159950 0x7f4844149d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.629+0000 7f4849613640 1 --2- 172.21.15.84:0/837604633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4844070b60 0x7f48441521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.629+0000 7f4849613640 1 --2- 172.21.15.84:0/837604633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f484406c100 0x7f484415c470 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.629+0000 7f4849613640 1 -- 172.21.15.84:0/837604633 >> 172.21.15.84:0/837604633 conn(0x7f48440ce6a0 msgr2=0x7f48440cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:56.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.629+0000 7f4849613640 1 -- 172.21.15.84:0/837604633 shutdown_connections 2023-10-16T10:04:56.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.630+0000 7f4849613640 1 -- 172.21.15.84:0/837604633 wait complete. 2023-10-16T10:04:56.649 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T10:04:56.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.854+0000 7f1668a40640 1 Processor -- start 2023-10-16T10:04:56.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.854+0000 7f1668a40640 1 -- start start 2023-10-16T10:04:56.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.856+0000 7f1668a40640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f166406c040 0x7f166406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:56.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.856+0000 7f1668a40640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16640ad970 0x7f166415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:56.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.856+0000 7f1668a40640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f166415ebc0 0x7f166415f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:56.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.856+0000 7f1668a40640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166406c960 con 0x7f166406c040 2023-10-16T10:04:56.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.856+0000 7f1668a40640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166415b9d0 con 0x7f166415ebc0 2023-10-16T10:04:56.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.856+0000 7f1668a40640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166415bcb0 con 0x7f16640ad970 2023-10-16T10:04:56.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.856+0000 7f16637fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f166406c040 0x7f166406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:56.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.856+0000 7f16637fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f166406c040 0x7f166406c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50210/0 (socket says 172.21.15.84:50210) 2023-10-16T10:04:56.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.856+0000 7f16637fe640 1 -- 172.21.15.84:0/2757983640 learned_addr learned my addr 172.21.15.84:0/2757983640 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:56.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.857+0000 7f165bfff640 1 --2- 172.21.15.84:0/2757983640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16640ad970 0x7f166415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:56.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.857+0000 7f1663fff640 1 --2- 172.21.15.84:0/2757983640 >> v2:172.21.15.157:3300/0 conn(0x7f166415ebc0 0x7f166415f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:56.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.857+0000 7f16637fe640 1 -- 172.21.15.84:0/2757983640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16640ad970 msgr2=0x7f166415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:56.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.857+0000 7f16637fe640 1 --2- 172.21.15.84:0/2757983640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16640ad970 0x7f166415e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.857+0000 7f16637fe640 1 -- 172.21.15.84:0/2757983640 >> v2:172.21.15.157:3300/0 conn(0x7f166415ebc0 msgr2=0x7f166415f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:56.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.857+0000 7f16637fe640 1 --2- 172.21.15.84:0/2757983640 >> v2:172.21.15.157:3300/0 conn(0x7f166415ebc0 0x7f166415f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.857+0000 7f16637fe640 1 -- 172.21.15.84:0/2757983640 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166415bf90 con 0x7f166406c040 2023-10-16T10:04:56.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.858+0000 7f16637fe640 1 --2- 172.21.15.84:0/2757983640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f166406c040 0x7f166406c420 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1654009870 tx=0x7f165402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc65d7f2d2f537ce server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:56.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.859+0000 7f1662ffd640 1 -- 172.21.15.84:0/2757983640 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1654031640 con 0x7f166406c040 2023-10-16T10:04:56.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.859+0000 7f1662ffd640 1 -- 172.21.15.84:0/2757983640 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16540317a0 con 0x7f166406c040 2023-10-16T10:04:56.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.859+0000 7f1662ffd640 1 -- 172.21.15.84:0/2757983640 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1654031b30 con 0x7f166406c040 2023-10-16T10:04:56.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.859+0000 7f1668a40640 1 -- 172.21.15.84:0/2757983640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f166406c040 msgr2=0x7f166406c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:56.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.859+0000 7f1668a40640 1 --2- 172.21.15.84:0/2757983640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f166406c040 0x7f166406c420 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1654009870 tx=0x7f165402b8f0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.860+0000 7f1668a40640 1 -- 172.21.15.84:0/2757983640 shutdown_connections 2023-10-16T10:04:56.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.860+0000 7f1668a40640 1 --2- 172.21.15.84:0/2757983640 >> v2:172.21.15.157:3300/0 conn(0x7f166415ebc0 0x7f166415f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.860+0000 7f1668a40640 1 --2- 172.21.15.84:0/2757983640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16640ad970 0x7f166415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.860+0000 7f1668a40640 1 --2- 172.21.15.84:0/2757983640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f166406c040 0x7f166406c420 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.860+0000 7f1668a40640 1 -- 172.21.15.84:0/2757983640 >> 172.21.15.84:0/2757983640 conn(0x7f16640ce6a0 msgr2=0x7f16640ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:56.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.860+0000 7f1668a40640 1 -- 172.21.15.84:0/2757983640 shutdown_connections 2023-10-16T10:04:56.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.860+0000 7f1668a40640 1 -- 172.21.15.84:0/2757983640 wait complete. 2023-10-16T10:04:56.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.861+0000 7f1668a40640 1 Processor -- start 2023-10-16T10:04:56.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.862+0000 7f1668a40640 1 -- start start 2023-10-16T10:04:56.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.863+0000 7f1668a40640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f166406c040 0x7f1664154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:56.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.863+0000 7f1668a40640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16640ad970 0x7f16640c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:56.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.863+0000 7f1668a40640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f166415ebc0 0x7f1664153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:56.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.863+0000 7f1668a40640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16640c7da0 con 0x7f166406c040 2023-10-16T10:04:56.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.863+0000 7f1668a40640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1664154140 con 0x7f166415ebc0 2023-10-16T10:04:56.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.863+0000 7f1668a40640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16641554c0 con 0x7f16640ad970 2023-10-16T10:04:56.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.863+0000 7f16637fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f166406c040 0x7f1664154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:56.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.863+0000 7f16637fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f166406c040 0x7f1664154de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50216/0 (socket says 172.21.15.84:50216) 2023-10-16T10:04:56.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.863+0000 7f16637fe640 1 -- 172.21.15.84:0/858177179 learned_addr learned my addr 172.21.15.84:0/858177179 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:56.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.863+0000 7f1663fff640 1 --2- 172.21.15.84:0/858177179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f166415ebc0 0x7f1664153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:56.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.863+0000 7f16637fe640 1 -- 172.21.15.84:0/858177179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16640ad970 msgr2=0x7f16640c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:56.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.863+0000 7f16637fe640 1 --2- 172.21.15.84:0/858177179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16640ad970 0x7f16640c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.863+0000 7f16637fe640 1 -- 172.21.15.84:0/858177179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f166415ebc0 msgr2=0x7f1664153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:56.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.863+0000 7f16637fe640 1 --2- 172.21.15.84:0/858177179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f166415ebc0 0x7f1664153c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:56.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.863+0000 7f16637fe640 1 -- 172.21.15.84:0/858177179 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16540094d0 con 0x7f166406c040 2023-10-16T10:04:56.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.864+0000 7f16637fe640 1 --2- 172.21.15.84:0/858177179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f166406c040 0x7f1664154de0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1654000c00 tx=0x7f16540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:56.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.864+0000 7f1660ff9640 1 -- 172.21.15.84:0/858177179 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165403ac60 con 0x7f166406c040 2023-10-16T10:04:56.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.864+0000 7f1660ff9640 1 -- 172.21.15.84:0/858177179 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165403adc0 con 0x7f166406c040 2023-10-16T10:04:56.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.864+0000 7f1660ff9640 1 -- 172.21.15.84:0/858177179 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1654044630 con 0x7f166406c040 2023-10-16T10:04:56.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.864+0000 7f1668a40640 1 -- 172.21.15.84:0/858177179 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16641557a0 con 0x7f166406c040 2023-10-16T10:04:56.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.864+0000 7f1668a40640 1 -- 172.21.15.84:0/858177179 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1664155d60 con 0x7f166406c040 2023-10-16T10:04:56.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.865+0000 7f1668a40640 1 -- 172.21.15.84:0/858177179 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f166406b330 con 0x7f166406c040 2023-10-16T10:04:56.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.866+0000 7f1660ff9640 1 -- 172.21.15.84:0/858177179 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 40) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f165402f050 con 0x7f166406c040 2023-10-16T10:04:56.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.867+0000 7f1660ff9640 1 --2- 172.21.15.84:0/858177179 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f164406af30 0x7f164406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:56.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.867+0000 7f1660ff9640 1 -- 172.21.15.84:0/858177179 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f16540ab8c0 con 0x7f166406c040 2023-10-16T10:04:56.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.867+0000 7f165bfff640 1 --2- 172.21.15.84:0/858177179 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f164406af30 0x7f164406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:56.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.868+0000 7f165bfff640 1 --2- 172.21.15.84:0/858177179 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f164406af30 0x7f164406d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f16640b1230 tx=0x7f164c008040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:56.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:56.872+0000 7f1660ff9640 1 -- 172.21.15.84:0/858177179 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1654076560 con 0x7f166406c040 2023-10-16T10:04:57.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.107+0000 7f1668a40640 1 -- 172.21.15.84:0/858177179 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f166404e9b0 con 0x7f166406c040 2023-10-16T10:04:57.109 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-16T10:04:57.108+0000 7fca0c7ff640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T10:04:57.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.111+0000 7f1660ff9640 1 -- 172.21.15.84:0/858177179 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1654031e60 con 0x7f166406c040 2023-10-16T10:04:57.113 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T10:04:57.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.114+0000 7f1668a40640 1 -- 172.21.15.84:0/858177179 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f164406af30 msgr2=0x7f164406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:57.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.114+0000 7f1668a40640 1 --2- 172.21.15.84:0/858177179 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f164406af30 0x7f164406d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f16640b1230 tx=0x7f164c008040 comp rx=0 tx=0).stop 2023-10-16T10:04:57.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.115+0000 7f1668a40640 1 -- 172.21.15.84:0/858177179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f166406c040 msgr2=0x7f1664154de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:57.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.115+0000 7f1668a40640 1 --2- 172.21.15.84:0/858177179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f166406c040 0x7f1664154de0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1654000c00 tx=0x7f16540046c0 comp rx=0 tx=0).stop 2023-10-16T10:04:57.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.115+0000 7f1668a40640 1 -- 172.21.15.84:0/858177179 shutdown_connections 2023-10-16T10:04:57.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.115+0000 7f1668a40640 1 --2- 172.21.15.84:0/858177179 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f164406af30 0x7f164406d3f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:57.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.115+0000 7f1668a40640 1 --2- 172.21.15.84:0/858177179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f166415ebc0 0x7f1664153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:57.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.115+0000 7f1668a40640 1 --2- 172.21.15.84:0/858177179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16640ad970 0x7f16640c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:57.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.115+0000 7f1668a40640 1 --2- 172.21.15.84:0/858177179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f166406c040 0x7f1664154de0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:57.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.115+0000 7f1668a40640 1 -- 172.21.15.84:0/858177179 >> 172.21.15.84:0/858177179 conn(0x7f16640ce6a0 msgr2=0x7f16640cd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:57.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.116+0000 7f1668a40640 1 -- 172.21.15.84:0/858177179 shutdown_connections 2023-10-16T10:04:57.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.116+0000 7f1668a40640 1 -- 172.21.15.84:0/858177179 wait complete. 2023-10-16T10:04:57.126 INFO:tasks.ceph.mds.c.smithi084.stderr: -3949> 2023-10-16T10:04:37.086+0000 7f47ff7bc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:04:57.138 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T10:04:57.178 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T10:04:57.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.324+0000 7fc4e9fa5640 1 Processor -- start 2023-10-16T10:04:57.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.324+0000 7fc4e9fa5640 1 -- start start 2023-10-16T10:04:57.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.326+0000 7fc4e9fa5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4e406a4b0 0x7fc4e40ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:57.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.326+0000 7fc4e9fa5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e40c9ca0 0x7fc4e40ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:57.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.326+0000 7fc4e9fa5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e40cb030 con 0x7fc4e415ad70 2023-10-16T10:04:57.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.326+0000 7fc4e9fa5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e40cae30 con 0x7fc4e406a4b0 2023-10-16T10:04:57.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.326+0000 7fc4e9fa5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40c91c0 con 0x7fc4e40c9ca0 2023-10-16T10:04:57.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.326+0000 7fc4e37fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc4e415ad70 0x7fc4e4069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57314/0 (socket says 172.21.15.84:57314) 2023-10-16T10:04:57.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.326+0000 7fc4e37fe640 1 -- 172.21.15.84:0/1612961584 learned_addr learned my addr 172.21.15.84:0/1612961584 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:57.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.327+0000 7fc4e3fff640 1 --2- 172.21.15.84:0/1612961584 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e40c9ca0 0x7fc4e40ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:57.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.327+0000 7fc4dbfff640 1 --2- 172.21.15.84:0/1612961584 >> v2:172.21.15.157:3300/0 conn(0x7fc4e406a4b0 0x7fc4e40ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:57.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.327+0000 7fc4e2ffd640 1 -- 172.21.15.84:0/1612961584 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 236036482 0 0) 0x7fc4e40cb030 con 0x7fc4e415ad70 2023-10-16T10:04:57.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.327+0000 7fc4e2ffd640 1 -- 172.21.15.84:0/1612961584 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4d0003520 con 0x7fc4e415ad70 2023-10-16T10:04:57.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.327+0000 7fc4dbfff640 1 -- 172.21.15.84:0/1612961584 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e40c9ca0 msgr2=0x7fc4e40ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:57.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.327+0000 7fc4dbfff640 1 --2- 172.21.15.84:0/1612961584 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e40c9ca0 0x7fc4e40ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:57.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.327+0000 7fc4dbfff640 1 -- 172.21.15.84:0/1612961584 >> v1:172.21.15.84:6789/0 conn(0x7fc4e415ad70 legacy=0x7fc4e4069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:57.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.327+0000 7fc4dbfff640 1 -- 172.21.15.84:0/1612961584 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e40c94a0 con 0x7fc4e406a4b0 2023-10-16T10:04:57.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.329+0000 7fc4dbfff640 1 --2- 172.21.15.84:0/1612961584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e406a4b0 0x7fc4e40ca3b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0097a0 tx=0x7fc4cc02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89edc67c8b4f6e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:57.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.329+0000 7fc4e2ffd640 1 -- 172.21.15.84:0/1612961584 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc02d040 con 0x7fc4e406a4b0 2023-10-16T10:04:57.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.329+0000 7fc4e2ffd640 1 -- 172.21.15.84:0/1612961584 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4cc031810 con 0x7fc4e406a4b0 2023-10-16T10:04:57.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.329+0000 7fc4e2ffd640 1 -- 172.21.15.84:0/1612961584 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc031ba0 con 0x7fc4e406a4b0 2023-10-16T10:04:57.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.330+0000 7fc4e9fa5640 1 -- 172.21.15.84:0/1612961584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e406a4b0 msgr2=0x7fc4e40ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:57.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.330+0000 7fc4e9fa5640 1 --2- 172.21.15.84:0/1612961584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e406a4b0 0x7fc4e40ca3b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0097a0 tx=0x7fc4cc02b8b0 comp rx=0 tx=0).stop 2023-10-16T10:04:57.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.330+0000 7fc4e9fa5640 1 -- 172.21.15.84:0/1612961584 shutdown_connections 2023-10-16T10:04:57.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.330+0000 7fc4e9fa5640 1 --2- 172.21.15.84:0/1612961584 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e40c9ca0 0x7fc4e40ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:57.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.330+0000 7fc4e9fa5640 1 --2- 172.21.15.84:0/1612961584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e406a4b0 0x7fc4e40ca3b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:57.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.330+0000 7fc4e9fa5640 1 -- 172.21.15.84:0/1612961584 >> 172.21.15.84:0/1612961584 conn(0x7fc4e40ce9b0 msgr2=0x7fc4e40cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:57.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.330+0000 7fc4e9fa5640 1 -- 172.21.15.84:0/1612961584 shutdown_connections 2023-10-16T10:04:57.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.330+0000 7fc4e9fa5640 1 -- 172.21.15.84:0/1612961584 wait complete. 2023-10-16T10:04:57.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.332+0000 7fc4e9fa5640 1 Processor -- start 2023-10-16T10:04:57.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.332+0000 7fc4e9fa5640 1 -- start start 2023-10-16T10:04:57.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.333+0000 7fc4e9fa5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e406a4b0 0x7fc4e415cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:57.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.333+0000 7fc4e9fa5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e40c9ca0 0x7fc4e40cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:57.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.333+0000 7fc4e9fa5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4e415ad70 0x7fc4e40ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:57.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.333+0000 7fc4e9fa5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40c63d0 con 0x7fc4e415ad70 2023-10-16T10:04:57.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.333+0000 7fc4e9fa5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40aee60 con 0x7fc4e40c9ca0 2023-10-16T10:04:57.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.333+0000 7fc4e9fa5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40ae250 con 0x7fc4e406a4b0 2023-10-16T10:04:57.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.333+0000 7fc4e3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4e415ad70 0x7fc4e40ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:57.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.333+0000 7fc4e3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4e415ad70 0x7fc4e40ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50258/0 (socket says 172.21.15.84:50258) 2023-10-16T10:04:57.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.333+0000 7fc4e3fff640 1 -- 172.21.15.84:0/2790004145 learned_addr learned my addr 172.21.15.84:0/2790004145 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:57.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.334+0000 7fc4e37fe640 1 --2- 172.21.15.84:0/2790004145 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e406a4b0 0x7fc4e415cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:57.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.334+0000 7fc4e3fff640 1 -- 172.21.15.84:0/2790004145 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e406a4b0 msgr2=0x7fc4e415cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:57.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.334+0000 7fc4e3fff640 1 --2- 172.21.15.84:0/2790004145 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e406a4b0 0x7fc4e415cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:57.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.334+0000 7fc4dbfff640 1 --2- 172.21.15.84:0/2790004145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e40c9ca0 0x7fc4e40cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:57.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.334+0000 7fc4e3fff640 1 -- 172.21.15.84:0/2790004145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e40c9ca0 msgr2=0x7fc4e40cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:57.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.334+0000 7fc4e3fff640 1 --2- 172.21.15.84:0/2790004145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e40c9ca0 0x7fc4e40cc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:57.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.334+0000 7fc4e3fff640 1 -- 172.21.15.84:0/2790004145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc009400 con 0x7fc4e415ad70 2023-10-16T10:04:57.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.334+0000 7fc4e3fff640 1 --2- 172.21.15.84:0/2790004145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4e415ad70 0x7fc4e40ccac0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00efc0 tx=0x7fc4dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:57.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.334+0000 7fc4e0ff9640 1 -- 172.21.15.84:0/2790004145 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc014480 con 0x7fc4e415ad70 2023-10-16T10:04:57.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.334+0000 7fc4e0ff9640 1 -- 172.21.15.84:0/2790004145 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4dc00f040 con 0x7fc4e415ad70 2023-10-16T10:04:57.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.334+0000 7fc4e9fa5640 1 -- 172.21.15.84:0/2790004145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e40ae530 con 0x7fc4e415ad70 2023-10-16T10:04:57.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.334+0000 7fc4e0ff9640 1 -- 172.21.15.84:0/2790004145 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc014820 con 0x7fc4e415ad70 2023-10-16T10:04:57.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.335+0000 7fc4e9fa5640 1 -- 172.21.15.84:0/2790004145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e40aeaf0 con 0x7fc4e415ad70 2023-10-16T10:04:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.335+0000 7fc4e9fa5640 1 -- 172.21.15.84:0/2790004145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e40c6540 con 0x7fc4e415ad70 2023-10-16T10:04:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.337+0000 7fc4e0ff9640 1 -- 172.21.15.84:0/2790004145 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 40) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fc4dc014980 con 0x7fc4e415ad70 2023-10-16T10:04:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.337+0000 7fc4e0ff9640 1 --2- 172.21.15.84:0/2790004145 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc4c406af00 0x7fc4c406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:57.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.338+0000 7fc4e0ff9640 1 -- 172.21.15.84:0/2790004145 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fc4dc08a6f0 con 0x7fc4e415ad70 2023-10-16T10:04:57.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.338+0000 7fc4e37fe640 1 --2- 172.21.15.84:0/2790004145 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc4c406af00 0x7fc4c406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:57.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.338+0000 7fc4e37fe640 1 --2- 172.21.15.84:0/2790004145 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc4c406af00 0x7fc4c406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40011e0 tx=0x7fc4d4009040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:57.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.342+0000 7fc4e0ff9640 1 -- 172.21.15.84:0/2790004145 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc4dc055320 con 0x7fc4e415ad70 2023-10-16T10:04:57.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:57.568+0000 7fc4e9fa5640 1 -- 172.21.15.84:0/2790004145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc4e40c9750 con 0x7fc4e415ad70 2023-10-16T10:04:58.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.114+0000 7fc4e0ff9640 1 -- 172.21.15.84:0/2790004145 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc4dc058940 con 0x7fc4e415ad70 2023-10-16T10:04:58.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.118+0000 7fc4e9fa5640 1 -- 172.21.15.84:0/2790004145 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc4c406af00 msgr2=0x7fc4c406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:58.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.118+0000 7fc4e9fa5640 1 --2- 172.21.15.84:0/2790004145 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc4c406af00 0x7fc4c406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40011e0 tx=0x7fc4d4009040 comp rx=0 tx=0).stop 2023-10-16T10:04:58.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.118+0000 7fc4e9fa5640 1 -- 172.21.15.84:0/2790004145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4e415ad70 msgr2=0x7fc4e40ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:58.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.118+0000 7fc4e9fa5640 1 --2- 172.21.15.84:0/2790004145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4e415ad70 0x7fc4e40ccac0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00efc0 tx=0x7fc4dc00c490 comp rx=0 tx=0).stop 2023-10-16T10:04:58.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.118+0000 7fc4e9fa5640 1 -- 172.21.15.84:0/2790004145 shutdown_connections 2023-10-16T10:04:58.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.118+0000 7fc4e9fa5640 1 --2- 172.21.15.84:0/2790004145 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc4c406af00 0x7fc4c406d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:58.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.118+0000 7fc4e9fa5640 1 --2- 172.21.15.84:0/2790004145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4e415ad70 0x7fc4e40ccac0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:58.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.118+0000 7fc4e9fa5640 1 --2- 172.21.15.84:0/2790004145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e40c9ca0 0x7fc4e40cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:58.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.118+0000 7fc4e9fa5640 1 --2- 172.21.15.84:0/2790004145 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e406a4b0 0x7fc4e415cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:58.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.118+0000 7fc4e9fa5640 1 -- 172.21.15.84:0/2790004145 >> 172.21.15.84:0/2790004145 conn(0x7fc4e40ce9b0 msgr2=0x7fc4e40afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:58.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.118+0000 7fc4e9fa5640 1 -- 172.21.15.84:0/2790004145 shutdown_connections 2023-10-16T10:04:58.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.118+0000 7fc4e9fa5640 1 -- 172.21.15.84:0/2790004145 wait complete. 2023-10-16T10:04:58.140 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T10:04:58.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.405+0000 7f5ecd685640 1 Processor -- start 2023-10-16T10:04:58.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.406+0000 7f5ecd685640 1 -- start start 2023-10-16T10:04:58.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.407+0000 7f5ecd685640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec80b8480 0x7f5ec80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:58.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.407+0000 7f5ecd685640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5ec80d0be0 0x7f5ec80cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:58.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.407+0000 7f5ecd685640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec815d150 con 0x7f5ec815cd70 2023-10-16T10:04:58.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.407+0000 7f5ecd685640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec80d1180 con 0x7f5ec80d0be0 2023-10-16T10:04:58.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.407+0000 7f5ecd685640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80d1460 con 0x7f5ec80b8480 2023-10-16T10:04:58.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.407+0000 7f5ecce84640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5ec80d0be0 0x7f5ec80cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:58.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.407+0000 7f5ec77fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec80b8480 0x7f5ec80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:58.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.407+0000 7f5ecce84640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5ec80d0be0 0x7f5ec80cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:39488/0 (socket says 172.21.15.84:39488) 2023-10-16T10:04:58.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.407+0000 7f5ecce84640 1 -- 172.21.15.84:0/1058122648 learned_addr learned my addr 172.21.15.84:0/1058122648 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:58.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.408+0000 7f5ecce84640 1 -- 172.21.15.84:0/1058122648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec80b8480 msgr2=0x7f5ec80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:58.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.408+0000 7f5ecce84640 1 --2- 172.21.15.84:0/1058122648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec80b8480 0x7f5ec80cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:58.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.408+0000 7f5ecce84640 1 -- 172.21.15.84:0/1058122648 >> v1:172.21.15.157:6789/0 conn(0x7f5ec815cd70 legacy=0x7f5ec80d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:58.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.408+0000 7f5ecce84640 1 -- 172.21.15.84:0/1058122648 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec80d1740 con 0x7f5ec80d0be0 2023-10-16T10:04:58.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.410+0000 7f5ecce84640 1 --2- 172.21.15.84:0/1058122648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec80d0be0 0x7f5ec80cdd00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009870 tx=0x7f5ec002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cbcf7930150289f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:58.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.410+0000 7f5ec6ffd640 1 -- 172.21.15.84:0/1058122648 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0030840 con 0x7f5ec80d0be0 2023-10-16T10:04:58.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.410+0000 7f5ec6ffd640 1 -- 172.21.15.84:0/1058122648 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec00309a0 con 0x7f5ec80d0be0 2023-10-16T10:04:58.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.410+0000 7f5ec6ffd640 1 -- 172.21.15.84:0/1058122648 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0030d30 con 0x7f5ec80d0be0 2023-10-16T10:04:58.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.411+0000 7f5ecd685640 1 -- 172.21.15.84:0/1058122648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec80d0be0 msgr2=0x7f5ec80cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:58.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.411+0000 7f5ecd685640 1 --2- 172.21.15.84:0/1058122648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec80d0be0 0x7f5ec80cdd00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009870 tx=0x7f5ec002b980 comp rx=0 tx=0).stop 2023-10-16T10:04:58.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.411+0000 7f5ecd685640 1 -- 172.21.15.84:0/1058122648 shutdown_connections 2023-10-16T10:04:58.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.411+0000 7f5ecd685640 1 --2- 172.21.15.84:0/1058122648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec80d0be0 0x7f5ec80cdd00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:58.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.411+0000 7f5ecd685640 1 --2- 172.21.15.84:0/1058122648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec80b8480 0x7f5ec80cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:58.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.411+0000 7f5ecd685640 1 -- 172.21.15.84:0/1058122648 >> 172.21.15.84:0/1058122648 conn(0x7f5ec80cf570 msgr2=0x7f5ec80cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:58.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.411+0000 7f5ecd685640 1 -- 172.21.15.84:0/1058122648 shutdown_connections 2023-10-16T10:04:58.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.411+0000 7f5ecd685640 1 -- 172.21.15.84:0/1058122648 wait complete. 2023-10-16T10:04:58.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.413+0000 7f5ecd685640 1 Processor -- start 2023-10-16T10:04:58.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.413+0000 7f5ecd685640 1 -- start start 2023-10-16T10:04:58.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.414+0000 7f5ecd685640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ec80b8480 0x7f5ec815fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:58.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.414+0000 7f5ecd685640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec80d0be0 0x7f5ec80c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:58.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.414+0000 7f5ecd685640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec815cd70 0x7f5ec80ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:58.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.414+0000 7f5ecd685640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec8151820 con 0x7f5ec80b8480 2023-10-16T10:04:58.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.414+0000 7f5ecd685640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80cb320 con 0x7f5ec815cd70 2023-10-16T10:04:58.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.414+0000 7f5ecd685640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80ca7a0 con 0x7f5ec80d0be0 2023-10-16T10:04:58.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.414+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ec80b8480 0x7f5ec815fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:58.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.414+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ec80b8480 0x7f5ec815fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50266/0 (socket says 172.21.15.84:50266) 2023-10-16T10:04:58.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.414+0000 7f5ec7fff640 1 -- 172.21.15.84:0/2767759616 learned_addr learned my addr 172.21.15.84:0/2767759616 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:58.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.414+0000 7f5ecce84640 1 --2- 172.21.15.84:0/2767759616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec815cd70 0x7f5ec80ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:58.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.414+0000 7f5ec77fe640 1 --2- 172.21.15.84:0/2767759616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec80d0be0 0x7f5ec80c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:58.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.414+0000 7f5ec7fff640 1 -- 172.21.15.84:0/2767759616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec80d0be0 msgr2=0x7f5ec80c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:58.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.414+0000 7f5ec7fff640 1 --2- 172.21.15.84:0/2767759616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec80d0be0 0x7f5ec80c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:58.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.414+0000 7f5ec7fff640 1 -- 172.21.15.84:0/2767759616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec815cd70 msgr2=0x7f5ec80ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:58.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.414+0000 7f5ec7fff640 1 --2- 172.21.15.84:0/2767759616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec815cd70 0x7f5ec80ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:58.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.414+0000 7f5ec7fff640 1 -- 172.21.15.84:0/2767759616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec00094d0 con 0x7f5ec80b8480 2023-10-16T10:04:58.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.415+0000 7f5ec7fff640 1 --2- 172.21.15.84:0/2767759616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ec80b8480 0x7f5ec815fa90 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800c870 tx=0x7f5eb800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:58.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.415+0000 7f5ec4ff9640 1 -- 172.21.15.84:0/2767759616 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8003b70 con 0x7f5ec80b8480 2023-10-16T10:04:58.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.415+0000 7f5ec4ff9640 1 -- 172.21.15.84:0/2767759616 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb8003cf0 con 0x7f5ec80b8480 2023-10-16T10:04:58.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.415+0000 7f5ecd685640 1 -- 172.21.15.84:0/2767759616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec80caa80 con 0x7f5ec80b8480 2023-10-16T10:04:58.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.415+0000 7f5ec4ff9640 1 -- 172.21.15.84:0/2767759616 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb800aa90 con 0x7f5ec80b8480 2023-10-16T10:04:58.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.415+0000 7f5ecd685640 1 -- 172.21.15.84:0/2767759616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec80cb040 con 0x7f5ec80b8480 2023-10-16T10:04:58.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.416+0000 7f5ecd685640 1 -- 172.21.15.84:0/2767759616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec804e9b0 con 0x7f5ec80b8480 2023-10-16T10:04:58.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.417+0000 7f5ec4ff9640 1 -- 172.21.15.84:0/2767759616 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5eb800abf0 con 0x7f5ec80b8480 2023-10-16T10:04:58.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.418+0000 7f5ec4ff9640 1 --2- 172.21.15.84:0/2767759616 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f5ea806af90 0x7f5ea806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:58.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.418+0000 7f5ec4ff9640 1 -- 172.21.15.84:0/2767759616 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f5eb8089980 con 0x7f5ec80b8480 2023-10-16T10:04:58.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.418+0000 7f5ec77fe640 1 --2- 172.21.15.84:0/2767759616 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f5ea806af90 0x7f5ea806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:58.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.419+0000 7f5ec77fe640 1 --2- 172.21.15.84:0/2767759616 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f5ea806af90 0x7f5ea806d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5ec815f7b0 tx=0x7f5eb0008040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:58.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.422+0000 7f5ec4ff9640 1 -- 172.21.15.84:0/2767759616 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5eb8054600 con 0x7f5ec80b8480 2023-10-16T10:04:58.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:58.614+0000 7f5ecd685640 1 -- 172.21.15.84:0/2767759616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5ec80b6800 con 0x7f5ec80b8480 2023-10-16T10:04:59.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.231+0000 7f5ec4ff9640 1 -- 172.21.15.84:0/2767759616 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5eb8003e60 con 0x7f5ec80b8480 2023-10-16T10:04:59.233 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-16T10:04:59.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.236+0000 7f5ecd685640 1 -- 172.21.15.84:0/2767759616 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f5ea806af90 msgr2=0x7f5ea806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:59.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.236+0000 7f5ecd685640 1 --2- 172.21.15.84:0/2767759616 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f5ea806af90 0x7f5ea806d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5ec815f7b0 tx=0x7f5eb0008040 comp rx=0 tx=0).stop 2023-10-16T10:04:59.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.236+0000 7f5ecd685640 1 -- 172.21.15.84:0/2767759616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ec80b8480 msgr2=0x7f5ec815fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:59.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.236+0000 7f5ecd685640 1 --2- 172.21.15.84:0/2767759616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ec80b8480 0x7f5ec815fa90 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800c870 tx=0x7f5eb800cd30 comp rx=0 tx=0).stop 2023-10-16T10:04:59.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.237+0000 7f5ecd685640 1 -- 172.21.15.84:0/2767759616 shutdown_connections 2023-10-16T10:04:59.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.237+0000 7f5ecd685640 1 --2- 172.21.15.84:0/2767759616 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f5ea806af90 0x7f5ea806d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:59.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.237+0000 7f5ecd685640 1 --2- 172.21.15.84:0/2767759616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec815cd70 0x7f5ec80ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:59.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.237+0000 7f5ecd685640 1 --2- 172.21.15.84:0/2767759616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ec80d0be0 0x7f5ec80c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:59.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.237+0000 7f5ecd685640 1 --2- 172.21.15.84:0/2767759616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ec80b8480 0x7f5ec815fa90 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:59.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.237+0000 7f5ecd685640 1 -- 172.21.15.84:0/2767759616 >> 172.21.15.84:0/2767759616 conn(0x7f5ec80cf570 msgr2=0x7f5ec80d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:59.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.240+0000 7f5ecd685640 1 -- 172.21.15.84:0/2767759616 shutdown_connections 2023-10-16T10:04:59.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.240+0000 7f5ecd685640 1 -- 172.21.15.84:0/2767759616 wait complete. 2023-10-16T10:04:59.265 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T10:04:59.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.465+0000 7f92d08ec640 1 Processor -- start 2023-10-16T10:04:59.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.465+0000 7f92d08ec640 1 -- start start 2023-10-16T10:04:59.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.466+0000 7f92d08ec640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f92cc0b8480 0x7f92cc0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:59.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.466+0000 7f92d08ec640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92cc15d150 con 0x7f92cc0d0be0 2023-10-16T10:04:59.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.466+0000 7f92d08ec640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92cc0d1180 con 0x7f92cc15cd70 2023-10-16T10:04:59.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.466+0000 7f92d08ec640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92cc0d1460 con 0x7f92cc0b8480 2023-10-16T10:04:59.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.467+0000 7f92cbfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f92cc0d0be0 0x7f92cc0b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57316/0 (socket says 172.21.15.84:57316) 2023-10-16T10:04:59.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.467+0000 7f92cbfff640 1 -- 172.21.15.84:0/3403454571 learned_addr learned my addr 172.21.15.84:0/3403454571 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:04:59.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.467+0000 7f92c3fff640 1 --2- 172.21.15.84:0/3403454571 >> v2:172.21.15.84:3300/0 conn(0x7f92cc0b8480 0x7f92cc0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:59.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.467+0000 7f92c3fff640 1 -- 172.21.15.84:0/3403454571 >> v1:172.21.15.157:6789/0 conn(0x7f92cc15cd70 legacy=0x7f92cc0d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:59.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.467+0000 7f92c3fff640 1 -- 172.21.15.84:0/3403454571 >> v1:172.21.15.84:6789/0 conn(0x7f92cc0d0be0 legacy=0x7f92cc0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:59.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.467+0000 7f92c3fff640 1 -- 172.21.15.84:0/3403454571 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc0d1740 con 0x7f92cc0b8480 2023-10-16T10:04:59.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.468+0000 7f92c3fff640 1 --2- 172.21.15.84:0/3403454571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92cc0b8480 0x7f92cc0cd7c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f92b40097a0 tx=0x7f92b402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6170d19fe5e2e2cc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:59.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.469+0000 7f92caffd640 1 -- 172.21.15.84:0/3403454571 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b402e070 con 0x7f92cc0b8480 2023-10-16T10:04:59.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.469+0000 7f92caffd640 1 -- 172.21.15.84:0/3403454571 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b40315a0 con 0x7f92cc0b8480 2023-10-16T10:04:59.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.469+0000 7f92caffd640 1 -- 172.21.15.84:0/3403454571 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4031930 con 0x7f92cc0b8480 2023-10-16T10:04:59.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.469+0000 7f92d08ec640 1 -- 172.21.15.84:0/3403454571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92cc0b8480 msgr2=0x7f92cc0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:59.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.469+0000 7f92d08ec640 1 --2- 172.21.15.84:0/3403454571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92cc0b8480 0x7f92cc0cd7c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f92b40097a0 tx=0x7f92b402b840 comp rx=0 tx=0).stop 2023-10-16T10:04:59.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.469+0000 7f92d08ec640 1 -- 172.21.15.84:0/3403454571 shutdown_connections 2023-10-16T10:04:59.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.469+0000 7f92d08ec640 1 --2- 172.21.15.84:0/3403454571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92cc0b8480 0x7f92cc0cd7c0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:59.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.469+0000 7f92d08ec640 1 -- 172.21.15.84:0/3403454571 >> 172.21.15.84:0/3403454571 conn(0x7f92cc0cf570 msgr2=0x7f92cc0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:59.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.470+0000 7f92d08ec640 1 -- 172.21.15.84:0/3403454571 shutdown_connections 2023-10-16T10:04:59.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.470+0000 7f92d08ec640 1 -- 172.21.15.84:0/3403454571 wait complete. 2023-10-16T10:04:59.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.471+0000 7f92d08ec640 1 Processor -- start 2023-10-16T10:04:59.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.472+0000 7f92d08ec640 1 -- start start 2023-10-16T10:04:59.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.472+0000 7f92d08ec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92cc0b8480 0x7f92cc1f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:59.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.473+0000 7f92d08ec640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92cc0d0be0 0x7f92cc14cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:59.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.473+0000 7f92d08ec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92cc15cd70 0x7f92cc1f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:59.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.473+0000 7f92d08ec640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc0ccb40 con 0x7f92cc15cd70 2023-10-16T10:04:59.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.473+0000 7f92d08ec640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc1f66c0 con 0x7f92cc0d0be0 2023-10-16T10:04:59.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.473+0000 7f92d08ec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc1f69a0 con 0x7f92cc0b8480 2023-10-16T10:04:59.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.473+0000 7f92cbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92cc15cd70 0x7f92cc1f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:59.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.473+0000 7f92cbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92cc15cd70 0x7f92cc1f5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50282/0 (socket says 172.21.15.84:50282) 2023-10-16T10:04:59.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.473+0000 7f92cbfff640 1 -- 172.21.15.84:0/3130990330 learned_addr learned my addr 172.21.15.84:0/3130990330 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:04:59.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.473+0000 7f92c3fff640 1 --2- 172.21.15.84:0/3130990330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92cc0d0be0 0x7f92cc14cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:59.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.473+0000 7f92cb7fe640 1 --2- 172.21.15.84:0/3130990330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92cc0b8480 0x7f92cc1f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:59.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.473+0000 7f92cbfff640 1 -- 172.21.15.84:0/3130990330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92cc0b8480 msgr2=0x7f92cc1f26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:59.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.473+0000 7f92cbfff640 1 --2- 172.21.15.84:0/3130990330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92cc0b8480 0x7f92cc1f26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:59.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.473+0000 7f92cbfff640 1 -- 172.21.15.84:0/3130990330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92cc0d0be0 msgr2=0x7f92cc14cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:59.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.473+0000 7f92cbfff640 1 --2- 172.21.15.84:0/3130990330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92cc0d0be0 0x7f92cc14cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:59.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.473+0000 7f92cbfff640 1 -- 172.21.15.84:0/3130990330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b4009400 con 0x7f92cc15cd70 2023-10-16T10:04:59.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.473+0000 7f92c3fff640 1 --2- 172.21.15.84:0/3130990330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92cc0d0be0 0x7f92cc14cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:59.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.474+0000 7f92cbfff640 1 --2- 172.21.15.84:0/3130990330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92cc15cd70 0x7f92cc1f5c70 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f92c400ca70 tx=0x7f92c400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:59.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.474+0000 7f92c8ff9640 1 -- 172.21.15.84:0/3130990330 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c400abd0 con 0x7f92cc15cd70 2023-10-16T10:04:59.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.474+0000 7f92c8ff9640 1 -- 172.21.15.84:0/3130990330 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c400ad30 con 0x7f92cc15cd70 2023-10-16T10:04:59.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.474+0000 7f92d08ec640 1 -- 172.21.15.84:0/3130990330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92cc15f020 con 0x7f92cc15cd70 2023-10-16T10:04:59.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.474+0000 7f92c8ff9640 1 -- 172.21.15.84:0/3130990330 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c4012600 con 0x7f92cc15cd70 2023-10-16T10:04:59.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.474+0000 7f92d08ec640 1 -- 172.21.15.84:0/3130990330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92cc1f2c30 con 0x7f92cc15cd70 2023-10-16T10:04:59.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.475+0000 7f92d08ec640 1 -- 172.21.15.84:0/3130990330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92cc15f540 con 0x7f92cc15cd70 2023-10-16T10:04:59.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.476+0000 7f92c8ff9640 1 -- 172.21.15.84:0/3130990330 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f92c4007070 con 0x7f92cc15cd70 2023-10-16T10:04:59.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.477+0000 7f92c8ff9640 1 --2- 172.21.15.84:0/3130990330 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f92ac06af90 0x7f92ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:59.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.478+0000 7f92c8ff9640 1 -- 172.21.15.84:0/3130990330 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6045+0+0 (secure 0 0 0) 0x7f92c40896c0 con 0x7f92cc15cd70 2023-10-16T10:04:59.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.478+0000 7f92cb7fe640 1 --2- 172.21.15.84:0/3130990330 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f92ac06af90 0x7f92ac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:59.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.478+0000 7f92cb7fe640 1 --2- 172.21.15.84:0/3130990330 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f92ac06af90 0x7f92ac06d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f92bc003d90 tx=0x7f92bc009040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:59.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.482+0000 7f92c8ff9640 1 -- 172.21.15.84:0/3130990330 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f92cc15f540 con 0x7f92cc15cd70 2023-10-16T10:04:59.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:04:59.676+0000 7f92d08ec640 1 -- 172.21.15.84:0/3130990330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f92cc15f540 con 0x7f92cc15cd70 2023-10-16T10:05:00.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.254+0000 7f92c8ff9640 1 -- 172.21.15.84:0/3130990330 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7f92cc15f540 con 0x7f92cc15cd70 2023-10-16T10:05:00.256 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-16T10:05:00.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.258+0000 7f92d08ec640 1 -- 172.21.15.84:0/3130990330 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f92ac06af90 msgr2=0x7f92ac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:00.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.258+0000 7f92d08ec640 1 --2- 172.21.15.84:0/3130990330 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f92ac06af90 0x7f92ac06d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f92bc003d90 tx=0x7f92bc009040 comp rx=0 tx=0).stop 2023-10-16T10:05:00.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.258+0000 7f92d08ec640 1 -- 172.21.15.84:0/3130990330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92cc15cd70 msgr2=0x7f92cc1f5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:00.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.258+0000 7f92d08ec640 1 --2- 172.21.15.84:0/3130990330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92cc15cd70 0x7f92cc1f5c70 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f92c400ca70 tx=0x7f92c400cf30 comp rx=0 tx=0).stop 2023-10-16T10:05:00.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.260+0000 7f92d08ec640 1 -- 172.21.15.84:0/3130990330 shutdown_connections 2023-10-16T10:05:00.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.260+0000 7f92d08ec640 1 --2- 172.21.15.84:0/3130990330 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f92ac06af90 0x7f92ac06d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.260+0000 7f92d08ec640 1 --2- 172.21.15.84:0/3130990330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92cc15cd70 0x7f92cc1f5c70 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.260+0000 7f92d08ec640 1 --2- 172.21.15.84:0/3130990330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92cc0d0be0 0x7f92cc14cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.260+0000 7f92d08ec640 1 --2- 172.21.15.84:0/3130990330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92cc0b8480 0x7f92cc1f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.260+0000 7f92d08ec640 1 -- 172.21.15.84:0/3130990330 >> 172.21.15.84:0/3130990330 conn(0x7f92cc0cf570 msgr2=0x7f92cc0b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:00.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.262+0000 7f92d08ec640 1 -- 172.21.15.84:0/3130990330 shutdown_connections 2023-10-16T10:05:00.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.262+0000 7f92d08ec640 1 -- 172.21.15.84:0/3130990330 wait complete. 2023-10-16T10:05:00.288 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T10:05:00.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.486+0000 7f08c86da640 1 Processor -- start 2023-10-16T10:05:00.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.487+0000 7f08c86da640 1 -- start start 2023-10-16T10:05:00.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.488+0000 7f08c86da640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f08c00caac0 0x7f08c00ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:00.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.488+0000 7f08c86da640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c00cb030 con 0x7f08c006a4b0 2023-10-16T10:05:00.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.488+0000 7f08c86da640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c00c91c0 con 0x7f08c015ad70 2023-10-16T10:05:00.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.488+0000 7f08c86da640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c00c94a0 con 0x7f08c00caac0 2023-10-16T10:05:00.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.488+0000 7f08c6c77640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f08c00caac0 0x7f08c00ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:00.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.488+0000 7f08c5c75640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f08c006a4b0 0x7f08c00ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57328/0 (socket says 172.21.15.84:57328) 2023-10-16T10:05:00.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.488+0000 7f08c5c75640 1 -- 172.21.15.84:0/1601203083 learned_addr learned my addr 172.21.15.84:0/1601203083 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:00.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.489+0000 7f08c6c77640 1 -- 172.21.15.84:0/1601203083 >> v1:172.21.15.157:6789/0 conn(0x7f08c015ad70 legacy=0x7f08c0069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.489+0000 7f08c6c77640 1 -- 172.21.15.84:0/1601203083 >> v1:172.21.15.84:6789/0 conn(0x7f08c006a4b0 legacy=0x7f08c00ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.489+0000 7f08c6c77640 1 -- 172.21.15.84:0/1601203083 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c00c9780 con 0x7f08c00caac0 2023-10-16T10:05:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.489+0000 7f08c5474640 1 -- 172.21.15.84:0/1601203083 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942595669 0 0) 0x7f08c00cb030 con 0x7f08c006a4b0 2023-10-16T10:05:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.490+0000 7f08c6c77640 1 --2- 172.21.15.84:0/1601203083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c00caac0 0x7f08c00ad4d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009d90 tx=0x7f08b802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a062d63670323f50 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.490+0000 7f08c5474640 1 -- 172.21.15.84:0/1601203083 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b802cc60 con 0x7f08c00caac0 2023-10-16T10:05:00.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.490+0000 7f08c5474640 1 -- 172.21.15.84:0/1601203083 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b802cdc0 con 0x7f08c00caac0 2023-10-16T10:05:00.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.490+0000 7f08c5474640 1 -- 172.21.15.84:0/1601203083 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8034660 con 0x7f08c00caac0 2023-10-16T10:05:00.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.491+0000 7f08c86da640 1 -- 172.21.15.84:0/1601203083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c00caac0 msgr2=0x7f08c00ad4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:00.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.491+0000 7f08c86da640 1 --2- 172.21.15.84:0/1601203083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c00caac0 0x7f08c00ad4d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009d90 tx=0x7f08b802beb0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.491+0000 7f08c86da640 1 -- 172.21.15.84:0/1601203083 shutdown_connections 2023-10-16T10:05:00.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.491+0000 7f08c86da640 1 --2- 172.21.15.84:0/1601203083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c00caac0 0x7f08c00ad4d0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.491+0000 7f08c86da640 1 -- 172.21.15.84:0/1601203083 >> 172.21.15.84:0/1601203083 conn(0x7f08c00ce9b0 msgr2=0x7f08c00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:00.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.491+0000 7f08c86da640 1 -- 172.21.15.84:0/1601203083 shutdown_connections 2023-10-16T10:05:00.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.491+0000 7f08c86da640 1 -- 172.21.15.84:0/1601203083 wait complete. 2023-10-16T10:05:00.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.493+0000 7f08c86da640 1 Processor -- start 2023-10-16T10:05:00.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.493+0000 7f08c86da640 1 -- start start 2023-10-16T10:05:00.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.494+0000 7f08c86da640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08c006a4b0 0x7f08c00a8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:00.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.494+0000 7f08c86da640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c00caac0 0x7f08c00c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:00.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.494+0000 7f08c86da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08c015ad70 0x7f08c014ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:00.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.494+0000 7f08c86da640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c00c5010 con 0x7f08c00caac0 2023-10-16T10:05:00.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.494+0000 7f08c86da640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c014f260 con 0x7f08c015ad70 2023-10-16T10:05:00.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.494+0000 7f08c86da640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c00c5200 con 0x7f08c006a4b0 2023-10-16T10:05:00.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.494+0000 7f08c5c75640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c00caac0 0x7f08c00c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:00.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.494+0000 7f08c5c75640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c00caac0 0x7f08c00c4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50294/0 (socket says 172.21.15.84:50294) 2023-10-16T10:05:00.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.494+0000 7f08c5c75640 1 -- 172.21.15.84:0/159450101 learned_addr learned my addr 172.21.15.84:0/159450101 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:00.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.495+0000 7f08c6c77640 1 --2- 172.21.15.84:0/159450101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08c015ad70 0x7f08c014ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:00.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.495+0000 7f08c5c75640 1 -- 172.21.15.84:0/159450101 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08c006a4b0 msgr2=0x7f08c00a8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:00.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.495+0000 7f08c5c75640 1 --2- 172.21.15.84:0/159450101 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08c006a4b0 0x7f08c00a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.495+0000 7f08c5c75640 1 -- 172.21.15.84:0/159450101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08c015ad70 msgr2=0x7f08c014ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:00.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.495+0000 7f08c5c75640 1 --2- 172.21.15.84:0/159450101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08c015ad70 0x7f08c014ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.495+0000 7f08c5c75640 1 -- 172.21.15.84:0/159450101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b8009a70 con 0x7f08c00caac0 2023-10-16T10:05:00.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.495+0000 7f08c5c75640 1 --2- 172.21.15.84:0/159450101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c00caac0 0x7f08c00c4ad0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f08b000c4d0 tx=0x7f08b000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:00.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.495+0000 7f08b6ffd640 1 -- 172.21.15.84:0/159450101 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b00074e0 con 0x7f08c00caac0 2023-10-16T10:05:00.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.496+0000 7f08b6ffd640 1 -- 172.21.15.84:0/159450101 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b0007660 con 0x7f08c00caac0 2023-10-16T10:05:00.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.496+0000 7f08c86da640 1 -- 172.21.15.84:0/159450101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c00c6b30 con 0x7f08c00caac0 2023-10-16T10:05:00.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.496+0000 7f08b6ffd640 1 -- 172.21.15.84:0/159450101 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b0004a20 con 0x7f08c00caac0 2023-10-16T10:05:00.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.496+0000 7f08c86da640 1 -- 172.21.15.84:0/159450101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c00c70f0 con 0x7f08c00caac0 2023-10-16T10:05:00.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.497+0000 7f08c86da640 1 -- 172.21.15.84:0/159450101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c00cf2f0 con 0x7f08c00caac0 2023-10-16T10:05:00.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.498+0000 7f08b6ffd640 1 -- 172.21.15.84:0/159450101 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f08b0004b80 con 0x7f08c00caac0 2023-10-16T10:05:00.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.499+0000 7f08b6ffd640 1 --2- 172.21.15.84:0/159450101 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f08a406af90 0x7f08a406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:00.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.499+0000 7f08b6ffd640 1 -- 172.21.15.84:0/159450101 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f08b0088bc0 con 0x7f08c00caac0 2023-10-16T10:05:00.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.499+0000 7f08c6476640 1 --2- 172.21.15.84:0/159450101 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f08a406af90 0x7f08a406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:00.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.499+0000 7f08c6476640 1 --2- 172.21.15.84:0/159450101 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f08a406af90 0x7f08a406d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f08bc006fd0 tx=0x7f08bc006bc0 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:00.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.503+0000 7f08b6ffd640 1 -- 172.21.15.84:0/159450101 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f08b0053c40 con 0x7f08c00caac0 2023-10-16T10:05:00.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.708+0000 7f08c86da640 1 -- 172.21.15.84:0/159450101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f08c004e160 con 0x7f08c00caac0 2023-10-16T10:05:00.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.713+0000 7f08b6ffd640 1 -- 172.21.15.84:0/159450101 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f08b00180a0 con 0x7f08c00caac0 2023-10-16T10:05:00.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.715+0000 7f08c86da640 1 -- 172.21.15.84:0/159450101 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f08a406af90 msgr2=0x7f08a406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:00.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.716+0000 7f08c86da640 1 --2- 172.21.15.84:0/159450101 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f08a406af90 0x7f08a406d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f08bc006fd0 tx=0x7f08bc006bc0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.716+0000 7f08c86da640 1 -- 172.21.15.84:0/159450101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c00caac0 msgr2=0x7f08c00c4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:00.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.716+0000 7f08c86da640 1 --2- 172.21.15.84:0/159450101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c00caac0 0x7f08c00c4ad0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f08b000c4d0 tx=0x7f08b000c990 comp rx=0 tx=0).stop 2023-10-16T10:05:00.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.716+0000 7f08c86da640 1 -- 172.21.15.84:0/159450101 shutdown_connections 2023-10-16T10:05:00.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.716+0000 7f08c86da640 1 --2- 172.21.15.84:0/159450101 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f08a406af90 0x7f08a406d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.716+0000 7f08c86da640 1 --2- 172.21.15.84:0/159450101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08c015ad70 0x7f08c014ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.716+0000 7f08c86da640 1 --2- 172.21.15.84:0/159450101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c00caac0 0x7f08c00c4ad0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.716+0000 7f08c86da640 1 --2- 172.21.15.84:0/159450101 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08c006a4b0 0x7f08c00a8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.716+0000 7f08c86da640 1 -- 172.21.15.84:0/159450101 >> 172.21.15.84:0/159450101 conn(0x7f08c00ce9b0 msgr2=0x7f08c0069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:00.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.716+0000 7f08c86da640 1 -- 172.21.15.84:0/159450101 shutdown_connections 2023-10-16T10:05:00.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.716+0000 7f08c86da640 1 -- 172.21.15.84:0/159450101 wait complete. 2023-10-16T10:05:00.733 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T10:05:00.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.922+0000 7fc76cb33640 1 Processor -- start 2023-10-16T10:05:00.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.922+0000 7fc76cb33640 1 -- start start 2023-10-16T10:05:00.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.924+0000 7fc76cb33640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76806a4b0 0x7fc7680ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:00.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.924+0000 7fc76cb33640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7680c9ca0 0x7fc7680ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:00.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.924+0000 7fc76cb33640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7680cb030 con 0x7fc76815ad70 2023-10-16T10:05:00.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.924+0000 7fc76cb33640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7680cae30 con 0x7fc7680c9ca0 2023-10-16T10:05:00.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.924+0000 7fc76cb33640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680c91c0 con 0x7fc76806a4b0 2023-10-16T10:05:00.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.924+0000 7fc76659c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc76815ad70 0x7fc768069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57344/0 (socket says 172.21.15.84:57344) 2023-10-16T10:05:00.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.924+0000 7fc76659c640 1 -- 172.21.15.84:0/4170728900 learned_addr learned my addr 172.21.15.84:0/4170728900 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:00.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.925+0000 7fc766d9d640 1 --2- 172.21.15.84:0/4170728900 >> v2:172.21.15.157:3300/0 conn(0x7fc7680c9ca0 0x7fc7680ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:00.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.925+0000 7fc75ffff640 1 --2- 172.21.15.84:0/4170728900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76806a4b0 0x7fc7680ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:00.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.925+0000 7fc765d9b640 1 -- 172.21.15.84:0/4170728900 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400474413 0 0) 0x7fc7680cb030 con 0x7fc76815ad70 2023-10-16T10:05:00.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.925+0000 7fc765d9b640 1 -- 172.21.15.84:0/4170728900 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc754003520 con 0x7fc76815ad70 2023-10-16T10:05:00.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.925+0000 7fc765d9b640 1 -- 172.21.15.84:0/4170728900 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 913708671 0 0) 0x7fc754003520 con 0x7fc76815ad70 2023-10-16T10:05:00.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.925+0000 7fc765d9b640 1 -- 172.21.15.84:0/4170728900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76806a4b0 msgr2=0x7fc7680ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:00.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.925+0000 7fc765d9b640 1 --2- 172.21.15.84:0/4170728900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76806a4b0 0x7fc7680ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.925+0000 7fc765d9b640 1 -- 172.21.15.84:0/4170728900 >> v2:172.21.15.157:3300/0 conn(0x7fc7680c9ca0 msgr2=0x7fc7680ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:00.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.925+0000 7fc765d9b640 1 --2- 172.21.15.84:0/4170728900 >> v2:172.21.15.157:3300/0 conn(0x7fc7680c9ca0 0x7fc7680ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.925+0000 7fc765d9b640 1 -- 172.21.15.84:0/4170728900 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7680c94a0 con 0x7fc76815ad70 2023-10-16T10:05:00.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.925+0000 7fc765d9b640 1 -- 172.21.15.84:0/4170728900 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc758001670 con 0x7fc76815ad70 2023-10-16T10:05:00.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.925+0000 7fc765d9b640 1 -- 172.21.15.84:0/4170728900 >> v1:172.21.15.84:6789/0 conn(0x7fc76815ad70 legacy=0x7fc768069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:00.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.925+0000 7fc765d9b640 1 --2- 172.21.15.84:0/4170728900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76806a4b0 0x7fc754003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:00.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.925+0000 7fc765d9b640 1 --2- 172.21.15.84:0/4170728900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc754003ef0 0x7fc7540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:00.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.925+0000 7fc765d9b640 1 --2- 172.21.15.84:0/4170728900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc754004d00 0x7fc754005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:00.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.925+0000 7fc765d9b640 1 -- 172.21.15.84:0/4170728900 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680cae30 con 0x7fc754004d00 2023-10-16T10:05:00.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.925+0000 7fc765d9b640 1 -- 172.21.15.84:0/4170728900 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680c91c0 con 0x7fc754003ef0 2023-10-16T10:05:00.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.925+0000 7fc765d9b640 1 -- 172.21.15.84:0/4170728900 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75400b980 con 0x7fc76806a4b0 2023-10-16T10:05:00.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.926+0000 7fc766d9d640 1 --2- 172.21.15.84:0/4170728900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc754004d00 0x7fc754005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:00.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.926+0000 7fc75ffff640 1 --2- 172.21.15.84:0/4170728900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc754003ef0 0x7fc7540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:00.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.926+0000 7fc76659c640 1 --2- 172.21.15.84:0/4170728900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76806a4b0 0x7fc754003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:00.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.926+0000 7fc766d9d640 1 -- 172.21.15.84:0/4170728900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76806a4b0 msgr2=0x7fc754003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:00.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.926+0000 7fc766d9d640 1 --2- 172.21.15.84:0/4170728900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76806a4b0 0x7fc754003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.926+0000 7fc766d9d640 1 -- 172.21.15.84:0/4170728900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc754003ef0 msgr2=0x7fc7540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:00.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.926+0000 7fc766d9d640 1 --2- 172.21.15.84:0/4170728900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc754003ef0 0x7fc7540047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.926+0000 7fc766d9d640 1 -- 172.21.15.84:0/4170728900 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc754003520 con 0x7fc754004d00 2023-10-16T10:05:00.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.927+0000 7fc766d9d640 1 --2- 172.21.15.84:0/4170728900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc754004d00 0x7fc754005180 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc760009fc0 tx=0x7fc7540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e940d9e59c432e3c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:00.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.927+0000 7fc765d9b640 1 -- 172.21.15.84:0/4170728900 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76002cc40 con 0x7fc754004d00 2023-10-16T10:05:00.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.927+0000 7fc765d9b640 1 -- 172.21.15.84:0/4170728900 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76000f040 con 0x7fc754004d00 2023-10-16T10:05:00.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.927+0000 7fc765d9b640 1 -- 172.21.15.84:0/4170728900 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc760034440 con 0x7fc754004d00 2023-10-16T10:05:00.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.928+0000 7fc76cb33640 1 -- 172.21.15.84:0/4170728900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc754004d00 msgr2=0x7fc754005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:00.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.928+0000 7fc76cb33640 1 --2- 172.21.15.84:0/4170728900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc754004d00 0x7fc754005180 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc760009fc0 tx=0x7fc7540057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.928+0000 7fc766d9d640 1 -- 172.21.15.84:0/4170728900 reap_dead start 2023-10-16T10:05:00.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.928+0000 7fc76cb33640 1 -- 172.21.15.84:0/4170728900 shutdown_connections 2023-10-16T10:05:00.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.928+0000 7fc76cb33640 1 -- 172.21.15.84:0/4170728900 >> 172.21.15.84:0/4170728900 conn(0x7fc7680ce9b0 msgr2=0x7fc7680cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:00.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.928+0000 7fc76cb33640 1 -- 172.21.15.84:0/4170728900 shutdown_connections 2023-10-16T10:05:00.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.928+0000 7fc76cb33640 1 -- 172.21.15.84:0/4170728900 wait complete. 2023-10-16T10:05:00.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.930+0000 7fc76cb33640 1 Processor -- start 2023-10-16T10:05:00.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.930+0000 7fc76cb33640 1 -- start start 2023-10-16T10:05:00.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.931+0000 7fc76cb33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc754004d00 0x7fc7680b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:00.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.931+0000 7fc76cb33640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7680b1b10 0x7fc76814e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:00.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.931+0000 7fc76cb33640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76814ec10 0x7fc76814f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:00.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.931+0000 7fc76cb33640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680b1f90 con 0x7fc7680b1b10 2023-10-16T10:05:00.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.931+0000 7fc76cb33640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680af4c0 con 0x7fc754004d00 2023-10-16T10:05:00.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.931+0000 7fc76cb33640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76814c7c0 con 0x7fc76814ec10 2023-10-16T10:05:00.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.932+0000 7fc75ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7680b1b10 0x7fc76814e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:00.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.932+0000 7fc75ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7680b1b10 0x7fc76814e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50314/0 (socket says 172.21.15.84:50314) 2023-10-16T10:05:00.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.932+0000 7fc75ffff640 1 -- 172.21.15.84:0/2588518067 learned_addr learned my addr 172.21.15.84:0/2588518067 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:00.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.932+0000 7fc76659c640 1 --2- 172.21.15.84:0/2588518067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc754004d00 0x7fc7680b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:00.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.932+0000 7fc766d9d640 1 --2- 172.21.15.84:0/2588518067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76814ec10 0x7fc76814f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:00.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.932+0000 7fc75ffff640 1 -- 172.21.15.84:0/2588518067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76814ec10 msgr2=0x7fc76814f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:00.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.932+0000 7fc75ffff640 1 --2- 172.21.15.84:0/2588518067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76814ec10 0x7fc76814f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.932+0000 7fc75ffff640 1 -- 172.21.15.84:0/2588518067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc754004d00 msgr2=0x7fc7680b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:00.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.932+0000 7fc75ffff640 1 --2- 172.21.15.84:0/2588518067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc754004d00 0x7fc7680b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:00.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.932+0000 7fc75ffff640 1 -- 172.21.15.84:0/2588518067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76814caa0 con 0x7fc7680b1b10 2023-10-16T10:05:00.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.932+0000 7fc76659c640 1 --2- 172.21.15.84:0/2588518067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc754004d00 0x7fc7680b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:00.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.932+0000 7fc75ffff640 1 --2- 172.21.15.84:0/2588518067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7680b1b10 0x7fc76814e6d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc750011540 tx=0x7fc750011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:00.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.932+0000 7fc75effd640 1 -- 172.21.15.84:0/2588518067 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc750015930 con 0x7fc7680b1b10 2023-10-16T10:05:00.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.933+0000 7fc75effd640 1 -- 172.21.15.84:0/2588518067 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc750015a90 con 0x7fc7680b1b10 2023-10-16T10:05:00.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.933+0000 7fc76cb33640 1 -- 172.21.15.84:0/2588518067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76814cd80 con 0x7fc7680b1b10 2023-10-16T10:05:00.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.933+0000 7fc75effd640 1 -- 172.21.15.84:0/2588518067 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc750015e60 con 0x7fc7680b1b10 2023-10-16T10:05:00.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.933+0000 7fc76cb33640 1 -- 172.21.15.84:0/2588518067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76814d480 con 0x7fc7680b1b10 2023-10-16T10:05:00.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.934+0000 7fc76cb33640 1 -- 172.21.15.84:0/2588518067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76804e160 con 0x7fc7680b1b10 2023-10-16T10:05:00.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.935+0000 7fc75effd640 1 -- 172.21.15.84:0/2588518067 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc75000f460 con 0x7fc7680b1b10 2023-10-16T10:05:00.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.936+0000 7fc75effd640 1 --2- 172.21.15.84:0/2588518067 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc74806af90 0x7fc74806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:00.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.936+0000 7fc75effd640 1 -- 172.21.15.84:0/2588518067 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5631+0+0 (secure 0 0 0) 0x7fc750089f10 con 0x7fc7680b1b10 2023-10-16T10:05:00.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.936+0000 7fc76659c640 1 --2- 172.21.15.84:0/2588518067 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc74806af90 0x7fc74806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:00.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.937+0000 7fc76659c640 1 --2- 172.21.15.84:0/2588518067 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc74806af90 0x7fc74806d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc758001050 tx=0x7fc758001580 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:00.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:00.940+0000 7fc75effd640 1 -- 172.21.15.84:0/2588518067 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc750054ef0 con 0x7fc7680b1b10 2023-10-16T10:05:01.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.128+0000 7fc76cb33640 1 -- 172.21.15.84:0/2588518067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fc7680c9750 con 0x7fc7680b1b10 2023-10-16T10:05:01.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.135+0000 7fc75effd640 1 -- 172.21.15.84:0/2588518067 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v26) v1 ==== 120+0+0 (secure 0 0 0) 0x7fc750058510 con 0x7fc7680b1b10 2023-10-16T10:05:01.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.138+0000 7fc76cb33640 1 -- 172.21.15.84:0/2588518067 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc74806af90 msgr2=0x7fc74806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:01.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.138+0000 7fc76cb33640 1 --2- 172.21.15.84:0/2588518067 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc74806af90 0x7fc74806d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc758001050 tx=0x7fc758001580 comp rx=0 tx=0).stop 2023-10-16T10:05:01.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.138+0000 7fc76cb33640 1 -- 172.21.15.84:0/2588518067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7680b1b10 msgr2=0x7fc76814e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:01.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.138+0000 7fc76cb33640 1 --2- 172.21.15.84:0/2588518067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7680b1b10 0x7fc76814e6d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc750011540 tx=0x7fc750011a00 comp rx=0 tx=0).stop 2023-10-16T10:05:01.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.138+0000 7fc76cb33640 1 -- 172.21.15.84:0/2588518067 shutdown_connections 2023-10-16T10:05:01.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.138+0000 7fc76cb33640 1 --2- 172.21.15.84:0/2588518067 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc74806af90 0x7fc74806d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.138+0000 7fc76cb33640 1 --2- 172.21.15.84:0/2588518067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76814ec10 0x7fc76814f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.138+0000 7fc76cb33640 1 --2- 172.21.15.84:0/2588518067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7680b1b10 0x7fc76814e6d0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.138+0000 7fc76cb33640 1 --2- 172.21.15.84:0/2588518067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc754004d00 0x7fc7680b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.138+0000 7fc76cb33640 1 -- 172.21.15.84:0/2588518067 >> 172.21.15.84:0/2588518067 conn(0x7fc7680ce9b0 msgr2=0x7fc7680b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:01.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.138+0000 7fc76cb33640 1 -- 172.21.15.84:0/2588518067 shutdown_connections 2023-10-16T10:05:01.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.138+0000 7fc76cb33640 1 -- 172.21.15.84:0/2588518067 wait complete. 2023-10-16T10:05:01.157 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T10:05:01.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.364+0000 7fb47293e640 1 Processor -- start 2023-10-16T10:05:01.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.364+0000 7fb47293e640 1 -- start start 2023-10-16T10:05:01.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.365+0000 7fb47293e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46c15aaa0 0x7fb46c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:01.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.365+0000 7fb47293e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb46c15ae80 0x7fb46c0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:01.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.365+0000 7fb47293e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46c0707e0 con 0x7fb46c0ccaa0 2023-10-16T10:05:01.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.365+0000 7fb47293e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46c06b7a0 con 0x7fb46c15ae80 2023-10-16T10:05:01.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.365+0000 7fb47293e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c15ef20 con 0x7fb46c15aaa0 2023-10-16T10:05:01.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.366+0000 7fb463fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb46c15ae80 0x7fb46c0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:01.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.366+0000 7fb470edb640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb46c0ccaa0 0x7fb46c070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57358/0 (socket says 172.21.15.84:57358) 2023-10-16T10:05:01.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.366+0000 7fb470edb640 1 -- 172.21.15.84:0/2206705311 learned_addr learned my addr 172.21.15.84:0/2206705311 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:01.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.366+0000 7fb46bfff640 1 --2- 172.21.15.84:0/2206705311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46c15aaa0 0x7fb46c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:01.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.366+0000 7fb463fff640 1 -- 172.21.15.84:0/2206705311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46c15aaa0 msgr2=0x7fb46c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:01.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.366+0000 7fb463fff640 1 --2- 172.21.15.84:0/2206705311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46c15aaa0 0x7fb46c15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.366+0000 7fb463fff640 1 -- 172.21.15.84:0/2206705311 >> v1:172.21.15.84:6789/0 conn(0x7fb46c0ccaa0 legacy=0x7fb46c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:01.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.366+0000 7fb46b7fe640 1 -- 172.21.15.84:0/2206705311 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473488167 0 0) 0x7fb46c0707e0 con 0x7fb46c0ccaa0 2023-10-16T10:05:01.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.366+0000 7fb463fff640 1 -- 172.21.15.84:0/2206705311 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c15f200 con 0x7fb46c15ae80 2023-10-16T10:05:01.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.366+0000 7fb46bfff640 1 --2- 172.21.15.84:0/2206705311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46c15aaa0 0x7fb46c15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:01.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.367+0000 7fb463fff640 1 --2- 172.21.15.84:0/2206705311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46c15ae80 0x7fb46c0cc560 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb4540097a0 tx=0x7fb45402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6130b44e06e0b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:01.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.368+0000 7fb46b7fe640 1 -- 172.21.15.84:0/2206705311 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454031780 con 0x7fb46c15ae80 2023-10-16T10:05:01.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.368+0000 7fb46b7fe640 1 -- 172.21.15.84:0/2206705311 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4540318e0 con 0x7fb46c15ae80 2023-10-16T10:05:01.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.368+0000 7fb46b7fe640 1 -- 172.21.15.84:0/2206705311 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454031c70 con 0x7fb46c15ae80 2023-10-16T10:05:01.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.368+0000 7fb47293e640 1 -- 172.21.15.84:0/2206705311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46c15ae80 msgr2=0x7fb46c0cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:01.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.368+0000 7fb47293e640 1 --2- 172.21.15.84:0/2206705311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46c15ae80 0x7fb46c0cc560 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb4540097a0 tx=0x7fb45402b820 comp rx=0 tx=0).stop 2023-10-16T10:05:01.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.368+0000 7fb47293e640 1 -- 172.21.15.84:0/2206705311 shutdown_connections 2023-10-16T10:05:01.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.368+0000 7fb47293e640 1 --2- 172.21.15.84:0/2206705311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46c15ae80 0x7fb46c0cc560 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.368+0000 7fb47293e640 1 --2- 172.21.15.84:0/2206705311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46c15aaa0 0x7fb46c15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.368+0000 7fb47293e640 1 -- 172.21.15.84:0/2206705311 >> 172.21.15.84:0/2206705311 conn(0x7fb46c0cb410 msgr2=0x7fb46c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:01.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.368+0000 7fb47293e640 1 -- 172.21.15.84:0/2206705311 shutdown_connections 2023-10-16T10:05:01.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.368+0000 7fb47293e640 1 -- 172.21.15.84:0/2206705311 wait complete. 2023-10-16T10:05:01.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.370+0000 7fb47293e640 1 Processor -- start 2023-10-16T10:05:01.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.370+0000 7fb47293e640 1 -- start start 2023-10-16T10:05:01.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.371+0000 7fb47293e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46c0ccaa0 0x7fb46c15dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:01.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.371+0000 7fb47293e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46c15aaa0 0x7fb46c158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:01.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.371+0000 7fb47293e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb46c15ae80 0x7fb46c158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:01.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.371+0000 7fb47293e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c0c65e0 con 0x7fb46c0ccaa0 2023-10-16T10:05:01.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.371+0000 7fb47293e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c0c67f0 con 0x7fb46c15ae80 2023-10-16T10:05:01.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.371+0000 7fb47293e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c0c6ad0 con 0x7fb46c15aaa0 2023-10-16T10:05:01.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.371+0000 7fb46bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46c0ccaa0 0x7fb46c15dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:01.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.371+0000 7fb46bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46c0ccaa0 0x7fb46c15dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50344/0 (socket says 172.21.15.84:50344) 2023-10-16T10:05:01.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.371+0000 7fb46bfff640 1 -- 172.21.15.84:0/274079674 learned_addr learned my addr 172.21.15.84:0/274079674 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:01.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.372+0000 7fb46bfff640 1 -- 172.21.15.84:0/274079674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46c15aaa0 msgr2=0x7fb46c158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:01.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.372+0000 7fb463fff640 1 --2- 172.21.15.84:0/274079674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46c15aaa0 0x7fb46c158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:01.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.372+0000 7fb46bfff640 1 --2- 172.21.15.84:0/274079674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46c15aaa0 0x7fb46c158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.372+0000 7fb46bfff640 1 -- 172.21.15.84:0/274079674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb46c15ae80 msgr2=0x7fb46c158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:01.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.372+0000 7fb46bfff640 1 --2- 172.21.15.84:0/274079674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb46c15ae80 0x7fb46c158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.372+0000 7fb46bfff640 1 -- 172.21.15.84:0/274079674 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb454009400 con 0x7fb46c0ccaa0 2023-10-16T10:05:01.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.372+0000 7fb46bfff640 1 --2- 172.21.15.84:0/274079674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46c0ccaa0 0x7fb46c15dba0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00c7f0 tx=0x7fb45c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:01.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.372+0000 7fb4697fa640 1 -- 172.21.15.84:0/274079674 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c007a30 con 0x7fb46c0ccaa0 2023-10-16T10:05:01.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.372+0000 7fb4697fa640 1 -- 172.21.15.84:0/274079674 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c007b90 con 0x7fb46c0ccaa0 2023-10-16T10:05:01.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.372+0000 7fb4697fa640 1 -- 172.21.15.84:0/274079674 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c00ce40 con 0x7fb46c0ccaa0 2023-10-16T10:05:01.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.372+0000 7fb47293e640 1 -- 172.21.15.84:0/274079674 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46c0c6db0 con 0x7fb46c0ccaa0 2023-10-16T10:05:01.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.373+0000 7fb47293e640 1 -- 172.21.15.84:0/274079674 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46c0c7370 con 0x7fb46c0ccaa0 2023-10-16T10:05:01.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.374+0000 7fb47293e640 1 -- 172.21.15.84:0/274079674 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46c04e160 con 0x7fb46c0ccaa0 2023-10-16T10:05:01.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.375+0000 7fb4697fa640 1 -- 172.21.15.84:0/274079674 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb45c0113f0 con 0x7fb46c0ccaa0 2023-10-16T10:05:01.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.375+0000 7fb4697fa640 1 --2- 172.21.15.84:0/274079674 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fb44c06af90 0x7fb44c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:01.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.375+0000 7fb4697fa640 1 -- 172.21.15.84:0/274079674 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5631+0+0 (secure 0 0 0) 0x7fb45c089a90 con 0x7fb46c0ccaa0 2023-10-16T10:05:01.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.376+0000 7fb463fff640 1 --2- 172.21.15.84:0/274079674 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fb44c06af90 0x7fb44c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:01.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.376+0000 7fb463fff640 1 --2- 172.21.15.84:0/274079674 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fb44c06af90 0x7fb44c06d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb454036000 tx=0x7fb45403d040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:01.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.380+0000 7fb4697fa640 1 -- 172.21.15.84:0/274079674 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb45c0548d0 con 0x7fb46c0ccaa0 2023-10-16T10:05:01.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.571+0000 7fb47293e640 1 -- 172.21.15.84:0/274079674 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fb46c0cdc60 con 0x7fb46c0ccaa0 2023-10-16T10:05:01.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.578+0000 7fb4697fa640 1 -- 172.21.15.84:0/274079674 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v27) v1 ==== 115+0+0 (secure 0 0 0) 0x7fb45c01f0b0 con 0x7fb46c0ccaa0 2023-10-16T10:05:01.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.581+0000 7fb47293e640 1 -- 172.21.15.84:0/274079674 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fb44c06af90 msgr2=0x7fb44c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:01.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.581+0000 7fb47293e640 1 --2- 172.21.15.84:0/274079674 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fb44c06af90 0x7fb44c06d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb454036000 tx=0x7fb45403d040 comp rx=0 tx=0).stop 2023-10-16T10:05:01.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.581+0000 7fb47293e640 1 -- 172.21.15.84:0/274079674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46c0ccaa0 msgr2=0x7fb46c15dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:01.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.581+0000 7fb47293e640 1 --2- 172.21.15.84:0/274079674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46c0ccaa0 0x7fb46c15dba0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00c7f0 tx=0x7fb45c00cbf0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.581+0000 7fb47293e640 1 -- 172.21.15.84:0/274079674 shutdown_connections 2023-10-16T10:05:01.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.581+0000 7fb47293e640 1 --2- 172.21.15.84:0/274079674 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fb44c06af90 0x7fb44c06d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.581+0000 7fb47293e640 1 --2- 172.21.15.84:0/274079674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb46c15ae80 0x7fb46c158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.581+0000 7fb47293e640 1 --2- 172.21.15.84:0/274079674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb46c15aaa0 0x7fb46c158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.581+0000 7fb47293e640 1 --2- 172.21.15.84:0/274079674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46c0ccaa0 0x7fb46c15dba0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.581+0000 7fb47293e640 1 -- 172.21.15.84:0/274079674 >> 172.21.15.84:0/274079674 conn(0x7fb46c0cb410 msgr2=0x7fb46c0b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:01.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.582+0000 7fb47293e640 1 -- 172.21.15.84:0/274079674 shutdown_connections 2023-10-16T10:05:01.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.582+0000 7fb47293e640 1 -- 172.21.15.84:0/274079674 wait complete. 2023-10-16T10:05:01.601 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-10-16T10:05:01.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.802+0000 7f5d979c9640 1 Processor -- start 2023-10-16T10:05:01.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.802+0000 7f5d979c9640 1 -- start start 2023-10-16T10:05:01.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.803+0000 7f5d979c9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5d90159f00 0x7f5d900b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:01.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.803+0000 7f5d979c9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9006f7f0 con 0x7f5d90159b20 2023-10-16T10:05:01.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.803+0000 7f5d979c9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9006c5e0 con 0x7f5d9015d880 2023-10-16T10:05:01.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.803+0000 7f5d979c9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9006c8c0 con 0x7f5d90159f00 2023-10-16T10:05:01.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.804+0000 7f5d95765640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f5d90159b20 0x7f5d9006bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57372/0 (socket says 172.21.15.84:57372) 2023-10-16T10:05:01.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.804+0000 7f5d95765640 1 -- 172.21.15.84:0/2917205729 learned_addr learned my addr 172.21.15.84:0/2917205729 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:01.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.804+0000 7f5d94f64640 1 --2- 172.21.15.84:0/2917205729 >> v2:172.21.15.157:3300/0 conn(0x7f5d90159f00 0x7f5d900b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:01.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.804+0000 7f5d8ffff640 1 -- 172.21.15.84:0/2917205729 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027833936 0 0) 0x7f5d9006f7f0 con 0x7f5d90159b20 2023-10-16T10:05:01.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.804+0000 7f5d8ffff640 1 -- 172.21.15.84:0/2917205729 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d78003520 con 0x7f5d90159b20 2023-10-16T10:05:01.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.804+0000 7f5d8ffff640 1 -- 172.21.15.84:0/2917205729 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1212116940 0 0) 0x7f5d78003520 con 0x7f5d90159b20 2023-10-16T10:05:01.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.804+0000 7f5d8ffff640 1 -- 172.21.15.84:0/2917205729 >> v2:172.21.15.157:3300/0 conn(0x7f5d90159f00 msgr2=0x7f5d900b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:01.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.804+0000 7f5d8ffff640 1 --2- 172.21.15.84:0/2917205729 >> v2:172.21.15.157:3300/0 conn(0x7f5d90159f00 0x7f5d900b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.804+0000 7f5d8ffff640 1 -- 172.21.15.84:0/2917205729 >> v1:172.21.15.157:6789/0 conn(0x7f5d9015d880 legacy=0x7f5d9006fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:01.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.804+0000 7f5d8ffff640 1 -- 172.21.15.84:0/2917205729 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9006cba0 con 0x7f5d90159b20 2023-10-16T10:05:01.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.804+0000 7f5d8ffff640 1 -- 172.21.15.84:0/2917205729 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f5d800019f0 con 0x7f5d90159b20 2023-10-16T10:05:01.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.804+0000 7f5d8ffff640 1 -- 172.21.15.84:0/2917205729 >> v1:172.21.15.84:6789/0 conn(0x7f5d90159b20 legacy=0x7f5d9006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:01.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.805+0000 7f5d8ffff640 1 --2- 172.21.15.84:0/2917205729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d78003970 0x7f5d78003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:01.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.805+0000 7f5d8ffff640 1 --2- 172.21.15.84:0/2917205729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d780047c0 0x7f5d78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:01.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.805+0000 7f5d8ffff640 1 --2- 172.21.15.84:0/2917205729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d78005180 0x7f5d7800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:01.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.805+0000 7f5d8ffff640 1 -- 172.21.15.84:0/2917205729 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9006c8c0 con 0x7f5d780047c0 2023-10-16T10:05:01.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.805+0000 7f5d8ffff640 1 -- 172.21.15.84:0/2917205729 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7800c080 con 0x7f5d78005180 2023-10-16T10:05:01.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.805+0000 7f5d8ffff640 1 -- 172.21.15.84:0/2917205729 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7800c240 con 0x7f5d78003970 2023-10-16T10:05:01.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.805+0000 7f5d94f64640 1 --2- 172.21.15.84:0/2917205729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d780047c0 0x7f5d78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:01.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.805+0000 7f5d94f64640 1 -- 172.21.15.84:0/2917205729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d78003970 msgr2=0x7f5d78003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:01.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.805+0000 7f5d94f64640 1 --2- 172.21.15.84:0/2917205729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d78003970 0x7f5d78003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.805+0000 7f5d94f64640 1 -- 172.21.15.84:0/2917205729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d78005180 msgr2=0x7f5d7800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:01.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.805+0000 7f5d95f66640 1 --2- 172.21.15.84:0/2917205729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d78005180 0x7f5d7800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:01.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.805+0000 7f5d94f64640 1 --2- 172.21.15.84:0/2917205729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d78005180 0x7f5d7800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.805+0000 7f5d94f64640 1 -- 172.21.15.84:0/2917205729 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9006cba0 con 0x7f5d780047c0 2023-10-16T10:05:01.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.805+0000 7f5d95f66640 1 --2- 172.21.15.84:0/2917205729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d78005180 0x7f5d7800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:01.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.805+0000 7f5d95f66640 1 -- 172.21.15.84:0/2917205729 reap_dead start 2023-10-16T10:05:01.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.806+0000 7f5d94f64640 1 --2- 172.21.15.84:0/2917205729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d780047c0 0x7f5d78004c40 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400ba00 tx=0x7f5d780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b654c8e417777a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:01.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.807+0000 7f5d8ffff640 1 -- 172.21.15.84:0/2917205729 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84032620 con 0x7f5d780047c0 2023-10-16T10:05:01.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.807+0000 7f5d8ffff640 1 -- 172.21.15.84:0/2917205729 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d84032780 con 0x7f5d780047c0 2023-10-16T10:05:01.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.807+0000 7f5d8ffff640 1 -- 172.21.15.84:0/2917205729 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84032b10 con 0x7f5d780047c0 2023-10-16T10:05:01.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.807+0000 7f5d979c9640 1 -- 172.21.15.84:0/2917205729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d780047c0 msgr2=0x7f5d78004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:01.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.807+0000 7f5d979c9640 1 --2- 172.21.15.84:0/2917205729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d780047c0 0x7f5d78004c40 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400ba00 tx=0x7f5d780057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.807+0000 7f5d979c9640 1 -- 172.21.15.84:0/2917205729 shutdown_connections 2023-10-16T10:05:01.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.807+0000 7f5d979c9640 1 --2- 172.21.15.84:0/2917205729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d780047c0 0x7f5d78004c40 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.807+0000 7f5d979c9640 1 -- 172.21.15.84:0/2917205729 >> 172.21.15.84:0/2917205729 conn(0x7f5d900cb410 msgr2=0x7f5d900cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:01.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.807+0000 7f5d979c9640 1 -- 172.21.15.84:0/2917205729 shutdown_connections 2023-10-16T10:05:01.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.807+0000 7f5d979c9640 1 -- 172.21.15.84:0/2917205729 wait complete. 2023-10-16T10:05:01.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.809+0000 7f5d979c9640 1 Processor -- start 2023-10-16T10:05:01.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.809+0000 7f5d979c9640 1 -- start start 2023-10-16T10:05:01.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.810+0000 7f5d979c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d780047c0 0x7f5d90157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:01.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.810+0000 7f5d979c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d90153350 0x7f5d901581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:01.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.810+0000 7f5d979c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d9015bdc0 0x7f5d9015c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:01.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.810+0000 7f5d979c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d901f35d0 con 0x7f5d90153350 2023-10-16T10:05:01.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.810+0000 7f5d979c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d901f2d10 con 0x7f5d780047c0 2023-10-16T10:05:01.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.810+0000 7f5d979c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d901f2f80 con 0x7f5d9015bdc0 2023-10-16T10:05:01.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.810+0000 7f5d94f64640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d90153350 0x7f5d901581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:01.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.810+0000 7f5d94f64640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d90153350 0x7f5d901581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50370/0 (socket says 172.21.15.84:50370) 2023-10-16T10:05:01.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.810+0000 7f5d94f64640 1 -- 172.21.15.84:0/3753415941 learned_addr learned my addr 172.21.15.84:0/3753415941 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:01.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.811+0000 7f5d94f64640 1 -- 172.21.15.84:0/3753415941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d9015bdc0 msgr2=0x7f5d9015c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:01.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.811+0000 7f5d95765640 1 --2- 172.21.15.84:0/3753415941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d780047c0 0x7f5d90157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:01.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.811+0000 7f5d95f66640 1 --2- 172.21.15.84:0/3753415941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d9015bdc0 0x7f5d9015c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:01.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.811+0000 7f5d94f64640 1 --2- 172.21.15.84:0/3753415941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d9015bdc0 0x7f5d9015c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.811+0000 7f5d94f64640 1 -- 172.21.15.84:0/3753415941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d780047c0 msgr2=0x7f5d90157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:01.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.811+0000 7f5d94f64640 1 --2- 172.21.15.84:0/3753415941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d780047c0 0x7f5d90157c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:01.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.811+0000 7f5d94f64640 1 -- 172.21.15.84:0/3753415941 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d901f3260 con 0x7f5d90153350 2023-10-16T10:05:01.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.811+0000 7f5d94f64640 1 --2- 172.21.15.84:0/3753415941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d90153350 0x7f5d901581d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400bb20 tx=0x7f5d84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:01.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.811+0000 7f5d8dffb640 1 -- 172.21.15.84:0/3753415941 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84004020 con 0x7f5d90153350 2023-10-16T10:05:01.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.811+0000 7f5d8dffb640 1 -- 172.21.15.84:0/3753415941 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d840041a0 con 0x7f5d90153350 2023-10-16T10:05:01.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.811+0000 7f5d8dffb640 1 -- 172.21.15.84:0/3753415941 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84003610 con 0x7f5d90153350 2023-10-16T10:05:01.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.811+0000 7f5d979c9640 1 -- 172.21.15.84:0/3753415941 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9015c790 con 0x7f5d90153350 2023-10-16T10:05:01.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.812+0000 7f5d979c9640 1 -- 172.21.15.84:0/3753415941 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9015cad0 con 0x7f5d90153350 2023-10-16T10:05:01.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.813+0000 7f5d979c9640 1 -- 172.21.15.84:0/3753415941 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9004e160 con 0x7f5d90153350 2023-10-16T10:05:01.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.814+0000 7f5d8dffb640 1 -- 172.21.15.84:0/3753415941 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5d84003770 con 0x7f5d90153350 2023-10-16T10:05:01.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.815+0000 7f5d8dffb640 1 --2- 172.21.15.84:0/3753415941 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f5d7c06afc0 0x7f5d7c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:01.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.815+0000 7f5d8dffb640 1 -- 172.21.15.84:0/3753415941 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f5d840ab640 con 0x7f5d90153350 2023-10-16T10:05:01.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.815+0000 7f5d95765640 1 --2- 172.21.15.84:0/3753415941 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f5d7c06afc0 0x7f5d7c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:01.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.815+0000 7f5d95765640 1 --2- 172.21.15.84:0/3753415941 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f5d7c06afc0 0x7f5d7c06d480 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5d80001330 tx=0x7f5d80009040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:01.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:01.819+0000 7f5d8dffb640 1 -- 172.21.15.84:0/3753415941 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5d84076640 con 0x7f5d90153350 2023-10-16T10:05:02.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.086+0000 7f5d979c9640 1 -- 172.21.15.84:0/3753415941 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f5d900b10b0 con 0x7f5d90153350 2023-10-16T10:05:02.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.266+0000 7f5d8dffb640 1 -- 172.21.15.84:0/3753415941 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v591) v1 ==== 156+0+0 (secure 0 0 0) 0x7f5d84079c60 con 0x7f5d90153350 2023-10-16T10:05:02.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.270+0000 7f5d979c9640 1 -- 172.21.15.84:0/3753415941 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f5d7c06afc0 msgr2=0x7f5d7c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:02.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.270+0000 7f5d979c9640 1 --2- 172.21.15.84:0/3753415941 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f5d7c06afc0 0x7f5d7c06d480 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5d80001330 tx=0x7f5d80009040 comp rx=0 tx=0).stop 2023-10-16T10:05:02.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.270+0000 7f5d979c9640 1 -- 172.21.15.84:0/3753415941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d90153350 msgr2=0x7f5d901581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:02.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.270+0000 7f5d979c9640 1 --2- 172.21.15.84:0/3753415941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d90153350 0x7f5d901581d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400bb20 tx=0x7f5d84004730 comp rx=0 tx=0).stop 2023-10-16T10:05:02.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.270+0000 7f5d979c9640 1 -- 172.21.15.84:0/3753415941 shutdown_connections 2023-10-16T10:05:02.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.270+0000 7f5d979c9640 1 --2- 172.21.15.84:0/3753415941 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f5d7c06afc0 0x7f5d7c06d480 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.270+0000 7f5d979c9640 1 --2- 172.21.15.84:0/3753415941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d9015bdc0 0x7f5d9015c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.270+0000 7f5d979c9640 1 --2- 172.21.15.84:0/3753415941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d90153350 0x7f5d901581d0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.270+0000 7f5d979c9640 1 --2- 172.21.15.84:0/3753415941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d780047c0 0x7f5d90157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.270+0000 7f5d979c9640 1 -- 172.21.15.84:0/3753415941 >> 172.21.15.84:0/3753415941 conn(0x7f5d900cb410 msgr2=0x7f5d900cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:02.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.270+0000 7f5d979c9640 1 -- 172.21.15.84:0/3753415941 shutdown_connections 2023-10-16T10:05:02.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.270+0000 7f5d979c9640 1 -- 172.21.15.84:0/3753415941 wait complete. 2023-10-16T10:05:02.292 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2023-10-16T10:05:02.293 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2023-10-16T10:05:02.293 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T10:05:02.294 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:05:02.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.509+0000 7f6354edf640 1 Processor -- start 2023-10-16T10:05:02.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.509+0000 7f6354edf640 1 -- start start 2023-10-16T10:05:02.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.511+0000 7f6354edf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f63500ad970 0x7f63500cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:02.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.511+0000 7f6354edf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f63500ccfb0 0x7f63500cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:02.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.511+0000 7f6354edf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635015b9d0 con 0x7f635006c040 2023-10-16T10:05:02.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.511+0000 7f6354edf640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f635015bcb0 con 0x7f63500ccfb0 2023-10-16T10:05:02.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.511+0000 7f6354edf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f635015bf90 con 0x7f63500ad970 2023-10-16T10:05:02.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.511+0000 7f634f7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f635006c040 0x7f635006c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57386/0 (socket says 172.21.15.84:57386) 2023-10-16T10:05:02.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.511+0000 7f634f7fe640 1 -- 172.21.15.84:0/265768998 learned_addr learned my addr 172.21.15.84:0/265768998 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:02.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.511+0000 7f634ffff640 1 --2- 172.21.15.84:0/265768998 >> v2:172.21.15.84:3300/0 conn(0x7f63500ccfb0 0x7f63500cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:02.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.512+0000 7f6347fff640 1 --2- 172.21.15.84:0/265768998 >> v2:172.21.15.157:3300/0 conn(0x7f63500ad970 0x7f63500cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:02.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.512+0000 7f634ffff640 1 -- 172.21.15.84:0/265768998 >> v2:172.21.15.157:3300/0 conn(0x7f63500ad970 msgr2=0x7f63500cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:02.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.512+0000 7f634ffff640 1 --2- 172.21.15.84:0/265768998 >> v2:172.21.15.157:3300/0 conn(0x7f63500ad970 0x7f63500cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.512+0000 7f634ffff640 1 -- 172.21.15.84:0/265768998 >> v1:172.21.15.84:6789/0 conn(0x7f635006c040 legacy=0x7f635006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:02.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.512+0000 7f634ffff640 1 -- 172.21.15.84:0/265768998 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63500cc270 con 0x7f63500ccfb0 2023-10-16T10:05:02.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.513+0000 7f634ffff640 1 --2- 172.21.15.84:0/265768998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63500ccfb0 0x7f63500cd460 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f6348009d90 tx=0x7f634802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774bbbf83d1641f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:02.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.513+0000 7f634effd640 1 -- 172.21.15.84:0/265768998 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634802cc60 con 0x7f63500ccfb0 2023-10-16T10:05:02.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.513+0000 7f634effd640 1 -- 172.21.15.84:0/265768998 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634802cdc0 con 0x7f63500ccfb0 2023-10-16T10:05:02.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.513+0000 7f634effd640 1 -- 172.21.15.84:0/265768998 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6348034660 con 0x7f63500ccfb0 2023-10-16T10:05:02.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.514+0000 7f6354edf640 1 -- 172.21.15.84:0/265768998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63500ccfb0 msgr2=0x7f63500cd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:02.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.514+0000 7f6354edf640 1 --2- 172.21.15.84:0/265768998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63500ccfb0 0x7f63500cd460 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f6348009d90 tx=0x7f634802beb0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.514+0000 7f6354edf640 1 -- 172.21.15.84:0/265768998 shutdown_connections 2023-10-16T10:05:02.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.514+0000 7f6354edf640 1 --2- 172.21.15.84:0/265768998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63500ccfb0 0x7f63500cd460 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.514+0000 7f6354edf640 1 --2- 172.21.15.84:0/265768998 >> v2:172.21.15.157:3300/0 conn(0x7f63500ad970 0x7f63500cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.514+0000 7f6354edf640 1 -- 172.21.15.84:0/265768998 >> 172.21.15.84:0/265768998 conn(0x7f63500ce6a0 msgr2=0x7f63500ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:02.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.514+0000 7f6354edf640 1 -- 172.21.15.84:0/265768998 shutdown_connections 2023-10-16T10:05:02.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.514+0000 7f6354edf640 1 -- 172.21.15.84:0/265768998 wait complete. 2023-10-16T10:05:02.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.516+0000 7f6354edf640 1 Processor -- start 2023-10-16T10:05:02.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.516+0000 7f6354edf640 1 -- start start 2023-10-16T10:05:02.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.517+0000 7f6354edf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635006c040 0x7f63500b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:02.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.517+0000 7f6354edf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63500ad970 0x7f63500ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:02.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.517+0000 7f6354edf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63500ccfb0 0x7f63500c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:02.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.517+0000 7f6354edf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63500af2f0 con 0x7f63500ccfb0 2023-10-16T10:05:02.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.517+0000 7f6354edf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63501e4620 con 0x7f635006c040 2023-10-16T10:05:02.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.517+0000 7f6354edf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63500aea50 con 0x7f63500ad970 2023-10-16T10:05:02.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.517+0000 7f634ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63500ccfb0 0x7f63500c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:02.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.517+0000 7f634ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63500ccfb0 0x7f63500c7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50388/0 (socket says 172.21.15.84:50388) 2023-10-16T10:05:02.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.517+0000 7f634ffff640 1 -- 172.21.15.84:0/1781572215 learned_addr learned my addr 172.21.15.84:0/1781572215 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:02.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.517+0000 7f634f7fe640 1 --2- 172.21.15.84:0/1781572215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635006c040 0x7f63500b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:02.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.517+0000 7f6347fff640 1 --2- 172.21.15.84:0/1781572215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63500ad970 0x7f63500ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:02.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.517+0000 7f634ffff640 1 -- 172.21.15.84:0/1781572215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63500ad970 msgr2=0x7f63500ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:02.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.517+0000 7f634ffff640 1 --2- 172.21.15.84:0/1781572215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63500ad970 0x7f63500ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.517+0000 7f634ffff640 1 -- 172.21.15.84:0/1781572215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635006c040 msgr2=0x7f63500b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:02.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.517+0000 7f634ffff640 1 --2- 172.21.15.84:0/1781572215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635006c040 0x7f63500b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.517+0000 7f634ffff640 1 -- 172.21.15.84:0/1781572215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6348009a70 con 0x7f63500ccfb0 2023-10-16T10:05:02.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.517+0000 7f634f7fe640 1 --2- 172.21.15.84:0/1781572215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635006c040 0x7f63500b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:02.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.518+0000 7f634ffff640 1 --2- 172.21.15.84:0/1781572215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63500ccfb0 0x7f63500c7620 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6348009d90 tx=0x7f6348004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:02.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.518+0000 7f634cff9640 1 -- 172.21.15.84:0/1781572215 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6348040650 con 0x7f63500ccfb0 2023-10-16T10:05:02.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.518+0000 7f634cff9640 1 -- 172.21.15.84:0/1781572215 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63480407d0 con 0x7f63500ccfb0 2023-10-16T10:05:02.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.518+0000 7f634cff9640 1 -- 172.21.15.84:0/1781572215 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6348040b80 con 0x7f63500ccfb0 2023-10-16T10:05:02.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.518+0000 7f6354edf640 1 -- 172.21.15.84:0/1781572215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63500aed30 con 0x7f63500ccfb0 2023-10-16T10:05:02.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.518+0000 7f6354edf640 1 -- 172.21.15.84:0/1781572215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63500ae640 con 0x7f63500ccfb0 2023-10-16T10:05:02.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.520+0000 7f6354edf640 1 -- 172.21.15.84:0/1781572215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635006b330 con 0x7f63500ccfb0 2023-10-16T10:05:02.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.520+0000 7f634cff9640 1 -- 172.21.15.84:0/1781572215 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6348040ce0 con 0x7f63500ccfb0 2023-10-16T10:05:02.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.521+0000 7f634cff9640 1 --2- 172.21.15.84:0/1781572215 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f633006afc0 0x7f633006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:02.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.521+0000 7f634cff9640 1 -- 172.21.15.84:0/1781572215 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f63480ac600 con 0x7f63500ccfb0 2023-10-16T10:05:02.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.522+0000 7f634f7fe640 1 --2- 172.21.15.84:0/1781572215 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f633006afc0 0x7f633006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:02.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.522+0000 7f634f7fe640 1 --2- 172.21.15.84:0/1781572215 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f633006afc0 0x7f633006d480 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6340005880 tx=0x7f6340009040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:02.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.525+0000 7f634cff9640 1 -- 172.21.15.84:0/1781572215 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6348077680 con 0x7f63500ccfb0 2023-10-16T10:05:02.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.717+0000 7f6354edf640 1 -- 172.21.15.84:0/1781572215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63500ae7d0 con 0x7f63500ccfb0 2023-10-16T10:05:02.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.719+0000 7f634cff9640 1 -- 172.21.15.84:0/1781572215 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+13965 (secure 0 0 0) 0x7f634807aca0 con 0x7f63500ccfb0 2023-10-16T10:05:02.720 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:02.721 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":87,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:05:00.245418+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6835/2669751864":"2023-10-17T10:04:57.109386+0000","172.21.15.84:6832/2103121481":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/3340998984":"2023-10-17T10:04:53.436086+0000","172.21.15.84:6837/2669751864":"2023-10-17T10:04:57.109386+0000","172.21.15.84:0/2855693098":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/1664791074":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/2565413151":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/890477594":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/3177129458":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/1145018607":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/1499920052":"2023-10-17T10:04:47.124608+0000","172.21.15.84:6833/2103121481":"2023-10-17T10:04:53.436086+0000","172.21.15.84:6833/2091947430":"2023-10-17T10:04:47.124608+0000","172.21.15.84:6832/2091947430":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/1204215627":"2023-10-17T10:04:53.436086+0000","172.21.15.84:0/2509405053":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/3991569032":"2023-10-17T10:04:47.124608+0000","172.21.15.84:0/3599228496":"2023-10-17T10:04:47.124608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:05:02.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.721+0000 7f6354edf640 1 -- 172.21.15.84:0/1781572215 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f633006afc0 msgr2=0x7f633006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:02.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.721+0000 7f6354edf640 1 --2- 172.21.15.84:0/1781572215 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f633006afc0 0x7f633006d480 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6340005880 tx=0x7f6340009040 comp rx=0 tx=0).stop 2023-10-16T10:05:02.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.721+0000 7f6354edf640 1 -- 172.21.15.84:0/1781572215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63500ccfb0 msgr2=0x7f63500c7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:02.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.721+0000 7f6354edf640 1 --2- 172.21.15.84:0/1781572215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63500ccfb0 0x7f63500c7620 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6348009d90 tx=0x7f6348004a40 comp rx=0 tx=0).stop 2023-10-16T10:05:02.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.721+0000 7f6354edf640 1 -- 172.21.15.84:0/1781572215 shutdown_connections 2023-10-16T10:05:02.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.722+0000 7f6354edf640 1 --2- 172.21.15.84:0/1781572215 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f633006afc0 0x7f633006d480 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.722+0000 7f6354edf640 1 --2- 172.21.15.84:0/1781572215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63500ccfb0 0x7f63500c7620 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.722+0000 7f6354edf640 1 --2- 172.21.15.84:0/1781572215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63500ad970 0x7f63500ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.722+0000 7f6354edf640 1 --2- 172.21.15.84:0/1781572215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635006c040 0x7f63500b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.722+0000 7f6354edf640 1 -- 172.21.15.84:0/1781572215 >> 172.21.15.84:0/1781572215 conn(0x7f63500ce6a0 msgr2=0x7f635015c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:02.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.722+0000 7f6354edf640 1 -- 172.21.15.84:0/1781572215 shutdown_connections 2023-10-16T10:05:02.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.722+0000 7f6354edf640 1 -- 172.21.15.84:0/1781572215 wait complete. 2023-10-16T10:05:02.741 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:05:02.741 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:05:02.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.962+0000 7f4ddffff640 1 Processor -- start 2023-10-16T10:05:02.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.962+0000 7f4ddffff640 1 -- start start 2023-10-16T10:05:02.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.964+0000 7f4ddffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4de015cbf0 0x7f4de01613b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:02.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.964+0000 7f4ddffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4de015cfd0 0x7f4de00cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:02.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.964+0000 7f4ddffff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4de0160f20 con 0x7f4de00cd3f0 2023-10-16T10:05:02.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.964+0000 7f4ddffff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4de0160910 con 0x7f4de015cfd0 2023-10-16T10:05:02.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.964+0000 7f4ddffff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de00cd870 con 0x7f4de015cbf0 2023-10-16T10:05:02.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.964+0000 7f4ddf7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4de00cd3f0 0x7f4de006c550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57394/0 (socket says 172.21.15.84:57394) 2023-10-16T10:05:02.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.964+0000 7f4ddf7fe640 1 -- 172.21.15.84:0/2043801168 learned_addr learned my addr 172.21.15.84:0/2043801168 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:02.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.965+0000 7f4dd7fff640 1 --2- 172.21.15.84:0/2043801168 >> v2:172.21.15.157:3300/0 conn(0x7f4de015cfd0 0x7f4de00cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:02.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.965+0000 7f4ddeffd640 1 --2- 172.21.15.84:0/2043801168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4de015cbf0 0x7f4de01613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:02.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.965+0000 7f4dde7fc640 1 -- 172.21.15.84:0/2043801168 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542423615 0 0) 0x7f4de0160f20 con 0x7f4de00cd3f0 2023-10-16T10:05:02.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.965+0000 7f4dde7fc640 1 -- 172.21.15.84:0/2043801168 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dcc003520 con 0x7f4de00cd3f0 2023-10-16T10:05:02.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.965+0000 7f4dde7fc640 1 -- 172.21.15.84:0/2043801168 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1190498815 0 0) 0x7f4dcc003520 con 0x7f4de00cd3f0 2023-10-16T10:05:02.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.965+0000 7f4dde7fc640 1 -- 172.21.15.84:0/2043801168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4de015cbf0 msgr2=0x7f4de01613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:02.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.965+0000 7f4dde7fc640 1 --2- 172.21.15.84:0/2043801168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4de015cbf0 0x7f4de01613b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.965+0000 7f4dde7fc640 1 -- 172.21.15.84:0/2043801168 >> v2:172.21.15.157:3300/0 conn(0x7f4de015cfd0 msgr2=0x7f4de00cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:02.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.965+0000 7f4dde7fc640 1 --2- 172.21.15.84:0/2043801168 >> v2:172.21.15.157:3300/0 conn(0x7f4de015cfd0 0x7f4de00cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.965+0000 7f4dde7fc640 1 -- 172.21.15.84:0/2043801168 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de00cdb50 con 0x7f4de00cd3f0 2023-10-16T10:05:02.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.965+0000 7f4dde7fc640 1 -- 172.21.15.84:0/2043801168 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f4dd80019f0 con 0x7f4de00cd3f0 2023-10-16T10:05:02.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.965+0000 7f4dde7fc640 1 -- 172.21.15.84:0/2043801168 >> v1:172.21.15.84:6789/0 conn(0x7f4de00cd3f0 legacy=0x7f4de006c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:02.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.965+0000 7f4dde7fc640 1 --2- 172.21.15.84:0/2043801168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4de015cbf0 0x7f4dcc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:02.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.965+0000 7f4dde7fc640 1 --2- 172.21.15.84:0/2043801168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dcc003ef0 0x7f4dcc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:02.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.965+0000 7f4dde7fc640 1 --2- 172.21.15.84:0/2043801168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dcc004d00 0x7f4dcc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:02.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.965+0000 7f4dde7fc640 1 -- 172.21.15.84:0/2043801168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de0160910 con 0x7f4dcc004d00 2023-10-16T10:05:02.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.965+0000 7f4dde7fc640 1 -- 172.21.15.84:0/2043801168 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de00cd870 con 0x7f4dcc003ef0 2023-10-16T10:05:02.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.965+0000 7f4dde7fc640 1 -- 172.21.15.84:0/2043801168 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc00b980 con 0x7f4de015cbf0 2023-10-16T10:05:02.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.966+0000 7f4ddf7fe640 1 --2- 172.21.15.84:0/2043801168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dcc004d00 0x7f4dcc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:02.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.966+0000 7f4dd7fff640 1 --2- 172.21.15.84:0/2043801168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dcc003ef0 0x7f4dcc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:02.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.966+0000 7f4ddeffd640 1 --2- 172.21.15.84:0/2043801168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4de015cbf0 0x7f4dcc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:02.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.966+0000 7f4ddf7fe640 1 -- 172.21.15.84:0/2043801168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4de015cbf0 msgr2=0x7f4dcc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:02.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.966+0000 7f4ddf7fe640 1 --2- 172.21.15.84:0/2043801168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4de015cbf0 0x7f4dcc003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.966+0000 7f4ddf7fe640 1 -- 172.21.15.84:0/2043801168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dcc003ef0 msgr2=0x7f4dcc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:02.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.966+0000 7f4ddf7fe640 1 --2- 172.21.15.84:0/2043801168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dcc003ef0 0x7f4dcc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.966+0000 7f4ddf7fe640 1 -- 172.21.15.84:0/2043801168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dcc003520 con 0x7f4dcc004d00 2023-10-16T10:05:02.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.966+0000 7f4dd7fff640 1 --2- 172.21.15.84:0/2043801168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dcc003ef0 0x7f4dcc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:02.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.967+0000 7f4ddf7fe640 1 --2- 172.21.15.84:0/2043801168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dcc004d00 0x7f4dcc005180 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800a830 tx=0x7f4dcc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9a5ad97f1dee37 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:02.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.967+0000 7f4dde7fc640 1 -- 172.21.15.84:0/2043801168 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd802cea0 con 0x7f4dcc004d00 2023-10-16T10:05:02.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.967+0000 7f4dde7fc640 1 -- 172.21.15.84:0/2043801168 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd802f070 con 0x7f4dcc004d00 2023-10-16T10:05:02.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.967+0000 7f4dde7fc640 1 -- 172.21.15.84:0/2043801168 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd80348d0 con 0x7f4dcc004d00 2023-10-16T10:05:02.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.968+0000 7f4ddffff640 1 -- 172.21.15.84:0/2043801168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dcc004d00 msgr2=0x7f4dcc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:02.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.968+0000 7f4ddffff640 1 --2- 172.21.15.84:0/2043801168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dcc004d00 0x7f4dcc005180 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800a830 tx=0x7f4dcc0057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.968+0000 7f4ddf7fe640 1 -- 172.21.15.84:0/2043801168 reap_dead start 2023-10-16T10:05:02.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.968+0000 7f4ddffff640 1 -- 172.21.15.84:0/2043801168 shutdown_connections 2023-10-16T10:05:02.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.968+0000 7f4ddffff640 1 -- 172.21.15.84:0/2043801168 >> 172.21.15.84:0/2043801168 conn(0x7f4de00b23b0 msgr2=0x7f4de00b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:02.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.968+0000 7f4ddffff640 1 -- 172.21.15.84:0/2043801168 shutdown_connections 2023-10-16T10:05:02.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.968+0000 7f4ddffff640 1 -- 172.21.15.84:0/2043801168 wait complete. 2023-10-16T10:05:02.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.970+0000 7f4ddffff640 1 Processor -- start 2023-10-16T10:05:02.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.970+0000 7f4ddffff640 1 -- start start 2023-10-16T10:05:02.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.971+0000 7f4ddffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dcc004d00 0x7f4de00b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:02.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.971+0000 7f4ddffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4de0157cf0 0x7f4de0158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:02.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.971+0000 7f4ddffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4de01f54b0 0x7f4de01f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:02.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.971+0000 7f4ddffff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de00ac890 con 0x7f4de01f54b0 2023-10-16T10:05:02.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.971+0000 7f4ddffff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de01586b0 con 0x7f4de0157cf0 2023-10-16T10:05:02.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.971+0000 7f4ddffff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de01f61e0 con 0x7f4dcc004d00 2023-10-16T10:05:02.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.971+0000 7f4ddf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4de01f54b0 0x7f4de01f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:02.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.971+0000 7f4ddf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4de01f54b0 0x7f4de01f5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50404/0 (socket says 172.21.15.84:50404) 2023-10-16T10:05:02.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.971+0000 7f4ddf7fe640 1 -- 172.21.15.84:0/1648091439 learned_addr learned my addr 172.21.15.84:0/1648091439 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:02.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.972+0000 7f4ddeffd640 1 --2- 172.21.15.84:0/1648091439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dcc004d00 0x7f4de00b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:02.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.972+0000 7f4ddf7fe640 1 -- 172.21.15.84:0/1648091439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dcc004d00 msgr2=0x7f4de00b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:02.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.972+0000 7f4ddf7fe640 1 --2- 172.21.15.84:0/1648091439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dcc004d00 0x7f4de00b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.972+0000 7f4ddf7fe640 1 -- 172.21.15.84:0/1648091439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4de0157cf0 msgr2=0x7f4de0158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:02.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.972+0000 7f4ddf7fe640 1 --2- 172.21.15.84:0/1648091439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4de0157cf0 0x7f4de0158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:02.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.972+0000 7f4ddf7fe640 1 -- 172.21.15.84:0/1648091439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de00acaa0 con 0x7f4de01f54b0 2023-10-16T10:05:02.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.972+0000 7f4ddf7fe640 1 --2- 172.21.15.84:0/1648091439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4de01f54b0 0x7f4de01f5940 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f4dd802cbc0 tx=0x7f4dd8001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:02.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.972+0000 7f4dd77fe640 1 -- 172.21.15.84:0/1648091439 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8034840 con 0x7f4de01f54b0 2023-10-16T10:05:02.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.972+0000 7f4dd77fe640 1 -- 172.21.15.84:0/1648091439 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd80349a0 con 0x7f4de01f54b0 2023-10-16T10:05:02.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.972+0000 7f4dd77fe640 1 -- 172.21.15.84:0/1648091439 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8034d30 con 0x7f4de01f54b0 2023-10-16T10:05:02.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.972+0000 7f4ddffff640 1 -- 172.21.15.84:0/1648091439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de00acd80 con 0x7f4de01f54b0 2023-10-16T10:05:02.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.972+0000 7f4ddffff640 1 -- 172.21.15.84:0/1648091439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de01f5e80 con 0x7f4de01f54b0 2023-10-16T10:05:02.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.974+0000 7f4ddffff640 1 -- 172.21.15.84:0/1648091439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de00ad2a0 con 0x7f4de01f54b0 2023-10-16T10:05:02.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.975+0000 7f4dd77fe640 1 -- 172.21.15.84:0/1648091439 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4dd8031030 con 0x7f4de01f54b0 2023-10-16T10:05:02.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.975+0000 7f4dd77fe640 1 --2- 172.21.15.84:0/1648091439 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f4dc006afc0 0x7f4dc006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:02.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.975+0000 7f4dd77fe640 1 -- 172.21.15.84:0/1648091439 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f4dd80aac30 con 0x7f4de01f54b0 2023-10-16T10:05:02.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.976+0000 7f4ddeffd640 1 --2- 172.21.15.84:0/1648091439 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f4dc006afc0 0x7f4dc006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:02.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.976+0000 7f4ddeffd640 1 --2- 172.21.15.84:0/1648091439 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f4dc006afc0 0x7f4dc006d480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4de00afd00 tx=0x7f4dd0008040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:02.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:02.980+0000 7f4dd77fe640 1 -- 172.21.15.84:0/1648091439 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4de00ad2a0 con 0x7f4de01f54b0 2023-10-16T10:05:03.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.173+0000 7f4ddffff640 1 -- 172.21.15.84:0/1648091439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4de00ad2a0 con 0x7f4de01f54b0 2023-10-16T10:05:03.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.173+0000 7f4dd77fe640 1 -- 172.21.15.84:0/1648091439 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4de00ad2a0 con 0x7f4de01f54b0 2023-10-16T10:05:03.175 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:05:03.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.176+0000 7f4ddffff640 1 -- 172.21.15.84:0/1648091439 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f4dc006afc0 msgr2=0x7f4dc006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:03.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.176+0000 7f4ddffff640 1 --2- 172.21.15.84:0/1648091439 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f4dc006afc0 0x7f4dc006d480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4de00afd00 tx=0x7f4dd0008040 comp rx=0 tx=0).stop 2023-10-16T10:05:03.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.176+0000 7f4ddffff640 1 -- 172.21.15.84:0/1648091439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4de01f54b0 msgr2=0x7f4de01f5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:03.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.176+0000 7f4ddffff640 1 --2- 172.21.15.84:0/1648091439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4de01f54b0 0x7f4de01f5940 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f4dd802cbc0 tx=0x7f4dd8001d80 comp rx=0 tx=0).stop 2023-10-16T10:05:03.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.176+0000 7f4ddffff640 1 -- 172.21.15.84:0/1648091439 shutdown_connections 2023-10-16T10:05:03.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.176+0000 7f4ddffff640 1 --2- 172.21.15.84:0/1648091439 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f4dc006afc0 0x7f4dc006d480 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:03.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.176+0000 7f4ddffff640 1 --2- 172.21.15.84:0/1648091439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4de01f54b0 0x7f4de01f5940 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:03.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.176+0000 7f4ddffff640 1 --2- 172.21.15.84:0/1648091439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4de0157cf0 0x7f4de0158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:03.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.176+0000 7f4ddffff640 1 --2- 172.21.15.84:0/1648091439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dcc004d00 0x7f4de00b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:03.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.176+0000 7f4ddffff640 1 -- 172.21.15.84:0/1648091439 >> 172.21.15.84:0/1648091439 conn(0x7f4de00b23b0 msgr2=0x7f4de015ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:03.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.176+0000 7f4ddffff640 1 -- 172.21.15.84:0/1648091439 shutdown_connections 2023-10-16T10:05:03.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.176+0000 7f4ddffff640 1 -- 172.21.15.84:0/1648091439 wait complete. 2023-10-16T10:05:03.196 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-10-16T10:05:03.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.392+0000 7fd5f9c26640 1 Processor -- start 2023-10-16T10:05:03.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.393+0000 7fd5f9c26640 1 -- start start 2023-10-16T10:05:03.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.394+0000 7fd5f9c26640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd5f4159f00 0x7fd5f40b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:03.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.394+0000 7fd5f9c26640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f406f7f0 con 0x7fd5f4159b20 2023-10-16T10:05:03.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.394+0000 7fd5f9c26640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f406c5e0 con 0x7fd5f415d880 2023-10-16T10:05:03.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.394+0000 7fd5f9c26640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f406c8c0 con 0x7fd5f4159f00 2023-10-16T10:05:03.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.394+0000 7fd5f37fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd5f4159b20 0x7fd5f406bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57396/0 (socket says 172.21.15.84:57396) 2023-10-16T10:05:03.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.394+0000 7fd5f37fe640 1 -- 172.21.15.84:0/765373651 learned_addr learned my addr 172.21.15.84:0/765373651 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:03.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.395+0000 7fd5f27fc640 1 -- 172.21.15.84:0/765373651 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767781826 0 0) 0x7fd5f406f7f0 con 0x7fd5f4159b20 2023-10-16T10:05:03.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.395+0000 7fd5f27fc640 1 -- 172.21.15.84:0/765373651 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5e0003520 con 0x7fd5f4159b20 2023-10-16T10:05:03.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.395+0000 7fd5f2ffd640 1 --2- 172.21.15.84:0/765373651 >> v2:172.21.15.157:3300/0 conn(0x7fd5f4159f00 0x7fd5f40b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:03.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.395+0000 7fd5f27fc640 1 -- 172.21.15.84:0/765373651 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4114355001 0 0) 0x7fd5e0003520 con 0x7fd5f4159b20 2023-10-16T10:05:03.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.395+0000 7fd5f27fc640 1 -- 172.21.15.84:0/765373651 >> v2:172.21.15.157:3300/0 conn(0x7fd5f4159f00 msgr2=0x7fd5f40b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:03.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.395+0000 7fd5f27fc640 1 --2- 172.21.15.84:0/765373651 >> v2:172.21.15.157:3300/0 conn(0x7fd5f4159f00 0x7fd5f40b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:03.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.395+0000 7fd5f27fc640 1 -- 172.21.15.84:0/765373651 >> v1:172.21.15.157:6789/0 conn(0x7fd5f415d880 legacy=0x7fd5f406fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:03.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.395+0000 7fd5f27fc640 1 -- 172.21.15.84:0/765373651 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f406cba0 con 0x7fd5f4159b20 2023-10-16T10:05:03.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.395+0000 7fd5f27fc640 1 -- 172.21.15.84:0/765373651 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd5e40019f0 con 0x7fd5f4159b20 2023-10-16T10:05:03.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.395+0000 7fd5f27fc640 1 -- 172.21.15.84:0/765373651 >> v1:172.21.15.84:6789/0 conn(0x7fd5f4159b20 legacy=0x7fd5f406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:03.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.395+0000 7fd5f27fc640 1 --2- 172.21.15.84:0/765373651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5e00039c0 0x7fd5e0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:03.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.395+0000 7fd5f27fc640 1 --2- 172.21.15.84:0/765373651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5e00047c0 0x7fd5e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:03.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.395+0000 7fd5f27fc640 1 --2- 172.21.15.84:0/765373651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e00051d0 0x7fd5e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:03.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.395+0000 7fd5f27fc640 1 -- 172.21.15.84:0/765373651 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f406c8c0 con 0x7fd5e00051d0 2023-10-16T10:05:03.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.395+0000 7fd5f27fc640 1 -- 172.21.15.84:0/765373651 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e000c120 con 0x7fd5e00039c0 2023-10-16T10:05:03.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.395+0000 7fd5f27fc640 1 -- 172.21.15.84:0/765373651 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e000e4f0 con 0x7fd5e00047c0 2023-10-16T10:05:03.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.396+0000 7fd5f3fff640 1 --2- 172.21.15.84:0/765373651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e00051d0 0x7fd5e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:03.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.396+0000 7fd5f37fe640 1 --2- 172.21.15.84:0/765373651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5e00039c0 0x7fd5e0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:03.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.396+0000 7fd5f2ffd640 1 --2- 172.21.15.84:0/765373651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5e00047c0 0x7fd5e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:03.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.396+0000 7fd5f3fff640 1 -- 172.21.15.84:0/765373651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5e00047c0 msgr2=0x7fd5e0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:03.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.396+0000 7fd5f3fff640 1 --2- 172.21.15.84:0/765373651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5e00047c0 0x7fd5e0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:03.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.396+0000 7fd5f3fff640 1 -- 172.21.15.84:0/765373651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5e00039c0 msgr2=0x7fd5e0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:03.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.396+0000 7fd5f3fff640 1 --2- 172.21.15.84:0/765373651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5e00039c0 0x7fd5e0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:03.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.396+0000 7fd5f3fff640 1 -- 172.21.15.84:0/765373651 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e0003520 con 0x7fd5e00051d0 2023-10-16T10:05:03.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.397+0000 7fd5f3fff640 1 -- 172.21.15.84:0/765373651 reap_dead start 2023-10-16T10:05:03.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.397+0000 7fd5f3fff640 1 --2- 172.21.15.84:0/765373651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e00051d0 0x7fd5e000b980 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec001750 tx=0x7fd5e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2674b2c4d56f916 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:03.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.397+0000 7fd5f27fc640 1 -- 172.21.15.84:0/765373651 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec0052d0 con 0x7fd5e00051d0 2023-10-16T10:05:03.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.398+0000 7fd5f27fc640 1 -- 172.21.15.84:0/765373651 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ec02e050 con 0x7fd5e00051d0 2023-10-16T10:05:03.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.398+0000 7fd5f27fc640 1 -- 172.21.15.84:0/765373651 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec0035f0 con 0x7fd5e00051d0 2023-10-16T10:05:03.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.398+0000 7fd5f9c26640 1 -- 172.21.15.84:0/765373651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e00051d0 msgr2=0x7fd5e000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:03.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.398+0000 7fd5f9c26640 1 --2- 172.21.15.84:0/765373651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e00051d0 0x7fd5e000b980 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec001750 tx=0x7fd5e00057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:03.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.398+0000 7fd5f9c26640 1 -- 172.21.15.84:0/765373651 shutdown_connections 2023-10-16T10:05:03.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.398+0000 7fd5f9c26640 1 --2- 172.21.15.84:0/765373651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e00051d0 0x7fd5e000b980 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:03.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.398+0000 7fd5f9c26640 1 -- 172.21.15.84:0/765373651 >> 172.21.15.84:0/765373651 conn(0x7fd5f40cb410 msgr2=0x7fd5f40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:03.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.398+0000 7fd5f9c26640 1 -- 172.21.15.84:0/765373651 shutdown_connections 2023-10-16T10:05:03.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.398+0000 7fd5f9c26640 1 -- 172.21.15.84:0/765373651 wait complete. 2023-10-16T10:05:03.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.400+0000 7fd5f9c26640 1 Processor -- start 2023-10-16T10:05:03.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.400+0000 7fd5f9c26640 1 -- start start 2023-10-16T10:05:03.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.401+0000 7fd5f9c26640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e00051d0 0x7fd5f4148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:03.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.401+0000 7fd5f9c26640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5f4155cf0 0x7fd5f40b1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:03.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.401+0000 7fd5f9c26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5f415bc60 0x7fd5f415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:03.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.401+0000 7fd5f9c26640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f415c620 con 0x7fd5e00051d0 2023-10-16T10:05:03.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.401+0000 7fd5f9c26640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f415ba20 con 0x7fd5f415bc60 2023-10-16T10:05:03.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.401+0000 7fd5f9c26640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f40c64c0 con 0x7fd5f4155cf0 2023-10-16T10:05:03.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.401+0000 7fd5f37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e00051d0 0x7fd5f4148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:03.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.401+0000 7fd5f37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e00051d0 0x7fd5f4148d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50414/0 (socket says 172.21.15.84:50414) 2023-10-16T10:05:03.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.401+0000 7fd5f37fe640 1 -- 172.21.15.84:0/3548143945 learned_addr learned my addr 172.21.15.84:0/3548143945 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:03.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.401+0000 7fd5f37fe640 1 -- 172.21.15.84:0/3548143945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5f4155cf0 msgr2=0x7fd5f40b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:03.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.401+0000 7fd5f37fe640 1 --2- 172.21.15.84:0/3548143945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5f4155cf0 0x7fd5f40b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:03.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.401+0000 7fd5f37fe640 1 -- 172.21.15.84:0/3548143945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5f415bc60 msgr2=0x7fd5f415c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:03.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.401+0000 7fd5f37fe640 1 --2- 172.21.15.84:0/3548143945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5f415bc60 0x7fd5f415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:03.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.401+0000 7fd5f37fe640 1 -- 172.21.15.84:0/3548143945 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f41564d0 con 0x7fd5e00051d0 2023-10-16T10:05:03.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.402+0000 7fd5f37fe640 1 --2- 172.21.15.84:0/3548143945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e00051d0 0x7fd5f4148d60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4007a10 tx=0x7fd5e400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:03.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.402+0000 7fd5ebfff640 1 -- 172.21.15.84:0/3548143945 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e400edf0 con 0x7fd5e00051d0 2023-10-16T10:05:03.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.402+0000 7fd5ebfff640 1 -- 172.21.15.84:0/3548143945 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e4013420 con 0x7fd5e00051d0 2023-10-16T10:05:03.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.402+0000 7fd5ebfff640 1 -- 172.21.15.84:0/3548143945 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40137d0 con 0x7fd5e00051d0 2023-10-16T10:05:03.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.402+0000 7fd5f9c26640 1 -- 172.21.15.84:0/3548143945 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f40c5b90 con 0x7fd5e00051d0 2023-10-16T10:05:03.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.402+0000 7fd5f9c26640 1 -- 172.21.15.84:0/3548143945 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f40c6150 con 0x7fd5e00051d0 2023-10-16T10:05:03.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.403+0000 7fd5f9c26640 1 -- 172.21.15.84:0/3548143945 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f40c6310 con 0x7fd5e00051d0 2023-10-16T10:05:03.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.404+0000 7fd5ebfff640 1 -- 172.21.15.84:0/3548143945 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd5e4013930 con 0x7fd5e00051d0 2023-10-16T10:05:03.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.405+0000 7fd5ebfff640 1 --2- 172.21.15.84:0/3548143945 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fd5d406afc0 0x7fd5d406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:03.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.406+0000 7fd5ebfff640 1 -- 172.21.15.84:0/3548143945 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5631+0+0 (secure 0 0 0) 0x7fd5e4089da0 con 0x7fd5e00051d0 2023-10-16T10:05:03.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.406+0000 7fd5f2ffd640 1 --2- 172.21.15.84:0/3548143945 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fd5d406afc0 0x7fd5d406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:03.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.406+0000 7fd5f2ffd640 1 --2- 172.21.15.84:0/3548143945 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fd5d406afc0 0x7fd5d406d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc006a90 tx=0x7fd5dc006a20 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:03.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.410+0000 7fd5ebfff640 1 -- 172.21.15.84:0/3548143945 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd5e4054d80 con 0x7fd5e00051d0 2023-10-16T10:05:03.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:03.671+0000 7fd5f9c26640 1 -- 172.21.15.84:0/3548143945 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7fd5f40c62e0 con 0x7fd5e00051d0 2023-10-16T10:05:03.763 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:05:03.773 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:05:04.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.279+0000 7fd5ebfff640 1 -- 172.21.15.84:0/3548143945 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v593) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd5e4001850 con 0x7fd5e00051d0 2023-10-16T10:05:04.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.283+0000 7fd5f9c26640 1 -- 172.21.15.84:0/3548143945 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fd5d406afc0 msgr2=0x7fd5d406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:04.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.283+0000 7fd5f9c26640 1 --2- 172.21.15.84:0/3548143945 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fd5d406afc0 0x7fd5d406d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc006a90 tx=0x7fd5dc006a20 comp rx=0 tx=0).stop 2023-10-16T10:05:04.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.283+0000 7fd5f9c26640 1 -- 172.21.15.84:0/3548143945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e00051d0 msgr2=0x7fd5f4148d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:04.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.283+0000 7fd5f9c26640 1 --2- 172.21.15.84:0/3548143945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e00051d0 0x7fd5f4148d60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4007a10 tx=0x7fd5e400e530 comp rx=0 tx=0).stop 2023-10-16T10:05:04.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.283+0000 7fd5f9c26640 1 -- 172.21.15.84:0/3548143945 shutdown_connections 2023-10-16T10:05:04.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.283+0000 7fd5f9c26640 1 --2- 172.21.15.84:0/3548143945 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fd5d406afc0 0x7fd5d406d480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:04.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.283+0000 7fd5f9c26640 1 --2- 172.21.15.84:0/3548143945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5f415bc60 0x7fd5f415c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:04.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.283+0000 7fd5f9c26640 1 --2- 172.21.15.84:0/3548143945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5f4155cf0 0x7fd5f40b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:04.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.283+0000 7fd5f9c26640 1 --2- 172.21.15.84:0/3548143945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5e00051d0 0x7fd5f4148d60 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:04.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.283+0000 7fd5f9c26640 1 -- 172.21.15.84:0/3548143945 >> 172.21.15.84:0/3548143945 conn(0x7fd5f40cb410 msgr2=0x7fd5f4158de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:04.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.283+0000 7fd5f9c26640 1 -- 172.21.15.84:0/3548143945 shutdown_connections 2023-10-16T10:05:04.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.283+0000 7fd5f9c26640 1 -- 172.21.15.84:0/3548143945 wait complete. 2023-10-16T10:05:04.306 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T10:05:04.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.523+0000 7f5301814640 1 Processor -- start 2023-10-16T10:05:04.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.523+0000 7f5301814640 1 -- start start 2023-10-16T10:05:04.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.524+0000 7f5301814640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f52fc06c040 0x7f52fc06c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:04.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.524+0000 7f5301814640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f52fc0ad970 0x7f52fc15e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:04.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.524+0000 7f5301814640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52fc06c960 con 0x7f52fc15ebc0 2023-10-16T10:05:04.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.524+0000 7f5301814640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52fc15b9d0 con 0x7f52fc0ad970 2023-10-16T10:05:04.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.524+0000 7f5301814640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52fc15bcb0 con 0x7f52fc06c040 2023-10-16T10:05:04.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.525+0000 7f52fbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f52fc0ad970 0x7f52fc15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:04.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.525+0000 7f52fbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f52fc0ad970 0x7f52fc15e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50416/0 (socket says 172.21.15.84:50416) 2023-10-16T10:05:04.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.525+0000 7f52fbfff640 1 -- 172.21.15.84:0/1261073829 learned_addr learned my addr 172.21.15.84:0/1261073829 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:04.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.525+0000 7f5300812640 1 --2- 172.21.15.84:0/1261073829 >> v2:172.21.15.157:3300/0 conn(0x7f52fc06c040 0x7f52fc06c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:04.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.525+0000 7f52fbfff640 1 -- 172.21.15.84:0/1261073829 >> v2:172.21.15.157:3300/0 conn(0x7f52fc06c040 msgr2=0x7f52fc06c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:04.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.525+0000 7f52fbfff640 1 --2- 172.21.15.84:0/1261073829 >> v2:172.21.15.157:3300/0 conn(0x7f52fc06c040 0x7f52fc06c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:04.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.525+0000 7f52fbfff640 1 -- 172.21.15.84:0/1261073829 >> v1:172.21.15.84:6789/0 conn(0x7f52fc15ebc0 legacy=0x7f52fc0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:04.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.525+0000 7f52fbfff640 1 -- 172.21.15.84:0/1261073829 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc15bf90 con 0x7f52fc0ad970 2023-10-16T10:05:04.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.525+0000 7f52fb7fe640 1 -- 172.21.15.84:0/1261073829 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164145432 0 0) 0x7f52fc06c960 con 0x7f52fc15ebc0 2023-10-16T10:05:04.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.525+0000 7f5300812640 1 --2- 172.21.15.84:0/1261073829 >> v2:172.21.15.157:3300/0 conn(0x7f52fc06c040 0x7f52fc06c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:04.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.526+0000 7f52fbfff640 1 --2- 172.21.15.84:0/1261073829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52fc0ad970 0x7f52fc15e680 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0cf350 tx=0x7f52e802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9afc02efc2855d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:04.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.527+0000 7f52fb7fe640 1 -- 172.21.15.84:0/1261073829 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e8030860 con 0x7f52fc0ad970 2023-10-16T10:05:04.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.527+0000 7f52fb7fe640 1 -- 172.21.15.84:0/1261073829 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e80309c0 con 0x7f52fc0ad970 2023-10-16T10:05:04.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.527+0000 7f52fb7fe640 1 -- 172.21.15.84:0/1261073829 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e8030d50 con 0x7f52fc0ad970 2023-10-16T10:05:04.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.527+0000 7f5301814640 1 -- 172.21.15.84:0/1261073829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52fc0ad970 msgr2=0x7f52fc15e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:04.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.527+0000 7f5301814640 1 --2- 172.21.15.84:0/1261073829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52fc0ad970 0x7f52fc15e680 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0cf350 tx=0x7f52e802b9a0 comp rx=0 tx=0).stop 2023-10-16T10:05:04.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.527+0000 7f5301814640 1 -- 172.21.15.84:0/1261073829 shutdown_connections 2023-10-16T10:05:04.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.527+0000 7f5301814640 1 --2- 172.21.15.84:0/1261073829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52fc0ad970 0x7f52fc15e680 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:04.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.527+0000 7f5301814640 1 --2- 172.21.15.84:0/1261073829 >> v2:172.21.15.157:3300/0 conn(0x7f52fc06c040 0x7f52fc06c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:04.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.527+0000 7f5301814640 1 -- 172.21.15.84:0/1261073829 >> 172.21.15.84:0/1261073829 conn(0x7f52fc0ce6a0 msgr2=0x7f52fc0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:04.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.527+0000 7f5301814640 1 -- 172.21.15.84:0/1261073829 shutdown_connections 2023-10-16T10:05:04.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.527+0000 7f5301814640 1 -- 172.21.15.84:0/1261073829 wait complete. 2023-10-16T10:05:04.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.529+0000 7f5301814640 1 Processor -- start 2023-10-16T10:05:04.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.529+0000 7f5301814640 1 -- start start 2023-10-16T10:05:04.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.530+0000 7f5301814640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52fc06c040 0x7f52fc0ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:04.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.530+0000 7f5301814640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52fc0ad970 0x7f52fc14ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:04.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.530+0000 7f5301814640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52fc15ebc0 0x7f52fc14f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:04.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.530+0000 7f5301814640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc0b2440 con 0x7f52fc06c040 2023-10-16T10:05:04.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.530+0000 7f5301814640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc15aba0 con 0x7f52fc15ebc0 2023-10-16T10:05:04.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.530+0000 7f5301814640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc0b17c0 con 0x7f52fc0ad970 2023-10-16T10:05:04.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.530+0000 7f5300812640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52fc06c040 0x7f52fc0ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:04.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.530+0000 7f5300812640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52fc06c040 0x7f52fc0ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50424/0 (socket says 172.21.15.84:50424) 2023-10-16T10:05:04.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.530+0000 7f5300812640 1 -- 172.21.15.84:0/1254199143 learned_addr learned my addr 172.21.15.84:0/1254199143 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:04.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.531+0000 7f5300812640 1 -- 172.21.15.84:0/1254199143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52fc0ad970 msgr2=0x7f52fc14ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:04.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.531+0000 7f5300812640 1 --2- 172.21.15.84:0/1254199143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52fc0ad970 0x7f52fc14ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:04.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.531+0000 7f5300812640 1 -- 172.21.15.84:0/1254199143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52fc15ebc0 msgr2=0x7f52fc14f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:04.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.531+0000 7f5300812640 1 --2- 172.21.15.84:0/1254199143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52fc15ebc0 0x7f52fc14f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:04.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.531+0000 7f5300812640 1 -- 172.21.15.84:0/1254199143 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e80094d0 con 0x7f52fc06c040 2023-10-16T10:05:04.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.531+0000 7f5300812640 1 --2- 172.21.15.84:0/1254199143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52fc06c040 0x7f52fc0ca740 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f52f000c7f0 tx=0x7f52f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:04.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.531+0000 7f52f97fa640 1 -- 172.21.15.84:0/1254199143 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0007820 con 0x7f52fc06c040 2023-10-16T10:05:04.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.531+0000 7f52f97fa640 1 -- 172.21.15.84:0/1254199143 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f0007980 con 0x7f52fc06c040 2023-10-16T10:05:04.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.531+0000 7f52f97fa640 1 -- 172.21.15.84:0/1254199143 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0007cf0 con 0x7f52fc06c040 2023-10-16T10:05:04.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.531+0000 7f5301814640 1 -- 172.21.15.84:0/1254199143 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52fc0b1aa0 con 0x7f52fc06c040 2023-10-16T10:05:04.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.531+0000 7f5301814640 1 -- 172.21.15.84:0/1254199143 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52fc0b2060 con 0x7f52fc06c040 2023-10-16T10:05:04.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.533+0000 7f5301814640 1 -- 172.21.15.84:0/1254199143 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52fc06b330 con 0x7f52fc06c040 2023-10-16T10:05:04.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.534+0000 7f52f97fa640 1 -- 172.21.15.84:0/1254199143 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f52f00113f0 con 0x7f52fc06c040 2023-10-16T10:05:04.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.534+0000 7f52f97fa640 1 --2- 172.21.15.84:0/1254199143 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f52e006af90 0x7f52e006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:04.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.534+0000 7f52f97fa640 1 -- 172.21.15.84:0/1254199143 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f52f0089ad0 con 0x7f52fc06c040 2023-10-16T10:05:04.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.535+0000 7f52fbfff640 1 --2- 172.21.15.84:0/1254199143 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f52e006af90 0x7f52e006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:04.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.535+0000 7f52fbfff640 1 --2- 172.21.15.84:0/1254199143 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f52e006af90 0x7f52e006d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f52e8036000 tx=0x7f52e8030000 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:04.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.539+0000 7f52f97fa640 1 -- 172.21.15.84:0/1254199143 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f52f0054a40 con 0x7f52fc06c040 2023-10-16T10:05:04.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.730+0000 7f5301814640 1 -- 172.21.15.84:0/1254199143 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f52fc04e9b0 con 0x7f52fc06c040 2023-10-16T10:05:04.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.735+0000 7f52f97fa640 1 -- 172.21.15.84:0/1254199143 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f52f001f0c0 con 0x7f52fc06c040 2023-10-16T10:05:04.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.737+0000 7f5301814640 1 -- 172.21.15.84:0/1254199143 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f52e006af90 msgr2=0x7f52e006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:04.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.738+0000 7f5301814640 1 --2- 172.21.15.84:0/1254199143 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f52e006af90 0x7f52e006d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f52e8036000 tx=0x7f52e8030000 comp rx=0 tx=0).stop 2023-10-16T10:05:04.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.738+0000 7f5301814640 1 -- 172.21.15.84:0/1254199143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52fc06c040 msgr2=0x7f52fc0ca740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:04.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.738+0000 7f5301814640 1 --2- 172.21.15.84:0/1254199143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52fc06c040 0x7f52fc0ca740 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f52f000c7f0 tx=0x7f52f000ccb0 comp rx=0 tx=0).stop 2023-10-16T10:05:04.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.738+0000 7f5301814640 1 -- 172.21.15.84:0/1254199143 shutdown_connections 2023-10-16T10:05:04.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.738+0000 7f5301814640 1 --2- 172.21.15.84:0/1254199143 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f52e006af90 0x7f52e006d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:04.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.738+0000 7f5301814640 1 --2- 172.21.15.84:0/1254199143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52fc15ebc0 0x7f52fc14f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:04.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.738+0000 7f5301814640 1 --2- 172.21.15.84:0/1254199143 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52fc0ad970 0x7f52fc14ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:04.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.738+0000 7f5301814640 1 --2- 172.21.15.84:0/1254199143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52fc06c040 0x7f52fc0ca740 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:04.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.738+0000 7f5301814640 1 -- 172.21.15.84:0/1254199143 >> 172.21.15.84:0/1254199143 conn(0x7f52fc0ce6a0 msgr2=0x7f52fc15dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:04.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.738+0000 7f5301814640 1 -- 172.21.15.84:0/1254199143 shutdown_connections 2023-10-16T10:05:04.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:04.738+0000 7f5301814640 1 -- 172.21.15.84:0/1254199143 wait complete. 2023-10-16T10:05:04.758 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:05:04.759 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:05:04.875 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:05:05.000 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:05:05.000 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:05:05.001 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:05:05.062 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:05:05.062 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:05:05.062 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:05:05.122 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:05:05.122 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:05:05.123 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:05:05.183 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:05:05.184 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:05:05.184 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:05:05.245 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:05:05.245 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T10:05:05.245 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:05:05.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.466+0000 7fa426a3f640 1 Processor -- start 2023-10-16T10:05:05.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.467+0000 7fa426a3f640 1 -- start start 2023-10-16T10:05:05.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.468+0000 7fa426a3f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa4200b0b20 0x7fa4201615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:05.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.468+0000 7fa426a3f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa42015e3e0 con 0x7fa4200cd0c0 2023-10-16T10:05:05.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.468+0000 7fa426a3f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4200cd4a0 con 0x7fa42015d3f0 2023-10-16T10:05:05.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.468+0000 7fa426a3f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa42015e100 con 0x7fa4200b0b20 2023-10-16T10:05:05.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.468+0000 7fa41ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa4200cd0c0 0x7fa4200b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57412/0 (socket says 172.21.15.84:57412) 2023-10-16T10:05:05.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.468+0000 7fa41ffff640 1 -- 172.21.15.84:0/1130518607 learned_addr learned my addr 172.21.15.84:0/1130518607 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:05.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.469+0000 7fa41f7fe640 1 --2- 172.21.15.84:0/1130518607 >> v2:172.21.15.157:3300/0 conn(0x7fa4200b0b20 0x7fa4201615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:05.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.469+0000 7fa41effd640 1 -- 172.21.15.84:0/1130518607 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756877492 0 0) 0x7fa42015e3e0 con 0x7fa4200cd0c0 2023-10-16T10:05:05.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.469+0000 7fa41effd640 1 -- 172.21.15.84:0/1130518607 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa40c003520 con 0x7fa4200cd0c0 2023-10-16T10:05:05.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.469+0000 7fa41effd640 1 -- 172.21.15.84:0/1130518607 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3044677274 0 0) 0x7fa40c003520 con 0x7fa4200cd0c0 2023-10-16T10:05:05.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.469+0000 7fa41effd640 1 -- 172.21.15.84:0/1130518607 >> v2:172.21.15.157:3300/0 conn(0x7fa4200b0b20 msgr2=0x7fa4201615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:05.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.469+0000 7fa41effd640 1 --2- 172.21.15.84:0/1130518607 >> v2:172.21.15.157:3300/0 conn(0x7fa4200b0b20 0x7fa4201615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:05.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.469+0000 7fa41effd640 1 -- 172.21.15.84:0/1130518607 >> v1:172.21.15.157:6789/0 conn(0x7fa42015d3f0 legacy=0x7fa42015d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:05.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.469+0000 7fa41effd640 1 -- 172.21.15.84:0/1130518607 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa420160b10 con 0x7fa4200cd0c0 2023-10-16T10:05:05.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.469+0000 7fa41effd640 1 -- 172.21.15.84:0/1130518607 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fa4100019f0 con 0x7fa4200cd0c0 2023-10-16T10:05:05.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.469+0000 7fa41effd640 1 -- 172.21.15.84:0/1130518607 >> v1:172.21.15.84:6789/0 conn(0x7fa4200cd0c0 legacy=0x7fa4200b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:05.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.469+0000 7fa41effd640 1 --2- 172.21.15.84:0/1130518607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa40c0039c0 0x7fa40c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:05.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.469+0000 7fa41effd640 1 --2- 172.21.15.84:0/1130518607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa40c0047c0 0x7fa40c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:05.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.469+0000 7fa41effd640 1 --2- 172.21.15.84:0/1130518607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40c0051d0 0x7fa40c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:05.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.469+0000 7fa41effd640 1 -- 172.21.15.84:0/1130518607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42015e100 con 0x7fa40c0051d0 2023-10-16T10:05:05.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.470+0000 7fa41effd640 1 -- 172.21.15.84:0/1130518607 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c00c120 con 0x7fa40c0039c0 2023-10-16T10:05:05.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.470+0000 7fa41effd640 1 -- 172.21.15.84:0/1130518607 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c00c2e0 con 0x7fa40c0047c0 2023-10-16T10:05:05.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.470+0000 7fa424fdc640 1 --2- 172.21.15.84:0/1130518607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40c0051d0 0x7fa40c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:05.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.470+0000 7fa41ffff640 1 --2- 172.21.15.84:0/1130518607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa40c0039c0 0x7fa40c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:05.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.470+0000 7fa41f7fe640 1 --2- 172.21.15.84:0/1130518607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa40c0047c0 0x7fa40c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:05.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.470+0000 7fa424fdc640 1 -- 172.21.15.84:0/1130518607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa40c0047c0 msgr2=0x7fa40c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:05.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.470+0000 7fa424fdc640 1 --2- 172.21.15.84:0/1130518607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa40c0047c0 0x7fa40c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:05.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.470+0000 7fa424fdc640 1 -- 172.21.15.84:0/1130518607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa40c0039c0 msgr2=0x7fa40c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:05.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.470+0000 7fa424fdc640 1 --2- 172.21.15.84:0/1130518607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa40c0039c0 0x7fa40c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:05.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.470+0000 7fa424fdc640 1 -- 172.21.15.84:0/1130518607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c003520 con 0x7fa40c0051d0 2023-10-16T10:05:05.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.471+0000 7fa424fdc640 1 -- 172.21.15.84:0/1130518607 reap_dead start 2023-10-16T10:05:05.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.471+0000 7fa424fdc640 1 --2- 172.21.15.84:0/1130518607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40c0051d0 0x7fa40c00b980 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fa418001720 tx=0x7fa40c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd2999c4f7eeeb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:05.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.471+0000 7fa41effd640 1 -- 172.21.15.84:0/1130518607 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4180052d0 con 0x7fa40c0051d0 2023-10-16T10:05:05.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.472+0000 7fa41effd640 1 -- 172.21.15.84:0/1130518607 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41802e050 con 0x7fa40c0051d0 2023-10-16T10:05:05.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.472+0000 7fa41effd640 1 -- 172.21.15.84:0/1130518607 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4180035f0 con 0x7fa40c0051d0 2023-10-16T10:05:05.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.472+0000 7fa426a3f640 1 -- 172.21.15.84:0/1130518607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40c0051d0 msgr2=0x7fa40c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:05.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.472+0000 7fa426a3f640 1 --2- 172.21.15.84:0/1130518607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40c0051d0 0x7fa40c00b980 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fa418001720 tx=0x7fa40c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:05.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.472+0000 7fa426a3f640 1 -- 172.21.15.84:0/1130518607 shutdown_connections 2023-10-16T10:05:05.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.472+0000 7fa426a3f640 1 --2- 172.21.15.84:0/1130518607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40c0051d0 0x7fa40c00b980 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:05.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.472+0000 7fa426a3f640 1 -- 172.21.15.84:0/1130518607 >> 172.21.15.84:0/1130518607 conn(0x7fa4200cb410 msgr2=0x7fa4200cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:05.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.472+0000 7fa426a3f640 1 -- 172.21.15.84:0/1130518607 shutdown_connections 2023-10-16T10:05:05.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.472+0000 7fa426a3f640 1 -- 172.21.15.84:0/1130518607 wait complete. 2023-10-16T10:05:05.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.474+0000 7fa426a3f640 1 Processor -- start 2023-10-16T10:05:05.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.474+0000 7fa426a3f640 1 -- start start 2023-10-16T10:05:05.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.475+0000 7fa426a3f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40c0051d0 0x7fa420146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:05.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.475+0000 7fa426a3f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4200c86b0 0x7fa4200c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:05.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.475+0000 7fa426a3f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4200c5fa0 0x7fa42015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:05.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.475+0000 7fa426a3f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200b3bc0 con 0x7fa40c0051d0 2023-10-16T10:05:05.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.475+0000 7fa426a3f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200c6430 con 0x7fa4200c5fa0 2023-10-16T10:05:05.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.475+0000 7fa426a3f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4201551f0 con 0x7fa4200c86b0 2023-10-16T10:05:05.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.475+0000 7fa41ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40c0051d0 0x7fa420146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:05.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.475+0000 7fa41ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40c0051d0 0x7fa420146f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50446/0 (socket says 172.21.15.84:50446) 2023-10-16T10:05:05.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.475+0000 7fa41ffff640 1 -- 172.21.15.84:0/3150027723 learned_addr learned my addr 172.21.15.84:0/3150027723 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:05.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.476+0000 7fa41ffff640 1 -- 172.21.15.84:0/3150027723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4200c86b0 msgr2=0x7fa4200c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:05.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.476+0000 7fa41f7fe640 1 --2- 172.21.15.84:0/3150027723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4200c86b0 0x7fa4200c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:05.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.476+0000 7fa424fdc640 1 --2- 172.21.15.84:0/3150027723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4200c5fa0 0x7fa42015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:05.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.476+0000 7fa41ffff640 1 --2- 172.21.15.84:0/3150027723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4200c86b0 0x7fa4200c5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:05.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.476+0000 7fa41ffff640 1 -- 172.21.15.84:0/3150027723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4200c5fa0 msgr2=0x7fa42015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:05.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.476+0000 7fa41ffff640 1 --2- 172.21.15.84:0/3150027723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4200c5fa0 0x7fa42015ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:05.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.476+0000 7fa41ffff640 1 -- 172.21.15.84:0/3150027723 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4200c8b10 con 0x7fa40c0051d0 2023-10-16T10:05:05.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.476+0000 7fa41ffff640 1 --2- 172.21.15.84:0/3150027723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40c0051d0 0x7fa420146f40 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fa410001f70 tx=0x7fa41000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:05.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.476+0000 7fa41cff9640 1 -- 172.21.15.84:0/3150027723 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa410004b60 con 0x7fa40c0051d0 2023-10-16T10:05:05.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.477+0000 7fa41cff9640 1 -- 172.21.15.84:0/3150027723 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa410004cc0 con 0x7fa40c0051d0 2023-10-16T10:05:05.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.477+0000 7fa41cff9640 1 -- 172.21.15.84:0/3150027723 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4100072d0 con 0x7fa40c0051d0 2023-10-16T10:05:05.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.477+0000 7fa426a3f640 1 -- 172.21.15.84:0/3150027723 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4201f2eb0 con 0x7fa40c0051d0 2023-10-16T10:05:05.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.477+0000 7fa426a3f640 1 -- 172.21.15.84:0/3150027723 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4200b20e0 con 0x7fa40c0051d0 2023-10-16T10:05:05.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.478+0000 7fa426a3f640 1 -- 172.21.15.84:0/3150027723 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4200b1c60 con 0x7fa40c0051d0 2023-10-16T10:05:05.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.479+0000 7fa41cff9640 1 -- 172.21.15.84:0/3150027723 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa410007430 con 0x7fa40c0051d0 2023-10-16T10:05:05.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.479+0000 7fa41cff9640 1 --2- 172.21.15.84:0/3150027723 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fa40006af90 0x7fa40006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:05.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.480+0000 7fa41cff9640 1 -- 172.21.15.84:0/3150027723 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5631+0+0 (secure 0 0 0) 0x7fa4100897e0 con 0x7fa40c0051d0 2023-10-16T10:05:05.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.480+0000 7fa41f7fe640 1 --2- 172.21.15.84:0/3150027723 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fa40006af90 0x7fa40006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:05.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.480+0000 7fa41f7fe640 1 --2- 172.21.15.84:0/3150027723 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fa40006af90 0x7fa40006d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa408009910 tx=0x7fa408008040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:05.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.484+0000 7fa41cff9640 1 -- 172.21.15.84:0/3150027723 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa4200b1c60 con 0x7fa40c0051d0 2023-10-16T10:05:05.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.714+0000 7fa426a3f640 1 -- 172.21.15.84:0/3150027723 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4200b1c60 con 0x7fa40c0051d0 2023-10-16T10:05:05.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.715+0000 7fa41cff9640 1 -- 172.21.15.84:0/3150027723 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fa4200b1c60 con 0x7fa40c0051d0 2023-10-16T10:05:05.717 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:05.717 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5066,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1223610299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1223610299},{"type":"v1","addr":"172.21.15.157:6833","nonce":1223610299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3636443426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3636443426},{"type":"v1","addr":"172.21.15.157:6835","nonce":3636443426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6537,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/3575464874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3575464874},{"type":"v1","addr":"172.21.15.84:6836","nonce":3575464874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6933,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/458796559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":458796559},{"type":"v1","addr":"172.21.15.84:6837","nonce":458796559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[]} 2023-10-16T10:05:05.717 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 37 2023-10-16T10:05:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.717+0000 7fa426a3f640 1 -- 172.21.15.84:0/3150027723 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fa40006af90 msgr2=0x7fa40006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.717+0000 7fa426a3f640 1 --2- 172.21.15.84:0/3150027723 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fa40006af90 0x7fa40006d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa408009910 tx=0x7fa408008040 comp rx=0 tx=0).stop 2023-10-16T10:05:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.717+0000 7fa426a3f640 1 -- 172.21.15.84:0/3150027723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40c0051d0 msgr2=0x7fa420146f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.717+0000 7fa426a3f640 1 --2- 172.21.15.84:0/3150027723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40c0051d0 0x7fa420146f40 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fa410001f70 tx=0x7fa41000b4a0 comp rx=0 tx=0).stop 2023-10-16T10:05:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.718+0000 7fa426a3f640 1 -- 172.21.15.84:0/3150027723 shutdown_connections 2023-10-16T10:05:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.718+0000 7fa426a3f640 1 --2- 172.21.15.84:0/3150027723 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fa40006af90 0x7fa40006d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.718+0000 7fa426a3f640 1 --2- 172.21.15.84:0/3150027723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4200c5fa0 0x7fa42015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.718+0000 7fa426a3f640 1 --2- 172.21.15.84:0/3150027723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4200c86b0 0x7fa4200c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.718+0000 7fa426a3f640 1 --2- 172.21.15.84:0/3150027723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40c0051d0 0x7fa420146f40 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.718+0000 7fa426a3f640 1 -- 172.21.15.84:0/3150027723 >> 172.21.15.84:0/3150027723 conn(0x7fa4200cb410 msgr2=0x7fa4200caac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.718+0000 7fa426a3f640 1 -- 172.21.15.84:0/3150027723 shutdown_connections 2023-10-16T10:05:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:05.718+0000 7fa426a3f640 1 -- 172.21.15.84:0/3150027723 wait complete. 2023-10-16T10:05:05.737 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T10:05:05.737 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T10:05:05.737 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T10:05:05.737 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:05:05.738 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T10:05:05.738 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T10:05:05.738 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T10:05:05.739 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:05:05.739 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T10:05:05.739 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T10:05:05.739 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T10:05:05.740 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:05:05.740 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T10:05:05.740 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T10:05:05.741 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T10:05:05.741 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:05:05.755 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T10:05:05.755 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T10:05:05.760 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T10:05:05.761 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T10:05:05.761 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T10:05:05.762 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T10:05:05.763 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T10:05:05.763 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T10:05:05.809 INFO:tasks.ceph.mds.b:Started 2023-10-16T10:05:05.809 DEBUG:teuthology.parallel:result is None 2023-10-16T10:05:05.811 INFO:tasks.ceph.mds.a:Started 2023-10-16T10:05:05.811 DEBUG:teuthology.parallel:result is None 2023-10-16T10:05:05.815 INFO:tasks.ceph.mds.d:Started 2023-10-16T10:05:05.816 DEBUG:teuthology.parallel:result is None 2023-10-16T10:05:05.818 INFO:tasks.ceph.mds.c:Started 2023-10-16T10:05:05.819 DEBUG:teuthology.parallel:result is None 2023-10-16T10:05:05.819 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T10:05:05.970 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T10:05:05.970 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T10:05:05.972 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-10-16T10:05:05.978 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T10:05:06.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.014+0000 7f6bd31e3640 1 Processor -- start 2023-10-16T10:05:06.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.014+0000 7f6bd31e3640 1 -- start start 2023-10-16T10:05:06.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.015+0000 7f6bd31e3640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6bcc15af80 0x7f6bcc0cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:06.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.015+0000 7f6bd31e3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6bcc065ab0 0x7f6bcc065f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:06.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.015+0000 7f6bd31e3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bcc15eff0 con 0x7f6bcc06faf0 2023-10-16T10:05:06.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.015+0000 7f6bd31e3640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bcc066480 con 0x7f6bcc15af80 2023-10-16T10:05:06.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.015+0000 7f6bd31e3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bcc0d0930 con 0x7f6bcc065ab0 2023-10-16T10:05:06.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.015+0000 7f6bd0f7f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6bcc15af80 0x7f6bcc0cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:06.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.015+0000 7f6bd0f7f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6bcc15af80 0x7f6bcc0cb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42522/0 (socket says 172.21.15.84:42522) 2023-10-16T10:05:06.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.015+0000 7f6bd0f7f640 1 -- 172.21.15.84:0/4006506746 learned_addr learned my addr 172.21.15.84:0/4006506746 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:06.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.015+0000 7f6bd1780640 1 --2- 172.21.15.84:0/4006506746 >> v2:172.21.15.157:3300/0 conn(0x7f6bcc065ab0 0x7f6bcc065f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:06.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.016+0000 7f6bd0f7f640 1 -- 172.21.15.84:0/4006506746 >> v2:172.21.15.157:3300/0 conn(0x7f6bcc065ab0 msgr2=0x7f6bcc065f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:06.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.016+0000 7f6bd0f7f640 1 --2- 172.21.15.84:0/4006506746 >> v2:172.21.15.157:3300/0 conn(0x7f6bcc065ab0 0x7f6bcc065f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.016+0000 7f6bd0f7f640 1 -- 172.21.15.84:0/4006506746 >> v1:172.21.15.157:6789/0 conn(0x7f6bcc06faf0 legacy=0x7f6bcc15f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:06.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.016+0000 7f6bcb7fe640 1 -- 172.21.15.84:0/4006506746 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201931543 0 0) 0x7f6bcc15eff0 con 0x7f6bcc06faf0 2023-10-16T10:05:06.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.016+0000 7f6bd0f7f640 1 -- 172.21.15.84:0/4006506746 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc15ecc0 con 0x7f6bcc15af80 2023-10-16T10:05:06.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.017+0000 7f6bd0f7f640 1 --2- 172.21.15.84:0/4006506746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bcc15af80 0x7f6bcc0cb1c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0b6c00 tx=0x7f6bc002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf9d7a439936c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:06.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.017+0000 7f6bcb7fe640 1 -- 172.21.15.84:0/4006506746 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0031870 con 0x7f6bcc15af80 2023-10-16T10:05:06.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.017+0000 7f6bcb7fe640 1 -- 172.21.15.84:0/4006506746 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc00319d0 con 0x7f6bcc15af80 2023-10-16T10:05:06.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.017+0000 7f6bcb7fe640 1 -- 172.21.15.84:0/4006506746 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0031d60 con 0x7f6bcc15af80 2023-10-16T10:05:06.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.018+0000 7f6bd31e3640 1 -- 172.21.15.84:0/4006506746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bcc15af80 msgr2=0x7f6bcc0cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:06.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.018+0000 7f6bd31e3640 1 --2- 172.21.15.84:0/4006506746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bcc15af80 0x7f6bcc0cb1c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0b6c00 tx=0x7f6bc002b910 comp rx=0 tx=0).stop 2023-10-16T10:05:06.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.018+0000 7f6bd31e3640 1 -- 172.21.15.84:0/4006506746 shutdown_connections 2023-10-16T10:05:06.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.018+0000 7f6bd31e3640 1 --2- 172.21.15.84:0/4006506746 >> v2:172.21.15.157:3300/0 conn(0x7f6bcc065ab0 0x7f6bcc065f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.018+0000 7f6bd31e3640 1 --2- 172.21.15.84:0/4006506746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bcc15af80 0x7f6bcc0cb1c0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.018+0000 7f6bd31e3640 1 -- 172.21.15.84:0/4006506746 >> 172.21.15.84:0/4006506746 conn(0x7f6bcc0d0ab0 msgr2=0x7f6bcc0d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:06.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.018+0000 7f6bd31e3640 1 -- 172.21.15.84:0/4006506746 shutdown_connections 2023-10-16T10:05:06.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.018+0000 7f6bd31e3640 1 -- 172.21.15.84:0/4006506746 wait complete. 2023-10-16T10:05:06.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.020+0000 7f6bd31e3640 1 Processor -- start 2023-10-16T10:05:06.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.020+0000 7f6bd31e3640 1 -- start start 2023-10-16T10:05:06.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.021+0000 7f6bd31e3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bcc065ab0 0x7f6bcc1578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:06.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.021+0000 7f6bd31e3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bcc06faf0 0x7f6bcc157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:06.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.021+0000 7f6bd31e3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bcc15af80 0x7f6bcc149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:06.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.021+0000 7f6bd31e3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc0ac890 con 0x7f6bcc15af80 2023-10-16T10:05:06.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.021+0000 7f6bd31e3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc0ae590 con 0x7f6bcc065ab0 2023-10-16T10:05:06.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.021+0000 7f6bd31e3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc149520 con 0x7f6bcc06faf0 2023-10-16T10:05:06.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.021+0000 7f6bd1780640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bcc15af80 0x7f6bcc149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:06.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.021+0000 7f6bd1780640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bcc15af80 0x7f6bcc149be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42532/0 (socket says 172.21.15.84:42532) 2023-10-16T10:05:06.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.021+0000 7f6bd1780640 1 -- 172.21.15.84:0/2907315598 learned_addr learned my addr 172.21.15.84:0/2907315598 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:06.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.021+0000 7f6bd0f7f640 1 --2- 172.21.15.84:0/2907315598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bcc065ab0 0x7f6bcc1578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:06.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.021+0000 7f6bcbfff640 1 --2- 172.21.15.84:0/2907315598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bcc06faf0 0x7f6bcc157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:06.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.021+0000 7f6bd1780640 1 -- 172.21.15.84:0/2907315598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bcc06faf0 msgr2=0x7f6bcc157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:06.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.021+0000 7f6bd1780640 1 --2- 172.21.15.84:0/2907315598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bcc06faf0 0x7f6bcc157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.021+0000 7f6bd1780640 1 -- 172.21.15.84:0/2907315598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bcc065ab0 msgr2=0x7f6bcc1578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:06.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.021+0000 7f6bd1780640 1 --2- 172.21.15.84:0/2907315598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bcc065ab0 0x7f6bcc1578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.021+0000 7f6bd1780640 1 -- 172.21.15.84:0/2907315598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc00094d0 con 0x7f6bcc15af80 2023-10-16T10:05:06.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.021+0000 7f6bd0f7f640 1 --2- 172.21.15.84:0/2907315598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bcc065ab0 0x7f6bcc1578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:05:06.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.022+0000 7f6bd1780640 1 --2- 172.21.15.84:0/2907315598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bcc15af80 0x7f6bcc149be0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400efc0 tx=0x7f6bc400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:06.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.022+0000 7f6bc97fa640 1 -- 172.21.15.84:0/2907315598 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4009ba0 con 0x7f6bcc15af80 2023-10-16T10:05:06.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.022+0000 7f6bc97fa640 1 -- 172.21.15.84:0/2907315598 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc400f040 con 0x7f6bcc15af80 2023-10-16T10:05:06.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.022+0000 7f6bc97fa640 1 -- 172.21.15.84:0/2907315598 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc40146c0 con 0x7f6bcc15af80 2023-10-16T10:05:06.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.022+0000 7f6bd31e3640 1 -- 172.21.15.84:0/2907315598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bcc149800 con 0x7f6bcc15af80 2023-10-16T10:05:06.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.022+0000 7f6bd31e3640 1 -- 172.21.15.84:0/2907315598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bcc0acdc0 con 0x7f6bcc15af80 2023-10-16T10:05:06.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.023+0000 7f6bd31e3640 1 -- 172.21.15.84:0/2907315598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bcc15d620 con 0x7f6bcc15af80 2023-10-16T10:05:06.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.024+0000 7f6bc97fa640 1 -- 172.21.15.84:0/2907315598 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6bc4014820 con 0x7f6bcc15af80 2023-10-16T10:05:06.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.025+0000 7f6bc97fa640 1 --2- 172.21.15.84:0/2907315598 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f6bb006af90 0x7f6bb006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:06.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.025+0000 7f6bc97fa640 1 -- 172.21.15.84:0/2907315598 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f6bc408a660 con 0x7f6bcc15af80 2023-10-16T10:05:06.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.026+0000 7f6bd0f7f640 1 --2- 172.21.15.84:0/2907315598 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f6bb006af90 0x7f6bb006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:06.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.026+0000 7f6bd0f7f640 1 --2- 172.21.15.84:0/2907315598 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f6bb006af90 0x7f6bb006d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0036000 tx=0x7f6bc003c040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:06.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.030+0000 7f6bc97fa640 1 -- 172.21.15.84:0/2907315598 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6bc4055530 con 0x7f6bcc15af80 2023-10-16T10:05:06.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.233+0000 7f6bd31e3640 1 -- 172.21.15.84:0/2907315598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6bcc0acfe0 con 0x7f6bcc15af80 2023-10-16T10:05:06.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.302+0000 7f6bc97fa640 1 -- 172.21.15.84:0/2907315598 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v89) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6bc4058b50 con 0x7f6bcc15af80 2023-10-16T10:05:06.304 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-10-16T10:05:06.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.305+0000 7f6bd31e3640 1 -- 172.21.15.84:0/2907315598 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f6bb006af90 msgr2=0x7f6bb006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:06.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.305+0000 7f6bd31e3640 1 --2- 172.21.15.84:0/2907315598 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f6bb006af90 0x7f6bb006d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0036000 tx=0x7f6bc003c040 comp rx=0 tx=0).stop 2023-10-16T10:05:06.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.305+0000 7f6bd31e3640 1 -- 172.21.15.84:0/2907315598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bcc15af80 msgr2=0x7f6bcc149be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:06.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.305+0000 7f6bd31e3640 1 --2- 172.21.15.84:0/2907315598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bcc15af80 0x7f6bcc149be0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400efc0 tx=0x7f6bc400c490 comp rx=0 tx=0).stop 2023-10-16T10:05:06.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.305+0000 7f6bd31e3640 1 -- 172.21.15.84:0/2907315598 shutdown_connections 2023-10-16T10:05:06.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.305+0000 7f6bd31e3640 1 --2- 172.21.15.84:0/2907315598 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f6bb006af90 0x7f6bb006d450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.305+0000 7f6bd31e3640 1 --2- 172.21.15.84:0/2907315598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bcc15af80 0x7f6bcc149be0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.305+0000 7f6bd31e3640 1 --2- 172.21.15.84:0/2907315598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bcc06faf0 0x7f6bcc157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.305+0000 7f6bd31e3640 1 --2- 172.21.15.84:0/2907315598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bcc065ab0 0x7f6bcc1578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.306+0000 7f6bd31e3640 1 -- 172.21.15.84:0/2907315598 >> 172.21.15.84:0/2907315598 conn(0x7f6bcc0d0ab0 msgr2=0x7f6bcc0b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:06.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.306+0000 7f6bd31e3640 1 -- 172.21.15.84:0/2907315598 shutdown_connections 2023-10-16T10:05:06.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.306+0000 7f6bd31e3640 1 -- 172.21.15.84:0/2907315598 wait complete. 2023-10-16T10:05:06.328 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T10:05:06.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.532+0000 7f8825fb0640 1 Processor -- start 2023-10-16T10:05:06.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.533+0000 7f8825fb0640 1 -- start start 2023-10-16T10:05:06.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.534+0000 7f8825fb0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88200adb50 0x7f882006f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:06.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.534+0000 7f8825fb0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f88200b0b20 0x7f8820164f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:06.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.534+0000 7f8825fb0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8820071450 con 0x7f8820070fd0 2023-10-16T10:05:06.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.534+0000 7f8825fb0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88200cc600 con 0x7f88200b0b20 2023-10-16T10:05:06.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.534+0000 7f8825fb0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88200cc8e0 con 0x7f88200adb50 2023-10-16T10:05:06.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.535+0000 7f881ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f88200b0b20 0x7f8820164f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:06.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.535+0000 7f881ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f88200b0b20 0x7f8820164f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42544/0 (socket says 172.21.15.84:42544) 2023-10-16T10:05:06.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.535+0000 7f881ffff640 1 -- 172.21.15.84:0/3828674832 learned_addr learned my addr 172.21.15.84:0/3828674832 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:06.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.535+0000 7f881ffff640 1 -- 172.21.15.84:0/3828674832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88200adb50 msgr2=0x7f882006f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:06.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.535+0000 7f881f7fe640 1 --2- 172.21.15.84:0/3828674832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88200adb50 0x7f882006f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:06.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.535+0000 7f881ffff640 1 --2- 172.21.15.84:0/3828674832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88200adb50 0x7f882006f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.535+0000 7f881ffff640 1 -- 172.21.15.84:0/3828674832 >> v1:172.21.15.157:6789/0 conn(0x7f8820070fd0 legacy=0x7f88200b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:06.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.535+0000 7f881ffff640 1 -- 172.21.15.84:0/3828674832 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88200ccbc0 con 0x7f88200b0b20 2023-10-16T10:05:06.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.535+0000 7f881f7fe640 1 --2- 172.21.15.84:0/3828674832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88200adb50 0x7f882006f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:06.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.536+0000 7f881ffff640 1 --2- 172.21.15.84:0/3828674832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88200b0b20 0x7f8820164f60 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8818009e50 tx=0x7f881802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431f33bbde7f3225 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:06.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.537+0000 7f881effd640 1 -- 172.21.15.84:0/3828674832 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881802ca70 con 0x7f88200b0b20 2023-10-16T10:05:06.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.537+0000 7f881effd640 1 -- 172.21.15.84:0/3828674832 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881802cbd0 con 0x7f88200b0b20 2023-10-16T10:05:06.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.537+0000 7f881effd640 1 -- 172.21.15.84:0/3828674832 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88180343f0 con 0x7f88200b0b20 2023-10-16T10:05:06.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.537+0000 7f8825fb0640 1 -- 172.21.15.84:0/3828674832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88200b0b20 msgr2=0x7f8820164f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:06.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.537+0000 7f8825fb0640 1 --2- 172.21.15.84:0/3828674832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88200b0b20 0x7f8820164f60 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8818009e50 tx=0x7f881802bed0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.537+0000 7f8825fb0640 1 -- 172.21.15.84:0/3828674832 shutdown_connections 2023-10-16T10:05:06.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.537+0000 7f8825fb0640 1 --2- 172.21.15.84:0/3828674832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88200b0b20 0x7f8820164f60 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.537+0000 7f8825fb0640 1 --2- 172.21.15.84:0/3828674832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88200adb50 0x7f882006f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.537+0000 7f8825fb0640 1 -- 172.21.15.84:0/3828674832 >> 172.21.15.84:0/3828674832 conn(0x7f88200cb410 msgr2=0x7f88200cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:06.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.537+0000 7f8825fb0640 1 -- 172.21.15.84:0/3828674832 shutdown_connections 2023-10-16T10:05:06.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.538+0000 7f8825fb0640 1 -- 172.21.15.84:0/3828674832 wait complete. 2023-10-16T10:05:06.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.539+0000 7f8825fb0640 1 Processor -- start 2023-10-16T10:05:06.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.539+0000 7f8825fb0640 1 -- start start 2023-10-16T10:05:06.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.540+0000 7f8825fb0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8820070fd0 0x7f88200c38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:06.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.540+0000 7f8825fb0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88200adb50 0x7f88201634e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:06.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.540+0000 7f8825fb0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88200b0b20 0x7f882015fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:06.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.540+0000 7f8825fb0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88200c3eb0 con 0x7f88200b0b20 2023-10-16T10:05:06.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.540+0000 7f8825fb0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88200c4190 con 0x7f8820070fd0 2023-10-16T10:05:06.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.540+0000 7f8825fb0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88200c4470 con 0x7f88200adb50 2023-10-16T10:05:06.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.540+0000 7f881ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88200b0b20 0x7f882015fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:06.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.541+0000 7f881ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88200b0b20 0x7f882015fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42560/0 (socket says 172.21.15.84:42560) 2023-10-16T10:05:06.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.541+0000 7f881ffff640 1 -- 172.21.15.84:0/1438524795 learned_addr learned my addr 172.21.15.84:0/1438524795 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:06.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.541+0000 7f881f7fe640 1 --2- 172.21.15.84:0/1438524795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8820070fd0 0x7f88200c38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:06.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.541+0000 7f8817fff640 1 --2- 172.21.15.84:0/1438524795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88200adb50 0x7f88201634e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:06.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.541+0000 7f881ffff640 1 -- 172.21.15.84:0/1438524795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88200adb50 msgr2=0x7f88201634e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:06.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.541+0000 7f881ffff640 1 --2- 172.21.15.84:0/1438524795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88200adb50 0x7f88201634e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.541+0000 7f881ffff640 1 -- 172.21.15.84:0/1438524795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8820070fd0 msgr2=0x7f88200c38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:06.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.541+0000 7f881ffff640 1 --2- 172.21.15.84:0/1438524795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8820070fd0 0x7f88200c38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.541+0000 7f881ffff640 1 -- 172.21.15.84:0/1438524795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8818009b30 con 0x7f88200b0b20 2023-10-16T10:05:06.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.541+0000 7f881ffff640 1 --2- 172.21.15.84:0/1438524795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88200b0b20 0x7f882015fcf0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8818036040 tx=0x7f8818004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:06.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.542+0000 7f881cff9640 1 -- 172.21.15.84:0/1438524795 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8818040670 con 0x7f88200b0b20 2023-10-16T10:05:06.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.542+0000 7f881cff9640 1 -- 172.21.15.84:0/1438524795 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88180407f0 con 0x7f88200b0b20 2023-10-16T10:05:06.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.542+0000 7f881cff9640 1 -- 172.21.15.84:0/1438524795 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8818040ba0 con 0x7f88200b0b20 2023-10-16T10:05:06.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.542+0000 7f8825fb0640 1 -- 172.21.15.84:0/1438524795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88200c4750 con 0x7f88200b0b20 2023-10-16T10:05:06.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.542+0000 7f8825fb0640 1 -- 172.21.15.84:0/1438524795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882015f1a0 con 0x7f88200b0b20 2023-10-16T10:05:06.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.543+0000 7f8825fb0640 1 -- 172.21.15.84:0/1438524795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88200c48e0 con 0x7f88200b0b20 2023-10-16T10:05:06.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.544+0000 7f881cff9640 1 -- 172.21.15.84:0/1438524795 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8818040d00 con 0x7f88200b0b20 2023-10-16T10:05:06.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.545+0000 7f881cff9640 1 --2- 172.21.15.84:0/1438524795 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f880006afc0 0x7f880006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:06.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.545+0000 7f881cff9640 1 -- 172.21.15.84:0/1438524795 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f88180ac2f0 con 0x7f88200b0b20 2023-10-16T10:05:06.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.545+0000 7f881f7fe640 1 --2- 172.21.15.84:0/1438524795 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f880006afc0 0x7f880006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:06.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.546+0000 7f881f7fe640 1 --2- 172.21.15.84:0/1438524795 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f880006afc0 0x7f880006d480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f881000a990 tx=0x7f8810008040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:06.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.549+0000 7f881cff9640 1 -- 172.21.15.84:0/1438524795 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8818077700 con 0x7f88200b0b20 2023-10-16T10:05:06.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.807+0000 7f8825fb0640 1 -- 172.21.15.84:0/1438524795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f882004e160 con 0x7f88200b0b20 2023-10-16T10:05:06.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.808+0000 7f881cff9640 1 -- 172.21.15.84:0/1438524795 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f881807ad20 con 0x7f88200b0b20 2023-10-16T10:05:06.810 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:06.810 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T10:05:06.810 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-10-16T10:05:06.811 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.811 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-10-16T10:05:06.811 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l6dhjKRAA0pMvY6AK6qtFXDprNhwsqg==", 2023-10-16T10:05:06.811 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.811 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:05:06.811 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:05:06.812 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:05:06.812 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:06.812 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.812 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:06.812 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.812 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-10-16T10:05:06.813 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lFpMxEBAA0TLPRt5q3FwwBvMG019cjQ==", 2023-10-16T10:05:06.813 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.813 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:05:06.813 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:05:06.813 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:05:06.814 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:06.814 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.814 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:06.814 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.814 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-10-16T10:05:06.815 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l73RBOhAAOMEPxfUAMHhn5Blyvd5oMw==", 2023-10-16T10:05:06.815 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.815 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:05:06.815 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:05:06.815 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:05:06.816 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:06.816 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.816 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:06.816 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.816 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-10-16T10:05:06.817 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lcTIRIhAA0+FWm4b4yw77LD3+WKbk7w==", 2023-10-16T10:05:06.817 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.817 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:05:06.817 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:05:06.817 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:05:06.818 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:06.818 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.818 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:06.818 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.819 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-10-16T10:05:06.819 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDYBi1l1zBLERAA3YLW3xIYQoRVw7KSkwN3Og==", 2023-10-16T10:05:06.819 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.819 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:05:06.819 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:05:06.819 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:06.820 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.820 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:06.820 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.820 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-10-16T10:05:06.820 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDbBi1lFUUxKRAAQbus8SmYprJRmWTjoCznNA==", 2023-10-16T10:05:06.820 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.821 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:05:06.821 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:05:06.821 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:06.821 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.821 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:06.821 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.822 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-10-16T10:05:06.822 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDeBi1l+9lBMxAArl6sb/h15uBG/a9lVH40Hg==", 2023-10-16T10:05:06.822 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.822 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:05:06.823 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:05:06.823 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:06.823 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.823 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:06.823 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.823 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-10-16T10:05:06.824 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDiBi1l2rnEERAA3oRD71xa0fEOynM8zKuBEg==", 2023-10-16T10:05:06.824 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.824 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:05:06.824 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:05:06.824 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:06.825 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.825 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:06.825 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.825 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-10-16T10:05:06.825 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDnBi1lR7oLChAAZto8fDovDbpOd3Fcnc7C6w==", 2023-10-16T10:05:06.826 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.826 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:05:06.826 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:05:06.826 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:06.827 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.827 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:06.827 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.827 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-10-16T10:05:06.827 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDqBi1lmjg/JBAAlINKRGBc0/UWJJHMVFw57w==", 2023-10-16T10:05:06.827 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.828 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:05:06.828 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:05:06.828 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:06.828 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.828 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:06.828 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.829 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-10-16T10:05:06.829 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDuBi1lu0kMARAAtuSeM8F3yr8CVrc3p1biaQ==", 2023-10-16T10:05:06.829 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.829 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:05:06.829 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:05:06.830 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:06.830 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.830 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:06.830 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.831 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-10-16T10:05:06.831 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDxBi1lf6mNGBAAHGbqzJ0fmDxQIsasboHMfg==", 2023-10-16T10:05:06.831 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.832 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:05:06.832 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:05:06.832 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:06.832 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.832 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:06.832 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.833 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-10-16T10:05:06.833 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ==", 2023-10-16T10:05:06.833 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.833 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:05:06.833 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow r", 2023-10-16T10:05:06.834 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T10:05:06.834 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.834 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:06.834 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.834 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-10-16T10:05:06.835 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1lHU+4BxAAd8ET8hQ8Y+EUwdTJr9/+Ig==", 2023-10-16T10:05:06.835 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.835 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:05:06.835 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:05:06.835 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:05:06.836 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:05:06.836 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.836 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:06.836 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.837 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-10-16T10:05:06.837 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lA1BOORAAFNyfqgEF34MVR9YX3KBKkQ==", 2023-10-16T10:05:06.837 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.837 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:05:06.837 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:05:06.838 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:05:06.838 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:05:06.838 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.838 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:06.839 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.839 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-10-16T10:05:06.839 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1l848oERAANMfrZz9JrlgNVh7ZFbtnaA==", 2023-10-16T10:05:06.839 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.839 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:05:06.840 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:05:06.840 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:05:06.840 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:05:06.840 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.840 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:06.841 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.841 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-10-16T10:05:06.841 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDRBi1lTAxTGRAAMptxnxy3Qit/cTCXie6CrQ==", 2023-10-16T10:05:06.841 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.841 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:05:06.842 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:05:06.842 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:05:06.842 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:06.842 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.842 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:06.843 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.843 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-10-16T10:05:06.843 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lPuJxHRAABAnGPVCrkq0JR4Y6iFGZlw==", 2023-10-16T10:05:06.843 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.843 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:05:06.844 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T10:05:06.844 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:06.844 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.844 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:06.844 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:06.845 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-10-16T10:05:06.845 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lLxnKEBAA+a/pOHUp7Sg3m4weAust5w==", 2023-10-16T10:05:06.845 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:06.845 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:05:06.845 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T10:05:06.846 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:06.846 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.846 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:06.846 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-10-16T10:05:06.847 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T10:05:06.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.810+0000 7f8825fb0640 1 -- 172.21.15.84:0/1438524795 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f880006afc0 msgr2=0x7f880006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:06.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.810+0000 7f8825fb0640 1 --2- 172.21.15.84:0/1438524795 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f880006afc0 0x7f880006d480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f881000a990 tx=0x7f8810008040 comp rx=0 tx=0).stop 2023-10-16T10:05:06.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.810+0000 7f8825fb0640 1 -- 172.21.15.84:0/1438524795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88200b0b20 msgr2=0x7f882015fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:06.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.810+0000 7f8825fb0640 1 --2- 172.21.15.84:0/1438524795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88200b0b20 0x7f882015fcf0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8818036040 tx=0x7f8818004a40 comp rx=0 tx=0).stop 2023-10-16T10:05:06.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.810+0000 7f8825fb0640 1 -- 172.21.15.84:0/1438524795 shutdown_connections 2023-10-16T10:05:06.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.811+0000 7f8825fb0640 1 --2- 172.21.15.84:0/1438524795 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f880006afc0 0x7f880006d480 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.811+0000 7f8825fb0640 1 --2- 172.21.15.84:0/1438524795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88200b0b20 0x7f882015fcf0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.811+0000 7f8825fb0640 1 --2- 172.21.15.84:0/1438524795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88200adb50 0x7f88201634e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.811+0000 7f8825fb0640 1 --2- 172.21.15.84:0/1438524795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8820070fd0 0x7f88200c38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:06.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.811+0000 7f8825fb0640 1 -- 172.21.15.84:0/1438524795 >> 172.21.15.84:0/1438524795 conn(0x7f88200cb410 msgr2=0x7f882006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:06.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.811+0000 7f8825fb0640 1 -- 172.21.15.84:0/1438524795 shutdown_connections 2023-10-16T10:05:06.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:06.811+0000 7f8825fb0640 1 -- 172.21.15.84:0/1438524795 wait complete. 2023-10-16T10:05:06.853 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:05:07.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.068+0000 7f4ae2761640 1 Processor -- start 2023-10-16T10:05:07.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.068+0000 7f4ae2761640 1 -- start start 2023-10-16T10:05:07.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.069+0000 7f4ae2761640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4adc159f00 0x7f4adc0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:07.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.069+0000 7f4ae2761640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4adc06f7f0 con 0x7f4adc159b20 2023-10-16T10:05:07.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.069+0000 7f4ae2761640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4adc06c5e0 con 0x7f4adc15d880 2023-10-16T10:05:07.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.069+0000 7f4ae2761640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4adc06c8c0 con 0x7f4adc159f00 2023-10-16T10:05:07.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.069+0000 7f4adb7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4adc159f00 0x7f4adc0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:07.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.070+0000 7f4adb7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4adc159f00 0x7f4adc0b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42572/0 (socket says 172.21.15.84:42572) 2023-10-16T10:05:07.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.070+0000 7f4adb7fe640 1 -- 172.21.15.84:0/1917555616 learned_addr learned my addr 172.21.15.84:0/1917555616 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:07.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.070+0000 7f4adb7fe640 1 -- 172.21.15.84:0/1917555616 >> v1:172.21.15.157:6789/0 conn(0x7f4adc15d880 legacy=0x7f4adc06fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:07.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.070+0000 7f4adb7fe640 1 -- 172.21.15.84:0/1917555616 >> v1:172.21.15.84:6789/0 conn(0x7f4adc159b20 legacy=0x7f4adc06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:07.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.070+0000 7f4adb7fe640 1 -- 172.21.15.84:0/1917555616 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc06cba0 con 0x7f4adc159f00 2023-10-16T10:05:07.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.071+0000 7f4adb7fe640 1 --2- 172.21.15.84:0/1917555616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4adc159f00 0x7f4adc0b0410 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009870 tx=0x7f4ad002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de1253619216e9f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:07.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.071+0000 7f4adaffd640 1 -- 172.21.15.84:0/1917555616 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0030850 con 0x7f4adc159f00 2023-10-16T10:05:07.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.071+0000 7f4adaffd640 1 -- 172.21.15.84:0/1917555616 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad00309b0 con 0x7f4adc159f00 2023-10-16T10:05:07.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.071+0000 7f4adaffd640 1 -- 172.21.15.84:0/1917555616 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0030d40 con 0x7f4adc159f00 2023-10-16T10:05:07.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.072+0000 7f4ae2761640 1 -- 172.21.15.84:0/1917555616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4adc159f00 msgr2=0x7f4adc0b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:07.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.072+0000 7f4ae2761640 1 --2- 172.21.15.84:0/1917555616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4adc159f00 0x7f4adc0b0410 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009870 tx=0x7f4ad002b990 comp rx=0 tx=0).stop 2023-10-16T10:05:07.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.072+0000 7f4ae2761640 1 -- 172.21.15.84:0/1917555616 shutdown_connections 2023-10-16T10:05:07.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.072+0000 7f4ae2761640 1 --2- 172.21.15.84:0/1917555616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4adc159f00 0x7f4adc0b0410 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.072+0000 7f4ae2761640 1 -- 172.21.15.84:0/1917555616 >> 172.21.15.84:0/1917555616 conn(0x7f4adc0cb410 msgr2=0x7f4adc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:07.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.072+0000 7f4ae2761640 1 -- 172.21.15.84:0/1917555616 shutdown_connections 2023-10-16T10:05:07.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.072+0000 7f4ae2761640 1 -- 172.21.15.84:0/1917555616 wait complete. 2023-10-16T10:05:07.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.074+0000 7f4ae2761640 1 Processor -- start 2023-10-16T10:05:07.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.074+0000 7f4ae2761640 1 -- start start 2023-10-16T10:05:07.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.075+0000 7f4ae2761640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4adc159b20 0x7f4adc0b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:07.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.075+0000 7f4ae2761640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4adc159f00 0x7f4adc0ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:07.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.075+0000 7f4ae2761640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4adc15d880 0x7f4adc0ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:07.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.075+0000 7f4ae2761640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc0c6d80 con 0x7f4adc159b20 2023-10-16T10:05:07.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.075+0000 7f4ae2761640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc0b1e30 con 0x7f4adc159f00 2023-10-16T10:05:07.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.075+0000 7f4ae2761640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc0b2110 con 0x7f4adc15d880 2023-10-16T10:05:07.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.075+0000 7f4adbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4adc159b20 0x7f4adc0b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:07.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.075+0000 7f4adbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4adc159b20 0x7f4adc0b17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42578/0 (socket says 172.21.15.84:42578) 2023-10-16T10:05:07.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.075+0000 7f4adbfff640 1 -- 172.21.15.84:0/1022418902 learned_addr learned my addr 172.21.15.84:0/1022418902 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:07.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.075+0000 7f4adbfff640 1 -- 172.21.15.84:0/1022418902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4adc15d880 msgr2=0x7f4adc0ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:07.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.075+0000 7f4adbfff640 1 --2- 172.21.15.84:0/1022418902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4adc15d880 0x7f4adc0ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.075+0000 7f4adbfff640 1 -- 172.21.15.84:0/1022418902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4adc159f00 msgr2=0x7f4adc0ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:07.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.075+0000 7f4adbfff640 1 --2- 172.21.15.84:0/1022418902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4adc159f00 0x7f4adc0ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.075+0000 7f4adbfff640 1 -- 172.21.15.84:0/1022418902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad00094d0 con 0x7f4adc159b20 2023-10-16T10:05:07.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.076+0000 7f4adbfff640 1 --2- 172.21.15.84:0/1022418902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4adc159b20 0x7f4adc0b17a0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00c400 tx=0x7f4acc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:07.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.076+0000 7f4ad8ff9640 1 -- 172.21.15.84:0/1022418902 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc0052d0 con 0x7f4adc159b20 2023-10-16T10:05:07.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.076+0000 7f4ad8ff9640 1 -- 172.21.15.84:0/1022418902 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4acc005430 con 0x7f4adc159b20 2023-10-16T10:05:07.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.076+0000 7f4ad8ff9640 1 -- 172.21.15.84:0/1022418902 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc003830 con 0x7f4adc159b20 2023-10-16T10:05:07.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.076+0000 7f4ae2761640 1 -- 172.21.15.84:0/1022418902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4adc0b23f0 con 0x7f4adc159b20 2023-10-16T10:05:07.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.076+0000 7f4ae2761640 1 -- 172.21.15.84:0/1022418902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4adc15ce60 con 0x7f4adc159b20 2023-10-16T10:05:07.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.077+0000 7f4ae2761640 1 -- 172.21.15.84:0/1022418902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4adc0c6ef0 con 0x7f4adc159b20 2023-10-16T10:05:07.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.078+0000 7f4ad8ff9640 1 -- 172.21.15.84:0/1022418902 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4acc003990 con 0x7f4adc159b20 2023-10-16T10:05:07.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.079+0000 7f4ad8ff9640 1 --2- 172.21.15.84:0/1022418902 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f4ac806af90 0x7f4ac806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:07.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.079+0000 7f4ad8ff9640 1 -- 172.21.15.84:0/1022418902 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4acc0889c0 con 0x7f4adc159b20 2023-10-16T10:05:07.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.079+0000 7f4adb7fe640 1 --2- 172.21.15.84:0/1022418902 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f4ac806af90 0x7f4ac806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:07.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.080+0000 7f4adb7fe640 1 --2- 172.21.15.84:0/1022418902 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f4ac806af90 0x7f4ac806d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0036000 tx=0x7f4ad003b040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:07.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.083+0000 7f4ad8ff9640 1 -- 172.21.15.84:0/1022418902 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4adc0c6ef0 con 0x7f4adc159b20 2023-10-16T10:05:07.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.277+0000 7f4ae2761640 1 -- 172.21.15.84:0/1022418902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4adc0c6ec0 con 0x7f4adc159b20 2023-10-16T10:05:07.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.280+0000 7f4ad8ff9640 1 -- 172.21.15.84:0/1022418902 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+12833 (secure 0 0 0) 0x7f4acc017090 con 0x7f4adc159b20 2023-10-16T10:05:07.281 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:07.282 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":91,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:05:06.315576+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:05:07.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.282+0000 7f4ae2761640 1 -- 172.21.15.84:0/1022418902 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f4ac806af90 msgr2=0x7f4ac806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:07.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.282+0000 7f4ae2761640 1 --2- 172.21.15.84:0/1022418902 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f4ac806af90 0x7f4ac806d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0036000 tx=0x7f4ad003b040 comp rx=0 tx=0).stop 2023-10-16T10:05:07.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.282+0000 7f4ae2761640 1 -- 172.21.15.84:0/1022418902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4adc159b20 msgr2=0x7f4adc0b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:07.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.282+0000 7f4ae2761640 1 --2- 172.21.15.84:0/1022418902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4adc159b20 0x7f4adc0b17a0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00c400 tx=0x7f4acc00c8c0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.283+0000 7f4ae2761640 1 -- 172.21.15.84:0/1022418902 shutdown_connections 2023-10-16T10:05:07.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.283+0000 7f4ae2761640 1 --2- 172.21.15.84:0/1022418902 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f4ac806af90 0x7f4ac806d450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.283+0000 7f4ae2761640 1 --2- 172.21.15.84:0/1022418902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4adc15d880 0x7f4adc0ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.283+0000 7f4ae2761640 1 --2- 172.21.15.84:0/1022418902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4adc159f00 0x7f4adc0ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.283+0000 7f4ae2761640 1 --2- 172.21.15.84:0/1022418902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4adc159b20 0x7f4adc0b17a0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.283+0000 7f4ae2761640 1 -- 172.21.15.84:0/1022418902 >> 172.21.15.84:0/1022418902 conn(0x7f4adc0cb410 msgr2=0x7f4adc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:07.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.283+0000 7f4ae2761640 1 -- 172.21.15.84:0/1022418902 shutdown_connections 2023-10-16T10:05:07.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.283+0000 7f4ae2761640 1 -- 172.21.15.84:0/1022418902 wait complete. 2023-10-16T10:05:07.303 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:05:07.304 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:05:07.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.507+0000 7f00da761640 1 Processor -- start 2023-10-16T10:05:07.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.507+0000 7f00da761640 1 -- start start 2023-10-16T10:05:07.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.509+0000 7f00da761640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00d4160ad0 0x7f00d40ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:07.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.509+0000 7f00da761640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f00d40ce350 0x7f00d4160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:07.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.509+0000 7f00da761640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f00d40ae260 0x7f00d40ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:07.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.509+0000 7f00da761640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d40ce800 con 0x7f00d40ae260 2023-10-16T10:05:07.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.509+0000 7f00da761640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d40cdec0 con 0x7f00d40ce350 2023-10-16T10:05:07.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.509+0000 7f00da761640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d40acdd0 con 0x7f00d4160ad0 2023-10-16T10:05:07.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.509+0000 7f00d8cfe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f00d40ae260 0x7f00d40ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:07.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.509+0000 7f00d8cfe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f00d40ae260 0x7f00d40ccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42580/0 (socket says 172.21.15.84:42580) 2023-10-16T10:05:07.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.509+0000 7f00d8cfe640 1 -- 172.21.15.84:0/2811437726 learned_addr learned my addr 172.21.15.84:0/2811437726 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:07.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.510+0000 7f00d3fff640 1 --2- 172.21.15.84:0/2811437726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00d4160ad0 0x7f00d40ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:07.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.510+0000 7f00cbfff640 1 --2- 172.21.15.84:0/2811437726 >> v2:172.21.15.157:3300/0 conn(0x7f00d40ce350 0x7f00d4160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:07.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.510+0000 7f00d8cfe640 1 -- 172.21.15.84:0/2811437726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00d4160ad0 msgr2=0x7f00d40ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:07.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.510+0000 7f00d8cfe640 1 --2- 172.21.15.84:0/2811437726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00d4160ad0 0x7f00d40ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.510+0000 7f00d8cfe640 1 -- 172.21.15.84:0/2811437726 >> v2:172.21.15.157:3300/0 conn(0x7f00d40ce350 msgr2=0x7f00d4160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:07.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.510+0000 7f00d8cfe640 1 --2- 172.21.15.84:0/2811437726 >> v2:172.21.15.157:3300/0 conn(0x7f00d40ce350 0x7f00d4160eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.510+0000 7f00d8cfe640 1 -- 172.21.15.84:0/2811437726 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d41613f0 con 0x7f00d40ae260 2023-10-16T10:05:07.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.510+0000 7f00d3fff640 1 --2- 172.21.15.84:0/2811437726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00d4160ad0 0x7f00d40ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:07.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.510+0000 7f00cbfff640 1 --2- 172.21.15.84:0/2811437726 >> v2:172.21.15.157:3300/0 conn(0x7f00d40ce350 0x7f00d4160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:07.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.511+0000 7f00d8cfe640 1 --2- 172.21.15.84:0/2811437726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d40ae260 0x7f00d40ccd80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f00d406f6b0 tx=0x7f00cc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cabcb7692ccea0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:07.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.511+0000 7f00d37fe640 1 -- 172.21.15.84:0/2811437726 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc02ca80 con 0x7f00d40ae260 2023-10-16T10:05:07.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.511+0000 7f00d37fe640 1 -- 172.21.15.84:0/2811437726 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00cc02cbe0 con 0x7f00d40ae260 2023-10-16T10:05:07.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.511+0000 7f00d37fe640 1 -- 172.21.15.84:0/2811437726 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc0343f0 con 0x7f00d40ae260 2023-10-16T10:05:07.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.512+0000 7f00da761640 1 -- 172.21.15.84:0/2811437726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d40ae260 msgr2=0x7f00d40ccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:07.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.512+0000 7f00da761640 1 --2- 172.21.15.84:0/2811437726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d40ae260 0x7f00d40ccd80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f00d406f6b0 tx=0x7f00cc02bee0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.512+0000 7f00da761640 1 -- 172.21.15.84:0/2811437726 shutdown_connections 2023-10-16T10:05:07.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.512+0000 7f00da761640 1 --2- 172.21.15.84:0/2811437726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d40ae260 0x7f00d40ccd80 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.512+0000 7f00da761640 1 --2- 172.21.15.84:0/2811437726 >> v2:172.21.15.157:3300/0 conn(0x7f00d40ce350 0x7f00d4160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.512+0000 7f00da761640 1 --2- 172.21.15.84:0/2811437726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00d4160ad0 0x7f00d40ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.512+0000 7f00da761640 1 -- 172.21.15.84:0/2811437726 >> 172.21.15.84:0/2811437726 conn(0x7f00d406bc10 msgr2=0x7f00d406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:07.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.512+0000 7f00da761640 1 -- 172.21.15.84:0/2811437726 shutdown_connections 2023-10-16T10:05:07.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.512+0000 7f00da761640 1 -- 172.21.15.84:0/2811437726 wait complete. 2023-10-16T10:05:07.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.514+0000 7f00da761640 1 Processor -- start 2023-10-16T10:05:07.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.514+0000 7f00da761640 1 -- start start 2023-10-16T10:05:07.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.515+0000 7f00da761640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d40ae260 0x7f00d40c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:07.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.515+0000 7f00da761640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d40ce350 0x7f00d40b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:07.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.515+0000 7f00da761640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00d4160ad0 0x7f00d40c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:07.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.515+0000 7f00da761640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d40b4ee0 con 0x7f00d40ce350 2023-10-16T10:05:07.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.515+0000 7f00da761640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d415d2e0 con 0x7f00d40ae260 2023-10-16T10:05:07.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.515+0000 7f00da761640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d415e0b0 con 0x7f00d4160ad0 2023-10-16T10:05:07.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.515+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d40ce350 0x7f00d40b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:07.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.515+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d40ce350 0x7f00d40b49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42592/0 (socket says 172.21.15.84:42592) 2023-10-16T10:05:07.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.515+0000 7f00cbfff640 1 -- 172.21.15.84:0/1801887925 learned_addr learned my addr 172.21.15.84:0/1801887925 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:07.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.516+0000 7f00d3fff640 1 --2- 172.21.15.84:0/1801887925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d40ae260 0x7f00d40c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:07.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.516+0000 7f00d8cfe640 1 --2- 172.21.15.84:0/1801887925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00d4160ad0 0x7f00d40c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:07.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.516+0000 7f00cbfff640 1 -- 172.21.15.84:0/1801887925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00d4160ad0 msgr2=0x7f00d40c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:07.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.516+0000 7f00cbfff640 1 --2- 172.21.15.84:0/1801887925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00d4160ad0 0x7f00d40c6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.516+0000 7f00cbfff640 1 -- 172.21.15.84:0/1801887925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d40ae260 msgr2=0x7f00d40c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:07.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.516+0000 7f00cbfff640 1 --2- 172.21.15.84:0/1801887925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d40ae260 0x7f00d40c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.516+0000 7f00cbfff640 1 -- 172.21.15.84:0/1801887925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc009b30 con 0x7f00d40ce350 2023-10-16T10:05:07.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.516+0000 7f00cbfff640 1 --2- 172.21.15.84:0/1801887925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d40ce350 0x7f00d40b49a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00c7f0 tx=0x7f00bc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:07.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.516+0000 7f00d17fa640 1 -- 172.21.15.84:0/1801887925 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc007820 con 0x7f00d40ce350 2023-10-16T10:05:07.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.516+0000 7f00d17fa640 1 -- 172.21.15.84:0/1801887925 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc007980 con 0x7f00d40ce350 2023-10-16T10:05:07.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.516+0000 7f00d17fa640 1 -- 172.21.15.84:0/1801887925 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc007d10 con 0x7f00d40ce350 2023-10-16T10:05:07.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.516+0000 7f00da761640 1 -- 172.21.15.84:0/1801887925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d415d530 con 0x7f00d40ce350 2023-10-16T10:05:07.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.517+0000 7f00da761640 1 -- 172.21.15.84:0/1801887925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d415daf0 con 0x7f00d40ce350 2023-10-16T10:05:07.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.518+0000 7f00da761640 1 -- 172.21.15.84:0/1801887925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d41f57b0 con 0x7f00d40ce350 2023-10-16T10:05:07.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.519+0000 7f00d17fa640 1 -- 172.21.15.84:0/1801887925 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f00bc0113f0 con 0x7f00d40ce350 2023-10-16T10:05:07.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.519+0000 7f00d17fa640 1 --2- 172.21.15.84:0/1801887925 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f00b406af90 0x7f00b406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:07.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.520+0000 7f00d17fa640 1 -- 172.21.15.84:0/1801887925 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f00bc089870 con 0x7f00d40ce350 2023-10-16T10:05:07.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.520+0000 7f00d3fff640 1 --2- 172.21.15.84:0/1801887925 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f00b406af90 0x7f00b406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:07.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.520+0000 7f00d3fff640 1 --2- 172.21.15.84:0/1801887925 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f00b406af90 0x7f00b406d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f00c400a990 tx=0x7f00c4008040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:07.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.524+0000 7f00d17fa640 1 -- 172.21.15.84:0/1801887925 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f00bc054a40 con 0x7f00d40ce350 2023-10-16T10:05:07.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.702+0000 7f00da761640 1 -- 172.21.15.84:0/1801887925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f00d404e160 con 0x7f00d40ce350 2023-10-16T10:05:07.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.703+0000 7f00d17fa640 1 -- 172.21.15.84:0/1801887925 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f00bc058060 con 0x7f00d40ce350 2023-10-16T10:05:07.705 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:05:07.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.705+0000 7f00da761640 1 -- 172.21.15.84:0/1801887925 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f00b406af90 msgr2=0x7f00b406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:07.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.705+0000 7f00da761640 1 --2- 172.21.15.84:0/1801887925 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f00b406af90 0x7f00b406d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f00c400a990 tx=0x7f00c4008040 comp rx=0 tx=0).stop 2023-10-16T10:05:07.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.705+0000 7f00da761640 1 -- 172.21.15.84:0/1801887925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d40ce350 msgr2=0x7f00d40b49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:07.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.705+0000 7f00da761640 1 --2- 172.21.15.84:0/1801887925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d40ce350 0x7f00d40b49a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00c7f0 tx=0x7f00bc00ccb0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.706+0000 7f00da761640 1 -- 172.21.15.84:0/1801887925 shutdown_connections 2023-10-16T10:05:07.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.706+0000 7f00da761640 1 --2- 172.21.15.84:0/1801887925 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f00b406af90 0x7f00b406d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.706+0000 7f00da761640 1 --2- 172.21.15.84:0/1801887925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00d4160ad0 0x7f00d40c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.706+0000 7f00da761640 1 --2- 172.21.15.84:0/1801887925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d40ce350 0x7f00d40b49a0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.706+0000 7f00da761640 1 --2- 172.21.15.84:0/1801887925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d40ae260 0x7f00d40c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.706+0000 7f00da761640 1 -- 172.21.15.84:0/1801887925 >> 172.21.15.84:0/1801887925 conn(0x7f00d406bc10 msgr2=0x7f00d40ca5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:07.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.706+0000 7f00da761640 1 -- 172.21.15.84:0/1801887925 shutdown_connections 2023-10-16T10:05:07.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.706+0000 7f00da761640 1 -- 172.21.15.84:0/1801887925 wait complete. 2023-10-16T10:05:07.724 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T10:05:07.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.920+0000 7f54be2df640 1 Processor -- start 2023-10-16T10:05:07.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.921+0000 7f54be2df640 1 -- start start 2023-10-16T10:05:07.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.922+0000 7f54be2df640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f54b815b320 0x7f54b80cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:07.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.922+0000 7f54be2df640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b815b9d0 con 0x7f54b80ad970 2023-10-16T10:05:07.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.922+0000 7f54be2df640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b815bcb0 con 0x7f54b806c040 2023-10-16T10:05:07.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.922+0000 7f54be2df640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b815b7d0 con 0x7f54b815b320 2023-10-16T10:05:07.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.922+0000 7f54bcadc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f54b80ad970 0x7f54b80cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47024/0 (socket says 172.21.15.84:47024) 2023-10-16T10:05:07.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.922+0000 7f54bcadc640 1 -- 172.21.15.84:0/172801721 learned_addr learned my addr 172.21.15.84:0/172801721 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:07.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.922+0000 7f54bdade640 1 --2- 172.21.15.84:0/172801721 >> v2:172.21.15.84:3300/0 conn(0x7f54b815b320 0x7f54b80cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:07.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.923+0000 7f54bdade640 1 -- 172.21.15.84:0/172801721 >> v1:172.21.15.157:6789/0 conn(0x7f54b806c040 legacy=0x7f54b806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:07.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.923+0000 7f54bdade640 1 -- 172.21.15.84:0/172801721 >> v1:172.21.15.84:6789/0 conn(0x7f54b80ad970 legacy=0x7f54b80cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:07.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.923+0000 7f54bdade640 1 -- 172.21.15.84:0/172801721 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b815bf90 con 0x7f54b815b320 2023-10-16T10:05:07.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.924+0000 7f54bdade640 1 --2- 172.21.15.84:0/172801721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54b815b320 0x7f54b80cd180 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f54b0009dc0 tx=0x7f54b002be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4607cebbbd0d0321 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:07.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.924+0000 7f54abfff640 1 -- 172.21.15.84:0/172801721 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b002cc40 con 0x7f54b815b320 2023-10-16T10:05:07.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.924+0000 7f54abfff640 1 -- 172.21.15.84:0/172801721 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b002cda0 con 0x7f54b815b320 2023-10-16T10:05:07.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.924+0000 7f54abfff640 1 -- 172.21.15.84:0/172801721 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b0034660 con 0x7f54b815b320 2023-10-16T10:05:07.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.925+0000 7f54be2df640 1 -- 172.21.15.84:0/172801721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54b815b320 msgr2=0x7f54b80cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:07.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.925+0000 7f54be2df640 1 --2- 172.21.15.84:0/172801721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54b815b320 0x7f54b80cd180 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f54b0009dc0 tx=0x7f54b002be90 comp rx=0 tx=0).stop 2023-10-16T10:05:07.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.925+0000 7f54be2df640 1 -- 172.21.15.84:0/172801721 shutdown_connections 2023-10-16T10:05:07.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.925+0000 7f54be2df640 1 --2- 172.21.15.84:0/172801721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54b815b320 0x7f54b80cd180 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.925+0000 7f54be2df640 1 -- 172.21.15.84:0/172801721 >> 172.21.15.84:0/172801721 conn(0x7f54b80ce6a0 msgr2=0x7f54b80ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:07.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.925+0000 7f54be2df640 1 -- 172.21.15.84:0/172801721 shutdown_connections 2023-10-16T10:05:07.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.925+0000 7f54be2df640 1 -- 172.21.15.84:0/172801721 wait complete. 2023-10-16T10:05:07.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.927+0000 7f54be2df640 1 Processor -- start 2023-10-16T10:05:07.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.927+0000 7f54be2df640 1 -- start start 2023-10-16T10:05:07.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.928+0000 7f54be2df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54b806c040 0x7f54b80b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:07.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.928+0000 7f54be2df640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54b80ad970 0x7f54b8159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:07.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.928+0000 7f54be2df640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54b815b320 0x7f54b8159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:07.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.928+0000 7f54be2df640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b8152650 con 0x7f54b80ad970 2023-10-16T10:05:07.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.928+0000 7f54be2df640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b80aefb0 con 0x7f54b806c040 2023-10-16T10:05:07.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.928+0000 7f54be2df640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b80ae100 con 0x7f54b815b320 2023-10-16T10:05:07.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.928+0000 7f54bcadc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54b80ad970 0x7f54b8159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:07.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.928+0000 7f54bcadc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54b80ad970 0x7f54b8159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42616/0 (socket says 172.21.15.84:42616) 2023-10-16T10:05:07.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.928+0000 7f54bcadc640 1 -- 172.21.15.84:0/535135856 learned_addr learned my addr 172.21.15.84:0/535135856 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:07.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.929+0000 7f54bcadc640 1 -- 172.21.15.84:0/535135856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54b815b320 msgr2=0x7f54b8159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:07.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.929+0000 7f54bcadc640 1 --2- 172.21.15.84:0/535135856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54b815b320 0x7f54b8159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.929+0000 7f54bcadc640 1 -- 172.21.15.84:0/535135856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54b806c040 msgr2=0x7f54b80b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:07.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.929+0000 7f54bd2dd640 1 --2- 172.21.15.84:0/535135856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54b806c040 0x7f54b80b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:07.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.929+0000 7f54bcadc640 1 --2- 172.21.15.84:0/535135856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54b806c040 0x7f54b80b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:07.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.929+0000 7f54bcadc640 1 -- 172.21.15.84:0/535135856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b0009a70 con 0x7f54b80ad970 2023-10-16T10:05:07.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.929+0000 7f54bd2dd640 1 --2- 172.21.15.84:0/535135856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54b806c040 0x7f54b80b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:07.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.929+0000 7f54bcadc640 1 --2- 172.21.15.84:0/535135856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54b80ad970 0x7f54b8159730 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00c500 tx=0x7f54ac00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:07.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.929+0000 7f54a9ffb640 1 -- 172.21.15.84:0/535135856 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac005ce0 con 0x7f54b80ad970 2023-10-16T10:05:07.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.929+0000 7f54a9ffb640 1 -- 172.21.15.84:0/535135856 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ac005e40 con 0x7f54b80ad970 2023-10-16T10:05:07.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.929+0000 7f54a9ffb640 1 -- 172.21.15.84:0/535135856 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0072f0 con 0x7f54b80ad970 2023-10-16T10:05:07.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.929+0000 7f54be2df640 1 -- 172.21.15.84:0/535135856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b80ae3a0 con 0x7f54b80ad970 2023-10-16T10:05:07.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.930+0000 7f54be2df640 1 -- 172.21.15.84:0/535135856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b80ae960 con 0x7f54b80ad970 2023-10-16T10:05:07.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.931+0000 7f54be2df640 1 -- 172.21.15.84:0/535135856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b806b330 con 0x7f54b80ad970 2023-10-16T10:05:07.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.932+0000 7f54a9ffb640 1 -- 172.21.15.84:0/535135856 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f54ac007510 con 0x7f54b80ad970 2023-10-16T10:05:07.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.933+0000 7f54a9ffb640 1 --2- 172.21.15.84:0/535135856 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f549c06af90 0x7f549c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:07.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.933+0000 7f54a9ffb640 1 -- 172.21.15.84:0/535135856 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f54ac0887f0 con 0x7f54b80ad970 2023-10-16T10:05:07.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.933+0000 7f54bd2dd640 1 --2- 172.21.15.84:0/535135856 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f549c06af90 0x7f549c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:07.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.933+0000 7f54bd2dd640 1 --2- 172.21.15.84:0/535135856 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f549c06af90 0x7f549c06d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f54b400a830 tx=0x7f54b4008040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:07.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:07.937+0000 7f54a9ffb640 1 -- 172.21.15.84:0/535135856 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f54ac053c20 con 0x7f54b80ad970 2023-10-16T10:05:08.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.130+0000 7f54be2df640 1 -- 172.21.15.84:0/535135856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f54b80af120 con 0x7f54b80ad970 2023-10-16T10:05:08.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.130+0000 7f54a9ffb640 1 -- 172.21.15.84:0/535135856 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v91) v1 ==== 84+0+65 (secure 0 0 0) 0x7f54ac017020 con 0x7f54b80ad970 2023-10-16T10:05:08.132 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:08.132 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T10:05:08.132 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:08.133 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-16T10:05:08.133 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-16T10:05:08.133 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:08.133 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T10:05:08.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.133+0000 7f54be2df640 1 -- 172.21.15.84:0/535135856 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f549c06af90 msgr2=0x7f549c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:08.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.133+0000 7f54be2df640 1 --2- 172.21.15.84:0/535135856 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f549c06af90 0x7f549c06d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f54b400a830 tx=0x7f54b4008040 comp rx=0 tx=0).stop 2023-10-16T10:05:08.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.133+0000 7f54be2df640 1 -- 172.21.15.84:0/535135856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54b80ad970 msgr2=0x7f54b8159730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:08.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.133+0000 7f54be2df640 1 --2- 172.21.15.84:0/535135856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54b80ad970 0x7f54b8159730 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00c500 tx=0x7f54ac00c9c0 comp rx=0 tx=0).stop 2023-10-16T10:05:08.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.133+0000 7f54be2df640 1 -- 172.21.15.84:0/535135856 shutdown_connections 2023-10-16T10:05:08.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.133+0000 7f54be2df640 1 --2- 172.21.15.84:0/535135856 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f549c06af90 0x7f549c06d450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:08.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.133+0000 7f54be2df640 1 --2- 172.21.15.84:0/535135856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54b815b320 0x7f54b8159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:08.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.133+0000 7f54be2df640 1 --2- 172.21.15.84:0/535135856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54b80ad970 0x7f54b8159730 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:08.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.133+0000 7f54be2df640 1 --2- 172.21.15.84:0/535135856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54b806c040 0x7f54b80b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:08.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.133+0000 7f54be2df640 1 -- 172.21.15.84:0/535135856 >> 172.21.15.84:0/535135856 conn(0x7f54b80ce6a0 msgr2=0x7f54b80c93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:08.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.133+0000 7f54be2df640 1 -- 172.21.15.84:0/535135856 shutdown_connections 2023-10-16T10:05:08.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.133+0000 7f54be2df640 1 -- 172.21.15.84:0/535135856 wait complete. 2023-10-16T10:05:08.153 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T10:05:08.154 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T10:05:08.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.374+0000 7fc3ad7a9640 1 Processor -- start 2023-10-16T10:05:08.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.375+0000 7fc3ad7a9640 1 -- start start 2023-10-16T10:05:08.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.376+0000 7fc3ad7a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc3a815ac10 0x7fc3a806c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:08.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.376+0000 7fc3ad7a9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a815b310 con 0x7fc3a80cac20 2023-10-16T10:05:08.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.376+0000 7fc3ad7a9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a806f640 con 0x7fc3a80b0360 2023-10-16T10:05:08.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.376+0000 7fc3ad7a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a815b5b0 con 0x7fc3a815ac10 2023-10-16T10:05:08.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.377+0000 7fc3a6ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc3a815ac10 0x7fc3a806c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:08.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.377+0000 7fc3a6ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc3a815ac10 0x7fc3a806c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42620/0 (socket says 172.21.15.84:42620) 2023-10-16T10:05:08.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.377+0000 7fc3a6ffd640 1 -- 172.21.15.84:0/3337298667 learned_addr learned my addr 172.21.15.84:0/3337298667 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:08.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.377+0000 7fc3a6ffd640 1 -- 172.21.15.84:0/3337298667 >> v1:172.21.15.157:6789/0 conn(0x7fc3a80b0360 legacy=0x7fc3a80ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:08.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.377+0000 7fc3a6ffd640 1 -- 172.21.15.84:0/3337298667 >> v1:172.21.15.84:6789/0 conn(0x7fc3a80cac20 legacy=0x7fc3a815e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:08.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.377+0000 7fc3a6ffd640 1 -- 172.21.15.84:0/3337298667 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a815b890 con 0x7fc3a815ac10 2023-10-16T10:05:08.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.378+0000 7fc3a6ffd640 1 --2- 172.21.15.84:0/3337298667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a815ac10 0x7fc3a806c730 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fc398009870 tx=0x7fc39802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea2521c6af3c698 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:08.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.378+0000 7fc3a67fc640 1 -- 172.21.15.84:0/3337298667 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398031860 con 0x7fc3a815ac10 2023-10-16T10:05:08.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.378+0000 7fc3a67fc640 1 -- 172.21.15.84:0/3337298667 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3980319c0 con 0x7fc3a815ac10 2023-10-16T10:05:08.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.378+0000 7fc3a67fc640 1 -- 172.21.15.84:0/3337298667 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398031d50 con 0x7fc3a815ac10 2023-10-16T10:05:08.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.379+0000 7fc3ad7a9640 1 -- 172.21.15.84:0/3337298667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a815ac10 msgr2=0x7fc3a806c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:08.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.379+0000 7fc3ad7a9640 1 --2- 172.21.15.84:0/3337298667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a815ac10 0x7fc3a806c730 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fc398009870 tx=0x7fc39802b900 comp rx=0 tx=0).stop 2023-10-16T10:05:08.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.379+0000 7fc3ad7a9640 1 -- 172.21.15.84:0/3337298667 shutdown_connections 2023-10-16T10:05:08.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.379+0000 7fc3ad7a9640 1 --2- 172.21.15.84:0/3337298667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a815ac10 0x7fc3a806c730 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:08.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.379+0000 7fc3ad7a9640 1 -- 172.21.15.84:0/3337298667 >> 172.21.15.84:0/3337298667 conn(0x7fc3a80cb360 msgr2=0x7fc3a80cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:08.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.379+0000 7fc3ad7a9640 1 -- 172.21.15.84:0/3337298667 shutdown_connections 2023-10-16T10:05:08.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.379+0000 7fc3ad7a9640 1 -- 172.21.15.84:0/3337298667 wait complete. 2023-10-16T10:05:08.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.381+0000 7fc3ad7a9640 1 Processor -- start 2023-10-16T10:05:08.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.381+0000 7fc3ad7a9640 1 -- start start 2023-10-16T10:05:08.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.382+0000 7fc3ad7a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3a80b0360 0x7fc3a814edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:08.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.382+0000 7fc3ad7a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3a80cac20 0x7fc3a80c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:08.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.382+0000 7fc3ad7a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a815ac10 0x7fc3a80c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:08.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.382+0000 7fc3ad7a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a814f310 con 0x7fc3a815ac10 2023-10-16T10:05:08.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.382+0000 7fc3ad7a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a81f63b0 con 0x7fc3a80cac20 2023-10-16T10:05:08.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.382+0000 7fc3ad7a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a81f6650 con 0x7fc3a80b0360 2023-10-16T10:05:08.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.382+0000 7fc3a77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a815ac10 0x7fc3a80c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:08.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.382+0000 7fc3a77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a815ac10 0x7fc3a80c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42624/0 (socket says 172.21.15.84:42624) 2023-10-16T10:05:08.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.382+0000 7fc3a77fe640 1 -- 172.21.15.84:0/3959311357 learned_addr learned my addr 172.21.15.84:0/3959311357 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:08.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.383+0000 7fc3a6ffd640 1 --2- 172.21.15.84:0/3959311357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3a80b0360 0x7fc3a814edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:08.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.383+0000 7fc39e7fc640 1 --2- 172.21.15.84:0/3959311357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3a80cac20 0x7fc3a80c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:08.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.383+0000 7fc3a77fe640 1 -- 172.21.15.84:0/3959311357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3a80b0360 msgr2=0x7fc3a814edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:08.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.383+0000 7fc3a77fe640 1 --2- 172.21.15.84:0/3959311357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3a80b0360 0x7fc3a814edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:08.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.383+0000 7fc3a77fe640 1 -- 172.21.15.84:0/3959311357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3a80cac20 msgr2=0x7fc3a80c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:08.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.383+0000 7fc3a77fe640 1 --2- 172.21.15.84:0/3959311357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3a80cac20 0x7fc3a80c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:08.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.383+0000 7fc3a77fe640 1 -- 172.21.15.84:0/3959311357 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3980094d0 con 0x7fc3a815ac10 2023-10-16T10:05:08.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.383+0000 7fc3a6ffd640 1 --2- 172.21.15.84:0/3959311357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3a80b0360 0x7fc3a814edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:08.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.383+0000 7fc39e7fc640 1 --2- 172.21.15.84:0/3959311357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3a80cac20 0x7fc3a80c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:08.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.383+0000 7fc3a77fe640 1 --2- 172.21.15.84:0/3959311357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a815ac10 0x7fc3a80c7b30 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000c7a0 tx=0x7fc3a000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:08.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.384+0000 7fc39ffff640 1 -- 172.21.15.84:0/3959311357 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0003b70 con 0x7fc3a815ac10 2023-10-16T10:05:08.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.384+0000 7fc39ffff640 1 -- 172.21.15.84:0/3959311357 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a0003cf0 con 0x7fc3a815ac10 2023-10-16T10:05:08.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.384+0000 7fc3ad7a9640 1 -- 172.21.15.84:0/3959311357 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a81f6930 con 0x7fc3a815ac10 2023-10-16T10:05:08.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.384+0000 7fc39ffff640 1 -- 172.21.15.84:0/3959311357 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0003ea0 con 0x7fc3a815ac10 2023-10-16T10:05:08.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.384+0000 7fc3ad7a9640 1 -- 172.21.15.84:0/3959311357 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a81f6ef0 con 0x7fc3a815ac10 2023-10-16T10:05:08.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.385+0000 7fc3ad7a9640 1 -- 172.21.15.84:0/3959311357 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a804e160 con 0x7fc3a815ac10 2023-10-16T10:05:08.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.386+0000 7fc39ffff640 1 -- 172.21.15.84:0/3959311357 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc3a000aa90 con 0x7fc3a815ac10 2023-10-16T10:05:08.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.387+0000 7fc39ffff640 1 --2- 172.21.15.84:0/3959311357 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc38806afe0 0x7fc38806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:08.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.387+0000 7fc39ffff640 1 -- 172.21.15.84:0/3959311357 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc3a0089110 con 0x7fc3a815ac10 2023-10-16T10:05:08.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.387+0000 7fc3a6ffd640 1 --2- 172.21.15.84:0/3959311357 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc38806afe0 0x7fc38806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:08.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.388+0000 7fc3a6ffd640 1 --2- 172.21.15.84:0/3959311357 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc38806afe0 0x7fc38806d4a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc398036000 tx=0x7fc39803c040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:08.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.391+0000 7fc39ffff640 1 -- 172.21.15.84:0/3959311357 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc3a0054570 con 0x7fc3a815ac10 2023-10-16T10:05:08.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:08.582+0000 7fc3ad7a9640 1 -- 172.21.15.84:0/3959311357 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc3a806c2e0 con 0x7fc3a815ac10 2023-10-16T10:05:09.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.334+0000 7fc39ffff640 1 -- 172.21.15.84:0/3959311357 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v92) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc3a0057b90 con 0x7fc3a815ac10 2023-10-16T10:05:09.336 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-16T10:05:09.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.338+0000 7fc3ad7a9640 1 -- 172.21.15.84:0/3959311357 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc38806afe0 msgr2=0x7fc38806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:09.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.338+0000 7fc3ad7a9640 1 --2- 172.21.15.84:0/3959311357 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc38806afe0 0x7fc38806d4a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc398036000 tx=0x7fc39803c040 comp rx=0 tx=0).stop 2023-10-16T10:05:09.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.338+0000 7fc3ad7a9640 1 -- 172.21.15.84:0/3959311357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a815ac10 msgr2=0x7fc3a80c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:09.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.338+0000 7fc3ad7a9640 1 --2- 172.21.15.84:0/3959311357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a815ac10 0x7fc3a80c7b30 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000c7a0 tx=0x7fc3a000cc60 comp rx=0 tx=0).stop 2023-10-16T10:05:09.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.338+0000 7fc3ad7a9640 1 -- 172.21.15.84:0/3959311357 shutdown_connections 2023-10-16T10:05:09.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.338+0000 7fc3ad7a9640 1 --2- 172.21.15.84:0/3959311357 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc38806afe0 0x7fc38806d4a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:09.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.338+0000 7fc3ad7a9640 1 --2- 172.21.15.84:0/3959311357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a815ac10 0x7fc3a80c7b30 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:09.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.338+0000 7fc3ad7a9640 1 --2- 172.21.15.84:0/3959311357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3a80cac20 0x7fc3a80c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:09.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.338+0000 7fc3ad7a9640 1 --2- 172.21.15.84:0/3959311357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3a80b0360 0x7fc3a814edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:09.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.338+0000 7fc3ad7a9640 1 -- 172.21.15.84:0/3959311357 >> 172.21.15.84:0/3959311357 conn(0x7fc3a80cb360 msgr2=0x7fc3a80ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:09.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.339+0000 7fc3ad7a9640 1 -- 172.21.15.84:0/3959311357 shutdown_connections 2023-10-16T10:05:09.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.339+0000 7fc3ad7a9640 1 -- 172.21.15.84:0/3959311357 wait complete. 2023-10-16T10:05:09.367 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T10:05:09.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.582+0000 7fc39f690640 1 Processor -- start 2023-10-16T10:05:09.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.582+0000 7fc39f690640 1 -- start start 2023-10-16T10:05:09.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.584+0000 7fc39f690640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc3981614c0 0x7fc3980c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:09.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.584+0000 7fc39f690640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3980c1b90 0x7fc39806b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:09.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.584+0000 7fc39f690640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc39806b9c0 0x7fc398165700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:09.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.584+0000 7fc39f690640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3980cf810 con 0x7fc39806b9c0 2023-10-16T10:05:09.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.584+0000 7fc39f690640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc39806d7b0 con 0x7fc3981614c0 2023-10-16T10:05:09.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.584+0000 7fc39f690640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3980cf550 con 0x7fc3980c1b90 2023-10-16T10:05:09.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.584+0000 7fc39dc2d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc39806b9c0 0x7fc398165700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:09.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.585+0000 7fc39dc2d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc39806b9c0 0x7fc398165700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42638/0 (socket says 172.21.15.84:42638) 2023-10-16T10:05:09.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.585+0000 7fc39dc2d640 1 -- 172.21.15.84:0/2015269407 learned_addr learned my addr 172.21.15.84:0/2015269407 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:09.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.585+0000 7fc39cc2b640 1 --2- 172.21.15.84:0/2015269407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3980c1b90 0x7fc39806b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:09.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.585+0000 7fc39d42c640 1 --2- 172.21.15.84:0/2015269407 >> v2:172.21.15.157:3300/0 conn(0x7fc3981614c0 0x7fc3980c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:09.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.585+0000 7fc39dc2d640 1 -- 172.21.15.84:0/2015269407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3980c1b90 msgr2=0x7fc39806b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:09.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.585+0000 7fc39dc2d640 1 --2- 172.21.15.84:0/2015269407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3980c1b90 0x7fc39806b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:09.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.585+0000 7fc39dc2d640 1 -- 172.21.15.84:0/2015269407 >> v2:172.21.15.157:3300/0 conn(0x7fc3981614c0 msgr2=0x7fc3980c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:09.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.585+0000 7fc39dc2d640 1 --2- 172.21.15.84:0/2015269407 >> v2:172.21.15.157:3300/0 conn(0x7fc3981614c0 0x7fc3980c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:09.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.585+0000 7fc39dc2d640 1 -- 172.21.15.84:0/2015269407 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3980c1770 con 0x7fc39806b9c0 2023-10-16T10:05:09.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.585+0000 7fc39d42c640 1 --2- 172.21.15.84:0/2015269407 >> v2:172.21.15.157:3300/0 conn(0x7fc3981614c0 0x7fc3980c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:09.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.586+0000 7fc39dc2d640 1 --2- 172.21.15.84:0/2015269407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc39806b9c0 0x7fc398165700 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fc39806dfc0 tx=0x7fc39002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14309a7f351c3db8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:09.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.587+0000 7fc38ffff640 1 -- 172.21.15.84:0/2015269407 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39002ca70 con 0x7fc39806b9c0 2023-10-16T10:05:09.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.587+0000 7fc38ffff640 1 -- 172.21.15.84:0/2015269407 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39002cbd0 con 0x7fc39806b9c0 2023-10-16T10:05:09.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.587+0000 7fc38ffff640 1 -- 172.21.15.84:0/2015269407 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3900343f0 con 0x7fc39806b9c0 2023-10-16T10:05:09.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.588+0000 7fc39f690640 1 -- 172.21.15.84:0/2015269407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc39806b9c0 msgr2=0x7fc398165700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:09.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.588+0000 7fc39f690640 1 --2- 172.21.15.84:0/2015269407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc39806b9c0 0x7fc398165700 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fc39806dfc0 tx=0x7fc39002bed0 comp rx=0 tx=0).stop 2023-10-16T10:05:09.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.588+0000 7fc39f690640 1 -- 172.21.15.84:0/2015269407 shutdown_connections 2023-10-16T10:05:09.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.588+0000 7fc39f690640 1 --2- 172.21.15.84:0/2015269407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc39806b9c0 0x7fc398165700 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:09.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.588+0000 7fc39f690640 1 --2- 172.21.15.84:0/2015269407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3980c1b90 0x7fc39806b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:09.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.588+0000 7fc39f690640 1 --2- 172.21.15.84:0/2015269407 >> v2:172.21.15.157:3300/0 conn(0x7fc3981614c0 0x7fc3980c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:09.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.588+0000 7fc39f690640 1 -- 172.21.15.84:0/2015269407 >> 172.21.15.84:0/2015269407 conn(0x7fc3980ce8b0 msgr2=0x7fc3980cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:09.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.588+0000 7fc39f690640 1 -- 172.21.15.84:0/2015269407 shutdown_connections 2023-10-16T10:05:09.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.588+0000 7fc39f690640 1 -- 172.21.15.84:0/2015269407 wait complete. 2023-10-16T10:05:09.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.590+0000 7fc39f690640 1 Processor -- start 2023-10-16T10:05:09.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.590+0000 7fc39f690640 1 -- start start 2023-10-16T10:05:09.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.591+0000 7fc39f690640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc39806b9c0 0x7fc39815d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:09.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.591+0000 7fc39f690640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3980c1b90 0x7fc398163ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:09.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.591+0000 7fc39f690640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3981614c0 0x7fc398164410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:09.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.591+0000 7fc39f690640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3981625e0 con 0x7fc3981614c0 2023-10-16T10:05:09.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.591+0000 7fc39f690640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc398162aa0 con 0x7fc3980c1b90 2023-10-16T10:05:09.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.591+0000 7fc39f690640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc398164950 con 0x7fc39806b9c0 2023-10-16T10:05:09.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.591+0000 7fc39dc2d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3981614c0 0x7fc398164410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:09.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.591+0000 7fc39dc2d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3981614c0 0x7fc398164410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42642/0 (socket says 172.21.15.84:42642) 2023-10-16T10:05:09.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.591+0000 7fc39dc2d640 1 -- 172.21.15.84:0/685503809 learned_addr learned my addr 172.21.15.84:0/685503809 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:09.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.591+0000 7fc39dc2d640 1 -- 172.21.15.84:0/685503809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc39806b9c0 msgr2=0x7fc39815d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:09.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.591+0000 7fc39dc2d640 1 --2- 172.21.15.84:0/685503809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc39806b9c0 0x7fc39815d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:09.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.591+0000 7fc39dc2d640 1 -- 172.21.15.84:0/685503809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3980c1b90 msgr2=0x7fc398163ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:09.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.591+0000 7fc39dc2d640 1 --2- 172.21.15.84:0/685503809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3980c1b90 0x7fc398163ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:09.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.591+0000 7fc39dc2d640 1 -- 172.21.15.84:0/685503809 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc390009b30 con 0x7fc3981614c0 2023-10-16T10:05:09.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.592+0000 7fc39dc2d640 1 --2- 172.21.15.84:0/685503809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3981614c0 0x7fc398164410 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fc390036040 tx=0x7fc390004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:09.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.592+0000 7fc38dffb640 1 -- 172.21.15.84:0/685503809 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39003e450 con 0x7fc3981614c0 2023-10-16T10:05:09.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.592+0000 7fc38dffb640 1 -- 172.21.15.84:0/685503809 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39003e5b0 con 0x7fc3981614c0 2023-10-16T10:05:09.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.592+0000 7fc38dffb640 1 -- 172.21.15.84:0/685503809 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39003e960 con 0x7fc3981614c0 2023-10-16T10:05:09.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.592+0000 7fc39f690640 1 -- 172.21.15.84:0/685503809 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3980c6590 con 0x7fc3981614c0 2023-10-16T10:05:09.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.592+0000 7fc39f690640 1 -- 172.21.15.84:0/685503809 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3980c7c70 con 0x7fc3981614c0 2023-10-16T10:05:09.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.593+0000 7fc39f690640 1 -- 172.21.15.84:0/685503809 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc39804e160 con 0x7fc3981614c0 2023-10-16T10:05:09.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.595+0000 7fc38dffb640 1 -- 172.21.15.84:0/685503809 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc39003eac0 con 0x7fc3981614c0 2023-10-16T10:05:09.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.596+0000 7fc38dffb640 1 --2- 172.21.15.84:0/685503809 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc37c06afc0 0x7fc37c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:09.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.596+0000 7fc38dffb640 1 -- 172.21.15.84:0/685503809 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fc3900ac760 con 0x7fc3981614c0 2023-10-16T10:05:09.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.596+0000 7fc39d42c640 1 --2- 172.21.15.84:0/685503809 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc37c06afc0 0x7fc37c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:09.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.597+0000 7fc39d42c640 1 --2- 172.21.15.84:0/685503809 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc37c06afc0 0x7fc37c06d480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc394006fd0 tx=0x7fc394008040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:09.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.600+0000 7fc38dffb640 1 -- 172.21.15.84:0/685503809 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc390077ae0 con 0x7fc3981614c0 2023-10-16T10:05:09.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:09.790+0000 7fc39f690640 1 -- 172.21.15.84:0/685503809 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc3980cc740 con 0x7fc3981614c0 2023-10-16T10:05:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.341+0000 7fc38dffb640 1 -- 172.21.15.84:0/685503809 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v93) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc390033e80 con 0x7fc3981614c0 2023-10-16T10:05:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-16T10:05:10.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.346+0000 7fc39f690640 1 -- 172.21.15.84:0/685503809 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc37c06afc0 msgr2=0x7fc37c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:10.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.346+0000 7fc39f690640 1 --2- 172.21.15.84:0/685503809 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc37c06afc0 0x7fc37c06d480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc394006fd0 tx=0x7fc394008040 comp rx=0 tx=0).stop 2023-10-16T10:05:10.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.346+0000 7fc39f690640 1 -- 172.21.15.84:0/685503809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3981614c0 msgr2=0x7fc398164410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:10.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.346+0000 7fc39f690640 1 --2- 172.21.15.84:0/685503809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3981614c0 0x7fc398164410 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fc390036040 tx=0x7fc390004a40 comp rx=0 tx=0).stop 2023-10-16T10:05:10.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.347+0000 7fc39f690640 1 -- 172.21.15.84:0/685503809 shutdown_connections 2023-10-16T10:05:10.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.347+0000 7fc39f690640 1 --2- 172.21.15.84:0/685503809 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc37c06afc0 0x7fc37c06d480 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:10.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.347+0000 7fc39f690640 1 --2- 172.21.15.84:0/685503809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3981614c0 0x7fc398164410 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:10.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.347+0000 7fc39f690640 1 --2- 172.21.15.84:0/685503809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3980c1b90 0x7fc398163ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:10.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.347+0000 7fc39f690640 1 --2- 172.21.15.84:0/685503809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc39806b9c0 0x7fc39815d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:10.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.347+0000 7fc39f690640 1 -- 172.21.15.84:0/685503809 >> 172.21.15.84:0/685503809 conn(0x7fc3980ce8b0 msgr2=0x7fc3980ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:10.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.351+0000 7fc39f690640 1 -- 172.21.15.84:0/685503809 shutdown_connections 2023-10-16T10:05:10.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.353+0000 7fc39f690640 1 -- 172.21.15.84:0/685503809 wait complete. 2023-10-16T10:05:10.386 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T10:05:10.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.576+0000 7fb6dd2d4640 1 Processor -- start 2023-10-16T10:05:10.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.577+0000 7fb6dd2d4640 1 -- start start 2023-10-16T10:05:10.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.578+0000 7fb6dd2d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6d815ad70 0x7fb6d8069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:10.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.578+0000 7fb6dd2d4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb6d80c9ca0 0x7fb6d806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:10.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.578+0000 7fb6dd2d4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d80cb030 con 0x7fb6d80ca3b0 2023-10-16T10:05:10.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.578+0000 7fb6dd2d4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d80c91c0 con 0x7fb6d80c9ca0 2023-10-16T10:05:10.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.578+0000 7fb6dd2d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d80c94a0 con 0x7fb6d815ad70 2023-10-16T10:05:10.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.579+0000 7fb6d6ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6d815ad70 0x7fb6d8069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:10.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.579+0000 7fb6d67fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb6d80c9ca0 0x7fb6d806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:10.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.579+0000 7fb6d77fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb6d80ca3b0 0x7fb6d80ca840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:47592/0 (socket says 172.21.15.84:47592) 2023-10-16T10:05:10.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.579+0000 7fb6d77fe640 1 -- 172.21.15.84:0/417482099 learned_addr learned my addr 172.21.15.84:0/417482099 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:10.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.580+0000 7fb6d6ffd640 1 -- 172.21.15.84:0/417482099 >> v2:172.21.15.157:3300/0 conn(0x7fb6d80c9ca0 msgr2=0x7fb6d806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:10.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.580+0000 7fb6d6ffd640 1 --2- 172.21.15.84:0/417482099 >> v2:172.21.15.157:3300/0 conn(0x7fb6d80c9ca0 0x7fb6d806a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:10.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.580+0000 7fb6d6ffd640 1 -- 172.21.15.84:0/417482099 >> v1:172.21.15.157:6789/0 conn(0x7fb6d80ca3b0 legacy=0x7fb6d80ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:10.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.580+0000 7fb6d6ffd640 1 -- 172.21.15.84:0/417482099 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d80c9780 con 0x7fb6d815ad70 2023-10-16T10:05:10.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.580+0000 7fb6d67fc640 1 --2- 172.21.15.84:0/417482099 >> v2:172.21.15.157:3300/0 conn(0x7fb6d80c9ca0 0x7fb6d806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:05:10.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.581+0000 7fb6d6ffd640 1 --2- 172.21.15.84:0/417482099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6d815ad70 0x7fb6d8069da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009d90 tx=0x7fb6c802beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45746c7aa72b58c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:10.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.582+0000 7fb6d5ffb640 1 -- 172.21.15.84:0/417482099 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c802cc60 con 0x7fb6d815ad70 2023-10-16T10:05:10.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.582+0000 7fb6d5ffb640 1 -- 172.21.15.84:0/417482099 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c802cdc0 con 0x7fb6d815ad70 2023-10-16T10:05:10.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.582+0000 7fb6d5ffb640 1 -- 172.21.15.84:0/417482099 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8034660 con 0x7fb6d815ad70 2023-10-16T10:05:10.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.583+0000 7fb6dd2d4640 1 -- 172.21.15.84:0/417482099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6d815ad70 msgr2=0x7fb6d8069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:10.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.583+0000 7fb6dd2d4640 1 --2- 172.21.15.84:0/417482099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6d815ad70 0x7fb6d8069da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009d90 tx=0x7fb6c802beb0 comp rx=0 tx=0).stop 2023-10-16T10:05:10.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.583+0000 7fb6dd2d4640 1 -- 172.21.15.84:0/417482099 shutdown_connections 2023-10-16T10:05:10.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.583+0000 7fb6dd2d4640 1 --2- 172.21.15.84:0/417482099 >> v2:172.21.15.157:3300/0 conn(0x7fb6d80c9ca0 0x7fb6d806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:10.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.583+0000 7fb6dd2d4640 1 --2- 172.21.15.84:0/417482099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6d815ad70 0x7fb6d8069da0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:10.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.583+0000 7fb6dd2d4640 1 -- 172.21.15.84:0/417482099 >> 172.21.15.84:0/417482099 conn(0x7fb6d80ce9b0 msgr2=0x7fb6d80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:10.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.583+0000 7fb6dd2d4640 1 -- 172.21.15.84:0/417482099 shutdown_connections 2023-10-16T10:05:10.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.583+0000 7fb6dd2d4640 1 -- 172.21.15.84:0/417482099 wait complete. 2023-10-16T10:05:10.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.585+0000 7fb6dd2d4640 1 Processor -- start 2023-10-16T10:05:10.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.585+0000 7fb6dd2d4640 1 -- start start 2023-10-16T10:05:10.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.586+0000 7fb6dd2d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6d80c9ca0 0x7fb6d81f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:10.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.586+0000 7fb6dd2d4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6d80ca3b0 0x7fb6d8156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:10.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.586+0000 7fb6dd2d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6d815ad70 0x7fb6d81540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:10.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.586+0000 7fb6dd2d4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d8156090 con 0x7fb6d80ca3b0 2023-10-16T10:05:10.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.586+0000 7fb6dd2d4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d8156910 con 0x7fb6d815ad70 2023-10-16T10:05:10.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.586+0000 7fb6dd2d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d81545e0 con 0x7fb6d80c9ca0 2023-10-16T10:05:10.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.586+0000 7fb6d67fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6d80ca3b0 0x7fb6d8156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:10.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.586+0000 7fb6d67fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6d80ca3b0 0x7fb6d8156da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42648/0 (socket says 172.21.15.84:42648) 2023-10-16T10:05:10.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.586+0000 7fb6d67fc640 1 -- 172.21.15.84:0/889795065 learned_addr learned my addr 172.21.15.84:0/889795065 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:10.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.586+0000 7fb6d6ffd640 1 --2- 172.21.15.84:0/889795065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6d80c9ca0 0x7fb6d81f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:10.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.586+0000 7fb6d77fe640 1 --2- 172.21.15.84:0/889795065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6d815ad70 0x7fb6d81540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:10.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.586+0000 7fb6d67fc640 1 -- 172.21.15.84:0/889795065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6d80c9ca0 msgr2=0x7fb6d81f6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:10.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.586+0000 7fb6d67fc640 1 --2- 172.21.15.84:0/889795065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6d80c9ca0 0x7fb6d81f6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:10.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.586+0000 7fb6d67fc640 1 -- 172.21.15.84:0/889795065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6d815ad70 msgr2=0x7fb6d81540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:10.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.586+0000 7fb6d67fc640 1 --2- 172.21.15.84:0/889795065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6d815ad70 0x7fb6d81540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:10.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.586+0000 7fb6d67fc640 1 -- 172.21.15.84:0/889795065 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c0009400 con 0x7fb6d80ca3b0 2023-10-16T10:05:10.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.587+0000 7fb6d67fc640 1 --2- 172.21.15.84:0/889795065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6d80ca3b0 0x7fb6d8156da0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000ebe0 tx=0x7fb6c000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:10.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.587+0000 7fb6cf7fe640 1 -- 172.21.15.84:0/889795065 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0009760 con 0x7fb6d80ca3b0 2023-10-16T10:05:10.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.587+0000 7fb6cf7fe640 1 -- 172.21.15.84:0/889795065 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c000cc30 con 0x7fb6d80ca3b0 2023-10-16T10:05:10.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.587+0000 7fb6cf7fe640 1 -- 172.21.15.84:0/889795065 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c000eee0 con 0x7fb6d80ca3b0 2023-10-16T10:05:10.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.587+0000 7fb6dd2d4640 1 -- 172.21.15.84:0/889795065 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c8009a70 con 0x7fb6d80ca3b0 2023-10-16T10:05:10.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.587+0000 7fb6dd2d4640 1 -- 172.21.15.84:0/889795065 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d81562c0 con 0x7fb6d80ca3b0 2023-10-16T10:05:10.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.589+0000 7fb6dd2d4640 1 -- 172.21.15.84:0/889795065 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d804e160 con 0x7fb6d80ca3b0 2023-10-16T10:05:10.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.590+0000 7fb6cf7fe640 1 -- 172.21.15.84:0/889795065 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb6c0013470 con 0x7fb6d80ca3b0 2023-10-16T10:05:10.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.592+0000 7fb6cf7fe640 1 --2- 172.21.15.84:0/889795065 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fb6b806af90 0x7fb6b806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:10.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.592+0000 7fb6cf7fe640 1 -- 172.21.15.84:0/889795065 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fb6c008a1c0 con 0x7fb6d80ca3b0 2023-10-16T10:05:10.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.592+0000 7fb6d6ffd640 1 --2- 172.21.15.84:0/889795065 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fb6b806af90 0x7fb6b806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:10.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.593+0000 7fb6d6ffd640 1 --2- 172.21.15.84:0/889795065 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fb6b806af90 0x7fb6b806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8036000 tx=0x7fb6c8030000 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:10.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.596+0000 7fb6cf7fe640 1 -- 172.21.15.84:0/889795065 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb6c00552a0 con 0x7fb6d80ca3b0 2023-10-16T10:05:10.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:10.826+0000 7fb6dd2d4640 1 -- 172.21.15.84:0/889795065 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb6d80ae8d0 con 0x7fb6d80ca3b0 2023-10-16T10:05:11.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.365+0000 7fb6cf7fe640 1 -- 172.21.15.84:0/889795065 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v40) v1 ==== 174+0+0 (secure 0 0 0) 0x7fb6c00588c0 con 0x7fb6d80ca3b0 2023-10-16T10:05:11.368 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 10 and data pool 11 2023-10-16T10:05:11.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.371+0000 7fb6dd2d4640 1 -- 172.21.15.84:0/889795065 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fb6b806af90 msgr2=0x7fb6b806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:11.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.371+0000 7fb6dd2d4640 1 --2- 172.21.15.84:0/889795065 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fb6b806af90 0x7fb6b806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8036000 tx=0x7fb6c8030000 comp rx=0 tx=0).stop 2023-10-16T10:05:11.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.371+0000 7fb6dd2d4640 1 -- 172.21.15.84:0/889795065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6d80ca3b0 msgr2=0x7fb6d8156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:11.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.371+0000 7fb6dd2d4640 1 --2- 172.21.15.84:0/889795065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6d80ca3b0 0x7fb6d8156da0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000ebe0 tx=0x7fb6c000c520 comp rx=0 tx=0).stop 2023-10-16T10:05:11.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.371+0000 7fb6dd2d4640 1 -- 172.21.15.84:0/889795065 shutdown_connections 2023-10-16T10:05:11.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.371+0000 7fb6dd2d4640 1 --2- 172.21.15.84:0/889795065 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fb6b806af90 0x7fb6b806d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:11.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.371+0000 7fb6dd2d4640 1 --2- 172.21.15.84:0/889795065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6d815ad70 0x7fb6d81540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:11.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.371+0000 7fb6dd2d4640 1 --2- 172.21.15.84:0/889795065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6d80ca3b0 0x7fb6d8156da0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:11.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.371+0000 7fb6dd2d4640 1 --2- 172.21.15.84:0/889795065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6d80c9ca0 0x7fb6d81f6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:11.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.371+0000 7fb6dd2d4640 1 -- 172.21.15.84:0/889795065 >> 172.21.15.84:0/889795065 conn(0x7fb6d80ce9b0 msgr2=0x7fb6d80ca840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:11.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.372+0000 7fb6dd2d4640 1 -- 172.21.15.84:0/889795065 shutdown_connections 2023-10-16T10:05:11.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.373+0000 7fb6dd2d4640 1 -- 172.21.15.84:0/889795065 wait complete. 2023-10-16T10:05:11.414 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:05:11.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.589+0000 7f71faccf640 1 Processor -- start 2023-10-16T10:05:11.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.589+0000 7f71faccf640 1 -- start start 2023-10-16T10:05:11.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.590+0000 7f71faccf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f415aaa0 0x7f71f415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:11.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.590+0000 7f71faccf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f415ed60 con 0x7f71f415ae80 2023-10-16T10:05:11.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.590+0000 7f71faccf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f40707e0 con 0x7f71f40ce390 2023-10-16T10:05:11.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.590+0000 7f71faccf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f406b810 con 0x7f71f415aaa0 2023-10-16T10:05:11.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.590+0000 7f71f3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f71f415ae80 0x7f71f40cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47036/0 (socket says 172.21.15.84:47036) 2023-10-16T10:05:11.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.590+0000 7f71f3fff640 1 -- 172.21.15.84:0/327829388 learned_addr learned my addr 172.21.15.84:0/327829388 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:11.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.591+0000 7f71f37fe640 1 -- 172.21.15.84:0/327829388 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 444980346 0 0) 0x7f71f415ed60 con 0x7f71f415ae80 2023-10-16T10:05:11.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.591+0000 7f71f8a6b640 1 --2- 172.21.15.84:0/327829388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f415aaa0 0x7f71f415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:11.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.591+0000 7f71f37fe640 1 -- 172.21.15.84:0/327829388 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71e0003520 con 0x7f71f415ae80 2023-10-16T10:05:11.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.591+0000 7f71f37fe640 1 -- 172.21.15.84:0/327829388 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1316849214 0 0) 0x7f71e0003520 con 0x7f71f415ae80 2023-10-16T10:05:11.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.591+0000 7f71f37fe640 1 -- 172.21.15.84:0/327829388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f415aaa0 msgr2=0x7f71f415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:11.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.591+0000 7f71f37fe640 1 --2- 172.21.15.84:0/327829388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f415aaa0 0x7f71f415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:11.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.591+0000 7f71f37fe640 1 -- 172.21.15.84:0/327829388 >> v1:172.21.15.157:6789/0 conn(0x7f71f40ce390 legacy=0x7f71f4070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:11.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.591+0000 7f71f37fe640 1 -- 172.21.15.84:0/327829388 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f415ef90 con 0x7f71f415ae80 2023-10-16T10:05:11.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.591+0000 7f71f37fe640 1 -- 172.21.15.84:0/327829388 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f71dc001670 con 0x7f71f415ae80 2023-10-16T10:05:11.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.591+0000 7f71f37fe640 1 -- 172.21.15.84:0/327829388 >> v1:172.21.15.84:6789/0 conn(0x7f71f415ae80 legacy=0x7f71f40cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:11.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.591+0000 7f71f37fe640 1 --2- 172.21.15.84:0/327829388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71e00039a0 0x7f71e0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:11.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.591+0000 7f71f37fe640 1 --2- 172.21.15.84:0/327829388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f415aaa0 0x7f71e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:11.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.591+0000 7f71f37fe640 1 --2- 172.21.15.84:0/327829388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71e0004d00 0x7f71e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:11.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.591+0000 7f71f37fe640 1 -- 172.21.15.84:0/327829388 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f406b810 con 0x7f71e00039a0 2023-10-16T10:05:11.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.591+0000 7f71f37fe640 1 -- 172.21.15.84:0/327829388 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e000bb10 con 0x7f71e0004d00 2023-10-16T10:05:11.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.591+0000 7f71f37fe640 1 -- 172.21.15.84:0/327829388 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e000bca0 con 0x7f71f415aaa0 2023-10-16T10:05:11.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.591+0000 7f71f8a6b640 1 --2- 172.21.15.84:0/327829388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71e00039a0 0x7f71e0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:11.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.592+0000 7f71f8a6b640 1 -- 172.21.15.84:0/327829388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f415aaa0 msgr2=0x7f71e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:11.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.592+0000 7f71f3fff640 1 --2- 172.21.15.84:0/327829388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f415aaa0 0x7f71e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:11.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.592+0000 7f71f8a6b640 1 --2- 172.21.15.84:0/327829388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f415aaa0 0x7f71e00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:11.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.592+0000 7f71f8a6b640 1 -- 172.21.15.84:0/327829388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71e0004d00 msgr2=0x7f71e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:11.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.592+0000 7f71f8a6b640 1 --2- 172.21.15.84:0/327829388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71e0004d00 0x7f71e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:11.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.592+0000 7f71f8a6b640 1 -- 172.21.15.84:0/327829388 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e0003520 con 0x7f71e00039a0 2023-10-16T10:05:11.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.593+0000 7f71f8a6b640 1 --2- 172.21.15.84:0/327829388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71e00039a0 0x7f71e0003dc0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f71ec007390 tx=0x7f71e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8985cf2b39b33c09 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:11.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.593+0000 7f71f37fe640 1 -- 172.21.15.84:0/327829388 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec02ccb0 con 0x7f71e00039a0 2023-10-16T10:05:11.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.593+0000 7f71f37fe640 1 -- 172.21.15.84:0/327829388 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ec02ce10 con 0x7f71e00039a0 2023-10-16T10:05:11.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.593+0000 7f71f37fe640 1 -- 172.21.15.84:0/327829388 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec034660 con 0x7f71e00039a0 2023-10-16T10:05:11.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.594+0000 7f71faccf640 1 -- 172.21.15.84:0/327829388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71e00039a0 msgr2=0x7f71e0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:11.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.594+0000 7f71faccf640 1 --2- 172.21.15.84:0/327829388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71e00039a0 0x7f71e0003dc0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f71ec007390 tx=0x7f71e00057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:11.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.594+0000 7f71f926c640 1 -- 172.21.15.84:0/327829388 reap_dead start 2023-10-16T10:05:11.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.594+0000 7f71faccf640 1 -- 172.21.15.84:0/327829388 shutdown_connections 2023-10-16T10:05:11.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.594+0000 7f71faccf640 1 -- 172.21.15.84:0/327829388 >> 172.21.15.84:0/327829388 conn(0x7f71f40cb410 msgr2=0x7f71f40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:11.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.594+0000 7f71faccf640 1 -- 172.21.15.84:0/327829388 shutdown_connections 2023-10-16T10:05:11.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.594+0000 7f71faccf640 1 -- 172.21.15.84:0/327829388 wait complete. 2023-10-16T10:05:11.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.595+0000 7f71faccf640 1 Processor -- start 2023-10-16T10:05:11.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.596+0000 7f71faccf640 1 -- start start 2023-10-16T10:05:11.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.596+0000 7f71faccf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71e00039a0 0x7f71f415c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:11.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.596+0000 7f71faccf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f4155a40 0x7f71f40c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:11.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.597+0000 7f71faccf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f40af680 0x7f71f4154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:11.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.597+0000 7f71faccf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f40c39e0 con 0x7f71f4155a40 2023-10-16T10:05:11.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.597+0000 7f71faccf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f40c4840 con 0x7f71e00039a0 2023-10-16T10:05:11.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.597+0000 7f71faccf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f40c3c30 con 0x7f71f40af680 2023-10-16T10:05:11.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.597+0000 7f71f3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f4155a40 0x7f71f40c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:11.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.597+0000 7f71f3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f4155a40 0x7f71f40c4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42666/0 (socket says 172.21.15.84:42666) 2023-10-16T10:05:11.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.597+0000 7f71f3fff640 1 -- 172.21.15.84:0/3316343244 learned_addr learned my addr 172.21.15.84:0/3316343244 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:11.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.597+0000 7f71f3fff640 1 -- 172.21.15.84:0/3316343244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f40af680 msgr2=0x7f71f4154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:11.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.597+0000 7f71f926c640 1 --2- 172.21.15.84:0/3316343244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f40af680 0x7f71f4154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:11.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.597+0000 7f71f3fff640 1 --2- 172.21.15.84:0/3316343244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f40af680 0x7f71f4154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:11.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.597+0000 7f71f3fff640 1 -- 172.21.15.84:0/3316343244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71e00039a0 msgr2=0x7f71f415c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:11.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.597+0000 7f71f3fff640 1 --2- 172.21.15.84:0/3316343244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71e00039a0 0x7f71f415c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:11.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.597+0000 7f71f3fff640 1 -- 172.21.15.84:0/3316343244 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f40c3f10 con 0x7f71f4155a40 2023-10-16T10:05:11.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.597+0000 7f71f926c640 1 --2- 172.21.15.84:0/3316343244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f40af680 0x7f71f4154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:11.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.597+0000 7f71f3fff640 1 --2- 172.21.15.84:0/3316343244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f4155a40 0x7f71f40c4cd0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00d7d0 tx=0x7f71dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:11.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.598+0000 7f71f17fa640 1 -- 172.21.15.84:0/3316343244 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc0074e0 con 0x7f71f4155a40 2023-10-16T10:05:11.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.598+0000 7f71f17fa640 1 -- 172.21.15.84:0/3316343244 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71dc007640 con 0x7f71f4155a40 2023-10-16T10:05:11.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.598+0000 7f71f17fa640 1 -- 172.21.15.84:0/3316343244 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc004a20 con 0x7f71f4155a40 2023-10-16T10:05:11.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.598+0000 7f71faccf640 1 -- 172.21.15.84:0/3316343244 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f40c41f0 con 0x7f71f4155a40 2023-10-16T10:05:11.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.598+0000 7f71faccf640 1 -- 172.21.15.84:0/3316343244 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f40c5210 con 0x7f71f4155a40 2023-10-16T10:05:11.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.599+0000 7f71faccf640 1 -- 172.21.15.84:0/3316343244 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f404e160 con 0x7f71f4155a40 2023-10-16T10:05:11.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.600+0000 7f71f17fa640 1 -- 172.21.15.84:0/3316343244 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f71dc004b80 con 0x7f71f4155a40 2023-10-16T10:05:11.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.601+0000 7f71f17fa640 1 --2- 172.21.15.84:0/3316343244 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f71d406af90 0x7f71d406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:11.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.601+0000 7f71f17fa640 1 -- 172.21.15.84:0/3316343244 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71dc0891c0 con 0x7f71f4155a40 2023-10-16T10:05:11.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.602+0000 7f71f8a6b640 1 --2- 172.21.15.84:0/3316343244 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f71d406af90 0x7f71d406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:11.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.602+0000 7f71f8a6b640 1 --2- 172.21.15.84:0/3316343244 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f71d406af90 0x7f71d406d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f71ec036000 tx=0x7f71ec02e000 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:11.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.606+0000 7f71f17fa640 1 -- 172.21.15.84:0/3316343244 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f71dc054050 con 0x7f71f4155a40 2023-10-16T10:05:11.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.795+0000 7f71faccf640 1 -- 172.21.15.84:0/3316343244 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71f40c49d0 con 0x7f71f4155a40 2023-10-16T10:05:11.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.797+0000 7f71f17fa640 1 -- 172.21.15.84:0/3316343244 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16557 (secure 0 0 0) 0x7f71dc057670 con 0x7f71f4155a40 2023-10-16T10:05:11.799 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:11.799 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":94,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:05:10.828655+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:05:08.636077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-16T10:05:09.841985+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:05:11.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.800+0000 7f71faccf640 1 -- 172.21.15.84:0/3316343244 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f71d406af90 msgr2=0x7f71d406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:11.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.800+0000 7f71faccf640 1 --2- 172.21.15.84:0/3316343244 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f71d406af90 0x7f71d406d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f71ec036000 tx=0x7f71ec02e000 comp rx=0 tx=0).stop 2023-10-16T10:05:11.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.800+0000 7f71faccf640 1 -- 172.21.15.84:0/3316343244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f4155a40 msgr2=0x7f71f40c4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:11.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.800+0000 7f71faccf640 1 --2- 172.21.15.84:0/3316343244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f4155a40 0x7f71f40c4cd0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00d7d0 tx=0x7f71dc00dc90 comp rx=0 tx=0).stop 2023-10-16T10:05:11.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.800+0000 7f71faccf640 1 -- 172.21.15.84:0/3316343244 shutdown_connections 2023-10-16T10:05:11.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.800+0000 7f71faccf640 1 --2- 172.21.15.84:0/3316343244 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f71d406af90 0x7f71d406d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:11.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.800+0000 7f71faccf640 1 --2- 172.21.15.84:0/3316343244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71f40af680 0x7f71f4154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:11.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.800+0000 7f71faccf640 1 --2- 172.21.15.84:0/3316343244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f4155a40 0x7f71f40c4cd0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:11.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.800+0000 7f71faccf640 1 --2- 172.21.15.84:0/3316343244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71e00039a0 0x7f71f415c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:11.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.800+0000 7f71faccf640 1 -- 172.21.15.84:0/3316343244 >> 172.21.15.84:0/3316343244 conn(0x7f71f40cb410 msgr2=0x7f71f40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:11.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.800+0000 7f71faccf640 1 -- 172.21.15.84:0/3316343244 shutdown_connections 2023-10-16T10:05:11.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:11.800+0000 7f71faccf640 1 -- 172.21.15.84:0/3316343244 wait complete. 2023-10-16T10:05:11.818 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:05:12.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.010+0000 7fc736d97640 1 Processor -- start 2023-10-16T10:05:12.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.010+0000 7fc736d97640 1 -- start start 2023-10-16T10:05:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.011+0000 7fc736d97640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73015ae80 0x7fc73015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.011+0000 7fc736d97640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7300ce390 0x7fc730070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.011+0000 7fc736d97640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73015ed60 con 0x7fc73015aaa0 2023-10-16T10:05:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.012+0000 7fc736d97640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73006b810 con 0x7fc7300ce390 2023-10-16T10:05:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.012+0000 7fc736d97640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300707e0 con 0x7fc73015ae80 2023-10-16T10:05:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.012+0000 7fc734b33640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc73015aaa0 0x7fc7300cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47050/0 (socket says 172.21.15.84:47050) 2023-10-16T10:05:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.012+0000 7fc734b33640 1 -- 172.21.15.84:0/2529035465 learned_addr learned my addr 172.21.15.84:0/2529035465 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.012+0000 7fc727fff640 1 --2- 172.21.15.84:0/2529035465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73015ae80 0x7fc73015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.012+0000 7fc735334640 1 --2- 172.21.15.84:0/2529035465 >> v2:172.21.15.157:3300/0 conn(0x7fc7300ce390 0x7fc730070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.012+0000 7fc7277fe640 1 -- 172.21.15.84:0/2529035465 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160640650 0 0) 0x7fc73015ed60 con 0x7fc73015aaa0 2023-10-16T10:05:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.012+0000 7fc7277fe640 1 -- 172.21.15.84:0/2529035465 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc718003520 con 0x7fc73015aaa0 2023-10-16T10:05:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.013+0000 7fc7277fe640 1 -- 172.21.15.84:0/2529035465 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1964545154 0 0) 0x7fc718003520 con 0x7fc73015aaa0 2023-10-16T10:05:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.013+0000 7fc7277fe640 1 -- 172.21.15.84:0/2529035465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73015ae80 msgr2=0x7fc73015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.013+0000 7fc7277fe640 1 --2- 172.21.15.84:0/2529035465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73015ae80 0x7fc73015e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.013+0000 7fc7277fe640 1 -- 172.21.15.84:0/2529035465 >> v2:172.21.15.157:3300/0 conn(0x7fc7300ce390 msgr2=0x7fc730070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.013+0000 7fc7277fe640 1 --2- 172.21.15.84:0/2529035465 >> v2:172.21.15.157:3300/0 conn(0x7fc7300ce390 0x7fc730070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.013+0000 7fc7277fe640 1 -- 172.21.15.84:0/2529035465 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73015ef90 con 0x7fc73015aaa0 2023-10-16T10:05:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.013+0000 7fc7277fe640 1 -- 172.21.15.84:0/2529035465 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc72c001670 con 0x7fc73015aaa0 2023-10-16T10:05:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.013+0000 7fc7277fe640 1 -- 172.21.15.84:0/2529035465 >> v1:172.21.15.84:6789/0 conn(0x7fc73015aaa0 legacy=0x7fc7300cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:12.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.013+0000 7fc7277fe640 1 --2- 172.21.15.84:0/2529035465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc718003970 0x7fc718003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:12.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.013+0000 7fc7277fe640 1 --2- 172.21.15.84:0/2529035465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7180047c0 0x7fc718004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:12.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.013+0000 7fc7277fe640 1 --2- 172.21.15.84:0/2529035465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73015ae80 0x7fc718005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:12.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.013+0000 7fc7277fe640 1 -- 172.21.15.84:0/2529035465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73006b810 con 0x7fc7180047c0 2023-10-16T10:05:12.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.013+0000 7fc7277fe640 1 -- 172.21.15.84:0/2529035465 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300707e0 con 0x7fc718003970 2023-10-16T10:05:12.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.013+0000 7fc7277fe640 1 -- 172.21.15.84:0/2529035465 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71800b980 con 0x7fc73015ae80 2023-10-16T10:05:12.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.013+0000 7fc727fff640 1 --2- 172.21.15.84:0/2529035465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7180047c0 0x7fc718004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:12.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.014+0000 7fc727fff640 1 -- 172.21.15.84:0/2529035465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73015ae80 msgr2=0x7fc718005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:12.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.014+0000 7fc735334640 1 --2- 172.21.15.84:0/2529035465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73015ae80 0x7fc718005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:12.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.014+0000 7fc727fff640 1 --2- 172.21.15.84:0/2529035465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73015ae80 0x7fc718005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:12.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.014+0000 7fc727fff640 1 -- 172.21.15.84:0/2529035465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc718003970 msgr2=0x7fc718003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:12.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.014+0000 7fc727fff640 1 --2- 172.21.15.84:0/2529035465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc718003970 0x7fc718003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:12.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.014+0000 7fc727fff640 1 -- 172.21.15.84:0/2529035465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc718003520 con 0x7fc7180047c0 2023-10-16T10:05:12.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.015+0000 7fc727fff640 1 --2- 172.21.15.84:0/2529035465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7180047c0 0x7fc718004c40 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fc720009ac0 tx=0x7fc7180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9bec9a652937f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:12.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.015+0000 7fc7277fe640 1 -- 172.21.15.84:0/2529035465 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72002c6b0 con 0x7fc7180047c0 2023-10-16T10:05:12.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.015+0000 7fc7277fe640 1 -- 172.21.15.84:0/2529035465 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72002c810 con 0x7fc7180047c0 2023-10-16T10:05:12.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.015+0000 7fc7277fe640 1 -- 172.21.15.84:0/2529035465 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72002cba0 con 0x7fc7180047c0 2023-10-16T10:05:12.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.015+0000 7fc736d97640 1 -- 172.21.15.84:0/2529035465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7180047c0 msgr2=0x7fc718004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:12.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.015+0000 7fc736d97640 1 --2- 172.21.15.84:0/2529035465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7180047c0 0x7fc718004c40 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fc720009ac0 tx=0x7fc7180057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:12.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.016+0000 7fc735334640 1 -- 172.21.15.84:0/2529035465 reap_dead start 2023-10-16T10:05:12.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.016+0000 7fc736d97640 1 -- 172.21.15.84:0/2529035465 shutdown_connections 2023-10-16T10:05:12.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.016+0000 7fc736d97640 1 -- 172.21.15.84:0/2529035465 >> 172.21.15.84:0/2529035465 conn(0x7fc7300cb410 msgr2=0x7fc7300cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:12.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.016+0000 7fc736d97640 1 -- 172.21.15.84:0/2529035465 shutdown_connections 2023-10-16T10:05:12.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.016+0000 7fc736d97640 1 -- 172.21.15.84:0/2529035465 wait complete. 2023-10-16T10:05:12.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.017+0000 7fc736d97640 1 Processor -- start 2023-10-16T10:05:12.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.018+0000 7fc736d97640 1 -- start start 2023-10-16T10:05:12.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.018+0000 7fc736d97640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7180047c0 0x7fc7301f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:12.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.018+0000 7fc736d97640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7300c5120 0x7fc73014ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:12.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.018+0000 7fc736d97640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7300c24a0 0x7fc7300ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:12.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.018+0000 7fc736d97640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300c4d00 con 0x7fc7180047c0 2023-10-16T10:05:12.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.018+0000 7fc736d97640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300c40c0 con 0x7fc7300c24a0 2023-10-16T10:05:12.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.018+0000 7fc736d97640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73014f3c0 con 0x7fc7300c5120 2023-10-16T10:05:12.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.019+0000 7fc734b33640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7180047c0 0x7fc7301f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:12.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.019+0000 7fc734b33640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7180047c0 0x7fc7301f6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42678/0 (socket says 172.21.15.84:42678) 2023-10-16T10:05:12.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.019+0000 7fc734b33640 1 -- 172.21.15.84:0/230155942 learned_addr learned my addr 172.21.15.84:0/230155942 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:12.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.019+0000 7fc727fff640 1 --2- 172.21.15.84:0/230155942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7300c5120 0x7fc73014ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:12.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.019+0000 7fc734b33640 1 -- 172.21.15.84:0/230155942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7300c5120 msgr2=0x7fc73014ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:12.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.019+0000 7fc735334640 1 --2- 172.21.15.84:0/230155942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7300c24a0 0x7fc7300ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:12.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.019+0000 7fc734b33640 1 --2- 172.21.15.84:0/230155942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7300c5120 0x7fc73014ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:12.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.019+0000 7fc734b33640 1 -- 172.21.15.84:0/230155942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7300c24a0 msgr2=0x7fc7300ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:12.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.019+0000 7fc734b33640 1 --2- 172.21.15.84:0/230155942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7300c24a0 0x7fc7300ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:12.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.019+0000 7fc734b33640 1 -- 172.21.15.84:0/230155942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7300c4360 con 0x7fc7180047c0 2023-10-16T10:05:12.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.019+0000 7fc727fff640 1 --2- 172.21.15.84:0/230155942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7300c5120 0x7fc73014ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:12.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.020+0000 7fc734b33640 1 --2- 172.21.15.84:0/230155942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7180047c0 0x7fc7301f6d70 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00c750 tx=0x7fc72c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:12.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.020+0000 7fc7257fa640 1 -- 172.21.15.84:0/230155942 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c0074e0 con 0x7fc7180047c0 2023-10-16T10:05:12.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.020+0000 7fc7257fa640 1 -- 172.21.15.84:0/230155942 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c007660 con 0x7fc7180047c0 2023-10-16T10:05:12.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.020+0000 7fc736d97640 1 -- 172.21.15.84:0/230155942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7300c4640 con 0x7fc7180047c0 2023-10-16T10:05:12.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.020+0000 7fc7257fa640 1 -- 172.21.15.84:0/230155942 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c004a20 con 0x7fc7180047c0 2023-10-16T10:05:12.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.020+0000 7fc736d97640 1 -- 172.21.15.84:0/230155942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7300c2920 con 0x7fc7180047c0 2023-10-16T10:05:12.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.021+0000 7fc736d97640 1 -- 172.21.15.84:0/230155942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7300c2b10 con 0x7fc7180047c0 2023-10-16T10:05:12.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.022+0000 7fc7257fa640 1 -- 172.21.15.84:0/230155942 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc72c004b80 con 0x7fc7180047c0 2023-10-16T10:05:12.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.023+0000 7fc7257fa640 1 --2- 172.21.15.84:0/230155942 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc71c06af90 0x7fc71c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:12.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.023+0000 7fc7257fa640 1 -- 172.21.15.84:0/230155942 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc72c088ed0 con 0x7fc7180047c0 2023-10-16T10:05:12.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.023+0000 7fc727fff640 1 --2- 172.21.15.84:0/230155942 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc71c06af90 0x7fc71c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:12.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.024+0000 7fc727fff640 1 --2- 172.21.15.84:0/230155942 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc71c06af90 0x7fc71c06d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc720036000 tx=0x7fc720030000 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:12.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.027+0000 7fc7257fa640 1 -- 172.21.15.84:0/230155942 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc72c057990 con 0x7fc7180047c0 2023-10-16T10:05:12.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.237+0000 7fc736d97640 1 -- 172.21.15.84:0/230155942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7300c4e40 con 0x7fc7180047c0 2023-10-16T10:05:12.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.240+0000 7fc7257fa640 1 -- 172.21.15.84:0/230155942 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16557 (secure 0 0 0) 0x7fc7300c4e40 con 0x7fc7180047c0 2023-10-16T10:05:12.242 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:12.242 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":94,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:05:10.828655+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:05:08.636077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-16T10:05:09.841985+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:05:12.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.242+0000 7fc736d97640 1 -- 172.21.15.84:0/230155942 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc71c06af90 msgr2=0x7fc71c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:12.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.242+0000 7fc736d97640 1 --2- 172.21.15.84:0/230155942 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc71c06af90 0x7fc71c06d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc720036000 tx=0x7fc720030000 comp rx=0 tx=0).stop 2023-10-16T10:05:12.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.242+0000 7fc736d97640 1 -- 172.21.15.84:0/230155942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7180047c0 msgr2=0x7fc7301f6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:12.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.242+0000 7fc736d97640 1 --2- 172.21.15.84:0/230155942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7180047c0 0x7fc7301f6d70 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00c750 tx=0x7fc72c00cc10 comp rx=0 tx=0).stop 2023-10-16T10:05:12.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.243+0000 7fc736d97640 1 -- 172.21.15.84:0/230155942 shutdown_connections 2023-10-16T10:05:12.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.243+0000 7fc736d97640 1 --2- 172.21.15.84:0/230155942 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fc71c06af90 0x7fc71c06d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:12.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.243+0000 7fc736d97640 1 --2- 172.21.15.84:0/230155942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7300c24a0 0x7fc7300ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:12.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.243+0000 7fc736d97640 1 --2- 172.21.15.84:0/230155942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7300c5120 0x7fc73014ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:12.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.243+0000 7fc736d97640 1 --2- 172.21.15.84:0/230155942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7180047c0 0x7fc7301f6d70 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:12.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.243+0000 7fc736d97640 1 -- 172.21.15.84:0/230155942 >> 172.21.15.84:0/230155942 conn(0x7fc7300cb410 msgr2=0x7fc7300b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:12.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.243+0000 7fc736d97640 1 -- 172.21.15.84:0/230155942 shutdown_connections 2023-10-16T10:05:12.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.243+0000 7fc736d97640 1 -- 172.21.15.84:0/230155942 wait complete. 2023-10-16T10:05:12.263 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T10:05:12.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.480+0000 7f6e3bdb0640 1 Processor -- start 2023-10-16T10:05:12.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.480+0000 7f6e3bdb0640 1 -- start start 2023-10-16T10:05:12.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.482+0000 7f6e3bdb0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e34160530 0x7f6e341609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:12.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.482+0000 7f6e3bdb0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6e34065ab0 0x7f6e34160f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:12.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.482+0000 7f6e3bdb0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6e34161470 0x7f6e340ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:12.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.482+0000 7f6e3bdb0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e340ac890 con 0x7f6e34161470 2023-10-16T10:05:12.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.482+0000 7f6e3bdb0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e340ae800 con 0x7f6e34065ab0 2023-10-16T10:05:12.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.482+0000 7f6e3bdb0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340ae1f0 con 0x7f6e34160530 2023-10-16T10:05:12.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.482+0000 7f6e3a34d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6e34161470 0x7f6e340ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:12.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.482+0000 7f6e3a34d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6e34161470 0x7f6e340ccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42690/0 (socket says 172.21.15.84:42690) 2023-10-16T10:05:12.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.482+0000 7f6e3a34d640 1 -- 172.21.15.84:0/452315220 learned_addr learned my addr 172.21.15.84:0/452315220 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:12.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.483+0000 7f6e3a34d640 1 -- 172.21.15.84:0/452315220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e34160530 msgr2=0x7f6e341609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:12.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.483+0000 7f6e39b4c640 1 --2- 172.21.15.84:0/452315220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e34160530 0x7f6e341609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:12.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.483+0000 7f6e3934b640 1 --2- 172.21.15.84:0/452315220 >> v2:172.21.15.157:3300/0 conn(0x7f6e34065ab0 0x7f6e34160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:12.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.483+0000 7f6e3a34d640 1 --2- 172.21.15.84:0/452315220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e34160530 0x7f6e341609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:12.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.483+0000 7f6e3a34d640 1 -- 172.21.15.84:0/452315220 >> v2:172.21.15.157:3300/0 conn(0x7f6e34065ab0 msgr2=0x7f6e34160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:12.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.483+0000 7f6e3a34d640 1 --2- 172.21.15.84:0/452315220 >> v2:172.21.15.157:3300/0 conn(0x7f6e34065ab0 0x7f6e34160f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:12.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.483+0000 7f6e3a34d640 1 -- 172.21.15.84:0/452315220 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e340acae0 con 0x7f6e34161470 2023-10-16T10:05:12.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.483+0000 7f6e3934b640 1 --2- 172.21.15.84:0/452315220 >> v2:172.21.15.157:3300/0 conn(0x7f6e34065ab0 0x7f6e34160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:12.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.483+0000 7f6e39b4c640 1 --2- 172.21.15.84:0/452315220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e34160530 0x7f6e341609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:12.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.484+0000 7f6e3a34d640 1 --2- 172.21.15.84:0/452315220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e34161470 0x7f6e340ccd80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f6e3406f840 tx=0x7f6e2c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60882f7997aa90e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:12.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.484+0000 7f6e38b4a640 1 -- 172.21.15.84:0/452315220 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c02c9f0 con 0x7f6e34161470 2023-10-16T10:05:12.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.484+0000 7f6e38b4a640 1 -- 172.21.15.84:0/452315220 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2c02cb50 con 0x7f6e34161470 2023-10-16T10:05:12.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.484+0000 7f6e38b4a640 1 -- 172.21.15.84:0/452315220 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0343f0 con 0x7f6e34161470 2023-10-16T10:05:12.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.485+0000 7f6e3bdb0640 1 -- 172.21.15.84:0/452315220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e34161470 msgr2=0x7f6e340ccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:12.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.485+0000 7f6e3bdb0640 1 --2- 172.21.15.84:0/452315220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e34161470 0x7f6e340ccd80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f6e3406f840 tx=0x7f6e2c02be50 comp rx=0 tx=0).stop 2023-10-16T10:05:12.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.485+0000 7f6e3bdb0640 1 -- 172.21.15.84:0/452315220 shutdown_connections 2023-10-16T10:05:12.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.485+0000 7f6e3bdb0640 1 --2- 172.21.15.84:0/452315220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e34161470 0x7f6e340ccd80 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:12.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.485+0000 7f6e3bdb0640 1 --2- 172.21.15.84:0/452315220 >> v2:172.21.15.157:3300/0 conn(0x7f6e34065ab0 0x7f6e34160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:12.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.485+0000 7f6e3bdb0640 1 --2- 172.21.15.84:0/452315220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e34160530 0x7f6e341609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:12.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.485+0000 7f6e3bdb0640 1 -- 172.21.15.84:0/452315220 >> 172.21.15.84:0/452315220 conn(0x7f6e3406bc10 msgr2=0x7f6e3406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:12.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.485+0000 7f6e3bdb0640 1 -- 172.21.15.84:0/452315220 shutdown_connections 2023-10-16T10:05:12.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.485+0000 7f6e3bdb0640 1 -- 172.21.15.84:0/452315220 wait complete. 2023-10-16T10:05:12.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.487+0000 7f6e3bdb0640 1 Processor -- start 2023-10-16T10:05:12.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.487+0000 7f6e3bdb0640 1 -- start start 2023-10-16T10:05:12.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.488+0000 7f6e3bdb0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e34065ab0 0x7f6e341578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:12.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.488+0000 7f6e3bdb0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e34160530 0x7f6e340b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:12.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.488+0000 7f6e3bdb0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e34161470 0x7f6e3415c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:12.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.488+0000 7f6e3bdb0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e34157e10 con 0x7f6e34065ab0 2023-10-16T10:05:12.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.488+0000 7f6e3bdb0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340b3c90 con 0x7f6e34161470 2023-10-16T10:05:12.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.488+0000 7f6e3bdb0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340b3f70 con 0x7f6e34160530 2023-10-16T10:05:12.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.488+0000 7f6e39b4c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e34065ab0 0x7f6e341578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:12.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.488+0000 7f6e39b4c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e34065ab0 0x7f6e341578d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42696/0 (socket says 172.21.15.84:42696) 2023-10-16T10:05:12.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.488+0000 7f6e39b4c640 1 -- 172.21.15.84:0/3691413046 learned_addr learned my addr 172.21.15.84:0/3691413046 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:12.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.489+0000 7f6e39b4c640 1 -- 172.21.15.84:0/3691413046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e34160530 msgr2=0x7f6e340b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:12.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.489+0000 7f6e39b4c640 1 --2- 172.21.15.84:0/3691413046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e34160530 0x7f6e340b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:12.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.489+0000 7f6e39b4c640 1 -- 172.21.15.84:0/3691413046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e34161470 msgr2=0x7f6e3415c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:12.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.489+0000 7f6e39b4c640 1 --2- 172.21.15.84:0/3691413046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e34161470 0x7f6e3415c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:12.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.489+0000 7f6e39b4c640 1 -- 172.21.15.84:0/3691413046 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c009b30 con 0x7f6e34065ab0 2023-10-16T10:05:12.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.489+0000 7f6e39b4c640 1 --2- 172.21.15.84:0/3691413046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e34065ab0 0x7f6e341578d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000c7f0 tx=0x7f6e3000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:12.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.489+0000 7f6e2a7fc640 1 -- 172.21.15.84:0/3691413046 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e30007a30 con 0x7f6e34065ab0 2023-10-16T10:05:12.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.489+0000 7f6e2a7fc640 1 -- 172.21.15.84:0/3691413046 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e30007b90 con 0x7f6e34065ab0 2023-10-16T10:05:12.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.489+0000 7f6e2a7fc640 1 -- 172.21.15.84:0/3691413046 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3000ce40 con 0x7f6e34065ab0 2023-10-16T10:05:12.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.489+0000 7f6e3bdb0640 1 -- 172.21.15.84:0/3691413046 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e340b4250 con 0x7f6e34065ab0 2023-10-16T10:05:12.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.490+0000 7f6e3bdb0640 1 -- 172.21.15.84:0/3691413046 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e340b4810 con 0x7f6e34065ab0 2023-10-16T10:05:12.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.491+0000 7f6e3bdb0640 1 -- 172.21.15.84:0/3691413046 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e340b4a30 con 0x7f6e34065ab0 2023-10-16T10:05:12.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.492+0000 7f6e2a7fc640 1 -- 172.21.15.84:0/3691413046 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6e300113f0 con 0x7f6e34065ab0 2023-10-16T10:05:12.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.493+0000 7f6e2a7fc640 1 --2- 172.21.15.84:0/3691413046 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f6e1806af90 0x7f6e1806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:12.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.493+0000 7f6e2a7fc640 1 -- 172.21.15.84:0/3691413046 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e30089a90 con 0x7f6e34065ab0 2023-10-16T10:05:12.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.493+0000 7f6e3934b640 1 --2- 172.21.15.84:0/3691413046 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f6e1806af90 0x7f6e1806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:12.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.494+0000 7f6e3934b640 1 --2- 172.21.15.84:0/3691413046 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f6e1806af90 0x7f6e1806d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e24006b10 tx=0x7f6e24008040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:12.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.497+0000 7f6e2a7fc640 1 -- 172.21.15.84:0/3691413046 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6e30054920 con 0x7f6e34065ab0 2023-10-16T10:05:12.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:12.730+0000 7f6e3bdb0640 1 -- 172.21.15.84:0/3691413046 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6e3404e160 con 0x7f6e34065ab0 2023-10-16T10:05:13.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.407+0000 7f6e2a7fc640 1 -- 172.21.15.84:0/3691413046 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6e3001f0b0 con 0x7f6e34065ab0 2023-10-16T10:05:13.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.411+0000 7f6e3bdb0640 1 -- 172.21.15.84:0/3691413046 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f6e1806af90 msgr2=0x7f6e1806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:13.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.411+0000 7f6e3bdb0640 1 --2- 172.21.15.84:0/3691413046 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f6e1806af90 0x7f6e1806d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e24006b10 tx=0x7f6e24008040 comp rx=0 tx=0).stop 2023-10-16T10:05:13.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.411+0000 7f6e3bdb0640 1 -- 172.21.15.84:0/3691413046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e34065ab0 msgr2=0x7f6e341578d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:13.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.411+0000 7f6e3bdb0640 1 --2- 172.21.15.84:0/3691413046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e34065ab0 0x7f6e341578d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000c7f0 tx=0x7f6e3000cbf0 comp rx=0 tx=0).stop 2023-10-16T10:05:13.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.412+0000 7f6e3bdb0640 1 -- 172.21.15.84:0/3691413046 shutdown_connections 2023-10-16T10:05:13.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.412+0000 7f6e3bdb0640 1 --2- 172.21.15.84:0/3691413046 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f6e1806af90 0x7f6e1806d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:13.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.412+0000 7f6e3bdb0640 1 --2- 172.21.15.84:0/3691413046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e34161470 0x7f6e3415c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:13.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.412+0000 7f6e3bdb0640 1 --2- 172.21.15.84:0/3691413046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e34160530 0x7f6e340b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:13.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.412+0000 7f6e3bdb0640 1 --2- 172.21.15.84:0/3691413046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e34065ab0 0x7f6e341578d0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:13.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.412+0000 7f6e3bdb0640 1 -- 172.21.15.84:0/3691413046 >> 172.21.15.84:0/3691413046 conn(0x7f6e3406bc10 msgr2=0x7f6e3415f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:13.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.412+0000 7f6e3bdb0640 1 -- 172.21.15.84:0/3691413046 shutdown_connections 2023-10-16T10:05:13.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.412+0000 7f6e3bdb0640 1 -- 172.21.15.84:0/3691413046 wait complete. 2023-10-16T10:05:13.434 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T10:05:13.435 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T10:05:13.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.646+0000 7f8130651640 1 Processor -- start 2023-10-16T10:05:13.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.646+0000 7f8130651640 1 -- start start 2023-10-16T10:05:13.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.648+0000 7f8130651640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8128159ab0 0x7f81280ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:13.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.648+0000 7f8130651640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f812806c140 0x7f812806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:13.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.648+0000 7f8130651640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8128070360 con 0x7f8128159e90 2023-10-16T10:05:13.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.648+0000 7f8130651640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812806cb10 con 0x7f812806c140 2023-10-16T10:05:13.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.648+0000 7f8130651640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812815d890 con 0x7f8128159ab0 2023-10-16T10:05:13.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.649+0000 7f812e3ed640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8128159ab0 0x7f81280ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:13.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.649+0000 7f812e3ed640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8128159ab0 0x7f81280ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:47904/0 (socket says 172.21.15.84:47904) 2023-10-16T10:05:13.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.649+0000 7f812e3ed640 1 -- 172.21.15.84:0/2771117528 learned_addr learned my addr 172.21.15.84:0/2771117528 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:13.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.649+0000 7f812ebee640 1 --2- 172.21.15.84:0/2771117528 >> v2:172.21.15.157:3300/0 conn(0x7f812806c140 0x7f812806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:13.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.649+0000 7f812e3ed640 1 -- 172.21.15.84:0/2771117528 >> v2:172.21.15.157:3300/0 conn(0x7f812806c140 msgr2=0x7f812806c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:13.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.649+0000 7f812e3ed640 1 --2- 172.21.15.84:0/2771117528 >> v2:172.21.15.157:3300/0 conn(0x7f812806c140 0x7f812806c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:13.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.649+0000 7f812e3ed640 1 -- 172.21.15.84:0/2771117528 >> v1:172.21.15.157:6789/0 conn(0x7f8128159e90 legacy=0x7f812806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:13.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.650+0000 7f812e3ed640 1 -- 172.21.15.84:0/2771117528 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81280cd1f0 con 0x7f8128159ab0 2023-10-16T10:05:13.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.651+0000 7f812e3ed640 1 --2- 172.21.15.84:0/2771117528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8128159ab0 0x7f81280ce370 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f812400bb40 tx=0x7f812402e030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cabe408bf60e4aa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:13.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.652+0000 7f812d3eb640 1 -- 172.21.15.84:0/2771117528 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812402ede0 con 0x7f8128159ab0 2023-10-16T10:05:13.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.652+0000 7f812d3eb640 1 -- 172.21.15.84:0/2771117528 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8124036440 con 0x7f8128159ab0 2023-10-16T10:05:13.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.652+0000 7f812d3eb640 1 -- 172.21.15.84:0/2771117528 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81240367d0 con 0x7f8128159ab0 2023-10-16T10:05:13.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.653+0000 7f8130651640 1 -- 172.21.15.84:0/2771117528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8128159ab0 msgr2=0x7f81280ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:13.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.653+0000 7f8130651640 1 --2- 172.21.15.84:0/2771117528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8128159ab0 0x7f81280ce370 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f812400bb40 tx=0x7f812402e030 comp rx=0 tx=0).stop 2023-10-16T10:05:13.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.653+0000 7f8130651640 1 -- 172.21.15.84:0/2771117528 shutdown_connections 2023-10-16T10:05:13.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.653+0000 7f8130651640 1 --2- 172.21.15.84:0/2771117528 >> v2:172.21.15.157:3300/0 conn(0x7f812806c140 0x7f812806c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:13.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.653+0000 7f8130651640 1 --2- 172.21.15.84:0/2771117528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8128159ab0 0x7f81280ce370 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:13.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.653+0000 7f8130651640 1 -- 172.21.15.84:0/2771117528 >> 172.21.15.84:0/2771117528 conn(0x7f81280ce9b0 msgr2=0x7f81280cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:13.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.653+0000 7f8130651640 1 -- 172.21.15.84:0/2771117528 shutdown_connections 2023-10-16T10:05:13.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.653+0000 7f8130651640 1 -- 172.21.15.84:0/2771117528 wait complete. 2023-10-16T10:05:13.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.655+0000 7f8130651640 1 Processor -- start 2023-10-16T10:05:13.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.655+0000 7f8130651640 1 -- start start 2023-10-16T10:05:13.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.656+0000 7f8130651640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f812806c140 0x7f81280c3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:13.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.656+0000 7f8130651640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8128159ab0 0x7f812806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:13.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.656+0000 7f8130651640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8128159e90 0x7f812815bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:13.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.656+0000 7f8130651640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81281516e0 con 0x7f8128159ab0 2023-10-16T10:05:13.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.656+0000 7f8130651640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8128150df0 con 0x7f8128159e90 2023-10-16T10:05:13.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.656+0000 7f8130651640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8128151090 con 0x7f812806c140 2023-10-16T10:05:13.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.656+0000 7f812dbec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8128159ab0 0x7f812806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:13.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.656+0000 7f812dbec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8128159ab0 0x7f812806a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42712/0 (socket says 172.21.15.84:42712) 2023-10-16T10:05:13.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.656+0000 7f812dbec640 1 -- 172.21.15.84:0/109762364 learned_addr learned my addr 172.21.15.84:0/109762364 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:13.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.657+0000 7f812e3ed640 1 --2- 172.21.15.84:0/109762364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f812806c140 0x7f81280c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:13.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.657+0000 7f812ebee640 1 --2- 172.21.15.84:0/109762364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8128159e90 0x7f812815bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:13.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.657+0000 7f812dbec640 1 -- 172.21.15.84:0/109762364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f812806c140 msgr2=0x7f81280c3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:13.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.657+0000 7f812dbec640 1 --2- 172.21.15.84:0/109762364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f812806c140 0x7f81280c3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:13.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.657+0000 7f812dbec640 1 -- 172.21.15.84:0/109762364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8128159e90 msgr2=0x7f812815bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:13.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.657+0000 7f812dbec640 1 --2- 172.21.15.84:0/109762364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8128159e90 0x7f812815bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:13.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.657+0000 7f812dbec640 1 -- 172.21.15.84:0/109762364 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812400b770 con 0x7f8128159ab0 2023-10-16T10:05:13.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.657+0000 7f812dbec640 1 --2- 172.21.15.84:0/109762364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8128159ab0 0x7f812806a010 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f811800c450 tx=0x7f811800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:13.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.657+0000 7f811effd640 1 -- 172.21.15.84:0/109762364 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81180052d0 con 0x7f8128159ab0 2023-10-16T10:05:13.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.657+0000 7f811effd640 1 -- 172.21.15.84:0/109762364 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8118005430 con 0x7f8128159ab0 2023-10-16T10:05:13.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.657+0000 7f811effd640 1 -- 172.21.15.84:0/109762364 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118012870 con 0x7f8128159ab0 2023-10-16T10:05:13.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.657+0000 7f8130651640 1 -- 172.21.15.84:0/109762364 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8128151370 con 0x7f8128159ab0 2023-10-16T10:05:13.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.658+0000 7f8130651640 1 -- 172.21.15.84:0/109762364 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812806a550 con 0x7f8128159ab0 2023-10-16T10:05:13.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.659+0000 7f8130651640 1 -- 172.21.15.84:0/109762364 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8128151890 con 0x7f8128159ab0 2023-10-16T10:05:13.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.660+0000 7f811effd640 1 -- 172.21.15.84:0/109762364 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f81180129d0 con 0x7f8128159ab0 2023-10-16T10:05:13.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.661+0000 7f811effd640 1 --2- 172.21.15.84:0/109762364 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f810c06af90 0x7f810c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:13.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.661+0000 7f811effd640 1 -- 172.21.15.84:0/109762364 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8118089230 con 0x7f8128159ab0 2023-10-16T10:05:13.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.661+0000 7f812e3ed640 1 --2- 172.21.15.84:0/109762364 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f810c06af90 0x7f810c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:13.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.662+0000 7f812e3ed640 1 --2- 172.21.15.84:0/109762364 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f810c06af90 0x7f810c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8124038000 tx=0x7f8124008040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:13.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.666+0000 7f811effd640 1 -- 172.21.15.84:0/109762364 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8128151890 con 0x7f8128159ab0 2023-10-16T10:05:13.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:13.891+0000 7f8130651640 1 -- 172.21.15.84:0/109762364 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8128151890 con 0x7f8128159ab0 2023-10-16T10:05:14.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.418+0000 7f811effd640 1 -- 172.21.15.84:0/109762364 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8128151890 con 0x7f8128159ab0 2023-10-16T10:05:14.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.422+0000 7f8130651640 1 -- 172.21.15.84:0/109762364 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f810c06af90 msgr2=0x7f810c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:14.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.422+0000 7f8130651640 1 --2- 172.21.15.84:0/109762364 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f810c06af90 0x7f810c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8124038000 tx=0x7f8124008040 comp rx=0 tx=0).stop 2023-10-16T10:05:14.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.422+0000 7f8130651640 1 -- 172.21.15.84:0/109762364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8128159ab0 msgr2=0x7f812806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:14.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.422+0000 7f8130651640 1 --2- 172.21.15.84:0/109762364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8128159ab0 0x7f812806a010 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f811800c450 tx=0x7f811800c910 comp rx=0 tx=0).stop 2023-10-16T10:05:14.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.422+0000 7f8130651640 1 -- 172.21.15.84:0/109762364 shutdown_connections 2023-10-16T10:05:14.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.422+0000 7f8130651640 1 --2- 172.21.15.84:0/109762364 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f810c06af90 0x7f810c06d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:14.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.422+0000 7f8130651640 1 --2- 172.21.15.84:0/109762364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8128159e90 0x7f812815bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:14.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.422+0000 7f8130651640 1 --2- 172.21.15.84:0/109762364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8128159ab0 0x7f812806a010 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:14.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.422+0000 7f8130651640 1 --2- 172.21.15.84:0/109762364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f812806c140 0x7f81280c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:14.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.422+0000 7f8130651640 1 -- 172.21.15.84:0/109762364 >> 172.21.15.84:0/109762364 conn(0x7f81280ce9b0 msgr2=0x7f81280c9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:14.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.422+0000 7f8130651640 1 -- 172.21.15.84:0/109762364 shutdown_connections 2023-10-16T10:05:14.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.422+0000 7f8130651640 1 -- 172.21.15.84:0/109762364 wait complete. 2023-10-16T10:05:14.444 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T10:05:14.444 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:05:14.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.643+0000 7f5179475640 1 Processor -- start 2023-10-16T10:05:14.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.644+0000 7f5179475640 1 -- start start 2023-10-16T10:05:14.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.645+0000 7f5179475640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f517415aaa0 0x7f517415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:14.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.645+0000 7f5179475640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51740ce390 0x7f5174070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:14.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.645+0000 7f5179475640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f517415ed60 con 0x7f517415ae80 2023-10-16T10:05:14.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.645+0000 7f5179475640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f517406b810 con 0x7f517415aaa0 2023-10-16T10:05:14.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.645+0000 7f5179475640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51740707e0 con 0x7f51740ce390 2023-10-16T10:05:14.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.646+0000 7f51737fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51740ce390 0x7f5174070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:14.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.646+0000 7f51737fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51740ce390 0x7f5174070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:47908/0 (socket says 172.21.15.84:47908) 2023-10-16T10:05:14.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.646+0000 7f51737fe640 1 -- 172.21.15.84:0/2185704617 learned_addr learned my addr 172.21.15.84:0/2185704617 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:14.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.646+0000 7f5172ffd640 1 --2- 172.21.15.84:0/2185704617 >> v2:172.21.15.157:3300/0 conn(0x7f517415aaa0 0x7f517415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:14.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.647+0000 7f51737fe640 1 -- 172.21.15.84:0/2185704617 >> v2:172.21.15.157:3300/0 conn(0x7f517415aaa0 msgr2=0x7f517415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:14.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.647+0000 7f51737fe640 1 --2- 172.21.15.84:0/2185704617 >> v2:172.21.15.157:3300/0 conn(0x7f517415aaa0 0x7f517415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:14.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.647+0000 7f51737fe640 1 -- 172.21.15.84:0/2185704617 >> v1:172.21.15.157:6789/0 conn(0x7f517415ae80 legacy=0x7f51740cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:14.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.647+0000 7f5171ffb640 1 -- 172.21.15.84:0/2185704617 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1634476883 0 0) 0x7f517415ed60 con 0x7f517415ae80 2023-10-16T10:05:14.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.647+0000 7f51737fe640 1 -- 172.21.15.84:0/2185704617 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517415ef90 con 0x7f51740ce390 2023-10-16T10:05:14.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.648+0000 7f51737fe640 1 --2- 172.21.15.84:0/2185704617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51740ce390 0x7f5174070f50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f516c009870 tx=0x7f516c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31747ff9e92aa24b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:14.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.649+0000 7f5171ffb640 1 -- 172.21.15.84:0/2185704617 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c030840 con 0x7f51740ce390 2023-10-16T10:05:14.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.649+0000 7f5171ffb640 1 -- 172.21.15.84:0/2185704617 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c0309a0 con 0x7f51740ce390 2023-10-16T10:05:14.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.649+0000 7f5171ffb640 1 -- 172.21.15.84:0/2185704617 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c030d30 con 0x7f51740ce390 2023-10-16T10:05:14.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.650+0000 7f5179475640 1 -- 172.21.15.84:0/2185704617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51740ce390 msgr2=0x7f5174070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:14.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.650+0000 7f5179475640 1 --2- 172.21.15.84:0/2185704617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51740ce390 0x7f5174070f50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f516c009870 tx=0x7f516c02b980 comp rx=0 tx=0).stop 2023-10-16T10:05:14.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.650+0000 7f5179475640 1 -- 172.21.15.84:0/2185704617 shutdown_connections 2023-10-16T10:05:14.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.650+0000 7f5179475640 1 --2- 172.21.15.84:0/2185704617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51740ce390 0x7f5174070f50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:14.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.650+0000 7f5179475640 1 --2- 172.21.15.84:0/2185704617 >> v2:172.21.15.157:3300/0 conn(0x7f517415aaa0 0x7f517415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:14.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.650+0000 7f5179475640 1 -- 172.21.15.84:0/2185704617 >> 172.21.15.84:0/2185704617 conn(0x7f51740cb410 msgr2=0x7f51740cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:14.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.650+0000 7f5179475640 1 -- 172.21.15.84:0/2185704617 shutdown_connections 2023-10-16T10:05:14.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.650+0000 7f5179475640 1 -- 172.21.15.84:0/2185704617 wait complete. 2023-10-16T10:05:14.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.651+0000 7f5179475640 1 Processor -- start 2023-10-16T10:05:14.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.652+0000 7f5179475640 1 -- start start 2023-10-16T10:05:14.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.652+0000 7f5179475640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51740ce390 0x7f517414a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:14.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.652+0000 7f5179475640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f517415aaa0 0x7f51740c3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:14.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.652+0000 7f5179475640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f517415ae80 0x7f51740c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:14.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.652+0000 7f5179475640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517415e3e0 con 0x7f517415ae80 2023-10-16T10:05:14.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.652+0000 7f5179475640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517414b790 con 0x7f517415aaa0 2023-10-16T10:05:14.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.652+0000 7f5179475640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517415b320 con 0x7f51740ce390 2023-10-16T10:05:14.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.653+0000 7f51737fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f517415ae80 0x7f51740c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:14.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.653+0000 7f51737fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f517415ae80 0x7f51740c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42716/0 (socket says 172.21.15.84:42716) 2023-10-16T10:05:14.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.653+0000 7f51737fe640 1 -- 172.21.15.84:0/3807450268 learned_addr learned my addr 172.21.15.84:0/3807450268 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:14.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.653+0000 7f5172ffd640 1 --2- 172.21.15.84:0/3807450268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51740ce390 0x7f517414a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:14.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.653+0000 7f51727fc640 1 --2- 172.21.15.84:0/3807450268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f517415aaa0 0x7f51740c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:14.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.653+0000 7f51737fe640 1 -- 172.21.15.84:0/3807450268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51740ce390 msgr2=0x7f517414a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:14.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.653+0000 7f51737fe640 1 --2- 172.21.15.84:0/3807450268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51740ce390 0x7f517414a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:14.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.653+0000 7f51737fe640 1 -- 172.21.15.84:0/3807450268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f517415aaa0 msgr2=0x7f51740c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:14.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.653+0000 7f51737fe640 1 --2- 172.21.15.84:0/3807450268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f517415aaa0 0x7f51740c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:14.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.653+0000 7f51737fe640 1 -- 172.21.15.84:0/3807450268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516c0094d0 con 0x7f517415ae80 2023-10-16T10:05:14.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.654+0000 7f51737fe640 1 --2- 172.21.15.84:0/3807450268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f517415ae80 0x7f51740c3a50 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f516c02b9c0 tx=0x7f516c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:14.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.654+0000 7f516b7fe640 1 -- 172.21.15.84:0/3807450268 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c039c90 con 0x7f517415ae80 2023-10-16T10:05:14.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.654+0000 7f516b7fe640 1 -- 172.21.15.84:0/3807450268 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c039df0 con 0x7f517415ae80 2023-10-16T10:05:14.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.654+0000 7f516b7fe640 1 -- 172.21.15.84:0/3807450268 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c043640 con 0x7f517415ae80 2023-10-16T10:05:14.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.654+0000 7f5179475640 1 -- 172.21.15.84:0/3807450268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517414ac20 con 0x7f517415ae80 2023-10-16T10:05:14.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.654+0000 7f5179475640 1 -- 172.21.15.84:0/3807450268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517414b0d0 con 0x7f517415ae80 2023-10-16T10:05:14.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.655+0000 7f5179475640 1 -- 172.21.15.84:0/3807450268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f517414af30 con 0x7f517415ae80 2023-10-16T10:05:14.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.656+0000 7f516b7fe640 1 -- 172.21.15.84:0/3807450268 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f516c0437a0 con 0x7f517415ae80 2023-10-16T10:05:14.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.657+0000 7f516b7fe640 1 --2- 172.21.15.84:0/3807450268 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f515406af90 0x7f515406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:14.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.657+0000 7f516b7fe640 1 -- 172.21.15.84:0/3807450268 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f516c03e070 con 0x7f517415ae80 2023-10-16T10:05:14.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.657+0000 7f5172ffd640 1 --2- 172.21.15.84:0/3807450268 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f515406af90 0x7f515406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:14.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.658+0000 7f5172ffd640 1 --2- 172.21.15.84:0/3807450268 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f515406af90 0x7f515406d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f51640074a0 tx=0x7f516400b040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:14.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.661+0000 7f516b7fe640 1 -- 172.21.15.84:0/3807450268 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f517414af30 con 0x7f517415ae80 2023-10-16T10:05:14.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.892+0000 7f5179475640 1 -- 172.21.15.84:0/3807450268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f517414b8d0 con 0x7f517415ae80 2023-10-16T10:05:14.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.893+0000 7f516b7fe640 1 -- 172.21.15.84:0/3807450268 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f517414b8d0 con 0x7f517415ae80 2023-10-16T10:05:14.895 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:14.895 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/89534235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":89534235},{"type":"v1","addr":"172.21.15.157:6835","nonce":89534235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3807666966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3807666966},{"type":"v1","addr":"172.21.15.157:6834","nonce":3807666966}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3448417053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3448417053},{"type":"v1","addr":"172.21.15.84:6835","nonce":3448417053}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:05:10.829122+0000","modified":"2023-10-16T10:05:13.893618+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7044":{"gid":7044,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/1939610713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":1939610713},{"type":"v1","addr":"172.21.15.84:6837","nonce":1939610713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-16T10:05:14.895 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 44 2023-10-16T10:05:14.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.896+0000 7f5179475640 1 -- 172.21.15.84:0/3807450268 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f515406af90 msgr2=0x7f515406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:14.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.896+0000 7f5179475640 1 --2- 172.21.15.84:0/3807450268 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f515406af90 0x7f515406d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f51640074a0 tx=0x7f516400b040 comp rx=0 tx=0).stop 2023-10-16T10:05:14.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.896+0000 7f5179475640 1 -- 172.21.15.84:0/3807450268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f517415ae80 msgr2=0x7f51740c3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:14.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.896+0000 7f5179475640 1 --2- 172.21.15.84:0/3807450268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f517415ae80 0x7f51740c3a50 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f516c02b9c0 tx=0x7f516c004730 comp rx=0 tx=0).stop 2023-10-16T10:05:14.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.896+0000 7f5179475640 1 -- 172.21.15.84:0/3807450268 shutdown_connections 2023-10-16T10:05:14.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.896+0000 7f5179475640 1 --2- 172.21.15.84:0/3807450268 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f515406af90 0x7f515406d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:14.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.896+0000 7f5179475640 1 --2- 172.21.15.84:0/3807450268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f517415ae80 0x7f51740c3a50 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:14.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.896+0000 7f5179475640 1 --2- 172.21.15.84:0/3807450268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f517415aaa0 0x7f51740c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:14.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.896+0000 7f5179475640 1 --2- 172.21.15.84:0/3807450268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51740ce390 0x7f517414a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:14.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.896+0000 7f5179475640 1 -- 172.21.15.84:0/3807450268 >> 172.21.15.84:0/3807450268 conn(0x7f51740cb410 msgr2=0x7f517406fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:14.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.896+0000 7f5179475640 1 -- 172.21.15.84:0/3807450268 shutdown_connections 2023-10-16T10:05:14.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:14.896+0000 7f5179475640 1 -- 172.21.15.84:0/3807450268 wait complete. 2023-10-16T10:05:14.915 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:05:15.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.123+0000 7fcf7734e640 1 Processor -- start 2023-10-16T10:05:15.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.123+0000 7fcf7734e640 1 -- start start 2023-10-16T10:05:15.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.124+0000 7fcf7734e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf70070b60 0x7fcf7015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:15.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.124+0000 7fcf7734e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcf7006c100 0x7fcf7006c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:15.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.125+0000 7fcf7734e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf7015aef0 con 0x7fcf70159950 2023-10-16T10:05:15.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.125+0000 7fcf7734e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf7015e490 con 0x7fcf7006c100 2023-10-16T10:05:15.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.125+0000 7fcf7734e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7015d910 con 0x7fcf70070b60 2023-10-16T10:05:15.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.125+0000 7fcf7634c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fcf70159950 0x7fcf70070450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47062/0 (socket says 172.21.15.84:47062) 2023-10-16T10:05:15.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.125+0000 7fcf7634c640 1 -- 172.21.15.84:0/788919473 learned_addr learned my addr 172.21.15.84:0/788919473 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:15.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.125+0000 7fcf7534a640 1 -- 172.21.15.84:0/788919473 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837350681 0 0) 0x7fcf6c001110 con 0x7fcf70159950 2023-10-16T10:05:15.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.125+0000 7fcf7534a640 1 -- 172.21.15.84:0/788919473 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf5c003520 con 0x7fcf70159950 2023-10-16T10:05:15.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.125+0000 7fcf76b4d640 1 --2- 172.21.15.84:0/788919473 >> v2:172.21.15.157:3300/0 conn(0x7fcf7006c100 0x7fcf7006c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:15.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.125+0000 7fcf75b4b640 1 --2- 172.21.15.84:0/788919473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf70070b60 0x7fcf7015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:15.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.125+0000 7fcf7534a640 1 -- 172.21.15.84:0/788919473 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2869200011 0 0) 0x7fcf5c003520 con 0x7fcf70159950 2023-10-16T10:05:15.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.125+0000 7fcf7534a640 1 -- 172.21.15.84:0/788919473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf70070b60 msgr2=0x7fcf7015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:15.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf7534a640 1 --2- 172.21.15.84:0/788919473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf70070b60 0x7fcf7015a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf7534a640 1 -- 172.21.15.84:0/788919473 >> v2:172.21.15.157:3300/0 conn(0x7fcf7006c100 msgr2=0x7fcf7006c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:15.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf7534a640 1 --2- 172.21.15.84:0/788919473 >> v2:172.21.15.157:3300/0 conn(0x7fcf7006c100 0x7fcf7006c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf7534a640 1 -- 172.21.15.84:0/788919473 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7015dbf0 con 0x7fcf70159950 2023-10-16T10:05:15.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf7534a640 1 -- 172.21.15.84:0/788919473 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fcf6c001800 con 0x7fcf70159950 2023-10-16T10:05:15.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf7534a640 1 -- 172.21.15.84:0/788919473 >> v1:172.21.15.84:6789/0 conn(0x7fcf70159950 legacy=0x7fcf70070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:15.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf7534a640 1 --2- 172.21.15.84:0/788919473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf70070b60 0x7fcf5c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:15.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf7534a640 1 --2- 172.21.15.84:0/788919473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf5c003fa0 0x7fcf5c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:15.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf7534a640 1 --2- 172.21.15.84:0/788919473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf5c004d00 0x7fcf5c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:15.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf7534a640 1 -- 172.21.15.84:0/788919473 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7015e490 con 0x7fcf5c004d00 2023-10-16T10:05:15.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf7534a640 1 -- 172.21.15.84:0/788919473 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7015d910 con 0x7fcf5c003fa0 2023-10-16T10:05:15.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf7534a640 1 -- 172.21.15.84:0/788919473 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c00bb40 con 0x7fcf70070b60 2023-10-16T10:05:15.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf76b4d640 1 --2- 172.21.15.84:0/788919473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf5c004d00 0x7fcf5c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:15.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf7634c640 1 --2- 172.21.15.84:0/788919473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf70070b60 0x7fcf5c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:15.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf75b4b640 1 --2- 172.21.15.84:0/788919473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf5c003fa0 0x7fcf5c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:15.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf76b4d640 1 -- 172.21.15.84:0/788919473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf70070b60 msgr2=0x7fcf5c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:15.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf76b4d640 1 --2- 172.21.15.84:0/788919473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf70070b60 0x7fcf5c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf76b4d640 1 -- 172.21.15.84:0/788919473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf5c003fa0 msgr2=0x7fcf5c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:15.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf76b4d640 1 --2- 172.21.15.84:0/788919473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf5c003fa0 0x7fcf5c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.126+0000 7fcf76b4d640 1 -- 172.21.15.84:0/788919473 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c003520 con 0x7fcf5c004d00 2023-10-16T10:05:15.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.128+0000 7fcf76b4d640 1 --2- 172.21.15.84:0/788919473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf5c004d00 0x7fcf5c005180 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009ba0 tx=0x7fcf5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb42573993d3b8a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:15.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.128+0000 7fcf7534a640 1 -- 172.21.15.84:0/788919473 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68036440 con 0x7fcf5c004d00 2023-10-16T10:05:15.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.128+0000 7fcf7534a640 1 -- 172.21.15.84:0/788919473 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf680365a0 con 0x7fcf5c004d00 2023-10-16T10:05:15.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.128+0000 7fcf7534a640 1 -- 172.21.15.84:0/788919473 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68036930 con 0x7fcf5c004d00 2023-10-16T10:05:15.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.128+0000 7fcf7734e640 1 -- 172.21.15.84:0/788919473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf5c004d00 msgr2=0x7fcf5c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:15.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.128+0000 7fcf7734e640 1 --2- 172.21.15.84:0/788919473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf5c004d00 0x7fcf5c005180 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009ba0 tx=0x7fcf5c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.128+0000 7fcf76b4d640 1 -- 172.21.15.84:0/788919473 reap_dead start 2023-10-16T10:05:15.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.128+0000 7fcf7734e640 1 -- 172.21.15.84:0/788919473 shutdown_connections 2023-10-16T10:05:15.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.128+0000 7fcf7734e640 1 -- 172.21.15.84:0/788919473 >> 172.21.15.84:0/788919473 conn(0x7fcf700ce6a0 msgr2=0x7fcf700ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:15.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.128+0000 7fcf7734e640 1 -- 172.21.15.84:0/788919473 shutdown_connections 2023-10-16T10:05:15.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.129+0000 7fcf7734e640 1 -- 172.21.15.84:0/788919473 wait complete. 2023-10-16T10:05:15.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.130+0000 7fcf7734e640 1 Processor -- start 2023-10-16T10:05:15.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.130+0000 7fcf7734e640 1 -- start start 2023-10-16T10:05:15.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.131+0000 7fcf7734e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf5c004d00 0x7fcf701557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:15.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.131+0000 7fcf7734e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf701f5b20 0x7fcf700c1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:15.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.131+0000 7fcf7734e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf701f2530 0x7fcf700ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:15.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.131+0000 7fcf7734e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf700c7760 con 0x7fcf701f2530 2023-10-16T10:05:15.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.131+0000 7fcf7734e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf701f2df0 con 0x7fcf701f5b20 2023-10-16T10:05:15.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.131+0000 7fcf7734e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf701f30d0 con 0x7fcf5c004d00 2023-10-16T10:05:15.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.132+0000 7fcf76b4d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf701f2530 0x7fcf700ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.132+0000 7fcf76b4d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf701f2530 0x7fcf700ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42720/0 (socket says 172.21.15.84:42720) 2023-10-16T10:05:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.132+0000 7fcf76b4d640 1 -- 172.21.15.84:0/1678586860 learned_addr learned my addr 172.21.15.84:0/1678586860 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.132+0000 7fcf75b4b640 1 --2- 172.21.15.84:0/1678586860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf701f5b20 0x7fcf700c1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.132+0000 7fcf76b4d640 1 -- 172.21.15.84:0/1678586860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf5c004d00 msgr2=0x7fcf701557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:15.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.132+0000 7fcf7634c640 1 --2- 172.21.15.84:0/1678586860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf5c004d00 0x7fcf701557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.132+0000 7fcf76b4d640 1 --2- 172.21.15.84:0/1678586860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf5c004d00 0x7fcf701557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.132+0000 7fcf76b4d640 1 -- 172.21.15.84:0/1678586860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf701f5b20 msgr2=0x7fcf700c1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.132+0000 7fcf76b4d640 1 --2- 172.21.15.84:0/1678586860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf701f5b20 0x7fcf700c1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.132+0000 7fcf76b4d640 1 -- 172.21.15.84:0/1678586860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf701f33b0 con 0x7fcf701f2530 2023-10-16T10:05:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.132+0000 7fcf75b4b640 1 --2- 172.21.15.84:0/1678586860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf701f5b20 0x7fcf700c1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:15.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.132+0000 7fcf76b4d640 1 --2- 172.21.15.84:0/1678586860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf701f2530 0x7fcf700ac890 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fcf68039040 tx=0x7fcf68004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:15.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.133+0000 7fcf66ffd640 1 -- 172.21.15.84:0/1678586860 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6803f830 con 0x7fcf701f2530 2023-10-16T10:05:15.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.133+0000 7fcf66ffd640 1 -- 172.21.15.84:0/1678586860 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6803f990 con 0x7fcf701f2530 2023-10-16T10:05:15.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.133+0000 7fcf7734e640 1 -- 172.21.15.84:0/1678586860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf700c7a00 con 0x7fcf701f2530 2023-10-16T10:05:15.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.133+0000 7fcf66ffd640 1 -- 172.21.15.84:0/1678586860 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6803fd00 con 0x7fcf701f2530 2023-10-16T10:05:15.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.133+0000 7fcf7734e640 1 -- 172.21.15.84:0/1678586860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf700c2270 con 0x7fcf701f2530 2023-10-16T10:05:15.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.134+0000 7fcf7734e640 1 -- 172.21.15.84:0/1678586860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7006cc40 con 0x7fcf701f2530 2023-10-16T10:05:15.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.135+0000 7fcf66ffd640 1 -- 172.21.15.84:0/1678586860 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcf6803d070 con 0x7fcf701f2530 2023-10-16T10:05:15.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.136+0000 7fcf66ffd640 1 --2- 172.21.15.84:0/1678586860 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fcf5406afc0 0x7fcf5406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:15.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.137+0000 7fcf66ffd640 1 -- 172.21.15.84:0/1678586860 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf680abc10 con 0x7fcf701f2530 2023-10-16T10:05:15.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.137+0000 7fcf7634c640 1 --2- 172.21.15.84:0/1678586860 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fcf5406afc0 0x7fcf5406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:15.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.137+0000 7fcf7634c640 1 --2- 172.21.15.84:0/1678586860 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fcf5406afc0 0x7fcf5406d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c000f60 tx=0x7fcf6c000ed0 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:15.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.141+0000 7fcf66ffd640 1 -- 172.21.15.84:0/1678586860 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcf68076bb0 con 0x7fcf701f2530 2023-10-16T10:05:15.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.336+0000 7fcf7734e640 1 -- 172.21.15.84:0/1678586860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf7006cc40 con 0x7fcf701f2530 2023-10-16T10:05:15.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.338+0000 7fcf66ffd640 1 -- 172.21.15.84:0/1678586860 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16544 (secure 0 0 0) 0x7fcf6803b020 con 0x7fcf701f2530 2023-10-16T10:05:15.340 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:15.340 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":95,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:05:12.383998+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:05:08.636077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-16T10:05:09.841985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:05:15.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.340+0000 7fcf7734e640 1 -- 172.21.15.84:0/1678586860 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fcf5406afc0 msgr2=0x7fcf5406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:15.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.340+0000 7fcf7734e640 1 --2- 172.21.15.84:0/1678586860 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fcf5406afc0 0x7fcf5406d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c000f60 tx=0x7fcf6c000ed0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.340+0000 7fcf7734e640 1 -- 172.21.15.84:0/1678586860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf701f2530 msgr2=0x7fcf700ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:15.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.340+0000 7fcf7734e640 1 --2- 172.21.15.84:0/1678586860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf701f2530 0x7fcf700ac890 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fcf68039040 tx=0x7fcf68004ab0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.341+0000 7fcf7734e640 1 -- 172.21.15.84:0/1678586860 shutdown_connections 2023-10-16T10:05:15.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.341+0000 7fcf7734e640 1 --2- 172.21.15.84:0/1678586860 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fcf5406afc0 0x7fcf5406d480 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.341+0000 7fcf7734e640 1 --2- 172.21.15.84:0/1678586860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf701f2530 0x7fcf700ac890 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.341+0000 7fcf7734e640 1 --2- 172.21.15.84:0/1678586860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf701f5b20 0x7fcf700c1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.341+0000 7fcf7734e640 1 --2- 172.21.15.84:0/1678586860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf5c004d00 0x7fcf701557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.341+0000 7fcf7734e640 1 -- 172.21.15.84:0/1678586860 >> 172.21.15.84:0/1678586860 conn(0x7fcf700ce6a0 msgr2=0x7fcf700cd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:15.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.341+0000 7fcf7734e640 1 -- 172.21.15.84:0/1678586860 shutdown_connections 2023-10-16T10:05:15.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.341+0000 7fcf7734e640 1 -- 172.21.15.84:0/1678586860 wait complete. 2023-10-16T10:05:15.362 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T10:05:15.363 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:05:15.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.579+0000 7f51ca861640 1 Processor -- start 2023-10-16T10:05:15.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.579+0000 7f51ca861640 1 -- start start 2023-10-16T10:05:15.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.581+0000 7f51ca861640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f51c40b0b20 0x7f51c4164b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:15.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.581+0000 7f51ca861640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c406f7f0 con 0x7f51c4159b20 2023-10-16T10:05:15.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.581+0000 7f51ca861640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c406c5e0 con 0x7f51c4159f00 2023-10-16T10:05:15.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.581+0000 7f51ca861640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c406c8c0 con 0x7f51c40b0b20 2023-10-16T10:05:15.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.581+0000 7f51c3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f51c4159b20 0x7f51c406bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:59316/0 (socket says 172.21.15.84:59316) 2023-10-16T10:05:15.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.581+0000 7f51c3fff640 1 -- 172.21.15.84:0/1797935215 learned_addr learned my addr 172.21.15.84:0/1797935215 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:15.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.582+0000 7f51c37fe640 1 -- 172.21.15.84:0/1797935215 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4005448014 0 0) 0x7f51c406f7f0 con 0x7f51c4159b20 2023-10-16T10:05:15.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.582+0000 7f51c37fe640 1 -- 172.21.15.84:0/1797935215 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51b0003520 con 0x7f51c4159b20 2023-10-16T10:05:15.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.582+0000 7f51c8dfe640 1 --2- 172.21.15.84:0/1797935215 >> v2:172.21.15.157:3300/0 conn(0x7f51c40b0b20 0x7f51c4164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:15.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.582+0000 7f51c37fe640 1 -- 172.21.15.84:0/1797935215 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1590879570 0 0) 0x7f51b0003520 con 0x7f51c4159b20 2023-10-16T10:05:15.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.582+0000 7f51c37fe640 1 -- 172.21.15.84:0/1797935215 >> v2:172.21.15.157:3300/0 conn(0x7f51c40b0b20 msgr2=0x7f51c4164b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:15.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.582+0000 7f51c37fe640 1 --2- 172.21.15.84:0/1797935215 >> v2:172.21.15.157:3300/0 conn(0x7f51c40b0b20 0x7f51c4164b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.582+0000 7f51c37fe640 1 -- 172.21.15.84:0/1797935215 >> v1:172.21.15.157:6789/0 conn(0x7f51c4159f00 legacy=0x7f51c40b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:15.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.582+0000 7f51c37fe640 1 -- 172.21.15.84:0/1797935215 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c406cba0 con 0x7f51c4159b20 2023-10-16T10:05:15.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.582+0000 7f51c37fe640 1 -- 172.21.15.84:0/1797935215 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f51b4001670 con 0x7f51c4159b20 2023-10-16T10:05:15.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.582+0000 7f51c37fe640 1 -- 172.21.15.84:0/1797935215 >> v1:172.21.15.84:6789/0 conn(0x7f51c4159b20 legacy=0x7f51c406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:15.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.582+0000 7f51c37fe640 1 --2- 172.21.15.84:0/1797935215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51b0003970 0x7f51b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:15.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.582+0000 7f51c37fe640 1 --2- 172.21.15.84:0/1797935215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51b00047c0 0x7f51b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:15.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.582+0000 7f51c37fe640 1 --2- 172.21.15.84:0/1797935215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b0005180 0x7f51b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:15.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.582+0000 7f51c37fe640 1 -- 172.21.15.84:0/1797935215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c406c8c0 con 0x7f51b0003970 2023-10-16T10:05:15.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.582+0000 7f51c37fe640 1 -- 172.21.15.84:0/1797935215 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b000c0d0 con 0x7f51b00047c0 2023-10-16T10:05:15.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.582+0000 7f51c37fe640 1 -- 172.21.15.84:0/1797935215 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b000c290 con 0x7f51b0005180 2023-10-16T10:05:15.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.583+0000 7f51c3fff640 1 --2- 172.21.15.84:0/1797935215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51b0003970 0x7f51b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:15.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.583+0000 7f51bbfff640 1 --2- 172.21.15.84:0/1797935215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51b00047c0 0x7f51b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:15.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.583+0000 7f51c3fff640 1 -- 172.21.15.84:0/1797935215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b0005180 msgr2=0x7f51b000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:15.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.583+0000 7f51c3fff640 1 --2- 172.21.15.84:0/1797935215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b0005180 0x7f51b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.583+0000 7f51c3fff640 1 -- 172.21.15.84:0/1797935215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51b00047c0 msgr2=0x7f51b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:15.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.583+0000 7f51c3fff640 1 --2- 172.21.15.84:0/1797935215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51b00047c0 0x7f51b0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.583+0000 7f51c3fff640 1 -- 172.21.15.84:0/1797935215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b0003520 con 0x7f51b0003970 2023-10-16T10:05:15.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.583+0000 7f51bbfff640 1 --2- 172.21.15.84:0/1797935215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51b00047c0 0x7f51b0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:15.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.583+0000 7f51c8dfe640 1 -- 172.21.15.84:0/1797935215 reap_dead start 2023-10-16T10:05:15.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.584+0000 7f51c3fff640 1 --2- 172.21.15.84:0/1797935215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51b0003970 0x7f51b0003d90 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f51b400c190 tx=0x7f51b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d802b899d5439894 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:15.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.584+0000 7f51c37fe640 1 -- 172.21.15.84:0/1797935215 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b4005ad0 con 0x7f51b0003970 2023-10-16T10:05:15.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.585+0000 7f51c37fe640 1 -- 172.21.15.84:0/1797935215 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b4005c30 con 0x7f51b0003970 2023-10-16T10:05:15.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.585+0000 7f51c37fe640 1 -- 172.21.15.84:0/1797935215 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b402c8a0 con 0x7f51b0003970 2023-10-16T10:05:15.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.585+0000 7f51ca861640 1 -- 172.21.15.84:0/1797935215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51b0003970 msgr2=0x7f51b0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:15.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.585+0000 7f51ca861640 1 --2- 172.21.15.84:0/1797935215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51b0003970 0x7f51b0003d90 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f51b400c190 tx=0x7f51b00057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.585+0000 7f51ca861640 1 -- 172.21.15.84:0/1797935215 shutdown_connections 2023-10-16T10:05:15.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.585+0000 7f51ca861640 1 --2- 172.21.15.84:0/1797935215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51b0003970 0x7f51b0003d90 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.585+0000 7f51ca861640 1 -- 172.21.15.84:0/1797935215 >> 172.21.15.84:0/1797935215 conn(0x7f51c40cb410 msgr2=0x7f51c40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:15.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.585+0000 7f51ca861640 1 -- 172.21.15.84:0/1797935215 shutdown_connections 2023-10-16T10:05:15.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.585+0000 7f51ca861640 1 -- 172.21.15.84:0/1797935215 wait complete. 2023-10-16T10:05:15.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.587+0000 7f51ca861640 1 Processor -- start 2023-10-16T10:05:15.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.587+0000 7f51ca861640 1 -- start start 2023-10-16T10:05:15.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.588+0000 7f51ca861640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b0003970 0x7f51c40b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:15.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.588+0000 7f51ca861640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51c414f1c0 0x7f51c41f3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:15.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.588+0000 7f51ca861640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51c40b2d10 0x7f51c406fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:15.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.588+0000 7f51ca861640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40b3ee0 con 0x7f51c40b2d10 2023-10-16T10:05:15.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.588+0000 7f51ca861640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40c5280 con 0x7f51c414f1c0 2023-10-16T10:05:15.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.588+0000 7f51ca861640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c41f3e60 con 0x7f51b0003970 2023-10-16T10:05:15.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.588+0000 7f51c8dfe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51c40b2d10 0x7f51c406fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:15.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.588+0000 7f51c8dfe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51c40b2d10 0x7f51c406fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35490/0 (socket says 172.21.15.84:35490) 2023-10-16T10:05:15.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.588+0000 7f51c8dfe640 1 -- 172.21.15.84:0/4042078355 learned_addr learned my addr 172.21.15.84:0/4042078355 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:15.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.589+0000 7f51c3fff640 1 --2- 172.21.15.84:0/4042078355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b0003970 0x7f51c40b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:15.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.589+0000 7f51bbfff640 1 --2- 172.21.15.84:0/4042078355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51c414f1c0 0x7f51c41f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:15.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.589+0000 7f51c8dfe640 1 -- 172.21.15.84:0/4042078355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b0003970 msgr2=0x7f51c40b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:15.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.589+0000 7f51c8dfe640 1 --2- 172.21.15.84:0/4042078355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b0003970 0x7f51c40b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.589+0000 7f51c8dfe640 1 -- 172.21.15.84:0/4042078355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51c414f1c0 msgr2=0x7f51c41f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:15.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.589+0000 7f51c8dfe640 1 --2- 172.21.15.84:0/4042078355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51c414f1c0 0x7f51c41f3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.589+0000 7f51c8dfe640 1 -- 172.21.15.84:0/4042078355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c40b32d0 con 0x7f51c40b2d10 2023-10-16T10:05:15.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.589+0000 7f51c3fff640 1 --2- 172.21.15.84:0/4042078355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b0003970 0x7f51c40b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:15.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.589+0000 7f51c8dfe640 1 --2- 172.21.15.84:0/4042078355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51c40b2d10 0x7f51c406fc80 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f51c406d340 tx=0x7f51bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:15.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.589+0000 7f51c17fa640 1 -- 172.21.15.84:0/4042078355 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc007e40 con 0x7f51c40b2d10 2023-10-16T10:05:15.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.589+0000 7f51c17fa640 1 -- 172.21.15.84:0/4042078355 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51bc014420 con 0x7f51c40b2d10 2023-10-16T10:05:15.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.589+0000 7f51c17fa640 1 -- 172.21.15.84:0/4042078355 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc0147d0 con 0x7f51c40b2d10 2023-10-16T10:05:15.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.589+0000 7f51ca861640 1 -- 172.21.15.84:0/4042078355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c40b35b0 con 0x7f51c40b2d10 2023-10-16T10:05:15.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.590+0000 7f51ca861640 1 -- 172.21.15.84:0/4042078355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c40b3b70 con 0x7f51c40b2d10 2023-10-16T10:05:15.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.591+0000 7f51ca861640 1 -- 172.21.15.84:0/4042078355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c40b3d30 con 0x7f51c40b2d10 2023-10-16T10:05:15.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.592+0000 7f51c17fa640 1 -- 172.21.15.84:0/4042078355 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f51bc014930 con 0x7f51c40b2d10 2023-10-16T10:05:15.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.593+0000 7f51c17fa640 1 --2- 172.21.15.84:0/4042078355 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f51a406afc0 0x7f51a406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:15.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.593+0000 7f51c17fa640 1 -- 172.21.15.84:0/4042078355 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51bc089be0 con 0x7f51c40b2d10 2023-10-16T10:05:15.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.593+0000 7f51c3fff640 1 --2- 172.21.15.84:0/4042078355 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f51a406afc0 0x7f51a406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:15.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.594+0000 7f51c3fff640 1 --2- 172.21.15.84:0/4042078355 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f51a406afc0 0x7f51a406d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f51b4034000 tx=0x7f51b4009030 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:15.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.597+0000 7f51c17fa640 1 -- 172.21.15.84:0/4042078355 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f51bc054b80 con 0x7f51c40b2d10 2023-10-16T10:05:15.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.786+0000 7f51ca861640 1 -- 172.21.15.84:0/4042078355 --> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f51c406ce80 con 0x7f51a406afc0 2023-10-16T10:05:15.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.815+0000 7f51c17fa640 1 -- 172.21.15.84:0/4042078355 <== mgr.6786 v2:172.21.15.84:6832/147697688 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287048 (secure 0 0 0) 0x7f51b4005970 con 0x7f51a406afc0 2023-10-16T10:05:15.820 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:15.822 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T10:05:15.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.821+0000 7f51ca861640 1 -- 172.21.15.84:0/4042078355 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f51a406afc0 msgr2=0x7f51a406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:15.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.821+0000 7f51ca861640 1 --2- 172.21.15.84:0/4042078355 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f51a406afc0 0x7f51a406d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f51b4034000 tx=0x7f51b4009030 comp rx=0 tx=0).stop 2023-10-16T10:05:15.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.821+0000 7f51ca861640 1 -- 172.21.15.84:0/4042078355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51c40b2d10 msgr2=0x7f51c406fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:15.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.821+0000 7f51ca861640 1 --2- 172.21.15.84:0/4042078355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51c40b2d10 0x7f51c406fc80 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f51c406d340 tx=0x7f51bc00e4b0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.821+0000 7f51ca861640 1 -- 172.21.15.84:0/4042078355 shutdown_connections 2023-10-16T10:05:15.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.821+0000 7f51ca861640 1 --2- 172.21.15.84:0/4042078355 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f51a406afc0 0x7f51a406d480 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.821+0000 7f51ca861640 1 --2- 172.21.15.84:0/4042078355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51c40b2d10 0x7f51c406fc80 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.821+0000 7f51ca861640 1 --2- 172.21.15.84:0/4042078355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51c414f1c0 0x7f51c41f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.821+0000 7f51ca861640 1 --2- 172.21.15.84:0/4042078355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b0003970 0x7f51c40b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:15.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.822+0000 7f51ca861640 1 -- 172.21.15.84:0/4042078355 >> 172.21.15.84:0/4042078355 conn(0x7f51c40cb410 msgr2=0x7f51c41579b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:15.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.822+0000 7f51ca861640 1 -- 172.21.15.84:0/4042078355 shutdown_connections 2023-10-16T10:05:15.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:15.822+0000 7f51ca861640 1 -- 172.21.15.84:0/4042078355 wait complete. 2023-10-16T10:05:15.843 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-16T10:05:15.474186+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":600,"num_read_kb":1133,"num_write":303,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":316612,"kb_used_data":10244,"kb_used_omap":12,"kb_used_meta":306227,"kb_avail":754658108,"statfs":{"total":773094113280,"available":772769902592,"internally_reserved":0,"allocated":10489856,"data_stored":5582712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":313577017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001327"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.323308+0000","last_change":"2023-10-16T10:05:11.414668+0000","last_active":"2023-10-16T10:05:14.323308+0000","last_peered":"2023-10-16T10:05:14.323308+0000","last_clean":"2023-10-16T10:05:14.323308+0000","last_became_active":"2023-10-16T10:05:11.414182+0000","last_became_peered":"2023-10-16T10:05:11.414182+0000","last_unstale":"2023-10-16T10:05:14.323308+0000","last_undegraded":"2023-10-16T10:05:14.323308+0000","last_fullsized":"2023-10-16T10:05:14.323308+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:58:48.434293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.198774+0000","last_change":"2023-10-16T10:05:11.403026+0000","last_active":"2023-10-16T10:05:13.198774+0000","last_peered":"2023-10-16T10:05:13.198774+0000","last_clean":"2023-10-16T10:05:13.198774+0000","last_became_active":"2023-10-16T10:05:11.401337+0000","last_became_peered":"2023-10-16T10:05:11.401337+0000","last_unstale":"2023-10-16T10:05:13.198774+0000","last_undegraded":"2023-10-16T10:05:13.198774+0000","last_fullsized":"2023-10-16T10:05:13.198774+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:38:09.960436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.399655+0000","last_change":"2023-10-16T10:05:11.396721+0000","last_active":"2023-10-16T10:05:11.399655+0000","last_peered":"2023-10-16T10:05:11.399655+0000","last_clean":"2023-10-16T10:05:11.399655+0000","last_became_active":"2023-10-16T10:05:11.395538+0000","last_became_peered":"2023-10-16T10:05:11.395538+0000","last_unstale":"2023-10-16T10:05:11.399655+0000","last_undegraded":"2023-10-16T10:05:11.399655+0000","last_fullsized":"2023-10-16T10:05:11.399655+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:56.490714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.400652+0000","last_change":"2023-10-16T10:05:11.395295+0000","last_active":"2023-10-16T10:05:11.400652+0000","last_peered":"2023-10-16T10:05:11.400652+0000","last_clean":"2023-10-16T10:05:11.400652+0000","last_became_active":"2023-10-16T10:05:11.394833+0000","last_became_peered":"2023-10-16T10:05:11.394833+0000","last_unstale":"2023-10-16T10:05:11.400652+0000","last_undegraded":"2023-10-16T10:05:11.400652+0000","last_fullsized":"2023-10-16T10:05:11.400652+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:06:07.971004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.397367+0000","last_change":"2023-10-16T10:05:11.412427+0000","last_active":"2023-10-16T10:05:12.397367+0000","last_peered":"2023-10-16T10:05:12.397367+0000","last_clean":"2023-10-16T10:05:12.397367+0000","last_became_active":"2023-10-16T10:05:11.411718+0000","last_became_peered":"2023-10-16T10:05:11.411718+0000","last_unstale":"2023-10-16T10:05:12.397367+0000","last_undegraded":"2023-10-16T10:05:12.397367+0000","last_fullsized":"2023-10-16T10:05:12.397367+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:51.178511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.198597+0000","last_change":"2023-10-16T10:05:11.403384+0000","last_active":"2023-10-16T10:05:13.198597+0000","last_peered":"2023-10-16T10:05:13.198597+0000","last_clean":"2023-10-16T10:05:13.198597+0000","last_became_active":"2023-10-16T10:05:11.401451+0000","last_became_peered":"2023-10-16T10:05:11.401451+0000","last_unstale":"2023-10-16T10:05:13.198597+0000","last_undegraded":"2023-10-16T10:05:13.198597+0000","last_fullsized":"2023-10-16T10:05:13.198597+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:57.580285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.082540+0000","last_change":"2023-10-16T10:05:11.406465+0000","last_active":"2023-10-16T10:05:13.082540+0000","last_peered":"2023-10-16T10:05:13.082540+0000","last_clean":"2023-10-16T10:05:13.082540+0000","last_became_active":"2023-10-16T10:05:11.404685+0000","last_became_peered":"2023-10-16T10:05:11.404685+0000","last_unstale":"2023-10-16T10:05:13.082540+0000","last_undegraded":"2023-10-16T10:05:13.082540+0000","last_fullsized":"2023-10-16T10:05:13.082540+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:21:57.974477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.198240+0000","last_change":"2023-10-16T10:05:11.388725+0000","last_active":"2023-10-16T10:05:13.198240+0000","last_peered":"2023-10-16T10:05:13.198240+0000","last_clean":"2023-10-16T10:05:13.198240+0000","last_became_active":"2023-10-16T10:05:11.387352+0000","last_became_peered":"2023-10-16T10:05:11.387352+0000","last_unstale":"2023-10-16T10:05:13.198240+0000","last_undegraded":"2023-10-16T10:05:13.198240+0000","last_fullsized":"2023-10-16T10:05:13.198240+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:55.605058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.397888+0000","last_change":"2023-10-16T10:05:11.398654+0000","last_active":"2023-10-16T10:05:12.397888+0000","last_peered":"2023-10-16T10:05:12.397888+0000","last_clean":"2023-10-16T10:05:12.397888+0000","last_became_active":"2023-10-16T10:05:11.398131+0000","last_became_peered":"2023-10-16T10:05:11.398131+0000","last_unstale":"2023-10-16T10:05:12.397888+0000","last_undegraded":"2023-10-16T10:05:12.397888+0000","last_fullsized":"2023-10-16T10:05:12.397888+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:03:16.251267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.401043+0000","last_change":"2023-10-16T10:05:11.395215+0000","last_active":"2023-10-16T10:05:12.401043+0000","last_peered":"2023-10-16T10:05:12.401043+0000","last_clean":"2023-10-16T10:05:12.401043+0000","last_became_active":"2023-10-16T10:05:11.394873+0000","last_became_peered":"2023-10-16T10:05:11.394873+0000","last_unstale":"2023-10-16T10:05:12.401043+0000","last_undegraded":"2023-10-16T10:05:12.401043+0000","last_fullsized":"2023-10-16T10:05:12.401043+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:23.942054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.323206+0000","last_change":"2023-10-16T10:05:11.413876+0000","last_active":"2023-10-16T10:05:14.323206+0000","last_peered":"2023-10-16T10:05:14.323206+0000","last_clean":"2023-10-16T10:05:14.323206+0000","last_became_active":"2023-10-16T10:05:11.413576+0000","last_became_peered":"2023-10-16T10:05:11.413576+0000","last_unstale":"2023-10-16T10:05:14.323206+0000","last_undegraded":"2023-10-16T10:05:14.323206+0000","last_fullsized":"2023-10-16T10:05:14.323206+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:22.397230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.399291+0000","last_change":"2023-10-16T10:05:11.393132+0000","last_active":"2023-10-16T10:05:11.399291+0000","last_peered":"2023-10-16T10:05:11.399291+0000","last_clean":"2023-10-16T10:05:11.399291+0000","last_became_active":"2023-10-16T10:05:11.392519+0000","last_became_peered":"2023-10-16T10:05:11.392519+0000","last_unstale":"2023-10-16T10:05:11.399291+0000","last_undegraded":"2023-10-16T10:05:11.399291+0000","last_fullsized":"2023-10-16T10:05:11.399291+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:33:37.874737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.398866+0000","last_change":"2023-10-16T10:05:11.393902+0000","last_active":"2023-10-16T10:05:12.398866+0000","last_peered":"2023-10-16T10:05:12.398866+0000","last_clean":"2023-10-16T10:05:12.398866+0000","last_became_active":"2023-10-16T10:05:11.393633+0000","last_became_peered":"2023-10-16T10:05:11.393633+0000","last_unstale":"2023-10-16T10:05:12.398866+0000","last_undegraded":"2023-10-16T10:05:12.398866+0000","last_fullsized":"2023-10-16T10:05:12.398866+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:38:09.960436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.397519+0000","last_change":"2023-10-16T10:05:11.405003+0000","last_active":"2023-10-16T10:05:12.397519+0000","last_peered":"2023-10-16T10:05:12.397519+0000","last_clean":"2023-10-16T10:05:12.397519+0000","last_became_active":"2023-10-16T10:05:11.404691+0000","last_became_peered":"2023-10-16T10:05:11.404691+0000","last_unstale":"2023-10-16T10:05:12.397519+0000","last_undegraded":"2023-10-16T10:05:12.397519+0000","last_fullsized":"2023-10-16T10:05:12.397519+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:46:53.588751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.399221+0000","last_change":"2023-10-16T10:05:11.387839+0000","last_active":"2023-10-16T10:05:12.399221+0000","last_peered":"2023-10-16T10:05:12.399221+0000","last_clean":"2023-10-16T10:05:12.399221+0000","last_became_active":"2023-10-16T10:05:11.387575+0000","last_became_peered":"2023-10-16T10:05:11.387575+0000","last_unstale":"2023-10-16T10:05:12.399221+0000","last_undegraded":"2023-10-16T10:05:12.399221+0000","last_fullsized":"2023-10-16T10:05:12.399221+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:35.765791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.399587+0000","last_change":"2023-10-16T10:05:11.397970+0000","last_active":"2023-10-16T10:05:12.399587+0000","last_peered":"2023-10-16T10:05:12.399587+0000","last_clean":"2023-10-16T10:05:12.399587+0000","last_became_active":"2023-10-16T10:05:11.397686+0000","last_became_peered":"2023-10-16T10:05:11.397686+0000","last_unstale":"2023-10-16T10:05:12.399587+0000","last_undegraded":"2023-10-16T10:05:12.399587+0000","last_fullsized":"2023-10-16T10:05:12.399587+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:14.948508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.398035+0000","last_change":"2023-10-16T10:05:11.398459+0000","last_active":"2023-10-16T10:05:12.398035+0000","last_peered":"2023-10-16T10:05:12.398035+0000","last_clean":"2023-10-16T10:05:12.398035+0000","last_became_active":"2023-10-16T10:05:11.397678+0000","last_became_peered":"2023-10-16T10:05:11.397678+0000","last_unstale":"2023-10-16T10:05:12.398035+0000","last_undegraded":"2023-10-16T10:05:12.398035+0000","last_fullsized":"2023-10-16T10:05:12.398035+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:06:20.963899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.403886+0000","last_change":"2023-10-16T10:05:11.401189+0000","last_active":"2023-10-16T10:05:11.403886+0000","last_peered":"2023-10-16T10:05:11.403886+0000","last_clean":"2023-10-16T10:05:11.403886+0000","last_became_active":"2023-10-16T10:05:11.400916+0000","last_became_peered":"2023-10-16T10:05:11.400916+0000","last_unstale":"2023-10-16T10:05:11.403886+0000","last_undegraded":"2023-10-16T10:05:11.403886+0000","last_fullsized":"2023-10-16T10:05:11.403886+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:59:06.547242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.198186+0000","last_change":"2023-10-16T10:05:11.391390+0000","last_active":"2023-10-16T10:05:13.198186+0000","last_peered":"2023-10-16T10:05:13.198186+0000","last_clean":"2023-10-16T10:05:13.198186+0000","last_became_active":"2023-10-16T10:05:11.391087+0000","last_became_peered":"2023-10-16T10:05:11.391087+0000","last_unstale":"2023-10-16T10:05:13.198186+0000","last_undegraded":"2023-10-16T10:05:13.198186+0000","last_fullsized":"2023-10-16T10:05:13.198186+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:20:34.386787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.399044+0000","last_change":"2023-10-16T10:05:11.401586+0000","last_active":"2023-10-16T10:05:12.399044+0000","last_peered":"2023-10-16T10:05:12.399044+0000","last_clean":"2023-10-16T10:05:12.399044+0000","last_became_active":"2023-10-16T10:05:11.401303+0000","last_became_peered":"2023-10-16T10:05:11.401303+0000","last_unstale":"2023-10-16T10:05:12.399044+0000","last_undegraded":"2023-10-16T10:05:12.399044+0000","last_fullsized":"2023-10-16T10:05:12.399044+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:23:38.009256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.400099+0000","last_change":"2023-10-16T10:05:11.392880+0000","last_active":"2023-10-16T10:05:12.400099+0000","last_peered":"2023-10-16T10:05:12.400099+0000","last_clean":"2023-10-16T10:05:12.400099+0000","last_became_active":"2023-10-16T10:05:11.392540+0000","last_became_peered":"2023-10-16T10:05:11.392540+0000","last_unstale":"2023-10-16T10:05:12.400099+0000","last_undegraded":"2023-10-16T10:05:12.400099+0000","last_fullsized":"2023-10-16T10:05:12.400099+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:43:06.561050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.400009+0000","last_change":"2023-10-16T10:05:11.389170+0000","last_active":"2023-10-16T10:05:12.400009+0000","last_peered":"2023-10-16T10:05:12.400009+0000","last_clean":"2023-10-16T10:05:12.400009+0000","last_became_active":"2023-10-16T10:05:11.388765+0000","last_became_peered":"2023-10-16T10:05:11.388765+0000","last_unstale":"2023-10-16T10:05:12.400009+0000","last_undegraded":"2023-10-16T10:05:12.400009+0000","last_fullsized":"2023-10-16T10:05:12.400009+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:33:53.035562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.322993+0000","last_change":"2023-10-16T10:05:11.414359+0000","last_active":"2023-10-16T10:05:14.322993+0000","last_peered":"2023-10-16T10:05:14.322993+0000","last_clean":"2023-10-16T10:05:14.322993+0000","last_became_active":"2023-10-16T10:05:11.414015+0000","last_became_peered":"2023-10-16T10:05:11.414015+0000","last_unstale":"2023-10-16T10:05:14.322993+0000","last_undegraded":"2023-10-16T10:05:14.322993+0000","last_fullsized":"2023-10-16T10:05:14.322993+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:57:56.336265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.322581+0000","last_change":"2023-10-16T10:05:11.403248+0000","last_active":"2023-10-16T10:05:14.322581+0000","last_peered":"2023-10-16T10:05:14.322581+0000","last_clean":"2023-10-16T10:05:14.322581+0000","last_became_active":"2023-10-16T10:05:11.402947+0000","last_became_peered":"2023-10-16T10:05:11.402947+0000","last_unstale":"2023-10-16T10:05:14.322581+0000","last_undegraded":"2023-10-16T10:05:14.322581+0000","last_fullsized":"2023-10-16T10:05:14.322581+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:28:10.619415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.401081+0000","last_change":"2023-10-16T10:05:11.408662+0000","last_active":"2023-10-16T10:05:12.401081+0000","last_peered":"2023-10-16T10:05:12.401081+0000","last_clean":"2023-10-16T10:05:12.401081+0000","last_became_active":"2023-10-16T10:05:11.407404+0000","last_became_peered":"2023-10-16T10:05:11.407404+0000","last_unstale":"2023-10-16T10:05:12.401081+0000","last_undegraded":"2023-10-16T10:05:12.401081+0000","last_fullsized":"2023-10-16T10:05:12.401081+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:43.832121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.082939+0000","last_change":"2023-10-16T10:05:11.404472+0000","last_active":"2023-10-16T10:05:13.082939+0000","last_peered":"2023-10-16T10:05:13.082939+0000","last_clean":"2023-10-16T10:05:13.082939+0000","last_became_active":"2023-10-16T10:05:11.404069+0000","last_became_peered":"2023-10-16T10:05:11.404069+0000","last_unstale":"2023-10-16T10:05:13.082939+0000","last_undegraded":"2023-10-16T10:05:13.082939+0000","last_fullsized":"2023-10-16T10:05:13.082939+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:27:45.274018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.412492+0000","last_change":"2023-10-16T10:05:11.411252+0000","last_active":"2023-10-16T10:05:11.412492+0000","last_peered":"2023-10-16T10:05:11.412492+0000","last_clean":"2023-10-16T10:05:11.412492+0000","last_became_active":"2023-10-16T10:05:11.410883+0000","last_became_peered":"2023-10-16T10:05:11.410883+0000","last_unstale":"2023-10-16T10:05:11.412492+0000","last_undegraded":"2023-10-16T10:05:11.412492+0000","last_fullsized":"2023-10-16T10:05:11.412492+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:51.160809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.400726+0000","last_change":"2023-10-16T10:05:11.394898+0000","last_active":"2023-10-16T10:05:12.400726+0000","last_peered":"2023-10-16T10:05:12.400726+0000","last_clean":"2023-10-16T10:05:12.400726+0000","last_became_active":"2023-10-16T10:05:11.394561+0000","last_became_peered":"2023-10-16T10:05:11.394561+0000","last_unstale":"2023-10-16T10:05:12.400726+0000","last_undegraded":"2023-10-16T10:05:12.400726+0000","last_fullsized":"2023-10-16T10:05:12.400726+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:02:20.574735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.400021+0000","last_change":"2023-10-16T10:05:11.397623+0000","last_active":"2023-10-16T10:05:11.400021+0000","last_peered":"2023-10-16T10:05:11.400021+0000","last_clean":"2023-10-16T10:05:11.400021+0000","last_became_active":"2023-10-16T10:05:11.397357+0000","last_became_peered":"2023-10-16T10:05:11.397357+0000","last_unstale":"2023-10-16T10:05:11.400021+0000","last_undegraded":"2023-10-16T10:05:11.400021+0000","last_fullsized":"2023-10-16T10:05:11.400021+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:18:54.637543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.082692+0000","last_change":"2023-10-16T10:05:11.405893+0000","last_active":"2023-10-16T10:05:13.082692+0000","last_peered":"2023-10-16T10:05:13.082692+0000","last_clean":"2023-10-16T10:05:13.082692+0000","last_became_active":"2023-10-16T10:05:11.405500+0000","last_became_peered":"2023-10-16T10:05:11.405500+0000","last_unstale":"2023-10-16T10:05:13.082692+0000","last_undegraded":"2023-10-16T10:05:13.082692+0000","last_fullsized":"2023-10-16T10:05:13.082692+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:36:38.823109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.082897+0000","last_change":"2023-10-16T10:05:11.409375+0000","last_active":"2023-10-16T10:05:13.082897+0000","last_peered":"2023-10-16T10:05:13.082897+0000","last_clean":"2023-10-16T10:05:13.082897+0000","last_became_active":"2023-10-16T10:05:11.409050+0000","last_became_peered":"2023-10-16T10:05:11.409050+0000","last_unstale":"2023-10-16T10:05:13.082897+0000","last_undegraded":"2023-10-16T10:05:13.082897+0000","last_fullsized":"2023-10-16T10:05:13.082897+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:53:31.471053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.198162+0000","last_change":"2023-10-16T10:05:11.403415+0000","last_active":"2023-10-16T10:05:13.198162+0000","last_peered":"2023-10-16T10:05:13.198162+0000","last_clean":"2023-10-16T10:05:13.198162+0000","last_became_active":"2023-10-16T10:05:11.403027+0000","last_became_peered":"2023-10-16T10:05:11.403027+0000","last_unstale":"2023-10-16T10:05:13.198162+0000","last_undegraded":"2023-10-16T10:05:13.198162+0000","last_fullsized":"2023-10-16T10:05:13.198162+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:05:42.255119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.323385+0000","last_change":"2023-10-16T10:05:11.420161+0000","last_active":"2023-10-16T10:05:14.323385+0000","last_peered":"2023-10-16T10:05:14.323385+0000","last_clean":"2023-10-16T10:05:14.323385+0000","last_became_active":"2023-10-16T10:05:11.419839+0000","last_became_peered":"2023-10-16T10:05:11.419839+0000","last_unstale":"2023-10-16T10:05:14.323385+0000","last_undegraded":"2023-10-16T10:05:14.323385+0000","last_fullsized":"2023-10-16T10:05:14.323385+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:10.036163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"94'4","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.418053+0000","last_change":"2023-10-16T10:05:10.382295+0000","last_active":"2023-10-16T10:05:11.418053+0000","last_peered":"2023-10-16T10:05:11.418053+0000","last_clean":"2023-10-16T10:05:11.418053+0000","last_became_active":"2023-10-16T10:05:10.381968+0000","last_became_peered":"2023-10-16T10:05:10.381968+0000","last_unstale":"2023-10-16T10:05:11.418053+0000","last_undegraded":"2023-10-16T10:05:11.418053+0000","last_fullsized":"2023-10-16T10:05:11.418053+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:31.389526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.398217+0000","last_change":"2023-10-16T10:05:11.398829+0000","last_active":"2023-10-16T10:05:12.398217+0000","last_peered":"2023-10-16T10:05:12.398217+0000","last_clean":"2023-10-16T10:05:12.398217+0000","last_became_active":"2023-10-16T10:05:11.398520+0000","last_became_peered":"2023-10-16T10:05:11.398520+0000","last_unstale":"2023-10-16T10:05:12.398217+0000","last_undegraded":"2023-10-16T10:05:12.398217+0000","last_fullsized":"2023-10-16T10:05:12.398217+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:38:13.502289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"94'1","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.386044+0000","last_change":"2023-10-16T10:05:10.370126+0000","last_active":"2023-10-16T10:05:11.386044+0000","last_peered":"2023-10-16T10:05:11.386044+0000","last_clean":"2023-10-16T10:05:11.386044+0000","last_became_active":"2023-10-16T10:05:10.368626+0000","last_became_peered":"2023-10-16T10:05:10.368626+0000","last_unstale":"2023-10-16T10:05:11.386044+0000","last_undegraded":"2023-10-16T10:05:11.386044+0000","last_fullsized":"2023-10-16T10:05:11.386044+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:20:33.370299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.198204+0000","last_change":"2023-10-16T10:05:11.413038+0000","last_active":"2023-10-16T10:05:13.198204+0000","last_peered":"2023-10-16T10:05:13.198204+0000","last_clean":"2023-10-16T10:05:13.198204+0000","last_became_active":"2023-10-16T10:05:11.412589+0000","last_became_peered":"2023-10-16T10:05:11.412589+0000","last_unstale":"2023-10-16T10:05:13.198204+0000","last_undegraded":"2023-10-16T10:05:13.198204+0000","last_fullsized":"2023-10-16T10:05:13.198204+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:32.785131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.411199+0000","last_change":"2023-10-16T10:05:10.384291+0000","last_active":"2023-10-16T10:05:11.411199+0000","last_peered":"2023-10-16T10:05:11.411199+0000","last_clean":"2023-10-16T10:05:11.411199+0000","last_became_active":"2023-10-16T10:05:10.382504+0000","last_became_peered":"2023-10-16T10:05:10.382504+0000","last_unstale":"2023-10-16T10:05:11.411199+0000","last_undegraded":"2023-10-16T10:05:11.411199+0000","last_fullsized":"2023-10-16T10:05:11.411199+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:07:26.048925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.323315+0000","last_change":"2023-10-16T10:05:11.416290+0000","last_active":"2023-10-16T10:05:14.323315+0000","last_peered":"2023-10-16T10:05:14.323315+0000","last_clean":"2023-10-16T10:05:14.323315+0000","last_became_active":"2023-10-16T10:05:11.415862+0000","last_became_peered":"2023-10-16T10:05:11.415862+0000","last_unstale":"2023-10-16T10:05:14.323315+0000","last_undegraded":"2023-10-16T10:05:14.323315+0000","last_fullsized":"2023-10-16T10:05:14.323315+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:19.013927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.416676+0000","last_change":"2023-10-16T10:05:10.375080+0000","last_active":"2023-10-16T10:05:11.416676+0000","last_peered":"2023-10-16T10:05:11.416676+0000","last_clean":"2023-10-16T10:05:11.416676+0000","last_became_active":"2023-10-16T10:05:10.374523+0000","last_became_peered":"2023-10-16T10:05:10.374523+0000","last_unstale":"2023-10-16T10:05:11.416676+0000","last_undegraded":"2023-10-16T10:05:11.416676+0000","last_fullsized":"2023-10-16T10:05:11.416676+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:11:52.684115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.083180+0000","last_change":"2023-10-16T10:05:11.401995+0000","last_active":"2023-10-16T10:05:13.083180+0000","last_peered":"2023-10-16T10:05:13.083180+0000","last_clean":"2023-10-16T10:05:13.083180+0000","last_became_active":"2023-10-16T10:05:11.401680+0000","last_became_peered":"2023-10-16T10:05:11.401680+0000","last_unstale":"2023-10-16T10:05:13.083180+0000","last_undegraded":"2023-10-16T10:05:13.083180+0000","last_fullsized":"2023-10-16T10:05:13.083180+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:10:13.772569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"94'1","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.391838+0000","last_change":"2023-10-16T10:05:10.373250+0000","last_active":"2023-10-16T10:05:11.391838+0000","last_peered":"2023-10-16T10:05:11.391838+0000","last_clean":"2023-10-16T10:05:11.391838+0000","last_became_active":"2023-10-16T10:05:10.372761+0000","last_became_peered":"2023-10-16T10:05:10.372761+0000","last_unstale":"2023-10-16T10:05:11.391838+0000","last_undegraded":"2023-10-16T10:05:11.391838+0000","last_fullsized":"2023-10-16T10:05:11.391838+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:28:09.602927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.400468+0000","last_change":"2023-10-16T10:05:11.418394+0000","last_active":"2023-10-16T10:05:12.400468+0000","last_peered":"2023-10-16T10:05:12.400468+0000","last_clean":"2023-10-16T10:05:12.400468+0000","last_became_active":"2023-10-16T10:05:11.418068+0000","last_became_peered":"2023-10-16T10:05:11.418068+0000","last_unstale":"2023-10-16T10:05:12.400468+0000","last_undegraded":"2023-10-16T10:05:12.400468+0000","last_fullsized":"2023-10-16T10:05:12.400468+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:35:46.196661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"94'1","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.416712+0000","last_change":"2023-10-16T10:05:10.377635+0000","last_active":"2023-10-16T10:05:11.416712+0000","last_peered":"2023-10-16T10:05:11.416712+0000","last_clean":"2023-10-16T10:05:11.416712+0000","last_became_active":"2023-10-16T10:05:10.377357+0000","last_became_peered":"2023-10-16T10:05:10.377357+0000","last_unstale":"2023-10-16T10:05:11.416712+0000","last_undegraded":"2023-10-16T10:05:11.416712+0000","last_fullsized":"2023-10-16T10:05:11.416712+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:18:35.706363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.397048+0000","last_change":"2023-10-16T10:05:11.391607+0000","last_active":"2023-10-16T10:05:12.397048+0000","last_peered":"2023-10-16T10:05:12.397048+0000","last_clean":"2023-10-16T10:05:12.397048+0000","last_became_active":"2023-10-16T10:05:11.391184+0000","last_became_peered":"2023-10-16T10:05:11.391184+0000","last_unstale":"2023-10-16T10:05:12.397048+0000","last_undegraded":"2023-10-16T10:05:12.397048+0000","last_fullsized":"2023-10-16T10:05:12.397048+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:35.765791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.379842+0000","last_change":"2023-10-16T10:05:10.372922+0000","last_active":"2023-10-16T10:05:11.379842+0000","last_peered":"2023-10-16T10:05:11.379842+0000","last_clean":"2023-10-16T10:05:11.379842+0000","last_became_active":"2023-10-16T10:05:10.372409+0000","last_became_peered":"2023-10-16T10:05:10.372409+0000","last_unstale":"2023-10-16T10:05:11.379842+0000","last_undegraded":"2023-10-16T10:05:11.379842+0000","last_fullsized":"2023-10-16T10:05:11.379842+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:38:12.485801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.399189+0000","last_change":"2023-10-16T10:05:11.407542+0000","last_active":"2023-10-16T10:05:12.399189+0000","last_peered":"2023-10-16T10:05:12.399189+0000","last_clean":"2023-10-16T10:05:12.399189+0000","last_became_active":"2023-10-16T10:05:11.407042+0000","last_became_peered":"2023-10-16T10:05:11.407042+0000","last_unstale":"2023-10-16T10:05:12.399189+0000","last_undegraded":"2023-10-16T10:05:12.399189+0000","last_fullsized":"2023-10-16T10:05:12.399189+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:22:26.235553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.372073+0000","last_change":"2023-10-16T10:05:10.368323+0000","last_active":"2023-10-16T10:05:11.372073+0000","last_peered":"2023-10-16T10:05:11.372073+0000","last_clean":"2023-10-16T10:05:11.372073+0000","last_became_active":"2023-10-16T10:05:10.367902+0000","last_became_peered":"2023-10-16T10:05:10.367902+0000","last_unstale":"2023-10-16T10:05:11.372073+0000","last_undegraded":"2023-10-16T10:05:11.372073+0000","last_fullsized":"2023-10-16T10:05:11.372073+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:06.896544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.083323+0000","last_change":"2023-10-16T10:05:11.416169+0000","last_active":"2023-10-16T10:05:13.083323+0000","last_peered":"2023-10-16T10:05:13.083323+0000","last_clean":"2023-10-16T10:05:13.083323+0000","last_became_active":"2023-10-16T10:05:11.415411+0000","last_became_peered":"2023-10-16T10:05:11.415411+0000","last_unstale":"2023-10-16T10:05:13.083323+0000","last_undegraded":"2023-10-16T10:05:13.083323+0000","last_fullsized":"2023-10-16T10:05:13.083323+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:17:13.781609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.395549+0000","last_change":"2023-10-16T10:05:10.374183+0000","last_active":"2023-10-16T10:05:11.395549+0000","last_peered":"2023-10-16T10:05:11.395549+0000","last_clean":"2023-10-16T10:05:11.395549+0000","last_became_active":"2023-10-16T10:05:10.373838+0000","last_became_peered":"2023-10-16T10:05:10.373838+0000","last_unstale":"2023-10-16T10:05:11.395549+0000","last_undegraded":"2023-10-16T10:05:11.395549+0000","last_fullsized":"2023-10-16T10:05:11.395549+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:41.390739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.323555+0000","last_change":"2023-10-16T10:05:11.417790+0000","last_active":"2023-10-16T10:05:14.323555+0000","last_peered":"2023-10-16T10:05:14.323555+0000","last_clean":"2023-10-16T10:05:14.323555+0000","last_became_active":"2023-10-16T10:05:11.417438+0000","last_became_peered":"2023-10-16T10:05:11.417438+0000","last_unstale":"2023-10-16T10:05:14.323555+0000","last_undegraded":"2023-10-16T10:05:14.323555+0000","last_fullsized":"2023-10-16T10:05:14.323555+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:55.298078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.384396+0000","last_change":"2023-10-16T10:05:10.379280+0000","last_active":"2023-10-16T10:05:11.384396+0000","last_peered":"2023-10-16T10:05:11.384396+0000","last_clean":"2023-10-16T10:05:11.384396+0000","last_became_active":"2023-10-16T10:05:10.379005+0000","last_became_peered":"2023-10-16T10:05:10.379005+0000","last_unstale":"2023-10-16T10:05:11.384396+0000","last_undegraded":"2023-10-16T10:05:11.384396+0000","last_fullsized":"2023-10-16T10:05:11.384396+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:25.051680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.198384+0000","last_change":"2023-10-16T10:05:11.403722+0000","last_active":"2023-10-16T10:05:13.198384+0000","last_peered":"2023-10-16T10:05:13.198384+0000","last_clean":"2023-10-16T10:05:13.198384+0000","last_became_active":"2023-10-16T10:05:11.403413+0000","last_became_peered":"2023-10-16T10:05:11.403413+0000","last_unstale":"2023-10-16T10:05:13.198384+0000","last_undegraded":"2023-10-16T10:05:13.198384+0000","last_fullsized":"2023-10-16T10:05:13.198384+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:05:03.062283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"94'3","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.404422+0000","last_change":"2023-10-16T10:05:10.375760+0000","last_active":"2023-10-16T10:05:11.404422+0000","last_peered":"2023-10-16T10:05:11.404422+0000","last_clean":"2023-10-16T10:05:11.404422+0000","last_became_active":"2023-10-16T10:05:10.375599+0000","last_became_peered":"2023-10-16T10:05:10.375599+0000","last_unstale":"2023-10-16T10:05:11.404422+0000","last_undegraded":"2023-10-16T10:05:11.404422+0000","last_fullsized":"2023-10-16T10:05:11.404422+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:24:29.169411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.397461+0000","last_change":"2023-10-16T10:05:11.392052+0000","last_active":"2023-10-16T10:05:12.397461+0000","last_peered":"2023-10-16T10:05:12.397461+0000","last_clean":"2023-10-16T10:05:12.397461+0000","last_became_active":"2023-10-16T10:05:11.391717+0000","last_became_peered":"2023-10-16T10:05:11.391717+0000","last_unstale":"2023-10-16T10:05:12.397461+0000","last_undegraded":"2023-10-16T10:05:12.397461+0000","last_fullsized":"2023-10-16T10:05:12.397461+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:45:56.717864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"94'6","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.403348+0000","last_change":"2023-10-16T10:05:10.363723+0000","last_active":"2023-10-16T10:05:11.403348+0000","last_peered":"2023-10-16T10:05:11.403348+0000","last_clean":"2023-10-16T10:05:11.403348+0000","last_became_active":"2023-10-16T10:05:10.363368+0000","last_became_peered":"2023-10-16T10:05:10.363368+0000","last_unstale":"2023-10-16T10:05:11.403348+0000","last_undegraded":"2023-10-16T10:05:11.403348+0000","last_fullsized":"2023-10-16T10:05:11.403348+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:21.263378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.322683+0000","last_change":"2023-10-16T10:05:11.411845+0000","last_active":"2023-10-16T10:05:14.322683+0000","last_peered":"2023-10-16T10:05:14.322683+0000","last_clean":"2023-10-16T10:05:14.322683+0000","last_became_active":"2023-10-16T10:05:11.411563+0000","last_became_peered":"2023-10-16T10:05:11.411563+0000","last_unstale":"2023-10-16T10:05:14.322683+0000","last_undegraded":"2023-10-16T10:05:14.322683+0000","last_fullsized":"2023-10-16T10:05:14.322683+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:24:30.185899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"94'2","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.416937+0000","last_change":"2023-10-16T10:05:10.378284+0000","last_active":"2023-10-16T10:05:11.416937+0000","last_peered":"2023-10-16T10:05:11.416937+0000","last_clean":"2023-10-16T10:05:11.416937+0000","last_became_active":"2023-10-16T10:05:10.378024+0000","last_became_peered":"2023-10-16T10:05:10.378024+0000","last_unstale":"2023-10-16T10:05:11.416937+0000","last_undegraded":"2023-10-16T10:05:11.416937+0000","last_fullsized":"2023-10-16T10:05:11.416937+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:39:22.865400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.082775+0000","last_change":"2023-10-16T10:05:11.398771+0000","last_active":"2023-10-16T10:05:13.082775+0000","last_peered":"2023-10-16T10:05:13.082775+0000","last_clean":"2023-10-16T10:05:13.082775+0000","last_became_active":"2023-10-16T10:05:11.398025+0000","last_became_peered":"2023-10-16T10:05:11.398025+0000","last_unstale":"2023-10-16T10:05:13.082775+0000","last_undegraded":"2023-10-16T10:05:13.082775+0000","last_fullsized":"2023-10-16T10:05:13.082775+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:46.619715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.383670+0000","last_change":"2023-10-16T10:05:10.368901+0000","last_active":"2023-10-16T10:05:11.383670+0000","last_peered":"2023-10-16T10:05:11.383670+0000","last_clean":"2023-10-16T10:05:11.383670+0000","last_became_active":"2023-10-16T10:05:10.368631+0000","last_became_peered":"2023-10-16T10:05:10.368631+0000","last_unstale":"2023-10-16T10:05:11.383670+0000","last_undegraded":"2023-10-16T10:05:11.383670+0000","last_fullsized":"2023-10-16T10:05:11.383670+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:37.991581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.398725+0000","last_change":"2023-10-16T10:05:11.395702+0000","last_active":"2023-10-16T10:05:12.398725+0000","last_peered":"2023-10-16T10:05:12.398725+0000","last_clean":"2023-10-16T10:05:12.398725+0000","last_became_active":"2023-10-16T10:05:11.395057+0000","last_became_peered":"2023-10-16T10:05:11.395057+0000","last_unstale":"2023-10-16T10:05:12.398725+0000","last_undegraded":"2023-10-16T10:05:12.398725+0000","last_fullsized":"2023-10-16T10:05:12.398725+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:57.580285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.399968+0000","last_change":"2023-10-16T10:05:10.372553+0000","last_active":"2023-10-16T10:05:11.399968+0000","last_peered":"2023-10-16T10:05:11.399968+0000","last_clean":"2023-10-16T10:05:11.399968+0000","last_became_active":"2023-10-16T10:05:10.372272+0000","last_became_peered":"2023-10-16T10:05:10.372272+0000","last_unstale":"2023-10-16T10:05:11.399968+0000","last_undegraded":"2023-10-16T10:05:11.399968+0000","last_fullsized":"2023-10-16T10:05:11.399968+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:39:27.567450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.397958+0000","last_change":"2023-10-16T10:05:11.398182+0000","last_active":"2023-10-16T10:05:12.397958+0000","last_peered":"2023-10-16T10:05:12.397958+0000","last_clean":"2023-10-16T10:05:12.397958+0000","last_became_active":"2023-10-16T10:05:11.397897+0000","last_became_peered":"2023-10-16T10:05:11.397897+0000","last_unstale":"2023-10-16T10:05:12.397958+0000","last_undegraded":"2023-10-16T10:05:12.397958+0000","last_fullsized":"2023-10-16T10:05:12.397958+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:58.218659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"94'8","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.419352+0000","last_change":"2023-10-16T10:05:10.384020+0000","last_active":"2023-10-16T10:05:11.419352+0000","last_peered":"2023-10-16T10:05:11.419352+0000","last_clean":"2023-10-16T10:05:11.419352+0000","last_became_active":"2023-10-16T10:05:10.383325+0000","last_became_peered":"2023-10-16T10:05:10.383325+0000","last_unstale":"2023-10-16T10:05:11.419352+0000","last_undegraded":"2023-10-16T10:05:11.419352+0000","last_fullsized":"2023-10-16T10:05:11.419352+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:57.297999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.401422+0000","last_change":"2023-10-16T10:05:11.394591+0000","last_active":"2023-10-16T10:05:12.401422+0000","last_peered":"2023-10-16T10:05:12.401422+0000","last_clean":"2023-10-16T10:05:12.401422+0000","last_became_active":"2023-10-16T10:05:11.393980+0000","last_became_peered":"2023-10-16T10:05:11.393980+0000","last_unstale":"2023-10-16T10:05:12.401422+0000","last_undegraded":"2023-10-16T10:05:12.401422+0000","last_fullsized":"2023-10-16T10:05:12.401422+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:16.786433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"94'1","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.416543+0000","last_change":"2023-10-16T10:05:10.371329+0000","last_active":"2023-10-16T10:05:11.416543+0000","last_peered":"2023-10-16T10:05:11.416543+0000","last_clean":"2023-10-16T10:05:11.416543+0000","last_became_active":"2023-10-16T10:05:10.370944+0000","last_became_peered":"2023-10-16T10:05:10.370944+0000","last_unstale":"2023-10-16T10:05:11.416543+0000","last_undegraded":"2023-10-16T10:05:11.416543+0000","last_fullsized":"2023-10-16T10:05:11.416543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:56:20.497615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.323681+0000","last_change":"2023-10-16T10:05:11.414462+0000","last_active":"2023-10-16T10:05:14.323681+0000","last_peered":"2023-10-16T10:05:14.323681+0000","last_clean":"2023-10-16T10:05:14.323681+0000","last_became_active":"2023-10-16T10:05:11.413836+0000","last_became_peered":"2023-10-16T10:05:11.413836+0000","last_unstale":"2023-10-16T10:05:14.323681+0000","last_undegraded":"2023-10-16T10:05:14.323681+0000","last_fullsized":"2023-10-16T10:05:14.323681+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:19:31.171652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.415487+0000","last_change":"2023-10-16T10:05:10.384943+0000","last_active":"2023-10-16T10:05:11.415487+0000","last_peered":"2023-10-16T10:05:11.415487+0000","last_clean":"2023-10-16T10:05:11.415487+0000","last_became_active":"2023-10-16T10:05:10.382395+0000","last_became_peered":"2023-10-16T10:05:10.382395+0000","last_unstale":"2023-10-16T10:05:11.415487+0000","last_undegraded":"2023-10-16T10:05:11.415487+0000","last_fullsized":"2023-10-16T10:05:11.415487+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:10:02.377817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.399312+0000","last_change":"2023-10-16T10:05:11.393326+0000","last_active":"2023-10-16T10:05:11.399312+0000","last_peered":"2023-10-16T10:05:11.399312+0000","last_clean":"2023-10-16T10:05:11.399312+0000","last_became_active":"2023-10-16T10:05:11.392871+0000","last_became_peered":"2023-10-16T10:05:11.392871+0000","last_unstale":"2023-10-16T10:05:11.399312+0000","last_undegraded":"2023-10-16T10:05:11.399312+0000","last_fullsized":"2023-10-16T10:05:11.399312+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:57:19.149543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"94'1","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.416579+0000","last_change":"2023-10-16T10:05:10.380465+0000","last_active":"2023-10-16T10:05:11.416579+0000","last_peered":"2023-10-16T10:05:11.416579+0000","last_clean":"2023-10-16T10:05:11.416579+0000","last_became_active":"2023-10-16T10:05:10.380134+0000","last_became_peered":"2023-10-16T10:05:10.380134+0000","last_unstale":"2023-10-16T10:05:11.416579+0000","last_undegraded":"2023-10-16T10:05:11.416579+0000","last_fullsized":"2023-10-16T10:05:11.416579+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:22:28.852480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.397541+0000","last_change":"2023-10-16T10:05:11.391498+0000","last_active":"2023-10-16T10:05:12.397541+0000","last_peered":"2023-10-16T10:05:12.397541+0000","last_clean":"2023-10-16T10:05:12.397541+0000","last_became_active":"2023-10-16T10:05:11.391176+0000","last_became_peered":"2023-10-16T10:05:11.391176+0000","last_unstale":"2023-10-16T10:05:12.397541+0000","last_undegraded":"2023-10-16T10:05:12.397541+0000","last_fullsized":"2023-10-16T10:05:12.397541+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:57.685307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.406557+0000","last_change":"2023-10-16T10:05:10.384125+0000","last_active":"2023-10-16T10:05:11.406557+0000","last_peered":"2023-10-16T10:05:11.406557+0000","last_clean":"2023-10-16T10:05:11.406557+0000","last_became_active":"2023-10-16T10:05:10.383816+0000","last_became_peered":"2023-10-16T10:05:10.383816+0000","last_unstale":"2023-10-16T10:05:11.406557+0000","last_undegraded":"2023-10-16T10:05:11.406557+0000","last_fullsized":"2023-10-16T10:05:11.406557+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:43:09.223869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.398319+0000","last_change":"2023-10-16T10:05:11.394052+0000","last_active":"2023-10-16T10:05:11.398319+0000","last_peered":"2023-10-16T10:05:11.398319+0000","last_clean":"2023-10-16T10:05:11.398319+0000","last_became_active":"2023-10-16T10:05:11.393768+0000","last_became_peered":"2023-10-16T10:05:11.393768+0000","last_unstale":"2023-10-16T10:05:11.398319+0000","last_undegraded":"2023-10-16T10:05:11.398319+0000","last_fullsized":"2023-10-16T10:05:11.398319+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:24:13.019540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.383904+0000","last_change":"2023-10-16T10:05:10.375863+0000","last_active":"2023-10-16T10:05:11.383904+0000","last_peered":"2023-10-16T10:05:11.383904+0000","last_clean":"2023-10-16T10:05:11.383904+0000","last_became_active":"2023-10-16T10:05:10.372876+0000","last_became_peered":"2023-10-16T10:05:10.372876+0000","last_unstale":"2023-10-16T10:05:11.383904+0000","last_undegraded":"2023-10-16T10:05:11.383904+0000","last_fullsized":"2023-10-16T10:05:11.383904+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:05.484396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.400533+0000","last_change":"2023-10-16T10:05:11.399178+0000","last_active":"2023-10-16T10:05:11.400533+0000","last_peered":"2023-10-16T10:05:11.400533+0000","last_clean":"2023-10-16T10:05:11.400533+0000","last_became_active":"2023-10-16T10:05:11.398628+0000","last_became_peered":"2023-10-16T10:05:11.398628+0000","last_unstale":"2023-10-16T10:05:11.400533+0000","last_undegraded":"2023-10-16T10:05:11.400533+0000","last_fullsized":"2023-10-16T10:05:11.400533+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:03.454542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.394321+0000","last_change":"2023-10-16T10:05:10.374041+0000","last_active":"2023-10-16T10:05:11.394321+0000","last_peered":"2023-10-16T10:05:11.394321+0000","last_clean":"2023-10-16T10:05:11.394321+0000","last_became_active":"2023-10-16T10:05:10.373753+0000","last_became_peered":"2023-10-16T10:05:10.373753+0000","last_unstale":"2023-10-16T10:05:11.394321+0000","last_undegraded":"2023-10-16T10:05:11.394321+0000","last_fullsized":"2023-10-16T10:05:11.394321+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:30:42.712451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.408815+0000","last_change":"2023-10-16T10:05:11.407798+0000","last_active":"2023-10-16T10:05:11.408815+0000","last_peered":"2023-10-16T10:05:11.408815+0000","last_clean":"2023-10-16T10:05:11.408815+0000","last_became_active":"2023-10-16T10:05:11.407406+0000","last_became_peered":"2023-10-16T10:05:11.407406+0000","last_unstale":"2023-10-16T10:05:11.408815+0000","last_undegraded":"2023-10-16T10:05:11.408815+0000","last_fullsized":"2023-10-16T10:05:11.408815+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:27:29.884109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.382740+0000","last_change":"2023-10-16T10:05:10.369914+0000","last_active":"2023-10-16T10:05:11.382740+0000","last_peered":"2023-10-16T10:05:11.382740+0000","last_clean":"2023-10-16T10:05:11.382740+0000","last_became_active":"2023-10-16T10:05:10.369624+0000","last_became_peered":"2023-10-16T10:05:10.369624+0000","last_unstale":"2023-10-16T10:05:11.382740+0000","last_undegraded":"2023-10-16T10:05:11.382740+0000","last_fullsized":"2023-10-16T10:05:11.382740+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:54:36.927537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.399429+0000","last_change":"2023-10-16T10:05:11.386939+0000","last_active":"2023-10-16T10:05:12.399429+0000","last_peered":"2023-10-16T10:05:12.399429+0000","last_clean":"2023-10-16T10:05:12.399429+0000","last_became_active":"2023-10-16T10:05:11.386672+0000","last_became_peered":"2023-10-16T10:05:11.386672+0000","last_unstale":"2023-10-16T10:05:12.399429+0000","last_undegraded":"2023-10-16T10:05:12.399429+0000","last_fullsized":"2023-10-16T10:05:12.399429+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:33:51.440378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.416627+0000","last_change":"2023-10-16T10:05:10.373636+0000","last_active":"2023-10-16T10:05:11.416627+0000","last_peered":"2023-10-16T10:05:11.416627+0000","last_clean":"2023-10-16T10:05:11.416627+0000","last_became_active":"2023-10-16T10:05:10.373364+0000","last_became_peered":"2023-10-16T10:05:10.373364+0000","last_unstale":"2023-10-16T10:05:11.416627+0000","last_undegraded":"2023-10-16T10:05:11.416627+0000","last_fullsized":"2023-10-16T10:05:11.416627+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:21:39.317846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.083423+0000","last_change":"2023-10-16T10:05:11.407988+0000","last_active":"2023-10-16T10:05:13.083423+0000","last_peered":"2023-10-16T10:05:13.083423+0000","last_clean":"2023-10-16T10:05:13.083423+0000","last_became_active":"2023-10-16T10:05:11.406715+0000","last_became_peered":"2023-10-16T10:05:11.406715+0000","last_unstale":"2023-10-16T10:05:13.083423+0000","last_undegraded":"2023-10-16T10:05:13.083423+0000","last_fullsized":"2023-10-16T10:05:13.083423+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:59:03.669248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.399291+0000","last_change":"2023-10-16T10:05:10.372440+0000","last_active":"2023-10-16T10:05:11.399291+0000","last_peered":"2023-10-16T10:05:11.399291+0000","last_clean":"2023-10-16T10:05:11.399291+0000","last_became_active":"2023-10-16T10:05:10.372047+0000","last_became_peered":"2023-10-16T10:05:10.372047+0000","last_unstale":"2023-10-16T10:05:11.399291+0000","last_undegraded":"2023-10-16T10:05:11.399291+0000","last_fullsized":"2023-10-16T10:05:11.399291+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:51:21.096892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.323679+0000","last_change":"2023-10-16T10:05:11.418525+0000","last_active":"2023-10-16T10:05:14.323679+0000","last_peered":"2023-10-16T10:05:14.323679+0000","last_clean":"2023-10-16T10:05:14.323679+0000","last_became_active":"2023-10-16T10:05:11.418237+0000","last_became_peered":"2023-10-16T10:05:11.418237+0000","last_unstale":"2023-10-16T10:05:14.323679+0000","last_undegraded":"2023-10-16T10:05:14.323679+0000","last_fullsized":"2023-10-16T10:05:14.323679+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:51.860331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.374447+0000","last_change":"2023-10-16T10:05:10.369414+0000","last_active":"2023-10-16T10:05:11.374447+0000","last_peered":"2023-10-16T10:05:11.374447+0000","last_clean":"2023-10-16T10:05:11.374447+0000","last_became_active":"2023-10-16T10:05:10.369199+0000","last_became_peered":"2023-10-16T10:05:10.369199+0000","last_unstale":"2023-10-16T10:05:11.374447+0000","last_undegraded":"2023-10-16T10:05:11.374447+0000","last_fullsized":"2023-10-16T10:05:11.374447+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:56.563797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.323473+0000","last_change":"2023-10-16T10:05:11.418473+0000","last_active":"2023-10-16T10:05:14.323473+0000","last_peered":"2023-10-16T10:05:14.323473+0000","last_clean":"2023-10-16T10:05:14.323473+0000","last_became_active":"2023-10-16T10:05:11.418166+0000","last_became_peered":"2023-10-16T10:05:11.418166+0000","last_unstale":"2023-10-16T10:05:14.323473+0000","last_undegraded":"2023-10-16T10:05:14.323473+0000","last_fullsized":"2023-10-16T10:05:14.323473+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:30:16.921858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.381389+0000","last_change":"2023-10-16T10:05:10.384003+0000","last_active":"2023-10-16T10:05:11.381389+0000","last_peered":"2023-10-16T10:05:11.381389+0000","last_clean":"2023-10-16T10:05:11.381389+0000","last_became_active":"2023-10-16T10:05:10.383804+0000","last_became_peered":"2023-10-16T10:05:10.383804+0000","last_unstale":"2023-10-16T10:05:11.381389+0000","last_undegraded":"2023-10-16T10:05:11.381389+0000","last_fullsized":"2023-10-16T10:05:11.381389+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:31.768643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.397327+0000","last_change":"2023-10-16T10:05:11.396702+0000","last_active":"2023-10-16T10:05:12.397327+0000","last_peered":"2023-10-16T10:05:12.397327+0000","last_clean":"2023-10-16T10:05:12.397327+0000","last_became_active":"2023-10-16T10:05:11.396343+0000","last_became_peered":"2023-10-16T10:05:11.396343+0000","last_unstale":"2023-10-16T10:05:12.397327+0000","last_undegraded":"2023-10-16T10:05:12.397327+0000","last_fullsized":"2023-10-16T10:05:12.397327+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:33:51.440378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.374190+0000","last_change":"2023-10-16T10:05:10.376744+0000","last_active":"2023-10-16T10:05:11.374190+0000","last_peered":"2023-10-16T10:05:11.374190+0000","last_clean":"2023-10-16T10:05:11.374190+0000","last_became_active":"2023-10-16T10:05:10.376146+0000","last_became_peered":"2023-10-16T10:05:10.376146+0000","last_unstale":"2023-10-16T10:05:11.374190+0000","last_undegraded":"2023-10-16T10:05:11.374190+0000","last_fullsized":"2023-10-16T10:05:11.374190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:54:36.927537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.197882+0000","last_change":"2023-10-16T10:05:11.416337+0000","last_active":"2023-10-16T10:05:13.197882+0000","last_peered":"2023-10-16T10:05:13.197882+0000","last_clean":"2023-10-16T10:05:13.197882+0000","last_became_active":"2023-10-16T10:05:11.415777+0000","last_became_peered":"2023-10-16T10:05:11.415777+0000","last_unstale":"2023-10-16T10:05:13.197882+0000","last_undegraded":"2023-10-16T10:05:13.197882+0000","last_fullsized":"2023-10-16T10:05:13.197882+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:07.913032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.380283+0000","last_change":"2023-10-16T10:05:10.376543+0000","last_active":"2023-10-16T10:05:11.380283+0000","last_peered":"2023-10-16T10:05:11.380283+0000","last_clean":"2023-10-16T10:05:11.380283+0000","last_became_active":"2023-10-16T10:05:10.376224+0000","last_became_peered":"2023-10-16T10:05:10.376224+0000","last_unstale":"2023-10-16T10:05:11.380283+0000","last_undegraded":"2023-10-16T10:05:11.380283+0000","last_fullsized":"2023-10-16T10:05:11.380283+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:08:26.700080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.401740+0000","last_change":"2023-10-16T10:05:11.398956+0000","last_active":"2023-10-16T10:05:11.401740+0000","last_peered":"2023-10-16T10:05:11.401740+0000","last_clean":"2023-10-16T10:05:11.401740+0000","last_became_active":"2023-10-16T10:05:11.398700+0000","last_became_peered":"2023-10-16T10:05:11.398700+0000","last_unstale":"2023-10-16T10:05:11.401740+0000","last_undegraded":"2023-10-16T10:05:11.401740+0000","last_fullsized":"2023-10-16T10:05:11.401740+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:41.583697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.378203+0000","last_change":"2023-10-16T10:05:10.384152+0000","last_active":"2023-10-16T10:05:11.378203+0000","last_peered":"2023-10-16T10:05:11.378203+0000","last_clean":"2023-10-16T10:05:11.378203+0000","last_became_active":"2023-10-16T10:05:10.383688+0000","last_became_peered":"2023-10-16T10:05:10.383688+0000","last_unstale":"2023-10-16T10:05:11.378203+0000","last_undegraded":"2023-10-16T10:05:11.378203+0000","last_fullsized":"2023-10-16T10:05:11.378203+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:46:13.198009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.198248+0000","last_change":"2023-10-16T10:05:11.404146+0000","last_active":"2023-10-16T10:05:13.198248+0000","last_peered":"2023-10-16T10:05:13.198248+0000","last_clean":"2023-10-16T10:05:13.198248+0000","last_became_active":"2023-10-16T10:05:11.403805+0000","last_became_peered":"2023-10-16T10:05:11.403805+0000","last_unstale":"2023-10-16T10:05:13.198248+0000","last_undegraded":"2023-10-16T10:05:13.198248+0000","last_fullsized":"2023-10-16T10:05:13.198248+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:54:37.944025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.382300+0000","last_change":"2023-10-16T10:05:10.374754+0000","last_active":"2023-10-16T10:05:11.382300+0000","last_peered":"2023-10-16T10:05:11.382300+0000","last_clean":"2023-10-16T10:05:11.382300+0000","last_became_active":"2023-10-16T10:05:10.374447+0000","last_became_peered":"2023-10-16T10:05:10.374447+0000","last_unstale":"2023-10-16T10:05:11.382300+0000","last_undegraded":"2023-10-16T10:05:11.382300+0000","last_fullsized":"2023-10-16T10:05:11.382300+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:32:26.229914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.400829+0000","last_change":"2023-10-16T10:05:11.398995+0000","last_active":"2023-10-16T10:05:11.400829+0000","last_peered":"2023-10-16T10:05:11.400829+0000","last_clean":"2023-10-16T10:05:11.400829+0000","last_became_active":"2023-10-16T10:05:11.398696+0000","last_became_peered":"2023-10-16T10:05:11.398696+0000","last_unstale":"2023-10-16T10:05:11.400829+0000","last_undegraded":"2023-10-16T10:05:11.400829+0000","last_fullsized":"2023-10-16T10:05:11.400829+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:43:10.240357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.416536+0000","last_change":"2023-10-16T10:05:10.366612+0000","last_active":"2023-10-16T10:05:11.416536+0000","last_peered":"2023-10-16T10:05:11.416536+0000","last_clean":"2023-10-16T10:05:11.416536+0000","last_became_active":"2023-10-16T10:05:10.366310+0000","last_became_peered":"2023-10-16T10:05:10.366310+0000","last_unstale":"2023-10-16T10:05:11.416536+0000","last_undegraded":"2023-10-16T10:05:11.416536+0000","last_fullsized":"2023-10-16T10:05:11.416536+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:06.570078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"84'173","reported_seq":436,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.400553+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T10:05:11.400553+0000","last_peered":"2023-10-16T10:05:11.400553+0000","last_clean":"2023-10-16T10:05:11.400553+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T10:05:11.400553+0000","last_undegraded":"2023-10-16T10:05:11.400553+0000","last_fullsized":"2023-10-16T10:05:11.400553+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":600,"num_read_kb":1133,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":600,"num_read_kb":1133,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":173,"ondisk_log_size":173,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575043,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37056,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334784,"statfs":{"total":96636764160,"available":96598818816,"internally_reserved":0,"allocated":1748992,"data_stored":1136691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1639999999999999,"15min":1.167},"min":{"1min":0.65000000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.6599999999999999,"5min":1.853,"15min":1.998},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1719999999999999,"15min":1.208},"min":{"1min":0.58099999999999996,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.5629999999999999,"5min":1.75,"15min":3.694},"last":0.59199999999999997}]},{"osd":1,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.222,"15min":1.2290000000000001},"min":{"1min":0.63300000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.579,"5min":1.923,"15min":1.923},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.147,"5min":1.234,"15min":1.22},"min":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5309999999999999,"5min":1.704,"15min":1.7410000000000001},"last":0.749}]},{"osd":2,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2,"15min":1.2490000000000001},"min":{"1min":0.89000000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5820000000000001,"5min":1.837,"15min":1.885},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.284,"15min":1.306},"min":{"1min":0.81499999999999995,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.7430000000000001,"5min":1.8620000000000001,"15min":3.7759999999999998},"last":0.91300000000000003}]},{"osd":3,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.3120000000000001,"15min":1.3380000000000001},"min":{"1min":0.93500000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5389999999999999,"5min":1.996,"15min":1.996},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.3600000000000001,"15min":1.3859999999999999},"min":{"1min":1.0509999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6659999999999999,"5min":1.8759999999999999,"15min":2.141},"last":1.347}]},{"osd":4,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.367,"15min":1.3759999999999999},"min":{"1min":1.004,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7829999999999999,"5min":2.028,"15min":2.0310000000000001},"last":1.419},{"interface":"front","average":{"1min":1.274,"5min":1.3120000000000001,"15min":1.3200000000000001},"min":{"1min":0.79200000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.55,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.98899999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4690000000000001,"15min":1.5209999999999999},"min":{"1min":1.111,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.77,"5min":2.1320000000000001,"15min":2.4510000000000001},"last":1.478},{"interface":"front","average":{"1min":1.403,"5min":1.4379999999999999,"15min":1.4410000000000001},"min":{"1min":0.92200000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6899999999999999,"5min":2.2170000000000001,"15min":2.3210000000000002},"last":1.1080000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.413,"15min":1.4159999999999999},"min":{"1min":1.054,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.659,"5min":1.946,"15min":2.6259999999999999},"last":1.175},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.6040000000000001,"15min":1.623},"min":{"1min":1.21,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.986,"5min":2.3799999999999999,"15min":2.5670000000000002},"last":1.544}]}]},{"osd":6,"up_from":13,"seq":55834575043,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335360,"statfs":{"total":96636764160,"available":96599408640,"internally_reserved":0,"allocated":1167360,"data_stored":552099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.127,"15min":1.147},"min":{"1min":0.75900000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4790000000000001,"5min":1.6910000000000001,"15min":2.04},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.236,"15min":1.246},"min":{"1min":0.98599999999999999,"5min":0.65900000000000003,"15min":0.624},"max":{"1min":1.641,"5min":1.847,"15min":2.1000000000000001},"last":1.175}]},{"osd":1,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1240000000000001,"15min":1.1779999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.9259999999999999},"last":1.351},{"interface":"front","average":{"1min":1.292,"5min":1.274,"15min":1.3009999999999999},"min":{"1min":0.91900000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.288}]},{"osd":2,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.169,"15min":1.2010000000000001},"min":{"1min":0.745,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":2.036},"last":1.851},{"interface":"front","average":{"1min":1.238,"5min":1.288,"15min":1.3200000000000001},"min":{"1min":0.79900000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.925,"5min":1.925,"15min":2.1720000000000002},"last":0.90300000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.228,"15min":1.2390000000000001},"min":{"1min":0.69599999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.742,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1859999999999999,"15min":1.234},"min":{"1min":0.83399999999999996,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6020000000000001,"5min":1.6359999999999999,"15min":2.0640000000000001},"last":1.637}]},{"osd":4,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.196,"15min":1.2849999999999999},"min":{"1min":0.77800000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.794,"5min":2.1629999999999998,"15min":2.4540000000000002},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4299999999999999,"15min":1.45},"min":{"1min":0.77700000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.901,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.5860000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.276,"15min":1.323},"min":{"1min":0.80400000000000005,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.603,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.956},{"interface":"front","average":{"1min":1.544,"5min":1.482,"15min":1.5229999999999999},"min":{"1min":1.3089999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.865,"5min":2.3849999999999998,"15min":2.5289999999999999},"last":1.7350000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.456,"15min":1.4890000000000001},"min":{"1min":0.871,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.843,"5min":3.5739999999999998,"15min":3.5739999999999998},"last":1.79},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.431,"15min":1.5},"min":{"1min":0.92600000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8340000000000001,"5min":2.4249999999999998,"15min":3.5419999999999998},"last":2.0510000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575043,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36476,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335364,"statfs":{"total":96636764160,"available":96599412736,"internally_reserved":0,"allocated":1163264,"data_stored":552097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.1399999999999999,"15min":1.214},"min":{"1min":0.63100000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.486,"5min":1.675,"15min":2.1019999999999999},"last":1.663},{"interface":"front","average":{"1min":1.272,"5min":1.2789999999999999,"15min":1.3029999999999999},"min":{"1min":0.72399999999999998,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.724,"5min":1.8600000000000001,"15min":1.915},"last":1.4690000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.135,"15min":1.149},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.629,"5min":1.7729999999999999,"15min":2.4569999999999999},"last":1.026},{"interface":"front","average":{"1min":1.179,"5min":1.1279999999999999,"15min":1.1559999999999999},"min":{"1min":0.84299999999999997,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.526,"5min":1.6970000000000001,"15min":2.2410000000000001},"last":1.363}]},{"osd":2,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.157,"15min":1.163},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71299999999999997},"max":{"1min":1.675,"5min":1.7470000000000001,"15min":1.77},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.228,"15min":1.246},"min":{"1min":0.83499999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.5780000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.5569999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.288,"15min":1.2869999999999999},"min":{"1min":0.81699999999999995,"5min":0.76000000000000001,"15min":0.73799999999999999},"max":{"1min":1.8839999999999999,"5min":1.97,"15min":1.97},"last":1.129},{"interface":"front","average":{"1min":1.298,"5min":1.355,"15min":1.3660000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.623,"5min":1.7989999999999999,"15min":2.1160000000000001},"last":1.3129999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.2529999999999999,"15min":1.2949999999999999},"min":{"1min":0.79700000000000004,"5min":0.77100000000000002,"15min":0.52100000000000002},"max":{"1min":1.569,"5min":1.7989999999999999,"15min":2.355},"last":1.196},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4990000000000001,"15min":1.5229999999999999},"min":{"1min":0.91700000000000004,"5min":0.89500000000000002,"15min":0.63600000000000001},"max":{"1min":1.8069999999999999,"5min":2.0920000000000001,"15min":2.4079999999999999},"last":1.804}]},{"osd":6,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.3740000000000001,"15min":1.379},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.55500000000000005},"max":{"1min":1.726,"5min":1.847,"15min":2.3580000000000001},"last":1.417},{"interface":"front","average":{"1min":1.375,"5min":1.3839999999999999,"15min":1.4099999999999999},"min":{"1min":1.151,"5min":0.83299999999999996,"15min":0.58399999999999996},"max":{"1min":1.6120000000000001,"5min":1.889,"15min":2.4159999999999999},"last":1.0680000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.478,"15min":1.5109999999999999},"min":{"1min":1.0669999999999999,"5min":0.88600000000000001,"15min":0.78700000000000003},"max":{"1min":1.913,"5min":2.1989999999999998,"15min":2.4809999999999999},"last":1.714},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.575,"15min":1.5960000000000001},"min":{"1min":1.2509999999999999,"5min":0.99199999999999999,"15min":0.75600000000000001},"max":{"1min":1.851,"5min":1.9430000000000001,"15min":2.931},"last":1.8520000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575044,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46208,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325632,"statfs":{"total":96636764160,"available":96589447168,"internally_reserved":0,"allocated":1163264,"data_stored":552077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.107,"15min":1.163},"min":{"1min":0.74199999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.554,"5min":1.76,"15min":4.2640000000000002},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.3460000000000001,"15min":1.252},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.756,"5min":11.974,"15min":11.974},"last":1.6120000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.3580000000000001,"15min":1.2809999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.7549999999999999,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.2130000000000001,"15min":1.24},"min":{"1min":0.67700000000000005,"5min":0.63600000000000001,"15min":0.624},"max":{"1min":1.5309999999999999,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.3740000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.254,"15min":1.2490000000000001},"min":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.75,"5min":1.756,"15min":2.3130000000000002},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.377,"15min":1.3799999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.615,"5min":5.0549999999999997,"15min":5.0549999999999997},"last":1.2390000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3160000000000001,"15min":1.3129999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.6739999999999999,"5min":6.8079999999999998,"15min":6.8079999999999998},"last":1.165},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.3759999999999999,"15min":1.395},"min":{"1min":0.93100000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.885,"5min":1.923,"15min":3.3050000000000002},"last":1.1020000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.498,"15min":1.423},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9590000000000001,"5min":8.702,"15min":8.702},"last":1.536},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.69099999999999995},"max":{"1min":1.754,"5min":3.0430000000000001,"15min":3.669},"last":1.2889999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.4810000000000001,"15min":1.4299999999999999},"min":{"1min":1.0049999999999999,"5min":0.80700000000000005,"15min":0.56599999999999995},"max":{"1min":1.9850000000000001,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.534,"5min":1.605,"15min":1.5509999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.70299999999999996},"max":{"1min":2.1179999999999999,"5min":6.4089999999999998,"15min":6.4089999999999998},"last":1.667}]},{"osd":7,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.508,"15min":1.5029999999999999},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.60999999999999999},"max":{"1min":2.177,"5min":2.9399999999999999,"15min":3.5150000000000001},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.631,"5min":1.585,"15min":1.5860000000000001},"min":{"1min":1.1220000000000001,"5min":0.98399999999999999,"15min":0.89600000000000002},"max":{"1min":2.0539999999999998,"5min":2.6120000000000001,"15min":3.9849999999999999},"last":1.3340000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575043,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36476,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335364,"statfs":{"total":96636764160,"available":96599412736,"internally_reserved":0,"allocated":1155072,"data_stored":545819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.83099999999999996,"15min":0.85999999999999999},"min":{"1min":0.60799999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4339999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.95599999999999996,"15min":1.004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.29999999999999999},"max":{"1min":1.385,"5min":1.708,"15min":1.708},"last":0.66600000000000004}]},{"osd":1,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.91200000000000003,"15min":0.94299999999999995},"min":{"1min":0.42799999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.2210000000000001,"5min":1.49,"15min":1.7150000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.006,"15min":1.0509999999999999},"min":{"1min":0.52900000000000003,"5min":0.42099999999999999,"15min":0.39600000000000002},"max":{"1min":1.4079999999999999,"5min":1.603,"15min":1.643},"last":0.81499999999999995}]},{"osd":2,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":1.0149999999999999,"15min":1.0580000000000001},"min":{"1min":0.46500000000000002,"5min":0.44800000000000001,"15min":0.439},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.8640000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.141,"15min":1.163},"min":{"1min":0.55100000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.534,"5min":1.776,"15min":2.028},"last":0.90300000000000002}]},{"osd":4,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.24,"15min":1.276},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.5740000000000001,"5min":1.8069999999999999,"15min":2.1619999999999999},"last":1.202},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.296,"15min":1.3220000000000001},"min":{"1min":0.75600000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.6779999999999999,"5min":1.8500000000000001,"15min":2.532},"last":1.29}]},{"osd":5,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.278,"15min":1.3169999999999999},"min":{"1min":0.77800000000000002,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.5629999999999999,"5min":1.7350000000000001,"15min":2.0019999999999998},"last":1.542},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3029999999999999,"15min":1.3460000000000001},"min":{"1min":0.91200000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.5589999999999999,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":1.319}]},{"osd":6,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.375,"15min":1.429},"min":{"1min":0.72099999999999997,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6799999999999999,"5min":1.891,"15min":2.3029999999999999},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3999999999999999,"15min":1.4410000000000001},"min":{"1min":0.88300000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.619,"5min":1.976,"15min":2.1240000000000001},"last":1.373}]},{"osd":7,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3959999999999999,"15min":1.4490000000000001},"min":{"1min":0.73199999999999998,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7450000000000001,"5min":2.032,"15min":2.403},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.504,"15min":1.5640000000000001},"min":{"1min":0.68600000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8320000000000001,"5min":2.2429999999999999,"15min":2.6099999999999999},"last":1.581}]}]},{"osd":2,"up_from":13,"seq":55834575042,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40652,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331188,"statfs":{"total":96636764160,"available":96595136512,"internally_reserved":0,"allocated":1171456,"data_stored":553619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.875,"15min":0.86699999999999999},"min":{"1min":0.499,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.506,"5min":1.5469999999999999,"15min":2.8690000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":1,"5min":0.98099999999999998,"15min":1.028},"min":{"1min":0.52100000000000002,"5min":0.34999999999999998,"15min":0.28899999999999998},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.155}]},{"osd":1,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.044,"15min":1.036},"min":{"1min":0.56999999999999995,"5min":0.52900000000000003,"15min":0.42899999999999999},"max":{"1min":1.5329999999999999,"5min":1.8080000000000001,"15min":2.6059999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0629999999999999,"15min":1.1140000000000001},"min":{"1min":0.622,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4770000000000001,"5min":1.792,"15min":2.6499999999999999},"last":1.3999999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":1.006,"15min":1.0089999999999999},"min":{"1min":0.55400000000000005,"5min":0.41399999999999998,"15min":0.38},"max":{"1min":1.3300000000000001,"5min":1.776,"15min":2.5339999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.226,"5min":1.169,"15min":1.1970000000000001},"min":{"1min":0.59799999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.7969999999999999,"5min":1.8,"15min":2.9700000000000002},"last":0.88700000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.3420000000000001,"15min":1.3480000000000001},"min":{"1min":1.083,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":2.2309999999999999},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3240000000000001,"15min":1.302},"min":{"1min":0.97899999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.2000000000000002},"last":1.2030000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3600000000000001,"15min":1.373},"min":{"1min":1.0249999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.9810000000000001,"5min":2.1640000000000001,"15min":2.7770000000000001},"last":1.361},{"interface":"front","average":{"1min":1.462,"5min":1.391,"15min":1.415},"min":{"1min":0.72699999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.885,"5min":1.956,"15min":2.952},"last":1.341}]},{"osd":6,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3400000000000001,"15min":1.3360000000000001},"min":{"1min":0.89100000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.831,"5min":2.1970000000000001,"15min":2.2469999999999999},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.4330000000000001,"15min":1.4410000000000001},"min":{"1min":1.1180000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.8140000000000001,"5min":2.044,"15min":2.8180000000000001},"last":1.2509999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.4630000000000001,"15min":1.4850000000000001},"min":{"1min":1.0109999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.012,"5min":2.0739999999999998,"15min":3.746},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.411,"15min":1.431},"min":{"1min":1.085,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.9279999999999999,"5min":2.3109999999999999,"15min":2.3769999999999998},"last":1.5329999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575042,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50372,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321468,"statfs":{"total":96636764160,"available":96585183232,"internally_reserved":0,"allocated":1167360,"data_stored":547935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.83399999999999996,"15min":0.88},"min":{"1min":0.67400000000000004,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2909999999999999,"5min":1.363,"15min":2.2010000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.087,"5min":0.99099999999999999,"15min":0.998},"min":{"1min":0.66100000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.6200000000000001,"5min":1.798,"15min":2.1619999999999999},"last":0.76600000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.98199999999999998,"15min":0.995},"min":{"1min":0.60399999999999998,"5min":0.45300000000000001,"15min":0.42299999999999999},"max":{"1min":1.3700000000000001,"5min":1.526,"15min":2.3849999999999998},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2430000000000001},"min":{"1min":0.70199999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5669999999999999,"5min":1.667,"15min":2.2959999999999998},"last":0.46000000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.113,"15min":1.143},"min":{"1min":0.70199999999999996,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":2.254},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.141,"5min":1.0660000000000001,"15min":1.1060000000000001},"min":{"1min":0.83299999999999996,"5min":0.40699999999999997,"15min":0.36099999999999999},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":2.8109999999999999},"last":0.91800000000000004}]},{"osd":4,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.206,"15min":1.2450000000000001},"min":{"1min":0.84099999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5189999999999999,"5min":1.7010000000000001,"15min":2.4540000000000002},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3069999999999999,"15min":1.3799999999999999},"min":{"1min":1.0309999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5389999999999999,"5min":1.7789999999999999,"15min":2.851},"last":1.0209999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3129999999999999,"15min":1.345},"min":{"1min":1.155,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5820000000000001,"5min":1.7889999999999999,"15min":2.7120000000000002},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.298,"15min":1.335},"min":{"1min":1.034,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.7290000000000001,"5min":1.7869999999999999,"15min":2.512},"last":0.98399999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.385,"15min":1.4099999999999999},"min":{"1min":1.0760000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6830000000000001,"5min":1.7649999999999999,"15min":2.75},"last":1.236},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.3620000000000001,"15min":1.401},"min":{"1min":1.014,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.8029999999999999,"5min":1.851,"15min":2.669},"last":0.80800000000000005}]},{"osd":7,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4339999999999999,"15min":1.494},"min":{"1min":1.1619999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.784,"5min":1.8799999999999999,"15min":2.8799999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.4910000000000001,"15min":1.506},"min":{"1min":1.2589999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":3.1400000000000001},"last":1.1659999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575043,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32892,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338948,"statfs":{"total":96636764160,"available":96603082752,"internally_reserved":0,"allocated":1753088,"data_stored":1142375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.86899999999999999,"15min":0.85699999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.585,"5min":2.766,"15min":2.766},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.998,"15min":1.03},"min":{"1min":0.44900000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.5720000000000001,"5min":3.96,"15min":3.96},"last":0.73499999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0660000000000001,"15min":1.0429999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.38900000000000001},"max":{"1min":1.526,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.048,"5min":1.107,"15min":1.089},"min":{"1min":0.50700000000000001,"5min":0.48999999999999999,"15min":0.46600000000000003},"max":{"1min":1.722,"5min":4.4290000000000003,"15min":4.4290000000000003},"last":0.86299999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.054,"15min":1.032},"min":{"1min":0.495,"5min":0.38,"15min":0.38},"max":{"1min":1.498,"5min":4.7380000000000004,"15min":4.7380000000000004},"last":1.032},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.24,"15min":1.202},"min":{"1min":0.47099999999999997,"5min":0.44,"15min":0.44},"max":{"1min":2.1859999999999999,"5min":5.1500000000000004,"15min":5.1500000000000004},"last":1.278}]},{"osd":4,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.341,"15min":1.3340000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.859,"5min":4.1970000000000001,"15min":4.1970000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.3049999999999999,"15min":1.2909999999999999},"min":{"1min":0.77900000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.623,"5min":4.9630000000000001,"15min":4.9630000000000001},"last":1.2450000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3280000000000001,"15min":1.3220000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6799999999999999,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.399,"5min":1.4239999999999999,"15min":1.407},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.1269999999999998,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":1.1619999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.464,"15min":1.4390000000000001},"min":{"1min":0.98099999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0739999999999998,"5min":3.8250000000000002,"15min":3.8250000000000002},"last":1.401},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.444,"15min":1.4259999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.84999999999999998},"max":{"1min":2.02,"5min":4.0979999999999999,"15min":4.0979999999999999},"last":1.2090000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.49,"15min":1.4750000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7929999999999999,"5min":4.3369999999999997,"15min":4.3369999999999997},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4610000000000001,"15min":1.4359999999999999},"min":{"1min":0.83599999999999997,"5min":0.755,"15min":0.755},"max":{"1min":1.8169999999999999,"5min":5.0570000000000004,"15min":5.0570000000000004},"last":1.3680000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:05:15.850 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:05:16.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.101+0000 7f5c0a8e7640 1 Processor -- start 2023-10-16T10:05:16.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.101+0000 7f5c0a8e7640 1 -- start start 2023-10-16T10:05:16.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.102+0000 7f5c0a8e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c040adb50 0x7f5c0406f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:16.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.103+0000 7f5c0a8e7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5c040b0b20 0x7f5c04164f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:16.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.103+0000 7f5c0a8e7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c04071450 con 0x7f5c04070fd0 2023-10-16T10:05:16.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.103+0000 7f5c0a8e7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c040cc600 con 0x7f5c040b0b20 2023-10-16T10:05:16.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.103+0000 7f5c0a8e7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c040cc8e0 con 0x7f5c040adb50 2023-10-16T10:05:16.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.103+0000 7f5c08e84640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5c040b0b20 0x7f5c04164f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:16.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.103+0000 7f5c08e84640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5c040b0b20 0x7f5c04164f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35498/0 (socket says 172.21.15.84:35498) 2023-10-16T10:05:16.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.103+0000 7f5c08e84640 1 -- 172.21.15.84:0/4213236630 learned_addr learned my addr 172.21.15.84:0/4213236630 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:16.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.103+0000 7f5c03fff640 1 --2- 172.21.15.84:0/4213236630 >> v2:172.21.15.157:3300/0 conn(0x7f5c040adb50 0x7f5c0406f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:16.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.103+0000 7f5c08e84640 1 -- 172.21.15.84:0/4213236630 >> v2:172.21.15.157:3300/0 conn(0x7f5c040adb50 msgr2=0x7f5c0406f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:16.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.103+0000 7f5c08e84640 1 --2- 172.21.15.84:0/4213236630 >> v2:172.21.15.157:3300/0 conn(0x7f5c040adb50 0x7f5c0406f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.104+0000 7f5c08e84640 1 -- 172.21.15.84:0/4213236630 >> v1:172.21.15.157:6789/0 conn(0x7f5c04070fd0 legacy=0x7f5c040b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:16.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.104+0000 7f5c08e84640 1 -- 172.21.15.84:0/4213236630 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c040ccbc0 con 0x7f5c040b0b20 2023-10-16T10:05:16.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.105+0000 7f5c08e84640 1 --2- 172.21.15.84:0/4213236630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c040b0b20 0x7f5c04164f60 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5c04071a00 tx=0x7f5bfc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271ada2c0b684021 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:16.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.105+0000 7f5c037fe640 1 -- 172.21.15.84:0/4213236630 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc02ca80 con 0x7f5c040b0b20 2023-10-16T10:05:16.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.105+0000 7f5c037fe640 1 -- 172.21.15.84:0/4213236630 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bfc02cbe0 con 0x7f5c040b0b20 2023-10-16T10:05:16.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.105+0000 7f5c037fe640 1 -- 172.21.15.84:0/4213236630 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0343f0 con 0x7f5c040b0b20 2023-10-16T10:05:16.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.105+0000 7f5c0a8e7640 1 -- 172.21.15.84:0/4213236630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c040b0b20 msgr2=0x7f5c04164f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:16.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.105+0000 7f5c0a8e7640 1 --2- 172.21.15.84:0/4213236630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c040b0b20 0x7f5c04164f60 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5c04071a00 tx=0x7f5bfc02bee0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.106+0000 7f5c0a8e7640 1 -- 172.21.15.84:0/4213236630 shutdown_connections 2023-10-16T10:05:16.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.106+0000 7f5c0a8e7640 1 --2- 172.21.15.84:0/4213236630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c040b0b20 0x7f5c04164f60 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.106+0000 7f5c0a8e7640 1 --2- 172.21.15.84:0/4213236630 >> v2:172.21.15.157:3300/0 conn(0x7f5c040adb50 0x7f5c0406f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.106+0000 7f5c0a8e7640 1 -- 172.21.15.84:0/4213236630 >> 172.21.15.84:0/4213236630 conn(0x7f5c040cb410 msgr2=0x7f5c040cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:16.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.106+0000 7f5c0a8e7640 1 -- 172.21.15.84:0/4213236630 shutdown_connections 2023-10-16T10:05:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.106+0000 7f5c0a8e7640 1 -- 172.21.15.84:0/4213236630 wait complete. 2023-10-16T10:05:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.107+0000 7f5c0a8e7640 1 Processor -- start 2023-10-16T10:05:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.107+0000 7f5c0a8e7640 1 -- start start 2023-10-16T10:05:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.108+0000 7f5c0a8e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c04070fd0 0x7f5c040bd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:16.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.108+0000 7f5c0a8e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c040adb50 0x7f5c0415e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:16.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.108+0000 7f5c0a8e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c040b0b20 0x7f5c040b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:16.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.108+0000 7f5c0a8e7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0415e940 con 0x7f5c040b0b20 2023-10-16T10:05:16.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.108+0000 7f5c0a8e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c040b2910 con 0x7f5c04070fd0 2023-10-16T10:05:16.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.108+0000 7f5c0a8e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c040b2bb0 con 0x7f5c040adb50 2023-10-16T10:05:16.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.109+0000 7f5c08e84640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c040b0b20 0x7f5c040b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:16.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.109+0000 7f5c08e84640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c040b0b20 0x7f5c040b23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35508/0 (socket says 172.21.15.84:35508) 2023-10-16T10:05:16.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.109+0000 7f5c08e84640 1 -- 172.21.15.84:0/4211185727 learned_addr learned my addr 172.21.15.84:0/4211185727 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:16.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.109+0000 7f5c08e84640 1 -- 172.21.15.84:0/4211185727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c040adb50 msgr2=0x7f5c0415e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:16.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.109+0000 7f5c08e84640 1 --2- 172.21.15.84:0/4211185727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c040adb50 0x7f5c0415e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.109+0000 7f5c08e84640 1 -- 172.21.15.84:0/4211185727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c04070fd0 msgr2=0x7f5c040bd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:16.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.109+0000 7f5c08e84640 1 --2- 172.21.15.84:0/4211185727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c04070fd0 0x7f5c040bd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.109+0000 7f5c08e84640 1 -- 172.21.15.84:0/4211185727 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bfc009b30 con 0x7f5c040b0b20 2023-10-16T10:05:16.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.109+0000 7f5c08e84640 1 --2- 172.21.15.84:0/4211185727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c040b0b20 0x7f5c040b23d0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc036040 tx=0x7f5bfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:16.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.110+0000 7f5c017fa640 1 -- 172.21.15.84:0/4211185727 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc009510 con 0x7f5c040b0b20 2023-10-16T10:05:16.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.110+0000 7f5c017fa640 1 -- 172.21.15.84:0/4211185727 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bfc03e450 con 0x7f5c040b0b20 2023-10-16T10:05:16.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.110+0000 7f5c017fa640 1 -- 172.21.15.84:0/4211185727 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc03e7e0 con 0x7f5c040b0b20 2023-10-16T10:05:16.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.110+0000 7f5c0a8e7640 1 -- 172.21.15.84:0/4211185727 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c040c3a20 con 0x7f5c040b0b20 2023-10-16T10:05:16.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.110+0000 7f5c0a8e7640 1 -- 172.21.15.84:0/4211185727 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c040c3fe0 con 0x7f5c040b0b20 2023-10-16T10:05:16.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.111+0000 7f5c0a8e7640 1 -- 172.21.15.84:0/4211185727 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c040c41d0 con 0x7f5c040b0b20 2023-10-16T10:05:16.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.112+0000 7f5c017fa640 1 -- 172.21.15.84:0/4211185727 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5bfc02cd50 con 0x7f5c040b0b20 2023-10-16T10:05:16.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.113+0000 7f5c017fa640 1 --2- 172.21.15.84:0/4211185727 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f5be406af90 0x7f5be406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:16.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.113+0000 7f5c017fa640 1 -- 172.21.15.84:0/4211185727 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bfc0ac3d0 con 0x7f5c040b0b20 2023-10-16T10:05:16.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.114+0000 7f5c03fff640 1 --2- 172.21.15.84:0/4211185727 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f5be406af90 0x7f5be406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:16.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.114+0000 7f5c03fff640 1 --2- 172.21.15.84:0/4211185727 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f5be406af90 0x7f5be406d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4006fd0 tx=0x7f5bf4008040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:16.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.118+0000 7f5c017fa640 1 -- 172.21.15.84:0/4211185727 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5bfc0773f0 con 0x7f5c040b0b20 2023-10-16T10:05:16.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.305+0000 7f5c0a8e7640 1 -- 172.21.15.84:0/4211185727 --> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5c040c2150 con 0x7f5be406af90 2023-10-16T10:05:16.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.327+0000 7f5c017fa640 1 -- 172.21.15.84:0/4211185727 <== mgr.6786 v2:172.21.15.84:6832/147697688 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287048 (secure 0 0 0) 0x7f5bf400be20 con 0x7f5be406af90 2023-10-16T10:05:16.334 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:16.335 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T10:05:16.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.334+0000 7f5c0a8e7640 1 -- 172.21.15.84:0/4211185727 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f5be406af90 msgr2=0x7f5be406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:16.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.334+0000 7f5c0a8e7640 1 --2- 172.21.15.84:0/4211185727 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f5be406af90 0x7f5be406d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4006fd0 tx=0x7f5bf4008040 comp rx=0 tx=0).stop 2023-10-16T10:05:16.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.334+0000 7f5c0a8e7640 1 -- 172.21.15.84:0/4211185727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c040b0b20 msgr2=0x7f5c040b23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:16.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.334+0000 7f5c0a8e7640 1 --2- 172.21.15.84:0/4211185727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c040b0b20 0x7f5c040b23d0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc036040 tx=0x7f5bfc004a40 comp rx=0 tx=0).stop 2023-10-16T10:05:16.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.335+0000 7f5c0a8e7640 1 -- 172.21.15.84:0/4211185727 shutdown_connections 2023-10-16T10:05:16.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.335+0000 7f5c0a8e7640 1 --2- 172.21.15.84:0/4211185727 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f5be406af90 0x7f5be406d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.335+0000 7f5c0a8e7640 1 --2- 172.21.15.84:0/4211185727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c040b0b20 0x7f5c040b23d0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.335+0000 7f5c0a8e7640 1 --2- 172.21.15.84:0/4211185727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c040adb50 0x7f5c0415e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.335+0000 7f5c0a8e7640 1 --2- 172.21.15.84:0/4211185727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c04070fd0 0x7f5c040bd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.335+0000 7f5c0a8e7640 1 -- 172.21.15.84:0/4211185727 >> 172.21.15.84:0/4211185727 conn(0x7f5c040cb410 msgr2=0x7f5c04160ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:16.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.335+0000 7f5c0a8e7640 1 -- 172.21.15.84:0/4211185727 shutdown_connections 2023-10-16T10:05:16.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.335+0000 7f5c0a8e7640 1 -- 172.21.15.84:0/4211185727 wait complete. 2023-10-16T10:05:16.356 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-16T10:05:15.474186+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":600,"num_read_kb":1133,"num_write":303,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":316612,"kb_used_data":10244,"kb_used_omap":12,"kb_used_meta":306227,"kb_avail":754658108,"statfs":{"total":773094113280,"available":772769902592,"internally_reserved":0,"allocated":10489856,"data_stored":5582712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":313577017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001327"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.323308+0000","last_change":"2023-10-16T10:05:11.414668+0000","last_active":"2023-10-16T10:05:14.323308+0000","last_peered":"2023-10-16T10:05:14.323308+0000","last_clean":"2023-10-16T10:05:14.323308+0000","last_became_active":"2023-10-16T10:05:11.414182+0000","last_became_peered":"2023-10-16T10:05:11.414182+0000","last_unstale":"2023-10-16T10:05:14.323308+0000","last_undegraded":"2023-10-16T10:05:14.323308+0000","last_fullsized":"2023-10-16T10:05:14.323308+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:58:48.434293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.198774+0000","last_change":"2023-10-16T10:05:11.403026+0000","last_active":"2023-10-16T10:05:13.198774+0000","last_peered":"2023-10-16T10:05:13.198774+0000","last_clean":"2023-10-16T10:05:13.198774+0000","last_became_active":"2023-10-16T10:05:11.401337+0000","last_became_peered":"2023-10-16T10:05:11.401337+0000","last_unstale":"2023-10-16T10:05:13.198774+0000","last_undegraded":"2023-10-16T10:05:13.198774+0000","last_fullsized":"2023-10-16T10:05:13.198774+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:38:09.960436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.399655+0000","last_change":"2023-10-16T10:05:11.396721+0000","last_active":"2023-10-16T10:05:11.399655+0000","last_peered":"2023-10-16T10:05:11.399655+0000","last_clean":"2023-10-16T10:05:11.399655+0000","last_became_active":"2023-10-16T10:05:11.395538+0000","last_became_peered":"2023-10-16T10:05:11.395538+0000","last_unstale":"2023-10-16T10:05:11.399655+0000","last_undegraded":"2023-10-16T10:05:11.399655+0000","last_fullsized":"2023-10-16T10:05:11.399655+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:56.490714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.400652+0000","last_change":"2023-10-16T10:05:11.395295+0000","last_active":"2023-10-16T10:05:11.400652+0000","last_peered":"2023-10-16T10:05:11.400652+0000","last_clean":"2023-10-16T10:05:11.400652+0000","last_became_active":"2023-10-16T10:05:11.394833+0000","last_became_peered":"2023-10-16T10:05:11.394833+0000","last_unstale":"2023-10-16T10:05:11.400652+0000","last_undegraded":"2023-10-16T10:05:11.400652+0000","last_fullsized":"2023-10-16T10:05:11.400652+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:06:07.971004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.397367+0000","last_change":"2023-10-16T10:05:11.412427+0000","last_active":"2023-10-16T10:05:12.397367+0000","last_peered":"2023-10-16T10:05:12.397367+0000","last_clean":"2023-10-16T10:05:12.397367+0000","last_became_active":"2023-10-16T10:05:11.411718+0000","last_became_peered":"2023-10-16T10:05:11.411718+0000","last_unstale":"2023-10-16T10:05:12.397367+0000","last_undegraded":"2023-10-16T10:05:12.397367+0000","last_fullsized":"2023-10-16T10:05:12.397367+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:51.178511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.198597+0000","last_change":"2023-10-16T10:05:11.403384+0000","last_active":"2023-10-16T10:05:13.198597+0000","last_peered":"2023-10-16T10:05:13.198597+0000","last_clean":"2023-10-16T10:05:13.198597+0000","last_became_active":"2023-10-16T10:05:11.401451+0000","last_became_peered":"2023-10-16T10:05:11.401451+0000","last_unstale":"2023-10-16T10:05:13.198597+0000","last_undegraded":"2023-10-16T10:05:13.198597+0000","last_fullsized":"2023-10-16T10:05:13.198597+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:57.580285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.082540+0000","last_change":"2023-10-16T10:05:11.406465+0000","last_active":"2023-10-16T10:05:13.082540+0000","last_peered":"2023-10-16T10:05:13.082540+0000","last_clean":"2023-10-16T10:05:13.082540+0000","last_became_active":"2023-10-16T10:05:11.404685+0000","last_became_peered":"2023-10-16T10:05:11.404685+0000","last_unstale":"2023-10-16T10:05:13.082540+0000","last_undegraded":"2023-10-16T10:05:13.082540+0000","last_fullsized":"2023-10-16T10:05:13.082540+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:21:57.974477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.198240+0000","last_change":"2023-10-16T10:05:11.388725+0000","last_active":"2023-10-16T10:05:13.198240+0000","last_peered":"2023-10-16T10:05:13.198240+0000","last_clean":"2023-10-16T10:05:13.198240+0000","last_became_active":"2023-10-16T10:05:11.387352+0000","last_became_peered":"2023-10-16T10:05:11.387352+0000","last_unstale":"2023-10-16T10:05:13.198240+0000","last_undegraded":"2023-10-16T10:05:13.198240+0000","last_fullsized":"2023-10-16T10:05:13.198240+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:55.605058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.397888+0000","last_change":"2023-10-16T10:05:11.398654+0000","last_active":"2023-10-16T10:05:12.397888+0000","last_peered":"2023-10-16T10:05:12.397888+0000","last_clean":"2023-10-16T10:05:12.397888+0000","last_became_active":"2023-10-16T10:05:11.398131+0000","last_became_peered":"2023-10-16T10:05:11.398131+0000","last_unstale":"2023-10-16T10:05:12.397888+0000","last_undegraded":"2023-10-16T10:05:12.397888+0000","last_fullsized":"2023-10-16T10:05:12.397888+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:03:16.251267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.401043+0000","last_change":"2023-10-16T10:05:11.395215+0000","last_active":"2023-10-16T10:05:12.401043+0000","last_peered":"2023-10-16T10:05:12.401043+0000","last_clean":"2023-10-16T10:05:12.401043+0000","last_became_active":"2023-10-16T10:05:11.394873+0000","last_became_peered":"2023-10-16T10:05:11.394873+0000","last_unstale":"2023-10-16T10:05:12.401043+0000","last_undegraded":"2023-10-16T10:05:12.401043+0000","last_fullsized":"2023-10-16T10:05:12.401043+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:23.942054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.323206+0000","last_change":"2023-10-16T10:05:11.413876+0000","last_active":"2023-10-16T10:05:14.323206+0000","last_peered":"2023-10-16T10:05:14.323206+0000","last_clean":"2023-10-16T10:05:14.323206+0000","last_became_active":"2023-10-16T10:05:11.413576+0000","last_became_peered":"2023-10-16T10:05:11.413576+0000","last_unstale":"2023-10-16T10:05:14.323206+0000","last_undegraded":"2023-10-16T10:05:14.323206+0000","last_fullsized":"2023-10-16T10:05:14.323206+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:22.397230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.399291+0000","last_change":"2023-10-16T10:05:11.393132+0000","last_active":"2023-10-16T10:05:11.399291+0000","last_peered":"2023-10-16T10:05:11.399291+0000","last_clean":"2023-10-16T10:05:11.399291+0000","last_became_active":"2023-10-16T10:05:11.392519+0000","last_became_peered":"2023-10-16T10:05:11.392519+0000","last_unstale":"2023-10-16T10:05:11.399291+0000","last_undegraded":"2023-10-16T10:05:11.399291+0000","last_fullsized":"2023-10-16T10:05:11.399291+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:33:37.874737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.398866+0000","last_change":"2023-10-16T10:05:11.393902+0000","last_active":"2023-10-16T10:05:12.398866+0000","last_peered":"2023-10-16T10:05:12.398866+0000","last_clean":"2023-10-16T10:05:12.398866+0000","last_became_active":"2023-10-16T10:05:11.393633+0000","last_became_peered":"2023-10-16T10:05:11.393633+0000","last_unstale":"2023-10-16T10:05:12.398866+0000","last_undegraded":"2023-10-16T10:05:12.398866+0000","last_fullsized":"2023-10-16T10:05:12.398866+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:38:09.960436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.397519+0000","last_change":"2023-10-16T10:05:11.405003+0000","last_active":"2023-10-16T10:05:12.397519+0000","last_peered":"2023-10-16T10:05:12.397519+0000","last_clean":"2023-10-16T10:05:12.397519+0000","last_became_active":"2023-10-16T10:05:11.404691+0000","last_became_peered":"2023-10-16T10:05:11.404691+0000","last_unstale":"2023-10-16T10:05:12.397519+0000","last_undegraded":"2023-10-16T10:05:12.397519+0000","last_fullsized":"2023-10-16T10:05:12.397519+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:46:53.588751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.399221+0000","last_change":"2023-10-16T10:05:11.387839+0000","last_active":"2023-10-16T10:05:12.399221+0000","last_peered":"2023-10-16T10:05:12.399221+0000","last_clean":"2023-10-16T10:05:12.399221+0000","last_became_active":"2023-10-16T10:05:11.387575+0000","last_became_peered":"2023-10-16T10:05:11.387575+0000","last_unstale":"2023-10-16T10:05:12.399221+0000","last_undegraded":"2023-10-16T10:05:12.399221+0000","last_fullsized":"2023-10-16T10:05:12.399221+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:35.765791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.399587+0000","last_change":"2023-10-16T10:05:11.397970+0000","last_active":"2023-10-16T10:05:12.399587+0000","last_peered":"2023-10-16T10:05:12.399587+0000","last_clean":"2023-10-16T10:05:12.399587+0000","last_became_active":"2023-10-16T10:05:11.397686+0000","last_became_peered":"2023-10-16T10:05:11.397686+0000","last_unstale":"2023-10-16T10:05:12.399587+0000","last_undegraded":"2023-10-16T10:05:12.399587+0000","last_fullsized":"2023-10-16T10:05:12.399587+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:14.948508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.398035+0000","last_change":"2023-10-16T10:05:11.398459+0000","last_active":"2023-10-16T10:05:12.398035+0000","last_peered":"2023-10-16T10:05:12.398035+0000","last_clean":"2023-10-16T10:05:12.398035+0000","last_became_active":"2023-10-16T10:05:11.397678+0000","last_became_peered":"2023-10-16T10:05:11.397678+0000","last_unstale":"2023-10-16T10:05:12.398035+0000","last_undegraded":"2023-10-16T10:05:12.398035+0000","last_fullsized":"2023-10-16T10:05:12.398035+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:06:20.963899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.403886+0000","last_change":"2023-10-16T10:05:11.401189+0000","last_active":"2023-10-16T10:05:11.403886+0000","last_peered":"2023-10-16T10:05:11.403886+0000","last_clean":"2023-10-16T10:05:11.403886+0000","last_became_active":"2023-10-16T10:05:11.400916+0000","last_became_peered":"2023-10-16T10:05:11.400916+0000","last_unstale":"2023-10-16T10:05:11.403886+0000","last_undegraded":"2023-10-16T10:05:11.403886+0000","last_fullsized":"2023-10-16T10:05:11.403886+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:59:06.547242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.198186+0000","last_change":"2023-10-16T10:05:11.391390+0000","last_active":"2023-10-16T10:05:13.198186+0000","last_peered":"2023-10-16T10:05:13.198186+0000","last_clean":"2023-10-16T10:05:13.198186+0000","last_became_active":"2023-10-16T10:05:11.391087+0000","last_became_peered":"2023-10-16T10:05:11.391087+0000","last_unstale":"2023-10-16T10:05:13.198186+0000","last_undegraded":"2023-10-16T10:05:13.198186+0000","last_fullsized":"2023-10-16T10:05:13.198186+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:20:34.386787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.399044+0000","last_change":"2023-10-16T10:05:11.401586+0000","last_active":"2023-10-16T10:05:12.399044+0000","last_peered":"2023-10-16T10:05:12.399044+0000","last_clean":"2023-10-16T10:05:12.399044+0000","last_became_active":"2023-10-16T10:05:11.401303+0000","last_became_peered":"2023-10-16T10:05:11.401303+0000","last_unstale":"2023-10-16T10:05:12.399044+0000","last_undegraded":"2023-10-16T10:05:12.399044+0000","last_fullsized":"2023-10-16T10:05:12.399044+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:23:38.009256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.400099+0000","last_change":"2023-10-16T10:05:11.392880+0000","last_active":"2023-10-16T10:05:12.400099+0000","last_peered":"2023-10-16T10:05:12.400099+0000","last_clean":"2023-10-16T10:05:12.400099+0000","last_became_active":"2023-10-16T10:05:11.392540+0000","last_became_peered":"2023-10-16T10:05:11.392540+0000","last_unstale":"2023-10-16T10:05:12.400099+0000","last_undegraded":"2023-10-16T10:05:12.400099+0000","last_fullsized":"2023-10-16T10:05:12.400099+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:43:06.561050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.400009+0000","last_change":"2023-10-16T10:05:11.389170+0000","last_active":"2023-10-16T10:05:12.400009+0000","last_peered":"2023-10-16T10:05:12.400009+0000","last_clean":"2023-10-16T10:05:12.400009+0000","last_became_active":"2023-10-16T10:05:11.388765+0000","last_became_peered":"2023-10-16T10:05:11.388765+0000","last_unstale":"2023-10-16T10:05:12.400009+0000","last_undegraded":"2023-10-16T10:05:12.400009+0000","last_fullsized":"2023-10-16T10:05:12.400009+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:33:53.035562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.322993+0000","last_change":"2023-10-16T10:05:11.414359+0000","last_active":"2023-10-16T10:05:14.322993+0000","last_peered":"2023-10-16T10:05:14.322993+0000","last_clean":"2023-10-16T10:05:14.322993+0000","last_became_active":"2023-10-16T10:05:11.414015+0000","last_became_peered":"2023-10-16T10:05:11.414015+0000","last_unstale":"2023-10-16T10:05:14.322993+0000","last_undegraded":"2023-10-16T10:05:14.322993+0000","last_fullsized":"2023-10-16T10:05:14.322993+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:57:56.336265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.322581+0000","last_change":"2023-10-16T10:05:11.403248+0000","last_active":"2023-10-16T10:05:14.322581+0000","last_peered":"2023-10-16T10:05:14.322581+0000","last_clean":"2023-10-16T10:05:14.322581+0000","last_became_active":"2023-10-16T10:05:11.402947+0000","last_became_peered":"2023-10-16T10:05:11.402947+0000","last_unstale":"2023-10-16T10:05:14.322581+0000","last_undegraded":"2023-10-16T10:05:14.322581+0000","last_fullsized":"2023-10-16T10:05:14.322581+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:28:10.619415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.401081+0000","last_change":"2023-10-16T10:05:11.408662+0000","last_active":"2023-10-16T10:05:12.401081+0000","last_peered":"2023-10-16T10:05:12.401081+0000","last_clean":"2023-10-16T10:05:12.401081+0000","last_became_active":"2023-10-16T10:05:11.407404+0000","last_became_peered":"2023-10-16T10:05:11.407404+0000","last_unstale":"2023-10-16T10:05:12.401081+0000","last_undegraded":"2023-10-16T10:05:12.401081+0000","last_fullsized":"2023-10-16T10:05:12.401081+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:43.832121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.082939+0000","last_change":"2023-10-16T10:05:11.404472+0000","last_active":"2023-10-16T10:05:13.082939+0000","last_peered":"2023-10-16T10:05:13.082939+0000","last_clean":"2023-10-16T10:05:13.082939+0000","last_became_active":"2023-10-16T10:05:11.404069+0000","last_became_peered":"2023-10-16T10:05:11.404069+0000","last_unstale":"2023-10-16T10:05:13.082939+0000","last_undegraded":"2023-10-16T10:05:13.082939+0000","last_fullsized":"2023-10-16T10:05:13.082939+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:27:45.274018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.412492+0000","last_change":"2023-10-16T10:05:11.411252+0000","last_active":"2023-10-16T10:05:11.412492+0000","last_peered":"2023-10-16T10:05:11.412492+0000","last_clean":"2023-10-16T10:05:11.412492+0000","last_became_active":"2023-10-16T10:05:11.410883+0000","last_became_peered":"2023-10-16T10:05:11.410883+0000","last_unstale":"2023-10-16T10:05:11.412492+0000","last_undegraded":"2023-10-16T10:05:11.412492+0000","last_fullsized":"2023-10-16T10:05:11.412492+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:51.160809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.400726+0000","last_change":"2023-10-16T10:05:11.394898+0000","last_active":"2023-10-16T10:05:12.400726+0000","last_peered":"2023-10-16T10:05:12.400726+0000","last_clean":"2023-10-16T10:05:12.400726+0000","last_became_active":"2023-10-16T10:05:11.394561+0000","last_became_peered":"2023-10-16T10:05:11.394561+0000","last_unstale":"2023-10-16T10:05:12.400726+0000","last_undegraded":"2023-10-16T10:05:12.400726+0000","last_fullsized":"2023-10-16T10:05:12.400726+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:02:20.574735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.400021+0000","last_change":"2023-10-16T10:05:11.397623+0000","last_active":"2023-10-16T10:05:11.400021+0000","last_peered":"2023-10-16T10:05:11.400021+0000","last_clean":"2023-10-16T10:05:11.400021+0000","last_became_active":"2023-10-16T10:05:11.397357+0000","last_became_peered":"2023-10-16T10:05:11.397357+0000","last_unstale":"2023-10-16T10:05:11.400021+0000","last_undegraded":"2023-10-16T10:05:11.400021+0000","last_fullsized":"2023-10-16T10:05:11.400021+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:18:54.637543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.082692+0000","last_change":"2023-10-16T10:05:11.405893+0000","last_active":"2023-10-16T10:05:13.082692+0000","last_peered":"2023-10-16T10:05:13.082692+0000","last_clean":"2023-10-16T10:05:13.082692+0000","last_became_active":"2023-10-16T10:05:11.405500+0000","last_became_peered":"2023-10-16T10:05:11.405500+0000","last_unstale":"2023-10-16T10:05:13.082692+0000","last_undegraded":"2023-10-16T10:05:13.082692+0000","last_fullsized":"2023-10-16T10:05:13.082692+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:36:38.823109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.082897+0000","last_change":"2023-10-16T10:05:11.409375+0000","last_active":"2023-10-16T10:05:13.082897+0000","last_peered":"2023-10-16T10:05:13.082897+0000","last_clean":"2023-10-16T10:05:13.082897+0000","last_became_active":"2023-10-16T10:05:11.409050+0000","last_became_peered":"2023-10-16T10:05:11.409050+0000","last_unstale":"2023-10-16T10:05:13.082897+0000","last_undegraded":"2023-10-16T10:05:13.082897+0000","last_fullsized":"2023-10-16T10:05:13.082897+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:53:31.471053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.198162+0000","last_change":"2023-10-16T10:05:11.403415+0000","last_active":"2023-10-16T10:05:13.198162+0000","last_peered":"2023-10-16T10:05:13.198162+0000","last_clean":"2023-10-16T10:05:13.198162+0000","last_became_active":"2023-10-16T10:05:11.403027+0000","last_became_peered":"2023-10-16T10:05:11.403027+0000","last_unstale":"2023-10-16T10:05:13.198162+0000","last_undegraded":"2023-10-16T10:05:13.198162+0000","last_fullsized":"2023-10-16T10:05:13.198162+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:05:42.255119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.323385+0000","last_change":"2023-10-16T10:05:11.420161+0000","last_active":"2023-10-16T10:05:14.323385+0000","last_peered":"2023-10-16T10:05:14.323385+0000","last_clean":"2023-10-16T10:05:14.323385+0000","last_became_active":"2023-10-16T10:05:11.419839+0000","last_became_peered":"2023-10-16T10:05:11.419839+0000","last_unstale":"2023-10-16T10:05:14.323385+0000","last_undegraded":"2023-10-16T10:05:14.323385+0000","last_fullsized":"2023-10-16T10:05:14.323385+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:10.036163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"94'4","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.418053+0000","last_change":"2023-10-16T10:05:10.382295+0000","last_active":"2023-10-16T10:05:11.418053+0000","last_peered":"2023-10-16T10:05:11.418053+0000","last_clean":"2023-10-16T10:05:11.418053+0000","last_became_active":"2023-10-16T10:05:10.381968+0000","last_became_peered":"2023-10-16T10:05:10.381968+0000","last_unstale":"2023-10-16T10:05:11.418053+0000","last_undegraded":"2023-10-16T10:05:11.418053+0000","last_fullsized":"2023-10-16T10:05:11.418053+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:31.389526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.398217+0000","last_change":"2023-10-16T10:05:11.398829+0000","last_active":"2023-10-16T10:05:12.398217+0000","last_peered":"2023-10-16T10:05:12.398217+0000","last_clean":"2023-10-16T10:05:12.398217+0000","last_became_active":"2023-10-16T10:05:11.398520+0000","last_became_peered":"2023-10-16T10:05:11.398520+0000","last_unstale":"2023-10-16T10:05:12.398217+0000","last_undegraded":"2023-10-16T10:05:12.398217+0000","last_fullsized":"2023-10-16T10:05:12.398217+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:38:13.502289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"94'1","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.386044+0000","last_change":"2023-10-16T10:05:10.370126+0000","last_active":"2023-10-16T10:05:11.386044+0000","last_peered":"2023-10-16T10:05:11.386044+0000","last_clean":"2023-10-16T10:05:11.386044+0000","last_became_active":"2023-10-16T10:05:10.368626+0000","last_became_peered":"2023-10-16T10:05:10.368626+0000","last_unstale":"2023-10-16T10:05:11.386044+0000","last_undegraded":"2023-10-16T10:05:11.386044+0000","last_fullsized":"2023-10-16T10:05:11.386044+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:20:33.370299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.198204+0000","last_change":"2023-10-16T10:05:11.413038+0000","last_active":"2023-10-16T10:05:13.198204+0000","last_peered":"2023-10-16T10:05:13.198204+0000","last_clean":"2023-10-16T10:05:13.198204+0000","last_became_active":"2023-10-16T10:05:11.412589+0000","last_became_peered":"2023-10-16T10:05:11.412589+0000","last_unstale":"2023-10-16T10:05:13.198204+0000","last_undegraded":"2023-10-16T10:05:13.198204+0000","last_fullsized":"2023-10-16T10:05:13.198204+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:32.785131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.411199+0000","last_change":"2023-10-16T10:05:10.384291+0000","last_active":"2023-10-16T10:05:11.411199+0000","last_peered":"2023-10-16T10:05:11.411199+0000","last_clean":"2023-10-16T10:05:11.411199+0000","last_became_active":"2023-10-16T10:05:10.382504+0000","last_became_peered":"2023-10-16T10:05:10.382504+0000","last_unstale":"2023-10-16T10:05:11.411199+0000","last_undegraded":"2023-10-16T10:05:11.411199+0000","last_fullsized":"2023-10-16T10:05:11.411199+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:07:26.048925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.323315+0000","last_change":"2023-10-16T10:05:11.416290+0000","last_active":"2023-10-16T10:05:14.323315+0000","last_peered":"2023-10-16T10:05:14.323315+0000","last_clean":"2023-10-16T10:05:14.323315+0000","last_became_active":"2023-10-16T10:05:11.415862+0000","last_became_peered":"2023-10-16T10:05:11.415862+0000","last_unstale":"2023-10-16T10:05:14.323315+0000","last_undegraded":"2023-10-16T10:05:14.323315+0000","last_fullsized":"2023-10-16T10:05:14.323315+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:19.013927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.416676+0000","last_change":"2023-10-16T10:05:10.375080+0000","last_active":"2023-10-16T10:05:11.416676+0000","last_peered":"2023-10-16T10:05:11.416676+0000","last_clean":"2023-10-16T10:05:11.416676+0000","last_became_active":"2023-10-16T10:05:10.374523+0000","last_became_peered":"2023-10-16T10:05:10.374523+0000","last_unstale":"2023-10-16T10:05:11.416676+0000","last_undegraded":"2023-10-16T10:05:11.416676+0000","last_fullsized":"2023-10-16T10:05:11.416676+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:11:52.684115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.083180+0000","last_change":"2023-10-16T10:05:11.401995+0000","last_active":"2023-10-16T10:05:13.083180+0000","last_peered":"2023-10-16T10:05:13.083180+0000","last_clean":"2023-10-16T10:05:13.083180+0000","last_became_active":"2023-10-16T10:05:11.401680+0000","last_became_peered":"2023-10-16T10:05:11.401680+0000","last_unstale":"2023-10-16T10:05:13.083180+0000","last_undegraded":"2023-10-16T10:05:13.083180+0000","last_fullsized":"2023-10-16T10:05:13.083180+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:10:13.772569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"94'1","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.391838+0000","last_change":"2023-10-16T10:05:10.373250+0000","last_active":"2023-10-16T10:05:11.391838+0000","last_peered":"2023-10-16T10:05:11.391838+0000","last_clean":"2023-10-16T10:05:11.391838+0000","last_became_active":"2023-10-16T10:05:10.372761+0000","last_became_peered":"2023-10-16T10:05:10.372761+0000","last_unstale":"2023-10-16T10:05:11.391838+0000","last_undegraded":"2023-10-16T10:05:11.391838+0000","last_fullsized":"2023-10-16T10:05:11.391838+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:28:09.602927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.400468+0000","last_change":"2023-10-16T10:05:11.418394+0000","last_active":"2023-10-16T10:05:12.400468+0000","last_peered":"2023-10-16T10:05:12.400468+0000","last_clean":"2023-10-16T10:05:12.400468+0000","last_became_active":"2023-10-16T10:05:11.418068+0000","last_became_peered":"2023-10-16T10:05:11.418068+0000","last_unstale":"2023-10-16T10:05:12.400468+0000","last_undegraded":"2023-10-16T10:05:12.400468+0000","last_fullsized":"2023-10-16T10:05:12.400468+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:35:46.196661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"94'1","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.416712+0000","last_change":"2023-10-16T10:05:10.377635+0000","last_active":"2023-10-16T10:05:11.416712+0000","last_peered":"2023-10-16T10:05:11.416712+0000","last_clean":"2023-10-16T10:05:11.416712+0000","last_became_active":"2023-10-16T10:05:10.377357+0000","last_became_peered":"2023-10-16T10:05:10.377357+0000","last_unstale":"2023-10-16T10:05:11.416712+0000","last_undegraded":"2023-10-16T10:05:11.416712+0000","last_fullsized":"2023-10-16T10:05:11.416712+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:18:35.706363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.397048+0000","last_change":"2023-10-16T10:05:11.391607+0000","last_active":"2023-10-16T10:05:12.397048+0000","last_peered":"2023-10-16T10:05:12.397048+0000","last_clean":"2023-10-16T10:05:12.397048+0000","last_became_active":"2023-10-16T10:05:11.391184+0000","last_became_peered":"2023-10-16T10:05:11.391184+0000","last_unstale":"2023-10-16T10:05:12.397048+0000","last_undegraded":"2023-10-16T10:05:12.397048+0000","last_fullsized":"2023-10-16T10:05:12.397048+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:35.765791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.379842+0000","last_change":"2023-10-16T10:05:10.372922+0000","last_active":"2023-10-16T10:05:11.379842+0000","last_peered":"2023-10-16T10:05:11.379842+0000","last_clean":"2023-10-16T10:05:11.379842+0000","last_became_active":"2023-10-16T10:05:10.372409+0000","last_became_peered":"2023-10-16T10:05:10.372409+0000","last_unstale":"2023-10-16T10:05:11.379842+0000","last_undegraded":"2023-10-16T10:05:11.379842+0000","last_fullsized":"2023-10-16T10:05:11.379842+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:38:12.485801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.399189+0000","last_change":"2023-10-16T10:05:11.407542+0000","last_active":"2023-10-16T10:05:12.399189+0000","last_peered":"2023-10-16T10:05:12.399189+0000","last_clean":"2023-10-16T10:05:12.399189+0000","last_became_active":"2023-10-16T10:05:11.407042+0000","last_became_peered":"2023-10-16T10:05:11.407042+0000","last_unstale":"2023-10-16T10:05:12.399189+0000","last_undegraded":"2023-10-16T10:05:12.399189+0000","last_fullsized":"2023-10-16T10:05:12.399189+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:22:26.235553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.372073+0000","last_change":"2023-10-16T10:05:10.368323+0000","last_active":"2023-10-16T10:05:11.372073+0000","last_peered":"2023-10-16T10:05:11.372073+0000","last_clean":"2023-10-16T10:05:11.372073+0000","last_became_active":"2023-10-16T10:05:10.367902+0000","last_became_peered":"2023-10-16T10:05:10.367902+0000","last_unstale":"2023-10-16T10:05:11.372073+0000","last_undegraded":"2023-10-16T10:05:11.372073+0000","last_fullsized":"2023-10-16T10:05:11.372073+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:06.896544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.083323+0000","last_change":"2023-10-16T10:05:11.416169+0000","last_active":"2023-10-16T10:05:13.083323+0000","last_peered":"2023-10-16T10:05:13.083323+0000","last_clean":"2023-10-16T10:05:13.083323+0000","last_became_active":"2023-10-16T10:05:11.415411+0000","last_became_peered":"2023-10-16T10:05:11.415411+0000","last_unstale":"2023-10-16T10:05:13.083323+0000","last_undegraded":"2023-10-16T10:05:13.083323+0000","last_fullsized":"2023-10-16T10:05:13.083323+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:17:13.781609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.395549+0000","last_change":"2023-10-16T10:05:10.374183+0000","last_active":"2023-10-16T10:05:11.395549+0000","last_peered":"2023-10-16T10:05:11.395549+0000","last_clean":"2023-10-16T10:05:11.395549+0000","last_became_active":"2023-10-16T10:05:10.373838+0000","last_became_peered":"2023-10-16T10:05:10.373838+0000","last_unstale":"2023-10-16T10:05:11.395549+0000","last_undegraded":"2023-10-16T10:05:11.395549+0000","last_fullsized":"2023-10-16T10:05:11.395549+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:41.390739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.323555+0000","last_change":"2023-10-16T10:05:11.417790+0000","last_active":"2023-10-16T10:05:14.323555+0000","last_peered":"2023-10-16T10:05:14.323555+0000","last_clean":"2023-10-16T10:05:14.323555+0000","last_became_active":"2023-10-16T10:05:11.417438+0000","last_became_peered":"2023-10-16T10:05:11.417438+0000","last_unstale":"2023-10-16T10:05:14.323555+0000","last_undegraded":"2023-10-16T10:05:14.323555+0000","last_fullsized":"2023-10-16T10:05:14.323555+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:55.298078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.384396+0000","last_change":"2023-10-16T10:05:10.379280+0000","last_active":"2023-10-16T10:05:11.384396+0000","last_peered":"2023-10-16T10:05:11.384396+0000","last_clean":"2023-10-16T10:05:11.384396+0000","last_became_active":"2023-10-16T10:05:10.379005+0000","last_became_peered":"2023-10-16T10:05:10.379005+0000","last_unstale":"2023-10-16T10:05:11.384396+0000","last_undegraded":"2023-10-16T10:05:11.384396+0000","last_fullsized":"2023-10-16T10:05:11.384396+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:25.051680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.198384+0000","last_change":"2023-10-16T10:05:11.403722+0000","last_active":"2023-10-16T10:05:13.198384+0000","last_peered":"2023-10-16T10:05:13.198384+0000","last_clean":"2023-10-16T10:05:13.198384+0000","last_became_active":"2023-10-16T10:05:11.403413+0000","last_became_peered":"2023-10-16T10:05:11.403413+0000","last_unstale":"2023-10-16T10:05:13.198384+0000","last_undegraded":"2023-10-16T10:05:13.198384+0000","last_fullsized":"2023-10-16T10:05:13.198384+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:05:03.062283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"94'3","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.404422+0000","last_change":"2023-10-16T10:05:10.375760+0000","last_active":"2023-10-16T10:05:11.404422+0000","last_peered":"2023-10-16T10:05:11.404422+0000","last_clean":"2023-10-16T10:05:11.404422+0000","last_became_active":"2023-10-16T10:05:10.375599+0000","last_became_peered":"2023-10-16T10:05:10.375599+0000","last_unstale":"2023-10-16T10:05:11.404422+0000","last_undegraded":"2023-10-16T10:05:11.404422+0000","last_fullsized":"2023-10-16T10:05:11.404422+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:24:29.169411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.397461+0000","last_change":"2023-10-16T10:05:11.392052+0000","last_active":"2023-10-16T10:05:12.397461+0000","last_peered":"2023-10-16T10:05:12.397461+0000","last_clean":"2023-10-16T10:05:12.397461+0000","last_became_active":"2023-10-16T10:05:11.391717+0000","last_became_peered":"2023-10-16T10:05:11.391717+0000","last_unstale":"2023-10-16T10:05:12.397461+0000","last_undegraded":"2023-10-16T10:05:12.397461+0000","last_fullsized":"2023-10-16T10:05:12.397461+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:45:56.717864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"94'6","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.403348+0000","last_change":"2023-10-16T10:05:10.363723+0000","last_active":"2023-10-16T10:05:11.403348+0000","last_peered":"2023-10-16T10:05:11.403348+0000","last_clean":"2023-10-16T10:05:11.403348+0000","last_became_active":"2023-10-16T10:05:10.363368+0000","last_became_peered":"2023-10-16T10:05:10.363368+0000","last_unstale":"2023-10-16T10:05:11.403348+0000","last_undegraded":"2023-10-16T10:05:11.403348+0000","last_fullsized":"2023-10-16T10:05:11.403348+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:21.263378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.322683+0000","last_change":"2023-10-16T10:05:11.411845+0000","last_active":"2023-10-16T10:05:14.322683+0000","last_peered":"2023-10-16T10:05:14.322683+0000","last_clean":"2023-10-16T10:05:14.322683+0000","last_became_active":"2023-10-16T10:05:11.411563+0000","last_became_peered":"2023-10-16T10:05:11.411563+0000","last_unstale":"2023-10-16T10:05:14.322683+0000","last_undegraded":"2023-10-16T10:05:14.322683+0000","last_fullsized":"2023-10-16T10:05:14.322683+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:24:30.185899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"94'2","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.416937+0000","last_change":"2023-10-16T10:05:10.378284+0000","last_active":"2023-10-16T10:05:11.416937+0000","last_peered":"2023-10-16T10:05:11.416937+0000","last_clean":"2023-10-16T10:05:11.416937+0000","last_became_active":"2023-10-16T10:05:10.378024+0000","last_became_peered":"2023-10-16T10:05:10.378024+0000","last_unstale":"2023-10-16T10:05:11.416937+0000","last_undegraded":"2023-10-16T10:05:11.416937+0000","last_fullsized":"2023-10-16T10:05:11.416937+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:39:22.865400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.082775+0000","last_change":"2023-10-16T10:05:11.398771+0000","last_active":"2023-10-16T10:05:13.082775+0000","last_peered":"2023-10-16T10:05:13.082775+0000","last_clean":"2023-10-16T10:05:13.082775+0000","last_became_active":"2023-10-16T10:05:11.398025+0000","last_became_peered":"2023-10-16T10:05:11.398025+0000","last_unstale":"2023-10-16T10:05:13.082775+0000","last_undegraded":"2023-10-16T10:05:13.082775+0000","last_fullsized":"2023-10-16T10:05:13.082775+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:46.619715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.383670+0000","last_change":"2023-10-16T10:05:10.368901+0000","last_active":"2023-10-16T10:05:11.383670+0000","last_peered":"2023-10-16T10:05:11.383670+0000","last_clean":"2023-10-16T10:05:11.383670+0000","last_became_active":"2023-10-16T10:05:10.368631+0000","last_became_peered":"2023-10-16T10:05:10.368631+0000","last_unstale":"2023-10-16T10:05:11.383670+0000","last_undegraded":"2023-10-16T10:05:11.383670+0000","last_fullsized":"2023-10-16T10:05:11.383670+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:37.991581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.398725+0000","last_change":"2023-10-16T10:05:11.395702+0000","last_active":"2023-10-16T10:05:12.398725+0000","last_peered":"2023-10-16T10:05:12.398725+0000","last_clean":"2023-10-16T10:05:12.398725+0000","last_became_active":"2023-10-16T10:05:11.395057+0000","last_became_peered":"2023-10-16T10:05:11.395057+0000","last_unstale":"2023-10-16T10:05:12.398725+0000","last_undegraded":"2023-10-16T10:05:12.398725+0000","last_fullsized":"2023-10-16T10:05:12.398725+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:57.580285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.399968+0000","last_change":"2023-10-16T10:05:10.372553+0000","last_active":"2023-10-16T10:05:11.399968+0000","last_peered":"2023-10-16T10:05:11.399968+0000","last_clean":"2023-10-16T10:05:11.399968+0000","last_became_active":"2023-10-16T10:05:10.372272+0000","last_became_peered":"2023-10-16T10:05:10.372272+0000","last_unstale":"2023-10-16T10:05:11.399968+0000","last_undegraded":"2023-10-16T10:05:11.399968+0000","last_fullsized":"2023-10-16T10:05:11.399968+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:39:27.567450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.397958+0000","last_change":"2023-10-16T10:05:11.398182+0000","last_active":"2023-10-16T10:05:12.397958+0000","last_peered":"2023-10-16T10:05:12.397958+0000","last_clean":"2023-10-16T10:05:12.397958+0000","last_became_active":"2023-10-16T10:05:11.397897+0000","last_became_peered":"2023-10-16T10:05:11.397897+0000","last_unstale":"2023-10-16T10:05:12.397958+0000","last_undegraded":"2023-10-16T10:05:12.397958+0000","last_fullsized":"2023-10-16T10:05:12.397958+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:58.218659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"94'8","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.419352+0000","last_change":"2023-10-16T10:05:10.384020+0000","last_active":"2023-10-16T10:05:11.419352+0000","last_peered":"2023-10-16T10:05:11.419352+0000","last_clean":"2023-10-16T10:05:11.419352+0000","last_became_active":"2023-10-16T10:05:10.383325+0000","last_became_peered":"2023-10-16T10:05:10.383325+0000","last_unstale":"2023-10-16T10:05:11.419352+0000","last_undegraded":"2023-10-16T10:05:11.419352+0000","last_fullsized":"2023-10-16T10:05:11.419352+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:57.297999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.401422+0000","last_change":"2023-10-16T10:05:11.394591+0000","last_active":"2023-10-16T10:05:12.401422+0000","last_peered":"2023-10-16T10:05:12.401422+0000","last_clean":"2023-10-16T10:05:12.401422+0000","last_became_active":"2023-10-16T10:05:11.393980+0000","last_became_peered":"2023-10-16T10:05:11.393980+0000","last_unstale":"2023-10-16T10:05:12.401422+0000","last_undegraded":"2023-10-16T10:05:12.401422+0000","last_fullsized":"2023-10-16T10:05:12.401422+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:16.786433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"94'1","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.416543+0000","last_change":"2023-10-16T10:05:10.371329+0000","last_active":"2023-10-16T10:05:11.416543+0000","last_peered":"2023-10-16T10:05:11.416543+0000","last_clean":"2023-10-16T10:05:11.416543+0000","last_became_active":"2023-10-16T10:05:10.370944+0000","last_became_peered":"2023-10-16T10:05:10.370944+0000","last_unstale":"2023-10-16T10:05:11.416543+0000","last_undegraded":"2023-10-16T10:05:11.416543+0000","last_fullsized":"2023-10-16T10:05:11.416543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:56:20.497615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.323681+0000","last_change":"2023-10-16T10:05:11.414462+0000","last_active":"2023-10-16T10:05:14.323681+0000","last_peered":"2023-10-16T10:05:14.323681+0000","last_clean":"2023-10-16T10:05:14.323681+0000","last_became_active":"2023-10-16T10:05:11.413836+0000","last_became_peered":"2023-10-16T10:05:11.413836+0000","last_unstale":"2023-10-16T10:05:14.323681+0000","last_undegraded":"2023-10-16T10:05:14.323681+0000","last_fullsized":"2023-10-16T10:05:14.323681+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:19:31.171652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.415487+0000","last_change":"2023-10-16T10:05:10.384943+0000","last_active":"2023-10-16T10:05:11.415487+0000","last_peered":"2023-10-16T10:05:11.415487+0000","last_clean":"2023-10-16T10:05:11.415487+0000","last_became_active":"2023-10-16T10:05:10.382395+0000","last_became_peered":"2023-10-16T10:05:10.382395+0000","last_unstale":"2023-10-16T10:05:11.415487+0000","last_undegraded":"2023-10-16T10:05:11.415487+0000","last_fullsized":"2023-10-16T10:05:11.415487+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:10:02.377817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.399312+0000","last_change":"2023-10-16T10:05:11.393326+0000","last_active":"2023-10-16T10:05:11.399312+0000","last_peered":"2023-10-16T10:05:11.399312+0000","last_clean":"2023-10-16T10:05:11.399312+0000","last_became_active":"2023-10-16T10:05:11.392871+0000","last_became_peered":"2023-10-16T10:05:11.392871+0000","last_unstale":"2023-10-16T10:05:11.399312+0000","last_undegraded":"2023-10-16T10:05:11.399312+0000","last_fullsized":"2023-10-16T10:05:11.399312+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:57:19.149543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"94'1","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.416579+0000","last_change":"2023-10-16T10:05:10.380465+0000","last_active":"2023-10-16T10:05:11.416579+0000","last_peered":"2023-10-16T10:05:11.416579+0000","last_clean":"2023-10-16T10:05:11.416579+0000","last_became_active":"2023-10-16T10:05:10.380134+0000","last_became_peered":"2023-10-16T10:05:10.380134+0000","last_unstale":"2023-10-16T10:05:11.416579+0000","last_undegraded":"2023-10-16T10:05:11.416579+0000","last_fullsized":"2023-10-16T10:05:11.416579+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:22:28.852480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.397541+0000","last_change":"2023-10-16T10:05:11.391498+0000","last_active":"2023-10-16T10:05:12.397541+0000","last_peered":"2023-10-16T10:05:12.397541+0000","last_clean":"2023-10-16T10:05:12.397541+0000","last_became_active":"2023-10-16T10:05:11.391176+0000","last_became_peered":"2023-10-16T10:05:11.391176+0000","last_unstale":"2023-10-16T10:05:12.397541+0000","last_undegraded":"2023-10-16T10:05:12.397541+0000","last_fullsized":"2023-10-16T10:05:12.397541+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:57.685307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.406557+0000","last_change":"2023-10-16T10:05:10.384125+0000","last_active":"2023-10-16T10:05:11.406557+0000","last_peered":"2023-10-16T10:05:11.406557+0000","last_clean":"2023-10-16T10:05:11.406557+0000","last_became_active":"2023-10-16T10:05:10.383816+0000","last_became_peered":"2023-10-16T10:05:10.383816+0000","last_unstale":"2023-10-16T10:05:11.406557+0000","last_undegraded":"2023-10-16T10:05:11.406557+0000","last_fullsized":"2023-10-16T10:05:11.406557+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:43:09.223869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.398319+0000","last_change":"2023-10-16T10:05:11.394052+0000","last_active":"2023-10-16T10:05:11.398319+0000","last_peered":"2023-10-16T10:05:11.398319+0000","last_clean":"2023-10-16T10:05:11.398319+0000","last_became_active":"2023-10-16T10:05:11.393768+0000","last_became_peered":"2023-10-16T10:05:11.393768+0000","last_unstale":"2023-10-16T10:05:11.398319+0000","last_undegraded":"2023-10-16T10:05:11.398319+0000","last_fullsized":"2023-10-16T10:05:11.398319+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:24:13.019540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.383904+0000","last_change":"2023-10-16T10:05:10.375863+0000","last_active":"2023-10-16T10:05:11.383904+0000","last_peered":"2023-10-16T10:05:11.383904+0000","last_clean":"2023-10-16T10:05:11.383904+0000","last_became_active":"2023-10-16T10:05:10.372876+0000","last_became_peered":"2023-10-16T10:05:10.372876+0000","last_unstale":"2023-10-16T10:05:11.383904+0000","last_undegraded":"2023-10-16T10:05:11.383904+0000","last_fullsized":"2023-10-16T10:05:11.383904+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:05.484396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.400533+0000","last_change":"2023-10-16T10:05:11.399178+0000","last_active":"2023-10-16T10:05:11.400533+0000","last_peered":"2023-10-16T10:05:11.400533+0000","last_clean":"2023-10-16T10:05:11.400533+0000","last_became_active":"2023-10-16T10:05:11.398628+0000","last_became_peered":"2023-10-16T10:05:11.398628+0000","last_unstale":"2023-10-16T10:05:11.400533+0000","last_undegraded":"2023-10-16T10:05:11.400533+0000","last_fullsized":"2023-10-16T10:05:11.400533+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:03.454542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.394321+0000","last_change":"2023-10-16T10:05:10.374041+0000","last_active":"2023-10-16T10:05:11.394321+0000","last_peered":"2023-10-16T10:05:11.394321+0000","last_clean":"2023-10-16T10:05:11.394321+0000","last_became_active":"2023-10-16T10:05:10.373753+0000","last_became_peered":"2023-10-16T10:05:10.373753+0000","last_unstale":"2023-10-16T10:05:11.394321+0000","last_undegraded":"2023-10-16T10:05:11.394321+0000","last_fullsized":"2023-10-16T10:05:11.394321+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:30:42.712451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.408815+0000","last_change":"2023-10-16T10:05:11.407798+0000","last_active":"2023-10-16T10:05:11.408815+0000","last_peered":"2023-10-16T10:05:11.408815+0000","last_clean":"2023-10-16T10:05:11.408815+0000","last_became_active":"2023-10-16T10:05:11.407406+0000","last_became_peered":"2023-10-16T10:05:11.407406+0000","last_unstale":"2023-10-16T10:05:11.408815+0000","last_undegraded":"2023-10-16T10:05:11.408815+0000","last_fullsized":"2023-10-16T10:05:11.408815+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:27:29.884109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.382740+0000","last_change":"2023-10-16T10:05:10.369914+0000","last_active":"2023-10-16T10:05:11.382740+0000","last_peered":"2023-10-16T10:05:11.382740+0000","last_clean":"2023-10-16T10:05:11.382740+0000","last_became_active":"2023-10-16T10:05:10.369624+0000","last_became_peered":"2023-10-16T10:05:10.369624+0000","last_unstale":"2023-10-16T10:05:11.382740+0000","last_undegraded":"2023-10-16T10:05:11.382740+0000","last_fullsized":"2023-10-16T10:05:11.382740+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:54:36.927537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.399429+0000","last_change":"2023-10-16T10:05:11.386939+0000","last_active":"2023-10-16T10:05:12.399429+0000","last_peered":"2023-10-16T10:05:12.399429+0000","last_clean":"2023-10-16T10:05:12.399429+0000","last_became_active":"2023-10-16T10:05:11.386672+0000","last_became_peered":"2023-10-16T10:05:11.386672+0000","last_unstale":"2023-10-16T10:05:12.399429+0000","last_undegraded":"2023-10-16T10:05:12.399429+0000","last_fullsized":"2023-10-16T10:05:12.399429+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:33:51.440378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.416627+0000","last_change":"2023-10-16T10:05:10.373636+0000","last_active":"2023-10-16T10:05:11.416627+0000","last_peered":"2023-10-16T10:05:11.416627+0000","last_clean":"2023-10-16T10:05:11.416627+0000","last_became_active":"2023-10-16T10:05:10.373364+0000","last_became_peered":"2023-10-16T10:05:10.373364+0000","last_unstale":"2023-10-16T10:05:11.416627+0000","last_undegraded":"2023-10-16T10:05:11.416627+0000","last_fullsized":"2023-10-16T10:05:11.416627+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:21:39.317846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.083423+0000","last_change":"2023-10-16T10:05:11.407988+0000","last_active":"2023-10-16T10:05:13.083423+0000","last_peered":"2023-10-16T10:05:13.083423+0000","last_clean":"2023-10-16T10:05:13.083423+0000","last_became_active":"2023-10-16T10:05:11.406715+0000","last_became_peered":"2023-10-16T10:05:11.406715+0000","last_unstale":"2023-10-16T10:05:13.083423+0000","last_undegraded":"2023-10-16T10:05:13.083423+0000","last_fullsized":"2023-10-16T10:05:13.083423+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:59:03.669248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.399291+0000","last_change":"2023-10-16T10:05:10.372440+0000","last_active":"2023-10-16T10:05:11.399291+0000","last_peered":"2023-10-16T10:05:11.399291+0000","last_clean":"2023-10-16T10:05:11.399291+0000","last_became_active":"2023-10-16T10:05:10.372047+0000","last_became_peered":"2023-10-16T10:05:10.372047+0000","last_unstale":"2023-10-16T10:05:11.399291+0000","last_undegraded":"2023-10-16T10:05:11.399291+0000","last_fullsized":"2023-10-16T10:05:11.399291+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:51:21.096892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.323679+0000","last_change":"2023-10-16T10:05:11.418525+0000","last_active":"2023-10-16T10:05:14.323679+0000","last_peered":"2023-10-16T10:05:14.323679+0000","last_clean":"2023-10-16T10:05:14.323679+0000","last_became_active":"2023-10-16T10:05:11.418237+0000","last_became_peered":"2023-10-16T10:05:11.418237+0000","last_unstale":"2023-10-16T10:05:14.323679+0000","last_undegraded":"2023-10-16T10:05:14.323679+0000","last_fullsized":"2023-10-16T10:05:14.323679+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:51.860331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.374447+0000","last_change":"2023-10-16T10:05:10.369414+0000","last_active":"2023-10-16T10:05:11.374447+0000","last_peered":"2023-10-16T10:05:11.374447+0000","last_clean":"2023-10-16T10:05:11.374447+0000","last_became_active":"2023-10-16T10:05:10.369199+0000","last_became_peered":"2023-10-16T10:05:10.369199+0000","last_unstale":"2023-10-16T10:05:11.374447+0000","last_undegraded":"2023-10-16T10:05:11.374447+0000","last_fullsized":"2023-10-16T10:05:11.374447+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:56.563797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:14.323473+0000","last_change":"2023-10-16T10:05:11.418473+0000","last_active":"2023-10-16T10:05:14.323473+0000","last_peered":"2023-10-16T10:05:14.323473+0000","last_clean":"2023-10-16T10:05:14.323473+0000","last_became_active":"2023-10-16T10:05:11.418166+0000","last_became_peered":"2023-10-16T10:05:11.418166+0000","last_unstale":"2023-10-16T10:05:14.323473+0000","last_undegraded":"2023-10-16T10:05:14.323473+0000","last_fullsized":"2023-10-16T10:05:14.323473+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:30:16.921858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.381389+0000","last_change":"2023-10-16T10:05:10.384003+0000","last_active":"2023-10-16T10:05:11.381389+0000","last_peered":"2023-10-16T10:05:11.381389+0000","last_clean":"2023-10-16T10:05:11.381389+0000","last_became_active":"2023-10-16T10:05:10.383804+0000","last_became_peered":"2023-10-16T10:05:10.383804+0000","last_unstale":"2023-10-16T10:05:11.381389+0000","last_undegraded":"2023-10-16T10:05:11.381389+0000","last_fullsized":"2023-10-16T10:05:11.381389+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:31.768643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:12.397327+0000","last_change":"2023-10-16T10:05:11.396702+0000","last_active":"2023-10-16T10:05:12.397327+0000","last_peered":"2023-10-16T10:05:12.397327+0000","last_clean":"2023-10-16T10:05:12.397327+0000","last_became_active":"2023-10-16T10:05:11.396343+0000","last_became_peered":"2023-10-16T10:05:11.396343+0000","last_unstale":"2023-10-16T10:05:12.397327+0000","last_undegraded":"2023-10-16T10:05:12.397327+0000","last_fullsized":"2023-10-16T10:05:12.397327+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:33:51.440378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.374190+0000","last_change":"2023-10-16T10:05:10.376744+0000","last_active":"2023-10-16T10:05:11.374190+0000","last_peered":"2023-10-16T10:05:11.374190+0000","last_clean":"2023-10-16T10:05:11.374190+0000","last_became_active":"2023-10-16T10:05:10.376146+0000","last_became_peered":"2023-10-16T10:05:10.376146+0000","last_unstale":"2023-10-16T10:05:11.374190+0000","last_undegraded":"2023-10-16T10:05:11.374190+0000","last_fullsized":"2023-10-16T10:05:11.374190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:54:36.927537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.197882+0000","last_change":"2023-10-16T10:05:11.416337+0000","last_active":"2023-10-16T10:05:13.197882+0000","last_peered":"2023-10-16T10:05:13.197882+0000","last_clean":"2023-10-16T10:05:13.197882+0000","last_became_active":"2023-10-16T10:05:11.415777+0000","last_became_peered":"2023-10-16T10:05:11.415777+0000","last_unstale":"2023-10-16T10:05:13.197882+0000","last_undegraded":"2023-10-16T10:05:13.197882+0000","last_fullsized":"2023-10-16T10:05:13.197882+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:07.913032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.380283+0000","last_change":"2023-10-16T10:05:10.376543+0000","last_active":"2023-10-16T10:05:11.380283+0000","last_peered":"2023-10-16T10:05:11.380283+0000","last_clean":"2023-10-16T10:05:11.380283+0000","last_became_active":"2023-10-16T10:05:10.376224+0000","last_became_peered":"2023-10-16T10:05:10.376224+0000","last_unstale":"2023-10-16T10:05:11.380283+0000","last_undegraded":"2023-10-16T10:05:11.380283+0000","last_fullsized":"2023-10-16T10:05:11.380283+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:08:26.700080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.401740+0000","last_change":"2023-10-16T10:05:11.398956+0000","last_active":"2023-10-16T10:05:11.401740+0000","last_peered":"2023-10-16T10:05:11.401740+0000","last_clean":"2023-10-16T10:05:11.401740+0000","last_became_active":"2023-10-16T10:05:11.398700+0000","last_became_peered":"2023-10-16T10:05:11.398700+0000","last_unstale":"2023-10-16T10:05:11.401740+0000","last_undegraded":"2023-10-16T10:05:11.401740+0000","last_fullsized":"2023-10-16T10:05:11.401740+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:41.583697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.378203+0000","last_change":"2023-10-16T10:05:10.384152+0000","last_active":"2023-10-16T10:05:11.378203+0000","last_peered":"2023-10-16T10:05:11.378203+0000","last_clean":"2023-10-16T10:05:11.378203+0000","last_became_active":"2023-10-16T10:05:10.383688+0000","last_became_peered":"2023-10-16T10:05:10.383688+0000","last_unstale":"2023-10-16T10:05:11.378203+0000","last_undegraded":"2023-10-16T10:05:11.378203+0000","last_fullsized":"2023-10-16T10:05:11.378203+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:46:13.198009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-10-16T10:05:13.198248+0000","last_change":"2023-10-16T10:05:11.404146+0000","last_active":"2023-10-16T10:05:13.198248+0000","last_peered":"2023-10-16T10:05:13.198248+0000","last_clean":"2023-10-16T10:05:13.198248+0000","last_became_active":"2023-10-16T10:05:11.403805+0000","last_became_peered":"2023-10-16T10:05:11.403805+0000","last_unstale":"2023-10-16T10:05:13.198248+0000","last_undegraded":"2023-10-16T10:05:13.198248+0000","last_fullsized":"2023-10-16T10:05:13.198248+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:54:37.944025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.382300+0000","last_change":"2023-10-16T10:05:10.374754+0000","last_active":"2023-10-16T10:05:11.382300+0000","last_peered":"2023-10-16T10:05:11.382300+0000","last_clean":"2023-10-16T10:05:11.382300+0000","last_became_active":"2023-10-16T10:05:10.374447+0000","last_became_peered":"2023-10-16T10:05:10.374447+0000","last_unstale":"2023-10-16T10:05:11.382300+0000","last_undegraded":"2023-10-16T10:05:11.382300+0000","last_fullsized":"2023-10-16T10:05:11.382300+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:32:26.229914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.400829+0000","last_change":"2023-10-16T10:05:11.398995+0000","last_active":"2023-10-16T10:05:11.400829+0000","last_peered":"2023-10-16T10:05:11.400829+0000","last_clean":"2023-10-16T10:05:11.400829+0000","last_became_active":"2023-10-16T10:05:11.398696+0000","last_became_peered":"2023-10-16T10:05:11.398696+0000","last_unstale":"2023-10-16T10:05:11.400829+0000","last_undegraded":"2023-10-16T10:05:11.400829+0000","last_fullsized":"2023-10-16T10:05:11.400829+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:10.335544+0000","last_clean_scrub_stamp":"2023-10-16T10:05:10.335544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:43:10.240357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.416536+0000","last_change":"2023-10-16T10:05:10.366612+0000","last_active":"2023-10-16T10:05:11.416536+0000","last_peered":"2023-10-16T10:05:11.416536+0000","last_clean":"2023-10-16T10:05:11.416536+0000","last_became_active":"2023-10-16T10:05:10.366310+0000","last_became_peered":"2023-10-16T10:05:10.366310+0000","last_unstale":"2023-10-16T10:05:11.416536+0000","last_undegraded":"2023-10-16T10:05:11.416536+0000","last_fullsized":"2023-10-16T10:05:11.416536+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:09.319056+0000","last_clean_scrub_stamp":"2023-10-16T10:05:09.319056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:06.570078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"84'173","reported_seq":436,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-16T10:05:11.400553+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T10:05:11.400553+0000","last_peered":"2023-10-16T10:05:11.400553+0000","last_clean":"2023-10-16T10:05:11.400553+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T10:05:11.400553+0000","last_undegraded":"2023-10-16T10:05:11.400553+0000","last_fullsized":"2023-10-16T10:05:11.400553+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":600,"num_read_kb":1133,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":600,"num_read_kb":1133,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":173,"ondisk_log_size":173,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575043,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37056,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334784,"statfs":{"total":96636764160,"available":96598818816,"internally_reserved":0,"allocated":1748992,"data_stored":1136691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1639999999999999,"15min":1.167},"min":{"1min":0.65000000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.6599999999999999,"5min":1.853,"15min":1.998},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1719999999999999,"15min":1.208},"min":{"1min":0.58099999999999996,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.5629999999999999,"5min":1.75,"15min":3.694},"last":0.59199999999999997}]},{"osd":1,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.222,"15min":1.2290000000000001},"min":{"1min":0.63300000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.579,"5min":1.923,"15min":1.923},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.147,"5min":1.234,"15min":1.22},"min":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5309999999999999,"5min":1.704,"15min":1.7410000000000001},"last":0.749}]},{"osd":2,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2,"15min":1.2490000000000001},"min":{"1min":0.89000000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5820000000000001,"5min":1.837,"15min":1.885},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.284,"15min":1.306},"min":{"1min":0.81499999999999995,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.7430000000000001,"5min":1.8620000000000001,"15min":3.7759999999999998},"last":0.91300000000000003}]},{"osd":3,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.3120000000000001,"15min":1.3380000000000001},"min":{"1min":0.93500000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5389999999999999,"5min":1.996,"15min":1.996},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.3600000000000001,"15min":1.3859999999999999},"min":{"1min":1.0509999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6659999999999999,"5min":1.8759999999999999,"15min":2.141},"last":1.347}]},{"osd":4,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.367,"15min":1.3759999999999999},"min":{"1min":1.004,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7829999999999999,"5min":2.028,"15min":2.0310000000000001},"last":1.419},{"interface":"front","average":{"1min":1.274,"5min":1.3120000000000001,"15min":1.3200000000000001},"min":{"1min":0.79200000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.55,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.98899999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4690000000000001,"15min":1.5209999999999999},"min":{"1min":1.111,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.77,"5min":2.1320000000000001,"15min":2.4510000000000001},"last":1.478},{"interface":"front","average":{"1min":1.403,"5min":1.4379999999999999,"15min":1.4410000000000001},"min":{"1min":0.92200000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6899999999999999,"5min":2.2170000000000001,"15min":2.3210000000000002},"last":1.1080000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.413,"15min":1.4159999999999999},"min":{"1min":1.054,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.659,"5min":1.946,"15min":2.6259999999999999},"last":1.175},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.6040000000000001,"15min":1.623},"min":{"1min":1.21,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.986,"5min":2.3799999999999999,"15min":2.5670000000000002},"last":1.544}]}]},{"osd":6,"up_from":13,"seq":55834575043,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335360,"statfs":{"total":96636764160,"available":96599408640,"internally_reserved":0,"allocated":1167360,"data_stored":552099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.127,"15min":1.147},"min":{"1min":0.75900000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4790000000000001,"5min":1.6910000000000001,"15min":2.04},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.236,"15min":1.246},"min":{"1min":0.98599999999999999,"5min":0.65900000000000003,"15min":0.624},"max":{"1min":1.641,"5min":1.847,"15min":2.1000000000000001},"last":1.175}]},{"osd":1,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1240000000000001,"15min":1.1779999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.9259999999999999},"last":1.351},{"interface":"front","average":{"1min":1.292,"5min":1.274,"15min":1.3009999999999999},"min":{"1min":0.91900000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.288}]},{"osd":2,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.169,"15min":1.2010000000000001},"min":{"1min":0.745,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":2.036},"last":1.851},{"interface":"front","average":{"1min":1.238,"5min":1.288,"15min":1.3200000000000001},"min":{"1min":0.79900000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.925,"5min":1.925,"15min":2.1720000000000002},"last":0.90300000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.228,"15min":1.2390000000000001},"min":{"1min":0.69599999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.742,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1859999999999999,"15min":1.234},"min":{"1min":0.83399999999999996,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6020000000000001,"5min":1.6359999999999999,"15min":2.0640000000000001},"last":1.637}]},{"osd":4,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.196,"15min":1.2849999999999999},"min":{"1min":0.77800000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.794,"5min":2.1629999999999998,"15min":2.4540000000000002},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4299999999999999,"15min":1.45},"min":{"1min":0.77700000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.901,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.5860000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.276,"15min":1.323},"min":{"1min":0.80400000000000005,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.603,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.956},{"interface":"front","average":{"1min":1.544,"5min":1.482,"15min":1.5229999999999999},"min":{"1min":1.3089999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.865,"5min":2.3849999999999998,"15min":2.5289999999999999},"last":1.7350000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.456,"15min":1.4890000000000001},"min":{"1min":0.871,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.843,"5min":3.5739999999999998,"15min":3.5739999999999998},"last":1.79},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.431,"15min":1.5},"min":{"1min":0.92600000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8340000000000001,"5min":2.4249999999999998,"15min":3.5419999999999998},"last":2.0510000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575043,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36476,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335364,"statfs":{"total":96636764160,"available":96599412736,"internally_reserved":0,"allocated":1163264,"data_stored":552097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.1399999999999999,"15min":1.214},"min":{"1min":0.63100000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.486,"5min":1.675,"15min":2.1019999999999999},"last":1.663},{"interface":"front","average":{"1min":1.272,"5min":1.2789999999999999,"15min":1.3029999999999999},"min":{"1min":0.72399999999999998,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.724,"5min":1.8600000000000001,"15min":1.915},"last":1.4690000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.135,"15min":1.149},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.629,"5min":1.7729999999999999,"15min":2.4569999999999999},"last":1.026},{"interface":"front","average":{"1min":1.179,"5min":1.1279999999999999,"15min":1.1559999999999999},"min":{"1min":0.84299999999999997,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.526,"5min":1.6970000000000001,"15min":2.2410000000000001},"last":1.363}]},{"osd":2,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.157,"15min":1.163},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71299999999999997},"max":{"1min":1.675,"5min":1.7470000000000001,"15min":1.77},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.228,"15min":1.246},"min":{"1min":0.83499999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.5780000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.5569999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.288,"15min":1.2869999999999999},"min":{"1min":0.81699999999999995,"5min":0.76000000000000001,"15min":0.73799999999999999},"max":{"1min":1.8839999999999999,"5min":1.97,"15min":1.97},"last":1.129},{"interface":"front","average":{"1min":1.298,"5min":1.355,"15min":1.3660000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.623,"5min":1.7989999999999999,"15min":2.1160000000000001},"last":1.3129999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.2529999999999999,"15min":1.2949999999999999},"min":{"1min":0.79700000000000004,"5min":0.77100000000000002,"15min":0.52100000000000002},"max":{"1min":1.569,"5min":1.7989999999999999,"15min":2.355},"last":1.196},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4990000000000001,"15min":1.5229999999999999},"min":{"1min":0.91700000000000004,"5min":0.89500000000000002,"15min":0.63600000000000001},"max":{"1min":1.8069999999999999,"5min":2.0920000000000001,"15min":2.4079999999999999},"last":1.804}]},{"osd":6,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.3740000000000001,"15min":1.379},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.55500000000000005},"max":{"1min":1.726,"5min":1.847,"15min":2.3580000000000001},"last":1.417},{"interface":"front","average":{"1min":1.375,"5min":1.3839999999999999,"15min":1.4099999999999999},"min":{"1min":1.151,"5min":0.83299999999999996,"15min":0.58399999999999996},"max":{"1min":1.6120000000000001,"5min":1.889,"15min":2.4159999999999999},"last":1.0680000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:04:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.478,"15min":1.5109999999999999},"min":{"1min":1.0669999999999999,"5min":0.88600000000000001,"15min":0.78700000000000003},"max":{"1min":1.913,"5min":2.1989999999999998,"15min":2.4809999999999999},"last":1.714},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.575,"15min":1.5960000000000001},"min":{"1min":1.2509999999999999,"5min":0.99199999999999999,"15min":0.75600000000000001},"max":{"1min":1.851,"5min":1.9430000000000001,"15min":2.931},"last":1.8520000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575044,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46208,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325632,"statfs":{"total":96636764160,"available":96589447168,"internally_reserved":0,"allocated":1163264,"data_stored":552077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.107,"15min":1.163},"min":{"1min":0.74199999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.554,"5min":1.76,"15min":4.2640000000000002},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.3460000000000001,"15min":1.252},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.756,"5min":11.974,"15min":11.974},"last":1.6120000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.3580000000000001,"15min":1.2809999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.7549999999999999,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.2130000000000001,"15min":1.24},"min":{"1min":0.67700000000000005,"5min":0.63600000000000001,"15min":0.624},"max":{"1min":1.5309999999999999,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.3740000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.254,"15min":1.2490000000000001},"min":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.75,"5min":1.756,"15min":2.3130000000000002},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.377,"15min":1.3799999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.615,"5min":5.0549999999999997,"15min":5.0549999999999997},"last":1.2390000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3160000000000001,"15min":1.3129999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.6739999999999999,"5min":6.8079999999999998,"15min":6.8079999999999998},"last":1.165},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.3759999999999999,"15min":1.395},"min":{"1min":0.93100000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.885,"5min":1.923,"15min":3.3050000000000002},"last":1.1020000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.498,"15min":1.423},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9590000000000001,"5min":8.702,"15min":8.702},"last":1.536},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.69099999999999995},"max":{"1min":1.754,"5min":3.0430000000000001,"15min":3.669},"last":1.2889999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.4810000000000001,"15min":1.4299999999999999},"min":{"1min":1.0049999999999999,"5min":0.80700000000000005,"15min":0.56599999999999995},"max":{"1min":1.9850000000000001,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.534,"5min":1.605,"15min":1.5509999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.70299999999999996},"max":{"1min":2.1179999999999999,"5min":6.4089999999999998,"15min":6.4089999999999998},"last":1.667}]},{"osd":7,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.508,"15min":1.5029999999999999},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.60999999999999999},"max":{"1min":2.177,"5min":2.9399999999999999,"15min":3.5150000000000001},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.631,"5min":1.585,"15min":1.5860000000000001},"min":{"1min":1.1220000000000001,"5min":0.98399999999999999,"15min":0.89600000000000002},"max":{"1min":2.0539999999999998,"5min":2.6120000000000001,"15min":3.9849999999999999},"last":1.3340000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575043,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36476,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335364,"statfs":{"total":96636764160,"available":96599412736,"internally_reserved":0,"allocated":1155072,"data_stored":545819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.83099999999999996,"15min":0.85999999999999999},"min":{"1min":0.60799999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4339999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.95599999999999996,"15min":1.004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.29999999999999999},"max":{"1min":1.385,"5min":1.708,"15min":1.708},"last":0.66600000000000004}]},{"osd":1,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.91200000000000003,"15min":0.94299999999999995},"min":{"1min":0.42799999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.2210000000000001,"5min":1.49,"15min":1.7150000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.006,"15min":1.0509999999999999},"min":{"1min":0.52900000000000003,"5min":0.42099999999999999,"15min":0.39600000000000002},"max":{"1min":1.4079999999999999,"5min":1.603,"15min":1.643},"last":0.81499999999999995}]},{"osd":2,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":1.0149999999999999,"15min":1.0580000000000001},"min":{"1min":0.46500000000000002,"5min":0.44800000000000001,"15min":0.439},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.8640000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.141,"15min":1.163},"min":{"1min":0.55100000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.534,"5min":1.776,"15min":2.028},"last":0.90300000000000002}]},{"osd":4,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.24,"15min":1.276},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.5740000000000001,"5min":1.8069999999999999,"15min":2.1619999999999999},"last":1.202},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.296,"15min":1.3220000000000001},"min":{"1min":0.75600000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.6779999999999999,"5min":1.8500000000000001,"15min":2.532},"last":1.29}]},{"osd":5,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.278,"15min":1.3169999999999999},"min":{"1min":0.77800000000000002,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.5629999999999999,"5min":1.7350000000000001,"15min":2.0019999999999998},"last":1.542},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3029999999999999,"15min":1.3460000000000001},"min":{"1min":0.91200000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.5589999999999999,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":1.319}]},{"osd":6,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.375,"15min":1.429},"min":{"1min":0.72099999999999997,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6799999999999999,"5min":1.891,"15min":2.3029999999999999},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3999999999999999,"15min":1.4410000000000001},"min":{"1min":0.88300000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.619,"5min":1.976,"15min":2.1240000000000001},"last":1.373}]},{"osd":7,"last update":"Mon Oct 16 10:04:27 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3959999999999999,"15min":1.4490000000000001},"min":{"1min":0.73199999999999998,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7450000000000001,"5min":2.032,"15min":2.403},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.504,"15min":1.5640000000000001},"min":{"1min":0.68600000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8320000000000001,"5min":2.2429999999999999,"15min":2.6099999999999999},"last":1.581}]}]},{"osd":2,"up_from":13,"seq":55834575042,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40652,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331188,"statfs":{"total":96636764160,"available":96595136512,"internally_reserved":0,"allocated":1171456,"data_stored":553619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.875,"15min":0.86699999999999999},"min":{"1min":0.499,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.506,"5min":1.5469999999999999,"15min":2.8690000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":1,"5min":0.98099999999999998,"15min":1.028},"min":{"1min":0.52100000000000002,"5min":0.34999999999999998,"15min":0.28899999999999998},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.155}]},{"osd":1,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.044,"15min":1.036},"min":{"1min":0.56999999999999995,"5min":0.52900000000000003,"15min":0.42899999999999999},"max":{"1min":1.5329999999999999,"5min":1.8080000000000001,"15min":2.6059999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0629999999999999,"15min":1.1140000000000001},"min":{"1min":0.622,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4770000000000001,"5min":1.792,"15min":2.6499999999999999},"last":1.3999999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":1.006,"15min":1.0089999999999999},"min":{"1min":0.55400000000000005,"5min":0.41399999999999998,"15min":0.38},"max":{"1min":1.3300000000000001,"5min":1.776,"15min":2.5339999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.226,"5min":1.169,"15min":1.1970000000000001},"min":{"1min":0.59799999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.7969999999999999,"5min":1.8,"15min":2.9700000000000002},"last":0.88700000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.3420000000000001,"15min":1.3480000000000001},"min":{"1min":1.083,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":2.2309999999999999},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3240000000000001,"15min":1.302},"min":{"1min":0.97899999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.2000000000000002},"last":1.2030000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3600000000000001,"15min":1.373},"min":{"1min":1.0249999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.9810000000000001,"5min":2.1640000000000001,"15min":2.7770000000000001},"last":1.361},{"interface":"front","average":{"1min":1.462,"5min":1.391,"15min":1.415},"min":{"1min":0.72699999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.885,"5min":1.956,"15min":2.952},"last":1.341}]},{"osd":6,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3400000000000001,"15min":1.3360000000000001},"min":{"1min":0.89100000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.831,"5min":2.1970000000000001,"15min":2.2469999999999999},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.4330000000000001,"15min":1.4410000000000001},"min":{"1min":1.1180000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.8140000000000001,"5min":2.044,"15min":2.8180000000000001},"last":1.2509999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.4630000000000001,"15min":1.4850000000000001},"min":{"1min":1.0109999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.012,"5min":2.0739999999999998,"15min":3.746},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.411,"15min":1.431},"min":{"1min":1.085,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.9279999999999999,"5min":2.3109999999999999,"15min":2.3769999999999998},"last":1.5329999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575042,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50372,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321468,"statfs":{"total":96636764160,"available":96585183232,"internally_reserved":0,"allocated":1167360,"data_stored":547935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.83399999999999996,"15min":0.88},"min":{"1min":0.67400000000000004,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2909999999999999,"5min":1.363,"15min":2.2010000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.087,"5min":0.99099999999999999,"15min":0.998},"min":{"1min":0.66100000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.6200000000000001,"5min":1.798,"15min":2.1619999999999999},"last":0.76600000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.98199999999999998,"15min":0.995},"min":{"1min":0.60399999999999998,"5min":0.45300000000000001,"15min":0.42299999999999999},"max":{"1min":1.3700000000000001,"5min":1.526,"15min":2.3849999999999998},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2430000000000001},"min":{"1min":0.70199999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5669999999999999,"5min":1.667,"15min":2.2959999999999998},"last":0.46000000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.113,"15min":1.143},"min":{"1min":0.70199999999999996,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":2.254},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.141,"5min":1.0660000000000001,"15min":1.1060000000000001},"min":{"1min":0.83299999999999996,"5min":0.40699999999999997,"15min":0.36099999999999999},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":2.8109999999999999},"last":0.91800000000000004}]},{"osd":4,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.206,"15min":1.2450000000000001},"min":{"1min":0.84099999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5189999999999999,"5min":1.7010000000000001,"15min":2.4540000000000002},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3069999999999999,"15min":1.3799999999999999},"min":{"1min":1.0309999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5389999999999999,"5min":1.7789999999999999,"15min":2.851},"last":1.0209999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3129999999999999,"15min":1.345},"min":{"1min":1.155,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5820000000000001,"5min":1.7889999999999999,"15min":2.7120000000000002},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.298,"15min":1.335},"min":{"1min":1.034,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.7290000000000001,"5min":1.7869999999999999,"15min":2.512},"last":0.98399999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.385,"15min":1.4099999999999999},"min":{"1min":1.0760000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6830000000000001,"5min":1.7649999999999999,"15min":2.75},"last":1.236},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.3620000000000001,"15min":1.401},"min":{"1min":1.014,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.8029999999999999,"5min":1.851,"15min":2.669},"last":0.80800000000000005}]},{"osd":7,"last update":"Mon Oct 16 10:04:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4339999999999999,"15min":1.494},"min":{"1min":1.1619999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.784,"5min":1.8799999999999999,"15min":2.8799999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.4910000000000001,"15min":1.506},"min":{"1min":1.2589999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":3.1400000000000001},"last":1.1659999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575043,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32892,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338948,"statfs":{"total":96636764160,"available":96603082752,"internally_reserved":0,"allocated":1753088,"data_stored":1142375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.86899999999999999,"15min":0.85699999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.585,"5min":2.766,"15min":2.766},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.998,"15min":1.03},"min":{"1min":0.44900000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.5720000000000001,"5min":3.96,"15min":3.96},"last":0.73499999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0660000000000001,"15min":1.0429999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.38900000000000001},"max":{"1min":1.526,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.048,"5min":1.107,"15min":1.089},"min":{"1min":0.50700000000000001,"5min":0.48999999999999999,"15min":0.46600000000000003},"max":{"1min":1.722,"5min":4.4290000000000003,"15min":4.4290000000000003},"last":0.86299999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.054,"15min":1.032},"min":{"1min":0.495,"5min":0.38,"15min":0.38},"max":{"1min":1.498,"5min":4.7380000000000004,"15min":4.7380000000000004},"last":1.032},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.24,"15min":1.202},"min":{"1min":0.47099999999999997,"5min":0.44,"15min":0.44},"max":{"1min":2.1859999999999999,"5min":5.1500000000000004,"15min":5.1500000000000004},"last":1.278}]},{"osd":4,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.341,"15min":1.3340000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.859,"5min":4.1970000000000001,"15min":4.1970000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.3049999999999999,"15min":1.2909999999999999},"min":{"1min":0.77900000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.623,"5min":4.9630000000000001,"15min":4.9630000000000001},"last":1.2450000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3280000000000001,"15min":1.3220000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6799999999999999,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.399,"5min":1.4239999999999999,"15min":1.407},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.1269999999999998,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":1.1619999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.464,"15min":1.4390000000000001},"min":{"1min":0.98099999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0739999999999998,"5min":3.8250000000000002,"15min":3.8250000000000002},"last":1.401},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.444,"15min":1.4259999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.84999999999999998},"max":{"1min":2.02,"5min":4.0979999999999999,"15min":4.0979999999999999},"last":1.2090000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:04:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.49,"15min":1.4750000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7929999999999999,"5min":4.3369999999999997,"15min":4.3369999999999997},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4610000000000001,"15min":1.4359999999999999},"min":{"1min":0.83599999999999997,"5min":0.755,"15min":0.755},"max":{"1min":1.8169999999999999,"5min":5.0570000000000004,"15min":5.0570000000000004},"last":1.3680000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:05:16.362 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T10:05:16.363 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T10:05:16.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.617+0000 7fea2541f640 1 Processor -- start 2023-10-16T10:05:16.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.617+0000 7fea2541f640 1 -- start start 2023-10-16T10:05:16.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.618+0000 7fea2541f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fea200adbe0 0x7fea200cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:16.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.619+0000 7fea2541f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea200adfc0 0x7fea200b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:16.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.619+0000 7fea2541f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea200cc600 con 0x7fea20070fd0 2023-10-16T10:05:16.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.619+0000 7fea2541f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea20071470 con 0x7fea200adbe0 2023-10-16T10:05:16.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.619+0000 7fea2541f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea200cc870 con 0x7fea200adfc0 2023-10-16T10:05:16.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.619+0000 7fea1e7fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fea20070fd0 0x7fea2006f550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:37688/0 (socket says 172.21.15.84:37688) 2023-10-16T10:05:16.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.619+0000 7fea1e7fc640 1 -- 172.21.15.84:0/3884846528 learned_addr learned my addr 172.21.15.84:0/3884846528 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:16.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.619+0000 7fea1effd640 1 --2- 172.21.15.84:0/3884846528 >> v2:172.21.15.157:3300/0 conn(0x7fea200adbe0 0x7fea200cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:16.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.620+0000 7fea1f7fe640 1 --2- 172.21.15.84:0/3884846528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea200adfc0 0x7fea200b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:16.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.620+0000 7fea1effd640 1 -- 172.21.15.84:0/3884846528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea200adfc0 msgr2=0x7fea200b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:16.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.620+0000 7fea1effd640 1 --2- 172.21.15.84:0/3884846528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea200adfc0 0x7fea200b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.620+0000 7fea1effd640 1 -- 172.21.15.84:0/3884846528 >> v1:172.21.15.157:6789/0 conn(0x7fea20070fd0 legacy=0x7fea2006f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:16.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.620+0000 7fea1effd640 1 -- 172.21.15.84:0/3884846528 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea200ccb50 con 0x7fea200adbe0 2023-10-16T10:05:16.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.622+0000 7fea1effd640 1 --2- 172.21.15.84:0/3884846528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea200adbe0 0x7fea200cd070 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fea100097a0 tx=0x7fea1002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c589c8d79b9dafe8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:16.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.622+0000 7fea1dffb640 1 -- 172.21.15.84:0/3884846528 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea10031830 con 0x7fea200adbe0 2023-10-16T10:05:16.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.622+0000 7fea1dffb640 1 -- 172.21.15.84:0/3884846528 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea10031990 con 0x7fea200adbe0 2023-10-16T10:05:16.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.622+0000 7fea1dffb640 1 -- 172.21.15.84:0/3884846528 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea10031d20 con 0x7fea200adbe0 2023-10-16T10:05:16.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.623+0000 7fea2541f640 1 -- 172.21.15.84:0/3884846528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea200adbe0 msgr2=0x7fea200cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:16.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.623+0000 7fea2541f640 1 --2- 172.21.15.84:0/3884846528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea200adbe0 0x7fea200cd070 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fea100097a0 tx=0x7fea1002b8d0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.623+0000 7fea2541f640 1 -- 172.21.15.84:0/3884846528 shutdown_connections 2023-10-16T10:05:16.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.623+0000 7fea2541f640 1 --2- 172.21.15.84:0/3884846528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea200adfc0 0x7fea200b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.623+0000 7fea2541f640 1 --2- 172.21.15.84:0/3884846528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea200adbe0 0x7fea200cd070 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.623+0000 7fea2541f640 1 -- 172.21.15.84:0/3884846528 >> 172.21.15.84:0/3884846528 conn(0x7fea200cb410 msgr2=0x7fea200cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:16.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.623+0000 7fea2541f640 1 -- 172.21.15.84:0/3884846528 shutdown_connections 2023-10-16T10:05:16.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.623+0000 7fea2541f640 1 -- 172.21.15.84:0/3884846528 wait complete. 2023-10-16T10:05:16.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.625+0000 7fea2541f640 1 Processor -- start 2023-10-16T10:05:16.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.625+0000 7fea2541f640 1 -- start start 2023-10-16T10:05:16.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.626+0000 7fea2541f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea20070fd0 0x7fea2015b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:16.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.626+0000 7fea2541f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea200adbe0 0x7fea20164420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:16.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.626+0000 7fea2541f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea200adfc0 0x7fea200afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:16.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.626+0000 7fea2541f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2015e2f0 con 0x7fea20070fd0 2023-10-16T10:05:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.626+0000 7fea2541f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2015da50 con 0x7fea200adfc0 2023-10-16T10:05:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.626+0000 7fea2541f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2015dd30 con 0x7fea200adbe0 2023-10-16T10:05:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.626+0000 7fea1effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea20070fd0 0x7fea2015b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.626+0000 7fea1effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea20070fd0 0x7fea2015b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35524/0 (socket says 172.21.15.84:35524) 2023-10-16T10:05:16.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.626+0000 7fea1effd640 1 -- 172.21.15.84:0/908071183 learned_addr learned my addr 172.21.15.84:0/908071183 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:16.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.627+0000 7fea1e7fc640 1 --2- 172.21.15.84:0/908071183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea200adbe0 0x7fea20164420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:16.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.627+0000 7fea1f7fe640 1 --2- 172.21.15.84:0/908071183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea200adfc0 0x7fea200afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:16.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.627+0000 7fea1effd640 1 -- 172.21.15.84:0/908071183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea200adbe0 msgr2=0x7fea20164420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:16.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.627+0000 7fea1effd640 1 --2- 172.21.15.84:0/908071183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea200adbe0 0x7fea20164420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.627+0000 7fea1effd640 1 -- 172.21.15.84:0/908071183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea200adfc0 msgr2=0x7fea200afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:16.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.627+0000 7fea1effd640 1 --2- 172.21.15.84:0/908071183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea200adfc0 0x7fea200afa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.627+0000 7fea1effd640 1 -- 172.21.15.84:0/908071183 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea10009400 con 0x7fea20070fd0 2023-10-16T10:05:16.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.627+0000 7fea1e7fc640 1 --2- 172.21.15.84:0/908071183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea200adbe0 0x7fea20164420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:16.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.627+0000 7fea1effd640 1 --2- 172.21.15.84:0/908071183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea20070fd0 0x7fea2015b000 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fea1002bee0 tx=0x7fea100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:16.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.627+0000 7fea077fe640 1 -- 172.21.15.84:0/908071183 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1003a6f0 con 0x7fea20070fd0 2023-10-16T10:05:16.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.627+0000 7fea077fe640 1 -- 172.21.15.84:0/908071183 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1003a870 con 0x7fea20070fd0 2023-10-16T10:05:16.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.627+0000 7fea2541f640 1 -- 172.21.15.84:0/908071183 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2015e010 con 0x7fea20070fd0 2023-10-16T10:05:16.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.627+0000 7fea077fe640 1 -- 172.21.15.84:0/908071183 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1003ac20 con 0x7fea20070fd0 2023-10-16T10:05:16.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.628+0000 7fea2541f640 1 -- 172.21.15.84:0/908071183 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2015b540 con 0x7fea20070fd0 2023-10-16T10:05:16.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.629+0000 7fea2541f640 1 -- 172.21.15.84:0/908071183 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2004e160 con 0x7fea20070fd0 2023-10-16T10:05:16.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.630+0000 7fea077fe640 1 -- 172.21.15.84:0/908071183 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fea1004c3b0 con 0x7fea20070fd0 2023-10-16T10:05:16.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.630+0000 7fea077fe640 1 --2- 172.21.15.84:0/908071183 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fea0c06afc0 0x7fea0c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:16.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.631+0000 7fea077fe640 1 -- 172.21.15.84:0/908071183 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea100ab3d0 con 0x7fea20070fd0 2023-10-16T10:05:16.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.631+0000 7fea1e7fc640 1 --2- 172.21.15.84:0/908071183 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fea0c06afc0 0x7fea0c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:16.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.631+0000 7fea1e7fc640 1 --2- 172.21.15.84:0/908071183 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fea0c06afc0 0x7fea0c06d480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fea2015ee20 tx=0x7fea14009040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:16.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.635+0000 7fea077fe640 1 -- 172.21.15.84:0/908071183 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fea10076410 con 0x7fea20070fd0 2023-10-16T10:05:16.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.896+0000 7fea2541f640 1 -- 172.21.15.84:0/908071183 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fea200b46c0 con 0x7fea20070fd0 2023-10-16T10:05:16.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.901+0000 7fea077fe640 1 -- 172.21.15.84:0/908071183 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7fea10079a30 con 0x7fea20070fd0 2023-10-16T10:05:16.903 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2023-10-16T10:05:16.903 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ== 2023-10-16T10:05:16.903 INFO:teuthology.orchestra.run.smithi084.stdout: caps mds = "allow" 2023-10-16T10:05:16.904 INFO:teuthology.orchestra.run.smithi084.stdout: caps mon = "allow r" 2023-10-16T10:05:16.904 INFO:teuthology.orchestra.run.smithi084.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T10:05:16.904 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-16T10:05:16.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.904+0000 7fea2541f640 1 -- 172.21.15.84:0/908071183 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fea0c06afc0 msgr2=0x7fea0c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:16.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.904+0000 7fea2541f640 1 --2- 172.21.15.84:0/908071183 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fea0c06afc0 0x7fea0c06d480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fea2015ee20 tx=0x7fea14009040 comp rx=0 tx=0).stop 2023-10-16T10:05:16.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.904+0000 7fea2541f640 1 -- 172.21.15.84:0/908071183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea20070fd0 msgr2=0x7fea2015b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:16.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.904+0000 7fea2541f640 1 --2- 172.21.15.84:0/908071183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea20070fd0 0x7fea2015b000 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fea1002bee0 tx=0x7fea100046c0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.904+0000 7fea2541f640 1 -- 172.21.15.84:0/908071183 shutdown_connections 2023-10-16T10:05:16.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.904+0000 7fea2541f640 1 --2- 172.21.15.84:0/908071183 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fea0c06afc0 0x7fea0c06d480 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.904+0000 7fea2541f640 1 --2- 172.21.15.84:0/908071183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea200adfc0 0x7fea200afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.904+0000 7fea2541f640 1 --2- 172.21.15.84:0/908071183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea200adbe0 0x7fea20164420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.904+0000 7fea2541f640 1 --2- 172.21.15.84:0/908071183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea20070fd0 0x7fea2015b000 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:16.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.904+0000 7fea2541f640 1 -- 172.21.15.84:0/908071183 >> 172.21.15.84:0/908071183 conn(0x7fea200cb410 msgr2=0x7fea200aec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:16.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.905+0000 7fea2541f640 1 -- 172.21.15.84:0/908071183 shutdown_connections 2023-10-16T10:05:16.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:16.905+0000 7fea2541f640 1 -- 172.21.15.84:0/908071183 wait complete. 2023-10-16T10:05:16.925 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:05:17.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.131+0000 7fda758c9640 1 Processor -- start 2023-10-16T10:05:17.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.131+0000 7fda758c9640 1 -- start start 2023-10-16T10:05:17.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.132+0000 7fda758c9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fda7015ae80 0x7fda7015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:17.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.132+0000 7fda758c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda700ce390 0x7fda70070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:17.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.133+0000 7fda758c9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda7015ed60 con 0x7fda7015aaa0 2023-10-16T10:05:17.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.133+0000 7fda758c9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda7006b810 con 0x7fda7015ae80 2023-10-16T10:05:17.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.133+0000 7fda758c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda700707e0 con 0x7fda700ce390 2023-10-16T10:05:17.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.133+0000 7fda67fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fda7015ae80 0x7fda7015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:17.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.133+0000 7fda67fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fda7015ae80 0x7fda7015e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35526/0 (socket says 172.21.15.84:35526) 2023-10-16T10:05:17.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.133+0000 7fda67fff640 1 -- 172.21.15.84:0/3365884034 learned_addr learned my addr 172.21.15.84:0/3365884034 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:17.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.133+0000 7fda67fff640 1 -- 172.21.15.84:0/3365884034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda700ce390 msgr2=0x7fda70070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:17.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.133+0000 7fda67fff640 1 --2- 172.21.15.84:0/3365884034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda700ce390 0x7fda70070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:17.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.133+0000 7fda67fff640 1 -- 172.21.15.84:0/3365884034 >> v1:172.21.15.84:6789/0 conn(0x7fda7015aaa0 legacy=0x7fda700cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:17.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.133+0000 7fda67fff640 1 -- 172.21.15.84:0/3365884034 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7015ef90 con 0x7fda7015ae80 2023-10-16T10:05:17.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.134+0000 7fda67fff640 1 --2- 172.21.15.84:0/3365884034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda7015ae80 0x7fda7015e820 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fda58009870 tx=0x7fda5802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a110e49533dd0479 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:17.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.135+0000 7fda6e7fc640 1 -- 172.21.15.84:0/3365884034 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58030840 con 0x7fda7015ae80 2023-10-16T10:05:17.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.135+0000 7fda6e7fc640 1 -- 172.21.15.84:0/3365884034 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda580309a0 con 0x7fda7015ae80 2023-10-16T10:05:17.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.135+0000 7fda6e7fc640 1 -- 172.21.15.84:0/3365884034 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58030d30 con 0x7fda7015ae80 2023-10-16T10:05:17.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.135+0000 7fda758c9640 1 -- 172.21.15.84:0/3365884034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda7015ae80 msgr2=0x7fda7015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:17.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.135+0000 7fda758c9640 1 --2- 172.21.15.84:0/3365884034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda7015ae80 0x7fda7015e820 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fda58009870 tx=0x7fda5802b980 comp rx=0 tx=0).stop 2023-10-16T10:05:17.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.135+0000 7fda758c9640 1 -- 172.21.15.84:0/3365884034 shutdown_connections 2023-10-16T10:05:17.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.135+0000 7fda758c9640 1 --2- 172.21.15.84:0/3365884034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda700ce390 0x7fda70070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:17.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.135+0000 7fda758c9640 1 --2- 172.21.15.84:0/3365884034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda7015ae80 0x7fda7015e820 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:17.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.135+0000 7fda758c9640 1 -- 172.21.15.84:0/3365884034 >> 172.21.15.84:0/3365884034 conn(0x7fda700cb410 msgr2=0x7fda700cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:17.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.135+0000 7fda758c9640 1 -- 172.21.15.84:0/3365884034 shutdown_connections 2023-10-16T10:05:17.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.135+0000 7fda758c9640 1 -- 172.21.15.84:0/3365884034 wait complete. 2023-10-16T10:05:17.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.137+0000 7fda758c9640 1 Processor -- start 2023-10-16T10:05:17.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.137+0000 7fda758c9640 1 -- start start 2023-10-16T10:05:17.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.138+0000 7fda758c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda700ce390 0x7fda700c3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.138+0000 7fda758c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda7015aaa0 0x7fda70156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.138+0000 7fda758c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda7015ae80 0x7fda70141e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.138+0000 7fda758c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda700c3b80 con 0x7fda700ce390 2023-10-16T10:05:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.138+0000 7fda758c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda700c3e60 con 0x7fda7015aaa0 2023-10-16T10:05:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.138+0000 7fda758c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda700c4140 con 0x7fda7015ae80 2023-10-16T10:05:17.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.138+0000 7fda6effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda700ce390 0x7fda700c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:17.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.138+0000 7fda6effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda700ce390 0x7fda700c3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35530/0 (socket says 172.21.15.84:35530) 2023-10-16T10:05:17.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.138+0000 7fda6effd640 1 -- 172.21.15.84:0/1010342805 learned_addr learned my addr 172.21.15.84:0/1010342805 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:17.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.139+0000 7fda6f7fe640 1 --2- 172.21.15.84:0/1010342805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda7015ae80 0x7fda70141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:17.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.139+0000 7fda6effd640 1 -- 172.21.15.84:0/1010342805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda7015ae80 msgr2=0x7fda70141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.139+0000 7fda6effd640 1 --2- 172.21.15.84:0/1010342805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda7015ae80 0x7fda70141e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.139+0000 7fda6effd640 1 -- 172.21.15.84:0/1010342805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda7015aaa0 msgr2=0x7fda70156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.139+0000 7fda6effd640 1 --2- 172.21.15.84:0/1010342805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda7015aaa0 0x7fda70156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.139+0000 7fda6effd640 1 -- 172.21.15.84:0/1010342805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda580094d0 con 0x7fda700ce390 2023-10-16T10:05:17.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.139+0000 7fda6effd640 1 --2- 172.21.15.84:0/1010342805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda700ce390 0x7fda700c3510 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fda6000c4d0 tx=0x7fda6000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:17.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.139+0000 7fda6cff9640 1 -- 172.21.15.84:0/1010342805 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda600052d0 con 0x7fda700ce390 2023-10-16T10:05:17.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.139+0000 7fda6cff9640 1 -- 172.21.15.84:0/1010342805 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda60005430 con 0x7fda700ce390 2023-10-16T10:05:17.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.139+0000 7fda6cff9640 1 -- 172.21.15.84:0/1010342805 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda60012850 con 0x7fda700ce390 2023-10-16T10:05:17.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.139+0000 7fda758c9640 1 -- 172.21.15.84:0/1010342805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda700c4420 con 0x7fda700ce390 2023-10-16T10:05:17.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.140+0000 7fda758c9640 1 -- 172.21.15.84:0/1010342805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda701572c0 con 0x7fda700ce390 2023-10-16T10:05:17.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.141+0000 7fda758c9640 1 -- 172.21.15.84:0/1010342805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda700c4940 con 0x7fda700ce390 2023-10-16T10:05:17.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.142+0000 7fda6cff9640 1 -- 172.21.15.84:0/1010342805 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fda600129b0 con 0x7fda700ce390 2023-10-16T10:05:17.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.142+0000 7fda6cff9640 1 --2- 172.21.15.84:0/1010342805 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fda5006af90 0x7fda5006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:17.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.143+0000 7fda6cff9640 1 -- 172.21.15.84:0/1010342805 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda60089230 con 0x7fda700ce390 2023-10-16T10:05:17.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.143+0000 7fda67fff640 1 --2- 172.21.15.84:0/1010342805 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fda5006af90 0x7fda5006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:17.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.143+0000 7fda67fff640 1 --2- 172.21.15.84:0/1010342805 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fda5006af90 0x7fda5006d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fda58036000 tx=0x7fda5803b040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:17.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.147+0000 7fda6cff9640 1 -- 172.21.15.84:0/1010342805 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fda60054260 con 0x7fda700ce390 2023-10-16T10:05:17.317 INFO:tasks.ceph.mds.c.smithi084.stderr:2023-10-16T10:05:17.316+0000 7f8900b07640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:05:17.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.375+0000 7fda758c9640 1 -- 172.21.15.84:0/1010342805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda7004e160 con 0x7fda700ce390 2023-10-16T10:05:17.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.376+0000 7fda6cff9640 1 -- 172.21.15.84:0/1010342805 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fda60057880 con 0x7fda700ce390 2023-10-16T10:05:17.378 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:17.379 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/89534235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":89534235},{"type":"v1","addr":"172.21.15.157:6835","nonce":89534235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3807666966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3807666966},{"type":"v1","addr":"172.21.15.157:6834","nonce":3807666966}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3448417053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3448417053},{"type":"v1","addr":"172.21.15.84:6835","nonce":3448417053}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:05:10.829122+0000","modified":"2023-10-16T10:05:13.893618+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7044":{"gid":7044,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/1939610713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":1939610713},{"type":"v1","addr":"172.21.15.84:6837","nonce":1939610713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-16T10:05:17.379 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 44 2023-10-16T10:05:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.380+0000 7fda758c9640 1 -- 172.21.15.84:0/1010342805 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fda5006af90 msgr2=0x7fda5006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.380+0000 7fda758c9640 1 --2- 172.21.15.84:0/1010342805 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fda5006af90 0x7fda5006d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fda58036000 tx=0x7fda5803b040 comp rx=0 tx=0).stop 2023-10-16T10:05:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.380+0000 7fda758c9640 1 -- 172.21.15.84:0/1010342805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda700ce390 msgr2=0x7fda700c3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.380+0000 7fda758c9640 1 --2- 172.21.15.84:0/1010342805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda700ce390 0x7fda700c3510 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fda6000c4d0 tx=0x7fda6000c990 comp rx=0 tx=0).stop 2023-10-16T10:05:17.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.380+0000 7fda758c9640 1 -- 172.21.15.84:0/1010342805 shutdown_connections 2023-10-16T10:05:17.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.380+0000 7fda758c9640 1 --2- 172.21.15.84:0/1010342805 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fda5006af90 0x7fda5006d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:17.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.380+0000 7fda758c9640 1 --2- 172.21.15.84:0/1010342805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda7015ae80 0x7fda70141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:17.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.380+0000 7fda758c9640 1 --2- 172.21.15.84:0/1010342805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda7015aaa0 0x7fda70156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:17.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.380+0000 7fda758c9640 1 --2- 172.21.15.84:0/1010342805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda700ce390 0x7fda700c3510 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:17.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.380+0000 7fda758c9640 1 -- 172.21.15.84:0/1010342805 >> 172.21.15.84:0/1010342805 conn(0x7fda700cb410 msgr2=0x7fda7006f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:17.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.380+0000 7fda758c9640 1 -- 172.21.15.84:0/1010342805 shutdown_connections 2023-10-16T10:05:17.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:17.380+0000 7fda758c9640 1 -- 172.21.15.84:0/1010342805 wait complete. 2023-10-16T10:05:17.802 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T10:05:10.829122+0000', 'modified': '2023-10-16T10:05:13.893618+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7044}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7044': {'gid': 7044, 'name': 'c', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.84:6837/1939610713', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6836', 'nonce': 1939610713}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 1939610713}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T10:05:17.803 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T10:05:17.803 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-16T10:05:18.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.028+0000 7fef90a1f640 1 Processor -- start 2023-10-16T10:05:18.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.029+0000 7fef90a1f640 1 -- start start 2023-10-16T10:05:18.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.030+0000 7fef90a1f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fef8c06a4b0 0x7fef8c0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.030+0000 7fef90a1f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef8c0c9ca0 0x7fef8c0ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.030+0000 7fef90a1f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef8c0cb030 con 0x7fef8c15ad70 2023-10-16T10:05:18.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.030+0000 7fef90a1f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef8c0cae30 con 0x7fef8c06a4b0 2023-10-16T10:05:18.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.030+0000 7fef90a1f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef8c0c91c0 con 0x7fef8c0c9ca0 2023-10-16T10:05:18.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.030+0000 7fef83fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fef8c06a4b0 0x7fef8c0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.030+0000 7fef8a59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fef8c15ad70 0x7fef8c069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:59340/0 (socket says 172.21.15.84:59340) 2023-10-16T10:05:18.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.031+0000 7fef83fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fef8c06a4b0 0x7fef8c0ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35538/0 (socket says 172.21.15.84:35538) 2023-10-16T10:05:18.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.031+0000 7fef8a59c640 1 -- 172.21.15.84:0/3761090402 learned_addr learned my addr 172.21.15.84:0/3761090402 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:18.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.031+0000 7fef8ad9d640 1 --2- 172.21.15.84:0/3761090402 >> v2:172.21.15.157:3300/0 conn(0x7fef8c0c9ca0 0x7fef8c0ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.031+0000 7fef83fff640 1 -- 172.21.15.84:0/3761090402 >> v2:172.21.15.157:3300/0 conn(0x7fef8c0c9ca0 msgr2=0x7fef8c0ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:18.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.031+0000 7fef83fff640 1 --2- 172.21.15.84:0/3761090402 >> v2:172.21.15.157:3300/0 conn(0x7fef8c0c9ca0 0x7fef8c0ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.031+0000 7fef83fff640 1 -- 172.21.15.84:0/3761090402 >> v1:172.21.15.84:6789/0 conn(0x7fef8c15ad70 legacy=0x7fef8c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:18.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.031+0000 7fef83fff640 1 -- 172.21.15.84:0/3761090402 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c0c94a0 con 0x7fef8c06a4b0 2023-10-16T10:05:18.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.032+0000 7fef83fff640 1 --2- 172.21.15.84:0/3761090402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef8c06a4b0 0x7fef8c0ca3b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fef8c06f270 tx=0x7fef7402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2227c79bbaa4ebc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:18.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.032+0000 7fef89d9b640 1 -- 172.21.15.84:0/3761090402 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef74031830 con 0x7fef8c06a4b0 2023-10-16T10:05:18.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.032+0000 7fef89d9b640 1 -- 172.21.15.84:0/3761090402 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef74031990 con 0x7fef8c06a4b0 2023-10-16T10:05:18.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.032+0000 7fef89d9b640 1 -- 172.21.15.84:0/3761090402 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef74031d20 con 0x7fef8c06a4b0 2023-10-16T10:05:18.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.033+0000 7fef90a1f640 1 -- 172.21.15.84:0/3761090402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef8c06a4b0 msgr2=0x7fef8c0ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:18.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.033+0000 7fef90a1f640 1 --2- 172.21.15.84:0/3761090402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef8c06a4b0 0x7fef8c0ca3b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fef8c06f270 tx=0x7fef7402b8d0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.033+0000 7fef90a1f640 1 -- 172.21.15.84:0/3761090402 shutdown_connections 2023-10-16T10:05:18.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.033+0000 7fef90a1f640 1 --2- 172.21.15.84:0/3761090402 >> v2:172.21.15.157:3300/0 conn(0x7fef8c0c9ca0 0x7fef8c0ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.033+0000 7fef90a1f640 1 --2- 172.21.15.84:0/3761090402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef8c06a4b0 0x7fef8c0ca3b0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.033+0000 7fef90a1f640 1 -- 172.21.15.84:0/3761090402 >> 172.21.15.84:0/3761090402 conn(0x7fef8c0ce9b0 msgr2=0x7fef8c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:18.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.033+0000 7fef90a1f640 1 -- 172.21.15.84:0/3761090402 shutdown_connections 2023-10-16T10:05:18.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.033+0000 7fef90a1f640 1 -- 172.21.15.84:0/3761090402 wait complete. 2023-10-16T10:05:18.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.035+0000 7fef90a1f640 1 Processor -- start 2023-10-16T10:05:18.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.035+0000 7fef90a1f640 1 -- start start 2023-10-16T10:05:18.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.036+0000 7fef90a1f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef8c06a4b0 0x7fef8c0b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.036+0000 7fef90a1f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef8c0c9ca0 0x7fef8c15d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.036+0000 7fef90a1f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef8c15ad70 0x7fef8c1ef7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.036+0000 7fef90a1f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0b15d0 con 0x7fef8c06a4b0 2023-10-16T10:05:18.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.036+0000 7fef90a1f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c1f4760 con 0x7fef8c15ad70 2023-10-16T10:05:18.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.036+0000 7fef90a1f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0b1820 con 0x7fef8c0c9ca0 2023-10-16T10:05:18.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.036+0000 7fef8a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef8c06a4b0 0x7fef8c0b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.036+0000 7fef8a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef8c06a4b0 0x7fef8c0b2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35550/0 (socket says 172.21.15.84:35550) 2023-10-16T10:05:18.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.036+0000 7fef8a59c640 1 -- 172.21.15.84:0/4119242772 learned_addr learned my addr 172.21.15.84:0/4119242772 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:18.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.037+0000 7fef83fff640 1 --2- 172.21.15.84:0/4119242772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef8c0c9ca0 0x7fef8c15d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.037+0000 7fef8ad9d640 1 --2- 172.21.15.84:0/4119242772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef8c15ad70 0x7fef8c1ef7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.037+0000 7fef8a59c640 1 -- 172.21.15.84:0/4119242772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef8c0c9ca0 msgr2=0x7fef8c15d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:18.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.037+0000 7fef8a59c640 1 --2- 172.21.15.84:0/4119242772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef8c0c9ca0 0x7fef8c15d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.037+0000 7fef8a59c640 1 -- 172.21.15.84:0/4119242772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef8c15ad70 msgr2=0x7fef8c1ef7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:18.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.037+0000 7fef8a59c640 1 --2- 172.21.15.84:0/4119242772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef8c15ad70 0x7fef8c1ef7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.037+0000 7fef8a59c640 1 -- 172.21.15.84:0/4119242772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef74009400 con 0x7fef8c06a4b0 2023-10-16T10:05:18.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.037+0000 7fef8a59c640 1 --2- 172.21.15.84:0/4119242772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef8c06a4b0 0x7fef8c0b2300 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00c480 tx=0x7fef7c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:18.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.037+0000 7fef82ffd640 1 -- 172.21.15.84:0/4119242772 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c005ce0 con 0x7fef8c06a4b0 2023-10-16T10:05:18.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.038+0000 7fef82ffd640 1 -- 172.21.15.84:0/4119242772 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7c005e40 con 0x7fef8c06a4b0 2023-10-16T10:05:18.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.038+0000 7fef82ffd640 1 -- 172.21.15.84:0/4119242772 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c0072f0 con 0x7fef8c06a4b0 2023-10-16T10:05:18.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.038+0000 7fef90a1f640 1 -- 172.21.15.84:0/4119242772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8c0b1b00 con 0x7fef8c06a4b0 2023-10-16T10:05:18.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.038+0000 7fef90a1f640 1 -- 172.21.15.84:0/4119242772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8c15db50 con 0x7fef8c06a4b0 2023-10-16T10:05:18.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.040+0000 7fef82ffd640 1 -- 172.21.15.84:0/4119242772 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fef7c007510 con 0x7fef8c06a4b0 2023-10-16T10:05:18.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.042+0000 7fef82ffd640 1 --2- 172.21.15.84:0/4119242772 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fef6c06af90 0x7fef6c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.042+0000 7fef935b3740 1 Processor -- start 2023-10-16T10:05:18.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.042+0000 7fef82ffd640 1 -- 172.21.15.84:0/4119242772 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef7c088c60 con 0x7fef8c06a4b0 2023-10-16T10:05:18.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.042+0000 7fef935b3740 1 -- start start 2023-10-16T10:05:18.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.042+0000 7fef83fff640 1 --2- 172.21.15.84:0/4119242772 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fef6c06af90 0x7fef6c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.043+0000 7fef83fff640 1 --2- 172.21.15.84:0/4119242772 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fef6c06af90 0x7fef6c06d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fef74036000 tx=0x7fef74008030 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:18.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.043+0000 7fef935b3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73afb00 0x563ab73ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.043+0000 7fef935b3740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab738f340 0x563ab7367030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.043+0000 7fef935b3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab7367570 0x563ab7391d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.043+0000 7fef935b3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab72827a0 con 0x563ab73afb00 2023-10-16T10:05:18.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.043+0000 7fef935b3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab72d4740 con 0x563ab7367570 2023-10-16T10:05:18.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.043+0000 7fef935b3740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ab72d1e30 con 0x563ab738f340 2023-10-16T10:05:18.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.043+0000 7fef8a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73afb00 0x563ab73ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.043+0000 7fef8a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73afb00 0x563ab73ad6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35566/0 (socket says 172.21.15.84:35566) 2023-10-16T10:05:18.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.043+0000 7fef8a59c640 1 -- 172.21.15.84:0/781780994 learned_addr learned my addr 172.21.15.84:0/781780994 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:18.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.044+0000 7fef8a59c640 1 -- 172.21.15.84:0/781780994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab738f340 msgr2=0x563ab7367030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:18.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.044+0000 7fef8a59c640 1 --2- 172.21.15.84:0/781780994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab738f340 0x563ab7367030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.044+0000 7fef8a59c640 1 -- 172.21.15.84:0/781780994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab7367570 msgr2=0x563ab7391d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:18.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.044+0000 7fef8a59c640 1 --2- 172.21.15.84:0/781780994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab7367570 0x563ab7391d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.044+0000 7fef8a59c640 1 -- 172.21.15.84:0/781780994 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ab7291910 con 0x563ab73afb00 2023-10-16T10:05:18.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.044+0000 7fef8a59c640 1 --2- 172.21.15.84:0/781780994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73afb00 0x563ab73ad6c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fef7c000f60 tx=0x7fef7c05ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b9bf9420bc6f71 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:18.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.044+0000 7fef6bfff640 1 -- 172.21.15.84:0/781780994 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c055d00 con 0x563ab73afb00 2023-10-16T10:05:18.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.044+0000 7fef6bfff640 1 -- 172.21.15.84:0/781780994 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7c0870e0 con 0x563ab73afb00 2023-10-16T10:05:18.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.044+0000 7fef6bfff640 1 -- 172.21.15.84:0/781780994 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c087250 con 0x563ab73afb00 2023-10-16T10:05:18.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.045+0000 7fef935b3740 1 -- 172.21.15.84:0/781780994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73afb00 msgr2=0x563ab73ad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:18.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.045+0000 7fef935b3740 1 --2- 172.21.15.84:0/781780994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73afb00 0x563ab73ad6c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fef7c000f60 tx=0x7fef7c05ba00 comp rx=0 tx=0).stop 2023-10-16T10:05:18.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.045+0000 7fef935b3740 1 -- 172.21.15.84:0/781780994 shutdown_connections 2023-10-16T10:05:18.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.045+0000 7fef935b3740 1 --2- 172.21.15.84:0/781780994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab7367570 0x563ab7391d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.045+0000 7fef935b3740 1 --2- 172.21.15.84:0/781780994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab738f340 0x563ab7367030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.045+0000 7fef935b3740 1 --2- 172.21.15.84:0/781780994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73afb00 0x563ab73ad6c0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.045+0000 7fef935b3740 1 -- 172.21.15.84:0/781780994 >> 172.21.15.84:0/781780994 conn(0x563ab7364190 msgr2=0x563ab73acd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:18.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.045+0000 7fef935b3740 1 -- 172.21.15.84:0/781780994 shutdown_connections 2023-10-16T10:05:18.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.045+0000 7fef935b3740 1 -- 172.21.15.84:0/781780994 wait complete. 2023-10-16T10:05:18.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.046+0000 7fef935b3740 1 -- start start 2023-10-16T10:05:18.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.046+0000 7fef935b3740 1 Processor -- start 2023-10-16T10:05:18.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.047+0000 7fef935b3740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab738f340 0x563ab73791a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.047+0000 7fef935b3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab73afb00 0x563ab738a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.047+0000 7fef935b3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73a97d0 0x563ab73a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.047+0000 7fef935b3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab72cd720 con 0x563ab73a97d0 2023-10-16T10:05:18.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.047+0000 7fef935b3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab726b970 con 0x563ab73afb00 2023-10-16T10:05:18.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.047+0000 7fef935b3740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ab726b610 con 0x563ab738f340 2023-10-16T10:05:18.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.047+0000 7fef935b3740 10 client.0 _ll_register_callbacks cb 0x563ab728d9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:05:18.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.047+0000 7fef8ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73a97d0 0x563ab73a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.047+0000 7fef8ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73a97d0 0x563ab73a9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35574/0 (socket says 172.21.15.84:35574) 2023-10-16T10:05:18.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.047+0000 7fef8ad9d640 1 -- 172.21.15.84:0/598823298 learned_addr learned my addr 172.21.15.84:0/598823298 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:18.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.047+0000 7fef8a59c640 1 --2- 172.21.15.84:0/598823298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab738f340 0x563ab73791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.047+0000 7fef83fff640 1 --2- 172.21.15.84:0/598823298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab73afb00 0x563ab738a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.048+0000 7fef8ad9d640 1 -- 172.21.15.84:0/598823298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab738f340 msgr2=0x563ab73791a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:18.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.048+0000 7fef8ad9d640 1 --2- 172.21.15.84:0/598823298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab738f340 0x563ab73791a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.048+0000 7fef8ad9d640 1 -- 172.21.15.84:0/598823298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab73afb00 msgr2=0x563ab738a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:18.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.048+0000 7fef8ad9d640 1 --2- 172.21.15.84:0/598823298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab73afb00 0x563ab738a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.048+0000 7fef8ad9d640 1 -- 172.21.15.84:0/598823298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8c15db50 con 0x563ab73a97d0 2023-10-16T10:05:18.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.048+0000 7fef8a59c640 1 --2- 172.21.15.84:0/598823298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab738f340 0x563ab73791a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:18.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.048+0000 7fef8ad9d640 1 --2- 172.21.15.84:0/598823298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73a97d0 0x563ab73a9c50 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fef840079d0 tx=0x7fef84007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c80882702deb61 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:18.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.048+0000 7fef697fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T10:05:18.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.048+0000 7fef697fa640 1 -- 172.21.15.84:0/598823298 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8400fc00 con 0x563ab73a97d0 2023-10-16T10:05:18.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.048+0000 7fef935b3740 1 -- 172.21.15.84:0/598823298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ab72f4160 con 0x563ab73a97d0 2023-10-16T10:05:18.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.048+0000 7fef697fa640 1 -- 172.21.15.84:0/598823298 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef8400fd60 con 0x563ab73a97d0 2023-10-16T10:05:18.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.048+0000 7fef697fa640 1 -- 172.21.15.84:0/598823298 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef840143f0 con 0x563ab73a97d0 2023-10-16T10:05:18.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.048+0000 7fef935b3740 10 client.7164 fetch_fsmap learned FSMap version 44 2023-10-16T10:05:18.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.048+0000 7fef935b3740 1 -- 172.21.15.84:0/598823298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x563ab72cbda0 con 0x563ab73a97d0 2023-10-16T10:05:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.049+0000 7fef697fa640 1 -- 172.21.15.84:0/598823298 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563ab72f4160 con 0x563ab73a97d0 2023-10-16T10:05:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.049+0000 7fef697fa640 1 -- 172.21.15.84:0/598823298 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fef84014550 con 0x563ab73a97d0 2023-10-16T10:05:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.049+0000 7fef935b3740 10 client.7164 fetch_fsmap finished waiting for FSMap version 44 2023-10-16T10:05:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.049+0000 7fef935b3740 10 client.7164 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T10:05:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.049+0000 7fef935b3740 20 client.7164 populate_metadata read hostname 'smithi084' 2023-10-16T10:05:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.049+0000 7fef935b3740 1 --2- 172.21.15.84:0/598823298 >> [v2:172.21.15.84:6836/1939610713,v1:172.21.15.84:6837/1939610713] conn(0x563ab73c52d0 0x563ab738d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.049+0000 7fef935b3740 4 client.7164 mds_command: new command op to 7044 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T10:05:18.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.049+0000 7fef935b3740 1 -- 172.21.15.84:0/598823298 --> [v2:172.21.15.84:6836/1939610713,v1:172.21.15.84:6837/1939610713] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ab729fcc0 con 0x563ab73c52d0 2023-10-16T10:05:18.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.049+0000 7fef8a59c640 1 --2- 172.21.15.84:0/598823298 >> [v2:172.21.15.84:6836/1939610713,v1:172.21.15.84:6837/1939610713] conn(0x563ab73c52d0 0x563ab738d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.050+0000 7fef8a59c640 1 --2- 172.21.15.84:0/598823298 >> [v2:172.21.15.84:6836/1939610713,v1:172.21.15.84:6837/1939610713] conn(0x563ab73c52d0 0x563ab738d2d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e02068db5f0ef7b server_cookie=622348036c0e06c6 in_seq=0 out_seq=0 2023-10-16T10:05:18.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.050+0000 7fef697fa640 10 client.7164 ms_handle_connect on v2:172.21.15.84:6836/1939610713 2023-10-16T10:05:18.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.050+0000 7fef697fa640 1 -- 172.21.15.84:0/598823298 <== mds.0 v2:172.21.15.84:6836/1939610713 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563ab729fcc0 con 0x563ab73c52d0 2023-10-16T10:05:18.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.050+0000 7fef697fa640 10 client.7164 handle_command_reply: tid=0 2023-10-16T10:05:18.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.050+0000 7fef935b3740 1 client.7164 shutdown 2023-10-16T10:05:18.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.051+0000 7fef935b3740 1 -- 172.21.15.84:0/598823298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73a97d0 msgr2=0x563ab73a9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:18.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.051+0000 7fef935b3740 1 --2- 172.21.15.84:0/598823298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73a97d0 0x563ab73a9c50 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fef840079d0 tx=0x7fef84007dd0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.051+0000 7fef935b3740 1 -- 172.21.15.84:0/598823298 shutdown_connections 2023-10-16T10:05:18.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.051+0000 7fef935b3740 1 --2- 172.21.15.84:0/598823298 >> [v2:172.21.15.84:6836/1939610713,v1:172.21.15.84:6837/1939610713] conn(0x563ab73c52d0 0x563ab738d2d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.051+0000 7fef935b3740 1 --2- 172.21.15.84:0/598823298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73a97d0 0x563ab73a9c50 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.051+0000 7fef935b3740 1 --2- 172.21.15.84:0/598823298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab73afb00 0x563ab738a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.051+0000 7fef697fa640 0 client.7164 ms_handle_reset on v2:172.21.15.84:6836/1939610713 2023-10-16T10:05:18.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.051+0000 7fef935b3740 1 --2- 172.21.15.84:0/598823298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab738f340 0x563ab73791a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.051+0000 7fef935b3740 1 -- 172.21.15.84:0/598823298 >> 172.21.15.84:0/598823298 conn(0x563ab7367570 msgr2=0x563ab7360e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:18.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.051+0000 7fef935b3740 1 -- 172.21.15.84:0/598823298 shutdown_connections 2023-10-16T10:05:18.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.051+0000 7fef935b3740 1 -- 172.21.15.84:0/598823298 wait complete. 2023-10-16T10:05:18.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.051+0000 7fef935b3740 20 client.7164 trim_cache size 0 max 16384 2023-10-16T10:05:18.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.067+0000 7fef935b3740 1 Processor -- start 2023-10-16T10:05:18.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.067+0000 7fef935b3740 1 -- start start 2023-10-16T10:05:18.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.068+0000 7fef935b3740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab738f340 0x563ab73af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.068+0000 7fef935b3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab73a97d0 0x563ab7361480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.068+0000 7fef935b3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73afb00 0x563ab73a8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.068+0000 7fef935b3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab7031f20 con 0x563ab73afb00 2023-10-16T10:05:18.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.068+0000 7fef935b3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab7367dd0 con 0x563ab73a97d0 2023-10-16T10:05:18.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.068+0000 7fef935b3740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ab73a8ac0 con 0x563ab738f340 2023-10-16T10:05:18.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.069+0000 7fef8ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73afb00 0x563ab73a8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.069+0000 7fef8ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73afb00 0x563ab73a8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35582/0 (socket says 172.21.15.84:35582) 2023-10-16T10:05:18.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.069+0000 7fef8ad9d640 1 -- 172.21.15.84:0/2722101045 learned_addr learned my addr 172.21.15.84:0/2722101045 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:18.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.069+0000 7fef8ad9d640 1 -- 172.21.15.84:0/2722101045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab738f340 msgr2=0x563ab73af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:18.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.069+0000 7fef8ad9d640 1 --2- 172.21.15.84:0/2722101045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab738f340 0x563ab73af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.069+0000 7fef8ad9d640 1 -- 172.21.15.84:0/2722101045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab73a97d0 msgr2=0x563ab7361480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:18.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.069+0000 7fef8ad9d640 1 --2- 172.21.15.84:0/2722101045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab73a97d0 0x563ab7361480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.069+0000 7fef8ad9d640 1 -- 172.21.15.84:0/2722101045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ab7366280 con 0x563ab73afb00 2023-10-16T10:05:18.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.069+0000 7fef8ad9d640 1 --2- 172.21.15.84:0/2722101045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73afb00 0x563ab73a8f80 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fef84009ea0 tx=0x7fef84014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c1a05e870b2013 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:18.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.069+0000 7fef80ff9640 1 -- 172.21.15.84:0/2722101045 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84014e40 con 0x563ab73afb00 2023-10-16T10:05:18.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.069+0000 7fef80ff9640 1 -- 172.21.15.84:0/2722101045 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef84007040 con 0x563ab73afb00 2023-10-16T10:05:18.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.069+0000 7fef80ff9640 1 -- 172.21.15.84:0/2722101045 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840143f0 con 0x563ab73afb00 2023-10-16T10:05:18.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.070+0000 7fef935b3740 1 -- 172.21.15.84:0/2722101045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73afb00 msgr2=0x563ab73a8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:18.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.070+0000 7fef935b3740 1 --2- 172.21.15.84:0/2722101045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73afb00 0x563ab73a8f80 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fef84009ea0 tx=0x7fef84014000 comp rx=0 tx=0).stop 2023-10-16T10:05:18.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.070+0000 7fef935b3740 1 -- 172.21.15.84:0/2722101045 shutdown_connections 2023-10-16T10:05:18.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.070+0000 7fef935b3740 1 --2- 172.21.15.84:0/2722101045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73afb00 0x563ab73a8f80 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.070+0000 7fef935b3740 1 --2- 172.21.15.84:0/2722101045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab73a97d0 0x563ab7361480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.070+0000 7fef935b3740 1 --2- 172.21.15.84:0/2722101045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab738f340 0x563ab73af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.070+0000 7fef935b3740 1 -- 172.21.15.84:0/2722101045 >> 172.21.15.84:0/2722101045 conn(0x563ab7367570 msgr2=0x563ab7395ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:18.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.070+0000 7fef935b3740 1 -- 172.21.15.84:0/2722101045 shutdown_connections 2023-10-16T10:05:18.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.070+0000 7fef935b3740 1 -- 172.21.15.84:0/2722101045 wait complete. 2023-10-16T10:05:18.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.071+0000 7fef935b3740 1 -- start start 2023-10-16T10:05:18.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.071+0000 7fef935b3740 1 Processor -- start 2023-10-16T10:05:18.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.072+0000 7fef935b3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab738f340 0x563ab7416da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.072+0000 7fef935b3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73a97d0 0x563ab738e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.072+0000 7fef935b3740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab73afb00 0x563ab738ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.072+0000 7fef935b3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab73c3640 con 0x563ab73a97d0 2023-10-16T10:05:18.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.072+0000 7fef935b3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab738d510 con 0x563ab738f340 2023-10-16T10:05:18.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.072+0000 7fef935b3740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ab74172e0 con 0x563ab73afb00 2023-10-16T10:05:18.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.072+0000 7fef935b3740 10 client.0 _ll_register_callbacks cb 0x7fef84009730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:05:18.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.072+0000 7fef83fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73a97d0 0x563ab738e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.072+0000 7fef83fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73a97d0 0x563ab738e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35594/0 (socket says 172.21.15.84:35594) 2023-10-16T10:05:18.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.072+0000 7fef83fff640 1 -- 172.21.15.84:0/4005968376 learned_addr learned my addr 172.21.15.84:0/4005968376 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:18.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.073+0000 7fef83fff640 1 -- 172.21.15.84:0/4005968376 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab73afb00 msgr2=0x563ab738ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:18.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.073+0000 7fef83fff640 1 --2- 172.21.15.84:0/4005968376 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab73afb00 0x563ab738ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.073+0000 7fef8a59c640 1 --2- 172.21.15.84:0/4005968376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab738f340 0x563ab7416da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.073+0000 7fef83fff640 1 -- 172.21.15.84:0/4005968376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab738f340 msgr2=0x563ab7416da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:18.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.073+0000 7fef83fff640 1 --2- 172.21.15.84:0/4005968376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab738f340 0x563ab7416da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.073+0000 7fef83fff640 1 -- 172.21.15.84:0/4005968376 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ab72cbda0 con 0x563ab73a97d0 2023-10-16T10:05:18.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.073+0000 7fef83fff640 1 --2- 172.21.15.84:0/4005968376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73a97d0 0x563ab738e6a0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fef74009860 tx=0x7fef74031c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a234bf27e1132b7d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:18.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.073+0000 7fef697fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T10:05:18.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.073+0000 7fef697fa640 1 -- 172.21.15.84:0/4005968376 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7404b070 con 0x563ab73a97d0 2023-10-16T10:05:18.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.073+0000 7fef697fa640 1 -- 172.21.15.84:0/4005968376 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef74046ab0 con 0x563ab73a97d0 2023-10-16T10:05:18.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.073+0000 7fef935b3740 1 -- 172.21.15.84:0/4005968376 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef7c0870e0 con 0x563ab73a97d0 2023-10-16T10:05:18.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.073+0000 7fef697fa640 1 -- 172.21.15.84:0/4005968376 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef7403f070 con 0x563ab73a97d0 2023-10-16T10:05:18.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.073+0000 7fef935b3740 10 client.7170 fetch_fsmap learned FSMap version 44 2023-10-16T10:05:18.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.073+0000 7fef935b3740 1 -- 172.21.15.84:0/4005968376 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x563ab726ab80 con 0x563ab73a97d0 2023-10-16T10:05:18.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.074+0000 7fef697fa640 1 -- 172.21.15.84:0/4005968376 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7c0870e0 con 0x563ab73a97d0 2023-10-16T10:05:18.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.074+0000 7fef697fa640 1 -- 172.21.15.84:0/4005968376 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fef74046c70 con 0x563ab73a97d0 2023-10-16T10:05:18.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.074+0000 7fef935b3740 10 client.7170 fetch_fsmap finished waiting for FSMap version 44 2023-10-16T10:05:18.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.074+0000 7fef935b3740 10 client.7170 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T10:05:18.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.074+0000 7fef935b3740 20 client.7170 populate_metadata read hostname 'smithi084' 2023-10-16T10:05:18.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.074+0000 7fef935b3740 1 --2- 172.21.15.84:0/4005968376 >> [v2:172.21.15.84:6836/1939610713,v1:172.21.15.84:6837/1939610713] conn(0x563ab7419f90 0x563ab741a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.074+0000 7fef935b3740 4 client.7170 mds_command: new command op to 7044 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T10:05:18.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.074+0000 7fef935b3740 1 -- 172.21.15.84:0/4005968376 --> [v2:172.21.15.84:6836/1939610713,v1:172.21.15.84:6837/1939610713] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563ab72c6700 con 0x563ab7419f90 2023-10-16T10:05:18.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.074+0000 7fef8a59c640 1 --2- 172.21.15.84:0/4005968376 >> [v2:172.21.15.84:6836/1939610713,v1:172.21.15.84:6837/1939610713] conn(0x563ab7419f90 0x563ab741a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.074+0000 7fef697fa640 10 client.7170 ms_handle_connect on v2:172.21.15.84:6836/1939610713 2023-10-16T10:05:18.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.074+0000 7fef8a59c640 1 --2- 172.21.15.84:0/4005968376 >> [v2:172.21.15.84:6836/1939610713,v1:172.21.15.84:6837/1939610713] conn(0x563ab7419f90 0x563ab741a370 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82a9611c71a9b7a9 server_cookie=41bcddbfa61261e2 in_seq=0 out_seq=0 2023-10-16T10:05:18.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.075+0000 7fef697fa640 1 -- 172.21.15.84:0/4005968376 <== mds.0 v2:172.21.15.84:6836/1939610713 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x563ab72c6700 con 0x563ab7419f90 2023-10-16T10:05:18.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.075+0000 7fef697fa640 10 client.7170 handle_command_reply: tid=0 2023-10-16T10:05:18.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.075+0000 7fef935b3740 1 client.7170 shutdown 2023-10-16T10:05:18.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.075+0000 7fef935b3740 1 -- 172.21.15.84:0/4005968376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73a97d0 msgr2=0x563ab738e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:18.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.075+0000 7fef935b3740 1 --2- 172.21.15.84:0/4005968376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73a97d0 0x563ab738e6a0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fef74009860 tx=0x7fef74031c40 comp rx=0 tx=0).stop 2023-10-16T10:05:18.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.075+0000 7fef935b3740 1 -- 172.21.15.84:0/4005968376 shutdown_connections 2023-10-16T10:05:18.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.075+0000 7fef935b3740 1 --2- 172.21.15.84:0/4005968376 >> [v2:172.21.15.84:6836/1939610713,v1:172.21.15.84:6837/1939610713] conn(0x563ab7419f90 0x563ab741a370 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.075+0000 7fef935b3740 1 --2- 172.21.15.84:0/4005968376 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab73afb00 0x563ab738ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.075+0000 7fef697fa640 0 client.7170 ms_handle_reset on v2:172.21.15.84:6836/1939610713 2023-10-16T10:05:18.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.075+0000 7fef935b3740 1 --2- 172.21.15.84:0/4005968376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab73a97d0 0x563ab738e6a0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.075+0000 7fef935b3740 1 --2- 172.21.15.84:0/4005968376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab738f340 0x563ab7416da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.075+0000 7fef935b3740 1 -- 172.21.15.84:0/4005968376 >> 172.21.15.84:0/4005968376 conn(0x563ab7367570 msgr2=0x563ab738a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:18.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.075+0000 7fef935b3740 1 -- 172.21.15.84:0/4005968376 shutdown_connections 2023-10-16T10:05:18.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.075+0000 7fef935b3740 1 -- 172.21.15.84:0/4005968376 wait complete. 2023-10-16T10:05:18.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.075+0000 7fef935b3740 20 client.7170 trim_cache size 0 max 16384 2023-10-16T10:05:18.090 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T10:05:18.091 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "63fb4d68-f881-4694-b880-d10369105c0e", 2023-10-16T10:05:18.091 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-16T10:05:18.091 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 7044, 2023-10-16T10:05:18.091 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-16T10:05:18.091 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-16T10:05:18.092 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-16T10:05:18.092 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 6.7073693089999997, 2023-10-16T10:05:18.092 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 44, 2023-10-16T10:05:18.092 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 95, 2023-10-16T10:05:18.092 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 95, 2023-10-16T10:05:18.093 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 12.093265476999999 2023-10-16T10:05:18.093 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T10:05:18.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.076+0000 7fef90a1f640 1 -- 172.21.15.84:0/4119242772 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fef6c06af90 msgr2=0x7fef6c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:18.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.076+0000 7fef90a1f640 1 --2- 172.21.15.84:0/4119242772 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fef6c06af90 0x7fef6c06d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fef74036000 tx=0x7fef74008030 comp rx=0 tx=0).stop 2023-10-16T10:05:18.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.076+0000 7fef90a1f640 1 -- 172.21.15.84:0/4119242772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef8c06a4b0 msgr2=0x7fef8c0b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:18.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.076+0000 7fef90a1f640 1 --2- 172.21.15.84:0/4119242772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef8c06a4b0 0x7fef8c0b2300 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00c480 tx=0x7fef7c00c940 comp rx=0 tx=0).stop 2023-10-16T10:05:18.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.076+0000 7fef90a1f640 1 -- 172.21.15.84:0/4119242772 shutdown_connections 2023-10-16T10:05:18.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.076+0000 7fef90a1f640 1 --2- 172.21.15.84:0/4119242772 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fef6c06af90 0x7fef6c06d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.076+0000 7fef90a1f640 1 --2- 172.21.15.84:0/4119242772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef8c15ad70 0x7fef8c1ef7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.076+0000 7fef90a1f640 1 --2- 172.21.15.84:0/4119242772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef8c0c9ca0 0x7fef8c15d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.076+0000 7fef90a1f640 1 --2- 172.21.15.84:0/4119242772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef8c06a4b0 0x7fef8c0b2300 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.076+0000 7fef90a1f640 1 -- 172.21.15.84:0/4119242772 >> 172.21.15.84:0/4119242772 conn(0x7fef8c0ce9b0 msgr2=0x7fef8c0afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:18.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.076+0000 7fef90a1f640 1 -- 172.21.15.84:0/4119242772 shutdown_connections 2023-10-16T10:05:18.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.076+0000 7fef90a1f640 1 -- 172.21.15.84:0/4119242772 wait complete. 2023-10-16T10:05:18.097 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T10:05:18.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.306+0000 7f0c54fc9640 1 Processor -- start 2023-10-16T10:05:18.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.307+0000 7f0c54fc9640 1 -- start start 2023-10-16T10:05:18.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.308+0000 7f0c54fc9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0c500cd0c0 0x7f0c501615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.308+0000 7f0c54fc9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5015e3e0 con 0x7f0c5015db00 2023-10-16T10:05:18.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.308+0000 7f0c54fc9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c500cd4a0 con 0x7f0c500b0410 2023-10-16T10:05:18.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.308+0000 7f0c54fc9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c5015e100 con 0x7f0c500cd0c0 2023-10-16T10:05:18.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.309+0000 7f0c4ed9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0c5015db00 0x7f0c5006f550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:59346/0 (socket says 172.21.15.84:59346) 2023-10-16T10:05:18.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.309+0000 7f0c4ed9d640 1 -- 172.21.15.84:0/2765677670 learned_addr learned my addr 172.21.15.84:0/2765677670 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:18.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.309+0000 7f0c4e59c640 1 --2- 172.21.15.84:0/2765677670 >> v2:172.21.15.157:3300/0 conn(0x7f0c500cd0c0 0x7f0c501615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.309+0000 7f0c4d59a640 1 -- 172.21.15.84:0/2765677670 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513878312 0 0) 0x7f0c5015e3e0 con 0x7f0c5015db00 2023-10-16T10:05:18.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.309+0000 7f0c4d59a640 1 -- 172.21.15.84:0/2765677670 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c3c003520 con 0x7f0c5015db00 2023-10-16T10:05:18.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4d59a640 1 -- 172.21.15.84:0/2765677670 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2655308320 0 0) 0x7f0c3c003520 con 0x7f0c5015db00 2023-10-16T10:05:18.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4d59a640 1 -- 172.21.15.84:0/2765677670 >> v2:172.21.15.157:3300/0 conn(0x7f0c500cd0c0 msgr2=0x7f0c501615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:18.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4d59a640 1 --2- 172.21.15.84:0/2765677670 >> v2:172.21.15.157:3300/0 conn(0x7f0c500cd0c0 0x7f0c501615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4d59a640 1 -- 172.21.15.84:0/2765677670 >> v1:172.21.15.157:6789/0 conn(0x7f0c500b0410 legacy=0x7f0c5015d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:18.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4d59a640 1 -- 172.21.15.84:0/2765677670 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c50160b10 con 0x7f0c5015db00 2023-10-16T10:05:18.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4d59a640 1 -- 172.21.15.84:0/2765677670 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f0c480019f0 con 0x7f0c5015db00 2023-10-16T10:05:18.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4d59a640 1 -- 172.21.15.84:0/2765677670 >> v1:172.21.15.84:6789/0 conn(0x7f0c5015db00 legacy=0x7f0c5006f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:18.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4d59a640 1 --2- 172.21.15.84:0/2765677670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3c0039c0 0x7f0c3c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4d59a640 1 --2- 172.21.15.84:0/2765677670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c3c0047c0 0x7f0c3c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4d59a640 1 --2- 172.21.15.84:0/2765677670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c3c0051d0 0x7f0c3c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4d59a640 1 -- 172.21.15.84:0/2765677670 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5015e100 con 0x7f0c3c0039c0 2023-10-16T10:05:18.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4d59a640 1 -- 172.21.15.84:0/2765677670 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c00c120 con 0x7f0c3c0047c0 2023-10-16T10:05:18.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4d59a640 1 -- 172.21.15.84:0/2765677670 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c00c2e0 con 0x7f0c3c0051d0 2023-10-16T10:05:18.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4e59c640 1 --2- 172.21.15.84:0/2765677670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3c0039c0 0x7f0c3c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4e59c640 1 -- 172.21.15.84:0/2765677670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c3c0051d0 msgr2=0x7f0c3c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:18.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4e59c640 1 --2- 172.21.15.84:0/2765677670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c3c0051d0 0x7f0c3c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4e59c640 1 -- 172.21.15.84:0/2765677670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c3c0047c0 msgr2=0x7f0c3c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:18.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4e59c640 1 --2- 172.21.15.84:0/2765677670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c3c0047c0 0x7f0c3c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4e59c640 1 -- 172.21.15.84:0/2765677670 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c003520 con 0x7f0c3c0039c0 2023-10-16T10:05:18.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.310+0000 7f0c4ed9d640 1 -- 172.21.15.84:0/2765677670 reap_dead start 2023-10-16T10:05:18.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.312+0000 7f0c4e59c640 1 --2- 172.21.15.84:0/2765677670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3c0039c0 0x7f0c3c003de0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009a90 tx=0x7f0c3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a51f52fec29c2b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:18.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.312+0000 7f0c4d59a640 1 -- 172.21.15.84:0/2765677670 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3802c680 con 0x7f0c3c0039c0 2023-10-16T10:05:18.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.312+0000 7f0c4d59a640 1 -- 172.21.15.84:0/2765677670 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3802c7e0 con 0x7f0c3c0039c0 2023-10-16T10:05:18.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.312+0000 7f0c4d59a640 1 -- 172.21.15.84:0/2765677670 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3802cb70 con 0x7f0c3c0039c0 2023-10-16T10:05:18.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.312+0000 7f0c54fc9640 1 -- 172.21.15.84:0/2765677670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3c0039c0 msgr2=0x7f0c3c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:18.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.312+0000 7f0c54fc9640 1 --2- 172.21.15.84:0/2765677670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3c0039c0 0x7f0c3c003de0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009a90 tx=0x7f0c3c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.313+0000 7f0c54fc9640 1 -- 172.21.15.84:0/2765677670 shutdown_connections 2023-10-16T10:05:18.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.313+0000 7f0c54fc9640 1 --2- 172.21.15.84:0/2765677670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3c0039c0 0x7f0c3c003de0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.313+0000 7f0c54fc9640 1 -- 172.21.15.84:0/2765677670 >> 172.21.15.84:0/2765677670 conn(0x7f0c500cb410 msgr2=0x7f0c500cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:18.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.313+0000 7f0c54fc9640 1 -- 172.21.15.84:0/2765677670 shutdown_connections 2023-10-16T10:05:18.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.313+0000 7f0c54fc9640 1 -- 172.21.15.84:0/2765677670 wait complete. 2023-10-16T10:05:18.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.314+0000 7f0c54fc9640 1 Processor -- start 2023-10-16T10:05:18.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.315+0000 7f0c54fc9640 1 -- start start 2023-10-16T10:05:18.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.315+0000 7f0c54fc9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3c0039c0 0x7f0c50156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.315+0000 7f0c54fc9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c501e67e0 0x7f0c501e6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.315+0000 7f0c54fc9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c501e87d0 0x7f0c501e8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.315+0000 7f0c54fc9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c50156d70 con 0x7f0c3c0039c0 2023-10-16T10:05:18.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.315+0000 7f0c54fc9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c500af400 con 0x7f0c501e67e0 2023-10-16T10:05:18.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.315+0000 7f0c54fc9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c501e71a0 con 0x7f0c501e87d0 2023-10-16T10:05:18.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.316+0000 7f0c4e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3c0039c0 0x7f0c50156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.316+0000 7f0c4e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3c0039c0 0x7f0c50156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35604/0 (socket says 172.21.15.84:35604) 2023-10-16T10:05:18.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.316+0000 7f0c4e59c640 1 -- 172.21.15.84:0/3567277045 learned_addr learned my addr 172.21.15.84:0/3567277045 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:18.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.316+0000 7f0c4e59c640 1 -- 172.21.15.84:0/3567277045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c501e87d0 msgr2=0x7f0c501e8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:18.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.316+0000 7f0c4e59c640 1 --2- 172.21.15.84:0/3567277045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c501e87d0 0x7f0c501e8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.316+0000 7f0c4e59c640 1 -- 172.21.15.84:0/3567277045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c501e67e0 msgr2=0x7f0c501e6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:18.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.316+0000 7f0c4e59c640 1 --2- 172.21.15.84:0/3567277045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c501e67e0 0x7f0c501e6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.316+0000 7f0c4e59c640 1 -- 172.21.15.84:0/3567277045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c500aff40 con 0x7f0c3c0039c0 2023-10-16T10:05:18.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.316+0000 7f0c4e59c640 1 --2- 172.21.15.84:0/3567277045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3c0039c0 0x7f0c50156830 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0c3802c220 tx=0x7f0c38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:18.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.317+0000 7f0c46ffd640 1 -- 172.21.15.84:0/3567277045 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c38003810 con 0x7f0c3c0039c0 2023-10-16T10:05:18.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.317+0000 7f0c46ffd640 1 -- 172.21.15.84:0/3567277045 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c38003970 con 0x7f0c3c0039c0 2023-10-16T10:05:18.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.317+0000 7f0c54fc9640 1 -- 172.21.15.84:0/3567277045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c500af610 con 0x7f0c3c0039c0 2023-10-16T10:05:18.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.317+0000 7f0c46ffd640 1 -- 172.21.15.84:0/3567277045 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c38003d00 con 0x7f0c3c0039c0 2023-10-16T10:05:18.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.317+0000 7f0c54fc9640 1 -- 172.21.15.84:0/3567277045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c500afbd0 con 0x7f0c3c0039c0 2023-10-16T10:05:18.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.318+0000 7f0c54fc9640 1 -- 172.21.15.84:0/3567277045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5004e160 con 0x7f0c3c0039c0 2023-10-16T10:05:18.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.319+0000 7f0c46ffd640 1 -- 172.21.15.84:0/3567277045 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0c38034440 con 0x7f0c3c0039c0 2023-10-16T10:05:18.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.320+0000 7f0c46ffd640 1 --2- 172.21.15.84:0/3567277045 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f0c3006af90 0x7f0c3006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:18.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.320+0000 7f0c46ffd640 1 -- 172.21.15.84:0/3567277045 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c380ab790 con 0x7f0c3c0039c0 2023-10-16T10:05:18.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.320+0000 7f0c4dd9b640 1 --2- 172.21.15.84:0/3567277045 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f0c3006af90 0x7f0c3006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:18.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.321+0000 7f0c4dd9b640 1 --2- 172.21.15.84:0/3567277045 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f0c3006af90 0x7f0c3006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0c500c8160 tx=0x7f0c40007990 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:18.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.324+0000 7f0c46ffd640 1 -- 172.21.15.84:0/3567277045 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0c38076900 con 0x7f0c3c0039c0 2023-10-16T10:05:18.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.561+0000 7f0c54fc9640 1 -- 172.21.15.84:0/3567277045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0c500b4910 con 0x7f0c3c0039c0 2023-10-16T10:05:18.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.561+0000 7f0c46ffd640 1 -- 172.21.15.84:0/3567277045 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0c38003e60 con 0x7f0c3c0039c0 2023-10-16T10:05:18.563 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T10:05:18.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.564+0000 7f0c54fc9640 1 -- 172.21.15.84:0/3567277045 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f0c3006af90 msgr2=0x7f0c3006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:18.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.564+0000 7f0c54fc9640 1 --2- 172.21.15.84:0/3567277045 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f0c3006af90 0x7f0c3006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0c500c8160 tx=0x7f0c40007990 comp rx=0 tx=0).stop 2023-10-16T10:05:18.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.564+0000 7f0c54fc9640 1 -- 172.21.15.84:0/3567277045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3c0039c0 msgr2=0x7f0c50156830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:18.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.564+0000 7f0c54fc9640 1 --2- 172.21.15.84:0/3567277045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3c0039c0 0x7f0c50156830 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0c3802c220 tx=0x7f0c38004730 comp rx=0 tx=0).stop 2023-10-16T10:05:18.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.564+0000 7f0c54fc9640 1 -- 172.21.15.84:0/3567277045 shutdown_connections 2023-10-16T10:05:18.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.564+0000 7f0c54fc9640 1 --2- 172.21.15.84:0/3567277045 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f0c3006af90 0x7f0c3006d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.564+0000 7f0c54fc9640 1 --2- 172.21.15.84:0/3567277045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c501e87d0 0x7f0c501e8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.564+0000 7f0c54fc9640 1 --2- 172.21.15.84:0/3567277045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c501e67e0 0x7f0c501e6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.564+0000 7f0c54fc9640 1 --2- 172.21.15.84:0/3567277045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3c0039c0 0x7f0c50156830 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:18.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.564+0000 7f0c54fc9640 1 -- 172.21.15.84:0/3567277045 >> 172.21.15.84:0/3567277045 conn(0x7f0c500cb410 msgr2=0x7f0c5015b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:18.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.564+0000 7f0c54fc9640 1 -- 172.21.15.84:0/3567277045 shutdown_connections 2023-10-16T10:05:18.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:18.564+0000 7f0c54fc9640 1 -- 172.21.15.84:0/3567277045 wait complete. 2023-10-16T10:05:18.585 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T10:05:18.586 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T10:05:18.586 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-16T10:05:18.586 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T10:05:18.587 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T10:05:18.587 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T10:05:18.587 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T10:05:18.588 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T10:05:18.588 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:05:18.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:18.589 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-16T10:05:18.644 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T10:05:18.645 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T10:05:18.645 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T10:05:18.645 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:05:18.645 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T10:05:18.646 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:05:18.646 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:05:18.646 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:05:18.646 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-16T10:05:18.646 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:05:18.647 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:05:18.647 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-16T10:05:18.647 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:05:18.647 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:05:18.648 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T10:05:18.648 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:05:18.648 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T10:05:18.648 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 41432sec preferred_lft 41432sec 2023-10-16T10:05:18.648 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-16T10:05:18.648 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:05:18.649 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T10:05:18.649 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:05:18.649 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T10:05:18.649 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:05:18.649 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::20a8:eaff:fe9d:5cea/64 scope link 2023-10-16T10:05:18.650 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:05:18.650 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:05:18.650 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T10:05:18.650 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1854:13ff:fe47:be92/64 scope link 2023-10-16T10:05:18.651 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:05:18.651 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:05:18.651 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether ee:d5:41:a3:a5:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T10:05:18.651 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ecd5:41ff:fea3:a570/64 scope link 2023-10-16T10:05:18.651 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:05:18.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:18.653 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T10:05:18.744 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-16T10:05:18.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:18.749 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-16T10:05:18.804 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-16T10:05:18.804 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T10:05:18.805 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T10:05:18.805 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T10:05:18.805 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T10:05:18.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:18.806 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T10:05:18.806 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T10:05:18.807 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T10:05:18.807 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T10:05:18.807 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T10:05:18.808 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T10:05:18.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:18.989 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T10:05:19.043 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T10:05:19.043 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T10:05:19.044 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:05:19.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:19.045 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-16T10:05:19.140 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:05:19.141 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T10:05:19.197 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:05:19.197 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:05:19.198 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:05:19.251 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-16T10:05:19.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:19.252 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:05:19.398 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:05:19.398 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T10:05:19.431 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:05:19.431 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T10:05:19.431 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T10:05:19.432 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with new device syntax: 0@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T10:05:19.432 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T10:05:19.434 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-16T10:05:19.434 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2023-10-16T10:05:19.434 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T10:05:19.434 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T10:05:19.435 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T10:05:19.437 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T10:05:19.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:19.438 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:05:19.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:19.516 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T10:05:19.579 INFO:teuthology.orchestra.run.smithi084.stdout:client7179 2023-10-16T10:05:19.579 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:05:19.579 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:05:19.626 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T10:05:19.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.885+0000 7f6e1db78640 1 Processor -- start 2023-10-16T10:05:19.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.886+0000 7f6e1db78640 1 -- start start 2023-10-16T10:05:19.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.888+0000 7f6e1db78640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6e1806a4b0 0x7f6e180c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:19.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.888+0000 7f6e1db78640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e180ac890 con 0x7f6e1815d150 2023-10-16T10:05:19.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.888+0000 7f6e1db78640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e180ad7a0 con 0x7f6e180ad350 2023-10-16T10:05:19.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.888+0000 7f6e1db78640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e1815d530 con 0x7f6e1806a4b0 2023-10-16T10:05:19.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.888+0000 7f6e177fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6e1815d150 0x7f6e18160b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:59360/0 (socket says 172.21.15.84:59360) 2023-10-16T10:05:19.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.888+0000 7f6e177fe640 1 -- 172.21.15.84:0/2886150543 learned_addr learned my addr 172.21.15.84:0/2886150543 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:19.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.888+0000 7f6e17fff640 1 --2- 172.21.15.84:0/2886150543 >> v2:172.21.15.84:3300/0 conn(0x7f6e1806a4b0 0x7f6e180c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:19.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.888+0000 7f6e17fff640 1 -- 172.21.15.84:0/2886150543 >> v1:172.21.15.157:6789/0 conn(0x7f6e180ad350 legacy=0x7f6e18069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:19.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.888+0000 7f6e17fff640 1 -- 172.21.15.84:0/2886150543 >> v1:172.21.15.84:6789/0 conn(0x7f6e1815d150 legacy=0x7f6e18160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:19.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.888+0000 7f6e17fff640 1 -- 172.21.15.84:0/2886150543 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e18161650 con 0x7f6e1806a4b0 2023-10-16T10:05:19.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.888+0000 7f6e167fc640 1 -- 172.21.15.84:0/2886150543 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244187695 0 0) 0x7f6e180ac890 con 0x7f6e1815d150 2023-10-16T10:05:19.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.889+0000 7f6e17fff640 1 --2- 172.21.15.84:0/2886150543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e1806a4b0 0x7f6e180c9000 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f6e100097a0 tx=0x7f6e1002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f7e5a2082d10b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:19.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.890+0000 7f6e167fc640 1 -- 172.21.15.84:0/2886150543 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e10031830 con 0x7f6e1806a4b0 2023-10-16T10:05:19.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.890+0000 7f6e167fc640 1 -- 172.21.15.84:0/2886150543 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e10031990 con 0x7f6e1806a4b0 2023-10-16T10:05:19.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.890+0000 7f6e167fc640 1 -- 172.21.15.84:0/2886150543 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e10031d20 con 0x7f6e1806a4b0 2023-10-16T10:05:19.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.890+0000 7f6e1db78640 1 -- 172.21.15.84:0/2886150543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e1806a4b0 msgr2=0x7f6e180c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:19.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.890+0000 7f6e1db78640 1 --2- 172.21.15.84:0/2886150543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e1806a4b0 0x7f6e180c9000 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f6e100097a0 tx=0x7f6e1002b8d0 comp rx=0 tx=0).stop 2023-10-16T10:05:19.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.890+0000 7f6e1db78640 1 -- 172.21.15.84:0/2886150543 shutdown_connections 2023-10-16T10:05:19.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.890+0000 7f6e1db78640 1 --2- 172.21.15.84:0/2886150543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e1806a4b0 0x7f6e180c9000 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:19.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.890+0000 7f6e1db78640 1 -- 172.21.15.84:0/2886150543 >> 172.21.15.84:0/2886150543 conn(0x7f6e180ce9b0 msgr2=0x7f6e180cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:19.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.890+0000 7f6e1db78640 1 -- 172.21.15.84:0/2886150543 shutdown_connections 2023-10-16T10:05:19.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.890+0000 7f6e1db78640 1 -- 172.21.15.84:0/2886150543 wait complete. 2023-10-16T10:05:19.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.892+0000 7f6e1db78640 1 Processor -- start 2023-10-16T10:05:19.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.892+0000 7f6e1db78640 1 -- start start 2023-10-16T10:05:19.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.893+0000 7f6e1db78640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e1806a4b0 0x7f6e18155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:19.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.893+0000 7f6e1db78640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e180ad350 0x7f6e180c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:19.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.893+0000 7f6e1db78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e1815d150 0x7f6e180c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:19.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.893+0000 7f6e1db78640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e181f3410 con 0x7f6e180ad350 2023-10-16T10:05:19.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.893+0000 7f6e1db78640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e180c6ba0 con 0x7f6e1806a4b0 2023-10-16T10:05:19.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.893+0000 7f6e1db78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e180c2b30 con 0x7f6e1815d150 2023-10-16T10:05:19.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.893+0000 7f6e16ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e180ad350 0x7f6e180c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:19.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.893+0000 7f6e16ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e180ad350 0x7f6e180c79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35622/0 (socket says 172.21.15.84:35622) 2023-10-16T10:05:19.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.893+0000 7f6e16ffd640 1 -- 172.21.15.84:0/1315989292 learned_addr learned my addr 172.21.15.84:0/1315989292 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:19.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.894+0000 7f6e16ffd640 1 -- 172.21.15.84:0/1315989292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e1815d150 msgr2=0x7f6e180c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:19.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.894+0000 7f6e16ffd640 1 --2- 172.21.15.84:0/1315989292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e1815d150 0x7f6e180c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:19.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.894+0000 7f6e16ffd640 1 -- 172.21.15.84:0/1315989292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e1806a4b0 msgr2=0x7f6e18155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:19.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.894+0000 7f6e16ffd640 1 --2- 172.21.15.84:0/1315989292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e1806a4b0 0x7f6e18155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:19.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.894+0000 7f6e16ffd640 1 -- 172.21.15.84:0/1315989292 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e10009400 con 0x7f6e180ad350 2023-10-16T10:05:19.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.894+0000 7f6e16ffd640 1 --2- 172.21.15.84:0/1315989292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e180ad350 0x7f6e180c79b0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000c430 tx=0x7f6e0000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:19.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.894+0000 7f6e0ffff640 1 -- 172.21.15.84:0/1315989292 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e00011790 con 0x7f6e180ad350 2023-10-16T10:05:19.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.894+0000 7f6e0ffff640 1 -- 172.21.15.84:0/1315989292 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e000118f0 con 0x7f6e180ad350 2023-10-16T10:05:19.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.894+0000 7f6e1db78640 1 -- 172.21.15.84:0/1315989292 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e180c2850 con 0x7f6e180ad350 2023-10-16T10:05:19.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.894+0000 7f6e0ffff640 1 -- 172.21.15.84:0/1315989292 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e00011ce0 con 0x7f6e180ad350 2023-10-16T10:05:19.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.894+0000 7f6e1db78640 1 -- 172.21.15.84:0/1315989292 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e180c7120 con 0x7f6e180ad350 2023-10-16T10:05:19.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.896+0000 7f6e1db78640 1 -- 172.21.15.84:0/1315989292 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e180c6a00 con 0x7f6e180ad350 2023-10-16T10:05:19.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.897+0000 7f6e0ffff640 1 -- 172.21.15.84:0/1315989292 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6e0001c460 con 0x7f6e180ad350 2023-10-16T10:05:19.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.898+0000 7f6e0ffff640 1 --2- 172.21.15.84:0/1315989292 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f6df806af90 0x7f6df806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:19.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.898+0000 7f6e0ffff640 1 -- 172.21.15.84:0/1315989292 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e000896c0 con 0x7f6e180ad350 2023-10-16T10:05:19.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.898+0000 7f6e177fe640 1 --2- 172.21.15.84:0/1315989292 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f6df806af90 0x7f6df806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:19.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.899+0000 7f6e177fe640 1 --2- 172.21.15.84:0/1315989292 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f6df806af90 0x7f6df806d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6e08001560 tx=0x7f6e08007ca0 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:19.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:19.903+0000 7f6e0ffff640 1 -- 172.21.15.84:0/1315989292 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6e180c6a00 con 0x7f6e180ad350 2023-10-16T10:05:20.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.121+0000 7f6e1db78640 1 -- 172.21.15.84:0/1315989292 --> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e180acb50 con 0x7f6df806af90 2023-10-16T10:05:20.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.124+0000 7f6e0ffff640 1 -- 172.21.15.84:0/1315989292 <== mgr.6786 v2:172.21.15.84:6832/147697688 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6e08004b60 con 0x7f6df806af90 2023-10-16T10:05:20.126 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T10:05:20.126 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:20.126 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephfs" 2023-10-16T10:05:20.126 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:20.126 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T10:05:20.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.126+0000 7f6e1db78640 1 -- 172.21.15.84:0/1315989292 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f6df806af90 msgr2=0x7f6df806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:20.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.126+0000 7f6e1db78640 1 --2- 172.21.15.84:0/1315989292 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f6df806af90 0x7f6df806d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6e08001560 tx=0x7f6e08007ca0 comp rx=0 tx=0).stop 2023-10-16T10:05:20.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.126+0000 7f6e1db78640 1 -- 172.21.15.84:0/1315989292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e180ad350 msgr2=0x7f6e180c79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:20.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.126+0000 7f6e1db78640 1 --2- 172.21.15.84:0/1315989292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e180ad350 0x7f6e180c79b0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000c430 tx=0x7f6e0000c8f0 comp rx=0 tx=0).stop 2023-10-16T10:05:20.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.127+0000 7f6e1db78640 1 -- 172.21.15.84:0/1315989292 shutdown_connections 2023-10-16T10:05:20.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.127+0000 7f6e1db78640 1 --2- 172.21.15.84:0/1315989292 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f6df806af90 0x7f6df806d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:20.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.127+0000 7f6e1db78640 1 --2- 172.21.15.84:0/1315989292 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e1815d150 0x7f6e180c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:20.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.127+0000 7f6e1db78640 1 --2- 172.21.15.84:0/1315989292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e180ad350 0x7f6e180c79b0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:20.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.127+0000 7f6e1db78640 1 --2- 172.21.15.84:0/1315989292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e1806a4b0 0x7f6e18155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:20.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.127+0000 7f6e1db78640 1 -- 172.21.15.84:0/1315989292 >> 172.21.15.84:0/1315989292 conn(0x7f6e180ce9b0 msgr2=0x7f6e18069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:20.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.127+0000 7f6e1db78640 1 -- 172.21.15.84:0/1315989292 shutdown_connections 2023-10-16T10:05:20.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.127+0000 7f6e1db78640 1 -- 172.21.15.84:0/1315989292 wait complete. 2023-10-16T10:05:20.146 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T10:05:20.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.356+0000 7fb561015640 1 Processor -- start 2023-10-16T10:05:20.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.356+0000 7fb561015640 1 -- start start 2023-10-16T10:05:20.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.357+0000 7fb561015640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb55c06a4b0 0x7fb55c0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:20.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.357+0000 7fb561015640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb55c0c9ca0 0x7fb55c0ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:20.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.357+0000 7fb561015640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55c0cb030 con 0x7fb55c15ad70 2023-10-16T10:05:20.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.357+0000 7fb561015640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55c0cae30 con 0x7fb55c0c9ca0 2023-10-16T10:05:20.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.357+0000 7fb561015640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55c0c91c0 con 0x7fb55c06a4b0 2023-10-16T10:05:20.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.357+0000 7fb55ad9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb55c15ad70 0x7fb55c069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:59366/0 (socket says 172.21.15.84:59366) 2023-10-16T10:05:20.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.357+0000 7fb55ad9d640 1 -- 172.21.15.84:0/1470626511 learned_addr learned my addr 172.21.15.84:0/1470626511 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:20.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.357+0000 7fb55b59e640 1 --2- 172.21.15.84:0/1470626511 >> v2:172.21.15.84:3300/0 conn(0x7fb55c0c9ca0 0x7fb55c0ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:20.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.358+0000 7fb553fff640 1 --2- 172.21.15.84:0/1470626511 >> v2:172.21.15.157:3300/0 conn(0x7fb55c06a4b0 0x7fb55c0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:20.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.358+0000 7fb55a59c640 1 -- 172.21.15.84:0/1470626511 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646059002 0 0) 0x7fb55c0cb030 con 0x7fb55c15ad70 2023-10-16T10:05:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.358+0000 7fb55a59c640 1 -- 172.21.15.84:0/1470626511 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb548003520 con 0x7fb55c15ad70 2023-10-16T10:05:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.358+0000 7fb55b59e640 1 -- 172.21.15.84:0/1470626511 >> v2:172.21.15.157:3300/0 conn(0x7fb55c06a4b0 msgr2=0x7fb55c0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.358+0000 7fb55b59e640 1 --2- 172.21.15.84:0/1470626511 >> v2:172.21.15.157:3300/0 conn(0x7fb55c06a4b0 0x7fb55c0ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.358+0000 7fb55b59e640 1 -- 172.21.15.84:0/1470626511 >> v1:172.21.15.84:6789/0 conn(0x7fb55c15ad70 legacy=0x7fb55c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.358+0000 7fb55b59e640 1 -- 172.21.15.84:0/1470626511 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c0c94a0 con 0x7fb55c0c9ca0 2023-10-16T10:05:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.359+0000 7fb55b59e640 1 --2- 172.21.15.84:0/1470626511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb55c0c9ca0 0x7fb55c0ca8f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0ce870 tx=0x7fb55402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e3075378628bf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.359+0000 7fb55a59c640 1 -- 172.21.15.84:0/1470626511 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55402cc80 con 0x7fb55c0c9ca0 2023-10-16T10:05:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.360+0000 7fb55a59c640 1 -- 172.21.15.84:0/1470626511 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55402cde0 con 0x7fb55c0c9ca0 2023-10-16T10:05:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.360+0000 7fb55a59c640 1 -- 172.21.15.84:0/1470626511 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb554034660 con 0x7fb55c0c9ca0 2023-10-16T10:05:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.360+0000 7fb561015640 1 -- 172.21.15.84:0/1470626511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb55c0c9ca0 msgr2=0x7fb55c0ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.360+0000 7fb561015640 1 --2- 172.21.15.84:0/1470626511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb55c0c9ca0 0x7fb55c0ca8f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0ce870 tx=0x7fb55402bed0 comp rx=0 tx=0).stop 2023-10-16T10:05:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.360+0000 7fb561015640 1 -- 172.21.15.84:0/1470626511 shutdown_connections 2023-10-16T10:05:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.360+0000 7fb561015640 1 --2- 172.21.15.84:0/1470626511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb55c0c9ca0 0x7fb55c0ca8f0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.360+0000 7fb561015640 1 --2- 172.21.15.84:0/1470626511 >> v2:172.21.15.157:3300/0 conn(0x7fb55c06a4b0 0x7fb55c0ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.360+0000 7fb561015640 1 -- 172.21.15.84:0/1470626511 >> 172.21.15.84:0/1470626511 conn(0x7fb55c0ce9b0 msgr2=0x7fb55c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.360+0000 7fb561015640 1 -- 172.21.15.84:0/1470626511 shutdown_connections 2023-10-16T10:05:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.360+0000 7fb561015640 1 -- 172.21.15.84:0/1470626511 wait complete. 2023-10-16T10:05:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.362+0000 7fb561015640 1 Processor -- start 2023-10-16T10:05:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.362+0000 7fb561015640 1 -- start start 2023-10-16T10:05:20.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.363+0000 7fb561015640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb55c06a4b0 0x7fb55c159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:20.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.363+0000 7fb561015640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb55c0c9ca0 0x7fb55c0c3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:20.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.363+0000 7fb561015640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb55c15ad70 0x7fb55c0c3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:20.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.363+0000 7fb561015640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c0c4a40 con 0x7fb55c15ad70 2023-10-16T10:05:20.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.363+0000 7fb561015640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c0c3dc0 con 0x7fb55c06a4b0 2023-10-16T10:05:20.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.363+0000 7fb561015640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c0c40a0 con 0x7fb55c0c9ca0 2023-10-16T10:05:20.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.364+0000 7fb55b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb55c15ad70 0x7fb55c0c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:20.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.364+0000 7fb55b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb55c15ad70 0x7fb55c0c3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35650/0 (socket says 172.21.15.84:35650) 2023-10-16T10:05:20.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.364+0000 7fb55b59e640 1 -- 172.21.15.84:0/3305103037 learned_addr learned my addr 172.21.15.84:0/3305103037 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:20.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.364+0000 7fb553fff640 1 --2- 172.21.15.84:0/3305103037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb55c0c9ca0 0x7fb55c0c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:20.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.364+0000 7fb55ad9d640 1 --2- 172.21.15.84:0/3305103037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb55c06a4b0 0x7fb55c159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:20.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.364+0000 7fb55b59e640 1 -- 172.21.15.84:0/3305103037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb55c0c9ca0 msgr2=0x7fb55c0c3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:20.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.364+0000 7fb55b59e640 1 --2- 172.21.15.84:0/3305103037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb55c0c9ca0 0x7fb55c0c3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:20.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.364+0000 7fb55b59e640 1 -- 172.21.15.84:0/3305103037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb55c06a4b0 msgr2=0x7fb55c159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:20.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.364+0000 7fb55b59e640 1 --2- 172.21.15.84:0/3305103037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb55c06a4b0 0x7fb55c159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:20.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.364+0000 7fb55b59e640 1 -- 172.21.15.84:0/3305103037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb554009a70 con 0x7fb55c15ad70 2023-10-16T10:05:20.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.364+0000 7fb55b59e640 1 --2- 172.21.15.84:0/3305103037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb55c15ad70 0x7fb55c0c3740 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb554009e80 tx=0x7fb554004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:20.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.365+0000 7fb5537fe640 1 -- 172.21.15.84:0/3305103037 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb554040650 con 0x7fb55c15ad70 2023-10-16T10:05:20.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.365+0000 7fb5537fe640 1 -- 172.21.15.84:0/3305103037 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5540407b0 con 0x7fb55c15ad70 2023-10-16T10:05:20.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.365+0000 7fb5537fe640 1 -- 172.21.15.84:0/3305103037 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb554040b60 con 0x7fb55c15ad70 2023-10-16T10:05:20.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.365+0000 7fb561015640 1 -- 172.21.15.84:0/3305103037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55c0c4380 con 0x7fb55c15ad70 2023-10-16T10:05:20.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.365+0000 7fb561015640 1 -- 172.21.15.84:0/3305103037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55c15a2c0 con 0x7fb55c15ad70 2023-10-16T10:05:20.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.366+0000 7fb561015640 1 -- 172.21.15.84:0/3305103037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb55c0c4b80 con 0x7fb55c15ad70 2023-10-16T10:05:20.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.367+0000 7fb5537fe640 1 -- 172.21.15.84:0/3305103037 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb554040cc0 con 0x7fb55c15ad70 2023-10-16T10:05:20.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.368+0000 7fb5537fe640 1 --2- 172.21.15.84:0/3305103037 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fb53c06af90 0x7fb53c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:20.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.368+0000 7fb5537fe640 1 -- 172.21.15.84:0/3305103037 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5540ac580 con 0x7fb55c15ad70 2023-10-16T10:05:20.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.368+0000 7fb55ad9d640 1 --2- 172.21.15.84:0/3305103037 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fb53c06af90 0x7fb53c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:20.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.369+0000 7fb55ad9d640 1 --2- 172.21.15.84:0/3305103037 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fb53c06af90 0x7fb53c06d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0011e0 tx=0x7fb54c009040 comp rx=0 tx=0).ready entity=mgr.6786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:20.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.373+0000 7fb5537fe640 1 -- 172.21.15.84:0/3305103037 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb5540773f0 con 0x7fb55c15ad70 2023-10-16T10:05:20.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.613+0000 7fb561015640 1 -- 172.21.15.84:0/3305103037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fb55c04e160 con 0x7fb55c15ad70 2023-10-16T10:05:20.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.938+0000 7fb5537fe640 1 -- 172.21.15.84:0/3305103037 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v42) v1 ==== 92+0+0 (secure 0 0 0) 0x7fb554009450 con 0x7fb55c15ad70 2023-10-16T10:05:20.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.941+0000 7fb561015640 1 -- 172.21.15.84:0/3305103037 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fb53c06af90 msgr2=0x7fb53c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:20.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.941+0000 7fb561015640 1 --2- 172.21.15.84:0/3305103037 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fb53c06af90 0x7fb53c06d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0011e0 tx=0x7fb54c009040 comp rx=0 tx=0).stop 2023-10-16T10:05:20.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.941+0000 7fb561015640 1 -- 172.21.15.84:0/3305103037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb55c15ad70 msgr2=0x7fb55c0c3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:20.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.941+0000 7fb561015640 1 --2- 172.21.15.84:0/3305103037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb55c15ad70 0x7fb55c0c3740 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb554009e80 tx=0x7fb554004a40 comp rx=0 tx=0).stop 2023-10-16T10:05:20.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.943+0000 7fb561015640 1 -- 172.21.15.84:0/3305103037 shutdown_connections 2023-10-16T10:05:20.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.943+0000 7fb561015640 1 --2- 172.21.15.84:0/3305103037 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fb53c06af90 0x7fb53c06d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:20.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.943+0000 7fb561015640 1 --2- 172.21.15.84:0/3305103037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb55c15ad70 0x7fb55c0c3740 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:20.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.943+0000 7fb561015640 1 --2- 172.21.15.84:0/3305103037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb55c0c9ca0 0x7fb55c0c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:20.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.943+0000 7fb561015640 1 --2- 172.21.15.84:0/3305103037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb55c06a4b0 0x7fb55c159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:20.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.943+0000 7fb561015640 1 -- 172.21.15.84:0/3305103037 >> 172.21.15.84:0/3305103037 conn(0x7fb55c0ce9b0 msgr2=0x7fb55c0b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:20.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.944+0000 7fb561015640 1 -- 172.21.15.84:0/3305103037 shutdown_connections 2023-10-16T10:05:20.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:20.944+0000 7fb561015640 1 -- 172.21.15.84:0/3305103037 wait complete. 2023-10-16T10:05:20.966 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T10:05:21.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.158+0000 7fd693090640 1 Processor -- start 2023-10-16T10:05:21.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.159+0000 7fd693090640 1 -- start start 2023-10-16T10:05:21.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.160+0000 7fd693090640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd68c0ae260 0x7fd68c167e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:21.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.160+0000 7fd693090640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd68c0ce800 con 0x7fd68c0ce350 2023-10-16T10:05:21.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.160+0000 7fd693090640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd68c0cdec0 con 0x7fd68c160ad0 2023-10-16T10:05:21.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.160+0000 7fd693090640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd68c0ae740 con 0x7fd68c0ae260 2023-10-16T10:05:21.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.160+0000 7fd68bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd68c0ce350 0x7fd68c160eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:59380/0 (socket says 172.21.15.84:59380) 2023-10-16T10:05:21.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.160+0000 7fd68bfff640 1 -- 172.21.15.84:0/3852131820 learned_addr learned my addr 172.21.15.84:0/3852131820 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:21.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.161+0000 7fd69162d640 1 --2- 172.21.15.84:0/3852131820 >> v2:172.21.15.157:3300/0 conn(0x7fd68c0ae260 0x7fd68c167e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:21.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.161+0000 7fd68b7fe640 1 -- 172.21.15.84:0/3852131820 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051642592 0 0) 0x7fd68c0ce800 con 0x7fd68c0ce350 2023-10-16T10:05:21.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.161+0000 7fd68b7fe640 1 -- 172.21.15.84:0/3852131820 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd678003520 con 0x7fd68c0ce350 2023-10-16T10:05:21.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.161+0000 7fd68b7fe640 1 -- 172.21.15.84:0/3852131820 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2037509130 0 0) 0x7fd678003520 con 0x7fd68c0ce350 2023-10-16T10:05:21.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.161+0000 7fd68b7fe640 1 -- 172.21.15.84:0/3852131820 >> v2:172.21.15.157:3300/0 conn(0x7fd68c0ae260 msgr2=0x7fd68c167e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:21.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.161+0000 7fd68b7fe640 1 --2- 172.21.15.84:0/3852131820 >> v2:172.21.15.157:3300/0 conn(0x7fd68c0ae260 0x7fd68c167e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.161+0000 7fd68b7fe640 1 -- 172.21.15.84:0/3852131820 >> v1:172.21.15.157:6789/0 conn(0x7fd68c160ad0 legacy=0x7fd68c0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:21.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.161+0000 7fd68b7fe640 1 -- 172.21.15.84:0/3852131820 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c0cea50 con 0x7fd68c0ce350 2023-10-16T10:05:21.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.161+0000 7fd68b7fe640 1 -- 172.21.15.84:0/3852131820 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd674001670 con 0x7fd68c0ce350 2023-10-16T10:05:21.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.161+0000 7fd68b7fe640 1 -- 172.21.15.84:0/3852131820 >> v1:172.21.15.84:6789/0 conn(0x7fd68c0ce350 legacy=0x7fd68c160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:21.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.161+0000 7fd68b7fe640 1 --2- 172.21.15.84:0/3852131820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd678003970 0x7fd678003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:21.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.161+0000 7fd68b7fe640 1 --2- 172.21.15.84:0/3852131820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6780047c0 0x7fd678004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:21.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.161+0000 7fd68b7fe640 1 --2- 172.21.15.84:0/3852131820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd678005180 0x7fd67800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:21.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.161+0000 7fd68b7fe640 1 -- 172.21.15.84:0/3852131820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c0ae740 con 0x7fd678005180 2023-10-16T10:05:21.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.161+0000 7fd68b7fe640 1 -- 172.21.15.84:0/3852131820 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67800c020 con 0x7fd678003970 2023-10-16T10:05:21.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.161+0000 7fd68b7fe640 1 -- 172.21.15.84:0/3852131820 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67800c1e0 con 0x7fd6780047c0 2023-10-16T10:05:21.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.162+0000 7fd69162d640 1 --2- 172.21.15.84:0/3852131820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd678005180 0x7fd67800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:21.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.162+0000 7fd690e2c640 1 --2- 172.21.15.84:0/3852131820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd678003970 0x7fd678003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:21.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.162+0000 7fd69162d640 1 -- 172.21.15.84:0/3852131820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6780047c0 msgr2=0x7fd678004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:21.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.162+0000 7fd69162d640 1 --2- 172.21.15.84:0/3852131820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6780047c0 0x7fd678004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.162+0000 7fd69162d640 1 -- 172.21.15.84:0/3852131820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd678003970 msgr2=0x7fd678003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:21.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.162+0000 7fd69162d640 1 --2- 172.21.15.84:0/3852131820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd678003970 0x7fd678003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.162+0000 7fd69162d640 1 -- 172.21.15.84:0/3852131820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd678003520 con 0x7fd678005180 2023-10-16T10:05:21.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.162+0000 7fd690e2c640 1 --2- 172.21.15.84:0/3852131820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd678003970 0x7fd678003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:21.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.163+0000 7fd69162d640 1 -- 172.21.15.84:0/3852131820 reap_dead start 2023-10-16T10:05:21.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.163+0000 7fd69162d640 1 --2- 172.21.15.84:0/3852131820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd678005180 0x7fd67800b980 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd684009ac0 tx=0x7fd6780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1481516c1a3f2c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:21.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.163+0000 7fd68b7fe640 1 -- 172.21.15.84:0/3852131820 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd684004020 con 0x7fd678005180 2023-10-16T10:05:21.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.163+0000 7fd68b7fe640 1 -- 172.21.15.84:0/3852131820 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd684004180 con 0x7fd678005180 2023-10-16T10:05:21.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.163+0000 7fd68b7fe640 1 -- 172.21.15.84:0/3852131820 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68402c710 con 0x7fd678005180 2023-10-16T10:05:21.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.164+0000 7fd693090640 1 -- 172.21.15.84:0/3852131820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd678005180 msgr2=0x7fd67800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:21.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.164+0000 7fd693090640 1 --2- 172.21.15.84:0/3852131820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd678005180 0x7fd67800b980 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd684009ac0 tx=0x7fd6780057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.164+0000 7fd693090640 1 -- 172.21.15.84:0/3852131820 shutdown_connections 2023-10-16T10:05:21.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.164+0000 7fd693090640 1 --2- 172.21.15.84:0/3852131820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd678005180 0x7fd67800b980 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.164+0000 7fd693090640 1 -- 172.21.15.84:0/3852131820 >> 172.21.15.84:0/3852131820 conn(0x7fd68c06bc10 msgr2=0x7fd68c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:21.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.164+0000 7fd693090640 1 -- 172.21.15.84:0/3852131820 shutdown_connections 2023-10-16T10:05:21.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.164+0000 7fd693090640 1 -- 172.21.15.84:0/3852131820 wait complete. 2023-10-16T10:05:21.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.166+0000 7fd693090640 1 Processor -- start 2023-10-16T10:05:21.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.166+0000 7fd693090640 1 -- start start 2023-10-16T10:05:21.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.167+0000 7fd693090640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd678005180 0x7fd68c14e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:21.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.167+0000 7fd693090640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd68c070090 0x7fd68c0c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:21.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.167+0000 7fd693090640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd68c14f760 0x7fd68c0b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:21.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.167+0000 7fd693090640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c14f210 con 0x7fd68c070090 2023-10-16T10:05:21.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.167+0000 7fd693090640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c15e640 con 0x7fd68c14f760 2023-10-16T10:05:21.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.167+0000 7fd693090640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c15e920 con 0x7fd678005180 2023-10-16T10:05:21.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.167+0000 7fd68bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd68c070090 0x7fd68c0c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:21.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.167+0000 7fd68bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd68c070090 0x7fd68c0c6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35676/0 (socket says 172.21.15.84:35676) 2023-10-16T10:05:21.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.167+0000 7fd68bfff640 1 -- 172.21.15.84:0/4126552782 learned_addr learned my addr 172.21.15.84:0/4126552782 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:21.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.168+0000 7fd690e2c640 1 --2- 172.21.15.84:0/4126552782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd678005180 0x7fd68c14e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:21.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.168+0000 7fd69162d640 1 --2- 172.21.15.84:0/4126552782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd68c14f760 0x7fd68c0b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:21.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.168+0000 7fd68bfff640 1 -- 172.21.15.84:0/4126552782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd678005180 msgr2=0x7fd68c14e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:21.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.168+0000 7fd68bfff640 1 --2- 172.21.15.84:0/4126552782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd678005180 0x7fd68c14e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.168+0000 7fd68bfff640 1 -- 172.21.15.84:0/4126552782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd68c14f760 msgr2=0x7fd68c0b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:21.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.168+0000 7fd68bfff640 1 --2- 172.21.15.84:0/4126552782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd68c14f760 0x7fd68c0b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.168+0000 7fd68bfff640 1 -- 172.21.15.84:0/4126552782 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c15ec00 con 0x7fd68c070090 2023-10-16T10:05:21.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.168+0000 7fd69162d640 1 --2- 172.21.15.84:0/4126552782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd68c14f760 0x7fd68c0b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:05:21.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.168+0000 7fd68bfff640 1 --2- 172.21.15.84:0/4126552782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd68c070090 0x7fd68c0c6980 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd67400c700 tx=0x7fd67400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:21.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.169+0000 7fd6897fa640 1 -- 172.21.15.84:0/4126552782 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6740074e0 con 0x7fd68c070090 2023-10-16T10:05:21.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.169+0000 7fd6897fa640 1 -- 172.21.15.84:0/4126552782 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd674007660 con 0x7fd68c070090 2023-10-16T10:05:21.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.169+0000 7fd693090640 1 -- 172.21.15.84:0/4126552782 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68c15eee0 con 0x7fd68c070090 2023-10-16T10:05:21.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.169+0000 7fd693090640 1 -- 172.21.15.84:0/4126552782 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd68c14ef30 con 0x7fd68c070090 2023-10-16T10:05:21.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.169+0000 7fd6897fa640 1 -- 172.21.15.84:0/4126552782 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd674004a20 con 0x7fd68c070090 2023-10-16T10:05:21.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.171+0000 7fd693090640 1 -- 172.21.15.84:0/4126552782 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd68c04e160 con 0x7fd68c070090 2023-10-16T10:05:21.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.171+0000 7fd6897fa640 1 -- 172.21.15.84:0/4126552782 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 42) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fd674004b80 con 0x7fd68c070090 2023-10-16T10:05:21.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.173+0000 7fd6897fa640 1 --2- 172.21.15.84:0/4126552782 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fd66c06b030 0x7fd66c06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:21.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.173+0000 7fd690e2c640 1 -- 172.21.15.84:0/4126552782 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fd66c06b030 msgr2=0x7fd66c06d4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/147697688 2023-10-16T10:05:21.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.173+0000 7fd6897fa640 1 -- 172.21.15.84:0/4126552782 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd674088de0 con 0x7fd68c070090 2023-10-16T10:05:21.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.173+0000 7fd690e2c640 1 --2- 172.21.15.84:0/4126552782 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fd66c06b030 0x7fd66c06d4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:05:21.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.177+0000 7fd6897fa640 1 -- 172.21.15.84:0/4126552782 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd6740578c0 con 0x7fd68c070090 2023-10-16T10:05:21.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.371+0000 7fd693090640 1 -- 172.21.15.84:0/4126552782 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fd68c06c950 con 0x7fd68c070090 2023-10-16T10:05:21.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.374+0000 7fd690e2c640 1 -- 172.21.15.84:0/4126552782 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fd66c06b030 msgr2=0x7fd66c06d4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/147697688 2023-10-16T10:05:21.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.374+0000 7fd690e2c640 1 --2- 172.21.15.84:0/4126552782 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fd66c06b030 0x7fd66c06d4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:05:21.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.377+0000 7fd6897fa640 1 -- 172.21.15.84:0/4126552782 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd674018020 con 0x7fd68c070090 2023-10-16T10:05:21.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.380+0000 7fd693090640 1 -- 172.21.15.84:0/4126552782 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fd66c06b030 msgr2=0x7fd66c06d4f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:05:21.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.380+0000 7fd693090640 1 --2- 172.21.15.84:0/4126552782 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fd66c06b030 0x7fd66c06d4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.380+0000 7fd693090640 1 -- 172.21.15.84:0/4126552782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd68c070090 msgr2=0x7fd68c0c6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:21.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.380+0000 7fd693090640 1 --2- 172.21.15.84:0/4126552782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd68c070090 0x7fd68c0c6980 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd67400c700 tx=0x7fd67400cbc0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.380+0000 7fd693090640 1 -- 172.21.15.84:0/4126552782 shutdown_connections 2023-10-16T10:05:21.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.380+0000 7fd693090640 1 --2- 172.21.15.84:0/4126552782 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fd66c06b030 0x7fd66c06d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.380+0000 7fd693090640 1 --2- 172.21.15.84:0/4126552782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd68c14f760 0x7fd68c0b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.380+0000 7fd693090640 1 --2- 172.21.15.84:0/4126552782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd68c070090 0x7fd68c0c6980 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.380+0000 7fd693090640 1 --2- 172.21.15.84:0/4126552782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd678005180 0x7fd68c14e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.380+0000 7fd693090640 1 -- 172.21.15.84:0/4126552782 >> 172.21.15.84:0/4126552782 conn(0x7fd68c06bc10 msgr2=0x7fd68c0afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:21.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.380+0000 7fd693090640 1 -- 172.21.15.84:0/4126552782 shutdown_connections 2023-10-16T10:05:21.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.380+0000 7fd693090640 1 -- 172.21.15.84:0/4126552782 wait complete. 2023-10-16T10:05:21.398 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T10:05:21.470 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:21.469+0000 7f84d7c2b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:05:21.502 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:21.500+0000 7f5bc940f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:05:21.527 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:21.527+0000 7f84d7c2b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:05:21.557 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:21.555+0000 7f5bc940f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:05:21.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.610+0000 7fdbcf4d1640 1 Processor -- start 2023-10-16T10:05:21.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.610+0000 7fdbcf4d1640 1 -- start start 2023-10-16T10:05:21.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.611+0000 7fdbcf4d1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdbc80b8480 0x7fdbc80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:21.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.611+0000 7fdbcf4d1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdbc80d0be0 0x7fdbc80cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:21.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.611+0000 7fdbcf4d1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc815d150 con 0x7fdbc815cd70 2023-10-16T10:05:21.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.611+0000 7fdbcf4d1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc80d1180 con 0x7fdbc80b8480 2023-10-16T10:05:21.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.612+0000 7fdbcf4d1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc80d1460 con 0x7fdbc80d0be0 2023-10-16T10:05:21.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.612+0000 7fdbce4cf640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdbc815cd70 0x7fdbc80d2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:37726/0 (socket says 172.21.15.84:37726) 2023-10-16T10:05:21.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.612+0000 7fdbce4cf640 1 -- 172.21.15.84:0/3344981631 learned_addr learned my addr 172.21.15.84:0/3344981631 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:21.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.612+0000 7fdbcdcce640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdbc80b8480 0x7fdbc80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:21.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.612+0000 7fdbcecd0640 1 --2- 172.21.15.84:0/3344981631 >> v2:172.21.15.157:3300/0 conn(0x7fdbc80d0be0 0x7fdbc80cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:21.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.612+0000 7fdbcdcce640 1 -- 172.21.15.84:0/3344981631 >> v2:172.21.15.157:3300/0 conn(0x7fdbc80d0be0 msgr2=0x7fdbc80cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:21.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.612+0000 7fdbcdcce640 1 --2- 172.21.15.84:0/3344981631 >> v2:172.21.15.157:3300/0 conn(0x7fdbc80d0be0 0x7fdbc80cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.612+0000 7fdbcdcce640 1 -- 172.21.15.84:0/3344981631 >> v1:172.21.15.157:6789/0 conn(0x7fdbc815cd70 legacy=0x7fdbc80d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:21.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.612+0000 7fdbcdcce640 1 -- 172.21.15.84:0/3344981631 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc80d1740 con 0x7fdbc80b8480 2023-10-16T10:05:21.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.613+0000 7fdbcdcce640 1 --2- 172.21.15.84:0/3344981631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc80b8480 0x7fdbc80cd7c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80b8910 tx=0x7fdbbc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f608d769a51d96 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:21.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.614+0000 7fdbcd4cd640 1 -- 172.21.15.84:0/3344981631 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc02e070 con 0x7fdbc80b8480 2023-10-16T10:05:21.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.614+0000 7fdbcd4cd640 1 -- 172.21.15.84:0/3344981631 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbbc0315a0 con 0x7fdbc80b8480 2023-10-16T10:05:21.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.614+0000 7fdbcd4cd640 1 -- 172.21.15.84:0/3344981631 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc031930 con 0x7fdbc80b8480 2023-10-16T10:05:21.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.614+0000 7fdbcf4d1640 1 -- 172.21.15.84:0/3344981631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc80b8480 msgr2=0x7fdbc80cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:21.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.614+0000 7fdbcf4d1640 1 --2- 172.21.15.84:0/3344981631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc80b8480 0x7fdbc80cd7c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80b8910 tx=0x7fdbbc02b840 comp rx=0 tx=0).stop 2023-10-16T10:05:21.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.614+0000 7fdbcf4d1640 1 -- 172.21.15.84:0/3344981631 shutdown_connections 2023-10-16T10:05:21.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.614+0000 7fdbcf4d1640 1 --2- 172.21.15.84:0/3344981631 >> v2:172.21.15.157:3300/0 conn(0x7fdbc80d0be0 0x7fdbc80cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.614+0000 7fdbcf4d1640 1 --2- 172.21.15.84:0/3344981631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc80b8480 0x7fdbc80cd7c0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.614+0000 7fdbcf4d1640 1 -- 172.21.15.84:0/3344981631 >> 172.21.15.84:0/3344981631 conn(0x7fdbc80cf570 msgr2=0x7fdbc80cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:21.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.615+0000 7fdbcf4d1640 1 -- 172.21.15.84:0/3344981631 shutdown_connections 2023-10-16T10:05:21.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.615+0000 7fdbcf4d1640 1 -- 172.21.15.84:0/3344981631 wait complete. 2023-10-16T10:05:21.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.616+0000 7fdbcf4d1640 1 Processor -- start 2023-10-16T10:05:21.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.616+0000 7fdbcf4d1640 1 -- start start 2023-10-16T10:05:21.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.617+0000 7fdbcf4d1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbc80b8480 0x7fdbc81539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:21.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.617+0000 7fdbcf4d1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc80d0be0 0x7fdbc80c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:21.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.617+0000 7fdbcf4d1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbc815cd70 0x7fdbc80cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:21.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.617+0000 7fdbcf4d1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc815dc70 con 0x7fdbc80d0be0 2023-10-16T10:05:21.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.617+0000 7fdbcf4d1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc80cb400 con 0x7fdbc80b8480 2023-10-16T10:05:21.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.617+0000 7fdbcf4d1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc815d340 con 0x7fdbc815cd70 2023-10-16T10:05:21.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.617+0000 7fdbcdcce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc80d0be0 0x7fdbc80c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:21.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.617+0000 7fdbcdcce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc80d0be0 0x7fdbc80c8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35680/0 (socket says 172.21.15.84:35680) 2023-10-16T10:05:21.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.617+0000 7fdbcdcce640 1 -- 172.21.15.84:0/3638144205 learned_addr learned my addr 172.21.15.84:0/3638144205 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:21.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.618+0000 7fdbce4cf640 1 --2- 172.21.15.84:0/3638144205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbc80b8480 0x7fdbc81539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:21.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.618+0000 7fdbcecd0640 1 --2- 172.21.15.84:0/3638144205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbc815cd70 0x7fdbc80cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:21.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.618+0000 7fdbcdcce640 1 -- 172.21.15.84:0/3638144205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbc815cd70 msgr2=0x7fdbc80cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:21.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.618+0000 7fdbcdcce640 1 --2- 172.21.15.84:0/3638144205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbc815cd70 0x7fdbc80cbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.618+0000 7fdbcdcce640 1 -- 172.21.15.84:0/3638144205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbc80b8480 msgr2=0x7fdbc81539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:21.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.618+0000 7fdbcdcce640 1 --2- 172.21.15.84:0/3638144205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbc80b8480 0x7fdbc81539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.618+0000 7fdbcdcce640 1 -- 172.21.15.84:0/3638144205 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc009400 con 0x7fdbc80d0be0 2023-10-16T10:05:21.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.618+0000 7fdbcdcce640 1 --2- 172.21.15.84:0/3638144205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc80d0be0 0x7fdbc80c8690 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc009890 tx=0x7fdbbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:21.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.618+0000 7fdbc6ffd640 1 -- 172.21.15.84:0/3638144205 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc032840 con 0x7fdbc80d0be0 2023-10-16T10:05:21.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.618+0000 7fdbc6ffd640 1 -- 172.21.15.84:0/3638144205 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbbc0329a0 con 0x7fdbc80d0be0 2023-10-16T10:05:21.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.618+0000 7fdbc6ffd640 1 -- 172.21.15.84:0/3638144205 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc032d50 con 0x7fdbc80d0be0 2023-10-16T10:05:21.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.618+0000 7fdbcf4d1640 1 -- 172.21.15.84:0/3638144205 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc815d620 con 0x7fdbc80d0be0 2023-10-16T10:05:21.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.619+0000 7fdbcf4d1640 1 -- 172.21.15.84:0/3638144205 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc80cc0f0 con 0x7fdbc80d0be0 2023-10-16T10:05:21.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.620+0000 7fdbcf4d1640 1 -- 172.21.15.84:0/3638144205 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc815de20 con 0x7fdbc80d0be0 2023-10-16T10:05:21.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.621+0000 7fdbc6ffd640 1 -- 172.21.15.84:0/3638144205 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 42) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fdbbc03b3f0 con 0x7fdbc80d0be0 2023-10-16T10:05:21.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.623+0000 7fdbc6ffd640 1 --2- 172.21.15.84:0/3638144205 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fdbb406b010 0x7fdbb406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:21.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.623+0000 7fdbce4cf640 1 -- 172.21.15.84:0/3638144205 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fdbb406b010 msgr2=0x7fdbb406d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/147697688 2023-10-16T10:05:21.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.623+0000 7fdbc6ffd640 1 -- 172.21.15.84:0/3638144205 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbbc0ab870 con 0x7fdbc80d0be0 2023-10-16T10:05:21.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.623+0000 7fdbce4cf640 1 --2- 172.21.15.84:0/3638144205 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fdbb406b010 0x7fdbb406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:05:21.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.627+0000 7fdbc6ffd640 1 -- 172.21.15.84:0/3638144205 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdbc815de20 con 0x7fdbc80d0be0 2023-10-16T10:05:21.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.820+0000 7fdbcf4d1640 1 -- 172.21.15.84:0/3638144205 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fdbc80d1d00 con 0x7fdbc80d0be0 2023-10-16T10:05:21.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.824+0000 7fdbce4cf640 1 -- 172.21.15.84:0/3638144205 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fdbb406b010 msgr2=0x7fdbb406d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/147697688 2023-10-16T10:05:21.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.824+0000 7fdbce4cf640 1 --2- 172.21.15.84:0/3638144205 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fdbb406b010 0x7fdbb406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:05:21.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.825+0000 7fdbc6ffd640 1 -- 172.21.15.84:0/3638144205 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7fdbbc076700 con 0x7fdbc80d0be0 2023-10-16T10:05:21.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.828+0000 7fdbcf4d1640 1 -- 172.21.15.84:0/3638144205 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fdbb406b010 msgr2=0x7fdbb406d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:05:21.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.828+0000 7fdbcf4d1640 1 --2- 172.21.15.84:0/3638144205 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fdbb406b010 0x7fdbb406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.828+0000 7fdbcf4d1640 1 -- 172.21.15.84:0/3638144205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc80d0be0 msgr2=0x7fdbc80c8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:21.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.828+0000 7fdbcf4d1640 1 --2- 172.21.15.84:0/3638144205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc80d0be0 0x7fdbc80c8690 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc009890 tx=0x7fdbbc0046c0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.828+0000 7fdbcf4d1640 1 -- 172.21.15.84:0/3638144205 shutdown_connections 2023-10-16T10:05:21.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.828+0000 7fdbcf4d1640 1 --2- 172.21.15.84:0/3638144205 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7fdbb406b010 0x7fdbb406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.828+0000 7fdbcf4d1640 1 --2- 172.21.15.84:0/3638144205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbc815cd70 0x7fdbc80cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.828+0000 7fdbcf4d1640 1 --2- 172.21.15.84:0/3638144205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbc80d0be0 0x7fdbc80c8690 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.828+0000 7fdbcf4d1640 1 --2- 172.21.15.84:0/3638144205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbc80b8480 0x7fdbc81539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:21.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.828+0000 7fdbcf4d1640 1 -- 172.21.15.84:0/3638144205 >> 172.21.15.84:0/3638144205 conn(0x7fdbc80cf570 msgr2=0x7fdbc80b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:21.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.828+0000 7fdbcf4d1640 1 -- 172.21.15.84:0/3638144205 shutdown_connections 2023-10-16T10:05:21.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:21.828+0000 7fdbcf4d1640 1 -- 172.21.15.84:0/3638144205 wait complete. 2023-10-16T10:05:21.847 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-10-16T10:05:22.001 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:22.001+0000 7f84d7c2b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:05:22.006 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:22.005+0000 7f5bc940f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:05:22.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.052+0000 7f8da5259640 1 Processor -- start 2023-10-16T10:05:22.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.053+0000 7f8da5259640 1 -- start start 2023-10-16T10:05:22.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.054+0000 7f8da5259640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8da006d220 0x7f8da00c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:22.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.054+0000 7f8da5259640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da00acd70 con 0x7f8da00ce280 2023-10-16T10:05:22.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.054+0000 7f8da5259640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da00ac4a0 con 0x7f8da0065ab0 2023-10-16T10:05:22.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.054+0000 7f8da5259640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da0065f40 con 0x7f8da006d220 2023-10-16T10:05:22.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.054+0000 7f8d9ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8da006d220 0x7f8da00c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:22.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.054+0000 7f8d9ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8da006d220 0x7f8da00c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35688/0 (socket says 172.21.15.84:35688) 2023-10-16T10:05:22.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.054+0000 7f8d9ffff640 1 -- 172.21.15.84:0/696291643 learned_addr learned my addr 172.21.15.84:0/696291643 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:22.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.055+0000 7f8d9ffff640 1 -- 172.21.15.84:0/696291643 >> v1:172.21.15.157:6789/0 conn(0x7f8da0065ab0 legacy=0x7f8da00cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:22.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.055+0000 7f8d9ffff640 1 -- 172.21.15.84:0/696291643 >> v1:172.21.15.84:6789/0 conn(0x7f8da00ce280 legacy=0x7f8da00ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:22.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.055+0000 7f8d9ffff640 1 -- 172.21.15.84:0/696291643 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da00ac720 con 0x7f8da006d220 2023-10-16T10:05:22.058 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:22.056+0000 7f84d7c2b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:05:22.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.056+0000 7f8d9ffff640 1 --2- 172.21.15.84:0/696291643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da006d220 0x7f8da00c95e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009870 tx=0x7f8d9002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3132ab6da65f4bea server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:22.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.056+0000 7f8d9f7fe640 1 -- 172.21.15.84:0/696291643 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90031870 con 0x7f8da006d220 2023-10-16T10:05:22.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.056+0000 7f8d9f7fe640 1 -- 172.21.15.84:0/696291643 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d900319d0 con 0x7f8da006d220 2023-10-16T10:05:22.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.056+0000 7f8d9f7fe640 1 -- 172.21.15.84:0/696291643 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90031d60 con 0x7f8da006d220 2023-10-16T10:05:22.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.057+0000 7f8da5259640 1 -- 172.21.15.84:0/696291643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da006d220 msgr2=0x7f8da00c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:22.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.057+0000 7f8da5259640 1 --2- 172.21.15.84:0/696291643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da006d220 0x7f8da00c95e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009870 tx=0x7f8d9002b910 comp rx=0 tx=0).stop 2023-10-16T10:05:22.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.057+0000 7f8da5259640 1 -- 172.21.15.84:0/696291643 shutdown_connections 2023-10-16T10:05:22.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.057+0000 7f8da5259640 1 --2- 172.21.15.84:0/696291643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da006d220 0x7f8da00c95e0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:22.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.057+0000 7f8da5259640 1 -- 172.21.15.84:0/696291643 >> 172.21.15.84:0/696291643 conn(0x7f8da00b0a80 msgr2=0x7f8da00b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:22.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.057+0000 7f8da5259640 1 -- 172.21.15.84:0/696291643 shutdown_connections 2023-10-16T10:05:22.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.057+0000 7f8da5259640 1 -- 172.21.15.84:0/696291643 wait complete. 2023-10-16T10:05:22.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.059+0000 7f8da5259640 1 Processor -- start 2023-10-16T10:05:22.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.059+0000 7f8da5259640 1 -- start start 2023-10-16T10:05:22.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.060+0000 7f8da5259640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da006d220 0x7f8da00b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:22.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.060+0000 7f8da5259640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8da00ce280 0x7f8da00b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:22.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.060+0000 7f8da5259640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8da014cf60 0x7f8da015eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:22.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.060+0000 7f8da5259640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da009a110 con 0x7f8da006d220 2023-10-16T10:05:22.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.060+0000 7f8da5259640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da00b36b0 con 0x7f8da00ce280 2023-10-16T10:05:22.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.060+0000 7f8da5259640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da00b2e10 con 0x7f8da014cf60 2023-10-16T10:05:22.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.060+0000 7f8d9ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da006d220 0x7f8da00b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:22.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.060+0000 7f8d9ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da006d220 0x7f8da00b3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35692/0 (socket says 172.21.15.84:35692) 2023-10-16T10:05:22.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.060+0000 7f8d9ffff640 1 -- 172.21.15.84:0/4115962122 learned_addr learned my addr 172.21.15.84:0/4115962122 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:22.066 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:22.057+0000 7f5bc940f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:05:22.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.060+0000 7f8d9ffff640 1 -- 172.21.15.84:0/4115962122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8da014cf60 msgr2=0x7f8da015eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:22.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.060+0000 7f8d9ffff640 1 --2- 172.21.15.84:0/4115962122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8da014cf60 0x7f8da015eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:22.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.061+0000 7f8d97fff640 1 --2- 172.21.15.84:0/4115962122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8da00ce280 0x7f8da00b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:22.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.061+0000 7f8d9ffff640 1 -- 172.21.15.84:0/4115962122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8da00ce280 msgr2=0x7f8da00b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:22.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.061+0000 7f8d9ffff640 1 --2- 172.21.15.84:0/4115962122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8da00ce280 0x7f8da00b25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:22.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.061+0000 7f8d9ffff640 1 -- 172.21.15.84:0/4115962122 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d900094d0 con 0x7f8da006d220 2023-10-16T10:05:22.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.061+0000 7f8d9ffff640 1 --2- 172.21.15.84:0/4115962122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da006d220 0x7f8da00b3cf0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009870 tx=0x7f8d900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:22.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.061+0000 7f8d9d7fa640 1 -- 172.21.15.84:0/4115962122 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90031820 con 0x7f8da006d220 2023-10-16T10:05:22.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.061+0000 7f8d9d7fa640 1 -- 172.21.15.84:0/4115962122 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d900443f0 con 0x7f8da006d220 2023-10-16T10:05:22.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.061+0000 7f8d9d7fa640 1 -- 172.21.15.84:0/4115962122 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d900447a0 con 0x7f8da006d220 2023-10-16T10:05:22.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.061+0000 7f8da5259640 1 -- 172.21.15.84:0/4115962122 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da00b30f0 con 0x7f8da006d220 2023-10-16T10:05:22.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.061+0000 7f8da5259640 1 -- 172.21.15.84:0/4115962122 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da009a340 con 0x7f8da006d220 2023-10-16T10:05:22.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.063+0000 7f8da5259640 1 -- 172.21.15.84:0/4115962122 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da004e9b0 con 0x7f8da006d220 2023-10-16T10:05:22.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.064+0000 7f8d9d7fa640 1 -- 172.21.15.84:0/4115962122 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 42) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f8d9004d450 con 0x7f8da006d220 2023-10-16T10:05:22.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.065+0000 7f8d9d7fa640 1 --2- 172.21.15.84:0/4115962122 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f8d8006afe0 0x7f8d8006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:22.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.065+0000 7f8d9d7fa640 1 -- 172.21.15.84:0/4115962122 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d9002d040 con 0x7f8da006d220 2023-10-16T10:05:22.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.065+0000 7f8d97fff640 1 -- 172.21.15.84:0/4115962122 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f8d8006afe0 msgr2=0x7f8d8006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/147697688 2023-10-16T10:05:22.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.065+0000 7f8d97fff640 1 --2- 172.21.15.84:0/4115962122 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f8d8006afe0 0x7f8d8006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:05:22.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.070+0000 7f8d9d7fa640 1 -- 172.21.15.84:0/4115962122 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8d90076f80 con 0x7f8da006d220 2023-10-16T10:05:22.110 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:22.108+0000 7f5bc940f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:05:22.110 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:22.110+0000 7f84d7c2b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:05:22.171 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:22.170+0000 7f5bc940f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:05:22.174 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:22.173+0000 7f84d7c2b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:05:22.220 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:22.218+0000 7f5bc940f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:05:22.225 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:22.225+0000 7f84d7c2b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:05:22.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.266+0000 7f8d97fff640 1 -- 172.21.15.84:0/4115962122 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f8d8006afe0 msgr2=0x7f8d8006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/147697688 2023-10-16T10:05:22.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.266+0000 7f8d97fff640 1 --2- 172.21.15.84:0/4115962122 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f8d8006afe0 0x7f8d8006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:05:22.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.284+0000 7f8da5259640 1 -- 172.21.15.84:0/4115962122 --> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8da00aca00 con 0x7f8d8006afe0 2023-10-16T10:05:22.331 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:22.330+0000 7f5bc940f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:05:22.338 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:22.338+0000 7f84d7c2b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:05:22.394 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:22.392+0000 7f5bc940f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:05:22.400 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:22.400+0000 7f84d7c2b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:05:22.444 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:22.443+0000 7f5bc940f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:05:22.450 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:22.450+0000 7f84d7c2b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:05:22.501 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:22.499+0000 7f5bc940f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:05:22.506 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:22.506+0000 7f84d7c2b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:05:22.551 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:22.549+0000 7f5bc940f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:05:22.556 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:22.556+0000 7f84d7c2b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:05:22.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.667+0000 7f8d97fff640 1 -- 172.21.15.84:0/4115962122 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f8d8006afe0 msgr2=0x7f8d8006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/147697688 2023-10-16T10:05:22.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:22.667+0000 7f8d97fff640 1 --2- 172.21.15.84:0/4115962122 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f8d8006afe0 0x7f8d8006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T10:05:22.932 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:22.932+0000 7f84d7c2b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:05:22.934 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:22.932+0000 7f5bc940f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:05:23.091 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:23.091+0000 7f84d7c2b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:05:23.094 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:23.093+0000 7f5bc940f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:05:23.199 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:23.198+0000 7f84d7c2b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:05:23.202 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:23.200+0000 7f5bc940f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:05:23.248 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:23.248+0000 7f84d7c2b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:05:23.251 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:23.250+0000 7f5bc940f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:05:23.355 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:23.354+0000 7f84d7c2b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:05:23.359 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:23.358+0000 7f5bc940f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:05:23.462 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:23.462+0000 7f84d7c2b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:05:23.466 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:23.464+0000 7f5bc940f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:05:23.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:23.468+0000 7f8d97fff640 1 -- 172.21.15.84:0/4115962122 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f8d8006afe0 msgr2=0x7f8d8006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/147697688 2023-10-16T10:05:23.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:23.468+0000 7f8d97fff640 1 --2- 172.21.15.84:0/4115962122 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f8d8006afe0 0x7f8d8006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T10:05:23.619 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:23.618+0000 7f84d7c2b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:05:23.623 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:23.621+0000 7f5bc940f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:05:23.668 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:23.668+0000 7f84d7c2b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:05:23.672 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:23.671+0000 7f5bc940f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:05:23.787 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:23.786+0000 7f84d7c2b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:05:23.791 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:23.790+0000 7f5bc940f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:05:24.609 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:24.608+0000 7f5bc940f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:05:24.615 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:24.615+0000 7f84d7c2b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:05:24.902 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:24.900+0000 7f5bc940f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:05:24.908 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:24.908+0000 7f84d7c2b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:05:24.958 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:24.956+0000 7f5bc940f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:05:24.964 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:24.964+0000 7f84d7c2b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:05:25.070 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:25.068+0000 7f5bc940f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:05:25.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:25.070+0000 7f8d97fff640 1 -- 172.21.15.84:0/4115962122 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f8d8006afe0 msgr2=0x7f8d8006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/147697688 2023-10-16T10:05:25.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:25.070+0000 7f8d97fff640 1 --2- 172.21.15.84:0/4115962122 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f8d8006afe0 0x7f8d8006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T10:05:25.077 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:25.077+0000 7f84d7c2b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:05:25.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:25.094+0000 7f8d9d7fa640 1 -- 172.21.15.84:0/4115962122 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 43) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f8d90077350 con 0x7f8da006d220 2023-10-16T10:05:25.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:25.095+0000 7f8d9d7fa640 1 -- 172.21.15.84:0/4115962122 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f8d8006afe0 msgr2=0x7f8d8006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:05:25.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:25.095+0000 7f8d9d7fa640 1 --2- 172.21.15.84:0/4115962122 >> [v2:172.21.15.84:6832/147697688,v1:172.21.15.84:6833/147697688] conn(0x7f8d8006afe0 0x7f8d8006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:25.446 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:25.445+0000 7f8453f78640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:25.446 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:25.445+0000 7f8453f78640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:25.447 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:25.445+0000 7f8453f78640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:25.447 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:25.445+0000 7f8453f78640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:25.447 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:25.445+0000 7f8453f78640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:25.447 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:25.446+0000 7f84567bd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:25.448 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:25.446+0000 7f84567bd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:25.448 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:25.446+0000 7f84567bd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:25.448 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:25.446+0000 7f84567bd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:25.448 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:25.446+0000 7f84567bd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:26.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.152+0000 7f8d9d7fa640 1 -- 172.21.15.84:0/4115962122 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 44) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f8d90007000 con 0x7f8da006d220 2023-10-16T10:05:26.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.153+0000 7f8d9d7fa640 1 --2- 172.21.15.84:0/4115962122 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f8d800355a0 0x7f8d80035980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:26.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.153+0000 7f8d9d7fa640 1 -- 172.21.15.84:0/4115962122 --> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d80035ec0 con 0x7f8d800355a0 2023-10-16T10:05:26.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.165+0000 7f8d97fff640 1 --2- 172.21.15.84:0/4115962122 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f8d800355a0 0x7f8d80035980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:26.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.175+0000 7f8d97fff640 1 --2- 172.21.15.84:0/4115962122 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f8d800355a0 0x7f8d80035980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8da00b4230 tx=0x7f8d880091f0 comp rx=0 tx=0).ready entity=mgr.7206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:26.199 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:26.198+0000 7f845dfcc640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-16T10:05:26.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.199+0000 7f8d9d7fa640 1 -- 172.21.15.84:0/4115962122 <== mgr.7206 v2:172.21.15.84:6832/494168607 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f8d80035ec0 con 0x7f8d800355a0 2023-10-16T10:05:26.200 INFO:teuthology.orchestra.run.smithi084.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-16T10:05:26.221 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T10:05:26.222 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T10:05:26.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.440+0000 7f71dfaa9640 1 Processor -- start 2023-10-16T10:05:26.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.440+0000 7f71dfaa9640 1 -- start start 2023-10-16T10:05:26.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.441+0000 7f71dfaa9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71d815ad70 0x7f71d8069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:26.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.441+0000 7f71dfaa9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d80cb030 con 0x7f71d80caac0 2023-10-16T10:05:26.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.441+0000 7f71dfaa9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d80c91c0 con 0x7f71d80c9ca0 2023-10-16T10:05:26.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.441+0000 7f71dfaa9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d80c94a0 con 0x7f71d815ad70 2023-10-16T10:05:26.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.442+0000 7f71de046640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f71d80caac0 0x7f71d80ad4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57804/0 (socket says 172.21.15.84:57804) 2023-10-16T10:05:26.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.442+0000 7f71de046640 1 -- 172.21.15.84:0/2398067659 learned_addr learned my addr 172.21.15.84:0/2398067659 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:26.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.442+0000 7f71dd845640 1 --2- 172.21.15.84:0/2398067659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71d815ad70 0x7f71d8069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:26.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.442+0000 7f71dc843640 1 -- 172.21.15.84:0/2398067659 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3202667219 0 0) 0x7f71d80cb030 con 0x7f71d80caac0 2023-10-16T10:05:26.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.442+0000 7f71dc843640 1 -- 172.21.15.84:0/2398067659 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71c4003520 con 0x7f71d80caac0 2023-10-16T10:05:26.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.443+0000 7f71dc843640 1 -- 172.21.15.84:0/2398067659 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2819205779 0 0) 0x7f71c4003520 con 0x7f71d80caac0 2023-10-16T10:05:26.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.443+0000 7f71dc843640 1 -- 172.21.15.84:0/2398067659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71d815ad70 msgr2=0x7f71d8069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:26.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.443+0000 7f71dc843640 1 --2- 172.21.15.84:0/2398067659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71d815ad70 0x7f71d8069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:26.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.443+0000 7f71dc843640 1 -- 172.21.15.84:0/2398067659 >> v1:172.21.15.157:6789/0 conn(0x7f71d80c9ca0 legacy=0x7f71d80ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:26.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.443+0000 7f71dc843640 1 -- 172.21.15.84:0/2398067659 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d80c9780 con 0x7f71d80caac0 2023-10-16T10:05:26.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.443+0000 7f71dc843640 1 -- 172.21.15.84:0/2398067659 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f71d00019f0 con 0x7f71d80caac0 2023-10-16T10:05:26.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.443+0000 7f71dc843640 1 -- 172.21.15.84:0/2398067659 >> v1:172.21.15.84:6789/0 conn(0x7f71d80caac0 legacy=0x7f71d80ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:26.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.443+0000 7f71dc843640 1 --2- 172.21.15.84:0/2398067659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c40039a0 0x7f71c4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:26.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.443+0000 7f71dc843640 1 --2- 172.21.15.84:0/2398067659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71c40047c0 0x7f71c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:26.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.443+0000 7f71dc843640 1 --2- 172.21.15.84:0/2398067659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71d815ad70 0x7f71c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:26.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.443+0000 7f71dc843640 1 -- 172.21.15.84:0/2398067659 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d80c94a0 con 0x7f71c40047c0 2023-10-16T10:05:26.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.443+0000 7f71dc843640 1 -- 172.21.15.84:0/2398067659 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c400ba60 con 0x7f71c40039a0 2023-10-16T10:05:26.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.443+0000 7f71dc843640 1 -- 172.21.15.84:0/2398067659 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c400bc20 con 0x7f71d815ad70 2023-10-16T10:05:26.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.443+0000 7f71dd044640 1 --2- 172.21.15.84:0/2398067659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71c40047c0 0x7f71c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:26.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.443+0000 7f71dd845640 1 --2- 172.21.15.84:0/2398067659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c40039a0 0x7f71c4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:26.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.444+0000 7f71dd044640 1 -- 172.21.15.84:0/2398067659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71d815ad70 msgr2=0x7f71c4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:26.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.444+0000 7f71dd044640 1 --2- 172.21.15.84:0/2398067659 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71d815ad70 0x7f71c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:26.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.444+0000 7f71dd044640 1 -- 172.21.15.84:0/2398067659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c40039a0 msgr2=0x7f71c4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:26.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.444+0000 7f71dd044640 1 --2- 172.21.15.84:0/2398067659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c40039a0 0x7f71c4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:26.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.444+0000 7f71dd044640 1 -- 172.21.15.84:0/2398067659 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c4003520 con 0x7f71c40047c0 2023-10-16T10:05:26.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.445+0000 7f71dd044640 1 --2- 172.21.15.84:0/2398067659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71c40047c0 0x7f71c4004c40 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f71c8001720 tx=0x7f71c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96988f5299730b5d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:26.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.445+0000 7f71dc843640 1 -- 172.21.15.84:0/2398067659 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c80052d0 con 0x7f71c40047c0 2023-10-16T10:05:26.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.445+0000 7f71dc843640 1 -- 172.21.15.84:0/2398067659 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c802e050 con 0x7f71c40047c0 2023-10-16T10:05:26.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.445+0000 7f71dc843640 1 -- 172.21.15.84:0/2398067659 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c80035f0 con 0x7f71c40047c0 2023-10-16T10:05:26.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.446+0000 7f71dfaa9640 1 -- 172.21.15.84:0/2398067659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71c40047c0 msgr2=0x7f71c4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:26.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.446+0000 7f71dfaa9640 1 --2- 172.21.15.84:0/2398067659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71c40047c0 0x7f71c4004c40 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f71c8001720 tx=0x7f71c40057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:26.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.446+0000 7f71de046640 1 -- 172.21.15.84:0/2398067659 reap_dead start 2023-10-16T10:05:26.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.446+0000 7f71dfaa9640 1 -- 172.21.15.84:0/2398067659 shutdown_connections 2023-10-16T10:05:26.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.446+0000 7f71dfaa9640 1 -- 172.21.15.84:0/2398067659 >> 172.21.15.84:0/2398067659 conn(0x7f71d80ce9b0 msgr2=0x7f71d80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:26.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.446+0000 7f71dfaa9640 1 -- 172.21.15.84:0/2398067659 shutdown_connections 2023-10-16T10:05:26.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.446+0000 7f71dfaa9640 1 -- 172.21.15.84:0/2398067659 wait complete. 2023-10-16T10:05:26.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.448+0000 7f71dfaa9640 1 Processor -- start 2023-10-16T10:05:26.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.448+0000 7f71dfaa9640 1 -- start start 2023-10-16T10:05:26.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.449+0000 7f71dfaa9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71c40047c0 0x7f71d80c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:26.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.449+0000 7f71dfaa9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d8151f80 0x7f71d81f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:26.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.449+0000 7f71dfaa9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71d81f3970 0x7f71d815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:26.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.449+0000 7f71dfaa9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d81f3770 con 0x7f71d81f3970 2023-10-16T10:05:26.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.449+0000 7f71dfaa9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d81f3df0 con 0x7f71d8151f80 2023-10-16T10:05:26.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.449+0000 7f71dfaa9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d80c8110 con 0x7f71c40047c0 2023-10-16T10:05:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.449+0000 7f71de046640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71d81f3970 0x7f71d815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.449+0000 7f71de046640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71d81f3970 0x7f71d815b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56042/0 (socket says 172.21.15.84:56042) 2023-10-16T10:05:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.449+0000 7f71de046640 1 -- 172.21.15.84:0/3463834071 learned_addr learned my addr 172.21.15.84:0/3463834071 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.450+0000 7f71de046640 1 -- 172.21.15.84:0/3463834071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71c40047c0 msgr2=0x7f71d80c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.450+0000 7f71de046640 1 --2- 172.21.15.84:0/3463834071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71c40047c0 0x7f71d80c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.450+0000 7f71de046640 1 -- 172.21.15.84:0/3463834071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d8151f80 msgr2=0x7f71d81f27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.450+0000 7f71dd044640 1 --2- 172.21.15.84:0/3463834071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d8151f80 0x7f71d81f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.450+0000 7f71de046640 1 --2- 172.21.15.84:0/3463834071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d8151f80 0x7f71d81f27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.450+0000 7f71de046640 1 -- 172.21.15.84:0/3463834071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d81f2ed0 con 0x7f71d81f3970 2023-10-16T10:05:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.450+0000 7f71de046640 1 --2- 172.21.15.84:0/3463834071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71d81f3970 0x7f71d815b780 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f71d80cf780 tx=0x7f71d000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.450+0000 7f71ce7fc640 1 -- 172.21.15.84:0/3463834071 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d000a610 con 0x7f71d81f3970 2023-10-16T10:05:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.450+0000 7f71ce7fc640 1 -- 172.21.15.84:0/3463834071 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d000a790 con 0x7f71d81f3970 2023-10-16T10:05:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.450+0000 7f71dfaa9640 1 -- 172.21.15.84:0/3463834071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d81f31b0 con 0x7f71d81f3970 2023-10-16T10:05:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.450+0000 7f71ce7fc640 1 -- 172.21.15.84:0/3463834071 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d000ab40 con 0x7f71d81f3970 2023-10-16T10:05:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.450+0000 7f71dfaa9640 1 -- 172.21.15.84:0/3463834071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d80c25f0 con 0x7f71d81f3970 2023-10-16T10:05:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.452+0000 7f71dfaa9640 1 -- 172.21.15.84:0/3463834071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71d804e160 con 0x7f71d81f3970 2023-10-16T10:05:26.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.452+0000 7f71ce7fc640 1 -- 172.21.15.84:0/3463834071 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 44) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f71d000aca0 con 0x7f71d81f3970 2023-10-16T10:05:26.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.453+0000 7f71ce7fc640 1 --2- 172.21.15.84:0/3463834071 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f71bc06ad40 0x7f71bc06d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:26.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.453+0000 7f71ce7fc640 1 -- 172.21.15.84:0/3463834071 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f71d0088ca0 con 0x7f71d81f3970 2023-10-16T10:05:26.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.454+0000 7f71dd845640 1 --2- 172.21.15.84:0/3463834071 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f71bc06ad40 0x7f71bc06d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:26.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.454+0000 7f71dd845640 1 --2- 172.21.15.84:0/3463834071 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f71bc06ad40 0x7f71bc06d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f71d815e1c0 tx=0x7f71d4008040 comp rx=0 tx=0).ready entity=mgr.7206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:26.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.458+0000 7f71ce7fc640 1 -- 172.21.15.84:0/3463834071 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f71d0053f30 con 0x7f71d81f3970 2023-10-16T10:05:26.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:26.699+0000 7f71dfaa9640 1 -- 172.21.15.84:0/3463834071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f71d80cb020 con 0x7f71d81f3970 2023-10-16T10:05:27.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.152+0000 7f71ce7fc640 1 -- 172.21.15.84:0/3463834071 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v45) v1 ==== 93+0+0 (secure 0 0 0) 0x7f71d0007c40 con 0x7f71d81f3970 2023-10-16T10:05:27.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.156+0000 7f71dfaa9640 1 -- 172.21.15.84:0/3463834071 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f71bc06ad40 msgr2=0x7f71bc06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:27.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.156+0000 7f71dfaa9640 1 --2- 172.21.15.84:0/3463834071 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f71bc06ad40 0x7f71bc06d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f71d815e1c0 tx=0x7f71d4008040 comp rx=0 tx=0).stop 2023-10-16T10:05:27.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.156+0000 7f71dfaa9640 1 -- 172.21.15.84:0/3463834071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71d81f3970 msgr2=0x7f71d815b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:27.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.156+0000 7f71dfaa9640 1 --2- 172.21.15.84:0/3463834071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71d81f3970 0x7f71d815b780 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f71d80cf780 tx=0x7f71d000ccd0 comp rx=0 tx=0).stop 2023-10-16T10:05:27.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.156+0000 7f71dfaa9640 1 -- 172.21.15.84:0/3463834071 shutdown_connections 2023-10-16T10:05:27.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.156+0000 7f71dfaa9640 1 --2- 172.21.15.84:0/3463834071 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f71bc06ad40 0x7f71bc06d200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:27.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.156+0000 7f71dfaa9640 1 --2- 172.21.15.84:0/3463834071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71d81f3970 0x7f71d815b780 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:27.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.156+0000 7f71dfaa9640 1 --2- 172.21.15.84:0/3463834071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d8151f80 0x7f71d81f27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:27.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.156+0000 7f71dfaa9640 1 --2- 172.21.15.84:0/3463834071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71c40047c0 0x7f71d80c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:27.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.156+0000 7f71dfaa9640 1 -- 172.21.15.84:0/3463834071 >> 172.21.15.84:0/3463834071 conn(0x7f71d80ce9b0 msgr2=0x7f71d80ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:27.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.157+0000 7f71dfaa9640 1 -- 172.21.15.84:0/3463834071 shutdown_connections 2023-10-16T10:05:27.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.157+0000 7f71dfaa9640 1 -- 172.21.15.84:0/3463834071 wait complete. 2023-10-16T10:05:27.181 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:05:27.181 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:05:27.292 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:05:27.383 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:05:27.384 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:05:27.384 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:05:27.438 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:05:27.439 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:05:27.527 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:05:27.706 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:27.704+0000 7fed6500d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:05:27.711 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:27.711+0000 7f921864b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:05:27.765 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:27.763+0000 7fed6500d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:05:27.768 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:27.767+0000 7f921864b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:05:27.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.769+0000 7fa03212c640 1 Processor -- start 2023-10-16T10:05:27.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.769+0000 7fa03212c640 1 -- start start 2023-10-16T10:05:27.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.770+0000 7fa03212c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa02c06b4c0 0x7fa02c0ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:27.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.770+0000 7fa03212c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa02c06e170 con 0x7fa02c160800 2023-10-16T10:05:27.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.770+0000 7fa03212c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa02c0cd460 con 0x7fa02c160420 2023-10-16T10:05:27.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.771+0000 7fa03212c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa02c06e3c0 con 0x7fa02c06b4c0 2023-10-16T10:05:27.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.771+0000 7fa02bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa02c160800 0x7fa02c070cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57816/0 (socket says 172.21.15.84:57816) 2023-10-16T10:05:27.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.771+0000 7fa02bfff640 1 -- 172.21.15.84:0/3177386056 learned_addr learned my addr 172.21.15.84:0/3177386056 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:27.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.771+0000 7fa023fff640 1 --2- 172.21.15.84:0/3177386056 >> v2:172.21.15.157:3300/0 conn(0x7fa02c06b4c0 0x7fa02c0ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:27.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.771+0000 7fa02affd640 1 -- 172.21.15.84:0/3177386056 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982832468 0 0) 0x7fa02c06e170 con 0x7fa02c160800 2023-10-16T10:05:27.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.771+0000 7fa02affd640 1 -- 172.21.15.84:0/3177386056 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa018003520 con 0x7fa02c160800 2023-10-16T10:05:27.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.772+0000 7fa02affd640 1 -- 172.21.15.84:0/3177386056 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 834427538 0 0) 0x7fa018003520 con 0x7fa02c160800 2023-10-16T10:05:27.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.772+0000 7fa02affd640 1 -- 172.21.15.84:0/3177386056 >> v2:172.21.15.157:3300/0 conn(0x7fa02c06b4c0 msgr2=0x7fa02c0ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:27.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.772+0000 7fa02affd640 1 --2- 172.21.15.84:0/3177386056 >> v2:172.21.15.157:3300/0 conn(0x7fa02c06b4c0 0x7fa02c0ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:27.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.772+0000 7fa02affd640 1 -- 172.21.15.84:0/3177386056 >> v1:172.21.15.157:6789/0 conn(0x7fa02c160420 legacy=0x7fa02c06adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:27.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.772+0000 7fa02affd640 1 -- 172.21.15.84:0/3177386056 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02c06e6a0 con 0x7fa02c160800 2023-10-16T10:05:27.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.772+0000 7fa02affd640 1 -- 172.21.15.84:0/3177386056 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fa0240019f0 con 0x7fa02c160800 2023-10-16T10:05:27.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.772+0000 7fa02affd640 1 -- 172.21.15.84:0/3177386056 >> v1:172.21.15.84:6789/0 conn(0x7fa02c160800 legacy=0x7fa02c070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:27.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.772+0000 7fa02affd640 1 --2- 172.21.15.84:0/3177386056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0180039c0 0x7fa018003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:27.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.772+0000 7fa02affd640 1 --2- 172.21.15.84:0/3177386056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0180047c0 0x7fa018004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:27.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.772+0000 7fa02affd640 1 --2- 172.21.15.84:0/3177386056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa018005180 0x7fa01800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:27.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.772+0000 7fa02affd640 1 -- 172.21.15.84:0/3177386056 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c06e3c0 con 0x7fa0180047c0 2023-10-16T10:05:27.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.772+0000 7fa02affd640 1 -- 172.21.15.84:0/3177386056 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01800c080 con 0x7fa0180039c0 2023-10-16T10:05:27.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.772+0000 7fa02affd640 1 -- 172.21.15.84:0/3177386056 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01800c240 con 0x7fa018005180 2023-10-16T10:05:27.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.772+0000 7fa023fff640 1 --2- 172.21.15.84:0/3177386056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0180047c0 0x7fa018004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:27.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.772+0000 7fa02b7fe640 1 --2- 172.21.15.84:0/3177386056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0180039c0 0x7fa018003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:27.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.773+0000 7fa02bfff640 1 --2- 172.21.15.84:0/3177386056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa018005180 0x7fa01800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:27.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.773+0000 7fa023fff640 1 -- 172.21.15.84:0/3177386056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa018005180 msgr2=0x7fa01800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:27.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.773+0000 7fa023fff640 1 --2- 172.21.15.84:0/3177386056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa018005180 0x7fa01800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:27.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.773+0000 7fa023fff640 1 -- 172.21.15.84:0/3177386056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0180039c0 msgr2=0x7fa018003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:27.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.773+0000 7fa023fff640 1 --2- 172.21.15.84:0/3177386056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0180039c0 0x7fa018003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:27.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.773+0000 7fa023fff640 1 -- 172.21.15.84:0/3177386056 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa018003520 con 0x7fa0180047c0 2023-10-16T10:05:27.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.773+0000 7fa02bfff640 1 --2- 172.21.15.84:0/3177386056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa018005180 0x7fa01800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:27.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.773+0000 7fa02bfff640 1 -- 172.21.15.84:0/3177386056 reap_dead start 2023-10-16T10:05:27.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.774+0000 7fa023fff640 1 --2- 172.21.15.84:0/3177386056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0180047c0 0x7fa018004c40 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa014009ac0 tx=0x7fa0180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a501efd7304d0f98 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:27.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.775+0000 7fa02affd640 1 -- 172.21.15.84:0/3177386056 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01402c6b0 con 0x7fa0180047c0 2023-10-16T10:05:27.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.775+0000 7fa02affd640 1 -- 172.21.15.84:0/3177386056 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01402c810 con 0x7fa0180047c0 2023-10-16T10:05:27.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.775+0000 7fa02affd640 1 -- 172.21.15.84:0/3177386056 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01402cba0 con 0x7fa0180047c0 2023-10-16T10:05:27.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.775+0000 7fa03212c640 1 -- 172.21.15.84:0/3177386056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0180047c0 msgr2=0x7fa018004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:27.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.775+0000 7fa03212c640 1 --2- 172.21.15.84:0/3177386056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0180047c0 0x7fa018004c40 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa014009ac0 tx=0x7fa0180057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:27.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.776+0000 7fa03212c640 1 -- 172.21.15.84:0/3177386056 shutdown_connections 2023-10-16T10:05:27.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.776+0000 7fa03212c640 1 --2- 172.21.15.84:0/3177386056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0180047c0 0x7fa018004c40 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:27.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.776+0000 7fa03212c640 1 -- 172.21.15.84:0/3177386056 >> 172.21.15.84:0/3177386056 conn(0x7fa02c0b0d30 msgr2=0x7fa02c0b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:27.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.776+0000 7fa03212c640 1 -- 172.21.15.84:0/3177386056 shutdown_connections 2023-10-16T10:05:27.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.776+0000 7fa03212c640 1 -- 172.21.15.84:0/3177386056 wait complete. 2023-10-16T10:05:27.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.777+0000 7fa03212c640 1 Processor -- start 2023-10-16T10:05:27.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.778+0000 7fa03212c640 1 -- start start 2023-10-16T10:05:27.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.778+0000 7fa03212c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0180047c0 0x7fa02c099bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:27.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.778+0000 7fa03212c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa02c09a100 0x7fa02c09a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:27.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.778+0000 7fa03212c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa02c09aac0 0x7fa02c09e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:27.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.778+0000 7fa03212c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c0b3410 con 0x7fa0180047c0 2023-10-16T10:05:27.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.778+0000 7fa03212c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c09af40 con 0x7fa02c09aac0 2023-10-16T10:05:27.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.778+0000 7fa03212c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c0b2540 con 0x7fa02c09a100 2023-10-16T10:05:27.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.779+0000 7fa02b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0180047c0 0x7fa02c099bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:27.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.779+0000 7fa02b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0180047c0 0x7fa02c099bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56068/0 (socket says 172.21.15.84:56068) 2023-10-16T10:05:27.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.779+0000 7fa02b7fe640 1 -- 172.21.15.84:0/1974387099 learned_addr learned my addr 172.21.15.84:0/1974387099 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:27.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.779+0000 7fa02bfff640 1 --2- 172.21.15.84:0/1974387099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa02c09aac0 0x7fa02c09e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:27.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.779+0000 7fa02b7fe640 1 -- 172.21.15.84:0/1974387099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa02c09a100 msgr2=0x7fa02c09a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:27.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.779+0000 7fa02b7fe640 1 --2- 172.21.15.84:0/1974387099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa02c09a100 0x7fa02c09a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:27.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.779+0000 7fa02b7fe640 1 -- 172.21.15.84:0/1974387099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa02c09aac0 msgr2=0x7fa02c09e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:27.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.779+0000 7fa02b7fe640 1 --2- 172.21.15.84:0/1974387099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa02c09aac0 0x7fa02c09e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:27.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.779+0000 7fa02b7fe640 1 -- 172.21.15.84:0/1974387099 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02c15a660 con 0x7fa0180047c0 2023-10-16T10:05:27.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.779+0000 7fa02bfff640 1 --2- 172.21.15.84:0/1974387099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa02c09aac0 0x7fa02c09e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:27.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.779+0000 7fa02b7fe640 1 --2- 172.21.15.84:0/1974387099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0180047c0 0x7fa02c099bc0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00e5b0 tx=0x7fa01c00ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:27.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.780+0000 7fa028ff9640 1 -- 172.21.15.84:0/1974387099 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c0052e0 con 0x7fa0180047c0 2023-10-16T10:05:27.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.780+0000 7fa028ff9640 1 -- 172.21.15.84:0/1974387099 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01c005440 con 0x7fa0180047c0 2023-10-16T10:05:27.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.780+0000 7fa028ff9640 1 -- 172.21.15.84:0/1974387099 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c003810 con 0x7fa0180047c0 2023-10-16T10:05:27.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.780+0000 7fa03212c640 1 -- 172.21.15.84:0/1974387099 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa02c0b2790 con 0x7fa0180047c0 2023-10-16T10:05:27.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.780+0000 7fa03212c640 1 -- 172.21.15.84:0/1974387099 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa02c0b2d50 con 0x7fa0180047c0 2023-10-16T10:05:27.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.781+0000 7fa03212c640 1 -- 172.21.15.84:0/1974387099 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa02c04e160 con 0x7fa0180047c0 2023-10-16T10:05:27.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.783+0000 7fa028ff9640 1 -- 172.21.15.84:0/1974387099 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 45) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fa01c003970 con 0x7fa0180047c0 2023-10-16T10:05:27.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.784+0000 7fa028ff9640 1 --2- 172.21.15.84:0/1974387099 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fa00c06af90 0x7fa00c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:27.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.784+0000 7fa028ff9640 1 -- 172.21.15.84:0/1974387099 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fa01c089580 con 0x7fa0180047c0 2023-10-16T10:05:27.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.784+0000 7fa023fff640 1 -- 172.21.15.84:0/1974387099 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fa00c06af90 msgr2=0x7fa00c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/494168607 2023-10-16T10:05:27.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.784+0000 7fa023fff640 1 --2- 172.21.15.84:0/1974387099 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fa00c06af90 0x7fa00c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:05:27.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.788+0000 7fa028ff9640 1 -- 172.21.15.84:0/1974387099 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa01c054450 con 0x7fa0180047c0 2023-10-16T10:05:27.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.983+0000 7fa03212c640 1 -- 172.21.15.84:0/1974387099 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa02c0b51c0 con 0x7fa0180047c0 2023-10-16T10:05:27.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.985+0000 7fa028ff9640 1 -- 172.21.15.84:0/1974387099 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+17041 (secure 0 0 0) 0x7fa01c057a70 con 0x7fa0180047c0 2023-10-16T10:05:27.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.985+0000 7fa023fff640 1 -- 172.21.15.84:0/1974387099 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fa00c06af90 msgr2=0x7fa00c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/494168607 2023-10-16T10:05:27.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.985+0000 7fa023fff640 1 --2- 172.21.15.84:0/1974387099 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fa00c06af90 0x7fa00c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:05:27.986 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:27.986 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":96,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:05:25.080022+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:05:08.636077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-16T10:05:09.841985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/4027382473":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/3685429931":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/504267391":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/3778743167":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/150347074":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/3151132884":"2023-10-17T10:05:25.079928+0000","172.21.15.84:6833/147697688":"2023-10-17T10:05:25.079928+0000","172.21.15.84:6832/147697688":"2023-10-17T10:05:25.079928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:05:27.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.988+0000 7fa03212c640 1 -- 172.21.15.84:0/1974387099 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fa00c06af90 msgr2=0x7fa00c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:05:27.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.988+0000 7fa03212c640 1 --2- 172.21.15.84:0/1974387099 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fa00c06af90 0x7fa00c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:27.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.988+0000 7fa03212c640 1 -- 172.21.15.84:0/1974387099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0180047c0 msgr2=0x7fa02c099bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:27.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.988+0000 7fa03212c640 1 --2- 172.21.15.84:0/1974387099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0180047c0 0x7fa02c099bc0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00e5b0 tx=0x7fa01c00ea70 comp rx=0 tx=0).stop 2023-10-16T10:05:27.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.989+0000 7fa03212c640 1 -- 172.21.15.84:0/1974387099 shutdown_connections 2023-10-16T10:05:27.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.989+0000 7fa03212c640 1 --2- 172.21.15.84:0/1974387099 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fa00c06af90 0x7fa00c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:27.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.989+0000 7fa03212c640 1 --2- 172.21.15.84:0/1974387099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa02c09aac0 0x7fa02c09e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:27.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.989+0000 7fa03212c640 1 --2- 172.21.15.84:0/1974387099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa02c09a100 0x7fa02c09a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:27.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.989+0000 7fa03212c640 1 --2- 172.21.15.84:0/1974387099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0180047c0 0x7fa02c099bc0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:27.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.989+0000 7fa03212c640 1 -- 172.21.15.84:0/1974387099 >> 172.21.15.84:0/1974387099 conn(0x7fa02c0b0d30 msgr2=0x7fa02c0aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:27.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.989+0000 7fa03212c640 1 -- 172.21.15.84:0/1974387099 shutdown_connections 2023-10-16T10:05:27.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:27.989+0000 7fa03212c640 1 -- 172.21.15.84:0/1974387099 wait complete. 2023-10-16T10:05:28.008 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:05:08.636077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:05:09.841985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:05:28.009 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:05:28.219 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:28.218+0000 7fed6500d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:05:28.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.222+0000 7f4d30294640 1 Processor -- start 2023-10-16T10:05:28.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.222+0000 7f4d30294640 1 -- start start 2023-10-16T10:05:28.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.223+0000 7f4d30294640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4d2815d410 0x7f4d280d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:28.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.223+0000 7f4d30294640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d280b5d90 con 0x7f4d280cd260 2023-10-16T10:05:28.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.223+0000 7f4d30294640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d28160fc0 con 0x7f4d2815d030 2023-10-16T10:05:28.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.223+0000 7f4d30294640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d280d2b50 con 0x7f4d2815d410 2023-10-16T10:05:28.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.224+0000 7f4d2e831640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4d280cd260 0x7f4d280b4720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57826/0 (socket says 172.21.15.84:57826) 2023-10-16T10:05:28.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.224+0000 7f4d2e831640 1 -- 172.21.15.84:0/4069633521 learned_addr learned my addr 172.21.15.84:0/4069633521 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:28.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.224+0000 7f4d2d82f640 1 --2- 172.21.15.84:0/4069633521 >> v2:172.21.15.157:3300/0 conn(0x7f4d2815d410 0x7f4d280d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:28.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.224+0000 7f4d2d02e640 1 -- 172.21.15.84:0/4069633521 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3887509579 0 0) 0x7f4d280b5d90 con 0x7f4d280cd260 2023-10-16T10:05:28.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.224+0000 7f4d2d02e640 1 -- 172.21.15.84:0/4069633521 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d14003520 con 0x7f4d280cd260 2023-10-16T10:05:28.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.225+0000 7f4d2d02e640 1 -- 172.21.15.84:0/4069633521 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4174385932 0 0) 0x7f4d14003520 con 0x7f4d280cd260 2023-10-16T10:05:28.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.225+0000 7f4d2d02e640 1 -- 172.21.15.84:0/4069633521 >> v2:172.21.15.157:3300/0 conn(0x7f4d2815d410 msgr2=0x7f4d280d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:28.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.225+0000 7f4d2d02e640 1 --2- 172.21.15.84:0/4069633521 >> v2:172.21.15.157:3300/0 conn(0x7f4d2815d410 0x7f4d280d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.225+0000 7f4d2d02e640 1 -- 172.21.15.84:0/4069633521 >> v1:172.21.15.157:6789/0 conn(0x7f4d2815d030 legacy=0x7f4d28161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:28.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.225+0000 7f4d2d02e640 1 -- 172.21.15.84:0/4069633521 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d280d25d0 con 0x7f4d280cd260 2023-10-16T10:05:28.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.225+0000 7f4d2d02e640 1 -- 172.21.15.84:0/4069633521 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f4d200019f0 con 0x7f4d280cd260 2023-10-16T10:05:28.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.225+0000 7f4d2d02e640 1 -- 172.21.15.84:0/4069633521 >> v1:172.21.15.84:6789/0 conn(0x7f4d280cd260 legacy=0x7f4d280b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:28.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.225+0000 7f4d2d02e640 1 --2- 172.21.15.84:0/4069633521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d140039c0 0x7f4d14003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:28.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.225+0000 7f4d2d02e640 1 --2- 172.21.15.84:0/4069633521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d140047c0 0x7f4d14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:28.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.225+0000 7f4d2d02e640 1 --2- 172.21.15.84:0/4069633521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d14005180 0x7f4d1400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:28.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.225+0000 7f4d2d02e640 1 -- 172.21.15.84:0/4069633521 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1400c030 con 0x7f4d140047c0 2023-10-16T10:05:28.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.225+0000 7f4d2d02e640 1 -- 172.21.15.84:0/4069633521 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1400c1a0 con 0x7f4d140039c0 2023-10-16T10:05:28.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.225+0000 7f4d2d02e640 1 -- 172.21.15.84:0/4069633521 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1400c360 con 0x7f4d14005180 2023-10-16T10:05:28.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.225+0000 7f4d2d82f640 1 --2- 172.21.15.84:0/4069633521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d140047c0 0x7f4d14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:28.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.225+0000 7f4d2e030640 1 --2- 172.21.15.84:0/4069633521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d140039c0 0x7f4d14003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:28.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.225+0000 7f4d2e831640 1 --2- 172.21.15.84:0/4069633521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d14005180 0x7f4d1400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:28.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.226+0000 7f4d2d82f640 1 -- 172.21.15.84:0/4069633521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d14005180 msgr2=0x7f4d1400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:28.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.226+0000 7f4d2d82f640 1 --2- 172.21.15.84:0/4069633521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d14005180 0x7f4d1400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.226+0000 7f4d2d82f640 1 -- 172.21.15.84:0/4069633521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d140039c0 msgr2=0x7f4d14003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:28.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.226+0000 7f4d2d82f640 1 --2- 172.21.15.84:0/4069633521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d140039c0 0x7f4d14003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.226+0000 7f4d2d82f640 1 -- 172.21.15.84:0/4069633521 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d14003520 con 0x7f4d140047c0 2023-10-16T10:05:28.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.226+0000 7f4d2e831640 1 -- 172.21.15.84:0/4069633521 reap_dead start 2023-10-16T10:05:28.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.227+0000 7f4d2d82f640 1 --2- 172.21.15.84:0/4069633521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d140047c0 0x7f4d14004c40 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009a90 tx=0x7f4d140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3587f8be8d981cc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:28.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.227+0000 7f4d2d02e640 1 -- 172.21.15.84:0/4069633521 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1802c6b0 con 0x7f4d140047c0 2023-10-16T10:05:28.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.227+0000 7f4d2d02e640 1 -- 172.21.15.84:0/4069633521 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1802c810 con 0x7f4d140047c0 2023-10-16T10:05:28.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.227+0000 7f4d2d02e640 1 -- 172.21.15.84:0/4069633521 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1802cba0 con 0x7f4d140047c0 2023-10-16T10:05:28.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.227+0000 7f4d30294640 1 -- 172.21.15.84:0/4069633521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d140047c0 msgr2=0x7f4d14004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:28.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.227+0000 7f4d30294640 1 --2- 172.21.15.84:0/4069633521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d140047c0 0x7f4d14004c40 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009a90 tx=0x7f4d140057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.228+0000 7f4d30294640 1 -- 172.21.15.84:0/4069633521 shutdown_connections 2023-10-16T10:05:28.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.228+0000 7f4d30294640 1 --2- 172.21.15.84:0/4069633521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d140047c0 0x7f4d14004c40 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.228+0000 7f4d30294640 1 -- 172.21.15.84:0/4069633521 >> 172.21.15.84:0/4069633521 conn(0x7f4d280cf720 msgr2=0x7f4d280cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:28.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.228+0000 7f4d30294640 1 -- 172.21.15.84:0/4069633521 shutdown_connections 2023-10-16T10:05:28.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.228+0000 7f4d30294640 1 -- 172.21.15.84:0/4069633521 wait complete. 2023-10-16T10:05:28.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.229+0000 7f4d30294640 1 Processor -- start 2023-10-16T10:05:28.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.230+0000 7f4d30294640 1 -- start start 2023-10-16T10:05:28.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.230+0000 7f4d30294640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d140047c0 0x7f4d2815a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:28.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.230+0000 7f4d30294640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d280c84f0 0x7f4d280c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:28.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.230+0000 7f4d30294640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d280c9ce0 0x7f4d280ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:28.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.230+0000 7f4d30294640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d280ca6b0 con 0x7f4d140047c0 2023-10-16T10:05:28.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.230+0000 7f4d30294640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2815e990 con 0x7f4d280c84f0 2023-10-16T10:05:28.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.230+0000 7f4d30294640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d280b8750 con 0x7f4d280c9ce0 2023-10-16T10:05:28.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.231+0000 7f4d2e030640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d140047c0 0x7f4d2815a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:28.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.231+0000 7f4d2e030640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d140047c0 0x7f4d2815a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56076/0 (socket says 172.21.15.84:56076) 2023-10-16T10:05:28.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.231+0000 7f4d2e030640 1 -- 172.21.15.84:0/3257373395 learned_addr learned my addr 172.21.15.84:0/3257373395 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:28.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.231+0000 7f4d2e831640 1 --2- 172.21.15.84:0/3257373395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d280c9ce0 0x7f4d280ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:28.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.231+0000 7f4d2d82f640 1 --2- 172.21.15.84:0/3257373395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d280c84f0 0x7f4d280c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:28.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.231+0000 7f4d2e030640 1 -- 172.21.15.84:0/3257373395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d280c9ce0 msgr2=0x7f4d280ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:28.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.231+0000 7f4d2e030640 1 --2- 172.21.15.84:0/3257373395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d280c9ce0 0x7f4d280ca170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.231+0000 7f4d2e030640 1 -- 172.21.15.84:0/3257373395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d280c84f0 msgr2=0x7f4d280c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:28.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.231+0000 7f4d2e030640 1 --2- 172.21.15.84:0/3257373395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d280c84f0 0x7f4d280c8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.231+0000 7f4d2e030640 1 -- 172.21.15.84:0/3257373395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d280b7b40 con 0x7f4d140047c0 2023-10-16T10:05:28.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.231+0000 7f4d2e030640 1 --2- 172.21.15.84:0/3257373395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d140047c0 0x7f4d2815a430 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400e4c0 tx=0x7f4d2400e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:28.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.232+0000 7f4d1effd640 1 -- 172.21.15.84:0/3257373395 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d240052e0 con 0x7f4d140047c0 2023-10-16T10:05:28.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.232+0000 7f4d1effd640 1 -- 172.21.15.84:0/3257373395 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d24005440 con 0x7f4d140047c0 2023-10-16T10:05:28.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.232+0000 7f4d1effd640 1 -- 172.21.15.84:0/3257373395 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24003830 con 0x7f4d140047c0 2023-10-16T10:05:28.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.232+0000 7f4d30294640 1 -- 172.21.15.84:0/3257373395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d280b7e20 con 0x7f4d140047c0 2023-10-16T10:05:28.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.232+0000 7f4d30294640 1 -- 172.21.15.84:0/3257373395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d280b83e0 con 0x7f4d140047c0 2023-10-16T10:05:28.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.233+0000 7f4d30294640 1 -- 172.21.15.84:0/3257373395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2804e160 con 0x7f4d140047c0 2023-10-16T10:05:28.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.234+0000 7f4d1effd640 1 -- 172.21.15.84:0/3257373395 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 45) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f4d24003990 con 0x7f4d140047c0 2023-10-16T10:05:28.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.235+0000 7f4d1effd640 1 --2- 172.21.15.84:0/3257373395 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f4d0c06af90 0x7f4d0c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:28.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.235+0000 7f4d2d82f640 1 -- 172.21.15.84:0/3257373395 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f4d0c06af90 msgr2=0x7f4d0c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/494168607 2023-10-16T10:05:28.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.235+0000 7f4d1effd640 1 -- 172.21.15.84:0/3257373395 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f4d24089640 con 0x7f4d140047c0 2023-10-16T10:05:28.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.235+0000 7f4d2d82f640 1 --2- 172.21.15.84:0/3257373395 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f4d0c06af90 0x7f4d0c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:05:28.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.240+0000 7f4d1effd640 1 -- 172.21.15.84:0/3257373395 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4d24054480 con 0x7f4d140047c0 2023-10-16T10:05:28.245 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:28.244+0000 7f921864b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:05:28.273 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:28.272+0000 7fed6500d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:05:28.299 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:28.298+0000 7f921864b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:05:28.323 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:28.322+0000 7fed6500d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:05:28.351 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:28.350+0000 7f921864b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:05:28.384 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:28.383+0000 7fed6500d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:05:28.414 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:28.413+0000 7f921864b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:05:28.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.432+0000 7f4d30294640 1 -- 172.21.15.84:0/3257373395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4d280d2840 con 0x7f4d140047c0 2023-10-16T10:05:28.433 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:28.432+0000 7fed6500d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:05:28.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.433+0000 7f4d1effd640 1 -- 172.21.15.84:0/3257373395 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4d24057aa0 con 0x7f4d140047c0 2023-10-16T10:05:28.434 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:05:28.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.435+0000 7f4d30294640 1 -- 172.21.15.84:0/3257373395 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f4d0c06af90 msgr2=0x7f4d0c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:05:28.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.435+0000 7f4d30294640 1 --2- 172.21.15.84:0/3257373395 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f4d0c06af90 0x7f4d0c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.435+0000 7f4d30294640 1 -- 172.21.15.84:0/3257373395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d140047c0 msgr2=0x7f4d2815a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:28.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.435+0000 7f4d30294640 1 --2- 172.21.15.84:0/3257373395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d140047c0 0x7f4d2815a430 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400e4c0 tx=0x7f4d2400e980 comp rx=0 tx=0).stop 2023-10-16T10:05:28.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.435+0000 7f4d30294640 1 -- 172.21.15.84:0/3257373395 shutdown_connections 2023-10-16T10:05:28.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.435+0000 7f4d30294640 1 --2- 172.21.15.84:0/3257373395 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f4d0c06af90 0x7f4d0c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.435+0000 7f4d30294640 1 --2- 172.21.15.84:0/3257373395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d280c9ce0 0x7f4d280ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.435+0000 7f4d30294640 1 --2- 172.21.15.84:0/3257373395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d280c84f0 0x7f4d280c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.435+0000 7f4d30294640 1 --2- 172.21.15.84:0/3257373395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d140047c0 0x7f4d2815a430 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.435+0000 7f4d30294640 1 -- 172.21.15.84:0/3257373395 >> 172.21.15.84:0/3257373395 conn(0x7f4d280cf720 msgr2=0x7f4d280d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:28.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.436+0000 7f4d30294640 1 -- 172.21.15.84:0/3257373395 shutdown_connections 2023-10-16T10:05:28.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.436+0000 7f4d30294640 1 -- 172.21.15.84:0/3257373395 wait complete. 2023-10-16T10:05:28.454 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:05:28.465 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:28.465+0000 7f921864b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:05:28.544 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:28.542+0000 7fed6500d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:05:28.583 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:28.582+0000 7f921864b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:05:28.606 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:28.604+0000 7fed6500d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:05:28.649 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:28.648+0000 7f921864b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:05:28.656 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:28.654+0000 7fed6500d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:05:28.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.657+0000 7fa7aea90640 1 Processor -- start 2023-10-16T10:05:28.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.657+0000 7fa7aea90640 1 -- start start 2023-10-16T10:05:28.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.658+0000 7fa7aea90640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa7a815d030 0x7fa7a80d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:28.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.658+0000 7fa7aea90640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa7a815d410 0x7fa7a8161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:28.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.658+0000 7fa7aea90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7a8161990 0x7fa7a80cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:28.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.658+0000 7fa7aea90640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a80d2b50 con 0x7fa7a815d410 2023-10-16T10:05:28.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.658+0000 7fa7aea90640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a80b5d90 con 0x7fa7a815d030 2023-10-16T10:05:28.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.658+0000 7fa7aea90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a8160fc0 con 0x7fa7a8161990 2023-10-16T10:05:28.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.659+0000 7fa7a7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa7a815d410 0x7fa7a8161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:28.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.659+0000 7fa7a7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa7a815d410 0x7fa7a8161450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56092/0 (socket says 172.21.15.84:56092) 2023-10-16T10:05:28.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.659+0000 7fa7a7fff640 1 -- 172.21.15.84:0/40869508 learned_addr learned my addr 172.21.15.84:0/40869508 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:28.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.659+0000 7fa7ad02d640 1 --2- 172.21.15.84:0/40869508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7a8161990 0x7fa7a80cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:28.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.659+0000 7fa7ac82c640 1 --2- 172.21.15.84:0/40869508 >> v2:172.21.15.157:3300/0 conn(0x7fa7a815d030 0x7fa7a80d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:28.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.659+0000 7fa7a7fff640 1 -- 172.21.15.84:0/40869508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7a8161990 msgr2=0x7fa7a80cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:28.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.659+0000 7fa7a7fff640 1 --2- 172.21.15.84:0/40869508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7a8161990 0x7fa7a80cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.659+0000 7fa7a7fff640 1 -- 172.21.15.84:0/40869508 >> v2:172.21.15.157:3300/0 conn(0x7fa7a815d030 msgr2=0x7fa7a80d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:28.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.659+0000 7fa7a7fff640 1 --2- 172.21.15.84:0/40869508 >> v2:172.21.15.157:3300/0 conn(0x7fa7a815d030 0x7fa7a80d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.659+0000 7fa7a7fff640 1 -- 172.21.15.84:0/40869508 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a80d25d0 con 0x7fa7a815d410 2023-10-16T10:05:28.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.659+0000 7fa7ad02d640 1 --2- 172.21.15.84:0/40869508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7a8161990 0x7fa7a80cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:05:28.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.661+0000 7fa7a7fff640 1 --2- 172.21.15.84:0/40869508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7a815d410 0x7fa7a8161450 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa7a806cd10 tx=0x7fa79402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9538f83949b35ec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:28.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.661+0000 7fa7a77fe640 1 -- 172.21.15.84:0/40869508 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794030860 con 0x7fa7a815d410 2023-10-16T10:05:28.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.661+0000 7fa7a77fe640 1 -- 172.21.15.84:0/40869508 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7940309c0 con 0x7fa7a815d410 2023-10-16T10:05:28.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.661+0000 7fa7a77fe640 1 -- 172.21.15.84:0/40869508 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794030d50 con 0x7fa7a815d410 2023-10-16T10:05:28.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.662+0000 7fa7aea90640 1 -- 172.21.15.84:0/40869508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7a815d410 msgr2=0x7fa7a8161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:28.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.662+0000 7fa7aea90640 1 --2- 172.21.15.84:0/40869508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7a815d410 0x7fa7a8161450 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa7a806cd10 tx=0x7fa79402b9a0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.662+0000 7fa7aea90640 1 -- 172.21.15.84:0/40869508 shutdown_connections 2023-10-16T10:05:28.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.662+0000 7fa7aea90640 1 --2- 172.21.15.84:0/40869508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7a815d410 0x7fa7a8161450 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.662+0000 7fa7aea90640 1 --2- 172.21.15.84:0/40869508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7a8161990 0x7fa7a80cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.662+0000 7fa7aea90640 1 --2- 172.21.15.84:0/40869508 >> v2:172.21.15.157:3300/0 conn(0x7fa7a815d030 0x7fa7a80d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.662+0000 7fa7aea90640 1 -- 172.21.15.84:0/40869508 >> 172.21.15.84:0/40869508 conn(0x7fa7a80cf720 msgr2=0x7fa7a80cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:28.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.662+0000 7fa7aea90640 1 -- 172.21.15.84:0/40869508 shutdown_connections 2023-10-16T10:05:28.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.662+0000 7fa7aea90640 1 -- 172.21.15.84:0/40869508 wait complete. 2023-10-16T10:05:28.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.664+0000 7fa7aea90640 1 Processor -- start 2023-10-16T10:05:28.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.664+0000 7fa7aea90640 1 -- start start 2023-10-16T10:05:28.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.665+0000 7fa7aea90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a815d030 0x7fa7a81f7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:28.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.665+0000 7fa7aea90640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7a815d410 0x7fa7a81550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:28.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.665+0000 7fa7aea90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7a8161990 0x7fa7a80b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:28.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.665+0000 7fa7aea90640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a81f7000 con 0x7fa7a815d410 2023-10-16T10:05:28.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.665+0000 7fa7aea90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a8155a50 con 0x7fa7a815d030 2023-10-16T10:05:28.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.665+0000 7fa7aea90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a80b4c00 con 0x7fa7a8161990 2023-10-16T10:05:28.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.665+0000 7fa7a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7a815d410 0x7fa7a81550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:28.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.666+0000 7fa7a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7a815d410 0x7fa7a81550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56106/0 (socket says 172.21.15.84:56106) 2023-10-16T10:05:28.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.666+0000 7fa7a7fff640 1 -- 172.21.15.84:0/390451135 learned_addr learned my addr 172.21.15.84:0/390451135 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:28.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.666+0000 7fa7ac82c640 1 --2- 172.21.15.84:0/390451135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a815d030 0x7fa7a81f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:28.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.666+0000 7fa7ad02d640 1 --2- 172.21.15.84:0/390451135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7a8161990 0x7fa7a80b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:28.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.666+0000 7fa7a7fff640 1 -- 172.21.15.84:0/390451135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7a8161990 msgr2=0x7fa7a80b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:28.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.666+0000 7fa7a7fff640 1 --2- 172.21.15.84:0/390451135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7a8161990 0x7fa7a80b46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.666+0000 7fa7a7fff640 1 -- 172.21.15.84:0/390451135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a815d030 msgr2=0x7fa7a81f7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:28.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.666+0000 7fa7a7fff640 1 --2- 172.21.15.84:0/390451135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a815d030 0x7fa7a81f7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.666+0000 7fa7a7fff640 1 -- 172.21.15.84:0/390451135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7940094d0 con 0x7fa7a815d410 2023-10-16T10:05:28.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.666+0000 7fa7a7fff640 1 --2- 172.21.15.84:0/390451135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7a815d410 0x7fa7a81550d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa794000c00 tx=0x7fa794004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:28.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.667+0000 7fa7a57fa640 1 -- 172.21.15.84:0/390451135 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794039d60 con 0x7fa7a815d410 2023-10-16T10:05:28.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.667+0000 7fa7a57fa640 1 -- 172.21.15.84:0/390451135 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa794044480 con 0x7fa7a815d410 2023-10-16T10:05:28.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.667+0000 7fa7a57fa640 1 -- 172.21.15.84:0/390451135 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794044830 con 0x7fa7a815d410 2023-10-16T10:05:28.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.667+0000 7fa7aea90640 1 -- 172.21.15.84:0/390451135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a81606e0 con 0x7fa7a815d410 2023-10-16T10:05:28.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.667+0000 7fa7aea90640 1 -- 172.21.15.84:0/390451135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a81f69b0 con 0x7fa7a815d410 2023-10-16T10:05:28.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.668+0000 7fa7aea90640 1 -- 172.21.15.84:0/390451135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a81f71b0 con 0x7fa7a815d410 2023-10-16T10:05:28.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.670+0000 7fa7a57fa640 1 -- 172.21.15.84:0/390451135 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 45) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fa79404d3b0 con 0x7fa7a815d410 2023-10-16T10:05:28.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.671+0000 7fa7a57fa640 1 --2- 172.21.15.84:0/390451135 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fa78c06af90 0x7fa78c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:28.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.671+0000 7fa7ac82c640 1 -- 172.21.15.84:0/390451135 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fa78c06af90 msgr2=0x7fa78c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/494168607 2023-10-16T10:05:28.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.671+0000 7fa7a57fa640 1 -- 172.21.15.84:0/390451135 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fa79403e070 con 0x7fa7a815d410 2023-10-16T10:05:28.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.671+0000 7fa7ac82c640 1 --2- 172.21.15.84:0/390451135 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fa78c06af90 0x7fa78c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:05:28.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.676+0000 7fa7a57fa640 1 -- 172.21.15.84:0/390451135 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa7a81f71b0 con 0x7fa7a815d410 2023-10-16T10:05:28.704 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:28.704+0000 7f921864b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:05:28.713 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:28.712+0000 7fed6500d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:05:28.763 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:28.763+0000 7f921864b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:05:28.764 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:28.762+0000 7fed6500d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:05:28.816 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:28.816+0000 7f921864b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:05:28.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.871+0000 7fa7aea90640 1 -- 172.21.15.84:0/390451135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa7a81f71b0 con 0x7fa7a815d410 2023-10-16T10:05:28.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.872+0000 7fa7a57fa640 1 -- 172.21.15.84:0/390451135 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa7a81f71b0 con 0x7fa7a815d410 2023-10-16T10:05:28.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.872+0000 7fa7ac82c640 1 -- 172.21.15.84:0/390451135 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fa78c06af90 msgr2=0x7fa78c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/494168607 2023-10-16T10:05:28.873 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T10:05:28.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.872+0000 7fa7ac82c640 1 --2- 172.21.15.84:0/390451135 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fa78c06af90 0x7fa78c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:05:28.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.874+0000 7fa7aea90640 1 -- 172.21.15.84:0/390451135 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fa78c06af90 msgr2=0x7fa78c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:05:28.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.874+0000 7fa7aea90640 1 --2- 172.21.15.84:0/390451135 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fa78c06af90 0x7fa78c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.874+0000 7fa7aea90640 1 -- 172.21.15.84:0/390451135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7a815d410 msgr2=0x7fa7a81550d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:28.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.874+0000 7fa7aea90640 1 --2- 172.21.15.84:0/390451135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7a815d410 0x7fa7a81550d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa794000c00 tx=0x7fa794004730 comp rx=0 tx=0).stop 2023-10-16T10:05:28.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.874+0000 7fa7aea90640 1 -- 172.21.15.84:0/390451135 shutdown_connections 2023-10-16T10:05:28.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.874+0000 7fa7aea90640 1 --2- 172.21.15.84:0/390451135 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fa78c06af90 0x7fa78c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.874+0000 7fa7aea90640 1 --2- 172.21.15.84:0/390451135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7a8161990 0x7fa7a80b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.874+0000 7fa7aea90640 1 --2- 172.21.15.84:0/390451135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7a815d410 0x7fa7a81550d0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.874+0000 7fa7aea90640 1 --2- 172.21.15.84:0/390451135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a815d030 0x7fa7a81f7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:28.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.874+0000 7fa7aea90640 1 -- 172.21.15.84:0/390451135 >> 172.21.15.84:0/390451135 conn(0x7fa7a80cf720 msgr2=0x7fa7a80d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:28.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.874+0000 7fa7aea90640 1 -- 172.21.15.84:0/390451135 shutdown_connections 2023-10-16T10:05:28.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:28.874+0000 7fa7aea90640 1 -- 172.21.15.84:0/390451135 wait complete. 2023-10-16T10:05:28.894 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:05:29.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.143+0000 7fc79e4e0640 1 Processor -- start 2023-10-16T10:05:29.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.143+0000 7fc79e4e0640 1 -- start start 2023-10-16T10:05:29.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.145+0000 7fc79e4e0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc79815ce00 0x7fc798160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:29.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.145+0000 7fc79e4e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79806ea40 0x7fc798069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:29.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.145+0000 7fc79e4e0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79806a720 con 0x7fc79806a2e0 2023-10-16T10:05:29.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.145+0000 7fc79e4e0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7980cdc60 con 0x7fc79815ce00 2023-10-16T10:05:29.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.145+0000 7fc79e4e0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7981618b0 con 0x7fc79806ea40 2023-10-16T10:05:29.147 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:29.145+0000 7fed6500d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:05:29.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.145+0000 7fc797fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc79815ce00 0x7fc798160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:29.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.145+0000 7fc797fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc79815ce00 0x7fc798160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56112/0 (socket says 172.21.15.84:56112) 2023-10-16T10:05:29.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.145+0000 7fc797fff640 1 -- 172.21.15.84:0/2880625408 learned_addr learned my addr 172.21.15.84:0/2880625408 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:29.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.146+0000 7fc78ffff640 1 --2- 172.21.15.84:0/2880625408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79806ea40 0x7fc798069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:29.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.146+0000 7fc797fff640 1 -- 172.21.15.84:0/2880625408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79806ea40 msgr2=0x7fc798069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:29.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.146+0000 7fc797fff640 1 --2- 172.21.15.84:0/2880625408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79806ea40 0x7fc798069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.146+0000 7fc797fff640 1 -- 172.21.15.84:0/2880625408 >> v1:172.21.15.157:6789/0 conn(0x7fc79806a2e0 legacy=0x7fc798070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:29.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.146+0000 7fc797fff640 1 -- 172.21.15.84:0/2880625408 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc798161010 con 0x7fc79815ce00 2023-10-16T10:05:29.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.147+0000 7fc797fff640 1 --2- 172.21.15.84:0/2880625408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc79815ce00 0x7fc798160980 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fc788009870 tx=0x7fc78802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f3a82fc98099f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:29.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.147+0000 7fc7977fe640 1 -- 172.21.15.84:0/2880625408 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788031640 con 0x7fc79815ce00 2023-10-16T10:05:29.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.147+0000 7fc7977fe640 1 -- 172.21.15.84:0/2880625408 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7880317a0 con 0x7fc79815ce00 2023-10-16T10:05:29.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.147+0000 7fc7977fe640 1 -- 172.21.15.84:0/2880625408 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788031b30 con 0x7fc79815ce00 2023-10-16T10:05:29.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.148+0000 7fc79e4e0640 1 -- 172.21.15.84:0/2880625408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc79815ce00 msgr2=0x7fc798160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:29.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.148+0000 7fc79e4e0640 1 --2- 172.21.15.84:0/2880625408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc79815ce00 0x7fc798160980 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fc788009870 tx=0x7fc78802b8f0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.148+0000 7fc79e4e0640 1 -- 172.21.15.84:0/2880625408 shutdown_connections 2023-10-16T10:05:29.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.148+0000 7fc79e4e0640 1 --2- 172.21.15.84:0/2880625408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79806ea40 0x7fc798069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.148+0000 7fc79e4e0640 1 --2- 172.21.15.84:0/2880625408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc79815ce00 0x7fc798160980 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.148+0000 7fc79e4e0640 1 -- 172.21.15.84:0/2880625408 >> 172.21.15.84:0/2880625408 conn(0x7fc7980cc460 msgr2=0x7fc7980cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:29.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.148+0000 7fc79e4e0640 1 -- 172.21.15.84:0/2880625408 shutdown_connections 2023-10-16T10:05:29.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.148+0000 7fc79e4e0640 1 -- 172.21.15.84:0/2880625408 wait complete. 2023-10-16T10:05:29.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.150+0000 7fc79e4e0640 1 Processor -- start 2023-10-16T10:05:29.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.150+0000 7fc79e4e0640 1 -- start start 2023-10-16T10:05:29.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.151+0000 7fc79e4e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc79806a2e0 0x7fc79815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:29.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.151+0000 7fc79e4e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79806ea40 0x7fc7981f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:29.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.151+0000 7fc79e4e0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc79815ce00 0x7fc7980b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:29.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.151+0000 7fc79e4e0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980c92e0 con 0x7fc79806a2e0 2023-10-16T10:05:29.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.151+0000 7fc79e4e0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980c8fb0 con 0x7fc79815ce00 2023-10-16T10:05:29.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.151+0000 7fc79e4e0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79815e320 con 0x7fc79806ea40 2023-10-16T10:05:29.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.151+0000 7fc797fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc79806a2e0 0x7fc79815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:29.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.151+0000 7fc797fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc79806a2e0 0x7fc79815d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56128/0 (socket says 172.21.15.84:56128) 2023-10-16T10:05:29.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.151+0000 7fc797fff640 1 -- 172.21.15.84:0/2627189636 learned_addr learned my addr 172.21.15.84:0/2627189636 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:29.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.151+0000 7fc78ffff640 1 --2- 172.21.15.84:0/2627189636 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79806ea40 0x7fc7981f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:29.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.151+0000 7fc79ca7d640 1 --2- 172.21.15.84:0/2627189636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc79815ce00 0x7fc7980b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:29.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.151+0000 7fc797fff640 1 -- 172.21.15.84:0/2627189636 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79806ea40 msgr2=0x7fc7981f2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:29.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.151+0000 7fc797fff640 1 --2- 172.21.15.84:0/2627189636 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79806ea40 0x7fc7981f2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.151+0000 7fc797fff640 1 -- 172.21.15.84:0/2627189636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc79815ce00 msgr2=0x7fc7980b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:29.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.151+0000 7fc797fff640 1 --2- 172.21.15.84:0/2627189636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc79815ce00 0x7fc7980b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.152+0000 7fc797fff640 1 -- 172.21.15.84:0/2627189636 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7880094d0 con 0x7fc79806a2e0 2023-10-16T10:05:29.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.152+0000 7fc797fff640 1 --2- 172.21.15.84:0/2627189636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc79806a2e0 0x7fc79815d3c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fc78802b930 tx=0x7fc7880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:29.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.152+0000 7fc7957fa640 1 -- 172.21.15.84:0/2627189636 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78803ac60 con 0x7fc79806a2e0 2023-10-16T10:05:29.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.152+0000 7fc7957fa640 1 -- 172.21.15.84:0/2627189636 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78803adc0 con 0x7fc79806a2e0 2023-10-16T10:05:29.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.152+0000 7fc7957fa640 1 -- 172.21.15.84:0/2627189636 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788042630 con 0x7fc79806a2e0 2023-10-16T10:05:29.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.152+0000 7fc79e4e0640 1 -- 172.21.15.84:0/2627189636 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7981f3510 con 0x7fc79806a2e0 2023-10-16T10:05:29.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.152+0000 7fc79e4e0640 1 -- 172.21.15.84:0/2627189636 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79815dd60 con 0x7fc79806a2e0 2023-10-16T10:05:29.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.154+0000 7fc79e4e0640 1 -- 172.21.15.84:0/2627189636 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7980c9420 con 0x7fc79806a2e0 2023-10-16T10:05:29.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.155+0000 7fc7957fa640 1 -- 172.21.15.84:0/2627189636 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 45) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fc78802f050 con 0x7fc79806a2e0 2023-10-16T10:05:29.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.156+0000 7fc7957fa640 1 --2- 172.21.15.84:0/2627189636 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fc77806af90 0x7fc77806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:29.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.156+0000 7fc78ffff640 1 -- 172.21.15.84:0/2627189636 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fc77806af90 msgr2=0x7fc77806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/494168607 2023-10-16T10:05:29.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.156+0000 7fc7957fa640 1 -- 172.21.15.84:0/2627189636 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fc7880ab240 con 0x7fc79806a2e0 2023-10-16T10:05:29.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.156+0000 7fc78ffff640 1 --2- 172.21.15.84:0/2627189636 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fc77806af90 0x7fc77806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:05:29.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.160+0000 7fc7957fa640 1 -- 172.21.15.84:0/2627189636 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc788076000 con 0x7fc79806a2e0 2023-10-16T10:05:29.221 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:29.221+0000 7f921864b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:05:29.308 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:29.307+0000 7fed6500d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:05:29.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.352+0000 7fc79e4e0640 1 -- 172.21.15.84:0/2627189636 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc79804e160 con 0x7fc79806a2e0 2023-10-16T10:05:29.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.353+0000 7fc7957fa640 1 -- 172.21.15.84:0/2627189636 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc788031e60 con 0x7fc79806a2e0 2023-10-16T10:05:29.354 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T10:05:29.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.356+0000 7fc79e4e0640 1 -- 172.21.15.84:0/2627189636 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fc77806af90 msgr2=0x7fc77806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:05:29.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.356+0000 7fc79e4e0640 1 --2- 172.21.15.84:0/2627189636 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fc77806af90 0x7fc77806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.356+0000 7fc79e4e0640 1 -- 172.21.15.84:0/2627189636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc79806a2e0 msgr2=0x7fc79815d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:29.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.356+0000 7fc79e4e0640 1 --2- 172.21.15.84:0/2627189636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc79806a2e0 0x7fc79815d3c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fc78802b930 tx=0x7fc7880046c0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.356+0000 7fc79e4e0640 1 -- 172.21.15.84:0/2627189636 shutdown_connections 2023-10-16T10:05:29.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.356+0000 7fc79e4e0640 1 --2- 172.21.15.84:0/2627189636 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fc77806af90 0x7fc77806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.356+0000 7fc79e4e0640 1 --2- 172.21.15.84:0/2627189636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc79815ce00 0x7fc7980b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.356+0000 7fc79e4e0640 1 --2- 172.21.15.84:0/2627189636 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79806ea40 0x7fc7981f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.356+0000 7fc79e4e0640 1 --2- 172.21.15.84:0/2627189636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc79806a2e0 0x7fc79815d3c0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.356+0000 7fc79e4e0640 1 -- 172.21.15.84:0/2627189636 >> 172.21.15.84:0/2627189636 conn(0x7fc7980cc460 msgr2=0x7fc7980ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:29.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.356+0000 7fc79e4e0640 1 -- 172.21.15.84:0/2627189636 shutdown_connections 2023-10-16T10:05:29.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.356+0000 7fc79e4e0640 1 -- 172.21.15.84:0/2627189636 wait complete. 2023-10-16T10:05:29.374 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T10:05:29.388 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:29.387+0000 7f921864b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:05:29.416 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:29.414+0000 7fed6500d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:05:29.466 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:29.465+0000 7fed6500d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:05:29.502 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:29.501+0000 7f921864b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:05:29.555 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:29.553+0000 7f921864b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:05:29.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.567+0000 7fd442d35640 1 Processor -- start 2023-10-16T10:05:29.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.567+0000 7fd442d35640 1 -- start start 2023-10-16T10:05:29.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.568+0000 7fd442d35640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd43c15cd70 0x7fd43c0d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:29.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.568+0000 7fd442d35640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd43c0cded0 0x7fd43c0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:29.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.568+0000 7fd442d35640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43c15d150 con 0x7fd43c0b8480 2023-10-16T10:05:29.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.568+0000 7fd442d35640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43c0d1180 con 0x7fd43c15cd70 2023-10-16T10:05:29.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.569+0000 7fd442d35640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43c0d1460 con 0x7fd43c0cded0 2023-10-16T10:05:29.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.569+0000 7fd441d33640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd43c15cd70 0x7fd43c0d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:29.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.569+0000 7fd441d33640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd43c15cd70 0x7fd43c0d2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56132/0 (socket says 172.21.15.84:56132) 2023-10-16T10:05:29.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.569+0000 7fd441d33640 1 -- 172.21.15.84:0/468360171 learned_addr learned my addr 172.21.15.84:0/468360171 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:29.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.569+0000 7fd441532640 1 --1- 172.21.15.84:0/468360171 >> v1:172.21.15.84:6789/0 conn(0x7fd43c0b8480 0x7fd43c0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57828/0 (socket says 172.21.15.84:57828) 2023-10-16T10:05:29.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.569+0000 7fd442534640 1 --2- 172.21.15.84:0/468360171 >> v2:172.21.15.157:3300/0 conn(0x7fd43c0cded0 0x7fd43c0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:29.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.569+0000 7fd441d33640 1 -- 172.21.15.84:0/468360171 >> v2:172.21.15.157:3300/0 conn(0x7fd43c0cded0 msgr2=0x7fd43c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:29.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.569+0000 7fd441d33640 1 --2- 172.21.15.84:0/468360171 >> v2:172.21.15.157:3300/0 conn(0x7fd43c0cded0 0x7fd43c0b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.569+0000 7fd441d33640 1 -- 172.21.15.84:0/468360171 >> v1:172.21.15.84:6789/0 conn(0x7fd43c0b8480 legacy=0x7fd43c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:29.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.569+0000 7fd441d33640 1 -- 172.21.15.84:0/468360171 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c0d1740 con 0x7fd43c15cd70 2023-10-16T10:05:29.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.569+0000 7fd442534640 1 --2- 172.21.15.84:0/468360171 >> v2:172.21.15.157:3300/0 conn(0x7fd43c0cded0 0x7fd43c0b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:29.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.569+0000 7fd440d31640 1 -- 172.21.15.84:0/468360171 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752638835 0 0) 0x7fd43c15d150 con 0x7fd43c0b8480 2023-10-16T10:05:29.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.571+0000 7fd441d33640 1 --2- 172.21.15.84:0/468360171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd43c15cd70 0x7fd43c0d2220 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0b8910 tx=0x7fd43802b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadf518d4b4ccedf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:29.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.571+0000 7fd440d31640 1 -- 172.21.15.84:0/468360171 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438030870 con 0x7fd43c15cd70 2023-10-16T10:05:29.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.571+0000 7fd440d31640 1 -- 172.21.15.84:0/468360171 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4380309d0 con 0x7fd43c15cd70 2023-10-16T10:05:29.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.571+0000 7fd440d31640 1 -- 172.21.15.84:0/468360171 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438030d60 con 0x7fd43c15cd70 2023-10-16T10:05:29.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.571+0000 7fd442d35640 1 -- 172.21.15.84:0/468360171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd43c15cd70 msgr2=0x7fd43c0d2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:29.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.571+0000 7fd442d35640 1 --2- 172.21.15.84:0/468360171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd43c15cd70 0x7fd43c0d2220 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0b8910 tx=0x7fd43802b9b0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.571+0000 7fd442d35640 1 -- 172.21.15.84:0/468360171 shutdown_connections 2023-10-16T10:05:29.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.571+0000 7fd442d35640 1 --2- 172.21.15.84:0/468360171 >> v2:172.21.15.157:3300/0 conn(0x7fd43c0cded0 0x7fd43c0b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.571+0000 7fd442d35640 1 --2- 172.21.15.84:0/468360171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd43c15cd70 0x7fd43c0d2220 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.571+0000 7fd442d35640 1 -- 172.21.15.84:0/468360171 >> 172.21.15.84:0/468360171 conn(0x7fd43c0cf570 msgr2=0x7fd43c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:29.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.572+0000 7fd442d35640 1 -- 172.21.15.84:0/468360171 shutdown_connections 2023-10-16T10:05:29.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.572+0000 7fd442d35640 1 -- 172.21.15.84:0/468360171 wait complete. 2023-10-16T10:05:29.578 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:29.572+0000 7fed6500d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:05:29.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.573+0000 7fd442d35640 1 Processor -- start 2023-10-16T10:05:29.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.573+0000 7fd442d35640 1 -- start start 2023-10-16T10:05:29.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.574+0000 7fd442d35640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd43c0b8480 0x7fd43c144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:29.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.574+0000 7fd442d35640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd43c0cded0 0x7fd43c1f1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:29.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.574+0000 7fd442d35640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd43c15cd70 0x7fd43c1f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:29.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.574+0000 7fd442d35640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c0cc4f0 con 0x7fd43c15cd70 2023-10-16T10:05:29.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.574+0000 7fd442d35640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c1f2390 con 0x7fd43c0cded0 2023-10-16T10:05:29.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.574+0000 7fd442d35640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c0cb620 con 0x7fd43c0b8480 2023-10-16T10:05:29.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.575+0000 7fd442534640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd43c15cd70 0x7fd43c1f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.575+0000 7fd442534640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd43c15cd70 0x7fd43c1f1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56140/0 (socket says 172.21.15.84:56140) 2023-10-16T10:05:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.575+0000 7fd442534640 1 -- 172.21.15.84:0/3062424671 learned_addr learned my addr 172.21.15.84:0/3062424671 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.575+0000 7fd441532640 1 --2- 172.21.15.84:0/3062424671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd43c0cded0 0x7fd43c1f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.575+0000 7fd441d33640 1 --2- 172.21.15.84:0/3062424671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd43c0b8480 0x7fd43c144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.575+0000 7fd442534640 1 -- 172.21.15.84:0/3062424671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd43c0b8480 msgr2=0x7fd43c144fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.575+0000 7fd442534640 1 --2- 172.21.15.84:0/3062424671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd43c0b8480 0x7fd43c144fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.575+0000 7fd442534640 1 -- 172.21.15.84:0/3062424671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd43c0cded0 msgr2=0x7fd43c1f1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.575+0000 7fd442534640 1 --2- 172.21.15.84:0/3062424671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd43c0cded0 0x7fd43c1f1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.575+0000 7fd442534640 1 -- 172.21.15.84:0/3062424671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4380094d0 con 0x7fd43c15cd70 2023-10-16T10:05:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.576+0000 7fd442534640 1 --2- 172.21.15.84:0/3062424671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd43c15cd70 0x7fd43c1f1e50 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fd43400ce70 tx=0x7fd43400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.576+0000 7fd4327fc640 1 -- 172.21.15.84:0/3062424671 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43400d840 con 0x7fd43c15cd70 2023-10-16T10:05:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.576+0000 7fd4327fc640 1 -- 172.21.15.84:0/3062424671 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43400d9a0 con 0x7fd43c15cd70 2023-10-16T10:05:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.576+0000 7fd442d35640 1 -- 172.21.15.84:0/3062424671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd43c145ba0 con 0x7fd43c15cd70 2023-10-16T10:05:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.576+0000 7fd4327fc640 1 -- 172.21.15.84:0/3062424671 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43400dd50 con 0x7fd43c15cd70 2023-10-16T10:05:29.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.576+0000 7fd442d35640 1 -- 172.21.15.84:0/3062424671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd43c0cbb50 con 0x7fd43c15cd70 2023-10-16T10:05:29.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.577+0000 7fd442d35640 1 -- 172.21.15.84:0/3062424671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43c04e9b0 con 0x7fd43c15cd70 2023-10-16T10:05:29.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.578+0000 7fd4327fc640 1 -- 172.21.15.84:0/3062424671 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 45) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fd43401e3b0 con 0x7fd43c15cd70 2023-10-16T10:05:29.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.579+0000 7fd4327fc640 1 --2- 172.21.15.84:0/3062424671 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fd42006af90 0x7fd42006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:29.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.579+0000 7fd4327fc640 1 -- 172.21.15.84:0/3062424671 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fd43408a290 con 0x7fd43c15cd70 2023-10-16T10:05:29.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.579+0000 7fd441d33640 1 -- 172.21.15.84:0/3062424671 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fd42006af90 msgr2=0x7fd42006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/494168607 2023-10-16T10:05:29.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.579+0000 7fd441d33640 1 --2- 172.21.15.84:0/3062424671 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fd42006af90 0x7fd42006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:05:29.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.583+0000 7fd4327fc640 1 -- 172.21.15.84:0/3062424671 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd4340550c0 con 0x7fd43c15cd70 2023-10-16T10:05:29.667 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:29.666+0000 7f921864b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:05:29.681 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:29.680+0000 7fed6500d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:05:29.776 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:29.774+0000 7f921864b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:05:29.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.780+0000 7fd441d33640 1 -- 172.21.15.84:0/3062424671 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fd42006af90 msgr2=0x7fd42006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/494168607 2023-10-16T10:05:29.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.780+0000 7fd441d33640 1 --2- 172.21.15.84:0/3062424671 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fd42006af90 0x7fd42006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:05:29.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.789+0000 7fd442d35640 1 -- 172.21.15.84:0/3062424671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd43c0b2510 con 0x7fd43c15cd70 2023-10-16T10:05:29.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.790+0000 7fd4327fc640 1 -- 172.21.15.84:0/3062424671 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v96) v1 ==== 80+0+466 (secure 0 0 0) 0x7fd4340586e0 con 0x7fd43c15cd70 2023-10-16T10:05:29.791 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/4027382473 2023-10-17T10:05:25.079928+0000 2023-10-16T10:05:29.792 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3685429931 2023-10-17T10:05:25.079928+0000 2023-10-16T10:05:29.792 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/504267391 2023-10-17T10:05:25.079928+0000 2023-10-16T10:05:29.792 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3778743167 2023-10-17T10:05:25.079928+0000 2023-10-16T10:05:29.792 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/150347074 2023-10-17T10:05:25.079928+0000 2023-10-16T10:05:29.793 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3151132884 2023-10-17T10:05:25.079928+0000 2023-10-16T10:05:29.793 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6833/147697688 2023-10-17T10:05:25.079928+0000 2023-10-16T10:05:29.793 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6832/147697688 2023-10-17T10:05:25.079928+0000 2023-10-16T10:05:29.794 INFO:teuthology.orchestra.run.smithi084.stderr:listed 8 entries 2023-10-16T10:05:29.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.792+0000 7fd442d35640 1 -- 172.21.15.84:0/3062424671 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fd42006af90 msgr2=0x7fd42006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:05:29.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.792+0000 7fd442d35640 1 --2- 172.21.15.84:0/3062424671 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fd42006af90 0x7fd42006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.792+0000 7fd442d35640 1 -- 172.21.15.84:0/3062424671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd43c15cd70 msgr2=0x7fd43c1f1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:29.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.792+0000 7fd442d35640 1 --2- 172.21.15.84:0/3062424671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd43c15cd70 0x7fd43c1f1e50 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fd43400ce70 tx=0x7fd43400a710 comp rx=0 tx=0).stop 2023-10-16T10:05:29.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.792+0000 7fd442d35640 1 -- 172.21.15.84:0/3062424671 shutdown_connections 2023-10-16T10:05:29.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.792+0000 7fd442d35640 1 --2- 172.21.15.84:0/3062424671 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fd42006af90 0x7fd42006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.792+0000 7fd442d35640 1 --2- 172.21.15.84:0/3062424671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd43c15cd70 0x7fd43c1f1e50 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.792+0000 7fd442d35640 1 --2- 172.21.15.84:0/3062424671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd43c0cded0 0x7fd43c1f1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.792+0000 7fd442d35640 1 --2- 172.21.15.84:0/3062424671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd43c0b8480 0x7fd43c144fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:29.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.792+0000 7fd442d35640 1 -- 172.21.15.84:0/3062424671 >> 172.21.15.84:0/3062424671 conn(0x7fd43c0cf570 msgr2=0x7fd43c0b7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:29.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.793+0000 7fd442d35640 1 -- 172.21.15.84:0/3062424671 shutdown_connections 2023-10-16T10:05:29.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:29.793+0000 7fd442d35640 1 -- 172.21.15.84:0/3062424671 wait complete. 2023-10-16T10:05:29.810 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T10:05:29.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:29.811 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T10:05:29.837 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:29.835+0000 7fed6500d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:05:29.885 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:29.884+0000 7fed6500d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:05:29.943 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:29.942+0000 7f921864b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:05:29.954 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T10:05:29.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:29.955 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T10:05:30.001 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:30.000+0000 7f921864b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:05:30.002 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:05:30.003 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:05:30.012 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:30.004+0000 7fed6500d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:05:30.060 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:05:30.061 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:05:30.062 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:05:30.115 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:05:30.116 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:05:30.116 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:05:30.125 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:30.123+0000 7f921864b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:05:30.172 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:05:30.173 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:05:30.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.375+0000 7f7985652640 1 Processor -- start 2023-10-16T10:05:30.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.375+0000 7f7985652640 1 -- start start 2023-10-16T10:05:30.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.376+0000 7f7985652640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7980159950 0x7f798015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:30.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.376+0000 7f7985652640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7980070450 0x7f79800708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:30.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.376+0000 7f7985652640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7980070df0 con 0x7f798006c100 2023-10-16T10:05:30.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.376+0000 7f7985652640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798015e490 con 0x7f7980070450 2023-10-16T10:05:30.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.376+0000 7f7985652640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798015aef0 con 0x7f7980159950 2023-10-16T10:05:30.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.377+0000 7f797f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7980070450 0x7f79800708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:30.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.377+0000 7f797f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7980070450 0x7f79800708b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56154/0 (socket says 172.21.15.84:56154) 2023-10-16T10:05:30.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.377+0000 7f797f7fe640 1 -- 172.21.15.84:0/453222193 learned_addr learned my addr 172.21.15.84:0/453222193 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:30.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.377+0000 7f797ffff640 1 --2- 172.21.15.84:0/453222193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7980159950 0x7f798015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:30.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.377+0000 7f797f7fe640 1 -- 172.21.15.84:0/453222193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7980159950 msgr2=0x7f798015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:30.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.377+0000 7f797f7fe640 1 --2- 172.21.15.84:0/453222193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7980159950 0x7f798015a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:30.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.377+0000 7f797f7fe640 1 -- 172.21.15.84:0/453222193 >> v1:172.21.15.157:6789/0 conn(0x7f798006c100 legacy=0x7f798006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:30.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.377+0000 7f797f7fe640 1 -- 172.21.15.84:0/453222193 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798006cc80 con 0x7f7980070450 2023-10-16T10:05:30.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.377+0000 7f797ffff640 1 --2- 172.21.15.84:0/453222193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7980159950 0x7f798015a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:30.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.378+0000 7f797f7fe640 1 --2- 172.21.15.84:0/453222193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7980070450 0x7f79800708b0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f7974009870 tx=0x7f797402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2094e2419d0ca8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:30.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.379+0000 7f797effd640 1 -- 172.21.15.84:0/453222193 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79740317c0 con 0x7f7980070450 2023-10-16T10:05:30.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.379+0000 7f797effd640 1 -- 172.21.15.84:0/453222193 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7974031920 con 0x7f7980070450 2023-10-16T10:05:30.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.379+0000 7f797effd640 1 -- 172.21.15.84:0/453222193 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974031cb0 con 0x7f7980070450 2023-10-16T10:05:30.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.379+0000 7f7985652640 1 -- 172.21.15.84:0/453222193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7980070450 msgr2=0x7f79800708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:30.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.379+0000 7f7985652640 1 --2- 172.21.15.84:0/453222193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7980070450 0x7f79800708b0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f7974009870 tx=0x7f797402b860 comp rx=0 tx=0).stop 2023-10-16T10:05:30.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.379+0000 7f7985652640 1 -- 172.21.15.84:0/453222193 shutdown_connections 2023-10-16T10:05:30.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.379+0000 7f7985652640 1 --2- 172.21.15.84:0/453222193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7980070450 0x7f79800708b0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:30.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.379+0000 7f7985652640 1 --2- 172.21.15.84:0/453222193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7980159950 0x7f798015a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:30.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.379+0000 7f7985652640 1 -- 172.21.15.84:0/453222193 >> 172.21.15.84:0/453222193 conn(0x7f79800ce6a0 msgr2=0x7f79800ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:30.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.379+0000 7f7985652640 1 -- 172.21.15.84:0/453222193 shutdown_connections 2023-10-16T10:05:30.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.379+0000 7f7985652640 1 -- 172.21.15.84:0/453222193 wait complete. 2023-10-16T10:05:30.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.381+0000 7f7985652640 1 Processor -- start 2023-10-16T10:05:30.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.381+0000 7f7985652640 1 -- start start 2023-10-16T10:05:30.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.382+0000 7f7985652640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f798006c100 0x7f79801f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:30.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.382+0000 7f7985652640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7980070450 0x7f79800af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:30.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.382+0000 7f7985652640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7980159950 0x7f79800af600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:30.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.382+0000 7f7985652640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79800c6590 con 0x7f7980070450 2023-10-16T10:05:30.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.382+0000 7f7985652640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7980157d30 con 0x7f798006c100 2023-10-16T10:05:30.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.382+0000 7f7985652640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79800c5910 con 0x7f7980159950 2023-10-16T10:05:30.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.382+0000 7f797f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7980070450 0x7f79800af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:30.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.382+0000 7f797f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7980070450 0x7f79800af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56164/0 (socket says 172.21.15.84:56164) 2023-10-16T10:05:30.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.382+0000 7f797f7fe640 1 -- 172.21.15.84:0/884976321 learned_addr learned my addr 172.21.15.84:0/884976321 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:30.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.383+0000 7f797f7fe640 1 -- 172.21.15.84:0/884976321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7980159950 msgr2=0x7f79800af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:30.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.383+0000 7f797ffff640 1 --2- 172.21.15.84:0/884976321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f798006c100 0x7f79801f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:30.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.383+0000 7f7984e51640 1 --2- 172.21.15.84:0/884976321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7980159950 0x7f79800af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:30.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.383+0000 7f797f7fe640 1 --2- 172.21.15.84:0/884976321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7980159950 0x7f79800af600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:30.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.383+0000 7f797f7fe640 1 -- 172.21.15.84:0/884976321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f798006c100 msgr2=0x7f79801f1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:30.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.383+0000 7f797f7fe640 1 --2- 172.21.15.84:0/884976321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f798006c100 0x7f79801f1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:30.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.383+0000 7f797f7fe640 1 -- 172.21.15.84:0/884976321 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79740094d0 con 0x7f7980070450 2023-10-16T10:05:30.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.383+0000 7f7984e51640 1 --2- 172.21.15.84:0/884976321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7980159950 0x7f79800af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:30.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.383+0000 7f797ffff640 1 --2- 172.21.15.84:0/884976321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f798006c100 0x7f79801f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:30.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.383+0000 7f797f7fe640 1 --2- 172.21.15.84:0/884976321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7980070450 0x7f79800af0c0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f797402b8a0 tx=0x7f79740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:30.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.383+0000 7f797cff9640 1 -- 172.21.15.84:0/884976321 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79740329a0 con 0x7f7980070450 2023-10-16T10:05:30.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.383+0000 7f797cff9640 1 -- 172.21.15.84:0/884976321 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7974032b40 con 0x7f7980070450 2023-10-16T10:05:30.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.383+0000 7f797cff9640 1 -- 172.21.15.84:0/884976321 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79740453f0 con 0x7f7980070450 2023-10-16T10:05:30.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.383+0000 7f7985652640 1 -- 172.21.15.84:0/884976321 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79800c5bf0 con 0x7f7980070450 2023-10-16T10:05:30.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.384+0000 7f7985652640 1 -- 172.21.15.84:0/884976321 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79800c61b0 con 0x7f7980070450 2023-10-16T10:05:30.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.385+0000 7f7985652640 1 -- 172.21.15.84:0/884976321 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79800c66d0 con 0x7f7980070450 2023-10-16T10:05:30.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.385+0000 7f797cff9640 1 -- 172.21.15.84:0/884976321 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 45) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f797404e450 con 0x7f7980070450 2023-10-16T10:05:30.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.386+0000 7f797cff9640 1 --2- 172.21.15.84:0/884976321 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f796c06af90 0x7f796c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:30.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.386+0000 7f797cff9640 1 -- 172.21.15.84:0/884976321 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f79740abbd0 con 0x7f7980070450 2023-10-16T10:05:30.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.386+0000 7f797ffff640 1 -- 172.21.15.84:0/884976321 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f796c06af90 msgr2=0x7f796c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/494168607 2023-10-16T10:05:30.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.386+0000 7f797ffff640 1 --2- 172.21.15.84:0/884976321 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f796c06af90 0x7f796c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:05:30.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.390+0000 7f797cff9640 1 -- 172.21.15.84:0/884976321 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f79800c66d0 con 0x7f7980070450 2023-10-16T10:05:30.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.587+0000 7f797ffff640 1 -- 172.21.15.84:0/884976321 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f796c06af90 msgr2=0x7f796c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/494168607 2023-10-16T10:05:30.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.587+0000 7f797ffff640 1 --2- 172.21.15.84:0/884976321 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f796c06af90 0x7f796c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:05:30.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.607+0000 7f7985652640 1 -- 172.21.15.84:0/884976321 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79800c66d0 con 0x7f7980070450 2023-10-16T10:05:30.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.608+0000 7f797cff9640 1 -- 172.21.15.84:0/884976321 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f79800c66d0 con 0x7f7980070450 2023-10-16T10:05:30.610 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:30.610 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/89534235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":89534235},{"type":"v1","addr":"172.21.15.157:6835","nonce":89534235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3807666966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3807666966},{"type":"v1","addr":"172.21.15.157:6834","nonce":3807666966}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3448417053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3448417053},{"type":"v1","addr":"172.21.15.84:6835","nonce":3448417053}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:05:10.829122+0000","modified":"2023-10-16T10:05:13.893618+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7044":{"gid":7044,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/1939610713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":1939610713},{"type":"v1","addr":"172.21.15.84:6837","nonce":1939610713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-16T10:05:30.610 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 44 2023-10-16T10:05:30.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.610+0000 7f7985652640 1 -- 172.21.15.84:0/884976321 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f796c06af90 msgr2=0x7f796c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:05:30.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.610+0000 7f7985652640 1 --2- 172.21.15.84:0/884976321 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f796c06af90 0x7f796c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:30.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.610+0000 7f7985652640 1 -- 172.21.15.84:0/884976321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7980070450 msgr2=0x7f79800af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:30.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.610+0000 7f7985652640 1 --2- 172.21.15.84:0/884976321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7980070450 0x7f79800af0c0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f797402b8a0 tx=0x7f79740046c0 comp rx=0 tx=0).stop 2023-10-16T10:05:30.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.611+0000 7f7985652640 1 -- 172.21.15.84:0/884976321 shutdown_connections 2023-10-16T10:05:30.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.611+0000 7f7985652640 1 --2- 172.21.15.84:0/884976321 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f796c06af90 0x7f796c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:30.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.611+0000 7f7985652640 1 --2- 172.21.15.84:0/884976321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7980159950 0x7f79800af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:30.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.611+0000 7f7985652640 1 --2- 172.21.15.84:0/884976321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7980070450 0x7f79800af0c0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:30.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.611+0000 7f7985652640 1 --2- 172.21.15.84:0/884976321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f798006c100 0x7f79801f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:30.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.611+0000 7f7985652640 1 -- 172.21.15.84:0/884976321 >> 172.21.15.84:0/884976321 conn(0x7f79800ce6a0 msgr2=0x7f79800cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:30.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.611+0000 7f7985652640 1 -- 172.21.15.84:0/884976321 shutdown_connections 2023-10-16T10:05:30.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.611+0000 7f7985652640 1 -- 172.21.15.84:0/884976321 wait complete. 2023-10-16T10:05:30.629 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:05:30.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.812+0000 7f821ab33640 1 Processor -- start 2023-10-16T10:05:30.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.813+0000 7f821ab33640 1 -- start start 2023-10-16T10:05:30.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.814+0000 7f821ab33640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f821415aaa0 0x7f821415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:30.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.814+0000 7f821ab33640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f821415ae80 0x7f82140cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:30.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.814+0000 7f821ab33640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821415ed60 con 0x7f82140ccaa0 2023-10-16T10:05:30.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.814+0000 7f821ab33640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f821406b810 con 0x7f821415aaa0 2023-10-16T10:05:30.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.814+0000 7f821ab33640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82140707e0 con 0x7f821415ae80 2023-10-16T10:05:30.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.814+0000 7f82190d0640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f82140ccaa0 0x7f8214070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57832/0 (socket says 172.21.15.84:57832) 2023-10-16T10:05:30.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.814+0000 7f82190d0640 1 -- 172.21.15.84:0/3204212847 learned_addr learned my addr 172.21.15.84:0/3204212847 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:30.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.815+0000 7f8213fff640 1 --2- 172.21.15.84:0/3204212847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f821415ae80 0x7f82140cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:30.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.815+0000 7f82137fe640 1 -- 172.21.15.84:0/3204212847 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657103786 0 0) 0x7f821415ed60 con 0x7f82140ccaa0 2023-10-16T10:05:30.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.815+0000 7f82137fe640 1 -- 172.21.15.84:0/3204212847 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8204003520 con 0x7f82140ccaa0 2023-10-16T10:05:30.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.815+0000 7f82188cf640 1 --2- 172.21.15.84:0/3204212847 >> v2:172.21.15.157:3300/0 conn(0x7f821415aaa0 0x7f821415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:30.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.815+0000 7f82137fe640 1 -- 172.21.15.84:0/3204212847 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2737865594 0 0) 0x7f8204003520 con 0x7f82140ccaa0 2023-10-16T10:05:30.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.815+0000 7f82137fe640 1 -- 172.21.15.84:0/3204212847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f821415ae80 msgr2=0x7f82140cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:30.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.815+0000 7f82137fe640 1 --2- 172.21.15.84:0/3204212847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f821415ae80 0x7f82140cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:30.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.815+0000 7f82137fe640 1 -- 172.21.15.84:0/3204212847 >> v2:172.21.15.157:3300/0 conn(0x7f821415aaa0 msgr2=0x7f821415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:30.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.815+0000 7f82137fe640 1 --2- 172.21.15.84:0/3204212847 >> v2:172.21.15.157:3300/0 conn(0x7f821415aaa0 0x7f821415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:30.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.815+0000 7f82137fe640 1 -- 172.21.15.84:0/3204212847 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821415ef90 con 0x7f82140ccaa0 2023-10-16T10:05:30.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.815+0000 7f82137fe640 1 -- 172.21.15.84:0/3204212847 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f820c0019f0 con 0x7f82140ccaa0 2023-10-16T10:05:30.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.815+0000 7f82137fe640 1 -- 172.21.15.84:0/3204212847 >> v1:172.21.15.84:6789/0 conn(0x7f82140ccaa0 legacy=0x7f8214070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:30.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.816+0000 7f82137fe640 1 --2- 172.21.15.84:0/3204212847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f821415ae80 0x7f8204003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:30.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.816+0000 7f82137fe640 1 --2- 172.21.15.84:0/3204212847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8204003ef0 0x7f82040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:30.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.816+0000 7f82137fe640 1 --2- 172.21.15.84:0/3204212847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8204004d00 0x7f8204005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:30.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.816+0000 7f82137fe640 1 -- 172.21.15.84:0/3204212847 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821406b810 con 0x7f8204003ef0 2023-10-16T10:05:30.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.816+0000 7f82137fe640 1 -- 172.21.15.84:0/3204212847 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82140707e0 con 0x7f8204004d00 2023-10-16T10:05:30.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.816+0000 7f82137fe640 1 -- 172.21.15.84:0/3204212847 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820400b980 con 0x7f821415ae80 2023-10-16T10:05:30.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.816+0000 7f8213fff640 1 --2- 172.21.15.84:0/3204212847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8204003ef0 0x7f82040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:30.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.816+0000 7f8213fff640 1 -- 172.21.15.84:0/3204212847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f821415ae80 msgr2=0x7f8204003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:30.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.816+0000 7f82190d0640 1 --2- 172.21.15.84:0/3204212847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8204004d00 0x7f8204005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:30.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.816+0000 7f8213fff640 1 --2- 172.21.15.84:0/3204212847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f821415ae80 0x7f8204003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:30.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.816+0000 7f8213fff640 1 -- 172.21.15.84:0/3204212847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8204004d00 msgr2=0x7f8204005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:30.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.816+0000 7f8213fff640 1 --2- 172.21.15.84:0/3204212847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8204004d00 0x7f8204005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:30.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.816+0000 7f8213fff640 1 -- 172.21.15.84:0/3204212847 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8204003520 con 0x7f8204003ef0 2023-10-16T10:05:30.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.816+0000 7f82190d0640 1 --2- 172.21.15.84:0/3204212847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8204004d00 0x7f8204005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:30.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.817+0000 7f8213fff640 1 --2- 172.21.15.84:0/3204212847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8204003ef0 0x7f82040047c0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f8200009ac0 tx=0x7f82040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22be2cb1dbd866be server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:30.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.818+0000 7f82137fe640 1 -- 172.21.15.84:0/3204212847 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820002c6d0 con 0x7f8204003ef0 2023-10-16T10:05:30.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.818+0000 7f82137fe640 1 -- 172.21.15.84:0/3204212847 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820002c830 con 0x7f8204003ef0 2023-10-16T10:05:30.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.818+0000 7f82137fe640 1 -- 172.21.15.84:0/3204212847 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820002cbc0 con 0x7f8204003ef0 2023-10-16T10:05:30.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.818+0000 7f821ab33640 1 -- 172.21.15.84:0/3204212847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8204003ef0 msgr2=0x7f82040047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:30.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.818+0000 7f821ab33640 1 --2- 172.21.15.84:0/3204212847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8204003ef0 0x7f82040047c0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f8200009ac0 tx=0x7f82040057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:30.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.818+0000 7f82190d0640 1 -- 172.21.15.84:0/3204212847 reap_dead start 2023-10-16T10:05:30.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.818+0000 7f821ab33640 1 -- 172.21.15.84:0/3204212847 shutdown_connections 2023-10-16T10:05:30.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.818+0000 7f821ab33640 1 -- 172.21.15.84:0/3204212847 >> 172.21.15.84:0/3204212847 conn(0x7f82140cb410 msgr2=0x7f82140cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:30.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.818+0000 7f821ab33640 1 -- 172.21.15.84:0/3204212847 shutdown_connections 2023-10-16T10:05:30.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.818+0000 7f821ab33640 1 -- 172.21.15.84:0/3204212847 wait complete. 2023-10-16T10:05:30.831 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:30.820+0000 7fed6500d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:05:30.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.821+0000 7f821ab33640 1 Processor -- start 2023-10-16T10:05:30.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.821+0000 7f821ab33640 1 -- start start 2023-10-16T10:05:30.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.823+0000 7f821ab33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8204003ef0 0x7f82141efc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:30.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.823+0000 7f821ab33640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82140c5a60 0x7f82140c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:30.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.823+0000 7f821ab33640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82140c7a90 0x7f82140c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:30.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.823+0000 7f821ab33640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82140c8450 con 0x7f82140c5a60 2023-10-16T10:05:30.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.823+0000 7f821ab33640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82140c7130 con 0x7f8204003ef0 2023-10-16T10:05:30.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.823+0000 7f821ab33640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82140c6520 con 0x7f82140c7a90 2023-10-16T10:05:30.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.823+0000 7f8213fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82140c5a60 0x7f82140c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:30.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.823+0000 7f8213fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82140c5a60 0x7f82140c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56190/0 (socket says 172.21.15.84:56190) 2023-10-16T10:05:30.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.823+0000 7f8213fff640 1 -- 172.21.15.84:0/2850228701 learned_addr learned my addr 172.21.15.84:0/2850228701 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:30.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.823+0000 7f8213fff640 1 -- 172.21.15.84:0/2850228701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82140c7a90 msgr2=0x7f82140c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:30.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.823+0000 7f8213fff640 1 --2- 172.21.15.84:0/2850228701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82140c7a90 0x7f82140c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:30.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.823+0000 7f8213fff640 1 -- 172.21.15.84:0/2850228701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8204003ef0 msgr2=0x7f82141efc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:30.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.823+0000 7f8213fff640 1 --2- 172.21.15.84:0/2850228701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8204003ef0 0x7f82141efc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:30.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.823+0000 7f8213fff640 1 -- 172.21.15.84:0/2850228701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82140c6800 con 0x7f82140c5a60 2023-10-16T10:05:30.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.824+0000 7f8213fff640 1 --2- 172.21.15.84:0/2850228701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82140c5a60 0x7f82140c5ec0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f8200037040 tx=0x7f8200004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:30.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.824+0000 7f82117fa640 1 -- 172.21.15.84:0/2850228701 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8200003810 con 0x7f82140c5a60 2023-10-16T10:05:30.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.824+0000 7f82117fa640 1 -- 172.21.15.84:0/2850228701 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8200003970 con 0x7f82140c5a60 2023-10-16T10:05:30.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.824+0000 7f82117fa640 1 -- 172.21.15.84:0/2850228701 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8200003d00 con 0x7f82140c5a60 2023-10-16T10:05:30.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.824+0000 7f821ab33640 1 -- 172.21.15.84:0/2850228701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82140c6ae0 con 0x7f82140c5a60 2023-10-16T10:05:30.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.824+0000 7f821ab33640 1 -- 172.21.15.84:0/2850228701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82140c3ee0 con 0x7f82140c5a60 2023-10-16T10:05:30.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.825+0000 7f821ab33640 1 -- 172.21.15.84:0/2850228701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f821404e160 con 0x7f82140c5a60 2023-10-16T10:05:30.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.826+0000 7f82117fa640 1 -- 172.21.15.84:0/2850228701 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 45) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f8200034440 con 0x7f82140c5a60 2023-10-16T10:05:30.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.827+0000 7f82117fa640 1 --2- 172.21.15.84:0/2850228701 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f81f806af90 0x7f81f806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:30.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.827+0000 7f82117fa640 1 -- 172.21.15.84:0/2850228701 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f82000ab9e0 con 0x7f82140c5a60 2023-10-16T10:05:30.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.827+0000 7f82188cf640 1 -- 172.21.15.84:0/2850228701 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f81f806af90 msgr2=0x7f81f806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/494168607 2023-10-16T10:05:30.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.827+0000 7f82188cf640 1 --2- 172.21.15.84:0/2850228701 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f81f806af90 0x7f81f806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:05:30.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:30.831+0000 7f82117fa640 1 -- 172.21.15.84:0/2850228701 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8200076840 con 0x7f82140c5a60 2023-10-16T10:05:30.961 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:30.959+0000 7f921864b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:05:31.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.013+0000 7f821ab33640 1 -- 172.21.15.84:0/2850228701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82140cdc60 con 0x7f82140c5a60 2023-10-16T10:05:31.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.016+0000 7f82117fa640 1 -- 172.21.15.84:0/2850228701 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+17041 (secure 0 0 0) 0x7f8200039090 con 0x7f82140c5a60 2023-10-16T10:05:31.018 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:31.018 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":96,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:05:25.080022+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:05:08.636077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-16T10:05:09.841985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/4027382473":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/3685429931":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/504267391":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/3778743167":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/150347074":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/3151132884":"2023-10-17T10:05:25.079928+0000","172.21.15.84:6833/147697688":"2023-10-17T10:05:25.079928+0000","172.21.15.84:6832/147697688":"2023-10-17T10:05:25.079928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:05:31.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.018+0000 7f821ab33640 1 -- 172.21.15.84:0/2850228701 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f81f806af90 msgr2=0x7f81f806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:05:31.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.018+0000 7f821ab33640 1 --2- 172.21.15.84:0/2850228701 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f81f806af90 0x7f81f806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.019+0000 7f821ab33640 1 -- 172.21.15.84:0/2850228701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82140c5a60 msgr2=0x7f82140c5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:31.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.019+0000 7f821ab33640 1 --2- 172.21.15.84:0/2850228701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82140c5a60 0x7f82140c5ec0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f8200037040 tx=0x7f8200004730 comp rx=0 tx=0).stop 2023-10-16T10:05:31.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.019+0000 7f821ab33640 1 -- 172.21.15.84:0/2850228701 shutdown_connections 2023-10-16T10:05:31.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.019+0000 7f821ab33640 1 --2- 172.21.15.84:0/2850228701 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7f81f806af90 0x7f81f806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.019+0000 7f821ab33640 1 --2- 172.21.15.84:0/2850228701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82140c7a90 0x7f82140c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.019+0000 7f821ab33640 1 --2- 172.21.15.84:0/2850228701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82140c5a60 0x7f82140c5ec0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.019+0000 7f821ab33640 1 --2- 172.21.15.84:0/2850228701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8204003ef0 0x7f82141efc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.019+0000 7f821ab33640 1 -- 172.21.15.84:0/2850228701 >> 172.21.15.84:0/2850228701 conn(0x7f82140cb410 msgr2=0x7f82140b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:31.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.019+0000 7f821ab33640 1 -- 172.21.15.84:0/2850228701 shutdown_connections 2023-10-16T10:05:31.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.019+0000 7f821ab33640 1 -- 172.21.15.84:0/2850228701 wait complete. 2023-10-16T10:05:31.038 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:05:08.636077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:05:09.841985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:05:31.039 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:05:31.115 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:31.114+0000 7fed6500d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:05:31.173 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:31.171+0000 7fed6500d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:05:31.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.228+0000 7fea4e91d640 1 Processor -- start 2023-10-16T10:05:31.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.228+0000 7fea4e91d640 1 -- start start 2023-10-16T10:05:31.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.229+0000 7fea4e91d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fea4806c850 0x7fea480ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:31.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.229+0000 7fea4e91d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea48070360 con 0x7fea48159ab0 2023-10-16T10:05:31.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.229+0000 7fea4e91d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea480cd1f0 con 0x7fea48159e90 2023-10-16T10:05:31.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.229+0000 7fea4e91d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4815d890 con 0x7fea4806c850 2023-10-16T10:05:31.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.230+0000 7fea4ceba640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fea4806c850 0x7fea480ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:31.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.230+0000 7fea4ceba640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fea4806c850 0x7fea480ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56200/0 (socket says 172.21.15.84:56200) 2023-10-16T10:05:31.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.230+0000 7fea4ceba640 1 -- 172.21.15.84:0/2724060412 learned_addr learned my addr 172.21.15.84:0/2724060412 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:31.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.230+0000 7fea4ceba640 1 -- 172.21.15.84:0/2724060412 >> v1:172.21.15.157:6789/0 conn(0x7fea48159e90 legacy=0x7fea4806c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:31.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.230+0000 7fea4ceba640 1 -- 172.21.15.84:0/2724060412 >> v1:172.21.15.84:6789/0 conn(0x7fea48159ab0 legacy=0x7fea4806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:31.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.230+0000 7fea4ceba640 1 -- 172.21.15.84:0/2724060412 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea480cd4d0 con 0x7fea4806c850 2023-10-16T10:05:31.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.230+0000 7fea46ffd640 1 -- 172.21.15.84:0/2724060412 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387258162 0 0) 0x7fea48070360 con 0x7fea48159ab0 2023-10-16T10:05:31.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.231+0000 7fea4ceba640 1 --2- 172.21.15.84:0/2724060412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea4806c850 0x7fea480ce370 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fea4000d150 tx=0x7fea4002e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184a99f5325a82b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:31.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.231+0000 7fea46ffd640 1 -- 172.21.15.84:0/2724060412 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4002ea00 con 0x7fea4806c850 2023-10-16T10:05:31.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.231+0000 7fea46ffd640 1 -- 172.21.15.84:0/2724060412 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4002eb60 con 0x7fea4806c850 2023-10-16T10:05:31.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.231+0000 7fea46ffd640 1 -- 172.21.15.84:0/2724060412 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40009600 con 0x7fea4806c850 2023-10-16T10:05:31.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.232+0000 7fea4e91d640 1 -- 172.21.15.84:0/2724060412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea4806c850 msgr2=0x7fea480ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:31.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.232+0000 7fea4e91d640 1 --2- 172.21.15.84:0/2724060412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea4806c850 0x7fea480ce370 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fea4000d150 tx=0x7fea4002e060 comp rx=0 tx=0).stop 2023-10-16T10:05:31.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.232+0000 7fea4e91d640 1 -- 172.21.15.84:0/2724060412 shutdown_connections 2023-10-16T10:05:31.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.232+0000 7fea4e91d640 1 --2- 172.21.15.84:0/2724060412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea4806c850 0x7fea480ce370 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.232+0000 7fea4e91d640 1 -- 172.21.15.84:0/2724060412 >> 172.21.15.84:0/2724060412 conn(0x7fea480ce9b0 msgr2=0x7fea480cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:31.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.232+0000 7fea4e91d640 1 -- 172.21.15.84:0/2724060412 shutdown_connections 2023-10-16T10:05:31.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.233+0000 7fea4e91d640 1 -- 172.21.15.84:0/2724060412 wait complete. 2023-10-16T10:05:31.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.234+0000 7fea4e91d640 1 Processor -- start 2023-10-16T10:05:31.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.234+0000 7fea4e91d640 1 -- start start 2023-10-16T10:05:31.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.235+0000 7fea4e91d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea4806c850 0x7fea4815c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:31.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.235+0000 7fea4e91d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea48159ab0 0x7fea480c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:31.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.235+0000 7fea4e91d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea48159e90 0x7fea4815ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:31.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.235+0000 7fea4e91d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4815a700 con 0x7fea48159ab0 2023-10-16T10:05:31.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.235+0000 7fea4e91d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4815ca50 con 0x7fea48159e90 2023-10-16T10:05:31.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.235+0000 7fea4e91d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea480c3f60 con 0x7fea4806c850 2023-10-16T10:05:31.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.235+0000 7fea477fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea48159ab0 0x7fea480c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:31.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.236+0000 7fea477fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea48159ab0 0x7fea480c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56212/0 (socket says 172.21.15.84:56212) 2023-10-16T10:05:31.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.236+0000 7fea477fe640 1 -- 172.21.15.84:0/2144815835 learned_addr learned my addr 172.21.15.84:0/2144815835 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:31.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.236+0000 7fea47fff640 1 --2- 172.21.15.84:0/2144815835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea4806c850 0x7fea4815c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:31.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.236+0000 7fea4ceba640 1 --2- 172.21.15.84:0/2144815835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea48159e90 0x7fea4815ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:31.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.236+0000 7fea477fe640 1 -- 172.21.15.84:0/2144815835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea4806c850 msgr2=0x7fea4815c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:31.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.236+0000 7fea477fe640 1 --2- 172.21.15.84:0/2144815835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea4806c850 0x7fea4815c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.236+0000 7fea477fe640 1 -- 172.21.15.84:0/2144815835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea48159e90 msgr2=0x7fea4815ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:31.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.236+0000 7fea477fe640 1 --2- 172.21.15.84:0/2144815835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea48159e90 0x7fea4815ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.236+0000 7fea477fe640 1 -- 172.21.15.84:0/2144815835 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4000bd40 con 0x7fea48159ab0 2023-10-16T10:05:31.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.236+0000 7fea4ceba640 1 --2- 172.21.15.84:0/2144815835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea48159e90 0x7fea4815ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:31.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.236+0000 7fea477fe640 1 --2- 172.21.15.84:0/2144815835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea48159ab0 0x7fea480c3660 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00c430 tx=0x7fea3c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:31.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.237+0000 7fea44ff9640 1 -- 172.21.15.84:0/2144815835 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c011790 con 0x7fea48159ab0 2023-10-16T10:05:31.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.237+0000 7fea44ff9640 1 -- 172.21.15.84:0/2144815835 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c0118f0 con 0x7fea48159ab0 2023-10-16T10:05:31.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.237+0000 7fea4e91d640 1 -- 172.21.15.84:0/2144815835 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea480c4240 con 0x7fea48159ab0 2023-10-16T10:05:31.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.237+0000 7fea44ff9640 1 -- 172.21.15.84:0/2144815835 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c011ce0 con 0x7fea48159ab0 2023-10-16T10:05:31.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.237+0000 7fea4e91d640 1 -- 172.21.15.84:0/2144815835 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea480c4520 con 0x7fea48159ab0 2023-10-16T10:05:31.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.238+0000 7fea4e91d640 1 -- 172.21.15.84:0/2144815835 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4815a870 con 0x7fea48159ab0 2023-10-16T10:05:31.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.240+0000 7fea44ff9640 1 -- 172.21.15.84:0/2144815835 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 45) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fea3c01c460 con 0x7fea48159ab0 2023-10-16T10:05:31.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.240+0000 7fea44ff9640 1 --2- 172.21.15.84:0/2144815835 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fea3406af90 0x7fea3406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:31.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.241+0000 7fea44ff9640 1 -- 172.21.15.84:0/2144815835 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fea3c089800 con 0x7fea48159ab0 2023-10-16T10:05:31.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.241+0000 7fea47fff640 1 -- 172.21.15.84:0/2144815835 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fea3406af90 msgr2=0x7fea3406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/494168607 2023-10-16T10:05:31.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.241+0000 7fea47fff640 1 --2- 172.21.15.84:0/2144815835 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fea3406af90 0x7fea3406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:05:31.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.245+0000 7fea44ff9640 1 -- 172.21.15.84:0/2144815835 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fea3c0546d0 con 0x7fea48159ab0 2023-10-16T10:05:31.273 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:31.271+0000 7f921864b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:05:31.288 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:31.287+0000 7fed6500d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:05:31.331 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:31.330+0000 7f921864b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:05:31.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.438+0000 7fea4e91d640 1 -- 172.21.15.84:0/2144815835 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fea480cda60 con 0x7fea48159ab0 2023-10-16T10:05:31.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.439+0000 7fea44ff9640 1 -- 172.21.15.84:0/2144815835 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7fea3c057cf0 con 0x7fea48159ab0 2023-10-16T10:05:31.441 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:05:31.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.441+0000 7fea4e91d640 1 -- 172.21.15.84:0/2144815835 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fea3406af90 msgr2=0x7fea3406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:05:31.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.441+0000 7fea4e91d640 1 --2- 172.21.15.84:0/2144815835 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fea3406af90 0x7fea3406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.441+0000 7fea4e91d640 1 -- 172.21.15.84:0/2144815835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea48159ab0 msgr2=0x7fea480c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:31.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.441+0000 7fea4e91d640 1 --2- 172.21.15.84:0/2144815835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea48159ab0 0x7fea480c3660 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00c430 tx=0x7fea3c00c8f0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.441+0000 7fea4e91d640 1 -- 172.21.15.84:0/2144815835 shutdown_connections 2023-10-16T10:05:31.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.441+0000 7fea4e91d640 1 --2- 172.21.15.84:0/2144815835 >> [v2:172.21.15.84:6832/494168607,v1:172.21.15.84:6833/494168607] conn(0x7fea3406af90 0x7fea3406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.441+0000 7fea4e91d640 1 --2- 172.21.15.84:0/2144815835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea48159e90 0x7fea4815ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.441+0000 7fea4e91d640 1 --2- 172.21.15.84:0/2144815835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea48159ab0 0x7fea480c3660 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.441+0000 7fea4e91d640 1 --2- 172.21.15.84:0/2144815835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea4806c850 0x7fea4815c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.441+0000 7fea4e91d640 1 -- 172.21.15.84:0/2144815835 >> 172.21.15.84:0/2144815835 conn(0x7fea480ce9b0 msgr2=0x7fea480ae430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:31.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.442+0000 7fea4e91d640 1 -- 172.21.15.84:0/2144815835 shutdown_connections 2023-10-16T10:05:31.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.442+0000 7fea4e91d640 1 -- 172.21.15.84:0/2144815835 wait complete. 2023-10-16T10:05:31.447 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:31.446+0000 7f921864b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:05:31.463 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:05:31.527 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:31.525+0000 7f919d9a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:31.527 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:31.525+0000 7f919d9a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:31.528 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:31.525+0000 7f919d9a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:31.528 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:31.525+0000 7f919d9a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:31.528 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:31.525+0000 7f919d9a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:31.528 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:31.526+0000 7f919a9a2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:31.528 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:31.526+0000 7f919a9a2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:31.529 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:31.526+0000 7f919a9a2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:31.529 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:31.526+0000 7f919a9a2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:31.529 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:31.526+0000 7f919a9a2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:31.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.646+0000 7f04e0733640 1 Processor -- start 2023-10-16T10:05:31.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.646+0000 7f04e0733640 1 -- start start 2023-10-16T10:05:31.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.648+0000 7f04e0733640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f04d80ad350 0x7f04d8069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:31.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.648+0000 7f04e0733640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d806a730 con 0x7f04d806a2e0 2023-10-16T10:05:31.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.648+0000 7f04e0733640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d80ac890 con 0x7f04d815d150 2023-10-16T10:05:31.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.648+0000 7f04e0733640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d815d530 con 0x7f04d80ad350 2023-10-16T10:05:31.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.648+0000 7f04ddcce640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f04d80ad350 0x7f04d8069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:31.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.648+0000 7f04ddcce640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f04d80ad350 0x7f04d8069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56284/0 (socket says 172.21.15.84:56284) 2023-10-16T10:05:31.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.648+0000 7f04ddcce640 1 -- 172.21.15.84:0/327227951 learned_addr learned my addr 172.21.15.84:0/327227951 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:31.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.648+0000 7f04decd0640 1 --1- 172.21.15.84:0/327227951 >> v1:172.21.15.84:6789/0 conn(0x7f04d806a2e0 0x7f04d80c9000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57844/0 (socket says 172.21.15.84:57844) 2023-10-16T10:05:31.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.648+0000 7f04ddcce640 1 -- 172.21.15.84:0/327227951 >> v1:172.21.15.157:6789/0 conn(0x7f04d815d150 legacy=0x7f04d8160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:31.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.648+0000 7f04dd4cd640 1 -- 172.21.15.84:0/327227951 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3822546619 0 0) 0x7f04d0001490 con 0x7f04d806a2e0 2023-10-16T10:05:31.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.648+0000 7f04ddcce640 1 -- 172.21.15.84:0/327227951 >> v1:172.21.15.84:6789/0 conn(0x7f04d806a2e0 legacy=0x7f04d80c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:31.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.648+0000 7f04ddcce640 1 -- 172.21.15.84:0/327227951 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d80ad7a0 con 0x7f04d80ad350 2023-10-16T10:05:31.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.649+0000 7f04ddcce640 1 --2- 172.21.15.84:0/327227951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04d80ad350 0x7f04d8069da0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009870 tx=0x7f04cc02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1641d2871dc24881 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:31.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.650+0000 7f04dd4cd640 1 -- 172.21.15.84:0/327227951 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc030850 con 0x7f04d80ad350 2023-10-16T10:05:31.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.650+0000 7f04dd4cd640 1 -- 172.21.15.84:0/327227951 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04cc0309b0 con 0x7f04d80ad350 2023-10-16T10:05:31.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.650+0000 7f04dd4cd640 1 -- 172.21.15.84:0/327227951 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc030d40 con 0x7f04d80ad350 2023-10-16T10:05:31.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.650+0000 7f04e0733640 1 -- 172.21.15.84:0/327227951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04d80ad350 msgr2=0x7f04d8069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:31.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.650+0000 7f04e0733640 1 --2- 172.21.15.84:0/327227951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04d80ad350 0x7f04d8069da0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009870 tx=0x7f04cc02b990 comp rx=0 tx=0).stop 2023-10-16T10:05:31.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.650+0000 7f04e0733640 1 -- 172.21.15.84:0/327227951 shutdown_connections 2023-10-16T10:05:31.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.650+0000 7f04e0733640 1 --2- 172.21.15.84:0/327227951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04d80ad350 0x7f04d8069da0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.650+0000 7f04e0733640 1 -- 172.21.15.84:0/327227951 >> 172.21.15.84:0/327227951 conn(0x7f04d80ce9b0 msgr2=0x7f04d80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:31.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.650+0000 7f04e0733640 1 -- 172.21.15.84:0/327227951 shutdown_connections 2023-10-16T10:05:31.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.650+0000 7f04e0733640 1 -- 172.21.15.84:0/327227951 wait complete. 2023-10-16T10:05:31.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.652+0000 7f04e0733640 1 Processor -- start 2023-10-16T10:05:31.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.652+0000 7f04e0733640 1 -- start start 2023-10-16T10:05:31.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.653+0000 7f04e0733640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04d806a2e0 0x7f04d8155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:31.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.653+0000 7f04e0733640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04d80ad350 0x7f04d80b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:31.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.653+0000 7f04e0733640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04d815d150 0x7f04d8158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:31.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.653+0000 7f04e0733640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d8155cd0 con 0x7f04d806a2e0 2023-10-16T10:05:31.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.653+0000 7f04e0733640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d80aecc0 con 0x7f04d80ad350 2023-10-16T10:05:31.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.653+0000 7f04e0733640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d80c3ce0 con 0x7f04d815d150 2023-10-16T10:05:31.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.653+0000 7f04de4cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04d806a2e0 0x7f04d8155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:31.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.653+0000 7f04de4cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04d806a2e0 0x7f04d8155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56298/0 (socket says 172.21.15.84:56298) 2023-10-16T10:05:31.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.653+0000 7f04de4cf640 1 -- 172.21.15.84:0/3941416123 learned_addr learned my addr 172.21.15.84:0/3941416123 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:31.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.654+0000 7f04de4cf640 1 -- 172.21.15.84:0/3941416123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04d815d150 msgr2=0x7f04d8158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:31.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.654+0000 7f04de4cf640 1 --2- 172.21.15.84:0/3941416123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04d815d150 0x7f04d8158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.654+0000 7f04de4cf640 1 -- 172.21.15.84:0/3941416123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04d80ad350 msgr2=0x7f04d80b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:31.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.654+0000 7f04de4cf640 1 --2- 172.21.15.84:0/3941416123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04d80ad350 0x7f04d80b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.654+0000 7f04de4cf640 1 -- 172.21.15.84:0/3941416123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc0094d0 con 0x7f04d806a2e0 2023-10-16T10:05:31.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.654+0000 7f04de4cf640 1 --2- 172.21.15.84:0/3941416123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04d806a2e0 0x7f04d8155790 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f04d400b3f0 tx=0x7f04d400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:31.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.654+0000 7f04caffd640 1 -- 172.21.15.84:0/3941416123 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d4011830 con 0x7f04d806a2e0 2023-10-16T10:05:31.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.654+0000 7f04caffd640 1 -- 172.21.15.84:0/3941416123 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d4011990 con 0x7f04d806a2e0 2023-10-16T10:05:31.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.654+0000 7f04e0733640 1 -- 172.21.15.84:0/3941416123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d80b0f60 con 0x7f04d806a2e0 2023-10-16T10:05:31.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.654+0000 7f04caffd640 1 -- 172.21.15.84:0/3941416123 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d4011d40 con 0x7f04d806a2e0 2023-10-16T10:05:31.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.655+0000 7f04e0733640 1 -- 172.21.15.84:0/3941416123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d80c4210 con 0x7f04d806a2e0 2023-10-16T10:05:31.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.656+0000 7f04e0733640 1 -- 172.21.15.84:0/3941416123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d804e160 con 0x7f04d806a2e0 2023-10-16T10:05:31.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.656+0000 7f04caffd640 1 -- 172.21.15.84:0/3941416123 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 46) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f04d401a400 con 0x7f04d806a2e0 2023-10-16T10:05:31.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.658+0000 7f04caffd640 1 -- 172.21.15.84:0/3941416123 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f04d4089200 con 0x7f04d806a2e0 2023-10-16T10:05:31.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.662+0000 7f04caffd640 1 -- 172.21.15.84:0/3941416123 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f04d4054450 con 0x7f04d806a2e0 2023-10-16T10:05:31.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.847+0000 7f04e0733640 1 -- 172.21.15.84:0/3941416123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f04d80c5510 con 0x7f04d806a2e0 2023-10-16T10:05:31.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.848+0000 7f04caffd640 1 -- 172.21.15.84:0/3941416123 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+11 (secure 0 0 0) 0x7f04d4057a70 con 0x7f04d806a2e0 2023-10-16T10:05:31.849 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T10:05:31.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.850+0000 7f04e0733640 1 -- 172.21.15.84:0/3941416123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04d806a2e0 msgr2=0x7f04d8155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:31.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.850+0000 7f04e0733640 1 --2- 172.21.15.84:0/3941416123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04d806a2e0 0x7f04d8155790 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f04d400b3f0 tx=0x7f04d400b8b0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.850+0000 7f04e0733640 1 -- 172.21.15.84:0/3941416123 shutdown_connections 2023-10-16T10:05:31.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.850+0000 7f04e0733640 1 --2- 172.21.15.84:0/3941416123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04d815d150 0x7f04d8158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.850+0000 7f04e0733640 1 --2- 172.21.15.84:0/3941416123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04d80ad350 0x7f04d80b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.850+0000 7f04e0733640 1 --2- 172.21.15.84:0/3941416123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04d806a2e0 0x7f04d8155790 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:31.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.850+0000 7f04e0733640 1 -- 172.21.15.84:0/3941416123 >> 172.21.15.84:0/3941416123 conn(0x7f04d80ce9b0 msgr2=0x7f04d806f770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:31.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.850+0000 7f04e0733640 1 -- 172.21.15.84:0/3941416123 shutdown_connections 2023-10-16T10:05:31.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:31.851+0000 7f04e0733640 1 -- 172.21.15.84:0/3941416123 wait complete. 2023-10-16T10:05:31.868 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:05:32.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.070+0000 7fdc064c6640 1 Processor -- start 2023-10-16T10:05:32.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.070+0000 7fdc064c6640 1 -- start start 2023-10-16T10:05:32.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.071+0000 7fdc064c6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc0006b660 0x7fdc0006ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:32.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.071+0000 7fdc064c6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdc001428c0 0x7fdc000c9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:32.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.071+0000 7fdc064c6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc000c9ae0 con 0x7fdc000c9690 2023-10-16T10:05:32.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.071+0000 7fdc064c6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc000cc6b0 con 0x7fdc001428c0 2023-10-16T10:05:32.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.071+0000 7fdc064c6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc001599b0 con 0x7fdc0006b660 2023-10-16T10:05:32.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.072+0000 7fdbff7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdc001428c0 0x7fdc000c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:32.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.072+0000 7fdbff7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdc001428c0 0x7fdc000c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56304/0 (socket says 172.21.15.84:56304) 2023-10-16T10:05:32.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.072+0000 7fdbff7fe640 1 -- 172.21.15.84:0/2012596138 learned_addr learned my addr 172.21.15.84:0/2012596138 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:32.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.072+0000 7fdbfffff640 1 --2- 172.21.15.84:0/2012596138 >> v2:172.21.15.157:3300/0 conn(0x7fdc0006b660 0x7fdc0006ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:32.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.072+0000 7fdbff7fe640 1 -- 172.21.15.84:0/2012596138 >> v2:172.21.15.157:3300/0 conn(0x7fdc0006b660 msgr2=0x7fdc0006ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:32.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.072+0000 7fdbff7fe640 1 --2- 172.21.15.84:0/2012596138 >> v2:172.21.15.157:3300/0 conn(0x7fdc0006b660 0x7fdc0006ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:32.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.072+0000 7fdbff7fe640 1 -- 172.21.15.84:0/2012596138 >> v1:172.21.15.157:6789/0 conn(0x7fdc000c9690 legacy=0x7fdc0015d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:32.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.072+0000 7fdbff7fe640 1 -- 172.21.15.84:0/2012596138 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0006c290 con 0x7fdc001428c0 2023-10-16T10:05:32.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.073+0000 7fdbff7fe640 1 --2- 172.21.15.84:0/2012596138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc001428c0 0x7fdc000c9150 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fdc000c9120 tx=0x7fdbf402deb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e854fca0a9da9d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:32.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.073+0000 7fdbfeffd640 1 -- 172.21.15.84:0/2012596138 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf402ec60 con 0x7fdc001428c0 2023-10-16T10:05:32.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.074+0000 7fdbfeffd640 1 -- 172.21.15.84:0/2012596138 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf4032070 con 0x7fdc001428c0 2023-10-16T10:05:32.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.074+0000 7fdbfeffd640 1 -- 172.21.15.84:0/2012596138 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4038400 con 0x7fdc001428c0 2023-10-16T10:05:32.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.074+0000 7fdc064c6640 1 -- 172.21.15.84:0/2012596138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc001428c0 msgr2=0x7fdc000c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:32.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.074+0000 7fdc064c6640 1 --2- 172.21.15.84:0/2012596138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc001428c0 0x7fdc000c9150 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fdc000c9120 tx=0x7fdbf402deb0 comp rx=0 tx=0).stop 2023-10-16T10:05:32.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.074+0000 7fdc064c6640 1 -- 172.21.15.84:0/2012596138 shutdown_connections 2023-10-16T10:05:32.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.074+0000 7fdc064c6640 1 --2- 172.21.15.84:0/2012596138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc001428c0 0x7fdc000c9150 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:32.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.074+0000 7fdc064c6640 1 --2- 172.21.15.84:0/2012596138 >> v2:172.21.15.157:3300/0 conn(0x7fdc0006b660 0x7fdc0006ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:32.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.074+0000 7fdc064c6640 1 -- 172.21.15.84:0/2012596138 >> 172.21.15.84:0/2012596138 conn(0x7fdc000ce9b0 msgr2=0x7fdc000cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:32.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.074+0000 7fdc064c6640 1 -- 172.21.15.84:0/2012596138 shutdown_connections 2023-10-16T10:05:32.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.074+0000 7fdc064c6640 1 -- 172.21.15.84:0/2012596138 wait complete. 2023-10-16T10:05:32.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.076+0000 7fdc064c6640 1 Processor -- start 2023-10-16T10:05:32.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.076+0000 7fdc064c6640 1 -- start start 2023-10-16T10:05:32.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.077+0000 7fdc064c6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc0006b660 0x7fdc000c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:32.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.077+0000 7fdc064c6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc000c9690 0x7fdc000c4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:32.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.077+0000 7fdc064c6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc001428c0 0x7fdc0015bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:32.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.077+0000 7fdc064c6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0015af60 con 0x7fdc000c9690 2023-10-16T10:05:32.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.077+0000 7fdc064c6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000c47d0 con 0x7fdc001428c0 2023-10-16T10:05:32.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.077+0000 7fdc064c6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000c8a20 con 0x7fdc0006b660 2023-10-16T10:05:32.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.077+0000 7fdbff7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc000c9690 0x7fdc000c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:32.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.077+0000 7fdbff7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc000c9690 0x7fdc000c4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56316/0 (socket says 172.21.15.84:56316) 2023-10-16T10:05:32.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.077+0000 7fdbff7fe640 1 -- 172.21.15.84:0/3320791907 learned_addr learned my addr 172.21.15.84:0/3320791907 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:32.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.078+0000 7fdbfffff640 1 --2- 172.21.15.84:0/3320791907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc0006b660 0x7fdc000c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:32.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.078+0000 7fdbff7fe640 1 -- 172.21.15.84:0/3320791907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc0006b660 msgr2=0x7fdc000c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:32.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.078+0000 7fdbff7fe640 1 --2- 172.21.15.84:0/3320791907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc0006b660 0x7fdc000c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:32.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.078+0000 7fdbff7fe640 1 -- 172.21.15.84:0/3320791907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc001428c0 msgr2=0x7fdc0015bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:32.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.078+0000 7fdc04a63640 1 --2- 172.21.15.84:0/3320791907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc001428c0 0x7fdc0015bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:32.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.078+0000 7fdbff7fe640 1 --2- 172.21.15.84:0/3320791907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc001428c0 0x7fdc0015bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:32.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.078+0000 7fdbff7fe640 1 -- 172.21.15.84:0/3320791907 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf400b760 con 0x7fdc000c9690 2023-10-16T10:05:32.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.078+0000 7fdbff7fe640 1 --2- 172.21.15.84:0/3320791907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc000c9690 0x7fdc000c4290 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4000c00 tx=0x7fdbf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:32.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.079+0000 7fdbfcff9640 1 -- 172.21.15.84:0/3320791907 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4006df0 con 0x7fdc000c9690 2023-10-16T10:05:32.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.079+0000 7fdbfcff9640 1 -- 172.21.15.84:0/3320791907 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf4043470 con 0x7fdc000c9690 2023-10-16T10:05:32.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.079+0000 7fdc064c6640 1 -- 172.21.15.84:0/3320791907 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0015a630 con 0x7fdc000c9690 2023-10-16T10:05:32.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.079+0000 7fdbfcff9640 1 -- 172.21.15.84:0/3320791907 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4043820 con 0x7fdc000c9690 2023-10-16T10:05:32.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.079+0000 7fdc064c6640 1 -- 172.21.15.84:0/3320791907 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0015abf0 con 0x7fdc000c9690 2023-10-16T10:05:32.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.080+0000 7fdc064c6640 1 -- 172.21.15.84:0/3320791907 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc0015adb0 con 0x7fdc000c9690 2023-10-16T10:05:32.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.081+0000 7fdbfcff9640 1 -- 172.21.15.84:0/3320791907 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 46) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fdbf4043980 con 0x7fdc000c9690 2023-10-16T10:05:32.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.082+0000 7fdbfcff9640 1 -- 172.21.15.84:0/3320791907 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6397+0+0 (secure 0 0 0) 0x7fdbf40adf90 con 0x7fdc000c9690 2023-10-16T10:05:32.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.086+0000 7fdbfcff9640 1 -- 172.21.15.84:0/3320791907 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdbf40790f0 con 0x7fdc000c9690 2023-10-16T10:05:32.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.281+0000 7fdc064c6640 1 -- 172.21.15.84:0/3320791907 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc000c9ae0 con 0x7fdc000c9690 2023-10-16T10:05:32.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.282+0000 7fdbfcff9640 1 -- 172.21.15.84:0/3320791907 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc000c9ae0 con 0x7fdc000c9690 2023-10-16T10:05:32.283 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T10:05:32.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.284+0000 7fdc064c6640 1 -- 172.21.15.84:0/3320791907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc000c9690 msgr2=0x7fdc000c4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:32.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.284+0000 7fdc064c6640 1 --2- 172.21.15.84:0/3320791907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc000c9690 0x7fdc000c4290 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4000c00 tx=0x7fdbf40046c0 comp rx=0 tx=0).stop 2023-10-16T10:05:32.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.285+0000 7fdc064c6640 1 -- 172.21.15.84:0/3320791907 shutdown_connections 2023-10-16T10:05:32.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.285+0000 7fdc064c6640 1 --2- 172.21.15.84:0/3320791907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc001428c0 0x7fdc0015bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:32.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.285+0000 7fdc064c6640 1 --2- 172.21.15.84:0/3320791907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc000c9690 0x7fdc000c4290 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:32.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.285+0000 7fdc064c6640 1 --2- 172.21.15.84:0/3320791907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc0006b660 0x7fdc000c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:32.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.285+0000 7fdc064c6640 1 -- 172.21.15.84:0/3320791907 >> 172.21.15.84:0/3320791907 conn(0x7fdc000ce9b0 msgr2=0x7fdc00157c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:32.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.285+0000 7fdc064c6640 1 -- 172.21.15.84:0/3320791907 shutdown_connections 2023-10-16T10:05:32.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.285+0000 7fdc064c6640 1 -- 172.21.15.84:0/3320791907 wait complete. 2023-10-16T10:05:32.304 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:05:32.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.525+0000 7f75f8541640 1 Processor -- start 2023-10-16T10:05:32.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.525+0000 7f75f8541640 1 -- start start 2023-10-16T10:05:32.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.526+0000 7f75f8541640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f75f00b0b20 0x7f75f0164b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:32.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.526+0000 7f75f8541640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f006f7f0 con 0x7f75f0159f00 2023-10-16T10:05:32.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.526+0000 7f75f8541640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f006c5e0 con 0x7f75f0159b20 2023-10-16T10:05:32.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.526+0000 7f75f8541640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f006c8c0 con 0x7f75f00b0b20 2023-10-16T10:05:32.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.527+0000 7f75f6ade640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f75f00b0b20 0x7f75f0164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:32.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.527+0000 7f75f5adc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f75f0159f00 0x7f75f00b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57850/0 (socket says 172.21.15.84:57850) 2023-10-16T10:05:32.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.527+0000 7f75f5adc640 1 -- 172.21.15.84:0/402907588 learned_addr learned my addr 172.21.15.84:0/402907588 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:32.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.527+0000 7f75f6ade640 1 -- 172.21.15.84:0/402907588 >> v1:172.21.15.157:6789/0 conn(0x7f75f0159b20 legacy=0x7f75f006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:32.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.527+0000 7f75f52db640 1 -- 172.21.15.84:0/402907588 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139905655 0 0) 0x7f75f006f7f0 con 0x7f75f0159f00 2023-10-16T10:05:32.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.527+0000 7f75f6ade640 1 -- 172.21.15.84:0/402907588 >> v1:172.21.15.84:6789/0 conn(0x7f75f0159f00 legacy=0x7f75f00b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:32.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.527+0000 7f75f6ade640 1 -- 172.21.15.84:0/402907588 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f006cba0 con 0x7f75f00b0b20 2023-10-16T10:05:32.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.528+0000 7f75f6ade640 1 --2- 172.21.15.84:0/402907588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f00b0b20 0x7f75f0164b40 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009d90 tx=0x7f75e802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bac686d7946f8a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:32.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.529+0000 7f75f52db640 1 -- 172.21.15.84:0/402907588 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e802cc60 con 0x7f75f00b0b20 2023-10-16T10:05:32.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.529+0000 7f75f52db640 1 -- 172.21.15.84:0/402907588 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e802cdc0 con 0x7f75f00b0b20 2023-10-16T10:05:32.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.529+0000 7f75f52db640 1 -- 172.21.15.84:0/402907588 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8034660 con 0x7f75f00b0b20 2023-10-16T10:05:32.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.529+0000 7f75f8541640 1 -- 172.21.15.84:0/402907588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f00b0b20 msgr2=0x7f75f0164b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:32.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.529+0000 7f75f8541640 1 --2- 172.21.15.84:0/402907588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f00b0b20 0x7f75f0164b40 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009d90 tx=0x7f75e802beb0 comp rx=0 tx=0).stop 2023-10-16T10:05:32.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.529+0000 7f75f8541640 1 -- 172.21.15.84:0/402907588 shutdown_connections 2023-10-16T10:05:32.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.529+0000 7f75f8541640 1 --2- 172.21.15.84:0/402907588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f00b0b20 0x7f75f0164b40 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:32.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.529+0000 7f75f8541640 1 -- 172.21.15.84:0/402907588 >> 172.21.15.84:0/402907588 conn(0x7f75f00cb410 msgr2=0x7f75f00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:32.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.529+0000 7f75f8541640 1 -- 172.21.15.84:0/402907588 shutdown_connections 2023-10-16T10:05:32.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.530+0000 7f75f8541640 1 -- 172.21.15.84:0/402907588 wait complete. 2023-10-16T10:05:32.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.531+0000 7f75f8541640 1 Processor -- start 2023-10-16T10:05:32.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.531+0000 7f75f8541640 1 -- start start 2023-10-16T10:05:32.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.532+0000 7f75f8541640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75f00b0b20 0x7f75f00c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:32.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.532+0000 7f75f8541640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0159b20 0x7f75f00b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:32.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.532+0000 7f75f8541640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f0159f00 0x7f75f014c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:32.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.532+0000 7f75f8541640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f00c7ce0 con 0x7f75f0159f00 2023-10-16T10:05:32.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.532+0000 7f75f8541640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f00c4700 con 0x7f75f0159b20 2023-10-16T10:05:32.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.532+0000 7f75f8541640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f00c49e0 con 0x7f75f00b0b20 2023-10-16T10:05:32.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.532+0000 7f75f6ade640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f0159f00 0x7f75f014c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:32.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.533+0000 7f75f6ade640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f0159f00 0x7f75f014c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56320/0 (socket says 172.21.15.84:56320) 2023-10-16T10:05:32.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.533+0000 7f75f6ade640 1 -- 172.21.15.84:0/2340667805 learned_addr learned my addr 172.21.15.84:0/2340667805 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:32.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.533+0000 7f75f62dd640 1 --2- 172.21.15.84:0/2340667805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75f00b0b20 0x7f75f00c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:32.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.533+0000 7f75f6ade640 1 -- 172.21.15.84:0/2340667805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75f00b0b20 msgr2=0x7f75f00c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:32.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.533+0000 7f75f6ade640 1 --2- 172.21.15.84:0/2340667805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75f00b0b20 0x7f75f00c77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:32.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.533+0000 7f75f5adc640 1 --2- 172.21.15.84:0/2340667805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0159b20 0x7f75f00b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:32.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.533+0000 7f75f6ade640 1 -- 172.21.15.84:0/2340667805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0159b20 msgr2=0x7f75f00b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:32.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.533+0000 7f75f6ade640 1 --2- 172.21.15.84:0/2340667805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0159b20 0x7f75f00b17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:32.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.533+0000 7f75f6ade640 1 -- 172.21.15.84:0/2340667805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e8009a70 con 0x7f75f0159f00 2023-10-16T10:05:32.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.533+0000 7f75f6ade640 1 --2- 172.21.15.84:0/2340667805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f0159f00 0x7f75f014c1e0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009eb0 tx=0x7f75e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:32.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.533+0000 7f75e6ffd640 1 -- 172.21.15.84:0/2340667805 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e803f420 con 0x7f75f0159f00 2023-10-16T10:05:32.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.534+0000 7f75e6ffd640 1 -- 172.21.15.84:0/2340667805 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e803f580 con 0x7f75f0159f00 2023-10-16T10:05:32.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.534+0000 7f75f8541640 1 -- 172.21.15.84:0/2340667805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f00c4cc0 con 0x7f75f0159f00 2023-10-16T10:05:32.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.534+0000 7f75e6ffd640 1 -- 172.21.15.84:0/2340667805 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e803f8f0 con 0x7f75f0159f00 2023-10-16T10:05:32.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.534+0000 7f75f8541640 1 -- 172.21.15.84:0/2340667805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f00c5280 con 0x7f75f0159f00 2023-10-16T10:05:32.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.535+0000 7f75f8541640 1 -- 172.21.15.84:0/2340667805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f004e160 con 0x7f75f0159f00 2023-10-16T10:05:32.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.536+0000 7f75e6ffd640 1 -- 172.21.15.84:0/2340667805 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 47) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f75e803fa50 con 0x7f75f0159f00 2023-10-16T10:05:32.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.536+0000 7f75e6ffd640 1 --2- 172.21.15.84:0/2340667805 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f75d406acc0 0x7f75d406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:32.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.537+0000 7f75e6ffd640 1 -- 172.21.15.84:0/2340667805 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f75e80ac2d0 con 0x7f75f0159f00 2023-10-16T10:05:32.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.537+0000 7f75f62dd640 1 --2- 172.21.15.84:0/2340667805 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f75d406acc0 0x7f75d406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:32.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.537+0000 7f75f62dd640 1 --2- 172.21.15.84:0/2340667805 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f75d406acc0 0x7f75d406d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f75ec003d90 tx=0x7f75ec009040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:32.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.541+0000 7f75e6ffd640 1 -- 172.21.15.84:0/2340667805 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f75e8077340 con 0x7f75f0159f00 2023-10-16T10:05:32.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.770+0000 7f75f8541640 1 -- 172.21.15.84:0/2340667805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75f006ce50 con 0x7f75f0159f00 2023-10-16T10:05:32.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.771+0000 7f75e6ffd640 1 -- 172.21.15.84:0/2340667805 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f75e807a960 con 0x7f75f0159f00 2023-10-16T10:05:32.773 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:32.773 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/89534235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":89534235},{"type":"v1","addr":"172.21.15.157:6835","nonce":89534235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3807666966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3807666966},{"type":"v1","addr":"172.21.15.157:6834","nonce":3807666966}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3448417053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3448417053},{"type":"v1","addr":"172.21.15.84:6835","nonce":3448417053}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:05:10.829122+0000","modified":"2023-10-16T10:05:13.893618+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7044":{"gid":7044,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/1939610713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":1939610713},{"type":"v1","addr":"172.21.15.84:6837","nonce":1939610713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-16T10:05:32.774 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 44 2023-10-16T10:05:32.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.774+0000 7f75f8541640 1 -- 172.21.15.84:0/2340667805 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f75d406acc0 msgr2=0x7f75d406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:32.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.774+0000 7f75f8541640 1 --2- 172.21.15.84:0/2340667805 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f75d406acc0 0x7f75d406d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f75ec003d90 tx=0x7f75ec009040 comp rx=0 tx=0).stop 2023-10-16T10:05:32.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.774+0000 7f75f8541640 1 -- 172.21.15.84:0/2340667805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f0159f00 msgr2=0x7f75f014c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:32.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.774+0000 7f75f8541640 1 --2- 172.21.15.84:0/2340667805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f0159f00 0x7f75f014c1e0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009eb0 tx=0x7f75e8004a40 comp rx=0 tx=0).stop 2023-10-16T10:05:32.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.774+0000 7f75f8541640 1 -- 172.21.15.84:0/2340667805 shutdown_connections 2023-10-16T10:05:32.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.774+0000 7f75f8541640 1 --2- 172.21.15.84:0/2340667805 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f75d406acc0 0x7f75d406d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:32.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.774+0000 7f75f8541640 1 --2- 172.21.15.84:0/2340667805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f0159f00 0x7f75f014c1e0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:32.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.774+0000 7f75f8541640 1 --2- 172.21.15.84:0/2340667805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0159b20 0x7f75f00b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:32.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.774+0000 7f75f8541640 1 --2- 172.21.15.84:0/2340667805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75f00b0b20 0x7f75f00c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:32.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.774+0000 7f75f8541640 1 -- 172.21.15.84:0/2340667805 >> 172.21.15.84:0/2340667805 conn(0x7f75f00cb410 msgr2=0x7f75f00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:32.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.774+0000 7f75f8541640 1 -- 172.21.15.84:0/2340667805 shutdown_connections 2023-10-16T10:05:32.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:32.774+0000 7f75f8541640 1 -- 172.21.15.84:0/2340667805 wait complete. 2023-10-16T10:05:32.794 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:05:33.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.013+0000 7f800fb5e640 1 Processor -- start 2023-10-16T10:05:33.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.013+0000 7f800fb5e640 1 -- start start 2023-10-16T10:05:33.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.015+0000 7f800fb5e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f80080b0b20 0x7f8008164f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:33.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.015+0000 7f800fb5e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8008071450 con 0x7f80080adb50 2023-10-16T10:05:33.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.015+0000 7f800fb5e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80080cc600 con 0x7f8008070fd0 2023-10-16T10:05:33.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.015+0000 7f800fb5e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80080cc8e0 con 0x7f80080b0b20 2023-10-16T10:05:33.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.015+0000 7f800d8fa640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f80080adb50 0x7f800806f550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57856/0 (socket says 172.21.15.84:57856) 2023-10-16T10:05:33.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.015+0000 7f800d8fa640 1 -- 172.21.15.84:0/983285457 learned_addr learned my addr 172.21.15.84:0/983285457 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:33.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.015+0000 7f800e0fb640 1 --2- 172.21.15.84:0/983285457 >> v2:172.21.15.84:3300/0 conn(0x7f80080b0b20 0x7f8008164f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:33.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.015+0000 7f800c8f8640 1 -- 172.21.15.84:0/983285457 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070599844 0 0) 0x7f8008071450 con 0x7f80080adb50 2023-10-16T10:05:33.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.016+0000 7f800c8f8640 1 -- 172.21.15.84:0/983285457 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ff4003520 con 0x7f80080adb50 2023-10-16T10:05:33.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.016+0000 7f800e0fb640 1 -- 172.21.15.84:0/983285457 >> v1:172.21.15.157:6789/0 conn(0x7f8008070fd0 legacy=0x7f80080b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:33.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.016+0000 7f800e0fb640 1 -- 172.21.15.84:0/983285457 >> v1:172.21.15.84:6789/0 conn(0x7f80080adb50 legacy=0x7f800806f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:33.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.016+0000 7f800e0fb640 1 -- 172.21.15.84:0/983285457 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80080ccbc0 con 0x7f80080b0b20 2023-10-16T10:05:33.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.017+0000 7f800e0fb640 1 --2- 172.21.15.84:0/983285457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80080b0b20 0x7f8008164f60 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8000009d90 tx=0x7f800002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afed7f06e2aa0eac server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:33.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.017+0000 7f800c8f8640 1 -- 172.21.15.84:0/983285457 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800002cc60 con 0x7f80080b0b20 2023-10-16T10:05:33.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.017+0000 7f800c8f8640 1 -- 172.21.15.84:0/983285457 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800002cdc0 con 0x7f80080b0b20 2023-10-16T10:05:33.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.017+0000 7f800c8f8640 1 -- 172.21.15.84:0/983285457 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000034660 con 0x7f80080b0b20 2023-10-16T10:05:33.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.018+0000 7f800fb5e640 1 -- 172.21.15.84:0/983285457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80080b0b20 msgr2=0x7f8008164f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:33.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.018+0000 7f800fb5e640 1 --2- 172.21.15.84:0/983285457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80080b0b20 0x7f8008164f60 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8000009d90 tx=0x7f800002beb0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.018+0000 7f800fb5e640 1 -- 172.21.15.84:0/983285457 shutdown_connections 2023-10-16T10:05:33.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.018+0000 7f800fb5e640 1 --2- 172.21.15.84:0/983285457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80080b0b20 0x7f8008164f60 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.018+0000 7f800fb5e640 1 -- 172.21.15.84:0/983285457 >> 172.21.15.84:0/983285457 conn(0x7f80080cb410 msgr2=0x7f80080cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:33.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.018+0000 7f800fb5e640 1 -- 172.21.15.84:0/983285457 shutdown_connections 2023-10-16T10:05:33.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.018+0000 7f800fb5e640 1 -- 172.21.15.84:0/983285457 wait complete. 2023-10-16T10:05:33.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.019+0000 7f800fb5e640 1 Processor -- start 2023-10-16T10:05:33.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.020+0000 7f800fb5e640 1 -- start start 2023-10-16T10:05:33.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.020+0000 7f800fb5e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8008070fd0 0x7f80080c3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:33.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.021+0000 7f800fb5e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80080adb50 0x7f80081619f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:33.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.021+0000 7f800fb5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80080b0b20 0x7f800815e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:33.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.021+0000 7f800fb5e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8008162b90 con 0x7f80080adb50 2023-10-16T10:05:33.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.021+0000 7f800fb5e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8008161f30 con 0x7f80080b0b20 2023-10-16T10:05:33.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.021+0000 7f800fb5e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80081621d0 con 0x7f8008070fd0 2023-10-16T10:05:33.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.021+0000 7f800d0f9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80080adb50 0x7f80081619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:33.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.021+0000 7f800d0f9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80080adb50 0x7f80081619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56338/0 (socket says 172.21.15.84:56338) 2023-10-16T10:05:33.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.021+0000 7f800d0f9640 1 -- 172.21.15.84:0/3972020387 learned_addr learned my addr 172.21.15.84:0/3972020387 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:33.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.021+0000 7f800d8fa640 1 --2- 172.21.15.84:0/3972020387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8008070fd0 0x7f80080c3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:33.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.021+0000 7f800e0fb640 1 --2- 172.21.15.84:0/3972020387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80080b0b20 0x7f800815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:33.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.021+0000 7f800d0f9640 1 -- 172.21.15.84:0/3972020387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8008070fd0 msgr2=0x7f80080c3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:33.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.021+0000 7f800d0f9640 1 --2- 172.21.15.84:0/3972020387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8008070fd0 0x7f80080c3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.021+0000 7f800d0f9640 1 -- 172.21.15.84:0/3972020387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80080b0b20 msgr2=0x7f800815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:33.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.021+0000 7f800d0f9640 1 --2- 172.21.15.84:0/3972020387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80080b0b20 0x7f800815e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.021+0000 7f800d0f9640 1 -- 172.21.15.84:0/3972020387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8000009a70 con 0x7f80080adb50 2023-10-16T10:05:33.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.022+0000 7f800d0f9640 1 --2- 172.21.15.84:0/3972020387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80080adb50 0x7f80081619f0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00b3f0 tx=0x7f7ffc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:33.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.022+0000 7f7ffa7fc640 1 -- 172.21.15.84:0/3972020387 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc011830 con 0x7f80080adb50 2023-10-16T10:05:33.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.022+0000 7f7ffa7fc640 1 -- 172.21.15.84:0/3972020387 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ffc011990 con 0x7f80080adb50 2023-10-16T10:05:33.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.022+0000 7f7ffa7fc640 1 -- 172.21.15.84:0/3972020387 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc011d40 con 0x7f80080adb50 2023-10-16T10:05:33.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.022+0000 7f800fb5e640 1 -- 172.21.15.84:0/3972020387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80081624b0 con 0x7f80080adb50 2023-10-16T10:05:33.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.022+0000 7f800fb5e640 1 -- 172.21.15.84:0/3972020387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8008162da0 con 0x7f80080adb50 2023-10-16T10:05:33.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.023+0000 7f800fb5e640 1 -- 172.21.15.84:0/3972020387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8008162fc0 con 0x7f80080adb50 2023-10-16T10:05:33.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.024+0000 7f7ffa7fc640 1 -- 172.21.15.84:0/3972020387 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 47) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f7ffc01a400 con 0x7f80080adb50 2023-10-16T10:05:33.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.025+0000 7f7ffa7fc640 1 --2- 172.21.15.84:0/3972020387 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f7fec06acc0 0x7f7fec06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:33.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.025+0000 7f7ffa7fc640 1 -- 172.21.15.84:0/3972020387 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f7ffc0893d0 con 0x7f80080adb50 2023-10-16T10:05:33.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.025+0000 7f800d8fa640 1 --2- 172.21.15.84:0/3972020387 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f7fec06acc0 0x7f7fec06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:33.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.026+0000 7f800d8fa640 1 --2- 172.21.15.84:0/3972020387 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f7fec06acc0 0x7f7fec06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f80040011e0 tx=0x7f8004009040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:33.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.030+0000 7f7ffa7fc640 1 -- 172.21.15.84:0/3972020387 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7ffc054440 con 0x7f80080adb50 2023-10-16T10:05:33.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.227+0000 7f800fb5e640 1 -- 172.21.15.84:0/3972020387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f800804e160 con 0x7f80080adb50 2023-10-16T10:05:33.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.231+0000 7f7ffa7fc640 1 -- 172.21.15.84:0/3972020387 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+17540 (secure 0 0 0) 0x7f7ffc057a60 con 0x7f80080adb50 2023-10-16T10:05:33.233 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:33.233 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":97,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:05:31.449789+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:05:08.636077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-16T10:05:09.841985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2152330716":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/3965108955":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/1978512312":"2023-10-17T10:05:31.449737+0000","172.21.15.84:6833/494168607":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/2341097922":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/4027382473":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/3685429931":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/3830790285":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/821251662":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/504267391":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/3778743167":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/150347074":"2023-10-17T10:05:25.079928+0000","172.21.15.84:6832/494168607":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/3151132884":"2023-10-17T10:05:25.079928+0000","172.21.15.84:6833/147697688":"2023-10-17T10:05:25.079928+0000","172.21.15.84:6832/147697688":"2023-10-17T10:05:25.079928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:05:33.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.233+0000 7f800fb5e640 1 -- 172.21.15.84:0/3972020387 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f7fec06acc0 msgr2=0x7f7fec06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:33.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.233+0000 7f800fb5e640 1 --2- 172.21.15.84:0/3972020387 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f7fec06acc0 0x7f7fec06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f80040011e0 tx=0x7f8004009040 comp rx=0 tx=0).stop 2023-10-16T10:05:33.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.233+0000 7f800fb5e640 1 -- 172.21.15.84:0/3972020387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80080adb50 msgr2=0x7f80081619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:33.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.233+0000 7f800fb5e640 1 --2- 172.21.15.84:0/3972020387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80080adb50 0x7f80081619f0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00b3f0 tx=0x7f7ffc00b8b0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.233+0000 7f800fb5e640 1 -- 172.21.15.84:0/3972020387 shutdown_connections 2023-10-16T10:05:33.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.233+0000 7f800fb5e640 1 --2- 172.21.15.84:0/3972020387 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f7fec06acc0 0x7f7fec06d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.233+0000 7f800fb5e640 1 --2- 172.21.15.84:0/3972020387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80080b0b20 0x7f800815e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.233+0000 7f800fb5e640 1 --2- 172.21.15.84:0/3972020387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80080adb50 0x7f80081619f0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.233+0000 7f800fb5e640 1 --2- 172.21.15.84:0/3972020387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8008070fd0 0x7f80080c3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.233+0000 7f800fb5e640 1 -- 172.21.15.84:0/3972020387 >> 172.21.15.84:0/3972020387 conn(0x7f80080cb410 msgr2=0x7f80080ae630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:33.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.234+0000 7f800fb5e640 1 -- 172.21.15.84:0/3972020387 shutdown_connections 2023-10-16T10:05:33.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.234+0000 7f800fb5e640 1 -- 172.21.15.84:0/3972020387 wait complete. 2023-10-16T10:05:33.254 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T10:05:33.255 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:05:33.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.462+0000 7f0567fff640 1 Processor -- start 2023-10-16T10:05:33.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.463+0000 7f0567fff640 1 -- start start 2023-10-16T10:05:33.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.464+0000 7f0567fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f056806c040 0x7f056806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:33.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.464+0000 7f0567fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05680ad970 0x7f056815e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:33.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.464+0000 7f0567fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056806c960 con 0x7f056815ebc0 2023-10-16T10:05:33.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.464+0000 7f0567fff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f056815b9d0 con 0x7f056806c040 2023-10-16T10:05:33.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.464+0000 7f0567fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056815bcb0 con 0x7f05680ad970 2023-10-16T10:05:33.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.464+0000 7f05677fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f056815ebc0 0x7f05680cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57862/0 (socket says 172.21.15.84:57862) 2023-10-16T10:05:33.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.464+0000 7f05677fe640 1 -- 172.21.15.84:0/2063694206 learned_addr learned my addr 172.21.15.84:0/2063694206 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:33.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.464+0000 7f05667fc640 1 --2- 172.21.15.84:0/2063694206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05680ad970 0x7f056815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:33.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.464+0000 7f0566ffd640 1 --2- 172.21.15.84:0/2063694206 >> v2:172.21.15.157:3300/0 conn(0x7f056806c040 0x7f056806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:33.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.465+0000 7f05667fc640 1 -- 172.21.15.84:0/2063694206 >> v2:172.21.15.157:3300/0 conn(0x7f056806c040 msgr2=0x7f056806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:33.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.465+0000 7f05667fc640 1 --2- 172.21.15.84:0/2063694206 >> v2:172.21.15.157:3300/0 conn(0x7f056806c040 0x7f056806c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.465+0000 7f05667fc640 1 -- 172.21.15.84:0/2063694206 >> v1:172.21.15.84:6789/0 conn(0x7f056815ebc0 legacy=0x7f05680cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:33.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.465+0000 7f05667fc640 1 -- 172.21.15.84:0/2063694206 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056815bf90 con 0x7f05680ad970 2023-10-16T10:05:33.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.467+0000 7f05667fc640 1 --2- 172.21.15.84:0/2063694206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05680ad970 0x7f056815e680 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f055c0097d0 tx=0x7f055c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a830fc06af22e717 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:33.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.470+0000 7f0565ffb640 1 -- 172.21.15.84:0/2063694206 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c02d040 con 0x7f05680ad970 2023-10-16T10:05:33.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.470+0000 7f0565ffb640 1 -- 172.21.15.84:0/2063694206 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c031810 con 0x7f05680ad970 2023-10-16T10:05:33.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.470+0000 7f0565ffb640 1 -- 172.21.15.84:0/2063694206 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c031ba0 con 0x7f05680ad970 2023-10-16T10:05:33.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.470+0000 7f0567fff640 1 -- 172.21.15.84:0/2063694206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05680ad970 msgr2=0x7f056815e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:33.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.470+0000 7f0567fff640 1 --2- 172.21.15.84:0/2063694206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05680ad970 0x7f056815e680 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f055c0097d0 tx=0x7f055c02b8b0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.471+0000 7f0567fff640 1 -- 172.21.15.84:0/2063694206 shutdown_connections 2023-10-16T10:05:33.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.471+0000 7f0567fff640 1 --2- 172.21.15.84:0/2063694206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05680ad970 0x7f056815e680 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.471+0000 7f0567fff640 1 --2- 172.21.15.84:0/2063694206 >> v2:172.21.15.157:3300/0 conn(0x7f056806c040 0x7f056806c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.471+0000 7f0567fff640 1 -- 172.21.15.84:0/2063694206 >> 172.21.15.84:0/2063694206 conn(0x7f05680ce6a0 msgr2=0x7f05680ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:33.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.471+0000 7f0567fff640 1 -- 172.21.15.84:0/2063694206 shutdown_connections 2023-10-16T10:05:33.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.471+0000 7f0567fff640 1 -- 172.21.15.84:0/2063694206 wait complete. 2023-10-16T10:05:33.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.472+0000 7f0567fff640 1 Processor -- start 2023-10-16T10:05:33.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.472+0000 7f0567fff640 1 -- start start 2023-10-16T10:05:33.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.473+0000 7f0567fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f056806c040 0x7f05680b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:33.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.473+0000 7f0567fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05680ad970 0x7f056815d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:33.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.473+0000 7f0567fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f056815ebc0 0x7f056815d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:33.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.473+0000 7f0567fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0568149830 con 0x7f05680ad970 2023-10-16T10:05:33.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.473+0000 7f0567fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05680ae0c0 con 0x7f056815ebc0 2023-10-16T10:05:33.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.473+0000 7f0567fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05680ae960 con 0x7f056806c040 2023-10-16T10:05:33.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.474+0000 7f05667fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05680ad970 0x7f056815d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:33.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.474+0000 7f05667fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05680ad970 0x7f056815d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56344/0 (socket says 172.21.15.84:56344) 2023-10-16T10:05:33.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.474+0000 7f05667fc640 1 -- 172.21.15.84:0/2434560223 learned_addr learned my addr 172.21.15.84:0/2434560223 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:33.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.474+0000 7f05667fc640 1 -- 172.21.15.84:0/2434560223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f056806c040 msgr2=0x7f05680b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:33.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.474+0000 7f05667fc640 1 --2- 172.21.15.84:0/2434560223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f056806c040 0x7f05680b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.474+0000 7f05677fe640 1 --2- 172.21.15.84:0/2434560223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f056815ebc0 0x7f056815d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:33.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.474+0000 7f05667fc640 1 -- 172.21.15.84:0/2434560223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f056815ebc0 msgr2=0x7f056815d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:33.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.474+0000 7f05667fc640 1 --2- 172.21.15.84:0/2434560223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f056815ebc0 0x7f056815d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.474+0000 7f05667fc640 1 -- 172.21.15.84:0/2434560223 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c009400 con 0x7f05680ad970 2023-10-16T10:05:33.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.474+0000 7f05667fc640 1 --2- 172.21.15.84:0/2434560223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05680ad970 0x7f056815d350 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f055c0098f0 tx=0x7f055c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:33.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.475+0000 7f054f7fe640 1 -- 172.21.15.84:0/2434560223 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c03ad60 con 0x7f05680ad970 2023-10-16T10:05:33.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.475+0000 7f054f7fe640 1 -- 172.21.15.84:0/2434560223 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c0423f0 con 0x7f05680ad970 2023-10-16T10:05:33.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.475+0000 7f054f7fe640 1 -- 172.21.15.84:0/2434560223 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c0427a0 con 0x7f05680ad970 2023-10-16T10:05:33.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.475+0000 7f0567fff640 1 -- 172.21.15.84:0/2434560223 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05680c76b0 con 0x7f05680ad970 2023-10-16T10:05:33.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.475+0000 7f0567fff640 1 -- 172.21.15.84:0/2434560223 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05680ae5f0 con 0x7f05680ad970 2023-10-16T10:05:33.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.476+0000 7f0567fff640 1 -- 172.21.15.84:0/2434560223 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056806b330 con 0x7f05680ad970 2023-10-16T10:05:33.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.477+0000 7f054f7fe640 1 -- 172.21.15.84:0/2434560223 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 47) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f055c04b3b0 con 0x7f05680ad970 2023-10-16T10:05:33.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.478+0000 7f054f7fe640 1 --2- 172.21.15.84:0/2434560223 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f055406acc0 0x7f055406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:33.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.478+0000 7f054f7fe640 1 -- 172.21.15.84:0/2434560223 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f055c0ab630 con 0x7f05680ad970 2023-10-16T10:05:33.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.478+0000 7f0566ffd640 1 --2- 172.21.15.84:0/2434560223 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f055406acc0 0x7f055406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:33.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.478+0000 7f0566ffd640 1 --2- 172.21.15.84:0/2434560223 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f055406acc0 0x7f055406d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0558006a90 tx=0x7f0558006a20 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:33.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.482+0000 7f054f7fe640 1 -- 172.21.15.84:0/2434560223 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f055c0766a0 con 0x7f05680ad970 2023-10-16T10:05:33.501 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:33.499+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:33.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.710+0000 7f0567fff640 1 -- 172.21.15.84:0/2434560223 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f056804e9b0 con 0x7f05680ad970 2023-10-16T10:05:33.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.710+0000 7f054f7fe640 1 -- 172.21.15.84:0/2434560223 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f055c079cc0 con 0x7f05680ad970 2023-10-16T10:05:33.712 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:33.712 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/89534235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":89534235},{"type":"v1","addr":"172.21.15.157:6835","nonce":89534235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3807666966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3807666966},{"type":"v1","addr":"172.21.15.157:6834","nonce":3807666966}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3448417053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3448417053},{"type":"v1","addr":"172.21.15.84:6835","nonce":3448417053}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:05:10.829122+0000","modified":"2023-10-16T10:05:13.893618+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7044":{"gid":7044,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/1939610713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":1939610713},{"type":"v1","addr":"172.21.15.84:6837","nonce":1939610713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-16T10:05:33.713 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 44 2023-10-16T10:05:33.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.713+0000 7f0567fff640 1 -- 172.21.15.84:0/2434560223 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f055406acc0 msgr2=0x7f055406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:33.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.713+0000 7f0567fff640 1 --2- 172.21.15.84:0/2434560223 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f055406acc0 0x7f055406d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0558006a90 tx=0x7f0558006a20 comp rx=0 tx=0).stop 2023-10-16T10:05:33.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.713+0000 7f0567fff640 1 -- 172.21.15.84:0/2434560223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05680ad970 msgr2=0x7f056815d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:33.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.713+0000 7f0567fff640 1 --2- 172.21.15.84:0/2434560223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05680ad970 0x7f056815d350 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f055c0098f0 tx=0x7f055c0046c0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.713+0000 7f0567fff640 1 -- 172.21.15.84:0/2434560223 shutdown_connections 2023-10-16T10:05:33.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.713+0000 7f0567fff640 1 --2- 172.21.15.84:0/2434560223 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f055406acc0 0x7f055406d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.713+0000 7f0567fff640 1 --2- 172.21.15.84:0/2434560223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f056815ebc0 0x7f056815d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.713+0000 7f0567fff640 1 --2- 172.21.15.84:0/2434560223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05680ad970 0x7f056815d350 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.713+0000 7f0567fff640 1 --2- 172.21.15.84:0/2434560223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f056806c040 0x7f05680b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.713+0000 7f0567fff640 1 -- 172.21.15.84:0/2434560223 >> 172.21.15.84:0/2434560223 conn(0x7f05680ce6a0 msgr2=0x7f05680ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:33.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.713+0000 7f0567fff640 1 -- 172.21.15.84:0/2434560223 shutdown_connections 2023-10-16T10:05:33.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.713+0000 7f0567fff640 1 -- 172.21.15.84:0/2434560223 wait complete. 2023-10-16T10:05:33.731 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:05:33.909 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:05:33.918 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:05:33.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.942+0000 7fb0bbfff640 1 Processor -- start 2023-10-16T10:05:33.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.942+0000 7fb0bbfff640 1 -- start start 2023-10-16T10:05:33.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.944+0000 7fb0bbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb0bc15cbe0 0x7fb0bc0d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:33.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.944+0000 7fb0bbfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0bc15cfc0 0x7fb0bc0b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:33.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.944+0000 7fb0bbfff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0bc0d15a0 con 0x7fb0bc0d1090 2023-10-16T10:05:33.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.944+0000 7fb0bbfff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0bc0d1880 con 0x7fb0bc15cbe0 2023-10-16T10:05:33.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.944+0000 7fb0bbfff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc0d1b60 con 0x7fb0bc15cfc0 2023-10-16T10:05:33.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.944+0000 7fb0baffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb0bc15cbe0 0x7fb0bc0d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:33.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.944+0000 7fb0baffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb0bc15cbe0 0x7fb0bc0d0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56352/0 (socket says 172.21.15.84:56352) 2023-10-16T10:05:33.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.944+0000 7fb0baffd640 1 -- 172.21.15.84:0/195049148 learned_addr learned my addr 172.21.15.84:0/195049148 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:33.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.944+0000 7fb0bb7fe640 1 --1- 172.21.15.84:0/195049148 >> v1:172.21.15.84:6789/0 conn(0x7fb0bc0d1090 0x7fb0bc06bbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57876/0 (socket says 172.21.15.84:57876) 2023-10-16T10:05:33.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.944+0000 7fb0ba7fc640 1 --2- 172.21.15.84:0/195049148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0bc15cfc0 0x7fb0bc0b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:33.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.945+0000 7fb0baffd640 1 -- 172.21.15.84:0/195049148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0bc15cfc0 msgr2=0x7fb0bc0b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:33.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.945+0000 7fb0baffd640 1 --2- 172.21.15.84:0/195049148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0bc15cfc0 0x7fb0bc0b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.945+0000 7fb0baffd640 1 -- 172.21.15.84:0/195049148 >> v1:172.21.15.84:6789/0 conn(0x7fb0bc0d1090 legacy=0x7fb0bc06bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:33.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.945+0000 7fb0b9ffb640 1 -- 172.21.15.84:0/195049148 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3715826252 0 0) 0x7fb0bc0d15a0 con 0x7fb0bc0d1090 2023-10-16T10:05:33.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.945+0000 7fb0baffd640 1 -- 172.21.15.84:0/195049148 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc0d1e40 con 0x7fb0bc15cbe0 2023-10-16T10:05:33.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.946+0000 7fb0baffd640 1 --2- 172.21.15.84:0/195049148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0bc15cbe0 0x7fb0bc0d0b50 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009870 tx=0x7fb0ac02ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551ae4e35ce7f228 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:33.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.946+0000 7fb0b9ffb640 1 -- 172.21.15.84:0/195049148 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac02f830 con 0x7fb0bc15cbe0 2023-10-16T10:05:33.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.946+0000 7fb0b9ffb640 1 -- 172.21.15.84:0/195049148 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ac02f990 con 0x7fb0bc15cbe0 2023-10-16T10:05:33.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.946+0000 7fb0b9ffb640 1 -- 172.21.15.84:0/195049148 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac02fd20 con 0x7fb0bc15cbe0 2023-10-16T10:05:33.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.947+0000 7fb0bbfff640 1 -- 172.21.15.84:0/195049148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0bc15cbe0 msgr2=0x7fb0bc0d0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:33.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.947+0000 7fb0bbfff640 1 --2- 172.21.15.84:0/195049148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0bc15cbe0 0x7fb0bc0d0b50 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009870 tx=0x7fb0ac02ba10 comp rx=0 tx=0).stop 2023-10-16T10:05:33.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.947+0000 7fb0bbfff640 1 -- 172.21.15.84:0/195049148 shutdown_connections 2023-10-16T10:05:33.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.947+0000 7fb0bbfff640 1 --2- 172.21.15.84:0/195049148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0bc15cfc0 0x7fb0bc0b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.947+0000 7fb0bbfff640 1 --2- 172.21.15.84:0/195049148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0bc15cbe0 0x7fb0bc0d0b50 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.947+0000 7fb0bbfff640 1 -- 172.21.15.84:0/195049148 >> 172.21.15.84:0/195049148 conn(0x7fb0bc0cf570 msgr2=0x7fb0bc0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:33.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.947+0000 7fb0bbfff640 1 -- 172.21.15.84:0/195049148 shutdown_connections 2023-10-16T10:05:33.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.947+0000 7fb0bbfff640 1 -- 172.21.15.84:0/195049148 wait complete. 2023-10-16T10:05:33.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.949+0000 7fb0bbfff640 1 Processor -- start 2023-10-16T10:05:33.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.949+0000 7fb0bbfff640 1 -- start start 2023-10-16T10:05:33.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.950+0000 7fb0bbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0bc0d1090 0x7fb0bc155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:33.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.950+0000 7fb0bbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0bc15cbe0 0x7fb0bc153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:33.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.950+0000 7fb0bbfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0bc15cfc0 0x7fb0bc0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:33.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.950+0000 7fb0bbfff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc153880 con 0x7fb0bc15cbe0 2023-10-16T10:05:33.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.950+0000 7fb0bbfff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc0c9480 con 0x7fb0bc0d1090 2023-10-16T10:05:33.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.950+0000 7fb0bbfff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc153a90 con 0x7fb0bc15cfc0 2023-10-16T10:05:33.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.950+0000 7fb0ba7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0bc15cbe0 0x7fb0bc153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:33.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.950+0000 7fb0ba7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0bc15cbe0 0x7fb0bc153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56366/0 (socket says 172.21.15.84:56366) 2023-10-16T10:05:33.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.950+0000 7fb0ba7fc640 1 -- 172.21.15.84:0/1733440381 learned_addr learned my addr 172.21.15.84:0/1733440381 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:33.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.951+0000 7fb0baffd640 1 --2- 172.21.15.84:0/1733440381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0bc0d1090 0x7fb0bc155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:33.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.951+0000 7fb0bb7fe640 1 --2- 172.21.15.84:0/1733440381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0bc15cfc0 0x7fb0bc0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:33.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.951+0000 7fb0ba7fc640 1 -- 172.21.15.84:0/1733440381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0bc15cfc0 msgr2=0x7fb0bc0c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:33.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.951+0000 7fb0ba7fc640 1 --2- 172.21.15.84:0/1733440381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0bc15cfc0 0x7fb0bc0c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.951+0000 7fb0ba7fc640 1 -- 172.21.15.84:0/1733440381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0bc0d1090 msgr2=0x7fb0bc155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:33.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.951+0000 7fb0ba7fc640 1 --2- 172.21.15.84:0/1733440381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0bc0d1090 0x7fb0bc155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:33.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.951+0000 7fb0ba7fc640 1 -- 172.21.15.84:0/1733440381 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac0094d0 con 0x7fb0bc15cbe0 2023-10-16T10:05:33.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.951+0000 7fb0bb7fe640 1 --2- 172.21.15.84:0/1733440381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0bc15cfc0 0x7fb0bc0c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:05:33.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.951+0000 7fb0ba7fc640 1 --2- 172.21.15.84:0/1733440381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0bc15cbe0 0x7fb0bc153340 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000d5f0 tx=0x7fb0b000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:33.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.951+0000 7fb0a37fe640 1 -- 172.21.15.84:0/1733440381 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b00125f0 con 0x7fb0bc15cbe0 2023-10-16T10:05:33.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.952+0000 7fb0a37fe640 1 -- 172.21.15.84:0/1733440381 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b0012750 con 0x7fb0bc15cbe0 2023-10-16T10:05:33.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.952+0000 7fb0a37fe640 1 -- 172.21.15.84:0/1733440381 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0012b00 con 0x7fb0bc15cbe0 2023-10-16T10:05:33.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.952+0000 7fb0bbfff640 1 -- 172.21.15.84:0/1733440381 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0bc15ec10 con 0x7fb0bc15cbe0 2023-10-16T10:05:33.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.952+0000 7fb0bbfff640 1 -- 172.21.15.84:0/1733440381 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0bc15f1d0 con 0x7fb0bc15cbe0 2023-10-16T10:05:33.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.953+0000 7fb0bbfff640 1 -- 172.21.15.84:0/1733440381 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0bc04e9b0 con 0x7fb0bc15cbe0 2023-10-16T10:05:33.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.954+0000 7fb0a37fe640 1 -- 172.21.15.84:0/1733440381 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 47) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fb0b0012c60 con 0x7fb0bc15cbe0 2023-10-16T10:05:33.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.955+0000 7fb0a37fe640 1 --2- 172.21.15.84:0/1733440381 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fb0a806acf0 0x7fb0a806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:33.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.955+0000 7fb0a37fe640 1 -- 172.21.15.84:0/1733440381 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6397+0+0 (secure 0 0 0) 0x7fb0b0089930 con 0x7fb0bc15cbe0 2023-10-16T10:05:33.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.955+0000 7fb0baffd640 1 --2- 172.21.15.84:0/1733440381 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fb0a806acf0 0x7fb0a806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:33.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.956+0000 7fb0baffd640 1 --2- 172.21.15.84:0/1733440381 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fb0a806acf0 0x7fb0a806d1b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac036000 tx=0x7fb0ac008030 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:33.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:33.959+0000 7fb0a37fe640 1 -- 172.21.15.84:0/1733440381 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb0b00549e0 con 0x7fb0bc15cbe0 2023-10-16T10:05:34.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.201+0000 7fb0a37fe640 1 -- 172.21.15.84:0/1733440381 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fb0b0054db0 con 0x7fb0bc15cbe0 2023-10-16T10:05:34.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.212+0000 7fb0bbfff640 1 -- 172.21.15.84:0/1733440381 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0bc0b4d80 con 0x7fb0bc15cbe0 2023-10-16T10:05:34.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.214+0000 7fb0a37fe640 1 -- 172.21.15.84:0/1733440381 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fb0b003e020 con 0x7fb0bc15cbe0 2023-10-16T10:05:34.215 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:34.216 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/89534235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":89534235},{"type":"v1","addr":"172.21.15.157:6835","nonce":89534235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3807666966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3807666966},{"type":"v1","addr":"172.21.15.157:6834","nonce":3807666966}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3448417053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3448417053},{"type":"v1","addr":"172.21.15.84:6835","nonce":3448417053}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:05:10.829122+0000","modified":"2023-10-16T10:05:13.893618+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7044":{"gid":7044,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/1939610713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":1939610713},{"type":"v1","addr":"172.21.15.84:6837","nonce":1939610713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-16T10:05:34.216 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 44 2023-10-16T10:05:34.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.216+0000 7fb0bbfff640 1 -- 172.21.15.84:0/1733440381 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fb0a806acf0 msgr2=0x7fb0a806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:34.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.216+0000 7fb0bbfff640 1 --2- 172.21.15.84:0/1733440381 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fb0a806acf0 0x7fb0a806d1b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac036000 tx=0x7fb0ac008030 comp rx=0 tx=0).stop 2023-10-16T10:05:34.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.216+0000 7fb0bbfff640 1 -- 172.21.15.84:0/1733440381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0bc15cbe0 msgr2=0x7fb0bc153340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:34.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.216+0000 7fb0bbfff640 1 --2- 172.21.15.84:0/1733440381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0bc15cbe0 0x7fb0bc153340 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000d5f0 tx=0x7fb0b000dab0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.216+0000 7fb0bbfff640 1 -- 172.21.15.84:0/1733440381 shutdown_connections 2023-10-16T10:05:34.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.216+0000 7fb0bbfff640 1 --2- 172.21.15.84:0/1733440381 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fb0a806acf0 0x7fb0a806d1b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.216+0000 7fb0bbfff640 1 --2- 172.21.15.84:0/1733440381 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0bc15cfc0 0x7fb0bc0c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.216+0000 7fb0bbfff640 1 --2- 172.21.15.84:0/1733440381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0bc15cbe0 0x7fb0bc153340 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.216+0000 7fb0bbfff640 1 --2- 172.21.15.84:0/1733440381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0bc0d1090 0x7fb0bc155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.216+0000 7fb0bbfff640 1 -- 172.21.15.84:0/1733440381 >> 172.21.15.84:0/1733440381 conn(0x7fb0bc0cf570 msgr2=0x7fb0bc15b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:34.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.217+0000 7fb0bbfff640 1 -- 172.21.15.84:0/1733440381 shutdown_connections 2023-10-16T10:05:34.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.217+0000 7fb0bbfff640 1 -- 172.21.15.84:0/1733440381 wait complete. 2023-10-16T10:05:34.234 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:05:34.376 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:34.373+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:34.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.441+0000 7fde6be56640 1 Processor -- start 2023-10-16T10:05:34.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.441+0000 7fde6be56640 1 -- start start 2023-10-16T10:05:34.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.443+0000 7fde6be56640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde6415aaa0 0x7fde6415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:34.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.443+0000 7fde6be56640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde6415ed60 con 0x7fde640ce390 2023-10-16T10:05:34.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.443+0000 7fde6be56640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde640707e0 con 0x7fde6415ae80 2023-10-16T10:05:34.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.443+0000 7fde6be56640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6406b810 con 0x7fde6415aaa0 2023-10-16T10:05:34.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.443+0000 7fde6a3f3640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fde640ce390 0x7fde64070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57880/0 (socket says 172.21.15.84:57880) 2023-10-16T10:05:34.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.443+0000 7fde6a3f3640 1 -- 172.21.15.84:0/1940973486 learned_addr learned my addr 172.21.15.84:0/1940973486 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:34.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.444+0000 7fde69bf2640 1 --2- 172.21.15.84:0/1940973486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde6415aaa0 0x7fde6415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:34.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.444+0000 7fde68bf0640 1 -- 172.21.15.84:0/1940973486 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493454867 0 0) 0x7fde6415ed60 con 0x7fde640ce390 2023-10-16T10:05:34.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.444+0000 7fde68bf0640 1 -- 172.21.15.84:0/1940973486 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde50003520 con 0x7fde640ce390 2023-10-16T10:05:34.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.444+0000 7fde68bf0640 1 -- 172.21.15.84:0/1940973486 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1789337061 0 0) 0x7fde50003520 con 0x7fde640ce390 2023-10-16T10:05:34.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.444+0000 7fde68bf0640 1 -- 172.21.15.84:0/1940973486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde6415aaa0 msgr2=0x7fde6415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:34.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.444+0000 7fde68bf0640 1 --2- 172.21.15.84:0/1940973486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde6415aaa0 0x7fde6415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.444+0000 7fde68bf0640 1 -- 172.21.15.84:0/1940973486 >> v1:172.21.15.157:6789/0 conn(0x7fde6415ae80 legacy=0x7fde640cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:34.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.444+0000 7fde68bf0640 1 -- 172.21.15.84:0/1940973486 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6415ef90 con 0x7fde640ce390 2023-10-16T10:05:34.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.444+0000 7fde68bf0640 1 -- 172.21.15.84:0/1940973486 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fde5c0019f0 con 0x7fde640ce390 2023-10-16T10:05:34.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.444+0000 7fde68bf0640 1 -- 172.21.15.84:0/1940973486 >> v1:172.21.15.84:6789/0 conn(0x7fde640ce390 legacy=0x7fde64070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:34.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.444+0000 7fde68bf0640 1 --2- 172.21.15.84:0/1940973486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde500039a0 0x7fde50003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:34.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.444+0000 7fde68bf0640 1 --2- 172.21.15.84:0/1940973486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde500047c0 0x7fde50004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:34.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.444+0000 7fde68bf0640 1 --2- 172.21.15.84:0/1940973486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde50005180 0x7fde5000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:34.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.444+0000 7fde68bf0640 1 -- 172.21.15.84:0/1940973486 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6406b810 con 0x7fde50005180 2023-10-16T10:05:34.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.444+0000 7fde68bf0640 1 -- 172.21.15.84:0/1940973486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5000c080 con 0x7fde500047c0 2023-10-16T10:05:34.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.444+0000 7fde68bf0640 1 -- 172.21.15.84:0/1940973486 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5000c240 con 0x7fde500039a0 2023-10-16T10:05:34.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.444+0000 7fde69bf2640 1 --2- 172.21.15.84:0/1940973486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde6415aaa0 0x7fde6415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:05:34.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.445+0000 7fde6a3f3640 1 --2- 172.21.15.84:0/1940973486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde50005180 0x7fde5000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:34.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.445+0000 7fde6a3f3640 1 -- 172.21.15.84:0/1940973486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde500039a0 msgr2=0x7fde50003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:34.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.445+0000 7fde6a3f3640 1 --2- 172.21.15.84:0/1940973486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde500039a0 0x7fde50003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.445+0000 7fde6a3f3640 1 -- 172.21.15.84:0/1940973486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde500047c0 msgr2=0x7fde50004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:34.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.445+0000 7fde6a3f3640 1 --2- 172.21.15.84:0/1940973486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde500047c0 0x7fde50004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.445+0000 7fde6a3f3640 1 -- 172.21.15.84:0/1940973486 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde50003520 con 0x7fde50005180 2023-10-16T10:05:34.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.446+0000 7fde6a3f3640 1 --2- 172.21.15.84:0/1940973486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde50005180 0x7fde5000b980 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00a760 tx=0x7fde500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0007a8a7f02bc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:34.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.446+0000 7fde68bf0640 1 -- 172.21.15.84:0/1940973486 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c034630 con 0x7fde50005180 2023-10-16T10:05:34.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.446+0000 7fde68bf0640 1 -- 172.21.15.84:0/1940973486 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c02f070 con 0x7fde50005180 2023-10-16T10:05:34.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.446+0000 7fde68bf0640 1 -- 172.21.15.84:0/1940973486 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c034a50 con 0x7fde50005180 2023-10-16T10:05:34.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.447+0000 7fde6be56640 1 -- 172.21.15.84:0/1940973486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde50005180 msgr2=0x7fde5000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:34.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.447+0000 7fde6be56640 1 --2- 172.21.15.84:0/1940973486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde50005180 0x7fde5000b980 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00a760 tx=0x7fde500057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.447+0000 7fde6a3f3640 1 -- 172.21.15.84:0/1940973486 reap_dead start 2023-10-16T10:05:34.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.447+0000 7fde6be56640 1 -- 172.21.15.84:0/1940973486 shutdown_connections 2023-10-16T10:05:34.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.447+0000 7fde6be56640 1 -- 172.21.15.84:0/1940973486 >> 172.21.15.84:0/1940973486 conn(0x7fde640cb410 msgr2=0x7fde640cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:34.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.447+0000 7fde6be56640 1 -- 172.21.15.84:0/1940973486 shutdown_connections 2023-10-16T10:05:34.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.447+0000 7fde6be56640 1 -- 172.21.15.84:0/1940973486 wait complete. 2023-10-16T10:05:34.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.449+0000 7fde6be56640 1 Processor -- start 2023-10-16T10:05:34.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.449+0000 7fde6be56640 1 -- start start 2023-10-16T10:05:34.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.450+0000 7fde6be56640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde50005180 0x7fde64151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:34.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.450+0000 7fde6be56640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde6415db00 0x7fde640c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:34.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.450+0000 7fde6be56640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde640c4c90 0x7fde6415bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:34.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.450+0000 7fde6be56640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6415d410 con 0x7fde6415db00 2023-10-16T10:05:34.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.450+0000 7fde6be56640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6415ce50 con 0x7fde50005180 2023-10-16T10:05:34.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.450+0000 7fde6be56640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6415d130 con 0x7fde640c4c90 2023-10-16T10:05:34.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.450+0000 7fde693f1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde6415db00 0x7fde640c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:34.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.450+0000 7fde693f1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde6415db00 0x7fde640c4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56380/0 (socket says 172.21.15.84:56380) 2023-10-16T10:05:34.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.450+0000 7fde693f1640 1 -- 172.21.15.84:0/3613296514 learned_addr learned my addr 172.21.15.84:0/3613296514 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:34.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.451+0000 7fde69bf2640 1 --2- 172.21.15.84:0/3613296514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde50005180 0x7fde64151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:34.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.451+0000 7fde693f1640 1 -- 172.21.15.84:0/3613296514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde640c4c90 msgr2=0x7fde6415bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:34.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.451+0000 7fde693f1640 1 --2- 172.21.15.84:0/3613296514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde640c4c90 0x7fde6415bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.451+0000 7fde693f1640 1 -- 172.21.15.84:0/3613296514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde50005180 msgr2=0x7fde64151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:34.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.451+0000 7fde693f1640 1 --2- 172.21.15.84:0/3613296514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde50005180 0x7fde64151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.451+0000 7fde693f1640 1 -- 172.21.15.84:0/3613296514 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde641560f0 con 0x7fde6415db00 2023-10-16T10:05:34.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.451+0000 7fde693f1640 1 --2- 172.21.15.84:0/3613296514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde6415db00 0x7fde640c4750 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fde5400c450 tx=0x7fde5400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:34.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.451+0000 7fde5a7fc640 1 -- 172.21.15.84:0/3613296514 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540052d0 con 0x7fde6415db00 2023-10-16T10:05:34.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.451+0000 7fde5a7fc640 1 -- 172.21.15.84:0/3613296514 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde54005430 con 0x7fde6415db00 2023-10-16T10:05:34.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.451+0000 7fde5a7fc640 1 -- 172.21.15.84:0/3613296514 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54012870 con 0x7fde6415db00 2023-10-16T10:05:34.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.451+0000 7fde6be56640 1 -- 172.21.15.84:0/3613296514 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde641563d0 con 0x7fde6415db00 2023-10-16T10:05:34.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.451+0000 7fde6be56640 1 -- 172.21.15.84:0/3613296514 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde64152170 con 0x7fde6415db00 2023-10-16T10:05:34.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.453+0000 7fde6be56640 1 -- 172.21.15.84:0/3613296514 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6404e160 con 0x7fde6415db00 2023-10-16T10:05:34.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.453+0000 7fde5a7fc640 1 -- 172.21.15.84:0/3613296514 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fde540129d0 con 0x7fde6415db00 2023-10-16T10:05:34.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.455+0000 7fde5a7fc640 1 --2- 172.21.15.84:0/3613296514 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fde4806af00 0x7fde4806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:34.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.455+0000 7fde5a7fc640 1 -- 172.21.15.84:0/3613296514 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6397+0+0 (secure 0 0 0) 0x7fde540893a0 con 0x7fde6415db00 2023-10-16T10:05:34.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.455+0000 7fde69bf2640 1 --2- 172.21.15.84:0/3613296514 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fde4806af00 0x7fde4806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:34.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.456+0000 7fde69bf2640 1 --2- 172.21.15.84:0/3613296514 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fde4806af00 0x7fde4806d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fde640cd990 tx=0x7fde60008040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:34.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.459+0000 7fde5a7fc640 1 -- 172.21.15.84:0/3613296514 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fde54054240 con 0x7fde6415db00 2023-10-16T10:05:34.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.660+0000 7fde6be56640 1 -- 172.21.15.84:0/3613296514 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde640cdc60 con 0x7fde6415db00 2023-10-16T10:05:34.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.662+0000 7fde5a7fc640 1 -- 172.21.15.84:0/3613296514 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+17540 (secure 0 0 0) 0x7fde54057860 con 0x7fde6415db00 2023-10-16T10:05:34.664 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:34.664 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":97,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:05:31.449789+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:05:08.636077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-16T10:05:09.841985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2152330716":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/3965108955":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/1978512312":"2023-10-17T10:05:31.449737+0000","172.21.15.84:6833/494168607":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/2341097922":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/4027382473":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/3685429931":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/3830790285":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/821251662":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/504267391":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/3778743167":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/150347074":"2023-10-17T10:05:25.079928+0000","172.21.15.84:6832/494168607":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/3151132884":"2023-10-17T10:05:25.079928+0000","172.21.15.84:6833/147697688":"2023-10-17T10:05:25.079928+0000","172.21.15.84:6832/147697688":"2023-10-17T10:05:25.079928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:05:34.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.664+0000 7fde6be56640 1 -- 172.21.15.84:0/3613296514 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fde4806af00 msgr2=0x7fde4806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:34.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.664+0000 7fde6be56640 1 --2- 172.21.15.84:0/3613296514 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fde4806af00 0x7fde4806d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fde640cd990 tx=0x7fde60008040 comp rx=0 tx=0).stop 2023-10-16T10:05:34.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.664+0000 7fde6be56640 1 -- 172.21.15.84:0/3613296514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde6415db00 msgr2=0x7fde640c4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:34.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.664+0000 7fde6be56640 1 --2- 172.21.15.84:0/3613296514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde6415db00 0x7fde640c4750 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fde5400c450 tx=0x7fde5400c910 comp rx=0 tx=0).stop 2023-10-16T10:05:34.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.664+0000 7fde6be56640 1 -- 172.21.15.84:0/3613296514 shutdown_connections 2023-10-16T10:05:34.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.664+0000 7fde6be56640 1 --2- 172.21.15.84:0/3613296514 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fde4806af00 0x7fde4806d3c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.664+0000 7fde6be56640 1 --2- 172.21.15.84:0/3613296514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde640c4c90 0x7fde6415bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.664+0000 7fde6be56640 1 --2- 172.21.15.84:0/3613296514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde6415db00 0x7fde640c4750 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.664+0000 7fde6be56640 1 --2- 172.21.15.84:0/3613296514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde50005180 0x7fde64151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.664+0000 7fde6be56640 1 -- 172.21.15.84:0/3613296514 >> 172.21.15.84:0/3613296514 conn(0x7fde640cb410 msgr2=0x7fde6406ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:34.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.665+0000 7fde6be56640 1 -- 172.21.15.84:0/3613296514 shutdown_connections 2023-10-16T10:05:34.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.665+0000 7fde6be56640 1 -- 172.21.15.84:0/3613296514 wait complete. 2023-10-16T10:05:34.686 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T10:05:34.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.900+0000 7f55d27e2640 1 Processor -- start 2023-10-16T10:05:34.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.900+0000 7f55d27e2640 1 -- start start 2023-10-16T10:05:34.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.901+0000 7f55d27e2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55cc0b8480 0x7f55cc0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:34.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.901+0000 7f55d27e2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55cc15d150 con 0x7f55cc15cd70 2023-10-16T10:05:34.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.902+0000 7f55d27e2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55cc0d1180 con 0x7f55cc0d0be0 2023-10-16T10:05:34.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.902+0000 7f55d27e2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0d1460 con 0x7f55cc0b8480 2023-10-16T10:05:34.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.902+0000 7f55d17e0640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f55cc15cd70 0x7f55cc0d2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57886/0 (socket says 172.21.15.84:57886) 2023-10-16T10:05:34.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.902+0000 7f55d17e0640 1 -- 172.21.15.84:0/2409997271 learned_addr learned my addr 172.21.15.84:0/2409997271 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:34.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.902+0000 7f55d0fdf640 1 --2- 172.21.15.84:0/2409997271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55cc0b8480 0x7f55cc0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:34.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.902+0000 7f55c3fff640 1 -- 172.21.15.84:0/2409997271 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416673467 0 0) 0x7f55cc15d150 con 0x7f55cc15cd70 2023-10-16T10:05:34.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.902+0000 7f55c3fff640 1 -- 172.21.15.84:0/2409997271 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55b8003520 con 0x7f55cc15cd70 2023-10-16T10:05:34.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55c3fff640 1 -- 172.21.15.84:0/2409997271 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3450554368 0 0) 0x7f55b8003520 con 0x7f55cc15cd70 2023-10-16T10:05:34.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55c3fff640 1 -- 172.21.15.84:0/2409997271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55cc0b8480 msgr2=0x7f55cc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:34.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55c3fff640 1 --2- 172.21.15.84:0/2409997271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55cc0b8480 0x7f55cc0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55c3fff640 1 -- 172.21.15.84:0/2409997271 >> v1:172.21.15.157:6789/0 conn(0x7f55cc0d0be0 legacy=0x7f55cc0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:34.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55c3fff640 1 -- 172.21.15.84:0/2409997271 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc0d1740 con 0x7f55cc15cd70 2023-10-16T10:05:34.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55c3fff640 1 -- 172.21.15.84:0/2409997271 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f55c8001670 con 0x7f55cc15cd70 2023-10-16T10:05:34.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55d0fdf640 1 --2- 172.21.15.84:0/2409997271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55cc0b8480 0x7f55cc0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:05:34.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55c3fff640 1 -- 172.21.15.84:0/2409997271 >> v1:172.21.15.84:6789/0 conn(0x7f55cc15cd70 legacy=0x7f55cc0d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:34.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55c3fff640 1 --2- 172.21.15.84:0/2409997271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55cc0b8480 0x7f55b80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:34.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55c3fff640 1 --2- 172.21.15.84:0/2409997271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55b8003fe0 0x7f55b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:34.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55c3fff640 1 --2- 172.21.15.84:0/2409997271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55b8004d00 0x7f55b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:34.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55c3fff640 1 -- 172.21.15.84:0/2409997271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0d1460 con 0x7f55b8003fe0 2023-10-16T10:05:34.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55c3fff640 1 -- 172.21.15.84:0/2409997271 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b800bb40 con 0x7f55b8004d00 2023-10-16T10:05:34.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55c3fff640 1 -- 172.21.15.84:0/2409997271 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b800bd00 con 0x7f55cc0b8480 2023-10-16T10:05:34.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55d0fdf640 1 --2- 172.21.15.84:0/2409997271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55b8003fe0 0x7f55b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:34.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55d17e0640 1 --2- 172.21.15.84:0/2409997271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55cc0b8480 0x7f55b80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:34.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55d1fe1640 1 --2- 172.21.15.84:0/2409997271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55b8004d00 0x7f55b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:34.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55d0fdf640 1 -- 172.21.15.84:0/2409997271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55cc0b8480 msgr2=0x7f55b80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:34.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55d0fdf640 1 --2- 172.21.15.84:0/2409997271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55cc0b8480 0x7f55b80039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55d0fdf640 1 -- 172.21.15.84:0/2409997271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55b8004d00 msgr2=0x7f55b8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:34.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55d0fdf640 1 --2- 172.21.15.84:0/2409997271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55b8004d00 0x7f55b8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.903+0000 7f55d0fdf640 1 -- 172.21.15.84:0/2409997271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b8003520 con 0x7f55b8003fe0 2023-10-16T10:05:34.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.904+0000 7f55d1fe1640 1 --2- 172.21.15.84:0/2409997271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55b8004d00 0x7f55b8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:34.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.905+0000 7f55d0fdf640 1 --2- 172.21.15.84:0/2409997271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55b8003fe0 0x7f55b80047c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0098a0 tx=0x7f55b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf491d2bb517e52 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:34.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.905+0000 7f55c3fff640 1 -- 172.21.15.84:0/2409997271 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc004230 con 0x7f55b8003fe0 2023-10-16T10:05:34.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.905+0000 7f55c3fff640 1 -- 172.21.15.84:0/2409997271 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55bc004390 con 0x7f55b8003fe0 2023-10-16T10:05:34.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.905+0000 7f55c3fff640 1 -- 172.21.15.84:0/2409997271 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc003830 con 0x7f55b8003fe0 2023-10-16T10:05:34.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.905+0000 7f55d27e2640 1 -- 172.21.15.84:0/2409997271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55b8003fe0 msgr2=0x7f55b80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:34.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.905+0000 7f55d27e2640 1 --2- 172.21.15.84:0/2409997271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55b8003fe0 0x7f55b80047c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0098a0 tx=0x7f55b80057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.905+0000 7f55d1fe1640 1 -- 172.21.15.84:0/2409997271 reap_dead start 2023-10-16T10:05:34.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.906+0000 7f55d27e2640 1 -- 172.21.15.84:0/2409997271 shutdown_connections 2023-10-16T10:05:34.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.906+0000 7f55d27e2640 1 -- 172.21.15.84:0/2409997271 >> 172.21.15.84:0/2409997271 conn(0x7f55cc0cf570 msgr2=0x7f55cc0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:34.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.906+0000 7f55d27e2640 1 -- 172.21.15.84:0/2409997271 shutdown_connections 2023-10-16T10:05:34.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.906+0000 7f55d27e2640 1 -- 172.21.15.84:0/2409997271 wait complete. 2023-10-16T10:05:34.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.907+0000 7f55d27e2640 1 Processor -- start 2023-10-16T10:05:34.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.908+0000 7f55d27e2640 1 -- start start 2023-10-16T10:05:34.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.908+0000 7f55d27e2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55b8003fe0 0x7f55cc1f3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:34.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.908+0000 7f55d27e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55cc156f60 0x7f55cc1526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:34.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.909+0000 7f55d27e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55cc158a30 0x7f55cc158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:34.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.909+0000 7f55d27e2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc1f3980 con 0x7f55cc156f60 2023-10-16T10:05:34.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.909+0000 7f55d27e2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc157420 con 0x7f55cc158a30 2023-10-16T10:05:34.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.909+0000 7f55d27e2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc156550 con 0x7f55b8003fe0 2023-10-16T10:05:34.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.909+0000 7f55d0fdf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55cc156f60 0x7f55cc1526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:34.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.909+0000 7f55d0fdf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55cc156f60 0x7f55cc1526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56392/0 (socket says 172.21.15.84:56392) 2023-10-16T10:05:34.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.909+0000 7f55d0fdf640 1 -- 172.21.15.84:0/234131499 learned_addr learned my addr 172.21.15.84:0/234131499 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:34.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.909+0000 7f55d17e0640 1 --2- 172.21.15.84:0/234131499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55b8003fe0 0x7f55cc1f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:34.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.909+0000 7f55d1fe1640 1 --2- 172.21.15.84:0/234131499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55cc158a30 0x7f55cc158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:34.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.909+0000 7f55d0fdf640 1 -- 172.21.15.84:0/234131499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55b8003fe0 msgr2=0x7f55cc1f3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:34.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.909+0000 7f55d0fdf640 1 --2- 172.21.15.84:0/234131499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55b8003fe0 0x7f55cc1f3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.909+0000 7f55d0fdf640 1 -- 172.21.15.84:0/234131499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55cc158a30 msgr2=0x7f55cc158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:34.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.909+0000 7f55d0fdf640 1 --2- 172.21.15.84:0/234131499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55cc158a30 0x7f55cc158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:34.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.909+0000 7f55d0fdf640 1 -- 172.21.15.84:0/234131499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc156830 con 0x7f55cc156f60 2023-10-16T10:05:34.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.910+0000 7f55d0fdf640 1 --2- 172.21.15.84:0/234131499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55cc156f60 0x7f55cc1526a0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f55bc02bea0 tx=0x7f55bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:34.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.910+0000 7f55c1ffb640 1 -- 172.21.15.84:0/234131499 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc030610 con 0x7f55cc156f60 2023-10-16T10:05:34.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.910+0000 7f55c1ffb640 1 -- 172.21.15.84:0/234131499 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55bc030770 con 0x7f55cc156f60 2023-10-16T10:05:34.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.910+0000 7f55d27e2640 1 -- 172.21.15.84:0/234131499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55cc156b10 con 0x7f55cc156f60 2023-10-16T10:05:34.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.910+0000 7f55c1ffb640 1 -- 172.21.15.84:0/234131499 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc030b20 con 0x7f55cc156f60 2023-10-16T10:05:34.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.910+0000 7f55d27e2640 1 -- 172.21.15.84:0/234131499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55cc159400 con 0x7f55cc156f60 2023-10-16T10:05:34.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.911+0000 7f55d27e2640 1 -- 172.21.15.84:0/234131499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55cc0b1f10 con 0x7f55cc156f60 2023-10-16T10:05:34.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.912+0000 7f55c1ffb640 1 -- 172.21.15.84:0/234131499 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f55bc030c80 con 0x7f55cc156f60 2023-10-16T10:05:34.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.913+0000 7f55c1ffb640 1 --2- 172.21.15.84:0/234131499 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f55b006af00 0x7f55b006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:34.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.913+0000 7f55c1ffb640 1 -- 172.21.15.84:0/234131499 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f55bc0ab550 con 0x7f55cc156f60 2023-10-16T10:05:34.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.914+0000 7f55d17e0640 1 --2- 172.21.15.84:0/234131499 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f55b006af00 0x7f55b006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:34.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.914+0000 7f55d17e0640 1 --2- 172.21.15.84:0/234131499 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f55b006af00 0x7f55b006d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f55c8000ff0 tx=0x7f55c8009040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:34.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:34.918+0000 7f55c1ffb640 1 -- 172.21.15.84:0/234131499 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f55bc076260 con 0x7f55cc156f60 2023-10-16T10:05:35.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.159+0000 7f55d27e2640 1 -- 172.21.15.84:0/234131499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f55cc1f3980 con 0x7f55cc156f60 2023-10-16T10:05:35.161 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-16T10:05:35.160+0000 7fca0c7ff640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T10:05:35.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.163+0000 7f55c1ffb640 1 -- 172.21.15.84:0/234131499 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) v1 ==== 161+0+0 (secure 0 0 0) 0x7f55bc006e10 con 0x7f55cc156f60 2023-10-16T10:05:35.165 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T10:05:35.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.166+0000 7f55d27e2640 1 -- 172.21.15.84:0/234131499 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f55b006af00 msgr2=0x7f55b006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:35.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.166+0000 7f55d27e2640 1 --2- 172.21.15.84:0/234131499 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f55b006af00 0x7f55b006d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f55c8000ff0 tx=0x7f55c8009040 comp rx=0 tx=0).stop 2023-10-16T10:05:35.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.166+0000 7f55d27e2640 1 -- 172.21.15.84:0/234131499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55cc156f60 msgr2=0x7f55cc1526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:35.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.166+0000 7f55d27e2640 1 --2- 172.21.15.84:0/234131499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55cc156f60 0x7f55cc1526a0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f55bc02bea0 tx=0x7f55bc004730 comp rx=0 tx=0).stop 2023-10-16T10:05:35.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.166+0000 7f55d27e2640 1 -- 172.21.15.84:0/234131499 shutdown_connections 2023-10-16T10:05:35.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.166+0000 7f55d27e2640 1 --2- 172.21.15.84:0/234131499 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f55b006af00 0x7f55b006d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:35.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.166+0000 7f55d27e2640 1 --2- 172.21.15.84:0/234131499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55cc158a30 0x7f55cc158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:35.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.166+0000 7f55d27e2640 1 --2- 172.21.15.84:0/234131499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55cc156f60 0x7f55cc1526a0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:35.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.166+0000 7f55d27e2640 1 --2- 172.21.15.84:0/234131499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55b8003fe0 0x7f55cc1f3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:35.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.166+0000 7f55d27e2640 1 -- 172.21.15.84:0/234131499 >> 172.21.15.84:0/234131499 conn(0x7f55cc0cf570 msgr2=0x7f55cc06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:35.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.167+0000 7f55d27e2640 1 -- 172.21.15.84:0/234131499 shutdown_connections 2023-10-16T10:05:35.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.167+0000 7f55d27e2640 1 -- 172.21.15.84:0/234131499 wait complete. 2023-10-16T10:05:35.180 INFO:tasks.ceph.mds.c.smithi084.stderr: -2600> 2023-10-16T10:05:17.316+0000 7f8900b07640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:05:35.188 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T10:05:35.235 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T10:05:35.248 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:35.246+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:35.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.382+0000 7f96e7fff640 1 Processor -- start 2023-10-16T10:05:35.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.382+0000 7f96e7fff640 1 -- start start 2023-10-16T10:05:35.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.383+0000 7f96e7fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f96e8160620 0x7f96e80caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:35.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.383+0000 7f96e7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f96e80b1ca0 0x7f96e80cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:35.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.383+0000 7f96e7fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e806afe0 con 0x7f96e8160240 2023-10-16T10:05:35.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.383+0000 7f96e7fff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e80cf540 con 0x7f96e80b1ca0 2023-10-16T10:05:35.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.383+0000 7f96e7fff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e80cf820 con 0x7f96e8160620 2023-10-16T10:05:35.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.383+0000 7f96e77fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f96e80b1ca0 0x7f96e80cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:35.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.383+0000 7f96e77fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f96e80b1ca0 0x7f96e80cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56396/0 (socket says 172.21.15.84:56396) 2023-10-16T10:05:35.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.383+0000 7f96e6ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f96e8160240 0x7f96e80ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57890/0 (socket says 172.21.15.84:57890) 2023-10-16T10:05:35.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.383+0000 7f96e77fe640 1 -- 172.21.15.84:0/2142265333 learned_addr learned my addr 172.21.15.84:0/2142265333 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:35.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.384+0000 7f96e67fc640 1 --2- 172.21.15.84:0/2142265333 >> v2:172.21.15.157:3300/0 conn(0x7f96e8160620 0x7f96e80caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:35.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.384+0000 7f96e77fe640 1 -- 172.21.15.84:0/2142265333 >> v2:172.21.15.157:3300/0 conn(0x7f96e8160620 msgr2=0x7f96e80caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:35.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.384+0000 7f96e77fe640 1 --2- 172.21.15.84:0/2142265333 >> v2:172.21.15.157:3300/0 conn(0x7f96e8160620 0x7f96e80caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:35.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.384+0000 7f96e77fe640 1 -- 172.21.15.84:0/2142265333 >> v1:172.21.15.84:6789/0 conn(0x7f96e8160240 legacy=0x7f96e80ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:35.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.384+0000 7f96e77fe640 1 -- 172.21.15.84:0/2142265333 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e80cfb00 con 0x7f96e80b1ca0 2023-10-16T10:05:35.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.384+0000 7f96e5ffb640 1 -- 172.21.15.84:0/2142265333 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 485483743 0 0) 0x7f96e806afe0 con 0x7f96e8160240 2023-10-16T10:05:35.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.385+0000 7f96e77fe640 1 --2- 172.21.15.84:0/2142265333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e80b1ca0 0x7f96e80cd220 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f96e80739f0 tx=0x7f96e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21bc1258fdeb8d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:35.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.385+0000 7f96e5ffb640 1 -- 172.21.15.84:0/2142265333 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0031870 con 0x7f96e80b1ca0 2023-10-16T10:05:35.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.385+0000 7f96e5ffb640 1 -- 172.21.15.84:0/2142265333 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e00319d0 con 0x7f96e80b1ca0 2023-10-16T10:05:35.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.385+0000 7f96e5ffb640 1 -- 172.21.15.84:0/2142265333 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0031d60 con 0x7f96e80b1ca0 2023-10-16T10:05:35.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.386+0000 7f96e7fff640 1 -- 172.21.15.84:0/2142265333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e80b1ca0 msgr2=0x7f96e80cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:35.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.386+0000 7f96e7fff640 1 --2- 172.21.15.84:0/2142265333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e80b1ca0 0x7f96e80cd220 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f96e80739f0 tx=0x7f96e002b910 comp rx=0 tx=0).stop 2023-10-16T10:05:35.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.386+0000 7f96e7fff640 1 -- 172.21.15.84:0/2142265333 shutdown_connections 2023-10-16T10:05:35.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.386+0000 7f96e7fff640 1 --2- 172.21.15.84:0/2142265333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e80b1ca0 0x7f96e80cd220 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:35.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.386+0000 7f96e7fff640 1 --2- 172.21.15.84:0/2142265333 >> v2:172.21.15.157:3300/0 conn(0x7f96e8160620 0x7f96e80caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:35.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.386+0000 7f96e7fff640 1 -- 172.21.15.84:0/2142265333 >> 172.21.15.84:0/2142265333 conn(0x7f96e8099a10 msgr2=0x7f96e80d0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:35.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.386+0000 7f96e7fff640 1 -- 172.21.15.84:0/2142265333 shutdown_connections 2023-10-16T10:05:35.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.386+0000 7f96e7fff640 1 -- 172.21.15.84:0/2142265333 wait complete. 2023-10-16T10:05:35.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.388+0000 7f96e7fff640 1 Processor -- start 2023-10-16T10:05:35.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.388+0000 7f96e7fff640 1 -- start start 2023-10-16T10:05:35.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.389+0000 7f96e7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e80b1ca0 0x7f96e81e5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:35.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.389+0000 7f96e7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e8160240 0x7f96e81e70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:35.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.389+0000 7f96e7fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96e8160620 0x7f96e8155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:35.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.389+0000 7f96e7fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80c4d30 con 0x7f96e8160240 2023-10-16T10:05:35.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.389+0000 7f96e7fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e8155640 con 0x7f96e80b1ca0 2023-10-16T10:05:35.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.389+0000 7f96e7fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80d12c0 con 0x7f96e8160620 2023-10-16T10:05:35.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.389+0000 7f96e67fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e8160240 0x7f96e81e70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:35.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.389+0000 7f96e67fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e8160240 0x7f96e81e70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56410/0 (socket says 172.21.15.84:56410) 2023-10-16T10:05:35.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.389+0000 7f96e67fc640 1 -- 172.21.15.84:0/4036148045 learned_addr learned my addr 172.21.15.84:0/4036148045 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:35.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.390+0000 7f96e6ffd640 1 --2- 172.21.15.84:0/4036148045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e80b1ca0 0x7f96e81e5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:35.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.390+0000 7f96e77fe640 1 --2- 172.21.15.84:0/4036148045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96e8160620 0x7f96e8155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:35.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.390+0000 7f96e67fc640 1 -- 172.21.15.84:0/4036148045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96e8160620 msgr2=0x7f96e8155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:35.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.390+0000 7f96e67fc640 1 --2- 172.21.15.84:0/4036148045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96e8160620 0x7f96e8155100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:35.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.390+0000 7f96e67fc640 1 -- 172.21.15.84:0/4036148045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e80b1ca0 msgr2=0x7f96e81e5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:35.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.390+0000 7f96e67fc640 1 --2- 172.21.15.84:0/4036148045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e80b1ca0 0x7f96e81e5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:35.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.390+0000 7f96e67fc640 1 -- 172.21.15.84:0/4036148045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e00094d0 con 0x7f96e8160240 2023-10-16T10:05:35.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.390+0000 7f96e67fc640 1 --2- 172.21.15.84:0/4036148045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e8160240 0x7f96e81e70c0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f96d000d5f0 tx=0x7f96d000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:35.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.390+0000 7f96dffff640 1 -- 172.21.15.84:0/4036148045 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d00125f0 con 0x7f96e8160240 2023-10-16T10:05:35.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.391+0000 7f96dffff640 1 -- 172.21.15.84:0/4036148045 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d0012750 con 0x7f96e8160240 2023-10-16T10:05:35.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.391+0000 7f96dffff640 1 -- 172.21.15.84:0/4036148045 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0012b00 con 0x7f96e8160240 2023-10-16T10:05:35.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.391+0000 7f96e7fff640 1 -- 172.21.15.84:0/4036148045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e81e5d10 con 0x7f96e8160240 2023-10-16T10:05:35.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.391+0000 7f96e7fff640 1 -- 172.21.15.84:0/4036148045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e81e62d0 con 0x7f96e8160240 2023-10-16T10:05:35.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.392+0000 7f96e7fff640 1 -- 172.21.15.84:0/4036148045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e80c4a90 con 0x7f96e8160240 2023-10-16T10:05:35.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.393+0000 7f96dffff640 1 -- 172.21.15.84:0/4036148045 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f96d0012c60 con 0x7f96e8160240 2023-10-16T10:05:35.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.395+0000 7f96dffff640 1 --2- 172.21.15.84:0/4036148045 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f96c806af00 0x7f96c806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:35.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.395+0000 7f96dffff640 1 -- 172.21.15.84:0/4036148045 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f96d0089ed0 con 0x7f96e8160240 2023-10-16T10:05:35.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.395+0000 7f96e6ffd640 1 --2- 172.21.15.84:0/4036148045 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f96c806af00 0x7f96c806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:35.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.396+0000 7f96e6ffd640 1 --2- 172.21.15.84:0/4036148045 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f96c806af00 0x7f96c806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f96d80012e0 tx=0x7f96d8007ed0 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:35.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.400+0000 7f96dffff640 1 -- 172.21.15.84:0/4036148045 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f96e80c4a90 con 0x7f96e8160240 2023-10-16T10:05:35.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:35.634+0000 7f96e7fff640 1 -- 172.21.15.84:0/4036148045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f96e80c4a90 con 0x7f96e8160240 2023-10-16T10:05:36.173 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:36.170+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:36.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.199+0000 7f96dffff640 1 -- 172.21.15.84:0/4036148045 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) v1 ==== 104+0+0 (secure 0 0 0) 0x7f96e80c4a90 con 0x7f96e8160240 2023-10-16T10:05:36.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.202+0000 7f96e7fff640 1 -- 172.21.15.84:0/4036148045 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f96c806af00 msgr2=0x7f96c806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:36.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.202+0000 7f96e7fff640 1 --2- 172.21.15.84:0/4036148045 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f96c806af00 0x7f96c806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f96d80012e0 tx=0x7f96d8007ed0 comp rx=0 tx=0).stop 2023-10-16T10:05:36.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.202+0000 7f96e7fff640 1 -- 172.21.15.84:0/4036148045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e8160240 msgr2=0x7f96e81e70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:36.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.202+0000 7f96e7fff640 1 --2- 172.21.15.84:0/4036148045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e8160240 0x7f96e81e70c0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f96d000d5f0 tx=0x7f96d000dab0 comp rx=0 tx=0).stop 2023-10-16T10:05:36.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.203+0000 7f96e7fff640 1 -- 172.21.15.84:0/4036148045 shutdown_connections 2023-10-16T10:05:36.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.203+0000 7f96e7fff640 1 --2- 172.21.15.84:0/4036148045 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f96c806af00 0x7f96c806d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:36.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.203+0000 7f96e7fff640 1 --2- 172.21.15.84:0/4036148045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96e8160620 0x7f96e8155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:36.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.203+0000 7f96e7fff640 1 --2- 172.21.15.84:0/4036148045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e8160240 0x7f96e81e70c0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:36.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.203+0000 7f96e7fff640 1 --2- 172.21.15.84:0/4036148045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e80b1ca0 0x7f96e81e5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:36.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.203+0000 7f96e7fff640 1 -- 172.21.15.84:0/4036148045 >> 172.21.15.84:0/4036148045 conn(0x7f96e8099a10 msgr2=0x7f96e80c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:36.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.203+0000 7f96e7fff640 1 -- 172.21.15.84:0/4036148045 shutdown_connections 2023-10-16T10:05:36.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.203+0000 7f96e7fff640 1 -- 172.21.15.84:0/4036148045 wait complete. 2023-10-16T10:05:36.225 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T10:05:36.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.495+0000 7fdccdb48640 1 Processor -- start 2023-10-16T10:05:36.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.495+0000 7fdccdb48640 1 -- start start 2023-10-16T10:05:36.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.497+0000 7fdccdb48640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcc80c9300 0x7fdcc806c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:36.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.497+0000 7fdccdb48640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc815dfb0 con 0x7fdcc8159da0 2023-10-16T10:05:36.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.497+0000 7fdccdb48640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc815e290 con 0x7fdcc81599c0 2023-10-16T10:05:36.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.497+0000 7fdccdb48640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc80c9740 con 0x7fdcc80c9300 2023-10-16T10:05:36.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.497+0000 7fdcbffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdcc8159da0 0x7fdcc80cce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39904/0 (socket says 172.21.15.84:39904) 2023-10-16T10:05:36.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.497+0000 7fdcbffff640 1 -- 172.21.15.84:0/399725880 learned_addr learned my addr 172.21.15.84:0/399725880 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:36.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.498+0000 7fdcc7fff640 1 --2- 172.21.15.84:0/399725880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcc80c9300 0x7fdcc806c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:36.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.498+0000 7fdcc6ffd640 1 -- 172.21.15.84:0/399725880 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1062558926 0 0) 0x7fdcc815dfb0 con 0x7fdcc8159da0 2023-10-16T10:05:36.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.498+0000 7fdcc6ffd640 1 -- 172.21.15.84:0/399725880 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb4005730 con 0x7fdcc8159da0 2023-10-16T10:05:36.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.498+0000 7fdcc7fff640 1 -- 172.21.15.84:0/399725880 >> v1:172.21.15.157:6789/0 conn(0x7fdcc81599c0 legacy=0x7fdcc815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:36.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.498+0000 7fdcc6ffd640 1 -- 172.21.15.84:0/399725880 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3391005887 0 0) 0x7fdcc815e290 con 0x7fdcc81599c0 2023-10-16T10:05:36.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.498+0000 7fdcc7fff640 1 -- 172.21.15.84:0/399725880 >> v1:172.21.15.84:6789/0 conn(0x7fdcc8159da0 legacy=0x7fdcc80cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:36.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.498+0000 7fdcc7fff640 1 -- 172.21.15.84:0/399725880 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc815e570 con 0x7fdcc80c9300 2023-10-16T10:05:36.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.500+0000 7fdcc7fff640 1 --2- 172.21.15.84:0/399725880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcc80c9300 0x7fdcc806c0d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009dc0 tx=0x7fdcc002bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0dcc7b17a03fe5b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:36.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.500+0000 7fdcc6ffd640 1 -- 172.21.15.84:0/399725880 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc002ccc0 con 0x7fdcc80c9300 2023-10-16T10:05:36.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.501+0000 7fdcc6ffd640 1 -- 172.21.15.84:0/399725880 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc002ce20 con 0x7fdcc80c9300 2023-10-16T10:05:36.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.501+0000 7fdcc6ffd640 1 -- 172.21.15.84:0/399725880 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0034660 con 0x7fdcc80c9300 2023-10-16T10:05:36.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.501+0000 7fdccdb48640 1 -- 172.21.15.84:0/399725880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcc80c9300 msgr2=0x7fdcc806c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:36.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.501+0000 7fdccdb48640 1 --2- 172.21.15.84:0/399725880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcc80c9300 0x7fdcc806c0d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009dc0 tx=0x7fdcc002bf10 comp rx=0 tx=0).stop 2023-10-16T10:05:36.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.501+0000 7fdccdb48640 1 -- 172.21.15.84:0/399725880 shutdown_connections 2023-10-16T10:05:36.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.501+0000 7fdccdb48640 1 --2- 172.21.15.84:0/399725880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcc80c9300 0x7fdcc806c0d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:36.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.501+0000 7fdccdb48640 1 -- 172.21.15.84:0/399725880 >> 172.21.15.84:0/399725880 conn(0x7fdcc80cb360 msgr2=0x7fdcc80cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:36.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.502+0000 7fdccdb48640 1 -- 172.21.15.84:0/399725880 shutdown_connections 2023-10-16T10:05:36.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.502+0000 7fdccdb48640 1 -- 172.21.15.84:0/399725880 wait complete. 2023-10-16T10:05:36.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.503+0000 7fdccdb48640 1 Processor -- start 2023-10-16T10:05:36.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.503+0000 7fdccdb48640 1 -- start start 2023-10-16T10:05:36.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.504+0000 7fdccdb48640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcc80c9300 0x7fdcc814d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:36.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.504+0000 7fdccdb48640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcc81599c0 0x7fdcc814d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:36.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.504+0000 7fdccdb48640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcc8159da0 0x7fdcc806b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:36.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.504+0000 7fdccdb48640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc814d760 con 0x7fdcc81599c0 2023-10-16T10:05:36.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.504+0000 7fdccdb48640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc80b21b0 con 0x7fdcc80c9300 2023-10-16T10:05:36.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.504+0000 7fdccdb48640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc80b1630 con 0x7fdcc8159da0 2023-10-16T10:05:36.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.505+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcc81599c0 0x7fdcc814d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:36.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.505+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcc81599c0 0x7fdcc814d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46092/0 (socket says 172.21.15.84:46092) 2023-10-16T10:05:36.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.505+0000 7fdcbffff640 1 -- 172.21.15.84:0/1746894757 learned_addr learned my addr 172.21.15.84:0/1746894757 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:36.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.505+0000 7fdcc77fe640 1 --2- 172.21.15.84:0/1746894757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcc80c9300 0x7fdcc814d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:36.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.505+0000 7fdcbffff640 1 -- 172.21.15.84:0/1746894757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcc8159da0 msgr2=0x7fdcc806b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:36.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.505+0000 7fdcbffff640 1 --2- 172.21.15.84:0/1746894757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcc8159da0 0x7fdcc806b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:36.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.505+0000 7fdcbffff640 1 -- 172.21.15.84:0/1746894757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcc80c9300 msgr2=0x7fdcc814d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:36.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.505+0000 7fdcbffff640 1 --2- 172.21.15.84:0/1746894757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcc80c9300 0x7fdcc814d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:36.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.505+0000 7fdcbffff640 1 -- 172.21.15.84:0/1746894757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc0009a70 con 0x7fdcc81599c0 2023-10-16T10:05:36.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.506+0000 7fdcbffff640 1 --2- 172.21.15.84:0/1746894757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcc81599c0 0x7fdcc814d930 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000c410 tx=0x7fdcb000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:36.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.506+0000 7fdcc4ff9640 1 -- 172.21.15.84:0/1746894757 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0005ce0 con 0x7fdcc81599c0 2023-10-16T10:05:36.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.506+0000 7fdcc4ff9640 1 -- 172.21.15.84:0/1746894757 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb0005e40 con 0x7fdcc81599c0 2023-10-16T10:05:36.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.506+0000 7fdccdb48640 1 -- 172.21.15.84:0/1746894757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc80b1910 con 0x7fdcc81599c0 2023-10-16T10:05:36.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.506+0000 7fdcc4ff9640 1 -- 172.21.15.84:0/1746894757 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00072f0 con 0x7fdcc81599c0 2023-10-16T10:05:36.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.506+0000 7fdccdb48640 1 -- 172.21.15.84:0/1746894757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc80b1ed0 con 0x7fdcc81599c0 2023-10-16T10:05:36.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.507+0000 7fdccdb48640 1 -- 172.21.15.84:0/1746894757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc804e160 con 0x7fdcc81599c0 2023-10-16T10:05:36.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.508+0000 7fdcc4ff9640 1 -- 172.21.15.84:0/1746894757 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fdcb0007450 con 0x7fdcc81599c0 2023-10-16T10:05:36.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.509+0000 7fdcc4ff9640 1 --2- 172.21.15.84:0/1746894757 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fdca806af30 0x7fdca806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:36.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.509+0000 7fdcc4ff9640 1 -- 172.21.15.84:0/1746894757 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fdcb0088f30 con 0x7fdcc81599c0 2023-10-16T10:05:36.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.509+0000 7fdcc77fe640 1 --2- 172.21.15.84:0/1746894757 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fdca806af30 0x7fdca806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:36.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.510+0000 7fdcc77fe640 1 --2- 172.21.15.84:0/1746894757 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fdca806af30 0x7fdca806d3f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8000f60 tx=0x7fdcb8009040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:36.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.513+0000 7fdcc4ff9640 1 -- 172.21.15.84:0/1746894757 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdcb00bbfe0 con 0x7fdcc81599c0 2023-10-16T10:05:36.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:36.712+0000 7fdccdb48640 1 -- 172.21.15.84:0/1746894757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdcc80ca8e0 con 0x7fdcc81599c0 2023-10-16T10:05:37.104 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:37.101+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:37.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.221+0000 7fdcc4ff9640 1 -- 172.21.15.84:0/1746894757 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) v1 ==== 181+0+0 (secure 0 0 0) 0x7fdcb00170d0 con 0x7fdcc81599c0 2023-10-16T10:05:37.223 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-16T10:05:37.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.225+0000 7fdccdb48640 1 -- 172.21.15.84:0/1746894757 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fdca806af30 msgr2=0x7fdca806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:37.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.225+0000 7fdccdb48640 1 --2- 172.21.15.84:0/1746894757 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fdca806af30 0x7fdca806d3f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8000f60 tx=0x7fdcb8009040 comp rx=0 tx=0).stop 2023-10-16T10:05:37.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.225+0000 7fdccdb48640 1 -- 172.21.15.84:0/1746894757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcc81599c0 msgr2=0x7fdcc814d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:37.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.225+0000 7fdccdb48640 1 --2- 172.21.15.84:0/1746894757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcc81599c0 0x7fdcc814d930 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000c410 tx=0x7fdcb000c8d0 comp rx=0 tx=0).stop 2023-10-16T10:05:37.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.226+0000 7fdccdb48640 1 -- 172.21.15.84:0/1746894757 shutdown_connections 2023-10-16T10:05:37.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.226+0000 7fdccdb48640 1 --2- 172.21.15.84:0/1746894757 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fdca806af30 0x7fdca806d3f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:37.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.226+0000 7fdccdb48640 1 --2- 172.21.15.84:0/1746894757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcc8159da0 0x7fdcc806b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:37.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.226+0000 7fdccdb48640 1 --2- 172.21.15.84:0/1746894757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcc81599c0 0x7fdcc814d930 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:37.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.226+0000 7fdccdb48640 1 --2- 172.21.15.84:0/1746894757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcc80c9300 0x7fdcc814d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:37.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.226+0000 7fdccdb48640 1 -- 172.21.15.84:0/1746894757 >> 172.21.15.84:0/1746894757 conn(0x7fdcc80cb360 msgr2=0x7fdcc8155da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:37.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.227+0000 7fdccdb48640 1 -- 172.21.15.84:0/1746894757 shutdown_connections 2023-10-16T10:05:37.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.227+0000 7fdccdb48640 1 -- 172.21.15.84:0/1746894757 wait complete. 2023-10-16T10:05:37.251 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T10:05:37.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.459+0000 7f369ab35640 1 Processor -- start 2023-10-16T10:05:37.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.459+0000 7f369ab35640 1 -- start start 2023-10-16T10:05:37.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.460+0000 7f369ab35640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369406c040 0x7f369406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:37.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.460+0000 7f369ab35640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f36940ad970 0x7f369415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:37.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.460+0000 7f369ab35640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f369406c960 con 0x7f369415ebc0 2023-10-16T10:05:37.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.461+0000 7f369ab35640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f369415b9d0 con 0x7f36940ad970 2023-10-16T10:05:37.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.461+0000 7f369ab35640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369415bcb0 con 0x7f369406c040 2023-10-16T10:05:37.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.461+0000 7f3699332640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f36940ad970 0x7f369415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:37.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.461+0000 7f369a334640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f369415ebc0 0x7f36940cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39918/0 (socket says 172.21.15.84:39918) 2023-10-16T10:05:37.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.461+0000 7f369a334640 1 -- 172.21.15.84:0/1747787529 learned_addr learned my addr 172.21.15.84:0/1747787529 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:37.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.461+0000 7f3699b33640 1 --2- 172.21.15.84:0/1747787529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369406c040 0x7f369406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:37.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.461+0000 7f3699332640 1 -- 172.21.15.84:0/1747787529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369406c040 msgr2=0x7f369406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:37.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.461+0000 7f3699332640 1 --2- 172.21.15.84:0/1747787529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369406c040 0x7f369406c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:37.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.461+0000 7f3699332640 1 -- 172.21.15.84:0/1747787529 >> v1:172.21.15.84:6789/0 conn(0x7f369415ebc0 legacy=0x7f36940cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:37.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.461+0000 7f3699332640 1 -- 172.21.15.84:0/1747787529 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369415bf90 con 0x7f36940ad970 2023-10-16T10:05:37.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.461+0000 7f3698b31640 1 -- 172.21.15.84:0/1747787529 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172664202 0 0) 0x7f369406c960 con 0x7f369415ebc0 2023-10-16T10:05:37.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.463+0000 7f3699332640 1 --2- 172.21.15.84:0/1747787529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36940ad970 0x7f369415e680 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f36840097d0 tx=0x7f368402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6770a3490a1f4073 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:37.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.463+0000 7f3698b31640 1 -- 172.21.15.84:0/1747787529 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3684031800 con 0x7f36940ad970 2023-10-16T10:05:37.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.463+0000 7f3698b31640 1 -- 172.21.15.84:0/1747787529 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3684031960 con 0x7f36940ad970 2023-10-16T10:05:37.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.463+0000 7f3698b31640 1 -- 172.21.15.84:0/1747787529 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3684031cf0 con 0x7f36940ad970 2023-10-16T10:05:37.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.463+0000 7f369ab35640 1 -- 172.21.15.84:0/1747787529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36940ad970 msgr2=0x7f369415e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:37.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.463+0000 7f369ab35640 1 --2- 172.21.15.84:0/1747787529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36940ad970 0x7f369415e680 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f36840097d0 tx=0x7f368402b8a0 comp rx=0 tx=0).stop 2023-10-16T10:05:37.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.463+0000 7f369ab35640 1 -- 172.21.15.84:0/1747787529 shutdown_connections 2023-10-16T10:05:37.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.463+0000 7f369ab35640 1 --2- 172.21.15.84:0/1747787529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36940ad970 0x7f369415e680 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:37.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.463+0000 7f369ab35640 1 --2- 172.21.15.84:0/1747787529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369406c040 0x7f369406c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:37.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.463+0000 7f369ab35640 1 -- 172.21.15.84:0/1747787529 >> 172.21.15.84:0/1747787529 conn(0x7f36940ce6a0 msgr2=0x7f36940ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:37.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.464+0000 7f369ab35640 1 -- 172.21.15.84:0/1747787529 shutdown_connections 2023-10-16T10:05:37.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.464+0000 7f369ab35640 1 -- 172.21.15.84:0/1747787529 wait complete. 2023-10-16T10:05:37.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.465+0000 7f369ab35640 1 Processor -- start 2023-10-16T10:05:37.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.466+0000 7f369ab35640 1 -- start start 2023-10-16T10:05:37.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.467+0000 7f369ab35640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369406c040 0x7f36941f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:37.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.467+0000 7f369ab35640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f369415ebc0 0x7f36940c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:37.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.467+0000 7f369ab35640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36941f25e0 0x7f36940c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:37.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.467+0000 7f369ab35640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36941477c0 con 0x7f36941f25e0 2023-10-16T10:05:37.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.467+0000 7f369ab35640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36940b0bd0 con 0x7f369415ebc0 2023-10-16T10:05:37.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.467+0000 7f369ab35640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36940b0eb0 con 0x7f369406c040 2023-10-16T10:05:37.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.467+0000 7f369a334640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36941f25e0 0x7f36940c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:37.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.467+0000 7f369a334640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36941f25e0 0x7f36940c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46108/0 (socket says 172.21.15.84:46108) 2023-10-16T10:05:37.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.467+0000 7f369a334640 1 -- 172.21.15.84:0/1003509097 learned_addr learned my addr 172.21.15.84:0/1003509097 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:37.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.467+0000 7f3699b33640 1 --2- 172.21.15.84:0/1003509097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369406c040 0x7f36941f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:37.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.467+0000 7f3699332640 1 --2- 172.21.15.84:0/1003509097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f369415ebc0 0x7f36940c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:37.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.467+0000 7f369a334640 1 -- 172.21.15.84:0/1003509097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369406c040 msgr2=0x7f36941f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:37.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.467+0000 7f369a334640 1 --2- 172.21.15.84:0/1003509097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369406c040 0x7f36941f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:37.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.467+0000 7f369a334640 1 -- 172.21.15.84:0/1003509097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f369415ebc0 msgr2=0x7f36940c67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:37.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.467+0000 7f369a334640 1 --2- 172.21.15.84:0/1003509097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f369415ebc0 0x7f36940c67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:37.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.467+0000 7f369a334640 1 -- 172.21.15.84:0/1003509097 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3684009400 con 0x7f36941f25e0 2023-10-16T10:05:37.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.468+0000 7f369a334640 1 --2- 172.21.15.84:0/1003509097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36941f25e0 0x7f36940c6d00 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f368c00c7e0 tx=0x7f368c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:37.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.468+0000 7f368a7fc640 1 -- 172.21.15.84:0/1003509097 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c00a640 con 0x7f36941f25e0 2023-10-16T10:05:37.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.468+0000 7f368a7fc640 1 -- 172.21.15.84:0/1003509097 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c00a7c0 con 0x7f36941f25e0 2023-10-16T10:05:37.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.468+0000 7f368a7fc640 1 -- 172.21.15.84:0/1003509097 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c00ab70 con 0x7f36941f25e0 2023-10-16T10:05:37.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.468+0000 7f369ab35640 1 -- 172.21.15.84:0/1003509097 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36941479d0 con 0x7f36941f25e0 2023-10-16T10:05:37.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.468+0000 7f369ab35640 1 -- 172.21.15.84:0/1003509097 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36940b0860 con 0x7f36941f25e0 2023-10-16T10:05:37.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.469+0000 7f369ab35640 1 -- 172.21.15.84:0/1003509097 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f369406b330 con 0x7f36941f25e0 2023-10-16T10:05:37.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.470+0000 7f368a7fc640 1 -- 172.21.15.84:0/1003509097 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f368c00acd0 con 0x7f36941f25e0 2023-10-16T10:05:37.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.471+0000 7f368a7fc640 1 --2- 172.21.15.84:0/1003509097 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f367806af00 0x7f367806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:37.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.471+0000 7f368a7fc640 1 -- 172.21.15.84:0/1003509097 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6045+0+0 (secure 0 0 0) 0x7f368c088d40 con 0x7f36941f25e0 2023-10-16T10:05:37.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.471+0000 7f3699b33640 1 --2- 172.21.15.84:0/1003509097 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f367806af00 0x7f367806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:37.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.472+0000 7f3699b33640 1 --2- 172.21.15.84:0/1003509097 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f367806af00 0x7f367806d3c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3690006fd0 tx=0x7f3690008040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:37.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.476+0000 7f368a7fc640 1 -- 172.21.15.84:0/1003509097 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f368c053bb0 con 0x7f36941f25e0 2023-10-16T10:05:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:37.678+0000 7f369ab35640 1 -- 172.21.15.84:0/1003509097 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3694147ef0 con 0x7f36941f25e0 2023-10-16T10:05:38.047 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:38.045+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:38.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.231+0000 7f368a7fc640 1 -- 172.21.15.84:0/1003509097 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3694147ef0 con 0x7f36941f25e0 2023-10-16T10:05:38.233 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-16T10:05:38.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.236+0000 7f369ab35640 1 -- 172.21.15.84:0/1003509097 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f367806af00 msgr2=0x7f367806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:38.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.236+0000 7f369ab35640 1 --2- 172.21.15.84:0/1003509097 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f367806af00 0x7f367806d3c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3690006fd0 tx=0x7f3690008040 comp rx=0 tx=0).stop 2023-10-16T10:05:38.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.236+0000 7f369ab35640 1 -- 172.21.15.84:0/1003509097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36941f25e0 msgr2=0x7f36940c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:38.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.236+0000 7f369ab35640 1 --2- 172.21.15.84:0/1003509097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36941f25e0 0x7f36940c6d00 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f368c00c7e0 tx=0x7f368c00cca0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.237+0000 7f369ab35640 1 -- 172.21.15.84:0/1003509097 shutdown_connections 2023-10-16T10:05:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.237+0000 7f369ab35640 1 --2- 172.21.15.84:0/1003509097 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f367806af00 0x7f367806d3c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.237+0000 7f369ab35640 1 --2- 172.21.15.84:0/1003509097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36941f25e0 0x7f36940c6d00 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.237+0000 7f369ab35640 1 --2- 172.21.15.84:0/1003509097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f369415ebc0 0x7f36940c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.237+0000 7f369ab35640 1 --2- 172.21.15.84:0/1003509097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369406c040 0x7f36941f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.237+0000 7f369ab35640 1 -- 172.21.15.84:0/1003509097 >> 172.21.15.84:0/1003509097 conn(0x7f36940ce6a0 msgr2=0x7f3694069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.238+0000 7f369ab35640 1 -- 172.21.15.84:0/1003509097 shutdown_connections 2023-10-16T10:05:38.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.239+0000 7f369ab35640 1 -- 172.21.15.84:0/1003509097 wait complete. 2023-10-16T10:05:38.267 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T10:05:38.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.462+0000 7fde268f2640 1 Processor -- start 2023-10-16T10:05:38.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.463+0000 7fde268f2640 1 -- start start 2023-10-16T10:05:38.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.464+0000 7fde268f2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fde2015ad70 0x7fde20069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:38.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.464+0000 7fde268f2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fde200caac0 0x7fde2006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:38.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.464+0000 7fde268f2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde200cb030 con 0x7fde200c9ca0 2023-10-16T10:05:38.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.464+0000 7fde268f2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde200c91c0 con 0x7fde2015ad70 2023-10-16T10:05:38.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.464+0000 7fde268f2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde200c94a0 con 0x7fde200caac0 2023-10-16T10:05:38.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.464+0000 7fde1ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fde2015ad70 0x7fde20069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:38.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.465+0000 7fde1ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fde2015ad70 0x7fde20069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46122/0 (socket says 172.21.15.84:46122) 2023-10-16T10:05:38.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.465+0000 7fde1ffff640 1 -- 172.21.15.84:0/1869481051 learned_addr learned my addr 172.21.15.84:0/1869481051 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:38.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.465+0000 7fde24e8f640 1 --2- 172.21.15.84:0/1869481051 >> v2:172.21.15.157:3300/0 conn(0x7fde200caac0 0x7fde2006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:38.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.465+0000 7fde1ffff640 1 -- 172.21.15.84:0/1869481051 >> v2:172.21.15.157:3300/0 conn(0x7fde200caac0 msgr2=0x7fde2006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:38.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.465+0000 7fde1ffff640 1 --2- 172.21.15.84:0/1869481051 >> v2:172.21.15.157:3300/0 conn(0x7fde200caac0 0x7fde2006a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.465+0000 7fde1ffff640 1 -- 172.21.15.84:0/1869481051 >> v1:172.21.15.84:6789/0 conn(0x7fde200c9ca0 legacy=0x7fde200ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:38.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.465+0000 7fde1ffff640 1 -- 172.21.15.84:0/1869481051 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde200c9780 con 0x7fde2015ad70 2023-10-16T10:05:38.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.465+0000 7fde24e8f640 1 --2- 172.21.15.84:0/1869481051 >> v2:172.21.15.157:3300/0 conn(0x7fde200caac0 0x7fde2006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:38.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.466+0000 7fde1ffff640 1 --2- 172.21.15.84:0/1869481051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde2015ad70 0x7fde20069da0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fde2006d090 tx=0x7fde1002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f3bcac1e034315 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:38.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.466+0000 7fde1f7fe640 1 -- 172.21.15.84:0/1869481051 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10030860 con 0x7fde2015ad70 2023-10-16T10:05:38.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.466+0000 7fde1f7fe640 1 -- 172.21.15.84:0/1869481051 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde100309c0 con 0x7fde2015ad70 2023-10-16T10:05:38.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.466+0000 7fde1f7fe640 1 -- 172.21.15.84:0/1869481051 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10030d50 con 0x7fde2015ad70 2023-10-16T10:05:38.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.467+0000 7fde268f2640 1 -- 172.21.15.84:0/1869481051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde2015ad70 msgr2=0x7fde20069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:38.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.467+0000 7fde268f2640 1 --2- 172.21.15.84:0/1869481051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde2015ad70 0x7fde20069da0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fde2006d090 tx=0x7fde1002b9a0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.467+0000 7fde268f2640 1 -- 172.21.15.84:0/1869481051 shutdown_connections 2023-10-16T10:05:38.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.467+0000 7fde268f2640 1 --2- 172.21.15.84:0/1869481051 >> v2:172.21.15.157:3300/0 conn(0x7fde200caac0 0x7fde2006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.467+0000 7fde268f2640 1 --2- 172.21.15.84:0/1869481051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde2015ad70 0x7fde20069da0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.467+0000 7fde268f2640 1 -- 172.21.15.84:0/1869481051 >> 172.21.15.84:0/1869481051 conn(0x7fde200ce9b0 msgr2=0x7fde200cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:38.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.467+0000 7fde268f2640 1 -- 172.21.15.84:0/1869481051 shutdown_connections 2023-10-16T10:05:38.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.467+0000 7fde268f2640 1 -- 172.21.15.84:0/1869481051 wait complete. 2023-10-16T10:05:38.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.469+0000 7fde268f2640 1 Processor -- start 2023-10-16T10:05:38.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.469+0000 7fde268f2640 1 -- start start 2023-10-16T10:05:38.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.470+0000 7fde268f2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde200c9ca0 0x7fde201f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:38.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.470+0000 7fde268f2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde200caac0 0x7fde200ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:38.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.470+0000 7fde268f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2015ad70 0x7fde200acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:38.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.470+0000 7fde268f2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2015b7a0 con 0x7fde200caac0 2023-10-16T10:05:38.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.470+0000 7fde268f2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde201f65b0 con 0x7fde2015ad70 2023-10-16T10:05:38.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.470+0000 7fde268f2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2014cc00 con 0x7fde200c9ca0 2023-10-16T10:05:38.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.471+0000 7fde17fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde200caac0 0x7fde200ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:38.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.471+0000 7fde17fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde200caac0 0x7fde200ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46132/0 (socket says 172.21.15.84:46132) 2023-10-16T10:05:38.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.471+0000 7fde17fff640 1 -- 172.21.15.84:0/2349568308 learned_addr learned my addr 172.21.15.84:0/2349568308 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:38.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.471+0000 7fde17fff640 1 -- 172.21.15.84:0/2349568308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde200c9ca0 msgr2=0x7fde201f6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:38.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.471+0000 7fde17fff640 1 --2- 172.21.15.84:0/2349568308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde200c9ca0 0x7fde201f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.471+0000 7fde17fff640 1 -- 172.21.15.84:0/2349568308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2015ad70 msgr2=0x7fde200acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:38.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.471+0000 7fde17fff640 1 --2- 172.21.15.84:0/2349568308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2015ad70 0x7fde200acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.471+0000 7fde17fff640 1 -- 172.21.15.84:0/2349568308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde100094d0 con 0x7fde200caac0 2023-10-16T10:05:38.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.471+0000 7fde17fff640 1 --2- 172.21.15.84:0/2349568308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde200caac0 0x7fde200ac890 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fde0800c450 tx=0x7fde0800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:38.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.472+0000 7fde1d7fa640 1 -- 172.21.15.84:0/2349568308 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde080052d0 con 0x7fde200caac0 2023-10-16T10:05:38.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.472+0000 7fde1d7fa640 1 -- 172.21.15.84:0/2349568308 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde08005430 con 0x7fde200caac0 2023-10-16T10:05:38.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.472+0000 7fde1d7fa640 1 -- 172.21.15.84:0/2349568308 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08012870 con 0x7fde200caac0 2023-10-16T10:05:38.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.472+0000 7fde268f2640 1 -- 172.21.15.84:0/2349568308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde2014cee0 con 0x7fde200caac0 2023-10-16T10:05:38.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.472+0000 7fde268f2640 1 -- 172.21.15.84:0/2349568308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde2014d4a0 con 0x7fde200caac0 2023-10-16T10:05:38.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.473+0000 7fde268f2640 1 -- 172.21.15.84:0/2349568308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde2004e160 con 0x7fde200caac0 2023-10-16T10:05:38.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.474+0000 7fde1d7fa640 1 -- 172.21.15.84:0/2349568308 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fde080129d0 con 0x7fde200caac0 2023-10-16T10:05:38.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.475+0000 7fde1d7fa640 1 --2- 172.21.15.84:0/2349568308 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fde0006af00 0x7fde0006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:38.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.475+0000 7fde1d7fa640 1 -- 172.21.15.84:0/2349568308 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5631+0+0 (secure 0 0 0) 0x7fde08089190 con 0x7fde200caac0 2023-10-16T10:05:38.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.475+0000 7fde1ffff640 1 --2- 172.21.15.84:0/2349568308 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fde0006af00 0x7fde0006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:38.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.476+0000 7fde1ffff640 1 --2- 172.21.15.84:0/2349568308 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fde0006af00 0x7fde0006d3c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fde10036000 tx=0x7fde1003b040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:38.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.480+0000 7fde1d7fa640 1 -- 172.21.15.84:0/2349568308 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fde08054230 con 0x7fde200caac0 2023-10-16T10:05:38.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.681+0000 7fde268f2640 1 -- 172.21.15.84:0/2349568308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fde200c52d0 con 0x7fde200caac0 2023-10-16T10:05:38.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.687+0000 7fde1d7fa640 1 -- 172.21.15.84:0/2349568308 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7fde08057850 con 0x7fde200caac0 2023-10-16T10:05:38.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.690+0000 7fde268f2640 1 -- 172.21.15.84:0/2349568308 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fde0006af00 msgr2=0x7fde0006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:38.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.690+0000 7fde268f2640 1 --2- 172.21.15.84:0/2349568308 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fde0006af00 0x7fde0006d3c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fde10036000 tx=0x7fde1003b040 comp rx=0 tx=0).stop 2023-10-16T10:05:38.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.690+0000 7fde268f2640 1 -- 172.21.15.84:0/2349568308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde200caac0 msgr2=0x7fde200ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:38.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.690+0000 7fde268f2640 1 --2- 172.21.15.84:0/2349568308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde200caac0 0x7fde200ac890 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fde0800c450 tx=0x7fde0800c910 comp rx=0 tx=0).stop 2023-10-16T10:05:38.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.690+0000 7fde268f2640 1 -- 172.21.15.84:0/2349568308 shutdown_connections 2023-10-16T10:05:38.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.690+0000 7fde268f2640 1 --2- 172.21.15.84:0/2349568308 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fde0006af00 0x7fde0006d3c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.690+0000 7fde268f2640 1 --2- 172.21.15.84:0/2349568308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2015ad70 0x7fde200acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.690+0000 7fde268f2640 1 --2- 172.21.15.84:0/2349568308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde200caac0 0x7fde200ac890 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.690+0000 7fde268f2640 1 --2- 172.21.15.84:0/2349568308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde200c9ca0 0x7fde201f6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.690+0000 7fde268f2640 1 -- 172.21.15.84:0/2349568308 >> 172.21.15.84:0/2349568308 conn(0x7fde200ce9b0 msgr2=0x7fde200afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:38.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.690+0000 7fde268f2640 1 -- 172.21.15.84:0/2349568308 shutdown_connections 2023-10-16T10:05:38.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.690+0000 7fde268f2640 1 -- 172.21.15.84:0/2349568308 wait complete. 2023-10-16T10:05:38.710 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T10:05:38.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.927+0000 7fca46733640 1 Processor -- start 2023-10-16T10:05:38.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.927+0000 7fca46733640 1 -- start start 2023-10-16T10:05:38.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.928+0000 7fca46733640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca40065ab0 0x7fca40161100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:38.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.928+0000 7fca46733640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fca40069fa0 0x7fca4006a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:38.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.928+0000 7fca46733640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca400ac890 con 0x7fca40160530 2023-10-16T10:05:38.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.928+0000 7fca46733640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca400ae800 con 0x7fca40069fa0 2023-10-16T10:05:38.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.928+0000 7fca46733640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca400ae1f0 con 0x7fca40065ab0 2023-10-16T10:05:38.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.929+0000 7fca44cd0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fca40069fa0 0x7fca4006a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:38.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.929+0000 7fca3f7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca40065ab0 0x7fca40161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:38.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.929+0000 7fca44cd0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fca40069fa0 0x7fca4006a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:54016/0 (socket says 172.21.15.84:54016) 2023-10-16T10:05:38.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.929+0000 7fca44cd0640 1 -- 172.21.15.84:0/1093890922 learned_addr learned my addr 172.21.15.84:0/1093890922 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:38.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.930+0000 7fca44cd0640 1 -- 172.21.15.84:0/1093890922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca40065ab0 msgr2=0x7fca40161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:38.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.930+0000 7fca44cd0640 1 --2- 172.21.15.84:0/1093890922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca40065ab0 0x7fca40161100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.930+0000 7fca44cd0640 1 -- 172.21.15.84:0/1093890922 >> v1:172.21.15.157:6789/0 conn(0x7fca40160530 legacy=0x7fca401609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:38.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.930+0000 7fca44cd0640 1 -- 172.21.15.84:0/1093890922 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca400acae0 con 0x7fca40069fa0 2023-10-16T10:05:38.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.930+0000 7fca3f7fe640 1 --2- 172.21.15.84:0/1093890922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca40065ab0 0x7fca40161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:05:38.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.931+0000 7fca44cd0640 1 --2- 172.21.15.84:0/1093890922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca40069fa0 0x7fca4006a400 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fca38009e50 tx=0x7fca3802bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3634dff4b9e3d6b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:38.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.932+0000 7fca3effd640 1 -- 172.21.15.84:0/1093890922 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3802ca80 con 0x7fca40069fa0 2023-10-16T10:05:38.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.932+0000 7fca3effd640 1 -- 172.21.15.84:0/1093890922 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3802cbe0 con 0x7fca40069fa0 2023-10-16T10:05:38.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.932+0000 7fca3effd640 1 -- 172.21.15.84:0/1093890922 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca380343f0 con 0x7fca40069fa0 2023-10-16T10:05:38.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.933+0000 7fca46733640 1 -- 172.21.15.84:0/1093890922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca40069fa0 msgr2=0x7fca4006a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:38.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.933+0000 7fca46733640 1 --2- 172.21.15.84:0/1093890922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca40069fa0 0x7fca4006a400 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fca38009e50 tx=0x7fca3802bee0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.933+0000 7fca46733640 1 -- 172.21.15.84:0/1093890922 shutdown_connections 2023-10-16T10:05:38.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.933+0000 7fca46733640 1 --2- 172.21.15.84:0/1093890922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca40069fa0 0x7fca4006a400 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.933+0000 7fca46733640 1 --2- 172.21.15.84:0/1093890922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca40065ab0 0x7fca40161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.933+0000 7fca46733640 1 -- 172.21.15.84:0/1093890922 >> 172.21.15.84:0/1093890922 conn(0x7fca4006bc10 msgr2=0x7fca4006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:38.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.933+0000 7fca46733640 1 -- 172.21.15.84:0/1093890922 shutdown_connections 2023-10-16T10:05:38.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.933+0000 7fca46733640 1 -- 172.21.15.84:0/1093890922 wait complete. 2023-10-16T10:05:38.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.935+0000 7fca46733640 1 Processor -- start 2023-10-16T10:05:38.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.935+0000 7fca46733640 1 -- start start 2023-10-16T10:05:38.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.936+0000 7fca46733640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca40065ab0 0x7fca401f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:38.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.936+0000 7fca46733640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca40069fa0 0x7fca40157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:38.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.936+0000 7fca46733640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca40160530 0x7fca400c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:38.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.936+0000 7fca46733640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca40157d40 con 0x7fca40069fa0 2023-10-16T10:05:38.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.936+0000 7fca46733640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca400c3b00 con 0x7fca40065ab0 2023-10-16T10:05:38.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.936+0000 7fca46733640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca401f5dc0 con 0x7fca40160530 2023-10-16T10:05:38.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.936+0000 7fca3f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca40069fa0 0x7fca40157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:38.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.936+0000 7fca3f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca40069fa0 0x7fca40157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46140/0 (socket says 172.21.15.84:46140) 2023-10-16T10:05:38.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.936+0000 7fca3f7fe640 1 -- 172.21.15.84:0/1749812672 learned_addr learned my addr 172.21.15.84:0/1749812672 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:38.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.937+0000 7fca44cd0640 1 --2- 172.21.15.84:0/1749812672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca40160530 0x7fca400c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:38.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.937+0000 7fca3ffff640 1 --2- 172.21.15.84:0/1749812672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca40065ab0 0x7fca401f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:38.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.937+0000 7fca3f7fe640 1 -- 172.21.15.84:0/1749812672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca40160530 msgr2=0x7fca400c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:38.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.937+0000 7fca3f7fe640 1 --2- 172.21.15.84:0/1749812672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca40160530 0x7fca400c2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.937+0000 7fca3f7fe640 1 -- 172.21.15.84:0/1749812672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca40065ab0 msgr2=0x7fca401f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:38.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.937+0000 7fca3f7fe640 1 --2- 172.21.15.84:0/1749812672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca40065ab0 0x7fca401f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:38.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.937+0000 7fca3f7fe640 1 -- 172.21.15.84:0/1749812672 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca34009400 con 0x7fca40069fa0 2023-10-16T10:05:38.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.937+0000 7fca3f7fe640 1 --2- 172.21.15.84:0/1749812672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca40069fa0 0x7fca40157800 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fca3400e960 tx=0x7fca3400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:38.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.937+0000 7fca3cff9640 1 -- 172.21.15.84:0/1749812672 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3400ca00 con 0x7fca40069fa0 2023-10-16T10:05:38.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.937+0000 7fca3cff9640 1 -- 172.21.15.84:0/1749812672 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3400cb60 con 0x7fca40069fa0 2023-10-16T10:05:38.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.937+0000 7fca46733640 1 -- 172.21.15.84:0/1749812672 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca38009b30 con 0x7fca40069fa0 2023-10-16T10:05:38.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.938+0000 7fca3cff9640 1 -- 172.21.15.84:0/1749812672 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34013470 con 0x7fca40069fa0 2023-10-16T10:05:38.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.938+0000 7fca46733640 1 -- 172.21.15.84:0/1749812672 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca401f6380 con 0x7fca40069fa0 2023-10-16T10:05:38.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.939+0000 7fca46733640 1 -- 172.21.15.84:0/1749812672 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4004e160 con 0x7fca40069fa0 2023-10-16T10:05:38.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.940+0000 7fca3cff9640 1 -- 172.21.15.84:0/1749812672 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fca340135d0 con 0x7fca40069fa0 2023-10-16T10:05:38.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.941+0000 7fca3cff9640 1 --2- 172.21.15.84:0/1749812672 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fca2c06af30 0x7fca2c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:38.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.941+0000 7fca3cff9640 1 -- 172.21.15.84:0/1749812672 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5631+0+0 (secure 0 0 0) 0x7fca340898d0 con 0x7fca40069fa0 2023-10-16T10:05:38.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.941+0000 7fca3ffff640 1 --2- 172.21.15.84:0/1749812672 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fca2c06af30 0x7fca2c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:38.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.942+0000 7fca3ffff640 1 --2- 172.21.15.84:0/1749812672 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fca2c06af30 0x7fca2c06d3f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fca30001620 tx=0x7fca30009040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:38.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:38.946+0000 7fca3cff9640 1 -- 172.21.15.84:0/1749812672 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fca34054970 con 0x7fca40069fa0 2023-10-16T10:05:39.002 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:38.999+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:39.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.147+0000 7fca46733640 1 -- 172.21.15.84:0/1749812672 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fca400ae460 con 0x7fca40069fa0 2023-10-16T10:05:39.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.154+0000 7fca3cff9640 1 -- 172.21.15.84:0/1749812672 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v32) v1 ==== 120+0+0 (secure 0 0 0) 0x7fca34057f90 con 0x7fca40069fa0 2023-10-16T10:05:39.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.158+0000 7fca46733640 1 -- 172.21.15.84:0/1749812672 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fca2c06af30 msgr2=0x7fca2c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:39.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.158+0000 7fca46733640 1 --2- 172.21.15.84:0/1749812672 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fca2c06af30 0x7fca2c06d3f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fca30001620 tx=0x7fca30009040 comp rx=0 tx=0).stop 2023-10-16T10:05:39.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.158+0000 7fca46733640 1 -- 172.21.15.84:0/1749812672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca40069fa0 msgr2=0x7fca40157800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:39.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.158+0000 7fca46733640 1 --2- 172.21.15.84:0/1749812672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca40069fa0 0x7fca40157800 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fca3400e960 tx=0x7fca3400ee20 comp rx=0 tx=0).stop 2023-10-16T10:05:39.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.158+0000 7fca46733640 1 -- 172.21.15.84:0/1749812672 shutdown_connections 2023-10-16T10:05:39.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.158+0000 7fca46733640 1 --2- 172.21.15.84:0/1749812672 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fca2c06af30 0x7fca2c06d3f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:39.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.158+0000 7fca46733640 1 --2- 172.21.15.84:0/1749812672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca40160530 0x7fca400c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:39.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.158+0000 7fca46733640 1 --2- 172.21.15.84:0/1749812672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca40069fa0 0x7fca40157800 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:39.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.158+0000 7fca46733640 1 --2- 172.21.15.84:0/1749812672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca40065ab0 0x7fca401f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:39.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.158+0000 7fca46733640 1 -- 172.21.15.84:0/1749812672 >> 172.21.15.84:0/1749812672 conn(0x7fca4006bc10 msgr2=0x7fca400aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:39.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.158+0000 7fca46733640 1 -- 172.21.15.84:0/1749812672 shutdown_connections 2023-10-16T10:05:39.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.158+0000 7fca46733640 1 -- 172.21.15.84:0/1749812672 wait complete. 2023-10-16T10:05:39.179 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T10:05:39.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.396+0000 7fe8a6490640 1 Processor -- start 2023-10-16T10:05:39.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.396+0000 7fe8a6490640 1 -- start start 2023-10-16T10:05:39.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.397+0000 7fe8a6490640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe8a00b0ea0 0x7fe8a015a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:39.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.397+0000 7fe8a6490640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8a00cd970 0x7fe8a015a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:39.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.397+0000 7fe8a6490640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a006be30 con 0x7fe8a00b3ec0 2023-10-16T10:05:39.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.397+0000 7fe8a6490640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a006ca00 con 0x7fe8a00b0ea0 2023-10-16T10:05:39.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.397+0000 7fe8a6490640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a00ada10 con 0x7fe8a00cd970 2023-10-16T10:05:39.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.398+0000 7fe8a4c8d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe8a00b0ea0 0x7fe8a015a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:39.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.398+0000 7fe8a5c8f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8a00cd970 0x7fe8a015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:39.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.398+0000 7fe8a4c8d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe8a00b0ea0 0x7fe8a015a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:54040/0 (socket says 172.21.15.84:54040) 2023-10-16T10:05:39.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.398+0000 7fe8a4c8d640 1 -- 172.21.15.84:0/1461178288 learned_addr learned my addr 172.21.15.84:0/1461178288 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:39.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.399+0000 7fe8a5c8f640 1 -- 172.21.15.84:0/1461178288 >> v2:172.21.15.157:3300/0 conn(0x7fe8a00b0ea0 msgr2=0x7fe8a015a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:39.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.399+0000 7fe8a5c8f640 1 --2- 172.21.15.84:0/1461178288 >> v2:172.21.15.157:3300/0 conn(0x7fe8a00b0ea0 0x7fe8a015a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:39.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.399+0000 7fe8a5c8f640 1 -- 172.21.15.84:0/1461178288 >> v1:172.21.15.157:6789/0 conn(0x7fe8a00b3ec0 legacy=0x7fe8a00cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:39.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.399+0000 7fe8a5c8f640 1 -- 172.21.15.84:0/1461178288 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a006c060 con 0x7fe8a00cd970 2023-10-16T10:05:39.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.399+0000 7fe8a4c8d640 1 --2- 172.21.15.84:0/1461178288 >> v2:172.21.15.157:3300/0 conn(0x7fe8a00b0ea0 0x7fe8a015a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:05:39.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.400+0000 7fe8a5c8f640 1 --2- 172.21.15.84:0/1461178288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8a00cd970 0x7fe8a015a830 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe898009d90 tx=0x7fe89802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58c0310d80a97809 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:39.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.401+0000 7fe897fff640 1 -- 172.21.15.84:0/1461178288 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89802cbc0 con 0x7fe8a00cd970 2023-10-16T10:05:39.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.401+0000 7fe897fff640 1 -- 172.21.15.84:0/1461178288 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89802cd20 con 0x7fe8a00cd970 2023-10-16T10:05:39.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.401+0000 7fe897fff640 1 -- 172.21.15.84:0/1461178288 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898034660 con 0x7fe8a00cd970 2023-10-16T10:05:39.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.401+0000 7fe8a6490640 1 -- 172.21.15.84:0/1461178288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8a00cd970 msgr2=0x7fe8a015a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:39.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.401+0000 7fe8a6490640 1 --2- 172.21.15.84:0/1461178288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8a00cd970 0x7fe8a015a830 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe898009d90 tx=0x7fe89802be10 comp rx=0 tx=0).stop 2023-10-16T10:05:39.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.402+0000 7fe8a6490640 1 -- 172.21.15.84:0/1461178288 shutdown_connections 2023-10-16T10:05:39.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.402+0000 7fe8a6490640 1 --2- 172.21.15.84:0/1461178288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8a00cd970 0x7fe8a015a830 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:39.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.402+0000 7fe8a6490640 1 --2- 172.21.15.84:0/1461178288 >> v2:172.21.15.157:3300/0 conn(0x7fe8a00b0ea0 0x7fe8a015a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:39.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.402+0000 7fe8a6490640 1 -- 172.21.15.84:0/1461178288 >> 172.21.15.84:0/1461178288 conn(0x7fe8a00b0170 msgr2=0x7fe8a00b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:39.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.402+0000 7fe8a6490640 1 -- 172.21.15.84:0/1461178288 shutdown_connections 2023-10-16T10:05:39.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.402+0000 7fe8a6490640 1 -- 172.21.15.84:0/1461178288 wait complete. 2023-10-16T10:05:39.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.403+0000 7fe8a6490640 1 Processor -- start 2023-10-16T10:05:39.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.403+0000 7fe8a6490640 1 -- start start 2023-10-16T10:05:39.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.404+0000 7fe8a6490640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8a00b0ea0 0x7fe8a00c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:39.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.404+0000 7fe8a6490640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8a00b3ec0 0x7fe8a00c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:39.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.404+0000 7fe8a6490640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8a00cd970 0x7fe8a0153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:39.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.404+0000 7fe8a6490640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a00c7b20 con 0x7fe8a00cd970 2023-10-16T10:05:39.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.404+0000 7fe8a6490640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a00c62f0 con 0x7fe8a00b0ea0 2023-10-16T10:05:39.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.404+0000 7fe8a6490640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a01543d0 con 0x7fe8a00b3ec0 2023-10-16T10:05:39.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.405+0000 7fe8a5c8f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8a00cd970 0x7fe8a0153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:39.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.405+0000 7fe8a5c8f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8a00cd970 0x7fe8a0153e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46142/0 (socket says 172.21.15.84:46142) 2023-10-16T10:05:39.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.405+0000 7fe8a5c8f640 1 -- 172.21.15.84:0/1239992301 learned_addr learned my addr 172.21.15.84:0/1239992301 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:39.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.405+0000 7fe8a4c8d640 1 --2- 172.21.15.84:0/1239992301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8a00b3ec0 0x7fe8a00c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:39.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.405+0000 7fe8a548e640 1 --2- 172.21.15.84:0/1239992301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8a00b0ea0 0x7fe8a00c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:39.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.405+0000 7fe8a5c8f640 1 -- 172.21.15.84:0/1239992301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8a00b3ec0 msgr2=0x7fe8a00c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:39.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.405+0000 7fe8a5c8f640 1 --2- 172.21.15.84:0/1239992301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8a00b3ec0 0x7fe8a00c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:39.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.405+0000 7fe8a5c8f640 1 -- 172.21.15.84:0/1239992301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8a00b0ea0 msgr2=0x7fe8a00c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:39.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.405+0000 7fe8a5c8f640 1 --2- 172.21.15.84:0/1239992301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8a00b0ea0 0x7fe8a00c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:39.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.405+0000 7fe8a5c8f640 1 -- 172.21.15.84:0/1239992301 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8900094d0 con 0x7fe8a00cd970 2023-10-16T10:05:39.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.405+0000 7fe8a548e640 1 --2- 172.21.15.84:0/1239992301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8a00b0ea0 0x7fe8a00c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:39.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.405+0000 7fe8a5c8f640 1 --2- 172.21.15.84:0/1239992301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8a00cd970 0x7fe8a0153e90 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fe898009d90 tx=0x7fe898004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:39.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.406+0000 7fe895ffb640 1 -- 172.21.15.84:0/1239992301 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898040650 con 0x7fe8a00cd970 2023-10-16T10:05:39.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.406+0000 7fe895ffb640 1 -- 172.21.15.84:0/1239992301 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89802ce90 con 0x7fe8a00cd970 2023-10-16T10:05:39.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.406+0000 7fe8a6490640 1 -- 172.21.15.84:0/1239992301 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe898009a70 con 0x7fe8a00cd970 2023-10-16T10:05:39.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.406+0000 7fe895ffb640 1 -- 172.21.15.84:0/1239992301 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8980409f0 con 0x7fe8a00cd970 2023-10-16T10:05:39.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.406+0000 7fe8a6490640 1 -- 172.21.15.84:0/1239992301 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a00c6500 con 0x7fe8a00cd970 2023-10-16T10:05:39.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.407+0000 7fe8a6490640 1 -- 172.21.15.84:0/1239992301 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a004e9b0 con 0x7fe8a00cd970 2023-10-16T10:05:39.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.408+0000 7fe895ffb640 1 -- 172.21.15.84:0/1239992301 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fe898040b50 con 0x7fe8a00cd970 2023-10-16T10:05:39.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.409+0000 7fe895ffb640 1 --2- 172.21.15.84:0/1239992301 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe88406af30 0x7fe88406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:39.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.409+0000 7fe895ffb640 1 -- 172.21.15.84:0/1239992301 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5631+0+0 (secure 0 0 0) 0x7fe8980ac200 con 0x7fe8a00cd970 2023-10-16T10:05:39.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.410+0000 7fe8a548e640 1 --2- 172.21.15.84:0/1239992301 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe88406af30 0x7fe88406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:39.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.410+0000 7fe8a548e640 1 --2- 172.21.15.84:0/1239992301 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe88406af30 0x7fe88406d3f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe89c000f60 tx=0x7fe89c009040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:39.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.414+0000 7fe895ffb640 1 -- 172.21.15.84:0/1239992301 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe898077430 con 0x7fe8a00cd970 2023-10-16T10:05:39.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.614+0000 7fe8a6490640 1 -- 172.21.15.84:0/1239992301 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fe8a006abd0 con 0x7fe8a00cd970 2023-10-16T10:05:39.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.620+0000 7fe895ffb640 1 -- 172.21.15.84:0/1239992301 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v33) v1 ==== 115+0+0 (secure 0 0 0) 0x7fe898009450 con 0x7fe8a00cd970 2023-10-16T10:05:39.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.623+0000 7fe8a6490640 1 -- 172.21.15.84:0/1239992301 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe88406af30 msgr2=0x7fe88406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:39.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.623+0000 7fe8a6490640 1 --2- 172.21.15.84:0/1239992301 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe88406af30 0x7fe88406d3f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe89c000f60 tx=0x7fe89c009040 comp rx=0 tx=0).stop 2023-10-16T10:05:39.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.623+0000 7fe8a6490640 1 -- 172.21.15.84:0/1239992301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8a00cd970 msgr2=0x7fe8a0153e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:39.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.623+0000 7fe8a6490640 1 --2- 172.21.15.84:0/1239992301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8a00cd970 0x7fe8a0153e90 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fe898009d90 tx=0x7fe898004a40 comp rx=0 tx=0).stop 2023-10-16T10:05:39.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.623+0000 7fe8a6490640 1 -- 172.21.15.84:0/1239992301 shutdown_connections 2023-10-16T10:05:39.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.623+0000 7fe8a6490640 1 --2- 172.21.15.84:0/1239992301 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe88406af30 0x7fe88406d3f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:39.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.623+0000 7fe8a6490640 1 --2- 172.21.15.84:0/1239992301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8a00cd970 0x7fe8a0153e90 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:39.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.623+0000 7fe8a6490640 1 --2- 172.21.15.84:0/1239992301 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8a00b3ec0 0x7fe8a00c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:39.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.623+0000 7fe8a6490640 1 --2- 172.21.15.84:0/1239992301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8a00b0ea0 0x7fe8a00c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:39.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.623+0000 7fe8a6490640 1 -- 172.21.15.84:0/1239992301 >> 172.21.15.84:0/1239992301 conn(0x7fe8a00b0170 msgr2=0x7fe8a00ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:39.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.624+0000 7fe8a6490640 1 -- 172.21.15.84:0/1239992301 shutdown_connections 2023-10-16T10:05:39.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.624+0000 7fe8a6490640 1 -- 172.21.15.84:0/1239992301 wait complete. 2023-10-16T10:05:39.646 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-10-16T10:05:39.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.849+0000 7f6a9cd1d640 1 Processor -- start 2023-10-16T10:05:39.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.849+0000 7f6a9cd1d640 1 -- start start 2023-10-16T10:05:39.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.851+0000 7f6a9cd1d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6a980b0b20 0x7f6a981615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:39.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.851+0000 7f6a9cd1d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6a9815d3f0 0x7f6a9815d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:39.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.851+0000 7f6a9cd1d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a9815e3e0 con 0x7f6a980cd0c0 2023-10-16T10:05:39.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.851+0000 7f6a9cd1d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a980cd4a0 con 0x7f6a980b0b20 2023-10-16T10:05:39.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.851+0000 7f6a9cd1d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a9815e100 con 0x7f6a9815d3f0 2023-10-16T10:05:39.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.851+0000 7f6a8ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6a980b0b20 0x7f6a981615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:39.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.851+0000 7f6a8ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6a980b0b20 0x7f6a981615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46148/0 (socket says 172.21.15.84:46148) 2023-10-16T10:05:39.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.851+0000 7f6a8ffff640 1 -- 172.21.15.84:0/3621853574 learned_addr learned my addr 172.21.15.84:0/3621853574 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:39.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.851+0000 7f6a96d9d640 1 --2- 172.21.15.84:0/3621853574 >> v2:172.21.15.157:3300/0 conn(0x7f6a9815d3f0 0x7f6a9815d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:39.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.851+0000 7f6a8ffff640 1 -- 172.21.15.84:0/3621853574 >> v2:172.21.15.157:3300/0 conn(0x7f6a9815d3f0 msgr2=0x7f6a9815d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:39.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.851+0000 7f6a8ffff640 1 --2- 172.21.15.84:0/3621853574 >> v2:172.21.15.157:3300/0 conn(0x7f6a9815d3f0 0x7f6a9815d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:39.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.851+0000 7f6a8ffff640 1 -- 172.21.15.84:0/3621853574 >> v1:172.21.15.157:6789/0 conn(0x7f6a980cd0c0 legacy=0x7f6a980b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:39.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.852+0000 7f6a8ffff640 1 -- 172.21.15.84:0/3621853574 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a98160b10 con 0x7f6a980b0b20 2023-10-16T10:05:39.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.853+0000 7f6a8ffff640 1 --2- 172.21.15.84:0/3621853574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a980b0b20 0x7f6a981615f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f6a9806da60 tx=0x7f6a8002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82231ea816d92598 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:39.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.853+0000 7f6a95d9b640 1 -- 172.21.15.84:0/3621853574 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a800317e0 con 0x7f6a980b0b20 2023-10-16T10:05:39.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.853+0000 7f6a95d9b640 1 -- 172.21.15.84:0/3621853574 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a80031940 con 0x7f6a980b0b20 2023-10-16T10:05:39.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.853+0000 7f6a95d9b640 1 -- 172.21.15.84:0/3621853574 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a80031cd0 con 0x7f6a980b0b20 2023-10-16T10:05:39.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.854+0000 7f6a9cd1d640 1 -- 172.21.15.84:0/3621853574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a980b0b20 msgr2=0x7f6a981615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:39.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.854+0000 7f6a9cd1d640 1 --2- 172.21.15.84:0/3621853574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a980b0b20 0x7f6a981615f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f6a9806da60 tx=0x7f6a8002b880 comp rx=0 tx=0).stop 2023-10-16T10:05:39.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.854+0000 7f6a9cd1d640 1 -- 172.21.15.84:0/3621853574 shutdown_connections 2023-10-16T10:05:39.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.854+0000 7f6a9cd1d640 1 --2- 172.21.15.84:0/3621853574 >> v2:172.21.15.157:3300/0 conn(0x7f6a9815d3f0 0x7f6a9815d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:39.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.854+0000 7f6a9cd1d640 1 --2- 172.21.15.84:0/3621853574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a980b0b20 0x7f6a981615f0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:39.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.854+0000 7f6a9cd1d640 1 -- 172.21.15.84:0/3621853574 >> 172.21.15.84:0/3621853574 conn(0x7f6a980cb410 msgr2=0x7f6a980cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:39.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.854+0000 7f6a9cd1d640 1 -- 172.21.15.84:0/3621853574 shutdown_connections 2023-10-16T10:05:39.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.854+0000 7f6a9cd1d640 1 -- 172.21.15.84:0/3621853574 wait complete. 2023-10-16T10:05:39.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.856+0000 7f6a9cd1d640 1 Processor -- start 2023-10-16T10:05:39.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.856+0000 7f6a9cd1d640 1 -- start start 2023-10-16T10:05:39.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.857+0000 7f6a9cd1d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a980b0b20 0x7f6a980ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:39.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.857+0000 7f6a9cd1d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a980cd0c0 0x7f6a9814c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:39.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.857+0000 7f6a9cd1d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a9815d3f0 0x7f6a980c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:39.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.857+0000 7f6a9cd1d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a980c4f80 con 0x7f6a9815d3f0 2023-10-16T10:05:39.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.857+0000 7f6a9cd1d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a980af780 con 0x7f6a980b0b20 2023-10-16T10:05:39.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.857+0000 7f6a9cd1d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a980ae8f0 con 0x7f6a980cd0c0 2023-10-16T10:05:39.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.857+0000 7f6a96d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a9815d3f0 0x7f6a980c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:39.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.857+0000 7f6a96d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a9815d3f0 0x7f6a980c5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46164/0 (socket says 172.21.15.84:46164) 2023-10-16T10:05:39.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.857+0000 7f6a96d9d640 1 -- 172.21.15.84:0/1365407612 learned_addr learned my addr 172.21.15.84:0/1365407612 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:39.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.857+0000 7f6a9659c640 1 --2- 172.21.15.84:0/1365407612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a980b0b20 0x7f6a980ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:39.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.857+0000 7f6a8ffff640 1 --2- 172.21.15.84:0/1365407612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a980cd0c0 0x7f6a9814c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:39.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.857+0000 7f6a96d9d640 1 -- 172.21.15.84:0/1365407612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a980cd0c0 msgr2=0x7f6a9814c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:39.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.857+0000 7f6a96d9d640 1 --2- 172.21.15.84:0/1365407612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a980cd0c0 0x7f6a9814c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:39.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.857+0000 7f6a96d9d640 1 -- 172.21.15.84:0/1365407612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a980b0b20 msgr2=0x7f6a980ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:39.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.857+0000 7f6a96d9d640 1 --2- 172.21.15.84:0/1365407612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a980b0b20 0x7f6a980ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:39.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.858+0000 7f6a96d9d640 1 -- 172.21.15.84:0/1365407612 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a800094d0 con 0x7f6a9815d3f0 2023-10-16T10:05:39.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.858+0000 7f6a8ffff640 1 --2- 172.21.15.84:0/1365407612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a980cd0c0 0x7f6a9814c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:39.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.858+0000 7f6a96d9d640 1 --2- 172.21.15.84:0/1365407612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a9815d3f0 0x7f6a980c5a60 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000cd70 tx=0x7f6a9000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:39.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.858+0000 7f6a8effd640 1 -- 172.21.15.84:0/1365407612 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90003b90 con 0x7f6a9815d3f0 2023-10-16T10:05:39.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.858+0000 7f6a8effd640 1 -- 172.21.15.84:0/1365407612 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a90003cf0 con 0x7f6a9815d3f0 2023-10-16T10:05:39.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.858+0000 7f6a8effd640 1 -- 172.21.15.84:0/1365407612 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90004240 con 0x7f6a9815d3f0 2023-10-16T10:05:39.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.858+0000 7f6a9cd1d640 1 -- 172.21.15.84:0/1365407612 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a980aeb70 con 0x7f6a9815d3f0 2023-10-16T10:05:39.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.858+0000 7f6a9cd1d640 1 -- 172.21.15.84:0/1365407612 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a980af130 con 0x7f6a9815d3f0 2023-10-16T10:05:39.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.860+0000 7f6a9cd1d640 1 -- 172.21.15.84:0/1365407612 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9804e160 con 0x7f6a9815d3f0 2023-10-16T10:05:39.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.860+0000 7f6a8effd640 1 -- 172.21.15.84:0/1365407612 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f6a900043a0 con 0x7f6a9815d3f0 2023-10-16T10:05:39.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.862+0000 7f6a8effd640 1 --2- 172.21.15.84:0/1365407612 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f6a7806af00 0x7f6a7806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:39.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.862+0000 7f6a8effd640 1 -- 172.21.15.84:0/1365407612 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f6a90012070 con 0x7f6a9815d3f0 2023-10-16T10:05:39.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.862+0000 7f6a9659c640 1 --2- 172.21.15.84:0/1365407612 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f6a7806af00 0x7f6a7806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:39.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.863+0000 7f6a9659c640 1 --2- 172.21.15.84:0/1365407612 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f6a7806af00 0x7f6a7806d3c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6a88006fd0 tx=0x7f6a88008040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:39.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:39.866+0000 7f6a8effd640 1 -- 172.21.15.84:0/1365407612 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6a90055130 con 0x7f6a9815d3f0 2023-10-16T10:05:39.916 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:39.913+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:40.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.140+0000 7f6a9cd1d640 1 -- 172.21.15.84:0/1365407612 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f6a980b4910 con 0x7f6a9815d3f0 2023-10-16T10:05:40.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.244+0000 7f6a8effd640 1 -- 172.21.15.84:0/1365407612 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v625) v1 ==== 166+0+0 (secure 0 0 0) 0x7f6a90058750 con 0x7f6a9815d3f0 2023-10-16T10:05:40.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.247+0000 7f6a9cd1d640 1 -- 172.21.15.84:0/1365407612 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f6a7806af00 msgr2=0x7f6a7806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:40.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.248+0000 7f6a9cd1d640 1 --2- 172.21.15.84:0/1365407612 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f6a7806af00 0x7f6a7806d3c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6a88006fd0 tx=0x7f6a88008040 comp rx=0 tx=0).stop 2023-10-16T10:05:40.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.248+0000 7f6a9cd1d640 1 -- 172.21.15.84:0/1365407612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a9815d3f0 msgr2=0x7f6a980c5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:40.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.248+0000 7f6a9cd1d640 1 --2- 172.21.15.84:0/1365407612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a9815d3f0 0x7f6a980c5a60 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000cd70 tx=0x7f6a9000a660 comp rx=0 tx=0).stop 2023-10-16T10:05:40.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.248+0000 7f6a9cd1d640 1 -- 172.21.15.84:0/1365407612 shutdown_connections 2023-10-16T10:05:40.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.248+0000 7f6a9cd1d640 1 --2- 172.21.15.84:0/1365407612 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f6a7806af00 0x7f6a7806d3c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.248+0000 7f6a9cd1d640 1 --2- 172.21.15.84:0/1365407612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a9815d3f0 0x7f6a980c5a60 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.248+0000 7f6a9cd1d640 1 --2- 172.21.15.84:0/1365407612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a980cd0c0 0x7f6a9814c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.248+0000 7f6a9cd1d640 1 --2- 172.21.15.84:0/1365407612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a980b0b20 0x7f6a980ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.248+0000 7f6a9cd1d640 1 -- 172.21.15.84:0/1365407612 >> 172.21.15.84:0/1365407612 conn(0x7f6a980cb410 msgr2=0x7f6a980ca6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:40.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.248+0000 7f6a9cd1d640 1 -- 172.21.15.84:0/1365407612 shutdown_connections 2023-10-16T10:05:40.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.248+0000 7f6a9cd1d640 1 -- 172.21.15.84:0/1365407612 wait complete. 2023-10-16T10:05:40.270 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2023-10-16T10:05:40.270 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that a non-existent path leads to schedule deactivation after a few retries. 2023-10-16T10:05:40.271 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T10:05:40.271 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:05:40.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.486+0000 7fe9f3fd5640 1 Processor -- start 2023-10-16T10:05:40.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.487+0000 7fe9f3fd5640 1 -- start start 2023-10-16T10:05:40.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.488+0000 7fe9f3fd5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ec15aaa0 0x7fe9ec15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:40.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.488+0000 7fe9f3fd5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9ec15ae80 0x7fe9ec0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:40.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.488+0000 7fe9f3fd5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9ec15ed60 con 0x7fe9ec0ccaa0 2023-10-16T10:05:40.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.488+0000 7fe9f3fd5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9ec06b810 con 0x7fe9ec15ae80 2023-10-16T10:05:40.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.488+0000 7fe9f3fd5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec0707e0 con 0x7fe9ec15aaa0 2023-10-16T10:05:40.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.488+0000 7fe9f2572640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe9ec0ccaa0 0x7fe9ec070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39934/0 (socket says 172.21.15.84:39934) 2023-10-16T10:05:40.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.488+0000 7fe9f2572640 1 -- 172.21.15.84:0/3552937343 learned_addr learned my addr 172.21.15.84:0/3552937343 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:40.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f1d71640 1 --2- 172.21.15.84:0/3552937343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ec15aaa0 0x7fe9ec15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:40.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f0d6f640 1 -- 172.21.15.84:0/3552937343 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810678123 0 0) 0x7fe9ec15ed60 con 0x7fe9ec0ccaa0 2023-10-16T10:05:40.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f1570640 1 --2- 172.21.15.84:0/3552937343 >> v2:172.21.15.157:3300/0 conn(0x7fe9ec15ae80 0x7fe9ec0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:40.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f0d6f640 1 -- 172.21.15.84:0/3552937343 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9d8003520 con 0x7fe9ec0ccaa0 2023-10-16T10:05:40.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f0d6f640 1 -- 172.21.15.84:0/3552937343 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3630470526 0 0) 0x7fe9d8003520 con 0x7fe9ec0ccaa0 2023-10-16T10:05:40.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f0d6f640 1 -- 172.21.15.84:0/3552937343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ec15aaa0 msgr2=0x7fe9ec15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:40.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f0d6f640 1 --2- 172.21.15.84:0/3552937343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ec15aaa0 0x7fe9ec15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f0d6f640 1 -- 172.21.15.84:0/3552937343 >> v2:172.21.15.157:3300/0 conn(0x7fe9ec15ae80 msgr2=0x7fe9ec0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:40.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f0d6f640 1 --2- 172.21.15.84:0/3552937343 >> v2:172.21.15.157:3300/0 conn(0x7fe9ec15ae80 0x7fe9ec0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f0d6f640 1 -- 172.21.15.84:0/3552937343 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ec15ef90 con 0x7fe9ec0ccaa0 2023-10-16T10:05:40.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f0d6f640 1 -- 172.21.15.84:0/3552937343 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fe9e40019f0 con 0x7fe9ec0ccaa0 2023-10-16T10:05:40.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f0d6f640 1 -- 172.21.15.84:0/3552937343 >> v1:172.21.15.84:6789/0 conn(0x7fe9ec0ccaa0 legacy=0x7fe9ec070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:40.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f0d6f640 1 --2- 172.21.15.84:0/3552937343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d8003970 0x7fe9d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:40.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f0d6f640 1 --2- 172.21.15.84:0/3552937343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9d80047c0 0x7fe9d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:40.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f0d6f640 1 --2- 172.21.15.84:0/3552937343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9d8005180 0x7fe9d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:40.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f0d6f640 1 -- 172.21.15.84:0/3552937343 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec06b810 con 0x7fe9d8003970 2023-10-16T10:05:40.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f0d6f640 1 -- 172.21.15.84:0/3552937343 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec0707e0 con 0x7fe9d80047c0 2023-10-16T10:05:40.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f0d6f640 1 -- 172.21.15.84:0/3552937343 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d800bf70 con 0x7fe9d8005180 2023-10-16T10:05:40.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.489+0000 7fe9f1d71640 1 --2- 172.21.15.84:0/3552937343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ec15aaa0 0x7fe9ec15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:05:40.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.490+0000 7fe9f1d71640 1 --2- 172.21.15.84:0/3552937343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d8003970 0x7fe9d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:40.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.490+0000 7fe9f1570640 1 --2- 172.21.15.84:0/3552937343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9d80047c0 0x7fe9d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:40.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.490+0000 7fe9f2572640 1 --2- 172.21.15.84:0/3552937343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9d8005180 0x7fe9d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:40.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.490+0000 7fe9f1d71640 1 -- 172.21.15.84:0/3552937343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9d8005180 msgr2=0x7fe9d800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:40.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.490+0000 7fe9f1d71640 1 --2- 172.21.15.84:0/3552937343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9d8005180 0x7fe9d800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.490+0000 7fe9f1d71640 1 -- 172.21.15.84:0/3552937343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9d80047c0 msgr2=0x7fe9d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:40.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.490+0000 7fe9f1d71640 1 --2- 172.21.15.84:0/3552937343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9d80047c0 0x7fe9d8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.490+0000 7fe9f1d71640 1 -- 172.21.15.84:0/3552937343 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d8003520 con 0x7fe9d8003970 2023-10-16T10:05:40.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.491+0000 7fe9f1d71640 1 --2- 172.21.15.84:0/3552937343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d8003970 0x7fe9d8003d90 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8006fd0 tx=0x7fe9d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36048f6df2ee5b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:40.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.491+0000 7fe9f0d6f640 1 -- 172.21.15.84:0/3552937343 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8033600 con 0x7fe9d8003970 2023-10-16T10:05:40.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.492+0000 7fe9f0d6f640 1 -- 172.21.15.84:0/3552937343 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e802f050 con 0x7fe9d8003970 2023-10-16T10:05:40.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.492+0000 7fe9f0d6f640 1 -- 172.21.15.84:0/3552937343 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8033980 con 0x7fe9d8003970 2023-10-16T10:05:40.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.492+0000 7fe9f3fd5640 1 -- 172.21.15.84:0/3552937343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d8003970 msgr2=0x7fe9d8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:40.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.492+0000 7fe9f3fd5640 1 --2- 172.21.15.84:0/3552937343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d8003970 0x7fe9d8003d90 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8006fd0 tx=0x7fe9d80057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.492+0000 7fe9f2572640 1 -- 172.21.15.84:0/3552937343 reap_dead start 2023-10-16T10:05:40.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.492+0000 7fe9f3fd5640 1 -- 172.21.15.84:0/3552937343 shutdown_connections 2023-10-16T10:05:40.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.492+0000 7fe9f3fd5640 1 -- 172.21.15.84:0/3552937343 >> 172.21.15.84:0/3552937343 conn(0x7fe9ec0cb410 msgr2=0x7fe9ec0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:40.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.492+0000 7fe9f3fd5640 1 -- 172.21.15.84:0/3552937343 shutdown_connections 2023-10-16T10:05:40.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.492+0000 7fe9f3fd5640 1 -- 172.21.15.84:0/3552937343 wait complete. 2023-10-16T10:05:40.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.494+0000 7fe9f3fd5640 1 Processor -- start 2023-10-16T10:05:40.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.494+0000 7fe9f3fd5640 1 -- start start 2023-10-16T10:05:40.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.495+0000 7fe9f3fd5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9d8003970 0x7fe9ec1f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:40.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.495+0000 7fe9f3fd5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec159cb0 0x7fe9ec0c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:40.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.495+0000 7fe9f3fd5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9ec0c4bb0 0x7fe9ec1f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:40.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.495+0000 7fe9f3fd5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec0c5030 con 0x7fe9ec0c4bb0 2023-10-16T10:05:40.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.495+0000 7fe9f3fd5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec1f3900 con 0x7fe9ec159cb0 2023-10-16T10:05:40.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.495+0000 7fe9f3fd5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec0c6740 con 0x7fe9d8003970 2023-10-16T10:05:40.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.496+0000 7fe9f2572640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9ec0c4bb0 0x7fe9ec1f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:40.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.496+0000 7fe9f2572640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9ec0c4bb0 0x7fe9ec1f5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46190/0 (socket says 172.21.15.84:46190) 2023-10-16T10:05:40.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.496+0000 7fe9f2572640 1 -- 172.21.15.84:0/3188200906 learned_addr learned my addr 172.21.15.84:0/3188200906 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:40.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.496+0000 7fe9f1d71640 1 --2- 172.21.15.84:0/3188200906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9d8003970 0x7fe9ec1f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:40.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.496+0000 7fe9f1570640 1 --2- 172.21.15.84:0/3188200906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec159cb0 0x7fe9ec0c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:40.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.496+0000 7fe9f2572640 1 -- 172.21.15.84:0/3188200906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9d8003970 msgr2=0x7fe9ec1f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:40.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.496+0000 7fe9f2572640 1 --2- 172.21.15.84:0/3188200906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9d8003970 0x7fe9ec1f33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.496+0000 7fe9f2572640 1 -- 172.21.15.84:0/3188200906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec159cb0 msgr2=0x7fe9ec0c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:40.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.496+0000 7fe9f2572640 1 --2- 172.21.15.84:0/3188200906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec159cb0 0x7fe9ec0c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.496+0000 7fe9f2572640 1 -- 172.21.15.84:0/3188200906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ec0c6a20 con 0x7fe9ec0c4bb0 2023-10-16T10:05:40.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.496+0000 7fe9f1570640 1 --2- 172.21.15.84:0/3188200906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec159cb0 0x7fe9ec0c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:40.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.497+0000 7fe9f2572640 1 --2- 172.21.15.84:0/3188200906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9ec0c4bb0 0x7fe9ec1f5f30 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec0c59c0 tx=0x7fe9e4007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:40.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.497+0000 7fe9e27fc640 1 -- 172.21.15.84:0/3188200906 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e400ead0 con 0x7fe9ec0c4bb0 2023-10-16T10:05:40.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.497+0000 7fe9e27fc640 1 -- 172.21.15.84:0/3188200906 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e400ec50 con 0x7fe9ec0c4bb0 2023-10-16T10:05:40.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.497+0000 7fe9e27fc640 1 -- 172.21.15.84:0/3188200906 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4014430 con 0x7fe9ec0c4bb0 2023-10-16T10:05:40.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.497+0000 7fe9f3fd5640 1 -- 172.21.15.84:0/3188200906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ec1f3ba0 con 0x7fe9ec0c4bb0 2023-10-16T10:05:40.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.497+0000 7fe9f3fd5640 1 -- 172.21.15.84:0/3188200906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ec1f4160 con 0x7fe9ec0c4bb0 2023-10-16T10:05:40.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.498+0000 7fe9f3fd5640 1 -- 172.21.15.84:0/3188200906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ec04e160 con 0x7fe9ec0c4bb0 2023-10-16T10:05:40.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.499+0000 7fe9e27fc640 1 -- 172.21.15.84:0/3188200906 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fe9e4014590 con 0x7fe9ec0c4bb0 2023-10-16T10:05:40.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.500+0000 7fe9e27fc640 1 --2- 172.21.15.84:0/3188200906 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe9d006af30 0x7fe9d006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:40.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.500+0000 7fe9e27fc640 1 -- 172.21.15.84:0/3188200906 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5631+0+0 (secure 0 0 0) 0x7fe9e4089280 con 0x7fe9ec0c4bb0 2023-10-16T10:05:40.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.500+0000 7fe9f1d71640 1 --2- 172.21.15.84:0/3188200906 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe9d006af30 0x7fe9d006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:40.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.501+0000 7fe9f1d71640 1 --2- 172.21.15.84:0/3188200906 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe9d006af30 0x7fe9d006d3f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8036000 tx=0x7fe9e8031000 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:40.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.505+0000 7fe9e27fc640 1 -- 172.21.15.84:0/3188200906 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe9e4054390 con 0x7fe9ec0c4bb0 2023-10-16T10:05:40.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.701+0000 7fe9f3fd5640 1 -- 172.21.15.84:0/3188200906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9ec15f520 con 0x7fe9ec0c4bb0 2023-10-16T10:05:40.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.702+0000 7fe9e27fc640 1 -- 172.21.15.84:0/3188200906 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+13961 (secure 0 0 0) 0x7fe9e40579b0 con 0x7fe9ec0c4bb0 2023-10-16T10:05:40.704 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:40.705 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":100,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:05:38.222507+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6836/1939610713":"2023-10-17T10:05:35.160900+0000","172.21.15.84:0/2152330716":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/3965108955":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/1978512312":"2023-10-17T10:05:31.449737+0000","172.21.15.84:6833/494168607":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/2341097922":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/4027382473":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/3685429931":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/3830790285":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/821251662":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/504267391":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/3778743167":"2023-10-17T10:05:25.079928+0000","172.21.15.84:0/150347074":"2023-10-17T10:05:25.079928+0000","172.21.15.84:6837/1939610713":"2023-10-17T10:05:35.160900+0000","172.21.15.84:6832/494168607":"2023-10-17T10:05:31.449737+0000","172.21.15.84:0/3151132884":"2023-10-17T10:05:25.079928+0000","172.21.15.84:6833/147697688":"2023-10-17T10:05:25.079928+0000","172.21.15.84:6832/147697688":"2023-10-17T10:05:25.079928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:05:40.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.705+0000 7fe9f3fd5640 1 -- 172.21.15.84:0/3188200906 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe9d006af30 msgr2=0x7fe9d006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:40.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.705+0000 7fe9f3fd5640 1 --2- 172.21.15.84:0/3188200906 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe9d006af30 0x7fe9d006d3f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8036000 tx=0x7fe9e8031000 comp rx=0 tx=0).stop 2023-10-16T10:05:40.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.705+0000 7fe9f3fd5640 1 -- 172.21.15.84:0/3188200906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9ec0c4bb0 msgr2=0x7fe9ec1f5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:40.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.705+0000 7fe9f3fd5640 1 --2- 172.21.15.84:0/3188200906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9ec0c4bb0 0x7fe9ec1f5f30 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec0c59c0 tx=0x7fe9e4007bc0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.705+0000 7fe9f3fd5640 1 -- 172.21.15.84:0/3188200906 shutdown_connections 2023-10-16T10:05:40.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.705+0000 7fe9f3fd5640 1 --2- 172.21.15.84:0/3188200906 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe9d006af30 0x7fe9d006d3f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.705+0000 7fe9f3fd5640 1 --2- 172.21.15.84:0/3188200906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9ec0c4bb0 0x7fe9ec1f5f30 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.705+0000 7fe9f3fd5640 1 --2- 172.21.15.84:0/3188200906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ec159cb0 0x7fe9ec0c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.705+0000 7fe9f3fd5640 1 --2- 172.21.15.84:0/3188200906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9d8003970 0x7fe9ec1f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.705+0000 7fe9f3fd5640 1 -- 172.21.15.84:0/3188200906 >> 172.21.15.84:0/3188200906 conn(0x7fe9ec0cb410 msgr2=0x7fe9ec070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:40.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.705+0000 7fe9f3fd5640 1 -- 172.21.15.84:0/3188200906 shutdown_connections 2023-10-16T10:05:40.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.705+0000 7fe9f3fd5640 1 -- 172.21.15.84:0/3188200906 wait complete. 2023-10-16T10:05:40.727 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:05:40.728 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:05:40.860 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:40.857+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:40.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.950+0000 7f08bed0c640 1 Processor -- start 2023-10-16T10:05:40.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.950+0000 7f08bed0c640 1 -- start start 2023-10-16T10:05:40.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.951+0000 7f08bed0c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f08b815ad70 0x7f08b8069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:40.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.951+0000 7f08bed0c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b80c9ca0 0x7f08b806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:40.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.951+0000 7f08bed0c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b80cb030 con 0x7f08b80ca3b0 2023-10-16T10:05:40.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.952+0000 7f08bed0c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b80c91c0 con 0x7f08b815ad70 2023-10-16T10:05:40.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.952+0000 7f08bed0c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80c94a0 con 0x7f08b80c9ca0 2023-10-16T10:05:40.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.952+0000 7f08bcaa8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f08b815ad70 0x7f08b8069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:40.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.952+0000 7f08bcaa8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f08b815ad70 0x7f08b8069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46196/0 (socket says 172.21.15.84:46196) 2023-10-16T10:05:40.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.952+0000 7f08bcaa8640 1 -- 172.21.15.84:0/2881923085 learned_addr learned my addr 172.21.15.84:0/2881923085 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:40.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.952+0000 7f08affff640 1 --2- 172.21.15.84:0/2881923085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b80c9ca0 0x7f08b806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:40.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.952+0000 7f08bcaa8640 1 -- 172.21.15.84:0/2881923085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b80c9ca0 msgr2=0x7f08b806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:40.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.952+0000 7f08bcaa8640 1 --2- 172.21.15.84:0/2881923085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b80c9ca0 0x7f08b806a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.952+0000 7f08bcaa8640 1 -- 172.21.15.84:0/2881923085 >> v1:172.21.15.157:6789/0 conn(0x7f08b80ca3b0 legacy=0x7f08b80ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:40.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.952+0000 7f08bcaa8640 1 -- 172.21.15.84:0/2881923085 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b80c9780 con 0x7f08b815ad70 2023-10-16T10:05:40.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.954+0000 7f08bcaa8640 1 --2- 172.21.15.84:0/2881923085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08b815ad70 0x7f08b8069da0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009e50 tx=0x7f08b402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78183b831dafaca server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:40.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.954+0000 7f08af7fe640 1 -- 172.21.15.84:0/2881923085 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b402ca70 con 0x7f08b815ad70 2023-10-16T10:05:40.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.954+0000 7f08af7fe640 1 -- 172.21.15.84:0/2881923085 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b402cbd0 con 0x7f08b815ad70 2023-10-16T10:05:40.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.954+0000 7f08af7fe640 1 -- 172.21.15.84:0/2881923085 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b40343f0 con 0x7f08b815ad70 2023-10-16T10:05:40.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.955+0000 7f08bed0c640 1 -- 172.21.15.84:0/2881923085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08b815ad70 msgr2=0x7f08b8069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:40.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.955+0000 7f08bed0c640 1 --2- 172.21.15.84:0/2881923085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08b815ad70 0x7f08b8069da0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009e50 tx=0x7f08b402bed0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.955+0000 7f08bed0c640 1 -- 172.21.15.84:0/2881923085 shutdown_connections 2023-10-16T10:05:40.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.955+0000 7f08bed0c640 1 --2- 172.21.15.84:0/2881923085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b80c9ca0 0x7f08b806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.955+0000 7f08bed0c640 1 --2- 172.21.15.84:0/2881923085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08b815ad70 0x7f08b8069da0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.955+0000 7f08bed0c640 1 -- 172.21.15.84:0/2881923085 >> 172.21.15.84:0/2881923085 conn(0x7f08b80ce9b0 msgr2=0x7f08b80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:40.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.955+0000 7f08bed0c640 1 -- 172.21.15.84:0/2881923085 shutdown_connections 2023-10-16T10:05:40.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.955+0000 7f08bed0c640 1 -- 172.21.15.84:0/2881923085 wait complete. 2023-10-16T10:05:40.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.957+0000 7f08bed0c640 1 Processor -- start 2023-10-16T10:05:40.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.957+0000 7f08bed0c640 1 -- start start 2023-10-16T10:05:40.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.958+0000 7f08bed0c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08b80c9ca0 0x7f08b80b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:40.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.958+0000 7f08bed0c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b80ca3b0 0x7f08b8156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:40.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.958+0000 7f08bed0c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b815ad70 0x7f08b8156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:40.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.958+0000 7f08bed0c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80c8880 con 0x7f08b80c9ca0 2023-10-16T10:05:40.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.958+0000 7f08bed0c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80c79b0 con 0x7f08b80ca3b0 2023-10-16T10:05:40.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.958+0000 7f08bed0c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80c7c70 con 0x7f08b815ad70 2023-10-16T10:05:40.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.959+0000 7f08bcaa8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08b80c9ca0 0x7f08b80b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:40.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.959+0000 7f08bcaa8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08b80c9ca0 0x7f08b80b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46212/0 (socket says 172.21.15.84:46212) 2023-10-16T10:05:40.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.959+0000 7f08bcaa8640 1 -- 172.21.15.84:0/1822460489 learned_addr learned my addr 172.21.15.84:0/1822460489 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:40.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.959+0000 7f08affff640 1 --2- 172.21.15.84:0/1822460489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b80ca3b0 0x7f08b8156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:40.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.959+0000 7f08bd2a9640 1 --2- 172.21.15.84:0/1822460489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b815ad70 0x7f08b8156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:40.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.959+0000 7f08bcaa8640 1 -- 172.21.15.84:0/1822460489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b815ad70 msgr2=0x7f08b8156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:40.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.959+0000 7f08bcaa8640 1 --2- 172.21.15.84:0/1822460489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b815ad70 0x7f08b8156670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.959+0000 7f08bcaa8640 1 -- 172.21.15.84:0/1822460489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b80ca3b0 msgr2=0x7f08b8156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:40.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.959+0000 7f08bcaa8640 1 --2- 172.21.15.84:0/1822460489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b80ca3b0 0x7f08b8156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:40.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.959+0000 7f08bcaa8640 1 -- 172.21.15.84:0/1822460489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b4009b30 con 0x7f08b80c9ca0 2023-10-16T10:05:40.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.959+0000 7f08affff640 1 --2- 172.21.15.84:0/1822460489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b80ca3b0 0x7f08b8156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:40.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.960+0000 7f08bcaa8640 1 --2- 172.21.15.84:0/1822460489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08b80c9ca0 0x7f08b80b35e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f08b4000f80 tx=0x7f08b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:40.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.960+0000 7f08ad7fa640 1 -- 172.21.15.84:0/1822460489 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b4009510 con 0x7f08b80c9ca0 2023-10-16T10:05:40.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.960+0000 7f08ad7fa640 1 -- 172.21.15.84:0/1822460489 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b403e450 con 0x7f08b80c9ca0 2023-10-16T10:05:40.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.960+0000 7f08bed0c640 1 -- 172.21.15.84:0/1822460489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b80c7f50 con 0x7f08b80c9ca0 2023-10-16T10:05:40.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.960+0000 7f08ad7fa640 1 -- 172.21.15.84:0/1822460489 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b403e800 con 0x7f08b80c9ca0 2023-10-16T10:05:40.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.960+0000 7f08bed0c640 1 -- 172.21.15.84:0/1822460489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b80c8510 con 0x7f08b80c9ca0 2023-10-16T10:05:40.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.961+0000 7f08bed0c640 1 -- 172.21.15.84:0/1822460489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b804e160 con 0x7f08b80c9ca0 2023-10-16T10:05:40.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.962+0000 7f08ad7fa640 1 -- 172.21.15.84:0/1822460489 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f08b403e960 con 0x7f08b80c9ca0 2023-10-16T10:05:40.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.963+0000 7f08ad7fa640 1 --2- 172.21.15.84:0/1822460489 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f08a406af00 0x7f08a406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:40.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.963+0000 7f08ad7fa640 1 -- 172.21.15.84:0/1822460489 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f08b40ac520 con 0x7f08b80c9ca0 2023-10-16T10:05:40.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.964+0000 7f08affff640 1 --2- 172.21.15.84:0/1822460489 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f08a406af00 0x7f08a406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:40.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.964+0000 7f08affff640 1 --2- 172.21.15.84:0/1822460489 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f08a406af00 0x7f08a406d3c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f08b815e1b0 tx=0x7f08a8008040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:40.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:40.968+0000 7f08ad7fa640 1 -- 172.21.15.84:0/1822460489 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f08b40774a0 con 0x7f08b80c9ca0 2023-10-16T10:05:41.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.160+0000 7f08bed0c640 1 -- 172.21.15.84:0/1822460489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08b806c700 con 0x7f08b80c9ca0 2023-10-16T10:05:41.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.161+0000 7f08ad7fa640 1 -- 172.21.15.84:0/1822460489 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08b407aac0 con 0x7f08b80c9ca0 2023-10-16T10:05:41.163 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:05:41.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.163+0000 7f08bed0c640 1 -- 172.21.15.84:0/1822460489 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f08a406af00 msgr2=0x7f08a406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:41.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.163+0000 7f08bed0c640 1 --2- 172.21.15.84:0/1822460489 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f08a406af00 0x7f08a406d3c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f08b815e1b0 tx=0x7f08a8008040 comp rx=0 tx=0).stop 2023-10-16T10:05:41.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.163+0000 7f08bed0c640 1 -- 172.21.15.84:0/1822460489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08b80c9ca0 msgr2=0x7f08b80b35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:41.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.163+0000 7f08bed0c640 1 --2- 172.21.15.84:0/1822460489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08b80c9ca0 0x7f08b80b35e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f08b4000f80 tx=0x7f08b4004a40 comp rx=0 tx=0).stop 2023-10-16T10:05:41.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.164+0000 7f08bed0c640 1 -- 172.21.15.84:0/1822460489 shutdown_connections 2023-10-16T10:05:41.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.164+0000 7f08bed0c640 1 --2- 172.21.15.84:0/1822460489 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f08a406af00 0x7f08a406d3c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:41.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.164+0000 7f08bed0c640 1 --2- 172.21.15.84:0/1822460489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b815ad70 0x7f08b8156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:41.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.164+0000 7f08bed0c640 1 --2- 172.21.15.84:0/1822460489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b80ca3b0 0x7f08b8156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:41.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.164+0000 7f08bed0c640 1 --2- 172.21.15.84:0/1822460489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08b80c9ca0 0x7f08b80b35e0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:41.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.164+0000 7f08bed0c640 1 -- 172.21.15.84:0/1822460489 >> 172.21.15.84:0/1822460489 conn(0x7f08b80ce9b0 msgr2=0x7f08b80b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:41.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.164+0000 7f08bed0c640 1 -- 172.21.15.84:0/1822460489 shutdown_connections 2023-10-16T10:05:41.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.164+0000 7f08bed0c640 1 -- 172.21.15.84:0/1822460489 wait complete. 2023-10-16T10:05:41.185 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-10-16T10:05:41.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.397+0000 7f3a5f31d640 1 Processor -- start 2023-10-16T10:05:41.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.398+0000 7f3a5f31d640 1 -- start start 2023-10-16T10:05:41.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.399+0000 7f3a5f31d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a58159f00 0x7f3a580b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:41.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.399+0000 7f3a5f31d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a5806f7f0 con 0x7f3a58159b20 2023-10-16T10:05:41.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.399+0000 7f3a5f31d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a5806c5e0 con 0x7f3a5815d880 2023-10-16T10:05:41.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.399+0000 7f3a5f31d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5806c8c0 con 0x7f3a58159f00 2023-10-16T10:05:41.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.400+0000 7f3a5d0b9640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3a58159b20 0x7f3a5806bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39946/0 (socket says 172.21.15.84:39946) 2023-10-16T10:05:41.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.400+0000 7f3a5d0b9640 1 -- 172.21.15.84:0/2964164469 learned_addr learned my addr 172.21.15.84:0/2964164469 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:41.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.400+0000 7f3a5c8b8640 1 --2- 172.21.15.84:0/2964164469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a58159f00 0x7f3a580b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:41.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.400+0000 7f3a4ffff640 1 -- 172.21.15.84:0/2964164469 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3522913540 0 0) 0x7f3a5806f7f0 con 0x7f3a58159b20 2023-10-16T10:05:41.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.400+0000 7f3a4ffff640 1 -- 172.21.15.84:0/2964164469 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a44003520 con 0x7f3a58159b20 2023-10-16T10:05:41.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.400+0000 7f3a4ffff640 1 -- 172.21.15.84:0/2964164469 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3872567115 0 0) 0x7f3a44003520 con 0x7f3a58159b20 2023-10-16T10:05:41.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.400+0000 7f3a4ffff640 1 -- 172.21.15.84:0/2964164469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a58159f00 msgr2=0x7f3a580b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:41.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.400+0000 7f3a4ffff640 1 --2- 172.21.15.84:0/2964164469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a58159f00 0x7f3a580b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:41.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.400+0000 7f3a4ffff640 1 -- 172.21.15.84:0/2964164469 >> v1:172.21.15.157:6789/0 conn(0x7f3a5815d880 legacy=0x7f3a5806fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:41.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.400+0000 7f3a4ffff640 1 -- 172.21.15.84:0/2964164469 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5806cba0 con 0x7f3a58159b20 2023-10-16T10:05:41.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.400+0000 7f3a4ffff640 1 -- 172.21.15.84:0/2964164469 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f3a54001670 con 0x7f3a58159b20 2023-10-16T10:05:41.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.401+0000 7f3a4ffff640 1 -- 172.21.15.84:0/2964164469 >> v1:172.21.15.84:6789/0 conn(0x7f3a58159b20 legacy=0x7f3a5806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:41.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.401+0000 7f3a4ffff640 1 --2- 172.21.15.84:0/2964164469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a440039a0 0x7f3a44003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:41.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.401+0000 7f3a4ffff640 1 --2- 172.21.15.84:0/2964164469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a440047c0 0x7f3a44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:41.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.401+0000 7f3a4ffff640 1 --2- 172.21.15.84:0/2964164469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a58159f00 0x7f3a44005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:41.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.401+0000 7f3a4ffff640 1 -- 172.21.15.84:0/2964164469 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5806c8c0 con 0x7f3a440047c0 2023-10-16T10:05:41.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.401+0000 7f3a4ffff640 1 -- 172.21.15.84:0/2964164469 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4400dc40 con 0x7f3a440039a0 2023-10-16T10:05:41.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.401+0000 7f3a4ffff640 1 -- 172.21.15.84:0/2964164469 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4400de00 con 0x7f3a58159f00 2023-10-16T10:05:41.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.401+0000 7f3a5c8b8640 1 --2- 172.21.15.84:0/2964164469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a440047c0 0x7f3a44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:41.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.401+0000 7f3a5d0b9640 1 --2- 172.21.15.84:0/2964164469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a440039a0 0x7f3a44003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:41.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.401+0000 7f3a5d8ba640 1 --2- 172.21.15.84:0/2964164469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a58159f00 0x7f3a44005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:41.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.401+0000 7f3a5c8b8640 1 -- 172.21.15.84:0/2964164469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a58159f00 msgr2=0x7f3a44005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:41.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.401+0000 7f3a5c8b8640 1 --2- 172.21.15.84:0/2964164469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a58159f00 0x7f3a44005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:41.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.401+0000 7f3a5c8b8640 1 -- 172.21.15.84:0/2964164469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a440039a0 msgr2=0x7f3a44003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:41.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.401+0000 7f3a5c8b8640 1 --2- 172.21.15.84:0/2964164469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a440039a0 0x7f3a44003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:41.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.401+0000 7f3a5c8b8640 1 -- 172.21.15.84:0/2964164469 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a44003520 con 0x7f3a440047c0 2023-10-16T10:05:41.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.401+0000 7f3a5d0b9640 1 --2- 172.21.15.84:0/2964164469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a440039a0 0x7f3a44003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:41.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.401+0000 7f3a5d8ba640 1 --2- 172.21.15.84:0/2964164469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a58159f00 0x7f3a44005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:41.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.402+0000 7f3a5c8b8640 1 --2- 172.21.15.84:0/2964164469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a440047c0 0x7f3a44004c40 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f3a5815a2e0 tx=0x7f3a440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e5e10e369e3447 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.403+0000 7f3a4ffff640 1 -- 172.21.15.84:0/2964164469 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4802c630 con 0x7f3a440047c0 2023-10-16T10:05:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.403+0000 7f3a4ffff640 1 -- 172.21.15.84:0/2964164469 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4802c790 con 0x7f3a440047c0 2023-10-16T10:05:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.403+0000 7f3a4ffff640 1 -- 172.21.15.84:0/2964164469 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4802cb20 con 0x7f3a440047c0 2023-10-16T10:05:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.403+0000 7f3a5f31d640 1 -- 172.21.15.84:0/2964164469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a440047c0 msgr2=0x7f3a44004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:41.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.403+0000 7f3a5f31d640 1 --2- 172.21.15.84:0/2964164469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a440047c0 0x7f3a44004c40 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f3a5815a2e0 tx=0x7f3a440057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:41.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.403+0000 7f3a5d8ba640 1 -- 172.21.15.84:0/2964164469 reap_dead start 2023-10-16T10:05:41.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.403+0000 7f3a5f31d640 1 -- 172.21.15.84:0/2964164469 shutdown_connections 2023-10-16T10:05:41.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.403+0000 7f3a5f31d640 1 -- 172.21.15.84:0/2964164469 >> 172.21.15.84:0/2964164469 conn(0x7f3a580cb410 msgr2=0x7f3a580cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:41.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.404+0000 7f3a5f31d640 1 -- 172.21.15.84:0/2964164469 shutdown_connections 2023-10-16T10:05:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.404+0000 7f3a5f31d640 1 -- 172.21.15.84:0/2964164469 wait complete. 2023-10-16T10:05:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.405+0000 7f3a5f31d640 1 Processor -- start 2023-10-16T10:05:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.405+0000 7f3a5f31d640 1 -- start start 2023-10-16T10:05:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.406+0000 7f3a5f31d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a440047c0 0x7f3a5815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.406+0000 7f3a5f31d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a58154cd0 0x7f3a580c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.406+0000 7f3a5f31d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a580c3fd0 0x7f3a580ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.406+0000 7f3a5f31d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5815c3b0 con 0x7f3a580c3fd0 2023-10-16T10:05:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.406+0000 7f3a5f31d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5815d1a0 con 0x7f3a58154cd0 2023-10-16T10:05:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.406+0000 7f3a5f31d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5815b730 con 0x7f3a440047c0 2023-10-16T10:05:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.407+0000 7f3a5d8ba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a580c3fd0 0x7f3a580ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:41.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.407+0000 7f3a5d8ba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a580c3fd0 0x7f3a580ce870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46226/0 (socket says 172.21.15.84:46226) 2023-10-16T10:05:41.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.407+0000 7f3a5d8ba640 1 -- 172.21.15.84:0/3372731978 learned_addr learned my addr 172.21.15.84:0/3372731978 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:41.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.407+0000 7f3a5c8b8640 1 --2- 172.21.15.84:0/3372731978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a58154cd0 0x7f3a580c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:41.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.407+0000 7f3a5d0b9640 1 --2- 172.21.15.84:0/3372731978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a440047c0 0x7f3a5815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:41.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.407+0000 7f3a5d8ba640 1 -- 172.21.15.84:0/3372731978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a440047c0 msgr2=0x7f3a5815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:41.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.407+0000 7f3a5d8ba640 1 --2- 172.21.15.84:0/3372731978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a440047c0 0x7f3a5815c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:41.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.407+0000 7f3a5d8ba640 1 -- 172.21.15.84:0/3372731978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a58154cd0 msgr2=0x7f3a580c3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:41.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.407+0000 7f3a5d8ba640 1 --2- 172.21.15.84:0/3372731978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a58154cd0 0x7f3a580c3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:41.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.407+0000 7f3a5d8ba640 1 -- 172.21.15.84:0/3372731978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5815ba10 con 0x7f3a580c3fd0 2023-10-16T10:05:41.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.407+0000 7f3a5c8b8640 1 --2- 172.21.15.84:0/3372731978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a58154cd0 0x7f3a580c3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:41.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.407+0000 7f3a5d8ba640 1 --2- 172.21.15.84:0/3372731978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a580c3fd0 0x7f3a580ce870 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f3a580c8bb0 tx=0x7f3a5000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:41.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.408+0000 7f3a4dffb640 1 -- 172.21.15.84:0/3372731978 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50006d50 con 0x7f3a580c3fd0 2023-10-16T10:05:41.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.408+0000 7f3a4dffb640 1 -- 172.21.15.84:0/3372731978 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a50003970 con 0x7f3a580c3fd0 2023-10-16T10:05:41.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.408+0000 7f3a4dffb640 1 -- 172.21.15.84:0/3372731978 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50003d60 con 0x7f3a580c3fd0 2023-10-16T10:05:41.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.408+0000 7f3a5f31d640 1 -- 172.21.15.84:0/3372731978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5815bcf0 con 0x7f3a580c3fd0 2023-10-16T10:05:41.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.408+0000 7f3a5f31d640 1 -- 172.21.15.84:0/3372731978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5815cd10 con 0x7f3a580c3fd0 2023-10-16T10:05:41.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.409+0000 7f3a5f31d640 1 -- 172.21.15.84:0/3372731978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5815c4f0 con 0x7f3a580c3fd0 2023-10-16T10:05:41.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.410+0000 7f3a4dffb640 1 -- 172.21.15.84:0/3372731978 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f3a50005070 con 0x7f3a580c3fd0 2023-10-16T10:05:41.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.411+0000 7f3a4dffb640 1 --2- 172.21.15.84:0/3372731978 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f3a3c06af00 0x7f3a3c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:41.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.411+0000 7f3a4dffb640 1 -- 172.21.15.84:0/3372731978 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f3a50089530 con 0x7f3a580c3fd0 2023-10-16T10:05:41.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.411+0000 7f3a5d0b9640 1 --2- 172.21.15.84:0/3372731978 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f3a3c06af00 0x7f3a3c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:41.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.412+0000 7f3a5d0b9640 1 --2- 172.21.15.84:0/3372731978 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f3a3c06af00 0x7f3a3c06d3c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3a54000c00 tx=0x7f3a54009040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:41.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.416+0000 7f3a4dffb640 1 -- 172.21.15.84:0/3372731978 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3a5815c4f0 con 0x7f3a580c3fd0 2023-10-16T10:05:41.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:41.685+0000 7f3a5f31d640 1 -- 172.21.15.84:0/3372731978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f3a5815c4f0 con 0x7f3a580c3fd0 2023-10-16T10:05:41.811 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:41.808+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:42.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.249+0000 7f3a4dffb640 1 -- 172.21.15.84:0/3372731978 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v627) v1 ==== 186+0+0 (secure 0 0 0) 0x7f3a5815c4f0 con 0x7f3a580c3fd0 2023-10-16T10:05:42.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.254+0000 7f3a5f31d640 1 -- 172.21.15.84:0/3372731978 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f3a3c06af00 msgr2=0x7f3a3c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:42.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.254+0000 7f3a5f31d640 1 --2- 172.21.15.84:0/3372731978 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f3a3c06af00 0x7f3a3c06d3c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3a54000c00 tx=0x7f3a54009040 comp rx=0 tx=0).stop 2023-10-16T10:05:42.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.254+0000 7f3a5f31d640 1 -- 172.21.15.84:0/3372731978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a580c3fd0 msgr2=0x7f3a580ce870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:42.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.254+0000 7f3a5f31d640 1 --2- 172.21.15.84:0/3372731978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a580c3fd0 0x7f3a580ce870 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f3a580c8bb0 tx=0x7f3a5000dc00 comp rx=0 tx=0).stop 2023-10-16T10:05:42.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.254+0000 7f3a5f31d640 1 -- 172.21.15.84:0/3372731978 shutdown_connections 2023-10-16T10:05:42.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.254+0000 7f3a5f31d640 1 --2- 172.21.15.84:0/3372731978 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f3a3c06af00 0x7f3a3c06d3c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:42.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.254+0000 7f3a5f31d640 1 --2- 172.21.15.84:0/3372731978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a580c3fd0 0x7f3a580ce870 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:42.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.254+0000 7f3a5f31d640 1 --2- 172.21.15.84:0/3372731978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a58154cd0 0x7f3a580c3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:42.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.254+0000 7f3a5f31d640 1 --2- 172.21.15.84:0/3372731978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a440047c0 0x7f3a5815c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:42.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.254+0000 7f3a5f31d640 1 -- 172.21.15.84:0/3372731978 >> 172.21.15.84:0/3372731978 conn(0x7f3a580cb410 msgr2=0x7f3a580cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:42.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.254+0000 7f3a5f31d640 1 -- 172.21.15.84:0/3372731978 shutdown_connections 2023-10-16T10:05:42.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.254+0000 7f3a5f31d640 1 -- 172.21.15.84:0/3372731978 wait complete. 2023-10-16T10:05:42.277 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T10:05:42.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.493+0000 7f49c0141640 1 Processor -- start 2023-10-16T10:05:42.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.494+0000 7f49c0141640 1 -- start start 2023-10-16T10:05:42.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.495+0000 7f49c0141640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49b80cf710 0x7f49b80b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:42.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.495+0000 7f49c0141640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f49b80cfaf0 0x7f49b80b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:42.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.495+0000 7f49c0141640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b80d3980 con 0x7f49b80b38e0 2023-10-16T10:05:42.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.495+0000 7f49c0141640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b80d3c60 con 0x7f49b80cfaf0 2023-10-16T10:05:42.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.495+0000 7f49c0141640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b80ce630 con 0x7f49b80cf710 2023-10-16T10:05:42.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.495+0000 7f49be6de640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f49b80b38e0 0x7f49b80d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39958/0 (socket says 172.21.15.84:39958) 2023-10-16T10:05:42.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.495+0000 7f49be6de640 1 -- 172.21.15.84:0/1969052428 learned_addr learned my addr 172.21.15.84:0/1969052428 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:42.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.496+0000 7f49bdedd640 1 --2- 172.21.15.84:0/1969052428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49b80cf710 0x7f49b80b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:42.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.496+0000 7f49bd6dc640 1 --2- 172.21.15.84:0/1969052428 >> v2:172.21.15.157:3300/0 conn(0x7f49b80cfaf0 0x7f49b80b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:42.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.496+0000 7f49bcedb640 1 -- 172.21.15.84:0/1969052428 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454762861 0 0) 0x7f49b80d3980 con 0x7f49b80b38e0 2023-10-16T10:05:42.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.496+0000 7f49bcedb640 1 -- 172.21.15.84:0/1969052428 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49a0003520 con 0x7f49b80b38e0 2023-10-16T10:05:42.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.496+0000 7f49bcedb640 1 -- 172.21.15.84:0/1969052428 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 32602213 0 0) 0x7f49a0003520 con 0x7f49b80b38e0 2023-10-16T10:05:42.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.496+0000 7f49bcedb640 1 -- 172.21.15.84:0/1969052428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49b80cf710 msgr2=0x7f49b80b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:42.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.496+0000 7f49bcedb640 1 --2- 172.21.15.84:0/1969052428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49b80cf710 0x7f49b80b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:42.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.496+0000 7f49bcedb640 1 -- 172.21.15.84:0/1969052428 >> v2:172.21.15.157:3300/0 conn(0x7f49b80cfaf0 msgr2=0x7f49b80b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:42.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.496+0000 7f49bcedb640 1 --2- 172.21.15.84:0/1969052428 >> v2:172.21.15.157:3300/0 conn(0x7f49b80cfaf0 0x7f49b80b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:42.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.496+0000 7f49bcedb640 1 -- 172.21.15.84:0/1969052428 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b80ce910 con 0x7f49b80b38e0 2023-10-16T10:05:42.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.496+0000 7f49bcedb640 1 -- 172.21.15.84:0/1969052428 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f49b0001670 con 0x7f49b80b38e0 2023-10-16T10:05:42.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.496+0000 7f49bcedb640 1 -- 172.21.15.84:0/1969052428 >> v1:172.21.15.84:6789/0 conn(0x7f49b80b38e0 legacy=0x7f49b80d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:42.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.497+0000 7f49bcedb640 1 --2- 172.21.15.84:0/1969052428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49a0003970 0x7f49a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:42.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.497+0000 7f49bcedb640 1 --2- 172.21.15.84:0/1969052428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49a00047c0 0x7f49a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:42.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.497+0000 7f49bcedb640 1 --2- 172.21.15.84:0/1969052428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49b80cf710 0x7f49a0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:42.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.497+0000 7f49bcedb640 1 -- 172.21.15.84:0/1969052428 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b80d3c60 con 0x7f49a00047c0 2023-10-16T10:05:42.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.497+0000 7f49bcedb640 1 -- 172.21.15.84:0/1969052428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b80ce630 con 0x7f49a0003970 2023-10-16T10:05:42.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.497+0000 7f49bcedb640 1 -- 172.21.15.84:0/1969052428 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a000b980 con 0x7f49b80cf710 2023-10-16T10:05:42.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.497+0000 7f49bd6dc640 1 --2- 172.21.15.84:0/1969052428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49a00047c0 0x7f49a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:42.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.497+0000 7f49bdedd640 1 --2- 172.21.15.84:0/1969052428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49a0003970 0x7f49a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:42.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.497+0000 7f49be6de640 1 --2- 172.21.15.84:0/1969052428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49b80cf710 0x7f49a0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:42.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.497+0000 7f49bd6dc640 1 -- 172.21.15.84:0/1969052428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49b80cf710 msgr2=0x7f49a0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:42.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.497+0000 7f49bd6dc640 1 --2- 172.21.15.84:0/1969052428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49b80cf710 0x7f49a0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:42.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.497+0000 7f49bd6dc640 1 -- 172.21.15.84:0/1969052428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49a0003970 msgr2=0x7f49a0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:42.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.497+0000 7f49bd6dc640 1 --2- 172.21.15.84:0/1969052428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49a0003970 0x7f49a0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:42.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.497+0000 7f49bd6dc640 1 -- 172.21.15.84:0/1969052428 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a0003520 con 0x7f49a00047c0 2023-10-16T10:05:42.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.497+0000 7f49bdedd640 1 --2- 172.21.15.84:0/1969052428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49a0003970 0x7f49a0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:42.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.498+0000 7f49bd6dc640 1 --2- 172.21.15.84:0/1969052428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49a00047c0 0x7f49a0004c40 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f49ac009af0 tx=0x7f49a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce05efa4175b9539 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:42.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.499+0000 7f49bcedb640 1 -- 172.21.15.84:0/1969052428 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac02c710 con 0x7f49a00047c0 2023-10-16T10:05:42.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.499+0000 7f49bcedb640 1 -- 172.21.15.84:0/1969052428 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ac02c870 con 0x7f49a00047c0 2023-10-16T10:05:42.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.499+0000 7f49bcedb640 1 -- 172.21.15.84:0/1969052428 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac02cc00 con 0x7f49a00047c0 2023-10-16T10:05:42.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.499+0000 7f49c0141640 1 -- 172.21.15.84:0/1969052428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49a00047c0 msgr2=0x7f49a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:42.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.499+0000 7f49c0141640 1 --2- 172.21.15.84:0/1969052428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49a00047c0 0x7f49a0004c40 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f49ac009af0 tx=0x7f49a00057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:42.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.499+0000 7f49be6de640 1 -- 172.21.15.84:0/1969052428 reap_dead start 2023-10-16T10:05:42.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.499+0000 7f49c0141640 1 -- 172.21.15.84:0/1969052428 shutdown_connections 2023-10-16T10:05:42.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.499+0000 7f49c0141640 1 -- 172.21.15.84:0/1969052428 >> 172.21.15.84:0/1969052428 conn(0x7f49b80d2cc0 msgr2=0x7f49b80d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:42.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.500+0000 7f49c0141640 1 -- 172.21.15.84:0/1969052428 shutdown_connections 2023-10-16T10:05:42.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.500+0000 7f49c0141640 1 -- 172.21.15.84:0/1969052428 wait complete. 2023-10-16T10:05:42.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.501+0000 7f49c0141640 1 Processor -- start 2023-10-16T10:05:42.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.501+0000 7f49c0141640 1 -- start start 2023-10-16T10:05:42.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.502+0000 7f49c0141640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49a00047c0 0x7f49b814e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:42.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.502+0000 7f49c0141640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49b80c9d90 0x7f49b80cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:42.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.502+0000 7f49c0141640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49b80b6090 0x7f49b80b6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:42.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.502+0000 7f49c0141640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b814e6c0 con 0x7f49b80b6090 2023-10-16T10:05:42.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.502+0000 7f49c0141640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b80b6a50 con 0x7f49b80c9d90 2023-10-16T10:05:42.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.502+0000 7f49c0141640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b80cc470 con 0x7f49a00047c0 2023-10-16T10:05:42.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.503+0000 7f49be6de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49b80b6090 0x7f49b80b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:42.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.503+0000 7f49be6de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49b80b6090 0x7f49b80b6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46238/0 (socket says 172.21.15.84:46238) 2023-10-16T10:05:42.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.503+0000 7f49be6de640 1 -- 172.21.15.84:0/3697535731 learned_addr learned my addr 172.21.15.84:0/3697535731 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:42.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.503+0000 7f49bdedd640 1 --2- 172.21.15.84:0/3697535731 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49a00047c0 0x7f49b814e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:42.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.503+0000 7f49bd6dc640 1 --2- 172.21.15.84:0/3697535731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49b80c9d90 0x7f49b80cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:42.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.503+0000 7f49be6de640 1 -- 172.21.15.84:0/3697535731 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49a00047c0 msgr2=0x7f49b814e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:42.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.503+0000 7f49be6de640 1 --2- 172.21.15.84:0/3697535731 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49a00047c0 0x7f49b814e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:42.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.503+0000 7f49be6de640 1 -- 172.21.15.84:0/3697535731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49b80c9d90 msgr2=0x7f49b80cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:42.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.503+0000 7f49be6de640 1 --2- 172.21.15.84:0/3697535731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49b80c9d90 0x7f49b80cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:42.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.503+0000 7f49be6de640 1 -- 172.21.15.84:0/3697535731 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b80c6df0 con 0x7f49b80b6090 2023-10-16T10:05:42.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.503+0000 7f49be6de640 1 --2- 172.21.15.84:0/3697535731 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49b80b6090 0x7f49b80b6510 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f49b808c160 tx=0x7f49b000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:42.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.504+0000 7f49b67fc640 1 -- 172.21.15.84:0/3697535731 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0007230 con 0x7f49b80b6090 2023-10-16T10:05:42.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.504+0000 7f49b67fc640 1 -- 172.21.15.84:0/3697535731 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b0007390 con 0x7f49b80b6090 2023-10-16T10:05:42.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.504+0000 7f49c0141640 1 -- 172.21.15.84:0/3697535731 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b815fe50 con 0x7f49b80b6090 2023-10-16T10:05:42.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.504+0000 7f49b67fc640 1 -- 172.21.15.84:0/3697535731 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b00047e0 con 0x7f49b80b6090 2023-10-16T10:05:42.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.504+0000 7f49c0141640 1 -- 172.21.15.84:0/3697535731 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b80ca5e0 con 0x7f49b80b6090 2023-10-16T10:05:42.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.505+0000 7f49c0141640 1 -- 172.21.15.84:0/3697535731 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b804e160 con 0x7f49b80b6090 2023-10-16T10:05:42.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.506+0000 7f49b67fc640 1 -- 172.21.15.84:0/3697535731 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f49b0004940 con 0x7f49b80b6090 2023-10-16T10:05:42.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.507+0000 7f49b67fc640 1 --2- 172.21.15.84:0/3697535731 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f49a406af30 0x7f49a406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:42.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.507+0000 7f49b67fc640 1 -- 172.21.15.84:0/3697535731 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f49b0089430 con 0x7f49b80b6090 2023-10-16T10:05:42.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.507+0000 7f49bdedd640 1 --2- 172.21.15.84:0/3697535731 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f49a406af30 0x7f49a406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:42.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.508+0000 7f49bdedd640 1 --2- 172.21.15.84:0/3697535731 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f49a406af30 0x7f49a406d3f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f49a80074a0 tx=0x7f49a800d040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:42.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.511+0000 7f49b67fc640 1 -- 172.21.15.84:0/3697535731 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f49b0054370 con 0x7f49b80b6090 2023-10-16T10:05:42.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.710+0000 7f49c0141640 1 -- 172.21.15.84:0/3697535731 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f49b80d1270 con 0x7f49b80b6090 2023-10-16T10:05:42.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.717+0000 7f49b67fc640 1 -- 172.21.15.84:0/3697535731 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f49b0057990 con 0x7f49b80b6090 2023-10-16T10:05:42.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.720+0000 7f49c0141640 1 -- 172.21.15.84:0/3697535731 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f49a406af30 msgr2=0x7f49a406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:42.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.720+0000 7f49c0141640 1 --2- 172.21.15.84:0/3697535731 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f49a406af30 0x7f49a406d3f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f49a80074a0 tx=0x7f49a800d040 comp rx=0 tx=0).stop 2023-10-16T10:05:42.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.720+0000 7f49c0141640 1 -- 172.21.15.84:0/3697535731 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49b80b6090 msgr2=0x7f49b80b6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:42.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.720+0000 7f49c0141640 1 --2- 172.21.15.84:0/3697535731 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49b80b6090 0x7f49b80b6510 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f49b808c160 tx=0x7f49b000dc00 comp rx=0 tx=0).stop 2023-10-16T10:05:42.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.720+0000 7f49c0141640 1 -- 172.21.15.84:0/3697535731 shutdown_connections 2023-10-16T10:05:42.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.720+0000 7f49c0141640 1 --2- 172.21.15.84:0/3697535731 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f49a406af30 0x7f49a406d3f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:42.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.720+0000 7f49c0141640 1 --2- 172.21.15.84:0/3697535731 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49b80b6090 0x7f49b80b6510 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:42.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.720+0000 7f49c0141640 1 --2- 172.21.15.84:0/3697535731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49b80c9d90 0x7f49b80cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:42.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.720+0000 7f49c0141640 1 --2- 172.21.15.84:0/3697535731 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49a00047c0 0x7f49b814e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:42.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.720+0000 7f49c0141640 1 -- 172.21.15.84:0/3697535731 >> 172.21.15.84:0/3697535731 conn(0x7f49b80d2cc0 msgr2=0x7f49b815bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:42.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.721+0000 7f49c0141640 1 -- 172.21.15.84:0/3697535731 shutdown_connections 2023-10-16T10:05:42.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:42.721+0000 7f49c0141640 1 -- 172.21.15.84:0/3697535731 wait complete. 2023-10-16T10:05:42.742 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:05:42.742 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:05:42.797 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:42.795+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:42.860 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:05:42.978 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:05:42.979 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:05:42.979 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:05:43.039 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:05:43.040 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:05:43.040 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:05:43.097 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:05:43.098 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:05:43.098 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:05:43.160 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:05:43.160 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:05:43.161 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:05:43.221 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:05:43.221 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T10:05:43.221 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:05:43.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.450+0000 7fe296ebb640 1 Processor -- start 2023-10-16T10:05:43.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.450+0000 7fe296ebb640 1 -- start start 2023-10-16T10:05:43.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.452+0000 7fe296ebb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe290070b60 0x7fe29015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:43.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.452+0000 7fe296ebb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe29006c100 0x7fe29006c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:43.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.452+0000 7fe296ebb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29015aef0 con 0x7fe290159950 2023-10-16T10:05:43.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.452+0000 7fe296ebb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe29015e490 con 0x7fe290070b60 2023-10-16T10:05:43.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.452+0000 7fe296ebb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29015d910 con 0x7fe29006c100 2023-10-16T10:05:43.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.452+0000 7fe2956b8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe290070b60 0x7fe29015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:43.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.452+0000 7fe2956b8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe290070b60 0x7fe29015a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46246/0 (socket says 172.21.15.84:46246) 2023-10-16T10:05:43.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.452+0000 7fe2956b8640 1 -- 172.21.15.84:0/753046814 learned_addr learned my addr 172.21.15.84:0/753046814 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:43.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.453+0000 7fe2956b8640 1 -- 172.21.15.84:0/753046814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe29006c100 msgr2=0x7fe29006c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:43.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.453+0000 7fe2966ba640 1 --2- 172.21.15.84:0/753046814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe29006c100 0x7fe29006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:43.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.453+0000 7fe2956b8640 1 --2- 172.21.15.84:0/753046814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe29006c100 0x7fe29006c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:43.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.453+0000 7fe2956b8640 1 -- 172.21.15.84:0/753046814 >> v1:172.21.15.157:6789/0 conn(0x7fe290159950 legacy=0x7fe290070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:43.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.453+0000 7fe2956b8640 1 -- 172.21.15.84:0/753046814 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29015dbf0 con 0x7fe290070b60 2023-10-16T10:05:43.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.453+0000 7fe2966ba640 1 --2- 172.21.15.84:0/753046814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe29006c100 0x7fe29006c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:43.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.454+0000 7fe2956b8640 1 --2- 172.21.15.84:0/753046814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe290070b60 0x7fe29015a9b0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009870 tx=0x7fe28c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5db2743167359af server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:43.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.454+0000 7fe294eb7640 1 -- 172.21.15.84:0/753046814 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c0317c0 con 0x7fe290070b60 2023-10-16T10:05:43.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.454+0000 7fe294eb7640 1 -- 172.21.15.84:0/753046814 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c031920 con 0x7fe290070b60 2023-10-16T10:05:43.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.454+0000 7fe294eb7640 1 -- 172.21.15.84:0/753046814 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c031cb0 con 0x7fe290070b60 2023-10-16T10:05:43.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.455+0000 7fe296ebb640 1 -- 172.21.15.84:0/753046814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe290070b60 msgr2=0x7fe29015a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:43.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.455+0000 7fe296ebb640 1 --2- 172.21.15.84:0/753046814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe290070b60 0x7fe29015a9b0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009870 tx=0x7fe28c02b860 comp rx=0 tx=0).stop 2023-10-16T10:05:43.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.455+0000 7fe296ebb640 1 -- 172.21.15.84:0/753046814 shutdown_connections 2023-10-16T10:05:43.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.455+0000 7fe296ebb640 1 --2- 172.21.15.84:0/753046814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe29006c100 0x7fe29006c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:43.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.455+0000 7fe296ebb640 1 --2- 172.21.15.84:0/753046814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe290070b60 0x7fe29015a9b0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:43.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.455+0000 7fe296ebb640 1 -- 172.21.15.84:0/753046814 >> 172.21.15.84:0/753046814 conn(0x7fe2900ce6a0 msgr2=0x7fe2900ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:43.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.455+0000 7fe296ebb640 1 -- 172.21.15.84:0/753046814 shutdown_connections 2023-10-16T10:05:43.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.455+0000 7fe296ebb640 1 -- 172.21.15.84:0/753046814 wait complete. 2023-10-16T10:05:43.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.457+0000 7fe296ebb640 1 Processor -- start 2023-10-16T10:05:43.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.457+0000 7fe296ebb640 1 -- start start 2023-10-16T10:05:43.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.458+0000 7fe296ebb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe29006c100 0x7fe2900c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:43.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.458+0000 7fe296ebb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe290070b60 0x7fe2900ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:43.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.458+0000 7fe296ebb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe290159950 0x7fe2900acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:43.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.458+0000 7fe296ebb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900c7e80 con 0x7fe290159950 2023-10-16T10:05:43.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.458+0000 7fe296ebb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900c59f0 con 0x7fe290070b60 2023-10-16T10:05:43.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.458+0000 7fe296ebb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29015c750 con 0x7fe29006c100 2023-10-16T10:05:43.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.458+0000 7fe2966ba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe290159950 0x7fe2900acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:43.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.458+0000 7fe2966ba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe290159950 0x7fe2900acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46258/0 (socket says 172.21.15.84:46258) 2023-10-16T10:05:43.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.458+0000 7fe2966ba640 1 -- 172.21.15.84:0/2119096762 learned_addr learned my addr 172.21.15.84:0/2119096762 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:43.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.459+0000 7fe2956b8640 1 --2- 172.21.15.84:0/2119096762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe290070b60 0x7fe2900ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:43.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.459+0000 7fe2966ba640 1 -- 172.21.15.84:0/2119096762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe29006c100 msgr2=0x7fe2900c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:43.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.459+0000 7fe295eb9640 1 --2- 172.21.15.84:0/2119096762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe29006c100 0x7fe2900c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:43.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.459+0000 7fe2966ba640 1 --2- 172.21.15.84:0/2119096762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe29006c100 0x7fe2900c21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:43.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.459+0000 7fe2966ba640 1 -- 172.21.15.84:0/2119096762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe290070b60 msgr2=0x7fe2900ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:43.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.459+0000 7fe2966ba640 1 --2- 172.21.15.84:0/2119096762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe290070b60 0x7fe2900ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:43.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.459+0000 7fe2966ba640 1 -- 172.21.15.84:0/2119096762 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c0094d0 con 0x7fe290159950 2023-10-16T10:05:43.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.459+0000 7fe2966ba640 1 --2- 172.21.15.84:0/2119096762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe290159950 0x7fe2900acdd0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fe28800cef0 tx=0x7fe28800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:43.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.459+0000 7fe2827fc640 1 -- 172.21.15.84:0/2119096762 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800d840 con 0x7fe290159950 2023-10-16T10:05:43.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.459+0000 7fe2827fc640 1 -- 172.21.15.84:0/2119096762 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28800d9c0 con 0x7fe290159950 2023-10-16T10:05:43.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.459+0000 7fe2827fc640 1 -- 172.21.15.84:0/2119096762 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800dd70 con 0x7fe290159950 2023-10-16T10:05:43.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.459+0000 7fe296ebb640 1 -- 172.21.15.84:0/2119096762 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29015ca30 con 0x7fe290159950 2023-10-16T10:05:43.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.459+0000 7fe296ebb640 1 -- 172.21.15.84:0/2119096762 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2900c5f20 con 0x7fe290159950 2023-10-16T10:05:43.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.461+0000 7fe296ebb640 1 -- 172.21.15.84:0/2119096762 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2900c6140 con 0x7fe290159950 2023-10-16T10:05:43.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.462+0000 7fe2827fc640 1 -- 172.21.15.84:0/2119096762 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fe28801e3b0 con 0x7fe290159950 2023-10-16T10:05:43.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.462+0000 7fe2827fc640 1 --2- 172.21.15.84:0/2119096762 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe27406af00 0x7fe27406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:43.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.463+0000 7fe2827fc640 1 -- 172.21.15.84:0/2119096762 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5631+0+0 (secure 0 0 0) 0x7fe28808a030 con 0x7fe290159950 2023-10-16T10:05:43.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.463+0000 7fe295eb9640 1 --2- 172.21.15.84:0/2119096762 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe27406af00 0x7fe27406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:43.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.463+0000 7fe295eb9640 1 --2- 172.21.15.84:0/2119096762 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe27406af00 0x7fe27406d3c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe284006fd0 tx=0x7fe284008040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:43.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.467+0000 7fe2827fc640 1 -- 172.21.15.84:0/2119096762 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe288055290 con 0x7fe290159950 2023-10-16T10:05:43.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.708+0000 7fe296ebb640 1 -- 172.21.15.84:0/2119096762 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe29004e9b0 con 0x7fe290159950 2023-10-16T10:05:43.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.709+0000 7fe2827fc640 1 -- 172.21.15.84:0/2119096762 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+3699 (secure 0 0 0) 0x7fe2880588b0 con 0x7fe290159950 2023-10-16T10:05:43.710 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:43.711 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/89534235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":89534235},{"type":"v1","addr":"172.21.15.157:6835","nonce":89534235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3807666966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3807666966},{"type":"v1","addr":"172.21.15.157:6834","nonce":3807666966}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3448417053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3448417053},{"type":"v1","addr":"172.21.15.84:6835","nonce":3448417053}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7428,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/1220944458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":1220944458},{"type":"v1","addr":"172.21.15.84:6837","nonce":1220944458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[]} 2023-10-16T10:05:43.711 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 46 2023-10-16T10:05:43.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.711+0000 7fe296ebb640 1 -- 172.21.15.84:0/2119096762 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe27406af00 msgr2=0x7fe27406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:43.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.711+0000 7fe296ebb640 1 --2- 172.21.15.84:0/2119096762 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe27406af00 0x7fe27406d3c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe284006fd0 tx=0x7fe284008040 comp rx=0 tx=0).stop 2023-10-16T10:05:43.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.711+0000 7fe296ebb640 1 -- 172.21.15.84:0/2119096762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe290159950 msgr2=0x7fe2900acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:43.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.711+0000 7fe296ebb640 1 --2- 172.21.15.84:0/2119096762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe290159950 0x7fe2900acdd0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fe28800cef0 tx=0x7fe28800a710 comp rx=0 tx=0).stop 2023-10-16T10:05:43.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.711+0000 7fe296ebb640 1 -- 172.21.15.84:0/2119096762 shutdown_connections 2023-10-16T10:05:43.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.711+0000 7fe296ebb640 1 --2- 172.21.15.84:0/2119096762 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe27406af00 0x7fe27406d3c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:43.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.711+0000 7fe296ebb640 1 --2- 172.21.15.84:0/2119096762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe290159950 0x7fe2900acdd0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:43.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.711+0000 7fe296ebb640 1 --2- 172.21.15.84:0/2119096762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe290070b60 0x7fe2900ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:43.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.711+0000 7fe296ebb640 1 --2- 172.21.15.84:0/2119096762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe29006c100 0x7fe2900c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:43.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.711+0000 7fe296ebb640 1 -- 172.21.15.84:0/2119096762 >> 172.21.15.84:0/2119096762 conn(0x7fe2900ce6a0 msgr2=0x7fe2900cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:43.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.711+0000 7fe296ebb640 1 -- 172.21.15.84:0/2119096762 shutdown_connections 2023-10-16T10:05:43.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:43.712+0000 7fe296ebb640 1 -- 172.21.15.84:0/2119096762 wait complete. 2023-10-16T10:05:43.734 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:43.730+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:43.734 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T10:05:43.734 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T10:05:43.735 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T10:05:43.735 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:05:43.735 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T10:05:43.735 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T10:05:43.736 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T10:05:43.736 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:05:43.736 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T10:05:43.737 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T10:05:43.737 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T10:05:43.737 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:05:43.737 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T10:05:43.738 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T10:05:43.738 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T10:05:43.738 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:05:43.755 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T10:05:43.755 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T10:05:43.756 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T10:05:43.757 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T10:05:43.758 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T10:05:43.761 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T10:05:43.762 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T10:05:43.762 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T10:05:43.804 INFO:tasks.ceph.mds.a:Started 2023-10-16T10:05:43.804 DEBUG:teuthology.parallel:result is None 2023-10-16T10:05:43.808 INFO:tasks.ceph.mds.b:Started 2023-10-16T10:05:43.808 DEBUG:teuthology.parallel:result is None 2023-10-16T10:05:43.811 INFO:tasks.ceph.mds.c:Started 2023-10-16T10:05:43.811 DEBUG:teuthology.parallel:result is None 2023-10-16T10:05:43.815 INFO:tasks.ceph.mds.d:Started 2023-10-16T10:05:43.815 DEBUG:teuthology.parallel:result is None 2023-10-16T10:05:43.815 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T10:05:43.968 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T10:05:43.969 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T10:05:43.973 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T10:05:43.976 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-10-16T10:05:44.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.008+0000 7f9fc5ead640 1 Processor -- start 2023-10-16T10:05:44.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.008+0000 7f9fc5ead640 1 -- start start 2023-10-16T10:05:44.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.009+0000 7f9fc5ead640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9fc015cd70 0x7f9fc00d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:44.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.009+0000 7f9fc5ead640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc00cded0 0x7f9fc00b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:44.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.009+0000 7f9fc5ead640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc015d150 con 0x7f9fc00b8480 2023-10-16T10:05:44.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.009+0000 7f9fc5ead640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc00d1180 con 0x7f9fc015cd70 2023-10-16T10:05:44.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.009+0000 7f9fc5ead640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc00d1460 con 0x7f9fc00cded0 2023-10-16T10:05:44.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.010+0000 7f9fc4eab640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9fc015cd70 0x7f9fc00d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:44.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.010+0000 7f9fc4eab640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9fc015cd70 0x7f9fc00d2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46270/0 (socket says 172.21.15.84:46270) 2023-10-16T10:05:44.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.010+0000 7f9fc4eab640 1 -- 172.21.15.84:0/3711275835 learned_addr learned my addr 172.21.15.84:0/3711275835 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:44.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.010+0000 7f9fc56ac640 1 --2- 172.21.15.84:0/3711275835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc00cded0 0x7f9fc00b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:44.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.010+0000 7f9fc4eab640 1 -- 172.21.15.84:0/3711275835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc00cded0 msgr2=0x7f9fc00b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:44.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.010+0000 7f9fc4eab640 1 --2- 172.21.15.84:0/3711275835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc00cded0 0x7f9fc00b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.010+0000 7f9fc4eab640 1 -- 172.21.15.84:0/3711275835 >> v1:172.21.15.157:6789/0 conn(0x7f9fc00b8480 legacy=0x7f9fc00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:44.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.010+0000 7f9fc4eab640 1 -- 172.21.15.84:0/3711275835 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc00d1740 con 0x7f9fc015cd70 2023-10-16T10:05:44.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.012+0000 7f9fc4eab640 1 --2- 172.21.15.84:0/3711275835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc015cd70 0x7f9fc00d2220 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009870 tx=0x7f9fb402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e73cd9e8fef363 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:44.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.012+0000 7f9fbf7fe640 1 -- 172.21.15.84:0/3711275835 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4030840 con 0x7f9fc015cd70 2023-10-16T10:05:44.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.012+0000 7f9fbf7fe640 1 -- 172.21.15.84:0/3711275835 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb40309a0 con 0x7f9fc015cd70 2023-10-16T10:05:44.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.012+0000 7f9fbf7fe640 1 -- 172.21.15.84:0/3711275835 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4030d30 con 0x7f9fc015cd70 2023-10-16T10:05:44.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.012+0000 7f9fc5ead640 1 -- 172.21.15.84:0/3711275835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc015cd70 msgr2=0x7f9fc00d2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:44.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.012+0000 7f9fc5ead640 1 --2- 172.21.15.84:0/3711275835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc015cd70 0x7f9fc00d2220 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009870 tx=0x7f9fb402b980 comp rx=0 tx=0).stop 2023-10-16T10:05:44.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.013+0000 7f9fc5ead640 1 -- 172.21.15.84:0/3711275835 shutdown_connections 2023-10-16T10:05:44.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.013+0000 7f9fc5ead640 1 --2- 172.21.15.84:0/3711275835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc00cded0 0x7f9fc00b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.013+0000 7f9fc5ead640 1 --2- 172.21.15.84:0/3711275835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc015cd70 0x7f9fc00d2220 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.013+0000 7f9fc5ead640 1 -- 172.21.15.84:0/3711275835 >> 172.21.15.84:0/3711275835 conn(0x7f9fc00cf570 msgr2=0x7f9fc00cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:44.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.013+0000 7f9fc5ead640 1 -- 172.21.15.84:0/3711275835 shutdown_connections 2023-10-16T10:05:44.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.013+0000 7f9fc5ead640 1 -- 172.21.15.84:0/3711275835 wait complete. 2023-10-16T10:05:44.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.014+0000 7f9fc5ead640 1 Processor -- start 2023-10-16T10:05:44.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.015+0000 7f9fc5ead640 1 -- start start 2023-10-16T10:05:44.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.015+0000 7f9fc5ead640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc00b8480 0x7f9fc01570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:44.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.015+0000 7f9fc5ead640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc00cded0 0x7f9fc00b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:44.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.015+0000 7f9fc5ead640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc015cd70 0x7f9fc00c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:44.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.015+0000 7f9fc5ead640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc0152f10 con 0x7f9fc00cded0 2023-10-16T10:05:44.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.015+0000 7f9fc5ead640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc0156e60 con 0x7f9fc00b8480 2023-10-16T10:05:44.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.015+0000 7f9fc5ead640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc0156b80 con 0x7f9fc015cd70 2023-10-16T10:05:44.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.016+0000 7f9fbffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc00cded0 0x7f9fc00b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:44.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.016+0000 7f9fbffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc00cded0 0x7f9fc00b5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46276/0 (socket says 172.21.15.84:46276) 2023-10-16T10:05:44.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.016+0000 7f9fbffff640 1 -- 172.21.15.84:0/621030489 learned_addr learned my addr 172.21.15.84:0/621030489 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:44.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.016+0000 7f9fc4eab640 1 --2- 172.21.15.84:0/621030489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc00b8480 0x7f9fc01570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:44.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.016+0000 7f9fc56ac640 1 --2- 172.21.15.84:0/621030489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc015cd70 0x7f9fc00c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:44.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.016+0000 7f9fbffff640 1 -- 172.21.15.84:0/621030489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc015cd70 msgr2=0x7f9fc00c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:44.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.016+0000 7f9fbffff640 1 --2- 172.21.15.84:0/621030489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc015cd70 0x7f9fc00c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.016+0000 7f9fbffff640 1 -- 172.21.15.84:0/621030489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc00b8480 msgr2=0x7f9fc01570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:44.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.016+0000 7f9fbffff640 1 --2- 172.21.15.84:0/621030489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc00b8480 0x7f9fc01570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.016+0000 7f9fbffff640 1 -- 172.21.15.84:0/621030489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb40094d0 con 0x7f9fc00cded0 2023-10-16T10:05:44.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.016+0000 7f9fc4eab640 1 --2- 172.21.15.84:0/621030489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc00b8480 0x7f9fc01570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:05:44.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.017+0000 7f9fbffff640 1 --2- 172.21.15.84:0/621030489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc00cded0 0x7f9fc00b5ec0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00c4d0 tx=0x7f9fac00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:44.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.017+0000 7f9fbd7fa640 1 -- 172.21.15.84:0/621030489 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0052d0 con 0x7f9fc00cded0 2023-10-16T10:05:44.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.017+0000 7f9fbd7fa640 1 -- 172.21.15.84:0/621030489 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac005430 con 0x7f9fc00cded0 2023-10-16T10:05:44.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.017+0000 7f9fbd7fa640 1 -- 172.21.15.84:0/621030489 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac012850 con 0x7f9fc00cded0 2023-10-16T10:05:44.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.017+0000 7f9fc5ead640 1 -- 172.21.15.84:0/621030489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc0144160 con 0x7f9fc00cded0 2023-10-16T10:05:44.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.017+0000 7f9fc5ead640 1 -- 172.21.15.84:0/621030489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc0144720 con 0x7f9fc00cded0 2023-10-16T10:05:44.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.018+0000 7f9fc5ead640 1 -- 172.21.15.84:0/621030489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc004e9b0 con 0x7f9fc00cded0 2023-10-16T10:05:44.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.019+0000 7f9fbd7fa640 1 -- 172.21.15.84:0/621030489 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f9fac0129b0 con 0x7f9fc00cded0 2023-10-16T10:05:44.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.020+0000 7f9fbd7fa640 1 --2- 172.21.15.84:0/621030489 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f9fa406af30 0x7f9fa406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:44.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.020+0000 7f9fbd7fa640 1 -- 172.21.15.84:0/621030489 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f9fac089010 con 0x7f9fc00cded0 2023-10-16T10:05:44.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.021+0000 7f9fc4eab640 1 --2- 172.21.15.84:0/621030489 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f9fa406af30 0x7f9fa406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:44.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.021+0000 7f9fc4eab640 1 --2- 172.21.15.84:0/621030489 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f9fa406af30 0x7f9fa406d3f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4036000 tx=0x7f9fb403b040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:44.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.025+0000 7f9fbd7fa640 1 -- 172.21.15.84:0/621030489 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9fac0541b0 con 0x7f9fc00cded0 2023-10-16T10:05:44.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.234+0000 7f9fc5ead640 1 -- 172.21.15.84:0/621030489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9fc01448e0 con 0x7f9fc00cded0 2023-10-16T10:05:44.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.273+0000 7f9fbd7fa640 1 -- 172.21.15.84:0/621030489 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v102) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9fac0577d0 con 0x7f9fc00cded0 2023-10-16T10:05:44.275 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-10-16T10:05:44.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.276+0000 7f9fc5ead640 1 -- 172.21.15.84:0/621030489 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f9fa406af30 msgr2=0x7f9fa406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:44.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.276+0000 7f9fc5ead640 1 --2- 172.21.15.84:0/621030489 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f9fa406af30 0x7f9fa406d3f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4036000 tx=0x7f9fb403b040 comp rx=0 tx=0).stop 2023-10-16T10:05:44.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.276+0000 7f9fc5ead640 1 -- 172.21.15.84:0/621030489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc00cded0 msgr2=0x7f9fc00b5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:44.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.276+0000 7f9fc5ead640 1 --2- 172.21.15.84:0/621030489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc00cded0 0x7f9fc00b5ec0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00c4d0 tx=0x7f9fac00c990 comp rx=0 tx=0).stop 2023-10-16T10:05:44.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.276+0000 7f9fc5ead640 1 -- 172.21.15.84:0/621030489 shutdown_connections 2023-10-16T10:05:44.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.277+0000 7f9fc5ead640 1 --2- 172.21.15.84:0/621030489 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f9fa406af30 0x7f9fa406d3f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.277+0000 7f9fc5ead640 1 --2- 172.21.15.84:0/621030489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9fc015cd70 0x7f9fc00c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.277+0000 7f9fc5ead640 1 --2- 172.21.15.84:0/621030489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc00cded0 0x7f9fc00b5ec0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.277+0000 7f9fc5ead640 1 --2- 172.21.15.84:0/621030489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc00b8480 0x7f9fc01570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.277+0000 7f9fc5ead640 1 -- 172.21.15.84:0/621030489 >> 172.21.15.84:0/621030489 conn(0x7f9fc00cf570 msgr2=0x7f9fc00b7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:44.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.277+0000 7f9fc5ead640 1 -- 172.21.15.84:0/621030489 shutdown_connections 2023-10-16T10:05:44.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.277+0000 7f9fc5ead640 1 -- 172.21.15.84:0/621030489 wait complete. 2023-10-16T10:05:44.299 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T10:05:44.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.484+0000 7f8346f27640 1 Processor -- start 2023-10-16T10:05:44.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.484+0000 7f8346f27640 1 -- start start 2023-10-16T10:05:44.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.486+0000 7f8346f27640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8340065ab0 0x7f83400c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:44.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.486+0000 7f8346f27640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83400ce280 0x7f83400ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:44.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.486+0000 7f8346f27640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83400acd70 con 0x7f834006d220 2023-10-16T10:05:44.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.486+0000 7f8346f27640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83400ac4a0 con 0x7f83400ce280 2023-10-16T10:05:44.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.486+0000 7f8346f27640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8340065f40 con 0x7f8340065ab0 2023-10-16T10:05:44.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.486+0000 7f8346726640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83400ce280 0x7f83400ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:44.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.486+0000 7f8346726640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83400ce280 0x7f83400ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46282/0 (socket says 172.21.15.84:46282) 2023-10-16T10:05:44.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.486+0000 7f8346726640 1 -- 172.21.15.84:0/2157428780 learned_addr learned my addr 172.21.15.84:0/2157428780 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:44.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.487+0000 7f8346726640 1 -- 172.21.15.84:0/2157428780 >> v2:172.21.15.157:3300/0 conn(0x7f8340065ab0 msgr2=0x7f83400c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:44.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.487+0000 7f8346726640 1 --2- 172.21.15.84:0/2157428780 >> v2:172.21.15.157:3300/0 conn(0x7f8340065ab0 0x7f83400c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.487+0000 7f8346726640 1 -- 172.21.15.84:0/2157428780 >> v1:172.21.15.157:6789/0 conn(0x7f834006d220 legacy=0x7f83400cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:44.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.487+0000 7f8346726640 1 -- 172.21.15.84:0/2157428780 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83400ac720 con 0x7f83400ce280 2023-10-16T10:05:44.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.488+0000 7f8346726640 1 --2- 172.21.15.84:0/2157428780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83400ce280 0x7f83400ad200 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f83400cb8f0 tx=0x7f833802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05fecfd9202fbe2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:44.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.488+0000 7f8344f23640 1 -- 172.21.15.84:0/2157428780 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833802c9f0 con 0x7f83400ce280 2023-10-16T10:05:44.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.488+0000 7f8344f23640 1 -- 172.21.15.84:0/2157428780 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833802cb50 con 0x7f83400ce280 2023-10-16T10:05:44.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.488+0000 7f8344f23640 1 -- 172.21.15.84:0/2157428780 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83380343f0 con 0x7f83400ce280 2023-10-16T10:05:44.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.489+0000 7f8346f27640 1 -- 172.21.15.84:0/2157428780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83400ce280 msgr2=0x7f83400ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:44.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.489+0000 7f8346f27640 1 --2- 172.21.15.84:0/2157428780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83400ce280 0x7f83400ad200 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f83400cb8f0 tx=0x7f833802be50 comp rx=0 tx=0).stop 2023-10-16T10:05:44.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.489+0000 7f8346f27640 1 -- 172.21.15.84:0/2157428780 shutdown_connections 2023-10-16T10:05:44.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.489+0000 7f8346f27640 1 --2- 172.21.15.84:0/2157428780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83400ce280 0x7f83400ad200 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.489+0000 7f8346f27640 1 --2- 172.21.15.84:0/2157428780 >> v2:172.21.15.157:3300/0 conn(0x7f8340065ab0 0x7f83400c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.489+0000 7f8346f27640 1 -- 172.21.15.84:0/2157428780 >> 172.21.15.84:0/2157428780 conn(0x7f83400b0a80 msgr2=0x7f83400b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:44.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.489+0000 7f8346f27640 1 -- 172.21.15.84:0/2157428780 shutdown_connections 2023-10-16T10:05:44.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.489+0000 7f8346f27640 1 -- 172.21.15.84:0/2157428780 wait complete. 2023-10-16T10:05:44.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.491+0000 7f8346f27640 1 Processor -- start 2023-10-16T10:05:44.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.491+0000 7f8346f27640 1 -- start start 2023-10-16T10:05:44.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.492+0000 7f8346f27640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f834006d220 0x7f834015ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:44.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.492+0000 7f8346f27640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83400ce280 0x7f834006c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:44.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.492+0000 7f8346f27640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f834015a760 0x7f8340099b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:44.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.492+0000 7f8346f27640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834006ccd0 con 0x7f83400ce280 2023-10-16T10:05:44.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.492+0000 7f8346f27640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83401605a0 con 0x7f834015a760 2023-10-16T10:05:44.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.492+0000 7f8346f27640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8340160880 con 0x7f834006d220 2023-10-16T10:05:44.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.492+0000 7f8345724640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83400ce280 0x7f834006c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:44.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.492+0000 7f8345724640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83400ce280 0x7f834006c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46288/0 (socket says 172.21.15.84:46288) 2023-10-16T10:05:44.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.492+0000 7f8345724640 1 -- 172.21.15.84:0/290984478 learned_addr learned my addr 172.21.15.84:0/290984478 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:44.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.493+0000 7f8345724640 1 -- 172.21.15.84:0/290984478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f834006d220 msgr2=0x7f834015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:44.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.493+0000 7f8345f25640 1 --2- 172.21.15.84:0/290984478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f834006d220 0x7f834015ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:44.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.493+0000 7f8346726640 1 --2- 172.21.15.84:0/290984478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f834015a760 0x7f8340099b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:44.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.493+0000 7f8345724640 1 --2- 172.21.15.84:0/290984478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f834006d220 0x7f834015ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.493+0000 7f8345724640 1 -- 172.21.15.84:0/290984478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f834015a760 msgr2=0x7f8340099b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:44.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.493+0000 7f8345724640 1 --2- 172.21.15.84:0/290984478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f834015a760 0x7f8340099b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.493+0000 7f8345724640 1 -- 172.21.15.84:0/290984478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8338009b30 con 0x7f83400ce280 2023-10-16T10:05:44.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.493+0000 7f8345724640 1 --2- 172.21.15.84:0/290984478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83400ce280 0x7f834006c790 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f833000b410 tx=0x7f833000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:44.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.493+0000 7f83367fc640 1 -- 172.21.15.84:0/290984478 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330011830 con 0x7f83400ce280 2023-10-16T10:05:44.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.493+0000 7f83367fc640 1 -- 172.21.15.84:0/290984478 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83300119b0 con 0x7f83400ce280 2023-10-16T10:05:44.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.493+0000 7f8346f27640 1 -- 172.21.15.84:0/290984478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8340160b60 con 0x7f83400ce280 2023-10-16T10:05:44.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.493+0000 7f83367fc640 1 -- 172.21.15.84:0/290984478 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330011d60 con 0x7f83400ce280 2023-10-16T10:05:44.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.493+0000 7f8346f27640 1 -- 172.21.15.84:0/290984478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8340161120 con 0x7f83400ce280 2023-10-16T10:05:44.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.494+0000 7f8346f27640 1 -- 172.21.15.84:0/290984478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f834004e9b0 con 0x7f83400ce280 2023-10-16T10:05:44.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.495+0000 7f83367fc640 1 -- 172.21.15.84:0/290984478 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f833001a400 con 0x7f83400ce280 2023-10-16T10:05:44.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.496+0000 7f83367fc640 1 --2- 172.21.15.84:0/290984478 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f832406af00 0x7f832406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:44.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.496+0000 7f83367fc640 1 -- 172.21.15.84:0/290984478 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f83300894d0 con 0x7f83400ce280 2023-10-16T10:05:44.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.496+0000 7f8345f25640 1 --2- 172.21.15.84:0/290984478 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f832406af00 0x7f832406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:44.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.497+0000 7f8345f25640 1 --2- 172.21.15.84:0/290984478 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f832406af00 0x7f832406d3c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f833c0015d0 tx=0x7f833c009040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:44.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.500+0000 7f83367fc640 1 -- 172.21.15.84:0/290984478 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8330054870 con 0x7f83400ce280 2023-10-16T10:05:44.648 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:44.645+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:44.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.752+0000 7f8346f27640 1 -- 172.21.15.84:0/290984478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f83400b4f10 con 0x7f83400ce280 2023-10-16T10:05:44.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.753+0000 7f83367fc640 1 -- 172.21.15.84:0/290984478 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8330057e90 con 0x7f83400ce280 2023-10-16T10:05:44.755 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:44.755 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T10:05:44.755 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-10-16T10:05:44.755 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.755 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-10-16T10:05:44.756 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l6dhjKRAA0pMvY6AK6qtFXDprNhwsqg==", 2023-10-16T10:05:44.756 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.756 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:05:44.756 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:05:44.756 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:05:44.757 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:44.757 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.757 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:44.757 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.757 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-10-16T10:05:44.758 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lFpMxEBAA0TLPRt5q3FwwBvMG019cjQ==", 2023-10-16T10:05:44.758 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.758 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:05:44.758 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:05:44.758 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:05:44.759 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:44.759 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.759 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:44.759 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.759 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-10-16T10:05:44.760 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l73RBOhAAOMEPxfUAMHhn5Blyvd5oMw==", 2023-10-16T10:05:44.760 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.760 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:05:44.760 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:05:44.760 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:05:44.761 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:44.761 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.761 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:44.761 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.761 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-10-16T10:05:44.761 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lcTIRIhAA0+FWm4b4yw77LD3+WKbk7w==", 2023-10-16T10:05:44.762 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.762 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:05:44.762 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:05:44.762 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:05:44.763 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:44.763 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.763 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:44.763 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.763 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-10-16T10:05:44.764 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDYBi1l1zBLERAA3YLW3xIYQoRVw7KSkwN3Og==", 2023-10-16T10:05:44.764 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.764 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:05:44.764 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:05:44.764 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:44.764 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.765 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:44.765 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.765 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-10-16T10:05:44.765 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDbBi1lFUUxKRAAQbus8SmYprJRmWTjoCznNA==", 2023-10-16T10:05:44.765 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.766 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:05:44.766 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:05:44.766 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:44.766 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.766 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:44.767 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.767 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-10-16T10:05:44.767 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDeBi1l+9lBMxAArl6sb/h15uBG/a9lVH40Hg==", 2023-10-16T10:05:44.767 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.767 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:05:44.768 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:05:44.768 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:44.768 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.768 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:44.768 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.768 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-10-16T10:05:44.769 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDiBi1l2rnEERAA3oRD71xa0fEOynM8zKuBEg==", 2023-10-16T10:05:44.769 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.769 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:05:44.769 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:05:44.769 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:44.770 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.770 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:44.770 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.770 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-10-16T10:05:44.770 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDnBi1lR7oLChAAZto8fDovDbpOd3Fcnc7C6w==", 2023-10-16T10:05:44.771 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.771 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:05:44.771 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:05:44.771 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:44.771 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.771 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:44.772 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.772 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-10-16T10:05:44.772 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDqBi1lmjg/JBAAlINKRGBc0/UWJJHMVFw57w==", 2023-10-16T10:05:44.772 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.772 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:05:44.773 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:05:44.773 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:44.773 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.773 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:44.773 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.774 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-10-16T10:05:44.774 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDuBi1lu0kMARAAtuSeM8F3yr8CVrc3p1biaQ==", 2023-10-16T10:05:44.774 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.774 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:05:44.774 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:05:44.775 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:44.775 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.775 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:44.775 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.775 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-10-16T10:05:44.776 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDxBi1lf6mNGBAAHGbqzJ0fmDxQIsasboHMfg==", 2023-10-16T10:05:44.776 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.776 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:05:44.776 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:05:44.776 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:44.777 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.777 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:44.777 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.777 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-10-16T10:05:44.777 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ==", 2023-10-16T10:05:44.778 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.778 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:05:44.778 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow r", 2023-10-16T10:05:44.778 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T10:05:44.778 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.779 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:44.779 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.779 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-10-16T10:05:44.779 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1lHU+4BxAAd8ET8hQ8Y+EUwdTJr9/+Ig==", 2023-10-16T10:05:44.779 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.780 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:05:44.780 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:05:44.780 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:05:44.780 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:05:44.780 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.781 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:44.781 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.781 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-10-16T10:05:44.781 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lA1BOORAAFNyfqgEF34MVR9YX3KBKkQ==", 2023-10-16T10:05:44.781 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.781 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:05:44.782 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:05:44.782 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:05:44.782 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:05:44.782 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.783 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:44.783 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.783 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-10-16T10:05:44.783 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1l848oERAANMfrZz9JrlgNVh7ZFbtnaA==", 2023-10-16T10:05:44.783 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.783 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:05:44.784 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:05:44.784 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:05:44.784 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:05:44.784 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.784 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:44.785 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.785 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-10-16T10:05:44.785 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDRBi1lTAxTGRAAMptxnxy3Qit/cTCXie6CrQ==", 2023-10-16T10:05:44.785 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.785 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:05:44.786 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:05:44.786 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:05:44.786 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:44.786 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.786 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:44.787 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.787 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-10-16T10:05:44.787 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lPuJxHRAABAnGPVCrkq0JR4Y6iFGZlw==", 2023-10-16T10:05:44.787 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.787 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:05:44.787 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T10:05:44.788 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:44.788 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.788 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:05:44.788 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:44.788 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-10-16T10:05:44.789 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lLxnKEBAA+a/pOHUp7Sg3m4weAust5w==", 2023-10-16T10:05:44.789 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:05:44.789 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:05:44.789 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T10:05:44.789 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:05:44.789 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.790 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:44.790 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-10-16T10:05:44.790 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T10:05:44.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.755+0000 7f8346f27640 1 -- 172.21.15.84:0/290984478 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f832406af00 msgr2=0x7f832406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:44.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.755+0000 7f8346f27640 1 --2- 172.21.15.84:0/290984478 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f832406af00 0x7f832406d3c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f833c0015d0 tx=0x7f833c009040 comp rx=0 tx=0).stop 2023-10-16T10:05:44.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.755+0000 7f8346f27640 1 -- 172.21.15.84:0/290984478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83400ce280 msgr2=0x7f834006c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:44.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.755+0000 7f8346f27640 1 --2- 172.21.15.84:0/290984478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83400ce280 0x7f834006c790 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f833000b410 tx=0x7f833000b8d0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.755+0000 7f8346f27640 1 -- 172.21.15.84:0/290984478 shutdown_connections 2023-10-16T10:05:44.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.755+0000 7f8346f27640 1 --2- 172.21.15.84:0/290984478 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f832406af00 0x7f832406d3c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.755+0000 7f8346f27640 1 --2- 172.21.15.84:0/290984478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f834015a760 0x7f8340099b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.755+0000 7f8346f27640 1 --2- 172.21.15.84:0/290984478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83400ce280 0x7f834006c790 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.755+0000 7f8346f27640 1 --2- 172.21.15.84:0/290984478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f834006d220 0x7f834015ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:44.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.755+0000 7f8346f27640 1 -- 172.21.15.84:0/290984478 >> 172.21.15.84:0/290984478 conn(0x7f83400b0a80 msgr2=0x7f834009c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:44.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.756+0000 7f8346f27640 1 -- 172.21.15.84:0/290984478 shutdown_connections 2023-10-16T10:05:44.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:44.756+0000 7f8346f27640 1 -- 172.21.15.84:0/290984478 wait complete. 2023-10-16T10:05:44.794 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:05:45.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.002+0000 7f04f1b67640 1 Processor -- start 2023-10-16T10:05:45.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.002+0000 7f04f1b67640 1 -- start start 2023-10-16T10:05:45.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.004+0000 7f04f1b67640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f04ec0b8480 0x7f04ec0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.004+0000 7f04f1b67640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04ec0d0be0 0x7f04ec0cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.004+0000 7f04f1b67640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04ec15d150 con 0x7f04ec15cd70 2023-10-16T10:05:45.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.004+0000 7f04f1b67640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04ec0d1180 con 0x7f04ec0b8480 2023-10-16T10:05:45.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.004+0000 7f04f1b67640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec0d1460 con 0x7f04ec0d0be0 2023-10-16T10:05:45.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.004+0000 7f04ebfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f04ec0b8480 0x7f04ec0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.005+0000 7f04ebfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f04ec0b8480 0x7f04ec0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46300/0 (socket says 172.21.15.84:46300) 2023-10-16T10:05:45.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.005+0000 7f04ebfff640 1 -- 172.21.15.84:0/768805632 learned_addr learned my addr 172.21.15.84:0/768805632 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:45.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.005+0000 7f04ebfff640 1 -- 172.21.15.84:0/768805632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04ec0d0be0 msgr2=0x7f04ec0cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:45.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.005+0000 7f04f1366640 1 --2- 172.21.15.84:0/768805632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04ec0d0be0 0x7f04ec0cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.005+0000 7f04ebfff640 1 --2- 172.21.15.84:0/768805632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04ec0d0be0 0x7f04ec0cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.005+0000 7f04ebfff640 1 -- 172.21.15.84:0/768805632 >> v1:172.21.15.157:6789/0 conn(0x7f04ec15cd70 legacy=0x7f04ec0d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:45.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.005+0000 7f04ebfff640 1 -- 172.21.15.84:0/768805632 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ec0d1740 con 0x7f04ec0b8480 2023-10-16T10:05:45.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.006+0000 7f04ebfff640 1 --2- 172.21.15.84:0/768805632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04ec0b8480 0x7f04ec0cd7c0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009870 tx=0x7f04d802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1358914bea27419 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:45.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.006+0000 7f04eb7fe640 1 -- 172.21.15.84:0/768805632 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d8031640 con 0x7f04ec0b8480 2023-10-16T10:05:45.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.006+0000 7f04eb7fe640 1 -- 172.21.15.84:0/768805632 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d80317a0 con 0x7f04ec0b8480 2023-10-16T10:05:45.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.006+0000 7f04eb7fe640 1 -- 172.21.15.84:0/768805632 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d8031b30 con 0x7f04ec0b8480 2023-10-16T10:05:45.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.007+0000 7f04f1b67640 1 -- 172.21.15.84:0/768805632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04ec0b8480 msgr2=0x7f04ec0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:45.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.007+0000 7f04f1b67640 1 --2- 172.21.15.84:0/768805632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04ec0b8480 0x7f04ec0cd7c0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009870 tx=0x7f04d802b8f0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.007+0000 7f04f1b67640 1 -- 172.21.15.84:0/768805632 shutdown_connections 2023-10-16T10:05:45.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.007+0000 7f04f1b67640 1 --2- 172.21.15.84:0/768805632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04ec0d0be0 0x7f04ec0cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.007+0000 7f04f1b67640 1 --2- 172.21.15.84:0/768805632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04ec0b8480 0x7f04ec0cd7c0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.007+0000 7f04f1b67640 1 -- 172.21.15.84:0/768805632 >> 172.21.15.84:0/768805632 conn(0x7f04ec0cf570 msgr2=0x7f04ec0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:45.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.007+0000 7f04f1b67640 1 -- 172.21.15.84:0/768805632 shutdown_connections 2023-10-16T10:05:45.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.007+0000 7f04f1b67640 1 -- 172.21.15.84:0/768805632 wait complete. 2023-10-16T10:05:45.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.009+0000 7f04f1b67640 1 Processor -- start 2023-10-16T10:05:45.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.009+0000 7f04f1b67640 1 -- start start 2023-10-16T10:05:45.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.010+0000 7f04f1b67640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04ec0b8480 0x7f04ec15ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.010+0000 7f04f1b67640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04ec0d0be0 0x7f04ec15d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.010+0000 7f04f1b67640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04ec15cd70 0x7f04ec0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.010+0000 7f04f1b67640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec15dce0 con 0x7f04ec15cd70 2023-10-16T10:05:45.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.010+0000 7f04f1b67640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec15dfc0 con 0x7f04ec0d0be0 2023-10-16T10:05:45.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.010+0000 7f04f1b67640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec15e2a0 con 0x7f04ec0b8480 2023-10-16T10:05:45.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.010+0000 7f04f1366640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04ec15cd70 0x7f04ec0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.010+0000 7f04f1366640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04ec15cd70 0x7f04ec0b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46312/0 (socket says 172.21.15.84:46312) 2023-10-16T10:05:45.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.010+0000 7f04f1366640 1 -- 172.21.15.84:0/2160154200 learned_addr learned my addr 172.21.15.84:0/2160154200 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:45.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.010+0000 7f04f0b65640 1 --2- 172.21.15.84:0/2160154200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04ec0b8480 0x7f04ec15ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.010+0000 7f04f1366640 1 -- 172.21.15.84:0/2160154200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04ec0b8480 msgr2=0x7f04ec15ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:45.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.011+0000 7f04f1366640 1 --2- 172.21.15.84:0/2160154200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04ec0b8480 0x7f04ec15ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.011+0000 7f04f1366640 1 -- 172.21.15.84:0/2160154200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04ec0d0be0 msgr2=0x7f04ec15d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:45.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.011+0000 7f04f1366640 1 --2- 172.21.15.84:0/2160154200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04ec0d0be0 0x7f04ec15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.011+0000 7f04f1366640 1 -- 172.21.15.84:0/2160154200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d80094d0 con 0x7f04ec15cd70 2023-10-16T10:05:45.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.011+0000 7f04f1366640 1 --2- 172.21.15.84:0/2160154200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04ec15cd70 0x7f04ec0b3e40 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f04e400cd20 tx=0x7f04e400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:45.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.011+0000 7f04e97fa640 1 -- 172.21.15.84:0/2160154200 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4003b90 con 0x7f04ec15cd70 2023-10-16T10:05:45.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.011+0000 7f04e97fa640 1 -- 172.21.15.84:0/2160154200 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e4003cf0 con 0x7f04ec15cd70 2023-10-16T10:05:45.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.011+0000 7f04e97fa640 1 -- 172.21.15.84:0/2160154200 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4004240 con 0x7f04ec15cd70 2023-10-16T10:05:45.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.011+0000 7f04f1b67640 1 -- 172.21.15.84:0/2160154200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ec15e580 con 0x7f04ec15cd70 2023-10-16T10:05:45.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.011+0000 7f04f1b67640 1 -- 172.21.15.84:0/2160154200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ec15d8a0 con 0x7f04ec15cd70 2023-10-16T10:05:45.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.012+0000 7f04f1b67640 1 -- 172.21.15.84:0/2160154200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04ec15eaa0 con 0x7f04ec15cd70 2023-10-16T10:05:45.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.013+0000 7f04e97fa640 1 -- 172.21.15.84:0/2160154200 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f04e40043a0 con 0x7f04ec15cd70 2023-10-16T10:05:45.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.014+0000 7f04e97fa640 1 --2- 172.21.15.84:0/2160154200 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f04d006af00 0x7f04d006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.014+0000 7f04e97fa640 1 -- 172.21.15.84:0/2160154200 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f04e4012070 con 0x7f04ec15cd70 2023-10-16T10:05:45.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.014+0000 7f04f0b65640 1 --2- 172.21.15.84:0/2160154200 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f04d006af00 0x7f04d006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.015+0000 7f04f0b65640 1 --2- 172.21.15.84:0/2160154200 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f04d006af00 0x7f04d006d3c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f04e000a750 tx=0x7f04e0008040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:45.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.019+0000 7f04e97fa640 1 -- 172.21.15.84:0/2160154200 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f04ec15eaa0 con 0x7f04ec15cd70 2023-10-16T10:05:45.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.208+0000 7f04f1b67640 1 -- 172.21.15.84:0/2160154200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04ec15eaa0 con 0x7f04ec15cd70 2023-10-16T10:05:45.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.209+0000 7f04e97fa640 1 -- 172.21.15.84:0/2160154200 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+12835 (secure 0 0 0) 0x7f04ec15eaa0 con 0x7f04ec15cd70 2023-10-16T10:05:45.211 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:45.211 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":104,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:05:44.286926+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:05:45.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.212+0000 7f04f1b67640 1 -- 172.21.15.84:0/2160154200 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f04d006af00 msgr2=0x7f04d006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:45.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.212+0000 7f04f1b67640 1 --2- 172.21.15.84:0/2160154200 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f04d006af00 0x7f04d006d3c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f04e000a750 tx=0x7f04e0008040 comp rx=0 tx=0).stop 2023-10-16T10:05:45.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.212+0000 7f04f1b67640 1 -- 172.21.15.84:0/2160154200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04ec15cd70 msgr2=0x7f04ec0b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:45.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.212+0000 7f04f1b67640 1 --2- 172.21.15.84:0/2160154200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04ec15cd70 0x7f04ec0b3e40 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f04e400cd20 tx=0x7f04e400a660 comp rx=0 tx=0).stop 2023-10-16T10:05:45.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.212+0000 7f04f1b67640 1 -- 172.21.15.84:0/2160154200 shutdown_connections 2023-10-16T10:05:45.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.212+0000 7f04f1b67640 1 --2- 172.21.15.84:0/2160154200 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f04d006af00 0x7f04d006d3c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.212+0000 7f04f1b67640 1 --2- 172.21.15.84:0/2160154200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04ec15cd70 0x7f04ec0b3e40 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.212+0000 7f04f1b67640 1 --2- 172.21.15.84:0/2160154200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04ec0d0be0 0x7f04ec15d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.212+0000 7f04f1b67640 1 --2- 172.21.15.84:0/2160154200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04ec0b8480 0x7f04ec15ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.212+0000 7f04f1b67640 1 -- 172.21.15.84:0/2160154200 >> 172.21.15.84:0/2160154200 conn(0x7f04ec0cf570 msgr2=0x7f04ec0b7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:45.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.212+0000 7f04f1b67640 1 -- 172.21.15.84:0/2160154200 shutdown_connections 2023-10-16T10:05:45.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.212+0000 7f04f1b67640 1 -- 172.21.15.84:0/2160154200 wait complete. 2023-10-16T10:05:45.232 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:05:45.233 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:05:45.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.432+0000 7f565721a640 1 Processor -- start 2023-10-16T10:05:45.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.432+0000 7f565721a640 1 -- start start 2023-10-16T10:05:45.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.433+0000 7f565721a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5650070f30 0x7f565006aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.433+0000 7f565721a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f565006b3e0 0x7f56500c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.434+0000 7f565721a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56500cd260 con 0x7f5650159ad0 2023-10-16T10:05:45.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.434+0000 7f565721a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f565015de30 con 0x7f565006b3e0 2023-10-16T10:05:45.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.434+0000 7f565721a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565015e110 con 0x7f5650070f30 2023-10-16T10:05:45.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.434+0000 7f5656218640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f5650159ad0 0x7f565015d650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39976/0 (socket says 172.21.15.84:39976) 2023-10-16T10:05:45.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.434+0000 7f5656218640 1 -- 172.21.15.84:0/2701563135 learned_addr learned my addr 172.21.15.84:0/2701563135 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:45.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.434+0000 7f5655216640 1 -- 172.21.15.84:0/2701563135 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3191520426 0 0) 0x7f56500cd260 con 0x7f5650159ad0 2023-10-16T10:05:45.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.434+0000 7f5655216640 1 -- 172.21.15.84:0/2701563135 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f563c005730 con 0x7f5650159ad0 2023-10-16T10:05:45.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.434+0000 7f5655a17640 1 --2- 172.21.15.84:0/2701563135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5650070f30 0x7f565006aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.434+0000 7f5656a19640 1 --2- 172.21.15.84:0/2701563135 >> v2:172.21.15.157:3300/0 conn(0x7f565006b3e0 0x7f56500c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655216640 1 -- 172.21.15.84:0/2701563135 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2125966704 0 0) 0x7f563c005730 con 0x7f5650159ad0 2023-10-16T10:05:45.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655216640 1 -- 172.21.15.84:0/2701563135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5650070f30 msgr2=0x7f565006aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:45.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655216640 1 --2- 172.21.15.84:0/2701563135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5650070f30 0x7f565006aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655216640 1 -- 172.21.15.84:0/2701563135 >> v2:172.21.15.157:3300/0 conn(0x7f565006b3e0 msgr2=0x7f56500c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:45.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655216640 1 --2- 172.21.15.84:0/2701563135 >> v2:172.21.15.157:3300/0 conn(0x7f565006b3e0 0x7f56500c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655216640 1 -- 172.21.15.84:0/2701563135 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565015e3f0 con 0x7f5650159ad0 2023-10-16T10:05:45.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655216640 1 -- 172.21.15.84:0/2701563135 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f564c001670 con 0x7f5650159ad0 2023-10-16T10:05:45.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655216640 1 -- 172.21.15.84:0/2701563135 >> v1:172.21.15.84:6789/0 conn(0x7f5650159ad0 legacy=0x7f565015d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:45.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655216640 1 --2- 172.21.15.84:0/2701563135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f563c005b80 0x7f563c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655216640 1 --2- 172.21.15.84:0/2701563135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f563c0069d0 0x7f563c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655216640 1 --2- 172.21.15.84:0/2701563135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5650070f30 0x7f563c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655216640 1 -- 172.21.15.84:0/2701563135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565015de30 con 0x7f563c0069d0 2023-10-16T10:05:45.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655216640 1 -- 172.21.15.84:0/2701563135 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565015e110 con 0x7f563c005b80 2023-10-16T10:05:45.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655216640 1 -- 172.21.15.84:0/2701563135 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c00dc90 con 0x7f5650070f30 2023-10-16T10:05:45.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655a17640 1 --2- 172.21.15.84:0/2701563135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f563c0069d0 0x7f563c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5656218640 1 --2- 172.21.15.84:0/2701563135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f563c005b80 0x7f563c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5656a19640 1 --2- 172.21.15.84:0/2701563135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5650070f30 0x7f563c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655a17640 1 -- 172.21.15.84:0/2701563135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5650070f30 msgr2=0x7f563c007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:45.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655a17640 1 --2- 172.21.15.84:0/2701563135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5650070f30 0x7f563c007390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655a17640 1 -- 172.21.15.84:0/2701563135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f563c005b80 msgr2=0x7f563c005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:45.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655a17640 1 --2- 172.21.15.84:0/2701563135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f563c005b80 0x7f563c005fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.435+0000 7f5655a17640 1 -- 172.21.15.84:0/2701563135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c005730 con 0x7f563c0069d0 2023-10-16T10:05:45.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.436+0000 7f5655a17640 1 --2- 172.21.15.84:0/2701563135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f563c0069d0 0x7f563c006e50 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f564000ba00 tx=0x7f563c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb9c3151a877b72 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:45.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.437+0000 7f5655216640 1 -- 172.21.15.84:0/2701563135 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56400325f0 con 0x7f563c0069d0 2023-10-16T10:05:45.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.437+0000 7f5655216640 1 -- 172.21.15.84:0/2701563135 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5640032750 con 0x7f563c0069d0 2023-10-16T10:05:45.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.437+0000 7f5655216640 1 -- 172.21.15.84:0/2701563135 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640032ae0 con 0x7f563c0069d0 2023-10-16T10:05:45.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.437+0000 7f565721a640 1 -- 172.21.15.84:0/2701563135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f563c0069d0 msgr2=0x7f563c006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:45.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.437+0000 7f565721a640 1 --2- 172.21.15.84:0/2701563135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f563c0069d0 0x7f563c006e50 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f564000ba00 tx=0x7f563c007a00 comp rx=0 tx=0).stop 2023-10-16T10:05:45.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.437+0000 7f5656a19640 1 -- 172.21.15.84:0/2701563135 reap_dead start 2023-10-16T10:05:45.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.437+0000 7f565721a640 1 -- 172.21.15.84:0/2701563135 shutdown_connections 2023-10-16T10:05:45.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.437+0000 7f565721a640 1 -- 172.21.15.84:0/2701563135 >> 172.21.15.84:0/2701563135 conn(0x7f56500b0170 msgr2=0x7f56500b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:45.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.437+0000 7f565721a640 1 -- 172.21.15.84:0/2701563135 shutdown_connections 2023-10-16T10:05:45.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.437+0000 7f565721a640 1 -- 172.21.15.84:0/2701563135 wait complete. 2023-10-16T10:05:45.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.439+0000 7f565721a640 1 Processor -- start 2023-10-16T10:05:45.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.439+0000 7f565721a640 1 -- start start 2023-10-16T10:05:45.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.440+0000 7f565721a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f563c0069d0 0x7f5650150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.440+0000 7f565721a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56500c6880 0x7f5650151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.440+0000 7f565721a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56500b1d50 0x7f565015c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.440+0000 7f565721a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56500c7a00 con 0x7f56500b1d50 2023-10-16T10:05:45.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.440+0000 7f565721a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56500c6df0 con 0x7f56500c6880 2023-10-16T10:05:45.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.440+0000 7f565721a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56500c70d0 con 0x7f563c0069d0 2023-10-16T10:05:45.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.441+0000 7f5656a19640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56500b1d50 0x7f565015c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.441+0000 7f5656a19640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56500b1d50 0x7f565015c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46322/0 (socket says 172.21.15.84:46322) 2023-10-16T10:05:45.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.441+0000 7f5656a19640 1 -- 172.21.15.84:0/3833436730 learned_addr learned my addr 172.21.15.84:0/3833436730 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:45.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.441+0000 7f5656218640 1 --2- 172.21.15.84:0/3833436730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f563c0069d0 0x7f5650150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.441+0000 7f5656a19640 1 -- 172.21.15.84:0/3833436730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f563c0069d0 msgr2=0x7f5650150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:45.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.441+0000 7f5656a19640 1 --2- 172.21.15.84:0/3833436730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f563c0069d0 0x7f5650150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.441+0000 7f5655a17640 1 --2- 172.21.15.84:0/3833436730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56500c6880 0x7f5650151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.441+0000 7f5656a19640 1 -- 172.21.15.84:0/3833436730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56500c6880 msgr2=0x7f5650151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:45.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.441+0000 7f5656a19640 1 --2- 172.21.15.84:0/3833436730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56500c6880 0x7f5650151b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.441+0000 7f5656a19640 1 -- 172.21.15.84:0/3833436730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56500c73b0 con 0x7f56500b1d50 2023-10-16T10:05:45.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.441+0000 7f5656a19640 1 --2- 172.21.15.84:0/3833436730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56500b1d50 0x7f565015c160 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f5650073ce0 tx=0x7f5648009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:45.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.442+0000 7f5646ffd640 1 -- 172.21.15.84:0/3833436730 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648007cf0 con 0x7f56500b1d50 2023-10-16T10:05:45.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.442+0000 7f5646ffd640 1 -- 172.21.15.84:0/3833436730 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5648007e70 con 0x7f56500b1d50 2023-10-16T10:05:45.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.442+0000 7f5646ffd640 1 -- 172.21.15.84:0/3833436730 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648014620 con 0x7f56500b1d50 2023-10-16T10:05:45.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.442+0000 7f565721a640 1 -- 172.21.15.84:0/3833436730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56500c7690 con 0x7f56500b1d50 2023-10-16T10:05:45.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.442+0000 7f565721a640 1 -- 172.21.15.84:0/3833436730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56501520d0 con 0x7f56500b1d50 2023-10-16T10:05:45.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.443+0000 7f565721a640 1 -- 172.21.15.84:0/3833436730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56500b0fd0 con 0x7f56500b1d50 2023-10-16T10:05:45.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.444+0000 7f5646ffd640 1 -- 172.21.15.84:0/3833436730 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f564801d3b0 con 0x7f56500b1d50 2023-10-16T10:05:45.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.445+0000 7f5646ffd640 1 --2- 172.21.15.84:0/3833436730 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f563406af00 0x7f563406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.446+0000 7f5646ffd640 1 -- 172.21.15.84:0/3833436730 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f56480891b0 con 0x7f56500b1d50 2023-10-16T10:05:45.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.446+0000 7f5656218640 1 --2- 172.21.15.84:0/3833436730 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f563406af00 0x7f563406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.446+0000 7f5656218640 1 --2- 172.21.15.84:0/3833436730 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f563406af00 0x7f563406d3c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f564c000c00 tx=0x7f564c009040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:45.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.451+0000 7f5646ffd640 1 -- 172.21.15.84:0/3833436730 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f56480544c0 con 0x7f56500b1d50 2023-10-16T10:05:45.604 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:45.601+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:45.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.642+0000 7f565721a640 1 -- 172.21.15.84:0/3833436730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f565004e9b0 con 0x7f56500b1d50 2023-10-16T10:05:45.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.643+0000 7f5646ffd640 1 -- 172.21.15.84:0/3833436730 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5648057ae0 con 0x7f56500b1d50 2023-10-16T10:05:45.644 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:05:45.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.645+0000 7f565721a640 1 -- 172.21.15.84:0/3833436730 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f563406af00 msgr2=0x7f563406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:45.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.645+0000 7f565721a640 1 --2- 172.21.15.84:0/3833436730 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f563406af00 0x7f563406d3c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f564c000c00 tx=0x7f564c009040 comp rx=0 tx=0).stop 2023-10-16T10:05:45.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.645+0000 7f565721a640 1 -- 172.21.15.84:0/3833436730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56500b1d50 msgr2=0x7f565015c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:45.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.645+0000 7f565721a640 1 --2- 172.21.15.84:0/3833436730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56500b1d50 0x7f565015c160 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f5650073ce0 tx=0x7f5648009d40 comp rx=0 tx=0).stop 2023-10-16T10:05:45.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.645+0000 7f565721a640 1 -- 172.21.15.84:0/3833436730 shutdown_connections 2023-10-16T10:05:45.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.645+0000 7f565721a640 1 --2- 172.21.15.84:0/3833436730 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f563406af00 0x7f563406d3c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.645+0000 7f565721a640 1 --2- 172.21.15.84:0/3833436730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56500b1d50 0x7f565015c160 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.645+0000 7f565721a640 1 --2- 172.21.15.84:0/3833436730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56500c6880 0x7f5650151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.645+0000 7f565721a640 1 --2- 172.21.15.84:0/3833436730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f563c0069d0 0x7f5650150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.645+0000 7f565721a640 1 -- 172.21.15.84:0/3833436730 >> 172.21.15.84:0/3833436730 conn(0x7f56500b0170 msgr2=0x7f565006c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:45.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.645+0000 7f565721a640 1 -- 172.21.15.84:0/3833436730 shutdown_connections 2023-10-16T10:05:45.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.646+0000 7f565721a640 1 -- 172.21.15.84:0/3833436730 wait complete. 2023-10-16T10:05:45.665 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T10:05:45.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.873+0000 7f9f65632640 1 Processor -- start 2023-10-16T10:05:45.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.873+0000 7f9f65632640 1 -- start start 2023-10-16T10:05:45.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.875+0000 7f9f65632640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9f600cdf30 0x7f9f60071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.875+0000 7f9f65632640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f600d12f0 con 0x7f9f6015cc80 2023-10-16T10:05:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.875+0000 7f9f65632640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f600d06e0 con 0x7f9f60071fa0 2023-10-16T10:05:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.875+0000 7f9f65632640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f600d0500 con 0x7f9f600cdf30 2023-10-16T10:05:45.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.875+0000 7f9f5ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9f6015cc80 0x7f9f600cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53646/0 (socket says 172.21.15.84:53646) 2023-10-16T10:05:45.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.875+0000 7f9f5ffff640 1 -- 172.21.15.84:0/1952449922 learned_addr learned my addr 172.21.15.84:0/1952449922 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:45.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.876+0000 7f9f5effd640 1 -- 172.21.15.84:0/1952449922 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3234031797 0 0) 0x7f9f600d12f0 con 0x7f9f6015cc80 2023-10-16T10:05:45.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.876+0000 7f9f5effd640 1 -- 172.21.15.84:0/1952449922 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f48003520 con 0x7f9f6015cc80 2023-10-16T10:05:45.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.876+0000 7f9f5f7fe640 1 --2- 172.21.15.84:0/1952449922 >> v2:172.21.15.157:3300/0 conn(0x7f9f600cdf30 0x7f9f60071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.876+0000 7f9f5effd640 1 -- 172.21.15.84:0/1952449922 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2728763443 0 0) 0x7f9f48003520 con 0x7f9f6015cc80 2023-10-16T10:05:45.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.876+0000 7f9f5effd640 1 -- 172.21.15.84:0/1952449922 >> v2:172.21.15.157:3300/0 conn(0x7f9f600cdf30 msgr2=0x7f9f60071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:45.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.876+0000 7f9f5effd640 1 --2- 172.21.15.84:0/1952449922 >> v2:172.21.15.157:3300/0 conn(0x7f9f600cdf30 0x7f9f60071a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.876+0000 7f9f5effd640 1 -- 172.21.15.84:0/1952449922 >> v1:172.21.15.157:6789/0 conn(0x7f9f60071fa0 legacy=0x7f9f6006aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:45.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.876+0000 7f9f5effd640 1 -- 172.21.15.84:0/1952449922 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f600d09c0 con 0x7f9f6015cc80 2023-10-16T10:05:45.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.876+0000 7f9f5effd640 1 -- 172.21.15.84:0/1952449922 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f9f50001670 con 0x7f9f6015cc80 2023-10-16T10:05:45.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.876+0000 7f9f5f7fe640 1 --2- 172.21.15.84:0/1952449922 >> v2:172.21.15.157:3300/0 conn(0x7f9f600cdf30 0x7f9f60071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:45.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.876+0000 7f9f5effd640 1 -- 172.21.15.84:0/1952449922 >> v1:172.21.15.84:6789/0 conn(0x7f9f6015cc80 legacy=0x7f9f600cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:45.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.876+0000 7f9f5effd640 1 --2- 172.21.15.84:0/1952449922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f48003970 0x7f9f48003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.876+0000 7f9f5effd640 1 --2- 172.21.15.84:0/1952449922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f480047c0 0x7f9f48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.876+0000 7f9f5effd640 1 --2- 172.21.15.84:0/1952449922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f480052a0 0x7f9f4800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.876+0000 7f9f5effd640 1 -- 172.21.15.84:0/1952449922 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f600d0500 con 0x7f9f480052a0 2023-10-16T10:05:45.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.876+0000 7f9f5effd640 1 -- 172.21.15.84:0/1952449922 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4800c1a0 con 0x7f9f480047c0 2023-10-16T10:05:45.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.876+0000 7f9f5effd640 1 -- 172.21.15.84:0/1952449922 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4800c360 con 0x7f9f48003970 2023-10-16T10:05:45.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.877+0000 7f9f64e31640 1 --2- 172.21.15.84:0/1952449922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f480052a0 0x7f9f4800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.877+0000 7f9f5f7fe640 1 --2- 172.21.15.84:0/1952449922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f480047c0 0x7f9f48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.877+0000 7f9f5ffff640 1 --2- 172.21.15.84:0/1952449922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f48003970 0x7f9f48003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.877+0000 7f9f64e31640 1 -- 172.21.15.84:0/1952449922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f48003970 msgr2=0x7f9f48003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:45.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.877+0000 7f9f64e31640 1 --2- 172.21.15.84:0/1952449922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f48003970 0x7f9f48003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.877+0000 7f9f64e31640 1 -- 172.21.15.84:0/1952449922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f480047c0 msgr2=0x7f9f48004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:45.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.877+0000 7f9f64e31640 1 --2- 172.21.15.84:0/1952449922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f480047c0 0x7f9f48004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.877+0000 7f9f64e31640 1 -- 172.21.15.84:0/1952449922 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f48003520 con 0x7f9f480052a0 2023-10-16T10:05:45.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.877+0000 7f9f5ffff640 1 --2- 172.21.15.84:0/1952449922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f48003970 0x7f9f48003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:45.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.878+0000 7f9f64e31640 1 -- 172.21.15.84:0/1952449922 reap_dead start 2023-10-16T10:05:45.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.878+0000 7f9f64e31640 1 --2- 172.21.15.84:0/1952449922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f480052a0 0x7f9f4800b980 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f9f58001e50 tx=0x7f9f480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8edbd851cd1d11 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:45.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.878+0000 7f9f5effd640 1 -- 172.21.15.84:0/1952449922 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f58003b70 con 0x7f9f480052a0 2023-10-16T10:05:45.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.878+0000 7f9f5effd640 1 -- 172.21.15.84:0/1952449922 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f58003d10 con 0x7f9f480052a0 2023-10-16T10:05:45.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.878+0000 7f9f5effd640 1 -- 172.21.15.84:0/1952449922 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f58005310 con 0x7f9f480052a0 2023-10-16T10:05:45.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.879+0000 7f9f65632640 1 -- 172.21.15.84:0/1952449922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f480052a0 msgr2=0x7f9f4800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:45.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.879+0000 7f9f65632640 1 --2- 172.21.15.84:0/1952449922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f480052a0 0x7f9f4800b980 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f9f58001e50 tx=0x7f9f480057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.879+0000 7f9f65632640 1 -- 172.21.15.84:0/1952449922 shutdown_connections 2023-10-16T10:05:45.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.879+0000 7f9f65632640 1 --2- 172.21.15.84:0/1952449922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f480052a0 0x7f9f4800b980 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.879+0000 7f9f65632640 1 -- 172.21.15.84:0/1952449922 >> 172.21.15.84:0/1952449922 conn(0x7f9f60065ed0 msgr2=0x7f9f600662e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:45.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.879+0000 7f9f65632640 1 -- 172.21.15.84:0/1952449922 shutdown_connections 2023-10-16T10:05:45.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.879+0000 7f9f65632640 1 -- 172.21.15.84:0/1952449922 wait complete. 2023-10-16T10:05:45.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.881+0000 7f9f65632640 1 Processor -- start 2023-10-16T10:05:45.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.881+0000 7f9f65632640 1 -- start start 2023-10-16T10:05:45.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.882+0000 7f9f65632640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f480052a0 0x7f9f600adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.882+0000 7f9f65632640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f6015f960 0x7f9f600ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.882+0000 7f9f65632640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f60156d50 0x7f9f600b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.882+0000 7f9f65632640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f600ad980 con 0x7f9f60156d50 2023-10-16T10:05:45.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.882+0000 7f9f65632640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f600ad0e0 con 0x7f9f6015f960 2023-10-16T10:05:45.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.882+0000 7f9f65632640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f600ad3c0 con 0x7f9f480052a0 2023-10-16T10:05:45.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.882+0000 7f9f64e31640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f60156d50 0x7f9f600b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.882+0000 7f9f64e31640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f60156d50 0x7f9f600b3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60528/0 (socket says 172.21.15.84:60528) 2023-10-16T10:05:45.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.882+0000 7f9f64e31640 1 -- 172.21.15.84:0/1722008948 learned_addr learned my addr 172.21.15.84:0/1722008948 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:45.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.883+0000 7f9f5ffff640 1 --2- 172.21.15.84:0/1722008948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f480052a0 0x7f9f600adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.883+0000 7f9f64e31640 1 -- 172.21.15.84:0/1722008948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f480052a0 msgr2=0x7f9f600adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:45.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.883+0000 7f9f5f7fe640 1 --2- 172.21.15.84:0/1722008948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f6015f960 0x7f9f600ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.883+0000 7f9f64e31640 1 --2- 172.21.15.84:0/1722008948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f480052a0 0x7f9f600adbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.883+0000 7f9f64e31640 1 -- 172.21.15.84:0/1722008948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f6015f960 msgr2=0x7f9f600ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:45.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.883+0000 7f9f64e31640 1 --2- 172.21.15.84:0/1722008948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f6015f960 0x7f9f600ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:45.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.883+0000 7f9f64e31640 1 -- 172.21.15.84:0/1722008948 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f600ad6a0 con 0x7f9f60156d50 2023-10-16T10:05:45.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.883+0000 7f9f5ffff640 1 --2- 172.21.15.84:0/1722008948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f480052a0 0x7f9f600adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:45.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.883+0000 7f9f64e31640 1 --2- 172.21.15.84:0/1722008948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f60156d50 0x7f9f600b3500 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f9f58001f70 tx=0x7f9f580014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:45.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.883+0000 7f9f5cff9640 1 -- 172.21.15.84:0/1722008948 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f58006d50 con 0x7f9f60156d50 2023-10-16T10:05:45.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.883+0000 7f9f5cff9640 1 -- 172.21.15.84:0/1722008948 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f58003b80 con 0x7f9f60156d50 2023-10-16T10:05:45.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.883+0000 7f9f5cff9640 1 -- 172.21.15.84:0/1722008948 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f58007250 con 0x7f9f60156d50 2023-10-16T10:05:45.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.883+0000 7f9f65632640 1 -- 172.21.15.84:0/1722008948 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f600ad980 con 0x7f9f60156d50 2023-10-16T10:05:45.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.884+0000 7f9f65632640 1 -- 172.21.15.84:0/1722008948 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f600ae100 con 0x7f9f60156d50 2023-10-16T10:05:45.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.885+0000 7f9f65632640 1 -- 172.21.15.84:0/1722008948 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f600b5350 con 0x7f9f60156d50 2023-10-16T10:05:45.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.886+0000 7f9f5cff9640 1 -- 172.21.15.84:0/1722008948 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f9f580073b0 con 0x7f9f60156d50 2023-10-16T10:05:45.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.886+0000 7f9f5cff9640 1 --2- 172.21.15.84:0/1722008948 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f9f4c06af00 0x7f9f4c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:45.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.886+0000 7f9f5cff9640 1 -- 172.21.15.84:0/1722008948 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9f580aacf0 con 0x7f9f60156d50 2023-10-16T10:05:45.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.887+0000 7f9f5ffff640 1 --2- 172.21.15.84:0/1722008948 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f9f4c06af00 0x7f9f4c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:45.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.887+0000 7f9f5ffff640 1 --2- 172.21.15.84:0/1722008948 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f9f4c06af00 0x7f9f4c06d3c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9f50001eb0 tx=0x7f9f50009040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:45.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:45.891+0000 7f9f5cff9640 1 -- 172.21.15.84:0/1722008948 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9f600b5350 con 0x7f9f60156d50 2023-10-16T10:05:46.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.089+0000 7f9f65632640 1 -- 172.21.15.84:0/1722008948 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9f600b5730 con 0x7f9f60156d50 2023-10-16T10:05:46.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.090+0000 7f9f5cff9640 1 -- 172.21.15.84:0/1722008948 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v104) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9f600b5730 con 0x7f9f60156d50 2023-10-16T10:05:46.092 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:46.092 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T10:05:46.092 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:46.092 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-16T10:05:46.093 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-16T10:05:46.093 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:46.093 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T10:05:46.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.092+0000 7f9f65632640 1 -- 172.21.15.84:0/1722008948 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f9f4c06af00 msgr2=0x7f9f4c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:46.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.092+0000 7f9f65632640 1 --2- 172.21.15.84:0/1722008948 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f9f4c06af00 0x7f9f4c06d3c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9f50001eb0 tx=0x7f9f50009040 comp rx=0 tx=0).stop 2023-10-16T10:05:46.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.092+0000 7f9f65632640 1 -- 172.21.15.84:0/1722008948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f60156d50 msgr2=0x7f9f600b3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:46.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.092+0000 7f9f65632640 1 --2- 172.21.15.84:0/1722008948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f60156d50 0x7f9f600b3500 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f9f58001f70 tx=0x7f9f580014a0 comp rx=0 tx=0).stop 2023-10-16T10:05:46.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.092+0000 7f9f65632640 1 -- 172.21.15.84:0/1722008948 shutdown_connections 2023-10-16T10:05:46.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.092+0000 7f9f65632640 1 --2- 172.21.15.84:0/1722008948 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f9f4c06af00 0x7f9f4c06d3c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:46.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.092+0000 7f9f65632640 1 --2- 172.21.15.84:0/1722008948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f60156d50 0x7f9f600b3500 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:46.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.092+0000 7f9f65632640 1 --2- 172.21.15.84:0/1722008948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f6015f960 0x7f9f600ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:46.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.092+0000 7f9f65632640 1 --2- 172.21.15.84:0/1722008948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f480052a0 0x7f9f600adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:46.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.092+0000 7f9f65632640 1 -- 172.21.15.84:0/1722008948 >> 172.21.15.84:0/1722008948 conn(0x7f9f60065ed0 msgr2=0x7f9f600cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:46.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.093+0000 7f9f65632640 1 -- 172.21.15.84:0/1722008948 shutdown_connections 2023-10-16T10:05:46.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.093+0000 7f9f65632640 1 -- 172.21.15.84:0/1722008948 wait complete. 2023-10-16T10:05:46.113 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T10:05:46.113 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T10:05:46.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.326+0000 7fa12f59e640 1 Processor -- start 2023-10-16T10:05:46.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.327+0000 7fa12f59e640 1 -- start start 2023-10-16T10:05:46.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.328+0000 7fa12f59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa13015cd20 0x7fa1300cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:46.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.328+0000 7fa12f59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1300b4ca0 0x7fa1300d06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:46.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.328+0000 7fa12f59e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1300b8430 con 0x7fa13015d100 2023-10-16T10:05:46.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.328+0000 7fa12f59e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1300d2160 con 0x7fa13015cd20 2023-10-16T10:05:46.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.328+0000 7fa12f59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1300ce0b0 con 0x7fa1300b4ca0 2023-10-16T10:05:46.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.328+0000 7fa12dd9b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa13015d100 0x7fa1300b4590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53654/0 (socket says 172.21.15.84:53654) 2023-10-16T10:05:46.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.328+0000 7fa12dd9b640 1 -- 172.21.15.84:0/837652002 learned_addr learned my addr 172.21.15.84:0/837652002 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:46.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.329+0000 7fa12d59a640 1 -- 172.21.15.84:0/837652002 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904420306 0 0) 0x7fa1300b8430 con 0x7fa13015d100 2023-10-16T10:05:46.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.329+0000 7fa12d59a640 1 -- 172.21.15.84:0/837652002 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa118003520 con 0x7fa13015d100 2023-10-16T10:05:46.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.329+0000 7fa12e59c640 1 --2- 172.21.15.84:0/837652002 >> v2:172.21.15.157:3300/0 conn(0x7fa13015cd20 0x7fa1300cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:46.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.329+0000 7fa12ed9d640 1 --2- 172.21.15.84:0/837652002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1300b4ca0 0x7fa1300d06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:46.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.329+0000 7fa12d59a640 1 -- 172.21.15.84:0/837652002 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 738868088 0 0) 0x7fa118003520 con 0x7fa13015d100 2023-10-16T10:05:46.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.329+0000 7fa12d59a640 1 -- 172.21.15.84:0/837652002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1300b4ca0 msgr2=0x7fa1300d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:46.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.329+0000 7fa12d59a640 1 --2- 172.21.15.84:0/837652002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1300b4ca0 0x7fa1300d06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:46.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.329+0000 7fa12d59a640 1 -- 172.21.15.84:0/837652002 >> v2:172.21.15.157:3300/0 conn(0x7fa13015cd20 msgr2=0x7fa1300cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:46.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.329+0000 7fa12d59a640 1 --2- 172.21.15.84:0/837652002 >> v2:172.21.15.157:3300/0 conn(0x7fa13015cd20 0x7fa1300cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:46.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.329+0000 7fa12d59a640 1 -- 172.21.15.84:0/837652002 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1300b8680 con 0x7fa13015d100 2023-10-16T10:05:46.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.329+0000 7fa12d59a640 1 -- 172.21.15.84:0/837652002 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fa124001670 con 0x7fa13015d100 2023-10-16T10:05:46.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.329+0000 7fa12d59a640 1 -- 172.21.15.84:0/837652002 >> v1:172.21.15.84:6789/0 conn(0x7fa13015d100 legacy=0x7fa1300b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:46.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.329+0000 7fa12d59a640 1 --2- 172.21.15.84:0/837652002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1300b4ca0 0x7fa1180039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:46.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.329+0000 7fa12d59a640 1 --2- 172.21.15.84:0/837652002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa118004010 0x7fa1180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:46.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.329+0000 7fa12d59a640 1 --2- 172.21.15.84:0/837652002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa118004d00 0x7fa118005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:46.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.329+0000 7fa12d59a640 1 -- 172.21.15.84:0/837652002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1300d2160 con 0x7fa118004d00 2023-10-16T10:05:46.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.329+0000 7fa12d59a640 1 -- 172.21.15.84:0/837652002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1300ce0b0 con 0x7fa118004010 2023-10-16T10:05:46.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.329+0000 7fa12d59a640 1 -- 172.21.15.84:0/837652002 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11800bc80 con 0x7fa1300b4ca0 2023-10-16T10:05:46.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.330+0000 7fa12ed9d640 1 --2- 172.21.15.84:0/837652002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa118004d00 0x7fa118005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:46.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.330+0000 7fa12e59c640 1 --2- 172.21.15.84:0/837652002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1300b4ca0 0x7fa1180039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:46.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.330+0000 7fa12dd9b640 1 --2- 172.21.15.84:0/837652002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa118004010 0x7fa1180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:46.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.330+0000 7fa12ed9d640 1 -- 172.21.15.84:0/837652002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1300b4ca0 msgr2=0x7fa1180039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:46.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.330+0000 7fa12ed9d640 1 --2- 172.21.15.84:0/837652002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1300b4ca0 0x7fa1180039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:46.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.330+0000 7fa12ed9d640 1 -- 172.21.15.84:0/837652002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa118004010 msgr2=0x7fa1180047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:46.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.330+0000 7fa12ed9d640 1 --2- 172.21.15.84:0/837652002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa118004010 0x7fa1180047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:46.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.330+0000 7fa12ed9d640 1 -- 172.21.15.84:0/837652002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa118003520 con 0x7fa118004d00 2023-10-16T10:05:46.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.331+0000 7fa12ed9d640 1 --2- 172.21.15.84:0/837652002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa118004d00 0x7fa118005160 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fa128007f30 tx=0x7fa1180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95746199c1d115b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:46.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.331+0000 7fa12d59a640 1 -- 172.21.15.84:0/837652002 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa128033600 con 0x7fa118004d00 2023-10-16T10:05:46.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.332+0000 7fa12d59a640 1 -- 172.21.15.84:0/837652002 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12802f050 con 0x7fa118004d00 2023-10-16T10:05:46.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.332+0000 7fa12d59a640 1 -- 172.21.15.84:0/837652002 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1280339a0 con 0x7fa118004d00 2023-10-16T10:05:46.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.332+0000 7fa12f59e640 1 -- 172.21.15.84:0/837652002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa118004d00 msgr2=0x7fa118005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:46.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.332+0000 7fa12f59e640 1 --2- 172.21.15.84:0/837652002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa118004d00 0x7fa118005160 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fa128007f30 tx=0x7fa1180057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:46.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.332+0000 7fa12ed9d640 1 -- 172.21.15.84:0/837652002 reap_dead start 2023-10-16T10:05:46.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.332+0000 7fa12f59e640 1 -- 172.21.15.84:0/837652002 shutdown_connections 2023-10-16T10:05:46.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.332+0000 7fa12f59e640 1 -- 172.21.15.84:0/837652002 >> 172.21.15.84:0/837652002 conn(0x7fa1300cf590 msgr2=0x7fa1300cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:46.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.332+0000 7fa12f59e640 1 -- 172.21.15.84:0/837652002 shutdown_connections 2023-10-16T10:05:46.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.332+0000 7fa12f59e640 1 -- 172.21.15.84:0/837652002 wait complete. 2023-10-16T10:05:46.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.334+0000 7fa12f59e640 1 Processor -- start 2023-10-16T10:05:46.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.334+0000 7fa12f59e640 1 -- start start 2023-10-16T10:05:46.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.335+0000 7fa12f59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa118004d00 0x7fa1301f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:46.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.335+0000 7fa12f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1301f63e0 0x7fa1301f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:46.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.335+0000 7fa12f59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1301f2f60 0x7fa1301427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:46.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.335+0000 7fa12f59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1301f6860 con 0x7fa1301f63e0 2023-10-16T10:05:46.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.335+0000 7fa12f59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13015df10 con 0x7fa118004d00 2023-10-16T10:05:46.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.335+0000 7fa12f59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13015f4c0 con 0x7fa1301f2f60 2023-10-16T10:05:46.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.336+0000 7fa12dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1301f63e0 0x7fa1301f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:46.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.336+0000 7fa12dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1301f63e0 0x7fa1301f2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60550/0 (socket says 172.21.15.84:60550) 2023-10-16T10:05:46.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.336+0000 7fa12dd9b640 1 -- 172.21.15.84:0/601403037 learned_addr learned my addr 172.21.15.84:0/601403037 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:46.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.336+0000 7fa12e59c640 1 --2- 172.21.15.84:0/601403037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa118004d00 0x7fa1301f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:46.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.336+0000 7fa12dd9b640 1 -- 172.21.15.84:0/601403037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1301f2f60 msgr2=0x7fa1301427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:46.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.336+0000 7fa12ed9d640 1 --2- 172.21.15.84:0/601403037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1301f2f60 0x7fa1301427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:46.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.336+0000 7fa12dd9b640 1 --2- 172.21.15.84:0/601403037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1301f2f60 0x7fa1301427d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:46.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.336+0000 7fa12dd9b640 1 -- 172.21.15.84:0/601403037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa118004d00 msgr2=0x7fa1301f5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:46.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.336+0000 7fa12dd9b640 1 --2- 172.21.15.84:0/601403037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa118004d00 0x7fa1301f5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:46.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.336+0000 7fa12dd9b640 1 -- 172.21.15.84:0/601403037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13015f730 con 0x7fa1301f63e0 2023-10-16T10:05:46.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.336+0000 7fa12dd9b640 1 --2- 172.21.15.84:0/601403037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1301f63e0 0x7fa1301f2a20 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fa1240017a0 tx=0x7fa12400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:46.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.337+0000 7fa11effd640 1 -- 172.21.15.84:0/601403037 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1240074e0 con 0x7fa1301f63e0 2023-10-16T10:05:46.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.337+0000 7fa11effd640 1 -- 172.21.15.84:0/601403037 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa124007640 con 0x7fa1301f63e0 2023-10-16T10:05:46.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.337+0000 7fa11effd640 1 -- 172.21.15.84:0/601403037 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1240049e0 con 0x7fa1301f63e0 2023-10-16T10:05:46.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.337+0000 7fa12f59e640 1 -- 172.21.15.84:0/601403037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13015fa10 con 0x7fa1301f63e0 2023-10-16T10:05:46.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.337+0000 7fa12f59e640 1 -- 172.21.15.84:0/601403037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1301f37a0 con 0x7fa1301f63e0 2023-10-16T10:05:46.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.338+0000 7fa12f59e640 1 -- 172.21.15.84:0/601403037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13015ff30 con 0x7fa1301f63e0 2023-10-16T10:05:46.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.339+0000 7fa11effd640 1 -- 172.21.15.84:0/601403037 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa124004b40 con 0x7fa1301f63e0 2023-10-16T10:05:46.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.340+0000 7fa11effd640 1 --2- 172.21.15.84:0/601403037 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fa11006af30 0x7fa11006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:46.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.340+0000 7fa11effd640 1 -- 172.21.15.84:0/601403037 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa124088de0 con 0x7fa1301f63e0 2023-10-16T10:05:46.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.340+0000 7fa12e59c640 1 --2- 172.21.15.84:0/601403037 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fa11006af30 0x7fa11006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:46.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.341+0000 7fa12e59c640 1 --2- 172.21.15.84:0/601403037 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fa11006af30 0x7fa11006d3f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa1300d1580 tx=0x7fa120008040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:46.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.344+0000 7fa11effd640 1 -- 172.21.15.84:0/601403037 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa13015ff30 con 0x7fa1301f63e0 2023-10-16T10:05:46.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:46.544+0000 7fa12f59e640 1 -- 172.21.15.84:0/601403037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa13015ff30 con 0x7fa1301f63e0 2023-10-16T10:05:46.568 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:46.566+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:47.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.304+0000 7fa11effd640 1 -- 172.21.15.84:0/601403037 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v105) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa13015ff30 con 0x7fa1301f63e0 2023-10-16T10:05:47.306 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-16T10:05:47.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.309+0000 7fa12f59e640 1 -- 172.21.15.84:0/601403037 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fa11006af30 msgr2=0x7fa11006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:47.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.309+0000 7fa12f59e640 1 --2- 172.21.15.84:0/601403037 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fa11006af30 0x7fa11006d3f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa1300d1580 tx=0x7fa120008040 comp rx=0 tx=0).stop 2023-10-16T10:05:47.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.309+0000 7fa12f59e640 1 -- 172.21.15.84:0/601403037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1301f63e0 msgr2=0x7fa1301f2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:47.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.309+0000 7fa12f59e640 1 --2- 172.21.15.84:0/601403037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1301f63e0 0x7fa1301f2a20 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fa1240017a0 tx=0x7fa12400dc90 comp rx=0 tx=0).stop 2023-10-16T10:05:47.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.309+0000 7fa12f59e640 1 -- 172.21.15.84:0/601403037 shutdown_connections 2023-10-16T10:05:47.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.309+0000 7fa12f59e640 1 --2- 172.21.15.84:0/601403037 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fa11006af30 0x7fa11006d3f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:47.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.309+0000 7fa12f59e640 1 --2- 172.21.15.84:0/601403037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1301f2f60 0x7fa1301427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:47.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.310+0000 7fa12f59e640 1 --2- 172.21.15.84:0/601403037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1301f63e0 0x7fa1301f2a20 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:47.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.310+0000 7fa12f59e640 1 --2- 172.21.15.84:0/601403037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa118004d00 0x7fa1301f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:47.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.310+0000 7fa12f59e640 1 -- 172.21.15.84:0/601403037 >> 172.21.15.84:0/601403037 conn(0x7fa1300cf590 msgr2=0x7fa13015ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:47.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.312+0000 7fa12f59e640 1 -- 172.21.15.84:0/601403037 shutdown_connections 2023-10-16T10:05:47.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.313+0000 7fa12f59e640 1 -- 172.21.15.84:0/601403037 wait complete. 2023-10-16T10:05:47.339 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T10:05:47.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.551+0000 7ff520cdf640 1 Processor -- start 2023-10-16T10:05:47.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.551+0000 7ff520cdf640 1 -- start start 2023-10-16T10:05:47.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.552+0000 7ff520cdf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c0b4c80 0x7ff51c167b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:47.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.552+0000 7ff520cdf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff51c0cdbf0 con 0x7ff51c0b1690 2023-10-16T10:05:47.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.552+0000 7ff520cdf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff51c0cde90 con 0x7ff51c15cb50 2023-10-16T10:05:47.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.552+0000 7ff520cdf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c0ce170 con 0x7ff51c0b4c80 2023-10-16T10:05:47.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.553+0000 7ff51affd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff51c0b1690 0x7ff51c0b4570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53668/0 (socket says 172.21.15.84:53668) 2023-10-16T10:05:47.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.553+0000 7ff51affd640 1 -- 172.21.15.84:0/3919406697 learned_addr learned my addr 172.21.15.84:0/3919406697 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:47.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.553+0000 7ff51a7fc640 1 -- 172.21.15.84:0/3919406697 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1576467220 0 0) 0x7ff51c0cdbf0 con 0x7ff51c0b1690 2023-10-16T10:05:47.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.553+0000 7ff51bfff640 1 --2- 172.21.15.84:0/3919406697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c0b4c80 0x7ff51c167b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:47.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.553+0000 7ff51a7fc640 1 -- 172.21.15.84:0/3919406697 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff504003520 con 0x7ff51c0b1690 2023-10-16T10:05:47.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.554+0000 7ff51a7fc640 1 -- 172.21.15.84:0/3919406697 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3379035867 0 0) 0x7ff504003520 con 0x7ff51c0b1690 2023-10-16T10:05:47.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.554+0000 7ff51a7fc640 1 -- 172.21.15.84:0/3919406697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c0b4c80 msgr2=0x7ff51c167b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:47.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.554+0000 7ff51a7fc640 1 --2- 172.21.15.84:0/3919406697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c0b4c80 0x7ff51c167b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:47.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.554+0000 7ff51a7fc640 1 -- 172.21.15.84:0/3919406697 >> v1:172.21.15.157:6789/0 conn(0x7ff51c15cb50 legacy=0x7ff51c161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:47.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.554+0000 7ff51a7fc640 1 -- 172.21.15.84:0/3919406697 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c0d0f30 con 0x7ff51c0b1690 2023-10-16T10:05:47.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.554+0000 7ff51a7fc640 1 -- 172.21.15.84:0/3919406697 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ff510001670 con 0x7ff51c0b1690 2023-10-16T10:05:47.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.554+0000 7ff51a7fc640 1 -- 172.21.15.84:0/3919406697 >> v1:172.21.15.84:6789/0 conn(0x7ff51c0b1690 legacy=0x7ff51c0b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:47.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.554+0000 7ff51a7fc640 1 --2- 172.21.15.84:0/3919406697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5040039a0 0x7ff504003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:47.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.554+0000 7ff51a7fc640 1 --2- 172.21.15.84:0/3919406697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5040047c0 0x7ff504004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:47.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.554+0000 7ff51a7fc640 1 --2- 172.21.15.84:0/3919406697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c0b4c80 0x7ff50400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:47.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.554+0000 7ff51a7fc640 1 -- 172.21.15.84:0/3919406697 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c0ce170 con 0x7ff5040039a0 2023-10-16T10:05:47.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.554+0000 7ff51a7fc640 1 -- 172.21.15.84:0/3919406697 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff504005500 con 0x7ff5040047c0 2023-10-16T10:05:47.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.554+0000 7ff51a7fc640 1 -- 172.21.15.84:0/3919406697 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50400bec0 con 0x7ff51c0b4c80 2023-10-16T10:05:47.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.554+0000 7ff51b7fe640 1 --2- 172.21.15.84:0/3919406697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5040039a0 0x7ff504003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:47.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.554+0000 7ff51affd640 1 --2- 172.21.15.84:0/3919406697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5040047c0 0x7ff504004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:47.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.555+0000 7ff51bfff640 1 --2- 172.21.15.84:0/3919406697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c0b4c80 0x7ff50400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:47.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.555+0000 7ff51b7fe640 1 -- 172.21.15.84:0/3919406697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c0b4c80 msgr2=0x7ff50400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:47.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.555+0000 7ff51b7fe640 1 --2- 172.21.15.84:0/3919406697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c0b4c80 0x7ff50400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:47.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.555+0000 7ff51b7fe640 1 -- 172.21.15.84:0/3919406697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5040047c0 msgr2=0x7ff504004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:47.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.555+0000 7ff51b7fe640 1 --2- 172.21.15.84:0/3919406697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5040047c0 0x7ff504004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:47.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.555+0000 7ff51b7fe640 1 -- 172.21.15.84:0/3919406697 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff504003520 con 0x7ff5040039a0 2023-10-16T10:05:47.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.556+0000 7ff51b7fe640 1 --2- 172.21.15.84:0/3919406697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5040039a0 0x7ff504003dc0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7ff50c001750 tx=0x7ff5040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66c5acc0dcb5ade server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:47.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.556+0000 7ff51a7fc640 1 -- 172.21.15.84:0/3919406697 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0052d0 con 0x7ff5040039a0 2023-10-16T10:05:47.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.556+0000 7ff51a7fc640 1 -- 172.21.15.84:0/3919406697 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50c02e050 con 0x7ff5040039a0 2023-10-16T10:05:47.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.556+0000 7ff51a7fc640 1 -- 172.21.15.84:0/3919406697 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0035f0 con 0x7ff5040039a0 2023-10-16T10:05:47.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.557+0000 7ff520cdf640 1 -- 172.21.15.84:0/3919406697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5040039a0 msgr2=0x7ff504003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:47.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.557+0000 7ff520cdf640 1 --2- 172.21.15.84:0/3919406697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5040039a0 0x7ff504003dc0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7ff50c001750 tx=0x7ff5040057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:47.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.557+0000 7ff51bfff640 1 -- 172.21.15.84:0/3919406697 reap_dead start 2023-10-16T10:05:47.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.557+0000 7ff520cdf640 1 -- 172.21.15.84:0/3919406697 shutdown_connections 2023-10-16T10:05:47.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.557+0000 7ff520cdf640 1 -- 172.21.15.84:0/3919406697 >> 172.21.15.84:0/3919406697 conn(0x7ff51c0cf570 msgr2=0x7ff51c0cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:47.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.557+0000 7ff520cdf640 1 -- 172.21.15.84:0/3919406697 shutdown_connections 2023-10-16T10:05:47.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.557+0000 7ff520cdf640 1 -- 172.21.15.84:0/3919406697 wait complete. 2023-10-16T10:05:47.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.559+0000 7ff520cdf640 1 Processor -- start 2023-10-16T10:05:47.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.559+0000 7ff520cdf640 1 -- start start 2023-10-16T10:05:47.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.560+0000 7ff520cdf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5040039a0 0x7ff51c1500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:47.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.560+0000 7ff520cdf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff51c0c9bc0 0x7ff51c0c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:47.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.560+0000 7ff520cdf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c15e310 0x7ff51c15e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:47.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.560+0000 7ff520cdf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c15ecd0 con 0x7ff51c0c9bc0 2023-10-16T10:05:47.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.560+0000 7ff520cdf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c0ca130 con 0x7ff5040039a0 2023-10-16T10:05:47.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.560+0000 7ff520cdf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c0ca410 con 0x7ff51c15e310 2023-10-16T10:05:47.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.560+0000 7ff51affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff51c0c9bc0 0x7ff51c0c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:47.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.560+0000 7ff51affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff51c0c9bc0 0x7ff51c0c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60566/0 (socket says 172.21.15.84:60566) 2023-10-16T10:05:47.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.560+0000 7ff51affd640 1 -- 172.21.15.84:0/3840554239 learned_addr learned my addr 172.21.15.84:0/3840554239 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:47.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.561+0000 7ff51b7fe640 1 --2- 172.21.15.84:0/3840554239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5040039a0 0x7ff51c1500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:47.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.561+0000 7ff51bfff640 1 --2- 172.21.15.84:0/3840554239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c15e310 0x7ff51c15e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:47.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.561+0000 7ff51affd640 1 -- 172.21.15.84:0/3840554239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c15e310 msgr2=0x7ff51c15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:47.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.561+0000 7ff51affd640 1 --2- 172.21.15.84:0/3840554239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c15e310 0x7ff51c15e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:47.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.561+0000 7ff51affd640 1 -- 172.21.15.84:0/3840554239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5040039a0 msgr2=0x7ff51c1500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:47.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.561+0000 7ff51affd640 1 --2- 172.21.15.84:0/3840554239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5040039a0 0x7ff51c1500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:47.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.561+0000 7ff51affd640 1 -- 172.21.15.84:0/3840554239 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c0ca6f0 con 0x7ff51c0c9bc0 2023-10-16T10:05:47.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.561+0000 7ff51b7fe640 1 --2- 172.21.15.84:0/3840554239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5040039a0 0x7ff51c1500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:47.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.561+0000 7ff51affd640 1 --2- 172.21.15.84:0/3840554239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff51c0c9bc0 0x7ff51c0c92c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff51000e700 tx=0x7ff51000ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:47.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.561+0000 7ff503fff640 1 -- 172.21.15.84:0/3840554239 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51000c6c0 con 0x7ff51c0c9bc0 2023-10-16T10:05:47.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.561+0000 7ff503fff640 1 -- 172.21.15.84:0/3840554239 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51000c820 con 0x7ff51c0c9bc0 2023-10-16T10:05:47.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.561+0000 7ff503fff640 1 -- 172.21.15.84:0/3840554239 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51000cbd0 con 0x7ff51c0c9bc0 2023-10-16T10:05:47.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.561+0000 7ff520cdf640 1 -- 172.21.15.84:0/3840554239 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff51c0ca9d0 con 0x7ff51c0c9bc0 2023-10-16T10:05:47.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.562+0000 7ff520cdf640 1 -- 172.21.15.84:0/3840554239 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51c0c99e0 con 0x7ff51c0c9bc0 2023-10-16T10:05:47.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.563+0000 7ff520cdf640 1 -- 172.21.15.84:0/3840554239 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51c0cab60 con 0x7ff51c0c9bc0 2023-10-16T10:05:47.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.564+0000 7ff503fff640 1 -- 172.21.15.84:0/3840554239 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7ff51000cd30 con 0x7ff51c0c9bc0 2023-10-16T10:05:47.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.565+0000 7ff503fff640 1 --2- 172.21.15.84:0/3840554239 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7ff50806af00 0x7ff50806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:47.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.565+0000 7ff503fff640 1 -- 172.21.15.84:0/3840554239 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff510089400 con 0x7ff51c0c9bc0 2023-10-16T10:05:47.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.565+0000 7ff51b7fe640 1 --2- 172.21.15.84:0/3840554239 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7ff50806af00 0x7ff50806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:47.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.566+0000 7ff51b7fe640 1 --2- 172.21.15.84:0/3840554239 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7ff50806af00 0x7ff50806d3c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff50c033000 tx=0x7ff50c009030 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:47.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.570+0000 7ff503fff640 1 -- 172.21.15.84:0/3840554239 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff5100545a0 con 0x7ff51c0c9bc0 2023-10-16T10:05:47.605 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:47.602+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:47.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:47.772+0000 7ff520cdf640 1 -- 172.21.15.84:0/3840554239 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff51c0d26c0 con 0x7ff51c0c9bc0 2023-10-16T10:05:48.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.317+0000 7ff503fff640 1 -- 172.21.15.84:0/3840554239 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v106) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff510057bc0 con 0x7ff51c0c9bc0 2023-10-16T10:05:48.319 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-16T10:05:48.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.321+0000 7ff520cdf640 1 -- 172.21.15.84:0/3840554239 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7ff50806af00 msgr2=0x7ff50806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:48.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.321+0000 7ff520cdf640 1 --2- 172.21.15.84:0/3840554239 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7ff50806af00 0x7ff50806d3c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff50c033000 tx=0x7ff50c009030 comp rx=0 tx=0).stop 2023-10-16T10:05:48.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.321+0000 7ff520cdf640 1 -- 172.21.15.84:0/3840554239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff51c0c9bc0 msgr2=0x7ff51c0c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:48.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.321+0000 7ff520cdf640 1 --2- 172.21.15.84:0/3840554239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff51c0c9bc0 0x7ff51c0c92c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff51000e700 tx=0x7ff51000ebc0 comp rx=0 tx=0).stop 2023-10-16T10:05:48.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.321+0000 7ff520cdf640 1 -- 172.21.15.84:0/3840554239 shutdown_connections 2023-10-16T10:05:48.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.321+0000 7ff520cdf640 1 --2- 172.21.15.84:0/3840554239 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7ff50806af00 0x7ff50806d3c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:48.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.322+0000 7ff520cdf640 1 --2- 172.21.15.84:0/3840554239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c15e310 0x7ff51c15e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:48.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.322+0000 7ff520cdf640 1 --2- 172.21.15.84:0/3840554239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff51c0c9bc0 0x7ff51c0c92c0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:48.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.322+0000 7ff520cdf640 1 --2- 172.21.15.84:0/3840554239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5040039a0 0x7ff51c1500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:48.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.322+0000 7ff520cdf640 1 -- 172.21.15.84:0/3840554239 >> 172.21.15.84:0/3840554239 conn(0x7ff51c0cf570 msgr2=0x7ff51c0b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:48.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.323+0000 7ff520cdf640 1 -- 172.21.15.84:0/3840554239 shutdown_connections 2023-10-16T10:05:48.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.323+0000 7ff520cdf640 1 -- 172.21.15.84:0/3840554239 wait complete. 2023-10-16T10:05:48.356 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T10:05:48.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.545+0000 7fe7edcbb640 1 Processor -- start 2023-10-16T10:05:48.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.545+0000 7fe7edcbb640 1 -- start start 2023-10-16T10:05:48.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.546+0000 7fe7edcbb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe7e8159ab0 0x7fe7e80ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:48.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.546+0000 7fe7edcbb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e8070360 con 0x7fe7e8159e90 2023-10-16T10:05:48.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.546+0000 7fe7edcbb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e80cd1f0 con 0x7fe7e806c140 2023-10-16T10:05:48.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.546+0000 7fe7edcbb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7e815d890 con 0x7fe7e8159ab0 2023-10-16T10:05:48.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.546+0000 7fe7e77fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe7e8159ab0 0x7fe7e80ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:48.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.546+0000 7fe7dffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe7e8159e90 0x7fe7e806fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53678/0 (socket says 172.21.15.84:53678) 2023-10-16T10:05:48.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.547+0000 7fe7dffff640 1 -- 172.21.15.84:0/2088338394 learned_addr learned my addr 172.21.15.84:0/2088338394 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:48.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.547+0000 7fe7e77fe640 1 -- 172.21.15.84:0/2088338394 >> v1:172.21.15.157:6789/0 conn(0x7fe7e806c140 legacy=0x7fe7e815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:48.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.547+0000 7fe7e6ffd640 1 -- 172.21.15.84:0/2088338394 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807612777 0 0) 0x7fe7e8070360 con 0x7fe7e8159e90 2023-10-16T10:05:48.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.547+0000 7fe7e77fe640 1 -- 172.21.15.84:0/2088338394 >> v1:172.21.15.84:6789/0 conn(0x7fe7e8159e90 legacy=0x7fe7e806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:48.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.547+0000 7fe7e77fe640 1 -- 172.21.15.84:0/2088338394 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e80cd4d0 con 0x7fe7e8159ab0 2023-10-16T10:05:48.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.548+0000 7fe7e77fe640 1 --2- 172.21.15.84:0/2088338394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e8159ab0 0x7fe7e80ce370 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80097a0 tx=0x7fe7d802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338c0f749265cd59 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:48.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.549+0000 7fe7e6ffd640 1 -- 172.21.15.84:0/2088338394 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d802f050 con 0x7fe7e8159ab0 2023-10-16T10:05:48.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.549+0000 7fe7e6ffd640 1 -- 172.21.15.84:0/2088338394 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d8031700 con 0x7fe7e8159ab0 2023-10-16T10:05:48.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.549+0000 7fe7e6ffd640 1 -- 172.21.15.84:0/2088338394 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8031a90 con 0x7fe7e8159ab0 2023-10-16T10:05:48.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.549+0000 7fe7edcbb640 1 -- 172.21.15.84:0/2088338394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e8159ab0 msgr2=0x7fe7e80ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:48.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.549+0000 7fe7edcbb640 1 --2- 172.21.15.84:0/2088338394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e8159ab0 0x7fe7e80ce370 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80097a0 tx=0x7fe7d802b7a0 comp rx=0 tx=0).stop 2023-10-16T10:05:48.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.549+0000 7fe7edcbb640 1 -- 172.21.15.84:0/2088338394 shutdown_connections 2023-10-16T10:05:48.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.549+0000 7fe7edcbb640 1 --2- 172.21.15.84:0/2088338394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e8159ab0 0x7fe7e80ce370 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:48.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.549+0000 7fe7edcbb640 1 -- 172.21.15.84:0/2088338394 >> 172.21.15.84:0/2088338394 conn(0x7fe7e80ce9b0 msgr2=0x7fe7e80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:48.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.549+0000 7fe7edcbb640 1 -- 172.21.15.84:0/2088338394 shutdown_connections 2023-10-16T10:05:48.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.549+0000 7fe7edcbb640 1 -- 172.21.15.84:0/2088338394 wait complete. 2023-10-16T10:05:48.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.551+0000 7fe7edcbb640 1 Processor -- start 2023-10-16T10:05:48.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.551+0000 7fe7edcbb640 1 -- start start 2023-10-16T10:05:48.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.552+0000 7fe7edcbb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e806c140 0x7fe7e80c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:48.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.552+0000 7fe7edcbb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7e8159ab0 0x7fe7e8069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:48.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.552+0000 7fe7edcbb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7e8159e90 0x7fe7e806a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:48.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.552+0000 7fe7edcbb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e80c4160 con 0x7fe7e806c140 2023-10-16T10:05:48.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.552+0000 7fe7edcbb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e80c4440 con 0x7fe7e8159e90 2023-10-16T10:05:48.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.552+0000 7fe7edcbb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e80c4720 con 0x7fe7e8159ab0 2023-10-16T10:05:48.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.553+0000 7fe7e77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e806c140 0x7fe7e80c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:48.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.553+0000 7fe7e77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e806c140 0x7fe7e80c37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60588/0 (socket says 172.21.15.84:60588) 2023-10-16T10:05:48.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.553+0000 7fe7e77fe640 1 -- 172.21.15.84:0/2065101088 learned_addr learned my addr 172.21.15.84:0/2065101088 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:48.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.553+0000 7fe7e77fe640 1 -- 172.21.15.84:0/2065101088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7e8159ab0 msgr2=0x7fe7e8069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:48.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.553+0000 7fe7e77fe640 1 --2- 172.21.15.84:0/2065101088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7e8159ab0 0x7fe7e8069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:48.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.553+0000 7fe7e77fe640 1 -- 172.21.15.84:0/2065101088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7e8159e90 msgr2=0x7fe7e806a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:48.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.553+0000 7fe7e7fff640 1 --2- 172.21.15.84:0/2065101088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7e8159e90 0x7fe7e806a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:48.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.553+0000 7fe7e77fe640 1 --2- 172.21.15.84:0/2065101088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7e8159e90 0x7fe7e806a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:48.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.553+0000 7fe7e77fe640 1 -- 172.21.15.84:0/2065101088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d8009400 con 0x7fe7e806c140 2023-10-16T10:05:48.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.553+0000 7fe7e77fe640 1 --2- 172.21.15.84:0/2065101088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e806c140 0x7fe7e80c37f0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80097a0 tx=0x7fe7d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:48.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.554+0000 7fe7e4ff9640 1 -- 172.21.15.84:0/2065101088 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8033c30 con 0x7fe7e806c140 2023-10-16T10:05:48.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.554+0000 7fe7e4ff9640 1 -- 172.21.15.84:0/2065101088 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d8033d90 con 0x7fe7e806c140 2023-10-16T10:05:48.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.554+0000 7fe7e4ff9640 1 -- 172.21.15.84:0/2065101088 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d803d6b0 con 0x7fe7e806c140 2023-10-16T10:05:48.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.554+0000 7fe7edcbb640 1 -- 172.21.15.84:0/2065101088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e80c6430 con 0x7fe7e806c140 2023-10-16T10:05:48.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.554+0000 7fe7edcbb640 1 -- 172.21.15.84:0/2065101088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e80c3d30 con 0x7fe7e806c140 2023-10-16T10:05:48.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.555+0000 7fe7edcbb640 1 -- 172.21.15.84:0/2065101088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e80c3f20 con 0x7fe7e806c140 2023-10-16T10:05:48.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.557+0000 7fe7e4ff9640 1 -- 172.21.15.84:0/2065101088 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fe7d803a040 con 0x7fe7e806c140 2023-10-16T10:05:48.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.558+0000 7fe7e4ff9640 1 --2- 172.21.15.84:0/2065101088 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe7c806af30 0x7fe7c806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:48.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.558+0000 7fe7e4ff9640 1 -- 172.21.15.84:0/2065101088 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe7d80abdc0 con 0x7fe7e806c140 2023-10-16T10:05:48.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.558+0000 7fe7dffff640 1 --2- 172.21.15.84:0/2065101088 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe7c806af30 0x7fe7c806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:48.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.559+0000 7fe7dffff640 1 --2- 172.21.15.84:0/2065101088 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe7c806af30 0x7fe7c806d3f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0007fd0 tx=0x7fe7d0007920 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:48.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.562+0000 7fe7e4ff9640 1 -- 172.21.15.84:0/2065101088 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe7d8076f30 con 0x7fe7e806c140 2023-10-16T10:05:48.619 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:48.616+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:48.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:48.806+0000 7fe7edcbb640 1 -- 172.21.15.84:0/2065101088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe7e80c3f20 con 0x7fe7e806c140 2023-10-16T10:05:49.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.321+0000 7fe7e4ff9640 1 -- 172.21.15.84:0/2065101088 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v49) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe7e80c3f20 con 0x7fe7e806c140 2023-10-16T10:05:49.323 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 12 and data pool 13 2023-10-16T10:05:49.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.327+0000 7fe7edcbb640 1 -- 172.21.15.84:0/2065101088 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe7c806af30 msgr2=0x7fe7c806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:49.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.327+0000 7fe7edcbb640 1 --2- 172.21.15.84:0/2065101088 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe7c806af30 0x7fe7c806d3f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0007fd0 tx=0x7fe7d0007920 comp rx=0 tx=0).stop 2023-10-16T10:05:49.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.327+0000 7fe7edcbb640 1 -- 172.21.15.84:0/2065101088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e806c140 msgr2=0x7fe7e80c37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:49.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.327+0000 7fe7edcbb640 1 --2- 172.21.15.84:0/2065101088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e806c140 0x7fe7e80c37f0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80097a0 tx=0x7fe7d80046c0 comp rx=0 tx=0).stop 2023-10-16T10:05:49.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.327+0000 7fe7edcbb640 1 -- 172.21.15.84:0/2065101088 shutdown_connections 2023-10-16T10:05:49.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.327+0000 7fe7edcbb640 1 --2- 172.21.15.84:0/2065101088 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fe7c806af30 0x7fe7c806d3f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:49.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.327+0000 7fe7edcbb640 1 --2- 172.21.15.84:0/2065101088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7e8159e90 0x7fe7e806a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:49.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.327+0000 7fe7edcbb640 1 --2- 172.21.15.84:0/2065101088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7e8159ab0 0x7fe7e8069fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:49.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.327+0000 7fe7edcbb640 1 --2- 172.21.15.84:0/2065101088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e806c140 0x7fe7e80c37f0 secure :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80097a0 tx=0x7fe7d80046c0 comp rx=0 tx=0).stop 2023-10-16T10:05:49.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.327+0000 7fe7edcbb640 1 -- 172.21.15.84:0/2065101088 >> 172.21.15.84:0/2065101088 conn(0x7fe7e80ce9b0 msgr2=0x7fe7e814edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:49.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.329+0000 7fe7edcbb640 1 -- 172.21.15.84:0/2065101088 shutdown_connections 2023-10-16T10:05:49.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.330+0000 7fe7edcbb640 1 -- 172.21.15.84:0/2065101088 wait complete. 2023-10-16T10:05:49.370 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:05:49.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.557+0000 7f2704451640 1 Processor -- start 2023-10-16T10:05:49.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.557+0000 7f2704451640 1 -- start start 2023-10-16T10:05:49.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.559+0000 7f2704451640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26fc160420 0x7f26fc0ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:49.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.559+0000 7f2704451640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f26fc06b4c0 0x7f26fc0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:49.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.559+0000 7f2704451640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26fc06e170 con 0x7f26fc160800 2023-10-16T10:05:49.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.559+0000 7f2704451640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26fc0cde30 con 0x7f26fc06b4c0 2023-10-16T10:05:49.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.559+0000 7f2704451640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0cd460 con 0x7f26fc160420 2023-10-16T10:05:49.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.559+0000 7f27029ee640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f26fc06b4c0 0x7f26fc0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:49.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.559+0000 7f27029ee640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f26fc06b4c0 0x7f26fc0cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60600/0 (socket says 172.21.15.84:60600) 2023-10-16T10:05:49.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.559+0000 7f27029ee640 1 -- 172.21.15.84:0/2914449272 learned_addr learned my addr 172.21.15.84:0/2914449272 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:49.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.559+0000 7f27021ed640 1 --2- 172.21.15.84:0/2914449272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26fc160420 0x7f26fc0ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:49.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.559+0000 7f27029ee640 1 -- 172.21.15.84:0/2914449272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26fc160420 msgr2=0x7f26fc0ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:49.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.559+0000 7f27029ee640 1 --2- 172.21.15.84:0/2914449272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26fc160420 0x7f26fc0ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:49.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.559+0000 7f27029ee640 1 -- 172.21.15.84:0/2914449272 >> v1:172.21.15.157:6789/0 conn(0x7f26fc160800 legacy=0x7f26fc06adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:49.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.559+0000 7f27029ee640 1 -- 172.21.15.84:0/2914449272 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc06e3c0 con 0x7f26fc06b4c0 2023-10-16T10:05:49.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.561+0000 7f27029ee640 1 --2- 172.21.15.84:0/2914449272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26fc06b4c0 0x7f26fc0cd8f0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009e50 tx=0x7f26f402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad080a7f21c8c50 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:49.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.561+0000 7f27011eb640 1 -- 172.21.15.84:0/2914449272 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f402ca70 con 0x7f26fc06b4c0 2023-10-16T10:05:49.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.561+0000 7f27011eb640 1 -- 172.21.15.84:0/2914449272 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f402cbd0 con 0x7f26fc06b4c0 2023-10-16T10:05:49.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.561+0000 7f27011eb640 1 -- 172.21.15.84:0/2914449272 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f40343f0 con 0x7f26fc06b4c0 2023-10-16T10:05:49.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.562+0000 7f2704451640 1 -- 172.21.15.84:0/2914449272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26fc06b4c0 msgr2=0x7f26fc0cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:49.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.562+0000 7f2704451640 1 --2- 172.21.15.84:0/2914449272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26fc06b4c0 0x7f26fc0cd8f0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009e50 tx=0x7f26f402bed0 comp rx=0 tx=0).stop 2023-10-16T10:05:49.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.562+0000 7f2704451640 1 -- 172.21.15.84:0/2914449272 shutdown_connections 2023-10-16T10:05:49.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.562+0000 7f2704451640 1 --2- 172.21.15.84:0/2914449272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26fc06b4c0 0x7f26fc0cd8f0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:49.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.562+0000 7f2704451640 1 --2- 172.21.15.84:0/2914449272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26fc160420 0x7f26fc0ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:49.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.562+0000 7f2704451640 1 -- 172.21.15.84:0/2914449272 >> 172.21.15.84:0/2914449272 conn(0x7f26fc0b0d30 msgr2=0x7f26fc0b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:49.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.562+0000 7f2704451640 1 -- 172.21.15.84:0/2914449272 shutdown_connections 2023-10-16T10:05:49.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.562+0000 7f2704451640 1 -- 172.21.15.84:0/2914449272 wait complete. 2023-10-16T10:05:49.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.564+0000 7f2704451640 1 Processor -- start 2023-10-16T10:05:49.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.564+0000 7f2704451640 1 -- start start 2023-10-16T10:05:49.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.565+0000 7f2704451640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26fc06b4c0 0x7f26fc0b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:49.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.565+0000 7f2704451640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26fc160420 0x7f26fc0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:49.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.565+0000 7f2704451640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26fc160800 0x7f26fc0b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:49.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.565+0000 7f2704451640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0c34c0 con 0x7f26fc160420 2023-10-16T10:05:49.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.565+0000 7f2704451640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0c4040 con 0x7f26fc06b4c0 2023-10-16T10:05:49.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.565+0000 7f2704451640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0b4780 con 0x7f26fc160800 2023-10-16T10:05:49.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.565+0000 7f27019ec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26fc160420 0x7f26fc0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:49.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.565+0000 7f27019ec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26fc160420 0x7f26fc0b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60610/0 (socket says 172.21.15.84:60610) 2023-10-16T10:05:49.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.565+0000 7f27019ec640 1 -- 172.21.15.84:0/3986932677 learned_addr learned my addr 172.21.15.84:0/3986932677 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:49.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.565+0000 7f27021ed640 1 --2- 172.21.15.84:0/3986932677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26fc06b4c0 0x7f26fc0b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:49.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.565+0000 7f27019ec640 1 -- 172.21.15.84:0/3986932677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26fc160800 msgr2=0x7f26fc0b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:49.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.565+0000 7f27019ec640 1 --2- 172.21.15.84:0/3986932677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26fc160800 0x7f26fc0b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:49.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.565+0000 7f27019ec640 1 -- 172.21.15.84:0/3986932677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26fc06b4c0 msgr2=0x7f26fc0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:49.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.565+0000 7f27019ec640 1 --2- 172.21.15.84:0/3986932677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26fc06b4c0 0x7f26fc0b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:49.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.566+0000 7f27019ec640 1 -- 172.21.15.84:0/3986932677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f4009b30 con 0x7f26fc160420 2023-10-16T10:05:49.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.566+0000 7f27019ec640 1 --2- 172.21.15.84:0/3986932677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26fc160420 0x7f26fc0b3d00 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00b420 tx=0x7f26ec00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:49.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.566+0000 7f26f2ffd640 1 -- 172.21.15.84:0/3986932677 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec011830 con 0x7f26fc160420 2023-10-16T10:05:49.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.566+0000 7f26f2ffd640 1 -- 172.21.15.84:0/3986932677 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec011990 con 0x7f26fc160420 2023-10-16T10:05:49.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.566+0000 7f26f2ffd640 1 -- 172.21.15.84:0/3986932677 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec011d40 con 0x7f26fc160420 2023-10-16T10:05:49.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.566+0000 7f2704451640 1 -- 172.21.15.84:0/3986932677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26fc0c3710 con 0x7f26fc160420 2023-10-16T10:05:49.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.566+0000 7f2704451640 1 -- 172.21.15.84:0/3986932677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26fc0c3cd0 con 0x7f26fc160420 2023-10-16T10:05:49.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.568+0000 7f2704451640 1 -- 172.21.15.84:0/3986932677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26fc04e160 con 0x7f26fc160420 2023-10-16T10:05:49.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.569+0000 7f26f2ffd640 1 -- 172.21.15.84:0/3986932677 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f26ec01b460 con 0x7f26fc160420 2023-10-16T10:05:49.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.570+0000 7f26f2ffd640 1 --2- 172.21.15.84:0/3986932677 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f26e006af00 0x7f26e006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:49.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.570+0000 7f26f2ffd640 1 -- 172.21.15.84:0/3986932677 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26ec089130 con 0x7f26fc160420 2023-10-16T10:05:49.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.570+0000 7f27021ed640 1 --2- 172.21.15.84:0/3986932677 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f26e006af00 0x7f26e006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:49.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.571+0000 7f27021ed640 1 --2- 172.21.15.84:0/3986932677 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f26e006af00 0x7f26e006d3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f26f8006fd0 tx=0x7f26f8008040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:49.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.574+0000 7f26f2ffd640 1 -- 172.21.15.84:0/3986932677 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f26ec054280 con 0x7f26fc160420 2023-10-16T10:05:49.651 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:49.649+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:49.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.773+0000 7f2704451640 1 -- 172.21.15.84:0/3986932677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26fc0c4180 con 0x7f26fc160420 2023-10-16T10:05:49.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.775+0000 7f26f2ffd640 1 -- 172.21.15.84:0/3986932677 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16560 (secure 0 0 0) 0x7f26ec0578a0 con 0x7f26fc160420 2023-10-16T10:05:49.777 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:49.777 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":107,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:05:48.808646+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:05:46.595888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-16T10:05:47.826019+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:05:49.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.778+0000 7f2704451640 1 -- 172.21.15.84:0/3986932677 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f26e006af00 msgr2=0x7f26e006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:49.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.778+0000 7f2704451640 1 --2- 172.21.15.84:0/3986932677 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f26e006af00 0x7f26e006d3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f26f8006fd0 tx=0x7f26f8008040 comp rx=0 tx=0).stop 2023-10-16T10:05:49.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.778+0000 7f2704451640 1 -- 172.21.15.84:0/3986932677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26fc160420 msgr2=0x7f26fc0b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:49.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.778+0000 7f2704451640 1 --2- 172.21.15.84:0/3986932677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26fc160420 0x7f26fc0b3d00 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00b420 tx=0x7f26ec00b8e0 comp rx=0 tx=0).stop 2023-10-16T10:05:49.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.778+0000 7f2704451640 1 -- 172.21.15.84:0/3986932677 shutdown_connections 2023-10-16T10:05:49.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.778+0000 7f2704451640 1 --2- 172.21.15.84:0/3986932677 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f26e006af00 0x7f26e006d3c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:49.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.778+0000 7f2704451640 1 --2- 172.21.15.84:0/3986932677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26fc160800 0x7f26fc0b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:49.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.778+0000 7f2704451640 1 --2- 172.21.15.84:0/3986932677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26fc160420 0x7f26fc0b3d00 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:49.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.778+0000 7f2704451640 1 --2- 172.21.15.84:0/3986932677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26fc06b4c0 0x7f26fc0b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:49.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.778+0000 7f2704451640 1 -- 172.21.15.84:0/3986932677 >> 172.21.15.84:0/3986932677 conn(0x7f26fc0b0d30 msgr2=0x7f26fc0af990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:49.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.778+0000 7f2704451640 1 -- 172.21.15.84:0/3986932677 shutdown_connections 2023-10-16T10:05:49.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:49.778+0000 7f2704451640 1 -- 172.21.15.84:0/3986932677 wait complete. 2023-10-16T10:05:49.798 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:05:50.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.013+0000 7f61788e6640 1 Processor -- start 2023-10-16T10:05:50.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.014+0000 7f61788e6640 1 -- start start 2023-10-16T10:05:50.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.015+0000 7f61788e6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f617415aaa0 0x7f61740b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:50.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.015+0000 7f61788e6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61740ca8a0 0x7f61740b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:50.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.015+0000 7f61788e6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61740cad00 con 0x7f6174070240 2023-10-16T10:05:50.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.015+0000 7f61788e6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61740b2d00 con 0x7f617415aaa0 2023-10-16T10:05:50.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.015+0000 7f61788e6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61740b2fe0 con 0x7f61740ca8a0 2023-10-16T10:05:50.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.015+0000 7f6172d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f617415aaa0 0x7f61740b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:50.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.015+0000 7f6172d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f617415aaa0 0x7f61740b2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60614/0 (socket says 172.21.15.84:60614) 2023-10-16T10:05:50.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.015+0000 7f6172d9d640 1 -- 172.21.15.84:0/141897866 learned_addr learned my addr 172.21.15.84:0/141897866 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:50.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.016+0000 7f6172d9d640 1 -- 172.21.15.84:0/141897866 >> v2:172.21.15.157:3300/0 conn(0x7f61740ca8a0 msgr2=0x7f61740b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:50.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.016+0000 7f6172d9d640 1 --2- 172.21.15.84:0/141897866 >> v2:172.21.15.157:3300/0 conn(0x7f61740ca8a0 0x7f61740b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:50.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.016+0000 7f6172d9d640 1 -- 172.21.15.84:0/141897866 >> v1:172.21.15.157:6789/0 conn(0x7f6174070240 legacy=0x7f61740c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:50.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.016+0000 7f6172d9d640 1 -- 172.21.15.84:0/141897866 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61740b32c0 con 0x7f617415aaa0 2023-10-16T10:05:50.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.017+0000 7f6172d9d640 1 --2- 172.21.15.84:0/141897866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f617415aaa0 0x7f61740b2600 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f61740cf650 tx=0x7f616402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3e7ca717dad421 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:50.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.017+0000 7f617259c640 1 -- 172.21.15.84:0/141897866 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164031870 con 0x7f617415aaa0 2023-10-16T10:05:50.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.017+0000 7f617259c640 1 -- 172.21.15.84:0/141897866 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61640319d0 con 0x7f617415aaa0 2023-10-16T10:05:50.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.017+0000 7f617259c640 1 -- 172.21.15.84:0/141897866 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164031d60 con 0x7f617415aaa0 2023-10-16T10:05:50.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.018+0000 7f61788e6640 1 -- 172.21.15.84:0/141897866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f617415aaa0 msgr2=0x7f61740b2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:50.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.018+0000 7f61788e6640 1 --2- 172.21.15.84:0/141897866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f617415aaa0 0x7f61740b2600 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f61740cf650 tx=0x7f616402b910 comp rx=0 tx=0).stop 2023-10-16T10:05:50.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.018+0000 7f61788e6640 1 -- 172.21.15.84:0/141897866 shutdown_connections 2023-10-16T10:05:50.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.018+0000 7f61788e6640 1 --2- 172.21.15.84:0/141897866 >> v2:172.21.15.157:3300/0 conn(0x7f61740ca8a0 0x7f61740b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:50.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.018+0000 7f61788e6640 1 --2- 172.21.15.84:0/141897866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f617415aaa0 0x7f61740b2600 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:50.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.018+0000 7f61788e6640 1 -- 172.21.15.84:0/141897866 >> 172.21.15.84:0/141897866 conn(0x7f61740cc230 msgr2=0x7f61740cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:50.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.018+0000 7f61788e6640 1 -- 172.21.15.84:0/141897866 shutdown_connections 2023-10-16T10:05:50.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.018+0000 7f61788e6640 1 -- 172.21.15.84:0/141897866 wait complete. 2023-10-16T10:05:50.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.020+0000 7f61788e6640 1 Processor -- start 2023-10-16T10:05:50.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.020+0000 7f61788e6640 1 -- start start 2023-10-16T10:05:50.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.021+0000 7f61788e6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6174070240 0x7f617414ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:50.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.021+0000 7f61788e6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61740ca8a0 0x7f617415c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:50.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.021+0000 7f61788e6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f617415aaa0 0x7f61740c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:50.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.021+0000 7f61788e6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61740c6e40 con 0x7f61740ca8a0 2023-10-16T10:05:50.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.021+0000 7f61788e6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617415d380 con 0x7f617415aaa0 2023-10-16T10:05:50.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.021+0000 7f61788e6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617415d660 con 0x7f6174070240 2023-10-16T10:05:50.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.021+0000 7f616bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61740ca8a0 0x7f617415c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:50.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.021+0000 7f616bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61740ca8a0 0x7f617415c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60630/0 (socket says 172.21.15.84:60630) 2023-10-16T10:05:50.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.021+0000 7f616bfff640 1 -- 172.21.15.84:0/295339335 learned_addr learned my addr 172.21.15.84:0/295339335 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:50.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.022+0000 7f6172d9d640 1 --2- 172.21.15.84:0/295339335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6174070240 0x7f617414ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:50.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.022+0000 7f616bfff640 1 -- 172.21.15.84:0/295339335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6174070240 msgr2=0x7f617414ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:50.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.022+0000 7f616bfff640 1 --2- 172.21.15.84:0/295339335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6174070240 0x7f617414ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:50.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.022+0000 7f616bfff640 1 -- 172.21.15.84:0/295339335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f617415aaa0 msgr2=0x7f61740c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:50.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.022+0000 7f616bfff640 1 --2- 172.21.15.84:0/295339335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f617415aaa0 0x7f61740c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:50.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.022+0000 7f616bfff640 1 -- 172.21.15.84:0/295339335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61640094d0 con 0x7f61740ca8a0 2023-10-16T10:05:50.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.022+0000 7f616bfff640 1 --2- 172.21.15.84:0/295339335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61740ca8a0 0x7f617415c930 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f615c00b4a0 tx=0x7f615c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:50.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.022+0000 7f616b7fe640 1 -- 172.21.15.84:0/295339335 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c010610 con 0x7f61740ca8a0 2023-10-16T10:05:50.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.022+0000 7f616b7fe640 1 -- 172.21.15.84:0/295339335 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c010770 con 0x7f61740ca8a0 2023-10-16T10:05:50.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.022+0000 7f616b7fe640 1 -- 172.21.15.84:0/295339335 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c010b20 con 0x7f61740ca8a0 2023-10-16T10:05:50.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.022+0000 7f61788e6640 1 -- 172.21.15.84:0/295339335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61740c6ad0 con 0x7f61740ca8a0 2023-10-16T10:05:50.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.022+0000 7f61788e6640 1 -- 172.21.15.84:0/295339335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6174159cb0 con 0x7f61740ca8a0 2023-10-16T10:05:50.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.024+0000 7f61788e6640 1 -- 172.21.15.84:0/295339335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f617415d880 con 0x7f61740ca8a0 2023-10-16T10:05:50.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.025+0000 7f616b7fe640 1 -- 172.21.15.84:0/295339335 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f615c010c80 con 0x7f61740ca8a0 2023-10-16T10:05:50.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.025+0000 7f616b7fe640 1 --2- 172.21.15.84:0/295339335 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f615406af00 0x7f615406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:50.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.026+0000 7f616b7fe640 1 -- 172.21.15.84:0/295339335 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f615c089330 con 0x7f61740ca8a0 2023-10-16T10:05:50.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.026+0000 7f6172d9d640 1 --2- 172.21.15.84:0/295339335 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f615406af00 0x7f615406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:50.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.026+0000 7f6172d9d640 1 --2- 172.21.15.84:0/295339335 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f615406af00 0x7f615406d3c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6164036000 tx=0x7f616403c040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:50.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.030+0000 7f616b7fe640 1 -- 172.21.15.84:0/295339335 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f615c0542f0 con 0x7f61740ca8a0 2023-10-16T10:05:50.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.231+0000 7f61788e6640 1 -- 172.21.15.84:0/295339335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f617404e9b0 con 0x7f61740ca8a0 2023-10-16T10:05:50.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.233+0000 7f616b7fe640 1 -- 172.21.15.84:0/295339335 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16560 (secure 0 0 0) 0x7f615c057910 con 0x7f61740ca8a0 2023-10-16T10:05:50.235 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:50.235 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":107,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:05:48.808646+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:05:46.595888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-16T10:05:47.826019+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:05:50.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.235+0000 7f61788e6640 1 -- 172.21.15.84:0/295339335 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f615406af00 msgr2=0x7f615406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:50.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.235+0000 7f61788e6640 1 --2- 172.21.15.84:0/295339335 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f615406af00 0x7f615406d3c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6164036000 tx=0x7f616403c040 comp rx=0 tx=0).stop 2023-10-16T10:05:50.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.235+0000 7f61788e6640 1 -- 172.21.15.84:0/295339335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61740ca8a0 msgr2=0x7f617415c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:50.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.235+0000 7f61788e6640 1 --2- 172.21.15.84:0/295339335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61740ca8a0 0x7f617415c930 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f615c00b4a0 tx=0x7f615c00b960 comp rx=0 tx=0).stop 2023-10-16T10:05:50.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.235+0000 7f61788e6640 1 -- 172.21.15.84:0/295339335 shutdown_connections 2023-10-16T10:05:50.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.235+0000 7f61788e6640 1 --2- 172.21.15.84:0/295339335 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f615406af00 0x7f615406d3c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:50.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.235+0000 7f61788e6640 1 --2- 172.21.15.84:0/295339335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f617415aaa0 0x7f61740c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:50.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.235+0000 7f61788e6640 1 --2- 172.21.15.84:0/295339335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61740ca8a0 0x7f617415c930 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:50.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.235+0000 7f61788e6640 1 --2- 172.21.15.84:0/295339335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6174070240 0x7f617414ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:50.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.235+0000 7f61788e6640 1 -- 172.21.15.84:0/295339335 >> 172.21.15.84:0/295339335 conn(0x7f61740cc230 msgr2=0x7f61740c51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:50.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.236+0000 7f61788e6640 1 -- 172.21.15.84:0/295339335 shutdown_connections 2023-10-16T10:05:50.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.236+0000 7f61788e6640 1 -- 172.21.15.84:0/295339335 wait complete. 2023-10-16T10:05:50.255 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T10:05:50.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.467+0000 7ff9ac81a640 1 Processor -- start 2023-10-16T10:05:50.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.468+0000 7ff9ac81a640 1 -- start start 2023-10-16T10:05:50.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.469+0000 7ff9ac81a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a8070950 0x7ff9a806bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:50.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.469+0000 7ff9ac81a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a80ca3a0 con 0x7ff9a81573e0 2023-10-16T10:05:50.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.469+0000 7ff9ac81a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a80c9d70 con 0x7ff9a80ca830 2023-10-16T10:05:50.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.469+0000 7ff9ac81a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a81577c0 con 0x7ff9a8070950 2023-10-16T10:05:50.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.469+0000 7ff9a77fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff9a81573e0 0x7ff9a8159e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53686/0 (socket says 172.21.15.84:53686) 2023-10-16T10:05:50.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.469+0000 7ff9a77fe640 1 -- 172.21.15.84:0/3659930156 learned_addr learned my addr 172.21.15.84:0/3659930156 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:50.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.470+0000 7ff9a7fff640 1 --2- 172.21.15.84:0/3659930156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a8070950 0x7ff9a806bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:50.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.470+0000 7ff9a6ffd640 1 -- 172.21.15.84:0/3659930156 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2434733077 0 0) 0x7ff9a80ca3a0 con 0x7ff9a81573e0 2023-10-16T10:05:50.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.470+0000 7ff9a6ffd640 1 -- 172.21.15.84:0/3659930156 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff994003520 con 0x7ff9a81573e0 2023-10-16T10:05:50.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.470+0000 7ff9a6ffd640 1 -- 172.21.15.84:0/3659930156 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2562922010 0 0) 0x7ff994003520 con 0x7ff9a81573e0 2023-10-16T10:05:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.470+0000 7ff9a6ffd640 1 -- 172.21.15.84:0/3659930156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a8070950 msgr2=0x7ff9a806bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.470+0000 7ff9a6ffd640 1 --2- 172.21.15.84:0/3659930156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a8070950 0x7ff9a806bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.470+0000 7ff9a6ffd640 1 -- 172.21.15.84:0/3659930156 >> v1:172.21.15.157:6789/0 conn(0x7ff9a80ca830 legacy=0x7ff9a8070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.470+0000 7ff9a6ffd640 1 -- 172.21.15.84:0/3659930156 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a815a9a0 con 0x7ff9a81573e0 2023-10-16T10:05:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.470+0000 7ff9a6ffd640 1 -- 172.21.15.84:0/3659930156 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ff998001670 con 0x7ff9a81573e0 2023-10-16T10:05:50.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.470+0000 7ff9a7fff640 1 --2- 172.21.15.84:0/3659930156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a8070950 0x7ff9a806bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:05:50.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.470+0000 7ff9a6ffd640 1 -- 172.21.15.84:0/3659930156 >> v1:172.21.15.84:6789/0 conn(0x7ff9a81573e0 legacy=0x7ff9a8159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:50.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.470+0000 7ff9a6ffd640 1 --2- 172.21.15.84:0/3659930156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9940039a0 0x7ff994003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:50.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.470+0000 7ff9a6ffd640 1 --2- 172.21.15.84:0/3659930156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9940047c0 0x7ff994004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:50.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.470+0000 7ff9a6ffd640 1 --2- 172.21.15.84:0/3659930156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a8070950 0x7ff994005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:50.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.470+0000 7ff9a6ffd640 1 -- 172.21.15.84:0/3659930156 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a81577c0 con 0x7ff9940039a0 2023-10-16T10:05:50.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.470+0000 7ff9a6ffd640 1 -- 172.21.15.84:0/3659930156 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99400bab0 con 0x7ff9940047c0 2023-10-16T10:05:50.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.470+0000 7ff9a6ffd640 1 -- 172.21.15.84:0/3659930156 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99400bc70 con 0x7ff9a8070950 2023-10-16T10:05:50.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.470+0000 7ff9a77fe640 1 --2- 172.21.15.84:0/3659930156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9940039a0 0x7ff994003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:50.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.471+0000 7ff9a7fff640 1 --2- 172.21.15.84:0/3659930156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a8070950 0x7ff994005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:50.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.471+0000 7ff9a77fe640 1 -- 172.21.15.84:0/3659930156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a8070950 msgr2=0x7ff994005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:50.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.471+0000 7ff9a77fe640 1 --2- 172.21.15.84:0/3659930156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a8070950 0x7ff994005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:50.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.471+0000 7ff9a77fe640 1 -- 172.21.15.84:0/3659930156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9940047c0 msgr2=0x7ff994004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:50.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.471+0000 7ff9a77fe640 1 --2- 172.21.15.84:0/3659930156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9940047c0 0x7ff994004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:50.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.471+0000 7ff9a77fe640 1 -- 172.21.15.84:0/3659930156 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff994003520 con 0x7ff9940039a0 2023-10-16T10:05:50.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.471+0000 7ff9a7fff640 1 --2- 172.21.15.84:0/3659930156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a8070950 0x7ff994005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:50.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.472+0000 7ff9a77fe640 1 --2- 172.21.15.84:0/3659930156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9940039a0 0x7ff994003dc0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff99800c190 tx=0x7ff9940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f780e5c080dde2c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:50.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.472+0000 7ff9a6ffd640 1 -- 172.21.15.84:0/3659930156 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998005ad0 con 0x7ff9940039a0 2023-10-16T10:05:50.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.472+0000 7ff9a6ffd640 1 -- 172.21.15.84:0/3659930156 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff998005c30 con 0x7ff9940039a0 2023-10-16T10:05:50.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.472+0000 7ff9a6ffd640 1 -- 172.21.15.84:0/3659930156 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99802c8a0 con 0x7ff9940039a0 2023-10-16T10:05:50.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.473+0000 7ff9ac81a640 1 -- 172.21.15.84:0/3659930156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9940039a0 msgr2=0x7ff994003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:50.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.473+0000 7ff9ac81a640 1 --2- 172.21.15.84:0/3659930156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9940039a0 0x7ff994003dc0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff99800c190 tx=0x7ff9940057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:50.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.473+0000 7ff9a7fff640 1 -- 172.21.15.84:0/3659930156 reap_dead start 2023-10-16T10:05:50.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.473+0000 7ff9ac81a640 1 -- 172.21.15.84:0/3659930156 shutdown_connections 2023-10-16T10:05:50.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.473+0000 7ff9ac81a640 1 -- 172.21.15.84:0/3659930156 >> 172.21.15.84:0/3659930156 conn(0x7ff9a80cc230 msgr2=0x7ff9a80cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:50.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.473+0000 7ff9ac81a640 1 -- 172.21.15.84:0/3659930156 shutdown_connections 2023-10-16T10:05:50.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.473+0000 7ff9ac81a640 1 -- 172.21.15.84:0/3659930156 wait complete. 2023-10-16T10:05:50.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.475+0000 7ff9ac81a640 1 Processor -- start 2023-10-16T10:05:50.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.475+0000 7ff9ac81a640 1 -- start start 2023-10-16T10:05:50.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.476+0000 7ff9ac81a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9940039a0 0x7ff9a8159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:50.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.476+0000 7ff9ac81a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a81f23e0 0x7ff9a81f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:50.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.476+0000 7ff9ac81a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a8157990 0x7ff9a8157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:50.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.476+0000 7ff9ac81a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a80c6eb0 con 0x7ff9a8157990 2023-10-16T10:05:50.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.476+0000 7ff9ac81a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a8158450 con 0x7ff9940039a0 2023-10-16T10:05:50.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.476+0000 7ff9ac81a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a8158730 con 0x7ff9a81f23e0 2023-10-16T10:05:50.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.476+0000 7ff9a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a8157990 0x7ff9a8157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:50.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.476+0000 7ff9a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a8157990 0x7ff9a8157e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60650/0 (socket says 172.21.15.84:60650) 2023-10-16T10:05:50.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.476+0000 7ff9a7fff640 1 -- 172.21.15.84:0/2533268361 learned_addr learned my addr 172.21.15.84:0/2533268361 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:50.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.477+0000 7ff9a7fff640 1 -- 172.21.15.84:0/2533268361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a81f23e0 msgr2=0x7ff9a81f2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:50.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.477+0000 7ff9a7fff640 1 --2- 172.21.15.84:0/2533268361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a81f23e0 0x7ff9a81f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:50.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.477+0000 7ff9a7fff640 1 -- 172.21.15.84:0/2533268361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9940039a0 msgr2=0x7ff9a8159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:50.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.477+0000 7ff9a7fff640 1 --2- 172.21.15.84:0/2533268361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9940039a0 0x7ff9a8159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:50.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.477+0000 7ff9a7fff640 1 -- 172.21.15.84:0/2533268361 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a80c67f0 con 0x7ff9a8157990 2023-10-16T10:05:50.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.477+0000 7ff9a7fff640 1 --2- 172.21.15.84:0/2533268361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a8157990 0x7ff9a8157e10 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8073ad0 tx=0x7ff9a0004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:50.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.477+0000 7ff9a4ff9640 1 -- 172.21.15.84:0/2533268361 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0007cf0 con 0x7ff9a8157990 2023-10-16T10:05:50.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.477+0000 7ff9a4ff9640 1 -- 172.21.15.84:0/2533268361 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a0007e50 con 0x7ff9a8157990 2023-10-16T10:05:50.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.477+0000 7ff9a4ff9640 1 -- 172.21.15.84:0/2533268361 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0014640 con 0x7ff9a8157990 2023-10-16T10:05:50.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.477+0000 7ff9ac81a640 1 -- 172.21.15.84:0/2533268361 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a80c6ad0 con 0x7ff9a8157990 2023-10-16T10:05:50.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.477+0000 7ff9ac81a640 1 -- 172.21.15.84:0/2533268361 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a815a480 con 0x7ff9a8157990 2023-10-16T10:05:50.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.478+0000 7ff9ac81a640 1 -- 172.21.15.84:0/2533268361 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a804e9b0 con 0x7ff9a8157990 2023-10-16T10:05:50.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.480+0000 7ff9a4ff9640 1 -- 172.21.15.84:0/2533268361 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7ff9a001d3b0 con 0x7ff9a8157990 2023-10-16T10:05:50.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.480+0000 7ff9a4ff9640 1 --2- 172.21.15.84:0/2533268361 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7ff98806af20 0x7ff98806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:50.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.481+0000 7ff9a4ff9640 1 -- 172.21.15.84:0/2533268361 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9a0089380 con 0x7ff9a8157990 2023-10-16T10:05:50.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.481+0000 7ff9a77fe640 1 --2- 172.21.15.84:0/2533268361 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7ff98806af20 0x7ff98806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:50.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.481+0000 7ff9a77fe640 1 --2- 172.21.15.84:0/2533268361 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7ff98806af20 0x7ff98806d3e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff998034000 tx=0x7ff998007000 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:50.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.485+0000 7ff9a4ff9640 1 -- 172.21.15.84:0/2533268361 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff9a00543d0 con 0x7ff9a8157990 2023-10-16T10:05:50.663 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:50.660+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:50.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:50.723+0000 7ff9ac81a640 1 -- 172.21.15.84:0/2533268361 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff9a80b5730 con 0x7ff9a8157990 2023-10-16T10:05:51.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.363+0000 7ff9a4ff9640 1 -- 172.21.15.84:0/2533268361 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff9a00579f0 con 0x7ff9a8157990 2023-10-16T10:05:51.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.367+0000 7ff9ac81a640 1 -- 172.21.15.84:0/2533268361 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7ff98806af20 msgr2=0x7ff98806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:51.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.367+0000 7ff9ac81a640 1 --2- 172.21.15.84:0/2533268361 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7ff98806af20 0x7ff98806d3e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff998034000 tx=0x7ff998007000 comp rx=0 tx=0).stop 2023-10-16T10:05:51.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.367+0000 7ff9ac81a640 1 -- 172.21.15.84:0/2533268361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a8157990 msgr2=0x7ff9a8157e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:51.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.367+0000 7ff9ac81a640 1 --2- 172.21.15.84:0/2533268361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a8157990 0x7ff9a8157e10 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8073ad0 tx=0x7ff9a0004780 comp rx=0 tx=0).stop 2023-10-16T10:05:51.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.367+0000 7ff9ac81a640 1 -- 172.21.15.84:0/2533268361 shutdown_connections 2023-10-16T10:05:51.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.367+0000 7ff9ac81a640 1 --2- 172.21.15.84:0/2533268361 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7ff98806af20 0x7ff98806d3e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:51.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.367+0000 7ff9ac81a640 1 --2- 172.21.15.84:0/2533268361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a8157990 0x7ff9a8157e10 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:51.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.367+0000 7ff9ac81a640 1 --2- 172.21.15.84:0/2533268361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9a81f23e0 0x7ff9a81f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:51.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.367+0000 7ff9ac81a640 1 --2- 172.21.15.84:0/2533268361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9940039a0 0x7ff9a8159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:51.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.367+0000 7ff9ac81a640 1 -- 172.21.15.84:0/2533268361 >> 172.21.15.84:0/2533268361 conn(0x7ff9a80cc230 msgr2=0x7ff9a80ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:51.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.368+0000 7ff9ac81a640 1 -- 172.21.15.84:0/2533268361 shutdown_connections 2023-10-16T10:05:51.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.368+0000 7ff9ac81a640 1 -- 172.21.15.84:0/2533268361 wait complete. 2023-10-16T10:05:51.390 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T10:05:51.391 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T10:05:51.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.606+0000 7fac52d47640 1 Processor -- start 2023-10-16T10:05:51.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.606+0000 7fac52d47640 1 -- start start 2023-10-16T10:05:51.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.607+0000 7fac52d47640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fac4c15cd70 0x7fac4c0d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:51.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.607+0000 7fac52d47640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac4c0b8480 0x7fac4c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:51.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.607+0000 7fac52d47640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fac4c0d0be0 0x7fac4c0cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:51.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.607+0000 7fac52d47640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4c15d150 con 0x7fac4c0d0be0 2023-10-16T10:05:51.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.607+0000 7fac52d47640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4c0d1180 con 0x7fac4c15cd70 2023-10-16T10:05:51.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.607+0000 7fac52d47640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c0d1460 con 0x7fac4c0b8480 2023-10-16T10:05:51.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.608+0000 7fac52546640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fac4c0d0be0 0x7fac4c0cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:51.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.608+0000 7fac52546640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fac4c0d0be0 0x7fac4c0cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60654/0 (socket says 172.21.15.84:60654) 2023-10-16T10:05:51.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.608+0000 7fac52546640 1 -- 172.21.15.84:0/2727145098 learned_addr learned my addr 172.21.15.84:0/2727145098 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:51.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.608+0000 7fac51d45640 1 --2- 172.21.15.84:0/2727145098 >> v2:172.21.15.157:3300/0 conn(0x7fac4c15cd70 0x7fac4c0d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:51.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.608+0000 7fac51544640 1 --2- 172.21.15.84:0/2727145098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac4c0b8480 0x7fac4c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:51.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.608+0000 7fac52546640 1 -- 172.21.15.84:0/2727145098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac4c0b8480 msgr2=0x7fac4c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:51.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.608+0000 7fac52546640 1 --2- 172.21.15.84:0/2727145098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac4c0b8480 0x7fac4c0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:51.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.608+0000 7fac52546640 1 -- 172.21.15.84:0/2727145098 >> v2:172.21.15.157:3300/0 conn(0x7fac4c15cd70 msgr2=0x7fac4c0d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:51.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.608+0000 7fac52546640 1 --2- 172.21.15.84:0/2727145098 >> v2:172.21.15.157:3300/0 conn(0x7fac4c15cd70 0x7fac4c0d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:51.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.608+0000 7fac52546640 1 -- 172.21.15.84:0/2727145098 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c0d1740 con 0x7fac4c0d0be0 2023-10-16T10:05:51.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.609+0000 7fac52546640 1 --2- 172.21.15.84:0/2727145098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac4c0d0be0 0x7fac4c0cdd00 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0b56a0 tx=0x7fac4402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c339b32ef6fa72 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:51.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.610+0000 7fac50d43640 1 -- 172.21.15.84:0/2727145098 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4402ca70 con 0x7fac4c0d0be0 2023-10-16T10:05:51.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.610+0000 7fac50d43640 1 -- 172.21.15.84:0/2727145098 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac4402cbd0 con 0x7fac4c0d0be0 2023-10-16T10:05:51.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.610+0000 7fac50d43640 1 -- 172.21.15.84:0/2727145098 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac440343f0 con 0x7fac4c0d0be0 2023-10-16T10:05:51.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.610+0000 7fac52d47640 1 -- 172.21.15.84:0/2727145098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac4c0d0be0 msgr2=0x7fac4c0cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:51.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.610+0000 7fac52d47640 1 --2- 172.21.15.84:0/2727145098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac4c0d0be0 0x7fac4c0cdd00 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0b56a0 tx=0x7fac4402bed0 comp rx=0 tx=0).stop 2023-10-16T10:05:51.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.610+0000 7fac52d47640 1 -- 172.21.15.84:0/2727145098 shutdown_connections 2023-10-16T10:05:51.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.610+0000 7fac52d47640 1 --2- 172.21.15.84:0/2727145098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac4c0d0be0 0x7fac4c0cdd00 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:51.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.610+0000 7fac52d47640 1 --2- 172.21.15.84:0/2727145098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac4c0b8480 0x7fac4c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:51.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.610+0000 7fac52d47640 1 --2- 172.21.15.84:0/2727145098 >> v2:172.21.15.157:3300/0 conn(0x7fac4c15cd70 0x7fac4c0d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:51.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.610+0000 7fac52d47640 1 -- 172.21.15.84:0/2727145098 >> 172.21.15.84:0/2727145098 conn(0x7fac4c0cf570 msgr2=0x7fac4c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:51.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.611+0000 7fac52d47640 1 -- 172.21.15.84:0/2727145098 shutdown_connections 2023-10-16T10:05:51.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.611+0000 7fac52d47640 1 -- 172.21.15.84:0/2727145098 wait complete. 2023-10-16T10:05:51.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.612+0000 7fac52d47640 1 Processor -- start 2023-10-16T10:05:51.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.612+0000 7fac52d47640 1 -- start start 2023-10-16T10:05:51.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.613+0000 7fac52d47640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac4c0b8480 0x7fac4c153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:51.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.614+0000 7fac52d47640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac4c0d0be0 0x7fac4c0b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:51.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.614+0000 7fac52d47640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac4c15cd70 0x7fac4c1f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:51.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.614+0000 7fac52d47640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c1602f0 con 0x7fac4c0b8480 2023-10-16T10:05:51.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.614+0000 7fac52d47640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c154760 con 0x7fac4c15cd70 2023-10-16T10:05:51.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.614+0000 7fac52d47640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c154a40 con 0x7fac4c0d0be0 2023-10-16T10:05:51.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.614+0000 7fac51d45640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac4c0b8480 0x7fac4c153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:51.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.614+0000 7fac51d45640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac4c0b8480 0x7fac4c153a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60660/0 (socket says 172.21.15.84:60660) 2023-10-16T10:05:51.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.614+0000 7fac51d45640 1 -- 172.21.15.84:0/338568437 learned_addr learned my addr 172.21.15.84:0/338568437 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:51.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.614+0000 7fac51d45640 1 -- 172.21.15.84:0/338568437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac4c0d0be0 msgr2=0x7fac4c0b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:51.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.614+0000 7fac51d45640 1 --2- 172.21.15.84:0/338568437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac4c0d0be0 0x7fac4c0b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:51.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.614+0000 7fac51d45640 1 -- 172.21.15.84:0/338568437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac4c15cd70 msgr2=0x7fac4c1f50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:51.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.614+0000 7fac52546640 1 --2- 172.21.15.84:0/338568437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac4c15cd70 0x7fac4c1f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:51.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.614+0000 7fac51d45640 1 --2- 172.21.15.84:0/338568437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac4c15cd70 0x7fac4c1f50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:51.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.614+0000 7fac51d45640 1 -- 172.21.15.84:0/338568437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac44009b30 con 0x7fac4c0b8480 2023-10-16T10:05:51.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.615+0000 7fac51d45640 1 --2- 172.21.15.84:0/338568437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac4c0b8480 0x7fac4c153a50 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fac4800d800 tx=0x7fac4800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:51.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.615+0000 7fac427fc640 1 -- 172.21.15.84:0/338568437 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4800b6c0 con 0x7fac4c0b8480 2023-10-16T10:05:51.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.615+0000 7fac427fc640 1 -- 172.21.15.84:0/338568437 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac4800b820 con 0x7fac4c0b8480 2023-10-16T10:05:51.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.615+0000 7fac427fc640 1 -- 172.21.15.84:0/338568437 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4800bbb0 con 0x7fac4c0b8480 2023-10-16T10:05:51.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.615+0000 7fac52d47640 1 -- 172.21.15.84:0/338568437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4c154d20 con 0x7fac4c0b8480 2023-10-16T10:05:51.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.615+0000 7fac52d47640 1 -- 172.21.15.84:0/338568437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c153f90 con 0x7fac4c0b8480 2023-10-16T10:05:51.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.616+0000 7fac52d47640 1 -- 172.21.15.84:0/338568437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4c04e9b0 con 0x7fac4c0b8480 2023-10-16T10:05:51.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.617+0000 7fac427fc640 1 -- 172.21.15.84:0/338568437 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fac4800bd10 con 0x7fac4c0b8480 2023-10-16T10:05:51.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.618+0000 7fac427fc640 1 --2- 172.21.15.84:0/338568437 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fac3006af00 0x7fac3006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:51.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.618+0000 7fac427fc640 1 -- 172.21.15.84:0/338568437 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac48089c50 con 0x7fac4c0b8480 2023-10-16T10:05:51.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.618+0000 7fac51544640 1 --2- 172.21.15.84:0/338568437 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fac3006af00 0x7fac3006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:51.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.619+0000 7fac51544640 1 --2- 172.21.15.84:0/338568437 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fac3006af00 0x7fac3006d3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fac4c1f0240 tx=0x7fac3c008040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:51.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.623+0000 7fac427fc640 1 -- 172.21.15.84:0/338568437 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fac48054c10 con 0x7fac4c0b8480 2023-10-16T10:05:51.729 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:51.726+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:51.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:51.854+0000 7fac52d47640 1 -- 172.21.15.84:0/338568437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fac4c0c85d0 con 0x7fac4c0b8480 2023-10-16T10:05:52.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.377+0000 7fac427fc640 1 -- 172.21.15.84:0/338568437 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) v1 ==== 122+0+0 (secure 0 0 0) 0x7fac48058230 con 0x7fac4c0b8480 2023-10-16T10:05:52.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.382+0000 7fac52d47640 1 -- 172.21.15.84:0/338568437 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fac3006af00 msgr2=0x7fac3006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:52.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.382+0000 7fac52d47640 1 --2- 172.21.15.84:0/338568437 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fac3006af00 0x7fac3006d3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fac4c1f0240 tx=0x7fac3c008040 comp rx=0 tx=0).stop 2023-10-16T10:05:52.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.382+0000 7fac52d47640 1 -- 172.21.15.84:0/338568437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac4c0b8480 msgr2=0x7fac4c153a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:52.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.382+0000 7fac52d47640 1 --2- 172.21.15.84:0/338568437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac4c0b8480 0x7fac4c153a50 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fac4800d800 tx=0x7fac4800dcc0 comp rx=0 tx=0).stop 2023-10-16T10:05:52.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.382+0000 7fac52d47640 1 -- 172.21.15.84:0/338568437 shutdown_connections 2023-10-16T10:05:52.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.382+0000 7fac52d47640 1 --2- 172.21.15.84:0/338568437 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fac3006af00 0x7fac3006d3c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:52.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.382+0000 7fac52d47640 1 --2- 172.21.15.84:0/338568437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac4c15cd70 0x7fac4c1f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:52.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.382+0000 7fac52d47640 1 --2- 172.21.15.84:0/338568437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac4c0d0be0 0x7fac4c0b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:52.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.382+0000 7fac52d47640 1 --2- 172.21.15.84:0/338568437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac4c0b8480 0x7fac4c153a50 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:52.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.382+0000 7fac52d47640 1 -- 172.21.15.84:0/338568437 >> 172.21.15.84:0/338568437 conn(0x7fac4c0cf570 msgr2=0x7fac4c0d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:52.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.383+0000 7fac52d47640 1 -- 172.21.15.84:0/338568437 shutdown_connections 2023-10-16T10:05:52.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.383+0000 7fac52d47640 1 -- 172.21.15.84:0/338568437 wait complete. 2023-10-16T10:05:52.403 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T10:05:52.404 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:05:52.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.612+0000 7fa249e1f640 1 Processor -- start 2023-10-16T10:05:52.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.613+0000 7fa249e1f640 1 -- start start 2023-10-16T10:05:52.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.614+0000 7fa249e1f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa24406b8c0 0x7fa2440cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:52.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.614+0000 7fa249e1f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa24406bca0 0x7fa2440b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:52.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.614+0000 7fa249e1f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2440cf940 con 0x7fa2440b09c0 2023-10-16T10:05:52.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.614+0000 7fa249e1f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2440ce990 con 0x7fa24406bca0 2023-10-16T10:05:52.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.614+0000 7fa249e1f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2440d0310 con 0x7fa24406b8c0 2023-10-16T10:05:52.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.615+0000 7fa2437fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa24406b8c0 0x7fa2440cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:52.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.615+0000 7fa2437fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa24406b8c0 0x7fa2440cfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:40690/0 (socket says 172.21.15.84:40690) 2023-10-16T10:05:52.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.615+0000 7fa2437fe640 1 -- 172.21.15.84:0/850733253 learned_addr learned my addr 172.21.15.84:0/850733253 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:52.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.615+0000 7fa243fff640 1 --1- 172.21.15.84:0/850733253 >> v1:172.21.15.157:6789/0 conn(0x7fa2440b09c0 0x7fa244160a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:58250/0 (socket says 172.21.15.84:58250) 2023-10-16T10:05:52.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.615+0000 7fa242ffd640 1 --2- 172.21.15.84:0/850733253 >> v2:172.21.15.157:3300/0 conn(0x7fa24406bca0 0x7fa2440b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:52.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.616+0000 7fa2437fe640 1 -- 172.21.15.84:0/850733253 >> v2:172.21.15.157:3300/0 conn(0x7fa24406bca0 msgr2=0x7fa2440b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:52.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.616+0000 7fa2437fe640 1 --2- 172.21.15.84:0/850733253 >> v2:172.21.15.157:3300/0 conn(0x7fa24406bca0 0x7fa2440b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:52.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.616+0000 7fa2437fe640 1 -- 172.21.15.84:0/850733253 >> v1:172.21.15.157:6789/0 conn(0x7fa2440b09c0 legacy=0x7fa244160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:52.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.616+0000 7fa2437fe640 1 -- 172.21.15.84:0/850733253 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24415e290 con 0x7fa24406b8c0 2023-10-16T10:05:52.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.616+0000 7fa2427fc640 1 -- 172.21.15.84:0/850733253 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404060392 0 0) 0x7fa2440cf940 con 0x7fa2440b09c0 2023-10-16T10:05:52.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.617+0000 7fa2437fe640 1 --2- 172.21.15.84:0/850733253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa24406b8c0 0x7fa2440cfdd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa234009ea0 tx=0x7fa23402bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45bd9805ad9a1039 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:52.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.618+0000 7fa2427fc640 1 -- 172.21.15.84:0/850733253 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23402cb40 con 0x7fa24406b8c0 2023-10-16T10:05:52.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.618+0000 7fa2427fc640 1 -- 172.21.15.84:0/850733253 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23402cca0 con 0x7fa24406b8c0 2023-10-16T10:05:52.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.618+0000 7fa2427fc640 1 -- 172.21.15.84:0/850733253 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa234034610 con 0x7fa24406b8c0 2023-10-16T10:05:52.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.618+0000 7fa249e1f640 1 -- 172.21.15.84:0/850733253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa24406b8c0 msgr2=0x7fa2440cfdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:52.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.618+0000 7fa249e1f640 1 --2- 172.21.15.84:0/850733253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa24406b8c0 0x7fa2440cfdd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa234009ea0 tx=0x7fa23402bfa0 comp rx=0 tx=0).stop 2023-10-16T10:05:52.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.618+0000 7fa249e1f640 1 -- 172.21.15.84:0/850733253 shutdown_connections 2023-10-16T10:05:52.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.618+0000 7fa249e1f640 1 --2- 172.21.15.84:0/850733253 >> v2:172.21.15.157:3300/0 conn(0x7fa24406bca0 0x7fa2440b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:52.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.618+0000 7fa249e1f640 1 --2- 172.21.15.84:0/850733253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa24406b8c0 0x7fa2440cfdd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:52.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.618+0000 7fa249e1f640 1 -- 172.21.15.84:0/850733253 >> 172.21.15.84:0/850733253 conn(0x7fa244099ea0 msgr2=0x7fa2440d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:52.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.618+0000 7fa249e1f640 1 -- 172.21.15.84:0/850733253 shutdown_connections 2023-10-16T10:05:52.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.618+0000 7fa249e1f640 1 -- 172.21.15.84:0/850733253 wait complete. 2023-10-16T10:05:52.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.620+0000 7fa249e1f640 1 Processor -- start 2023-10-16T10:05:52.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.620+0000 7fa249e1f640 1 -- start start 2023-10-16T10:05:52.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.621+0000 7fa249e1f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa24406b8c0 0x7fa2441f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:52.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.621+0000 7fa249e1f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa24406bca0 0x7fa2440c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:52.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.621+0000 7fa249e1f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2440b09c0 0x7fa2440c5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:52.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.621+0000 7fa249e1f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2440ce0c0 con 0x7fa24406b8c0 2023-10-16T10:05:52.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.621+0000 7fa249e1f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2441f3340 con 0x7fa24406bca0 2023-10-16T10:05:52.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.621+0000 7fa249e1f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2441f2770 con 0x7fa2440b09c0 2023-10-16T10:05:52.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.622+0000 7fa2437fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa24406b8c0 0x7fa2441f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:52.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.622+0000 7fa2437fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa24406b8c0 0x7fa2441f1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60670/0 (socket says 172.21.15.84:60670) 2023-10-16T10:05:52.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.622+0000 7fa2437fe640 1 -- 172.21.15.84:0/1005618819 learned_addr learned my addr 172.21.15.84:0/1005618819 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:52.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.622+0000 7fa243fff640 1 --2- 172.21.15.84:0/1005618819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2440b09c0 0x7fa2440c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:52.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.622+0000 7fa242ffd640 1 --2- 172.21.15.84:0/1005618819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa24406bca0 0x7fa2440c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:52.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.622+0000 7fa2437fe640 1 -- 172.21.15.84:0/1005618819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2440b09c0 msgr2=0x7fa2440c5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:52.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.622+0000 7fa2437fe640 1 --2- 172.21.15.84:0/1005618819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2440b09c0 0x7fa2440c5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:52.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.622+0000 7fa2437fe640 1 -- 172.21.15.84:0/1005618819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa24406bca0 msgr2=0x7fa2440c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:52.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.622+0000 7fa2437fe640 1 --2- 172.21.15.84:0/1005618819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa24406bca0 0x7fa2440c4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:52.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.622+0000 7fa2437fe640 1 -- 172.21.15.84:0/1005618819 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa234009b30 con 0x7fa24406b8c0 2023-10-16T10:05:52.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.622+0000 7fa243fff640 1 --2- 172.21.15.84:0/1005618819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2440b09c0 0x7fa2440c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:52.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.622+0000 7fa2437fe640 1 --2- 172.21.15.84:0/1005618819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa24406b8c0 0x7fa2441f1da0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fa234036040 tx=0x7fa234004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:52.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.623+0000 7fa23bfff640 1 -- 172.21.15.84:0/1005618819 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa234009510 con 0x7fa24406b8c0 2023-10-16T10:05:52.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.623+0000 7fa23bfff640 1 -- 172.21.15.84:0/1005618819 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23403e450 con 0x7fa24406b8c0 2023-10-16T10:05:52.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.623+0000 7fa249e1f640 1 -- 172.21.15.84:0/1005618819 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2441f2a10 con 0x7fa24406b8c0 2023-10-16T10:05:52.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.623+0000 7fa23bfff640 1 -- 172.21.15.84:0/1005618819 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23403e7e0 con 0x7fa24406b8c0 2023-10-16T10:05:52.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.623+0000 7fa249e1f640 1 -- 172.21.15.84:0/1005618819 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2441f2fd0 con 0x7fa24406b8c0 2023-10-16T10:05:52.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.624+0000 7fa249e1f640 1 -- 172.21.15.84:0/1005618819 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2441f3160 con 0x7fa24406b8c0 2023-10-16T10:05:52.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.625+0000 7fa23bfff640 1 -- 172.21.15.84:0/1005618819 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa23403e940 con 0x7fa24406b8c0 2023-10-16T10:05:52.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.626+0000 7fa23bfff640 1 --2- 172.21.15.84:0/1005618819 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fa22406af00 0x7fa22406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:52.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.626+0000 7fa23bfff640 1 -- 172.21.15.84:0/1005618819 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2340ac3a0 con 0x7fa24406b8c0 2023-10-16T10:05:52.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.626+0000 7fa242ffd640 1 --2- 172.21.15.84:0/1005618819 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fa22406af00 0x7fa22406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:52.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.627+0000 7fa242ffd640 1 --2- 172.21.15.84:0/1005618819 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fa22406af00 0x7fa22406d3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa2440c7ac0 tx=0x7fa22c008040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:52.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.630+0000 7fa23bfff640 1 -- 172.21.15.84:0/1005618819 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa234077470 con 0x7fa24406b8c0 2023-10-16T10:05:52.781 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:52.778+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:52.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.873+0000 7fa249e1f640 1 -- 172.21.15.84:0/1005618819 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa24404e160 con 0x7fa24406b8c0 2023-10-16T10:05:52.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.875+0000 7fa23bfff640 1 -- 172.21.15.84:0/1005618819 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fa23407aa90 con 0x7fa24406b8c0 2023-10-16T10:05:52.877 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:52.877 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3458903351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3458903351},{"type":"v1","addr":"172.21.15.84:6837","nonce":3458903351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4002935522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4002935522},{"type":"v1","addr":"172.21.15.157:6833","nonce":4002935522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5809,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1890551305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1890551305},{"type":"v1","addr":"172.21.15.157:6835","nonce":1890551305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:05:48.809508+0000","modified":"2023-10-16T10:05:51.856862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7521":{"gid":7521,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/3621445269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3621445269},{"type":"v1","addr":"172.21.15.84:6836","nonce":3621445269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-16T10:05:52.877 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 53 2023-10-16T10:05:52.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.877+0000 7fa249e1f640 1 -- 172.21.15.84:0/1005618819 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fa22406af00 msgr2=0x7fa22406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:52.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.877+0000 7fa249e1f640 1 --2- 172.21.15.84:0/1005618819 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fa22406af00 0x7fa22406d3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa2440c7ac0 tx=0x7fa22c008040 comp rx=0 tx=0).stop 2023-10-16T10:05:52.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.877+0000 7fa249e1f640 1 -- 172.21.15.84:0/1005618819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa24406b8c0 msgr2=0x7fa2441f1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:52.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.877+0000 7fa249e1f640 1 --2- 172.21.15.84:0/1005618819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa24406b8c0 0x7fa2441f1da0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fa234036040 tx=0x7fa234004a40 comp rx=0 tx=0).stop 2023-10-16T10:05:52.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.877+0000 7fa249e1f640 1 -- 172.21.15.84:0/1005618819 shutdown_connections 2023-10-16T10:05:52.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.878+0000 7fa249e1f640 1 --2- 172.21.15.84:0/1005618819 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fa22406af00 0x7fa22406d3c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:52.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.878+0000 7fa249e1f640 1 --2- 172.21.15.84:0/1005618819 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2440b09c0 0x7fa2440c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:52.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.878+0000 7fa249e1f640 1 --2- 172.21.15.84:0/1005618819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa24406bca0 0x7fa2440c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:52.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.878+0000 7fa249e1f640 1 --2- 172.21.15.84:0/1005618819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa24406b8c0 0x7fa2441f1da0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:52.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.878+0000 7fa249e1f640 1 -- 172.21.15.84:0/1005618819 >> 172.21.15.84:0/1005618819 conn(0x7fa244099ea0 msgr2=0x7fa24415b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:52.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.878+0000 7fa249e1f640 1 -- 172.21.15.84:0/1005618819 shutdown_connections 2023-10-16T10:05:52.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:52.878+0000 7fa249e1f640 1 -- 172.21.15.84:0/1005618819 wait complete. 2023-10-16T10:05:52.898 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:05:53.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.112+0000 7f8809f01640 1 Processor -- start 2023-10-16T10:05:53.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.112+0000 7f8809f01640 1 -- start start 2023-10-16T10:05:53.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.113+0000 7f8809f01640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880415ae80 0x7f880415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:53.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.113+0000 7f8809f01640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f880415ed60 con 0x7f88040ce390 2023-10-16T10:05:53.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.113+0000 7f8809f01640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88040707e0 con 0x7f880415aaa0 2023-10-16T10:05:53.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.113+0000 7f8809f01640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880406b810 con 0x7f880415ae80 2023-10-16T10:05:53.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.114+0000 7f8803fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f88040ce390 0x7f8804070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53688/0 (socket says 172.21.15.84:53688) 2023-10-16T10:05:53.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.114+0000 7f8803fff640 1 -- 172.21.15.84:0/2967921319 learned_addr learned my addr 172.21.15.84:0/2967921319 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.114+0000 7f8802ffd640 1 --2- 172.21.15.84:0/2967921319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880415ae80 0x7f880415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.114+0000 7f88027fc640 1 -- 172.21.15.84:0/2967921319 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3983490613 0 0) 0x7f880415ed60 con 0x7f88040ce390 2023-10-16T10:05:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.114+0000 7f88027fc640 1 -- 172.21.15.84:0/2967921319 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87ec003520 con 0x7f88040ce390 2023-10-16T10:05:53.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.114+0000 7f88027fc640 1 -- 172.21.15.84:0/2967921319 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2739395914 0 0) 0x7f87ec003520 con 0x7f88040ce390 2023-10-16T10:05:53.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f88027fc640 1 -- 172.21.15.84:0/2967921319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880415ae80 msgr2=0x7f880415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:53.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f88027fc640 1 --2- 172.21.15.84:0/2967921319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880415ae80 0x7f880415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f88027fc640 1 -- 172.21.15.84:0/2967921319 >> v1:172.21.15.157:6789/0 conn(0x7f880415aaa0 legacy=0x7f88040cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:53.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f88027fc640 1 -- 172.21.15.84:0/2967921319 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880415ef90 con 0x7f88040ce390 2023-10-16T10:05:53.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f88027fc640 1 -- 172.21.15.84:0/2967921319 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f87fc0019f0 con 0x7f88040ce390 2023-10-16T10:05:53.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f88027fc640 1 -- 172.21.15.84:0/2967921319 >> v1:172.21.15.84:6789/0 conn(0x7f88040ce390 legacy=0x7f8804070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:53.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f88027fc640 1 --2- 172.21.15.84:0/2967921319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880415ae80 0x7f87ec0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:53.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f88027fc640 1 --2- 172.21.15.84:0/2967921319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87ec003f20 0x7f87ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:53.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f88027fc640 1 --2- 172.21.15.84:0/2967921319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87ec004d00 0x7f87ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:53.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f88027fc640 1 -- 172.21.15.84:0/2967921319 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880406b810 con 0x7f87ec003f20 2023-10-16T10:05:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f88027fc640 1 -- 172.21.15.84:0/2967921319 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec00ba60 con 0x7f87ec004d00 2023-10-16T10:05:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f88027fc640 1 -- 172.21.15.84:0/2967921319 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec00bc20 con 0x7f880415ae80 2023-10-16T10:05:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f8802ffd640 1 --2- 172.21.15.84:0/2967921319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87ec003f20 0x7f87ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f88037fe640 1 --2- 172.21.15.84:0/2967921319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880415ae80 0x7f87ec0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f8803fff640 1 --2- 172.21.15.84:0/2967921319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87ec004d00 0x7f87ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:53.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f8802ffd640 1 -- 172.21.15.84:0/2967921319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880415ae80 msgr2=0x7f87ec0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:53.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f8802ffd640 1 --2- 172.21.15.84:0/2967921319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880415ae80 0x7f87ec0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f8802ffd640 1 -- 172.21.15.84:0/2967921319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87ec004d00 msgr2=0x7f87ec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:53.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f8802ffd640 1 --2- 172.21.15.84:0/2967921319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87ec004d00 0x7f87ec005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.115+0000 7f8802ffd640 1 -- 172.21.15.84:0/2967921319 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec003520 con 0x7f87ec003f20 2023-10-16T10:05:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.116+0000 7f88037fe640 1 --2- 172.21.15.84:0/2967921319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880415ae80 0x7f87ec0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.116+0000 7f8803fff640 1 --2- 172.21.15.84:0/2967921319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87ec004d00 0x7f87ec005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.117+0000 7f8802ffd640 1 --2- 172.21.15.84:0/2967921319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87ec003f20 0x7f87ec0047c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009ac0 tx=0x7f87ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9e10cce84c90ec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.117+0000 7f88027fc640 1 -- 172.21.15.84:0/2967921319 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f802c6b0 con 0x7f87ec003f20 2023-10-16T10:05:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.117+0000 7f88027fc640 1 -- 172.21.15.84:0/2967921319 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f802c810 con 0x7f87ec003f20 2023-10-16T10:05:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.117+0000 7f88027fc640 1 -- 172.21.15.84:0/2967921319 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f802cba0 con 0x7f87ec003f20 2023-10-16T10:05:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.117+0000 7f8809f01640 1 -- 172.21.15.84:0/2967921319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87ec003f20 msgr2=0x7f87ec0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.117+0000 7f8809f01640 1 --2- 172.21.15.84:0/2967921319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87ec003f20 0x7f87ec0047c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009ac0 tx=0x7f87ec0057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.117+0000 7f8803fff640 1 -- 172.21.15.84:0/2967921319 reap_dead start 2023-10-16T10:05:53.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.118+0000 7f8809f01640 1 -- 172.21.15.84:0/2967921319 shutdown_connections 2023-10-16T10:05:53.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.118+0000 7f8809f01640 1 -- 172.21.15.84:0/2967921319 >> 172.21.15.84:0/2967921319 conn(0x7f88040cb410 msgr2=0x7f88040cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:53.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.118+0000 7f8809f01640 1 -- 172.21.15.84:0/2967921319 shutdown_connections 2023-10-16T10:05:53.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.118+0000 7f8809f01640 1 -- 172.21.15.84:0/2967921319 wait complete. 2023-10-16T10:05:53.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.119+0000 7f8809f01640 1 Processor -- start 2023-10-16T10:05:53.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.120+0000 7f8809f01640 1 -- start start 2023-10-16T10:05:53.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.121+0000 7f8809f01640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87ec003f20 0x7f8804158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:53.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.121+0000 7f8809f01640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880415db40 0x7f880415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:53.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.121+0000 7f8809f01640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804155970 0x7f8804155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:53.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.121+0000 7f8809f01640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8804156360 con 0x7f87ec003f20 2023-10-16T10:05:53.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.121+0000 7f8809f01640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88040c2660 con 0x7f8804155970 2023-10-16T10:05:53.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.121+0000 7f8809f01640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88040c2940 con 0x7f880415db40 2023-10-16T10:05:53.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.121+0000 7f88037fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87ec003f20 0x7f8804158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:53.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.121+0000 7f88037fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87ec003f20 0x7f8804158c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60688/0 (socket says 172.21.15.84:60688) 2023-10-16T10:05:53.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.121+0000 7f88037fe640 1 -- 172.21.15.84:0/436509844 learned_addr learned my addr 172.21.15.84:0/436509844 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:53.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.121+0000 7f8802ffd640 1 --2- 172.21.15.84:0/436509844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880415db40 0x7f880415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:53.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.121+0000 7f8803fff640 1 --2- 172.21.15.84:0/436509844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804155970 0x7f8804155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:53.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.121+0000 7f88037fe640 1 -- 172.21.15.84:0/436509844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880415db40 msgr2=0x7f880415dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:53.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.121+0000 7f88037fe640 1 --2- 172.21.15.84:0/436509844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880415db40 0x7f880415dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.121+0000 7f88037fe640 1 -- 172.21.15.84:0/436509844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804155970 msgr2=0x7f8804155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:53.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.121+0000 7f88037fe640 1 --2- 172.21.15.84:0/436509844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804155970 0x7f8804155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.121+0000 7f88037fe640 1 -- 172.21.15.84:0/436509844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88040c2c20 con 0x7f87ec003f20 2023-10-16T10:05:53.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.122+0000 7f88037fe640 1 --2- 172.21.15.84:0/436509844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87ec003f20 0x7f8804158c70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f87f400d770 tx=0x7f87f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:53.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.122+0000 7f87ebfff640 1 -- 172.21.15.84:0/436509844 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4005270 con 0x7f87ec003f20 2023-10-16T10:05:53.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.122+0000 7f87ebfff640 1 -- 172.21.15.84:0/436509844 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f40053d0 con 0x7f87ec003f20 2023-10-16T10:05:53.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.122+0000 7f87ebfff640 1 -- 172.21.15.84:0/436509844 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f400b8f0 con 0x7f87ec003f20 2023-10-16T10:05:53.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.122+0000 7f8809f01640 1 -- 172.21.15.84:0/436509844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88040c2f00 con 0x7f87ec003f20 2023-10-16T10:05:53.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.122+0000 7f8809f01640 1 -- 172.21.15.84:0/436509844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880415c5b0 con 0x7f87ec003f20 2023-10-16T10:05:53.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.123+0000 7f8809f01640 1 -- 172.21.15.84:0/436509844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f880404e160 con 0x7f87ec003f20 2023-10-16T10:05:53.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.124+0000 7f87ebfff640 1 -- 172.21.15.84:0/436509844 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f87f400ba50 con 0x7f87ec003f20 2023-10-16T10:05:53.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.125+0000 7f87ebfff640 1 --2- 172.21.15.84:0/436509844 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f87f006af00 0x7f87f006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:53.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.125+0000 7f87ebfff640 1 -- 172.21.15.84:0/436509844 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87f40895e0 con 0x7f87ec003f20 2023-10-16T10:05:53.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.125+0000 7f8802ffd640 1 --2- 172.21.15.84:0/436509844 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f87f006af00 0x7f87f006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:53.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.126+0000 7f8802ffd640 1 --2- 172.21.15.84:0/436509844 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f87f006af00 0x7f87f006d3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f87f8036000 tx=0x7f87f8030000 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:53.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.130+0000 7f87ebfff640 1 -- 172.21.15.84:0/436509844 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f87f4054620 con 0x7f87ec003f20 2023-10-16T10:05:53.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.329+0000 7f8809f01640 1 -- 172.21.15.84:0/436509844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88040cdc60 con 0x7f87ec003f20 2023-10-16T10:05:53.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.331+0000 7f87ebfff640 1 -- 172.21.15.84:0/436509844 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16547 (secure 0 0 0) 0x7f87f405e030 con 0x7f87ec003f20 2023-10-16T10:05:53.333 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:53.333 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":108,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:05:50.339706+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:05:46.595888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-16T10:05:47.826019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:05:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.334+0000 7f8809f01640 1 -- 172.21.15.84:0/436509844 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f87f006af00 msgr2=0x7f87f006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.334+0000 7f8809f01640 1 --2- 172.21.15.84:0/436509844 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f87f006af00 0x7f87f006d3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f87f8036000 tx=0x7f87f8030000 comp rx=0 tx=0).stop 2023-10-16T10:05:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.334+0000 7f8809f01640 1 -- 172.21.15.84:0/436509844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87ec003f20 msgr2=0x7f8804158c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.334+0000 7f8809f01640 1 --2- 172.21.15.84:0/436509844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87ec003f20 0x7f8804158c70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f87f400d770 tx=0x7f87f400dc30 comp rx=0 tx=0).stop 2023-10-16T10:05:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.334+0000 7f8809f01640 1 -- 172.21.15.84:0/436509844 shutdown_connections 2023-10-16T10:05:53.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.334+0000 7f8809f01640 1 --2- 172.21.15.84:0/436509844 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f87f006af00 0x7f87f006d3c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.334+0000 7f8809f01640 1 --2- 172.21.15.84:0/436509844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804155970 0x7f8804155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.334+0000 7f8809f01640 1 --2- 172.21.15.84:0/436509844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880415db40 0x7f880415dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.334+0000 7f8809f01640 1 --2- 172.21.15.84:0/436509844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87ec003f20 0x7f8804158c70 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.334+0000 7f8809f01640 1 -- 172.21.15.84:0/436509844 >> 172.21.15.84:0/436509844 conn(0x7f88040cb410 msgr2=0x7f880406ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:53.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.334+0000 7f8809f01640 1 -- 172.21.15.84:0/436509844 shutdown_connections 2023-10-16T10:05:53.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.334+0000 7f8809f01640 1 -- 172.21.15.84:0/436509844 wait complete. 2023-10-16T10:05:53.353 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T10:05:53.354 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:05:53.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.554+0000 7facfb3b0640 1 Processor -- start 2023-10-16T10:05:53.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.554+0000 7facfb3b0640 1 -- start start 2023-10-16T10:05:53.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.556+0000 7facfb3b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facf40cd0c0 0x7facf41615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:53.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.556+0000 7facfb3b0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7facf40b0410 0x7facf40b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:53.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.556+0000 7facfb3b0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf40b0db0 con 0x7facf415d3f0 2023-10-16T10:05:53.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.556+0000 7facfb3b0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf415e3e0 con 0x7facf40b0410 2023-10-16T10:05:53.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.556+0000 7facfb3b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf40cd4a0 con 0x7facf40cd0c0 2023-10-16T10:05:53.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.557+0000 7facf914c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facf40cd0c0 0x7facf41615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:53.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.557+0000 7facf914c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facf40cd0c0 0x7facf41615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:40726/0 (socket says 172.21.15.84:40726) 2023-10-16T10:05:53.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.557+0000 7facf914c640 1 -- 172.21.15.84:0/4169608974 learned_addr learned my addr 172.21.15.84:0/4169608974 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:53.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.557+0000 7facf894b640 1 --2- 172.21.15.84:0/4169608974 >> v2:172.21.15.157:3300/0 conn(0x7facf40b0410 0x7facf40b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:53.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.557+0000 7facebfff640 1 -- 172.21.15.84:0/4169608974 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821276936 0 0) 0x7facf40b0db0 con 0x7facf415d3f0 2023-10-16T10:05:53.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.557+0000 7facebfff640 1 -- 172.21.15.84:0/4169608974 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7face0003520 con 0x7facf415d3f0 2023-10-16T10:05:53.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.558+0000 7facf914c640 1 -- 172.21.15.84:0/4169608974 >> v2:172.21.15.157:3300/0 conn(0x7facf40b0410 msgr2=0x7facf40b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:53.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.558+0000 7facf914c640 1 --2- 172.21.15.84:0/4169608974 >> v2:172.21.15.157:3300/0 conn(0x7facf40b0410 0x7facf40b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.558+0000 7facf914c640 1 -- 172.21.15.84:0/4169608974 >> v1:172.21.15.157:6789/0 conn(0x7facf415d3f0 legacy=0x7facf415d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:53.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.558+0000 7facf914c640 1 -- 172.21.15.84:0/4169608974 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf415e100 con 0x7facf40cd0c0 2023-10-16T10:05:53.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.559+0000 7facf914c640 1 --2- 172.21.15.84:0/4169608974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facf40cd0c0 0x7facf41615f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7facf0009e50 tx=0x7facf002bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=682f303843524b4c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:53.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.560+0000 7facebfff640 1 -- 172.21.15.84:0/4169608974 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf002ca70 con 0x7facf40cd0c0 2023-10-16T10:05:53.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.560+0000 7facebfff640 1 -- 172.21.15.84:0/4169608974 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf002cbd0 con 0x7facf40cd0c0 2023-10-16T10:05:53.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.560+0000 7facebfff640 1 -- 172.21.15.84:0/4169608974 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf00343f0 con 0x7facf40cd0c0 2023-10-16T10:05:53.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.561+0000 7facfb3b0640 1 -- 172.21.15.84:0/4169608974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facf40cd0c0 msgr2=0x7facf41615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:53.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.561+0000 7facfb3b0640 1 --2- 172.21.15.84:0/4169608974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facf40cd0c0 0x7facf41615f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7facf0009e50 tx=0x7facf002bed0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.561+0000 7facfb3b0640 1 -- 172.21.15.84:0/4169608974 shutdown_connections 2023-10-16T10:05:53.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.561+0000 7facfb3b0640 1 --2- 172.21.15.84:0/4169608974 >> v2:172.21.15.157:3300/0 conn(0x7facf40b0410 0x7facf40b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.561+0000 7facfb3b0640 1 --2- 172.21.15.84:0/4169608974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facf40cd0c0 0x7facf41615f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.561+0000 7facfb3b0640 1 -- 172.21.15.84:0/4169608974 >> 172.21.15.84:0/4169608974 conn(0x7facf40cb410 msgr2=0x7facf40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:53.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.561+0000 7facfb3b0640 1 -- 172.21.15.84:0/4169608974 shutdown_connections 2023-10-16T10:05:53.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.561+0000 7facfb3b0640 1 -- 172.21.15.84:0/4169608974 wait complete. 2023-10-16T10:05:53.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.563+0000 7facfb3b0640 1 Processor -- start 2023-10-16T10:05:53.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.563+0000 7facfb3b0640 1 -- start start 2023-10-16T10:05:53.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.564+0000 7facfb3b0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf40b0410 0x7facf4156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:53.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.564+0000 7facfb3b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facf40cd0c0 0x7facf415c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:53.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.564+0000 7facfb3b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facf415d3f0 0x7facf415c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:53.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.564+0000 7facfb3b0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf41574f0 con 0x7facf40b0410 2023-10-16T10:05:53.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.564+0000 7facfb3b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf4158e20 con 0x7facf40cd0c0 2023-10-16T10:05:53.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.564+0000 7facfb3b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf40b2260 con 0x7facf415d3f0 2023-10-16T10:05:53.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.564+0000 7facf914c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf40b0410 0x7facf4156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:53.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.564+0000 7facf914c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf40b0410 0x7facf4156fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60698/0 (socket says 172.21.15.84:60698) 2023-10-16T10:05:53.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.564+0000 7facf914c640 1 -- 172.21.15.84:0/4064095821 learned_addr learned my addr 172.21.15.84:0/4064095821 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:53.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.565+0000 7facf894b640 1 --2- 172.21.15.84:0/4064095821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facf40cd0c0 0x7facf415c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:53.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.565+0000 7facf994d640 1 --2- 172.21.15.84:0/4064095821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facf415d3f0 0x7facf415c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:53.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.565+0000 7facf914c640 1 -- 172.21.15.84:0/4064095821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facf415d3f0 msgr2=0x7facf415c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:53.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.565+0000 7facf914c640 1 --2- 172.21.15.84:0/4064095821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facf415d3f0 0x7facf415c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.565+0000 7facf914c640 1 -- 172.21.15.84:0/4064095821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facf40cd0c0 msgr2=0x7facf415c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:53.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.565+0000 7facf914c640 1 --2- 172.21.15.84:0/4064095821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facf40cd0c0 0x7facf415c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.565+0000 7facf914c640 1 -- 172.21.15.84:0/4064095821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf0009b30 con 0x7facf40b0410 2023-10-16T10:05:53.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.565+0000 7facf914c640 1 --2- 172.21.15.84:0/4064095821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf40b0410 0x7facf4156fb0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7facf0036040 tx=0x7facf0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:53.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.565+0000 7face9ffb640 1 -- 172.21.15.84:0/4064095821 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0009510 con 0x7facf40b0410 2023-10-16T10:05:53.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.565+0000 7face9ffb640 1 -- 172.21.15.84:0/4064095821 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf003e450 con 0x7facf40b0410 2023-10-16T10:05:53.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.565+0000 7facfb3b0640 1 -- 172.21.15.84:0/4064095821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf40b2540 con 0x7facf40b0410 2023-10-16T10:05:53.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.565+0000 7face9ffb640 1 -- 172.21.15.84:0/4064095821 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf003e800 con 0x7facf40b0410 2023-10-16T10:05:53.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.566+0000 7facfb3b0640 1 -- 172.21.15.84:0/4064095821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf40b2b00 con 0x7facf40b0410 2023-10-16T10:05:53.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.567+0000 7facfb3b0640 1 -- 172.21.15.84:0/4064095821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf404e160 con 0x7facf40b0410 2023-10-16T10:05:53.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.568+0000 7face9ffb640 1 -- 172.21.15.84:0/4064095821 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7facf003e960 con 0x7facf40b0410 2023-10-16T10:05:53.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.569+0000 7face9ffb640 1 --2- 172.21.15.84:0/4064095821 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7facd806af30 0x7facd806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:53.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.569+0000 7face9ffb640 1 -- 172.21.15.84:0/4064095821 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7facf00ac280 con 0x7facf40b0410 2023-10-16T10:05:53.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.569+0000 7facf894b640 1 --2- 172.21.15.84:0/4064095821 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7facd806af30 0x7facd806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:53.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.569+0000 7facf894b640 1 --2- 172.21.15.84:0/4064095821 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7facd806af30 0x7facd806d3f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7facf40cf320 tx=0x7face4008040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:53.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.574+0000 7face9ffb640 1 -- 172.21.15.84:0/4064095821 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7facf0077340 con 0x7facf40b0410 2023-10-16T10:05:53.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.765+0000 7facfb3b0640 1 -- 172.21.15.84:0/4064095821 --> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7facf4160df0 con 0x7facd806af30 2023-10-16T10:05:53.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.795+0000 7face9ffb640 1 -- 172.21.15.84:0/4064095821 <== mgr.7284 v2:172.21.15.84:6832/3250823028 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287716 (secure 0 0 0) 0x7face40118b0 con 0x7facd806af30 2023-10-16T10:05:53.800 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:53.802 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T10:05:53.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.802+0000 7facfb3b0640 1 -- 172.21.15.84:0/4064095821 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7facd806af30 msgr2=0x7facd806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:53.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.802+0000 7facfb3b0640 1 --2- 172.21.15.84:0/4064095821 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7facd806af30 0x7facd806d3f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7facf40cf320 tx=0x7face4008040 comp rx=0 tx=0).stop 2023-10-16T10:05:53.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.802+0000 7facfb3b0640 1 -- 172.21.15.84:0/4064095821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf40b0410 msgr2=0x7facf4156fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:53.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.802+0000 7facfb3b0640 1 --2- 172.21.15.84:0/4064095821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf40b0410 0x7facf4156fb0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7facf0036040 tx=0x7facf0004a40 comp rx=0 tx=0).stop 2023-10-16T10:05:53.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.802+0000 7facfb3b0640 1 -- 172.21.15.84:0/4064095821 shutdown_connections 2023-10-16T10:05:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.802+0000 7facfb3b0640 1 --2- 172.21.15.84:0/4064095821 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7facd806af30 0x7facd806d3f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.802+0000 7facfb3b0640 1 --2- 172.21.15.84:0/4064095821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facf415d3f0 0x7facf415c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.802+0000 7facfb3b0640 1 --2- 172.21.15.84:0/4064095821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facf40cd0c0 0x7facf415c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.802+0000 7facfb3b0640 1 --2- 172.21.15.84:0/4064095821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf40b0410 0x7facf4156fb0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.802+0000 7facfb3b0640 1 -- 172.21.15.84:0/4064095821 >> 172.21.15.84:0/4064095821 conn(0x7facf40cb410 msgr2=0x7facf40ca3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.802+0000 7facfb3b0640 1 -- 172.21.15.84:0/4064095821 shutdown_connections 2023-10-16T10:05:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:53.802+0000 7facfb3b0640 1 -- 172.21.15.84:0/4064095821 wait complete. 2023-10-16T10:05:53.818 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:53.815+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:53.825 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-16T10:05:53.478282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":640,"num_read_kb":1170,"num_write":305,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2186,"ondisk_log_size":2186,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":322088,"kb_used_data":11592,"kb_used_omap":12,"kb_used_meta":310387,"kb_avail":754652632,"statfs":{"total":773094113280,"available":772764295168,"internally_reserved":0,"allocated":11870208,"data_stored":6242012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":317836857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001870"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.361271+0000","last_change":"2023-10-16T10:05:49.362542+0000","last_active":"2023-10-16T10:05:50.361271+0000","last_peered":"2023-10-16T10:05:50.361271+0000","last_clean":"2023-10-16T10:05:50.361271+0000","last_became_active":"2023-10-16T10:05:49.361614+0000","last_became_peered":"2023-10-16T10:05:49.361614+0000","last_unstale":"2023-10-16T10:05:50.361271+0000","last_undegraded":"2023-10-16T10:05:50.361271+0000","last_fullsized":"2023-10-16T10:05:50.361271+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:54.756998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.899649+0000","last_change":"2023-10-16T10:05:49.372635+0000","last_active":"2023-10-16T10:05:50.899649+0000","last_peered":"2023-10-16T10:05:50.899649+0000","last_clean":"2023-10-16T10:05:50.899649+0000","last_became_active":"2023-10-16T10:05:49.372346+0000","last_became_peered":"2023-10-16T10:05:49.372346+0000","last_unstale":"2023-10-16T10:05:50.899649+0000","last_undegraded":"2023-10-16T10:05:50.899649+0000","last_fullsized":"2023-10-16T10:05:50.899649+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:24:34.556467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.697038+0000","last_change":"2023-10-16T10:05:49.369954+0000","last_active":"2023-10-16T10:05:50.697038+0000","last_peered":"2023-10-16T10:05:50.697038+0000","last_clean":"2023-10-16T10:05:50.697038+0000","last_became_active":"2023-10-16T10:05:49.368928+0000","last_became_peered":"2023-10-16T10:05:49.368928+0000","last_unstale":"2023-10-16T10:05:50.697038+0000","last_undegraded":"2023-10-16T10:05:50.697038+0000","last_fullsized":"2023-10-16T10:05:50.697038+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:28:59.129150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.353862+0000","last_change":"2023-10-16T10:05:49.360594+0000","last_active":"2023-10-16T10:05:50.353862+0000","last_peered":"2023-10-16T10:05:50.353862+0000","last_clean":"2023-10-16T10:05:50.353862+0000","last_became_active":"2023-10-16T10:05:49.360319+0000","last_became_peered":"2023-10-16T10:05:49.360319+0000","last_unstale":"2023-10-16T10:05:50.353862+0000","last_undegraded":"2023-10-16T10:05:50.353862+0000","last_fullsized":"2023-10-16T10:05:50.353862+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:10:41.364870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.357425+0000","last_change":"2023-10-16T10:05:49.357925+0000","last_active":"2023-10-16T10:05:50.357425+0000","last_peered":"2023-10-16T10:05:50.357425+0000","last_clean":"2023-10-16T10:05:50.357425+0000","last_became_active":"2023-10-16T10:05:49.357658+0000","last_became_peered":"2023-10-16T10:05:49.357658+0000","last_unstale":"2023-10-16T10:05:50.357425+0000","last_undegraded":"2023-10-16T10:05:50.357425+0000","last_fullsized":"2023-10-16T10:05:50.357425+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:18.108605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.897841+0000","last_change":"2023-10-16T10:05:49.369444+0000","last_active":"2023-10-16T10:05:50.897841+0000","last_peered":"2023-10-16T10:05:50.897841+0000","last_clean":"2023-10-16T10:05:50.897841+0000","last_became_active":"2023-10-16T10:05:49.369082+0000","last_became_peered":"2023-10-16T10:05:49.369082+0000","last_unstale":"2023-10-16T10:05:50.897841+0000","last_undegraded":"2023-10-16T10:05:50.897841+0000","last_fullsized":"2023-10-16T10:05:50.897841+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:30:54.892423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.353466+0000","last_change":"2023-10-16T10:05:49.365766+0000","last_active":"2023-10-16T10:05:50.353466+0000","last_peered":"2023-10-16T10:05:50.353466+0000","last_clean":"2023-10-16T10:05:50.353466+0000","last_became_active":"2023-10-16T10:05:49.365494+0000","last_became_peered":"2023-10-16T10:05:49.365494+0000","last_unstale":"2023-10-16T10:05:50.353466+0000","last_undegraded":"2023-10-16T10:05:50.353466+0000","last_fullsized":"2023-10-16T10:05:50.353466+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:21.782751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.897553+0000","last_change":"2023-10-16T10:05:49.358313+0000","last_active":"2023-10-16T10:05:50.897553+0000","last_peered":"2023-10-16T10:05:50.897553+0000","last_clean":"2023-10-16T10:05:50.897553+0000","last_became_active":"2023-10-16T10:05:49.357332+0000","last_became_peered":"2023-10-16T10:05:49.357332+0000","last_unstale":"2023-10-16T10:05:50.897553+0000","last_undegraded":"2023-10-16T10:05:50.897553+0000","last_fullsized":"2023-10-16T10:05:50.897553+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:56.984492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.898611+0000","last_change":"2023-10-16T10:05:49.365756+0000","last_active":"2023-10-16T10:05:50.898611+0000","last_peered":"2023-10-16T10:05:50.898611+0000","last_clean":"2023-10-16T10:05:50.898611+0000","last_became_active":"2023-10-16T10:05:49.363995+0000","last_became_peered":"2023-10-16T10:05:49.363995+0000","last_unstale":"2023-10-16T10:05:50.898611+0000","last_undegraded":"2023-10-16T10:05:50.898611+0000","last_fullsized":"2023-10-16T10:05:50.898611+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:53:29.830897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.361002+0000","last_change":"2023-10-16T10:05:49.362906+0000","last_active":"2023-10-16T10:05:50.361002+0000","last_peered":"2023-10-16T10:05:50.361002+0000","last_clean":"2023-10-16T10:05:50.361002+0000","last_became_active":"2023-10-16T10:05:49.362569+0000","last_became_peered":"2023-10-16T10:05:49.362569+0000","last_unstale":"2023-10-16T10:05:50.361002+0000","last_undegraded":"2023-10-16T10:05:50.361002+0000","last_fullsized":"2023-10-16T10:05:50.361002+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:21.802686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.900551+0000","last_change":"2023-10-16T10:05:49.357681+0000","last_active":"2023-10-16T10:05:50.900551+0000","last_peered":"2023-10-16T10:05:50.900551+0000","last_clean":"2023-10-16T10:05:50.900551+0000","last_became_active":"2023-10-16T10:05:49.357394+0000","last_became_peered":"2023-10-16T10:05:49.357394+0000","last_unstale":"2023-10-16T10:05:50.900551+0000","last_undegraded":"2023-10-16T10:05:50.900551+0000","last_fullsized":"2023-10-16T10:05:50.900551+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:06:38.741132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.359436+0000","last_change":"2023-10-16T10:05:49.364749+0000","last_active":"2023-10-16T10:05:50.359436+0000","last_peered":"2023-10-16T10:05:50.359436+0000","last_clean":"2023-10-16T10:05:50.359436+0000","last_became_active":"2023-10-16T10:05:49.364308+0000","last_became_peered":"2023-10-16T10:05:49.364308+0000","last_unstale":"2023-10-16T10:05:50.359436+0000","last_undegraded":"2023-10-16T10:05:50.359436+0000","last_fullsized":"2023-10-16T10:05:50.359436+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:52.919073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.696690+0000","last_change":"2023-10-16T10:05:49.370396+0000","last_active":"2023-10-16T10:05:50.696690+0000","last_peered":"2023-10-16T10:05:50.696690+0000","last_clean":"2023-10-16T10:05:50.696690+0000","last_became_active":"2023-10-16T10:05:49.369607+0000","last_became_peered":"2023-10-16T10:05:49.369607+0000","last_unstale":"2023-10-16T10:05:50.696690+0000","last_undegraded":"2023-10-16T10:05:50.696690+0000","last_fullsized":"2023-10-16T10:05:50.696690+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:03:23.479796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.353492+0000","last_change":"2023-10-16T10:05:49.361044+0000","last_active":"2023-10-16T10:05:50.353492+0000","last_peered":"2023-10-16T10:05:50.353492+0000","last_clean":"2023-10-16T10:05:50.353492+0000","last_became_active":"2023-10-16T10:05:49.360562+0000","last_became_peered":"2023-10-16T10:05:49.360562+0000","last_unstale":"2023-10-16T10:05:50.353492+0000","last_undegraded":"2023-10-16T10:05:50.353492+0000","last_fullsized":"2023-10-16T10:05:50.353492+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:39:33.028029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.898245+0000","last_change":"2023-10-16T10:05:49.356572+0000","last_active":"2023-10-16T10:05:50.898245+0000","last_peered":"2023-10-16T10:05:50.898245+0000","last_clean":"2023-10-16T10:05:50.898245+0000","last_became_active":"2023-10-16T10:05:49.354173+0000","last_became_peered":"2023-10-16T10:05:49.354173+0000","last_unstale":"2023-10-16T10:05:50.898245+0000","last_undegraded":"2023-10-16T10:05:50.898245+0000","last_fullsized":"2023-10-16T10:05:50.898245+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:06.670523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.356640+0000","last_change":"2023-10-16T10:05:49.363448+0000","last_active":"2023-10-16T10:05:50.356640+0000","last_peered":"2023-10-16T10:05:50.356640+0000","last_clean":"2023-10-16T10:05:50.356640+0000","last_became_active":"2023-10-16T10:05:49.363099+0000","last_became_peered":"2023-10-16T10:05:49.363099+0000","last_unstale":"2023-10-16T10:05:50.356640+0000","last_undegraded":"2023-10-16T10:05:50.356640+0000","last_fullsized":"2023-10-16T10:05:50.356640+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:17:51.752174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.360033+0000","last_change":"2023-10-16T10:05:49.345381+0000","last_active":"2023-10-16T10:05:50.360033+0000","last_peered":"2023-10-16T10:05:50.360033+0000","last_clean":"2023-10-16T10:05:50.360033+0000","last_became_active":"2023-10-16T10:05:49.345087+0000","last_became_peered":"2023-10-16T10:05:49.345087+0000","last_unstale":"2023-10-16T10:05:50.360033+0000","last_undegraded":"2023-10-16T10:05:50.360033+0000","last_fullsized":"2023-10-16T10:05:50.360033+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:44:34.031744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.354356+0000","last_change":"2023-10-16T10:05:49.367804+0000","last_active":"2023-10-16T10:05:50.354356+0000","last_peered":"2023-10-16T10:05:50.354356+0000","last_clean":"2023-10-16T10:05:50.354356+0000","last_became_active":"2023-10-16T10:05:49.367512+0000","last_became_peered":"2023-10-16T10:05:49.367512+0000","last_unstale":"2023-10-16T10:05:50.354356+0000","last_undegraded":"2023-10-16T10:05:50.354356+0000","last_fullsized":"2023-10-16T10:05:50.354356+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:43:44.531615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.690068+0000","last_change":"2023-10-16T10:05:49.348920+0000","last_active":"2023-10-16T10:05:50.690068+0000","last_peered":"2023-10-16T10:05:50.690068+0000","last_clean":"2023-10-16T10:05:50.690068+0000","last_became_active":"2023-10-16T10:05:49.348283+0000","last_became_peered":"2023-10-16T10:05:49.348283+0000","last_unstale":"2023-10-16T10:05:50.690068+0000","last_undegraded":"2023-10-16T10:05:50.690068+0000","last_fullsized":"2023-10-16T10:05:50.690068+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:39:33.028029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.353875+0000","last_change":"2023-10-16T10:05:49.358870+0000","last_active":"2023-10-16T10:05:50.353875+0000","last_peered":"2023-10-16T10:05:50.353875+0000","last_clean":"2023-10-16T10:05:50.353875+0000","last_became_active":"2023-10-16T10:05:49.358422+0000","last_became_peered":"2023-10-16T10:05:49.358422+0000","last_unstale":"2023-10-16T10:05:50.353875+0000","last_undegraded":"2023-10-16T10:05:50.353875+0000","last_fullsized":"2023-10-16T10:05:50.353875+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:52.919073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.353789+0000","last_change":"2023-10-16T10:05:49.341994+0000","last_active":"2023-10-16T10:05:50.353789+0000","last_peered":"2023-10-16T10:05:50.353789+0000","last_clean":"2023-10-16T10:05:50.353789+0000","last_became_active":"2023-10-16T10:05:49.341723+0000","last_became_peered":"2023-10-16T10:05:49.341723+0000","last_unstale":"2023-10-16T10:05:50.353789+0000","last_undegraded":"2023-10-16T10:05:50.353789+0000","last_fullsized":"2023-10-16T10:05:50.353789+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:20.377792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.898245+0000","last_change":"2023-10-16T10:05:49.357323+0000","last_active":"2023-10-16T10:05:50.898245+0000","last_peered":"2023-10-16T10:05:50.898245+0000","last_clean":"2023-10-16T10:05:50.898245+0000","last_became_active":"2023-10-16T10:05:49.357017+0000","last_became_peered":"2023-10-16T10:05:49.357017+0000","last_unstale":"2023-10-16T10:05:50.898245+0000","last_undegraded":"2023-10-16T10:05:50.898245+0000","last_fullsized":"2023-10-16T10:05:50.898245+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:12:58.465258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.360485+0000","last_change":"2023-10-16T10:05:49.373922+0000","last_active":"2023-10-16T10:05:50.360485+0000","last_peered":"2023-10-16T10:05:50.360485+0000","last_clean":"2023-10-16T10:05:50.360485+0000","last_became_active":"2023-10-16T10:05:49.373519+0000","last_became_peered":"2023-10-16T10:05:49.373519+0000","last_unstale":"2023-10-16T10:05:50.360485+0000","last_undegraded":"2023-10-16T10:05:50.360485+0000","last_fullsized":"2023-10-16T10:05:50.360485+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:02:58.545300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.353632+0000","last_change":"2023-10-16T10:05:49.362481+0000","last_active":"2023-10-16T10:05:50.353632+0000","last_peered":"2023-10-16T10:05:50.353632+0000","last_clean":"2023-10-16T10:05:50.353632+0000","last_became_active":"2023-10-16T10:05:49.362127+0000","last_became_peered":"2023-10-16T10:05:49.362127+0000","last_unstale":"2023-10-16T10:05:50.353632+0000","last_undegraded":"2023-10-16T10:05:50.353632+0000","last_fullsized":"2023-10-16T10:05:50.353632+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:08:05.035979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.899217+0000","last_change":"2023-10-16T10:05:49.364611+0000","last_active":"2023-10-16T10:05:50.899217+0000","last_peered":"2023-10-16T10:05:50.899217+0000","last_clean":"2023-10-16T10:05:50.899217+0000","last_became_active":"2023-10-16T10:05:49.364384+0000","last_became_peered":"2023-10-16T10:05:49.364384+0000","last_unstale":"2023-10-16T10:05:50.899217+0000","last_undegraded":"2023-10-16T10:05:50.899217+0000","last_fullsized":"2023-10-16T10:05:50.899217+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:05.264470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.697230+0000","last_change":"2023-10-16T10:05:49.356210+0000","last_active":"2023-10-16T10:05:50.697230+0000","last_peered":"2023-10-16T10:05:50.697230+0000","last_clean":"2023-10-16T10:05:50.697230+0000","last_became_active":"2023-10-16T10:05:49.355931+0000","last_became_peered":"2023-10-16T10:05:49.355931+0000","last_unstale":"2023-10-16T10:05:50.697230+0000","last_undegraded":"2023-10-16T10:05:50.697230+0000","last_fullsized":"2023-10-16T10:05:50.697230+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:28:30.066110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.357152+0000","last_change":"2023-10-16T10:05:49.349801+0000","last_active":"2023-10-16T10:05:50.357152+0000","last_peered":"2023-10-16T10:05:50.357152+0000","last_clean":"2023-10-16T10:05:50.357152+0000","last_became_active":"2023-10-16T10:05:49.349467+0000","last_became_peered":"2023-10-16T10:05:49.349467+0000","last_unstale":"2023-10-16T10:05:50.357152+0000","last_undegraded":"2023-10-16T10:05:50.357152+0000","last_fullsized":"2023-10-16T10:05:50.357152+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:02.441122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.359625+0000","last_change":"2023-10-16T10:05:49.364564+0000","last_active":"2023-10-16T10:05:50.359625+0000","last_peered":"2023-10-16T10:05:50.359625+0000","last_clean":"2023-10-16T10:05:50.359625+0000","last_became_active":"2023-10-16T10:05:49.363888+0000","last_became_peered":"2023-10-16T10:05:49.363888+0000","last_unstale":"2023-10-16T10:05:50.359625+0000","last_undegraded":"2023-10-16T10:05:50.359625+0000","last_fullsized":"2023-10-16T10:05:50.359625+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:34:31.006127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.899760+0000","last_change":"2023-10-16T10:05:49.350648+0000","last_active":"2023-10-16T10:05:50.899760+0000","last_peered":"2023-10-16T10:05:50.899760+0000","last_clean":"2023-10-16T10:05:50.899760+0000","last_became_active":"2023-10-16T10:05:49.346019+0000","last_became_peered":"2023-10-16T10:05:49.346019+0000","last_unstale":"2023-10-16T10:05:50.899760+0000","last_undegraded":"2023-10-16T10:05:50.899760+0000","last_fullsized":"2023-10-16T10:05:50.899760+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:03.181971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.898613+0000","last_change":"2023-10-16T10:05:49.352496+0000","last_active":"2023-10-16T10:05:50.898613+0000","last_peered":"2023-10-16T10:05:50.898613+0000","last_clean":"2023-10-16T10:05:50.898613+0000","last_became_active":"2023-10-16T10:05:49.351819+0000","last_became_peered":"2023-10-16T10:05:49.351819+0000","last_unstale":"2023-10-16T10:05:50.898613+0000","last_undegraded":"2023-10-16T10:05:50.898613+0000","last_fullsized":"2023-10-16T10:05:50.898613+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:54:12.892643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.356939+0000","last_change":"2023-10-16T10:05:49.341898+0000","last_active":"2023-10-16T10:05:50.356939+0000","last_peered":"2023-10-16T10:05:50.356939+0000","last_clean":"2023-10-16T10:05:50.356939+0000","last_became_active":"2023-10-16T10:05:49.341209+0000","last_became_peered":"2023-10-16T10:05:49.341209+0000","last_unstale":"2023-10-16T10:05:50.356939+0000","last_undegraded":"2023-10-16T10:05:50.356939+0000","last_fullsized":"2023-10-16T10:05:50.356939+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:59:41.639814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.357362+0000","last_change":"2023-10-16T10:05:49.362375+0000","last_active":"2023-10-16T10:05:50.357362+0000","last_peered":"2023-10-16T10:05:50.357362+0000","last_clean":"2023-10-16T10:05:50.357362+0000","last_became_active":"2023-10-16T10:05:49.361934+0000","last_became_peered":"2023-10-16T10:05:49.361934+0000","last_unstale":"2023-10-16T10:05:50.357362+0000","last_undegraded":"2023-10-16T10:05:50.357362+0000","last_fullsized":"2023-10-16T10:05:50.357362+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:10:41.364870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.359222+0000","last_change":"2023-10-16T10:05:49.365913+0000","last_active":"2023-10-16T10:05:50.359222+0000","last_peered":"2023-10-16T10:05:50.359222+0000","last_clean":"2023-10-16T10:05:50.359222+0000","last_became_active":"2023-10-16T10:05:49.364886+0000","last_became_peered":"2023-10-16T10:05:49.364886+0000","last_unstale":"2023-10-16T10:05:50.359222+0000","last_undegraded":"2023-10-16T10:05:50.359222+0000","last_fullsized":"2023-10-16T10:05:50.359222+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:13:55.500718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"107'4","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.373750+0000","last_change":"2023-10-16T10:05:48.353738+0000","last_active":"2023-10-16T10:05:49.373750+0000","last_peered":"2023-10-16T10:05:49.373750+0000","last_clean":"2023-10-16T10:05:49.373750+0000","last_became_active":"2023-10-16T10:05:48.352267+0000","last_became_peered":"2023-10-16T10:05:48.352267+0000","last_unstale":"2023-10-16T10:05:49.373750+0000","last_undegraded":"2023-10-16T10:05:49.373750+0000","last_fullsized":"2023-10-16T10:05:49.373750+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:51:22.616889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.361429+0000","last_change":"2023-10-16T10:05:49.368896+0000","last_active":"2023-10-16T10:05:50.361429+0000","last_peered":"2023-10-16T10:05:50.361429+0000","last_clean":"2023-10-16T10:05:50.361429+0000","last_became_active":"2023-10-16T10:05:49.367068+0000","last_became_peered":"2023-10-16T10:05:49.367068+0000","last_unstale":"2023-10-16T10:05:50.361429+0000","last_undegraded":"2023-10-16T10:05:50.361429+0000","last_fullsized":"2023-10-16T10:05:50.361429+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:15.458983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.356193+0000","last_change":"2023-10-16T10:05:48.349504+0000","last_active":"2023-10-16T10:05:49.356193+0000","last_peered":"2023-10-16T10:05:49.356193+0000","last_clean":"2023-10-16T10:05:49.356193+0000","last_became_active":"2023-10-16T10:05:48.347072+0000","last_became_peered":"2023-10-16T10:05:48.347072+0000","last_unstale":"2023-10-16T10:05:49.356193+0000","last_undegraded":"2023-10-16T10:05:49.356193+0000","last_fullsized":"2023-10-16T10:05:49.356193+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:55:23.570570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.359736+0000","last_change":"2023-10-16T10:05:49.362764+0000","last_active":"2023-10-16T10:05:50.359736+0000","last_peered":"2023-10-16T10:05:50.359736+0000","last_clean":"2023-10-16T10:05:50.359736+0000","last_became_active":"2023-10-16T10:05:49.361980+0000","last_became_peered":"2023-10-16T10:05:49.361980+0000","last_unstale":"2023-10-16T10:05:50.359736+0000","last_undegraded":"2023-10-16T10:05:50.359736+0000","last_fullsized":"2023-10-16T10:05:50.359736+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:43:44.531615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.369705+0000","last_change":"2023-10-16T10:05:48.337816+0000","last_active":"2023-10-16T10:05:49.369705+0000","last_peered":"2023-10-16T10:05:49.369705+0000","last_clean":"2023-10-16T10:05:49.369705+0000","last_became_active":"2023-10-16T10:05:48.337502+0000","last_became_peered":"2023-10-16T10:05:48.337502+0000","last_unstale":"2023-10-16T10:05:49.369705+0000","last_undegraded":"2023-10-16T10:05:49.369705+0000","last_fullsized":"2023-10-16T10:05:49.369705+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:51:22.616889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.697244+0000","last_change":"2023-10-16T10:05:49.354692+0000","last_active":"2023-10-16T10:05:50.697244+0000","last_peered":"2023-10-16T10:05:50.697244+0000","last_clean":"2023-10-16T10:05:50.697244+0000","last_became_active":"2023-10-16T10:05:49.353407+0000","last_became_peered":"2023-10-16T10:05:49.353407+0000","last_unstale":"2023-10-16T10:05:50.697244+0000","last_undegraded":"2023-10-16T10:05:50.697244+0000","last_fullsized":"2023-10-16T10:05:50.697244+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:20:46.312140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.369922+0000","last_change":"2023-10-16T10:05:48.340316+0000","last_active":"2023-10-16T10:05:49.369922+0000","last_peered":"2023-10-16T10:05:49.369922+0000","last_clean":"2023-10-16T10:05:49.369922+0000","last_became_active":"2023-10-16T10:05:48.339905+0000","last_became_peered":"2023-10-16T10:05:48.339905+0000","last_unstale":"2023-10-16T10:05:49.369922+0000","last_undegraded":"2023-10-16T10:05:49.369922+0000","last_fullsized":"2023-10-16T10:05:49.369922+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:11.984515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.898748+0000","last_change":"2023-10-16T10:05:49.349718+0000","last_active":"2023-10-16T10:05:50.898748+0000","last_peered":"2023-10-16T10:05:50.898748+0000","last_clean":"2023-10-16T10:05:50.898748+0000","last_became_active":"2023-10-16T10:05:49.349444+0000","last_became_peered":"2023-10-16T10:05:49.349444+0000","last_unstale":"2023-10-16T10:05:50.898748+0000","last_undegraded":"2023-10-16T10:05:50.898748+0000","last_fullsized":"2023-10-16T10:05:50.898748+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:58:33.795729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.357358+0000","last_change":"2023-10-16T10:05:48.344054+0000","last_active":"2023-10-16T10:05:49.357358+0000","last_peered":"2023-10-16T10:05:49.357358+0000","last_clean":"2023-10-16T10:05:49.357358+0000","last_became_active":"2023-10-16T10:05:48.343091+0000","last_became_peered":"2023-10-16T10:05:48.343091+0000","last_unstale":"2023-10-16T10:05:49.357358+0000","last_undegraded":"2023-10-16T10:05:49.357358+0000","last_fullsized":"2023-10-16T10:05:49.357358+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:10:50.723424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.357890+0000","last_change":"2023-10-16T10:05:49.339498+0000","last_active":"2023-10-16T10:05:50.357890+0000","last_peered":"2023-10-16T10:05:50.357890+0000","last_clean":"2023-10-16T10:05:50.357890+0000","last_became_active":"2023-10-16T10:05:49.338147+0000","last_became_peered":"2023-10-16T10:05:49.338147+0000","last_unstale":"2023-10-16T10:05:50.357890+0000","last_undegraded":"2023-10-16T10:05:50.357890+0000","last_fullsized":"2023-10-16T10:05:50.357890+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:41:09.556814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.355826+0000","last_change":"2023-10-16T10:05:48.346998+0000","last_active":"2023-10-16T10:05:49.355826+0000","last_peered":"2023-10-16T10:05:49.355826+0000","last_clean":"2023-10-16T10:05:49.355826+0000","last_became_active":"2023-10-16T10:05:48.345222+0000","last_became_peered":"2023-10-16T10:05:48.345222+0000","last_unstale":"2023-10-16T10:05:49.355826+0000","last_undegraded":"2023-10-16T10:05:49.355826+0000","last_fullsized":"2023-10-16T10:05:49.355826+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:09.964060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.696427+0000","last_change":"2023-10-16T10:05:49.360514+0000","last_active":"2023-10-16T10:05:50.696427+0000","last_peered":"2023-10-16T10:05:50.696427+0000","last_clean":"2023-10-16T10:05:50.696427+0000","last_became_active":"2023-10-16T10:05:49.359628+0000","last_became_peered":"2023-10-16T10:05:49.359628+0000","last_unstale":"2023-10-16T10:05:50.696427+0000","last_undegraded":"2023-10-16T10:05:50.696427+0000","last_fullsized":"2023-10-16T10:05:50.696427+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:52.671244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.338344+0000","last_change":"2023-10-16T10:05:48.353689+0000","last_active":"2023-10-16T10:05:49.338344+0000","last_peered":"2023-10-16T10:05:49.338344+0000","last_clean":"2023-10-16T10:05:49.338344+0000","last_became_active":"2023-10-16T10:05:48.353179+0000","last_became_peered":"2023-10-16T10:05:48.353179+0000","last_unstale":"2023-10-16T10:05:49.338344+0000","last_undegraded":"2023-10-16T10:05:49.338344+0000","last_fullsized":"2023-10-16T10:05:49.338344+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:48.020621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.898611+0000","last_change":"2023-10-16T10:05:49.356583+0000","last_active":"2023-10-16T10:05:50.898611+0000","last_peered":"2023-10-16T10:05:50.898611+0000","last_clean":"2023-10-16T10:05:50.898611+0000","last_became_active":"2023-10-16T10:05:49.356258+0000","last_became_peered":"2023-10-16T10:05:49.356258+0000","last_unstale":"2023-10-16T10:05:50.898611+0000","last_undegraded":"2023-10-16T10:05:50.898611+0000","last_fullsized":"2023-10-16T10:05:50.898611+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:47.366846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.340244+0000","last_change":"2023-10-16T10:05:48.348940+0000","last_active":"2023-10-16T10:05:49.340244+0000","last_peered":"2023-10-16T10:05:49.340244+0000","last_clean":"2023-10-16T10:05:49.340244+0000","last_became_active":"2023-10-16T10:05:48.348569+0000","last_became_peered":"2023-10-16T10:05:48.348569+0000","last_unstale":"2023-10-16T10:05:49.340244+0000","last_undegraded":"2023-10-16T10:05:49.340244+0000","last_fullsized":"2023-10-16T10:05:49.340244+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:05.681629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.900389+0000","last_change":"2023-10-16T10:05:49.346825+0000","last_active":"2023-10-16T10:05:50.900389+0000","last_peered":"2023-10-16T10:05:50.900389+0000","last_clean":"2023-10-16T10:05:50.900389+0000","last_became_active":"2023-10-16T10:05:49.346547+0000","last_became_peered":"2023-10-16T10:05:49.346547+0000","last_unstale":"2023-10-16T10:05:50.900389+0000","last_undegraded":"2023-10-16T10:05:50.900389+0000","last_fullsized":"2023-10-16T10:05:50.900389+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:45.381047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.351837+0000","last_change":"2023-10-16T10:05:48.356174+0000","last_active":"2023-10-16T10:05:49.351837+0000","last_peered":"2023-10-16T10:05:49.351837+0000","last_clean":"2023-10-16T10:05:49.351837+0000","last_became_active":"2023-10-16T10:05:48.355766+0000","last_became_peered":"2023-10-16T10:05:48.355766+0000","last_unstale":"2023-10-16T10:05:49.351837+0000","last_undegraded":"2023-10-16T10:05:49.351837+0000","last_fullsized":"2023-10-16T10:05:49.351837+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:39.658226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.898746+0000","last_change":"2023-10-16T10:05:49.369245+0000","last_active":"2023-10-16T10:05:50.898746+0000","last_peered":"2023-10-16T10:05:50.898746+0000","last_clean":"2023-10-16T10:05:50.898746+0000","last_became_active":"2023-10-16T10:05:49.368931+0000","last_became_peered":"2023-10-16T10:05:49.368931+0000","last_unstale":"2023-10-16T10:05:50.898746+0000","last_undegraded":"2023-10-16T10:05:50.898746+0000","last_fullsized":"2023-10-16T10:05:50.898746+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:36.705668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.351081+0000","last_change":"2023-10-16T10:05:48.356237+0000","last_active":"2023-10-16T10:05:49.351081+0000","last_peered":"2023-10-16T10:05:49.351081+0000","last_clean":"2023-10-16T10:05:49.351081+0000","last_became_active":"2023-10-16T10:05:48.355917+0000","last_became_peered":"2023-10-16T10:05:48.355917+0000","last_unstale":"2023-10-16T10:05:49.351081+0000","last_undegraded":"2023-10-16T10:05:49.351081+0000","last_fullsized":"2023-10-16T10:05:49.351081+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:11.984515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.690127+0000","last_change":"2023-10-16T10:05:49.373308+0000","last_active":"2023-10-16T10:05:50.690127+0000","last_peered":"2023-10-16T10:05:50.690127+0000","last_clean":"2023-10-16T10:05:50.690127+0000","last_became_active":"2023-10-16T10:05:49.372947+0000","last_became_peered":"2023-10-16T10:05:49.372947+0000","last_unstale":"2023-10-16T10:05:50.690127+0000","last_undegraded":"2023-10-16T10:05:50.690127+0000","last_fullsized":"2023-10-16T10:05:50.690127+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:12:35.449167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"107'3","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.375015+0000","last_change":"2023-10-16T10:05:48.349374+0000","last_active":"2023-10-16T10:05:49.375015+0000","last_peered":"2023-10-16T10:05:49.375015+0000","last_clean":"2023-10-16T10:05:49.375015+0000","last_became_active":"2023-10-16T10:05:48.348869+0000","last_became_peered":"2023-10-16T10:05:48.348869+0000","last_unstale":"2023-10-16T10:05:49.375015+0000","last_undegraded":"2023-10-16T10:05:49.375015+0000","last_fullsized":"2023-10-16T10:05:49.375015+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:39:19.257795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.358788+0000","last_change":"2023-10-16T10:05:49.346890+0000","last_active":"2023-10-16T10:05:50.358788+0000","last_peered":"2023-10-16T10:05:50.358788+0000","last_clean":"2023-10-16T10:05:50.358788+0000","last_became_active":"2023-10-16T10:05:49.346229+0000","last_became_peered":"2023-10-16T10:05:49.346229+0000","last_unstale":"2023-10-16T10:05:50.358788+0000","last_undegraded":"2023-10-16T10:05:50.358788+0000","last_fullsized":"2023-10-16T10:05:50.358788+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:28.603799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"107'6","reported_seq":18,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.375017+0000","last_change":"2023-10-16T10:05:48.354401+0000","last_active":"2023-10-16T10:05:49.375017+0000","last_peered":"2023-10-16T10:05:49.375017+0000","last_clean":"2023-10-16T10:05:49.375017+0000","last_became_active":"2023-10-16T10:05:48.353686+0000","last_became_peered":"2023-10-16T10:05:48.353686+0000","last_unstale":"2023-10-16T10:05:49.375017+0000","last_undegraded":"2023-10-16T10:05:49.375017+0000","last_fullsized":"2023-10-16T10:05:49.375017+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:25.385147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.689786+0000","last_change":"2023-10-16T10:05:49.362658+0000","last_active":"2023-10-16T10:05:50.689786+0000","last_peered":"2023-10-16T10:05:50.689786+0000","last_clean":"2023-10-16T10:05:50.689786+0000","last_became_active":"2023-10-16T10:05:49.357650+0000","last_became_peered":"2023-10-16T10:05:49.357650+0000","last_unstale":"2023-10-16T10:05:50.689786+0000","last_undegraded":"2023-10-16T10:05:50.689786+0000","last_fullsized":"2023-10-16T10:05:50.689786+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:21.782751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.357344+0000","last_change":"2023-10-16T10:05:48.345760+0000","last_active":"2023-10-16T10:05:49.357344+0000","last_peered":"2023-10-16T10:05:49.357344+0000","last_clean":"2023-10-16T10:05:49.357344+0000","last_became_active":"2023-10-16T10:05:48.345487+0000","last_became_peered":"2023-10-16T10:05:48.345487+0000","last_unstale":"2023-10-16T10:05:49.357344+0000","last_undegraded":"2023-10-16T10:05:49.357344+0000","last_fullsized":"2023-10-16T10:05:49.357344+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:47:15.203419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.696721+0000","last_change":"2023-10-16T10:05:49.354706+0000","last_active":"2023-10-16T10:05:50.696721+0000","last_peered":"2023-10-16T10:05:50.696721+0000","last_clean":"2023-10-16T10:05:50.696721+0000","last_became_active":"2023-10-16T10:05:49.354158+0000","last_became_peered":"2023-10-16T10:05:49.354158+0000","last_unstale":"2023-10-16T10:05:50.696721+0000","last_undegraded":"2023-10-16T10:05:50.696721+0000","last_fullsized":"2023-10-16T10:05:50.696721+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:30:01.349607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.328714+0000","last_change":"2023-10-16T10:05:48.349903+0000","last_active":"2023-10-16T10:05:49.328714+0000","last_peered":"2023-10-16T10:05:49.328714+0000","last_clean":"2023-10-16T10:05:49.328714+0000","last_became_active":"2023-10-16T10:05:48.349590+0000","last_became_peered":"2023-10-16T10:05:48.349590+0000","last_unstale":"2023-10-16T10:05:49.328714+0000","last_undegraded":"2023-10-16T10:05:49.328714+0000","last_fullsized":"2023-10-16T10:05:49.328714+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:40.208983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.897606+0000","last_change":"2023-10-16T10:05:49.345088+0000","last_active":"2023-10-16T10:05:50.897606+0000","last_peered":"2023-10-16T10:05:50.897606+0000","last_clean":"2023-10-16T10:05:50.897606+0000","last_became_active":"2023-10-16T10:05:49.344395+0000","last_became_peered":"2023-10-16T10:05:49.344395+0000","last_unstale":"2023-10-16T10:05:50.897606+0000","last_undegraded":"2023-10-16T10:05:50.897606+0000","last_fullsized":"2023-10-16T10:05:50.897606+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:05:15.203028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.351850+0000","last_change":"2023-10-16T10:05:48.354759+0000","last_active":"2023-10-16T10:05:49.351850+0000","last_peered":"2023-10-16T10:05:49.351850+0000","last_clean":"2023-10-16T10:05:49.351850+0000","last_became_active":"2023-10-16T10:05:48.354308+0000","last_became_peered":"2023-10-16T10:05:48.354308+0000","last_unstale":"2023-10-16T10:05:49.351850+0000","last_undegraded":"2023-10-16T10:05:49.351850+0000","last_fullsized":"2023-10-16T10:05:49.351850+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:59.348085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.897541+0000","last_change":"2023-10-16T10:05:49.367238+0000","last_active":"2023-10-16T10:05:50.897541+0000","last_peered":"2023-10-16T10:05:50.897541+0000","last_clean":"2023-10-16T10:05:50.897541+0000","last_became_active":"2023-10-16T10:05:49.366929+0000","last_became_peered":"2023-10-16T10:05:49.366929+0000","last_unstale":"2023-10-16T10:05:50.897541+0000","last_undegraded":"2023-10-16T10:05:50.897541+0000","last_fullsized":"2023-10-16T10:05:50.897541+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:48.194506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"107'8","reported_seq":20,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.371133+0000","last_change":"2023-10-16T10:05:48.349165+0000","last_active":"2023-10-16T10:05:49.371133+0000","last_peered":"2023-10-16T10:05:49.371133+0000","last_clean":"2023-10-16T10:05:49.371133+0000","last_became_active":"2023-10-16T10:05:48.348882+0000","last_became_peered":"2023-10-16T10:05:48.348882+0000","last_unstale":"2023-10-16T10:05:49.371133+0000","last_undegraded":"2023-10-16T10:05:49.371133+0000","last_fullsized":"2023-10-16T10:05:49.371133+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:22.991998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.697262+0000","last_change":"2023-10-16T10:05:49.365826+0000","last_active":"2023-10-16T10:05:50.697262+0000","last_peered":"2023-10-16T10:05:50.697262+0000","last_clean":"2023-10-16T10:05:50.697262+0000","last_became_active":"2023-10-16T10:05:49.365053+0000","last_became_peered":"2023-10-16T10:05:49.365053+0000","last_unstale":"2023-10-16T10:05:50.697262+0000","last_undegraded":"2023-10-16T10:05:50.697262+0000","last_fullsized":"2023-10-16T10:05:50.697262+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:43:19.965258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.366296+0000","last_change":"2023-10-16T10:05:48.356751+0000","last_active":"2023-10-16T10:05:49.366296+0000","last_peered":"2023-10-16T10:05:49.366296+0000","last_clean":"2023-10-16T10:05:49.366296+0000","last_became_active":"2023-10-16T10:05:48.356436+0000","last_became_peered":"2023-10-16T10:05:48.356436+0000","last_unstale":"2023-10-16T10:05:49.366296+0000","last_undegraded":"2023-10-16T10:05:49.366296+0000","last_fullsized":"2023-10-16T10:05:49.366296+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:46:51.165351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.361548+0000","last_change":"2023-10-16T10:05:49.369757+0000","last_active":"2023-10-16T10:05:50.361548+0000","last_peered":"2023-10-16T10:05:50.361548+0000","last_clean":"2023-10-16T10:05:50.361548+0000","last_became_active":"2023-10-16T10:05:49.368859+0000","last_became_peered":"2023-10-16T10:05:49.368859+0000","last_unstale":"2023-10-16T10:05:50.361548+0000","last_undegraded":"2023-10-16T10:05:50.361548+0000","last_fullsized":"2023-10-16T10:05:50.361548+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:30.616685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.364588+0000","last_change":"2023-10-16T10:05:48.349507+0000","last_active":"2023-10-16T10:05:49.364588+0000","last_peered":"2023-10-16T10:05:49.364588+0000","last_clean":"2023-10-16T10:05:49.364588+0000","last_became_active":"2023-10-16T10:05:48.349214+0000","last_became_peered":"2023-10-16T10:05:48.349214+0000","last_unstale":"2023-10-16T10:05:49.364588+0000","last_undegraded":"2023-10-16T10:05:49.364588+0000","last_fullsized":"2023-10-16T10:05:49.364588+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:39.658226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.359049+0000","last_change":"2023-10-16T10:05:49.344432+0000","last_active":"2023-10-16T10:05:50.359049+0000","last_peered":"2023-10-16T10:05:50.359049+0000","last_clean":"2023-10-16T10:05:50.359049+0000","last_became_active":"2023-10-16T10:05:49.343192+0000","last_became_peered":"2023-10-16T10:05:49.343192+0000","last_unstale":"2023-10-16T10:05:50.359049+0000","last_undegraded":"2023-10-16T10:05:50.359049+0000","last_fullsized":"2023-10-16T10:05:50.359049+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:52.562154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.338675+0000","last_change":"2023-10-16T10:05:48.355145+0000","last_active":"2023-10-16T10:05:49.338675+0000","last_peered":"2023-10-16T10:05:49.338675+0000","last_clean":"2023-10-16T10:05:49.338675+0000","last_became_active":"2023-10-16T10:05:48.354837+0000","last_became_peered":"2023-10-16T10:05:48.354837+0000","last_unstale":"2023-10-16T10:05:49.338675+0000","last_undegraded":"2023-10-16T10:05:49.338675+0000","last_fullsized":"2023-10-16T10:05:49.338675+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:35.265342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.353526+0000","last_change":"2023-10-16T10:05:49.360866+0000","last_active":"2023-10-16T10:05:50.353526+0000","last_peered":"2023-10-16T10:05:50.353526+0000","last_clean":"2023-10-16T10:05:50.353526+0000","last_became_active":"2023-10-16T10:05:49.360153+0000","last_became_peered":"2023-10-16T10:05:49.360153+0000","last_unstale":"2023-10-16T10:05:50.353526+0000","last_undegraded":"2023-10-16T10:05:50.353526+0000","last_fullsized":"2023-10-16T10:05:50.353526+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:12:35.449167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.349025+0000","last_change":"2023-10-16T10:05:48.343597+0000","last_active":"2023-10-16T10:05:49.349025+0000","last_peered":"2023-10-16T10:05:49.349025+0000","last_clean":"2023-10-16T10:05:49.349025+0000","last_became_active":"2023-10-16T10:05:48.343241+0000","last_became_peered":"2023-10-16T10:05:48.343241+0000","last_unstale":"2023-10-16T10:05:49.349025+0000","last_undegraded":"2023-10-16T10:05:49.349025+0000","last_fullsized":"2023-10-16T10:05:49.349025+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:30:46.063285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.897951+0000","last_change":"2023-10-16T10:05:49.360727+0000","last_active":"2023-10-16T10:05:50.897951+0000","last_peered":"2023-10-16T10:05:50.897951+0000","last_clean":"2023-10-16T10:05:50.897951+0000","last_became_active":"2023-10-16T10:05:49.360458+0000","last_became_peered":"2023-10-16T10:05:49.360458+0000","last_unstale":"2023-10-16T10:05:50.897951+0000","last_undegraded":"2023-10-16T10:05:50.897951+0000","last_fullsized":"2023-10-16T10:05:50.897951+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:20:09.142217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.329315+0000","last_change":"2023-10-16T10:05:48.336655+0000","last_active":"2023-10-16T10:05:49.329315+0000","last_peered":"2023-10-16T10:05:49.329315+0000","last_clean":"2023-10-16T10:05:49.329315+0000","last_became_active":"2023-10-16T10:05:48.332928+0000","last_became_peered":"2023-10-16T10:05:48.332928+0000","last_unstale":"2023-10-16T10:05:49.329315+0000","last_undegraded":"2023-10-16T10:05:49.329315+0000","last_fullsized":"2023-10-16T10:05:49.329315+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:35.265342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.353909+0000","last_change":"2023-10-16T10:05:49.355703+0000","last_active":"2023-10-16T10:05:50.353909+0000","last_peered":"2023-10-16T10:05:50.353909+0000","last_clean":"2023-10-16T10:05:50.353909+0000","last_became_active":"2023-10-16T10:05:49.355443+0000","last_became_peered":"2023-10-16T10:05:49.355443+0000","last_unstale":"2023-10-16T10:05:50.353909+0000","last_undegraded":"2023-10-16T10:05:50.353909+0000","last_fullsized":"2023-10-16T10:05:50.353909+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:34:31.006127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.328588+0000","last_change":"2023-10-16T10:05:48.356672+0000","last_active":"2023-10-16T10:05:49.328588+0000","last_peered":"2023-10-16T10:05:49.328588+0000","last_clean":"2023-10-16T10:05:49.328588+0000","last_became_active":"2023-10-16T10:05:48.356367+0000","last_became_peered":"2023-10-16T10:05:48.356367+0000","last_unstale":"2023-10-16T10:05:49.328588+0000","last_undegraded":"2023-10-16T10:05:49.328588+0000","last_fullsized":"2023-10-16T10:05:49.328588+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:48.020621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.897763+0000","last_change":"2023-10-16T10:05:49.365910+0000","last_active":"2023-10-16T10:05:50.897763+0000","last_peered":"2023-10-16T10:05:50.897763+0000","last_clean":"2023-10-16T10:05:50.897763+0000","last_became_active":"2023-10-16T10:05:49.361383+0000","last_became_peered":"2023-10-16T10:05:49.361383+0000","last_unstale":"2023-10-16T10:05:50.897763+0000","last_undegraded":"2023-10-16T10:05:50.897763+0000","last_fullsized":"2023-10-16T10:05:50.897763+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:48.006728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.328376+0000","last_change":"2023-10-16T10:05:48.341885+0000","last_active":"2023-10-16T10:05:49.328376+0000","last_peered":"2023-10-16T10:05:49.328376+0000","last_clean":"2023-10-16T10:05:49.328376+0000","last_became_active":"2023-10-16T10:05:48.341590+0000","last_became_peered":"2023-10-16T10:05:48.341590+0000","last_unstale":"2023-10-16T10:05:49.328376+0000","last_undegraded":"2023-10-16T10:05:49.328376+0000","last_fullsized":"2023-10-16T10:05:49.328376+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:16:59.947077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.359337+0000","last_change":"2023-10-16T10:05:49.362847+0000","last_active":"2023-10-16T10:05:50.359337+0000","last_peered":"2023-10-16T10:05:50.359337+0000","last_clean":"2023-10-16T10:05:50.359337+0000","last_became_active":"2023-10-16T10:05:49.362516+0000","last_became_peered":"2023-10-16T10:05:49.362516+0000","last_unstale":"2023-10-16T10:05:50.359337+0000","last_undegraded":"2023-10-16T10:05:50.359337+0000","last_fullsized":"2023-10-16T10:05:50.359337+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:02.721303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.367239+0000","last_change":"2023-10-16T10:05:48.351887+0000","last_active":"2023-10-16T10:05:49.367239+0000","last_peered":"2023-10-16T10:05:49.367239+0000","last_clean":"2023-10-16T10:05:49.367239+0000","last_became_active":"2023-10-16T10:05:48.351613+0000","last_became_peered":"2023-10-16T10:05:48.351613+0000","last_unstale":"2023-10-16T10:05:49.367239+0000","last_undegraded":"2023-10-16T10:05:49.367239+0000","last_fullsized":"2023-10-16T10:05:49.367239+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:14.439273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.697481+0000","last_change":"2023-10-16T10:05:49.367441+0000","last_active":"2023-10-16T10:05:50.697481+0000","last_peered":"2023-10-16T10:05:50.697481+0000","last_clean":"2023-10-16T10:05:50.697481+0000","last_became_active":"2023-10-16T10:05:49.366237+0000","last_became_peered":"2023-10-16T10:05:49.366237+0000","last_unstale":"2023-10-16T10:05:50.697481+0000","last_undegraded":"2023-10-16T10:05:50.697481+0000","last_fullsized":"2023-10-16T10:05:50.697481+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:16:11.590897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.343256+0000","last_change":"2023-10-16T10:05:48.346236+0000","last_active":"2023-10-16T10:05:49.343256+0000","last_peered":"2023-10-16T10:05:49.343256+0000","last_clean":"2023-10-16T10:05:49.343256+0000","last_became_active":"2023-10-16T10:05:48.345957+0000","last_became_peered":"2023-10-16T10:05:48.345957+0000","last_unstale":"2023-10-16T10:05:49.343256+0000","last_undegraded":"2023-10-16T10:05:49.343256+0000","last_fullsized":"2023-10-16T10:05:49.343256+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:13.642849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.359993+0000","last_change":"2023-10-16T10:05:49.371515+0000","last_active":"2023-10-16T10:05:50.359993+0000","last_peered":"2023-10-16T10:05:50.359993+0000","last_clean":"2023-10-16T10:05:50.359993+0000","last_became_active":"2023-10-16T10:05:49.371098+0000","last_became_peered":"2023-10-16T10:05:49.371098+0000","last_unstale":"2023-10-16T10:05:50.359993+0000","last_undegraded":"2023-10-16T10:05:50.359993+0000","last_fullsized":"2023-10-16T10:05:50.359993+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:36:24.167226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.348761+0000","last_change":"2023-10-16T10:05:48.352753+0000","last_active":"2023-10-16T10:05:49.348761+0000","last_peered":"2023-10-16T10:05:49.348761+0000","last_clean":"2023-10-16T10:05:49.348761+0000","last_became_active":"2023-10-16T10:05:48.352350+0000","last_became_peered":"2023-10-16T10:05:48.352350+0000","last_unstale":"2023-10-16T10:05:49.348761+0000","last_undegraded":"2023-10-16T10:05:49.348761+0000","last_fullsized":"2023-10-16T10:05:49.348761+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:48.020621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.359185+0000","last_change":"2023-10-16T10:05:49.346070+0000","last_active":"2023-10-16T10:05:50.359185+0000","last_peered":"2023-10-16T10:05:50.359185+0000","last_clean":"2023-10-16T10:05:50.359185+0000","last_became_active":"2023-10-16T10:05:49.345616+0000","last_became_peered":"2023-10-16T10:05:49.345616+0000","last_unstale":"2023-10-16T10:05:50.359185+0000","last_undegraded":"2023-10-16T10:05:50.359185+0000","last_fullsized":"2023-10-16T10:05:50.359185+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:56:09.445211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.357094+0000","last_change":"2023-10-16T10:05:48.350655+0000","last_active":"2023-10-16T10:05:49.357094+0000","last_peered":"2023-10-16T10:05:49.357094+0000","last_clean":"2023-10-16T10:05:49.357094+0000","last_became_active":"2023-10-16T10:05:48.350221+0000","last_became_peered":"2023-10-16T10:05:48.350221+0000","last_unstale":"2023-10-16T10:05:49.357094+0000","last_undegraded":"2023-10-16T10:05:49.357094+0000","last_fullsized":"2023-10-16T10:05:49.357094+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:08:04.016268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.897876+0000","last_change":"2023-10-16T10:05:49.354934+0000","last_active":"2023-10-16T10:05:50.897876+0000","last_peered":"2023-10-16T10:05:50.897876+0000","last_clean":"2023-10-16T10:05:50.897876+0000","last_became_active":"2023-10-16T10:05:49.354256+0000","last_became_peered":"2023-10-16T10:05:49.354256+0000","last_unstale":"2023-10-16T10:05:50.897876+0000","last_undegraded":"2023-10-16T10:05:50.897876+0000","last_fullsized":"2023-10-16T10:05:50.897876+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:56:33.268643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.339088+0000","last_change":"2023-10-16T10:05:48.349016+0000","last_active":"2023-10-16T10:05:49.339088+0000","last_peered":"2023-10-16T10:05:49.339088+0000","last_clean":"2023-10-16T10:05:49.339088+0000","last_became_active":"2023-10-16T10:05:48.348724+0000","last_became_peered":"2023-10-16T10:05:48.348724+0000","last_unstale":"2023-10-16T10:05:49.339088+0000","last_undegraded":"2023-10-16T10:05:49.339088+0000","last_fullsized":"2023-10-16T10:05:49.339088+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:46:51.165351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.690694+0000","last_change":"2023-10-16T10:05:49.345581+0000","last_active":"2023-10-16T10:05:50.690694+0000","last_peered":"2023-10-16T10:05:50.690694+0000","last_clean":"2023-10-16T10:05:50.690694+0000","last_became_active":"2023-10-16T10:05:49.345319+0000","last_became_peered":"2023-10-16T10:05:49.345319+0000","last_unstale":"2023-10-16T10:05:50.690694+0000","last_undegraded":"2023-10-16T10:05:50.690694+0000","last_fullsized":"2023-10-16T10:05:50.690694+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:08:05.035979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.333033+0000","last_change":"2023-10-16T10:05:48.346770+0000","last_active":"2023-10-16T10:05:49.333033+0000","last_peered":"2023-10-16T10:05:49.333033+0000","last_clean":"2023-10-16T10:05:49.333033+0000","last_became_active":"2023-10-16T10:05:48.343689+0000","last_became_peered":"2023-10-16T10:05:48.343689+0000","last_unstale":"2023-10-16T10:05:49.333033+0000","last_undegraded":"2023-10-16T10:05:49.333033+0000","last_fullsized":"2023-10-16T10:05:49.333033+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:51:59.064234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.696529+0000","last_change":"2023-10-16T10:05:49.364526+0000","last_active":"2023-10-16T10:05:50.696529+0000","last_peered":"2023-10-16T10:05:50.696529+0000","last_clean":"2023-10-16T10:05:50.696529+0000","last_became_active":"2023-10-16T10:05:49.364242+0000","last_became_peered":"2023-10-16T10:05:49.364242+0000","last_unstale":"2023-10-16T10:05:50.696529+0000","last_undegraded":"2023-10-16T10:05:50.696529+0000","last_fullsized":"2023-10-16T10:05:50.696529+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:59.428532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.362334+0000","last_change":"2023-10-16T10:05:48.354276+0000","last_active":"2023-10-16T10:05:49.362334+0000","last_peered":"2023-10-16T10:05:49.362334+0000","last_clean":"2023-10-16T10:05:49.362334+0000","last_became_active":"2023-10-16T10:05:48.353951+0000","last_became_peered":"2023-10-16T10:05:48.353951+0000","last_unstale":"2023-10-16T10:05:49.362334+0000","last_undegraded":"2023-10-16T10:05:49.362334+0000","last_fullsized":"2023-10-16T10:05:49.362334+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:19.358081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.360642+0000","last_change":"2023-10-16T10:05:49.373411+0000","last_active":"2023-10-16T10:05:50.360642+0000","last_peered":"2023-10-16T10:05:50.360642+0000","last_clean":"2023-10-16T10:05:50.360642+0000","last_became_active":"2023-10-16T10:05:49.371728+0000","last_became_peered":"2023-10-16T10:05:49.371728+0000","last_unstale":"2023-10-16T10:05:50.360642+0000","last_undegraded":"2023-10-16T10:05:50.360642+0000","last_fullsized":"2023-10-16T10:05:50.360642+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:47:01.912619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.336572+0000","last_change":"2023-10-16T10:05:48.352039+0000","last_active":"2023-10-16T10:05:49.336572+0000","last_peered":"2023-10-16T10:05:49.336572+0000","last_clean":"2023-10-16T10:05:49.336572+0000","last_became_active":"2023-10-16T10:05:48.351721+0000","last_became_peered":"2023-10-16T10:05:48.351721+0000","last_unstale":"2023-10-16T10:05:49.336572+0000","last_undegraded":"2023-10-16T10:05:49.336572+0000","last_fullsized":"2023-10-16T10:05:49.336572+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:28:22.224872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.899354+0000","last_change":"2023-10-16T10:05:49.366056+0000","last_active":"2023-10-16T10:05:50.899354+0000","last_peered":"2023-10-16T10:05:50.899354+0000","last_clean":"2023-10-16T10:05:50.899354+0000","last_became_active":"2023-10-16T10:05:49.365789+0000","last_became_peered":"2023-10-16T10:05:49.365789+0000","last_unstale":"2023-10-16T10:05:50.899354+0000","last_undegraded":"2023-10-16T10:05:50.899354+0000","last_fullsized":"2023-10-16T10:05:50.899354+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:10:07.060924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.368690+0000","last_change":"2023-10-16T10:05:48.351584+0000","last_active":"2023-10-16T10:05:49.368690+0000","last_peered":"2023-10-16T10:05:49.368690+0000","last_clean":"2023-10-16T10:05:49.368690+0000","last_became_active":"2023-10-16T10:05:48.351313+0000","last_became_peered":"2023-10-16T10:05:48.351313+0000","last_unstale":"2023-10-16T10:05:49.368690+0000","last_undegraded":"2023-10-16T10:05:49.368690+0000","last_fullsized":"2023-10-16T10:05:49.368690+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:40.208983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"108'2148","reported_seq":2450,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:52.471408+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T10:05:52.471408+0000","last_peered":"2023-10-16T10:05:52.471408+0000","last_clean":"2023-10-16T10:05:52.471408+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T10:05:52.471408+0000","last_undegraded":"2023-10-16T10:05:52.471408+0000","last_fullsized":"2023-10-16T10:05:52.471408+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2148,"log_dups_size":0,"ondisk_log_size":2148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":640,"num_read_kb":1170,"num_write":261,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":640,"num_read_kb":1170,"num_write":261,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2148,"ondisk_log_size":2148,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575050,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28896,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342944,"statfs":{"total":96636764160,"available":96607174656,"internally_reserved":0,"allocated":1925120,"data_stored":1222495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.125,"15min":1.1619999999999999},"min":{"1min":0.67200000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5429999999999999,"5min":1.853,"15min":1.925},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.008,"5min":1.1319999999999999,"15min":1.1899999999999999},"min":{"1min":0.59199999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.5169999999999999,"5min":1.75,"15min":2.016},"last":1.343}]},{"osd":1,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1899999999999999,"15min":1.23},"min":{"1min":0.71799999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6259999999999999,"5min":1.923,"15min":1.923},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.171,"15min":1.214},"min":{"1min":0.749,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.464,"5min":1.595,"15min":1.7410000000000001},"last":0.996}]},{"osd":2,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1799999999999999,"15min":1.244},"min":{"1min":0.68899999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7410000000000001,"5min":1.837,"15min":1.8740000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.2490000000000001,"15min":1.2929999999999999},"min":{"1min":0.84799999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4450000000000001,"5min":1.7430000000000001,"15min":1.8620000000000001},"last":1.3}]},{"osd":3,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.2989999999999999,"15min":1.339},"min":{"1min":0.89400000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.661,"5min":1.7929999999999999,"15min":1.996},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.3320000000000001,"15min":1.381},"min":{"1min":0.69099999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.8540000000000001,"5min":1.8759999999999999,"15min":2.004},"last":1.5289999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3720000000000001,"15min":1.3700000000000001},"min":{"1min":0.99199999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.994,"5min":2.028,"15min":2.0310000000000001},"last":1.151},{"interface":"front","average":{"1min":1.359,"5min":1.3149999999999999,"15min":1.319},"min":{"1min":0.96299999999999997,"5min":0.502,"15min":0.502},"max":{"1min":1.9039999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.0589999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.472,"15min":1.516},"min":{"1min":0.91100000000000003,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.081,"5min":2.1320000000000001,"15min":2.165},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4299999999999999,"15min":1.4339999999999999},"min":{"1min":1.097,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.9630000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.2509999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.417,"15min":1.4059999999999999},"min":{"1min":1.175,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.109},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.601,"15min":1.619},"min":{"1min":1.3300000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":1.6479999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575050,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36648,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335192,"statfs":{"total":96636764160,"available":96599236608,"internally_reserved":0,"allocated":1339392,"data_stored":632751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:05:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.127,"15min":1.145},"min":{"1min":0.71799999999999997,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6240000000000001,"5min":1.667,"15min":1.7689999999999999},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.2290000000000001,"15min":1.246},"min":{"1min":0.92300000000000004,"5min":0.65900000000000003,"15min":0.65300000000000002},"max":{"1min":1.7010000000000001,"5min":1.847,"15min":1.8720000000000001},"last":1.466}]},{"osd":1,"last update":"Mon Oct 16 10:05:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1160000000000001,"15min":1.175},"min":{"1min":0.83699999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5189999999999999,"5min":1.6180000000000001,"15min":1.6970000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.276,"15min":1.3},"min":{"1min":0.69099999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.738,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.7070000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:05:20 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.179,"15min":1.2090000000000001},"min":{"1min":0.82599999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.851,"5min":1.851,"15min":2.036},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.2949999999999999,"15min":1.3100000000000001},"min":{"1min":0.90300000000000002,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.5269999999999999,"5min":1.925,"15min":1.925},"last":1.845}]},{"osd":3,"last update":"Mon Oct 16 10:05:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1990000000000001,"15min":1.2250000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.6990000000000001,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.343},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.1850000000000001,"15min":1.2330000000000001},"min":{"1min":0.84999999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.865},"last":1.103}]},{"osd":4,"last update":"Mon Oct 16 10:05:20 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.244,"15min":1.2709999999999999},"min":{"1min":1.0289999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.7729999999999999,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.4470000000000001,"15min":1.4339999999999999},"min":{"1min":0.91900000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.865,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.8069999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:05:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.2869999999999999,"15min":1.3160000000000001},"min":{"1min":0.91800000000000004,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.0790000000000002,"5min":2.3090000000000002,"15min":2.5649999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.4970000000000001,"15min":1.51},"min":{"1min":0.97799999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.867,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.996}]},{"osd":7,"last update":"Mon Oct 16 10:05:20 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4630000000000001,"15min":1.476},"min":{"1min":1.2030000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.9510000000000001,"5min":3.5739999999999998,"15min":3.5739999999999998},"last":1.9390000000000001},{"interface":"front","average":{"1min":1.585,"5min":1.446,"15min":1.4890000000000001},"min":{"1min":0.96099999999999997,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.4249999999999998},"last":1.7669999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575050,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36652,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335188,"statfs":{"total":96636764160,"available":96599232512,"internally_reserved":0,"allocated":1343488,"data_stored":633691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.1499999999999999,"15min":1.224},"min":{"1min":0.59299999999999997,"5min":0.496,"15min":0.496},"max":{"1min":2.3959999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.6779999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.2749999999999999,"15min":1.3080000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.2390000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.111,"15min":1.141},"min":{"1min":0.60499999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.7310000000000001,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.149,"5min":1.1180000000000001,"15min":1.151},"min":{"1min":0.73499999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6970000000000001},"last":1.5149999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.153,"15min":1.167},"min":{"1min":0.83899999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.5609999999999999,"5min":1.7470000000000001,"15min":1.77},"last":1.139},{"interface":"front","average":{"1min":1.244,"5min":1.2230000000000001,"15min":1.242},"min":{"1min":0.77300000000000002,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.6739999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.3100000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.26,"15min":1.2789999999999999},"min":{"1min":0.91600000000000004,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.496,"5min":1.9530000000000001,"15min":1.97},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.323,"15min":1.355},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.5860000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2490000000000001,"15min":1.276},"min":{"1min":0.86599999999999999,"5min":0.77100000000000002,"15min":0.73499999999999999},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":2.355},"last":1.7769999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.496,"15min":1.5149999999999999},"min":{"1min":1.028,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.861,"5min":2.0819999999999999,"15min":2.4079999999999999},"last":1.349}]},{"osd":6,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3460000000000001,"15min":1.365},"min":{"1min":0.85499999999999998,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.786,"5min":1.847,"15min":1.847},"last":1.206},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.3759999999999999,"15min":1.3919999999999999},"min":{"1min":1.0389999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":2.2549999999999999},"last":1.869}]},{"osd":7,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.466,"15min":1.4910000000000001},"min":{"1min":0.93300000000000005,"5min":0.88600000000000001,"15min":0.872},"max":{"1min":1.996,"5min":2.1989999999999998,"15min":2.4660000000000002},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5620000000000001,"15min":1.5760000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":0.99199999999999999},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.407},"last":1.4410000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575051,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46364,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325476,"statfs":{"total":96636764160,"available":96589287424,"internally_reserved":0,"allocated":1327104,"data_stored":632057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:05:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0900000000000001,"15min":1.1479999999999999},"min":{"1min":0.81299999999999994,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.6399999999999999,"5min":1.76,"15min":1.827},"last":1.625},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3620000000000001,"15min":1.256},"min":{"1min":0.94499999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8939999999999999,"5min":11.974,"15min":11.974},"last":1.367}]},{"osd":1,"last update":"Mon Oct 16 10:05:22 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.3700000000000001,"15min":1.292},"min":{"1min":0.98999999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.8939999999999999,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":1.109},{"interface":"front","average":{"1min":1.135,"5min":1.194,"15min":1.238},"min":{"1min":0.73399999999999999,"5min":0.63600000000000001,"15min":0.624},"max":{"1min":1.603,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.0620000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:05:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.24,"15min":1.2430000000000001},"min":{"1min":0.76900000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.4770000000000001,"5min":1.756,"15min":1.843},"last":1.897},{"interface":"front","average":{"1min":1.431,"5min":1.3879999999999999,"15min":1.385},"min":{"1min":1.0549999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.1219999999999999,"5min":5.0549999999999997,"15min":5.0549999999999997},"last":1.266}]},{"osd":3,"last update":"Mon Oct 16 10:05:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3220000000000001,"15min":1.3},"min":{"1min":0.98699999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.744,"5min":6.8079999999999998,"15min":6.8079999999999998},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.369,"15min":1.379},"min":{"1min":1.0289999999999999,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.7350000000000001,"5min":1.885,"15min":1.923},"last":1.744}]},{"osd":5,"last update":"Mon Oct 16 10:05:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.524,"15min":1.4219999999999999},"min":{"1min":1.036,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.1539999999999999,"5min":8.702,"15min":8.702},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3879999999999999,"15min":1.3720000000000001},"min":{"1min":0.92600000000000005,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.032,"5min":3.0430000000000001,"15min":3.0430000000000001},"last":1.236}]},{"osd":6,"last update":"Mon Oct 16 10:05:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4930000000000001,"15min":1.4119999999999999},"min":{"1min":0.84699999999999998,"5min":0.80700000000000005,"15min":0.79000000000000004},"max":{"1min":1.8979999999999999,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":1.8480000000000001},{"interface":"front","average":{"1min":1.714,"5min":1.6279999999999999,"15min":1.554},"min":{"1min":1.3200000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.2549999999999999,"5min":6.4089999999999998,"15min":6.4089999999999998},"last":1.569}]},{"osd":7,"last update":"Mon Oct 16 10:05:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5880000000000001,"5min":1.5289999999999999,"15min":1.4930000000000001},"min":{"1min":1.1439999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.3410000000000002,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.302},{"interface":"front","average":{"1min":1.647,"5min":1.603,"15min":1.569},"min":{"1min":1.282,"5min":1.1220000000000001,"15min":0.96799999999999997},"max":{"1min":2.0030000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":1.8}]}]},{"osd":3,"up_from":13,"seq":55834575050,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36648,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335192,"statfs":{"total":96636764160,"available":96599236608,"internally_reserved":0,"allocated":1339392,"data_stored":633669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:05:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.82499999999999996,"15min":0.85799999999999998},"min":{"1min":0.39900000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.318},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.89200000000000002,"15min":0.995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.29999999999999999},"max":{"1min":1.3540000000000001,"5min":1.488,"15min":1.708},"last":0.49099999999999999}]},{"osd":1,"last update":"Mon Oct 16 10:05:32 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.86899999999999999,"15min":0.93999999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.0270000000000001,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.98399999999999999,"15min":1.054},"min":{"1min":0.441,"5min":0.42099999999999999,"15min":0.39600000000000002},"max":{"1min":3.363,"5min":3.363,"15min":3.363},"last":0.36699999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:05:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.98099999999999998,"15min":1.054},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.069,"5min":1.093,"15min":1.165},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":4.1230000000000002,"5min":4.1230000000000002,"15min":4.1230000000000002},"last":0.70799999999999996}]},{"osd":4,"last update":"Mon Oct 16 10:05:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2270000000000001,"15min":1.2749999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":4.0650000000000004,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.2809999999999999,"15min":1.3220000000000001},"min":{"1min":0.72999999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":3.5550000000000002,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.81499999999999995}]},{"osd":5,"last update":"Mon Oct 16 10:05:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.2849999999999999,"15min":1.3180000000000001},"min":{"1min":0.81799999999999995,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":4.1829999999999998,"5min":4.1829999999999998,"15min":4.1829999999999998},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.28,"15min":1.3360000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.0129999999999999,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.73799999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:05:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.3640000000000001,"15min":1.4259999999999999},"min":{"1min":0.72199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":4.0069999999999997,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.355,"15min":1.429},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.96999999999999997}]},{"osd":7,"last update":"Mon Oct 16 10:05:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.375,"15min":1.4330000000000001},"min":{"1min":0.747,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":3.649,"5min":3.649,"15min":3.649},"last":1},{"interface":"front","average":{"1min":1.5,"5min":1.4830000000000001,"15min":1.5529999999999999},"min":{"1min":0.95699999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":3.8730000000000002,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":1.0369999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575049,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40804,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331036,"statfs":{"total":96636764160,"available":96594980864,"internally_reserved":0,"allocated":1331200,"data_stored":632639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.89800000000000002,"15min":0.878},"min":{"1min":0.35899999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.1829999999999998,"5min":5.1829999999999998,"15min":5.1829999999999998},"last":1.26},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.92900000000000005,"15min":1.028},"min":{"1min":0.50700000000000001,"5min":0.34999999999999998,"15min":0.28899999999999998},"max":{"1min":1.452,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.80900000000000005}]},{"osd":1,"last update":"Mon Oct 16 10:05:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.04,"15min":1.0389999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":3.6040000000000001,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":1.5900000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.016,"15min":1.1120000000000001},"min":{"1min":0.38300000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3999999999999999,"5min":1.792,"15min":2.6499999999999999},"last":1.0329999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.95199999999999996,"15min":1.0069999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.3260000000000001,"5min":1.776,"15min":2.5339999999999998},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.171,"15min":1.204},"min":{"1min":0.69999999999999996,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":3.6970000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":1.7230000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:05:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.321,"15min":1.3440000000000001},"min":{"1min":0.84399999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.345},{"interface":"front","average":{"1min":1.258,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.71899999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.262,"5min":3.262,"15min":3.262},"last":1.6459999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:05:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.367,"15min":1.3740000000000001},"min":{"1min":0.84099999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.9169999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":2.0259999999999998},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.3480000000000001,"15min":1.399},"min":{"1min":0.73699999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.774,"5min":1.956,"15min":1.956},"last":1.4259999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:05:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3280000000000001,"15min":1.345},"min":{"1min":0.91500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":3.169,"5min":3.169,"15min":3.169},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.4239999999999999,"15min":1.4419999999999999},"min":{"1min":0.88800000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":3.8340000000000001,"5min":3.8340000000000001,"15min":3.8340000000000001},"last":2.2549999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:05:35 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.429,"15min":1.4730000000000001},"min":{"1min":0.81200000000000006,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.907,"5min":2.0739999999999998,"15min":2.4620000000000002},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.391,"15min":1.4350000000000001},"min":{"1min":0.98799999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":3.3780000000000001,"5min":3.3780000000000001,"15min":3.3780000000000001},"last":1.774}]}]},{"osd":1,"up_from":13,"seq":55834575049,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54700,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317140,"statfs":{"total":96636764160,"available":96580751360,"internally_reserved":0,"allocated":1339392,"data_stored":632251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:05:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":0.875},"min":{"1min":0.47599999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.631},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.96099999999999997,"15min":0.99099999999999999},"min":{"1min":0.42299999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4339999999999999,"5min":1.798,"15min":1.798},"last":0.93899999999999995}]},{"osd":2,"last update":"Mon Oct 16 10:05:15 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.96799999999999997,"15min":0.98499999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.42299999999999999},"max":{"1min":1.5069999999999999,"5min":1.526,"15min":1.673},"last":1.339},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.151,"15min":1.2290000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.482,"5min":1.6419999999999999,"15min":1.7669999999999999},"last":1.476}]},{"osd":3,"last update":"Mon Oct 16 10:05:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.0469999999999999,"15min":1.129},"min":{"1min":0.442,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.873},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.026,"15min":1.099},"min":{"1min":0.70699999999999996,"5min":0.40699999999999997,"15min":0.36099999999999999},"max":{"1min":1.3779999999999999,"5min":1.6879999999999999,"15min":1.8879999999999999},"last":1.1220000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1879999999999999,"15min":1.236},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":2.0259999999999998},"last":1.03},{"interface":"front","average":{"1min":1.228,"5min":1.2629999999999999,"15min":1.361},"min":{"1min":0.67600000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":2.1320000000000001},"last":1.3169999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2729999999999999,"15min":1.3320000000000001},"min":{"1min":0.77300000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.696,"5min":1.7470000000000001,"15min":1.9359999999999999},"last":1.276},{"interface":"front","average":{"1min":1.26,"5min":1.27,"15min":1.327},"min":{"1min":0.77800000000000002,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.661,"5min":1.7869999999999999,"15min":2.0470000000000002},"last":1.2290000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3400000000000001,"15min":1.395},"min":{"1min":0.88200000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.325,"15min":1.3859999999999999},"min":{"1min":0.80800000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.718,"5min":1.851,"15min":2.1680000000000001},"last":1.4350000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.3640000000000001,"15min":1.4710000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.1699999999999999},"last":1.375},{"interface":"front","average":{"1min":1.361,"5min":1.4530000000000001,"15min":1.49},"min":{"1min":0.95199999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.8280000000000001,"5min":2.1829999999999998,"15min":2.1960000000000002},"last":1.419}]}]},{"osd":0,"up_from":13,"seq":55834575050,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41376,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330464,"statfs":{"total":96636764160,"available":96594395136,"internally_reserved":0,"allocated":1925120,"data_stored":1222459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.874,"15min":0.85999999999999999},"min":{"1min":0.43099999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.7589999999999999,"5min":2.766,"15min":2.766},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.98299999999999998,"15min":1.034},"min":{"1min":0.51300000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.571,"5min":3.96,"15min":3.96},"last":1.3109999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:05:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0580000000000001,"15min":1.044},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.0449999999999999,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.081,"15min":1.0820000000000001},"min":{"1min":0.54000000000000004,"5min":0.48999999999999999,"15min":0.46600000000000003},"max":{"1min":1.3500000000000001,"5min":4.4290000000000003,"15min":4.4290000000000003},"last":1.1719999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.0189999999999999,"15min":1.03},"min":{"1min":0.442,"5min":0.38,"15min":0.38},"max":{"1min":1.5920000000000001,"5min":4.7380000000000004,"15min":4.7380000000000004},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.123,"5min":1.204,"15min":1.2010000000000001},"min":{"1min":0.505,"5min":0.44,"15min":0.44},"max":{"1min":1.8759999999999999,"5min":5.1500000000000004,"15min":5.1500000000000004},"last":1.73}]},{"osd":4,"last update":"Mon Oct 16 10:05:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3280000000000001,"15min":1.3320000000000001},"min":{"1min":0.83399999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5920000000000001,"5min":4.1970000000000001,"15min":4.1970000000000001},"last":1.411},{"interface":"front","average":{"1min":1.181,"5min":1.2769999999999999,"15min":1.2869999999999999},"min":{"1min":0.81799999999999995,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.502,"5min":4.9630000000000001,"15min":4.9630000000000001},"last":1.1180000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:05:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.304,"15min":1.3140000000000001},"min":{"1min":0.76600000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.593,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.4159999999999999,"15min":1.403},"min":{"1min":0.89600000000000002,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.0190000000000001,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":1.6679999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:05:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.462,"15min":1.4410000000000001},"min":{"1min":0.84899999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.1030000000000002,"5min":3.8250000000000002,"15min":3.8250000000000002},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.4350000000000001,"15min":1.427},"min":{"1min":1.0309999999999999,"5min":0.85799999999999998,"15min":0.84999999999999998},"max":{"1min":1.998,"5min":4.0979999999999999,"15min":4.0979999999999999},"last":1.605}]},{"osd":7,"last update":"Mon Oct 16 10:05:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4890000000000001,"15min":1.482},"min":{"1min":0.89000000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.0750000000000002,"5min":4.3369999999999997,"15min":4.3369999999999997},"last":1.51},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.4259999999999999,"15min":1.4319999999999999},"min":{"1min":0.78900000000000003,"5min":0.755,"15min":0.755},"max":{"1min":1.7569999999999999,"5min":5.0570000000000004,"15min":5.0570000000000004},"last":1.375}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:05:53.831 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:05:54.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.090+0000 7f96a1f99640 1 Processor -- start 2023-10-16T10:05:54.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.090+0000 7f96a1f99640 1 -- start start 2023-10-16T10:05:54.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.092+0000 7f96a1f99640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f969c159950 0x7f969c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:54.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.092+0000 7f96a1f99640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f969c06c810 0x7f969c0708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:54.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.092+0000 7f96a1f99640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969c070df0 con 0x7f969c070450 2023-10-16T10:05:54.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.092+0000 7f96a1f99640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969c15e490 con 0x7f969c06c810 2023-10-16T10:05:54.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.092+0000 7f96a1f99640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c15aef0 con 0x7f969c159950 2023-10-16T10:05:54.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.092+0000 7f969bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f969c070450 0x7f969c06c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53692/0 (socket says 172.21.15.84:53692) 2023-10-16T10:05:54.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.092+0000 7f969bfff640 1 -- 172.21.15.84:0/3333837087 learned_addr learned my addr 172.21.15.84:0/3333837087 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:54.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.092+0000 7f96a1798640 1 --2- 172.21.15.84:0/3333837087 >> v2:172.21.15.84:3300/0 conn(0x7f969c06c810 0x7f969c0708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:54.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.092+0000 7f96a0f97640 1 --2- 172.21.15.84:0/3333837087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f969c159950 0x7f969c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:54.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.093+0000 7f969b7fe640 1 -- 172.21.15.84:0/3333837087 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257909746 0 0) 0x7f969c070df0 con 0x7f969c070450 2023-10-16T10:05:54.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.093+0000 7f969b7fe640 1 -- 172.21.15.84:0/3333837087 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f968c003520 con 0x7f969c070450 2023-10-16T10:05:54.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.093+0000 7f96a1798640 1 -- 172.21.15.84:0/3333837087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f969c159950 msgr2=0x7f969c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:54.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.093+0000 7f96a1798640 1 --2- 172.21.15.84:0/3333837087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f969c159950 0x7f969c15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.093+0000 7f96a1798640 1 -- 172.21.15.84:0/3333837087 >> v1:172.21.15.84:6789/0 conn(0x7f969c070450 legacy=0x7f969c06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:54.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.093+0000 7f96a1798640 1 -- 172.21.15.84:0/3333837087 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c06cc80 con 0x7f969c06c810 2023-10-16T10:05:54.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.093+0000 7f96a0f97640 1 --2- 172.21.15.84:0/3333837087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f969c159950 0x7f969c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:05:54.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.094+0000 7f96a1798640 1 --2- 172.21.15.84:0/3333837087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f969c06c810 0x7f969c0708b0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f9694009d90 tx=0x7f969402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d11c2519d929196 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:54.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.094+0000 7f969b7fe640 1 -- 172.21.15.84:0/3333837087 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969402cbc0 con 0x7f969c06c810 2023-10-16T10:05:54.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.094+0000 7f969b7fe640 1 -- 172.21.15.84:0/3333837087 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969402cd20 con 0x7f969c06c810 2023-10-16T10:05:54.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.094+0000 7f969b7fe640 1 -- 172.21.15.84:0/3333837087 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694034640 con 0x7f969c06c810 2023-10-16T10:05:54.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.095+0000 7f96a1f99640 1 -- 172.21.15.84:0/3333837087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f969c06c810 msgr2=0x7f969c0708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:54.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.095+0000 7f96a1f99640 1 --2- 172.21.15.84:0/3333837087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f969c06c810 0x7f969c0708b0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f9694009d90 tx=0x7f969402be10 comp rx=0 tx=0).stop 2023-10-16T10:05:54.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.095+0000 7f96a1f99640 1 -- 172.21.15.84:0/3333837087 shutdown_connections 2023-10-16T10:05:54.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.095+0000 7f96a1f99640 1 --2- 172.21.15.84:0/3333837087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f969c06c810 0x7f969c0708b0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.095+0000 7f96a1f99640 1 --2- 172.21.15.84:0/3333837087 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f969c159950 0x7f969c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.095+0000 7f96a1f99640 1 -- 172.21.15.84:0/3333837087 >> 172.21.15.84:0/3333837087 conn(0x7f969c0ce6a0 msgr2=0x7f969c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:54.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.095+0000 7f96a1f99640 1 -- 172.21.15.84:0/3333837087 shutdown_connections 2023-10-16T10:05:54.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.095+0000 7f96a1f99640 1 -- 172.21.15.84:0/3333837087 wait complete. 2023-10-16T10:05:54.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.097+0000 7f96a1f99640 1 Processor -- start 2023-10-16T10:05:54.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.097+0000 7f96a1f99640 1 -- start start 2023-10-16T10:05:54.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.098+0000 7f96a1f99640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f969c06c810 0x7f969c15b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:54.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.098+0000 7f96a1f99640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f969c070450 0x7f969c0afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:54.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.098+0000 7f96a1f99640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f969c159950 0x7f969c152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:54.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.098+0000 7f96a1f99640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c15b6e0 con 0x7f969c070450 2023-10-16T10:05:54.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.098+0000 7f96a1f99640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c15b9c0 con 0x7f969c159950 2023-10-16T10:05:54.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.098+0000 7f96a1f99640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c15bca0 con 0x7f969c06c810 2023-10-16T10:05:54.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.098+0000 7f969bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f969c070450 0x7f969c0afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:54.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.098+0000 7f969bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f969c070450 0x7f969c0afee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60722/0 (socket says 172.21.15.84:60722) 2023-10-16T10:05:54.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.098+0000 7f969bfff640 1 -- 172.21.15.84:0/4202040259 learned_addr learned my addr 172.21.15.84:0/4202040259 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:54.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.099+0000 7f969bfff640 1 -- 172.21.15.84:0/4202040259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f969c06c810 msgr2=0x7f969c15b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:54.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.099+0000 7f96a1798640 1 --2- 172.21.15.84:0/4202040259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f969c159950 0x7f969c152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:54.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.099+0000 7f96a0f97640 1 --2- 172.21.15.84:0/4202040259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f969c06c810 0x7f969c15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:54.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.099+0000 7f969bfff640 1 --2- 172.21.15.84:0/4202040259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f969c06c810 0x7f969c15b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.099+0000 7f969bfff640 1 -- 172.21.15.84:0/4202040259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f969c159950 msgr2=0x7f969c152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:54.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.099+0000 7f969bfff640 1 --2- 172.21.15.84:0/4202040259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f969c159950 0x7f969c152b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.099+0000 7f969bfff640 1 -- 172.21.15.84:0/4202040259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9694009a70 con 0x7f969c070450 2023-10-16T10:05:54.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.099+0000 7f969bfff640 1 --2- 172.21.15.84:0/4202040259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f969c070450 0x7f969c0afee0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f968800c430 tx=0x7f968800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:54.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.099+0000 7f96997fa640 1 -- 172.21.15.84:0/4202040259 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688005ce0 con 0x7f969c070450 2023-10-16T10:05:54.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.099+0000 7f96997fa640 1 -- 172.21.15.84:0/4202040259 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9688005e40 con 0x7f969c070450 2023-10-16T10:05:54.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.099+0000 7f96a1f99640 1 -- 172.21.15.84:0/4202040259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969c15bf80 con 0x7f969c070450 2023-10-16T10:05:54.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.099+0000 7f96997fa640 1 -- 172.21.15.84:0/4202040259 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96880072f0 con 0x7f969c070450 2023-10-16T10:05:54.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.099+0000 7f96a1f99640 1 -- 172.21.15.84:0/4202040259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c0b0420 con 0x7f969c070450 2023-10-16T10:05:54.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.101+0000 7f96a1f99640 1 -- 172.21.15.84:0/4202040259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969c0b1a60 con 0x7f969c070450 2023-10-16T10:05:54.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.102+0000 7f96997fa640 1 -- 172.21.15.84:0/4202040259 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f9688007450 con 0x7f969c070450 2023-10-16T10:05:54.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.102+0000 7f96997fa640 1 --2- 172.21.15.84:0/4202040259 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f968006af00 0x7f968006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:54.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.103+0000 7f96997fa640 1 -- 172.21.15.84:0/4202040259 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9688088dc0 con 0x7f969c070450 2023-10-16T10:05:54.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.103+0000 7f96a0f97640 1 --2- 172.21.15.84:0/4202040259 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f968006af00 0x7f968006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:54.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.103+0000 7f96a0f97640 1 --2- 172.21.15.84:0/4202040259 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f968006af00 0x7f968006d3c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9690006fd0 tx=0x7f9690008040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:54.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.107+0000 7f96997fa640 1 -- 172.21.15.84:0/4202040259 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f969c0b1a60 con 0x7f969c070450 2023-10-16T10:05:54.280 INFO:tasks.ceph.mds.a.smithi084.stderr:2023-10-16T10:05:54.278+0000 7f8cd56dc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:05:54.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.301+0000 7f96a1f99640 1 -- 172.21.15.84:0/4202040259 --> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f969c15dbf0 con 0x7f968006af00 2023-10-16T10:05:54.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.314+0000 7f96997fa640 1 -- 172.21.15.84:0/4202040259 <== mgr.7284 v2:172.21.15.84:6832/3250823028 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287716 (secure 0 0 0) 0x7f969000de20 con 0x7f968006af00 2023-10-16T10:05:54.320 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:54.322 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T10:05:54.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.320+0000 7f96a1f99640 1 -- 172.21.15.84:0/4202040259 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f968006af00 msgr2=0x7f968006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:54.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.321+0000 7f96a1f99640 1 --2- 172.21.15.84:0/4202040259 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f968006af00 0x7f968006d3c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9690006fd0 tx=0x7f9690008040 comp rx=0 tx=0).stop 2023-10-16T10:05:54.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.321+0000 7f96a1f99640 1 -- 172.21.15.84:0/4202040259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f969c070450 msgr2=0x7f969c0afee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:54.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.321+0000 7f96a1f99640 1 --2- 172.21.15.84:0/4202040259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f969c070450 0x7f969c0afee0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f968800c430 tx=0x7f968800c8f0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.321+0000 7f96a1f99640 1 -- 172.21.15.84:0/4202040259 shutdown_connections 2023-10-16T10:05:54.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.321+0000 7f96a1f99640 1 --2- 172.21.15.84:0/4202040259 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f968006af00 0x7f968006d3c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.321+0000 7f96a1f99640 1 --2- 172.21.15.84:0/4202040259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f969c159950 0x7f969c152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.321+0000 7f96a1f99640 1 --2- 172.21.15.84:0/4202040259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f969c070450 0x7f969c0afee0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.321+0000 7f96a1f99640 1 --2- 172.21.15.84:0/4202040259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f969c06c810 0x7f969c15b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.321+0000 7f96a1f99640 1 -- 172.21.15.84:0/4202040259 >> 172.21.15.84:0/4202040259 conn(0x7f969c0ce6a0 msgr2=0x7f969c06aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:54.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.321+0000 7f96a1f99640 1 -- 172.21.15.84:0/4202040259 shutdown_connections 2023-10-16T10:05:54.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.321+0000 7f96a1f99640 1 -- 172.21.15.84:0/4202040259 wait complete. 2023-10-16T10:05:54.343 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-16T10:05:53.478282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":640,"num_read_kb":1170,"num_write":305,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2186,"ondisk_log_size":2186,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":322088,"kb_used_data":11592,"kb_used_omap":12,"kb_used_meta":310387,"kb_avail":754652632,"statfs":{"total":773094113280,"available":772764295168,"internally_reserved":0,"allocated":11870208,"data_stored":6242012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":317836857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001870"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.361271+0000","last_change":"2023-10-16T10:05:49.362542+0000","last_active":"2023-10-16T10:05:50.361271+0000","last_peered":"2023-10-16T10:05:50.361271+0000","last_clean":"2023-10-16T10:05:50.361271+0000","last_became_active":"2023-10-16T10:05:49.361614+0000","last_became_peered":"2023-10-16T10:05:49.361614+0000","last_unstale":"2023-10-16T10:05:50.361271+0000","last_undegraded":"2023-10-16T10:05:50.361271+0000","last_fullsized":"2023-10-16T10:05:50.361271+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:54.756998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.899649+0000","last_change":"2023-10-16T10:05:49.372635+0000","last_active":"2023-10-16T10:05:50.899649+0000","last_peered":"2023-10-16T10:05:50.899649+0000","last_clean":"2023-10-16T10:05:50.899649+0000","last_became_active":"2023-10-16T10:05:49.372346+0000","last_became_peered":"2023-10-16T10:05:49.372346+0000","last_unstale":"2023-10-16T10:05:50.899649+0000","last_undegraded":"2023-10-16T10:05:50.899649+0000","last_fullsized":"2023-10-16T10:05:50.899649+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:24:34.556467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.697038+0000","last_change":"2023-10-16T10:05:49.369954+0000","last_active":"2023-10-16T10:05:50.697038+0000","last_peered":"2023-10-16T10:05:50.697038+0000","last_clean":"2023-10-16T10:05:50.697038+0000","last_became_active":"2023-10-16T10:05:49.368928+0000","last_became_peered":"2023-10-16T10:05:49.368928+0000","last_unstale":"2023-10-16T10:05:50.697038+0000","last_undegraded":"2023-10-16T10:05:50.697038+0000","last_fullsized":"2023-10-16T10:05:50.697038+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:28:59.129150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.353862+0000","last_change":"2023-10-16T10:05:49.360594+0000","last_active":"2023-10-16T10:05:50.353862+0000","last_peered":"2023-10-16T10:05:50.353862+0000","last_clean":"2023-10-16T10:05:50.353862+0000","last_became_active":"2023-10-16T10:05:49.360319+0000","last_became_peered":"2023-10-16T10:05:49.360319+0000","last_unstale":"2023-10-16T10:05:50.353862+0000","last_undegraded":"2023-10-16T10:05:50.353862+0000","last_fullsized":"2023-10-16T10:05:50.353862+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:10:41.364870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.357425+0000","last_change":"2023-10-16T10:05:49.357925+0000","last_active":"2023-10-16T10:05:50.357425+0000","last_peered":"2023-10-16T10:05:50.357425+0000","last_clean":"2023-10-16T10:05:50.357425+0000","last_became_active":"2023-10-16T10:05:49.357658+0000","last_became_peered":"2023-10-16T10:05:49.357658+0000","last_unstale":"2023-10-16T10:05:50.357425+0000","last_undegraded":"2023-10-16T10:05:50.357425+0000","last_fullsized":"2023-10-16T10:05:50.357425+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:18.108605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.897841+0000","last_change":"2023-10-16T10:05:49.369444+0000","last_active":"2023-10-16T10:05:50.897841+0000","last_peered":"2023-10-16T10:05:50.897841+0000","last_clean":"2023-10-16T10:05:50.897841+0000","last_became_active":"2023-10-16T10:05:49.369082+0000","last_became_peered":"2023-10-16T10:05:49.369082+0000","last_unstale":"2023-10-16T10:05:50.897841+0000","last_undegraded":"2023-10-16T10:05:50.897841+0000","last_fullsized":"2023-10-16T10:05:50.897841+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:30:54.892423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.353466+0000","last_change":"2023-10-16T10:05:49.365766+0000","last_active":"2023-10-16T10:05:50.353466+0000","last_peered":"2023-10-16T10:05:50.353466+0000","last_clean":"2023-10-16T10:05:50.353466+0000","last_became_active":"2023-10-16T10:05:49.365494+0000","last_became_peered":"2023-10-16T10:05:49.365494+0000","last_unstale":"2023-10-16T10:05:50.353466+0000","last_undegraded":"2023-10-16T10:05:50.353466+0000","last_fullsized":"2023-10-16T10:05:50.353466+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:21.782751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.897553+0000","last_change":"2023-10-16T10:05:49.358313+0000","last_active":"2023-10-16T10:05:50.897553+0000","last_peered":"2023-10-16T10:05:50.897553+0000","last_clean":"2023-10-16T10:05:50.897553+0000","last_became_active":"2023-10-16T10:05:49.357332+0000","last_became_peered":"2023-10-16T10:05:49.357332+0000","last_unstale":"2023-10-16T10:05:50.897553+0000","last_undegraded":"2023-10-16T10:05:50.897553+0000","last_fullsized":"2023-10-16T10:05:50.897553+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:56.984492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.898611+0000","last_change":"2023-10-16T10:05:49.365756+0000","last_active":"2023-10-16T10:05:50.898611+0000","last_peered":"2023-10-16T10:05:50.898611+0000","last_clean":"2023-10-16T10:05:50.898611+0000","last_became_active":"2023-10-16T10:05:49.363995+0000","last_became_peered":"2023-10-16T10:05:49.363995+0000","last_unstale":"2023-10-16T10:05:50.898611+0000","last_undegraded":"2023-10-16T10:05:50.898611+0000","last_fullsized":"2023-10-16T10:05:50.898611+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:53:29.830897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.361002+0000","last_change":"2023-10-16T10:05:49.362906+0000","last_active":"2023-10-16T10:05:50.361002+0000","last_peered":"2023-10-16T10:05:50.361002+0000","last_clean":"2023-10-16T10:05:50.361002+0000","last_became_active":"2023-10-16T10:05:49.362569+0000","last_became_peered":"2023-10-16T10:05:49.362569+0000","last_unstale":"2023-10-16T10:05:50.361002+0000","last_undegraded":"2023-10-16T10:05:50.361002+0000","last_fullsized":"2023-10-16T10:05:50.361002+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:21.802686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.900551+0000","last_change":"2023-10-16T10:05:49.357681+0000","last_active":"2023-10-16T10:05:50.900551+0000","last_peered":"2023-10-16T10:05:50.900551+0000","last_clean":"2023-10-16T10:05:50.900551+0000","last_became_active":"2023-10-16T10:05:49.357394+0000","last_became_peered":"2023-10-16T10:05:49.357394+0000","last_unstale":"2023-10-16T10:05:50.900551+0000","last_undegraded":"2023-10-16T10:05:50.900551+0000","last_fullsized":"2023-10-16T10:05:50.900551+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:06:38.741132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.359436+0000","last_change":"2023-10-16T10:05:49.364749+0000","last_active":"2023-10-16T10:05:50.359436+0000","last_peered":"2023-10-16T10:05:50.359436+0000","last_clean":"2023-10-16T10:05:50.359436+0000","last_became_active":"2023-10-16T10:05:49.364308+0000","last_became_peered":"2023-10-16T10:05:49.364308+0000","last_unstale":"2023-10-16T10:05:50.359436+0000","last_undegraded":"2023-10-16T10:05:50.359436+0000","last_fullsized":"2023-10-16T10:05:50.359436+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:52.919073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.696690+0000","last_change":"2023-10-16T10:05:49.370396+0000","last_active":"2023-10-16T10:05:50.696690+0000","last_peered":"2023-10-16T10:05:50.696690+0000","last_clean":"2023-10-16T10:05:50.696690+0000","last_became_active":"2023-10-16T10:05:49.369607+0000","last_became_peered":"2023-10-16T10:05:49.369607+0000","last_unstale":"2023-10-16T10:05:50.696690+0000","last_undegraded":"2023-10-16T10:05:50.696690+0000","last_fullsized":"2023-10-16T10:05:50.696690+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:03:23.479796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.353492+0000","last_change":"2023-10-16T10:05:49.361044+0000","last_active":"2023-10-16T10:05:50.353492+0000","last_peered":"2023-10-16T10:05:50.353492+0000","last_clean":"2023-10-16T10:05:50.353492+0000","last_became_active":"2023-10-16T10:05:49.360562+0000","last_became_peered":"2023-10-16T10:05:49.360562+0000","last_unstale":"2023-10-16T10:05:50.353492+0000","last_undegraded":"2023-10-16T10:05:50.353492+0000","last_fullsized":"2023-10-16T10:05:50.353492+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:39:33.028029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.898245+0000","last_change":"2023-10-16T10:05:49.356572+0000","last_active":"2023-10-16T10:05:50.898245+0000","last_peered":"2023-10-16T10:05:50.898245+0000","last_clean":"2023-10-16T10:05:50.898245+0000","last_became_active":"2023-10-16T10:05:49.354173+0000","last_became_peered":"2023-10-16T10:05:49.354173+0000","last_unstale":"2023-10-16T10:05:50.898245+0000","last_undegraded":"2023-10-16T10:05:50.898245+0000","last_fullsized":"2023-10-16T10:05:50.898245+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:06.670523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.356640+0000","last_change":"2023-10-16T10:05:49.363448+0000","last_active":"2023-10-16T10:05:50.356640+0000","last_peered":"2023-10-16T10:05:50.356640+0000","last_clean":"2023-10-16T10:05:50.356640+0000","last_became_active":"2023-10-16T10:05:49.363099+0000","last_became_peered":"2023-10-16T10:05:49.363099+0000","last_unstale":"2023-10-16T10:05:50.356640+0000","last_undegraded":"2023-10-16T10:05:50.356640+0000","last_fullsized":"2023-10-16T10:05:50.356640+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:17:51.752174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.360033+0000","last_change":"2023-10-16T10:05:49.345381+0000","last_active":"2023-10-16T10:05:50.360033+0000","last_peered":"2023-10-16T10:05:50.360033+0000","last_clean":"2023-10-16T10:05:50.360033+0000","last_became_active":"2023-10-16T10:05:49.345087+0000","last_became_peered":"2023-10-16T10:05:49.345087+0000","last_unstale":"2023-10-16T10:05:50.360033+0000","last_undegraded":"2023-10-16T10:05:50.360033+0000","last_fullsized":"2023-10-16T10:05:50.360033+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:44:34.031744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.354356+0000","last_change":"2023-10-16T10:05:49.367804+0000","last_active":"2023-10-16T10:05:50.354356+0000","last_peered":"2023-10-16T10:05:50.354356+0000","last_clean":"2023-10-16T10:05:50.354356+0000","last_became_active":"2023-10-16T10:05:49.367512+0000","last_became_peered":"2023-10-16T10:05:49.367512+0000","last_unstale":"2023-10-16T10:05:50.354356+0000","last_undegraded":"2023-10-16T10:05:50.354356+0000","last_fullsized":"2023-10-16T10:05:50.354356+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:43:44.531615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.690068+0000","last_change":"2023-10-16T10:05:49.348920+0000","last_active":"2023-10-16T10:05:50.690068+0000","last_peered":"2023-10-16T10:05:50.690068+0000","last_clean":"2023-10-16T10:05:50.690068+0000","last_became_active":"2023-10-16T10:05:49.348283+0000","last_became_peered":"2023-10-16T10:05:49.348283+0000","last_unstale":"2023-10-16T10:05:50.690068+0000","last_undegraded":"2023-10-16T10:05:50.690068+0000","last_fullsized":"2023-10-16T10:05:50.690068+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:39:33.028029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.353875+0000","last_change":"2023-10-16T10:05:49.358870+0000","last_active":"2023-10-16T10:05:50.353875+0000","last_peered":"2023-10-16T10:05:50.353875+0000","last_clean":"2023-10-16T10:05:50.353875+0000","last_became_active":"2023-10-16T10:05:49.358422+0000","last_became_peered":"2023-10-16T10:05:49.358422+0000","last_unstale":"2023-10-16T10:05:50.353875+0000","last_undegraded":"2023-10-16T10:05:50.353875+0000","last_fullsized":"2023-10-16T10:05:50.353875+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:52.919073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.353789+0000","last_change":"2023-10-16T10:05:49.341994+0000","last_active":"2023-10-16T10:05:50.353789+0000","last_peered":"2023-10-16T10:05:50.353789+0000","last_clean":"2023-10-16T10:05:50.353789+0000","last_became_active":"2023-10-16T10:05:49.341723+0000","last_became_peered":"2023-10-16T10:05:49.341723+0000","last_unstale":"2023-10-16T10:05:50.353789+0000","last_undegraded":"2023-10-16T10:05:50.353789+0000","last_fullsized":"2023-10-16T10:05:50.353789+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:20.377792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.898245+0000","last_change":"2023-10-16T10:05:49.357323+0000","last_active":"2023-10-16T10:05:50.898245+0000","last_peered":"2023-10-16T10:05:50.898245+0000","last_clean":"2023-10-16T10:05:50.898245+0000","last_became_active":"2023-10-16T10:05:49.357017+0000","last_became_peered":"2023-10-16T10:05:49.357017+0000","last_unstale":"2023-10-16T10:05:50.898245+0000","last_undegraded":"2023-10-16T10:05:50.898245+0000","last_fullsized":"2023-10-16T10:05:50.898245+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:12:58.465258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.360485+0000","last_change":"2023-10-16T10:05:49.373922+0000","last_active":"2023-10-16T10:05:50.360485+0000","last_peered":"2023-10-16T10:05:50.360485+0000","last_clean":"2023-10-16T10:05:50.360485+0000","last_became_active":"2023-10-16T10:05:49.373519+0000","last_became_peered":"2023-10-16T10:05:49.373519+0000","last_unstale":"2023-10-16T10:05:50.360485+0000","last_undegraded":"2023-10-16T10:05:50.360485+0000","last_fullsized":"2023-10-16T10:05:50.360485+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:02:58.545300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.353632+0000","last_change":"2023-10-16T10:05:49.362481+0000","last_active":"2023-10-16T10:05:50.353632+0000","last_peered":"2023-10-16T10:05:50.353632+0000","last_clean":"2023-10-16T10:05:50.353632+0000","last_became_active":"2023-10-16T10:05:49.362127+0000","last_became_peered":"2023-10-16T10:05:49.362127+0000","last_unstale":"2023-10-16T10:05:50.353632+0000","last_undegraded":"2023-10-16T10:05:50.353632+0000","last_fullsized":"2023-10-16T10:05:50.353632+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:08:05.035979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.899217+0000","last_change":"2023-10-16T10:05:49.364611+0000","last_active":"2023-10-16T10:05:50.899217+0000","last_peered":"2023-10-16T10:05:50.899217+0000","last_clean":"2023-10-16T10:05:50.899217+0000","last_became_active":"2023-10-16T10:05:49.364384+0000","last_became_peered":"2023-10-16T10:05:49.364384+0000","last_unstale":"2023-10-16T10:05:50.899217+0000","last_undegraded":"2023-10-16T10:05:50.899217+0000","last_fullsized":"2023-10-16T10:05:50.899217+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:05.264470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.697230+0000","last_change":"2023-10-16T10:05:49.356210+0000","last_active":"2023-10-16T10:05:50.697230+0000","last_peered":"2023-10-16T10:05:50.697230+0000","last_clean":"2023-10-16T10:05:50.697230+0000","last_became_active":"2023-10-16T10:05:49.355931+0000","last_became_peered":"2023-10-16T10:05:49.355931+0000","last_unstale":"2023-10-16T10:05:50.697230+0000","last_undegraded":"2023-10-16T10:05:50.697230+0000","last_fullsized":"2023-10-16T10:05:50.697230+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:28:30.066110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.357152+0000","last_change":"2023-10-16T10:05:49.349801+0000","last_active":"2023-10-16T10:05:50.357152+0000","last_peered":"2023-10-16T10:05:50.357152+0000","last_clean":"2023-10-16T10:05:50.357152+0000","last_became_active":"2023-10-16T10:05:49.349467+0000","last_became_peered":"2023-10-16T10:05:49.349467+0000","last_unstale":"2023-10-16T10:05:50.357152+0000","last_undegraded":"2023-10-16T10:05:50.357152+0000","last_fullsized":"2023-10-16T10:05:50.357152+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:02.441122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.359625+0000","last_change":"2023-10-16T10:05:49.364564+0000","last_active":"2023-10-16T10:05:50.359625+0000","last_peered":"2023-10-16T10:05:50.359625+0000","last_clean":"2023-10-16T10:05:50.359625+0000","last_became_active":"2023-10-16T10:05:49.363888+0000","last_became_peered":"2023-10-16T10:05:49.363888+0000","last_unstale":"2023-10-16T10:05:50.359625+0000","last_undegraded":"2023-10-16T10:05:50.359625+0000","last_fullsized":"2023-10-16T10:05:50.359625+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:34:31.006127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.899760+0000","last_change":"2023-10-16T10:05:49.350648+0000","last_active":"2023-10-16T10:05:50.899760+0000","last_peered":"2023-10-16T10:05:50.899760+0000","last_clean":"2023-10-16T10:05:50.899760+0000","last_became_active":"2023-10-16T10:05:49.346019+0000","last_became_peered":"2023-10-16T10:05:49.346019+0000","last_unstale":"2023-10-16T10:05:50.899760+0000","last_undegraded":"2023-10-16T10:05:50.899760+0000","last_fullsized":"2023-10-16T10:05:50.899760+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:03.181971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.898613+0000","last_change":"2023-10-16T10:05:49.352496+0000","last_active":"2023-10-16T10:05:50.898613+0000","last_peered":"2023-10-16T10:05:50.898613+0000","last_clean":"2023-10-16T10:05:50.898613+0000","last_became_active":"2023-10-16T10:05:49.351819+0000","last_became_peered":"2023-10-16T10:05:49.351819+0000","last_unstale":"2023-10-16T10:05:50.898613+0000","last_undegraded":"2023-10-16T10:05:50.898613+0000","last_fullsized":"2023-10-16T10:05:50.898613+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:54:12.892643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.356939+0000","last_change":"2023-10-16T10:05:49.341898+0000","last_active":"2023-10-16T10:05:50.356939+0000","last_peered":"2023-10-16T10:05:50.356939+0000","last_clean":"2023-10-16T10:05:50.356939+0000","last_became_active":"2023-10-16T10:05:49.341209+0000","last_became_peered":"2023-10-16T10:05:49.341209+0000","last_unstale":"2023-10-16T10:05:50.356939+0000","last_undegraded":"2023-10-16T10:05:50.356939+0000","last_fullsized":"2023-10-16T10:05:50.356939+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:59:41.639814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.357362+0000","last_change":"2023-10-16T10:05:49.362375+0000","last_active":"2023-10-16T10:05:50.357362+0000","last_peered":"2023-10-16T10:05:50.357362+0000","last_clean":"2023-10-16T10:05:50.357362+0000","last_became_active":"2023-10-16T10:05:49.361934+0000","last_became_peered":"2023-10-16T10:05:49.361934+0000","last_unstale":"2023-10-16T10:05:50.357362+0000","last_undegraded":"2023-10-16T10:05:50.357362+0000","last_fullsized":"2023-10-16T10:05:50.357362+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:10:41.364870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.359222+0000","last_change":"2023-10-16T10:05:49.365913+0000","last_active":"2023-10-16T10:05:50.359222+0000","last_peered":"2023-10-16T10:05:50.359222+0000","last_clean":"2023-10-16T10:05:50.359222+0000","last_became_active":"2023-10-16T10:05:49.364886+0000","last_became_peered":"2023-10-16T10:05:49.364886+0000","last_unstale":"2023-10-16T10:05:50.359222+0000","last_undegraded":"2023-10-16T10:05:50.359222+0000","last_fullsized":"2023-10-16T10:05:50.359222+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:13:55.500718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"107'4","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.373750+0000","last_change":"2023-10-16T10:05:48.353738+0000","last_active":"2023-10-16T10:05:49.373750+0000","last_peered":"2023-10-16T10:05:49.373750+0000","last_clean":"2023-10-16T10:05:49.373750+0000","last_became_active":"2023-10-16T10:05:48.352267+0000","last_became_peered":"2023-10-16T10:05:48.352267+0000","last_unstale":"2023-10-16T10:05:49.373750+0000","last_undegraded":"2023-10-16T10:05:49.373750+0000","last_fullsized":"2023-10-16T10:05:49.373750+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:51:22.616889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.361429+0000","last_change":"2023-10-16T10:05:49.368896+0000","last_active":"2023-10-16T10:05:50.361429+0000","last_peered":"2023-10-16T10:05:50.361429+0000","last_clean":"2023-10-16T10:05:50.361429+0000","last_became_active":"2023-10-16T10:05:49.367068+0000","last_became_peered":"2023-10-16T10:05:49.367068+0000","last_unstale":"2023-10-16T10:05:50.361429+0000","last_undegraded":"2023-10-16T10:05:50.361429+0000","last_fullsized":"2023-10-16T10:05:50.361429+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:15.458983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.356193+0000","last_change":"2023-10-16T10:05:48.349504+0000","last_active":"2023-10-16T10:05:49.356193+0000","last_peered":"2023-10-16T10:05:49.356193+0000","last_clean":"2023-10-16T10:05:49.356193+0000","last_became_active":"2023-10-16T10:05:48.347072+0000","last_became_peered":"2023-10-16T10:05:48.347072+0000","last_unstale":"2023-10-16T10:05:49.356193+0000","last_undegraded":"2023-10-16T10:05:49.356193+0000","last_fullsized":"2023-10-16T10:05:49.356193+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:55:23.570570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.359736+0000","last_change":"2023-10-16T10:05:49.362764+0000","last_active":"2023-10-16T10:05:50.359736+0000","last_peered":"2023-10-16T10:05:50.359736+0000","last_clean":"2023-10-16T10:05:50.359736+0000","last_became_active":"2023-10-16T10:05:49.361980+0000","last_became_peered":"2023-10-16T10:05:49.361980+0000","last_unstale":"2023-10-16T10:05:50.359736+0000","last_undegraded":"2023-10-16T10:05:50.359736+0000","last_fullsized":"2023-10-16T10:05:50.359736+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:43:44.531615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.369705+0000","last_change":"2023-10-16T10:05:48.337816+0000","last_active":"2023-10-16T10:05:49.369705+0000","last_peered":"2023-10-16T10:05:49.369705+0000","last_clean":"2023-10-16T10:05:49.369705+0000","last_became_active":"2023-10-16T10:05:48.337502+0000","last_became_peered":"2023-10-16T10:05:48.337502+0000","last_unstale":"2023-10-16T10:05:49.369705+0000","last_undegraded":"2023-10-16T10:05:49.369705+0000","last_fullsized":"2023-10-16T10:05:49.369705+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:51:22.616889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.697244+0000","last_change":"2023-10-16T10:05:49.354692+0000","last_active":"2023-10-16T10:05:50.697244+0000","last_peered":"2023-10-16T10:05:50.697244+0000","last_clean":"2023-10-16T10:05:50.697244+0000","last_became_active":"2023-10-16T10:05:49.353407+0000","last_became_peered":"2023-10-16T10:05:49.353407+0000","last_unstale":"2023-10-16T10:05:50.697244+0000","last_undegraded":"2023-10-16T10:05:50.697244+0000","last_fullsized":"2023-10-16T10:05:50.697244+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:20:46.312140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.369922+0000","last_change":"2023-10-16T10:05:48.340316+0000","last_active":"2023-10-16T10:05:49.369922+0000","last_peered":"2023-10-16T10:05:49.369922+0000","last_clean":"2023-10-16T10:05:49.369922+0000","last_became_active":"2023-10-16T10:05:48.339905+0000","last_became_peered":"2023-10-16T10:05:48.339905+0000","last_unstale":"2023-10-16T10:05:49.369922+0000","last_undegraded":"2023-10-16T10:05:49.369922+0000","last_fullsized":"2023-10-16T10:05:49.369922+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:11.984515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.898748+0000","last_change":"2023-10-16T10:05:49.349718+0000","last_active":"2023-10-16T10:05:50.898748+0000","last_peered":"2023-10-16T10:05:50.898748+0000","last_clean":"2023-10-16T10:05:50.898748+0000","last_became_active":"2023-10-16T10:05:49.349444+0000","last_became_peered":"2023-10-16T10:05:49.349444+0000","last_unstale":"2023-10-16T10:05:50.898748+0000","last_undegraded":"2023-10-16T10:05:50.898748+0000","last_fullsized":"2023-10-16T10:05:50.898748+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:58:33.795729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.357358+0000","last_change":"2023-10-16T10:05:48.344054+0000","last_active":"2023-10-16T10:05:49.357358+0000","last_peered":"2023-10-16T10:05:49.357358+0000","last_clean":"2023-10-16T10:05:49.357358+0000","last_became_active":"2023-10-16T10:05:48.343091+0000","last_became_peered":"2023-10-16T10:05:48.343091+0000","last_unstale":"2023-10-16T10:05:49.357358+0000","last_undegraded":"2023-10-16T10:05:49.357358+0000","last_fullsized":"2023-10-16T10:05:49.357358+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:10:50.723424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.357890+0000","last_change":"2023-10-16T10:05:49.339498+0000","last_active":"2023-10-16T10:05:50.357890+0000","last_peered":"2023-10-16T10:05:50.357890+0000","last_clean":"2023-10-16T10:05:50.357890+0000","last_became_active":"2023-10-16T10:05:49.338147+0000","last_became_peered":"2023-10-16T10:05:49.338147+0000","last_unstale":"2023-10-16T10:05:50.357890+0000","last_undegraded":"2023-10-16T10:05:50.357890+0000","last_fullsized":"2023-10-16T10:05:50.357890+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:41:09.556814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.355826+0000","last_change":"2023-10-16T10:05:48.346998+0000","last_active":"2023-10-16T10:05:49.355826+0000","last_peered":"2023-10-16T10:05:49.355826+0000","last_clean":"2023-10-16T10:05:49.355826+0000","last_became_active":"2023-10-16T10:05:48.345222+0000","last_became_peered":"2023-10-16T10:05:48.345222+0000","last_unstale":"2023-10-16T10:05:49.355826+0000","last_undegraded":"2023-10-16T10:05:49.355826+0000","last_fullsized":"2023-10-16T10:05:49.355826+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:09.964060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.696427+0000","last_change":"2023-10-16T10:05:49.360514+0000","last_active":"2023-10-16T10:05:50.696427+0000","last_peered":"2023-10-16T10:05:50.696427+0000","last_clean":"2023-10-16T10:05:50.696427+0000","last_became_active":"2023-10-16T10:05:49.359628+0000","last_became_peered":"2023-10-16T10:05:49.359628+0000","last_unstale":"2023-10-16T10:05:50.696427+0000","last_undegraded":"2023-10-16T10:05:50.696427+0000","last_fullsized":"2023-10-16T10:05:50.696427+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:52.671244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.338344+0000","last_change":"2023-10-16T10:05:48.353689+0000","last_active":"2023-10-16T10:05:49.338344+0000","last_peered":"2023-10-16T10:05:49.338344+0000","last_clean":"2023-10-16T10:05:49.338344+0000","last_became_active":"2023-10-16T10:05:48.353179+0000","last_became_peered":"2023-10-16T10:05:48.353179+0000","last_unstale":"2023-10-16T10:05:49.338344+0000","last_undegraded":"2023-10-16T10:05:49.338344+0000","last_fullsized":"2023-10-16T10:05:49.338344+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:48.020621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.898611+0000","last_change":"2023-10-16T10:05:49.356583+0000","last_active":"2023-10-16T10:05:50.898611+0000","last_peered":"2023-10-16T10:05:50.898611+0000","last_clean":"2023-10-16T10:05:50.898611+0000","last_became_active":"2023-10-16T10:05:49.356258+0000","last_became_peered":"2023-10-16T10:05:49.356258+0000","last_unstale":"2023-10-16T10:05:50.898611+0000","last_undegraded":"2023-10-16T10:05:50.898611+0000","last_fullsized":"2023-10-16T10:05:50.898611+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:47.366846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.340244+0000","last_change":"2023-10-16T10:05:48.348940+0000","last_active":"2023-10-16T10:05:49.340244+0000","last_peered":"2023-10-16T10:05:49.340244+0000","last_clean":"2023-10-16T10:05:49.340244+0000","last_became_active":"2023-10-16T10:05:48.348569+0000","last_became_peered":"2023-10-16T10:05:48.348569+0000","last_unstale":"2023-10-16T10:05:49.340244+0000","last_undegraded":"2023-10-16T10:05:49.340244+0000","last_fullsized":"2023-10-16T10:05:49.340244+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:05.681629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.900389+0000","last_change":"2023-10-16T10:05:49.346825+0000","last_active":"2023-10-16T10:05:50.900389+0000","last_peered":"2023-10-16T10:05:50.900389+0000","last_clean":"2023-10-16T10:05:50.900389+0000","last_became_active":"2023-10-16T10:05:49.346547+0000","last_became_peered":"2023-10-16T10:05:49.346547+0000","last_unstale":"2023-10-16T10:05:50.900389+0000","last_undegraded":"2023-10-16T10:05:50.900389+0000","last_fullsized":"2023-10-16T10:05:50.900389+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:45.381047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.351837+0000","last_change":"2023-10-16T10:05:48.356174+0000","last_active":"2023-10-16T10:05:49.351837+0000","last_peered":"2023-10-16T10:05:49.351837+0000","last_clean":"2023-10-16T10:05:49.351837+0000","last_became_active":"2023-10-16T10:05:48.355766+0000","last_became_peered":"2023-10-16T10:05:48.355766+0000","last_unstale":"2023-10-16T10:05:49.351837+0000","last_undegraded":"2023-10-16T10:05:49.351837+0000","last_fullsized":"2023-10-16T10:05:49.351837+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:39.658226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.898746+0000","last_change":"2023-10-16T10:05:49.369245+0000","last_active":"2023-10-16T10:05:50.898746+0000","last_peered":"2023-10-16T10:05:50.898746+0000","last_clean":"2023-10-16T10:05:50.898746+0000","last_became_active":"2023-10-16T10:05:49.368931+0000","last_became_peered":"2023-10-16T10:05:49.368931+0000","last_unstale":"2023-10-16T10:05:50.898746+0000","last_undegraded":"2023-10-16T10:05:50.898746+0000","last_fullsized":"2023-10-16T10:05:50.898746+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:36.705668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.351081+0000","last_change":"2023-10-16T10:05:48.356237+0000","last_active":"2023-10-16T10:05:49.351081+0000","last_peered":"2023-10-16T10:05:49.351081+0000","last_clean":"2023-10-16T10:05:49.351081+0000","last_became_active":"2023-10-16T10:05:48.355917+0000","last_became_peered":"2023-10-16T10:05:48.355917+0000","last_unstale":"2023-10-16T10:05:49.351081+0000","last_undegraded":"2023-10-16T10:05:49.351081+0000","last_fullsized":"2023-10-16T10:05:49.351081+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:39:11.984515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.690127+0000","last_change":"2023-10-16T10:05:49.373308+0000","last_active":"2023-10-16T10:05:50.690127+0000","last_peered":"2023-10-16T10:05:50.690127+0000","last_clean":"2023-10-16T10:05:50.690127+0000","last_became_active":"2023-10-16T10:05:49.372947+0000","last_became_peered":"2023-10-16T10:05:49.372947+0000","last_unstale":"2023-10-16T10:05:50.690127+0000","last_undegraded":"2023-10-16T10:05:50.690127+0000","last_fullsized":"2023-10-16T10:05:50.690127+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:12:35.449167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"107'3","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.375015+0000","last_change":"2023-10-16T10:05:48.349374+0000","last_active":"2023-10-16T10:05:49.375015+0000","last_peered":"2023-10-16T10:05:49.375015+0000","last_clean":"2023-10-16T10:05:49.375015+0000","last_became_active":"2023-10-16T10:05:48.348869+0000","last_became_peered":"2023-10-16T10:05:48.348869+0000","last_unstale":"2023-10-16T10:05:49.375015+0000","last_undegraded":"2023-10-16T10:05:49.375015+0000","last_fullsized":"2023-10-16T10:05:49.375015+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:39:19.257795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.358788+0000","last_change":"2023-10-16T10:05:49.346890+0000","last_active":"2023-10-16T10:05:50.358788+0000","last_peered":"2023-10-16T10:05:50.358788+0000","last_clean":"2023-10-16T10:05:50.358788+0000","last_became_active":"2023-10-16T10:05:49.346229+0000","last_became_peered":"2023-10-16T10:05:49.346229+0000","last_unstale":"2023-10-16T10:05:50.358788+0000","last_undegraded":"2023-10-16T10:05:50.358788+0000","last_fullsized":"2023-10-16T10:05:50.358788+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:28.603799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"107'6","reported_seq":18,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.375017+0000","last_change":"2023-10-16T10:05:48.354401+0000","last_active":"2023-10-16T10:05:49.375017+0000","last_peered":"2023-10-16T10:05:49.375017+0000","last_clean":"2023-10-16T10:05:49.375017+0000","last_became_active":"2023-10-16T10:05:48.353686+0000","last_became_peered":"2023-10-16T10:05:48.353686+0000","last_unstale":"2023-10-16T10:05:49.375017+0000","last_undegraded":"2023-10-16T10:05:49.375017+0000","last_fullsized":"2023-10-16T10:05:49.375017+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:25.385147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.689786+0000","last_change":"2023-10-16T10:05:49.362658+0000","last_active":"2023-10-16T10:05:50.689786+0000","last_peered":"2023-10-16T10:05:50.689786+0000","last_clean":"2023-10-16T10:05:50.689786+0000","last_became_active":"2023-10-16T10:05:49.357650+0000","last_became_peered":"2023-10-16T10:05:49.357650+0000","last_unstale":"2023-10-16T10:05:50.689786+0000","last_undegraded":"2023-10-16T10:05:50.689786+0000","last_fullsized":"2023-10-16T10:05:50.689786+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:21.782751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.357344+0000","last_change":"2023-10-16T10:05:48.345760+0000","last_active":"2023-10-16T10:05:49.357344+0000","last_peered":"2023-10-16T10:05:49.357344+0000","last_clean":"2023-10-16T10:05:49.357344+0000","last_became_active":"2023-10-16T10:05:48.345487+0000","last_became_peered":"2023-10-16T10:05:48.345487+0000","last_unstale":"2023-10-16T10:05:49.357344+0000","last_undegraded":"2023-10-16T10:05:49.357344+0000","last_fullsized":"2023-10-16T10:05:49.357344+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:47:15.203419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.696721+0000","last_change":"2023-10-16T10:05:49.354706+0000","last_active":"2023-10-16T10:05:50.696721+0000","last_peered":"2023-10-16T10:05:50.696721+0000","last_clean":"2023-10-16T10:05:50.696721+0000","last_became_active":"2023-10-16T10:05:49.354158+0000","last_became_peered":"2023-10-16T10:05:49.354158+0000","last_unstale":"2023-10-16T10:05:50.696721+0000","last_undegraded":"2023-10-16T10:05:50.696721+0000","last_fullsized":"2023-10-16T10:05:50.696721+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:30:01.349607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.328714+0000","last_change":"2023-10-16T10:05:48.349903+0000","last_active":"2023-10-16T10:05:49.328714+0000","last_peered":"2023-10-16T10:05:49.328714+0000","last_clean":"2023-10-16T10:05:49.328714+0000","last_became_active":"2023-10-16T10:05:48.349590+0000","last_became_peered":"2023-10-16T10:05:48.349590+0000","last_unstale":"2023-10-16T10:05:49.328714+0000","last_undegraded":"2023-10-16T10:05:49.328714+0000","last_fullsized":"2023-10-16T10:05:49.328714+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:40.208983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.897606+0000","last_change":"2023-10-16T10:05:49.345088+0000","last_active":"2023-10-16T10:05:50.897606+0000","last_peered":"2023-10-16T10:05:50.897606+0000","last_clean":"2023-10-16T10:05:50.897606+0000","last_became_active":"2023-10-16T10:05:49.344395+0000","last_became_peered":"2023-10-16T10:05:49.344395+0000","last_unstale":"2023-10-16T10:05:50.897606+0000","last_undegraded":"2023-10-16T10:05:50.897606+0000","last_fullsized":"2023-10-16T10:05:50.897606+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:05:15.203028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.351850+0000","last_change":"2023-10-16T10:05:48.354759+0000","last_active":"2023-10-16T10:05:49.351850+0000","last_peered":"2023-10-16T10:05:49.351850+0000","last_clean":"2023-10-16T10:05:49.351850+0000","last_became_active":"2023-10-16T10:05:48.354308+0000","last_became_peered":"2023-10-16T10:05:48.354308+0000","last_unstale":"2023-10-16T10:05:49.351850+0000","last_undegraded":"2023-10-16T10:05:49.351850+0000","last_fullsized":"2023-10-16T10:05:49.351850+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:59.348085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.897541+0000","last_change":"2023-10-16T10:05:49.367238+0000","last_active":"2023-10-16T10:05:50.897541+0000","last_peered":"2023-10-16T10:05:50.897541+0000","last_clean":"2023-10-16T10:05:50.897541+0000","last_became_active":"2023-10-16T10:05:49.366929+0000","last_became_peered":"2023-10-16T10:05:49.366929+0000","last_unstale":"2023-10-16T10:05:50.897541+0000","last_undegraded":"2023-10-16T10:05:50.897541+0000","last_fullsized":"2023-10-16T10:05:50.897541+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:48.194506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"107'8","reported_seq":20,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.371133+0000","last_change":"2023-10-16T10:05:48.349165+0000","last_active":"2023-10-16T10:05:49.371133+0000","last_peered":"2023-10-16T10:05:49.371133+0000","last_clean":"2023-10-16T10:05:49.371133+0000","last_became_active":"2023-10-16T10:05:48.348882+0000","last_became_peered":"2023-10-16T10:05:48.348882+0000","last_unstale":"2023-10-16T10:05:49.371133+0000","last_undegraded":"2023-10-16T10:05:49.371133+0000","last_fullsized":"2023-10-16T10:05:49.371133+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:22.991998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.697262+0000","last_change":"2023-10-16T10:05:49.365826+0000","last_active":"2023-10-16T10:05:50.697262+0000","last_peered":"2023-10-16T10:05:50.697262+0000","last_clean":"2023-10-16T10:05:50.697262+0000","last_became_active":"2023-10-16T10:05:49.365053+0000","last_became_peered":"2023-10-16T10:05:49.365053+0000","last_unstale":"2023-10-16T10:05:50.697262+0000","last_undegraded":"2023-10-16T10:05:50.697262+0000","last_fullsized":"2023-10-16T10:05:50.697262+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:43:19.965258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.366296+0000","last_change":"2023-10-16T10:05:48.356751+0000","last_active":"2023-10-16T10:05:49.366296+0000","last_peered":"2023-10-16T10:05:49.366296+0000","last_clean":"2023-10-16T10:05:49.366296+0000","last_became_active":"2023-10-16T10:05:48.356436+0000","last_became_peered":"2023-10-16T10:05:48.356436+0000","last_unstale":"2023-10-16T10:05:49.366296+0000","last_undegraded":"2023-10-16T10:05:49.366296+0000","last_fullsized":"2023-10-16T10:05:49.366296+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:46:51.165351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.361548+0000","last_change":"2023-10-16T10:05:49.369757+0000","last_active":"2023-10-16T10:05:50.361548+0000","last_peered":"2023-10-16T10:05:50.361548+0000","last_clean":"2023-10-16T10:05:50.361548+0000","last_became_active":"2023-10-16T10:05:49.368859+0000","last_became_peered":"2023-10-16T10:05:49.368859+0000","last_unstale":"2023-10-16T10:05:50.361548+0000","last_undegraded":"2023-10-16T10:05:50.361548+0000","last_fullsized":"2023-10-16T10:05:50.361548+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:30.616685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.364588+0000","last_change":"2023-10-16T10:05:48.349507+0000","last_active":"2023-10-16T10:05:49.364588+0000","last_peered":"2023-10-16T10:05:49.364588+0000","last_clean":"2023-10-16T10:05:49.364588+0000","last_became_active":"2023-10-16T10:05:48.349214+0000","last_became_peered":"2023-10-16T10:05:48.349214+0000","last_unstale":"2023-10-16T10:05:49.364588+0000","last_undegraded":"2023-10-16T10:05:49.364588+0000","last_fullsized":"2023-10-16T10:05:49.364588+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:39.658226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.359049+0000","last_change":"2023-10-16T10:05:49.344432+0000","last_active":"2023-10-16T10:05:50.359049+0000","last_peered":"2023-10-16T10:05:50.359049+0000","last_clean":"2023-10-16T10:05:50.359049+0000","last_became_active":"2023-10-16T10:05:49.343192+0000","last_became_peered":"2023-10-16T10:05:49.343192+0000","last_unstale":"2023-10-16T10:05:50.359049+0000","last_undegraded":"2023-10-16T10:05:50.359049+0000","last_fullsized":"2023-10-16T10:05:50.359049+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:52.562154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.338675+0000","last_change":"2023-10-16T10:05:48.355145+0000","last_active":"2023-10-16T10:05:49.338675+0000","last_peered":"2023-10-16T10:05:49.338675+0000","last_clean":"2023-10-16T10:05:49.338675+0000","last_became_active":"2023-10-16T10:05:48.354837+0000","last_became_peered":"2023-10-16T10:05:48.354837+0000","last_unstale":"2023-10-16T10:05:49.338675+0000","last_undegraded":"2023-10-16T10:05:49.338675+0000","last_fullsized":"2023-10-16T10:05:49.338675+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:35.265342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.353526+0000","last_change":"2023-10-16T10:05:49.360866+0000","last_active":"2023-10-16T10:05:50.353526+0000","last_peered":"2023-10-16T10:05:50.353526+0000","last_clean":"2023-10-16T10:05:50.353526+0000","last_became_active":"2023-10-16T10:05:49.360153+0000","last_became_peered":"2023-10-16T10:05:49.360153+0000","last_unstale":"2023-10-16T10:05:50.353526+0000","last_undegraded":"2023-10-16T10:05:50.353526+0000","last_fullsized":"2023-10-16T10:05:50.353526+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:12:35.449167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.349025+0000","last_change":"2023-10-16T10:05:48.343597+0000","last_active":"2023-10-16T10:05:49.349025+0000","last_peered":"2023-10-16T10:05:49.349025+0000","last_clean":"2023-10-16T10:05:49.349025+0000","last_became_active":"2023-10-16T10:05:48.343241+0000","last_became_peered":"2023-10-16T10:05:48.343241+0000","last_unstale":"2023-10-16T10:05:49.349025+0000","last_undegraded":"2023-10-16T10:05:49.349025+0000","last_fullsized":"2023-10-16T10:05:49.349025+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:30:46.063285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.897951+0000","last_change":"2023-10-16T10:05:49.360727+0000","last_active":"2023-10-16T10:05:50.897951+0000","last_peered":"2023-10-16T10:05:50.897951+0000","last_clean":"2023-10-16T10:05:50.897951+0000","last_became_active":"2023-10-16T10:05:49.360458+0000","last_became_peered":"2023-10-16T10:05:49.360458+0000","last_unstale":"2023-10-16T10:05:50.897951+0000","last_undegraded":"2023-10-16T10:05:50.897951+0000","last_fullsized":"2023-10-16T10:05:50.897951+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:20:09.142217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.329315+0000","last_change":"2023-10-16T10:05:48.336655+0000","last_active":"2023-10-16T10:05:49.329315+0000","last_peered":"2023-10-16T10:05:49.329315+0000","last_clean":"2023-10-16T10:05:49.329315+0000","last_became_active":"2023-10-16T10:05:48.332928+0000","last_became_peered":"2023-10-16T10:05:48.332928+0000","last_unstale":"2023-10-16T10:05:49.329315+0000","last_undegraded":"2023-10-16T10:05:49.329315+0000","last_fullsized":"2023-10-16T10:05:49.329315+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:35.265342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.353909+0000","last_change":"2023-10-16T10:05:49.355703+0000","last_active":"2023-10-16T10:05:50.353909+0000","last_peered":"2023-10-16T10:05:50.353909+0000","last_clean":"2023-10-16T10:05:50.353909+0000","last_became_active":"2023-10-16T10:05:49.355443+0000","last_became_peered":"2023-10-16T10:05:49.355443+0000","last_unstale":"2023-10-16T10:05:50.353909+0000","last_undegraded":"2023-10-16T10:05:50.353909+0000","last_fullsized":"2023-10-16T10:05:50.353909+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:34:31.006127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.328588+0000","last_change":"2023-10-16T10:05:48.356672+0000","last_active":"2023-10-16T10:05:49.328588+0000","last_peered":"2023-10-16T10:05:49.328588+0000","last_clean":"2023-10-16T10:05:49.328588+0000","last_became_active":"2023-10-16T10:05:48.356367+0000","last_became_peered":"2023-10-16T10:05:48.356367+0000","last_unstale":"2023-10-16T10:05:49.328588+0000","last_undegraded":"2023-10-16T10:05:49.328588+0000","last_fullsized":"2023-10-16T10:05:49.328588+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:48.020621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.897763+0000","last_change":"2023-10-16T10:05:49.365910+0000","last_active":"2023-10-16T10:05:50.897763+0000","last_peered":"2023-10-16T10:05:50.897763+0000","last_clean":"2023-10-16T10:05:50.897763+0000","last_became_active":"2023-10-16T10:05:49.361383+0000","last_became_peered":"2023-10-16T10:05:49.361383+0000","last_unstale":"2023-10-16T10:05:50.897763+0000","last_undegraded":"2023-10-16T10:05:50.897763+0000","last_fullsized":"2023-10-16T10:05:50.897763+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:48.006728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.328376+0000","last_change":"2023-10-16T10:05:48.341885+0000","last_active":"2023-10-16T10:05:49.328376+0000","last_peered":"2023-10-16T10:05:49.328376+0000","last_clean":"2023-10-16T10:05:49.328376+0000","last_became_active":"2023-10-16T10:05:48.341590+0000","last_became_peered":"2023-10-16T10:05:48.341590+0000","last_unstale":"2023-10-16T10:05:49.328376+0000","last_undegraded":"2023-10-16T10:05:49.328376+0000","last_fullsized":"2023-10-16T10:05:49.328376+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:16:59.947077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.359337+0000","last_change":"2023-10-16T10:05:49.362847+0000","last_active":"2023-10-16T10:05:50.359337+0000","last_peered":"2023-10-16T10:05:50.359337+0000","last_clean":"2023-10-16T10:05:50.359337+0000","last_became_active":"2023-10-16T10:05:49.362516+0000","last_became_peered":"2023-10-16T10:05:49.362516+0000","last_unstale":"2023-10-16T10:05:50.359337+0000","last_undegraded":"2023-10-16T10:05:50.359337+0000","last_fullsized":"2023-10-16T10:05:50.359337+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:02.721303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"107'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.367239+0000","last_change":"2023-10-16T10:05:48.351887+0000","last_active":"2023-10-16T10:05:49.367239+0000","last_peered":"2023-10-16T10:05:49.367239+0000","last_clean":"2023-10-16T10:05:49.367239+0000","last_became_active":"2023-10-16T10:05:48.351613+0000","last_became_peered":"2023-10-16T10:05:48.351613+0000","last_unstale":"2023-10-16T10:05:49.367239+0000","last_undegraded":"2023-10-16T10:05:49.367239+0000","last_fullsized":"2023-10-16T10:05:49.367239+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:14.439273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.697481+0000","last_change":"2023-10-16T10:05:49.367441+0000","last_active":"2023-10-16T10:05:50.697481+0000","last_peered":"2023-10-16T10:05:50.697481+0000","last_clean":"2023-10-16T10:05:50.697481+0000","last_became_active":"2023-10-16T10:05:49.366237+0000","last_became_peered":"2023-10-16T10:05:49.366237+0000","last_unstale":"2023-10-16T10:05:50.697481+0000","last_undegraded":"2023-10-16T10:05:50.697481+0000","last_fullsized":"2023-10-16T10:05:50.697481+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:16:11.590897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.343256+0000","last_change":"2023-10-16T10:05:48.346236+0000","last_active":"2023-10-16T10:05:49.343256+0000","last_peered":"2023-10-16T10:05:49.343256+0000","last_clean":"2023-10-16T10:05:49.343256+0000","last_became_active":"2023-10-16T10:05:48.345957+0000","last_became_peered":"2023-10-16T10:05:48.345957+0000","last_unstale":"2023-10-16T10:05:49.343256+0000","last_undegraded":"2023-10-16T10:05:49.343256+0000","last_fullsized":"2023-10-16T10:05:49.343256+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:13.642849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.359993+0000","last_change":"2023-10-16T10:05:49.371515+0000","last_active":"2023-10-16T10:05:50.359993+0000","last_peered":"2023-10-16T10:05:50.359993+0000","last_clean":"2023-10-16T10:05:50.359993+0000","last_became_active":"2023-10-16T10:05:49.371098+0000","last_became_peered":"2023-10-16T10:05:49.371098+0000","last_unstale":"2023-10-16T10:05:50.359993+0000","last_undegraded":"2023-10-16T10:05:50.359993+0000","last_fullsized":"2023-10-16T10:05:50.359993+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:36:24.167226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.348761+0000","last_change":"2023-10-16T10:05:48.352753+0000","last_active":"2023-10-16T10:05:49.348761+0000","last_peered":"2023-10-16T10:05:49.348761+0000","last_clean":"2023-10-16T10:05:49.348761+0000","last_became_active":"2023-10-16T10:05:48.352350+0000","last_became_peered":"2023-10-16T10:05:48.352350+0000","last_unstale":"2023-10-16T10:05:49.348761+0000","last_undegraded":"2023-10-16T10:05:49.348761+0000","last_fullsized":"2023-10-16T10:05:49.348761+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:48.020621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.359185+0000","last_change":"2023-10-16T10:05:49.346070+0000","last_active":"2023-10-16T10:05:50.359185+0000","last_peered":"2023-10-16T10:05:50.359185+0000","last_clean":"2023-10-16T10:05:50.359185+0000","last_became_active":"2023-10-16T10:05:49.345616+0000","last_became_peered":"2023-10-16T10:05:49.345616+0000","last_unstale":"2023-10-16T10:05:50.359185+0000","last_undegraded":"2023-10-16T10:05:50.359185+0000","last_fullsized":"2023-10-16T10:05:50.359185+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:56:09.445211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.357094+0000","last_change":"2023-10-16T10:05:48.350655+0000","last_active":"2023-10-16T10:05:49.357094+0000","last_peered":"2023-10-16T10:05:49.357094+0000","last_clean":"2023-10-16T10:05:49.357094+0000","last_became_active":"2023-10-16T10:05:48.350221+0000","last_became_peered":"2023-10-16T10:05:48.350221+0000","last_unstale":"2023-10-16T10:05:49.357094+0000","last_undegraded":"2023-10-16T10:05:49.357094+0000","last_fullsized":"2023-10-16T10:05:49.357094+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:08:04.016268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.897876+0000","last_change":"2023-10-16T10:05:49.354934+0000","last_active":"2023-10-16T10:05:50.897876+0000","last_peered":"2023-10-16T10:05:50.897876+0000","last_clean":"2023-10-16T10:05:50.897876+0000","last_became_active":"2023-10-16T10:05:49.354256+0000","last_became_peered":"2023-10-16T10:05:49.354256+0000","last_unstale":"2023-10-16T10:05:50.897876+0000","last_undegraded":"2023-10-16T10:05:50.897876+0000","last_fullsized":"2023-10-16T10:05:50.897876+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:56:33.268643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.339088+0000","last_change":"2023-10-16T10:05:48.349016+0000","last_active":"2023-10-16T10:05:49.339088+0000","last_peered":"2023-10-16T10:05:49.339088+0000","last_clean":"2023-10-16T10:05:49.339088+0000","last_became_active":"2023-10-16T10:05:48.348724+0000","last_became_peered":"2023-10-16T10:05:48.348724+0000","last_unstale":"2023-10-16T10:05:49.339088+0000","last_undegraded":"2023-10-16T10:05:49.339088+0000","last_fullsized":"2023-10-16T10:05:49.339088+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:46:51.165351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.690694+0000","last_change":"2023-10-16T10:05:49.345581+0000","last_active":"2023-10-16T10:05:50.690694+0000","last_peered":"2023-10-16T10:05:50.690694+0000","last_clean":"2023-10-16T10:05:50.690694+0000","last_became_active":"2023-10-16T10:05:49.345319+0000","last_became_peered":"2023-10-16T10:05:49.345319+0000","last_unstale":"2023-10-16T10:05:50.690694+0000","last_undegraded":"2023-10-16T10:05:50.690694+0000","last_fullsized":"2023-10-16T10:05:50.690694+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:08:05.035979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.333033+0000","last_change":"2023-10-16T10:05:48.346770+0000","last_active":"2023-10-16T10:05:49.333033+0000","last_peered":"2023-10-16T10:05:49.333033+0000","last_clean":"2023-10-16T10:05:49.333033+0000","last_became_active":"2023-10-16T10:05:48.343689+0000","last_became_peered":"2023-10-16T10:05:48.343689+0000","last_unstale":"2023-10-16T10:05:49.333033+0000","last_undegraded":"2023-10-16T10:05:49.333033+0000","last_fullsized":"2023-10-16T10:05:49.333033+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:51:59.064234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.696529+0000","last_change":"2023-10-16T10:05:49.364526+0000","last_active":"2023-10-16T10:05:50.696529+0000","last_peered":"2023-10-16T10:05:50.696529+0000","last_clean":"2023-10-16T10:05:50.696529+0000","last_became_active":"2023-10-16T10:05:49.364242+0000","last_became_peered":"2023-10-16T10:05:49.364242+0000","last_unstale":"2023-10-16T10:05:50.696529+0000","last_undegraded":"2023-10-16T10:05:50.696529+0000","last_fullsized":"2023-10-16T10:05:50.696529+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:59.428532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.362334+0000","last_change":"2023-10-16T10:05:48.354276+0000","last_active":"2023-10-16T10:05:49.362334+0000","last_peered":"2023-10-16T10:05:49.362334+0000","last_clean":"2023-10-16T10:05:49.362334+0000","last_became_active":"2023-10-16T10:05:48.353951+0000","last_became_peered":"2023-10-16T10:05:48.353951+0000","last_unstale":"2023-10-16T10:05:49.362334+0000","last_undegraded":"2023-10-16T10:05:49.362334+0000","last_fullsized":"2023-10-16T10:05:49.362334+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:19.358081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.360642+0000","last_change":"2023-10-16T10:05:49.373411+0000","last_active":"2023-10-16T10:05:50.360642+0000","last_peered":"2023-10-16T10:05:50.360642+0000","last_clean":"2023-10-16T10:05:50.360642+0000","last_became_active":"2023-10-16T10:05:49.371728+0000","last_became_peered":"2023-10-16T10:05:49.371728+0000","last_unstale":"2023-10-16T10:05:50.360642+0000","last_undegraded":"2023-10-16T10:05:50.360642+0000","last_fullsized":"2023-10-16T10:05:50.360642+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:47:01.912619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.336572+0000","last_change":"2023-10-16T10:05:48.352039+0000","last_active":"2023-10-16T10:05:49.336572+0000","last_peered":"2023-10-16T10:05:49.336572+0000","last_clean":"2023-10-16T10:05:49.336572+0000","last_became_active":"2023-10-16T10:05:48.351721+0000","last_became_peered":"2023-10-16T10:05:48.351721+0000","last_unstale":"2023-10-16T10:05:49.336572+0000","last_undegraded":"2023-10-16T10:05:49.336572+0000","last_fullsized":"2023-10-16T10:05:49.336572+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:28:22.224872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:50.899354+0000","last_change":"2023-10-16T10:05:49.366056+0000","last_active":"2023-10-16T10:05:50.899354+0000","last_peered":"2023-10-16T10:05:50.899354+0000","last_clean":"2023-10-16T10:05:50.899354+0000","last_became_active":"2023-10-16T10:05:49.365789+0000","last_became_peered":"2023-10-16T10:05:49.365789+0000","last_unstale":"2023-10-16T10:05:50.899354+0000","last_undegraded":"2023-10-16T10:05:50.899354+0000","last_fullsized":"2023-10-16T10:05:50.899354+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:48.306109+0000","last_clean_scrub_stamp":"2023-10-16T10:05:48.306109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:10:07.060924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"107'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-16T10:05:49.368690+0000","last_change":"2023-10-16T10:05:48.351584+0000","last_active":"2023-10-16T10:05:49.368690+0000","last_peered":"2023-10-16T10:05:49.368690+0000","last_clean":"2023-10-16T10:05:49.368690+0000","last_became_active":"2023-10-16T10:05:48.351313+0000","last_became_peered":"2023-10-16T10:05:48.351313+0000","last_unstale":"2023-10-16T10:05:49.368690+0000","last_undegraded":"2023-10-16T10:05:49.368690+0000","last_fullsized":"2023-10-16T10:05:49.368690+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:05:47.286399+0000","last_clean_scrub_stamp":"2023-10-16T10:05:47.286399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:40.208983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"108'2148","reported_seq":2450,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-10-16T10:05:52.471408+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T10:05:52.471408+0000","last_peered":"2023-10-16T10:05:52.471408+0000","last_clean":"2023-10-16T10:05:52.471408+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T10:05:52.471408+0000","last_undegraded":"2023-10-16T10:05:52.471408+0000","last_fullsized":"2023-10-16T10:05:52.471408+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2148,"log_dups_size":0,"ondisk_log_size":2148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":640,"num_read_kb":1170,"num_write":261,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":640,"num_read_kb":1170,"num_write":261,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2148,"ondisk_log_size":2148,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575050,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28896,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342944,"statfs":{"total":96636764160,"available":96607174656,"internally_reserved":0,"allocated":1925120,"data_stored":1222495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.125,"15min":1.1619999999999999},"min":{"1min":0.67200000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5429999999999999,"5min":1.853,"15min":1.925},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.008,"5min":1.1319999999999999,"15min":1.1899999999999999},"min":{"1min":0.59199999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.5169999999999999,"5min":1.75,"15min":2.016},"last":1.343}]},{"osd":1,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1899999999999999,"15min":1.23},"min":{"1min":0.71799999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6259999999999999,"5min":1.923,"15min":1.923},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.171,"15min":1.214},"min":{"1min":0.749,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.464,"5min":1.595,"15min":1.7410000000000001},"last":0.996}]},{"osd":2,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1799999999999999,"15min":1.244},"min":{"1min":0.68899999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7410000000000001,"5min":1.837,"15min":1.8740000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.2490000000000001,"15min":1.2929999999999999},"min":{"1min":0.84799999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4450000000000001,"5min":1.7430000000000001,"15min":1.8620000000000001},"last":1.3}]},{"osd":3,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.2989999999999999,"15min":1.339},"min":{"1min":0.89400000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.661,"5min":1.7929999999999999,"15min":1.996},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.3320000000000001,"15min":1.381},"min":{"1min":0.69099999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.8540000000000001,"5min":1.8759999999999999,"15min":2.004},"last":1.5289999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3720000000000001,"15min":1.3700000000000001},"min":{"1min":0.99199999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.994,"5min":2.028,"15min":2.0310000000000001},"last":1.151},{"interface":"front","average":{"1min":1.359,"5min":1.3149999999999999,"15min":1.319},"min":{"1min":0.96299999999999997,"5min":0.502,"15min":0.502},"max":{"1min":1.9039999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.0589999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.472,"15min":1.516},"min":{"1min":0.91100000000000003,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.081,"5min":2.1320000000000001,"15min":2.165},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4299999999999999,"15min":1.4339999999999999},"min":{"1min":1.097,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.9630000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.2509999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.417,"15min":1.4059999999999999},"min":{"1min":1.175,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.109},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.601,"15min":1.619},"min":{"1min":1.3300000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":1.6479999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575050,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36648,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335192,"statfs":{"total":96636764160,"available":96599236608,"internally_reserved":0,"allocated":1339392,"data_stored":632751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:05:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.127,"15min":1.145},"min":{"1min":0.71799999999999997,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6240000000000001,"5min":1.667,"15min":1.7689999999999999},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.2290000000000001,"15min":1.246},"min":{"1min":0.92300000000000004,"5min":0.65900000000000003,"15min":0.65300000000000002},"max":{"1min":1.7010000000000001,"5min":1.847,"15min":1.8720000000000001},"last":1.466}]},{"osd":1,"last update":"Mon Oct 16 10:05:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1160000000000001,"15min":1.175},"min":{"1min":0.83699999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5189999999999999,"5min":1.6180000000000001,"15min":1.6970000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.276,"15min":1.3},"min":{"1min":0.69099999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.738,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.7070000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:05:20 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.179,"15min":1.2090000000000001},"min":{"1min":0.82599999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.851,"5min":1.851,"15min":2.036},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.2949999999999999,"15min":1.3100000000000001},"min":{"1min":0.90300000000000002,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.5269999999999999,"5min":1.925,"15min":1.925},"last":1.845}]},{"osd":3,"last update":"Mon Oct 16 10:05:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1990000000000001,"15min":1.2250000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.6990000000000001,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.343},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.1850000000000001,"15min":1.2330000000000001},"min":{"1min":0.84999999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.865},"last":1.103}]},{"osd":4,"last update":"Mon Oct 16 10:05:20 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.244,"15min":1.2709999999999999},"min":{"1min":1.0289999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.7729999999999999,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.4470000000000001,"15min":1.4339999999999999},"min":{"1min":0.91900000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.865,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.8069999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:05:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.2869999999999999,"15min":1.3160000000000001},"min":{"1min":0.91800000000000004,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.0790000000000002,"5min":2.3090000000000002,"15min":2.5649999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.4970000000000001,"15min":1.51},"min":{"1min":0.97799999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.867,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.996}]},{"osd":7,"last update":"Mon Oct 16 10:05:20 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4630000000000001,"15min":1.476},"min":{"1min":1.2030000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.9510000000000001,"5min":3.5739999999999998,"15min":3.5739999999999998},"last":1.9390000000000001},{"interface":"front","average":{"1min":1.585,"5min":1.446,"15min":1.4890000000000001},"min":{"1min":0.96099999999999997,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.4249999999999998},"last":1.7669999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575050,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36652,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335188,"statfs":{"total":96636764160,"available":96599232512,"internally_reserved":0,"allocated":1343488,"data_stored":633691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.1499999999999999,"15min":1.224},"min":{"1min":0.59299999999999997,"5min":0.496,"15min":0.496},"max":{"1min":2.3959999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.6779999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.2749999999999999,"15min":1.3080000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.2390000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.111,"15min":1.141},"min":{"1min":0.60499999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.7310000000000001,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.149,"5min":1.1180000000000001,"15min":1.151},"min":{"1min":0.73499999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6970000000000001},"last":1.5149999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.153,"15min":1.167},"min":{"1min":0.83899999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.5609999999999999,"5min":1.7470000000000001,"15min":1.77},"last":1.139},{"interface":"front","average":{"1min":1.244,"5min":1.2230000000000001,"15min":1.242},"min":{"1min":0.77300000000000002,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.6739999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.3100000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.26,"15min":1.2789999999999999},"min":{"1min":0.91600000000000004,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.496,"5min":1.9530000000000001,"15min":1.97},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.323,"15min":1.355},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.5860000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2490000000000001,"15min":1.276},"min":{"1min":0.86599999999999999,"5min":0.77100000000000002,"15min":0.73499999999999999},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":2.355},"last":1.7769999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.496,"15min":1.5149999999999999},"min":{"1min":1.028,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.861,"5min":2.0819999999999999,"15min":2.4079999999999999},"last":1.349}]},{"osd":6,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3460000000000001,"15min":1.365},"min":{"1min":0.85499999999999998,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.786,"5min":1.847,"15min":1.847},"last":1.206},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.3759999999999999,"15min":1.3919999999999999},"min":{"1min":1.0389999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":2.2549999999999999},"last":1.869}]},{"osd":7,"last update":"Mon Oct 16 10:05:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.466,"15min":1.4910000000000001},"min":{"1min":0.93300000000000005,"5min":0.88600000000000001,"15min":0.872},"max":{"1min":1.996,"5min":2.1989999999999998,"15min":2.4660000000000002},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5620000000000001,"15min":1.5760000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":0.99199999999999999},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.407},"last":1.4410000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575051,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46364,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325476,"statfs":{"total":96636764160,"available":96589287424,"internally_reserved":0,"allocated":1327104,"data_stored":632057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:05:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0900000000000001,"15min":1.1479999999999999},"min":{"1min":0.81299999999999994,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.6399999999999999,"5min":1.76,"15min":1.827},"last":1.625},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3620000000000001,"15min":1.256},"min":{"1min":0.94499999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8939999999999999,"5min":11.974,"15min":11.974},"last":1.367}]},{"osd":1,"last update":"Mon Oct 16 10:05:22 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.3700000000000001,"15min":1.292},"min":{"1min":0.98999999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.8939999999999999,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":1.109},{"interface":"front","average":{"1min":1.135,"5min":1.194,"15min":1.238},"min":{"1min":0.73399999999999999,"5min":0.63600000000000001,"15min":0.624},"max":{"1min":1.603,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.0620000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:05:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.24,"15min":1.2430000000000001},"min":{"1min":0.76900000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.4770000000000001,"5min":1.756,"15min":1.843},"last":1.897},{"interface":"front","average":{"1min":1.431,"5min":1.3879999999999999,"15min":1.385},"min":{"1min":1.0549999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.1219999999999999,"5min":5.0549999999999997,"15min":5.0549999999999997},"last":1.266}]},{"osd":3,"last update":"Mon Oct 16 10:05:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3220000000000001,"15min":1.3},"min":{"1min":0.98699999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.744,"5min":6.8079999999999998,"15min":6.8079999999999998},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.369,"15min":1.379},"min":{"1min":1.0289999999999999,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.7350000000000001,"5min":1.885,"15min":1.923},"last":1.744}]},{"osd":5,"last update":"Mon Oct 16 10:05:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.524,"15min":1.4219999999999999},"min":{"1min":1.036,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.1539999999999999,"5min":8.702,"15min":8.702},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3879999999999999,"15min":1.3720000000000001},"min":{"1min":0.92600000000000005,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.032,"5min":3.0430000000000001,"15min":3.0430000000000001},"last":1.236}]},{"osd":6,"last update":"Mon Oct 16 10:05:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4930000000000001,"15min":1.4119999999999999},"min":{"1min":0.84699999999999998,"5min":0.80700000000000005,"15min":0.79000000000000004},"max":{"1min":1.8979999999999999,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":1.8480000000000001},{"interface":"front","average":{"1min":1.714,"5min":1.6279999999999999,"15min":1.554},"min":{"1min":1.3200000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.2549999999999999,"5min":6.4089999999999998,"15min":6.4089999999999998},"last":1.569}]},{"osd":7,"last update":"Mon Oct 16 10:05:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5880000000000001,"5min":1.5289999999999999,"15min":1.4930000000000001},"min":{"1min":1.1439999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.3410000000000002,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.302},{"interface":"front","average":{"1min":1.647,"5min":1.603,"15min":1.569},"min":{"1min":1.282,"5min":1.1220000000000001,"15min":0.96799999999999997},"max":{"1min":2.0030000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":1.8}]}]},{"osd":3,"up_from":13,"seq":55834575050,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36648,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335192,"statfs":{"total":96636764160,"available":96599236608,"internally_reserved":0,"allocated":1339392,"data_stored":633669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:05:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.82499999999999996,"15min":0.85799999999999998},"min":{"1min":0.39900000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.318},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.89200000000000002,"15min":0.995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.29999999999999999},"max":{"1min":1.3540000000000001,"5min":1.488,"15min":1.708},"last":0.49099999999999999}]},{"osd":1,"last update":"Mon Oct 16 10:05:32 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.86899999999999999,"15min":0.93999999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.0270000000000001,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.98399999999999999,"15min":1.054},"min":{"1min":0.441,"5min":0.42099999999999999,"15min":0.39600000000000002},"max":{"1min":3.363,"5min":3.363,"15min":3.363},"last":0.36699999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:05:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.98099999999999998,"15min":1.054},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.069,"5min":1.093,"15min":1.165},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":4.1230000000000002,"5min":4.1230000000000002,"15min":4.1230000000000002},"last":0.70799999999999996}]},{"osd":4,"last update":"Mon Oct 16 10:05:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2270000000000001,"15min":1.2749999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":4.0650000000000004,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.2809999999999999,"15min":1.3220000000000001},"min":{"1min":0.72999999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":3.5550000000000002,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.81499999999999995}]},{"osd":5,"last update":"Mon Oct 16 10:05:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.2849999999999999,"15min":1.3180000000000001},"min":{"1min":0.81799999999999995,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":4.1829999999999998,"5min":4.1829999999999998,"15min":4.1829999999999998},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.28,"15min":1.3360000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.0129999999999999,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.73799999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:05:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.3640000000000001,"15min":1.4259999999999999},"min":{"1min":0.72199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":4.0069999999999997,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.355,"15min":1.429},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.96999999999999997}]},{"osd":7,"last update":"Mon Oct 16 10:05:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.375,"15min":1.4330000000000001},"min":{"1min":0.747,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":3.649,"5min":3.649,"15min":3.649},"last":1},{"interface":"front","average":{"1min":1.5,"5min":1.4830000000000001,"15min":1.5529999999999999},"min":{"1min":0.95699999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":3.8730000000000002,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":1.0369999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575049,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40804,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331036,"statfs":{"total":96636764160,"available":96594980864,"internally_reserved":0,"allocated":1331200,"data_stored":632639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.89800000000000002,"15min":0.878},"min":{"1min":0.35899999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.1829999999999998,"5min":5.1829999999999998,"15min":5.1829999999999998},"last":1.26},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.92900000000000005,"15min":1.028},"min":{"1min":0.50700000000000001,"5min":0.34999999999999998,"15min":0.28899999999999998},"max":{"1min":1.452,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.80900000000000005}]},{"osd":1,"last update":"Mon Oct 16 10:05:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.04,"15min":1.0389999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":3.6040000000000001,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":1.5900000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.016,"15min":1.1120000000000001},"min":{"1min":0.38300000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3999999999999999,"5min":1.792,"15min":2.6499999999999999},"last":1.0329999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.95199999999999996,"15min":1.0069999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.3260000000000001,"5min":1.776,"15min":2.5339999999999998},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.171,"15min":1.204},"min":{"1min":0.69999999999999996,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":3.6970000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":1.7230000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:05:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.321,"15min":1.3440000000000001},"min":{"1min":0.84399999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.345},{"interface":"front","average":{"1min":1.258,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.71899999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.262,"5min":3.262,"15min":3.262},"last":1.6459999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:05:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.367,"15min":1.3740000000000001},"min":{"1min":0.84099999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.9169999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":2.0259999999999998},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.3480000000000001,"15min":1.399},"min":{"1min":0.73699999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.774,"5min":1.956,"15min":1.956},"last":1.4259999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:05:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3280000000000001,"15min":1.345},"min":{"1min":0.91500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":3.169,"5min":3.169,"15min":3.169},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.4239999999999999,"15min":1.4419999999999999},"min":{"1min":0.88800000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":3.8340000000000001,"5min":3.8340000000000001,"15min":3.8340000000000001},"last":2.2549999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:05:35 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.429,"15min":1.4730000000000001},"min":{"1min":0.81200000000000006,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.907,"5min":2.0739999999999998,"15min":2.4620000000000002},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.391,"15min":1.4350000000000001},"min":{"1min":0.98799999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":3.3780000000000001,"5min":3.3780000000000001,"15min":3.3780000000000001},"last":1.774}]}]},{"osd":1,"up_from":13,"seq":55834575049,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54700,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317140,"statfs":{"total":96636764160,"available":96580751360,"internally_reserved":0,"allocated":1339392,"data_stored":632251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:05:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":0.875},"min":{"1min":0.47599999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.631},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.96099999999999997,"15min":0.99099999999999999},"min":{"1min":0.42299999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4339999999999999,"5min":1.798,"15min":1.798},"last":0.93899999999999995}]},{"osd":2,"last update":"Mon Oct 16 10:05:15 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.96799999999999997,"15min":0.98499999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.42299999999999999},"max":{"1min":1.5069999999999999,"5min":1.526,"15min":1.673},"last":1.339},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.151,"15min":1.2290000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.482,"5min":1.6419999999999999,"15min":1.7669999999999999},"last":1.476}]},{"osd":3,"last update":"Mon Oct 16 10:05:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.0469999999999999,"15min":1.129},"min":{"1min":0.442,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.873},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.026,"15min":1.099},"min":{"1min":0.70699999999999996,"5min":0.40699999999999997,"15min":0.36099999999999999},"max":{"1min":1.3779999999999999,"5min":1.6879999999999999,"15min":1.8879999999999999},"last":1.1220000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1879999999999999,"15min":1.236},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":2.0259999999999998},"last":1.03},{"interface":"front","average":{"1min":1.228,"5min":1.2629999999999999,"15min":1.361},"min":{"1min":0.67600000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":2.1320000000000001},"last":1.3169999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2729999999999999,"15min":1.3320000000000001},"min":{"1min":0.77300000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.696,"5min":1.7470000000000001,"15min":1.9359999999999999},"last":1.276},{"interface":"front","average":{"1min":1.26,"5min":1.27,"15min":1.327},"min":{"1min":0.77800000000000002,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.661,"5min":1.7869999999999999,"15min":2.0470000000000002},"last":1.2290000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3400000000000001,"15min":1.395},"min":{"1min":0.88200000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.325,"15min":1.3859999999999999},"min":{"1min":0.80800000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.718,"5min":1.851,"15min":2.1680000000000001},"last":1.4350000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.3640000000000001,"15min":1.4710000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.1699999999999999},"last":1.375},{"interface":"front","average":{"1min":1.361,"5min":1.4530000000000001,"15min":1.49},"min":{"1min":0.95199999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.8280000000000001,"5min":2.1829999999999998,"15min":2.1960000000000002},"last":1.419}]}]},{"osd":0,"up_from":13,"seq":55834575050,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41376,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330464,"statfs":{"total":96636764160,"available":96594395136,"internally_reserved":0,"allocated":1925120,"data_stored":1222459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.874,"15min":0.85999999999999999},"min":{"1min":0.43099999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.7589999999999999,"5min":2.766,"15min":2.766},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.98299999999999998,"15min":1.034},"min":{"1min":0.51300000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.571,"5min":3.96,"15min":3.96},"last":1.3109999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:05:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0580000000000001,"15min":1.044},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.0449999999999999,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.081,"15min":1.0820000000000001},"min":{"1min":0.54000000000000004,"5min":0.48999999999999999,"15min":0.46600000000000003},"max":{"1min":1.3500000000000001,"5min":4.4290000000000003,"15min":4.4290000000000003},"last":1.1719999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.0189999999999999,"15min":1.03},"min":{"1min":0.442,"5min":0.38,"15min":0.38},"max":{"1min":1.5920000000000001,"5min":4.7380000000000004,"15min":4.7380000000000004},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.123,"5min":1.204,"15min":1.2010000000000001},"min":{"1min":0.505,"5min":0.44,"15min":0.44},"max":{"1min":1.8759999999999999,"5min":5.1500000000000004,"15min":5.1500000000000004},"last":1.73}]},{"osd":4,"last update":"Mon Oct 16 10:05:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3280000000000001,"15min":1.3320000000000001},"min":{"1min":0.83399999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5920000000000001,"5min":4.1970000000000001,"15min":4.1970000000000001},"last":1.411},{"interface":"front","average":{"1min":1.181,"5min":1.2769999999999999,"15min":1.2869999999999999},"min":{"1min":0.81799999999999995,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.502,"5min":4.9630000000000001,"15min":4.9630000000000001},"last":1.1180000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:05:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.304,"15min":1.3140000000000001},"min":{"1min":0.76600000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.593,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.4159999999999999,"15min":1.403},"min":{"1min":0.89600000000000002,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.0190000000000001,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":1.6679999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:05:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.462,"15min":1.4410000000000001},"min":{"1min":0.84899999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.1030000000000002,"5min":3.8250000000000002,"15min":3.8250000000000002},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.4350000000000001,"15min":1.427},"min":{"1min":1.0309999999999999,"5min":0.85799999999999998,"15min":0.84999999999999998},"max":{"1min":1.998,"5min":4.0979999999999999,"15min":4.0979999999999999},"last":1.605}]},{"osd":7,"last update":"Mon Oct 16 10:05:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4890000000000001,"15min":1.482},"min":{"1min":0.89000000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.0750000000000002,"5min":4.3369999999999997,"15min":4.3369999999999997},"last":1.51},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.4259999999999999,"15min":1.4319999999999999},"min":{"1min":0.78900000000000003,"5min":0.755,"15min":0.755},"max":{"1min":1.7569999999999999,"5min":5.0570000000000004,"15min":5.0570000000000004},"last":1.375}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:05:54.349 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T10:05:54.349 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T10:05:54.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.607+0000 7fdc8413e640 1 Processor -- start 2023-10-16T10:05:54.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.607+0000 7fdc8413e640 1 -- start start 2023-10-16T10:05:54.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.609+0000 7fdc8413e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc7c0b1840 0x7fdc7c0b0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:54.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.609+0000 7fdc8413e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc7c0b07b0 con 0x7fdc7c0cdb00 2023-10-16T10:05:54.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.609+0000 7fdc8413e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc7c0ccb50 con 0x7fdc7c15f4c0 2023-10-16T10:05:54.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.609+0000 7fdc8413e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c0b1c90 con 0x7fdc7c0b1840 2023-10-16T10:05:54.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.609+0000 7fdc816d9640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdc7c0cdb00 0x7fdc7c0ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53694/0 (socket says 172.21.15.84:53694) 2023-10-16T10:05:54.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.609+0000 7fdc816d9640 1 -- 172.21.15.84:0/2389660490 learned_addr learned my addr 172.21.15.84:0/2389660490 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:54.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.610+0000 7fdc826db640 1 --2- 172.21.15.84:0/2389660490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc7c0b1840 0x7fdc7c0b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:54.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.610+0000 7fdc80ed8640 1 -- 172.21.15.84:0/2389660490 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2936196235 0 0) 0x7fdc7c0b07b0 con 0x7fdc7c0cdb00 2023-10-16T10:05:54.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.610+0000 7fdc80ed8640 1 -- 172.21.15.84:0/2389660490 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc68003520 con 0x7fdc7c0cdb00 2023-10-16T10:05:54.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.610+0000 7fdc80ed8640 1 -- 172.21.15.84:0/2389660490 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1490263529 0 0) 0x7fdc68003520 con 0x7fdc7c0cdb00 2023-10-16T10:05:54.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.610+0000 7fdc80ed8640 1 -- 172.21.15.84:0/2389660490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc7c0b1840 msgr2=0x7fdc7c0b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:54.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.610+0000 7fdc80ed8640 1 --2- 172.21.15.84:0/2389660490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc7c0b1840 0x7fdc7c0b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.610+0000 7fdc80ed8640 1 -- 172.21.15.84:0/2389660490 >> v1:172.21.15.157:6789/0 conn(0x7fdc7c15f4c0 legacy=0x7fdc7c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:54.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.610+0000 7fdc80ed8640 1 -- 172.21.15.84:0/2389660490 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c0b15f0 con 0x7fdc7c0cdb00 2023-10-16T10:05:54.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.610+0000 7fdc80ed8640 1 -- 172.21.15.84:0/2389660490 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fdc6c001670 con 0x7fdc7c0cdb00 2023-10-16T10:05:54.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.610+0000 7fdc80ed8640 1 -- 172.21.15.84:0/2389660490 >> v1:172.21.15.84:6789/0 conn(0x7fdc7c0cdb00 legacy=0x7fdc7c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:54.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.610+0000 7fdc80ed8640 1 --2- 172.21.15.84:0/2389660490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc680039a0 0x7fdc68003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:54.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.610+0000 7fdc80ed8640 1 --2- 172.21.15.84:0/2389660490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc7c0b1840 0x7fdc680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:54.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.610+0000 7fdc80ed8640 1 --2- 172.21.15.84:0/2389660490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc68004d00 0x7fdc68005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:54.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.611+0000 7fdc80ed8640 1 -- 172.21.15.84:0/2389660490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6800ba30 con 0x7fdc680039a0 2023-10-16T10:05:54.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.611+0000 7fdc80ed8640 1 -- 172.21.15.84:0/2389660490 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6800bba0 con 0x7fdc68004d00 2023-10-16T10:05:54.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.611+0000 7fdc80ed8640 1 -- 172.21.15.84:0/2389660490 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6800bd60 con 0x7fdc7c0b1840 2023-10-16T10:05:54.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.611+0000 7fdc81eda640 1 --2- 172.21.15.84:0/2389660490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc680039a0 0x7fdc68003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:54.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.611+0000 7fdc816d9640 1 --2- 172.21.15.84:0/2389660490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc7c0b1840 0x7fdc680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:54.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.611+0000 7fdc81eda640 1 -- 172.21.15.84:0/2389660490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc7c0b1840 msgr2=0x7fdc680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:54.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.611+0000 7fdc81eda640 1 --2- 172.21.15.84:0/2389660490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc7c0b1840 0x7fdc680047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.611+0000 7fdc826db640 1 --2- 172.21.15.84:0/2389660490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc68004d00 0x7fdc68005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:54.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.611+0000 7fdc81eda640 1 -- 172.21.15.84:0/2389660490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc68004d00 msgr2=0x7fdc68005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:54.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.611+0000 7fdc81eda640 1 --2- 172.21.15.84:0/2389660490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc68004d00 0x7fdc68005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.611+0000 7fdc81eda640 1 -- 172.21.15.84:0/2389660490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc68003520 con 0x7fdc680039a0 2023-10-16T10:05:54.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.613+0000 7fdc81eda640 1 --2- 172.21.15.84:0/2389660490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc680039a0 0x7fdc68003dc0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fdc78007fd0 tx=0x7fdc680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ce59db03a2ce77 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:54.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.613+0000 7fdc80ed8640 1 -- 172.21.15.84:0/2389660490 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc780052d0 con 0x7fdc680039a0 2023-10-16T10:05:54.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.613+0000 7fdc80ed8640 1 -- 172.21.15.84:0/2389660490 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7802e050 con 0x7fdc680039a0 2023-10-16T10:05:54.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.613+0000 7fdc80ed8640 1 -- 172.21.15.84:0/2389660490 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc78003640 con 0x7fdc680039a0 2023-10-16T10:05:54.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.614+0000 7fdc8413e640 1 -- 172.21.15.84:0/2389660490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc680039a0 msgr2=0x7fdc68003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:54.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.614+0000 7fdc8413e640 1 --2- 172.21.15.84:0/2389660490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc680039a0 0x7fdc68003dc0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fdc78007fd0 tx=0x7fdc680057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.614+0000 7fdc826db640 1 -- 172.21.15.84:0/2389660490 reap_dead start 2023-10-16T10:05:54.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.614+0000 7fdc8413e640 1 -- 172.21.15.84:0/2389660490 shutdown_connections 2023-10-16T10:05:54.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.614+0000 7fdc8413e640 1 -- 172.21.15.84:0/2389660490 >> 172.21.15.84:0/2389660490 conn(0x7fdc7c099ea0 msgr2=0x7fdc7c0d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:54.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.614+0000 7fdc8413e640 1 -- 172.21.15.84:0/2389660490 shutdown_connections 2023-10-16T10:05:54.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.614+0000 7fdc8413e640 1 -- 172.21.15.84:0/2389660490 wait complete. 2023-10-16T10:05:54.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.616+0000 7fdc8413e640 1 Processor -- start 2023-10-16T10:05:54.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.616+0000 7fdc8413e640 1 -- start start 2023-10-16T10:05:54.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.617+0000 7fdc8413e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc680039a0 0x7fdc7c0c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:54.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.617+0000 7fdc8413e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc7c0c9940 0x7fdc7c0c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:54.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.617+0000 7fdc8413e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc7c1f3bb0 0x7fdc7c1f4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:54.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.617+0000 7fdc8413e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c1f4570 con 0x7fdc680039a0 2023-10-16T10:05:54.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.617+0000 7fdc8413e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c0c88a0 con 0x7fdc7c0c9940 2023-10-16T10:05:54.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.617+0000 7fdc8413e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c0c8b80 con 0x7fdc7c1f3bb0 2023-10-16T10:05:54.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.617+0000 7fdc81eda640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc680039a0 0x7fdc7c0c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:54.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.617+0000 7fdc81eda640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc680039a0 0x7fdc7c0c75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60738/0 (socket says 172.21.15.84:60738) 2023-10-16T10:05:54.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.617+0000 7fdc81eda640 1 -- 172.21.15.84:0/2467697393 learned_addr learned my addr 172.21.15.84:0/2467697393 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:54.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.617+0000 7fdc81eda640 1 -- 172.21.15.84:0/2467697393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc7c1f3bb0 msgr2=0x7fdc7c1f4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.617+0000 7fdc81eda640 1 --2- 172.21.15.84:0/2467697393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc7c1f3bb0 0x7fdc7c1f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.618+0000 7fdc81eda640 1 -- 172.21.15.84:0/2467697393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc7c0c9940 msgr2=0x7fdc7c0c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.618+0000 7fdc81eda640 1 --2- 172.21.15.84:0/2467697393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc7c0c9940 0x7fdc7c0c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.618+0000 7fdc81eda640 1 -- 172.21.15.84:0/2467697393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c0c8e60 con 0x7fdc680039a0 2023-10-16T10:05:54.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.618+0000 7fdc81eda640 1 --2- 172.21.15.84:0/2467697393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc680039a0 0x7fdc7c0c75b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fdc7802bbc0 tx=0x7fdc78001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:54.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.618+0000 7fdc727fc640 1 -- 172.21.15.84:0/2467697393 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc78006e20 con 0x7fdc680039a0 2023-10-16T10:05:54.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.618+0000 7fdc727fc640 1 -- 172.21.15.84:0/2467697393 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc78000ed0 con 0x7fdc680039a0 2023-10-16T10:05:54.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.618+0000 7fdc727fc640 1 -- 172.21.15.84:0/2467697393 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc78007470 con 0x7fdc680039a0 2023-10-16T10:05:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.618+0000 7fdc8413e640 1 -- 172.21.15.84:0/2467697393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7c0c9140 con 0x7fdc680039a0 2023-10-16T10:05:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.618+0000 7fdc8413e640 1 -- 172.21.15.84:0/2467697393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7c152ff0 con 0x7fdc680039a0 2023-10-16T10:05:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.620+0000 7fdc8413e640 1 -- 172.21.15.84:0/2467697393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7c04e160 con 0x7fdc680039a0 2023-10-16T10:05:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.621+0000 7fdc727fc640 1 -- 172.21.15.84:0/2467697393 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fdc780075d0 con 0x7fdc680039a0 2023-10-16T10:05:54.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.621+0000 7fdc727fc640 1 --2- 172.21.15.84:0/2467697393 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fdc6006af00 0x7fdc6006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:54.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.622+0000 7fdc727fc640 1 -- 172.21.15.84:0/2467697393 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc780ab420 con 0x7fdc680039a0 2023-10-16T10:05:54.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.622+0000 7fdc816d9640 1 --2- 172.21.15.84:0/2467697393 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fdc6006af00 0x7fdc6006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:54.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.622+0000 7fdc816d9640 1 --2- 172.21.15.84:0/2467697393 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fdc6006af00 0x7fdc6006d3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c001050 tx=0x7fdc6c000f40 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:54.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.626+0000 7fdc727fc640 1 -- 172.21.15.84:0/2467697393 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdc78076360 con 0x7fdc680039a0 2023-10-16T10:05:54.882 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:54.879+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:54.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.888+0000 7fdc8413e640 1 -- 172.21.15.84:0/2467697393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdc7c0d1a30 con 0x7fdc680039a0 2023-10-16T10:05:54.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.892+0000 7fdc727fc640 1 -- 172.21.15.84:0/2467697393 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+147 (secure 0 0 0) 0x7fdc78079980 con 0x7fdc680039a0 2023-10-16T10:05:54.894 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2023-10-16T10:05:54.894 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ== 2023-10-16T10:05:54.895 INFO:teuthology.orchestra.run.smithi084.stdout: caps mds = "allow" 2023-10-16T10:05:54.895 INFO:teuthology.orchestra.run.smithi084.stdout: caps mon = "allow r" 2023-10-16T10:05:54.895 INFO:teuthology.orchestra.run.smithi084.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T10:05:54.895 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-16T10:05:54.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.895+0000 7fdc8413e640 1 -- 172.21.15.84:0/2467697393 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fdc6006af00 msgr2=0x7fdc6006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:54.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.895+0000 7fdc8413e640 1 --2- 172.21.15.84:0/2467697393 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fdc6006af00 0x7fdc6006d3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c001050 tx=0x7fdc6c000f40 comp rx=0 tx=0).stop 2023-10-16T10:05:54.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.895+0000 7fdc8413e640 1 -- 172.21.15.84:0/2467697393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc680039a0 msgr2=0x7fdc7c0c75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:54.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.895+0000 7fdc8413e640 1 --2- 172.21.15.84:0/2467697393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc680039a0 0x7fdc7c0c75b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fdc7802bbc0 tx=0x7fdc78001170 comp rx=0 tx=0).stop 2023-10-16T10:05:54.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.896+0000 7fdc8413e640 1 -- 172.21.15.84:0/2467697393 shutdown_connections 2023-10-16T10:05:54.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.896+0000 7fdc8413e640 1 --2- 172.21.15.84:0/2467697393 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fdc6006af00 0x7fdc6006d3c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.896+0000 7fdc8413e640 1 --2- 172.21.15.84:0/2467697393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc7c1f3bb0 0x7fdc7c1f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.896+0000 7fdc8413e640 1 --2- 172.21.15.84:0/2467697393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc7c0c9940 0x7fdc7c0c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.896+0000 7fdc8413e640 1 --2- 172.21.15.84:0/2467697393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc680039a0 0x7fdc7c0c75b0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:54.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.896+0000 7fdc8413e640 1 -- 172.21.15.84:0/2467697393 >> 172.21.15.84:0/2467697393 conn(0x7fdc7c099ea0 msgr2=0x7fdc7c15a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:54.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.896+0000 7fdc8413e640 1 -- 172.21.15.84:0/2467697393 shutdown_connections 2023-10-16T10:05:54.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:54.896+0000 7fdc8413e640 1 -- 172.21.15.84:0/2467697393 wait complete. 2023-10-16T10:05:54.917 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:05:55.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.132+0000 7f2174f5a640 1 Processor -- start 2023-10-16T10:05:55.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.132+0000 7f2174f5a640 1 -- start start 2023-10-16T10:05:55.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.134+0000 7f2174f5a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2170159f00 0x7f21700b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:55.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.134+0000 7f2174f5a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f217015d880 0x7f21700b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:55.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.134+0000 7f2174f5a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217006f7f0 con 0x7f2170159b20 2023-10-16T10:05:55.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.134+0000 7f2174f5a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f217006c5e0 con 0x7f217015d880 2023-10-16T10:05:55.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.134+0000 7f2174f5a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217006c8c0 con 0x7f2170159f00 2023-10-16T10:05:55.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.134+0000 7f216ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f217015d880 0x7f21700b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:55.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.134+0000 7f216ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f217015d880 0x7f21700b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60740/0 (socket says 172.21.15.84:60740) 2023-10-16T10:05:55.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.134+0000 7f216ed9d640 1 -- 172.21.15.84:0/4282191959 learned_addr learned my addr 172.21.15.84:0/4282191959 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:55.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.134+0000 7f2165d9b640 1 --2- 172.21.15.84:0/4282191959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2170159f00 0x7f21700b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:55.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.135+0000 7f216ed9d640 1 -- 172.21.15.84:0/4282191959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2170159f00 msgr2=0x7f21700b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:55.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.135+0000 7f216ed9d640 1 --2- 172.21.15.84:0/4282191959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2170159f00 0x7f21700b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:55.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.135+0000 7f216ed9d640 1 -- 172.21.15.84:0/4282191959 >> v1:172.21.15.157:6789/0 conn(0x7f2170159b20 legacy=0x7f217006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:55.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.135+0000 7f216ed9d640 1 -- 172.21.15.84:0/4282191959 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217006cba0 con 0x7f217015d880 2023-10-16T10:05:55.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.136+0000 7f216ed9d640 1 --2- 172.21.15.84:0/4282191959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f217015d880 0x7f21700b0950 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f2168009e50 tx=0x7f216802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f405aa95724b38 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:55.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.136+0000 7f216dd9b640 1 -- 172.21.15.84:0/4282191959 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216802ca70 con 0x7f217015d880 2023-10-16T10:05:55.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.136+0000 7f216dd9b640 1 -- 172.21.15.84:0/4282191959 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216802cbd0 con 0x7f217015d880 2023-10-16T10:05:55.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.136+0000 7f216dd9b640 1 -- 172.21.15.84:0/4282191959 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21680343f0 con 0x7f217015d880 2023-10-16T10:05:55.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.137+0000 7f2174f5a640 1 -- 172.21.15.84:0/4282191959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f217015d880 msgr2=0x7f21700b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:55.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.137+0000 7f2174f5a640 1 --2- 172.21.15.84:0/4282191959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f217015d880 0x7f21700b0950 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f2168009e50 tx=0x7f216802bed0 comp rx=0 tx=0).stop 2023-10-16T10:05:55.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.137+0000 7f2174f5a640 1 -- 172.21.15.84:0/4282191959 shutdown_connections 2023-10-16T10:05:55.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.137+0000 7f2174f5a640 1 --2- 172.21.15.84:0/4282191959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f217015d880 0x7f21700b0950 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:55.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.137+0000 7f2174f5a640 1 --2- 172.21.15.84:0/4282191959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2170159f00 0x7f21700b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:55.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.137+0000 7f2174f5a640 1 -- 172.21.15.84:0/4282191959 >> 172.21.15.84:0/4282191959 conn(0x7f21700cb410 msgr2=0x7f21700cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:55.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.137+0000 7f2174f5a640 1 -- 172.21.15.84:0/4282191959 shutdown_connections 2023-10-16T10:05:55.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.137+0000 7f2174f5a640 1 -- 172.21.15.84:0/4282191959 wait complete. 2023-10-16T10:05:55.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.139+0000 7f2174f5a640 1 Processor -- start 2023-10-16T10:05:55.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.139+0000 7f2174f5a640 1 -- start start 2023-10-16T10:05:55.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.140+0000 7f2174f5a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2170159b20 0x7f21700c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:55.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.140+0000 7f2174f5a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2170159f00 0x7f2170153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:55.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.140+0000 7f2174f5a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f217015d880 0x7f2170154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:55.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.140+0000 7f2174f5a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21700c5fa0 con 0x7f2170159b20 2023-10-16T10:05:55.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.140+0000 7f2174f5a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21700c61d0 con 0x7f2170159f00 2023-10-16T10:05:55.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.140+0000 7f2174f5a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21700c4e00 con 0x7f217015d880 2023-10-16T10:05:55.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.140+0000 7f216e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2170159b20 0x7f21700c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:55.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.140+0000 7f216e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2170159b20 0x7f21700c5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60750/0 (socket says 172.21.15.84:60750) 2023-10-16T10:05:55.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.140+0000 7f216e59c640 1 -- 172.21.15.84:0/3466810739 learned_addr learned my addr 172.21.15.84:0/3466810739 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:55.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.141+0000 7f2165d9b640 1 --2- 172.21.15.84:0/3466810739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2170159f00 0x7f2170153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:55.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.141+0000 7f216ed9d640 1 --2- 172.21.15.84:0/3466810739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f217015d880 0x7f2170154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:55.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.141+0000 7f216e59c640 1 -- 172.21.15.84:0/3466810739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f217015d880 msgr2=0x7f2170154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:55.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.141+0000 7f216e59c640 1 --2- 172.21.15.84:0/3466810739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f217015d880 0x7f2170154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:55.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.141+0000 7f216e59c640 1 -- 172.21.15.84:0/3466810739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2170159f00 msgr2=0x7f2170153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:55.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.141+0000 7f216e59c640 1 --2- 172.21.15.84:0/3466810739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2170159f00 0x7f2170153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:55.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.141+0000 7f216e59c640 1 -- 172.21.15.84:0/3466810739 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168009b30 con 0x7f2170159b20 2023-10-16T10:05:55.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.141+0000 7f2165d9b640 1 --2- 172.21.15.84:0/3466810739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2170159f00 0x7f2170153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:55.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.141+0000 7f216e59c640 1 --2- 172.21.15.84:0/3466810739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2170159b20 0x7f21700c5a60 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f216000b3f0 tx=0x7f216000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:55.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.141+0000 7f21677fe640 1 -- 172.21.15.84:0/3466810739 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160011830 con 0x7f2170159b20 2023-10-16T10:05:55.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.141+0000 7f21677fe640 1 -- 172.21.15.84:0/3466810739 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2160011990 con 0x7f2170159b20 2023-10-16T10:05:55.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.141+0000 7f21677fe640 1 -- 172.21.15.84:0/3466810739 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160011d40 con 0x7f2170159b20 2023-10-16T10:05:55.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.142+0000 7f2174f5a640 1 -- 172.21.15.84:0/3466810739 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21700c50e0 con 0x7f2170159b20 2023-10-16T10:05:55.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.142+0000 7f2174f5a640 1 -- 172.21.15.84:0/3466810739 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217015b770 con 0x7f2170159b20 2023-10-16T10:05:55.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.143+0000 7f2174f5a640 1 -- 172.21.15.84:0/3466810739 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217004e160 con 0x7f2170159b20 2023-10-16T10:05:55.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.144+0000 7f21677fe640 1 -- 172.21.15.84:0/3466810739 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f216001a400 con 0x7f2170159b20 2023-10-16T10:05:55.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.145+0000 7f21677fe640 1 --2- 172.21.15.84:0/3466810739 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f215006af00 0x7f215006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:55.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.145+0000 7f21677fe640 1 -- 172.21.15.84:0/3466810739 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21600894e0 con 0x7f2170159b20 2023-10-16T10:05:55.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.145+0000 7f2165d9b640 1 --2- 172.21.15.84:0/3466810739 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f215006af00 0x7f215006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:55.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.146+0000 7f2165d9b640 1 --2- 172.21.15.84:0/3466810739 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f215006af00 0x7f215006d3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f217015b660 tx=0x7f2158008040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:55.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.150+0000 7f21677fe640 1 -- 172.21.15.84:0/3466810739 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2160054530 con 0x7f2170159b20 2023-10-16T10:05:55.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.383+0000 7f2174f5a640 1 -- 172.21.15.84:0/3466810739 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21700b10b0 con 0x7f2170159b20 2023-10-16T10:05:55.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.384+0000 7f21677fe640 1 -- 172.21.15.84:0/3466810739 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f2160057b50 con 0x7f2170159b20 2023-10-16T10:05:55.386 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:05:55.386 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3458903351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3458903351},{"type":"v1","addr":"172.21.15.84:6837","nonce":3458903351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4002935522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4002935522},{"type":"v1","addr":"172.21.15.157:6833","nonce":4002935522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5809,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1890551305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1890551305},{"type":"v1","addr":"172.21.15.157:6835","nonce":1890551305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:05:48.809508+0000","modified":"2023-10-16T10:05:51.856862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7521":{"gid":7521,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/3621445269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3621445269},{"type":"v1","addr":"172.21.15.84:6836","nonce":3621445269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-16T10:05:55.387 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 53 2023-10-16T10:05:55.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.387+0000 7f2174f5a640 1 -- 172.21.15.84:0/3466810739 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f215006af00 msgr2=0x7f215006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:55.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.387+0000 7f2174f5a640 1 --2- 172.21.15.84:0/3466810739 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f215006af00 0x7f215006d3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f217015b660 tx=0x7f2158008040 comp rx=0 tx=0).stop 2023-10-16T10:05:55.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.387+0000 7f2174f5a640 1 -- 172.21.15.84:0/3466810739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2170159b20 msgr2=0x7f21700c5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:55.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.387+0000 7f2174f5a640 1 --2- 172.21.15.84:0/3466810739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2170159b20 0x7f21700c5a60 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f216000b3f0 tx=0x7f216000b8b0 comp rx=0 tx=0).stop 2023-10-16T10:05:55.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.387+0000 7f2174f5a640 1 -- 172.21.15.84:0/3466810739 shutdown_connections 2023-10-16T10:05:55.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.387+0000 7f2174f5a640 1 --2- 172.21.15.84:0/3466810739 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f215006af00 0x7f215006d3c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:55.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.387+0000 7f2174f5a640 1 --2- 172.21.15.84:0/3466810739 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f217015d880 0x7f2170154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:55.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.387+0000 7f2174f5a640 1 --2- 172.21.15.84:0/3466810739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2170159f00 0x7f2170153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:55.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.387+0000 7f2174f5a640 1 --2- 172.21.15.84:0/3466810739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2170159b20 0x7f21700c5a60 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:55.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.387+0000 7f2174f5a640 1 -- 172.21.15.84:0/3466810739 >> 172.21.15.84:0/3466810739 conn(0x7f21700cb410 msgr2=0x7f21700cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:55.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.387+0000 7f2174f5a640 1 -- 172.21.15.84:0/3466810739 shutdown_connections 2023-10-16T10:05:55.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:55.387+0000 7f2174f5a640 1 -- 172.21.15.84:0/3466810739 wait complete. 2023-10-16T10:05:55.809 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T10:05:48.809508+0000', 'modified': '2023-10-16T10:05:51.856862+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7521}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7521': {'gid': 7521, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.84:6836/3621445269', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6834', 'nonce': 3621445269}, {'type': 'v1', 'addr': '172.21.15.84:6836', 'nonce': 3621445269}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T10:05:55.810 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T10:05:55.811 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-16T10:05:55.980 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:55.977+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:56.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.034+0000 7ffbadce7640 1 Processor -- start 2023-10-16T10:05:56.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.034+0000 7ffbadce7640 1 -- start start 2023-10-16T10:05:56.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.036+0000 7ffbadce7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffba815ad70 0x7ffba8069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.036+0000 7ffbadce7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba80c9ca0 0x7ffba806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.036+0000 7ffbadce7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba80cb030 con 0x7ffba80ca3b0 2023-10-16T10:05:56.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.036+0000 7ffbadce7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba80c91c0 con 0x7ffba815ad70 2023-10-16T10:05:56.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.036+0000 7ffbadce7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80c94a0 con 0x7ffba80c9ca0 2023-10-16T10:05:56.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.036+0000 7ffba7fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ffba80ca3b0 0x7ffba80ca840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35226/0 (socket says 172.21.15.84:35226) 2023-10-16T10:05:56.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.036+0000 7ffba7fff640 1 -- 172.21.15.84:0/2564784800 learned_addr learned my addr 172.21.15.84:0/2564784800 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:56.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.037+0000 7ffba77fe640 1 --2- 172.21.15.84:0/2564784800 >> v2:172.21.15.157:3300/0 conn(0x7ffba815ad70 0x7ffba8069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.037+0000 7ffb9ffff640 1 --2- 172.21.15.84:0/2564784800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba80c9ca0 0x7ffba806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.037+0000 7ffba6ffd640 1 -- 172.21.15.84:0/2564784800 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3997085879 0 0) 0x7ffba80cb030 con 0x7ffba80ca3b0 2023-10-16T10:05:56.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.037+0000 7ffba6ffd640 1 -- 172.21.15.84:0/2564784800 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb94003520 con 0x7ffba80ca3b0 2023-10-16T10:05:56.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.037+0000 7ffba6ffd640 1 -- 172.21.15.84:0/2564784800 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2561331749 0 0) 0x7ffb94003520 con 0x7ffba80ca3b0 2023-10-16T10:05:56.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.037+0000 7ffba6ffd640 1 -- 172.21.15.84:0/2564784800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba80c9ca0 msgr2=0x7ffba806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:56.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.037+0000 7ffba6ffd640 1 --2- 172.21.15.84:0/2564784800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba80c9ca0 0x7ffba806a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.037+0000 7ffba6ffd640 1 -- 172.21.15.84:0/2564784800 >> v2:172.21.15.157:3300/0 conn(0x7ffba815ad70 msgr2=0x7ffba8069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:56.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.037+0000 7ffba6ffd640 1 --2- 172.21.15.84:0/2564784800 >> v2:172.21.15.157:3300/0 conn(0x7ffba815ad70 0x7ffba8069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.037+0000 7ffba6ffd640 1 -- 172.21.15.84:0/2564784800 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba80c9780 con 0x7ffba80ca3b0 2023-10-16T10:05:56.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.037+0000 7ffba6ffd640 1 -- 172.21.15.84:0/2564784800 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ffba00019f0 con 0x7ffba80ca3b0 2023-10-16T10:05:56.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.037+0000 7ffba6ffd640 1 -- 172.21.15.84:0/2564784800 >> v1:172.21.15.84:6789/0 conn(0x7ffba80ca3b0 legacy=0x7ffba80ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:56.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.037+0000 7ffba6ffd640 1 --2- 172.21.15.84:0/2564784800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb94003970 0x7ffb94003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.037+0000 7ffba6ffd640 1 --2- 172.21.15.84:0/2564784800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb940047c0 0x7ffb94004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.037+0000 7ffba6ffd640 1 --2- 172.21.15.84:0/2564784800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba80c9ca0 0x7ffb94005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.037+0000 7ffba6ffd640 1 -- 172.21.15.84:0/2564784800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80c91c0 con 0x7ffb940047c0 2023-10-16T10:05:56.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.037+0000 7ffba6ffd640 1 -- 172.21.15.84:0/2564784800 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80c94a0 con 0x7ffb94003970 2023-10-16T10:05:56.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.037+0000 7ffba6ffd640 1 -- 172.21.15.84:0/2564784800 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9400b980 con 0x7ffba80c9ca0 2023-10-16T10:05:56.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.038+0000 7ffb9ffff640 1 --2- 172.21.15.84:0/2564784800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb940047c0 0x7ffb94004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.038+0000 7ffba77fe640 1 --2- 172.21.15.84:0/2564784800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb94003970 0x7ffb94003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.038+0000 7ffb9ffff640 1 -- 172.21.15.84:0/2564784800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba80c9ca0 msgr2=0x7ffb94005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:56.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.038+0000 7ffb9ffff640 1 --2- 172.21.15.84:0/2564784800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba80c9ca0 0x7ffb94005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.038+0000 7ffb9ffff640 1 -- 172.21.15.84:0/2564784800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb94003970 msgr2=0x7ffb94003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:56.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.038+0000 7ffb9ffff640 1 --2- 172.21.15.84:0/2564784800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb94003970 0x7ffb94003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.038+0000 7ffb9ffff640 1 -- 172.21.15.84:0/2564784800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb94003520 con 0x7ffb940047c0 2023-10-16T10:05:56.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.039+0000 7ffb9ffff640 1 --2- 172.21.15.84:0/2564784800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb940047c0 0x7ffb94004c40 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009ac0 tx=0x7ffb940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c62ec5573bfe48 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:56.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.039+0000 7ffba6ffd640 1 -- 172.21.15.84:0/2564784800 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9002c6d0 con 0x7ffb940047c0 2023-10-16T10:05:56.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.040+0000 7ffba6ffd640 1 -- 172.21.15.84:0/2564784800 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9002c830 con 0x7ffb940047c0 2023-10-16T10:05:56.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.040+0000 7ffba6ffd640 1 -- 172.21.15.84:0/2564784800 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9002cbc0 con 0x7ffb940047c0 2023-10-16T10:05:56.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.040+0000 7ffbadce7640 1 -- 172.21.15.84:0/2564784800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb940047c0 msgr2=0x7ffb94004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:56.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.040+0000 7ffbadce7640 1 --2- 172.21.15.84:0/2564784800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb940047c0 0x7ffb94004c40 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009ac0 tx=0x7ffb940057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.040+0000 7ffba7fff640 1 -- 172.21.15.84:0/2564784800 reap_dead start 2023-10-16T10:05:56.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.040+0000 7ffbadce7640 1 -- 172.21.15.84:0/2564784800 shutdown_connections 2023-10-16T10:05:56.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.040+0000 7ffbadce7640 1 -- 172.21.15.84:0/2564784800 >> 172.21.15.84:0/2564784800 conn(0x7ffba80ce9b0 msgr2=0x7ffba80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:56.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.040+0000 7ffbadce7640 1 -- 172.21.15.84:0/2564784800 shutdown_connections 2023-10-16T10:05:56.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.040+0000 7ffbadce7640 1 -- 172.21.15.84:0/2564784800 wait complete. 2023-10-16T10:05:56.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.042+0000 7ffbadce7640 1 Processor -- start 2023-10-16T10:05:56.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.042+0000 7ffbadce7640 1 -- start start 2023-10-16T10:05:56.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.043+0000 7ffbadce7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb940047c0 0x7ffba80ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.043+0000 7ffbadce7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba80ac890 0x7ffba8151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.043+0000 7ffbadce7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba80c75c0 0x7ffba80c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.044+0000 7ffbadce7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80c7f80 con 0x7ffb940047c0 2023-10-16T10:05:56.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.044+0000 7ffbadce7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80ae3c0 con 0x7ffba80ac890 2023-10-16T10:05:56.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.044+0000 7ffbadce7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba81524c0 con 0x7ffba80c75c0 2023-10-16T10:05:56.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.044+0000 7ffba77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb940047c0 0x7ffba80ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.044+0000 7ffba77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb940047c0 0x7ffba80ade80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54428/0 (socket says 172.21.15.84:54428) 2023-10-16T10:05:56.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.044+0000 7ffba77fe640 1 -- 172.21.15.84:0/2697496159 learned_addr learned my addr 172.21.15.84:0/2697496159 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:56.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.044+0000 7ffba77fe640 1 -- 172.21.15.84:0/2697496159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba80c75c0 msgr2=0x7ffba80c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:56.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.044+0000 7ffba77fe640 1 --2- 172.21.15.84:0/2697496159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba80c75c0 0x7ffba80c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.044+0000 7ffba77fe640 1 -- 172.21.15.84:0/2697496159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba80ac890 msgr2=0x7ffba8151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:56.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.044+0000 7ffba77fe640 1 --2- 172.21.15.84:0/2697496159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba80ac890 0x7ffba8151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.044+0000 7ffba77fe640 1 -- 172.21.15.84:0/2697496159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba80acde0 con 0x7ffb940047c0 2023-10-16T10:05:56.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.044+0000 7ffba77fe640 1 --2- 172.21.15.84:0/2697496159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb940047c0 0x7ffba80ade80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ffb980108e0 tx=0x7ffb98010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:56.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.045+0000 7ffba4ff9640 1 -- 172.21.15.84:0/2697496159 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9800eb60 con 0x7ffb940047c0 2023-10-16T10:05:56.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.045+0000 7ffba4ff9640 1 -- 172.21.15.84:0/2697496159 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9800ecc0 con 0x7ffb940047c0 2023-10-16T10:05:56.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.045+0000 7ffbadce7640 1 -- 172.21.15.84:0/2697496159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba80ad0c0 con 0x7ffb940047c0 2023-10-16T10:05:56.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.045+0000 7ffba4ff9640 1 -- 172.21.15.84:0/2697496159 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9801c400 con 0x7ffb940047c0 2023-10-16T10:05:56.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.045+0000 7ffbadce7640 1 -- 172.21.15.84:0/2697496159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba80ad680 con 0x7ffb940047c0 2023-10-16T10:05:56.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.047+0000 7ffba4ff9640 1 -- 172.21.15.84:0/2697496159 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7ffb9801c560 con 0x7ffb940047c0 2023-10-16T10:05:56.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.048+0000 7ffba4ff9640 1 --2- 172.21.15.84:0/2697496159 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7ffb8806af00 0x7ffb8806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.048+0000 7ffba4ff9640 1 -- 172.21.15.84:0/2697496159 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb98089e30 con 0x7ffb940047c0 2023-10-16T10:05:56.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.048+0000 7ffb9ffff640 1 --2- 172.21.15.84:0/2697496159 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7ffb8806af00 0x7ffb8806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.049+0000 7ffbb087f740 1 Processor -- start 2023-10-16T10:05:56.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.049+0000 7ffb9ffff640 1 --2- 172.21.15.84:0/2697496159 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7ffb8806af00 0x7ffb8806d3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffb90036000 tx=0x7ffb9002e000 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:56.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.049+0000 7ffbb087f740 1 -- start start 2023-10-16T10:05:56.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.050+0000 7ffbb087f740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560b7d15ff80 0x560b7d132e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.050+0000 7ffbb087f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b7d160360 0x560b7d162d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.050+0000 7ffbb087f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d1333c0 0x560b7d163250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.050+0000 7ffbb087f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b7d0537a0 con 0x560b7d1333c0 2023-10-16T10:05:56.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.050+0000 7ffbb087f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b7d0a5740 con 0x560b7d160360 2023-10-16T10:05:56.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.050+0000 7ffbb087f740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b7d0a2e30 con 0x560b7d15ff80 2023-10-16T10:05:56.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.050+0000 7ffba7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d1333c0 0x560b7d163250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.050+0000 7ffba7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d1333c0 0x560b7d163250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54436/0 (socket says 172.21.15.84:54436) 2023-10-16T10:05:56.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.050+0000 7ffba7fff640 1 -- 172.21.15.84:0/201659765 learned_addr learned my addr 172.21.15.84:0/201659765 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:56.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.051+0000 7ffba7fff640 1 -- 172.21.15.84:0/201659765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560b7d15ff80 msgr2=0x560b7d132e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:56.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.051+0000 7ffba77fe640 1 --2- 172.21.15.84:0/201659765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560b7d15ff80 0x560b7d132e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.051+0000 7ffb9ffff640 1 --2- 172.21.15.84:0/201659765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b7d160360 0x560b7d162d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.051+0000 7ffba7fff640 1 --2- 172.21.15.84:0/201659765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560b7d15ff80 0x560b7d132e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.051+0000 7ffba7fff640 1 -- 172.21.15.84:0/201659765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b7d160360 msgr2=0x560b7d162d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:56.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.051+0000 7ffba7fff640 1 --2- 172.21.15.84:0/201659765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b7d160360 0x560b7d162d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.051+0000 7ffba7fff640 1 -- 172.21.15.84:0/201659765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b7d062910 con 0x560b7d1333c0 2023-10-16T10:05:56.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.051+0000 7ffba7fff640 1 --2- 172.21.15.84:0/201659765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d1333c0 0x560b7d163250 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ffba80cfa90 tx=0x7ffba0002140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66d6e335b094fc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:56.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.051+0000 7ffb9d7fa640 1 -- 172.21.15.84:0/201659765 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba0007cd0 con 0x560b7d1333c0 2023-10-16T10:05:56.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.051+0000 7ffb9d7fa640 1 -- 172.21.15.84:0/201659765 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba000c840 con 0x560b7d1333c0 2023-10-16T10:05:56.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.051+0000 7ffb9d7fa640 1 -- 172.21.15.84:0/201659765 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba000cbf0 con 0x560b7d1333c0 2023-10-16T10:05:56.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.052+0000 7ffbb087f740 1 -- 172.21.15.84:0/201659765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d1333c0 msgr2=0x560b7d163250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:56.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.052+0000 7ffbb087f740 1 --2- 172.21.15.84:0/201659765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d1333c0 0x560b7d163250 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ffba80cfa90 tx=0x7ffba0002140 comp rx=0 tx=0).stop 2023-10-16T10:05:56.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.052+0000 7ffbb087f740 1 -- 172.21.15.84:0/201659765 shutdown_connections 2023-10-16T10:05:56.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.052+0000 7ffbb087f740 1 --2- 172.21.15.84:0/201659765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d1333c0 0x560b7d163250 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.052+0000 7ffbb087f740 1 --2- 172.21.15.84:0/201659765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b7d160360 0x560b7d162d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.052+0000 7ffbb087f740 1 --2- 172.21.15.84:0/201659765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560b7d15ff80 0x560b7d132e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.052+0000 7ffbb087f740 1 -- 172.21.15.84:0/201659765 >> 172.21.15.84:0/201659765 conn(0x560b7d135190 msgr2=0x560b7d17dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:56.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.052+0000 7ffbb087f740 1 -- 172.21.15.84:0/201659765 shutdown_connections 2023-10-16T10:05:56.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.052+0000 7ffbb087f740 1 -- 172.21.15.84:0/201659765 wait complete. 2023-10-16T10:05:56.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.053+0000 7ffbb087f740 1 -- start start 2023-10-16T10:05:56.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.053+0000 7ffbb087f740 1 Processor -- start 2023-10-16T10:05:56.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.054+0000 7ffbb087f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b7d15ff80 0x560b7d178340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.054+0000 7ffbb087f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d160360 0x560b7d178880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.054+0000 7ffbb087f740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560b7d14dc90 0x560b7d165a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.054+0000 7ffbb087f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b7d09e720 con 0x560b7d160360 2023-10-16T10:05:56.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.054+0000 7ffbb087f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b7d03c970 con 0x560b7d15ff80 2023-10-16T10:05:56.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.054+0000 7ffbb087f740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b7d03c610 con 0x560b7d14dc90 2023-10-16T10:05:56.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.054+0000 7ffbb087f740 10 client.0 _ll_register_callbacks cb 0x560b7d05e9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:05:56.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.055+0000 7ffb9ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d160360 0x560b7d178880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.055+0000 7ffb9ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d160360 0x560b7d178880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54448/0 (socket says 172.21.15.84:54448) 2023-10-16T10:05:56.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.055+0000 7ffb9ffff640 1 -- 172.21.15.84:0/2365092716 learned_addr learned my addr 172.21.15.84:0/2365092716 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:56.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.055+0000 7ffba7fff640 1 --2- 172.21.15.84:0/2365092716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560b7d14dc90 0x560b7d165a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.055+0000 7ffb9ffff640 1 -- 172.21.15.84:0/2365092716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560b7d14dc90 msgr2=0x560b7d165a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:56.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.055+0000 7ffb9ffff640 1 --2- 172.21.15.84:0/2365092716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560b7d14dc90 0x560b7d165a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.055+0000 7ffb9ffff640 1 -- 172.21.15.84:0/2365092716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b7d15ff80 msgr2=0x560b7d178340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:56.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.055+0000 7ffb9ffff640 1 --2- 172.21.15.84:0/2365092716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b7d15ff80 0x560b7d178340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.055+0000 7ffb9ffff640 1 -- 172.21.15.84:0/2365092716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba80c9780 con 0x560b7d160360 2023-10-16T10:05:56.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.055+0000 7ffb9ffff640 1 --2- 172.21.15.84:0/2365092716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d160360 0x560b7d178880 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ffb9002c3a0 tx=0x7ffb900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9391685a11e951 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:56.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.055+0000 7ffb867fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T10:05:56.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.056+0000 7ffb867fc640 1 -- 172.21.15.84:0/2365092716 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90047070 con 0x560b7d160360 2023-10-16T10:05:56.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.056+0000 7ffb867fc640 1 -- 172.21.15.84:0/2365092716 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb9002cdb0 con 0x560b7d160360 2023-10-16T10:05:56.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.056+0000 7ffbb087f740 1 -- 172.21.15.84:0/2365092716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b7d0c5160 con 0x560b7d160360 2023-10-16T10:05:56.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.056+0000 7ffb867fc640 1 -- 172.21.15.84:0/2365092716 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb90005990 con 0x560b7d160360 2023-10-16T10:05:56.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.056+0000 7ffbb087f740 10 client.7650 fetch_fsmap learned FSMap version 53 2023-10-16T10:05:56.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.056+0000 7ffbb087f740 1 -- 172.21.15.84:0/2365092716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x560b7d09cda0 con 0x560b7d160360 2023-10-16T10:05:56.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.056+0000 7ffb867fc640 1 -- 172.21.15.84:0/2365092716 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b7d0c5160 con 0x560b7d160360 2023-10-16T10:05:56.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.056+0000 7ffb867fc640 1 -- 172.21.15.84:0/2365092716 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ffb90005af0 con 0x560b7d160360 2023-10-16T10:05:56.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.056+0000 7ffbb087f740 10 client.7650 fetch_fsmap finished waiting for FSMap version 53 2023-10-16T10:05:56.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.056+0000 7ffbb087f740 10 client.7650 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T10:05:56.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.056+0000 7ffbb087f740 20 client.7650 populate_metadata read hostname 'smithi084' 2023-10-16T10:05:56.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.056+0000 7ffbb087f740 1 --2- 172.21.15.84:0/2365092716 >> [v2:172.21.15.84:6834/3621445269,v1:172.21.15.84:6836/3621445269] conn(0x560b7d14fdc0 0x560b7d150270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.056+0000 7ffbb087f740 4 client.7650 mds_command: new command op to 7521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T10:05:56.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.056+0000 7ffbb087f740 1 -- 172.21.15.84:0/2365092716 --> [v2:172.21.15.84:6834/3621445269,v1:172.21.15.84:6836/3621445269] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b7d070cc0 con 0x560b7d14fdc0 2023-10-16T10:05:56.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.057+0000 7ffba77fe640 1 --2- 172.21.15.84:0/2365092716 >> [v2:172.21.15.84:6834/3621445269,v1:172.21.15.84:6836/3621445269] conn(0x560b7d14fdc0 0x560b7d150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.057+0000 7ffba77fe640 1 --2- 172.21.15.84:0/2365092716 >> [v2:172.21.15.84:6834/3621445269,v1:172.21.15.84:6836/3621445269] conn(0x560b7d14fdc0 0x560b7d150270 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dfd13a72c37cbed server_cookie=8e146bf58527af8 in_seq=0 out_seq=0 2023-10-16T10:05:56.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.057+0000 7ffb867fc640 10 client.7650 ms_handle_connect on v2:172.21.15.84:6834/3621445269 2023-10-16T10:05:56.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.058+0000 7ffb867fc640 1 -- 172.21.15.84:0/2365092716 <== mds.0 v2:172.21.15.84:6834/3621445269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560b7d070cc0 con 0x560b7d14fdc0 2023-10-16T10:05:56.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.058+0000 7ffb867fc640 10 client.7650 handle_command_reply: tid=0 2023-10-16T10:05:56.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.058+0000 7ffbb087f740 1 client.7650 shutdown 2023-10-16T10:05:56.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.058+0000 7ffbb087f740 1 -- 172.21.15.84:0/2365092716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d160360 msgr2=0x560b7d178880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:56.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.058+0000 7ffbb087f740 1 --2- 172.21.15.84:0/2365092716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d160360 0x560b7d178880 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ffb9002c3a0 tx=0x7ffb900046c0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.058+0000 7ffbb087f740 1 -- 172.21.15.84:0/2365092716 shutdown_connections 2023-10-16T10:05:56.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.058+0000 7ffbb087f740 1 --2- 172.21.15.84:0/2365092716 >> [v2:172.21.15.84:6834/3621445269,v1:172.21.15.84:6836/3621445269] conn(0x560b7d14fdc0 0x560b7d150270 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.058+0000 7ffbb087f740 1 --2- 172.21.15.84:0/2365092716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560b7d14dc90 0x560b7d165a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.058+0000 7ffbb087f740 1 --2- 172.21.15.84:0/2365092716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d160360 0x560b7d178880 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.058+0000 7ffbb087f740 1 --2- 172.21.15.84:0/2365092716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b7d15ff80 0x560b7d178340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.058+0000 7ffb867fc640 0 client.7650 ms_handle_reset on v2:172.21.15.84:6834/3621445269 2023-10-16T10:05:56.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.058+0000 7ffbb087f740 1 -- 172.21.15.84:0/2365092716 >> 172.21.15.84:0/2365092716 conn(0x560b7d1333c0 msgr2=0x560b7d136d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:56.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.059+0000 7ffbb087f740 1 -- 172.21.15.84:0/2365092716 shutdown_connections 2023-10-16T10:05:56.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.059+0000 7ffbb087f740 1 -- 172.21.15.84:0/2365092716 wait complete. 2023-10-16T10:05:56.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.059+0000 7ffbb087f740 20 client.7650 trim_cache size 0 max 16384 2023-10-16T10:05:56.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.076+0000 7ffbb087f740 1 Processor -- start 2023-10-16T10:05:56.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.076+0000 7ffbb087f740 1 -- start start 2023-10-16T10:05:56.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.077+0000 7ffbb087f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b7d14dc90 0x560b7d1a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.077+0000 7ffbb087f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d15ff80 0x560b7d177a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.077+0000 7ffbb087f740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560b7d160360 0x560b7d136d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.077+0000 7ffbb087f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b7ce02f20 con 0x560b7d15ff80 2023-10-16T10:05:56.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.077+0000 7ffbb087f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b7d1a0720 con 0x560b7d14dc90 2023-10-16T10:05:56.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.077+0000 7ffbb087f740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b7d136690 con 0x560b7d160360 2023-10-16T10:05:56.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.077+0000 7ffb9ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d15ff80 0x560b7d177a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.077+0000 7ffb9ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d15ff80 0x560b7d177a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54462/0 (socket says 172.21.15.84:54462) 2023-10-16T10:05:56.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.077+0000 7ffb9ffff640 1 -- 172.21.15.84:0/709444187 learned_addr learned my addr 172.21.15.84:0/709444187 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:56.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.078+0000 7ffb9ffff640 1 -- 172.21.15.84:0/709444187 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560b7d160360 msgr2=0x560b7d136d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:56.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.078+0000 7ffb9ffff640 1 --2- 172.21.15.84:0/709444187 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560b7d160360 0x560b7d136d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.078+0000 7ffb9ffff640 1 -- 172.21.15.84:0/709444187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b7d14dc90 msgr2=0x560b7d1a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:56.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.078+0000 7ffb9ffff640 1 --2- 172.21.15.84:0/709444187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b7d14dc90 0x560b7d1a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.078+0000 7ffb9ffff640 1 -- 172.21.15.84:0/709444187 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b7d03bb80 con 0x560b7d15ff80 2023-10-16T10:05:56.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.078+0000 7ffb9ffff640 1 --2- 172.21.15.84:0/709444187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d15ff80 0x560b7d177a20 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7ffb9002c370 tx=0x7ffb90047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ffe74247504540 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:56.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.078+0000 7ffb9dffb640 1 -- 172.21.15.84:0/709444187 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9000e070 con 0x560b7d15ff80 2023-10-16T10:05:56.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.078+0000 7ffb9dffb640 1 -- 172.21.15.84:0/709444187 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb90005b90 con 0x560b7d15ff80 2023-10-16T10:05:56.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.078+0000 7ffb9dffb640 1 -- 172.21.15.84:0/709444187 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90003830 con 0x560b7d15ff80 2023-10-16T10:05:56.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.079+0000 7ffbb087f740 1 -- 172.21.15.84:0/709444187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d15ff80 msgr2=0x560b7d177a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:56.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.079+0000 7ffbb087f740 1 --2- 172.21.15.84:0/709444187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d15ff80 0x560b7d177a20 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7ffb9002c370 tx=0x7ffb90047000 comp rx=0 tx=0).stop 2023-10-16T10:05:56.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.079+0000 7ffbb087f740 1 -- 172.21.15.84:0/709444187 shutdown_connections 2023-10-16T10:05:56.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.079+0000 7ffbb087f740 1 --2- 172.21.15.84:0/709444187 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560b7d160360 0x560b7d136d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.079+0000 7ffbb087f740 1 --2- 172.21.15.84:0/709444187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d15ff80 0x560b7d177a20 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.079+0000 7ffbb087f740 1 --2- 172.21.15.84:0/709444187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b7d14dc90 0x560b7d1a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.079+0000 7ffbb087f740 1 -- 172.21.15.84:0/709444187 >> 172.21.15.84:0/709444187 conn(0x560b7d1333c0 msgr2=0x560b7d166920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:56.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.079+0000 7ffbb087f740 1 -- 172.21.15.84:0/709444187 shutdown_connections 2023-10-16T10:05:56.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.079+0000 7ffbb087f740 1 -- 172.21.15.84:0/709444187 wait complete. 2023-10-16T10:05:56.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.080+0000 7ffbb087f740 1 -- start start 2023-10-16T10:05:56.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.080+0000 7ffbb087f740 1 Processor -- start 2023-10-16T10:05:56.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.081+0000 7ffbb087f740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560b7d14dc90 0x560b7d173ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.081+0000 7ffbb087f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d15ff80 0x560b7d1e4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.081+0000 7ffbb087f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b7d160360 0x560b7d1cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.081+0000 7ffbb087f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b7d1d1250 con 0x560b7d15ff80 2023-10-16T10:05:56.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.081+0000 7ffbb087f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b7d1e5510 con 0x560b7d160360 2023-10-16T10:05:56.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.081+0000 7ffbb087f740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b7d1d1670 con 0x560b7d14dc90 2023-10-16T10:05:56.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.081+0000 7ffbb087f740 10 client.0 _ll_register_callbacks cb 0x7ffb90037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:05:56.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.081+0000 7ffb9ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d15ff80 0x560b7d1e4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.081+0000 7ffb9ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d15ff80 0x560b7d1e4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54476/0 (socket says 172.21.15.84:54476) 2023-10-16T10:05:56.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.081+0000 7ffb9ffff640 1 -- 172.21.15.84:0/2745429002 learned_addr learned my addr 172.21.15.84:0/2745429002 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:56.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.081+0000 7ffb9ffff640 1 -- 172.21.15.84:0/2745429002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560b7d14dc90 msgr2=0x560b7d173ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:56.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.081+0000 7ffb9ffff640 1 --2- 172.21.15.84:0/2745429002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560b7d14dc90 0x560b7d173ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.081+0000 7ffb9ffff640 1 -- 172.21.15.84:0/2745429002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b7d160360 msgr2=0x560b7d1cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:56.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.081+0000 7ffb9ffff640 1 --2- 172.21.15.84:0/2745429002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b7d160360 0x560b7d1cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.082+0000 7ffb9ffff640 1 -- 172.21.15.84:0/2745429002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b7d09cda0 con 0x560b7d15ff80 2023-10-16T10:05:56.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.082+0000 7ffb9ffff640 1 --2- 172.21.15.84:0/2745429002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d15ff80 0x560b7d1e4d00 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7ffb9002c3a0 tx=0x7ffb9003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41df87086cb76a7a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:56.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.082+0000 7ffb867fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T10:05:56.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.082+0000 7ffb867fc640 1 -- 172.21.15.84:0/2745429002 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90047040 con 0x560b7d15ff80 2023-10-16T10:05:56.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.082+0000 7ffb867fc640 1 -- 172.21.15.84:0/2745429002 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb90003d10 con 0x560b7d15ff80 2023-10-16T10:05:56.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.082+0000 7ffbb087f740 1 -- 172.21.15.84:0/2745429002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffba000c840 con 0x560b7d15ff80 2023-10-16T10:05:56.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.082+0000 7ffb867fc640 1 -- 172.21.15.84:0/2745429002 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb9000e070 con 0x560b7d15ff80 2023-10-16T10:05:56.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.082+0000 7ffbb087f740 10 client.7656 fetch_fsmap learned FSMap version 53 2023-10-16T10:05:56.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.082+0000 7ffbb087f740 1 -- 172.21.15.84:0/2745429002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x560b7d061170 con 0x560b7d15ff80 2023-10-16T10:05:56.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.083+0000 7ffb867fc640 1 -- 172.21.15.84:0/2745429002 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba000c840 con 0x560b7d15ff80 2023-10-16T10:05:56.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.083+0000 7ffb867fc640 1 -- 172.21.15.84:0/2745429002 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ffb90005fb0 con 0x560b7d15ff80 2023-10-16T10:05:56.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.083+0000 7ffbb087f740 10 client.7656 fetch_fsmap finished waiting for FSMap version 53 2023-10-16T10:05:56.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.083+0000 7ffbb087f740 10 client.7656 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T10:05:56.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.083+0000 7ffbb087f740 20 client.7656 populate_metadata read hostname 'smithi084' 2023-10-16T10:05:56.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.083+0000 7ffbb087f740 1 --2- 172.21.15.84:0/2745429002 >> [v2:172.21.15.84:6834/3621445269,v1:172.21.15.84:6836/3621445269] conn(0x560b7d15e270 0x560b7d15e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.083+0000 7ffbb087f740 4 client.7656 mds_command: new command op to 7521 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T10:05:56.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.083+0000 7ffbb087f740 1 -- 172.21.15.84:0/2745429002 --> [v2:172.21.15.84:6834/3621445269,v1:172.21.15.84:6836/3621445269] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560b7d097700 con 0x560b7d15e270 2023-10-16T10:05:56.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.083+0000 7ffba77fe640 1 --2- 172.21.15.84:0/2745429002 >> [v2:172.21.15.84:6834/3621445269,v1:172.21.15.84:6836/3621445269] conn(0x560b7d15e270 0x560b7d15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.084+0000 7ffba77fe640 1 --2- 172.21.15.84:0/2745429002 >> [v2:172.21.15.84:6834/3621445269,v1:172.21.15.84:6836/3621445269] conn(0x560b7d15e270 0x560b7d15e650 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99eaba259b1daa69 server_cookie=c17d573b11e1bf1 in_seq=0 out_seq=0 2023-10-16T10:05:56.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.084+0000 7ffb867fc640 10 client.7656 ms_handle_connect on v2:172.21.15.84:6834/3621445269 2023-10-16T10:05:56.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.084+0000 7ffb867fc640 1 -- 172.21.15.84:0/2745429002 <== mds.0 v2:172.21.15.84:6834/3621445269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x560b7d097700 con 0x560b7d15e270 2023-10-16T10:05:56.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.084+0000 7ffb867fc640 10 client.7656 handle_command_reply: tid=0 2023-10-16T10:05:56.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.084+0000 7ffbb087f740 1 client.7656 shutdown 2023-10-16T10:05:56.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.084+0000 7ffbb087f740 1 -- 172.21.15.84:0/2745429002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d15ff80 msgr2=0x560b7d1e4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:56.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.084+0000 7ffbb087f740 1 --2- 172.21.15.84:0/2745429002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d15ff80 0x560b7d1e4d00 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7ffb9002c3a0 tx=0x7ffb9003d000 comp rx=0 tx=0).stop 2023-10-16T10:05:56.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.084+0000 7ffbb087f740 1 -- 172.21.15.84:0/2745429002 shutdown_connections 2023-10-16T10:05:56.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.084+0000 7ffbb087f740 1 --2- 172.21.15.84:0/2745429002 >> [v2:172.21.15.84:6834/3621445269,v1:172.21.15.84:6836/3621445269] conn(0x560b7d15e270 0x560b7d15e650 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.084+0000 7ffbb087f740 1 --2- 172.21.15.84:0/2745429002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b7d160360 0x560b7d1cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.084+0000 7ffbb087f740 1 --2- 172.21.15.84:0/2745429002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b7d15ff80 0x560b7d1e4d00 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.084+0000 7ffb867fc640 0 client.7656 ms_handle_reset on v2:172.21.15.84:6834/3621445269 2023-10-16T10:05:56.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.084+0000 7ffbb087f740 1 --2- 172.21.15.84:0/2745429002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560b7d14dc90 0x560b7d173ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.084+0000 7ffbb087f740 1 -- 172.21.15.84:0/2745429002 >> 172.21.15.84:0/2745429002 conn(0x560b7d1333c0 msgr2=0x560b7d1668e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:56.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.084+0000 7ffbb087f740 1 -- 172.21.15.84:0/2745429002 shutdown_connections 2023-10-16T10:05:56.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.084+0000 7ffbb087f740 1 -- 172.21.15.84:0/2745429002 wait complete. 2023-10-16T10:05:56.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.085+0000 7ffbb087f740 20 client.7656 trim_cache size 0 max 16384 2023-10-16T10:05:56.102 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T10:05:56.102 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "63fb4d68-f881-4694-b880-d10369105c0e", 2023-10-16T10:05:56.102 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-16T10:05:56.103 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 7521, 2023-10-16T10:05:56.103 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-16T10:05:56.103 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-16T10:05:56.103 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-16T10:05:56.103 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 6.7612151159999998, 2023-10-16T10:05:56.104 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 53, 2023-10-16T10:05:56.104 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 108, 2023-10-16T10:05:56.104 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 108, 2023-10-16T10:05:56.104 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 12.112512554 2023-10-16T10:05:56.104 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T10:05:56.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.085+0000 7ffbadce7640 1 -- 172.21.15.84:0/2697496159 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7ffb8806af00 msgr2=0x7ffb8806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:56.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.085+0000 7ffbadce7640 1 --2- 172.21.15.84:0/2697496159 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7ffb8806af00 0x7ffb8806d3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffb90036000 tx=0x7ffb9002e000 comp rx=0 tx=0).stop 2023-10-16T10:05:56.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.085+0000 7ffbadce7640 1 -- 172.21.15.84:0/2697496159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb940047c0 msgr2=0x7ffba80ade80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:56.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.085+0000 7ffbadce7640 1 --2- 172.21.15.84:0/2697496159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb940047c0 0x7ffba80ade80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ffb980108e0 tx=0x7ffb98010da0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.085+0000 7ffbadce7640 1 -- 172.21.15.84:0/2697496159 shutdown_connections 2023-10-16T10:05:56.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.085+0000 7ffbadce7640 1 --2- 172.21.15.84:0/2697496159 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7ffb8806af00 0x7ffb8806d3c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.085+0000 7ffbadce7640 1 --2- 172.21.15.84:0/2697496159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba80c75c0 0x7ffba80c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.085+0000 7ffbadce7640 1 --2- 172.21.15.84:0/2697496159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba80ac890 0x7ffba8151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.085+0000 7ffbadce7640 1 --2- 172.21.15.84:0/2697496159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb940047c0 0x7ffba80ade80 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.085+0000 7ffbadce7640 1 -- 172.21.15.84:0/2697496159 >> 172.21.15.84:0/2697496159 conn(0x7ffba80ce9b0 msgr2=0x7ffba80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:56.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.085+0000 7ffbadce7640 1 -- 172.21.15.84:0/2697496159 shutdown_connections 2023-10-16T10:05:56.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.085+0000 7ffbadce7640 1 -- 172.21.15.84:0/2697496159 wait complete. 2023-10-16T10:05:56.109 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T10:05:56.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.321+0000 7f86faa56640 1 Processor -- start 2023-10-16T10:05:56.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.322+0000 7f86faa56640 1 -- start start 2023-10-16T10:05:56.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.323+0000 7f86faa56640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f86f406c3f0 0x7f86f406c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.323+0000 7f86faa56640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f40cd630 con 0x7f86f415e0c0 2023-10-16T10:05:56.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.324+0000 7f86faa56640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f40ce1b0 con 0x7f86f415e540 2023-10-16T10:05:56.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.324+0000 7f86faa56640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f406cd10 con 0x7f86f406c3f0 2023-10-16T10:05:56.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.324+0000 7f86eb7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f86f415e0c0 0x7f86f40b4730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35238/0 (socket says 172.21.15.84:35238) 2023-10-16T10:05:56.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.324+0000 7f86eb7fe640 1 -- 172.21.15.84:0/925018974 learned_addr learned my addr 172.21.15.84:0/925018974 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:56.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.324+0000 7f86f3fff640 1 --2- 172.21.15.84:0/925018974 >> v2:172.21.15.157:3300/0 conn(0x7f86f406c3f0 0x7f86f406c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.324+0000 7f86f37fe640 1 -- 172.21.15.84:0/925018974 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535557153 0 0) 0x7f86f40cd630 con 0x7f86f415e0c0 2023-10-16T10:05:56.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.324+0000 7f86f37fe640 1 -- 172.21.15.84:0/925018974 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86e0003520 con 0x7f86f415e0c0 2023-10-16T10:05:56.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.325+0000 7f86f37fe640 1 -- 172.21.15.84:0/925018974 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3573112653 0 0) 0x7f86e0003520 con 0x7f86f415e0c0 2023-10-16T10:05:56.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.325+0000 7f86f37fe640 1 -- 172.21.15.84:0/925018974 >> v2:172.21.15.157:3300/0 conn(0x7f86f406c3f0 msgr2=0x7f86f406c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:56.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.325+0000 7f86f37fe640 1 --2- 172.21.15.84:0/925018974 >> v2:172.21.15.157:3300/0 conn(0x7f86f406c3f0 0x7f86f406c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.325+0000 7f86f37fe640 1 -- 172.21.15.84:0/925018974 >> v1:172.21.15.157:6789/0 conn(0x7f86f415e540 legacy=0x7f86f40b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:56.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.325+0000 7f86f37fe640 1 -- 172.21.15.84:0/925018974 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f40d2170 con 0x7f86f415e0c0 2023-10-16T10:05:56.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.325+0000 7f86f37fe640 1 -- 172.21.15.84:0/925018974 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f86dc001670 con 0x7f86f415e0c0 2023-10-16T10:05:56.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.325+0000 7f86f37fe640 1 -- 172.21.15.84:0/925018974 >> v1:172.21.15.84:6789/0 conn(0x7f86f415e0c0 legacy=0x7f86f40b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:56.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.325+0000 7f86f37fe640 1 --2- 172.21.15.84:0/925018974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00039c0 0x7f86e0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.325+0000 7f86f37fe640 1 --2- 172.21.15.84:0/925018974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e00047c0 0x7f86e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.325+0000 7f86f37fe640 1 --2- 172.21.15.84:0/925018974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e0005180 0x7f86e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.325+0000 7f86f37fe640 1 -- 172.21.15.84:0/925018974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f406cd10 con 0x7f86e00039c0 2023-10-16T10:05:56.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.325+0000 7f86f37fe640 1 -- 172.21.15.84:0/925018974 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e000c080 con 0x7f86e00047c0 2023-10-16T10:05:56.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.325+0000 7f86f37fe640 1 -- 172.21.15.84:0/925018974 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e000c240 con 0x7f86e0005180 2023-10-16T10:05:56.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.325+0000 7f86f3fff640 1 --2- 172.21.15.84:0/925018974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00039c0 0x7f86e0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.326+0000 7f86f3fff640 1 -- 172.21.15.84:0/925018974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e0005180 msgr2=0x7f86e000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:56.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.326+0000 7f86f8ff3640 1 --2- 172.21.15.84:0/925018974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e0005180 0x7f86e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.326+0000 7f86f3fff640 1 --2- 172.21.15.84:0/925018974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86e0005180 0x7f86e000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.326+0000 7f86f3fff640 1 -- 172.21.15.84:0/925018974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e00047c0 msgr2=0x7f86e0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:56.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.326+0000 7f86f3fff640 1 --2- 172.21.15.84:0/925018974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e00047c0 0x7f86e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.326+0000 7f86f3fff640 1 -- 172.21.15.84:0/925018974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e0003520 con 0x7f86e00039c0 2023-10-16T10:05:56.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.326+0000 7f86f8ff3640 1 -- 172.21.15.84:0/925018974 reap_dead start 2023-10-16T10:05:56.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.327+0000 7f86f3fff640 1 --2- 172.21.15.84:0/925018974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00039c0 0x7f86e0003de0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009ac0 tx=0x7f86e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd91d9f26eea09b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:56.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.327+0000 7f86f37fe640 1 -- 172.21.15.84:0/925018974 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e402c6b0 con 0x7f86e00039c0 2023-10-16T10:05:56.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.327+0000 7f86f37fe640 1 -- 172.21.15.84:0/925018974 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e402c810 con 0x7f86e00039c0 2023-10-16T10:05:56.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.327+0000 7f86f37fe640 1 -- 172.21.15.84:0/925018974 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e402cba0 con 0x7f86e00039c0 2023-10-16T10:05:56.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.328+0000 7f86faa56640 1 -- 172.21.15.84:0/925018974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00039c0 msgr2=0x7f86e0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:56.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.328+0000 7f86faa56640 1 --2- 172.21.15.84:0/925018974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00039c0 0x7f86e0003de0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009ac0 tx=0x7f86e00057f0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.328+0000 7f86faa56640 1 -- 172.21.15.84:0/925018974 shutdown_connections 2023-10-16T10:05:56.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.328+0000 7f86faa56640 1 --2- 172.21.15.84:0/925018974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00039c0 0x7f86e0003de0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.328+0000 7f86faa56640 1 -- 172.21.15.84:0/925018974 >> 172.21.15.84:0/925018974 conn(0x7f86f40d2cb0 msgr2=0x7f86f40d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:56.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.328+0000 7f86faa56640 1 -- 172.21.15.84:0/925018974 shutdown_connections 2023-10-16T10:05:56.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.328+0000 7f86faa56640 1 -- 172.21.15.84:0/925018974 wait complete. 2023-10-16T10:05:56.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.330+0000 7f86faa56640 1 Processor -- start 2023-10-16T10:05:56.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.330+0000 7f86faa56640 1 -- start start 2023-10-16T10:05:56.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.331+0000 7f86faa56640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00039c0 0x7f86f4154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.331+0000 7f86faa56640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86f4160000 0x7f86f4160480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.331+0000 7f86faa56640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f40c81b0 0x7f86f40c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.331+0000 7f86faa56640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f41609c0 con 0x7f86e00039c0 2023-10-16T10:05:56.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.331+0000 7f86faa56640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f40cd340 con 0x7f86f40c81b0 2023-10-16T10:05:56.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.331+0000 7f86faa56640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f4155510 con 0x7f86f4160000 2023-10-16T10:05:56.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.331+0000 7f86f3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00039c0 0x7f86f4154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.331+0000 7f86f3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00039c0 0x7f86f4154fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54498/0 (socket says 172.21.15.84:54498) 2023-10-16T10:05:56.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.331+0000 7f86f3fff640 1 -- 172.21.15.84:0/659952986 learned_addr learned my addr 172.21.15.84:0/659952986 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:56.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.331+0000 7f86f3fff640 1 -- 172.21.15.84:0/659952986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86f4160000 msgr2=0x7f86f4160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:56.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.332+0000 7f86f3fff640 1 --2- 172.21.15.84:0/659952986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86f4160000 0x7f86f4160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.332+0000 7f86f3fff640 1 -- 172.21.15.84:0/659952986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f40c81b0 msgr2=0x7f86f40c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:56.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.332+0000 7f86f3fff640 1 --2- 172.21.15.84:0/659952986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f40c81b0 0x7f86f40c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.332+0000 7f86f3fff640 1 -- 172.21.15.84:0/659952986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f40c8b70 con 0x7f86e00039c0 2023-10-16T10:05:56.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.332+0000 7f86f3fff640 1 --2- 172.21.15.84:0/659952986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00039c0 0x7f86f4154fd0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f86e4037040 tx=0x7f86e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:56.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.332+0000 7f86f17fa640 1 -- 172.21.15.84:0/659952986 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4003810 con 0x7f86e00039c0 2023-10-16T10:05:56.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.332+0000 7f86f17fa640 1 -- 172.21.15.84:0/659952986 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e4003970 con 0x7f86e00039c0 2023-10-16T10:05:56.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.332+0000 7f86faa56640 1 -- 172.21.15.84:0/659952986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f40c8e00 con 0x7f86e00039c0 2023-10-16T10:05:56.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.332+0000 7f86f17fa640 1 -- 172.21.15.84:0/659952986 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4003d00 con 0x7f86e00039c0 2023-10-16T10:05:56.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.332+0000 7f86faa56640 1 -- 172.21.15.84:0/659952986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f40c93c0 con 0x7f86e00039c0 2023-10-16T10:05:56.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.334+0000 7f86faa56640 1 -- 172.21.15.84:0/659952986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f40d1b40 con 0x7f86e00039c0 2023-10-16T10:05:56.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.334+0000 7f86f17fa640 1 -- 172.21.15.84:0/659952986 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f86e4040450 con 0x7f86e00039c0 2023-10-16T10:05:56.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.335+0000 7f86f17fa640 1 --2- 172.21.15.84:0/659952986 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f86d406af30 0x7f86d406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:56.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.336+0000 7f86f17fa640 1 -- 172.21.15.84:0/659952986 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86e40ab630 con 0x7f86e00039c0 2023-10-16T10:05:56.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.336+0000 7f86eb7fe640 1 --2- 172.21.15.84:0/659952986 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f86d406af30 0x7f86d406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:56.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.336+0000 7f86eb7fe640 1 --2- 172.21.15.84:0/659952986 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f86d406af30 0x7f86d406d3f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f86dc000ff0 tx=0x7f86dc001510 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:56.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.340+0000 7f86f17fa640 1 -- 172.21.15.84:0/659952986 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f86e40765f0 con 0x7f86e00039c0 2023-10-16T10:05:56.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.585+0000 7f86faa56640 1 -- 172.21.15.84:0/659952986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f86f404e160 con 0x7f86e00039c0 2023-10-16T10:05:56.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.585+0000 7f86f17fa640 1 -- 172.21.15.84:0/659952986 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7f86e4003e60 con 0x7f86e00039c0 2023-10-16T10:05:56.587 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T10:05:56.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.588+0000 7f86faa56640 1 -- 172.21.15.84:0/659952986 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f86d406af30 msgr2=0x7f86d406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:56.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.588+0000 7f86faa56640 1 --2- 172.21.15.84:0/659952986 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f86d406af30 0x7f86d406d3f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f86dc000ff0 tx=0x7f86dc001510 comp rx=0 tx=0).stop 2023-10-16T10:05:56.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.588+0000 7f86faa56640 1 -- 172.21.15.84:0/659952986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00039c0 msgr2=0x7f86f4154fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:56.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.588+0000 7f86faa56640 1 --2- 172.21.15.84:0/659952986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00039c0 0x7f86f4154fd0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f86e4037040 tx=0x7f86e4004730 comp rx=0 tx=0).stop 2023-10-16T10:05:56.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.588+0000 7f86faa56640 1 -- 172.21.15.84:0/659952986 shutdown_connections 2023-10-16T10:05:56.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.588+0000 7f86faa56640 1 --2- 172.21.15.84:0/659952986 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f86d406af30 0x7f86d406d3f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.588+0000 7f86faa56640 1 --2- 172.21.15.84:0/659952986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f40c81b0 0x7f86f40c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.588+0000 7f86faa56640 1 --2- 172.21.15.84:0/659952986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86f4160000 0x7f86f4160480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.588+0000 7f86faa56640 1 --2- 172.21.15.84:0/659952986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00039c0 0x7f86f4154fd0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:56.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.588+0000 7f86faa56640 1 -- 172.21.15.84:0/659952986 >> 172.21.15.84:0/659952986 conn(0x7f86f40d2cb0 msgr2=0x7f86f40d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:56.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.589+0000 7f86faa56640 1 -- 172.21.15.84:0/659952986 shutdown_connections 2023-10-16T10:05:56.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:56.589+0000 7f86faa56640 1 -- 172.21.15.84:0/659952986 wait complete. 2023-10-16T10:05:56.608 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T10:05:56.608 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T10:05:56.608 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-16T10:05:56.609 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T10:05:56.609 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T10:05:56.609 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T10:05:56.610 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T10:05:56.610 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T10:05:56.610 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:05:56.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:56.611 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-16T10:05:56.662 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T10:05:56.662 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T10:05:56.662 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T10:05:56.663 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:05:56.663 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T10:05:56.663 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:05:56.663 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:05:56.663 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:05:56.663 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-16T10:05:56.664 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:05:56.664 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:05:56.664 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-16T10:05:56.664 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:05:56.664 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:05:56.665 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T10:05:56.665 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:05:56.665 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T10:05:56.665 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 41394sec preferred_lft 41394sec 2023-10-16T10:05:56.665 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-16T10:05:56.666 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:05:56.666 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T10:05:56.666 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:05:56.666 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T10:05:56.666 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:05:56.667 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::20a8:eaff:fe9d:5cea/64 scope link 2023-10-16T10:05:56.667 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:05:56.667 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:05:56.667 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T10:05:56.667 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1854:13ff:fe47:be92/64 scope link 2023-10-16T10:05:56.667 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:05:56.668 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:05:56.668 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether ee:d5:41:a3:a5:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T10:05:56.668 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ecd5:41ff:fea3:a570/64 scope link 2023-10-16T10:05:56.668 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:05:56.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:56.670 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T10:05:56.805 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-16T10:05:56.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:56.809 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-16T10:05:56.861 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-16T10:05:56.862 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T10:05:56.862 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T10:05:56.862 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T10:05:56.862 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T10:05:56.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:56.863 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T10:05:56.864 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T10:05:56.864 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T10:05:56.864 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T10:05:56.865 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T10:05:56.865 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T10:05:57.021 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:05:57.019+0000 7f91b2251640 -1 client.7344: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.7236:de96542f-d8f3-48f7-983f-97efe1989b1c:v1:172.21.15.84:0/3830790285] 2023-10-16T10:05:57.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:57.036 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T10:05:57.087 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T10:05:57.087 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T10:05:57.088 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:05:57.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:57.089 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-16T10:05:57.178 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:05:57.179 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T10:05:57.232 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:05:57.233 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:05:57.233 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:05:57.284 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-16T10:05:57.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:57.285 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:05:57.464 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:05:57.465 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T10:05:57.502 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:05:57.502 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T10:05:57.502 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T10:05:57.503 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with new device syntax: 0@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T10:05:57.503 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T10:05:57.504 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-16T10:05:57.504 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2023-10-16T10:05:57.504 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T10:05:57.504 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T10:05:57.505 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T10:05:57.508 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T10:05:57.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:57.508 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:05:57.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:05:57.581 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T10:05:57.636 INFO:teuthology.orchestra.run.smithi084.stdout:client5483 2023-10-16T10:05:57.637 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:05:57.637 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:05:57.684 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T10:05:57.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.941+0000 7f0ba634e640 1 Processor -- start 2023-10-16T10:05:57.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.941+0000 7f0ba634e640 1 -- start start 2023-10-16T10:05:57.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.942+0000 7f0ba634e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ba0159ab0 0x7f0ba00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:57.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.942+0000 7f0ba634e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0ba0159e90 0x7f0ba006fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:57.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.943+0000 7f0ba634e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0ba006c140 0x7f0ba006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:57.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.943+0000 7f0ba634e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ba006cb10 con 0x7f0ba006c140 2023-10-16T10:05:57.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.943+0000 7f0ba634e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ba015d890 con 0x7f0ba0159e90 2023-10-16T10:05:57.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.943+0000 7f0ba634e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba00cd1f0 con 0x7f0ba0159ab0 2023-10-16T10:05:57.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.943+0000 7f0ba48eb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0ba006c140 0x7f0ba006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:57.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.943+0000 7f0ba48eb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0ba006c140 0x7f0ba006c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54534/0 (socket says 172.21.15.84:54534) 2023-10-16T10:05:57.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.943+0000 7f0ba48eb640 1 -- 172.21.15.84:0/983239675 learned_addr learned my addr 172.21.15.84:0/983239675 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:57.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.943+0000 7f0b9ffff640 1 --2- 172.21.15.84:0/983239675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ba0159ab0 0x7f0ba00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:57.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.943+0000 7f0ba48eb640 1 -- 172.21.15.84:0/983239675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ba0159ab0 msgr2=0x7f0ba00ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:57.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.943+0000 7f0ba48eb640 1 --2- 172.21.15.84:0/983239675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ba0159ab0 0x7f0ba00ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:57.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.943+0000 7f0b97fff640 1 --2- 172.21.15.84:0/983239675 >> v2:172.21.15.157:3300/0 conn(0x7f0ba0159e90 0x7f0ba006fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:57.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.943+0000 7f0ba48eb640 1 -- 172.21.15.84:0/983239675 >> v2:172.21.15.157:3300/0 conn(0x7f0ba0159e90 msgr2=0x7f0ba006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:57.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.943+0000 7f0ba48eb640 1 --2- 172.21.15.84:0/983239675 >> v2:172.21.15.157:3300/0 conn(0x7f0ba0159e90 0x7f0ba006fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:57.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.944+0000 7f0ba48eb640 1 -- 172.21.15.84:0/983239675 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba00cd4d0 con 0x7f0ba006c140 2023-10-16T10:05:57.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.945+0000 7f0ba48eb640 1 --2- 172.21.15.84:0/983239675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ba006c140 0x7f0ba006c5d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009e50 tx=0x7f0b9802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74afea4dbffda7e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:57.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.945+0000 7f0b9f7fe640 1 -- 172.21.15.84:0/983239675 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9802c950 con 0x7f0ba006c140 2023-10-16T10:05:57.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.945+0000 7f0b9f7fe640 1 -- 172.21.15.84:0/983239675 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9802cab0 con 0x7f0ba006c140 2023-10-16T10:05:57.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.945+0000 7f0b9f7fe640 1 -- 172.21.15.84:0/983239675 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9802ce40 con 0x7f0ba006c140 2023-10-16T10:05:57.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.946+0000 7f0ba634e640 1 -- 172.21.15.84:0/983239675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ba006c140 msgr2=0x7f0ba006c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:57.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.946+0000 7f0ba634e640 1 --2- 172.21.15.84:0/983239675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ba006c140 0x7f0ba006c5d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009e50 tx=0x7f0b9802bdb0 comp rx=0 tx=0).stop 2023-10-16T10:05:57.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.946+0000 7f0ba634e640 1 -- 172.21.15.84:0/983239675 shutdown_connections 2023-10-16T10:05:57.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.946+0000 7f0ba634e640 1 --2- 172.21.15.84:0/983239675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ba006c140 0x7f0ba006c5d0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:57.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.946+0000 7f0ba634e640 1 --2- 172.21.15.84:0/983239675 >> v2:172.21.15.157:3300/0 conn(0x7f0ba0159e90 0x7f0ba006fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:57.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.946+0000 7f0ba634e640 1 --2- 172.21.15.84:0/983239675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ba0159ab0 0x7f0ba00ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:57.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.946+0000 7f0ba634e640 1 -- 172.21.15.84:0/983239675 >> 172.21.15.84:0/983239675 conn(0x7f0ba00ce9b0 msgr2=0x7f0ba00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:57.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.946+0000 7f0ba634e640 1 -- 172.21.15.84:0/983239675 shutdown_connections 2023-10-16T10:05:57.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.946+0000 7f0ba634e640 1 -- 172.21.15.84:0/983239675 wait complete. 2023-10-16T10:05:57.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.948+0000 7f0ba634e640 1 Processor -- start 2023-10-16T10:05:57.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.948+0000 7f0ba634e640 1 -- start start 2023-10-16T10:05:57.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.949+0000 7f0ba634e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ba006c140 0x7f0ba0156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:57.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.949+0000 7f0ba634e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ba0159ab0 0x7f0ba00b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:57.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.949+0000 7f0ba634e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ba0159e90 0x7f0ba01f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:57.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.949+0000 7f0ba634e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba00c3530 con 0x7f0ba0159ab0 2023-10-16T10:05:57.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.949+0000 7f0ba634e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba00b2950 con 0x7f0ba006c140 2023-10-16T10:05:57.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.949+0000 7f0ba634e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba00c3780 con 0x7f0ba0159e90 2023-10-16T10:05:57.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.949+0000 7f0b97fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ba0159ab0 0x7f0ba00b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:57.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.949+0000 7f0b97fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ba0159ab0 0x7f0ba00b1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54546/0 (socket says 172.21.15.84:54546) 2023-10-16T10:05:57.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.949+0000 7f0b97fff640 1 -- 172.21.15.84:0/4058062966 learned_addr learned my addr 172.21.15.84:0/4058062966 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:57.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.949+0000 7f0b9ffff640 1 --2- 172.21.15.84:0/4058062966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ba006c140 0x7f0ba0156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:57.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.949+0000 7f0b97fff640 1 -- 172.21.15.84:0/4058062966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ba0159e90 msgr2=0x7f0ba01f27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:57.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.949+0000 7f0b97fff640 1 --2- 172.21.15.84:0/4058062966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ba0159e90 0x7f0ba01f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:57.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.949+0000 7f0b97fff640 1 -- 172.21.15.84:0/4058062966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ba006c140 msgr2=0x7f0ba0156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:57.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.949+0000 7f0b97fff640 1 --2- 172.21.15.84:0/4058062966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ba006c140 0x7f0ba0156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:57.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.949+0000 7f0b97fff640 1 -- 172.21.15.84:0/4058062966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b98009b30 con 0x7f0ba0159ab0 2023-10-16T10:05:57.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.950+0000 7f0b97fff640 1 --2- 172.21.15.84:0/4058062966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ba0159ab0 0x7f0ba00b1fe0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800c7f0 tx=0x7f0b8800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:57.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.950+0000 7f0b9d7fa640 1 -- 172.21.15.84:0/4058062966 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b88004220 con 0x7f0ba0159ab0 2023-10-16T10:05:57.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.950+0000 7f0b9d7fa640 1 -- 172.21.15.84:0/4058062966 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b88004380 con 0x7f0ba0159ab0 2023-10-16T10:05:57.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.950+0000 7f0ba634e640 1 -- 172.21.15.84:0/4058062966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba00c3a60 con 0x7f0ba0159ab0 2023-10-16T10:05:57.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.950+0000 7f0b9d7fa640 1 -- 172.21.15.84:0/4058062966 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b88007840 con 0x7f0ba0159ab0 2023-10-16T10:05:57.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.950+0000 7f0ba634e640 1 -- 172.21.15.84:0/4058062966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba0157510 con 0x7f0ba0159ab0 2023-10-16T10:05:57.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.951+0000 7f0ba634e640 1 -- 172.21.15.84:0/4058062966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba00c3f80 con 0x7f0ba0159ab0 2023-10-16T10:05:57.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.952+0000 7f0b9d7fa640 1 -- 172.21.15.84:0/4058062966 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f0b88007bb0 con 0x7f0ba0159ab0 2023-10-16T10:05:57.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.953+0000 7f0b9d7fa640 1 --2- 172.21.15.84:0/4058062966 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f0b8406af00 0x7f0b8406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:57.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.953+0000 7f0b9d7fa640 1 -- 172.21.15.84:0/4058062966 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5752+0+0 (secure 0 0 0) 0x7f0b88089770 con 0x7f0ba0159ab0 2023-10-16T10:05:57.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.953+0000 7f0b9ffff640 1 --2- 172.21.15.84:0/4058062966 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f0b8406af00 0x7f0b8406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:57.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.954+0000 7f0b9ffff640 1 --2- 172.21.15.84:0/4058062966 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f0b8406af00 0x7f0b8406d3c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0b90006fd0 tx=0x7f0b90008040 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:57.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:57.958+0000 7f0b9d7fa640 1 -- 172.21.15.84:0/4058062966 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0ba00c3f80 con 0x7f0ba0159ab0 2023-10-16T10:05:58.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.175+0000 7f0ba634e640 1 -- 172.21.15.84:0/4058062966 --> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ba00cda90 con 0x7f0b8406af00 2023-10-16T10:05:58.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.178+0000 7f0b9d7fa640 1 -- 172.21.15.84:0/4058062966 <== mgr.7284 v2:172.21.15.84:6832/3250823028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f0b9000e510 con 0x7f0b8406af00 2023-10-16T10:05:58.179 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T10:05:58.180 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:05:58.180 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephfs" 2023-10-16T10:05:58.180 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:05:58.180 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T10:05:58.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.180+0000 7f0ba634e640 1 -- 172.21.15.84:0/4058062966 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f0b8406af00 msgr2=0x7f0b8406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:58.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.180+0000 7f0ba634e640 1 --2- 172.21.15.84:0/4058062966 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f0b8406af00 0x7f0b8406d3c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0b90006fd0 tx=0x7f0b90008040 comp rx=0 tx=0).stop 2023-10-16T10:05:58.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.180+0000 7f0ba634e640 1 -- 172.21.15.84:0/4058062966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ba0159ab0 msgr2=0x7f0ba00b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:58.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.180+0000 7f0ba634e640 1 --2- 172.21.15.84:0/4058062966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ba0159ab0 0x7f0ba00b1fe0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800c7f0 tx=0x7f0b8800ccb0 comp rx=0 tx=0).stop 2023-10-16T10:05:58.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.181+0000 7f0ba634e640 1 -- 172.21.15.84:0/4058062966 shutdown_connections 2023-10-16T10:05:58.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.181+0000 7f0ba634e640 1 --2- 172.21.15.84:0/4058062966 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f0b8406af00 0x7f0b8406d3c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:58.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.181+0000 7f0ba634e640 1 --2- 172.21.15.84:0/4058062966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ba0159e90 0x7f0ba01f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:58.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.181+0000 7f0ba634e640 1 --2- 172.21.15.84:0/4058062966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ba0159ab0 0x7f0ba00b1fe0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:58.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.181+0000 7f0ba634e640 1 --2- 172.21.15.84:0/4058062966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ba006c140 0x7f0ba0156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:58.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.181+0000 7f0ba634e640 1 -- 172.21.15.84:0/4058062966 >> 172.21.15.84:0/4058062966 conn(0x7f0ba00ce9b0 msgr2=0x7f0ba0158f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:58.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.181+0000 7f0ba634e640 1 -- 172.21.15.84:0/4058062966 shutdown_connections 2023-10-16T10:05:58.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.181+0000 7f0ba634e640 1 -- 172.21.15.84:0/4058062966 wait complete. 2023-10-16T10:05:58.200 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T10:05:58.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.409+0000 7f48789a9640 1 Processor -- start 2023-10-16T10:05:58.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.409+0000 7f48789a9640 1 -- start start 2023-10-16T10:05:58.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.410+0000 7f48789a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4874159ab0 0x7f48740ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:58.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.410+0000 7f48789a9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f487406c140 0x7f487406c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:58.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.410+0000 7f48789a9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487406cad0 con 0x7f4874159e90 2023-10-16T10:05:58.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.410+0000 7f48789a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487415d890 con 0x7f4874159ab0 2023-10-16T10:05:58.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.410+0000 7f48789a9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4874070360 con 0x7f487406c140 2023-10-16T10:05:58.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.410+0000 7f487259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4874159ab0 0x7f48740ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:58.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.410+0000 7f4871d9b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4874159e90 0x7f487406fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35244/0 (socket says 172.21.15.84:35244) 2023-10-16T10:05:58.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.410+0000 7f4871d9b640 1 -- 172.21.15.84:0/1588322623 learned_addr learned my addr 172.21.15.84:0/1588322623 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:05:58.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.411+0000 7f487259c640 1 -- 172.21.15.84:0/1588322623 >> v2:172.21.15.157:3300/0 conn(0x7f487406c140 msgr2=0x7f487406c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:58.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.411+0000 7f4872d9d640 1 --2- 172.21.15.84:0/1588322623 >> v2:172.21.15.157:3300/0 conn(0x7f487406c140 0x7f487406c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:58.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.411+0000 7f487259c640 1 --2- 172.21.15.84:0/1588322623 >> v2:172.21.15.157:3300/0 conn(0x7f487406c140 0x7f487406c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:58.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.411+0000 7f487259c640 1 -- 172.21.15.84:0/1588322623 >> v1:172.21.15.84:6789/0 conn(0x7f4874159e90 legacy=0x7f487406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:58.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.411+0000 7f487159a640 1 -- 172.21.15.84:0/1588322623 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1985346645 0 0) 0x7f485c001490 con 0x7f4874159e90 2023-10-16T10:05:58.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.411+0000 7f487259c640 1 -- 172.21.15.84:0/1588322623 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48740cd1f0 con 0x7f4874159ab0 2023-10-16T10:05:58.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.411+0000 7f4872d9d640 1 --2- 172.21.15.84:0/1588322623 >> v2:172.21.15.157:3300/0 conn(0x7f487406c140 0x7f487406c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:58.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.412+0000 7f487259c640 1 --2- 172.21.15.84:0/1588322623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4874159ab0 0x7f48740ce370 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f48740adcd0 tx=0x7f486402b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512ae6b0ea749c05 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:58.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.412+0000 7f487159a640 1 -- 172.21.15.84:0/1588322623 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864031710 con 0x7f4874159ab0 2023-10-16T10:05:58.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.412+0000 7f487159a640 1 -- 172.21.15.84:0/1588322623 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486402f050 con 0x7f4874159ab0 2023-10-16T10:05:58.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.412+0000 7f487159a640 1 -- 172.21.15.84:0/1588322623 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864031a90 con 0x7f4874159ab0 2023-10-16T10:05:58.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.413+0000 7f48789a9640 1 -- 172.21.15.84:0/1588322623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4874159ab0 msgr2=0x7f48740ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:58.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.413+0000 7f48789a9640 1 --2- 172.21.15.84:0/1588322623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4874159ab0 0x7f48740ce370 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f48740adcd0 tx=0x7f486402b7b0 comp rx=0 tx=0).stop 2023-10-16T10:05:58.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.413+0000 7f48789a9640 1 -- 172.21.15.84:0/1588322623 shutdown_connections 2023-10-16T10:05:58.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.413+0000 7f48789a9640 1 --2- 172.21.15.84:0/1588322623 >> v2:172.21.15.157:3300/0 conn(0x7f487406c140 0x7f487406c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:58.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.413+0000 7f48789a9640 1 --2- 172.21.15.84:0/1588322623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4874159ab0 0x7f48740ce370 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:58.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.413+0000 7f48789a9640 1 -- 172.21.15.84:0/1588322623 >> 172.21.15.84:0/1588322623 conn(0x7f48740ce9b0 msgr2=0x7f48740cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:58.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.413+0000 7f48789a9640 1 -- 172.21.15.84:0/1588322623 shutdown_connections 2023-10-16T10:05:58.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.413+0000 7f48789a9640 1 -- 172.21.15.84:0/1588322623 wait complete. 2023-10-16T10:05:58.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.415+0000 7f48789a9640 1 Processor -- start 2023-10-16T10:05:58.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.415+0000 7f48789a9640 1 -- start start 2023-10-16T10:05:58.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.416+0000 7f48789a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f487406c140 0x7f48740c4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:58.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.416+0000 7f48789a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4874159ab0 0x7f48740c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:58.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.416+0000 7f48789a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4874159e90 0x7f48741583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:58.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.416+0000 7f48789a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48740c8d30 con 0x7f4874159e90 2023-10-16T10:05:58.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.416+0000 7f48789a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48740c4fa0 con 0x7f4874159ab0 2023-10-16T10:05:58.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.416+0000 7f48789a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48740c4700 con 0x7f487406c140 2023-10-16T10:05:58.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.416+0000 7f4872d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4874159e90 0x7f48741583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:58.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.416+0000 7f4872d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4874159e90 0x7f48741583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54574/0 (socket says 172.21.15.84:54574) 2023-10-16T10:05:58.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.417+0000 7f4872d9d640 1 -- 172.21.15.84:0/2125830040 learned_addr learned my addr 172.21.15.84:0/2125830040 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:58.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.417+0000 7f487259c640 1 --2- 172.21.15.84:0/2125830040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f487406c140 0x7f48740c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:58.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.417+0000 7f4872d9d640 1 -- 172.21.15.84:0/2125830040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f487406c140 msgr2=0x7f48740c4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:58.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.417+0000 7f4872d9d640 1 --2- 172.21.15.84:0/2125830040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f487406c140 0x7f48740c4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:58.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.417+0000 7f4872d9d640 1 -- 172.21.15.84:0/2125830040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4874159ab0 msgr2=0x7f48740c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:58.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.417+0000 7f4872d9d640 1 --2- 172.21.15.84:0/2125830040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4874159ab0 0x7f48740c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:58.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.417+0000 7f4872d9d640 1 -- 172.21.15.84:0/2125830040 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4864009400 con 0x7f4874159e90 2023-10-16T10:05:58.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.417+0000 7f4872d9d640 1 --2- 172.21.15.84:0/2125830040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4874159e90 0x7f48741583c0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f486c00c7c0 tx=0x7f486c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:58.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.418+0000 7f486affd640 1 -- 172.21.15.84:0/2125830040 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c004230 con 0x7f4874159e90 2023-10-16T10:05:58.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.418+0000 7f486affd640 1 -- 172.21.15.84:0/2125830040 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c004390 con 0x7f4874159e90 2023-10-16T10:05:58.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.418+0000 7f486affd640 1 -- 172.21.15.84:0/2125830040 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c007860 con 0x7f4874159e90 2023-10-16T10:05:58.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.418+0000 7f48789a9640 1 -- 172.21.15.84:0/2125830040 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48740c49e0 con 0x7f4874159e90 2023-10-16T10:05:58.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.418+0000 7f48789a9640 1 -- 172.21.15.84:0/2125830040 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48741f0600 con 0x7f4874159e90 2023-10-16T10:05:58.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.419+0000 7f48789a9640 1 -- 172.21.15.84:0/2125830040 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48741f0460 con 0x7f4874159e90 2023-10-16T10:05:58.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.420+0000 7f486affd640 1 -- 172.21.15.84:0/2125830040 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f486c0079c0 con 0x7f4874159e90 2023-10-16T10:05:58.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.421+0000 7f486affd640 1 --2- 172.21.15.84:0/2125830040 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f485406af00 0x7f485406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:58.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.421+0000 7f486affd640 1 -- 172.21.15.84:0/2125830040 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5752+0+0 (secure 0 0 0) 0x7f486c089ac0 con 0x7f4874159e90 2023-10-16T10:05:58.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.421+0000 7f487259c640 1 --2- 172.21.15.84:0/2125830040 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f485406af00 0x7f485406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:58.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.422+0000 7f487259c640 1 --2- 172.21.15.84:0/2125830040 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f485406af00 0x7f485406d3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4864035000 tx=0x7f4864008030 comp rx=0 tx=0).ready entity=mgr.7284 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:58.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.425+0000 7f486affd640 1 -- 172.21.15.84:0/2125830040 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f48741f0460 con 0x7f4874159e90 2023-10-16T10:05:58.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:58.672+0000 7f48789a9640 1 -- 172.21.15.84:0/2125830040 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f48741f0460 con 0x7f4874159e90 2023-10-16T10:05:59.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.438+0000 7f486affd640 1 -- 172.21.15.84:0/2125830040 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v49) v1 ==== 92+0+0 (secure 0 0 0) 0x7f48741f0460 con 0x7f4874159e90 2023-10-16T10:05:59.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.442+0000 7f48789a9640 1 -- 172.21.15.84:0/2125830040 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f485406af00 msgr2=0x7f485406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:59.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.442+0000 7f48789a9640 1 --2- 172.21.15.84:0/2125830040 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f485406af00 0x7f485406d3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4864035000 tx=0x7f4864008030 comp rx=0 tx=0).stop 2023-10-16T10:05:59.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.442+0000 7f48789a9640 1 -- 172.21.15.84:0/2125830040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4874159e90 msgr2=0x7f48741583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:59.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.442+0000 7f48789a9640 1 --2- 172.21.15.84:0/2125830040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4874159e90 0x7f48741583c0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f486c00c7c0 tx=0x7f486c00cc80 comp rx=0 tx=0).stop 2023-10-16T10:05:59.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.442+0000 7f48789a9640 1 -- 172.21.15.84:0/2125830040 shutdown_connections 2023-10-16T10:05:59.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.442+0000 7f48789a9640 1 --2- 172.21.15.84:0/2125830040 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7f485406af00 0x7f485406d3c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:59.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.442+0000 7f48789a9640 1 --2- 172.21.15.84:0/2125830040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4874159e90 0x7f48741583c0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:59.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.442+0000 7f48789a9640 1 --2- 172.21.15.84:0/2125830040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4874159ab0 0x7f48740c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:59.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.442+0000 7f48789a9640 1 --2- 172.21.15.84:0/2125830040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f487406c140 0x7f48740c4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:59.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.442+0000 7f48789a9640 1 -- 172.21.15.84:0/2125830040 >> 172.21.15.84:0/2125830040 conn(0x7f48740ce9b0 msgr2=0x7f487406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:59.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.442+0000 7f48789a9640 1 -- 172.21.15.84:0/2125830040 shutdown_connections 2023-10-16T10:05:59.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.442+0000 7f48789a9640 1 -- 172.21.15.84:0/2125830040 wait complete. 2023-10-16T10:05:59.466 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T10:05:59.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.669+0000 7fb135b01640 1 Processor -- start 2023-10-16T10:05:59.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.670+0000 7fb135b01640 1 -- start start 2023-10-16T10:05:59.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.671+0000 7fb135b01640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb1300c9e10 0x7fb13006ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:59.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.671+0000 7fb135b01640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1300ad8b0 0x7fb1300704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:59.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.671+0000 7fb135b01640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13006d7e0 con 0x7fb130159c60 2023-10-16T10:05:59.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.671+0000 7fb135b01640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1300c8e90 con 0x7fb1300c9e10 2023-10-16T10:05:59.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.671+0000 7fb135b01640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13006fb00 con 0x7fb1300ad8b0 2023-10-16T10:05:59.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.671+0000 7fb127fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb1300c9e10 0x7fb13006ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:59.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.671+0000 7fb127fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb1300c9e10 0x7fb13006ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54602/0 (socket says 172.21.15.84:54602) 2023-10-16T10:05:59.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.671+0000 7fb127fff640 1 -- 172.21.15.84:0/119992488 learned_addr learned my addr 172.21.15.84:0/119992488 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:59.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.671+0000 7fb12f7fe640 1 --1- 172.21.15.84:0/119992488 >> v1:172.21.15.84:6789/0 conn(0x7fb130159c60 0x7fb1300c9700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35250/0 (socket says 172.21.15.84:35250) 2023-10-16T10:05:59.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.672+0000 7fb12ffff640 1 --2- 172.21.15.84:0/119992488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1300ad8b0 0x7fb1300704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:59.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.672+0000 7fb127fff640 1 -- 172.21.15.84:0/119992488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1300ad8b0 msgr2=0x7fb1300704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:05:59.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.672+0000 7fb127fff640 1 --2- 172.21.15.84:0/119992488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1300ad8b0 0x7fb1300704f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:59.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.672+0000 7fb127fff640 1 -- 172.21.15.84:0/119992488 >> v1:172.21.15.84:6789/0 conn(0x7fb130159c60 legacy=0x7fb1300c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:59.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.672+0000 7fb127fff640 1 -- 172.21.15.84:0/119992488 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13006b8c0 con 0x7fb1300c9e10 2023-10-16T10:05:59.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.673+0000 7fb127fff640 1 --2- 172.21.15.84:0/119992488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1300c9e10 0x7fb13006ffb0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb118009870 tx=0x7fb11802ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38b945f93492b96 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:59.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.673+0000 7fb12effd640 1 -- 172.21.15.84:0/119992488 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11802f830 con 0x7fb1300c9e10 2023-10-16T10:05:59.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.673+0000 7fb12effd640 1 -- 172.21.15.84:0/119992488 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11802f990 con 0x7fb1300c9e10 2023-10-16T10:05:59.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.673+0000 7fb12effd640 1 -- 172.21.15.84:0/119992488 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11802fd20 con 0x7fb1300c9e10 2023-10-16T10:05:59.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.674+0000 7fb135b01640 1 -- 172.21.15.84:0/119992488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1300c9e10 msgr2=0x7fb13006ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:59.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.674+0000 7fb135b01640 1 --2- 172.21.15.84:0/119992488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1300c9e10 0x7fb13006ffb0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb118009870 tx=0x7fb11802ba10 comp rx=0 tx=0).stop 2023-10-16T10:05:59.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.674+0000 7fb135b01640 1 -- 172.21.15.84:0/119992488 shutdown_connections 2023-10-16T10:05:59.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.674+0000 7fb135b01640 1 --2- 172.21.15.84:0/119992488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1300ad8b0 0x7fb1300704f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:59.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.674+0000 7fb135b01640 1 --2- 172.21.15.84:0/119992488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1300c9e10 0x7fb13006ffb0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:59.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.674+0000 7fb135b01640 1 -- 172.21.15.84:0/119992488 >> 172.21.15.84:0/119992488 conn(0x7fb1300ce840 msgr2=0x7fb1300cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:59.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.674+0000 7fb135b01640 1 -- 172.21.15.84:0/119992488 shutdown_connections 2023-10-16T10:05:59.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.674+0000 7fb135b01640 1 -- 172.21.15.84:0/119992488 wait complete. 2023-10-16T10:05:59.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.676+0000 7fb135b01640 1 Processor -- start 2023-10-16T10:05:59.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.676+0000 7fb135b01640 1 -- start start 2023-10-16T10:05:59.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.677+0000 7fb135b01640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1300ad8b0 0x7fb1300c3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:59.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.677+0000 7fb135b01640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1300c9e10 0x7fb1300ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:59.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.677+0000 7fb135b01640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb130159c60 0x7fb1300acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:59.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.677+0000 7fb135b01640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1300c2510 con 0x7fb130159c60 2023-10-16T10:05:59.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.677+0000 7fb135b01640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1300c27b0 con 0x7fb1300c9e10 2023-10-16T10:05:59.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.677+0000 7fb135b01640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1300c2a90 con 0x7fb1300ad8b0 2023-10-16T10:05:59.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.677+0000 7fb12ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb130159c60 0x7fb1300acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:59.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.677+0000 7fb12ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb130159c60 0x7fb1300acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54608/0 (socket says 172.21.15.84:54608) 2023-10-16T10:05:59.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.677+0000 7fb12ffff640 1 -- 172.21.15.84:0/96664717 learned_addr learned my addr 172.21.15.84:0/96664717 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:05:59.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.677+0000 7fb12f7fe640 1 --2- 172.21.15.84:0/96664717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1300ad8b0 0x7fb1300c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:59.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.677+0000 7fb127fff640 1 --2- 172.21.15.84:0/96664717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1300c9e10 0x7fb1300ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:05:59.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.677+0000 7fb12ffff640 1 -- 172.21.15.84:0/96664717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1300ad8b0 msgr2=0x7fb1300c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:59.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.677+0000 7fb12ffff640 1 --2- 172.21.15.84:0/96664717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1300ad8b0 0x7fb1300c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:59.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.677+0000 7fb12ffff640 1 -- 172.21.15.84:0/96664717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1300c9e10 msgr2=0x7fb1300ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:59.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.677+0000 7fb12ffff640 1 --2- 172.21.15.84:0/96664717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1300c9e10 0x7fb1300ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:59.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.677+0000 7fb12ffff640 1 -- 172.21.15.84:0/96664717 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1180094d0 con 0x7fb130159c60 2023-10-16T10:05:59.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.677+0000 7fb12f7fe640 1 --2- 172.21.15.84:0/96664717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1300ad8b0 0x7fb1300c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:59.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.678+0000 7fb127fff640 1 --2- 172.21.15.84:0/96664717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1300c9e10 0x7fb1300ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:05:59.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.678+0000 7fb12ffff640 1 --2- 172.21.15.84:0/96664717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb130159c60 0x7fb1300acdd0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb12800cd20 tx=0x7fb12800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:05:59.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.678+0000 7fb12cff9640 1 -- 172.21.15.84:0/96664717 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb128003b90 con 0x7fb130159c60 2023-10-16T10:05:59.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.678+0000 7fb12cff9640 1 -- 172.21.15.84:0/96664717 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb128003cf0 con 0x7fb130159c60 2023-10-16T10:05:59.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.678+0000 7fb135b01640 1 -- 172.21.15.84:0/96664717 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1300c2d70 con 0x7fb130159c60 2023-10-16T10:05:59.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.678+0000 7fb12cff9640 1 -- 172.21.15.84:0/96664717 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb128004240 con 0x7fb130159c60 2023-10-16T10:05:59.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.678+0000 7fb135b01640 1 -- 172.21.15.84:0/96664717 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1300cb150 con 0x7fb130159c60 2023-10-16T10:05:59.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.680+0000 7fb135b01640 1 -- 172.21.15.84:0/96664717 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1300caf20 con 0x7fb130159c60 2023-10-16T10:05:59.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.680+0000 7fb12cff9640 1 -- 172.21.15.84:0/96664717 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 49) v1 ==== 89292+0+0 (secure 0 0 0) 0x7fb1280043a0 con 0x7fb130159c60 2023-10-16T10:05:59.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.681+0000 7fb12cff9640 1 --2- 172.21.15.84:0/96664717 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fb11406af50 0x7fb11406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:05:59.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.681+0000 7fb12cff9640 1 -- 172.21.15.84:0/96664717 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5752+0+0 (secure 0 0 0) 0x7fb128012070 con 0x7fb130159c60 2023-10-16T10:05:59.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.681+0000 7fb12f7fe640 1 -- 172.21.15.84:0/96664717 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fb11406af50 msgr2=0x7fb11406d410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3250823028 2023-10-16T10:05:59.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.681+0000 7fb12f7fe640 1 --2- 172.21.15.84:0/96664717 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fb11406af50 0x7fb11406d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:05:59.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.685+0000 7fb12cff9640 1 -- 172.21.15.84:0/96664717 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb1280550f0 con 0x7fb130159c60 2023-10-16T10:05:59.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.882+0000 7fb12f7fe640 1 -- 172.21.15.84:0/96664717 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fb11406af50 msgr2=0x7fb11406d410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3250823028 2023-10-16T10:05:59.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.882+0000 7fb12f7fe640 1 --2- 172.21.15.84:0/96664717 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fb11406af50 0x7fb11406d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:05:59.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.906+0000 7fb135b01640 1 -- 172.21.15.84:0/96664717 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fb1300c3290 con 0x7fb130159c60 2023-10-16T10:05:59.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.913+0000 7fb12cff9640 1 -- 172.21.15.84:0/96664717 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb1300c3290 con 0x7fb130159c60 2023-10-16T10:05:59.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.915+0000 7fb135b01640 1 -- 172.21.15.84:0/96664717 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fb11406af50 msgr2=0x7fb11406d410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:05:59.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.915+0000 7fb135b01640 1 --2- 172.21.15.84:0/96664717 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fb11406af50 0x7fb11406d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:59.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.915+0000 7fb135b01640 1 -- 172.21.15.84:0/96664717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb130159c60 msgr2=0x7fb1300acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:05:59.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.915+0000 7fb135b01640 1 --2- 172.21.15.84:0/96664717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb130159c60 0x7fb1300acdd0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb12800cd20 tx=0x7fb12800a660 comp rx=0 tx=0).stop 2023-10-16T10:05:59.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.915+0000 7fb135b01640 1 -- 172.21.15.84:0/96664717 shutdown_connections 2023-10-16T10:05:59.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.915+0000 7fb135b01640 1 --2- 172.21.15.84:0/96664717 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fb11406af50 0x7fb11406d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:59.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.915+0000 7fb135b01640 1 --2- 172.21.15.84:0/96664717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb130159c60 0x7fb1300acdd0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:59.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.915+0000 7fb135b01640 1 --2- 172.21.15.84:0/96664717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1300c9e10 0x7fb1300ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:59.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.915+0000 7fb135b01640 1 --2- 172.21.15.84:0/96664717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1300ad8b0 0x7fb1300c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:05:59.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.915+0000 7fb135b01640 1 -- 172.21.15.84:0/96664717 >> 172.21.15.84:0/96664717 conn(0x7fb1300ce840 msgr2=0x7fb1300cc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:05:59.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.916+0000 7fb135b01640 1 -- 172.21.15.84:0/96664717 shutdown_connections 2023-10-16T10:05:59.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:05:59.916+0000 7fb135b01640 1 -- 172.21.15.84:0/96664717 wait complete. 2023-10-16T10:05:59.933 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T10:05:59.967 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:05:59.965+0000 7f246983f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:06:00.004 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:00.002+0000 7f8d25a3a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:06:00.021 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:00.020+0000 7f246983f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:06:00.060 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:00.059+0000 7f8d25a3a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:06:00.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.134+0000 7fc279b33640 1 Processor -- start 2023-10-16T10:06:00.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.134+0000 7fc279b33640 1 -- start start 2023-10-16T10:06:00.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.135+0000 7fc279b33640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc2740cf710 0x7fc2740b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:06:00.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.136+0000 7fc279b33640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2740d18c0 con 0x7fc2740b6cd0 2023-10-16T10:06:00.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.136+0000 7fc279b33640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2740d3980 con 0x7fc2740cfaf0 2023-10-16T10:06:00.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.136+0000 7fc279b33640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2740d3c60 con 0x7fc2740cf710 2023-10-16T10:06:00.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.136+0000 7fc2737fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc2740cf710 0x7fc2740b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:06:00.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.136+0000 7fc2737fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc2740cf710 0x7fc2740b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54624/0 (socket says 172.21.15.84:54624) 2023-10-16T10:06:00.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.136+0000 7fc2737fe640 1 -- 172.21.15.84:0/2407915094 learned_addr learned my addr 172.21.15.84:0/2407915094 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:06:00.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.136+0000 7fc2737fe640 1 -- 172.21.15.84:0/2407915094 >> v1:172.21.15.157:6789/0 conn(0x7fc2740cfaf0 legacy=0x7fc2740d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:06:00.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.136+0000 7fc2737fe640 1 -- 172.21.15.84:0/2407915094 >> v1:172.21.15.84:6789/0 conn(0x7fc2740b6cd0 legacy=0x7fc2740d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:06:00.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.136+0000 7fc2737fe640 1 -- 172.21.15.84:0/2407915094 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2740ce630 con 0x7fc2740cf710 2023-10-16T10:06:00.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.136+0000 7fc272ffd640 1 -- 172.21.15.84:0/2407915094 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328236146 0 0) 0x7fc2740d18c0 con 0x7fc2740b6cd0 2023-10-16T10:06:00.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.137+0000 7fc2737fe640 1 --2- 172.21.15.84:0/2407915094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2740cf710 0x7fc2740b2d10 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fc264009850 tx=0x7fc26402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b06f6163ff2be82 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:06:00.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.138+0000 7fc272ffd640 1 -- 172.21.15.84:0/2407915094 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264004230 con 0x7fc2740cf710 2023-10-16T10:06:00.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.138+0000 7fc272ffd640 1 -- 172.21.15.84:0/2407915094 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc264004390 con 0x7fc2740cf710 2023-10-16T10:06:00.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.138+0000 7fc272ffd640 1 -- 172.21.15.84:0/2407915094 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264003810 con 0x7fc2740cf710 2023-10-16T10:06:00.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.138+0000 7fc279b33640 1 -- 172.21.15.84:0/2407915094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2740cf710 msgr2=0x7fc2740b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:06:00.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.138+0000 7fc279b33640 1 --2- 172.21.15.84:0/2407915094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2740cf710 0x7fc2740b2d10 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fc264009850 tx=0x7fc26402b8f0 comp rx=0 tx=0).stop 2023-10-16T10:06:00.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.138+0000 7fc279b33640 1 -- 172.21.15.84:0/2407915094 shutdown_connections 2023-10-16T10:06:00.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.138+0000 7fc279b33640 1 --2- 172.21.15.84:0/2407915094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2740cf710 0x7fc2740b2d10 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:00.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.138+0000 7fc279b33640 1 -- 172.21.15.84:0/2407915094 >> 172.21.15.84:0/2407915094 conn(0x7fc2740d2cc0 msgr2=0x7fc2740d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:06:00.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.139+0000 7fc279b33640 1 -- 172.21.15.84:0/2407915094 shutdown_connections 2023-10-16T10:06:00.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.139+0000 7fc279b33640 1 -- 172.21.15.84:0/2407915094 wait complete. 2023-10-16T10:06:00.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.140+0000 7fc279b33640 1 Processor -- start 2023-10-16T10:06:00.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.140+0000 7fc279b33640 1 -- start start 2023-10-16T10:06:00.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.141+0000 7fc279b33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2740b6cd0 0x7fc274142860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:06:00.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.141+0000 7fc279b33640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2740cf710 0x7fc274142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:06:00.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.141+0000 7fc279b33640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2740cfaf0 0x7fc2741e5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:06:00.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.141+0000 7fc279b33640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2741e6c70 con 0x7fc2740cf710 2023-10-16T10:06:00.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.141+0000 7fc279b33640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2741f2680 con 0x7fc2740b6cd0 2023-10-16T10:06:00.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.141+0000 7fc279b33640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc274160730 con 0x7fc2740cfaf0 2023-10-16T10:06:00.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.141+0000 7fc26bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2740cf710 0x7fc274142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:06:00.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.141+0000 7fc26bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2740cf710 0x7fc274142da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54638/0 (socket says 172.21.15.84:54638) 2023-10-16T10:06:00.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.141+0000 7fc26bfff640 1 -- 172.21.15.84:0/1855641703 learned_addr learned my addr 172.21.15.84:0/1855641703 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:06:00.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.142+0000 7fc2737fe640 1 --2- 172.21.15.84:0/1855641703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2740b6cd0 0x7fc274142860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:06:00.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.142+0000 7fc26bfff640 1 -- 172.21.15.84:0/1855641703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2740cfaf0 msgr2=0x7fc2741e5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:06:00.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.142+0000 7fc273fff640 1 --2- 172.21.15.84:0/1855641703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2740cfaf0 0x7fc2741e5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:06:00.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.142+0000 7fc26bfff640 1 --2- 172.21.15.84:0/1855641703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2740cfaf0 0x7fc2741e5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:00.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.142+0000 7fc26bfff640 1 -- 172.21.15.84:0/1855641703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2740b6cd0 msgr2=0x7fc274142860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:06:00.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.142+0000 7fc26bfff640 1 --2- 172.21.15.84:0/1855641703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2740b6cd0 0x7fc274142860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:00.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.142+0000 7fc26bfff640 1 -- 172.21.15.84:0/1855641703 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc264009460 con 0x7fc2740cf710 2023-10-16T10:06:00.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.142+0000 7fc2737fe640 1 --2- 172.21.15.84:0/1855641703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2740b6cd0 0x7fc274142860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:06:00.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.142+0000 7fc273fff640 1 --2- 172.21.15.84:0/1855641703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2740cfaf0 0x7fc2741e5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:06:00.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.142+0000 7fc26bfff640 1 --2- 172.21.15.84:0/1855641703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2740cf710 0x7fc274142da0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fc25c00b3f0 tx=0x7fc25c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:06:00.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.142+0000 7fc270ff9640 1 -- 172.21.15.84:0/1855641703 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c011830 con 0x7fc2740cf710 2023-10-16T10:06:00.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.142+0000 7fc270ff9640 1 -- 172.21.15.84:0/1855641703 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25c011990 con 0x7fc2740cf710 2023-10-16T10:06:00.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.142+0000 7fc270ff9640 1 -- 172.21.15.84:0/1855641703 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c011d40 con 0x7fc2740cf710 2023-10-16T10:06:00.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.143+0000 7fc279b33640 1 -- 172.21.15.84:0/1855641703 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2741e6620 con 0x7fc2740cf710 2023-10-16T10:06:00.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.143+0000 7fc279b33640 1 -- 172.21.15.84:0/1855641703 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2741435d0 con 0x7fc2740cf710 2023-10-16T10:06:00.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.144+0000 7fc279b33640 1 -- 172.21.15.84:0/1855641703 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27404e160 con 0x7fc2740cf710 2023-10-16T10:06:00.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.145+0000 7fc270ff9640 1 -- 172.21.15.84:0/1855641703 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 49) v1 ==== 89292+0+0 (secure 0 0 0) 0x7fc25c01a400 con 0x7fc2740cf710 2023-10-16T10:06:00.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.146+0000 7fc270ff9640 1 --2- 172.21.15.84:0/1855641703 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc25406af80 0x7fc25406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:06:00.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.146+0000 7fc270ff9640 1 -- 172.21.15.84:0/1855641703 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5752+0+0 (secure 0 0 0) 0x7fc25c089600 con 0x7fc2740cf710 2023-10-16T10:06:00.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.146+0000 7fc2737fe640 1 -- 172.21.15.84:0/1855641703 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc25406af80 msgr2=0x7fc25406d440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3250823028 2023-10-16T10:06:00.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.146+0000 7fc2737fe640 1 --2- 172.21.15.84:0/1855641703 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc25406af80 0x7fc25406d440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:06:00.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.150+0000 7fc270ff9640 1 -- 172.21.15.84:0/1855641703 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc25c0545a0 con 0x7fc2740cf710 2023-10-16T10:06:00.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.332+0000 7fc279b33640 1 -- 172.21.15.84:0/1855641703 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fc2740d1200 con 0x7fc2740cf710 2023-10-16T10:06:00.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.339+0000 7fc270ff9640 1 -- 172.21.15.84:0/1855641703 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v36)=0 v36) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc25c057bc0 con 0x7fc2740cf710 2023-10-16T10:06:00.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.343+0000 7fc279b33640 1 -- 172.21.15.84:0/1855641703 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc25406af80 msgr2=0x7fc25406d440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:06:00.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.343+0000 7fc279b33640 1 --2- 172.21.15.84:0/1855641703 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc25406af80 0x7fc25406d440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:00.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.343+0000 7fc279b33640 1 -- 172.21.15.84:0/1855641703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2740cf710 msgr2=0x7fc274142da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:06:00.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.343+0000 7fc279b33640 1 --2- 172.21.15.84:0/1855641703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2740cf710 0x7fc274142da0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fc25c00b3f0 tx=0x7fc25c00b8b0 comp rx=0 tx=0).stop 2023-10-16T10:06:00.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.343+0000 7fc279b33640 1 -- 172.21.15.84:0/1855641703 shutdown_connections 2023-10-16T10:06:00.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.343+0000 7fc279b33640 1 --2- 172.21.15.84:0/1855641703 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc25406af80 0x7fc25406d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:00.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.343+0000 7fc279b33640 1 --2- 172.21.15.84:0/1855641703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2740cfaf0 0x7fc2741e5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:00.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.343+0000 7fc279b33640 1 --2- 172.21.15.84:0/1855641703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2740cf710 0x7fc274142da0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:00.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.343+0000 7fc279b33640 1 --2- 172.21.15.84:0/1855641703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2740b6cd0 0x7fc274142860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:00.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.343+0000 7fc279b33640 1 -- 172.21.15.84:0/1855641703 >> 172.21.15.84:0/1855641703 conn(0x7fc2740d2cc0 msgr2=0x7fc2740b8170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:06:00.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.343+0000 7fc279b33640 1 -- 172.21.15.84:0/1855641703 shutdown_connections 2023-10-16T10:06:00.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.343+0000 7fc279b33640 1 -- 172.21.15.84:0/1855641703 wait complete. 2023-10-16T10:06:00.363 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /bad-path 1m 2023-10-16T10:06:00.473 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:00.472+0000 7f246983f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:06:00.526 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:00.525+0000 7f246983f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:06:00.528 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:00.526+0000 7f8d25a3a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:06:00.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.562+0000 7fc4cf7a9640 1 Processor -- start 2023-10-16T10:06:00.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.562+0000 7fc4cf7a9640 1 -- start start 2023-10-16T10:06:00.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.564+0000 7fc4cf7a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c815d330 0x7fc4c81618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:06:00.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.564+0000 7fc4cf7a9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4c80661c0 0x7fc4c80ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:06:00.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.564+0000 7fc4cf7a9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c80d0890 con 0x7fc4c806c7d0 2023-10-16T10:06:00.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.564+0000 7fc4cf7a9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c806b1e0 con 0x7fc4c80661c0 2023-10-16T10:06:00.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.564+0000 7fc4cf7a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c80d05b0 con 0x7fc4c815d330 2023-10-16T10:06:00.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.564+0000 7fc4cd545640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c815d330 0x7fc4c81618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:06:00.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.564+0000 7fc4cdd46640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4c80661c0 0x7fc4c80ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:06:00.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.565+0000 7fc4cd545640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c815d330 0x7fc4c81618b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:60764/0 (socket says 172.21.15.84:60764) 2023-10-16T10:06:00.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.565+0000 7fc4cd545640 1 -- 172.21.15.84:0/1519131067 learned_addr learned my addr 172.21.15.84:0/1519131067 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:06:00.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.565+0000 7fc4cd545640 1 -- 172.21.15.84:0/1519131067 >> v2:172.21.15.157:3300/0 conn(0x7fc4c80661c0 msgr2=0x7fc4c80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:06:00.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.565+0000 7fc4cd545640 1 --2- 172.21.15.84:0/1519131067 >> v2:172.21.15.157:3300/0 conn(0x7fc4c80661c0 0x7fc4c80ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:00.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.565+0000 7fc4cd545640 1 -- 172.21.15.84:0/1519131067 >> v1:172.21.15.157:6789/0 conn(0x7fc4c806c7d0 legacy=0x7fc4c8065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:06:00.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.565+0000 7fc4cd545640 1 -- 172.21.15.84:0/1519131067 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c8160dd0 con 0x7fc4c815d330 2023-10-16T10:06:00.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.566+0000 7fc4cd545640 1 --2- 172.21.15.84:0/1519131067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c815d330 0x7fc4c81618b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009e50 tx=0x7fc4c402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5f120df5b895b75 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:06:00.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.567+0000 7fc4bffff640 1 -- 172.21.15.84:0/1519131067 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c402ca70 con 0x7fc4c815d330 2023-10-16T10:06:00.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.567+0000 7fc4bffff640 1 -- 172.21.15.84:0/1519131067 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c402cbd0 con 0x7fc4c815d330 2023-10-16T10:06:00.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.567+0000 7fc4bffff640 1 -- 172.21.15.84:0/1519131067 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c40343f0 con 0x7fc4c815d330 2023-10-16T10:06:00.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.567+0000 7fc4cf7a9640 1 -- 172.21.15.84:0/1519131067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c815d330 msgr2=0x7fc4c81618b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:06:00.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.567+0000 7fc4cf7a9640 1 --2- 172.21.15.84:0/1519131067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c815d330 0x7fc4c81618b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009e50 tx=0x7fc4c402bed0 comp rx=0 tx=0).stop 2023-10-16T10:06:00.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.568+0000 7fc4cf7a9640 1 -- 172.21.15.84:0/1519131067 shutdown_connections 2023-10-16T10:06:00.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.568+0000 7fc4cf7a9640 1 --2- 172.21.15.84:0/1519131067 >> v2:172.21.15.157:3300/0 conn(0x7fc4c80661c0 0x7fc4c80ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:00.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.568+0000 7fc4cf7a9640 1 --2- 172.21.15.84:0/1519131067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c815d330 0x7fc4c81618b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:00.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.568+0000 7fc4cf7a9640 1 -- 172.21.15.84:0/1519131067 >> 172.21.15.84:0/1519131067 conn(0x7fc4c80d0ab0 msgr2=0x7fc4c80d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:06:00.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.568+0000 7fc4cf7a9640 1 -- 172.21.15.84:0/1519131067 shutdown_connections 2023-10-16T10:06:00.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.568+0000 7fc4cf7a9640 1 -- 172.21.15.84:0/1519131067 wait complete. 2023-10-16T10:06:00.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.569+0000 7fc4cf7a9640 1 Processor -- start 2023-10-16T10:06:00.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.569+0000 7fc4cf7a9640 1 -- start start 2023-10-16T10:06:00.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.570+0000 7fc4cf7a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4c80661c0 0x7fc4c8155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:06:00.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.570+0000 7fc4cf7a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c806c7d0 0x7fc4c815e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:06:00.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.570+0000 7fc4cf7a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c815d330 0x7fc4c8147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:06:00.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.570+0000 7fc4cf7a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c8148490 con 0x7fc4c80661c0 2023-10-16T10:06:00.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.570+0000 7fc4cf7a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c815c8c0 con 0x7fc4c806c7d0 2023-10-16T10:06:00.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.570+0000 7fc4cf7a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c815e750 con 0x7fc4c815d330 2023-10-16T10:06:00.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.571+0000 7fc4cd545640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4c80661c0 0x7fc4c8155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:06:00.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.571+0000 7fc4cd545640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4c80661c0 0x7fc4c8155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54654/0 (socket says 172.21.15.84:54654) 2023-10-16T10:06:00.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.571+0000 7fc4cd545640 1 -- 172.21.15.84:0/3824596202 learned_addr learned my addr 172.21.15.84:0/3824596202 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:06:00.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.571+0000 7fc4ccd44640 1 --2- 172.21.15.84:0/3824596202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c806c7d0 0x7fc4c815e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:06:00.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.571+0000 7fc4cd545640 1 -- 172.21.15.84:0/3824596202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c815d330 msgr2=0x7fc4c8147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:06:00.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.571+0000 7fc4cdd46640 1 --2- 172.21.15.84:0/3824596202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c815d330 0x7fc4c8147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:06:00.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.571+0000 7fc4cd545640 1 --2- 172.21.15.84:0/3824596202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c815d330 0x7fc4c8147f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:00.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.571+0000 7fc4cd545640 1 -- 172.21.15.84:0/3824596202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c806c7d0 msgr2=0x7fc4c815e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:06:00.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.571+0000 7fc4cd545640 1 --2- 172.21.15.84:0/3824596202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c806c7d0 0x7fc4c815e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:00.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.571+0000 7fc4cd545640 1 -- 172.21.15.84:0/3824596202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c4009b30 con 0x7fc4c80661c0 2023-10-16T10:06:00.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.571+0000 7fc4cd545640 1 --2- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4c80661c0 0x7fc4c8155230 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4000f80 tx=0x7fc4c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:06:00.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.572+0000 7fc4bdffb640 1 -- 172.21.15.84:0/3824596202 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4009510 con 0x7fc4c80661c0 2023-10-16T10:06:00.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.572+0000 7fc4bdffb640 1 -- 172.21.15.84:0/3824596202 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c403e450 con 0x7fc4c80661c0 2023-10-16T10:06:00.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.572+0000 7fc4bdffb640 1 -- 172.21.15.84:0/3824596202 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c403e800 con 0x7fc4c80661c0 2023-10-16T10:06:00.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.572+0000 7fc4cf7a9640 1 -- 172.21.15.84:0/3824596202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c815bf90 con 0x7fc4c80661c0 2023-10-16T10:06:00.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.572+0000 7fc4cf7a9640 1 -- 172.21.15.84:0/3824596202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c815c550 con 0x7fc4c80661c0 2023-10-16T10:06:00.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.573+0000 7fc4cf7a9640 1 -- 172.21.15.84:0/3824596202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c804e160 con 0x7fc4c80661c0 2023-10-16T10:06:00.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.574+0000 7fc4bdffb640 1 -- 172.21.15.84:0/3824596202 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 49) v1 ==== 89292+0+0 (secure 0 0 0) 0x7fc4c403e960 con 0x7fc4c80661c0 2023-10-16T10:06:00.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.575+0000 7fc4bdffb640 1 --2- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc4ac06af50 0x7fc4ac06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:06:00.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.575+0000 7fc4bdffb640 1 -- 172.21.15.84:0/3824596202 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5752+0+0 (secure 0 0 0) 0x7fc4c40ac4c0 con 0x7fc4c80661c0 2023-10-16T10:06:00.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.575+0000 7fc4ccd44640 1 -- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc4ac06af50 msgr2=0x7fc4ac06d410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3250823028 2023-10-16T10:06:00.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.575+0000 7fc4ccd44640 1 --2- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc4ac06af50 0x7fc4ac06d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:06:00.580 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:00.576+0000 7f246983f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:06:00.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.579+0000 7fc4bdffb640 1 -- 172.21.15.84:0/3824596202 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc4c4077460 con 0x7fc4c80661c0 2023-10-16T10:06:00.583 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:00.582+0000 7f8d25a3a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:06:00.636 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:00.635+0000 7f8d25a3a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:06:00.638 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:00.637+0000 7f246983f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:06:00.687 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:00.685+0000 7f246983f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:06:00.699 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:00.697+0000 7f8d25a3a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:06:00.749 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:00.748+0000 7f8d25a3a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:06:00.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.776+0000 7fc4ccd44640 1 -- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc4ac06af50 msgr2=0x7fc4ac06d410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3250823028 2023-10-16T10:06:00.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.776+0000 7fc4ccd44640 1 --2- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc4ac06af50 0x7fc4ac06d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:06:00.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:00.792+0000 7fc4cf7a9640 1 -- 172.21.15.84:0/3824596202 --> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/bad-path", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4c8161390 con 0x7fc4ac06af50 2023-10-16T10:06:00.797 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:00.796+0000 7f246983f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:06:00.859 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:00.857+0000 7f246983f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:06:00.862 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:00.860+0000 7f8d25a3a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:06:00.907 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:00.906+0000 7f246983f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:06:00.924 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:00.922+0000 7f8d25a3a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:06:00.965 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:00.963+0000 7f246983f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:06:00.973 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:00.971+0000 7f8d25a3a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:06:01.015 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:01.013+0000 7f246983f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:06:01.029 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:01.027+0000 7f8d25a3a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:06:01.079 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:01.078+0000 7f8d25a3a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:06:01.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:01.177+0000 7fc4ccd44640 1 -- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc4ac06af50 msgr2=0x7fc4ac06d410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3250823028 2023-10-16T10:06:01.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:01.177+0000 7fc4ccd44640 1 --2- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc4ac06af50 0x7fc4ac06d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T10:06:01.394 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:01.393+0000 7f246983f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:06:01.459 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:01.457+0000 7f8d25a3a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:06:01.555 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:01.553+0000 7f246983f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:06:01.616 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:01.615+0000 7f8d25a3a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:06:01.662 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:01.660+0000 7f246983f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:06:01.712 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:01.710+0000 7f246983f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:06:01.723 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:01.721+0000 7f8d25a3a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:06:01.773 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:01.771+0000 7f8d25a3a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:06:01.819 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:01.817+0000 7f246983f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:06:01.880 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:01.878+0000 7f8d25a3a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:06:01.925 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:01.924+0000 7f246983f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:06:01.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:01.978+0000 7fc4ccd44640 1 -- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc4ac06af50 msgr2=0x7fc4ac06d410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3250823028 2023-10-16T10:06:01.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:01.978+0000 7fc4ccd44640 1 --2- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc4ac06af50 0x7fc4ac06d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T10:06:01.985 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:01.984+0000 7f8d25a3a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:06:02.081 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:02.079+0000 7f246983f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:06:02.130 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:02.129+0000 7f246983f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:06:02.142 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:02.139+0000 7f8d25a3a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:06:02.189 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:02.188+0000 7f8d25a3a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:06:02.249 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:02.247+0000 7f246983f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:06:02.308 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:02.306+0000 7f8d25a3a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:06:03.061 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:03.060+0000 7f246983f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:06:03.134 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:03.133+0000 7f8d25a3a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:06:03.357 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:03.356+0000 7f246983f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:06:03.414 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:03.413+0000 7f246983f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:06:03.430 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:03.428+0000 7f8d25a3a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:06:03.486 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:03.484+0000 7f8d25a3a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:06:03.527 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:06:03.525+0000 7f246983f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:06:03.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:03.580+0000 7fc4ccd44640 1 -- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc4ac06af50 msgr2=0x7fc4ac06d410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3250823028 2023-10-16T10:06:03.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:03.580+0000 7fc4ccd44640 1 --2- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc4ac06af50 0x7fc4ac06d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T10:06:03.600 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:03.598+0000 7f8d25a3a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:06:03.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:03.616+0000 7fc4bdffb640 1 -- 172.21.15.84:0/3824596202 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 50) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fc4c4077830 con 0x7fc4c80661c0 2023-10-16T10:06:03.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:03.617+0000 7fc4bdffb640 1 -- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc4ac06af50 msgr2=0x7fc4ac06d410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:06:03.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:03.617+0000 7fc4bdffb640 1 --2- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:6832/3250823028,v1:172.21.15.84:6833/3250823028] conn(0x7fc4ac06af50 0x7fc4ac06d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:03.911 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:03.910+0000 7f8ca24f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:06:03.912 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:03.910+0000 7f8ca24f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:06:03.912 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:03.910+0000 7f8ca24f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:06:03.912 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:03.910+0000 7f8ca24f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:06:03.913 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:03.910+0000 7f8ca24f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:06:03.913 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:03.910+0000 7f8ca4d3d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:06:03.913 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:03.910+0000 7f8ca4d3d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:06:03.913 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:03.910+0000 7f8ca4d3d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:06:03.914 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:03.910+0000 7f8ca4d3d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:06:03.914 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:06:03.910+0000 7f8ca4d3d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:06:04.057 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:06:04.067 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:06:04.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:04.625+0000 7fc4bdffb640 1 -- 172.21.15.84:0/3824596202 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 51) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fc4c403ec10 con 0x7fc4c80661c0 2023-10-16T10:06:04.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:04.626+0000 7fc4bdffb640 1 --2- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fc4ac035b30 0x7fc4ac074ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:06:04.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:04.627+0000 7fc4bdffb640 1 -- 172.21.15.84:0/3824596202 --> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/bad-path", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4ac003890 con 0x7fc4ac035b30 2023-10-16T10:06:04.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:04.629+0000 7fc4ccd44640 1 --2- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fc4ac035b30 0x7fc4ac074ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:06:04.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:04.644+0000 7fc4ccd44640 1 --2- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fc4ac035b30 0x7fc4ac074ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80b6b90 tx=0x7fc4b8007d20 comp rx=0 tx=0).ready entity=mgr.7692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:06:04.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:04.832+0000 7fc4bdffb640 1 -- 172.21.15.84:0/3824596202 <== mgr.7692 v2:172.21.15.84:6832/985901542 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fc4b8001520 con 0x7fc4ac035b30 2023-10-16T10:06:04.834 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path /bad-path 2023-10-16T10:06:04.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:04.835+0000 7fc4cf7a9640 1 -- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fc4ac035b30 msgr2=0x7fc4ac074ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:06:04.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:04.835+0000 7fc4cf7a9640 1 --2- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fc4ac035b30 0x7fc4ac074ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80b6b90 tx=0x7fc4b8007d20 comp rx=0 tx=0).stop 2023-10-16T10:06:04.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:04.835+0000 7fc4cf7a9640 1 -- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4c80661c0 msgr2=0x7fc4c8155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:06:04.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:04.835+0000 7fc4cf7a9640 1 --2- 172.21.15.84:0/3824596202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4c80661c0 0x7fc4c8155230 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4000f80 tx=0x7fc4c4004a40 comp rx=0 tx=0).stop 2023-10-16T10:06:04.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:04.835+0000 7fc4cdd46640 1 -- 172.21.15.84:0/3824596202 reap_dead start 2023-10-16T10:06:04.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:04.835+0000 7fc4cf7a9640 1 -- 172.21.15.84:0/3824596202 shutdown_connections 2023-10-16T10:06:04.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:04.835+0000 7fc4cf7a9640 1 -- 172.21.15.84:0/3824596202 >> 172.21.15.84:0/3824596202 conn(0x7fc4c80d0ab0 msgr2=0x7fc4c8158750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:06:04.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:04.835+0000 7fc4cf7a9640 1 -- 172.21.15.84:0/3824596202 shutdown_connections 2023-10-16T10:06:04.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:04.835+0000 7fc4cf7a9640 1 -- 172.21.15.84:0/3824596202 wait complete. 2023-10-16T10:06:04.856 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2023-10-16T10:06:04.856 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /bad-path 2023-10-16T10:06:05.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.064+0000 7fb87f832640 1 Processor -- start 2023-10-16T10:06:05.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.064+0000 7fb87f832640 1 -- start start 2023-10-16T10:06:05.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.066+0000 7fb87f832640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb878159ab0 0x7fb8780ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:06:05.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.066+0000 7fb87f832640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb878070360 con 0x7fb87806c140 2023-10-16T10:06:05.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.066+0000 7fb87f832640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8780cd1f0 con 0x7fb878159e90 2023-10-16T10:06:05.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.066+0000 7fb87f832640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb87815d890 con 0x7fb878159ab0 2023-10-16T10:06:05.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.066+0000 7fb87d5ce640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb878159ab0 0x7fb8780ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:06:05.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.066+0000 7fb87d5ce640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb878159ab0 0x7fb8780ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54808/0 (socket says 172.21.15.84:54808) 2023-10-16T10:06:05.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.066+0000 7fb87d5ce640 1 -- 172.21.15.84:0/2440742583 learned_addr learned my addr 172.21.15.84:0/2440742583 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:06:05.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.066+0000 7fb87ddcf640 1 --1- 172.21.15.84:0/2440742583 >> v1:172.21.15.84:6789/0 conn(0x7fb87806c140 0x7fb87815dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35272/0 (socket says 172.21.15.84:35272) 2023-10-16T10:06:05.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.066+0000 7fb87d5ce640 1 -- 172.21.15.84:0/2440742583 >> v1:172.21.15.157:6789/0 conn(0x7fb878159e90 legacy=0x7fb87806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:06:05.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.066+0000 7fb87d5ce640 1 -- 172.21.15.84:0/2440742583 >> v1:172.21.15.84:6789/0 conn(0x7fb87806c140 legacy=0x7fb87815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:06:05.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.067+0000 7fb87d5ce640 1 -- 172.21.15.84:0/2440742583 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8780cd4d0 con 0x7fb878159ab0 2023-10-16T10:06:05.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.067+0000 7fb86ffff640 1 -- 172.21.15.84:0/2440742583 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3006101785 0 0) 0x7fb878070360 con 0x7fb87806c140 2023-10-16T10:06:05.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.068+0000 7fb87d5ce640 1 --2- 172.21.15.84:0/2440742583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb878159ab0 0x7fb8780ce370 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fb874009870 tx=0x7fb87402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0f1aada306d7cf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:06:05.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.068+0000 7fb86ffff640 1 -- 172.21.15.84:0/2440742583 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8740317e0 con 0x7fb878159ab0 2023-10-16T10:06:05.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.068+0000 7fb86ffff640 1 -- 172.21.15.84:0/2440742583 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb874031940 con 0x7fb878159ab0 2023-10-16T10:06:05.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.068+0000 7fb86ffff640 1 -- 172.21.15.84:0/2440742583 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874031cd0 con 0x7fb878159ab0 2023-10-16T10:06:05.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.069+0000 7fb87f832640 1 -- 172.21.15.84:0/2440742583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb878159ab0 msgr2=0x7fb8780ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:06:05.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.069+0000 7fb87f832640 1 --2- 172.21.15.84:0/2440742583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb878159ab0 0x7fb8780ce370 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fb874009870 tx=0x7fb87402b880 comp rx=0 tx=0).stop 2023-10-16T10:06:05.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.069+0000 7fb87f832640 1 -- 172.21.15.84:0/2440742583 shutdown_connections 2023-10-16T10:06:05.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.069+0000 7fb87f832640 1 --2- 172.21.15.84:0/2440742583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb878159ab0 0x7fb8780ce370 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:05.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.069+0000 7fb87f832640 1 -- 172.21.15.84:0/2440742583 >> 172.21.15.84:0/2440742583 conn(0x7fb8780ce9b0 msgr2=0x7fb8780cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:06:05.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.069+0000 7fb87f832640 1 -- 172.21.15.84:0/2440742583 shutdown_connections 2023-10-16T10:06:05.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.069+0000 7fb87f832640 1 -- 172.21.15.84:0/2440742583 wait complete. 2023-10-16T10:06:05.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.071+0000 7fb87f832640 1 Processor -- start 2023-10-16T10:06:05.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.071+0000 7fb87f832640 1 -- start start 2023-10-16T10:06:05.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.072+0000 7fb87f832640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87806c140 0x7fb8780c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:06:05.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.072+0000 7fb87f832640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb878159ab0 0x7fb87815d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:06:05.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.072+0000 7fb87f832640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb878159e90 0x7fb878155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:06:05.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.072+0000 7fb87f832640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87815af90 con 0x7fb87806c140 2023-10-16T10:06:05.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.072+0000 7fb87f832640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87815d560 con 0x7fb878159ab0 2023-10-16T10:06:05.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.072+0000 7fb87f832640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87815a330 con 0x7fb878159e90 2023-10-16T10:06:05.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.073+0000 7fb87d5ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87806c140 0x7fb8780c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:06:05.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.073+0000 7fb87d5ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87806c140 0x7fb8780c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54818/0 (socket says 172.21.15.84:54818) 2023-10-16T10:06:05.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.073+0000 7fb87d5ce640 1 -- 172.21.15.84:0/2768580375 learned_addr learned my addr 172.21.15.84:0/2768580375 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:06:05.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.073+0000 7fb87d5ce640 1 -- 172.21.15.84:0/2768580375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb878159e90 msgr2=0x7fb878155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:06:05.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.073+0000 7fb87d5ce640 1 --2- 172.21.15.84:0/2768580375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb878159e90 0x7fb878155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:05.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.073+0000 7fb87d5ce640 1 -- 172.21.15.84:0/2768580375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb878159ab0 msgr2=0x7fb87815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:06:05.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.073+0000 7fb87d5ce640 1 --2- 172.21.15.84:0/2768580375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb878159ab0 0x7fb87815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:05.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.073+0000 7fb87d5ce640 1 -- 172.21.15.84:0/2768580375 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8740094d0 con 0x7fb87806c140 2023-10-16T10:06:05.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.073+0000 7fb87d5ce640 1 --2- 172.21.15.84:0/2768580375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87806c140 0x7fb8780c8900 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fb874000c00 tx=0x7fb8740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:06:05.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.074+0000 7fb86dffb640 1 -- 172.21.15.84:0/2768580375 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874032900 con 0x7fb87806c140 2023-10-16T10:06:05.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.074+0000 7fb86dffb640 1 -- 172.21.15.84:0/2768580375 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb874032a80 con 0x7fb87806c140 2023-10-16T10:06:05.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.074+0000 7fb86dffb640 1 -- 172.21.15.84:0/2768580375 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874032e30 con 0x7fb87806c140 2023-10-16T10:06:05.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.074+0000 7fb87f832640 1 -- 172.21.15.84:0/2768580375 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb87815a5f0 con 0x7fb87806c140 2023-10-16T10:06:05.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.074+0000 7fb87f832640 1 -- 172.21.15.84:0/2768580375 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87815abb0 con 0x7fb87806c140 2023-10-16T10:06:05.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.075+0000 7fb87f832640 1 -- 172.21.15.84:0/2768580375 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87804e160 con 0x7fb87806c140 2023-10-16T10:06:05.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.076+0000 7fb86dffb640 1 -- 172.21.15.84:0/2768580375 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 51) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fb87404d3f0 con 0x7fb87806c140 2023-10-16T10:06:05.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.077+0000 7fb86dffb640 1 --2- 172.21.15.84:0/2768580375 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fb85c06ad40 0x7fb85c06d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:06:05.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.077+0000 7fb86dffb640 1 -- 172.21.15.84:0/2768580375 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fb8740ab3d0 con 0x7fb87806c140 2023-10-16T10:06:05.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.077+0000 7fb87cdcd640 1 --2- 172.21.15.84:0/2768580375 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fb85c06ad40 0x7fb85c06d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:06:05.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.078+0000 7fb87cdcd640 1 --2- 172.21.15.84:0/2768580375 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fb85c06ad40 0x7fb85c06d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb868007fd0 tx=0x7fb8680079a0 comp rx=0 tx=0).ready entity=mgr.7692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:06:05.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.082+0000 7fb86dffb640 1 -- 172.21.15.84:0/2768580375 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb874076640 con 0x7fb87806c140 2023-10-16T10:06:05.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.296+0000 7fb87f832640 1 -- 172.21.15.84:0/2768580375 --> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb8780cda90 con 0x7fb85c06ad40 2023-10-16T10:06:05.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.310+0000 7fb86dffb640 1 -- 172.21.15.84:0/2768580375 <== mgr.7692 v2:172.21.15.84:6832/985901542 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+302 (secure 0 0 0) 0x7fb868014400 con 0x7fb85c06ad40 2023-10-16T10:06:05.312 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:06:05.312 INFO:teuthology.orchestra.run.smithi084.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T10:06:04", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-16T10:06:05.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.313+0000 7fb87f832640 1 -- 172.21.15.84:0/2768580375 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fb85c06ad40 msgr2=0x7fb85c06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:06:05.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.313+0000 7fb87f832640 1 --2- 172.21.15.84:0/2768580375 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fb85c06ad40 0x7fb85c06d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb868007fd0 tx=0x7fb8680079a0 comp rx=0 tx=0).stop 2023-10-16T10:06:05.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.313+0000 7fb87f832640 1 -- 172.21.15.84:0/2768580375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87806c140 msgr2=0x7fb8780c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:06:05.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.313+0000 7fb87f832640 1 --2- 172.21.15.84:0/2768580375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87806c140 0x7fb8780c8900 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fb874000c00 tx=0x7fb8740046c0 comp rx=0 tx=0).stop 2023-10-16T10:06:05.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.313+0000 7fb87f832640 1 -- 172.21.15.84:0/2768580375 shutdown_connections 2023-10-16T10:06:05.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.313+0000 7fb87f832640 1 --2- 172.21.15.84:0/2768580375 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fb85c06ad40 0x7fb85c06d200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:05.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.313+0000 7fb87f832640 1 --2- 172.21.15.84:0/2768580375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb878159e90 0x7fb878155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:05.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.313+0000 7fb87f832640 1 --2- 172.21.15.84:0/2768580375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb878159ab0 0x7fb87815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:05.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.313+0000 7fb87f832640 1 --2- 172.21.15.84:0/2768580375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87806c140 0x7fb8780c8900 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:06:05.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.313+0000 7fb87f832640 1 -- 172.21.15.84:0/2768580375 >> 172.21.15.84:0/2768580375 conn(0x7fb8780ce9b0 msgr2=0x7fb8780ae310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:06:05.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.314+0000 7fb87f832640 1 -- 172.21.15.84:0/2768580375 shutdown_connections 2023-10-16T10:06:05.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:06:05.314+0000 7fb87f832640 1 -- 172.21.15.84:0/2768580375 wait complete. 2023-10-16T10:06:05.334 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T10:06:04", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-16T10:06:34.202 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:06:34.212 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:07:00.856 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:00.854+0000 7f8c94cdd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:00.856 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:00.854+0000 7f8c94cdd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:00.857 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:00.854+0000 7f8c94cdd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:00.857 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:00.854+0000 7f8c94cdd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:00.857 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:00.854+0000 7f8c94cdd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:04.350 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:07:04.360 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:07:05.335 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /bad-path 2023-10-16T10:07:05.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.558+0000 7f6abf4c9640 1 Processor -- start 2023-10-16T10:07:05.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.559+0000 7f6abf4c9640 1 -- start start 2023-10-16T10:07:05.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.560+0000 7f6abf4c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ab8159ab0 0x7f6ab80ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:05.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.560+0000 7f6abf4c9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6ab806c140 0x7f6ab806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:05.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.560+0000 7f6abf4c9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ab8070360 con 0x7f6ab8159e90 2023-10-16T10:07:05.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.560+0000 7f6abf4c9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab806cb10 con 0x7f6ab806c140 2023-10-16T10:07:05.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.560+0000 7f6abf4c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab815d890 con 0x7f6ab8159ab0 2023-10-16T10:07:05.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.561+0000 7f6abda66640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6ab806c140 0x7f6ab806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:05.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.561+0000 7f6abda66640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6ab806c140 0x7f6ab806c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38630/0 (socket says 172.21.15.84:38630) 2023-10-16T10:07:05.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.561+0000 7f6abda66640 1 -- 172.21.15.84:0/2180885052 learned_addr learned my addr 172.21.15.84:0/2180885052 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:05.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.561+0000 7f6abd265640 1 --2- 172.21.15.84:0/2180885052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ab8159ab0 0x7f6ab80ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:05.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.561+0000 7f6abda66640 1 -- 172.21.15.84:0/2180885052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ab8159ab0 msgr2=0x7f6ab80ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:05.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.561+0000 7f6abda66640 1 --2- 172.21.15.84:0/2180885052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ab8159ab0 0x7f6ab80ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:05.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.561+0000 7f6abda66640 1 -- 172.21.15.84:0/2180885052 >> v1:172.21.15.157:6789/0 conn(0x7f6ab8159e90 legacy=0x7f6ab806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:05.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.561+0000 7f6abda66640 1 -- 172.21.15.84:0/2180885052 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab80cd1f0 con 0x7f6ab806c140 2023-10-16T10:07:05.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.562+0000 7f6abda66640 1 --2- 172.21.15.84:0/2180885052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ab806c140 0x7f6ab806c5d0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009e80 tx=0x7f6ab002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2734bc7951dc267 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:05.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.563+0000 7f6aaffff640 1 -- 172.21.15.84:0/2180885052 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab002c9c0 con 0x7f6ab806c140 2023-10-16T10:07:05.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.563+0000 7f6aaffff640 1 -- 172.21.15.84:0/2180885052 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab002cb20 con 0x7f6ab806c140 2023-10-16T10:07:05.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.563+0000 7f6aaffff640 1 -- 172.21.15.84:0/2180885052 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab00343f0 con 0x7f6ab806c140 2023-10-16T10:07:05.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.563+0000 7f6abf4c9640 1 -- 172.21.15.84:0/2180885052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ab806c140 msgr2=0x7f6ab806c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:05.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.563+0000 7f6abf4c9640 1 --2- 172.21.15.84:0/2180885052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ab806c140 0x7f6ab806c5d0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009e80 tx=0x7f6ab002be20 comp rx=0 tx=0).stop 2023-10-16T10:07:05.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.563+0000 7f6abf4c9640 1 -- 172.21.15.84:0/2180885052 shutdown_connections 2023-10-16T10:07:05.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.563+0000 7f6abf4c9640 1 --2- 172.21.15.84:0/2180885052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ab806c140 0x7f6ab806c5d0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:05.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.563+0000 7f6abf4c9640 1 --2- 172.21.15.84:0/2180885052 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ab8159ab0 0x7f6ab80ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:05.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.563+0000 7f6abf4c9640 1 -- 172.21.15.84:0/2180885052 >> 172.21.15.84:0/2180885052 conn(0x7f6ab80ce9b0 msgr2=0x7f6ab80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:05.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.563+0000 7f6abf4c9640 1 -- 172.21.15.84:0/2180885052 shutdown_connections 2023-10-16T10:07:05.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.564+0000 7f6abf4c9640 1 -- 172.21.15.84:0/2180885052 wait complete. 2023-10-16T10:07:05.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.565+0000 7f6abf4c9640 1 Processor -- start 2023-10-16T10:07:05.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.565+0000 7f6abf4c9640 1 -- start start 2023-10-16T10:07:05.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.566+0000 7f6abf4c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ab806c140 0x7f6ab80c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:05.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.566+0000 7f6abf4c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ab8159ab0 0x7f6ab8150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:05.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.566+0000 7f6abf4c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ab8159e90 0x7f6ab814a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:05.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.566+0000 7f6abf4c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab80c34e0 con 0x7f6ab806c140 2023-10-16T10:07:05.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.566+0000 7f6abf4c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab81f6c60 con 0x7f6ab8159ab0 2023-10-16T10:07:05.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.566+0000 7f6abf4c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab80c5620 con 0x7f6ab8159e90 2023-10-16T10:07:05.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.567+0000 7f6abd265640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ab806c140 0x7f6ab80c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:05.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.567+0000 7f6abd265640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ab806c140 0x7f6ab80c50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38640/0 (socket says 172.21.15.84:38640) 2023-10-16T10:07:05.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.567+0000 7f6abd265640 1 -- 172.21.15.84:0/209496785 learned_addr learned my addr 172.21.15.84:0/209496785 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:05.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.567+0000 7f6abd265640 1 -- 172.21.15.84:0/209496785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ab8159e90 msgr2=0x7f6ab814a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:05.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.567+0000 7f6abd265640 1 --2- 172.21.15.84:0/209496785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ab8159e90 0x7f6ab814a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:05.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.567+0000 7f6abd265640 1 -- 172.21.15.84:0/209496785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ab8159ab0 msgr2=0x7f6ab8150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:05.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.567+0000 7f6abca64640 1 --2- 172.21.15.84:0/209496785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ab8159ab0 0x7f6ab8150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:05.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.567+0000 7f6abd265640 1 --2- 172.21.15.84:0/209496785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ab8159ab0 0x7f6ab8150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:05.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.567+0000 7f6abd265640 1 -- 172.21.15.84:0/209496785 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab0009b30 con 0x7f6ab806c140 2023-10-16T10:07:05.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.567+0000 7f6abca64640 1 --2- 172.21.15.84:0/209496785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ab8159ab0 0x7f6ab8150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:05.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.567+0000 7f6abd265640 1 --2- 172.21.15.84:0/209496785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ab806c140 0x7f6ab80c50e0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400d5d0 tx=0x7f6ab400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:05.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.568+0000 7f6aadffb640 1 -- 172.21.15.84:0/209496785 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab40125f0 con 0x7f6ab806c140 2023-10-16T10:07:05.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.568+0000 7f6aadffb640 1 -- 172.21.15.84:0/209496785 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab4012770 con 0x7f6ab806c140 2023-10-16T10:07:05.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.568+0000 7f6aadffb640 1 -- 172.21.15.84:0/209496785 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4012b20 con 0x7f6ab806c140 2023-10-16T10:07:05.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.568+0000 7f6abf4c9640 1 -- 172.21.15.84:0/209496785 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab80c2b40 con 0x7f6ab806c140 2023-10-16T10:07:05.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.568+0000 7f6abf4c9640 1 -- 172.21.15.84:0/209496785 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab80c3100 con 0x7f6ab806c140 2023-10-16T10:07:05.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.569+0000 7f6abf4c9640 1 -- 172.21.15.84:0/209496785 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab804e160 con 0x7f6ab806c140 2023-10-16T10:07:05.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.570+0000 7f6aadffb640 1 -- 172.21.15.84:0/209496785 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 54) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f6ab4012c80 con 0x7f6ab806c140 2023-10-16T10:07:05.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.571+0000 7f6aadffb640 1 --2- 172.21.15.84:0/209496785 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f6a9c06b100 0x7f6a9c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:05.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.571+0000 7f6aadffb640 1 -- 172.21.15.84:0/209496785 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6082+0+0 (secure 0 0 0) 0x7f6ab408a160 con 0x7f6ab806c140 2023-10-16T10:07:05.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.571+0000 7f6abca64640 1 --2- 172.21.15.84:0/209496785 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f6a9c06b100 0x7f6a9c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:05.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.572+0000 7f6abca64640 1 --2- 172.21.15.84:0/209496785 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f6a9c06b100 0x7f6a9c06d5c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80ca670 tx=0x7f6aa8009040 comp rx=0 tx=0).ready entity=mgr.7692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:05.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.576+0000 7f6aadffb640 1 -- 172.21.15.84:0/209496785 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6ab4054d20 con 0x7f6ab806c140 2023-10-16T10:07:05.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.793+0000 7f6abf4c9640 1 -- 172.21.15.84:0/209496785 --> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ab80cd7b0 con 0x7f6a9c06b100 2023-10-16T10:07:05.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.812+0000 7f6aadffb640 1 -- 172.21.15.84:0/209496785 <== mgr.7692 v2:172.21.15.84:6832/985901542 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+303 (secure 0 0 0) 0x7f6aa80013e0 con 0x7f6a9c06b100 2023-10-16T10:07:05.814 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:05.814 INFO:teuthology.orchestra.run.smithi084.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T10:06:04", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-10-16T10:07:05.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.815+0000 7f6abf4c9640 1 -- 172.21.15.84:0/209496785 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f6a9c06b100 msgr2=0x7f6a9c06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:05.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.815+0000 7f6abf4c9640 1 --2- 172.21.15.84:0/209496785 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f6a9c06b100 0x7f6a9c06d5c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80ca670 tx=0x7f6aa8009040 comp rx=0 tx=0).stop 2023-10-16T10:07:05.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.815+0000 7f6abf4c9640 1 -- 172.21.15.84:0/209496785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ab806c140 msgr2=0x7f6ab80c50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:05.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.815+0000 7f6abf4c9640 1 --2- 172.21.15.84:0/209496785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ab806c140 0x7f6ab80c50e0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400d5d0 tx=0x7f6ab400da90 comp rx=0 tx=0).stop 2023-10-16T10:07:05.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.815+0000 7f6abf4c9640 1 -- 172.21.15.84:0/209496785 shutdown_connections 2023-10-16T10:07:05.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.815+0000 7f6abf4c9640 1 --2- 172.21.15.84:0/209496785 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f6a9c06b100 0x7f6a9c06d5c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:05.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.815+0000 7f6abf4c9640 1 --2- 172.21.15.84:0/209496785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ab8159e90 0x7f6ab814a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:05.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.815+0000 7f6abf4c9640 1 --2- 172.21.15.84:0/209496785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ab8159ab0 0x7f6ab8150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:05.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.815+0000 7f6abf4c9640 1 --2- 172.21.15.84:0/209496785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ab806c140 0x7f6ab80c50e0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:05.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.815+0000 7f6abf4c9640 1 -- 172.21.15.84:0/209496785 >> 172.21.15.84:0/209496785 conn(0x7f6ab80ce9b0 msgr2=0x7f6ab80b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:05.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.815+0000 7f6abf4c9640 1 -- 172.21.15.84:0/209496785 shutdown_connections 2023-10-16T10:07:05.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:05.815+0000 7f6abf4c9640 1 -- 172.21.15.84:0/209496785 wait complete. 2023-10-16T10:07:05.836 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T10:06:04", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-10-16T10:07:05.837 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /bad-path 2023-10-16T10:07:06.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.046+0000 7f1e1e6a2640 1 Processor -- start 2023-10-16T10:07:06.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.046+0000 7f1e1e6a2640 1 -- start start 2023-10-16T10:07:06.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.048+0000 7f1e1e6a2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1e1806a4b0 0x7f1e180c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:06.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.048+0000 7f1e1e6a2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e180ac890 con 0x7f1e180ad350 2023-10-16T10:07:06.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.048+0000 7f1e1e6a2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e180ad7a0 con 0x7f1e1815d150 2023-10-16T10:07:06.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.048+0000 7f1e1e6a2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1815d530 con 0x7f1e1806a4b0 2023-10-16T10:07:06.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.048+0000 7f1e0ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1e180ad350 0x7f1e18069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54948/0 (socket says 172.21.15.84:54948) 2023-10-16T10:07:06.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.048+0000 7f1e0ffff640 1 -- 172.21.15.84:0/3771459326 learned_addr learned my addr 172.21.15.84:0/3771459326 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:06.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.048+0000 7f1e1cc3f640 1 --2- 172.21.15.84:0/3771459326 >> v2:172.21.15.84:3300/0 conn(0x7f1e1806a4b0 0x7f1e180c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:06.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.049+0000 7f1e177fe640 1 -- 172.21.15.84:0/3771459326 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635099331 0 0) 0x7f1e180ac890 con 0x7f1e180ad350 2023-10-16T10:07:06.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.049+0000 7f1e177fe640 1 -- 172.21.15.84:0/3771459326 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e04003520 con 0x7f1e180ad350 2023-10-16T10:07:06.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.049+0000 7f1e1cc3f640 1 -- 172.21.15.84:0/3771459326 >> v1:172.21.15.157:6789/0 conn(0x7f1e1815d150 legacy=0x7f1e18160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:06.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.049+0000 7f1e1cc3f640 1 -- 172.21.15.84:0/3771459326 >> v1:172.21.15.84:6789/0 conn(0x7f1e180ad350 legacy=0x7f1e18069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:06.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.049+0000 7f1e1cc3f640 1 -- 172.21.15.84:0/3771459326 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e18161650 con 0x7f1e1806a4b0 2023-10-16T10:07:06.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.050+0000 7f1e1cc3f640 1 --2- 172.21.15.84:0/3771459326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e1806a4b0 0x7f1e180c9000 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009d90 tx=0x7f1e1002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837379b216aade2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:06.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.050+0000 7f1e177fe640 1 -- 172.21.15.84:0/3771459326 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1002cd00 con 0x7f1e1806a4b0 2023-10-16T10:07:06.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.050+0000 7f1e177fe640 1 -- 172.21.15.84:0/3771459326 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1002ce60 con 0x7f1e1806a4b0 2023-10-16T10:07:06.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.050+0000 7f1e177fe640 1 -- 172.21.15.84:0/3771459326 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10034660 con 0x7f1e1806a4b0 2023-10-16T10:07:06.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.051+0000 7f1e1e6a2640 1 -- 172.21.15.84:0/3771459326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e1806a4b0 msgr2=0x7f1e180c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:06.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.051+0000 7f1e1e6a2640 1 --2- 172.21.15.84:0/3771459326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e1806a4b0 0x7f1e180c9000 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009d90 tx=0x7f1e1002bf50 comp rx=0 tx=0).stop 2023-10-16T10:07:06.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.051+0000 7f1e1e6a2640 1 -- 172.21.15.84:0/3771459326 shutdown_connections 2023-10-16T10:07:06.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.051+0000 7f1e1e6a2640 1 --2- 172.21.15.84:0/3771459326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e1806a4b0 0x7f1e180c9000 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:06.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.051+0000 7f1e1e6a2640 1 -- 172.21.15.84:0/3771459326 >> 172.21.15.84:0/3771459326 conn(0x7f1e180ce9b0 msgr2=0x7f1e180cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:06.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.051+0000 7f1e1e6a2640 1 -- 172.21.15.84:0/3771459326 shutdown_connections 2023-10-16T10:07:06.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.051+0000 7f1e1e6a2640 1 -- 172.21.15.84:0/3771459326 wait complete. 2023-10-16T10:07:06.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.053+0000 7f1e1e6a2640 1 Processor -- start 2023-10-16T10:07:06.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.053+0000 7f1e1e6a2640 1 -- start start 2023-10-16T10:07:06.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.054+0000 7f1e1e6a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e1806a4b0 0x7f1e180ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:06.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.054+0000 7f1e1e6a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e180ad350 0x7f1e18155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:06.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.054+0000 7f1e1e6a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1815d150 0x7f1e18156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:06.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.054+0000 7f1e1e6a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180c6880 con 0x7f1e180ad350 2023-10-16T10:07:06.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.054+0000 7f1e1e6a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180c5c00 con 0x7f1e1806a4b0 2023-10-16T10:07:06.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.054+0000 7f1e1e6a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180c5ee0 con 0x7f1e1815d150 2023-10-16T10:07:06.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.054+0000 7f1e0ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e180ad350 0x7f1e18155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:06.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.054+0000 7f1e0ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e180ad350 0x7f1e18155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54612/0 (socket says 172.21.15.84:54612) 2023-10-16T10:07:06.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.054+0000 7f1e0ffff640 1 -- 172.21.15.84:0/1934618389 learned_addr learned my addr 172.21.15.84:0/1934618389 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:06.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.055+0000 7f1e17fff640 1 --2- 172.21.15.84:0/1934618389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e1806a4b0 0x7f1e180ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:06.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.055+0000 7f1e0ffff640 1 -- 172.21.15.84:0/1934618389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1815d150 msgr2=0x7f1e18156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:06.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.055+0000 7f1e0ffff640 1 --2- 172.21.15.84:0/1934618389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1815d150 0x7f1e18156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:06.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.055+0000 7f1e0ffff640 1 -- 172.21.15.84:0/1934618389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e1806a4b0 msgr2=0x7f1e180ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:06.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.055+0000 7f1e0ffff640 1 --2- 172.21.15.84:0/1934618389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e1806a4b0 0x7f1e180ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:06.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.055+0000 7f1e0ffff640 1 -- 172.21.15.84:0/1934618389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e10009a70 con 0x7f1e180ad350 2023-10-16T10:07:06.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.055+0000 7f1e17fff640 1 --2- 172.21.15.84:0/1934618389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e1806a4b0 0x7f1e180ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:06.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.055+0000 7f1e0ffff640 1 --2- 172.21.15.84:0/1934618389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e180ad350 0x7f1e18155b20 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000c500 tx=0x7f1e0000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:06.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.055+0000 7f1e157fa640 1 -- 172.21.15.84:0/1934618389 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e00005ce0 con 0x7f1e180ad350 2023-10-16T10:07:06.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.055+0000 7f1e157fa640 1 -- 172.21.15.84:0/1934618389 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e00005e40 con 0x7f1e180ad350 2023-10-16T10:07:06.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.055+0000 7f1e1e6a2640 1 -- 172.21.15.84:0/1934618389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e180c61c0 con 0x7f1e180ad350 2023-10-16T10:07:06.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.055+0000 7f1e157fa640 1 -- 172.21.15.84:0/1934618389 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e00007330 con 0x7f1e180ad350 2023-10-16T10:07:06.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.055+0000 7f1e1e6a2640 1 -- 172.21.15.84:0/1934618389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e180cadb0 con 0x7f1e180ad350 2023-10-16T10:07:06.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.056+0000 7f1e1e6a2640 1 -- 172.21.15.84:0/1934618389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e180cafa0 con 0x7f1e180ad350 2023-10-16T10:07:06.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.057+0000 7f1e157fa640 1 -- 172.21.15.84:0/1934618389 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 54) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f1e00007550 con 0x7f1e180ad350 2023-10-16T10:07:06.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.058+0000 7f1e157fa640 1 --2- 172.21.15.84:0/1934618389 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f1df806b100 0x7f1df806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:06.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.058+0000 7f1e157fa640 1 -- 172.21.15.84:0/1934618389 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6082+0+0 (secure 0 0 0) 0x7f1e000892e0 con 0x7f1e180ad350 2023-10-16T10:07:06.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.059+0000 7f1e17fff640 1 --2- 172.21.15.84:0/1934618389 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f1df806b100 0x7f1df806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:06.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.059+0000 7f1e17fff640 1 --2- 172.21.15.84:0/1934618389 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f1df806b100 0x7f1df806d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1e08000f60 tx=0x7f1e08009040 comp rx=0 tx=0).ready entity=mgr.7692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:06.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.063+0000 7f1e157fa640 1 -- 172.21.15.84:0/1934618389 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1e00053f60 con 0x7f1e180ad350 2023-10-16T10:07:06.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.277+0000 7f1e1e6a2640 1 -- 172.21.15.84:0/1934618389 --> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/bad-path", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e180b1e20 con 0x7f1df806b100 2023-10-16T10:07:06.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.461+0000 7f1e157fa640 1 -- 172.21.15.84:0/1934618389 <== mgr.7692 v2:172.21.15.84:6832/985901542 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7f1e180b1e20 con 0x7f1df806b100 2023-10-16T10:07:06.464 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path /bad-path 2023-10-16T10:07:06.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.464+0000 7f1e1e6a2640 1 -- 172.21.15.84:0/1934618389 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f1df806b100 msgr2=0x7f1df806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:06.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.464+0000 7f1e1e6a2640 1 --2- 172.21.15.84:0/1934618389 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f1df806b100 0x7f1df806d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1e08000f60 tx=0x7f1e08009040 comp rx=0 tx=0).stop 2023-10-16T10:07:06.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.464+0000 7f1e1e6a2640 1 -- 172.21.15.84:0/1934618389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e180ad350 msgr2=0x7f1e18155b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:06.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.465+0000 7f1e1e6a2640 1 --2- 172.21.15.84:0/1934618389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e180ad350 0x7f1e18155b20 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000c500 tx=0x7f1e0000c9c0 comp rx=0 tx=0).stop 2023-10-16T10:07:06.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.465+0000 7f1e1e6a2640 1 -- 172.21.15.84:0/1934618389 shutdown_connections 2023-10-16T10:07:06.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.465+0000 7f1e1e6a2640 1 --2- 172.21.15.84:0/1934618389 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f1df806b100 0x7f1df806d5c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:06.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.465+0000 7f1e1e6a2640 1 --2- 172.21.15.84:0/1934618389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1815d150 0x7f1e18156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:06.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.465+0000 7f1e1e6a2640 1 --2- 172.21.15.84:0/1934618389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e180ad350 0x7f1e18155b20 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:06.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.465+0000 7f1e1e6a2640 1 --2- 172.21.15.84:0/1934618389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e1806a4b0 0x7f1e180ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:06.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.465+0000 7f1e1e6a2640 1 -- 172.21.15.84:0/1934618389 >> 172.21.15.84:0/1934618389 conn(0x7f1e180ce9b0 msgr2=0x7f1e1806f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:06.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.465+0000 7f1e1e6a2640 1 -- 172.21.15.84:0/1934618389 shutdown_connections 2023-10-16T10:07:06.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.465+0000 7f1e1e6a2640 1 -- 172.21.15.84:0/1934618389 wait complete. 2023-10-16T10:07:06.486 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2023-10-16T10:07:06.487 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T10:07:06.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.696+0000 7f16f814e640 1 Processor -- start 2023-10-16T10:07:06.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.696+0000 7f16f814e640 1 -- start start 2023-10-16T10:07:06.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.697+0000 7f16f814e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16f0159ab0 0x7f16f00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:06.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.697+0000 7f16f814e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f16f006c140 0x7f16f006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:06.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.697+0000 7f16f814e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f0070360 con 0x7f16f0159e90 2023-10-16T10:07:06.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.698+0000 7f16f814e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16f006cb10 con 0x7f16f006c140 2023-10-16T10:07:06.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.698+0000 7f16f814e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f015d890 con 0x7f16f0159ab0 2023-10-16T10:07:06.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.698+0000 7f16f5eea640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16f0159ab0 0x7f16f00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:06.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.698+0000 7f16f66eb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f16f006c140 0x7f16f006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:06.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.698+0000 7f16f66eb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f16f006c140 0x7f16f006c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:46826/0 (socket says 172.21.15.84:46826) 2023-10-16T10:07:06.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.698+0000 7f16f66eb640 1 -- 172.21.15.84:0/1393136974 learned_addr learned my addr 172.21.15.84:0/1393136974 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:06.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.698+0000 7f16f5eea640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16f0159ab0 0x7f16f00ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:34764/0 (socket says 172.21.15.84:34764) 2023-10-16T10:07:06.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.699+0000 7f16f66eb640 1 -- 172.21.15.84:0/1393136974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16f0159ab0 msgr2=0x7f16f00ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:06.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.699+0000 7f16f66eb640 1 --2- 172.21.15.84:0/1393136974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16f0159ab0 0x7f16f00ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:06.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.699+0000 7f16f66eb640 1 -- 172.21.15.84:0/1393136974 >> v1:172.21.15.157:6789/0 conn(0x7f16f0159e90 legacy=0x7f16f006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:06.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.699+0000 7f16f66eb640 1 -- 172.21.15.84:0/1393136974 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f00cd1f0 con 0x7f16f006c140 2023-10-16T10:07:06.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.701+0000 7f16f66eb640 1 --2- 172.21.15.84:0/1393136974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16f006c140 0x7f16f006c5d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009e80 tx=0x7f16e802be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec5b6a2a99c4345d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:06.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.701+0000 7f16f4ee8640 1 -- 172.21.15.84:0/1393136974 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e802c9c0 con 0x7f16f006c140 2023-10-16T10:07:06.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.701+0000 7f16f4ee8640 1 -- 172.21.15.84:0/1393136974 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e802cb20 con 0x7f16f006c140 2023-10-16T10:07:06.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.701+0000 7f16f4ee8640 1 -- 172.21.15.84:0/1393136974 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e80343f0 con 0x7f16f006c140 2023-10-16T10:07:06.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.702+0000 7f16f814e640 1 -- 172.21.15.84:0/1393136974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16f006c140 msgr2=0x7f16f006c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:06.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.702+0000 7f16f814e640 1 --2- 172.21.15.84:0/1393136974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16f006c140 0x7f16f006c5d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009e80 tx=0x7f16e802be20 comp rx=0 tx=0).stop 2023-10-16T10:07:06.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.702+0000 7f16f814e640 1 -- 172.21.15.84:0/1393136974 shutdown_connections 2023-10-16T10:07:06.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.702+0000 7f16f814e640 1 --2- 172.21.15.84:0/1393136974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16f006c140 0x7f16f006c5d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:06.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.702+0000 7f16f814e640 1 --2- 172.21.15.84:0/1393136974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16f0159ab0 0x7f16f00ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:06.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.702+0000 7f16f814e640 1 -- 172.21.15.84:0/1393136974 >> 172.21.15.84:0/1393136974 conn(0x7f16f00ce9b0 msgr2=0x7f16f00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:06.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.702+0000 7f16f814e640 1 -- 172.21.15.84:0/1393136974 shutdown_connections 2023-10-16T10:07:06.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.702+0000 7f16f814e640 1 -- 172.21.15.84:0/1393136974 wait complete. 2023-10-16T10:07:06.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.703+0000 7f16f814e640 1 Processor -- start 2023-10-16T10:07:06.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.704+0000 7f16f814e640 1 -- start start 2023-10-16T10:07:06.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.704+0000 7f16f814e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16f006c140 0x7f16f01f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:06.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.704+0000 7f16f814e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16f0159ab0 0x7f16f00b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:06.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.704+0000 7f16f814e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16f0159e90 0x7f16f00b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:06.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.704+0000 7f16f814e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f0154bc0 con 0x7f16f0159e90 2023-10-16T10:07:06.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.704+0000 7f16f814e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f00b40c0 con 0x7f16f0159ab0 2023-10-16T10:07:06.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.704+0000 7f16f814e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f00b43a0 con 0x7f16f006c140 2023-10-16T10:07:06.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.705+0000 7f16f66eb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16f0159e90 0x7f16f00b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:06.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.705+0000 7f16f66eb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16f0159e90 0x7f16f00b09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54616/0 (socket says 172.21.15.84:54616) 2023-10-16T10:07:06.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.705+0000 7f16f66eb640 1 -- 172.21.15.84:0/2841243319 learned_addr learned my addr 172.21.15.84:0/2841243319 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:06.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.705+0000 7f16f5eea640 1 --2- 172.21.15.84:0/2841243319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16f006c140 0x7f16f01f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:06.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.705+0000 7f16f66eb640 1 -- 172.21.15.84:0/2841243319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16f006c140 msgr2=0x7f16f01f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:06.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.705+0000 7f16f66eb640 1 --2- 172.21.15.84:0/2841243319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16f006c140 0x7f16f01f2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:06.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.705+0000 7f16f56e9640 1 --2- 172.21.15.84:0/2841243319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16f0159ab0 0x7f16f00b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:06.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.705+0000 7f16f66eb640 1 -- 172.21.15.84:0/2841243319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16f0159ab0 msgr2=0x7f16f00b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:06.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.705+0000 7f16f66eb640 1 --2- 172.21.15.84:0/2841243319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16f0159ab0 0x7f16f00b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:06.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.705+0000 7f16f66eb640 1 -- 172.21.15.84:0/2841243319 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e8009b30 con 0x7f16f0159e90 2023-10-16T10:07:06.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.706+0000 7f16f66eb640 1 --2- 172.21.15.84:0/2841243319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16f0159e90 0x7f16f00b09c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f16e8036040 tx=0x7f16e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:06.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.706+0000 7f16e67fc640 1 -- 172.21.15.84:0/2841243319 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e8009510 con 0x7f16f0159e90 2023-10-16T10:07:06.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.706+0000 7f16e67fc640 1 -- 172.21.15.84:0/2841243319 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e803e450 con 0x7f16f0159e90 2023-10-16T10:07:06.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.706+0000 7f16e67fc640 1 -- 172.21.15.84:0/2841243319 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e803e800 con 0x7f16f0159e90 2023-10-16T10:07:06.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.706+0000 7f16f814e640 1 -- 172.21.15.84:0/2841243319 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f01547e0 con 0x7f16f0159e90 2023-10-16T10:07:06.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.706+0000 7f16f814e640 1 -- 172.21.15.84:0/2841243319 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f0157450 con 0x7f16f0159e90 2023-10-16T10:07:06.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.707+0000 7f16f814e640 1 -- 172.21.15.84:0/2841243319 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f0154d00 con 0x7f16f0159e90 2023-10-16T10:07:06.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.708+0000 7f16e67fc640 1 -- 172.21.15.84:0/2841243319 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 54) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f16e803e960 con 0x7f16f0159e90 2023-10-16T10:07:06.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.709+0000 7f16e67fc640 1 --2- 172.21.15.84:0/2841243319 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f16d406b100 0x7f16d406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:06.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.709+0000 7f16e67fc640 1 -- 172.21.15.84:0/2841243319 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6082+0+0 (secure 0 0 0) 0x7f16e80ac9c0 con 0x7f16f0159e90 2023-10-16T10:07:06.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.709+0000 7f16f5eea640 1 --2- 172.21.15.84:0/2841243319 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f16d406b100 0x7f16d406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:06.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.710+0000 7f16f5eea640 1 --2- 172.21.15.84:0/2841243319 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f16d406b100 0x7f16d406d5c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009970 tx=0x7f16ec008040 comp rx=0 tx=0).ready entity=mgr.7692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:06.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.713+0000 7f16e67fc640 1 -- 172.21.15.84:0/2841243319 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f16f0154d00 con 0x7f16f0159e90 2023-10-16T10:07:06.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:06.956+0000 7f16f814e640 1 -- 172.21.15.84:0/2841243319 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f16f0154d00 con 0x7f16f0159e90 2023-10-16T10:07:07.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.154+0000 7f16e67fc640 1 -- 172.21.15.84:0/2841243319 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v55) v1 ==== 93+0+0 (secure 0 0 0) 0x7f16f0154d00 con 0x7f16f0159e90 2023-10-16T10:07:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.157+0000 7f16f814e640 1 -- 172.21.15.84:0/2841243319 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f16d406b100 msgr2=0x7f16d406d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.157+0000 7f16f814e640 1 --2- 172.21.15.84:0/2841243319 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f16d406b100 0x7f16d406d5c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009970 tx=0x7f16ec008040 comp rx=0 tx=0).stop 2023-10-16T10:07:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.157+0000 7f16f814e640 1 -- 172.21.15.84:0/2841243319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16f0159e90 msgr2=0x7f16f00b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:07.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.157+0000 7f16f814e640 1 --2- 172.21.15.84:0/2841243319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16f0159e90 0x7f16f00b09c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f16e8036040 tx=0x7f16e8004a40 comp rx=0 tx=0).stop 2023-10-16T10:07:07.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.158+0000 7f16f814e640 1 -- 172.21.15.84:0/2841243319 shutdown_connections 2023-10-16T10:07:07.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.158+0000 7f16f814e640 1 --2- 172.21.15.84:0/2841243319 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f16d406b100 0x7f16d406d5c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:07.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.158+0000 7f16f814e640 1 --2- 172.21.15.84:0/2841243319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16f0159e90 0x7f16f00b09c0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:07.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.158+0000 7f16f814e640 1 --2- 172.21.15.84:0/2841243319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16f0159ab0 0x7f16f00b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:07.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.158+0000 7f16f814e640 1 --2- 172.21.15.84:0/2841243319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16f006c140 0x7f16f01f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:07.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.158+0000 7f16f814e640 1 -- 172.21.15.84:0/2841243319 >> 172.21.15.84:0/2841243319 conn(0x7f16f00ce9b0 msgr2=0x7f16f006fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:07.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.159+0000 7f16f814e640 1 -- 172.21.15.84:0/2841243319 shutdown_connections 2023-10-16T10:07:07.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.159+0000 7f16f814e640 1 -- 172.21.15.84:0/2841243319 wait complete. 2023-10-16T10:07:07.181 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:07:07.182 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:07:07.274 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:07:07.385 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:07:07.386 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:07:07.386 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:07:07.439 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:07:07.440 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:07:07.489 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:07:07.683 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:07.681+0000 7f9b4122b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:07:07.705 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:07.703+0000 7fe27260f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:07:07.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.728+0000 7fd07d252640 1 Processor -- start 2023-10-16T10:07:07.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.728+0000 7fd07d252640 1 -- start start 2023-10-16T10:07:07.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.729+0000 7fd07d252640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd078159950 0x7fd07815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:07.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.729+0000 7fd07d252640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd078070450 0x7fd0780708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:07.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.729+0000 7fd07d252640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd078070df0 con 0x7fd07806c100 2023-10-16T10:07:07.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.729+0000 7fd07d252640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd07815e490 con 0x7fd078159950 2023-10-16T10:07:07.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.729+0000 7fd07d252640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07815aef0 con 0x7fd078070450 2023-10-16T10:07:07.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.730+0000 7fd077fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd078159950 0x7fd07815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:07.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.730+0000 7fd07ca51640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd07806c100 0x7fd07806c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:44478/0 (socket says 172.21.15.84:44478) 2023-10-16T10:07:07.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.730+0000 7fd07ca51640 1 -- 172.21.15.84:0/2241772971 learned_addr learned my addr 172.21.15.84:0/2241772971 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:07.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.730+0000 7fd0777fe640 1 --2- 172.21.15.84:0/2241772971 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd078070450 0x7fd0780708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:07.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.731+0000 7fd076ffd640 1 -- 172.21.15.84:0/2241772971 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 949542612 0 0) 0x7fd078070df0 con 0x7fd07806c100 2023-10-16T10:07:07.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.731+0000 7fd076ffd640 1 -- 172.21.15.84:0/2241772971 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd060003520 con 0x7fd07806c100 2023-10-16T10:07:07.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.731+0000 7fd077fff640 1 -- 172.21.15.84:0/2241772971 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd078070450 msgr2=0x7fd0780708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:07.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.731+0000 7fd077fff640 1 --2- 172.21.15.84:0/2241772971 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd078070450 0x7fd0780708b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:07.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.731+0000 7fd077fff640 1 -- 172.21.15.84:0/2241772971 >> v1:172.21.15.157:6789/0 conn(0x7fd07806c100 legacy=0x7fd07806c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:07.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.731+0000 7fd077fff640 1 -- 172.21.15.84:0/2241772971 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07806cc80 con 0x7fd078159950 2023-10-16T10:07:07.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.732+0000 7fd077fff640 1 --2- 172.21.15.84:0/2241772971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd078159950 0x7fd07815a9b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd0680097a0 tx=0x7fd06802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e998a850a44cc8b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:07.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.733+0000 7fd076ffd640 1 -- 172.21.15.84:0/2241772971 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068031780 con 0x7fd078159950 2023-10-16T10:07:07.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.733+0000 7fd076ffd640 1 -- 172.21.15.84:0/2241772971 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0680318e0 con 0x7fd078159950 2023-10-16T10:07:07.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.733+0000 7fd076ffd640 1 -- 172.21.15.84:0/2241772971 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068031c50 con 0x7fd078159950 2023-10-16T10:07:07.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.733+0000 7fd07d252640 1 -- 172.21.15.84:0/2241772971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd078159950 msgr2=0x7fd07815a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:07.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.733+0000 7fd07d252640 1 --2- 172.21.15.84:0/2241772971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd078159950 0x7fd07815a9b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd0680097a0 tx=0x7fd06802b820 comp rx=0 tx=0).stop 2023-10-16T10:07:07.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.734+0000 7fd07d252640 1 -- 172.21.15.84:0/2241772971 shutdown_connections 2023-10-16T10:07:07.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.734+0000 7fd07d252640 1 --2- 172.21.15.84:0/2241772971 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd078070450 0x7fd0780708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:07.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.734+0000 7fd07d252640 1 --2- 172.21.15.84:0/2241772971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd078159950 0x7fd07815a9b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:07.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.734+0000 7fd07d252640 1 -- 172.21.15.84:0/2241772971 >> 172.21.15.84:0/2241772971 conn(0x7fd0780ce6a0 msgr2=0x7fd0780ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:07.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.734+0000 7fd07d252640 1 -- 172.21.15.84:0/2241772971 shutdown_connections 2023-10-16T10:07:07.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.734+0000 7fd07d252640 1 -- 172.21.15.84:0/2241772971 wait complete. 2023-10-16T10:07:07.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.736+0000 7fd07d252640 1 Processor -- start 2023-10-16T10:07:07.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.736+0000 7fd07d252640 1 -- start start 2023-10-16T10:07:07.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.737+0000 7fd07d252640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd07806c100 0x7fd078149a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:07.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.737+0000 7fd07d252640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd078070450 0x7fd0780c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:07.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.737+0000 7fd07d252640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd078159950 0x7fd0781f0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:07.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.737+0000 7fd07d252640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0781f15c0 con 0x7fd07806c100 2023-10-16T10:07:07.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.737+0000 7fd07d252640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0781f1000 con 0x7fd078159950 2023-10-16T10:07:07.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.737+0000 7fd07d252640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0781f12e0 con 0x7fd078070450 2023-10-16T10:07:07.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.737+0000 7fd077fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd07806c100 0x7fd078149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:07.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.737+0000 7fd077fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd07806c100 0x7fd078149a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54656/0 (socket says 172.21.15.84:54656) 2023-10-16T10:07:07.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.737+0000 7fd077fff640 1 -- 172.21.15.84:0/3664663771 learned_addr learned my addr 172.21.15.84:0/3664663771 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:07.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.738+0000 7fd07ca51640 1 --2- 172.21.15.84:0/3664663771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd078159950 0x7fd0781f0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:07.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.738+0000 7fd0777fe640 1 --2- 172.21.15.84:0/3664663771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd078070450 0x7fd0780c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:07.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.738+0000 7fd077fff640 1 -- 172.21.15.84:0/3664663771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd078070450 msgr2=0x7fd0780c3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:07.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.738+0000 7fd077fff640 1 --2- 172.21.15.84:0/3664663771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd078070450 0x7fd0780c3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:07.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.738+0000 7fd077fff640 1 -- 172.21.15.84:0/3664663771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd078159950 msgr2=0x7fd0781f0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:07.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.738+0000 7fd077fff640 1 --2- 172.21.15.84:0/3664663771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd078159950 0x7fd0781f0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:07.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.738+0000 7fd077fff640 1 -- 172.21.15.84:0/3664663771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd068009400 con 0x7fd07806c100 2023-10-16T10:07:07.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.738+0000 7fd077fff640 1 --2- 172.21.15.84:0/3664663771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd07806c100 0x7fd078149a60 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fd0680098c0 tx=0x7fd0680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:07.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.738+0000 7fd074ff9640 1 -- 172.21.15.84:0/3664663771 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068003810 con 0x7fd07806c100 2023-10-16T10:07:07.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.738+0000 7fd074ff9640 1 -- 172.21.15.84:0/3664663771 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd068040070 con 0x7fd07806c100 2023-10-16T10:07:07.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.738+0000 7fd074ff9640 1 -- 172.21.15.84:0/3664663771 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068003b90 con 0x7fd07806c100 2023-10-16T10:07:07.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.738+0000 7fd07d252640 1 -- 172.21.15.84:0/3664663771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0780c33a0 con 0x7fd07806c100 2023-10-16T10:07:07.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.739+0000 7fd07d252640 1 -- 172.21.15.84:0/3664663771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd078149fa0 con 0x7fd07806c100 2023-10-16T10:07:07.745 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:07.739+0000 7f9b4122b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:07:07.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.740+0000 7fd07d252640 1 -- 172.21.15.84:0/3664663771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0780c3200 con 0x7fd07806c100 2023-10-16T10:07:07.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.741+0000 7fd074ff9640 1 -- 172.21.15.84:0/3664663771 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 55) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fd068003cf0 con 0x7fd07806c100 2023-10-16T10:07:07.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.742+0000 7fd074ff9640 1 --2- 172.21.15.84:0/3664663771 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fd06406b0b0 0x7fd06406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:07.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.743+0000 7fd074ff9640 1 -- 172.21.15.84:0/3664663771 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6082+0+0 (secure 0 0 0) 0x7fd068042070 con 0x7fd07806c100 2023-10-16T10:07:07.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.743+0000 7fd0777fe640 1 -- 172.21.15.84:0/3664663771 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fd06406b0b0 msgr2=0x7fd06406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/985901542 2023-10-16T10:07:07.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.743+0000 7fd0777fe640 1 --2- 172.21.15.84:0/3664663771 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fd06406b0b0 0x7fd06406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:07:07.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.747+0000 7fd074ff9640 1 -- 172.21.15.84:0/3664663771 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd0780c3200 con 0x7fd07806c100 2023-10-16T10:07:07.760 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:07.758+0000 7fe27260f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:07:07.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.939+0000 7fd07d252640 1 -- 172.21.15.84:0/3664663771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0780c3200 con 0x7fd07806c100 2023-10-16T10:07:07.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.940+0000 7fd074ff9640 1 -- 172.21.15.84:0/3664663771 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+17096 (secure 0 0 0) 0x7fd0780c3200 con 0x7fd07806c100 2023-10-16T10:07:07.942 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:07.943 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":112,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:07:05.125527+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:05:46.595888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-16T10:05:47.826019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.d","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2051153636":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/965715898":"2023-10-17T10:06:03.602359+0000","172.21.15.84:6833/3250823028":"2023-10-17T10:06:03.602359+0000","172.21.15.84:6832/3250823028":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/3381824109":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/3871500870":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/1287074955":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/3830790285":"2023-10-16T11:05:57.166133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:07:07.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.943+0000 7fd07d252640 1 -- 172.21.15.84:0/3664663771 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fd06406b0b0 msgr2=0x7fd06406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:07:07.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.943+0000 7fd07d252640 1 --2- 172.21.15.84:0/3664663771 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fd06406b0b0 0x7fd06406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:07.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.943+0000 7fd07d252640 1 -- 172.21.15.84:0/3664663771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd07806c100 msgr2=0x7fd078149a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:07.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.943+0000 7fd07d252640 1 --2- 172.21.15.84:0/3664663771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd07806c100 0x7fd078149a60 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fd0680098c0 tx=0x7fd0680046c0 comp rx=0 tx=0).stop 2023-10-16T10:07:07.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.943+0000 7fd07d252640 1 -- 172.21.15.84:0/3664663771 shutdown_connections 2023-10-16T10:07:07.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.943+0000 7fd07d252640 1 --2- 172.21.15.84:0/3664663771 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fd06406b0b0 0x7fd06406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:07.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.943+0000 7fd07d252640 1 --2- 172.21.15.84:0/3664663771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd078159950 0x7fd0781f0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:07.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.943+0000 7fd07d252640 1 --2- 172.21.15.84:0/3664663771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd078070450 0x7fd0780c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:07.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.943+0000 7fd07d252640 1 --2- 172.21.15.84:0/3664663771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd07806c100 0x7fd078149a60 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:07.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.943+0000 7fd07d252640 1 -- 172.21.15.84:0/3664663771 >> 172.21.15.84:0/3664663771 conn(0x7fd0780ce6a0 msgr2=0x7fd0780cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:07.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.943+0000 7fd07d252640 1 -- 172.21.15.84:0/3664663771 shutdown_connections 2023-10-16T10:07:07.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:07.943+0000 7fd07d252640 1 -- 172.21.15.84:0/3664663771 wait complete. 2023-10-16T10:07:07.963 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:05:46.595888+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:05:47.826019+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:07:07.964 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:07:08.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.168+0000 7f9c0d09f640 1 Processor -- start 2023-10-16T10:07:08.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.168+0000 7f9c0d09f640 1 -- start start 2023-10-16T10:07:08.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.170+0000 7f9c0d09f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c080b1ca0 0x7f9c080d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:08.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.170+0000 7f9c0d09f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9c080b2080 0x7f9c080ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:08.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.170+0000 7f9c0d09f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c080d0970 con 0x7f9c080ce960 2023-10-16T10:07:08.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.170+0000 7f9c0d09f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c080d0ea0 con 0x7f9c080b2080 2023-10-16T10:07:08.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.170+0000 7f9c0d09f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c080d0bc0 con 0x7f9c080b1ca0 2023-10-16T10:07:08.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.170+0000 7f9c0c89e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9c080ce960 0x7f9c080b24c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54950/0 (socket says 172.21.15.84:54950) 2023-10-16T10:07:08.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.170+0000 7f9c0c89e640 1 -- 172.21.15.84:0/1790180639 learned_addr learned my addr 172.21.15.84:0/1790180639 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:08.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.170+0000 7f9c07fff640 1 --2- 172.21.15.84:0/1790180639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c080b1ca0 0x7f9c080d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:08.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.170+0000 7f9c077fe640 1 --2- 172.21.15.84:0/1790180639 >> v2:172.21.15.157:3300/0 conn(0x7f9c080b2080 0x7f9c080ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:08.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.171+0000 7f9c06ffd640 1 -- 172.21.15.84:0/1790180639 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1993255045 0 0) 0x7f9c080d0970 con 0x7f9c080ce960 2023-10-16T10:07:08.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.171+0000 7f9c06ffd640 1 -- 172.21.15.84:0/1790180639 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bf0003520 con 0x7f9c080ce960 2023-10-16T10:07:08.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.171+0000 7f9c07fff640 1 -- 172.21.15.84:0/1790180639 >> v2:172.21.15.157:3300/0 conn(0x7f9c080b2080 msgr2=0x7f9c080ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:08.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.171+0000 7f9c07fff640 1 --2- 172.21.15.84:0/1790180639 >> v2:172.21.15.157:3300/0 conn(0x7f9c080b2080 0x7f9c080ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.171+0000 7f9c07fff640 1 -- 172.21.15.84:0/1790180639 >> v1:172.21.15.84:6789/0 conn(0x7f9c080ce960 legacy=0x7f9c080b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:08.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.171+0000 7f9c07fff640 1 -- 172.21.15.84:0/1790180639 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c080cd940 con 0x7f9c080b1ca0 2023-10-16T10:07:08.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.171+0000 7f9c077fe640 1 --2- 172.21.15.84:0/1790180639 >> v2:172.21.15.157:3300/0 conn(0x7f9c080b2080 0x7f9c080ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:07:08.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.173+0000 7f9c07fff640 1 --2- 172.21.15.84:0/1790180639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c080b1ca0 0x7f9c080d36b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80097a0 tx=0x7f9bf802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49bceff9ff6b9e80 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:08.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.178+0000 7f9c06ffd640 1 -- 172.21.15.84:0/1790180639 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8030800 con 0x7f9c080b1ca0 2023-10-16T10:07:08.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.179+0000 7f9c06ffd640 1 -- 172.21.15.84:0/1790180639 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf8030960 con 0x7f9c080b1ca0 2023-10-16T10:07:08.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.179+0000 7f9c06ffd640 1 -- 172.21.15.84:0/1790180639 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8030cd0 con 0x7f9c080b1ca0 2023-10-16T10:07:08.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.179+0000 7f9c0d09f640 1 -- 172.21.15.84:0/1790180639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c080b1ca0 msgr2=0x7f9c080d36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:08.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.179+0000 7f9c0d09f640 1 --2- 172.21.15.84:0/1790180639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c080b1ca0 0x7f9c080d36b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80097a0 tx=0x7f9bf802b940 comp rx=0 tx=0).stop 2023-10-16T10:07:08.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.179+0000 7f9c0d09f640 1 -- 172.21.15.84:0/1790180639 shutdown_connections 2023-10-16T10:07:08.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.179+0000 7f9c0d09f640 1 --2- 172.21.15.84:0/1790180639 >> v2:172.21.15.157:3300/0 conn(0x7f9c080b2080 0x7f9c080ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.179+0000 7f9c0d09f640 1 --2- 172.21.15.84:0/1790180639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c080b1ca0 0x7f9c080d36b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.179+0000 7f9c0d09f640 1 -- 172.21.15.84:0/1790180639 >> 172.21.15.84:0/1790180639 conn(0x7f9c080d2b90 msgr2=0x7f9c080d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:08.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.180+0000 7f9c0d09f640 1 -- 172.21.15.84:0/1790180639 shutdown_connections 2023-10-16T10:07:08.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.180+0000 7f9c0d09f640 1 -- 172.21.15.84:0/1790180639 wait complete. 2023-10-16T10:07:08.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.181+0000 7f9c0d09f640 1 Processor -- start 2023-10-16T10:07:08.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.181+0000 7f9c0d09f640 1 -- start start 2023-10-16T10:07:08.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.182+0000 7f9c0d09f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c080b1ca0 0x7f9c08159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:08.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.182+0000 7f9c0d09f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c080b2080 0x7f9c08153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:08.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.182+0000 7f9c0d09f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c080ce960 0x7f9c0815d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:08.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.182+0000 7f9c0d09f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c081589b0 con 0x7f9c080b1ca0 2023-10-16T10:07:08.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.182+0000 7f9c0d09f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0815a330 con 0x7f9c080b2080 2023-10-16T10:07:08.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.182+0000 7f9c0d09f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c081592e0 con 0x7f9c080ce960 2023-10-16T10:07:08.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.183+0000 7f9c07fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c080b1ca0 0x7f9c08159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:08.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.183+0000 7f9c07fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c080b1ca0 0x7f9c08159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54668/0 (socket says 172.21.15.84:54668) 2023-10-16T10:07:08.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.183+0000 7f9c07fff640 1 -- 172.21.15.84:0/2115858804 learned_addr learned my addr 172.21.15.84:0/2115858804 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:08.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.183+0000 7f9c077fe640 1 --2- 172.21.15.84:0/2115858804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c080b2080 0x7f9c08153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:08.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.183+0000 7f9c0c89e640 1 --2- 172.21.15.84:0/2115858804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c080ce960 0x7f9c0815d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:08.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.183+0000 7f9c07fff640 1 -- 172.21.15.84:0/2115858804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c080ce960 msgr2=0x7f9c0815d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:08.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.183+0000 7f9c07fff640 1 --2- 172.21.15.84:0/2115858804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c080ce960 0x7f9c0815d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.183+0000 7f9c07fff640 1 -- 172.21.15.84:0/2115858804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c080b2080 msgr2=0x7f9c08153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:08.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.183+0000 7f9c07fff640 1 --2- 172.21.15.84:0/2115858804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c080b2080 0x7f9c08153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.183+0000 7f9c07fff640 1 -- 172.21.15.84:0/2115858804 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc009400 con 0x7f9c080b1ca0 2023-10-16T10:07:08.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.183+0000 7f9c0c89e640 1 --2- 172.21.15.84:0/2115858804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c080ce960 0x7f9c0815d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:08.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.183+0000 7f9c07fff640 1 --2- 172.21.15.84:0/2115858804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c080b1ca0 0x7f9c08159df0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f9bf802b980 tx=0x7f9bf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:08.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.184+0000 7f9c04ff9640 1 -- 172.21.15.84:0/2115858804 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf803e070 con 0x7f9c080b1ca0 2023-10-16T10:07:08.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.184+0000 7f9c04ff9640 1 -- 172.21.15.84:0/2115858804 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf8006e10 con 0x7f9c080b1ca0 2023-10-16T10:07:08.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.184+0000 7f9c04ff9640 1 -- 172.21.15.84:0/2115858804 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8003830 con 0x7f9c080b1ca0 2023-10-16T10:07:08.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.184+0000 7f9c0d09f640 1 -- 172.21.15.84:0/2115858804 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf8009400 con 0x7f9c080b1ca0 2023-10-16T10:07:08.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.184+0000 7f9c0d09f640 1 -- 172.21.15.84:0/2115858804 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c08158f30 con 0x7f9c080b1ca0 2023-10-16T10:07:08.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.185+0000 7f9c0d09f640 1 -- 172.21.15.84:0/2115858804 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c0804e9b0 con 0x7f9c080b1ca0 2023-10-16T10:07:08.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.186+0000 7f9c04ff9640 1 -- 172.21.15.84:0/2115858804 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 55) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f9bf8003990 con 0x7f9c080b1ca0 2023-10-16T10:07:08.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.187+0000 7f9c04ff9640 1 --2- 172.21.15.84:0/2115858804 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f9bf406b0e0 0x7f9bf406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:08.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.187+0000 7f9c04ff9640 1 -- 172.21.15.84:0/2115858804 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6082+0+0 (secure 0 0 0) 0x7f9bf80abb10 con 0x7f9c080b1ca0 2023-10-16T10:07:08.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.187+0000 7f9c077fe640 1 -- 172.21.15.84:0/2115858804 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f9bf406b0e0 msgr2=0x7f9bf406d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/985901542 2023-10-16T10:07:08.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.187+0000 7f9c077fe640 1 --2- 172.21.15.84:0/2115858804 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f9bf406b0e0 0x7f9bf406d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:07:08.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.191+0000 7f9c04ff9640 1 -- 172.21.15.84:0/2115858804 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9bf8076540 con 0x7f9c080b1ca0 2023-10-16T10:07:08.210 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:08.209+0000 7f9b4122b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:07:08.213 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:08.212+0000 7fe27260f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:07:08.264 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:08.263+0000 7f9b4122b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:07:08.265 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:08.264+0000 7fe27260f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:07:08.315 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:08.314+0000 7fe27260f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:07:08.316 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:08.315+0000 7f9b4122b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:07:08.379 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:08.377+0000 7fe27260f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:07:08.382 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:08.381+0000 7f9b4122b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:07:08.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.385+0000 7f9c0d09f640 1 -- 172.21.15.84:0/2115858804 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9c08073a40 con 0x7f9c080b1ca0 2023-10-16T10:07:08.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.386+0000 7f9c04ff9640 1 -- 172.21.15.84:0/2115858804 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9bf8079b60 con 0x7f9c080b1ca0 2023-10-16T10:07:08.388 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:07:08.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.388+0000 7f9c077fe640 1 -- 172.21.15.84:0/2115858804 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f9bf406b0e0 msgr2=0x7f9bf406d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/985901542 2023-10-16T10:07:08.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.388+0000 7f9c077fe640 1 --2- 172.21.15.84:0/2115858804 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f9bf406b0e0 0x7f9bf406d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:07:08.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.388+0000 7f9c0d09f640 1 -- 172.21.15.84:0/2115858804 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f9bf406b0e0 msgr2=0x7f9bf406d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:07:08.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.388+0000 7f9c0d09f640 1 --2- 172.21.15.84:0/2115858804 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f9bf406b0e0 0x7f9bf406d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.388+0000 7f9c0d09f640 1 -- 172.21.15.84:0/2115858804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c080b1ca0 msgr2=0x7f9c08159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:08.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.388+0000 7f9c0d09f640 1 --2- 172.21.15.84:0/2115858804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c080b1ca0 0x7f9c08159df0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f9bf802b980 tx=0x7f9bf80046c0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.388+0000 7f9c0d09f640 1 -- 172.21.15.84:0/2115858804 shutdown_connections 2023-10-16T10:07:08.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.388+0000 7f9c0d09f640 1 --2- 172.21.15.84:0/2115858804 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f9bf406b0e0 0x7f9bf406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.388+0000 7f9c0d09f640 1 --2- 172.21.15.84:0/2115858804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c080ce960 0x7f9c0815d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.388+0000 7f9c0d09f640 1 --2- 172.21.15.84:0/2115858804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c080b2080 0x7f9c08153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.388+0000 7f9c0d09f640 1 --2- 172.21.15.84:0/2115858804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c080b1ca0 0x7f9c08159df0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.388+0000 7f9c0d09f640 1 -- 172.21.15.84:0/2115858804 >> 172.21.15.84:0/2115858804 conn(0x7f9c080d2b90 msgr2=0x7f9c080b0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:08.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.389+0000 7f9c0d09f640 1 -- 172.21.15.84:0/2115858804 shutdown_connections 2023-10-16T10:07:08.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.389+0000 7f9c0d09f640 1 -- 172.21.15.84:0/2115858804 wait complete. 2023-10-16T10:07:08.407 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:07:08.429 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:08.427+0000 7fe27260f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:07:08.435 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:08.434+0000 7f9b4122b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:07:08.540 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:08.539+0000 7fe27260f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:07:08.552 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:08.551+0000 7f9b4122b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:07:08.602 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:08.601+0000 7fe27260f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:07:08.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.605+0000 7fc7bd0bb640 1 Processor -- start 2023-10-16T10:07:08.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.605+0000 7fc7bd0bb640 1 -- start start 2023-10-16T10:07:08.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.607+0000 7fc7bd0bb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7b815cba0 0x7fc7b8071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:08.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.607+0000 7fc7bd0bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b80cdf10 0x7fc7b806aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:08.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.607+0000 7fc7bd0bb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b80d0500 con 0x7fc7b80720d0 2023-10-16T10:07:08.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.607+0000 7fc7bd0bb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b80d07c0 con 0x7fc7b815cba0 2023-10-16T10:07:08.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.607+0000 7fc7bd0bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b80d0aa0 con 0x7fc7b80cdf10 2023-10-16T10:07:08.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.608+0000 7fc7b7fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7b815cba0 0x7fc7b8071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:08.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.608+0000 7fc7b77fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc7b80720d0 0x7fc7b80cd800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:44492/0 (socket says 172.21.15.84:44492) 2023-10-16T10:07:08.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.608+0000 7fc7b77fe640 1 -- 172.21.15.84:0/3989275234 learned_addr learned my addr 172.21.15.84:0/3989275234 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:08.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.608+0000 7fc7bc8ba640 1 --2- 172.21.15.84:0/3989275234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b80cdf10 0x7fc7b806aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:08.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.608+0000 7fc7b6ffd640 1 -- 172.21.15.84:0/3989275234 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1975016778 0 0) 0x7fc7b80d0500 con 0x7fc7b80720d0 2023-10-16T10:07:08.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.608+0000 7fc7b6ffd640 1 -- 172.21.15.84:0/3989275234 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7a0003520 con 0x7fc7b80720d0 2023-10-16T10:07:08.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.608+0000 7fc7b7fff640 1 -- 172.21.15.84:0/3989275234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b80cdf10 msgr2=0x7fc7b806aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:08.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.608+0000 7fc7b7fff640 1 --2- 172.21.15.84:0/3989275234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b80cdf10 0x7fc7b806aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.608+0000 7fc7b7fff640 1 -- 172.21.15.84:0/3989275234 >> v1:172.21.15.157:6789/0 conn(0x7fc7b80720d0 legacy=0x7fc7b80cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:08.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.608+0000 7fc7b7fff640 1 -- 172.21.15.84:0/3989275234 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b80d0d80 con 0x7fc7b815cba0 2023-10-16T10:07:08.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.610+0000 7fc7b7fff640 1 --2- 172.21.15.84:0/3989275234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b815cba0 0x7fc7b8071b90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80097a0 tx=0x7fc7a802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c6b0d2ebeb19ddc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:08.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.610+0000 7fc7b6ffd640 1 -- 172.21.15.84:0/3989275234 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8031830 con 0x7fc7b815cba0 2023-10-16T10:07:08.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.610+0000 7fc7b6ffd640 1 -- 172.21.15.84:0/3989275234 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a8031990 con 0x7fc7b815cba0 2023-10-16T10:07:08.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.610+0000 7fc7b6ffd640 1 -- 172.21.15.84:0/3989275234 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8031d00 con 0x7fc7b815cba0 2023-10-16T10:07:08.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.611+0000 7fc7bd0bb640 1 -- 172.21.15.84:0/3989275234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b815cba0 msgr2=0x7fc7b8071b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:08.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.611+0000 7fc7bd0bb640 1 --2- 172.21.15.84:0/3989275234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b815cba0 0x7fc7b8071b90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80097a0 tx=0x7fc7a802b8d0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.611+0000 7fc7bd0bb640 1 -- 172.21.15.84:0/3989275234 shutdown_connections 2023-10-16T10:07:08.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.611+0000 7fc7bd0bb640 1 --2- 172.21.15.84:0/3989275234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b80cdf10 0x7fc7b806aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.611+0000 7fc7bd0bb640 1 --2- 172.21.15.84:0/3989275234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b815cba0 0x7fc7b8071b90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.611+0000 7fc7bd0bb640 1 -- 172.21.15.84:0/3989275234 >> 172.21.15.84:0/3989275234 conn(0x7fc7b8065ed0 msgr2=0x7fc7b80662e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:08.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.611+0000 7fc7bd0bb640 1 -- 172.21.15.84:0/3989275234 shutdown_connections 2023-10-16T10:07:08.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.611+0000 7fc7bd0bb640 1 -- 172.21.15.84:0/3989275234 wait complete. 2023-10-16T10:07:08.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.613+0000 7fc7bd0bb640 1 Processor -- start 2023-10-16T10:07:08.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.613+0000 7fc7bd0bb640 1 -- start start 2023-10-16T10:07:08.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.614+0000 7fc7bd0bb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7b80720d0 0x7fc7b80b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:08.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.614+0000 7fc7bd0bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b80cdf10 0x7fc7b814acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:08.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.614+0000 7fc7bd0bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b815cba0 0x7fc7b814b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:08.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.614+0000 7fc7bd0bb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b814a9f0 con 0x7fc7b80720d0 2023-10-16T10:07:08.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.614+0000 7fc7bd0bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b8149950 con 0x7fc7b815cba0 2023-10-16T10:07:08.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.614+0000 7fc7bd0bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b8149c30 con 0x7fc7b80cdf10 2023-10-16T10:07:08.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.614+0000 7fc7b7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7b80720d0 0x7fc7b80b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:08.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.614+0000 7fc7b7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7b80720d0 0x7fc7b80b1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54676/0 (socket says 172.21.15.84:54676) 2023-10-16T10:07:08.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.614+0000 7fc7b7fff640 1 -- 172.21.15.84:0/2051240629 learned_addr learned my addr 172.21.15.84:0/2051240629 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:08.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.614+0000 7fc7bc8ba640 1 --2- 172.21.15.84:0/2051240629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b815cba0 0x7fc7b814b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:08.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.614+0000 7fc7b77fe640 1 --2- 172.21.15.84:0/2051240629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b80cdf10 0x7fc7b814acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:08.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.614+0000 7fc7b7fff640 1 -- 172.21.15.84:0/2051240629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b80cdf10 msgr2=0x7fc7b814acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:08.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.614+0000 7fc7b7fff640 1 --2- 172.21.15.84:0/2051240629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b80cdf10 0x7fc7b814acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.614+0000 7fc7b7fff640 1 -- 172.21.15.84:0/2051240629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b815cba0 msgr2=0x7fc7b814b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:08.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.614+0000 7fc7b7fff640 1 --2- 172.21.15.84:0/2051240629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b815cba0 0x7fc7b814b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.614+0000 7fc7b7fff640 1 -- 172.21.15.84:0/2051240629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a8009400 con 0x7fc7b80720d0 2023-10-16T10:07:08.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.615+0000 7fc7b7fff640 1 --2- 172.21.15.84:0/2051240629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7b80720d0 0x7fc7b80b1400 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8000c00 tx=0x7fc7a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:08.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.615+0000 7fc7b4ff9640 1 -- 172.21.15.84:0/2051240629 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8004450 con 0x7fc7b80720d0 2023-10-16T10:07:08.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.615+0000 7fc7b4ff9640 1 -- 172.21.15.84:0/2051240629 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a803f070 con 0x7fc7b80720d0 2023-10-16T10:07:08.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.615+0000 7fc7bd0bb640 1 -- 172.21.15.84:0/2051240629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b8149f10 con 0x7fc7b80720d0 2023-10-16T10:07:08.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.615+0000 7fc7b4ff9640 1 -- 172.21.15.84:0/2051240629 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8003810 con 0x7fc7b80720d0 2023-10-16T10:07:08.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.615+0000 7fc7bd0bb640 1 -- 172.21.15.84:0/2051240629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b814a4d0 con 0x7fc7b80720d0 2023-10-16T10:07:08.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.616+0000 7fc7bd0bb640 1 -- 172.21.15.84:0/2051240629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b804e9b0 con 0x7fc7b80720d0 2023-10-16T10:07:08.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.617+0000 7fc7b4ff9640 1 -- 172.21.15.84:0/2051240629 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 55) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fc7a8003970 con 0x7fc7b80720d0 2023-10-16T10:07:08.624 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:08.618+0000 7f9b4122b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:07:08.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.618+0000 7fc7b4ff9640 1 --2- 172.21.15.84:0/2051240629 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fc7a406b0e0 0x7fc7a406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:08.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.618+0000 7fc7b4ff9640 1 -- 172.21.15.84:0/2051240629 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6082+0+0 (secure 0 0 0) 0x7fc7a8041070 con 0x7fc7b80720d0 2023-10-16T10:07:08.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.618+0000 7fc7b77fe640 1 -- 172.21.15.84:0/2051240629 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fc7a406b0e0 msgr2=0x7fc7a406d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/985901542 2023-10-16T10:07:08.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.618+0000 7fc7b77fe640 1 --2- 172.21.15.84:0/2051240629 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fc7a406b0e0 0x7fc7a406d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:07:08.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.626+0000 7fc7b4ff9640 1 -- 172.21.15.84:0/2051240629 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc7a8076b50 con 0x7fc7b80720d0 2023-10-16T10:07:08.654 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:08.653+0000 7fe27260f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:07:08.674 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:08.672+0000 7f9b4122b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:07:08.712 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:08.710+0000 7fe27260f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:07:08.733 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:08.732+0000 7f9b4122b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:07:08.761 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:08.760+0000 7fe27260f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:07:08.785 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:08.783+0000 7f9b4122b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:07:08.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.819+0000 7fc7b77fe640 1 -- 172.21.15.84:0/2051240629 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fc7a406b0e0 msgr2=0x7fc7a406d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/985901542 2023-10-16T10:07:08.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.819+0000 7fc7b77fe640 1 --2- 172.21.15.84:0/2051240629 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fc7a406b0e0 0x7fc7a406d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:07:08.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.821+0000 7fc7bd0bb640 1 -- 172.21.15.84:0/2051240629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc7b81604e0 con 0x7fc7b80720d0 2023-10-16T10:07:08.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.822+0000 7fc7b4ff9640 1 -- 172.21.15.84:0/2051240629 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc7a8035020 con 0x7fc7b80720d0 2023-10-16T10:07:08.824 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T10:07:08.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.825+0000 7fc7bd0bb640 1 -- 172.21.15.84:0/2051240629 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fc7a406b0e0 msgr2=0x7fc7a406d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:07:08.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.825+0000 7fc7bd0bb640 1 --2- 172.21.15.84:0/2051240629 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fc7a406b0e0 0x7fc7a406d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.825+0000 7fc7bd0bb640 1 -- 172.21.15.84:0/2051240629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7b80720d0 msgr2=0x7fc7b80b1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:08.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.825+0000 7fc7bd0bb640 1 --2- 172.21.15.84:0/2051240629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7b80720d0 0x7fc7b80b1400 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8000c00 tx=0x7fc7a80046c0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.825+0000 7fc7bd0bb640 1 -- 172.21.15.84:0/2051240629 shutdown_connections 2023-10-16T10:07:08.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.825+0000 7fc7bd0bb640 1 --2- 172.21.15.84:0/2051240629 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fc7a406b0e0 0x7fc7a406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.825+0000 7fc7bd0bb640 1 --2- 172.21.15.84:0/2051240629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7b815cba0 0x7fc7b814b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.825+0000 7fc7bd0bb640 1 --2- 172.21.15.84:0/2051240629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7b80cdf10 0x7fc7b814acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.825+0000 7fc7bd0bb640 1 --2- 172.21.15.84:0/2051240629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7b80720d0 0x7fc7b80b1400 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:08.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.825+0000 7fc7bd0bb640 1 -- 172.21.15.84:0/2051240629 >> 172.21.15.84:0/2051240629 conn(0x7fc7b8065ed0 msgr2=0x7fc7b80b5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:08.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.825+0000 7fc7bd0bb640 1 -- 172.21.15.84:0/2051240629 shutdown_connections 2023-10-16T10:07:08.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:08.825+0000 7fc7bd0bb640 1 -- 172.21.15.84:0/2051240629 wait complete. 2023-10-16T10:07:08.845 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:07:09.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.093+0000 7f13a7821640 1 Processor -- start 2023-10-16T10:07:09.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.093+0000 7f13a7821640 1 -- start start 2023-10-16T10:07:09.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.095+0000 7f13a7821640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a015d150 0x7f13a00ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:09.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.095+0000 7f13a7821640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f13a0160b90 0x7f13a0160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:09.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.095+0000 7f13a7821640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a00ac890 con 0x7f13a0069da0 2023-10-16T10:07:09.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.095+0000 7f13a7821640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a015d530 con 0x7f13a0160b90 2023-10-16T10:07:09.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.095+0000 7f13a7821640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a0161650 con 0x7f13a015d150 2023-10-16T10:07:09.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.095+0000 7f13a4dbc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f13a0160b90 0x7f13a0160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:09.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.095+0000 7f13a4dbc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f13a0160b90 0x7f13a0160fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54682/0 (socket says 172.21.15.84:54682) 2023-10-16T10:07:09.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.095+0000 7f13a4dbc640 1 -- 172.21.15.84:0/1600942167 learned_addr learned my addr 172.21.15.84:0/1600942167 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:09.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.095+0000 7f13a5dbe640 1 --1- 172.21.15.84:0/1600942167 >> v1:172.21.15.84:6789/0 conn(0x7f13a0069da0 0x7f13a006a1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54964/0 (socket says 172.21.15.84:54964) 2023-10-16T10:07:09.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.096+0000 7f13a55bd640 1 --2- 172.21.15.84:0/1600942167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a015d150 0x7f13a00ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:09.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.096+0000 7f13a4dbc640 1 -- 172.21.15.84:0/1600942167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a015d150 msgr2=0x7f13a00ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:09.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.096+0000 7f13a4dbc640 1 --2- 172.21.15.84:0/1600942167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a015d150 0x7f13a00ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.096+0000 7f13a4dbc640 1 -- 172.21.15.84:0/1600942167 >> v1:172.21.15.84:6789/0 conn(0x7f13a0069da0 legacy=0x7f13a006a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:09.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.096+0000 7f13a4dbc640 1 -- 172.21.15.84:0/1600942167 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a0161930 con 0x7f13a0160b90 2023-10-16T10:07:09.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.096+0000 7f1397fff640 1 -- 172.21.15.84:0/1600942167 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763902335 0 0) 0x7f13a00ac890 con 0x7f13a0069da0 2023-10-16T10:07:09.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.097+0000 7f13a4dbc640 1 --2- 172.21.15.84:0/1600942167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13a0160b90 0x7f13a0160fb0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f1390009870 tx=0x7f139002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93ecfe3108d0201 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:09.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.098+0000 7f1397fff640 1 -- 172.21.15.84:0/1600942167 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390030840 con 0x7f13a0160b90 2023-10-16T10:07:09.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.098+0000 7f1397fff640 1 -- 172.21.15.84:0/1600942167 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13900309a0 con 0x7f13a0160b90 2023-10-16T10:07:09.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.098+0000 7f1397fff640 1 -- 172.21.15.84:0/1600942167 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390030d30 con 0x7f13a0160b90 2023-10-16T10:07:09.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.098+0000 7f13a7821640 1 -- 172.21.15.84:0/1600942167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13a0160b90 msgr2=0x7f13a0160fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:09.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.098+0000 7f13a7821640 1 --2- 172.21.15.84:0/1600942167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13a0160b90 0x7f13a0160fb0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f1390009870 tx=0x7f139002b980 comp rx=0 tx=0).stop 2023-10-16T10:07:09.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.099+0000 7f13a7821640 1 -- 172.21.15.84:0/1600942167 shutdown_connections 2023-10-16T10:07:09.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.099+0000 7f13a7821640 1 --2- 172.21.15.84:0/1600942167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13a0160b90 0x7f13a0160fb0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.099+0000 7f13a7821640 1 --2- 172.21.15.84:0/1600942167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a015d150 0x7f13a00ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.099+0000 7f13a7821640 1 -- 172.21.15.84:0/1600942167 >> 172.21.15.84:0/1600942167 conn(0x7f13a00ce9b0 msgr2=0x7f13a00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:09.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.099+0000 7f13a7821640 1 -- 172.21.15.84:0/1600942167 shutdown_connections 2023-10-16T10:07:09.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.099+0000 7f13a7821640 1 -- 172.21.15.84:0/1600942167 wait complete. 2023-10-16T10:07:09.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.101+0000 7f13a7821640 1 Processor -- start 2023-10-16T10:07:09.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.101+0000 7f13a7821640 1 -- start start 2023-10-16T10:07:09.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.102+0000 7f13a7821640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a0069da0 0x7f13a00c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:09.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.102+0000 7f13a7821640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13a015d150 0x7f13a00c3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:09.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.102+0000 7f13a7821640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13a0160b90 0x7f13a01f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:09.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.102+0000 7f13a7821640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a00c3d10 con 0x7f13a015d150 2023-10-16T10:07:09.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.102+0000 7f13a7821640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a00c3ff0 con 0x7f13a0160b90 2023-10-16T10:07:09.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.102+0000 7f13a7821640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a00c42d0 con 0x7f13a0069da0 2023-10-16T10:07:09.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.102+0000 7f13a4dbc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13a015d150 0x7f13a00c3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:09.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.102+0000 7f13a4dbc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13a015d150 0x7f13a00c3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54694/0 (socket says 172.21.15.84:54694) 2023-10-16T10:07:09.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.102+0000 7f13a4dbc640 1 -- 172.21.15.84:0/3669624471 learned_addr learned my addr 172.21.15.84:0/3669624471 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:09.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.102+0000 7f13a5dbe640 1 --2- 172.21.15.84:0/3669624471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13a0160b90 0x7f13a01f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:09.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.102+0000 7f13a55bd640 1 --2- 172.21.15.84:0/3669624471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a0069da0 0x7f13a00c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:09.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.102+0000 7f13a4dbc640 1 -- 172.21.15.84:0/3669624471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a0069da0 msgr2=0x7f13a00c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:09.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.102+0000 7f13a4dbc640 1 --2- 172.21.15.84:0/3669624471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a0069da0 0x7f13a00c6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.102+0000 7f13a4dbc640 1 -- 172.21.15.84:0/3669624471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13a0160b90 msgr2=0x7f13a01f1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:09.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.102+0000 7f13a4dbc640 1 --2- 172.21.15.84:0/3669624471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13a0160b90 0x7f13a01f1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.103+0000 7f13a4dbc640 1 -- 172.21.15.84:0/3669624471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13900094d0 con 0x7f13a015d150 2023-10-16T10:07:09.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.103+0000 7f13a4dbc640 1 --2- 172.21.15.84:0/3669624471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13a015d150 0x7f13a00c3610 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f13900090b0 tx=0x7f1390004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:09.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.103+0000 7f1395ffb640 1 -- 172.21.15.84:0/3669624471 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390039b50 con 0x7f13a015d150 2023-10-16T10:07:09.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.103+0000 7f1395ffb640 1 -- 172.21.15.84:0/3669624471 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1390039cf0 con 0x7f13a015d150 2023-10-16T10:07:09.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.103+0000 7f13a7821640 1 -- 172.21.15.84:0/3669624471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a00c45b0 con 0x7f13a015d150 2023-10-16T10:07:09.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.103+0000 7f1395ffb640 1 -- 172.21.15.84:0/3669624471 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390043650 con 0x7f13a015d150 2023-10-16T10:07:09.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.103+0000 7f13a7821640 1 -- 172.21.15.84:0/3669624471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a01f2400 con 0x7f13a015d150 2023-10-16T10:07:09.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.104+0000 7f13a7821640 1 -- 172.21.15.84:0/3669624471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a01f2650 con 0x7f13a015d150 2023-10-16T10:07:09.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.106+0000 7f1395ffb640 1 -- 172.21.15.84:0/3669624471 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 55) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f13900437b0 con 0x7f13a015d150 2023-10-16T10:07:09.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.107+0000 7f1395ffb640 1 --2- 172.21.15.84:0/3669624471 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f138406b0b0 0x7f138406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:09.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.107+0000 7f13a55bd640 1 -- 172.21.15.84:0/3669624471 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f138406b0b0 msgr2=0x7f138406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/985901542 2023-10-16T10:07:09.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.107+0000 7f1395ffb640 1 -- 172.21.15.84:0/3669624471 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6082+0+0 (secure 0 0 0) 0x7f13900ac220 con 0x7f13a015d150 2023-10-16T10:07:09.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.107+0000 7f13a55bd640 1 --2- 172.21.15.84:0/3669624471 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f138406b0b0 0x7f138406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:07:09.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.115+0000 7f1395ffb640 1 -- 172.21.15.84:0/3669624471 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f13a01f2650 con 0x7f13a015d150 2023-10-16T10:07:09.144 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:09.142+0000 7fe27260f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:07:09.189 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:09.188+0000 7f9b4122b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:07:09.307 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:09.305+0000 7fe27260f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:07:09.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.308+0000 7f13a55bd640 1 -- 172.21.15.84:0/3669624471 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f138406b0b0 msgr2=0x7f138406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/985901542 2023-10-16T10:07:09.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.308+0000 7f13a55bd640 1 --2- 172.21.15.84:0/3669624471 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f138406b0b0 0x7f138406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:07:09.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.308+0000 7f13a7821640 1 -- 172.21.15.84:0/3669624471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f13a00c4ad0 con 0x7f13a015d150 2023-10-16T10:07:09.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.309+0000 7f1395ffb640 1 -- 172.21.15.84:0/3669624471 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7f13a00c4ad0 con 0x7f13a015d150 2023-10-16T10:07:09.311 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T10:07:09.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.312+0000 7f13a7821640 1 -- 172.21.15.84:0/3669624471 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f138406b0b0 msgr2=0x7f138406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:07:09.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.312+0000 7f13a7821640 1 --2- 172.21.15.84:0/3669624471 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f138406b0b0 0x7f138406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.312+0000 7f13a7821640 1 -- 172.21.15.84:0/3669624471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13a015d150 msgr2=0x7f13a00c3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:09.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.312+0000 7f13a7821640 1 --2- 172.21.15.84:0/3669624471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13a015d150 0x7f13a00c3610 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f13900090b0 tx=0x7f1390004730 comp rx=0 tx=0).stop 2023-10-16T10:07:09.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.312+0000 7f13a7821640 1 -- 172.21.15.84:0/3669624471 shutdown_connections 2023-10-16T10:07:09.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.312+0000 7f13a7821640 1 --2- 172.21.15.84:0/3669624471 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f138406b0b0 0x7f138406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.312+0000 7f13a7821640 1 --2- 172.21.15.84:0/3669624471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13a0160b90 0x7f13a01f1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.313+0000 7f13a7821640 1 --2- 172.21.15.84:0/3669624471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13a015d150 0x7f13a00c3610 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.313+0000 7f13a7821640 1 --2- 172.21.15.84:0/3669624471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a0069da0 0x7f13a00c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.313+0000 7f13a7821640 1 -- 172.21.15.84:0/3669624471 >> 172.21.15.84:0/3669624471 conn(0x7f13a00ce9b0 msgr2=0x7f13a00c8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:09.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.313+0000 7f13a7821640 1 -- 172.21.15.84:0/3669624471 shutdown_connections 2023-10-16T10:07:09.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.313+0000 7f13a7821640 1 -- 172.21.15.84:0/3669624471 wait complete. 2023-10-16T10:07:09.333 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T10:07:09.357 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:09.356+0000 7f9b4122b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:07:09.421 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:09.419+0000 7fe27260f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:07:09.471 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:09.469+0000 7fe27260f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:07:09.474 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:09.472+0000 7f9b4122b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:07:09.526 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:09.525+0000 7f9b4122b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:07:09.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.528+0000 7f8c26415640 1 Processor -- start 2023-10-16T10:07:09.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.528+0000 7f8c26415640 1 -- start start 2023-10-16T10:07:09.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.530+0000 7f8c26415640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8c2015cef0 0x7f8c200ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:09.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.530+0000 7f8c26415640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c200ae070 con 0x7f8c200adc00 2023-10-16T10:07:09.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.530+0000 7f8c26415640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c20070530 con 0x7f8c200ccd30 2023-10-16T10:07:09.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.530+0000 7f8c26415640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c200711b0 con 0x7f8c2015cef0 2023-10-16T10:07:09.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.530+0000 7f8c1ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8c2015cef0 0x7f8c200ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:09.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.530+0000 7f8c1ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8c2015cef0 0x7f8c200ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54702/0 (socket says 172.21.15.84:54702) 2023-10-16T10:07:09.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.530+0000 7f8c1ffff640 1 -- 172.21.15.84:0/294714311 learned_addr learned my addr 172.21.15.84:0/294714311 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:09.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.530+0000 7f8c1ffff640 1 -- 172.21.15.84:0/294714311 >> v1:172.21.15.157:6789/0 conn(0x7f8c200ccd30 legacy=0x7f8c2006b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:09.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.530+0000 7f8c1ffff640 1 -- 172.21.15.84:0/294714311 >> v1:172.21.15.84:6789/0 conn(0x7f8c200adc00 legacy=0x7f8c200cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:09.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.530+0000 7f8c1ffff640 1 -- 172.21.15.84:0/294714311 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c20070810 con 0x7f8c2015cef0 2023-10-16T10:07:09.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.531+0000 7f8c1ffff640 1 --2- 172.21.15.84:0/294714311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c2015cef0 0x7f8c200ad2a0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009870 tx=0x7f8c1002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b37ba5b227970fe server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:09.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.532+0000 7f8c1effd640 1 -- 172.21.15.84:0/294714311 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10031870 con 0x7f8c2015cef0 2023-10-16T10:07:09.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.532+0000 7f8c1effd640 1 -- 172.21.15.84:0/294714311 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c100319d0 con 0x7f8c2015cef0 2023-10-16T10:07:09.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.532+0000 7f8c1effd640 1 -- 172.21.15.84:0/294714311 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10031d60 con 0x7f8c2015cef0 2023-10-16T10:07:09.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.532+0000 7f8c26415640 1 -- 172.21.15.84:0/294714311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c2015cef0 msgr2=0x7f8c200ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:09.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.532+0000 7f8c26415640 1 --2- 172.21.15.84:0/294714311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c2015cef0 0x7f8c200ad2a0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009870 tx=0x7f8c1002b910 comp rx=0 tx=0).stop 2023-10-16T10:07:09.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.532+0000 7f8c26415640 1 -- 172.21.15.84:0/294714311 shutdown_connections 2023-10-16T10:07:09.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.532+0000 7f8c26415640 1 --2- 172.21.15.84:0/294714311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c2015cef0 0x7f8c200ad2a0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.532+0000 7f8c26415640 1 -- 172.21.15.84:0/294714311 >> 172.21.15.84:0/294714311 conn(0x7f8c200ce840 msgr2=0x7f8c200cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:09.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.533+0000 7f8c26415640 1 -- 172.21.15.84:0/294714311 shutdown_connections 2023-10-16T10:07:09.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.533+0000 7f8c26415640 1 -- 172.21.15.84:0/294714311 wait complete. 2023-10-16T10:07:09.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.534+0000 7f8c26415640 1 Processor -- start 2023-10-16T10:07:09.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.534+0000 7f8c26415640 1 -- start start 2023-10-16T10:07:09.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.535+0000 7f8c26415640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c200adc00 0x7f8c201f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:09.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.535+0000 7f8c26415640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c200ccd30 0x7f8c200c2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:09.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.535+0000 7f8c26415640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c2015cef0 0x7f8c200b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:09.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.535+0000 7f8c26415640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c200c3960 con 0x7f8c200adc00 2023-10-16T10:07:09.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.535+0000 7f8c26415640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c200c2d50 con 0x7f8c2015cef0 2023-10-16T10:07:09.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.535+0000 7f8c26415640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c200c3030 con 0x7f8c200ccd30 2023-10-16T10:07:09.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.536+0000 7f8c1ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c200adc00 0x7f8c201f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:09.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.536+0000 7f8c1ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c200adc00 0x7f8c201f1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54714/0 (socket says 172.21.15.84:54714) 2023-10-16T10:07:09.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.536+0000 7f8c1ffff640 1 -- 172.21.15.84:0/393989270 learned_addr learned my addr 172.21.15.84:0/393989270 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:09.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.536+0000 7f8c249b2640 1 --2- 172.21.15.84:0/393989270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c2015cef0 0x7f8c200b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:09.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.536+0000 7f8c1f7fe640 1 --2- 172.21.15.84:0/393989270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c200ccd30 0x7f8c200c2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:09.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.536+0000 7f8c1ffff640 1 -- 172.21.15.84:0/393989270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c200ccd30 msgr2=0x7f8c200c2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:09.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.536+0000 7f8c1ffff640 1 --2- 172.21.15.84:0/393989270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c200ccd30 0x7f8c200c2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.536+0000 7f8c1ffff640 1 -- 172.21.15.84:0/393989270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c2015cef0 msgr2=0x7f8c200b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:09.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.536+0000 7f8c1ffff640 1 --2- 172.21.15.84:0/393989270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c2015cef0 0x7f8c200b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.536+0000 7f8c1ffff640 1 -- 172.21.15.84:0/393989270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c100094d0 con 0x7f8c200adc00 2023-10-16T10:07:09.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.536+0000 7f8c1ffff640 1 --2- 172.21.15.84:0/393989270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c200adc00 0x7f8c201f1950 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009870 tx=0x7f8c100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:09.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.537+0000 7f8c1cff9640 1 -- 172.21.15.84:0/393989270 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10031820 con 0x7f8c200adc00 2023-10-16T10:07:09.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.537+0000 7f8c1cff9640 1 -- 172.21.15.84:0/393989270 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c100443f0 con 0x7f8c200adc00 2023-10-16T10:07:09.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.537+0000 7f8c1cff9640 1 -- 172.21.15.84:0/393989270 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c100447a0 con 0x7f8c200adc00 2023-10-16T10:07:09.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.537+0000 7f8c26415640 1 -- 172.21.15.84:0/393989270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c200c3310 con 0x7f8c200adc00 2023-10-16T10:07:09.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.537+0000 7f8c26415640 1 -- 172.21.15.84:0/393989270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c20154fa0 con 0x7f8c200adc00 2023-10-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.538+0000 7f8c26415640 1 -- 172.21.15.84:0/393989270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c20155190 con 0x7f8c200adc00 2023-10-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.540+0000 7f8c1cff9640 1 -- 172.21.15.84:0/393989270 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 55) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f8c1004d450 con 0x7f8c200adc00 2023-10-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.541+0000 7f8c1cff9640 1 --2- 172.21.15.84:0/393989270 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f8c0c06b0e0 0x7f8c0c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.541+0000 7f8c1cff9640 1 -- 172.21.15.84:0/393989270 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6082+0+0 (secure 0 0 0) 0x7f8c1002d040 con 0x7f8c200adc00 2023-10-16T10:07:09.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.541+0000 7f8c1f7fe640 1 -- 172.21.15.84:0/393989270 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f8c0c06b0e0 msgr2=0x7f8c0c06d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/985901542 2023-10-16T10:07:09.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.541+0000 7f8c1f7fe640 1 --2- 172.21.15.84:0/393989270 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f8c0c06b0e0 0x7f8c0c06d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:07:09.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.545+0000 7f8c1cff9640 1 -- 172.21.15.84:0/393989270 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8c20155190 con 0x7f8c200adc00 2023-10-16T10:07:09.579 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:09.578+0000 7fe27260f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:07:09.642 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:09.640+0000 7f9b4122b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:07:09.686 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:09.685+0000 7fe27260f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:07:09.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.742+0000 7f8c1f7fe640 1 -- 172.21.15.84:0/393989270 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f8c0c06b0e0 msgr2=0x7f8c0c06d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/985901542 2023-10-16T10:07:09.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.742+0000 7f8c1f7fe640 1 --2- 172.21.15.84:0/393989270 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f8c0c06b0e0 0x7f8c0c06d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:07:09.753 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:09.751+0000 7f9b4122b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:07:09.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.755+0000 7f8c26415640 1 -- 172.21.15.84:0/393989270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8c20155190 con 0x7f8c200adc00 2023-10-16T10:07:09.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.755+0000 7f8c1cff9640 1 -- 172.21.15.84:0/393989270 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v112) v1 ==== 80+0+469 (secure 0 0 0) 0x7f8c20155190 con 0x7f8c200adc00 2023-10-16T10:07:09.757 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2051153636 2023-10-17T10:06:03.602359+0000 2023-10-16T10:07:09.757 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/965715898 2023-10-17T10:06:03.602359+0000 2023-10-16T10:07:09.758 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6833/3250823028 2023-10-17T10:06:03.602359+0000 2023-10-16T10:07:09.758 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6832/3250823028 2023-10-17T10:06:03.602359+0000 2023-10-16T10:07:09.758 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3381824109 2023-10-17T10:06:03.602359+0000 2023-10-16T10:07:09.758 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3871500870 2023-10-17T10:06:03.602359+0000 2023-10-16T10:07:09.759 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1287074955 2023-10-17T10:06:03.602359+0000 2023-10-16T10:07:09.759 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3830790285 2023-10-16T11:05:57.166133+0000 2023-10-16T10:07:09.760 INFO:teuthology.orchestra.run.smithi084.stderr:listed 8 entries 2023-10-16T10:07:09.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.758+0000 7f8c26415640 1 -- 172.21.15.84:0/393989270 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f8c0c06b0e0 msgr2=0x7f8c0c06d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:07:09.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.758+0000 7f8c26415640 1 --2- 172.21.15.84:0/393989270 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f8c0c06b0e0 0x7f8c0c06d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.758+0000 7f8c26415640 1 -- 172.21.15.84:0/393989270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c200adc00 msgr2=0x7f8c201f1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:09.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.758+0000 7f8c26415640 1 --2- 172.21.15.84:0/393989270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c200adc00 0x7f8c201f1950 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009870 tx=0x7f8c100046c0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.758+0000 7f8c26415640 1 -- 172.21.15.84:0/393989270 shutdown_connections 2023-10-16T10:07:09.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.758+0000 7f8c26415640 1 --2- 172.21.15.84:0/393989270 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f8c0c06b0e0 0x7f8c0c06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.758+0000 7f8c26415640 1 --2- 172.21.15.84:0/393989270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c2015cef0 0x7f8c200b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.758+0000 7f8c26415640 1 --2- 172.21.15.84:0/393989270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c200ccd30 0x7f8c200c2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.758+0000 7f8c26415640 1 --2- 172.21.15.84:0/393989270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c200adc00 0x7f8c201f1950 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:09.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.758+0000 7f8c26415640 1 -- 172.21.15.84:0/393989270 >> 172.21.15.84:0/393989270 conn(0x7f8c200ce840 msgr2=0x7f8c200ae290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:09.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.758+0000 7f8c26415640 1 -- 172.21.15.84:0/393989270 shutdown_connections 2023-10-16T10:07:09.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:09.758+0000 7f8c26415640 1 -- 172.21.15.84:0/393989270 wait complete. 2023-10-16T10:07:09.777 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T10:07:09.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:09.778 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T10:07:09.842 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:09.840+0000 7fe27260f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:07:09.890 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:09.889+0000 7fe27260f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:07:09.915 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T10:07:09.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:09.915 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T10:07:09.921 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:09.920+0000 7f9b4122b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:07:09.961 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:07:09.961 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:07:09.974 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:09.972+0000 7f9b4122b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:07:10.013 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:10.011+0000 7fe27260f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:07:10.016 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:07:10.017 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:07:10.017 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:07:10.071 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:07:10.072 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:07:10.072 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:07:10.096 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:10.095+0000 7f9b4122b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:07:10.129 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:07:10.129 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:07:10.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.335+0000 7f27f6fc0640 1 Processor -- start 2023-10-16T10:07:10.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.335+0000 7f27f6fc0640 1 -- start start 2023-10-16T10:07:10.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.336+0000 7f27f6fc0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f27f0160800 0x7f27f00ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:10.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.336+0000 7f27f6fc0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f006e170 con 0x7f27f0160420 2023-10-16T10:07:10.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.336+0000 7f27f6fc0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f00cd460 con 0x7f27f006b4c0 2023-10-16T10:07:10.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.336+0000 7f27f6fc0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f006e3c0 con 0x7f27f0160800 2023-10-16T10:07:10.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.337+0000 7f27f4d5c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f27f0160420 0x7f27f006adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54976/0 (socket says 172.21.15.84:54976) 2023-10-16T10:07:10.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.337+0000 7f27f4d5c640 1 -- 172.21.15.84:0/970120324 learned_addr learned my addr 172.21.15.84:0/970120324 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:10.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.337+0000 7f27f555d640 1 --2- 172.21.15.84:0/970120324 >> v2:172.21.15.84:3300/0 conn(0x7f27f0160800 0x7f27f00ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.337+0000 7f27ef7fe640 1 -- 172.21.15.84:0/970120324 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968798081 0 0) 0x7f27f006e170 con 0x7f27f0160420 2023-10-16T10:07:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.337+0000 7f27ef7fe640 1 -- 172.21.15.84:0/970120324 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27e0003520 con 0x7f27f0160420 2023-10-16T10:07:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.337+0000 7f27f555d640 1 -- 172.21.15.84:0/970120324 >> v1:172.21.15.157:6789/0 conn(0x7f27f006b4c0 legacy=0x7f27f0070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.337+0000 7f27f555d640 1 -- 172.21.15.84:0/970120324 >> v1:172.21.15.84:6789/0 conn(0x7f27f0160420 legacy=0x7f27f006adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.337+0000 7f27f555d640 1 -- 172.21.15.84:0/970120324 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f006e6a0 con 0x7f27f0160800 2023-10-16T10:07:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.338+0000 7f27f555d640 1 --2- 172.21.15.84:0/970120324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27f0160800 0x7f27f00ae170 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009d90 tx=0x7f27e802bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2091c1f803f93 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.339+0000 7f27ef7fe640 1 -- 172.21.15.84:0/970120324 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e802ccf0 con 0x7f27f0160800 2023-10-16T10:07:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.339+0000 7f27ef7fe640 1 -- 172.21.15.84:0/970120324 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e802ce50 con 0x7f27f0160800 2023-10-16T10:07:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.339+0000 7f27ef7fe640 1 -- 172.21.15.84:0/970120324 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e8034660 con 0x7f27f0160800 2023-10-16T10:07:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.339+0000 7f27f6fc0640 1 -- 172.21.15.84:0/970120324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27f0160800 msgr2=0x7f27f00ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.339+0000 7f27f6fc0640 1 --2- 172.21.15.84:0/970120324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27f0160800 0x7f27f00ae170 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009d90 tx=0x7f27e802bf40 comp rx=0 tx=0).stop 2023-10-16T10:07:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.339+0000 7f27f6fc0640 1 -- 172.21.15.84:0/970120324 shutdown_connections 2023-10-16T10:07:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.339+0000 7f27f6fc0640 1 --2- 172.21.15.84:0/970120324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27f0160800 0x7f27f00ae170 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.339+0000 7f27f6fc0640 1 -- 172.21.15.84:0/970120324 >> 172.21.15.84:0/970120324 conn(0x7f27f00b0d30 msgr2=0x7f27f00b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.339+0000 7f27f6fc0640 1 -- 172.21.15.84:0/970120324 shutdown_connections 2023-10-16T10:07:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.339+0000 7f27f6fc0640 1 -- 172.21.15.84:0/970120324 wait complete. 2023-10-16T10:07:10.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.341+0000 7f27f6fc0640 1 Processor -- start 2023-10-16T10:07:10.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.341+0000 7f27f6fc0640 1 -- start start 2023-10-16T10:07:10.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.342+0000 7f27f6fc0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27f006b4c0 0x7f27f0159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:10.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.342+0000 7f27f6fc0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f0160420 0x7f27f00b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:10.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.342+0000 7f27f6fc0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27f0160800 0x7f27f00b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:10.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.342+0000 7f27f6fc0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f00b35e0 con 0x7f27f006b4c0 2023-10-16T10:07:10.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.342+0000 7f27f6fc0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f01597d0 con 0x7f27f0160420 2023-10-16T10:07:10.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.342+0000 7f27f6fc0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f009d970 con 0x7f27f0160800 2023-10-16T10:07:10.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.342+0000 7f27f4d5c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27f006b4c0 0x7f27f0159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:10.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.342+0000 7f27f4d5c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27f006b4c0 0x7f27f0159290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54734/0 (socket says 172.21.15.84:54734) 2023-10-16T10:07:10.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.342+0000 7f27f4d5c640 1 -- 172.21.15.84:0/3018278636 learned_addr learned my addr 172.21.15.84:0/3018278636 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:10.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.343+0000 7f27effff640 1 --2- 172.21.15.84:0/3018278636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f0160420 0x7f27f00b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:10.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.343+0000 7f27f4d5c640 1 -- 172.21.15.84:0/3018278636 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27f0160800 msgr2=0x7f27f00b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:10.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.343+0000 7f27f4d5c640 1 --2- 172.21.15.84:0/3018278636 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27f0160800 0x7f27f00b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:10.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.343+0000 7f27f4d5c640 1 -- 172.21.15.84:0/3018278636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f0160420 msgr2=0x7f27f00b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:10.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.343+0000 7f27f4d5c640 1 --2- 172.21.15.84:0/3018278636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f0160420 0x7f27f00b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:10.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.343+0000 7f27f4d5c640 1 -- 172.21.15.84:0/3018278636 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e8009a70 con 0x7f27f006b4c0 2023-10-16T10:07:10.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.343+0000 7f27effff640 1 --2- 172.21.15.84:0/3018278636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f0160420 0x7f27f00b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:10.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.343+0000 7f27f4d5c640 1 --2- 172.21.15.84:0/3018278636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27f006b4c0 0x7f27f0159290 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f27e400c500 tx=0x7f27e400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:10.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.343+0000 7f27ed7fa640 1 -- 172.21.15.84:0/3018278636 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e4005ce0 con 0x7f27f006b4c0 2023-10-16T10:07:10.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.343+0000 7f27ed7fa640 1 -- 172.21.15.84:0/3018278636 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e4005e40 con 0x7f27f006b4c0 2023-10-16T10:07:10.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.343+0000 7f27f6fc0640 1 -- 172.21.15.84:0/3018278636 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f00b2a60 con 0x7f27f006b4c0 2023-10-16T10:07:10.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.343+0000 7f27ed7fa640 1 -- 172.21.15.84:0/3018278636 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e4007330 con 0x7f27f006b4c0 2023-10-16T10:07:10.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.343+0000 7f27f6fc0640 1 -- 172.21.15.84:0/3018278636 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f00b3020 con 0x7f27f006b4c0 2023-10-16T10:07:10.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.345+0000 7f27f6fc0640 1 -- 172.21.15.84:0/3018278636 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f004e160 con 0x7f27f006b4c0 2023-10-16T10:07:10.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.345+0000 7f27ed7fa640 1 -- 172.21.15.84:0/3018278636 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 55) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f27e4007550 con 0x7f27f006b4c0 2023-10-16T10:07:10.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.346+0000 7f27ed7fa640 1 --2- 172.21.15.84:0/3018278636 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f27d406b0b0 0x7f27d406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:10.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.346+0000 7f27ed7fa640 1 -- 172.21.15.84:0/3018278636 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6082+0+0 (secure 0 0 0) 0x7f27e4089280 con 0x7f27f006b4c0 2023-10-16T10:07:10.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.346+0000 7f27effff640 1 -- 172.21.15.84:0/3018278636 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f27d406b0b0 msgr2=0x7f27d406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/985901542 2023-10-16T10:07:10.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.347+0000 7f27effff640 1 --2- 172.21.15.84:0/3018278636 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f27d406b0b0 0x7f27d406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:07:10.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.350+0000 7f27ed7fa640 1 -- 172.21.15.84:0/3018278636 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f27e4053f00 con 0x7f27f006b4c0 2023-10-16T10:07:10.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.548+0000 7f27effff640 1 -- 172.21.15.84:0/3018278636 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f27d406b0b0 msgr2=0x7f27d406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/985901542 2023-10-16T10:07:10.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.548+0000 7f27effff640 1 --2- 172.21.15.84:0/3018278636 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f27d406b0b0 0x7f27d406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:07:10.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.580+0000 7f27f6fc0640 1 -- 172.21.15.84:0/3018278636 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27f00b51c0 con 0x7f27f006b4c0 2023-10-16T10:07:10.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.580+0000 7f27ed7fa640 1 -- 172.21.15.84:0/3018278636 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f27e4057520 con 0x7f27f006b4c0 2023-10-16T10:07:10.583 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:10.583 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3458903351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3458903351},{"type":"v1","addr":"172.21.15.84:6837","nonce":3458903351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4002935522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4002935522},{"type":"v1","addr":"172.21.15.157:6833","nonce":4002935522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5809,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1890551305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1890551305},{"type":"v1","addr":"172.21.15.157:6835","nonce":1890551305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:05:48.809508+0000","modified":"2023-10-16T10:05:51.856862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7521":{"gid":7521,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/3621445269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3621445269},{"type":"v1","addr":"172.21.15.84:6836","nonce":3621445269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-16T10:07:10.583 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 53 2023-10-16T10:07:10.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.583+0000 7f27f6fc0640 1 -- 172.21.15.84:0/3018278636 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f27d406b0b0 msgr2=0x7f27d406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:07:10.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.583+0000 7f27f6fc0640 1 --2- 172.21.15.84:0/3018278636 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f27d406b0b0 0x7f27d406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:10.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.583+0000 7f27f6fc0640 1 -- 172.21.15.84:0/3018278636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27f006b4c0 msgr2=0x7f27f0159290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:10.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.583+0000 7f27f6fc0640 1 --2- 172.21.15.84:0/3018278636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27f006b4c0 0x7f27f0159290 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f27e400c500 tx=0x7f27e400c9c0 comp rx=0 tx=0).stop 2023-10-16T10:07:10.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.583+0000 7f27f6fc0640 1 -- 172.21.15.84:0/3018278636 shutdown_connections 2023-10-16T10:07:10.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.583+0000 7f27f6fc0640 1 --2- 172.21.15.84:0/3018278636 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f27d406b0b0 0x7f27d406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:10.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.583+0000 7f27f6fc0640 1 --2- 172.21.15.84:0/3018278636 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27f0160800 0x7f27f00b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:10.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.583+0000 7f27f6fc0640 1 --2- 172.21.15.84:0/3018278636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f0160420 0x7f27f00b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:10.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.583+0000 7f27f6fc0640 1 --2- 172.21.15.84:0/3018278636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27f006b4c0 0x7f27f0159290 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:10.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.583+0000 7f27f6fc0640 1 -- 172.21.15.84:0/3018278636 >> 172.21.15.84:0/3018278636 conn(0x7f27f00b0d30 msgr2=0x7f27f0070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:10.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.583+0000 7f27f6fc0640 1 -- 172.21.15.84:0/3018278636 shutdown_connections 2023-10-16T10:07:10.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.583+0000 7f27f6fc0640 1 -- 172.21.15.84:0/3018278636 wait complete. 2023-10-16T10:07:10.601 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:07:10.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.794+0000 7fe7c34bb640 1 Processor -- start 2023-10-16T10:07:10.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.794+0000 7fe7c34bb640 1 -- start start 2023-10-16T10:07:10.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.795+0000 7fe7c34bb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe7bc160c80 0x7fe7bc161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:10.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.795+0000 7fe7c34bb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc0ae1a0 con 0x7fe7bc0ad7a0 2023-10-16T10:07:10.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.795+0000 7fe7c34bb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc0ae480 con 0x7fe7bc070950 2023-10-16T10:07:10.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.796+0000 7fe7c34bb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7bc0ae760 con 0x7fe7bc160c80 2023-10-16T10:07:10.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.796+0000 7fe7c24b9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe7bc160c80 0x7fe7bc161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:10.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.796+0000 7fe7c1cb8640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe7bc0ad7a0 0x7fe7bc070240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54986/0 (socket says 172.21.15.84:54986) 2023-10-16T10:07:10.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.796+0000 7fe7c1cb8640 1 -- 172.21.15.84:0/2452995291 learned_addr learned my addr 172.21.15.84:0/2452995291 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:10.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.796+0000 7fe7c14b7640 1 -- 172.21.15.84:0/2452995291 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3595655857 0 0) 0x7fe7bc0ae1a0 con 0x7fe7bc0ad7a0 2023-10-16T10:07:10.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.796+0000 7fe7c14b7640 1 -- 172.21.15.84:0/2452995291 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7a8003520 con 0x7fe7bc0ad7a0 2023-10-16T10:07:10.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.796+0000 7fe7c24b9640 1 -- 172.21.15.84:0/2452995291 >> v1:172.21.15.157:6789/0 conn(0x7fe7bc070950 legacy=0x7fe7bc140400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:10.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.796+0000 7fe7c24b9640 1 -- 172.21.15.84:0/2452995291 >> v1:172.21.15.84:6789/0 conn(0x7fe7bc0ad7a0 legacy=0x7fe7bc070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:10.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.796+0000 7fe7c24b9640 1 -- 172.21.15.84:0/2452995291 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc0aea40 con 0x7fe7bc160c80 2023-10-16T10:07:10.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.797+0000 7fe7c24b9640 1 --2- 172.21.15.84:0/2452995291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7bc160c80 0x7fe7bc161060 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80097a0 tx=0x7fe7b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e415fc91ae9d32a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:10.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.798+0000 7fe7c14b7640 1 -- 172.21.15.84:0/2452995291 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8031830 con 0x7fe7bc160c80 2023-10-16T10:07:10.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.798+0000 7fe7c14b7640 1 -- 172.21.15.84:0/2452995291 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b8031990 con 0x7fe7bc160c80 2023-10-16T10:07:10.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.798+0000 7fe7c14b7640 1 -- 172.21.15.84:0/2452995291 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8031d20 con 0x7fe7bc160c80 2023-10-16T10:07:10.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.798+0000 7fe7c34bb640 1 -- 172.21.15.84:0/2452995291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7bc160c80 msgr2=0x7fe7bc161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:10.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.798+0000 7fe7c34bb640 1 --2- 172.21.15.84:0/2452995291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7bc160c80 0x7fe7bc161060 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80097a0 tx=0x7fe7b802b8d0 comp rx=0 tx=0).stop 2023-10-16T10:07:10.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.798+0000 7fe7c34bb640 1 -- 172.21.15.84:0/2452995291 shutdown_connections 2023-10-16T10:07:10.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.798+0000 7fe7c34bb640 1 --2- 172.21.15.84:0/2452995291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7bc160c80 0x7fe7bc161060 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:10.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.798+0000 7fe7c34bb640 1 -- 172.21.15.84:0/2452995291 >> 172.21.15.84:0/2452995291 conn(0x7fe7bc0cc230 msgr2=0x7fe7bc0cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:10.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.799+0000 7fe7c34bb640 1 -- 172.21.15.84:0/2452995291 shutdown_connections 2023-10-16T10:07:10.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.799+0000 7fe7c34bb640 1 -- 172.21.15.84:0/2452995291 wait complete. 2023-10-16T10:07:10.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.800+0000 7fe7c34bb640 1 Processor -- start 2023-10-16T10:07:10.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.800+0000 7fe7c34bb640 1 -- start start 2023-10-16T10:07:10.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.801+0000 7fe7c34bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7bc070950 0x7fe7bc0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:10.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.801+0000 7fe7c34bb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7bc0ad7a0 0x7fe7bc0b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:10.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.801+0000 7fe7c34bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7bc160c80 0x7fe7bc0b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:10.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.801+0000 7fe7c34bb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0b3090 con 0x7fe7bc0ad7a0 2023-10-16T10:07:10.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.801+0000 7fe7c34bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0b4e00 con 0x7fe7bc070950 2023-10-16T10:07:10.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.801+0000 7fe7c34bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0b50e0 con 0x7fe7bc160c80 2023-10-16T10:07:10.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.801+0000 7fe7c1cb8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7bc0ad7a0 0x7fe7bc0b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:10.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.801+0000 7fe7c1cb8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7bc0ad7a0 0x7fe7bc0b3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54758/0 (socket says 172.21.15.84:54758) 2023-10-16T10:07:10.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.801+0000 7fe7c1cb8640 1 -- 172.21.15.84:0/4101086117 learned_addr learned my addr 172.21.15.84:0/4101086117 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:10.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.802+0000 7fe7c24b9640 1 --2- 172.21.15.84:0/4101086117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7bc070950 0x7fe7bc0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:10.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.802+0000 7fe7c2cba640 1 --2- 172.21.15.84:0/4101086117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7bc160c80 0x7fe7bc0b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:10.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.802+0000 7fe7c1cb8640 1 -- 172.21.15.84:0/4101086117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7bc160c80 msgr2=0x7fe7bc0b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:10.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.802+0000 7fe7c1cb8640 1 --2- 172.21.15.84:0/4101086117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7bc160c80 0x7fe7bc0b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:10.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.802+0000 7fe7c1cb8640 1 -- 172.21.15.84:0/4101086117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7bc070950 msgr2=0x7fe7bc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:10.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.802+0000 7fe7c1cb8640 1 --2- 172.21.15.84:0/4101086117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7bc070950 0x7fe7bc0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:10.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.802+0000 7fe7c1cb8640 1 -- 172.21.15.84:0/4101086117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b8009400 con 0x7fe7bc0ad7a0 2023-10-16T10:07:10.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.802+0000 7fe7c24b9640 1 --2- 172.21.15.84:0/4101086117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7bc070950 0x7fe7bc0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:07:10.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.802+0000 7fe7c1cb8640 1 --2- 172.21.15.84:0/4101086117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7bc0ad7a0 0x7fe7bc0b3a60 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00c500 tx=0x7fe7ac00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:10.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.802+0000 7fe7b2ffd640 1 -- 172.21.15.84:0/4101086117 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac005ce0 con 0x7fe7bc0ad7a0 2023-10-16T10:07:10.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.803+0000 7fe7b2ffd640 1 -- 172.21.15.84:0/4101086117 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ac005e40 con 0x7fe7bc0ad7a0 2023-10-16T10:07:10.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.803+0000 7fe7b2ffd640 1 -- 172.21.15.84:0/4101086117 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac007330 con 0x7fe7bc0ad7a0 2023-10-16T10:07:10.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.803+0000 7fe7c34bb640 1 -- 172.21.15.84:0/4101086117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7bc0b2a40 con 0x7fe7bc0ad7a0 2023-10-16T10:07:10.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.803+0000 7fe7c34bb640 1 -- 172.21.15.84:0/4101086117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7bc1e8df0 con 0x7fe7bc0ad7a0 2023-10-16T10:07:10.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.804+0000 7fe7c34bb640 1 -- 172.21.15.84:0/4101086117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7bc04e9b0 con 0x7fe7bc0ad7a0 2023-10-16T10:07:10.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.805+0000 7fe7b2ffd640 1 -- 172.21.15.84:0/4101086117 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 55) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fe7ac007550 con 0x7fe7bc0ad7a0 2023-10-16T10:07:10.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.806+0000 7fe7b2ffd640 1 --2- 172.21.15.84:0/4101086117 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fe7a006b0b0 0x7fe7a006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:10.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.806+0000 7fe7b2ffd640 1 -- 172.21.15.84:0/4101086117 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6082+0+0 (secure 0 0 0) 0x7fe7ac0891e0 con 0x7fe7bc0ad7a0 2023-10-16T10:07:10.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.806+0000 7fe7c24b9640 1 -- 172.21.15.84:0/4101086117 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fe7a006b0b0 msgr2=0x7fe7a006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/985901542 2023-10-16T10:07:10.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.806+0000 7fe7c24b9640 1 --2- 172.21.15.84:0/4101086117 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fe7a006b0b0 0x7fe7a006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:07:10.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:10.810+0000 7fe7b2ffd640 1 -- 172.21.15.84:0/4101086117 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe7ac053f00 con 0x7fe7bc0ad7a0 2023-10-16T10:07:10.833 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:10.832+0000 7fe27260f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:07:10.919 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:10.918+0000 7f9b4122b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:07:11.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.005+0000 7fe7c34bb640 1 -- 172.21.15.84:0/4101086117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7bc1e9410 con 0x7fe7bc0ad7a0 2023-10-16T10:07:11.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.007+0000 7fe7c24b9640 1 -- 172.21.15.84:0/4101086117 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fe7a006b0b0 msgr2=0x7fe7a006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/985901542 2023-10-16T10:07:11.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.007+0000 7fe7c24b9640 1 --2- 172.21.15.84:0/4101086117 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fe7a006b0b0 0x7fe7a006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:07:11.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.007+0000 7fe7b2ffd640 1 -- 172.21.15.84:0/4101086117 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+17096 (secure 0 0 0) 0x7fe7ac057520 con 0x7fe7bc0ad7a0 2023-10-16T10:07:11.010 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:11.010 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":112,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:07:05.125527+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:05:46.595888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-16T10:05:47.826019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.d","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2051153636":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/965715898":"2023-10-17T10:06:03.602359+0000","172.21.15.84:6833/3250823028":"2023-10-17T10:06:03.602359+0000","172.21.15.84:6832/3250823028":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/3381824109":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/3871500870":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/1287074955":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/3830790285":"2023-10-16T11:05:57.166133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:07:11.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.010+0000 7fe7c34bb640 1 -- 172.21.15.84:0/4101086117 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fe7a006b0b0 msgr2=0x7fe7a006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:07:11.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.010+0000 7fe7c34bb640 1 --2- 172.21.15.84:0/4101086117 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fe7a006b0b0 0x7fe7a006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.010+0000 7fe7c34bb640 1 -- 172.21.15.84:0/4101086117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7bc0ad7a0 msgr2=0x7fe7bc0b3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:11.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.010+0000 7fe7c34bb640 1 --2- 172.21.15.84:0/4101086117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7bc0ad7a0 0x7fe7bc0b3a60 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00c500 tx=0x7fe7ac00c9c0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.010+0000 7fe7c34bb640 1 -- 172.21.15.84:0/4101086117 shutdown_connections 2023-10-16T10:07:11.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.010+0000 7fe7c34bb640 1 --2- 172.21.15.84:0/4101086117 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7fe7a006b0b0 0x7fe7a006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.010+0000 7fe7c34bb640 1 --2- 172.21.15.84:0/4101086117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7bc160c80 0x7fe7bc0b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.010+0000 7fe7c34bb640 1 --2- 172.21.15.84:0/4101086117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7bc0ad7a0 0x7fe7bc0b3a60 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.010+0000 7fe7c34bb640 1 --2- 172.21.15.84:0/4101086117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7bc070950 0x7fe7bc0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.010+0000 7fe7c34bb640 1 -- 172.21.15.84:0/4101086117 >> 172.21.15.84:0/4101086117 conn(0x7fe7bc0cc230 msgr2=0x7fe7bc0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:11.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.010+0000 7fe7c34bb640 1 -- 172.21.15.84:0/4101086117 shutdown_connections 2023-10-16T10:07:11.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.010+0000 7fe7c34bb640 1 -- 172.21.15.84:0/4101086117 wait complete. 2023-10-16T10:07:11.029 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:05:46.595888+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:05:47.826019+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:07:11.030 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:07:11.129 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:11.127+0000 7fe27260f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:07:11.188 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:11.186+0000 7fe27260f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:07:11.230 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:11.229+0000 7f9b4122b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:07:11.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.234+0000 7f33932d1640 1 Processor -- start 2023-10-16T10:07:11.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.234+0000 7f33932d1640 1 -- start start 2023-10-16T10:07:11.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.235+0000 7f33932d1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f338c0b1ca0 0x7f338c0d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:11.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.235+0000 7f33932d1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f338c0b2080 0x7f338c0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:11.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.235+0000 7f33932d1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f338c0d0970 con 0x7f338c0ce960 2023-10-16T10:07:11.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.235+0000 7f33932d1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338c0d0ea0 con 0x7f338c0b1ca0 2023-10-16T10:07:11.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.235+0000 7f33932d1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338c0d0bc0 con 0x7f338c0b2080 2023-10-16T10:07:11.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.236+0000 7f33922cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f338c0b1ca0 0x7f338c0d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:11.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.236+0000 7f33922cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f338c0b1ca0 0x7f338c0d36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54768/0 (socket says 172.21.15.84:54768) 2023-10-16T10:07:11.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.236+0000 7f33922cf640 1 -- 172.21.15.84:0/1977867082 learned_addr learned my addr 172.21.15.84:0/1977867082 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:11.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.236+0000 7f3391ace640 1 --2- 172.21.15.84:0/1977867082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f338c0b2080 0x7f338c0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:11.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.236+0000 7f33922cf640 1 -- 172.21.15.84:0/1977867082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f338c0b2080 msgr2=0x7f338c0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:11.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.236+0000 7f33922cf640 1 --2- 172.21.15.84:0/1977867082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f338c0b2080 0x7f338c0ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.236+0000 7f33922cf640 1 -- 172.21.15.84:0/1977867082 >> v1:172.21.15.157:6789/0 conn(0x7f338c0ce960 legacy=0x7f338c0b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:11.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.236+0000 7f33922cf640 1 -- 172.21.15.84:0/1977867082 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c0cd940 con 0x7f338c0b1ca0 2023-10-16T10:07:11.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.236+0000 7f3391ace640 1 --2- 172.21.15.84:0/1977867082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f338c0b2080 0x7f338c0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:11.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.237+0000 7f33922cf640 1 --2- 172.21.15.84:0/1977867082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f338c0b1ca0 0x7f338c0d36b0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f3388009870 tx=0x7f338802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78c198e5f98fac server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:11.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.238+0000 7f33912cd640 1 -- 172.21.15.84:0/1977867082 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3388030840 con 0x7f338c0b1ca0 2023-10-16T10:07:11.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.238+0000 7f33912cd640 1 -- 172.21.15.84:0/1977867082 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33880309a0 con 0x7f338c0b1ca0 2023-10-16T10:07:11.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.238+0000 7f33912cd640 1 -- 172.21.15.84:0/1977867082 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3388030d30 con 0x7f338c0b1ca0 2023-10-16T10:07:11.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.238+0000 7f33932d1640 1 -- 172.21.15.84:0/1977867082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f338c0b1ca0 msgr2=0x7f338c0d36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:11.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.238+0000 7f33932d1640 1 --2- 172.21.15.84:0/1977867082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f338c0b1ca0 0x7f338c0d36b0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f3388009870 tx=0x7f338802b980 comp rx=0 tx=0).stop 2023-10-16T10:07:11.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.238+0000 7f33932d1640 1 -- 172.21.15.84:0/1977867082 shutdown_connections 2023-10-16T10:07:11.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.238+0000 7f33932d1640 1 --2- 172.21.15.84:0/1977867082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f338c0b2080 0x7f338c0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.238+0000 7f33932d1640 1 --2- 172.21.15.84:0/1977867082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f338c0b1ca0 0x7f338c0d36b0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.238+0000 7f33932d1640 1 -- 172.21.15.84:0/1977867082 >> 172.21.15.84:0/1977867082 conn(0x7f338c0d2b90 msgr2=0x7f338c0d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:11.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.238+0000 7f33932d1640 1 -- 172.21.15.84:0/1977867082 shutdown_connections 2023-10-16T10:07:11.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.239+0000 7f33932d1640 1 -- 172.21.15.84:0/1977867082 wait complete. 2023-10-16T10:07:11.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.240+0000 7f33932d1640 1 Processor -- start 2023-10-16T10:07:11.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.240+0000 7f33932d1640 1 -- start start 2023-10-16T10:07:11.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.241+0000 7f33932d1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f338c0b1ca0 0x7f338c0c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:11.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.241+0000 7f33932d1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f338c0b2080 0x7f338c15d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:11.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.241+0000 7f33932d1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f338c0ce960 0x7f338c09ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:11.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.241+0000 7f33932d1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c0a0520 con 0x7f338c0b2080 2023-10-16T10:07:11.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.241+0000 7f33932d1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c0a07a0 con 0x7f338c0ce960 2023-10-16T10:07:11.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.241+0000 7f33932d1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338c09c970 con 0x7f338c0b1ca0 2023-10-16T10:07:11.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.241+0000 7f3391ace640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f338c0b2080 0x7f338c15d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:11.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.242+0000 7f3391ace640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f338c0b2080 0x7f338c15d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54778/0 (socket says 172.21.15.84:54778) 2023-10-16T10:07:11.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.242+0000 7f3391ace640 1 -- 172.21.15.84:0/1045275296 learned_addr learned my addr 172.21.15.84:0/1045275296 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:11.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.242+0000 7f33922cf640 1 --2- 172.21.15.84:0/1045275296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f338c0b1ca0 0x7f338c0c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:11.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.242+0000 7f3392ad0640 1 --2- 172.21.15.84:0/1045275296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f338c0ce960 0x7f338c09ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:11.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.242+0000 7f3391ace640 1 -- 172.21.15.84:0/1045275296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f338c0b1ca0 msgr2=0x7f338c0c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:11.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.242+0000 7f3391ace640 1 --2- 172.21.15.84:0/1045275296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f338c0b1ca0 0x7f338c0c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.242+0000 7f3391ace640 1 -- 172.21.15.84:0/1045275296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f338c0ce960 msgr2=0x7f338c09ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:11.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.242+0000 7f3391ace640 1 --2- 172.21.15.84:0/1045275296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f338c0ce960 0x7f338c09ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.242+0000 7f3391ace640 1 -- 172.21.15.84:0/1045275296 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33880094d0 con 0x7f338c0b2080 2023-10-16T10:07:11.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.242+0000 7f3391ace640 1 --2- 172.21.15.84:0/1045275296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f338c0b2080 0x7f338c15d240 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f337c00c7f0 tx=0x7f337c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:11.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.242+0000 7f3382ffd640 1 -- 172.21.15.84:0/1045275296 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c007820 con 0x7f338c0b2080 2023-10-16T10:07:11.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.243+0000 7f3382ffd640 1 -- 172.21.15.84:0/1045275296 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337c007980 con 0x7f338c0b2080 2023-10-16T10:07:11.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.243+0000 7f3382ffd640 1 -- 172.21.15.84:0/1045275296 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c007cf0 con 0x7f338c0b2080 2023-10-16T10:07:11.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.243+0000 7f33932d1640 1 -- 172.21.15.84:0/1045275296 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338c09cc50 con 0x7f338c0b2080 2023-10-16T10:07:11.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.243+0000 7f33932d1640 1 -- 172.21.15.84:0/1045275296 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338c09d210 con 0x7f338c0b2080 2023-10-16T10:07:11.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.244+0000 7f33932d1640 1 -- 172.21.15.84:0/1045275296 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338c04e9b0 con 0x7f338c0b2080 2023-10-16T10:07:11.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.245+0000 7f3382ffd640 1 -- 172.21.15.84:0/1045275296 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 55) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f337c0113f0 con 0x7f338c0b2080 2023-10-16T10:07:11.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.246+0000 7f3382ffd640 1 --2- 172.21.15.84:0/1045275296 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f337006b0e0 0x7f337006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:11.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.246+0000 7f33922cf640 1 -- 172.21.15.84:0/1045275296 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f337006b0e0 msgr2=0x7f337006d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/985901542 2023-10-16T10:07:11.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.246+0000 7f3382ffd640 1 -- 172.21.15.84:0/1045275296 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6082+0+0 (secure 0 0 0) 0x7f337c08a0a0 con 0x7f338c0b2080 2023-10-16T10:07:11.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.246+0000 7f33922cf640 1 --2- 172.21.15.84:0/1045275296 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f337006b0e0 0x7f337006d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:07:11.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.250+0000 7f3382ffd640 1 -- 172.21.15.84:0/1045275296 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f337c054ac0 con 0x7f338c0b2080 2023-10-16T10:07:11.292 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:11.291+0000 7f9b4122b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:07:11.302 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:11.301+0000 7fe27260f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:07:11.410 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:11.409+0000 7f9b4122b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:07:11.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.424+0000 7f3382ffd640 1 -- 172.21.15.84:0/1045275296 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 56) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f337c054e90 con 0x7f338c0b2080 2023-10-16T10:07:11.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.425+0000 7f3382ffd640 1 -- 172.21.15.84:0/1045275296 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f337006b0e0 msgr2=0x7f337006d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:07:11.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.425+0000 7f3382ffd640 1 --2- 172.21.15.84:0/1045275296 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f337006b0e0 0x7f337006d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.453+0000 7f33932d1640 1 -- 172.21.15.84:0/1045275296 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f338c073a40 con 0x7f338c0b2080 2023-10-16T10:07:11.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.454+0000 7f3382ffd640 1 -- 172.21.15.84:0/1045275296 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f337c087c30 con 0x7f338c0b2080 2023-10-16T10:07:11.456 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:07:11.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.456+0000 7f33932d1640 1 -- 172.21.15.84:0/1045275296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f338c0b2080 msgr2=0x7f338c15d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:11.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.456+0000 7f33932d1640 1 --2- 172.21.15.84:0/1045275296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f338c0b2080 0x7f338c15d240 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f337c00c7f0 tx=0x7f337c00ccb0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.457+0000 7f33932d1640 1 -- 172.21.15.84:0/1045275296 shutdown_connections 2023-10-16T10:07:11.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.457+0000 7f33932d1640 1 --2- 172.21.15.84:0/1045275296 >> [v2:172.21.15.84:6832/985901542,v1:172.21.15.84:6833/985901542] conn(0x7f337006b0e0 0x7f337006d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.457+0000 7f33932d1640 1 --2- 172.21.15.84:0/1045275296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f338c0ce960 0x7f338c09ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.457+0000 7f33932d1640 1 --2- 172.21.15.84:0/1045275296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f338c0b2080 0x7f338c15d240 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.457+0000 7f33932d1640 1 --2- 172.21.15.84:0/1045275296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f338c0b1ca0 0x7f338c0c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.457+0000 7f33932d1640 1 -- 172.21.15.84:0/1045275296 >> 172.21.15.84:0/1045275296 conn(0x7f338c0d2b90 msgr2=0x7f338c15fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:11.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.457+0000 7f33932d1640 1 -- 172.21.15.84:0/1045275296 shutdown_connections 2023-10-16T10:07:11.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.457+0000 7f33932d1640 1 -- 172.21.15.84:0/1045275296 wait complete. 2023-10-16T10:07:11.475 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:07:11.502 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:11.500+0000 7f9ac4ddf640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:11.502 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:11.500+0000 7f9ac4ddf640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:11.502 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:11.500+0000 7f9ac4ddf640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:11.502 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:11.500+0000 7f9ac4ddf640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:11.502 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:11.500+0000 7f9ac4ddf640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:11.503 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:11.501+0000 7f9ac2ddb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:11.503 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:11.501+0000 7f9ac2ddb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:11.503 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:11.501+0000 7f9ac2ddb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:11.504 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:11.501+0000 7f9ac2ddb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:11.504 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:11.501+0000 7f9ac2ddb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:11.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.661+0000 7ff5598da640 1 Processor -- start 2023-10-16T10:07:11.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.661+0000 7ff5598da640 1 -- start start 2023-10-16T10:07:11.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.662+0000 7ff5598da640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff55415ad70 0x7ff554069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:11.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.662+0000 7ff5598da640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff5540caac0 0x7ff55406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:11.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.662+0000 7ff5598da640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5540cb030 con 0x7ff5540c9ca0 2023-10-16T10:07:11.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.662+0000 7ff5598da640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5540c91c0 con 0x7ff5540caac0 2023-10-16T10:07:11.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.662+0000 7ff5598da640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540c94a0 con 0x7ff55415ad70 2023-10-16T10:07:11.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.663+0000 7ff5537fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff5540caac0 0x7ff55406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:11.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.663+0000 7ff5537fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff5540caac0 0x7ff55406a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54880/0 (socket says 172.21.15.84:54880) 2023-10-16T10:07:11.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.663+0000 7ff5537fe640 1 -- 172.21.15.84:0/1229584086 learned_addr learned my addr 172.21.15.84:0/1229584086 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:11.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.663+0000 7ff552ffd640 1 --2- 172.21.15.84:0/1229584086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff55415ad70 0x7ff554069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:11.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.663+0000 7ff5537fe640 1 -- 172.21.15.84:0/1229584086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff55415ad70 msgr2=0x7ff554069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:11.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.663+0000 7ff5537fe640 1 --2- 172.21.15.84:0/1229584086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff55415ad70 0x7ff554069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.663+0000 7ff5537fe640 1 -- 172.21.15.84:0/1229584086 >> v1:172.21.15.157:6789/0 conn(0x7ff5540c9ca0 legacy=0x7ff5540ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:11.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.663+0000 7ff5537fe640 1 -- 172.21.15.84:0/1229584086 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5540c9780 con 0x7ff5540caac0 2023-10-16T10:07:11.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.664+0000 7ff5537fe640 1 --2- 172.21.15.84:0/1229584086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5540caac0 0x7ff55406a2e0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009870 tx=0x7ff54c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed89dde836a555c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:11.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.665+0000 7ff551ffb640 1 -- 172.21.15.84:0/1229584086 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c031640 con 0x7ff5540caac0 2023-10-16T10:07:11.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.665+0000 7ff551ffb640 1 -- 172.21.15.84:0/1229584086 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c0317a0 con 0x7ff5540caac0 2023-10-16T10:07:11.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.665+0000 7ff551ffb640 1 -- 172.21.15.84:0/1229584086 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c031b30 con 0x7ff5540caac0 2023-10-16T10:07:11.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.665+0000 7ff5598da640 1 -- 172.21.15.84:0/1229584086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5540caac0 msgr2=0x7ff55406a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:11.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.665+0000 7ff5598da640 1 --2- 172.21.15.84:0/1229584086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5540caac0 0x7ff55406a2e0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009870 tx=0x7ff54c02b8f0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.665+0000 7ff5598da640 1 -- 172.21.15.84:0/1229584086 shutdown_connections 2023-10-16T10:07:11.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.665+0000 7ff5598da640 1 --2- 172.21.15.84:0/1229584086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5540caac0 0x7ff55406a2e0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.665+0000 7ff5598da640 1 --2- 172.21.15.84:0/1229584086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff55415ad70 0x7ff554069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.665+0000 7ff5598da640 1 -- 172.21.15.84:0/1229584086 >> 172.21.15.84:0/1229584086 conn(0x7ff5540ce9b0 msgr2=0x7ff5540cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:11.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.666+0000 7ff5598da640 1 -- 172.21.15.84:0/1229584086 shutdown_connections 2023-10-16T10:07:11.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.666+0000 7ff5598da640 1 -- 172.21.15.84:0/1229584086 wait complete. 2023-10-16T10:07:11.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.667+0000 7ff5598da640 1 Processor -- start 2023-10-16T10:07:11.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.667+0000 7ff5598da640 1 -- start start 2023-10-16T10:07:11.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.668+0000 7ff5598da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5540c9ca0 0x7ff554158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:11.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.668+0000 7ff5598da640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5540caac0 0x7ff5540aca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:11.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.668+0000 7ff5598da640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff55415ad70 0x7ff5540acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:11.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.668+0000 7ff5598da640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540c3c70 con 0x7ff55415ad70 2023-10-16T10:07:11.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.668+0000 7ff5598da640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540cc6b0 con 0x7ff5540c9ca0 2023-10-16T10:07:11.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.668+0000 7ff5598da640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540cc990 con 0x7ff5540caac0 2023-10-16T10:07:11.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.669+0000 7ff5537fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff55415ad70 0x7ff5540acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:11.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.669+0000 7ff5537fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff55415ad70 0x7ff5540acf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54886/0 (socket says 172.21.15.84:54886) 2023-10-16T10:07:11.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.669+0000 7ff5537fe640 1 -- 172.21.15.84:0/2463253953 learned_addr learned my addr 172.21.15.84:0/2463253953 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:11.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.669+0000 7ff5527fc640 1 --2- 172.21.15.84:0/2463253953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5540caac0 0x7ff5540aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:11.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.669+0000 7ff552ffd640 1 --2- 172.21.15.84:0/2463253953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5540c9ca0 0x7ff554158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:11.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.669+0000 7ff5537fe640 1 -- 172.21.15.84:0/2463253953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5540caac0 msgr2=0x7ff5540aca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:11.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.669+0000 7ff5537fe640 1 --2- 172.21.15.84:0/2463253953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5540caac0 0x7ff5540aca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.669+0000 7ff5537fe640 1 -- 172.21.15.84:0/2463253953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5540c9ca0 msgr2=0x7ff554158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:11.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.669+0000 7ff5537fe640 1 --2- 172.21.15.84:0/2463253953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5540c9ca0 0x7ff554158280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.669+0000 7ff5537fe640 1 -- 172.21.15.84:0/2463253953 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c0094d0 con 0x7ff55415ad70 2023-10-16T10:07:11.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.669+0000 7ff5537fe640 1 --2- 172.21.15.84:0/2463253953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff55415ad70 0x7ff5540acf60 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009870 tx=0x7ff54c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:11.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.670+0000 7ff54bfff640 1 -- 172.21.15.84:0/2463253953 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c03ac60 con 0x7ff55415ad70 2023-10-16T10:07:11.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.670+0000 7ff54bfff640 1 -- 172.21.15.84:0/2463253953 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c03ade0 con 0x7ff55415ad70 2023-10-16T10:07:11.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.670+0000 7ff5598da640 1 -- 172.21.15.84:0/2463253953 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5540ccc70 con 0x7ff55415ad70 2023-10-16T10:07:11.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.670+0000 7ff54bfff640 1 -- 172.21.15.84:0/2463253953 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c044630 con 0x7ff55415ad70 2023-10-16T10:07:11.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.670+0000 7ff5598da640 1 -- 172.21.15.84:0/2463253953 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5540cd230 con 0x7ff55415ad70 2023-10-16T10:07:11.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.671+0000 7ff5598da640 1 -- 172.21.15.84:0/2463253953 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5540cd420 con 0x7ff55415ad70 2023-10-16T10:07:11.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.672+0000 7ff54bfff640 1 -- 172.21.15.84:0/2463253953 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 56) v1 ==== 88940+0+0 (secure 0 0 0) 0x7ff54c02f050 con 0x7ff55415ad70 2023-10-16T10:07:11.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.673+0000 7ff54bfff640 1 -- 172.21.15.84:0/2463253953 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6512+0+0 (secure 0 0 0) 0x7ff54c0ab280 con 0x7ff55415ad70 2023-10-16T10:07:11.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.678+0000 7ff54bfff640 1 -- 172.21.15.84:0/2463253953 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff54c076460 con 0x7ff55415ad70 2023-10-16T10:07:11.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.855+0000 7ff5598da640 1 -- 172.21.15.84:0/2463253953 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff55404e160 con 0x7ff55415ad70 2023-10-16T10:07:11.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.856+0000 7ff54bfff640 1 -- 172.21.15.84:0/2463253953 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff54c079a80 con 0x7ff55415ad70 2023-10-16T10:07:11.858 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T10:07:11.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.858+0000 7ff5598da640 1 -- 172.21.15.84:0/2463253953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff55415ad70 msgr2=0x7ff5540acf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:11.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.858+0000 7ff5598da640 1 --2- 172.21.15.84:0/2463253953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff55415ad70 0x7ff5540acf60 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009870 tx=0x7ff54c0046c0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.858+0000 7ff5598da640 1 -- 172.21.15.84:0/2463253953 shutdown_connections 2023-10-16T10:07:11.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.858+0000 7ff5598da640 1 --2- 172.21.15.84:0/2463253953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff55415ad70 0x7ff5540acf60 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.858+0000 7ff5598da640 1 --2- 172.21.15.84:0/2463253953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5540caac0 0x7ff5540aca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.858+0000 7ff5598da640 1 --2- 172.21.15.84:0/2463253953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5540c9ca0 0x7ff554158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:11.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.858+0000 7ff5598da640 1 -- 172.21.15.84:0/2463253953 >> 172.21.15.84:0/2463253953 conn(0x7ff5540ce9b0 msgr2=0x7ff5540ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:11.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.858+0000 7ff5598da640 1 -- 172.21.15.84:0/2463253953 shutdown_connections 2023-10-16T10:07:11.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:11.858+0000 7ff5598da640 1 -- 172.21.15.84:0/2463253953 wait complete. 2023-10-16T10:07:11.877 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:07:12.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.134+0000 7f7b086da640 1 Processor -- start 2023-10-16T10:07:12.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.135+0000 7f7b086da640 1 -- start start 2023-10-16T10:07:12.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.136+0000 7f7b086da640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7b000b3420 0x7f7b000b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:12.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.136+0000 7f7b086da640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b000cfaf0 0x7f7b000b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:12.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.136+0000 7f7b086da640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b000d3980 con 0x7f7b000cf710 2023-10-16T10:07:12.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.136+0000 7f7b086da640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b000d3c60 con 0x7f7b000b3420 2023-10-16T10:07:12.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.136+0000 7f7b086da640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b000ce630 con 0x7f7b000cfaf0 2023-10-16T10:07:12.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.137+0000 7f7b06c77640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b000cfaf0 0x7f7b000b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:12.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.137+0000 7f7b06476640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7b000cf710 0x7f7b000b2d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:44546/0 (socket says 172.21.15.84:44546) 2023-10-16T10:07:12.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.137+0000 7f7b06476640 1 -- 172.21.15.84:0/1274212116 learned_addr learned my addr 172.21.15.84:0/1274212116 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:12.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.137+0000 7f7b05c75640 1 --2- 172.21.15.84:0/1274212116 >> v2:172.21.15.157:3300/0 conn(0x7f7b000b3420 0x7f7b000b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:12.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.138+0000 7f7b05c75640 1 -- 172.21.15.84:0/1274212116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b000cfaf0 msgr2=0x7f7b000b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:12.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.138+0000 7f7b05c75640 1 --2- 172.21.15.84:0/1274212116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b000cfaf0 0x7f7b000b6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.138+0000 7f7b05c75640 1 -- 172.21.15.84:0/1274212116 >> v1:172.21.15.157:6789/0 conn(0x7f7b000cf710 legacy=0x7f7b000b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:12.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.138+0000 7f7b05c75640 1 -- 172.21.15.84:0/1274212116 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b000ce910 con 0x7f7b000b3420 2023-10-16T10:07:12.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.139+0000 7f7b05c75640 1 --2- 172.21.15.84:0/1274212116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b000b3420 0x7f7b000b5cc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7af40097a0 tx=0x7f7af402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1a403da27645e02 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:12.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.140+0000 7f7b05474640 1 -- 172.21.15.84:0/1274212116 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4031780 con 0x7f7b000b3420 2023-10-16T10:07:12.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.140+0000 7f7b05474640 1 -- 172.21.15.84:0/1274212116 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af40318e0 con 0x7f7b000b3420 2023-10-16T10:07:12.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.140+0000 7f7b05474640 1 -- 172.21.15.84:0/1274212116 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4031c70 con 0x7f7b000b3420 2023-10-16T10:07:12.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.140+0000 7f7b086da640 1 -- 172.21.15.84:0/1274212116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b000b3420 msgr2=0x7f7b000b5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:12.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.140+0000 7f7b086da640 1 --2- 172.21.15.84:0/1274212116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b000b3420 0x7f7b000b5cc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7af40097a0 tx=0x7f7af402b820 comp rx=0 tx=0).stop 2023-10-16T10:07:12.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.141+0000 7f7b086da640 1 -- 172.21.15.84:0/1274212116 shutdown_connections 2023-10-16T10:07:12.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.141+0000 7f7b086da640 1 --2- 172.21.15.84:0/1274212116 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b000cfaf0 0x7f7b000b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.141+0000 7f7b086da640 1 --2- 172.21.15.84:0/1274212116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b000b3420 0x7f7b000b5cc0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.141+0000 7f7b086da640 1 -- 172.21.15.84:0/1274212116 >> 172.21.15.84:0/1274212116 conn(0x7f7b000d2cc0 msgr2=0x7f7b000d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:12.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.141+0000 7f7b086da640 1 -- 172.21.15.84:0/1274212116 shutdown_connections 2023-10-16T10:07:12.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.141+0000 7f7b086da640 1 -- 172.21.15.84:0/1274212116 wait complete. 2023-10-16T10:07:12.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.143+0000 7f7b086da640 1 Processor -- start 2023-10-16T10:07:12.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.143+0000 7f7b086da640 1 -- start start 2023-10-16T10:07:12.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.144+0000 7f7b086da640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b000b3420 0x7f7b000a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:12.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.144+0000 7f7b086da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b000cf710 0x7f7b0015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:12.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.144+0000 7f7b086da640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b000cfaf0 0x7f7b0009d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:12.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.144+0000 7f7b086da640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b000a0ab0 con 0x7f7b000cfaf0 2023-10-16T10:07:12.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.144+0000 7f7b086da640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0009d450 con 0x7f7b000cf710 2023-10-16T10:07:12.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.144+0000 7f7b086da640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0009cb20 con 0x7f7b000b3420 2023-10-16T10:07:12.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.145+0000 7f7b06c77640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b000cfaf0 0x7f7b0009d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:12.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.145+0000 7f7b06c77640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b000cfaf0 0x7f7b0009d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54890/0 (socket says 172.21.15.84:54890) 2023-10-16T10:07:12.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.145+0000 7f7b06c77640 1 -- 172.21.15.84:0/355962174 learned_addr learned my addr 172.21.15.84:0/355962174 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:12.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.145+0000 7f7b06476640 1 --2- 172.21.15.84:0/355962174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b000b3420 0x7f7b000a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:12.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.145+0000 7f7b05c75640 1 --2- 172.21.15.84:0/355962174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b000cf710 0x7f7b0015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:12.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.145+0000 7f7b06c77640 1 -- 172.21.15.84:0/355962174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b000b3420 msgr2=0x7f7b000a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:12.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.145+0000 7f7b06c77640 1 --2- 172.21.15.84:0/355962174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b000b3420 0x7f7b000a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.145+0000 7f7b06c77640 1 -- 172.21.15.84:0/355962174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b000cf710 msgr2=0x7f7b0015e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:12.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.145+0000 7f7b06c77640 1 --2- 172.21.15.84:0/355962174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b000cf710 0x7f7b0015e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.145+0000 7f7b06c77640 1 -- 172.21.15.84:0/355962174 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af4009400 con 0x7f7b000cfaf0 2023-10-16T10:07:12.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.145+0000 7f7b06c77640 1 --2- 172.21.15.84:0/355962174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b000cfaf0 0x7f7b0009d8e0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f7af800e930 tx=0x7f7af800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:12.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.146+0000 7f7afeffd640 1 -- 172.21.15.84:0/355962174 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af8009690 con 0x7f7b000cfaf0 2023-10-16T10:07:12.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.146+0000 7f7afeffd640 1 -- 172.21.15.84:0/355962174 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af800c980 con 0x7f7b000cfaf0 2023-10-16T10:07:12.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.146+0000 7f7b086da640 1 -- 172.21.15.84:0/355962174 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0009ce00 con 0x7f7b000cfaf0 2023-10-16T10:07:12.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.146+0000 7f7afeffd640 1 -- 172.21.15.84:0/355962174 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af800cd30 con 0x7f7b000cfaf0 2023-10-16T10:07:12.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.146+0000 7f7b086da640 1 -- 172.21.15.84:0/355962174 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0015ed40 con 0x7f7b000cfaf0 2023-10-16T10:07:12.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.147+0000 7f7b086da640 1 -- 172.21.15.84:0/355962174 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0004e160 con 0x7f7b000cfaf0 2023-10-16T10:07:12.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.148+0000 7f7afeffd640 1 -- 172.21.15.84:0/355962174 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 56) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f7af8013470 con 0x7f7b000cfaf0 2023-10-16T10:07:12.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.149+0000 7f7afeffd640 1 -- 172.21.15.84:0/355962174 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6512+0+0 (secure 0 0 0) 0x7f7af80896e0 con 0x7f7b000cfaf0 2023-10-16T10:07:12.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.154+0000 7f7afeffd640 1 -- 172.21.15.84:0/355962174 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7af80548b0 con 0x7f7b000cfaf0 2023-10-16T10:07:12.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.345+0000 7f7b086da640 1 -- 172.21.15.84:0/355962174 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b000ce2d0 con 0x7f7b000cfaf0 2023-10-16T10:07:12.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.345+0000 7f7afeffd640 1 -- 172.21.15.84:0/355962174 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7af8057ed0 con 0x7f7b000cfaf0 2023-10-16T10:07:12.347 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T10:07:12.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.348+0000 7f7b086da640 1 -- 172.21.15.84:0/355962174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b000cfaf0 msgr2=0x7f7b0009d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:12.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.348+0000 7f7b086da640 1 --2- 172.21.15.84:0/355962174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b000cfaf0 0x7f7b0009d8e0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f7af800e930 tx=0x7f7af800edf0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.348+0000 7f7b086da640 1 -- 172.21.15.84:0/355962174 shutdown_connections 2023-10-16T10:07:12.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.348+0000 7f7b086da640 1 --2- 172.21.15.84:0/355962174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b000cfaf0 0x7f7b0009d8e0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.348+0000 7f7b086da640 1 --2- 172.21.15.84:0/355962174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b000cf710 0x7f7b0015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.348+0000 7f7b086da640 1 --2- 172.21.15.84:0/355962174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b000b3420 0x7f7b000a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.348+0000 7f7b086da640 1 -- 172.21.15.84:0/355962174 >> 172.21.15.84:0/355962174 conn(0x7f7b000d2cc0 msgr2=0x7f7b000b5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:12.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.348+0000 7f7b086da640 1 -- 172.21.15.84:0/355962174 shutdown_connections 2023-10-16T10:07:12.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.348+0000 7f7b086da640 1 -- 172.21.15.84:0/355962174 wait complete. 2023-10-16T10:07:12.370 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:07:12.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.586+0000 7f7271ca2640 1 Processor -- start 2023-10-16T10:07:12.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.586+0000 7f7271ca2640 1 -- start start 2023-10-16T10:07:12.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.586+0000 7f7271ca2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f726c0caa10 0x7f726c06fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:12.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.587+0000 7f7271ca2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f726c0cc920 0x7f726c0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:12.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.587+0000 7f7271ca2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f726c159790 con 0x7f726c1593b0 2023-10-16T10:07:12.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.587+0000 7f7271ca2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726c0c9b20 con 0x7f726c0caa10 2023-10-16T10:07:12.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.587+0000 7f7271ca2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726c0c9e00 con 0x7f726c0cc920 2023-10-16T10:07:12.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.587+0000 7f7263fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f726c0caa10 0x7f726c06fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:12.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.587+0000 7f7263fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f726c0caa10 0x7f726c06fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54902/0 (socket says 172.21.15.84:54902) 2023-10-16T10:07:12.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.587+0000 7f7263fff640 1 -- 172.21.15.84:0/4011352169 learned_addr learned my addr 172.21.15.84:0/4011352169 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:12.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.587+0000 7f7263fff640 1 -- 172.21.15.84:0/4011352169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f726c0cc920 msgr2=0x7f726c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:12.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.587+0000 7f7263fff640 1 --2- 172.21.15.84:0/4011352169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f726c0cc920 0x7f726c0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.587+0000 7f7263fff640 1 -- 172.21.15.84:0/4011352169 >> v1:172.21.15.157:6789/0 conn(0x7f726c1593b0 legacy=0x7f726c06f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:12.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.587+0000 7f7263fff640 1 -- 172.21.15.84:0/4011352169 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726c0ca0e0 con 0x7f726c0caa10 2023-10-16T10:07:12.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.588+0000 7f7263fff640 1 --2- 172.21.15.84:0/4011352169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f726c0caa10 0x7f726c06fc60 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f7254009870 tx=0x7f725402b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78926ef0491ae6f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:12.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.589+0000 7f726affd640 1 -- 172.21.15.84:0/4011352169 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254033620 con 0x7f726c0caa10 2023-10-16T10:07:12.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.589+0000 7f726affd640 1 -- 172.21.15.84:0/4011352169 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7254033780 con 0x7f726c0caa10 2023-10-16T10:07:12.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.589+0000 7f726affd640 1 -- 172.21.15.84:0/4011352169 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254033b10 con 0x7f726c0caa10 2023-10-16T10:07:12.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.589+0000 7f7271ca2640 1 -- 172.21.15.84:0/4011352169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f726c0caa10 msgr2=0x7f726c06fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:12.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.589+0000 7f7271ca2640 1 --2- 172.21.15.84:0/4011352169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f726c0caa10 0x7f726c06fc60 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f7254009870 tx=0x7f725402b7d0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.589+0000 7f7271ca2640 1 -- 172.21.15.84:0/4011352169 shutdown_connections 2023-10-16T10:07:12.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.589+0000 7f7271ca2640 1 --2- 172.21.15.84:0/4011352169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f726c0cc920 0x7f726c0ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.590+0000 7f7271ca2640 1 --2- 172.21.15.84:0/4011352169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f726c0caa10 0x7f726c06fc60 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.590+0000 7f7271ca2640 1 -- 172.21.15.84:0/4011352169 >> 172.21.15.84:0/4011352169 conn(0x7f726c0cb410 msgr2=0x7f726c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:12.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.590+0000 7f7271ca2640 1 -- 172.21.15.84:0/4011352169 shutdown_connections 2023-10-16T10:07:12.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.590+0000 7f7271ca2640 1 -- 172.21.15.84:0/4011352169 wait complete. 2023-10-16T10:07:12.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.592+0000 7f7271ca2640 1 Processor -- start 2023-10-16T10:07:12.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.592+0000 7f7271ca2640 1 -- start start 2023-10-16T10:07:12.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.593+0000 7f7271ca2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f726c0caa10 0x7f726c1587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:12.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.593+0000 7f7271ca2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f726c0cc920 0x7f726c15c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:12.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.593+0000 7f7271ca2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f726c1593b0 0x7f726c1579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:12.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.593+0000 7f7271ca2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726c13fc80 con 0x7f726c0caa10 2023-10-16T10:07:12.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.593+0000 7f7271ca2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726c13ff60 con 0x7f726c0cc920 2023-10-16T10:07:12.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.593+0000 7f7271ca2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726c140240 con 0x7f726c1593b0 2023-10-16T10:07:12.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.594+0000 7f726b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f726c0caa10 0x7f726c1587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:12.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.594+0000 7f726b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f726c0caa10 0x7f726c1587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54906/0 (socket says 172.21.15.84:54906) 2023-10-16T10:07:12.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.594+0000 7f726b7fe640 1 -- 172.21.15.84:0/1730341666 learned_addr learned my addr 172.21.15.84:0/1730341666 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:12.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.594+0000 7f726b7fe640 1 -- 172.21.15.84:0/1730341666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f726c1593b0 msgr2=0x7f726c1579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:12.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.594+0000 7f726b7fe640 1 --2- 172.21.15.84:0/1730341666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f726c1593b0 0x7f726c1579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.594+0000 7f726b7fe640 1 -- 172.21.15.84:0/1730341666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f726c0cc920 msgr2=0x7f726c15c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:12.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.594+0000 7f726b7fe640 1 --2- 172.21.15.84:0/1730341666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f726c0cc920 0x7f726c15c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.594+0000 7f726b7fe640 1 -- 172.21.15.84:0/1730341666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72540094d0 con 0x7f726c0caa10 2023-10-16T10:07:12.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.594+0000 7f726b7fe640 1 --2- 172.21.15.84:0/1730341666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f726c0caa10 0x7f726c1587a0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f725c00c410 tx=0x7f725c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:12.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.594+0000 7f7268ff9640 1 -- 172.21.15.84:0/1730341666 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c0035f0 con 0x7f726c0caa10 2023-10-16T10:07:12.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.594+0000 7f7268ff9640 1 -- 172.21.15.84:0/1730341666 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c003750 con 0x7f726c0caa10 2023-10-16T10:07:12.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.594+0000 7f7268ff9640 1 -- 172.21.15.84:0/1730341666 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c003b40 con 0x7f726c0caa10 2023-10-16T10:07:12.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.594+0000 7f7271ca2640 1 -- 172.21.15.84:0/1730341666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726c140520 con 0x7f726c0caa10 2023-10-16T10:07:12.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.595+0000 7f7271ca2640 1 -- 172.21.15.84:0/1730341666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f726c15c600 con 0x7f726c0caa10 2023-10-16T10:07:12.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.596+0000 7f7271ca2640 1 -- 172.21.15.84:0/1730341666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f726c13d880 con 0x7f726c0caa10 2023-10-16T10:07:12.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.597+0000 7f7268ff9640 1 -- 172.21.15.84:0/1730341666 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 57) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f725c003ca0 con 0x7f726c0caa10 2023-10-16T10:07:12.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.598+0000 7f7268ff9640 1 --2- 172.21.15.84:0/1730341666 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f724c06acc0 0x7f724c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:12.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.598+0000 7f7268ff9640 1 -- 172.21.15.84:0/1730341666 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6512+0+0 (secure 0 0 0) 0x7f725c088860 con 0x7f726c0caa10 2023-10-16T10:07:12.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.598+0000 7f7263fff640 1 --2- 172.21.15.84:0/1730341666 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f724c06acc0 0x7f724c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:12.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.599+0000 7f7263fff640 1 --2- 172.21.15.84:0/1730341666 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f724c06acc0 0x7f724c06d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7254035000 tx=0x7f7254030000 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:12.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.602+0000 7f7268ff9640 1 -- 172.21.15.84:0/1730341666 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f726c13d880 con 0x7f726c0caa10 2023-10-16T10:07:12.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.836+0000 7f7271ca2640 1 -- 172.21.15.84:0/1730341666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f726c13d880 con 0x7f726c0caa10 2023-10-16T10:07:12.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.837+0000 7f7268ff9640 1 -- 172.21.15.84:0/1730341666 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f726c13d880 con 0x7f726c0caa10 2023-10-16T10:07:12.839 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:12.839 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3458903351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3458903351},{"type":"v1","addr":"172.21.15.84:6837","nonce":3458903351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4002935522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4002935522},{"type":"v1","addr":"172.21.15.157:6833","nonce":4002935522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5809,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1890551305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1890551305},{"type":"v1","addr":"172.21.15.157:6835","nonce":1890551305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:05:48.809508+0000","modified":"2023-10-16T10:05:51.856862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7521":{"gid":7521,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/3621445269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3621445269},{"type":"v1","addr":"172.21.15.84:6836","nonce":3621445269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-16T10:07:12.839 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 53 2023-10-16T10:07:12.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.839+0000 7f7271ca2640 1 -- 172.21.15.84:0/1730341666 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f724c06acc0 msgr2=0x7f724c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:12.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.840+0000 7f7271ca2640 1 --2- 172.21.15.84:0/1730341666 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f724c06acc0 0x7f724c06d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7254035000 tx=0x7f7254030000 comp rx=0 tx=0).stop 2023-10-16T10:07:12.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.840+0000 7f7271ca2640 1 -- 172.21.15.84:0/1730341666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f726c0caa10 msgr2=0x7f726c1587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:12.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.840+0000 7f7271ca2640 1 --2- 172.21.15.84:0/1730341666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f726c0caa10 0x7f726c1587a0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f725c00c410 tx=0x7f725c00c8d0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.840+0000 7f7271ca2640 1 -- 172.21.15.84:0/1730341666 shutdown_connections 2023-10-16T10:07:12.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.840+0000 7f7271ca2640 1 --2- 172.21.15.84:0/1730341666 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f724c06acc0 0x7f724c06d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.840+0000 7f7271ca2640 1 --2- 172.21.15.84:0/1730341666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f726c1593b0 0x7f726c1579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.840+0000 7f7271ca2640 1 --2- 172.21.15.84:0/1730341666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f726c0cc920 0x7f726c15c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.840+0000 7f7271ca2640 1 --2- 172.21.15.84:0/1730341666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f726c0caa10 0x7f726c1587a0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:12.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.840+0000 7f7271ca2640 1 -- 172.21.15.84:0/1730341666 >> 172.21.15.84:0/1730341666 conn(0x7f726c0cb410 msgr2=0x7f726c0ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:12.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.840+0000 7f7271ca2640 1 -- 172.21.15.84:0/1730341666 shutdown_connections 2023-10-16T10:07:12.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:12.840+0000 7f7271ca2640 1 -- 172.21.15.84:0/1730341666 wait complete. 2023-10-16T10:07:12.859 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:07:13.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.063+0000 7f4dce73a640 1 Processor -- start 2023-10-16T10:07:13.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.063+0000 7f4dce73a640 1 -- start start 2023-10-16T10:07:13.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.064+0000 7f4dce73a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dc815aaa0 0x7f4dc815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:13.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.064+0000 7f4dce73a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4dc815ae80 0x7f4dc80cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:13.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.064+0000 7f4dce73a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4dc80ccaa0 0x7f4dc8070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:13.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.064+0000 7f4dce73a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc806b810 con 0x7f4dc80ccaa0 2023-10-16T10:07:13.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.064+0000 7f4dce73a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc815ed60 con 0x7f4dc815ae80 2023-10-16T10:07:13.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.064+0000 7f4dce73a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc80707e0 con 0x7f4dc815aaa0 2023-10-16T10:07:13.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.065+0000 7f4dcccd7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4dc80ccaa0 0x7f4dc8070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:13.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.065+0000 7f4dcccd7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4dc80ccaa0 0x7f4dc8070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54910/0 (socket says 172.21.15.84:54910) 2023-10-16T10:07:13.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.065+0000 7f4dcccd7640 1 -- 172.21.15.84:0/3470097100 learned_addr learned my addr 172.21.15.84:0/3470097100 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:13.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.065+0000 7f4dcccd7640 1 -- 172.21.15.84:0/3470097100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dc815aaa0 msgr2=0x7f4dc815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:13.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.065+0000 7f4dc7fff640 1 --2- 172.21.15.84:0/3470097100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dc815aaa0 0x7f4dc815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:13.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.065+0000 7f4dbffff640 1 --2- 172.21.15.84:0/3470097100 >> v2:172.21.15.157:3300/0 conn(0x7f4dc815ae80 0x7f4dc80cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:13.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.065+0000 7f4dcccd7640 1 --2- 172.21.15.84:0/3470097100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dc815aaa0 0x7f4dc815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.065+0000 7f4dcccd7640 1 -- 172.21.15.84:0/3470097100 >> v2:172.21.15.157:3300/0 conn(0x7f4dc815ae80 msgr2=0x7f4dc80cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:13.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.065+0000 7f4dcccd7640 1 --2- 172.21.15.84:0/3470097100 >> v2:172.21.15.157:3300/0 conn(0x7f4dc815ae80 0x7f4dc80cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.065+0000 7f4dcccd7640 1 -- 172.21.15.84:0/3470097100 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc815ef90 con 0x7f4dc80ccaa0 2023-10-16T10:07:13.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.065+0000 7f4dc7fff640 1 --2- 172.21.15.84:0/3470097100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dc815aaa0 0x7f4dc815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:13.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.065+0000 7f4dbffff640 1 --2- 172.21.15.84:0/3470097100 >> v2:172.21.15.157:3300/0 conn(0x7f4dc815ae80 0x7f4dc80cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:13.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.066+0000 7f4dcccd7640 1 --2- 172.21.15.84:0/3470097100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dc80ccaa0 0x7f4dc8070f50 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80cb270 tx=0x7f4dc002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd8bc931d29733d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:13.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.067+0000 7f4dc77fe640 1 -- 172.21.15.84:0/3470097100 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc002ca80 con 0x7f4dc80ccaa0 2023-10-16T10:07:13.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.067+0000 7f4dc77fe640 1 -- 172.21.15.84:0/3470097100 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc002cbe0 con 0x7f4dc80ccaa0 2023-10-16T10:07:13.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.067+0000 7f4dc77fe640 1 -- 172.21.15.84:0/3470097100 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc00343f0 con 0x7f4dc80ccaa0 2023-10-16T10:07:13.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.067+0000 7f4dce73a640 1 -- 172.21.15.84:0/3470097100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dc80ccaa0 msgr2=0x7f4dc8070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:13.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.067+0000 7f4dce73a640 1 --2- 172.21.15.84:0/3470097100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dc80ccaa0 0x7f4dc8070f50 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80cb270 tx=0x7f4dc002bee0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.067+0000 7f4dce73a640 1 -- 172.21.15.84:0/3470097100 shutdown_connections 2023-10-16T10:07:13.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.067+0000 7f4dce73a640 1 --2- 172.21.15.84:0/3470097100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dc80ccaa0 0x7f4dc8070f50 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.067+0000 7f4dce73a640 1 --2- 172.21.15.84:0/3470097100 >> v2:172.21.15.157:3300/0 conn(0x7f4dc815ae80 0x7f4dc80cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.067+0000 7f4dce73a640 1 --2- 172.21.15.84:0/3470097100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dc815aaa0 0x7f4dc815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.067+0000 7f4dce73a640 1 -- 172.21.15.84:0/3470097100 >> 172.21.15.84:0/3470097100 conn(0x7f4dc80cb410 msgr2=0x7f4dc80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:13.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.067+0000 7f4dce73a640 1 -- 172.21.15.84:0/3470097100 shutdown_connections 2023-10-16T10:07:13.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.068+0000 7f4dce73a640 1 -- 172.21.15.84:0/3470097100 wait complete. 2023-10-16T10:07:13.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.069+0000 7f4dce73a640 1 Processor -- start 2023-10-16T10:07:13.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.069+0000 7f4dce73a640 1 -- start start 2023-10-16T10:07:13.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.070+0000 7f4dce73a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc80ccaa0 0x7f4dc8158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:13.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.070+0000 7f4dce73a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dc815aaa0 0x7f4dc8070da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:13.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.070+0000 7f4dce73a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dc815ae80 0x7f4dc80712e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:13.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.070+0000 7f4dce73a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc815d380 con 0x7f4dc815ae80 2023-10-16T10:07:13.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.070+0000 7f4dce73a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc815d660 con 0x7f4dc80ccaa0 2023-10-16T10:07:13.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.070+0000 7f4dce73a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc815d940 con 0x7f4dc815aaa0 2023-10-16T10:07:13.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.071+0000 7f4dcccd7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dc815ae80 0x7f4dc80712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:13.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.071+0000 7f4dcccd7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dc815ae80 0x7f4dc80712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54926/0 (socket says 172.21.15.84:54926) 2023-10-16T10:07:13.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.071+0000 7f4dcccd7640 1 -- 172.21.15.84:0/692298252 learned_addr learned my addr 172.21.15.84:0/692298252 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:13.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.071+0000 7f4dc7fff640 1 --2- 172.21.15.84:0/692298252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc80ccaa0 0x7f4dc8158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:13.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.071+0000 7f4dbffff640 1 --2- 172.21.15.84:0/692298252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dc815aaa0 0x7f4dc8070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:13.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.071+0000 7f4dcccd7640 1 -- 172.21.15.84:0/692298252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dc815aaa0 msgr2=0x7f4dc8070da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:13.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.071+0000 7f4dcccd7640 1 --2- 172.21.15.84:0/692298252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dc815aaa0 0x7f4dc8070da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.071+0000 7f4dcccd7640 1 -- 172.21.15.84:0/692298252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc80ccaa0 msgr2=0x7f4dc8158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:13.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.071+0000 7f4dcccd7640 1 --2- 172.21.15.84:0/692298252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc80ccaa0 0x7f4dc8158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.071+0000 7f4dcccd7640 1 -- 172.21.15.84:0/692298252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc0009b30 con 0x7f4dc815ae80 2023-10-16T10:07:13.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.072+0000 7f4dcccd7640 1 --2- 172.21.15.84:0/692298252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dc815ae80 0x7f4dc80712e0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0036040 tx=0x7f4dc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:13.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.072+0000 7f4dc57fa640 1 -- 172.21.15.84:0/692298252 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0009510 con 0x7f4dc815ae80 2023-10-16T10:07:13.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.072+0000 7f4dc57fa640 1 -- 172.21.15.84:0/692298252 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc003e450 con 0x7f4dc815ae80 2023-10-16T10:07:13.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.072+0000 7f4dc57fa640 1 -- 172.21.15.84:0/692298252 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc003e800 con 0x7f4dc815ae80 2023-10-16T10:07:13.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.072+0000 7f4dce73a640 1 -- 172.21.15.84:0/692298252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc815dc20 con 0x7f4dc815ae80 2023-10-16T10:07:13.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.072+0000 7f4dce73a640 1 -- 172.21.15.84:0/692298252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc80c43a0 con 0x7f4dc815ae80 2023-10-16T10:07:13.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.073+0000 7f4dce73a640 1 -- 172.21.15.84:0/692298252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc804e160 con 0x7f4dc815ae80 2023-10-16T10:07:13.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.074+0000 7f4dc57fa640 1 -- 172.21.15.84:0/692298252 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 57) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f4dc002cd50 con 0x7f4dc815ae80 2023-10-16T10:07:13.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.075+0000 7f4dc57fa640 1 --2- 172.21.15.84:0/692298252 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f4da806acc0 0x7f4da806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:13.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.075+0000 7f4dc57fa640 1 -- 172.21.15.84:0/692298252 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6512+0+0 (secure 0 0 0) 0x7f4dc00ac150 con 0x7f4dc815ae80 2023-10-16T10:07:13.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.075+0000 7f4dc7fff640 1 --2- 172.21.15.84:0/692298252 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f4da806acc0 0x7f4da806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:13.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.076+0000 7f4dc7fff640 1 --2- 172.21.15.84:0/692298252 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f4da806acc0 0x7f4da806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4db800a980 tx=0x7f4db8008040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:13.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.080+0000 7f4dc57fa640 1 -- 172.21.15.84:0/692298252 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4dc00773f0 con 0x7f4dc815ae80 2023-10-16T10:07:13.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.271+0000 7f4dce73a640 1 -- 172.21.15.84:0/692298252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4dc80c24c0 con 0x7f4dc815ae80 2023-10-16T10:07:13.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.273+0000 7f4dc57fa640 1 -- 172.21.15.84:0/692298252 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+17718 (secure 0 0 0) 0x7f4dc007aa10 con 0x7f4dc815ae80 2023-10-16T10:07:13.275 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:13.275 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":113,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:07:11.412966+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:05:46.595888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-16T10:05:47.826019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.d","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2143212776":"2023-10-17T10:07:11.412892+0000","172.21.15.84:6833/985901542":"2023-10-17T10:07:11.412892+0000","172.21.15.84:6832/985901542":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/636650964":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/2411644314":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/2627061388":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/1139885639":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/1946898486":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/2051153636":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/628187054":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/965715898":"2023-10-17T10:06:03.602359+0000","172.21.15.84:6833/3250823028":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/3505558848":"2023-10-17T10:07:11.412892+0000","172.21.15.84:6832/3250823028":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/3381824109":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/3871500870":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/1287074955":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/3830790285":"2023-10-16T11:05:57.166133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:07:13.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.275+0000 7f4dce73a640 1 -- 172.21.15.84:0/692298252 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f4da806acc0 msgr2=0x7f4da806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:13.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.275+0000 7f4dce73a640 1 --2- 172.21.15.84:0/692298252 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f4da806acc0 0x7f4da806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4db800a980 tx=0x7f4db8008040 comp rx=0 tx=0).stop 2023-10-16T10:07:13.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.275+0000 7f4dce73a640 1 -- 172.21.15.84:0/692298252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dc815ae80 msgr2=0x7f4dc80712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:13.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.275+0000 7f4dce73a640 1 --2- 172.21.15.84:0/692298252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dc815ae80 0x7f4dc80712e0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0036040 tx=0x7f4dc0004a40 comp rx=0 tx=0).stop 2023-10-16T10:07:13.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.275+0000 7f4dce73a640 1 -- 172.21.15.84:0/692298252 shutdown_connections 2023-10-16T10:07:13.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.275+0000 7f4dce73a640 1 --2- 172.21.15.84:0/692298252 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f4da806acc0 0x7f4da806d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.275+0000 7f4dce73a640 1 --2- 172.21.15.84:0/692298252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dc815ae80 0x7f4dc80712e0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.275+0000 7f4dce73a640 1 --2- 172.21.15.84:0/692298252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dc815aaa0 0x7f4dc8070da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.275+0000 7f4dce73a640 1 --2- 172.21.15.84:0/692298252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc80ccaa0 0x7f4dc8158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.275+0000 7f4dce73a640 1 -- 172.21.15.84:0/692298252 >> 172.21.15.84:0/692298252 conn(0x7f4dc80cb410 msgr2=0x7f4dc80b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:13.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.276+0000 7f4dce73a640 1 -- 172.21.15.84:0/692298252 shutdown_connections 2023-10-16T10:07:13.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.276+0000 7f4dce73a640 1 -- 172.21.15.84:0/692298252 wait complete. 2023-10-16T10:07:13.296 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T10:07:13.296 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:07:13.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.514+0000 7f45106e0640 1 Processor -- start 2023-10-16T10:07:13.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.514+0000 7f45106e0640 1 -- start start 2023-10-16T10:07:13.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.516+0000 7f45106e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450815ae80 0x7f450815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:13.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.516+0000 7f45106e0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f45080ce390 0x7f4508070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:13.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.516+0000 7f45106e0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f450815ed60 con 0x7f450815aaa0 2023-10-16T10:07:13.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.516+0000 7f45106e0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f450806b810 con 0x7f45080ce390 2023-10-16T10:07:13.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.516+0000 7f45106e0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45080707e0 con 0x7f450815ae80 2023-10-16T10:07:13.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.516+0000 7f450e47c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f450815aaa0 0x7f45080cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55000/0 (socket says 172.21.15.84:55000) 2023-10-16T10:07:13.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.516+0000 7f450e47c640 1 -- 172.21.15.84:0/797802121 learned_addr learned my addr 172.21.15.84:0/797802121 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:13.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.516+0000 7f450dc7b640 1 --2- 172.21.15.84:0/797802121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450815ae80 0x7f450815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:13.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.516+0000 7f450ec7d640 1 --2- 172.21.15.84:0/797802121 >> v2:172.21.15.157:3300/0 conn(0x7f45080ce390 0x7f4508070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:13.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.516+0000 7f450d47a640 1 -- 172.21.15.84:0/797802121 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3817171653 0 0) 0x7f450815ed60 con 0x7f450815aaa0 2023-10-16T10:07:13.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.517+0000 7f450d47a640 1 -- 172.21.15.84:0/797802121 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44f4003520 con 0x7f450815aaa0 2023-10-16T10:07:13.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.517+0000 7f450d47a640 1 -- 172.21.15.84:0/797802121 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2064004608 0 0) 0x7f44f4003520 con 0x7f450815aaa0 2023-10-16T10:07:13.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.517+0000 7f450d47a640 1 -- 172.21.15.84:0/797802121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450815ae80 msgr2=0x7f450815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:13.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.517+0000 7f450d47a640 1 --2- 172.21.15.84:0/797802121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450815ae80 0x7f450815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.517+0000 7f450d47a640 1 -- 172.21.15.84:0/797802121 >> v2:172.21.15.157:3300/0 conn(0x7f45080ce390 msgr2=0x7f4508070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:13.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.517+0000 7f450d47a640 1 --2- 172.21.15.84:0/797802121 >> v2:172.21.15.157:3300/0 conn(0x7f45080ce390 0x7f4508070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.517+0000 7f450d47a640 1 -- 172.21.15.84:0/797802121 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450815ef90 con 0x7f450815aaa0 2023-10-16T10:07:13.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.517+0000 7f450d47a640 1 -- 172.21.15.84:0/797802121 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f4504001670 con 0x7f450815aaa0 2023-10-16T10:07:13.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.517+0000 7f450d47a640 1 -- 172.21.15.84:0/797802121 >> v1:172.21.15.84:6789/0 conn(0x7f450815aaa0 legacy=0x7f45080cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:13.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.517+0000 7f450d47a640 1 --2- 172.21.15.84:0/797802121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450815ae80 0x7f44f4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:13.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.517+0000 7f450d47a640 1 --2- 172.21.15.84:0/797802121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f4003ef0 0x7f44f40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:13.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.517+0000 7f450d47a640 1 --2- 172.21.15.84:0/797802121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44f4004d00 0x7f44f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:13.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.517+0000 7f450d47a640 1 -- 172.21.15.84:0/797802121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450806b810 con 0x7f44f4004d00 2023-10-16T10:07:13.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.517+0000 7f450d47a640 1 -- 172.21.15.84:0/797802121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45080707e0 con 0x7f44f4003ef0 2023-10-16T10:07:13.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.517+0000 7f450d47a640 1 -- 172.21.15.84:0/797802121 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f400b980 con 0x7f450815ae80 2023-10-16T10:07:13.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.517+0000 7f450ec7d640 1 --2- 172.21.15.84:0/797802121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44f4004d00 0x7f44f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:13.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.518+0000 7f450dc7b640 1 --2- 172.21.15.84:0/797802121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f4003ef0 0x7f44f40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:13.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.518+0000 7f450ec7d640 1 -- 172.21.15.84:0/797802121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450815ae80 msgr2=0x7f44f4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:13.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.518+0000 7f450ec7d640 1 --2- 172.21.15.84:0/797802121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450815ae80 0x7f44f4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.518+0000 7f450ec7d640 1 -- 172.21.15.84:0/797802121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f4003ef0 msgr2=0x7f44f40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:13.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.518+0000 7f450ec7d640 1 --2- 172.21.15.84:0/797802121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f4003ef0 0x7f44f40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.518+0000 7f450ec7d640 1 -- 172.21.15.84:0/797802121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f4003520 con 0x7f44f4004d00 2023-10-16T10:07:13.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.519+0000 7f450ec7d640 1 --2- 172.21.15.84:0/797802121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44f4004d00 0x7f44f4005180 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4500009fc0 tx=0x7f44f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9038416e79f5254 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:13.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.519+0000 7f450d47a640 1 -- 172.21.15.84:0/797802121 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450002cc40 con 0x7f44f4004d00 2023-10-16T10:07:13.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.519+0000 7f450d47a640 1 -- 172.21.15.84:0/797802121 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450000f040 con 0x7f44f4004d00 2023-10-16T10:07:13.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.519+0000 7f450d47a640 1 -- 172.21.15.84:0/797802121 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500034440 con 0x7f44f4004d00 2023-10-16T10:07:13.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.520+0000 7f45106e0640 1 -- 172.21.15.84:0/797802121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44f4004d00 msgr2=0x7f44f4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:13.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.520+0000 7f45106e0640 1 --2- 172.21.15.84:0/797802121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44f4004d00 0x7f44f4005180 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4500009fc0 tx=0x7f44f40057f0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.520+0000 7f450ec7d640 1 -- 172.21.15.84:0/797802121 reap_dead start 2023-10-16T10:07:13.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.520+0000 7f45106e0640 1 -- 172.21.15.84:0/797802121 shutdown_connections 2023-10-16T10:07:13.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.520+0000 7f45106e0640 1 -- 172.21.15.84:0/797802121 >> 172.21.15.84:0/797802121 conn(0x7f45080cb410 msgr2=0x7f45080cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:13.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.520+0000 7f45106e0640 1 -- 172.21.15.84:0/797802121 shutdown_connections 2023-10-16T10:07:13.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.520+0000 7f45106e0640 1 -- 172.21.15.84:0/797802121 wait complete. 2023-10-16T10:07:13.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.522+0000 7f45106e0640 1 Processor -- start 2023-10-16T10:07:13.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.522+0000 7f45106e0640 1 -- start start 2023-10-16T10:07:13.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.523+0000 7f45106e0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f4004d00 0x7f45081f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:13.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.523+0000 7f45106e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45080b0670 0x7f45080c3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:13.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.523+0000 7f45106e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4508158fd0 0x7f450815ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:13.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.523+0000 7f45106e0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45080b1a70 con 0x7f4508158fd0 2023-10-16T10:07:13.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.523+0000 7f45106e0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45081f6850 con 0x7f44f4004d00 2023-10-16T10:07:13.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.523+0000 7f45106e0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450815bad0 con 0x7f45080b0670 2023-10-16T10:07:13.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.523+0000 7f450ec7d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4508158fd0 0x7f450815ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:13.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.523+0000 7f450ec7d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4508158fd0 0x7f450815ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54938/0 (socket says 172.21.15.84:54938) 2023-10-16T10:07:13.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.524+0000 7f450ec7d640 1 -- 172.21.15.84:0/804076591 learned_addr learned my addr 172.21.15.84:0/804076591 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:13.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.524+0000 7f450e47c640 1 --2- 172.21.15.84:0/804076591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f4004d00 0x7f45081f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:13.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.524+0000 7f450dc7b640 1 --2- 172.21.15.84:0/804076591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45080b0670 0x7f45080c3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:13.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.524+0000 7f450ec7d640 1 -- 172.21.15.84:0/804076591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45080b0670 msgr2=0x7f45080c3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:13.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.524+0000 7f450ec7d640 1 --2- 172.21.15.84:0/804076591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45080b0670 0x7f45080c3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.524+0000 7f450ec7d640 1 -- 172.21.15.84:0/804076591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f4004d00 msgr2=0x7f45081f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:13.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.524+0000 7f450ec7d640 1 --2- 172.21.15.84:0/804076591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f4004d00 0x7f45081f6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.524+0000 7f450ec7d640 1 -- 172.21.15.84:0/804076591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45080b14b0 con 0x7f4508158fd0 2023-10-16T10:07:13.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.524+0000 7f450ec7d640 1 --2- 172.21.15.84:0/804076591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4508158fd0 0x7f450815ce20 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f450000e230 tx=0x7f4500009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:13.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.525+0000 7f44feffd640 1 -- 172.21.15.84:0/804076591 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500034d50 con 0x7f4508158fd0 2023-10-16T10:07:13.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.525+0000 7f44feffd640 1 -- 172.21.15.84:0/804076591 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4500003990 con 0x7f4508158fd0 2023-10-16T10:07:13.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.525+0000 7f44feffd640 1 -- 172.21.15.84:0/804076591 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500003d40 con 0x7f4508158fd0 2023-10-16T10:07:13.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.525+0000 7f45106e0640 1 -- 172.21.15.84:0/804076591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45080b1790 con 0x7f4508158fd0 2023-10-16T10:07:13.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.525+0000 7f45106e0640 1 -- 172.21.15.84:0/804076591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450815b760 con 0x7f4508158fd0 2023-10-16T10:07:13.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.526+0000 7f45106e0640 1 -- 172.21.15.84:0/804076591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f450815b2e0 con 0x7f4508158fd0 2023-10-16T10:07:13.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.527+0000 7f44feffd640 1 -- 172.21.15.84:0/804076591 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 57) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f4500004030 con 0x7f4508158fd0 2023-10-16T10:07:13.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.528+0000 7f44feffd640 1 --2- 172.21.15.84:0/804076591 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f44ec06acc0 0x7f44ec06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:13.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.529+0000 7f44feffd640 1 -- 172.21.15.84:0/804076591 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6512+0+0 (secure 0 0 0) 0x7f45000ab130 con 0x7f4508158fd0 2023-10-16T10:07:13.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.529+0000 7f450e47c640 1 --2- 172.21.15.84:0/804076591 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f44ec06acc0 0x7f44ec06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:13.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.529+0000 7f450e47c640 1 --2- 172.21.15.84:0/804076591 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f44ec06acc0 0x7f44ec06d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4504000f60 tx=0x7f4504001710 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:13.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.533+0000 7f44feffd640 1 -- 172.21.15.84:0/804076591 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f450815b2e0 con 0x7f4508158fd0 2023-10-16T10:07:13.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.775+0000 7f45106e0640 1 -- 172.21.15.84:0/804076591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f450815b2e0 con 0x7f4508158fd0 2023-10-16T10:07:13.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.775+0000 7f44feffd640 1 -- 172.21.15.84:0/804076591 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f450815b2e0 con 0x7f4508158fd0 2023-10-16T10:07:13.776 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:13.776 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3458903351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3458903351},{"type":"v1","addr":"172.21.15.84:6837","nonce":3458903351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4002935522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4002935522},{"type":"v1","addr":"172.21.15.157:6833","nonce":4002935522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5809,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1890551305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1890551305},{"type":"v1","addr":"172.21.15.157:6835","nonce":1890551305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:05:48.809508+0000","modified":"2023-10-16T10:05:51.856862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7521":{"gid":7521,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/3621445269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3621445269},{"type":"v1","addr":"172.21.15.84:6836","nonce":3621445269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-16T10:07:13.777 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 53 2023-10-16T10:07:13.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.778+0000 7f45106e0640 1 -- 172.21.15.84:0/804076591 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f44ec06acc0 msgr2=0x7f44ec06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:13.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.778+0000 7f45106e0640 1 --2- 172.21.15.84:0/804076591 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f44ec06acc0 0x7f44ec06d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4504000f60 tx=0x7f4504001710 comp rx=0 tx=0).stop 2023-10-16T10:07:13.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.778+0000 7f45106e0640 1 -- 172.21.15.84:0/804076591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4508158fd0 msgr2=0x7f450815ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:13.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.778+0000 7f45106e0640 1 --2- 172.21.15.84:0/804076591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4508158fd0 0x7f450815ce20 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f450000e230 tx=0x7f4500009750 comp rx=0 tx=0).stop 2023-10-16T10:07:13.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.778+0000 7f45106e0640 1 -- 172.21.15.84:0/804076591 shutdown_connections 2023-10-16T10:07:13.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.778+0000 7f45106e0640 1 --2- 172.21.15.84:0/804076591 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f44ec06acc0 0x7f44ec06d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.778+0000 7f45106e0640 1 --2- 172.21.15.84:0/804076591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4508158fd0 0x7f450815ce20 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.778+0000 7f45106e0640 1 --2- 172.21.15.84:0/804076591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45080b0670 0x7f45080c3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.778+0000 7f45106e0640 1 --2- 172.21.15.84:0/804076591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f4004d00 0x7f45081f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:13.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.778+0000 7f45106e0640 1 -- 172.21.15.84:0/804076591 >> 172.21.15.84:0/804076591 conn(0x7f45080cb410 msgr2=0x7f4508070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:13.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.778+0000 7f45106e0640 1 -- 172.21.15.84:0/804076591 shutdown_connections 2023-10-16T10:07:13.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.779+0000 7f45106e0640 1 -- 172.21.15.84:0/804076591 wait complete. 2023-10-16T10:07:13.797 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:07:14.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.999+0000 7f5079c07640 1 Processor -- start 2023-10-16T10:07:14.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:13.999+0000 7f5079c07640 1 -- start start 2023-10-16T10:07:14.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.001+0000 7f5079c07640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f507415aaa0 0x7f507415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.001+0000 7f5079c07640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f507415ae80 0x7f50740cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.001+0000 7f5079c07640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f50740ccaa0 0x7f5074070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.001+0000 7f5079c07640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507406b810 con 0x7f507415ae80 2023-10-16T10:07:14.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.001+0000 7f5079c07640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507415ed60 con 0x7f50740ccaa0 2023-10-16T10:07:14.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.001+0000 7f5079c07640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50740707e0 con 0x7f507415aaa0 2023-10-16T10:07:14.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.001+0000 7f506bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f507415ae80 0x7f50740cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.001+0000 7f506bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f507415ae80 0x7f50740cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54968/0 (socket says 172.21.15.84:54968) 2023-10-16T10:07:14.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.001+0000 7f506bfff640 1 -- 172.21.15.84:0/3494182167 learned_addr learned my addr 172.21.15.84:0/3494182167 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:14.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.002+0000 7f50737fe640 1 --2- 172.21.15.84:0/3494182167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f507415aaa0 0x7f507415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.002+0000 7f5073fff640 1 --2- 172.21.15.84:0/3494182167 >> v2:172.21.15.157:3300/0 conn(0x7f50740ccaa0 0x7f5074070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.002+0000 7f506bfff640 1 -- 172.21.15.84:0/3494182167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f507415aaa0 msgr2=0x7f507415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:14.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.002+0000 7f506bfff640 1 --2- 172.21.15.84:0/3494182167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f507415aaa0 0x7f507415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.002+0000 7f506bfff640 1 -- 172.21.15.84:0/3494182167 >> v2:172.21.15.157:3300/0 conn(0x7f50740ccaa0 msgr2=0x7f5074070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:14.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.002+0000 7f506bfff640 1 --2- 172.21.15.84:0/3494182167 >> v2:172.21.15.157:3300/0 conn(0x7f50740ccaa0 0x7f5074070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.002+0000 7f506bfff640 1 -- 172.21.15.84:0/3494182167 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507415ef90 con 0x7f507415ae80 2023-10-16T10:07:14.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.003+0000 7f506bfff640 1 --2- 172.21.15.84:0/3494182167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507415ae80 0x7f50740cc560 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f50740ae3c0 tx=0x7f505c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4735d54321f04cf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:14.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.004+0000 7f5072ffd640 1 -- 172.21.15.84:0/3494182167 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c031870 con 0x7f507415ae80 2023-10-16T10:07:14.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.004+0000 7f5072ffd640 1 -- 172.21.15.84:0/3494182167 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505c0319d0 con 0x7f507415ae80 2023-10-16T10:07:14.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.004+0000 7f5072ffd640 1 -- 172.21.15.84:0/3494182167 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c031d60 con 0x7f507415ae80 2023-10-16T10:07:14.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.004+0000 7f5079c07640 1 -- 172.21.15.84:0/3494182167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507415ae80 msgr2=0x7f50740cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:14.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.005+0000 7f5079c07640 1 --2- 172.21.15.84:0/3494182167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507415ae80 0x7f50740cc560 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f50740ae3c0 tx=0x7f505c02b910 comp rx=0 tx=0).stop 2023-10-16T10:07:14.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.005+0000 7f5079c07640 1 -- 172.21.15.84:0/3494182167 shutdown_connections 2023-10-16T10:07:14.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.005+0000 7f5079c07640 1 --2- 172.21.15.84:0/3494182167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507415ae80 0x7f50740cc560 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.005+0000 7f5079c07640 1 --2- 172.21.15.84:0/3494182167 >> v2:172.21.15.157:3300/0 conn(0x7f50740ccaa0 0x7f5074070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.005+0000 7f5079c07640 1 --2- 172.21.15.84:0/3494182167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f507415aaa0 0x7f507415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.005+0000 7f5079c07640 1 -- 172.21.15.84:0/3494182167 >> 172.21.15.84:0/3494182167 conn(0x7f50740cb410 msgr2=0x7f50740cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:14.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.005+0000 7f5079c07640 1 -- 172.21.15.84:0/3494182167 shutdown_connections 2023-10-16T10:07:14.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.005+0000 7f5079c07640 1 -- 172.21.15.84:0/3494182167 wait complete. 2023-10-16T10:07:14.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.007+0000 7f5079c07640 1 Processor -- start 2023-10-16T10:07:14.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.007+0000 7f5079c07640 1 -- start start 2023-10-16T10:07:14.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.008+0000 7f5079c07640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50740ccaa0 0x7f5074155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.008+0000 7f5079c07640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507415aaa0 0x7f5074155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.008+0000 7f5079c07640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f507415ae80 0x7f5074158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.008+0000 7f5079c07640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50740c5370 con 0x7f507415aaa0 2023-10-16T10:07:14.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.008+0000 7f5079c07640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50740c5090 con 0x7f50740ccaa0 2023-10-16T10:07:14.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.008+0000 7f5079c07640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5074158090 con 0x7f507415ae80 2023-10-16T10:07:14.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.008+0000 7f506bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507415aaa0 0x7f5074155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.008+0000 7f506bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507415aaa0 0x7f5074155e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54970/0 (socket says 172.21.15.84:54970) 2023-10-16T10:07:14.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.008+0000 7f506bfff640 1 -- 172.21.15.84:0/1426572015 learned_addr learned my addr 172.21.15.84:0/1426572015 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:14.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.008+0000 7f50737fe640 1 --2- 172.21.15.84:0/1426572015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50740ccaa0 0x7f5074155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.008+0000 7f5073fff640 1 --2- 172.21.15.84:0/1426572015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f507415ae80 0x7f5074158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.008+0000 7f506bfff640 1 -- 172.21.15.84:0/1426572015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f507415ae80 msgr2=0x7f5074158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:14.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.008+0000 7f506bfff640 1 --2- 172.21.15.84:0/1426572015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f507415ae80 0x7f5074158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.008+0000 7f506bfff640 1 -- 172.21.15.84:0/1426572015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50740ccaa0 msgr2=0x7f5074155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:14.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.008+0000 7f506bfff640 1 --2- 172.21.15.84:0/1426572015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50740ccaa0 0x7f5074155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.009+0000 7f506bfff640 1 -- 172.21.15.84:0/1426572015 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c0094d0 con 0x7f507415aaa0 2023-10-16T10:07:14.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.009+0000 7f50737fe640 1 --2- 172.21.15.84:0/1426572015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50740ccaa0 0x7f5074155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:14.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.009+0000 7f5073fff640 1 --2- 172.21.15.84:0/1426572015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f507415ae80 0x7f5074158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:14.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.009+0000 7f506bfff640 1 --2- 172.21.15.84:0/1426572015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507415aaa0 0x7f5074155e80 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f505c02b950 tx=0x7f505c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:14.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.009+0000 7f5070ff9640 1 -- 172.21.15.84:0/1426572015 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c03bcc0 con 0x7f507415aaa0 2023-10-16T10:07:14.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.009+0000 7f5070ff9640 1 -- 172.21.15.84:0/1426572015 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505c03be40 con 0x7f507415aaa0 2023-10-16T10:07:14.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.009+0000 7f5070ff9640 1 -- 172.21.15.84:0/1426572015 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c045630 con 0x7f507415aaa0 2023-10-16T10:07:14.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.009+0000 7f5079c07640 1 -- 172.21.15.84:0/1426572015 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5074158370 con 0x7f507415aaa0 2023-10-16T10:07:14.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.009+0000 7f5079c07640 1 -- 172.21.15.84:0/1426572015 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50740c4cd0 con 0x7f507415aaa0 2023-10-16T10:07:14.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.011+0000 7f5079c07640 1 -- 172.21.15.84:0/1426572015 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f507404e160 con 0x7f507415aaa0 2023-10-16T10:07:14.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.012+0000 7f5070ff9640 1 -- 172.21.15.84:0/1426572015 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 57) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f505c04e450 con 0x7f507415aaa0 2023-10-16T10:07:14.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.012+0000 7f5070ff9640 1 --2- 172.21.15.84:0/1426572015 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f505406acc0 0x7f505406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.013+0000 7f5070ff9640 1 -- 172.21.15.84:0/1426572015 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6512+0+0 (secure 0 0 0) 0x7f505c02d040 con 0x7f507415aaa0 2023-10-16T10:07:14.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.013+0000 7f50737fe640 1 --2- 172.21.15.84:0/1426572015 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f505406acc0 0x7f505406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.013+0000 7f50737fe640 1 --2- 172.21.15.84:0/1426572015 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f505406acc0 0x7f505406d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5064006fd0 tx=0x7f5064008040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:14.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.018+0000 7f5070ff9640 1 -- 172.21.15.84:0/1426572015 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f505c076c50 con 0x7f507415aaa0 2023-10-16T10:07:14.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.206+0000 7f5070ff9640 1 -- 172.21.15.84:0/1426572015 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 58) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f505c077020 con 0x7f507415aaa0 2023-10-16T10:07:14.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.255+0000 7f5079c07640 1 -- 172.21.15.84:0/1426572015 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5074070a00 con 0x7f507415aaa0 2023-10-16T10:07:14.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.257+0000 7f5070ff9640 1 -- 172.21.15.84:0/1426572015 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f505c0910b0 con 0x7f507415aaa0 2023-10-16T10:07:14.258 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:14.258 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3458903351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3458903351},{"type":"v1","addr":"172.21.15.84:6837","nonce":3458903351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4002935522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4002935522},{"type":"v1","addr":"172.21.15.157:6833","nonce":4002935522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5809,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1890551305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1890551305},{"type":"v1","addr":"172.21.15.157:6835","nonce":1890551305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:05:48.809508+0000","modified":"2023-10-16T10:05:51.856862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7521":{"gid":7521,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/3621445269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3621445269},{"type":"v1","addr":"172.21.15.84:6836","nonce":3621445269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-16T10:07:14.258 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 53 2023-10-16T10:07:14.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.259+0000 7f5079c07640 1 -- 172.21.15.84:0/1426572015 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f505406acc0 msgr2=0x7f505406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:14.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.260+0000 7f5079c07640 1 --2- 172.21.15.84:0/1426572015 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f505406acc0 0x7f505406d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5064006fd0 tx=0x7f5064008040 comp rx=0 tx=0).stop 2023-10-16T10:07:14.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.260+0000 7f5079c07640 1 -- 172.21.15.84:0/1426572015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507415aaa0 msgr2=0x7f5074155e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:14.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.260+0000 7f5079c07640 1 --2- 172.21.15.84:0/1426572015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507415aaa0 0x7f5074155e80 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f505c02b950 tx=0x7f505c0046c0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.260+0000 7f5079c07640 1 -- 172.21.15.84:0/1426572015 shutdown_connections 2023-10-16T10:07:14.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.260+0000 7f5079c07640 1 --2- 172.21.15.84:0/1426572015 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f505406acc0 0x7f505406d180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.260+0000 7f5079c07640 1 --2- 172.21.15.84:0/1426572015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f507415ae80 0x7f5074158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.260+0000 7f5079c07640 1 --2- 172.21.15.84:0/1426572015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507415aaa0 0x7f5074155e80 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.260+0000 7f5079c07640 1 --2- 172.21.15.84:0/1426572015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50740ccaa0 0x7f5074155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.260+0000 7f5079c07640 1 -- 172.21.15.84:0/1426572015 >> 172.21.15.84:0/1426572015 conn(0x7f50740cb410 msgr2=0x7f50740ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:14.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.260+0000 7f5079c07640 1 -- 172.21.15.84:0/1426572015 shutdown_connections 2023-10-16T10:07:14.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.260+0000 7f5079c07640 1 -- 172.21.15.84:0/1426572015 wait complete. 2023-10-16T10:07:14.279 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:07:14.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.486+0000 7f8234ac9640 1 Processor -- start 2023-10-16T10:07:14.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.487+0000 7f8234ac9640 1 -- start start 2023-10-16T10:07:14.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.488+0000 7f8234ac9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82300ce390 0x7f823015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.488+0000 7f8234ac9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823015ed60 con 0x7f823015aaa0 2023-10-16T10:07:14.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.488+0000 7f8234ac9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82300707e0 con 0x7f823015ae80 2023-10-16T10:07:14.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.488+0000 7f8234ac9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823006b810 con 0x7f82300ce390 2023-10-16T10:07:14.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.488+0000 7f822e59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f823015aaa0 0x7f82300cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55010/0 (socket says 172.21.15.84:55010) 2023-10-16T10:07:14.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.488+0000 7f822e59c640 1 -- 172.21.15.84:0/1186476864 learned_addr learned my addr 172.21.15.84:0/1186476864 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:14.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.489+0000 7f822ed9d640 1 --2- 172.21.15.84:0/1186476864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82300ce390 0x7f823015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.489+0000 7f822dd9b640 1 -- 172.21.15.84:0/1186476864 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627403445 0 0) 0x7f823015ed60 con 0x7f823015aaa0 2023-10-16T10:07:14.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.489+0000 7f822dd9b640 1 -- 172.21.15.84:0/1186476864 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f821c003520 con 0x7f823015aaa0 2023-10-16T10:07:14.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.489+0000 7f822dd9b640 1 -- 172.21.15.84:0/1186476864 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3857239164 0 0) 0x7f821c003520 con 0x7f823015aaa0 2023-10-16T10:07:14.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.489+0000 7f822dd9b640 1 -- 172.21.15.84:0/1186476864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82300ce390 msgr2=0x7f823015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:14.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.489+0000 7f822dd9b640 1 --2- 172.21.15.84:0/1186476864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82300ce390 0x7f823015e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.489+0000 7f822dd9b640 1 -- 172.21.15.84:0/1186476864 >> v1:172.21.15.157:6789/0 conn(0x7f823015ae80 legacy=0x7f8230070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:14.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.489+0000 7f822dd9b640 1 -- 172.21.15.84:0/1186476864 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823015ef90 con 0x7f823015aaa0 2023-10-16T10:07:14.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.489+0000 7f822dd9b640 1 -- 172.21.15.84:0/1186476864 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f8220001690 con 0x7f823015aaa0 2023-10-16T10:07:14.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.489+0000 7f822dd9b640 1 -- 172.21.15.84:0/1186476864 >> v1:172.21.15.84:6789/0 conn(0x7f823015aaa0 legacy=0x7f82300cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:14.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.489+0000 7f822dd9b640 1 --2- 172.21.15.84:0/1186476864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f821c0039a0 0x7f821c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.490+0000 7f822dd9b640 1 --2- 172.21.15.84:0/1186476864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f821c0047c0 0x7f821c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.490+0000 7f822dd9b640 1 --2- 172.21.15.84:0/1186476864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82300ce390 0x7f821c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.490+0000 7f822dd9b640 1 -- 172.21.15.84:0/1186476864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823006b810 con 0x7f821c0047c0 2023-10-16T10:07:14.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.490+0000 7f822dd9b640 1 -- 172.21.15.84:0/1186476864 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c00ba60 con 0x7f821c0039a0 2023-10-16T10:07:14.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.490+0000 7f822dd9b640 1 -- 172.21.15.84:0/1186476864 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821c00bc20 con 0x7f82300ce390 2023-10-16T10:07:14.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.490+0000 7f8227fff640 1 --2- 172.21.15.84:0/1186476864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f821c0047c0 0x7f821c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.490+0000 7f822ed9d640 1 --2- 172.21.15.84:0/1186476864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82300ce390 0x7f821c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.490+0000 7f8227fff640 1 -- 172.21.15.84:0/1186476864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82300ce390 msgr2=0x7f821c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:14.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.490+0000 7f822e59c640 1 --2- 172.21.15.84:0/1186476864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f821c0039a0 0x7f821c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.490+0000 7f8227fff640 1 --2- 172.21.15.84:0/1186476864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82300ce390 0x7f821c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.490+0000 7f8227fff640 1 -- 172.21.15.84:0/1186476864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f821c0039a0 msgr2=0x7f821c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:14.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.490+0000 7f8227fff640 1 --2- 172.21.15.84:0/1186476864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f821c0039a0 0x7f821c003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.490+0000 7f8227fff640 1 -- 172.21.15.84:0/1186476864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c003520 con 0x7f821c0047c0 2023-10-16T10:07:14.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.491+0000 7f8227fff640 1 --2- 172.21.15.84:0/1186476864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f821c0047c0 0x7f821c004c40 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f823015b2f0 tx=0x7f821c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1497875e7e12d397 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:14.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.492+0000 7f822dd9b640 1 -- 172.21.15.84:0/1186476864 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82180052d0 con 0x7f821c0047c0 2023-10-16T10:07:14.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.492+0000 7f822dd9b640 1 -- 172.21.15.84:0/1186476864 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821802e050 con 0x7f821c0047c0 2023-10-16T10:07:14.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.492+0000 7f822dd9b640 1 -- 172.21.15.84:0/1186476864 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82180035f0 con 0x7f821c0047c0 2023-10-16T10:07:14.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.492+0000 7f8234ac9640 1 -- 172.21.15.84:0/1186476864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f821c0047c0 msgr2=0x7f821c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:14.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.492+0000 7f8234ac9640 1 --2- 172.21.15.84:0/1186476864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f821c0047c0 0x7f821c004c40 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f823015b2f0 tx=0x7f821c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.492+0000 7f822ed9d640 1 -- 172.21.15.84:0/1186476864 reap_dead start 2023-10-16T10:07:14.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.492+0000 7f8234ac9640 1 -- 172.21.15.84:0/1186476864 shutdown_connections 2023-10-16T10:07:14.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.492+0000 7f8234ac9640 1 -- 172.21.15.84:0/1186476864 >> 172.21.15.84:0/1186476864 conn(0x7f82300cb410 msgr2=0x7f82300cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:14.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.492+0000 7f8234ac9640 1 -- 172.21.15.84:0/1186476864 shutdown_connections 2023-10-16T10:07:14.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.493+0000 7f8234ac9640 1 -- 172.21.15.84:0/1186476864 wait complete. 2023-10-16T10:07:14.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.494+0000 7f8234ac9640 1 Processor -- start 2023-10-16T10:07:14.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.494+0000 7f8234ac9640 1 -- start start 2023-10-16T10:07:14.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.495+0000 7f8234ac9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f821c0047c0 0x7f82300ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.495+0000 7f8234ac9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82300c2090 0x7f82300c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.495+0000 7f8234ac9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82300c4900 0x7f8230158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.495+0000 7f8234ac9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82300ca070 con 0x7f82300c4900 2023-10-16T10:07:14.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.495+0000 7f8234ac9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82300c9460 con 0x7f82300c2090 2023-10-16T10:07:14.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.495+0000 7f8234ac9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82300c9740 con 0x7f821c0047c0 2023-10-16T10:07:14.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.496+0000 7f822ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82300c4900 0x7f8230158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.496+0000 7f822ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82300c4900 0x7f8230158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54998/0 (socket says 172.21.15.84:54998) 2023-10-16T10:07:14.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.496+0000 7f822ed9d640 1 -- 172.21.15.84:0/3496087688 learned_addr learned my addr 172.21.15.84:0/3496087688 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:14.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.496+0000 7f822e59c640 1 --2- 172.21.15.84:0/3496087688 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f821c0047c0 0x7f82300ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.496+0000 7f8227fff640 1 --2- 172.21.15.84:0/3496087688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82300c2090 0x7f82300c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.496+0000 7f822ed9d640 1 -- 172.21.15.84:0/3496087688 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f821c0047c0 msgr2=0x7f82300ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:14.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.496+0000 7f822ed9d640 1 --2- 172.21.15.84:0/3496087688 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f821c0047c0 0x7f82300ca500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.496+0000 7f822ed9d640 1 -- 172.21.15.84:0/3496087688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82300c2090 msgr2=0x7f82300c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:14.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.496+0000 7f822ed9d640 1 --2- 172.21.15.84:0/3496087688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82300c2090 0x7f82300c5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.496+0000 7f822ed9d640 1 -- 172.21.15.84:0/3496087688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82300c9a20 con 0x7f82300c4900 2023-10-16T10:07:14.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.496+0000 7f822ed9d640 1 --2- 172.21.15.84:0/3496087688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82300c4900 0x7f8230158f60 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f82300c5990 tx=0x7f822800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:14.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.496+0000 7f8226ffd640 1 -- 172.21.15.84:0/3496087688 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228017400 con 0x7f82300c4900 2023-10-16T10:07:14.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.497+0000 7f8226ffd640 1 -- 172.21.15.84:0/3496087688 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8228011040 con 0x7f82300c4900 2023-10-16T10:07:14.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.497+0000 7f8234ac9640 1 -- 172.21.15.84:0/3496087688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82300c9d00 con 0x7f82300c4900 2023-10-16T10:07:14.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.497+0000 7f8226ffd640 1 -- 172.21.15.84:0/3496087688 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82280177c0 con 0x7f82300c4900 2023-10-16T10:07:14.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.497+0000 7f8234ac9640 1 -- 172.21.15.84:0/3496087688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82300c4f70 con 0x7f82300c4900 2023-10-16T10:07:14.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.498+0000 7f8234ac9640 1 -- 172.21.15.84:0/3496087688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82300b14b0 con 0x7f82300c4900 2023-10-16T10:07:14.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.499+0000 7f8226ffd640 1 -- 172.21.15.84:0/3496087688 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 58) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f8228017920 con 0x7f82300c4900 2023-10-16T10:07:14.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.500+0000 7f8226ffd640 1 --2- 172.21.15.84:0/3496087688 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f821006af00 0x7f821006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.500+0000 7f8226ffd640 1 -- 172.21.15.84:0/3496087688 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6512+0+0 (secure 0 0 0) 0x7f8228089fb0 con 0x7f82300c4900 2023-10-16T10:07:14.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.500+0000 7f822e59c640 1 --2- 172.21.15.84:0/3496087688 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f821006af00 0x7f821006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.501+0000 7f822e59c640 1 --2- 172.21.15.84:0/3496087688 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f821006af00 0x7f821006d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8220001200 tx=0x7f8220009040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.504+0000 7f8226ffd640 1 -- 172.21.15.84:0/3496087688 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8228054c60 con 0x7f82300c4900 2023-10-16T10:07:14.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.698+0000 7f8234ac9640 1 -- 172.21.15.84:0/3496087688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f823004e160 con 0x7f82300c4900 2023-10-16T10:07:14.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.699+0000 7f8226ffd640 1 -- 172.21.15.84:0/3496087688 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+17718 (secure 0 0 0) 0x7f8228058280 con 0x7f82300c4900 2023-10-16T10:07:14.700 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:14.701 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":113,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:07:11.412966+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:05:46.595888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-16T10:05:47.826019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.d","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2143212776":"2023-10-17T10:07:11.412892+0000","172.21.15.84:6833/985901542":"2023-10-17T10:07:11.412892+0000","172.21.15.84:6832/985901542":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/636650964":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/2411644314":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/2627061388":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/1139885639":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/1946898486":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/2051153636":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/628187054":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/965715898":"2023-10-17T10:06:03.602359+0000","172.21.15.84:6833/3250823028":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/3505558848":"2023-10-17T10:07:11.412892+0000","172.21.15.84:6832/3250823028":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/3381824109":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/3871500870":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/1287074955":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/3830790285":"2023-10-16T11:05:57.166133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:07:14.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.702+0000 7f8234ac9640 1 -- 172.21.15.84:0/3496087688 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f821006af00 msgr2=0x7f821006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:14.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.702+0000 7f8234ac9640 1 --2- 172.21.15.84:0/3496087688 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f821006af00 0x7f821006d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8220001200 tx=0x7f8220009040 comp rx=0 tx=0).stop 2023-10-16T10:07:14.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.702+0000 7f8234ac9640 1 -- 172.21.15.84:0/3496087688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82300c4900 msgr2=0x7f8230158f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:14.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.702+0000 7f8234ac9640 1 --2- 172.21.15.84:0/3496087688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82300c4900 0x7f8230158f60 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f82300c5990 tx=0x7f822800f4a0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.702+0000 7f8234ac9640 1 -- 172.21.15.84:0/3496087688 shutdown_connections 2023-10-16T10:07:14.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.702+0000 7f8234ac9640 1 --2- 172.21.15.84:0/3496087688 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f821006af00 0x7f821006d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.702+0000 7f8234ac9640 1 --2- 172.21.15.84:0/3496087688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82300c4900 0x7f8230158f60 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.702+0000 7f8234ac9640 1 --2- 172.21.15.84:0/3496087688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82300c2090 0x7f82300c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.702+0000 7f8234ac9640 1 --2- 172.21.15.84:0/3496087688 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f821c0047c0 0x7f82300ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.702+0000 7f8234ac9640 1 -- 172.21.15.84:0/3496087688 >> 172.21.15.84:0/3496087688 conn(0x7f82300cb410 msgr2=0x7f82300b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:14.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.703+0000 7f8234ac9640 1 -- 172.21.15.84:0/3496087688 shutdown_connections 2023-10-16T10:07:14.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.703+0000 7f8234ac9640 1 -- 172.21.15.84:0/3496087688 wait complete. 2023-10-16T10:07:14.722 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T10:07:14.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.933+0000 7f61590a6640 1 Processor -- start 2023-10-16T10:07:14.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.933+0000 7f61590a6640 1 -- start start 2023-10-16T10:07:14.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.935+0000 7f61590a6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6154065ab0 0x7f61540c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.935+0000 7f61590a6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61540ce280 0x7f61540ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.935+0000 7f61590a6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61540acd70 con 0x7f615406d220 2023-10-16T10:07:14.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.935+0000 7f61590a6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61540ac4a0 con 0x7f6154065ab0 2023-10-16T10:07:14.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.935+0000 7f61590a6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6154065f40 con 0x7f61540ce280 2023-10-16T10:07:14.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.935+0000 7f61537fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6154065ab0 0x7f61540c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.935+0000 7f61537fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6154065ab0 0x7f61540c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55012/0 (socket says 172.21.15.84:55012) 2023-10-16T10:07:14.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.935+0000 7f61537fe640 1 -- 172.21.15.84:0/2424910213 learned_addr learned my addr 172.21.15.84:0/2424910213 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:14.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.935+0000 7f61537fe640 1 -- 172.21.15.84:0/2424910213 >> v2:172.21.15.157:3300/0 conn(0x7f61540ce280 msgr2=0x7f61540ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:14.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.935+0000 7f61588a5640 1 --2- 172.21.15.84:0/2424910213 >> v2:172.21.15.157:3300/0 conn(0x7f61540ce280 0x7f61540ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.935+0000 7f61537fe640 1 --2- 172.21.15.84:0/2424910213 >> v2:172.21.15.157:3300/0 conn(0x7f61540ce280 0x7f61540ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.936+0000 7f61537fe640 1 -- 172.21.15.84:0/2424910213 >> v1:172.21.15.84:6789/0 conn(0x7f615406d220 legacy=0x7f61540cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:14.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.936+0000 7f61537fe640 1 -- 172.21.15.84:0/2424910213 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61540ac720 con 0x7f6154065ab0 2023-10-16T10:07:14.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.936+0000 7f6152ffd640 1 -- 172.21.15.84:0/2424910213 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704576763 0 0) 0x7f61540acd70 con 0x7f615406d220 2023-10-16T10:07:14.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.937+0000 7f61537fe640 1 --2- 172.21.15.84:0/2424910213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6154065ab0 0x7f61540c95e0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f6154071fd0 tx=0x7f614402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1aa2e80b102e91 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:14.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.937+0000 7f6152ffd640 1 -- 172.21.15.84:0/2424910213 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614402ca80 con 0x7f6154065ab0 2023-10-16T10:07:14.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.937+0000 7f6152ffd640 1 -- 172.21.15.84:0/2424910213 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614402cbe0 con 0x7f6154065ab0 2023-10-16T10:07:14.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.937+0000 7f6152ffd640 1 -- 172.21.15.84:0/2424910213 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61440343f0 con 0x7f6154065ab0 2023-10-16T10:07:14.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.937+0000 7f61590a6640 1 -- 172.21.15.84:0/2424910213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6154065ab0 msgr2=0x7f61540c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:14.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.937+0000 7f61590a6640 1 --2- 172.21.15.84:0/2424910213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6154065ab0 0x7f61540c95e0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f6154071fd0 tx=0x7f614402bee0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.938+0000 7f61590a6640 1 -- 172.21.15.84:0/2424910213 shutdown_connections 2023-10-16T10:07:14.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.938+0000 7f61590a6640 1 --2- 172.21.15.84:0/2424910213 >> v2:172.21.15.157:3300/0 conn(0x7f61540ce280 0x7f61540ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.938+0000 7f61590a6640 1 --2- 172.21.15.84:0/2424910213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6154065ab0 0x7f61540c95e0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.938+0000 7f61590a6640 1 -- 172.21.15.84:0/2424910213 >> 172.21.15.84:0/2424910213 conn(0x7f61540b0a80 msgr2=0x7f61540b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:14.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.938+0000 7f61590a6640 1 -- 172.21.15.84:0/2424910213 shutdown_connections 2023-10-16T10:07:14.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.938+0000 7f61590a6640 1 -- 172.21.15.84:0/2424910213 wait complete. 2023-10-16T10:07:14.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.939+0000 7f61590a6640 1 Processor -- start 2023-10-16T10:07:14.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.940+0000 7f61590a6640 1 -- start start 2023-10-16T10:07:14.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.941+0000 7f61590a6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f615406d220 0x7f61541505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.941+0000 7f61590a6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61540ce280 0x7f61540c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.941+0000 7f61590a6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61540b3f90 0x7f61540c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.941+0000 7f61590a6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61540b2ce0 con 0x7f61540b3f90 2023-10-16T10:07:14.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.941+0000 7f61590a6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61540b2ef0 con 0x7f61540ce280 2023-10-16T10:07:14.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.941+0000 7f61590a6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61540b31d0 con 0x7f615406d220 2023-10-16T10:07:14.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.941+0000 7f61588a5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61540b3f90 0x7f61540c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.941+0000 7f61588a5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61540b3f90 0x7f61540c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55020/0 (socket says 172.21.15.84:55020) 2023-10-16T10:07:14.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.941+0000 7f61588a5640 1 -- 172.21.15.84:0/3170186459 learned_addr learned my addr 172.21.15.84:0/3170186459 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:14.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.941+0000 7f6153fff640 1 --2- 172.21.15.84:0/3170186459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f615406d220 0x7f61541505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.941+0000 7f61537fe640 1 --2- 172.21.15.84:0/3170186459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61540ce280 0x7f61540c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.941+0000 7f61588a5640 1 -- 172.21.15.84:0/3170186459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f615406d220 msgr2=0x7f61541505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:14.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.941+0000 7f61588a5640 1 --2- 172.21.15.84:0/3170186459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f615406d220 0x7f61541505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.941+0000 7f61588a5640 1 -- 172.21.15.84:0/3170186459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61540ce280 msgr2=0x7f61540c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:14.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.941+0000 7f61588a5640 1 --2- 172.21.15.84:0/3170186459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61540ce280 0x7f61540c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:14.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.941+0000 7f61588a5640 1 -- 172.21.15.84:0/3170186459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6144009b30 con 0x7f61540b3f90 2023-10-16T10:07:14.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.941+0000 7f61537fe640 1 --2- 172.21.15.84:0/3170186459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61540ce280 0x7f61540c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:14.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.942+0000 7f61588a5640 1 --2- 172.21.15.84:0/3170186459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61540b3f90 0x7f61540c9d40 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f614c00c7f0 tx=0x7f614c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:14.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.942+0000 7f6150ff9640 1 -- 172.21.15.84:0/3170186459 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c004220 con 0x7f61540b3f90 2023-10-16T10:07:14.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.942+0000 7f6150ff9640 1 -- 172.21.15.84:0/3170186459 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c004380 con 0x7f61540b3f90 2023-10-16T10:07:14.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.942+0000 7f6150ff9640 1 -- 172.21.15.84:0/3170186459 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c007860 con 0x7f61540b3f90 2023-10-16T10:07:14.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.942+0000 7f61590a6640 1 -- 172.21.15.84:0/3170186459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61540b34b0 con 0x7f61540b3f90 2023-10-16T10:07:14.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.942+0000 7f61590a6640 1 -- 172.21.15.84:0/3170186459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61540b3a70 con 0x7f61540b3f90 2023-10-16T10:07:14.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.943+0000 7f61590a6640 1 -- 172.21.15.84:0/3170186459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f615404e9b0 con 0x7f61540b3f90 2023-10-16T10:07:14.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.944+0000 7f6150ff9640 1 -- 172.21.15.84:0/3170186459 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 58) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f614c007bd0 con 0x7f61540b3f90 2023-10-16T10:07:14.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.945+0000 7f6150ff9640 1 --2- 172.21.15.84:0/3170186459 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f613406af30 0x7f613406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:14.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.945+0000 7f6150ff9640 1 -- 172.21.15.84:0/3170186459 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6512+0+0 (secure 0 0 0) 0x7f614c089a80 con 0x7f61540b3f90 2023-10-16T10:07:14.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.945+0000 7f6153fff640 1 --2- 172.21.15.84:0/3170186459 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f613406af30 0x7f613406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:14.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.946+0000 7f6153fff640 1 --2- 172.21.15.84:0/3170186459 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f613406af30 0x7f613406d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f613c000f60 tx=0x7f613c009040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:14.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:14.950+0000 7f6150ff9640 1 -- 172.21.15.84:0/3170186459 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f614c054630 con 0x7f61540b3f90 2023-10-16T10:07:15.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.179+0000 7f61590a6640 1 -- 172.21.15.84:0/3170186459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f61540b4f10 con 0x7f61540b3f90 2023-10-16T10:07:15.180 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-16T10:07:15.180+0000 7fca0c7ff640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T10:07:15.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.184+0000 7f6150ff9640 1 -- 172.21.15.84:0/3170186459 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) v1 ==== 161+0+0 (secure 0 0 0) 0x7f614c01e090 con 0x7f61540b3f90 2023-10-16T10:07:15.184 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T10:07:15.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.187+0000 7f61590a6640 1 -- 172.21.15.84:0/3170186459 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f613406af30 msgr2=0x7f613406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:15.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.187+0000 7f61590a6640 1 --2- 172.21.15.84:0/3170186459 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f613406af30 0x7f613406d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f613c000f60 tx=0x7f613c009040 comp rx=0 tx=0).stop 2023-10-16T10:07:15.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.187+0000 7f61590a6640 1 -- 172.21.15.84:0/3170186459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61540b3f90 msgr2=0x7f61540c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:15.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.187+0000 7f61590a6640 1 --2- 172.21.15.84:0/3170186459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61540b3f90 0x7f61540c9d40 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f614c00c7f0 tx=0x7f614c00ccb0 comp rx=0 tx=0).stop 2023-10-16T10:07:15.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.187+0000 7f61590a6640 1 -- 172.21.15.84:0/3170186459 shutdown_connections 2023-10-16T10:07:15.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.187+0000 7f61590a6640 1 --2- 172.21.15.84:0/3170186459 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f613406af30 0x7f613406d3f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:15.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.187+0000 7f61590a6640 1 --2- 172.21.15.84:0/3170186459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61540b3f90 0x7f61540c9d40 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:15.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.187+0000 7f61590a6640 1 --2- 172.21.15.84:0/3170186459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61540ce280 0x7f61540c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:15.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.187+0000 7f61590a6640 1 --2- 172.21.15.84:0/3170186459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f615406d220 0x7f61541505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:15.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.187+0000 7f61590a6640 1 -- 172.21.15.84:0/3170186459 >> 172.21.15.84:0/3170186459 conn(0x7f61540b0a80 msgr2=0x7f61540ae860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:15.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.187+0000 7f61590a6640 1 -- 172.21.15.84:0/3170186459 shutdown_connections 2023-10-16T10:07:15.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.188+0000 7f61590a6640 1 -- 172.21.15.84:0/3170186459 wait complete. 2023-10-16T10:07:15.205 INFO:tasks.ceph.mds.a.smithi084.stderr: -6910> 2023-10-16T10:05:54.278+0000 7f8cd56dc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:07:15.206 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T10:07:15.265 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T10:07:15.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.395+0000 7f31fb1c9640 1 Processor -- start 2023-10-16T10:07:15.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.395+0000 7f31fb1c9640 1 -- start start 2023-10-16T10:07:15.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.397+0000 7f31fb1c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f415ad70 0x7f31f4069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:15.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.397+0000 7f31fb1c9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31f40c9ca0 0x7f31f406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:15.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.397+0000 7f31fb1c9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f40cb030 con 0x7f31f40ca3b0 2023-10-16T10:07:15.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.397+0000 7f31fb1c9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f40c91c0 con 0x7f31f40c9ca0 2023-10-16T10:07:15.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.397+0000 7f31fb1c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f40c94a0 con 0x7f31f415ad70 2023-10-16T10:07:15.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.397+0000 7f31f3fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31f40c9ca0 0x7f31f406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:15.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.398+0000 7f31f8f65640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f415ad70 0x7f31f4069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:15.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.398+0000 7f31f3fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31f40c9ca0 0x7f31f406a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:47216/0 (socket says 172.21.15.84:47216) 2023-10-16T10:07:15.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.398+0000 7f31f3fff640 1 -- 172.21.15.84:0/769441131 learned_addr learned my addr 172.21.15.84:0/769441131 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:15.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.398+0000 7f31f3fff640 1 -- 172.21.15.84:0/769441131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f415ad70 msgr2=0x7f31f4069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:15.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.398+0000 7f31f3fff640 1 --2- 172.21.15.84:0/769441131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f415ad70 0x7f31f4069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:15.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.398+0000 7f31f3fff640 1 -- 172.21.15.84:0/769441131 >> v1:172.21.15.157:6789/0 conn(0x7f31f40ca3b0 legacy=0x7f31f40ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:15.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.398+0000 7f31f3fff640 1 -- 172.21.15.84:0/769441131 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f40c9780 con 0x7f31f40c9ca0 2023-10-16T10:07:15.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.400+0000 7f31f3fff640 1 --2- 172.21.15.84:0/769441131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f40c9ca0 0x7f31f406a2e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009e50 tx=0x7f31dc02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a89642101f4002be server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:15.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.400+0000 7f31f37fe640 1 -- 172.21.15.84:0/769441131 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc02cb00 con 0x7f31f40c9ca0 2023-10-16T10:07:15.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.400+0000 7f31f37fe640 1 -- 172.21.15.84:0/769441131 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31dc02cc60 con 0x7f31f40c9ca0 2023-10-16T10:07:15.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.400+0000 7f31f37fe640 1 -- 172.21.15.84:0/769441131 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc0343f0 con 0x7f31f40c9ca0 2023-10-16T10:07:15.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.401+0000 7f31fb1c9640 1 -- 172.21.15.84:0/769441131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f40c9ca0 msgr2=0x7f31f406a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:15.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.401+0000 7f31fb1c9640 1 --2- 172.21.15.84:0/769441131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f40c9ca0 0x7f31f406a2e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009e50 tx=0x7f31dc02bf60 comp rx=0 tx=0).stop 2023-10-16T10:07:15.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.401+0000 7f31fb1c9640 1 -- 172.21.15.84:0/769441131 shutdown_connections 2023-10-16T10:07:15.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.401+0000 7f31fb1c9640 1 --2- 172.21.15.84:0/769441131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f40c9ca0 0x7f31f406a2e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:15.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.401+0000 7f31fb1c9640 1 --2- 172.21.15.84:0/769441131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f415ad70 0x7f31f4069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:15.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.401+0000 7f31fb1c9640 1 -- 172.21.15.84:0/769441131 >> 172.21.15.84:0/769441131 conn(0x7f31f40ce9b0 msgr2=0x7f31f40cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:15.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.401+0000 7f31fb1c9640 1 -- 172.21.15.84:0/769441131 shutdown_connections 2023-10-16T10:07:15.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.401+0000 7f31fb1c9640 1 -- 172.21.15.84:0/769441131 wait complete. 2023-10-16T10:07:15.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.402+0000 7f31fb1c9640 1 Processor -- start 2023-10-16T10:07:15.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.403+0000 7f31fb1c9640 1 -- start start 2023-10-16T10:07:15.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.403+0000 7f31fb1c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31f40c9ca0 0x7f31f40c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:15.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.403+0000 7f31fb1c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f40ca3b0 0x7f31f40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:15.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.403+0000 7f31fb1c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f415ad70 0x7f31f40acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:15.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.403+0000 7f31fb1c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f40cd3a0 con 0x7f31f40c9ca0 2023-10-16T10:07:15.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.404+0000 7f31fb1c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f40cc720 con 0x7f31f40ca3b0 2023-10-16T10:07:15.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.404+0000 7f31fb1c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f40cca00 con 0x7f31f415ad70 2023-10-16T10:07:15.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.404+0000 7f31f8f65640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31f40c9ca0 0x7f31f40c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:15.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.404+0000 7f31f8f65640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31f40c9ca0 0x7f31f40c84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55066/0 (socket says 172.21.15.84:55066) 2023-10-16T10:07:15.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.404+0000 7f31f8f65640 1 -- 172.21.15.84:0/3540407095 learned_addr learned my addr 172.21.15.84:0/3540407095 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:15.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.404+0000 7f31f3fff640 1 --2- 172.21.15.84:0/3540407095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f40ca3b0 0x7f31f40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:15.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.404+0000 7f31f9766640 1 --2- 172.21.15.84:0/3540407095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f415ad70 0x7f31f40acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:15.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.404+0000 7f31f8f65640 1 -- 172.21.15.84:0/3540407095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f415ad70 msgr2=0x7f31f40acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:15.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.404+0000 7f31f8f65640 1 --2- 172.21.15.84:0/3540407095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f415ad70 0x7f31f40acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:15.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.404+0000 7f31f8f65640 1 -- 172.21.15.84:0/3540407095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f40ca3b0 msgr2=0x7f31f40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:15.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.404+0000 7f31f8f65640 1 --2- 172.21.15.84:0/3540407095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f40ca3b0 0x7f31f40cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:15.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.404+0000 7f31f8f65640 1 -- 172.21.15.84:0/3540407095 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31dc009b30 con 0x7f31f40c9ca0 2023-10-16T10:07:15.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.405+0000 7f31f8f65640 1 --2- 172.21.15.84:0/3540407095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31f40c9ca0 0x7f31f40c84d0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f31e400e870 tx=0x7f31e400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:15.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.405+0000 7f31f17fa640 1 -- 172.21.15.84:0/3540407095 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e40094f0 con 0x7f31f40c9ca0 2023-10-16T10:07:15.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.405+0000 7f31f17fa640 1 -- 172.21.15.84:0/3540407095 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e400c940 con 0x7f31f40c9ca0 2023-10-16T10:07:15.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.405+0000 7f31fb1c9640 1 -- 172.21.15.84:0/3540407095 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f40ccce0 con 0x7f31f40c9ca0 2023-10-16T10:07:15.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.405+0000 7f31f17fa640 1 -- 172.21.15.84:0/3540407095 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e400ccf0 con 0x7f31f40c9ca0 2023-10-16T10:07:15.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.405+0000 7f31fb1c9640 1 -- 172.21.15.84:0/3540407095 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f40cdd00 con 0x7f31f40c9ca0 2023-10-16T10:07:15.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.406+0000 7f31fb1c9640 1 -- 172.21.15.84:0/3540407095 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f404e160 con 0x7f31f40c9ca0 2023-10-16T10:07:15.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.407+0000 7f31f17fa640 1 -- 172.21.15.84:0/3540407095 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 58) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f31e4013470 con 0x7f31f40c9ca0 2023-10-16T10:07:15.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.408+0000 7f31f17fa640 1 --2- 172.21.15.84:0/3540407095 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f31d806af00 0x7f31d806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:15.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.408+0000 7f31f17fa640 1 -- 172.21.15.84:0/3540407095 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6598+0+0 (secure 0 0 0) 0x7f31e4089c20 con 0x7f31f40c9ca0 2023-10-16T10:07:15.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.408+0000 7f31f3fff640 1 --2- 172.21.15.84:0/3540407095 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f31d806af00 0x7f31d806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:15.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.409+0000 7f31f3fff640 1 --2- 172.21.15.84:0/3540407095 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f31d806af00 0x7f31d806d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f31dc035000 tx=0x7f31dc030000 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:15.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.412+0000 7f31f17fa640 1 -- 172.21.15.84:0/3540407095 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f31e4054860 con 0x7f31f40c9ca0 2023-10-16T10:07:15.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:15.624+0000 7f31fb1c9640 1 -- 172.21.15.84:0/3540407095 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f31f40736e0 con 0x7f31f40c9ca0 2023-10-16T10:07:16.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.226+0000 7f31f17fa640 1 -- 172.21.15.84:0/3540407095 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7f31e4057e80 con 0x7f31f40c9ca0 2023-10-16T10:07:16.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.229+0000 7f31fb1c9640 1 -- 172.21.15.84:0/3540407095 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f31d806af00 msgr2=0x7f31d806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:16.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.229+0000 7f31fb1c9640 1 --2- 172.21.15.84:0/3540407095 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f31d806af00 0x7f31d806d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f31dc035000 tx=0x7f31dc030000 comp rx=0 tx=0).stop 2023-10-16T10:07:16.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.229+0000 7f31fb1c9640 1 -- 172.21.15.84:0/3540407095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31f40c9ca0 msgr2=0x7f31f40c84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:16.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.229+0000 7f31fb1c9640 1 --2- 172.21.15.84:0/3540407095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31f40c9ca0 0x7f31f40c84d0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f31e400e870 tx=0x7f31e400ed30 comp rx=0 tx=0).stop 2023-10-16T10:07:16.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.229+0000 7f31fb1c9640 1 -- 172.21.15.84:0/3540407095 shutdown_connections 2023-10-16T10:07:16.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.229+0000 7f31fb1c9640 1 --2- 172.21.15.84:0/3540407095 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f31d806af00 0x7f31d806d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:16.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.229+0000 7f31fb1c9640 1 --2- 172.21.15.84:0/3540407095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f415ad70 0x7f31f40acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:16.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.229+0000 7f31fb1c9640 1 --2- 172.21.15.84:0/3540407095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f40ca3b0 0x7f31f40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:16.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.229+0000 7f31fb1c9640 1 --2- 172.21.15.84:0/3540407095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31f40c9ca0 0x7f31f40c84d0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:16.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.229+0000 7f31fb1c9640 1 -- 172.21.15.84:0/3540407095 >> 172.21.15.84:0/3540407095 conn(0x7f31f40ce9b0 msgr2=0x7f31f4159fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:16.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.229+0000 7f31fb1c9640 1 -- 172.21.15.84:0/3540407095 shutdown_connections 2023-10-16T10:07:16.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.230+0000 7f31fb1c9640 1 -- 172.21.15.84:0/3540407095 wait complete. 2023-10-16T10:07:16.253 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T10:07:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.451+0000 7efc7f99e640 1 Processor -- start 2023-10-16T10:07:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.451+0000 7efc7f99e640 1 -- start start 2023-10-16T10:07:16.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.453+0000 7efc7f99e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efc780c9ab0 0x7efc780c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.453+0000 7efc7f99e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efc780b0360 0x7efc780b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.453+0000 7efc7f99e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc780b0cf0 con 0x7efc7815cf60 2023-10-16T10:07:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.453+0000 7efc7f99e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc7815d340 con 0x7efc780c9ab0 2023-10-16T10:07:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.453+0000 7efc7f99e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc780cc860 con 0x7efc780b0360 2023-10-16T10:07:16.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.453+0000 7efc7d73a640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7efc7815cf60 0x7efc7806f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48282/0 (socket says 172.21.15.84:48282) 2023-10-16T10:07:16.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.453+0000 7efc7d73a640 1 -- 172.21.15.84:0/3852785808 learned_addr learned my addr 172.21.15.84:0/3852785808 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:16.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.453+0000 7efc7cf39640 1 --2- 172.21.15.84:0/3852785808 >> v2:172.21.15.84:3300/0 conn(0x7efc780c9ab0 0x7efc780c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:16.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.454+0000 7efc6ffff640 1 -- 172.21.15.84:0/3852785808 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 485428954 0 0) 0x7efc74001110 con 0x7efc7815cf60 2023-10-16T10:07:16.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.454+0000 7efc7df3b640 1 --2- 172.21.15.84:0/3852785808 >> v2:172.21.15.157:3300/0 conn(0x7efc780b0360 0x7efc780b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:16.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.454+0000 7efc6ffff640 1 -- 172.21.15.84:0/3852785808 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc64003520 con 0x7efc7815cf60 2023-10-16T10:07:16.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.454+0000 7efc7cf39640 1 -- 172.21.15.84:0/3852785808 >> v2:172.21.15.157:3300/0 conn(0x7efc780b0360 msgr2=0x7efc780b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:16.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.454+0000 7efc7cf39640 1 --2- 172.21.15.84:0/3852785808 >> v2:172.21.15.157:3300/0 conn(0x7efc780b0360 0x7efc780b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:16.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.454+0000 7efc7cf39640 1 -- 172.21.15.84:0/3852785808 >> v1:172.21.15.84:6789/0 conn(0x7efc7815cf60 legacy=0x7efc7806f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:16.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.454+0000 7efc7cf39640 1 -- 172.21.15.84:0/3852785808 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc780c9620 con 0x7efc780c9ab0 2023-10-16T10:07:16.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.454+0000 7efc7df3b640 1 --2- 172.21.15.84:0/3852785808 >> v2:172.21.15.157:3300/0 conn(0x7efc780b0360 0x7efc780b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:16.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.455+0000 7efc7cf39640 1 --2- 172.21.15.84:0/3852785808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc780c9ab0 0x7efc780c9f40 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7efc680097a0 tx=0x7efc6802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cfef18a975f6d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:16.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.455+0000 7efc6ffff640 1 -- 172.21.15.84:0/3852785808 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc68031830 con 0x7efc780c9ab0 2023-10-16T10:07:16.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.455+0000 7efc6ffff640 1 -- 172.21.15.84:0/3852785808 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc68031990 con 0x7efc780c9ab0 2023-10-16T10:07:16.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.455+0000 7efc6ffff640 1 -- 172.21.15.84:0/3852785808 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc68031d20 con 0x7efc780c9ab0 2023-10-16T10:07:16.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.456+0000 7efc7f99e640 1 -- 172.21.15.84:0/3852785808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc780c9ab0 msgr2=0x7efc780c9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:16.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.456+0000 7efc7f99e640 1 --2- 172.21.15.84:0/3852785808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc780c9ab0 0x7efc780c9f40 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7efc680097a0 tx=0x7efc6802b8d0 comp rx=0 tx=0).stop 2023-10-16T10:07:16.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.456+0000 7efc7f99e640 1 -- 172.21.15.84:0/3852785808 shutdown_connections 2023-10-16T10:07:16.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.456+0000 7efc7f99e640 1 --2- 172.21.15.84:0/3852785808 >> v2:172.21.15.157:3300/0 conn(0x7efc780b0360 0x7efc780b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:16.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.456+0000 7efc7f99e640 1 --2- 172.21.15.84:0/3852785808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc780c9ab0 0x7efc780c9f40 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:16.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.456+0000 7efc7f99e640 1 -- 172.21.15.84:0/3852785808 >> 172.21.15.84:0/3852785808 conn(0x7efc780cb360 msgr2=0x7efc780cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:16.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.456+0000 7efc7f99e640 1 -- 172.21.15.84:0/3852785808 shutdown_connections 2023-10-16T10:07:16.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.456+0000 7efc7f99e640 1 -- 172.21.15.84:0/3852785808 wait complete. 2023-10-16T10:07:16.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.458+0000 7efc7f99e640 1 Processor -- start 2023-10-16T10:07:16.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.458+0000 7efc7f99e640 1 -- start start 2023-10-16T10:07:16.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.459+0000 7efc7f99e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc780b0360 0x7efc78151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:16.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.459+0000 7efc7f99e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc780c9ab0 0x7efc781571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:16.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.459+0000 7efc7f99e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc7815cf60 0x7efc780b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:16.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.459+0000 7efc7f99e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc78156d50 con 0x7efc780b0360 2023-10-16T10:07:16.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.459+0000 7efc7f99e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc78156140 con 0x7efc780c9ab0 2023-10-16T10:07:16.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.459+0000 7efc7f99e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc78156420 con 0x7efc7815cf60 2023-10-16T10:07:16.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.459+0000 7efc7d73a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc780b0360 0x7efc78151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:16.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.459+0000 7efc7d73a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc780b0360 0x7efc78151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59480/0 (socket says 172.21.15.84:59480) 2023-10-16T10:07:16.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.459+0000 7efc7d73a640 1 -- 172.21.15.84:0/1578459677 learned_addr learned my addr 172.21.15.84:0/1578459677 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:16.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.460+0000 7efc7d73a640 1 -- 172.21.15.84:0/1578459677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc7815cf60 msgr2=0x7efc780b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:16.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.460+0000 7efc7d73a640 1 --2- 172.21.15.84:0/1578459677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc7815cf60 0x7efc780b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:16.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.460+0000 7efc7d73a640 1 -- 172.21.15.84:0/1578459677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc780c9ab0 msgr2=0x7efc781571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:16.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.460+0000 7efc7cf39640 1 --2- 172.21.15.84:0/1578459677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc780c9ab0 0x7efc781571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:16.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.460+0000 7efc7d73a640 1 --2- 172.21.15.84:0/1578459677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc780c9ab0 0x7efc781571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:16.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.460+0000 7efc7d73a640 1 -- 172.21.15.84:0/1578459677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc68009400 con 0x7efc780b0360 2023-10-16T10:07:16.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.460+0000 7efc7d73a640 1 --2- 172.21.15.84:0/1578459677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc780b0360 0x7efc78151150 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7efc7400c700 tx=0x7efc7400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:16.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.460+0000 7efc6dffb640 1 -- 172.21.15.84:0/1578459677 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74005ce0 con 0x7efc780b0360 2023-10-16T10:07:16.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.460+0000 7efc6dffb640 1 -- 172.21.15.84:0/1578459677 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc74005e40 con 0x7efc780b0360 2023-10-16T10:07:16.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.460+0000 7efc7f99e640 1 -- 172.21.15.84:0/1578459677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc78156700 con 0x7efc780b0360 2023-10-16T10:07:16.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.460+0000 7efc6dffb640 1 -- 172.21.15.84:0/1578459677 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74007290 con 0x7efc780b0360 2023-10-16T10:07:16.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.461+0000 7efc7f99e640 1 -- 172.21.15.84:0/1578459677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc780c76c0 con 0x7efc780b0360 2023-10-16T10:07:16.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.461+0000 7efc7f99e640 1 -- 172.21.15.84:0/1578459677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc7804e160 con 0x7efc780b0360 2023-10-16T10:07:16.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.463+0000 7efc6dffb640 1 -- 172.21.15.84:0/1578459677 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efc740074b0 con 0x7efc780b0360 2023-10-16T10:07:16.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.463+0000 7efc6dffb640 1 --2- 172.21.15.84:0/1578459677 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7efc5c06af90 0x7efc5c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:16.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.464+0000 7efc6dffb640 1 -- 172.21.15.84:0/1578459677 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6598+0+0 (secure 0 0 0) 0x7efc74088fa0 con 0x7efc780b0360 2023-10-16T10:07:16.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.464+0000 7efc7cf39640 1 --2- 172.21.15.84:0/1578459677 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7efc5c06af90 0x7efc5c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:16.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.464+0000 7efc7cf39640 1 --2- 172.21.15.84:0/1578459677 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7efc5c06af90 0x7efc5c06d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efc68036000 tx=0x7efc6803c040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:16.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.468+0000 7efc6dffb640 1 -- 172.21.15.84:0/1578459677 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efc740bbfe0 con 0x7efc780b0360 2023-10-16T10:07:16.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:16.662+0000 7efc7f99e640 1 -- 172.21.15.84:0/1578459677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efc780b0cf0 con 0x7efc780b0360 2023-10-16T10:07:17.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.229+0000 7efc6dffb640 1 -- 172.21.15.84:0/1578459677 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v115) v1 ==== 181+0+0 (secure 0 0 0) 0x7efc780b0cf0 con 0x7efc780b0360 2023-10-16T10:07:17.230 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-16T10:07:17.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.233+0000 7efc7f99e640 1 -- 172.21.15.84:0/1578459677 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7efc5c06af90 msgr2=0x7efc5c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:17.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.233+0000 7efc7f99e640 1 --2- 172.21.15.84:0/1578459677 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7efc5c06af90 0x7efc5c06d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efc68036000 tx=0x7efc6803c040 comp rx=0 tx=0).stop 2023-10-16T10:07:17.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.233+0000 7efc7f99e640 1 -- 172.21.15.84:0/1578459677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc780b0360 msgr2=0x7efc78151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:17.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.233+0000 7efc7f99e640 1 --2- 172.21.15.84:0/1578459677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc780b0360 0x7efc78151150 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7efc7400c700 tx=0x7efc7400cb00 comp rx=0 tx=0).stop 2023-10-16T10:07:17.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.234+0000 7efc7f99e640 1 -- 172.21.15.84:0/1578459677 shutdown_connections 2023-10-16T10:07:17.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.234+0000 7efc7f99e640 1 --2- 172.21.15.84:0/1578459677 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7efc5c06af90 0x7efc5c06d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:17.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.234+0000 7efc7f99e640 1 --2- 172.21.15.84:0/1578459677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc7815cf60 0x7efc780b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:17.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.234+0000 7efc7f99e640 1 --2- 172.21.15.84:0/1578459677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc780c9ab0 0x7efc781571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:17.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.234+0000 7efc7f99e640 1 --2- 172.21.15.84:0/1578459677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc780b0360 0x7efc78151150 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:17.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.234+0000 7efc7f99e640 1 -- 172.21.15.84:0/1578459677 >> 172.21.15.84:0/1578459677 conn(0x7efc780cb360 msgr2=0x7efc780ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:17.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.235+0000 7efc7f99e640 1 -- 172.21.15.84:0/1578459677 shutdown_connections 2023-10-16T10:07:17.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.236+0000 7efc7f99e640 1 -- 172.21.15.84:0/1578459677 wait complete. 2023-10-16T10:07:17.261 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T10:07:17.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.489+0000 7fe86d6a2640 1 Processor -- start 2023-10-16T10:07:17.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.489+0000 7fe86d6a2640 1 -- start start 2023-10-16T10:07:17.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.491+0000 7fe86d6a2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe86815ac10 0x7fe86806c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:17.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.491+0000 7fe86d6a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8680cac20 0x7fe8680b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:17.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.491+0000 7fe86d6a2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe86815b310 con 0x7fe8680b0360 2023-10-16T10:07:17.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.491+0000 7fe86d6a2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe86815f800 con 0x7fe86815ac10 2023-10-16T10:07:17.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.491+0000 7fe86d6a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8680b0d30 con 0x7fe8680cac20 2023-10-16T10:07:17.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.491+0000 7fe866ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe86815ac10 0x7fe86806c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:17.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.491+0000 7fe866ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe86815ac10 0x7fe86806c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59494/0 (socket says 172.21.15.84:59494) 2023-10-16T10:07:17.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.491+0000 7fe866ffd640 1 -- 172.21.15.84:0/806519664 learned_addr learned my addr 172.21.15.84:0/806519664 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:17.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.492+0000 7fe8677fe640 1 --2- 172.21.15.84:0/806519664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8680cac20 0x7fe8680b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:17.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.492+0000 7fe866ffd640 1 -- 172.21.15.84:0/806519664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8680cac20 msgr2=0x7fe8680b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:17.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.492+0000 7fe866ffd640 1 --2- 172.21.15.84:0/806519664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8680cac20 0x7fe8680b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:17.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.492+0000 7fe866ffd640 1 -- 172.21.15.84:0/806519664 >> v1:172.21.15.157:6789/0 conn(0x7fe8680b0360 legacy=0x7fe8680ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:17.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.492+0000 7fe866ffd640 1 -- 172.21.15.84:0/806519664 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86806f640 con 0x7fe86815ac10 2023-10-16T10:07:17.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.493+0000 7fe866ffd640 1 --2- 172.21.15.84:0/806519664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe86815ac10 0x7fe86806c730 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fe8580098a0 tx=0x7fe85802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1f89afe496a137 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:17.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.493+0000 7fe8667fc640 1 -- 172.21.15.84:0/806519664 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8580317c0 con 0x7fe86815ac10 2023-10-16T10:07:17.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.493+0000 7fe8667fc640 1 -- 172.21.15.84:0/806519664 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe858031920 con 0x7fe86815ac10 2023-10-16T10:07:17.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.493+0000 7fe8667fc640 1 -- 172.21.15.84:0/806519664 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858031cb0 con 0x7fe86815ac10 2023-10-16T10:07:17.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.494+0000 7fe86d6a2640 1 -- 172.21.15.84:0/806519664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe86815ac10 msgr2=0x7fe86806c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:17.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.494+0000 7fe86d6a2640 1 --2- 172.21.15.84:0/806519664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe86815ac10 0x7fe86806c730 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fe8580098a0 tx=0x7fe85802b860 comp rx=0 tx=0).stop 2023-10-16T10:07:17.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.494+0000 7fe86d6a2640 1 -- 172.21.15.84:0/806519664 shutdown_connections 2023-10-16T10:07:17.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.494+0000 7fe86d6a2640 1 --2- 172.21.15.84:0/806519664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8680cac20 0x7fe8680b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:17.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.494+0000 7fe86d6a2640 1 --2- 172.21.15.84:0/806519664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe86815ac10 0x7fe86806c730 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:17.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.494+0000 7fe86d6a2640 1 -- 172.21.15.84:0/806519664 >> 172.21.15.84:0/806519664 conn(0x7fe8680cb360 msgr2=0x7fe8680cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:17.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.494+0000 7fe86d6a2640 1 -- 172.21.15.84:0/806519664 shutdown_connections 2023-10-16T10:07:17.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.494+0000 7fe86d6a2640 1 -- 172.21.15.84:0/806519664 wait complete. 2023-10-16T10:07:17.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.496+0000 7fe86d6a2640 1 Processor -- start 2023-10-16T10:07:17.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.496+0000 7fe86d6a2640 1 -- start start 2023-10-16T10:07:17.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.497+0000 7fe86d6a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8680b0360 0x7fe868155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:17.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.497+0000 7fe86d6a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8680cac20 0x7fe8681f0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:17.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.497+0000 7fe86d6a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe86815ac10 0x7fe868153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:17.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.497+0000 7fe86d6a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8681f05c0 con 0x7fe86815ac10 2023-10-16T10:07:17.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.497+0000 7fe86d6a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe868154450 con 0x7fe8680b0360 2023-10-16T10:07:17.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.497+0000 7fe86d6a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8681f3ca0 con 0x7fe8680cac20 2023-10-16T10:07:17.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.497+0000 7fe8677fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe86815ac10 0x7fe868153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:17.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.497+0000 7fe8677fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe86815ac10 0x7fe868153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59496/0 (socket says 172.21.15.84:59496) 2023-10-16T10:07:17.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.497+0000 7fe8677fe640 1 -- 172.21.15.84:0/2818477131 learned_addr learned my addr 172.21.15.84:0/2818477131 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:17.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.498+0000 7fe866ffd640 1 --2- 172.21.15.84:0/2818477131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8680b0360 0x7fe868155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:17.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.498+0000 7fe85ffff640 1 --2- 172.21.15.84:0/2818477131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8680cac20 0x7fe8681f0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:17.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.498+0000 7fe8677fe640 1 -- 172.21.15.84:0/2818477131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8680cac20 msgr2=0x7fe8681f0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:17.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.498+0000 7fe8677fe640 1 --2- 172.21.15.84:0/2818477131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8680cac20 0x7fe8681f0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:17.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.498+0000 7fe8677fe640 1 -- 172.21.15.84:0/2818477131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8680b0360 msgr2=0x7fe868155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:17.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.498+0000 7fe8677fe640 1 --2- 172.21.15.84:0/2818477131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8680b0360 0x7fe868155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:17.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.498+0000 7fe8677fe640 1 -- 172.21.15.84:0/2818477131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8580094d0 con 0x7fe86815ac10 2023-10-16T10:07:17.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.498+0000 7fe866ffd640 1 --2- 172.21.15.84:0/2818477131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8680b0360 0x7fe868155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:17.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.498+0000 7fe8677fe640 1 --2- 172.21.15.84:0/2818477131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe86815ac10 0x7fe868153ce0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fe860007e60 tx=0x7fe86000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:17.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.498+0000 7fe85f7fe640 1 -- 172.21.15.84:0/2818477131 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860014420 con 0x7fe86815ac10 2023-10-16T10:07:17.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.498+0000 7fe85f7fe640 1 -- 172.21.15.84:0/2818477131 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8600145a0 con 0x7fe86815ac10 2023-10-16T10:07:17.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.498+0000 7fe86d6a2640 1 -- 172.21.15.84:0/2818477131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe868154220 con 0x7fe86815ac10 2023-10-16T10:07:17.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.498+0000 7fe85f7fe640 1 -- 172.21.15.84:0/2818477131 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860014950 con 0x7fe86815ac10 2023-10-16T10:07:17.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.499+0000 7fe86d6a2640 1 -- 172.21.15.84:0/2818477131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8681f47e0 con 0x7fe86815ac10 2023-10-16T10:07:17.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.500+0000 7fe86d6a2640 1 -- 172.21.15.84:0/2818477131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8681f4970 con 0x7fe86815ac10 2023-10-16T10:07:17.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.501+0000 7fe85f7fe640 1 -- 172.21.15.84:0/2818477131 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe860014ab0 con 0x7fe86815ac10 2023-10-16T10:07:17.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.502+0000 7fe85f7fe640 1 --2- 172.21.15.84:0/2818477131 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fe84806af90 0x7fe84806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:17.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.502+0000 7fe85f7fe640 1 -- 172.21.15.84:0/2818477131 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6160+0+0 (secure 0 0 0) 0x7fe86008a250 con 0x7fe86815ac10 2023-10-16T10:07:17.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.502+0000 7fe866ffd640 1 --2- 172.21.15.84:0/2818477131 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fe84806af90 0x7fe84806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:17.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.503+0000 7fe866ffd640 1 --2- 172.21.15.84:0/2818477131 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fe84806af90 0x7fe84806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe858036000 tx=0x7fe85803d040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:17.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.506+0000 7fe85f7fe640 1 -- 172.21.15.84:0/2818477131 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe860055030 con 0x7fe86815ac10 2023-10-16T10:07:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:17.698+0000 7fe86d6a2640 1 -- 172.21.15.84:0/2818477131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe86804e160 con 0x7fe86815ac10 2023-10-16T10:07:18.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.239+0000 7fe85f7fe640 1 -- 172.21.15.84:0/2818477131 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v116) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe860058650 con 0x7fe86815ac10 2023-10-16T10:07:18.240 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-16T10:07:18.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.243+0000 7fe86d6a2640 1 -- 172.21.15.84:0/2818477131 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fe84806af90 msgr2=0x7fe84806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:18.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.243+0000 7fe86d6a2640 1 --2- 172.21.15.84:0/2818477131 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fe84806af90 0x7fe84806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe858036000 tx=0x7fe85803d040 comp rx=0 tx=0).stop 2023-10-16T10:07:18.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.243+0000 7fe86d6a2640 1 -- 172.21.15.84:0/2818477131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe86815ac10 msgr2=0x7fe868153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:18.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.243+0000 7fe86d6a2640 1 --2- 172.21.15.84:0/2818477131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe86815ac10 0x7fe868153ce0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fe860007e60 tx=0x7fe86000e4b0 comp rx=0 tx=0).stop 2023-10-16T10:07:18.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.244+0000 7fe86d6a2640 1 -- 172.21.15.84:0/2818477131 shutdown_connections 2023-10-16T10:07:18.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.244+0000 7fe86d6a2640 1 --2- 172.21.15.84:0/2818477131 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fe84806af90 0x7fe84806d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:18.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.244+0000 7fe86d6a2640 1 --2- 172.21.15.84:0/2818477131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe86815ac10 0x7fe868153ce0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:18.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.244+0000 7fe86d6a2640 1 --2- 172.21.15.84:0/2818477131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8680cac20 0x7fe8681f0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:18.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.244+0000 7fe86d6a2640 1 --2- 172.21.15.84:0/2818477131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8680b0360 0x7fe868155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:18.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.244+0000 7fe86d6a2640 1 -- 172.21.15.84:0/2818477131 >> 172.21.15.84:0/2818477131 conn(0x7fe8680cb360 msgr2=0x7fe86806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:18.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.244+0000 7fe86d6a2640 1 -- 172.21.15.84:0/2818477131 shutdown_connections 2023-10-16T10:07:18.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.244+0000 7fe86d6a2640 1 -- 172.21.15.84:0/2818477131 wait complete. 2023-10-16T10:07:18.272 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T10:07:18.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.469+0000 7f7f9931d640 1 Processor -- start 2023-10-16T10:07:18.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.470+0000 7f7f9931d640 1 -- start start 2023-10-16T10:07:18.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.471+0000 7f7f9931d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7f94159ab0 0x7f7f940ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:18.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.471+0000 7f7f9931d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7f94159e90 0x7f7f9406fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:18.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.471+0000 7f7f9931d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f940cd1f0 con 0x7f7f9406c140 2023-10-16T10:07:18.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.471+0000 7f7f9931d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9415d890 con 0x7f7f94159e90 2023-10-16T10:07:18.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.471+0000 7f7f9931d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f940cd4d0 con 0x7f7f94159ab0 2023-10-16T10:07:18.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.472+0000 7f7f8bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7f94159e90 0x7f7f9406fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:18.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.472+0000 7f7f8bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7f94159e90 0x7f7f9406fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59510/0 (socket says 172.21.15.84:59510) 2023-10-16T10:07:18.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.472+0000 7f7f8bfff640 1 -- 172.21.15.84:0/787354364 learned_addr learned my addr 172.21.15.84:0/787354364 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:18.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.472+0000 7f7f8bfff640 1 -- 172.21.15.84:0/787354364 >> v2:172.21.15.157:3300/0 conn(0x7f7f94159ab0 msgr2=0x7f7f940ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:18.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.472+0000 7f7f8bfff640 1 --2- 172.21.15.84:0/787354364 >> v2:172.21.15.157:3300/0 conn(0x7f7f94159ab0 0x7f7f940ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:18.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.472+0000 7f7f8bfff640 1 -- 172.21.15.84:0/787354364 >> v1:172.21.15.157:6789/0 conn(0x7f7f9406c140 legacy=0x7f7f9415dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:18.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.472+0000 7f7f8bfff640 1 -- 172.21.15.84:0/787354364 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f940cd7b0 con 0x7f7f94159e90 2023-10-16T10:07:18.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.473+0000 7f7f8bfff640 1 --2- 172.21.15.84:0/787354364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f94159e90 0x7f7f9406fc50 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f7f940cfa00 tx=0x7f7f7c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373ec88518c3c81 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:18.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.473+0000 7f7f927fc640 1 -- 172.21.15.84:0/787354364 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c031750 con 0x7f7f94159e90 2023-10-16T10:07:18.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.474+0000 7f7f927fc640 1 -- 172.21.15.84:0/787354364 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7c0318b0 con 0x7f7f94159e90 2023-10-16T10:07:18.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.474+0000 7f7f927fc640 1 -- 172.21.15.84:0/787354364 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c031c40 con 0x7f7f94159e90 2023-10-16T10:07:18.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.474+0000 7f7f9931d640 1 -- 172.21.15.84:0/787354364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f94159e90 msgr2=0x7f7f9406fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:18.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.474+0000 7f7f9931d640 1 --2- 172.21.15.84:0/787354364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f94159e90 0x7f7f9406fc50 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f7f940cfa00 tx=0x7f7f7c02b7f0 comp rx=0 tx=0).stop 2023-10-16T10:07:18.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.474+0000 7f7f9931d640 1 -- 172.21.15.84:0/787354364 shutdown_connections 2023-10-16T10:07:18.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.474+0000 7f7f9931d640 1 --2- 172.21.15.84:0/787354364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f94159e90 0x7f7f9406fc50 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:18.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.474+0000 7f7f9931d640 1 --2- 172.21.15.84:0/787354364 >> v2:172.21.15.157:3300/0 conn(0x7f7f94159ab0 0x7f7f940ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:18.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.474+0000 7f7f9931d640 1 -- 172.21.15.84:0/787354364 >> 172.21.15.84:0/787354364 conn(0x7f7f940ce9b0 msgr2=0x7f7f940cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:18.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.474+0000 7f7f9931d640 1 -- 172.21.15.84:0/787354364 shutdown_connections 2023-10-16T10:07:18.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.474+0000 7f7f9931d640 1 -- 172.21.15.84:0/787354364 wait complete. 2023-10-16T10:07:18.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.477+0000 7f7f9931d640 1 Processor -- start 2023-10-16T10:07:18.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.477+0000 7f7f9931d640 1 -- start start 2023-10-16T10:07:18.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.478+0000 7f7f9931d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f9406c140 0x7f7f940c5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:18.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.478+0000 7f7f9931d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f94159ab0 0x7f7f94156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:18.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.478+0000 7f7f9931d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f94159e90 0x7f7f941e4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:18.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.478+0000 7f7f9931d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f940c4850 con 0x7f7f9406c140 2023-10-16T10:07:18.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.478+0000 7f7f9931d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f940c3bd0 con 0x7f7f94159ab0 2023-10-16T10:07:18.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.478+0000 7f7f9931d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f940c3eb0 con 0x7f7f94159e90 2023-10-16T10:07:18.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.478+0000 7f7f92ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f9406c140 0x7f7f940c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:18.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.479+0000 7f7f92ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f9406c140 0x7f7f940c5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59520/0 (socket says 172.21.15.84:59520) 2023-10-16T10:07:18.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.479+0000 7f7f92ffd640 1 -- 172.21.15.84:0/673044921 learned_addr learned my addr 172.21.15.84:0/673044921 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:18.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.479+0000 7f7f8bfff640 1 --2- 172.21.15.84:0/673044921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f94159ab0 0x7f7f94156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:18.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.479+0000 7f7f937fe640 1 --2- 172.21.15.84:0/673044921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f94159e90 0x7f7f941e4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:18.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.479+0000 7f7f92ffd640 1 -- 172.21.15.84:0/673044921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f94159e90 msgr2=0x7f7f941e4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:18.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.479+0000 7f7f92ffd640 1 --2- 172.21.15.84:0/673044921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f94159e90 0x7f7f941e4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:18.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.479+0000 7f7f92ffd640 1 -- 172.21.15.84:0/673044921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f94159ab0 msgr2=0x7f7f94156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:18.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.479+0000 7f7f92ffd640 1 --2- 172.21.15.84:0/673044921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f94159ab0 0x7f7f94156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:18.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.479+0000 7f7f92ffd640 1 -- 172.21.15.84:0/673044921 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c0094d0 con 0x7f7f9406c140 2023-10-16T10:07:18.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.479+0000 7f7f8bfff640 1 --2- 172.21.15.84:0/673044921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f94159ab0 0x7f7f94156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:18.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.479+0000 7f7f92ffd640 1 --2- 172.21.15.84:0/673044921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f9406c140 0x7f7f940c5190 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400b4a0 tx=0x7f7f8400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:18.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.480+0000 7f7f8b7fe640 1 -- 172.21.15.84:0/673044921 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84010610 con 0x7f7f9406c140 2023-10-16T10:07:18.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.480+0000 7f7f8b7fe640 1 -- 172.21.15.84:0/673044921 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f84010770 con 0x7f7f9406c140 2023-10-16T10:07:18.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.480+0000 7f7f9931d640 1 -- 172.21.15.84:0/673044921 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f940c4190 con 0x7f7f9406c140 2023-10-16T10:07:18.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.480+0000 7f7f8b7fe640 1 -- 172.21.15.84:0/673044921 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84010b20 con 0x7f7f9406c140 2023-10-16T10:07:18.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.480+0000 7f7f9931d640 1 -- 172.21.15.84:0/673044921 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f940c4eb0 con 0x7f7f9406c140 2023-10-16T10:07:18.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.481+0000 7f7f9931d640 1 -- 172.21.15.84:0/673044921 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f941e5c30 con 0x7f7f9406c140 2023-10-16T10:07:18.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.482+0000 7f7f8b7fe640 1 -- 172.21.15.84:0/673044921 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7f84010c80 con 0x7f7f9406c140 2023-10-16T10:07:18.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.483+0000 7f7f8b7fe640 1 --2- 172.21.15.84:0/673044921 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f7f7406af90 0x7f7f7406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:18.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.483+0000 7f7f8b7fe640 1 -- 172.21.15.84:0/673044921 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f7f840894d0 con 0x7f7f9406c140 2023-10-16T10:07:18.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.483+0000 7f7f8bfff640 1 --2- 172.21.15.84:0/673044921 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f7f7406af90 0x7f7f7406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:18.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.484+0000 7f7f8bfff640 1 --2- 172.21.15.84:0/673044921 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f7f7406af90 0x7f7f7406d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c035000 tx=0x7f7f7c008030 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:18.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.489+0000 7f7f8b7fe640 1 -- 172.21.15.84:0/673044921 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7f84054340 con 0x7f7f9406c140 2023-10-16T10:07:18.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.681+0000 7f7f9931d640 1 -- 172.21.15.84:0/673044921 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7f9406b1d0 con 0x7f7f9406c140 2023-10-16T10:07:18.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.686+0000 7f7f8b7fe640 1 -- 172.21.15.84:0/673044921 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7f84057960 con 0x7f7f9406c140 2023-10-16T10:07:18.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.689+0000 7f7f9931d640 1 -- 172.21.15.84:0/673044921 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f7f7406af90 msgr2=0x7f7f7406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:18.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.689+0000 7f7f9931d640 1 --2- 172.21.15.84:0/673044921 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f7f7406af90 0x7f7f7406d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c035000 tx=0x7f7f7c008030 comp rx=0 tx=0).stop 2023-10-16T10:07:18.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.689+0000 7f7f9931d640 1 -- 172.21.15.84:0/673044921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f9406c140 msgr2=0x7f7f940c5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:18.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.689+0000 7f7f9931d640 1 --2- 172.21.15.84:0/673044921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f9406c140 0x7f7f940c5190 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400b4a0 tx=0x7f7f8400b960 comp rx=0 tx=0).stop 2023-10-16T10:07:18.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.689+0000 7f7f9931d640 1 -- 172.21.15.84:0/673044921 shutdown_connections 2023-10-16T10:07:18.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.689+0000 7f7f9931d640 1 --2- 172.21.15.84:0/673044921 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f7f7406af90 0x7f7f7406d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:18.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.689+0000 7f7f9931d640 1 --2- 172.21.15.84:0/673044921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f94159e90 0x7f7f941e4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:18.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.689+0000 7f7f9931d640 1 --2- 172.21.15.84:0/673044921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f94159ab0 0x7f7f94156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:18.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.689+0000 7f7f9931d640 1 --2- 172.21.15.84:0/673044921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f9406c140 0x7f7f940c5190 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:18.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.689+0000 7f7f9931d640 1 -- 172.21.15.84:0/673044921 >> 172.21.15.84:0/673044921 conn(0x7f7f940ce9b0 msgr2=0x7f7f940ae410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:18.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.689+0000 7f7f9931d640 1 -- 172.21.15.84:0/673044921 shutdown_connections 2023-10-16T10:07:18.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.689+0000 7f7f9931d640 1 -- 172.21.15.84:0/673044921 wait complete. 2023-10-16T10:07:18.709 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T10:07:18.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.915+0000 7f1758861640 1 Processor -- start 2023-10-16T10:07:18.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.915+0000 7f1758861640 1 -- start start 2023-10-16T10:07:18.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.917+0000 7f1758861640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1754159ab0 0x7f17540ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:18.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.917+0000 7f1758861640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1754070360 con 0x7f1754159e90 2023-10-16T10:07:18.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.917+0000 7f1758861640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17540cd1f0 con 0x7f175406c140 2023-10-16T10:07:18.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.917+0000 7f1758861640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f175415d890 con 0x7f1754159ab0 2023-10-16T10:07:18.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.917+0000 7f175259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1754159ab0 0x7f17540ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:18.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.917+0000 7f175259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1754159ab0 0x7f17540ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59528/0 (socket says 172.21.15.84:59528) 2023-10-16T10:07:18.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.917+0000 7f175259c640 1 -- 172.21.15.84:0/798757217 learned_addr learned my addr 172.21.15.84:0/798757217 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:18.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.917+0000 7f1749d9b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1754159e90 0x7f175406fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48298/0 (socket says 172.21.15.84:48298) 2023-10-16T10:07:18.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.918+0000 7f175259c640 1 -- 172.21.15.84:0/798757217 >> v1:172.21.15.157:6789/0 conn(0x7f175406c140 legacy=0x7f175415dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:18.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.918+0000 7f175259c640 1 -- 172.21.15.84:0/798757217 >> v1:172.21.15.84:6789/0 conn(0x7f1754159e90 legacy=0x7f175406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:18.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.918+0000 7f175259c640 1 -- 172.21.15.84:0/798757217 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17540cd4d0 con 0x7f1754159ab0 2023-10-16T10:07:18.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.919+0000 7f175259c640 1 --2- 172.21.15.84:0/798757217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1754159ab0 0x7f17540ce370 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f1744009870 tx=0x7f174402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5df02d791175b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:18.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.919+0000 7f1751d9b640 1 -- 172.21.15.84:0/798757217 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17440317e0 con 0x7f1754159ab0 2023-10-16T10:07:18.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.919+0000 7f1751d9b640 1 -- 172.21.15.84:0/798757217 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1744031940 con 0x7f1754159ab0 2023-10-16T10:07:18.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.919+0000 7f1751d9b640 1 -- 172.21.15.84:0/798757217 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1744031cd0 con 0x7f1754159ab0 2023-10-16T10:07:18.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.920+0000 7f1758861640 1 -- 172.21.15.84:0/798757217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1754159ab0 msgr2=0x7f17540ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:18.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.920+0000 7f1758861640 1 --2- 172.21.15.84:0/798757217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1754159ab0 0x7f17540ce370 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f1744009870 tx=0x7f174402b880 comp rx=0 tx=0).stop 2023-10-16T10:07:18.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.920+0000 7f1758861640 1 -- 172.21.15.84:0/798757217 shutdown_connections 2023-10-16T10:07:18.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.920+0000 7f1758861640 1 --2- 172.21.15.84:0/798757217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1754159ab0 0x7f17540ce370 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:18.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.920+0000 7f1758861640 1 -- 172.21.15.84:0/798757217 >> 172.21.15.84:0/798757217 conn(0x7f17540ce9b0 msgr2=0x7f17540cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:18.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.920+0000 7f1758861640 1 -- 172.21.15.84:0/798757217 shutdown_connections 2023-10-16T10:07:18.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.920+0000 7f1758861640 1 -- 172.21.15.84:0/798757217 wait complete. 2023-10-16T10:07:18.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.921+0000 7f1758861640 1 Processor -- start 2023-10-16T10:07:18.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.922+0000 7f1758861640 1 -- start start 2023-10-16T10:07:18.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.922+0000 7f1758861640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f175406c140 0x7f17541f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:18.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.922+0000 7f1758861640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1754159ab0 0x7f1754157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:18.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.922+0000 7f1758861640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1754159e90 0x7f175415ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:18.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.923+0000 7f1758861640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17541f1520 con 0x7f1754159ab0 2023-10-16T10:07:18.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.923+0000 7f1758861640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1754157d60 con 0x7f175406c140 2023-10-16T10:07:18.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.923+0000 7f1758861640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17541f09a0 con 0x7f1754159e90 2023-10-16T10:07:18.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.923+0000 7f1749d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1754159ab0 0x7f1754157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:18.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.923+0000 7f1749d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1754159ab0 0x7f1754157820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59540/0 (socket says 172.21.15.84:59540) 2023-10-16T10:07:18.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.923+0000 7f1749d9b640 1 -- 172.21.15.84:0/3018033252 learned_addr learned my addr 172.21.15.84:0/3018033252 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:18.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.923+0000 7f175259c640 1 --2- 172.21.15.84:0/3018033252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f175406c140 0x7f17541f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:18.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.923+0000 7f1752d9d640 1 --2- 172.21.15.84:0/3018033252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1754159e90 0x7f175415ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:18.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.923+0000 7f1749d9b640 1 -- 172.21.15.84:0/3018033252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1754159e90 msgr2=0x7f175415ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:18.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.923+0000 7f1749d9b640 1 --2- 172.21.15.84:0/3018033252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1754159e90 0x7f175415ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:18.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.923+0000 7f1749d9b640 1 -- 172.21.15.84:0/3018033252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f175406c140 msgr2=0x7f17541f1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:18.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.923+0000 7f1749d9b640 1 --2- 172.21.15.84:0/3018033252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f175406c140 0x7f17541f1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:18.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.923+0000 7f1749d9b640 1 -- 172.21.15.84:0/3018033252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17440094d0 con 0x7f1754159ab0 2023-10-16T10:07:18.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.923+0000 7f175259c640 1 --2- 172.21.15.84:0/3018033252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f175406c140 0x7f17541f1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:18.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.924+0000 7f1749d9b640 1 --2- 172.21.15.84:0/3018033252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1754159ab0 0x7f1754157820 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f173c00c4d0 tx=0x7f173c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:18.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.924+0000 7f174b7fe640 1 -- 172.21.15.84:0/3018033252 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c007480 con 0x7f1754159ab0 2023-10-16T10:07:18.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.924+0000 7f174b7fe640 1 -- 172.21.15.84:0/3018033252 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173c0075e0 con 0x7f1754159ab0 2023-10-16T10:07:18.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.924+0000 7f174b7fe640 1 -- 172.21.15.84:0/3018033252 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c0049e0 con 0x7f1754159ab0 2023-10-16T10:07:18.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.924+0000 7f1758861640 1 -- 172.21.15.84:0/3018033252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17541f0c80 con 0x7f1754159ab0 2023-10-16T10:07:18.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.924+0000 7f1758861640 1 -- 172.21.15.84:0/3018033252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17541f1240 con 0x7f1754159ab0 2023-10-16T10:07:18.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.925+0000 7f1758861640 1 -- 172.21.15.84:0/3018033252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175404e160 con 0x7f1754159ab0 2023-10-16T10:07:18.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.926+0000 7f174b7fe640 1 -- 172.21.15.84:0/3018033252 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f173c004b40 con 0x7f1754159ab0 2023-10-16T10:07:18.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.927+0000 7f174b7fe640 1 --2- 172.21.15.84:0/3018033252 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f173406af90 0x7f173406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:18.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.927+0000 7f174b7fe640 1 -- 172.21.15.84:0/3018033252 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f173c088830 con 0x7f1754159ab0 2023-10-16T10:07:18.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.927+0000 7f175259c640 1 --2- 172.21.15.84:0/3018033252 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f173406af90 0x7f173406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:18.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.928+0000 7f175259c640 1 --2- 172.21.15.84:0/3018033252 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f173406af90 0x7f173406d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1744036000 tx=0x7f174403d040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:18.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:18.932+0000 7f174b7fe640 1 -- 172.21.15.84:0/3018033252 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f173c053860 con 0x7f1754159ab0 2023-10-16T10:07:19.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.121+0000 7f1758861640 1 -- 172.21.15.84:0/3018033252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f17540c2570 con 0x7f1754159ab0 2023-10-16T10:07:19.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.127+0000 7f174b7fe640 1 -- 172.21.15.84:0/3018033252 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v38) v1 ==== 120+0+0 (secure 0 0 0) 0x7f173c056e80 con 0x7f1754159ab0 2023-10-16T10:07:19.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.130+0000 7f1758861640 1 -- 172.21.15.84:0/3018033252 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f173406af90 msgr2=0x7f173406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:19.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.130+0000 7f1758861640 1 --2- 172.21.15.84:0/3018033252 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f173406af90 0x7f173406d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1744036000 tx=0x7f174403d040 comp rx=0 tx=0).stop 2023-10-16T10:07:19.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.130+0000 7f1758861640 1 -- 172.21.15.84:0/3018033252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1754159ab0 msgr2=0x7f1754157820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:19.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.130+0000 7f1758861640 1 --2- 172.21.15.84:0/3018033252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1754159ab0 0x7f1754157820 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f173c00c4d0 tx=0x7f173c00c990 comp rx=0 tx=0).stop 2023-10-16T10:07:19.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.130+0000 7f1758861640 1 -- 172.21.15.84:0/3018033252 shutdown_connections 2023-10-16T10:07:19.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.130+0000 7f1758861640 1 --2- 172.21.15.84:0/3018033252 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f173406af90 0x7f173406d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.130+0000 7f1758861640 1 --2- 172.21.15.84:0/3018033252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1754159e90 0x7f175415ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.130+0000 7f1758861640 1 --2- 172.21.15.84:0/3018033252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1754159ab0 0x7f1754157820 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.130+0000 7f1758861640 1 --2- 172.21.15.84:0/3018033252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f175406c140 0x7f17541f1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.130+0000 7f1758861640 1 -- 172.21.15.84:0/3018033252 >> 172.21.15.84:0/3018033252 conn(0x7f17540ce9b0 msgr2=0x7f1754070dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:19.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.130+0000 7f1758861640 1 -- 172.21.15.84:0/3018033252 shutdown_connections 2023-10-16T10:07:19.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.130+0000 7f1758861640 1 -- 172.21.15.84:0/3018033252 wait complete. 2023-10-16T10:07:19.149 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T10:07:19.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.353+0000 7f34011b3640 1 Processor -- start 2023-10-16T10:07:19.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.354+0000 7f34011b3640 1 -- start start 2023-10-16T10:07:19.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.355+0000 7f34011b3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f33fc0b3ec0 0x7f33fc0cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:19.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.355+0000 7f34011b3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33fc0b0ea0 0x7f33fc0cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:19.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.355+0000 7f34011b3640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f33fc0b43c0 0x7f33fc15a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:19.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.355+0000 7f34011b3640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33fc06ca00 con 0x7f33fc0b43c0 2023-10-16T10:07:19.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.355+0000 7f34011b3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33fc06be30 con 0x7f33fc0b3ec0 2023-10-16T10:07:19.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.355+0000 7f34011b3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc0ada10 con 0x7f33fc0b0ea0 2023-10-16T10:07:19.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.355+0000 7f34009b2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f33fc0b43c0 0x7f33fc15a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:19.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.355+0000 7f34009b2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f33fc0b43c0 0x7f33fc15a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59544/0 (socket says 172.21.15.84:59544) 2023-10-16T10:07:19.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.355+0000 7f34009b2640 1 -- 172.21.15.84:0/3331472229 learned_addr learned my addr 172.21.15.84:0/3331472229 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:19.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.356+0000 7f34009b2640 1 -- 172.21.15.84:0/3331472229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33fc0b0ea0 msgr2=0x7f33fc0cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:19.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.356+0000 7f34009b2640 1 --2- 172.21.15.84:0/3331472229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33fc0b0ea0 0x7f33fc0cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.356+0000 7f34009b2640 1 -- 172.21.15.84:0/3331472229 >> v2:172.21.15.157:3300/0 conn(0x7f33fc0b3ec0 msgr2=0x7f33fc0cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:19.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.356+0000 7f34009b2640 1 --2- 172.21.15.84:0/3331472229 >> v2:172.21.15.157:3300/0 conn(0x7f33fc0b3ec0 0x7f33fc0cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.356+0000 7f34009b2640 1 -- 172.21.15.84:0/3331472229 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc06c060 con 0x7f33fc0b43c0 2023-10-16T10:07:19.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.357+0000 7f34009b2640 1 --2- 172.21.15.84:0/3331472229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33fc0b43c0 0x7f33fc15a2f0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0742a0 tx=0x7f33f402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00c38e924aa701f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:19.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.357+0000 7f33fb7fe640 1 -- 172.21.15.84:0/3331472229 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f402c9f0 con 0x7f33fc0b43c0 2023-10-16T10:07:19.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.357+0000 7f33fb7fe640 1 -- 172.21.15.84:0/3331472229 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f402cb50 con 0x7f33fc0b43c0 2023-10-16T10:07:19.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.357+0000 7f33fb7fe640 1 -- 172.21.15.84:0/3331472229 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f40343f0 con 0x7f33fc0b43c0 2023-10-16T10:07:19.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.358+0000 7f34011b3640 1 -- 172.21.15.84:0/3331472229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33fc0b43c0 msgr2=0x7f33fc15a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:19.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.358+0000 7f34011b3640 1 --2- 172.21.15.84:0/3331472229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33fc0b43c0 0x7f33fc15a2f0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0742a0 tx=0x7f33f402be50 comp rx=0 tx=0).stop 2023-10-16T10:07:19.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.358+0000 7f34011b3640 1 -- 172.21.15.84:0/3331472229 shutdown_connections 2023-10-16T10:07:19.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.358+0000 7f34011b3640 1 --2- 172.21.15.84:0/3331472229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33fc0b43c0 0x7f33fc15a2f0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.358+0000 7f34011b3640 1 --2- 172.21.15.84:0/3331472229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33fc0b0ea0 0x7f33fc0cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.358+0000 7f34011b3640 1 --2- 172.21.15.84:0/3331472229 >> v2:172.21.15.157:3300/0 conn(0x7f33fc0b3ec0 0x7f33fc0cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.358+0000 7f34011b3640 1 -- 172.21.15.84:0/3331472229 >> 172.21.15.84:0/3331472229 conn(0x7f33fc0b0170 msgr2=0x7f33fc0b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:19.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.358+0000 7f34011b3640 1 -- 172.21.15.84:0/3331472229 shutdown_connections 2023-10-16T10:07:19.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.358+0000 7f34011b3640 1 -- 172.21.15.84:0/3331472229 wait complete. 2023-10-16T10:07:19.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.360+0000 7f34011b3640 1 Processor -- start 2023-10-16T10:07:19.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.360+0000 7f34011b3640 1 -- start start 2023-10-16T10:07:19.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.361+0000 7f34011b3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33fc0b0ea0 0x7f33fc157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:19.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.361+0000 7f34011b3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33fc0b3ec0 0x7f33fc0c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:19.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.361+0000 7f34011b3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33fc0b43c0 0x7f33fc0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:19.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.361+0000 7f34011b3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc1f3990 con 0x7f33fc0b0ea0 2023-10-16T10:07:19.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.361+0000 7f34011b3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc06fd40 con 0x7f33fc0b43c0 2023-10-16T10:07:19.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.361+0000 7f34011b3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc06f410 con 0x7f33fc0b3ec0 2023-10-16T10:07:19.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.362+0000 7f33fbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33fc0b0ea0 0x7f33fc157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:19.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.362+0000 7f33fbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33fc0b0ea0 0x7f33fc157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59556/0 (socket says 172.21.15.84:59556) 2023-10-16T10:07:19.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.362+0000 7f33fbfff640 1 -- 172.21.15.84:0/1429562642 learned_addr learned my addr 172.21.15.84:0/1429562642 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:19.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.362+0000 7f33fbfff640 1 -- 172.21.15.84:0/1429562642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33fc0b3ec0 msgr2=0x7f33fc0c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:19.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.362+0000 7f33fbfff640 1 --2- 172.21.15.84:0/1429562642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33fc0b3ec0 0x7f33fc0c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.362+0000 7f33fbfff640 1 -- 172.21.15.84:0/1429562642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33fc0b43c0 msgr2=0x7f33fc0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:19.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.362+0000 7f33fbfff640 1 --2- 172.21.15.84:0/1429562642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33fc0b43c0 0x7f33fc0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.362+0000 7f33fbfff640 1 -- 172.21.15.84:0/1429562642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f4009b30 con 0x7f33fc0b0ea0 2023-10-16T10:07:19.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.362+0000 7f33fbfff640 1 --2- 172.21.15.84:0/1429562642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33fc0b0ea0 0x7f33fc157d40 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00b410 tx=0x7f33ec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:19.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.363+0000 7f33f97fa640 1 -- 172.21.15.84:0/1429562642 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec011830 con 0x7f33fc0b0ea0 2023-10-16T10:07:19.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.363+0000 7f33f97fa640 1 -- 172.21.15.84:0/1429562642 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ec0119b0 con 0x7f33fc0b0ea0 2023-10-16T10:07:19.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.363+0000 7f33f97fa640 1 -- 172.21.15.84:0/1429562642 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec011d60 con 0x7f33fc0b0ea0 2023-10-16T10:07:19.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.363+0000 7f34011b3640 1 -- 172.21.15.84:0/1429562642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33fc06f6f0 con 0x7f33fc0b0ea0 2023-10-16T10:07:19.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.363+0000 7f34011b3640 1 -- 172.21.15.84:0/1429562642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33fc158280 con 0x7f33fc0b0ea0 2023-10-16T10:07:19.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.364+0000 7f34011b3640 1 -- 172.21.15.84:0/1429562642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33fc04e9b0 con 0x7f33fc0b0ea0 2023-10-16T10:07:19.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.365+0000 7f33f97fa640 1 -- 172.21.15.84:0/1429562642 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f33ec01a400 con 0x7f33fc0b0ea0 2023-10-16T10:07:19.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.366+0000 7f33f97fa640 1 --2- 172.21.15.84:0/1429562642 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f33dc06af90 0x7f33dc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:19.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.366+0000 7f33f97fa640 1 -- 172.21.15.84:0/1429562642 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f33ec0899b0 con 0x7f33fc0b0ea0 2023-10-16T10:07:19.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.366+0000 7f33f3fff640 1 --2- 172.21.15.84:0/1429562642 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f33dc06af90 0x7f33dc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:19.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.367+0000 7f33f3fff640 1 --2- 172.21.15.84:0/1429562642 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f33dc06af90 0x7f33dc06d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f33fc1f0570 tx=0x7f33e4006bf0 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:19.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.371+0000 7f33f97fa640 1 -- 172.21.15.84:0/1429562642 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f33ec0548b0 con 0x7f33fc0b0ea0 2023-10-16T10:07:19.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.563+0000 7f34011b3640 1 -- 172.21.15.84:0/1429562642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f33fc06abd0 con 0x7f33fc0b0ea0 2023-10-16T10:07:19.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.569+0000 7f33f97fa640 1 -- 172.21.15.84:0/1429562642 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v39) v1 ==== 115+0+0 (secure 0 0 0) 0x7f33ec015070 con 0x7f33fc0b0ea0 2023-10-16T10:07:19.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.572+0000 7f34011b3640 1 -- 172.21.15.84:0/1429562642 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f33dc06af90 msgr2=0x7f33dc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:19.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.572+0000 7f34011b3640 1 --2- 172.21.15.84:0/1429562642 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f33dc06af90 0x7f33dc06d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f33fc1f0570 tx=0x7f33e4006bf0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.572+0000 7f34011b3640 1 -- 172.21.15.84:0/1429562642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33fc0b0ea0 msgr2=0x7f33fc157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:19.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.572+0000 7f34011b3640 1 --2- 172.21.15.84:0/1429562642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33fc0b0ea0 0x7f33fc157d40 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00b410 tx=0x7f33ec00b8d0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.572+0000 7f34011b3640 1 -- 172.21.15.84:0/1429562642 shutdown_connections 2023-10-16T10:07:19.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.572+0000 7f34011b3640 1 --2- 172.21.15.84:0/1429562642 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f33dc06af90 0x7f33dc06d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.572+0000 7f34011b3640 1 --2- 172.21.15.84:0/1429562642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33fc0b43c0 0x7f33fc0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.572+0000 7f34011b3640 1 --2- 172.21.15.84:0/1429562642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33fc0b3ec0 0x7f33fc0c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.572+0000 7f34011b3640 1 --2- 172.21.15.84:0/1429562642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33fc0b0ea0 0x7f33fc157d40 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.572+0000 7f34011b3640 1 -- 172.21.15.84:0/1429562642 >> 172.21.15.84:0/1429562642 conn(0x7f33fc0b0170 msgr2=0x7f33fc0af780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:19.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.572+0000 7f34011b3640 1 -- 172.21.15.84:0/1429562642 shutdown_connections 2023-10-16T10:07:19.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.572+0000 7f34011b3640 1 -- 172.21.15.84:0/1429562642 wait complete. 2023-10-16T10:07:19.592 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-10-16T10:07:19.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.778+0000 7f494fbdc640 1 Processor -- start 2023-10-16T10:07:19.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.778+0000 7f494fbdc640 1 -- start start 2023-10-16T10:07:19.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.779+0000 7f494fbdc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f49481593b0 0x7f494806f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:19.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.779+0000 7f494fbdc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f49480caa10 0x7f494806fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:19.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.779+0000 7f494fbdc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4948159790 con 0x7f494806ffd0 2023-10-16T10:07:19.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.779+0000 7f494fbdc640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49480c9b20 con 0x7f49480caa10 2023-10-16T10:07:19.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.779+0000 7f494fbdc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49480c9e00 con 0x7f49481593b0 2023-10-16T10:07:19.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.780+0000 7f494d177640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f49480caa10 0x7f494806fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:19.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.780+0000 7f494e179640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f494806ffd0 0x7f494815ceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48304/0 (socket says 172.21.15.84:48304) 2023-10-16T10:07:19.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.780+0000 7f494e179640 1 -- 172.21.15.84:0/3386668406 learned_addr learned my addr 172.21.15.84:0/3386668406 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:19.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.780+0000 7f494d177640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f49480caa10 0x7f494806fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59566/0 (socket says 172.21.15.84:59566) 2023-10-16T10:07:19.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.780+0000 7f494d177640 1 -- 172.21.15.84:0/3386668406 >> v2:172.21.15.157:3300/0 conn(0x7f49481593b0 msgr2=0x7f494806f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:19.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.780+0000 7f494d978640 1 --2- 172.21.15.84:0/3386668406 >> v2:172.21.15.157:3300/0 conn(0x7f49481593b0 0x7f494806f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:19.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.780+0000 7f494d177640 1 --2- 172.21.15.84:0/3386668406 >> v2:172.21.15.157:3300/0 conn(0x7f49481593b0 0x7f494806f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.780+0000 7f494d177640 1 -- 172.21.15.84:0/3386668406 >> v1:172.21.15.84:6789/0 conn(0x7f494806ffd0 legacy=0x7f494815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:19.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.780+0000 7f494c976640 1 -- 172.21.15.84:0/3386668406 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19211583 0 0) 0x7f4948159790 con 0x7f494806ffd0 2023-10-16T10:07:19.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.780+0000 7f494d177640 1 -- 172.21.15.84:0/3386668406 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49480ca0e0 con 0x7f49480caa10 2023-10-16T10:07:19.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.780+0000 7f494d978640 1 --2- 172.21.15.84:0/3386668406 >> v2:172.21.15.157:3300/0 conn(0x7f49481593b0 0x7f494806f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:19.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.781+0000 7f494d177640 1 --2- 172.21.15.84:0/3386668406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49480caa10 0x7f494806fa90 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f49480ce8f0 tx=0x7f493802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87404bf67f776f08 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:19.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.782+0000 7f494c976640 1 -- 172.21.15.84:0/3386668406 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938031710 con 0x7f49480caa10 2023-10-16T10:07:19.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.782+0000 7f494c976640 1 -- 172.21.15.84:0/3386668406 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493802f050 con 0x7f49480caa10 2023-10-16T10:07:19.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.782+0000 7f494c976640 1 -- 172.21.15.84:0/3386668406 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938031a90 con 0x7f49480caa10 2023-10-16T10:07:19.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.782+0000 7f494fbdc640 1 -- 172.21.15.84:0/3386668406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49480caa10 msgr2=0x7f494806fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:19.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.782+0000 7f494fbdc640 1 --2- 172.21.15.84:0/3386668406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49480caa10 0x7f494806fa90 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f49480ce8f0 tx=0x7f493802b7b0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.782+0000 7f494fbdc640 1 -- 172.21.15.84:0/3386668406 shutdown_connections 2023-10-16T10:07:19.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.782+0000 7f494fbdc640 1 --2- 172.21.15.84:0/3386668406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49480caa10 0x7f494806fa90 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.782+0000 7f494fbdc640 1 --2- 172.21.15.84:0/3386668406 >> v2:172.21.15.157:3300/0 conn(0x7f49481593b0 0x7f494806f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.782+0000 7f494fbdc640 1 -- 172.21.15.84:0/3386668406 >> 172.21.15.84:0/3386668406 conn(0x7f49480cb410 msgr2=0x7f49480cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:19.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.783+0000 7f494fbdc640 1 -- 172.21.15.84:0/3386668406 shutdown_connections 2023-10-16T10:07:19.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.783+0000 7f494fbdc640 1 -- 172.21.15.84:0/3386668406 wait complete. 2023-10-16T10:07:19.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.784+0000 7f494fbdc640 1 Processor -- start 2023-10-16T10:07:19.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.784+0000 7f494fbdc640 1 -- start start 2023-10-16T10:07:19.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.785+0000 7f494fbdc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f494806ffd0 0x7f494815b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:19.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.785+0000 7f494fbdc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49480caa10 0x7f4948155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:19.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.785+0000 7f494fbdc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49481593b0 0x7f49480c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:19.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.785+0000 7f494fbdc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49481575b0 con 0x7f49480caa10 2023-10-16T10:07:19.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.785+0000 7f494fbdc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49481569a0 con 0x7f49481593b0 2023-10-16T10:07:19.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.785+0000 7f494fbdc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4948156c80 con 0x7f494806ffd0 2023-10-16T10:07:19.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.785+0000 7f494d177640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49480caa10 0x7f4948155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:19.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.785+0000 7f494d177640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49480caa10 0x7f4948155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59580/0 (socket says 172.21.15.84:59580) 2023-10-16T10:07:19.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.785+0000 7f494d177640 1 -- 172.21.15.84:0/3279886962 learned_addr learned my addr 172.21.15.84:0/3279886962 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:19.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.786+0000 7f494d177640 1 -- 172.21.15.84:0/3279886962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f494806ffd0 msgr2=0x7f494815b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:19.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.786+0000 7f494d177640 1 --2- 172.21.15.84:0/3279886962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f494806ffd0 0x7f494815b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.786+0000 7f494d177640 1 -- 172.21.15.84:0/3279886962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49481593b0 msgr2=0x7f49480c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:19.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.786+0000 7f494d177640 1 --2- 172.21.15.84:0/3279886962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49481593b0 0x7f49480c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:19.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.786+0000 7f494d177640 1 -- 172.21.15.84:0/3279886962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4938009400 con 0x7f49480caa10 2023-10-16T10:07:19.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.786+0000 7f494d177640 1 --2- 172.21.15.84:0/3279886962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49480caa10 0x7f4948155df0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f4938000c00 tx=0x7f49380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:19.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.786+0000 7f493e7fc640 1 -- 172.21.15.84:0/3279886962 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938033a50 con 0x7f49480caa10 2023-10-16T10:07:19.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.786+0000 7f493e7fc640 1 -- 172.21.15.84:0/3279886962 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4938033bb0 con 0x7f49480caa10 2023-10-16T10:07:19.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.786+0000 7f493e7fc640 1 -- 172.21.15.84:0/3279886962 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493803c400 con 0x7f49480caa10 2023-10-16T10:07:19.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.786+0000 7f494fbdc640 1 -- 172.21.15.84:0/3279886962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4948156f60 con 0x7f49480caa10 2023-10-16T10:07:19.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.787+0000 7f494fbdc640 1 -- 172.21.15.84:0/3279886962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494814cad0 con 0x7f49480caa10 2023-10-16T10:07:19.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.788+0000 7f494fbdc640 1 -- 172.21.15.84:0/3279886962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494804e160 con 0x7f49480caa10 2023-10-16T10:07:19.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.788+0000 7f493e7fc640 1 -- 172.21.15.84:0/3279886962 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f493803a040 con 0x7f49480caa10 2023-10-16T10:07:19.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.789+0000 7f493e7fc640 1 --2- 172.21.15.84:0/3279886962 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f492c06afc0 0x7f492c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:19.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.789+0000 7f493e7fc640 1 -- 172.21.15.84:0/3279886962 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f49380ab3c0 con 0x7f49480caa10 2023-10-16T10:07:19.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.790+0000 7f494d978640 1 --2- 172.21.15.84:0/3279886962 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f492c06afc0 0x7f492c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:19.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.790+0000 7f494d978640 1 --2- 172.21.15.84:0/3279886962 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f492c06afc0 0x7f492c06d480 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4944006fd0 tx=0x7f4944008040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:19.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:19.794+0000 7f493e7fc640 1 -- 172.21.15.84:0/3279886962 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f49380761a0 con 0x7f49480caa10 2023-10-16T10:07:20.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.046+0000 7f494fbdc640 1 -- 172.21.15.84:0/3279886962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f49481576f0 con 0x7f49480caa10 2023-10-16T10:07:20.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.252+0000 7f493e7fc640 1 -- 172.21.15.84:0/3279886962 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v693) v1 ==== 183+0+0 (secure 0 0 0) 0x7f49481576f0 con 0x7f49480caa10 2023-10-16T10:07:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.255+0000 7f494fbdc640 1 -- 172.21.15.84:0/3279886962 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f492c06afc0 msgr2=0x7f492c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.255+0000 7f494fbdc640 1 --2- 172.21.15.84:0/3279886962 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f492c06afc0 0x7f492c06d480 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4944006fd0 tx=0x7f4944008040 comp rx=0 tx=0).stop 2023-10-16T10:07:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.255+0000 7f494fbdc640 1 -- 172.21.15.84:0/3279886962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49480caa10 msgr2=0x7f4948155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.255+0000 7f494fbdc640 1 --2- 172.21.15.84:0/3279886962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49480caa10 0x7f4948155df0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f4938000c00 tx=0x7f49380046c0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.255+0000 7f494fbdc640 1 -- 172.21.15.84:0/3279886962 shutdown_connections 2023-10-16T10:07:20.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.255+0000 7f494fbdc640 1 --2- 172.21.15.84:0/3279886962 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f492c06afc0 0x7f492c06d480 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.255+0000 7f494fbdc640 1 --2- 172.21.15.84:0/3279886962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49481593b0 0x7f49480c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.255+0000 7f494fbdc640 1 --2- 172.21.15.84:0/3279886962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49480caa10 0x7f4948155df0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.255+0000 7f494fbdc640 1 --2- 172.21.15.84:0/3279886962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f494806ffd0 0x7f494815b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.255+0000 7f494fbdc640 1 -- 172.21.15.84:0/3279886962 >> 172.21.15.84:0/3279886962 conn(0x7f49480cb410 msgr2=0x7f49480cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:20.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.255+0000 7f494fbdc640 1 -- 172.21.15.84:0/3279886962 shutdown_connections 2023-10-16T10:07:20.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.255+0000 7f494fbdc640 1 -- 172.21.15.84:0/3279886962 wait complete. 2023-10-16T10:07:20.275 INFO:tasks.cephfs_test_runner:Test that a non-existent path leads to schedule deactivation after a few retries. ... ok 2023-10-16T10:07:20.276 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2023-10-16T10:07:20.276 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T10:07:20.277 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:07:20.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.475+0000 7f58e7d48640 1 Processor -- start 2023-10-16T10:07:20.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.475+0000 7f58e7d48640 1 -- start start 2023-10-16T10:07:20.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.477+0000 7f58e7d48640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58e01593b0 0x7f58e006f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:20.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.477+0000 7f58e7d48640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58e00caa10 0x7f58e006fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:20.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.477+0000 7f58e7d48640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58e0159790 con 0x7f58e006ffd0 2023-10-16T10:07:20.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.477+0000 7f58e7d48640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e00c9b20 con 0x7f58e01593b0 2023-10-16T10:07:20.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.477+0000 7f58e7d48640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e00c9e00 con 0x7f58e00caa10 2023-10-16T10:07:20.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.477+0000 7f58e62e5640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f58e006ffd0 0x7f58e015ceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48316/0 (socket says 172.21.15.84:48316) 2023-10-16T10:07:20.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.477+0000 7f58e62e5640 1 -- 172.21.15.84:0/1827895986 learned_addr learned my addr 172.21.15.84:0/1827895986 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:20.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.477+0000 7f58e52e3640 1 --2- 172.21.15.84:0/1827895986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58e00caa10 0x7f58e006fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:20.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.478+0000 7f58e5ae4640 1 --2- 172.21.15.84:0/1827895986 >> v2:172.21.15.157:3300/0 conn(0x7f58e01593b0 0x7f58e006f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:20.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.478+0000 7f58e4ae2640 1 -- 172.21.15.84:0/1827895986 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2253790742 0 0) 0x7f58e0159790 con 0x7f58e006ffd0 2023-10-16T10:07:20.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.478+0000 7f58e4ae2640 1 -- 172.21.15.84:0/1827895986 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58cc003520 con 0x7f58e006ffd0 2023-10-16T10:07:20.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.478+0000 7f58e4ae2640 1 -- 172.21.15.84:0/1827895986 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2620590065 0 0) 0x7f58cc003520 con 0x7f58e006ffd0 2023-10-16T10:07:20.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.478+0000 7f58e4ae2640 1 -- 172.21.15.84:0/1827895986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58e00caa10 msgr2=0x7f58e006fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:20.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.478+0000 7f58e4ae2640 1 --2- 172.21.15.84:0/1827895986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58e00caa10 0x7f58e006fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.478+0000 7f58e4ae2640 1 -- 172.21.15.84:0/1827895986 >> v2:172.21.15.157:3300/0 conn(0x7f58e01593b0 msgr2=0x7f58e006f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:20.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.478+0000 7f58e4ae2640 1 --2- 172.21.15.84:0/1827895986 >> v2:172.21.15.157:3300/0 conn(0x7f58e01593b0 0x7f58e006f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.478+0000 7f58e4ae2640 1 -- 172.21.15.84:0/1827895986 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e00ca0e0 con 0x7f58e006ffd0 2023-10-16T10:07:20.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.478+0000 7f58e4ae2640 1 -- 172.21.15.84:0/1827895986 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f58d80019f0 con 0x7f58e006ffd0 2023-10-16T10:07:20.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.478+0000 7f58e4ae2640 1 -- 172.21.15.84:0/1827895986 >> v1:172.21.15.84:6789/0 conn(0x7f58e006ffd0 legacy=0x7f58e015ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:20.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.478+0000 7f58e4ae2640 1 --2- 172.21.15.84:0/1827895986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58cc003b40 0x7f58cc003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:20.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.478+0000 7f58e4ae2640 1 --2- 172.21.15.84:0/1827895986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58cc008850 0x7f58cc00ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:20.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.478+0000 7f58e4ae2640 1 --2- 172.21.15.84:0/1827895986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58e00caa10 0x7f58cc00d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:20.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.478+0000 7f58e4ae2640 1 -- 172.21.15.84:0/1827895986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e00c9b20 con 0x7f58cc008850 2023-10-16T10:07:20.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.478+0000 7f58e4ae2640 1 -- 172.21.15.84:0/1827895986 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e00c9e00 con 0x7f58cc003b40 2023-10-16T10:07:20.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.478+0000 7f58e4ae2640 1 -- 172.21.15.84:0/1827895986 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc00dc20 con 0x7f58e00caa10 2023-10-16T10:07:20.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.478+0000 7f58e52e3640 1 --2- 172.21.15.84:0/1827895986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58cc008850 0x7f58cc00ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:20.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.479+0000 7f58e5ae4640 1 --2- 172.21.15.84:0/1827895986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58cc003b40 0x7f58cc003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:20.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.479+0000 7f58e52e3640 1 -- 172.21.15.84:0/1827895986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58e00caa10 msgr2=0x7f58cc00d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:20.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.479+0000 7f58e52e3640 1 --2- 172.21.15.84:0/1827895986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58e00caa10 0x7f58cc00d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.479+0000 7f58e52e3640 1 -- 172.21.15.84:0/1827895986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58cc003b40 msgr2=0x7f58cc003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:20.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.479+0000 7f58e52e3640 1 --2- 172.21.15.84:0/1827895986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58cc003b40 0x7f58cc003f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.479+0000 7f58e52e3640 1 -- 172.21.15.84:0/1827895986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58cc003520 con 0x7f58cc008850 2023-10-16T10:07:20.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.479+0000 7f58e5ae4640 1 --2- 172.21.15.84:0/1827895986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58cc003b40 0x7f58cc003f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:20.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.480+0000 7f58e52e3640 1 --2- 172.21.15.84:0/1827895986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58cc008850 0x7f58cc00ace0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009ac0 tx=0x7f58cc0037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18645c89e85db890 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:20.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.480+0000 7f58e4ae2640 1 -- 172.21.15.84:0/1827895986 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d002c710 con 0x7f58cc008850 2023-10-16T10:07:20.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.480+0000 7f58e4ae2640 1 -- 172.21.15.84:0/1827895986 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d002c870 con 0x7f58cc008850 2023-10-16T10:07:20.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.480+0000 7f58e4ae2640 1 -- 172.21.15.84:0/1827895986 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d002cc00 con 0x7f58cc008850 2023-10-16T10:07:20.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.481+0000 7f58e7d48640 1 -- 172.21.15.84:0/1827895986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58cc008850 msgr2=0x7f58cc00ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:20.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.481+0000 7f58e7d48640 1 --2- 172.21.15.84:0/1827895986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58cc008850 0x7f58cc00ace0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009ac0 tx=0x7f58cc0037d0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.481+0000 7f58e62e5640 1 -- 172.21.15.84:0/1827895986 reap_dead start 2023-10-16T10:07:20.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.481+0000 7f58e7d48640 1 -- 172.21.15.84:0/1827895986 shutdown_connections 2023-10-16T10:07:20.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.481+0000 7f58e7d48640 1 -- 172.21.15.84:0/1827895986 >> 172.21.15.84:0/1827895986 conn(0x7f58e00cb410 msgr2=0x7f58e00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:20.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.481+0000 7f58e7d48640 1 -- 172.21.15.84:0/1827895986 shutdown_connections 2023-10-16T10:07:20.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.481+0000 7f58e7d48640 1 -- 172.21.15.84:0/1827895986 wait complete. 2023-10-16T10:07:20.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.482+0000 7f58e7d48640 1 Processor -- start 2023-10-16T10:07:20.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.483+0000 7f58e7d48640 1 -- start start 2023-10-16T10:07:20.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.483+0000 7f58e7d48640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58cc008850 0x7f58e0156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:20.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.483+0000 7f58e7d48640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58e00c4510 0x7f58e015c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:20.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.483+0000 7f58e7d48640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58e00b1530 0x7f58e00b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:20.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.484+0000 7f58e7d48640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e0156bc0 con 0x7f58cc008850 2023-10-16T10:07:20.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.484+0000 7f58e7d48640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e0156ea0 con 0x7f58e00b1530 2023-10-16T10:07:20.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.484+0000 7f58e7d48640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e0157180 con 0x7f58e00c4510 2023-10-16T10:07:20.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.484+0000 7f58e5ae4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58cc008850 0x7f58e0156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:20.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.484+0000 7f58e5ae4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58cc008850 0x7f58e0156590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59596/0 (socket says 172.21.15.84:59596) 2023-10-16T10:07:20.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.484+0000 7f58e5ae4640 1 -- 172.21.15.84:0/1440797349 learned_addr learned my addr 172.21.15.84:0/1440797349 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:20.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.484+0000 7f58e52e3640 1 --2- 172.21.15.84:0/1440797349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58e00c4510 0x7f58e015c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:20.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.484+0000 7f58e62e5640 1 --2- 172.21.15.84:0/1440797349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58e00b1530 0x7f58e00b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:20.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.484+0000 7f58e5ae4640 1 -- 172.21.15.84:0/1440797349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58e00c4510 msgr2=0x7f58e015c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:20.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.484+0000 7f58e5ae4640 1 --2- 172.21.15.84:0/1440797349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58e00c4510 0x7f58e015c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.484+0000 7f58e5ae4640 1 -- 172.21.15.84:0/1440797349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58e00b1530 msgr2=0x7f58e00b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:20.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.484+0000 7f58e5ae4640 1 --2- 172.21.15.84:0/1440797349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58e00b1530 0x7f58e00b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.484+0000 7f58e5ae4640 1 -- 172.21.15.84:0/1440797349 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e0157460 con 0x7f58cc008850 2023-10-16T10:07:20.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.484+0000 7f58e62e5640 1 --2- 172.21.15.84:0/1440797349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58e00b1530 0x7f58e00b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:20.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.485+0000 7f58e5ae4640 1 --2- 172.21.15.84:0/1440797349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58cc008850 0x7f58e0156590 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f58dc010890 tx=0x7f58dc010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:20.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.485+0000 7f58d67fc640 1 -- 172.21.15.84:0/1440797349 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc00eb60 con 0x7f58cc008850 2023-10-16T10:07:20.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.485+0000 7f58d67fc640 1 -- 172.21.15.84:0/1440797349 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc00ecc0 con 0x7f58cc008850 2023-10-16T10:07:20.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.485+0000 7f58d67fc640 1 -- 172.21.15.84:0/1440797349 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc01c400 con 0x7f58cc008850 2023-10-16T10:07:20.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.485+0000 7f58e7d48640 1 -- 172.21.15.84:0/1440797349 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e0157740 con 0x7f58cc008850 2023-10-16T10:07:20.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.485+0000 7f58e7d48640 1 -- 172.21.15.84:0/1440797349 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e00c4990 con 0x7f58cc008850 2023-10-16T10:07:20.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.486+0000 7f58e7d48640 1 -- 172.21.15.84:0/1440797349 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e004e160 con 0x7f58cc008850 2023-10-16T10:07:20.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.487+0000 7f58d67fc640 1 -- 172.21.15.84:0/1440797349 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f58dc0143f0 con 0x7f58cc008850 2023-10-16T10:07:20.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.488+0000 7f58d67fc640 1 --2- 172.21.15.84:0/1440797349 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f58c406af90 0x7f58c406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:20.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.488+0000 7f58d67fc640 1 -- 172.21.15.84:0/1440797349 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f58dc0899e0 con 0x7f58cc008850 2023-10-16T10:07:20.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.488+0000 7f58e52e3640 1 --2- 172.21.15.84:0/1440797349 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f58c406af90 0x7f58c406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:20.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.489+0000 7f58e52e3640 1 --2- 172.21.15.84:0/1440797349 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f58c406af90 0x7f58c406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f58d0036000 tx=0x7f58d0030000 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:20.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.492+0000 7f58d67fc640 1 -- 172.21.15.84:0/1440797349 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f58dc054960 con 0x7f58cc008850 2023-10-16T10:07:20.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.685+0000 7f58e7d48640 1 -- 172.21.15.84:0/1440797349 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58e00b4910 con 0x7f58cc008850 2023-10-16T10:07:20.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.687+0000 7f58d67fc640 1 -- 172.21.15.84:0/1440797349 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+14095 (secure 0 0 0) 0x7f58dc057f80 con 0x7f58cc008850 2023-10-16T10:07:20.688 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:20.689 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":116,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:07:18.229704+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6834/3621445269":"2023-10-17T10:07:15.180223+0000","172.21.15.84:0/2143212776":"2023-10-17T10:07:11.412892+0000","172.21.15.84:6833/985901542":"2023-10-17T10:07:11.412892+0000","172.21.15.84:6832/985901542":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/636650964":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/2411644314":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/2627061388":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/1139885639":"2023-10-17T10:07:11.412892+0000","172.21.15.84:6836/3621445269":"2023-10-17T10:07:15.180223+0000","172.21.15.84:0/1946898486":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/2051153636":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/628187054":"2023-10-17T10:07:11.412892+0000","172.21.15.84:0/965715898":"2023-10-17T10:06:03.602359+0000","172.21.15.84:6833/3250823028":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/3505558848":"2023-10-17T10:07:11.412892+0000","172.21.15.84:6832/3250823028":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/3381824109":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/3871500870":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/1287074955":"2023-10-17T10:06:03.602359+0000","172.21.15.84:0/3830790285":"2023-10-16T11:05:57.166133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:07:20.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.690+0000 7f58e7d48640 1 -- 172.21.15.84:0/1440797349 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f58c406af90 msgr2=0x7f58c406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:20.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.690+0000 7f58e7d48640 1 --2- 172.21.15.84:0/1440797349 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f58c406af90 0x7f58c406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f58d0036000 tx=0x7f58d0030000 comp rx=0 tx=0).stop 2023-10-16T10:07:20.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.690+0000 7f58e7d48640 1 -- 172.21.15.84:0/1440797349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58cc008850 msgr2=0x7f58e0156590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:20.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.690+0000 7f58e7d48640 1 --2- 172.21.15.84:0/1440797349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58cc008850 0x7f58e0156590 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f58dc010890 tx=0x7f58dc010d50 comp rx=0 tx=0).stop 2023-10-16T10:07:20.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.690+0000 7f58e7d48640 1 -- 172.21.15.84:0/1440797349 shutdown_connections 2023-10-16T10:07:20.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.690+0000 7f58e7d48640 1 --2- 172.21.15.84:0/1440797349 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f58c406af90 0x7f58c406d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.690+0000 7f58e7d48640 1 --2- 172.21.15.84:0/1440797349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58e00b1530 0x7f58e00b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.690+0000 7f58e7d48640 1 --2- 172.21.15.84:0/1440797349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58e00c4510 0x7f58e015c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.690+0000 7f58e7d48640 1 --2- 172.21.15.84:0/1440797349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58cc008850 0x7f58e0156590 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.690+0000 7f58e7d48640 1 -- 172.21.15.84:0/1440797349 >> 172.21.15.84:0/1440797349 conn(0x7f58e00cb410 msgr2=0x7f58e00cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:20.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.690+0000 7f58e7d48640 1 -- 172.21.15.84:0/1440797349 shutdown_connections 2023-10-16T10:07:20.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.690+0000 7f58e7d48640 1 -- 172.21.15.84:0/1440797349 wait complete. 2023-10-16T10:07:20.710 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:07:20.710 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:07:20.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.925+0000 7fcc61189640 1 Processor -- start 2023-10-16T10:07:20.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.925+0000 7fcc61189640 1 -- start start 2023-10-16T10:07:20.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.926+0000 7fcc61189640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcc5c0caac0 0x7fcc5c0ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:20.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.927+0000 7fcc61189640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5c0cb030 con 0x7fcc5c15ad70 2023-10-16T10:07:20.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.927+0000 7fcc61189640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5c0c91c0 con 0x7fcc5c06a4b0 2023-10-16T10:07:20.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.927+0000 7fcc61189640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc5c0c94a0 con 0x7fcc5c0caac0 2023-10-16T10:07:20.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.927+0000 7fcc5ad9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fcc5c15ad70 0x7fcc5c069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48328/0 (socket says 172.21.15.84:48328) 2023-10-16T10:07:20.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.927+0000 7fcc5ad9d640 1 -- 172.21.15.84:0/3919179445 learned_addr learned my addr 172.21.15.84:0/3919179445 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:20.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.927+0000 7fcc5b59e640 1 --2- 172.21.15.84:0/3919179445 >> v2:172.21.15.84:3300/0 conn(0x7fcc5c0caac0 0x7fcc5c0ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:20.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.927+0000 7fcc5a59c640 1 -- 172.21.15.84:0/3919179445 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810864210 0 0) 0x7fcc5c0cb030 con 0x7fcc5c15ad70 2023-10-16T10:07:20.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.927+0000 7fcc5a59c640 1 -- 172.21.15.84:0/3919179445 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc48003520 con 0x7fcc5c15ad70 2023-10-16T10:07:20.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.928+0000 7fcc5b59e640 1 -- 172.21.15.84:0/3919179445 >> v1:172.21.15.157:6789/0 conn(0x7fcc5c06a4b0 legacy=0x7fcc5c0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:20.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.928+0000 7fcc5b59e640 1 -- 172.21.15.84:0/3919179445 >> v1:172.21.15.84:6789/0 conn(0x7fcc5c15ad70 legacy=0x7fcc5c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:20.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.928+0000 7fcc5b59e640 1 -- 172.21.15.84:0/3919179445 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c0c9780 con 0x7fcc5c0caac0 2023-10-16T10:07:20.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.929+0000 7fcc5b59e640 1 --2- 172.21.15.84:0/3919179445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc5c0caac0 0x7fcc5c0ad4d0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fcc54009d90 tx=0x7fcc5402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7482a7d089d9c09 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:20.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.929+0000 7fcc5a59c640 1 -- 172.21.15.84:0/3919179445 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5402cd00 con 0x7fcc5c0caac0 2023-10-16T10:07:20.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.929+0000 7fcc5a59c640 1 -- 172.21.15.84:0/3919179445 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5402ce60 con 0x7fcc5c0caac0 2023-10-16T10:07:20.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.929+0000 7fcc5a59c640 1 -- 172.21.15.84:0/3919179445 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc54034660 con 0x7fcc5c0caac0 2023-10-16T10:07:20.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.930+0000 7fcc61189640 1 -- 172.21.15.84:0/3919179445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc5c0caac0 msgr2=0x7fcc5c0ad4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:20.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.930+0000 7fcc61189640 1 --2- 172.21.15.84:0/3919179445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc5c0caac0 0x7fcc5c0ad4d0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fcc54009d90 tx=0x7fcc5402bf50 comp rx=0 tx=0).stop 2023-10-16T10:07:20.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.930+0000 7fcc61189640 1 -- 172.21.15.84:0/3919179445 shutdown_connections 2023-10-16T10:07:20.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.930+0000 7fcc61189640 1 --2- 172.21.15.84:0/3919179445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc5c0caac0 0x7fcc5c0ad4d0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.930+0000 7fcc61189640 1 -- 172.21.15.84:0/3919179445 >> 172.21.15.84:0/3919179445 conn(0x7fcc5c0ce9b0 msgr2=0x7fcc5c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:20.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.930+0000 7fcc61189640 1 -- 172.21.15.84:0/3919179445 shutdown_connections 2023-10-16T10:07:20.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.930+0000 7fcc61189640 1 -- 172.21.15.84:0/3919179445 wait complete. 2023-10-16T10:07:20.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.932+0000 7fcc61189640 1 Processor -- start 2023-10-16T10:07:20.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.932+0000 7fcc61189640 1 -- start start 2023-10-16T10:07:20.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.933+0000 7fcc61189640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc5c06a4b0 0x7fcc5c0cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:20.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.933+0000 7fcc61189640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc5c0caac0 0x7fcc5c1f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:20.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.933+0000 7fcc61189640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc5c15ad70 0x7fcc5c15c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:20.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.933+0000 7fcc61189640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c0ad8d0 con 0x7fcc5c06a4b0 2023-10-16T10:07:20.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.933+0000 7fcc61189640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c0adf30 con 0x7fcc5c15ad70 2023-10-16T10:07:20.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.933+0000 7fcc61189640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c15ce20 con 0x7fcc5c0caac0 2023-10-16T10:07:20.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.933+0000 7fcc5ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc5c06a4b0 0x7fcc5c0cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:20.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.933+0000 7fcc5ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc5c06a4b0 0x7fcc5c0cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59620/0 (socket says 172.21.15.84:59620) 2023-10-16T10:07:20.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.933+0000 7fcc5ad9d640 1 -- 172.21.15.84:0/455508437 learned_addr learned my addr 172.21.15.84:0/455508437 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:20.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.933+0000 7fcc5ad9d640 1 -- 172.21.15.84:0/455508437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc5c0caac0 msgr2=0x7fcc5c1f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:20.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.933+0000 7fcc5ad9d640 1 --2- 172.21.15.84:0/455508437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc5c0caac0 0x7fcc5c1f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.933+0000 7fcc5ad9d640 1 -- 172.21.15.84:0/455508437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc5c15ad70 msgr2=0x7fcc5c15c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:20.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.933+0000 7fcc5ad9d640 1 --2- 172.21.15.84:0/455508437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc5c15ad70 0x7fcc5c15c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:20.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.933+0000 7fcc5ad9d640 1 -- 172.21.15.84:0/455508437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc54009a70 con 0x7fcc5c06a4b0 2023-10-16T10:07:20.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.934+0000 7fcc5ad9d640 1 --2- 172.21.15.84:0/455508437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc5c06a4b0 0x7fcc5c0cc7f0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00c410 tx=0x7fcc4c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:20.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.934+0000 7fcc537fe640 1 -- 172.21.15.84:0/455508437 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c005ce0 con 0x7fcc5c06a4b0 2023-10-16T10:07:20.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.934+0000 7fcc537fe640 1 -- 172.21.15.84:0/455508437 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4c005e40 con 0x7fcc5c06a4b0 2023-10-16T10:07:20.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.934+0000 7fcc537fe640 1 -- 172.21.15.84:0/455508437 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c0072f0 con 0x7fcc5c06a4b0 2023-10-16T10:07:20.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.934+0000 7fcc61189640 1 -- 172.21.15.84:0/455508437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5c0ae1f0 con 0x7fcc5c06a4b0 2023-10-16T10:07:20.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.934+0000 7fcc61189640 1 -- 172.21.15.84:0/455508437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5c0ae7b0 con 0x7fcc5c06a4b0 2023-10-16T10:07:20.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.935+0000 7fcc61189640 1 -- 172.21.15.84:0/455508437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5c04e160 con 0x7fcc5c06a4b0 2023-10-16T10:07:20.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.936+0000 7fcc537fe640 1 -- 172.21.15.84:0/455508437 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcc4c007510 con 0x7fcc5c06a4b0 2023-10-16T10:07:20.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.937+0000 7fcc537fe640 1 --2- 172.21.15.84:0/455508437 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fcc3c06af90 0x7fcc3c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:20.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.937+0000 7fcc537fe640 1 -- 172.21.15.84:0/455508437 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5717+0+0 (secure 0 0 0) 0x7fcc4c088de0 con 0x7fcc5c06a4b0 2023-10-16T10:07:20.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.937+0000 7fcc53fff640 1 --2- 172.21.15.84:0/455508437 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fcc3c06af90 0x7fcc3c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:20.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.938+0000 7fcc53fff640 1 --2- 172.21.15.84:0/455508437 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fcc3c06af90 0x7fcc3c06d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcc44000f60 tx=0x7fcc440079b0 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:20.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:20.941+0000 7fcc537fe640 1 -- 172.21.15.84:0/455508437 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcc4c053ec0 con 0x7fcc5c06a4b0 2023-10-16T10:07:21.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.117+0000 7fcc61189640 1 -- 172.21.15.84:0/455508437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc5c0cdfa0 con 0x7fcc5c06a4b0 2023-10-16T10:07:21.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.118+0000 7fcc537fe640 1 -- 172.21.15.84:0/455508437 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc4c0170a0 con 0x7fcc5c06a4b0 2023-10-16T10:07:21.119 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:07:21.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.120+0000 7fcc61189640 1 -- 172.21.15.84:0/455508437 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fcc3c06af90 msgr2=0x7fcc3c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:21.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.120+0000 7fcc61189640 1 --2- 172.21.15.84:0/455508437 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fcc3c06af90 0x7fcc3c06d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcc44000f60 tx=0x7fcc440079b0 comp rx=0 tx=0).stop 2023-10-16T10:07:21.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.120+0000 7fcc61189640 1 -- 172.21.15.84:0/455508437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc5c06a4b0 msgr2=0x7fcc5c0cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:21.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.120+0000 7fcc61189640 1 --2- 172.21.15.84:0/455508437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc5c06a4b0 0x7fcc5c0cc7f0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00c410 tx=0x7fcc4c00c8d0 comp rx=0 tx=0).stop 2023-10-16T10:07:21.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.120+0000 7fcc61189640 1 -- 172.21.15.84:0/455508437 shutdown_connections 2023-10-16T10:07:21.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.120+0000 7fcc61189640 1 --2- 172.21.15.84:0/455508437 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fcc3c06af90 0x7fcc3c06d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:21.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.120+0000 7fcc61189640 1 --2- 172.21.15.84:0/455508437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc5c15ad70 0x7fcc5c15c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:21.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.120+0000 7fcc61189640 1 --2- 172.21.15.84:0/455508437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc5c0caac0 0x7fcc5c1f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:21.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.120+0000 7fcc61189640 1 --2- 172.21.15.84:0/455508437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc5c06a4b0 0x7fcc5c0cc7f0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:21.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.120+0000 7fcc61189640 1 -- 172.21.15.84:0/455508437 >> 172.21.15.84:0/455508437 conn(0x7fcc5c0ce9b0 msgr2=0x7fcc5c069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:21.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.121+0000 7fcc61189640 1 -- 172.21.15.84:0/455508437 shutdown_connections 2023-10-16T10:07:21.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.121+0000 7fcc61189640 1 -- 172.21.15.84:0/455508437 wait complete. 2023-10-16T10:07:21.137 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-10-16T10:07:21.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.332+0000 7f30623a9640 1 Processor -- start 2023-10-16T10:07:21.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.332+0000 7f30623a9640 1 -- start start 2023-10-16T10:07:21.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.334+0000 7f30623a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f305c0ce390 0x7f305c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:21.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.334+0000 7f30623a9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f305c15ed60 con 0x7f305c15aaa0 2023-10-16T10:07:21.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.334+0000 7f30623a9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f305c0707e0 con 0x7f305c15ae80 2023-10-16T10:07:21.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.334+0000 7f30623a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f305c06b810 con 0x7f305c0ce390 2023-10-16T10:07:21.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.334+0000 7f3060946640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f305c0ce390 0x7f305c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:21.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.334+0000 7f3060946640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f305c0ce390 0x7f305c15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59630/0 (socket says 172.21.15.84:59630) 2023-10-16T10:07:21.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.334+0000 7f3060946640 1 -- 172.21.15.84:0/3551578169 learned_addr learned my addr 172.21.15.84:0/3551578169 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:21.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.335+0000 7f3060946640 1 -- 172.21.15.84:0/3551578169 >> v1:172.21.15.157:6789/0 conn(0x7f305c15ae80 legacy=0x7f305c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:21.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.335+0000 7f3060946640 1 -- 172.21.15.84:0/3551578169 >> v1:172.21.15.84:6789/0 conn(0x7f305c15aaa0 legacy=0x7f305c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:21.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.335+0000 7f3060946640 1 -- 172.21.15.84:0/3551578169 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c15ef90 con 0x7f305c0ce390 2023-10-16T10:07:21.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.336+0000 7f3060946640 1 --2- 172.21.15.84:0/3551578169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f305c0ce390 0x7f305c15e820 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f3054009e50 tx=0x7f305402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf9ab56cedbd921 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:21.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.336+0000 7f305b7fe640 1 -- 172.21.15.84:0/3551578169 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305402cb00 con 0x7f305c0ce390 2023-10-16T10:07:21.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.336+0000 7f305b7fe640 1 -- 172.21.15.84:0/3551578169 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305402cc60 con 0x7f305c0ce390 2023-10-16T10:07:21.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.336+0000 7f305b7fe640 1 -- 172.21.15.84:0/3551578169 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30540343f0 con 0x7f305c0ce390 2023-10-16T10:07:21.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.336+0000 7f30623a9640 1 -- 172.21.15.84:0/3551578169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f305c0ce390 msgr2=0x7f305c15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:21.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.336+0000 7f30623a9640 1 --2- 172.21.15.84:0/3551578169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f305c0ce390 0x7f305c15e820 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f3054009e50 tx=0x7f305402bf60 comp rx=0 tx=0).stop 2023-10-16T10:07:21.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.337+0000 7f30623a9640 1 -- 172.21.15.84:0/3551578169 shutdown_connections 2023-10-16T10:07:21.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.337+0000 7f30623a9640 1 --2- 172.21.15.84:0/3551578169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f305c0ce390 0x7f305c15e820 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:21.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.337+0000 7f30623a9640 1 -- 172.21.15.84:0/3551578169 >> 172.21.15.84:0/3551578169 conn(0x7f305c0cb410 msgr2=0x7f305c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:21.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.337+0000 7f30623a9640 1 -- 172.21.15.84:0/3551578169 shutdown_connections 2023-10-16T10:07:21.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.337+0000 7f30623a9640 1 -- 172.21.15.84:0/3551578169 wait complete. 2023-10-16T10:07:21.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.338+0000 7f30623a9640 1 Processor -- start 2023-10-16T10:07:21.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.338+0000 7f30623a9640 1 -- start start 2023-10-16T10:07:21.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.339+0000 7f30623a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f305c0ce390 0x7f305c14de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:21.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.339+0000 7f30623a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f305c15aaa0 0x7f305c0a8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:21.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.339+0000 7f30623a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f305c15ae80 0x7f305c0a89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:21.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.339+0000 7f30623a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c14e3d0 con 0x7f305c15ae80 2023-10-16T10:07:21.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.339+0000 7f30623a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c15c7c0 con 0x7f305c15aaa0 2023-10-16T10:07:21.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.339+0000 7f30623a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305c15bbb0 con 0x7f305c0ce390 2023-10-16T10:07:21.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.340+0000 7f3060946640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f305c15ae80 0x7f305c0a89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:21.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.340+0000 7f3060946640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f305c15ae80 0x7f305c0a89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59644/0 (socket says 172.21.15.84:59644) 2023-10-16T10:07:21.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.340+0000 7f3060946640 1 -- 172.21.15.84:0/1285067141 learned_addr learned my addr 172.21.15.84:0/1285067141 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:21.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.340+0000 7f3060946640 1 -- 172.21.15.84:0/1285067141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f305c0ce390 msgr2=0x7f305c14de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:21.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.340+0000 7f3060946640 1 --2- 172.21.15.84:0/1285067141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f305c0ce390 0x7f305c14de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:21.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.340+0000 7f3060946640 1 -- 172.21.15.84:0/1285067141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f305c15aaa0 msgr2=0x7f305c0a8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:21.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.340+0000 7f3053fff640 1 --2- 172.21.15.84:0/1285067141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f305c15aaa0 0x7f305c0a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:21.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.340+0000 7f3060946640 1 --2- 172.21.15.84:0/1285067141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f305c15aaa0 0x7f305c0a8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:21.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.340+0000 7f3060946640 1 -- 172.21.15.84:0/1285067141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3054009b30 con 0x7f305c15ae80 2023-10-16T10:07:21.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.340+0000 7f3060946640 1 --2- 172.21.15.84:0/1285067141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f305c15ae80 0x7f305c0a89c0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f3054009e50 tx=0x7f3054004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:21.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.341+0000 7f30597fa640 1 -- 172.21.15.84:0/1285067141 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305403e450 con 0x7f305c15ae80 2023-10-16T10:07:21.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.341+0000 7f30597fa640 1 -- 172.21.15.84:0/1285067141 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305403e5d0 con 0x7f305c15ae80 2023-10-16T10:07:21.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.341+0000 7f30597fa640 1 -- 172.21.15.84:0/1285067141 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305403e980 con 0x7f305c15ae80 2023-10-16T10:07:21.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.341+0000 7f30623a9640 1 -- 172.21.15.84:0/1285067141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305c15be90 con 0x7f305c15ae80 2023-10-16T10:07:21.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.341+0000 7f30623a9640 1 -- 172.21.15.84:0/1285067141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305c15c450 con 0x7f305c15ae80 2023-10-16T10:07:21.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.342+0000 7f30623a9640 1 -- 172.21.15.84:0/1285067141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f305c15c610 con 0x7f305c15ae80 2023-10-16T10:07:21.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.343+0000 7f30597fa640 1 -- 172.21.15.84:0/1285067141 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f305403eae0 con 0x7f305c15ae80 2023-10-16T10:07:21.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.344+0000 7f30597fa640 1 --2- 172.21.15.84:0/1285067141 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f303c06f3a0 0x7f303c071860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:21.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.344+0000 7f30597fa640 1 -- 172.21.15.84:0/1285067141 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f30540ac790 con 0x7f305c15ae80 2023-10-16T10:07:21.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.344+0000 7f305bfff640 1 --2- 172.21.15.84:0/1285067141 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f303c06f3a0 0x7f303c071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:21.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.344+0000 7f305bfff640 1 --2- 172.21.15.84:0/1285067141 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f303c06f3a0 0x7f303c071860 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f304c000f60 tx=0x7f304c0079d0 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:21.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.348+0000 7f30597fa640 1 -- 172.21.15.84:0/1285067141 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3054077710 con 0x7f305c15ae80 2023-10-16T10:07:21.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:21.589+0000 7f30623a9640 1 -- 172.21.15.84:0/1285067141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f305c04e160 con 0x7f305c15ae80 2023-10-16T10:07:22.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.265+0000 7f30597fa640 1 -- 172.21.15.84:0/1285067141 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v695) v1 ==== 151+0+0 (secure 0 0 0) 0x7f305407ad30 con 0x7f305c15ae80 2023-10-16T10:07:22.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.269+0000 7f30623a9640 1 -- 172.21.15.84:0/1285067141 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f303c06f3a0 msgr2=0x7f303c071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:22.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.269+0000 7f30623a9640 1 --2- 172.21.15.84:0/1285067141 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f303c06f3a0 0x7f303c071860 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f304c000f60 tx=0x7f304c0079d0 comp rx=0 tx=0).stop 2023-10-16T10:07:22.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.269+0000 7f30623a9640 1 -- 172.21.15.84:0/1285067141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f305c15ae80 msgr2=0x7f305c0a89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:22.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.269+0000 7f30623a9640 1 --2- 172.21.15.84:0/1285067141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f305c15ae80 0x7f305c0a89c0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f3054009e50 tx=0x7f3054004a40 comp rx=0 tx=0).stop 2023-10-16T10:07:22.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.270+0000 7f30623a9640 1 -- 172.21.15.84:0/1285067141 shutdown_connections 2023-10-16T10:07:22.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.270+0000 7f30623a9640 1 --2- 172.21.15.84:0/1285067141 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f303c06f3a0 0x7f303c071860 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:22.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.270+0000 7f30623a9640 1 --2- 172.21.15.84:0/1285067141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f305c15ae80 0x7f305c0a89c0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:22.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.270+0000 7f30623a9640 1 --2- 172.21.15.84:0/1285067141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f305c15aaa0 0x7f305c0a8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:22.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.270+0000 7f30623a9640 1 --2- 172.21.15.84:0/1285067141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f305c0ce390 0x7f305c14de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:22.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.270+0000 7f30623a9640 1 -- 172.21.15.84:0/1285067141 >> 172.21.15.84:0/1285067141 conn(0x7f305c0cb410 msgr2=0x7f305c06f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:22.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.270+0000 7f30623a9640 1 -- 172.21.15.84:0/1285067141 shutdown_connections 2023-10-16T10:07:22.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.270+0000 7f30623a9640 1 -- 172.21.15.84:0/1285067141 wait complete. 2023-10-16T10:07:22.292 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T10:07:22.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.503+0000 7f9ad6133640 1 Processor -- start 2023-10-16T10:07:22.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.503+0000 7f9ad6133640 1 -- start start 2023-10-16T10:07:22.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.504+0000 7f9ad6133640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ad006c850 0x7f9ad00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:22.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.504+0000 7f9ad6133640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad0070360 con 0x7f9ad0159e90 2023-10-16T10:07:22.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.504+0000 7f9ad6133640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad00cd1f0 con 0x7f9ad0159ab0 2023-10-16T10:07:22.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.504+0000 7f9ad6133640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad015d890 con 0x7f9ad006c850 2023-10-16T10:07:22.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.504+0000 7f9ac6ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9ad0159e90 0x7f9ad006c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48348/0 (socket says 172.21.15.84:48348) 2023-10-16T10:07:22.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.504+0000 7f9ac6ffd640 1 -- 172.21.15.84:0/3968114564 learned_addr learned my addr 172.21.15.84:0/3968114564 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:22.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.505+0000 7f9acffff640 1 --2- 172.21.15.84:0/3968114564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ad006c850 0x7f9ad00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:22.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.505+0000 7f9aceffd640 1 -- 172.21.15.84:0/3968114564 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2993582130 0 0) 0x7f9ad0070360 con 0x7f9ad0159e90 2023-10-16T10:07:22.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.505+0000 7f9aceffd640 1 -- 172.21.15.84:0/3968114564 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9abc003520 con 0x7f9ad0159e90 2023-10-16T10:07:22.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.505+0000 7f9aceffd640 1 -- 172.21.15.84:0/3968114564 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1381264181 0 0) 0x7f9abc003520 con 0x7f9ad0159e90 2023-10-16T10:07:22.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.505+0000 7f9aceffd640 1 -- 172.21.15.84:0/3968114564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ad006c850 msgr2=0x7f9ad00ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:22.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.505+0000 7f9aceffd640 1 --2- 172.21.15.84:0/3968114564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ad006c850 0x7f9ad00ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:22.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.505+0000 7f9aceffd640 1 -- 172.21.15.84:0/3968114564 >> v1:172.21.15.157:6789/0 conn(0x7f9ad0159ab0 legacy=0x7f9ad006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:22.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.505+0000 7f9aceffd640 1 -- 172.21.15.84:0/3968114564 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad00cd4d0 con 0x7f9ad0159e90 2023-10-16T10:07:22.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.505+0000 7f9aceffd640 1 -- 172.21.15.84:0/3968114564 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f9ab8001670 con 0x7f9ad0159e90 2023-10-16T10:07:22.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.505+0000 7f9aceffd640 1 -- 172.21.15.84:0/3968114564 >> v1:172.21.15.84:6789/0 conn(0x7f9ad0159e90 legacy=0x7f9ad006c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:22.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.505+0000 7f9aceffd640 1 --2- 172.21.15.84:0/3968114564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ad006c850 0x7f9abc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:22.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.505+0000 7f9aceffd640 1 --2- 172.21.15.84:0/3968114564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9abc004020 0x7f9abc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:22.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.505+0000 7f9aceffd640 1 --2- 172.21.15.84:0/3968114564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9abc004e20 0x7f9abc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:22.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.505+0000 7f9aceffd640 1 -- 172.21.15.84:0/3968114564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad015d890 con 0x7f9abc004020 2023-10-16T10:07:22.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.505+0000 7f9aceffd640 1 -- 172.21.15.84:0/3968114564 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9abc005640 con 0x7f9abc004e20 2023-10-16T10:07:22.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.505+0000 7f9aceffd640 1 -- 172.21.15.84:0/3968114564 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9abc00bec0 con 0x7f9ad006c850 2023-10-16T10:07:22.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.506+0000 7f9ac6ffd640 1 --2- 172.21.15.84:0/3968114564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9abc004020 0x7f9abc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:22.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.506+0000 7f9acf7fe640 1 --2- 172.21.15.84:0/3968114564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ad006c850 0x7f9abc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:22.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.506+0000 7f9acffff640 1 --2- 172.21.15.84:0/3968114564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9abc004e20 0x7f9abc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:22.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.506+0000 7f9ac6ffd640 1 -- 172.21.15.84:0/3968114564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ad006c850 msgr2=0x7f9abc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:22.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.506+0000 7f9ac6ffd640 1 --2- 172.21.15.84:0/3968114564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ad006c850 0x7f9abc0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:22.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.506+0000 7f9ac6ffd640 1 -- 172.21.15.84:0/3968114564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9abc004e20 msgr2=0x7f9abc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:22.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.506+0000 7f9ac6ffd640 1 --2- 172.21.15.84:0/3968114564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9abc004e20 0x7f9abc00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:22.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.506+0000 7f9ac6ffd640 1 -- 172.21.15.84:0/3968114564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9abc003520 con 0x7f9abc004020 2023-10-16T10:07:22.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.507+0000 7f9ac6ffd640 1 --2- 172.21.15.84:0/3968114564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9abc004020 0x7f9abc0047c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800c0e0 tx=0x7f9abc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4414d2e255f1f50 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:22.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.507+0000 7f9aceffd640 1 -- 172.21.15.84:0/3968114564 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8005ad0 con 0x7f9abc004020 2023-10-16T10:07:22.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.507+0000 7f9aceffd640 1 -- 172.21.15.84:0/3968114564 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab8005c30 con 0x7f9abc004020 2023-10-16T10:07:22.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.507+0000 7f9aceffd640 1 -- 172.21.15.84:0/3968114564 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab802ca30 con 0x7f9abc004020 2023-10-16T10:07:22.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.508+0000 7f9ad6133640 1 -- 172.21.15.84:0/3968114564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9abc004020 msgr2=0x7f9abc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:22.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.508+0000 7f9ad6133640 1 --2- 172.21.15.84:0/3968114564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9abc004020 0x7f9abc0047c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800c0e0 tx=0x7f9abc0057f0 comp rx=0 tx=0).stop 2023-10-16T10:07:22.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.508+0000 7f9acffff640 1 -- 172.21.15.84:0/3968114564 reap_dead start 2023-10-16T10:07:22.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.508+0000 7f9ad6133640 1 -- 172.21.15.84:0/3968114564 shutdown_connections 2023-10-16T10:07:22.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.508+0000 7f9ad6133640 1 -- 172.21.15.84:0/3968114564 >> 172.21.15.84:0/3968114564 conn(0x7f9ad00ce9b0 msgr2=0x7f9ad00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:22.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.508+0000 7f9ad6133640 1 -- 172.21.15.84:0/3968114564 shutdown_connections 2023-10-16T10:07:22.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.508+0000 7f9ad6133640 1 -- 172.21.15.84:0/3968114564 wait complete. 2023-10-16T10:07:22.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.510+0000 7f9ad6133640 1 Processor -- start 2023-10-16T10:07:22.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.510+0000 7f9ad6133640 1 -- start start 2023-10-16T10:07:22.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.511+0000 7f9ad6133640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9abc004020 0x7f9ad015d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:22.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.511+0000 7f9ad6133640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ad00ace40 0x7f9ad0156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:22.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.511+0000 7f9ad6133640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad00af2e0 0x7f9ad00af760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:22.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.511+0000 7f9ad6133640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad015d6b0 con 0x7f9abc004020 2023-10-16T10:07:22.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.511+0000 7f9ad6133640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad00afca0 con 0x7f9ad00af2e0 2023-10-16T10:07:22.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.511+0000 7f9ad6133640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad00b23f0 con 0x7f9ad00ace40 2023-10-16T10:07:22.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.511+0000 7f9acf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9abc004020 0x7f9ad015d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:22.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.511+0000 7f9acf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9abc004020 0x7f9ad015d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59670/0 (socket says 172.21.15.84:59670) 2023-10-16T10:07:22.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.511+0000 7f9acf7fe640 1 -- 172.21.15.84:0/2159894151 learned_addr learned my addr 172.21.15.84:0/2159894151 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:22.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.512+0000 7f9acf7fe640 1 -- 172.21.15.84:0/2159894151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ad00ace40 msgr2=0x7f9ad0156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:22.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.512+0000 7f9acf7fe640 1 --2- 172.21.15.84:0/2159894151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ad00ace40 0x7f9ad0156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:22.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.512+0000 7f9acf7fe640 1 -- 172.21.15.84:0/2159894151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad00af2e0 msgr2=0x7f9ad00af760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:22.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.512+0000 7f9acf7fe640 1 --2- 172.21.15.84:0/2159894151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad00af2e0 0x7f9ad00af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:22.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.512+0000 7f9acf7fe640 1 -- 172.21.15.84:0/2159894151 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad00b1670 con 0x7f9abc004020 2023-10-16T10:07:22.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.512+0000 7f9acf7fe640 1 --2- 172.21.15.84:0/2159894151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9abc004020 0x7f9ad015d170 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000e550 tx=0x7f9ac000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:22.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.512+0000 7f9accff9640 1 -- 172.21.15.84:0/2159894151 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac00052e0 con 0x7f9abc004020 2023-10-16T10:07:22.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.512+0000 7f9accff9640 1 -- 172.21.15.84:0/2159894151 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac0005440 con 0x7f9abc004020 2023-10-16T10:07:22.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.512+0000 7f9ad6133640 1 -- 172.21.15.84:0/2159894151 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad00b1910 con 0x7f9abc004020 2023-10-16T10:07:22.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.512+0000 7f9accff9640 1 -- 172.21.15.84:0/2159894151 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0003810 con 0x7f9abc004020 2023-10-16T10:07:22.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.512+0000 7f9ad6133640 1 -- 172.21.15.84:0/2159894151 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad00b1ed0 con 0x7f9abc004020 2023-10-16T10:07:22.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.513+0000 7f9ad6133640 1 -- 172.21.15.84:0/2159894151 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad00b2060 con 0x7f9abc004020 2023-10-16T10:07:22.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.514+0000 7f9accff9640 1 -- 172.21.15.84:0/2159894151 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9ac0003970 con 0x7f9abc004020 2023-10-16T10:07:22.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.515+0000 7f9accff9640 1 --2- 172.21.15.84:0/2159894151 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f9ab006afc0 0x7f9ab006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:22.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.515+0000 7f9accff9640 1 -- 172.21.15.84:0/2159894151 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f9ac0089610 con 0x7f9abc004020 2023-10-16T10:07:22.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.516+0000 7f9ac6ffd640 1 --2- 172.21.15.84:0/2159894151 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f9ab006afc0 0x7f9ab006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:22.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.516+0000 7f9ac6ffd640 1 --2- 172.21.15.84:0/2159894151 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f9ab006afc0 0x7f9ab006d480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8034000 tx=0x7f9ab8007000 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:22.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.520+0000 7f9accff9640 1 -- 172.21.15.84:0/2159894151 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9ac00544b0 con 0x7f9abc004020 2023-10-16T10:07:22.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.696+0000 7f9ad6133640 1 -- 172.21.15.84:0/2159894151 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9ad004e160 con 0x7f9abc004020 2023-10-16T10:07:22.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.702+0000 7f9accff9640 1 -- 172.21.15.84:0/2159894151 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9ac0057ad0 con 0x7f9abc004020 2023-10-16T10:07:22.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.705+0000 7f9ad6133640 1 -- 172.21.15.84:0/2159894151 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f9ab006afc0 msgr2=0x7f9ab006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:22.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.705+0000 7f9ad6133640 1 --2- 172.21.15.84:0/2159894151 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f9ab006afc0 0x7f9ab006d480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8034000 tx=0x7f9ab8007000 comp rx=0 tx=0).stop 2023-10-16T10:07:22.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.705+0000 7f9ad6133640 1 -- 172.21.15.84:0/2159894151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9abc004020 msgr2=0x7f9ad015d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:22.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.705+0000 7f9ad6133640 1 --2- 172.21.15.84:0/2159894151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9abc004020 0x7f9ad015d170 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000e550 tx=0x7f9ac000ea10 comp rx=0 tx=0).stop 2023-10-16T10:07:22.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.705+0000 7f9ad6133640 1 -- 172.21.15.84:0/2159894151 shutdown_connections 2023-10-16T10:07:22.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.705+0000 7f9ad6133640 1 --2- 172.21.15.84:0/2159894151 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f9ab006afc0 0x7f9ab006d480 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:22.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.705+0000 7f9ad6133640 1 --2- 172.21.15.84:0/2159894151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad00af2e0 0x7f9ad00af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:22.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.705+0000 7f9ad6133640 1 --2- 172.21.15.84:0/2159894151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ad00ace40 0x7f9ad0156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:22.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.705+0000 7f9ad6133640 1 --2- 172.21.15.84:0/2159894151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9abc004020 0x7f9ad015d170 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:22.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.705+0000 7f9ad6133640 1 -- 172.21.15.84:0/2159894151 >> 172.21.15.84:0/2159894151 conn(0x7f9ad00ce9b0 msgr2=0x7f9ad006b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:22.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.705+0000 7f9ad6133640 1 -- 172.21.15.84:0/2159894151 shutdown_connections 2023-10-16T10:07:22.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:22.705+0000 7f9ad6133640 1 -- 172.21.15.84:0/2159894151 wait complete. 2023-10-16T10:07:22.723 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:07:22.724 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:07:22.831 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:07:22.955 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:07:22.956 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:07:22.957 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:07:23.019 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:07:23.019 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:07:23.020 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:07:23.079 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:07:23.080 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:07:23.080 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:07:23.139 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:07:23.140 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:07:23.140 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:07:23.201 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:07:23.202 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T10:07:23.202 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:07:23.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.425+0000 7f4345bc0640 1 Processor -- start 2023-10-16T10:07:23.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.425+0000 7f4345bc0640 1 -- start start 2023-10-16T10:07:23.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.426+0000 7f4345bc0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f43400adb50 0x7f434006f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:23.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.426+0000 7f4345bc0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4340070fd0 0x7f43400b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:23.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.426+0000 7f4345bc0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4340071450 con 0x7f43400adf30 2023-10-16T10:07:23.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.426+0000 7f4345bc0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43400cc600 con 0x7f43400adb50 2023-10-16T10:07:23.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.426+0000 7f4345bc0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43400cc8e0 con 0x7f4340070fd0 2023-10-16T10:07:23.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.427+0000 7f433f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f43400adb50 0x7f434006f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:23.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.427+0000 7f4337fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4340070fd0 0x7f43400b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:23.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.427+0000 7f433ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f43400adf30 0x7f4340164f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:60608/0 (socket says 172.21.15.84:60608) 2023-10-16T10:07:23.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.427+0000 7f433ffff640 1 -- 172.21.15.84:0/2599718268 learned_addr learned my addr 172.21.15.84:0/2599718268 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:23.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.427+0000 7f433f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f43400adb50 0x7f434006f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:40040/0 (socket says 172.21.15.84:40040) 2023-10-16T10:07:23.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.428+0000 7f433f7fe640 1 -- 172.21.15.84:0/2599718268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4340070fd0 msgr2=0x7f43400b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:23.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.428+0000 7f433f7fe640 1 --2- 172.21.15.84:0/2599718268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4340070fd0 0x7f43400b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:23.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.428+0000 7f433f7fe640 1 -- 172.21.15.84:0/2599718268 >> v1:172.21.15.157:6789/0 conn(0x7f43400adf30 legacy=0x7f4340164f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:23.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.428+0000 7f433f7fe640 1 -- 172.21.15.84:0/2599718268 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43400ccbc0 con 0x7f43400adb50 2023-10-16T10:07:23.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.428+0000 7f4337fff640 1 --2- 172.21.15.84:0/2599718268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4340070fd0 0x7f43400b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:07:23.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.428+0000 7f433effd640 1 -- 172.21.15.84:0/2599718268 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3086432568 0 0) 0x7f4340071450 con 0x7f43400adf30 2023-10-16T10:07:23.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.429+0000 7f433f7fe640 1 --2- 172.21.15.84:0/2599718268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43400adb50 0x7f434006f550 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f43300097a0 tx=0x7f433002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b526670f03ec386 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:23.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.430+0000 7f433effd640 1 -- 172.21.15.84:0/2599718268 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433002d040 con 0x7f43400adb50 2023-10-16T10:07:23.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.430+0000 7f433effd640 1 -- 172.21.15.84:0/2599718268 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4330031810 con 0x7f43400adb50 2023-10-16T10:07:23.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.430+0000 7f433effd640 1 -- 172.21.15.84:0/2599718268 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4330031ba0 con 0x7f43400adb50 2023-10-16T10:07:23.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.431+0000 7f4345bc0640 1 -- 172.21.15.84:0/2599718268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43400adb50 msgr2=0x7f434006f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:23.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.431+0000 7f4345bc0640 1 --2- 172.21.15.84:0/2599718268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43400adb50 0x7f434006f550 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f43300097a0 tx=0x7f433002b8b0 comp rx=0 tx=0).stop 2023-10-16T10:07:23.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.431+0000 7f4345bc0640 1 -- 172.21.15.84:0/2599718268 shutdown_connections 2023-10-16T10:07:23.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.431+0000 7f4345bc0640 1 --2- 172.21.15.84:0/2599718268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4340070fd0 0x7f43400b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:23.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.431+0000 7f4345bc0640 1 --2- 172.21.15.84:0/2599718268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43400adb50 0x7f434006f550 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:23.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.431+0000 7f4345bc0640 1 -- 172.21.15.84:0/2599718268 >> 172.21.15.84:0/2599718268 conn(0x7f43400cb410 msgr2=0x7f43400cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:23.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.431+0000 7f4345bc0640 1 -- 172.21.15.84:0/2599718268 shutdown_connections 2023-10-16T10:07:23.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.431+0000 7f4345bc0640 1 -- 172.21.15.84:0/2599718268 wait complete. 2023-10-16T10:07:23.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.433+0000 7f4345bc0640 1 Processor -- start 2023-10-16T10:07:23.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.433+0000 7f4345bc0640 1 -- start start 2023-10-16T10:07:23.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.434+0000 7f4345bc0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4340070fd0 0x7f43400b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:23.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.434+0000 7f4345bc0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43400adb50 0x7f43400c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:23.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.434+0000 7f4345bc0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43400adf30 0x7f4340155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:23.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.434+0000 7f4345bc0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43400b2e30 con 0x7f43400adf30 2023-10-16T10:07:23.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.434+0000 7f4345bc0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43400b2260 con 0x7f43400adb50 2023-10-16T10:07:23.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.434+0000 7f4345bc0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43400b2500 con 0x7f4340070fd0 2023-10-16T10:07:23.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.435+0000 7f433ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43400adf30 0x7f4340155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:23.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.435+0000 7f433ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43400adf30 0x7f4340155be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59672/0 (socket says 172.21.15.84:59672) 2023-10-16T10:07:23.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.435+0000 7f433ffff640 1 -- 172.21.15.84:0/3212004067 learned_addr learned my addr 172.21.15.84:0/3212004067 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:23.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.435+0000 7f433f7fe640 1 --2- 172.21.15.84:0/3212004067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4340070fd0 0x7f43400b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:23.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.435+0000 7f4337fff640 1 --2- 172.21.15.84:0/3212004067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43400adb50 0x7f43400c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:23.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.435+0000 7f433ffff640 1 -- 172.21.15.84:0/3212004067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4340070fd0 msgr2=0x7f43400b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:23.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.435+0000 7f433ffff640 1 --2- 172.21.15.84:0/3212004067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4340070fd0 0x7f43400b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:23.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.435+0000 7f433ffff640 1 -- 172.21.15.84:0/3212004067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43400adb50 msgr2=0x7f43400c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:23.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.435+0000 7f433ffff640 1 --2- 172.21.15.84:0/3212004067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43400adb50 0x7f43400c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:23.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.435+0000 7f433ffff640 1 -- 172.21.15.84:0/3212004067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4328009400 con 0x7f43400adf30 2023-10-16T10:07:23.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.436+0000 7f433ffff640 1 --2- 172.21.15.84:0/3212004067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43400adf30 0x7f4340155be0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f433800c7e0 tx=0x7f433800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:23.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.436+0000 7f433cff9640 1 -- 172.21.15.84:0/3212004067 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433800a640 con 0x7f43400adf30 2023-10-16T10:07:23.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.436+0000 7f433cff9640 1 -- 172.21.15.84:0/3212004067 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433800a7c0 con 0x7f43400adf30 2023-10-16T10:07:23.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.436+0000 7f4345bc0640 1 -- 172.21.15.84:0/3212004067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4330009400 con 0x7f43400adf30 2023-10-16T10:07:23.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.436+0000 7f433cff9640 1 -- 172.21.15.84:0/3212004067 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433800ab70 con 0x7f43400adf30 2023-10-16T10:07:23.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.436+0000 7f4345bc0640 1 -- 172.21.15.84:0/3212004067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43400b2ac0 con 0x7f43400adf30 2023-10-16T10:07:23.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.437+0000 7f4345bc0640 1 -- 172.21.15.84:0/3212004067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43400b2fa0 con 0x7f43400adf30 2023-10-16T10:07:23.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.438+0000 7f433cff9640 1 -- 172.21.15.84:0/3212004067 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f433800acd0 con 0x7f43400adf30 2023-10-16T10:07:23.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.439+0000 7f433cff9640 1 --2- 172.21.15.84:0/3212004067 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f432006af90 0x7f432006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:23.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.439+0000 7f433cff9640 1 -- 172.21.15.84:0/3212004067 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f4338088ef0 con 0x7f43400adf30 2023-10-16T10:07:23.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.439+0000 7f433f7fe640 1 --2- 172.21.15.84:0/3212004067 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f432006af90 0x7f432006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:23.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.440+0000 7f433f7fe640 1 --2- 172.21.15.84:0/3212004067 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f432006af90 0x7f432006d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4330036000 tx=0x7f433003c040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:23.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.444+0000 7f433cff9640 1 -- 172.21.15.84:0/3212004067 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4338053d50 con 0x7f43400adf30 2023-10-16T10:07:23.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.672+0000 7f4345bc0640 1 -- 172.21.15.84:0/3212004067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f434004e160 con 0x7f43400adf30 2023-10-16T10:07:23.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.673+0000 7f433cff9640 1 -- 172.21.15.84:0/3212004067 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f4338018030 con 0x7f43400adf30 2023-10-16T10:07:23.675 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:23.675 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3458903351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3458903351},{"type":"v1","addr":"172.21.15.84:6837","nonce":3458903351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5806,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4002935522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4002935522},{"type":"v1","addr":"172.21.15.157:6833","nonce":4002935522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5809,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1890551305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1890551305},{"type":"v1","addr":"172.21.15.157:6835","nonce":1890551305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7944,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/3043443828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3043443828},{"type":"v1","addr":"172.21.15.84:6836","nonce":3043443828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[]} 2023-10-16T10:07:23.675 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 55 2023-10-16T10:07:23.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.675+0000 7f4345bc0640 1 -- 172.21.15.84:0/3212004067 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f432006af90 msgr2=0x7f432006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:23.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.675+0000 7f4345bc0640 1 --2- 172.21.15.84:0/3212004067 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f432006af90 0x7f432006d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4330036000 tx=0x7f433003c040 comp rx=0 tx=0).stop 2023-10-16T10:07:23.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.675+0000 7f4345bc0640 1 -- 172.21.15.84:0/3212004067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43400adf30 msgr2=0x7f4340155be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:23.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.675+0000 7f4345bc0640 1 --2- 172.21.15.84:0/3212004067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43400adf30 0x7f4340155be0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f433800c7e0 tx=0x7f433800cca0 comp rx=0 tx=0).stop 2023-10-16T10:07:23.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.676+0000 7f4345bc0640 1 -- 172.21.15.84:0/3212004067 shutdown_connections 2023-10-16T10:07:23.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.676+0000 7f4345bc0640 1 --2- 172.21.15.84:0/3212004067 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f432006af90 0x7f432006d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:23.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.676+0000 7f4345bc0640 1 --2- 172.21.15.84:0/3212004067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43400adf30 0x7f4340155be0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:23.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.676+0000 7f4345bc0640 1 --2- 172.21.15.84:0/3212004067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43400adb50 0x7f43400c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:23.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.676+0000 7f4345bc0640 1 --2- 172.21.15.84:0/3212004067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4340070fd0 0x7f43400b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:23.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.676+0000 7f4345bc0640 1 -- 172.21.15.84:0/3212004067 >> 172.21.15.84:0/3212004067 conn(0x7f43400cb410 msgr2=0x7f43400c9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:23.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.676+0000 7f4345bc0640 1 -- 172.21.15.84:0/3212004067 shutdown_connections 2023-10-16T10:07:23.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.676+0000 7f4345bc0640 1 -- 172.21.15.84:0/3212004067 wait complete. 2023-10-16T10:07:23.698 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T10:07:23.698 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T10:07:23.699 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T10:07:23.699 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:07:23.699 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T10:07:23.699 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T10:07:23.700 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T10:07:23.700 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:07:23.700 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T10:07:23.701 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T10:07:23.701 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T10:07:23.701 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:07:23.701 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T10:07:23.702 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T10:07:23.702 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T10:07:23.702 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:07:23.717 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T10:07:23.717 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T10:07:23.720 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T10:07:23.720 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T10:07:23.721 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T10:07:23.721 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T10:07:23.731 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T10:07:23.731 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T10:07:23.768 INFO:tasks.ceph.mds.a:Started 2023-10-16T10:07:23.769 DEBUG:teuthology.parallel:result is None 2023-10-16T10:07:23.776 INFO:tasks.ceph.mds.c:Started 2023-10-16T10:07:23.776 DEBUG:teuthology.parallel:result is None 2023-10-16T10:07:23.779 INFO:tasks.ceph.mds.d:Started 2023-10-16T10:07:23.780 DEBUG:teuthology.parallel:result is None 2023-10-16T10:07:23.786 INFO:tasks.ceph.mds.b:Started 2023-10-16T10:07:23.787 DEBUG:teuthology.parallel:result is None 2023-10-16T10:07:23.787 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T10:07:23.928 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T10:07:23.930 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T10:07:23.951 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-10-16T10:07:23.956 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T10:07:23.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.976+0000 7fa79bd8a640 1 Processor -- start 2023-10-16T10:07:23.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.976+0000 7fa79bd8a640 1 -- start start 2023-10-16T10:07:23.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.977+0000 7fa79bd8a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa79415ad70 0x7fa794069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:23.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.977+0000 7fa79bd8a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7940caac0 0x7fa79406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:23.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.977+0000 7fa79bd8a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7940cb030 con 0x7fa7940c9ca0 2023-10-16T10:07:23.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.977+0000 7fa79bd8a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7940c91c0 con 0x7fa79415ad70 2023-10-16T10:07:23.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.977+0000 7fa79bd8a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7940c94a0 con 0x7fa7940caac0 2023-10-16T10:07:23.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.978+0000 7fa799325640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa7940c9ca0 0x7fa7940ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48362/0 (socket says 172.21.15.84:48362) 2023-10-16T10:07:23.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.978+0000 7fa799325640 1 -- 172.21.15.84:0/1542196697 learned_addr learned my addr 172.21.15.84:0/1542196697 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:23.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.978+0000 7fa799b26640 1 --2- 172.21.15.84:0/1542196697 >> v2:172.21.15.84:3300/0 conn(0x7fa79415ad70 0x7fa794069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:23.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.978+0000 7fa799b26640 1 -- 172.21.15.84:0/1542196697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7940caac0 msgr2=0x7fa79406a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:23.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.978+0000 7fa79a327640 1 --2- 172.21.15.84:0/1542196697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7940caac0 0x7fa79406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:23.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.978+0000 7fa799b26640 1 --2- 172.21.15.84:0/1542196697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7940caac0 0x7fa79406a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:23.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.978+0000 7fa798b24640 1 -- 172.21.15.84:0/1542196697 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1699812748 0 0) 0x7fa7940cb030 con 0x7fa7940c9ca0 2023-10-16T10:07:23.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.978+0000 7fa799b26640 1 -- 172.21.15.84:0/1542196697 >> v1:172.21.15.84:6789/0 conn(0x7fa7940c9ca0 legacy=0x7fa7940ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:23.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.978+0000 7fa799b26640 1 -- 172.21.15.84:0/1542196697 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7940c9780 con 0x7fa79415ad70 2023-10-16T10:07:23.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.978+0000 7fa79a327640 1 --2- 172.21.15.84:0/1542196697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7940caac0 0x7fa79406a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:23.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.979+0000 7fa799b26640 1 --2- 172.21.15.84:0/1542196697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa79415ad70 0x7fa794069da0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fa7900097a0 tx=0x7fa79002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71aed41c5315a298 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:23.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.980+0000 7fa798b24640 1 -- 172.21.15.84:0/1542196697 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79002d040 con 0x7fa79415ad70 2023-10-16T10:07:23.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.980+0000 7fa798b24640 1 -- 172.21.15.84:0/1542196697 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa790031810 con 0x7fa79415ad70 2023-10-16T10:07:23.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.980+0000 7fa798b24640 1 -- 172.21.15.84:0/1542196697 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa790031ba0 con 0x7fa79415ad70 2023-10-16T10:07:23.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.980+0000 7fa79bd8a640 1 -- 172.21.15.84:0/1542196697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa79415ad70 msgr2=0x7fa794069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:23.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.980+0000 7fa79bd8a640 1 --2- 172.21.15.84:0/1542196697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa79415ad70 0x7fa794069da0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fa7900097a0 tx=0x7fa79002b8b0 comp rx=0 tx=0).stop 2023-10-16T10:07:23.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.980+0000 7fa79bd8a640 1 -- 172.21.15.84:0/1542196697 shutdown_connections 2023-10-16T10:07:23.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.980+0000 7fa79bd8a640 1 --2- 172.21.15.84:0/1542196697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7940caac0 0x7fa79406a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:23.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.980+0000 7fa79bd8a640 1 --2- 172.21.15.84:0/1542196697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa79415ad70 0x7fa794069da0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:23.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.980+0000 7fa79bd8a640 1 -- 172.21.15.84:0/1542196697 >> 172.21.15.84:0/1542196697 conn(0x7fa7940ce9b0 msgr2=0x7fa7940cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:23.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.980+0000 7fa79bd8a640 1 -- 172.21.15.84:0/1542196697 shutdown_connections 2023-10-16T10:07:23.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.980+0000 7fa79bd8a640 1 -- 172.21.15.84:0/1542196697 wait complete. 2023-10-16T10:07:23.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.982+0000 7fa79bd8a640 1 Processor -- start 2023-10-16T10:07:23.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.982+0000 7fa79bd8a640 1 -- start start 2023-10-16T10:07:23.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.983+0000 7fa79bd8a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7940c9ca0 0x7fa794148090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:23.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.983+0000 7fa79bd8a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7940caac0 0x7fa7940c4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:23.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.983+0000 7fa79bd8a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa79415ad70 0x7fa7940c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:23.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.983+0000 7fa79bd8a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79414c0c0 con 0x7fa7940c9ca0 2023-10-16T10:07:23.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.983+0000 7fa79bd8a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7941485d0 con 0x7fa7940caac0 2023-10-16T10:07:23.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.983+0000 7fa79bd8a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79414bd50 con 0x7fa79415ad70 2023-10-16T10:07:23.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.983+0000 7fa799b26640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7940c9ca0 0x7fa794148090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:23.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.983+0000 7fa799b26640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7940c9ca0 0x7fa794148090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59742/0 (socket says 172.21.15.84:59742) 2023-10-16T10:07:23.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.983+0000 7fa799b26640 1 -- 172.21.15.84:0/3898442264 learned_addr learned my addr 172.21.15.84:0/3898442264 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:23.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.984+0000 7fa79a327640 1 --2- 172.21.15.84:0/3898442264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa79415ad70 0x7fa7940c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:23.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.984+0000 7fa799325640 1 --2- 172.21.15.84:0/3898442264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7940caac0 0x7fa7940c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:23.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.984+0000 7fa799b26640 1 -- 172.21.15.84:0/3898442264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa79415ad70 msgr2=0x7fa7940c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:23.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.984+0000 7fa799b26640 1 --2- 172.21.15.84:0/3898442264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa79415ad70 0x7fa7940c7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:23.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.984+0000 7fa799b26640 1 -- 172.21.15.84:0/3898442264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7940caac0 msgr2=0x7fa7940c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:23.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.984+0000 7fa799b26640 1 --2- 172.21.15.84:0/3898442264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7940caac0 0x7fa7940c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:23.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.984+0000 7fa799b26640 1 -- 172.21.15.84:0/3898442264 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa790009400 con 0x7fa7940c9ca0 2023-10-16T10:07:23.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.984+0000 7fa799325640 1 --2- 172.21.15.84:0/3898442264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7940caac0 0x7fa7940c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:23.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.984+0000 7fa799b26640 1 --2- 172.21.15.84:0/3898442264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7940c9ca0 0x7fa794148090 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa7900098c0 tx=0x7fa7900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:23.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.984+0000 7fa78a7fc640 1 -- 172.21.15.84:0/3898442264 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79003ad60 con 0x7fa7940c9ca0 2023-10-16T10:07:23.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.984+0000 7fa78a7fc640 1 -- 172.21.15.84:0/3898442264 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7900423f0 con 0x7fa7940c9ca0 2023-10-16T10:07:23.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.984+0000 7fa78a7fc640 1 -- 172.21.15.84:0/3898442264 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7900427a0 con 0x7fa7940c9ca0 2023-10-16T10:07:23.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.984+0000 7fa79bd8a640 1 -- 172.21.15.84:0/3898442264 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7940c4d60 con 0x7fa7940c9ca0 2023-10-16T10:07:23.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.985+0000 7fa79bd8a640 1 -- 172.21.15.84:0/3898442264 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7940c2760 con 0x7fa7940c9ca0 2023-10-16T10:07:23.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.986+0000 7fa79bd8a640 1 -- 172.21.15.84:0/3898442264 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa79404e160 con 0x7fa7940c9ca0 2023-10-16T10:07:23.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.987+0000 7fa78a7fc640 1 -- 172.21.15.84:0/3898442264 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa79004b3b0 con 0x7fa7940c9ca0 2023-10-16T10:07:23.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.988+0000 7fa78a7fc640 1 --2- 172.21.15.84:0/3898442264 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fa77806f3a0 0x7fa778071860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:23.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.988+0000 7fa78a7fc640 1 -- 172.21.15.84:0/3898442264 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5717+0+0 (secure 0 0 0) 0x7fa7900ab940 con 0x7fa7940c9ca0 2023-10-16T10:07:23.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.988+0000 7fa799325640 1 --2- 172.21.15.84:0/3898442264 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fa77806f3a0 0x7fa778071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:23.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.989+0000 7fa799325640 1 --2- 172.21.15.84:0/3898442264 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fa77806f3a0 0x7fa778071860 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa7940afc00 tx=0x7fa784009040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:23.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:23.992+0000 7fa78a7fc640 1 -- 172.21.15.84:0/3898442264 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa790076850 con 0x7fa7940c9ca0 2023-10-16T10:07:24.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.202+0000 7fa79bd8a640 1 -- 172.21.15.84:0/3898442264 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa7940b1a50 con 0x7fa7940c9ca0 2023-10-16T10:07:24.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.288+0000 7fa78a7fc640 1 -- 172.21.15.84:0/3898442264 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v118) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa790079e70 con 0x7fa7940c9ca0 2023-10-16T10:07:24.290 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-10-16T10:07:24.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.292+0000 7fa79bd8a640 1 -- 172.21.15.84:0/3898442264 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fa77806f3a0 msgr2=0x7fa778071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:24.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.292+0000 7fa79bd8a640 1 --2- 172.21.15.84:0/3898442264 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fa77806f3a0 0x7fa778071860 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa7940afc00 tx=0x7fa784009040 comp rx=0 tx=0).stop 2023-10-16T10:07:24.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.292+0000 7fa79bd8a640 1 -- 172.21.15.84:0/3898442264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7940c9ca0 msgr2=0x7fa794148090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:24.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.292+0000 7fa79bd8a640 1 --2- 172.21.15.84:0/3898442264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7940c9ca0 0x7fa794148090 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa7900098c0 tx=0x7fa7900046c0 comp rx=0 tx=0).stop 2023-10-16T10:07:24.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.292+0000 7fa79bd8a640 1 -- 172.21.15.84:0/3898442264 shutdown_connections 2023-10-16T10:07:24.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.292+0000 7fa79bd8a640 1 --2- 172.21.15.84:0/3898442264 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fa77806f3a0 0x7fa778071860 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:24.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.292+0000 7fa79bd8a640 1 --2- 172.21.15.84:0/3898442264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa79415ad70 0x7fa7940c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:24.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.292+0000 7fa79bd8a640 1 --2- 172.21.15.84:0/3898442264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7940caac0 0x7fa7940c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:24.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.292+0000 7fa79bd8a640 1 --2- 172.21.15.84:0/3898442264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7940c9ca0 0x7fa794148090 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:24.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.292+0000 7fa79bd8a640 1 -- 172.21.15.84:0/3898442264 >> 172.21.15.84:0/3898442264 conn(0x7fa7940ce9b0 msgr2=0x7fa794070d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:24.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.292+0000 7fa79bd8a640 1 -- 172.21.15.84:0/3898442264 shutdown_connections 2023-10-16T10:07:24.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.292+0000 7fa79bd8a640 1 -- 172.21.15.84:0/3898442264 wait complete. 2023-10-16T10:07:24.314 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T10:07:24.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.519+0000 7f0b568bb640 1 Processor -- start 2023-10-16T10:07:24.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.519+0000 7f0b568bb640 1 -- start start 2023-10-16T10:07:24.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.520+0000 7f0b568bb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0b500cd0c0 0x7f0b501615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:24.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.520+0000 7f0b568bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500b0410 0x7f0b500b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:24.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.520+0000 7f0b568bb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b500b0db0 con 0x7f0b5015d3f0 2023-10-16T10:07:24.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.520+0000 7f0b568bb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b5015e3e0 con 0x7f0b500cd0c0 2023-10-16T10:07:24.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.520+0000 7f0b568bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b500cd4a0 con 0x7f0b500b0410 2023-10-16T10:07:24.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.521+0000 7f0b4ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0b500cd0c0 0x7f0b501615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:24.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.521+0000 7f0b4ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0b500cd0c0 0x7f0b501615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59744/0 (socket says 172.21.15.84:59744) 2023-10-16T10:07:24.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.521+0000 7f0b4ffff640 1 -- 172.21.15.84:0/363193800 learned_addr learned my addr 172.21.15.84:0/363193800 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:24.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.521+0000 7f0b47fff640 1 --2- 172.21.15.84:0/363193800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500b0410 0x7f0b500b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:24.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.521+0000 7f0b4ffff640 1 -- 172.21.15.84:0/363193800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500b0410 msgr2=0x7f0b500b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:24.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.521+0000 7f0b4ffff640 1 --2- 172.21.15.84:0/363193800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500b0410 0x7f0b500b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:24.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.521+0000 7f0b4ffff640 1 -- 172.21.15.84:0/363193800 >> v1:172.21.15.157:6789/0 conn(0x7f0b5015d3f0 legacy=0x7f0b5015d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:24.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.521+0000 7f0b4ffff640 1 -- 172.21.15.84:0/363193800 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5015e100 con 0x7f0b500cd0c0 2023-10-16T10:07:24.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.522+0000 7f0b4ffff640 1 --2- 172.21.15.84:0/363193800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b500cd0c0 0x7f0b501615f0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009870 tx=0x7f0b4002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6fd93c9282dac9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:24.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.523+0000 7f0b4f7fe640 1 -- 172.21.15.84:0/363193800 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40031640 con 0x7f0b500cd0c0 2023-10-16T10:07:24.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.523+0000 7f0b4f7fe640 1 -- 172.21.15.84:0/363193800 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b400317a0 con 0x7f0b500cd0c0 2023-10-16T10:07:24.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.523+0000 7f0b4f7fe640 1 -- 172.21.15.84:0/363193800 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40031b30 con 0x7f0b500cd0c0 2023-10-16T10:07:24.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.523+0000 7f0b568bb640 1 -- 172.21.15.84:0/363193800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b500cd0c0 msgr2=0x7f0b501615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:24.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.523+0000 7f0b568bb640 1 --2- 172.21.15.84:0/363193800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b500cd0c0 0x7f0b501615f0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009870 tx=0x7f0b4002b8f0 comp rx=0 tx=0).stop 2023-10-16T10:07:24.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.523+0000 7f0b568bb640 1 -- 172.21.15.84:0/363193800 shutdown_connections 2023-10-16T10:07:24.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.523+0000 7f0b568bb640 1 --2- 172.21.15.84:0/363193800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500b0410 0x7f0b500b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:24.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.523+0000 7f0b568bb640 1 --2- 172.21.15.84:0/363193800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b500cd0c0 0x7f0b501615f0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:24.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.523+0000 7f0b568bb640 1 -- 172.21.15.84:0/363193800 >> 172.21.15.84:0/363193800 conn(0x7f0b500cb410 msgr2=0x7f0b500cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:24.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.523+0000 7f0b568bb640 1 -- 172.21.15.84:0/363193800 shutdown_connections 2023-10-16T10:07:24.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.523+0000 7f0b568bb640 1 -- 172.21.15.84:0/363193800 wait complete. 2023-10-16T10:07:24.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.525+0000 7f0b568bb640 1 Processor -- start 2023-10-16T10:07:24.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.525+0000 7f0b568bb640 1 -- start start 2023-10-16T10:07:24.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.526+0000 7f0b568bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b500b0410 0x7f0b50156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:24.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.526+0000 7f0b568bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500cd0c0 0x7f0b5015b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:24.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.526+0000 7f0b568bb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5015d3f0 0x7f0b5015b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:24.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.526+0000 7f0b568bb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b50156d70 con 0x7f0b5015d3f0 2023-10-16T10:07:24.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.526+0000 7f0b568bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b501e6d40 con 0x7f0b500b0410 2023-10-16T10:07:24.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.526+0000 7f0b568bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b500b23a0 con 0x7f0b500cd0c0 2023-10-16T10:07:24.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.526+0000 7f0b54e58640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5015d3f0 0x7f0b5015b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:24.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.527+0000 7f0b54e58640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5015d3f0 0x7f0b5015b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59750/0 (socket says 172.21.15.84:59750) 2023-10-16T10:07:24.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.527+0000 7f0b54e58640 1 -- 172.21.15.84:0/308283159 learned_addr learned my addr 172.21.15.84:0/308283159 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:24.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.527+0000 7f0b4ffff640 1 --2- 172.21.15.84:0/308283159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b500b0410 0x7f0b50156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:24.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.527+0000 7f0b47fff640 1 --2- 172.21.15.84:0/308283159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500cd0c0 0x7f0b5015b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:24.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.527+0000 7f0b54e58640 1 -- 172.21.15.84:0/308283159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500cd0c0 msgr2=0x7f0b5015b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:24.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.527+0000 7f0b54e58640 1 --2- 172.21.15.84:0/308283159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500cd0c0 0x7f0b5015b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:24.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.527+0000 7f0b54e58640 1 -- 172.21.15.84:0/308283159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b500b0410 msgr2=0x7f0b50156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:24.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.527+0000 7f0b54e58640 1 --2- 172.21.15.84:0/308283159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b500b0410 0x7f0b50156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:24.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.527+0000 7f0b54e58640 1 -- 172.21.15.84:0/308283159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b400094d0 con 0x7f0b5015d3f0 2023-10-16T10:07:24.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.527+0000 7f0b4ffff640 1 --2- 172.21.15.84:0/308283159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b500b0410 0x7f0b50156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:24.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.527+0000 7f0b54e58640 1 --2- 172.21.15.84:0/308283159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5015d3f0 0x7f0b5015b830 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800c7a0 tx=0x7f0b4800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:24.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.527+0000 7f0b4d7fa640 1 -- 172.21.15.84:0/308283159 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b48003b70 con 0x7f0b5015d3f0 2023-10-16T10:07:24.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.528+0000 7f0b4d7fa640 1 -- 172.21.15.84:0/308283159 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b48003cf0 con 0x7f0b5015d3f0 2023-10-16T10:07:24.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.528+0000 7f0b4d7fa640 1 -- 172.21.15.84:0/308283159 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4800aa90 con 0x7f0b5015d3f0 2023-10-16T10:07:24.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.528+0000 7f0b568bb640 1 -- 172.21.15.84:0/308283159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b501e6410 con 0x7f0b5015d3f0 2023-10-16T10:07:24.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.528+0000 7f0b568bb640 1 -- 172.21.15.84:0/308283159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b501e69d0 con 0x7f0b5015d3f0 2023-10-16T10:07:24.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.529+0000 7f0b568bb640 1 -- 172.21.15.84:0/308283159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b501e6ed0 con 0x7f0b5015d3f0 2023-10-16T10:07:24.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.530+0000 7f0b4d7fa640 1 -- 172.21.15.84:0/308283159 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0b4800abf0 con 0x7f0b5015d3f0 2023-10-16T10:07:24.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.531+0000 7f0b4d7fa640 1 --2- 172.21.15.84:0/308283159 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0b3006afc0 0x7f0b3006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:24.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.531+0000 7f0b4d7fa640 1 -- 172.21.15.84:0/308283159 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0b48089340 con 0x7f0b5015d3f0 2023-10-16T10:07:24.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.531+0000 7f0b4ffff640 1 --2- 172.21.15.84:0/308283159 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0b3006afc0 0x7f0b3006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:24.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.532+0000 7f0b4ffff640 1 --2- 172.21.15.84:0/308283159 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0b3006afc0 0x7f0b3006d480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0b40036000 tx=0x7f0b4003c040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:24.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.535+0000 7f0b4d7fa640 1 -- 172.21.15.84:0/308283159 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0b48054590 con 0x7f0b5015d3f0 2023-10-16T10:07:24.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.794+0000 7f0b568bb640 1 -- 172.21.15.84:0/308283159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0b5004e160 con 0x7f0b5015d3f0 2023-10-16T10:07:24.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.795+0000 7f0b4d7fa640 1 -- 172.21.15.84:0/308283159 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0b48003e60 con 0x7f0b5015d3f0 2023-10-16T10:07:24.797 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:24.797 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T10:07:24.797 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-10-16T10:07:24.797 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.797 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-10-16T10:07:24.798 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l6dhjKRAA0pMvY6AK6qtFXDprNhwsqg==", 2023-10-16T10:07:24.798 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.798 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:07:24.798 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:07:24.798 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:07:24.799 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:07:24.799 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.799 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:07:24.799 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.799 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-10-16T10:07:24.800 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lFpMxEBAA0TLPRt5q3FwwBvMG019cjQ==", 2023-10-16T10:07:24.800 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.800 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:07:24.800 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:07:24.800 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:07:24.801 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:07:24.801 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.801 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:07:24.801 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.801 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-10-16T10:07:24.802 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l73RBOhAAOMEPxfUAMHhn5Blyvd5oMw==", 2023-10-16T10:07:24.802 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.802 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:07:24.802 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:07:24.803 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:07:24.803 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:07:24.803 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.803 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:07:24.803 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.804 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-10-16T10:07:24.804 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lcTIRIhAA0+FWm4b4yw77LD3+WKbk7w==", 2023-10-16T10:07:24.804 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.804 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:07:24.804 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:07:24.805 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:07:24.805 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:07:24.805 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.805 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:07:24.805 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.806 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-10-16T10:07:24.806 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDYBi1l1zBLERAA3YLW3xIYQoRVw7KSkwN3Og==", 2023-10-16T10:07:24.806 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.806 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:07:24.806 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:07:24.807 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:07:24.807 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.807 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:07:24.807 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.807 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-10-16T10:07:24.808 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDbBi1lFUUxKRAAQbus8SmYprJRmWTjoCznNA==", 2023-10-16T10:07:24.808 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.808 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:07:24.808 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:07:24.808 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:07:24.808 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.809 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:07:24.809 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.809 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-10-16T10:07:24.809 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDeBi1l+9lBMxAArl6sb/h15uBG/a9lVH40Hg==", 2023-10-16T10:07:24.809 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.810 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:07:24.810 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:07:24.810 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:07:24.810 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.810 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:07:24.811 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.811 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-10-16T10:07:24.811 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDiBi1l2rnEERAA3oRD71xa0fEOynM8zKuBEg==", 2023-10-16T10:07:24.811 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.811 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:07:24.812 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:07:24.812 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:07:24.812 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.812 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:07:24.812 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.812 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-10-16T10:07:24.813 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDnBi1lR7oLChAAZto8fDovDbpOd3Fcnc7C6w==", 2023-10-16T10:07:24.813 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.813 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:07:24.813 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:07:24.813 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:07:24.814 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.814 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:07:24.814 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.814 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-10-16T10:07:24.814 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDqBi1lmjg/JBAAlINKRGBc0/UWJJHMVFw57w==", 2023-10-16T10:07:24.815 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.815 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:07:24.815 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:07:24.815 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:07:24.815 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.816 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:07:24.816 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.816 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-10-16T10:07:24.816 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDuBi1lu0kMARAAtuSeM8F3yr8CVrc3p1biaQ==", 2023-10-16T10:07:24.816 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.816 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:07:24.817 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:07:24.817 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:07:24.817 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.817 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:07:24.817 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.817 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-10-16T10:07:24.818 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDxBi1lf6mNGBAAHGbqzJ0fmDxQIsasboHMfg==", 2023-10-16T10:07:24.818 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.818 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:07:24.818 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:07:24.818 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:07:24.819 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.819 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:07:24.819 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.819 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-10-16T10:07:24.819 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ==", 2023-10-16T10:07:24.819 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.820 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:07:24.820 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow r", 2023-10-16T10:07:24.820 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T10:07:24.820 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.820 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:07:24.821 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.821 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-10-16T10:07:24.821 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1lHU+4BxAAd8ET8hQ8Y+EUwdTJr9/+Ig==", 2023-10-16T10:07:24.821 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.821 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:07:24.821 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:07:24.822 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:07:24.822 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:07:24.822 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.822 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:07:24.822 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.823 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-10-16T10:07:24.823 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lA1BOORAAFNyfqgEF34MVR9YX3KBKkQ==", 2023-10-16T10:07:24.823 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.823 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:07:24.823 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:07:24.824 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:07:24.824 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:07:24.824 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.824 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:07:24.824 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.824 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-10-16T10:07:24.825 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1l848oERAANMfrZz9JrlgNVh7ZFbtnaA==", 2023-10-16T10:07:24.825 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.825 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:07:24.825 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:07:24.825 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:07:24.826 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:07:24.826 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.826 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:07:24.826 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.826 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-10-16T10:07:24.827 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDRBi1lTAxTGRAAMptxnxy3Qit/cTCXie6CrQ==", 2023-10-16T10:07:24.827 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.827 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:07:24.827 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:07:24.827 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:07:24.828 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:07:24.828 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.828 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:07:24.828 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.828 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-10-16T10:07:24.828 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lPuJxHRAABAnGPVCrkq0JR4Y6iFGZlw==", 2023-10-16T10:07:24.829 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.829 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:07:24.829 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T10:07:24.829 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:07:24.829 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.830 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:07:24.830 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:24.830 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-10-16T10:07:24.830 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lLxnKEBAA+a/pOHUp7Sg3m4weAust5w==", 2023-10-16T10:07:24.830 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:07:24.831 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:07:24.831 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T10:07:24.831 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:07:24.831 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.831 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:24.832 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-10-16T10:07:24.832 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T10:07:24.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.798+0000 7f0b568bb640 1 -- 172.21.15.84:0/308283159 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0b3006afc0 msgr2=0x7f0b3006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:24.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.798+0000 7f0b568bb640 1 --2- 172.21.15.84:0/308283159 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0b3006afc0 0x7f0b3006d480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0b40036000 tx=0x7f0b4003c040 comp rx=0 tx=0).stop 2023-10-16T10:07:24.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.798+0000 7f0b568bb640 1 -- 172.21.15.84:0/308283159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5015d3f0 msgr2=0x7f0b5015b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:24.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.798+0000 7f0b568bb640 1 --2- 172.21.15.84:0/308283159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5015d3f0 0x7f0b5015b830 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800c7a0 tx=0x7f0b4800cc60 comp rx=0 tx=0).stop 2023-10-16T10:07:24.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.798+0000 7f0b568bb640 1 -- 172.21.15.84:0/308283159 shutdown_connections 2023-10-16T10:07:24.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.798+0000 7f0b568bb640 1 --2- 172.21.15.84:0/308283159 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0b3006afc0 0x7f0b3006d480 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:24.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.798+0000 7f0b568bb640 1 --2- 172.21.15.84:0/308283159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b5015d3f0 0x7f0b5015b830 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:24.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.798+0000 7f0b568bb640 1 --2- 172.21.15.84:0/308283159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b500cd0c0 0x7f0b5015b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:24.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.798+0000 7f0b568bb640 1 --2- 172.21.15.84:0/308283159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b500b0410 0x7f0b50156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:24.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.798+0000 7f0b568bb640 1 -- 172.21.15.84:0/308283159 >> 172.21.15.84:0/308283159 conn(0x7f0b500cb410 msgr2=0x7f0b500c9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:24.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.798+0000 7f0b568bb640 1 -- 172.21.15.84:0/308283159 shutdown_connections 2023-10-16T10:07:24.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:24.798+0000 7f0b568bb640 1 -- 172.21.15.84:0/308283159 wait complete. 2023-10-16T10:07:24.836 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:07:25.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.051+0000 7f2e7c27c640 1 Processor -- start 2023-10-16T10:07:25.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.051+0000 7f2e7c27c640 1 -- start start 2023-10-16T10:07:25.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.052+0000 7f2e7c27c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e74159b20 0x7f2e7406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.052+0000 7f2e7c27c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2e740b0b20 0x7f2e7406fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.052+0000 7f2e7c27c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e7406f7f0 con 0x7f2e74159f00 2023-10-16T10:07:25.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.052+0000 7f2e7c27c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e7406c5e0 con 0x7f2e740b0b20 2023-10-16T10:07:25.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.052+0000 7f2e7c27c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7406c8c0 con 0x7f2e74159b20 2023-10-16T10:07:25.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.053+0000 7f2e79817640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2e74159f00 0x7f2e740b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48368/0 (socket says 172.21.15.84:48368) 2023-10-16T10:07:25.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.053+0000 7f2e7a819640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2e740b0b20 0x7f2e7406fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:25.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.053+0000 7f2e79817640 1 -- 172.21.15.84:0/1517096769 learned_addr learned my addr 172.21.15.84:0/1517096769 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:25.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.053+0000 7f2e7a018640 1 --2- 172.21.15.84:0/1517096769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e74159b20 0x7f2e7406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:25.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.053+0000 7f2e7a819640 1 -- 172.21.15.84:0/1517096769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e74159b20 msgr2=0x7f2e7406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:25.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.053+0000 7f2e7a819640 1 --2- 172.21.15.84:0/1517096769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e74159b20 0x7f2e7406bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.053+0000 7f2e7a819640 1 -- 172.21.15.84:0/1517096769 >> v1:172.21.15.84:6789/0 conn(0x7f2e74159f00 legacy=0x7f2e740b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:25.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.053+0000 7f2e7a819640 1 -- 172.21.15.84:0/1517096769 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7406cba0 con 0x7f2e740b0b20 2023-10-16T10:07:25.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.053+0000 7f2e79016640 1 -- 172.21.15.84:0/1517096769 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959734178 0 0) 0x7f2e7406f7f0 con 0x7f2e74159f00 2023-10-16T10:07:25.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.054+0000 7f2e7a819640 1 --2- 172.21.15.84:0/1517096769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e740b0b20 0x7f2e7406fc80 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009d90 tx=0x7f2e6c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5b5e2ab39c07d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:25.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.055+0000 7f2e79016640 1 -- 172.21.15.84:0/1517096769 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c02cc50 con 0x7f2e740b0b20 2023-10-16T10:07:25.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.055+0000 7f2e79016640 1 -- 172.21.15.84:0/1517096769 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6c02cdb0 con 0x7f2e740b0b20 2023-10-16T10:07:25.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.055+0000 7f2e79016640 1 -- 172.21.15.84:0/1517096769 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c034660 con 0x7f2e740b0b20 2023-10-16T10:07:25.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.055+0000 7f2e7c27c640 1 -- 172.21.15.84:0/1517096769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e740b0b20 msgr2=0x7f2e7406fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:25.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.055+0000 7f2e7c27c640 1 --2- 172.21.15.84:0/1517096769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e740b0b20 0x7f2e7406fc80 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009d90 tx=0x7f2e6c02bea0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.055+0000 7f2e7c27c640 1 -- 172.21.15.84:0/1517096769 shutdown_connections 2023-10-16T10:07:25.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.055+0000 7f2e7c27c640 1 --2- 172.21.15.84:0/1517096769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e740b0b20 0x7f2e7406fc80 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.055+0000 7f2e7c27c640 1 --2- 172.21.15.84:0/1517096769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e74159b20 0x7f2e7406bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.055+0000 7f2e7c27c640 1 -- 172.21.15.84:0/1517096769 >> 172.21.15.84:0/1517096769 conn(0x7f2e740cb410 msgr2=0x7f2e740cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:25.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.055+0000 7f2e7c27c640 1 -- 172.21.15.84:0/1517096769 shutdown_connections 2023-10-16T10:07:25.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.056+0000 7f2e7c27c640 1 -- 172.21.15.84:0/1517096769 wait complete. 2023-10-16T10:07:25.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.057+0000 7f2e7c27c640 1 Processor -- start 2023-10-16T10:07:25.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.057+0000 7f2e7c27c640 1 -- start start 2023-10-16T10:07:25.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.058+0000 7f2e7c27c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e740b0b20 0x7f2e7415a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.058+0000 7f2e7c27c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e74159b20 0x7f2e74157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.058+0000 7f2e7c27c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e74159f00 0x7f2e740c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.058+0000 7f2e7c27c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e740c70a0 con 0x7f2e74159f00 2023-10-16T10:07:25.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.058+0000 7f2e7c27c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7415b7c0 con 0x7f2e74159b20 2023-10-16T10:07:25.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.058+0000 7f2e7c27c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7415b200 con 0x7f2e740b0b20 2023-10-16T10:07:25.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.059+0000 7f2e7a819640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e74159f00 0x7f2e740c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:25.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.059+0000 7f2e7a819640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e74159f00 0x7f2e740c4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59770/0 (socket says 172.21.15.84:59770) 2023-10-16T10:07:25.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.059+0000 7f2e7a819640 1 -- 172.21.15.84:0/2449937387 learned_addr learned my addr 172.21.15.84:0/2449937387 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:25.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.059+0000 7f2e7a018640 1 --2- 172.21.15.84:0/2449937387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e740b0b20 0x7f2e7415a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:25.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.059+0000 7f2e79817640 1 --2- 172.21.15.84:0/2449937387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e74159b20 0x7f2e74157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:25.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.059+0000 7f2e7a819640 1 -- 172.21.15.84:0/2449937387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e740b0b20 msgr2=0x7f2e7415a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:25.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.059+0000 7f2e7a819640 1 --2- 172.21.15.84:0/2449937387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e740b0b20 0x7f2e7415a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.059+0000 7f2e7a819640 1 -- 172.21.15.84:0/2449937387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e74159b20 msgr2=0x7f2e74157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:25.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.059+0000 7f2e7a819640 1 --2- 172.21.15.84:0/2449937387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e74159b20 0x7f2e74157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.059+0000 7f2e7a819640 1 -- 172.21.15.84:0/2449937387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6c009a70 con 0x7f2e74159f00 2023-10-16T10:07:25.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.059+0000 7f2e7a819640 1 --2- 172.21.15.84:0/2449937387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e74159f00 0x7f2e740c4690 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009d90 tx=0x7f2e6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:25.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.060+0000 7f2e6affd640 1 -- 172.21.15.84:0/2449937387 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c040650 con 0x7f2e74159f00 2023-10-16T10:07:25.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.060+0000 7f2e6affd640 1 -- 172.21.15.84:0/2449937387 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6c0407d0 con 0x7f2e74159f00 2023-10-16T10:07:25.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.060+0000 7f2e6affd640 1 -- 172.21.15.84:0/2449937387 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c040b80 con 0x7f2e74159f00 2023-10-16T10:07:25.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.060+0000 7f2e7c27c640 1 -- 172.21.15.84:0/2449937387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e7415b4e0 con 0x7f2e74159f00 2023-10-16T10:07:25.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.060+0000 7f2e7c27c640 1 -- 172.21.15.84:0/2449937387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e7415a9a0 con 0x7f2e74159f00 2023-10-16T10:07:25.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.062+0000 7f2e7c27c640 1 -- 172.21.15.84:0/2449937387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e740c6b20 con 0x7f2e74159f00 2023-10-16T10:07:25.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.062+0000 7f2e6affd640 1 -- 172.21.15.84:0/2449937387 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2e6c040ce0 con 0x7f2e74159f00 2023-10-16T10:07:25.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.063+0000 7f2e6affd640 1 --2- 172.21.15.84:0/2449937387 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f2e5806af90 0x7f2e5806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.063+0000 7f2e6affd640 1 -- 172.21.15.84:0/2449937387 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2e6c0ac1d0 con 0x7f2e74159f00 2023-10-16T10:07:25.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.064+0000 7f2e7a018640 1 --2- 172.21.15.84:0/2449937387 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f2e5806af90 0x7f2e5806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:25.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.064+0000 7f2e7a018640 1 --2- 172.21.15.84:0/2449937387 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f2e5806af90 0x7f2e5806d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2e70006fd0 tx=0x7f2e70008040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:25.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.068+0000 7f2e6affd640 1 -- 172.21.15.84:0/2449937387 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2e740c6b20 con 0x7f2e74159f00 2023-10-16T10:07:25.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.259+0000 7f2e7c27c640 1 -- 172.21.15.84:0/2449937387 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e740c71e0 con 0x7f2e74159f00 2023-10-16T10:07:25.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.260+0000 7f2e6affd640 1 -- 172.21.15.84:0/2449937387 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+12843 (secure 0 0 0) 0x7f2e740c71e0 con 0x7f2e74159f00 2023-10-16T10:07:25.262 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:25.262 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":120,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:07:24.304270+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:07:25.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.262+0000 7f2e7c27c640 1 -- 172.21.15.84:0/2449937387 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f2e5806af90 msgr2=0x7f2e5806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:25.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.262+0000 7f2e7c27c640 1 --2- 172.21.15.84:0/2449937387 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f2e5806af90 0x7f2e5806d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2e70006fd0 tx=0x7f2e70008040 comp rx=0 tx=0).stop 2023-10-16T10:07:25.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.262+0000 7f2e7c27c640 1 -- 172.21.15.84:0/2449937387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e74159f00 msgr2=0x7f2e740c4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:25.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.262+0000 7f2e7c27c640 1 --2- 172.21.15.84:0/2449937387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e74159f00 0x7f2e740c4690 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009d90 tx=0x7f2e6c004a40 comp rx=0 tx=0).stop 2023-10-16T10:07:25.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.263+0000 7f2e7c27c640 1 -- 172.21.15.84:0/2449937387 shutdown_connections 2023-10-16T10:07:25.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.263+0000 7f2e7c27c640 1 --2- 172.21.15.84:0/2449937387 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f2e5806af90 0x7f2e5806d450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.263+0000 7f2e7c27c640 1 --2- 172.21.15.84:0/2449937387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e74159f00 0x7f2e740c4690 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.263+0000 7f2e7c27c640 1 --2- 172.21.15.84:0/2449937387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e74159b20 0x7f2e74157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.263+0000 7f2e7c27c640 1 --2- 172.21.15.84:0/2449937387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e740b0b20 0x7f2e7415a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.263+0000 7f2e7c27c640 1 -- 172.21.15.84:0/2449937387 >> 172.21.15.84:0/2449937387 conn(0x7f2e740cb410 msgr2=0x7f2e740cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:25.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.263+0000 7f2e7c27c640 1 -- 172.21.15.84:0/2449937387 shutdown_connections 2023-10-16T10:07:25.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.263+0000 7f2e7c27c640 1 -- 172.21.15.84:0/2449937387 wait complete. 2023-10-16T10:07:25.282 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:07:25.283 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:07:25.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.489+0000 7fccd9561640 1 Processor -- start 2023-10-16T10:07:25.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.489+0000 7fccd9561640 1 -- start start 2023-10-16T10:07:25.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.491+0000 7fccd9561640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fccd415ad70 0x7fccd4069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.491+0000 7fccd9561640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccd40c9ca0 0x7fccd406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.491+0000 7fccd9561640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd40cb030 con 0x7fccd40ca3b0 2023-10-16T10:07:25.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.491+0000 7fccd9561640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd40c91c0 con 0x7fccd415ad70 2023-10-16T10:07:25.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.491+0000 7fccd9561640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd40c94a0 con 0x7fccd40c9ca0 2023-10-16T10:07:25.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.491+0000 7fccd27fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccd40c9ca0 0x7fccd406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:25.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.491+0000 7fccd27fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccd40c9ca0 0x7fccd406a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:42600/0 (socket says 172.21.15.84:42600) 2023-10-16T10:07:25.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.491+0000 7fccd27fc640 1 -- 172.21.15.84:0/3728984705 learned_addr learned my addr 172.21.15.84:0/3728984705 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:25.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.491+0000 7fccd2ffd640 1 --2- 172.21.15.84:0/3728984705 >> v2:172.21.15.157:3300/0 conn(0x7fccd415ad70 0x7fccd4069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:25.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.492+0000 7fccd27fc640 1 -- 172.21.15.84:0/3728984705 >> v2:172.21.15.157:3300/0 conn(0x7fccd415ad70 msgr2=0x7fccd4069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:25.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.492+0000 7fccd27fc640 1 --2- 172.21.15.84:0/3728984705 >> v2:172.21.15.157:3300/0 conn(0x7fccd415ad70 0x7fccd4069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.492+0000 7fccd27fc640 1 -- 172.21.15.84:0/3728984705 >> v1:172.21.15.157:6789/0 conn(0x7fccd40ca3b0 legacy=0x7fccd40ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:25.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.492+0000 7fccd27fc640 1 -- 172.21.15.84:0/3728984705 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd40c9780 con 0x7fccd40c9ca0 2023-10-16T10:07:25.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.494+0000 7fccd27fc640 1 --2- 172.21.15.84:0/3728984705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccd40c9ca0 0x7fccd406a2e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009870 tx=0x7fccc802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19802b69f5442673 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:25.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.495+0000 7fccd1ffb640 1 -- 172.21.15.84:0/3728984705 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8030840 con 0x7fccd40c9ca0 2023-10-16T10:07:25.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.495+0000 7fccd1ffb640 1 -- 172.21.15.84:0/3728984705 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc80309a0 con 0x7fccd40c9ca0 2023-10-16T10:07:25.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.495+0000 7fccd1ffb640 1 -- 172.21.15.84:0/3728984705 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8030d30 con 0x7fccd40c9ca0 2023-10-16T10:07:25.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.495+0000 7fccd9561640 1 -- 172.21.15.84:0/3728984705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccd40c9ca0 msgr2=0x7fccd406a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:25.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.495+0000 7fccd9561640 1 --2- 172.21.15.84:0/3728984705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccd40c9ca0 0x7fccd406a2e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009870 tx=0x7fccc802b980 comp rx=0 tx=0).stop 2023-10-16T10:07:25.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.496+0000 7fccd9561640 1 -- 172.21.15.84:0/3728984705 shutdown_connections 2023-10-16T10:07:25.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.496+0000 7fccd9561640 1 --2- 172.21.15.84:0/3728984705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccd40c9ca0 0x7fccd406a2e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.496+0000 7fccd9561640 1 --2- 172.21.15.84:0/3728984705 >> v2:172.21.15.157:3300/0 conn(0x7fccd415ad70 0x7fccd4069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.496+0000 7fccd9561640 1 -- 172.21.15.84:0/3728984705 >> 172.21.15.84:0/3728984705 conn(0x7fccd40ce9b0 msgr2=0x7fccd40cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:25.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.496+0000 7fccd9561640 1 -- 172.21.15.84:0/3728984705 shutdown_connections 2023-10-16T10:07:25.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.496+0000 7fccd9561640 1 -- 172.21.15.84:0/3728984705 wait complete. 2023-10-16T10:07:25.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.497+0000 7fccd9561640 1 Processor -- start 2023-10-16T10:07:25.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.497+0000 7fccd9561640 1 -- start start 2023-10-16T10:07:25.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.498+0000 7fccd9561640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd40c9ca0 0x7fccd414ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.498+0000 7fccd9561640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccd40ca3b0 0x7fccd40b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.498+0000 7fccd9561640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccd415ad70 0x7fccd40b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.498+0000 7fccd9561640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd40b27d0 con 0x7fccd40c9ca0 2023-10-16T10:07:25.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.498+0000 7fccd9561640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd414d290 con 0x7fccd40ca3b0 2023-10-16T10:07:25.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.498+0000 7fccd9561640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd414c1f0 con 0x7fccd415ad70 2023-10-16T10:07:25.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.499+0000 7fccd2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd40c9ca0 0x7fccd414ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:25.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.499+0000 7fccd2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd40c9ca0 0x7fccd414ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59780/0 (socket says 172.21.15.84:59780) 2023-10-16T10:07:25.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.499+0000 7fccd2ffd640 1 -- 172.21.15.84:0/890372639 learned_addr learned my addr 172.21.15.84:0/890372639 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:25.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.499+0000 7fccd27fc640 1 --2- 172.21.15.84:0/890372639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccd40ca3b0 0x7fccd40b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:25.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.499+0000 7fccd37fe640 1 --2- 172.21.15.84:0/890372639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccd415ad70 0x7fccd40b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:25.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.499+0000 7fccd2ffd640 1 -- 172.21.15.84:0/890372639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccd415ad70 msgr2=0x7fccd40b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:25.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.499+0000 7fccd2ffd640 1 --2- 172.21.15.84:0/890372639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccd415ad70 0x7fccd40b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.499+0000 7fccd2ffd640 1 -- 172.21.15.84:0/890372639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccd40ca3b0 msgr2=0x7fccd40b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:25.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.499+0000 7fccd2ffd640 1 --2- 172.21.15.84:0/890372639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccd40ca3b0 0x7fccd40b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.499+0000 7fccd2ffd640 1 -- 172.21.15.84:0/890372639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc80094d0 con 0x7fccd40c9ca0 2023-10-16T10:07:25.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.500+0000 7fccd2ffd640 1 --2- 172.21.15.84:0/890372639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd40c9ca0 0x7fccd414ab50 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fccc400e870 tx=0x7fccc400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:25.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.500+0000 7fccbb7fe640 1 -- 172.21.15.84:0/890372639 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc40094f0 con 0x7fccd40c9ca0 2023-10-16T10:07:25.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.500+0000 7fccbb7fe640 1 -- 172.21.15.84:0/890372639 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc400c940 con 0x7fccd40c9ca0 2023-10-16T10:07:25.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.500+0000 7fccd9561640 1 -- 172.21.15.84:0/890372639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd414c4d0 con 0x7fccd40c9ca0 2023-10-16T10:07:25.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.500+0000 7fccbb7fe640 1 -- 172.21.15.84:0/890372639 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc400ccf0 con 0x7fccd40c9ca0 2023-10-16T10:07:25.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.500+0000 7fccd9561640 1 -- 172.21.15.84:0/890372639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd414ca90 con 0x7fccd40c9ca0 2023-10-16T10:07:25.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.501+0000 7fccd9561640 1 -- 172.21.15.84:0/890372639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd404e160 con 0x7fccd40c9ca0 2023-10-16T10:07:25.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.502+0000 7fccbb7fe640 1 -- 172.21.15.84:0/890372639 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fccc4013470 con 0x7fccd40c9ca0 2023-10-16T10:07:25.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.504+0000 7fccbb7fe640 1 --2- 172.21.15.84:0/890372639 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fccc006af90 0x7fccc006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.504+0000 7fccbb7fe640 1 -- 172.21.15.84:0/890372639 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fccc40894d0 con 0x7fccd40c9ca0 2023-10-16T10:07:25.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.504+0000 7fccd27fc640 1 --2- 172.21.15.84:0/890372639 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fccc006af90 0x7fccc006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:25.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.505+0000 7fccd27fc640 1 --2- 172.21.15.84:0/890372639 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fccc006af90 0x7fccc006d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fccc8036000 tx=0x7fccc8008030 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:25.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.508+0000 7fccbb7fe640 1 -- 172.21.15.84:0/890372639 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fccc4054850 con 0x7fccd40c9ca0 2023-10-16T10:07:25.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.698+0000 7fccd9561640 1 -- 172.21.15.84:0/890372639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fccd40ce570 con 0x7fccd40c9ca0 2023-10-16T10:07:25.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.699+0000 7fccbb7fe640 1 -- 172.21.15.84:0/890372639 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7fccc4089030 con 0x7fccd40c9ca0 2023-10-16T10:07:25.701 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:07:25.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.702+0000 7fccd9561640 1 -- 172.21.15.84:0/890372639 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fccc006af90 msgr2=0x7fccc006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:25.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.702+0000 7fccd9561640 1 --2- 172.21.15.84:0/890372639 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fccc006af90 0x7fccc006d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fccc8036000 tx=0x7fccc8008030 comp rx=0 tx=0).stop 2023-10-16T10:07:25.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.702+0000 7fccd9561640 1 -- 172.21.15.84:0/890372639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd40c9ca0 msgr2=0x7fccd414ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:25.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.702+0000 7fccd9561640 1 --2- 172.21.15.84:0/890372639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd40c9ca0 0x7fccd414ab50 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fccc400e870 tx=0x7fccc400ed30 comp rx=0 tx=0).stop 2023-10-16T10:07:25.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.702+0000 7fccd9561640 1 -- 172.21.15.84:0/890372639 shutdown_connections 2023-10-16T10:07:25.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.702+0000 7fccd9561640 1 --2- 172.21.15.84:0/890372639 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fccc006af90 0x7fccc006d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.702+0000 7fccd9561640 1 --2- 172.21.15.84:0/890372639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccd415ad70 0x7fccd40b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.702+0000 7fccd9561640 1 --2- 172.21.15.84:0/890372639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccd40ca3b0 0x7fccd40b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.702+0000 7fccd9561640 1 --2- 172.21.15.84:0/890372639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd40c9ca0 0x7fccd414ab50 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.702+0000 7fccd9561640 1 -- 172.21.15.84:0/890372639 >> 172.21.15.84:0/890372639 conn(0x7fccd40ce9b0 msgr2=0x7fccd40ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:25.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.702+0000 7fccd9561640 1 -- 172.21.15.84:0/890372639 shutdown_connections 2023-10-16T10:07:25.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.702+0000 7fccd9561640 1 -- 172.21.15.84:0/890372639 wait complete. 2023-10-16T10:07:25.720 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T10:07:25.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.928+0000 7f76e2856640 1 Processor -- start 2023-10-16T10:07:25.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.929+0000 7f76e2856640 1 -- start start 2023-10-16T10:07:25.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.930+0000 7f76e2856640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f76dc0b0b20 0x7f76dc164f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.930+0000 7f76e2856640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76dc071450 con 0x7f76dc070fd0 2023-10-16T10:07:25.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.930+0000 7f76e2856640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76dc0cc600 con 0x7f76dc0adb50 2023-10-16T10:07:25.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.930+0000 7f76e2856640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76dc0cc8e0 con 0x7f76dc0b0b20 2023-10-16T10:07:25.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.930+0000 7f76d3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f76dc070fd0 0x7f76dc0b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37794/0 (socket says 172.21.15.84:37794) 2023-10-16T10:07:25.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.931+0000 7f76d3fff640 1 -- 172.21.15.84:0/745723481 learned_addr learned my addr 172.21.15.84:0/745723481 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:25.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.931+0000 7f76e0df3640 1 --2- 172.21.15.84:0/745723481 >> v2:172.21.15.157:3300/0 conn(0x7f76dc0b0b20 0x7f76dc164f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:25.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.931+0000 7f76db7fe640 1 -- 172.21.15.84:0/745723481 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428575412 0 0) 0x7f76dc071450 con 0x7f76dc070fd0 2023-10-16T10:07:25.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.931+0000 7f76db7fe640 1 -- 172.21.15.84:0/745723481 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76c8003520 con 0x7f76dc070fd0 2023-10-16T10:07:25.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.931+0000 7f76db7fe640 1 -- 172.21.15.84:0/745723481 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1710779498 0 0) 0x7f76c8003520 con 0x7f76dc070fd0 2023-10-16T10:07:25.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.931+0000 7f76db7fe640 1 -- 172.21.15.84:0/745723481 >> v2:172.21.15.157:3300/0 conn(0x7f76dc0b0b20 msgr2=0x7f76dc164f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:25.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.931+0000 7f76db7fe640 1 --2- 172.21.15.84:0/745723481 >> v2:172.21.15.157:3300/0 conn(0x7f76dc0b0b20 0x7f76dc164f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.931+0000 7f76db7fe640 1 -- 172.21.15.84:0/745723481 >> v1:172.21.15.157:6789/0 conn(0x7f76dc0adb50 legacy=0x7f76dc06f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:25.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.931+0000 7f76db7fe640 1 -- 172.21.15.84:0/745723481 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc0ccbc0 con 0x7f76dc070fd0 2023-10-16T10:07:25.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.931+0000 7f76db7fe640 1 -- 172.21.15.84:0/745723481 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f76c4001670 con 0x7f76dc070fd0 2023-10-16T10:07:25.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.932+0000 7f76db7fe640 1 -- 172.21.15.84:0/745723481 >> v1:172.21.15.84:6789/0 conn(0x7f76dc070fd0 legacy=0x7f76dc0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:25.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.932+0000 7f76db7fe640 1 --2- 172.21.15.84:0/745723481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76c80039c0 0x7f76c8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.932+0000 7f76db7fe640 1 --2- 172.21.15.84:0/745723481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76c80047c0 0x7f76c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.932+0000 7f76db7fe640 1 --2- 172.21.15.84:0/745723481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76c8005180 0x7f76c800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.932+0000 7f76db7fe640 1 -- 172.21.15.84:0/745723481 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc0cc8e0 con 0x7f76c8005180 2023-10-16T10:07:25.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.932+0000 7f76db7fe640 1 -- 172.21.15.84:0/745723481 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c800c080 con 0x7f76c80047c0 2023-10-16T10:07:25.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.932+0000 7f76db7fe640 1 -- 172.21.15.84:0/745723481 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c800c240 con 0x7f76c80039c0 2023-10-16T10:07:25.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.932+0000 7f76e0df3640 1 --2- 172.21.15.84:0/745723481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76c8005180 0x7f76c800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:25.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.932+0000 7f76e0df3640 1 -- 172.21.15.84:0/745723481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76c80039c0 msgr2=0x7f76c8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:25.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.932+0000 7f76e0df3640 1 --2- 172.21.15.84:0/745723481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76c80039c0 0x7f76c8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.932+0000 7f76e0df3640 1 -- 172.21.15.84:0/745723481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76c80047c0 msgr2=0x7f76c8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:25.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.932+0000 7f76e0df3640 1 --2- 172.21.15.84:0/745723481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76c80047c0 0x7f76c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.932+0000 7f76e0df3640 1 -- 172.21.15.84:0/745723481 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c8003520 con 0x7f76c8005180 2023-10-16T10:07:25.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.933+0000 7f76e0df3640 1 -- 172.21.15.84:0/745723481 reap_dead start 2023-10-16T10:07:25.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.933+0000 7f76e0df3640 1 --2- 172.21.15.84:0/745723481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76c8005180 0x7f76c800b980 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009f90 tx=0x7f76c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd29585b5659cb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:25.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.934+0000 7f76db7fe640 1 -- 172.21.15.84:0/745723481 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d402cc20 con 0x7f76c8005180 2023-10-16T10:07:25.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.934+0000 7f76db7fe640 1 -- 172.21.15.84:0/745723481 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d400f040 con 0x7f76c8005180 2023-10-16T10:07:25.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.934+0000 7f76db7fe640 1 -- 172.21.15.84:0/745723481 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4003990 con 0x7f76c8005180 2023-10-16T10:07:25.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.934+0000 7f76e2856640 1 -- 172.21.15.84:0/745723481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76c8005180 msgr2=0x7f76c800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:25.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.934+0000 7f76e2856640 1 --2- 172.21.15.84:0/745723481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76c8005180 0x7f76c800b980 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009f90 tx=0x7f76c80057f0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.934+0000 7f76e2856640 1 -- 172.21.15.84:0/745723481 shutdown_connections 2023-10-16T10:07:25.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.934+0000 7f76e2856640 1 --2- 172.21.15.84:0/745723481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76c8005180 0x7f76c800b980 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.934+0000 7f76e2856640 1 -- 172.21.15.84:0/745723481 >> 172.21.15.84:0/745723481 conn(0x7f76dc0cb410 msgr2=0x7f76dc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:25.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.934+0000 7f76e2856640 1 -- 172.21.15.84:0/745723481 shutdown_connections 2023-10-16T10:07:25.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.934+0000 7f76e2856640 1 -- 172.21.15.84:0/745723481 wait complete. 2023-10-16T10:07:25.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.936+0000 7f76e2856640 1 Processor -- start 2023-10-16T10:07:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.936+0000 7f76e2856640 1 -- start start 2023-10-16T10:07:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.937+0000 7f76e2856640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76c8005180 0x7f76dc0c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.937+0000 7f76e2856640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76dc0c5a60 0x7f76dc1fa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.937+0000 7f76e2856640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76dc1f82e0 0x7f76dc1fa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.937+0000 7f76e2856640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc0c85f0 con 0x7f76dc1f82e0 2023-10-16T10:07:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.937+0000 7f76e2856640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc0c8820 con 0x7f76c8005180 2023-10-16T10:07:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.937+0000 7f76e2856640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc0c8b00 con 0x7f76dc0c5a60 2023-10-16T10:07:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.938+0000 7f76e0df3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76dc1f82e0 0x7f76dc1fa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.938+0000 7f76e0df3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76dc1f82e0 0x7f76dc1fa920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51974/0 (socket says 172.21.15.84:51974) 2023-10-16T10:07:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.938+0000 7f76e0df3640 1 -- 172.21.15.84:0/1327566006 learned_addr learned my addr 172.21.15.84:0/1327566006 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:25.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.938+0000 7f76d3fff640 1 --2- 172.21.15.84:0/1327566006 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76dc0c5a60 0x7f76dc1fa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:25.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.938+0000 7f76dbfff640 1 --2- 172.21.15.84:0/1327566006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76c8005180 0x7f76dc0c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:25.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.938+0000 7f76e0df3640 1 -- 172.21.15.84:0/1327566006 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76dc0c5a60 msgr2=0x7f76dc1fa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:25.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.938+0000 7f76e0df3640 1 --2- 172.21.15.84:0/1327566006 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76dc0c5a60 0x7f76dc1fa3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.938+0000 7f76e0df3640 1 -- 172.21.15.84:0/1327566006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76c8005180 msgr2=0x7f76dc0c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:25.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.938+0000 7f76e0df3640 1 --2- 172.21.15.84:0/1327566006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76c8005180 0x7f76dc0c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:25.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.938+0000 7f76e0df3640 1 -- 172.21.15.84:0/1327566006 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc0b24d0 con 0x7f76dc1f82e0 2023-10-16T10:07:25.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.938+0000 7f76dbfff640 1 --2- 172.21.15.84:0/1327566006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76c8005180 0x7f76dc0c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:25.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.938+0000 7f76e0df3640 1 --2- 172.21.15.84:0/1327566006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76dc1f82e0 0x7f76dc1fa920 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f76d402c690 tx=0x7f76d4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:25.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.939+0000 7f76d97fa640 1 -- 172.21.15.84:0/1327566006 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4003da0 con 0x7f76dc1f82e0 2023-10-16T10:07:25.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.939+0000 7f76d97fa640 1 -- 172.21.15.84:0/1327566006 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d40358d0 con 0x7f76dc1f82e0 2023-10-16T10:07:25.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.939+0000 7f76e2856640 1 -- 172.21.15.84:0/1327566006 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76dc0b27b0 con 0x7f76dc1f82e0 2023-10-16T10:07:25.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.939+0000 7f76d97fa640 1 -- 172.21.15.84:0/1327566006 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4035c60 con 0x7f76dc1f82e0 2023-10-16T10:07:25.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.939+0000 7f76e2856640 1 -- 172.21.15.84:0/1327566006 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76dc0b2fb0 con 0x7f76dc1f82e0 2023-10-16T10:07:25.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.940+0000 7f76e2856640 1 -- 172.21.15.84:0/1327566006 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76dc04e160 con 0x7f76dc1f82e0 2023-10-16T10:07:25.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.941+0000 7f76d97fa640 1 -- 172.21.15.84:0/1327566006 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f76d4035dc0 con 0x7f76dc1f82e0 2023-10-16T10:07:25.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.942+0000 7f76d97fa640 1 --2- 172.21.15.84:0/1327566006 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f76bc06afc0 0x7f76bc06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:25.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.942+0000 7f76d97fa640 1 -- 172.21.15.84:0/1327566006 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f76d40ac190 con 0x7f76dc1f82e0 2023-10-16T10:07:25.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.942+0000 7f76dbfff640 1 --2- 172.21.15.84:0/1327566006 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f76bc06afc0 0x7f76bc06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:25.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.943+0000 7f76dbfff640 1 --2- 172.21.15.84:0/1327566006 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f76bc06afc0 0x7f76bc06d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0b43e0 tx=0x7f76cc009040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:25.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:25.946+0000 7f76d97fa640 1 -- 172.21.15.84:0/1327566006 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f76d40773e0 con 0x7f76dc1f82e0 2023-10-16T10:07:26.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.124+0000 7f76e2856640 1 -- 172.21.15.84:0/1327566006 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f76dc164190 con 0x7f76dc1f82e0 2023-10-16T10:07:26.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.125+0000 7f76d97fa640 1 -- 172.21.15.84:0/1327566006 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v120) v1 ==== 84+0+65 (secure 0 0 0) 0x7f76d407aa00 con 0x7f76dc1f82e0 2023-10-16T10:07:26.127 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:26.127 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T10:07:26.127 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:26.127 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-16T10:07:26.127 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-16T10:07:26.128 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:26.128 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T10:07:26.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.128+0000 7f76e2856640 1 -- 172.21.15.84:0/1327566006 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f76bc06afc0 msgr2=0x7f76bc06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:26.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.128+0000 7f76e2856640 1 --2- 172.21.15.84:0/1327566006 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f76bc06afc0 0x7f76bc06d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0b43e0 tx=0x7f76cc009040 comp rx=0 tx=0).stop 2023-10-16T10:07:26.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.128+0000 7f76e2856640 1 -- 172.21.15.84:0/1327566006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76dc1f82e0 msgr2=0x7f76dc1fa920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:26.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.128+0000 7f76e2856640 1 --2- 172.21.15.84:0/1327566006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76dc1f82e0 0x7f76dc1fa920 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f76d402c690 tx=0x7f76d4009750 comp rx=0 tx=0).stop 2023-10-16T10:07:26.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.128+0000 7f76e2856640 1 -- 172.21.15.84:0/1327566006 shutdown_connections 2023-10-16T10:07:26.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.128+0000 7f76e2856640 1 --2- 172.21.15.84:0/1327566006 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f76bc06afc0 0x7f76bc06d480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:26.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.128+0000 7f76e2856640 1 --2- 172.21.15.84:0/1327566006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76dc1f82e0 0x7f76dc1fa920 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:26.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.128+0000 7f76e2856640 1 --2- 172.21.15.84:0/1327566006 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76dc0c5a60 0x7f76dc1fa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:26.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.128+0000 7f76e2856640 1 --2- 172.21.15.84:0/1327566006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76c8005180 0x7f76dc0c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:26.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.128+0000 7f76e2856640 1 -- 172.21.15.84:0/1327566006 >> 172.21.15.84:0/1327566006 conn(0x7f76dc0cb410 msgr2=0x7f76dc0cd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:26.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.128+0000 7f76e2856640 1 -- 172.21.15.84:0/1327566006 shutdown_connections 2023-10-16T10:07:26.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.128+0000 7f76e2856640 1 -- 172.21.15.84:0/1327566006 wait complete. 2023-10-16T10:07:26.147 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T10:07:26.148 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T10:07:26.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.358+0000 7f2da4a55640 1 Processor -- start 2023-10-16T10:07:26.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.359+0000 7f2da4a55640 1 -- start start 2023-10-16T10:07:26.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.360+0000 7f2da4a55640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2da00cd900 0x7f2da00cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:26.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.360+0000 7f2da4a55640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2da00d1030 0x7f2da00d1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:26.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.360+0000 7f2da4a55640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da0160ca0 con 0x7f2da015d0e0 2023-10-16T10:07:26.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.360+0000 7f2da4a55640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da015d4c0 con 0x7f2da00cd900 2023-10-16T10:07:26.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.360+0000 7f2da4a55640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da00ce2d0 con 0x7f2da00d1030 2023-10-16T10:07:26.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.360+0000 7f2d9e59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2da00cd900 0x7f2da00cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:26.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.361+0000 7f2d9e59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2da00cd900 0x7f2da00cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51984/0 (socket says 172.21.15.84:51984) 2023-10-16T10:07:26.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.361+0000 7f2d9e59c640 1 -- 172.21.15.84:0/4066913518 learned_addr learned my addr 172.21.15.84:0/4066913518 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:26.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.361+0000 7f2d9f59e640 1 --2- 172.21.15.84:0/4066913518 >> v2:172.21.15.157:3300/0 conn(0x7f2da00d1030 0x7f2da00d1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:26.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.361+0000 7f2d9e59c640 1 -- 172.21.15.84:0/4066913518 >> v2:172.21.15.157:3300/0 conn(0x7f2da00d1030 msgr2=0x7f2da00d1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:26.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.361+0000 7f2d9e59c640 1 --2- 172.21.15.84:0/4066913518 >> v2:172.21.15.157:3300/0 conn(0x7f2da00d1030 0x7f2da00d1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:26.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.361+0000 7f2d9e59c640 1 -- 172.21.15.84:0/4066913518 >> v1:172.21.15.157:6789/0 conn(0x7f2da015d0e0 legacy=0x7f2da00b84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:26.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.361+0000 7f2d9e59c640 1 -- 172.21.15.84:0/4066913518 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da0160f80 con 0x7f2da00cd900 2023-10-16T10:07:26.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.362+0000 7f2d9e59c640 1 --2- 172.21.15.84:0/4066913518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da00cd900 0x7f2da00cdd90 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f2da00d04f0 tx=0x7f2d8802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6577b1e394d9eab server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:26.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.362+0000 7f2d9dd9b640 1 -- 172.21.15.84:0/4066913518 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d88031870 con 0x7f2da00cd900 2023-10-16T10:07:26.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.362+0000 7f2d9dd9b640 1 -- 172.21.15.84:0/4066913518 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d880319d0 con 0x7f2da00cd900 2023-10-16T10:07:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.362+0000 7f2d9dd9b640 1 -- 172.21.15.84:0/4066913518 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d88031d60 con 0x7f2da00cd900 2023-10-16T10:07:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.363+0000 7f2da4a55640 1 -- 172.21.15.84:0/4066913518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da00cd900 msgr2=0x7f2da00cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.363+0000 7f2da4a55640 1 --2- 172.21.15.84:0/4066913518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da00cd900 0x7f2da00cdd90 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f2da00d04f0 tx=0x7f2d8802b910 comp rx=0 tx=0).stop 2023-10-16T10:07:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.363+0000 7f2da4a55640 1 -- 172.21.15.84:0/4066913518 shutdown_connections 2023-10-16T10:07:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.363+0000 7f2da4a55640 1 --2- 172.21.15.84:0/4066913518 >> v2:172.21.15.157:3300/0 conn(0x7f2da00d1030 0x7f2da00d1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.363+0000 7f2da4a55640 1 --2- 172.21.15.84:0/4066913518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da00cd900 0x7f2da00cdd90 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.363+0000 7f2da4a55640 1 -- 172.21.15.84:0/4066913518 >> 172.21.15.84:0/4066913518 conn(0x7f2da006bc60 msgr2=0x7f2da006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.363+0000 7f2da4a55640 1 -- 172.21.15.84:0/4066913518 shutdown_connections 2023-10-16T10:07:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.363+0000 7f2da4a55640 1 -- 172.21.15.84:0/4066913518 wait complete. 2023-10-16T10:07:26.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.365+0000 7f2da4a55640 1 Processor -- start 2023-10-16T10:07:26.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.365+0000 7f2da4a55640 1 -- start start 2023-10-16T10:07:26.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.366+0000 7f2da4a55640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da00cd900 0x7f2da01f2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:26.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.366+0000 7f2da4a55640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da00d1030 0x7f2da015b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:26.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.367+0000 7f2da4a55640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2da015d0e0 0x7f2da00b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:26.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.367+0000 7f2da4a55640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00b36a0 con 0x7f2da00d1030 2023-10-16T10:07:26.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.367+0000 7f2da4a55640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00b3b30 con 0x7f2da00cd900 2023-10-16T10:07:26.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.367+0000 7f2da4a55640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00b2a90 con 0x7f2da015d0e0 2023-10-16T10:07:26.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.367+0000 7f2d9e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da00d1030 0x7f2da015b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:26.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.367+0000 7f2d9e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da00d1030 0x7f2da015b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51986/0 (socket says 172.21.15.84:51986) 2023-10-16T10:07:26.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.367+0000 7f2d9e59c640 1 -- 172.21.15.84:0/3284042707 learned_addr learned my addr 172.21.15.84:0/3284042707 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:26.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.367+0000 7f2d9e59c640 1 -- 172.21.15.84:0/3284042707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2da015d0e0 msgr2=0x7f2da00b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:26.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.367+0000 7f2d9e59c640 1 --2- 172.21.15.84:0/3284042707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2da015d0e0 0x7f2da00b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:26.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.367+0000 7f2d9e59c640 1 -- 172.21.15.84:0/3284042707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da00cd900 msgr2=0x7f2da01f2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:26.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.367+0000 7f2d9e59c640 1 --2- 172.21.15.84:0/3284042707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da00cd900 0x7f2da01f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:26.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.367+0000 7f2d9e59c640 1 -- 172.21.15.84:0/3284042707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d880094d0 con 0x7f2da00d1030 2023-10-16T10:07:26.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.367+0000 7f2d9e59c640 1 --2- 172.21.15.84:0/3284042707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da00d1030 0x7f2da015b700 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f2d88000c00 tx=0x7f2d880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:26.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.368+0000 7f2d977fe640 1 -- 172.21.15.84:0/3284042707 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d88031840 con 0x7f2da00d1030 2023-10-16T10:07:26.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.368+0000 7f2d977fe640 1 -- 172.21.15.84:0/3284042707 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d880443f0 con 0x7f2da00d1030 2023-10-16T10:07:26.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.368+0000 7f2d977fe640 1 -- 172.21.15.84:0/3284042707 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d880447a0 con 0x7f2da00d1030 2023-10-16T10:07:26.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.368+0000 7f2da4a55640 1 -- 172.21.15.84:0/3284042707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da00b2d70 con 0x7f2da00d1030 2023-10-16T10:07:26.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.368+0000 7f2da4a55640 1 -- 172.21.15.84:0/3284042707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da00b3330 con 0x7f2da00d1030 2023-10-16T10:07:26.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.369+0000 7f2da4a55640 1 -- 172.21.15.84:0/3284042707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da0072190 con 0x7f2da00d1030 2023-10-16T10:07:26.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.370+0000 7f2d977fe640 1 -- 172.21.15.84:0/3284042707 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2d8804d450 con 0x7f2da00d1030 2023-10-16T10:07:26.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.371+0000 7f2d977fe640 1 --2- 172.21.15.84:0/3284042707 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f2d8006af90 0x7f2d8006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:26.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.371+0000 7f2d977fe640 1 -- 172.21.15.84:0/3284042707 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2d8802d040 con 0x7f2da00d1030 2023-10-16T10:07:26.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.372+0000 7f2d9ed9d640 1 --2- 172.21.15.84:0/3284042707 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f2d8006af90 0x7f2d8006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:26.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.372+0000 7f2d9ed9d640 1 --2- 172.21.15.84:0/3284042707 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f2d8006af90 0x7f2d8006d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2da00d27d0 tx=0x7f2d90007d10 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:26.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.376+0000 7f2d977fe640 1 -- 172.21.15.84:0/3284042707 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2d88076e70 con 0x7f2da00d1030 2023-10-16T10:07:26.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:26.553+0000 7f2da4a55640 1 -- 172.21.15.84:0/3284042707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2da004e9b0 con 0x7f2da00d1030 2023-10-16T10:07:27.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.318+0000 7f2d977fe640 1 -- 172.21.15.84:0/3284042707 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v121) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2d88035020 con 0x7f2da00d1030 2023-10-16T10:07:27.321 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-16T10:07:27.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.323+0000 7f2da4a55640 1 -- 172.21.15.84:0/3284042707 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f2d8006af90 msgr2=0x7f2d8006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:27.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.323+0000 7f2da4a55640 1 --2- 172.21.15.84:0/3284042707 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f2d8006af90 0x7f2d8006d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2da00d27d0 tx=0x7f2d90007d10 comp rx=0 tx=0).stop 2023-10-16T10:07:27.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.323+0000 7f2da4a55640 1 -- 172.21.15.84:0/3284042707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da00d1030 msgr2=0x7f2da015b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:27.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.323+0000 7f2da4a55640 1 --2- 172.21.15.84:0/3284042707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da00d1030 0x7f2da015b700 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f2d88000c00 tx=0x7f2d880046c0 comp rx=0 tx=0).stop 2023-10-16T10:07:27.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.324+0000 7f2da4a55640 1 -- 172.21.15.84:0/3284042707 shutdown_connections 2023-10-16T10:07:27.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.324+0000 7f2da4a55640 1 --2- 172.21.15.84:0/3284042707 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f2d8006af90 0x7f2d8006d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:27.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.324+0000 7f2da4a55640 1 --2- 172.21.15.84:0/3284042707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2da015d0e0 0x7f2da00b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:27.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.324+0000 7f2da4a55640 1 --2- 172.21.15.84:0/3284042707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da00d1030 0x7f2da015b700 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:27.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.324+0000 7f2da4a55640 1 --2- 172.21.15.84:0/3284042707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da00cd900 0x7f2da01f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:27.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.324+0000 7f2da4a55640 1 -- 172.21.15.84:0/3284042707 >> 172.21.15.84:0/3284042707 conn(0x7f2da006bc60 msgr2=0x7f2da00d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:27.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.325+0000 7f2da4a55640 1 -- 172.21.15.84:0/3284042707 shutdown_connections 2023-10-16T10:07:27.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.325+0000 7f2da4a55640 1 -- 172.21.15.84:0/3284042707 wait complete. 2023-10-16T10:07:27.354 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T10:07:27.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.562+0000 7febea741640 1 Processor -- start 2023-10-16T10:07:27.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.563+0000 7febea741640 1 -- start start 2023-10-16T10:07:27.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.564+0000 7febea741640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7febe40adbe0 0x7febe40cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:27.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.564+0000 7febea741640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febe40adfc0 0x7febe40b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:27.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.564+0000 7febea741640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe40cc600 con 0x7febe4070fd0 2023-10-16T10:07:27.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.564+0000 7febea741640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe4071470 con 0x7febe40adbe0 2023-10-16T10:07:27.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.564+0000 7febea741640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe40cc870 con 0x7febe40adfc0 2023-10-16T10:07:27.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.565+0000 7febe3fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7febe40adbe0 0x7febe40cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:27.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.565+0000 7febe8cde640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febe40adfc0 0x7febe40b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:27.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.565+0000 7febe3fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7febe40adbe0 0x7febe40cd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:38158/0 (socket says 172.21.15.84:38158) 2023-10-16T10:07:27.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.565+0000 7febe3fff640 1 -- 172.21.15.84:0/2633392582 learned_addr learned my addr 172.21.15.84:0/2633392582 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:27.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.566+0000 7febe3fff640 1 -- 172.21.15.84:0/2633392582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febe40adfc0 msgr2=0x7febe40b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:27.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.566+0000 7febe3fff640 1 --2- 172.21.15.84:0/2633392582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febe40adfc0 0x7febe40b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:27.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.566+0000 7febe3fff640 1 -- 172.21.15.84:0/2633392582 >> v1:172.21.15.157:6789/0 conn(0x7febe4070fd0 legacy=0x7febe406f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:27.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.566+0000 7febe3fff640 1 -- 172.21.15.84:0/2633392582 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe40ccb50 con 0x7febe40adbe0 2023-10-16T10:07:27.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.566+0000 7febe8cde640 1 --2- 172.21.15.84:0/2633392582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febe40adfc0 0x7febe40b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:07:27.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.568+0000 7febe3fff640 1 --2- 172.21.15.84:0/2633392582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febe40adbe0 0x7febe40cd070 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7febd4009870 tx=0x7febd402b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=112afcf575df12ba server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:27.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.568+0000 7febe2ffd640 1 -- 172.21.15.84:0/2633392582 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd4030710 con 0x7febe40adbe0 2023-10-16T10:07:27.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.569+0000 7febe2ffd640 1 -- 172.21.15.84:0/2633392582 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd402e050 con 0x7febe40adbe0 2023-10-16T10:07:27.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.569+0000 7febe2ffd640 1 -- 172.21.15.84:0/2633392582 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd4030a90 con 0x7febe40adbe0 2023-10-16T10:07:27.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.569+0000 7febea741640 1 -- 172.21.15.84:0/2633392582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febe40adbe0 msgr2=0x7febe40cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:27.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.569+0000 7febea741640 1 --2- 172.21.15.84:0/2633392582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febe40adbe0 0x7febe40cd070 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7febd4009870 tx=0x7febd402b7c0 comp rx=0 tx=0).stop 2023-10-16T10:07:27.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.569+0000 7febea741640 1 -- 172.21.15.84:0/2633392582 shutdown_connections 2023-10-16T10:07:27.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.569+0000 7febea741640 1 --2- 172.21.15.84:0/2633392582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febe40adfc0 0x7febe40b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:27.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.569+0000 7febea741640 1 --2- 172.21.15.84:0/2633392582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febe40adbe0 0x7febe40cd070 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:27.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.570+0000 7febea741640 1 -- 172.21.15.84:0/2633392582 >> 172.21.15.84:0/2633392582 conn(0x7febe40cb410 msgr2=0x7febe40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:27.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.570+0000 7febea741640 1 -- 172.21.15.84:0/2633392582 shutdown_connections 2023-10-16T10:07:27.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.570+0000 7febea741640 1 -- 172.21.15.84:0/2633392582 wait complete. 2023-10-16T10:07:27.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.572+0000 7febea741640 1 Processor -- start 2023-10-16T10:07:27.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.572+0000 7febea741640 1 -- start start 2023-10-16T10:07:27.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.573+0000 7febea741640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febe4070fd0 0x7febe40be9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:27.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.573+0000 7febea741640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febe40adbe0 0x7febe40c0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:27.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.573+0000 7febea741640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febe40adfc0 0x7febe4156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:27.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.573+0000 7febea741640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe40c0770 con 0x7febe4070fd0 2023-10-16T10:07:27.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.573+0000 7febea741640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe40c3860 con 0x7febe40adfc0 2023-10-16T10:07:27.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.573+0000 7febea741640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe40c0950 con 0x7febe40adbe0 2023-10-16T10:07:27.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.573+0000 7febe3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febe4070fd0 0x7febe40be9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:27.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.573+0000 7febe3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febe4070fd0 0x7febe40be9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52000/0 (socket says 172.21.15.84:52000) 2023-10-16T10:07:27.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.573+0000 7febe3fff640 1 -- 172.21.15.84:0/2932816639 learned_addr learned my addr 172.21.15.84:0/2932816639 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:27.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.574+0000 7febe8cde640 1 --2- 172.21.15.84:0/2932816639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febe40adfc0 0x7febe4156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:27.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.574+0000 7febe37fe640 1 --2- 172.21.15.84:0/2932816639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febe40adbe0 0x7febe40c0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:27.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.574+0000 7febe3fff640 1 -- 172.21.15.84:0/2932816639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febe40adbe0 msgr2=0x7febe40c0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:27.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.574+0000 7febe3fff640 1 --2- 172.21.15.84:0/2932816639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febe40adbe0 0x7febe40c0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:27.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.574+0000 7febe3fff640 1 -- 172.21.15.84:0/2932816639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febe40adfc0 msgr2=0x7febe4156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:27.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.574+0000 7febe3fff640 1 --2- 172.21.15.84:0/2932816639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febe40adfc0 0x7febe4156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:27.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.574+0000 7febe3fff640 1 -- 172.21.15.84:0/2932816639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febdc009a70 con 0x7febe4070fd0 2023-10-16T10:07:27.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.574+0000 7febe3fff640 1 --2- 172.21.15.84:0/2932816639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febe4070fd0 0x7febe40be9d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7febd4000c00 tx=0x7febd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:27.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.574+0000 7febe0ff9640 1 -- 172.21.15.84:0/2932816639 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd403b810 con 0x7febe4070fd0 2023-10-16T10:07:27.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.575+0000 7febe0ff9640 1 -- 172.21.15.84:0/2932816639 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd403b990 con 0x7febe4070fd0 2023-10-16T10:07:27.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.575+0000 7febe0ff9640 1 -- 172.21.15.84:0/2932816639 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd403bd40 con 0x7febe4070fd0 2023-10-16T10:07:27.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.575+0000 7febea741640 1 -- 172.21.15.84:0/2932816639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd40094d0 con 0x7febe4070fd0 2023-10-16T10:07:27.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.575+0000 7febea741640 1 -- 172.21.15.84:0/2932816639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe40c0f10 con 0x7febe4070fd0 2023-10-16T10:07:27.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.576+0000 7febea741640 1 -- 172.21.15.84:0/2932816639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe404e160 con 0x7febe4070fd0 2023-10-16T10:07:27.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.577+0000 7febe0ff9640 1 -- 172.21.15.84:0/2932816639 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7febd4039040 con 0x7febe4070fd0 2023-10-16T10:07:27.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.578+0000 7febe0ff9640 1 --2- 172.21.15.84:0/2932816639 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7febd006afc0 0x7febd006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:27.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.578+0000 7febe0ff9640 1 -- 172.21.15.84:0/2932816639 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5227+0+0 (secure 0 0 0) 0x7febd40aaed0 con 0x7febe4070fd0 2023-10-16T10:07:27.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.578+0000 7febe37fe640 1 --2- 172.21.15.84:0/2932816639 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7febd006afc0 0x7febd006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:27.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.579+0000 7febe37fe640 1 --2- 172.21.15.84:0/2932816639 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7febd006afc0 0x7febd006d480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7febe406c560 tx=0x7febd8009040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:27.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.583+0000 7febe0ff9640 1 -- 172.21.15.84:0/2932816639 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7febd4075fc0 con 0x7febe4070fd0 2023-10-16T10:07:27.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:27.776+0000 7febea741640 1 -- 172.21.15.84:0/2932816639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7febe40b4910 con 0x7febe4070fd0 2023-10-16T10:07:28.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.330+0000 7febe0ff9640 1 -- 172.21.15.84:0/2932816639 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v122) v1 ==== 170+0+0 (secure 0 0 0) 0x7febd40370d0 con 0x7febe4070fd0 2023-10-16T10:07:28.333 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-16T10:07:28.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.335+0000 7febea741640 1 -- 172.21.15.84:0/2932816639 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7febd006afc0 msgr2=0x7febd006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:28.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.335+0000 7febea741640 1 --2- 172.21.15.84:0/2932816639 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7febd006afc0 0x7febd006d480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7febe406c560 tx=0x7febd8009040 comp rx=0 tx=0).stop 2023-10-16T10:07:28.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.335+0000 7febea741640 1 -- 172.21.15.84:0/2932816639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febe4070fd0 msgr2=0x7febe40be9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:28.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.335+0000 7febea741640 1 --2- 172.21.15.84:0/2932816639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febe4070fd0 0x7febe40be9d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7febd4000c00 tx=0x7febd40046c0 comp rx=0 tx=0).stop 2023-10-16T10:07:28.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.335+0000 7febea741640 1 -- 172.21.15.84:0/2932816639 shutdown_connections 2023-10-16T10:07:28.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.335+0000 7febea741640 1 --2- 172.21.15.84:0/2932816639 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7febd006afc0 0x7febd006d480 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:28.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.335+0000 7febea741640 1 --2- 172.21.15.84:0/2932816639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febe40adfc0 0x7febe4156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:28.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.335+0000 7febea741640 1 --2- 172.21.15.84:0/2932816639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7febe40adbe0 0x7febe40c0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:28.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.335+0000 7febea741640 1 --2- 172.21.15.84:0/2932816639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febe4070fd0 0x7febe40be9d0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:28.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.335+0000 7febea741640 1 -- 172.21.15.84:0/2932816639 >> 172.21.15.84:0/2932816639 conn(0x7febe40cb410 msgr2=0x7febe40b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:28.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.336+0000 7febea741640 1 -- 172.21.15.84:0/2932816639 shutdown_connections 2023-10-16T10:07:28.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.337+0000 7febea741640 1 -- 172.21.15.84:0/2932816639 wait complete. 2023-10-16T10:07:28.372 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T10:07:28.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.557+0000 7f47e0889640 1 Processor -- start 2023-10-16T10:07:28.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.557+0000 7f47e0889640 1 -- start start 2023-10-16T10:07:28.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.559+0000 7f47e0889640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f47dc0b2210 0x7f47dc0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:28.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.559+0000 7f47e0889640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47dc0d0820 con 0x7f47dc0b46d0 2023-10-16T10:07:28.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.559+0000 7f47e0889640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47dc0ce5c0 con 0x7f47dc0b1c50 2023-10-16T10:07:28.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.559+0000 7f47e0889640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47dc0d0a70 con 0x7f47dc0b2210 2023-10-16T10:07:28.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.559+0000 7f47da59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f47dc0b46d0 0x7f47dc0b4ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37806/0 (socket says 172.21.15.84:37806) 2023-10-16T10:07:28.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.559+0000 7f47d3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f47dc0b2210 0x7f47dc0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:28.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.559+0000 7f47da59c640 1 -- 172.21.15.84:0/2557800741 learned_addr learned my addr 172.21.15.84:0/2557800741 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:28.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.560+0000 7f47d3fff640 1 -- 172.21.15.84:0/2557800741 >> v1:172.21.15.157:6789/0 conn(0x7f47dc0b1c50 legacy=0x7f47dc0d1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:28.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.560+0000 7f47d9d9b640 1 -- 172.21.15.84:0/2557800741 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70338779 0 0) 0x7f47dc0d0820 con 0x7f47dc0b46d0 2023-10-16T10:07:28.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.560+0000 7f47d3fff640 1 -- 172.21.15.84:0/2557800741 >> v1:172.21.15.84:6789/0 conn(0x7f47dc0b46d0 legacy=0x7f47dc0b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.560+0000 7f47d3fff640 1 -- 172.21.15.84:0/2557800741 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc0d0d50 con 0x7f47dc0b2210 2023-10-16T10:07:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.561+0000 7f47d3fff640 1 --2- 172.21.15.84:0/2557800741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47dc0b2210 0x7f47dc0d1550 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f47c40097a0 tx=0x7f47c402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e431716d4373b68 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.561+0000 7f47d9d9b640 1 -- 172.21.15.84:0/2557800741 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c402e070 con 0x7f47dc0b2210 2023-10-16T10:07:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.561+0000 7f47d9d9b640 1 -- 172.21.15.84:0/2557800741 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c40315a0 con 0x7f47dc0b2210 2023-10-16T10:07:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.561+0000 7f47d9d9b640 1 -- 172.21.15.84:0/2557800741 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4031930 con 0x7f47dc0b2210 2023-10-16T10:07:28.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.562+0000 7f47e0889640 1 -- 172.21.15.84:0/2557800741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47dc0b2210 msgr2=0x7f47dc0d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:28.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.562+0000 7f47e0889640 1 --2- 172.21.15.84:0/2557800741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47dc0b2210 0x7f47dc0d1550 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f47c40097a0 tx=0x7f47c402b840 comp rx=0 tx=0).stop 2023-10-16T10:07:28.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.562+0000 7f47e0889640 1 -- 172.21.15.84:0/2557800741 shutdown_connections 2023-10-16T10:07:28.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.562+0000 7f47e0889640 1 --2- 172.21.15.84:0/2557800741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47dc0b2210 0x7f47dc0d1550 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:28.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.562+0000 7f47e0889640 1 -- 172.21.15.84:0/2557800741 >> 172.21.15.84:0/2557800741 conn(0x7f47dc0cf6d0 msgr2=0x7f47dc0cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:28.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.562+0000 7f47e0889640 1 -- 172.21.15.84:0/2557800741 shutdown_connections 2023-10-16T10:07:28.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.562+0000 7f47e0889640 1 -- 172.21.15.84:0/2557800741 wait complete. 2023-10-16T10:07:28.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.564+0000 7f47e0889640 1 Processor -- start 2023-10-16T10:07:28.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.564+0000 7f47e0889640 1 -- start start 2023-10-16T10:07:28.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.565+0000 7f47e0889640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47dc0b1c50 0x7f47dc0cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:28.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.565+0000 7f47e0889640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47dc0b2210 0x7f47dc0ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:28.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.565+0000 7f47e0889640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47dc0b46d0 0x7f47dc0ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:28.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.565+0000 7f47e0889640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc149700 con 0x7f47dc0b2210 2023-10-16T10:07:28.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.565+0000 7f47e0889640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc148e60 con 0x7f47dc0b46d0 2023-10-16T10:07:28.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.565+0000 7f47e0889640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc149140 con 0x7f47dc0b1c50 2023-10-16T10:07:28.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.565+0000 7f47d3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47dc0b2210 0x7f47dc0ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:28.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.565+0000 7f47d3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47dc0b2210 0x7f47dc0ca370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52016/0 (socket says 172.21.15.84:52016) 2023-10-16T10:07:28.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.565+0000 7f47d3fff640 1 -- 172.21.15.84:0/1556415580 learned_addr learned my addr 172.21.15.84:0/1556415580 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:28.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.566+0000 7f47d3fff640 1 -- 172.21.15.84:0/1556415580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47dc0b1c50 msgr2=0x7f47dc0cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:28.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.566+0000 7f47d3fff640 1 --2- 172.21.15.84:0/1556415580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47dc0b1c50 0x7f47dc0cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:28.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.566+0000 7f47d3fff640 1 -- 172.21.15.84:0/1556415580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47dc0b46d0 msgr2=0x7f47dc0ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:28.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.566+0000 7f47dad9d640 1 --2- 172.21.15.84:0/1556415580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47dc0b46d0 0x7f47dc0ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:28.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.566+0000 7f47d3fff640 1 --2- 172.21.15.84:0/1556415580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47dc0b46d0 0x7f47dc0ca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:28.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.566+0000 7f47d3fff640 1 -- 172.21.15.84:0/1556415580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c4009400 con 0x7f47dc0b2210 2023-10-16T10:07:28.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.566+0000 7f47d3fff640 1 --2- 172.21.15.84:0/1556415580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47dc0b2210 0x7f47dc0ca370 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f47c40097a0 tx=0x7f47c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:28.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.566+0000 7f47d2ffd640 1 -- 172.21.15.84:0/1556415580 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4004230 con 0x7f47dc0b2210 2023-10-16T10:07:28.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.566+0000 7f47d2ffd640 1 -- 172.21.15.84:0/1556415580 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c40043b0 con 0x7f47dc0b2210 2023-10-16T10:07:28.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.566+0000 7f47d2ffd640 1 -- 172.21.15.84:0/1556415580 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4004520 con 0x7f47dc0b2210 2023-10-16T10:07:28.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.567+0000 7f47e0889640 1 -- 172.21.15.84:0/1556415580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc149420 con 0x7f47dc0b2210 2023-10-16T10:07:28.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.567+0000 7f47e0889640 1 -- 172.21.15.84:0/1556415580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc15a030 con 0x7f47dc0b2210 2023-10-16T10:07:28.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.568+0000 7f47e0889640 1 -- 172.21.15.84:0/1556415580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47dc0ca090 con 0x7f47dc0b2210 2023-10-16T10:07:28.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.569+0000 7f47d2ffd640 1 -- 172.21.15.84:0/1556415580 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f47c4032880 con 0x7f47dc0b2210 2023-10-16T10:07:28.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.570+0000 7f47d2ffd640 1 --2- 172.21.15.84:0/1556415580 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f47bc06af90 0x7f47bc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:28.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.570+0000 7f47d2ffd640 1 -- 172.21.15.84:0/1556415580 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f47c40abb60 con 0x7f47dc0b2210 2023-10-16T10:07:28.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.570+0000 7f47da59c640 1 --2- 172.21.15.84:0/1556415580 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f47bc06af90 0x7f47bc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:28.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.571+0000 7f47da59c640 1 --2- 172.21.15.84:0/1556415580 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f47bc06af90 0x7f47bc06d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f47cc000c00 tx=0x7f47cc0079c0 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:28.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.574+0000 7f47d2ffd640 1 -- 172.21.15.84:0/1556415580 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f47c4076be0 con 0x7f47dc0b2210 2023-10-16T10:07:28.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:28.795+0000 7f47e0889640 1 -- 172.21.15.84:0/1556415580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f47dc0ca090 con 0x7f47dc0b2210 2023-10-16T10:07:29.362 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 14 and data pool 15 2023-10-16T10:07:29.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.356+0000 7f47d2ffd640 1 -- 172.21.15.84:0/1556415580 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v58) v1 ==== 174+0+0 (secure 0 0 0) 0x7f47dc0ca090 con 0x7f47dc0b2210 2023-10-16T10:07:29.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.365+0000 7f47e0889640 1 -- 172.21.15.84:0/1556415580 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f47bc06af90 msgr2=0x7f47bc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:29.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.365+0000 7f47e0889640 1 --2- 172.21.15.84:0/1556415580 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f47bc06af90 0x7f47bc06d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f47cc000c00 tx=0x7f47cc0079c0 comp rx=0 tx=0).stop 2023-10-16T10:07:29.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.365+0000 7f47e0889640 1 -- 172.21.15.84:0/1556415580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47dc0b2210 msgr2=0x7f47dc0ca370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:29.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.365+0000 7f47e0889640 1 --2- 172.21.15.84:0/1556415580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47dc0b2210 0x7f47dc0ca370 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f47c40097a0 tx=0x7f47c40046c0 comp rx=0 tx=0).stop 2023-10-16T10:07:29.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.365+0000 7f47e0889640 1 -- 172.21.15.84:0/1556415580 shutdown_connections 2023-10-16T10:07:29.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.365+0000 7f47e0889640 1 --2- 172.21.15.84:0/1556415580 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f47bc06af90 0x7f47bc06d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:29.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.365+0000 7f47e0889640 1 --2- 172.21.15.84:0/1556415580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47dc0b46d0 0x7f47dc0ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:29.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.365+0000 7f47e0889640 1 --2- 172.21.15.84:0/1556415580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47dc0b2210 0x7f47dc0ca370 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:29.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.365+0000 7f47e0889640 1 --2- 172.21.15.84:0/1556415580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47dc0b1c50 0x7f47dc0cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:29.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.365+0000 7f47e0889640 1 -- 172.21.15.84:0/1556415580 >> 172.21.15.84:0/1556415580 conn(0x7f47dc0cf6d0 msgr2=0x7f47dc0cfab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:29.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.366+0000 7f47e0889640 1 -- 172.21.15.84:0/1556415580 shutdown_connections 2023-10-16T10:07:29.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.367+0000 7f47e0889640 1 -- 172.21.15.84:0/1556415580 wait complete. 2023-10-16T10:07:29.399 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:07:29.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.575+0000 7f0cbb53e640 1 Processor -- start 2023-10-16T10:07:29.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.575+0000 7f0cbb53e640 1 -- start start 2023-10-16T10:07:29.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.577+0000 7f0cbb53e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0cb415ae80 0x7f0cb415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:29.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.577+0000 7f0cbb53e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cb40ce390 0x7f0cb4070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:29.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.577+0000 7f0cbb53e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb415ed60 con 0x7f0cb415aaa0 2023-10-16T10:07:29.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.577+0000 7f0cbb53e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb406b810 con 0x7f0cb415ae80 2023-10-16T10:07:29.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.577+0000 7f0cbb53e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb40707e0 con 0x7f0cb40ce390 2023-10-16T10:07:29.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.577+0000 7f0cb8ad9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0cb415ae80 0x7f0cb415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:29.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.578+0000 7f0cb9adb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cb40ce390 0x7f0cb4070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:29.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.578+0000 7f0cb8ad9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0cb415ae80 0x7f0cb415e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:38186/0 (socket says 172.21.15.84:38186) 2023-10-16T10:07:29.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.578+0000 7f0cb8ad9640 1 -- 172.21.15.84:0/4034577311 learned_addr learned my addr 172.21.15.84:0/4034577311 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:29.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.578+0000 7f0cb8ad9640 1 -- 172.21.15.84:0/4034577311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cb40ce390 msgr2=0x7f0cb4070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:29.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.578+0000 7f0cb8ad9640 1 --2- 172.21.15.84:0/4034577311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cb40ce390 0x7f0cb4070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:29.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.578+0000 7f0cb8ad9640 1 -- 172.21.15.84:0/4034577311 >> v1:172.21.15.157:6789/0 conn(0x7f0cb415aaa0 legacy=0x7f0cb40cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:29.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.578+0000 7f0cb8ad9640 1 -- 172.21.15.84:0/4034577311 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb415ef90 con 0x7f0cb415ae80 2023-10-16T10:07:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.580+0000 7f0cb8ad9640 1 --2- 172.21.15.84:0/4034577311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cb415ae80 0x7f0cb415e820 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009870 tx=0x7f0ca402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19556fd40aec2aeb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.580+0000 7f0cabfff640 1 -- 172.21.15.84:0/4034577311 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4030840 con 0x7f0cb415ae80 2023-10-16T10:07:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.580+0000 7f0cabfff640 1 -- 172.21.15.84:0/4034577311 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca40309a0 con 0x7f0cb415ae80 2023-10-16T10:07:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.580+0000 7f0cabfff640 1 -- 172.21.15.84:0/4034577311 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4030d30 con 0x7f0cb415ae80 2023-10-16T10:07:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.581+0000 7f0cbb53e640 1 -- 172.21.15.84:0/4034577311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cb415ae80 msgr2=0x7f0cb415e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.581+0000 7f0cbb53e640 1 --2- 172.21.15.84:0/4034577311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cb415ae80 0x7f0cb415e820 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009870 tx=0x7f0ca402b980 comp rx=0 tx=0).stop 2023-10-16T10:07:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.581+0000 7f0cbb53e640 1 -- 172.21.15.84:0/4034577311 shutdown_connections 2023-10-16T10:07:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.581+0000 7f0cbb53e640 1 --2- 172.21.15.84:0/4034577311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cb40ce390 0x7f0cb4070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.581+0000 7f0cbb53e640 1 --2- 172.21.15.84:0/4034577311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cb415ae80 0x7f0cb415e820 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.581+0000 7f0cbb53e640 1 -- 172.21.15.84:0/4034577311 >> 172.21.15.84:0/4034577311 conn(0x7f0cb40cb410 msgr2=0x7f0cb40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.581+0000 7f0cbb53e640 1 -- 172.21.15.84:0/4034577311 shutdown_connections 2023-10-16T10:07:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.581+0000 7f0cbb53e640 1 -- 172.21.15.84:0/4034577311 wait complete. 2023-10-16T10:07:29.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.583+0000 7f0cbb53e640 1 Processor -- start 2023-10-16T10:07:29.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.583+0000 7f0cbb53e640 1 -- start start 2023-10-16T10:07:29.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.584+0000 7f0cbb53e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cb40ce390 0x7f0cb414eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:29.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.584+0000 7f0cbb53e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cb415aaa0 0x7f0cb40c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:29.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.584+0000 7f0cbb53e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cb415ae80 0x7f0cb40c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:29.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.584+0000 7f0cbb53e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb414f3e0 con 0x7f0cb40ce390 2023-10-16T10:07:29.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.584+0000 7f0cbb53e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb40c5b90 con 0x7f0cb415ae80 2023-10-16T10:07:29.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.584+0000 7f0cbb53e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb40c75f0 con 0x7f0cb415aaa0 2023-10-16T10:07:29.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.585+0000 7f0cb92da640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cb40ce390 0x7f0cb414eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:29.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.585+0000 7f0cb92da640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cb40ce390 0x7f0cb414eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52018/0 (socket says 172.21.15.84:52018) 2023-10-16T10:07:29.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.585+0000 7f0cb92da640 1 -- 172.21.15.84:0/1980181314 learned_addr learned my addr 172.21.15.84:0/1980181314 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:29.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.585+0000 7f0cb9adb640 1 --2- 172.21.15.84:0/1980181314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cb415ae80 0x7f0cb40c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:29.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.585+0000 7f0cb8ad9640 1 --2- 172.21.15.84:0/1980181314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cb415aaa0 0x7f0cb40c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:29.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.585+0000 7f0cb92da640 1 -- 172.21.15.84:0/1980181314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cb415aaa0 msgr2=0x7f0cb40c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:29.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.585+0000 7f0cb92da640 1 --2- 172.21.15.84:0/1980181314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cb415aaa0 0x7f0cb40c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:29.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.585+0000 7f0cb92da640 1 -- 172.21.15.84:0/1980181314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cb415ae80 msgr2=0x7f0cb40c2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:29.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.585+0000 7f0cb92da640 1 --2- 172.21.15.84:0/1980181314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cb415ae80 0x7f0cb40c2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:29.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.585+0000 7f0cb92da640 1 -- 172.21.15.84:0/1980181314 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca40094d0 con 0x7f0cb40ce390 2023-10-16T10:07:29.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.586+0000 7f0cb92da640 1 --2- 172.21.15.84:0/1980181314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cb40ce390 0x7f0cb414eea0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000c4d0 tx=0x7f0cb000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:29.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.586+0000 7f0ca9ffb640 1 -- 172.21.15.84:0/1980181314 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb00052d0 con 0x7f0cb40ce390 2023-10-16T10:07:29.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.586+0000 7f0ca9ffb640 1 -- 172.21.15.84:0/1980181314 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb0005430 con 0x7f0cb40ce390 2023-10-16T10:07:29.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.586+0000 7f0ca9ffb640 1 -- 172.21.15.84:0/1980181314 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0012850 con 0x7f0cb40ce390 2023-10-16T10:07:29.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.586+0000 7f0cbb53e640 1 -- 172.21.15.84:0/1980181314 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb40c78d0 con 0x7f0cb40ce390 2023-10-16T10:07:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.586+0000 7f0cbb53e640 1 -- 172.21.15.84:0/1980181314 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb40c7bb0 con 0x7f0cb40ce390 2023-10-16T10:07:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.587+0000 7f0cbb53e640 1 -- 172.21.15.84:0/1980181314 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb40c6070 con 0x7f0cb40ce390 2023-10-16T10:07:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.588+0000 7f0ca9ffb640 1 -- 172.21.15.84:0/1980181314 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0cb00129b0 con 0x7f0cb40ce390 2023-10-16T10:07:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.589+0000 7f0ca9ffb640 1 --2- 172.21.15.84:0/1980181314 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0c9806af90 0x7f0c9806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.589+0000 7f0ca9ffb640 1 -- 172.21.15.84:0/1980181314 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0cb0089330 con 0x7f0cb40ce390 2023-10-16T10:07:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.589+0000 7f0cb8ad9640 1 --2- 172.21.15.84:0/1980181314 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0c9806af90 0x7f0c9806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.590+0000 7f0cb8ad9640 1 --2- 172.21.15.84:0/1980181314 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0c9806af90 0x7f0c9806d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4036000 tx=0x7f0ca403b040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:29.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.594+0000 7f0ca9ffb640 1 -- 172.21.15.84:0/1980181314 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0cb00542c0 con 0x7f0cb40ce390 2023-10-16T10:07:29.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.773+0000 7f0cbb53e640 1 -- 172.21.15.84:0/1980181314 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0cb404e160 con 0x7f0cb40ce390 2023-10-16T10:07:29.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.775+0000 7f0ca9ffb640 1 -- 172.21.15.84:0/1980181314 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16508 (secure 0 0 0) 0x7f0cb00578e0 con 0x7f0cb40ce390 2023-10-16T10:07:29.777 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:29.777 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":123,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:07:28.797937+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:07:26.607199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-16T10:07:27.832706+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:07:29.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.777+0000 7f0cbb53e640 1 -- 172.21.15.84:0/1980181314 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0c9806af90 msgr2=0x7f0c9806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:29.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.777+0000 7f0cbb53e640 1 --2- 172.21.15.84:0/1980181314 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0c9806af90 0x7f0c9806d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4036000 tx=0x7f0ca403b040 comp rx=0 tx=0).stop 2023-10-16T10:07:29.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.777+0000 7f0cbb53e640 1 -- 172.21.15.84:0/1980181314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cb40ce390 msgr2=0x7f0cb414eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:29.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.777+0000 7f0cbb53e640 1 --2- 172.21.15.84:0/1980181314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cb40ce390 0x7f0cb414eea0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000c4d0 tx=0x7f0cb000c990 comp rx=0 tx=0).stop 2023-10-16T10:07:29.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.777+0000 7f0cbb53e640 1 -- 172.21.15.84:0/1980181314 shutdown_connections 2023-10-16T10:07:29.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.777+0000 7f0cbb53e640 1 --2- 172.21.15.84:0/1980181314 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0c9806af90 0x7f0c9806d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:29.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.777+0000 7f0cbb53e640 1 --2- 172.21.15.84:0/1980181314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cb415ae80 0x7f0cb40c2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:29.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.777+0000 7f0cbb53e640 1 --2- 172.21.15.84:0/1980181314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cb415aaa0 0x7f0cb40c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:29.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.777+0000 7f0cbb53e640 1 --2- 172.21.15.84:0/1980181314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cb40ce390 0x7f0cb414eea0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:29.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.777+0000 7f0cbb53e640 1 -- 172.21.15.84:0/1980181314 >> 172.21.15.84:0/1980181314 conn(0x7f0cb40cb410 msgr2=0x7f0cb40b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:29.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.777+0000 7f0cbb53e640 1 -- 172.21.15.84:0/1980181314 shutdown_connections 2023-10-16T10:07:29.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:29.777+0000 7f0cbb53e640 1 -- 172.21.15.84:0/1980181314 wait complete. 2023-10-16T10:07:29.796 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:07:30.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.000+0000 7f3d1eb78640 1 Processor -- start 2023-10-16T10:07:30.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.000+0000 7f3d1eb78640 1 -- start start 2023-10-16T10:07:30.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.001+0000 7f3d1eb78640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3d1815aaa0 0x7f3d1815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:30.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.001+0000 7f3d1eb78640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3d1815ae80 0x7f3d180cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:30.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.001+0000 7f3d1eb78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d180ccaa0 0x7f3d18070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:30.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.001+0000 7f3d1eb78640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1806b810 con 0x7f3d1815ae80 2023-10-16T10:07:30.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.001+0000 7f3d1eb78640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1815ed60 con 0x7f3d1815aaa0 2023-10-16T10:07:30.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.001+0000 7f3d1eb78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d180707e0 con 0x7f3d180ccaa0 2023-10-16T10:07:30.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.001+0000 7f3d17fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3d1815ae80 0x7f3d180cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:30.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.001+0000 7f3d17fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3d1815ae80 0x7f3d180cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52034/0 (socket says 172.21.15.84:52034) 2023-10-16T10:07:30.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.001+0000 7f3d17fff640 1 -- 172.21.15.84:0/212325066 learned_addr learned my addr 172.21.15.84:0/212325066 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:30.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.002+0000 7f3d1c914640 1 --2- 172.21.15.84:0/212325066 >> v2:172.21.15.157:3300/0 conn(0x7f3d1815aaa0 0x7f3d1815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:30.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.002+0000 7f3d1d115640 1 --2- 172.21.15.84:0/212325066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d180ccaa0 0x7f3d18070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:30.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.002+0000 7f3d17fff640 1 -- 172.21.15.84:0/212325066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d180ccaa0 msgr2=0x7f3d18070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:30.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.002+0000 7f3d17fff640 1 --2- 172.21.15.84:0/212325066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d180ccaa0 0x7f3d18070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.002+0000 7f3d17fff640 1 -- 172.21.15.84:0/212325066 >> v2:172.21.15.157:3300/0 conn(0x7f3d1815aaa0 msgr2=0x7f3d1815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:30.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.002+0000 7f3d17fff640 1 --2- 172.21.15.84:0/212325066 >> v2:172.21.15.157:3300/0 conn(0x7f3d1815aaa0 0x7f3d1815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.002+0000 7f3d17fff640 1 -- 172.21.15.84:0/212325066 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1815ef90 con 0x7f3d1815ae80 2023-10-16T10:07:30.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.003+0000 7f3d17fff640 1 --2- 172.21.15.84:0/212325066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d1815ae80 0x7f3d180cc560 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f3d180cd990 tx=0x7f3d0402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929580c4e01e284 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:30.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.003+0000 7f3d177fe640 1 -- 172.21.15.84:0/212325066 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d04031870 con 0x7f3d1815ae80 2023-10-16T10:07:30.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.003+0000 7f3d177fe640 1 -- 172.21.15.84:0/212325066 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d040319d0 con 0x7f3d1815ae80 2023-10-16T10:07:30.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.003+0000 7f3d177fe640 1 -- 172.21.15.84:0/212325066 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d04031d60 con 0x7f3d1815ae80 2023-10-16T10:07:30.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.004+0000 7f3d1eb78640 1 -- 172.21.15.84:0/212325066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d1815ae80 msgr2=0x7f3d180cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:30.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.004+0000 7f3d1eb78640 1 --2- 172.21.15.84:0/212325066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d1815ae80 0x7f3d180cc560 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f3d180cd990 tx=0x7f3d0402b910 comp rx=0 tx=0).stop 2023-10-16T10:07:30.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.004+0000 7f3d1eb78640 1 -- 172.21.15.84:0/212325066 shutdown_connections 2023-10-16T10:07:30.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.004+0000 7f3d1eb78640 1 --2- 172.21.15.84:0/212325066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d1815ae80 0x7f3d180cc560 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.004+0000 7f3d1eb78640 1 --2- 172.21.15.84:0/212325066 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d180ccaa0 0x7f3d18070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.004+0000 7f3d1eb78640 1 --2- 172.21.15.84:0/212325066 >> v2:172.21.15.157:3300/0 conn(0x7f3d1815aaa0 0x7f3d1815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.004+0000 7f3d1eb78640 1 -- 172.21.15.84:0/212325066 >> 172.21.15.84:0/212325066 conn(0x7f3d180cb410 msgr2=0x7f3d180cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:30.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.004+0000 7f3d1eb78640 1 -- 172.21.15.84:0/212325066 shutdown_connections 2023-10-16T10:07:30.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.004+0000 7f3d1eb78640 1 -- 172.21.15.84:0/212325066 wait complete. 2023-10-16T10:07:30.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.006+0000 7f3d1eb78640 1 Processor -- start 2023-10-16T10:07:30.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.006+0000 7f3d1eb78640 1 -- start start 2023-10-16T10:07:30.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.007+0000 7f3d1eb78640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d180ccaa0 0x7f3d1814df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:30.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.007+0000 7f3d1eb78640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d1815aaa0 0x7f3d180aa210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:30.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.007+0000 7f3d1eb78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d1815ae80 0x7f3d1815bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:30.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.007+0000 7f3d1eb78640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1815d2d0 con 0x7f3d1815aaa0 2023-10-16T10:07:30.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.007+0000 7f3d1eb78640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d181581b0 con 0x7f3d180ccaa0 2023-10-16T10:07:30.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.007+0000 7f3d1eb78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1815c6c0 con 0x7f3d1815ae80 2023-10-16T10:07:30.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.007+0000 7f3d17fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d1815aaa0 0x7f3d180aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:30.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.007+0000 7f3d17fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d1815aaa0 0x7f3d180aa210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52042/0 (socket says 172.21.15.84:52042) 2023-10-16T10:07:30.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.007+0000 7f3d17fff640 1 -- 172.21.15.84:0/2945015626 learned_addr learned my addr 172.21.15.84:0/2945015626 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:30.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.007+0000 7f3d1c914640 1 --2- 172.21.15.84:0/2945015626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d180ccaa0 0x7f3d1814df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:30.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.007+0000 7f3d17fff640 1 -- 172.21.15.84:0/2945015626 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d1815ae80 msgr2=0x7f3d1815bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:30.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.007+0000 7f3d17fff640 1 --2- 172.21.15.84:0/2945015626 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d1815ae80 0x7f3d1815bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.007+0000 7f3d17fff640 1 -- 172.21.15.84:0/2945015626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d180ccaa0 msgr2=0x7f3d1814df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.007+0000 7f3d17fff640 1 --2- 172.21.15.84:0/2945015626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d180ccaa0 0x7f3d1814df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.007+0000 7f3d17fff640 1 -- 172.21.15.84:0/2945015626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d040094d0 con 0x7f3d1815aaa0 2023-10-16T10:07:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.008+0000 7f3d17fff640 1 --2- 172.21.15.84:0/2945015626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d1815aaa0 0x7f3d180aa210 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f3d0402b950 tx=0x7f3d040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.008+0000 7f3d157fa640 1 -- 172.21.15.84:0/2945015626 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0403ad30 con 0x7f3d1815aaa0 2023-10-16T10:07:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.008+0000 7f3d157fa640 1 -- 172.21.15.84:0/2945015626 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d040443f0 con 0x7f3d1815aaa0 2023-10-16T10:07:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.008+0000 7f3d1eb78640 1 -- 172.21.15.84:0/2945015626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1815c9a0 con 0x7f3d1815aaa0 2023-10-16T10:07:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.008+0000 7f3d157fa640 1 -- 172.21.15.84:0/2945015626 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d040447a0 con 0x7f3d1815aaa0 2023-10-16T10:07:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.008+0000 7f3d1eb78640 1 -- 172.21.15.84:0/2945015626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1815cf60 con 0x7f3d1815aaa0 2023-10-16T10:07:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.009+0000 7f3d1eb78640 1 -- 172.21.15.84:0/2945015626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1815d440 con 0x7f3d1815aaa0 2023-10-16T10:07:30.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.010+0000 7f3d157fa640 1 -- 172.21.15.84:0/2945015626 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3d0404d450 con 0x7f3d1815aaa0 2023-10-16T10:07:30.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.011+0000 7f3d157fa640 1 --2- 172.21.15.84:0/2945015626 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f3cfc06afc0 0x7f3cfc06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:30.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.011+0000 7f3d157fa640 1 -- 172.21.15.84:0/2945015626 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d0402d040 con 0x7f3d1815aaa0 2023-10-16T10:07:30.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.012+0000 7f3d1c914640 1 --2- 172.21.15.84:0/2945015626 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f3cfc06afc0 0x7f3cfc06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:30.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.012+0000 7f3d1c914640 1 --2- 172.21.15.84:0/2945015626 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f3cfc06afc0 0x7f3cfc06d480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c006fd0 tx=0x7f3d0c008040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:30.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.016+0000 7f3d157fa640 1 -- 172.21.15.84:0/2945015626 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3d04076d20 con 0x7f3d1815aaa0 2023-10-16T10:07:30.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.208+0000 7f3d1eb78640 1 -- 172.21.15.84:0/2945015626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d1804e160 con 0x7f3d1815aaa0 2023-10-16T10:07:30.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.210+0000 7f3d157fa640 1 -- 172.21.15.84:0/2945015626 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16508 (secure 0 0 0) 0x7f3d04035020 con 0x7f3d1815aaa0 2023-10-16T10:07:30.212 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:30.212 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":123,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:07:28.797937+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:07:26.607199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-16T10:07:27.832706+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:07:30.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.212+0000 7f3d1eb78640 1 -- 172.21.15.84:0/2945015626 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f3cfc06afc0 msgr2=0x7f3cfc06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:30.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.212+0000 7f3d1eb78640 1 --2- 172.21.15.84:0/2945015626 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f3cfc06afc0 0x7f3cfc06d480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c006fd0 tx=0x7f3d0c008040 comp rx=0 tx=0).stop 2023-10-16T10:07:30.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.212+0000 7f3d1eb78640 1 -- 172.21.15.84:0/2945015626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d1815aaa0 msgr2=0x7f3d180aa210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:30.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.212+0000 7f3d1eb78640 1 --2- 172.21.15.84:0/2945015626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d1815aaa0 0x7f3d180aa210 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f3d0402b950 tx=0x7f3d040046c0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.212+0000 7f3d1eb78640 1 -- 172.21.15.84:0/2945015626 shutdown_connections 2023-10-16T10:07:30.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.212+0000 7f3d1eb78640 1 --2- 172.21.15.84:0/2945015626 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f3cfc06afc0 0x7f3cfc06d480 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.212+0000 7f3d1eb78640 1 --2- 172.21.15.84:0/2945015626 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d1815ae80 0x7f3d1815bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.212+0000 7f3d1eb78640 1 --2- 172.21.15.84:0/2945015626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d1815aaa0 0x7f3d180aa210 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.212+0000 7f3d1eb78640 1 --2- 172.21.15.84:0/2945015626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d180ccaa0 0x7f3d1814df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.212+0000 7f3d1eb78640 1 -- 172.21.15.84:0/2945015626 >> 172.21.15.84:0/2945015626 conn(0x7f3d180cb410 msgr2=0x7f3d18070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:30.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.213+0000 7f3d1eb78640 1 -- 172.21.15.84:0/2945015626 shutdown_connections 2023-10-16T10:07:30.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.213+0000 7f3d1eb78640 1 -- 172.21.15.84:0/2945015626 wait complete. 2023-10-16T10:07:30.232 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T10:07:30.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.448+0000 7f30d5090640 1 Processor -- start 2023-10-16T10:07:30.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.448+0000 7f30d5090640 1 -- start start 2023-10-16T10:07:30.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.450+0000 7f30d5090640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d015d330 0x7f30d01618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:30.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.450+0000 7f30d5090640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f30d006c7d0 0x7f30d0065ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:30.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.450+0000 7f30d5090640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f30d006cc80 0x7f30d0065ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:30.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.450+0000 7f30d5090640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d006b1e0 con 0x7f30d006c7d0 2023-10-16T10:07:30.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.450+0000 7f30d5090640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d00d0890 con 0x7f30d006cc80 2023-10-16T10:07:30.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.450+0000 7f30d5090640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d00d05b0 con 0x7f30d015d330 2023-10-16T10:07:30.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.450+0000 7f30ce59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f30d006c7d0 0x7f30d0065ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:30.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.450+0000 7f30ce59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f30d006c7d0 0x7f30d0065ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52046/0 (socket says 172.21.15.84:52046) 2023-10-16T10:07:30.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.450+0000 7f30ce59c640 1 -- 172.21.15.84:0/2131578183 learned_addr learned my addr 172.21.15.84:0/2131578183 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:30.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.451+0000 7f30ce59c640 1 -- 172.21.15.84:0/2131578183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d015d330 msgr2=0x7f30d01618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:30.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.451+0000 7f30ce59c640 1 --2- 172.21.15.84:0/2131578183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d015d330 0x7f30d01618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.451+0000 7f30ce59c640 1 -- 172.21.15.84:0/2131578183 >> v2:172.21.15.157:3300/0 conn(0x7f30d006cc80 msgr2=0x7f30d0065ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:30.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.451+0000 7f30ce59c640 1 --2- 172.21.15.84:0/2131578183 >> v2:172.21.15.157:3300/0 conn(0x7f30d006cc80 0x7f30d0065ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.451+0000 7f30ce59c640 1 -- 172.21.15.84:0/2131578183 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d0160dd0 con 0x7f30d006c7d0 2023-10-16T10:07:30.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.452+0000 7f30ce59c640 1 --2- 172.21.15.84:0/2131578183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30d006c7d0 0x7f30d0065ab0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f30d00afa70 tx=0x7f30b802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c9087f3977d1c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:30.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.452+0000 7f30cdd9b640 1 -- 172.21.15.84:0/2131578183 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b80317e0 con 0x7f30d006c7d0 2023-10-16T10:07:30.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.452+0000 7f30cdd9b640 1 -- 172.21.15.84:0/2131578183 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b8031940 con 0x7f30d006c7d0 2023-10-16T10:07:30.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.452+0000 7f30cdd9b640 1 -- 172.21.15.84:0/2131578183 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8031cd0 con 0x7f30d006c7d0 2023-10-16T10:07:30.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.453+0000 7f30d5090640 1 -- 172.21.15.84:0/2131578183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30d006c7d0 msgr2=0x7f30d0065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:30.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.453+0000 7f30d5090640 1 --2- 172.21.15.84:0/2131578183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30d006c7d0 0x7f30d0065ab0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f30d00afa70 tx=0x7f30b802b880 comp rx=0 tx=0).stop 2023-10-16T10:07:30.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.453+0000 7f30d5090640 1 -- 172.21.15.84:0/2131578183 shutdown_connections 2023-10-16T10:07:30.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.453+0000 7f30d5090640 1 --2- 172.21.15.84:0/2131578183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30d006c7d0 0x7f30d0065ab0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.453+0000 7f30d5090640 1 --2- 172.21.15.84:0/2131578183 >> v2:172.21.15.157:3300/0 conn(0x7f30d006cc80 0x7f30d0065ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.453+0000 7f30d5090640 1 --2- 172.21.15.84:0/2131578183 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d015d330 0x7f30d01618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.453+0000 7f30d5090640 1 -- 172.21.15.84:0/2131578183 >> 172.21.15.84:0/2131578183 conn(0x7f30d00d0ab0 msgr2=0x7f30d00d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.453+0000 7f30d5090640 1 -- 172.21.15.84:0/2131578183 shutdown_connections 2023-10-16T10:07:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.453+0000 7f30d5090640 1 -- 172.21.15.84:0/2131578183 wait complete. 2023-10-16T10:07:30.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.455+0000 7f30d5090640 1 Processor -- start 2023-10-16T10:07:30.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.455+0000 7f30d5090640 1 -- start start 2023-10-16T10:07:30.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.456+0000 7f30d5090640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30d006c7d0 0x7f30d0160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:30.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.456+0000 7f30d5090640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d006cc80 0x7f30d00af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:30.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.456+0000 7f30d5090640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d015d330 0x7f30d00ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:30.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.456+0000 7f30d5090640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d015fd80 con 0x7f30d006c7d0 2023-10-16T10:07:30.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.456+0000 7f30d5090640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d015f170 con 0x7f30d015d330 2023-10-16T10:07:30.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.456+0000 7f30d5090640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d015f450 con 0x7f30d006cc80 2023-10-16T10:07:30.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.457+0000 7f30ced9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30d006c7d0 0x7f30d0160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:30.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.457+0000 7f30ced9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30d006c7d0 0x7f30d0160210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52056/0 (socket says 172.21.15.84:52056) 2023-10-16T10:07:30.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.457+0000 7f30ced9d640 1 -- 172.21.15.84:0/1036810542 learned_addr learned my addr 172.21.15.84:0/1036810542 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:30.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.457+0000 7f30ce59c640 1 --2- 172.21.15.84:0/1036810542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d006cc80 0x7f30d00af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:30.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.457+0000 7f30cf59e640 1 --2- 172.21.15.84:0/1036810542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d015d330 0x7f30d00ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.457+0000 7f30ced9d640 1 -- 172.21.15.84:0/1036810542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d006cc80 msgr2=0x7f30d00af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.457+0000 7f30ced9d640 1 --2- 172.21.15.84:0/1036810542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d006cc80 0x7f30d00af1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.457+0000 7f30ced9d640 1 -- 172.21.15.84:0/1036810542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d015d330 msgr2=0x7f30d00ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.457+0000 7f30ced9d640 1 --2- 172.21.15.84:0/1036810542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d015d330 0x7f30d00ad2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.457+0000 7f30ced9d640 1 -- 172.21.15.84:0/1036810542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b80094d0 con 0x7f30d006c7d0 2023-10-16T10:07:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.457+0000 7f30cf59e640 1 --2- 172.21.15.84:0/1036810542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d015d330 0x7f30d00ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.458+0000 7f30ced9d640 1 --2- 172.21.15.84:0/1036810542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30d006c7d0 0x7f30d0160210 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f30c000ba50 tx=0x7f30c000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.458+0000 7f30c77fe640 1 -- 172.21.15.84:0/1036810542 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c000cbe0 con 0x7f30d006c7d0 2023-10-16T10:07:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.458+0000 7f30c77fe640 1 -- 172.21.15.84:0/1036810542 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c000cd60 con 0x7f30d006c7d0 2023-10-16T10:07:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.458+0000 7f30c77fe640 1 -- 172.21.15.84:0/1036810542 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c0014680 con 0x7f30d006c7d0 2023-10-16T10:07:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.458+0000 7f30d5090640 1 -- 172.21.15.84:0/1036810542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d015f730 con 0x7f30d006c7d0 2023-10-16T10:07:30.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.458+0000 7f30d5090640 1 -- 172.21.15.84:0/1036810542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d00af710 con 0x7f30d006c7d0 2023-10-16T10:07:30.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.459+0000 7f30d5090640 1 -- 172.21.15.84:0/1036810542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d00af900 con 0x7f30d006c7d0 2023-10-16T10:07:30.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.460+0000 7f30c77fe640 1 -- 172.21.15.84:0/1036810542 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f30c001d3b0 con 0x7f30d006c7d0 2023-10-16T10:07:30.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.461+0000 7f30c77fe640 1 --2- 172.21.15.84:0/1036810542 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f30b006af90 0x7f30b006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:30.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.461+0000 7f30c77fe640 1 -- 172.21.15.84:0/1036810542 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30c0089780 con 0x7f30d006c7d0 2023-10-16T10:07:30.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.461+0000 7f30ce59c640 1 --2- 172.21.15.84:0/1036810542 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f30b006af90 0x7f30b006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:30.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.462+0000 7f30ce59c640 1 --2- 172.21.15.84:0/1036810542 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f30b006af90 0x7f30b006d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f30b8036000 tx=0x7f30b803d040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:30.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.466+0000 7f30c77fe640 1 -- 172.21.15.84:0/1036810542 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f30d00af900 con 0x7f30d006c7d0 2023-10-16T10:07:30.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:30.700+0000 7f30d5090640 1 -- 172.21.15.84:0/1036810542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f30d00af900 con 0x7f30d006c7d0 2023-10-16T10:07:31.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.375+0000 7f30c77fe640 1 -- 172.21.15.84:0/1036810542 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v61) v1 ==== 118+0+0 (secure 0 0 0) 0x7f30d00af900 con 0x7f30d006c7d0 2023-10-16T10:07:31.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.379+0000 7f30d5090640 1 -- 172.21.15.84:0/1036810542 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f30b006af90 msgr2=0x7f30b006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:31.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.379+0000 7f30d5090640 1 --2- 172.21.15.84:0/1036810542 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f30b006af90 0x7f30b006d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f30b8036000 tx=0x7f30b803d040 comp rx=0 tx=0).stop 2023-10-16T10:07:31.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.379+0000 7f30d5090640 1 -- 172.21.15.84:0/1036810542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30d006c7d0 msgr2=0x7f30d0160210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:31.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.379+0000 7f30d5090640 1 --2- 172.21.15.84:0/1036810542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30d006c7d0 0x7f30d0160210 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f30c000ba50 tx=0x7f30c000bf10 comp rx=0 tx=0).stop 2023-10-16T10:07:31.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.379+0000 7f30d5090640 1 -- 172.21.15.84:0/1036810542 shutdown_connections 2023-10-16T10:07:31.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.379+0000 7f30d5090640 1 --2- 172.21.15.84:0/1036810542 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f30b006af90 0x7f30b006d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:31.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.379+0000 7f30d5090640 1 --2- 172.21.15.84:0/1036810542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d015d330 0x7f30d00ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:31.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.379+0000 7f30d5090640 1 --2- 172.21.15.84:0/1036810542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f30d006cc80 0x7f30d00af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:31.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.379+0000 7f30d5090640 1 --2- 172.21.15.84:0/1036810542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30d006c7d0 0x7f30d0160210 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:31.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.379+0000 7f30d5090640 1 -- 172.21.15.84:0/1036810542 >> 172.21.15.84:0/1036810542 conn(0x7f30d00d0ab0 msgr2=0x7f30d0159780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:31.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.379+0000 7f30d5090640 1 -- 172.21.15.84:0/1036810542 shutdown_connections 2023-10-16T10:07:31.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.379+0000 7f30d5090640 1 -- 172.21.15.84:0/1036810542 wait complete. 2023-10-16T10:07:31.401 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T10:07:31.401 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T10:07:31.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.608+0000 7f4abc906640 1 Processor -- start 2023-10-16T10:07:31.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.609+0000 7f4abc906640 1 -- start start 2023-10-16T10:07:31.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.610+0000 7f4abc906640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ab8159950 0x7f4ab815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:31.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.610+0000 7f4abc906640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4ab806c810 0x7f4ab80708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:31.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.610+0000 7f4abc906640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab815aef0 con 0x7f4ab8070450 2023-10-16T10:07:31.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.610+0000 7f4abc906640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab8070e20 con 0x7f4ab806c810 2023-10-16T10:07:31.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.610+0000 7f4abc906640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab815e490 con 0x7f4ab8159950 2023-10-16T10:07:31.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.610+0000 7f4ab7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4ab806c810 0x7f4ab80708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:31.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.610+0000 7f4ab7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4ab806c810 0x7f4ab80708e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52068/0 (socket says 172.21.15.84:52068) 2023-10-16T10:07:31.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.610+0000 7f4ab7fff640 1 -- 172.21.15.84:0/2104530990 learned_addr learned my addr 172.21.15.84:0/2104530990 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:31.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.611+0000 7f4ab77fe640 1 --2- 172.21.15.84:0/2104530990 >> v2:172.21.15.157:3300/0 conn(0x7f4ab8159950 0x7f4ab815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:31.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.611+0000 7f4ab7fff640 1 -- 172.21.15.84:0/2104530990 >> v2:172.21.15.157:3300/0 conn(0x7f4ab8159950 msgr2=0x7f4ab815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:31.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.611+0000 7f4ab7fff640 1 --2- 172.21.15.84:0/2104530990 >> v2:172.21.15.157:3300/0 conn(0x7f4ab8159950 0x7f4ab815a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:31.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.611+0000 7f4ab7fff640 1 -- 172.21.15.84:0/2104530990 >> v1:172.21.15.157:6789/0 conn(0x7f4ab8070450 legacy=0x7f4ab806c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:31.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.611+0000 7f4ab7fff640 1 -- 172.21.15.84:0/2104530990 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab815d910 con 0x7f4ab806c810 2023-10-16T10:07:31.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.612+0000 7f4ab7fff640 1 --2- 172.21.15.84:0/2104530990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ab806c810 0x7f4ab80708e0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80ca950 tx=0x7f4ab002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00736fab33b695f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:31.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.612+0000 7f4ab67fc640 1 -- 172.21.15.84:0/2104530990 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab002c9f0 con 0x7f4ab806c810 2023-10-16T10:07:31.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.612+0000 7f4ab67fc640 1 -- 172.21.15.84:0/2104530990 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab002cb50 con 0x7f4ab806c810 2023-10-16T10:07:31.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.612+0000 7f4ab67fc640 1 -- 172.21.15.84:0/2104530990 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab00343f0 con 0x7f4ab806c810 2023-10-16T10:07:31.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.613+0000 7f4abc906640 1 -- 172.21.15.84:0/2104530990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ab806c810 msgr2=0x7f4ab80708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:31.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.613+0000 7f4abc906640 1 --2- 172.21.15.84:0/2104530990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ab806c810 0x7f4ab80708e0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80ca950 tx=0x7f4ab002be50 comp rx=0 tx=0).stop 2023-10-16T10:07:31.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.613+0000 7f4abc906640 1 -- 172.21.15.84:0/2104530990 shutdown_connections 2023-10-16T10:07:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.613+0000 7f4abc906640 1 --2- 172.21.15.84:0/2104530990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ab806c810 0x7f4ab80708e0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.613+0000 7f4abc906640 1 --2- 172.21.15.84:0/2104530990 >> v2:172.21.15.157:3300/0 conn(0x7f4ab8159950 0x7f4ab815a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.613+0000 7f4abc906640 1 -- 172.21.15.84:0/2104530990 >> 172.21.15.84:0/2104530990 conn(0x7f4ab80ce6a0 msgr2=0x7f4ab80ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.613+0000 7f4abc906640 1 -- 172.21.15.84:0/2104530990 shutdown_connections 2023-10-16T10:07:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.613+0000 7f4abc906640 1 -- 172.21.15.84:0/2104530990 wait complete. 2023-10-16T10:07:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.615+0000 7f4abc906640 1 Processor -- start 2023-10-16T10:07:31.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.615+0000 7f4abc906640 1 -- start start 2023-10-16T10:07:31.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.616+0000 7f4abc906640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab806c810 0x7f4ab8150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:31.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.616+0000 7f4abc906640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ab8070450 0x7f4ab814da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:31.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.616+0000 7f4abc906640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ab8159950 0x7f4ab80c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:31.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.616+0000 7f4abc906640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab80c38e0 con 0x7f4ab8070450 2023-10-16T10:07:31.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.616+0000 7f4abc906640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab814df40 con 0x7f4ab8159950 2023-10-16T10:07:31.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.616+0000 7f4abc906640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab80c2a00 con 0x7f4ab806c810 2023-10-16T10:07:31.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.616+0000 7f4ab6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ab8070450 0x7f4ab814da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:31.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.616+0000 7f4ab6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ab8070450 0x7f4ab814da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52082/0 (socket says 172.21.15.84:52082) 2023-10-16T10:07:31.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.616+0000 7f4ab6ffd640 1 -- 172.21.15.84:0/1028956045 learned_addr learned my addr 172.21.15.84:0/1028956045 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:31.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.617+0000 7f4ab77fe640 1 --2- 172.21.15.84:0/1028956045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab806c810 0x7f4ab8150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:31.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.617+0000 7f4ab7fff640 1 --2- 172.21.15.84:0/1028956045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ab8159950 0x7f4ab80c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:31.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.617+0000 7f4ab6ffd640 1 -- 172.21.15.84:0/1028956045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab806c810 msgr2=0x7f4ab8150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:31.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.617+0000 7f4ab6ffd640 1 --2- 172.21.15.84:0/1028956045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab806c810 0x7f4ab8150590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:31.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.617+0000 7f4ab6ffd640 1 -- 172.21.15.84:0/1028956045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ab8159950 msgr2=0x7f4ab80c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:31.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.617+0000 7f4ab6ffd640 1 --2- 172.21.15.84:0/1028956045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ab8159950 0x7f4ab80c21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:31.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.617+0000 7f4ab6ffd640 1 -- 172.21.15.84:0/1028956045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab0009b30 con 0x7f4ab8070450 2023-10-16T10:07:31.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.617+0000 7f4ab77fe640 1 --2- 172.21.15.84:0/1028956045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab806c810 0x7f4ab8150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:31.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.617+0000 7f4ab7fff640 1 --2- 172.21.15.84:0/1028956045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ab8159950 0x7f4ab80c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:31.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.617+0000 7f4ab6ffd640 1 --2- 172.21.15.84:0/1028956045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ab8070450 0x7f4ab814da00 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000c440 tx=0x7f4aa000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:31.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.617+0000 7f4aaffff640 1 -- 172.21.15.84:0/1028956045 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0005480 con 0x7f4ab8070450 2023-10-16T10:07:31.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.617+0000 7f4aaffff640 1 -- 172.21.15.84:0/1028956045 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa00035f0 con 0x7f4ab8070450 2023-10-16T10:07:31.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.617+0000 7f4abc906640 1 -- 172.21.15.84:0/1028956045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab80c2ce0 con 0x7f4ab8070450 2023-10-16T10:07:31.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.617+0000 7f4aaffff640 1 -- 172.21.15.84:0/1028956045 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa00039c0 con 0x7f4ab8070450 2023-10-16T10:07:31.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.618+0000 7f4abc906640 1 -- 172.21.15.84:0/1028956045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab80c3500 con 0x7f4ab8070450 2023-10-16T10:07:31.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.619+0000 7f4abc906640 1 -- 172.21.15.84:0/1028956045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab80c3a20 con 0x7f4ab8070450 2023-10-16T10:07:31.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.620+0000 7f4aaffff640 1 -- 172.21.15.84:0/1028956045 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4aa0003b20 con 0x7f4ab8070450 2023-10-16T10:07:31.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.620+0000 7f4aaffff640 1 --2- 172.21.15.84:0/1028956045 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f4a9806af90 0x7f4a9806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.621+0000 7f4aaffff640 1 -- 172.21.15.84:0/1028956045 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4aa0088c90 con 0x7f4ab8070450 2023-10-16T10:07:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.621+0000 7f4ab77fe640 1 --2- 172.21.15.84:0/1028956045 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f4a9806af90 0x7f4a9806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.622+0000 7f4ab77fe640 1 --2- 172.21.15.84:0/1028956045 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f4a9806af90 0x7f4a9806d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8006fd0 tx=0x7f4aa8008040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:31.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.626+0000 7f4aaffff640 1 -- 172.21.15.84:0/1028956045 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4ab80c3a20 con 0x7f4ab8070450 2023-10-16T10:07:31.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:31.855+0000 7f4abc906640 1 -- 172.21.15.84:0/1028956045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4ab80c3a20 con 0x7f4ab8070450 2023-10-16T10:07:32.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.385+0000 7f4aaffff640 1 -- 172.21.15.84:0/1028956045 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v62) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4ab80c3a20 con 0x7f4ab8070450 2023-10-16T10:07:32.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.389+0000 7f4abc906640 1 -- 172.21.15.84:0/1028956045 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f4a9806af90 msgr2=0x7f4a9806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:32.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.389+0000 7f4abc906640 1 --2- 172.21.15.84:0/1028956045 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f4a9806af90 0x7f4a9806d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8006fd0 tx=0x7f4aa8008040 comp rx=0 tx=0).stop 2023-10-16T10:07:32.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.389+0000 7f4abc906640 1 -- 172.21.15.84:0/1028956045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ab8070450 msgr2=0x7f4ab814da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:32.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.389+0000 7f4abc906640 1 --2- 172.21.15.84:0/1028956045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ab8070450 0x7f4ab814da00 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000c440 tx=0x7f4aa000c900 comp rx=0 tx=0).stop 2023-10-16T10:07:32.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.390+0000 7f4abc906640 1 -- 172.21.15.84:0/1028956045 shutdown_connections 2023-10-16T10:07:32.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.390+0000 7f4abc906640 1 --2- 172.21.15.84:0/1028956045 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f4a9806af90 0x7f4a9806d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:32.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.390+0000 7f4abc906640 1 --2- 172.21.15.84:0/1028956045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ab8159950 0x7f4ab80c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:32.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.390+0000 7f4abc906640 1 --2- 172.21.15.84:0/1028956045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ab8070450 0x7f4ab814da00 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:32.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.390+0000 7f4abc906640 1 --2- 172.21.15.84:0/1028956045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ab806c810 0x7f4ab8150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:32.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.390+0000 7f4abc906640 1 -- 172.21.15.84:0/1028956045 >> 172.21.15.84:0/1028956045 conn(0x7f4ab80ce6a0 msgr2=0x7f4ab8151d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:32.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.390+0000 7f4abc906640 1 -- 172.21.15.84:0/1028956045 shutdown_connections 2023-10-16T10:07:32.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.390+0000 7f4abc906640 1 -- 172.21.15.84:0/1028956045 wait complete. 2023-10-16T10:07:32.411 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T10:07:32.412 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:07:32.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.617+0000 7f97a3e4b640 1 Processor -- start 2023-10-16T10:07:32.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.618+0000 7f97a3e4b640 1 -- start start 2023-10-16T10:07:32.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.619+0000 7f97a3e4b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f979c0adb50 0x7f979c06f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:32.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.619+0000 7f97a3e4b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f979c0b0b20 0x7f979c164f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:32.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.619+0000 7f97a3e4b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f979c071450 con 0x7f979c070fd0 2023-10-16T10:07:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.619+0000 7f97a3e4b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979c0cc600 con 0x7f979c0adb50 2023-10-16T10:07:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.619+0000 7f97a3e4b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979c0cc8e0 con 0x7f979c0b0b20 2023-10-16T10:07:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.619+0000 7f97a1be7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f979c0adb50 0x7f979c06f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.620+0000 7f97a1be7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f979c0adb50 0x7f979c06f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52094/0 (socket says 172.21.15.84:52094) 2023-10-16T10:07:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.620+0000 7f97a1be7640 1 -- 172.21.15.84:0/2898555995 learned_addr learned my addr 172.21.15.84:0/2898555995 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:32.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.620+0000 7f97a1be7640 1 -- 172.21.15.84:0/2898555995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f979c0b0b20 msgr2=0x7f979c164f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:32.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.620+0000 7f97a1be7640 1 --2- 172.21.15.84:0/2898555995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f979c0b0b20 0x7f979c164f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:32.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.620+0000 7f97a1be7640 1 -- 172.21.15.84:0/2898555995 >> v1:172.21.15.157:6789/0 conn(0x7f979c070fd0 legacy=0x7f979c0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:32.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.620+0000 7f97a1be7640 1 -- 172.21.15.84:0/2898555995 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c0ccbc0 con 0x7f979c0adb50 2023-10-16T10:07:32.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.621+0000 7f97a1be7640 1 --2- 172.21.15.84:0/2898555995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979c0adb50 0x7f979c06f550 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f9798009870 tx=0x7f979802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3462d85eb94be17f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:32.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.621+0000 7f97a0be5640 1 -- 172.21.15.84:0/2898555995 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798031640 con 0x7f979c0adb50 2023-10-16T10:07:32.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.621+0000 7f97a0be5640 1 -- 172.21.15.84:0/2898555995 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97980317a0 con 0x7f979c0adb50 2023-10-16T10:07:32.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.621+0000 7f97a0be5640 1 -- 172.21.15.84:0/2898555995 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798031b30 con 0x7f979c0adb50 2023-10-16T10:07:32.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.622+0000 7f97a3e4b640 1 -- 172.21.15.84:0/2898555995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979c0adb50 msgr2=0x7f979c06f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:32.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.622+0000 7f97a3e4b640 1 --2- 172.21.15.84:0/2898555995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979c0adb50 0x7f979c06f550 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f9798009870 tx=0x7f979802b8f0 comp rx=0 tx=0).stop 2023-10-16T10:07:32.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.622+0000 7f97a3e4b640 1 -- 172.21.15.84:0/2898555995 shutdown_connections 2023-10-16T10:07:32.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.622+0000 7f97a3e4b640 1 --2- 172.21.15.84:0/2898555995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f979c0b0b20 0x7f979c164f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:32.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.622+0000 7f97a3e4b640 1 --2- 172.21.15.84:0/2898555995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979c0adb50 0x7f979c06f550 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:32.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.622+0000 7f97a3e4b640 1 -- 172.21.15.84:0/2898555995 >> 172.21.15.84:0/2898555995 conn(0x7f979c0cb410 msgr2=0x7f979c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:32.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.622+0000 7f97a3e4b640 1 -- 172.21.15.84:0/2898555995 shutdown_connections 2023-10-16T10:07:32.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.622+0000 7f97a3e4b640 1 -- 172.21.15.84:0/2898555995 wait complete. 2023-10-16T10:07:32.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.624+0000 7f97a3e4b640 1 Processor -- start 2023-10-16T10:07:32.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.624+0000 7f97a3e4b640 1 -- start start 2023-10-16T10:07:32.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.625+0000 7f97a3e4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979c070fd0 0x7f979c15f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:32.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.625+0000 7f97a3e4b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979c0adb50 0x7f979c1f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:32.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.625+0000 7f97a3e4b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f979c0b0b20 0x7f979c0af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:32.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.625+0000 7f97a3e4b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c0b23d0 con 0x7f979c0adb50 2023-10-16T10:07:32.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.625+0000 7f97a3e4b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c0c5b70 con 0x7f979c070fd0 2023-10-16T10:07:32.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.625+0000 7f97a3e4b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979c1f86c0 con 0x7f979c0b0b20 2023-10-16T10:07:32.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.625+0000 7f97a13e6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979c0adb50 0x7f979c1f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:32.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.625+0000 7f97a13e6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979c0adb50 0x7f979c1f8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52108/0 (socket says 172.21.15.84:52108) 2023-10-16T10:07:32.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.625+0000 7f97a13e6640 1 -- 172.21.15.84:0/1175585238 learned_addr learned my addr 172.21.15.84:0/1175585238 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:32.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.625+0000 7f97a13e6640 1 -- 172.21.15.84:0/1175585238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f979c0b0b20 msgr2=0x7f979c0af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:32.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.625+0000 7f97a13e6640 1 --2- 172.21.15.84:0/1175585238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f979c0b0b20 0x7f979c0af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:32.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.625+0000 7f97a13e6640 1 -- 172.21.15.84:0/1175585238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979c070fd0 msgr2=0x7f979c15f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:32.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.625+0000 7f97a13e6640 1 --2- 172.21.15.84:0/1175585238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979c070fd0 0x7f979c15f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:32.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.625+0000 7f97a13e6640 1 -- 172.21.15.84:0/1175585238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97980094d0 con 0x7f979c0adb50 2023-10-16T10:07:32.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.626+0000 7f97a13e6640 1 --2- 172.21.15.84:0/1175585238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979c0adb50 0x7f979c1f8180 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f979000b3e0 tx=0x7f979000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:32.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.626+0000 7f978e7fc640 1 -- 172.21.15.84:0/1175585238 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790011830 con 0x7f979c0adb50 2023-10-16T10:07:32.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.626+0000 7f978e7fc640 1 -- 172.21.15.84:0/1175585238 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97900119b0 con 0x7f979c0adb50 2023-10-16T10:07:32.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.626+0000 7f97a3e4b640 1 -- 172.21.15.84:0/1175585238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979c0b1750 con 0x7f979c0adb50 2023-10-16T10:07:32.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.626+0000 7f978e7fc640 1 -- 172.21.15.84:0/1175585238 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790011d60 con 0x7f979c0adb50 2023-10-16T10:07:32.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.626+0000 7f97a3e4b640 1 -- 172.21.15.84:0/1175585238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979c0b1d10 con 0x7f979c0adb50 2023-10-16T10:07:32.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.627+0000 7f97a3e4b640 1 -- 172.21.15.84:0/1175585238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979c0c2130 con 0x7f979c0adb50 2023-10-16T10:07:32.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.628+0000 7f978e7fc640 1 -- 172.21.15.84:0/1175585238 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f979001a400 con 0x7f979c0adb50 2023-10-16T10:07:32.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.629+0000 7f978e7fc640 1 --2- 172.21.15.84:0/1175585238 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f978006af90 0x7f978006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:32.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.629+0000 7f978e7fc640 1 -- 172.21.15.84:0/1175585238 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9790089a40 con 0x7f979c0adb50 2023-10-16T10:07:32.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.630+0000 7f97a1be7640 1 --2- 172.21.15.84:0/1175585238 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f978006af90 0x7f978006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:32.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.630+0000 7f97a1be7640 1 --2- 172.21.15.84:0/1175585238 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f978006af90 0x7f978006d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9798036000 tx=0x7f9798030000 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:32.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.633+0000 7f978e7fc640 1 -- 172.21.15.84:0/1175585238 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f979c0c2130 con 0x7f979c0adb50 2023-10-16T10:07:32.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.844+0000 7f97a3e4b640 1 -- 172.21.15.84:0/1175585238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f979c0c2130 con 0x7f979c0adb50 2023-10-16T10:07:32.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.845+0000 7f978e7fc640 1 -- 172.21.15.84:0/1175585238 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f979c0c2130 con 0x7f979c0adb50 2023-10-16T10:07:32.847 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:32.847 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2012273152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2012273152},{"type":"v1","addr":"172.21.15.157:6835","nonce":2012273152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/435606514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":435606514},{"type":"v1","addr":"172.21.15.157:6833","nonce":435606514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2286048798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2286048798},{"type":"v1","addr":"172.21.15.84:6836","nonce":2286048798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:07:28.798746+0000","modified":"2023-10-16T10:07:31.857629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2859088731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2859088731},{"type":"v1","addr":"172.21.15.84:6837","nonce":2859088731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-16T10:07:32.847 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 62 2023-10-16T10:07:32.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.847+0000 7f97a3e4b640 1 -- 172.21.15.84:0/1175585238 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f978006af90 msgr2=0x7f978006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:32.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.847+0000 7f97a3e4b640 1 --2- 172.21.15.84:0/1175585238 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f978006af90 0x7f978006d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9798036000 tx=0x7f9798030000 comp rx=0 tx=0).stop 2023-10-16T10:07:32.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.847+0000 7f97a3e4b640 1 -- 172.21.15.84:0/1175585238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979c0adb50 msgr2=0x7f979c1f8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.847+0000 7f97a3e4b640 1 --2- 172.21.15.84:0/1175585238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979c0adb50 0x7f979c1f8180 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f979000b3e0 tx=0x7f979000b8a0 comp rx=0 tx=0).stop 2023-10-16T10:07:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.848+0000 7f97a3e4b640 1 -- 172.21.15.84:0/1175585238 shutdown_connections 2023-10-16T10:07:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.848+0000 7f97a3e4b640 1 --2- 172.21.15.84:0/1175585238 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f978006af90 0x7f978006d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.848+0000 7f97a3e4b640 1 --2- 172.21.15.84:0/1175585238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f979c0b0b20 0x7f979c0af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.848+0000 7f97a3e4b640 1 --2- 172.21.15.84:0/1175585238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979c0adb50 0x7f979c1f8180 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.848+0000 7f97a3e4b640 1 --2- 172.21.15.84:0/1175585238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979c070fd0 0x7f979c15f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.848+0000 7f97a3e4b640 1 -- 172.21.15.84:0/1175585238 >> 172.21.15.84:0/1175585238 conn(0x7f979c0cb410 msgr2=0x7f979c160f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.848+0000 7f97a3e4b640 1 -- 172.21.15.84:0/1175585238 shutdown_connections 2023-10-16T10:07:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:32.848+0000 7f97a3e4b640 1 -- 172.21.15.84:0/1175585238 wait complete. 2023-10-16T10:07:32.867 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:07:33.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.075+0000 7fd17932c640 1 Processor -- start 2023-10-16T10:07:33.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.075+0000 7fd17932c640 1 -- start start 2023-10-16T10:07:33.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.077+0000 7fd17932c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd174159b20 0x7fd17406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:33.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.077+0000 7fd17932c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1740b0b20 0x7fd17406fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:33.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.077+0000 7fd17932c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17406f7f0 con 0x7fd174159f00 2023-10-16T10:07:33.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.077+0000 7fd17932c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17406c5e0 con 0x7fd174159b20 2023-10-16T10:07:33.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.077+0000 7fd17932c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17406c8c0 con 0x7fd1740b0b20 2023-10-16T10:07:33.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.077+0000 7fd1727fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd174159f00 0x7fd1740b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37816/0 (socket says 172.21.15.84:37816) 2023-10-16T10:07:33.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.077+0000 7fd172ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd174159b20 0x7fd17406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:33.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.077+0000 7fd1727fc640 1 -- 172.21.15.84:0/639646100 learned_addr learned my addr 172.21.15.84:0/639646100 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:33.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.078+0000 7fd1737fe640 1 --2- 172.21.15.84:0/639646100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1740b0b20 0x7fd17406fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:33.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.078+0000 7fd171ffb640 1 -- 172.21.15.84:0/639646100 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2264363292 0 0) 0x7fd17406f7f0 con 0x7fd174159f00 2023-10-16T10:07:33.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.078+0000 7fd172ffd640 1 -- 172.21.15.84:0/639646100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1740b0b20 msgr2=0x7fd17406fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:33.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.078+0000 7fd172ffd640 1 --2- 172.21.15.84:0/639646100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1740b0b20 0x7fd17406fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.078+0000 7fd172ffd640 1 -- 172.21.15.84:0/639646100 >> v1:172.21.15.84:6789/0 conn(0x7fd174159f00 legacy=0x7fd1740b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:33.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.078+0000 7fd172ffd640 1 -- 172.21.15.84:0/639646100 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17406cba0 con 0x7fd174159b20 2023-10-16T10:07:33.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.079+0000 7fd172ffd640 1 --2- 172.21.15.84:0/639646100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd174159b20 0x7fd17406bbe0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fd1640097a0 tx=0x7fd16402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b25cd1b311ba7f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:33.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.080+0000 7fd171ffb640 1 -- 172.21.15.84:0/639646100 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16402d040 con 0x7fd174159b20 2023-10-16T10:07:33.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.080+0000 7fd171ffb640 1 -- 172.21.15.84:0/639646100 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd164031810 con 0x7fd174159b20 2023-10-16T10:07:33.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.080+0000 7fd171ffb640 1 -- 172.21.15.84:0/639646100 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164031ba0 con 0x7fd174159b20 2023-10-16T10:07:33.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.080+0000 7fd17932c640 1 -- 172.21.15.84:0/639646100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd174159b20 msgr2=0x7fd17406bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:33.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.080+0000 7fd17932c640 1 --2- 172.21.15.84:0/639646100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd174159b20 0x7fd17406bbe0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fd1640097a0 tx=0x7fd16402b8b0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.080+0000 7fd17932c640 1 -- 172.21.15.84:0/639646100 shutdown_connections 2023-10-16T10:07:33.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.080+0000 7fd17932c640 1 --2- 172.21.15.84:0/639646100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1740b0b20 0x7fd17406fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.080+0000 7fd17932c640 1 --2- 172.21.15.84:0/639646100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd174159b20 0x7fd17406bbe0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.080+0000 7fd17932c640 1 -- 172.21.15.84:0/639646100 >> 172.21.15.84:0/639646100 conn(0x7fd1740cb410 msgr2=0x7fd1740cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:33.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.081+0000 7fd17932c640 1 -- 172.21.15.84:0/639646100 shutdown_connections 2023-10-16T10:07:33.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.081+0000 7fd17932c640 1 -- 172.21.15.84:0/639646100 wait complete. 2023-10-16T10:07:33.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.082+0000 7fd17932c640 1 Processor -- start 2023-10-16T10:07:33.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.082+0000 7fd17932c640 1 -- start start 2023-10-16T10:07:33.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.083+0000 7fd17932c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1740b0b20 0x7fd1740c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:33.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.083+0000 7fd17932c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd174159b20 0x7fd174148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:33.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.083+0000 7fd17932c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd174159f00 0x7fd1740c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:33.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.083+0000 7fd17932c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1740c6290 con 0x7fd1740b0b20 2023-10-16T10:07:33.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.083+0000 7fd17932c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1740b2ef0 con 0x7fd174159f00 2023-10-16T10:07:33.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.083+0000 7fd17932c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1740c8a60 con 0x7fd174159b20 2023-10-16T10:07:33.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.083+0000 7fd172ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1740b0b20 0x7fd1740c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:33.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.084+0000 7fd172ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1740b0b20 0x7fd1740c5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52122/0 (socket says 172.21.15.84:52122) 2023-10-16T10:07:33.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.084+0000 7fd172ffd640 1 -- 172.21.15.84:0/364294619 learned_addr learned my addr 172.21.15.84:0/364294619 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:33.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.084+0000 7fd172ffd640 1 -- 172.21.15.84:0/364294619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd174159b20 msgr2=0x7fd174148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:33.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.084+0000 7fd172ffd640 1 --2- 172.21.15.84:0/364294619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd174159b20 0x7fd174148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.084+0000 7fd172ffd640 1 -- 172.21.15.84:0/364294619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd174159f00 msgr2=0x7fd1740c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:33.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.084+0000 7fd172ffd640 1 --2- 172.21.15.84:0/364294619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd174159f00 0x7fd1740c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.084+0000 7fd172ffd640 1 -- 172.21.15.84:0/364294619 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd164009400 con 0x7fd1740b0b20 2023-10-16T10:07:33.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.084+0000 7fd172ffd640 1 --2- 172.21.15.84:0/364294619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1740b0b20 0x7fd1740c5080 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fd164000c00 tx=0x7fd1640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:33.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.084+0000 7fd15b7fe640 1 -- 172.21.15.84:0/364294619 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16403ba60 con 0x7fd1740b0b20 2023-10-16T10:07:33.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.085+0000 7fd15b7fe640 1 -- 172.21.15.84:0/364294619 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16403bbc0 con 0x7fd1740b0b20 2023-10-16T10:07:33.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.085+0000 7fd17932c640 1 -- 172.21.15.84:0/364294619 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd174148690 con 0x7fd1740b0b20 2023-10-16T10:07:33.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.085+0000 7fd15b7fe640 1 -- 172.21.15.84:0/364294619 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164004030 con 0x7fd1740b0b20 2023-10-16T10:07:33.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.085+0000 7fd17932c640 1 -- 172.21.15.84:0/364294619 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1740c8780 con 0x7fd1740b0b20 2023-10-16T10:07:33.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.086+0000 7fd17932c640 1 -- 172.21.15.84:0/364294619 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1740ae110 con 0x7fd1740b0b20 2023-10-16T10:07:33.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.087+0000 7fd15b7fe640 1 -- 172.21.15.84:0/364294619 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd164043070 con 0x7fd1740b0b20 2023-10-16T10:07:33.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.087+0000 7fd15b7fe640 1 --2- 172.21.15.84:0/364294619 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fd16006af90 0x7fd16006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:33.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.088+0000 7fd15b7fe640 1 -- 172.21.15.84:0/364294619 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1640ab7e0 con 0x7fd1740b0b20 2023-10-16T10:07:33.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.088+0000 7fd1727fc640 1 --2- 172.21.15.84:0/364294619 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fd16006af90 0x7fd16006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:33.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.088+0000 7fd1727fc640 1 --2- 172.21.15.84:0/364294619 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fd16006af90 0x7fd16006d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd168007fd0 tx=0x7fd168007960 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:33.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.092+0000 7fd15b7fe640 1 -- 172.21.15.84:0/364294619 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd1640765d0 con 0x7fd1740b0b20 2023-10-16T10:07:33.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.284+0000 7fd17932c640 1 -- 172.21.15.84:0/364294619 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1740b2ce0 con 0x7fd1740b0b20 2023-10-16T10:07:33.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.286+0000 7fd15b7fe640 1 -- 172.21.15.84:0/364294619 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16495 (secure 0 0 0) 0x7fd164035090 con 0x7fd1740b0b20 2023-10-16T10:07:33.288 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:33.288 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":124,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:07:30.355003+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:07:26.607199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-16T10:07:27.832706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:07:33.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.288+0000 7fd17932c640 1 -- 172.21.15.84:0/364294619 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fd16006af90 msgr2=0x7fd16006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:33.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.288+0000 7fd17932c640 1 --2- 172.21.15.84:0/364294619 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fd16006af90 0x7fd16006d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd168007fd0 tx=0x7fd168007960 comp rx=0 tx=0).stop 2023-10-16T10:07:33.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.288+0000 7fd17932c640 1 -- 172.21.15.84:0/364294619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1740b0b20 msgr2=0x7fd1740c5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:33.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.288+0000 7fd17932c640 1 --2- 172.21.15.84:0/364294619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1740b0b20 0x7fd1740c5080 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fd164000c00 tx=0x7fd1640046c0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.288+0000 7fd17932c640 1 -- 172.21.15.84:0/364294619 shutdown_connections 2023-10-16T10:07:33.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.288+0000 7fd17932c640 1 --2- 172.21.15.84:0/364294619 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fd16006af90 0x7fd16006d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.288+0000 7fd17932c640 1 --2- 172.21.15.84:0/364294619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd174159f00 0x7fd1740c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.288+0000 7fd17932c640 1 --2- 172.21.15.84:0/364294619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd174159b20 0x7fd174148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.289+0000 7fd17932c640 1 --2- 172.21.15.84:0/364294619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1740b0b20 0x7fd1740c5080 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.289+0000 7fd17932c640 1 -- 172.21.15.84:0/364294619 >> 172.21.15.84:0/364294619 conn(0x7fd1740cb410 msgr2=0x7fd17406fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:33.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.289+0000 7fd17932c640 1 -- 172.21.15.84:0/364294619 shutdown_connections 2023-10-16T10:07:33.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.289+0000 7fd17932c640 1 -- 172.21.15.84:0/364294619 wait complete. 2023-10-16T10:07:33.310 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T10:07:33.310 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:07:33.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.511+0000 7f05acbe7640 1 Processor -- start 2023-10-16T10:07:33.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.511+0000 7f05acbe7640 1 -- start start 2023-10-16T10:07:33.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.513+0000 7f05acbe7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f05a8159f00 0x7f05a80b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:33.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.513+0000 7f05acbe7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a806f7f0 con 0x7f05a815d880 2023-10-16T10:07:33.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.513+0000 7f05acbe7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a806c5e0 con 0x7f05a8159b20 2023-10-16T10:07:33.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.513+0000 7f05acbe7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a806c8c0 con 0x7f05a8159f00 2023-10-16T10:07:33.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.513+0000 7f05a6d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f05a815d880 0x7f05a806fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37828/0 (socket says 172.21.15.84:37828) 2023-10-16T10:07:33.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.513+0000 7f05a6d9d640 1 -- 172.21.15.84:0/3599636169 learned_addr learned my addr 172.21.15.84:0/3599636169 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:33.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.514+0000 7f059ffff640 1 --2- 172.21.15.84:0/3599636169 >> v2:172.21.15.157:3300/0 conn(0x7f05a8159f00 0x7f05a80b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:33.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.514+0000 7f05a5d9b640 1 -- 172.21.15.84:0/3599636169 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785358759 0 0) 0x7f05a806f7f0 con 0x7f05a815d880 2023-10-16T10:07:33.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.514+0000 7f05a5d9b640 1 -- 172.21.15.84:0/3599636169 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0594003520 con 0x7f05a815d880 2023-10-16T10:07:33.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.514+0000 7f05a5d9b640 1 -- 172.21.15.84:0/3599636169 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1234952819 0 0) 0x7f0594003520 con 0x7f05a815d880 2023-10-16T10:07:33.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.514+0000 7f05a5d9b640 1 -- 172.21.15.84:0/3599636169 >> v2:172.21.15.157:3300/0 conn(0x7f05a8159f00 msgr2=0x7f05a80b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:33.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.514+0000 7f05a5d9b640 1 --2- 172.21.15.84:0/3599636169 >> v2:172.21.15.157:3300/0 conn(0x7f05a8159f00 0x7f05a80b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.514+0000 7f05a5d9b640 1 -- 172.21.15.84:0/3599636169 >> v1:172.21.15.157:6789/0 conn(0x7f05a8159b20 legacy=0x7f05a806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:33.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.514+0000 7f05a5d9b640 1 -- 172.21.15.84:0/3599636169 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a806cba0 con 0x7f05a815d880 2023-10-16T10:07:33.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.514+0000 7f05a5d9b640 1 -- 172.21.15.84:0/3599636169 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f05a00019f0 con 0x7f05a815d880 2023-10-16T10:07:33.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.514+0000 7f05a5d9b640 1 -- 172.21.15.84:0/3599636169 >> v1:172.21.15.84:6789/0 conn(0x7f05a815d880 legacy=0x7f05a806fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:33.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.514+0000 7f05a5d9b640 1 --2- 172.21.15.84:0/3599636169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0594003970 0x7f0594003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:33.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.514+0000 7f05a5d9b640 1 --2- 172.21.15.84:0/3599636169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05940047c0 0x7f0594004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:33.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.514+0000 7f05a5d9b640 1 --2- 172.21.15.84:0/3599636169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0594005180 0x7f059400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:33.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.514+0000 7f05a5d9b640 1 -- 172.21.15.84:0/3599636169 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a806c8c0 con 0x7f0594003970 2023-10-16T10:07:33.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.514+0000 7f05a5d9b640 1 -- 172.21.15.84:0/3599636169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059400c020 con 0x7f0594005180 2023-10-16T10:07:33.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.514+0000 7f05a5d9b640 1 -- 172.21.15.84:0/3599636169 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059400c1e0 con 0x7f05940047c0 2023-10-16T10:07:33.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.515+0000 7f05a659c640 1 --2- 172.21.15.84:0/3599636169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0594003970 0x7f0594003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:33.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.515+0000 7f05a659c640 1 -- 172.21.15.84:0/3599636169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05940047c0 msgr2=0x7f0594004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:33.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.515+0000 7f05a659c640 1 --2- 172.21.15.84:0/3599636169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05940047c0 0x7f0594004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.515+0000 7f05a659c640 1 -- 172.21.15.84:0/3599636169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0594005180 msgr2=0x7f059400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:33.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.515+0000 7f05a659c640 1 --2- 172.21.15.84:0/3599636169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0594005180 0x7f059400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.515+0000 7f05a659c640 1 -- 172.21.15.84:0/3599636169 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0594003520 con 0x7f0594003970 2023-10-16T10:07:33.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.515+0000 7f05a6d9d640 1 -- 172.21.15.84:0/3599636169 reap_dead start 2023-10-16T10:07:33.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.516+0000 7f05a659c640 1 --2- 172.21.15.84:0/3599636169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0594003970 0x7f0594003d90 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f0598001750 tx=0x7f05940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334b936b004915d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:33.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.516+0000 7f05a5d9b640 1 -- 172.21.15.84:0/3599636169 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05980052d0 con 0x7f0594003970 2023-10-16T10:07:33.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.516+0000 7f05a5d9b640 1 -- 172.21.15.84:0/3599636169 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059802e050 con 0x7f0594003970 2023-10-16T10:07:33.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.516+0000 7f05a5d9b640 1 -- 172.21.15.84:0/3599636169 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598003830 con 0x7f0594003970 2023-10-16T10:07:33.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.517+0000 7f05acbe7640 1 -- 172.21.15.84:0/3599636169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0594003970 msgr2=0x7f0594003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:33.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.517+0000 7f05acbe7640 1 --2- 172.21.15.84:0/3599636169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0594003970 0x7f0594003d90 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f0598001750 tx=0x7f05940057f0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.517+0000 7f05acbe7640 1 -- 172.21.15.84:0/3599636169 shutdown_connections 2023-10-16T10:07:33.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.517+0000 7f05acbe7640 1 --2- 172.21.15.84:0/3599636169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0594003970 0x7f0594003d90 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.517+0000 7f05acbe7640 1 -- 172.21.15.84:0/3599636169 >> 172.21.15.84:0/3599636169 conn(0x7f05a80cb410 msgr2=0x7f05a80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:33.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.517+0000 7f05acbe7640 1 -- 172.21.15.84:0/3599636169 shutdown_connections 2023-10-16T10:07:33.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.517+0000 7f05acbe7640 1 -- 172.21.15.84:0/3599636169 wait complete. 2023-10-16T10:07:33.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.519+0000 7f05acbe7640 1 Processor -- start 2023-10-16T10:07:33.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.519+0000 7f05acbe7640 1 -- start start 2023-10-16T10:07:33.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.520+0000 7f05acbe7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0594003970 0x7f05a815a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:33.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.520+0000 7f05acbe7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a80b2470 0x7f05a80b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:33.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.520+0000 7f05acbe7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05a8070330 0x7f05a81f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:33.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.520+0000 7f05acbe7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80b2a30 con 0x7f05a80b2470 2023-10-16T10:07:33.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.520+0000 7f05acbe7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80b2d10 con 0x7f0594003970 2023-10-16T10:07:33.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.520+0000 7f05acbe7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80b2ff0 con 0x7f05a8070330 2023-10-16T10:07:33.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.520+0000 7f059ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a80b2470 0x7f05a80b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:33.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.520+0000 7f059ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a80b2470 0x7f05a80b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52154/0 (socket says 172.21.15.84:52154) 2023-10-16T10:07:33.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.520+0000 7f059ffff640 1 -- 172.21.15.84:0/1992994203 learned_addr learned my addr 172.21.15.84:0/1992994203 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:33.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.521+0000 7f05a659c640 1 --2- 172.21.15.84:0/1992994203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0594003970 0x7f05a815a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:33.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.521+0000 7f05a6d9d640 1 --2- 172.21.15.84:0/1992994203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05a8070330 0x7f05a81f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:33.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.521+0000 7f059ffff640 1 -- 172.21.15.84:0/1992994203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05a8070330 msgr2=0x7f05a81f24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:33.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.521+0000 7f059ffff640 1 --2- 172.21.15.84:0/1992994203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05a8070330 0x7f05a81f24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.521+0000 7f059ffff640 1 -- 172.21.15.84:0/1992994203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0594003970 msgr2=0x7f05a815a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:33.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.521+0000 7f059ffff640 1 --2- 172.21.15.84:0/1992994203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0594003970 0x7f05a815a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.521+0000 7f059ffff640 1 -- 172.21.15.84:0/1992994203 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a80b32d0 con 0x7f05a80b2470 2023-10-16T10:07:33.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.521+0000 7f05a659c640 1 --2- 172.21.15.84:0/1992994203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0594003970 0x7f05a815a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:33.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.521+0000 7f059ffff640 1 --2- 172.21.15.84:0/1992994203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a80b2470 0x7f05a80b3ad0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f059000e7a0 tx=0x7f059000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:33.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.521+0000 7f059effd640 1 -- 172.21.15.84:0/1992994203 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059000ee50 con 0x7f05a80b2470 2023-10-16T10:07:33.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.521+0000 7f059effd640 1 -- 172.21.15.84:0/1992994203 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059000c940 con 0x7f05a80b2470 2023-10-16T10:07:33.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.521+0000 7f059effd640 1 -- 172.21.15.84:0/1992994203 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059000ccf0 con 0x7f05a80b2470 2023-10-16T10:07:33.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.521+0000 7f05acbe7640 1 -- 172.21.15.84:0/1992994203 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a80b35b0 con 0x7f05a80b2470 2023-10-16T10:07:33.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.522+0000 7f05acbe7640 1 -- 172.21.15.84:0/1992994203 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a80b4010 con 0x7f05a80b2470 2023-10-16T10:07:33.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.523+0000 7f05acbe7640 1 -- 172.21.15.84:0/1992994203 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a804e160 con 0x7f05a80b2470 2023-10-16T10:07:33.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.523+0000 7f059effd640 1 -- 172.21.15.84:0/1992994203 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0590013470 con 0x7f05a80b2470 2023-10-16T10:07:33.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.524+0000 7f059effd640 1 --2- 172.21.15.84:0/1992994203 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f058806af90 0x7f058806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:33.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.524+0000 7f059effd640 1 -- 172.21.15.84:0/1992994203 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05900898b0 con 0x7f05a80b2470 2023-10-16T10:07:33.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.525+0000 7f05a659c640 1 --2- 172.21.15.84:0/1992994203 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f058806af90 0x7f058806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:33.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.525+0000 7f05a659c640 1 --2- 172.21.15.84:0/1992994203 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f058806af90 0x7f058806d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0598033000 tx=0x7f0598009030 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:33.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.529+0000 7f059effd640 1 -- 172.21.15.84:0/1992994203 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0590054720 con 0x7f05a80b2470 2023-10-16T10:07:33.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.714+0000 7f05acbe7640 1 -- 172.21.15.84:0/1992994203 --> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05a80b2e80 con 0x7f058806af90 2023-10-16T10:07:33.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.726+0000 7f059effd640 1 -- 172.21.15.84:0/1992994203 <== mgr.7794 v2:172.21.15.84:6832/4292025878 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287707 (secure 0 0 0) 0x7f05a80b2e80 con 0x7f058806af90 2023-10-16T10:07:33.731 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:33.732 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T10:07:33.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.731+0000 7f05acbe7640 1 -- 172.21.15.84:0/1992994203 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f058806af90 msgr2=0x7f058806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:33.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.731+0000 7f05acbe7640 1 --2- 172.21.15.84:0/1992994203 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f058806af90 0x7f058806d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0598033000 tx=0x7f0598009030 comp rx=0 tx=0).stop 2023-10-16T10:07:33.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.731+0000 7f05acbe7640 1 -- 172.21.15.84:0/1992994203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a80b2470 msgr2=0x7f05a80b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:33.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.731+0000 7f05acbe7640 1 --2- 172.21.15.84:0/1992994203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a80b2470 0x7f05a80b3ad0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f059000e7a0 tx=0x7f059000ec60 comp rx=0 tx=0).stop 2023-10-16T10:07:33.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.731+0000 7f05acbe7640 1 -- 172.21.15.84:0/1992994203 shutdown_connections 2023-10-16T10:07:33.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.731+0000 7f05acbe7640 1 --2- 172.21.15.84:0/1992994203 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f058806af90 0x7f058806d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.731+0000 7f05acbe7640 1 --2- 172.21.15.84:0/1992994203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05a8070330 0x7f05a81f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.731+0000 7f05acbe7640 1 --2- 172.21.15.84:0/1992994203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a80b2470 0x7f05a80b3ad0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.731+0000 7f05acbe7640 1 --2- 172.21.15.84:0/1992994203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0594003970 0x7f05a815a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:33.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.731+0000 7f05acbe7640 1 -- 172.21.15.84:0/1992994203 >> 172.21.15.84:0/1992994203 conn(0x7f05a80cb410 msgr2=0x7f05a80ad5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:33.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.732+0000 7f05acbe7640 1 -- 172.21.15.84:0/1992994203 shutdown_connections 2023-10-16T10:07:33.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:33.732+0000 7f05acbe7640 1 -- 172.21.15.84:0/1992994203 wait complete. 2023-10-16T10:07:33.752 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-16T10:07:33.455588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":928,"num_read_kb":1790,"num_write":365,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2662,"ondisk_log_size":2662,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333440,"kb_used_data":13192,"kb_used_omap":12,"kb_used_meta":320115,"kb_avail":754641280,"statfs":{"total":773094113280,"available":772752670720,"internally_reserved":0,"allocated":13508608,"data_stored":7037284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":327798329},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001718"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.038152+0000","last_change":"2023-10-16T10:07:29.382093+0000","last_active":"2023-10-16T10:07:31.038152+0000","last_peered":"2023-10-16T10:07:31.038152+0000","last_clean":"2023-10-16T10:07:31.038152+0000","last_became_active":"2023-10-16T10:07:29.381687+0000","last_became_peered":"2023-10-16T10:07:29.381687+0000","last_unstale":"2023-10-16T10:07:31.038152+0000","last_undegraded":"2023-10-16T10:07:31.038152+0000","last_fullsized":"2023-10-16T10:07:31.038152+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:26.661030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.039568+0000","last_change":"2023-10-16T10:07:29.379842+0000","last_active":"2023-10-16T10:07:31.039568+0000","last_peered":"2023-10-16T10:07:31.039568+0000","last_clean":"2023-10-16T10:07:31.039568+0000","last_became_active":"2023-10-16T10:07:29.379482+0000","last_became_peered":"2023-10-16T10:07:29.379482+0000","last_unstale":"2023-10-16T10:07:31.039568+0000","last_undegraded":"2023-10-16T10:07:31.039568+0000","last_fullsized":"2023-10-16T10:07:31.039568+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:33:28.020956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.037563+0000","last_change":"2023-10-16T10:07:29.386733+0000","last_active":"2023-10-16T10:07:31.037563+0000","last_peered":"2023-10-16T10:07:31.037563+0000","last_clean":"2023-10-16T10:07:31.037563+0000","last_became_active":"2023-10-16T10:07:29.382142+0000","last_became_peered":"2023-10-16T10:07:29.382142+0000","last_unstale":"2023-10-16T10:07:31.037563+0000","last_undegraded":"2023-10-16T10:07:31.037563+0000","last_fullsized":"2023-10-16T10:07:31.037563+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:24.429514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.038414+0000","last_change":"2023-10-16T10:07:29.371178+0000","last_active":"2023-10-16T10:07:31.038414+0000","last_peered":"2023-10-16T10:07:31.038414+0000","last_clean":"2023-10-16T10:07:31.038414+0000","last_became_active":"2023-10-16T10:07:29.369217+0000","last_became_peered":"2023-10-16T10:07:29.369217+0000","last_unstale":"2023-10-16T10:07:31.038414+0000","last_undegraded":"2023-10-16T10:07:31.038414+0000","last_fullsized":"2023-10-16T10:07:31.038414+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:36:21.816584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.372732+0000","last_change":"2023-10-16T10:07:29.368997+0000","last_active":"2023-10-16T10:07:30.372732+0000","last_peered":"2023-10-16T10:07:30.372732+0000","last_clean":"2023-10-16T10:07:30.372732+0000","last_became_active":"2023-10-16T10:07:29.368028+0000","last_became_peered":"2023-10-16T10:07:29.368028+0000","last_unstale":"2023-10-16T10:07:30.372732+0000","last_undegraded":"2023-10-16T10:07:30.372732+0000","last_fullsized":"2023-10-16T10:07:30.372732+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:08:36.998720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036734+0000","last_change":"2023-10-16T10:07:29.372561+0000","last_active":"2023-10-16T10:07:31.036734+0000","last_peered":"2023-10-16T10:07:31.036734+0000","last_clean":"2023-10-16T10:07:31.036734+0000","last_became_active":"2023-10-16T10:07:29.372275+0000","last_became_peered":"2023-10-16T10:07:29.372275+0000","last_unstale":"2023-10-16T10:07:31.036734+0000","last_undegraded":"2023-10-16T10:07:31.036734+0000","last_fullsized":"2023-10-16T10:07:31.036734+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:25:12.697183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036416+0000","last_change":"2023-10-16T10:07:29.372938+0000","last_active":"2023-10-16T10:07:31.036416+0000","last_peered":"2023-10-16T10:07:31.036416+0000","last_clean":"2023-10-16T10:07:31.036416+0000","last_became_active":"2023-10-16T10:07:29.372197+0000","last_became_peered":"2023-10-16T10:07:29.372197+0000","last_unstale":"2023-10-16T10:07:31.036416+0000","last_undegraded":"2023-10-16T10:07:31.036416+0000","last_fullsized":"2023-10-16T10:07:31.036416+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:57:48.375352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036049+0000","last_change":"2023-10-16T10:07:29.377578+0000","last_active":"2023-10-16T10:07:31.036049+0000","last_peered":"2023-10-16T10:07:31.036049+0000","last_clean":"2023-10-16T10:07:31.036049+0000","last_became_active":"2023-10-16T10:07:29.377291+0000","last_became_peered":"2023-10-16T10:07:29.377291+0000","last_unstale":"2023-10-16T10:07:31.036049+0000","last_undegraded":"2023-10-16T10:07:31.036049+0000","last_fullsized":"2023-10-16T10:07:31.036049+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:40.382024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.035912+0000","last_change":"2023-10-16T10:07:29.369758+0000","last_active":"2023-10-16T10:07:31.035912+0000","last_peered":"2023-10-16T10:07:31.035912+0000","last_clean":"2023-10-16T10:07:31.035912+0000","last_became_active":"2023-10-16T10:07:29.369464+0000","last_became_peered":"2023-10-16T10:07:29.369464+0000","last_unstale":"2023-10-16T10:07:31.035912+0000","last_undegraded":"2023-10-16T10:07:31.035912+0000","last_fullsized":"2023-10-16T10:07:31.035912+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:11:24.261638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.372287+0000","last_change":"2023-10-16T10:07:29.376624+0000","last_active":"2023-10-16T10:07:30.372287+0000","last_peered":"2023-10-16T10:07:30.372287+0000","last_clean":"2023-10-16T10:07:30.372287+0000","last_became_active":"2023-10-16T10:07:29.372821+0000","last_became_peered":"2023-10-16T10:07:29.372821+0000","last_unstale":"2023-10-16T10:07:30.372287+0000","last_undegraded":"2023-10-16T10:07:30.372287+0000","last_fullsized":"2023-10-16T10:07:30.372287+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:26:48.170692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036971+0000","last_change":"2023-10-16T10:07:29.376016+0000","last_active":"2023-10-16T10:07:31.036971+0000","last_peered":"2023-10-16T10:07:31.036971+0000","last_clean":"2023-10-16T10:07:31.036971+0000","last_became_active":"2023-10-16T10:07:29.373333+0000","last_became_peered":"2023-10-16T10:07:29.373333+0000","last_unstale":"2023-10-16T10:07:31.036971+0000","last_undegraded":"2023-10-16T10:07:31.036971+0000","last_fullsized":"2023-10-16T10:07:31.036971+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:48.025764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.035987+0000","last_change":"2023-10-16T10:07:29.374068+0000","last_active":"2023-10-16T10:07:31.035987+0000","last_peered":"2023-10-16T10:07:31.035987+0000","last_clean":"2023-10-16T10:07:31.035987+0000","last_became_active":"2023-10-16T10:07:29.373810+0000","last_became_peered":"2023-10-16T10:07:29.373810+0000","last_unstale":"2023-10-16T10:07:31.035987+0000","last_undegraded":"2023-10-16T10:07:31.035987+0000","last_fullsized":"2023-10-16T10:07:31.035987+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:17.260480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.039352+0000","last_change":"2023-10-16T10:07:29.358718+0000","last_active":"2023-10-16T10:07:31.039352+0000","last_peered":"2023-10-16T10:07:31.039352+0000","last_clean":"2023-10-16T10:07:31.039352+0000","last_became_active":"2023-10-16T10:07:29.356998+0000","last_became_peered":"2023-10-16T10:07:29.356998+0000","last_unstale":"2023-10-16T10:07:31.039352+0000","last_undegraded":"2023-10-16T10:07:31.039352+0000","last_fullsized":"2023-10-16T10:07:31.039352+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:33:43.615838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.367762+0000","last_change":"2023-10-16T10:07:29.372847+0000","last_active":"2023-10-16T10:07:30.367762+0000","last_peered":"2023-10-16T10:07:30.367762+0000","last_clean":"2023-10-16T10:07:30.367762+0000","last_became_active":"2023-10-16T10:07:29.369970+0000","last_became_peered":"2023-10-16T10:07:29.369970+0000","last_unstale":"2023-10-16T10:07:30.367762+0000","last_undegraded":"2023-10-16T10:07:30.367762+0000","last_fullsized":"2023-10-16T10:07:30.367762+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:47.868903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.037078+0000","last_change":"2023-10-16T10:07:29.367146+0000","last_active":"2023-10-16T10:07:31.037078+0000","last_peered":"2023-10-16T10:07:31.037078+0000","last_clean":"2023-10-16T10:07:31.037078+0000","last_became_active":"2023-10-16T10:07:29.365999+0000","last_became_peered":"2023-10-16T10:07:29.365999+0000","last_unstale":"2023-10-16T10:07:31.037078+0000","last_undegraded":"2023-10-16T10:07:31.037078+0000","last_fullsized":"2023-10-16T10:07:31.037078+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:02:50.044180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036744+0000","last_change":"2023-10-16T10:07:29.373230+0000","last_active":"2023-10-16T10:07:31.036744+0000","last_peered":"2023-10-16T10:07:31.036744+0000","last_clean":"2023-10-16T10:07:31.036744+0000","last_became_active":"2023-10-16T10:07:29.372983+0000","last_became_peered":"2023-10-16T10:07:29.372983+0000","last_unstale":"2023-10-16T10:07:31.036744+0000","last_undegraded":"2023-10-16T10:07:31.036744+0000","last_fullsized":"2023-10-16T10:07:31.036744+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:25.040628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.039963+0000","last_change":"2023-10-16T10:07:29.392036+0000","last_active":"2023-10-16T10:07:31.039963+0000","last_peered":"2023-10-16T10:07:31.039963+0000","last_clean":"2023-10-16T10:07:31.039963+0000","last_became_active":"2023-10-16T10:07:29.390853+0000","last_became_peered":"2023-10-16T10:07:29.390853+0000","last_unstale":"2023-10-16T10:07:31.039963+0000","last_undegraded":"2023-10-16T10:07:31.039963+0000","last_fullsized":"2023-10-16T10:07:31.039963+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:58:13.282872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.038705+0000","last_change":"2023-10-16T10:07:29.367515+0000","last_active":"2023-10-16T10:07:31.038705+0000","last_peered":"2023-10-16T10:07:31.038705+0000","last_clean":"2023-10-16T10:07:31.038705+0000","last_became_active":"2023-10-16T10:07:29.367182+0000","last_became_peered":"2023-10-16T10:07:29.367182+0000","last_unstale":"2023-10-16T10:07:31.038705+0000","last_undegraded":"2023-10-16T10:07:31.038705+0000","last_fullsized":"2023-10-16T10:07:31.038705+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:45:14.229013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.368022+0000","last_change":"2023-10-16T10:07:29.374134+0000","last_active":"2023-10-16T10:07:30.368022+0000","last_peered":"2023-10-16T10:07:30.368022+0000","last_clean":"2023-10-16T10:07:30.368022+0000","last_became_active":"2023-10-16T10:07:29.373871+0000","last_became_peered":"2023-10-16T10:07:29.373871+0000","last_unstale":"2023-10-16T10:07:30.368022+0000","last_undegraded":"2023-10-16T10:07:30.368022+0000","last_fullsized":"2023-10-16T10:07:30.368022+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:40.382024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036362+0000","last_change":"2023-10-16T10:07:29.356960+0000","last_active":"2023-10-16T10:07:31.036362+0000","last_peered":"2023-10-16T10:07:31.036362+0000","last_clean":"2023-10-16T10:07:31.036362+0000","last_became_active":"2023-10-16T10:07:29.356180+0000","last_became_peered":"2023-10-16T10:07:29.356180+0000","last_unstale":"2023-10-16T10:07:31.036362+0000","last_undegraded":"2023-10-16T10:07:31.036362+0000","last_fullsized":"2023-10-16T10:07:31.036362+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:16:22.572793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.367597+0000","last_change":"2023-10-16T10:07:29.376964+0000","last_active":"2023-10-16T10:07:30.367597+0000","last_peered":"2023-10-16T10:07:30.367597+0000","last_clean":"2023-10-16T10:07:30.367597+0000","last_became_active":"2023-10-16T10:07:29.376643+0000","last_became_peered":"2023-10-16T10:07:29.376643+0000","last_unstale":"2023-10-16T10:07:30.367597+0000","last_undegraded":"2023-10-16T10:07:30.367597+0000","last_fullsized":"2023-10-16T10:07:30.367597+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:12.622336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.038569+0000","last_change":"2023-10-16T10:07:29.379469+0000","last_active":"2023-10-16T10:07:31.038569+0000","last_peered":"2023-10-16T10:07:31.038569+0000","last_clean":"2023-10-16T10:07:31.038569+0000","last_became_active":"2023-10-16T10:07:29.379148+0000","last_became_peered":"2023-10-16T10:07:29.379148+0000","last_unstale":"2023-10-16T10:07:31.038569+0000","last_undegraded":"2023-10-16T10:07:31.038569+0000","last_fullsized":"2023-10-16T10:07:31.038569+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:06.419087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.035664+0000","last_change":"2023-10-16T10:07:29.377683+0000","last_active":"2023-10-16T10:07:31.035664+0000","last_peered":"2023-10-16T10:07:31.035664+0000","last_clean":"2023-10-16T10:07:31.035664+0000","last_became_active":"2023-10-16T10:07:29.377372+0000","last_became_peered":"2023-10-16T10:07:29.377372+0000","last_unstale":"2023-10-16T10:07:31.035664+0000","last_undegraded":"2023-10-16T10:07:31.035664+0000","last_fullsized":"2023-10-16T10:07:31.035664+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:00:14.321059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036953+0000","last_change":"2023-10-16T10:07:29.371436+0000","last_active":"2023-10-16T10:07:31.036953+0000","last_peered":"2023-10-16T10:07:31.036953+0000","last_clean":"2023-10-16T10:07:31.036953+0000","last_became_active":"2023-10-16T10:07:29.371150+0000","last_became_peered":"2023-10-16T10:07:29.371150+0000","last_unstale":"2023-10-16T10:07:31.036953+0000","last_undegraded":"2023-10-16T10:07:31.036953+0000","last_fullsized":"2023-10-16T10:07:31.036953+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:09:49.494032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.372741+0000","last_change":"2023-10-16T10:07:29.382549+0000","last_active":"2023-10-16T10:07:30.372741+0000","last_peered":"2023-10-16T10:07:30.372741+0000","last_clean":"2023-10-16T10:07:30.372741+0000","last_became_active":"2023-10-16T10:07:29.380506+0000","last_became_peered":"2023-10-16T10:07:29.380506+0000","last_unstale":"2023-10-16T10:07:30.372741+0000","last_undegraded":"2023-10-16T10:07:30.372741+0000","last_fullsized":"2023-10-16T10:07:30.372741+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:33:28.020956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.039938+0000","last_change":"2023-10-16T10:07:29.386247+0000","last_active":"2023-10-16T10:07:31.039938+0000","last_peered":"2023-10-16T10:07:31.039938+0000","last_clean":"2023-10-16T10:07:31.039938+0000","last_became_active":"2023-10-16T10:07:29.385605+0000","last_became_peered":"2023-10-16T10:07:29.385605+0000","last_unstale":"2023-10-16T10:07:31.039938+0000","last_undegraded":"2023-10-16T10:07:31.039938+0000","last_fullsized":"2023-10-16T10:07:31.039938+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:34.906651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.037230+0000","last_change":"2023-10-16T10:07:29.365734+0000","last_active":"2023-10-16T10:07:31.037230+0000","last_peered":"2023-10-16T10:07:31.037230+0000","last_clean":"2023-10-16T10:07:31.037230+0000","last_became_active":"2023-10-16T10:07:29.365459+0000","last_became_peered":"2023-10-16T10:07:29.365459+0000","last_unstale":"2023-10-16T10:07:31.037230+0000","last_undegraded":"2023-10-16T10:07:31.037230+0000","last_fullsized":"2023-10-16T10:07:31.037230+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:54.133205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.372597+0000","last_change":"2023-10-16T10:07:29.374472+0000","last_active":"2023-10-16T10:07:30.372597+0000","last_peered":"2023-10-16T10:07:30.372597+0000","last_clean":"2023-10-16T10:07:30.372597+0000","last_became_active":"2023-10-16T10:07:29.374039+0000","last_became_peered":"2023-10-16T10:07:29.374039+0000","last_unstale":"2023-10-16T10:07:30.372597+0000","last_undegraded":"2023-10-16T10:07:30.372597+0000","last_fullsized":"2023-10-16T10:07:30.372597+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:40.382024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.038823+0000","last_change":"2023-10-16T10:07:29.384272+0000","last_active":"2023-10-16T10:07:31.038823+0000","last_peered":"2023-10-16T10:07:31.038823+0000","last_clean":"2023-10-16T10:07:31.038823+0000","last_became_active":"2023-10-16T10:07:29.383866+0000","last_became_peered":"2023-10-16T10:07:29.383866+0000","last_unstale":"2023-10-16T10:07:31.038823+0000","last_undegraded":"2023-10-16T10:07:31.038823+0000","last_fullsized":"2023-10-16T10:07:31.038823+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:08:36.998720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.367955+0000","last_change":"2023-10-16T10:07:29.368739+0000","last_active":"2023-10-16T10:07:30.367955+0000","last_peered":"2023-10-16T10:07:30.367955+0000","last_clean":"2023-10-16T10:07:30.367955+0000","last_became_active":"2023-10-16T10:07:29.368461+0000","last_became_peered":"2023-10-16T10:07:29.368461+0000","last_unstale":"2023-10-16T10:07:30.367955+0000","last_undegraded":"2023-10-16T10:07:30.367955+0000","last_fullsized":"2023-10-16T10:07:30.367955+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:26:48.170692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.037517+0000","last_change":"2023-10-16T10:07:29.374317+0000","last_active":"2023-10-16T10:07:31.037517+0000","last_peered":"2023-10-16T10:07:31.037517+0000","last_clean":"2023-10-16T10:07:31.037517+0000","last_became_active":"2023-10-16T10:07:29.373509+0000","last_became_peered":"2023-10-16T10:07:29.373509+0000","last_unstale":"2023-10-16T10:07:31.037517+0000","last_undegraded":"2023-10-16T10:07:31.037517+0000","last_fullsized":"2023-10-16T10:07:31.037517+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:43:25.862353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.035659+0000","last_change":"2023-10-16T10:07:29.375192+0000","last_active":"2023-10-16T10:07:31.035659+0000","last_peered":"2023-10-16T10:07:31.035659+0000","last_clean":"2023-10-16T10:07:31.035659+0000","last_became_active":"2023-10-16T10:07:29.373537+0000","last_became_peered":"2023-10-16T10:07:29.373537+0000","last_unstale":"2023-10-16T10:07:31.035659+0000","last_undegraded":"2023-10-16T10:07:31.035659+0000","last_fullsized":"2023-10-16T10:07:31.035659+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:49:03.178809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.368270+0000","last_change":"2023-10-16T10:07:29.368773+0000","last_active":"2023-10-16T10:07:30.368270+0000","last_peered":"2023-10-16T10:07:30.368270+0000","last_clean":"2023-10-16T10:07:30.368270+0000","last_became_active":"2023-10-16T10:07:29.366701+0000","last_became_peered":"2023-10-16T10:07:29.366701+0000","last_unstale":"2023-10-16T10:07:30.368270+0000","last_undegraded":"2023-10-16T10:07:30.368270+0000","last_fullsized":"2023-10-16T10:07:30.368270+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:08:36.998720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"123'4","reported_seq":16,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.395391+0000","last_change":"2023-10-16T10:07:28.361614+0000","last_active":"2023-10-16T10:07:29.395391+0000","last_peered":"2023-10-16T10:07:29.395391+0000","last_clean":"2023-10-16T10:07:29.395391+0000","last_became_active":"2023-10-16T10:07:28.361357+0000","last_became_peered":"2023-10-16T10:07:28.361357+0000","last_unstale":"2023-10-16T10:07:29.395391+0000","last_undegraded":"2023-10-16T10:07:29.395391+0000","last_fullsized":"2023-10-16T10:07:29.395391+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:27:20.422020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.038888+0000","last_change":"2023-10-16T10:07:29.373919+0000","last_active":"2023-10-16T10:07:31.038888+0000","last_peered":"2023-10-16T10:07:31.038888+0000","last_clean":"2023-10-16T10:07:31.038888+0000","last_became_active":"2023-10-16T10:07:29.373562+0000","last_became_peered":"2023-10-16T10:07:29.373562+0000","last_unstale":"2023-10-16T10:07:31.038888+0000","last_undegraded":"2023-10-16T10:07:31.038888+0000","last_fullsized":"2023-10-16T10:07:31.038888+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:07:13.553899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"123'1","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.392986+0000","last_change":"2023-10-16T10:07:28.370286+0000","last_active":"2023-10-16T10:07:29.392986+0000","last_peered":"2023-10-16T10:07:29.392986+0000","last_clean":"2023-10-16T10:07:29.392986+0000","last_became_active":"2023-10-16T10:07:28.369845+0000","last_became_peered":"2023-10-16T10:07:28.369845+0000","last_unstale":"2023-10-16T10:07:29.392986+0000","last_undegraded":"2023-10-16T10:07:29.392986+0000","last_fullsized":"2023-10-16T10:07:29.392986+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:33:42.598522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036378+0000","last_change":"2023-10-16T10:07:29.374864+0000","last_active":"2023-10-16T10:07:31.036378+0000","last_peered":"2023-10-16T10:07:31.036378+0000","last_clean":"2023-10-16T10:07:31.036378+0000","last_became_active":"2023-10-16T10:07:29.374593+0000","last_became_peered":"2023-10-16T10:07:29.374593+0000","last_unstale":"2023-10-16T10:07:31.036378+0000","last_undegraded":"2023-10-16T10:07:31.036378+0000","last_fullsized":"2023-10-16T10:07:31.036378+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:30.854910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"123'2","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.392985+0000","last_change":"2023-10-16T10:07:28.360255+0000","last_active":"2023-10-16T10:07:29.392985+0000","last_peered":"2023-10-16T10:07:29.392985+0000","last_clean":"2023-10-16T10:07:29.392985+0000","last_became_active":"2023-10-16T10:07:28.359785+0000","last_became_peered":"2023-10-16T10:07:28.359785+0000","last_unstale":"2023-10-16T10:07:29.392985+0000","last_undegraded":"2023-10-16T10:07:29.392985+0000","last_fullsized":"2023-10-16T10:07:29.392985+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:11.605020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036341+0000","last_change":"2023-10-16T10:07:29.377813+0000","last_active":"2023-10-16T10:07:31.036341+0000","last_peered":"2023-10-16T10:07:31.036341+0000","last_clean":"2023-10-16T10:07:31.036341+0000","last_became_active":"2023-10-16T10:07:29.377533+0000","last_became_peered":"2023-10-16T10:07:29.377533+0000","last_unstale":"2023-10-16T10:07:31.036341+0000","last_undegraded":"2023-10-16T10:07:31.036341+0000","last_fullsized":"2023-10-16T10:07:31.036341+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:18.346286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"123'2","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.394146+0000","last_change":"2023-10-16T10:07:28.367658+0000","last_active":"2023-10-16T10:07:29.394146+0000","last_peered":"2023-10-16T10:07:29.394146+0000","last_clean":"2023-10-16T10:07:29.394146+0000","last_became_active":"2023-10-16T10:07:28.367182+0000","last_became_peered":"2023-10-16T10:07:28.367182+0000","last_unstale":"2023-10-16T10:07:29.394146+0000","last_undegraded":"2023-10-16T10:07:29.394146+0000","last_fullsized":"2023-10-16T10:07:29.394146+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:27.586892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.369853+0000","last_change":"2023-10-16T10:07:29.377214+0000","last_active":"2023-10-16T10:07:30.369853+0000","last_peered":"2023-10-16T10:07:30.369853+0000","last_clean":"2023-10-16T10:07:30.369853+0000","last_became_active":"2023-10-16T10:07:29.373847+0000","last_became_peered":"2023-10-16T10:07:29.373847+0000","last_unstale":"2023-10-16T10:07:30.369853+0000","last_undegraded":"2023-10-16T10:07:30.369853+0000","last_fullsized":"2023-10-16T10:07:30.369853+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:00.092762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"123'1","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.392467+0000","last_change":"2023-10-16T10:07:28.357437+0000","last_active":"2023-10-16T10:07:29.392467+0000","last_peered":"2023-10-16T10:07:29.392467+0000","last_clean":"2023-10-16T10:07:29.392467+0000","last_became_active":"2023-10-16T10:07:28.356851+0000","last_became_peered":"2023-10-16T10:07:28.356851+0000","last_unstale":"2023-10-16T10:07:29.392467+0000","last_undegraded":"2023-10-16T10:07:29.392467+0000","last_fullsized":"2023-10-16T10:07:29.392467+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:08.827809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.037365+0000","last_change":"2023-10-16T10:07:29.374434+0000","last_active":"2023-10-16T10:07:31.037365+0000","last_peered":"2023-10-16T10:07:31.037365+0000","last_clean":"2023-10-16T10:07:31.037365+0000","last_became_active":"2023-10-16T10:07:29.373410+0000","last_became_peered":"2023-10-16T10:07:29.373410+0000","last_unstale":"2023-10-16T10:07:31.037365+0000","last_undegraded":"2023-10-16T10:07:31.037365+0000","last_fullsized":"2023-10-16T10:07:31.037365+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:25.786399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"123'1","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.393694+0000","last_change":"2023-10-16T10:07:28.359504+0000","last_active":"2023-10-16T10:07:29.393694+0000","last_peered":"2023-10-16T10:07:29.393694+0000","last_clean":"2023-10-16T10:07:29.393694+0000","last_became_active":"2023-10-16T10:07:28.359189+0000","last_became_peered":"2023-10-16T10:07:28.359189+0000","last_unstale":"2023-10-16T10:07:29.393694+0000","last_undegraded":"2023-10-16T10:07:29.393694+0000","last_fullsized":"2023-10-16T10:07:29.393694+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:27.207835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036494+0000","last_change":"2023-10-16T10:07:29.371383+0000","last_active":"2023-10-16T10:07:31.036494+0000","last_peered":"2023-10-16T10:07:31.036494+0000","last_clean":"2023-10-16T10:07:31.036494+0000","last_became_active":"2023-10-16T10:07:29.369223+0000","last_became_peered":"2023-10-16T10:07:29.369223+0000","last_unstale":"2023-10-16T10:07:31.036494+0000","last_undegraded":"2023-10-16T10:07:31.036494+0000","last_fullsized":"2023-10-16T10:07:31.036494+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:42:07.800731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.365532+0000","last_change":"2023-10-16T10:07:28.366092+0000","last_active":"2023-10-16T10:07:29.365532+0000","last_peered":"2023-10-16T10:07:29.365532+0000","last_clean":"2023-10-16T10:07:29.365532+0000","last_became_active":"2023-10-16T10:07:28.365734+0000","last_became_peered":"2023-10-16T10:07:28.365734+0000","last_unstale":"2023-10-16T10:07:29.365532+0000","last_undegraded":"2023-10-16T10:07:29.365532+0000","last_fullsized":"2023-10-16T10:07:29.365532+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:27.207835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.035303+0000","last_change":"2023-10-16T10:07:29.368389+0000","last_active":"2023-10-16T10:07:31.035303+0000","last_peered":"2023-10-16T10:07:31.035303+0000","last_clean":"2023-10-16T10:07:31.035303+0000","last_became_active":"2023-10-16T10:07:29.368032+0000","last_became_peered":"2023-10-16T10:07:29.368032+0000","last_unstale":"2023-10-16T10:07:31.035303+0000","last_undegraded":"2023-10-16T10:07:31.035303+0000","last_fullsized":"2023-10-16T10:07:31.035303+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:17:51.605125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.361176+0000","last_change":"2023-10-16T10:07:28.356999+0000","last_active":"2023-10-16T10:07:29.361176+0000","last_peered":"2023-10-16T10:07:29.361176+0000","last_clean":"2023-10-16T10:07:29.361176+0000","last_became_active":"2023-10-16T10:07:28.356707+0000","last_became_peered":"2023-10-16T10:07:28.356707+0000","last_unstale":"2023-10-16T10:07:29.361176+0000","last_undegraded":"2023-10-16T10:07:29.361176+0000","last_fullsized":"2023-10-16T10:07:29.361176+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:33.889335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.040130+0000","last_change":"2023-10-16T10:07:29.386744+0000","last_active":"2023-10-16T10:07:31.040130+0000","last_peered":"2023-10-16T10:07:31.040130+0000","last_clean":"2023-10-16T10:07:31.040130+0000","last_became_active":"2023-10-16T10:07:29.386476+0000","last_became_peered":"2023-10-16T10:07:29.386476+0000","last_unstale":"2023-10-16T10:07:31.040130+0000","last_undegraded":"2023-10-16T10:07:31.040130+0000","last_fullsized":"2023-10-16T10:07:31.040130+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:21:49.156446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.362781+0000","last_change":"2023-10-16T10:07:28.362884+0000","last_active":"2023-10-16T10:07:29.362781+0000","last_peered":"2023-10-16T10:07:29.362781+0000","last_clean":"2023-10-16T10:07:29.362781+0000","last_became_active":"2023-10-16T10:07:28.361604+0000","last_became_peered":"2023-10-16T10:07:28.361604+0000","last_unstale":"2023-10-16T10:07:29.362781+0000","last_undegraded":"2023-10-16T10:07:29.362781+0000","last_fullsized":"2023-10-16T10:07:29.362781+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:15.702581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.035956+0000","last_change":"2023-10-16T10:07:29.382243+0000","last_active":"2023-10-16T10:07:31.035956+0000","last_peered":"2023-10-16T10:07:31.035956+0000","last_clean":"2023-10-16T10:07:31.035956+0000","last_became_active":"2023-10-16T10:07:29.381957+0000","last_became_peered":"2023-10-16T10:07:29.381957+0000","last_unstale":"2023-10-16T10:07:31.035956+0000","last_undegraded":"2023-10-16T10:07:31.035956+0000","last_fullsized":"2023-10-16T10:07:31.035956+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:26.463597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.355031+0000","last_change":"2023-10-16T10:07:28.366495+0000","last_active":"2023-10-16T10:07:29.355031+0000","last_peered":"2023-10-16T10:07:29.355031+0000","last_clean":"2023-10-16T10:07:29.355031+0000","last_became_active":"2023-10-16T10:07:28.365028+0000","last_became_peered":"2023-10-16T10:07:28.365028+0000","last_unstale":"2023-10-16T10:07:29.355031+0000","last_undegraded":"2023-10-16T10:07:29.355031+0000","last_fullsized":"2023-10-16T10:07:29.355031+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:00.086702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.372867+0000","last_change":"2023-10-16T10:07:29.391473+0000","last_active":"2023-10-16T10:07:30.372867+0000","last_peered":"2023-10-16T10:07:30.372867+0000","last_clean":"2023-10-16T10:07:30.372867+0000","last_became_active":"2023-10-16T10:07:29.391128+0000","last_became_peered":"2023-10-16T10:07:29.391128+0000","last_unstale":"2023-10-16T10:07:30.372867+0000","last_undegraded":"2023-10-16T10:07:30.372867+0000","last_fullsized":"2023-10-16T10:07:30.372867+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:34.906651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"123'3","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.390415+0000","last_change":"2023-10-16T10:07:28.369936+0000","last_active":"2023-10-16T10:07:29.390415+0000","last_peered":"2023-10-16T10:07:29.390415+0000","last_clean":"2023-10-16T10:07:29.390415+0000","last_became_active":"2023-10-16T10:07:28.369613+0000","last_became_peered":"2023-10-16T10:07:28.369613+0000","last_unstale":"2023-10-16T10:07:29.390415+0000","last_undegraded":"2023-10-16T10:07:29.390415+0000","last_fullsized":"2023-10-16T10:07:29.390415+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:00:40.716420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.037529+0000","last_change":"2023-10-16T10:07:29.381218+0000","last_active":"2023-10-16T10:07:31.037529+0000","last_peered":"2023-10-16T10:07:31.037529+0000","last_clean":"2023-10-16T10:07:31.037529+0000","last_became_active":"2023-10-16T10:07:29.380889+0000","last_became_peered":"2023-10-16T10:07:29.380889+0000","last_unstale":"2023-10-16T10:07:31.037529+0000","last_undegraded":"2023-10-16T10:07:31.037529+0000","last_fullsized":"2023-10-16T10:07:31.037529+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:24:36.397822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"123'6","reported_seq":18,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.394464+0000","last_change":"2023-10-16T10:07:28.357341+0000","last_active":"2023-10-16T10:07:29.394464+0000","last_peered":"2023-10-16T10:07:29.394464+0000","last_clean":"2023-10-16T10:07:29.394464+0000","last_became_active":"2023-10-16T10:07:28.357050+0000","last_became_peered":"2023-10-16T10:07:28.357050+0000","last_unstale":"2023-10-16T10:07:29.394464+0000","last_undegraded":"2023-10-16T10:07:29.394464+0000","last_fullsized":"2023-10-16T10:07:29.394464+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:30:09.063022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.037616+0000","last_change":"2023-10-16T10:07:29.376998+0000","last_active":"2023-10-16T10:07:31.037616+0000","last_peered":"2023-10-16T10:07:31.037616+0000","last_clean":"2023-10-16T10:07:31.037616+0000","last_became_active":"2023-10-16T10:07:29.376740+0000","last_became_peered":"2023-10-16T10:07:29.376740+0000","last_unstale":"2023-10-16T10:07:31.037616+0000","last_undegraded":"2023-10-16T10:07:31.037616+0000","last_fullsized":"2023-10-16T10:07:31.037616+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:02.053186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"123'2","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.391985+0000","last_change":"2023-10-16T10:07:28.360250+0000","last_active":"2023-10-16T10:07:29.391985+0000","last_peered":"2023-10-16T10:07:29.391985+0000","last_clean":"2023-10-16T10:07:29.391985+0000","last_became_active":"2023-10-16T10:07:28.358653+0000","last_became_peered":"2023-10-16T10:07:28.358653+0000","last_unstale":"2023-10-16T10:07:29.391985+0000","last_undegraded":"2023-10-16T10:07:29.391985+0000","last_fullsized":"2023-10-16T10:07:29.391985+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:49.373492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036989+0000","last_change":"2023-10-16T10:07:29.388058+0000","last_active":"2023-10-16T10:07:31.036989+0000","last_peered":"2023-10-16T10:07:31.036989+0000","last_clean":"2023-10-16T10:07:31.036989+0000","last_became_active":"2023-10-16T10:07:29.382803+0000","last_became_peered":"2023-10-16T10:07:29.382803+0000","last_unstale":"2023-10-16T10:07:31.036989+0000","last_undegraded":"2023-10-16T10:07:31.036989+0000","last_fullsized":"2023-10-16T10:07:31.036989+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:17:22.404985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.348509+0000","last_change":"2023-10-16T10:07:28.358918+0000","last_active":"2023-10-16T10:07:29.348509+0000","last_peered":"2023-10-16T10:07:29.348509+0000","last_clean":"2023-10-16T10:07:29.348509+0000","last_became_active":"2023-10-16T10:07:28.358669+0000","last_became_peered":"2023-10-16T10:07:28.358669+0000","last_unstale":"2023-10-16T10:07:29.348509+0000","last_undegraded":"2023-10-16T10:07:29.348509+0000","last_fullsized":"2023-10-16T10:07:29.348509+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:13.028657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.035212+0000","last_change":"2023-10-16T10:07:29.366841+0000","last_active":"2023-10-16T10:07:31.035212+0000","last_peered":"2023-10-16T10:07:31.035212+0000","last_clean":"2023-10-16T10:07:31.035212+0000","last_became_active":"2023-10-16T10:07:29.365834+0000","last_became_peered":"2023-10-16T10:07:29.365834+0000","last_unstale":"2023-10-16T10:07:31.035212+0000","last_undegraded":"2023-10-16T10:07:31.035212+0000","last_fullsized":"2023-10-16T10:07:31.035212+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:44:59.979486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.343960+0000","last_change":"2023-10-16T10:07:28.363095+0000","last_active":"2023-10-16T10:07:29.343960+0000","last_peered":"2023-10-16T10:07:29.343960+0000","last_clean":"2023-10-16T10:07:29.343960+0000","last_became_active":"2023-10-16T10:07:28.362803+0000","last_became_peered":"2023-10-16T10:07:28.362803+0000","last_unstale":"2023-10-16T10:07:29.343960+0000","last_undegraded":"2023-10-16T10:07:29.343960+0000","last_fullsized":"2023-10-16T10:07:29.343960+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:30:31.668157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.367606+0000","last_change":"2023-10-16T10:07:29.362200+0000","last_active":"2023-10-16T10:07:30.367606+0000","last_peered":"2023-10-16T10:07:30.367606+0000","last_clean":"2023-10-16T10:07:30.367606+0000","last_became_active":"2023-10-16T10:07:29.356693+0000","last_became_peered":"2023-10-16T10:07:29.356693+0000","last_unstale":"2023-10-16T10:07:30.367606+0000","last_undegraded":"2023-10-16T10:07:30.367606+0000","last_fullsized":"2023-10-16T10:07:30.367606+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:27:21.439336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"123'8","reported_seq":20,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.391556+0000","last_change":"2023-10-16T10:07:28.365215+0000","last_active":"2023-10-16T10:07:29.391556+0000","last_peered":"2023-10-16T10:07:29.391556+0000","last_clean":"2023-10-16T10:07:29.391556+0000","last_became_active":"2023-10-16T10:07:28.364919+0000","last_became_peered":"2023-10-16T10:07:28.364919+0000","last_unstale":"2023-10-16T10:07:29.391556+0000","last_undegraded":"2023-10-16T10:07:29.391556+0000","last_fullsized":"2023-10-16T10:07:29.391556+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:58:12.265556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.372699+0000","last_change":"2023-10-16T10:07:29.378755+0000","last_active":"2023-10-16T10:07:30.372699+0000","last_peered":"2023-10-16T10:07:30.372699+0000","last_clean":"2023-10-16T10:07:30.372699+0000","last_became_active":"2023-10-16T10:07:29.377743+0000","last_became_peered":"2023-10-16T10:07:29.377743+0000","last_unstale":"2023-10-16T10:07:30.372699+0000","last_undegraded":"2023-10-16T10:07:30.372699+0000","last_fullsized":"2023-10-16T10:07:30.372699+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:06.419087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"123'2","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.387918+0000","last_change":"2023-10-16T10:07:28.357710+0000","last_active":"2023-10-16T10:07:29.387918+0000","last_peered":"2023-10-16T10:07:29.387918+0000","last_clean":"2023-10-16T10:07:29.387918+0000","last_became_active":"2023-10-16T10:07:28.356541+0000","last_became_peered":"2023-10-16T10:07:28.356541+0000","last_unstale":"2023-10-16T10:07:29.387918+0000","last_undegraded":"2023-10-16T10:07:29.387918+0000","last_fullsized":"2023-10-16T10:07:29.387918+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:38.126063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.370638+0000","last_change":"2023-10-16T10:07:29.385528+0000","last_active":"2023-10-16T10:07:30.370638+0000","last_peered":"2023-10-16T10:07:30.370638+0000","last_clean":"2023-10-16T10:07:30.370638+0000","last_became_active":"2023-10-16T10:07:29.385240+0000","last_became_peered":"2023-10-16T10:07:29.385240+0000","last_unstale":"2023-10-16T10:07:30.370638+0000","last_undegraded":"2023-10-16T10:07:30.370638+0000","last_fullsized":"2023-10-16T10:07:30.370638+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:16.686978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"123'1","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.389827+0000","last_change":"2023-10-16T10:07:28.362939+0000","last_active":"2023-10-16T10:07:29.389827+0000","last_peered":"2023-10-16T10:07:29.389827+0000","last_clean":"2023-10-16T10:07:29.389827+0000","last_became_active":"2023-10-16T10:07:28.362649+0000","last_became_peered":"2023-10-16T10:07:28.362649+0000","last_unstale":"2023-10-16T10:07:29.389827+0000","last_undegraded":"2023-10-16T10:07:29.389827+0000","last_fullsized":"2023-10-16T10:07:29.389827+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:04:13.458192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.037548+0000","last_change":"2023-10-16T10:07:29.374321+0000","last_active":"2023-10-16T10:07:31.037548+0000","last_peered":"2023-10-16T10:07:31.037548+0000","last_clean":"2023-10-16T10:07:31.037548+0000","last_became_active":"2023-10-16T10:07:29.372935+0000","last_became_peered":"2023-10-16T10:07:29.372935+0000","last_unstale":"2023-10-16T10:07:31.037548+0000","last_undegraded":"2023-10-16T10:07:31.037548+0000","last_fullsized":"2023-10-16T10:07:31.037548+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:07:34.048822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.377582+0000","last_change":"2023-10-16T10:07:28.367804+0000","last_active":"2023-10-16T10:07:29.377582+0000","last_peered":"2023-10-16T10:07:29.377582+0000","last_clean":"2023-10-16T10:07:29.377582+0000","last_became_active":"2023-10-16T10:07:28.367548+0000","last_became_peered":"2023-10-16T10:07:28.367548+0000","last_unstale":"2023-10-16T10:07:29.377582+0000","last_undegraded":"2023-10-16T10:07:29.377582+0000","last_fullsized":"2023-10-16T10:07:29.377582+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:26:47.153376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.367884+0000","last_change":"2023-10-16T10:07:29.365272+0000","last_active":"2023-10-16T10:07:30.367884+0000","last_peered":"2023-10-16T10:07:30.367884+0000","last_clean":"2023-10-16T10:07:30.367884+0000","last_became_active":"2023-10-16T10:07:29.364935+0000","last_became_peered":"2023-10-16T10:07:29.364935+0000","last_unstale":"2023-10-16T10:07:30.367884+0000","last_undegraded":"2023-10-16T10:07:30.367884+0000","last_fullsized":"2023-10-16T10:07:30.367884+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:28.604208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"123'2","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.394194+0000","last_change":"2023-10-16T10:07:28.364807+0000","last_active":"2023-10-16T10:07:29.394194+0000","last_peered":"2023-10-16T10:07:29.394194+0000","last_clean":"2023-10-16T10:07:29.394194+0000","last_became_active":"2023-10-16T10:07:28.364530+0000","last_became_peered":"2023-10-16T10:07:28.364530+0000","last_unstale":"2023-10-16T10:07:29.394194+0000","last_undegraded":"2023-10-16T10:07:29.394194+0000","last_fullsized":"2023-10-16T10:07:29.394194+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:29.987017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.368225+0000","last_change":"2023-10-16T10:07:29.387904+0000","last_active":"2023-10-16T10:07:30.368225+0000","last_peered":"2023-10-16T10:07:30.368225+0000","last_clean":"2023-10-16T10:07:30.368225+0000","last_became_active":"2023-10-16T10:07:29.387621+0000","last_became_peered":"2023-10-16T10:07:29.387621+0000","last_unstale":"2023-10-16T10:07:30.368225+0000","last_undegraded":"2023-10-16T10:07:30.368225+0000","last_fullsized":"2023-10-16T10:07:30.368225+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:06.419087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.367382+0000","last_change":"2023-10-16T10:07:28.359507+0000","last_active":"2023-10-16T10:07:29.367382+0000","last_peered":"2023-10-16T10:07:29.367382+0000","last_clean":"2023-10-16T10:07:29.367382+0000","last_became_active":"2023-10-16T10:07:28.359170+0000","last_became_peered":"2023-10-16T10:07:28.359170+0000","last_unstale":"2023-10-16T10:07:29.367382+0000","last_undegraded":"2023-10-16T10:07:29.367382+0000","last_fullsized":"2023-10-16T10:07:29.367382+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:17.328970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.367409+0000","last_change":"2023-10-16T10:07:29.367315+0000","last_active":"2023-10-16T10:07:30.367409+0000","last_peered":"2023-10-16T10:07:30.367409+0000","last_clean":"2023-10-16T10:07:30.367409+0000","last_became_active":"2023-10-16T10:07:29.364519+0000","last_became_peered":"2023-10-16T10:07:29.364519+0000","last_unstale":"2023-10-16T10:07:30.367409+0000","last_undegraded":"2023-10-16T10:07:30.367409+0000","last_fullsized":"2023-10-16T10:07:30.367409+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:43:09.145602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.360808+0000","last_change":"2023-10-16T10:07:28.357580+0000","last_active":"2023-10-16T10:07:29.360808+0000","last_peered":"2023-10-16T10:07:29.360808+0000","last_clean":"2023-10-16T10:07:29.360808+0000","last_became_active":"2023-10-16T10:07:28.357220+0000","last_became_peered":"2023-10-16T10:07:28.357220+0000","last_unstale":"2023-10-16T10:07:29.360808+0000","last_undegraded":"2023-10-16T10:07:29.360808+0000","last_fullsized":"2023-10-16T10:07:29.360808+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:25.446281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.371604+0000","last_change":"2023-10-16T10:07:29.373609+0000","last_active":"2023-10-16T10:07:30.371604+0000","last_peered":"2023-10-16T10:07:30.371604+0000","last_clean":"2023-10-16T10:07:30.371604+0000","last_became_active":"2023-10-16T10:07:29.373300+0000","last_became_peered":"2023-10-16T10:07:29.373300+0000","last_unstale":"2023-10-16T10:07:30.371604+0000","last_undegraded":"2023-10-16T10:07:30.371604+0000","last_fullsized":"2023-10-16T10:07:30.371604+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:28.604208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.371278+0000","last_change":"2023-10-16T10:07:28.363956+0000","last_active":"2023-10-16T10:07:29.371278+0000","last_peered":"2023-10-16T10:07:29.371278+0000","last_clean":"2023-10-16T10:07:29.371278+0000","last_became_active":"2023-10-16T10:07:28.363620+0000","last_became_peered":"2023-10-16T10:07:28.363620+0000","last_unstale":"2023-10-16T10:07:29.371278+0000","last_undegraded":"2023-10-16T10:07:29.371278+0000","last_fullsized":"2023-10-16T10:07:29.371278+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:49.373492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.040278+0000","last_change":"2023-10-16T10:07:29.381611+0000","last_active":"2023-10-16T10:07:31.040278+0000","last_peered":"2023-10-16T10:07:31.040278+0000","last_clean":"2023-10-16T10:07:31.040278+0000","last_became_active":"2023-10-16T10:07:29.381353+0000","last_became_peered":"2023-10-16T10:07:29.381353+0000","last_unstale":"2023-10-16T10:07:31.040278+0000","last_undegraded":"2023-10-16T10:07:31.040278+0000","last_fullsized":"2023-10-16T10:07:31.040278+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:09.845125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"123'2","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.390458+0000","last_change":"2023-10-16T10:07:28.367328+0000","last_active":"2023-10-16T10:07:29.390458+0000","last_peered":"2023-10-16T10:07:29.390458+0000","last_clean":"2023-10-16T10:07:29.390458+0000","last_became_active":"2023-10-16T10:07:28.366999+0000","last_became_peered":"2023-10-16T10:07:28.366999+0000","last_unstale":"2023-10-16T10:07:29.390458+0000","last_undegraded":"2023-10-16T10:07:29.390458+0000","last_fullsized":"2023-10-16T10:07:29.390458+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:16:21.555477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.038295+0000","last_change":"2023-10-16T10:07:29.368994+0000","last_active":"2023-10-16T10:07:31.038295+0000","last_peered":"2023-10-16T10:07:31.038295+0000","last_clean":"2023-10-16T10:07:31.038295+0000","last_became_active":"2023-10-16T10:07:29.368397+0000","last_became_peered":"2023-10-16T10:07:29.368397+0000","last_unstale":"2023-10-16T10:07:31.038295+0000","last_undegraded":"2023-10-16T10:07:31.038295+0000","last_fullsized":"2023-10-16T10:07:31.038295+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:45.568635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.375647+0000","last_change":"2023-10-16T10:07:28.365844+0000","last_active":"2023-10-16T10:07:29.375647+0000","last_peered":"2023-10-16T10:07:29.375647+0000","last_clean":"2023-10-16T10:07:29.375647+0000","last_became_active":"2023-10-16T10:07:28.365443+0000","last_became_peered":"2023-10-16T10:07:28.365443+0000","last_unstale":"2023-10-16T10:07:29.375647+0000","last_undegraded":"2023-10-16T10:07:29.375647+0000","last_fullsized":"2023-10-16T10:07:29.375647+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:43:08.128286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.038020+0000","last_change":"2023-10-16T10:07:29.379737+0000","last_active":"2023-10-16T10:07:31.038020+0000","last_peered":"2023-10-16T10:07:31.038020+0000","last_clean":"2023-10-16T10:07:31.038020+0000","last_became_active":"2023-10-16T10:07:29.379212+0000","last_became_peered":"2023-10-16T10:07:29.379212+0000","last_unstale":"2023-10-16T10:07:31.038020+0000","last_undegraded":"2023-10-16T10:07:31.038020+0000","last_fullsized":"2023-10-16T10:07:31.038020+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:23:11.152599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.376102+0000","last_change":"2023-10-16T10:07:28.360063+0000","last_active":"2023-10-16T10:07:29.376102+0000","last_peered":"2023-10-16T10:07:29.376102+0000","last_clean":"2023-10-16T10:07:29.376102+0000","last_became_active":"2023-10-16T10:07:28.359469+0000","last_became_peered":"2023-10-16T10:07:28.359469+0000","last_unstale":"2023-10-16T10:07:29.376102+0000","last_undegraded":"2023-10-16T10:07:29.376102+0000","last_fullsized":"2023-10-16T10:07:29.376102+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:46.851587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.372375+0000","last_change":"2023-10-16T10:07:29.382896+0000","last_active":"2023-10-16T10:07:30.372375+0000","last_peered":"2023-10-16T10:07:30.372375+0000","last_clean":"2023-10-16T10:07:30.372375+0000","last_became_active":"2023-10-16T10:07:29.382613+0000","last_became_peered":"2023-10-16T10:07:29.382613+0000","last_unstale":"2023-10-16T10:07:30.372375+0000","last_undegraded":"2023-10-16T10:07:30.372375+0000","last_fullsized":"2023-10-16T10:07:30.372375+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:00:14.321059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.375014+0000","last_change":"2023-10-16T10:07:28.361287+0000","last_active":"2023-10-16T10:07:29.375014+0000","last_peered":"2023-10-16T10:07:29.375014+0000","last_clean":"2023-10-16T10:07:29.375014+0000","last_became_active":"2023-10-16T10:07:28.360220+0000","last_became_peered":"2023-10-16T10:07:28.360220+0000","last_unstale":"2023-10-16T10:07:29.375014+0000","last_undegraded":"2023-10-16T10:07:29.375014+0000","last_fullsized":"2023-10-16T10:07:29.375014+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:43:08.128286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.368062+0000","last_change":"2023-10-16T10:07:29.359015+0000","last_active":"2023-10-16T10:07:30.368062+0000","last_peered":"2023-10-16T10:07:30.368062+0000","last_clean":"2023-10-16T10:07:30.368062+0000","last_became_active":"2023-10-16T10:07:29.358652+0000","last_became_peered":"2023-10-16T10:07:29.358652+0000","last_unstale":"2023-10-16T10:07:30.368062+0000","last_undegraded":"2023-10-16T10:07:30.368062+0000","last_fullsized":"2023-10-16T10:07:30.368062+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:00:14.321059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.375705+0000","last_change":"2023-10-16T10:07:28.364313+0000","last_active":"2023-10-16T10:07:29.375705+0000","last_peered":"2023-10-16T10:07:29.375705+0000","last_clean":"2023-10-16T10:07:29.375705+0000","last_became_active":"2023-10-16T10:07:28.363948+0000","last_became_peered":"2023-10-16T10:07:28.363948+0000","last_unstale":"2023-10-16T10:07:29.375705+0000","last_undegraded":"2023-10-16T10:07:29.375705+0000","last_fullsized":"2023-10-16T10:07:29.375705+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:04:13.458192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.035832+0000","last_change":"2023-10-16T10:07:29.379290+0000","last_active":"2023-10-16T10:07:31.035832+0000","last_peered":"2023-10-16T10:07:31.035832+0000","last_clean":"2023-10-16T10:07:31.035832+0000","last_became_active":"2023-10-16T10:07:29.378793+0000","last_became_peered":"2023-10-16T10:07:29.378793+0000","last_unstale":"2023-10-16T10:07:31.035832+0000","last_undegraded":"2023-10-16T10:07:31.035832+0000","last_fullsized":"2023-10-16T10:07:31.035832+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:18:14.725928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.365939+0000","last_change":"2023-10-16T10:07:28.357283+0000","last_active":"2023-10-16T10:07:29.365939+0000","last_peered":"2023-10-16T10:07:29.365939+0000","last_clean":"2023-10-16T10:07:29.365939+0000","last_became_active":"2023-10-16T10:07:28.356946+0000","last_became_peered":"2023-10-16T10:07:28.356946+0000","last_unstale":"2023-10-16T10:07:29.365939+0000","last_undegraded":"2023-10-16T10:07:29.365939+0000","last_fullsized":"2023-10-16T10:07:29.365939+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:29.987017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.367607+0000","last_change":"2023-10-16T10:07:29.358961+0000","last_active":"2023-10-16T10:07:30.367607+0000","last_peered":"2023-10-16T10:07:30.367607+0000","last_clean":"2023-10-16T10:07:30.367607+0000","last_became_active":"2023-10-16T10:07:29.358547+0000","last_became_peered":"2023-10-16T10:07:29.358547+0000","last_unstale":"2023-10-16T10:07:30.367607+0000","last_undegraded":"2023-10-16T10:07:30.367607+0000","last_fullsized":"2023-10-16T10:07:30.367607+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:04:14.475508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.352520+0000","last_change":"2023-10-16T10:07:28.355774+0000","last_active":"2023-10-16T10:07:29.352520+0000","last_peered":"2023-10-16T10:07:29.352520+0000","last_clean":"2023-10-16T10:07:29.352520+0000","last_became_active":"2023-10-16T10:07:28.355485+0000","last_became_peered":"2023-10-16T10:07:28.355485+0000","last_unstale":"2023-10-16T10:07:29.352520+0000","last_undegraded":"2023-10-16T10:07:29.352520+0000","last_fullsized":"2023-10-16T10:07:29.352520+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:25:38.877788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.371601+0000","last_change":"2023-10-16T10:07:29.367503+0000","last_active":"2023-10-16T10:07:30.371601+0000","last_peered":"2023-10-16T10:07:30.371601+0000","last_clean":"2023-10-16T10:07:30.371601+0000","last_became_active":"2023-10-16T10:07:29.367190+0000","last_became_peered":"2023-10-16T10:07:29.367190+0000","last_unstale":"2023-10-16T10:07:30.371601+0000","last_undegraded":"2023-10-16T10:07:30.371601+0000","last_fullsized":"2023-10-16T10:07:30.371601+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:47.868903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.343361+0000","last_change":"2023-10-16T10:07:28.367204+0000","last_active":"2023-10-16T10:07:29.343361+0000","last_peered":"2023-10-16T10:07:29.343361+0000","last_clean":"2023-10-16T10:07:29.343361+0000","last_became_active":"2023-10-16T10:07:28.366430+0000","last_became_peered":"2023-10-16T10:07:28.366430+0000","last_unstale":"2023-10-16T10:07:29.343361+0000","last_undegraded":"2023-10-16T10:07:29.343361+0000","last_fullsized":"2023-10-16T10:07:29.343361+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:01:20.636726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.368306+0000","last_change":"2023-10-16T10:07:29.365302+0000","last_active":"2023-10-16T10:07:30.368306+0000","last_peered":"2023-10-16T10:07:30.368306+0000","last_clean":"2023-10-16T10:07:30.368306+0000","last_became_active":"2023-10-16T10:07:29.364119+0000","last_became_peered":"2023-10-16T10:07:29.364119+0000","last_unstale":"2023-10-16T10:07:30.368306+0000","last_undegraded":"2023-10-16T10:07:30.368306+0000","last_fullsized":"2023-10-16T10:07:30.368306+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:33:28.020956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"123'1","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.389574+0000","last_change":"2023-10-16T10:07:28.370898+0000","last_active":"2023-10-16T10:07:29.389574+0000","last_peered":"2023-10-16T10:07:29.389574+0000","last_clean":"2023-10-16T10:07:29.389574+0000","last_became_active":"2023-10-16T10:07:28.370566+0000","last_became_peered":"2023-10-16T10:07:28.370566+0000","last_unstale":"2023-10-16T10:07:29.389574+0000","last_undegraded":"2023-10-16T10:07:29.389574+0000","last_fullsized":"2023-10-16T10:07:29.389574+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:27.011207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"113'2624","reported_seq":3029,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.361123+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T10:07:29.361123+0000","last_peered":"2023-10-16T10:07:29.361123+0000","last_clean":"2023-10-16T10:07:29.361123+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T10:07:29.361123+0000","last_undegraded":"2023-10-16T10:07:29.361123+0000","last_fullsized":"2023-10-16T10:07:29.361123+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2624,"log_dups_size":0,"ondisk_log_size":2624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":928,"num_read_kb":1790,"num_write":321,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":928,"num_read_kb":1790,"num_write":321,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2624,"ondisk_log_size":2624,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575069,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2134016,"data_stored":1321946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1699999999999999,"15min":1.171},"min":{"1min":0.81200000000000006,"5min":0.65000000000000002,"15min":0.54400000000000004},"max":{"1min":1.6160000000000001,"5min":1.7310000000000001,"15min":1.925},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.27,"5min":1.159,"15min":1.1899999999999999},"min":{"1min":0.90400000000000003,"5min":0.58099999999999996,"15min":0.33600000000000002},"max":{"1min":1.702,"5min":1.7370000000000001,"15min":2.016},"last":1.1259999999999999}]},{"osd":1,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1759999999999999,"15min":1.2210000000000001},"min":{"1min":0.72899999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.48,"5min":1.788,"15min":1.923},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.161,"15min":1.2090000000000001},"min":{"1min":0.746,"5min":0.67000000000000004,"15min":0.60999999999999999},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.77500000000000002}]},{"osd":2,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.21,"15min":1.24},"min":{"1min":0.63900000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8109999999999999,"5min":1.837,"15min":1.8740000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2629999999999999,"15min":1.288},"min":{"1min":0.89400000000000002,"5min":0.65900000000000003,"15min":0.64500000000000002},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8620000000000001},"last":0.89400000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3069999999999999,"15min":1.339},"min":{"1min":1.054,"5min":0.747,"15min":0.67700000000000005},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3280000000000001,"15min":1.3720000000000001},"min":{"1min":0.97699999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6799999999999999,"5min":1.8540000000000001,"15min":2.004},"last":1.0389999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3859999999999999,"15min":1.367},"min":{"1min":0.86499999999999999,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6970000000000001,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.3140000000000001,"15min":1.3129999999999999},"min":{"1min":1.0009999999999999,"5min":0.63200000000000001,"15min":0.502},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.361}]},{"osd":5,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.5069999999999999,"15min":1.52},"min":{"1min":1.1879999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.9330000000000001,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.411},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4199999999999999,"15min":1.4279999999999999},"min":{"1min":1.1910000000000001,"5min":0.72199999999999998,"15min":0.45400000000000001},"max":{"1min":1.6699999999999999,"5min":1.9630000000000001,"15min":2.2170000000000001},"last":1.452}]},{"osd":6,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.419,"15min":1.4039999999999999},"min":{"1min":0.97599999999999998,"5min":0.82899999999999996,"15min":0.54100000000000004},"max":{"1min":2.0899999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.631,"15min":1.617},"min":{"1min":1.212,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.9279999999999999,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.548}]}]},{"osd":6,"up_from":13,"seq":55834575069,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36844,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334996,"statfs":{"total":96636764160,"available":96599035904,"internally_reserved":0,"allocated":1540096,"data_stored":732118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.1439999999999999,"15min":1.1499999999999999},"min":{"1min":0.68000000000000005,"5min":0.61599999999999999,"15min":0.61099999999999999},"max":{"1min":1.736,"5min":1.736,"15min":1.7689999999999999},"last":1.298},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2430000000000001,"15min":1.24},"min":{"1min":0.71199999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.6899999999999999,"5min":1.7010000000000001,"15min":1.847},"last":1.5329999999999999}]},{"osd":1,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.153,"15min":1.173},"min":{"1min":0.84599999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6970000000000001},"last":1.7789999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.2949999999999999,"15min":1.298},"min":{"1min":0.72599999999999998,"5min":0.69099999999999995,"15min":0.66300000000000003},"max":{"1min":1.7190000000000001,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.6539999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2090000000000001,"15min":1.216},"min":{"1min":0.67800000000000005,"5min":0.66600000000000004,"15min":0.45100000000000001},"max":{"1min":1.748,"5min":1.851,"15min":2.036},"last":1.026},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.29,"15min":1.304},"min":{"1min":0.96899999999999997,"5min":0.79900000000000004,"15min":0.58999999999999997},"max":{"1min":1.794,"5min":1.925,"15min":1.925},"last":1.5800000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.177,"15min":1.2090000000000001},"min":{"1min":0.78800000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5069999999999999,"5min":1.742,"15min":2.8239999999999998},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.1899999999999999,"15min":1.232},"min":{"1min":0.78900000000000003,"5min":0.67400000000000004,"15min":0.63900000000000001},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.865},"last":1.131}]},{"osd":4,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2490000000000001,"15min":1.2629999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.46000000000000002},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":2.1629999999999998},"last":1.9690000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.4350000000000001,"15min":1.429},"min":{"1min":0.81100000000000005,"5min":0.77700000000000002,"15min":0.56599999999999995},"max":{"1min":2,"5min":2,"15min":3.6880000000000002},"last":1.8520000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3009999999999999,"15min":1.3149999999999999},"min":{"1min":0.88,"5min":0.80400000000000005,"15min":0.38900000000000001},"max":{"1min":1.6559999999999999,"5min":2.0790000000000002,"15min":2.5649999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.5029999999999999,"15min":1.5089999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.495},"max":{"1min":1.796,"5min":1.996,"15min":2.3849999999999998},"last":1.7190000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":1.028,"5min":0.79800000000000004,"15min":0.46600000000000003},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":3.5739999999999998},"last":2.0070000000000001},{"interface":"front","average":{"1min":1.53,"5min":1.5049999999999999,"15min":1.4910000000000001},"min":{"1min":0.96299999999999997,"5min":0.92600000000000005,"15min":0.65300000000000002},"max":{"1min":1.917,"5min":2.2000000000000002,"15min":2.4249999999999998},"last":1.4830000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575069,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36840,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335000,"statfs":{"total":96636764160,"available":96599040000,"internally_reserved":0,"allocated":1536000,"data_stored":731536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.179,"15min":1.204},"min":{"1min":0.65600000000000003,"5min":0.57799999999999996,"15min":0.496},"max":{"1min":1.6279999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.151},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.236,"15min":1.2849999999999999},"min":{"1min":0.72599999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.4219999999999999,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.6830000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.119,"15min":1.1499999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.647,"5min":1.7310000000000001,"15min":1.7729999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.139,"15min":1.1439999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.478,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.345}]},{"osd":2,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1339999999999999,"15min":1.155},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5700000000000001,"5min":1.738,"15min":1.77},"last":1.7709999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.21,"15min":1.232},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.645,"5min":1.6739999999999999,"15min":1.8109999999999999},"last":1.4159999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2609999999999999,"15min":1.27},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.76000000000000001},"max":{"1min":1.8260000000000001,"5min":1.9530000000000001,"15min":1.97},"last":1.044},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3149999999999999,"15min":1.347},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6100000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.232}]},{"osd":4,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.278,"15min":1.2689999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.73499999999999999},"max":{"1min":1.6850000000000001,"5min":1.984,"15min":1.984},"last":1.476},{"interface":"front","average":{"1min":1.444,"5min":1.51,"15min":1.502},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.7470000000000001,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.3660000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3360000000000001,"15min":1.3540000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.722,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.381,"15min":1.3799999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.613,"5min":1.986,"15min":1.986},"last":1.2789999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.464,"15min":1.47},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.9219999999999999,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":1.571},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.5489999999999999,"15min":1.5620000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.788,"5min":2.234,"15min":2.234},"last":1.5089999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575070,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46588,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325252,"statfs":{"total":96636764160,"available":96589058048,"internally_reserved":0,"allocated":1552384,"data_stored":733202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.079,"15min":1.141},"min":{"1min":0.64300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.5349999999999999,"5min":1.7609999999999999,"15min":1.827},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3220000000000001,"15min":1.2689999999999999},"min":{"1min":0.89600000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.7070000000000001,"5min":11.974,"15min":11.974},"last":1.4450000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3260000000000001,"15min":1.2989999999999999},"min":{"1min":0.63900000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.7969999999999999,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.2010000000000001,"15min":1.226},"min":{"1min":0.70699999999999996,"5min":0.63600000000000001,"15min":0.624},"max":{"1min":1.5800000000000001,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.24}]},{"osd":2,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.246,"15min":1.248},"min":{"1min":0.78200000000000003,"5min":0.61799999999999999,"15min":0.59399999999999997},"max":{"1min":1.712,"5min":1.897,"15min":1.897},"last":1.155},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3540000000000001,"15min":1.383},"min":{"1min":0.81499999999999995,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6160000000000001,"5min":3.673,"15min":5.0549999999999997},"last":1.1850000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3200000000000001,"15min":1.3},"min":{"1min":0.94399999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.625,"5min":6.8079999999999998,"15min":6.8079999999999998},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.327,"5min":1.3620000000000001,"15min":1.3779999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.67700000000000005},"max":{"1min":1.768,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.70599999999999996}]},{"osd":5,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.496,"15min":1.4350000000000001},"min":{"1min":0.77400000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9990000000000001,"5min":8.702,"15min":8.702},"last":1.343},{"interface":"front","average":{"1min":1.337,"5min":1.383,"15min":1.3720000000000001},"min":{"1min":0.93799999999999994,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.331,"5min":2.331,"15min":3.0430000000000001},"last":1.1120000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4850000000000001,"15min":1.417},"min":{"1min":0.95399999999999996,"5min":0.84699999999999998,"15min":0.79000000000000004},"max":{"1min":1.6990000000000001,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":1.048},{"interface":"front","average":{"1min":1.496,"5min":1.585,"15min":1.556},"min":{"1min":0.88900000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.891,"5min":4.625,"15min":6.4089999999999998},"last":1.52}]},{"osd":7,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.5189999999999999,"15min":1.492},"min":{"1min":1.167,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.927,"5min":2.4060000000000001,"15min":2.9399999999999999},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.579,"15min":1.5700000000000001},"min":{"1min":1.2589999999999999,"5min":1.071,"15min":0.96799999999999997},"max":{"1min":1.9139999999999999,"5min":2.2690000000000001,"15min":2.6120000000000001},"last":1.3759999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575069,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36852,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334988,"statfs":{"total":96636764160,"available":96599027712,"internally_reserved":0,"allocated":1548288,"data_stored":733100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:06:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.81299999999999994,"15min":0.85399999999999998},"min":{"1min":0.318,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1060000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.89600000000000002,"15min":1.002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.708},"last":0.80200000000000005}]},{"osd":1,"last update":"Mon Oct 16 10:06:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.84399999999999997,"15min":0.93600000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.5800000000000001,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.94899999999999995,"15min":1.0489999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4950000000000001,"5min":3.363,"15min":3.363},"last":0.38300000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:06:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.94899999999999995,"15min":1.0569999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.3400000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0600000000000001,"15min":1.1679999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.734,"5min":4.1230000000000002,"15min":4.1230000000000002},"last":0.51900000000000002}]},{"osd":4,"last update":"Mon Oct 16 10:06:33 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2090000000000001,"15min":1.272},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.637,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2729999999999999,"15min":1.3240000000000001},"min":{"1min":0.74199999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.599,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.56499999999999995}]},{"osd":5,"last update":"Mon Oct 16 10:06:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.262,"15min":1.3160000000000001},"min":{"1min":0.75600000000000001,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.667,"5min":4.1829999999999998,"15min":4.1829999999999998},"last":0.497},{"interface":"front","average":{"1min":1.242,"5min":1.258,"15min":1.335},"min":{"1min":0.73799999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.9199999999999999,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.89900000000000002}]},{"osd":6,"last update":"Mon Oct 16 10:06:33 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.3520000000000001,"15min":1.423},"min":{"1min":0.81100000000000005,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.7789999999999999,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.399,"5min":1.347,"15min":1.4330000000000001},"min":{"1min":0.81599999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.0339999999999998,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.93899999999999995}]},{"osd":7,"last update":"Mon Oct 16 10:06:33 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.3779999999999999,"15min":1.4399999999999999},"min":{"1min":1,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":2.1000000000000001,"5min":3.649,"15min":3.649},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.506,"5min":1.4750000000000001,"15min":1.5569999999999999},"min":{"1min":0.84599999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0430000000000001,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":0.63400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575068,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41008,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330832,"statfs":{"total":96636764160,"available":96594771968,"internally_reserved":0,"allocated":1536000,"data_stored":731500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.89300000000000002,"15min":0.877},"min":{"1min":0.35299999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.2749999999999999,"5min":5.1829999999999998,"15min":5.1829999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.90600000000000003,"15min":1.0229999999999999},"min":{"1min":0.35999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.6479999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.94999999999999996}]},{"osd":1,"last update":"Mon Oct 16 10:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0169999999999999,"15min":1.042},"min":{"1min":0.51900000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.5900000000000001,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":1.5029999999999999},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.96499999999999997,"15min":1.1020000000000001},"min":{"1min":0.48899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3979999999999999,"5min":1.792,"15min":1.792},"last":1.0880000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.92700000000000005,"15min":1.004},"min":{"1min":0.48499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.3160000000000001,"5min":1.776,"15min":1.776},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.147,"15min":1.1950000000000001},"min":{"1min":0.47499999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.841,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":1.3380000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:06:37 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.284,"15min":1.341},"min":{"1min":0.79900000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.71,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.548},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2709999999999999,"15min":1.304},"min":{"1min":0.89200000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.6459999999999999,"5min":3.262,"15min":3.262},"last":1.5720000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:06:37 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3560000000000001,"15min":1.373},"min":{"1min":0.79700000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.0259999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.3069999999999999,"15min":1.397},"min":{"1min":0.76200000000000001,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.573,"5min":1.885,"15min":1.956},"last":1.2649999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:06:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2889999999999999,"15min":1.3420000000000001},"min":{"1min":0.84299999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.716,"5min":3.169,"15min":3.169},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.417,"5min":1.415,"15min":1.4399999999999999},"min":{"1min":0.90100000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.2549999999999999,"5min":3.8340000000000001,"15min":3.8340000000000001},"last":1.7749999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:06:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.391,"15min":1.466},"min":{"1min":0.86499999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6870000000000001,"5min":2.0739999999999998,"15min":2.262},"last":1.472},{"interface":"front","average":{"1min":1.359,"5min":1.3600000000000001,"15min":1.429},"min":{"1min":0.84199999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.776,"5min":3.3780000000000001,"15min":3.3780000000000001},"last":1.7410000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575068,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54892,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316948,"statfs":{"total":96636764160,"available":96580554752,"internally_reserved":0,"allocated":1531904,"data_stored":731466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.79500000000000004,"15min":0.85599999999999998},"min":{"1min":0.372,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.196,"5min":1.4690000000000001,"15min":1.631},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.94499999999999995,"15min":0.98899999999999999},"min":{"1min":0.48699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.262,"5min":1.798,"15min":1.798},"last":0.51700000000000002}]},{"osd":2,"last update":"Mon Oct 16 10:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.93200000000000005,"15min":0.97699999999999998},"min":{"1min":0.44800000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.476,"5min":1.5069999999999999,"15min":1.673},"last":0.58499999999999996},{"interface":"front","average":{"1min":1.159,"5min":1.119,"15min":1.214},"min":{"1min":0.505,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.591,"5min":1.609,"15min":1.7669999999999999},"last":0.59699999999999998}]},{"osd":3,"last update":"Mon Oct 16 10:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.02,"15min":1.113},"min":{"1min":0.42999999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5429999999999999,"5min":1.798,"15min":1.798},"last":0.53800000000000003},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0269999999999999,"15min":1.1020000000000001},"min":{"1min":0.52800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5089999999999999,"5min":1.6879999999999999,"15min":1.8879999999999999},"last":0.70699999999999996}]},{"osd":4,"last update":"Mon Oct 16 10:07:22 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.159,"15min":1.2190000000000001},"min":{"1min":0.61299999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4610000000000001,"5min":1.7450000000000001,"15min":2.0259999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.2789999999999999,"15min":1.351},"min":{"1min":0.79200000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.823,"5min":1.8680000000000001,"15min":2.1320000000000001},"last":0.75700000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:07:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.2569999999999999,"15min":1.3240000000000001},"min":{"1min":0.82399999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6220000000000001,"5min":1.7310000000000001,"15min":1.9359999999999999},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.246,"15min":1.3149999999999999},"min":{"1min":0.73099999999999998,"5min":0.67200000000000004,"15min":0.60499999999999998},"max":{"1min":1.643,"5min":1.7869999999999999,"15min":2.0470000000000002},"last":0.84199999999999997}]},{"osd":6,"last update":"Mon Oct 16 10:07:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.325,"15min":1.387},"min":{"1min":0.76900000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.7230000000000001,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.302,"5min":1.321,"15min":1.3799999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6180000000000001,"5min":1.8779999999999999,"15min":2.1680000000000001},"last":0.98299999999999998}]},{"osd":7,"last update":"Mon Oct 16 10:07:22 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.355,"15min":1.448},"min":{"1min":0.81299999999999994,"5min":0.62,"15min":0.62},"max":{"1min":1.7330000000000001,"5min":2.0739999999999998,"15min":2.1699999999999999},"last":1.046},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.427,"15min":1.4850000000000001},"min":{"1min":1.0369999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.798,"5min":2.1829999999999998,"15min":2.1960000000000002},"last":1.0700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575069,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47148,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324692,"statfs":{"total":96636764160,"available":96588484608,"internally_reserved":0,"allocated":2129920,"data_stored":1322416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.83399999999999996,"15min":0.85499999999999998},"min":{"1min":0.51700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1819999999999999,"5min":1.7589999999999999,"15min":2.766},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.093,"5min":0.95999999999999996,"15min":1.03},"min":{"1min":0.45600000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.51,"5min":1.581,"15min":3.96},"last":1.01}]},{"osd":2,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":0.999,"15min":1.0409999999999999},"min":{"1min":0.60699999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.5089999999999999,"5min":2.0449999999999999,"15min":2.9260000000000002},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.024,"15min":1.075},"min":{"1min":0.49299999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":4.4290000000000003},"last":1.21}]},{"osd":3,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.94599999999999995,"15min":1.022},"min":{"1min":0.44400000000000001,"5min":0.41999999999999998,"15min":0.38},"max":{"1min":1.4370000000000001,"5min":1.5920000000000001,"15min":4.7380000000000004},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.226,"5min":1.151,"15min":1.202},"min":{"1min":0.80400000000000005,"5min":0.44,"15min":0.44},"max":{"1min":1.6579999999999999,"5min":2.1859999999999999,"15min":5.1500000000000004},"last":1.1160000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.296,"15min":1.331},"min":{"1min":0.78300000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.696,"5min":2.1749999999999998,"15min":4.1970000000000001},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.224,"15min":1.2889999999999999},"min":{"1min":0.76300000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6080000000000001,"5min":1.766,"15min":4.9630000000000001},"last":1.3999999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2410000000000001,"15min":1.306},"min":{"1min":0.872,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.657,"5min":1.7070000000000001,"15min":4.8600000000000003},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3500000000000001,"15min":1.3979999999999999},"min":{"1min":0.93899999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.639,"5min":2.1269999999999998,"15min":4.6779999999999999},"last":1.2470000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.4330000000000001,"15min":1.448},"min":{"1min":1.3149999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.8100000000000001,"5min":2.1030000000000002,"15min":3.8250000000000002},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.377,"15min":1.427},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7669999999999999,"5min":2.117,"15min":4.0979999999999999},"last":1.333}]},{"osd":7,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.452,"15min":1.4850000000000001},"min":{"1min":0.63300000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.756,"5min":2.0750000000000002,"15min":4.3369999999999997},"last":1.484},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.353,"15min":1.429},"min":{"1min":0.79700000000000004,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":5.0570000000000004},"last":1.4590000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:07:33.759 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:07:34.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.008+0000 7f0b95f47640 1 Processor -- start 2023-10-16T10:07:34.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.008+0000 7f0b95f47640 1 -- start start 2023-10-16T10:07:34.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.010+0000 7f0b95f47640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b900b0b20 0x7f0b90164b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:34.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.010+0000 7f0b95f47640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b9006f7f0 con 0x7f0b90159b20 2023-10-16T10:07:34.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.010+0000 7f0b95f47640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b9006c5e0 con 0x7f0b90159f00 2023-10-16T10:07:34.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.010+0000 7f0b95f47640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9006c8c0 con 0x7f0b900b0b20 2023-10-16T10:07:34.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.010+0000 7f0b8f7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0b90159b20 0x7f0b9006bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37840/0 (socket says 172.21.15.84:37840) 2023-10-16T10:07:34.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.010+0000 7f0b8f7fe640 1 -- 172.21.15.84:0/2030176490 learned_addr learned my addr 172.21.15.84:0/2030176490 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:34.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.011+0000 7f0b8ffff640 1 --2- 172.21.15.84:0/2030176490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b900b0b20 0x7f0b90164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:34.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.011+0000 7f0b8effd640 1 -- 172.21.15.84:0/2030176490 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2738783953 0 0) 0x7f0b9006f7f0 con 0x7f0b90159b20 2023-10-16T10:07:34.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.011+0000 7f0b8effd640 1 -- 172.21.15.84:0/2030176490 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b7c003520 con 0x7f0b90159b20 2023-10-16T10:07:34.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.011+0000 7f0b8effd640 1 -- 172.21.15.84:0/2030176490 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1931568833 0 0) 0x7f0b7c003520 con 0x7f0b90159b20 2023-10-16T10:07:34.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.011+0000 7f0b8effd640 1 -- 172.21.15.84:0/2030176490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b900b0b20 msgr2=0x7f0b90164b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:34.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.011+0000 7f0b8effd640 1 --2- 172.21.15.84:0/2030176490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b900b0b20 0x7f0b90164b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.011+0000 7f0b8effd640 1 -- 172.21.15.84:0/2030176490 >> v1:172.21.15.157:6789/0 conn(0x7f0b90159f00 legacy=0x7f0b900b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:34.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.011+0000 7f0b8effd640 1 -- 172.21.15.84:0/2030176490 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9006cba0 con 0x7f0b90159b20 2023-10-16T10:07:34.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.011+0000 7f0b8effd640 1 -- 172.21.15.84:0/2030176490 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f0b80001670 con 0x7f0b90159b20 2023-10-16T10:07:34.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.011+0000 7f0b8effd640 1 -- 172.21.15.84:0/2030176490 >> v1:172.21.15.84:6789/0 conn(0x7f0b90159b20 legacy=0x7f0b9006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:34.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.011+0000 7f0b8effd640 1 --2- 172.21.15.84:0/2030176490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b7c0039a0 0x7f0b7c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:34.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.011+0000 7f0b8effd640 1 --2- 172.21.15.84:0/2030176490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b900b0b20 0x7f0b7c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:34.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.011+0000 7f0b8effd640 1 --2- 172.21.15.84:0/2030176490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b7c004d00 0x7f0b7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:34.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.011+0000 7f0b8effd640 1 -- 172.21.15.84:0/2030176490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9006c8c0 con 0x7f0b7c0039a0 2023-10-16T10:07:34.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.011+0000 7f0b8effd640 1 -- 172.21.15.84:0/2030176490 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c00bb10 con 0x7f0b7c004d00 2023-10-16T10:07:34.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.011+0000 7f0b8effd640 1 -- 172.21.15.84:0/2030176490 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c00dee0 con 0x7f0b900b0b20 2023-10-16T10:07:34.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.011+0000 7f0b8f7fe640 1 --2- 172.21.15.84:0/2030176490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b7c0039a0 0x7f0b7c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:34.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.012+0000 7f0b8ffff640 1 --2- 172.21.15.84:0/2030176490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b7c004d00 0x7f0b7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:34.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.012+0000 7f0b8f7fe640 1 -- 172.21.15.84:0/2030176490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b900b0b20 msgr2=0x7f0b7c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:34.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.012+0000 7f0b8f7fe640 1 --2- 172.21.15.84:0/2030176490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b900b0b20 0x7f0b7c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.012+0000 7f0b8f7fe640 1 -- 172.21.15.84:0/2030176490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b7c004d00 msgr2=0x7f0b7c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:34.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.012+0000 7f0b8f7fe640 1 --2- 172.21.15.84:0/2030176490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b7c004d00 0x7f0b7c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.012+0000 7f0b8f7fe640 1 -- 172.21.15.84:0/2030176490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c003520 con 0x7f0b7c0039a0 2023-10-16T10:07:34.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.013+0000 7f0b8f7fe640 1 --2- 172.21.15.84:0/2030176490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b7c0039a0 0x7f0b7c003dc0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000c240 tx=0x7f0b7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720e5a09d03ae221 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:34.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.013+0000 7f0b8effd640 1 -- 172.21.15.84:0/2030176490 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b80005ad0 con 0x7f0b7c0039a0 2023-10-16T10:07:34.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.013+0000 7f0b8effd640 1 -- 172.21.15.84:0/2030176490 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b80005c30 con 0x7f0b7c0039a0 2023-10-16T10:07:34.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.013+0000 7f0b8effd640 1 -- 172.21.15.84:0/2030176490 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8002c8b0 con 0x7f0b7c0039a0 2023-10-16T10:07:34.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.014+0000 7f0b95f47640 1 -- 172.21.15.84:0/2030176490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b7c0039a0 msgr2=0x7f0b7c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:34.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.014+0000 7f0b95f47640 1 --2- 172.21.15.84:0/2030176490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b7c0039a0 0x7f0b7c003dc0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000c240 tx=0x7f0b7c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.014+0000 7f0b8ffff640 1 -- 172.21.15.84:0/2030176490 reap_dead start 2023-10-16T10:07:34.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.014+0000 7f0b95f47640 1 -- 172.21.15.84:0/2030176490 shutdown_connections 2023-10-16T10:07:34.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.014+0000 7f0b95f47640 1 -- 172.21.15.84:0/2030176490 >> 172.21.15.84:0/2030176490 conn(0x7f0b900cb410 msgr2=0x7f0b900cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:34.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.014+0000 7f0b95f47640 1 -- 172.21.15.84:0/2030176490 shutdown_connections 2023-10-16T10:07:34.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.014+0000 7f0b95f47640 1 -- 172.21.15.84:0/2030176490 wait complete. 2023-10-16T10:07:34.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.016+0000 7f0b95f47640 1 Processor -- start 2023-10-16T10:07:34.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.016+0000 7f0b95f47640 1 -- start start 2023-10-16T10:07:34.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.017+0000 7f0b95f47640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b7c0039a0 0x7f0b90155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:34.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.017+0000 7f0b95f47640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b9015ab00 0x7f0b900b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:34.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.017+0000 7f0b95f47640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b9015af80 0x7f0b900b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:34.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.017+0000 7f0b95f47640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b900c85c0 con 0x7f0b9015af80 2023-10-16T10:07:34.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.017+0000 7f0b95f47640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b900c79f0 con 0x7f0b7c0039a0 2023-10-16T10:07:34.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.017+0000 7f0b95f47640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b900c7c90 con 0x7f0b9015ab00 2023-10-16T10:07:34.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.017+0000 7f0b8ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b9015af80 0x7f0b900b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:34.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.017+0000 7f0b8ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b9015af80 0x7f0b900b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52176/0 (socket says 172.21.15.84:52176) 2023-10-16T10:07:34.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.017+0000 7f0b8ffff640 1 -- 172.21.15.84:0/1132954638 learned_addr learned my addr 172.21.15.84:0/1132954638 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:34.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.017+0000 7f0b8ffff640 1 -- 172.21.15.84:0/1132954638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b9015ab00 msgr2=0x7f0b900b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:34.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.017+0000 7f0b8f7fe640 1 --2- 172.21.15.84:0/1132954638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b7c0039a0 0x7f0b90155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:34.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.017+0000 7f0b87fff640 1 --2- 172.21.15.84:0/1132954638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b9015ab00 0x7f0b900b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:34.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.017+0000 7f0b8ffff640 1 --2- 172.21.15.84:0/1132954638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b9015ab00 0x7f0b900b22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.017+0000 7f0b8ffff640 1 -- 172.21.15.84:0/1132954638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b7c0039a0 msgr2=0x7f0b90155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:34.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.017+0000 7f0b8ffff640 1 --2- 172.21.15.84:0/1132954638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b7c0039a0 0x7f0b90155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.017+0000 7f0b8ffff640 1 -- 172.21.15.84:0/1132954638 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b900c7f70 con 0x7f0b9015af80 2023-10-16T10:07:34.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.018+0000 7f0b8ffff640 1 --2- 172.21.15.84:0/1132954638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b9015af80 0x7f0b900b2810 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f0b900c9ad0 tx=0x7f0b88009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:34.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.018+0000 7f0b8cff9640 1 -- 172.21.15.84:0/1132954638 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b88007db0 con 0x7f0b9015af80 2023-10-16T10:07:34.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.018+0000 7f0b8cff9640 1 -- 172.21.15.84:0/1132954638 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b88014400 con 0x7f0b9015af80 2023-10-16T10:07:34.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.018+0000 7f0b8cff9640 1 -- 172.21.15.84:0/1132954638 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b880147b0 con 0x7f0b9015af80 2023-10-16T10:07:34.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.018+0000 7f0b95f47640 1 -- 172.21.15.84:0/1132954638 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b900c8250 con 0x7f0b9015af80 2023-10-16T10:07:34.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.018+0000 7f0b95f47640 1 -- 172.21.15.84:0/1132954638 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b90156160 con 0x7f0b9015af80 2023-10-16T10:07:34.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.019+0000 7f0b95f47640 1 -- 172.21.15.84:0/1132954638 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b900c8730 con 0x7f0b9015af80 2023-10-16T10:07:34.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.020+0000 7f0b8cff9640 1 -- 172.21.15.84:0/1132954638 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0b8801d3b0 con 0x7f0b9015af80 2023-10-16T10:07:34.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.021+0000 7f0b8cff9640 1 --2- 172.21.15.84:0/1132954638 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0b7006af90 0x7f0b7006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:34.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.021+0000 7f0b8cff9640 1 -- 172.21.15.84:0/1132954638 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b88089490 con 0x7f0b9015af80 2023-10-16T10:07:34.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.021+0000 7f0b8f7fe640 1 --2- 172.21.15.84:0/1132954638 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0b7006af90 0x7f0b7006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:34.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.022+0000 7f0b8f7fe640 1 --2- 172.21.15.84:0/1132954638 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0b7006af90 0x7f0b7006d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000c240 tx=0x7f0b80009030 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:34.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.026+0000 7f0b8cff9640 1 -- 172.21.15.84:0/1132954638 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0b88054420 con 0x7f0b9015af80 2023-10-16T10:07:34.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.201+0000 7f0b95f47640 1 -- 172.21.15.84:0/1132954638 --> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b9006c2f0 con 0x7f0b7006af90 2023-10-16T10:07:34.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.223+0000 7f0b8cff9640 1 -- 172.21.15.84:0/1132954638 <== mgr.7794 v2:172.21.15.84:6832/4292025878 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287707 (secure 0 0 0) 0x7f0b80005970 con 0x7f0b7006af90 2023-10-16T10:07:34.229 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:34.230 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T10:07:34.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.228+0000 7f0b95f47640 1 -- 172.21.15.84:0/1132954638 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0b7006af90 msgr2=0x7f0b7006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:34.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.229+0000 7f0b95f47640 1 --2- 172.21.15.84:0/1132954638 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0b7006af90 0x7f0b7006d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000c240 tx=0x7f0b80009030 comp rx=0 tx=0).stop 2023-10-16T10:07:34.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.229+0000 7f0b95f47640 1 -- 172.21.15.84:0/1132954638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b9015af80 msgr2=0x7f0b900b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:34.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.229+0000 7f0b95f47640 1 --2- 172.21.15.84:0/1132954638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b9015af80 0x7f0b900b2810 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f0b900c9ad0 tx=0x7f0b88009dc0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.229+0000 7f0b95f47640 1 -- 172.21.15.84:0/1132954638 shutdown_connections 2023-10-16T10:07:34.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.229+0000 7f0b95f47640 1 --2- 172.21.15.84:0/1132954638 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0b7006af90 0x7f0b7006d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.229+0000 7f0b95f47640 1 --2- 172.21.15.84:0/1132954638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b9015af80 0x7f0b900b2810 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.229+0000 7f0b95f47640 1 --2- 172.21.15.84:0/1132954638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b9015ab00 0x7f0b900b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.229+0000 7f0b95f47640 1 --2- 172.21.15.84:0/1132954638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b7c0039a0 0x7f0b90155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.229+0000 7f0b95f47640 1 -- 172.21.15.84:0/1132954638 >> 172.21.15.84:0/1132954638 conn(0x7f0b900cb410 msgr2=0x7f0b90158f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:34.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.229+0000 7f0b95f47640 1 -- 172.21.15.84:0/1132954638 shutdown_connections 2023-10-16T10:07:34.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.229+0000 7f0b95f47640 1 -- 172.21.15.84:0/1132954638 wait complete. 2023-10-16T10:07:34.247 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-16T10:07:33.455588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":928,"num_read_kb":1790,"num_write":365,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2662,"ondisk_log_size":2662,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333440,"kb_used_data":13192,"kb_used_omap":12,"kb_used_meta":320115,"kb_avail":754641280,"statfs":{"total":773094113280,"available":772752670720,"internally_reserved":0,"allocated":13508608,"data_stored":7037284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":327798329},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001718"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.038152+0000","last_change":"2023-10-16T10:07:29.382093+0000","last_active":"2023-10-16T10:07:31.038152+0000","last_peered":"2023-10-16T10:07:31.038152+0000","last_clean":"2023-10-16T10:07:31.038152+0000","last_became_active":"2023-10-16T10:07:29.381687+0000","last_became_peered":"2023-10-16T10:07:29.381687+0000","last_unstale":"2023-10-16T10:07:31.038152+0000","last_undegraded":"2023-10-16T10:07:31.038152+0000","last_fullsized":"2023-10-16T10:07:31.038152+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:26.661030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.039568+0000","last_change":"2023-10-16T10:07:29.379842+0000","last_active":"2023-10-16T10:07:31.039568+0000","last_peered":"2023-10-16T10:07:31.039568+0000","last_clean":"2023-10-16T10:07:31.039568+0000","last_became_active":"2023-10-16T10:07:29.379482+0000","last_became_peered":"2023-10-16T10:07:29.379482+0000","last_unstale":"2023-10-16T10:07:31.039568+0000","last_undegraded":"2023-10-16T10:07:31.039568+0000","last_fullsized":"2023-10-16T10:07:31.039568+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:33:28.020956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.037563+0000","last_change":"2023-10-16T10:07:29.386733+0000","last_active":"2023-10-16T10:07:31.037563+0000","last_peered":"2023-10-16T10:07:31.037563+0000","last_clean":"2023-10-16T10:07:31.037563+0000","last_became_active":"2023-10-16T10:07:29.382142+0000","last_became_peered":"2023-10-16T10:07:29.382142+0000","last_unstale":"2023-10-16T10:07:31.037563+0000","last_undegraded":"2023-10-16T10:07:31.037563+0000","last_fullsized":"2023-10-16T10:07:31.037563+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:24.429514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.038414+0000","last_change":"2023-10-16T10:07:29.371178+0000","last_active":"2023-10-16T10:07:31.038414+0000","last_peered":"2023-10-16T10:07:31.038414+0000","last_clean":"2023-10-16T10:07:31.038414+0000","last_became_active":"2023-10-16T10:07:29.369217+0000","last_became_peered":"2023-10-16T10:07:29.369217+0000","last_unstale":"2023-10-16T10:07:31.038414+0000","last_undegraded":"2023-10-16T10:07:31.038414+0000","last_fullsized":"2023-10-16T10:07:31.038414+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:36:21.816584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.372732+0000","last_change":"2023-10-16T10:07:29.368997+0000","last_active":"2023-10-16T10:07:30.372732+0000","last_peered":"2023-10-16T10:07:30.372732+0000","last_clean":"2023-10-16T10:07:30.372732+0000","last_became_active":"2023-10-16T10:07:29.368028+0000","last_became_peered":"2023-10-16T10:07:29.368028+0000","last_unstale":"2023-10-16T10:07:30.372732+0000","last_undegraded":"2023-10-16T10:07:30.372732+0000","last_fullsized":"2023-10-16T10:07:30.372732+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:08:36.998720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036734+0000","last_change":"2023-10-16T10:07:29.372561+0000","last_active":"2023-10-16T10:07:31.036734+0000","last_peered":"2023-10-16T10:07:31.036734+0000","last_clean":"2023-10-16T10:07:31.036734+0000","last_became_active":"2023-10-16T10:07:29.372275+0000","last_became_peered":"2023-10-16T10:07:29.372275+0000","last_unstale":"2023-10-16T10:07:31.036734+0000","last_undegraded":"2023-10-16T10:07:31.036734+0000","last_fullsized":"2023-10-16T10:07:31.036734+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:25:12.697183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036416+0000","last_change":"2023-10-16T10:07:29.372938+0000","last_active":"2023-10-16T10:07:31.036416+0000","last_peered":"2023-10-16T10:07:31.036416+0000","last_clean":"2023-10-16T10:07:31.036416+0000","last_became_active":"2023-10-16T10:07:29.372197+0000","last_became_peered":"2023-10-16T10:07:29.372197+0000","last_unstale":"2023-10-16T10:07:31.036416+0000","last_undegraded":"2023-10-16T10:07:31.036416+0000","last_fullsized":"2023-10-16T10:07:31.036416+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:57:48.375352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036049+0000","last_change":"2023-10-16T10:07:29.377578+0000","last_active":"2023-10-16T10:07:31.036049+0000","last_peered":"2023-10-16T10:07:31.036049+0000","last_clean":"2023-10-16T10:07:31.036049+0000","last_became_active":"2023-10-16T10:07:29.377291+0000","last_became_peered":"2023-10-16T10:07:29.377291+0000","last_unstale":"2023-10-16T10:07:31.036049+0000","last_undegraded":"2023-10-16T10:07:31.036049+0000","last_fullsized":"2023-10-16T10:07:31.036049+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:40.382024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.035912+0000","last_change":"2023-10-16T10:07:29.369758+0000","last_active":"2023-10-16T10:07:31.035912+0000","last_peered":"2023-10-16T10:07:31.035912+0000","last_clean":"2023-10-16T10:07:31.035912+0000","last_became_active":"2023-10-16T10:07:29.369464+0000","last_became_peered":"2023-10-16T10:07:29.369464+0000","last_unstale":"2023-10-16T10:07:31.035912+0000","last_undegraded":"2023-10-16T10:07:31.035912+0000","last_fullsized":"2023-10-16T10:07:31.035912+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:11:24.261638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.372287+0000","last_change":"2023-10-16T10:07:29.376624+0000","last_active":"2023-10-16T10:07:30.372287+0000","last_peered":"2023-10-16T10:07:30.372287+0000","last_clean":"2023-10-16T10:07:30.372287+0000","last_became_active":"2023-10-16T10:07:29.372821+0000","last_became_peered":"2023-10-16T10:07:29.372821+0000","last_unstale":"2023-10-16T10:07:30.372287+0000","last_undegraded":"2023-10-16T10:07:30.372287+0000","last_fullsized":"2023-10-16T10:07:30.372287+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:26:48.170692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036971+0000","last_change":"2023-10-16T10:07:29.376016+0000","last_active":"2023-10-16T10:07:31.036971+0000","last_peered":"2023-10-16T10:07:31.036971+0000","last_clean":"2023-10-16T10:07:31.036971+0000","last_became_active":"2023-10-16T10:07:29.373333+0000","last_became_peered":"2023-10-16T10:07:29.373333+0000","last_unstale":"2023-10-16T10:07:31.036971+0000","last_undegraded":"2023-10-16T10:07:31.036971+0000","last_fullsized":"2023-10-16T10:07:31.036971+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:48.025764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.035987+0000","last_change":"2023-10-16T10:07:29.374068+0000","last_active":"2023-10-16T10:07:31.035987+0000","last_peered":"2023-10-16T10:07:31.035987+0000","last_clean":"2023-10-16T10:07:31.035987+0000","last_became_active":"2023-10-16T10:07:29.373810+0000","last_became_peered":"2023-10-16T10:07:29.373810+0000","last_unstale":"2023-10-16T10:07:31.035987+0000","last_undegraded":"2023-10-16T10:07:31.035987+0000","last_fullsized":"2023-10-16T10:07:31.035987+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:17.260480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.039352+0000","last_change":"2023-10-16T10:07:29.358718+0000","last_active":"2023-10-16T10:07:31.039352+0000","last_peered":"2023-10-16T10:07:31.039352+0000","last_clean":"2023-10-16T10:07:31.039352+0000","last_became_active":"2023-10-16T10:07:29.356998+0000","last_became_peered":"2023-10-16T10:07:29.356998+0000","last_unstale":"2023-10-16T10:07:31.039352+0000","last_undegraded":"2023-10-16T10:07:31.039352+0000","last_fullsized":"2023-10-16T10:07:31.039352+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:33:43.615838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.367762+0000","last_change":"2023-10-16T10:07:29.372847+0000","last_active":"2023-10-16T10:07:30.367762+0000","last_peered":"2023-10-16T10:07:30.367762+0000","last_clean":"2023-10-16T10:07:30.367762+0000","last_became_active":"2023-10-16T10:07:29.369970+0000","last_became_peered":"2023-10-16T10:07:29.369970+0000","last_unstale":"2023-10-16T10:07:30.367762+0000","last_undegraded":"2023-10-16T10:07:30.367762+0000","last_fullsized":"2023-10-16T10:07:30.367762+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:47.868903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.037078+0000","last_change":"2023-10-16T10:07:29.367146+0000","last_active":"2023-10-16T10:07:31.037078+0000","last_peered":"2023-10-16T10:07:31.037078+0000","last_clean":"2023-10-16T10:07:31.037078+0000","last_became_active":"2023-10-16T10:07:29.365999+0000","last_became_peered":"2023-10-16T10:07:29.365999+0000","last_unstale":"2023-10-16T10:07:31.037078+0000","last_undegraded":"2023-10-16T10:07:31.037078+0000","last_fullsized":"2023-10-16T10:07:31.037078+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:02:50.044180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036744+0000","last_change":"2023-10-16T10:07:29.373230+0000","last_active":"2023-10-16T10:07:31.036744+0000","last_peered":"2023-10-16T10:07:31.036744+0000","last_clean":"2023-10-16T10:07:31.036744+0000","last_became_active":"2023-10-16T10:07:29.372983+0000","last_became_peered":"2023-10-16T10:07:29.372983+0000","last_unstale":"2023-10-16T10:07:31.036744+0000","last_undegraded":"2023-10-16T10:07:31.036744+0000","last_fullsized":"2023-10-16T10:07:31.036744+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:25.040628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.039963+0000","last_change":"2023-10-16T10:07:29.392036+0000","last_active":"2023-10-16T10:07:31.039963+0000","last_peered":"2023-10-16T10:07:31.039963+0000","last_clean":"2023-10-16T10:07:31.039963+0000","last_became_active":"2023-10-16T10:07:29.390853+0000","last_became_peered":"2023-10-16T10:07:29.390853+0000","last_unstale":"2023-10-16T10:07:31.039963+0000","last_undegraded":"2023-10-16T10:07:31.039963+0000","last_fullsized":"2023-10-16T10:07:31.039963+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:58:13.282872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.038705+0000","last_change":"2023-10-16T10:07:29.367515+0000","last_active":"2023-10-16T10:07:31.038705+0000","last_peered":"2023-10-16T10:07:31.038705+0000","last_clean":"2023-10-16T10:07:31.038705+0000","last_became_active":"2023-10-16T10:07:29.367182+0000","last_became_peered":"2023-10-16T10:07:29.367182+0000","last_unstale":"2023-10-16T10:07:31.038705+0000","last_undegraded":"2023-10-16T10:07:31.038705+0000","last_fullsized":"2023-10-16T10:07:31.038705+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:45:14.229013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.368022+0000","last_change":"2023-10-16T10:07:29.374134+0000","last_active":"2023-10-16T10:07:30.368022+0000","last_peered":"2023-10-16T10:07:30.368022+0000","last_clean":"2023-10-16T10:07:30.368022+0000","last_became_active":"2023-10-16T10:07:29.373871+0000","last_became_peered":"2023-10-16T10:07:29.373871+0000","last_unstale":"2023-10-16T10:07:30.368022+0000","last_undegraded":"2023-10-16T10:07:30.368022+0000","last_fullsized":"2023-10-16T10:07:30.368022+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:40.382024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036362+0000","last_change":"2023-10-16T10:07:29.356960+0000","last_active":"2023-10-16T10:07:31.036362+0000","last_peered":"2023-10-16T10:07:31.036362+0000","last_clean":"2023-10-16T10:07:31.036362+0000","last_became_active":"2023-10-16T10:07:29.356180+0000","last_became_peered":"2023-10-16T10:07:29.356180+0000","last_unstale":"2023-10-16T10:07:31.036362+0000","last_undegraded":"2023-10-16T10:07:31.036362+0000","last_fullsized":"2023-10-16T10:07:31.036362+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:16:22.572793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.367597+0000","last_change":"2023-10-16T10:07:29.376964+0000","last_active":"2023-10-16T10:07:30.367597+0000","last_peered":"2023-10-16T10:07:30.367597+0000","last_clean":"2023-10-16T10:07:30.367597+0000","last_became_active":"2023-10-16T10:07:29.376643+0000","last_became_peered":"2023-10-16T10:07:29.376643+0000","last_unstale":"2023-10-16T10:07:30.367597+0000","last_undegraded":"2023-10-16T10:07:30.367597+0000","last_fullsized":"2023-10-16T10:07:30.367597+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:12.622336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.038569+0000","last_change":"2023-10-16T10:07:29.379469+0000","last_active":"2023-10-16T10:07:31.038569+0000","last_peered":"2023-10-16T10:07:31.038569+0000","last_clean":"2023-10-16T10:07:31.038569+0000","last_became_active":"2023-10-16T10:07:29.379148+0000","last_became_peered":"2023-10-16T10:07:29.379148+0000","last_unstale":"2023-10-16T10:07:31.038569+0000","last_undegraded":"2023-10-16T10:07:31.038569+0000","last_fullsized":"2023-10-16T10:07:31.038569+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:06.419087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.035664+0000","last_change":"2023-10-16T10:07:29.377683+0000","last_active":"2023-10-16T10:07:31.035664+0000","last_peered":"2023-10-16T10:07:31.035664+0000","last_clean":"2023-10-16T10:07:31.035664+0000","last_became_active":"2023-10-16T10:07:29.377372+0000","last_became_peered":"2023-10-16T10:07:29.377372+0000","last_unstale":"2023-10-16T10:07:31.035664+0000","last_undegraded":"2023-10-16T10:07:31.035664+0000","last_fullsized":"2023-10-16T10:07:31.035664+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:00:14.321059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036953+0000","last_change":"2023-10-16T10:07:29.371436+0000","last_active":"2023-10-16T10:07:31.036953+0000","last_peered":"2023-10-16T10:07:31.036953+0000","last_clean":"2023-10-16T10:07:31.036953+0000","last_became_active":"2023-10-16T10:07:29.371150+0000","last_became_peered":"2023-10-16T10:07:29.371150+0000","last_unstale":"2023-10-16T10:07:31.036953+0000","last_undegraded":"2023-10-16T10:07:31.036953+0000","last_fullsized":"2023-10-16T10:07:31.036953+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:09:49.494032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.372741+0000","last_change":"2023-10-16T10:07:29.382549+0000","last_active":"2023-10-16T10:07:30.372741+0000","last_peered":"2023-10-16T10:07:30.372741+0000","last_clean":"2023-10-16T10:07:30.372741+0000","last_became_active":"2023-10-16T10:07:29.380506+0000","last_became_peered":"2023-10-16T10:07:29.380506+0000","last_unstale":"2023-10-16T10:07:30.372741+0000","last_undegraded":"2023-10-16T10:07:30.372741+0000","last_fullsized":"2023-10-16T10:07:30.372741+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:33:28.020956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.039938+0000","last_change":"2023-10-16T10:07:29.386247+0000","last_active":"2023-10-16T10:07:31.039938+0000","last_peered":"2023-10-16T10:07:31.039938+0000","last_clean":"2023-10-16T10:07:31.039938+0000","last_became_active":"2023-10-16T10:07:29.385605+0000","last_became_peered":"2023-10-16T10:07:29.385605+0000","last_unstale":"2023-10-16T10:07:31.039938+0000","last_undegraded":"2023-10-16T10:07:31.039938+0000","last_fullsized":"2023-10-16T10:07:31.039938+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:34.906651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.037230+0000","last_change":"2023-10-16T10:07:29.365734+0000","last_active":"2023-10-16T10:07:31.037230+0000","last_peered":"2023-10-16T10:07:31.037230+0000","last_clean":"2023-10-16T10:07:31.037230+0000","last_became_active":"2023-10-16T10:07:29.365459+0000","last_became_peered":"2023-10-16T10:07:29.365459+0000","last_unstale":"2023-10-16T10:07:31.037230+0000","last_undegraded":"2023-10-16T10:07:31.037230+0000","last_fullsized":"2023-10-16T10:07:31.037230+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:54.133205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.372597+0000","last_change":"2023-10-16T10:07:29.374472+0000","last_active":"2023-10-16T10:07:30.372597+0000","last_peered":"2023-10-16T10:07:30.372597+0000","last_clean":"2023-10-16T10:07:30.372597+0000","last_became_active":"2023-10-16T10:07:29.374039+0000","last_became_peered":"2023-10-16T10:07:29.374039+0000","last_unstale":"2023-10-16T10:07:30.372597+0000","last_undegraded":"2023-10-16T10:07:30.372597+0000","last_fullsized":"2023-10-16T10:07:30.372597+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:40.382024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.038823+0000","last_change":"2023-10-16T10:07:29.384272+0000","last_active":"2023-10-16T10:07:31.038823+0000","last_peered":"2023-10-16T10:07:31.038823+0000","last_clean":"2023-10-16T10:07:31.038823+0000","last_became_active":"2023-10-16T10:07:29.383866+0000","last_became_peered":"2023-10-16T10:07:29.383866+0000","last_unstale":"2023-10-16T10:07:31.038823+0000","last_undegraded":"2023-10-16T10:07:31.038823+0000","last_fullsized":"2023-10-16T10:07:31.038823+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:08:36.998720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.367955+0000","last_change":"2023-10-16T10:07:29.368739+0000","last_active":"2023-10-16T10:07:30.367955+0000","last_peered":"2023-10-16T10:07:30.367955+0000","last_clean":"2023-10-16T10:07:30.367955+0000","last_became_active":"2023-10-16T10:07:29.368461+0000","last_became_peered":"2023-10-16T10:07:29.368461+0000","last_unstale":"2023-10-16T10:07:30.367955+0000","last_undegraded":"2023-10-16T10:07:30.367955+0000","last_fullsized":"2023-10-16T10:07:30.367955+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:26:48.170692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.037517+0000","last_change":"2023-10-16T10:07:29.374317+0000","last_active":"2023-10-16T10:07:31.037517+0000","last_peered":"2023-10-16T10:07:31.037517+0000","last_clean":"2023-10-16T10:07:31.037517+0000","last_became_active":"2023-10-16T10:07:29.373509+0000","last_became_peered":"2023-10-16T10:07:29.373509+0000","last_unstale":"2023-10-16T10:07:31.037517+0000","last_undegraded":"2023-10-16T10:07:31.037517+0000","last_fullsized":"2023-10-16T10:07:31.037517+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:43:25.862353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.035659+0000","last_change":"2023-10-16T10:07:29.375192+0000","last_active":"2023-10-16T10:07:31.035659+0000","last_peered":"2023-10-16T10:07:31.035659+0000","last_clean":"2023-10-16T10:07:31.035659+0000","last_became_active":"2023-10-16T10:07:29.373537+0000","last_became_peered":"2023-10-16T10:07:29.373537+0000","last_unstale":"2023-10-16T10:07:31.035659+0000","last_undegraded":"2023-10-16T10:07:31.035659+0000","last_fullsized":"2023-10-16T10:07:31.035659+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:49:03.178809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.368270+0000","last_change":"2023-10-16T10:07:29.368773+0000","last_active":"2023-10-16T10:07:30.368270+0000","last_peered":"2023-10-16T10:07:30.368270+0000","last_clean":"2023-10-16T10:07:30.368270+0000","last_became_active":"2023-10-16T10:07:29.366701+0000","last_became_peered":"2023-10-16T10:07:29.366701+0000","last_unstale":"2023-10-16T10:07:30.368270+0000","last_undegraded":"2023-10-16T10:07:30.368270+0000","last_fullsized":"2023-10-16T10:07:30.368270+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:08:36.998720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"123'4","reported_seq":16,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.395391+0000","last_change":"2023-10-16T10:07:28.361614+0000","last_active":"2023-10-16T10:07:29.395391+0000","last_peered":"2023-10-16T10:07:29.395391+0000","last_clean":"2023-10-16T10:07:29.395391+0000","last_became_active":"2023-10-16T10:07:28.361357+0000","last_became_peered":"2023-10-16T10:07:28.361357+0000","last_unstale":"2023-10-16T10:07:29.395391+0000","last_undegraded":"2023-10-16T10:07:29.395391+0000","last_fullsized":"2023-10-16T10:07:29.395391+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:27:20.422020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.038888+0000","last_change":"2023-10-16T10:07:29.373919+0000","last_active":"2023-10-16T10:07:31.038888+0000","last_peered":"2023-10-16T10:07:31.038888+0000","last_clean":"2023-10-16T10:07:31.038888+0000","last_became_active":"2023-10-16T10:07:29.373562+0000","last_became_peered":"2023-10-16T10:07:29.373562+0000","last_unstale":"2023-10-16T10:07:31.038888+0000","last_undegraded":"2023-10-16T10:07:31.038888+0000","last_fullsized":"2023-10-16T10:07:31.038888+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:07:13.553899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"123'1","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.392986+0000","last_change":"2023-10-16T10:07:28.370286+0000","last_active":"2023-10-16T10:07:29.392986+0000","last_peered":"2023-10-16T10:07:29.392986+0000","last_clean":"2023-10-16T10:07:29.392986+0000","last_became_active":"2023-10-16T10:07:28.369845+0000","last_became_peered":"2023-10-16T10:07:28.369845+0000","last_unstale":"2023-10-16T10:07:29.392986+0000","last_undegraded":"2023-10-16T10:07:29.392986+0000","last_fullsized":"2023-10-16T10:07:29.392986+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:33:42.598522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036378+0000","last_change":"2023-10-16T10:07:29.374864+0000","last_active":"2023-10-16T10:07:31.036378+0000","last_peered":"2023-10-16T10:07:31.036378+0000","last_clean":"2023-10-16T10:07:31.036378+0000","last_became_active":"2023-10-16T10:07:29.374593+0000","last_became_peered":"2023-10-16T10:07:29.374593+0000","last_unstale":"2023-10-16T10:07:31.036378+0000","last_undegraded":"2023-10-16T10:07:31.036378+0000","last_fullsized":"2023-10-16T10:07:31.036378+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:30.854910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"123'2","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.392985+0000","last_change":"2023-10-16T10:07:28.360255+0000","last_active":"2023-10-16T10:07:29.392985+0000","last_peered":"2023-10-16T10:07:29.392985+0000","last_clean":"2023-10-16T10:07:29.392985+0000","last_became_active":"2023-10-16T10:07:28.359785+0000","last_became_peered":"2023-10-16T10:07:28.359785+0000","last_unstale":"2023-10-16T10:07:29.392985+0000","last_undegraded":"2023-10-16T10:07:29.392985+0000","last_fullsized":"2023-10-16T10:07:29.392985+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:11.605020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036341+0000","last_change":"2023-10-16T10:07:29.377813+0000","last_active":"2023-10-16T10:07:31.036341+0000","last_peered":"2023-10-16T10:07:31.036341+0000","last_clean":"2023-10-16T10:07:31.036341+0000","last_became_active":"2023-10-16T10:07:29.377533+0000","last_became_peered":"2023-10-16T10:07:29.377533+0000","last_unstale":"2023-10-16T10:07:31.036341+0000","last_undegraded":"2023-10-16T10:07:31.036341+0000","last_fullsized":"2023-10-16T10:07:31.036341+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:18.346286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"123'2","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.394146+0000","last_change":"2023-10-16T10:07:28.367658+0000","last_active":"2023-10-16T10:07:29.394146+0000","last_peered":"2023-10-16T10:07:29.394146+0000","last_clean":"2023-10-16T10:07:29.394146+0000","last_became_active":"2023-10-16T10:07:28.367182+0000","last_became_peered":"2023-10-16T10:07:28.367182+0000","last_unstale":"2023-10-16T10:07:29.394146+0000","last_undegraded":"2023-10-16T10:07:29.394146+0000","last_fullsized":"2023-10-16T10:07:29.394146+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:27.586892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.369853+0000","last_change":"2023-10-16T10:07:29.377214+0000","last_active":"2023-10-16T10:07:30.369853+0000","last_peered":"2023-10-16T10:07:30.369853+0000","last_clean":"2023-10-16T10:07:30.369853+0000","last_became_active":"2023-10-16T10:07:29.373847+0000","last_became_peered":"2023-10-16T10:07:29.373847+0000","last_unstale":"2023-10-16T10:07:30.369853+0000","last_undegraded":"2023-10-16T10:07:30.369853+0000","last_fullsized":"2023-10-16T10:07:30.369853+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:00.092762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"123'1","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.392467+0000","last_change":"2023-10-16T10:07:28.357437+0000","last_active":"2023-10-16T10:07:29.392467+0000","last_peered":"2023-10-16T10:07:29.392467+0000","last_clean":"2023-10-16T10:07:29.392467+0000","last_became_active":"2023-10-16T10:07:28.356851+0000","last_became_peered":"2023-10-16T10:07:28.356851+0000","last_unstale":"2023-10-16T10:07:29.392467+0000","last_undegraded":"2023-10-16T10:07:29.392467+0000","last_fullsized":"2023-10-16T10:07:29.392467+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:08.827809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.037365+0000","last_change":"2023-10-16T10:07:29.374434+0000","last_active":"2023-10-16T10:07:31.037365+0000","last_peered":"2023-10-16T10:07:31.037365+0000","last_clean":"2023-10-16T10:07:31.037365+0000","last_became_active":"2023-10-16T10:07:29.373410+0000","last_became_peered":"2023-10-16T10:07:29.373410+0000","last_unstale":"2023-10-16T10:07:31.037365+0000","last_undegraded":"2023-10-16T10:07:31.037365+0000","last_fullsized":"2023-10-16T10:07:31.037365+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:25.786399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"123'1","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.393694+0000","last_change":"2023-10-16T10:07:28.359504+0000","last_active":"2023-10-16T10:07:29.393694+0000","last_peered":"2023-10-16T10:07:29.393694+0000","last_clean":"2023-10-16T10:07:29.393694+0000","last_became_active":"2023-10-16T10:07:28.359189+0000","last_became_peered":"2023-10-16T10:07:28.359189+0000","last_unstale":"2023-10-16T10:07:29.393694+0000","last_undegraded":"2023-10-16T10:07:29.393694+0000","last_fullsized":"2023-10-16T10:07:29.393694+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:27.207835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036494+0000","last_change":"2023-10-16T10:07:29.371383+0000","last_active":"2023-10-16T10:07:31.036494+0000","last_peered":"2023-10-16T10:07:31.036494+0000","last_clean":"2023-10-16T10:07:31.036494+0000","last_became_active":"2023-10-16T10:07:29.369223+0000","last_became_peered":"2023-10-16T10:07:29.369223+0000","last_unstale":"2023-10-16T10:07:31.036494+0000","last_undegraded":"2023-10-16T10:07:31.036494+0000","last_fullsized":"2023-10-16T10:07:31.036494+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:42:07.800731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.365532+0000","last_change":"2023-10-16T10:07:28.366092+0000","last_active":"2023-10-16T10:07:29.365532+0000","last_peered":"2023-10-16T10:07:29.365532+0000","last_clean":"2023-10-16T10:07:29.365532+0000","last_became_active":"2023-10-16T10:07:28.365734+0000","last_became_peered":"2023-10-16T10:07:28.365734+0000","last_unstale":"2023-10-16T10:07:29.365532+0000","last_undegraded":"2023-10-16T10:07:29.365532+0000","last_fullsized":"2023-10-16T10:07:29.365532+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:27.207835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.035303+0000","last_change":"2023-10-16T10:07:29.368389+0000","last_active":"2023-10-16T10:07:31.035303+0000","last_peered":"2023-10-16T10:07:31.035303+0000","last_clean":"2023-10-16T10:07:31.035303+0000","last_became_active":"2023-10-16T10:07:29.368032+0000","last_became_peered":"2023-10-16T10:07:29.368032+0000","last_unstale":"2023-10-16T10:07:31.035303+0000","last_undegraded":"2023-10-16T10:07:31.035303+0000","last_fullsized":"2023-10-16T10:07:31.035303+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:17:51.605125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.361176+0000","last_change":"2023-10-16T10:07:28.356999+0000","last_active":"2023-10-16T10:07:29.361176+0000","last_peered":"2023-10-16T10:07:29.361176+0000","last_clean":"2023-10-16T10:07:29.361176+0000","last_became_active":"2023-10-16T10:07:28.356707+0000","last_became_peered":"2023-10-16T10:07:28.356707+0000","last_unstale":"2023-10-16T10:07:29.361176+0000","last_undegraded":"2023-10-16T10:07:29.361176+0000","last_fullsized":"2023-10-16T10:07:29.361176+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:33.889335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.040130+0000","last_change":"2023-10-16T10:07:29.386744+0000","last_active":"2023-10-16T10:07:31.040130+0000","last_peered":"2023-10-16T10:07:31.040130+0000","last_clean":"2023-10-16T10:07:31.040130+0000","last_became_active":"2023-10-16T10:07:29.386476+0000","last_became_peered":"2023-10-16T10:07:29.386476+0000","last_unstale":"2023-10-16T10:07:31.040130+0000","last_undegraded":"2023-10-16T10:07:31.040130+0000","last_fullsized":"2023-10-16T10:07:31.040130+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:21:49.156446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.362781+0000","last_change":"2023-10-16T10:07:28.362884+0000","last_active":"2023-10-16T10:07:29.362781+0000","last_peered":"2023-10-16T10:07:29.362781+0000","last_clean":"2023-10-16T10:07:29.362781+0000","last_became_active":"2023-10-16T10:07:28.361604+0000","last_became_peered":"2023-10-16T10:07:28.361604+0000","last_unstale":"2023-10-16T10:07:29.362781+0000","last_undegraded":"2023-10-16T10:07:29.362781+0000","last_fullsized":"2023-10-16T10:07:29.362781+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:15.702581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.035956+0000","last_change":"2023-10-16T10:07:29.382243+0000","last_active":"2023-10-16T10:07:31.035956+0000","last_peered":"2023-10-16T10:07:31.035956+0000","last_clean":"2023-10-16T10:07:31.035956+0000","last_became_active":"2023-10-16T10:07:29.381957+0000","last_became_peered":"2023-10-16T10:07:29.381957+0000","last_unstale":"2023-10-16T10:07:31.035956+0000","last_undegraded":"2023-10-16T10:07:31.035956+0000","last_fullsized":"2023-10-16T10:07:31.035956+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:26.463597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.355031+0000","last_change":"2023-10-16T10:07:28.366495+0000","last_active":"2023-10-16T10:07:29.355031+0000","last_peered":"2023-10-16T10:07:29.355031+0000","last_clean":"2023-10-16T10:07:29.355031+0000","last_became_active":"2023-10-16T10:07:28.365028+0000","last_became_peered":"2023-10-16T10:07:28.365028+0000","last_unstale":"2023-10-16T10:07:29.355031+0000","last_undegraded":"2023-10-16T10:07:29.355031+0000","last_fullsized":"2023-10-16T10:07:29.355031+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:00.086702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.372867+0000","last_change":"2023-10-16T10:07:29.391473+0000","last_active":"2023-10-16T10:07:30.372867+0000","last_peered":"2023-10-16T10:07:30.372867+0000","last_clean":"2023-10-16T10:07:30.372867+0000","last_became_active":"2023-10-16T10:07:29.391128+0000","last_became_peered":"2023-10-16T10:07:29.391128+0000","last_unstale":"2023-10-16T10:07:30.372867+0000","last_undegraded":"2023-10-16T10:07:30.372867+0000","last_fullsized":"2023-10-16T10:07:30.372867+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:34.906651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"123'3","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.390415+0000","last_change":"2023-10-16T10:07:28.369936+0000","last_active":"2023-10-16T10:07:29.390415+0000","last_peered":"2023-10-16T10:07:29.390415+0000","last_clean":"2023-10-16T10:07:29.390415+0000","last_became_active":"2023-10-16T10:07:28.369613+0000","last_became_peered":"2023-10-16T10:07:28.369613+0000","last_unstale":"2023-10-16T10:07:29.390415+0000","last_undegraded":"2023-10-16T10:07:29.390415+0000","last_fullsized":"2023-10-16T10:07:29.390415+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:00:40.716420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.037529+0000","last_change":"2023-10-16T10:07:29.381218+0000","last_active":"2023-10-16T10:07:31.037529+0000","last_peered":"2023-10-16T10:07:31.037529+0000","last_clean":"2023-10-16T10:07:31.037529+0000","last_became_active":"2023-10-16T10:07:29.380889+0000","last_became_peered":"2023-10-16T10:07:29.380889+0000","last_unstale":"2023-10-16T10:07:31.037529+0000","last_undegraded":"2023-10-16T10:07:31.037529+0000","last_fullsized":"2023-10-16T10:07:31.037529+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:24:36.397822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"123'6","reported_seq":18,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.394464+0000","last_change":"2023-10-16T10:07:28.357341+0000","last_active":"2023-10-16T10:07:29.394464+0000","last_peered":"2023-10-16T10:07:29.394464+0000","last_clean":"2023-10-16T10:07:29.394464+0000","last_became_active":"2023-10-16T10:07:28.357050+0000","last_became_peered":"2023-10-16T10:07:28.357050+0000","last_unstale":"2023-10-16T10:07:29.394464+0000","last_undegraded":"2023-10-16T10:07:29.394464+0000","last_fullsized":"2023-10-16T10:07:29.394464+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:30:09.063022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.037616+0000","last_change":"2023-10-16T10:07:29.376998+0000","last_active":"2023-10-16T10:07:31.037616+0000","last_peered":"2023-10-16T10:07:31.037616+0000","last_clean":"2023-10-16T10:07:31.037616+0000","last_became_active":"2023-10-16T10:07:29.376740+0000","last_became_peered":"2023-10-16T10:07:29.376740+0000","last_unstale":"2023-10-16T10:07:31.037616+0000","last_undegraded":"2023-10-16T10:07:31.037616+0000","last_fullsized":"2023-10-16T10:07:31.037616+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:02.053186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"123'2","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.391985+0000","last_change":"2023-10-16T10:07:28.360250+0000","last_active":"2023-10-16T10:07:29.391985+0000","last_peered":"2023-10-16T10:07:29.391985+0000","last_clean":"2023-10-16T10:07:29.391985+0000","last_became_active":"2023-10-16T10:07:28.358653+0000","last_became_peered":"2023-10-16T10:07:28.358653+0000","last_unstale":"2023-10-16T10:07:29.391985+0000","last_undegraded":"2023-10-16T10:07:29.391985+0000","last_fullsized":"2023-10-16T10:07:29.391985+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:49.373492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.036989+0000","last_change":"2023-10-16T10:07:29.388058+0000","last_active":"2023-10-16T10:07:31.036989+0000","last_peered":"2023-10-16T10:07:31.036989+0000","last_clean":"2023-10-16T10:07:31.036989+0000","last_became_active":"2023-10-16T10:07:29.382803+0000","last_became_peered":"2023-10-16T10:07:29.382803+0000","last_unstale":"2023-10-16T10:07:31.036989+0000","last_undegraded":"2023-10-16T10:07:31.036989+0000","last_fullsized":"2023-10-16T10:07:31.036989+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:17:22.404985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.348509+0000","last_change":"2023-10-16T10:07:28.358918+0000","last_active":"2023-10-16T10:07:29.348509+0000","last_peered":"2023-10-16T10:07:29.348509+0000","last_clean":"2023-10-16T10:07:29.348509+0000","last_became_active":"2023-10-16T10:07:28.358669+0000","last_became_peered":"2023-10-16T10:07:28.358669+0000","last_unstale":"2023-10-16T10:07:29.348509+0000","last_undegraded":"2023-10-16T10:07:29.348509+0000","last_fullsized":"2023-10-16T10:07:29.348509+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:13.028657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.035212+0000","last_change":"2023-10-16T10:07:29.366841+0000","last_active":"2023-10-16T10:07:31.035212+0000","last_peered":"2023-10-16T10:07:31.035212+0000","last_clean":"2023-10-16T10:07:31.035212+0000","last_became_active":"2023-10-16T10:07:29.365834+0000","last_became_peered":"2023-10-16T10:07:29.365834+0000","last_unstale":"2023-10-16T10:07:31.035212+0000","last_undegraded":"2023-10-16T10:07:31.035212+0000","last_fullsized":"2023-10-16T10:07:31.035212+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:44:59.979486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.343960+0000","last_change":"2023-10-16T10:07:28.363095+0000","last_active":"2023-10-16T10:07:29.343960+0000","last_peered":"2023-10-16T10:07:29.343960+0000","last_clean":"2023-10-16T10:07:29.343960+0000","last_became_active":"2023-10-16T10:07:28.362803+0000","last_became_peered":"2023-10-16T10:07:28.362803+0000","last_unstale":"2023-10-16T10:07:29.343960+0000","last_undegraded":"2023-10-16T10:07:29.343960+0000","last_fullsized":"2023-10-16T10:07:29.343960+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:30:31.668157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.367606+0000","last_change":"2023-10-16T10:07:29.362200+0000","last_active":"2023-10-16T10:07:30.367606+0000","last_peered":"2023-10-16T10:07:30.367606+0000","last_clean":"2023-10-16T10:07:30.367606+0000","last_became_active":"2023-10-16T10:07:29.356693+0000","last_became_peered":"2023-10-16T10:07:29.356693+0000","last_unstale":"2023-10-16T10:07:30.367606+0000","last_undegraded":"2023-10-16T10:07:30.367606+0000","last_fullsized":"2023-10-16T10:07:30.367606+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:27:21.439336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"123'8","reported_seq":20,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.391556+0000","last_change":"2023-10-16T10:07:28.365215+0000","last_active":"2023-10-16T10:07:29.391556+0000","last_peered":"2023-10-16T10:07:29.391556+0000","last_clean":"2023-10-16T10:07:29.391556+0000","last_became_active":"2023-10-16T10:07:28.364919+0000","last_became_peered":"2023-10-16T10:07:28.364919+0000","last_unstale":"2023-10-16T10:07:29.391556+0000","last_undegraded":"2023-10-16T10:07:29.391556+0000","last_fullsized":"2023-10-16T10:07:29.391556+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:58:12.265556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.372699+0000","last_change":"2023-10-16T10:07:29.378755+0000","last_active":"2023-10-16T10:07:30.372699+0000","last_peered":"2023-10-16T10:07:30.372699+0000","last_clean":"2023-10-16T10:07:30.372699+0000","last_became_active":"2023-10-16T10:07:29.377743+0000","last_became_peered":"2023-10-16T10:07:29.377743+0000","last_unstale":"2023-10-16T10:07:30.372699+0000","last_undegraded":"2023-10-16T10:07:30.372699+0000","last_fullsized":"2023-10-16T10:07:30.372699+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:06.419087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"123'2","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.387918+0000","last_change":"2023-10-16T10:07:28.357710+0000","last_active":"2023-10-16T10:07:29.387918+0000","last_peered":"2023-10-16T10:07:29.387918+0000","last_clean":"2023-10-16T10:07:29.387918+0000","last_became_active":"2023-10-16T10:07:28.356541+0000","last_became_peered":"2023-10-16T10:07:28.356541+0000","last_unstale":"2023-10-16T10:07:29.387918+0000","last_undegraded":"2023-10-16T10:07:29.387918+0000","last_fullsized":"2023-10-16T10:07:29.387918+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:38.126063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.370638+0000","last_change":"2023-10-16T10:07:29.385528+0000","last_active":"2023-10-16T10:07:30.370638+0000","last_peered":"2023-10-16T10:07:30.370638+0000","last_clean":"2023-10-16T10:07:30.370638+0000","last_became_active":"2023-10-16T10:07:29.385240+0000","last_became_peered":"2023-10-16T10:07:29.385240+0000","last_unstale":"2023-10-16T10:07:30.370638+0000","last_undegraded":"2023-10-16T10:07:30.370638+0000","last_fullsized":"2023-10-16T10:07:30.370638+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:16.686978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"123'1","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.389827+0000","last_change":"2023-10-16T10:07:28.362939+0000","last_active":"2023-10-16T10:07:29.389827+0000","last_peered":"2023-10-16T10:07:29.389827+0000","last_clean":"2023-10-16T10:07:29.389827+0000","last_became_active":"2023-10-16T10:07:28.362649+0000","last_became_peered":"2023-10-16T10:07:28.362649+0000","last_unstale":"2023-10-16T10:07:29.389827+0000","last_undegraded":"2023-10-16T10:07:29.389827+0000","last_fullsized":"2023-10-16T10:07:29.389827+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:04:13.458192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.037548+0000","last_change":"2023-10-16T10:07:29.374321+0000","last_active":"2023-10-16T10:07:31.037548+0000","last_peered":"2023-10-16T10:07:31.037548+0000","last_clean":"2023-10-16T10:07:31.037548+0000","last_became_active":"2023-10-16T10:07:29.372935+0000","last_became_peered":"2023-10-16T10:07:29.372935+0000","last_unstale":"2023-10-16T10:07:31.037548+0000","last_undegraded":"2023-10-16T10:07:31.037548+0000","last_fullsized":"2023-10-16T10:07:31.037548+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:07:34.048822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.377582+0000","last_change":"2023-10-16T10:07:28.367804+0000","last_active":"2023-10-16T10:07:29.377582+0000","last_peered":"2023-10-16T10:07:29.377582+0000","last_clean":"2023-10-16T10:07:29.377582+0000","last_became_active":"2023-10-16T10:07:28.367548+0000","last_became_peered":"2023-10-16T10:07:28.367548+0000","last_unstale":"2023-10-16T10:07:29.377582+0000","last_undegraded":"2023-10-16T10:07:29.377582+0000","last_fullsized":"2023-10-16T10:07:29.377582+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:26:47.153376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.367884+0000","last_change":"2023-10-16T10:07:29.365272+0000","last_active":"2023-10-16T10:07:30.367884+0000","last_peered":"2023-10-16T10:07:30.367884+0000","last_clean":"2023-10-16T10:07:30.367884+0000","last_became_active":"2023-10-16T10:07:29.364935+0000","last_became_peered":"2023-10-16T10:07:29.364935+0000","last_unstale":"2023-10-16T10:07:30.367884+0000","last_undegraded":"2023-10-16T10:07:30.367884+0000","last_fullsized":"2023-10-16T10:07:30.367884+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:28.604208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"123'2","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.394194+0000","last_change":"2023-10-16T10:07:28.364807+0000","last_active":"2023-10-16T10:07:29.394194+0000","last_peered":"2023-10-16T10:07:29.394194+0000","last_clean":"2023-10-16T10:07:29.394194+0000","last_became_active":"2023-10-16T10:07:28.364530+0000","last_became_peered":"2023-10-16T10:07:28.364530+0000","last_unstale":"2023-10-16T10:07:29.394194+0000","last_undegraded":"2023-10-16T10:07:29.394194+0000","last_fullsized":"2023-10-16T10:07:29.394194+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:29.987017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.368225+0000","last_change":"2023-10-16T10:07:29.387904+0000","last_active":"2023-10-16T10:07:30.368225+0000","last_peered":"2023-10-16T10:07:30.368225+0000","last_clean":"2023-10-16T10:07:30.368225+0000","last_became_active":"2023-10-16T10:07:29.387621+0000","last_became_peered":"2023-10-16T10:07:29.387621+0000","last_unstale":"2023-10-16T10:07:30.368225+0000","last_undegraded":"2023-10-16T10:07:30.368225+0000","last_fullsized":"2023-10-16T10:07:30.368225+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:06.419087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.367382+0000","last_change":"2023-10-16T10:07:28.359507+0000","last_active":"2023-10-16T10:07:29.367382+0000","last_peered":"2023-10-16T10:07:29.367382+0000","last_clean":"2023-10-16T10:07:29.367382+0000","last_became_active":"2023-10-16T10:07:28.359170+0000","last_became_peered":"2023-10-16T10:07:28.359170+0000","last_unstale":"2023-10-16T10:07:29.367382+0000","last_undegraded":"2023-10-16T10:07:29.367382+0000","last_fullsized":"2023-10-16T10:07:29.367382+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:17.328970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.367409+0000","last_change":"2023-10-16T10:07:29.367315+0000","last_active":"2023-10-16T10:07:30.367409+0000","last_peered":"2023-10-16T10:07:30.367409+0000","last_clean":"2023-10-16T10:07:30.367409+0000","last_became_active":"2023-10-16T10:07:29.364519+0000","last_became_peered":"2023-10-16T10:07:29.364519+0000","last_unstale":"2023-10-16T10:07:30.367409+0000","last_undegraded":"2023-10-16T10:07:30.367409+0000","last_fullsized":"2023-10-16T10:07:30.367409+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:43:09.145602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.360808+0000","last_change":"2023-10-16T10:07:28.357580+0000","last_active":"2023-10-16T10:07:29.360808+0000","last_peered":"2023-10-16T10:07:29.360808+0000","last_clean":"2023-10-16T10:07:29.360808+0000","last_became_active":"2023-10-16T10:07:28.357220+0000","last_became_peered":"2023-10-16T10:07:28.357220+0000","last_unstale":"2023-10-16T10:07:29.360808+0000","last_undegraded":"2023-10-16T10:07:29.360808+0000","last_fullsized":"2023-10-16T10:07:29.360808+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:25.446281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.371604+0000","last_change":"2023-10-16T10:07:29.373609+0000","last_active":"2023-10-16T10:07:30.371604+0000","last_peered":"2023-10-16T10:07:30.371604+0000","last_clean":"2023-10-16T10:07:30.371604+0000","last_became_active":"2023-10-16T10:07:29.373300+0000","last_became_peered":"2023-10-16T10:07:29.373300+0000","last_unstale":"2023-10-16T10:07:30.371604+0000","last_undegraded":"2023-10-16T10:07:30.371604+0000","last_fullsized":"2023-10-16T10:07:30.371604+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:28.604208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.371278+0000","last_change":"2023-10-16T10:07:28.363956+0000","last_active":"2023-10-16T10:07:29.371278+0000","last_peered":"2023-10-16T10:07:29.371278+0000","last_clean":"2023-10-16T10:07:29.371278+0000","last_became_active":"2023-10-16T10:07:28.363620+0000","last_became_peered":"2023-10-16T10:07:28.363620+0000","last_unstale":"2023-10-16T10:07:29.371278+0000","last_undegraded":"2023-10-16T10:07:29.371278+0000","last_fullsized":"2023-10-16T10:07:29.371278+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:49.373492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.040278+0000","last_change":"2023-10-16T10:07:29.381611+0000","last_active":"2023-10-16T10:07:31.040278+0000","last_peered":"2023-10-16T10:07:31.040278+0000","last_clean":"2023-10-16T10:07:31.040278+0000","last_became_active":"2023-10-16T10:07:29.381353+0000","last_became_peered":"2023-10-16T10:07:29.381353+0000","last_unstale":"2023-10-16T10:07:31.040278+0000","last_undegraded":"2023-10-16T10:07:31.040278+0000","last_fullsized":"2023-10-16T10:07:31.040278+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:09.845125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"123'2","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.390458+0000","last_change":"2023-10-16T10:07:28.367328+0000","last_active":"2023-10-16T10:07:29.390458+0000","last_peered":"2023-10-16T10:07:29.390458+0000","last_clean":"2023-10-16T10:07:29.390458+0000","last_became_active":"2023-10-16T10:07:28.366999+0000","last_became_peered":"2023-10-16T10:07:28.366999+0000","last_unstale":"2023-10-16T10:07:29.390458+0000","last_undegraded":"2023-10-16T10:07:29.390458+0000","last_fullsized":"2023-10-16T10:07:29.390458+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:16:21.555477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.038295+0000","last_change":"2023-10-16T10:07:29.368994+0000","last_active":"2023-10-16T10:07:31.038295+0000","last_peered":"2023-10-16T10:07:31.038295+0000","last_clean":"2023-10-16T10:07:31.038295+0000","last_became_active":"2023-10-16T10:07:29.368397+0000","last_became_peered":"2023-10-16T10:07:29.368397+0000","last_unstale":"2023-10-16T10:07:31.038295+0000","last_undegraded":"2023-10-16T10:07:31.038295+0000","last_fullsized":"2023-10-16T10:07:31.038295+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:45.568635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.375647+0000","last_change":"2023-10-16T10:07:28.365844+0000","last_active":"2023-10-16T10:07:29.375647+0000","last_peered":"2023-10-16T10:07:29.375647+0000","last_clean":"2023-10-16T10:07:29.375647+0000","last_became_active":"2023-10-16T10:07:28.365443+0000","last_became_peered":"2023-10-16T10:07:28.365443+0000","last_unstale":"2023-10-16T10:07:29.375647+0000","last_undegraded":"2023-10-16T10:07:29.375647+0000","last_fullsized":"2023-10-16T10:07:29.375647+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:43:08.128286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.038020+0000","last_change":"2023-10-16T10:07:29.379737+0000","last_active":"2023-10-16T10:07:31.038020+0000","last_peered":"2023-10-16T10:07:31.038020+0000","last_clean":"2023-10-16T10:07:31.038020+0000","last_became_active":"2023-10-16T10:07:29.379212+0000","last_became_peered":"2023-10-16T10:07:29.379212+0000","last_unstale":"2023-10-16T10:07:31.038020+0000","last_undegraded":"2023-10-16T10:07:31.038020+0000","last_fullsized":"2023-10-16T10:07:31.038020+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:23:11.152599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.376102+0000","last_change":"2023-10-16T10:07:28.360063+0000","last_active":"2023-10-16T10:07:29.376102+0000","last_peered":"2023-10-16T10:07:29.376102+0000","last_clean":"2023-10-16T10:07:29.376102+0000","last_became_active":"2023-10-16T10:07:28.359469+0000","last_became_peered":"2023-10-16T10:07:28.359469+0000","last_unstale":"2023-10-16T10:07:29.376102+0000","last_undegraded":"2023-10-16T10:07:29.376102+0000","last_fullsized":"2023-10-16T10:07:29.376102+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:46.851587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.372375+0000","last_change":"2023-10-16T10:07:29.382896+0000","last_active":"2023-10-16T10:07:30.372375+0000","last_peered":"2023-10-16T10:07:30.372375+0000","last_clean":"2023-10-16T10:07:30.372375+0000","last_became_active":"2023-10-16T10:07:29.382613+0000","last_became_peered":"2023-10-16T10:07:29.382613+0000","last_unstale":"2023-10-16T10:07:30.372375+0000","last_undegraded":"2023-10-16T10:07:30.372375+0000","last_fullsized":"2023-10-16T10:07:30.372375+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:00:14.321059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.375014+0000","last_change":"2023-10-16T10:07:28.361287+0000","last_active":"2023-10-16T10:07:29.375014+0000","last_peered":"2023-10-16T10:07:29.375014+0000","last_clean":"2023-10-16T10:07:29.375014+0000","last_became_active":"2023-10-16T10:07:28.360220+0000","last_became_peered":"2023-10-16T10:07:28.360220+0000","last_unstale":"2023-10-16T10:07:29.375014+0000","last_undegraded":"2023-10-16T10:07:29.375014+0000","last_fullsized":"2023-10-16T10:07:29.375014+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:43:08.128286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.368062+0000","last_change":"2023-10-16T10:07:29.359015+0000","last_active":"2023-10-16T10:07:30.368062+0000","last_peered":"2023-10-16T10:07:30.368062+0000","last_clean":"2023-10-16T10:07:30.368062+0000","last_became_active":"2023-10-16T10:07:29.358652+0000","last_became_peered":"2023-10-16T10:07:29.358652+0000","last_unstale":"2023-10-16T10:07:30.368062+0000","last_undegraded":"2023-10-16T10:07:30.368062+0000","last_fullsized":"2023-10-16T10:07:30.368062+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:00:14.321059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.375705+0000","last_change":"2023-10-16T10:07:28.364313+0000","last_active":"2023-10-16T10:07:29.375705+0000","last_peered":"2023-10-16T10:07:29.375705+0000","last_clean":"2023-10-16T10:07:29.375705+0000","last_became_active":"2023-10-16T10:07:28.363948+0000","last_became_peered":"2023-10-16T10:07:28.363948+0000","last_unstale":"2023-10-16T10:07:29.375705+0000","last_undegraded":"2023-10-16T10:07:29.375705+0000","last_fullsized":"2023-10-16T10:07:29.375705+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:04:13.458192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:31.035832+0000","last_change":"2023-10-16T10:07:29.379290+0000","last_active":"2023-10-16T10:07:31.035832+0000","last_peered":"2023-10-16T10:07:31.035832+0000","last_clean":"2023-10-16T10:07:31.035832+0000","last_became_active":"2023-10-16T10:07:29.378793+0000","last_became_peered":"2023-10-16T10:07:29.378793+0000","last_unstale":"2023-10-16T10:07:31.035832+0000","last_undegraded":"2023-10-16T10:07:31.035832+0000","last_fullsized":"2023-10-16T10:07:31.035832+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:18:14.725928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.365939+0000","last_change":"2023-10-16T10:07:28.357283+0000","last_active":"2023-10-16T10:07:29.365939+0000","last_peered":"2023-10-16T10:07:29.365939+0000","last_clean":"2023-10-16T10:07:29.365939+0000","last_became_active":"2023-10-16T10:07:28.356946+0000","last_became_peered":"2023-10-16T10:07:28.356946+0000","last_unstale":"2023-10-16T10:07:29.365939+0000","last_undegraded":"2023-10-16T10:07:29.365939+0000","last_fullsized":"2023-10-16T10:07:29.365939+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:29.987017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.367607+0000","last_change":"2023-10-16T10:07:29.358961+0000","last_active":"2023-10-16T10:07:30.367607+0000","last_peered":"2023-10-16T10:07:30.367607+0000","last_clean":"2023-10-16T10:07:30.367607+0000","last_became_active":"2023-10-16T10:07:29.358547+0000","last_became_peered":"2023-10-16T10:07:29.358547+0000","last_unstale":"2023-10-16T10:07:30.367607+0000","last_undegraded":"2023-10-16T10:07:30.367607+0000","last_fullsized":"2023-10-16T10:07:30.367607+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:04:14.475508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.352520+0000","last_change":"2023-10-16T10:07:28.355774+0000","last_active":"2023-10-16T10:07:29.352520+0000","last_peered":"2023-10-16T10:07:29.352520+0000","last_clean":"2023-10-16T10:07:29.352520+0000","last_became_active":"2023-10-16T10:07:28.355485+0000","last_became_peered":"2023-10-16T10:07:28.355485+0000","last_unstale":"2023-10-16T10:07:29.352520+0000","last_undegraded":"2023-10-16T10:07:29.352520+0000","last_fullsized":"2023-10-16T10:07:29.352520+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:25:38.877788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.371601+0000","last_change":"2023-10-16T10:07:29.367503+0000","last_active":"2023-10-16T10:07:30.371601+0000","last_peered":"2023-10-16T10:07:30.371601+0000","last_clean":"2023-10-16T10:07:30.371601+0000","last_became_active":"2023-10-16T10:07:29.367190+0000","last_became_peered":"2023-10-16T10:07:29.367190+0000","last_unstale":"2023-10-16T10:07:30.371601+0000","last_undegraded":"2023-10-16T10:07:30.371601+0000","last_fullsized":"2023-10-16T10:07:30.371601+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:47.868903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.343361+0000","last_change":"2023-10-16T10:07:28.367204+0000","last_active":"2023-10-16T10:07:29.343361+0000","last_peered":"2023-10-16T10:07:29.343361+0000","last_clean":"2023-10-16T10:07:29.343361+0000","last_became_active":"2023-10-16T10:07:28.366430+0000","last_became_peered":"2023-10-16T10:07:28.366430+0000","last_unstale":"2023-10-16T10:07:29.343361+0000","last_undegraded":"2023-10-16T10:07:29.343361+0000","last_fullsized":"2023-10-16T10:07:29.343361+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:01:20.636726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-16T10:07:30.368306+0000","last_change":"2023-10-16T10:07:29.365302+0000","last_active":"2023-10-16T10:07:30.368306+0000","last_peered":"2023-10-16T10:07:30.368306+0000","last_clean":"2023-10-16T10:07:30.368306+0000","last_became_active":"2023-10-16T10:07:29.364119+0000","last_became_peered":"2023-10-16T10:07:29.364119+0000","last_unstale":"2023-10-16T10:07:30.368306+0000","last_undegraded":"2023-10-16T10:07:30.368306+0000","last_fullsized":"2023-10-16T10:07:30.368306+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:28.320338+0000","last_clean_scrub_stamp":"2023-10-16T10:07:28.320338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:33:28.020956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"123'1","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.389574+0000","last_change":"2023-10-16T10:07:28.370898+0000","last_active":"2023-10-16T10:07:29.389574+0000","last_peered":"2023-10-16T10:07:29.389574+0000","last_clean":"2023-10-16T10:07:29.389574+0000","last_became_active":"2023-10-16T10:07:28.370566+0000","last_became_peered":"2023-10-16T10:07:28.370566+0000","last_unstale":"2023-10-16T10:07:29.389574+0000","last_undegraded":"2023-10-16T10:07:29.389574+0000","last_fullsized":"2023-10-16T10:07:29.389574+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:07:27.303022+0000","last_clean_scrub_stamp":"2023-10-16T10:07:27.303022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:27.011207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"113'2624","reported_seq":3029,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-16T10:07:29.361123+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T10:07:29.361123+0000","last_peered":"2023-10-16T10:07:29.361123+0000","last_clean":"2023-10-16T10:07:29.361123+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T10:07:29.361123+0000","last_undegraded":"2023-10-16T10:07:29.361123+0000","last_fullsized":"2023-10-16T10:07:29.361123+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2624,"log_dups_size":0,"ondisk_log_size":2624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":928,"num_read_kb":1790,"num_write":321,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":928,"num_read_kb":1790,"num_write":321,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2624,"ondisk_log_size":2624,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575069,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2134016,"data_stored":1321946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1699999999999999,"15min":1.171},"min":{"1min":0.81200000000000006,"5min":0.65000000000000002,"15min":0.54400000000000004},"max":{"1min":1.6160000000000001,"5min":1.7310000000000001,"15min":1.925},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.27,"5min":1.159,"15min":1.1899999999999999},"min":{"1min":0.90400000000000003,"5min":0.58099999999999996,"15min":0.33600000000000002},"max":{"1min":1.702,"5min":1.7370000000000001,"15min":2.016},"last":1.1259999999999999}]},{"osd":1,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1759999999999999,"15min":1.2210000000000001},"min":{"1min":0.72899999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.48,"5min":1.788,"15min":1.923},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.161,"15min":1.2090000000000001},"min":{"1min":0.746,"5min":0.67000000000000004,"15min":0.60999999999999999},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.77500000000000002}]},{"osd":2,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.21,"15min":1.24},"min":{"1min":0.63900000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8109999999999999,"5min":1.837,"15min":1.8740000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2629999999999999,"15min":1.288},"min":{"1min":0.89400000000000002,"5min":0.65900000000000003,"15min":0.64500000000000002},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8620000000000001},"last":0.89400000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3069999999999999,"15min":1.339},"min":{"1min":1.054,"5min":0.747,"15min":0.67700000000000005},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3280000000000001,"15min":1.3720000000000001},"min":{"1min":0.97699999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6799999999999999,"5min":1.8540000000000001,"15min":2.004},"last":1.0389999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3859999999999999,"15min":1.367},"min":{"1min":0.86499999999999999,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6970000000000001,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.3140000000000001,"15min":1.3129999999999999},"min":{"1min":1.0009999999999999,"5min":0.63200000000000001,"15min":0.502},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.361}]},{"osd":5,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.5069999999999999,"15min":1.52},"min":{"1min":1.1879999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.9330000000000001,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.411},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4199999999999999,"15min":1.4279999999999999},"min":{"1min":1.1910000000000001,"5min":0.72199999999999998,"15min":0.45400000000000001},"max":{"1min":1.6699999999999999,"5min":1.9630000000000001,"15min":2.2170000000000001},"last":1.452}]},{"osd":6,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.419,"15min":1.4039999999999999},"min":{"1min":0.97599999999999998,"5min":0.82899999999999996,"15min":0.54100000000000004},"max":{"1min":2.0899999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.631,"15min":1.617},"min":{"1min":1.212,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.9279999999999999,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.548}]}]},{"osd":6,"up_from":13,"seq":55834575069,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36844,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334996,"statfs":{"total":96636764160,"available":96599035904,"internally_reserved":0,"allocated":1540096,"data_stored":732118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.1439999999999999,"15min":1.1499999999999999},"min":{"1min":0.68000000000000005,"5min":0.61599999999999999,"15min":0.61099999999999999},"max":{"1min":1.736,"5min":1.736,"15min":1.7689999999999999},"last":1.298},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2430000000000001,"15min":1.24},"min":{"1min":0.71199999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.6899999999999999,"5min":1.7010000000000001,"15min":1.847},"last":1.5329999999999999}]},{"osd":1,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.153,"15min":1.173},"min":{"1min":0.84599999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6970000000000001},"last":1.7789999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.2949999999999999,"15min":1.298},"min":{"1min":0.72599999999999998,"5min":0.69099999999999995,"15min":0.66300000000000003},"max":{"1min":1.7190000000000001,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.6539999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2090000000000001,"15min":1.216},"min":{"1min":0.67800000000000005,"5min":0.66600000000000004,"15min":0.45100000000000001},"max":{"1min":1.748,"5min":1.851,"15min":2.036},"last":1.026},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.29,"15min":1.304},"min":{"1min":0.96899999999999997,"5min":0.79900000000000004,"15min":0.58999999999999997},"max":{"1min":1.794,"5min":1.925,"15min":1.925},"last":1.5800000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.177,"15min":1.2090000000000001},"min":{"1min":0.78800000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5069999999999999,"5min":1.742,"15min":2.8239999999999998},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.1899999999999999,"15min":1.232},"min":{"1min":0.78900000000000003,"5min":0.67400000000000004,"15min":0.63900000000000001},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.865},"last":1.131}]},{"osd":4,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2490000000000001,"15min":1.2629999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.46000000000000002},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":2.1629999999999998},"last":1.9690000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.4350000000000001,"15min":1.429},"min":{"1min":0.81100000000000005,"5min":0.77700000000000002,"15min":0.56599999999999995},"max":{"1min":2,"5min":2,"15min":3.6880000000000002},"last":1.8520000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3009999999999999,"15min":1.3149999999999999},"min":{"1min":0.88,"5min":0.80400000000000005,"15min":0.38900000000000001},"max":{"1min":1.6559999999999999,"5min":2.0790000000000002,"15min":2.5649999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.5029999999999999,"15min":1.5089999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.495},"max":{"1min":1.796,"5min":1.996,"15min":2.3849999999999998},"last":1.7190000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":1.028,"5min":0.79800000000000004,"15min":0.46600000000000003},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":3.5739999999999998},"last":2.0070000000000001},{"interface":"front","average":{"1min":1.53,"5min":1.5049999999999999,"15min":1.4910000000000001},"min":{"1min":0.96299999999999997,"5min":0.92600000000000005,"15min":0.65300000000000002},"max":{"1min":1.917,"5min":2.2000000000000002,"15min":2.4249999999999998},"last":1.4830000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575069,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36840,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335000,"statfs":{"total":96636764160,"available":96599040000,"internally_reserved":0,"allocated":1536000,"data_stored":731536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.179,"15min":1.204},"min":{"1min":0.65600000000000003,"5min":0.57799999999999996,"15min":0.496},"max":{"1min":1.6279999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.151},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.236,"15min":1.2849999999999999},"min":{"1min":0.72599999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.4219999999999999,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.6830000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.119,"15min":1.1499999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.647,"5min":1.7310000000000001,"15min":1.7729999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.139,"15min":1.1439999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.478,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.345}]},{"osd":2,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1339999999999999,"15min":1.155},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5700000000000001,"5min":1.738,"15min":1.77},"last":1.7709999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.21,"15min":1.232},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.645,"5min":1.6739999999999999,"15min":1.8109999999999999},"last":1.4159999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2609999999999999,"15min":1.27},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.76000000000000001},"max":{"1min":1.8260000000000001,"5min":1.9530000000000001,"15min":1.97},"last":1.044},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3149999999999999,"15min":1.347},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6100000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.232}]},{"osd":4,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.278,"15min":1.2689999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.73499999999999999},"max":{"1min":1.6850000000000001,"5min":1.984,"15min":1.984},"last":1.476},{"interface":"front","average":{"1min":1.444,"5min":1.51,"15min":1.502},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.7470000000000001,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.3660000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3360000000000001,"15min":1.3540000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.722,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.381,"15min":1.3799999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.613,"5min":1.986,"15min":1.986},"last":1.2789999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.464,"15min":1.47},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.9219999999999999,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":1.571},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.5489999999999999,"15min":1.5620000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.788,"5min":2.234,"15min":2.234},"last":1.5089999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575070,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46588,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325252,"statfs":{"total":96636764160,"available":96589058048,"internally_reserved":0,"allocated":1552384,"data_stored":733202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.079,"15min":1.141},"min":{"1min":0.64300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.5349999999999999,"5min":1.7609999999999999,"15min":1.827},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3220000000000001,"15min":1.2689999999999999},"min":{"1min":0.89600000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.7070000000000001,"5min":11.974,"15min":11.974},"last":1.4450000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3260000000000001,"15min":1.2989999999999999},"min":{"1min":0.63900000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.7969999999999999,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.2010000000000001,"15min":1.226},"min":{"1min":0.70699999999999996,"5min":0.63600000000000001,"15min":0.624},"max":{"1min":1.5800000000000001,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.24}]},{"osd":2,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.246,"15min":1.248},"min":{"1min":0.78200000000000003,"5min":0.61799999999999999,"15min":0.59399999999999997},"max":{"1min":1.712,"5min":1.897,"15min":1.897},"last":1.155},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3540000000000001,"15min":1.383},"min":{"1min":0.81499999999999995,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6160000000000001,"5min":3.673,"15min":5.0549999999999997},"last":1.1850000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3200000000000001,"15min":1.3},"min":{"1min":0.94399999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.625,"5min":6.8079999999999998,"15min":6.8079999999999998},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.327,"5min":1.3620000000000001,"15min":1.3779999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.67700000000000005},"max":{"1min":1.768,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.70599999999999996}]},{"osd":5,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.496,"15min":1.4350000000000001},"min":{"1min":0.77400000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9990000000000001,"5min":8.702,"15min":8.702},"last":1.343},{"interface":"front","average":{"1min":1.337,"5min":1.383,"15min":1.3720000000000001},"min":{"1min":0.93799999999999994,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.331,"5min":2.331,"15min":3.0430000000000001},"last":1.1120000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4850000000000001,"15min":1.417},"min":{"1min":0.95399999999999996,"5min":0.84699999999999998,"15min":0.79000000000000004},"max":{"1min":1.6990000000000001,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":1.048},{"interface":"front","average":{"1min":1.496,"5min":1.585,"15min":1.556},"min":{"1min":0.88900000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.891,"5min":4.625,"15min":6.4089999999999998},"last":1.52}]},{"osd":7,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.5189999999999999,"15min":1.492},"min":{"1min":1.167,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.927,"5min":2.4060000000000001,"15min":2.9399999999999999},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.579,"15min":1.5700000000000001},"min":{"1min":1.2589999999999999,"5min":1.071,"15min":0.96799999999999997},"max":{"1min":1.9139999999999999,"5min":2.2690000000000001,"15min":2.6120000000000001},"last":1.3759999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575069,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36852,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334988,"statfs":{"total":96636764160,"available":96599027712,"internally_reserved":0,"allocated":1548288,"data_stored":733100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:06:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.81299999999999994,"15min":0.85399999999999998},"min":{"1min":0.318,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1060000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.89600000000000002,"15min":1.002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.708},"last":0.80200000000000005}]},{"osd":1,"last update":"Mon Oct 16 10:06:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.84399999999999997,"15min":0.93600000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.5800000000000001,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.94899999999999995,"15min":1.0489999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4950000000000001,"5min":3.363,"15min":3.363},"last":0.38300000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:06:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.94899999999999995,"15min":1.0569999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.3400000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0600000000000001,"15min":1.1679999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.734,"5min":4.1230000000000002,"15min":4.1230000000000002},"last":0.51900000000000002}]},{"osd":4,"last update":"Mon Oct 16 10:06:33 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2090000000000001,"15min":1.272},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.637,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2729999999999999,"15min":1.3240000000000001},"min":{"1min":0.74199999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.599,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.56499999999999995}]},{"osd":5,"last update":"Mon Oct 16 10:06:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.262,"15min":1.3160000000000001},"min":{"1min":0.75600000000000001,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.667,"5min":4.1829999999999998,"15min":4.1829999999999998},"last":0.497},{"interface":"front","average":{"1min":1.242,"5min":1.258,"15min":1.335},"min":{"1min":0.73799999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.9199999999999999,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.89900000000000002}]},{"osd":6,"last update":"Mon Oct 16 10:06:33 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.3520000000000001,"15min":1.423},"min":{"1min":0.81100000000000005,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.7789999999999999,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.399,"5min":1.347,"15min":1.4330000000000001},"min":{"1min":0.81599999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.0339999999999998,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.93899999999999995}]},{"osd":7,"last update":"Mon Oct 16 10:06:33 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.3779999999999999,"15min":1.4399999999999999},"min":{"1min":1,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":2.1000000000000001,"5min":3.649,"15min":3.649},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.506,"5min":1.4750000000000001,"15min":1.5569999999999999},"min":{"1min":0.84599999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0430000000000001,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":0.63400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575068,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41008,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330832,"statfs":{"total":96636764160,"available":96594771968,"internally_reserved":0,"allocated":1536000,"data_stored":731500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.89300000000000002,"15min":0.877},"min":{"1min":0.35299999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.2749999999999999,"5min":5.1829999999999998,"15min":5.1829999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.90600000000000003,"15min":1.0229999999999999},"min":{"1min":0.35999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.6479999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.94999999999999996}]},{"osd":1,"last update":"Mon Oct 16 10:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0169999999999999,"15min":1.042},"min":{"1min":0.51900000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.5900000000000001,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":1.5029999999999999},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.96499999999999997,"15min":1.1020000000000001},"min":{"1min":0.48899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3979999999999999,"5min":1.792,"15min":1.792},"last":1.0880000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.92700000000000005,"15min":1.004},"min":{"1min":0.48499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.3160000000000001,"5min":1.776,"15min":1.776},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.147,"15min":1.1950000000000001},"min":{"1min":0.47499999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.841,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":1.3380000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:06:37 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.284,"15min":1.341},"min":{"1min":0.79900000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.71,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.548},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2709999999999999,"15min":1.304},"min":{"1min":0.89200000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.6459999999999999,"5min":3.262,"15min":3.262},"last":1.5720000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:06:37 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3560000000000001,"15min":1.373},"min":{"1min":0.79700000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.0259999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.3069999999999999,"15min":1.397},"min":{"1min":0.76200000000000001,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.573,"5min":1.885,"15min":1.956},"last":1.2649999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:06:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2889999999999999,"15min":1.3420000000000001},"min":{"1min":0.84299999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.716,"5min":3.169,"15min":3.169},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.417,"5min":1.415,"15min":1.4399999999999999},"min":{"1min":0.90100000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.2549999999999999,"5min":3.8340000000000001,"15min":3.8340000000000001},"last":1.7749999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:06:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.391,"15min":1.466},"min":{"1min":0.86499999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6870000000000001,"5min":2.0739999999999998,"15min":2.262},"last":1.472},{"interface":"front","average":{"1min":1.359,"5min":1.3600000000000001,"15min":1.429},"min":{"1min":0.84199999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.776,"5min":3.3780000000000001,"15min":3.3780000000000001},"last":1.7410000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575068,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54892,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316948,"statfs":{"total":96636764160,"available":96580554752,"internally_reserved":0,"allocated":1531904,"data_stored":731466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.79500000000000004,"15min":0.85599999999999998},"min":{"1min":0.372,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.196,"5min":1.4690000000000001,"15min":1.631},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.94499999999999995,"15min":0.98899999999999999},"min":{"1min":0.48699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.262,"5min":1.798,"15min":1.798},"last":0.51700000000000002}]},{"osd":2,"last update":"Mon Oct 16 10:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.93200000000000005,"15min":0.97699999999999998},"min":{"1min":0.44800000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.476,"5min":1.5069999999999999,"15min":1.673},"last":0.58499999999999996},{"interface":"front","average":{"1min":1.159,"5min":1.119,"15min":1.214},"min":{"1min":0.505,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.591,"5min":1.609,"15min":1.7669999999999999},"last":0.59699999999999998}]},{"osd":3,"last update":"Mon Oct 16 10:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.02,"15min":1.113},"min":{"1min":0.42999999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5429999999999999,"5min":1.798,"15min":1.798},"last":0.53800000000000003},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0269999999999999,"15min":1.1020000000000001},"min":{"1min":0.52800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5089999999999999,"5min":1.6879999999999999,"15min":1.8879999999999999},"last":0.70699999999999996}]},{"osd":4,"last update":"Mon Oct 16 10:07:22 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.159,"15min":1.2190000000000001},"min":{"1min":0.61299999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4610000000000001,"5min":1.7450000000000001,"15min":2.0259999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.2789999999999999,"15min":1.351},"min":{"1min":0.79200000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.823,"5min":1.8680000000000001,"15min":2.1320000000000001},"last":0.75700000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:07:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.2569999999999999,"15min":1.3240000000000001},"min":{"1min":0.82399999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6220000000000001,"5min":1.7310000000000001,"15min":1.9359999999999999},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.246,"15min":1.3149999999999999},"min":{"1min":0.73099999999999998,"5min":0.67200000000000004,"15min":0.60499999999999998},"max":{"1min":1.643,"5min":1.7869999999999999,"15min":2.0470000000000002},"last":0.84199999999999997}]},{"osd":6,"last update":"Mon Oct 16 10:07:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.325,"15min":1.387},"min":{"1min":0.76900000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.7230000000000001,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.302,"5min":1.321,"15min":1.3799999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6180000000000001,"5min":1.8779999999999999,"15min":2.1680000000000001},"last":0.98299999999999998}]},{"osd":7,"last update":"Mon Oct 16 10:07:22 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.355,"15min":1.448},"min":{"1min":0.81299999999999994,"5min":0.62,"15min":0.62},"max":{"1min":1.7330000000000001,"5min":2.0739999999999998,"15min":2.1699999999999999},"last":1.046},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.427,"15min":1.4850000000000001},"min":{"1min":1.0369999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.798,"5min":2.1829999999999998,"15min":2.1960000000000002},"last":1.0700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575069,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47148,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324692,"statfs":{"total":96636764160,"available":96588484608,"internally_reserved":0,"allocated":2129920,"data_stored":1322416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.83399999999999996,"15min":0.85499999999999998},"min":{"1min":0.51700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1819999999999999,"5min":1.7589999999999999,"15min":2.766},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.093,"5min":0.95999999999999996,"15min":1.03},"min":{"1min":0.45600000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.51,"5min":1.581,"15min":3.96},"last":1.01}]},{"osd":2,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":0.999,"15min":1.0409999999999999},"min":{"1min":0.60699999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.5089999999999999,"5min":2.0449999999999999,"15min":2.9260000000000002},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.024,"15min":1.075},"min":{"1min":0.49299999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":4.4290000000000003},"last":1.21}]},{"osd":3,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.94599999999999995,"15min":1.022},"min":{"1min":0.44400000000000001,"5min":0.41999999999999998,"15min":0.38},"max":{"1min":1.4370000000000001,"5min":1.5920000000000001,"15min":4.7380000000000004},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.226,"5min":1.151,"15min":1.202},"min":{"1min":0.80400000000000005,"5min":0.44,"15min":0.44},"max":{"1min":1.6579999999999999,"5min":2.1859999999999999,"15min":5.1500000000000004},"last":1.1160000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.296,"15min":1.331},"min":{"1min":0.78300000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.696,"5min":2.1749999999999998,"15min":4.1970000000000001},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.224,"15min":1.2889999999999999},"min":{"1min":0.76300000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6080000000000001,"5min":1.766,"15min":4.9630000000000001},"last":1.3999999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2410000000000001,"15min":1.306},"min":{"1min":0.872,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.657,"5min":1.7070000000000001,"15min":4.8600000000000003},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3500000000000001,"15min":1.3979999999999999},"min":{"1min":0.93899999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.639,"5min":2.1269999999999998,"15min":4.6779999999999999},"last":1.2470000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.4330000000000001,"15min":1.448},"min":{"1min":1.3149999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.8100000000000001,"5min":2.1030000000000002,"15min":3.8250000000000002},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.377,"15min":1.427},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7669999999999999,"5min":2.117,"15min":4.0979999999999999},"last":1.333}]},{"osd":7,"last update":"Mon Oct 16 10:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.452,"15min":1.4850000000000001},"min":{"1min":0.63300000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.756,"5min":2.0750000000000002,"15min":4.3369999999999997},"last":1.484},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.353,"15min":1.429},"min":{"1min":0.79700000000000004,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":5.0570000000000004},"last":1.4590000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:07:34.253 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T10:07:34.254 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T10:07:34.497 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:07:34.503 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:07:34.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.515+0000 7fd6c2583640 1 Processor -- start 2023-10-16T10:07:34.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.515+0000 7fd6c2583640 1 -- start start 2023-10-16T10:07:34.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.516+0000 7fd6c2583640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd6bc0b3070 0x7fd6bc0cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:34.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.516+0000 7fd6c2583640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd6bc0d1b10 0x7fd6bc0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:34.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.516+0000 7fd6c2583640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6bc0b3940 con 0x7fd6bc0b3e70 2023-10-16T10:07:34.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.516+0000 7fd6c2583640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6bc0b45e0 con 0x7fd6bc0d1b10 2023-10-16T10:07:34.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.516+0000 7fd6c2583640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6bc0b3b90 con 0x7fd6bc0b3070 2023-10-16T10:07:34.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.517+0000 7fd6b3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd6bc0b3e70 0x7fd6bc0c6870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37842/0 (socket says 172.21.15.84:37842) 2023-10-16T10:07:34.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.517+0000 7fd6b3fff640 1 -- 172.21.15.84:0/1120096219 learned_addr learned my addr 172.21.15.84:0/1120096219 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:34.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.517+0000 7fd6c0b20640 1 --2- 172.21.15.84:0/1120096219 >> v2:172.21.15.84:3300/0 conn(0x7fd6bc0d1b10 0x7fd6bc0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:34.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.517+0000 7fd6bbfff640 1 --2- 172.21.15.84:0/1120096219 >> v2:172.21.15.157:3300/0 conn(0x7fd6bc0b3070 0x7fd6bc0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:34.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.517+0000 7fd6bb7fe640 1 -- 172.21.15.84:0/1120096219 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2748895890 0 0) 0x7fd6bc0b3940 con 0x7fd6bc0b3e70 2023-10-16T10:07:34.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.517+0000 7fd6bb7fe640 1 -- 172.21.15.84:0/1120096219 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6a8003520 con 0x7fd6bc0b3e70 2023-10-16T10:07:34.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.517+0000 7fd6c0b20640 1 -- 172.21.15.84:0/1120096219 >> v2:172.21.15.157:3300/0 conn(0x7fd6bc0b3070 msgr2=0x7fd6bc0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:34.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.517+0000 7fd6c0b20640 1 --2- 172.21.15.84:0/1120096219 >> v2:172.21.15.157:3300/0 conn(0x7fd6bc0b3070 0x7fd6bc0cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.517+0000 7fd6c0b20640 1 -- 172.21.15.84:0/1120096219 >> v1:172.21.15.84:6789/0 conn(0x7fd6bc0b3e70 legacy=0x7fd6bc0c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:34.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.517+0000 7fd6c0b20640 1 -- 172.21.15.84:0/1120096219 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6bc0d3840 con 0x7fd6bc0d1b10 2023-10-16T10:07:34.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.518+0000 7fd6c0b20640 1 --2- 172.21.15.84:0/1120096219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6bc0d1b10 0x7fd6bc0cfa40 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc08ba60 tx=0x7fd6b402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a299e81dc24b411 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:34.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.519+0000 7fd6bb7fe640 1 -- 172.21.15.84:0/1120096219 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b402cc70 con 0x7fd6bc0d1b10 2023-10-16T10:07:34.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.519+0000 7fd6bb7fe640 1 -- 172.21.15.84:0/1120096219 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b402cdd0 con 0x7fd6bc0d1b10 2023-10-16T10:07:34.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.519+0000 7fd6bb7fe640 1 -- 172.21.15.84:0/1120096219 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4034660 con 0x7fd6bc0d1b10 2023-10-16T10:07:34.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.519+0000 7fd6c2583640 1 -- 172.21.15.84:0/1120096219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6bc0d1b10 msgr2=0x7fd6bc0cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:34.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.519+0000 7fd6c2583640 1 --2- 172.21.15.84:0/1120096219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6bc0d1b10 0x7fd6bc0cfa40 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc08ba60 tx=0x7fd6b402bec0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.519+0000 7fd6c2583640 1 -- 172.21.15.84:0/1120096219 shutdown_connections 2023-10-16T10:07:34.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.519+0000 7fd6c2583640 1 --2- 172.21.15.84:0/1120096219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6bc0d1b10 0x7fd6bc0cfa40 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.519+0000 7fd6c2583640 1 --2- 172.21.15.84:0/1120096219 >> v2:172.21.15.157:3300/0 conn(0x7fd6bc0b3070 0x7fd6bc0cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.519+0000 7fd6c2583640 1 -- 172.21.15.84:0/1120096219 >> 172.21.15.84:0/1120096219 conn(0x7fd6bc0d2d20 msgr2=0x7fd6bc0d3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:34.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.520+0000 7fd6c2583640 1 -- 172.21.15.84:0/1120096219 shutdown_connections 2023-10-16T10:07:34.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.520+0000 7fd6c2583640 1 -- 172.21.15.84:0/1120096219 wait complete. 2023-10-16T10:07:34.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.521+0000 7fd6c2583640 1 Processor -- start 2023-10-16T10:07:34.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.521+0000 7fd6c2583640 1 -- start start 2023-10-16T10:07:34.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.522+0000 7fd6c2583640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6bc0b3070 0x7fd6bc0caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:34.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.522+0000 7fd6c2583640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6bc0b3e70 0x7fd6bc0b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:34.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.522+0000 7fd6c2583640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6bc0d1b10 0x7fd6bc160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:34.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.522+0000 7fd6c2583640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc0b6fb0 con 0x7fd6bc0b3070 2023-10-16T10:07:34.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.522+0000 7fd6c2583640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc0cb0b0 con 0x7fd6bc0b3e70 2023-10-16T10:07:34.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.522+0000 7fd6c2583640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc0cb390 con 0x7fd6bc0d1b10 2023-10-16T10:07:34.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.523+0000 7fd6bbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6bc0b3070 0x7fd6bc0caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:34.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.523+0000 7fd6bbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6bc0b3070 0x7fd6bc0caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52202/0 (socket says 172.21.15.84:52202) 2023-10-16T10:07:34.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.523+0000 7fd6bbfff640 1 -- 172.21.15.84:0/3535800906 learned_addr learned my addr 172.21.15.84:0/3535800906 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:34.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.523+0000 7fd6c0b20640 1 --2- 172.21.15.84:0/3535800906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6bc0d1b10 0x7fd6bc160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:34.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.523+0000 7fd6b3fff640 1 --2- 172.21.15.84:0/3535800906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6bc0b3e70 0x7fd6bc0b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:34.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.523+0000 7fd6bbfff640 1 -- 172.21.15.84:0/3535800906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6bc0d1b10 msgr2=0x7fd6bc160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:34.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.523+0000 7fd6bbfff640 1 --2- 172.21.15.84:0/3535800906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6bc0d1b10 0x7fd6bc160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.523+0000 7fd6bbfff640 1 -- 172.21.15.84:0/3535800906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6bc0b3e70 msgr2=0x7fd6bc0b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:34.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.523+0000 7fd6bbfff640 1 --2- 172.21.15.84:0/3535800906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6bc0b3e70 0x7fd6bc0b6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.523+0000 7fd6bbfff640 1 -- 172.21.15.84:0/3535800906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b4009a70 con 0x7fd6bc0b3070 2023-10-16T10:07:34.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.523+0000 7fd6c0b20640 1 --2- 172.21.15.84:0/3535800906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6bc0d1b10 0x7fd6bc160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:07:34.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.523+0000 7fd6bbfff640 1 --2- 172.21.15.84:0/3535800906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6bc0b3070 0x7fd6bc0caa20 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00d800 tx=0x7fd6ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:34.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.524+0000 7fd6b97fa640 1 -- 172.21.15.84:0/3535800906 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac00b6c0 con 0x7fd6bc0b3070 2023-10-16T10:07:34.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.524+0000 7fd6b97fa640 1 -- 172.21.15.84:0/3535800906 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ac00b820 con 0x7fd6bc0b3070 2023-10-16T10:07:34.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.524+0000 7fd6b97fa640 1 -- 172.21.15.84:0/3535800906 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac00bbd0 con 0x7fd6bc0b3070 2023-10-16T10:07:34.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.524+0000 7fd6c2583640 1 -- 172.21.15.84:0/3535800906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6bc0cb670 con 0x7fd6bc0b3070 2023-10-16T10:07:34.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.524+0000 7fd6c2583640 1 -- 172.21.15.84:0/3535800906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6bc1e6e40 con 0x7fd6bc0b3070 2023-10-16T10:07:34.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.525+0000 7fd6c2583640 1 -- 172.21.15.84:0/3535800906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6bc04e160 con 0x7fd6bc0b3070 2023-10-16T10:07:34.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.526+0000 7fd6b97fa640 1 -- 172.21.15.84:0/3535800906 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd6ac00bd30 con 0x7fd6bc0b3070 2023-10-16T10:07:34.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.527+0000 7fd6b97fa640 1 --2- 172.21.15.84:0/3535800906 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fd69c06af90 0x7fd69c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:34.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.527+0000 7fd6b97fa640 1 -- 172.21.15.84:0/3535800906 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6ac089d10 con 0x7fd6bc0b3070 2023-10-16T10:07:34.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.527+0000 7fd6b3fff640 1 --2- 172.21.15.84:0/3535800906 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fd69c06af90 0x7fd69c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:34.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.528+0000 7fd6b3fff640 1 --2- 172.21.15.84:0/3535800906 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fd69c06af90 0x7fd69c06d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc1f5140 tx=0x7fd6a4009040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:34.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.532+0000 7fd6b97fa640 1 -- 172.21.15.84:0/3535800906 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd6ac08f020 con 0x7fd6bc0b3070 2023-10-16T10:07:34.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.794+0000 7fd6c2583640 1 -- 172.21.15.84:0/3535800906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd6bc0cb1f0 con 0x7fd6bc0b3070 2023-10-16T10:07:34.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.799+0000 7fd6b97fa640 1 -- 172.21.15.84:0/3535800906 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+147 (secure 0 0 0) 0x7fd6bc0cb1f0 con 0x7fd6bc0b3070 2023-10-16T10:07:34.800 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2023-10-16T10:07:34.801 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ== 2023-10-16T10:07:34.801 INFO:teuthology.orchestra.run.smithi084.stdout: caps mds = "allow" 2023-10-16T10:07:34.801 INFO:teuthology.orchestra.run.smithi084.stdout: caps mon = "allow r" 2023-10-16T10:07:34.801 INFO:teuthology.orchestra.run.smithi084.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T10:07:34.802 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-16T10:07:34.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.801+0000 7fd6c2583640 1 -- 172.21.15.84:0/3535800906 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fd69c06af90 msgr2=0x7fd69c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:34.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.801+0000 7fd6c2583640 1 --2- 172.21.15.84:0/3535800906 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fd69c06af90 0x7fd69c06d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc1f5140 tx=0x7fd6a4009040 comp rx=0 tx=0).stop 2023-10-16T10:07:34.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.801+0000 7fd6c2583640 1 -- 172.21.15.84:0/3535800906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6bc0b3070 msgr2=0x7fd6bc0caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:34.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.801+0000 7fd6c2583640 1 --2- 172.21.15.84:0/3535800906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6bc0b3070 0x7fd6bc0caa20 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00d800 tx=0x7fd6ac00dcc0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.801+0000 7fd6c2583640 1 -- 172.21.15.84:0/3535800906 shutdown_connections 2023-10-16T10:07:34.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.801+0000 7fd6c2583640 1 --2- 172.21.15.84:0/3535800906 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fd69c06af90 0x7fd69c06d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.801+0000 7fd6c2583640 1 --2- 172.21.15.84:0/3535800906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6bc0d1b10 0x7fd6bc160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.801+0000 7fd6c2583640 1 --2- 172.21.15.84:0/3535800906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6bc0b3e70 0x7fd6bc0b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.801+0000 7fd6c2583640 1 --2- 172.21.15.84:0/3535800906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6bc0b3070 0x7fd6bc0caa20 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:34.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.801+0000 7fd6c2583640 1 -- 172.21.15.84:0/3535800906 >> 172.21.15.84:0/3535800906 conn(0x7fd6bc0d2d20 msgr2=0x7fd6bc0c68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:34.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.801+0000 7fd6c2583640 1 -- 172.21.15.84:0/3535800906 shutdown_connections 2023-10-16T10:07:34.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:34.802+0000 7fd6c2583640 1 -- 172.21.15.84:0/3535800906 wait complete. 2023-10-16T10:07:34.821 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:07:35.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.031+0000 7fa1df990640 1 Processor -- start 2023-10-16T10:07:35.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.031+0000 7fa1df990640 1 -- start start 2023-10-16T10:07:35.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.033+0000 7fa1df990640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1d815ae80 0x7fa1d815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.033+0000 7fa1df990640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa1d80ce390 0x7fa1d8070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.033+0000 7fa1df990640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d815ed60 con 0x7fa1d815aaa0 2023-10-16T10:07:35.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.033+0000 7fa1df990640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d806b810 con 0x7fa1d80ce390 2023-10-16T10:07:35.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.033+0000 7fa1df990640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d80707e0 con 0x7fa1d815ae80 2023-10-16T10:07:35.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.033+0000 7fa1ddf2d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa1d80ce390 0x7fa1d8070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.033+0000 7fa1ddf2d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa1d80ce390 0x7fa1d8070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52210/0 (socket says 172.21.15.84:52210) 2023-10-16T10:07:35.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.033+0000 7fa1ddf2d640 1 -- 172.21.15.84:0/2587298833 learned_addr learned my addr 172.21.15.84:0/2587298833 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:35.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.033+0000 7fa1dcf2b640 1 --2- 172.21.15.84:0/2587298833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1d815ae80 0x7fa1d815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.034+0000 7fa1ddf2d640 1 -- 172.21.15.84:0/2587298833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1d815ae80 msgr2=0x7fa1d815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:35.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.034+0000 7fa1ddf2d640 1 --2- 172.21.15.84:0/2587298833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1d815ae80 0x7fa1d815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.034+0000 7fa1ddf2d640 1 -- 172.21.15.84:0/2587298833 >> v1:172.21.15.157:6789/0 conn(0x7fa1d815aaa0 legacy=0x7fa1d80cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:35.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.034+0000 7fa1ddf2d640 1 -- 172.21.15.84:0/2587298833 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d815ef90 con 0x7fa1d80ce390 2023-10-16T10:07:35.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.035+0000 7fa1ddf2d640 1 --2- 172.21.15.84:0/2587298833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1d80ce390 0x7fa1d8070f50 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009e50 tx=0x7fa1d002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fa753b3c666b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:35.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.035+0000 7fa1cffff640 1 -- 172.21.15.84:0/2587298833 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d002caf0 con 0x7fa1d80ce390 2023-10-16T10:07:35.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.035+0000 7fa1cffff640 1 -- 172.21.15.84:0/2587298833 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d002cc50 con 0x7fa1d80ce390 2023-10-16T10:07:35.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.035+0000 7fa1cffff640 1 -- 172.21.15.84:0/2587298833 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d00343f0 con 0x7fa1d80ce390 2023-10-16T10:07:35.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.036+0000 7fa1df990640 1 -- 172.21.15.84:0/2587298833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1d80ce390 msgr2=0x7fa1d8070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:35.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.036+0000 7fa1df990640 1 --2- 172.21.15.84:0/2587298833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1d80ce390 0x7fa1d8070f50 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009e50 tx=0x7fa1d002bf50 comp rx=0 tx=0).stop 2023-10-16T10:07:35.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.036+0000 7fa1df990640 1 -- 172.21.15.84:0/2587298833 shutdown_connections 2023-10-16T10:07:35.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.036+0000 7fa1df990640 1 --2- 172.21.15.84:0/2587298833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1d80ce390 0x7fa1d8070f50 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.036+0000 7fa1df990640 1 --2- 172.21.15.84:0/2587298833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1d815ae80 0x7fa1d815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.036+0000 7fa1df990640 1 -- 172.21.15.84:0/2587298833 >> 172.21.15.84:0/2587298833 conn(0x7fa1d80cb410 msgr2=0x7fa1d80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:35.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.036+0000 7fa1df990640 1 -- 172.21.15.84:0/2587298833 shutdown_connections 2023-10-16T10:07:35.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.036+0000 7fa1df990640 1 -- 172.21.15.84:0/2587298833 wait complete. 2023-10-16T10:07:35.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.037+0000 7fa1df990640 1 Processor -- start 2023-10-16T10:07:35.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.038+0000 7fa1df990640 1 -- start start 2023-10-16T10:07:35.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.039+0000 7fa1df990640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1d80ce390 0x7fa1d80c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.039+0000 7fa1df990640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1d815aaa0 0x7fa1d80ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.039+0000 7fa1df990640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1d815ae80 0x7fa1d80acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.039+0000 7fa1df990640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d80c8600 con 0x7fa1d815aaa0 2023-10-16T10:07:35.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.039+0000 7fa1df990640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d80c5110 con 0x7fa1d815ae80 2023-10-16T10:07:35.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.039+0000 7fa1df990640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d81f60d0 con 0x7fa1d80ce390 2023-10-16T10:07:35.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.039+0000 7fa1dcf2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1d815aaa0 0x7fa1d80ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.039+0000 7fa1dcf2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1d815aaa0 0x7fa1d80ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52226/0 (socket says 172.21.15.84:52226) 2023-10-16T10:07:35.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.039+0000 7fa1dcf2b640 1 -- 172.21.15.84:0/2221703298 learned_addr learned my addr 172.21.15.84:0/2221703298 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:35.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.039+0000 7fa1dd72c640 1 --2- 172.21.15.84:0/2221703298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1d80ce390 0x7fa1d80c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.039+0000 7fa1dcf2b640 1 -- 172.21.15.84:0/2221703298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1d80ce390 msgr2=0x7fa1d80c1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:35.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.039+0000 7fa1ddf2d640 1 --2- 172.21.15.84:0/2221703298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1d815ae80 0x7fa1d80acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.039+0000 7fa1dcf2b640 1 --2- 172.21.15.84:0/2221703298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1d80ce390 0x7fa1d80c1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.039+0000 7fa1dcf2b640 1 -- 172.21.15.84:0/2221703298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1d815ae80 msgr2=0x7fa1d80acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:35.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.039+0000 7fa1dcf2b640 1 --2- 172.21.15.84:0/2221703298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1d815ae80 0x7fa1d80acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.039+0000 7fa1dcf2b640 1 -- 172.21.15.84:0/2221703298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d0009b30 con 0x7fa1d815aaa0 2023-10-16T10:07:35.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.039+0000 7fa1dd72c640 1 --2- 172.21.15.84:0/2221703298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1d80ce390 0x7fa1d80c1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:35.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.040+0000 7fa1dcf2b640 1 --2- 172.21.15.84:0/2221703298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1d815aaa0 0x7fa1d80ac890 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800d800 tx=0x7fa1c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:35.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.040+0000 7fa1cdffb640 1 -- 172.21.15.84:0/2221703298 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c800b6c0 con 0x7fa1d815aaa0 2023-10-16T10:07:35.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.040+0000 7fa1cdffb640 1 -- 172.21.15.84:0/2221703298 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c800b820 con 0x7fa1d815aaa0 2023-10-16T10:07:35.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.040+0000 7fa1cdffb640 1 -- 172.21.15.84:0/2221703298 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c800bbb0 con 0x7fa1d815aaa0 2023-10-16T10:07:35.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.040+0000 7fa1df990640 1 -- 172.21.15.84:0/2221703298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d81f63b0 con 0x7fa1d815aaa0 2023-10-16T10:07:35.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.040+0000 7fa1df990640 1 -- 172.21.15.84:0/2221703298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d80c2190 con 0x7fa1d815aaa0 2023-10-16T10:07:35.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.041+0000 7fa1df990640 1 -- 172.21.15.84:0/2221703298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d81f5f10 con 0x7fa1d815aaa0 2023-10-16T10:07:35.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.042+0000 7fa1cdffb640 1 -- 172.21.15.84:0/2221703298 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa1c800bd10 con 0x7fa1d815aaa0 2023-10-16T10:07:35.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.043+0000 7fa1cdffb640 1 --2- 172.21.15.84:0/2221703298 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fa1bc06af90 0x7fa1bc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.043+0000 7fa1cdffb640 1 -- 172.21.15.84:0/2221703298 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1c8089dd0 con 0x7fa1d815aaa0 2023-10-16T10:07:35.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.043+0000 7fa1dd72c640 1 --2- 172.21.15.84:0/2221703298 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fa1bc06af90 0x7fa1bc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.044+0000 7fa1dd72c640 1 --2- 172.21.15.84:0/2221703298 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fa1bc06af90 0x7fa1bc06d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4000f60 tx=0x7fa1d4009040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:35.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.047+0000 7fa1cdffb640 1 -- 172.21.15.84:0/2221703298 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa1c8054cd0 con 0x7fa1d815aaa0 2023-10-16T10:07:35.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.258+0000 7fa1df990640 1 -- 172.21.15.84:0/2221703298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1d804e160 con 0x7fa1d815aaa0 2023-10-16T10:07:35.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.259+0000 7fa1cdffb640 1 -- 172.21.15.84:0/2221703298 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fa1c80582f0 con 0x7fa1d815aaa0 2023-10-16T10:07:35.261 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:35.261 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2012273152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2012273152},{"type":"v1","addr":"172.21.15.157:6835","nonce":2012273152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/435606514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":435606514},{"type":"v1","addr":"172.21.15.157:6833","nonce":435606514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2286048798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2286048798},{"type":"v1","addr":"172.21.15.84:6836","nonce":2286048798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:07:28.798746+0000","modified":"2023-10-16T10:07:31.857629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2859088731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2859088731},{"type":"v1","addr":"172.21.15.84:6837","nonce":2859088731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-16T10:07:35.262 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 62 2023-10-16T10:07:35.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.261+0000 7fa1df990640 1 -- 172.21.15.84:0/2221703298 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fa1bc06af90 msgr2=0x7fa1bc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:35.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.261+0000 7fa1df990640 1 --2- 172.21.15.84:0/2221703298 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fa1bc06af90 0x7fa1bc06d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4000f60 tx=0x7fa1d4009040 comp rx=0 tx=0).stop 2023-10-16T10:07:35.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.262+0000 7fa1df990640 1 -- 172.21.15.84:0/2221703298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1d815aaa0 msgr2=0x7fa1d80ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:35.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.262+0000 7fa1df990640 1 --2- 172.21.15.84:0/2221703298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1d815aaa0 0x7fa1d80ac890 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800d800 tx=0x7fa1c800dcc0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.262+0000 7fa1df990640 1 -- 172.21.15.84:0/2221703298 shutdown_connections 2023-10-16T10:07:35.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.262+0000 7fa1df990640 1 --2- 172.21.15.84:0/2221703298 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fa1bc06af90 0x7fa1bc06d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.262+0000 7fa1df990640 1 --2- 172.21.15.84:0/2221703298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1d815ae80 0x7fa1d80acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.262+0000 7fa1df990640 1 --2- 172.21.15.84:0/2221703298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1d815aaa0 0x7fa1d80ac890 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.262+0000 7fa1df990640 1 --2- 172.21.15.84:0/2221703298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1d80ce390 0x7fa1d80c1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.262+0000 7fa1df990640 1 -- 172.21.15.84:0/2221703298 >> 172.21.15.84:0/2221703298 conn(0x7fa1d80cb410 msgr2=0x7fa1d80b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:35.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.262+0000 7fa1df990640 1 -- 172.21.15.84:0/2221703298 shutdown_connections 2023-10-16T10:07:35.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.262+0000 7fa1df990640 1 -- 172.21.15.84:0/2221703298 wait complete. 2023-10-16T10:07:35.684 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T10:07:28.798746+0000', 'modified': '2023-10-16T10:07:31.857629+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8040}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8040': {'gid': 8040, 'name': 'c', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.84:6837/2859088731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6835', 'nonce': 2859088731}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 2859088731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T10:07:35.684 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T10:07:35.685 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-16T10:07:35.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.906+0000 7f3f052ad640 1 Processor -- start 2023-10-16T10:07:35.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.907+0000 7f3f052ad640 1 -- start start 2023-10-16T10:07:35.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.908+0000 7f3f052ad640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3f000cded0 0x7f3f000b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.908+0000 7f3f052ad640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0015d150 con 0x7f3f0015cd70 2023-10-16T10:07:35.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.908+0000 7f3f052ad640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f000d1180 con 0x7f3f000b8480 2023-10-16T10:07:35.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.908+0000 7f3f052ad640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f000d1460 con 0x7f3f000cded0 2023-10-16T10:07:35.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.908+0000 7f3efffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3f0015cd70 0x7f3f000d2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:41280/0 (socket says 172.21.15.84:41280) 2023-10-16T10:07:35.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.908+0000 7f3efffff640 1 -- 172.21.15.84:0/2062047059 learned_addr learned my addr 172.21.15.84:0/2062047059 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:35.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.909+0000 7f3efeffd640 1 -- 172.21.15.84:0/2062047059 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3821105985 0 0) 0x7f3f0015d150 con 0x7f3f0015cd70 2023-10-16T10:07:35.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.909+0000 7f3f04aac640 1 --2- 172.21.15.84:0/2062047059 >> v2:172.21.15.157:3300/0 conn(0x7f3f000cded0 0x7f3f000b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.909+0000 7f3efeffd640 1 -- 172.21.15.84:0/2062047059 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3eec003520 con 0x7f3f0015cd70 2023-10-16T10:07:35.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.909+0000 7f3efeffd640 1 -- 172.21.15.84:0/2062047059 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 356680167 0 0) 0x7f3eec003520 con 0x7f3f0015cd70 2023-10-16T10:07:35.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.909+0000 7f3efeffd640 1 -- 172.21.15.84:0/2062047059 >> v2:172.21.15.157:3300/0 conn(0x7f3f000cded0 msgr2=0x7f3f000b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:35.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.909+0000 7f3efeffd640 1 --2- 172.21.15.84:0/2062047059 >> v2:172.21.15.157:3300/0 conn(0x7f3f000cded0 0x7f3f000b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.909+0000 7f3efeffd640 1 -- 172.21.15.84:0/2062047059 >> v1:172.21.15.157:6789/0 conn(0x7f3f000b8480 legacy=0x7f3f000cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:35.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.909+0000 7f3efeffd640 1 -- 172.21.15.84:0/2062047059 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f000d1740 con 0x7f3f0015cd70 2023-10-16T10:07:35.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.909+0000 7f3efeffd640 1 -- 172.21.15.84:0/2062047059 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f3ef00019f0 con 0x7f3f0015cd70 2023-10-16T10:07:35.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.909+0000 7f3efeffd640 1 -- 172.21.15.84:0/2062047059 >> v1:172.21.15.84:6789/0 conn(0x7f3f0015cd70 legacy=0x7f3f000d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:35.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.909+0000 7f3efeffd640 1 --2- 172.21.15.84:0/2062047059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eec003970 0x7f3eec003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.909+0000 7f3efeffd640 1 --2- 172.21.15.84:0/2062047059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec0047c0 0x7f3eec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.909+0000 7f3efeffd640 1 --2- 172.21.15.84:0/2062047059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3eec005180 0x7f3eec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.909+0000 7f3efeffd640 1 -- 172.21.15.84:0/2062047059 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f000d1460 con 0x7f3eec003970 2023-10-16T10:07:35.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.909+0000 7f3efeffd640 1 -- 172.21.15.84:0/2062047059 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec00c0b0 con 0x7f3eec0047c0 2023-10-16T10:07:35.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.909+0000 7f3efeffd640 1 -- 172.21.15.84:0/2062047059 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec00c270 con 0x7f3eec005180 2023-10-16T10:07:35.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.909+0000 7f3efffff640 1 --2- 172.21.15.84:0/2062047059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eec003970 0x7f3eec003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.910+0000 7f3efffff640 1 -- 172.21.15.84:0/2062047059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3eec005180 msgr2=0x7f3eec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:35.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.910+0000 7f3efffff640 1 --2- 172.21.15.84:0/2062047059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3eec005180 0x7f3eec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.910+0000 7f3efffff640 1 -- 172.21.15.84:0/2062047059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec0047c0 msgr2=0x7f3eec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:35.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.910+0000 7f3efffff640 1 --2- 172.21.15.84:0/2062047059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec0047c0 0x7f3eec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.910+0000 7f3efffff640 1 -- 172.21.15.84:0/2062047059 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec003520 con 0x7f3eec003970 2023-10-16T10:07:35.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.910+0000 7f3f04aac640 1 -- 172.21.15.84:0/2062047059 reap_dead start 2023-10-16T10:07:35.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.911+0000 7f3efffff640 1 --2- 172.21.15.84:0/2062047059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eec003970 0x7f3eec003d70 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0002230 tx=0x7f3eec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77422ba5d578528f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:35.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.911+0000 7f3efeffd640 1 -- 172.21.15.84:0/2062047059 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0034630 con 0x7f3eec003970 2023-10-16T10:07:35.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.911+0000 7f3efeffd640 1 -- 172.21.15.84:0/2062047059 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef002f070 con 0x7f3eec003970 2023-10-16T10:07:35.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.911+0000 7f3efeffd640 1 -- 172.21.15.84:0/2062047059 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0034a30 con 0x7f3eec003970 2023-10-16T10:07:35.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.912+0000 7f3f052ad640 1 -- 172.21.15.84:0/2062047059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eec003970 msgr2=0x7f3eec003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:35.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.912+0000 7f3f052ad640 1 --2- 172.21.15.84:0/2062047059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eec003970 0x7f3eec003d70 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0002230 tx=0x7f3eec0057f0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.912+0000 7f3f052ad640 1 -- 172.21.15.84:0/2062047059 shutdown_connections 2023-10-16T10:07:35.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.912+0000 7f3f052ad640 1 --2- 172.21.15.84:0/2062047059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eec003970 0x7f3eec003d70 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.912+0000 7f3f052ad640 1 -- 172.21.15.84:0/2062047059 >> 172.21.15.84:0/2062047059 conn(0x7f3f000cf570 msgr2=0x7f3f000cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:35.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.912+0000 7f3f052ad640 1 -- 172.21.15.84:0/2062047059 shutdown_connections 2023-10-16T10:07:35.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.912+0000 7f3f052ad640 1 -- 172.21.15.84:0/2062047059 wait complete. 2023-10-16T10:07:35.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.914+0000 7f3f052ad640 1 Processor -- start 2023-10-16T10:07:35.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.914+0000 7f3f052ad640 1 -- start start 2023-10-16T10:07:35.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.915+0000 7f3f052ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec003970 0x7f3f00150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.915+0000 7f3f052ad640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f00153210 0x7f3f000cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.915+0000 7f3f052ad640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f000cbee0 0x7f3f000c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.915+0000 7f3f052ad640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f000cc370 con 0x7f3f00153210 2023-10-16T10:07:35.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.915+0000 7f3f052ad640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f00151520 con 0x7f3eec003970 2023-10-16T10:07:35.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.915+0000 7f3f052ad640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f000cabd0 con 0x7f3f000cbee0 2023-10-16T10:07:35.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.915+0000 7f3eff7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f00153210 0x7f3f000cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.915+0000 7f3eff7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f00153210 0x7f3f000cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51282/0 (socket says 172.21.15.84:51282) 2023-10-16T10:07:35.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.915+0000 7f3eff7fe640 1 -- 172.21.15.84:0/183303082 learned_addr learned my addr 172.21.15.84:0/183303082 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:35.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.916+0000 7f3eff7fe640 1 -- 172.21.15.84:0/183303082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f000cbee0 msgr2=0x7f3f000c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:35.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.916+0000 7f3eff7fe640 1 --2- 172.21.15.84:0/183303082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f000cbee0 0x7f3f000c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.916+0000 7f3eff7fe640 1 -- 172.21.15.84:0/183303082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec003970 msgr2=0x7f3f00150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:35.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.916+0000 7f3eff7fe640 1 --2- 172.21.15.84:0/183303082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec003970 0x7f3f00150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.916+0000 7f3eff7fe640 1 -- 172.21.15.84:0/183303082 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0015d4e0 con 0x7f3f00153210 2023-10-16T10:07:35.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.916+0000 7f3eff7fe640 1 --2- 172.21.15.84:0/183303082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f00153210 0x7f3f000cb9a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800c480 tx=0x7f3ee800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:35.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.916+0000 7f3efcff9640 1 -- 172.21.15.84:0/183303082 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8012830 con 0x7f3f00153210 2023-10-16T10:07:35.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.916+0000 7f3efcff9640 1 -- 172.21.15.84:0/183303082 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee8012990 con 0x7f3f00153210 2023-10-16T10:07:35.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.916+0000 7f3efcff9640 1 -- 172.21.15.84:0/183303082 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8012d40 con 0x7f3f00153210 2023-10-16T10:07:35.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.916+0000 7f3f052ad640 1 -- 172.21.15.84:0/183303082 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0015d7c0 con 0x7f3f00153210 2023-10-16T10:07:35.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.916+0000 7f3f052ad640 1 -- 172.21.15.84:0/183303082 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f000ca3a0 con 0x7f3f00153210 2023-10-16T10:07:35.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.919+0000 7f3efcff9640 1 -- 172.21.15.84:0/183303082 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3ee801b400 con 0x7f3f00153210 2023-10-16T10:07:35.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.919+0000 7f3efcff9640 1 --2- 172.21.15.84:0/183303082 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f3ee006afc0 0x7f3ee006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.920+0000 7f3efcff9640 1 -- 172.21.15.84:0/183303082 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ee8089910 con 0x7f3f00153210 2023-10-16T10:07:35.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.920+0000 7f3efffff640 1 --2- 172.21.15.84:0/183303082 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f3ee006afc0 0x7f3ee006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.920+0000 7f3f0917a740 1 Processor -- start 2023-10-16T10:07:35.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.920+0000 7f3f0917a740 1 -- start start 2023-10-16T10:07:35.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.920+0000 7f3efffff640 1 --2- 172.21.15.84:0/183303082 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f3ee006afc0 0x7f3ee006d480 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0034000 tx=0x7f3ef0009030 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:35.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.921+0000 7f3f0917a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 0x55d439856070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.921+0000 7f3f0917a740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d439883550 0x55d439885f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.921+0000 7f3f0917a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4398565b0 0x55d439886440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.921+0000 7f3f0917a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4397767a0 con 0x55d439883170 2023-10-16T10:07:35.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.921+0000 7f3f0917a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4397c8740 con 0x55d4398565b0 2023-10-16T10:07:35.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.921+0000 7f3f0917a740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4397c5e30 con 0x55d439883550 2023-10-16T10:07:35.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.922+0000 7f3efffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 0x55d439856070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.922+0000 7f3efffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 0x55d439856070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51294/0 (socket says 172.21.15.84:51294) 2023-10-16T10:07:35.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.922+0000 7f3efffff640 1 -- 172.21.15.84:0/59474519 learned_addr learned my addr 172.21.15.84:0/59474519 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:35.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.922+0000 7f3eff7fe640 1 --2- 172.21.15.84:0/59474519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d439883550 0x55d439885f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.922+0000 7f3efffff640 1 -- 172.21.15.84:0/59474519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d439883550 msgr2=0x55d439885f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:35.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.922+0000 7f3f04aac640 1 --2- 172.21.15.84:0/59474519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4398565b0 0x55d439886440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.922+0000 7f3efffff640 1 --2- 172.21.15.84:0/59474519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d439883550 0x55d439885f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.922+0000 7f3efffff640 1 -- 172.21.15.84:0/59474519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4398565b0 msgr2=0x55d439886440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:35.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.922+0000 7f3efffff640 1 --2- 172.21.15.84:0/59474519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4398565b0 0x55d439886440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.922+0000 7f3efffff640 1 -- 172.21.15.84:0/59474519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d439785910 con 0x55d439883170 2023-10-16T10:07:35.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.922+0000 7f3efffff640 1 --2- 172.21.15.84:0/59474519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 0x55d439856070 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000a8f0 tx=0x7f3ef0030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e098af6f116863 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:35.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.923+0000 7f3ef5ffb640 1 -- 172.21.15.84:0/59474519 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0040c00 con 0x55d439883170 2023-10-16T10:07:35.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.923+0000 7f3ef5ffb640 1 -- 172.21.15.84:0/59474519 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef0040d60 con 0x55d439883170 2023-10-16T10:07:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.923+0000 7f3ef5ffb640 1 -- 172.21.15.84:0/59474519 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef004b640 con 0x55d439883170 2023-10-16T10:07:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.923+0000 7f3f0917a740 1 -- 172.21.15.84:0/59474519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 msgr2=0x55d439856070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.923+0000 7f3f0917a740 1 --2- 172.21.15.84:0/59474519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 0x55d439856070 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000a8f0 tx=0x7f3ef0030000 comp rx=0 tx=0).stop 2023-10-16T10:07:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.923+0000 7f3f0917a740 1 -- 172.21.15.84:0/59474519 shutdown_connections 2023-10-16T10:07:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.923+0000 7f3f0917a740 1 --2- 172.21.15.84:0/59474519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4398565b0 0x55d439886440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.923+0000 7f3f0917a740 1 --2- 172.21.15.84:0/59474519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d439883550 0x55d439885f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.923+0000 7f3f0917a740 1 --2- 172.21.15.84:0/59474519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 0x55d439856070 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.923+0000 7f3f0917a740 1 -- 172.21.15.84:0/59474519 >> 172.21.15.84:0/59474519 conn(0x55d439858380 msgr2=0x55d4398a0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:35.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.924+0000 7f3f0917a740 1 -- 172.21.15.84:0/59474519 shutdown_connections 2023-10-16T10:07:35.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.924+0000 7f3f0917a740 1 -- 172.21.15.84:0/59474519 wait complete. 2023-10-16T10:07:35.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.924+0000 7f3f0917a740 1 -- start start 2023-10-16T10:07:35.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.925+0000 7f3f0917a740 1 Processor -- start 2023-10-16T10:07:35.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.926+0000 7f3f0917a740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d439883170 0x55d4398b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.926+0000 7f3f0917a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d439883550 0x55d4398987d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.926+0000 7f3f0917a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d43989af80 0x55d43989b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.926+0000 7f3f0917a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4397c1720 con 0x55d43989af80 2023-10-16T10:07:35.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.926+0000 7f3f0917a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d43975f970 con 0x55d439883550 2023-10-16T10:07:35.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.926+0000 7f3f0917a740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d43975f610 con 0x55d439883170 2023-10-16T10:07:35.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.926+0000 7f3f0917a740 10 client.0 _ll_register_callbacks cb 0x55d4397819d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:07:35.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.926+0000 7f3f04aac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d43989af80 0x55d43989b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.926+0000 7f3f04aac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d43989af80 0x55d43989b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51306/0 (socket says 172.21.15.84:51306) 2023-10-16T10:07:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.926+0000 7f3f04aac640 1 -- 172.21.15.84:0/3393408524 learned_addr learned my addr 172.21.15.84:0/3393408524 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.926+0000 7f3efffff640 1 --2- 172.21.15.84:0/3393408524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d439883170 0x55d4398b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.926+0000 7f3f04aac640 1 -- 172.21.15.84:0/3393408524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d439883170 msgr2=0x55d4398b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.926+0000 7f3f04aac640 1 --2- 172.21.15.84:0/3393408524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d439883170 0x55d4398b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.926+0000 7f3f04aac640 1 -- 172.21.15.84:0/3393408524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d439883550 msgr2=0x55d4398987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.926+0000 7f3f04aac640 1 --2- 172.21.15.84:0/3393408524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d439883550 0x55d4398987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.926+0000 7f3f04aac640 1 -- 172.21.15.84:0/3393408524 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef0038070 con 0x55d43989af80 2023-10-16T10:07:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.927+0000 7f3f04aac640 1 --2- 172.21.15.84:0/3393408524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d43989af80 0x55d43989b400 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f3f000cd0f0 tx=0x7f3ef80094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fc2df7d06ca1ec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:35.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.927+0000 7f3edeffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T10:07:35.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.927+0000 7f3edeffd640 1 -- 172.21.15.84:0/3393408524 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8017070 con 0x55d43989af80 2023-10-16T10:07:35.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.927+0000 7f3edeffd640 1 -- 172.21.15.84:0/3393408524 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ef800ee70 con 0x55d43989af80 2023-10-16T10:07:35.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.927+0000 7f3f0917a740 1 -- 172.21.15.84:0/3393408524 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4397e8160 con 0x55d43989af80 2023-10-16T10:07:35.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.927+0000 7f3edeffd640 1 -- 172.21.15.84:0/3393408524 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ef800c070 con 0x55d43989af80 2023-10-16T10:07:35.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.927+0000 7f3f0917a740 10 client.8181 fetch_fsmap learned FSMap version 62 2023-10-16T10:07:35.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.927+0000 7f3f0917a740 1 -- 172.21.15.84:0/3393408524 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55d4397bfda0 con 0x55d43989af80 2023-10-16T10:07:35.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.927+0000 7f3edeffd640 1 -- 172.21.15.84:0/3393408524 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d4397e8160 con 0x55d43989af80 2023-10-16T10:07:35.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.928+0000 7f3edeffd640 1 -- 172.21.15.84:0/3393408524 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3ef801a020 con 0x55d43989af80 2023-10-16T10:07:35.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.928+0000 7f3f0917a740 10 client.8181 fetch_fsmap finished waiting for FSMap version 62 2023-10-16T10:07:35.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.928+0000 7f3f0917a740 10 client.8181 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T10:07:35.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.928+0000 7f3f0917a740 20 client.8181 populate_metadata read hostname 'smithi084' 2023-10-16T10:07:35.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.928+0000 7f3f0917a740 1 --2- 172.21.15.84:0/3393408524 >> [v2:172.21.15.84:6835/2859088731,v1:172.21.15.84:6837/2859088731] conn(0x55d439877670 0x55d4398b4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.928+0000 7f3f0917a740 4 client.8181 mds_command: new command op to 8040 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T10:07:35.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.928+0000 7f3f0917a740 1 -- 172.21.15.84:0/3393408524 --> [v2:172.21.15.84:6835/2859088731,v1:172.21.15.84:6837/2859088731] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d439793cc0 con 0x55d439877670 2023-10-16T10:07:35.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.928+0000 7f3efffff640 1 --2- 172.21.15.84:0/3393408524 >> [v2:172.21.15.84:6835/2859088731,v1:172.21.15.84:6837/2859088731] conn(0x55d439877670 0x55d4398b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.928+0000 7f3efffff640 1 --2- 172.21.15.84:0/3393408524 >> [v2:172.21.15.84:6835/2859088731,v1:172.21.15.84:6837/2859088731] conn(0x55d439877670 0x55d4398b4130 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87ed5a4a3fd24e5c server_cookie=2c45c5188007334b in_seq=0 out_seq=0 2023-10-16T10:07:35.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.928+0000 7f3edeffd640 10 client.8181 ms_handle_connect on v2:172.21.15.84:6835/2859088731 2023-10-16T10:07:35.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.929+0000 7f3edeffd640 1 -- 172.21.15.84:0/3393408524 <== mds.0 v2:172.21.15.84:6835/2859088731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d439793cc0 con 0x55d439877670 2023-10-16T10:07:35.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.929+0000 7f3edeffd640 10 client.8181 handle_command_reply: tid=0 2023-10-16T10:07:35.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.929+0000 7f3f0917a740 1 client.8181 shutdown 2023-10-16T10:07:35.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.929+0000 7f3f0917a740 1 -- 172.21.15.84:0/3393408524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d43989af80 msgr2=0x55d43989b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:35.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.929+0000 7f3f0917a740 1 --2- 172.21.15.84:0/3393408524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d43989af80 0x55d43989b400 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f3f000cd0f0 tx=0x7f3ef80094f0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.929+0000 7f3f0917a740 1 -- 172.21.15.84:0/3393408524 shutdown_connections 2023-10-16T10:07:35.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.929+0000 7f3f0917a740 1 --2- 172.21.15.84:0/3393408524 >> [v2:172.21.15.84:6835/2859088731,v1:172.21.15.84:6837/2859088731] conn(0x55d439877670 0x55d4398b4130 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.929+0000 7f3f0917a740 1 --2- 172.21.15.84:0/3393408524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d43989af80 0x55d43989b400 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.929+0000 7f3f0917a740 1 --2- 172.21.15.84:0/3393408524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d439883550 0x55d4398987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.929+0000 7f3f0917a740 1 --2- 172.21.15.84:0/3393408524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d439883170 0x55d4398b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.929+0000 7f3edeffd640 0 client.8181 ms_handle_reset on v2:172.21.15.84:6835/2859088731 2023-10-16T10:07:35.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.929+0000 7f3f0917a740 1 -- 172.21.15.84:0/3393408524 >> 172.21.15.84:0/3393408524 conn(0x55d4398565b0 msgr2=0x55d439885ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:35.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.930+0000 7f3f0917a740 1 -- 172.21.15.84:0/3393408524 shutdown_connections 2023-10-16T10:07:35.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.930+0000 7f3f0917a740 1 -- 172.21.15.84:0/3393408524 wait complete. 2023-10-16T10:07:35.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.930+0000 7f3f0917a740 20 client.8181 trim_cache size 0 max 16384 2023-10-16T10:07:35.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.946+0000 7f3f0917a740 1 Processor -- start 2023-10-16T10:07:35.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.946+0000 7f3f0917a740 1 -- start start 2023-10-16T10:07:35.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.947+0000 7f3f0917a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 0x55d4398a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.947+0000 7f3f0917a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d439883550 0x55d4398550f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.947+0000 7f3f0917a740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d43989af80 0x55d4398a4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.947+0000 7f3f0917a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d439525f20 con 0x55d439883170 2023-10-16T10:07:35.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.947+0000 7f3f0917a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d439881b60 con 0x55d439883550 2023-10-16T10:07:35.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.947+0000 7f3f0917a740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4398a1710 con 0x55d43989af80 2023-10-16T10:07:35.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.948+0000 7f3efffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 0x55d4398a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.948+0000 7f3efffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 0x55d4398a35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51308/0 (socket says 172.21.15.84:51308) 2023-10-16T10:07:35.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.948+0000 7f3efffff640 1 -- 172.21.15.84:0/3971934602 learned_addr learned my addr 172.21.15.84:0/3971934602 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:35.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.948+0000 7f3eff7fe640 1 --2- 172.21.15.84:0/3971934602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d439883550 0x55d4398550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.948+0000 7f3f04aac640 1 --2- 172.21.15.84:0/3971934602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d43989af80 0x55d4398a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.948+0000 7f3efffff640 1 -- 172.21.15.84:0/3971934602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d43989af80 msgr2=0x55d4398a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:35.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.948+0000 7f3efffff640 1 --2- 172.21.15.84:0/3971934602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d43989af80 0x55d4398a4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.948+0000 7f3efffff640 1 -- 172.21.15.84:0/3971934602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d439883550 msgr2=0x55d4398550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:35.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.948+0000 7f3efffff640 1 --2- 172.21.15.84:0/3971934602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d439883550 0x55d4398550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.948+0000 7f3efffff640 1 -- 172.21.15.84:0/3971934602 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d43975eb80 con 0x55d439883170 2023-10-16T10:07:35.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.948+0000 7f3efffff640 1 --2- 172.21.15.84:0/3971934602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 0x55d4398a35f0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f3ef002cc00 tx=0x7f3ef0038000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e8bebfcee47029 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:35.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.949+0000 7f3ef67fc640 1 -- 172.21.15.84:0/3971934602 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0056be0 con 0x55d439883170 2023-10-16T10:07:35.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.949+0000 7f3ef67fc640 1 -- 172.21.15.84:0/3971934602 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef0056d40 con 0x55d439883170 2023-10-16T10:07:35.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.949+0000 7f3ef67fc640 1 -- 172.21.15.84:0/3971934602 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0056230 con 0x55d439883170 2023-10-16T10:07:35.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.949+0000 7f3f0917a740 1 -- 172.21.15.84:0/3971934602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 msgr2=0x55d4398a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:35.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.949+0000 7f3f0917a740 1 --2- 172.21.15.84:0/3971934602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 0x55d4398a35f0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f3ef002cc00 tx=0x7f3ef0038000 comp rx=0 tx=0).stop 2023-10-16T10:07:35.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.949+0000 7f3f0917a740 1 -- 172.21.15.84:0/3971934602 shutdown_connections 2023-10-16T10:07:35.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.949+0000 7f3f0917a740 1 --2- 172.21.15.84:0/3971934602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d43989af80 0x55d4398a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.949+0000 7f3f0917a740 1 --2- 172.21.15.84:0/3971934602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d439883550 0x55d4398550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.949+0000 7f3f0917a740 1 --2- 172.21.15.84:0/3971934602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 0x55d4398a35f0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.949+0000 7f3f0917a740 1 -- 172.21.15.84:0/3971934602 >> 172.21.15.84:0/3971934602 conn(0x55d4398565b0 msgr2=0x55d43985a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:35.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.949+0000 7f3f0917a740 1 -- 172.21.15.84:0/3971934602 shutdown_connections 2023-10-16T10:07:35.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.949+0000 7f3f0917a740 1 -- 172.21.15.84:0/3971934602 wait complete. 2023-10-16T10:07:35.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.950+0000 7f3f0917a740 1 -- start start 2023-10-16T10:07:35.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.950+0000 7f3f0917a740 1 Processor -- start 2023-10-16T10:07:35.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.951+0000 7f3f0917a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 0x55d4398ef060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.951+0000 7f3f0917a740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d439883550 0x55d4398a4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.951+0000 7f3f0917a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d43989af80 0x55d4398a4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.951+0000 7f3f0917a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d439887de0 con 0x55d439883170 2023-10-16T10:07:35.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.951+0000 7f3f0917a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d439886d90 con 0x55d43989af80 2023-10-16T10:07:35.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.951+0000 7f3f0917a740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4398ef990 con 0x55d439883550 2023-10-16T10:07:35.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.951+0000 7f3f0917a740 10 client.0 _ll_register_callbacks cb 0x7f3f000741b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:07:35.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.951+0000 7f3efffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 0x55d4398ef060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.952+0000 7f3efffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 0x55d4398ef060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51322/0 (socket says 172.21.15.84:51322) 2023-10-16T10:07:35.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.952+0000 7f3efffff640 1 -- 172.21.15.84:0/1296587851 learned_addr learned my addr 172.21.15.84:0/1296587851 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:35.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.952+0000 7f3efffff640 1 -- 172.21.15.84:0/1296587851 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d439883550 msgr2=0x55d4398a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:35.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.952+0000 7f3efffff640 1 --2- 172.21.15.84:0/1296587851 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d439883550 0x55d4398a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.952+0000 7f3efffff640 1 -- 172.21.15.84:0/1296587851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d43989af80 msgr2=0x55d4398a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:35.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.952+0000 7f3efffff640 1 --2- 172.21.15.84:0/1296587851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d43989af80 0x55d4398a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.952+0000 7f3efffff640 1 -- 172.21.15.84:0/1296587851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d439785910 con 0x55d439883170 2023-10-16T10:07:35.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.952+0000 7f3efffff640 1 --2- 172.21.15.84:0/1296587851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 0x55d4398ef060 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f3ef005d040 tx=0x7f3ef002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21d7646b00d9b7b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:35.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.952+0000 7f3edeffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T10:07:35.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.952+0000 7f3edeffd640 1 -- 172.21.15.84:0/1296587851 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0056be0 con 0x55d439883170 2023-10-16T10:07:35.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.952+0000 7f3edeffd640 1 -- 172.21.15.84:0/1296587851 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ef0056040 con 0x55d439883170 2023-10-16T10:07:35.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.952+0000 7f3f0917a740 1 -- 172.21.15.84:0/1296587851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ef0040d60 con 0x55d439883170 2023-10-16T10:07:35.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.953+0000 7f3edeffd640 1 -- 172.21.15.84:0/1296587851 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ef002cc70 con 0x55d439883170 2023-10-16T10:07:35.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.953+0000 7f3f0917a740 10 client.8187 fetch_fsmap learned FSMap version 62 2023-10-16T10:07:35.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.953+0000 7f3f0917a740 1 -- 172.21.15.84:0/1296587851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55d439784170 con 0x55d439883170 2023-10-16T10:07:35.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.953+0000 7f3edeffd640 1 -- 172.21.15.84:0/1296587851 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef0040d60 con 0x55d439883170 2023-10-16T10:07:35.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.953+0000 7f3edeffd640 1 -- 172.21.15.84:0/1296587851 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3ef0061020 con 0x55d439883170 2023-10-16T10:07:35.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.953+0000 7f3f0917a740 10 client.8187 fetch_fsmap finished waiting for FSMap version 62 2023-10-16T10:07:35.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.953+0000 7f3f0917a740 10 client.8187 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T10:07:35.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.953+0000 7f3f0917a740 20 client.8187 populate_metadata read hostname 'smithi084' 2023-10-16T10:07:35.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.953+0000 7f3f0917a740 1 --2- 172.21.15.84:0/1296587851 >> [v2:172.21.15.84:6835/2859088731,v1:172.21.15.84:6837/2859088731] conn(0x55d439875c60 0x55d439876040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:35.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.953+0000 7f3f0917a740 4 client.8187 mds_command: new command op to 8040 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T10:07:35.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.953+0000 7f3f0917a740 1 -- 172.21.15.84:0/1296587851 --> [v2:172.21.15.84:6835/2859088731,v1:172.21.15.84:6837/2859088731] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d4397ba700 con 0x55d439875c60 2023-10-16T10:07:35.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.953+0000 7f3eff7fe640 1 --2- 172.21.15.84:0/1296587851 >> [v2:172.21.15.84:6835/2859088731,v1:172.21.15.84:6837/2859088731] conn(0x55d439875c60 0x55d439876040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:35.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.954+0000 7f3eff7fe640 1 --2- 172.21.15.84:0/1296587851 >> [v2:172.21.15.84:6835/2859088731,v1:172.21.15.84:6837/2859088731] conn(0x55d439875c60 0x55d439876040 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d531cb179b552677 server_cookie=be3f8ff4050c48de in_seq=0 out_seq=0 2023-10-16T10:07:35.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.954+0000 7f3edeffd640 10 client.8187 ms_handle_connect on v2:172.21.15.84:6835/2859088731 2023-10-16T10:07:35.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.954+0000 7f3edeffd640 1 -- 172.21.15.84:0/1296587851 <== mds.0 v2:172.21.15.84:6835/2859088731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55d4397ba700 con 0x55d439875c60 2023-10-16T10:07:35.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.954+0000 7f3edeffd640 10 client.8187 handle_command_reply: tid=0 2023-10-16T10:07:35.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.954+0000 7f3f0917a740 1 client.8187 shutdown 2023-10-16T10:07:35.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.954+0000 7f3f0917a740 1 -- 172.21.15.84:0/1296587851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 msgr2=0x55d4398ef060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:35.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.954+0000 7f3f0917a740 1 --2- 172.21.15.84:0/1296587851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 0x55d4398ef060 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f3ef005d040 tx=0x7f3ef002e000 comp rx=0 tx=0).stop 2023-10-16T10:07:35.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.954+0000 7f3f0917a740 1 -- 172.21.15.84:0/1296587851 shutdown_connections 2023-10-16T10:07:35.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.954+0000 7f3f0917a740 1 --2- 172.21.15.84:0/1296587851 >> [v2:172.21.15.84:6835/2859088731,v1:172.21.15.84:6837/2859088731] conn(0x55d439875c60 0x55d439876040 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.954+0000 7f3f0917a740 1 --2- 172.21.15.84:0/1296587851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d43989af80 0x55d4398a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.954+0000 7f3f0917a740 1 --2- 172.21.15.84:0/1296587851 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d439883550 0x55d4398a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.954+0000 7f3f0917a740 1 --2- 172.21.15.84:0/1296587851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d439883170 0x55d4398ef060 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.954+0000 7f3edeffd640 0 client.8187 ms_handle_reset on v2:172.21.15.84:6835/2859088731 2023-10-16T10:07:35.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.954+0000 7f3f0917a740 1 -- 172.21.15.84:0/1296587851 >> 172.21.15.84:0/1296587851 conn(0x55d4398565b0 msgr2=0x55d439899550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:35.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.955+0000 7f3f0917a740 1 -- 172.21.15.84:0/1296587851 shutdown_connections 2023-10-16T10:07:35.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.955+0000 7f3f0917a740 1 -- 172.21.15.84:0/1296587851 wait complete. 2023-10-16T10:07:35.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.955+0000 7f3f0917a740 20 client.8187 trim_cache size 0 max 16384 2023-10-16T10:07:35.977 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T10:07:35.977 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "63fb4d68-f881-4694-b880-d10369105c0e", 2023-10-16T10:07:35.977 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-16T10:07:35.978 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 8040, 2023-10-16T10:07:35.978 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-16T10:07:35.978 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-16T10:07:35.978 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-16T10:07:35.979 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 6.5982903610000001, 2023-10-16T10:07:35.979 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 62, 2023-10-16T10:07:35.979 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 124, 2023-10-16T10:07:35.979 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 124, 2023-10-16T10:07:35.979 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 12.021451260999999 2023-10-16T10:07:35.980 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T10:07:35.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.955+0000 7f3f052ad640 1 -- 172.21.15.84:0/183303082 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f3ee006afc0 msgr2=0x7f3ee006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:35.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.955+0000 7f3f052ad640 1 --2- 172.21.15.84:0/183303082 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f3ee006afc0 0x7f3ee006d480 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0034000 tx=0x7f3ef0009030 comp rx=0 tx=0).stop 2023-10-16T10:07:35.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.955+0000 7f3f052ad640 1 -- 172.21.15.84:0/183303082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f00153210 msgr2=0x7f3f000cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:35.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.955+0000 7f3f052ad640 1 --2- 172.21.15.84:0/183303082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f00153210 0x7f3f000cb9a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800c480 tx=0x7f3ee800c940 comp rx=0 tx=0).stop 2023-10-16T10:07:35.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.955+0000 7f3f052ad640 1 -- 172.21.15.84:0/183303082 shutdown_connections 2023-10-16T10:07:35.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.955+0000 7f3f052ad640 1 --2- 172.21.15.84:0/183303082 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f3ee006afc0 0x7f3ee006d480 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.955+0000 7f3f052ad640 1 --2- 172.21.15.84:0/183303082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f000cbee0 0x7f3f000c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.955+0000 7f3f052ad640 1 --2- 172.21.15.84:0/183303082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f00153210 0x7f3f000cb9a0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.955+0000 7f3f052ad640 1 --2- 172.21.15.84:0/183303082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec003970 0x7f3f00150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:35.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.955+0000 7f3f052ad640 1 -- 172.21.15.84:0/183303082 >> 172.21.15.84:0/183303082 conn(0x7f3f000cf570 msgr2=0x7f3f000b7640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:35.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.956+0000 7f3f052ad640 1 -- 172.21.15.84:0/183303082 shutdown_connections 2023-10-16T10:07:35.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:35.956+0000 7f3f052ad640 1 -- 172.21.15.84:0/183303082 wait complete. 2023-10-16T10:07:35.984 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T10:07:36.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.189+0000 7fa1b9fe2640 1 Processor -- start 2023-10-16T10:07:36.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.190+0000 7fa1b9fe2640 1 -- start start 2023-10-16T10:07:36.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.191+0000 7fa1b9fe2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa1b415ae80 0x7fa1b415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:36.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.191+0000 7fa1b9fe2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b415ed60 con 0x7fa1b40ce390 2023-10-16T10:07:36.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.191+0000 7fa1b9fe2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b40707e0 con 0x7fa1b415aaa0 2023-10-16T10:07:36.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.191+0000 7fa1b9fe2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b406b810 con 0x7fa1b415ae80 2023-10-16T10:07:36.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.192+0000 7fa1b3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa1b40ce390 0x7fa1b4070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:41296/0 (socket says 172.21.15.84:41296) 2023-10-16T10:07:36.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.192+0000 7fa1b3fff640 1 -- 172.21.15.84:0/152931465 learned_addr learned my addr 172.21.15.84:0/152931465 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:36.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.192+0000 7fa1abfff640 1 --2- 172.21.15.84:0/152931465 >> v2:172.21.15.157:3300/0 conn(0x7fa1b415ae80 0x7fa1b415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:36.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.192+0000 7fa1b2ffd640 1 -- 172.21.15.84:0/152931465 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3678241735 0 0) 0x7fa1b415ed60 con 0x7fa1b40ce390 2023-10-16T10:07:36.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.192+0000 7fa1b2ffd640 1 -- 172.21.15.84:0/152931465 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1a0003520 con 0x7fa1b40ce390 2023-10-16T10:07:36.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.192+0000 7fa1b2ffd640 1 -- 172.21.15.84:0/152931465 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3177093617 0 0) 0x7fa1a0003520 con 0x7fa1b40ce390 2023-10-16T10:07:36.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.192+0000 7fa1b2ffd640 1 -- 172.21.15.84:0/152931465 >> v2:172.21.15.157:3300/0 conn(0x7fa1b415ae80 msgr2=0x7fa1b415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:36.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.192+0000 7fa1b2ffd640 1 --2- 172.21.15.84:0/152931465 >> v2:172.21.15.157:3300/0 conn(0x7fa1b415ae80 0x7fa1b415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:36.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.192+0000 7fa1b2ffd640 1 -- 172.21.15.84:0/152931465 >> v1:172.21.15.157:6789/0 conn(0x7fa1b415aaa0 legacy=0x7fa1b40cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:36.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.192+0000 7fa1b2ffd640 1 -- 172.21.15.84:0/152931465 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b415ef90 con 0x7fa1b40ce390 2023-10-16T10:07:36.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.192+0000 7fa1b2ffd640 1 -- 172.21.15.84:0/152931465 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fa1ac0019f0 con 0x7fa1b40ce390 2023-10-16T10:07:36.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.192+0000 7fa1b2ffd640 1 -- 172.21.15.84:0/152931465 >> v1:172.21.15.84:6789/0 conn(0x7fa1b40ce390 legacy=0x7fa1b4070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:36.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.193+0000 7fa1b2ffd640 1 --2- 172.21.15.84:0/152931465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1a0003970 0x7fa1a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:36.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.193+0000 7fa1b2ffd640 1 --2- 172.21.15.84:0/152931465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1a00047c0 0x7fa1a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:36.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.193+0000 7fa1b2ffd640 1 --2- 172.21.15.84:0/152931465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1a0005180 0x7fa1a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:36.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.193+0000 7fa1b2ffd640 1 -- 172.21.15.84:0/152931465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b406b810 con 0x7fa1a0003970 2023-10-16T10:07:36.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.193+0000 7fa1b2ffd640 1 -- 172.21.15.84:0/152931465 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a000c020 con 0x7fa1a0005180 2023-10-16T10:07:36.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.193+0000 7fa1b2ffd640 1 -- 172.21.15.84:0/152931465 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a000c1e0 con 0x7fa1a00047c0 2023-10-16T10:07:36.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.193+0000 7fa1b37fe640 1 --2- 172.21.15.84:0/152931465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1a0003970 0x7fa1a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:36.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.193+0000 7fa1abfff640 1 --2- 172.21.15.84:0/152931465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1a00047c0 0x7fa1a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:36.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.193+0000 7fa1b3fff640 1 --2- 172.21.15.84:0/152931465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1a0005180 0x7fa1a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:36.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.193+0000 7fa1b37fe640 1 -- 172.21.15.84:0/152931465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1a00047c0 msgr2=0x7fa1a0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:36.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.193+0000 7fa1b37fe640 1 --2- 172.21.15.84:0/152931465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1a00047c0 0x7fa1a0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:36.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.193+0000 7fa1b37fe640 1 -- 172.21.15.84:0/152931465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1a0005180 msgr2=0x7fa1a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:36.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.193+0000 7fa1b37fe640 1 --2- 172.21.15.84:0/152931465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1a0005180 0x7fa1a000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:36.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.193+0000 7fa1b37fe640 1 -- 172.21.15.84:0/152931465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a0003520 con 0x7fa1a0003970 2023-10-16T10:07:36.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.193+0000 7fa1b3fff640 1 --2- 172.21.15.84:0/152931465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1a0005180 0x7fa1a000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:36.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.193+0000 7fa1b3fff640 1 -- 172.21.15.84:0/152931465 reap_dead start 2023-10-16T10:07:36.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.195+0000 7fa1b37fe640 1 --2- 172.21.15.84:0/152931465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1a0003970 0x7fa1a0003d90 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4001750 tx=0x7fa1a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a760989ed00ac46 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:36.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.195+0000 7fa1b2ffd640 1 -- 172.21.15.84:0/152931465 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40052d0 con 0x7fa1a0003970 2023-10-16T10:07:36.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.195+0000 7fa1b2ffd640 1 -- 172.21.15.84:0/152931465 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a402e050 con 0x7fa1a0003970 2023-10-16T10:07:36.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.195+0000 7fa1b2ffd640 1 -- 172.21.15.84:0/152931465 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4003830 con 0x7fa1a0003970 2023-10-16T10:07:36.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.195+0000 7fa1b9fe2640 1 -- 172.21.15.84:0/152931465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1a0003970 msgr2=0x7fa1a0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:36.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.195+0000 7fa1b9fe2640 1 --2- 172.21.15.84:0/152931465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1a0003970 0x7fa1a0003d90 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4001750 tx=0x7fa1a00057f0 comp rx=0 tx=0).stop 2023-10-16T10:07:36.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.195+0000 7fa1b9fe2640 1 -- 172.21.15.84:0/152931465 shutdown_connections 2023-10-16T10:07:36.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.195+0000 7fa1b9fe2640 1 --2- 172.21.15.84:0/152931465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1a0003970 0x7fa1a0003d90 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:36.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.196+0000 7fa1b9fe2640 1 -- 172.21.15.84:0/152931465 >> 172.21.15.84:0/152931465 conn(0x7fa1b40cb410 msgr2=0x7fa1b40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:36.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.196+0000 7fa1b9fe2640 1 -- 172.21.15.84:0/152931465 shutdown_connections 2023-10-16T10:07:36.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.196+0000 7fa1b9fe2640 1 -- 172.21.15.84:0/152931465 wait complete. 2023-10-16T10:07:36.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.197+0000 7fa1b9fe2640 1 Processor -- start 2023-10-16T10:07:36.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.198+0000 7fa1b9fe2640 1 -- start start 2023-10-16T10:07:36.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.198+0000 7fa1b9fe2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1a0003970 0x7fa1b41e6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:36.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.198+0000 7fa1b9fe2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1b40c5a60 0x7fa1b41e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:36.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.198+0000 7fa1b9fe2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b415da50 0x7fa1b414aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:36.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.199+0000 7fa1b9fe2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b415bc80 con 0x7fa1b40c5a60 2023-10-16T10:07:36.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.199+0000 7fa1b9fe2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b415b320 con 0x7fa1b415da50 2023-10-16T10:07:36.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.199+0000 7fa1b9fe2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b41e5810 con 0x7fa1a0003970 2023-10-16T10:07:36.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.199+0000 7fa1abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1b40c5a60 0x7fa1b41e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:36.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.199+0000 7fa1abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1b40c5a60 0x7fa1b41e4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51340/0 (socket says 172.21.15.84:51340) 2023-10-16T10:07:36.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.199+0000 7fa1abfff640 1 -- 172.21.15.84:0/2329237111 learned_addr learned my addr 172.21.15.84:0/2329237111 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:36.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.199+0000 7fa1b37fe640 1 --2- 172.21.15.84:0/2329237111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1a0003970 0x7fa1b41e6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:36.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.199+0000 7fa1abfff640 1 -- 172.21.15.84:0/2329237111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1a0003970 msgr2=0x7fa1b41e6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:36.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.199+0000 7fa1abfff640 1 --2- 172.21.15.84:0/2329237111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1a0003970 0x7fa1b41e6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:36.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.199+0000 7fa1abfff640 1 -- 172.21.15.84:0/2329237111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b415da50 msgr2=0x7fa1b414aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:36.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.199+0000 7fa1abfff640 1 --2- 172.21.15.84:0/2329237111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b415da50 0x7fa1b414aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:36.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.199+0000 7fa1abfff640 1 -- 172.21.15.84:0/2329237111 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b415b5c0 con 0x7fa1b40c5a60 2023-10-16T10:07:36.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.199+0000 7fa1b37fe640 1 --2- 172.21.15.84:0/2329237111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1a0003970 0x7fa1b41e6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:36.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.200+0000 7fa1abfff640 1 --2- 172.21.15.84:0/2329237111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1b40c5a60 0x7fa1b41e4fc0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0114c0 tx=0x7fa19c011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:36.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.200+0000 7fa1b0ff9640 1 -- 172.21.15.84:0/2329237111 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c015870 con 0x7fa1b40c5a60 2023-10-16T10:07:36.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.200+0000 7fa1b0ff9640 1 -- 172.21.15.84:0/2329237111 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19c0159d0 con 0x7fa1b40c5a60 2023-10-16T10:07:36.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.200+0000 7fa1b0ff9640 1 -- 172.21.15.84:0/2329237111 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c015d40 con 0x7fa1b40c5a60 2023-10-16T10:07:36.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.200+0000 7fa1b9fe2640 1 -- 172.21.15.84:0/2329237111 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b415b8a0 con 0x7fa1b40c5a60 2023-10-16T10:07:36.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.200+0000 7fa1b9fe2640 1 -- 172.21.15.84:0/2329237111 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b40c5ee0 con 0x7fa1b40c5a60 2023-10-16T10:07:36.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.201+0000 7fa1b9fe2640 1 -- 172.21.15.84:0/2329237111 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b415f530 con 0x7fa1b40c5a60 2023-10-16T10:07:36.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.202+0000 7fa1b0ff9640 1 -- 172.21.15.84:0/2329237111 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa19c013400 con 0x7fa1b40c5a60 2023-10-16T10:07:36.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.203+0000 7fa1b0ff9640 1 --2- 172.21.15.84:0/2329237111 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fa19406afc0 0x7fa19406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:36.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.203+0000 7fa1b0ff9640 1 -- 172.21.15.84:0/2329237111 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa19c089a90 con 0x7fa1b40c5a60 2023-10-16T10:07:36.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.203+0000 7fa1b37fe640 1 --2- 172.21.15.84:0/2329237111 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fa19406afc0 0x7fa19406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:36.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.204+0000 7fa1b37fe640 1 --2- 172.21.15.84:0/2329237111 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fa19406afc0 0x7fa19406d480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4033000 tx=0x7fa1a4009030 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:36.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.207+0000 7fa1b0ff9640 1 -- 172.21.15.84:0/2329237111 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa1b415f530 con 0x7fa1b40c5a60 2023-10-16T10:07:36.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.444+0000 7fa1b9fe2640 1 -- 172.21.15.84:0/2329237111 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa1b415f530 con 0x7fa1b40c5a60 2023-10-16T10:07:36.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.445+0000 7fa1b0ff9640 1 -- 172.21.15.84:0/2329237111 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa1b415f530 con 0x7fa1b40c5a60 2023-10-16T10:07:36.447 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T10:07:36.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.448+0000 7fa1b9fe2640 1 -- 172.21.15.84:0/2329237111 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fa19406afc0 msgr2=0x7fa19406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:36.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.448+0000 7fa1b9fe2640 1 --2- 172.21.15.84:0/2329237111 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fa19406afc0 0x7fa19406d480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4033000 tx=0x7fa1a4009030 comp rx=0 tx=0).stop 2023-10-16T10:07:36.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.448+0000 7fa1b9fe2640 1 -- 172.21.15.84:0/2329237111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1b40c5a60 msgr2=0x7fa1b41e4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:36.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.448+0000 7fa1b9fe2640 1 --2- 172.21.15.84:0/2329237111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1b40c5a60 0x7fa1b41e4fc0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0114c0 tx=0x7fa19c011980 comp rx=0 tx=0).stop 2023-10-16T10:07:36.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.448+0000 7fa1b9fe2640 1 -- 172.21.15.84:0/2329237111 shutdown_connections 2023-10-16T10:07:36.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.448+0000 7fa1b9fe2640 1 --2- 172.21.15.84:0/2329237111 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fa19406afc0 0x7fa19406d480 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:36.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.448+0000 7fa1b9fe2640 1 --2- 172.21.15.84:0/2329237111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b415da50 0x7fa1b414aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:36.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.448+0000 7fa1b9fe2640 1 --2- 172.21.15.84:0/2329237111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1b40c5a60 0x7fa1b41e4fc0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:36.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.448+0000 7fa1b9fe2640 1 --2- 172.21.15.84:0/2329237111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1a0003970 0x7fa1b41e6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:36.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.448+0000 7fa1b9fe2640 1 -- 172.21.15.84:0/2329237111 >> 172.21.15.84:0/2329237111 conn(0x7fa1b40cb410 msgr2=0x7fa1b406ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:36.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.448+0000 7fa1b9fe2640 1 -- 172.21.15.84:0/2329237111 shutdown_connections 2023-10-16T10:07:36.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:36.448+0000 7fa1b9fe2640 1 -- 172.21.15.84:0/2329237111 wait complete. 2023-10-16T10:07:36.468 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T10:07:36.469 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T10:07:36.469 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-16T10:07:36.469 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T10:07:36.469 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T10:07:36.470 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T10:07:36.470 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T10:07:36.470 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T10:07:36.471 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:07:36.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:36.472 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-16T10:07:36.523 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T10:07:36.523 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T10:07:36.524 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T10:07:36.524 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:07:36.524 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T10:07:36.524 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:07:36.525 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:07:36.525 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:07:36.525 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-16T10:07:36.525 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:07:36.525 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:07:36.526 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-16T10:07:36.526 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:07:36.526 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:07:36.526 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T10:07:36.526 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:07:36.527 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T10:07:36.527 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 41294sec preferred_lft 41294sec 2023-10-16T10:07:36.527 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-16T10:07:36.527 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:07:36.527 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T10:07:36.528 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:07:36.528 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T10:07:36.528 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:07:36.528 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::20a8:eaff:fe9d:5cea/64 scope link 2023-10-16T10:07:36.528 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:07:36.528 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:07:36.529 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T10:07:36.529 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1854:13ff:fe47:be92/64 scope link 2023-10-16T10:07:36.529 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:07:36.529 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:07:36.529 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether ee:d5:41:a3:a5:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T10:07:36.530 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ecd5:41ff:fea3:a570/64 scope link 2023-10-16T10:07:36.530 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:07:36.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:36.531 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T10:07:36.623 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-16T10:07:36.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:36.627 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-16T10:07:36.680 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-16T10:07:36.680 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T10:07:36.680 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T10:07:36.680 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T10:07:36.681 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T10:07:36.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:36.681 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T10:07:36.682 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T10:07:36.682 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T10:07:36.683 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T10:07:36.683 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T10:07:36.683 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T10:07:36.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:36.861 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T10:07:36.914 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T10:07:36.914 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T10:07:36.914 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:07:36.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:36.915 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-16T10:07:37.011 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:07:37.011 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T10:07:37.069 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:07:37.069 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:07:37.070 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:07:37.120 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-16T10:07:37.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:37.121 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:07:37.265 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:07:37.266 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T10:07:37.298 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:07:37.298 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T10:07:37.298 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T10:07:37.299 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with new device syntax: 0@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T10:07:37.299 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T10:07:37.300 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-16T10:07:37.300 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2023-10-16T10:07:37.300 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T10:07:37.301 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T10:07:37.301 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T10:07:37.304 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T10:07:37.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:37.305 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:07:37.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:37.395 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T10:07:37.460 INFO:teuthology.orchestra.run.smithi084.stdout:client8196 2023-10-16T10:07:37.461 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:07:37.461 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:07:37.515 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T10:07:37.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.770+0000 7f4c0d20e640 1 Processor -- start 2023-10-16T10:07:37.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.771+0000 7f4c0d20e640 1 -- start start 2023-10-16T10:07:37.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.772+0000 7f4c0d20e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4c0806a4b0 0x7f4c080ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:37.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.772+0000 7f4c0d20e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c080cb030 con 0x7f4c080c9ca0 2023-10-16T10:07:37.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.772+0000 7f4c0d20e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c080c91c0 con 0x7f4c0815ad70 2023-10-16T10:07:37.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.772+0000 7f4c0d20e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c080c94a0 con 0x7f4c0806a4b0 2023-10-16T10:07:37.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.772+0000 7f4c0659c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4c0806a4b0 0x7f4c080ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:37.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.772+0000 7f4c0659c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4c0806a4b0 0x7f4c080ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51342/0 (socket says 172.21.15.84:51342) 2023-10-16T10:07:37.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.772+0000 7f4c0659c640 1 -- 172.21.15.84:0/1252939064 learned_addr learned my addr 172.21.15.84:0/1252939064 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:37.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.772+0000 7f4c0759e640 1 --1- 172.21.15.84:0/1252939064 >> v1:172.21.15.84:6789/0 conn(0x7f4c080c9ca0 0x7f4c080ad4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:41302/0 (socket says 172.21.15.84:41302) 2023-10-16T10:07:37.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.773+0000 7f4c0659c640 1 -- 172.21.15.84:0/1252939064 >> v1:172.21.15.157:6789/0 conn(0x7f4c0815ad70 legacy=0x7f4c08069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:37.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.773+0000 7f4c0659c640 1 -- 172.21.15.84:0/1252939064 >> v1:172.21.15.84:6789/0 conn(0x7f4c080c9ca0 legacy=0x7f4c080ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:37.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.773+0000 7f4c0659c640 1 -- 172.21.15.84:0/1252939064 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c080c9780 con 0x7f4c0806a4b0 2023-10-16T10:07:37.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.774+0000 7f4c0659c640 1 --2- 172.21.15.84:0/1252939064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0806a4b0 0x7f4c080ca3b0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009870 tx=0x7f4bf002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dba101c8e094898 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:37.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.774+0000 7f4c05d9b640 1 -- 172.21.15.84:0/1252939064 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0030860 con 0x7f4c0806a4b0 2023-10-16T10:07:37.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.774+0000 7f4c05d9b640 1 -- 172.21.15.84:0/1252939064 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf00309c0 con 0x7f4c0806a4b0 2023-10-16T10:07:37.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.774+0000 7f4c05d9b640 1 -- 172.21.15.84:0/1252939064 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0030d50 con 0x7f4c0806a4b0 2023-10-16T10:07:37.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.775+0000 7f4c0d20e640 1 -- 172.21.15.84:0/1252939064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0806a4b0 msgr2=0x7f4c080ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:37.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.775+0000 7f4c0d20e640 1 --2- 172.21.15.84:0/1252939064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0806a4b0 0x7f4c080ca3b0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009870 tx=0x7f4bf002b9a0 comp rx=0 tx=0).stop 2023-10-16T10:07:37.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.775+0000 7f4c0d20e640 1 -- 172.21.15.84:0/1252939064 shutdown_connections 2023-10-16T10:07:37.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.775+0000 7f4c0d20e640 1 --2- 172.21.15.84:0/1252939064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0806a4b0 0x7f4c080ca3b0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:37.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.775+0000 7f4c0d20e640 1 -- 172.21.15.84:0/1252939064 >> 172.21.15.84:0/1252939064 conn(0x7f4c080ce9b0 msgr2=0x7f4c080cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:37.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.775+0000 7f4c0d20e640 1 -- 172.21.15.84:0/1252939064 shutdown_connections 2023-10-16T10:07:37.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.775+0000 7f4c0d20e640 1 -- 172.21.15.84:0/1252939064 wait complete. 2023-10-16T10:07:37.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.777+0000 7f4c0d20e640 1 Processor -- start 2023-10-16T10:07:37.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.777+0000 7f4c0d20e640 1 -- start start 2023-10-16T10:07:37.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.778+0000 7f4c0d20e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0806a4b0 0x7f4c081f3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:37.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.778+0000 7f4c0d20e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c080c9ca0 0x7f4c080c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:37.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.778+0000 7f4c0d20e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0815ad70 0x7f4c080c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:37.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.778+0000 7f4c0d20e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c080aac80 con 0x7f4c0815ad70 2023-10-16T10:07:37.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.778+0000 7f4c0d20e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c080aa9a0 con 0x7f4c0806a4b0 2023-10-16T10:07:37.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.778+0000 7f4c0d20e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c081f3870 con 0x7f4c080c9ca0 2023-10-16T10:07:37.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.778+0000 7f4c0759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0815ad70 0x7f4c080c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:37.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.778+0000 7f4c0759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0815ad70 0x7f4c080c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51358/0 (socket says 172.21.15.84:51358) 2023-10-16T10:07:37.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.778+0000 7f4c0759e640 1 -- 172.21.15.84:0/375310089 learned_addr learned my addr 172.21.15.84:0/375310089 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:37.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.779+0000 7f4c0659c640 1 --2- 172.21.15.84:0/375310089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c080c9ca0 0x7f4c080c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:37.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.779+0000 7f4c0759e640 1 -- 172.21.15.84:0/375310089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c080c9ca0 msgr2=0x7f4c080c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:37.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.779+0000 7f4c0759e640 1 --2- 172.21.15.84:0/375310089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c080c9ca0 0x7f4c080c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:37.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.779+0000 7f4c06d9d640 1 --2- 172.21.15.84:0/375310089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0806a4b0 0x7f4c081f3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:37.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.779+0000 7f4c0759e640 1 -- 172.21.15.84:0/375310089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0806a4b0 msgr2=0x7f4c081f3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:37.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.779+0000 7f4c0759e640 1 --2- 172.21.15.84:0/375310089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0806a4b0 0x7f4c081f3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:37.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.779+0000 7f4c0759e640 1 -- 172.21.15.84:0/375310089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf00094d0 con 0x7f4c0815ad70 2023-10-16T10:07:37.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.779+0000 7f4c0759e640 1 --2- 172.21.15.84:0/375310089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0815ad70 0x7f4c080c8150 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000c430 tx=0x7f4c0000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:37.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.779+0000 7f4bff7fe640 1 -- 172.21.15.84:0/375310089 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c000074e0 con 0x7f4c0815ad70 2023-10-16T10:07:37.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.779+0000 7f4bff7fe640 1 -- 172.21.15.84:0/375310089 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c00007640 con 0x7f4c0815ad70 2023-10-16T10:07:37.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.779+0000 7f4bff7fe640 1 -- 172.21.15.84:0/375310089 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c000049e0 con 0x7f4c0815ad70 2023-10-16T10:07:37.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.779+0000 7f4c0d20e640 1 -- 172.21.15.84:0/375310089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c080a9d20 con 0x7f4c0815ad70 2023-10-16T10:07:37.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.780+0000 7f4c0d20e640 1 -- 172.21.15.84:0/375310089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c080aa2e0 con 0x7f4c0815ad70 2023-10-16T10:07:37.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.781+0000 7f4c0d20e640 1 -- 172.21.15.84:0/375310089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c080aaae0 con 0x7f4c0815ad70 2023-10-16T10:07:37.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.782+0000 7f4bff7fe640 1 -- 172.21.15.84:0/375310089 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4c00004b40 con 0x7f4c0815ad70 2023-10-16T10:07:37.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.783+0000 7f4bff7fe640 1 --2- 172.21.15.84:0/375310089 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f4be806af90 0x7f4be806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:37.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.783+0000 7f4bff7fe640 1 -- 172.21.15.84:0/375310089 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c00089190 con 0x7f4c0815ad70 2023-10-16T10:07:37.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.784+0000 7f4c06d9d640 1 --2- 172.21.15.84:0/375310089 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f4be806af90 0x7f4be806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:37.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.784+0000 7f4c06d9d640 1 --2- 172.21.15.84:0/375310089 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f4be806af90 0x7f4be806d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8007bf0 tx=0x7f4bf8007240 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:37.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:37.788+0000 7f4bff7fe640 1 -- 172.21.15.84:0/375310089 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4c080aaae0 con 0x7f4c0815ad70 2023-10-16T10:07:38.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.007+0000 7f4c0d20e640 1 -- 172.21.15.84:0/375310089 --> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c080ca8f0 con 0x7f4be806af90 2023-10-16T10:07:38.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.009+0000 7f4bff7fe640 1 -- 172.21.15.84:0/375310089 <== mgr.7794 v2:172.21.15.84:6832/4292025878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f4bf800c9d0 con 0x7f4be806af90 2023-10-16T10:07:38.011 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T10:07:38.011 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:07:38.011 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephfs" 2023-10-16T10:07:38.011 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:07:38.011 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T10:07:38.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.011+0000 7f4c0d20e640 1 -- 172.21.15.84:0/375310089 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f4be806af90 msgr2=0x7f4be806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:38.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.012+0000 7f4c0d20e640 1 --2- 172.21.15.84:0/375310089 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f4be806af90 0x7f4be806d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8007bf0 tx=0x7f4bf8007240 comp rx=0 tx=0).stop 2023-10-16T10:07:38.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.012+0000 7f4c0d20e640 1 -- 172.21.15.84:0/375310089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0815ad70 msgr2=0x7f4c080c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:38.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.012+0000 7f4c0d20e640 1 --2- 172.21.15.84:0/375310089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0815ad70 0x7f4c080c8150 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000c430 tx=0x7f4c0000c8f0 comp rx=0 tx=0).stop 2023-10-16T10:07:38.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.012+0000 7f4c0d20e640 1 -- 172.21.15.84:0/375310089 shutdown_connections 2023-10-16T10:07:38.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.012+0000 7f4c0d20e640 1 --2- 172.21.15.84:0/375310089 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f4be806af90 0x7f4be806d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:38.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.012+0000 7f4c0d20e640 1 --2- 172.21.15.84:0/375310089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0815ad70 0x7f4c080c8150 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:38.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.012+0000 7f4c0d20e640 1 --2- 172.21.15.84:0/375310089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c080c9ca0 0x7f4c080c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:38.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.012+0000 7f4c0d20e640 1 --2- 172.21.15.84:0/375310089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0806a4b0 0x7f4c081f3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:38.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.012+0000 7f4c0d20e640 1 -- 172.21.15.84:0/375310089 >> 172.21.15.84:0/375310089 conn(0x7f4c080ce9b0 msgr2=0x7f4c080ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:38.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.012+0000 7f4c0d20e640 1 -- 172.21.15.84:0/375310089 shutdown_connections 2023-10-16T10:07:38.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.012+0000 7f4c0d20e640 1 -- 172.21.15.84:0/375310089 wait complete. 2023-10-16T10:07:38.033 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T10:07:38.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.250+0000 7f6bc6f5a640 1 Processor -- start 2023-10-16T10:07:38.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.250+0000 7f6bc6f5a640 1 -- start start 2023-10-16T10:07:38.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.251+0000 7f6bc6f5a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6bc015ad70 0x7f6bc0069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:38.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.251+0000 7f6bc6f5a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6bc00c9ca0 0x7f6bc006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:38.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.251+0000 7f6bc6f5a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bc00cb030 con 0x7f6bc00ca3b0 2023-10-16T10:07:38.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.251+0000 7f6bc6f5a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bc00c91c0 con 0x7f6bc015ad70 2023-10-16T10:07:38.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.251+0000 7f6bc6f5a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bc00c94a0 con 0x7f6bc00c9ca0 2023-10-16T10:07:38.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.252+0000 7f6bc4cf6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6bc015ad70 0x7f6bc0069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:38.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.252+0000 7f6bc4cf6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6bc015ad70 0x7f6bc0069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51370/0 (socket says 172.21.15.84:51370) 2023-10-16T10:07:38.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.252+0000 7f6bc4cf6640 1 -- 172.21.15.84:0/2559906634 learned_addr learned my addr 172.21.15.84:0/2559906634 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:38.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.252+0000 7f6bbffff640 1 --2- 172.21.15.84:0/2559906634 >> v2:172.21.15.157:3300/0 conn(0x7f6bc00c9ca0 0x7f6bc006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:38.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.252+0000 7f6bc4cf6640 1 -- 172.21.15.84:0/2559906634 >> v2:172.21.15.157:3300/0 conn(0x7f6bc00c9ca0 msgr2=0x7f6bc006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:38.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.252+0000 7f6bc4cf6640 1 --2- 172.21.15.84:0/2559906634 >> v2:172.21.15.157:3300/0 conn(0x7f6bc00c9ca0 0x7f6bc006a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:38.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.252+0000 7f6bc4cf6640 1 -- 172.21.15.84:0/2559906634 >> v1:172.21.15.84:6789/0 conn(0x7f6bc00ca3b0 legacy=0x7f6bc00ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:38.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.252+0000 7f6bc4cf6640 1 -- 172.21.15.84:0/2559906634 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc00c9780 con 0x7f6bc015ad70 2023-10-16T10:07:38.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.252+0000 7f6bbffff640 1 --2- 172.21.15.84:0/2559906634 >> v2:172.21.15.157:3300/0 conn(0x7f6bc00c9ca0 0x7f6bc006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:07:38.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.252+0000 7f6bbf7fe640 1 -- 172.21.15.84:0/2559906634 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611680013 0 0) 0x7f6bc00cb030 con 0x7f6bc00ca3b0 2023-10-16T10:07:38.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.253+0000 7f6bc4cf6640 1 --2- 172.21.15.84:0/2559906634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bc015ad70 0x7f6bc0069da0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00ce870 tx=0x7f6bb002b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f26b6fcb0d57246 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:38.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.254+0000 7f6bbf7fe640 1 -- 172.21.15.84:0/2559906634 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0030870 con 0x7f6bc015ad70 2023-10-16T10:07:38.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.254+0000 7f6bbf7fe640 1 -- 172.21.15.84:0/2559906634 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb00309d0 con 0x7f6bc015ad70 2023-10-16T10:07:38.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.254+0000 7f6bbf7fe640 1 -- 172.21.15.84:0/2559906634 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0030d60 con 0x7f6bc015ad70 2023-10-16T10:07:38.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.254+0000 7f6bc6f5a640 1 -- 172.21.15.84:0/2559906634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bc015ad70 msgr2=0x7f6bc0069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:38.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.254+0000 7f6bc6f5a640 1 --2- 172.21.15.84:0/2559906634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bc015ad70 0x7f6bc0069da0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00ce870 tx=0x7f6bb002b9b0 comp rx=0 tx=0).stop 2023-10-16T10:07:38.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.254+0000 7f6bc6f5a640 1 -- 172.21.15.84:0/2559906634 shutdown_connections 2023-10-16T10:07:38.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.254+0000 7f6bc6f5a640 1 --2- 172.21.15.84:0/2559906634 >> v2:172.21.15.157:3300/0 conn(0x7f6bc00c9ca0 0x7f6bc006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:38.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.254+0000 7f6bc6f5a640 1 --2- 172.21.15.84:0/2559906634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bc015ad70 0x7f6bc0069da0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:38.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.254+0000 7f6bc6f5a640 1 -- 172.21.15.84:0/2559906634 >> 172.21.15.84:0/2559906634 conn(0x7f6bc00ce9b0 msgr2=0x7f6bc00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:38.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.254+0000 7f6bc6f5a640 1 -- 172.21.15.84:0/2559906634 shutdown_connections 2023-10-16T10:07:38.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.254+0000 7f6bc6f5a640 1 -- 172.21.15.84:0/2559906634 wait complete. 2023-10-16T10:07:38.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.256+0000 7f6bc6f5a640 1 Processor -- start 2023-10-16T10:07:38.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.256+0000 7f6bc6f5a640 1 -- start start 2023-10-16T10:07:38.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.257+0000 7f6bc6f5a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bc00c9ca0 0x7f6bc015d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:38.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.257+0000 7f6bc6f5a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bc00ca3b0 0x7f6bc00b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:38.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.257+0000 7f6bc6f5a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bc015ad70 0x7f6bc01f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:38.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.257+0000 7f6bc6f5a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc00b15d0 con 0x7f6bc015ad70 2023-10-16T10:07:38.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.257+0000 7f6bc6f5a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc015db60 con 0x7f6bc00ca3b0 2023-10-16T10:07:38.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.257+0000 7f6bc6f5a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc015e110 con 0x7f6bc00c9ca0 2023-10-16T10:07:38.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.258+0000 7f6bc54f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bc015ad70 0x7f6bc01f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:38.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.258+0000 7f6bc54f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bc015ad70 0x7f6bc01f22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51378/0 (socket says 172.21.15.84:51378) 2023-10-16T10:07:38.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.258+0000 7f6bc54f7640 1 -- 172.21.15.84:0/1371583142 learned_addr learned my addr 172.21.15.84:0/1371583142 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:38.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.258+0000 7f6bbffff640 1 --2- 172.21.15.84:0/1371583142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bc00ca3b0 0x7f6bc00b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:38.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.258+0000 7f6bc4cf6640 1 --2- 172.21.15.84:0/1371583142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bc00c9ca0 0x7f6bc015d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:38.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.258+0000 7f6bc54f7640 1 -- 172.21.15.84:0/1371583142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bc00c9ca0 msgr2=0x7f6bc015d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:38.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.258+0000 7f6bc54f7640 1 --2- 172.21.15.84:0/1371583142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bc00c9ca0 0x7f6bc015d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:38.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.258+0000 7f6bc54f7640 1 -- 172.21.15.84:0/1371583142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bc00ca3b0 msgr2=0x7f6bc00b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:38.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.258+0000 7f6bc54f7640 1 --2- 172.21.15.84:0/1371583142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bc00ca3b0 0x7f6bc00b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:38.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.258+0000 7f6bc54f7640 1 -- 172.21.15.84:0/1371583142 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb00094d0 con 0x7f6bc015ad70 2023-10-16T10:07:38.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.258+0000 7f6bc54f7640 1 --2- 172.21.15.84:0/1371583142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bc015ad70 0x7f6bc01f22a0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800c7e0 tx=0x7f6bb800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:38.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.259+0000 7f6bbd7fa640 1 -- 172.21.15.84:0/1371583142 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8003da0 con 0x7f6bc015ad70 2023-10-16T10:07:38.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.259+0000 7f6bbd7fa640 1 -- 172.21.15.84:0/1371583142 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb8005070 con 0x7f6bc015ad70 2023-10-16T10:07:38.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.259+0000 7f6bbd7fa640 1 -- 172.21.15.84:0/1371583142 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8005420 con 0x7f6bc015ad70 2023-10-16T10:07:38.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.259+0000 7f6bc6f5a640 1 -- 172.21.15.84:0/1371583142 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc00b3a70 con 0x7f6bc015ad70 2023-10-16T10:07:38.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.259+0000 7f6bc6f5a640 1 -- 172.21.15.84:0/1371583142 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc015ccf0 con 0x7f6bc015ad70 2023-10-16T10:07:38.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.260+0000 7f6bc6f5a640 1 -- 172.21.15.84:0/1371583142 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc00b1740 con 0x7f6bc015ad70 2023-10-16T10:07:38.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.261+0000 7f6bbd7fa640 1 -- 172.21.15.84:0/1371583142 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 59) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6bb8005580 con 0x7f6bc015ad70 2023-10-16T10:07:38.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.262+0000 7f6bbd7fa640 1 --2- 172.21.15.84:0/1371583142 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f6bac06af90 0x7f6bac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:38.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.262+0000 7f6bbd7fa640 1 -- 172.21.15.84:0/1371583142 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bb8089110 con 0x7f6bc015ad70 2023-10-16T10:07:38.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.262+0000 7f6bc4cf6640 1 --2- 172.21.15.84:0/1371583142 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f6bac06af90 0x7f6bac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:38.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.263+0000 7f6bc4cf6640 1 --2- 172.21.15.84:0/1371583142 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f6bac06af90 0x7f6bac06d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0036000 tx=0x7f6bb003b040 comp rx=0 tx=0).ready entity=mgr.7794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:38.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.267+0000 7f6bbd7fa640 1 -- 172.21.15.84:0/1371583142 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6bc00b1740 con 0x7f6bc015ad70 2023-10-16T10:07:38.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:38.510+0000 7f6bc6f5a640 1 -- 172.21.15.84:0/1371583142 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f6bc00b1710 con 0x7f6bc015ad70 2023-10-16T10:07:39.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.423+0000 7f6bbd7fa640 1 -- 172.21.15.84:0/1371583142 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v60) v1 ==== 92+0+0 (secure 0 0 0) 0x7f6bb8054010 con 0x7f6bc015ad70 2023-10-16T10:07:39.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.427+0000 7f6bc6f5a640 1 -- 172.21.15.84:0/1371583142 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f6bac06af90 msgr2=0x7f6bac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:39.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.427+0000 7f6bc6f5a640 1 --2- 172.21.15.84:0/1371583142 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f6bac06af90 0x7f6bac06d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0036000 tx=0x7f6bb003b040 comp rx=0 tx=0).stop 2023-10-16T10:07:39.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.427+0000 7f6bc6f5a640 1 -- 172.21.15.84:0/1371583142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bc015ad70 msgr2=0x7f6bc01f22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:39.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.427+0000 7f6bc6f5a640 1 --2- 172.21.15.84:0/1371583142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bc015ad70 0x7f6bc01f22a0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800c7e0 tx=0x7f6bb800cca0 comp rx=0 tx=0).stop 2023-10-16T10:07:39.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.428+0000 7f6bc6f5a640 1 -- 172.21.15.84:0/1371583142 shutdown_connections 2023-10-16T10:07:39.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.428+0000 7f6bc6f5a640 1 --2- 172.21.15.84:0/1371583142 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f6bac06af90 0x7f6bac06d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:39.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.428+0000 7f6bc6f5a640 1 --2- 172.21.15.84:0/1371583142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bc015ad70 0x7f6bc01f22a0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:39.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.428+0000 7f6bc6f5a640 1 --2- 172.21.15.84:0/1371583142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bc00ca3b0 0x7f6bc00b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:39.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.428+0000 7f6bc6f5a640 1 --2- 172.21.15.84:0/1371583142 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bc00c9ca0 0x7f6bc015d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:39.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.428+0000 7f6bc6f5a640 1 -- 172.21.15.84:0/1371583142 >> 172.21.15.84:0/1371583142 conn(0x7f6bc00ce9b0 msgr2=0x7f6bc00ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:39.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.428+0000 7f6bc6f5a640 1 -- 172.21.15.84:0/1371583142 shutdown_connections 2023-10-16T10:07:39.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.428+0000 7f6bc6f5a640 1 -- 172.21.15.84:0/1371583142 wait complete. 2023-10-16T10:07:39.454 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T10:07:39.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.658+0000 7f108df61640 1 Processor -- start 2023-10-16T10:07:39.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.658+0000 7f108df61640 1 -- start start 2023-10-16T10:07:39.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.660+0000 7f108df61640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f108806b660 0x7f108806ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:39.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.660+0000 7f108df61640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10880cc6b0 con 0x7f10880c9860 2023-10-16T10:07:39.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.660+0000 7f108df61640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10881599b0 con 0x7f10881428c0 2023-10-16T10:07:39.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.660+0000 7f108df61640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108806c290 con 0x7f108806b660 2023-10-16T10:07:39.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.661+0000 7f1087fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f10880c9860 0x7f10880ccb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:41342/0 (socket says 172.21.15.84:41342) 2023-10-16T10:07:39.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.661+0000 7f1087fff640 1 -- 172.21.15.84:0/1028896558 learned_addr learned my addr 172.21.15.84:0/1028896558 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:39.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.661+0000 7f10877fe640 1 --2- 172.21.15.84:0/1028896558 >> v2:172.21.15.157:3300/0 conn(0x7f108806b660 0x7f108806ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:39.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.661+0000 7f1086ffd640 1 -- 172.21.15.84:0/1028896558 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511006419 0 0) 0x7f10880cc6b0 con 0x7f10880c9860 2023-10-16T10:07:39.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.661+0000 7f1086ffd640 1 -- 172.21.15.84:0/1028896558 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1074003440 con 0x7f10880c9860 2023-10-16T10:07:39.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.662+0000 7f1086ffd640 1 -- 172.21.15.84:0/1028896558 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 602675998 0 0) 0x7f1074003440 con 0x7f10880c9860 2023-10-16T10:07:39.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.662+0000 7f1086ffd640 1 -- 172.21.15.84:0/1028896558 >> v2:172.21.15.157:3300/0 conn(0x7f108806b660 msgr2=0x7f108806ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:39.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.662+0000 7f1086ffd640 1 --2- 172.21.15.84:0/1028896558 >> v2:172.21.15.157:3300/0 conn(0x7f108806b660 0x7f108806ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:39.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.662+0000 7f1086ffd640 1 -- 172.21.15.84:0/1028896558 >> v1:172.21.15.157:6789/0 conn(0x7f10881428c0 legacy=0x7f10880c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:39.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.662+0000 7f1086ffd640 1 -- 172.21.15.84:0/1028896558 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108806c570 con 0x7f10880c9860 2023-10-16T10:07:39.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.662+0000 7f1086ffd640 1 -- 172.21.15.84:0/1028896558 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f10800019f0 con 0x7f10880c9860 2023-10-16T10:07:39.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.662+0000 7f10877fe640 1 --2- 172.21.15.84:0/1028896558 >> v2:172.21.15.157:3300/0 conn(0x7f108806b660 0x7f108806ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:07:39.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.662+0000 7f1086ffd640 1 -- 172.21.15.84:0/1028896558 >> v1:172.21.15.84:6789/0 conn(0x7f10880c9860 legacy=0x7f10880ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:39.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.662+0000 7f1086ffd640 1 --2- 172.21.15.84:0/1028896558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1074003ab0 0x7f1074003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:39.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.662+0000 7f1086ffd640 1 --2- 172.21.15.84:0/1028896558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f107400ac50 0x7f107400d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:39.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.662+0000 7f1086ffd640 1 --2- 172.21.15.84:0/1028896558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f107400d760 0x7f107400fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:39.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.662+0000 7f1086ffd640 1 -- 172.21.15.84:0/1028896558 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108806c290 con 0x7f107400ac50 2023-10-16T10:07:39.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.662+0000 7f1086ffd640 1 -- 172.21.15.84:0/1028896558 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10740104d0 con 0x7f1074003ab0 2023-10-16T10:07:39.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.662+0000 7f1086ffd640 1 -- 172.21.15.84:0/1028896558 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1074010690 con 0x7f107400d760 2023-10-16T10:07:39.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.662+0000 7f107ffff640 1 --2- 172.21.15.84:0/1028896558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f107400ac50 0x7f107400d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:39.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.663+0000 7f10877fe640 1 --2- 172.21.15.84:0/1028896558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1074003ab0 0x7f1074003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:39.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.663+0000 7f1087fff640 1 --2- 172.21.15.84:0/1028896558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f107400d760 0x7f107400fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:39.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.663+0000 7f107ffff640 1 -- 172.21.15.84:0/1028896558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f107400d760 msgr2=0x7f107400fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:39.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.663+0000 7f107ffff640 1 --2- 172.21.15.84:0/1028896558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f107400d760 0x7f107400fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:39.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.663+0000 7f107ffff640 1 -- 172.21.15.84:0/1028896558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1074003ab0 msgr2=0x7f1074003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:39.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.663+0000 7f107ffff640 1 --2- 172.21.15.84:0/1028896558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1074003ab0 0x7f1074003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:39.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.663+0000 7f107ffff640 1 -- 172.21.15.84:0/1028896558 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1074003440 con 0x7f107400ac50 2023-10-16T10:07:39.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.663+0000 7f1087fff640 1 -- 172.21.15.84:0/1028896558 reap_dead start 2023-10-16T10:07:39.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.664+0000 7f107ffff640 1 --2- 172.21.15.84:0/1028896558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f107400ac50 0x7f107400d040 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f1070001750 tx=0x7f10740036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549f752c8f0f01c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:39.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.664+0000 7f1086ffd640 1 -- 172.21.15.84:0/1028896558 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10700052d0 con 0x7f107400ac50 2023-10-16T10:07:39.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.664+0000 7f1086ffd640 1 -- 172.21.15.84:0/1028896558 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107002e050 con 0x7f107400ac50 2023-10-16T10:07:39.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.664+0000 7f1086ffd640 1 -- 172.21.15.84:0/1028896558 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1070003830 con 0x7f107400ac50 2023-10-16T10:07:39.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.665+0000 7f108df61640 1 -- 172.21.15.84:0/1028896558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f107400ac50 msgr2=0x7f107400d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:39.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.665+0000 7f108df61640 1 --2- 172.21.15.84:0/1028896558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f107400ac50 0x7f107400d040 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f1070001750 tx=0x7f10740036f0 comp rx=0 tx=0).stop 2023-10-16T10:07:39.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.665+0000 7f108df61640 1 -- 172.21.15.84:0/1028896558 shutdown_connections 2023-10-16T10:07:39.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.665+0000 7f108df61640 1 --2- 172.21.15.84:0/1028896558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f107400ac50 0x7f107400d040 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:39.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.665+0000 7f108df61640 1 -- 172.21.15.84:0/1028896558 >> 172.21.15.84:0/1028896558 conn(0x7f10880ce9b0 msgr2=0x7f10880cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:39.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.666+0000 7f108df61640 1 -- 172.21.15.84:0/1028896558 shutdown_connections 2023-10-16T10:07:39.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.666+0000 7f108df61640 1 -- 172.21.15.84:0/1028896558 wait complete. 2023-10-16T10:07:39.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.668+0000 7f108df61640 1 Processor -- start 2023-10-16T10:07:39.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.669+0000 7f108df61640 1 -- start start 2023-10-16T10:07:39.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.670+0000 7f108df61640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f107400ac50 0x7f1088158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:39.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.670+0000 7f108df61640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10880b2960 0x7f10880b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:39.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.670+0000 7f108df61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10881e1ad0 0x7f10881e1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:39.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.670+0000 7f108df61640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10881e24c0 con 0x7f10880b2960 2023-10-16T10:07:39.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.670+0000 7f108df61640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1088159530 con 0x7f107400ac50 2023-10-16T10:07:39.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.670+0000 7f108df61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10880b3420 con 0x7f10881e1ad0 2023-10-16T10:07:39.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.671+0000 7f107ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10880b2960 0x7f10880b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:39.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.671+0000 7f107ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10880b2960 0x7f10880b2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51400/0 (socket says 172.21.15.84:51400) 2023-10-16T10:07:39.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.671+0000 7f107ffff640 1 -- 172.21.15.84:0/2556038186 learned_addr learned my addr 172.21.15.84:0/2556038186 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:39.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.671+0000 7f10877fe640 1 --2- 172.21.15.84:0/2556038186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f107400ac50 0x7f1088158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:39.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.671+0000 7f1087fff640 1 --2- 172.21.15.84:0/2556038186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10881e1ad0 0x7f10881e1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:39.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.671+0000 7f107ffff640 1 -- 172.21.15.84:0/2556038186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10881e1ad0 msgr2=0x7f10881e1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:39.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.671+0000 7f107ffff640 1 --2- 172.21.15.84:0/2556038186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10881e1ad0 0x7f10881e1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:39.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.671+0000 7f107ffff640 1 -- 172.21.15.84:0/2556038186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f107400ac50 msgr2=0x7f1088158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:39.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.671+0000 7f107ffff640 1 --2- 172.21.15.84:0/2556038186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f107400ac50 0x7f1088158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:39.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.671+0000 7f107ffff640 1 -- 172.21.15.84:0/2556038186 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10880b3700 con 0x7f10880b2960 2023-10-16T10:07:39.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.671+0000 7f107ffff640 1 --2- 172.21.15.84:0/2556038186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10880b2960 0x7f10880b2dc0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f107002bc30 tx=0x7f1070001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:39.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.672+0000 7f1084ff9640 1 -- 172.21.15.84:0/2556038186 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10700076c0 con 0x7f10880b2960 2023-10-16T10:07:39.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.672+0000 7f1084ff9640 1 -- 172.21.15.84:0/2556038186 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1070007a30 con 0x7f10880b2960 2023-10-16T10:07:39.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.672+0000 7f1084ff9640 1 -- 172.21.15.84:0/2556038186 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1070006c20 con 0x7f10880b2960 2023-10-16T10:07:39.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.672+0000 7f108df61640 1 -- 172.21.15.84:0/2556038186 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10880b39e0 con 0x7f10880b2960 2023-10-16T10:07:39.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.672+0000 7f108df61640 1 -- 172.21.15.84:0/2556038186 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10880b3fa0 con 0x7f10880b2960 2023-10-16T10:07:39.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.673+0000 7f108df61640 1 -- 172.21.15.84:0/2556038186 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10880b4130 con 0x7f10880b2960 2023-10-16T10:07:39.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.674+0000 7f1084ff9640 1 -- 172.21.15.84:0/2556038186 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 60) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f10700035f0 con 0x7f10880b2960 2023-10-16T10:07:39.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.675+0000 7f1084ff9640 1 --2- 172.21.15.84:0/2556038186 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f106806b010 0x7f106806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:39.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.675+0000 7f1084ff9640 1 -- 172.21.15.84:0/2556038186 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10700ab660 con 0x7f10880b2960 2023-10-16T10:07:39.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.675+0000 7f10877fe640 1 -- 172.21.15.84:0/2556038186 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f106806b010 msgr2=0x7f106806d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4292025878 2023-10-16T10:07:39.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.675+0000 7f10877fe640 1 --2- 172.21.15.84:0/2556038186 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f106806b010 0x7f106806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:07:39.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.679+0000 7f1084ff9640 1 -- 172.21.15.84:0/2556038186 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1070076490 con 0x7f10880b2960 2023-10-16T10:07:39.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.871+0000 7f108df61640 1 -- 172.21.15.84:0/2556038186 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f108804e160 con 0x7f10880b2960 2023-10-16T10:07:39.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.876+0000 7f10877fe640 1 -- 172.21.15.84:0/2556038186 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f106806b010 msgr2=0x7f106806d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4292025878 2023-10-16T10:07:39.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.876+0000 7f10877fe640 1 --2- 172.21.15.84:0/2556038186 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f106806b010 0x7f106806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:07:39.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.876+0000 7f1084ff9640 1 -- 172.21.15.84:0/2556038186 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v41)=0 v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1070079ab0 con 0x7f10880b2960 2023-10-16T10:07:39.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.879+0000 7f108df61640 1 -- 172.21.15.84:0/2556038186 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f106806b010 msgr2=0x7f106806d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:07:39.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.879+0000 7f108df61640 1 --2- 172.21.15.84:0/2556038186 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f106806b010 0x7f106806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:39.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.879+0000 7f108df61640 1 -- 172.21.15.84:0/2556038186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10880b2960 msgr2=0x7f10880b2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:39.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.879+0000 7f108df61640 1 --2- 172.21.15.84:0/2556038186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10880b2960 0x7f10880b2dc0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f107002bc30 tx=0x7f1070001120 comp rx=0 tx=0).stop 2023-10-16T10:07:39.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.879+0000 7f108df61640 1 -- 172.21.15.84:0/2556038186 shutdown_connections 2023-10-16T10:07:39.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.879+0000 7f108df61640 1 --2- 172.21.15.84:0/2556038186 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f106806b010 0x7f106806d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:39.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.879+0000 7f108df61640 1 --2- 172.21.15.84:0/2556038186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10881e1ad0 0x7f10881e1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:39.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.879+0000 7f108df61640 1 --2- 172.21.15.84:0/2556038186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10880b2960 0x7f10880b2dc0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:39.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.879+0000 7f108df61640 1 --2- 172.21.15.84:0/2556038186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f107400ac50 0x7f1088158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:39.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.879+0000 7f108df61640 1 -- 172.21.15.84:0/2556038186 >> 172.21.15.84:0/2556038186 conn(0x7f10880ce9b0 msgr2=0x7f1088070060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:39.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.879+0000 7f108df61640 1 -- 172.21.15.84:0/2556038186 shutdown_connections 2023-10-16T10:07:39.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:39.879+0000 7f108df61640 1 -- 172.21.15.84:0/2556038186 wait complete. 2023-10-16T10:07:39.898 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T10:07:39.941 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:39.940+0000 7f583797f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:07:39.974 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:39.972+0000 7ff7e5446040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:07:39.996 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:39.994+0000 7f583797f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:07:40.031 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:40.029+0000 7ff7e5446040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:07:40.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.097+0000 7f0a42cec640 1 Processor -- start 2023-10-16T10:07:40.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.097+0000 7f0a42cec640 1 -- start start 2023-10-16T10:07:40.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.098+0000 7f0a42cec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3c0b8480 0x7f0a3c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:40.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.098+0000 7f0a42cec640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3c15d150 con 0x7f0a3c0d0be0 2023-10-16T10:07:40.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.098+0000 7f0a42cec640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3c0d1180 con 0x7f0a3c15cd70 2023-10-16T10:07:40.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.098+0000 7f0a42cec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0d1460 con 0x7f0a3c0b8480 2023-10-16T10:07:40.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.099+0000 7f0a424eb640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0a3c0d0be0 0x7f0a3c0b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:41354/0 (socket says 172.21.15.84:41354) 2023-10-16T10:07:40.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.099+0000 7f0a424eb640 1 -- 172.21.15.84:0/1215992454 learned_addr learned my addr 172.21.15.84:0/1215992454 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:07:40.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.099+0000 7f0a414e9640 1 --2- 172.21.15.84:0/1215992454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3c0b8480 0x7f0a3c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:40.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.099+0000 7f0a40ce8640 1 -- 172.21.15.84:0/1215992454 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1212531112 0 0) 0x7f0a3c15d150 con 0x7f0a3c0d0be0 2023-10-16T10:07:40.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.099+0000 7f0a40ce8640 1 -- 172.21.15.84:0/1215992454 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a28003520 con 0x7f0a3c0d0be0 2023-10-16T10:07:40.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.099+0000 7f0a40ce8640 1 -- 172.21.15.84:0/1215992454 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2209669484 0 0) 0x7f0a28003520 con 0x7f0a3c0d0be0 2023-10-16T10:07:40.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.099+0000 7f0a40ce8640 1 -- 172.21.15.84:0/1215992454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3c0b8480 msgr2=0x7f0a3c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:40.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.099+0000 7f0a40ce8640 1 --2- 172.21.15.84:0/1215992454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3c0b8480 0x7f0a3c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:40.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.100+0000 7f0a40ce8640 1 -- 172.21.15.84:0/1215992454 >> v1:172.21.15.157:6789/0 conn(0x7f0a3c15cd70 legacy=0x7f0a3c0d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:40.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.100+0000 7f0a40ce8640 1 -- 172.21.15.84:0/1215992454 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c0d1740 con 0x7f0a3c0d0be0 2023-10-16T10:07:40.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.100+0000 7f0a40ce8640 1 -- 172.21.15.84:0/1215992454 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f0a340019f0 con 0x7f0a3c0d0be0 2023-10-16T10:07:40.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.100+0000 7f0a40ce8640 1 -- 172.21.15.84:0/1215992454 >> v1:172.21.15.84:6789/0 conn(0x7f0a3c0d0be0 legacy=0x7f0a3c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:40.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.100+0000 7f0a40ce8640 1 --2- 172.21.15.84:0/1215992454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a280039a0 0x7f0a28003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:40.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.100+0000 7f0a40ce8640 1 --2- 172.21.15.84:0/1215992454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3c0b8480 0x7f0a280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:40.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.100+0000 7f0a40ce8640 1 --2- 172.21.15.84:0/1215992454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a28004d00 0x7f0a28005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:40.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.100+0000 7f0a40ce8640 1 -- 172.21.15.84:0/1215992454 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0d1460 con 0x7f0a280039a0 2023-10-16T10:07:40.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.100+0000 7f0a40ce8640 1 -- 172.21.15.84:0/1215992454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2800ba30 con 0x7f0a28004d00 2023-10-16T10:07:40.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.100+0000 7f0a40ce8640 1 -- 172.21.15.84:0/1215992454 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2800bbf0 con 0x7f0a3c0b8480 2023-10-16T10:07:40.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.101+0000 7f0a41cea640 1 --2- 172.21.15.84:0/1215992454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a280039a0 0x7f0a28003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:40.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.101+0000 7f0a414e9640 1 --2- 172.21.15.84:0/1215992454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3c0b8480 0x7f0a280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:40.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.101+0000 7f0a424eb640 1 --2- 172.21.15.84:0/1215992454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a28004d00 0x7f0a28005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:40.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.101+0000 7f0a41cea640 1 -- 172.21.15.84:0/1215992454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3c0b8480 msgr2=0x7f0a280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:40.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.101+0000 7f0a41cea640 1 --2- 172.21.15.84:0/1215992454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3c0b8480 0x7f0a280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:40.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.101+0000 7f0a41cea640 1 -- 172.21.15.84:0/1215992454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a28004d00 msgr2=0x7f0a28005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:40.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.101+0000 7f0a41cea640 1 --2- 172.21.15.84:0/1215992454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a28004d00 0x7f0a28005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:40.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.101+0000 7f0a41cea640 1 -- 172.21.15.84:0/1215992454 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a28003520 con 0x7f0a280039a0 2023-10-16T10:07:40.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.102+0000 7f0a41cea640 1 --2- 172.21.15.84:0/1215992454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a280039a0 0x7f0a28003dc0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f0a38001720 tx=0x7f0a280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6685153abc56c3d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:40.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.102+0000 7f0a40ce8640 1 -- 172.21.15.84:0/1215992454 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a380052d0 con 0x7f0a280039a0 2023-10-16T10:07:40.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.102+0000 7f0a40ce8640 1 -- 172.21.15.84:0/1215992454 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3802e050 con 0x7f0a280039a0 2023-10-16T10:07:40.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.102+0000 7f0a40ce8640 1 -- 172.21.15.84:0/1215992454 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a380035f0 con 0x7f0a280039a0 2023-10-16T10:07:40.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.103+0000 7f0a42cec640 1 -- 172.21.15.84:0/1215992454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a280039a0 msgr2=0x7f0a28003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:40.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.103+0000 7f0a42cec640 1 --2- 172.21.15.84:0/1215992454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a280039a0 0x7f0a28003dc0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f0a38001720 tx=0x7f0a280057f0 comp rx=0 tx=0).stop 2023-10-16T10:07:40.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.103+0000 7f0a424eb640 1 -- 172.21.15.84:0/1215992454 reap_dead start 2023-10-16T10:07:40.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.103+0000 7f0a42cec640 1 -- 172.21.15.84:0/1215992454 shutdown_connections 2023-10-16T10:07:40.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.103+0000 7f0a42cec640 1 -- 172.21.15.84:0/1215992454 >> 172.21.15.84:0/1215992454 conn(0x7f0a3c0cf570 msgr2=0x7f0a3c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:40.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.103+0000 7f0a42cec640 1 -- 172.21.15.84:0/1215992454 shutdown_connections 2023-10-16T10:07:40.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.103+0000 7f0a42cec640 1 -- 172.21.15.84:0/1215992454 wait complete. 2023-10-16T10:07:40.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.105+0000 7f0a42cec640 1 Processor -- start 2023-10-16T10:07:40.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.105+0000 7f0a42cec640 1 -- start start 2023-10-16T10:07:40.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.106+0000 7f0a42cec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a280039a0 0x7f0a3c0cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:40.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.106+0000 7f0a42cec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3c15d780 0x7f0a3c151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:40.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.106+0000 7f0a42cec640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a3c15dc00 0x7f0a3c152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:40.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.106+0000 7f0a42cec640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0c9ce0 con 0x7f0a280039a0 2023-10-16T10:07:40.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.106+0000 7f0a42cec640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0c9f30 con 0x7f0a3c15dc00 2023-10-16T10:07:40.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.106+0000 7f0a42cec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0ca210 con 0x7f0a3c15d780 2023-10-16T10:07:40.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.106+0000 7f0a41cea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a280039a0 0x7f0a3c0cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:40.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.106+0000 7f0a41cea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a280039a0 0x7f0a3c0cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51424/0 (socket says 172.21.15.84:51424) 2023-10-16T10:07:40.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.106+0000 7f0a41cea640 1 -- 172.21.15.84:0/1031196910 learned_addr learned my addr 172.21.15.84:0/1031196910 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:40.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.106+0000 7f0a414e9640 1 --2- 172.21.15.84:0/1031196910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3c15d780 0x7f0a3c151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:40.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.107+0000 7f0a424eb640 1 --2- 172.21.15.84:0/1031196910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a3c15dc00 0x7f0a3c152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:40.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.107+0000 7f0a41cea640 1 -- 172.21.15.84:0/1031196910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3c15d780 msgr2=0x7f0a3c151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:40.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.107+0000 7f0a41cea640 1 --2- 172.21.15.84:0/1031196910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3c15d780 0x7f0a3c151be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:40.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.107+0000 7f0a41cea640 1 -- 172.21.15.84:0/1031196910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a3c15dc00 msgr2=0x7f0a3c152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:40.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.107+0000 7f0a41cea640 1 --2- 172.21.15.84:0/1031196910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a3c15dc00 0x7f0a3c152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:40.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.107+0000 7f0a41cea640 1 -- 172.21.15.84:0/1031196910 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c0ca4f0 con 0x7f0a280039a0 2023-10-16T10:07:40.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.107+0000 7f0a41cea640 1 --2- 172.21.15.84:0/1031196910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a280039a0 0x7f0a3c0cc4e0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f0a38008fd0 tx=0x7f0a38001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:40.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.107+0000 7f0a327fc640 1 -- 172.21.15.84:0/1031196910 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a380076c0 con 0x7f0a280039a0 2023-10-16T10:07:40.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.107+0000 7f0a327fc640 1 -- 172.21.15.84:0/1031196910 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a38007a30 con 0x7f0a280039a0 2023-10-16T10:07:40.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.107+0000 7f0a327fc640 1 -- 172.21.15.84:0/1031196910 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a38006be0 con 0x7f0a280039a0 2023-10-16T10:07:40.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.107+0000 7f0a42cec640 1 -- 172.21.15.84:0/1031196910 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3c0ca7d0 con 0x7f0a280039a0 2023-10-16T10:07:40.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.107+0000 7f0a42cec640 1 -- 172.21.15.84:0/1031196910 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3c0ccc00 con 0x7f0a280039a0 2023-10-16T10:07:40.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.108+0000 7f0a42cec640 1 -- 172.21.15.84:0/1031196910 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3c04e9b0 con 0x7f0a280039a0 2023-10-16T10:07:40.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.109+0000 7f0a327fc640 1 -- 172.21.15.84:0/1031196910 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 60) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f0a38006d40 con 0x7f0a280039a0 2023-10-16T10:07:40.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.110+0000 7f0a327fc640 1 --2- 172.21.15.84:0/1031196910 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0a2006afe0 0x7f0a2006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:40.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.110+0000 7f0a327fc640 1 -- 172.21.15.84:0/1031196910 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a380ab8b0 con 0x7f0a280039a0 2023-10-16T10:07:40.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.111+0000 7f0a414e9640 1 -- 172.21.15.84:0/1031196910 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0a2006afe0 msgr2=0x7f0a2006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4292025878 2023-10-16T10:07:40.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.111+0000 7f0a414e9640 1 --2- 172.21.15.84:0/1031196910 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0a2006afe0 0x7f0a2006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:07:40.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.118+0000 7f0a327fc640 1 -- 172.21.15.84:0/1031196910 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0a38076850 con 0x7f0a280039a0 2023-10-16T10:07:40.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.310+0000 7f0a42cec640 1 -- 172.21.15.84:0/1031196910 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f0a3c0ccc00 con 0x7f0a280039a0 2023-10-16T10:07:40.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.312+0000 7f0a414e9640 1 -- 172.21.15.84:0/1031196910 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0a2006afe0 msgr2=0x7f0a2006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4292025878 2023-10-16T10:07:40.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.312+0000 7f0a414e9640 1 --2- 172.21.15.84:0/1031196910 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0a2006afe0 0x7f0a2006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:07:40.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.315+0000 7f0a327fc640 1 -- 172.21.15.84:0/1031196910 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v42)=0 v42) v1 ==== 133+0+0 (secure 0 0 0) 0x7f0a3c0ccc00 con 0x7f0a280039a0 2023-10-16T10:07:40.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.317+0000 7f0a42cec640 1 -- 172.21.15.84:0/1031196910 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0a2006afe0 msgr2=0x7f0a2006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:07:40.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.317+0000 7f0a42cec640 1 --2- 172.21.15.84:0/1031196910 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0a2006afe0 0x7f0a2006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:40.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.317+0000 7f0a42cec640 1 -- 172.21.15.84:0/1031196910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a280039a0 msgr2=0x7f0a3c0cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:40.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.317+0000 7f0a42cec640 1 --2- 172.21.15.84:0/1031196910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a280039a0 0x7f0a3c0cc4e0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f0a38008fd0 tx=0x7f0a38001120 comp rx=0 tx=0).stop 2023-10-16T10:07:40.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.317+0000 7f0a42cec640 1 -- 172.21.15.84:0/1031196910 shutdown_connections 2023-10-16T10:07:40.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.317+0000 7f0a42cec640 1 --2- 172.21.15.84:0/1031196910 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7f0a2006afe0 0x7f0a2006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:40.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.317+0000 7f0a42cec640 1 --2- 172.21.15.84:0/1031196910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a3c15dc00 0x7f0a3c152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:40.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.317+0000 7f0a42cec640 1 --2- 172.21.15.84:0/1031196910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3c15d780 0x7f0a3c151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:40.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.317+0000 7f0a42cec640 1 --2- 172.21.15.84:0/1031196910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a280039a0 0x7f0a3c0cc4e0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:40.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.318+0000 7f0a42cec640 1 -- 172.21.15.84:0/1031196910 >> 172.21.15.84:0/1031196910 conn(0x7f0a3c0cf570 msgr2=0x7f0a3c0b4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:40.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.318+0000 7f0a42cec640 1 -- 172.21.15.84:0/1031196910 shutdown_connections 2023-10-16T10:07:40.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.318+0000 7f0a42cec640 1 -- 172.21.15.84:0/1031196910 wait complete. 2023-10-16T10:07:40.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:40.337 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-16T10:07:40.394 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-16T10:07:40.449 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:40.447+0000 7f583797f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:07:40.500 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:40.499+0000 7f583797f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:07:40.505 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:40.504+0000 7ff7e5446040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:07:40.550 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:40.549+0000 7f583797f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:07:40.560 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:40.558+0000 7ff7e5446040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:07:40.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.588+0000 7fc610a21640 1 Processor -- start 2023-10-16T10:07:40.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.588+0000 7fc610a21640 1 -- start start 2023-10-16T10:07:40.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.590+0000 7fc610a21640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc60c159ab0 0x7fc60c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:40.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.590+0000 7fc610a21640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc60c06c140 0x7fc60c06c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:40.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.590+0000 7fc610a21640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc60c06cad0 con 0x7fc60c159e90 2023-10-16T10:07:40.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.590+0000 7fc610a21640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc60c15d890 con 0x7fc60c06c140 2023-10-16T10:07:40.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.590+0000 7fc610a21640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc60c070360 con 0x7fc60c159ab0 2023-10-16T10:07:40.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.590+0000 7fc60ad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc60c06c140 0x7fc60c06c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:40.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.590+0000 7fc60ad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc60c06c140 0x7fc60c06c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51434/0 (socket says 172.21.15.84:51434) 2023-10-16T10:07:40.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.590+0000 7fc60ad9d640 1 -- 172.21.15.84:0/4013956874 learned_addr learned my addr 172.21.15.84:0/4013956874 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:40.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.590+0000 7fc60a59c640 1 --2- 172.21.15.84:0/4013956874 >> v2:172.21.15.157:3300/0 conn(0x7fc60c159ab0 0x7fc60c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:40.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.591+0000 7fc60ad9d640 1 -- 172.21.15.84:0/4013956874 >> v2:172.21.15.157:3300/0 conn(0x7fc60c159ab0 msgr2=0x7fc60c0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:40.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.591+0000 7fc60ad9d640 1 --2- 172.21.15.84:0/4013956874 >> v2:172.21.15.157:3300/0 conn(0x7fc60c159ab0 0x7fc60c0ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:40.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.591+0000 7fc60ad9d640 1 -- 172.21.15.84:0/4013956874 >> v1:172.21.15.157:6789/0 conn(0x7fc60c159e90 legacy=0x7fc60c06fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:40.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.591+0000 7fc60ad9d640 1 -- 172.21.15.84:0/4013956874 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c0cd1f0 con 0x7fc60c06c140 2023-10-16T10:07:40.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.592+0000 7fc60ad9d640 1 --2- 172.21.15.84:0/4013956874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc60c06c140 0x7fc60c06c590 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0cfa00 tx=0x7fc60402bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f85c3c94fdc7fcc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:40.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.592+0000 7fc609d9b640 1 -- 172.21.15.84:0/4013956874 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60402c960 con 0x7fc60c06c140 2023-10-16T10:07:40.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.592+0000 7fc609d9b640 1 -- 172.21.15.84:0/4013956874 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc60402cac0 con 0x7fc60c06c140 2023-10-16T10:07:40.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.592+0000 7fc609d9b640 1 -- 172.21.15.84:0/4013956874 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60402ce50 con 0x7fc60c06c140 2023-10-16T10:07:40.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.592+0000 7fc610a21640 1 -- 172.21.15.84:0/4013956874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc60c06c140 msgr2=0x7fc60c06c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:40.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.592+0000 7fc610a21640 1 --2- 172.21.15.84:0/4013956874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc60c06c140 0x7fc60c06c590 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0cfa00 tx=0x7fc60402bdc0 comp rx=0 tx=0).stop 2023-10-16T10:07:40.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.593+0000 7fc610a21640 1 -- 172.21.15.84:0/4013956874 shutdown_connections 2023-10-16T10:07:40.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.593+0000 7fc610a21640 1 --2- 172.21.15.84:0/4013956874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc60c06c140 0x7fc60c06c590 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:40.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.593+0000 7fc610a21640 1 --2- 172.21.15.84:0/4013956874 >> v2:172.21.15.157:3300/0 conn(0x7fc60c159ab0 0x7fc60c0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:40.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.593+0000 7fc610a21640 1 -- 172.21.15.84:0/4013956874 >> 172.21.15.84:0/4013956874 conn(0x7fc60c0ce9b0 msgr2=0x7fc60c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:40.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.593+0000 7fc610a21640 1 -- 172.21.15.84:0/4013956874 shutdown_connections 2023-10-16T10:07:40.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.593+0000 7fc610a21640 1 -- 172.21.15.84:0/4013956874 wait complete. 2023-10-16T10:07:40.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.594+0000 7fc610a21640 1 Processor -- start 2023-10-16T10:07:40.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.594+0000 7fc610a21640 1 -- start start 2023-10-16T10:07:40.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.595+0000 7fc610a21640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc60c06c140 0x7fc60c15cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:40.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.595+0000 7fc610a21640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc60c159ab0 0x7fc60c070bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:40.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.595+0000 7fc610a21640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc60c159e90 0x7fc60c1f2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:40.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.595+0000 7fc610a21640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c15a330 con 0x7fc60c06c140 2023-10-16T10:07:40.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.595+0000 7fc610a21640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c071130 con 0x7fc60c159ab0 2023-10-16T10:07:40.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.595+0000 7fc610a21640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c15a580 con 0x7fc60c159e90 2023-10-16T10:07:40.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.596+0000 7fc60a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc60c06c140 0x7fc60c15cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:40.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.596+0000 7fc60a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc60c06c140 0x7fc60c15cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51444/0 (socket says 172.21.15.84:51444) 2023-10-16T10:07:40.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.596+0000 7fc60a59c640 1 -- 172.21.15.84:0/1549404438 learned_addr learned my addr 172.21.15.84:0/1549404438 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:40.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.596+0000 7fc60a59c640 1 -- 172.21.15.84:0/1549404438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc60c159e90 msgr2=0x7fc60c1f2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:40.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.596+0000 7fc60a59c640 1 --2- 172.21.15.84:0/1549404438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc60c159e90 0x7fc60c1f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:40.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.596+0000 7fc60a59c640 1 -- 172.21.15.84:0/1549404438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc60c159ab0 msgr2=0x7fc60c070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:40.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.596+0000 7fc60a59c640 1 --2- 172.21.15.84:0/1549404438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc60c159ab0 0x7fc60c070bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:40.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.596+0000 7fc60a59c640 1 -- 172.21.15.84:0/1549404438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc604009b30 con 0x7fc60c06c140 2023-10-16T10:07:40.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.596+0000 7fc60a59c640 1 --2- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc60c06c140 0x7fc60c15cb80 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00d5f0 tx=0x7fc5fc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:40.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.596+0000 7fc6037fe640 1 -- 172.21.15.84:0/1549404438 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0125f0 con 0x7fc60c06c140 2023-10-16T10:07:40.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.597+0000 7fc6037fe640 1 -- 172.21.15.84:0/1549404438 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5fc012750 con 0x7fc60c06c140 2023-10-16T10:07:40.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.597+0000 7fc6037fe640 1 -- 172.21.15.84:0/1549404438 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc012b00 con 0x7fc60c06c140 2023-10-16T10:07:40.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.597+0000 7fc610a21640 1 -- 172.21.15.84:0/1549404438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60c15a860 con 0x7fc60c06c140 2023-10-16T10:07:40.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.597+0000 7fc610a21640 1 -- 172.21.15.84:0/1549404438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60c15ae20 con 0x7fc60c06c140 2023-10-16T10:07:40.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.598+0000 7fc610a21640 1 -- 172.21.15.84:0/1549404438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60c15afe0 con 0x7fc60c06c140 2023-10-16T10:07:40.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.599+0000 7fc6037fe640 1 -- 172.21.15.84:0/1549404438 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 60) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fc5fc012c60 con 0x7fc60c06c140 2023-10-16T10:07:40.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.600+0000 7fc6037fe640 1 --2- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fc5ec06afe0 0x7fc5ec06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:40.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.600+0000 7fc6037fe640 1 -- 172.21.15.84:0/1549404438 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5fc089bf0 con 0x7fc60c06c140 2023-10-16T10:07:40.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.600+0000 7fc601d9b640 1 -- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fc5ec06afe0 msgr2=0x7fc5ec06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4292025878 2023-10-16T10:07:40.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.600+0000 7fc601d9b640 1 --2- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fc5ec06afe0 0x7fc5ec06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:07:40.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.604+0000 7fc6037fe640 1 -- 172.21.15.84:0/1549404438 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc5fc054c20 con 0x7fc60c06c140 2023-10-16T10:07:40.612 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:40.611+0000 7f583797f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:07:40.614 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:40.613+0000 7ff7e5446040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:07:40.661 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:40.660+0000 7f583797f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:07:40.676 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:40.675+0000 7ff7e5446040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:07:40.726 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:40.724+0000 7ff7e5446040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:07:40.772 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:40.770+0000 7f583797f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:07:40.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.801+0000 7fc601d9b640 1 -- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fc5ec06afe0 msgr2=0x7fc5ec06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4292025878 2023-10-16T10:07:40.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.801+0000 7fc601d9b640 1 --2- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fc5ec06afe0 0x7fc5ec06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:07:40.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:40.814+0000 7fc610a21640 1 -- 172.21.15.84:0/1549404438 --> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fc60c0cd7b0 con 0x7fc5ec06afe0 2023-10-16T10:07:40.834 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:40.832+0000 7f583797f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:07:40.837 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:40.836+0000 7ff7e5446040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:07:40.883 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:40.882+0000 7f583797f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:07:40.899 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:40.898+0000 7ff7e5446040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:07:40.941 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:40.939+0000 7f583797f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:07:40.949 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:40.948+0000 7ff7e5446040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:07:40.991 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:40.989+0000 7f583797f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:07:41.005 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:41.003+0000 7ff7e5446040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:07:41.055 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:41.053+0000 7ff7e5446040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:07:41.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:41.202+0000 7fc601d9b640 1 -- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fc5ec06afe0 msgr2=0x7fc5ec06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4292025878 2023-10-16T10:07:41.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:41.202+0000 7fc601d9b640 1 --2- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fc5ec06afe0 0x7fc5ec06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T10:07:41.379 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:41.378+0000 7f583797f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:07:41.437 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:41.435+0000 7ff7e5446040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:07:41.539 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:41.538+0000 7f583797f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:07:41.595 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:41.593+0000 7ff7e5446040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:07:41.648 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:41.646+0000 7f583797f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:07:41.698 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:41.697+0000 7f583797f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:07:41.704 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:41.702+0000 7ff7e5446040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:07:41.753 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:41.752+0000 7ff7e5446040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:07:41.806 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:41.804+0000 7f583797f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:07:41.860 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:41.858+0000 7ff7e5446040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:07:41.913 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:41.911+0000 7f583797f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:07:41.966 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:41.964+0000 7ff7e5446040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:07:42.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:42.003+0000 7fc601d9b640 1 -- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fc5ec06afe0 msgr2=0x7fc5ec06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/4292025878 2023-10-16T10:07:42.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:42.003+0000 7fc601d9b640 1 --2- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fc5ec06afe0 0x7fc5ec06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T10:07:42.069 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:42.067+0000 7f583797f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:07:42.119 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:42.117+0000 7f583797f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:07:42.121 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:42.119+0000 7ff7e5446040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:07:42.169 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:42.168+0000 7ff7e5446040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:07:42.238 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:42.236+0000 7f583797f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:07:42.287 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:42.285+0000 7ff7e5446040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:07:43.071 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:43.069+0000 7f583797f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:07:43.112 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:43.111+0000 7ff7e5446040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:07:43.366 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:43.365+0000 7f583797f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:07:43.403 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:43.402+0000 7ff7e5446040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:07:43.422 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:43.421+0000 7f583797f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:07:43.460 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:43.458+0000 7ff7e5446040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:07:43.535 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:07:43.533+0000 7f583797f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:07:43.573 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:43.571+0000 7ff7e5446040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:07:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:43.592+0000 7fc6037fe640 1 -- 172.21.15.84:0/1549404438 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 61) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fc5fc054ff0 con 0x7fc60c06c140 2023-10-16T10:07:43.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:43.593+0000 7fc6037fe640 1 -- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fc5ec06afe0 msgr2=0x7fc5ec06d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:07:43.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:43.593+0000 7fc6037fe640 1 --2- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:6832/4292025878,v1:172.21.15.84:6833/4292025878] conn(0x7fc5ec06afe0 0x7fc5ec06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:43.902 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:43.900+0000 7ff76472e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:43.902 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:43.900+0000 7ff76472e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:43.902 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:43.900+0000 7ff76472e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:43.903 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:43.900+0000 7ff76472e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:43.903 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:43.900+0000 7ff76472e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:43.903 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:43.900+0000 7ff7606e6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:43.903 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:43.901+0000 7ff7606e6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:43.903 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:43.901+0000 7ff7606e6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:43.904 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:43.901+0000 7ff7606e6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:43.904 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:07:43.901+0000 7ff7606e6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:07:44.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:44.602+0000 7fc6037fe640 1 -- 172.21.15.84:0/1549404438 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 62) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fc5fc057dd0 con 0x7fc60c06c140 2023-10-16T10:07:44.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:44.603+0000 7fc6037fe640 1 --2- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7fc5ec035880 0x7fc5ec074ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:44.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:44.603+0000 7fc6037fe640 1 -- 172.21.15.84:0/1549404438 --> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5ec035c60 con 0x7fc5ec035880 2023-10-16T10:07:44.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:44.606+0000 7fc601d9b640 1 --2- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7fc5ec035880 0x7fc5ec074ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:44.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:44.617+0000 7fc601d9b640 1 --2- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7fc5ec035880 0x7fc5ec074ea0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc60c1f5f20 tx=0x7fc5f40079f0 comp rx=0 tx=0).ready entity=mgr.8223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:44.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:44.811+0000 7fc6037fe640 1 -- 172.21.15.84:0/1549404438 <== mgr.8223 v2:172.21.15.84:6832/2209954486 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc5ec035c60 con 0x7fc5ec035880 2023-10-16T10:07:44.813 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path snap_test_dir1 2023-10-16T10:07:44.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:44.814+0000 7fc610a21640 1 -- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7fc5ec035880 msgr2=0x7fc5ec074ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:44.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:44.814+0000 7fc610a21640 1 --2- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7fc5ec035880 0x7fc5ec074ea0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc60c1f5f20 tx=0x7fc5f40079f0 comp rx=0 tx=0).stop 2023-10-16T10:07:44.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:44.814+0000 7fc610a21640 1 -- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc60c06c140 msgr2=0x7fc60c15cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:44.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:44.814+0000 7fc610a21640 1 --2- 172.21.15.84:0/1549404438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc60c06c140 0x7fc60c15cb80 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00d5f0 tx=0x7fc5fc00dab0 comp rx=0 tx=0).stop 2023-10-16T10:07:44.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:44.814+0000 7fc60ad9d640 1 -- 172.21.15.84:0/1549404438 reap_dead start 2023-10-16T10:07:44.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:44.815+0000 7fc610a21640 1 -- 172.21.15.84:0/1549404438 shutdown_connections 2023-10-16T10:07:44.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:44.815+0000 7fc610a21640 1 -- 172.21.15.84:0/1549404438 >> 172.21.15.84:0/1549404438 conn(0x7fc60c0ce9b0 msgr2=0x7fc60c1580c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:44.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:44.815+0000 7fc610a21640 1 -- 172.21.15.84:0/1549404438 shutdown_connections 2023-10-16T10:07:44.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:44.815+0000 7fc610a21640 1 -- 172.21.15.84:0/1549404438 wait complete. 2023-10-16T10:07:44.836 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-16T10:07:44.837 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-16-10_08_00 in ~16s... 2023-10-16T10:07:44.837 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [] 2023-10-16T10:07:44.838 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-16T10:07:45.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.045+0000 7fc8c5d4e640 1 Processor -- start 2023-10-16T10:07:45.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.045+0000 7fc8c5d4e640 1 -- start start 2023-10-16T10:07:45.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.046+0000 7fc8c5d4e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc8c015ad70 0x7fc8c0069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:45.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.046+0000 7fc8c5d4e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8c00c9ca0 0x7fc8c006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:45.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.046+0000 7fc8c5d4e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8c00cb030 con 0x7fc8c00ca3b0 2023-10-16T10:07:45.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.046+0000 7fc8c5d4e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c00c91c0 con 0x7fc8c015ad70 2023-10-16T10:07:45.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.046+0000 7fc8c5d4e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c00c94a0 con 0x7fc8c00c9ca0 2023-10-16T10:07:45.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.047+0000 7fc8bf7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc8c015ad70 0x7fc8c0069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:45.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.047+0000 7fc8bf7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc8c015ad70 0x7fc8c0069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51604/0 (socket says 172.21.15.84:51604) 2023-10-16T10:07:45.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.047+0000 7fc8bf7fe640 1 -- 172.21.15.84:0/3551446791 learned_addr learned my addr 172.21.15.84:0/3551446791 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:45.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.047+0000 7fc8bf7fe640 1 -- 172.21.15.84:0/3551446791 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8c00c9ca0 msgr2=0x7fc8c006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:45.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.047+0000 7fc8bf7fe640 1 --2- 172.21.15.84:0/3551446791 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8c00c9ca0 0x7fc8c006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:45.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.047+0000 7fc8bf7fe640 1 -- 172.21.15.84:0/3551446791 >> v1:172.21.15.157:6789/0 conn(0x7fc8c00ca3b0 legacy=0x7fc8c00ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:07:45.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.047+0000 7fc8bf7fe640 1 -- 172.21.15.84:0/3551446791 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c00c9780 con 0x7fc8c015ad70 2023-10-16T10:07:45.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.048+0000 7fc8bf7fe640 1 --2- 172.21.15.84:0/3551446791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8c015ad70 0x7fc8c0069da0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009870 tx=0x7fc8b002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4916346efd4146a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:45.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.048+0000 7fc8beffd640 1 -- 172.21.15.84:0/3551446791 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0031640 con 0x7fc8c015ad70 2023-10-16T10:07:45.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.049+0000 7fc8beffd640 1 -- 172.21.15.84:0/3551446791 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b00317a0 con 0x7fc8c015ad70 2023-10-16T10:07:45.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.049+0000 7fc8beffd640 1 -- 172.21.15.84:0/3551446791 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0031b30 con 0x7fc8c015ad70 2023-10-16T10:07:45.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.049+0000 7fc8c5d4e640 1 -- 172.21.15.84:0/3551446791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8c015ad70 msgr2=0x7fc8c0069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:45.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.049+0000 7fc8c5d4e640 1 --2- 172.21.15.84:0/3551446791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8c015ad70 0x7fc8c0069da0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009870 tx=0x7fc8b002b8f0 comp rx=0 tx=0).stop 2023-10-16T10:07:45.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.049+0000 7fc8c5d4e640 1 -- 172.21.15.84:0/3551446791 shutdown_connections 2023-10-16T10:07:45.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.049+0000 7fc8c5d4e640 1 --2- 172.21.15.84:0/3551446791 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8c00c9ca0 0x7fc8c006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:45.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.049+0000 7fc8c5d4e640 1 --2- 172.21.15.84:0/3551446791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8c015ad70 0x7fc8c0069da0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:45.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.049+0000 7fc8c5d4e640 1 -- 172.21.15.84:0/3551446791 >> 172.21.15.84:0/3551446791 conn(0x7fc8c00ce9b0 msgr2=0x7fc8c00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:45.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.049+0000 7fc8c5d4e640 1 -- 172.21.15.84:0/3551446791 shutdown_connections 2023-10-16T10:07:45.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.049+0000 7fc8c5d4e640 1 -- 172.21.15.84:0/3551446791 wait complete. 2023-10-16T10:07:45.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.051+0000 7fc8c5d4e640 1 Processor -- start 2023-10-16T10:07:45.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.051+0000 7fc8c5d4e640 1 -- start start 2023-10-16T10:07:45.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.052+0000 7fc8c5d4e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8c00c9ca0 0x7fc8c0159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:45.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.052+0000 7fc8c5d4e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8c00ca3b0 0x7fc8c00b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:45.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.052+0000 7fc8c5d4e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8c015ad70 0x7fc8c00ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:45.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.052+0000 7fc8c5d4e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c00cd5e0 con 0x7fc8c015ad70 2023-10-16T10:07:45.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.052+0000 7fc8c5d4e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c00ce0a0 con 0x7fc8c00c9ca0 2023-10-16T10:07:45.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.052+0000 7fc8c5d4e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c00b3640 con 0x7fc8c00ca3b0 2023-10-16T10:07:45.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.053+0000 7fc8bffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8c015ad70 0x7fc8c00ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:45.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.053+0000 7fc8bffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8c015ad70 0x7fc8c00ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51616/0 (socket says 172.21.15.84:51616) 2023-10-16T10:07:45.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.053+0000 7fc8bffff640 1 -- 172.21.15.84:0/2659529440 learned_addr learned my addr 172.21.15.84:0/2659529440 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:07:45.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.053+0000 7fc8bf7fe640 1 --2- 172.21.15.84:0/2659529440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8c00c9ca0 0x7fc8c0159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:45.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.053+0000 7fc8bffff640 1 -- 172.21.15.84:0/2659529440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8c00ca3b0 msgr2=0x7fc8c00b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:45.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.053+0000 7fc8bffff640 1 --2- 172.21.15.84:0/2659529440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8c00ca3b0 0x7fc8c00b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:45.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.053+0000 7fc8bffff640 1 -- 172.21.15.84:0/2659529440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8c00c9ca0 msgr2=0x7fc8c0159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:45.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.053+0000 7fc8bffff640 1 --2- 172.21.15.84:0/2659529440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8c00c9ca0 0x7fc8c0159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:45.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.053+0000 7fc8bffff640 1 -- 172.21.15.84:0/2659529440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b00094d0 con 0x7fc8c015ad70 2023-10-16T10:07:45.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.053+0000 7fc8bffff640 1 --2- 172.21.15.84:0/2659529440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8c015ad70 0x7fc8c00ac890 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800c780 tx=0x7fc8b800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:45.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.053+0000 7fc8bcff9640 1 -- 172.21.15.84:0/2659529440 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8003d80 con 0x7fc8c015ad70 2023-10-16T10:07:45.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.054+0000 7fc8bcff9640 1 -- 172.21.15.84:0/2659529440 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b800ce90 con 0x7fc8c015ad70 2023-10-16T10:07:45.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.054+0000 7fc8c5d4e640 1 -- 172.21.15.84:0/2659529440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c00cd300 con 0x7fc8c015ad70 2023-10-16T10:07:45.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.054+0000 7fc8bcff9640 1 -- 172.21.15.84:0/2659529440 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b80052b0 con 0x7fc8c015ad70 2023-10-16T10:07:45.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.054+0000 7fc8c5d4e640 1 -- 172.21.15.84:0/2659529440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c00cddc0 con 0x7fc8c015ad70 2023-10-16T10:07:45.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.055+0000 7fc8c5d4e640 1 -- 172.21.15.84:0/2659529440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c00ad490 con 0x7fc8c015ad70 2023-10-16T10:07:45.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.056+0000 7fc8bcff9640 1 -- 172.21.15.84:0/2659529440 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 62) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fc8b8005410 con 0x7fc8c015ad70 2023-10-16T10:07:45.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.056+0000 7fc8bcff9640 1 --2- 172.21.15.84:0/2659529440 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7fc8a006ad10 0x7fc8a006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:07:45.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.057+0000 7fc8bcff9640 1 -- 172.21.15.84:0/2659529440 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fc8b8088d00 con 0x7fc8c015ad70 2023-10-16T10:07:45.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.057+0000 7fc8bf7fe640 1 --2- 172.21.15.84:0/2659529440 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7fc8a006ad10 0x7fc8a006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:07:45.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.057+0000 7fc8bf7fe640 1 --2- 172.21.15.84:0/2659529440 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7fc8a006ad10 0x7fc8a006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0036000 tx=0x7fc8b003c040 comp rx=0 tx=0).ready entity=mgr.8223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:07:45.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.061+0000 7fc8bcff9640 1 -- 172.21.15.84:0/2659529440 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc8b8053ec0 con 0x7fc8c015ad70 2023-10-16T10:07:45.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.275+0000 7fc8c5d4e640 1 -- 172.21.15.84:0/2659529440 --> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc8c00cd960 con 0x7fc8a006ad10 2023-10-16T10:07:45.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.286+0000 7fc8bcff9640 1 -- 172.21.15.84:0/2659529440 <== mgr.8223 v2:172.21.15.84:6832/2209954486 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7fc8b0031a90 con 0x7fc8a006ad10 2023-10-16T10:07:45.288 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:07:45.289 INFO:teuthology.orchestra.run.smithi084.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{}]} 2023-10-16T10:07:45.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.289+0000 7fc8c5d4e640 1 -- 172.21.15.84:0/2659529440 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7fc8a006ad10 msgr2=0x7fc8a006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:45.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.289+0000 7fc8c5d4e640 1 --2- 172.21.15.84:0/2659529440 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7fc8a006ad10 0x7fc8a006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0036000 tx=0x7fc8b003c040 comp rx=0 tx=0).stop 2023-10-16T10:07:45.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.289+0000 7fc8c5d4e640 1 -- 172.21.15.84:0/2659529440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8c015ad70 msgr2=0x7fc8c00ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:07:45.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.289+0000 7fc8c5d4e640 1 --2- 172.21.15.84:0/2659529440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8c015ad70 0x7fc8c00ac890 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800c780 tx=0x7fc8b800cc40 comp rx=0 tx=0).stop 2023-10-16T10:07:45.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.289+0000 7fc8c5d4e640 1 -- 172.21.15.84:0/2659529440 shutdown_connections 2023-10-16T10:07:45.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.289+0000 7fc8c5d4e640 1 --2- 172.21.15.84:0/2659529440 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7fc8a006ad10 0x7fc8a006d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:45.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.289+0000 7fc8c5d4e640 1 --2- 172.21.15.84:0/2659529440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8c015ad70 0x7fc8c00ac890 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:45.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.289+0000 7fc8c5d4e640 1 --2- 172.21.15.84:0/2659529440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8c00ca3b0 0x7fc8c00b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:45.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.289+0000 7fc8c5d4e640 1 --2- 172.21.15.84:0/2659529440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8c00c9ca0 0x7fc8c0159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:07:45.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.289+0000 7fc8c5d4e640 1 -- 172.21.15.84:0/2659529440 >> 172.21.15.84:0/2659529440 conn(0x7fc8c00ce9b0 msgr2=0x7fc8c00c7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:07:45.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.289+0000 7fc8c5d4e640 1 -- 172.21.15.84:0/2659529440 shutdown_connections 2023-10-16T10:07:45.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:07:45.289+0000 7fc8c5d4e640 1 -- 172.21.15.84:0/2659529440 wait complete. 2023-10-16T10:07:45.310 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{}]} 2023-10-16T10:07:45.311 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{}]} 2023-10-16T10:07:45.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:45.311 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:07:45.366 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:07:45.367 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:07:45.367 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:07:46.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:46.369 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:07:46.432 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:07:46.433 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:07:46.433 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:07:47.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:47.435 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:07:47.499 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:07:47.500 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:07:47.500 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:07:48.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:48.502 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:07:48.565 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:07:48.565 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:07:48.565 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:07:49.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:49.567 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:07:49.632 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:07:49.632 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:07:49.632 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:07:50.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:50.634 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:07:50.697 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:07:50.697 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:07:50.698 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:07:51.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:51.700 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:07:51.763 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:07:51.763 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:07:51.764 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:07:52.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:52.765 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:07:52.828 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:07:52.829 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:07:52.829 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:07:53.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:53.830 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:07:53.893 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:07:53.893 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:07:53.893 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:07:54.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:54.895 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:07:54.959 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:07:54.959 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:07:54.960 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:07:55.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:55.962 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:07:56.026 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:07:56.026 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:07:56.026 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:07:57.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:57.028 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:07:57.091 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:07:57.091 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:07:57.092 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:07:58.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:58.093 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:07:58.158 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:07:58.158 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:07:58.159 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:07:59.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:07:59.160 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:07:59.223 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:07:59.224 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:07:59.224 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:08:00.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:00.226 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:08:00.291 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:08:00.292 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:08:00.292 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:08:00.834 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:00.832+0000 7ff7546ce640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:00.834 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:00.832+0000 7ff7546ce640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:00.834 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:00.832+0000 7ff7546ce640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:00.835 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:00.832+0000 7ff7546ce640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:00.835 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:00.832+0000 7ff7546ce640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:01.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:01.294 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:08:01.356 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_08_00_UTC 2023-10-16T10:08:01.358 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_08_00_UTC'} 2023-10-16T10:08:01.359 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-16-10_08_00_UTC'} 2023-10-16T10:08:01.359 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:08:01.359 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-16-10_08_00_UTC'] 2023-10-16T10:08:01.360 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697450864.8372488, now = 1697450881.3600554, timo = 16 2023-10-16T10:08:02.361 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-16T10:08:02.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.587+0000 7f5811987640 1 Processor -- start 2023-10-16T10:08:02.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.587+0000 7f5811987640 1 -- start start 2023-10-16T10:08:02.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.588+0000 7f5811987640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f580c159950 0x7f580c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:02.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.588+0000 7f5811987640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580c06c810 0x7f580c0708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:02.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.588+0000 7f5811987640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f580c15aef0 con 0x7f580c070450 2023-10-16T10:08:02.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.588+0000 7f5811987640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580c070e20 con 0x7f580c159950 2023-10-16T10:08:02.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.588+0000 7f5811987640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580c15e490 con 0x7f580c06c810 2023-10-16T10:08:02.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.589+0000 7f5810985640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f580c159950 0x7f580c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:02.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.589+0000 7f580bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f580c070450 0x7f580c06c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47104/0 (socket says 172.21.15.84:47104) 2023-10-16T10:08:02.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.589+0000 7f580bfff640 1 -- 172.21.15.84:0/3418180745 learned_addr learned my addr 172.21.15.84:0/3418180745 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:02.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.589+0000 7f5810985640 1 -- 172.21.15.84:0/3418180745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580c06c810 msgr2=0x7f580c0708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:02.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.589+0000 7f5811186640 1 --2- 172.21.15.84:0/3418180745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580c06c810 0x7f580c0708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:02.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.589+0000 7f5810985640 1 --2- 172.21.15.84:0/3418180745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580c06c810 0x7f580c0708e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:02.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.589+0000 7f5810985640 1 -- 172.21.15.84:0/3418180745 >> v1:172.21.15.84:6789/0 conn(0x7f580c070450 legacy=0x7f580c06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:02.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.589+0000 7f5810985640 1 -- 172.21.15.84:0/3418180745 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c06cc60 con 0x7f580c159950 2023-10-16T10:08:02.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.590+0000 7f5810985640 1 --2- 172.21.15.84:0/3418180745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f580c159950 0x7f580c15a9b0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f57fc009d90 tx=0x7f57fc02bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f56ba6fd07e652 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:02.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.591+0000 7f580b7fe640 1 -- 172.21.15.84:0/3418180745 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57fc02c980 con 0x7f580c159950 2023-10-16T10:08:02.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.591+0000 7f580b7fe640 1 -- 172.21.15.84:0/3418180745 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57fc02cae0 con 0x7f580c159950 2023-10-16T10:08:02.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.591+0000 7f580b7fe640 1 -- 172.21.15.84:0/3418180745 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57fc02ce70 con 0x7f580c159950 2023-10-16T10:08:02.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.591+0000 7f5811987640 1 -- 172.21.15.84:0/3418180745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f580c159950 msgr2=0x7f580c15a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:02.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.591+0000 7f5811987640 1 --2- 172.21.15.84:0/3418180745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f580c159950 0x7f580c15a9b0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f57fc009d90 tx=0x7f57fc02bc50 comp rx=0 tx=0).stop 2023-10-16T10:08:02.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.592+0000 7f5811987640 1 -- 172.21.15.84:0/3418180745 shutdown_connections 2023-10-16T10:08:02.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.592+0000 7f5811987640 1 --2- 172.21.15.84:0/3418180745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580c06c810 0x7f580c0708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:02.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.592+0000 7f5811987640 1 --2- 172.21.15.84:0/3418180745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f580c159950 0x7f580c15a9b0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:02.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.592+0000 7f5811987640 1 -- 172.21.15.84:0/3418180745 >> 172.21.15.84:0/3418180745 conn(0x7f580c0ce6a0 msgr2=0x7f580c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:02.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.592+0000 7f5811987640 1 -- 172.21.15.84:0/3418180745 shutdown_connections 2023-10-16T10:08:02.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.592+0000 7f5811987640 1 -- 172.21.15.84:0/3418180745 wait complete. 2023-10-16T10:08:02.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.593+0000 7f5811987640 1 Processor -- start 2023-10-16T10:08:02.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.593+0000 7f5811987640 1 -- start start 2023-10-16T10:08:02.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.594+0000 7f5811987640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f580c06c810 0x7f580c15b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:02.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.594+0000 7f5811987640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f580c070450 0x7f580c0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:02.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.594+0000 7f5811987640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580c159950 0x7f580c0acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:02.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.594+0000 7f5811987640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0c2500 con 0x7f580c06c810 2023-10-16T10:08:02.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.594+0000 7f5811987640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0c65d0 con 0x7f580c070450 2023-10-16T10:08:02.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.594+0000 7f5811987640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0c21b0 con 0x7f580c159950 2023-10-16T10:08:02.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.595+0000 7f5810985640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f580c06c810 0x7f580c15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:02.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.595+0000 7f5810985640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f580c06c810 0x7f580c15b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58724/0 (socket says 172.21.15.84:58724) 2023-10-16T10:08:02.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.595+0000 7f5810985640 1 -- 172.21.15.84:0/1316303922 learned_addr learned my addr 172.21.15.84:0/1316303922 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:02.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.595+0000 7f580bfff640 1 --2- 172.21.15.84:0/1316303922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f580c070450 0x7f580c0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:02.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.595+0000 7f5811186640 1 --2- 172.21.15.84:0/1316303922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580c159950 0x7f580c0acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:02.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.595+0000 7f5810985640 1 -- 172.21.15.84:0/1316303922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580c159950 msgr2=0x7f580c0acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:02.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.595+0000 7f5810985640 1 --2- 172.21.15.84:0/1316303922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580c159950 0x7f580c0acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:02.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.595+0000 7f5810985640 1 -- 172.21.15.84:0/1316303922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f580c070450 msgr2=0x7f580c0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:02.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.595+0000 7f5810985640 1 --2- 172.21.15.84:0/1316303922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f580c070450 0x7f580c0ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:02.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.595+0000 7f5810985640 1 -- 172.21.15.84:0/1316303922 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57fc009a70 con 0x7f580c06c810 2023-10-16T10:08:02.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.596+0000 7f5810985640 1 --2- 172.21.15.84:0/1316303922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f580c06c810 0x7f580c15b090 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f57fc037040 tx=0x7f57fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:02.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.596+0000 7f58097fa640 1 -- 172.21.15.84:0/1316303922 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57fc034d50 con 0x7f580c06c810 2023-10-16T10:08:02.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.596+0000 7f58097fa640 1 -- 172.21.15.84:0/1316303922 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57fc040450 con 0x7f580c06c810 2023-10-16T10:08:02.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.596+0000 7f58097fa640 1 -- 172.21.15.84:0/1316303922 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57fc040800 con 0x7f580c06c810 2023-10-16T10:08:02.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.596+0000 7f5811987640 1 -- 172.21.15.84:0/1316303922 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580c15ba00 con 0x7f580c06c810 2023-10-16T10:08:02.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.596+0000 7f5811987640 1 -- 172.21.15.84:0/1316303922 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580c15bfc0 con 0x7f580c06c810 2023-10-16T10:08:02.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.597+0000 7f5811987640 1 -- 172.21.15.84:0/1316303922 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f580c04e9b0 con 0x7f580c06c810 2023-10-16T10:08:02.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.598+0000 7f58097fa640 1 -- 172.21.15.84:0/1316303922 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 65) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f57fc040960 con 0x7f580c06c810 2023-10-16T10:08:02.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.599+0000 7f58097fa640 1 --2- 172.21.15.84:0/1316303922 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f57f806b100 0x7f57f806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:02.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.599+0000 7f58097fa640 1 -- 172.21.15.84:0/1316303922 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f57fc0ac810 con 0x7f580c06c810 2023-10-16T10:08:02.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.599+0000 7f580bfff640 1 --2- 172.21.15.84:0/1316303922 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f57f806b100 0x7f57f806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:02.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.600+0000 7f580bfff640 1 --2- 172.21.15.84:0/1316303922 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f57f806b100 0x7f57f806d5c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f580c0af3a0 tx=0x7f5800009040 comp rx=0 tx=0).ready entity=mgr.8223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:02.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.604+0000 7f58097fa640 1 -- 172.21.15.84:0/1316303922 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f57fc0773f0 con 0x7f580c06c810 2023-10-16T10:08:02.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.816+0000 7f5811987640 1 -- 172.21.15.84:0/1316303922 --> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f580c15dbf0 con 0x7f57f806b100 2023-10-16T10:08:02.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.995+0000 7f58097fa640 1 -- 172.21.15.84:0/1316303922 <== mgr.8223 v2:172.21.15.84:6832/2209954486 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f58000070d0 con 0x7f57f806b100 2023-10-16T10:08:02.997 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path snap_test_dir1 2023-10-16T10:08:03.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.998+0000 7f5811987640 1 -- 172.21.15.84:0/1316303922 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f57f806b100 msgr2=0x7f57f806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:03.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.998+0000 7f5811987640 1 --2- 172.21.15.84:0/1316303922 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f57f806b100 0x7f57f806d5c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f580c0af3a0 tx=0x7f5800009040 comp rx=0 tx=0).stop 2023-10-16T10:08:03.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.998+0000 7f5811987640 1 -- 172.21.15.84:0/1316303922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f580c06c810 msgr2=0x7f580c15b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:03.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.998+0000 7f5811987640 1 --2- 172.21.15.84:0/1316303922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f580c06c810 0x7f580c15b090 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f57fc037040 tx=0x7f57fc004a40 comp rx=0 tx=0).stop 2023-10-16T10:08:03.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.998+0000 7f5811987640 1 -- 172.21.15.84:0/1316303922 shutdown_connections 2023-10-16T10:08:03.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.999+0000 7f5811987640 1 --2- 172.21.15.84:0/1316303922 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f57f806b100 0x7f57f806d5c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:03.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.999+0000 7f5811987640 1 --2- 172.21.15.84:0/1316303922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580c159950 0x7f580c0acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:03.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.999+0000 7f5811987640 1 --2- 172.21.15.84:0/1316303922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f580c070450 0x7f580c0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:03.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.999+0000 7f5811987640 1 --2- 172.21.15.84:0/1316303922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f580c06c810 0x7f580c15b090 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:03.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.999+0000 7f5811987640 1 -- 172.21.15.84:0/1316303922 >> 172.21.15.84:0/1316303922 conn(0x7f580c0ce6a0 msgr2=0x7f580c06b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:03.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.999+0000 7f5811987640 1 -- 172.21.15.84:0/1316303922 shutdown_connections 2023-10-16T10:08:03.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:02.999+0000 7f5811987640 1 -- 172.21.15.84:0/1316303922 wait complete. 2023-10-16T10:08:03.021 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-16T10:08:03.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:03.022 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:08:03.074 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_08_00_UTC 2023-10-16T10:08:03.075 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_08_00_UTC 2023-10-16T10:08:03.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:03.076 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_08_00_UTC) 2023-10-16T10:08:03.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:03.138 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-16T10:08:03.196 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T10:08:03.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.415+0000 7f695f844640 1 Processor -- start 2023-10-16T10:08:03.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.415+0000 7f695f844640 1 -- start start 2023-10-16T10:08:03.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.417+0000 7f695f844640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f695815d150 0x7f69580ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:03.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.417+0000 7f695f844640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f695806a4b0 0x7f6958160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:03.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.417+0000 7f695f844640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69580ac890 con 0x7f6958160b90 2023-10-16T10:08:03.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.417+0000 7f695f844640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f695815d530 con 0x7f695815d150 2023-10-16T10:08:03.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.417+0000 7f695f844640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6958161650 con 0x7f695806a4b0 2023-10-16T10:08:03.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.417+0000 7f695d5e0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f695815d150 0x7f69580ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:03.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.417+0000 7f695d5e0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f695815d150 0x7f69580ad350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58734/0 (socket says 172.21.15.84:58734) 2023-10-16T10:08:03.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.417+0000 7f695d5e0640 1 -- 172.21.15.84:0/1081500601 learned_addr learned my addr 172.21.15.84:0/1081500601 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:03.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.417+0000 7f695d5e0640 1 -- 172.21.15.84:0/1081500601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f695806a4b0 msgr2=0x7f6958160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:03.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.417+0000 7f695dde1640 1 --2- 172.21.15.84:0/1081500601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f695806a4b0 0x7f6958160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:03.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.418+0000 7f695d5e0640 1 --2- 172.21.15.84:0/1081500601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f695806a4b0 0x7f6958160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:03.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.418+0000 7f695d5e0640 1 -- 172.21.15.84:0/1081500601 >> v1:172.21.15.157:6789/0 conn(0x7f6958160b90 legacy=0x7f6958069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:03.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.418+0000 7f695d5e0640 1 -- 172.21.15.84:0/1081500601 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6958161930 con 0x7f695815d150 2023-10-16T10:08:03.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.419+0000 7f695d5e0640 1 --2- 172.21.15.84:0/1081500601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f695815d150 0x7f69580ad350 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f6954009870 tx=0x7f695402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853029c2a54cede2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:03.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.419+0000 7f694bfff640 1 -- 172.21.15.84:0/1081500601 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69540317c0 con 0x7f695815d150 2023-10-16T10:08:03.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.419+0000 7f694bfff640 1 -- 172.21.15.84:0/1081500601 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6954031920 con 0x7f695815d150 2023-10-16T10:08:03.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.419+0000 7f694bfff640 1 -- 172.21.15.84:0/1081500601 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6954031cb0 con 0x7f695815d150 2023-10-16T10:08:03.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.420+0000 7f695f844640 1 -- 172.21.15.84:0/1081500601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f695815d150 msgr2=0x7f69580ad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:03.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.420+0000 7f695f844640 1 --2- 172.21.15.84:0/1081500601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f695815d150 0x7f69580ad350 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f6954009870 tx=0x7f695402b860 comp rx=0 tx=0).stop 2023-10-16T10:08:03.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.420+0000 7f695f844640 1 -- 172.21.15.84:0/1081500601 shutdown_connections 2023-10-16T10:08:03.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.420+0000 7f695f844640 1 --2- 172.21.15.84:0/1081500601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f695806a4b0 0x7f6958160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:03.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.420+0000 7f695f844640 1 --2- 172.21.15.84:0/1081500601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f695815d150 0x7f69580ad350 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:03.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.420+0000 7f695f844640 1 -- 172.21.15.84:0/1081500601 >> 172.21.15.84:0/1081500601 conn(0x7f69580ce9b0 msgr2=0x7f69580cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:03.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.420+0000 7f695f844640 1 -- 172.21.15.84:0/1081500601 shutdown_connections 2023-10-16T10:08:03.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.420+0000 7f695f844640 1 -- 172.21.15.84:0/1081500601 wait complete. 2023-10-16T10:08:03.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.422+0000 7f695f844640 1 Processor -- start 2023-10-16T10:08:03.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.422+0000 7f695f844640 1 -- start start 2023-10-16T10:08:03.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.423+0000 7f695f844640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f695806a4b0 0x7f69580c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:03.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.423+0000 7f695f844640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f695815d150 0x7f69581f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:03.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.423+0000 7f695f844640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6958160b90 0x7f69580ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:03.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.423+0000 7f695f844640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69580c8480 con 0x7f695806a4b0 2023-10-16T10:08:03.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.423+0000 7f695f844640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69580b0bc0 con 0x7f6958160b90 2023-10-16T10:08:03.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.423+0000 7f695f844640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69581f2890 con 0x7f695815d150 2023-10-16T10:08:03.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.423+0000 7f695d5e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f695806a4b0 0x7f69580c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:03.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.423+0000 7f695d5e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f695806a4b0 0x7f69580c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58748/0 (socket says 172.21.15.84:58748) 2023-10-16T10:08:03.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.423+0000 7f695d5e0640 1 -- 172.21.15.84:0/3305181650 learned_addr learned my addr 172.21.15.84:0/3305181650 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:03.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.423+0000 7f695d5e0640 1 -- 172.21.15.84:0/3305181650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f695815d150 msgr2=0x7f69581f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:03.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.423+0000 7f695d5e0640 1 --2- 172.21.15.84:0/3305181650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f695815d150 0x7f69581f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:03.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.423+0000 7f695d5e0640 1 -- 172.21.15.84:0/3305181650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6958160b90 msgr2=0x7f69580ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:03.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.423+0000 7f695d5e0640 1 --2- 172.21.15.84:0/3305181650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6958160b90 0x7f69580ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:03.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.423+0000 7f695d5e0640 1 -- 172.21.15.84:0/3305181650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69540094d0 con 0x7f695806a4b0 2023-10-16T10:08:03.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.424+0000 7f695d5e0640 1 --2- 172.21.15.84:0/3305181650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f695806a4b0 0x7f69580c7f40 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f6954000c00 tx=0x7f69540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:03.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.424+0000 7f6949ffb640 1 -- 172.21.15.84:0/3305181650 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69540329a0 con 0x7f695806a4b0 2023-10-16T10:08:03.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.424+0000 7f6949ffb640 1 -- 172.21.15.84:0/3305181650 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6954032b20 con 0x7f695806a4b0 2023-10-16T10:08:03.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.424+0000 7f6949ffb640 1 -- 172.21.15.84:0/3305181650 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69540453f0 con 0x7f695806a4b0 2023-10-16T10:08:03.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.424+0000 7f695f844640 1 -- 172.21.15.84:0/3305181650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69580b0600 con 0x7f695806a4b0 2023-10-16T10:08:03.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.424+0000 7f695f844640 1 -- 172.21.15.84:0/3305181650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69580c5c00 con 0x7f695806a4b0 2023-10-16T10:08:03.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.425+0000 7f695f844640 1 -- 172.21.15.84:0/3305181650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69580cf2f0 con 0x7f695806a4b0 2023-10-16T10:08:03.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.427+0000 7f6949ffb640 1 -- 172.21.15.84:0/3305181650 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 65) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f695404d3f0 con 0x7f695806a4b0 2023-10-16T10:08:03.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.427+0000 7f6949ffb640 1 --2- 172.21.15.84:0/3305181650 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f693c06b100 0x7f693c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:03.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.427+0000 7f6949ffb640 1 -- 172.21.15.84:0/3305181650 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f69540abb00 con 0x7f695806a4b0 2023-10-16T10:08:03.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.428+0000 7f695cddf640 1 --2- 172.21.15.84:0/3305181650 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f693c06b100 0x7f693c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:03.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.428+0000 7f695cddf640 1 --2- 172.21.15.84:0/3305181650 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f693c06b100 0x7f693c06d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f694c006fd0 tx=0x7f694c006bc0 comp rx=0 tx=0).ready entity=mgr.8223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:03.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.433+0000 7f6949ffb640 1 -- 172.21.15.84:0/3305181650 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6954076780 con 0x7f695806a4b0 2023-10-16T10:08:03.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:03.676+0000 7f695f844640 1 -- 172.21.15.84:0/3305181650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f695804e160 con 0x7f695806a4b0 2023-10-16T10:08:04.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.031+0000 7f6949ffb640 1 -- 172.21.15.84:0/3305181650 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v66) v1 ==== 93+0+0 (secure 0 0 0) 0x7f6954079da0 con 0x7f695806a4b0 2023-10-16T10:08:04.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.034+0000 7f695f844640 1 -- 172.21.15.84:0/3305181650 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f693c06b100 msgr2=0x7f693c06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:04.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.034+0000 7f695f844640 1 --2- 172.21.15.84:0/3305181650 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f693c06b100 0x7f693c06d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f694c006fd0 tx=0x7f694c006bc0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.034+0000 7f695f844640 1 -- 172.21.15.84:0/3305181650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f695806a4b0 msgr2=0x7f69580c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:04.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.034+0000 7f695f844640 1 --2- 172.21.15.84:0/3305181650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f695806a4b0 0x7f69580c7f40 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f6954000c00 tx=0x7f69540046c0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.034+0000 7f695f844640 1 -- 172.21.15.84:0/3305181650 shutdown_connections 2023-10-16T10:08:04.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.034+0000 7f695f844640 1 --2- 172.21.15.84:0/3305181650 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f693c06b100 0x7f693c06d5c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.034+0000 7f695f844640 1 --2- 172.21.15.84:0/3305181650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6958160b90 0x7f69580ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.034+0000 7f695f844640 1 --2- 172.21.15.84:0/3305181650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f695815d150 0x7f69581f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.034+0000 7f695f844640 1 --2- 172.21.15.84:0/3305181650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f695806a4b0 0x7f69580c7f40 secure :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f6954000c00 tx=0x7f69540046c0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.034+0000 7f695f844640 1 -- 172.21.15.84:0/3305181650 >> 172.21.15.84:0/3305181650 conn(0x7f69580ce9b0 msgr2=0x7f695806f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:04.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.036+0000 7f695f844640 1 -- 172.21.15.84:0/3305181650 shutdown_connections 2023-10-16T10:08:04.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.036+0000 7f695f844640 1 -- 172.21.15.84:0/3305181650 wait complete. 2023-10-16T10:08:04.061 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:08:04.062 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:08:04.163 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:08:04.256 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:08:04.257 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:08:04.257 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:08:04.305 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:08:04.306 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:08:04.399 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:08:04.604 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:04.602+0000 7fc263432040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:08:04.623 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:04.622+0000 7facc7012040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:08:04.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.649+0000 7f7e7a66f640 1 Processor -- start 2023-10-16T10:08:04.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.649+0000 7f7e7a66f640 1 -- start start 2023-10-16T10:08:04.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.650+0000 7f7e7a66f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e74160420 0x7f7e740ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:04.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.650+0000 7f7e7a66f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7e74160800 0x7f7e740cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:04.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.650+0000 7f7e7a66f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7e740adc40 0x7f7e7406adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:04.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.650+0000 7f7e7a66f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e740cde30 con 0x7f7e740adc40 2023-10-16T10:08:04.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.650+0000 7f7e7a66f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7406e170 con 0x7f7e74160800 2023-10-16T10:08:04.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.650+0000 7f7e7a66f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e740cd460 con 0x7f7e74160420 2023-10-16T10:08:04.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.650+0000 7f7e78c0c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7e740adc40 0x7f7e7406adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:04.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.650+0000 7f7e78c0c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7e740adc40 0x7f7e7406adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58782/0 (socket says 172.21.15.84:58782) 2023-10-16T10:08:04.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.650+0000 7f7e78c0c640 1 -- 172.21.15.84:0/2486637592 learned_addr learned my addr 172.21.15.84:0/2486637592 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:04.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.651+0000 7f7e73fff640 1 --2- 172.21.15.84:0/2486637592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e74160420 0x7f7e740ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:04.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.651+0000 7f7e78c0c640 1 -- 172.21.15.84:0/2486637592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e74160420 msgr2=0x7f7e740ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:04.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.651+0000 7f7e78c0c640 1 --2- 172.21.15.84:0/2486637592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e74160420 0x7f7e740ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.651+0000 7f7e78c0c640 1 -- 172.21.15.84:0/2486637592 >> v2:172.21.15.157:3300/0 conn(0x7f7e74160800 msgr2=0x7f7e740cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:04.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.651+0000 7f7e737fe640 1 --2- 172.21.15.84:0/2486637592 >> v2:172.21.15.157:3300/0 conn(0x7f7e74160800 0x7f7e740cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:04.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.651+0000 7f7e78c0c640 1 --2- 172.21.15.84:0/2486637592 >> v2:172.21.15.157:3300/0 conn(0x7f7e74160800 0x7f7e740cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.651+0000 7f7e78c0c640 1 -- 172.21.15.84:0/2486637592 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7406e3c0 con 0x7f7e740adc40 2023-10-16T10:08:04.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.652+0000 7f7e78c0c640 1 --2- 172.21.15.84:0/2486637592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e740adc40 0x7f7e7406adb0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f7e7406f600 tx=0x7f7e6c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a6a98128ed0e88 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:04.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.652+0000 7f7e72ffd640 1 -- 172.21.15.84:0/2486637592 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c031870 con 0x7f7e740adc40 2023-10-16T10:08:04.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.652+0000 7f7e72ffd640 1 -- 172.21.15.84:0/2486637592 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6c0319d0 con 0x7f7e740adc40 2023-10-16T10:08:04.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.652+0000 7f7e72ffd640 1 -- 172.21.15.84:0/2486637592 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c031d60 con 0x7f7e740adc40 2023-10-16T10:08:04.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.653+0000 7f7e7a66f640 1 -- 172.21.15.84:0/2486637592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e740adc40 msgr2=0x7f7e7406adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:04.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.653+0000 7f7e7a66f640 1 --2- 172.21.15.84:0/2486637592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e740adc40 0x7f7e7406adb0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f7e7406f600 tx=0x7f7e6c02b910 comp rx=0 tx=0).stop 2023-10-16T10:08:04.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.653+0000 7f7e7a66f640 1 -- 172.21.15.84:0/2486637592 shutdown_connections 2023-10-16T10:08:04.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.653+0000 7f7e7a66f640 1 --2- 172.21.15.84:0/2486637592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e740adc40 0x7f7e7406adb0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.653+0000 7f7e7a66f640 1 --2- 172.21.15.84:0/2486637592 >> v2:172.21.15.157:3300/0 conn(0x7f7e74160800 0x7f7e740cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.653+0000 7f7e7a66f640 1 --2- 172.21.15.84:0/2486637592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e74160420 0x7f7e740ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.653+0000 7f7e7a66f640 1 -- 172.21.15.84:0/2486637592 >> 172.21.15.84:0/2486637592 conn(0x7f7e740b0d30 msgr2=0x7f7e740b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:04.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.653+0000 7f7e7a66f640 1 -- 172.21.15.84:0/2486637592 shutdown_connections 2023-10-16T10:08:04.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.653+0000 7f7e7a66f640 1 -- 172.21.15.84:0/2486637592 wait complete. 2023-10-16T10:08:04.660 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:04.657+0000 7fc263432040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:08:04.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.655+0000 7f7e7a66f640 1 Processor -- start 2023-10-16T10:08:04.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.655+0000 7f7e7a66f640 1 -- start start 2023-10-16T10:08:04.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.656+0000 7f7e7a66f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e740adc40 0x7f7e740c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:04.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.656+0000 7f7e7a66f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e74160420 0x7f7e7409fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:04.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.656+0000 7f7e7a66f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e74160800 0x7f7e7415b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:04.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.656+0000 7f7e7a66f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e740a03c0 con 0x7f7e74160420 2023-10-16T10:08:04.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.656+0000 7f7e7a66f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e740c8670 con 0x7f7e740adc40 2023-10-16T10:08:04.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.656+0000 7f7e7a66f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e740c7a60 con 0x7f7e74160800 2023-10-16T10:08:04.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.656+0000 7f7e737fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e74160420 0x7f7e7409fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:04.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.656+0000 7f7e737fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e74160420 0x7f7e7409fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58790/0 (socket says 172.21.15.84:58790) 2023-10-16T10:08:04.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.656+0000 7f7e737fe640 1 -- 172.21.15.84:0/578863472 learned_addr learned my addr 172.21.15.84:0/578863472 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:04.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.656+0000 7f7e78c0c640 1 --2- 172.21.15.84:0/578863472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e74160800 0x7f7e7415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:04.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.656+0000 7f7e73fff640 1 --2- 172.21.15.84:0/578863472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e740adc40 0x7f7e740c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:04.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.656+0000 7f7e737fe640 1 -- 172.21.15.84:0/578863472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e74160800 msgr2=0x7f7e7415b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:04.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.656+0000 7f7e737fe640 1 --2- 172.21.15.84:0/578863472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e74160800 0x7f7e7415b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.656+0000 7f7e737fe640 1 -- 172.21.15.84:0/578863472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e740adc40 msgr2=0x7f7e740c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:04.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.656+0000 7f7e737fe640 1 --2- 172.21.15.84:0/578863472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e740adc40 0x7f7e740c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.656+0000 7f7e737fe640 1 -- 172.21.15.84:0/578863472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c0094d0 con 0x7f7e74160420 2023-10-16T10:08:04.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.656+0000 7f7e73fff640 1 --2- 172.21.15.84:0/578863472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e740adc40 0x7f7e740c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:04.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.657+0000 7f7e737fe640 1 --2- 172.21.15.84:0/578863472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e74160420 0x7f7e7409fe80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800c7f0 tx=0x7f7e6800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:04.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.657+0000 7f7e70ff9640 1 -- 172.21.15.84:0/578863472 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68004220 con 0x7f7e74160420 2023-10-16T10:08:04.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.657+0000 7f7e70ff9640 1 -- 172.21.15.84:0/578863472 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e68004380 con 0x7f7e74160420 2023-10-16T10:08:04.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.657+0000 7f7e70ff9640 1 -- 172.21.15.84:0/578863472 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68007840 con 0x7f7e74160420 2023-10-16T10:08:04.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.657+0000 7f7e7a66f640 1 -- 172.21.15.84:0/578863472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e740c7d40 con 0x7f7e74160420 2023-10-16T10:08:04.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.657+0000 7f7e7a66f640 1 -- 172.21.15.84:0/578863472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e740c8300 con 0x7f7e74160420 2023-10-16T10:08:04.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.658+0000 7f7e7a66f640 1 -- 172.21.15.84:0/578863472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e740c8820 con 0x7f7e74160420 2023-10-16T10:08:04.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.659+0000 7f7e70ff9640 1 -- 172.21.15.84:0/578863472 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 66) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f7e68007bb0 con 0x7f7e74160420 2023-10-16T10:08:04.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.661+0000 7f7e70ff9640 1 --2- 172.21.15.84:0/578863472 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f7e6006b0e0 0x7f7e6006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:04.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.661+0000 7f7e73fff640 1 -- 172.21.15.84:0/578863472 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f7e6006b0e0 msgr2=0x7f7e6006d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2209954486 2023-10-16T10:08:04.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.661+0000 7f7e70ff9640 1 -- 172.21.15.84:0/578863472 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f7e68089a50 con 0x7f7e74160420 2023-10-16T10:08:04.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.661+0000 7f7e73fff640 1 --2- 172.21.15.84:0/578863472 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f7e6006b0e0 0x7f7e6006d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:04.670 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:08:04.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.665+0000 7f7e70ff9640 1 -- 172.21.15.84:0/578863472 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7e680545c0 con 0x7f7e74160420 2023-10-16T10:08:04.677 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:08:04.683 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:04.682+0000 7facc7012040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:08:04.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.860+0000 7f7e7a66f640 1 -- 172.21.15.84:0/578863472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e7406e670 con 0x7f7e74160420 2023-10-16T10:08:04.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.861+0000 7f7e70ff9640 1 -- 172.21.15.84:0/578863472 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16995 (secure 0 0 0) 0x7f7e6801e020 con 0x7f7e74160420 2023-10-16T10:08:04.864 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:04.864 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":125,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:07:43.574763+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:07:26.607199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-16T10:07:27.832706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1284479959":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/2038404786":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/644840609":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/3794895754":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/3269894694":"2023-10-17T10:07:43.574635+0000","172.21.15.84:6833/4292025878":"2023-10-17T10:07:43.574635+0000","172.21.15.84:6832/4292025878":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/1934706185":"2023-10-17T10:07:43.574635+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:08:04.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.862+0000 7f7e73fff640 1 -- 172.21.15.84:0/578863472 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f7e6006b0e0 msgr2=0x7f7e6006d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2209954486 2023-10-16T10:08:04.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.862+0000 7f7e73fff640 1 --2- 172.21.15.84:0/578863472 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f7e6006b0e0 0x7f7e6006d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:08:04.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.864+0000 7f7e7a66f640 1 -- 172.21.15.84:0/578863472 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f7e6006b0e0 msgr2=0x7f7e6006d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:04.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.864+0000 7f7e7a66f640 1 --2- 172.21.15.84:0/578863472 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f7e6006b0e0 0x7f7e6006d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.864+0000 7f7e7a66f640 1 -- 172.21.15.84:0/578863472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e74160420 msgr2=0x7f7e7409fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:04.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.864+0000 7f7e7a66f640 1 --2- 172.21.15.84:0/578863472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e74160420 0x7f7e7409fe80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800c7f0 tx=0x7f7e6800ccb0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.864+0000 7f7e7a66f640 1 -- 172.21.15.84:0/578863472 shutdown_connections 2023-10-16T10:08:04.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.864+0000 7f7e7a66f640 1 --2- 172.21.15.84:0/578863472 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f7e6006b0e0 0x7f7e6006d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.864+0000 7f7e7a66f640 1 --2- 172.21.15.84:0/578863472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e74160800 0x7f7e7415b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.864+0000 7f7e7a66f640 1 --2- 172.21.15.84:0/578863472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e74160420 0x7f7e7409fe80 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.864+0000 7f7e7a66f640 1 --2- 172.21.15.84:0/578863472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e740adc40 0x7f7e740c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:04.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.864+0000 7f7e7a66f640 1 -- 172.21.15.84:0/578863472 >> 172.21.15.84:0/578863472 conn(0x7f7e740b0d30 msgr2=0x7f7e740cdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:04.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.864+0000 7f7e7a66f640 1 -- 172.21.15.84:0/578863472 shutdown_connections 2023-10-16T10:08:04.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:04.864+0000 7f7e7a66f640 1 -- 172.21.15.84:0/578863472 wait complete. 2023-10-16T10:08:04.883 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:07:26.607199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:07:27.832706+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:08:04.884 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:08:05.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.079+0000 7f268b59e640 1 Processor -- start 2023-10-16T10:08:05.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.080+0000 7f268b59e640 1 -- start start 2023-10-16T10:08:05.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.081+0000 7f268b59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f268c0cfc40 0x7f268c0d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:05.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.081+0000 7f268b59e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f268c0d0970 con 0x7f268c0b1ca0 2023-10-16T10:08:05.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.081+0000 7f268b59e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f268c0d0ea0 con 0x7f268c0b2080 2023-10-16T10:08:05.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.081+0000 7f268b59e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268c0d0bc0 con 0x7f268c0cfc40 2023-10-16T10:08:05.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.081+0000 7f268a59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f268c0b1ca0 0x7f268c0ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47110/0 (socket says 172.21.15.84:47110) 2023-10-16T10:08:05.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.081+0000 7f268a59c640 1 -- 172.21.15.84:0/1194462695 learned_addr learned my addr 172.21.15.84:0/1194462695 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:05.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.082+0000 7f268ad9d640 1 --2- 172.21.15.84:0/1194462695 >> v2:172.21.15.157:3300/0 conn(0x7f268c0cfc40 0x7f268c0d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:05.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.082+0000 7f268959a640 1 -- 172.21.15.84:0/1194462695 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2319287949 0 0) 0x7f268c0d0970 con 0x7f268c0b1ca0 2023-10-16T10:08:05.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.082+0000 7f268959a640 1 -- 172.21.15.84:0/1194462695 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2674003520 con 0x7f268c0b1ca0 2023-10-16T10:08:05.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.082+0000 7f268959a640 1 -- 172.21.15.84:0/1194462695 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2166346431 0 0) 0x7f2674003520 con 0x7f268c0b1ca0 2023-10-16T10:08:05.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.082+0000 7f268959a640 1 -- 172.21.15.84:0/1194462695 >> v2:172.21.15.157:3300/0 conn(0x7f268c0cfc40 msgr2=0x7f268c0d36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:05.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.082+0000 7f268959a640 1 --2- 172.21.15.84:0/1194462695 >> v2:172.21.15.157:3300/0 conn(0x7f268c0cfc40 0x7f268c0d36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.082+0000 7f268959a640 1 -- 172.21.15.84:0/1194462695 >> v1:172.21.15.157:6789/0 conn(0x7f268c0b2080 legacy=0x7f268c0b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:05.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.082+0000 7f268959a640 1 -- 172.21.15.84:0/1194462695 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c0cd940 con 0x7f268c0b1ca0 2023-10-16T10:08:05.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.082+0000 7f268959a640 1 -- 172.21.15.84:0/1194462695 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f267c001670 con 0x7f268c0b1ca0 2023-10-16T10:08:05.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.082+0000 7f268959a640 1 -- 172.21.15.84:0/1194462695 >> v1:172.21.15.84:6789/0 conn(0x7f268c0b1ca0 legacy=0x7f268c0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:05.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.082+0000 7f268959a640 1 --2- 172.21.15.84:0/1194462695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2674003970 0x7f2674003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:05.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.082+0000 7f268959a640 1 --2- 172.21.15.84:0/1194462695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26740047c0 0x7f2674004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:05.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.082+0000 7f268959a640 1 --2- 172.21.15.84:0/1194462695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2674005180 0x7f267400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:05.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.082+0000 7f268959a640 1 -- 172.21.15.84:0/1194462695 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0d0bc0 con 0x7f2674003970 2023-10-16T10:08:05.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.082+0000 7f268959a640 1 -- 172.21.15.84:0/1194462695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267400c030 con 0x7f2674005180 2023-10-16T10:08:05.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.082+0000 7f268959a640 1 -- 172.21.15.84:0/1194462695 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267400c1f0 con 0x7f26740047c0 2023-10-16T10:08:05.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.083+0000 7f268a59c640 1 --2- 172.21.15.84:0/1194462695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2674003970 0x7f2674003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:05.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.083+0000 7f268ad9d640 1 --2- 172.21.15.84:0/1194462695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2674005180 0x7f267400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:05.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.083+0000 7f2689d9b640 1 --2- 172.21.15.84:0/1194462695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26740047c0 0x7f2674004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:05.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.083+0000 7f268a59c640 1 -- 172.21.15.84:0/1194462695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26740047c0 msgr2=0x7f2674004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:05.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.083+0000 7f268a59c640 1 --2- 172.21.15.84:0/1194462695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26740047c0 0x7f2674004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.083+0000 7f268a59c640 1 -- 172.21.15.84:0/1194462695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2674005180 msgr2=0x7f267400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:05.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.083+0000 7f268a59c640 1 --2- 172.21.15.84:0/1194462695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2674005180 0x7f267400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.083+0000 7f268a59c640 1 -- 172.21.15.84:0/1194462695 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2674003520 con 0x7f2674003970 2023-10-16T10:08:05.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.083+0000 7f268ad9d640 1 -- 172.21.15.84:0/1194462695 reap_dead start 2023-10-16T10:08:05.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.083+0000 7f2689d9b640 1 --2- 172.21.15.84:0/1194462695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26740047c0 0x7f2674004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:05.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.084+0000 7f268a59c640 1 --2- 172.21.15.84:0/1194462695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2674003970 0x7f2674003d90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f267c00c240 tx=0x7f26740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f121e0e80dba66 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:05.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.084+0000 7f268959a640 1 -- 172.21.15.84:0/1194462695 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c005ad0 con 0x7f2674003970 2023-10-16T10:08:05.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.084+0000 7f268959a640 1 -- 172.21.15.84:0/1194462695 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c005c30 con 0x7f2674003970 2023-10-16T10:08:05.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.084+0000 7f268959a640 1 -- 172.21.15.84:0/1194462695 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c02cab0 con 0x7f2674003970 2023-10-16T10:08:05.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.085+0000 7f268b59e640 1 -- 172.21.15.84:0/1194462695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2674003970 msgr2=0x7f2674003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:05.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.085+0000 7f268b59e640 1 --2- 172.21.15.84:0/1194462695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2674003970 0x7f2674003d90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f267c00c240 tx=0x7f26740057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.085+0000 7f268b59e640 1 -- 172.21.15.84:0/1194462695 shutdown_connections 2023-10-16T10:08:05.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.085+0000 7f268b59e640 1 --2- 172.21.15.84:0/1194462695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2674003970 0x7f2674003d90 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.085+0000 7f268b59e640 1 -- 172.21.15.84:0/1194462695 >> 172.21.15.84:0/1194462695 conn(0x7f268c0d2b90 msgr2=0x7f268c0d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:05.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.085+0000 7f268b59e640 1 -- 172.21.15.84:0/1194462695 shutdown_connections 2023-10-16T10:08:05.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.085+0000 7f268b59e640 1 -- 172.21.15.84:0/1194462695 wait complete. 2023-10-16T10:08:05.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.087+0000 7f268b59e640 1 Processor -- start 2023-10-16T10:08:05.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.087+0000 7f268b59e640 1 -- start start 2023-10-16T10:08:05.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.088+0000 7f268b59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2674003970 0x7f268c1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:05.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.088+0000 7f268b59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f268c1e7200 0x7f268c0b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:05.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.088+0000 7f268b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f268c1e7680 0x7f268c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:05.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.088+0000 7f268b59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c1f6760 con 0x7f268c1e7680 2023-10-16T10:08:05.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.088+0000 7f268b59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c1f5b90 con 0x7f268c1e7200 2023-10-16T10:08:05.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.088+0000 7f268b59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c1f5e30 con 0x7f2674003970 2023-10-16T10:08:05.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.088+0000 7f268ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f268c1e7680 0x7f268c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:05.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.088+0000 7f268ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f268c1e7680 0x7f268c0b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58802/0 (socket says 172.21.15.84:58802) 2023-10-16T10:08:05.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.088+0000 7f268ad9d640 1 -- 172.21.15.84:0/2086904796 learned_addr learned my addr 172.21.15.84:0/2086904796 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:05.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.089+0000 7f268a59c640 1 --2- 172.21.15.84:0/2086904796 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2674003970 0x7f268c1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:05.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.089+0000 7f2689d9b640 1 --2- 172.21.15.84:0/2086904796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f268c1e7200 0x7f268c0b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:05.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.089+0000 7f268ad9d640 1 -- 172.21.15.84:0/2086904796 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2674003970 msgr2=0x7f268c1539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:05.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.089+0000 7f268ad9d640 1 --2- 172.21.15.84:0/2086904796 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2674003970 0x7f268c1539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.089+0000 7f268ad9d640 1 -- 172.21.15.84:0/2086904796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f268c1e7200 msgr2=0x7f268c0b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:05.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.089+0000 7f268ad9d640 1 --2- 172.21.15.84:0/2086904796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f268c1e7200 0x7f268c0b2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.089+0000 7f268ad9d640 1 -- 172.21.15.84:0/2086904796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c1f6110 con 0x7f268c1e7680 2023-10-16T10:08:05.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.089+0000 7f268ad9d640 1 --2- 172.21.15.84:0/2086904796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f268c1e7680 0x7f268c0b31a0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f268c0d0020 tx=0x7f26840115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:05.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.089+0000 7f267affd640 1 -- 172.21.15.84:0/2086904796 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268400e070 con 0x7f268c1e7680 2023-10-16T10:08:05.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.089+0000 7f267affd640 1 -- 172.21.15.84:0/2086904796 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2684011db0 con 0x7f268c1e7680 2023-10-16T10:08:05.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.089+0000 7f268b59e640 1 -- 172.21.15.84:0/2086904796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268c1f63f0 con 0x7f268c1e7680 2023-10-16T10:08:05.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.089+0000 7f267affd640 1 -- 172.21.15.84:0/2086904796 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684003990 con 0x7f268c1e7680 2023-10-16T10:08:05.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.090+0000 7f268b59e640 1 -- 172.21.15.84:0/2086904796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268c1e6be0 con 0x7f268c1e7680 2023-10-16T10:08:05.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.091+0000 7f268b59e640 1 -- 172.21.15.84:0/2086904796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f268c04e9b0 con 0x7f268c1e7680 2023-10-16T10:08:05.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.092+0000 7f267affd640 1 -- 172.21.15.84:0/2086904796 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 66) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f2684003af0 con 0x7f268c1e7680 2023-10-16T10:08:05.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.092+0000 7f267affd640 1 --2- 172.21.15.84:0/2086904796 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f266c06b0b0 0x7f266c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:05.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.093+0000 7f267affd640 1 -- 172.21.15.84:0/2086904796 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f268408a430 con 0x7f268c1e7680 2023-10-16T10:08:05.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.093+0000 7f268a59c640 1 -- 172.21.15.84:0/2086904796 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f266c06b0b0 msgr2=0x7f266c06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2209954486 2023-10-16T10:08:05.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.093+0000 7f268a59c640 1 --2- 172.21.15.84:0/2086904796 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f266c06b0b0 0x7f266c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:05.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.101+0000 7f267affd640 1 -- 172.21.15.84:0/2086904796 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2684054f80 con 0x7f268c1e7680 2023-10-16T10:08:05.114 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:05.113+0000 7fc263432040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:08:05.163 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:05.162+0000 7facc7012040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:08:05.166 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:05.164+0000 7fc263432040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:08:05.216 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:05.214+0000 7fc263432040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:08:05.217 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:05.216+0000 7facc7012040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:08:05.269 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:05.268+0000 7facc7012040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:08:05.276 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:05.275+0000 7fc263432040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:08:05.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.294+0000 7f268a59c640 1 -- 172.21.15.84:0/2086904796 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f266c06b0b0 msgr2=0x7f266c06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2209954486 2023-10-16T10:08:05.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.294+0000 7f268a59c640 1 --2- 172.21.15.84:0/2086904796 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f266c06b0b0 0x7f266c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:08:05.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.296+0000 7f268b59e640 1 -- 172.21.15.84:0/2086904796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f268c1f5f70 con 0x7f268c1e7680 2023-10-16T10:08:05.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.297+0000 7f267affd640 1 -- 172.21.15.84:0/2086904796 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f268c1f5f70 con 0x7f268c1e7680 2023-10-16T10:08:05.299 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:08:05.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.299+0000 7f268b59e640 1 -- 172.21.15.84:0/2086904796 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f266c06b0b0 msgr2=0x7f266c06d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:05.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.299+0000 7f268b59e640 1 --2- 172.21.15.84:0/2086904796 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f266c06b0b0 0x7f266c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.299+0000 7f268b59e640 1 -- 172.21.15.84:0/2086904796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f268c1e7680 msgr2=0x7f268c0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:05.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.299+0000 7f268b59e640 1 --2- 172.21.15.84:0/2086904796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f268c1e7680 0x7f268c0b31a0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f268c0d0020 tx=0x7f26840115a0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.300+0000 7f268b59e640 1 -- 172.21.15.84:0/2086904796 shutdown_connections 2023-10-16T10:08:05.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.300+0000 7f268b59e640 1 --2- 172.21.15.84:0/2086904796 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f266c06b0b0 0x7f266c06d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.300+0000 7f268b59e640 1 --2- 172.21.15.84:0/2086904796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f268c1e7680 0x7f268c0b31a0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.300+0000 7f268b59e640 1 --2- 172.21.15.84:0/2086904796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f268c1e7200 0x7f268c0b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.300+0000 7f268b59e640 1 --2- 172.21.15.84:0/2086904796 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2674003970 0x7f268c1539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.300+0000 7f268b59e640 1 -- 172.21.15.84:0/2086904796 >> 172.21.15.84:0/2086904796 conn(0x7f268c0d2b90 msgr2=0x7f268c15f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:05.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.300+0000 7f268b59e640 1 -- 172.21.15.84:0/2086904796 shutdown_connections 2023-10-16T10:08:05.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.300+0000 7f268b59e640 1 -- 172.21.15.84:0/2086904796 wait complete. 2023-10-16T10:08:05.319 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:08:05.325 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:05.324+0000 7fc263432040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:08:05.334 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:05.332+0000 7facc7012040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:08:05.386 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:05.384+0000 7facc7012040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:08:05.435 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:05.433+0000 7fc263432040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:08:05.496 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:05.495+0000 7fc263432040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:08:05.503 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:05.502+0000 7facc7012040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:08:05.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.518+0000 7ff3ed27c640 1 Processor -- start 2023-10-16T10:08:05.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.518+0000 7ff3ed27c640 1 -- start start 2023-10-16T10:08:05.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.519+0000 7ff3ed27c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff3e815d330 0x7ff3e81618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:05.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.519+0000 7ff3ed27c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff3e80661c0 0x7ff3e80ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:05.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.520+0000 7ff3ed27c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e80d0890 con 0x7ff3e806c7d0 2023-10-16T10:08:05.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.520+0000 7ff3ed27c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e806b1e0 con 0x7ff3e80661c0 2023-10-16T10:08:05.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.520+0000 7ff3ed27c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e80d05b0 con 0x7ff3e815d330 2023-10-16T10:08:05.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.520+0000 7ff3e77fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff3e80661c0 0x7ff3e80ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:05.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.520+0000 7ff3e77fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff3e80661c0 0x7ff3e80ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58810/0 (socket says 172.21.15.84:58810) 2023-10-16T10:08:05.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.520+0000 7ff3e77fe640 1 -- 172.21.15.84:0/111680875 learned_addr learned my addr 172.21.15.84:0/111680875 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:05.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.520+0000 7ff3e6ffd640 1 --2- 172.21.15.84:0/111680875 >> v2:172.21.15.157:3300/0 conn(0x7ff3e815d330 0x7ff3e81618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:05.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.520+0000 7ff3e77fe640 1 -- 172.21.15.84:0/111680875 >> v2:172.21.15.157:3300/0 conn(0x7ff3e815d330 msgr2=0x7ff3e81618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:05.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.520+0000 7ff3e77fe640 1 --2- 172.21.15.84:0/111680875 >> v2:172.21.15.157:3300/0 conn(0x7ff3e815d330 0x7ff3e81618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.520+0000 7ff3e77fe640 1 -- 172.21.15.84:0/111680875 >> v1:172.21.15.157:6789/0 conn(0x7ff3e806c7d0 legacy=0x7ff3e8065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:05.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.521+0000 7ff3e77fe640 1 -- 172.21.15.84:0/111680875 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e8160dd0 con 0x7ff3e80661c0 2023-10-16T10:08:05.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.522+0000 7ff3e77fe640 1 --2- 172.21.15.84:0/111680875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80661c0 0x7ff3e80ce9b0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80744f0 tx=0x7ff3e002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d6e7afdeb121a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:05.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.522+0000 7ff3e5ffb640 1 -- 172.21.15.84:0/111680875 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e002c9f0 con 0x7ff3e80661c0 2023-10-16T10:08:05.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.522+0000 7ff3e5ffb640 1 -- 172.21.15.84:0/111680875 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e002cb50 con 0x7ff3e80661c0 2023-10-16T10:08:05.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.523+0000 7ff3e5ffb640 1 -- 172.21.15.84:0/111680875 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e00343f0 con 0x7ff3e80661c0 2023-10-16T10:08:05.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.523+0000 7ff3ed27c640 1 -- 172.21.15.84:0/111680875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80661c0 msgr2=0x7ff3e80ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:05.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.523+0000 7ff3ed27c640 1 --2- 172.21.15.84:0/111680875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80661c0 0x7ff3e80ce9b0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80744f0 tx=0x7ff3e002be50 comp rx=0 tx=0).stop 2023-10-16T10:08:05.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.523+0000 7ff3ed27c640 1 -- 172.21.15.84:0/111680875 shutdown_connections 2023-10-16T10:08:05.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.523+0000 7ff3ed27c640 1 --2- 172.21.15.84:0/111680875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80661c0 0x7ff3e80ce9b0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.523+0000 7ff3ed27c640 1 --2- 172.21.15.84:0/111680875 >> v2:172.21.15.157:3300/0 conn(0x7ff3e815d330 0x7ff3e81618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.523+0000 7ff3ed27c640 1 -- 172.21.15.84:0/111680875 >> 172.21.15.84:0/111680875 conn(0x7ff3e80d0ab0 msgr2=0x7ff3e80d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:05.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.523+0000 7ff3ed27c640 1 -- 172.21.15.84:0/111680875 shutdown_connections 2023-10-16T10:08:05.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.523+0000 7ff3ed27c640 1 -- 172.21.15.84:0/111680875 wait complete. 2023-10-16T10:08:05.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.525+0000 7ff3ed27c640 1 Processor -- start 2023-10-16T10:08:05.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.525+0000 7ff3ed27c640 1 -- start start 2023-10-16T10:08:05.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.526+0000 7ff3ed27c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80661c0 0x7ff3e80c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:05.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.526+0000 7ff3ed27c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e806c7d0 0x7ff3e81552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:05.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.526+0000 7ff3ed27c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3e815d330 0x7ff3e815eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:05.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.526+0000 7ff3ed27c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e8160500 con 0x7ff3e80661c0 2023-10-16T10:08:05.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.526+0000 7ff3ed27c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e81557f0 con 0x7ff3e806c7d0 2023-10-16T10:08:05.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.526+0000 7ff3ed27c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e815f740 con 0x7ff3e815d330 2023-10-16T10:08:05.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.526+0000 7ff3e6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80661c0 0x7ff3e80c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:05.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.526+0000 7ff3e6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80661c0 0x7ff3e80c9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58820/0 (socket says 172.21.15.84:58820) 2023-10-16T10:08:05.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.526+0000 7ff3e6ffd640 1 -- 172.21.15.84:0/2107422374 learned_addr learned my addr 172.21.15.84:0/2107422374 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:05.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.527+0000 7ff3e6ffd640 1 -- 172.21.15.84:0/2107422374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3e815d330 msgr2=0x7ff3e815eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:05.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.527+0000 7ff3e6ffd640 1 --2- 172.21.15.84:0/2107422374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3e815d330 0x7ff3e815eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.527+0000 7ff3e6ffd640 1 -- 172.21.15.84:0/2107422374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e806c7d0 msgr2=0x7ff3e81552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:05.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.527+0000 7ff3e67fc640 1 --2- 172.21.15.84:0/2107422374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e806c7d0 0x7ff3e81552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:05.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.527+0000 7ff3e6ffd640 1 --2- 172.21.15.84:0/2107422374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e806c7d0 0x7ff3e81552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.527+0000 7ff3e6ffd640 1 -- 172.21.15.84:0/2107422374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e0009b30 con 0x7ff3e80661c0 2023-10-16T10:08:05.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.527+0000 7ff3e6ffd640 1 --2- 172.21.15.84:0/2107422374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80661c0 0x7ff3e80c9910 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800d5f0 tx=0x7ff3d800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:05.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.527+0000 7ff3cf7fe640 1 -- 172.21.15.84:0/2107422374 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d80125f0 con 0x7ff3e80661c0 2023-10-16T10:08:05.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.527+0000 7ff3cf7fe640 1 -- 172.21.15.84:0/2107422374 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d8012750 con 0x7ff3e80661c0 2023-10-16T10:08:05.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.527+0000 7ff3ed27c640 1 -- 172.21.15.84:0/2107422374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e815d7b0 con 0x7ff3e80661c0 2023-10-16T10:08:05.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.527+0000 7ff3cf7fe640 1 -- 172.21.15.84:0/2107422374 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8012b00 con 0x7ff3e80661c0 2023-10-16T10:08:05.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.528+0000 7ff3ed27c640 1 -- 172.21.15.84:0/2107422374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e815dd70 con 0x7ff3e80661c0 2023-10-16T10:08:05.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.529+0000 7ff3ed27c640 1 -- 172.21.15.84:0/2107422374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e804e160 con 0x7ff3e80661c0 2023-10-16T10:08:05.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.530+0000 7ff3cf7fe640 1 -- 172.21.15.84:0/2107422374 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 66) v1 ==== 89445+0+0 (secure 0 0 0) 0x7ff3d8012c60 con 0x7ff3e80661c0 2023-10-16T10:08:05.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.530+0000 7ff3cf7fe640 1 --2- 172.21.15.84:0/2107422374 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7ff3d406b0b0 0x7ff3d406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:05.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.531+0000 7ff3e67fc640 1 -- 172.21.15.84:0/2107422374 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7ff3d406b0b0 msgr2=0x7ff3d406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2209954486 2023-10-16T10:08:05.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.531+0000 7ff3cf7fe640 1 -- 172.21.15.84:0/2107422374 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6053+0+0 (secure 0 0 0) 0x7ff3d808a0e0 con 0x7ff3e80661c0 2023-10-16T10:08:05.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.531+0000 7ff3e67fc640 1 --2- 172.21.15.84:0/2107422374 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7ff3d406b0b0 0x7ff3d406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:05.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.539+0000 7ff3cf7fe640 1 -- 172.21.15.84:0/2107422374 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff3d8054c30 con 0x7ff3e80661c0 2023-10-16T10:08:05.546 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:05.545+0000 7fc263432040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:08:05.574 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:05.573+0000 7facc7012040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:08:05.604 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:05.602+0000 7fc263432040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:08:05.626 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:05.624+0000 7facc7012040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:08:05.654 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:05.652+0000 7fc263432040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:08:05.685 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:05.683+0000 7facc7012040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:08:05.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.732+0000 7ff3e67fc640 1 -- 172.21.15.84:0/2107422374 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7ff3d406b0b0 msgr2=0x7ff3d406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2209954486 2023-10-16T10:08:05.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.732+0000 7ff3e67fc640 1 --2- 172.21.15.84:0/2107422374 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7ff3d406b0b0 0x7ff3d406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:08:05.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.732+0000 7ff3ed27c640 1 -- 172.21.15.84:0/2107422374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff3e806f3a0 con 0x7ff3e80661c0 2023-10-16T10:08:05.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.733+0000 7ff3cf7fe640 1 -- 172.21.15.84:0/2107422374 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff3d8058250 con 0x7ff3e80661c0 2023-10-16T10:08:05.735 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T10:08:05.736 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:05.735+0000 7facc7012040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:08:05.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.736+0000 7ff3ed27c640 1 -- 172.21.15.84:0/2107422374 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7ff3d406b0b0 msgr2=0x7ff3d406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:05.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.736+0000 7ff3ed27c640 1 --2- 172.21.15.84:0/2107422374 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7ff3d406b0b0 0x7ff3d406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.736+0000 7ff3ed27c640 1 -- 172.21.15.84:0/2107422374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80661c0 msgr2=0x7ff3e80c9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:05.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.736+0000 7ff3ed27c640 1 --2- 172.21.15.84:0/2107422374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80661c0 0x7ff3e80c9910 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800d5f0 tx=0x7ff3d800dab0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.736+0000 7ff3ed27c640 1 -- 172.21.15.84:0/2107422374 shutdown_connections 2023-10-16T10:08:05.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.736+0000 7ff3ed27c640 1 --2- 172.21.15.84:0/2107422374 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7ff3d406b0b0 0x7ff3d406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.736+0000 7ff3ed27c640 1 --2- 172.21.15.84:0/2107422374 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3e815d330 0x7ff3e815eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.736+0000 7ff3ed27c640 1 --2- 172.21.15.84:0/2107422374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e806c7d0 0x7ff3e81552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.736+0000 7ff3ed27c640 1 --2- 172.21.15.84:0/2107422374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80661c0 0x7ff3e80c9910 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:05.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.736+0000 7ff3ed27c640 1 -- 172.21.15.84:0/2107422374 >> 172.21.15.84:0/2107422374 conn(0x7ff3e80d0ab0 msgr2=0x7ff3e815a920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:05.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.736+0000 7ff3ed27c640 1 -- 172.21.15.84:0/2107422374 shutdown_connections 2023-10-16T10:08:05.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.736+0000 7ff3ed27c640 1 -- 172.21.15.84:0/2107422374 wait complete. 2023-10-16T10:08:05.755 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:08:05.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.993+0000 7f9ee741f640 1 Processor -- start 2023-10-16T10:08:05.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.994+0000 7f9ee741f640 1 -- start start 2023-10-16T10:08:05.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.995+0000 7f9ee741f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9ee00c9800 0x7f9ee015d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:05.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.995+0000 7f9ee741f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee006af40 0x7f9ee015dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:05.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.995+0000 7f9ee741f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee015a120 con 0x7f9ee0159d40 2023-10-16T10:08:05.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.995+0000 7f9ee741f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee015e3b0 con 0x7f9ee00c9800 2023-10-16T10:08:05.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.995+0000 7f9ee741f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee015e690 con 0x7f9ee006af40 2023-10-16T10:08:05.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.996+0000 7f9ee49ba640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9ee00c9800 0x7f9ee015d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:05.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.996+0000 7f9ee51bb640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9ee0159d40 0x7f9ee0071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33116/0 (socket says 172.21.15.84:33116) 2023-10-16T10:08:05.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.996+0000 7f9ee51bb640 1 -- 172.21.15.84:0/81345627 learned_addr learned my addr 172.21.15.84:0/81345627 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:05.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.996+0000 7f9ee59bc640 1 --2- 172.21.15.84:0/81345627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee006af40 0x7f9ee015dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:06.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.996+0000 7f9ee49ba640 1 -- 172.21.15.84:0/81345627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee006af40 msgr2=0x7f9ee015dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:06.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.996+0000 7f9ee49ba640 1 --2- 172.21.15.84:0/81345627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee006af40 0x7f9ee015dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.996+0000 7f9ee49ba640 1 -- 172.21.15.84:0/81345627 >> v1:172.21.15.84:6789/0 conn(0x7f9ee0159d40 legacy=0x7f9ee0071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:06.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.996+0000 7f9ee49ba640 1 -- 172.21.15.84:0/81345627 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee0070f60 con 0x7f9ee00c9800 2023-10-16T10:08:06.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.996+0000 7f9ed7fff640 1 -- 172.21.15.84:0/81345627 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240357151 0 0) 0x7f9ee015a120 con 0x7f9ee0159d40 2023-10-16T10:08:06.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.997+0000 7f9ee49ba640 1 --2- 172.21.15.84:0/81345627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee00c9800 0x7f9ee015d7e0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00097a0 tx=0x7f9ed002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173f6f07ae2789d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:06.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.998+0000 7f9ed7fff640 1 -- 172.21.15.84:0/81345627 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed002d040 con 0x7f9ee00c9800 2023-10-16T10:08:06.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.998+0000 7f9ed7fff640 1 -- 172.21.15.84:0/81345627 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed0031810 con 0x7f9ee00c9800 2023-10-16T10:08:06.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.998+0000 7f9ed7fff640 1 -- 172.21.15.84:0/81345627 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0031ba0 con 0x7f9ee00c9800 2023-10-16T10:08:06.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.998+0000 7f9ee741f640 1 -- 172.21.15.84:0/81345627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee00c9800 msgr2=0x7f9ee015d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:06.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.998+0000 7f9ee741f640 1 --2- 172.21.15.84:0/81345627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee00c9800 0x7f9ee015d7e0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00097a0 tx=0x7f9ed002b8b0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.998+0000 7f9ee741f640 1 -- 172.21.15.84:0/81345627 shutdown_connections 2023-10-16T10:08:06.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.998+0000 7f9ee741f640 1 --2- 172.21.15.84:0/81345627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee006af40 0x7f9ee015dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.998+0000 7f9ee741f640 1 --2- 172.21.15.84:0/81345627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee00c9800 0x7f9ee015d7e0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.998+0000 7f9ee741f640 1 -- 172.21.15.84:0/81345627 >> 172.21.15.84:0/81345627 conn(0x7f9ee006bc10 msgr2=0x7f9ee006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:06.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.999+0000 7f9ee741f640 1 -- 172.21.15.84:0/81345627 shutdown_connections 2023-10-16T10:08:06.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:05.999+0000 7f9ee741f640 1 -- 172.21.15.84:0/81345627 wait complete. 2023-10-16T10:08:06.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.000+0000 7f9ee741f640 1 Processor -- start 2023-10-16T10:08:06.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.000+0000 7f9ee741f640 1 -- start start 2023-10-16T10:08:06.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.001+0000 7f9ee741f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee006af40 0x7f9ee0153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:06.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.001+0000 7f9ee741f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee00c9800 0x7f9ee00ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:06.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.001+0000 7f9ee741f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee0159d40 0x7f9ee00acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:06.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.001+0000 7f9ee741f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee01f3000 con 0x7f9ee00c9800 2023-10-16T10:08:06.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.001+0000 7f9ee741f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee01f32e0 con 0x7f9ee0159d40 2023-10-16T10:08:06.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.001+0000 7f9ee741f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee01f35c0 con 0x7f9ee006af40 2023-10-16T10:08:06.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.002+0000 7f9ee49ba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee00c9800 0x7f9ee00ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:06.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.002+0000 7f9ee49ba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee00c9800 0x7f9ee00ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57390/0 (socket says 172.21.15.84:57390) 2023-10-16T10:08:06.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.002+0000 7f9ee49ba640 1 -- 172.21.15.84:0/4076018661 learned_addr learned my addr 172.21.15.84:0/4076018661 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:06.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.002+0000 7f9ee51bb640 1 --2- 172.21.15.84:0/4076018661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee006af40 0x7f9ee0153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:06.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.002+0000 7f9ee59bc640 1 --2- 172.21.15.84:0/4076018661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee0159d40 0x7f9ee00acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:06.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.002+0000 7f9ee49ba640 1 -- 172.21.15.84:0/4076018661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee006af40 msgr2=0x7f9ee0153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:06.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.002+0000 7f9ee49ba640 1 --2- 172.21.15.84:0/4076018661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee006af40 0x7f9ee0153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.002+0000 7f9ee49ba640 1 -- 172.21.15.84:0/4076018661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee0159d40 msgr2=0x7f9ee00acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:06.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.002+0000 7f9ee49ba640 1 --2- 172.21.15.84:0/4076018661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee0159d40 0x7f9ee00acdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.002+0000 7f9ee49ba640 1 -- 172.21.15.84:0/4076018661 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed0009400 con 0x7f9ee00c9800 2023-10-16T10:08:06.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.002+0000 7f9ee59bc640 1 --2- 172.21.15.84:0/4076018661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee0159d40 0x7f9ee00acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:06.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.003+0000 7f9ee49ba640 1 --2- 172.21.15.84:0/4076018661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee00c9800 0x7f9ee00ac890 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0000c00 tx=0x7f9ed00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:06.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.003+0000 7f9ed5ffb640 1 -- 172.21.15.84:0/4076018661 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed003ad60 con 0x7f9ee00c9800 2023-10-16T10:08:06.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.003+0000 7f9ed5ffb640 1 -- 172.21.15.84:0/4076018661 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed0044400 con 0x7f9ee00c9800 2023-10-16T10:08:06.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.003+0000 7f9ed5ffb640 1 -- 172.21.15.84:0/4076018661 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed00447b0 con 0x7f9ee00c9800 2023-10-16T10:08:06.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.003+0000 7f9ee741f640 1 -- 172.21.15.84:0/4076018661 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee01f38a0 con 0x7f9ee00c9800 2023-10-16T10:08:06.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.003+0000 7f9ee741f640 1 -- 172.21.15.84:0/4076018661 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee0153ff0 con 0x7f9ee00c9800 2023-10-16T10:08:06.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.005+0000 7f9ee741f640 1 -- 172.21.15.84:0/4076018661 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee00b1dd0 con 0x7f9ee00c9800 2023-10-16T10:08:06.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.005+0000 7f9ed5ffb640 1 -- 172.21.15.84:0/4076018661 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 66) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f9ed004d440 con 0x7f9ee00c9800 2023-10-16T10:08:06.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.006+0000 7f9ed5ffb640 1 --2- 172.21.15.84:0/4076018661 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f9ec406b0b0 0x7f9ec406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:06.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.006+0000 7f9ee51bb640 1 -- 172.21.15.84:0/4076018661 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f9ec406b0b0 msgr2=0x7f9ec406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2209954486 2023-10-16T10:08:06.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.007+0000 7f9ed5ffb640 1 -- 172.21.15.84:0/4076018661 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f9ed00ac0c0 con 0x7f9ee00c9800 2023-10-16T10:08:06.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.007+0000 7f9ee51bb640 1 --2- 172.21.15.84:0/4076018661 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f9ec406b0b0 0x7f9ec406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:06.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.011+0000 7f9ed5ffb640 1 -- 172.21.15.84:0/4076018661 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9ed0076c20 con 0x7f9ee00c9800 2023-10-16T10:08:06.033 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:06.032+0000 7fc263432040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:08:06.135 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:06.133+0000 7facc7012040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:08:06.195 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:06.193+0000 7fc263432040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:08:06.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.206+0000 7f9ee741f640 1 -- 172.21.15.84:0/4076018661 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ee00c6d00 con 0x7f9ee00c9800 2023-10-16T10:08:06.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.206+0000 7f9ed5ffb640 1 -- 172.21.15.84:0/4076018661 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9ed007a240 con 0x7f9ee00c9800 2023-10-16T10:08:06.208 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T10:08:06.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.208+0000 7f9ee51bb640 1 -- 172.21.15.84:0/4076018661 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f9ec406b0b0 msgr2=0x7f9ec406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2209954486 2023-10-16T10:08:06.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.208+0000 7f9ee51bb640 1 --2- 172.21.15.84:0/4076018661 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f9ec406b0b0 0x7f9ec406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:08:06.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.210+0000 7f9ee741f640 1 -- 172.21.15.84:0/4076018661 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f9ec406b0b0 msgr2=0x7f9ec406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:06.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.210+0000 7f9ee741f640 1 --2- 172.21.15.84:0/4076018661 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f9ec406b0b0 0x7f9ec406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.210+0000 7f9ee741f640 1 -- 172.21.15.84:0/4076018661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee00c9800 msgr2=0x7f9ee00ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:06.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.210+0000 7f9ee741f640 1 --2- 172.21.15.84:0/4076018661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee00c9800 0x7f9ee00ac890 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0000c00 tx=0x7f9ed00046c0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.210+0000 7f9ee741f640 1 -- 172.21.15.84:0/4076018661 shutdown_connections 2023-10-16T10:08:06.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.210+0000 7f9ee741f640 1 --2- 172.21.15.84:0/4076018661 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f9ec406b0b0 0x7f9ec406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.210+0000 7f9ee741f640 1 --2- 172.21.15.84:0/4076018661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee0159d40 0x7f9ee00acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.210+0000 7f9ee741f640 1 --2- 172.21.15.84:0/4076018661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee00c9800 0x7f9ee00ac890 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.210+0000 7f9ee741f640 1 --2- 172.21.15.84:0/4076018661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ee006af40 0x7f9ee0153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.210+0000 7f9ee741f640 1 -- 172.21.15.84:0/4076018661 >> 172.21.15.84:0/4076018661 conn(0x7f9ee006bc10 msgr2=0x7f9ee00ccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:06.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.210+0000 7f9ee741f640 1 -- 172.21.15.84:0/4076018661 shutdown_connections 2023-10-16T10:08:06.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.211+0000 7f9ee741f640 1 -- 172.21.15.84:0/4076018661 wait complete. 2023-10-16T10:08:06.238 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T10:08:06.302 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:06.300+0000 7fc263432040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:08:06.309 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:06.307+0000 7facc7012040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:08:06.351 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:06.350+0000 7fc263432040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:08:06.422 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:06.420+0000 7facc7012040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:08:06.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.444+0000 7f3237fff640 1 Processor -- start 2023-10-16T10:08:06.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.445+0000 7f3237fff640 1 -- start start 2023-10-16T10:08:06.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.446+0000 7f3237fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f323815cd70 0x7f32380d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:06.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.446+0000 7f3237fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323815d150 con 0x7f32380cded0 2023-10-16T10:08:06.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.446+0000 7f3237fff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32380d1180 con 0x7f32380b8480 2023-10-16T10:08:06.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.446+0000 7f3237fff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32380d1460 con 0x7f323815cd70 2023-10-16T10:08:06.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.447+0000 7f32377fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f32380cded0 0x7f32380b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33120/0 (socket says 172.21.15.84:33120) 2023-10-16T10:08:06.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.447+0000 7f32377fe640 1 -- 172.21.15.84:0/1472685438 learned_addr learned my addr 172.21.15.84:0/1472685438 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:06.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.447+0000 7f3236ffd640 1 --2- 172.21.15.84:0/1472685438 >> v2:172.21.15.157:3300/0 conn(0x7f323815cd70 0x7f32380d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:06.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.447+0000 7f32367fc640 1 -- 172.21.15.84:0/1472685438 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1527454579 0 0) 0x7f323815d150 con 0x7f32380cded0 2023-10-16T10:08:06.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.447+0000 7f32367fc640 1 -- 172.21.15.84:0/1472685438 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3224003520 con 0x7f32380cded0 2023-10-16T10:08:06.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f32367fc640 1 -- 172.21.15.84:0/1472685438 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4105785751 0 0) 0x7f3224003520 con 0x7f32380cded0 2023-10-16T10:08:06.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f32367fc640 1 -- 172.21.15.84:0/1472685438 >> v2:172.21.15.157:3300/0 conn(0x7f323815cd70 msgr2=0x7f32380d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:06.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f32367fc640 1 --2- 172.21.15.84:0/1472685438 >> v2:172.21.15.157:3300/0 conn(0x7f323815cd70 0x7f32380d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f32367fc640 1 -- 172.21.15.84:0/1472685438 >> v1:172.21.15.157:6789/0 conn(0x7f32380b8480 legacy=0x7f32380cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:06.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f32367fc640 1 -- 172.21.15.84:0/1472685438 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32380d1740 con 0x7f32380cded0 2023-10-16T10:08:06.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f32367fc640 1 -- 172.21.15.84:0/1472685438 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f32300019f0 con 0x7f32380cded0 2023-10-16T10:08:06.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f32367fc640 1 -- 172.21.15.84:0/1472685438 >> v1:172.21.15.84:6789/0 conn(0x7f32380cded0 legacy=0x7f32380b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:06.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f32367fc640 1 --2- 172.21.15.84:0/1472685438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3224003970 0x7f3224003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:06.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f32367fc640 1 --2- 172.21.15.84:0/1472685438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32240047c0 0x7f3224004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:06.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f32367fc640 1 --2- 172.21.15.84:0/1472685438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3224005180 0x7f322400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:06.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f32367fc640 1 -- 172.21.15.84:0/1472685438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32380d1460 con 0x7f3224005180 2023-10-16T10:08:06.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f32367fc640 1 -- 172.21.15.84:0/1472685438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322400c020 con 0x7f3224003970 2023-10-16T10:08:06.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f32367fc640 1 -- 172.21.15.84:0/1472685438 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322400c1e0 con 0x7f32240047c0 2023-10-16T10:08:06.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f32377fe640 1 --2- 172.21.15.84:0/1472685438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3224005180 0x7f322400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:06.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f322ffff640 1 --2- 172.21.15.84:0/1472685438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32240047c0 0x7f3224004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:06.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f3236ffd640 1 --2- 172.21.15.84:0/1472685438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3224003970 0x7f3224003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:06.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f32377fe640 1 -- 172.21.15.84:0/1472685438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32240047c0 msgr2=0x7f3224004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:06.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f32377fe640 1 --2- 172.21.15.84:0/1472685438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32240047c0 0x7f3224004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f32377fe640 1 -- 172.21.15.84:0/1472685438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3224003970 msgr2=0x7f3224003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:06.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.448+0000 7f32377fe640 1 --2- 172.21.15.84:0/1472685438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3224003970 0x7f3224003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.449+0000 7f32377fe640 1 -- 172.21.15.84:0/1472685438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3224003520 con 0x7f3224005180 2023-10-16T10:08:06.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.449+0000 7f3236ffd640 1 --2- 172.21.15.84:0/1472685438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3224003970 0x7f3224003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:06.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.449+0000 7f32377fe640 1 -- 172.21.15.84:0/1472685438 reap_dead start 2023-10-16T10:08:06.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.450+0000 7f32377fe640 1 --2- 172.21.15.84:0/1472685438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3224005180 0x7f322400b980 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f323000a830 tx=0x7f32240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce800f2538c7b400 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:06.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.450+0000 7f32367fc640 1 -- 172.21.15.84:0/1472685438 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32300072d0 con 0x7f3224005180 2023-10-16T10:08:06.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.450+0000 7f32367fc640 1 -- 172.21.15.84:0/1472685438 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323002f070 con 0x7f3224005180 2023-10-16T10:08:06.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.450+0000 7f32367fc640 1 -- 172.21.15.84:0/1472685438 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3230007690 con 0x7f3224005180 2023-10-16T10:08:06.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.451+0000 7f3237fff640 1 -- 172.21.15.84:0/1472685438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3224005180 msgr2=0x7f322400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:06.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.451+0000 7f3237fff640 1 --2- 172.21.15.84:0/1472685438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3224005180 0x7f322400b980 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f323000a830 tx=0x7f32240057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.451+0000 7f3237fff640 1 -- 172.21.15.84:0/1472685438 shutdown_connections 2023-10-16T10:08:06.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.451+0000 7f3237fff640 1 --2- 172.21.15.84:0/1472685438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3224005180 0x7f322400b980 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.451+0000 7f3237fff640 1 -- 172.21.15.84:0/1472685438 >> 172.21.15.84:0/1472685438 conn(0x7f32380cf570 msgr2=0x7f32380cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:06.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.451+0000 7f3237fff640 1 -- 172.21.15.84:0/1472685438 shutdown_connections 2023-10-16T10:08:06.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.451+0000 7f3237fff640 1 -- 172.21.15.84:0/1472685438 wait complete. 2023-10-16T10:08:06.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.453+0000 7f3237fff640 1 Processor -- start 2023-10-16T10:08:06.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.453+0000 7f3237fff640 1 -- start start 2023-10-16T10:08:06.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.454+0000 7f3237fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3224005180 0x7f32380c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:06.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.454+0000 7f3237fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f323815a990 0x7f323815a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:06.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.454+0000 7f3237fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32380aa360 0x7f32380aa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:06.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.454+0000 7f3237fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32380aad30 con 0x7f3224005180 2023-10-16T10:08:06.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.454+0000 7f3237fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32380b80e0 con 0x7f323815a990 2023-10-16T10:08:06.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.454+0000 7f3237fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32380ca1a0 con 0x7f32380aa360 2023-10-16T10:08:06.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.454+0000 7f3236ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3224005180 0x7f32380c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:06.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.454+0000 7f3236ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3224005180 0x7f32380c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57408/0 (socket says 172.21.15.84:57408) 2023-10-16T10:08:06.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.454+0000 7f3236ffd640 1 -- 172.21.15.84:0/4126710011 learned_addr learned my addr 172.21.15.84:0/4126710011 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:06.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.454+0000 7f322ffff640 1 --2- 172.21.15.84:0/4126710011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f323815a990 0x7f323815a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:06.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.454+0000 7f3236ffd640 1 -- 172.21.15.84:0/4126710011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32380aa360 msgr2=0x7f32380aa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:06.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.454+0000 7f3236ffd640 1 --2- 172.21.15.84:0/4126710011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32380aa360 0x7f32380aa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.454+0000 7f3236ffd640 1 -- 172.21.15.84:0/4126710011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f323815a990 msgr2=0x7f323815a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:06.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.454+0000 7f3236ffd640 1 --2- 172.21.15.84:0/4126710011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f323815a990 0x7f323815a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.454+0000 7f3236ffd640 1 -- 172.21.15.84:0/4126710011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32380ca480 con 0x7f3224005180 2023-10-16T10:08:06.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.455+0000 7f3236ffd640 1 --2- 172.21.15.84:0/4126710011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3224005180 0x7f32380c9bc0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f32280108b0 tx=0x7f3228010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:06.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.455+0000 7f322f7fe640 1 -- 172.21.15.84:0/4126710011 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322800eb60 con 0x7f3224005180 2023-10-16T10:08:06.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.455+0000 7f322f7fe640 1 -- 172.21.15.84:0/4126710011 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322800ecc0 con 0x7f3224005180 2023-10-16T10:08:06.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.455+0000 7f322f7fe640 1 -- 172.21.15.84:0/4126710011 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322801c400 con 0x7f3224005180 2023-10-16T10:08:06.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.455+0000 7f3237fff640 1 -- 172.21.15.84:0/4126710011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32380ca760 con 0x7f3224005180 2023-10-16T10:08:06.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.455+0000 7f3237fff640 1 -- 172.21.15.84:0/4126710011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32380cad20 con 0x7f3224005180 2023-10-16T10:08:06.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.456+0000 7f3237fff640 1 -- 172.21.15.84:0/4126710011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323804e9b0 con 0x7f3224005180 2023-10-16T10:08:06.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.457+0000 7f322f7fe640 1 -- 172.21.15.84:0/4126710011 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 66) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f322801c560 con 0x7f3224005180 2023-10-16T10:08:06.466 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:06.459+0000 7fc263432040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:08:06.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.459+0000 7f322f7fe640 1 --2- 172.21.15.84:0/4126710011 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f321806b0b0 0x7f321806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:06.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.459+0000 7f322f7fe640 1 -- 172.21.15.84:0/4126710011 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f322808a120 con 0x7f3224005180 2023-10-16T10:08:06.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.459+0000 7f322ffff640 1 -- 172.21.15.84:0/4126710011 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f321806b0b0 msgr2=0x7f321806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2209954486 2023-10-16T10:08:06.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.459+0000 7f322ffff640 1 --2- 172.21.15.84:0/4126710011 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f321806b0b0 0x7f321806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:06.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.463+0000 7f322f7fe640 1 -- 172.21.15.84:0/4126710011 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3228054e50 con 0x7f3224005180 2023-10-16T10:08:06.476 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:06.475+0000 7facc7012040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:08:06.568 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:06.566+0000 7fc263432040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:08:06.589 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:06.587+0000 7facc7012040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:08:06.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.660+0000 7f322ffff640 1 -- 172.21.15.84:0/4126710011 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f321806b0b0 msgr2=0x7f321806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2209954486 2023-10-16T10:08:06.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.660+0000 7f322ffff640 1 --2- 172.21.15.84:0/4126710011 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f321806b0b0 0x7f321806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:08:06.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.669+0000 7f3237fff640 1 -- 172.21.15.84:0/4126710011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f32380b5d30 con 0x7f3224005180 2023-10-16T10:08:06.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.670+0000 7f322f7fe640 1 -- 172.21.15.84:0/4126710011 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v125) v1 ==== 80+0+469 (secure 0 0 0) 0x7f3228058470 con 0x7f3224005180 2023-10-16T10:08:06.672 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1284479959 2023-10-17T10:07:43.574635+0000 2023-10-16T10:08:06.672 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2038404786 2023-10-17T10:07:43.574635+0000 2023-10-16T10:08:06.672 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/644840609 2023-10-17T10:07:43.574635+0000 2023-10-16T10:08:06.672 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3794895754 2023-10-17T10:07:43.574635+0000 2023-10-16T10:08:06.673 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3269894694 2023-10-17T10:07:43.574635+0000 2023-10-16T10:08:06.673 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6833/4292025878 2023-10-17T10:07:43.574635+0000 2023-10-16T10:08:06.673 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6832/4292025878 2023-10-17T10:07:43.574635+0000 2023-10-16T10:08:06.673 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1934706185 2023-10-17T10:07:43.574635+0000 2023-10-16T10:08:06.674 INFO:teuthology.orchestra.run.smithi084.stderr:listed 8 entries 2023-10-16T10:08:06.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.672+0000 7f3237fff640 1 -- 172.21.15.84:0/4126710011 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f321806b0b0 msgr2=0x7f321806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:06.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.672+0000 7f3237fff640 1 --2- 172.21.15.84:0/4126710011 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f321806b0b0 0x7f321806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.672+0000 7f3237fff640 1 -- 172.21.15.84:0/4126710011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3224005180 msgr2=0x7f32380c9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:06.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.672+0000 7f3237fff640 1 --2- 172.21.15.84:0/4126710011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3224005180 0x7f32380c9bc0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f32280108b0 tx=0x7f3228010d70 comp rx=0 tx=0).stop 2023-10-16T10:08:06.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.672+0000 7f3237fff640 1 -- 172.21.15.84:0/4126710011 shutdown_connections 2023-10-16T10:08:06.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.672+0000 7f3237fff640 1 --2- 172.21.15.84:0/4126710011 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f321806b0b0 0x7f321806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.672+0000 7f3237fff640 1 --2- 172.21.15.84:0/4126710011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32380aa360 0x7f32380aa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.672+0000 7f3237fff640 1 --2- 172.21.15.84:0/4126710011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f323815a990 0x7f323815a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.672+0000 7f3237fff640 1 --2- 172.21.15.84:0/4126710011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3224005180 0x7f32380c9bc0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:06.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.672+0000 7f3237fff640 1 -- 172.21.15.84:0/4126710011 >> 172.21.15.84:0/4126710011 conn(0x7f32380cf570 msgr2=0x7f32380b6530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:06.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.673+0000 7f3237fff640 1 -- 172.21.15.84:0/4126710011 shutdown_connections 2023-10-16T10:08:06.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:06.673+0000 7f3237fff640 1 -- 172.21.15.84:0/4126710011 wait complete. 2023-10-16T10:08:06.691 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T10:08:06.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:06.691 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T10:08:06.702 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:06.700+0000 7facc7012040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:08:06.723 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:06.721+0000 7fc263432040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:08:06.774 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:06.772+0000 7fc263432040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:08:06.780 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T10:08:06.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:06.781 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T10:08:06.867 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:06.866+0000 7facc7012040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:08:06.871 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:08:06.872 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:08:06.894 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:06.893+0000 7fc263432040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:08:06.918 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:06.917+0000 7facc7012040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:08:06.928 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:08:06.929 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:08:06.929 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:08:06.983 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:08:06.983 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:08:06.984 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:08:07.039 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:08:07.039 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:08:07.048 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:07.040+0000 7facc7012040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:08:07.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.247+0000 7f6f7b59e640 1 Processor -- start 2023-10-16T10:08:07.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.247+0000 7f6f7b59e640 1 -- start start 2023-10-16T10:08:07.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.248+0000 7f6f7b59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6f7c0b4c80 0x7f6f7c167910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:07.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.248+0000 7f6f7b59e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f7c0d15a0 con 0x7f6f7c15cfc0 2023-10-16T10:08:07.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.248+0000 7f6f7b59e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f7c0d1880 con 0x7f6f7c15cbe0 2023-10-16T10:08:07.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.248+0000 7f6f7b59e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f7c0d1b60 con 0x7f6f7c0b4c80 2023-10-16T10:08:07.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.249+0000 7f6f79d9b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6f7c15cfc0 0x7f6f7c0b4570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33126/0 (socket says 172.21.15.84:33126) 2023-10-16T10:08:07.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.249+0000 7f6f79d9b640 1 -- 172.21.15.84:0/4110143212 learned_addr learned my addr 172.21.15.84:0/4110143212 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:07.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.249+0000 7f6f7959a640 1 -- 172.21.15.84:0/4110143212 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255065149 0 0) 0x7f6f7c0d15a0 con 0x7f6f7c15cfc0 2023-10-16T10:08:07.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.249+0000 7f6f7959a640 1 -- 172.21.15.84:0/4110143212 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f64003520 con 0x7f6f7c15cfc0 2023-10-16T10:08:07.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.249+0000 7f6f7ad9d640 1 --2- 172.21.15.84:0/4110143212 >> v2:172.21.15.157:3300/0 conn(0x7f6f7c0b4c80 0x7f6f7c167910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:07.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.249+0000 7f6f7959a640 1 -- 172.21.15.84:0/4110143212 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1305668897 0 0) 0x7f6f64003520 con 0x7f6f7c15cfc0 2023-10-16T10:08:07.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.249+0000 7f6f7959a640 1 -- 172.21.15.84:0/4110143212 >> v2:172.21.15.157:3300/0 conn(0x7f6f7c0b4c80 msgr2=0x7f6f7c167910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:07.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.249+0000 7f6f7959a640 1 --2- 172.21.15.84:0/4110143212 >> v2:172.21.15.157:3300/0 conn(0x7f6f7c0b4c80 0x7f6f7c167910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.249+0000 7f6f7959a640 1 -- 172.21.15.84:0/4110143212 >> v1:172.21.15.157:6789/0 conn(0x7f6f7c15cbe0 legacy=0x7f6f7c0d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:07.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.249+0000 7f6f7959a640 1 -- 172.21.15.84:0/4110143212 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c0d1e40 con 0x7f6f7c15cfc0 2023-10-16T10:08:07.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.249+0000 7f6f7959a640 1 -- 172.21.15.84:0/4110143212 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f6f70001670 con 0x7f6f7c15cfc0 2023-10-16T10:08:07.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.250+0000 7f6f7959a640 1 -- 172.21.15.84:0/4110143212 >> v1:172.21.15.84:6789/0 conn(0x7f6f7c15cfc0 legacy=0x7f6f7c0b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:07.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.250+0000 7f6f7959a640 1 --2- 172.21.15.84:0/4110143212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f64003970 0x7f6f64003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:07.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.250+0000 7f6f7959a640 1 --2- 172.21.15.84:0/4110143212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640047c0 0x7f6f64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:07.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.250+0000 7f6f7959a640 1 --2- 172.21.15.84:0/4110143212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f64005180 0x7f6f6400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:07.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.250+0000 7f6f7959a640 1 -- 172.21.15.84:0/4110143212 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c0d1b60 con 0x7f6f64003970 2023-10-16T10:08:07.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.250+0000 7f6f7959a640 1 -- 172.21.15.84:0/4110143212 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6400c0d0 con 0x7f6f64005180 2023-10-16T10:08:07.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.250+0000 7f6f7959a640 1 -- 172.21.15.84:0/4110143212 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6400c290 con 0x7f6f640047c0 2023-10-16T10:08:07.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.250+0000 7f6f7a59c640 1 --2- 172.21.15.84:0/4110143212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f64003970 0x7f6f64003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:07.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.250+0000 7f6f7a59c640 1 -- 172.21.15.84:0/4110143212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640047c0 msgr2=0x7f6f64004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:07.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.250+0000 7f6f7a59c640 1 --2- 172.21.15.84:0/4110143212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640047c0 0x7f6f64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.250+0000 7f6f7a59c640 1 -- 172.21.15.84:0/4110143212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f64005180 msgr2=0x7f6f6400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:07.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.250+0000 7f6f7a59c640 1 --2- 172.21.15.84:0/4110143212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f64005180 0x7f6f6400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.250+0000 7f6f7a59c640 1 -- 172.21.15.84:0/4110143212 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f64003520 con 0x7f6f64003970 2023-10-16T10:08:07.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.250+0000 7f6f7ad9d640 1 -- 172.21.15.84:0/4110143212 reap_dead start 2023-10-16T10:08:07.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.251+0000 7f6f7a59c640 1 --2- 172.21.15.84:0/4110143212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f64003970 0x7f6f64003d90 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c001750 tx=0x7f6f640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546c0ae1b0ce2886 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:07.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.251+0000 7f6f7959a640 1 -- 172.21.15.84:0/4110143212 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c0052d0 con 0x7f6f64003970 2023-10-16T10:08:07.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.251+0000 7f6f7959a640 1 -- 172.21.15.84:0/4110143212 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c02e050 con 0x7f6f64003970 2023-10-16T10:08:07.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.251+0000 7f6f7959a640 1 -- 172.21.15.84:0/4110143212 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c003830 con 0x7f6f64003970 2023-10-16T10:08:07.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.252+0000 7f6f7b59e640 1 -- 172.21.15.84:0/4110143212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f64003970 msgr2=0x7f6f64003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:07.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.252+0000 7f6f7b59e640 1 --2- 172.21.15.84:0/4110143212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f64003970 0x7f6f64003d90 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c001750 tx=0x7f6f640057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.252+0000 7f6f7b59e640 1 -- 172.21.15.84:0/4110143212 shutdown_connections 2023-10-16T10:08:07.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.252+0000 7f6f7b59e640 1 --2- 172.21.15.84:0/4110143212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f64003970 0x7f6f64003d90 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.252+0000 7f6f7b59e640 1 -- 172.21.15.84:0/4110143212 >> 172.21.15.84:0/4110143212 conn(0x7f6f7c0cf570 msgr2=0x7f6f7c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:07.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.252+0000 7f6f7b59e640 1 -- 172.21.15.84:0/4110143212 shutdown_connections 2023-10-16T10:08:07.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.252+0000 7f6f7b59e640 1 -- 172.21.15.84:0/4110143212 wait complete. 2023-10-16T10:08:07.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.255+0000 7f6f7b59e640 1 Processor -- start 2023-10-16T10:08:07.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.255+0000 7f6f7b59e640 1 -- start start 2023-10-16T10:08:07.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.257+0000 7f6f7b59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f64003970 0x7f6f7c1f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:07.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.257+0000 7f6f7b59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f7c1ef860 0x7f6f7c1f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:07.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.257+0000 7f6f7b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f7c1f5ec0 0x7f6f7c154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:07.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.257+0000 7f6f7b59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c1f6340 con 0x7f6f7c1f5ec0 2023-10-16T10:08:07.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.257+0000 7f6f7b59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c158f90 con 0x7f6f64003970 2023-10-16T10:08:07.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.257+0000 7f6f7b59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c1f3350 con 0x7f6f7c1ef860 2023-10-16T10:08:07.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.257+0000 7f6f7ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f7c1f5ec0 0x7f6f7c154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:07.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.257+0000 7f6f7ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f7c1f5ec0 0x7f6f7c154fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57412/0 (socket says 172.21.15.84:57412) 2023-10-16T10:08:07.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.257+0000 7f6f7ad9d640 1 -- 172.21.15.84:0/1367751509 learned_addr learned my addr 172.21.15.84:0/1367751509 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:07.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.257+0000 7f6f79d9b640 1 --2- 172.21.15.84:0/1367751509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f7c1ef860 0x7f6f7c1f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:07.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.257+0000 7f6f7a59c640 1 --2- 172.21.15.84:0/1367751509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f64003970 0x7f6f7c1f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:07.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.258+0000 7f6f7ad9d640 1 -- 172.21.15.84:0/1367751509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f7c1ef860 msgr2=0x7f6f7c1f5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:07.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.258+0000 7f6f7ad9d640 1 --2- 172.21.15.84:0/1367751509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f7c1ef860 0x7f6f7c1f5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.258+0000 7f6f7ad9d640 1 -- 172.21.15.84:0/1367751509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f64003970 msgr2=0x7f6f7c1f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:07.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.258+0000 7f6f7ad9d640 1 --2- 172.21.15.84:0/1367751509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f64003970 0x7f6f7c1f2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.258+0000 7f6f7ad9d640 1 -- 172.21.15.84:0/1367751509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c158380 con 0x7f6f7c1f5ec0 2023-10-16T10:08:07.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.258+0000 7f6f79d9b640 1 --2- 172.21.15.84:0/1367751509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f7c1ef860 0x7f6f7c1f5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:07.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.258+0000 7f6f7a59c640 1 --2- 172.21.15.84:0/1367751509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f64003970 0x7f6f7c1f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:07.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.258+0000 7f6f7ad9d640 1 --2- 172.21.15.84:0/1367751509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f7c1f5ec0 0x7f6f7c154fb0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0d3b20 tx=0x7f6f74007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:07.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.258+0000 7f6f6affd640 1 -- 172.21.15.84:0/1367751509 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7400ec20 con 0x7f6f7c1f5ec0 2023-10-16T10:08:07.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.258+0000 7f6f6affd640 1 -- 172.21.15.84:0/1367751509 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7400eda0 con 0x7f6f7c1f5ec0 2023-10-16T10:08:07.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.258+0000 7f6f6affd640 1 -- 172.21.15.84:0/1367751509 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f74014660 con 0x7f6f7c1f5ec0 2023-10-16T10:08:07.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.258+0000 7f6f7b59e640 1 -- 172.21.15.84:0/1367751509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f7c158660 con 0x7f6f7c1f5ec0 2023-10-16T10:08:07.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.258+0000 7f6f7b59e640 1 -- 172.21.15.84:0/1367751509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f7c158c20 con 0x7f6f7c1f5ec0 2023-10-16T10:08:07.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.259+0000 7f6f7b59e640 1 -- 172.21.15.84:0/1367751509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7c158db0 con 0x7f6f7c1f5ec0 2023-10-16T10:08:07.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.260+0000 7f6f6affd640 1 -- 172.21.15.84:0/1367751509 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 66) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f6f740147c0 con 0x7f6f7c1f5ec0 2023-10-16T10:08:07.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.262+0000 7f6f6affd640 1 --2- 172.21.15.84:0/1367751509 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f6f5c06b0e0 0x7f6f5c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:07.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.262+0000 7f6f7a59c640 1 -- 172.21.15.84:0/1367751509 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f6f5c06b0e0 msgr2=0x7f6f5c06d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2209954486 2023-10-16T10:08:07.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.262+0000 7f6f6affd640 1 -- 172.21.15.84:0/1367751509 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f6f74089ff0 con 0x7f6f7c1f5ec0 2023-10-16T10:08:07.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.262+0000 7f6f7a59c640 1 --2- 172.21.15.84:0/1367751509 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f6f5c06b0e0 0x7f6f5c06d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:07.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.266+0000 7f6f6affd640 1 -- 172.21.15.84:0/1367751509 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6f740bcfe0 con 0x7f6f7c1f5ec0 2023-10-16T10:08:07.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.463+0000 7f6f7a59c640 1 -- 172.21.15.84:0/1367751509 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f6f5c06b0e0 msgr2=0x7f6f5c06d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2209954486 2023-10-16T10:08:07.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.463+0000 7f6f7a59c640 1 --2- 172.21.15.84:0/1367751509 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f6f5c06b0e0 0x7f6f5c06d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:08:07.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.500+0000 7f6f7b59e640 1 -- 172.21.15.84:0/1367751509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f7c04e9b0 con 0x7f6f7c1f5ec0 2023-10-16T10:08:07.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.501+0000 7f6f6affd640 1 -- 172.21.15.84:0/1367751509 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f6f74054c50 con 0x7f6f7c1f5ec0 2023-10-16T10:08:07.503 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:07.503 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2012273152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2012273152},{"type":"v1","addr":"172.21.15.157:6835","nonce":2012273152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/435606514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":435606514},{"type":"v1","addr":"172.21.15.157:6833","nonce":435606514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2286048798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2286048798},{"type":"v1","addr":"172.21.15.84:6836","nonce":2286048798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:07:28.798746+0000","modified":"2023-10-16T10:07:31.857629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2859088731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2859088731},{"type":"v1","addr":"172.21.15.84:6837","nonce":2859088731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-16T10:08:07.503 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 62 2023-10-16T10:08:07.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.503+0000 7f6f7b59e640 1 -- 172.21.15.84:0/1367751509 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f6f5c06b0e0 msgr2=0x7f6f5c06d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:07.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.503+0000 7f6f7b59e640 1 --2- 172.21.15.84:0/1367751509 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f6f5c06b0e0 0x7f6f5c06d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.503+0000 7f6f7b59e640 1 -- 172.21.15.84:0/1367751509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f7c1f5ec0 msgr2=0x7f6f7c154fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:07.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.503+0000 7f6f7b59e640 1 --2- 172.21.15.84:0/1367751509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f7c1f5ec0 0x7f6f7c154fb0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0d3b20 tx=0x7f6f74007f20 comp rx=0 tx=0).stop 2023-10-16T10:08:07.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.504+0000 7f6f7b59e640 1 -- 172.21.15.84:0/1367751509 shutdown_connections 2023-10-16T10:08:07.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.504+0000 7f6f7b59e640 1 --2- 172.21.15.84:0/1367751509 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f6f5c06b0e0 0x7f6f5c06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.504+0000 7f6f7b59e640 1 --2- 172.21.15.84:0/1367751509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f7c1f5ec0 0x7f6f7c154fb0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.504+0000 7f6f7b59e640 1 --2- 172.21.15.84:0/1367751509 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f7c1ef860 0x7f6f7c1f5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.504+0000 7f6f7b59e640 1 --2- 172.21.15.84:0/1367751509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f64003970 0x7f6f7c1f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.504+0000 7f6f7b59e640 1 -- 172.21.15.84:0/1367751509 >> 172.21.15.84:0/1367751509 conn(0x7f6f7c0cf570 msgr2=0x7f6f7c1597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:07.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.504+0000 7f6f7b59e640 1 -- 172.21.15.84:0/1367751509 shutdown_connections 2023-10-16T10:08:07.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.504+0000 7f6f7b59e640 1 -- 172.21.15.84:0/1367751509 wait complete. 2023-10-16T10:08:07.524 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:08:07.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.720+0000 7f5a41634640 1 Processor -- start 2023-10-16T10:08:07.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.720+0000 7f5a41634640 1 -- start start 2023-10-16T10:08:07.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.722+0000 7f5a41634640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a3c06c7d0 0x7f5a3c0b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:07.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.722+0000 7f5a41634640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a3c0cd630 con 0x7f5a3c06c3f0 2023-10-16T10:08:07.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.722+0000 7f5a41634640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a3c0ce1b0 con 0x7f5a3c0b4c70 2023-10-16T10:08:07.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.722+0000 7f5a41634640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c15e7d0 con 0x7f5a3c06c7d0 2023-10-16T10:08:07.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.722+0000 7f5a3affd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f5a3c06c3f0 0x7f5a3c15e0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33132/0 (socket says 172.21.15.84:33132) 2023-10-16T10:08:07.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.722+0000 7f5a3affd640 1 -- 172.21.15.84:0/1765900817 learned_addr learned my addr 172.21.15.84:0/1765900817 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:07.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.723+0000 7f5a3a7fc640 1 --2- 172.21.15.84:0/1765900817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a3c06c7d0 0x7f5a3c0b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:07.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.723+0000 7f5a39ffb640 1 -- 172.21.15.84:0/1765900817 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2533179219 0 0) 0x7f5a3c0cd630 con 0x7f5a3c06c3f0 2023-10-16T10:08:07.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.723+0000 7f5a39ffb640 1 -- 172.21.15.84:0/1765900817 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a24003520 con 0x7f5a3c06c3f0 2023-10-16T10:08:07.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.723+0000 7f5a39ffb640 1 -- 172.21.15.84:0/1765900817 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3909852983 0 0) 0x7f5a24003520 con 0x7f5a3c06c3f0 2023-10-16T10:08:07.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.723+0000 7f5a39ffb640 1 -- 172.21.15.84:0/1765900817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a3c06c7d0 msgr2=0x7f5a3c0b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:07.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.723+0000 7f5a39ffb640 1 --2- 172.21.15.84:0/1765900817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a3c06c7d0 0x7f5a3c0b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.723+0000 7f5a39ffb640 1 -- 172.21.15.84:0/1765900817 >> v1:172.21.15.157:6789/0 conn(0x7f5a3c0b4c70 legacy=0x7f5a3c0b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:07.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.723+0000 7f5a39ffb640 1 -- 172.21.15.84:0/1765900817 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c0d2170 con 0x7f5a3c06c3f0 2023-10-16T10:08:07.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.723+0000 7f5a39ffb640 1 -- 172.21.15.84:0/1765900817 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f5a2c001670 con 0x7f5a3c06c3f0 2023-10-16T10:08:07.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.723+0000 7f5a39ffb640 1 -- 172.21.15.84:0/1765900817 >> v1:172.21.15.84:6789/0 conn(0x7f5a3c06c3f0 legacy=0x7f5a3c15e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:07.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.723+0000 7f5a39ffb640 1 --2- 172.21.15.84:0/1765900817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a240039a0 0x7f5a24003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:07.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.723+0000 7f5a39ffb640 1 --2- 172.21.15.84:0/1765900817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a3c06c7d0 0x7f5a240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:07.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.723+0000 7f5a39ffb640 1 --2- 172.21.15.84:0/1765900817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a24004d00 0x7f5a24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:07.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.723+0000 7f5a39ffb640 1 -- 172.21.15.84:0/1765900817 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2400ba30 con 0x7f5a24004d00 2023-10-16T10:08:07.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.723+0000 7f5a39ffb640 1 -- 172.21.15.84:0/1765900817 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2400bba0 con 0x7f5a240039a0 2023-10-16T10:08:07.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.723+0000 7f5a39ffb640 1 -- 172.21.15.84:0/1765900817 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2400bd60 con 0x7f5a3c06c7d0 2023-10-16T10:08:07.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.724+0000 7f5a3b7fe640 1 --2- 172.21.15.84:0/1765900817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a24004d00 0x7f5a24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:07.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.724+0000 7f5a3affd640 1 --2- 172.21.15.84:0/1765900817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a240039a0 0x7f5a24003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:07.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.724+0000 7f5a3a7fc640 1 --2- 172.21.15.84:0/1765900817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a3c06c7d0 0x7f5a240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:07.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.724+0000 7f5a3b7fe640 1 -- 172.21.15.84:0/1765900817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a3c06c7d0 msgr2=0x7f5a240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:07.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.724+0000 7f5a3b7fe640 1 --2- 172.21.15.84:0/1765900817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a3c06c7d0 0x7f5a240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.724+0000 7f5a3b7fe640 1 -- 172.21.15.84:0/1765900817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a240039a0 msgr2=0x7f5a24003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:07.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.724+0000 7f5a3b7fe640 1 --2- 172.21.15.84:0/1765900817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a240039a0 0x7f5a24003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.724+0000 7f5a3b7fe640 1 -- 172.21.15.84:0/1765900817 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a24003520 con 0x7f5a24004d00 2023-10-16T10:08:07.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.724+0000 7f5a3affd640 1 --2- 172.21.15.84:0/1765900817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a240039a0 0x7f5a24003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:08:07.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.725+0000 7f5a3b7fe640 1 --2- 172.21.15.84:0/1765900817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a24004d00 0x7f5a24005180 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f5a34001e20 tx=0x7f5a240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f78864ec593aef5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:07.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.726+0000 7f5a39ffb640 1 -- 172.21.15.84:0/1765900817 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a34003b70 con 0x7f5a24004d00 2023-10-16T10:08:07.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.726+0000 7f5a39ffb640 1 -- 172.21.15.84:0/1765900817 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a34003cd0 con 0x7f5a24004d00 2023-10-16T10:08:07.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.726+0000 7f5a39ffb640 1 -- 172.21.15.84:0/1765900817 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a34005310 con 0x7f5a24004d00 2023-10-16T10:08:07.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.726+0000 7f5a41634640 1 -- 172.21.15.84:0/1765900817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a24004d00 msgr2=0x7f5a24005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:07.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.726+0000 7f5a41634640 1 --2- 172.21.15.84:0/1765900817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a24004d00 0x7f5a24005180 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f5a34001e20 tx=0x7f5a240057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.726+0000 7f5a3b7fe640 1 -- 172.21.15.84:0/1765900817 reap_dead start 2023-10-16T10:08:07.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.726+0000 7f5a41634640 1 -- 172.21.15.84:0/1765900817 shutdown_connections 2023-10-16T10:08:07.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.726+0000 7f5a41634640 1 -- 172.21.15.84:0/1765900817 >> 172.21.15.84:0/1765900817 conn(0x7f5a3c0d2cb0 msgr2=0x7f5a3c0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:07.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.726+0000 7f5a41634640 1 -- 172.21.15.84:0/1765900817 shutdown_connections 2023-10-16T10:08:07.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.727+0000 7f5a41634640 1 -- 172.21.15.84:0/1765900817 wait complete. 2023-10-16T10:08:07.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.728+0000 7f5a41634640 1 Processor -- start 2023-10-16T10:08:07.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.728+0000 7f5a41634640 1 -- start start 2023-10-16T10:08:07.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.729+0000 7f5a41634640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a24004d00 0x7f5a3c1f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:07.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.729+0000 7f5a41634640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a3c0b4520 0x7f5a3c1f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:07.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.729+0000 7f5a41634640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a3c1f6150 0x7f5a3c0ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:07.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.729+0000 7f5a41634640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c1f65e0 con 0x7f5a3c1f6150 2023-10-16T10:08:07.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.729+0000 7f5a41634640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c0cd0c0 con 0x7f5a3c0b4520 2023-10-16T10:08:07.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.729+0000 7f5a41634640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c0cb350 con 0x7f5a24004d00 2023-10-16T10:08:07.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.730+0000 7f5a3b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a3c1f6150 0x7f5a3c0ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:07.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.730+0000 7f5a3b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a3c1f6150 0x7f5a3c0ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57442/0 (socket says 172.21.15.84:57442) 2023-10-16T10:08:07.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.730+0000 7f5a3b7fe640 1 -- 172.21.15.84:0/3182783091 learned_addr learned my addr 172.21.15.84:0/3182783091 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:07.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.730+0000 7f5a3affd640 1 --2- 172.21.15.84:0/3182783091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a24004d00 0x7f5a3c1f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:07.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.730+0000 7f5a3a7fc640 1 --2- 172.21.15.84:0/3182783091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a3c0b4520 0x7f5a3c1f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:07.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.730+0000 7f5a3b7fe640 1 -- 172.21.15.84:0/3182783091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a24004d00 msgr2=0x7f5a3c1f2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:07.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.730+0000 7f5a3b7fe640 1 --2- 172.21.15.84:0/3182783091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a24004d00 0x7f5a3c1f2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.730+0000 7f5a3b7fe640 1 -- 172.21.15.84:0/3182783091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a3c0b4520 msgr2=0x7f5a3c1f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:07.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.730+0000 7f5a3b7fe640 1 --2- 172.21.15.84:0/3182783091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a3c0b4520 0x7f5a3c1f5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.730+0000 7f5a3b7fe640 1 -- 172.21.15.84:0/3182783091 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c158fd0 con 0x7f5a3c1f6150 2023-10-16T10:08:07.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.731+0000 7f5a3b7fe640 1 --2- 172.21.15.84:0/3182783091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a3c1f6150 0x7f5a3c0ca6b0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f5a34005040 tx=0x7f5a340014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:07.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.731+0000 7f5a237fe640 1 -- 172.21.15.84:0/3182783091 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3403e070 con 0x7f5a3c1f6150 2023-10-16T10:08:07.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.731+0000 7f5a237fe640 1 -- 172.21.15.84:0/3182783091 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a34003e40 con 0x7f5a3c1f6150 2023-10-16T10:08:07.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.731+0000 7f5a237fe640 1 -- 172.21.15.84:0/3182783091 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a34006d50 con 0x7f5a3c1f6150 2023-10-16T10:08:07.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.731+0000 7f5a41634640 1 -- 172.21.15.84:0/3182783091 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3c1592b0 con 0x7f5a3c1f6150 2023-10-16T10:08:07.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.731+0000 7f5a41634640 1 -- 172.21.15.84:0/3182783091 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3c159870 con 0x7f5a3c1f6150 2023-10-16T10:08:07.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.732+0000 7f5a41634640 1 -- 172.21.15.84:0/3182783091 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3c159a00 con 0x7f5a3c1f6150 2023-10-16T10:08:07.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.733+0000 7f5a237fe640 1 -- 172.21.15.84:0/3182783091 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 66) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f5a34007030 con 0x7f5a3c1f6150 2023-10-16T10:08:07.743 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:07.739+0000 7fc263432040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:08:07.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.734+0000 7f5a237fe640 1 --2- 172.21.15.84:0/3182783091 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f5a2806b0e0 0x7f5a2806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:07.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.735+0000 7f5a3affd640 1 -- 172.21.15.84:0/3182783091 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f5a2806b0e0 msgr2=0x7f5a2806d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2209954486 2023-10-16T10:08:07.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.735+0000 7f5a237fe640 1 -- 172.21.15.84:0/3182783091 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f5a340ab260 con 0x7f5a3c1f6150 2023-10-16T10:08:07.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.735+0000 7f5a3affd640 1 --2- 172.21.15.84:0/3182783091 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f5a2806b0e0 0x7f5a2806d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:07.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.739+0000 7f5a237fe640 1 -- 172.21.15.84:0/3182783091 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5a34075dd0 con 0x7f5a3c1f6150 2023-10-16T10:08:07.904 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:07.903+0000 7facc7012040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:08:07.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.933+0000 7f5a41634640 1 -- 172.21.15.84:0/3182783091 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a3c04e160 con 0x7f5a3c1f6150 2023-10-16T10:08:07.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.935+0000 7f5a237fe640 1 -- 172.21.15.84:0/3182783091 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16995 (secure 0 0 0) 0x7f5a34048090 con 0x7f5a3c1f6150 2023-10-16T10:08:07.937 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:07.937 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":125,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:07:43.574763+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:07:26.607199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-16T10:07:27.832706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1284479959":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/2038404786":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/644840609":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/3794895754":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/3269894694":"2023-10-17T10:07:43.574635+0000","172.21.15.84:6833/4292025878":"2023-10-17T10:07:43.574635+0000","172.21.15.84:6832/4292025878":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/1934706185":"2023-10-17T10:07:43.574635+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:08:07.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.936+0000 7f5a3affd640 1 -- 172.21.15.84:0/3182783091 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f5a2806b0e0 msgr2=0x7f5a2806d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2209954486 2023-10-16T10:08:07.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.936+0000 7f5a3affd640 1 --2- 172.21.15.84:0/3182783091 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f5a2806b0e0 0x7f5a2806d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:08:07.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.937+0000 7f5a41634640 1 -- 172.21.15.84:0/3182783091 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f5a2806b0e0 msgr2=0x7f5a2806d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:07.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.937+0000 7f5a41634640 1 --2- 172.21.15.84:0/3182783091 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f5a2806b0e0 0x7f5a2806d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.938+0000 7f5a41634640 1 -- 172.21.15.84:0/3182783091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a3c1f6150 msgr2=0x7f5a3c0ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:07.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.938+0000 7f5a41634640 1 --2- 172.21.15.84:0/3182783091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a3c1f6150 0x7f5a3c0ca6b0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f5a34005040 tx=0x7f5a340014a0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.938+0000 7f5a41634640 1 -- 172.21.15.84:0/3182783091 shutdown_connections 2023-10-16T10:08:07.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.938+0000 7f5a41634640 1 --2- 172.21.15.84:0/3182783091 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f5a2806b0e0 0x7f5a2806d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.938+0000 7f5a41634640 1 --2- 172.21.15.84:0/3182783091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a3c1f6150 0x7f5a3c0ca6b0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.938+0000 7f5a41634640 1 --2- 172.21.15.84:0/3182783091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a3c0b4520 0x7f5a3c1f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.938+0000 7f5a41634640 1 --2- 172.21.15.84:0/3182783091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a24004d00 0x7f5a3c1f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:07.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.938+0000 7f5a41634640 1 -- 172.21.15.84:0/3182783091 >> 172.21.15.84:0/3182783091 conn(0x7f5a3c0d2cb0 msgr2=0x7f5a3c15e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:07.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.938+0000 7f5a41634640 1 -- 172.21.15.84:0/3182783091 shutdown_connections 2023-10-16T10:08:07.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:07.938+0000 7f5a41634640 1 -- 172.21.15.84:0/3182783091 wait complete. 2023-10-16T10:08:07.959 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:07:26.607199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:07:27.832706+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:08:07.960 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:08:08.063 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:08.061+0000 7fc263432040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:08:08.120 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:08.119+0000 7fc263432040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:08:08.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.158+0000 7f2dfbffb640 1 Processor -- start 2023-10-16T10:08:08.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.158+0000 7f2dfbffb640 1 -- start start 2023-10-16T10:08:08.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.160+0000 7f2dfbffb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df40c9800 0x7f2df415d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:08.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.160+0000 7f2dfbffb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2df406af40 0x7f2df415dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:08.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.160+0000 7f2dfbffb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df415a120 con 0x7f2df4159d40 2023-10-16T10:08:08.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.160+0000 7f2dfbffb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df415e3b0 con 0x7f2df406af40 2023-10-16T10:08:08.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.160+0000 7f2dfbffb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df415e690 con 0x7f2df40c9800 2023-10-16T10:08:08.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.160+0000 7f2df9d97640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2df4159d40 0x7f2df4071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33134/0 (socket says 172.21.15.84:33134) 2023-10-16T10:08:08.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.160+0000 7f2df9d97640 1 -- 172.21.15.84:0/2961615828 learned_addr learned my addr 172.21.15.84:0/2961615828 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:08.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.160+0000 7f2df8d95640 1 -- 172.21.15.84:0/2961615828 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736654206 0 0) 0x7f2df415a120 con 0x7f2df4159d40 2023-10-16T10:08:08.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.160+0000 7f2df8d95640 1 -- 172.21.15.84:0/2961615828 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2de0005730 con 0x7f2df4159d40 2023-10-16T10:08:08.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.160+0000 7f2dfa598640 1 --2- 172.21.15.84:0/2961615828 >> v2:172.21.15.157:3300/0 conn(0x7f2df406af40 0x7f2df415dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:08.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.160+0000 7f2df9596640 1 --2- 172.21.15.84:0/2961615828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df40c9800 0x7f2df415d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:08.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.161+0000 7f2df8d95640 1 -- 172.21.15.84:0/2961615828 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2559967005 0 0) 0x7f2de0005730 con 0x7f2df4159d40 2023-10-16T10:08:08.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.161+0000 7f2df8d95640 1 -- 172.21.15.84:0/2961615828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df40c9800 msgr2=0x7f2df415d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:08.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.161+0000 7f2df8d95640 1 --2- 172.21.15.84:0/2961615828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df40c9800 0x7f2df415d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.161+0000 7f2df8d95640 1 -- 172.21.15.84:0/2961615828 >> v2:172.21.15.157:3300/0 conn(0x7f2df406af40 msgr2=0x7f2df415dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:08.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.161+0000 7f2df8d95640 1 --2- 172.21.15.84:0/2961615828 >> v2:172.21.15.157:3300/0 conn(0x7f2df406af40 0x7f2df415dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.161+0000 7f2df8d95640 1 -- 172.21.15.84:0/2961615828 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df4070f60 con 0x7f2df4159d40 2023-10-16T10:08:08.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.161+0000 7f2df8d95640 1 -- 172.21.15.84:0/2961615828 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f2df0001670 con 0x7f2df4159d40 2023-10-16T10:08:08.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.161+0000 7f2df8d95640 1 -- 172.21.15.84:0/2961615828 >> v1:172.21.15.84:6789/0 conn(0x7f2df4159d40 legacy=0x7f2df4071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:08.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.161+0000 7f2df8d95640 1 --2- 172.21.15.84:0/2961615828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de0005b80 0x7f2de0005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:08.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.161+0000 7f2df8d95640 1 --2- 172.21.15.84:0/2961615828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df40c9800 0x7f2de00069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:08.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.161+0000 7f2df8d95640 1 --2- 172.21.15.84:0/2961615828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de0006f10 0x7f2de0007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:08.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.161+0000 7f2df8d95640 1 -- 172.21.15.84:0/2961615828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df415e3b0 con 0x7f2de0006f10 2023-10-16T10:08:08.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.161+0000 7f2df8d95640 1 -- 172.21.15.84:0/2961615828 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df415e690 con 0x7f2de0005b80 2023-10-16T10:08:08.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.161+0000 7f2df8d95640 1 -- 172.21.15.84:0/2961615828 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de000dcc0 con 0x7f2df40c9800 2023-10-16T10:08:08.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.161+0000 7f2df9d97640 1 --2- 172.21.15.84:0/2961615828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de0005b80 0x7f2de0005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:08.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.161+0000 7f2dfa598640 1 --2- 172.21.15.84:0/2961615828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de0006f10 0x7f2de0007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:08.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.161+0000 7f2df9596640 1 --2- 172.21.15.84:0/2961615828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df40c9800 0x7f2de00069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:08.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.162+0000 7f2dfa598640 1 -- 172.21.15.84:0/2961615828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df40c9800 msgr2=0x7f2de00069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:08.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.162+0000 7f2dfa598640 1 --2- 172.21.15.84:0/2961615828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df40c9800 0x7f2de00069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.162+0000 7f2dfa598640 1 -- 172.21.15.84:0/2961615828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de0005b80 msgr2=0x7f2de0005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:08.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.162+0000 7f2dfa598640 1 --2- 172.21.15.84:0/2961615828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de0005b80 0x7f2de0005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.162+0000 7f2dfa598640 1 -- 172.21.15.84:0/2961615828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de0005730 con 0x7f2de0006f10 2023-10-16T10:08:08.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.163+0000 7f2dfa598640 1 --2- 172.21.15.84:0/2961615828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de0006f10 0x7f2de0007390 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f2dec007360 tx=0x7f2de0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9bd07ca6ba3abbf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:08.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.164+0000 7f2df8d95640 1 -- 172.21.15.84:0/2961615828 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec02ccf0 con 0x7f2de0006f10 2023-10-16T10:08:08.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.164+0000 7f2df8d95640 1 -- 172.21.15.84:0/2961615828 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dec02ce50 con 0x7f2de0006f10 2023-10-16T10:08:08.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.164+0000 7f2df8d95640 1 -- 172.21.15.84:0/2961615828 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec034660 con 0x7f2de0006f10 2023-10-16T10:08:08.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.164+0000 7f2dfbffb640 1 -- 172.21.15.84:0/2961615828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de0006f10 msgr2=0x7f2de0007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:08.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.164+0000 7f2dfbffb640 1 --2- 172.21.15.84:0/2961615828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de0006f10 0x7f2de0007390 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f2dec007360 tx=0x7f2de0007a00 comp rx=0 tx=0).stop 2023-10-16T10:08:08.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.164+0000 7f2dfa598640 1 -- 172.21.15.84:0/2961615828 reap_dead start 2023-10-16T10:08:08.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.165+0000 7f2dfbffb640 1 -- 172.21.15.84:0/2961615828 shutdown_connections 2023-10-16T10:08:08.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.165+0000 7f2dfbffb640 1 -- 172.21.15.84:0/2961615828 >> 172.21.15.84:0/2961615828 conn(0x7f2df406bc10 msgr2=0x7f2df406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:08.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.165+0000 7f2dfbffb640 1 -- 172.21.15.84:0/2961615828 shutdown_connections 2023-10-16T10:08:08.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.165+0000 7f2dfbffb640 1 -- 172.21.15.84:0/2961615828 wait complete. 2023-10-16T10:08:08.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.166+0000 7f2dfbffb640 1 Processor -- start 2023-10-16T10:08:08.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.167+0000 7f2dfbffb640 1 -- start start 2023-10-16T10:08:08.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.167+0000 7f2dfbffb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de0006f10 0x7f2df40c3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:08.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.167+0000 7f2dfbffb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2df4157700 0x7f2df41f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:08.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.167+0000 7f2dfbffb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2df4157b80 0x7f2df41f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:08.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.167+0000 7f2dfbffb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40c3440 con 0x7f2df4157b80 2023-10-16T10:08:08.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.167+0000 7f2dfbffb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df41f2c40 con 0x7f2df4157700 2023-10-16T10:08:08.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.167+0000 7f2dfbffb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df41f6a50 con 0x7f2de0006f10 2023-10-16T10:08:08.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.168+0000 7f2dfa598640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2df4157b80 0x7f2df41f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:08.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.168+0000 7f2dfa598640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2df4157b80 0x7f2df41f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57458/0 (socket says 172.21.15.84:57458) 2023-10-16T10:08:08.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.168+0000 7f2dfa598640 1 -- 172.21.15.84:0/3483629861 learned_addr learned my addr 172.21.15.84:0/3483629861 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:08.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.168+0000 7f2df9596640 1 --2- 172.21.15.84:0/3483629861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2df4157700 0x7f2df41f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:08.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.168+0000 7f2df9d97640 1 --2- 172.21.15.84:0/3483629861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de0006f10 0x7f2df40c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:08.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.168+0000 7f2dfa598640 1 -- 172.21.15.84:0/3483629861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de0006f10 msgr2=0x7f2df40c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:08.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.168+0000 7f2dfa598640 1 --2- 172.21.15.84:0/3483629861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de0006f10 0x7f2df40c3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.168+0000 7f2dfa598640 1 -- 172.21.15.84:0/3483629861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2df4157700 msgr2=0x7f2df41f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:08.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.168+0000 7f2dfa598640 1 --2- 172.21.15.84:0/3483629861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2df4157700 0x7f2df41f5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.168+0000 7f2dfa598640 1 -- 172.21.15.84:0/3483629861 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df41f6d30 con 0x7f2df4157b80 2023-10-16T10:08:08.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.168+0000 7f2df9596640 1 --2- 172.21.15.84:0/3483629861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2df4157700 0x7f2df41f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:08:08.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.169+0000 7f2dfa598640 1 --2- 172.21.15.84:0/3483629861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2df4157b80 0x7f2df41f6420 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f2dec02dfd0 tx=0x7f2dec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:08.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.169+0000 7f2dea7fc640 1 -- 172.21.15.84:0/3483629861 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec044070 con 0x7f2df4157b80 2023-10-16T10:08:08.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.169+0000 7f2dea7fc640 1 -- 172.21.15.84:0/3483629861 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dec03e400 con 0x7f2df4157b80 2023-10-16T10:08:08.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.169+0000 7f2dea7fc640 1 -- 172.21.15.84:0/3483629861 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec03e770 con 0x7f2df4157b80 2023-10-16T10:08:08.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.169+0000 7f2dfbffb640 1 -- 172.21.15.84:0/3483629861 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df40c2d80 con 0x7f2df4157b80 2023-10-16T10:08:08.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.169+0000 7f2dfbffb640 1 -- 172.21.15.84:0/3483629861 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df41522f0 con 0x7f2df4157b80 2023-10-16T10:08:08.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.170+0000 7f2dfbffb640 1 -- 172.21.15.84:0/3483629861 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df40c3580 con 0x7f2df4157b80 2023-10-16T10:08:08.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.172+0000 7f2dea7fc640 1 -- 172.21.15.84:0/3483629861 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 66) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f2dec03e8d0 con 0x7f2df4157b80 2023-10-16T10:08:08.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.173+0000 7f2dea7fc640 1 --2- 172.21.15.84:0/3483629861 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f2dd806b0b0 0x7f2dd806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:08.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.173+0000 7f2dea7fc640 1 -- 172.21.15.84:0/3483629861 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f2dec0ab9d0 con 0x7f2df4157b80 2023-10-16T10:08:08.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.174+0000 7f2df9d97640 1 -- 172.21.15.84:0/3483629861 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f2dd806b0b0 msgr2=0x7f2dd806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2209954486 2023-10-16T10:08:08.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.174+0000 7f2df9d97640 1 --2- 172.21.15.84:0/3483629861 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f2dd806b0b0 0x7f2dd806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:08.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.182+0000 7f2dea7fc640 1 -- 172.21.15.84:0/3483629861 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2df40c3580 con 0x7f2df4157b80 2023-10-16T10:08:08.237 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:08.235+0000 7fc263432040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:08:08.242 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:08.240+0000 7facc7012040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:08:08.301 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:08.299+0000 7facc7012040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:08:08.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.375+0000 7f2df9d97640 1 -- 172.21.15.84:0/3483629861 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f2dd806b0b0 msgr2=0x7f2dd806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2209954486 2023-10-16T10:08:08.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.375+0000 7f2df9d97640 1 --2- 172.21.15.84:0/3483629861 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f2dd806b0b0 0x7f2dd806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:08:08.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.377+0000 7f2dfbffb640 1 -- 172.21.15.84:0/3483629861 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2df40c3580 con 0x7f2df4157b80 2023-10-16T10:08:08.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.378+0000 7f2dea7fc640 1 -- 172.21.15.84:0/3483629861 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2df40c3580 con 0x7f2df4157b80 2023-10-16T10:08:08.380 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:08:08.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.380+0000 7f2dfbffb640 1 -- 172.21.15.84:0/3483629861 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f2dd806b0b0 msgr2=0x7f2dd806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:08.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.380+0000 7f2dfbffb640 1 --2- 172.21.15.84:0/3483629861 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f2dd806b0b0 0x7f2dd806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.380+0000 7f2dfbffb640 1 -- 172.21.15.84:0/3483629861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2df4157b80 msgr2=0x7f2df41f6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:08.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.380+0000 7f2dfbffb640 1 --2- 172.21.15.84:0/3483629861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2df4157b80 0x7f2df41f6420 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f2dec02dfd0 tx=0x7f2dec004ab0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.381+0000 7f2dfbffb640 1 -- 172.21.15.84:0/3483629861 shutdown_connections 2023-10-16T10:08:08.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.381+0000 7f2dfbffb640 1 --2- 172.21.15.84:0/3483629861 >> [v2:172.21.15.84:6832/2209954486,v1:172.21.15.84:6833/2209954486] conn(0x7f2dd806b0b0 0x7f2dd806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.381+0000 7f2dfbffb640 1 --2- 172.21.15.84:0/3483629861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2df4157b80 0x7f2df41f6420 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.381+0000 7f2dfbffb640 1 --2- 172.21.15.84:0/3483629861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2df4157700 0x7f2df41f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.381+0000 7f2dfbffb640 1 --2- 172.21.15.84:0/3483629861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de0006f10 0x7f2df40c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.381+0000 7f2dfbffb640 1 -- 172.21.15.84:0/3483629861 >> 172.21.15.84:0/3483629861 conn(0x7f2df406bc10 msgr2=0x7f2df406dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:08.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.381+0000 7f2dfbffb640 1 -- 172.21.15.84:0/3483629861 shutdown_connections 2023-10-16T10:08:08.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.381+0000 7f2dfbffb640 1 -- 172.21.15.84:0/3483629861 wait complete. 2023-10-16T10:08:08.399 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:08:08.423 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:08.422+0000 7facc7012040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:08:08.498 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:08.496+0000 7fac47696640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:08.499 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:08.496+0000 7fac47696640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:08.499 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:08.496+0000 7fac47696640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:08.499 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:08.497+0000 7fac47696640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:08.499 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:08.497+0000 7fac47696640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:08.500 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:08.497+0000 7fac4a69c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:08.500 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:08.497+0000 7fac4a69c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:08.500 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:08.497+0000 7fac4a69c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:08.500 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:08.497+0000 7fac4a69c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:08.501 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:08.497+0000 7fac4a69c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:08.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.591+0000 7f076e26f640 1 Processor -- start 2023-10-16T10:08:08.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.591+0000 7f076e26f640 1 -- start start 2023-10-16T10:08:08.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.592+0000 7f076e26f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f076815ad70 0x7f0768069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:08.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.592+0000 7f076e26f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07680c9ca0 0x7f076806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:08.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.592+0000 7f076e26f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f07680ca3b0 0x7f07680ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:08.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.592+0000 7f076e26f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07680cad80 con 0x7f07680ca3b0 2023-10-16T10:08:08.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.592+0000 7f076e26f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07680cb030 con 0x7f076815ad70 2023-10-16T10:08:08.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.592+0000 7f076e26f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07680c91c0 con 0x7f07680c9ca0 2023-10-16T10:08:08.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.592+0000 7f076c80c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f07680ca3b0 0x7f07680ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:08.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.593+0000 7f076c80c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f07680ca3b0 0x7f07680ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57580/0 (socket says 172.21.15.84:57580) 2023-10-16T10:08:08.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.593+0000 7f076c80c640 1 -- 172.21.15.84:0/3009323723 learned_addr learned my addr 172.21.15.84:0/3009323723 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:08.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.593+0000 7f07677fe640 1 --2- 172.21.15.84:0/3009323723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07680c9ca0 0x7f076806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:08.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.593+0000 7f076c80c640 1 -- 172.21.15.84:0/3009323723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07680c9ca0 msgr2=0x7f076806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:08.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.593+0000 7f076c80c640 1 --2- 172.21.15.84:0/3009323723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07680c9ca0 0x7f076806a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.593+0000 7f0767fff640 1 --2- 172.21.15.84:0/3009323723 >> v2:172.21.15.157:3300/0 conn(0x7f076815ad70 0x7f0768069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:08.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.593+0000 7f076c80c640 1 -- 172.21.15.84:0/3009323723 >> v2:172.21.15.157:3300/0 conn(0x7f076815ad70 msgr2=0x7f0768069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:08.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.593+0000 7f076c80c640 1 --2- 172.21.15.84:0/3009323723 >> v2:172.21.15.157:3300/0 conn(0x7f076815ad70 0x7f0768069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.593+0000 7f076c80c640 1 -- 172.21.15.84:0/3009323723 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07680c94a0 con 0x7f07680ca3b0 2023-10-16T10:08:08.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.594+0000 7f076c80c640 1 --2- 172.21.15.84:0/3009323723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07680ca3b0 0x7f07680ca840 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f07680cf780 tx=0x7f076002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73882750917b4d85 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:08.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.594+0000 7f0766ffd640 1 -- 172.21.15.84:0/3009323723 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076002ca80 con 0x7f07680ca3b0 2023-10-16T10:08:08.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.594+0000 7f0766ffd640 1 -- 172.21.15.84:0/3009323723 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076002cbe0 con 0x7f07680ca3b0 2023-10-16T10:08:08.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.594+0000 7f0766ffd640 1 -- 172.21.15.84:0/3009323723 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07600343f0 con 0x7f07680ca3b0 2023-10-16T10:08:08.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.595+0000 7f076e26f640 1 -- 172.21.15.84:0/3009323723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07680ca3b0 msgr2=0x7f07680ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:08.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.595+0000 7f076e26f640 1 --2- 172.21.15.84:0/3009323723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07680ca3b0 0x7f07680ca840 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f07680cf780 tx=0x7f076002bee0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.595+0000 7f076e26f640 1 -- 172.21.15.84:0/3009323723 shutdown_connections 2023-10-16T10:08:08.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.595+0000 7f076e26f640 1 --2- 172.21.15.84:0/3009323723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07680ca3b0 0x7f07680ca840 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.595+0000 7f076e26f640 1 --2- 172.21.15.84:0/3009323723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07680c9ca0 0x7f076806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.595+0000 7f076e26f640 1 --2- 172.21.15.84:0/3009323723 >> v2:172.21.15.157:3300/0 conn(0x7f076815ad70 0x7f0768069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.595+0000 7f076e26f640 1 -- 172.21.15.84:0/3009323723 >> 172.21.15.84:0/3009323723 conn(0x7f07680ce9b0 msgr2=0x7f07680cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:08.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.595+0000 7f076e26f640 1 -- 172.21.15.84:0/3009323723 shutdown_connections 2023-10-16T10:08:08.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.595+0000 7f076e26f640 1 -- 172.21.15.84:0/3009323723 wait complete. 2023-10-16T10:08:08.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.597+0000 7f076e26f640 1 Processor -- start 2023-10-16T10:08:08.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.597+0000 7f076e26f640 1 -- start start 2023-10-16T10:08:08.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.598+0000 7f076e26f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07680c9ca0 0x7f07680b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:08.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.598+0000 7f076e26f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07680ca3b0 0x7f0768159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:08.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.598+0000 7f076e26f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f076815ad70 0x7f07680cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:08.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.598+0000 7f076e26f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07680b1a00 con 0x7f076815ad70 2023-10-16T10:08:08.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.598+0000 7f076e26f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07681f2630 con 0x7f07680ca3b0 2023-10-16T10:08:08.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.598+0000 7f076e26f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07681f2910 con 0x7f07680c9ca0 2023-10-16T10:08:08.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.598+0000 7f076c80c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f076815ad70 0x7f07680cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:08.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.598+0000 7f076c80c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f076815ad70 0x7f07680cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57594/0 (socket says 172.21.15.84:57594) 2023-10-16T10:08:08.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.598+0000 7f076c80c640 1 -- 172.21.15.84:0/562059499 learned_addr learned my addr 172.21.15.84:0/562059499 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:08.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.598+0000 7f07677fe640 1 --2- 172.21.15.84:0/562059499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07680ca3b0 0x7f0768159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:08.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.598+0000 7f0767fff640 1 --2- 172.21.15.84:0/562059499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07680c9ca0 0x7f07680b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:08.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.598+0000 7f076c80c640 1 -- 172.21.15.84:0/562059499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07680c9ca0 msgr2=0x7f07680b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:08.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.598+0000 7f076c80c640 1 --2- 172.21.15.84:0/562059499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07680c9ca0 0x7f07680b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.598+0000 7f076c80c640 1 -- 172.21.15.84:0/562059499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07680ca3b0 msgr2=0x7f0768159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:08.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.598+0000 7f076c80c640 1 --2- 172.21.15.84:0/562059499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07680ca3b0 0x7f0768159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.598+0000 7f076c80c640 1 -- 172.21.15.84:0/562059499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0760009b30 con 0x7f076815ad70 2023-10-16T10:08:08.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.599+0000 7f076c80c640 1 --2- 172.21.15.84:0/562059499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f076815ad70 0x7f07680cdd60 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f0760036040 tx=0x7f0760004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:08.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.599+0000 7f0764ff9640 1 -- 172.21.15.84:0/562059499 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076003e450 con 0x7f076815ad70 2023-10-16T10:08:08.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.599+0000 7f0764ff9640 1 -- 172.21.15.84:0/562059499 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076003e5d0 con 0x7f076815ad70 2023-10-16T10:08:08.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.599+0000 7f0764ff9640 1 -- 172.21.15.84:0/562059499 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076003e980 con 0x7f076815ad70 2023-10-16T10:08:08.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.599+0000 7f076e26f640 1 -- 172.21.15.84:0/562059499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07681f2bf0 con 0x7f076815ad70 2023-10-16T10:08:08.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.599+0000 7f076e26f640 1 -- 172.21.15.84:0/562059499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0768159850 con 0x7f076815ad70 2023-10-16T10:08:08.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.600+0000 7f076e26f640 1 -- 172.21.15.84:0/562059499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f076804e160 con 0x7f076815ad70 2023-10-16T10:08:08.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.601+0000 7f0764ff9640 1 -- 172.21.15.84:0/562059499 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 67) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f076002cd50 con 0x7f076815ad70 2023-10-16T10:08:08.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.602+0000 7f0764ff9640 1 -- 172.21.15.84:0/562059499 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f07600ac0c0 con 0x7f076815ad70 2023-10-16T10:08:08.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.606+0000 7f0764ff9640 1 -- 172.21.15.84:0/562059499 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0760077270 con 0x7f076815ad70 2023-10-16T10:08:08.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.782+0000 7f076e26f640 1 -- 172.21.15.84:0/562059499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f076815da20 con 0x7f076815ad70 2023-10-16T10:08:08.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.783+0000 7f0764ff9640 1 -- 172.21.15.84:0/562059499 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) v1 ==== 104+0+11 (secure 0 0 0) 0x7f076007a890 con 0x7f076815ad70 2023-10-16T10:08:08.785 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T10:08:08.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.785+0000 7f076e26f640 1 -- 172.21.15.84:0/562059499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f076815ad70 msgr2=0x7f07680cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:08.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.785+0000 7f076e26f640 1 --2- 172.21.15.84:0/562059499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f076815ad70 0x7f07680cdd60 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f0760036040 tx=0x7f0760004a40 comp rx=0 tx=0).stop 2023-10-16T10:08:08.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.785+0000 7f076e26f640 1 -- 172.21.15.84:0/562059499 shutdown_connections 2023-10-16T10:08:08.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.785+0000 7f076e26f640 1 --2- 172.21.15.84:0/562059499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f076815ad70 0x7f07680cdd60 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.785+0000 7f076e26f640 1 --2- 172.21.15.84:0/562059499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07680ca3b0 0x7f0768159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.785+0000 7f076e26f640 1 --2- 172.21.15.84:0/562059499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07680c9ca0 0x7f07680b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:08.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.785+0000 7f076e26f640 1 -- 172.21.15.84:0/562059499 >> 172.21.15.84:0/562059499 conn(0x7f07680ce9b0 msgr2=0x7f076815dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:08.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.785+0000 7f076e26f640 1 -- 172.21.15.84:0/562059499 shutdown_connections 2023-10-16T10:08:08.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:08.785+0000 7f076e26f640 1 -- 172.21.15.84:0/562059499 wait complete. 2023-10-16T10:08:08.804 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:08:09.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.056+0000 7fd55e061640 1 Processor -- start 2023-10-16T10:08:09.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.056+0000 7fd55e061640 1 -- start start 2023-10-16T10:08:09.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.057+0000 7fd55e061640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd558159ab0 0x7fd5580ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:09.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.057+0000 7fd55e061640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd558159e90 0x7fd55806fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:09.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.057+0000 7fd55e061640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5580cd1f0 con 0x7fd55806c140 2023-10-16T10:08:09.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.057+0000 7fd55e061640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55815d890 con 0x7fd558159e90 2023-10-16T10:08:09.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.057+0000 7fd55e061640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580cd4d0 con 0x7fd558159ab0 2023-10-16T10:08:09.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.058+0000 7fd556ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd558159e90 0x7fd55806fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:09.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.058+0000 7fd556ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd558159e90 0x7fd55806fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:54724/0 (socket says 172.21.15.84:54724) 2023-10-16T10:08:09.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.058+0000 7fd556ffd640 1 -- 172.21.15.84:0/1333462806 learned_addr learned my addr 172.21.15.84:0/1333462806 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:09.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.058+0000 7fd5577fe640 1 --2- 172.21.15.84:0/1333462806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd558159ab0 0x7fd5580ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:09.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.059+0000 7fd556ffd640 1 -- 172.21.15.84:0/1333462806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd558159ab0 msgr2=0x7fd5580ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:09.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.059+0000 7fd556ffd640 1 --2- 172.21.15.84:0/1333462806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd558159ab0 0x7fd5580ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.059+0000 7fd556ffd640 1 -- 172.21.15.84:0/1333462806 >> v1:172.21.15.157:6789/0 conn(0x7fd55806c140 legacy=0x7fd55815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:09.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.059+0000 7fd556ffd640 1 -- 172.21.15.84:0/1333462806 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5580cd7b0 con 0x7fd558159e90 2023-10-16T10:08:09.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.059+0000 7fd5577fe640 1 --2- 172.21.15.84:0/1333462806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd558159ab0 0x7fd5580ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:08:09.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.061+0000 7fd556ffd640 1 --2- 172.21.15.84:0/1333462806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd558159e90 0x7fd55806fc50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0098a0 tx=0x7fd54c02b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f3db69b7c49b354 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:09.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.068+0000 7fd5567fc640 1 -- 172.21.15.84:0/1333462806 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c0317c0 con 0x7fd558159e90 2023-10-16T10:08:09.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.068+0000 7fd5567fc640 1 -- 172.21.15.84:0/1333462806 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54c031920 con 0x7fd558159e90 2023-10-16T10:08:09.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.068+0000 7fd5567fc640 1 -- 172.21.15.84:0/1333462806 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c031cb0 con 0x7fd558159e90 2023-10-16T10:08:09.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.068+0000 7fd55e061640 1 -- 172.21.15.84:0/1333462806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd558159e90 msgr2=0x7fd55806fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:09.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.068+0000 7fd55e061640 1 --2- 172.21.15.84:0/1333462806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd558159e90 0x7fd55806fc50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0098a0 tx=0x7fd54c02b860 comp rx=0 tx=0).stop 2023-10-16T10:08:09.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.070+0000 7fd55e061640 1 -- 172.21.15.84:0/1333462806 shutdown_connections 2023-10-16T10:08:09.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.070+0000 7fd55e061640 1 --2- 172.21.15.84:0/1333462806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd558159e90 0x7fd55806fc50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.070+0000 7fd55e061640 1 --2- 172.21.15.84:0/1333462806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd558159ab0 0x7fd5580ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.070+0000 7fd55e061640 1 -- 172.21.15.84:0/1333462806 >> 172.21.15.84:0/1333462806 conn(0x7fd5580ce9b0 msgr2=0x7fd5580cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:09.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.070+0000 7fd55e061640 1 -- 172.21.15.84:0/1333462806 shutdown_connections 2023-10-16T10:08:09.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.070+0000 7fd55e061640 1 -- 172.21.15.84:0/1333462806 wait complete. 2023-10-16T10:08:09.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.072+0000 7fd55e061640 1 Processor -- start 2023-10-16T10:08:09.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.072+0000 7fd55e061640 1 -- start start 2023-10-16T10:08:09.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.073+0000 7fd55e061640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd55806c140 0x7fd5580b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:09.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.073+0000 7fd55e061640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd558159ab0 0x7fd558154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:09.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.074+0000 7fd55e061640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd558159e90 0x7fd55815c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:09.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.074+0000 7fd55e061640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580c7a50 con 0x7fd55806c140 2023-10-16T10:08:09.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.074+0000 7fd55e061640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580c7d30 con 0x7fd558159e90 2023-10-16T10:08:09.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.074+0000 7fd55e061640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580c8010 con 0x7fd558159ab0 2023-10-16T10:08:09.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.074+0000 7fd5577fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd55806c140 0x7fd5580b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:09.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.074+0000 7fd5577fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd55806c140 0x7fd5580b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57606/0 (socket says 172.21.15.84:57606) 2023-10-16T10:08:09.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.074+0000 7fd5577fe640 1 -- 172.21.15.84:0/3609196669 learned_addr learned my addr 172.21.15.84:0/3609196669 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:09.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.074+0000 7fd5577fe640 1 -- 172.21.15.84:0/3609196669 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd558159ab0 msgr2=0x7fd558154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:09.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.074+0000 7fd5577fe640 1 --2- 172.21.15.84:0/3609196669 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd558159ab0 0x7fd558154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.074+0000 7fd5577fe640 1 -- 172.21.15.84:0/3609196669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd558159e90 msgr2=0x7fd55815c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:09.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.074+0000 7fd5577fe640 1 --2- 172.21.15.84:0/3609196669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd558159e90 0x7fd55815c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.074+0000 7fd5577fe640 1 -- 172.21.15.84:0/3609196669 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c0094d0 con 0x7fd55806c140 2023-10-16T10:08:09.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.075+0000 7fd5577fe640 1 --2- 172.21.15.84:0/3609196669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd55806c140 0x7fd5580b0860 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fd54800e7e0 tx=0x7fd54800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:09.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.075+0000 7fd53ffff640 1 -- 172.21.15.84:0/3609196669 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548004030 con 0x7fd55806c140 2023-10-16T10:08:09.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.075+0000 7fd53ffff640 1 -- 172.21.15.84:0/3609196669 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd548004190 con 0x7fd55806c140 2023-10-16T10:08:09.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.075+0000 7fd55e061640 1 -- 172.21.15.84:0/3609196669 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5580c82f0 con 0x7fd55806c140 2023-10-16T10:08:09.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.075+0000 7fd53ffff640 1 -- 172.21.15.84:0/3609196669 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548004520 con 0x7fd55806c140 2023-10-16T10:08:09.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.075+0000 7fd55e061640 1 -- 172.21.15.84:0/3609196669 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd558159040 con 0x7fd55806c140 2023-10-16T10:08:09.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.076+0000 7fd55e061640 1 -- 172.21.15.84:0/3609196669 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd55804e160 con 0x7fd55806c140 2023-10-16T10:08:09.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.077+0000 7fd53ffff640 1 -- 172.21.15.84:0/3609196669 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 67) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fd54800c6d0 con 0x7fd55806c140 2023-10-16T10:08:09.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.078+0000 7fd53ffff640 1 -- 172.21.15.84:0/3609196669 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fd548089290 con 0x7fd55806c140 2023-10-16T10:08:09.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.083+0000 7fd53ffff640 1 -- 172.21.15.84:0/3609196669 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd548054310 con 0x7fd55806c140 2023-10-16T10:08:09.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.273+0000 7fd55e061640 1 -- 172.21.15.84:0/3609196669 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd55806bd30 con 0x7fd55806c140 2023-10-16T10:08:09.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.274+0000 7fd53ffff640 1 -- 172.21.15.84:0/3609196669 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd54805ce60 con 0x7fd55806c140 2023-10-16T10:08:09.276 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T10:08:09.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.278+0000 7fd55e061640 1 -- 172.21.15.84:0/3609196669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd55806c140 msgr2=0x7fd5580b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:09.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.278+0000 7fd55e061640 1 --2- 172.21.15.84:0/3609196669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd55806c140 0x7fd5580b0860 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fd54800e7e0 tx=0x7fd54800eca0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.278+0000 7fd55e061640 1 -- 172.21.15.84:0/3609196669 shutdown_connections 2023-10-16T10:08:09.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.278+0000 7fd55e061640 1 --2- 172.21.15.84:0/3609196669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd558159e90 0x7fd55815c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.278+0000 7fd55e061640 1 --2- 172.21.15.84:0/3609196669 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd558159ab0 0x7fd558154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.278+0000 7fd55e061640 1 --2- 172.21.15.84:0/3609196669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd55806c140 0x7fd5580b0860 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.278+0000 7fd55e061640 1 -- 172.21.15.84:0/3609196669 >> 172.21.15.84:0/3609196669 conn(0x7fd5580ce9b0 msgr2=0x7fd5580af9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:09.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.278+0000 7fd55e061640 1 -- 172.21.15.84:0/3609196669 shutdown_connections 2023-10-16T10:08:09.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.278+0000 7fd55e061640 1 -- 172.21.15.84:0/3609196669 wait complete. 2023-10-16T10:08:09.297 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:08:09.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.518+0000 7f0fd7fff640 1 Processor -- start 2023-10-16T10:08:09.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.519+0000 7f0fd7fff640 1 -- start start 2023-10-16T10:08:09.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.521+0000 7f0fd7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0fd80c9060 0x7f0fd806f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:09.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.521+0000 7f0fd7fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd80ca560 con 0x7f0fd806c9e0 2023-10-16T10:08:09.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.521+0000 7f0fd7fff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd8165b70 con 0x7f0fd80c9440 2023-10-16T10:08:09.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.521+0000 7f0fd7fff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd80c9950 con 0x7f0fd80c9060 2023-10-16T10:08:09.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.521+0000 7f0fd6ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0fd80c9060 0x7f0fd806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:09.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.521+0000 7f0fcffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0fd806c9e0 0x7f0fd8164d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33144/0 (socket says 172.21.15.84:33144) 2023-10-16T10:08:09.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.521+0000 7f0fd6ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0fd80c9060 0x7f0fd806f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57612/0 (socket says 172.21.15.84:57612) 2023-10-16T10:08:09.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.521+0000 7f0fcffff640 1 -- 172.21.15.84:0/2084018612 learned_addr learned my addr 172.21.15.84:0/2084018612 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:09.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.521+0000 7f0fd6ffd640 1 -- 172.21.15.84:0/2084018612 >> v1:172.21.15.157:6789/0 conn(0x7f0fd80c9440 legacy=0x7f0fd8165460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:09.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.521+0000 7f0fd6ffd640 1 -- 172.21.15.84:0/2084018612 >> v1:172.21.15.84:6789/0 conn(0x7f0fd806c9e0 legacy=0x7f0fd8164d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:09.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.521+0000 7f0fd6ffd640 1 -- 172.21.15.84:0/2084018612 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd80c9c30 con 0x7f0fd80c9060 2023-10-16T10:08:09.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.522+0000 7f0fd6ffd640 1 --2- 172.21.15.84:0/2084018612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd80c9060 0x7f0fd806f440 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80097a0 tx=0x7f0fc802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f984d1d217c6eec8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:09.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.522+0000 7f0fd67fc640 1 -- 172.21.15.84:0/2084018612 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8031830 con 0x7f0fd80c9060 2023-10-16T10:08:09.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.522+0000 7f0fd67fc640 1 -- 172.21.15.84:0/2084018612 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc8031990 con 0x7f0fd80c9060 2023-10-16T10:08:09.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.522+0000 7f0fd67fc640 1 -- 172.21.15.84:0/2084018612 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8031d20 con 0x7f0fd80c9060 2023-10-16T10:08:09.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.522+0000 7f0fd7fff640 1 -- 172.21.15.84:0/2084018612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd80c9060 msgr2=0x7f0fd806f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:09.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.522+0000 7f0fd7fff640 1 --2- 172.21.15.84:0/2084018612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd80c9060 0x7f0fd806f440 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80097a0 tx=0x7f0fc802b8d0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.522+0000 7f0fd7fff640 1 -- 172.21.15.84:0/2084018612 shutdown_connections 2023-10-16T10:08:09.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.522+0000 7f0fd7fff640 1 --2- 172.21.15.84:0/2084018612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd80c9060 0x7f0fd806f440 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.522+0000 7f0fd7fff640 1 -- 172.21.15.84:0/2084018612 >> 172.21.15.84:0/2084018612 conn(0x7f0fd80ce6a0 msgr2=0x7f0fd80ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:09.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.523+0000 7f0fd7fff640 1 -- 172.21.15.84:0/2084018612 shutdown_connections 2023-10-16T10:08:09.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.523+0000 7f0fd7fff640 1 -- 172.21.15.84:0/2084018612 wait complete. 2023-10-16T10:08:09.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.524+0000 7f0fd7fff640 1 Processor -- start 2023-10-16T10:08:09.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.524+0000 7f0fd7fff640 1 -- start start 2023-10-16T10:08:09.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.525+0000 7f0fd7fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fd806c9e0 0x7f0fd8155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:09.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.525+0000 7f0fd7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd80c9060 0x7f0fd8157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:09.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.525+0000 7f0fd7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fd80c9440 0x7f0fd81583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:09.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.525+0000 7f0fd7fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd81f7a30 con 0x7f0fd80c9060 2023-10-16T10:08:09.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.525+0000 7f0fd7fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd81589e0 con 0x7f0fd80c9440 2023-10-16T10:08:09.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.525+0000 7f0fd7fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd8158cc0 con 0x7f0fd806c9e0 2023-10-16T10:08:09.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.525+0000 7f0fcffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd80c9060 0x7f0fd8157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:09.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.526+0000 7f0fcffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd80c9060 0x7f0fd8157e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57622/0 (socket says 172.21.15.84:57622) 2023-10-16T10:08:09.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.526+0000 7f0fcffff640 1 -- 172.21.15.84:0/434288697 learned_addr learned my addr 172.21.15.84:0/434288697 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:09.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.526+0000 7f0fd6ffd640 1 --2- 172.21.15.84:0/434288697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fd806c9e0 0x7f0fd8155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:09.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.526+0000 7f0fd77fe640 1 --2- 172.21.15.84:0/434288697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fd80c9440 0x7f0fd81583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:09.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.526+0000 7f0fcffff640 1 -- 172.21.15.84:0/434288697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fd806c9e0 msgr2=0x7f0fd8155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:09.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.526+0000 7f0fcffff640 1 --2- 172.21.15.84:0/434288697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fd806c9e0 0x7f0fd8155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.526+0000 7f0fcffff640 1 -- 172.21.15.84:0/434288697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fd80c9440 msgr2=0x7f0fd81583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:09.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.526+0000 7f0fcffff640 1 --2- 172.21.15.84:0/434288697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fd80c9440 0x7f0fd81583d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.526+0000 7f0fcffff640 1 -- 172.21.15.84:0/434288697 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc8009400 con 0x7f0fd80c9060 2023-10-16T10:08:09.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.526+0000 7f0fcffff640 1 --2- 172.21.15.84:0/434288697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd80c9060 0x7f0fd8157e90 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000c3c0 tx=0x7f0fc000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:09.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.526+0000 7f0fcf7fe640 1 -- 172.21.15.84:0/434288697 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0005ce0 con 0x7f0fd80c9060 2023-10-16T10:08:09.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.527+0000 7f0fcf7fe640 1 -- 172.21.15.84:0/434288697 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc0005e40 con 0x7f0fd80c9060 2023-10-16T10:08:09.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.527+0000 7f0fcf7fe640 1 -- 172.21.15.84:0/434288697 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc00072d0 con 0x7f0fd80c9060 2023-10-16T10:08:09.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.527+0000 7f0fd7fff640 1 -- 172.21.15.84:0/434288697 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd8158fa0 con 0x7f0fd80c9060 2023-10-16T10:08:09.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.527+0000 7f0fd7fff640 1 -- 172.21.15.84:0/434288697 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd8159960 con 0x7f0fd80c9060 2023-10-16T10:08:09.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.528+0000 7f0fd7fff640 1 -- 172.21.15.84:0/434288697 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd81f7be0 con 0x7f0fd80c9060 2023-10-16T10:08:09.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.529+0000 7f0fcf7fe640 1 -- 172.21.15.84:0/434288697 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 68) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f0fc00074f0 con 0x7f0fd80c9060 2023-10-16T10:08:09.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.530+0000 7f0fcf7fe640 1 --2- 172.21.15.84:0/434288697 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f0fb806acc0 0x7f0fb806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:09.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.530+0000 7f0fcf7fe640 1 -- 172.21.15.84:0/434288697 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6539+0+0 (secure 0 0 0) 0x7f0fc0088920 con 0x7f0fd80c9060 2023-10-16T10:08:09.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.530+0000 7f0fd6ffd640 1 --2- 172.21.15.84:0/434288697 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f0fb806acc0 0x7f0fb806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:09.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.531+0000 7f0fd6ffd640 1 --2- 172.21.15.84:0/434288697 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f0fb806acc0 0x7f0fb806d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8036000 tx=0x7f0fc8008030 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:09.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.534+0000 7f0fcf7fe640 1 -- 172.21.15.84:0/434288697 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0fd81f7be0 con 0x7f0fd80c9060 2023-10-16T10:08:09.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.758+0000 7f0fd7fff640 1 -- 172.21.15.84:0/434288697 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fd804e9b0 con 0x7f0fd80c9060 2023-10-16T10:08:09.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.759+0000 7f0fcf7fe640 1 -- 172.21.15.84:0/434288697 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f0fc0017090 con 0x7f0fd80c9060 2023-10-16T10:08:09.761 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:09.762 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2012273152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2012273152},{"type":"v1","addr":"172.21.15.157:6835","nonce":2012273152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/435606514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":435606514},{"type":"v1","addr":"172.21.15.157:6833","nonce":435606514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2286048798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2286048798},{"type":"v1","addr":"172.21.15.84:6836","nonce":2286048798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:07:28.798746+0000","modified":"2023-10-16T10:07:31.857629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2859088731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2859088731},{"type":"v1","addr":"172.21.15.84:6837","nonce":2859088731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-16T10:08:09.762 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 62 2023-10-16T10:08:09.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.762+0000 7f0fd7fff640 1 -- 172.21.15.84:0/434288697 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f0fb806acc0 msgr2=0x7f0fb806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:09.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.762+0000 7f0fd7fff640 1 --2- 172.21.15.84:0/434288697 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f0fb806acc0 0x7f0fb806d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8036000 tx=0x7f0fc8008030 comp rx=0 tx=0).stop 2023-10-16T10:08:09.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.762+0000 7f0fd7fff640 1 -- 172.21.15.84:0/434288697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd80c9060 msgr2=0x7f0fd8157e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:09.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.762+0000 7f0fd7fff640 1 --2- 172.21.15.84:0/434288697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd80c9060 0x7f0fd8157e90 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000c3c0 tx=0x7f0fc000c880 comp rx=0 tx=0).stop 2023-10-16T10:08:09.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.762+0000 7f0fd7fff640 1 -- 172.21.15.84:0/434288697 shutdown_connections 2023-10-16T10:08:09.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.762+0000 7f0fd7fff640 1 --2- 172.21.15.84:0/434288697 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f0fb806acc0 0x7f0fb806d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.762+0000 7f0fd7fff640 1 --2- 172.21.15.84:0/434288697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fd80c9440 0x7f0fd81583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.762+0000 7f0fd7fff640 1 --2- 172.21.15.84:0/434288697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd80c9060 0x7f0fd8157e90 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.762+0000 7f0fd7fff640 1 --2- 172.21.15.84:0/434288697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fd806c9e0 0x7f0fd8155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.762+0000 7f0fd7fff640 1 -- 172.21.15.84:0/434288697 >> 172.21.15.84:0/434288697 conn(0x7f0fd80ce6a0 msgr2=0x7f0fd806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:09.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.763+0000 7f0fd7fff640 1 -- 172.21.15.84:0/434288697 shutdown_connections 2023-10-16T10:08:09.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.763+0000 7f0fd7fff640 1 -- 172.21.15.84:0/434288697 wait complete. 2023-10-16T10:08:09.783 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:08:09.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.986+0000 7fa4181b0640 1 Processor -- start 2023-10-16T10:08:09.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.986+0000 7fa4181b0640 1 -- start start 2023-10-16T10:08:09.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.988+0000 7fa4181b0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa41015aaa0 0x7fa41015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:09.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.988+0000 7fa4181b0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa41015ae80 0x7fa4100cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:09.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.988+0000 7fa4181b0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41015ed60 con 0x7fa4100ccaa0 2023-10-16T10:08:09.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.988+0000 7fa4181b0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa41006b810 con 0x7fa41015aaa0 2023-10-16T10:08:09.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.988+0000 7fa4181b0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4100707e0 con 0x7fa41015ae80 2023-10-16T10:08:09.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.988+0000 7fa415f4c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa41015aaa0 0x7fa41015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:09.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.988+0000 7fa41674d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa4100ccaa0 0x7fa410070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33148/0 (socket says 172.21.15.84:33148) 2023-10-16T10:08:09.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.988+0000 7fa41674d640 1 -- 172.21.15.84:0/2639488667 learned_addr learned my addr 172.21.15.84:0/2639488667 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:09.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.988+0000 7fa415f4c640 1 -- 172.21.15.84:0/2639488667 >> v2:172.21.15.157:3300/0 conn(0x7fa41015ae80 msgr2=0x7fa4100cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:09.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.988+0000 7fa41574b640 1 --2- 172.21.15.84:0/2639488667 >> v2:172.21.15.157:3300/0 conn(0x7fa41015ae80 0x7fa4100cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:09.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.988+0000 7fa415f4c640 1 --2- 172.21.15.84:0/2639488667 >> v2:172.21.15.157:3300/0 conn(0x7fa41015ae80 0x7fa4100cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.988+0000 7fa415f4c640 1 -- 172.21.15.84:0/2639488667 >> v1:172.21.15.84:6789/0 conn(0x7fa4100ccaa0 legacy=0x7fa410070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:09.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.988+0000 7fa414f4a640 1 -- 172.21.15.84:0/2639488667 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769769177 0 0) 0x7fa41015ed60 con 0x7fa4100ccaa0 2023-10-16T10:08:09.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.988+0000 7fa415f4c640 1 -- 172.21.15.84:0/2639488667 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41015ef90 con 0x7fa41015aaa0 2023-10-16T10:08:09.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.989+0000 7fa415f4c640 1 --2- 172.21.15.84:0/2639488667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa41015aaa0 0x7fa41015e820 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa4100c59c0 tx=0x7fa40c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673d41523025557 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:09.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.989+0000 7fa414f4a640 1 -- 172.21.15.84:0/2639488667 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c031830 con 0x7fa41015aaa0 2023-10-16T10:08:09.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.989+0000 7fa414f4a640 1 -- 172.21.15.84:0/2639488667 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c031990 con 0x7fa41015aaa0 2023-10-16T10:08:09.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.989+0000 7fa414f4a640 1 -- 172.21.15.84:0/2639488667 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c031d20 con 0x7fa41015aaa0 2023-10-16T10:08:09.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.990+0000 7fa4181b0640 1 -- 172.21.15.84:0/2639488667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa41015aaa0 msgr2=0x7fa41015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:09.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.990+0000 7fa4181b0640 1 --2- 172.21.15.84:0/2639488667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa41015aaa0 0x7fa41015e820 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa4100c59c0 tx=0x7fa40c02b8d0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.990+0000 7fa4181b0640 1 -- 172.21.15.84:0/2639488667 shutdown_connections 2023-10-16T10:08:09.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.990+0000 7fa4181b0640 1 --2- 172.21.15.84:0/2639488667 >> v2:172.21.15.157:3300/0 conn(0x7fa41015ae80 0x7fa4100cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.990+0000 7fa4181b0640 1 --2- 172.21.15.84:0/2639488667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa41015aaa0 0x7fa41015e820 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:09.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.990+0000 7fa4181b0640 1 -- 172.21.15.84:0/2639488667 >> 172.21.15.84:0/2639488667 conn(0x7fa4100cb410 msgr2=0x7fa4100cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:09.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.990+0000 7fa4181b0640 1 -- 172.21.15.84:0/2639488667 shutdown_connections 2023-10-16T10:08:09.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.990+0000 7fa4181b0640 1 -- 172.21.15.84:0/2639488667 wait complete. 2023-10-16T10:08:09.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.992+0000 7fa4181b0640 1 Processor -- start 2023-10-16T10:08:09.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.992+0000 7fa4181b0640 1 -- start start 2023-10-16T10:08:09.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.993+0000 7fa4181b0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4100ccaa0 0x7fa4101578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:09.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.993+0000 7fa4181b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa41015aaa0 0x7fa410156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:09.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.993+0000 7fa4181b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41015ae80 0x7fa4100b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:09.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.993+0000 7fa4181b0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4101568c0 con 0x7fa4100ccaa0 2023-10-16T10:08:09.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.993+0000 7fa4181b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4100b1050 con 0x7fa41015ae80 2023-10-16T10:08:09.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.993+0000 7fa4181b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa410156b20 con 0x7fa41015aaa0 2023-10-16T10:08:09.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.993+0000 7fa415f4c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4100ccaa0 0x7fa4101578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:09.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.993+0000 7fa415f4c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4100ccaa0 0x7fa4101578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57638/0 (socket says 172.21.15.84:57638) 2023-10-16T10:08:09.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.993+0000 7fa415f4c640 1 -- 172.21.15.84:0/316674650 learned_addr learned my addr 172.21.15.84:0/316674650 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:09.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.993+0000 7fa41674d640 1 --2- 172.21.15.84:0/316674650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41015ae80 0x7fa4100b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:09.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.993+0000 7fa415f4c640 1 -- 172.21.15.84:0/316674650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa41015aaa0 msgr2=0x7fa410156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:10.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.993+0000 7fa415f4c640 1 --2- 172.21.15.84:0/316674650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa41015aaa0 0x7fa410156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.993+0000 7fa415f4c640 1 -- 172.21.15.84:0/316674650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41015ae80 msgr2=0x7fa4100b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:10.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.993+0000 7fa415f4c640 1 --2- 172.21.15.84:0/316674650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41015ae80 0x7fa4100b30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.993+0000 7fa415f4c640 1 -- 172.21.15.84:0/316674650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c009400 con 0x7fa4100ccaa0 2023-10-16T10:08:10.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.994+0000 7fa415f4c640 1 --2- 172.21.15.84:0/316674650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4100ccaa0 0x7fa4101578e0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009890 tx=0x7fa40c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:10.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.994+0000 7fa4067fc640 1 -- 172.21.15.84:0/316674650 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c03ba70 con 0x7fa4100ccaa0 2023-10-16T10:08:10.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.994+0000 7fa4067fc640 1 -- 172.21.15.84:0/316674650 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c03bbd0 con 0x7fa4100ccaa0 2023-10-16T10:08:10.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.994+0000 7fa4067fc640 1 -- 172.21.15.84:0/316674650 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0453f0 con 0x7fa4100ccaa0 2023-10-16T10:08:10.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.994+0000 7fa4181b0640 1 -- 172.21.15.84:0/316674650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa410156e00 con 0x7fa4100ccaa0 2023-10-16T10:08:10.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.994+0000 7fa4181b0640 1 -- 172.21.15.84:0/316674650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4101573c0 con 0x7fa4100ccaa0 2023-10-16T10:08:10.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.995+0000 7fa4181b0640 1 -- 172.21.15.84:0/316674650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4101575b0 con 0x7fa4100ccaa0 2023-10-16T10:08:10.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.996+0000 7fa4067fc640 1 -- 172.21.15.84:0/316674650 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 68) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fa40c045550 con 0x7fa4100ccaa0 2023-10-16T10:08:10.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.997+0000 7fa4067fc640 1 --2- 172.21.15.84:0/316674650 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fa3f406acc0 0x7fa3f406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:10.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.997+0000 7fa4067fc640 1 -- 172.21.15.84:0/316674650 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6539+0+0 (secure 0 0 0) 0x7fa40c0ab8d0 con 0x7fa4100ccaa0 2023-10-16T10:08:10.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.997+0000 7fa41574b640 1 --2- 172.21.15.84:0/316674650 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fa3f406acc0 0x7fa3f406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:10.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:09.998+0000 7fa41574b640 1 --2- 172.21.15.84:0/316674650 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fa3f406acc0 0x7fa3f406d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa41015a500 tx=0x7fa400008040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:10.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.003+0000 7fa4067fc640 1 -- 172.21.15.84:0/316674650 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa40c076980 con 0x7fa4100ccaa0 2023-10-16T10:08:10.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.200+0000 7fa4181b0640 1 -- 172.21.15.84:0/316674650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa41004e160 con 0x7fa4100ccaa0 2023-10-16T10:08:10.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.203+0000 7fa4067fc640 1 -- 172.21.15.84:0/316674650 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+17730 (secure 0 0 0) 0x7fa40c079fa0 con 0x7fa4100ccaa0 2023-10-16T10:08:10.204 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:10.204 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":127,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:08:09.432278+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:07:26.607199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-16T10:07:27.832706+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":127,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/621277862":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/1967825537":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/2700178020":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/1638353973":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/1637648657":"2023-10-17T10:08:08.425362+0000","172.21.15.84:6833/2209954486":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/3662496273":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/1284479959":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/2038404786":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/688441146":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/644840609":"2023-10-17T10:07:43.574635+0000","172.21.15.84:6832/2209954486":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/3794895754":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/1166553040":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/3269894694":"2023-10-17T10:07:43.574635+0000","172.21.15.84:6833/4292025878":"2023-10-17T10:07:43.574635+0000","172.21.15.84:6832/4292025878":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/1934706185":"2023-10-17T10:07:43.574635+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:08:10.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.205+0000 7fa4181b0640 1 -- 172.21.15.84:0/316674650 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fa3f406acc0 msgr2=0x7fa3f406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:10.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.205+0000 7fa4181b0640 1 --2- 172.21.15.84:0/316674650 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fa3f406acc0 0x7fa3f406d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa41015a500 tx=0x7fa400008040 comp rx=0 tx=0).stop 2023-10-16T10:08:10.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.205+0000 7fa4181b0640 1 -- 172.21.15.84:0/316674650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4100ccaa0 msgr2=0x7fa4101578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:10.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.205+0000 7fa4181b0640 1 --2- 172.21.15.84:0/316674650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4100ccaa0 0x7fa4101578e0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009890 tx=0x7fa40c0046c0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.206+0000 7fa4181b0640 1 -- 172.21.15.84:0/316674650 shutdown_connections 2023-10-16T10:08:10.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.206+0000 7fa4181b0640 1 --2- 172.21.15.84:0/316674650 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fa3f406acc0 0x7fa3f406d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.206+0000 7fa4181b0640 1 --2- 172.21.15.84:0/316674650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41015ae80 0x7fa4100b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.206+0000 7fa4181b0640 1 --2- 172.21.15.84:0/316674650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa41015aaa0 0x7fa410156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.206+0000 7fa4181b0640 1 --2- 172.21.15.84:0/316674650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4100ccaa0 0x7fa4101578e0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.206+0000 7fa4181b0640 1 -- 172.21.15.84:0/316674650 >> 172.21.15.84:0/316674650 conn(0x7fa4100cb410 msgr2=0x7fa4100cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:10.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.206+0000 7fa4181b0640 1 -- 172.21.15.84:0/316674650 shutdown_connections 2023-10-16T10:08:10.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.206+0000 7fa4181b0640 1 -- 172.21.15.84:0/316674650 wait complete. 2023-10-16T10:08:10.226 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T10:08:10.227 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:08:10.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.437+0000 7fb5e4f27640 1 Processor -- start 2023-10-16T10:08:10.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.437+0000 7fb5e4f27640 1 -- start start 2023-10-16T10:08:10.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.439+0000 7fb5e4f27640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb5e0069da0 0x7fb5e006a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:10.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.439+0000 7fb5e4f27640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb5e0145ab0 0x7fb5e0145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:10.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.439+0000 7fb5e4f27640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e015dc60 con 0x7fb5e00c6310 2023-10-16T10:08:10.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.439+0000 7fb5e4f27640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e006a790 con 0x7fb5e0145ab0 2023-10-16T10:08:10.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.439+0000 7fb5e4f27640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e015d370 con 0x7fb5e0069da0 2023-10-16T10:08:10.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.439+0000 7fb5dffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb5e0145ab0 0x7fb5e0145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:10.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.439+0000 7fb5dffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb5e0145ab0 0x7fb5e0145ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57648/0 (socket says 172.21.15.84:57648) 2023-10-16T10:08:10.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.439+0000 7fb5dffff640 1 -- 172.21.15.84:0/2305885962 learned_addr learned my addr 172.21.15.84:0/2305885962 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:10.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.440+0000 7fb5deffd640 1 --2- 172.21.15.84:0/2305885962 >> v2:172.21.15.157:3300/0 conn(0x7fb5e0069da0 0x7fb5e006a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:10.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.440+0000 7fb5dffff640 1 -- 172.21.15.84:0/2305885962 >> v2:172.21.15.157:3300/0 conn(0x7fb5e0069da0 msgr2=0x7fb5e006a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:10.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.440+0000 7fb5dffff640 1 --2- 172.21.15.84:0/2305885962 >> v2:172.21.15.157:3300/0 conn(0x7fb5e0069da0 0x7fb5e006a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.440+0000 7fb5dffff640 1 -- 172.21.15.84:0/2305885962 >> v1:172.21.15.157:6789/0 conn(0x7fb5e00c6310 legacy=0x7fb5e00cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:10.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.440+0000 7fb5dffff640 1 -- 172.21.15.84:0/2305885962 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e00c66f0 con 0x7fb5e0145ab0 2023-10-16T10:08:10.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.440+0000 7fb5deffd640 1 --2- 172.21.15.84:0/2305885962 >> v2:172.21.15.157:3300/0 conn(0x7fb5e0069da0 0x7fb5e006a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:10.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.441+0000 7fb5dffff640 1 --2- 172.21.15.84:0/2305885962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5e0145ab0 0x7fb5e0145ee0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb5e006ae80 tx=0x7fb5d802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb182de974bc5c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:10.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.441+0000 7fb5de7fc640 1 -- 172.21.15.84:0/2305885962 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d802c9f0 con 0x7fb5e0145ab0 2023-10-16T10:08:10.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.442+0000 7fb5de7fc640 1 -- 172.21.15.84:0/2305885962 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d802cb50 con 0x7fb5e0145ab0 2023-10-16T10:08:10.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.442+0000 7fb5de7fc640 1 -- 172.21.15.84:0/2305885962 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d80343f0 con 0x7fb5e0145ab0 2023-10-16T10:08:10.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.442+0000 7fb5e4f27640 1 -- 172.21.15.84:0/2305885962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5e0145ab0 msgr2=0x7fb5e0145ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:10.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.442+0000 7fb5e4f27640 1 --2- 172.21.15.84:0/2305885962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5e0145ab0 0x7fb5e0145ee0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb5e006ae80 tx=0x7fb5d802be50 comp rx=0 tx=0).stop 2023-10-16T10:08:10.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.442+0000 7fb5e4f27640 1 -- 172.21.15.84:0/2305885962 shutdown_connections 2023-10-16T10:08:10.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.442+0000 7fb5e4f27640 1 --2- 172.21.15.84:0/2305885962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5e0145ab0 0x7fb5e0145ee0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.442+0000 7fb5e4f27640 1 --2- 172.21.15.84:0/2305885962 >> v2:172.21.15.157:3300/0 conn(0x7fb5e0069da0 0x7fb5e006a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.442+0000 7fb5e4f27640 1 -- 172.21.15.84:0/2305885962 >> 172.21.15.84:0/2305885962 conn(0x7fb5e00cf910 msgr2=0x7fb5e00cfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:10.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.442+0000 7fb5e4f27640 1 -- 172.21.15.84:0/2305885962 shutdown_connections 2023-10-16T10:08:10.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.442+0000 7fb5e4f27640 1 -- 172.21.15.84:0/2305885962 wait complete. 2023-10-16T10:08:10.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.444+0000 7fb5e4f27640 1 Processor -- start 2023-10-16T10:08:10.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.444+0000 7fb5e4f27640 1 -- start start 2023-10-16T10:08:10.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.445+0000 7fb5e4f27640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5e0069da0 0x7fb5e01e6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:10.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.445+0000 7fb5e4f27640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5e00c6310 0x7fb5e006c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:10.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.445+0000 7fb5e4f27640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5e0145ab0 0x7fb5e006c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:10.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.445+0000 7fb5e4f27640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e00ac880 con 0x7fb5e00c6310 2023-10-16T10:08:10.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.445+0000 7fb5e4f27640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e006d8b0 con 0x7fb5e0145ab0 2023-10-16T10:08:10.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.445+0000 7fb5e4f27640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e006db90 con 0x7fb5e0069da0 2023-10-16T10:08:10.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.445+0000 7fb5deffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5e00c6310 0x7fb5e006c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:10.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.445+0000 7fb5deffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5e00c6310 0x7fb5e006c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57652/0 (socket says 172.21.15.84:57652) 2023-10-16T10:08:10.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.445+0000 7fb5deffd640 1 -- 172.21.15.84:0/1467890200 learned_addr learned my addr 172.21.15.84:0/1467890200 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:10.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.446+0000 7fb5dffff640 1 --2- 172.21.15.84:0/1467890200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5e0145ab0 0x7fb5e006c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:10.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.446+0000 7fb5df7fe640 1 --2- 172.21.15.84:0/1467890200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5e0069da0 0x7fb5e01e6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:10.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.446+0000 7fb5deffd640 1 -- 172.21.15.84:0/1467890200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5e0069da0 msgr2=0x7fb5e01e6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:10.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.446+0000 7fb5deffd640 1 --2- 172.21.15.84:0/1467890200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5e0069da0 0x7fb5e01e6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.446+0000 7fb5deffd640 1 -- 172.21.15.84:0/1467890200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5e0145ab0 msgr2=0x7fb5e006c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:10.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.446+0000 7fb5deffd640 1 --2- 172.21.15.84:0/1467890200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5e0145ab0 0x7fb5e006c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.446+0000 7fb5deffd640 1 -- 172.21.15.84:0/1467890200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d8009b30 con 0x7fb5e00c6310 2023-10-16T10:08:10.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.446+0000 7fb5df7fe640 1 --2- 172.21.15.84:0/1467890200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5e0069da0 0x7fb5e01e6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:10.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.446+0000 7fb5dffff640 1 --2- 172.21.15.84:0/1467890200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5e0145ab0 0x7fb5e006c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:10.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.446+0000 7fb5deffd640 1 --2- 172.21.15.84:0/1467890200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5e00c6310 0x7fb5e006c410 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400c7f0 tx=0x7fb5d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:10.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.446+0000 7fb5c7fff640 1 -- 172.21.15.84:0/1467890200 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4007820 con 0x7fb5e00c6310 2023-10-16T10:08:10.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.446+0000 7fb5c7fff640 1 -- 172.21.15.84:0/1467890200 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d4007980 con 0x7fb5e00c6310 2023-10-16T10:08:10.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.446+0000 7fb5c7fff640 1 -- 172.21.15.84:0/1467890200 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4007cf0 con 0x7fb5e00c6310 2023-10-16T10:08:10.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.446+0000 7fb5e4f27640 1 -- 172.21.15.84:0/1467890200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e00acad0 con 0x7fb5e00c6310 2023-10-16T10:08:10.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.447+0000 7fb5e4f27640 1 -- 172.21.15.84:0/1467890200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e01e6e50 con 0x7fb5e00c6310 2023-10-16T10:08:10.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.448+0000 7fb5e4f27640 1 -- 172.21.15.84:0/1467890200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e01e70d0 con 0x7fb5e00c6310 2023-10-16T10:08:10.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.449+0000 7fb5c7fff640 1 -- 172.21.15.84:0/1467890200 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 68) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fb5d40113f0 con 0x7fb5e00c6310 2023-10-16T10:08:10.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.450+0000 7fb5c7fff640 1 --2- 172.21.15.84:0/1467890200 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb5cc06acc0 0x7fb5cc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:10.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.450+0000 7fb5c7fff640 1 -- 172.21.15.84:0/1467890200 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6539+0+0 (secure 0 0 0) 0x7fb5d4089980 con 0x7fb5e00c6310 2023-10-16T10:08:10.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.450+0000 7fb5df7fe640 1 --2- 172.21.15.84:0/1467890200 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb5cc06acc0 0x7fb5cc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:10.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.451+0000 7fb5df7fe640 1 --2- 172.21.15.84:0/1467890200 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb5cc06acc0 0x7fb5cc06d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb5e01e6270 tx=0x7fb5d0008040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:10.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.454+0000 7fb5c7fff640 1 -- 172.21.15.84:0/1467890200 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb5d4054960 con 0x7fb5e00c6310 2023-10-16T10:08:10.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.693+0000 7fb5e4f27640 1 -- 172.21.15.84:0/1467890200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5e015e0d0 con 0x7fb5e00c6310 2023-10-16T10:08:10.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.693+0000 7fb5c7fff640 1 -- 172.21.15.84:0/1467890200 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fb5e015e0d0 con 0x7fb5e00c6310 2023-10-16T10:08:10.695 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:10.695 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2012273152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2012273152},{"type":"v1","addr":"172.21.15.157:6835","nonce":2012273152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/435606514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":435606514},{"type":"v1","addr":"172.21.15.157:6833","nonce":435606514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2286048798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2286048798},{"type":"v1","addr":"172.21.15.84:6836","nonce":2286048798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:07:28.798746+0000","modified":"2023-10-16T10:07:31.857629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2859088731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2859088731},{"type":"v1","addr":"172.21.15.84:6837","nonce":2859088731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-16T10:08:10.695 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 62 2023-10-16T10:08:10.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.696+0000 7fb5e4f27640 1 -- 172.21.15.84:0/1467890200 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb5cc06acc0 msgr2=0x7fb5cc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:10.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.696+0000 7fb5e4f27640 1 --2- 172.21.15.84:0/1467890200 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb5cc06acc0 0x7fb5cc06d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb5e01e6270 tx=0x7fb5d0008040 comp rx=0 tx=0).stop 2023-10-16T10:08:10.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.696+0000 7fb5e4f27640 1 -- 172.21.15.84:0/1467890200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5e00c6310 msgr2=0x7fb5e006c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:10.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.696+0000 7fb5e4f27640 1 --2- 172.21.15.84:0/1467890200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5e00c6310 0x7fb5e006c410 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400c7f0 tx=0x7fb5d400ccb0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.696+0000 7fb5e4f27640 1 -- 172.21.15.84:0/1467890200 shutdown_connections 2023-10-16T10:08:10.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.696+0000 7fb5e4f27640 1 --2- 172.21.15.84:0/1467890200 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb5cc06acc0 0x7fb5cc06d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.696+0000 7fb5e4f27640 1 --2- 172.21.15.84:0/1467890200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5e0145ab0 0x7fb5e006c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.696+0000 7fb5e4f27640 1 --2- 172.21.15.84:0/1467890200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5e00c6310 0x7fb5e006c410 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.696+0000 7fb5e4f27640 1 --2- 172.21.15.84:0/1467890200 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5e0069da0 0x7fb5e01e6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.696+0000 7fb5e4f27640 1 -- 172.21.15.84:0/1467890200 >> 172.21.15.84:0/1467890200 conn(0x7fb5e00cf910 msgr2=0x7fb5e0143fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:10.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.696+0000 7fb5e4f27640 1 -- 172.21.15.84:0/1467890200 shutdown_connections 2023-10-16T10:08:10.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.696+0000 7fb5e4f27640 1 -- 172.21.15.84:0/1467890200 wait complete. 2023-10-16T10:08:10.713 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:08:10.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.910+0000 7fb492d7c640 1 Processor -- start 2023-10-16T10:08:10.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.910+0000 7fb492d7c640 1 -- start start 2023-10-16T10:08:10.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.912+0000 7fb492d7c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb48c159f00 0x7fb48c0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:10.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.912+0000 7fb492d7c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb48c15d880 0x7fb48c0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:10.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.912+0000 7fb492d7c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48c06f7f0 con 0x7fb48c159b20 2023-10-16T10:08:10.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.912+0000 7fb492d7c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48c06c5e0 con 0x7fb48c159f00 2023-10-16T10:08:10.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.912+0000 7fb492d7c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48c06c8c0 con 0x7fb48c15d880 2023-10-16T10:08:10.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.912+0000 7fb483fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb48c159f00 0x7fb48c0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:10.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.912+0000 7fb483fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb48c159f00 0x7fb48c0b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57686/0 (socket says 172.21.15.84:57686) 2023-10-16T10:08:10.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.912+0000 7fb483fff640 1 -- 172.21.15.84:0/1485085156 learned_addr learned my addr 172.21.15.84:0/1485085156 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:10.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.913+0000 7fb491319640 1 --2- 172.21.15.84:0/1485085156 >> v2:172.21.15.157:3300/0 conn(0x7fb48c15d880 0x7fb48c0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:10.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.913+0000 7fb483fff640 1 -- 172.21.15.84:0/1485085156 >> v2:172.21.15.157:3300/0 conn(0x7fb48c15d880 msgr2=0x7fb48c0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:10.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.913+0000 7fb483fff640 1 --2- 172.21.15.84:0/1485085156 >> v2:172.21.15.157:3300/0 conn(0x7fb48c15d880 0x7fb48c0b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.913+0000 7fb483fff640 1 -- 172.21.15.84:0/1485085156 >> v1:172.21.15.157:6789/0 conn(0x7fb48c159b20 legacy=0x7fb48c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:10.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.913+0000 7fb483fff640 1 -- 172.21.15.84:0/1485085156 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c06cba0 con 0x7fb48c159f00 2023-10-16T10:08:10.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.913+0000 7fb491319640 1 --2- 172.21.15.84:0/1485085156 >> v2:172.21.15.157:3300/0 conn(0x7fb48c15d880 0x7fb48c0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:10.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.914+0000 7fb483fff640 1 --2- 172.21.15.84:0/1485085156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c159f00 0x7fb48c0b0410 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0c8bb0 tx=0x7fb47c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66345ff4583f499c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:10.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.914+0000 7fb4837fe640 1 -- 172.21.15.84:0/1485085156 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c031860 con 0x7fb48c159f00 2023-10-16T10:08:10.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.914+0000 7fb4837fe640 1 -- 172.21.15.84:0/1485085156 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47c0319c0 con 0x7fb48c159f00 2023-10-16T10:08:10.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.914+0000 7fb4837fe640 1 -- 172.21.15.84:0/1485085156 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c031d50 con 0x7fb48c159f00 2023-10-16T10:08:10.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.915+0000 7fb492d7c640 1 -- 172.21.15.84:0/1485085156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c159f00 msgr2=0x7fb48c0b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:10.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.915+0000 7fb492d7c640 1 --2- 172.21.15.84:0/1485085156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c159f00 0x7fb48c0b0410 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0c8bb0 tx=0x7fb47c02b900 comp rx=0 tx=0).stop 2023-10-16T10:08:10.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.915+0000 7fb492d7c640 1 -- 172.21.15.84:0/1485085156 shutdown_connections 2023-10-16T10:08:10.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.915+0000 7fb492d7c640 1 --2- 172.21.15.84:0/1485085156 >> v2:172.21.15.157:3300/0 conn(0x7fb48c15d880 0x7fb48c0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.915+0000 7fb492d7c640 1 --2- 172.21.15.84:0/1485085156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c159f00 0x7fb48c0b0410 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.915+0000 7fb492d7c640 1 -- 172.21.15.84:0/1485085156 >> 172.21.15.84:0/1485085156 conn(0x7fb48c0cb410 msgr2=0x7fb48c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:10.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.915+0000 7fb492d7c640 1 -- 172.21.15.84:0/1485085156 shutdown_connections 2023-10-16T10:08:10.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.915+0000 7fb492d7c640 1 -- 172.21.15.84:0/1485085156 wait complete. 2023-10-16T10:08:10.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.917+0000 7fb492d7c640 1 Processor -- start 2023-10-16T10:08:10.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.918+0000 7fb492d7c640 1 -- start start 2023-10-16T10:08:10.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.919+0000 7fb492d7c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb48c159b20 0x7fb48c14d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:10.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.919+0000 7fb492d7c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c159f00 0x7fb48c154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:10.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.919+0000 7fb492d7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb48c15d880 0x7fb48c13eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:10.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.919+0000 7fb492d7c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c14df50 con 0x7fb48c159f00 2023-10-16T10:08:10.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.919+0000 7fb492d7c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c1553c0 con 0x7fb48c15d880 2023-10-16T10:08:10.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.919+0000 7fb492d7c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0af1b0 con 0x7fb48c159b20 2023-10-16T10:08:10.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.919+0000 7fb483fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c159f00 0x7fb48c154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:10.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.919+0000 7fb483fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c159f00 0x7fb48c154e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57692/0 (socket says 172.21.15.84:57692) 2023-10-16T10:08:10.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.919+0000 7fb483fff640 1 -- 172.21.15.84:0/1067737308 learned_addr learned my addr 172.21.15.84:0/1067737308 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:10.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.919+0000 7fb491319640 1 --2- 172.21.15.84:0/1067737308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb48c15d880 0x7fb48c13eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:10.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.920+0000 7fb490b18640 1 --2- 172.21.15.84:0/1067737308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb48c159b20 0x7fb48c14d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:10.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.920+0000 7fb483fff640 1 -- 172.21.15.84:0/1067737308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb48c159b20 msgr2=0x7fb48c14d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:10.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.920+0000 7fb483fff640 1 --2- 172.21.15.84:0/1067737308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb48c159b20 0x7fb48c14d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.920+0000 7fb483fff640 1 -- 172.21.15.84:0/1067737308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb48c15d880 msgr2=0x7fb48c13eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:10.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.920+0000 7fb483fff640 1 --2- 172.21.15.84:0/1067737308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb48c15d880 0x7fb48c13eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:10.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.920+0000 7fb483fff640 1 -- 172.21.15.84:0/1067737308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47c0094d0 con 0x7fb48c159f00 2023-10-16T10:08:10.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.920+0000 7fb491319640 1 --2- 172.21.15.84:0/1067737308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb48c15d880 0x7fb48c13eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:10.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.920+0000 7fb483fff640 1 --2- 172.21.15.84:0/1067737308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c159f00 0x7fb48c154e80 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fb47c000c00 tx=0x7fb47c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:10.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.920+0000 7fb4817fa640 1 -- 172.21.15.84:0/1067737308 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c031810 con 0x7fb48c159f00 2023-10-16T10:08:10.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.920+0000 7fb4817fa640 1 -- 172.21.15.84:0/1067737308 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47c0443f0 con 0x7fb48c159f00 2023-10-16T10:08:10.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.920+0000 7fb492d7c640 1 -- 172.21.15.84:0/1067737308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c14dc70 con 0x7fb48c159f00 2023-10-16T10:08:10.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.920+0000 7fb4817fa640 1 -- 172.21.15.84:0/1067737308 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c0447a0 con 0x7fb48c159f00 2023-10-16T10:08:10.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.920+0000 7fb492d7c640 1 -- 172.21.15.84:0/1067737308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48c0af730 con 0x7fb48c159f00 2023-10-16T10:08:10.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.922+0000 7fb492d7c640 1 -- 172.21.15.84:0/1067737308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48c06c270 con 0x7fb48c159f00 2023-10-16T10:08:10.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.923+0000 7fb4817fa640 1 -- 172.21.15.84:0/1067737308 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 68) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fb47c04d450 con 0x7fb48c159f00 2023-10-16T10:08:10.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.924+0000 7fb4817fa640 1 --2- 172.21.15.84:0/1067737308 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb47806acc0 0x7fb47806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:10.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.924+0000 7fb4817fa640 1 -- 172.21.15.84:0/1067737308 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6539+0+0 (secure 0 0 0) 0x7fb47c0abc60 con 0x7fb48c159f00 2023-10-16T10:08:10.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.924+0000 7fb490b18640 1 --2- 172.21.15.84:0/1067737308 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb47806acc0 0x7fb47806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:10.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.925+0000 7fb490b18640 1 --2- 172.21.15.84:0/1067737308 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb47806acc0 0x7fb47806d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb48800a730 tx=0x7fb488008040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:10.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:10.928+0000 7fb4817fa640 1 -- 172.21.15.84:0/1067737308 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb47c076d10 con 0x7fb48c159f00 2023-10-16T10:08:11.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.089+0000 7fb4817fa640 1 -- 172.21.15.84:0/1067737308 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 69) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fb47c0770e0 con 0x7fb48c159f00 2023-10-16T10:08:11.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.163+0000 7fb492d7c640 1 -- 172.21.15.84:0/1067737308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb48c04e160 con 0x7fb48c159f00 2023-10-16T10:08:11.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.164+0000 7fb4817fa640 1 -- 172.21.15.84:0/1067737308 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fb47c090df0 con 0x7fb48c159f00 2023-10-16T10:08:11.166 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:11.167 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2012273152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2012273152},{"type":"v1","addr":"172.21.15.157:6835","nonce":2012273152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/435606514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":435606514},{"type":"v1","addr":"172.21.15.157:6833","nonce":435606514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2286048798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2286048798},{"type":"v1","addr":"172.21.15.84:6836","nonce":2286048798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:07:28.798746+0000","modified":"2023-10-16T10:07:31.857629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2859088731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2859088731},{"type":"v1","addr":"172.21.15.84:6837","nonce":2859088731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-16T10:08:11.168 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 62 2023-10-16T10:08:11.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.167+0000 7fb492d7c640 1 -- 172.21.15.84:0/1067737308 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb47806acc0 msgr2=0x7fb47806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:11.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.167+0000 7fb492d7c640 1 --2- 172.21.15.84:0/1067737308 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb47806acc0 0x7fb47806d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb48800a730 tx=0x7fb488008040 comp rx=0 tx=0).stop 2023-10-16T10:08:11.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.167+0000 7fb492d7c640 1 -- 172.21.15.84:0/1067737308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c159f00 msgr2=0x7fb48c154e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:11.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.167+0000 7fb492d7c640 1 --2- 172.21.15.84:0/1067737308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c159f00 0x7fb48c154e80 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fb47c000c00 tx=0x7fb47c0046c0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.167+0000 7fb492d7c640 1 -- 172.21.15.84:0/1067737308 shutdown_connections 2023-10-16T10:08:11.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.167+0000 7fb492d7c640 1 --2- 172.21.15.84:0/1067737308 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb47806acc0 0x7fb47806d180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.167+0000 7fb492d7c640 1 --2- 172.21.15.84:0/1067737308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb48c15d880 0x7fb48c13eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.167+0000 7fb492d7c640 1 --2- 172.21.15.84:0/1067737308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c159f00 0x7fb48c154e80 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.167+0000 7fb492d7c640 1 --2- 172.21.15.84:0/1067737308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb48c159b20 0x7fb48c14d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.167+0000 7fb492d7c640 1 -- 172.21.15.84:0/1067737308 >> 172.21.15.84:0/1067737308 conn(0x7fb48c0cb410 msgr2=0x7fb48c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:11.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.167+0000 7fb492d7c640 1 -- 172.21.15.84:0/1067737308 shutdown_connections 2023-10-16T10:08:11.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.167+0000 7fb492d7c640 1 -- 172.21.15.84:0/1067737308 wait complete. 2023-10-16T10:08:11.184 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:08:11.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.383+0000 7fe3e9147640 1 Processor -- start 2023-10-16T10:08:11.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.384+0000 7fe3e9147640 1 -- start start 2023-10-16T10:08:11.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.385+0000 7fe3e9147640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe3e40ad970 0x7fe3e40cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:11.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.385+0000 7fe3e9147640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3e40ccfb0 0x7fe3e40cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:11.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.385+0000 7fe3e9147640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e415b9d0 con 0x7fe3e406c040 2023-10-16T10:08:11.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.385+0000 7fe3e9147640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e415bcb0 con 0x7fe3e40ad970 2023-10-16T10:08:11.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.385+0000 7fe3e9147640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e415bf90 con 0x7fe3e40ccfb0 2023-10-16T10:08:11.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.385+0000 7fe3e3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe3e406c040 0x7fe3e406c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33160/0 (socket says 172.21.15.84:33160) 2023-10-16T10:08:11.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.385+0000 7fe3e3fff640 1 -- 172.21.15.84:0/361814421 learned_addr learned my addr 172.21.15.84:0/361814421 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:11.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.385+0000 7fe3dbfff640 1 --2- 172.21.15.84:0/361814421 >> v2:172.21.15.84:3300/0 conn(0x7fe3e40ad970 0x7fe3e40cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:11.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.386+0000 7fe3e37fe640 1 -- 172.21.15.84:0/361814421 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758028747 0 0) 0x7fe3e415b9d0 con 0x7fe3e406c040 2023-10-16T10:08:11.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.386+0000 7fe3e37fe640 1 -- 172.21.15.84:0/361814421 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3d0003520 con 0x7fe3e406c040 2023-10-16T10:08:11.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.386+0000 7fe3dbfff640 1 -- 172.21.15.84:0/361814421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3e40ccfb0 msgr2=0x7fe3e40cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:11.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.386+0000 7fe3e8946640 1 --2- 172.21.15.84:0/361814421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3e40ccfb0 0x7fe3e40cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:11.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.386+0000 7fe3dbfff640 1 --2- 172.21.15.84:0/361814421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3e40ccfb0 0x7fe3e40cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.386+0000 7fe3dbfff640 1 -- 172.21.15.84:0/361814421 >> v1:172.21.15.84:6789/0 conn(0x7fe3e406c040 legacy=0x7fe3e406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:11.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.386+0000 7fe3dbfff640 1 -- 172.21.15.84:0/361814421 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e40cc270 con 0x7fe3e40ad970 2023-10-16T10:08:11.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.386+0000 7fe3e8946640 1 --2- 172.21.15.84:0/361814421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3e40ccfb0 0x7fe3e40cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:11.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.387+0000 7fe3dbfff640 1 --2- 172.21.15.84:0/361814421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e40ad970 0x7fe3e40cca70 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0097a0 tx=0x7fe3cc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b0b6afaa935046 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:11.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.387+0000 7fe3e37fe640 1 -- 172.21.15.84:0/361814421 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc02d040 con 0x7fe3e40ad970 2023-10-16T10:08:11.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.387+0000 7fe3e37fe640 1 -- 172.21.15.84:0/361814421 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3cc031810 con 0x7fe3e40ad970 2023-10-16T10:08:11.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.387+0000 7fe3e37fe640 1 -- 172.21.15.84:0/361814421 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc031ba0 con 0x7fe3e40ad970 2023-10-16T10:08:11.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.388+0000 7fe3e9147640 1 -- 172.21.15.84:0/361814421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e40ad970 msgr2=0x7fe3e40cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:11.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.388+0000 7fe3e9147640 1 --2- 172.21.15.84:0/361814421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e40ad970 0x7fe3e40cca70 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0097a0 tx=0x7fe3cc02b8b0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.388+0000 7fe3e9147640 1 -- 172.21.15.84:0/361814421 shutdown_connections 2023-10-16T10:08:11.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.388+0000 7fe3e9147640 1 --2- 172.21.15.84:0/361814421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3e40ccfb0 0x7fe3e40cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.388+0000 7fe3e9147640 1 --2- 172.21.15.84:0/361814421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e40ad970 0x7fe3e40cca70 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.388+0000 7fe3e9147640 1 -- 172.21.15.84:0/361814421 >> 172.21.15.84:0/361814421 conn(0x7fe3e40ce6a0 msgr2=0x7fe3e40ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:11.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.388+0000 7fe3e9147640 1 -- 172.21.15.84:0/361814421 shutdown_connections 2023-10-16T10:08:11.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.388+0000 7fe3e9147640 1 -- 172.21.15.84:0/361814421 wait complete. 2023-10-16T10:08:11.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.390+0000 7fe3e9147640 1 Processor -- start 2023-10-16T10:08:11.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.390+0000 7fe3e9147640 1 -- start start 2023-10-16T10:08:11.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.391+0000 7fe3e9147640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e406c040 0x7fe3e40b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:11.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.391+0000 7fe3e9147640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3e40ad970 0x7fe3e40c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:11.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.391+0000 7fe3e9147640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3e40ccfb0 0x7fe3e4156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:11.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.391+0000 7fe3e9147640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e40c3610 con 0x7fe3e406c040 2023-10-16T10:08:11.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.391+0000 7fe3e9147640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e40c26f0 con 0x7fe3e40ad970 2023-10-16T10:08:11.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.391+0000 7fe3e9147640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e40c2990 con 0x7fe3e40ccfb0 2023-10-16T10:08:11.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.391+0000 7fe3e3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e406c040 0x7fe3e40b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:11.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.391+0000 7fe3e3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e406c040 0x7fe3e40b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57704/0 (socket says 172.21.15.84:57704) 2023-10-16T10:08:11.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.391+0000 7fe3e3fff640 1 -- 172.21.15.84:0/1237036240 learned_addr learned my addr 172.21.15.84:0/1237036240 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:11.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.391+0000 7fe3e3fff640 1 -- 172.21.15.84:0/1237036240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3e40ccfb0 msgr2=0x7fe3e4156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:11.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.391+0000 7fe3e3fff640 1 --2- 172.21.15.84:0/1237036240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3e40ccfb0 0x7fe3e4156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.391+0000 7fe3e3fff640 1 -- 172.21.15.84:0/1237036240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3e40ad970 msgr2=0x7fe3e40c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:11.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.391+0000 7fe3e3fff640 1 --2- 172.21.15.84:0/1237036240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3e40ad970 0x7fe3e40c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.391+0000 7fe3e3fff640 1 -- 172.21.15.84:0/1237036240 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc009400 con 0x7fe3e406c040 2023-10-16T10:08:11.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.392+0000 7fe3e3fff640 1 --2- 172.21.15.84:0/1237036240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e406c040 0x7fe3e40b0530 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400c430 tx=0x7fe3d400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:11.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.392+0000 7fe3e17fa640 1 -- 172.21.15.84:0/1237036240 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4005ce0 con 0x7fe3e406c040 2023-10-16T10:08:11.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.392+0000 7fe3e17fa640 1 -- 172.21.15.84:0/1237036240 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d4005e40 con 0x7fe3e406c040 2023-10-16T10:08:11.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.392+0000 7fe3e9147640 1 -- 172.21.15.84:0/1237036240 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e40c2c70 con 0x7fe3e406c040 2023-10-16T10:08:11.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.392+0000 7fe3e17fa640 1 -- 172.21.15.84:0/1237036240 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d40072f0 con 0x7fe3e406c040 2023-10-16T10:08:11.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.392+0000 7fe3e9147640 1 -- 172.21.15.84:0/1237036240 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e40c3230 con 0x7fe3e406c040 2023-10-16T10:08:11.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.393+0000 7fe3e9147640 1 -- 172.21.15.84:0/1237036240 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e40c33c0 con 0x7fe3e406c040 2023-10-16T10:08:11.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.394+0000 7fe3e17fa640 1 -- 172.21.15.84:0/1237036240 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 69) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fe3d4007450 con 0x7fe3e406c040 2023-10-16T10:08:11.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.395+0000 7fe3e17fa640 1 --2- 172.21.15.84:0/1237036240 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fe3c406af00 0x7fe3c406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:11.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.395+0000 7fe3e17fa640 1 -- 172.21.15.84:0/1237036240 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6539+0+0 (secure 0 0 0) 0x7fe3d4088f70 con 0x7fe3e406c040 2023-10-16T10:08:11.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.395+0000 7fe3dbfff640 1 --2- 172.21.15.84:0/1237036240 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fe3c406af00 0x7fe3c406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:11.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.396+0000 7fe3dbfff640 1 --2- 172.21.15.84:0/1237036240 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fe3c406af00 0x7fe3c406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc036000 tx=0x7fe3cc030000 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:11.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.399+0000 7fe3e17fa640 1 -- 172.21.15.84:0/1237036240 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe3d40bbfe0 con 0x7fe3e406c040 2023-10-16T10:08:11.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.578+0000 7fe3e9147640 1 -- 172.21.15.84:0/1237036240 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3e406b330 con 0x7fe3e406c040 2023-10-16T10:08:11.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.581+0000 7fe3e17fa640 1 -- 172.21.15.84:0/1237036240 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+17730 (secure 0 0 0) 0x7fe3d4017020 con 0x7fe3e406c040 2023-10-16T10:08:11.582 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:11.583 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":127,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:08:09.432278+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:07:26.607199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-16T10:07:27.832706+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":127,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/621277862":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/1967825537":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/2700178020":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/1638353973":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/1637648657":"2023-10-17T10:08:08.425362+0000","172.21.15.84:6833/2209954486":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/3662496273":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/1284479959":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/2038404786":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/688441146":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/644840609":"2023-10-17T10:07:43.574635+0000","172.21.15.84:6832/2209954486":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/3794895754":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/1166553040":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/3269894694":"2023-10-17T10:07:43.574635+0000","172.21.15.84:6833/4292025878":"2023-10-17T10:07:43.574635+0000","172.21.15.84:6832/4292025878":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/1934706185":"2023-10-17T10:07:43.574635+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:08:11.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.584+0000 7fe3e9147640 1 -- 172.21.15.84:0/1237036240 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fe3c406af00 msgr2=0x7fe3c406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:11.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.584+0000 7fe3e9147640 1 --2- 172.21.15.84:0/1237036240 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fe3c406af00 0x7fe3c406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc036000 tx=0x7fe3cc030000 comp rx=0 tx=0).stop 2023-10-16T10:08:11.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.584+0000 7fe3e9147640 1 -- 172.21.15.84:0/1237036240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e406c040 msgr2=0x7fe3e40b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:11.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.584+0000 7fe3e9147640 1 --2- 172.21.15.84:0/1237036240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e406c040 0x7fe3e40b0530 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400c430 tx=0x7fe3d400c8f0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.584+0000 7fe3e9147640 1 -- 172.21.15.84:0/1237036240 shutdown_connections 2023-10-16T10:08:11.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.584+0000 7fe3e9147640 1 --2- 172.21.15.84:0/1237036240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3e40ccfb0 0x7fe3e4156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.584+0000 7fe3e9147640 1 --2- 172.21.15.84:0/1237036240 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fe3c406af00 0x7fe3c406d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.584+0000 7fe3e9147640 1 --2- 172.21.15.84:0/1237036240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3e40ad970 0x7fe3e40c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.584+0000 7fe3e9147640 1 --2- 172.21.15.84:0/1237036240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e406c040 0x7fe3e40b0530 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.584+0000 7fe3e9147640 1 -- 172.21.15.84:0/1237036240 >> 172.21.15.84:0/1237036240 conn(0x7fe3e40ce6a0 msgr2=0x7fe3e40b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:11.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.584+0000 7fe3e9147640 1 -- 172.21.15.84:0/1237036240 shutdown_connections 2023-10-16T10:08:11.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.584+0000 7fe3e9147640 1 -- 172.21.15.84:0/1237036240 wait complete. 2023-10-16T10:08:11.602 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T10:08:11.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.812+0000 7fad07fff640 1 Processor -- start 2023-10-16T10:08:11.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.812+0000 7fad07fff640 1 -- start start 2023-10-16T10:08:11.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.813+0000 7fad07fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fad08159950 0x7fad0815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:11.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.813+0000 7fad07fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0815aef0 con 0x7fad08070450 2023-10-16T10:08:11.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.813+0000 7fad07fff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0815d910 con 0x7fad0806c810 2023-10-16T10:08:11.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.813+0000 7fad07fff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0815e490 con 0x7fad08159950 2023-10-16T10:08:11.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.814+0000 7fad06ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fad08159950 0x7fad0815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:11.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.814+0000 7fad067fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fad08070450 0x7fad0806c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33168/0 (socket says 172.21.15.84:33168) 2023-10-16T10:08:11.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.814+0000 7fad067fc640 1 -- 172.21.15.84:0/3624776506 learned_addr learned my addr 172.21.15.84:0/3624776506 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:11.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.814+0000 7fad06ffd640 1 -- 172.21.15.84:0/3624776506 >> v1:172.21.15.157:6789/0 conn(0x7fad0806c810 legacy=0x7fad080c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:11.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.814+0000 7fad06ffd640 1 -- 172.21.15.84:0/3624776506 >> v1:172.21.15.84:6789/0 conn(0x7fad08070450 legacy=0x7fad0806c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:11.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.814+0000 7fad06ffd640 1 -- 172.21.15.84:0/3624776506 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0815dbf0 con 0x7fad08159950 2023-10-16T10:08:11.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.815+0000 7fad06ffd640 1 --2- 172.21.15.84:0/3624776506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad08159950 0x7fad0815a9b0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7facf80097a0 tx=0x7facf802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1ebe41fa3f891f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:11.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.816+0000 7fad05ffb640 1 -- 172.21.15.84:0/3624776506 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf802e070 con 0x7fad08159950 2023-10-16T10:08:11.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.816+0000 7fad05ffb640 1 -- 172.21.15.84:0/3624776506 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf80315a0 con 0x7fad08159950 2023-10-16T10:08:11.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.816+0000 7fad05ffb640 1 -- 172.21.15.84:0/3624776506 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf8031930 con 0x7fad08159950 2023-10-16T10:08:11.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.816+0000 7fad07fff640 1 -- 172.21.15.84:0/3624776506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad08159950 msgr2=0x7fad0815a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:11.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.816+0000 7fad07fff640 1 --2- 172.21.15.84:0/3624776506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad08159950 0x7fad0815a9b0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7facf80097a0 tx=0x7facf802b840 comp rx=0 tx=0).stop 2023-10-16T10:08:11.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.816+0000 7fad07fff640 1 -- 172.21.15.84:0/3624776506 shutdown_connections 2023-10-16T10:08:11.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.816+0000 7fad07fff640 1 --2- 172.21.15.84:0/3624776506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad08159950 0x7fad0815a9b0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.816+0000 7fad07fff640 1 -- 172.21.15.84:0/3624776506 >> 172.21.15.84:0/3624776506 conn(0x7fad080ce6a0 msgr2=0x7fad080ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:11.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.816+0000 7fad07fff640 1 -- 172.21.15.84:0/3624776506 shutdown_connections 2023-10-16T10:08:11.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.816+0000 7fad07fff640 1 -- 172.21.15.84:0/3624776506 wait complete. 2023-10-16T10:08:11.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.818+0000 7fad07fff640 1 Processor -- start 2023-10-16T10:08:11.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.818+0000 7fad07fff640 1 -- start start 2023-10-16T10:08:11.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.819+0000 7fad07fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad0806c810 0x7fad080c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:11.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.819+0000 7fad07fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad08070450 0x7fad081469e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:11.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.819+0000 7fad07fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad08159950 0x7fad08146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:11.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.819+0000 7fad07fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad081501a0 con 0x7fad08070450 2023-10-16T10:08:11.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.819+0000 7fad07fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0814fb90 con 0x7fad0806c810 2023-10-16T10:08:11.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.819+0000 7fad07fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0815d310 con 0x7fad08159950 2023-10-16T10:08:11.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.819+0000 7fad067fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad08070450 0x7fad081469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:11.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.819+0000 7fad067fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad08070450 0x7fad081469e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57720/0 (socket says 172.21.15.84:57720) 2023-10-16T10:08:11.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.819+0000 7fad067fc640 1 -- 172.21.15.84:0/1361188807 learned_addr learned my addr 172.21.15.84:0/1361188807 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:11.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.820+0000 7fad06ffd640 1 --2- 172.21.15.84:0/1361188807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad0806c810 0x7fad080c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:11.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.820+0000 7fad067fc640 1 -- 172.21.15.84:0/1361188807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad08159950 msgr2=0x7fad08146f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:11.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.820+0000 7fad067fc640 1 --2- 172.21.15.84:0/1361188807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad08159950 0x7fad08146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.820+0000 7fad067fc640 1 -- 172.21.15.84:0/1361188807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad0806c810 msgr2=0x7fad080c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:11.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.820+0000 7fad067fc640 1 --2- 172.21.15.84:0/1361188807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad0806c810 0x7fad080c21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:11.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.820+0000 7fad067fc640 1 -- 172.21.15.84:0/1361188807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf8009400 con 0x7fad08070450 2023-10-16T10:08:11.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.820+0000 7fad067fc640 1 --2- 172.21.15.84:0/1361188807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad08070450 0x7fad081469e0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7facfc00c4a0 tx=0x7facfc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:11.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.820+0000 7facef7fe640 1 -- 172.21.15.84:0/1361188807 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc005ce0 con 0x7fad08070450 2023-10-16T10:08:11.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.820+0000 7facef7fe640 1 -- 172.21.15.84:0/1361188807 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc005e40 con 0x7fad08070450 2023-10-16T10:08:11.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.820+0000 7facef7fe640 1 -- 172.21.15.84:0/1361188807 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc007270 con 0x7fad08070450 2023-10-16T10:08:11.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.820+0000 7fad07fff640 1 -- 172.21.15.84:0/1361188807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0814fe30 con 0x7fad08070450 2023-10-16T10:08:11.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.820+0000 7fad07fff640 1 -- 172.21.15.84:0/1361188807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0815d5f0 con 0x7fad08070450 2023-10-16T10:08:11.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.822+0000 7fad07fff640 1 -- 172.21.15.84:0/1361188807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0804e9b0 con 0x7fad08070450 2023-10-16T10:08:11.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.823+0000 7facef7fe640 1 -- 172.21.15.84:0/1361188807 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 69) v1 ==== 89275+0+0 (secure 0 0 0) 0x7facfc007490 con 0x7fad08070450 2023-10-16T10:08:11.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.824+0000 7facef7fe640 1 --2- 172.21.15.84:0/1361188807 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7facf406af00 0x7facf406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:11.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.824+0000 7facef7fe640 1 -- 172.21.15.84:0/1361188807 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6539+0+0 (secure 0 0 0) 0x7facfc088c70 con 0x7fad08070450 2023-10-16T10:08:11.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.824+0000 7fad06ffd640 1 --2- 172.21.15.84:0/1361188807 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7facf406af00 0x7facf406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:11.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.825+0000 7fad06ffd640 1 --2- 172.21.15.84:0/1361188807 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7facf406af00 0x7facf406d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7facf8036000 tx=0x7facf803d040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:11.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:11.828+0000 7facef7fe640 1 -- 172.21.15.84:0/1361188807 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7facfc053a30 con 0x7fad08070450 2023-10-16T10:08:12.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.055+0000 7fad07fff640 1 -- 172.21.15.84:0/1361188807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fad080b4620 con 0x7fad08070450 2023-10-16T10:08:12.058 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-16T10:08:12.057+0000 7fca0c7ff640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T10:08:12.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.061+0000 7facef7fe640 1 -- 172.21.15.84:0/1361188807 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7facfc017090 con 0x7fad08070450 2023-10-16T10:08:12.062 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T10:08:12.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.064+0000 7fad07fff640 1 -- 172.21.15.84:0/1361188807 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7facf406af00 msgr2=0x7facf406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:12.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.064+0000 7fad07fff640 1 --2- 172.21.15.84:0/1361188807 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7facf406af00 0x7facf406d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7facf8036000 tx=0x7facf803d040 comp rx=0 tx=0).stop 2023-10-16T10:08:12.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.064+0000 7fad07fff640 1 -- 172.21.15.84:0/1361188807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad08070450 msgr2=0x7fad081469e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:12.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.064+0000 7fad07fff640 1 --2- 172.21.15.84:0/1361188807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad08070450 0x7fad081469e0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7facfc00c4a0 tx=0x7facfc00c960 comp rx=0 tx=0).stop 2023-10-16T10:08:12.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.065+0000 7fad07fff640 1 -- 172.21.15.84:0/1361188807 shutdown_connections 2023-10-16T10:08:12.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.065+0000 7fad07fff640 1 --2- 172.21.15.84:0/1361188807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad08159950 0x7fad08146f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:12.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.065+0000 7fad07fff640 1 --2- 172.21.15.84:0/1361188807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad08070450 0x7fad081469e0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:12.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.065+0000 7fad07fff640 1 --2- 172.21.15.84:0/1361188807 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7facf406af00 0x7facf406d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:12.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.065+0000 7fad07fff640 1 --2- 172.21.15.84:0/1361188807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad0806c810 0x7fad080c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:12.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.065+0000 7fad07fff640 1 -- 172.21.15.84:0/1361188807 >> 172.21.15.84:0/1361188807 conn(0x7fad080ce6a0 msgr2=0x7fad0806ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:12.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.066+0000 7fad07fff640 1 -- 172.21.15.84:0/1361188807 shutdown_connections 2023-10-16T10:08:12.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.066+0000 7fad07fff640 1 -- 172.21.15.84:0/1361188807 wait complete. 2023-10-16T10:08:12.088 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T10:08:12.166 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T10:08:12.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.268+0000 7f49fb29f640 1 Processor -- start 2023-10-16T10:08:12.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.268+0000 7f49fb29f640 1 -- start start 2023-10-16T10:08:12.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.269+0000 7f49fb29f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49f40ca0a0 0x7f49f406c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:12.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.269+0000 7f49fb29f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f415d910 con 0x7f49f4159950 2023-10-16T10:08:12.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.269+0000 7f49fb29f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f415dbf0 con 0x7f49f4070b60 2023-10-16T10:08:12.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.269+0000 7f49fb29f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f415e490 con 0x7f49f40ca0a0 2023-10-16T10:08:12.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.270+0000 7f49fa29d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f49f4159950 0x7f49f4070450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33182/0 (socket says 172.21.15.84:33182) 2023-10-16T10:08:12.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.270+0000 7f49fa29d640 1 -- 172.21.15.84:0/3001009596 learned_addr learned my addr 172.21.15.84:0/3001009596 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:12.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.270+0000 7f49f929b640 1 -- 172.21.15.84:0/3001009596 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194497419 0 0) 0x7f49f415d910 con 0x7f49f4159950 2023-10-16T10:08:12.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.270+0000 7f49f929b640 1 -- 172.21.15.84:0/3001009596 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49e0003520 con 0x7f49f4159950 2023-10-16T10:08:12.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.270+0000 7f49faa9e640 1 --2- 172.21.15.84:0/3001009596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49f40ca0a0 0x7f49f406c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:12.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.270+0000 7f49f929b640 1 -- 172.21.15.84:0/3001009596 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 292225640 0 0) 0x7f49e0003520 con 0x7f49f4159950 2023-10-16T10:08:12.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.270+0000 7f49f929b640 1 -- 172.21.15.84:0/3001009596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49f40ca0a0 msgr2=0x7f49f406c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:12.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.270+0000 7f49f929b640 1 --2- 172.21.15.84:0/3001009596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49f40ca0a0 0x7f49f406c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:12.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.270+0000 7f49f929b640 1 -- 172.21.15.84:0/3001009596 >> v1:172.21.15.157:6789/0 conn(0x7f49f4070b60 legacy=0x7f49f406c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:12.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.270+0000 7f49f929b640 1 -- 172.21.15.84:0/3001009596 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f415ded0 con 0x7f49f4159950 2023-10-16T10:08:12.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.270+0000 7f49f929b640 1 -- 172.21.15.84:0/3001009596 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f49f0001670 con 0x7f49f4159950 2023-10-16T10:08:12.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.271+0000 7f49f929b640 1 -- 172.21.15.84:0/3001009596 >> v1:172.21.15.84:6789/0 conn(0x7f49f4159950 legacy=0x7f49f4070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:12.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.271+0000 7f49f929b640 1 --2- 172.21.15.84:0/3001009596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49f40ca0a0 0x7f49e00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:12.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.271+0000 7f49f929b640 1 --2- 172.21.15.84:0/3001009596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49e0004020 0x7f49e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:12.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.271+0000 7f49f929b640 1 --2- 172.21.15.84:0/3001009596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49e0004d00 0x7f49e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:12.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.271+0000 7f49f929b640 1 -- 172.21.15.84:0/3001009596 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f415e490 con 0x7f49e0004d00 2023-10-16T10:08:12.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.271+0000 7f49f929b640 1 -- 172.21.15.84:0/3001009596 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e000bc60 con 0x7f49e0004020 2023-10-16T10:08:12.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.271+0000 7f49f929b640 1 -- 172.21.15.84:0/3001009596 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e000e030 con 0x7f49f40ca0a0 2023-10-16T10:08:12.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.271+0000 7f49faa9e640 1 --2- 172.21.15.84:0/3001009596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49e0004d00 0x7f49e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:12.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.271+0000 7f49f9a9c640 1 --2- 172.21.15.84:0/3001009596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49e0004020 0x7f49e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:12.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.271+0000 7f49fa29d640 1 --2- 172.21.15.84:0/3001009596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49f40ca0a0 0x7f49e00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:12.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.271+0000 7f49faa9e640 1 -- 172.21.15.84:0/3001009596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49f40ca0a0 msgr2=0x7f49e00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:12.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.271+0000 7f49faa9e640 1 --2- 172.21.15.84:0/3001009596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49f40ca0a0 0x7f49e00039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:12.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.271+0000 7f49faa9e640 1 -- 172.21.15.84:0/3001009596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49e0004020 msgr2=0x7f49e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:12.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.271+0000 7f49faa9e640 1 --2- 172.21.15.84:0/3001009596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49e0004020 0x7f49e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:12.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.271+0000 7f49faa9e640 1 -- 172.21.15.84:0/3001009596 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e0003520 con 0x7f49e0004d00 2023-10-16T10:08:12.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.271+0000 7f49f9a9c640 1 --2- 172.21.15.84:0/3001009596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49e0004020 0x7f49e00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:12.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.272+0000 7f49faa9e640 1 --2- 172.21.15.84:0/3001009596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49e0004d00 0x7f49e0005180 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f49ec007360 tx=0x7f49e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719bde6299dc6cc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:12.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.272+0000 7f49f929b640 1 -- 172.21.15.84:0/3001009596 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec02ccb0 con 0x7f49e0004d00 2023-10-16T10:08:12.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.273+0000 7f49f929b640 1 -- 172.21.15.84:0/3001009596 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec02ce10 con 0x7f49e0004d00 2023-10-16T10:08:12.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.273+0000 7f49f929b640 1 -- 172.21.15.84:0/3001009596 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec034660 con 0x7f49e0004d00 2023-10-16T10:08:12.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.273+0000 7f49fb29f640 1 -- 172.21.15.84:0/3001009596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49e0004d00 msgr2=0x7f49e0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:12.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.273+0000 7f49fb29f640 1 --2- 172.21.15.84:0/3001009596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49e0004d00 0x7f49e0005180 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f49ec007360 tx=0x7f49e00057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:12.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.273+0000 7f49faa9e640 1 -- 172.21.15.84:0/3001009596 reap_dead start 2023-10-16T10:08:12.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.273+0000 7f49fb29f640 1 -- 172.21.15.84:0/3001009596 shutdown_connections 2023-10-16T10:08:12.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.273+0000 7f49fb29f640 1 -- 172.21.15.84:0/3001009596 >> 172.21.15.84:0/3001009596 conn(0x7f49f40ce6a0 msgr2=0x7f49f40ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:12.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.273+0000 7f49fb29f640 1 -- 172.21.15.84:0/3001009596 shutdown_connections 2023-10-16T10:08:12.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.273+0000 7f49fb29f640 1 -- 172.21.15.84:0/3001009596 wait complete. 2023-10-16T10:08:12.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.275+0000 7f49fb29f640 1 Processor -- start 2023-10-16T10:08:12.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.275+0000 7f49fb29f640 1 -- start start 2023-10-16T10:08:12.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.276+0000 7f49fb29f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49e0004d00 0x7f49f40c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:12.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.276+0000 7f49fb29f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49f40aeb30 0x7f49f40c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:12.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.276+0000 7f49fb29f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49f414ead0 0x7f49f406b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:12.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.276+0000 7f49fb29f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f40c7e10 con 0x7f49e0004d00 2023-10-16T10:08:12.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.276+0000 7f49fb29f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f40c66c0 con 0x7f49f40aeb30 2023-10-16T10:08:12.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.276+0000 7f49fb29f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f40b24b0 con 0x7f49f414ead0 2023-10-16T10:08:12.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.276+0000 7f49fa29d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49e0004d00 0x7f49f40c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:12.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.276+0000 7f49fa29d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49e0004d00 0x7f49f40c78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57752/0 (socket says 172.21.15.84:57752) 2023-10-16T10:08:12.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.276+0000 7f49fa29d640 1 -- 172.21.15.84:0/2943121507 learned_addr learned my addr 172.21.15.84:0/2943121507 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:12.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.276+0000 7f49fa29d640 1 -- 172.21.15.84:0/2943121507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49f414ead0 msgr2=0x7f49f406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:12.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.276+0000 7f49fa29d640 1 --2- 172.21.15.84:0/2943121507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49f414ead0 0x7f49f406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:12.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.276+0000 7f49fa29d640 1 -- 172.21.15.84:0/2943121507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49f40aeb30 msgr2=0x7f49f40c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:12.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.276+0000 7f49fa29d640 1 --2- 172.21.15.84:0/2943121507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49f40aeb30 0x7f49f40c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:12.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.277+0000 7f49fa29d640 1 -- 172.21.15.84:0/2943121507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f40b2790 con 0x7f49e0004d00 2023-10-16T10:08:12.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.277+0000 7f49fa29d640 1 --2- 172.21.15.84:0/2943121507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49e0004d00 0x7f49f40c78d0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f49f000d770 tx=0x7f49f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:12.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.277+0000 7f49eaffd640 1 -- 172.21.15.84:0/2943121507 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f00074e0 con 0x7f49e0004d00 2023-10-16T10:08:12.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.277+0000 7f49eaffd640 1 -- 172.21.15.84:0/2943121507 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f0007640 con 0x7f49e0004d00 2023-10-16T10:08:12.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.277+0000 7f49fb29f640 1 -- 172.21.15.84:0/2943121507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f40b2a70 con 0x7f49e0004d00 2023-10-16T10:08:12.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.277+0000 7f49eaffd640 1 -- 172.21.15.84:0/2943121507 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f00049e0 con 0x7f49e0004d00 2023-10-16T10:08:12.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.277+0000 7f49fb29f640 1 -- 172.21.15.84:0/2943121507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f414ef50 con 0x7f49e0004d00 2023-10-16T10:08:12.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.278+0000 7f49fb29f640 1 -- 172.21.15.84:0/2943121507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f40b2f90 con 0x7f49e0004d00 2023-10-16T10:08:12.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.279+0000 7f49eaffd640 1 -- 172.21.15.84:0/2943121507 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 69) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f49f0004b40 con 0x7f49e0004d00 2023-10-16T10:08:12.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.280+0000 7f49eaffd640 1 --2- 172.21.15.84:0/2943121507 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f49d806af30 0x7f49d806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:12.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.281+0000 7f49eaffd640 1 -- 172.21.15.84:0/2943121507 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f49f0089820 con 0x7f49e0004d00 2023-10-16T10:08:12.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.281+0000 7f49f9a9c640 1 --2- 172.21.15.84:0/2943121507 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f49d806af30 0x7f49d806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:12.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.281+0000 7f49f9a9c640 1 --2- 172.21.15.84:0/2943121507 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f49d806af30 0x7f49d806d3f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f49f40c5520 tx=0x7f49e4009040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:12.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.285+0000 7f49eaffd640 1 -- 172.21.15.84:0/2943121507 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f49f40b2f90 con 0x7f49e0004d00 2023-10-16T10:08:12.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:12.501+0000 7f49fb29f640 1 -- 172.21.15.84:0/2943121507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f49f40b2f90 con 0x7f49e0004d00 2023-10-16T10:08:13.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.110+0000 7f49eaffd640 1 -- 172.21.15.84:0/2943121507 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) v1 ==== 104+0+0 (secure 0 0 0) 0x7f49f40b2f90 con 0x7f49e0004d00 2023-10-16T10:08:13.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.114+0000 7f49fb29f640 1 -- 172.21.15.84:0/2943121507 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f49d806af30 msgr2=0x7f49d806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:13.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.114+0000 7f49fb29f640 1 --2- 172.21.15.84:0/2943121507 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f49d806af30 0x7f49d806d3f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f49f40c5520 tx=0x7f49e4009040 comp rx=0 tx=0).stop 2023-10-16T10:08:13.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.114+0000 7f49fb29f640 1 -- 172.21.15.84:0/2943121507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49e0004d00 msgr2=0x7f49f40c78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:13.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.114+0000 7f49fb29f640 1 --2- 172.21.15.84:0/2943121507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49e0004d00 0x7f49f40c78d0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f49f000d770 tx=0x7f49f000dc30 comp rx=0 tx=0).stop 2023-10-16T10:08:13.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.114+0000 7f49fb29f640 1 -- 172.21.15.84:0/2943121507 shutdown_connections 2023-10-16T10:08:13.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.114+0000 7f49fb29f640 1 --2- 172.21.15.84:0/2943121507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49f414ead0 0x7f49f406b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:13.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.115+0000 7f49fb29f640 1 --2- 172.21.15.84:0/2943121507 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f49d806af30 0x7f49d806d3f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:13.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.115+0000 7f49fb29f640 1 --2- 172.21.15.84:0/2943121507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49f40aeb30 0x7f49f40c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:13.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.115+0000 7f49fb29f640 1 --2- 172.21.15.84:0/2943121507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49e0004d00 0x7f49f40c78d0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:13.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.115+0000 7f49fb29f640 1 -- 172.21.15.84:0/2943121507 >> 172.21.15.84:0/2943121507 conn(0x7f49f40ce6a0 msgr2=0x7f49f415c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:13.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.115+0000 7f49fb29f640 1 -- 172.21.15.84:0/2943121507 shutdown_connections 2023-10-16T10:08:13.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.115+0000 7f49fb29f640 1 -- 172.21.15.84:0/2943121507 wait complete. 2023-10-16T10:08:13.138 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T10:08:13.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.348+0000 7fd953600640 1 Processor -- start 2023-10-16T10:08:13.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.348+0000 7fd953600640 1 -- start start 2023-10-16T10:08:13.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.349+0000 7fd953600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c1599c0 0x7fd94c0c9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:13.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.350+0000 7fd953600640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd94c15dfb0 con 0x7fd94c0cce20 2023-10-16T10:08:13.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.350+0000 7fd953600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd94c15e290 con 0x7fd94c159da0 2023-10-16T10:08:13.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.350+0000 7fd953600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c15e570 con 0x7fd94c1599c0 2023-10-16T10:08:13.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.350+0000 7fd951b9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd94c0cce20 0x7fd94c06c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33190/0 (socket says 172.21.15.84:33190) 2023-10-16T10:08:13.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.350+0000 7fd951b9d640 1 -- 172.21.15.84:0/142450137 learned_addr learned my addr 172.21.15.84:0/142450137 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:13.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.350+0000 7fd95139c640 1 --2- 172.21.15.84:0/142450137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c1599c0 0x7fd94c0c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:13.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.350+0000 7fd943fff640 1 -- 172.21.15.84:0/142450137 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2683788871 0 0) 0x7fd94c15dfb0 con 0x7fd94c0cce20 2023-10-16T10:08:13.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.350+0000 7fd943fff640 1 -- 172.21.15.84:0/142450137 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd938003520 con 0x7fd94c0cce20 2023-10-16T10:08:13.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.351+0000 7fd943fff640 1 -- 172.21.15.84:0/142450137 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 887865104 0 0) 0x7fd938003520 con 0x7fd94c0cce20 2023-10-16T10:08:13.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.351+0000 7fd943fff640 1 -- 172.21.15.84:0/142450137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c1599c0 msgr2=0x7fd94c0c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:13.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.351+0000 7fd943fff640 1 --2- 172.21.15.84:0/142450137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c1599c0 0x7fd94c0c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:13.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.351+0000 7fd943fff640 1 -- 172.21.15.84:0/142450137 >> v1:172.21.15.157:6789/0 conn(0x7fd94c159da0 legacy=0x7fd94c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:13.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.351+0000 7fd943fff640 1 -- 172.21.15.84:0/142450137 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94c06fda0 con 0x7fd94c0cce20 2023-10-16T10:08:13.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.351+0000 7fd943fff640 1 -- 172.21.15.84:0/142450137 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd944001a10 con 0x7fd94c0cce20 2023-10-16T10:08:13.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.351+0000 7fd943fff640 1 -- 172.21.15.84:0/142450137 >> v1:172.21.15.84:6789/0 conn(0x7fd94c0cce20 legacy=0x7fd94c06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:13.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.351+0000 7fd943fff640 1 --2- 172.21.15.84:0/142450137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd938003a10 0x7fd938003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:13.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.351+0000 7fd943fff640 1 --2- 172.21.15.84:0/142450137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c1599c0 0x7fd9380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:13.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.351+0000 7fd943fff640 1 --2- 172.21.15.84:0/142450137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd938004d00 0x7fd938005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:13.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.351+0000 7fd943fff640 1 -- 172.21.15.84:0/142450137 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c15e570 con 0x7fd938004d00 2023-10-16T10:08:13.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.351+0000 7fd943fff640 1 -- 172.21.15.84:0/142450137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93800bb10 con 0x7fd938003a10 2023-10-16T10:08:13.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.351+0000 7fd943fff640 1 -- 172.21.15.84:0/142450137 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd93800bca0 con 0x7fd94c1599c0 2023-10-16T10:08:13.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.351+0000 7fd951b9d640 1 --2- 172.21.15.84:0/142450137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd938004d00 0x7fd938005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:13.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.352+0000 7fd95139c640 1 --2- 172.21.15.84:0/142450137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd938003a10 0x7fd938003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:13.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.352+0000 7fd950b9b640 1 --2- 172.21.15.84:0/142450137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c1599c0 0x7fd9380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:13.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.352+0000 7fd951b9d640 1 -- 172.21.15.84:0/142450137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c1599c0 msgr2=0x7fd9380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:13.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.352+0000 7fd951b9d640 1 --2- 172.21.15.84:0/142450137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c1599c0 0x7fd9380047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:13.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.352+0000 7fd951b9d640 1 -- 172.21.15.84:0/142450137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd938003a10 msgr2=0x7fd938003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:13.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.352+0000 7fd951b9d640 1 --2- 172.21.15.84:0/142450137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd938003a10 0x7fd938003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:13.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.352+0000 7fd951b9d640 1 -- 172.21.15.84:0/142450137 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd938003520 con 0x7fd938004d00 2023-10-16T10:08:13.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.353+0000 7fd951b9d640 1 --2- 172.21.15.84:0/142450137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd938004d00 0x7fd938005180 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fd94400a7e0 tx=0x7fd9380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797251c79ae6c0a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:13.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.353+0000 7fd943fff640 1 -- 172.21.15.84:0/142450137 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944034630 con 0x7fd938004d00 2023-10-16T10:08:13.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.353+0000 7fd943fff640 1 -- 172.21.15.84:0/142450137 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94402f070 con 0x7fd938004d00 2023-10-16T10:08:13.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.353+0000 7fd943fff640 1 -- 172.21.15.84:0/142450137 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9440349d0 con 0x7fd938004d00 2023-10-16T10:08:13.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.354+0000 7fd953600640 1 -- 172.21.15.84:0/142450137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd938004d00 msgr2=0x7fd938005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:13.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.354+0000 7fd953600640 1 --2- 172.21.15.84:0/142450137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd938004d00 0x7fd938005180 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fd94400a7e0 tx=0x7fd9380057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:13.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.354+0000 7fd951b9d640 1 -- 172.21.15.84:0/142450137 reap_dead start 2023-10-16T10:08:13.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.354+0000 7fd953600640 1 -- 172.21.15.84:0/142450137 shutdown_connections 2023-10-16T10:08:13.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.354+0000 7fd953600640 1 -- 172.21.15.84:0/142450137 >> 172.21.15.84:0/142450137 conn(0x7fd94c0cb360 msgr2=0x7fd94c0cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:13.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.354+0000 7fd953600640 1 -- 172.21.15.84:0/142450137 shutdown_connections 2023-10-16T10:08:13.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.354+0000 7fd953600640 1 -- 172.21.15.84:0/142450137 wait complete. 2023-10-16T10:08:13.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.356+0000 7fd953600640 1 Processor -- start 2023-10-16T10:08:13.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.356+0000 7fd953600640 1 -- start start 2023-10-16T10:08:13.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.357+0000 7fd953600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd938004d00 0x7fd94c0b1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:13.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.357+0000 7fd953600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd94c0acb00 0x7fd94c154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:13.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.357+0000 7fd953600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c0acf80 0x7fd94c1554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:13.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.357+0000 7fd953600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c15cda0 con 0x7fd94c0acb00 2023-10-16T10:08:13.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.357+0000 7fd953600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c15c120 con 0x7fd938004d00 2023-10-16T10:08:13.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.357+0000 7fd953600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c15c400 con 0x7fd94c0acf80 2023-10-16T10:08:13.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.357+0000 7fd950b9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd94c0acb00 0x7fd94c154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:13.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.357+0000 7fd950b9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd94c0acb00 0x7fd94c154fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57764/0 (socket says 172.21.15.84:57764) 2023-10-16T10:08:13.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.357+0000 7fd950b9b640 1 -- 172.21.15.84:0/2332461699 learned_addr learned my addr 172.21.15.84:0/2332461699 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:13.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.357+0000 7fd950b9b640 1 -- 172.21.15.84:0/2332461699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c0acf80 msgr2=0x7fd94c1554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:13.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.357+0000 7fd950b9b640 1 --2- 172.21.15.84:0/2332461699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c0acf80 0x7fd94c1554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:13.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.357+0000 7fd950b9b640 1 -- 172.21.15.84:0/2332461699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd938004d00 msgr2=0x7fd94c0b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:13.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.357+0000 7fd950b9b640 1 --2- 172.21.15.84:0/2332461699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd938004d00 0x7fd94c0b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:13.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.357+0000 7fd950b9b640 1 -- 172.21.15.84:0/2332461699 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94c15c6e0 con 0x7fd94c0acb00 2023-10-16T10:08:13.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.358+0000 7fd950b9b640 1 --2- 172.21.15.84:0/2332461699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd94c0acb00 0x7fd94c154fb0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00d570 tx=0x7fd93c00da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:13.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.358+0000 7fd941ffb640 1 -- 172.21.15.84:0/2332461699 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c003610 con 0x7fd94c0acb00 2023-10-16T10:08:13.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.358+0000 7fd941ffb640 1 -- 172.21.15.84:0/2332461699 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93c003770 con 0x7fd94c0acb00 2023-10-16T10:08:13.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.358+0000 7fd953600640 1 -- 172.21.15.84:0/2332461699 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94c15c9c0 con 0x7fd94c0acb00 2023-10-16T10:08:13.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.358+0000 7fd941ffb640 1 -- 172.21.15.84:0/2332461699 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c003ae0 con 0x7fd94c0acb00 2023-10-16T10:08:13.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.358+0000 7fd953600640 1 -- 172.21.15.84:0/2332461699 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94c155a30 con 0x7fd94c0acb00 2023-10-16T10:08:13.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.359+0000 7fd953600640 1 -- 172.21.15.84:0/2332461699 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd94c15d010 con 0x7fd94c0acb00 2023-10-16T10:08:13.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.360+0000 7fd941ffb640 1 -- 172.21.15.84:0/2332461699 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd93c003c40 con 0x7fd94c0acb00 2023-10-16T10:08:13.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.361+0000 7fd941ffb640 1 --2- 172.21.15.84:0/2332461699 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fd93006af90 0x7fd93006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:13.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.361+0000 7fd941ffb640 1 -- 172.21.15.84:0/2332461699 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fd93c089ac0 con 0x7fd94c0acb00 2023-10-16T10:08:13.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.361+0000 7fd95139c640 1 --2- 172.21.15.84:0/2332461699 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fd93006af90 0x7fd93006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:13.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.362+0000 7fd95139c640 1 --2- 172.21.15.84:0/2332461699 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fd93006af90 0x7fd93006d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd94800ba90 tx=0x7fd948008040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:13.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.366+0000 7fd941ffb640 1 -- 172.21.15.84:0/2332461699 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd93c054680 con 0x7fd94c0acb00 2023-10-16T10:08:13.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:13.557+0000 7fd953600640 1 -- 172.21.15.84:0/2332461699 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd94c04e160 con 0x7fd94c0acb00 2023-10-16T10:08:14.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.113+0000 7fd941ffb640 1 -- 172.21.15.84:0/2332461699 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v129) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd93c057ca0 con 0x7fd94c0acb00 2023-10-16T10:08:14.114 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-16T10:08:14.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.117+0000 7fd953600640 1 -- 172.21.15.84:0/2332461699 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fd93006af90 msgr2=0x7fd93006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:14.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.117+0000 7fd953600640 1 --2- 172.21.15.84:0/2332461699 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fd93006af90 0x7fd93006d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd94800ba90 tx=0x7fd948008040 comp rx=0 tx=0).stop 2023-10-16T10:08:14.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.117+0000 7fd953600640 1 -- 172.21.15.84:0/2332461699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd94c0acb00 msgr2=0x7fd94c154fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:14.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.117+0000 7fd953600640 1 --2- 172.21.15.84:0/2332461699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd94c0acb00 0x7fd94c154fb0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00d570 tx=0x7fd93c00da30 comp rx=0 tx=0).stop 2023-10-16T10:08:14.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.117+0000 7fd953600640 1 -- 172.21.15.84:0/2332461699 shutdown_connections 2023-10-16T10:08:14.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.117+0000 7fd953600640 1 --2- 172.21.15.84:0/2332461699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c0acf80 0x7fd94c1554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:14.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.117+0000 7fd953600640 1 --2- 172.21.15.84:0/2332461699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd94c0acb00 0x7fd94c154fb0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:14.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.117+0000 7fd953600640 1 --2- 172.21.15.84:0/2332461699 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fd93006af90 0x7fd93006d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:14.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.117+0000 7fd953600640 1 --2- 172.21.15.84:0/2332461699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd938004d00 0x7fd94c0b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:14.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.117+0000 7fd953600640 1 -- 172.21.15.84:0/2332461699 >> 172.21.15.84:0/2332461699 conn(0x7fd94c0cb360 msgr2=0x7fd94c072e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:14.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.117+0000 7fd953600640 1 -- 172.21.15.84:0/2332461699 shutdown_connections 2023-10-16T10:08:14.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.118+0000 7fd953600640 1 -- 172.21.15.84:0/2332461699 wait complete. 2023-10-16T10:08:14.145 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T10:08:14.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.345+0000 7fb1eadd5640 1 Processor -- start 2023-10-16T10:08:14.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.345+0000 7fb1eadd5640 1 -- start start 2023-10-16T10:08:14.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.346+0000 7fb1eadd5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40cac20 0x7fb1e40b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:14.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.346+0000 7fb1eadd5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e415b310 con 0x7fb1e415ac10 2023-10-16T10:08:14.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.347+0000 7fb1eadd5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e406f640 con 0x7fb1e40b0360 2023-10-16T10:08:14.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.347+0000 7fb1eadd5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e415f800 con 0x7fb1e40cac20 2023-10-16T10:08:14.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.347+0000 7fb1e8b71640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb1e415ac10 0x7fb1e406c730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33200/0 (socket says 172.21.15.84:33200) 2023-10-16T10:08:14.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.347+0000 7fb1e8b71640 1 -- 172.21.15.84:0/3934520072 learned_addr learned my addr 172.21.15.84:0/3934520072 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:14.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.347+0000 7fb1e37fe640 1 -- 172.21.15.84:0/3934520072 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237441190 0 0) 0x7fb1e415b310 con 0x7fb1e415ac10 2023-10-16T10:08:14.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.347+0000 7fb1e9372640 1 --2- 172.21.15.84:0/3934520072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40cac20 0x7fb1e40b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:14.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.347+0000 7fb1e37fe640 1 -- 172.21.15.84:0/3934520072 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1d4003520 con 0x7fb1e415ac10 2023-10-16T10:08:14.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.348+0000 7fb1e37fe640 1 -- 172.21.15.84:0/3934520072 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2138764281 0 0) 0x7fb1d4003520 con 0x7fb1e415ac10 2023-10-16T10:08:14.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.348+0000 7fb1e37fe640 1 -- 172.21.15.84:0/3934520072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40cac20 msgr2=0x7fb1e40b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:14.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.348+0000 7fb1e37fe640 1 --2- 172.21.15.84:0/3934520072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40cac20 0x7fb1e40b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:14.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.348+0000 7fb1e37fe640 1 -- 172.21.15.84:0/3934520072 >> v1:172.21.15.157:6789/0 conn(0x7fb1e40b0360 legacy=0x7fb1e40ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:14.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.348+0000 7fb1e37fe640 1 -- 172.21.15.84:0/3934520072 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e415b5b0 con 0x7fb1e415ac10 2023-10-16T10:08:14.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.348+0000 7fb1e37fe640 1 -- 172.21.15.84:0/3934520072 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fb1d8001670 con 0x7fb1e415ac10 2023-10-16T10:08:14.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.348+0000 7fb1e37fe640 1 -- 172.21.15.84:0/3934520072 >> v1:172.21.15.84:6789/0 conn(0x7fb1e415ac10 legacy=0x7fb1e406c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:14.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.348+0000 7fb1e37fe640 1 --2- 172.21.15.84:0/3934520072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d4003a10 0x7fb1d4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:14.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.348+0000 7fb1e37fe640 1 --2- 172.21.15.84:0/3934520072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40cac20 0x7fb1d40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:14.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.348+0000 7fb1e37fe640 1 --2- 172.21.15.84:0/3934520072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1d4004d00 0x7fb1d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:14.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.348+0000 7fb1e37fe640 1 -- 172.21.15.84:0/3934520072 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e415f800 con 0x7fb1d4004d00 2023-10-16T10:08:14.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.348+0000 7fb1e37fe640 1 -- 172.21.15.84:0/3934520072 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d400bb60 con 0x7fb1d4003a10 2023-10-16T10:08:14.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.348+0000 7fb1e37fe640 1 -- 172.21.15.84:0/3934520072 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d400bd20 con 0x7fb1e40cac20 2023-10-16T10:08:14.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.348+0000 7fb1e9372640 1 --2- 172.21.15.84:0/3934520072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1d4004d00 0x7fb1d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:14.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.349+0000 7fb1e8b71640 1 --2- 172.21.15.84:0/3934520072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d4003a10 0x7fb1d4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:14.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.349+0000 7fb1e3fff640 1 --2- 172.21.15.84:0/3934520072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40cac20 0x7fb1d40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:14.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.349+0000 7fb1e9372640 1 -- 172.21.15.84:0/3934520072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40cac20 msgr2=0x7fb1d40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:14.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.349+0000 7fb1e9372640 1 --2- 172.21.15.84:0/3934520072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40cac20 0x7fb1d40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:14.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.349+0000 7fb1e9372640 1 -- 172.21.15.84:0/3934520072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d4003a10 msgr2=0x7fb1d4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:14.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.349+0000 7fb1e9372640 1 --2- 172.21.15.84:0/3934520072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d4003a10 0x7fb1d4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:14.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.349+0000 7fb1e9372640 1 -- 172.21.15.84:0/3934520072 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d4003520 con 0x7fb1d4004d00 2023-10-16T10:08:14.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.349+0000 7fb1e3fff640 1 --2- 172.21.15.84:0/3934520072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40cac20 0x7fb1d40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:14.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.350+0000 7fb1e9372640 1 --2- 172.21.15.84:0/3934520072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1d4004d00 0x7fb1d4005180 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009fc0 tx=0x7fb1d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff79aaa91b8a949 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:14.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.350+0000 7fb1e37fe640 1 -- 172.21.15.84:0/3934520072 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc02cbf0 con 0x7fb1d4004d00 2023-10-16T10:08:14.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.350+0000 7fb1e37fe640 1 -- 172.21.15.84:0/3934520072 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1dc00f040 con 0x7fb1d4004d00 2023-10-16T10:08:14.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.350+0000 7fb1e37fe640 1 -- 172.21.15.84:0/3934520072 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc034440 con 0x7fb1d4004d00 2023-10-16T10:08:14.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.351+0000 7fb1eadd5640 1 -- 172.21.15.84:0/3934520072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1d4004d00 msgr2=0x7fb1d4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:14.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.351+0000 7fb1eadd5640 1 --2- 172.21.15.84:0/3934520072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1d4004d00 0x7fb1d4005180 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009fc0 tx=0x7fb1d40057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:14.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.351+0000 7fb1e9372640 1 -- 172.21.15.84:0/3934520072 reap_dead start 2023-10-16T10:08:14.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.351+0000 7fb1eadd5640 1 -- 172.21.15.84:0/3934520072 shutdown_connections 2023-10-16T10:08:14.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.351+0000 7fb1eadd5640 1 -- 172.21.15.84:0/3934520072 >> 172.21.15.84:0/3934520072 conn(0x7fb1e40cb360 msgr2=0x7fb1e40cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:14.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.351+0000 7fb1eadd5640 1 -- 172.21.15.84:0/3934520072 shutdown_connections 2023-10-16T10:08:14.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.351+0000 7fb1eadd5640 1 -- 172.21.15.84:0/3934520072 wait complete. 2023-10-16T10:08:14.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.353+0000 7fb1eadd5640 1 Processor -- start 2023-10-16T10:08:14.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.353+0000 7fb1eadd5640 1 -- start start 2023-10-16T10:08:14.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.354+0000 7fb1eadd5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d4004d00 0x7fb1e40acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:14.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.354+0000 7fb1eadd5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1e40cd840 0x7fb1e40cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:14.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.354+0000 7fb1eadd5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40c6010 0x7fb1e40c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:14.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.354+0000 7fb1eadd5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40cc7f0 con 0x7fb1e40cd840 2023-10-16T10:08:14.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.354+0000 7fb1eadd5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e41f66e0 con 0x7fb1d4004d00 2023-10-16T10:08:14.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.354+0000 7fb1eadd5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e41f69c0 con 0x7fb1e40c6010 2023-10-16T10:08:14.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.354+0000 7fb1e3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1e40cd840 0x7fb1e40cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:14.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.354+0000 7fb1e3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1e40cd840 0x7fb1e40cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57780/0 (socket says 172.21.15.84:57780) 2023-10-16T10:08:14.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.354+0000 7fb1e3fff640 1 -- 172.21.15.84:0/1425851937 learned_addr learned my addr 172.21.15.84:0/1425851937 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:14.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.354+0000 7fb1e8b71640 1 --2- 172.21.15.84:0/1425851937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d4004d00 0x7fb1e40acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:14.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.354+0000 7fb1e3fff640 1 -- 172.21.15.84:0/1425851937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40c6010 msgr2=0x7fb1e40c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:14.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.354+0000 7fb1e3fff640 1 --2- 172.21.15.84:0/1425851937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40c6010 0x7fb1e40c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:14.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.354+0000 7fb1e3fff640 1 -- 172.21.15.84:0/1425851937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d4004d00 msgr2=0x7fb1e40acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:14.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.354+0000 7fb1e3fff640 1 --2- 172.21.15.84:0/1425851937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d4004d00 0x7fb1e40acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:14.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.354+0000 7fb1e3fff640 1 -- 172.21.15.84:0/1425851937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e40ccd60 con 0x7fb1e40cd840 2023-10-16T10:08:14.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.355+0000 7fb1e3fff640 1 --2- 172.21.15.84:0/1425851937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1e40cd840 0x7fb1e40cdcc0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000d7a0 tx=0x7fb1d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:14.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.355+0000 7fb1e17fa640 1 -- 172.21.15.84:0/1425851937 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0017070 con 0x7fb1e40cd840 2023-10-16T10:08:14.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.355+0000 7fb1e17fa640 1 -- 172.21.15.84:0/1425851937 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d0005070 con 0x7fb1e40cd840 2023-10-16T10:08:14.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.355+0000 7fb1e17fa640 1 -- 172.21.15.84:0/1425851937 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0005420 con 0x7fb1e40cd840 2023-10-16T10:08:14.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.355+0000 7fb1eadd5640 1 -- 172.21.15.84:0/1425851937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e40cd040 con 0x7fb1e40cd840 2023-10-16T10:08:14.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.355+0000 7fb1eadd5640 1 -- 172.21.15.84:0/1425851937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e40ad040 con 0x7fb1e40cd840 2023-10-16T10:08:14.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.356+0000 7fb1eadd5640 1 -- 172.21.15.84:0/1425851937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e404e160 con 0x7fb1e40cd840 2023-10-16T10:08:14.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.357+0000 7fb1e17fa640 1 -- 172.21.15.84:0/1425851937 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb1d0007030 con 0x7fb1e40cd840 2023-10-16T10:08:14.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.358+0000 7fb1e17fa640 1 --2- 172.21.15.84:0/1425851937 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb1c806af90 0x7fb1c806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:14.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.358+0000 7fb1e17fa640 1 -- 172.21.15.84:0/1425851937 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fb1d00892a0 con 0x7fb1e40cd840 2023-10-16T10:08:14.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.358+0000 7fb1e8b71640 1 --2- 172.21.15.84:0/1425851937 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb1c806af90 0x7fb1c806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:14.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.359+0000 7fb1e8b71640 1 --2- 172.21.15.84:0/1425851937 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb1c806af90 0x7fb1c806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8001eb0 tx=0x7fb1d8000f50 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:14.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.362+0000 7fb1e17fa640 1 -- 172.21.15.84:0/1425851937 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb1d0053f60 con 0x7fb1e40cd840 2023-10-16T10:08:14.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:14.565+0000 7fb1eadd5640 1 -- 172.21.15.84:0/1425851937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb1e406c2e0 con 0x7fb1e40cd840 2023-10-16T10:08:15.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.126+0000 7fb1e17fa640 1 -- 172.21.15.84:0/1425851937 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v130) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb1d0057580 con 0x7fb1e40cd840 2023-10-16T10:08:15.127 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-16T10:08:15.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.129+0000 7fb1eadd5640 1 -- 172.21.15.84:0/1425851937 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb1c806af90 msgr2=0x7fb1c806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:15.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.129+0000 7fb1eadd5640 1 --2- 172.21.15.84:0/1425851937 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb1c806af90 0x7fb1c806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8001eb0 tx=0x7fb1d8000f50 comp rx=0 tx=0).stop 2023-10-16T10:08:15.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.129+0000 7fb1eadd5640 1 -- 172.21.15.84:0/1425851937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1e40cd840 msgr2=0x7fb1e40cdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:15.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.129+0000 7fb1eadd5640 1 --2- 172.21.15.84:0/1425851937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1e40cd840 0x7fb1e40cdcc0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000d7a0 tx=0x7fb1d000dc60 comp rx=0 tx=0).stop 2023-10-16T10:08:15.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.130+0000 7fb1eadd5640 1 -- 172.21.15.84:0/1425851937 shutdown_connections 2023-10-16T10:08:15.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.130+0000 7fb1eadd5640 1 --2- 172.21.15.84:0/1425851937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1e40c6010 0x7fb1e40c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.130+0000 7fb1eadd5640 1 --2- 172.21.15.84:0/1425851937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1e40cd840 0x7fb1e40cdcc0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.130+0000 7fb1eadd5640 1 --2- 172.21.15.84:0/1425851937 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb1c806af90 0x7fb1c806d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.130+0000 7fb1eadd5640 1 --2- 172.21.15.84:0/1425851937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d4004d00 0x7fb1e40acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.130+0000 7fb1eadd5640 1 -- 172.21.15.84:0/1425851937 >> 172.21.15.84:0/1425851937 conn(0x7fb1e40cb360 msgr2=0x7fb1e406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:15.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.130+0000 7fb1eadd5640 1 -- 172.21.15.84:0/1425851937 shutdown_connections 2023-10-16T10:08:15.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.130+0000 7fb1eadd5640 1 -- 172.21.15.84:0/1425851937 wait complete. 2023-10-16T10:08:15.152 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T10:08:15.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.363+0000 7f6bb359e640 1 Processor -- start 2023-10-16T10:08:15.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.364+0000 7f6bb359e640 1 -- start start 2023-10-16T10:08:15.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.365+0000 7f6bb359e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6bb406c040 0x7f6bb406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:15.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.365+0000 7f6bb359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bb40ad970 0x7f6bb415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:15.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.365+0000 7f6bb359e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb406c960 con 0x7f6bb415ebc0 2023-10-16T10:08:15.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.365+0000 7f6bb359e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb415b9d0 con 0x7f6bb406c040 2023-10-16T10:08:15.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.365+0000 7f6bb359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb415bcb0 con 0x7f6bb40ad970 2023-10-16T10:08:15.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.365+0000 7f6bb259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6bb406c040 0x7f6bb406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:15.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.365+0000 7f6bb259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6bb406c040 0x7f6bb406c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57796/0 (socket says 172.21.15.84:57796) 2023-10-16T10:08:15.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.365+0000 7f6bb259c640 1 -- 172.21.15.84:0/2399407097 learned_addr learned my addr 172.21.15.84:0/2399407097 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:15.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.366+0000 7f6bb259c640 1 -- 172.21.15.84:0/2399407097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bb40ad970 msgr2=0x7f6bb415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:15.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.366+0000 7f6bb259c640 1 --2- 172.21.15.84:0/2399407097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bb40ad970 0x7f6bb415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.366+0000 7f6bb259c640 1 -- 172.21.15.84:0/2399407097 >> v1:172.21.15.157:6789/0 conn(0x7f6bb415ebc0 legacy=0x7f6bb40cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:15.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.366+0000 7f6bb259c640 1 -- 172.21.15.84:0/2399407097 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb415bf90 con 0x7f6bb406c040 2023-10-16T10:08:15.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.367+0000 7f6bb259c640 1 --2- 172.21.15.84:0/2399407097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bb406c040 0x7f6bb406c420 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40098a0 tx=0x7f6ba402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d2e46c03dcb810 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:15.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.367+0000 7f6bb1d9b640 1 -- 172.21.15.84:0/2399407097 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4031640 con 0x7f6bb406c040 2023-10-16T10:08:15.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.367+0000 7f6bb1d9b640 1 -- 172.21.15.84:0/2399407097 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba40317a0 con 0x7f6bb406c040 2023-10-16T10:08:15.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.367+0000 7f6bb1d9b640 1 -- 172.21.15.84:0/2399407097 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4031b30 con 0x7f6bb406c040 2023-10-16T10:08:15.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.368+0000 7f6bb359e640 1 -- 172.21.15.84:0/2399407097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bb406c040 msgr2=0x7f6bb406c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:15.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.368+0000 7f6bb359e640 1 --2- 172.21.15.84:0/2399407097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bb406c040 0x7f6bb406c420 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40098a0 tx=0x7f6ba402b8f0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.368+0000 7f6bb359e640 1 -- 172.21.15.84:0/2399407097 shutdown_connections 2023-10-16T10:08:15.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.368+0000 7f6bb359e640 1 --2- 172.21.15.84:0/2399407097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bb40ad970 0x7f6bb415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.368+0000 7f6bb359e640 1 --2- 172.21.15.84:0/2399407097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bb406c040 0x7f6bb406c420 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.368+0000 7f6bb359e640 1 -- 172.21.15.84:0/2399407097 >> 172.21.15.84:0/2399407097 conn(0x7f6bb40ce6a0 msgr2=0x7f6bb40ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:15.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.368+0000 7f6bb359e640 1 -- 172.21.15.84:0/2399407097 shutdown_connections 2023-10-16T10:08:15.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.368+0000 7f6bb359e640 1 -- 172.21.15.84:0/2399407097 wait complete. 2023-10-16T10:08:15.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.370+0000 7f6bb359e640 1 Processor -- start 2023-10-16T10:08:15.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.370+0000 7f6bb359e640 1 -- start start 2023-10-16T10:08:15.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.371+0000 7f6bb359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bb406c040 0x7f6bb40acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:15.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.371+0000 7f6bb359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bb40ad970 0x7f6bb40ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:15.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.371+0000 7f6bb359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bb415ebc0 0x7f6bb40ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:15.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.371+0000 7f6bb359e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb4153e80 con 0x7f6bb40ad970 2023-10-16T10:08:15.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.371+0000 7f6bb359e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40cacd0 con 0x7f6bb415ebc0 2023-10-16T10:08:15.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.371+0000 7f6bb359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40c8d90 con 0x7f6bb406c040 2023-10-16T10:08:15.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.371+0000 7f6ba9d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bb40ad970 0x7f6bb40ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:15.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.371+0000 7f6ba9d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bb40ad970 0x7f6bb40ca0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57800/0 (socket says 172.21.15.84:57800) 2023-10-16T10:08:15.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.371+0000 7f6ba9d9b640 1 -- 172.21.15.84:0/3512001490 learned_addr learned my addr 172.21.15.84:0/3512001490 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:15.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.372+0000 7f6ba9d9b640 1 -- 172.21.15.84:0/3512001490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bb406c040 msgr2=0x7f6bb40acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:15.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.372+0000 7f6ba9d9b640 1 --2- 172.21.15.84:0/3512001490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bb406c040 0x7f6bb40acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.372+0000 7f6ba9d9b640 1 -- 172.21.15.84:0/3512001490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bb415ebc0 msgr2=0x7f6bb40ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:15.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.372+0000 7f6ba9d9b640 1 --2- 172.21.15.84:0/3512001490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bb415ebc0 0x7f6bb40ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.372+0000 7f6ba9d9b640 1 -- 172.21.15.84:0/3512001490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba40094d0 con 0x7f6bb40ad970 2023-10-16T10:08:15.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.372+0000 7f6ba9d9b640 1 --2- 172.21.15.84:0/3512001490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bb40ad970 0x7f6bb40ca0f0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00b410 tx=0x7f6b9c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:15.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.372+0000 7f6bab7fe640 1 -- 172.21.15.84:0/3512001490 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c011830 con 0x7f6bb40ad970 2023-10-16T10:08:15.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.372+0000 7f6bab7fe640 1 -- 172.21.15.84:0/3512001490 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c0119b0 con 0x7f6bb40ad970 2023-10-16T10:08:15.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.372+0000 7f6bab7fe640 1 -- 172.21.15.84:0/3512001490 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c011d60 con 0x7f6bb40ad970 2023-10-16T10:08:15.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.372+0000 7f6bb359e640 1 -- 172.21.15.84:0/3512001490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb40c9070 con 0x7f6bb40ad970 2023-10-16T10:08:15.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.372+0000 7f6bb359e640 1 -- 172.21.15.84:0/3512001490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb40c9630 con 0x7f6bb40ad970 2023-10-16T10:08:15.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.373+0000 7f6bb359e640 1 -- 172.21.15.84:0/3512001490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb406b330 con 0x7f6bb40ad970 2023-10-16T10:08:15.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.374+0000 7f6bab7fe640 1 -- 172.21.15.84:0/3512001490 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6b9c01a400 con 0x7f6bb40ad970 2023-10-16T10:08:15.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.376+0000 7f6bab7fe640 1 --2- 172.21.15.84:0/3512001490 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f6b9406af90 0x7f6b9406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:15.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.376+0000 7f6bab7fe640 1 -- 172.21.15.84:0/3512001490 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5745+0+0 (secure 0 0 0) 0x7f6b9c089a10 con 0x7f6bb40ad970 2023-10-16T10:08:15.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.376+0000 7f6bb259c640 1 --2- 172.21.15.84:0/3512001490 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f6b9406af90 0x7f6b9406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:15.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.377+0000 7f6bb259c640 1 --2- 172.21.15.84:0/3512001490 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f6b9406af90 0x7f6b9406d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4036000 tx=0x7f6ba4030000 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:15.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.380+0000 7f6bab7fe640 1 -- 172.21.15.84:0/3512001490 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6b9c0548e0 con 0x7f6bb40ad970 2023-10-16T10:08:15.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.571+0000 7f6bb359e640 1 -- 172.21.15.84:0/3512001490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6bb40c9850 con 0x7f6bb40ad970 2023-10-16T10:08:15.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.577+0000 7f6bab7fe640 1 -- 172.21.15.84:0/3512001490 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6b9c0150a0 con 0x7f6bb40ad970 2023-10-16T10:08:15.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.580+0000 7f6bb359e640 1 -- 172.21.15.84:0/3512001490 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f6b9406af90 msgr2=0x7f6b9406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:15.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.580+0000 7f6bb359e640 1 --2- 172.21.15.84:0/3512001490 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f6b9406af90 0x7f6b9406d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4036000 tx=0x7f6ba4030000 comp rx=0 tx=0).stop 2023-10-16T10:08:15.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.580+0000 7f6bb359e640 1 -- 172.21.15.84:0/3512001490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bb40ad970 msgr2=0x7f6bb40ca0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:15.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.580+0000 7f6bb359e640 1 --2- 172.21.15.84:0/3512001490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bb40ad970 0x7f6bb40ca0f0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00b410 tx=0x7f6b9c00b8d0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.580+0000 7f6bb359e640 1 -- 172.21.15.84:0/3512001490 shutdown_connections 2023-10-16T10:08:15.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.580+0000 7f6bb359e640 1 --2- 172.21.15.84:0/3512001490 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f6b9406af90 0x7f6b9406d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.580+0000 7f6bb359e640 1 --2- 172.21.15.84:0/3512001490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bb415ebc0 0x7f6bb40ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.580+0000 7f6bb359e640 1 --2- 172.21.15.84:0/3512001490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bb40ad970 0x7f6bb40ca0f0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.580+0000 7f6bb359e640 1 --2- 172.21.15.84:0/3512001490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bb406c040 0x7f6bb40acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.580+0000 7f6bb359e640 1 -- 172.21.15.84:0/3512001490 >> 172.21.15.84:0/3512001490 conn(0x7f6bb40ce6a0 msgr2=0x7f6bb415d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:15.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.580+0000 7f6bb359e640 1 -- 172.21.15.84:0/3512001490 shutdown_connections 2023-10-16T10:08:15.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.580+0000 7f6bb359e640 1 -- 172.21.15.84:0/3512001490 wait complete. 2023-10-16T10:08:15.599 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T10:08:15.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.812+0000 7facb1e44640 1 Processor -- start 2023-10-16T10:08:15.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.812+0000 7facb1e44640 1 -- start start 2023-10-16T10:08:15.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.814+0000 7facb1e44640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7facac15ad70 0x7facac069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:15.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.814+0000 7facb1e44640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac0caac0 0x7facac06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:15.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.814+0000 7facb1e44640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facac0cb030 con 0x7facac0c9ca0 2023-10-16T10:08:15.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.814+0000 7facb1e44640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facac0c91c0 con 0x7facac15ad70 2023-10-16T10:08:15.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.814+0000 7facb1e44640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facac0c94a0 con 0x7facac0caac0 2023-10-16T10:08:15.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.814+0000 7facab7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7facac15ad70 0x7facac069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:15.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.814+0000 7faca2ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7facac0c9ca0 0x7facac0ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:41514/0 (socket says 172.21.15.84:41514) 2023-10-16T10:08:15.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.814+0000 7faca2ffd640 1 -- 172.21.15.84:0/3774338570 learned_addr learned my addr 172.21.15.84:0/3774338570 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:15.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.815+0000 7facabfff640 1 --2- 172.21.15.84:0/3774338570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac0caac0 0x7facac06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:15.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.815+0000 7facab7fe640 1 -- 172.21.15.84:0/3774338570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac0caac0 msgr2=0x7facac06a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:15.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.815+0000 7facab7fe640 1 --2- 172.21.15.84:0/3774338570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac0caac0 0x7facac06a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.815+0000 7facab7fe640 1 -- 172.21.15.84:0/3774338570 >> v1:172.21.15.84:6789/0 conn(0x7facac0c9ca0 legacy=0x7facac0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:15.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.815+0000 7facab7fe640 1 -- 172.21.15.84:0/3774338570 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac0c9780 con 0x7facac15ad70 2023-10-16T10:08:15.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.815+0000 7facaaffd640 1 -- 172.21.15.84:0/3774338570 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4219712330 0 0) 0x7facac0cb030 con 0x7facac0c9ca0 2023-10-16T10:08:15.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.816+0000 7facab7fe640 1 --2- 172.21.15.84:0/3774338570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac15ad70 0x7facac069da0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009d90 tx=0x7fac9c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2a4c77735ea3fb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:15.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.816+0000 7facaaffd640 1 -- 172.21.15.84:0/3774338570 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c02cc50 con 0x7facac15ad70 2023-10-16T10:08:15.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.816+0000 7facaaffd640 1 -- 172.21.15.84:0/3774338570 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c02cdb0 con 0x7facac15ad70 2023-10-16T10:08:15.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.816+0000 7facaaffd640 1 -- 172.21.15.84:0/3774338570 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c034660 con 0x7facac15ad70 2023-10-16T10:08:15.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.817+0000 7facb1e44640 1 -- 172.21.15.84:0/3774338570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac15ad70 msgr2=0x7facac069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:15.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.817+0000 7facb1e44640 1 --2- 172.21.15.84:0/3774338570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac15ad70 0x7facac069da0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009d90 tx=0x7fac9c02bea0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.817+0000 7facb1e44640 1 -- 172.21.15.84:0/3774338570 shutdown_connections 2023-10-16T10:08:15.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.817+0000 7facb1e44640 1 --2- 172.21.15.84:0/3774338570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac0caac0 0x7facac06a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.817+0000 7facb1e44640 1 --2- 172.21.15.84:0/3774338570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac15ad70 0x7facac069da0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.817+0000 7facb1e44640 1 -- 172.21.15.84:0/3774338570 >> 172.21.15.84:0/3774338570 conn(0x7facac0ce9b0 msgr2=0x7facac0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:15.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.817+0000 7facb1e44640 1 -- 172.21.15.84:0/3774338570 shutdown_connections 2023-10-16T10:08:15.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.817+0000 7facb1e44640 1 -- 172.21.15.84:0/3774338570 wait complete. 2023-10-16T10:08:15.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.819+0000 7facb1e44640 1 Processor -- start 2023-10-16T10:08:15.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.819+0000 7facb1e44640 1 -- start start 2023-10-16T10:08:15.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.820+0000 7facb1e44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac0c9ca0 0x7facac0c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:15.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.820+0000 7facb1e44640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac0caac0 0x7facac156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:15.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.820+0000 7facb1e44640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facac15ad70 0x7facac0c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:15.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.820+0000 7facb1e44640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac0c7560 con 0x7facac0c9ca0 2023-10-16T10:08:15.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.820+0000 7facb1e44640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac0c6ca0 con 0x7facac15ad70 2023-10-16T10:08:15.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.820+0000 7facb1e44640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facac0c4770 con 0x7facac0caac0 2023-10-16T10:08:15.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.820+0000 7facab7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac0c9ca0 0x7facac0c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:15.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.820+0000 7facab7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac0c9ca0 0x7facac0c3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44278/0 (socket says 172.21.15.84:44278) 2023-10-16T10:08:15.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.820+0000 7facab7fe640 1 -- 172.21.15.84:0/505882047 learned_addr learned my addr 172.21.15.84:0/505882047 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:15.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.821+0000 7faca2ffd640 1 --2- 172.21.15.84:0/505882047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac0caac0 0x7facac156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:15.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.821+0000 7facab7fe640 1 -- 172.21.15.84:0/505882047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac0caac0 msgr2=0x7facac156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:15.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.821+0000 7facabfff640 1 --2- 172.21.15.84:0/505882047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facac15ad70 0x7facac0c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:15.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.821+0000 7facab7fe640 1 --2- 172.21.15.84:0/505882047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac0caac0 0x7facac156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.821+0000 7facab7fe640 1 -- 172.21.15.84:0/505882047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facac15ad70 msgr2=0x7facac0c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:15.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.821+0000 7facab7fe640 1 --2- 172.21.15.84:0/505882047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facac15ad70 0x7facac0c7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:15.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.821+0000 7facab7fe640 1 -- 172.21.15.84:0/505882047 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c009a70 con 0x7facac0c9ca0 2023-10-16T10:08:15.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.821+0000 7facab7fe640 1 --2- 172.21.15.84:0/505882047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac0c9ca0 0x7facac0c3ef0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009d90 tx=0x7fac9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:15.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.821+0000 7faca8ff9640 1 -- 172.21.15.84:0/505882047 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c040650 con 0x7facac0c9ca0 2023-10-16T10:08:15.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.821+0000 7faca8ff9640 1 -- 172.21.15.84:0/505882047 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c0407b0 con 0x7facac0c9ca0 2023-10-16T10:08:15.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.821+0000 7facb1e44640 1 -- 172.21.15.84:0/505882047 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facac0c6ea0 con 0x7facac0c9ca0 2023-10-16T10:08:15.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.822+0000 7faca8ff9640 1 -- 172.21.15.84:0/505882047 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c040b60 con 0x7facac0c9ca0 2023-10-16T10:08:15.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.822+0000 7facb1e44640 1 -- 172.21.15.84:0/505882047 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facac0c4430 con 0x7facac0c9ca0 2023-10-16T10:08:15.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.823+0000 7facb1e44640 1 -- 172.21.15.84:0/505882047 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facac04e160 con 0x7facac0c9ca0 2023-10-16T10:08:15.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.824+0000 7faca8ff9640 1 -- 172.21.15.84:0/505882047 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fac9c040cc0 con 0x7facac0c9ca0 2023-10-16T10:08:15.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.824+0000 7faca8ff9640 1 --2- 172.21.15.84:0/505882047 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fac8c06afc0 0x7fac8c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:15.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.825+0000 7faca8ff9640 1 -- 172.21.15.84:0/505882047 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5745+0+0 (secure 0 0 0) 0x7fac9c0ac670 con 0x7facac0c9ca0 2023-10-16T10:08:15.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.825+0000 7faca2ffd640 1 --2- 172.21.15.84:0/505882047 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fac8c06afc0 0x7fac8c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:15.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.825+0000 7faca2ffd640 1 --2- 172.21.15.84:0/505882047 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fac8c06afc0 0x7fac8c06d480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fac94007fd0 tx=0x7fac94009040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:15.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:15.829+0000 7faca8ff9640 1 -- 172.21.15.84:0/505882047 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fac9c077540 con 0x7facac0c9ca0 2023-10-16T10:08:16.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.032+0000 7facb1e44640 1 -- 172.21.15.84:0/505882047 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7facac0ca880 con 0x7facac0c9ca0 2023-10-16T10:08:16.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.039+0000 7faca8ff9640 1 -- 172.21.15.84:0/505882047 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v44) v1 ==== 120+0+0 (secure 0 0 0) 0x7fac9c07ab60 con 0x7facac0c9ca0 2023-10-16T10:08:16.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.042+0000 7facb1e44640 1 -- 172.21.15.84:0/505882047 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fac8c06afc0 msgr2=0x7fac8c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:16.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.042+0000 7facb1e44640 1 --2- 172.21.15.84:0/505882047 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fac8c06afc0 0x7fac8c06d480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fac94007fd0 tx=0x7fac94009040 comp rx=0 tx=0).stop 2023-10-16T10:08:16.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.042+0000 7facb1e44640 1 -- 172.21.15.84:0/505882047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac0c9ca0 msgr2=0x7facac0c3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:16.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.042+0000 7facb1e44640 1 --2- 172.21.15.84:0/505882047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac0c9ca0 0x7facac0c3ef0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009d90 tx=0x7fac9c004a40 comp rx=0 tx=0).stop 2023-10-16T10:08:16.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.042+0000 7facb1e44640 1 -- 172.21.15.84:0/505882047 shutdown_connections 2023-10-16T10:08:16.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.042+0000 7facb1e44640 1 --2- 172.21.15.84:0/505882047 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fac8c06afc0 0x7fac8c06d480 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:16.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.042+0000 7facb1e44640 1 --2- 172.21.15.84:0/505882047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facac15ad70 0x7facac0c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:16.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.042+0000 7facb1e44640 1 --2- 172.21.15.84:0/505882047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facac0caac0 0x7facac156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:16.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.042+0000 7facb1e44640 1 --2- 172.21.15.84:0/505882047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac0c9ca0 0x7facac0c3ef0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:16.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.042+0000 7facb1e44640 1 -- 172.21.15.84:0/505882047 >> 172.21.15.84:0/505882047 conn(0x7facac0ce9b0 msgr2=0x7facac069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:16.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.042+0000 7facb1e44640 1 -- 172.21.15.84:0/505882047 shutdown_connections 2023-10-16T10:08:16.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.042+0000 7facb1e44640 1 -- 172.21.15.84:0/505882047 wait complete. 2023-10-16T10:08:16.063 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T10:08:16.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.274+0000 7f0f018da640 1 Processor -- start 2023-10-16T10:08:16.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.274+0000 7f0f018da640 1 -- start start 2023-10-16T10:08:16.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.276+0000 7f0f018da640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0efc15ae80 0x7f0efc15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:16.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.276+0000 7f0f018da640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0efc0707e0 con 0x7f0efc15aaa0 2023-10-16T10:08:16.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.276+0000 7f0f018da640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0efc06b7a0 con 0x7f0efc0ce390 2023-10-16T10:08:16.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.276+0000 7f0f018da640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0efc15ef20 con 0x7f0efc15ae80 2023-10-16T10:08:16.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.276+0000 7f0ef27fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0efc15ae80 0x7f0efc15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:16.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.276+0000 7f0efaffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0efc15aaa0 0x7f0efc0cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:41518/0 (socket says 172.21.15.84:41518) 2023-10-16T10:08:16.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.276+0000 7f0efaffd640 1 -- 172.21.15.84:0/4066823778 learned_addr learned my addr 172.21.15.84:0/4066823778 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:16.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.277+0000 7f0efa7fc640 1 -- 172.21.15.84:0/4066823778 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154203799 0 0) 0x7f0efc0707e0 con 0x7f0efc15aaa0 2023-10-16T10:08:16.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.277+0000 7f0ef27fc640 1 -- 172.21.15.84:0/4066823778 >> v1:172.21.15.157:6789/0 conn(0x7f0efc0ce390 legacy=0x7f0efc070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:16.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.277+0000 7f0ef27fc640 1 -- 172.21.15.84:0/4066823778 >> v1:172.21.15.84:6789/0 conn(0x7f0efc15aaa0 legacy=0x7f0efc0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:16.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.277+0000 7f0ef27fc640 1 -- 172.21.15.84:0/4066823778 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0efc15f200 con 0x7f0efc15ae80 2023-10-16T10:08:16.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.278+0000 7f0ef27fc640 1 --2- 172.21.15.84:0/4066823778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0efc15ae80 0x7f0efc15e820 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40097a0 tx=0x7f0ee402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e78eec2114d6de server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:16.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.278+0000 7f0efa7fc640 1 -- 172.21.15.84:0/4066823778 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee402e070 con 0x7f0efc15ae80 2023-10-16T10:08:16.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.278+0000 7f0efa7fc640 1 -- 172.21.15.84:0/4066823778 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee40315a0 con 0x7f0efc15ae80 2023-10-16T10:08:16.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.278+0000 7f0efa7fc640 1 -- 172.21.15.84:0/4066823778 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4031930 con 0x7f0efc15ae80 2023-10-16T10:08:16.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.279+0000 7f0f018da640 1 -- 172.21.15.84:0/4066823778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0efc15ae80 msgr2=0x7f0efc15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:16.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.279+0000 7f0f018da640 1 --2- 172.21.15.84:0/4066823778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0efc15ae80 0x7f0efc15e820 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40097a0 tx=0x7f0ee402b840 comp rx=0 tx=0).stop 2023-10-16T10:08:16.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.279+0000 7f0f018da640 1 -- 172.21.15.84:0/4066823778 shutdown_connections 2023-10-16T10:08:16.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.279+0000 7f0f018da640 1 --2- 172.21.15.84:0/4066823778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0efc15ae80 0x7f0efc15e820 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:16.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.279+0000 7f0f018da640 1 -- 172.21.15.84:0/4066823778 >> 172.21.15.84:0/4066823778 conn(0x7f0efc0cb410 msgr2=0x7f0efc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:16.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.279+0000 7f0f018da640 1 -- 172.21.15.84:0/4066823778 shutdown_connections 2023-10-16T10:08:16.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.279+0000 7f0f018da640 1 -- 172.21.15.84:0/4066823778 wait complete. 2023-10-16T10:08:16.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.281+0000 7f0f018da640 1 Processor -- start 2023-10-16T10:08:16.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.281+0000 7f0f018da640 1 -- start start 2023-10-16T10:08:16.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.282+0000 7f0f018da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0efc0ce390 0x7f0efc158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:16.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.282+0000 7f0f018da640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0efc15aaa0 0x7f0efc15c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:16.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.282+0000 7f0f018da640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0efc15ae80 0x7f0efc06cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:16.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.282+0000 7f0f018da640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc0c25f0 con 0x7f0efc15ae80 2023-10-16T10:08:16.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.282+0000 7f0f018da640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc15cf80 con 0x7f0efc0ce390 2023-10-16T10:08:16.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.282+0000 7f0f018da640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc15d260 con 0x7f0efc15aaa0 2023-10-16T10:08:16.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.282+0000 7f0efb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0efc15ae80 0x7f0efc06cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:16.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.282+0000 7f0efb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0efc15ae80 0x7f0efc06cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44296/0 (socket says 172.21.15.84:44296) 2023-10-16T10:08:16.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.282+0000 7f0efb7fe640 1 -- 172.21.15.84:0/1759425446 learned_addr learned my addr 172.21.15.84:0/1759425446 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:16.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.283+0000 7f0efaffd640 1 --2- 172.21.15.84:0/1759425446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0efc0ce390 0x7f0efc158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:16.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.283+0000 7f0efb7fe640 1 -- 172.21.15.84:0/1759425446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0efc15aaa0 msgr2=0x7f0efc15c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:16.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.283+0000 7f0efb7fe640 1 --2- 172.21.15.84:0/1759425446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0efc15aaa0 0x7f0efc15c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:16.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.283+0000 7f0efb7fe640 1 -- 172.21.15.84:0/1759425446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0efc0ce390 msgr2=0x7f0efc158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:16.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.283+0000 7f0efb7fe640 1 --2- 172.21.15.84:0/1759425446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0efc0ce390 0x7f0efc158030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:16.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.283+0000 7f0efb7fe640 1 -- 172.21.15.84:0/1759425446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee4009400 con 0x7f0efc15ae80 2023-10-16T10:08:16.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.283+0000 7f0efb7fe640 1 --2- 172.21.15.84:0/1759425446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0efc15ae80 0x7f0efc06cc80 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400c780 tx=0x7f0ef400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:16.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.283+0000 7f0ef8ff9640 1 -- 172.21.15.84:0/1759425446 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4004b60 con 0x7f0efc15ae80 2023-10-16T10:08:16.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.283+0000 7f0ef8ff9640 1 -- 172.21.15.84:0/1759425446 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef400ce90 con 0x7f0efc15ae80 2023-10-16T10:08:16.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.283+0000 7f0ef8ff9640 1 -- 172.21.15.84:0/1759425446 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef400a850 con 0x7f0efc15ae80 2023-10-16T10:08:16.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.283+0000 7f0f018da640 1 -- 172.21.15.84:0/1759425446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0efc15d540 con 0x7f0efc15ae80 2023-10-16T10:08:16.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.283+0000 7f0f018da640 1 -- 172.21.15.84:0/1759425446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0efc0b2aa0 con 0x7f0efc15ae80 2023-10-16T10:08:16.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.284+0000 7f0f018da640 1 -- 172.21.15.84:0/1759425446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0efc0c2760 con 0x7f0efc15ae80 2023-10-16T10:08:16.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.286+0000 7f0ef8ff9640 1 -- 172.21.15.84:0/1759425446 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0ef400a9b0 con 0x7f0efc15ae80 2023-10-16T10:08:16.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.286+0000 7f0ef8ff9640 1 --2- 172.21.15.84:0/1759425446 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f0edc06af90 0x7f0edc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:16.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.287+0000 7f0ef8ff9640 1 -- 172.21.15.84:0/1759425446 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5745+0+0 (secure 0 0 0) 0x7f0ef4089a50 con 0x7f0efc15ae80 2023-10-16T10:08:16.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.287+0000 7f0efaffd640 1 --2- 172.21.15.84:0/1759425446 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f0edc06af90 0x7f0edc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:16.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.288+0000 7f0efaffd640 1 --2- 172.21.15.84:0/1759425446 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f0edc06af90 0x7f0edc06d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0eec007fd0 tx=0x7f0eec009040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:16.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.291+0000 7f0ef8ff9640 1 -- 172.21.15.84:0/1759425446 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0ef40549b0 con 0x7f0efc15ae80 2023-10-16T10:08:16.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.489+0000 7f0f018da640 1 -- 172.21.15.84:0/1759425446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f0efc0b2cf0 con 0x7f0efc15ae80 2023-10-16T10:08:16.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.496+0000 7f0ef8ff9640 1 -- 172.21.15.84:0/1759425446 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v45) v1 ==== 115+0+0 (secure 0 0 0) 0x7f0ef4057fd0 con 0x7f0efc15ae80 2023-10-16T10:08:16.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.499+0000 7f0f018da640 1 -- 172.21.15.84:0/1759425446 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f0edc06af90 msgr2=0x7f0edc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:16.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.499+0000 7f0f018da640 1 --2- 172.21.15.84:0/1759425446 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f0edc06af90 0x7f0edc06d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0eec007fd0 tx=0x7f0eec009040 comp rx=0 tx=0).stop 2023-10-16T10:08:16.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.499+0000 7f0f018da640 1 -- 172.21.15.84:0/1759425446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0efc15ae80 msgr2=0x7f0efc06cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:16.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.499+0000 7f0f018da640 1 --2- 172.21.15.84:0/1759425446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0efc15ae80 0x7f0efc06cc80 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400c780 tx=0x7f0ef400cc40 comp rx=0 tx=0).stop 2023-10-16T10:08:16.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.500+0000 7f0f018da640 1 -- 172.21.15.84:0/1759425446 shutdown_connections 2023-10-16T10:08:16.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.500+0000 7f0f018da640 1 --2- 172.21.15.84:0/1759425446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0efc15ae80 0x7f0efc06cc80 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:16.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.500+0000 7f0f018da640 1 --2- 172.21.15.84:0/1759425446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0efc15aaa0 0x7f0efc15c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:16.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.500+0000 7f0f018da640 1 --2- 172.21.15.84:0/1759425446 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f0edc06af90 0x7f0edc06d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:16.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.500+0000 7f0f018da640 1 --2- 172.21.15.84:0/1759425446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0efc0ce390 0x7f0efc158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:16.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.500+0000 7f0f018da640 1 -- 172.21.15.84:0/1759425446 >> 172.21.15.84:0/1759425446 conn(0x7f0efc0cb410 msgr2=0x7f0efc158a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:16.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.500+0000 7f0f018da640 1 -- 172.21.15.84:0/1759425446 shutdown_connections 2023-10-16T10:08:16.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.500+0000 7f0f018da640 1 -- 172.21.15.84:0/1759425446 wait complete. 2023-10-16T10:08:16.520 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-10-16T10:08:16.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.734+0000 7f8a55056640 1 Processor -- start 2023-10-16T10:08:16.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.734+0000 7f8a55056640 1 -- start start 2023-10-16T10:08:16.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.736+0000 7f8a55056640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8a5015ae80 0x7f8a5015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:16.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.736+0000 7f8a55056640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8a500ce390 0x7f8a50070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:16.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.736+0000 7f8a55056640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a5015ed60 con 0x7f8a5015aaa0 2023-10-16T10:08:16.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.736+0000 7f8a55056640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a5006b810 con 0x7f8a5015ae80 2023-10-16T10:08:16.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.736+0000 7f8a55056640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a500707e0 con 0x7f8a500ce390 2023-10-16T10:08:16.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.736+0000 7f8a4ed9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8a5015aaa0 0x7f8a500cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:41532/0 (socket says 172.21.15.84:41532) 2023-10-16T10:08:16.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.736+0000 7f8a4ed9d640 1 -- 172.21.15.84:0/1559144001 learned_addr learned my addr 172.21.15.84:0/1559144001 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:16.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.736+0000 7f8a4659c640 1 --2- 172.21.15.84:0/1559144001 >> v2:172.21.15.84:3300/0 conn(0x7f8a5015ae80 0x7f8a5015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:16.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.737+0000 7f8a4f59e640 1 --2- 172.21.15.84:0/1559144001 >> v2:172.21.15.157:3300/0 conn(0x7f8a500ce390 0x7f8a50070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:16.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.737+0000 7f8a4659c640 1 -- 172.21.15.84:0/1559144001 >> v2:172.21.15.157:3300/0 conn(0x7f8a500ce390 msgr2=0x7f8a50070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:16.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.737+0000 7f8a4659c640 1 --2- 172.21.15.84:0/1559144001 >> v2:172.21.15.157:3300/0 conn(0x7f8a500ce390 0x7f8a50070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:16.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.737+0000 7f8a4659c640 1 -- 172.21.15.84:0/1559144001 >> v1:172.21.15.84:6789/0 conn(0x7f8a5015aaa0 legacy=0x7f8a500cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:16.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.737+0000 7f8a4659c640 1 -- 172.21.15.84:0/1559144001 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5015ef90 con 0x7f8a5015ae80 2023-10-16T10:08:16.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.737+0000 7f8a4e59c640 1 -- 172.21.15.84:0/1559144001 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4206198921 0 0) 0x7f8a5015ed60 con 0x7f8a5015aaa0 2023-10-16T10:08:16.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.738+0000 7f8a4659c640 1 --2- 172.21.15.84:0/1559144001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a5015ae80 0x7f8a5015e820 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f8a500c2490 tx=0x7f8a3802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973809cf633a6b23 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:16.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.738+0000 7f8a4e59c640 1 -- 172.21.15.84:0/1559144001 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a38031830 con 0x7f8a5015ae80 2023-10-16T10:08:16.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.738+0000 7f8a4e59c640 1 -- 172.21.15.84:0/1559144001 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a38031990 con 0x7f8a5015ae80 2023-10-16T10:08:16.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.738+0000 7f8a4e59c640 1 -- 172.21.15.84:0/1559144001 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a38031d20 con 0x7f8a5015ae80 2023-10-16T10:08:16.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.739+0000 7f8a55056640 1 -- 172.21.15.84:0/1559144001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a5015ae80 msgr2=0x7f8a5015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:16.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.739+0000 7f8a55056640 1 --2- 172.21.15.84:0/1559144001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a5015ae80 0x7f8a5015e820 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f8a500c2490 tx=0x7f8a3802b8d0 comp rx=0 tx=0).stop 2023-10-16T10:08:16.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.739+0000 7f8a55056640 1 -- 172.21.15.84:0/1559144001 shutdown_connections 2023-10-16T10:08:16.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.739+0000 7f8a55056640 1 --2- 172.21.15.84:0/1559144001 >> v2:172.21.15.157:3300/0 conn(0x7f8a500ce390 0x7f8a50070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:16.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.739+0000 7f8a55056640 1 --2- 172.21.15.84:0/1559144001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a5015ae80 0x7f8a5015e820 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:16.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.739+0000 7f8a55056640 1 -- 172.21.15.84:0/1559144001 >> 172.21.15.84:0/1559144001 conn(0x7f8a500cb410 msgr2=0x7f8a500cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:16.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.739+0000 7f8a55056640 1 -- 172.21.15.84:0/1559144001 shutdown_connections 2023-10-16T10:08:16.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.739+0000 7f8a55056640 1 -- 172.21.15.84:0/1559144001 wait complete. 2023-10-16T10:08:16.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.741+0000 7f8a55056640 1 Processor -- start 2023-10-16T10:08:16.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.741+0000 7f8a55056640 1 -- start start 2023-10-16T10:08:16.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.742+0000 7f8a55056640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a500ce390 0x7f8a500c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:16.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.742+0000 7f8a55056640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a5015aaa0 0x7f8a5015b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:16.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.742+0000 7f8a55056640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a5015ae80 0x7f8a5014fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:16.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.742+0000 7f8a55056640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a500c5d20 con 0x7f8a5015ae80 2023-10-16T10:08:16.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.742+0000 7f8a55056640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5015b7f0 con 0x7f8a500ce390 2023-10-16T10:08:16.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.742+0000 7f8a55056640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a50150280 con 0x7f8a5015aaa0 2023-10-16T10:08:16.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.742+0000 7f8a4f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a5015ae80 0x7f8a5014fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:16.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.742+0000 7f8a4f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a5015ae80 0x7f8a5014fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44308/0 (socket says 172.21.15.84:44308) 2023-10-16T10:08:16.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.742+0000 7f8a4f59e640 1 -- 172.21.15.84:0/1482679145 learned_addr learned my addr 172.21.15.84:0/1482679145 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:16.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.742+0000 7f8a4ed9d640 1 --2- 172.21.15.84:0/1482679145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a500ce390 0x7f8a500c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:16.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.742+0000 7f8a4f59e640 1 -- 172.21.15.84:0/1482679145 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a5015aaa0 msgr2=0x7f8a5015b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:16.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.742+0000 7f8a4f59e640 1 --2- 172.21.15.84:0/1482679145 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a5015aaa0 0x7f8a5015b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:16.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.742+0000 7f8a4f59e640 1 -- 172.21.15.84:0/1482679145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a500ce390 msgr2=0x7f8a500c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:16.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.742+0000 7f8a4f59e640 1 --2- 172.21.15.84:0/1482679145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a500ce390 0x7f8a500c8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:16.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.743+0000 7f8a4f59e640 1 -- 172.21.15.84:0/1482679145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a38009400 con 0x7f8a5015ae80 2023-10-16T10:08:16.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.743+0000 7f8a4f59e640 1 --2- 172.21.15.84:0/1482679145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a5015ae80 0x7f8a5014fd40 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f8a48007e40 tx=0x7f8a4800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:16.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.743+0000 7f8a47fff640 1 -- 172.21.15.84:0/1482679145 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48014620 con 0x7f8a5015ae80 2023-10-16T10:08:16.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.743+0000 7f8a47fff640 1 -- 172.21.15.84:0/1482679145 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4800ee90 con 0x7f8a5015ae80 2023-10-16T10:08:16.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.743+0000 7f8a47fff640 1 -- 172.21.15.84:0/1482679145 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a480149e0 con 0x7f8a5015ae80 2023-10-16T10:08:16.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.743+0000 7f8a55056640 1 -- 172.21.15.84:0/1482679145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a500af520 con 0x7f8a5015ae80 2023-10-16T10:08:16.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.743+0000 7f8a55056640 1 -- 172.21.15.84:0/1482679145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a500afae0 con 0x7f8a5015ae80 2023-10-16T10:08:16.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.744+0000 7f8a55056640 1 -- 172.21.15.84:0/1482679145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a500c5e90 con 0x7f8a5015ae80 2023-10-16T10:08:16.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.745+0000 7f8a47fff640 1 -- 172.21.15.84:0/1482679145 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8a48014b40 con 0x7f8a5015ae80 2023-10-16T10:08:16.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.746+0000 7f8a47fff640 1 --2- 172.21.15.84:0/1482679145 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f8a3006af90 0x7f8a3006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:16.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.746+0000 7f8a47fff640 1 -- 172.21.15.84:0/1482679145 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5745+0+0 (secure 0 0 0) 0x7f8a4808a270 con 0x7f8a5015ae80 2023-10-16T10:08:16.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.746+0000 7f8a4ed9d640 1 --2- 172.21.15.84:0/1482679145 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f8a3006af90 0x7f8a3006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:16.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.747+0000 7f8a4ed9d640 1 --2- 172.21.15.84:0/1482679145 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f8a3006af90 0x7f8a3006d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8a40007fd0 tx=0x7f8a40009040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:16.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:16.750+0000 7f8a47fff640 1 -- 172.21.15.84:0/1482679145 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8a48055150 con 0x7f8a5015ae80 2023-10-16T10:08:17.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.013+0000 7f8a55056640 1 -- 172.21.15.84:0/1482679145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f8a5004e160 con 0x7f8a5015ae80 2023-10-16T10:08:17.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.139+0000 7f8a47fff640 1 -- 172.21.15.84:0/1482679145 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v737) v1 ==== 148+0+0 (secure 0 0 0) 0x7f8a48058770 con 0x7f8a5015ae80 2023-10-16T10:08:17.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.142+0000 7f8a55056640 1 -- 172.21.15.84:0/1482679145 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f8a3006af90 msgr2=0x7f8a3006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:17.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.142+0000 7f8a55056640 1 --2- 172.21.15.84:0/1482679145 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f8a3006af90 0x7f8a3006d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8a40007fd0 tx=0x7f8a40009040 comp rx=0 tx=0).stop 2023-10-16T10:08:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.142+0000 7f8a55056640 1 -- 172.21.15.84:0/1482679145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a5015ae80 msgr2=0x7f8a5014fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.142+0000 7f8a55056640 1 --2- 172.21.15.84:0/1482679145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a5015ae80 0x7f8a5014fd40 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f8a48007e40 tx=0x7f8a4800e4b0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.142+0000 7f8a55056640 1 -- 172.21.15.84:0/1482679145 shutdown_connections 2023-10-16T10:08:17.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.142+0000 7f8a55056640 1 --2- 172.21.15.84:0/1482679145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a5015ae80 0x7f8a5014fd40 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.142+0000 7f8a55056640 1 --2- 172.21.15.84:0/1482679145 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a5015aaa0 0x7f8a5015b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.142+0000 7f8a55056640 1 --2- 172.21.15.84:0/1482679145 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f8a3006af90 0x7f8a3006d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.142+0000 7f8a55056640 1 --2- 172.21.15.84:0/1482679145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a500ce390 0x7f8a500c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.142+0000 7f8a55056640 1 -- 172.21.15.84:0/1482679145 >> 172.21.15.84:0/1482679145 conn(0x7f8a500cb410 msgr2=0x7f8a5006cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.142+0000 7f8a55056640 1 -- 172.21.15.84:0/1482679145 shutdown_connections 2023-10-16T10:08:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.142+0000 7f8a55056640 1 -- 172.21.15.84:0/1482679145 wait complete. 2023-10-16T10:08:17.162 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2023-10-16T10:08:17.163 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_all_periods (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T10:08:17.163 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:08:17.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.371+0000 7fd0698de640 1 Processor -- start 2023-10-16T10:08:17.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.371+0000 7fd0698de640 1 -- start start 2023-10-16T10:08:17.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.372+0000 7fd0698de640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd0640b3070 0x7fd0640cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:17.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.372+0000 7fd0698de640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0640b3e70 0x7fd0640cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:17.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.372+0000 7fd0698de640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0640b3940 con 0x7fd0640b3450 2023-10-16T10:08:17.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.372+0000 7fd0698de640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0640b45e0 con 0x7fd0640b3070 2023-10-16T10:08:17.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.372+0000 7fd0698de640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640b3b90 con 0x7fd0640b3e70 2023-10-16T10:08:17.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.373+0000 7fd0627fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0640b3e70 0x7fd0640cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:17.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.373+0000 7fd062ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd0640b3070 0x7fd0640cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:17.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.373+0000 7fd0627fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0640b3e70 0x7fd0640cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:38172/0 (socket says 172.21.15.84:38172) 2023-10-16T10:08:17.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.373+0000 7fd0627fc640 1 -- 172.21.15.84:0/1676129466 learned_addr learned my addr 172.21.15.84:0/1676129466 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:17.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.374+0000 7fd0627fc640 1 -- 172.21.15.84:0/1676129466 >> v2:172.21.15.157:3300/0 conn(0x7fd0640b3070 msgr2=0x7fd0640cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:17.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.374+0000 7fd0627fc640 1 --2- 172.21.15.84:0/1676129466 >> v2:172.21.15.157:3300/0 conn(0x7fd0640b3070 0x7fd0640cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.374+0000 7fd0627fc640 1 -- 172.21.15.84:0/1676129466 >> v1:172.21.15.157:6789/0 conn(0x7fd0640b3450 legacy=0x7fd0640c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:17.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.374+0000 7fd0627fc640 1 -- 172.21.15.84:0/1676129466 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0640d3840 con 0x7fd0640b3e70 2023-10-16T10:08:17.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.375+0000 7fd0627fc640 1 --2- 172.21.15.84:0/1676129466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0640b3e70 0x7fd0640cfa40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009e50 tx=0x7fd04c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed09970f6d165e42 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:17.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.376+0000 7fd061ffb640 1 -- 172.21.15.84:0/1676129466 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c02cb00 con 0x7fd0640b3e70 2023-10-16T10:08:17.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.376+0000 7fd061ffb640 1 -- 172.21.15.84:0/1676129466 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04c02cc60 con 0x7fd0640b3e70 2023-10-16T10:08:17.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.376+0000 7fd061ffb640 1 -- 172.21.15.84:0/1676129466 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c0343f0 con 0x7fd0640b3e70 2023-10-16T10:08:17.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.377+0000 7fd0698de640 1 -- 172.21.15.84:0/1676129466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0640b3e70 msgr2=0x7fd0640cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:17.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.377+0000 7fd0698de640 1 --2- 172.21.15.84:0/1676129466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0640b3e70 0x7fd0640cfa40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009e50 tx=0x7fd04c02bf60 comp rx=0 tx=0).stop 2023-10-16T10:08:17.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.377+0000 7fd0698de640 1 -- 172.21.15.84:0/1676129466 shutdown_connections 2023-10-16T10:08:17.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.377+0000 7fd0698de640 1 --2- 172.21.15.84:0/1676129466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0640b3e70 0x7fd0640cfa40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.377+0000 7fd0698de640 1 --2- 172.21.15.84:0/1676129466 >> v2:172.21.15.157:3300/0 conn(0x7fd0640b3070 0x7fd0640cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.377+0000 7fd0698de640 1 -- 172.21.15.84:0/1676129466 >> 172.21.15.84:0/1676129466 conn(0x7fd0640d2d20 msgr2=0x7fd0640d3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:17.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.377+0000 7fd0698de640 1 -- 172.21.15.84:0/1676129466 shutdown_connections 2023-10-16T10:08:17.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.377+0000 7fd0698de640 1 -- 172.21.15.84:0/1676129466 wait complete. 2023-10-16T10:08:17.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.379+0000 7fd0698de640 1 Processor -- start 2023-10-16T10:08:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.379+0000 7fd0698de640 1 -- start start 2023-10-16T10:08:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.380+0000 7fd0698de640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0640b3070 0x7fd0640b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.380+0000 7fd0698de640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0640b3450 0x7fd0641e70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.380+0000 7fd0698de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0640b3e70 0x7fd06415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:17.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.380+0000 7fd0698de640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640b2550 con 0x7fd0640b3e70 2023-10-16T10:08:17.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.380+0000 7fd0698de640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0641e7610 con 0x7fd0640b3450 2023-10-16T10:08:17.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.380+0000 7fd0698de640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640cae30 con 0x7fd0640b3070 2023-10-16T10:08:17.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.380+0000 7fd0637fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0640b3e70 0x7fd06415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:17.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.380+0000 7fd0637fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0640b3e70 0x7fd06415dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44310/0 (socket says 172.21.15.84:44310) 2023-10-16T10:08:17.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.380+0000 7fd0637fe640 1 -- 172.21.15.84:0/1377537256 learned_addr learned my addr 172.21.15.84:0/1377537256 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:17.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.380+0000 7fd062ffd640 1 --2- 172.21.15.84:0/1377537256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0640b3070 0x7fd0640b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:17.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.380+0000 7fd0627fc640 1 --2- 172.21.15.84:0/1377537256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0640b3450 0x7fd0641e70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:17.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.381+0000 7fd0637fe640 1 -- 172.21.15.84:0/1377537256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0640b3070 msgr2=0x7fd0640b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:17.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.381+0000 7fd0637fe640 1 --2- 172.21.15.84:0/1377537256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0640b3070 0x7fd0640b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.381+0000 7fd0637fe640 1 -- 172.21.15.84:0/1377537256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0640b3450 msgr2=0x7fd0641e70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:17.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.381+0000 7fd0637fe640 1 --2- 172.21.15.84:0/1377537256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0640b3450 0x7fd0641e70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.381+0000 7fd0637fe640 1 -- 172.21.15.84:0/1377537256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c009b30 con 0x7fd0640b3e70 2023-10-16T10:08:17.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.381+0000 7fd0637fe640 1 --2- 172.21.15.84:0/1377537256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0640b3e70 0x7fd06415dc90 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fd05c001eb0 tx=0x7fd05c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:17.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.381+0000 7fd05bfff640 1 -- 172.21.15.84:0/1377537256 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c0052d0 con 0x7fd0640b3e70 2023-10-16T10:08:17.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.381+0000 7fd05bfff640 1 -- 172.21.15.84:0/1377537256 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05c005430 con 0x7fd0640b3e70 2023-10-16T10:08:17.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.381+0000 7fd05bfff640 1 -- 172.21.15.84:0/1377537256 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c003810 con 0x7fd0640b3e70 2023-10-16T10:08:17.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.381+0000 7fd0698de640 1 -- 172.21.15.84:0/1377537256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0640b1f90 con 0x7fd0640b3e70 2023-10-16T10:08:17.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.381+0000 7fd0698de640 1 -- 172.21.15.84:0/1377537256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0640cb0d0 con 0x7fd0640b3e70 2023-10-16T10:08:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.383+0000 7fd0698de640 1 -- 172.21.15.84:0/1377537256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06404e160 con 0x7fd0640b3e70 2023-10-16T10:08:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.384+0000 7fd05bfff640 1 -- 172.21.15.84:0/1377537256 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd05c003970 con 0x7fd0640b3e70 2023-10-16T10:08:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.384+0000 7fd05bfff640 1 --2- 172.21.15.84:0/1377537256 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fd04406af60 0x7fd04406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:17.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.385+0000 7fd05bfff640 1 -- 172.21.15.84:0/1377537256 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5745+0+0 (secure 0 0 0) 0x7fd05c088d90 con 0x7fd0640b3e70 2023-10-16T10:08:17.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.385+0000 7fd062ffd640 1 --2- 172.21.15.84:0/1377537256 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fd04406af60 0x7fd04406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:17.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.385+0000 7fd062ffd640 1 --2- 172.21.15.84:0/1377537256 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fd04406af60 0x7fd04406d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd054009ac0 tx=0x7fd054008040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:17.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.389+0000 7fd05bfff640 1 -- 172.21.15.84:0/1377537256 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd05c053d00 con 0x7fd0640b3e70 2023-10-16T10:08:17.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.581+0000 7fd0698de640 1 -- 172.21.15.84:0/1377537256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0640b8b50 con 0x7fd0640b3e70 2023-10-16T10:08:17.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.582+0000 7fd05bfff640 1 -- 172.21.15.84:0/1377537256 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+14141 (secure 0 0 0) 0x7fd05c017090 con 0x7fd0640b3e70 2023-10-16T10:08:17.583 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:17.584 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":130,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:08:15.114279+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6835/2859088731":"2023-10-17T10:08:12.057038+0000","172.21.15.84:0/621277862":"2023-10-17T10:08:08.425362+0000","172.21.15.84:6837/2859088731":"2023-10-17T10:08:12.057038+0000","172.21.15.84:0/1967825537":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/2700178020":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/1638353973":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/1637648657":"2023-10-17T10:08:08.425362+0000","172.21.15.84:6833/2209954486":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/3662496273":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/1284479959":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/2038404786":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/688441146":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/644840609":"2023-10-17T10:07:43.574635+0000","172.21.15.84:6832/2209954486":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/3794895754":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/1166553040":"2023-10-17T10:08:08.425362+0000","172.21.15.84:0/3269894694":"2023-10-17T10:07:43.574635+0000","172.21.15.84:6833/4292025878":"2023-10-17T10:07:43.574635+0000","172.21.15.84:6832/4292025878":"2023-10-17T10:07:43.574635+0000","172.21.15.84:0/1934706185":"2023-10-17T10:07:43.574635+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:08:17.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.585+0000 7fd0698de640 1 -- 172.21.15.84:0/1377537256 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fd04406af60 msgr2=0x7fd04406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:17.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.585+0000 7fd0698de640 1 --2- 172.21.15.84:0/1377537256 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fd04406af60 0x7fd04406d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd054009ac0 tx=0x7fd054008040 comp rx=0 tx=0).stop 2023-10-16T10:08:17.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.585+0000 7fd0698de640 1 -- 172.21.15.84:0/1377537256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0640b3e70 msgr2=0x7fd06415dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:17.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.585+0000 7fd0698de640 1 --2- 172.21.15.84:0/1377537256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0640b3e70 0x7fd06415dc90 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fd05c001eb0 tx=0x7fd05c00c890 comp rx=0 tx=0).stop 2023-10-16T10:08:17.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.585+0000 7fd0698de640 1 -- 172.21.15.84:0/1377537256 shutdown_connections 2023-10-16T10:08:17.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.585+0000 7fd0698de640 1 --2- 172.21.15.84:0/1377537256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0640b3e70 0x7fd06415dc90 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.585+0000 7fd0698de640 1 --2- 172.21.15.84:0/1377537256 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fd04406af60 0x7fd04406d420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.585+0000 7fd0698de640 1 --2- 172.21.15.84:0/1377537256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0640b3450 0x7fd0641e70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.585+0000 7fd0698de640 1 --2- 172.21.15.84:0/1377537256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0640b3070 0x7fd0640b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.585+0000 7fd0698de640 1 -- 172.21.15.84:0/1377537256 >> 172.21.15.84:0/1377537256 conn(0x7fd0640d2d20 msgr2=0x7fd0640c7580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:17.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.585+0000 7fd0698de640 1 -- 172.21.15.84:0/1377537256 shutdown_connections 2023-10-16T10:08:17.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.585+0000 7fd0698de640 1 -- 172.21.15.84:0/1377537256 wait complete. 2023-10-16T10:08:17.605 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:08:17.606 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:08:17.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.816+0000 7fbe74644640 1 Processor -- start 2023-10-16T10:08:17.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.816+0000 7fbe74644640 1 -- start start 2023-10-16T10:08:17.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.818+0000 7fbe74644640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbe6c159e90 0x7fbe6c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:17.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.818+0000 7fbe74644640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbe6c06c140 0x7fbe6c06c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:17.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.818+0000 7fbe74644640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe6c06cad0 con 0x7fbe6c159ab0 2023-10-16T10:08:17.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.818+0000 7fbe74644640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe6c15d890 con 0x7fbe6c159e90 2023-10-16T10:08:17.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.818+0000 7fbe74644640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe6c070360 con 0x7fbe6c06c140 2023-10-16T10:08:17.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.818+0000 7fbe71bdf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbe6c159e90 0x7fbe6c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:17.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.818+0000 7fbe71bdf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbe6c159e90 0x7fbe6c0ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44314/0 (socket says 172.21.15.84:44314) 2023-10-16T10:08:17.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.818+0000 7fbe71bdf640 1 -- 172.21.15.84:0/1494487599 learned_addr learned my addr 172.21.15.84:0/1494487599 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:17.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.818+0000 7fbe72be1640 1 --2- 172.21.15.84:0/1494487599 >> v2:172.21.15.157:3300/0 conn(0x7fbe6c06c140 0x7fbe6c06c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:17.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.819+0000 7fbe71bdf640 1 -- 172.21.15.84:0/1494487599 >> v2:172.21.15.157:3300/0 conn(0x7fbe6c06c140 msgr2=0x7fbe6c06c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:17.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.819+0000 7fbe71bdf640 1 --2- 172.21.15.84:0/1494487599 >> v2:172.21.15.157:3300/0 conn(0x7fbe6c06c140 0x7fbe6c06c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.819+0000 7fbe71bdf640 1 -- 172.21.15.84:0/1494487599 >> v1:172.21.15.157:6789/0 conn(0x7fbe6c159ab0 legacy=0x7fbe6c06fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:17.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.819+0000 7fbe71bdf640 1 -- 172.21.15.84:0/1494487599 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c0cd1f0 con 0x7fbe6c159e90 2023-10-16T10:08:17.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.820+0000 7fbe71bdf640 1 --2- 172.21.15.84:0/1494487599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c159e90 0x7fbe6c0ce370 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0adcd0 tx=0x7fbe5c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f796ef244e049e27 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:17.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.820+0000 7fbe713de640 1 -- 172.21.15.84:0/1494487599 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c031750 con 0x7fbe6c159e90 2023-10-16T10:08:17.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.820+0000 7fbe713de640 1 -- 172.21.15.84:0/1494487599 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5c0318b0 con 0x7fbe6c159e90 2023-10-16T10:08:17.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.820+0000 7fbe713de640 1 -- 172.21.15.84:0/1494487599 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c031c40 con 0x7fbe6c159e90 2023-10-16T10:08:17.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.821+0000 7fbe74644640 1 -- 172.21.15.84:0/1494487599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c159e90 msgr2=0x7fbe6c0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:17.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.821+0000 7fbe74644640 1 --2- 172.21.15.84:0/1494487599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c159e90 0x7fbe6c0ce370 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0adcd0 tx=0x7fbe5c02b7f0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.821+0000 7fbe74644640 1 -- 172.21.15.84:0/1494487599 shutdown_connections 2023-10-16T10:08:17.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.821+0000 7fbe74644640 1 --2- 172.21.15.84:0/1494487599 >> v2:172.21.15.157:3300/0 conn(0x7fbe6c06c140 0x7fbe6c06c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.821+0000 7fbe74644640 1 --2- 172.21.15.84:0/1494487599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c159e90 0x7fbe6c0ce370 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.821+0000 7fbe74644640 1 -- 172.21.15.84:0/1494487599 >> 172.21.15.84:0/1494487599 conn(0x7fbe6c0ce9b0 msgr2=0x7fbe6c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:17.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.821+0000 7fbe74644640 1 -- 172.21.15.84:0/1494487599 shutdown_connections 2023-10-16T10:08:17.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.821+0000 7fbe74644640 1 -- 172.21.15.84:0/1494487599 wait complete. 2023-10-16T10:08:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.823+0000 7fbe74644640 1 Processor -- start 2023-10-16T10:08:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.823+0000 7fbe74644640 1 -- start start 2023-10-16T10:08:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.824+0000 7fbe74644640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe6c06c140 0x7fbe6c0c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.824+0000 7fbe74644640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe6c159ab0 0x7fbe6c14bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.824+0000 7fbe74644640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c159e90 0x7fbe6c14c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.824+0000 7fbe74644640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c15cda0 con 0x7fbe6c159e90 2023-10-16T10:08:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.824+0000 7fbe74644640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c14c5c0 con 0x7fbe6c159ab0 2023-10-16T10:08:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.824+0000 7fbe74644640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c15d1c0 con 0x7fbe6c06c140 2023-10-16T10:08:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.824+0000 7fbe72be1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c159e90 0x7fbe6c14c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.824+0000 7fbe72be1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c159e90 0x7fbe6c14c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44326/0 (socket says 172.21.15.84:44326) 2023-10-16T10:08:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.824+0000 7fbe72be1640 1 -- 172.21.15.84:0/2631192926 learned_addr learned my addr 172.21.15.84:0/2631192926 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.824+0000 7fbe71bdf640 1 --2- 172.21.15.84:0/2631192926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe6c159ab0 0x7fbe6c14bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.825+0000 7fbe72be1640 1 -- 172.21.15.84:0/2631192926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe6c06c140 msgr2=0x7fbe6c0c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.825+0000 7fbe723e0640 1 --2- 172.21.15.84:0/2631192926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe6c06c140 0x7fbe6c0c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.825+0000 7fbe72be1640 1 --2- 172.21.15.84:0/2631192926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe6c06c140 0x7fbe6c0c4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.825+0000 7fbe72be1640 1 -- 172.21.15.84:0/2631192926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe6c159ab0 msgr2=0x7fbe6c14bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.825+0000 7fbe72be1640 1 --2- 172.21.15.84:0/2631192926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe6c159ab0 0x7fbe6c14bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.825+0000 7fbe72be1640 1 -- 172.21.15.84:0/2631192926 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe5c0094d0 con 0x7fbe6c159e90 2023-10-16T10:08:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.825+0000 7fbe71bdf640 1 --2- 172.21.15.84:0/2631192926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe6c159ab0 0x7fbe6c14bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.825+0000 7fbe72be1640 1 --2- 172.21.15.84:0/2631192926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c159e90 0x7fbe6c14c080 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fbe64007e90 tx=0x7fbe6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.825+0000 7fbe62ffd640 1 -- 172.21.15.84:0/2631192926 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe64014420 con 0x7fbe6c159e90 2023-10-16T10:08:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.825+0000 7fbe62ffd640 1 -- 172.21.15.84:0/2631192926 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe64014580 con 0x7fbe6c159e90 2023-10-16T10:08:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.825+0000 7fbe62ffd640 1 -- 172.21.15.84:0/2631192926 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe64014930 con 0x7fbe6c159e90 2023-10-16T10:08:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.825+0000 7fbe74644640 1 -- 172.21.15.84:0/2631192926 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe6c15c400 con 0x7fbe6c159e90 2023-10-16T10:08:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.825+0000 7fbe74644640 1 -- 172.21.15.84:0/2631192926 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6c15c9c0 con 0x7fbe6c159e90 2023-10-16T10:08:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.826+0000 7fbe74644640 1 -- 172.21.15.84:0/2631192926 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6c15cee0 con 0x7fbe6c159e90 2023-10-16T10:08:17.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.827+0000 7fbe62ffd640 1 -- 172.21.15.84:0/2631192926 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbe64014a90 con 0x7fbe6c159e90 2023-10-16T10:08:17.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.828+0000 7fbe62ffd640 1 --2- 172.21.15.84:0/2631192926 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fbe5006af90 0x7fbe5006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:17.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.828+0000 7fbe62ffd640 1 -- 172.21.15.84:0/2631192926 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5745+0+0 (secure 0 0 0) 0x7fbe6408a0e0 con 0x7fbe6c159e90 2023-10-16T10:08:17.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.829+0000 7fbe723e0640 1 --2- 172.21.15.84:0/2631192926 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fbe5006af90 0x7fbe5006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:17.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.829+0000 7fbe723e0640 1 --2- 172.21.15.84:0/2631192926 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fbe5006af90 0x7fbe5006d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0ad9f0 tx=0x7fbe68009040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:17.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:17.833+0000 7fbe62ffd640 1 -- 172.21.15.84:0/2631192926 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbe6c15cee0 con 0x7fbe6c159e90 2023-10-16T10:08:18.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.020+0000 7fbe74644640 1 -- 172.21.15.84:0/2631192926 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbe6c15cee0 con 0x7fbe6c159e90 2023-10-16T10:08:18.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.021+0000 7fbe62ffd640 1 -- 172.21.15.84:0/2631192926 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbe6c15cee0 con 0x7fbe6c159e90 2023-10-16T10:08:18.022 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:08:18.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.023+0000 7fbe74644640 1 -- 172.21.15.84:0/2631192926 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fbe5006af90 msgr2=0x7fbe5006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:18.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.023+0000 7fbe74644640 1 --2- 172.21.15.84:0/2631192926 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fbe5006af90 0x7fbe5006d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0ad9f0 tx=0x7fbe68009040 comp rx=0 tx=0).stop 2023-10-16T10:08:18.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.023+0000 7fbe74644640 1 -- 172.21.15.84:0/2631192926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c159e90 msgr2=0x7fbe6c14c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:18.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.023+0000 7fbe74644640 1 --2- 172.21.15.84:0/2631192926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c159e90 0x7fbe6c14c080 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fbe64007e90 tx=0x7fbe6400e4b0 comp rx=0 tx=0).stop 2023-10-16T10:08:18.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.023+0000 7fbe74644640 1 -- 172.21.15.84:0/2631192926 shutdown_connections 2023-10-16T10:08:18.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.023+0000 7fbe74644640 1 --2- 172.21.15.84:0/2631192926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c159e90 0x7fbe6c14c080 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:18.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.023+0000 7fbe74644640 1 --2- 172.21.15.84:0/2631192926 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fbe5006af90 0x7fbe5006d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:18.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.023+0000 7fbe74644640 1 --2- 172.21.15.84:0/2631192926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe6c159ab0 0x7fbe6c14bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:18.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.023+0000 7fbe74644640 1 --2- 172.21.15.84:0/2631192926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe6c06c140 0x7fbe6c0c4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:18.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.023+0000 7fbe74644640 1 -- 172.21.15.84:0/2631192926 >> 172.21.15.84:0/2631192926 conn(0x7fbe6c0ce9b0 msgr2=0x7fbe6c0b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:18.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.024+0000 7fbe74644640 1 -- 172.21.15.84:0/2631192926 shutdown_connections 2023-10-16T10:08:18.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.024+0000 7fbe74644640 1 -- 172.21.15.84:0/2631192926 wait complete. 2023-10-16T10:08:18.043 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods' 2023-10-16T10:08:18.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.248+0000 7efdc1f9e640 1 Processor -- start 2023-10-16T10:08:18.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.248+0000 7efdc1f9e640 1 -- start start 2023-10-16T10:08:18.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.250+0000 7efdc1f9e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efdbc0cded0 0x7efdbc0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:18.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.250+0000 7efdc1f9e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdbc15d150 con 0x7efdbc0b8480 2023-10-16T10:08:18.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.250+0000 7efdc1f9e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdbc0d1180 con 0x7efdbc15cd70 2023-10-16T10:08:18.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.250+0000 7efdc1f9e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdbc0d1460 con 0x7efdbc0cded0 2023-10-16T10:08:18.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.250+0000 7efdc179d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efdbc0cded0 0x7efdbc0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:18.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.250+0000 7efdc179d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efdbc0cded0 0x7efdbc0b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44338/0 (socket says 172.21.15.84:44338) 2023-10-16T10:08:18.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.250+0000 7efdc179d640 1 -- 172.21.15.84:0/4062372153 learned_addr learned my addr 172.21.15.84:0/4062372153 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:18.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.251+0000 7efdc179d640 1 -- 172.21.15.84:0/4062372153 >> v1:172.21.15.157:6789/0 conn(0x7efdbc15cd70 legacy=0x7efdbc0d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:18.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.251+0000 7efdc179d640 1 -- 172.21.15.84:0/4062372153 >> v1:172.21.15.84:6789/0 conn(0x7efdbc0b8480 legacy=0x7efdbc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:18.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.251+0000 7efdc179d640 1 -- 172.21.15.84:0/4062372153 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc0d1740 con 0x7efdbc0cded0 2023-10-16T10:08:18.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.252+0000 7efdc179d640 1 --2- 172.21.15.84:0/4062372153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdbc0cded0 0x7efdbc0b5c90 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009e50 tx=0x7efdb402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c241814dac2760f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:18.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.252+0000 7efdbb7fe640 1 -- 172.21.15.84:0/4062372153 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb402cb10 con 0x7efdbc0cded0 2023-10-16T10:08:18.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.252+0000 7efdbb7fe640 1 -- 172.21.15.84:0/4062372153 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb402cc70 con 0x7efdbc0cded0 2023-10-16T10:08:18.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.252+0000 7efdbb7fe640 1 -- 172.21.15.84:0/4062372153 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb40343f0 con 0x7efdbc0cded0 2023-10-16T10:08:18.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.252+0000 7efdc1f9e640 1 -- 172.21.15.84:0/4062372153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdbc0cded0 msgr2=0x7efdbc0b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:18.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.252+0000 7efdc1f9e640 1 --2- 172.21.15.84:0/4062372153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdbc0cded0 0x7efdbc0b5c90 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009e50 tx=0x7efdb402bf70 comp rx=0 tx=0).stop 2023-10-16T10:08:18.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.253+0000 7efdc1f9e640 1 -- 172.21.15.84:0/4062372153 shutdown_connections 2023-10-16T10:08:18.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.253+0000 7efdc1f9e640 1 --2- 172.21.15.84:0/4062372153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdbc0cded0 0x7efdbc0b5c90 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:18.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.253+0000 7efdc1f9e640 1 -- 172.21.15.84:0/4062372153 >> 172.21.15.84:0/4062372153 conn(0x7efdbc0cf570 msgr2=0x7efdbc0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:18.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.253+0000 7efdc1f9e640 1 -- 172.21.15.84:0/4062372153 shutdown_connections 2023-10-16T10:08:18.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.253+0000 7efdc1f9e640 1 -- 172.21.15.84:0/4062372153 wait complete. 2023-10-16T10:08:18.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.254+0000 7efdc1f9e640 1 Processor -- start 2023-10-16T10:08:18.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.254+0000 7efdc1f9e640 1 -- start start 2023-10-16T10:08:18.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.255+0000 7efdc1f9e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdbc0b8480 0x7efdbc157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:18.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.255+0000 7efdc1f9e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdbc0cded0 0x7efdbc0a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:18.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.255+0000 7efdc1f9e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdbc15cd70 0x7efdbc0cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:18.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.255+0000 7efdc1f9e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc0c9bc0 con 0x7efdbc0b8480 2023-10-16T10:08:18.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.255+0000 7efdc1f9e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc0a9ed0 con 0x7efdbc0cded0 2023-10-16T10:08:18.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.255+0000 7efdc1f9e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc0aa1b0 con 0x7efdbc15cd70 2023-10-16T10:08:18.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.256+0000 7efdc0f9c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdbc0b8480 0x7efdbc157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:18.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.256+0000 7efdc0f9c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdbc0b8480 0x7efdbc157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44348/0 (socket says 172.21.15.84:44348) 2023-10-16T10:08:18.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.256+0000 7efdc0f9c640 1 -- 172.21.15.84:0/4081784827 learned_addr learned my addr 172.21.15.84:0/4081784827 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:18.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.256+0000 7efdbbfff640 1 --2- 172.21.15.84:0/4081784827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdbc0cded0 0x7efdbc0a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:18.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.256+0000 7efdc0f9c640 1 -- 172.21.15.84:0/4081784827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdbc15cd70 msgr2=0x7efdbc0cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:18.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.256+0000 7efdc0f9c640 1 --2- 172.21.15.84:0/4081784827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdbc15cd70 0x7efdbc0cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:18.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.256+0000 7efdc0f9c640 1 -- 172.21.15.84:0/4081784827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdbc0cded0 msgr2=0x7efdbc0a94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:18.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.256+0000 7efdc0f9c640 1 --2- 172.21.15.84:0/4081784827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdbc0cded0 0x7efdbc0a94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:18.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.256+0000 7efdc0f9c640 1 -- 172.21.15.84:0/4081784827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb4009b30 con 0x7efdbc0b8480 2023-10-16T10:08:18.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.256+0000 7efdc0f9c640 1 --2- 172.21.15.84:0/4081784827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdbc0b8480 0x7efdbc157f70 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7efdb000b3e0 tx=0x7efdb000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:18.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.257+0000 7efdb97fa640 1 -- 172.21.15.84:0/4081784827 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb0011830 con 0x7efdbc0b8480 2023-10-16T10:08:18.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.257+0000 7efdb97fa640 1 -- 172.21.15.84:0/4081784827 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb00119b0 con 0x7efdbc0b8480 2023-10-16T10:08:18.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.257+0000 7efdb97fa640 1 -- 172.21.15.84:0/4081784827 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb0011d60 con 0x7efdbc0b8480 2023-10-16T10:08:18.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.257+0000 7efdc1f9e640 1 -- 172.21.15.84:0/4081784827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdbc0aa490 con 0x7efdbc0b8480 2023-10-16T10:08:18.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.257+0000 7efdc1f9e640 1 -- 172.21.15.84:0/4081784827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdbc1584b0 con 0x7efdbc0b8480 2023-10-16T10:08:18.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.258+0000 7efdc1f9e640 1 -- 172.21.15.84:0/4081784827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdbc0c9d70 con 0x7efdbc0b8480 2023-10-16T10:08:18.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.259+0000 7efdb97fa640 1 -- 172.21.15.84:0/4081784827 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efdb001a400 con 0x7efdbc0b8480 2023-10-16T10:08:18.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.260+0000 7efdb97fa640 1 --2- 172.21.15.84:0/4081784827 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7efda006af90 0x7efda006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:18.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.260+0000 7efdb97fa640 1 -- 172.21.15.84:0/4081784827 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5745+0+0 (secure 0 0 0) 0x7efdb0089950 con 0x7efdbc0b8480 2023-10-16T10:08:18.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.260+0000 7efdbbfff640 1 --2- 172.21.15.84:0/4081784827 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7efda006af90 0x7efda006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:18.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.261+0000 7efdbbfff640 1 --2- 172.21.15.84:0/4081784827 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7efda006af90 0x7efda006d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0cc2b0 tx=0x7efda8009040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:18.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.265+0000 7efdb97fa640 1 -- 172.21.15.84:0/4081784827 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efdbc0c9d70 con 0x7efdbc0b8480 2023-10-16T10:08:18.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:18.526+0000 7efdc1f9e640 1 -- 172.21.15.84:0/4081784827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]} v 0) v1 -- 0x7efdbc04e9b0 con 0x7efdbc0b8480 2023-10-16T10:08:19.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.150+0000 7efdb97fa640 1 -- 172.21.15.84:0/4081784827 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]}]=0 v739) v1 ==== 163+0+0 (secure 0 0 0) 0x7efdb0015020 con 0x7efdbc0b8480 2023-10-16T10:08:19.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.154+0000 7efdc1f9e640 1 -- 172.21.15.84:0/4081784827 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7efda006af90 msgr2=0x7efda006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:19.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.154+0000 7efdc1f9e640 1 --2- 172.21.15.84:0/4081784827 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7efda006af90 0x7efda006d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0cc2b0 tx=0x7efda8009040 comp rx=0 tx=0).stop 2023-10-16T10:08:19.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.154+0000 7efdc1f9e640 1 -- 172.21.15.84:0/4081784827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdbc0b8480 msgr2=0x7efdbc157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:19.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.154+0000 7efdc1f9e640 1 --2- 172.21.15.84:0/4081784827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdbc0b8480 0x7efdbc157f70 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7efdb000b3e0 tx=0x7efdb000b8a0 comp rx=0 tx=0).stop 2023-10-16T10:08:19.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.154+0000 7efdc1f9e640 1 -- 172.21.15.84:0/4081784827 shutdown_connections 2023-10-16T10:08:19.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.154+0000 7efdc1f9e640 1 --2- 172.21.15.84:0/4081784827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdbc15cd70 0x7efdbc0cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:19.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.154+0000 7efdc1f9e640 1 --2- 172.21.15.84:0/4081784827 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7efda006af90 0x7efda006d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:19.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.154+0000 7efdc1f9e640 1 --2- 172.21.15.84:0/4081784827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdbc0cded0 0x7efdbc0a94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:19.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.154+0000 7efdc1f9e640 1 --2- 172.21.15.84:0/4081784827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdbc0b8480 0x7efdbc157f70 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:19.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.154+0000 7efdc1f9e640 1 -- 172.21.15.84:0/4081784827 >> 172.21.15.84:0/4081784827 conn(0x7efdbc0cf570 msgr2=0x7efdbc0cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:19.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.155+0000 7efdc1f9e640 1 -- 172.21.15.84:0/4081784827 shutdown_connections 2023-10-16T10:08:19.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.155+0000 7efdc1f9e640 1 -- 172.21.15.84:0/4081784827 wait complete. 2023-10-16T10:08:19.181 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T10:08:19.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.399+0000 7f094b748640 1 Processor -- start 2023-10-16T10:08:19.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.399+0000 7f094b748640 1 -- start start 2023-10-16T10:08:19.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.400+0000 7f094b748640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0944159ab0 0x7f09440ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:19.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.400+0000 7f094b748640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f094406c140 0x7f094406c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:19.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.400+0000 7f094b748640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f094406cad0 con 0x7f0944159e90 2023-10-16T10:08:19.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.400+0000 7f094b748640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f094415d890 con 0x7f094406c140 2023-10-16T10:08:19.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.400+0000 7f094b748640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0944070360 con 0x7f0944159ab0 2023-10-16T10:08:19.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.401+0000 7f0949ce5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f094406c140 0x7f094406c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:19.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.401+0000 7f0948ce3640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0944159e90 0x7f094406fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:41558/0 (socket says 172.21.15.84:41558) 2023-10-16T10:08:19.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.401+0000 7f0948ce3640 1 -- 172.21.15.84:0/1931660067 learned_addr learned my addr 172.21.15.84:0/1931660067 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:19.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.401+0000 7f09494e4640 1 --2- 172.21.15.84:0/1931660067 >> v2:172.21.15.157:3300/0 conn(0x7f0944159ab0 0x7f09440ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:19.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.401+0000 7f0949ce5640 1 -- 172.21.15.84:0/1931660067 >> v2:172.21.15.157:3300/0 conn(0x7f0944159ab0 msgr2=0x7f09440ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:19.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.401+0000 7f0949ce5640 1 --2- 172.21.15.84:0/1931660067 >> v2:172.21.15.157:3300/0 conn(0x7f0944159ab0 0x7f09440ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:19.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.401+0000 7f0949ce5640 1 -- 172.21.15.84:0/1931660067 >> v1:172.21.15.84:6789/0 conn(0x7f0944159e90 legacy=0x7f094406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:19.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.401+0000 7f0949ce5640 1 -- 172.21.15.84:0/1931660067 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09440cd1f0 con 0x7f094406c140 2023-10-16T10:08:19.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.401+0000 7f093bfff640 1 -- 172.21.15.84:0/1931660067 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712744919 0 0) 0x7f0934001110 con 0x7f0944159e90 2023-10-16T10:08:19.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.401+0000 7f09494e4640 1 --2- 172.21.15.84:0/1931660067 >> v2:172.21.15.157:3300/0 conn(0x7f0944159ab0 0x7f09440ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:19.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.402+0000 7f0949ce5640 1 --2- 172.21.15.84:0/1931660067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f094406c140 0x7f094406c590 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f09440cfa00 tx=0x7f093c02bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f3f552ad11e26f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:19.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.403+0000 7f093bfff640 1 -- 172.21.15.84:0/1931660067 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c02cb50 con 0x7f094406c140 2023-10-16T10:08:19.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.403+0000 7f093bfff640 1 -- 172.21.15.84:0/1931660067 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093c02ccb0 con 0x7f094406c140 2023-10-16T10:08:19.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.403+0000 7f093bfff640 1 -- 172.21.15.84:0/1931660067 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c034660 con 0x7f094406c140 2023-10-16T10:08:19.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.403+0000 7f094b748640 1 -- 172.21.15.84:0/1931660067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f094406c140 msgr2=0x7f094406c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:19.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.403+0000 7f094b748640 1 --2- 172.21.15.84:0/1931660067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f094406c140 0x7f094406c590 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f09440cfa00 tx=0x7f093c02bda0 comp rx=0 tx=0).stop 2023-10-16T10:08:19.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.403+0000 7f094b748640 1 -- 172.21.15.84:0/1931660067 shutdown_connections 2023-10-16T10:08:19.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.403+0000 7f094b748640 1 --2- 172.21.15.84:0/1931660067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f094406c140 0x7f094406c590 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:19.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.403+0000 7f094b748640 1 --2- 172.21.15.84:0/1931660067 >> v2:172.21.15.157:3300/0 conn(0x7f0944159ab0 0x7f09440ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:19.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.403+0000 7f094b748640 1 -- 172.21.15.84:0/1931660067 >> 172.21.15.84:0/1931660067 conn(0x7f09440ce9b0 msgr2=0x7f09440cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:19.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.403+0000 7f094b748640 1 -- 172.21.15.84:0/1931660067 shutdown_connections 2023-10-16T10:08:19.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.403+0000 7f094b748640 1 -- 172.21.15.84:0/1931660067 wait complete. 2023-10-16T10:08:19.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.405+0000 7f094b748640 1 Processor -- start 2023-10-16T10:08:19.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.405+0000 7f094b748640 1 -- start start 2023-10-16T10:08:19.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.406+0000 7f094b748640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f094406c140 0x7f0944149d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:19.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.406+0000 7f094b748640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0944159ab0 0x7f09440ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:19.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.406+0000 7f094b748640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0944159e90 0x7f09440acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:19.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.406+0000 7f094b748640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09441e3820 con 0x7f0944159e90 2023-10-16T10:08:19.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.406+0000 7f094b748640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09441e3b00 con 0x7f094406c140 2023-10-16T10:08:19.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.406+0000 7f094b748640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09441e3de0 con 0x7f0944159ab0 2023-10-16T10:08:19.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.406+0000 7f0949ce5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0944159e90 0x7f09440acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:19.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.406+0000 7f0949ce5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0944159e90 0x7f09440acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44356/0 (socket says 172.21.15.84:44356) 2023-10-16T10:08:19.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.406+0000 7f0949ce5640 1 -- 172.21.15.84:0/3786405673 learned_addr learned my addr 172.21.15.84:0/3786405673 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:19.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.407+0000 7f0949ce5640 1 -- 172.21.15.84:0/3786405673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0944159ab0 msgr2=0x7f09440ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:19.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.407+0000 7f0949ce5640 1 --2- 172.21.15.84:0/3786405673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0944159ab0 0x7f09440ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:19.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.407+0000 7f0949ce5640 1 -- 172.21.15.84:0/3786405673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f094406c140 msgr2=0x7f0944149d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:19.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.407+0000 7f09494e4640 1 --2- 172.21.15.84:0/3786405673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f094406c140 0x7f0944149d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:19.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.407+0000 7f0949ce5640 1 --2- 172.21.15.84:0/3786405673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f094406c140 0x7f0944149d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:19.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.407+0000 7f0949ce5640 1 -- 172.21.15.84:0/3786405673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c009a70 con 0x7f0944159e90 2023-10-16T10:08:19.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.407+0000 7f0949ce5640 1 --2- 172.21.15.84:0/3786405673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0944159e90 0x7f09440acdd0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f093c009e80 tx=0x7f093c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:19.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.407+0000 7f0939ffb640 1 -- 172.21.15.84:0/3786405673 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c040650 con 0x7f0944159e90 2023-10-16T10:08:19.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.407+0000 7f0939ffb640 1 -- 172.21.15.84:0/3786405673 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093c0407b0 con 0x7f0944159e90 2023-10-16T10:08:19.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.407+0000 7f0939ffb640 1 -- 172.21.15.84:0/3786405673 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c040b60 con 0x7f0944159e90 2023-10-16T10:08:19.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.408+0000 7f094b748640 1 -- 172.21.15.84:0/3786405673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09441e40c0 con 0x7f0944159e90 2023-10-16T10:08:19.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.408+0000 7f094b748640 1 -- 172.21.15.84:0/3786405673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09441e48c0 con 0x7f0944159e90 2023-10-16T10:08:19.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.409+0000 7f094b748640 1 -- 172.21.15.84:0/3786405673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09441e45e0 con 0x7f0944159e90 2023-10-16T10:08:19.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.410+0000 7f0939ffb640 1 -- 172.21.15.84:0/3786405673 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f093c040cc0 con 0x7f0944159e90 2023-10-16T10:08:19.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.410+0000 7f0939ffb640 1 --2- 172.21.15.84:0/3786405673 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f092806af90 0x7f092806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:19.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.411+0000 7f0939ffb640 1 -- 172.21.15.84:0/3786405673 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5745+0+0 (secure 0 0 0) 0x7f093c0ac830 con 0x7f0944159e90 2023-10-16T10:08:19.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.411+0000 7f09494e4640 1 --2- 172.21.15.84:0/3786405673 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f092806af90 0x7f092806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:19.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.412+0000 7f09494e4640 1 --2- 172.21.15.84:0/3786405673 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f092806af90 0x7f092806d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0940008850 tx=0x7f0940008040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:19.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.415+0000 7f0939ffb640 1 -- 172.21.15.84:0/3786405673 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f09441e45e0 con 0x7f0944159e90 2023-10-16T10:08:19.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.607+0000 7f094b748640 1 -- 172.21.15.84:0/3786405673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f09441e45e0 con 0x7f0944159e90 2023-10-16T10:08:19.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.614+0000 7f0939ffb640 1 -- 172.21.15.84:0/3786405673 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7f09441e45e0 con 0x7f0944159e90 2023-10-16T10:08:19.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.616+0000 7f094b748640 1 -- 172.21.15.84:0/3786405673 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f092806af90 msgr2=0x7f092806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:19.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.616+0000 7f094b748640 1 --2- 172.21.15.84:0/3786405673 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f092806af90 0x7f092806d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0940008850 tx=0x7f0940008040 comp rx=0 tx=0).stop 2023-10-16T10:08:19.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.616+0000 7f094b748640 1 -- 172.21.15.84:0/3786405673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0944159e90 msgr2=0x7f09440acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:19.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.616+0000 7f094b748640 1 --2- 172.21.15.84:0/3786405673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0944159e90 0x7f09440acdd0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f093c009e80 tx=0x7f093c004a40 comp rx=0 tx=0).stop 2023-10-16T10:08:19.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.617+0000 7f094b748640 1 -- 172.21.15.84:0/3786405673 shutdown_connections 2023-10-16T10:08:19.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.617+0000 7f094b748640 1 --2- 172.21.15.84:0/3786405673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0944159e90 0x7f09440acdd0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:19.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.617+0000 7f094b748640 1 --2- 172.21.15.84:0/3786405673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0944159ab0 0x7f09440ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:19.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.617+0000 7f094b748640 1 --2- 172.21.15.84:0/3786405673 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f092806af90 0x7f092806d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:19.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.617+0000 7f094b748640 1 --2- 172.21.15.84:0/3786405673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f094406c140 0x7f0944149d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:19.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.617+0000 7f094b748640 1 -- 172.21.15.84:0/3786405673 >> 172.21.15.84:0/3786405673 conn(0x7f09440ce9b0 msgr2=0x7f09440b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:19.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.617+0000 7f094b748640 1 -- 172.21.15.84:0/3786405673 shutdown_connections 2023-10-16T10:08:19.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:19.617+0000 7f094b748640 1 -- 172.21.15.84:0/3786405673 wait complete. 2023-10-16T10:08:19.636 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:08:19.636 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:08:19.753 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:08:19.884 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:08:19.884 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:08:19.885 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:08:19.946 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:08:19.947 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:08:19.948 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:08:20.011 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:08:20.012 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:08:20.013 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:08:20.072 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:08:20.073 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:08:20.073 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:08:20.132 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:08:20.133 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T10:08:20.133 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:08:20.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.354+0000 7f45d831d640 1 Processor -- start 2023-10-16T10:08:20.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.354+0000 7f45d831d640 1 -- start start 2023-10-16T10:08:20.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.355+0000 7f45d831d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f45d0070fd0 0x7f45d00b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:20.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.355+0000 7f45d831d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d0071450 con 0x7f45d00adf30 2023-10-16T10:08:20.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.355+0000 7f45d831d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d00cc600 con 0x7f45d00adb50 2023-10-16T10:08:20.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.355+0000 7f45d831d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d00cc8e0 con 0x7f45d0070fd0 2023-10-16T10:08:20.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.356+0000 7f45d58b8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f45d0070fd0 0x7f45d00b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:20.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.356+0000 7f45d68ba640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f45d00adf30 0x7f45d0164f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:41568/0 (socket says 172.21.15.84:41568) 2023-10-16T10:08:20.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.356+0000 7f45d68ba640 1 -- 172.21.15.84:0/2263821152 learned_addr learned my addr 172.21.15.84:0/2263821152 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:20.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.356+0000 7f45d58b8640 1 -- 172.21.15.84:0/2263821152 >> v1:172.21.15.157:6789/0 conn(0x7f45d00adb50 legacy=0x7f45d006f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:20.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.356+0000 7f45d58b8640 1 -- 172.21.15.84:0/2263821152 >> v1:172.21.15.84:6789/0 conn(0x7f45d00adf30 legacy=0x7f45d0164f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:20.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.356+0000 7f45d58b8640 1 -- 172.21.15.84:0/2263821152 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d00ccbc0 con 0x7f45d0070fd0 2023-10-16T10:08:20.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.357+0000 7f45d58b8640 1 --2- 172.21.15.84:0/2263821152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45d0070fd0 0x7f45d00b0410 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f45c00097a0 tx=0x7f45c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade4eba6e8c552d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:20.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.358+0000 7f45d50b7640 1 -- 172.21.15.84:0/2263821152 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0031830 con 0x7f45d0070fd0 2023-10-16T10:08:20.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.358+0000 7f45d50b7640 1 -- 172.21.15.84:0/2263821152 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c0031990 con 0x7f45d0070fd0 2023-10-16T10:08:20.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.358+0000 7f45d50b7640 1 -- 172.21.15.84:0/2263821152 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0031d20 con 0x7f45d0070fd0 2023-10-16T10:08:20.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.358+0000 7f45d831d640 1 -- 172.21.15.84:0/2263821152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45d0070fd0 msgr2=0x7f45d00b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.358+0000 7f45d831d640 1 --2- 172.21.15.84:0/2263821152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45d0070fd0 0x7f45d00b0410 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f45c00097a0 tx=0x7f45c002b8d0 comp rx=0 tx=0).stop 2023-10-16T10:08:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.358+0000 7f45d831d640 1 -- 172.21.15.84:0/2263821152 shutdown_connections 2023-10-16T10:08:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.358+0000 7f45d831d640 1 --2- 172.21.15.84:0/2263821152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45d0070fd0 0x7f45d00b0410 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.358+0000 7f45d831d640 1 -- 172.21.15.84:0/2263821152 >> 172.21.15.84:0/2263821152 conn(0x7f45d00cb410 msgr2=0x7f45d00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.358+0000 7f45d831d640 1 -- 172.21.15.84:0/2263821152 shutdown_connections 2023-10-16T10:08:20.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.359+0000 7f45d831d640 1 -- 172.21.15.84:0/2263821152 wait complete. 2023-10-16T10:08:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.360+0000 7f45d831d640 1 Processor -- start 2023-10-16T10:08:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.360+0000 7f45d831d640 1 -- start start 2023-10-16T10:08:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.361+0000 7f45d831d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d0070fd0 0x7f45d0164570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.361+0000 7f45d831d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45d00adb50 0x7f45d0161a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.361+0000 7f45d831d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d00adf30 0x7f45d0161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.361+0000 7f45d831d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d00c3dd0 con 0x7f45d00adb50 2023-10-16T10:08:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.361+0000 7f45d831d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d01625d0 con 0x7f45d00adf30 2023-10-16T10:08:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.361+0000 7f45d831d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d01628b0 con 0x7f45d0070fd0 2023-10-16T10:08:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.362+0000 7f45d58b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45d00adb50 0x7f45d0161a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.362+0000 7f45d58b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45d00adb50 0x7f45d0161a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44372/0 (socket says 172.21.15.84:44372) 2023-10-16T10:08:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.362+0000 7f45d58b8640 1 -- 172.21.15.84:0/2942947628 learned_addr learned my addr 172.21.15.84:0/2942947628 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:20.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.362+0000 7f45d68ba640 1 --2- 172.21.15.84:0/2942947628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d00adf30 0x7f45d0161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:20.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.362+0000 7f45d58b8640 1 -- 172.21.15.84:0/2942947628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d0070fd0 msgr2=0x7f45d0164570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:20.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.362+0000 7f45d60b9640 1 --2- 172.21.15.84:0/2942947628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d0070fd0 0x7f45d0164570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:20.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.362+0000 7f45d58b8640 1 --2- 172.21.15.84:0/2942947628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d0070fd0 0x7f45d0164570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:20.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.362+0000 7f45d58b8640 1 -- 172.21.15.84:0/2942947628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d00adf30 msgr2=0x7f45d0161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:20.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.362+0000 7f45d58b8640 1 --2- 172.21.15.84:0/2942947628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d00adf30 0x7f45d0161fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:20.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.362+0000 7f45d58b8640 1 -- 172.21.15.84:0/2942947628 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c0009400 con 0x7f45d00adb50 2023-10-16T10:08:20.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.362+0000 7f45d58b8640 1 --2- 172.21.15.84:0/2942947628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45d00adb50 0x7f45d0161a60 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f45c0000c00 tx=0x7f45c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:20.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.363+0000 7f45c6ffd640 1 -- 172.21.15.84:0/2942947628 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c003aa60 con 0x7f45d00adb50 2023-10-16T10:08:20.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.363+0000 7f45c6ffd640 1 -- 172.21.15.84:0/2942947628 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c003abc0 con 0x7f45d00adb50 2023-10-16T10:08:20.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.363+0000 7f45c6ffd640 1 -- 172.21.15.84:0/2942947628 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c00443f0 con 0x7f45d00adb50 2023-10-16T10:08:20.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.363+0000 7f45d831d640 1 -- 172.21.15.84:0/2942947628 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d0162b90 con 0x7f45d00adb50 2023-10-16T10:08:20.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.363+0000 7f45d831d640 1 -- 172.21.15.84:0/2942947628 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d0163150 con 0x7f45d00adb50 2023-10-16T10:08:20.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.364+0000 7f45d831d640 1 -- 172.21.15.84:0/2942947628 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d0163340 con 0x7f45d00adb50 2023-10-16T10:08:20.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.365+0000 7f45c6ffd640 1 -- 172.21.15.84:0/2942947628 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f45c004d450 con 0x7f45d00adb50 2023-10-16T10:08:20.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.366+0000 7f45c6ffd640 1 --2- 172.21.15.84:0/2942947628 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f45b406afc0 0x7f45b406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:20.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.366+0000 7f45c6ffd640 1 -- 172.21.15.84:0/2942947628 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5745+0+0 (secure 0 0 0) 0x7f45c00abaa0 con 0x7f45d00adb50 2023-10-16T10:08:20.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.366+0000 7f45d60b9640 1 --2- 172.21.15.84:0/2942947628 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f45b406afc0 0x7f45b406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:20.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.367+0000 7f45d60b9640 1 --2- 172.21.15.84:0/2942947628 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f45b406afc0 0x7f45b406d480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f45cc003d90 tx=0x7f45cc009040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:20.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.370+0000 7f45c6ffd640 1 -- 172.21.15.84:0/2942947628 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f45c00768f0 con 0x7f45d00adb50 2023-10-16T10:08:20.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.604+0000 7f45d831d640 1 -- 172.21.15.84:0/2942947628 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45d004e160 con 0x7f45d00adb50 2023-10-16T10:08:20.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.606+0000 7f45c6ffd640 1 -- 172.21.15.84:0/2942947628 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f45c0035070 con 0x7f45d00adb50 2023-10-16T10:08:20.607 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:20.608 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":64,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2012273152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2012273152},{"type":"v1","addr":"172.21.15.157:6835","nonce":2012273152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/435606514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":435606514},{"type":"v1","addr":"172.21.15.157:6833","nonce":435606514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2286048798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2286048798},{"type":"v1","addr":"172.21.15.84:6836","nonce":2286048798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2628067877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2628067877},{"type":"v1","addr":"172.21.15.84:6837","nonce":2628067877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[]} 2023-10-16T10:08:20.608 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 64 2023-10-16T10:08:20.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.608+0000 7f45d831d640 1 -- 172.21.15.84:0/2942947628 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f45b406afc0 msgr2=0x7f45b406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:20.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.608+0000 7f45d831d640 1 --2- 172.21.15.84:0/2942947628 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f45b406afc0 0x7f45b406d480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f45cc003d90 tx=0x7f45cc009040 comp rx=0 tx=0).stop 2023-10-16T10:08:20.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.608+0000 7f45d831d640 1 -- 172.21.15.84:0/2942947628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45d00adb50 msgr2=0x7f45d0161a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:20.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.608+0000 7f45d831d640 1 --2- 172.21.15.84:0/2942947628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45d00adb50 0x7f45d0161a60 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f45c0000c00 tx=0x7f45c00046c0 comp rx=0 tx=0).stop 2023-10-16T10:08:20.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.608+0000 7f45d831d640 1 -- 172.21.15.84:0/2942947628 shutdown_connections 2023-10-16T10:08:20.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.608+0000 7f45d831d640 1 --2- 172.21.15.84:0/2942947628 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f45b406afc0 0x7f45b406d480 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:20.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.608+0000 7f45d831d640 1 --2- 172.21.15.84:0/2942947628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d00adf30 0x7f45d0161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:20.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.608+0000 7f45d831d640 1 --2- 172.21.15.84:0/2942947628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45d00adb50 0x7f45d0161a60 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:20.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.608+0000 7f45d831d640 1 --2- 172.21.15.84:0/2942947628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d0070fd0 0x7f45d0164570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:20.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.608+0000 7f45d831d640 1 -- 172.21.15.84:0/2942947628 >> 172.21.15.84:0/2942947628 conn(0x7f45d00cb410 msgr2=0x7f45d00aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:20.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.609+0000 7f45d831d640 1 -- 172.21.15.84:0/2942947628 shutdown_connections 2023-10-16T10:08:20.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.609+0000 7f45d831d640 1 -- 172.21.15.84:0/2942947628 wait complete. 2023-10-16T10:08:20.629 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T10:08:20.630 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T10:08:20.630 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T10:08:20.630 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:08:20.631 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T10:08:20.631 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T10:08:20.631 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T10:08:20.631 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:08:20.632 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T10:08:20.632 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T10:08:20.632 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T10:08:20.632 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:08:20.633 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T10:08:20.633 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T10:08:20.633 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T10:08:20.633 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:08:20.650 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T10:08:20.650 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T10:08:20.651 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T10:08:20.651 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T10:08:20.652 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T10:08:20.653 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T10:08:20.654 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T10:08:20.654 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T10:08:20.699 INFO:tasks.ceph.mds.a:Started 2023-10-16T10:08:20.700 DEBUG:teuthology.parallel:result is None 2023-10-16T10:08:20.702 INFO:tasks.ceph.mds.b:Started 2023-10-16T10:08:20.703 DEBUG:teuthology.parallel:result is None 2023-10-16T10:08:20.706 INFO:tasks.ceph.mds.c:Started 2023-10-16T10:08:20.706 DEBUG:teuthology.parallel:result is None 2023-10-16T10:08:20.710 INFO:tasks.ceph.mds.d:Started 2023-10-16T10:08:20.710 DEBUG:teuthology.parallel:result is None 2023-10-16T10:08:20.711 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T10:08:20.857 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T10:08:20.859 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T10:08:20.867 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-10-16T10:08:20.868 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T10:08:20.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.899+0000 7f8a808a2640 1 Processor -- start 2023-10-16T10:08:20.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.900+0000 7f8a808a2640 1 -- start start 2023-10-16T10:08:20.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.901+0000 7f8a808a2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8a7c0caa80 0x7f8a7c069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:20.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.901+0000 7f8a808a2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8a7c06a2e0 0x7f8a7c1650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:20.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.901+0000 7f8a808a2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7c0caef0 con 0x7f8a7c0addb0 2023-10-16T10:08:20.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.901+0000 7f8a808a2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7c161a50 con 0x7f8a7c06a2e0 2023-10-16T10:08:20.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.901+0000 7f8a808a2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7c06a6f0 con 0x7f8a7c0caa80 2023-10-16T10:08:20.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.901+0000 7f8a7ad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8a7c06a2e0 0x7f8a7c1650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:20.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.901+0000 7f8a7ad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8a7c06a2e0 0x7f8a7c1650d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44408/0 (socket says 172.21.15.84:44408) 2023-10-16T10:08:20.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.901+0000 7f8a7ad9d640 1 -- 172.21.15.84:0/1491000510 learned_addr learned my addr 172.21.15.84:0/1491000510 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:20.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.902+0000 7f8a71d9b640 1 --2- 172.21.15.84:0/1491000510 >> v2:172.21.15.157:3300/0 conn(0x7f8a7c0caa80 0x7f8a7c069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:20.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.902+0000 7f8a7ad9d640 1 -- 172.21.15.84:0/1491000510 >> v2:172.21.15.157:3300/0 conn(0x7f8a7c0caa80 msgr2=0x7f8a7c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:20.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.902+0000 7f8a7ad9d640 1 --2- 172.21.15.84:0/1491000510 >> v2:172.21.15.157:3300/0 conn(0x7f8a7c0caa80 0x7f8a7c069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:20.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.902+0000 7f8a7ad9d640 1 -- 172.21.15.84:0/1491000510 >> v1:172.21.15.157:6789/0 conn(0x7f8a7c0addb0 legacy=0x7f8a7c0c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:20.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.902+0000 7f8a7ad9d640 1 -- 172.21.15.84:0/1491000510 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c0cb140 con 0x7f8a7c06a2e0 2023-10-16T10:08:20.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.903+0000 7f8a7ad9d640 1 --2- 172.21.15.84:0/1491000510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7c06a2e0 0x7f8a7c1650d0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0cdbe0 tx=0x7f8a7402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961974dd6a2b8609 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:20.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.903+0000 7f8a79d9b640 1 -- 172.21.15.84:0/1491000510 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7402cb10 con 0x7f8a7c06a2e0 2023-10-16T10:08:20.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.903+0000 7f8a79d9b640 1 -- 172.21.15.84:0/1491000510 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7402cc70 con 0x7f8a7c06a2e0 2023-10-16T10:08:20.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.903+0000 7f8a79d9b640 1 -- 172.21.15.84:0/1491000510 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a740343f0 con 0x7f8a7c06a2e0 2023-10-16T10:08:20.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.904+0000 7f8a808a2640 1 -- 172.21.15.84:0/1491000510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7c06a2e0 msgr2=0x7f8a7c1650d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:20.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.904+0000 7f8a808a2640 1 --2- 172.21.15.84:0/1491000510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7c06a2e0 0x7f8a7c1650d0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0cdbe0 tx=0x7f8a7402bf70 comp rx=0 tx=0).stop 2023-10-16T10:08:20.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.904+0000 7f8a808a2640 1 -- 172.21.15.84:0/1491000510 shutdown_connections 2023-10-16T10:08:20.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.904+0000 7f8a808a2640 1 --2- 172.21.15.84:0/1491000510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7c06a2e0 0x7f8a7c1650d0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:20.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.904+0000 7f8a808a2640 1 --2- 172.21.15.84:0/1491000510 >> v2:172.21.15.157:3300/0 conn(0x7f8a7c0caa80 0x7f8a7c069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:20.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.904+0000 7f8a808a2640 1 -- 172.21.15.84:0/1491000510 >> 172.21.15.84:0/1491000510 conn(0x7f8a7c0ce9b0 msgr2=0x7f8a7c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:20.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.904+0000 7f8a808a2640 1 -- 172.21.15.84:0/1491000510 shutdown_connections 2023-10-16T10:08:20.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.904+0000 7f8a808a2640 1 -- 172.21.15.84:0/1491000510 wait complete. 2023-10-16T10:08:20.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.906+0000 7f8a808a2640 1 Processor -- start 2023-10-16T10:08:20.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.906+0000 7f8a808a2640 1 -- start start 2023-10-16T10:08:20.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.907+0000 7f8a808a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7c06a2e0 0x7f8a7c15dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:20.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.907+0000 7f8a808a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a7c0addb0 0x7f8a7c0c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:20.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.907+0000 7f8a808a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c0caa80 0x7f8a7c1f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:20.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.907+0000 7f8a808a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c1f72b0 con 0x7f8a7c06a2e0 2023-10-16T10:08:20.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.907+0000 7f8a808a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c0c5800 con 0x7f8a7c0caa80 2023-10-16T10:08:20.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.907+0000 7f8a808a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c0c70b0 con 0x7f8a7c0addb0 2023-10-16T10:08:20.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.907+0000 7f8a7a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7c06a2e0 0x7f8a7c15dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:20.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.907+0000 7f8a7a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7c06a2e0 0x7f8a7c15dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44412/0 (socket says 172.21.15.84:44412) 2023-10-16T10:08:20.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.907+0000 7f8a7a59c640 1 -- 172.21.15.84:0/896422014 learned_addr learned my addr 172.21.15.84:0/896422014 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:20.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.907+0000 7f8a71d9b640 1 --2- 172.21.15.84:0/896422014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a7c0addb0 0x7f8a7c0c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:20.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.907+0000 7f8a7ad9d640 1 --2- 172.21.15.84:0/896422014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c0caa80 0x7f8a7c1f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:20.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.907+0000 7f8a7a59c640 1 -- 172.21.15.84:0/896422014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a7c0addb0 msgr2=0x7f8a7c0c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:20.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.907+0000 7f8a7a59c640 1 --2- 172.21.15.84:0/896422014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a7c0addb0 0x7f8a7c0c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:20.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.907+0000 7f8a7a59c640 1 -- 172.21.15.84:0/896422014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c0caa80 msgr2=0x7f8a7c1f6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:20.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.907+0000 7f8a7a59c640 1 --2- 172.21.15.84:0/896422014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c0caa80 0x7f8a7c1f6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:20.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.907+0000 7f8a7a59c640 1 -- 172.21.15.84:0/896422014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a74009b30 con 0x7f8a7c06a2e0 2023-10-16T10:08:20.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.908+0000 7f8a7a59c640 1 --2- 172.21.15.84:0/896422014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7c06a2e0 0x7f8a7c15dee0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400c450 tx=0x7f8a6400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:20.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.908+0000 7f8a737fe640 1 -- 172.21.15.84:0/896422014 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a640052d0 con 0x7f8a7c06a2e0 2023-10-16T10:08:20.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.908+0000 7f8a737fe640 1 -- 172.21.15.84:0/896422014 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a64005430 con 0x7f8a7c06a2e0 2023-10-16T10:08:20.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.908+0000 7f8a737fe640 1 -- 172.21.15.84:0/896422014 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a64012870 con 0x7f8a7c06a2e0 2023-10-16T10:08:20.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.908+0000 7f8a808a2640 1 -- 172.21.15.84:0/896422014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7c0c7390 con 0x7f8a7c06a2e0 2023-10-16T10:08:20.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.908+0000 7f8a808a2640 1 -- 172.21.15.84:0/896422014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7c0c7950 con 0x7f8a7c06a2e0 2023-10-16T10:08:20.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.909+0000 7f8a808a2640 1 -- 172.21.15.84:0/896422014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a7c04e160 con 0x7f8a7c06a2e0 2023-10-16T10:08:20.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.910+0000 7f8a737fe640 1 -- 172.21.15.84:0/896422014 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8a640129d0 con 0x7f8a7c06a2e0 2023-10-16T10:08:20.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.911+0000 7f8a737fe640 1 --2- 172.21.15.84:0/896422014 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f8a5c06af90 0x7f8a5c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:20.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.911+0000 7f8a737fe640 1 -- 172.21.15.84:0/896422014 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5745+0+0 (secure 0 0 0) 0x7f8a64089250 con 0x7f8a7c06a2e0 2023-10-16T10:08:20.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.911+0000 7f8a71d9b640 1 --2- 172.21.15.84:0/896422014 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f8a5c06af90 0x7f8a5c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:20.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.912+0000 7f8a71d9b640 1 --2- 172.21.15.84:0/896422014 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f8a5c06af90 0x7f8a5c06d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0c28e0 tx=0x7f8a6c008040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:20.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:20.915+0000 7f8a737fe640 1 -- 172.21.15.84:0/896422014 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8a640541d0 con 0x7f8a7c06a2e0 2023-10-16T10:08:21.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.115+0000 7f8a808a2640 1 -- 172.21.15.84:0/896422014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8a7c06c5b0 con 0x7f8a7c06a2e0 2023-10-16T10:08:21.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.172+0000 7f8a737fe640 1 -- 172.21.15.84:0/896422014 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v132) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8a640577f0 con 0x7f8a7c06a2e0 2023-10-16T10:08:21.174 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-10-16T10:08:21.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.175+0000 7f8a808a2640 1 -- 172.21.15.84:0/896422014 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f8a5c06af90 msgr2=0x7f8a5c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:21.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.175+0000 7f8a808a2640 1 --2- 172.21.15.84:0/896422014 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f8a5c06af90 0x7f8a5c06d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0c28e0 tx=0x7f8a6c008040 comp rx=0 tx=0).stop 2023-10-16T10:08:21.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.175+0000 7f8a808a2640 1 -- 172.21.15.84:0/896422014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7c06a2e0 msgr2=0x7f8a7c15dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:21.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.175+0000 7f8a808a2640 1 --2- 172.21.15.84:0/896422014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7c06a2e0 0x7f8a7c15dee0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400c450 tx=0x7f8a6400c910 comp rx=0 tx=0).stop 2023-10-16T10:08:21.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.175+0000 7f8a808a2640 1 -- 172.21.15.84:0/896422014 shutdown_connections 2023-10-16T10:08:21.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.175+0000 7f8a808a2640 1 --2- 172.21.15.84:0/896422014 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f8a5c06af90 0x7f8a5c06d450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.175+0000 7f8a808a2640 1 --2- 172.21.15.84:0/896422014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c0caa80 0x7f8a7c1f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.175+0000 7f8a808a2640 1 --2- 172.21.15.84:0/896422014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a7c0addb0 0x7f8a7c0c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.175+0000 7f8a808a2640 1 --2- 172.21.15.84:0/896422014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7c06a2e0 0x7f8a7c15dee0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.175+0000 7f8a808a2640 1 -- 172.21.15.84:0/896422014 >> 172.21.15.84:0/896422014 conn(0x7f8a7c0ce9b0 msgr2=0x7f8a7c0cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:21.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.176+0000 7f8a808a2640 1 -- 172.21.15.84:0/896422014 shutdown_connections 2023-10-16T10:08:21.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.176+0000 7f8a808a2640 1 -- 172.21.15.84:0/896422014 wait complete. 2023-10-16T10:08:21.197 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T10:08:21.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.399+0000 7f665765d640 1 Processor -- start 2023-10-16T10:08:21.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.399+0000 7f665765d640 1 -- start start 2023-10-16T10:08:21.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.400+0000 7f665765d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6650159b20 0x7f665006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:21.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.400+0000 7f665765d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665006f7f0 con 0x7f66500b0b20 2023-10-16T10:08:21.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.400+0000 7f665765d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665006c5e0 con 0x7f6650159f00 2023-10-16T10:08:21.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.400+0000 7f665765d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665006c8c0 con 0x7f6650159b20 2023-10-16T10:08:21.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.400+0000 7f6655bfa640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f66500b0b20 0x7f665006fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:41578/0 (socket says 172.21.15.84:41578) 2023-10-16T10:08:21.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.400+0000 7f6655bfa640 1 -- 172.21.15.84:0/1938375999 learned_addr learned my addr 172.21.15.84:0/1938375999 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:21.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.401+0000 7f66553f9640 1 --2- 172.21.15.84:0/1938375999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6650159b20 0x7f665006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:21.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.401+0000 7f6647fff640 1 -- 172.21.15.84:0/1938375999 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585182586 0 0) 0x7f665006f7f0 con 0x7f66500b0b20 2023-10-16T10:08:21.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.401+0000 7f6647fff640 1 -- 172.21.15.84:0/1938375999 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f663c003520 con 0x7f66500b0b20 2023-10-16T10:08:21.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.401+0000 7f6647fff640 1 -- 172.21.15.84:0/1938375999 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 31942893 0 0) 0x7f663c003520 con 0x7f66500b0b20 2023-10-16T10:08:21.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.401+0000 7f6647fff640 1 -- 172.21.15.84:0/1938375999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6650159b20 msgr2=0x7f665006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:21.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.401+0000 7f6647fff640 1 --2- 172.21.15.84:0/1938375999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6650159b20 0x7f665006bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.401+0000 7f6647fff640 1 -- 172.21.15.84:0/1938375999 >> v1:172.21.15.157:6789/0 conn(0x7f6650159f00 legacy=0x7f66500b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:21.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.401+0000 7f6647fff640 1 -- 172.21.15.84:0/1938375999 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665006cba0 con 0x7f66500b0b20 2023-10-16T10:08:21.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.401+0000 7f6647fff640 1 -- 172.21.15.84:0/1938375999 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f66480019f0 con 0x7f66500b0b20 2023-10-16T10:08:21.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.401+0000 7f6647fff640 1 -- 172.21.15.84:0/1938375999 >> v1:172.21.15.84:6789/0 conn(0x7f66500b0b20 legacy=0x7f665006fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:21.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.401+0000 7f6647fff640 1 --2- 172.21.15.84:0/1938375999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f663c0039a0 0x7f663c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:21.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.401+0000 7f6647fff640 1 --2- 172.21.15.84:0/1938375999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f663c0047c0 0x7f663c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:21.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.401+0000 7f6647fff640 1 --2- 172.21.15.84:0/1938375999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6650159b20 0x7f663c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:21.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.401+0000 7f6647fff640 1 -- 172.21.15.84:0/1938375999 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665006c8c0 con 0x7f663c0039a0 2023-10-16T10:08:21.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.401+0000 7f6647fff640 1 -- 172.21.15.84:0/1938375999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c00ba30 con 0x7f663c0047c0 2023-10-16T10:08:21.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.401+0000 7f6647fff640 1 -- 172.21.15.84:0/1938375999 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c00bbf0 con 0x7f6650159b20 2023-10-16T10:08:21.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.402+0000 7f66553f9640 1 --2- 172.21.15.84:0/1938375999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f663c0039a0 0x7f663c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:21.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.402+0000 7f6654bf8640 1 --2- 172.21.15.84:0/1938375999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f663c0047c0 0x7f663c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:21.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.402+0000 7f66553f9640 1 -- 172.21.15.84:0/1938375999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6650159b20 msgr2=0x7f663c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:21.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.402+0000 7f66553f9640 1 --2- 172.21.15.84:0/1938375999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6650159b20 0x7f663c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.402+0000 7f66553f9640 1 -- 172.21.15.84:0/1938375999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f663c0047c0 msgr2=0x7f663c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:21.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.402+0000 7f66553f9640 1 --2- 172.21.15.84:0/1938375999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f663c0047c0 0x7f663c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.402+0000 7f66553f9640 1 -- 172.21.15.84:0/1938375999 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c003520 con 0x7f663c0039a0 2023-10-16T10:08:21.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.402+0000 7f6654bf8640 1 --2- 172.21.15.84:0/1938375999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f663c0047c0 0x7f663c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:21.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.403+0000 7f66553f9640 1 --2- 172.21.15.84:0/1938375999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f663c0039a0 0x7f663c003dc0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f664c00bba0 tx=0x7f663c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae05ce8d468e33b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:21.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.403+0000 7f6647fff640 1 -- 172.21.15.84:0/1938375999 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c033600 con 0x7f663c0039a0 2023-10-16T10:08:21.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.403+0000 7f6647fff640 1 -- 172.21.15.84:0/1938375999 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664c02f050 con 0x7f663c0039a0 2023-10-16T10:08:21.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.403+0000 7f6647fff640 1 -- 172.21.15.84:0/1938375999 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c033960 con 0x7f663c0039a0 2023-10-16T10:08:21.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.404+0000 7f665765d640 1 -- 172.21.15.84:0/1938375999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f663c0039a0 msgr2=0x7f663c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:21.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.404+0000 7f665765d640 1 --2- 172.21.15.84:0/1938375999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f663c0039a0 0x7f663c003dc0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f664c00bba0 tx=0x7f663c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.404+0000 7f6655bfa640 1 -- 172.21.15.84:0/1938375999 reap_dead start 2023-10-16T10:08:21.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.404+0000 7f665765d640 1 -- 172.21.15.84:0/1938375999 shutdown_connections 2023-10-16T10:08:21.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.404+0000 7f665765d640 1 -- 172.21.15.84:0/1938375999 >> 172.21.15.84:0/1938375999 conn(0x7f66500cb410 msgr2=0x7f66500cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:21.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.404+0000 7f665765d640 1 -- 172.21.15.84:0/1938375999 shutdown_connections 2023-10-16T10:08:21.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.404+0000 7f665765d640 1 -- 172.21.15.84:0/1938375999 wait complete. 2023-10-16T10:08:21.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.406+0000 7f665765d640 1 Processor -- start 2023-10-16T10:08:21.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.406+0000 7f665765d640 1 -- start start 2023-10-16T10:08:21.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.407+0000 7f665765d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f663c0039a0 0x7f66500c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:21.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.407+0000 7f665765d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f665015a310 0x7f665015a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:21.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.407+0000 7f665765d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6650144a80 0x7f6650144f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:21.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.407+0000 7f665765d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665015acd0 con 0x7f665015a310 2023-10-16T10:08:21.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.407+0000 7f665765d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66500c8470 con 0x7f6650144a80 2023-10-16T10:08:21.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.407+0000 7f665765d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66500c6e90 con 0x7f663c0039a0 2023-10-16T10:08:21.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.407+0000 7f6654bf8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f665015a310 0x7f665015a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:21.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.407+0000 7f6654bf8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f665015a310 0x7f665015a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44424/0 (socket says 172.21.15.84:44424) 2023-10-16T10:08:21.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.407+0000 7f6654bf8640 1 -- 172.21.15.84:0/3229620167 learned_addr learned my addr 172.21.15.84:0/3229620167 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:21.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.408+0000 7f66553f9640 1 --2- 172.21.15.84:0/3229620167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f663c0039a0 0x7f66500c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:21.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.408+0000 7f6655bfa640 1 --2- 172.21.15.84:0/3229620167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6650144a80 0x7f6650144f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:21.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.408+0000 7f6654bf8640 1 -- 172.21.15.84:0/3229620167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f663c0039a0 msgr2=0x7f66500c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:21.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.408+0000 7f6654bf8640 1 --2- 172.21.15.84:0/3229620167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f663c0039a0 0x7f66500c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.408+0000 7f6654bf8640 1 -- 172.21.15.84:0/3229620167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6650144a80 msgr2=0x7f6650144f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:21.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.408+0000 7f6654bf8640 1 --2- 172.21.15.84:0/3229620167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6650144a80 0x7f6650144f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.408+0000 7f6654bf8640 1 -- 172.21.15.84:0/3229620167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66500c7170 con 0x7f665015a310 2023-10-16T10:08:21.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.408+0000 7f6654bf8640 1 --2- 172.21.15.84:0/3229620167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f665015a310 0x7f665015a790 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f664000d7a0 tx=0x7f664000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:21.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.408+0000 7f6645ffb640 1 -- 172.21.15.84:0/3229620167 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6640017070 con 0x7f665015a310 2023-10-16T10:08:21.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.408+0000 7f6645ffb640 1 -- 172.21.15.84:0/3229620167 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6640005070 con 0x7f665015a310 2023-10-16T10:08:21.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.408+0000 7f665765d640 1 -- 172.21.15.84:0/3229620167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66500c7450 con 0x7f665015a310 2023-10-16T10:08:21.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.408+0000 7f6645ffb640 1 -- 172.21.15.84:0/3229620167 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6640005420 con 0x7f665015a310 2023-10-16T10:08:21.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.408+0000 7f665765d640 1 -- 172.21.15.84:0/3229620167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66500c7a10 con 0x7f665015a310 2023-10-16T10:08:21.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.410+0000 7f665765d640 1 -- 172.21.15.84:0/3229620167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f665004e160 con 0x7f665015a310 2023-10-16T10:08:21.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.410+0000 7f6645ffb640 1 -- 172.21.15.84:0/3229620167 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6640007030 con 0x7f665015a310 2023-10-16T10:08:21.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.411+0000 7f6645ffb640 1 --2- 172.21.15.84:0/3229620167 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f663406af90 0x7f663406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:21.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.412+0000 7f6645ffb640 1 -- 172.21.15.84:0/3229620167 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f6640088e20 con 0x7f665015a310 2023-10-16T10:08:21.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.412+0000 7f66553f9640 1 --2- 172.21.15.84:0/3229620167 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f663406af90 0x7f663406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:21.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.412+0000 7f66553f9640 1 --2- 172.21.15.84:0/3229620167 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f663406af90 0x7f663406d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f664c035000 tx=0x7f664c008030 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:21.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.416+0000 7f6645ffb640 1 -- 172.21.15.84:0/3229620167 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6640054090 con 0x7f665015a310 2023-10-16T10:08:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.679+0000 7f665765d640 1 -- 172.21.15.84:0/3229620167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f665006d390 con 0x7f665015a310 2023-10-16T10:08:21.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.680+0000 7f6645ffb640 1 -- 172.21.15.84:0/3229620167 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f66400576b0 con 0x7f665015a310 2023-10-16T10:08:21.681 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:21.682 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T10:08:21.682 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-10-16T10:08:21.682 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.682 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-10-16T10:08:21.683 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l6dhjKRAA0pMvY6AK6qtFXDprNhwsqg==", 2023-10-16T10:08:21.683 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.683 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:08:21.683 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:08:21.683 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:08:21.684 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:08:21.684 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.684 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:08:21.684 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.684 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-10-16T10:08:21.685 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lFpMxEBAA0TLPRt5q3FwwBvMG019cjQ==", 2023-10-16T10:08:21.685 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.685 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:08:21.685 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:08:21.685 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:08:21.686 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:08:21.686 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.686 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:08:21.686 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.686 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-10-16T10:08:21.687 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l73RBOhAAOMEPxfUAMHhn5Blyvd5oMw==", 2023-10-16T10:08:21.687 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.687 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:08:21.687 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:08:21.687 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:08:21.687 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:08:21.688 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.688 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:08:21.688 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.688 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-10-16T10:08:21.688 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lcTIRIhAA0+FWm4b4yw77LD3+WKbk7w==", 2023-10-16T10:08:21.689 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.689 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:08:21.689 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:08:21.689 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:08:21.689 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:08:21.690 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.690 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:08:21.690 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.690 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-10-16T10:08:21.690 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDYBi1l1zBLERAA3YLW3xIYQoRVw7KSkwN3Og==", 2023-10-16T10:08:21.691 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.691 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:08:21.691 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:08:21.691 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:08:21.691 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.692 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:08:21.692 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.692 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-10-16T10:08:21.692 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDbBi1lFUUxKRAAQbus8SmYprJRmWTjoCznNA==", 2023-10-16T10:08:21.692 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.693 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:08:21.693 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:08:21.693 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:08:21.693 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.694 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:08:21.694 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.694 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-10-16T10:08:21.694 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDeBi1l+9lBMxAArl6sb/h15uBG/a9lVH40Hg==", 2023-10-16T10:08:21.694 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.695 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:08:21.695 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:08:21.695 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:08:21.695 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.696 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:08:21.696 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.696 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-10-16T10:08:21.696 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDiBi1l2rnEERAA3oRD71xa0fEOynM8zKuBEg==", 2023-10-16T10:08:21.696 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.697 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:08:21.697 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:08:21.697 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:08:21.697 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.698 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:08:21.698 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.698 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-10-16T10:08:21.698 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDnBi1lR7oLChAAZto8fDovDbpOd3Fcnc7C6w==", 2023-10-16T10:08:21.699 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.699 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:08:21.699 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:08:21.699 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:08:21.699 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.700 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:08:21.700 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.700 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-10-16T10:08:21.700 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDqBi1lmjg/JBAAlINKRGBc0/UWJJHMVFw57w==", 2023-10-16T10:08:21.700 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.701 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:08:21.701 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:08:21.701 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:08:21.701 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.701 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:08:21.702 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.702 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-10-16T10:08:21.702 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDuBi1lu0kMARAAtuSeM8F3yr8CVrc3p1biaQ==", 2023-10-16T10:08:21.702 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.702 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:08:21.703 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:08:21.703 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:08:21.703 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.703 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:08:21.703 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.704 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-10-16T10:08:21.704 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDxBi1lf6mNGBAAHGbqzJ0fmDxQIsasboHMfg==", 2023-10-16T10:08:21.704 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.704 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:08:21.704 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:08:21.705 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:08:21.705 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.705 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:08:21.705 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.705 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-10-16T10:08:21.705 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ==", 2023-10-16T10:08:21.706 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.706 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:08:21.706 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow r", 2023-10-16T10:08:21.706 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T10:08:21.706 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.707 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:08:21.707 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.707 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-10-16T10:08:21.707 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1lHU+4BxAAd8ET8hQ8Y+EUwdTJr9/+Ig==", 2023-10-16T10:08:21.707 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.708 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:08:21.708 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:08:21.708 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:08:21.708 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:08:21.708 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.709 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:08:21.709 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.709 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-10-16T10:08:21.709 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lA1BOORAAFNyfqgEF34MVR9YX3KBKkQ==", 2023-10-16T10:08:21.709 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.710 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:08:21.710 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:08:21.710 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:08:21.710 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:08:21.710 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.710 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:08:21.711 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.711 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-10-16T10:08:21.711 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1l848oERAANMfrZz9JrlgNVh7ZFbtnaA==", 2023-10-16T10:08:21.711 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.711 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:08:21.712 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:08:21.712 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:08:21.712 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:08:21.712 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.712 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:08:21.713 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.713 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-10-16T10:08:21.713 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDRBi1lTAxTGRAAMptxnxy3Qit/cTCXie6CrQ==", 2023-10-16T10:08:21.713 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.713 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:08:21.713 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:08:21.714 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:08:21.714 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:08:21.714 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.714 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:08:21.714 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.715 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-10-16T10:08:21.715 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lPuJxHRAABAnGPVCrkq0JR4Y6iFGZlw==", 2023-10-16T10:08:21.715 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.715 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:08:21.715 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T10:08:21.716 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:08:21.716 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.716 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:08:21.716 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:21.716 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-10-16T10:08:21.716 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lLxnKEBAA+a/pOHUp7Sg3m4weAust5w==", 2023-10-16T10:08:21.717 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:08:21.717 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:08:21.717 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T10:08:21.717 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:08:21.717 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.718 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:21.718 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-10-16T10:08:21.718 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T10:08:21.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.682+0000 7f665765d640 1 -- 172.21.15.84:0/3229620167 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f663406af90 msgr2=0x7f663406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:21.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.682+0000 7f665765d640 1 --2- 172.21.15.84:0/3229620167 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f663406af90 0x7f663406d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f664c035000 tx=0x7f664c008030 comp rx=0 tx=0).stop 2023-10-16T10:08:21.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.682+0000 7f665765d640 1 -- 172.21.15.84:0/3229620167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f665015a310 msgr2=0x7f665015a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:21.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.682+0000 7f665765d640 1 --2- 172.21.15.84:0/3229620167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f665015a310 0x7f665015a790 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f664000d7a0 tx=0x7f664000dc60 comp rx=0 tx=0).stop 2023-10-16T10:08:21.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.682+0000 7f665765d640 1 -- 172.21.15.84:0/3229620167 shutdown_connections 2023-10-16T10:08:21.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.682+0000 7f665765d640 1 --2- 172.21.15.84:0/3229620167 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f663406af90 0x7f663406d450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.682+0000 7f665765d640 1 --2- 172.21.15.84:0/3229620167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6650144a80 0x7f6650144f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.682+0000 7f665765d640 1 --2- 172.21.15.84:0/3229620167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f665015a310 0x7f665015a790 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.682+0000 7f665765d640 1 --2- 172.21.15.84:0/3229620167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f663c0039a0 0x7f66500c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.682+0000 7f665765d640 1 -- 172.21.15.84:0/3229620167 >> 172.21.15.84:0/3229620167 conn(0x7f66500cb410 msgr2=0x7f66500ce8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:21.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.682+0000 7f665765d640 1 -- 172.21.15.84:0/3229620167 shutdown_connections 2023-10-16T10:08:21.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.683+0000 7f665765d640 1 -- 172.21.15.84:0/3229620167 wait complete. 2023-10-16T10:08:21.722 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:08:21.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.936+0000 7f5f86e61640 1 Processor -- start 2023-10-16T10:08:21.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.936+0000 7f5f86e61640 1 -- start start 2023-10-16T10:08:21.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.937+0000 7f5f86e61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f8015aaa0 0x7f5f8015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:21.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.937+0000 7f5f86e61640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5f8015ae80 0x7f5f800cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:21.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.937+0000 7f5f86e61640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8015ed60 con 0x7f5f800ccaa0 2023-10-16T10:08:21.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.938+0000 7f5f86e61640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f8006b810 con 0x7f5f8015ae80 2023-10-16T10:08:21.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.938+0000 7f5f86e61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f800707e0 con 0x7f5f8015aaa0 2023-10-16T10:08:21.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.938+0000 7f5f7ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5f8015ae80 0x7f5f800cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:21.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.938+0000 7f5f853fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f5f800ccaa0 0x7f5f80070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:41584/0 (socket says 172.21.15.84:41584) 2023-10-16T10:08:21.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.938+0000 7f5f853fe640 1 -- 172.21.15.84:0/40299909 learned_addr learned my addr 172.21.15.84:0/40299909 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:21.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.938+0000 7f5f84bfd640 1 --2- 172.21.15.84:0/40299909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f8015aaa0 0x7f5f8015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:21.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.938+0000 7f5f7ffff640 1 -- 172.21.15.84:0/40299909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f8015aaa0 msgr2=0x7f5f8015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:21.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.938+0000 7f5f7ffff640 1 --2- 172.21.15.84:0/40299909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f8015aaa0 0x7f5f8015e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.938+0000 7f5f7ffff640 1 -- 172.21.15.84:0/40299909 >> v1:172.21.15.84:6789/0 conn(0x7f5f800ccaa0 legacy=0x7f5f80070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:21.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.938+0000 7f5f7ffff640 1 -- 172.21.15.84:0/40299909 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8015ef90 con 0x7f5f8015ae80 2023-10-16T10:08:21.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.938+0000 7f5f7f7fe640 1 -- 172.21.15.84:0/40299909 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 468632225 0 0) 0x7f5f8015ed60 con 0x7f5f800ccaa0 2023-10-16T10:08:21.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.939+0000 7f5f7ffff640 1 --2- 172.21.15.84:0/40299909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f8015ae80 0x7f5f800cc560 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0097a0 tx=0x7f5f6c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32db055e470fa048 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:21.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.940+0000 7f5f7f7fe640 1 -- 172.21.15.84:0/40299909 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c02d040 con 0x7f5f8015ae80 2023-10-16T10:08:21.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.940+0000 7f5f7f7fe640 1 -- 172.21.15.84:0/40299909 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c031810 con 0x7f5f8015ae80 2023-10-16T10:08:21.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.940+0000 7f5f7f7fe640 1 -- 172.21.15.84:0/40299909 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c031ba0 con 0x7f5f8015ae80 2023-10-16T10:08:21.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.940+0000 7f5f86e61640 1 -- 172.21.15.84:0/40299909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f8015ae80 msgr2=0x7f5f800cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:21.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.940+0000 7f5f86e61640 1 --2- 172.21.15.84:0/40299909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f8015ae80 0x7f5f800cc560 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0097a0 tx=0x7f5f6c02b8b0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.940+0000 7f5f86e61640 1 -- 172.21.15.84:0/40299909 shutdown_connections 2023-10-16T10:08:21.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.940+0000 7f5f86e61640 1 --2- 172.21.15.84:0/40299909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f8015ae80 0x7f5f800cc560 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.940+0000 7f5f86e61640 1 --2- 172.21.15.84:0/40299909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f8015aaa0 0x7f5f8015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.940+0000 7f5f86e61640 1 -- 172.21.15.84:0/40299909 >> 172.21.15.84:0/40299909 conn(0x7f5f800cb410 msgr2=0x7f5f800cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:21.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.941+0000 7f5f86e61640 1 -- 172.21.15.84:0/40299909 shutdown_connections 2023-10-16T10:08:21.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.941+0000 7f5f86e61640 1 -- 172.21.15.84:0/40299909 wait complete. 2023-10-16T10:08:21.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.942+0000 7f5f86e61640 1 Processor -- start 2023-10-16T10:08:21.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.942+0000 7f5f86e61640 1 -- start start 2023-10-16T10:08:21.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.943+0000 7f5f86e61640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f800ccaa0 0x7f5f800c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:21.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.943+0000 7f5f86e61640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f8015aaa0 0x7f5f800c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:21.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.943+0000 7f5f86e61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f8015ae80 0x7f5f800c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:21.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.943+0000 7f5f86e61640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f801596e0 con 0x7f5f8015aaa0 2023-10-16T10:08:21.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.943+0000 7f5f86e61640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8015b320 con 0x7f5f800ccaa0 2023-10-16T10:08:21.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.943+0000 7f5f86e61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f800c1f00 con 0x7f5f8015ae80 2023-10-16T10:08:21.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.944+0000 7f5f7ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f8015aaa0 0x7f5f800c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:21.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.944+0000 7f5f7ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f8015aaa0 0x7f5f800c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44448/0 (socket says 172.21.15.84:44448) 2023-10-16T10:08:21.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.944+0000 7f5f7ffff640 1 -- 172.21.15.84:0/2557102386 learned_addr learned my addr 172.21.15.84:0/2557102386 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:21.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.944+0000 7f5f853fe640 1 --2- 172.21.15.84:0/2557102386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f8015ae80 0x7f5f800c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:21.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.944+0000 7f5f84bfd640 1 --2- 172.21.15.84:0/2557102386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f800ccaa0 0x7f5f800c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:21.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.944+0000 7f5f7ffff640 1 -- 172.21.15.84:0/2557102386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f8015ae80 msgr2=0x7f5f800c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:21.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.944+0000 7f5f7ffff640 1 --2- 172.21.15.84:0/2557102386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f8015ae80 0x7f5f800c4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.944+0000 7f5f7ffff640 1 -- 172.21.15.84:0/2557102386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f800ccaa0 msgr2=0x7f5f800c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:21.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.944+0000 7f5f7ffff640 1 --2- 172.21.15.84:0/2557102386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f800ccaa0 0x7f5f800c19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:21.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.944+0000 7f5f7ffff640 1 -- 172.21.15.84:0/2557102386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c009400 con 0x7f5f8015aaa0 2023-10-16T10:08:21.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.944+0000 7f5f84bfd640 1 --2- 172.21.15.84:0/2557102386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f800ccaa0 0x7f5f800c19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:21.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.945+0000 7f5f7ffff640 1 --2- 172.21.15.84:0/2557102386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f8015aaa0 0x7f5f800c4670 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0098c0 tx=0x7f5f6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:21.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.945+0000 7f5f7d7fa640 1 -- 172.21.15.84:0/2557102386 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c03ad60 con 0x7f5f8015aaa0 2023-10-16T10:08:21.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.945+0000 7f5f7d7fa640 1 -- 172.21.15.84:0/2557102386 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c0443f0 con 0x7f5f8015aaa0 2023-10-16T10:08:21.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.945+0000 7f5f86e61640 1 -- 172.21.15.84:0/2557102386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f800c7bc0 con 0x7f5f8015aaa0 2023-10-16T10:08:21.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.945+0000 7f5f7d7fa640 1 -- 172.21.15.84:0/2557102386 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c0447a0 con 0x7f5f8015aaa0 2023-10-16T10:08:21.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.945+0000 7f5f86e61640 1 -- 172.21.15.84:0/2557102386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f800c6880 con 0x7f5f8015aaa0 2023-10-16T10:08:21.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.946+0000 7f5f86e61640 1 -- 172.21.15.84:0/2557102386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8015b490 con 0x7f5f8015aaa0 2023-10-16T10:08:21.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.947+0000 7f5f7d7fa640 1 -- 172.21.15.84:0/2557102386 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5f6c04d450 con 0x7f5f8015aaa0 2023-10-16T10:08:21.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.948+0000 7f5f7d7fa640 1 --2- 172.21.15.84:0/2557102386 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f5f6406af90 0x7f5f6406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:21.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.948+0000 7f5f7d7fa640 1 -- 172.21.15.84:0/2557102386 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f5f6c0ab9c0 con 0x7f5f8015aaa0 2023-10-16T10:08:21.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.948+0000 7f5f84bfd640 1 --2- 172.21.15.84:0/2557102386 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f5f6406af90 0x7f5f6406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:21.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.949+0000 7f5f84bfd640 1 --2- 172.21.15.84:0/2557102386 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f5f6406af90 0x7f5f6406d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5f74006fd0 tx=0x7f5f74008040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:21.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:21.952+0000 7f5f7d7fa640 1 -- 172.21.15.84:0/2557102386 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5f8015b490 con 0x7f5f8015aaa0 2023-10-16T10:08:22.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.146+0000 7f5f86e61640 1 -- 172.21.15.84:0/2557102386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f8015b490 con 0x7f5f8015aaa0 2023-10-16T10:08:22.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.148+0000 7f5f7d7fa640 1 -- 172.21.15.84:0/2557102386 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+12887 (secure 0 0 0) 0x7f5f8015b490 con 0x7f5f8015aaa0 2023-10-16T10:08:22.149 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:22.150 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":134,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:08:21.183260+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:08:22.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.150+0000 7f5f86e61640 1 -- 172.21.15.84:0/2557102386 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f5f6406af90 msgr2=0x7f5f6406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:22.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.150+0000 7f5f86e61640 1 --2- 172.21.15.84:0/2557102386 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f5f6406af90 0x7f5f6406d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5f74006fd0 tx=0x7f5f74008040 comp rx=0 tx=0).stop 2023-10-16T10:08:22.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.150+0000 7f5f86e61640 1 -- 172.21.15.84:0/2557102386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f8015aaa0 msgr2=0x7f5f800c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:22.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.150+0000 7f5f86e61640 1 --2- 172.21.15.84:0/2557102386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f8015aaa0 0x7f5f800c4670 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0098c0 tx=0x7f5f6c0046c0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.150+0000 7f5f86e61640 1 -- 172.21.15.84:0/2557102386 shutdown_connections 2023-10-16T10:08:22.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.150+0000 7f5f86e61640 1 --2- 172.21.15.84:0/2557102386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f8015ae80 0x7f5f800c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.150+0000 7f5f86e61640 1 --2- 172.21.15.84:0/2557102386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f8015aaa0 0x7f5f800c4670 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.150+0000 7f5f86e61640 1 --2- 172.21.15.84:0/2557102386 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f5f6406af90 0x7f5f6406d450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.150+0000 7f5f86e61640 1 --2- 172.21.15.84:0/2557102386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f800ccaa0 0x7f5f800c19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.150+0000 7f5f86e61640 1 -- 172.21.15.84:0/2557102386 >> 172.21.15.84:0/2557102386 conn(0x7f5f800cb410 msgr2=0x7f5f80070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:22.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.150+0000 7f5f86e61640 1 -- 172.21.15.84:0/2557102386 shutdown_connections 2023-10-16T10:08:22.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.151+0000 7f5f86e61640 1 -- 172.21.15.84:0/2557102386 wait complete. 2023-10-16T10:08:22.170 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:08:22.171 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:08:22.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.385+0000 7f07e97f5640 1 Processor -- start 2023-10-16T10:08:22.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.385+0000 7f07e97f5640 1 -- start start 2023-10-16T10:08:22.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.387+0000 7f07e97f5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07e406a4b0 0x7f07e40ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:22.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.387+0000 7f07e97f5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f07e40c9ca0 0x7f07e40ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:22.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.387+0000 7f07e97f5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e40cb030 con 0x7f07e415ad70 2023-10-16T10:08:22.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.387+0000 7f07e97f5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e40cae30 con 0x7f07e40c9ca0 2023-10-16T10:08:22.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.387+0000 7f07e97f5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40c91c0 con 0x7f07e406a4b0 2023-10-16T10:08:22.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.387+0000 7f07e37fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f07e40c9ca0 0x7f07e40ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:22.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.388+0000 7f07e37fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f07e40c9ca0 0x7f07e40ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44452/0 (socket says 172.21.15.84:44452) 2023-10-16T10:08:22.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.388+0000 7f07e37fe640 1 -- 172.21.15.84:0/397198039 learned_addr learned my addr 172.21.15.84:0/397198039 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:22.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.388+0000 7f07dbfff640 1 --2- 172.21.15.84:0/397198039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07e406a4b0 0x7f07e40ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:22.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.388+0000 7f07e37fe640 1 -- 172.21.15.84:0/397198039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07e406a4b0 msgr2=0x7f07e40ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:22.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.388+0000 7f07e37fe640 1 --2- 172.21.15.84:0/397198039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07e406a4b0 0x7f07e40ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.388+0000 7f07e37fe640 1 -- 172.21.15.84:0/397198039 >> v1:172.21.15.157:6789/0 conn(0x7f07e415ad70 legacy=0x7f07e4069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:22.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.388+0000 7f07e37fe640 1 -- 172.21.15.84:0/397198039 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e40c94a0 con 0x7f07e40c9ca0 2023-10-16T10:08:22.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.389+0000 7f07e37fe640 1 --2- 172.21.15.84:0/397198039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07e40c9ca0 0x7f07e40ca8f0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009e50 tx=0x7f07dc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f3f1b643365487 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:22.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.389+0000 7f07e27fc640 1 -- 172.21.15.84:0/397198039 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc02ca70 con 0x7f07e40c9ca0 2023-10-16T10:08:22.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.389+0000 7f07e27fc640 1 -- 172.21.15.84:0/397198039 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07dc02cbd0 con 0x7f07e40c9ca0 2023-10-16T10:08:22.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.389+0000 7f07e27fc640 1 -- 172.21.15.84:0/397198039 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc0343f0 con 0x7f07e40c9ca0 2023-10-16T10:08:22.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.390+0000 7f07e97f5640 1 -- 172.21.15.84:0/397198039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07e40c9ca0 msgr2=0x7f07e40ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:22.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.390+0000 7f07e97f5640 1 --2- 172.21.15.84:0/397198039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07e40c9ca0 0x7f07e40ca8f0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009e50 tx=0x7f07dc02bed0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.390+0000 7f07e97f5640 1 -- 172.21.15.84:0/397198039 shutdown_connections 2023-10-16T10:08:22.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.390+0000 7f07e97f5640 1 --2- 172.21.15.84:0/397198039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07e40c9ca0 0x7f07e40ca8f0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.390+0000 7f07e97f5640 1 --2- 172.21.15.84:0/397198039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07e406a4b0 0x7f07e40ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.390+0000 7f07e97f5640 1 -- 172.21.15.84:0/397198039 >> 172.21.15.84:0/397198039 conn(0x7f07e40ce9b0 msgr2=0x7f07e40cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:22.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.390+0000 7f07e97f5640 1 -- 172.21.15.84:0/397198039 shutdown_connections 2023-10-16T10:08:22.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.390+0000 7f07e97f5640 1 -- 172.21.15.84:0/397198039 wait complete. 2023-10-16T10:08:22.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.392+0000 7f07e97f5640 1 Processor -- start 2023-10-16T10:08:22.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.392+0000 7f07e97f5640 1 -- start start 2023-10-16T10:08:22.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.393+0000 7f07e97f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07e406a4b0 0x7f07e40af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:22.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.393+0000 7f07e97f5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07e40c9ca0 0x7f07e41580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:22.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.393+0000 7f07e97f5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07e415ad70 0x7f07e40ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:22.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.393+0000 7f07e97f5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40ac890 con 0x7f07e406a4b0 2023-10-16T10:08:22.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.393+0000 7f07e97f5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e4158620 con 0x7f07e415ad70 2023-10-16T10:08:22.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.393+0000 7f07e97f5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40acac0 con 0x7f07e40c9ca0 2023-10-16T10:08:22.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.393+0000 7f07e2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07e406a4b0 0x7f07e40af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:22.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.394+0000 7f07e2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07e406a4b0 0x7f07e40af010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44468/0 (socket says 172.21.15.84:44468) 2023-10-16T10:08:22.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.394+0000 7f07e2ffd640 1 -- 172.21.15.84:0/1455515363 learned_addr learned my addr 172.21.15.84:0/1455515363 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:22.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.394+0000 7f07e2ffd640 1 -- 172.21.15.84:0/1455515363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07e40c9ca0 msgr2=0x7f07e41580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:22.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.394+0000 7f07e2ffd640 1 --2- 172.21.15.84:0/1455515363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07e40c9ca0 0x7f07e41580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.394+0000 7f07e37fe640 1 --2- 172.21.15.84:0/1455515363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07e415ad70 0x7f07e40ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.394+0000 7f07e2ffd640 1 -- 172.21.15.84:0/1455515363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07e415ad70 msgr2=0x7f07e40ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.394+0000 7f07e2ffd640 1 --2- 172.21.15.84:0/1455515363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07e415ad70 0x7f07e40ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.394+0000 7f07e2ffd640 1 -- 172.21.15.84:0/1455515363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc009b30 con 0x7f07e406a4b0 2023-10-16T10:08:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.394+0000 7f07e2ffd640 1 --2- 172.21.15.84:0/1455515363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07e406a4b0 0x7f07e40af010 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f07d400b3f0 tx=0x7f07d400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.395+0000 7f07db7fe640 1 -- 172.21.15.84:0/1455515363 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d4011830 con 0x7f07e406a4b0 2023-10-16T10:08:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.395+0000 7f07db7fe640 1 -- 172.21.15.84:0/1455515363 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d4011990 con 0x7f07e406a4b0 2023-10-16T10:08:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.395+0000 7f07db7fe640 1 -- 172.21.15.84:0/1455515363 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d4011d40 con 0x7f07e406a4b0 2023-10-16T10:08:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.395+0000 7f07e97f5640 1 -- 172.21.15.84:0/1455515363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e40acda0 con 0x7f07e406a4b0 2023-10-16T10:08:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.395+0000 7f07e97f5640 1 -- 172.21.15.84:0/1455515363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e40cc9e0 con 0x7f07e406a4b0 2023-10-16T10:08:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.396+0000 7f07e97f5640 1 -- 172.21.15.84:0/1455515363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e40acfc0 con 0x7f07e406a4b0 2023-10-16T10:08:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.397+0000 7f07db7fe640 1 -- 172.21.15.84:0/1455515363 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f07d401a400 con 0x7f07e406a4b0 2023-10-16T10:08:22.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.398+0000 7f07db7fe640 1 --2- 172.21.15.84:0/1455515363 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f07c406af90 0x7f07c406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:22.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.398+0000 7f07db7fe640 1 -- 172.21.15.84:0/1455515363 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f07d40892c0 con 0x7f07e406a4b0 2023-10-16T10:08:22.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.398+0000 7f07dbfff640 1 --2- 172.21.15.84:0/1455515363 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f07c406af90 0x7f07c406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:22.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.399+0000 7f07dbfff640 1 --2- 172.21.15.84:0/1455515363 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f07c406af90 0x7f07c406d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f07e406f580 tx=0x7f07cc008040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:22.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.402+0000 7f07db7fe640 1 -- 172.21.15.84:0/1455515363 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f07d4054590 con 0x7f07e406a4b0 2023-10-16T10:08:22.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.592+0000 7f07e97f5640 1 -- 172.21.15.84:0/1455515363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f07e404e160 con 0x7f07e406a4b0 2023-10-16T10:08:22.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.593+0000 7f07db7fe640 1 -- 172.21.15.84:0/1455515363 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) v1 ==== 93+0+10 (secure 0 0 0) 0x7f07d4057bb0 con 0x7f07e406a4b0 2023-10-16T10:08:22.595 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:08:22.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.596+0000 7f07e97f5640 1 -- 172.21.15.84:0/1455515363 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f07c406af90 msgr2=0x7f07c406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:22.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.596+0000 7f07e97f5640 1 --2- 172.21.15.84:0/1455515363 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f07c406af90 0x7f07c406d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f07e406f580 tx=0x7f07cc008040 comp rx=0 tx=0).stop 2023-10-16T10:08:22.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.596+0000 7f07e97f5640 1 -- 172.21.15.84:0/1455515363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07e406a4b0 msgr2=0x7f07e40af010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:22.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.596+0000 7f07e97f5640 1 --2- 172.21.15.84:0/1455515363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07e406a4b0 0x7f07e40af010 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f07d400b3f0 tx=0x7f07d400b8b0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.596+0000 7f07e97f5640 1 -- 172.21.15.84:0/1455515363 shutdown_connections 2023-10-16T10:08:22.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.596+0000 7f07e97f5640 1 --2- 172.21.15.84:0/1455515363 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f07c406af90 0x7f07c406d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.596+0000 7f07e97f5640 1 --2- 172.21.15.84:0/1455515363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07e415ad70 0x7f07e40ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.596+0000 7f07e97f5640 1 --2- 172.21.15.84:0/1455515363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07e40c9ca0 0x7f07e41580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.596+0000 7f07e97f5640 1 --2- 172.21.15.84:0/1455515363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07e406a4b0 0x7f07e40af010 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.596+0000 7f07e97f5640 1 -- 172.21.15.84:0/1455515363 >> 172.21.15.84:0/1455515363 conn(0x7f07e40ce9b0 msgr2=0x7f07e40b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:22.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.596+0000 7f07e97f5640 1 -- 172.21.15.84:0/1455515363 shutdown_connections 2023-10-16T10:08:22.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.596+0000 7f07e97f5640 1 -- 172.21.15.84:0/1455515363 wait complete. 2023-10-16T10:08:22.616 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T10:08:22.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.829+0000 7f0730b16640 1 Processor -- start 2023-10-16T10:08:22.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.830+0000 7f0730b16640 1 -- start start 2023-10-16T10:08:22.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.831+0000 7f0730b16640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f072c159950 0x7f072c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:22.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.831+0000 7f0730b16640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f072c070450 0x7f072c0708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:22.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.831+0000 7f0730b16640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f072c070df0 con 0x7f072c06c100 2023-10-16T10:08:22.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.831+0000 7f0730b16640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f072c15e490 con 0x7f072c070450 2023-10-16T10:08:22.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.831+0000 7f0730b16640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072c15aef0 con 0x7f072c159950 2023-10-16T10:08:22.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.832+0000 7f072bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f072c06c100 0x7f072c06c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:41600/0 (socket says 172.21.15.84:41600) 2023-10-16T10:08:22.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.832+0000 7f072bfff640 1 -- 172.21.15.84:0/604958780 learned_addr learned my addr 172.21.15.84:0/604958780 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:22.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.832+0000 7f072b7fe640 1 --2- 172.21.15.84:0/604958780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f072c159950 0x7f072c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:22.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.832+0000 7f0723fff640 1 --2- 172.21.15.84:0/604958780 >> v2:172.21.15.157:3300/0 conn(0x7f072c070450 0x7f072c0708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:22.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.832+0000 7f072affd640 1 -- 172.21.15.84:0/604958780 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947788261 0 0) 0x7f072c070df0 con 0x7f072c06c100 2023-10-16T10:08:22.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.832+0000 7f072affd640 1 -- 172.21.15.84:0/604958780 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0718003520 con 0x7f072c06c100 2023-10-16T10:08:22.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.832+0000 7f072affd640 1 -- 172.21.15.84:0/604958780 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3235157232 0 0) 0x7f0718003520 con 0x7f072c06c100 2023-10-16T10:08:22.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.832+0000 7f072affd640 1 -- 172.21.15.84:0/604958780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f072c159950 msgr2=0x7f072c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:22.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.832+0000 7f072affd640 1 --2- 172.21.15.84:0/604958780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f072c159950 0x7f072c15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.832+0000 7f072affd640 1 -- 172.21.15.84:0/604958780 >> v2:172.21.15.157:3300/0 conn(0x7f072c070450 msgr2=0x7f072c0708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:22.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.832+0000 7f072affd640 1 --2- 172.21.15.84:0/604958780 >> v2:172.21.15.157:3300/0 conn(0x7f072c070450 0x7f072c0708b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.832+0000 7f072affd640 1 -- 172.21.15.84:0/604958780 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c06cc80 con 0x7f072c06c100 2023-10-16T10:08:22.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.832+0000 7f072affd640 1 -- 172.21.15.84:0/604958780 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f07240019f0 con 0x7f072c06c100 2023-10-16T10:08:22.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.833+0000 7f072affd640 1 -- 172.21.15.84:0/604958780 >> v1:172.21.15.84:6789/0 conn(0x7f072c06c100 legacy=0x7f072c06c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:22.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.833+0000 7f072affd640 1 --2- 172.21.15.84:0/604958780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f072c159950 0x7f0718003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:22.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.833+0000 7f072affd640 1 --2- 172.21.15.84:0/604958780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0718003f40 0x7f07180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:22.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.833+0000 7f072affd640 1 --2- 172.21.15.84:0/604958780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0718004d00 0x7f0718005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:22.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.833+0000 7f072affd640 1 -- 172.21.15.84:0/604958780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c15e490 con 0x7f0718003f40 2023-10-16T10:08:22.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.833+0000 7f072affd640 1 -- 172.21.15.84:0/604958780 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c15aef0 con 0x7f0718004d00 2023-10-16T10:08:22.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.833+0000 7f072affd640 1 -- 172.21.15.84:0/604958780 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071800ba30 con 0x7f072c159950 2023-10-16T10:08:22.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.833+0000 7f0723fff640 1 --2- 172.21.15.84:0/604958780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0718003f40 0x7f07180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:22.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.833+0000 7f0723fff640 1 -- 172.21.15.84:0/604958780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f072c159950 msgr2=0x7f0718003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:22.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.833+0000 7f0723fff640 1 --2- 172.21.15.84:0/604958780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f072c159950 0x7f0718003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.833+0000 7f0723fff640 1 -- 172.21.15.84:0/604958780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0718004d00 msgr2=0x7f0718005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:22.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.833+0000 7f0723fff640 1 --2- 172.21.15.84:0/604958780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0718004d00 0x7f0718005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.833+0000 7f0723fff640 1 -- 172.21.15.84:0/604958780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0718003520 con 0x7f0718003f40 2023-10-16T10:08:22.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.834+0000 7f0723fff640 1 --2- 172.21.15.84:0/604958780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0718003f40 0x7f07180047c0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0714009ac0 tx=0x7f07180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4268013c6ae07c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:22.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.835+0000 7f072affd640 1 -- 172.21.15.84:0/604958780 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071402c710 con 0x7f0718003f40 2023-10-16T10:08:22.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.835+0000 7f072affd640 1 -- 172.21.15.84:0/604958780 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071402c870 con 0x7f0718003f40 2023-10-16T10:08:22.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.835+0000 7f072affd640 1 -- 172.21.15.84:0/604958780 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071402cc00 con 0x7f0718003f40 2023-10-16T10:08:22.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.835+0000 7f0730b16640 1 -- 172.21.15.84:0/604958780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0718003f40 msgr2=0x7f07180047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:22.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.835+0000 7f0730b16640 1 --2- 172.21.15.84:0/604958780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0718003f40 0x7f07180047c0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0714009ac0 tx=0x7f07180057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.835+0000 7f072bfff640 1 -- 172.21.15.84:0/604958780 reap_dead start 2023-10-16T10:08:22.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.835+0000 7f0730b16640 1 -- 172.21.15.84:0/604958780 shutdown_connections 2023-10-16T10:08:22.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.835+0000 7f0730b16640 1 -- 172.21.15.84:0/604958780 >> 172.21.15.84:0/604958780 conn(0x7f072c0ce6a0 msgr2=0x7f072c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:22.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.835+0000 7f0730b16640 1 -- 172.21.15.84:0/604958780 shutdown_connections 2023-10-16T10:08:22.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.835+0000 7f0730b16640 1 -- 172.21.15.84:0/604958780 wait complete. 2023-10-16T10:08:22.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.837+0000 7f0730b16640 1 Processor -- start 2023-10-16T10:08:22.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.837+0000 7f0730b16640 1 -- start start 2023-10-16T10:08:22.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.838+0000 7f0730b16640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0718003f40 0x7f072c0c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:22.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.838+0000 7f0730b16640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f072c158f10 0x7f072c0c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:22.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.838+0000 7f0730b16640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f072c151d00 0x7f072c0b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:22.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.838+0000 7f0730b16640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c0c6770 con 0x7f0718003f40 2023-10-16T10:08:22.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.838+0000 7f0730b16640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c0c5af0 con 0x7f072c151d00 2023-10-16T10:08:22.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.838+0000 7f0730b16640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072c0c5dd0 con 0x7f072c158f10 2023-10-16T10:08:22.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.839+0000 7f072b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0718003f40 0x7f072c0c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:22.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.839+0000 7f072b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0718003f40 0x7f072c0c8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44498/0 (socket says 172.21.15.84:44498) 2023-10-16T10:08:22.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.839+0000 7f072b7fe640 1 -- 172.21.15.84:0/1986453527 learned_addr learned my addr 172.21.15.84:0/1986453527 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:22.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.839+0000 7f072b7fe640 1 -- 172.21.15.84:0/1986453527 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f072c158f10 msgr2=0x7f072c0c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:22.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.839+0000 7f072b7fe640 1 --2- 172.21.15.84:0/1986453527 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f072c158f10 0x7f072c0c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.839+0000 7f072b7fe640 1 -- 172.21.15.84:0/1986453527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f072c151d00 msgr2=0x7f072c0b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:22.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.839+0000 7f072b7fe640 1 --2- 172.21.15.84:0/1986453527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f072c151d00 0x7f072c0b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:22.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.839+0000 7f072b7fe640 1 -- 172.21.15.84:0/1986453527 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c0c60b0 con 0x7f0718003f40 2023-10-16T10:08:22.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.839+0000 7f072b7fe640 1 --2- 172.21.15.84:0/1986453527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0718003f40 0x7f072c0c8f20 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f071c00e9d0 tx=0x7f071c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:22.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.840+0000 7f0728ff9640 1 -- 172.21.15.84:0/1986453527 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c00de60 con 0x7f0718003f40 2023-10-16T10:08:22.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.840+0000 7f0728ff9640 1 -- 172.21.15.84:0/1986453527 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071c019480 con 0x7f0718003f40 2023-10-16T10:08:22.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.840+0000 7f0730b16640 1 -- 172.21.15.84:0/1986453527 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072c0c6390 con 0x7f0718003f40 2023-10-16T10:08:22.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.840+0000 7f0728ff9640 1 -- 172.21.15.84:0/1986453527 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c019830 con 0x7f0718003f40 2023-10-16T10:08:22.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.840+0000 7f0730b16640 1 -- 172.21.15.84:0/1986453527 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072c0ad0f0 con 0x7f0718003f40 2023-10-16T10:08:22.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.841+0000 7f0730b16640 1 -- 172.21.15.84:0/1986453527 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f072c04e9b0 con 0x7f0718003f40 2023-10-16T10:08:22.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.842+0000 7f0728ff9640 1 -- 172.21.15.84:0/1986453527 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f071c019990 con 0x7f0718003f40 2023-10-16T10:08:22.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.843+0000 7f0728ff9640 1 --2- 172.21.15.84:0/1986453527 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f070c06afc0 0x7f070c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:22.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.843+0000 7f0728ff9640 1 -- 172.21.15.84:0/1986453527 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f071c014070 con 0x7f0718003f40 2023-10-16T10:08:22.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.843+0000 7f0723fff640 1 --2- 172.21.15.84:0/1986453527 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f070c06afc0 0x7f070c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:22.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.844+0000 7f0723fff640 1 --2- 172.21.15.84:0/1986453527 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f070c06afc0 0x7f070c06d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0714036000 tx=0x7f071402e000 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:22.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:22.847+0000 7f0728ff9640 1 -- 172.21.15.84:0/1986453527 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f071c055020 con 0x7f0718003f40 2023-10-16T10:08:23.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.041+0000 7f0730b16640 1 -- 172.21.15.84:0/1986453527 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f072c0b4620 con 0x7f0718003f40 2023-10-16T10:08:23.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.042+0000 7f0728ff9640 1 -- 172.21.15.84:0/1986453527 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v134) v1 ==== 84+0+65 (secure 0 0 0) 0x7f071c058640 con 0x7f0718003f40 2023-10-16T10:08:23.044 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:23.045 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T10:08:23.045 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:23.045 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-16T10:08:23.045 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-16T10:08:23.045 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:23.046 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T10:08:23.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.045+0000 7f0730b16640 1 -- 172.21.15.84:0/1986453527 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f070c06afc0 msgr2=0x7f070c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:23.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.045+0000 7f0730b16640 1 --2- 172.21.15.84:0/1986453527 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f070c06afc0 0x7f070c06d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0714036000 tx=0x7f071402e000 comp rx=0 tx=0).stop 2023-10-16T10:08:23.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.045+0000 7f0730b16640 1 -- 172.21.15.84:0/1986453527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0718003f40 msgr2=0x7f072c0c8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:23.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.045+0000 7f0730b16640 1 --2- 172.21.15.84:0/1986453527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0718003f40 0x7f072c0c8f20 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f071c00e9d0 tx=0x7f071c00ee90 comp rx=0 tx=0).stop 2023-10-16T10:08:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.045+0000 7f0730b16640 1 -- 172.21.15.84:0/1986453527 shutdown_connections 2023-10-16T10:08:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.045+0000 7f0730b16640 1 --2- 172.21.15.84:0/1986453527 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f070c06afc0 0x7f070c06d480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.045+0000 7f0730b16640 1 --2- 172.21.15.84:0/1986453527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f072c151d00 0x7f072c0b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.045+0000 7f0730b16640 1 --2- 172.21.15.84:0/1986453527 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f072c158f10 0x7f072c0c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.045+0000 7f0730b16640 1 --2- 172.21.15.84:0/1986453527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0718003f40 0x7f072c0c8f20 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.045+0000 7f0730b16640 1 -- 172.21.15.84:0/1986453527 >> 172.21.15.84:0/1986453527 conn(0x7f072c0ce6a0 msgr2=0x7f072c0cd990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:23.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.045+0000 7f0730b16640 1 -- 172.21.15.84:0/1986453527 shutdown_connections 2023-10-16T10:08:23.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.045+0000 7f0730b16640 1 -- 172.21.15.84:0/1986453527 wait complete. 2023-10-16T10:08:23.066 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T10:08:23.067 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T10:08:23.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.287+0000 7ff705def640 1 Processor -- start 2023-10-16T10:08:23.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.288+0000 7ff705def640 1 -- start start 2023-10-16T10:08:23.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.289+0000 7ff705def640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff70015cd20 0x7ff7000cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:23.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.289+0000 7ff705def640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70015d100 0x7ff7000b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:23.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.289+0000 7ff705def640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff7000b1810 0x7ff7000b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:23.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.289+0000 7ff705def640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7000b8430 con 0x7ff70015cd20 2023-10-16T10:08:23.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.289+0000 7ff705def640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7000d2160 con 0x7ff7000b1810 2023-10-16T10:08:23.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.289+0000 7ff705def640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7000ce0b0 con 0x7ff70015d100 2023-10-16T10:08:23.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.289+0000 7ff704ded640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff70015cd20 0x7ff7000cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:23.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.289+0000 7ff704ded640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff70015cd20 0x7ff7000cdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44502/0 (socket says 172.21.15.84:44502) 2023-10-16T10:08:23.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.289+0000 7ff704ded640 1 -- 172.21.15.84:0/275043512 learned_addr learned my addr 172.21.15.84:0/275043512 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:23.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.290+0000 7ff704ded640 1 -- 172.21.15.84:0/275043512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70015d100 msgr2=0x7ff7000b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:23.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.290+0000 7ff704ded640 1 --2- 172.21.15.84:0/275043512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70015d100 0x7ff7000b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:23.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.290+0000 7ff704ded640 1 -- 172.21.15.84:0/275043512 >> v2:172.21.15.157:3300/0 conn(0x7ff7000b1810 msgr2=0x7ff7000b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:23.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.290+0000 7ff704ded640 1 --2- 172.21.15.84:0/275043512 >> v2:172.21.15.157:3300/0 conn(0x7ff7000b1810 0x7ff7000b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:23.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.290+0000 7ff704ded640 1 -- 172.21.15.84:0/275043512 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7000b8680 con 0x7ff70015cd20 2023-10-16T10:08:23.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.291+0000 7ff704ded640 1 --2- 172.21.15.84:0/275043512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff70015cd20 0x7ff7000cdb70 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7ff7000b5b90 tx=0x7ff6f002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0c6c8ca324469a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:23.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.291+0000 7ff6ff7fe640 1 -- 172.21.15.84:0/275043512 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f002cb00 con 0x7ff70015cd20 2023-10-16T10:08:23.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.291+0000 7ff6ff7fe640 1 -- 172.21.15.84:0/275043512 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f002cc60 con 0x7ff70015cd20 2023-10-16T10:08:23.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.291+0000 7ff6ff7fe640 1 -- 172.21.15.84:0/275043512 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f00343f0 con 0x7ff70015cd20 2023-10-16T10:08:23.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.292+0000 7ff705def640 1 -- 172.21.15.84:0/275043512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff70015cd20 msgr2=0x7ff7000cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:23.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.292+0000 7ff705def640 1 --2- 172.21.15.84:0/275043512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff70015cd20 0x7ff7000cdb70 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7ff7000b5b90 tx=0x7ff6f002bf60 comp rx=0 tx=0).stop 2023-10-16T10:08:23.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.292+0000 7ff705def640 1 -- 172.21.15.84:0/275043512 shutdown_connections 2023-10-16T10:08:23.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.292+0000 7ff705def640 1 --2- 172.21.15.84:0/275043512 >> v2:172.21.15.157:3300/0 conn(0x7ff7000b1810 0x7ff7000b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:23.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.292+0000 7ff705def640 1 --2- 172.21.15.84:0/275043512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70015d100 0x7ff7000b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:23.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.292+0000 7ff705def640 1 --2- 172.21.15.84:0/275043512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff70015cd20 0x7ff7000cdb70 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:23.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.292+0000 7ff705def640 1 -- 172.21.15.84:0/275043512 >> 172.21.15.84:0/275043512 conn(0x7ff7000cf590 msgr2=0x7ff7000cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:23.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.292+0000 7ff705def640 1 -- 172.21.15.84:0/275043512 shutdown_connections 2023-10-16T10:08:23.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.292+0000 7ff705def640 1 -- 172.21.15.84:0/275043512 wait complete. 2023-10-16T10:08:23.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.294+0000 7ff705def640 1 Processor -- start 2023-10-16T10:08:23.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.294+0000 7ff705def640 1 -- start start 2023-10-16T10:08:23.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.295+0000 7ff705def640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7000b1810 0x7ff7000cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:23.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.295+0000 7ff705def640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70015cd20 0x7ff7000b7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:23.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.295+0000 7ff705def640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff70015d100 0x7ff7000b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:23.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.295+0000 7ff705def640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7000cc0d0 con 0x7ff70015d100 2023-10-16T10:08:23.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.295+0000 7ff705def640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7000cc9f0 con 0x7ff7000b1810 2023-10-16T10:08:23.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.295+0000 7ff705def640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7000cadd0 con 0x7ff70015cd20 2023-10-16T10:08:23.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.295+0000 7ff7055ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff70015d100 0x7ff7000b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:23.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.295+0000 7ff7055ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff70015d100 0x7ff7000b78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44510/0 (socket says 172.21.15.84:44510) 2023-10-16T10:08:23.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.295+0000 7ff7055ee640 1 -- 172.21.15.84:0/1459310272 learned_addr learned my addr 172.21.15.84:0/1459310272 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:23.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.296+0000 7ff704ded640 1 --2- 172.21.15.84:0/1459310272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7000b1810 0x7ff7000cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:23.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.296+0000 7ff6fffff640 1 --2- 172.21.15.84:0/1459310272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70015cd20 0x7ff7000b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:23.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.296+0000 7ff7055ee640 1 -- 172.21.15.84:0/1459310272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70015cd20 msgr2=0x7ff7000b7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:23.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.296+0000 7ff7055ee640 1 --2- 172.21.15.84:0/1459310272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70015cd20 0x7ff7000b7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:23.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.296+0000 7ff7055ee640 1 -- 172.21.15.84:0/1459310272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7000b1810 msgr2=0x7ff7000cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:23.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.296+0000 7ff7055ee640 1 --2- 172.21.15.84:0/1459310272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7000b1810 0x7ff7000cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:23.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.296+0000 7ff7055ee640 1 -- 172.21.15.84:0/1459310272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f0009b30 con 0x7ff70015d100 2023-10-16T10:08:23.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.296+0000 7ff7055ee640 1 --2- 172.21.15.84:0/1459310272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff70015d100 0x7ff7000b78b0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8009fd0 tx=0x7ff6f800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:23.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.297+0000 7ff6fd7fa640 1 -- 172.21.15.84:0/1459310272 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8011830 con 0x7ff70015d100 2023-10-16T10:08:23.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.297+0000 7ff6fd7fa640 1 -- 172.21.15.84:0/1459310272 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f8011990 con 0x7ff70015d100 2023-10-16T10:08:23.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.297+0000 7ff705def640 1 -- 172.21.15.84:0/1459310272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7000cb0b0 con 0x7ff70015d100 2023-10-16T10:08:23.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.297+0000 7ff6fd7fa640 1 -- 172.21.15.84:0/1459310272 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8011d20 con 0x7ff70015d100 2023-10-16T10:08:23.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.297+0000 7ff705def640 1 -- 172.21.15.84:0/1459310272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7000cb670 con 0x7ff70015d100 2023-10-16T10:08:23.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.298+0000 7ff705def640 1 -- 172.21.15.84:0/1459310272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff70004e9b0 con 0x7ff70015d100 2023-10-16T10:08:23.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.299+0000 7ff6fd7fa640 1 -- 172.21.15.84:0/1459310272 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff6f801b460 con 0x7ff70015d100 2023-10-16T10:08:23.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.300+0000 7ff6fd7fa640 1 --2- 172.21.15.84:0/1459310272 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7ff6e006af90 0x7ff6e006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:23.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.300+0000 7ff6fd7fa640 1 -- 172.21.15.84:0/1459310272 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4885+0+0 (secure 0 0 0) 0x7ff6f80891f0 con 0x7ff70015d100 2023-10-16T10:08:23.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.300+0000 7ff704ded640 1 --2- 172.21.15.84:0/1459310272 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7ff6e006af90 0x7ff6e006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:23.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.301+0000 7ff704ded640 1 --2- 172.21.15.84:0/1459310272 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7ff6e006af90 0x7ff6e006d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0035000 tx=0x7ff6f0030000 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:23.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.305+0000 7ff6fd7fa640 1 -- 172.21.15.84:0/1459310272 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff6f8057fe0 con 0x7ff70015d100 2023-10-16T10:08:23.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:23.501+0000 7ff705def640 1 -- 172.21.15.84:0/1459310272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff7000b15d0 con 0x7ff70015d100 2023-10-16T10:08:24.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.206+0000 7ff6fd7fa640 1 -- 172.21.15.84:0/1459310272 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v135) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff6f8057b70 con 0x7ff70015d100 2023-10-16T10:08:24.208 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-16T10:08:24.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.210+0000 7ff705def640 1 -- 172.21.15.84:0/1459310272 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7ff6e006af90 msgr2=0x7ff6e006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:24.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.210+0000 7ff705def640 1 --2- 172.21.15.84:0/1459310272 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7ff6e006af90 0x7ff6e006d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0035000 tx=0x7ff6f0030000 comp rx=0 tx=0).stop 2023-10-16T10:08:24.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.210+0000 7ff705def640 1 -- 172.21.15.84:0/1459310272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff70015d100 msgr2=0x7ff7000b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:24.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.210+0000 7ff705def640 1 --2- 172.21.15.84:0/1459310272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff70015d100 0x7ff7000b78b0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8009fd0 tx=0x7ff6f800b8e0 comp rx=0 tx=0).stop 2023-10-16T10:08:24.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.210+0000 7ff705def640 1 -- 172.21.15.84:0/1459310272 shutdown_connections 2023-10-16T10:08:24.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.210+0000 7ff705def640 1 --2- 172.21.15.84:0/1459310272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff70015d100 0x7ff7000b78b0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:24.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.210+0000 7ff705def640 1 --2- 172.21.15.84:0/1459310272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff70015cd20 0x7ff7000b7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:24.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.210+0000 7ff705def640 1 --2- 172.21.15.84:0/1459310272 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7ff6e006af90 0x7ff6e006d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:24.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.210+0000 7ff705def640 1 --2- 172.21.15.84:0/1459310272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7000b1810 0x7ff7000cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:24.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.210+0000 7ff705def640 1 -- 172.21.15.84:0/1459310272 >> 172.21.15.84:0/1459310272 conn(0x7ff7000cf590 msgr2=0x7ff7000b61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:24.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.210+0000 7ff705def640 1 -- 172.21.15.84:0/1459310272 shutdown_connections 2023-10-16T10:08:24.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.210+0000 7ff705def640 1 -- 172.21.15.84:0/1459310272 wait complete. 2023-10-16T10:08:24.237 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T10:08:24.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.441+0000 7fb346875640 1 Processor -- start 2023-10-16T10:08:24.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.441+0000 7fb346875640 1 -- start start 2023-10-16T10:08:24.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.442+0000 7fb346875640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb3400ccfc0 0x7fb3400b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:24.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.442+0000 7fb346875640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb3401615d0 0x7fb3400b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:24.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.442+0000 7fb346875640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb34015e3c0 con 0x7fb34015d3f0 2023-10-16T10:08:24.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.442+0000 7fb346875640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34015e0e0 con 0x7fb3401615d0 2023-10-16T10:08:24.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.442+0000 7fb346875640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb340160af0 con 0x7fb3400ccfc0 2023-10-16T10:08:24.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.443+0000 7fb337fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb3401615d0 0x7fb3400b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:24.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.443+0000 7fb337fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb3401615d0 0x7fb3400b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44518/0 (socket says 172.21.15.84:44518) 2023-10-16T10:08:24.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.443+0000 7fb337fff640 1 -- 172.21.15.84:0/3755288357 learned_addr learned my addr 172.21.15.84:0/3755288357 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:24.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.443+0000 7fb344e12640 1 --1- 172.21.15.84:0/3755288357 >> v1:172.21.15.84:6789/0 conn(0x7fb34015d3f0 0x7fb34015d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:41608/0 (socket says 172.21.15.84:41608) 2023-10-16T10:08:24.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.443+0000 7fb33ffff640 1 --2- 172.21.15.84:0/3755288357 >> v2:172.21.15.157:3300/0 conn(0x7fb3400ccfc0 0x7fb3400b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:24.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.443+0000 7fb337fff640 1 -- 172.21.15.84:0/3755288357 >> v2:172.21.15.157:3300/0 conn(0x7fb3400ccfc0 msgr2=0x7fb3400b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:24.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.443+0000 7fb337fff640 1 --2- 172.21.15.84:0/3755288357 >> v2:172.21.15.157:3300/0 conn(0x7fb3400ccfc0 0x7fb3400b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:24.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.443+0000 7fb337fff640 1 -- 172.21.15.84:0/3755288357 >> v1:172.21.15.84:6789/0 conn(0x7fb34015d3f0 legacy=0x7fb34015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:24.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.443+0000 7fb337fff640 1 -- 172.21.15.84:0/3755288357 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb340160dd0 con 0x7fb3401615d0 2023-10-16T10:08:24.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.443+0000 7fb33f7fe640 1 -- 172.21.15.84:0/3755288357 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455983345 0 0) 0x7fb34015e3c0 con 0x7fb34015d3f0 2023-10-16T10:08:24.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.445+0000 7fb337fff640 1 --2- 172.21.15.84:0/3755288357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3401615d0 0x7fb3400b0950 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fb3400cc4e0 tx=0x7fb32802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f83f0b3404fe0c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:24.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.445+0000 7fb33f7fe640 1 -- 172.21.15.84:0/3755288357 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328031870 con 0x7fb3401615d0 2023-10-16T10:08:24.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.445+0000 7fb33f7fe640 1 -- 172.21.15.84:0/3755288357 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3280319d0 con 0x7fb3401615d0 2023-10-16T10:08:24.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.445+0000 7fb33f7fe640 1 -- 172.21.15.84:0/3755288357 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328031d60 con 0x7fb3401615d0 2023-10-16T10:08:24.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.446+0000 7fb346875640 1 -- 172.21.15.84:0/3755288357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3401615d0 msgr2=0x7fb3400b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:24.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.446+0000 7fb346875640 1 --2- 172.21.15.84:0/3755288357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3401615d0 0x7fb3400b0950 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fb3400cc4e0 tx=0x7fb32802b910 comp rx=0 tx=0).stop 2023-10-16T10:08:24.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.446+0000 7fb346875640 1 -- 172.21.15.84:0/3755288357 shutdown_connections 2023-10-16T10:08:24.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.446+0000 7fb346875640 1 --2- 172.21.15.84:0/3755288357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3401615d0 0x7fb3400b0950 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:24.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.446+0000 7fb346875640 1 --2- 172.21.15.84:0/3755288357 >> v2:172.21.15.157:3300/0 conn(0x7fb3400ccfc0 0x7fb3400b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:24.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.446+0000 7fb346875640 1 -- 172.21.15.84:0/3755288357 >> 172.21.15.84:0/3755288357 conn(0x7fb3400cb410 msgr2=0x7fb3400cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:24.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.446+0000 7fb346875640 1 -- 172.21.15.84:0/3755288357 shutdown_connections 2023-10-16T10:08:24.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.446+0000 7fb346875640 1 -- 172.21.15.84:0/3755288357 wait complete. 2023-10-16T10:08:24.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.448+0000 7fb346875640 1 Processor -- start 2023-10-16T10:08:24.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.448+0000 7fb346875640 1 -- start start 2023-10-16T10:08:24.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.449+0000 7fb346875640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3400ccfc0 0x7fb340153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:24.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.449+0000 7fb346875640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb34015d3f0 0x7fb3400aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:24.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.449+0000 7fb346875640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3401615d0 0x7fb3400b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:24.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.449+0000 7fb346875640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400b4580 con 0x7fb3400ccfc0 2023-10-16T10:08:24.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.449+0000 7fb346875640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400c2590 con 0x7fb3401615d0 2023-10-16T10:08:24.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.449+0000 7fb346875640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400c2870 con 0x7fb34015d3f0 2023-10-16T10:08:24.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.449+0000 7fb33ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3400ccfc0 0x7fb340153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:24.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.449+0000 7fb33ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3400ccfc0 0x7fb340153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44524/0 (socket says 172.21.15.84:44524) 2023-10-16T10:08:24.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.449+0000 7fb33ffff640 1 -- 172.21.15.84:0/1319807852 learned_addr learned my addr 172.21.15.84:0/1319807852 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:24.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.449+0000 7fb344e12640 1 --2- 172.21.15.84:0/1319807852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3401615d0 0x7fb3400b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:24.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.449+0000 7fb33ffff640 1 -- 172.21.15.84:0/1319807852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb34015d3f0 msgr2=0x7fb3400aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:24.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.449+0000 7fb33ffff640 1 --2- 172.21.15.84:0/1319807852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb34015d3f0 0x7fb3400aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:24.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.449+0000 7fb33ffff640 1 -- 172.21.15.84:0/1319807852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3401615d0 msgr2=0x7fb3400b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:24.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.449+0000 7fb33ffff640 1 --2- 172.21.15.84:0/1319807852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3401615d0 0x7fb3400b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:24.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.449+0000 7fb33ffff640 1 -- 172.21.15.84:0/1319807852 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3280094d0 con 0x7fb3400ccfc0 2023-10-16T10:08:24.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.449+0000 7fb344e12640 1 --2- 172.21.15.84:0/1319807852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3401615d0 0x7fb3400b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:24.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.450+0000 7fb33ffff640 1 --2- 172.21.15.84:0/1319807852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3400ccfc0 0x7fb340153ff0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fb33000d800 tx=0x7fb33000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:24.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.450+0000 7fb33d7fa640 1 -- 172.21.15.84:0/1319807852 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33000b6c0 con 0x7fb3400ccfc0 2023-10-16T10:08:24.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.450+0000 7fb33d7fa640 1 -- 172.21.15.84:0/1319807852 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33000b820 con 0x7fb3400ccfc0 2023-10-16T10:08:24.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.450+0000 7fb346875640 1 -- 172.21.15.84:0/1319807852 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3400c2b50 con 0x7fb3400ccfc0 2023-10-16T10:08:24.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.450+0000 7fb33d7fa640 1 -- 172.21.15.84:0/1319807852 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33000bbb0 con 0x7fb3400ccfc0 2023-10-16T10:08:24.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.450+0000 7fb346875640 1 -- 172.21.15.84:0/1319807852 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3400c3110 con 0x7fb3400ccfc0 2023-10-16T10:08:24.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.451+0000 7fb346875640 1 -- 172.21.15.84:0/1319807852 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3400c32a0 con 0x7fb3400ccfc0 2023-10-16T10:08:24.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.452+0000 7fb33d7fa640 1 -- 172.21.15.84:0/1319807852 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb33000bd10 con 0x7fb3400ccfc0 2023-10-16T10:08:24.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.453+0000 7fb33d7fa640 1 --2- 172.21.15.84:0/1319807852 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb32006af90 0x7fb32006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:24.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.453+0000 7fb33d7fa640 1 -- 172.21.15.84:0/1319807852 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5255+0+0 (secure 0 0 0) 0x7fb33008a6a0 con 0x7fb3400ccfc0 2023-10-16T10:08:24.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.454+0000 7fb337fff640 1 --2- 172.21.15.84:0/1319807852 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb32006af90 0x7fb32006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:24.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.454+0000 7fb337fff640 1 --2- 172.21.15.84:0/1319807852 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb32006af90 0x7fb32006d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb328036000 tx=0x7fb328030000 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:24.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.458+0000 7fb33d7fa640 1 -- 172.21.15.84:0/1319807852 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb3300557e0 con 0x7fb3400ccfc0 2023-10-16T10:08:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:24.653+0000 7fb346875640 1 -- 172.21.15.84:0/1319807852 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb34004e160 con 0x7fb3400ccfc0 2023-10-16T10:08:25.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.224+0000 7fb33d7fa640 1 -- 172.21.15.84:0/1319807852 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v136) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb330058e00 con 0x7fb3400ccfc0 2023-10-16T10:08:25.227 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-16T10:08:25.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.228+0000 7fb346875640 1 -- 172.21.15.84:0/1319807852 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb32006af90 msgr2=0x7fb32006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:25.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.228+0000 7fb346875640 1 --2- 172.21.15.84:0/1319807852 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb32006af90 0x7fb32006d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb328036000 tx=0x7fb328030000 comp rx=0 tx=0).stop 2023-10-16T10:08:25.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.228+0000 7fb346875640 1 -- 172.21.15.84:0/1319807852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3400ccfc0 msgr2=0x7fb340153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:25.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.228+0000 7fb346875640 1 --2- 172.21.15.84:0/1319807852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3400ccfc0 0x7fb340153ff0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fb33000d800 tx=0x7fb33000dcc0 comp rx=0 tx=0).stop 2023-10-16T10:08:25.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.229+0000 7fb346875640 1 -- 172.21.15.84:0/1319807852 shutdown_connections 2023-10-16T10:08:25.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.229+0000 7fb346875640 1 --2- 172.21.15.84:0/1319807852 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb32006af90 0x7fb32006d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:25.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.229+0000 7fb346875640 1 --2- 172.21.15.84:0/1319807852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3401615d0 0x7fb3400b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:25.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.229+0000 7fb346875640 1 --2- 172.21.15.84:0/1319807852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb34015d3f0 0x7fb3400aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:25.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.229+0000 7fb346875640 1 --2- 172.21.15.84:0/1319807852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3400ccfc0 0x7fb340153ff0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:25.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.229+0000 7fb346875640 1 -- 172.21.15.84:0/1319807852 >> 172.21.15.84:0/1319807852 conn(0x7fb3400cb410 msgr2=0x7fb3400ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:25.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.231+0000 7fb346875640 1 -- 172.21.15.84:0/1319807852 shutdown_connections 2023-10-16T10:08:25.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.231+0000 7fb346875640 1 -- 172.21.15.84:0/1319807852 wait complete. 2023-10-16T10:08:25.267 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T10:08:25.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.450+0000 7f7ad78ec640 1 Processor -- start 2023-10-16T10:08:25.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.450+0000 7f7ad78ec640 1 -- start start 2023-10-16T10:08:25.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.452+0000 7f7ad78ec640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7ad015d150 0x7f7ad00ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:25.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.452+0000 7f7ad78ec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad006a4b0 0x7f7ad0160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:25.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.452+0000 7f7ad78ec640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad00ac890 con 0x7f7ad0160b90 2023-10-16T10:08:25.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.452+0000 7f7ad78ec640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad015d530 con 0x7f7ad015d150 2023-10-16T10:08:25.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.452+0000 7f7ad78ec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad0161650 con 0x7f7ad006a4b0 2023-10-16T10:08:25.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.453+0000 7f7ad4e87640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7ad0160b90 0x7f7ad0069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:48886/0 (socket says 172.21.15.84:48886) 2023-10-16T10:08:25.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.453+0000 7f7ad4e87640 1 -- 172.21.15.84:0/319158111 learned_addr learned my addr 172.21.15.84:0/319158111 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:25.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.453+0000 7f7ad5688640 1 --2- 172.21.15.84:0/319158111 >> v2:172.21.15.157:3300/0 conn(0x7f7ad015d150 0x7f7ad00ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:25.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.453+0000 7f7ad5e89640 1 --2- 172.21.15.84:0/319158111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad006a4b0 0x7f7ad0160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:25.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.453+0000 7f7ad5688640 1 -- 172.21.15.84:0/319158111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad006a4b0 msgr2=0x7f7ad0160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:25.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.454+0000 7f7ad5688640 1 --2- 172.21.15.84:0/319158111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad006a4b0 0x7f7ad0160fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:25.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.454+0000 7f7ad5688640 1 -- 172.21.15.84:0/319158111 >> v1:172.21.15.157:6789/0 conn(0x7f7ad0160b90 legacy=0x7f7ad0069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:25.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.454+0000 7f7ad5688640 1 -- 172.21.15.84:0/319158111 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad0161930 con 0x7f7ad015d150 2023-10-16T10:08:25.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.455+0000 7f7ad5688640 1 --2- 172.21.15.84:0/319158111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad015d150 0x7f7ad00ad350 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009d90 tx=0x7f7ac002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de21fe0d1ad7ccbf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:25.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.456+0000 7f7acffff640 1 -- 172.21.15.84:0/319158111 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac002cbc0 con 0x7f7ad015d150 2023-10-16T10:08:25.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.456+0000 7f7acffff640 1 -- 172.21.15.84:0/319158111 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac002cd20 con 0x7f7ad015d150 2023-10-16T10:08:25.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.456+0000 7f7acffff640 1 -- 172.21.15.84:0/319158111 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0034660 con 0x7f7ad015d150 2023-10-16T10:08:25.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.456+0000 7f7ad78ec640 1 -- 172.21.15.84:0/319158111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad015d150 msgr2=0x7f7ad00ad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:25.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.456+0000 7f7ad78ec640 1 --2- 172.21.15.84:0/319158111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad015d150 0x7f7ad00ad350 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009d90 tx=0x7f7ac002be10 comp rx=0 tx=0).stop 2023-10-16T10:08:25.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.457+0000 7f7ad78ec640 1 -- 172.21.15.84:0/319158111 shutdown_connections 2023-10-16T10:08:25.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.457+0000 7f7ad78ec640 1 --2- 172.21.15.84:0/319158111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad006a4b0 0x7f7ad0160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:25.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.457+0000 7f7ad78ec640 1 --2- 172.21.15.84:0/319158111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad015d150 0x7f7ad00ad350 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:25.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.457+0000 7f7ad78ec640 1 -- 172.21.15.84:0/319158111 >> 172.21.15.84:0/319158111 conn(0x7f7ad00ce9b0 msgr2=0x7f7ad00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:25.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.457+0000 7f7ad78ec640 1 -- 172.21.15.84:0/319158111 shutdown_connections 2023-10-16T10:08:25.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.457+0000 7f7ad78ec640 1 -- 172.21.15.84:0/319158111 wait complete. 2023-10-16T10:08:25.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.458+0000 7f7ad78ec640 1 Processor -- start 2023-10-16T10:08:25.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.459+0000 7f7ad78ec640 1 -- start start 2023-10-16T10:08:25.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.460+0000 7f7ad78ec640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad006a4b0 0x7f7ad015b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:25.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.460+0000 7f7ad78ec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad015d150 0x7f7ad00c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:25.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.460+0000 7f7ad78ec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ad0160b90 0x7f7ad015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:25.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.460+0000 7f7ad78ec640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad00c8990 con 0x7f7ad0160b90 2023-10-16T10:08:25.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.460+0000 7f7ad78ec640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad00b0ca0 con 0x7f7ad006a4b0 2023-10-16T10:08:25.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.460+0000 7f7ad78ec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad00c3bf0 con 0x7f7ad015d150 2023-10-16T10:08:25.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.460+0000 7f7ad5e89640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ad0160b90 0x7f7ad015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:25.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.460+0000 7f7ad5e89640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ad0160b90 0x7f7ad015b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44528/0 (socket says 172.21.15.84:44528) 2023-10-16T10:08:25.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.460+0000 7f7ad5e89640 1 -- 172.21.15.84:0/3730737359 learned_addr learned my addr 172.21.15.84:0/3730737359 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:25.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.460+0000 7f7ad5688640 1 --2- 172.21.15.84:0/3730737359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad006a4b0 0x7f7ad015b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:25.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.460+0000 7f7ad4e87640 1 --2- 172.21.15.84:0/3730737359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad015d150 0x7f7ad00c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:25.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.461+0000 7f7ad5e89640 1 -- 172.21.15.84:0/3730737359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad015d150 msgr2=0x7f7ad00c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:25.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.461+0000 7f7ad5e89640 1 --2- 172.21.15.84:0/3730737359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad015d150 0x7f7ad00c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:25.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.461+0000 7f7ad5e89640 1 -- 172.21.15.84:0/3730737359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad006a4b0 msgr2=0x7f7ad015b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:25.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.461+0000 7f7ad5e89640 1 --2- 172.21.15.84:0/3730737359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad006a4b0 0x7f7ad015b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:25.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.461+0000 7f7ad5e89640 1 -- 172.21.15.84:0/3730737359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac0009a70 con 0x7f7ad0160b90 2023-10-16T10:08:25.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.461+0000 7f7ad5e89640 1 --2- 172.21.15.84:0/3730737359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ad0160b90 0x7f7ad015b9c0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f7ac800e7f0 tx=0x7f7ac800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:25.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.461+0000 7f7acdffb640 1 -- 172.21.15.84:0/3730737359 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8009510 con 0x7f7ad0160b90 2023-10-16T10:08:25.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.461+0000 7f7acdffb640 1 -- 172.21.15.84:0/3730737359 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac800ee40 con 0x7f7ad0160b90 2023-10-16T10:08:25.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.461+0000 7f7ad78ec640 1 -- 172.21.15.84:0/3730737359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad00c3eb0 con 0x7f7ad0160b90 2023-10-16T10:08:25.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.461+0000 7f7acdffb640 1 -- 172.21.15.84:0/3730737359 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac800cb80 con 0x7f7ad0160b90 2023-10-16T10:08:25.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.462+0000 7f7ad78ec640 1 -- 172.21.15.84:0/3730737359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad00c4470 con 0x7f7ad0160b90 2023-10-16T10:08:25.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.463+0000 7f7ad78ec640 1 -- 172.21.15.84:0/3730737359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad00cf0a0 con 0x7f7ad0160b90 2023-10-16T10:08:25.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.465+0000 7f7acdffb640 1 -- 172.21.15.84:0/3730737359 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7ac800cce0 con 0x7f7ad0160b90 2023-10-16T10:08:25.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.466+0000 7f7acdffb640 1 --2- 172.21.15.84:0/3730737359 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f7abc06af90 0x7f7abc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:25.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.466+0000 7f7acdffb640 1 -- 172.21.15.84:0/3730737359 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5637+0+0 (secure 0 0 0) 0x7f7ac8089ed0 con 0x7f7ad0160b90 2023-10-16T10:08:25.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.467+0000 7f7ad5688640 1 --2- 172.21.15.84:0/3730737359 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f7abc06af90 0x7f7abc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:25.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.467+0000 7f7ad5688640 1 --2- 172.21.15.84:0/3730737359 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f7abc06af90 0x7f7abc06d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0036000 tx=0x7f7ac0030000 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:25.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.470+0000 7f7acdffb640 1 -- 172.21.15.84:0/3730737359 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7ac8054f60 con 0x7f7ad0160b90 2023-10-16T10:08:25.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:25.702+0000 7f7ad78ec640 1 -- 172.21.15.84:0/3730737359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7ad004e160 con 0x7f7ad0160b90 2023-10-16T10:08:26.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.229+0000 7f7acdffb640 1 -- 172.21.15.84:0/3730737359 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v67) v1 ==== 174+0+0 (secure 0 0 0) 0x7f7ac80170c0 con 0x7f7ad0160b90 2023-10-16T10:08:26.231 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 16 and data pool 17 2023-10-16T10:08:26.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.233+0000 7f7ad78ec640 1 -- 172.21.15.84:0/3730737359 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f7abc06af90 msgr2=0x7f7abc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:26.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.233+0000 7f7ad78ec640 1 --2- 172.21.15.84:0/3730737359 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f7abc06af90 0x7f7abc06d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0036000 tx=0x7f7ac0030000 comp rx=0 tx=0).stop 2023-10-16T10:08:26.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.233+0000 7f7ad78ec640 1 -- 172.21.15.84:0/3730737359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ad0160b90 msgr2=0x7f7ad015b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:26.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.233+0000 7f7ad78ec640 1 --2- 172.21.15.84:0/3730737359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ad0160b90 0x7f7ad015b9c0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f7ac800e7f0 tx=0x7f7ac800ebf0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.233+0000 7f7ad78ec640 1 -- 172.21.15.84:0/3730737359 shutdown_connections 2023-10-16T10:08:26.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.233+0000 7f7ad78ec640 1 --2- 172.21.15.84:0/3730737359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ad0160b90 0x7f7ad015b9c0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.233+0000 7f7ad78ec640 1 --2- 172.21.15.84:0/3730737359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ad015d150 0x7f7ad00c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.233+0000 7f7ad78ec640 1 --2- 172.21.15.84:0/3730737359 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f7abc06af90 0x7f7abc06d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.233+0000 7f7ad78ec640 1 --2- 172.21.15.84:0/3730737359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad006a4b0 0x7f7ad015b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.233+0000 7f7ad78ec640 1 -- 172.21.15.84:0/3730737359 >> 172.21.15.84:0/3730737359 conn(0x7f7ad00ce9b0 msgr2=0x7f7ad00c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:26.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.234+0000 7f7ad78ec640 1 -- 172.21.15.84:0/3730737359 shutdown_connections 2023-10-16T10:08:26.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.234+0000 7f7ad78ec640 1 -- 172.21.15.84:0/3730737359 wait complete. 2023-10-16T10:08:26.272 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:08:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.454+0000 7f71f27ce640 1 Processor -- start 2023-10-16T10:08:26.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.454+0000 7f71f27ce640 1 -- start start 2023-10-16T10:08:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.455+0000 7f71f27ce640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f71ec159f00 0x7f71ec0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:26.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.455+0000 7f71f27ce640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f71ec15d880 0x7f71ec0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.456+0000 7f71f27ce640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71ec06f7f0 con 0x7f71ec159b20 2023-10-16T10:08:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.456+0000 7f71f27ce640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71ec06c5e0 con 0x7f71ec159f00 2023-10-16T10:08:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.456+0000 7f71f27ce640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71ec06c8c0 con 0x7f71ec15d880 2023-10-16T10:08:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.456+0000 7f71e37fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f71ec159f00 0x7f71ec0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:26.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.456+0000 7f71e37fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f71ec159f00 0x7f71ec0b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51906/0 (socket says 172.21.15.84:51906) 2023-10-16T10:08:26.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.456+0000 7f71e37fe640 1 -- 172.21.15.84:0/703271873 learned_addr learned my addr 172.21.15.84:0/703271873 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:26.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.456+0000 7f71f0d6b640 1 --2- 172.21.15.84:0/703271873 >> v2:172.21.15.157:3300/0 conn(0x7f71ec15d880 0x7f71ec0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:26.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.456+0000 7f71e37fe640 1 -- 172.21.15.84:0/703271873 >> v2:172.21.15.157:3300/0 conn(0x7f71ec15d880 msgr2=0x7f71ec0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:26.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.456+0000 7f71e37fe640 1 --2- 172.21.15.84:0/703271873 >> v2:172.21.15.157:3300/0 conn(0x7f71ec15d880 0x7f71ec0b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.456+0000 7f71e37fe640 1 -- 172.21.15.84:0/703271873 >> v1:172.21.15.157:6789/0 conn(0x7f71ec159b20 legacy=0x7f71ec06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:26.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.456+0000 7f71e37fe640 1 -- 172.21.15.84:0/703271873 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ec06cba0 con 0x7f71ec159f00 2023-10-16T10:08:26.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.458+0000 7f71e37fe640 1 --2- 172.21.15.84:0/703271873 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71ec159f00 0x7f71ec0b0410 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0c8bb0 tx=0x7f71d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649578e490138ef8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:26.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.458+0000 7f71eb7fe640 1 -- 172.21.15.84:0/703271873 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4031870 con 0x7f71ec159f00 2023-10-16T10:08:26.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.458+0000 7f71eb7fe640 1 -- 172.21.15.84:0/703271873 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d40319d0 con 0x7f71ec159f00 2023-10-16T10:08:26.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.458+0000 7f71eb7fe640 1 -- 172.21.15.84:0/703271873 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4031d60 con 0x7f71ec159f00 2023-10-16T10:08:26.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.458+0000 7f71f27ce640 1 -- 172.21.15.84:0/703271873 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71ec159f00 msgr2=0x7f71ec0b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:26.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.458+0000 7f71f27ce640 1 --2- 172.21.15.84:0/703271873 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71ec159f00 0x7f71ec0b0410 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0c8bb0 tx=0x7f71d402b910 comp rx=0 tx=0).stop 2023-10-16T10:08:26.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.459+0000 7f71f27ce640 1 -- 172.21.15.84:0/703271873 shutdown_connections 2023-10-16T10:08:26.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.459+0000 7f71f27ce640 1 --2- 172.21.15.84:0/703271873 >> v2:172.21.15.157:3300/0 conn(0x7f71ec15d880 0x7f71ec0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.459+0000 7f71f27ce640 1 --2- 172.21.15.84:0/703271873 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71ec159f00 0x7f71ec0b0410 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.459+0000 7f71f27ce640 1 -- 172.21.15.84:0/703271873 >> 172.21.15.84:0/703271873 conn(0x7f71ec0cb410 msgr2=0x7f71ec0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:26.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.459+0000 7f71f27ce640 1 -- 172.21.15.84:0/703271873 shutdown_connections 2023-10-16T10:08:26.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.459+0000 7f71f27ce640 1 -- 172.21.15.84:0/703271873 wait complete. 2023-10-16T10:08:26.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.460+0000 7f71f27ce640 1 Processor -- start 2023-10-16T10:08:26.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.460+0000 7f71f27ce640 1 -- start start 2023-10-16T10:08:26.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.461+0000 7f71f27ce640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71ec159b20 0x7f71ec155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:26.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.461+0000 7f71f27ce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71ec159f00 0x7f71ec154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:26.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.461+0000 7f71f27ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71ec15d880 0x7f71ec15c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:26.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.462+0000 7f71f27ce640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec15d2c0 con 0x7f71ec15d880 2023-10-16T10:08:26.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.462+0000 7f71f27ce640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec154f80 con 0x7f71ec159f00 2023-10-16T10:08:26.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.462+0000 7f71f27ce640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec15cd00 con 0x7f71ec159b20 2023-10-16T10:08:26.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.462+0000 7f71f0d6b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71ec15d880 0x7f71ec15c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:26.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.462+0000 7f71f0d6b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71ec15d880 0x7f71ec15c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51908/0 (socket says 172.21.15.84:51908) 2023-10-16T10:08:26.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.462+0000 7f71f0d6b640 1 -- 172.21.15.84:0/3029762011 learned_addr learned my addr 172.21.15.84:0/3029762011 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:26.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.462+0000 7f71e37fe640 1 --2- 172.21.15.84:0/3029762011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71ec159f00 0x7f71ec154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:26.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.462+0000 7f71ebfff640 1 --2- 172.21.15.84:0/3029762011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71ec159b20 0x7f71ec155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:26.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.462+0000 7f71f0d6b640 1 -- 172.21.15.84:0/3029762011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71ec159b20 msgr2=0x7f71ec155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:26.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.462+0000 7f71f0d6b640 1 --2- 172.21.15.84:0/3029762011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71ec159b20 0x7f71ec155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.462+0000 7f71f0d6b640 1 -- 172.21.15.84:0/3029762011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71ec159f00 msgr2=0x7f71ec154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:26.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.462+0000 7f71f0d6b640 1 --2- 172.21.15.84:0/3029762011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71ec159f00 0x7f71ec154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.462+0000 7f71f0d6b640 1 -- 172.21.15.84:0/3029762011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d40094d0 con 0x7f71ec15d880 2023-10-16T10:08:26.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.463+0000 7f71f0d6b640 1 --2- 172.21.15.84:0/3029762011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71ec15d880 0x7f71ec15c440 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f71e400cd70 tx=0x7f71e400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:26.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.463+0000 7f71e97fa640 1 -- 172.21.15.84:0/3029762011 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e4003b90 con 0x7f71ec15d880 2023-10-16T10:08:26.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.463+0000 7f71e97fa640 1 -- 172.21.15.84:0/3029762011 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e4003cf0 con 0x7f71ec15d880 2023-10-16T10:08:26.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.463+0000 7f71e97fa640 1 -- 172.21.15.84:0/3029762011 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e4004240 con 0x7f71ec15d880 2023-10-16T10:08:26.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.463+0000 7f71f27ce640 1 -- 172.21.15.84:0/3029762011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ec15cfe0 con 0x7f71ec15d880 2023-10-16T10:08:26.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.463+0000 7f71f27ce640 1 -- 172.21.15.84:0/3029762011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ec0c5de0 con 0x7f71ec15d880 2023-10-16T10:08:26.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.464+0000 7f71f27ce640 1 -- 172.21.15.84:0/3029762011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ec06c270 con 0x7f71ec15d880 2023-10-16T10:08:26.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.466+0000 7f71e97fa640 1 -- 172.21.15.84:0/3029762011 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f71e40043a0 con 0x7f71ec15d880 2023-10-16T10:08:26.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.466+0000 7f71e97fa640 1 --2- 172.21.15.84:0/3029762011 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f71cc06af90 0x7f71cc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:26.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.467+0000 7f71e97fa640 1 -- 172.21.15.84:0/3029762011 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f71e4012070 con 0x7f71ec15d880 2023-10-16T10:08:26.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.467+0000 7f71ebfff640 1 --2- 172.21.15.84:0/3029762011 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f71cc06af90 0x7f71cc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:26.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.467+0000 7f71ebfff640 1 --2- 172.21.15.84:0/3029762011 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f71cc06af90 0x7f71cc06d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00a750 tx=0x7f71dc008040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:26.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.471+0000 7f71e97fa640 1 -- 172.21.15.84:0/3029762011 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f71e4055150 con 0x7f71ec15d880 2023-10-16T10:08:26.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.650+0000 7f71f27ce640 1 -- 172.21.15.84:0/3029762011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71ec04e160 con 0x7f71ec15d880 2023-10-16T10:08:26.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.652+0000 7f71e97fa640 1 -- 172.21.15.84:0/3029762011 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16560 (secure 0 0 0) 0x7f71e4058770 con 0x7f71ec15d880 2023-10-16T10:08:26.654 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:26.654 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":137,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:08:25.704831+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:08:23.558506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-16T10:08:24.709464+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:08:26.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.655+0000 7f71f27ce640 1 -- 172.21.15.84:0/3029762011 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f71cc06af90 msgr2=0x7f71cc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:26.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.655+0000 7f71f27ce640 1 --2- 172.21.15.84:0/3029762011 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f71cc06af90 0x7f71cc06d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00a750 tx=0x7f71dc008040 comp rx=0 tx=0).stop 2023-10-16T10:08:26.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.655+0000 7f71f27ce640 1 -- 172.21.15.84:0/3029762011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71ec15d880 msgr2=0x7f71ec15c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:26.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.655+0000 7f71f27ce640 1 --2- 172.21.15.84:0/3029762011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71ec15d880 0x7f71ec15c440 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f71e400cd70 tx=0x7f71e400a660 comp rx=0 tx=0).stop 2023-10-16T10:08:26.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.655+0000 7f71f27ce640 1 -- 172.21.15.84:0/3029762011 shutdown_connections 2023-10-16T10:08:26.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.655+0000 7f71f27ce640 1 --2- 172.21.15.84:0/3029762011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71ec15d880 0x7f71ec15c440 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.655+0000 7f71f27ce640 1 --2- 172.21.15.84:0/3029762011 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f71cc06af90 0x7f71cc06d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.655+0000 7f71f27ce640 1 --2- 172.21.15.84:0/3029762011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71ec159f00 0x7f71ec154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.655+0000 7f71f27ce640 1 --2- 172.21.15.84:0/3029762011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71ec159b20 0x7f71ec155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.655+0000 7f71f27ce640 1 -- 172.21.15.84:0/3029762011 >> 172.21.15.84:0/3029762011 conn(0x7f71ec0cb410 msgr2=0x7f71ec0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:26.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.656+0000 7f71f27ce640 1 -- 172.21.15.84:0/3029762011 shutdown_connections 2023-10-16T10:08:26.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.656+0000 7f71f27ce640 1 -- 172.21.15.84:0/3029762011 wait complete. 2023-10-16T10:08:26.675 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:08:26.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.876+0000 7f5ad3fff640 1 Processor -- start 2023-10-16T10:08:26.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.876+0000 7f5ad3fff640 1 -- start start 2023-10-16T10:08:26.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.878+0000 7f5ad3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5ad406c040 0x7f5ad406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:26.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.878+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ad40ad970 0x7f5ad415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:26.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.878+0000 7f5ad3fff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ad406c960 con 0x7f5ad415ebc0 2023-10-16T10:08:26.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.878+0000 7f5ad3fff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ad415b9d0 con 0x7f5ad406c040 2023-10-16T10:08:26.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.878+0000 7f5ad3fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad415bcb0 con 0x7f5ad40ad970 2023-10-16T10:08:26.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.878+0000 7f5ad2ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5ad406c040 0x7f5ad406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:26.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.879+0000 7f5ad2ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5ad406c040 0x7f5ad406c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51924/0 (socket says 172.21.15.84:51924) 2023-10-16T10:08:26.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.879+0000 7f5ad2ffd640 1 -- 172.21.15.84:0/1465405598 learned_addr learned my addr 172.21.15.84:0/1465405598 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:26.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.879+0000 7f5acbfff640 1 --2- 172.21.15.84:0/1465405598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ad40ad970 0x7f5ad415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:26.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.879+0000 7f5ad2ffd640 1 -- 172.21.15.84:0/1465405598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ad40ad970 msgr2=0x7f5ad415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:26.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.879+0000 7f5ad2ffd640 1 --2- 172.21.15.84:0/1465405598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ad40ad970 0x7f5ad415e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.879+0000 7f5ad2ffd640 1 -- 172.21.15.84:0/1465405598 >> v1:172.21.15.157:6789/0 conn(0x7f5ad415ebc0 legacy=0x7f5ad40cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:26.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.879+0000 7f5ad2ffd640 1 -- 172.21.15.84:0/1465405598 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad415bf90 con 0x7f5ad406c040 2023-10-16T10:08:26.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.880+0000 7f5ad2ffd640 1 --2- 172.21.15.84:0/1465405598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ad406c040 0x7f5ad406c420 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40098a0 tx=0x7f5ac402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e3f4d9a630cbe5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:26.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.881+0000 7f5ad27fc640 1 -- 172.21.15.84:0/1465405598 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4030840 con 0x7f5ad406c040 2023-10-16T10:08:26.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.881+0000 7f5ad27fc640 1 -- 172.21.15.84:0/1465405598 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac40309a0 con 0x7f5ad406c040 2023-10-16T10:08:26.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.881+0000 7f5ad27fc640 1 -- 172.21.15.84:0/1465405598 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4030d30 con 0x7f5ad406c040 2023-10-16T10:08:26.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.881+0000 7f5ad3fff640 1 -- 172.21.15.84:0/1465405598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ad406c040 msgr2=0x7f5ad406c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:26.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.881+0000 7f5ad3fff640 1 --2- 172.21.15.84:0/1465405598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ad406c040 0x7f5ad406c420 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40098a0 tx=0x7f5ac402b980 comp rx=0 tx=0).stop 2023-10-16T10:08:26.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.882+0000 7f5ad3fff640 1 -- 172.21.15.84:0/1465405598 shutdown_connections 2023-10-16T10:08:26.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.882+0000 7f5ad3fff640 1 --2- 172.21.15.84:0/1465405598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ad40ad970 0x7f5ad415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.882+0000 7f5ad3fff640 1 --2- 172.21.15.84:0/1465405598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ad406c040 0x7f5ad406c420 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.882+0000 7f5ad3fff640 1 -- 172.21.15.84:0/1465405598 >> 172.21.15.84:0/1465405598 conn(0x7f5ad40ce6a0 msgr2=0x7f5ad40ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:26.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.882+0000 7f5ad3fff640 1 -- 172.21.15.84:0/1465405598 shutdown_connections 2023-10-16T10:08:26.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.882+0000 7f5ad3fff640 1 -- 172.21.15.84:0/1465405598 wait complete. 2023-10-16T10:08:26.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.883+0000 7f5ad3fff640 1 Processor -- start 2023-10-16T10:08:26.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.883+0000 7f5ad3fff640 1 -- start start 2023-10-16T10:08:26.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.884+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ad406c040 0x7f5ad41f2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:26.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.884+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ad40ad970 0x7f5ad414a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:26.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.884+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ad415ebc0 0x7f5ad415c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:26.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.884+0000 7f5ad3fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad40b0a10 con 0x7f5ad415ebc0 2023-10-16T10:08:26.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.884+0000 7f5ad3fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad414a7f0 con 0x7f5ad406c040 2023-10-16T10:08:26.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.884+0000 7f5ad3fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad40b0cb0 con 0x7f5ad40ad970 2023-10-16T10:08:26.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.885+0000 7f5ad37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ad415ebc0 0x7f5ad415c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:26.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.885+0000 7f5ad37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ad415ebc0 0x7f5ad415c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51928/0 (socket says 172.21.15.84:51928) 2023-10-16T10:08:26.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.885+0000 7f5ad37fe640 1 -- 172.21.15.84:0/1003180063 learned_addr learned my addr 172.21.15.84:0/1003180063 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:26.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.885+0000 7f5acbfff640 1 --2- 172.21.15.84:0/1003180063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ad40ad970 0x7f5ad414a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:26.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.885+0000 7f5ad2ffd640 1 --2- 172.21.15.84:0/1003180063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ad406c040 0x7f5ad41f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:26.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.885+0000 7f5ad37fe640 1 -- 172.21.15.84:0/1003180063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ad40ad970 msgr2=0x7f5ad414a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:26.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.885+0000 7f5ad37fe640 1 --2- 172.21.15.84:0/1003180063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ad40ad970 0x7f5ad414a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.885+0000 7f5ad37fe640 1 -- 172.21.15.84:0/1003180063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ad406c040 msgr2=0x7f5ad41f2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:26.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.885+0000 7f5ad37fe640 1 --2- 172.21.15.84:0/1003180063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ad406c040 0x7f5ad41f2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:26.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.885+0000 7f5ad37fe640 1 -- 172.21.15.84:0/1003180063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac40094d0 con 0x7f5ad415ebc0 2023-10-16T10:08:26.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.885+0000 7f5acbfff640 1 --2- 172.21.15.84:0/1003180063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ad40ad970 0x7f5ad414a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:26.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.885+0000 7f5ad2ffd640 1 --2- 172.21.15.84:0/1003180063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ad406c040 0x7f5ad41f2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:26.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.885+0000 7f5ad37fe640 1 --2- 172.21.15.84:0/1003180063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ad415ebc0 0x7f5ad415c970 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00c7a0 tx=0x7f5acc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:26.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.886+0000 7f5ad0ff9640 1 -- 172.21.15.84:0/1003180063 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc003b70 con 0x7f5ad415ebc0 2023-10-16T10:08:26.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.886+0000 7f5ad0ff9640 1 -- 172.21.15.84:0/1003180063 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5acc003cf0 con 0x7f5ad415ebc0 2023-10-16T10:08:26.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.886+0000 7f5ad0ff9640 1 -- 172.21.15.84:0/1003180063 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc003ea0 con 0x7f5ad415ebc0 2023-10-16T10:08:26.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.886+0000 7f5ad3fff640 1 -- 172.21.15.84:0/1003180063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad41478f0 con 0x7f5ad415ebc0 2023-10-16T10:08:26.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.886+0000 7f5ad3fff640 1 -- 172.21.15.84:0/1003180063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad4147eb0 con 0x7f5ad415ebc0 2023-10-16T10:08:26.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.887+0000 7f5ad3fff640 1 -- 172.21.15.84:0/1003180063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad406b330 con 0x7f5ad415ebc0 2023-10-16T10:08:26.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.888+0000 7f5ad0ff9640 1 -- 172.21.15.84:0/1003180063 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5acc00aa90 con 0x7f5ad415ebc0 2023-10-16T10:08:26.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.889+0000 7f5ad0ff9640 1 --2- 172.21.15.84:0/1003180063 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f5ab406af90 0x7f5ab406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:26.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.889+0000 7f5ad0ff9640 1 -- 172.21.15.84:0/1003180063 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5acc0896b0 con 0x7f5ad415ebc0 2023-10-16T10:08:26.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.889+0000 7f5ad2ffd640 1 --2- 172.21.15.84:0/1003180063 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f5ab406af90 0x7f5ab406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:26.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.890+0000 7f5ad2ffd640 1 --2- 172.21.15.84:0/1003180063 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f5ab406af90 0x7f5ab406d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4036000 tx=0x7f5ac403b040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:26.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:26.893+0000 7f5ad0ff9640 1 -- 172.21.15.84:0/1003180063 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5acc054590 con 0x7f5ad415ebc0 2023-10-16T10:08:27.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.085+0000 7f5ad3fff640 1 -- 172.21.15.84:0/1003180063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ad41480d0 con 0x7f5ad415ebc0 2023-10-16T10:08:27.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.087+0000 7f5ad0ff9640 1 -- 172.21.15.84:0/1003180063 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16560 (secure 0 0 0) 0x7f5acc057bb0 con 0x7f5ad415ebc0 2023-10-16T10:08:27.090 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:27.090 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":137,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:08:25.704831+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:08:23.558506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-16T10:08:24.709464+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:08:27.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.089+0000 7f5ad3fff640 1 -- 172.21.15.84:0/1003180063 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f5ab406af90 msgr2=0x7f5ab406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:27.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.089+0000 7f5ad3fff640 1 --2- 172.21.15.84:0/1003180063 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f5ab406af90 0x7f5ab406d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4036000 tx=0x7f5ac403b040 comp rx=0 tx=0).stop 2023-10-16T10:08:27.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.090+0000 7f5ad3fff640 1 -- 172.21.15.84:0/1003180063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ad415ebc0 msgr2=0x7f5ad415c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:27.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.090+0000 7f5ad3fff640 1 --2- 172.21.15.84:0/1003180063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ad415ebc0 0x7f5ad415c970 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00c7a0 tx=0x7f5acc00cc60 comp rx=0 tx=0).stop 2023-10-16T10:08:27.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.090+0000 7f5ad3fff640 1 -- 172.21.15.84:0/1003180063 shutdown_connections 2023-10-16T10:08:27.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.090+0000 7f5ad3fff640 1 --2- 172.21.15.84:0/1003180063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ad415ebc0 0x7f5ad415c970 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:27.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.090+0000 7f5ad3fff640 1 --2- 172.21.15.84:0/1003180063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ad40ad970 0x7f5ad414a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:27.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.090+0000 7f5ad3fff640 1 --2- 172.21.15.84:0/1003180063 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f5ab406af90 0x7f5ab406d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:27.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.090+0000 7f5ad3fff640 1 --2- 172.21.15.84:0/1003180063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ad406c040 0x7f5ad41f2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:27.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.090+0000 7f5ad3fff640 1 -- 172.21.15.84:0/1003180063 >> 172.21.15.84:0/1003180063 conn(0x7f5ad40ce6a0 msgr2=0x7f5ad40c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:27.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.090+0000 7f5ad3fff640 1 -- 172.21.15.84:0/1003180063 shutdown_connections 2023-10-16T10:08:27.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.090+0000 7f5ad3fff640 1 -- 172.21.15.84:0/1003180063 wait complete. 2023-10-16T10:08:27.111 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T10:08:27.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.331+0000 7f33b9ecf640 1 Processor -- start 2023-10-16T10:08:27.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.332+0000 7f33b9ecf640 1 -- start start 2023-10-16T10:08:27.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.333+0000 7f33b9ecf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f33b406a4b0 0x7f33b40ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:27.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.333+0000 7f33b9ecf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b40cb030 con 0x7f33b40c9ca0 2023-10-16T10:08:27.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.333+0000 7f33b9ecf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b40c91c0 con 0x7f33b415ad70 2023-10-16T10:08:27.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.333+0000 7f33b9ecf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b40c94a0 con 0x7f33b406a4b0 2023-10-16T10:08:27.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.333+0000 7f33b3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f33b40c9ca0 0x7f33b40ad4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55388/0 (socket says 172.21.15.84:55388) 2023-10-16T10:08:27.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.333+0000 7f33b3fff640 1 -- 172.21.15.84:0/4225310655 learned_addr learned my addr 172.21.15.84:0/4225310655 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:27.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.334+0000 7f33b2ffd640 1 -- 172.21.15.84:0/4225310655 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219480971 0 0) 0x7f33b40cb030 con 0x7f33b40c9ca0 2023-10-16T10:08:27.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.334+0000 7f33b2ffd640 1 -- 172.21.15.84:0/4225310655 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33a0003520 con 0x7f33b40c9ca0 2023-10-16T10:08:27.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.334+0000 7f33abfff640 1 --2- 172.21.15.84:0/4225310655 >> v2:172.21.15.157:3300/0 conn(0x7f33b406a4b0 0x7f33b40ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:27.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.334+0000 7f33b2ffd640 1 -- 172.21.15.84:0/4225310655 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1313405966 0 0) 0x7f33a0003520 con 0x7f33b40c9ca0 2023-10-16T10:08:27.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.334+0000 7f33b2ffd640 1 -- 172.21.15.84:0/4225310655 >> v2:172.21.15.157:3300/0 conn(0x7f33b406a4b0 msgr2=0x7f33b40ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:27.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.334+0000 7f33b2ffd640 1 --2- 172.21.15.84:0/4225310655 >> v2:172.21.15.157:3300/0 conn(0x7f33b406a4b0 0x7f33b40ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:27.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.334+0000 7f33b2ffd640 1 -- 172.21.15.84:0/4225310655 >> v1:172.21.15.157:6789/0 conn(0x7f33b415ad70 legacy=0x7f33b4069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:27.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.334+0000 7f33b2ffd640 1 -- 172.21.15.84:0/4225310655 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b40c9780 con 0x7f33b40c9ca0 2023-10-16T10:08:27.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.334+0000 7f33b2ffd640 1 -- 172.21.15.84:0/4225310655 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f33ac0019f0 con 0x7f33b40c9ca0 2023-10-16T10:08:27.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.334+0000 7f33b2ffd640 1 -- 172.21.15.84:0/4225310655 >> v1:172.21.15.84:6789/0 conn(0x7f33b40c9ca0 legacy=0x7f33b40ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:27.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.334+0000 7f33b2ffd640 1 --2- 172.21.15.84:0/4225310655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a0003970 0x7f33a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:27.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.334+0000 7f33b2ffd640 1 --2- 172.21.15.84:0/4225310655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00047c0 0x7f33a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:27.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.334+0000 7f33b2ffd640 1 --2- 172.21.15.84:0/4225310655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a0005180 0x7f33a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:27.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.334+0000 7f33b2ffd640 1 -- 172.21.15.84:0/4225310655 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40c94a0 con 0x7f33a00047c0 2023-10-16T10:08:27.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.334+0000 7f33b2ffd640 1 -- 172.21.15.84:0/4225310655 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a000c0d0 con 0x7f33a0003970 2023-10-16T10:08:27.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.334+0000 7f33b2ffd640 1 -- 172.21.15.84:0/4225310655 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a000c290 con 0x7f33a0005180 2023-10-16T10:08:27.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.335+0000 7f33abfff640 1 --2- 172.21.15.84:0/4225310655 >> v2:172.21.15.157:3300/0 conn(0x7f33b406a4b0 0x7f33b40ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:08:27.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.335+0000 7f33abfff640 1 --2- 172.21.15.84:0/4225310655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00047c0 0x7f33a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:27.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.335+0000 7f33b37fe640 1 --2- 172.21.15.84:0/4225310655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a0003970 0x7f33a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:27.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.335+0000 7f33b3fff640 1 --2- 172.21.15.84:0/4225310655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a0005180 0x7f33a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:27.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.335+0000 7f33abfff640 1 -- 172.21.15.84:0/4225310655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a0005180 msgr2=0x7f33a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:27.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.335+0000 7f33abfff640 1 --2- 172.21.15.84:0/4225310655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a0005180 0x7f33a000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:27.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.335+0000 7f33abfff640 1 -- 172.21.15.84:0/4225310655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a0003970 msgr2=0x7f33a0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:27.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.335+0000 7f33abfff640 1 --2- 172.21.15.84:0/4225310655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a0003970 0x7f33a0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:27.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.335+0000 7f33abfff640 1 -- 172.21.15.84:0/4225310655 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a0003520 con 0x7f33a00047c0 2023-10-16T10:08:27.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.335+0000 7f33b3fff640 1 --2- 172.21.15.84:0/4225310655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a0005180 0x7f33a000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:27.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.335+0000 7f33b3fff640 1 -- 172.21.15.84:0/4225310655 reap_dead start 2023-10-16T10:08:27.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.336+0000 7f33abfff640 1 --2- 172.21.15.84:0/4225310655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00047c0 0x7f33a0004c40 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f339c0098a0 tx=0x7f33a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea308194845834c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:27.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.337+0000 7f33b2ffd640 1 -- 172.21.15.84:0/4225310655 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c030620 con 0x7f33a00047c0 2023-10-16T10:08:27.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.337+0000 7f33b2ffd640 1 -- 172.21.15.84:0/4225310655 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c030780 con 0x7f33a00047c0 2023-10-16T10:08:27.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.337+0000 7f33b2ffd640 1 -- 172.21.15.84:0/4225310655 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c030af0 con 0x7f33a00047c0 2023-10-16T10:08:27.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.337+0000 7f33b9ecf640 1 -- 172.21.15.84:0/4225310655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00047c0 msgr2=0x7f33a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:27.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.337+0000 7f33b9ecf640 1 --2- 172.21.15.84:0/4225310655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00047c0 0x7f33a0004c40 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f339c0098a0 tx=0x7f33a00057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:27.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.337+0000 7f33b9ecf640 1 -- 172.21.15.84:0/4225310655 shutdown_connections 2023-10-16T10:08:27.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.337+0000 7f33b9ecf640 1 --2- 172.21.15.84:0/4225310655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00047c0 0x7f33a0004c40 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:27.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.337+0000 7f33b9ecf640 1 -- 172.21.15.84:0/4225310655 >> 172.21.15.84:0/4225310655 conn(0x7f33b40ce9b0 msgr2=0x7f33b40cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:27.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.337+0000 7f33b9ecf640 1 -- 172.21.15.84:0/4225310655 shutdown_connections 2023-10-16T10:08:27.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.337+0000 7f33b9ecf640 1 -- 172.21.15.84:0/4225310655 wait complete. 2023-10-16T10:08:27.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.339+0000 7f33b9ecf640 1 Processor -- start 2023-10-16T10:08:27.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.339+0000 7f33b9ecf640 1 -- start start 2023-10-16T10:08:27.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.340+0000 7f33b9ecf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00047c0 0x7f33b40b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:27.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.340+0000 7f33b9ecf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b40a8e40 0x7f33b4158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:27.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.340+0000 7f33b9ecf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33b40c25a0 0x7f33b40c2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:27.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.340+0000 7f33b9ecf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40c2f90 con 0x7f33a00047c0 2023-10-16T10:08:27.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.340+0000 7f33b9ecf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40b34f0 con 0x7f33b40a8e40 2023-10-16T10:08:27.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.340+0000 7f33b9ecf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40a92c0 con 0x7f33b40c25a0 2023-10-16T10:08:27.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.340+0000 7f33b37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00047c0 0x7f33b40b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:27.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.340+0000 7f33b37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00047c0 0x7f33b40b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51940/0 (socket says 172.21.15.84:51940) 2023-10-16T10:08:27.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.340+0000 7f33b37fe640 1 -- 172.21.15.84:0/1146162144 learned_addr learned my addr 172.21.15.84:0/1146162144 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:27.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.341+0000 7f33b37fe640 1 -- 172.21.15.84:0/1146162144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33b40c25a0 msgr2=0x7f33b40c2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:27.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.341+0000 7f33b37fe640 1 --2- 172.21.15.84:0/1146162144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33b40c25a0 0x7f33b40c2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:27.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.341+0000 7f33b37fe640 1 -- 172.21.15.84:0/1146162144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b40a8e40 msgr2=0x7f33b4158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:27.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.341+0000 7f33b37fe640 1 --2- 172.21.15.84:0/1146162144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b40a8e40 0x7f33b4158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:27.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.341+0000 7f33b37fe640 1 -- 172.21.15.84:0/1146162144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b41592b0 con 0x7f33a00047c0 2023-10-16T10:08:27.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.341+0000 7f33b37fe640 1 --2- 172.21.15.84:0/1146162144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00047c0 0x7f33b40b2f10 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f33a400e580 tx=0x7f33a400ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:27.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.341+0000 7f33b0ff9640 1 -- 172.21.15.84:0/1146162144 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a40052e0 con 0x7f33a00047c0 2023-10-16T10:08:27.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.341+0000 7f33b0ff9640 1 -- 172.21.15.84:0/1146162144 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a4005440 con 0x7f33a00047c0 2023-10-16T10:08:27.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.341+0000 7f33b0ff9640 1 -- 172.21.15.84:0/1146162144 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4003810 con 0x7f33a00047c0 2023-10-16T10:08:27.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.341+0000 7f33b9ecf640 1 -- 172.21.15.84:0/1146162144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b40b05b0 con 0x7f33a00047c0 2023-10-16T10:08:27.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.341+0000 7f33b9ecf640 1 -- 172.21.15.84:0/1146162144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b40b0b70 con 0x7f33a00047c0 2023-10-16T10:08:27.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.342+0000 7f33b9ecf640 1 -- 172.21.15.84:0/1146162144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b404e160 con 0x7f33a00047c0 2023-10-16T10:08:27.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.343+0000 7f33b0ff9640 1 -- 172.21.15.84:0/1146162144 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f33a4003970 con 0x7f33a00047c0 2023-10-16T10:08:27.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.344+0000 7f33b0ff9640 1 --2- 172.21.15.84:0/1146162144 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f339406af90 0x7f339406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:27.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.344+0000 7f33b0ff9640 1 -- 172.21.15.84:0/1146162144 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f33a40894e0 con 0x7f33a00047c0 2023-10-16T10:08:27.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.344+0000 7f33abfff640 1 --2- 172.21.15.84:0/1146162144 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f339406af90 0x7f339406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:27.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.345+0000 7f33abfff640 1 --2- 172.21.15.84:0/1146162144 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f339406af90 0x7f339406d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f339c036000 tx=0x7f339c030000 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:27.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.348+0000 7f33b0ff9640 1 -- 172.21.15.84:0/1146162144 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f33a4054470 con 0x7f33a00047c0 2023-10-16T10:08:27.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:27.578+0000 7f33b9ecf640 1 -- 172.21.15.84:0/1146162144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f33b40af7a0 con 0x7f33a00047c0 2023-10-16T10:08:28.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.276+0000 7f33b0ff9640 1 -- 172.21.15.84:0/1146162144 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) v1 ==== 118+0+0 (secure 0 0 0) 0x7f33a4057a90 con 0x7f33a00047c0 2023-10-16T10:08:28.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.280+0000 7f33b9ecf640 1 -- 172.21.15.84:0/1146162144 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f339406af90 msgr2=0x7f339406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:28.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.280+0000 7f33b9ecf640 1 --2- 172.21.15.84:0/1146162144 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f339406af90 0x7f339406d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f339c036000 tx=0x7f339c030000 comp rx=0 tx=0).stop 2023-10-16T10:08:28.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.280+0000 7f33b9ecf640 1 -- 172.21.15.84:0/1146162144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00047c0 msgr2=0x7f33b40b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:28.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.280+0000 7f33b9ecf640 1 --2- 172.21.15.84:0/1146162144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00047c0 0x7f33b40b2f10 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f33a400e580 tx=0x7f33a400ea40 comp rx=0 tx=0).stop 2023-10-16T10:08:28.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.280+0000 7f33b9ecf640 1 -- 172.21.15.84:0/1146162144 shutdown_connections 2023-10-16T10:08:28.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.280+0000 7f33b9ecf640 1 --2- 172.21.15.84:0/1146162144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33b40c25a0 0x7f33b40c2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:28.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.280+0000 7f33b9ecf640 1 --2- 172.21.15.84:0/1146162144 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f339406af90 0x7f339406d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:28.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.280+0000 7f33b9ecf640 1 --2- 172.21.15.84:0/1146162144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b40a8e40 0x7f33b4158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:28.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.280+0000 7f33b9ecf640 1 --2- 172.21.15.84:0/1146162144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00047c0 0x7f33b40b2f10 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:28.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.280+0000 7f33b9ecf640 1 -- 172.21.15.84:0/1146162144 >> 172.21.15.84:0/1146162144 conn(0x7f33b40ce9b0 msgr2=0x7f33b415de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:28.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.281+0000 7f33b9ecf640 1 -- 172.21.15.84:0/1146162144 shutdown_connections 2023-10-16T10:08:28.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.281+0000 7f33b9ecf640 1 -- 172.21.15.84:0/1146162144 wait complete. 2023-10-16T10:08:28.302 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T10:08:28.302 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T10:08:28.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.514+0000 7fa84d861640 1 Processor -- start 2023-10-16T10:08:28.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.514+0000 7fa84d861640 1 -- start start 2023-10-16T10:08:28.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.516+0000 7fa84d861640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa84806a4b0 0x7fa8480ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:28.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.516+0000 7fa84d861640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa8480c9ca0 0x7fa8480ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:28.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.516+0000 7fa84d861640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8480cb030 con 0x7fa84815ad70 2023-10-16T10:08:28.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.516+0000 7fa84d861640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8480cae30 con 0x7fa84806a4b0 2023-10-16T10:08:28.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.516+0000 7fa84d861640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8480c91c0 con 0x7fa8480c9ca0 2023-10-16T10:08:28.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.516+0000 7fa83ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa84806a4b0 0x7fa8480ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:28.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.516+0000 7fa83ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa84806a4b0 0x7fa8480ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51946/0 (socket says 172.21.15.84:51946) 2023-10-16T10:08:28.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.516+0000 7fa846ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa84815ad70 0x7fa848069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55400/0 (socket says 172.21.15.84:55400) 2023-10-16T10:08:28.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.516+0000 7fa83ffff640 1 -- 172.21.15.84:0/2023463962 learned_addr learned my addr 172.21.15.84:0/2023463962 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:28.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.517+0000 7fa8477fe640 1 --2- 172.21.15.84:0/2023463962 >> v2:172.21.15.157:3300/0 conn(0x7fa8480c9ca0 0x7fa8480ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:28.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.517+0000 7fa83ffff640 1 -- 172.21.15.84:0/2023463962 >> v2:172.21.15.157:3300/0 conn(0x7fa8480c9ca0 msgr2=0x7fa8480ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:28.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.517+0000 7fa83ffff640 1 --2- 172.21.15.84:0/2023463962 >> v2:172.21.15.157:3300/0 conn(0x7fa8480c9ca0 0x7fa8480ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:28.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.517+0000 7fa83ffff640 1 -- 172.21.15.84:0/2023463962 >> v1:172.21.15.84:6789/0 conn(0x7fa84815ad70 legacy=0x7fa848069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:28.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.517+0000 7fa83ffff640 1 -- 172.21.15.84:0/2023463962 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8480c94a0 con 0x7fa84806a4b0 2023-10-16T10:08:28.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.517+0000 7fa8467fc640 1 -- 172.21.15.84:0/2023463962 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112715305 0 0) 0x7fa8480cb030 con 0x7fa84815ad70 2023-10-16T10:08:28.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.517+0000 7fa8477fe640 1 --2- 172.21.15.84:0/2023463962 >> v2:172.21.15.157:3300/0 conn(0x7fa8480c9ca0 0x7fa8480ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:28.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.518+0000 7fa83ffff640 1 --2- 172.21.15.84:0/2023463962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa84806a4b0 0x7fa8480ca3b0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fa84806f270 tx=0x7fa83002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa350364d4122454 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:28.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.518+0000 7fa8467fc640 1 -- 172.21.15.84:0/2023463962 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830030860 con 0x7fa84806a4b0 2023-10-16T10:08:28.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.518+0000 7fa8467fc640 1 -- 172.21.15.84:0/2023463962 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8300309c0 con 0x7fa84806a4b0 2023-10-16T10:08:28.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.518+0000 7fa8467fc640 1 -- 172.21.15.84:0/2023463962 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830030d50 con 0x7fa84806a4b0 2023-10-16T10:08:28.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.519+0000 7fa84d861640 1 -- 172.21.15.84:0/2023463962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa84806a4b0 msgr2=0x7fa8480ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:28.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.519+0000 7fa84d861640 1 --2- 172.21.15.84:0/2023463962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa84806a4b0 0x7fa8480ca3b0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fa84806f270 tx=0x7fa83002b9a0 comp rx=0 tx=0).stop 2023-10-16T10:08:28.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.519+0000 7fa84d861640 1 -- 172.21.15.84:0/2023463962 shutdown_connections 2023-10-16T10:08:28.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.519+0000 7fa84d861640 1 --2- 172.21.15.84:0/2023463962 >> v2:172.21.15.157:3300/0 conn(0x7fa8480c9ca0 0x7fa8480ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:28.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.519+0000 7fa84d861640 1 --2- 172.21.15.84:0/2023463962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa84806a4b0 0x7fa8480ca3b0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:28.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.519+0000 7fa84d861640 1 -- 172.21.15.84:0/2023463962 >> 172.21.15.84:0/2023463962 conn(0x7fa8480ce9b0 msgr2=0x7fa8480cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:28.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.519+0000 7fa84d861640 1 -- 172.21.15.84:0/2023463962 shutdown_connections 2023-10-16T10:08:28.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.519+0000 7fa84d861640 1 -- 172.21.15.84:0/2023463962 wait complete. 2023-10-16T10:08:28.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.521+0000 7fa84d861640 1 Processor -- start 2023-10-16T10:08:28.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.521+0000 7fa84d861640 1 -- start start 2023-10-16T10:08:28.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.522+0000 7fa84d861640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa84806a4b0 0x7fa848158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:28.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.522+0000 7fa84d861640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8480c9ca0 0x7fa8480c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:28.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.522+0000 7fa84d861640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84815ad70 0x7fa8480c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:28.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.522+0000 7fa84d861640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8481f0bb0 con 0x7fa8480c9ca0 2023-10-16T10:08:28.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.522+0000 7fa84d861640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa848159b60 con 0x7fa84815ad70 2023-10-16T10:08:28.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.522+0000 7fa84d861640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8481f08d0 con 0x7fa84806a4b0 2023-10-16T10:08:28.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.522+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8480c9ca0 0x7fa8480c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:28.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.522+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8480c9ca0 0x7fa8480c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51960/0 (socket says 172.21.15.84:51960) 2023-10-16T10:08:28.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.522+0000 7fa83ffff640 1 -- 172.21.15.84:0/2580079521 learned_addr learned my addr 172.21.15.84:0/2580079521 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:28.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.523+0000 7fa83ffff640 1 -- 172.21.15.84:0/2580079521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa84806a4b0 msgr2=0x7fa848158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:28.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.523+0000 7fa83ffff640 1 --2- 172.21.15.84:0/2580079521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa84806a4b0 0x7fa848158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:28.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.523+0000 7fa83ffff640 1 -- 172.21.15.84:0/2580079521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84815ad70 msgr2=0x7fa8480c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:28.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.523+0000 7fa8477fe640 1 --2- 172.21.15.84:0/2580079521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84815ad70 0x7fa8480c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:28.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.523+0000 7fa83ffff640 1 --2- 172.21.15.84:0/2580079521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84815ad70 0x7fa8480c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:28.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.523+0000 7fa83ffff640 1 -- 172.21.15.84:0/2580079521 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8300094d0 con 0x7fa8480c9ca0 2023-10-16T10:08:28.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.523+0000 7fa8477fe640 1 --2- 172.21.15.84:0/2580079521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84815ad70 0x7fa8480c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:28.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.523+0000 7fa83ffff640 1 --2- 172.21.15.84:0/2580079521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8480c9ca0 0x7fa8480c7580 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa830000c00 tx=0x7fa830004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:28.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.523+0000 7fa844ff9640 1 -- 172.21.15.84:0/2580079521 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830030850 con 0x7fa8480c9ca0 2023-10-16T10:08:28.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.523+0000 7fa844ff9640 1 -- 172.21.15.84:0/2580079521 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa830044480 con 0x7fa8480c9ca0 2023-10-16T10:08:28.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.523+0000 7fa844ff9640 1 -- 172.21.15.84:0/2580079521 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830044830 con 0x7fa8480c9ca0 2023-10-16T10:08:28.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.523+0000 7fa84d861640 1 -- 172.21.15.84:0/2580079521 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8481f0ee0 con 0x7fa8480c9ca0 2023-10-16T10:08:28.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.524+0000 7fa84d861640 1 -- 172.21.15.84:0/2580079521 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8481f14a0 con 0x7fa8480c9ca0 2023-10-16T10:08:28.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.525+0000 7fa84d861640 1 -- 172.21.15.84:0/2580079521 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84804e160 con 0x7fa8480c9ca0 2023-10-16T10:08:28.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.526+0000 7fa844ff9640 1 -- 172.21.15.84:0/2580079521 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa83004d3b0 con 0x7fa8480c9ca0 2023-10-16T10:08:28.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.527+0000 7fa844ff9640 1 --2- 172.21.15.84:0/2580079521 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fa82806af90 0x7fa82806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:28.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.527+0000 7fa844ff9640 1 -- 172.21.15.84:0/2580079521 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa83003e070 con 0x7fa8480c9ca0 2023-10-16T10:08:28.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.527+0000 7fa846ffd640 1 --2- 172.21.15.84:0/2580079521 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fa82806af90 0x7fa82806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:28.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.527+0000 7fa846ffd640 1 --2- 172.21.15.84:0/2580079521 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fa82806af90 0x7fa82806d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa838007fd0 tx=0x7fa838007960 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:28.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.531+0000 7fa844ff9640 1 -- 172.21.15.84:0/2580079521 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa830077640 con 0x7fa8480c9ca0 2023-10-16T10:08:28.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:28.761+0000 7fa84d861640 1 -- 172.21.15.84:0/2580079521 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa8481f1660 con 0x7fa8480c9ca0 2023-10-16T10:08:29.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.280+0000 7fa844ff9640 1 -- 172.21.15.84:0/2580079521 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa83007ac60 con 0x7fa8480c9ca0 2023-10-16T10:08:29.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.283+0000 7fa84d861640 1 -- 172.21.15.84:0/2580079521 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fa82806af90 msgr2=0x7fa82806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:29.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.283+0000 7fa84d861640 1 --2- 172.21.15.84:0/2580079521 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fa82806af90 0x7fa82806d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa838007fd0 tx=0x7fa838007960 comp rx=0 tx=0).stop 2023-10-16T10:08:29.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.283+0000 7fa84d861640 1 -- 172.21.15.84:0/2580079521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8480c9ca0 msgr2=0x7fa8480c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:29.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.283+0000 7fa84d861640 1 --2- 172.21.15.84:0/2580079521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8480c9ca0 0x7fa8480c7580 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa830000c00 tx=0x7fa830004730 comp rx=0 tx=0).stop 2023-10-16T10:08:29.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.283+0000 7fa84d861640 1 -- 172.21.15.84:0/2580079521 shutdown_connections 2023-10-16T10:08:29.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.283+0000 7fa84d861640 1 --2- 172.21.15.84:0/2580079521 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fa82806af90 0x7fa82806d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:29.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.283+0000 7fa84d861640 1 --2- 172.21.15.84:0/2580079521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84815ad70 0x7fa8480c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:29.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.283+0000 7fa84d861640 1 --2- 172.21.15.84:0/2580079521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8480c9ca0 0x7fa8480c7580 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:29.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.283+0000 7fa84d861640 1 --2- 172.21.15.84:0/2580079521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa84806a4b0 0x7fa848158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:29.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.283+0000 7fa84d861640 1 -- 172.21.15.84:0/2580079521 >> 172.21.15.84:0/2580079521 conn(0x7fa8480ce9b0 msgr2=0x7fa8480b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:29.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.284+0000 7fa84d861640 1 -- 172.21.15.84:0/2580079521 shutdown_connections 2023-10-16T10:08:29.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.284+0000 7fa84d861640 1 -- 172.21.15.84:0/2580079521 wait complete. 2023-10-16T10:08:29.310 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T10:08:29.311 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:08:29.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.532+0000 7f8012287640 1 Processor -- start 2023-10-16T10:08:29.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.532+0000 7f8012287640 1 -- start start 2023-10-16T10:08:29.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.533+0000 7f8012287640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f800c15aaa0 0x7f800c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:29.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.534+0000 7f8012287640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f800c0ce390 0x7f800c070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:29.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.534+0000 7f8012287640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f800c15ed60 con 0x7f800c15ae80 2023-10-16T10:08:29.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.534+0000 7f8012287640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f800c06b810 con 0x7f800c0ce390 2023-10-16T10:08:29.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.534+0000 7f8012287640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f800c0707e0 con 0x7f800c15aaa0 2023-10-16T10:08:29.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.534+0000 7f8010824640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f800c0ce390 0x7f800c070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:29.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.534+0000 7f8010824640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f800c0ce390 0x7f800c070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51976/0 (socket says 172.21.15.84:51976) 2023-10-16T10:08:29.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.534+0000 7f8010824640 1 -- 172.21.15.84:0/3354069212 learned_addr learned my addr 172.21.15.84:0/3354069212 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:29.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.534+0000 7f8010824640 1 -- 172.21.15.84:0/3354069212 >> v2:172.21.15.157:3300/0 conn(0x7f800c15aaa0 msgr2=0x7f800c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:29.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.534+0000 7f800bfff640 1 --2- 172.21.15.84:0/3354069212 >> v2:172.21.15.157:3300/0 conn(0x7f800c15aaa0 0x7f800c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.534+0000 7f8010824640 1 --2- 172.21.15.84:0/3354069212 >> v2:172.21.15.157:3300/0 conn(0x7f800c15aaa0 0x7f800c15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.534+0000 7f8010824640 1 -- 172.21.15.84:0/3354069212 >> v1:172.21.15.84:6789/0 conn(0x7f800c15ae80 legacy=0x7f800c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.534+0000 7f8010824640 1 -- 172.21.15.84:0/3354069212 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c15ef90 con 0x7f800c0ce390 2023-10-16T10:08:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.536+0000 7f8010824640 1 --2- 172.21.15.84:0/3354069212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f800c0ce390 0x7f800c070f50 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f800c0cd990 tx=0x7f800402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1233ed515573a262 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.536+0000 7f800b7fe640 1 -- 172.21.15.84:0/3354069212 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800402cb10 con 0x7f800c0ce390 2023-10-16T10:08:29.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.536+0000 7f800b7fe640 1 -- 172.21.15.84:0/3354069212 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800402cc70 con 0x7f800c0ce390 2023-10-16T10:08:29.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.536+0000 7f800b7fe640 1 -- 172.21.15.84:0/3354069212 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040343f0 con 0x7f800c0ce390 2023-10-16T10:08:29.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.536+0000 7f8012287640 1 -- 172.21.15.84:0/3354069212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f800c0ce390 msgr2=0x7f800c070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:29.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.536+0000 7f8012287640 1 --2- 172.21.15.84:0/3354069212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f800c0ce390 0x7f800c070f50 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f800c0cd990 tx=0x7f800402bf70 comp rx=0 tx=0).stop 2023-10-16T10:08:29.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.537+0000 7f8012287640 1 -- 172.21.15.84:0/3354069212 shutdown_connections 2023-10-16T10:08:29.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.537+0000 7f8012287640 1 --2- 172.21.15.84:0/3354069212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f800c0ce390 0x7f800c070f50 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:29.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.537+0000 7f8012287640 1 --2- 172.21.15.84:0/3354069212 >> v2:172.21.15.157:3300/0 conn(0x7f800c15aaa0 0x7f800c15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:29.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.537+0000 7f8012287640 1 -- 172.21.15.84:0/3354069212 >> 172.21.15.84:0/3354069212 conn(0x7f800c0cb410 msgr2=0x7f800c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:29.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.537+0000 7f8012287640 1 -- 172.21.15.84:0/3354069212 shutdown_connections 2023-10-16T10:08:29.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.537+0000 7f8012287640 1 -- 172.21.15.84:0/3354069212 wait complete. 2023-10-16T10:08:29.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.538+0000 7f8012287640 1 Processor -- start 2023-10-16T10:08:29.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.539+0000 7f8012287640 1 -- start start 2023-10-16T10:08:29.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.540+0000 7f8012287640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f800c0ce390 0x7f800c159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:29.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.540+0000 7f8012287640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800c15aaa0 0x7f800c15d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:29.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.540+0000 7f8012287640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f800c15ae80 0x7f800c159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:29.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.540+0000 7f8012287640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c15c260 con 0x7f800c0ce390 2023-10-16T10:08:29.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.540+0000 7f8012287640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c15c540 con 0x7f800c15aaa0 2023-10-16T10:08:29.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.540+0000 7f8012287640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800c15c820 con 0x7f800c15ae80 2023-10-16T10:08:29.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.540+0000 7f800bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f800c0ce390 0x7f800c159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:29.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.540+0000 7f800bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f800c0ce390 0x7f800c159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51990/0 (socket says 172.21.15.84:51990) 2023-10-16T10:08:29.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.540+0000 7f800bfff640 1 -- 172.21.15.84:0/2761755598 learned_addr learned my addr 172.21.15.84:0/2761755598 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:29.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.540+0000 7f800bfff640 1 -- 172.21.15.84:0/2761755598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f800c15ae80 msgr2=0x7f800c159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:29.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.540+0000 7f800bfff640 1 --2- 172.21.15.84:0/2761755598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f800c15ae80 0x7f800c159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:29.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.540+0000 7f800bfff640 1 -- 172.21.15.84:0/2761755598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800c15aaa0 msgr2=0x7f800c15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:29.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.540+0000 7f800bfff640 1 --2- 172.21.15.84:0/2761755598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800c15aaa0 0x7f800c15d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:29.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.540+0000 7f800bfff640 1 -- 172.21.15.84:0/2761755598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8004009b30 con 0x7f800c0ce390 2023-10-16T10:08:29.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.541+0000 7f800bfff640 1 --2- 172.21.15.84:0/2761755598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f800c0ce390 0x7f800c159b00 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00c7f0 tx=0x7f7ffc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:29.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.541+0000 7f80097fa640 1 -- 172.21.15.84:0/2761755598 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc004220 con 0x7f800c0ce390 2023-10-16T10:08:29.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.541+0000 7f80097fa640 1 -- 172.21.15.84:0/2761755598 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ffc0043a0 con 0x7f800c0ce390 2023-10-16T10:08:29.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.541+0000 7f80097fa640 1 -- 172.21.15.84:0/2761755598 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc007860 con 0x7f800c0ce390 2023-10-16T10:08:29.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.541+0000 7f8012287640 1 -- 172.21.15.84:0/2761755598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c15cb00 con 0x7f800c0ce390 2023-10-16T10:08:29.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.541+0000 7f8012287640 1 -- 172.21.15.84:0/2761755598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c15a270 con 0x7f800c0ce390 2023-10-16T10:08:29.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.542+0000 7f8012287640 1 -- 172.21.15.84:0/2761755598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800c15d020 con 0x7f800c0ce390 2023-10-16T10:08:29.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.543+0000 7f80097fa640 1 -- 172.21.15.84:0/2761755598 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7ffc007bd0 con 0x7f800c0ce390 2023-10-16T10:08:29.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.544+0000 7f80097fa640 1 --2- 172.21.15.84:0/2761755598 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f7fec06afe0 0x7f7fec06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:29.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.544+0000 7f80097fa640 1 -- 172.21.15.84:0/2761755598 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f7ffc089800 con 0x7f800c0ce390 2023-10-16T10:08:29.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.544+0000 7f8003fff640 1 --2- 172.21.15.84:0/2761755598 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f7fec06afe0 0x7f7fec06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:29.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.545+0000 7f8003fff640 1 --2- 172.21.15.84:0/2761755598 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f7fec06afe0 0x7f7fec06d4a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4000f60 tx=0x7f7ff40079b0 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:29.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.549+0000 7f80097fa640 1 -- 172.21.15.84:0/2761755598 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f800c15d020 con 0x7f800c0ce390 2023-10-16T10:08:29.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.783+0000 7f8012287640 1 -- 172.21.15.84:0/2761755598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f800c15d020 con 0x7f800c0ce390 2023-10-16T10:08:29.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.784+0000 7f80097fa640 1 -- 172.21.15.84:0/2761755598 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f7ffc01e090 con 0x7f800c0ce390 2023-10-16T10:08:29.786 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:29.787 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4118285132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":4118285132},{"type":"v1","addr":"172.21.15.84:6837","nonce":4118285132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3743702675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3743702675},{"type":"v1","addr":"172.21.15.157:6834","nonce":3743702675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2895418246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2895418246},{"type":"v1","addr":"172.21.15.157:6835","nonce":2895418246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:08:25.705299+0000","modified":"2023-10-16T10:08:28.762900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8574":{"gid":8574,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/791907708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":791907708},{"type":"v1","addr":"172.21.15.84:6836","nonce":791907708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-16T10:08:29.787 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 71 2023-10-16T10:08:29.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.787+0000 7f8012287640 1 -- 172.21.15.84:0/2761755598 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f7fec06afe0 msgr2=0x7f7fec06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:29.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.787+0000 7f8012287640 1 --2- 172.21.15.84:0/2761755598 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f7fec06afe0 0x7f7fec06d4a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4000f60 tx=0x7f7ff40079b0 comp rx=0 tx=0).stop 2023-10-16T10:08:29.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.787+0000 7f8012287640 1 -- 172.21.15.84:0/2761755598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f800c0ce390 msgr2=0x7f800c159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:29.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.787+0000 7f8012287640 1 --2- 172.21.15.84:0/2761755598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f800c0ce390 0x7f800c159b00 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00c7f0 tx=0x7f7ffc00ccb0 comp rx=0 tx=0).stop 2023-10-16T10:08:29.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.787+0000 7f8012287640 1 -- 172.21.15.84:0/2761755598 shutdown_connections 2023-10-16T10:08:29.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.787+0000 7f8012287640 1 --2- 172.21.15.84:0/2761755598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f800c15ae80 0x7f800c159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:29.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.787+0000 7f8012287640 1 --2- 172.21.15.84:0/2761755598 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f7fec06afe0 0x7f7fec06d4a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:29.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.787+0000 7f8012287640 1 --2- 172.21.15.84:0/2761755598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800c15aaa0 0x7f800c15d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:29.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.787+0000 7f8012287640 1 --2- 172.21.15.84:0/2761755598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f800c0ce390 0x7f800c159b00 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:29.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.787+0000 7f8012287640 1 -- 172.21.15.84:0/2761755598 >> 172.21.15.84:0/2761755598 conn(0x7f800c0cb410 msgr2=0x7f800c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:29.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.788+0000 7f8012287640 1 -- 172.21.15.84:0/2761755598 shutdown_connections 2023-10-16T10:08:29.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:29.788+0000 7f8012287640 1 -- 172.21.15.84:0/2761755598 wait complete. 2023-10-16T10:08:29.809 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:08:30.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.016+0000 7f31c01b7640 1 Processor -- start 2023-10-16T10:08:30.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.016+0000 7f31c01b7640 1 -- start start 2023-10-16T10:08:30.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.018+0000 7f31c01b7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f31b815aaa0 0x7f31b815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.018+0000 7f31c01b7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31b80ce390 0x7f31b8070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.018+0000 7f31c01b7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b815ed60 con 0x7f31b815ae80 2023-10-16T10:08:30.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.018+0000 7f31c01b7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b806b810 con 0x7f31b815aaa0 2023-10-16T10:08:30.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.018+0000 7f31c01b7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b80707e0 con 0x7f31b80ce390 2023-10-16T10:08:30.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.018+0000 7f31bdf53640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f31b815aaa0 0x7f31b815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:30.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.018+0000 7f31bdf53640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f31b815aaa0 0x7f31b815e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52002/0 (socket says 172.21.15.84:52002) 2023-10-16T10:08:30.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.018+0000 7f31bdf53640 1 -- 172.21.15.84:0/3365076411 learned_addr learned my addr 172.21.15.84:0/3365076411 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:30.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.018+0000 7f31bdf53640 1 -- 172.21.15.84:0/3365076411 >> v2:172.21.15.157:3300/0 conn(0x7f31b80ce390 msgr2=0x7f31b8070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:30.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.018+0000 7f31bdf53640 1 --2- 172.21.15.84:0/3365076411 >> v2:172.21.15.157:3300/0 conn(0x7f31b80ce390 0x7f31b8070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.018+0000 7f31bdf53640 1 -- 172.21.15.84:0/3365076411 >> v1:172.21.15.84:6789/0 conn(0x7f31b815ae80 legacy=0x7f31b80cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:30.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.018+0000 7f31bdf53640 1 -- 172.21.15.84:0/3365076411 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b815ef90 con 0x7f31b815aaa0 2023-10-16T10:08:30.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.020+0000 7f31bdf53640 1 --2- 172.21.15.84:0/3365076411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31b815aaa0 0x7f31b815e820 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f31b80c2490 tx=0x7f31b402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d0e367dec3c04b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:30.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.020+0000 7f31bcf51640 1 -- 172.21.15.84:0/3365076411 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b4030860 con 0x7f31b815aaa0 2023-10-16T10:08:30.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.020+0000 7f31bcf51640 1 -- 172.21.15.84:0/3365076411 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b40309c0 con 0x7f31b815aaa0 2023-10-16T10:08:30.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.020+0000 7f31bcf51640 1 -- 172.21.15.84:0/3365076411 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b4030d50 con 0x7f31b815aaa0 2023-10-16T10:08:30.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.020+0000 7f31c01b7640 1 -- 172.21.15.84:0/3365076411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31b815aaa0 msgr2=0x7f31b815e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:30.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.020+0000 7f31c01b7640 1 --2- 172.21.15.84:0/3365076411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31b815aaa0 0x7f31b815e820 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f31b80c2490 tx=0x7f31b402b9a0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.020+0000 7f31c01b7640 1 -- 172.21.15.84:0/3365076411 shutdown_connections 2023-10-16T10:08:30.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.020+0000 7f31c01b7640 1 --2- 172.21.15.84:0/3365076411 >> v2:172.21.15.157:3300/0 conn(0x7f31b80ce390 0x7f31b8070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.020+0000 7f31c01b7640 1 --2- 172.21.15.84:0/3365076411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31b815aaa0 0x7f31b815e820 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.020+0000 7f31c01b7640 1 -- 172.21.15.84:0/3365076411 >> 172.21.15.84:0/3365076411 conn(0x7f31b80cb410 msgr2=0x7f31b80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:30.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.021+0000 7f31c01b7640 1 -- 172.21.15.84:0/3365076411 shutdown_connections 2023-10-16T10:08:30.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.021+0000 7f31c01b7640 1 -- 172.21.15.84:0/3365076411 wait complete. 2023-10-16T10:08:30.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.022+0000 7f31c01b7640 1 Processor -- start 2023-10-16T10:08:30.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.022+0000 7f31c01b7640 1 -- start start 2023-10-16T10:08:30.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.023+0000 7f31c01b7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31b80ce390 0x7f31b8157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.023+0000 7f31c01b7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31b815aaa0 0x7f31b80aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.023+0000 7f31c01b7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31b815ae80 0x7f31b80a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.023+0000 7f31c01b7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b81584e0 con 0x7f31b815aaa0 2023-10-16T10:08:30.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.023+0000 7f31c01b7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b815d340 con 0x7f31b80ce390 2023-10-16T10:08:30.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.023+0000 7f31c01b7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b815c730 con 0x7f31b815ae80 2023-10-16T10:08:30.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.024+0000 7f31bd752640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31b815aaa0 0x7f31b80aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:30.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.024+0000 7f31bd752640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31b815aaa0 0x7f31b80aa8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52010/0 (socket says 172.21.15.84:52010) 2023-10-16T10:08:30.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.024+0000 7f31bd752640 1 -- 172.21.15.84:0/2552301140 learned_addr learned my addr 172.21.15.84:0/2552301140 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:30.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.024+0000 7f31bdf53640 1 --2- 172.21.15.84:0/2552301140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31b80ce390 0x7f31b8157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:30.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.024+0000 7f31bd752640 1 -- 172.21.15.84:0/2552301140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31b815ae80 msgr2=0x7f31b80a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:30.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.024+0000 7f31bd752640 1 --2- 172.21.15.84:0/2552301140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31b815ae80 0x7f31b80a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.024+0000 7f31bd752640 1 -- 172.21.15.84:0/2552301140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31b80ce390 msgr2=0x7f31b8157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:30.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.024+0000 7f31bd752640 1 --2- 172.21.15.84:0/2552301140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31b80ce390 0x7f31b8157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.024+0000 7f31bd752640 1 -- 172.21.15.84:0/2552301140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b40094d0 con 0x7f31b815aaa0 2023-10-16T10:08:30.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.024+0000 7f31bd752640 1 --2- 172.21.15.84:0/2552301140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31b815aaa0 0x7f31b80aa8f0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f31a800c3c0 tx=0x7f31a800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:30.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.025+0000 7f31ae7fc640 1 -- 172.21.15.84:0/2552301140 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a80052d0 con 0x7f31b815aaa0 2023-10-16T10:08:30.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.025+0000 7f31ae7fc640 1 -- 172.21.15.84:0/2552301140 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a8005450 con 0x7f31b815aaa0 2023-10-16T10:08:30.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.025+0000 7f31ae7fc640 1 -- 172.21.15.84:0/2552301140 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8003830 con 0x7f31b815aaa0 2023-10-16T10:08:30.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.025+0000 7f31c01b7640 1 -- 172.21.15.84:0/2552301140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b815ca10 con 0x7f31b815aaa0 2023-10-16T10:08:30.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.025+0000 7f31c01b7640 1 -- 172.21.15.84:0/2552301140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b815cfd0 con 0x7f31b815aaa0 2023-10-16T10:08:30.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.026+0000 7f31c01b7640 1 -- 172.21.15.84:0/2552301140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b815d4b0 con 0x7f31b815aaa0 2023-10-16T10:08:30.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.027+0000 7f31ae7fc640 1 -- 172.21.15.84:0/2552301140 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f31a8003990 con 0x7f31b815aaa0 2023-10-16T10:08:30.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.028+0000 7f31ae7fc640 1 --2- 172.21.15.84:0/2552301140 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f319c06af90 0x7f319c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.028+0000 7f31ae7fc640 1 -- 172.21.15.84:0/2552301140 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f31a8088cb0 con 0x7f31b815aaa0 2023-10-16T10:08:30.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.028+0000 7f31bdf53640 1 --2- 172.21.15.84:0/2552301140 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f319c06af90 0x7f319c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:30.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.029+0000 7f31bdf53640 1 --2- 172.21.15.84:0/2552301140 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f319c06af90 0x7f319c06d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f31b4036000 tx=0x7f31b4008030 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:30.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.032+0000 7f31ae7fc640 1 -- 172.21.15.84:0/2552301140 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f31a8053c30 con 0x7f31b815aaa0 2023-10-16T10:08:30.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.218+0000 7f31c01b7640 1 -- 172.21.15.84:0/2552301140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31b804e160 con 0x7f31b815aaa0 2023-10-16T10:08:30.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.221+0000 7f31ae7fc640 1 -- 172.21.15.84:0/2552301140 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16547 (secure 0 0 0) 0x7f31a8017090 con 0x7f31b815aaa0 2023-10-16T10:08:30.223 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:30.223 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":138,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:08:27.250122+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:08:23.558506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-16T10:08:24.709464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:08:30.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.223+0000 7f31c01b7640 1 -- 172.21.15.84:0/2552301140 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f319c06af90 msgr2=0x7f319c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:30.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.223+0000 7f31c01b7640 1 --2- 172.21.15.84:0/2552301140 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f319c06af90 0x7f319c06d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f31b4036000 tx=0x7f31b4008030 comp rx=0 tx=0).stop 2023-10-16T10:08:30.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.223+0000 7f31c01b7640 1 -- 172.21.15.84:0/2552301140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31b815aaa0 msgr2=0x7f31b80aa8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:30.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.223+0000 7f31c01b7640 1 --2- 172.21.15.84:0/2552301140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31b815aaa0 0x7f31b80aa8f0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f31a800c3c0 tx=0x7f31a800c880 comp rx=0 tx=0).stop 2023-10-16T10:08:30.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.223+0000 7f31c01b7640 1 -- 172.21.15.84:0/2552301140 shutdown_connections 2023-10-16T10:08:30.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.223+0000 7f31c01b7640 1 --2- 172.21.15.84:0/2552301140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31b815ae80 0x7f31b80a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.223+0000 7f31c01b7640 1 --2- 172.21.15.84:0/2552301140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31b815aaa0 0x7f31b80aa8f0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.224+0000 7f31c01b7640 1 --2- 172.21.15.84:0/2552301140 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f319c06af90 0x7f319c06d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.224+0000 7f31c01b7640 1 --2- 172.21.15.84:0/2552301140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31b80ce390 0x7f31b8157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.224+0000 7f31c01b7640 1 -- 172.21.15.84:0/2552301140 >> 172.21.15.84:0/2552301140 conn(0x7f31b80cb410 msgr2=0x7f31b80b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:30.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.224+0000 7f31c01b7640 1 -- 172.21.15.84:0/2552301140 shutdown_connections 2023-10-16T10:08:30.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.224+0000 7f31c01b7640 1 -- 172.21.15.84:0/2552301140 wait complete. 2023-10-16T10:08:30.243 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T10:08:30.244 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:08:30.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.454+0000 7f6152bc0640 1 Processor -- start 2023-10-16T10:08:30.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.455+0000 7f6152bc0640 1 -- start start 2023-10-16T10:08:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.456+0000 7f6152bc0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f614c15aaa0 0x7f614c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.456+0000 7f6152bc0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f614c15ae80 0x7f614c0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.456+0000 7f6152bc0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614c15ed60 con 0x7f614c0ccaa0 2023-10-16T10:08:30.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.456+0000 7f6152bc0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f614c06b810 con 0x7f614c15ae80 2023-10-16T10:08:30.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.456+0000 7f6152bc0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f614c0707e0 con 0x7f614c15aaa0 2023-10-16T10:08:30.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.456+0000 7f614bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f614c15ae80 0x7f614c0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:30.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.456+0000 7f614bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f614c15ae80 0x7f614c0cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52012/0 (socket says 172.21.15.84:52012) 2023-10-16T10:08:30.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.456+0000 7f614bfff640 1 -- 172.21.15.84:0/3759004014 learned_addr learned my addr 172.21.15.84:0/3759004014 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:30.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.457+0000 7f614bfff640 1 -- 172.21.15.84:0/3759004014 >> v2:172.21.15.157:3300/0 conn(0x7f614c15aaa0 msgr2=0x7f614c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:30.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.457+0000 7f614bfff640 1 --2- 172.21.15.84:0/3759004014 >> v2:172.21.15.157:3300/0 conn(0x7f614c15aaa0 0x7f614c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.457+0000 7f614bfff640 1 -- 172.21.15.84:0/3759004014 >> v1:172.21.15.157:6789/0 conn(0x7f614c0ccaa0 legacy=0x7f614c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:30.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.457+0000 7f614bfff640 1 -- 172.21.15.84:0/3759004014 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614c15ef90 con 0x7f614c15ae80 2023-10-16T10:08:30.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.458+0000 7f614bfff640 1 --2- 172.21.15.84:0/3759004014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f614c15ae80 0x7f614c0cc560 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f614c0cd990 tx=0x7f613802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa68a487ee7c6a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:30.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.458+0000 7f614b7fe640 1 -- 172.21.15.84:0/3759004014 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138031870 con 0x7f614c15ae80 2023-10-16T10:08:30.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.458+0000 7f614b7fe640 1 -- 172.21.15.84:0/3759004014 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61380319d0 con 0x7f614c15ae80 2023-10-16T10:08:30.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.458+0000 7f614b7fe640 1 -- 172.21.15.84:0/3759004014 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138031d60 con 0x7f614c15ae80 2023-10-16T10:08:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.459+0000 7f6152bc0640 1 -- 172.21.15.84:0/3759004014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f614c15ae80 msgr2=0x7f614c0cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.459+0000 7f6152bc0640 1 --2- 172.21.15.84:0/3759004014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f614c15ae80 0x7f614c0cc560 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f614c0cd990 tx=0x7f613802b910 comp rx=0 tx=0).stop 2023-10-16T10:08:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.459+0000 7f6152bc0640 1 -- 172.21.15.84:0/3759004014 shutdown_connections 2023-10-16T10:08:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.459+0000 7f6152bc0640 1 --2- 172.21.15.84:0/3759004014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f614c15ae80 0x7f614c0cc560 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.459+0000 7f6152bc0640 1 --2- 172.21.15.84:0/3759004014 >> v2:172.21.15.157:3300/0 conn(0x7f614c15aaa0 0x7f614c15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.459+0000 7f6152bc0640 1 -- 172.21.15.84:0/3759004014 >> 172.21.15.84:0/3759004014 conn(0x7f614c0cb410 msgr2=0x7f614c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.459+0000 7f6152bc0640 1 -- 172.21.15.84:0/3759004014 shutdown_connections 2023-10-16T10:08:30.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.459+0000 7f6152bc0640 1 -- 172.21.15.84:0/3759004014 wait complete. 2023-10-16T10:08:30.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.461+0000 7f6152bc0640 1 Processor -- start 2023-10-16T10:08:30.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.461+0000 7f6152bc0640 1 -- start start 2023-10-16T10:08:30.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.462+0000 7f6152bc0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f614c0ccaa0 0x7f614c15a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.462+0000 7f6152bc0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f614c15aaa0 0x7f614c15d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.462+0000 7f6152bc0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f614c15ae80 0x7f614c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.462+0000 7f6152bc0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c0c44a0 con 0x7f614c15ae80 2023-10-16T10:08:30.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.462+0000 7f6152bc0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c0c52c0 con 0x7f614c0ccaa0 2023-10-16T10:08:30.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.463+0000 7f6152bc0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614c15a550 con 0x7f614c15aaa0 2023-10-16T10:08:30.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.463+0000 7f615115d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f614c15ae80 0x7f614c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:30.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.463+0000 7f615115d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f614c15ae80 0x7f614c0b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52028/0 (socket says 172.21.15.84:52028) 2023-10-16T10:08:30.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.463+0000 7f615115d640 1 -- 172.21.15.84:0/1330758144 learned_addr learned my addr 172.21.15.84:0/1330758144 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:30.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.463+0000 7f615095c640 1 --2- 172.21.15.84:0/1330758144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f614c0ccaa0 0x7f614c15a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:30.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.463+0000 7f614bfff640 1 --2- 172.21.15.84:0/1330758144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f614c15aaa0 0x7f614c15d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:30.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.463+0000 7f615115d640 1 -- 172.21.15.84:0/1330758144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f614c15aaa0 msgr2=0x7f614c15d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:30.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.463+0000 7f615115d640 1 --2- 172.21.15.84:0/1330758144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f614c15aaa0 0x7f614c15d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.463+0000 7f615115d640 1 -- 172.21.15.84:0/1330758144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f614c0ccaa0 msgr2=0x7f614c15a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:30.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.463+0000 7f615115d640 1 --2- 172.21.15.84:0/1330758144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f614c0ccaa0 0x7f614c15a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.463+0000 7f615115d640 1 -- 172.21.15.84:0/1330758144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61380094d0 con 0x7f614c15ae80 2023-10-16T10:08:30.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.464+0000 7f615115d640 1 --2- 172.21.15.84:0/1330758144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f614c15ae80 0x7f614c0b2500 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f614400c780 tx=0x7f614400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:30.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.464+0000 7f61497fa640 1 -- 172.21.15.84:0/1330758144 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6144003d80 con 0x7f614c15ae80 2023-10-16T10:08:30.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.464+0000 7f61497fa640 1 -- 172.21.15.84:0/1330758144 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614400ce90 con 0x7f614c15ae80 2023-10-16T10:08:30.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.464+0000 7f6152bc0640 1 -- 172.21.15.84:0/1330758144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f614c0c4740 con 0x7f614c15ae80 2023-10-16T10:08:30.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.464+0000 7f61497fa640 1 -- 172.21.15.84:0/1330758144 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61440052b0 con 0x7f614c15ae80 2023-10-16T10:08:30.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.464+0000 7f6152bc0640 1 -- 172.21.15.84:0/1330758144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614c0c4d00 con 0x7f614c15ae80 2023-10-16T10:08:30.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.465+0000 7f6152bc0640 1 -- 172.21.15.84:0/1330758144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f614c0a8c40 con 0x7f614c15ae80 2023-10-16T10:08:30.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.466+0000 7f61497fa640 1 -- 172.21.15.84:0/1330758144 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6144005410 con 0x7f614c15ae80 2023-10-16T10:08:30.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.467+0000 7f61497fa640 1 --2- 172.21.15.84:0/1330758144 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f613006af90 0x7f613006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.467+0000 7f61497fa640 1 -- 172.21.15.84:0/1330758144 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6144088fe0 con 0x7f614c15ae80 2023-10-16T10:08:30.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.467+0000 7f615095c640 1 --2- 172.21.15.84:0/1330758144 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f613006af90 0x7f613006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:30.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.468+0000 7f615095c640 1 --2- 172.21.15.84:0/1330758144 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f613006af90 0x7f613006d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6140009890 tx=0x7f6140008040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:30.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.473+0000 7f61497fa640 1 -- 172.21.15.84:0/1330758144 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f614c0a8c40 con 0x7f614c15ae80 2023-10-16T10:08:30.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.662+0000 7f6152bc0640 1 -- 172.21.15.84:0/1330758144 --> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f614c15f550 con 0x7f613006af90 2023-10-16T10:08:30.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.679+0000 7f61497fa640 1 -- 172.21.15.84:0/1330758144 <== mgr.8322 v2:172.21.15.84:6832/2784642648 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287680 (secure 0 0 0) 0x7f6140009480 con 0x7f613006af90 2023-10-16T10:08:30.684 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:30.686 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T10:08:30.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.685+0000 7f6152bc0640 1 -- 172.21.15.84:0/1330758144 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f613006af90 msgr2=0x7f613006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:30.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.685+0000 7f6152bc0640 1 --2- 172.21.15.84:0/1330758144 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f613006af90 0x7f613006d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6140009890 tx=0x7f6140008040 comp rx=0 tx=0).stop 2023-10-16T10:08:30.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.685+0000 7f6152bc0640 1 -- 172.21.15.84:0/1330758144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f614c15ae80 msgr2=0x7f614c0b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:30.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.685+0000 7f6152bc0640 1 --2- 172.21.15.84:0/1330758144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f614c15ae80 0x7f614c0b2500 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f614400c780 tx=0x7f614400cc40 comp rx=0 tx=0).stop 2023-10-16T10:08:30.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.685+0000 7f6152bc0640 1 -- 172.21.15.84:0/1330758144 shutdown_connections 2023-10-16T10:08:30.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.685+0000 7f6152bc0640 1 --2- 172.21.15.84:0/1330758144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f614c15ae80 0x7f614c0b2500 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.685+0000 7f6152bc0640 1 --2- 172.21.15.84:0/1330758144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f614c15aaa0 0x7f614c15d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.685+0000 7f6152bc0640 1 --2- 172.21.15.84:0/1330758144 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f613006af90 0x7f613006d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.685+0000 7f6152bc0640 1 --2- 172.21.15.84:0/1330758144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f614c0ccaa0 0x7f614c15a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.686+0000 7f6152bc0640 1 -- 172.21.15.84:0/1330758144 >> 172.21.15.84:0/1330758144 conn(0x7f614c0cb410 msgr2=0x7f614c06cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:30.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.686+0000 7f6152bc0640 1 -- 172.21.15.84:0/1330758144 shutdown_connections 2023-10-16T10:08:30.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.686+0000 7f6152bc0640 1 -- 172.21.15.84:0/1330758144 wait complete. 2023-10-16T10:08:30.706 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-16T10:08:30.450231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1120,"num_read_kb":2202,"num_write":405,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2684,"ondisk_log_size":2684,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":357020,"kb_used_data":14552,"kb_used_omap":12,"kb_used_meta":342323,"kb_avail":754617700,"statfs":{"total":773094113280,"available":772728524800,"internally_reserved":0,"allocated":14901248,"data_stored":7716788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":350539321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001788"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.134208+0000","last_change":"2023-10-16T10:08:26.264876+0000","last_active":"2023-10-16T10:08:29.134208+0000","last_peered":"2023-10-16T10:08:29.134208+0000","last_clean":"2023-10-16T10:08:29.134208+0000","last_became_active":"2023-10-16T10:08:26.264607+0000","last_became_peered":"2023-10-16T10:08:26.264607+0000","last_unstale":"2023-10-16T10:08:29.134208+0000","last_undegraded":"2023-10-16T10:08:29.134208+0000","last_fullsized":"2023-10-16T10:08:29.134208+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:51:06.545129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.140515+0000","last_change":"2023-10-16T10:08:26.264953+0000","last_active":"2023-10-16T10:08:29.140515+0000","last_peered":"2023-10-16T10:08:29.140515+0000","last_clean":"2023-10-16T10:08:29.140515+0000","last_became_active":"2023-10-16T10:08:26.264517+0000","last_became_peered":"2023-10-16T10:08:26.264517+0000","last_unstale":"2023-10-16T10:08:29.140515+0000","last_undegraded":"2023-10-16T10:08:29.140515+0000","last_fullsized":"2023-10-16T10:08:29.140515+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:20:22.674056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.135971+0000","last_change":"2023-10-16T10:08:26.272140+0000","last_active":"2023-10-16T10:08:29.135971+0000","last_peered":"2023-10-16T10:08:29.135971+0000","last_clean":"2023-10-16T10:08:29.135971+0000","last_became_active":"2023-10-16T10:08:26.271791+0000","last_became_peered":"2023-10-16T10:08:26.271791+0000","last_unstale":"2023-10-16T10:08:29.135971+0000","last_undegraded":"2023-10-16T10:08:29.135971+0000","last_fullsized":"2023-10-16T10:08:29.135971+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:05.639602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.133993+0000","last_change":"2023-10-16T10:08:26.278089+0000","last_active":"2023-10-16T10:08:29.133993+0000","last_peered":"2023-10-16T10:08:29.133993+0000","last_clean":"2023-10-16T10:08:29.133993+0000","last_became_active":"2023-10-16T10:08:26.274964+0000","last_became_peered":"2023-10-16T10:08:26.274964+0000","last_unstale":"2023-10-16T10:08:29.133993+0000","last_undegraded":"2023-10-16T10:08:29.133993+0000","last_fullsized":"2023-10-16T10:08:29.133993+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:35:11.586105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.258690+0000","last_change":"2023-10-16T10:08:26.253003+0000","last_active":"2023-10-16T10:08:27.258690+0000","last_peered":"2023-10-16T10:08:27.258690+0000","last_clean":"2023-10-16T10:08:27.258690+0000","last_became_active":"2023-10-16T10:08:26.252733+0000","last_became_peered":"2023-10-16T10:08:26.252733+0000","last_unstale":"2023-10-16T10:08:27.258690+0000","last_undegraded":"2023-10-16T10:08:27.258690+0000","last_fullsized":"2023-10-16T10:08:27.258690+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:43:26.330501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.259749+0000","last_change":"2023-10-16T10:08:26.269263+0000","last_active":"2023-10-16T10:08:27.259749+0000","last_peered":"2023-10-16T10:08:27.259749+0000","last_clean":"2023-10-16T10:08:27.259749+0000","last_became_active":"2023-10-16T10:08:26.268401+0000","last_became_peered":"2023-10-16T10:08:26.268401+0000","last_unstale":"2023-10-16T10:08:27.259749+0000","last_undegraded":"2023-10-16T10:08:27.259749+0000","last_fullsized":"2023-10-16T10:08:27.259749+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:15:45.340527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.138700+0000","last_change":"2023-10-16T10:08:26.267596+0000","last_active":"2023-10-16T10:08:29.138700+0000","last_peered":"2023-10-16T10:08:29.138700+0000","last_clean":"2023-10-16T10:08:29.138700+0000","last_became_active":"2023-10-16T10:08:26.267282+0000","last_became_peered":"2023-10-16T10:08:26.267282+0000","last_unstale":"2023-10-16T10:08:29.138700+0000","last_undegraded":"2023-10-16T10:08:29.138700+0000","last_fullsized":"2023-10-16T10:08:29.138700+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:50:03.958131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.137411+0000","last_change":"2023-10-16T10:08:26.274393+0000","last_active":"2023-10-16T10:08:29.137411+0000","last_peered":"2023-10-16T10:08:29.137411+0000","last_clean":"2023-10-16T10:08:29.137411+0000","last_became_active":"2023-10-16T10:08:26.273836+0000","last_became_peered":"2023-10-16T10:08:26.273836+0000","last_unstale":"2023-10-16T10:08:29.137411+0000","last_undegraded":"2023-10-16T10:08:29.137411+0000","last_fullsized":"2023-10-16T10:08:29.137411+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:55.125995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.134338+0000","last_change":"2023-10-16T10:08:26.271401+0000","last_active":"2023-10-16T10:08:29.134338+0000","last_peered":"2023-10-16T10:08:29.134338+0000","last_clean":"2023-10-16T10:08:29.134338+0000","last_became_active":"2023-10-16T10:08:26.271085+0000","last_became_peered":"2023-10-16T10:08:26.271085+0000","last_unstale":"2023-10-16T10:08:29.134338+0000","last_undegraded":"2023-10-16T10:08:29.134338+0000","last_fullsized":"2023-10-16T10:08:29.134338+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:57.991675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.134380+0000","last_change":"2023-10-16T10:08:26.272795+0000","last_active":"2023-10-16T10:08:29.134380+0000","last_peered":"2023-10-16T10:08:29.134380+0000","last_clean":"2023-10-16T10:08:29.134380+0000","last_became_active":"2023-10-16T10:08:26.272594+0000","last_became_peered":"2023-10-16T10:08:26.272594+0000","last_unstale":"2023-10-16T10:08:29.134380+0000","last_undegraded":"2023-10-16T10:08:29.134380+0000","last_fullsized":"2023-10-16T10:08:29.134380+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:32.724082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.139959+0000","last_change":"2023-10-16T10:08:26.261122+0000","last_active":"2023-10-16T10:08:29.139959+0000","last_peered":"2023-10-16T10:08:29.139959+0000","last_clean":"2023-10-16T10:08:29.139959+0000","last_became_active":"2023-10-16T10:08:26.260809+0000","last_became_peered":"2023-10-16T10:08:26.260809+0000","last_unstale":"2023-10-16T10:08:29.139959+0000","last_undegraded":"2023-10-16T10:08:29.139959+0000","last_fullsized":"2023-10-16T10:08:29.139959+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:10:46.381690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.134513+0000","last_change":"2023-10-16T10:08:26.267984+0000","last_active":"2023-10-16T10:08:29.134513+0000","last_peered":"2023-10-16T10:08:29.134513+0000","last_clean":"2023-10-16T10:08:29.134513+0000","last_became_active":"2023-10-16T10:08:26.267645+0000","last_became_peered":"2023-10-16T10:08:26.267645+0000","last_unstale":"2023-10-16T10:08:29.134513+0000","last_undegraded":"2023-10-16T10:08:29.134513+0000","last_fullsized":"2023-10-16T10:08:29.134513+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:46.931837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.136336+0000","last_change":"2023-10-16T10:08:26.276003+0000","last_active":"2023-10-16T10:08:29.136336+0000","last_peered":"2023-10-16T10:08:29.136336+0000","last_clean":"2023-10-16T10:08:29.136336+0000","last_became_active":"2023-10-16T10:08:26.275735+0000","last_became_peered":"2023-10-16T10:08:26.275735+0000","last_unstale":"2023-10-16T10:08:29.136336+0000","last_undegraded":"2023-10-16T10:08:29.136336+0000","last_fullsized":"2023-10-16T10:08:29.136336+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:40.013907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.137957+0000","last_change":"2023-10-16T10:08:26.272373+0000","last_active":"2023-10-16T10:08:29.137957+0000","last_peered":"2023-10-16T10:08:29.137957+0000","last_clean":"2023-10-16T10:08:29.137957+0000","last_became_active":"2023-10-16T10:08:26.272046+0000","last_became_peered":"2023-10-16T10:08:26.272046+0000","last_unstale":"2023-10-16T10:08:29.137957+0000","last_undegraded":"2023-10-16T10:08:29.137957+0000","last_fullsized":"2023-10-16T10:08:29.137957+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:29:51.241530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.259773+0000","last_change":"2023-10-16T10:08:26.255042+0000","last_active":"2023-10-16T10:08:27.259773+0000","last_peered":"2023-10-16T10:08:27.259773+0000","last_clean":"2023-10-16T10:08:27.259773+0000","last_became_active":"2023-10-16T10:08:26.254220+0000","last_became_peered":"2023-10-16T10:08:26.254220+0000","last_unstale":"2023-10-16T10:08:27.259773+0000","last_undegraded":"2023-10-16T10:08:27.259773+0000","last_fullsized":"2023-10-16T10:08:27.259773+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:52.027174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.132971+0000","last_change":"2023-10-16T10:08:26.280057+0000","last_active":"2023-10-16T10:08:29.132971+0000","last_peered":"2023-10-16T10:08:29.132971+0000","last_clean":"2023-10-16T10:08:29.132971+0000","last_became_active":"2023-10-16T10:08:26.276903+0000","last_became_peered":"2023-10-16T10:08:26.276903+0000","last_unstale":"2023-10-16T10:08:29.132971+0000","last_undegraded":"2023-10-16T10:08:29.132971+0000","last_fullsized":"2023-10-16T10:08:29.132971+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:56:00.536576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.139850+0000","last_change":"2023-10-16T10:08:26.261410+0000","last_active":"2023-10-16T10:08:29.139850+0000","last_peered":"2023-10-16T10:08:29.139850+0000","last_clean":"2023-10-16T10:08:29.139850+0000","last_became_active":"2023-10-16T10:08:26.258093+0000","last_became_peered":"2023-10-16T10:08:26.258093+0000","last_unstale":"2023-10-16T10:08:29.139850+0000","last_undegraded":"2023-10-16T10:08:29.139850+0000","last_fullsized":"2023-10-16T10:08:29.139850+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:10:21.928285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.137289+0000","last_change":"2023-10-16T10:08:26.254586+0000","last_active":"2023-10-16T10:08:29.137289+0000","last_peered":"2023-10-16T10:08:29.137289+0000","last_clean":"2023-10-16T10:08:29.137289+0000","last_became_active":"2023-10-16T10:08:26.252590+0000","last_became_peered":"2023-10-16T10:08:26.252590+0000","last_unstale":"2023-10-16T10:08:29.137289+0000","last_undegraded":"2023-10-16T10:08:29.137289+0000","last_fullsized":"2023-10-16T10:08:29.137289+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:17:19.460450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.136345+0000","last_change":"2023-10-16T10:08:26.272835+0000","last_active":"2023-10-16T10:08:29.136345+0000","last_peered":"2023-10-16T10:08:29.136345+0000","last_clean":"2023-10-16T10:08:29.136345+0000","last_became_active":"2023-10-16T10:08:26.272356+0000","last_became_peered":"2023-10-16T10:08:26.272356+0000","last_unstale":"2023-10-16T10:08:29.136345+0000","last_undegraded":"2023-10-16T10:08:29.136345+0000","last_fullsized":"2023-10-16T10:08:29.136345+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:16:59.824826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.140704+0000","last_change":"2023-10-16T10:08:26.275943+0000","last_active":"2023-10-16T10:08:29.140704+0000","last_peered":"2023-10-16T10:08:29.140704+0000","last_clean":"2023-10-16T10:08:29.140704+0000","last_became_active":"2023-10-16T10:08:26.275660+0000","last_became_peered":"2023-10-16T10:08:26.275660+0000","last_unstale":"2023-10-16T10:08:29.140704+0000","last_undegraded":"2023-10-16T10:08:29.140704+0000","last_fullsized":"2023-10-16T10:08:29.140704+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:30.936479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.140626+0000","last_change":"2023-10-16T10:08:26.258709+0000","last_active":"2023-10-16T10:08:29.140626+0000","last_peered":"2023-10-16T10:08:29.140626+0000","last_clean":"2023-10-16T10:08:29.140626+0000","last_became_active":"2023-10-16T10:08:26.257441+0000","last_became_peered":"2023-10-16T10:08:26.257441+0000","last_unstale":"2023-10-16T10:08:29.140626+0000","last_undegraded":"2023-10-16T10:08:29.140626+0000","last_fullsized":"2023-10-16T10:08:29.140626+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:25:33.285479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.264271+0000","last_change":"2023-10-16T10:08:26.264675+0000","last_active":"2023-10-16T10:08:27.264271+0000","last_peered":"2023-10-16T10:08:27.264271+0000","last_clean":"2023-10-16T10:08:27.264271+0000","last_became_active":"2023-10-16T10:08:26.263180+0000","last_became_peered":"2023-10-16T10:08:26.263180+0000","last_unstale":"2023-10-16T10:08:27.264271+0000","last_undegraded":"2023-10-16T10:08:27.264271+0000","last_fullsized":"2023-10-16T10:08:27.264271+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:07:47.070939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.134845+0000","last_change":"2023-10-16T10:08:26.275062+0000","last_active":"2023-10-16T10:08:29.134845+0000","last_peered":"2023-10-16T10:08:29.134845+0000","last_clean":"2023-10-16T10:08:29.134845+0000","last_became_active":"2023-10-16T10:08:26.274796+0000","last_became_peered":"2023-10-16T10:08:26.274796+0000","last_unstale":"2023-10-16T10:08:29.134845+0000","last_undegraded":"2023-10-16T10:08:29.134845+0000","last_fullsized":"2023-10-16T10:08:29.134845+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:01:38.621393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.263686+0000","last_change":"2023-10-16T10:08:26.277305+0000","last_active":"2023-10-16T10:08:27.263686+0000","last_peered":"2023-10-16T10:08:27.263686+0000","last_clean":"2023-10-16T10:08:27.263686+0000","last_became_active":"2023-10-16T10:08:26.276841+0000","last_became_peered":"2023-10-16T10:08:26.276841+0000","last_unstale":"2023-10-16T10:08:27.263686+0000","last_undegraded":"2023-10-16T10:08:27.263686+0000","last_fullsized":"2023-10-16T10:08:27.263686+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:41.342475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.140230+0000","last_change":"2023-10-16T10:08:26.262582+0000","last_active":"2023-10-16T10:08:29.140230+0000","last_peered":"2023-10-16T10:08:29.140230+0000","last_clean":"2023-10-16T10:08:29.140230+0000","last_became_active":"2023-10-16T10:08:26.257735+0000","last_became_peered":"2023-10-16T10:08:26.257735+0000","last_unstale":"2023-10-16T10:08:29.140230+0000","last_undegraded":"2023-10-16T10:08:29.140230+0000","last_fullsized":"2023-10-16T10:08:29.140230+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:44.913421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.134119+0000","last_change":"2023-10-16T10:08:26.273268+0000","last_active":"2023-10-16T10:08:29.134119+0000","last_peered":"2023-10-16T10:08:29.134119+0000","last_clean":"2023-10-16T10:08:29.134119+0000","last_became_active":"2023-10-16T10:08:26.272951+0000","last_became_peered":"2023-10-16T10:08:26.272951+0000","last_unstale":"2023-10-16T10:08:29.134119+0000","last_undegraded":"2023-10-16T10:08:29.134119+0000","last_fullsized":"2023-10-16T10:08:29.134119+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:36.782761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.259802+0000","last_change":"2023-10-16T10:08:26.275814+0000","last_active":"2023-10-16T10:08:27.259802+0000","last_peered":"2023-10-16T10:08:27.259802+0000","last_clean":"2023-10-16T10:08:27.259802+0000","last_became_active":"2023-10-16T10:08:26.274884+0000","last_became_peered":"2023-10-16T10:08:26.274884+0000","last_unstale":"2023-10-16T10:08:27.259802+0000","last_undegraded":"2023-10-16T10:08:27.259802+0000","last_fullsized":"2023-10-16T10:08:27.259802+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:02:37.807028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.264821+0000","last_change":"2023-10-16T10:08:26.260936+0000","last_active":"2023-10-16T10:08:27.264821+0000","last_peered":"2023-10-16T10:08:27.264821+0000","last_clean":"2023-10-16T10:08:27.264821+0000","last_became_active":"2023-10-16T10:08:26.260674+0000","last_became_peered":"2023-10-16T10:08:26.260674+0000","last_unstale":"2023-10-16T10:08:27.264821+0000","last_undegraded":"2023-10-16T10:08:27.264821+0000","last_fullsized":"2023-10-16T10:08:27.264821+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:26:04.374190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.135491+0000","last_change":"2023-10-16T10:08:26.277928+0000","last_active":"2023-10-16T10:08:29.135491+0000","last_peered":"2023-10-16T10:08:29.135491+0000","last_clean":"2023-10-16T10:08:29.135491+0000","last_became_active":"2023-10-16T10:08:26.277632+0000","last_became_peered":"2023-10-16T10:08:26.277632+0000","last_unstale":"2023-10-16T10:08:29.135491+0000","last_undegraded":"2023-10-16T10:08:29.135491+0000","last_fullsized":"2023-10-16T10:08:29.135491+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:50:03.958131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.139876+0000","last_change":"2023-10-16T10:08:26.272140+0000","last_active":"2023-10-16T10:08:29.139876+0000","last_peered":"2023-10-16T10:08:29.139876+0000","last_clean":"2023-10-16T10:08:29.139876+0000","last_became_active":"2023-10-16T10:08:26.271500+0000","last_became_peered":"2023-10-16T10:08:26.271500+0000","last_unstale":"2023-10-16T10:08:29.139876+0000","last_undegraded":"2023-10-16T10:08:29.139876+0000","last_fullsized":"2023-10-16T10:08:29.139876+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:10:21.928285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.133619+0000","last_change":"2023-10-16T10:08:26.274534+0000","last_active":"2023-10-16T10:08:29.133619+0000","last_peered":"2023-10-16T10:08:29.133619+0000","last_clean":"2023-10-16T10:08:29.133619+0000","last_became_active":"2023-10-16T10:08:26.274273+0000","last_became_peered":"2023-10-16T10:08:26.274273+0000","last_unstale":"2023-10-16T10:08:29.133619+0000","last_undegraded":"2023-10-16T10:08:29.133619+0000","last_fullsized":"2023-10-16T10:08:29.133619+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:05.123633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.260280+0000","last_change":"2023-10-16T10:08:26.259118+0000","last_active":"2023-10-16T10:08:27.260280+0000","last_peered":"2023-10-16T10:08:27.260280+0000","last_clean":"2023-10-16T10:08:27.260280+0000","last_became_active":"2023-10-16T10:08:26.258532+0000","last_became_peered":"2023-10-16T10:08:26.258532+0000","last_unstale":"2023-10-16T10:08:27.260280+0000","last_undegraded":"2023-10-16T10:08:27.260280+0000","last_fullsized":"2023-10-16T10:08:27.260280+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:16:29.251757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.259974+0000","last_change":"2023-10-16T10:08:26.257766+0000","last_active":"2023-10-16T10:08:27.259974+0000","last_peered":"2023-10-16T10:08:27.259974+0000","last_clean":"2023-10-16T10:08:27.259974+0000","last_became_active":"2023-10-16T10:08:26.257515+0000","last_became_peered":"2023-10-16T10:08:26.257515+0000","last_unstale":"2023-10-16T10:08:27.259974+0000","last_undegraded":"2023-10-16T10:08:27.259974+0000","last_fullsized":"2023-10-16T10:08:27.259974+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:44.117413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"137'4","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.276280+0000","last_change":"2023-10-16T10:08:25.262040+0000","last_active":"2023-10-16T10:08:26.276280+0000","last_peered":"2023-10-16T10:08:26.276280+0000","last_clean":"2023-10-16T10:08:26.276280+0000","last_became_active":"2023-10-16T10:08:25.261768+0000","last_became_peered":"2023-10-16T10:08:25.261768+0000","last_unstale":"2023-10-16T10:08:26.276280+0000","last_undegraded":"2023-10-16T10:08:26.276280+0000","last_fullsized":"2023-10-16T10:08:26.276280+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:10.098275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.138668+0000","last_change":"2023-10-16T10:08:26.279348+0000","last_active":"2023-10-16T10:08:29.138668+0000","last_peered":"2023-10-16T10:08:29.138668+0000","last_clean":"2023-10-16T10:08:29.138668+0000","last_became_active":"2023-10-16T10:08:26.277014+0000","last_became_peered":"2023-10-16T10:08:26.277014+0000","last_unstale":"2023-10-16T10:08:29.138668+0000","last_undegraded":"2023-10-16T10:08:29.138668+0000","last_fullsized":"2023-10-16T10:08:29.138668+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:13.574636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.273364+0000","last_change":"2023-10-16T10:08:25.263087+0000","last_active":"2023-10-16T10:08:26.273364+0000","last_peered":"2023-10-16T10:08:26.273364+0000","last_clean":"2023-10-16T10:08:26.273364+0000","last_became_active":"2023-10-16T10:08:25.262767+0000","last_became_peered":"2023-10-16T10:08:25.262767+0000","last_unstale":"2023-10-16T10:08:26.273364+0000","last_undegraded":"2023-10-16T10:08:26.273364+0000","last_fullsized":"2023-10-16T10:08:26.273364+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:05.842577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.260666+0000","last_change":"2023-10-16T10:08:26.255964+0000","last_active":"2023-10-16T10:08:27.260666+0000","last_peered":"2023-10-16T10:08:27.260666+0000","last_clean":"2023-10-16T10:08:27.260666+0000","last_became_active":"2023-10-16T10:08:26.255691+0000","last_became_peered":"2023-10-16T10:08:26.255691+0000","last_unstale":"2023-10-16T10:08:27.260666+0000","last_undegraded":"2023-10-16T10:08:27.260666+0000","last_fullsized":"2023-10-16T10:08:27.260666+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:28:43.178170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.272108+0000","last_change":"2023-10-16T10:08:25.246351+0000","last_active":"2023-10-16T10:08:26.272108+0000","last_peered":"2023-10-16T10:08:26.272108+0000","last_clean":"2023-10-16T10:08:26.272108+0000","last_became_active":"2023-10-16T10:08:25.246053+0000","last_became_peered":"2023-10-16T10:08:25.246053+0000","last_unstale":"2023-10-16T10:08:26.272108+0000","last_undegraded":"2023-10-16T10:08:26.272108+0000","last_fullsized":"2023-10-16T10:08:26.272108+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:55.962023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.137087+0000","last_change":"2023-10-16T10:08:26.273727+0000","last_active":"2023-10-16T10:08:29.137087+0000","last_peered":"2023-10-16T10:08:29.137087+0000","last_clean":"2023-10-16T10:08:29.137087+0000","last_became_active":"2023-10-16T10:08:26.272652+0000","last_became_peered":"2023-10-16T10:08:26.272652+0000","last_unstale":"2023-10-16T10:08:29.137087+0000","last_undegraded":"2023-10-16T10:08:29.137087+0000","last_fullsized":"2023-10-16T10:08:29.137087+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:09:00.941907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.272401+0000","last_change":"2023-10-16T10:08:25.247100+0000","last_active":"2023-10-16T10:08:26.272401+0000","last_peered":"2023-10-16T10:08:26.272401+0000","last_clean":"2023-10-16T10:08:26.272401+0000","last_became_active":"2023-10-16T10:08:25.246782+0000","last_became_peered":"2023-10-16T10:08:25.246782+0000","last_unstale":"2023-10-16T10:08:26.272401+0000","last_undegraded":"2023-10-16T10:08:26.272401+0000","last_fullsized":"2023-10-16T10:08:26.272401+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:22.332858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.133245+0000","last_change":"2023-10-16T10:08:26.273760+0000","last_active":"2023-10-16T10:08:29.133245+0000","last_peered":"2023-10-16T10:08:29.133245+0000","last_clean":"2023-10-16T10:08:29.133245+0000","last_became_active":"2023-10-16T10:08:26.273508+0000","last_became_peered":"2023-10-16T10:08:26.273508+0000","last_unstale":"2023-10-16T10:08:29.133245+0000","last_undegraded":"2023-10-16T10:08:29.133245+0000","last_fullsized":"2023-10-16T10:08:29.133245+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:18:01.500362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.270333+0000","last_change":"2023-10-16T10:08:25.257406+0000","last_active":"2023-10-16T10:08:26.270333+0000","last_peered":"2023-10-16T10:08:26.270333+0000","last_clean":"2023-10-16T10:08:26.270333+0000","last_became_active":"2023-10-16T10:08:25.257103+0000","last_became_peered":"2023-10-16T10:08:25.257103+0000","last_unstale":"2023-10-16T10:08:26.270333+0000","last_undegraded":"2023-10-16T10:08:26.270333+0000","last_fullsized":"2023-10-16T10:08:26.270333+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:25.003965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.135856+0000","last_change":"2023-10-16T10:08:26.266279+0000","last_active":"2023-10-16T10:08:29.135856+0000","last_peered":"2023-10-16T10:08:29.135856+0000","last_clean":"2023-10-16T10:08:29.135856+0000","last_became_active":"2023-10-16T10:08:26.265351+0000","last_became_peered":"2023-10-16T10:08:26.265351+0000","last_unstale":"2023-10-16T10:08:29.135856+0000","last_undegraded":"2023-10-16T10:08:29.135856+0000","last_fullsized":"2023-10-16T10:08:29.135856+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:09.584840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.264847+0000","last_change":"2023-10-16T10:08:25.257495+0000","last_active":"2023-10-16T10:08:26.264847+0000","last_peered":"2023-10-16T10:08:26.264847+0000","last_clean":"2023-10-16T10:08:26.264847+0000","last_became_active":"2023-10-16T10:08:25.257208+0000","last_became_peered":"2023-10-16T10:08:25.257208+0000","last_unstale":"2023-10-16T10:08:26.264847+0000","last_undegraded":"2023-10-16T10:08:26.264847+0000","last_fullsized":"2023-10-16T10:08:26.264847+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:17.790922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.133251+0000","last_change":"2023-10-16T10:08:26.261989+0000","last_active":"2023-10-16T10:08:29.133251+0000","last_peered":"2023-10-16T10:08:29.133251+0000","last_clean":"2023-10-16T10:08:29.133251+0000","last_became_active":"2023-10-16T10:08:26.260844+0000","last_became_peered":"2023-10-16T10:08:26.260844+0000","last_unstale":"2023-10-16T10:08:29.133251+0000","last_undegraded":"2023-10-16T10:08:29.133251+0000","last_fullsized":"2023-10-16T10:08:29.133251+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:20:13.580053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.239692+0000","last_change":"2023-10-16T10:08:25.254308+0000","last_active":"2023-10-16T10:08:26.239692+0000","last_peered":"2023-10-16T10:08:26.239692+0000","last_clean":"2023-10-16T10:08:26.239692+0000","last_became_active":"2023-10-16T10:08:25.254010+0000","last_became_peered":"2023-10-16T10:08:25.254010+0000","last_unstale":"2023-10-16T10:08:26.239692+0000","last_undegraded":"2023-10-16T10:08:26.239692+0000","last_fullsized":"2023-10-16T10:08:26.239692+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:45.913441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.139278+0000","last_change":"2023-10-16T10:08:26.269223+0000","last_active":"2023-10-16T10:08:29.139278+0000","last_peered":"2023-10-16T10:08:29.139278+0000","last_clean":"2023-10-16T10:08:29.139278+0000","last_became_active":"2023-10-16T10:08:26.268395+0000","last_became_peered":"2023-10-16T10:08:26.268395+0000","last_unstale":"2023-10-16T10:08:29.139278+0000","last_undegraded":"2023-10-16T10:08:29.139278+0000","last_fullsized":"2023-10-16T10:08:29.139278+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:09.584840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.253138+0000","last_change":"2023-10-16T10:08:25.251903+0000","last_active":"2023-10-16T10:08:26.253138+0000","last_peered":"2023-10-16T10:08:26.253138+0000","last_clean":"2023-10-16T10:08:26.253138+0000","last_became_active":"2023-10-16T10:08:25.251481+0000","last_became_peered":"2023-10-16T10:08:25.251481+0000","last_unstale":"2023-10-16T10:08:26.253138+0000","last_undegraded":"2023-10-16T10:08:26.253138+0000","last_fullsized":"2023-10-16T10:08:26.253138+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:15:44.322131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.138392+0000","last_change":"2023-10-16T10:08:26.272437+0000","last_active":"2023-10-16T10:08:29.138392+0000","last_peered":"2023-10-16T10:08:29.138392+0000","last_clean":"2023-10-16T10:08:29.138392+0000","last_became_active":"2023-10-16T10:08:26.272119+0000","last_became_peered":"2023-10-16T10:08:26.272119+0000","last_unstale":"2023-10-16T10:08:29.138392+0000","last_undegraded":"2023-10-16T10:08:29.138392+0000","last_fullsized":"2023-10-16T10:08:29.138392+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:01:38.621393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.237051+0000","last_change":"2023-10-16T10:08:25.247973+0000","last_active":"2023-10-16T10:08:26.237051+0000","last_peered":"2023-10-16T10:08:26.237051+0000","last_clean":"2023-10-16T10:08:26.237051+0000","last_became_active":"2023-10-16T10:08:25.247532+0000","last_became_peered":"2023-10-16T10:08:25.247532+0000","last_unstale":"2023-10-16T10:08:26.237051+0000","last_undegraded":"2023-10-16T10:08:26.237051+0000","last_fullsized":"2023-10-16T10:08:26.237051+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:12.556240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.139030+0000","last_change":"2023-10-16T10:08:26.278409+0000","last_active":"2023-10-16T10:08:29.139030+0000","last_peered":"2023-10-16T10:08:29.139030+0000","last_clean":"2023-10-16T10:08:29.139030+0000","last_became_active":"2023-10-16T10:08:26.276646+0000","last_became_peered":"2023-10-16T10:08:26.276646+0000","last_unstale":"2023-10-16T10:08:29.139030+0000","last_undegraded":"2023-10-16T10:08:29.139030+0000","last_fullsized":"2023-10-16T10:08:29.139030+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:24:36.708728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.247393+0000","last_change":"2023-10-16T10:08:25.258954+0000","last_active":"2023-10-16T10:08:26.247393+0000","last_peered":"2023-10-16T10:08:26.247393+0000","last_clean":"2023-10-16T10:08:26.247393+0000","last_became_active":"2023-10-16T10:08:25.258629+0000","last_became_peered":"2023-10-16T10:08:25.258629+0000","last_unstale":"2023-10-16T10:08:26.247393+0000","last_undegraded":"2023-10-16T10:08:26.247393+0000","last_fullsized":"2023-10-16T10:08:26.247393+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:14.511192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.265114+0000","last_change":"2023-10-16T10:08:26.253237+0000","last_active":"2023-10-16T10:08:27.265114+0000","last_peered":"2023-10-16T10:08:27.265114+0000","last_clean":"2023-10-16T10:08:27.265114+0000","last_became_active":"2023-10-16T10:08:26.252936+0000","last_became_peered":"2023-10-16T10:08:26.252936+0000","last_unstale":"2023-10-16T10:08:27.265114+0000","last_undegraded":"2023-10-16T10:08:27.265114+0000","last_fullsized":"2023-10-16T10:08:27.265114+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:37:19.234903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"137'3","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.274251+0000","last_change":"2023-10-16T10:08:25.255885+0000","last_active":"2023-10-16T10:08:26.274251+0000","last_peered":"2023-10-16T10:08:26.274251+0000","last_clean":"2023-10-16T10:08:26.274251+0000","last_became_active":"2023-10-16T10:08:25.255645+0000","last_became_peered":"2023-10-16T10:08:25.255645+0000","last_unstale":"2023-10-16T10:08:26.274251+0000","last_undegraded":"2023-10-16T10:08:26.274251+0000","last_fullsized":"2023-10-16T10:08:26.274251+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:32:55.153046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.138703+0000","last_change":"2023-10-16T10:08:26.276221+0000","last_active":"2023-10-16T10:08:29.138703+0000","last_peered":"2023-10-16T10:08:29.138703+0000","last_clean":"2023-10-16T10:08:29.138703+0000","last_became_active":"2023-10-16T10:08:26.275740+0000","last_became_peered":"2023-10-16T10:08:26.275740+0000","last_unstale":"2023-10-16T10:08:29.138703+0000","last_undegraded":"2023-10-16T10:08:29.138703+0000","last_fullsized":"2023-10-16T10:08:29.138703+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:58:45.263010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"137'6","reported_seq":18,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.282278+0000","last_change":"2023-10-16T10:08:25.247740+0000","last_active":"2023-10-16T10:08:26.282278+0000","last_peered":"2023-10-16T10:08:26.282278+0000","last_clean":"2023-10-16T10:08:26.282278+0000","last_became_active":"2023-10-16T10:08:25.246511+0000","last_became_peered":"2023-10-16T10:08:25.246511+0000","last_unstale":"2023-10-16T10:08:26.282278+0000","last_undegraded":"2023-10-16T10:08:26.282278+0000","last_fullsized":"2023-10-16T10:08:26.282278+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:16:28.233361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.264545+0000","last_change":"2023-10-16T10:08:26.271952+0000","last_active":"2023-10-16T10:08:27.264545+0000","last_peered":"2023-10-16T10:08:27.264545+0000","last_clean":"2023-10-16T10:08:27.264545+0000","last_became_active":"2023-10-16T10:08:26.271656+0000","last_became_peered":"2023-10-16T10:08:26.271656+0000","last_unstale":"2023-10-16T10:08:27.264545+0000","last_undegraded":"2023-10-16T10:08:27.264545+0000","last_fullsized":"2023-10-16T10:08:27.264545+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:41.352237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.276104+0000","last_change":"2023-10-16T10:08:25.260285+0000","last_active":"2023-10-16T10:08:26.276104+0000","last_peered":"2023-10-16T10:08:26.276104+0000","last_clean":"2023-10-16T10:08:26.276104+0000","last_became_active":"2023-10-16T10:08:25.260075+0000","last_became_peered":"2023-10-16T10:08:25.260075+0000","last_unstale":"2023-10-16T10:08:26.276104+0000","last_undegraded":"2023-10-16T10:08:26.276104+0000","last_fullsized":"2023-10-16T10:08:26.276104+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:24:35.690332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.264461+0000","last_change":"2023-10-16T10:08:26.271063+0000","last_active":"2023-10-16T10:08:27.264461+0000","last_peered":"2023-10-16T10:08:27.264461+0000","last_clean":"2023-10-16T10:08:27.264461+0000","last_became_active":"2023-10-16T10:08:26.270788+0000","last_became_peered":"2023-10-16T10:08:26.270788+0000","last_unstale":"2023-10-16T10:08:27.264461+0000","last_undegraded":"2023-10-16T10:08:27.264461+0000","last_fullsized":"2023-10-16T10:08:27.264461+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:10:22.379381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.240068+0000","last_change":"2023-10-16T10:08:25.256935+0000","last_active":"2023-10-16T10:08:26.240068+0000","last_peered":"2023-10-16T10:08:26.240068+0000","last_clean":"2023-10-16T10:08:26.240068+0000","last_became_active":"2023-10-16T10:08:25.256646+0000","last_became_peered":"2023-10-16T10:08:25.256646+0000","last_unstale":"2023-10-16T10:08:26.240068+0000","last_undegraded":"2023-10-16T10:08:26.240068+0000","last_fullsized":"2023-10-16T10:08:26.240068+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:05.949599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.263508+0000","last_change":"2023-10-16T10:08:26.251738+0000","last_active":"2023-10-16T10:08:27.263508+0000","last_peered":"2023-10-16T10:08:27.263508+0000","last_clean":"2023-10-16T10:08:27.263508+0000","last_became_active":"2023-10-16T10:08:26.251464+0000","last_became_peered":"2023-10-16T10:08:26.251464+0000","last_unstale":"2023-10-16T10:08:27.263508+0000","last_undegraded":"2023-10-16T10:08:27.263508+0000","last_fullsized":"2023-10-16T10:08:27.263508+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:30:38.819179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"137'8","reported_seq":20,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.273613+0000","last_change":"2023-10-16T10:08:25.250724+0000","last_active":"2023-10-16T10:08:26.273613+0000","last_peered":"2023-10-16T10:08:26.273613+0000","last_clean":"2023-10-16T10:08:26.273613+0000","last_became_active":"2023-10-16T10:08:25.250460+0000","last_became_peered":"2023-10-16T10:08:25.250460+0000","last_unstale":"2023-10-16T10:08:26.273613+0000","last_undegraded":"2023-10-16T10:08:26.273613+0000","last_fullsized":"2023-10-16T10:08:26.273613+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:12:20.130900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.260933+0000","last_change":"2023-10-16T10:08:26.267823+0000","last_active":"2023-10-16T10:08:27.260933+0000","last_peered":"2023-10-16T10:08:27.260933+0000","last_clean":"2023-10-16T10:08:27.260933+0000","last_became_active":"2023-10-16T10:08:26.267411+0000","last_became_peered":"2023-10-16T10:08:26.267411+0000","last_unstale":"2023-10-16T10:08:27.260933+0000","last_undegraded":"2023-10-16T10:08:27.260933+0000","last_fullsized":"2023-10-16T10:08:27.260933+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:10:39.682619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.274033+0000","last_change":"2023-10-16T10:08:25.259547+0000","last_active":"2023-10-16T10:08:26.274033+0000","last_peered":"2023-10-16T10:08:26.274033+0000","last_clean":"2023-10-16T10:08:26.274033+0000","last_became_active":"2023-10-16T10:08:25.259330+0000","last_became_peered":"2023-10-16T10:08:25.259330+0000","last_unstale":"2023-10-16T10:08:26.274033+0000","last_undegraded":"2023-10-16T10:08:26.274033+0000","last_fullsized":"2023-10-16T10:08:26.274033+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:20:25.308279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.260297+0000","last_change":"2023-10-16T10:08:26.275460+0000","last_active":"2023-10-16T10:08:27.260297+0000","last_peered":"2023-10-16T10:08:27.260297+0000","last_clean":"2023-10-16T10:08:27.260297+0000","last_became_active":"2023-10-16T10:08:26.275179+0000","last_became_peered":"2023-10-16T10:08:26.275179+0000","last_unstale":"2023-10-16T10:08:27.260297+0000","last_undegraded":"2023-10-16T10:08:27.260297+0000","last_fullsized":"2023-10-16T10:08:27.260297+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:32:56.171441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.269678+0000","last_change":"2023-10-16T10:08:25.262287+0000","last_active":"2023-10-16T10:08:26.269678+0000","last_peered":"2023-10-16T10:08:26.269678+0000","last_clean":"2023-10-16T10:08:26.269678+0000","last_became_active":"2023-10-16T10:08:25.261948+0000","last_became_peered":"2023-10-16T10:08:25.261948+0000","last_unstale":"2023-10-16T10:08:26.269678+0000","last_undegraded":"2023-10-16T10:08:26.269678+0000","last_fullsized":"2023-10-16T10:08:26.269678+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:05:45.988873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.137304+0000","last_change":"2023-10-16T10:08:26.280257+0000","last_active":"2023-10-16T10:08:29.137304+0000","last_peered":"2023-10-16T10:08:29.137304+0000","last_clean":"2023-10-16T10:08:29.137304+0000","last_became_active":"2023-10-16T10:08:26.279895+0000","last_became_peered":"2023-10-16T10:08:26.279895+0000","last_unstale":"2023-10-16T10:08:29.137304+0000","last_undegraded":"2023-10-16T10:08:29.137304+0000","last_fullsized":"2023-10-16T10:08:29.137304+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:37.768708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.273454+0000","last_change":"2023-10-16T10:08:25.256427+0000","last_active":"2023-10-16T10:08:26.273454+0000","last_peered":"2023-10-16T10:08:26.273454+0000","last_clean":"2023-10-16T10:08:26.273454+0000","last_became_active":"2023-10-16T10:08:25.254558+0000","last_became_peered":"2023-10-16T10:08:25.254558+0000","last_unstale":"2023-10-16T10:08:26.273454+0000","last_undegraded":"2023-10-16T10:08:26.273454+0000","last_fullsized":"2023-10-16T10:08:26.273454+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:23:22.195630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.137462+0000","last_change":"2023-10-16T10:08:26.273363+0000","last_active":"2023-10-16T10:08:29.137462+0000","last_peered":"2023-10-16T10:08:29.137462+0000","last_clean":"2023-10-16T10:08:29.137462+0000","last_became_active":"2023-10-16T10:08:26.273072+0000","last_became_peered":"2023-10-16T10:08:26.273072+0000","last_unstale":"2023-10-16T10:08:29.137462+0000","last_undegraded":"2023-10-16T10:08:29.137462+0000","last_fullsized":"2023-10-16T10:08:29.137462+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:55.952398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.255907+0000","last_change":"2023-10-16T10:08:25.260069+0000","last_active":"2023-10-16T10:08:26.255907+0000","last_peered":"2023-10-16T10:08:26.255907+0000","last_clean":"2023-10-16T10:08:26.255907+0000","last_became_active":"2023-10-16T10:08:25.259758+0000","last_became_peered":"2023-10-16T10:08:25.259758+0000","last_unstale":"2023-10-16T10:08:26.255907+0000","last_undegraded":"2023-10-16T10:08:26.255907+0000","last_fullsized":"2023-10-16T10:08:26.255907+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:39.485099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.138059+0000","last_change":"2023-10-16T10:08:26.272485+0000","last_active":"2023-10-16T10:08:29.138059+0000","last_peered":"2023-10-16T10:08:29.138059+0000","last_clean":"2023-10-16T10:08:29.138059+0000","last_became_active":"2023-10-16T10:08:26.272194+0000","last_became_peered":"2023-10-16T10:08:26.272194+0000","last_unstale":"2023-10-16T10:08:29.138059+0000","last_undegraded":"2023-10-16T10:08:29.138059+0000","last_fullsized":"2023-10-16T10:08:29.138059+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:51:27.673790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.274408+0000","last_change":"2023-10-16T10:08:25.242736+0000","last_active":"2023-10-16T10:08:26.274408+0000","last_peered":"2023-10-16T10:08:26.274408+0000","last_clean":"2023-10-16T10:08:26.274408+0000","last_became_active":"2023-10-16T10:08:25.242031+0000","last_became_peered":"2023-10-16T10:08:25.242031+0000","last_unstale":"2023-10-16T10:08:26.274408+0000","last_undegraded":"2023-10-16T10:08:26.274408+0000","last_fullsized":"2023-10-16T10:08:26.274408+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:05:45.988873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.133208+0000","last_change":"2023-10-16T10:08:26.278628+0000","last_active":"2023-10-16T10:08:29.133208+0000","last_peered":"2023-10-16T10:08:29.133208+0000","last_clean":"2023-10-16T10:08:29.133208+0000","last_became_active":"2023-10-16T10:08:26.278368+0000","last_became_peered":"2023-10-16T10:08:26.278368+0000","last_unstale":"2023-10-16T10:08:29.133208+0000","last_undegraded":"2023-10-16T10:08:29.133208+0000","last_fullsized":"2023-10-16T10:08:29.133208+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:25:54.300572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.236479+0000","last_change":"2023-10-16T10:08:25.249107+0000","last_active":"2023-10-16T10:08:26.236479+0000","last_peered":"2023-10-16T10:08:26.236479+0000","last_clean":"2023-10-16T10:08:26.236479+0000","last_became_active":"2023-10-16T10:08:25.248477+0000","last_became_peered":"2023-10-16T10:08:25.248477+0000","last_unstale":"2023-10-16T10:08:26.236479+0000","last_undegraded":"2023-10-16T10:08:26.236479+0000","last_fullsized":"2023-10-16T10:08:26.236479+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:26:13.129741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.137671+0000","last_change":"2023-10-16T10:08:26.274562+0000","last_active":"2023-10-16T10:08:29.137671+0000","last_peered":"2023-10-16T10:08:29.137671+0000","last_clean":"2023-10-16T10:08:29.137671+0000","last_became_active":"2023-10-16T10:08:26.274105+0000","last_became_peered":"2023-10-16T10:08:26.274105+0000","last_unstale":"2023-10-16T10:08:29.137671+0000","last_undegraded":"2023-10-16T10:08:29.137671+0000","last_fullsized":"2023-10-16T10:08:29.137671+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:38:24.715623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.249085+0000","last_change":"2023-10-16T10:08:25.244509+0000","last_active":"2023-10-16T10:08:26.249085+0000","last_peered":"2023-10-16T10:08:26.249085+0000","last_clean":"2023-10-16T10:08:26.249085+0000","last_became_active":"2023-10-16T10:08:25.244196+0000","last_became_peered":"2023-10-16T10:08:25.244196+0000","last_unstale":"2023-10-16T10:08:26.249085+0000","last_undegraded":"2023-10-16T10:08:26.249085+0000","last_fullsized":"2023-10-16T10:08:26.249085+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:10:38.664224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.263744+0000","last_change":"2023-10-16T10:08:26.261335+0000","last_active":"2023-10-16T10:08:27.263744+0000","last_peered":"2023-10-16T10:08:27.263744+0000","last_clean":"2023-10-16T10:08:27.263744+0000","last_became_active":"2023-10-16T10:08:26.261049+0000","last_became_peered":"2023-10-16T10:08:26.261049+0000","last_unstale":"2023-10-16T10:08:27.263744+0000","last_undegraded":"2023-10-16T10:08:27.263744+0000","last_fullsized":"2023-10-16T10:08:27.263744+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:06.860973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.253288+0000","last_change":"2023-10-16T10:08:25.258208+0000","last_active":"2023-10-16T10:08:26.253288+0000","last_peered":"2023-10-16T10:08:26.253288+0000","last_clean":"2023-10-16T10:08:26.253288+0000","last_became_active":"2023-10-16T10:08:25.257929+0000","last_became_peered":"2023-10-16T10:08:25.257929+0000","last_unstale":"2023-10-16T10:08:26.253288+0000","last_undegraded":"2023-10-16T10:08:26.253288+0000","last_fullsized":"2023-10-16T10:08:26.253288+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:51.008778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.137900+0000","last_change":"2023-10-16T10:08:26.275698+0000","last_active":"2023-10-16T10:08:29.137900+0000","last_peered":"2023-10-16T10:08:29.137900+0000","last_clean":"2023-10-16T10:08:29.137900+0000","last_became_active":"2023-10-16T10:08:26.273577+0000","last_became_peered":"2023-10-16T10:08:26.273577+0000","last_unstale":"2023-10-16T10:08:29.137900+0000","last_undegraded":"2023-10-16T10:08:29.137900+0000","last_fullsized":"2023-10-16T10:08:29.137900+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:56.980419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.274954+0000","last_change":"2023-10-16T10:08:25.253364+0000","last_active":"2023-10-16T10:08:26.274954+0000","last_peered":"2023-10-16T10:08:26.274954+0000","last_clean":"2023-10-16T10:08:26.274954+0000","last_became_active":"2023-10-16T10:08:25.253048+0000","last_became_peered":"2023-10-16T10:08:25.253048+0000","last_unstale":"2023-10-16T10:08:26.274954+0000","last_undegraded":"2023-10-16T10:08:26.274954+0000","last_fullsized":"2023-10-16T10:08:26.274954+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:25:18.095377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.134624+0000","last_change":"2023-10-16T10:08:26.271057+0000","last_active":"2023-10-16T10:08:29.134624+0000","last_peered":"2023-10-16T10:08:29.134624+0000","last_clean":"2023-10-16T10:08:29.134624+0000","last_became_active":"2023-10-16T10:08:26.270780+0000","last_became_peered":"2023-10-16T10:08:26.270780+0000","last_unstale":"2023-10-16T10:08:29.134624+0000","last_undegraded":"2023-10-16T10:08:29.134624+0000","last_fullsized":"2023-10-16T10:08:29.134624+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:44:22.750010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.243382+0000","last_change":"2023-10-16T10:08:25.261749+0000","last_active":"2023-10-16T10:08:26.243382+0000","last_peered":"2023-10-16T10:08:26.243382+0000","last_clean":"2023-10-16T10:08:26.243382+0000","last_became_active":"2023-10-16T10:08:25.261106+0000","last_became_peered":"2023-10-16T10:08:25.261106+0000","last_unstale":"2023-10-16T10:08:26.243382+0000","last_undegraded":"2023-10-16T10:08:26.243382+0000","last_fullsized":"2023-10-16T10:08:26.243382+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:22.332858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.134381+0000","last_change":"2023-10-16T10:08:26.279489+0000","last_active":"2023-10-16T10:08:29.134381+0000","last_peered":"2023-10-16T10:08:29.134381+0000","last_clean":"2023-10-16T10:08:29.134381+0000","last_became_active":"2023-10-16T10:08:26.279149+0000","last_became_peered":"2023-10-16T10:08:26.279149+0000","last_unstale":"2023-10-16T10:08:29.134381+0000","last_undegraded":"2023-10-16T10:08:29.134381+0000","last_fullsized":"2023-10-16T10:08:29.134381+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:43:04.688389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.258084+0000","last_change":"2023-10-16T10:08:25.258390+0000","last_active":"2023-10-16T10:08:26.258084+0000","last_peered":"2023-10-16T10:08:26.258084+0000","last_clean":"2023-10-16T10:08:26.258084+0000","last_became_active":"2023-10-16T10:08:25.258098+0000","last_became_peered":"2023-10-16T10:08:25.258098+0000","last_unstale":"2023-10-16T10:08:26.258084+0000","last_undegraded":"2023-10-16T10:08:26.258084+0000","last_fullsized":"2023-10-16T10:08:26.258084+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:28:42.159774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.136806+0000","last_change":"2023-10-16T10:08:26.257765+0000","last_active":"2023-10-16T10:08:29.136806+0000","last_peered":"2023-10-16T10:08:29.136806+0000","last_clean":"2023-10-16T10:08:29.136806+0000","last_became_active":"2023-10-16T10:08:26.257255+0000","last_became_peered":"2023-10-16T10:08:26.257255+0000","last_unstale":"2023-10-16T10:08:29.136806+0000","last_undegraded":"2023-10-16T10:08:29.136806+0000","last_fullsized":"2023-10-16T10:08:29.136806+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:22:15.233943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.247655+0000","last_change":"2023-10-16T10:08:25.248506+0000","last_active":"2023-10-16T10:08:26.247655+0000","last_peered":"2023-10-16T10:08:26.247655+0000","last_clean":"2023-10-16T10:08:26.247655+0000","last_became_active":"2023-10-16T10:08:25.247947+0000","last_became_peered":"2023-10-16T10:08:25.247947+0000","last_unstale":"2023-10-16T10:08:26.247655+0000","last_undegraded":"2023-10-16T10:08:26.247655+0000","last_fullsized":"2023-10-16T10:08:26.247655+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:08.376496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.135694+0000","last_change":"2023-10-16T10:08:26.270421+0000","last_active":"2023-10-16T10:08:29.135694+0000","last_peered":"2023-10-16T10:08:29.135694+0000","last_clean":"2023-10-16T10:08:29.135694+0000","last_became_active":"2023-10-16T10:08:26.269775+0000","last_became_peered":"2023-10-16T10:08:26.269775+0000","last_unstale":"2023-10-16T10:08:29.135694+0000","last_undegraded":"2023-10-16T10:08:29.135694+0000","last_fullsized":"2023-10-16T10:08:29.135694+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:58:45.263010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.255594+0000","last_change":"2023-10-16T10:08:25.260303+0000","last_active":"2023-10-16T10:08:26.255594+0000","last_peered":"2023-10-16T10:08:26.255594+0000","last_clean":"2023-10-16T10:08:26.255594+0000","last_became_active":"2023-10-16T10:08:25.259967+0000","last_became_peered":"2023-10-16T10:08:25.259967+0000","last_unstale":"2023-10-16T10:08:26.255594+0000","last_undegraded":"2023-10-16T10:08:26.255594+0000","last_fullsized":"2023-10-16T10:08:26.255594+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:08:09.423160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.140312+0000","last_change":"2023-10-16T10:08:26.251617+0000","last_active":"2023-10-16T10:08:29.140312+0000","last_peered":"2023-10-16T10:08:29.140312+0000","last_clean":"2023-10-16T10:08:29.140312+0000","last_became_active":"2023-10-16T10:08:26.250783+0000","last_became_peered":"2023-10-16T10:08:26.250783+0000","last_unstale":"2023-10-16T10:08:29.140312+0000","last_undegraded":"2023-10-16T10:08:29.140312+0000","last_fullsized":"2023-10-16T10:08:29.140312+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:51.020862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.241985+0000","last_change":"2023-10-16T10:08:25.256421+0000","last_active":"2023-10-16T10:08:26.241985+0000","last_peered":"2023-10-16T10:08:26.241985+0000","last_clean":"2023-10-16T10:08:26.241985+0000","last_became_active":"2023-10-16T10:08:25.256156+0000","last_became_peered":"2023-10-16T10:08:25.256156+0000","last_unstale":"2023-10-16T10:08:26.241985+0000","last_undegraded":"2023-10-16T10:08:26.241985+0000","last_fullsized":"2023-10-16T10:08:26.241985+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:10.098275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.138038+0000","last_change":"2023-10-16T10:08:26.280924+0000","last_active":"2023-10-16T10:08:29.138038+0000","last_peered":"2023-10-16T10:08:29.138038+0000","last_clean":"2023-10-16T10:08:29.138038+0000","last_became_active":"2023-10-16T10:08:26.280413+0000","last_became_peered":"2023-10-16T10:08:26.280413+0000","last_unstale":"2023-10-16T10:08:29.138038+0000","last_undegraded":"2023-10-16T10:08:29.138038+0000","last_fullsized":"2023-10-16T10:08:29.138038+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:10.803585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.236587+0000","last_change":"2023-10-16T10:08:25.246369+0000","last_active":"2023-10-16T10:08:26.236587+0000","last_peered":"2023-10-16T10:08:26.236587+0000","last_clean":"2023-10-16T10:08:26.236587+0000","last_became_active":"2023-10-16T10:08:25.246101+0000","last_became_peered":"2023-10-16T10:08:25.246101+0000","last_unstale":"2023-10-16T10:08:26.236587+0000","last_undegraded":"2023-10-16T10:08:26.236587+0000","last_fullsized":"2023-10-16T10:08:26.236587+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:17:18.442054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.133984+0000","last_change":"2023-10-16T10:08:26.259001+0000","last_active":"2023-10-16T10:08:29.133984+0000","last_peered":"2023-10-16T10:08:29.133984+0000","last_clean":"2023-10-16T10:08:29.133984+0000","last_became_active":"2023-10-16T10:08:26.258741+0000","last_became_peered":"2023-10-16T10:08:26.258741+0000","last_unstale":"2023-10-16T10:08:29.133984+0000","last_undegraded":"2023-10-16T10:08:29.133984+0000","last_fullsized":"2023-10-16T10:08:29.133984+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:40:43.260342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.254672+0000","last_change":"2023-10-16T10:08:25.258263+0000","last_active":"2023-10-16T10:08:26.254672+0000","last_peered":"2023-10-16T10:08:26.254672+0000","last_clean":"2023-10-16T10:08:26.254672+0000","last_became_active":"2023-10-16T10:08:25.257979+0000","last_became_peered":"2023-10-16T10:08:25.257979+0000","last_unstale":"2023-10-16T10:08:26.254672+0000","last_undegraded":"2023-10-16T10:08:26.254672+0000","last_fullsized":"2023-10-16T10:08:26.254672+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:43.099017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.263742+0000","last_change":"2023-10-16T10:08:26.277114+0000","last_active":"2023-10-16T10:08:27.263742+0000","last_peered":"2023-10-16T10:08:27.263742+0000","last_clean":"2023-10-16T10:08:27.263742+0000","last_became_active":"2023-10-16T10:08:26.276485+0000","last_became_peered":"2023-10-16T10:08:26.276485+0000","last_unstale":"2023-10-16T10:08:27.263742+0000","last_undegraded":"2023-10-16T10:08:27.263742+0000","last_fullsized":"2023-10-16T10:08:27.263742+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:47:46.896643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.262322+0000","last_change":"2023-10-16T10:08:25.261482+0000","last_active":"2023-10-16T10:08:26.262322+0000","last_peered":"2023-10-16T10:08:26.262322+0000","last_clean":"2023-10-16T10:08:26.262322+0000","last_became_active":"2023-10-16T10:08:25.259231+0000","last_became_peered":"2023-10-16T10:08:25.259231+0000","last_unstale":"2023-10-16T10:08:26.262322+0000","last_undegraded":"2023-10-16T10:08:26.262322+0000","last_fullsized":"2023-10-16T10:08:26.262322+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:17:44.030296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"127'2646","reported_seq":3118,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.267923+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T10:08:26.267923+0000","last_peered":"2023-10-16T10:08:26.267923+0000","last_clean":"2023-10-16T10:08:26.267923+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T10:08:26.267923+0000","last_undegraded":"2023-10-16T10:08:26.267923+0000","last_fullsized":"2023-10-16T10:08:26.267923+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2646,"log_dups_size":0,"ondisk_log_size":2646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1120,"num_read_kb":2202,"num_write":361,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1120,"num_read_kb":2202,"num_write":361,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2646,"ondisk_log_size":2646,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575079,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33428,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338412,"statfs":{"total":96636764160,"available":96602533888,"internally_reserved":0,"allocated":2301952,"data_stored":1404224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1699999999999999,"15min":1.171},"min":{"1min":0.81200000000000006,"5min":0.65000000000000002,"15min":0.54400000000000004},"max":{"1min":1.6160000000000001,"5min":1.7310000000000001,"15min":1.925},"last":0.432},{"interface":"front","average":{"1min":1.27,"5min":1.159,"15min":1.1899999999999999},"min":{"1min":0.90400000000000003,"5min":0.58099999999999996,"15min":0.33600000000000002},"max":{"1min":1.702,"5min":1.7370000000000001,"15min":2.016},"last":0.55200000000000005}]},{"osd":1,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1759999999999999,"15min":1.2210000000000001},"min":{"1min":0.72899999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.48,"5min":1.788,"15min":1.923},"last":0.502},{"interface":"front","average":{"1min":1.214,"5min":1.161,"15min":1.2090000000000001},"min":{"1min":0.746,"5min":0.67000000000000004,"15min":0.60999999999999999},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.40699999999999997}]},{"osd":2,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.21,"15min":1.24},"min":{"1min":0.63900000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8109999999999999,"5min":1.837,"15min":1.8740000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2629999999999999,"15min":1.288},"min":{"1min":0.89400000000000002,"5min":0.65900000000000003,"15min":0.64500000000000002},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8620000000000001},"last":0.86699999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3069999999999999,"15min":1.339},"min":{"1min":1.054,"5min":0.747,"15min":0.67700000000000005},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3280000000000001,"15min":1.3720000000000001},"min":{"1min":0.97699999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6799999999999999,"5min":1.8540000000000001,"15min":2.004},"last":0.48199999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3859999999999999,"15min":1.367},"min":{"1min":0.86499999999999999,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6970000000000001,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.3140000000000001,"15min":1.3129999999999999},"min":{"1min":1.0009999999999999,"5min":0.63200000000000001,"15min":0.502},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.45200000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.5069999999999999,"15min":1.52},"min":{"1min":1.1879999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.9330000000000001,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.627},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4199999999999999,"15min":1.4279999999999999},"min":{"1min":1.1910000000000001,"5min":0.72199999999999998,"15min":0.45400000000000001},"max":{"1min":1.6699999999999999,"5min":1.9630000000000001,"15min":2.2170000000000001},"last":0.95199999999999996}]},{"osd":6,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.419,"15min":1.4039999999999999},"min":{"1min":0.97599999999999998,"5min":0.82899999999999996,"15min":0.54100000000000004},"max":{"1min":2.0899999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.631,"15min":1.617},"min":{"1min":1.212,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.9279999999999999,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.80900000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575079,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41176,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330664,"statfs":{"total":96636764160,"available":96594599936,"internally_reserved":0,"allocated":1716224,"data_stored":819638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.131,"15min":1.143},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.619,"5min":1.736,"15min":1.7689999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.216,"15min":1.228},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5329999999999999,"5min":1.7010000000000001,"15min":1.847},"last":0.76100000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1459999999999999,"15min":1.1659999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":1.171,"5min":1.278,"15min":1.286},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6539999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.1879999999999999,"15min":1.1990000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45100000000000001},"max":{"1min":1.4710000000000001,"5min":1.851,"15min":2.036},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.248,"15min":1.296},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.58999999999999997},"max":{"1min":1.6859999999999999,"5min":1.925,"15min":1.925},"last":0.73999999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1579999999999999,"15min":1.202},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":2.8239999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.087,"5min":1.1759999999999999,"15min":1.218},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5049999999999999,"5min":1.6719999999999999,"15min":1.865},"last":0.69099999999999995}]},{"osd":4,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.28,"15min":1.2490000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.46000000000000002},"max":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":2.1629999999999998},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.423,"15min":1.4239999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.9730000000000001,"5min":2,"15min":3.6880000000000002},"last":0.51800000000000002}]},{"osd":5,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3080000000000001,"15min":1.3160000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.38900000000000001},"max":{"1min":1.647,"5min":2.0790000000000002,"15min":2.5649999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.45,"5min":1.5,"15min":1.502},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.495},"max":{"1min":1.8440000000000001,"5min":1.996,"15min":2.3849999999999998},"last":0.79800000000000004}]},{"osd":7,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.478,"15min":1.47},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":3.5739999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.47,"5min":1.512,"15min":1.488},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.4249999999999998},"last":0.81899999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575079,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37016,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334824,"statfs":{"total":96636764160,"available":96598859776,"internally_reserved":0,"allocated":1708032,"data_stored":813844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.179,"15min":1.204},"min":{"1min":0.65600000000000003,"5min":0.57799999999999996,"15min":0.496},"max":{"1min":1.6279999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.236,"15min":1.2849999999999999},"min":{"1min":0.72599999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.4219999999999999,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.2370000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.119,"15min":1.1499999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.647,"5min":1.7310000000000001,"15min":1.7729999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.139,"15min":1.1439999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.478,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.2909999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1339999999999999,"15min":1.155},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5700000000000001,"5min":1.738,"15min":1.77},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.21,"15min":1.232},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.645,"5min":1.6739999999999999,"15min":1.8109999999999999},"last":1.0740000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2609999999999999,"15min":1.27},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.76000000000000001},"max":{"1min":1.8260000000000001,"5min":1.9530000000000001,"15min":1.97},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3149999999999999,"15min":1.347},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6100000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.3280000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.278,"15min":1.2689999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.73499999999999999},"max":{"1min":1.6850000000000001,"5min":1.984,"15min":1.984},"last":1.133},{"interface":"front","average":{"1min":1.444,"5min":1.51,"15min":1.502},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.7470000000000001,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.5289999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3360000000000001,"15min":1.3540000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.722,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.274},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.381,"15min":1.3799999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.613,"5min":1.986,"15min":1.986},"last":1.4990000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.464,"15min":1.47},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.9219999999999999,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":1.351},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.5489999999999999,"15min":1.5620000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.788,"5min":2.234,"15min":2.234},"last":1.383}]}]},{"osd":4,"up_from":13,"seq":55834575080,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46736,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325104,"statfs":{"total":96636764160,"available":96588906496,"internally_reserved":0,"allocated":1699840,"data_stored":813240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.079,"15min":1.141},"min":{"1min":0.64300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.5349999999999999,"5min":1.7609999999999999,"15min":1.827},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3220000000000001,"15min":1.2689999999999999},"min":{"1min":0.89600000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.7070000000000001,"5min":11.974,"15min":11.974},"last":0.76600000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3260000000000001,"15min":1.2989999999999999},"min":{"1min":0.63900000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.7969999999999999,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.204,"5min":1.2010000000000001,"15min":1.226},"min":{"1min":0.70699999999999996,"5min":0.63600000000000001,"15min":0.624},"max":{"1min":1.5800000000000001,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":0.98099999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.246,"15min":1.248},"min":{"1min":0.78200000000000003,"5min":0.61799999999999999,"15min":0.59399999999999997},"max":{"1min":1.712,"5min":1.897,"15min":1.897},"last":1.512},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3540000000000001,"15min":1.383},"min":{"1min":0.81499999999999995,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6160000000000001,"5min":3.673,"15min":5.0549999999999997},"last":1.0189999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3200000000000001,"15min":1.3},"min":{"1min":0.94399999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.625,"5min":6.8079999999999998,"15min":6.8079999999999998},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.327,"5min":1.3620000000000001,"15min":1.3779999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.67700000000000005},"max":{"1min":1.768,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.6180000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.496,"15min":1.4350000000000001},"min":{"1min":0.77400000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9990000000000001,"5min":8.702,"15min":8.702},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.383,"15min":1.3720000000000001},"min":{"1min":0.93799999999999994,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.331,"5min":2.331,"15min":3.0430000000000001},"last":1.167}]},{"osd":6,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4850000000000001,"15min":1.417},"min":{"1min":0.95399999999999996,"5min":0.84699999999999998,"15min":0.79000000000000004},"max":{"1min":1.6990000000000001,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":1.72},{"interface":"front","average":{"1min":1.496,"5min":1.585,"15min":1.556},"min":{"1min":0.88900000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.891,"5min":4.625,"15min":6.4089999999999998},"last":1.2070000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.5189999999999999,"15min":1.492},"min":{"1min":1.167,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.927,"5min":2.4060000000000001,"15min":2.9399999999999999},"last":1.085},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.579,"15min":1.5700000000000001},"min":{"1min":1.2589999999999999,"5min":1.071,"15min":0.96799999999999997},"max":{"1min":1.9139999999999999,"5min":2.2690000000000001,"15min":2.6120000000000001},"last":1.8260000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575079,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37036,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334804,"statfs":{"total":96636764160,"available":96598839296,"internally_reserved":0,"allocated":1728512,"data_stored":821250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.80100000000000005,"15min":0.84599999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.24399999999999999},"max":{"1min":1.405,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.91400000000000003,"15min":0.99399999999999999},"min":{"1min":0.35999999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.4239999999999999,"5min":1.5349999999999999,"15min":1.708},"last":0.58199999999999996}]},{"osd":1,"last update":"Mon Oct 16 10:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.83699999999999997,"15min":0.92200000000000004},"min":{"1min":0.40400000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1559999999999999,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.93799999999999994,"15min":1.028},"min":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.145,"5min":3.363,"15min":3.363},"last":0.67600000000000005}]},{"osd":2,"last update":"Mon Oct 16 10:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.96599999999999997,"15min":1.0469999999999999},"min":{"1min":0.54900000000000004,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.357,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.878},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.046,"15min":1.1539999999999999},"min":{"1min":0.51900000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.46,"5min":4.1230000000000002,"15min":4.1230000000000002},"last":0.70099999999999996}]},{"osd":4,"last update":"Mon Oct 16 10:07:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.196,"15min":1.2629999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.706,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":1.046},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.26,"15min":1.3169999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.54700000000000004},"max":{"1min":1.665,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.86299999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:07:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2410000000000001,"15min":1.306},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.6919999999999999,"5min":4.1829999999999998,"15min":4.1829999999999998},"last":1.147},{"interface":"front","average":{"1min":1.224,"5min":1.242,"15min":1.323},"min":{"1min":0.69499999999999995,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7030000000000001,"5min":2.0129999999999999,"15min":2.3140000000000001},"last":1.016}]},{"osd":6,"last update":"Mon Oct 16 10:07:35 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.353,"15min":1.4099999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.712,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.345,"15min":1.4239999999999999},"min":{"1min":0.73899999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.8089999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.429}]},{"osd":7,"last update":"Mon Oct 16 10:07:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3660000000000001,"15min":1.427},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8140000000000001,"5min":3.649,"15min":3.649},"last":1.264},{"interface":"front","average":{"1min":1.446,"5min":1.47,"15min":1.5429999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.907,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":1.2949999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575078,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59224,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312616,"statfs":{"total":96636764160,"available":96576118784,"internally_reserved":0,"allocated":1712128,"data_stored":819568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.89300000000000002,"15min":0.877},"min":{"1min":0.49299999999999999,"5min":0.35299999999999998,"15min":0.26500000000000001},"max":{"1min":1.236,"5min":5.1829999999999998,"15min":5.1829999999999998},"last":1.1930000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.94499999999999995,"15min":1.0109999999999999},"min":{"1min":0.51800000000000002,"5min":0.35999999999999999,"15min":0.34999999999999998},"max":{"1min":1.3520000000000001,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.82199999999999995}]},{"osd":1,"last update":"Mon Oct 16 10:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.0289999999999999,"15min":1.0389999999999999},"min":{"1min":0.5,"5min":0.438,"15min":0.438},"max":{"1min":1.5029999999999999,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":1.407},{"interface":"front","average":{"1min":1.05,"5min":1.002,"15min":1.0920000000000001},"min":{"1min":0.58299999999999996,"5min":0.38300000000000001,"15min":0.35499999999999998},"max":{"1min":1.395,"5min":1.4770000000000001,"15min":1.792},"last":1.272}]},{"osd":3,"last update":"Mon Oct 16 10:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.93700000000000006,"15min":0.998},"min":{"1min":0.54700000000000004,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.506,"5min":1.776,"15min":1.776},"last":1.131},{"interface":"front","average":{"1min":1.095,"5min":1.1539999999999999,"15min":1.1870000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41899999999999998},"max":{"1min":1.6899999999999999,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":1.6910000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:07:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.323,"15min":1.337},"min":{"1min":0.89900000000000002,"5min":0.79900000000000004,"15min":0.40300000000000002},"max":{"1min":1.732,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":2.1669999999999998},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.284,"15min":1.298},"min":{"1min":0.873,"5min":0.71899999999999997,"15min":0.46000000000000002},"max":{"1min":1.5720000000000001,"5min":3.262,"15min":3.262},"last":1.4650000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:07:38 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.3939999999999999,"15min":1.3779999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.59399999999999997},"max":{"1min":2.1589999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.9039999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.335,"15min":1.391},"min":{"1min":0.879,"5min":0.72699999999999998,"15min":0.60999999999999999},"max":{"1min":1.8069999999999999,"5min":1.885,"15min":1.956},"last":1.823}]},{"osd":6,"last update":"Mon Oct 16 10:07:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.3,"15min":1.3320000000000001},"min":{"1min":0.80700000000000005,"5min":0.78600000000000003,"15min":0.54400000000000004},"max":{"1min":1.7809999999999999,"5min":3.169,"15min":3.169},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4219999999999999,"15min":1.4379999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":2.0840000000000001,"5min":3.8340000000000001,"15min":3.8340000000000001},"last":1.7809999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:07:38 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.423,"15min":1.464},"min":{"1min":0.94099999999999995,"5min":0.81200000000000006,"15min":0.67000000000000004},"max":{"1min":1.8799999999999999,"5min":2.0739999999999998,"15min":2.262},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.395,"15min":1.425},"min":{"1min":0.91000000000000003,"5min":0.84199999999999997,"15min":0.63400000000000001},"max":{"1min":1.7709999999999999,"5min":3.3780000000000001,"15min":3.3780000000000001},"last":1.948}]}]},{"osd":1,"up_from":13,"seq":55834575078,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55072,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316768,"statfs":{"total":96636764160,"available":96580370432,"internally_reserved":0,"allocated":1720320,"data_stored":819692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.77500000000000002,"15min":0.84399999999999997},"min":{"1min":0.34399999999999997,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":1.0309999999999999,"5min":1.4690000000000001,"15min":1.631},"last":0.502},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.92500000000000004,"15min":0.97899999999999998},"min":{"1min":0.38200000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.5509999999999999,"5min":1.6200000000000001,"15min":1.798},"last":0.51200000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.90100000000000002,"15min":0.96299999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2529999999999999,"5min":1.5069999999999999,"15min":1.673},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.95199999999999996,"5min":1.0640000000000001,"15min":1.1930000000000001},"min":{"1min":0.56000000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3919999999999999,"5min":1.609,"15min":1.7669999999999999},"last":0.42299999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.0009999999999999,"15min":1.097},"min":{"1min":0.51800000000000002,"5min":0.42999999999999999,"15min":0.42599999999999999},"max":{"1min":1.4550000000000001,"5min":1.798,"15min":1.798},"last":0.46800000000000003},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0229999999999999,"15min":1.0920000000000001},"min":{"1min":0.47599999999999998,"5min":0.44,"15min":0.40699999999999997},"max":{"1min":1.5489999999999999,"5min":1.6879999999999999,"15min":1.8280000000000001},"last":0.54600000000000004}]},{"osd":4,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1599999999999999,"15min":1.212},"min":{"1min":0.69099999999999995,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5860000000000001,"5min":1.7450000000000001,"15min":1.885},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.27,"5min":1.2689999999999999,"15min":1.341},"min":{"1min":0.64700000000000002,"5min":0.63200000000000001,"15min":0.60999999999999999},"max":{"1min":1.605,"5min":1.8680000000000001,"15min":2.1320000000000001},"last":0.89000000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2589999999999999,"15min":1.3180000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.232,"5min":1.238,"15min":1.302},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.637,"5min":1.7290000000000001,"15min":2.0470000000000002},"last":0.94599999999999995}]},{"osd":6,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.3169999999999999,"15min":1.381},"min":{"1min":0.72099999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.3089999999999999,"15min":1.369},"min":{"1min":0.68600000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.762,"5min":1.8779999999999999,"15min":2.1680000000000001},"last":1.105}]},{"osd":7,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.331,"15min":1.4330000000000001},"min":{"1min":0.73799999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.6950000000000001,"5min":2.0739999999999998,"15min":2.1699999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4319999999999999,"15min":1.484},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.0389999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575079,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47332,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324508,"statfs":{"total":96636764160,"available":96588296192,"internally_reserved":0,"allocated":2314240,"data_stored":1405332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.80800000000000005,"15min":0.84799999999999998},"min":{"1min":0.35599999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.532,"5min":1.7589999999999999,"15min":2.766},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.92900000000000005,"15min":1.0129999999999999},"min":{"1min":0.51600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.228,"5min":1.581,"15min":3.96},"last":0.60099999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.97299999999999998,"15min":1.028},"min":{"1min":0.54200000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.278,"5min":2.0449999999999999,"15min":2.9260000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.98999999999999999,"15min":1.0660000000000001},"min":{"1min":0.47899999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.52,"5min":1.7789999999999999,"15min":4.4290000000000003},"last":0.75800000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.91100000000000003,"15min":1.012},"min":{"1min":0.5,"5min":0.41999999999999998,"15min":0.38},"max":{"1min":1.585,"5min":1.5920000000000001,"15min":4.7380000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.103,"15min":1.1859999999999999},"min":{"1min":0.51200000000000001,"5min":0.46800000000000003,"15min":0.44},"max":{"1min":1.3959999999999999,"5min":2.1859999999999999,"15min":5.1500000000000004},"last":0.63900000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2609999999999999,"15min":1.3200000000000001},"min":{"1min":0.83199999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5760000000000001,"5min":1.859,"15min":4.1970000000000001},"last":0.995},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1910000000000001,"15min":1.276},"min":{"1min":0.83899999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.3999999999999999,"5min":1.74,"15min":4.9630000000000001},"last":1.175}]},{"osd":5,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.202,"15min":1.296},"min":{"1min":0.877,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6799999999999999,"5min":1.6890000000000001,"15min":4.8600000000000003},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3300000000000001,"15min":1.385},"min":{"1min":0.76500000000000001,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6299999999999999,"5min":2.1269999999999998,"15min":4.6779999999999999},"last":1.202}]},{"osd":6,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.407,"15min":1.4330000000000001},"min":{"1min":0.93000000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.7390000000000001,"5min":2.1030000000000002,"15min":3.8250000000000002},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.254,"5min":1.339,"15min":1.4139999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.782,"5min":2.02,"15min":4.0979999999999999},"last":1.113}]},{"osd":7,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.4179999999999999,"15min":1.4710000000000001},"min":{"1min":0.79900000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.8600000000000001,"5min":2.0750000000000002,"15min":4.3369999999999997},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.3089999999999999,"15min":1.417},"min":{"1min":0.86299999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4650000000000001,"5min":1.9670000000000001,"15min":5.0570000000000004},"last":0.92800000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:08:30.712 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:08:30.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.964+0000 7f6b131ce640 1 Processor -- start 2023-10-16T10:08:30.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.964+0000 7f6b131ce640 1 -- start start 2023-10-16T10:08:30.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.965+0000 7f6b131ce640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b0c15ae80 0x7f6b0c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.965+0000 7f6b131ce640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b0c15ed60 con 0x7f6b0c0ce390 2023-10-16T10:08:30.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.965+0000 7f6b131ce640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b0c0707e0 con 0x7f6b0c15aaa0 2023-10-16T10:08:30.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.965+0000 7f6b131ce640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b0c06b810 con 0x7f6b0c15ae80 2023-10-16T10:08:30.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.966+0000 7f6b1176b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6b0c0ce390 0x7f6b0c070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55434/0 (socket says 172.21.15.84:55434) 2023-10-16T10:08:30.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.966+0000 7f6b1176b640 1 -- 172.21.15.84:0/3246225193 learned_addr learned my addr 172.21.15.84:0/3246225193 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:30.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.966+0000 7f6b0b7fe640 1 -- 172.21.15.84:0/3246225193 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208910590 0 0) 0x7f6b0c15ed60 con 0x7f6b0c0ce390 2023-10-16T10:08:30.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.966+0000 7f6b0b7fe640 1 -- 172.21.15.84:0/3246225193 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6af8003520 con 0x7f6b0c0ce390 2023-10-16T10:08:30.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.966+0000 7f6b0bfff640 1 --2- 172.21.15.84:0/3246225193 >> v2:172.21.15.157:3300/0 conn(0x7f6b0c15ae80 0x7f6b0c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:30.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.966+0000 7f6b0b7fe640 1 -- 172.21.15.84:0/3246225193 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3842361453 0 0) 0x7f6af8003520 con 0x7f6b0c0ce390 2023-10-16T10:08:30.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.966+0000 7f6b0b7fe640 1 -- 172.21.15.84:0/3246225193 >> v2:172.21.15.157:3300/0 conn(0x7f6b0c15ae80 msgr2=0x7f6b0c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:30.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.966+0000 7f6b0b7fe640 1 --2- 172.21.15.84:0/3246225193 >> v2:172.21.15.157:3300/0 conn(0x7f6b0c15ae80 0x7f6b0c15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.966+0000 7f6b0b7fe640 1 -- 172.21.15.84:0/3246225193 >> v1:172.21.15.157:6789/0 conn(0x7f6b0c15aaa0 legacy=0x7f6b0c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:30.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.966+0000 7f6b0b7fe640 1 -- 172.21.15.84:0/3246225193 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c15ef90 con 0x7f6b0c0ce390 2023-10-16T10:08:30.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.966+0000 7f6b0b7fe640 1 -- 172.21.15.84:0/3246225193 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f6b04001670 con 0x7f6b0c0ce390 2023-10-16T10:08:30.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.966+0000 7f6b0bfff640 1 --2- 172.21.15.84:0/3246225193 >> v2:172.21.15.157:3300/0 conn(0x7f6b0c15ae80 0x7f6b0c15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:08:30.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.967+0000 7f6b0b7fe640 1 -- 172.21.15.84:0/3246225193 >> v1:172.21.15.84:6789/0 conn(0x7f6b0c0ce390 legacy=0x7f6b0c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:30.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.967+0000 7f6b0b7fe640 1 --2- 172.21.15.84:0/3246225193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6af80039c0 0x7f6af8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.967+0000 7f6b0b7fe640 1 --2- 172.21.15.84:0/3246225193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6af80047c0 0x7f6af8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.967+0000 7f6b0b7fe640 1 --2- 172.21.15.84:0/3246225193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af80051d0 0x7f6af800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.967+0000 7f6b0b7fe640 1 -- 172.21.15.84:0/3246225193 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c06b810 con 0x7f6af80051d0 2023-10-16T10:08:30.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.967+0000 7f6b0b7fe640 1 -- 172.21.15.84:0/3246225193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af800c120 con 0x7f6af80047c0 2023-10-16T10:08:30.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.967+0000 7f6b0b7fe640 1 -- 172.21.15.84:0/3246225193 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af800c2e0 con 0x7f6af80039c0 2023-10-16T10:08:30.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.967+0000 7f6b1176b640 1 --2- 172.21.15.84:0/3246225193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af80051d0 0x7f6af800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:30.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.967+0000 7f6b10f6a640 1 --2- 172.21.15.84:0/3246225193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6af80039c0 0x7f6af8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:30.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.967+0000 7f6b0bfff640 1 --2- 172.21.15.84:0/3246225193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6af80047c0 0x7f6af8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:30.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.967+0000 7f6b1176b640 1 -- 172.21.15.84:0/3246225193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6af80039c0 msgr2=0x7f6af8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:30.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.967+0000 7f6b1176b640 1 --2- 172.21.15.84:0/3246225193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6af80039c0 0x7f6af8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.967+0000 7f6b1176b640 1 -- 172.21.15.84:0/3246225193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6af80047c0 msgr2=0x7f6af8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:30.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.967+0000 7f6b1176b640 1 --2- 172.21.15.84:0/3246225193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6af80047c0 0x7f6af8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.967+0000 7f6b1176b640 1 -- 172.21.15.84:0/3246225193 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af8003520 con 0x7f6af80051d0 2023-10-16T10:08:30.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.968+0000 7f6b1176b640 1 -- 172.21.15.84:0/3246225193 reap_dead start 2023-10-16T10:08:30.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.968+0000 7f6b1176b640 1 --2- 172.21.15.84:0/3246225193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af80051d0 0x7f6af800b980 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400c0e0 tx=0x7f6af80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3870a73af624453f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:30.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.969+0000 7f6b0b7fe640 1 -- 172.21.15.84:0/3246225193 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04005ad0 con 0x7f6af80051d0 2023-10-16T10:08:30.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.969+0000 7f6b0b7fe640 1 -- 172.21.15.84:0/3246225193 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b04005c50 con 0x7f6af80051d0 2023-10-16T10:08:30.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.969+0000 7f6b0b7fe640 1 -- 172.21.15.84:0/3246225193 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0402c7d0 con 0x7f6af80051d0 2023-10-16T10:08:30.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.969+0000 7f6b131ce640 1 -- 172.21.15.84:0/3246225193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af80051d0 msgr2=0x7f6af800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:30.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.969+0000 7f6b131ce640 1 --2- 172.21.15.84:0/3246225193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af80051d0 0x7f6af800b980 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400c0e0 tx=0x7f6af80057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.969+0000 7f6b131ce640 1 -- 172.21.15.84:0/3246225193 shutdown_connections 2023-10-16T10:08:30.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.969+0000 7f6b131ce640 1 --2- 172.21.15.84:0/3246225193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af80051d0 0x7f6af800b980 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.969+0000 7f6b131ce640 1 -- 172.21.15.84:0/3246225193 >> 172.21.15.84:0/3246225193 conn(0x7f6b0c0cb410 msgr2=0x7f6b0c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:30.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.969+0000 7f6b131ce640 1 -- 172.21.15.84:0/3246225193 shutdown_connections 2023-10-16T10:08:30.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.969+0000 7f6b131ce640 1 -- 172.21.15.84:0/3246225193 wait complete. 2023-10-16T10:08:30.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.971+0000 7f6b131ce640 1 Processor -- start 2023-10-16T10:08:30.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.971+0000 7f6b131ce640 1 -- start start 2023-10-16T10:08:30.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.972+0000 7f6b131ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af80051d0 0x7f6b0c158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.972+0000 7f6b131ce640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b0c152790 0x7f6b0c152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.972+0000 7f6b131ce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b0c154f50 0x7f6b0c1553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.972+0000 7f6b131ce640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c155c80 con 0x7f6af80051d0 2023-10-16T10:08:30.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.972+0000 7f6b131ce640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c153150 con 0x7f6b0c154f50 2023-10-16T10:08:30.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.972+0000 7f6b131ce640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c153d10 con 0x7f6b0c152790 2023-10-16T10:08:30.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.973+0000 7f6b10f6a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af80051d0 0x7f6b0c158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:30.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.973+0000 7f6b10f6a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af80051d0 0x7f6b0c158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52058/0 (socket says 172.21.15.84:52058) 2023-10-16T10:08:30.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.973+0000 7f6b10f6a640 1 -- 172.21.15.84:0/1304774474 learned_addr learned my addr 172.21.15.84:0/1304774474 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:30.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.973+0000 7f6b0bfff640 1 --2- 172.21.15.84:0/1304774474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b0c152790 0x7f6b0c152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:30.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.973+0000 7f6b10f6a640 1 -- 172.21.15.84:0/1304774474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b0c152790 msgr2=0x7f6b0c152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:30.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.973+0000 7f6b10f6a640 1 --2- 172.21.15.84:0/1304774474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b0c152790 0x7f6b0c152c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.973+0000 7f6b10f6a640 1 -- 172.21.15.84:0/1304774474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b0c154f50 msgr2=0x7f6b0c1553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:30.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.973+0000 7f6b10f6a640 1 --2- 172.21.15.84:0/1304774474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b0c154f50 0x7f6b0c1553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:30.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.973+0000 7f6b10f6a640 1 -- 172.21.15.84:0/1304774474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c153fb0 con 0x7f6af80051d0 2023-10-16T10:08:30.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.973+0000 7f6b10f6a640 1 --2- 172.21.15.84:0/1304774474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af80051d0 0x7f6b0c158be0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00e7f0 tx=0x7f6afc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:30.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.974+0000 7f6b097fa640 1 -- 172.21.15.84:0/1304774474 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc003b80 con 0x7f6af80051d0 2023-10-16T10:08:30.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.974+0000 7f6b097fa640 1 -- 172.21.15.84:0/1304774474 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc003ce0 con 0x7f6af80051d0 2023-10-16T10:08:30.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.974+0000 7f6b131ce640 1 -- 172.21.15.84:0/1304774474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0c154290 con 0x7f6af80051d0 2023-10-16T10:08:30.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.974+0000 7f6b097fa640 1 -- 172.21.15.84:0/1304774474 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc0052d0 con 0x7f6af80051d0 2023-10-16T10:08:30.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.974+0000 7f6b131ce640 1 -- 172.21.15.84:0/1304774474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0c155910 con 0x7f6af80051d0 2023-10-16T10:08:30.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.975+0000 7f6b131ce640 1 -- 172.21.15.84:0/1304774474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0c04e160 con 0x7f6af80051d0 2023-10-16T10:08:30.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.976+0000 7f6b097fa640 1 -- 172.21.15.84:0/1304774474 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6afc005430 con 0x7f6af80051d0 2023-10-16T10:08:30.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.977+0000 7f6b097fa640 1 --2- 172.21.15.84:0/1304774474 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f6aec06afb0 0x7f6aec06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:30.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.977+0000 7f6b097fa640 1 -- 172.21.15.84:0/1304774474 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6afc0897d0 con 0x7f6af80051d0 2023-10-16T10:08:30.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.977+0000 7f6b0bfff640 1 --2- 172.21.15.84:0/1304774474 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f6aec06afb0 0x7f6aec06d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:30.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.978+0000 7f6b0bfff640 1 --2- 172.21.15.84:0/1304774474 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f6aec06afb0 0x7f6aec06d470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6af4006fd0 tx=0x7f6af4008040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:30.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:30.981+0000 7f6b097fa640 1 -- 172.21.15.84:0/1304774474 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6afc054750 con 0x7f6af80051d0 2023-10-16T10:08:31.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.169+0000 7f6b131ce640 1 -- 172.21.15.84:0/1304774474 --> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6b0c15f550 con 0x7f6aec06afb0 2023-10-16T10:08:31.171 INFO:tasks.ceph.mds.a.smithi084.stderr:2023-10-16T10:08:31.169+0000 7fc32dd55640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:08:31.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.181+0000 7f6b097fa640 1 -- 172.21.15.84:0/1304774474 <== mgr.8322 v2:172.21.15.84:6832/2784642648 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287680 (secure 0 0 0) 0x7f6af400c040 con 0x7f6aec06afb0 2023-10-16T10:08:31.186 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:31.188 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T10:08:31.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.188+0000 7f6b131ce640 1 -- 172.21.15.84:0/1304774474 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f6aec06afb0 msgr2=0x7f6aec06d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:31.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.188+0000 7f6b131ce640 1 --2- 172.21.15.84:0/1304774474 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f6aec06afb0 0x7f6aec06d470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6af4006fd0 tx=0x7f6af4008040 comp rx=0 tx=0).stop 2023-10-16T10:08:31.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.188+0000 7f6b131ce640 1 -- 172.21.15.84:0/1304774474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af80051d0 msgr2=0x7f6b0c158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:31.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.188+0000 7f6b131ce640 1 --2- 172.21.15.84:0/1304774474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af80051d0 0x7f6b0c158be0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00e7f0 tx=0x7f6afc00ecb0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.188+0000 7f6b131ce640 1 -- 172.21.15.84:0/1304774474 shutdown_connections 2023-10-16T10:08:31.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.188+0000 7f6b131ce640 1 --2- 172.21.15.84:0/1304774474 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f6aec06afb0 0x7f6aec06d470 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.188+0000 7f6b131ce640 1 --2- 172.21.15.84:0/1304774474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b0c154f50 0x7f6b0c1553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.188+0000 7f6b131ce640 1 --2- 172.21.15.84:0/1304774474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b0c152790 0x7f6b0c152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.188+0000 7f6b131ce640 1 --2- 172.21.15.84:0/1304774474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af80051d0 0x7f6b0c158be0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.188+0000 7f6b131ce640 1 -- 172.21.15.84:0/1304774474 >> 172.21.15.84:0/1304774474 conn(0x7f6b0c0cb410 msgr2=0x7f6b0c0b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:31.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.189+0000 7f6b131ce640 1 -- 172.21.15.84:0/1304774474 shutdown_connections 2023-10-16T10:08:31.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.189+0000 7f6b131ce640 1 -- 172.21.15.84:0/1304774474 wait complete. 2023-10-16T10:08:31.209 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-16T10:08:30.450231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1120,"num_read_kb":2202,"num_write":405,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2684,"ondisk_log_size":2684,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":357020,"kb_used_data":14552,"kb_used_omap":12,"kb_used_meta":342323,"kb_avail":754617700,"statfs":{"total":773094113280,"available":772728524800,"internally_reserved":0,"allocated":14901248,"data_stored":7716788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":350539321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001788"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.134208+0000","last_change":"2023-10-16T10:08:26.264876+0000","last_active":"2023-10-16T10:08:29.134208+0000","last_peered":"2023-10-16T10:08:29.134208+0000","last_clean":"2023-10-16T10:08:29.134208+0000","last_became_active":"2023-10-16T10:08:26.264607+0000","last_became_peered":"2023-10-16T10:08:26.264607+0000","last_unstale":"2023-10-16T10:08:29.134208+0000","last_undegraded":"2023-10-16T10:08:29.134208+0000","last_fullsized":"2023-10-16T10:08:29.134208+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:51:06.545129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.140515+0000","last_change":"2023-10-16T10:08:26.264953+0000","last_active":"2023-10-16T10:08:29.140515+0000","last_peered":"2023-10-16T10:08:29.140515+0000","last_clean":"2023-10-16T10:08:29.140515+0000","last_became_active":"2023-10-16T10:08:26.264517+0000","last_became_peered":"2023-10-16T10:08:26.264517+0000","last_unstale":"2023-10-16T10:08:29.140515+0000","last_undegraded":"2023-10-16T10:08:29.140515+0000","last_fullsized":"2023-10-16T10:08:29.140515+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:20:22.674056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.135971+0000","last_change":"2023-10-16T10:08:26.272140+0000","last_active":"2023-10-16T10:08:29.135971+0000","last_peered":"2023-10-16T10:08:29.135971+0000","last_clean":"2023-10-16T10:08:29.135971+0000","last_became_active":"2023-10-16T10:08:26.271791+0000","last_became_peered":"2023-10-16T10:08:26.271791+0000","last_unstale":"2023-10-16T10:08:29.135971+0000","last_undegraded":"2023-10-16T10:08:29.135971+0000","last_fullsized":"2023-10-16T10:08:29.135971+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:05.639602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.133993+0000","last_change":"2023-10-16T10:08:26.278089+0000","last_active":"2023-10-16T10:08:29.133993+0000","last_peered":"2023-10-16T10:08:29.133993+0000","last_clean":"2023-10-16T10:08:29.133993+0000","last_became_active":"2023-10-16T10:08:26.274964+0000","last_became_peered":"2023-10-16T10:08:26.274964+0000","last_unstale":"2023-10-16T10:08:29.133993+0000","last_undegraded":"2023-10-16T10:08:29.133993+0000","last_fullsized":"2023-10-16T10:08:29.133993+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:35:11.586105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.258690+0000","last_change":"2023-10-16T10:08:26.253003+0000","last_active":"2023-10-16T10:08:27.258690+0000","last_peered":"2023-10-16T10:08:27.258690+0000","last_clean":"2023-10-16T10:08:27.258690+0000","last_became_active":"2023-10-16T10:08:26.252733+0000","last_became_peered":"2023-10-16T10:08:26.252733+0000","last_unstale":"2023-10-16T10:08:27.258690+0000","last_undegraded":"2023-10-16T10:08:27.258690+0000","last_fullsized":"2023-10-16T10:08:27.258690+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:43:26.330501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.259749+0000","last_change":"2023-10-16T10:08:26.269263+0000","last_active":"2023-10-16T10:08:27.259749+0000","last_peered":"2023-10-16T10:08:27.259749+0000","last_clean":"2023-10-16T10:08:27.259749+0000","last_became_active":"2023-10-16T10:08:26.268401+0000","last_became_peered":"2023-10-16T10:08:26.268401+0000","last_unstale":"2023-10-16T10:08:27.259749+0000","last_undegraded":"2023-10-16T10:08:27.259749+0000","last_fullsized":"2023-10-16T10:08:27.259749+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:15:45.340527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.138700+0000","last_change":"2023-10-16T10:08:26.267596+0000","last_active":"2023-10-16T10:08:29.138700+0000","last_peered":"2023-10-16T10:08:29.138700+0000","last_clean":"2023-10-16T10:08:29.138700+0000","last_became_active":"2023-10-16T10:08:26.267282+0000","last_became_peered":"2023-10-16T10:08:26.267282+0000","last_unstale":"2023-10-16T10:08:29.138700+0000","last_undegraded":"2023-10-16T10:08:29.138700+0000","last_fullsized":"2023-10-16T10:08:29.138700+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:50:03.958131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.137411+0000","last_change":"2023-10-16T10:08:26.274393+0000","last_active":"2023-10-16T10:08:29.137411+0000","last_peered":"2023-10-16T10:08:29.137411+0000","last_clean":"2023-10-16T10:08:29.137411+0000","last_became_active":"2023-10-16T10:08:26.273836+0000","last_became_peered":"2023-10-16T10:08:26.273836+0000","last_unstale":"2023-10-16T10:08:29.137411+0000","last_undegraded":"2023-10-16T10:08:29.137411+0000","last_fullsized":"2023-10-16T10:08:29.137411+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:55.125995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.134338+0000","last_change":"2023-10-16T10:08:26.271401+0000","last_active":"2023-10-16T10:08:29.134338+0000","last_peered":"2023-10-16T10:08:29.134338+0000","last_clean":"2023-10-16T10:08:29.134338+0000","last_became_active":"2023-10-16T10:08:26.271085+0000","last_became_peered":"2023-10-16T10:08:26.271085+0000","last_unstale":"2023-10-16T10:08:29.134338+0000","last_undegraded":"2023-10-16T10:08:29.134338+0000","last_fullsized":"2023-10-16T10:08:29.134338+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:57.991675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.134380+0000","last_change":"2023-10-16T10:08:26.272795+0000","last_active":"2023-10-16T10:08:29.134380+0000","last_peered":"2023-10-16T10:08:29.134380+0000","last_clean":"2023-10-16T10:08:29.134380+0000","last_became_active":"2023-10-16T10:08:26.272594+0000","last_became_peered":"2023-10-16T10:08:26.272594+0000","last_unstale":"2023-10-16T10:08:29.134380+0000","last_undegraded":"2023-10-16T10:08:29.134380+0000","last_fullsized":"2023-10-16T10:08:29.134380+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:32.724082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.139959+0000","last_change":"2023-10-16T10:08:26.261122+0000","last_active":"2023-10-16T10:08:29.139959+0000","last_peered":"2023-10-16T10:08:29.139959+0000","last_clean":"2023-10-16T10:08:29.139959+0000","last_became_active":"2023-10-16T10:08:26.260809+0000","last_became_peered":"2023-10-16T10:08:26.260809+0000","last_unstale":"2023-10-16T10:08:29.139959+0000","last_undegraded":"2023-10-16T10:08:29.139959+0000","last_fullsized":"2023-10-16T10:08:29.139959+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:10:46.381690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.134513+0000","last_change":"2023-10-16T10:08:26.267984+0000","last_active":"2023-10-16T10:08:29.134513+0000","last_peered":"2023-10-16T10:08:29.134513+0000","last_clean":"2023-10-16T10:08:29.134513+0000","last_became_active":"2023-10-16T10:08:26.267645+0000","last_became_peered":"2023-10-16T10:08:26.267645+0000","last_unstale":"2023-10-16T10:08:29.134513+0000","last_undegraded":"2023-10-16T10:08:29.134513+0000","last_fullsized":"2023-10-16T10:08:29.134513+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:46.931837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.136336+0000","last_change":"2023-10-16T10:08:26.276003+0000","last_active":"2023-10-16T10:08:29.136336+0000","last_peered":"2023-10-16T10:08:29.136336+0000","last_clean":"2023-10-16T10:08:29.136336+0000","last_became_active":"2023-10-16T10:08:26.275735+0000","last_became_peered":"2023-10-16T10:08:26.275735+0000","last_unstale":"2023-10-16T10:08:29.136336+0000","last_undegraded":"2023-10-16T10:08:29.136336+0000","last_fullsized":"2023-10-16T10:08:29.136336+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:40.013907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.137957+0000","last_change":"2023-10-16T10:08:26.272373+0000","last_active":"2023-10-16T10:08:29.137957+0000","last_peered":"2023-10-16T10:08:29.137957+0000","last_clean":"2023-10-16T10:08:29.137957+0000","last_became_active":"2023-10-16T10:08:26.272046+0000","last_became_peered":"2023-10-16T10:08:26.272046+0000","last_unstale":"2023-10-16T10:08:29.137957+0000","last_undegraded":"2023-10-16T10:08:29.137957+0000","last_fullsized":"2023-10-16T10:08:29.137957+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:29:51.241530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.259773+0000","last_change":"2023-10-16T10:08:26.255042+0000","last_active":"2023-10-16T10:08:27.259773+0000","last_peered":"2023-10-16T10:08:27.259773+0000","last_clean":"2023-10-16T10:08:27.259773+0000","last_became_active":"2023-10-16T10:08:26.254220+0000","last_became_peered":"2023-10-16T10:08:26.254220+0000","last_unstale":"2023-10-16T10:08:27.259773+0000","last_undegraded":"2023-10-16T10:08:27.259773+0000","last_fullsized":"2023-10-16T10:08:27.259773+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:52.027174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.132971+0000","last_change":"2023-10-16T10:08:26.280057+0000","last_active":"2023-10-16T10:08:29.132971+0000","last_peered":"2023-10-16T10:08:29.132971+0000","last_clean":"2023-10-16T10:08:29.132971+0000","last_became_active":"2023-10-16T10:08:26.276903+0000","last_became_peered":"2023-10-16T10:08:26.276903+0000","last_unstale":"2023-10-16T10:08:29.132971+0000","last_undegraded":"2023-10-16T10:08:29.132971+0000","last_fullsized":"2023-10-16T10:08:29.132971+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:56:00.536576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.139850+0000","last_change":"2023-10-16T10:08:26.261410+0000","last_active":"2023-10-16T10:08:29.139850+0000","last_peered":"2023-10-16T10:08:29.139850+0000","last_clean":"2023-10-16T10:08:29.139850+0000","last_became_active":"2023-10-16T10:08:26.258093+0000","last_became_peered":"2023-10-16T10:08:26.258093+0000","last_unstale":"2023-10-16T10:08:29.139850+0000","last_undegraded":"2023-10-16T10:08:29.139850+0000","last_fullsized":"2023-10-16T10:08:29.139850+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:10:21.928285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.137289+0000","last_change":"2023-10-16T10:08:26.254586+0000","last_active":"2023-10-16T10:08:29.137289+0000","last_peered":"2023-10-16T10:08:29.137289+0000","last_clean":"2023-10-16T10:08:29.137289+0000","last_became_active":"2023-10-16T10:08:26.252590+0000","last_became_peered":"2023-10-16T10:08:26.252590+0000","last_unstale":"2023-10-16T10:08:29.137289+0000","last_undegraded":"2023-10-16T10:08:29.137289+0000","last_fullsized":"2023-10-16T10:08:29.137289+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:17:19.460450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.136345+0000","last_change":"2023-10-16T10:08:26.272835+0000","last_active":"2023-10-16T10:08:29.136345+0000","last_peered":"2023-10-16T10:08:29.136345+0000","last_clean":"2023-10-16T10:08:29.136345+0000","last_became_active":"2023-10-16T10:08:26.272356+0000","last_became_peered":"2023-10-16T10:08:26.272356+0000","last_unstale":"2023-10-16T10:08:29.136345+0000","last_undegraded":"2023-10-16T10:08:29.136345+0000","last_fullsized":"2023-10-16T10:08:29.136345+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:16:59.824826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.140704+0000","last_change":"2023-10-16T10:08:26.275943+0000","last_active":"2023-10-16T10:08:29.140704+0000","last_peered":"2023-10-16T10:08:29.140704+0000","last_clean":"2023-10-16T10:08:29.140704+0000","last_became_active":"2023-10-16T10:08:26.275660+0000","last_became_peered":"2023-10-16T10:08:26.275660+0000","last_unstale":"2023-10-16T10:08:29.140704+0000","last_undegraded":"2023-10-16T10:08:29.140704+0000","last_fullsized":"2023-10-16T10:08:29.140704+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:30.936479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.140626+0000","last_change":"2023-10-16T10:08:26.258709+0000","last_active":"2023-10-16T10:08:29.140626+0000","last_peered":"2023-10-16T10:08:29.140626+0000","last_clean":"2023-10-16T10:08:29.140626+0000","last_became_active":"2023-10-16T10:08:26.257441+0000","last_became_peered":"2023-10-16T10:08:26.257441+0000","last_unstale":"2023-10-16T10:08:29.140626+0000","last_undegraded":"2023-10-16T10:08:29.140626+0000","last_fullsized":"2023-10-16T10:08:29.140626+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:25:33.285479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.264271+0000","last_change":"2023-10-16T10:08:26.264675+0000","last_active":"2023-10-16T10:08:27.264271+0000","last_peered":"2023-10-16T10:08:27.264271+0000","last_clean":"2023-10-16T10:08:27.264271+0000","last_became_active":"2023-10-16T10:08:26.263180+0000","last_became_peered":"2023-10-16T10:08:26.263180+0000","last_unstale":"2023-10-16T10:08:27.264271+0000","last_undegraded":"2023-10-16T10:08:27.264271+0000","last_fullsized":"2023-10-16T10:08:27.264271+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:07:47.070939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.134845+0000","last_change":"2023-10-16T10:08:26.275062+0000","last_active":"2023-10-16T10:08:29.134845+0000","last_peered":"2023-10-16T10:08:29.134845+0000","last_clean":"2023-10-16T10:08:29.134845+0000","last_became_active":"2023-10-16T10:08:26.274796+0000","last_became_peered":"2023-10-16T10:08:26.274796+0000","last_unstale":"2023-10-16T10:08:29.134845+0000","last_undegraded":"2023-10-16T10:08:29.134845+0000","last_fullsized":"2023-10-16T10:08:29.134845+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:01:38.621393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.263686+0000","last_change":"2023-10-16T10:08:26.277305+0000","last_active":"2023-10-16T10:08:27.263686+0000","last_peered":"2023-10-16T10:08:27.263686+0000","last_clean":"2023-10-16T10:08:27.263686+0000","last_became_active":"2023-10-16T10:08:26.276841+0000","last_became_peered":"2023-10-16T10:08:26.276841+0000","last_unstale":"2023-10-16T10:08:27.263686+0000","last_undegraded":"2023-10-16T10:08:27.263686+0000","last_fullsized":"2023-10-16T10:08:27.263686+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:41.342475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.140230+0000","last_change":"2023-10-16T10:08:26.262582+0000","last_active":"2023-10-16T10:08:29.140230+0000","last_peered":"2023-10-16T10:08:29.140230+0000","last_clean":"2023-10-16T10:08:29.140230+0000","last_became_active":"2023-10-16T10:08:26.257735+0000","last_became_peered":"2023-10-16T10:08:26.257735+0000","last_unstale":"2023-10-16T10:08:29.140230+0000","last_undegraded":"2023-10-16T10:08:29.140230+0000","last_fullsized":"2023-10-16T10:08:29.140230+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:44.913421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.134119+0000","last_change":"2023-10-16T10:08:26.273268+0000","last_active":"2023-10-16T10:08:29.134119+0000","last_peered":"2023-10-16T10:08:29.134119+0000","last_clean":"2023-10-16T10:08:29.134119+0000","last_became_active":"2023-10-16T10:08:26.272951+0000","last_became_peered":"2023-10-16T10:08:26.272951+0000","last_unstale":"2023-10-16T10:08:29.134119+0000","last_undegraded":"2023-10-16T10:08:29.134119+0000","last_fullsized":"2023-10-16T10:08:29.134119+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:36.782761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.259802+0000","last_change":"2023-10-16T10:08:26.275814+0000","last_active":"2023-10-16T10:08:27.259802+0000","last_peered":"2023-10-16T10:08:27.259802+0000","last_clean":"2023-10-16T10:08:27.259802+0000","last_became_active":"2023-10-16T10:08:26.274884+0000","last_became_peered":"2023-10-16T10:08:26.274884+0000","last_unstale":"2023-10-16T10:08:27.259802+0000","last_undegraded":"2023-10-16T10:08:27.259802+0000","last_fullsized":"2023-10-16T10:08:27.259802+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:02:37.807028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.264821+0000","last_change":"2023-10-16T10:08:26.260936+0000","last_active":"2023-10-16T10:08:27.264821+0000","last_peered":"2023-10-16T10:08:27.264821+0000","last_clean":"2023-10-16T10:08:27.264821+0000","last_became_active":"2023-10-16T10:08:26.260674+0000","last_became_peered":"2023-10-16T10:08:26.260674+0000","last_unstale":"2023-10-16T10:08:27.264821+0000","last_undegraded":"2023-10-16T10:08:27.264821+0000","last_fullsized":"2023-10-16T10:08:27.264821+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:26:04.374190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.135491+0000","last_change":"2023-10-16T10:08:26.277928+0000","last_active":"2023-10-16T10:08:29.135491+0000","last_peered":"2023-10-16T10:08:29.135491+0000","last_clean":"2023-10-16T10:08:29.135491+0000","last_became_active":"2023-10-16T10:08:26.277632+0000","last_became_peered":"2023-10-16T10:08:26.277632+0000","last_unstale":"2023-10-16T10:08:29.135491+0000","last_undegraded":"2023-10-16T10:08:29.135491+0000","last_fullsized":"2023-10-16T10:08:29.135491+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:50:03.958131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.139876+0000","last_change":"2023-10-16T10:08:26.272140+0000","last_active":"2023-10-16T10:08:29.139876+0000","last_peered":"2023-10-16T10:08:29.139876+0000","last_clean":"2023-10-16T10:08:29.139876+0000","last_became_active":"2023-10-16T10:08:26.271500+0000","last_became_peered":"2023-10-16T10:08:26.271500+0000","last_unstale":"2023-10-16T10:08:29.139876+0000","last_undegraded":"2023-10-16T10:08:29.139876+0000","last_fullsized":"2023-10-16T10:08:29.139876+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:10:21.928285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.133619+0000","last_change":"2023-10-16T10:08:26.274534+0000","last_active":"2023-10-16T10:08:29.133619+0000","last_peered":"2023-10-16T10:08:29.133619+0000","last_clean":"2023-10-16T10:08:29.133619+0000","last_became_active":"2023-10-16T10:08:26.274273+0000","last_became_peered":"2023-10-16T10:08:26.274273+0000","last_unstale":"2023-10-16T10:08:29.133619+0000","last_undegraded":"2023-10-16T10:08:29.133619+0000","last_fullsized":"2023-10-16T10:08:29.133619+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:05.123633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.260280+0000","last_change":"2023-10-16T10:08:26.259118+0000","last_active":"2023-10-16T10:08:27.260280+0000","last_peered":"2023-10-16T10:08:27.260280+0000","last_clean":"2023-10-16T10:08:27.260280+0000","last_became_active":"2023-10-16T10:08:26.258532+0000","last_became_peered":"2023-10-16T10:08:26.258532+0000","last_unstale":"2023-10-16T10:08:27.260280+0000","last_undegraded":"2023-10-16T10:08:27.260280+0000","last_fullsized":"2023-10-16T10:08:27.260280+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:16:29.251757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.259974+0000","last_change":"2023-10-16T10:08:26.257766+0000","last_active":"2023-10-16T10:08:27.259974+0000","last_peered":"2023-10-16T10:08:27.259974+0000","last_clean":"2023-10-16T10:08:27.259974+0000","last_became_active":"2023-10-16T10:08:26.257515+0000","last_became_peered":"2023-10-16T10:08:26.257515+0000","last_unstale":"2023-10-16T10:08:27.259974+0000","last_undegraded":"2023-10-16T10:08:27.259974+0000","last_fullsized":"2023-10-16T10:08:27.259974+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:44.117413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"137'4","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.276280+0000","last_change":"2023-10-16T10:08:25.262040+0000","last_active":"2023-10-16T10:08:26.276280+0000","last_peered":"2023-10-16T10:08:26.276280+0000","last_clean":"2023-10-16T10:08:26.276280+0000","last_became_active":"2023-10-16T10:08:25.261768+0000","last_became_peered":"2023-10-16T10:08:25.261768+0000","last_unstale":"2023-10-16T10:08:26.276280+0000","last_undegraded":"2023-10-16T10:08:26.276280+0000","last_fullsized":"2023-10-16T10:08:26.276280+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:10.098275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.138668+0000","last_change":"2023-10-16T10:08:26.279348+0000","last_active":"2023-10-16T10:08:29.138668+0000","last_peered":"2023-10-16T10:08:29.138668+0000","last_clean":"2023-10-16T10:08:29.138668+0000","last_became_active":"2023-10-16T10:08:26.277014+0000","last_became_peered":"2023-10-16T10:08:26.277014+0000","last_unstale":"2023-10-16T10:08:29.138668+0000","last_undegraded":"2023-10-16T10:08:29.138668+0000","last_fullsized":"2023-10-16T10:08:29.138668+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:13.574636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.273364+0000","last_change":"2023-10-16T10:08:25.263087+0000","last_active":"2023-10-16T10:08:26.273364+0000","last_peered":"2023-10-16T10:08:26.273364+0000","last_clean":"2023-10-16T10:08:26.273364+0000","last_became_active":"2023-10-16T10:08:25.262767+0000","last_became_peered":"2023-10-16T10:08:25.262767+0000","last_unstale":"2023-10-16T10:08:26.273364+0000","last_undegraded":"2023-10-16T10:08:26.273364+0000","last_fullsized":"2023-10-16T10:08:26.273364+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:05.842577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.260666+0000","last_change":"2023-10-16T10:08:26.255964+0000","last_active":"2023-10-16T10:08:27.260666+0000","last_peered":"2023-10-16T10:08:27.260666+0000","last_clean":"2023-10-16T10:08:27.260666+0000","last_became_active":"2023-10-16T10:08:26.255691+0000","last_became_peered":"2023-10-16T10:08:26.255691+0000","last_unstale":"2023-10-16T10:08:27.260666+0000","last_undegraded":"2023-10-16T10:08:27.260666+0000","last_fullsized":"2023-10-16T10:08:27.260666+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:28:43.178170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.272108+0000","last_change":"2023-10-16T10:08:25.246351+0000","last_active":"2023-10-16T10:08:26.272108+0000","last_peered":"2023-10-16T10:08:26.272108+0000","last_clean":"2023-10-16T10:08:26.272108+0000","last_became_active":"2023-10-16T10:08:25.246053+0000","last_became_peered":"2023-10-16T10:08:25.246053+0000","last_unstale":"2023-10-16T10:08:26.272108+0000","last_undegraded":"2023-10-16T10:08:26.272108+0000","last_fullsized":"2023-10-16T10:08:26.272108+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:55.962023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.137087+0000","last_change":"2023-10-16T10:08:26.273727+0000","last_active":"2023-10-16T10:08:29.137087+0000","last_peered":"2023-10-16T10:08:29.137087+0000","last_clean":"2023-10-16T10:08:29.137087+0000","last_became_active":"2023-10-16T10:08:26.272652+0000","last_became_peered":"2023-10-16T10:08:26.272652+0000","last_unstale":"2023-10-16T10:08:29.137087+0000","last_undegraded":"2023-10-16T10:08:29.137087+0000","last_fullsized":"2023-10-16T10:08:29.137087+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:09:00.941907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.272401+0000","last_change":"2023-10-16T10:08:25.247100+0000","last_active":"2023-10-16T10:08:26.272401+0000","last_peered":"2023-10-16T10:08:26.272401+0000","last_clean":"2023-10-16T10:08:26.272401+0000","last_became_active":"2023-10-16T10:08:25.246782+0000","last_became_peered":"2023-10-16T10:08:25.246782+0000","last_unstale":"2023-10-16T10:08:26.272401+0000","last_undegraded":"2023-10-16T10:08:26.272401+0000","last_fullsized":"2023-10-16T10:08:26.272401+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:22.332858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.133245+0000","last_change":"2023-10-16T10:08:26.273760+0000","last_active":"2023-10-16T10:08:29.133245+0000","last_peered":"2023-10-16T10:08:29.133245+0000","last_clean":"2023-10-16T10:08:29.133245+0000","last_became_active":"2023-10-16T10:08:26.273508+0000","last_became_peered":"2023-10-16T10:08:26.273508+0000","last_unstale":"2023-10-16T10:08:29.133245+0000","last_undegraded":"2023-10-16T10:08:29.133245+0000","last_fullsized":"2023-10-16T10:08:29.133245+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:18:01.500362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.270333+0000","last_change":"2023-10-16T10:08:25.257406+0000","last_active":"2023-10-16T10:08:26.270333+0000","last_peered":"2023-10-16T10:08:26.270333+0000","last_clean":"2023-10-16T10:08:26.270333+0000","last_became_active":"2023-10-16T10:08:25.257103+0000","last_became_peered":"2023-10-16T10:08:25.257103+0000","last_unstale":"2023-10-16T10:08:26.270333+0000","last_undegraded":"2023-10-16T10:08:26.270333+0000","last_fullsized":"2023-10-16T10:08:26.270333+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:25.003965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.135856+0000","last_change":"2023-10-16T10:08:26.266279+0000","last_active":"2023-10-16T10:08:29.135856+0000","last_peered":"2023-10-16T10:08:29.135856+0000","last_clean":"2023-10-16T10:08:29.135856+0000","last_became_active":"2023-10-16T10:08:26.265351+0000","last_became_peered":"2023-10-16T10:08:26.265351+0000","last_unstale":"2023-10-16T10:08:29.135856+0000","last_undegraded":"2023-10-16T10:08:29.135856+0000","last_fullsized":"2023-10-16T10:08:29.135856+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:09.584840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.264847+0000","last_change":"2023-10-16T10:08:25.257495+0000","last_active":"2023-10-16T10:08:26.264847+0000","last_peered":"2023-10-16T10:08:26.264847+0000","last_clean":"2023-10-16T10:08:26.264847+0000","last_became_active":"2023-10-16T10:08:25.257208+0000","last_became_peered":"2023-10-16T10:08:25.257208+0000","last_unstale":"2023-10-16T10:08:26.264847+0000","last_undegraded":"2023-10-16T10:08:26.264847+0000","last_fullsized":"2023-10-16T10:08:26.264847+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:17.790922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.133251+0000","last_change":"2023-10-16T10:08:26.261989+0000","last_active":"2023-10-16T10:08:29.133251+0000","last_peered":"2023-10-16T10:08:29.133251+0000","last_clean":"2023-10-16T10:08:29.133251+0000","last_became_active":"2023-10-16T10:08:26.260844+0000","last_became_peered":"2023-10-16T10:08:26.260844+0000","last_unstale":"2023-10-16T10:08:29.133251+0000","last_undegraded":"2023-10-16T10:08:29.133251+0000","last_fullsized":"2023-10-16T10:08:29.133251+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:20:13.580053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.239692+0000","last_change":"2023-10-16T10:08:25.254308+0000","last_active":"2023-10-16T10:08:26.239692+0000","last_peered":"2023-10-16T10:08:26.239692+0000","last_clean":"2023-10-16T10:08:26.239692+0000","last_became_active":"2023-10-16T10:08:25.254010+0000","last_became_peered":"2023-10-16T10:08:25.254010+0000","last_unstale":"2023-10-16T10:08:26.239692+0000","last_undegraded":"2023-10-16T10:08:26.239692+0000","last_fullsized":"2023-10-16T10:08:26.239692+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:45.913441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.139278+0000","last_change":"2023-10-16T10:08:26.269223+0000","last_active":"2023-10-16T10:08:29.139278+0000","last_peered":"2023-10-16T10:08:29.139278+0000","last_clean":"2023-10-16T10:08:29.139278+0000","last_became_active":"2023-10-16T10:08:26.268395+0000","last_became_peered":"2023-10-16T10:08:26.268395+0000","last_unstale":"2023-10-16T10:08:29.139278+0000","last_undegraded":"2023-10-16T10:08:29.139278+0000","last_fullsized":"2023-10-16T10:08:29.139278+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:09.584840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.253138+0000","last_change":"2023-10-16T10:08:25.251903+0000","last_active":"2023-10-16T10:08:26.253138+0000","last_peered":"2023-10-16T10:08:26.253138+0000","last_clean":"2023-10-16T10:08:26.253138+0000","last_became_active":"2023-10-16T10:08:25.251481+0000","last_became_peered":"2023-10-16T10:08:25.251481+0000","last_unstale":"2023-10-16T10:08:26.253138+0000","last_undegraded":"2023-10-16T10:08:26.253138+0000","last_fullsized":"2023-10-16T10:08:26.253138+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:15:44.322131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.138392+0000","last_change":"2023-10-16T10:08:26.272437+0000","last_active":"2023-10-16T10:08:29.138392+0000","last_peered":"2023-10-16T10:08:29.138392+0000","last_clean":"2023-10-16T10:08:29.138392+0000","last_became_active":"2023-10-16T10:08:26.272119+0000","last_became_peered":"2023-10-16T10:08:26.272119+0000","last_unstale":"2023-10-16T10:08:29.138392+0000","last_undegraded":"2023-10-16T10:08:29.138392+0000","last_fullsized":"2023-10-16T10:08:29.138392+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:01:38.621393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.237051+0000","last_change":"2023-10-16T10:08:25.247973+0000","last_active":"2023-10-16T10:08:26.237051+0000","last_peered":"2023-10-16T10:08:26.237051+0000","last_clean":"2023-10-16T10:08:26.237051+0000","last_became_active":"2023-10-16T10:08:25.247532+0000","last_became_peered":"2023-10-16T10:08:25.247532+0000","last_unstale":"2023-10-16T10:08:26.237051+0000","last_undegraded":"2023-10-16T10:08:26.237051+0000","last_fullsized":"2023-10-16T10:08:26.237051+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:12.556240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.139030+0000","last_change":"2023-10-16T10:08:26.278409+0000","last_active":"2023-10-16T10:08:29.139030+0000","last_peered":"2023-10-16T10:08:29.139030+0000","last_clean":"2023-10-16T10:08:29.139030+0000","last_became_active":"2023-10-16T10:08:26.276646+0000","last_became_peered":"2023-10-16T10:08:26.276646+0000","last_unstale":"2023-10-16T10:08:29.139030+0000","last_undegraded":"2023-10-16T10:08:29.139030+0000","last_fullsized":"2023-10-16T10:08:29.139030+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:24:36.708728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.247393+0000","last_change":"2023-10-16T10:08:25.258954+0000","last_active":"2023-10-16T10:08:26.247393+0000","last_peered":"2023-10-16T10:08:26.247393+0000","last_clean":"2023-10-16T10:08:26.247393+0000","last_became_active":"2023-10-16T10:08:25.258629+0000","last_became_peered":"2023-10-16T10:08:25.258629+0000","last_unstale":"2023-10-16T10:08:26.247393+0000","last_undegraded":"2023-10-16T10:08:26.247393+0000","last_fullsized":"2023-10-16T10:08:26.247393+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:14.511192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.265114+0000","last_change":"2023-10-16T10:08:26.253237+0000","last_active":"2023-10-16T10:08:27.265114+0000","last_peered":"2023-10-16T10:08:27.265114+0000","last_clean":"2023-10-16T10:08:27.265114+0000","last_became_active":"2023-10-16T10:08:26.252936+0000","last_became_peered":"2023-10-16T10:08:26.252936+0000","last_unstale":"2023-10-16T10:08:27.265114+0000","last_undegraded":"2023-10-16T10:08:27.265114+0000","last_fullsized":"2023-10-16T10:08:27.265114+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:37:19.234903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"137'3","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.274251+0000","last_change":"2023-10-16T10:08:25.255885+0000","last_active":"2023-10-16T10:08:26.274251+0000","last_peered":"2023-10-16T10:08:26.274251+0000","last_clean":"2023-10-16T10:08:26.274251+0000","last_became_active":"2023-10-16T10:08:25.255645+0000","last_became_peered":"2023-10-16T10:08:25.255645+0000","last_unstale":"2023-10-16T10:08:26.274251+0000","last_undegraded":"2023-10-16T10:08:26.274251+0000","last_fullsized":"2023-10-16T10:08:26.274251+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:32:55.153046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.138703+0000","last_change":"2023-10-16T10:08:26.276221+0000","last_active":"2023-10-16T10:08:29.138703+0000","last_peered":"2023-10-16T10:08:29.138703+0000","last_clean":"2023-10-16T10:08:29.138703+0000","last_became_active":"2023-10-16T10:08:26.275740+0000","last_became_peered":"2023-10-16T10:08:26.275740+0000","last_unstale":"2023-10-16T10:08:29.138703+0000","last_undegraded":"2023-10-16T10:08:29.138703+0000","last_fullsized":"2023-10-16T10:08:29.138703+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:58:45.263010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"137'6","reported_seq":18,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.282278+0000","last_change":"2023-10-16T10:08:25.247740+0000","last_active":"2023-10-16T10:08:26.282278+0000","last_peered":"2023-10-16T10:08:26.282278+0000","last_clean":"2023-10-16T10:08:26.282278+0000","last_became_active":"2023-10-16T10:08:25.246511+0000","last_became_peered":"2023-10-16T10:08:25.246511+0000","last_unstale":"2023-10-16T10:08:26.282278+0000","last_undegraded":"2023-10-16T10:08:26.282278+0000","last_fullsized":"2023-10-16T10:08:26.282278+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:16:28.233361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.264545+0000","last_change":"2023-10-16T10:08:26.271952+0000","last_active":"2023-10-16T10:08:27.264545+0000","last_peered":"2023-10-16T10:08:27.264545+0000","last_clean":"2023-10-16T10:08:27.264545+0000","last_became_active":"2023-10-16T10:08:26.271656+0000","last_became_peered":"2023-10-16T10:08:26.271656+0000","last_unstale":"2023-10-16T10:08:27.264545+0000","last_undegraded":"2023-10-16T10:08:27.264545+0000","last_fullsized":"2023-10-16T10:08:27.264545+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:41.352237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.276104+0000","last_change":"2023-10-16T10:08:25.260285+0000","last_active":"2023-10-16T10:08:26.276104+0000","last_peered":"2023-10-16T10:08:26.276104+0000","last_clean":"2023-10-16T10:08:26.276104+0000","last_became_active":"2023-10-16T10:08:25.260075+0000","last_became_peered":"2023-10-16T10:08:25.260075+0000","last_unstale":"2023-10-16T10:08:26.276104+0000","last_undegraded":"2023-10-16T10:08:26.276104+0000","last_fullsized":"2023-10-16T10:08:26.276104+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:24:35.690332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.264461+0000","last_change":"2023-10-16T10:08:26.271063+0000","last_active":"2023-10-16T10:08:27.264461+0000","last_peered":"2023-10-16T10:08:27.264461+0000","last_clean":"2023-10-16T10:08:27.264461+0000","last_became_active":"2023-10-16T10:08:26.270788+0000","last_became_peered":"2023-10-16T10:08:26.270788+0000","last_unstale":"2023-10-16T10:08:27.264461+0000","last_undegraded":"2023-10-16T10:08:27.264461+0000","last_fullsized":"2023-10-16T10:08:27.264461+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:10:22.379381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.240068+0000","last_change":"2023-10-16T10:08:25.256935+0000","last_active":"2023-10-16T10:08:26.240068+0000","last_peered":"2023-10-16T10:08:26.240068+0000","last_clean":"2023-10-16T10:08:26.240068+0000","last_became_active":"2023-10-16T10:08:25.256646+0000","last_became_peered":"2023-10-16T10:08:25.256646+0000","last_unstale":"2023-10-16T10:08:26.240068+0000","last_undegraded":"2023-10-16T10:08:26.240068+0000","last_fullsized":"2023-10-16T10:08:26.240068+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:05.949599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.263508+0000","last_change":"2023-10-16T10:08:26.251738+0000","last_active":"2023-10-16T10:08:27.263508+0000","last_peered":"2023-10-16T10:08:27.263508+0000","last_clean":"2023-10-16T10:08:27.263508+0000","last_became_active":"2023-10-16T10:08:26.251464+0000","last_became_peered":"2023-10-16T10:08:26.251464+0000","last_unstale":"2023-10-16T10:08:27.263508+0000","last_undegraded":"2023-10-16T10:08:27.263508+0000","last_fullsized":"2023-10-16T10:08:27.263508+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:30:38.819179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"137'8","reported_seq":20,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.273613+0000","last_change":"2023-10-16T10:08:25.250724+0000","last_active":"2023-10-16T10:08:26.273613+0000","last_peered":"2023-10-16T10:08:26.273613+0000","last_clean":"2023-10-16T10:08:26.273613+0000","last_became_active":"2023-10-16T10:08:25.250460+0000","last_became_peered":"2023-10-16T10:08:25.250460+0000","last_unstale":"2023-10-16T10:08:26.273613+0000","last_undegraded":"2023-10-16T10:08:26.273613+0000","last_fullsized":"2023-10-16T10:08:26.273613+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:12:20.130900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.260933+0000","last_change":"2023-10-16T10:08:26.267823+0000","last_active":"2023-10-16T10:08:27.260933+0000","last_peered":"2023-10-16T10:08:27.260933+0000","last_clean":"2023-10-16T10:08:27.260933+0000","last_became_active":"2023-10-16T10:08:26.267411+0000","last_became_peered":"2023-10-16T10:08:26.267411+0000","last_unstale":"2023-10-16T10:08:27.260933+0000","last_undegraded":"2023-10-16T10:08:27.260933+0000","last_fullsized":"2023-10-16T10:08:27.260933+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:10:39.682619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.274033+0000","last_change":"2023-10-16T10:08:25.259547+0000","last_active":"2023-10-16T10:08:26.274033+0000","last_peered":"2023-10-16T10:08:26.274033+0000","last_clean":"2023-10-16T10:08:26.274033+0000","last_became_active":"2023-10-16T10:08:25.259330+0000","last_became_peered":"2023-10-16T10:08:25.259330+0000","last_unstale":"2023-10-16T10:08:26.274033+0000","last_undegraded":"2023-10-16T10:08:26.274033+0000","last_fullsized":"2023-10-16T10:08:26.274033+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:20:25.308279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.260297+0000","last_change":"2023-10-16T10:08:26.275460+0000","last_active":"2023-10-16T10:08:27.260297+0000","last_peered":"2023-10-16T10:08:27.260297+0000","last_clean":"2023-10-16T10:08:27.260297+0000","last_became_active":"2023-10-16T10:08:26.275179+0000","last_became_peered":"2023-10-16T10:08:26.275179+0000","last_unstale":"2023-10-16T10:08:27.260297+0000","last_undegraded":"2023-10-16T10:08:27.260297+0000","last_fullsized":"2023-10-16T10:08:27.260297+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:32:56.171441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.269678+0000","last_change":"2023-10-16T10:08:25.262287+0000","last_active":"2023-10-16T10:08:26.269678+0000","last_peered":"2023-10-16T10:08:26.269678+0000","last_clean":"2023-10-16T10:08:26.269678+0000","last_became_active":"2023-10-16T10:08:25.261948+0000","last_became_peered":"2023-10-16T10:08:25.261948+0000","last_unstale":"2023-10-16T10:08:26.269678+0000","last_undegraded":"2023-10-16T10:08:26.269678+0000","last_fullsized":"2023-10-16T10:08:26.269678+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:05:45.988873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.137304+0000","last_change":"2023-10-16T10:08:26.280257+0000","last_active":"2023-10-16T10:08:29.137304+0000","last_peered":"2023-10-16T10:08:29.137304+0000","last_clean":"2023-10-16T10:08:29.137304+0000","last_became_active":"2023-10-16T10:08:26.279895+0000","last_became_peered":"2023-10-16T10:08:26.279895+0000","last_unstale":"2023-10-16T10:08:29.137304+0000","last_undegraded":"2023-10-16T10:08:29.137304+0000","last_fullsized":"2023-10-16T10:08:29.137304+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:37.768708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.273454+0000","last_change":"2023-10-16T10:08:25.256427+0000","last_active":"2023-10-16T10:08:26.273454+0000","last_peered":"2023-10-16T10:08:26.273454+0000","last_clean":"2023-10-16T10:08:26.273454+0000","last_became_active":"2023-10-16T10:08:25.254558+0000","last_became_peered":"2023-10-16T10:08:25.254558+0000","last_unstale":"2023-10-16T10:08:26.273454+0000","last_undegraded":"2023-10-16T10:08:26.273454+0000","last_fullsized":"2023-10-16T10:08:26.273454+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:23:22.195630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.137462+0000","last_change":"2023-10-16T10:08:26.273363+0000","last_active":"2023-10-16T10:08:29.137462+0000","last_peered":"2023-10-16T10:08:29.137462+0000","last_clean":"2023-10-16T10:08:29.137462+0000","last_became_active":"2023-10-16T10:08:26.273072+0000","last_became_peered":"2023-10-16T10:08:26.273072+0000","last_unstale":"2023-10-16T10:08:29.137462+0000","last_undegraded":"2023-10-16T10:08:29.137462+0000","last_fullsized":"2023-10-16T10:08:29.137462+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:55.952398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.255907+0000","last_change":"2023-10-16T10:08:25.260069+0000","last_active":"2023-10-16T10:08:26.255907+0000","last_peered":"2023-10-16T10:08:26.255907+0000","last_clean":"2023-10-16T10:08:26.255907+0000","last_became_active":"2023-10-16T10:08:25.259758+0000","last_became_peered":"2023-10-16T10:08:25.259758+0000","last_unstale":"2023-10-16T10:08:26.255907+0000","last_undegraded":"2023-10-16T10:08:26.255907+0000","last_fullsized":"2023-10-16T10:08:26.255907+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:39.485099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.138059+0000","last_change":"2023-10-16T10:08:26.272485+0000","last_active":"2023-10-16T10:08:29.138059+0000","last_peered":"2023-10-16T10:08:29.138059+0000","last_clean":"2023-10-16T10:08:29.138059+0000","last_became_active":"2023-10-16T10:08:26.272194+0000","last_became_peered":"2023-10-16T10:08:26.272194+0000","last_unstale":"2023-10-16T10:08:29.138059+0000","last_undegraded":"2023-10-16T10:08:29.138059+0000","last_fullsized":"2023-10-16T10:08:29.138059+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:51:27.673790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.274408+0000","last_change":"2023-10-16T10:08:25.242736+0000","last_active":"2023-10-16T10:08:26.274408+0000","last_peered":"2023-10-16T10:08:26.274408+0000","last_clean":"2023-10-16T10:08:26.274408+0000","last_became_active":"2023-10-16T10:08:25.242031+0000","last_became_peered":"2023-10-16T10:08:25.242031+0000","last_unstale":"2023-10-16T10:08:26.274408+0000","last_undegraded":"2023-10-16T10:08:26.274408+0000","last_fullsized":"2023-10-16T10:08:26.274408+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:05:45.988873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.133208+0000","last_change":"2023-10-16T10:08:26.278628+0000","last_active":"2023-10-16T10:08:29.133208+0000","last_peered":"2023-10-16T10:08:29.133208+0000","last_clean":"2023-10-16T10:08:29.133208+0000","last_became_active":"2023-10-16T10:08:26.278368+0000","last_became_peered":"2023-10-16T10:08:26.278368+0000","last_unstale":"2023-10-16T10:08:29.133208+0000","last_undegraded":"2023-10-16T10:08:29.133208+0000","last_fullsized":"2023-10-16T10:08:29.133208+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:25:54.300572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.236479+0000","last_change":"2023-10-16T10:08:25.249107+0000","last_active":"2023-10-16T10:08:26.236479+0000","last_peered":"2023-10-16T10:08:26.236479+0000","last_clean":"2023-10-16T10:08:26.236479+0000","last_became_active":"2023-10-16T10:08:25.248477+0000","last_became_peered":"2023-10-16T10:08:25.248477+0000","last_unstale":"2023-10-16T10:08:26.236479+0000","last_undegraded":"2023-10-16T10:08:26.236479+0000","last_fullsized":"2023-10-16T10:08:26.236479+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:26:13.129741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.137671+0000","last_change":"2023-10-16T10:08:26.274562+0000","last_active":"2023-10-16T10:08:29.137671+0000","last_peered":"2023-10-16T10:08:29.137671+0000","last_clean":"2023-10-16T10:08:29.137671+0000","last_became_active":"2023-10-16T10:08:26.274105+0000","last_became_peered":"2023-10-16T10:08:26.274105+0000","last_unstale":"2023-10-16T10:08:29.137671+0000","last_undegraded":"2023-10-16T10:08:29.137671+0000","last_fullsized":"2023-10-16T10:08:29.137671+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:38:24.715623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.249085+0000","last_change":"2023-10-16T10:08:25.244509+0000","last_active":"2023-10-16T10:08:26.249085+0000","last_peered":"2023-10-16T10:08:26.249085+0000","last_clean":"2023-10-16T10:08:26.249085+0000","last_became_active":"2023-10-16T10:08:25.244196+0000","last_became_peered":"2023-10-16T10:08:25.244196+0000","last_unstale":"2023-10-16T10:08:26.249085+0000","last_undegraded":"2023-10-16T10:08:26.249085+0000","last_fullsized":"2023-10-16T10:08:26.249085+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:10:38.664224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.263744+0000","last_change":"2023-10-16T10:08:26.261335+0000","last_active":"2023-10-16T10:08:27.263744+0000","last_peered":"2023-10-16T10:08:27.263744+0000","last_clean":"2023-10-16T10:08:27.263744+0000","last_became_active":"2023-10-16T10:08:26.261049+0000","last_became_peered":"2023-10-16T10:08:26.261049+0000","last_unstale":"2023-10-16T10:08:27.263744+0000","last_undegraded":"2023-10-16T10:08:27.263744+0000","last_fullsized":"2023-10-16T10:08:27.263744+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:06.860973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.253288+0000","last_change":"2023-10-16T10:08:25.258208+0000","last_active":"2023-10-16T10:08:26.253288+0000","last_peered":"2023-10-16T10:08:26.253288+0000","last_clean":"2023-10-16T10:08:26.253288+0000","last_became_active":"2023-10-16T10:08:25.257929+0000","last_became_peered":"2023-10-16T10:08:25.257929+0000","last_unstale":"2023-10-16T10:08:26.253288+0000","last_undegraded":"2023-10-16T10:08:26.253288+0000","last_fullsized":"2023-10-16T10:08:26.253288+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:51.008778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.137900+0000","last_change":"2023-10-16T10:08:26.275698+0000","last_active":"2023-10-16T10:08:29.137900+0000","last_peered":"2023-10-16T10:08:29.137900+0000","last_clean":"2023-10-16T10:08:29.137900+0000","last_became_active":"2023-10-16T10:08:26.273577+0000","last_became_peered":"2023-10-16T10:08:26.273577+0000","last_unstale":"2023-10-16T10:08:29.137900+0000","last_undegraded":"2023-10-16T10:08:29.137900+0000","last_fullsized":"2023-10-16T10:08:29.137900+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:56.980419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.274954+0000","last_change":"2023-10-16T10:08:25.253364+0000","last_active":"2023-10-16T10:08:26.274954+0000","last_peered":"2023-10-16T10:08:26.274954+0000","last_clean":"2023-10-16T10:08:26.274954+0000","last_became_active":"2023-10-16T10:08:25.253048+0000","last_became_peered":"2023-10-16T10:08:25.253048+0000","last_unstale":"2023-10-16T10:08:26.274954+0000","last_undegraded":"2023-10-16T10:08:26.274954+0000","last_fullsized":"2023-10-16T10:08:26.274954+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:25:18.095377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.134624+0000","last_change":"2023-10-16T10:08:26.271057+0000","last_active":"2023-10-16T10:08:29.134624+0000","last_peered":"2023-10-16T10:08:29.134624+0000","last_clean":"2023-10-16T10:08:29.134624+0000","last_became_active":"2023-10-16T10:08:26.270780+0000","last_became_peered":"2023-10-16T10:08:26.270780+0000","last_unstale":"2023-10-16T10:08:29.134624+0000","last_undegraded":"2023-10-16T10:08:29.134624+0000","last_fullsized":"2023-10-16T10:08:29.134624+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:44:22.750010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.243382+0000","last_change":"2023-10-16T10:08:25.261749+0000","last_active":"2023-10-16T10:08:26.243382+0000","last_peered":"2023-10-16T10:08:26.243382+0000","last_clean":"2023-10-16T10:08:26.243382+0000","last_became_active":"2023-10-16T10:08:25.261106+0000","last_became_peered":"2023-10-16T10:08:25.261106+0000","last_unstale":"2023-10-16T10:08:26.243382+0000","last_undegraded":"2023-10-16T10:08:26.243382+0000","last_fullsized":"2023-10-16T10:08:26.243382+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:22.332858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.134381+0000","last_change":"2023-10-16T10:08:26.279489+0000","last_active":"2023-10-16T10:08:29.134381+0000","last_peered":"2023-10-16T10:08:29.134381+0000","last_clean":"2023-10-16T10:08:29.134381+0000","last_became_active":"2023-10-16T10:08:26.279149+0000","last_became_peered":"2023-10-16T10:08:26.279149+0000","last_unstale":"2023-10-16T10:08:29.134381+0000","last_undegraded":"2023-10-16T10:08:29.134381+0000","last_fullsized":"2023-10-16T10:08:29.134381+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:43:04.688389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.258084+0000","last_change":"2023-10-16T10:08:25.258390+0000","last_active":"2023-10-16T10:08:26.258084+0000","last_peered":"2023-10-16T10:08:26.258084+0000","last_clean":"2023-10-16T10:08:26.258084+0000","last_became_active":"2023-10-16T10:08:25.258098+0000","last_became_peered":"2023-10-16T10:08:25.258098+0000","last_unstale":"2023-10-16T10:08:26.258084+0000","last_undegraded":"2023-10-16T10:08:26.258084+0000","last_fullsized":"2023-10-16T10:08:26.258084+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:28:42.159774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.136806+0000","last_change":"2023-10-16T10:08:26.257765+0000","last_active":"2023-10-16T10:08:29.136806+0000","last_peered":"2023-10-16T10:08:29.136806+0000","last_clean":"2023-10-16T10:08:29.136806+0000","last_became_active":"2023-10-16T10:08:26.257255+0000","last_became_peered":"2023-10-16T10:08:26.257255+0000","last_unstale":"2023-10-16T10:08:29.136806+0000","last_undegraded":"2023-10-16T10:08:29.136806+0000","last_fullsized":"2023-10-16T10:08:29.136806+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:22:15.233943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.247655+0000","last_change":"2023-10-16T10:08:25.248506+0000","last_active":"2023-10-16T10:08:26.247655+0000","last_peered":"2023-10-16T10:08:26.247655+0000","last_clean":"2023-10-16T10:08:26.247655+0000","last_became_active":"2023-10-16T10:08:25.247947+0000","last_became_peered":"2023-10-16T10:08:25.247947+0000","last_unstale":"2023-10-16T10:08:26.247655+0000","last_undegraded":"2023-10-16T10:08:26.247655+0000","last_fullsized":"2023-10-16T10:08:26.247655+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:08.376496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.135694+0000","last_change":"2023-10-16T10:08:26.270421+0000","last_active":"2023-10-16T10:08:29.135694+0000","last_peered":"2023-10-16T10:08:29.135694+0000","last_clean":"2023-10-16T10:08:29.135694+0000","last_became_active":"2023-10-16T10:08:26.269775+0000","last_became_peered":"2023-10-16T10:08:26.269775+0000","last_unstale":"2023-10-16T10:08:29.135694+0000","last_undegraded":"2023-10-16T10:08:29.135694+0000","last_fullsized":"2023-10-16T10:08:29.135694+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:58:45.263010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.255594+0000","last_change":"2023-10-16T10:08:25.260303+0000","last_active":"2023-10-16T10:08:26.255594+0000","last_peered":"2023-10-16T10:08:26.255594+0000","last_clean":"2023-10-16T10:08:26.255594+0000","last_became_active":"2023-10-16T10:08:25.259967+0000","last_became_peered":"2023-10-16T10:08:25.259967+0000","last_unstale":"2023-10-16T10:08:26.255594+0000","last_undegraded":"2023-10-16T10:08:26.255594+0000","last_fullsized":"2023-10-16T10:08:26.255594+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:08:09.423160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.140312+0000","last_change":"2023-10-16T10:08:26.251617+0000","last_active":"2023-10-16T10:08:29.140312+0000","last_peered":"2023-10-16T10:08:29.140312+0000","last_clean":"2023-10-16T10:08:29.140312+0000","last_became_active":"2023-10-16T10:08:26.250783+0000","last_became_peered":"2023-10-16T10:08:26.250783+0000","last_unstale":"2023-10-16T10:08:29.140312+0000","last_undegraded":"2023-10-16T10:08:29.140312+0000","last_fullsized":"2023-10-16T10:08:29.140312+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:51.020862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.241985+0000","last_change":"2023-10-16T10:08:25.256421+0000","last_active":"2023-10-16T10:08:26.241985+0000","last_peered":"2023-10-16T10:08:26.241985+0000","last_clean":"2023-10-16T10:08:26.241985+0000","last_became_active":"2023-10-16T10:08:25.256156+0000","last_became_peered":"2023-10-16T10:08:25.256156+0000","last_unstale":"2023-10-16T10:08:26.241985+0000","last_undegraded":"2023-10-16T10:08:26.241985+0000","last_fullsized":"2023-10-16T10:08:26.241985+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:10.098275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.138038+0000","last_change":"2023-10-16T10:08:26.280924+0000","last_active":"2023-10-16T10:08:29.138038+0000","last_peered":"2023-10-16T10:08:29.138038+0000","last_clean":"2023-10-16T10:08:29.138038+0000","last_became_active":"2023-10-16T10:08:26.280413+0000","last_became_peered":"2023-10-16T10:08:26.280413+0000","last_unstale":"2023-10-16T10:08:29.138038+0000","last_undegraded":"2023-10-16T10:08:29.138038+0000","last_fullsized":"2023-10-16T10:08:29.138038+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:10.803585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.236587+0000","last_change":"2023-10-16T10:08:25.246369+0000","last_active":"2023-10-16T10:08:26.236587+0000","last_peered":"2023-10-16T10:08:26.236587+0000","last_clean":"2023-10-16T10:08:26.236587+0000","last_became_active":"2023-10-16T10:08:25.246101+0000","last_became_peered":"2023-10-16T10:08:25.246101+0000","last_unstale":"2023-10-16T10:08:26.236587+0000","last_undegraded":"2023-10-16T10:08:26.236587+0000","last_fullsized":"2023-10-16T10:08:26.236587+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:17:18.442054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:29.133984+0000","last_change":"2023-10-16T10:08:26.259001+0000","last_active":"2023-10-16T10:08:29.133984+0000","last_peered":"2023-10-16T10:08:29.133984+0000","last_clean":"2023-10-16T10:08:29.133984+0000","last_became_active":"2023-10-16T10:08:26.258741+0000","last_became_peered":"2023-10-16T10:08:26.258741+0000","last_unstale":"2023-10-16T10:08:29.133984+0000","last_undegraded":"2023-10-16T10:08:29.133984+0000","last_fullsized":"2023-10-16T10:08:29.133984+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:40:43.260342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.254672+0000","last_change":"2023-10-16T10:08:25.258263+0000","last_active":"2023-10-16T10:08:26.254672+0000","last_peered":"2023-10-16T10:08:26.254672+0000","last_clean":"2023-10-16T10:08:26.254672+0000","last_became_active":"2023-10-16T10:08:25.257979+0000","last_became_peered":"2023-10-16T10:08:25.257979+0000","last_unstale":"2023-10-16T10:08:26.254672+0000","last_undegraded":"2023-10-16T10:08:26.254672+0000","last_fullsized":"2023-10-16T10:08:26.254672+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:43.099017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-16T10:08:27.263742+0000","last_change":"2023-10-16T10:08:26.277114+0000","last_active":"2023-10-16T10:08:27.263742+0000","last_peered":"2023-10-16T10:08:27.263742+0000","last_clean":"2023-10-16T10:08:27.263742+0000","last_became_active":"2023-10-16T10:08:26.276485+0000","last_became_peered":"2023-10-16T10:08:26.276485+0000","last_unstale":"2023-10-16T10:08:27.263742+0000","last_undegraded":"2023-10-16T10:08:27.263742+0000","last_fullsized":"2023-10-16T10:08:27.263742+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:25.207995+0000","last_clean_scrub_stamp":"2023-10-16T10:08:25.207995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:47:46.896643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.262322+0000","last_change":"2023-10-16T10:08:25.261482+0000","last_active":"2023-10-16T10:08:26.262322+0000","last_peered":"2023-10-16T10:08:26.262322+0000","last_clean":"2023-10-16T10:08:26.262322+0000","last_became_active":"2023-10-16T10:08:25.259231+0000","last_became_peered":"2023-10-16T10:08:25.259231+0000","last_unstale":"2023-10-16T10:08:26.262322+0000","last_undegraded":"2023-10-16T10:08:26.262322+0000","last_fullsized":"2023-10-16T10:08:26.262322+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:08:24.189599+0000","last_clean_scrub_stamp":"2023-10-16T10:08:24.189599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:17:44.030296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"127'2646","reported_seq":3118,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-16T10:08:26.267923+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T10:08:26.267923+0000","last_peered":"2023-10-16T10:08:26.267923+0000","last_clean":"2023-10-16T10:08:26.267923+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T10:08:26.267923+0000","last_undegraded":"2023-10-16T10:08:26.267923+0000","last_fullsized":"2023-10-16T10:08:26.267923+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2646,"log_dups_size":0,"ondisk_log_size":2646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1120,"num_read_kb":2202,"num_write":361,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1120,"num_read_kb":2202,"num_write":361,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2646,"ondisk_log_size":2646,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575079,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33428,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338412,"statfs":{"total":96636764160,"available":96602533888,"internally_reserved":0,"allocated":2301952,"data_stored":1404224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1699999999999999,"15min":1.171},"min":{"1min":0.81200000000000006,"5min":0.65000000000000002,"15min":0.54400000000000004},"max":{"1min":1.6160000000000001,"5min":1.7310000000000001,"15min":1.925},"last":0.432},{"interface":"front","average":{"1min":1.27,"5min":1.159,"15min":1.1899999999999999},"min":{"1min":0.90400000000000003,"5min":0.58099999999999996,"15min":0.33600000000000002},"max":{"1min":1.702,"5min":1.7370000000000001,"15min":2.016},"last":0.55200000000000005}]},{"osd":1,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1759999999999999,"15min":1.2210000000000001},"min":{"1min":0.72899999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.48,"5min":1.788,"15min":1.923},"last":0.502},{"interface":"front","average":{"1min":1.214,"5min":1.161,"15min":1.2090000000000001},"min":{"1min":0.746,"5min":0.67000000000000004,"15min":0.60999999999999999},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.40699999999999997}]},{"osd":2,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.21,"15min":1.24},"min":{"1min":0.63900000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8109999999999999,"5min":1.837,"15min":1.8740000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2629999999999999,"15min":1.288},"min":{"1min":0.89400000000000002,"5min":0.65900000000000003,"15min":0.64500000000000002},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8620000000000001},"last":0.86699999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3069999999999999,"15min":1.339},"min":{"1min":1.054,"5min":0.747,"15min":0.67700000000000005},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3280000000000001,"15min":1.3720000000000001},"min":{"1min":0.97699999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6799999999999999,"5min":1.8540000000000001,"15min":2.004},"last":0.48199999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3859999999999999,"15min":1.367},"min":{"1min":0.86499999999999999,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6970000000000001,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.3140000000000001,"15min":1.3129999999999999},"min":{"1min":1.0009999999999999,"5min":0.63200000000000001,"15min":0.502},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.45200000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.5069999999999999,"15min":1.52},"min":{"1min":1.1879999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.9330000000000001,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.627},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4199999999999999,"15min":1.4279999999999999},"min":{"1min":1.1910000000000001,"5min":0.72199999999999998,"15min":0.45400000000000001},"max":{"1min":1.6699999999999999,"5min":1.9630000000000001,"15min":2.2170000000000001},"last":0.95199999999999996}]},{"osd":6,"last update":"Mon Oct 16 10:07:30 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.419,"15min":1.4039999999999999},"min":{"1min":0.97599999999999998,"5min":0.82899999999999996,"15min":0.54100000000000004},"max":{"1min":2.0899999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.631,"15min":1.617},"min":{"1min":1.212,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.9279999999999999,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.80900000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575079,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41176,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330664,"statfs":{"total":96636764160,"available":96594599936,"internally_reserved":0,"allocated":1716224,"data_stored":819638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.131,"15min":1.143},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.619,"5min":1.736,"15min":1.7689999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.216,"15min":1.228},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5329999999999999,"5min":1.7010000000000001,"15min":1.847},"last":0.76100000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1459999999999999,"15min":1.1659999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":1.171,"5min":1.278,"15min":1.286},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6539999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.1879999999999999,"15min":1.1990000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45100000000000001},"max":{"1min":1.4710000000000001,"5min":1.851,"15min":2.036},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.248,"15min":1.296},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.58999999999999997},"max":{"1min":1.6859999999999999,"5min":1.925,"15min":1.925},"last":0.73999999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1579999999999999,"15min":1.202},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":2.8239999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.087,"5min":1.1759999999999999,"15min":1.218},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5049999999999999,"5min":1.6719999999999999,"15min":1.865},"last":0.69099999999999995}]},{"osd":4,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.28,"15min":1.2490000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.46000000000000002},"max":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":2.1629999999999998},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.423,"15min":1.4239999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.9730000000000001,"5min":2,"15min":3.6880000000000002},"last":0.51800000000000002}]},{"osd":5,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3080000000000001,"15min":1.3160000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.38900000000000001},"max":{"1min":1.647,"5min":2.0790000000000002,"15min":2.5649999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.45,"5min":1.5,"15min":1.502},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.495},"max":{"1min":1.8440000000000001,"5min":1.996,"15min":2.3849999999999998},"last":0.79800000000000004}]},{"osd":7,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.478,"15min":1.47},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":3.5739999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.47,"5min":1.512,"15min":1.488},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.4249999999999998},"last":0.81899999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575079,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37016,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334824,"statfs":{"total":96636764160,"available":96598859776,"internally_reserved":0,"allocated":1708032,"data_stored":813844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.179,"15min":1.204},"min":{"1min":0.65600000000000003,"5min":0.57799999999999996,"15min":0.496},"max":{"1min":1.6279999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.236,"15min":1.2849999999999999},"min":{"1min":0.72599999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.4219999999999999,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.2370000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.119,"15min":1.1499999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.647,"5min":1.7310000000000001,"15min":1.7729999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.139,"15min":1.1439999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.478,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.2909999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1339999999999999,"15min":1.155},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5700000000000001,"5min":1.738,"15min":1.77},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.21,"15min":1.232},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.645,"5min":1.6739999999999999,"15min":1.8109999999999999},"last":1.0740000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2609999999999999,"15min":1.27},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.76000000000000001},"max":{"1min":1.8260000000000001,"5min":1.9530000000000001,"15min":1.97},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3149999999999999,"15min":1.347},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6100000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.3280000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.278,"15min":1.2689999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.73499999999999999},"max":{"1min":1.6850000000000001,"5min":1.984,"15min":1.984},"last":1.133},{"interface":"front","average":{"1min":1.444,"5min":1.51,"15min":1.502},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.7470000000000001,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.5289999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3360000000000001,"15min":1.3540000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.722,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.274},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.381,"15min":1.3799999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.613,"5min":1.986,"15min":1.986},"last":1.4990000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:07:27 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.464,"15min":1.47},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.9219999999999999,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":1.351},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.5489999999999999,"15min":1.5620000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.788,"5min":2.234,"15min":2.234},"last":1.383}]}]},{"osd":4,"up_from":13,"seq":55834575080,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46736,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325104,"statfs":{"total":96636764160,"available":96588906496,"internally_reserved":0,"allocated":1699840,"data_stored":813240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.079,"15min":1.141},"min":{"1min":0.64300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.5349999999999999,"5min":1.7609999999999999,"15min":1.827},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3220000000000001,"15min":1.2689999999999999},"min":{"1min":0.89600000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.7070000000000001,"5min":11.974,"15min":11.974},"last":0.76600000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3260000000000001,"15min":1.2989999999999999},"min":{"1min":0.63900000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.7969999999999999,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.204,"5min":1.2010000000000001,"15min":1.226},"min":{"1min":0.70699999999999996,"5min":0.63600000000000001,"15min":0.624},"max":{"1min":1.5800000000000001,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":0.98099999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.246,"15min":1.248},"min":{"1min":0.78200000000000003,"5min":0.61799999999999999,"15min":0.59399999999999997},"max":{"1min":1.712,"5min":1.897,"15min":1.897},"last":1.512},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3540000000000001,"15min":1.383},"min":{"1min":0.81499999999999995,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6160000000000001,"5min":3.673,"15min":5.0549999999999997},"last":1.0189999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3200000000000001,"15min":1.3},"min":{"1min":0.94399999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.625,"5min":6.8079999999999998,"15min":6.8079999999999998},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.327,"5min":1.3620000000000001,"15min":1.3779999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.67700000000000005},"max":{"1min":1.768,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.6180000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.496,"15min":1.4350000000000001},"min":{"1min":0.77400000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9990000000000001,"5min":8.702,"15min":8.702},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.383,"15min":1.3720000000000001},"min":{"1min":0.93799999999999994,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.331,"5min":2.331,"15min":3.0430000000000001},"last":1.167}]},{"osd":6,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4850000000000001,"15min":1.417},"min":{"1min":0.95399999999999996,"5min":0.84699999999999998,"15min":0.79000000000000004},"max":{"1min":1.6990000000000001,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":1.72},{"interface":"front","average":{"1min":1.496,"5min":1.585,"15min":1.556},"min":{"1min":0.88900000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.891,"5min":4.625,"15min":6.4089999999999998},"last":1.2070000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:07:29 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.5189999999999999,"15min":1.492},"min":{"1min":1.167,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.927,"5min":2.4060000000000001,"15min":2.9399999999999999},"last":1.085},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.579,"15min":1.5700000000000001},"min":{"1min":1.2589999999999999,"5min":1.071,"15min":0.96799999999999997},"max":{"1min":1.9139999999999999,"5min":2.2690000000000001,"15min":2.6120000000000001},"last":1.8260000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575079,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37036,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334804,"statfs":{"total":96636764160,"available":96598839296,"internally_reserved":0,"allocated":1728512,"data_stored":821250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.80100000000000005,"15min":0.84599999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.24399999999999999},"max":{"1min":1.405,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.91400000000000003,"15min":0.99399999999999999},"min":{"1min":0.35999999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.4239999999999999,"5min":1.5349999999999999,"15min":1.708},"last":0.58199999999999996}]},{"osd":1,"last update":"Mon Oct 16 10:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.83699999999999997,"15min":0.92200000000000004},"min":{"1min":0.40400000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1559999999999999,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.93799999999999994,"15min":1.028},"min":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.145,"5min":3.363,"15min":3.363},"last":0.67600000000000005}]},{"osd":2,"last update":"Mon Oct 16 10:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.96599999999999997,"15min":1.0469999999999999},"min":{"1min":0.54900000000000004,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.357,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.878},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.046,"15min":1.1539999999999999},"min":{"1min":0.51900000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.46,"5min":4.1230000000000002,"15min":4.1230000000000002},"last":0.70099999999999996}]},{"osd":4,"last update":"Mon Oct 16 10:07:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.196,"15min":1.2629999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.706,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":1.046},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.26,"15min":1.3169999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.54700000000000004},"max":{"1min":1.665,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.86299999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:07:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2410000000000001,"15min":1.306},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.6919999999999999,"5min":4.1829999999999998,"15min":4.1829999999999998},"last":1.147},{"interface":"front","average":{"1min":1.224,"5min":1.242,"15min":1.323},"min":{"1min":0.69499999999999995,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7030000000000001,"5min":2.0129999999999999,"15min":2.3140000000000001},"last":1.016}]},{"osd":6,"last update":"Mon Oct 16 10:07:35 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.353,"15min":1.4099999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.712,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.345,"15min":1.4239999999999999},"min":{"1min":0.73899999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.8089999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.429}]},{"osd":7,"last update":"Mon Oct 16 10:07:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3660000000000001,"15min":1.427},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8140000000000001,"5min":3.649,"15min":3.649},"last":1.264},{"interface":"front","average":{"1min":1.446,"5min":1.47,"15min":1.5429999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.907,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":1.2949999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575078,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59224,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312616,"statfs":{"total":96636764160,"available":96576118784,"internally_reserved":0,"allocated":1712128,"data_stored":819568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.89300000000000002,"15min":0.877},"min":{"1min":0.49299999999999999,"5min":0.35299999999999998,"15min":0.26500000000000001},"max":{"1min":1.236,"5min":5.1829999999999998,"15min":5.1829999999999998},"last":1.1930000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.94499999999999995,"15min":1.0109999999999999},"min":{"1min":0.51800000000000002,"5min":0.35999999999999999,"15min":0.34999999999999998},"max":{"1min":1.3520000000000001,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.82199999999999995}]},{"osd":1,"last update":"Mon Oct 16 10:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.0289999999999999,"15min":1.0389999999999999},"min":{"1min":0.5,"5min":0.438,"15min":0.438},"max":{"1min":1.5029999999999999,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":1.407},{"interface":"front","average":{"1min":1.05,"5min":1.002,"15min":1.0920000000000001},"min":{"1min":0.58299999999999996,"5min":0.38300000000000001,"15min":0.35499999999999998},"max":{"1min":1.395,"5min":1.4770000000000001,"15min":1.792},"last":1.272}]},{"osd":3,"last update":"Mon Oct 16 10:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.93700000000000006,"15min":0.998},"min":{"1min":0.54700000000000004,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.506,"5min":1.776,"15min":1.776},"last":1.131},{"interface":"front","average":{"1min":1.095,"5min":1.1539999999999999,"15min":1.1870000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41899999999999998},"max":{"1min":1.6899999999999999,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":1.6910000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:07:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.323,"15min":1.337},"min":{"1min":0.89900000000000002,"5min":0.79900000000000004,"15min":0.40300000000000002},"max":{"1min":1.732,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":2.1669999999999998},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.284,"15min":1.298},"min":{"1min":0.873,"5min":0.71899999999999997,"15min":0.46000000000000002},"max":{"1min":1.5720000000000001,"5min":3.262,"15min":3.262},"last":1.4650000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:07:38 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.3939999999999999,"15min":1.3779999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.59399999999999997},"max":{"1min":2.1589999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.9039999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.335,"15min":1.391},"min":{"1min":0.879,"5min":0.72699999999999998,"15min":0.60999999999999999},"max":{"1min":1.8069999999999999,"5min":1.885,"15min":1.956},"last":1.823}]},{"osd":6,"last update":"Mon Oct 16 10:07:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.3,"15min":1.3320000000000001},"min":{"1min":0.80700000000000005,"5min":0.78600000000000003,"15min":0.54400000000000004},"max":{"1min":1.7809999999999999,"5min":3.169,"15min":3.169},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4219999999999999,"15min":1.4379999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":2.0840000000000001,"5min":3.8340000000000001,"15min":3.8340000000000001},"last":1.7809999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:07:38 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.423,"15min":1.464},"min":{"1min":0.94099999999999995,"5min":0.81200000000000006,"15min":0.67000000000000004},"max":{"1min":1.8799999999999999,"5min":2.0739999999999998,"15min":2.262},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.395,"15min":1.425},"min":{"1min":0.91000000000000003,"5min":0.84199999999999997,"15min":0.63400000000000001},"max":{"1min":1.7709999999999999,"5min":3.3780000000000001,"15min":3.3780000000000001},"last":1.948}]}]},{"osd":1,"up_from":13,"seq":55834575078,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55072,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316768,"statfs":{"total":96636764160,"available":96580370432,"internally_reserved":0,"allocated":1720320,"data_stored":819692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.77500000000000002,"15min":0.84399999999999997},"min":{"1min":0.34399999999999997,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":1.0309999999999999,"5min":1.4690000000000001,"15min":1.631},"last":0.502},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.92500000000000004,"15min":0.97899999999999998},"min":{"1min":0.38200000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.5509999999999999,"5min":1.6200000000000001,"15min":1.798},"last":0.51200000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.90100000000000002,"15min":0.96299999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2529999999999999,"5min":1.5069999999999999,"15min":1.673},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.95199999999999996,"5min":1.0640000000000001,"15min":1.1930000000000001},"min":{"1min":0.56000000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3919999999999999,"5min":1.609,"15min":1.7669999999999999},"last":0.42299999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.0009999999999999,"15min":1.097},"min":{"1min":0.51800000000000002,"5min":0.42999999999999999,"15min":0.42599999999999999},"max":{"1min":1.4550000000000001,"5min":1.798,"15min":1.798},"last":0.46800000000000003},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0229999999999999,"15min":1.0920000000000001},"min":{"1min":0.47599999999999998,"5min":0.44,"15min":0.40699999999999997},"max":{"1min":1.5489999999999999,"5min":1.6879999999999999,"15min":1.8280000000000001},"last":0.54600000000000004}]},{"osd":4,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1599999999999999,"15min":1.212},"min":{"1min":0.69099999999999995,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5860000000000001,"5min":1.7450000000000001,"15min":1.885},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.27,"5min":1.2689999999999999,"15min":1.341},"min":{"1min":0.64700000000000002,"5min":0.63200000000000001,"15min":0.60999999999999999},"max":{"1min":1.605,"5min":1.8680000000000001,"15min":2.1320000000000001},"last":0.89000000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2589999999999999,"15min":1.3180000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.232,"5min":1.238,"15min":1.302},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.637,"5min":1.7290000000000001,"15min":2.0470000000000002},"last":0.94599999999999995}]},{"osd":6,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.3169999999999999,"15min":1.381},"min":{"1min":0.72099999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.3089999999999999,"15min":1.369},"min":{"1min":0.68600000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.762,"5min":1.8779999999999999,"15min":2.1680000000000001},"last":1.105}]},{"osd":7,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.331,"15min":1.4330000000000001},"min":{"1min":0.73799999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.6950000000000001,"5min":2.0739999999999998,"15min":2.1699999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4319999999999999,"15min":1.484},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.0389999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575079,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47332,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324508,"statfs":{"total":96636764160,"available":96588296192,"internally_reserved":0,"allocated":2314240,"data_stored":1405332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.80800000000000005,"15min":0.84799999999999998},"min":{"1min":0.35599999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.532,"5min":1.7589999999999999,"15min":2.766},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.92900000000000005,"15min":1.0129999999999999},"min":{"1min":0.51600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.228,"5min":1.581,"15min":3.96},"last":0.60099999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.97299999999999998,"15min":1.028},"min":{"1min":0.54200000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.278,"5min":2.0449999999999999,"15min":2.9260000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.98999999999999999,"15min":1.0660000000000001},"min":{"1min":0.47899999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.52,"5min":1.7789999999999999,"15min":4.4290000000000003},"last":0.75800000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.91100000000000003,"15min":1.012},"min":{"1min":0.5,"5min":0.41999999999999998,"15min":0.38},"max":{"1min":1.585,"5min":1.5920000000000001,"15min":4.7380000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.103,"15min":1.1859999999999999},"min":{"1min":0.51200000000000001,"5min":0.46800000000000003,"15min":0.44},"max":{"1min":1.3959999999999999,"5min":2.1859999999999999,"15min":5.1500000000000004},"last":0.63900000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2609999999999999,"15min":1.3200000000000001},"min":{"1min":0.83199999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5760000000000001,"5min":1.859,"15min":4.1970000000000001},"last":0.995},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1910000000000001,"15min":1.276},"min":{"1min":0.83899999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.3999999999999999,"5min":1.74,"15min":4.9630000000000001},"last":1.175}]},{"osd":5,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.202,"15min":1.296},"min":{"1min":0.877,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6799999999999999,"5min":1.6890000000000001,"15min":4.8600000000000003},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3300000000000001,"15min":1.385},"min":{"1min":0.76500000000000001,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6299999999999999,"5min":2.1269999999999998,"15min":4.6779999999999999},"last":1.202}]},{"osd":6,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.407,"15min":1.4330000000000001},"min":{"1min":0.93000000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.7390000000000001,"5min":2.1030000000000002,"15min":3.8250000000000002},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.254,"5min":1.339,"15min":1.4139999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.782,"5min":2.02,"15min":4.0979999999999999},"last":1.113}]},{"osd":7,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.4179999999999999,"15min":1.4710000000000001},"min":{"1min":0.79900000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.8600000000000001,"5min":2.0750000000000002,"15min":4.3369999999999997},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.3089999999999999,"15min":1.417},"min":{"1min":0.86299999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4650000000000001,"5min":1.9670000000000001,"15min":5.0570000000000004},"last":0.92800000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:08:31.215 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T10:08:31.216 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T10:08:31.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.473+0000 7ff51af67640 1 Processor -- start 2023-10-16T10:08:31.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.473+0000 7ff51af67640 1 -- start start 2023-10-16T10:08:31.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.474+0000 7ff51af67640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff514159ac0 0x7ff51415e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:31.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.474+0000 7ff51af67640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff5140704e0 0x7ff514070970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:31.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.474+0000 7ff51af67640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff51415e5c0 con 0x7ff5140cc270 2023-10-16T10:08:31.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.474+0000 7ff51af67640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff514070eb0 con 0x7ff5140704e0 2023-10-16T10:08:31.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.474+0000 7ff51af67640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51406b810 con 0x7ff514159ac0 2023-10-16T10:08:31.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.475+0000 7ff519f65640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff514159ac0 0x7ff51415e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:31.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.475+0000 7ff51a766640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff5140704e0 0x7ff514070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:31.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.475+0000 7ff519764640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff5140cc270 0x7ff5140c8cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:33428/0 (socket says 172.21.15.84:33428) 2023-10-16T10:08:31.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.475+0000 7ff519764640 1 -- 172.21.15.84:0/1071512138 learned_addr learned my addr 172.21.15.84:0/1071512138 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:31.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.476+0000 7ff51a766640 1 -- 172.21.15.84:0/1071512138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff514159ac0 msgr2=0x7ff51415e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:31.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.476+0000 7ff51a766640 1 --2- 172.21.15.84:0/1071512138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff514159ac0 0x7ff51415e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.476+0000 7ff51a766640 1 -- 172.21.15.84:0/1071512138 >> v1:172.21.15.157:6789/0 conn(0x7ff5140cc270 legacy=0x7ff5140c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:31.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.476+0000 7ff51a766640 1 -- 172.21.15.84:0/1071512138 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5140cc810 con 0x7ff5140704e0 2023-10-16T10:08:31.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.478+0000 7ff51a766640 1 --2- 172.21.15.84:0/1071512138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5140704e0 0x7ff514070970 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009d90 tx=0x7ff50c02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dfe0e220cadd905 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:31.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.478+0000 7ff518f63640 1 -- 172.21.15.84:0/1071512138 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c02cc60 con 0x7ff5140704e0 2023-10-16T10:08:31.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.478+0000 7ff518f63640 1 -- 172.21.15.84:0/1071512138 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50c02cdc0 con 0x7ff5140704e0 2023-10-16T10:08:31.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.478+0000 7ff518f63640 1 -- 172.21.15.84:0/1071512138 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c034660 con 0x7ff5140704e0 2023-10-16T10:08:31.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.479+0000 7ff51af67640 1 -- 172.21.15.84:0/1071512138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5140704e0 msgr2=0x7ff514070970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:31.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.479+0000 7ff51af67640 1 --2- 172.21.15.84:0/1071512138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5140704e0 0x7ff514070970 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009d90 tx=0x7ff50c02beb0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.479+0000 7ff51af67640 1 -- 172.21.15.84:0/1071512138 shutdown_connections 2023-10-16T10:08:31.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.479+0000 7ff51af67640 1 --2- 172.21.15.84:0/1071512138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5140704e0 0x7ff514070970 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.479+0000 7ff51af67640 1 --2- 172.21.15.84:0/1071512138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff514159ac0 0x7ff51415e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.479+0000 7ff51af67640 1 -- 172.21.15.84:0/1071512138 >> 172.21.15.84:0/1071512138 conn(0x7ff5140ce6a0 msgr2=0x7ff5140ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:31.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.479+0000 7ff51af67640 1 -- 172.21.15.84:0/1071512138 shutdown_connections 2023-10-16T10:08:31.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.479+0000 7ff51af67640 1 -- 172.21.15.84:0/1071512138 wait complete. 2023-10-16T10:08:31.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.481+0000 7ff51af67640 1 Processor -- start 2023-10-16T10:08:31.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.481+0000 7ff51af67640 1 -- start start 2023-10-16T10:08:31.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.482+0000 7ff51af67640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5140704e0 0x7ff5140ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:31.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.482+0000 7ff51af67640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5140cc270 0x7ff5140c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:31.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.482+0000 7ff51af67640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff514159ac0 0x7ff514149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:31.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.482+0000 7ff51af67640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51415bbe0 con 0x7ff5140704e0 2023-10-16T10:08:31.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.482+0000 7ff51af67640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51415bdf0 con 0x7ff5140cc270 2023-10-16T10:08:31.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.482+0000 7ff51af67640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51415c0d0 con 0x7ff514159ac0 2023-10-16T10:08:31.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.482+0000 7ff519f65640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5140704e0 0x7ff5140ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:31.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.482+0000 7ff519f65640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5140704e0 0x7ff5140ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52070/0 (socket says 172.21.15.84:52070) 2023-10-16T10:08:31.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.482+0000 7ff519f65640 1 -- 172.21.15.84:0/1103320384 learned_addr learned my addr 172.21.15.84:0/1103320384 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:31.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.482+0000 7ff51a766640 1 --2- 172.21.15.84:0/1103320384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff514159ac0 0x7ff514149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:31.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.482+0000 7ff519f65640 1 -- 172.21.15.84:0/1103320384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff514159ac0 msgr2=0x7ff514149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:31.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.482+0000 7ff519f65640 1 --2- 172.21.15.84:0/1103320384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff514159ac0 0x7ff514149750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.482+0000 7ff519f65640 1 -- 172.21.15.84:0/1103320384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5140cc270 msgr2=0x7ff5140c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:31.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.482+0000 7ff519f65640 1 --2- 172.21.15.84:0/1103320384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5140cc270 0x7ff5140c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.482+0000 7ff519f65640 1 -- 172.21.15.84:0/1103320384 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c009a70 con 0x7ff5140704e0 2023-10-16T10:08:31.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.483+0000 7ff519f65640 1 --2- 172.21.15.84:0/1103320384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5140704e0 0x7ff5140ca2c0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7ff51000e870 tx=0x7ff51000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:31.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.483+0000 7ff5067fc640 1 -- 172.21.15.84:0/1103320384 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5100094f0 con 0x7ff5140704e0 2023-10-16T10:08:31.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.483+0000 7ff5067fc640 1 -- 172.21.15.84:0/1103320384 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51000c940 con 0x7ff5140704e0 2023-10-16T10:08:31.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.483+0000 7ff51af67640 1 -- 172.21.15.84:0/1103320384 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff51415c3b0 con 0x7ff5140704e0 2023-10-16T10:08:31.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.483+0000 7ff5067fc640 1 -- 172.21.15.84:0/1103320384 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51000ccf0 con 0x7ff5140704e0 2023-10-16T10:08:31.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.483+0000 7ff51af67640 1 -- 172.21.15.84:0/1103320384 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51415c970 con 0x7ff5140704e0 2023-10-16T10:08:31.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.484+0000 7ff51af67640 1 -- 172.21.15.84:0/1103320384 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51404e9b0 con 0x7ff5140704e0 2023-10-16T10:08:31.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.485+0000 7ff5067fc640 1 -- 172.21.15.84:0/1103320384 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff510013470 con 0x7ff5140704e0 2023-10-16T10:08:31.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.486+0000 7ff5067fc640 1 --2- 172.21.15.84:0/1103320384 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7ff4f806af90 0x7ff4f806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:31.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.486+0000 7ff5067fc640 1 -- 172.21.15.84:0/1103320384 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff5100898f0 con 0x7ff5140704e0 2023-10-16T10:08:31.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.487+0000 7ff519764640 1 --2- 172.21.15.84:0/1103320384 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7ff4f806af90 0x7ff4f806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:31.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.487+0000 7ff519764640 1 --2- 172.21.15.84:0/1103320384 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7ff4f806af90 0x7ff4f806d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff5140c5890 tx=0x7ff508007a40 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:31.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.490+0000 7ff5067fc640 1 -- 172.21.15.84:0/1103320384 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff5100547f0 con 0x7ff5140704e0 2023-10-16T10:08:31.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.751+0000 7ff51af67640 1 -- 172.21.15.84:0/1103320384 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff5140c78b0 con 0x7ff5140704e0 2023-10-16T10:08:31.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.757+0000 7ff5067fc640 1 -- 172.21.15.84:0/1103320384 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff5100210b0 con 0x7ff5140704e0 2023-10-16T10:08:31.759 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2023-10-16T10:08:31.759 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ== 2023-10-16T10:08:31.760 INFO:teuthology.orchestra.run.smithi084.stdout: caps mds = "allow" 2023-10-16T10:08:31.760 INFO:teuthology.orchestra.run.smithi084.stdout: caps mon = "allow r" 2023-10-16T10:08:31.760 INFO:teuthology.orchestra.run.smithi084.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T10:08:31.760 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-16T10:08:31.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.760+0000 7ff51af67640 1 -- 172.21.15.84:0/1103320384 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7ff4f806af90 msgr2=0x7ff4f806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:31.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.760+0000 7ff51af67640 1 --2- 172.21.15.84:0/1103320384 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7ff4f806af90 0x7ff4f806d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff5140c5890 tx=0x7ff508007a40 comp rx=0 tx=0).stop 2023-10-16T10:08:31.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.760+0000 7ff51af67640 1 -- 172.21.15.84:0/1103320384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5140704e0 msgr2=0x7ff5140ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:31.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.760+0000 7ff51af67640 1 --2- 172.21.15.84:0/1103320384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5140704e0 0x7ff5140ca2c0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7ff51000e870 tx=0x7ff51000ed30 comp rx=0 tx=0).stop 2023-10-16T10:08:31.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.760+0000 7ff51af67640 1 -- 172.21.15.84:0/1103320384 shutdown_connections 2023-10-16T10:08:31.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.760+0000 7ff51af67640 1 --2- 172.21.15.84:0/1103320384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff514159ac0 0x7ff514149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.760+0000 7ff51af67640 1 --2- 172.21.15.84:0/1103320384 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7ff4f806af90 0x7ff4f806d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.760+0000 7ff51af67640 1 --2- 172.21.15.84:0/1103320384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5140cc270 0x7ff5140c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.760+0000 7ff51af67640 1 --2- 172.21.15.84:0/1103320384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5140704e0 0x7ff5140ca2c0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.760+0000 7ff51af67640 1 -- 172.21.15.84:0/1103320384 >> 172.21.15.84:0/1103320384 conn(0x7ff5140ce6a0 msgr2=0x7ff5140ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:31.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.761+0000 7ff51af67640 1 -- 172.21.15.84:0/1103320384 shutdown_connections 2023-10-16T10:08:31.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.761+0000 7ff51af67640 1 -- 172.21.15.84:0/1103320384 wait complete. 2023-10-16T10:08:31.779 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:08:31.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.988+0000 7ff6dec9b640 1 Processor -- start 2023-10-16T10:08:31.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.988+0000 7ff6dec9b640 1 -- start start 2023-10-16T10:08:31.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.990+0000 7ff6dec9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff6d815ae80 0x7ff6d815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:31.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.990+0000 7ff6dec9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff6d80ce390 0x7ff6d8070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:31.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.990+0000 7ff6dec9b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6d815ed60 con 0x7ff6d815aaa0 2023-10-16T10:08:31.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.990+0000 7ff6dec9b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d806b810 con 0x7ff6d815ae80 2023-10-16T10:08:31.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.990+0000 7ff6dec9b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d80707e0 con 0x7ff6d80ce390 2023-10-16T10:08:31.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.990+0000 7ff6cffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff6d815ae80 0x7ff6d815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:31.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.990+0000 7ff6cffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff6d815ae80 0x7ff6d815e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52072/0 (socket says 172.21.15.84:52072) 2023-10-16T10:08:31.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.990+0000 7ff6cffff640 1 -- 172.21.15.84:0/2966505297 learned_addr learned my addr 172.21.15.84:0/2966505297 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:31.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.990+0000 7ff6dd238640 1 --2- 172.21.15.84:0/2966505297 >> v2:172.21.15.157:3300/0 conn(0x7ff6d80ce390 0x7ff6d8070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:31.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.991+0000 7ff6cffff640 1 -- 172.21.15.84:0/2966505297 >> v2:172.21.15.157:3300/0 conn(0x7ff6d80ce390 msgr2=0x7ff6d8070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:31.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.991+0000 7ff6cffff640 1 --2- 172.21.15.84:0/2966505297 >> v2:172.21.15.157:3300/0 conn(0x7ff6d80ce390 0x7ff6d8070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.991+0000 7ff6cffff640 1 -- 172.21.15.84:0/2966505297 >> v1:172.21.15.84:6789/0 conn(0x7ff6d815aaa0 legacy=0x7ff6d80cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:31.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.991+0000 7ff6cffff640 1 -- 172.21.15.84:0/2966505297 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d815ef90 con 0x7ff6d815ae80 2023-10-16T10:08:31.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.992+0000 7ff6cffff640 1 --2- 172.21.15.84:0/2966505297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6d815ae80 0x7ff6d815e820 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7ff6d80c2490 tx=0x7ff6c802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4983789586b6adb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:31.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.992+0000 7ff6cf7fe640 1 -- 172.21.15.84:0/2966505297 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8030860 con 0x7ff6d815ae80 2023-10-16T10:08:31.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.992+0000 7ff6cf7fe640 1 -- 172.21.15.84:0/2966505297 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c80309c0 con 0x7ff6d815ae80 2023-10-16T10:08:31.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.992+0000 7ff6cf7fe640 1 -- 172.21.15.84:0/2966505297 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8030d50 con 0x7ff6d815ae80 2023-10-16T10:08:31.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.992+0000 7ff6dec9b640 1 -- 172.21.15.84:0/2966505297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6d815ae80 msgr2=0x7ff6d815e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:31.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.992+0000 7ff6dec9b640 1 --2- 172.21.15.84:0/2966505297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6d815ae80 0x7ff6d815e820 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7ff6d80c2490 tx=0x7ff6c802b9a0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.993+0000 7ff6dec9b640 1 -- 172.21.15.84:0/2966505297 shutdown_connections 2023-10-16T10:08:31.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.993+0000 7ff6dec9b640 1 --2- 172.21.15.84:0/2966505297 >> v2:172.21.15.157:3300/0 conn(0x7ff6d80ce390 0x7ff6d8070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.993+0000 7ff6dec9b640 1 --2- 172.21.15.84:0/2966505297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6d815ae80 0x7ff6d815e820 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:31.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.993+0000 7ff6dec9b640 1 -- 172.21.15.84:0/2966505297 >> 172.21.15.84:0/2966505297 conn(0x7ff6d80cb410 msgr2=0x7ff6d80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:31.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.993+0000 7ff6dec9b640 1 -- 172.21.15.84:0/2966505297 shutdown_connections 2023-10-16T10:08:31.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.993+0000 7ff6dec9b640 1 -- 172.21.15.84:0/2966505297 wait complete. 2023-10-16T10:08:32.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.994+0000 7ff6dec9b640 1 Processor -- start 2023-10-16T10:08:32.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.994+0000 7ff6dec9b640 1 -- start start 2023-10-16T10:08:32.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.995+0000 7ff6dec9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6d80ce390 0x7ff6d80c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.995+0000 7ff6dec9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6d815aaa0 0x7ff6d80c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.995+0000 7ff6dec9b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6d815ae80 0x7ff6d81f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.995+0000 7ff6dec9b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d80c9160 con 0x7ff6d815aaa0 2023-10-16T10:08:32.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.995+0000 7ff6dec9b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d815d210 con 0x7ff6d80ce390 2023-10-16T10:08:32.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.995+0000 7ff6dec9b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d815d4f0 con 0x7ff6d815ae80 2023-10-16T10:08:32.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.996+0000 7ff6cffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6d815aaa0 0x7ff6d80c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:32.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.996+0000 7ff6cffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6d815aaa0 0x7ff6d80c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52080/0 (socket says 172.21.15.84:52080) 2023-10-16T10:08:32.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.996+0000 7ff6cffff640 1 -- 172.21.15.84:0/1124388502 learned_addr learned my addr 172.21.15.84:0/1124388502 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:32.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.996+0000 7ff6cffff640 1 -- 172.21.15.84:0/1124388502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6d815ae80 msgr2=0x7ff6d81f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:32.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.996+0000 7ff6cffff640 1 --2- 172.21.15.84:0/1124388502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6d815ae80 0x7ff6d81f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.996+0000 7ff6cffff640 1 -- 172.21.15.84:0/1124388502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6d80ce390 msgr2=0x7ff6d80c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:32.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.996+0000 7ff6cffff640 1 --2- 172.21.15.84:0/1124388502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6d80ce390 0x7ff6d80c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.996+0000 7ff6cffff640 1 -- 172.21.15.84:0/1124388502 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c80094d0 con 0x7ff6d815aaa0 2023-10-16T10:08:32.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.996+0000 7ff6cffff640 1 --2- 172.21.15.84:0/1124388502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6d815aaa0 0x7ff6d80c85e0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7ff6c802b9e0 tx=0x7ff6c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:32.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.996+0000 7ff6cd7fa640 1 -- 172.21.15.84:0/1124388502 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8030850 con 0x7ff6d815aaa0 2023-10-16T10:08:32.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.997+0000 7ff6cd7fa640 1 -- 172.21.15.84:0/1124388502 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c8039d90 con 0x7ff6d815aaa0 2023-10-16T10:08:32.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.997+0000 7ff6cd7fa640 1 -- 172.21.15.84:0/1124388502 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8043650 con 0x7ff6d815aaa0 2023-10-16T10:08:32.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.997+0000 7ff6dec9b640 1 -- 172.21.15.84:0/1124388502 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d815d7d0 con 0x7ff6d815aaa0 2023-10-16T10:08:32.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.997+0000 7ff6dec9b640 1 -- 172.21.15.84:0/1124388502 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d80c8b20 con 0x7ff6d815aaa0 2023-10-16T10:08:32.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.998+0000 7ff6dec9b640 1 -- 172.21.15.84:0/1124388502 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d815dcf0 con 0x7ff6d815aaa0 2023-10-16T10:08:32.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:31.999+0000 7ff6cd7fa640 1 -- 172.21.15.84:0/1124388502 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff6c802f3b0 con 0x7ff6d815aaa0 2023-10-16T10:08:32.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.000+0000 7ff6cd7fa640 1 --2- 172.21.15.84:0/1124388502 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7ff6c406af90 0x7ff6c406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.000+0000 7ff6cd7fa640 1 -- 172.21.15.84:0/1124388502 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff6c80abd10 con 0x7ff6d815aaa0 2023-10-16T10:08:32.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.000+0000 7ff6dca37640 1 --2- 172.21.15.84:0/1124388502 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7ff6c406af90 0x7ff6c406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:32.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.001+0000 7ff6dca37640 1 --2- 172.21.15.84:0/1124388502 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7ff6c406af90 0x7ff6c406d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40012e0 tx=0x7ff6d4007d30 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:32.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.004+0000 7ff6cd7fa640 1 -- 172.21.15.84:0/1124388502 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff6c8076ca0 con 0x7ff6d815aaa0 2023-10-16T10:08:32.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.216+0000 7ff6dec9b640 1 -- 172.21.15.84:0/1124388502 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6d804e160 con 0x7ff6d815aaa0 2023-10-16T10:08:32.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.217+0000 7ff6cd7fa640 1 -- 172.21.15.84:0/1124388502 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4918 (secure 0 0 0) 0x7ff6c8048090 con 0x7ff6d815aaa0 2023-10-16T10:08:32.218 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:32.219 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4118285132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":4118285132},{"type":"v1","addr":"172.21.15.84:6837","nonce":4118285132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3743702675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3743702675},{"type":"v1","addr":"172.21.15.157:6834","nonce":3743702675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2895418246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2895418246},{"type":"v1","addr":"172.21.15.157:6835","nonce":2895418246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:08:25.705299+0000","modified":"2023-10-16T10:08:28.762900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8574":{"gid":8574,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/791907708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":791907708},{"type":"v1","addr":"172.21.15.84:6836","nonce":791907708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-16T10:08:32.219 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 71 2023-10-16T10:08:32.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.219+0000 7ff6dec9b640 1 -- 172.21.15.84:0/1124388502 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7ff6c406af90 msgr2=0x7ff6c406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:32.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.219+0000 7ff6dec9b640 1 --2- 172.21.15.84:0/1124388502 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7ff6c406af90 0x7ff6c406d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40012e0 tx=0x7ff6d4007d30 comp rx=0 tx=0).stop 2023-10-16T10:08:32.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.219+0000 7ff6dec9b640 1 -- 172.21.15.84:0/1124388502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6d815aaa0 msgr2=0x7ff6d80c85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:32.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.219+0000 7ff6dec9b640 1 --2- 172.21.15.84:0/1124388502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6d815aaa0 0x7ff6d80c85e0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7ff6c802b9e0 tx=0x7ff6c8004730 comp rx=0 tx=0).stop 2023-10-16T10:08:32.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.219+0000 7ff6dec9b640 1 -- 172.21.15.84:0/1124388502 shutdown_connections 2023-10-16T10:08:32.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.219+0000 7ff6dec9b640 1 --2- 172.21.15.84:0/1124388502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6d815ae80 0x7ff6d81f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.219+0000 7ff6dec9b640 1 --2- 172.21.15.84:0/1124388502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6d815aaa0 0x7ff6d80c85e0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.219+0000 7ff6dec9b640 1 --2- 172.21.15.84:0/1124388502 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7ff6c406af90 0x7ff6c406d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.219+0000 7ff6dec9b640 1 --2- 172.21.15.84:0/1124388502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6d80ce390 0x7ff6d80c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.219+0000 7ff6dec9b640 1 -- 172.21.15.84:0/1124388502 >> 172.21.15.84:0/1124388502 conn(0x7ff6d80cb410 msgr2=0x7ff6d80b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:32.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.219+0000 7ff6dec9b640 1 -- 172.21.15.84:0/1124388502 shutdown_connections 2023-10-16T10:08:32.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.219+0000 7ff6dec9b640 1 -- 172.21.15.84:0/1124388502 wait complete. 2023-10-16T10:08:32.641 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T10:08:25.705299+0000', 'modified': '2023-10-16T10:08:28.762900+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8574}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8574': {'gid': 8574, 'name': 'a', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.84:6836/791907708', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6834', 'nonce': 791907708}, {'type': 'v1', 'addr': '172.21.15.84:6836', 'nonce': 791907708}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T10:08:32.642 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T10:08:32.642 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-16T10:08:32.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.867+0000 7fd91296c640 1 Processor -- start 2023-10-16T10:08:32.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.867+0000 7fd91296c640 1 -- start start 2023-10-16T10:08:32.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.869+0000 7fd91296c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd90c06ea40 0x7fd90c160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.869+0000 7fd91296c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd90c1618b0 con 0x7fd90c15ce00 2023-10-16T10:08:32.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.869+0000 7fd91296c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd90c161010 con 0x7fd90c06a4b0 2023-10-16T10:08:32.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.869+0000 7fd91296c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd90c0cdc60 con 0x7fd90c06ea40 2023-10-16T10:08:32.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.869+0000 7fd90bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd90c15ce00 0x7fd90c069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55458/0 (socket says 172.21.15.84:55458) 2023-10-16T10:08:32.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.869+0000 7fd90bfff640 1 -- 172.21.15.84:0/2639043615 learned_addr learned my addr 172.21.15.84:0/2639043615 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:32.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.869+0000 7fd90b7fe640 1 -- 172.21.15.84:0/2639043615 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166010179 0 0) 0x7fd90c1618b0 con 0x7fd90c15ce00 2023-10-16T10:08:32.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.870+0000 7fd90b7fe640 1 -- 172.21.15.84:0/2639043615 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8f8003520 con 0x7fd90c15ce00 2023-10-16T10:08:32.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.870+0000 7fd910f09640 1 --2- 172.21.15.84:0/2639043615 >> v2:172.21.15.157:3300/0 conn(0x7fd90c06ea40 0x7fd90c160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:32.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.870+0000 7fd90b7fe640 1 -- 172.21.15.84:0/2639043615 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2738819173 0 0) 0x7fd8f8003520 con 0x7fd90c15ce00 2023-10-16T10:08:32.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.870+0000 7fd90b7fe640 1 -- 172.21.15.84:0/2639043615 >> v2:172.21.15.157:3300/0 conn(0x7fd90c06ea40 msgr2=0x7fd90c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:32.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.870+0000 7fd90b7fe640 1 --2- 172.21.15.84:0/2639043615 >> v2:172.21.15.157:3300/0 conn(0x7fd90c06ea40 0x7fd90c160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.870+0000 7fd90b7fe640 1 -- 172.21.15.84:0/2639043615 >> v1:172.21.15.157:6789/0 conn(0x7fd90c06a4b0 legacy=0x7fd90c0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:32.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.870+0000 7fd90b7fe640 1 -- 172.21.15.84:0/2639043615 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c1612f0 con 0x7fd90c15ce00 2023-10-16T10:08:32.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.870+0000 7fd90b7fe640 1 -- 172.21.15.84:0/2639043615 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd8fc001670 con 0x7fd90c15ce00 2023-10-16T10:08:32.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.870+0000 7fd90b7fe640 1 -- 172.21.15.84:0/2639043615 >> v1:172.21.15.84:6789/0 conn(0x7fd90c15ce00 legacy=0x7fd90c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:32.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.870+0000 7fd90b7fe640 1 --2- 172.21.15.84:0/2639043615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f8003970 0x7fd8f8003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.870+0000 7fd90b7fe640 1 --2- 172.21.15.84:0/2639043615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8f80047c0 0x7fd8f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.870+0000 7fd90b7fe640 1 --2- 172.21.15.84:0/2639043615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f80052a0 0x7fd8f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.870+0000 7fd90b7fe640 1 -- 172.21.15.84:0/2639043615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c0cdc60 con 0x7fd8f80047c0 2023-10-16T10:08:32.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.870+0000 7fd90b7fe640 1 -- 172.21.15.84:0/2639043615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f800c150 con 0x7fd8f80052a0 2023-10-16T10:08:32.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.870+0000 7fd90b7fe640 1 -- 172.21.15.84:0/2639043615 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f800c310 con 0x7fd8f8003970 2023-10-16T10:08:32.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.870+0000 7fd9037fe640 1 --2- 172.21.15.84:0/2639043615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8f80047c0 0x7fd8f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:32.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.871+0000 7fd9037fe640 1 -- 172.21.15.84:0/2639043615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f8003970 msgr2=0x7fd8f8003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:32.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.871+0000 7fd90bfff640 1 --2- 172.21.15.84:0/2639043615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f8003970 0x7fd8f8003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:32.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.871+0000 7fd9037fe640 1 --2- 172.21.15.84:0/2639043615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8f8003970 0x7fd8f8003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.871+0000 7fd9037fe640 1 -- 172.21.15.84:0/2639043615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f80052a0 msgr2=0x7fd8f800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:32.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.871+0000 7fd9037fe640 1 --2- 172.21.15.84:0/2639043615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f80052a0 0x7fd8f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.871+0000 7fd9037fe640 1 -- 172.21.15.84:0/2639043615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f8003520 con 0x7fd8f80047c0 2023-10-16T10:08:32.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.871+0000 7fd910f09640 1 -- 172.21.15.84:0/2639043615 reap_dead start 2023-10-16T10:08:32.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.872+0000 7fd9037fe640 1 --2- 172.21.15.84:0/2639043615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8f80047c0 0x7fd8f8004c40 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4003d80 tx=0x7fd8f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765f49aa7a3f4c73 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:32.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.872+0000 7fd90b7fe640 1 -- 172.21.15.84:0/2639043615 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4032600 con 0x7fd8f80047c0 2023-10-16T10:08:32.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.872+0000 7fd90b7fe640 1 -- 172.21.15.84:0/2639043615 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f402e050 con 0x7fd8f80047c0 2023-10-16T10:08:32.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.872+0000 7fd90b7fe640 1 -- 172.21.15.84:0/2639043615 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f40329e0 con 0x7fd8f80047c0 2023-10-16T10:08:32.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.873+0000 7fd91296c640 1 -- 172.21.15.84:0/2639043615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8f80047c0 msgr2=0x7fd8f8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:32.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.873+0000 7fd91296c640 1 --2- 172.21.15.84:0/2639043615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8f80047c0 0x7fd8f8004c40 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4003d80 tx=0x7fd8f80057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.873+0000 7fd91296c640 1 -- 172.21.15.84:0/2639043615 shutdown_connections 2023-10-16T10:08:32.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.873+0000 7fd91296c640 1 --2- 172.21.15.84:0/2639043615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8f80047c0 0x7fd8f8004c40 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.873+0000 7fd91296c640 1 -- 172.21.15.84:0/2639043615 >> 172.21.15.84:0/2639043615 conn(0x7fd90c0cc460 msgr2=0x7fd90c0cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:32.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.873+0000 7fd91296c640 1 -- 172.21.15.84:0/2639043615 shutdown_connections 2023-10-16T10:08:32.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.873+0000 7fd91296c640 1 -- 172.21.15.84:0/2639043615 wait complete. 2023-10-16T10:08:32.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.875+0000 7fd91296c640 1 Processor -- start 2023-10-16T10:08:32.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.875+0000 7fd91296c640 1 -- start start 2023-10-16T10:08:32.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.876+0000 7fd91296c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f80047c0 0x7fd90c0ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.876+0000 7fd91296c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd90c0b4760 0x7fd90c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.876+0000 7fd91296c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd90c0b5120 0x7fd90c159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.876+0000 7fd91296c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c0c3010 con 0x7fd90c0b4760 2023-10-16T10:08:32.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.876+0000 7fd91296c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c0c53f0 con 0x7fd8f80047c0 2023-10-16T10:08:32.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.876+0000 7fd91296c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c0c56d0 con 0x7fd90c0b5120 2023-10-16T10:08:32.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.876+0000 7fd9037fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd90c0b4760 0x7fd90c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:32.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.876+0000 7fd9037fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd90c0b4760 0x7fd90c0b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52092/0 (socket says 172.21.15.84:52092) 2023-10-16T10:08:32.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.876+0000 7fd9037fe640 1 -- 172.21.15.84:0/1571967173 learned_addr learned my addr 172.21.15.84:0/1571967173 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:32.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.876+0000 7fd90bfff640 1 --2- 172.21.15.84:0/1571967173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f80047c0 0x7fd90c0ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:32.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.876+0000 7fd910f09640 1 --2- 172.21.15.84:0/1571967173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd90c0b5120 0x7fd90c159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:32.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.876+0000 7fd9037fe640 1 -- 172.21.15.84:0/1571967173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd90c0b5120 msgr2=0x7fd90c159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:32.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.876+0000 7fd9037fe640 1 --2- 172.21.15.84:0/1571967173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd90c0b5120 0x7fd90c159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.876+0000 7fd9037fe640 1 -- 172.21.15.84:0/1571967173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f80047c0 msgr2=0x7fd90c0ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:32.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.876+0000 7fd9037fe640 1 --2- 172.21.15.84:0/1571967173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f80047c0 0x7fd90c0ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.876+0000 7fd9037fe640 1 -- 172.21.15.84:0/1571967173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c0c59b0 con 0x7fd90c0b4760 2023-10-16T10:08:32.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.877+0000 7fd9037fe640 1 --2- 172.21.15.84:0/1571967173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd90c0b4760 0x7fd90c0b4be0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4035040 tx=0x7fd8f40036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:32.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.877+0000 7fd9097fa640 1 -- 172.21.15.84:0/1571967173 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f403c410 con 0x7fd90c0b4760 2023-10-16T10:08:32.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.877+0000 7fd9097fa640 1 -- 172.21.15.84:0/1571967173 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f403c570 con 0x7fd90c0b4760 2023-10-16T10:08:32.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.877+0000 7fd9097fa640 1 -- 172.21.15.84:0/1571967173 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f403c900 con 0x7fd90c0b4760 2023-10-16T10:08:32.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.877+0000 7fd91296c640 1 -- 172.21.15.84:0/1571967173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90c0c5c90 con 0x7fd90c0b4760 2023-10-16T10:08:32.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.877+0000 7fd91296c640 1 -- 172.21.15.84:0/1571967173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90c1465d0 con 0x7fd90c0b4760 2023-10-16T10:08:32.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.880+0000 7fd9097fa640 1 -- 172.21.15.84:0/1571967173 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd8f4039050 con 0x7fd90c0b4760 2023-10-16T10:08:32.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.881+0000 7fd9097fa640 1 --2- 172.21.15.84:0/1571967173 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fd8ec06af90 0x7fd8ec06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.881+0000 7fd9097fa640 1 -- 172.21.15.84:0/1571967173 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd8f40aad60 con 0x7fd90c0b4760 2023-10-16T10:08:32.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.881+0000 7fd90bfff640 1 --2- 172.21.15.84:0/1571967173 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fd8ec06af90 0x7fd8ec06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:32.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.882+0000 7fd9155de740 1 Processor -- start 2023-10-16T10:08:32.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.882+0000 7fd9155de740 1 -- start start 2023-10-16T10:08:32.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.882+0000 7fd90bfff640 1 --2- 172.21.15.84:0/1571967173 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fd8ec06af90 0x7fd8ec06d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0017a0 tx=0x7fd8fc009040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:32.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.883+0000 7fd9155de740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bd4ee914d0 0x55bd4eea52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.883+0000 7fd9155de740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 0x55bd4ee93d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.883+0000 7fd9155de740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bd4ee94250 0x55bd4ee946d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.883+0000 7fd9155de740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd4ed847a0 con 0x55bd4ee918b0 2023-10-16T10:08:32.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.883+0000 7fd9155de740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd4edd6740 con 0x55bd4ee914d0 2023-10-16T10:08:32.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.883+0000 7fd9155de740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd4edd3e30 con 0x55bd4ee94250 2023-10-16T10:08:32.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.883+0000 7fd9037fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 0x55bd4ee93d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:32.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.883+0000 7fd9037fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 0x55bd4ee93d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52096/0 (socket says 172.21.15.84:52096) 2023-10-16T10:08:32.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.883+0000 7fd9037fe640 1 -- 172.21.15.84:0/1360738482 learned_addr learned my addr 172.21.15.84:0/1360738482 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:32.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.884+0000 7fd9037fe640 1 -- 172.21.15.84:0/1360738482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bd4ee94250 msgr2=0x55bd4ee946d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:32.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.884+0000 7fd9037fe640 1 --2- 172.21.15.84:0/1360738482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bd4ee94250 0x55bd4ee946d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.884+0000 7fd9037fe640 1 -- 172.21.15.84:0/1360738482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bd4ee914d0 msgr2=0x55bd4eea52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:32.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.884+0000 7fd9037fe640 1 --2- 172.21.15.84:0/1360738482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bd4ee914d0 0x55bd4eea52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.884+0000 7fd9037fe640 1 -- 172.21.15.84:0/1360738482 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd4ed93910 con 0x55bd4ee918b0 2023-10-16T10:08:32.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.884+0000 7fd9037fe640 1 --2- 172.21.15.84:0/1360738482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 0x55bd4ee93d10 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40a8ee0 tx=0x7fd8f4075d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7806ffac4719f91 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:32.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.884+0000 7fd901ffb640 1 -- 172.21.15.84:0/1360738482 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4003980 con 0x55bd4ee918b0 2023-10-16T10:08:32.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.884+0000 7fd901ffb640 1 -- 172.21.15.84:0/1360738482 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f40a91c0 con 0x55bd4ee918b0 2023-10-16T10:08:32.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.884+0000 7fd901ffb640 1 -- 172.21.15.84:0/1360738482 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f40a9330 con 0x55bd4ee918b0 2023-10-16T10:08:32.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.885+0000 7fd9155de740 1 -- 172.21.15.84:0/1360738482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 msgr2=0x55bd4ee93d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:32.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.885+0000 7fd9155de740 1 --2- 172.21.15.84:0/1360738482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 0x55bd4ee93d10 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40a8ee0 tx=0x7fd8f4075d10 comp rx=0 tx=0).stop 2023-10-16T10:08:32.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.885+0000 7fd9155de740 1 -- 172.21.15.84:0/1360738482 shutdown_connections 2023-10-16T10:08:32.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.885+0000 7fd9155de740 1 --2- 172.21.15.84:0/1360738482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bd4ee94250 0x55bd4ee946d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.885+0000 7fd9155de740 1 --2- 172.21.15.84:0/1360738482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 0x55bd4ee93d10 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.885+0000 7fd9155de740 1 --2- 172.21.15.84:0/1360738482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bd4ee914d0 0x55bd4eea52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.885+0000 7fd9155de740 1 -- 172.21.15.84:0/1360738482 >> 172.21.15.84:0/1360738482 conn(0x55bd4ee66190 msgr2=0x55bd4eeaed10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:32.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.885+0000 7fd9155de740 1 -- 172.21.15.84:0/1360738482 shutdown_connections 2023-10-16T10:08:32.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.885+0000 7fd9155de740 1 -- 172.21.15.84:0/1360738482 wait complete. 2023-10-16T10:08:32.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.886+0000 7fd9155de740 1 -- start start 2023-10-16T10:08:32.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.886+0000 7fd9155de740 1 Processor -- start 2023-10-16T10:08:32.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.887+0000 7fd9155de740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 0x55bd4ee888e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.887+0000 7fd9155de740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bd4ee94250 0x55bd4eeb0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.887+0000 7fd9155de740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bd4ee67b80 0x55bd4ee68000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.887+0000 7fd9155de740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd4edcf720 con 0x55bd4ee918b0 2023-10-16T10:08:32.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.887+0000 7fd9155de740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd4ed6d970 con 0x55bd4ee67b80 2023-10-16T10:08:32.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.887+0000 7fd9155de740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd4ed6d610 con 0x55bd4ee94250 2023-10-16T10:08:32.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.887+0000 7fd9155de740 10 client.0 _ll_register_callbacks cb 0x55bd4ed8f9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:08:32.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.887+0000 7fd90bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 0x55bd4ee888e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:32.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.887+0000 7fd90bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 0x55bd4ee888e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52112/0 (socket says 172.21.15.84:52112) 2023-10-16T10:08:32.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.887+0000 7fd90bfff640 1 -- 172.21.15.84:0/3347362653 learned_addr learned my addr 172.21.15.84:0/3347362653 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:32.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.888+0000 7fd90bfff640 1 -- 172.21.15.84:0/3347362653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bd4ee94250 msgr2=0x55bd4eeb0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:32.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.888+0000 7fd90bfff640 1 --2- 172.21.15.84:0/3347362653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bd4ee94250 0x55bd4eeb0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.888+0000 7fd90bfff640 1 -- 172.21.15.84:0/3347362653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bd4ee67b80 msgr2=0x55bd4ee68000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:32.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.888+0000 7fd90bfff640 1 --2- 172.21.15.84:0/3347362653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bd4ee67b80 0x55bd4ee68000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.888+0000 7fd90bfff640 1 -- 172.21.15.84:0/3347362653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90c1465d0 con 0x55bd4ee918b0 2023-10-16T10:08:32.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.888+0000 7fd90bfff640 1 --2- 172.21.15.84:0/3347362653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 0x55bd4ee888e0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0156a0 tx=0x7fd8fc015aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab21f9ae5a9934ac server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:32.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.888+0000 7fd8eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T10:08:32.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.888+0000 7fd8eaffd640 1 -- 172.21.15.84:0/3347362653 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc012490 con 0x55bd4ee918b0 2023-10-16T10:08:32.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.888+0000 7fd8eaffd640 1 -- 172.21.15.84:0/3347362653 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd8fc023af0 con 0x55bd4ee918b0 2023-10-16T10:08:32.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.888+0000 7fd9155de740 1 -- 172.21.15.84:0/3347362653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd4edf6160 con 0x55bd4ee918b0 2023-10-16T10:08:32.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.889+0000 7fd8eaffd640 1 -- 172.21.15.84:0/3347362653 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8fc023cb0 con 0x55bd4ee918b0 2023-10-16T10:08:32.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.889+0000 7fd9155de740 10 client.8718 fetch_fsmap learned FSMap version 71 2023-10-16T10:08:32.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.889+0000 7fd9155de740 1 -- 172.21.15.84:0/3347362653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55bd4edcdda0 con 0x55bd4ee918b0 2023-10-16T10:08:32.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.889+0000 7fd8eaffd640 1 -- 172.21.15.84:0/3347362653 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bd4edf6160 con 0x55bd4ee918b0 2023-10-16T10:08:32.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.889+0000 7fd8eaffd640 1 -- 172.21.15.84:0/3347362653 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd8fc012070 con 0x55bd4ee918b0 2023-10-16T10:08:32.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.889+0000 7fd9155de740 10 client.8718 fetch_fsmap finished waiting for FSMap version 71 2023-10-16T10:08:32.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.889+0000 7fd9155de740 10 client.8718 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T10:08:32.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.889+0000 7fd9155de740 20 client.8718 populate_metadata read hostname 'smithi084' 2023-10-16T10:08:32.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.889+0000 7fd9155de740 1 --2- 172.21.15.84:0/3347362653 >> [v2:172.21.15.84:6834/791907708,v1:172.21.15.84:6836/791907708] conn(0x55bd4ee91cb0 0x55bd4ee92090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.889+0000 7fd9155de740 4 client.8718 mds_command: new command op to 8574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T10:08:32.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.889+0000 7fd9155de740 1 -- 172.21.15.84:0/3347362653 --> [v2:172.21.15.84:6834/791907708,v1:172.21.15.84:6836/791907708] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd4eda1cc0 con 0x55bd4ee91cb0 2023-10-16T10:08:32.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.889+0000 7fd9037fe640 1 --2- 172.21.15.84:0/3347362653 >> [v2:172.21.15.84:6834/791907708,v1:172.21.15.84:6836/791907708] conn(0x55bd4ee91cb0 0x55bd4ee92090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:32.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.890+0000 7fd9037fe640 1 --2- 172.21.15.84:0/3347362653 >> [v2:172.21.15.84:6834/791907708,v1:172.21.15.84:6836/791907708] conn(0x55bd4ee91cb0 0x55bd4ee92090 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=927e4241f4694ee3 server_cookie=2515565ca35e3de in_seq=0 out_seq=0 2023-10-16T10:08:32.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.890+0000 7fd8eaffd640 10 client.8718 ms_handle_connect on v2:172.21.15.84:6834/791907708 2023-10-16T10:08:32.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.891+0000 7fd8eaffd640 1 -- 172.21.15.84:0/3347362653 <== mds.0 v2:172.21.15.84:6834/791907708 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bd4eda1cc0 con 0x55bd4ee91cb0 2023-10-16T10:08:32.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.891+0000 7fd8eaffd640 10 client.8718 handle_command_reply: tid=0 2023-10-16T10:08:32.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.891+0000 7fd9155de740 1 client.8718 shutdown 2023-10-16T10:08:32.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.891+0000 7fd9155de740 1 -- 172.21.15.84:0/3347362653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 msgr2=0x55bd4ee888e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:32.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.891+0000 7fd9155de740 1 --2- 172.21.15.84:0/3347362653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 0x55bd4ee888e0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0156a0 tx=0x7fd8fc015aa0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.891+0000 7fd9155de740 1 -- 172.21.15.84:0/3347362653 shutdown_connections 2023-10-16T10:08:32.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.891+0000 7fd9155de740 1 --2- 172.21.15.84:0/3347362653 >> [v2:172.21.15.84:6834/791907708,v1:172.21.15.84:6836/791907708] conn(0x55bd4ee91cb0 0x55bd4ee92090 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.891+0000 7fd9155de740 1 --2- 172.21.15.84:0/3347362653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bd4ee67b80 0x55bd4ee68000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.891+0000 7fd9155de740 1 --2- 172.21.15.84:0/3347362653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bd4ee94250 0x55bd4eeb0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.891+0000 7fd8eaffd640 0 client.8718 ms_handle_reset on v2:172.21.15.84:6834/791907708 2023-10-16T10:08:32.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.891+0000 7fd9155de740 1 --2- 172.21.15.84:0/3347362653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 0x55bd4ee888e0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.891+0000 7fd9155de740 1 -- 172.21.15.84:0/3347362653 >> 172.21.15.84:0/3347362653 conn(0x55bd4ee914d0 msgr2=0x55bd4eeaec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:32.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.891+0000 7fd9155de740 1 -- 172.21.15.84:0/3347362653 shutdown_connections 2023-10-16T10:08:32.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.891+0000 7fd9155de740 1 -- 172.21.15.84:0/3347362653 wait complete. 2023-10-16T10:08:32.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.891+0000 7fd9155de740 20 client.8718 trim_cache size 0 max 16384 2023-10-16T10:08:32.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.908+0000 7fd9155de740 1 Processor -- start 2023-10-16T10:08:32.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.908+0000 7fd9155de740 1 -- start start 2023-10-16T10:08:32.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.909+0000 7fd9155de740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bd4ee67b80 0x55bd4ee6aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.909+0000 7fd9155de740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 0x55bd4ee96ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.909+0000 7fd9155de740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bd4ee94250 0x55bd4ee97530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.909+0000 7fd9155de740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd4eb33f20 con 0x55bd4ee918b0 2023-10-16T10:08:32.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.909+0000 7fd9155de740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd4ee982b0 con 0x55bd4ee67b80 2023-10-16T10:08:32.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.909+0000 7fd9155de740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd4ee92150 con 0x55bd4ee94250 2023-10-16T10:08:32.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.909+0000 7fd9037fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 0x55bd4ee96ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:32.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.909+0000 7fd9037fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 0x55bd4ee96ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52114/0 (socket says 172.21.15.84:52114) 2023-10-16T10:08:32.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.909+0000 7fd9037fe640 1 -- 172.21.15.84:0/1362468753 learned_addr learned my addr 172.21.15.84:0/1362468753 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:32.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.909+0000 7fd9037fe640 1 -- 172.21.15.84:0/1362468753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bd4ee94250 msgr2=0x55bd4ee97530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:32.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.909+0000 7fd9037fe640 1 --2- 172.21.15.84:0/1362468753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bd4ee94250 0x55bd4ee97530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.909+0000 7fd9037fe640 1 -- 172.21.15.84:0/1362468753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bd4ee67b80 msgr2=0x55bd4ee6aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:32.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.909+0000 7fd9037fe640 1 --2- 172.21.15.84:0/1362468753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bd4ee67b80 0x55bd4ee6aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.909+0000 7fd9037fe640 1 -- 172.21.15.84:0/1362468753 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd4ed6cb80 con 0x55bd4ee918b0 2023-10-16T10:08:32.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.909+0000 7fd9037fe640 1 --2- 172.21.15.84:0/1362468753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 0x55bd4ee96ff0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4075da0 tx=0x7fd8f407d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec76c16b1f6cdf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:32.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.910+0000 7fd9027fc640 1 -- 172.21.15.84:0/1362468753 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4003980 con 0x55bd4ee918b0 2023-10-16T10:08:32.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.910+0000 7fd9027fc640 1 -- 172.21.15.84:0/1362468753 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f4077c40 con 0x55bd4ee918b0 2023-10-16T10:08:32.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.910+0000 7fd9027fc640 1 -- 172.21.15.84:0/1362468753 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4075fb0 con 0x55bd4ee918b0 2023-10-16T10:08:32.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.910+0000 7fd9155de740 1 -- 172.21.15.84:0/1362468753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 msgr2=0x55bd4ee96ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:32.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.910+0000 7fd9155de740 1 --2- 172.21.15.84:0/1362468753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 0x55bd4ee96ff0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4075da0 tx=0x7fd8f407d000 comp rx=0 tx=0).stop 2023-10-16T10:08:32.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.910+0000 7fd9155de740 1 -- 172.21.15.84:0/1362468753 shutdown_connections 2023-10-16T10:08:32.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.910+0000 7fd9155de740 1 --2- 172.21.15.84:0/1362468753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bd4ee94250 0x55bd4ee97530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.910+0000 7fd9155de740 1 --2- 172.21.15.84:0/1362468753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee918b0 0x55bd4ee96ff0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.910+0000 7fd9155de740 1 --2- 172.21.15.84:0/1362468753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bd4ee67b80 0x55bd4ee6aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.910+0000 7fd9155de740 1 -- 172.21.15.84:0/1362468753 >> 172.21.15.84:0/1362468753 conn(0x55bd4ee914d0 msgr2=0x55bd4ee92470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:32.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.910+0000 7fd9155de740 1 -- 172.21.15.84:0/1362468753 shutdown_connections 2023-10-16T10:08:32.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.910+0000 7fd9155de740 1 -- 172.21.15.84:0/1362468753 wait complete. 2023-10-16T10:08:32.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.911+0000 7fd9155de740 1 -- start start 2023-10-16T10:08:32.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.911+0000 7fd9155de740 1 Processor -- start 2023-10-16T10:08:32.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.912+0000 7fd9155de740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee67b80 0x55bd4ef22730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.912+0000 7fd9155de740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bd4ee918b0 0x55bd4ef20240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.912+0000 7fd9155de740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bd4ee94250 0x55bd4ee879d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.912+0000 7fd9155de740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd4ef20b40 con 0x55bd4ee67b80 2023-10-16T10:08:32.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.912+0000 7fd9155de740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd4ee6a250 con 0x55bd4ee918b0 2023-10-16T10:08:32.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.912+0000 7fd9155de740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd4eeb17b0 con 0x55bd4ee94250 2023-10-16T10:08:32.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.912+0000 7fd9155de740 10 client.0 _ll_register_callbacks cb 0x55bd4ee94b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:08:32.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.912+0000 7fd90bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee67b80 0x55bd4ef22730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:32.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.913+0000 7fd90bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee67b80 0x55bd4ef22730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52126/0 (socket says 172.21.15.84:52126) 2023-10-16T10:08:32.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.913+0000 7fd90bfff640 1 -- 172.21.15.84:0/2002264277 learned_addr learned my addr 172.21.15.84:0/2002264277 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:32.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.913+0000 7fd90bfff640 1 -- 172.21.15.84:0/2002264277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bd4ee94250 msgr2=0x55bd4ee879d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:32.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.913+0000 7fd90bfff640 1 --2- 172.21.15.84:0/2002264277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bd4ee94250 0x55bd4ee879d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.913+0000 7fd90bfff640 1 -- 172.21.15.84:0/2002264277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bd4ee918b0 msgr2=0x55bd4ef20240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.913+0000 7fd90bfff640 1 --2- 172.21.15.84:0/2002264277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bd4ee918b0 0x55bd4ef20240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.913+0000 7fd90bfff640 1 -- 172.21.15.84:0/2002264277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd4ed93910 con 0x55bd4ee67b80 2023-10-16T10:08:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.913+0000 7fd90bfff640 1 --2- 172.21.15.84:0/2002264277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee67b80 0x55bd4ef22730 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc000f60 tx=0x7fd8fc00b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d4ad2b0e17ba8f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.913+0000 7fd8eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T10:08:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.913+0000 7fd8eaffd640 1 -- 172.21.15.84:0/2002264277 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc02b420 con 0x55bd4ee67b80 2023-10-16T10:08:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.914+0000 7fd8eaffd640 1 -- 172.21.15.84:0/2002264277 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd8fc030030 con 0x55bd4ee67b80 2023-10-16T10:08:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.914+0000 7fd9155de740 1 -- 172.21.15.84:0/2002264277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8f40a91c0 con 0x55bd4ee67b80 2023-10-16T10:08:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.914+0000 7fd8eaffd640 1 -- 172.21.15.84:0/2002264277 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8fc02b580 con 0x55bd4ee67b80 2023-10-16T10:08:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.914+0000 7fd9155de740 10 client.8724 fetch_fsmap learned FSMap version 71 2023-10-16T10:08:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.914+0000 7fd9155de740 1 -- 172.21.15.84:0/2002264277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55bd4ed92170 con 0x55bd4ee67b80 2023-10-16T10:08:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.914+0000 7fd8eaffd640 1 -- 172.21.15.84:0/2002264277 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f40a91c0 con 0x55bd4ee67b80 2023-10-16T10:08:32.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.914+0000 7fd8eaffd640 1 -- 172.21.15.84:0/2002264277 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd8fc012050 con 0x55bd4ee67b80 2023-10-16T10:08:32.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.914+0000 7fd9155de740 10 client.8724 fetch_fsmap finished waiting for FSMap version 71 2023-10-16T10:08:32.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.914+0000 7fd9155de740 10 client.8724 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T10:08:32.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.914+0000 7fd9155de740 20 client.8724 populate_metadata read hostname 'smithi084' 2023-10-16T10:08:32.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.914+0000 7fd9155de740 1 --2- 172.21.15.84:0/2002264277 >> [v2:172.21.15.84:6834/791907708,v1:172.21.15.84:6836/791907708] conn(0x55bd4eec96c0 0x55bd4eec9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.914+0000 7fd9155de740 4 client.8724 mds_command: new command op to 8574 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T10:08:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.914+0000 7fd9155de740 1 -- 172.21.15.84:0/2002264277 --> [v2:172.21.15.84:6834/791907708,v1:172.21.15.84:6836/791907708] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bd4edc8700 con 0x55bd4eec96c0 2023-10-16T10:08:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.914+0000 7fd9037fe640 1 --2- 172.21.15.84:0/2002264277 >> [v2:172.21.15.84:6834/791907708,v1:172.21.15.84:6836/791907708] conn(0x55bd4eec96c0 0x55bd4eec9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.915+0000 7fd9037fe640 1 --2- 172.21.15.84:0/2002264277 >> [v2:172.21.15.84:6834/791907708,v1:172.21.15.84:6836/791907708] conn(0x55bd4eec96c0 0x55bd4eec9aa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69b1c0fd83253670 server_cookie=a9fc3ee6b71cddae in_seq=0 out_seq=0 2023-10-16T10:08:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.915+0000 7fd8eaffd640 10 client.8724 ms_handle_connect on v2:172.21.15.84:6834/791907708 2023-10-16T10:08:32.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.915+0000 7fd8eaffd640 1 -- 172.21.15.84:0/2002264277 <== mds.0 v2:172.21.15.84:6834/791907708 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55bd4edc8700 con 0x55bd4eec96c0 2023-10-16T10:08:32.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.915+0000 7fd8eaffd640 10 client.8724 handle_command_reply: tid=0 2023-10-16T10:08:32.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.915+0000 7fd9155de740 1 client.8724 shutdown 2023-10-16T10:08:32.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.915+0000 7fd9155de740 1 -- 172.21.15.84:0/2002264277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee67b80 msgr2=0x55bd4ef22730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:32.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.915+0000 7fd9155de740 1 --2- 172.21.15.84:0/2002264277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee67b80 0x55bd4ef22730 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc000f60 tx=0x7fd8fc00b000 comp rx=0 tx=0).stop 2023-10-16T10:08:32.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.915+0000 7fd9155de740 1 -- 172.21.15.84:0/2002264277 shutdown_connections 2023-10-16T10:08:32.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.915+0000 7fd9155de740 1 --2- 172.21.15.84:0/2002264277 >> [v2:172.21.15.84:6834/791907708,v1:172.21.15.84:6836/791907708] conn(0x55bd4eec96c0 0x55bd4eec9aa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.915+0000 7fd9155de740 1 --2- 172.21.15.84:0/2002264277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55bd4ee94250 0x55bd4ee879d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.915+0000 7fd9155de740 1 --2- 172.21.15.84:0/2002264277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bd4ee918b0 0x55bd4ef20240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.915+0000 7fd9155de740 1 --2- 172.21.15.84:0/2002264277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd4ee67b80 0x55bd4ef22730 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.915+0000 7fd8eaffd640 0 client.8724 ms_handle_reset on v2:172.21.15.84:6834/791907708 2023-10-16T10:08:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.915+0000 7fd9155de740 1 -- 172.21.15.84:0/2002264277 >> 172.21.15.84:0/2002264277 conn(0x55bd4ee914d0 msgr2=0x55bd4ee92470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.916+0000 7fd9155de740 1 -- 172.21.15.84:0/2002264277 shutdown_connections 2023-10-16T10:08:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.916+0000 7fd9155de740 1 -- 172.21.15.84:0/2002264277 wait complete. 2023-10-16T10:08:32.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.916+0000 7fd9155de740 20 client.8724 trim_cache size 0 max 16384 2023-10-16T10:08:32.936 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T10:08:32.936 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "63fb4d68-f881-4694-b880-d10369105c0e", 2023-10-16T10:08:32.936 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-16T10:08:32.937 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 8574, 2023-10-16T10:08:32.937 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-16T10:08:32.937 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-16T10:08:32.937 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-16T10:08:32.937 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 6.6845438909999997, 2023-10-16T10:08:32.938 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 71, 2023-10-16T10:08:32.938 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 138, 2023-10-16T10:08:32.938 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 138, 2023-10-16T10:08:32.938 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 12.054878336 2023-10-16T10:08:32.939 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T10:08:32.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.916+0000 7fd91296c640 1 -- 172.21.15.84:0/1571967173 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fd8ec06af90 msgr2=0x7fd8ec06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:32.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.916+0000 7fd91296c640 1 --2- 172.21.15.84:0/1571967173 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fd8ec06af90 0x7fd8ec06d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0017a0 tx=0x7fd8fc009040 comp rx=0 tx=0).stop 2023-10-16T10:08:32.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.916+0000 7fd91296c640 1 -- 172.21.15.84:0/1571967173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd90c0b4760 msgr2=0x7fd90c0b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:32.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.916+0000 7fd91296c640 1 --2- 172.21.15.84:0/1571967173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd90c0b4760 0x7fd90c0b4be0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4035040 tx=0x7fd8f40036f0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.916+0000 7fd91296c640 1 -- 172.21.15.84:0/1571967173 shutdown_connections 2023-10-16T10:08:32.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.916+0000 7fd91296c640 1 --2- 172.21.15.84:0/1571967173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd90c0b5120 0x7fd90c159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.916+0000 7fd91296c640 1 --2- 172.21.15.84:0/1571967173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd90c0b4760 0x7fd90c0b4be0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.916+0000 7fd91296c640 1 --2- 172.21.15.84:0/1571967173 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fd8ec06af90 0x7fd8ec06d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.916+0000 7fd91296c640 1 --2- 172.21.15.84:0/1571967173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f80047c0 0x7fd90c0ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:32.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.916+0000 7fd91296c640 1 -- 172.21.15.84:0/1571967173 >> 172.21.15.84:0/1571967173 conn(0x7fd90c0cc460 msgr2=0x7fd90c0ce920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:32.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.917+0000 7fd91296c640 1 -- 172.21.15.84:0/1571967173 shutdown_connections 2023-10-16T10:08:32.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:32.917+0000 7fd91296c640 1 -- 172.21.15.84:0/1571967173 wait complete. 2023-10-16T10:08:32.943 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T10:08:33.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.152+0000 7f89658ec640 1 Processor -- start 2023-10-16T10:08:33.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.153+0000 7f89658ec640 1 -- start start 2023-10-16T10:08:33.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.154+0000 7f89658ec640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8960160420 0x7f89600ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:33.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.154+0000 7f89658ec640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f896006e170 con 0x7f8960160800 2023-10-16T10:08:33.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.154+0000 7f89658ec640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89600cd460 con 0x7f896006b4c0 2023-10-16T10:08:33.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.154+0000 7f89658ec640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f896006e3c0 con 0x7f8960160420 2023-10-16T10:08:33.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.154+0000 7f895e7fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8960160800 0x7f896006adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55472/0 (socket says 172.21.15.84:55472) 2023-10-16T10:08:33.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.154+0000 7f895e7fc640 1 -- 172.21.15.84:0/2134641329 learned_addr learned my addr 172.21.15.84:0/2134641329 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:33.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.154+0000 7f895dffb640 1 -- 172.21.15.84:0/2134641329 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3735731988 0 0) 0x7f896006e170 con 0x7f8960160800 2023-10-16T10:08:33.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.154+0000 7f895dffb640 1 -- 172.21.15.84:0/2134641329 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f894c003520 con 0x7f8960160800 2023-10-16T10:08:33.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.155+0000 7f895effd640 1 --2- 172.21.15.84:0/2134641329 >> v2:172.21.15.157:3300/0 conn(0x7f8960160420 0x7f89600ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:33.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.155+0000 7f895dffb640 1 -- 172.21.15.84:0/2134641329 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1937889207 0 0) 0x7f894c003520 con 0x7f8960160800 2023-10-16T10:08:33.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.155+0000 7f895dffb640 1 -- 172.21.15.84:0/2134641329 >> v2:172.21.15.157:3300/0 conn(0x7f8960160420 msgr2=0x7f89600ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:33.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.155+0000 7f895dffb640 1 --2- 172.21.15.84:0/2134641329 >> v2:172.21.15.157:3300/0 conn(0x7f8960160420 0x7f89600ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:33.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.155+0000 7f895dffb640 1 -- 172.21.15.84:0/2134641329 >> v1:172.21.15.157:6789/0 conn(0x7f896006b4c0 legacy=0x7f8960070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:33.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.155+0000 7f895dffb640 1 -- 172.21.15.84:0/2134641329 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896006e6a0 con 0x7f8960160800 2023-10-16T10:08:33.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.155+0000 7f895dffb640 1 -- 172.21.15.84:0/2134641329 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f8948001670 con 0x7f8960160800 2023-10-16T10:08:33.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.155+0000 7f895dffb640 1 -- 172.21.15.84:0/2134641329 >> v1:172.21.15.84:6789/0 conn(0x7f8960160800 legacy=0x7f896006adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:33.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.155+0000 7f895dffb640 1 --2- 172.21.15.84:0/2134641329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f894c0039c0 0x7f894c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:33.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.155+0000 7f895dffb640 1 --2- 172.21.15.84:0/2134641329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f894c0047c0 0x7f894c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:33.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.155+0000 7f895dffb640 1 --2- 172.21.15.84:0/2134641329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f894c0051d0 0x7f894c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:33.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.155+0000 7f895dffb640 1 -- 172.21.15.84:0/2134641329 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896006e3c0 con 0x7f894c0051d0 2023-10-16T10:08:33.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.155+0000 7f895dffb640 1 -- 172.21.15.84:0/2134641329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c00c120 con 0x7f894c0039c0 2023-10-16T10:08:33.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.155+0000 7f895dffb640 1 -- 172.21.15.84:0/2134641329 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894c00c2e0 con 0x7f894c0047c0 2023-10-16T10:08:33.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.155+0000 7f895f7fe640 1 --2- 172.21.15.84:0/2134641329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f894c0051d0 0x7f894c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:33.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.156+0000 7f895effd640 1 --2- 172.21.15.84:0/2134641329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f894c0039c0 0x7f894c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:33.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.156+0000 7f895e7fc640 1 --2- 172.21.15.84:0/2134641329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f894c0047c0 0x7f894c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:33.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.156+0000 7f895f7fe640 1 -- 172.21.15.84:0/2134641329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f894c0047c0 msgr2=0x7f894c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:33.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.156+0000 7f895f7fe640 1 --2- 172.21.15.84:0/2134641329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f894c0047c0 0x7f894c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:33.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.156+0000 7f895f7fe640 1 -- 172.21.15.84:0/2134641329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f894c0039c0 msgr2=0x7f894c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:33.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.156+0000 7f895f7fe640 1 --2- 172.21.15.84:0/2134641329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f894c0039c0 0x7f894c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:33.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.156+0000 7f895f7fe640 1 -- 172.21.15.84:0/2134641329 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c003520 con 0x7f894c0051d0 2023-10-16T10:08:33.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.157+0000 7f895f7fe640 1 -- 172.21.15.84:0/2134641329 reap_dead start 2023-10-16T10:08:33.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.157+0000 7f895f7fe640 1 --2- 172.21.15.84:0/2134641329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f894c0051d0 0x7f894c00b980 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f8958001720 tx=0x7f894c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691939fb49f7e79a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:33.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.157+0000 7f895dffb640 1 -- 172.21.15.84:0/2134641329 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89580052d0 con 0x7f894c0051d0 2023-10-16T10:08:33.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.157+0000 7f895dffb640 1 -- 172.21.15.84:0/2134641329 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895802e050 con 0x7f894c0051d0 2023-10-16T10:08:33.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.157+0000 7f895dffb640 1 -- 172.21.15.84:0/2134641329 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89580035f0 con 0x7f894c0051d0 2023-10-16T10:08:33.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.158+0000 7f89658ec640 1 -- 172.21.15.84:0/2134641329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f894c0051d0 msgr2=0x7f894c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:33.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.158+0000 7f89658ec640 1 --2- 172.21.15.84:0/2134641329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f894c0051d0 0x7f894c00b980 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f8958001720 tx=0x7f894c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:33.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.158+0000 7f89658ec640 1 -- 172.21.15.84:0/2134641329 shutdown_connections 2023-10-16T10:08:33.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.158+0000 7f89658ec640 1 --2- 172.21.15.84:0/2134641329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f894c0051d0 0x7f894c00b980 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:33.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.158+0000 7f89658ec640 1 -- 172.21.15.84:0/2134641329 >> 172.21.15.84:0/2134641329 conn(0x7f89600b0d30 msgr2=0x7f89600b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:33.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.158+0000 7f89658ec640 1 -- 172.21.15.84:0/2134641329 shutdown_connections 2023-10-16T10:08:33.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.158+0000 7f89658ec640 1 -- 172.21.15.84:0/2134641329 wait complete. 2023-10-16T10:08:33.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.160+0000 7f89658ec640 1 Processor -- start 2023-10-16T10:08:33.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.160+0000 7f89658ec640 1 -- start start 2023-10-16T10:08:33.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.161+0000 7f89658ec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f894c0051d0 0x7f89600b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:33.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.161+0000 7f89658ec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89600b47a0 0x7f89600b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:33.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.161+0000 7f89658ec640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89600b2530 0x7f89600c38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:33.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.161+0000 7f89658ec640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89600b29b0 con 0x7f894c0051d0 2023-10-16T10:08:33.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.161+0000 7f89658ec640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89600c4620 con 0x7f89600b2530 2023-10-16T10:08:33.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.161+0000 7f89658ec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89600c4900 con 0x7f89600b47a0 2023-10-16T10:08:33.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.161+0000 7f895effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f894c0051d0 0x7f89600b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:33.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.161+0000 7f895effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f894c0051d0 0x7f89600b4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52144/0 (socket says 172.21.15.84:52144) 2023-10-16T10:08:33.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.161+0000 7f895effd640 1 -- 172.21.15.84:0/1794864893 learned_addr learned my addr 172.21.15.84:0/1794864893 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:33.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.161+0000 7f895effd640 1 -- 172.21.15.84:0/1794864893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89600b47a0 msgr2=0x7f89600b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:33.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.161+0000 7f895effd640 1 --2- 172.21.15.84:0/1794864893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89600b47a0 0x7f89600b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:33.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.161+0000 7f895effd640 1 -- 172.21.15.84:0/1794864893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89600b2530 msgr2=0x7f89600c38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:33.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.161+0000 7f895effd640 1 --2- 172.21.15.84:0/1794864893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89600b2530 0x7f89600c38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:33.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.161+0000 7f895effd640 1 -- 172.21.15.84:0/1794864893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89600c4be0 con 0x7f894c0051d0 2023-10-16T10:08:33.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.161+0000 7f895effd640 1 --2- 172.21.15.84:0/1794864893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f894c0051d0 0x7f89600b4260 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f8950009cb0 tx=0x7f8950007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:33.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.162+0000 7f8957fff640 1 -- 172.21.15.84:0/1794864893 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950014610 con 0x7f894c0051d0 2023-10-16T10:08:33.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.162+0000 7f8957fff640 1 -- 172.21.15.84:0/1794864893 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895000f050 con 0x7f894c0051d0 2023-10-16T10:08:33.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.162+0000 7f8957fff640 1 -- 172.21.15.84:0/1794864893 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89500149d0 con 0x7f894c0051d0 2023-10-16T10:08:33.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.162+0000 7f89658ec640 1 -- 172.21.15.84:0/1794864893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89600c4ec0 con 0x7f894c0051d0 2023-10-16T10:08:33.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.162+0000 7f89658ec640 1 -- 172.21.15.84:0/1794864893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89600c3ff0 con 0x7f894c0051d0 2023-10-16T10:08:33.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.163+0000 7f89658ec640 1 -- 172.21.15.84:0/1794864893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8960071c90 con 0x7f894c0051d0 2023-10-16T10:08:33.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.164+0000 7f8957fff640 1 -- 172.21.15.84:0/1794864893 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8950014b30 con 0x7f894c0051d0 2023-10-16T10:08:33.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.165+0000 7f8957fff640 1 --2- 172.21.15.84:0/1794864893 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f894006af90 0x7f894006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:33.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.166+0000 7f8957fff640 1 -- 172.21.15.84:0/1794864893 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f895008a0e0 con 0x7f894c0051d0 2023-10-16T10:08:33.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.166+0000 7f895e7fc640 1 --2- 172.21.15.84:0/1794864893 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f894006af90 0x7f894006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:33.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.166+0000 7f895e7fc640 1 --2- 172.21.15.84:0/1794864893 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f894006af90 0x7f894006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f89600c6d30 tx=0x7f8948009040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:33.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.170+0000 7f8957fff640 1 -- 172.21.15.84:0/1794864893 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8950054fe0 con 0x7f894c0051d0 2023-10-16T10:08:33.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.405+0000 7f89658ec640 1 -- 172.21.15.84:0/1794864893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f896004e160 con 0x7f894c0051d0 2023-10-16T10:08:33.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.405+0000 7f8957fff640 1 -- 172.21.15.84:0/1794864893 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8950058600 con 0x7f894c0051d0 2023-10-16T10:08:33.407 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T10:08:33.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.408+0000 7f89658ec640 1 -- 172.21.15.84:0/1794864893 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f894006af90 msgr2=0x7f894006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:33.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.408+0000 7f89658ec640 1 --2- 172.21.15.84:0/1794864893 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f894006af90 0x7f894006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f89600c6d30 tx=0x7f8948009040 comp rx=0 tx=0).stop 2023-10-16T10:08:33.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.408+0000 7f89658ec640 1 -- 172.21.15.84:0/1794864893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f894c0051d0 msgr2=0x7f89600b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:33.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.408+0000 7f89658ec640 1 --2- 172.21.15.84:0/1794864893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f894c0051d0 0x7f89600b4260 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f8950009cb0 tx=0x7f8950007700 comp rx=0 tx=0).stop 2023-10-16T10:08:33.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.408+0000 7f89658ec640 1 -- 172.21.15.84:0/1794864893 shutdown_connections 2023-10-16T10:08:33.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.408+0000 7f89658ec640 1 --2- 172.21.15.84:0/1794864893 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f894006af90 0x7f894006d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:33.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.408+0000 7f89658ec640 1 --2- 172.21.15.84:0/1794864893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89600b2530 0x7f89600c38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:33.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.408+0000 7f89658ec640 1 --2- 172.21.15.84:0/1794864893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89600b47a0 0x7f89600b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:33.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.408+0000 7f89658ec640 1 --2- 172.21.15.84:0/1794864893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f894c0051d0 0x7f89600b4260 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:33.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.408+0000 7f89658ec640 1 -- 172.21.15.84:0/1794864893 >> 172.21.15.84:0/1794864893 conn(0x7f89600b0d30 msgr2=0x7f8960069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:33.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.408+0000 7f89658ec640 1 -- 172.21.15.84:0/1794864893 shutdown_connections 2023-10-16T10:08:33.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:33.408+0000 7f89658ec640 1 -- 172.21.15.84:0/1794864893 wait complete. 2023-10-16T10:08:33.429 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T10:08:33.429 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T10:08:33.430 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-16T10:08:33.430 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T10:08:33.430 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T10:08:33.431 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T10:08:33.431 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T10:08:33.431 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T10:08:33.432 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:08:33.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:33.432 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-16T10:08:33.485 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T10:08:33.486 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T10:08:33.486 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T10:08:33.486 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:08:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T10:08:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:08:33.487 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:08:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:08:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-16T10:08:33.488 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:08:33.488 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:08:33.488 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-16T10:08:33.488 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:08:33.488 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:08:33.489 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T10:08:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:08:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T10:08:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 41237sec preferred_lft 41237sec 2023-10-16T10:08:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-16T10:08:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:08:33.490 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T10:08:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:08:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T10:08:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:08:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::20a8:eaff:fe9d:5cea/64 scope link 2023-10-16T10:08:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:08:33.491 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:08:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T10:08:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1854:13ff:fe47:be92/64 scope link 2023-10-16T10:08:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:08:33.492 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:08:33.492 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether ee:d5:41:a3:a5:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T10:08:33.492 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ecd5:41ff:fea3:a570/64 scope link 2023-10-16T10:08:33.492 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:08:33.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:33.494 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T10:08:33.586 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-16T10:08:33.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:33.591 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-16T10:08:33.646 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-16T10:08:33.646 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T10:08:33.647 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T10:08:33.647 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T10:08:33.647 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T10:08:33.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:33.648 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T10:08:33.648 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T10:08:33.649 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T10:08:33.649 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T10:08:33.649 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T10:08:33.650 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T10:08:33.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:33.823 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T10:08:33.876 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T10:08:33.877 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T10:08:33.877 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:08:33.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:33.878 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-16T10:08:33.971 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:08:33.972 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T10:08:34.030 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:08:34.031 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:08:34.031 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:08:34.085 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-16T10:08:34.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:34.086 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:08:34.223 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:08:34.224 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T10:08:34.258 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:08:34.259 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T10:08:34.259 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T10:08:34.259 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with new device syntax: 0@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T10:08:34.259 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T10:08:34.262 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-16T10:08:34.262 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2023-10-16T10:08:34.262 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T10:08:34.262 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T10:08:34.263 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T10:08:34.266 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T10:08:34.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:34.267 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:08:34.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:34.348 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T10:08:34.405 INFO:teuthology.orchestra.run.smithi084.stdout:client5852 2023-10-16T10:08:34.406 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:08:34.406 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:08:34.458 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T10:08:34.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.664+0000 7f22e150f640 1 Processor -- start 2023-10-16T10:08:34.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.664+0000 7f22e150f640 1 -- start start 2023-10-16T10:08:34.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.666+0000 7f22e150f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22dc159ab0 0x7f22dc0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:34.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.666+0000 7f22e150f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f22dc159e90 0x7f22dc06fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:34.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.666+0000 7f22e150f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22dc0cd1f0 con 0x7f22dc06c140 2023-10-16T10:08:34.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.666+0000 7f22e150f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22dc15d890 con 0x7f22dc159e90 2023-10-16T10:08:34.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.666+0000 7f22e150f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc0cd4d0 con 0x7f22dc159ab0 2023-10-16T10:08:34.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.666+0000 7f22da7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f22dc159e90 0x7f22dc06fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:34.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.666+0000 7f22db7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f22dc06c140 0x7f22dc15dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55482/0 (socket says 172.21.15.84:55482) 2023-10-16T10:08:34.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.666+0000 7f22db7fe640 1 -- 172.21.15.84:0/3870296789 learned_addr learned my addr 172.21.15.84:0/3870296789 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:34.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.666+0000 7f22daffd640 1 --2- 172.21.15.84:0/3870296789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22dc159ab0 0x7f22dc0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:34.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.666+0000 7f22da7fc640 1 -- 172.21.15.84:0/3870296789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22dc159ab0 msgr2=0x7f22dc0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:34.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.666+0000 7f22da7fc640 1 --2- 172.21.15.84:0/3870296789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22dc159ab0 0x7f22dc0ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.666+0000 7f22da7fc640 1 -- 172.21.15.84:0/3870296789 >> v1:172.21.15.84:6789/0 conn(0x7f22dc06c140 legacy=0x7f22dc15dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.667+0000 7f22da7fc640 1 -- 172.21.15.84:0/3870296789 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc0cd7b0 con 0x7f22dc159e90 2023-10-16T10:08:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.667+0000 7f22d9ffb640 1 -- 172.21.15.84:0/3870296789 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859952764 0 0) 0x7f22dc0cd1f0 con 0x7f22dc06c140 2023-10-16T10:08:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.668+0000 7f22da7fc640 1 --2- 172.21.15.84:0/3870296789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22dc159e90 0x7f22dc06fc50 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009dc0 tx=0x7f22cc02bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494f4059bed7b82a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:34.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.668+0000 7f22d9ffb640 1 -- 172.21.15.84:0/3870296789 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc02cb20 con 0x7f22dc159e90 2023-10-16T10:08:34.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.668+0000 7f22d9ffb640 1 -- 172.21.15.84:0/3870296789 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22cc02cc80 con 0x7f22dc159e90 2023-10-16T10:08:34.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.668+0000 7f22d9ffb640 1 -- 172.21.15.84:0/3870296789 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc034440 con 0x7f22dc159e90 2023-10-16T10:08:34.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.669+0000 7f22e150f640 1 -- 172.21.15.84:0/3870296789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22dc159e90 msgr2=0x7f22dc06fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:34.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.669+0000 7f22e150f640 1 --2- 172.21.15.84:0/3870296789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22dc159e90 0x7f22dc06fc50 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009dc0 tx=0x7f22cc02bd70 comp rx=0 tx=0).stop 2023-10-16T10:08:34.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.669+0000 7f22e150f640 1 -- 172.21.15.84:0/3870296789 shutdown_connections 2023-10-16T10:08:34.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.669+0000 7f22e150f640 1 --2- 172.21.15.84:0/3870296789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22dc159e90 0x7f22dc06fc50 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:34.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.669+0000 7f22e150f640 1 --2- 172.21.15.84:0/3870296789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22dc159ab0 0x7f22dc0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:34.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.669+0000 7f22e150f640 1 -- 172.21.15.84:0/3870296789 >> 172.21.15.84:0/3870296789 conn(0x7f22dc0ce9b0 msgr2=0x7f22dc0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:34.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.669+0000 7f22e150f640 1 -- 172.21.15.84:0/3870296789 shutdown_connections 2023-10-16T10:08:34.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.669+0000 7f22e150f640 1 -- 172.21.15.84:0/3870296789 wait complete. 2023-10-16T10:08:34.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.670+0000 7f22e150f640 1 Processor -- start 2023-10-16T10:08:34.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.671+0000 7f22e150f640 1 -- start start 2023-10-16T10:08:34.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.671+0000 7f22e150f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22dc06c140 0x7f22dc167850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:34.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.671+0000 7f22e150f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22dc159ab0 0x7f22dc14f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:34.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.672+0000 7f22e150f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22dc159e90 0x7f22dc06c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:34.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.672+0000 7f22e150f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc15c6e0 con 0x7f22dc159e90 2023-10-16T10:08:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.672+0000 7f22e150f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc15a4d0 con 0x7f22dc159ab0 2023-10-16T10:08:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.672+0000 7f22e150f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc15a7b0 con 0x7f22dc06c140 2023-10-16T10:08:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.672+0000 7f22db7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22dc159e90 0x7f22dc06c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.672+0000 7f22db7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22dc159e90 0x7f22dc06c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52164/0 (socket says 172.21.15.84:52164) 2023-10-16T10:08:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.672+0000 7f22db7fe640 1 -- 172.21.15.84:0/376848309 learned_addr learned my addr 172.21.15.84:0/376848309 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:34.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.672+0000 7f22daffd640 1 --2- 172.21.15.84:0/376848309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22dc06c140 0x7f22dc167850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:34.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.672+0000 7f22db7fe640 1 -- 172.21.15.84:0/376848309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22dc06c140 msgr2=0x7f22dc167850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:34.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.672+0000 7f22db7fe640 1 --2- 172.21.15.84:0/376848309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22dc06c140 0x7f22dc167850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:34.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.672+0000 7f22db7fe640 1 -- 172.21.15.84:0/376848309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22dc159ab0 msgr2=0x7f22dc14f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:34.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.672+0000 7f22db7fe640 1 --2- 172.21.15.84:0/376848309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22dc159ab0 0x7f22dc14f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:34.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.672+0000 7f22db7fe640 1 -- 172.21.15.84:0/376848309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc009a70 con 0x7f22dc159e90 2023-10-16T10:08:34.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.673+0000 7f22db7fe640 1 --2- 172.21.15.84:0/376848309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22dc159e90 0x7f22dc06c5b0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f22d4001eb0 tx=0x7f22d400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:34.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.673+0000 7f22d37fe640 1 -- 172.21.15.84:0/376848309 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40074e0 con 0x7f22dc159e90 2023-10-16T10:08:34.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.673+0000 7f22d37fe640 1 -- 172.21.15.84:0/376848309 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d4007640 con 0x7f22dc159e90 2023-10-16T10:08:34.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.673+0000 7f22e150f640 1 -- 172.21.15.84:0/376848309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22dc15aa90 con 0x7f22dc159e90 2023-10-16T10:08:34.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.673+0000 7f22d37fe640 1 -- 172.21.15.84:0/376848309 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4004a00 con 0x7f22dc159e90 2023-10-16T10:08:34.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.673+0000 7f22e150f640 1 -- 172.21.15.84:0/376848309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22dc15b050 con 0x7f22dc159e90 2023-10-16T10:08:34.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.674+0000 7f22e150f640 1 -- 172.21.15.84:0/376848309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22dc15b240 con 0x7f22dc159e90 2023-10-16T10:08:34.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.675+0000 7f22d37fe640 1 -- 172.21.15.84:0/376848309 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f22d4004b60 con 0x7f22dc159e90 2023-10-16T10:08:34.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.676+0000 7f22d37fe640 1 --2- 172.21.15.84:0/376848309 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f22bc06af90 0x7f22bc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:34.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.676+0000 7f22d37fe640 1 -- 172.21.15.84:0/376848309 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f22d4089650 con 0x7f22dc159e90 2023-10-16T10:08:34.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.676+0000 7f22daffd640 1 --2- 172.21.15.84:0/376848309 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f22bc06af90 0x7f22bc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:34.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.677+0000 7f22daffd640 1 --2- 172.21.15.84:0/376848309 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f22bc06af90 0x7f22bc06d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f22c4006fd0 tx=0x7f22c4008040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:34.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.680+0000 7f22d37fe640 1 -- 172.21.15.84:0/376848309 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f22d40545c0 con 0x7f22dc159e90 2023-10-16T10:08:34.842 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:08:34.851 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:08:34.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.895+0000 7f22e150f640 1 -- 172.21.15.84:0/376848309 --> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f22dc0adf70 con 0x7f22bc06af90 2023-10-16T10:08:34.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.897+0000 7f22d37fe640 1 -- 172.21.15.84:0/376848309 <== mgr.8322 v2:172.21.15.84:6832/2784642648 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f22c400c040 con 0x7f22bc06af90 2023-10-16T10:08:34.899 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T10:08:34.899 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:08:34.899 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephfs" 2023-10-16T10:08:34.900 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:08:34.900 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T10:08:34.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.900+0000 7f22e150f640 1 -- 172.21.15.84:0/376848309 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f22bc06af90 msgr2=0x7f22bc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:34.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.900+0000 7f22e150f640 1 --2- 172.21.15.84:0/376848309 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f22bc06af90 0x7f22bc06d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f22c4006fd0 tx=0x7f22c4008040 comp rx=0 tx=0).stop 2023-10-16T10:08:34.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.900+0000 7f22e150f640 1 -- 172.21.15.84:0/376848309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22dc159e90 msgr2=0x7f22dc06c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:34.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.900+0000 7f22e150f640 1 --2- 172.21.15.84:0/376848309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22dc159e90 0x7f22dc06c5b0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f22d4001eb0 tx=0x7f22d400c870 comp rx=0 tx=0).stop 2023-10-16T10:08:34.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.900+0000 7f22e150f640 1 -- 172.21.15.84:0/376848309 shutdown_connections 2023-10-16T10:08:34.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.900+0000 7f22e150f640 1 --2- 172.21.15.84:0/376848309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22dc159e90 0x7f22dc06c5b0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:34.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.900+0000 7f22e150f640 1 --2- 172.21.15.84:0/376848309 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f22bc06af90 0x7f22bc06d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:34.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.900+0000 7f22e150f640 1 --2- 172.21.15.84:0/376848309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22dc159ab0 0x7f22dc14f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:34.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.900+0000 7f22e150f640 1 --2- 172.21.15.84:0/376848309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22dc06c140 0x7f22dc167850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:34.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.900+0000 7f22e150f640 1 -- 172.21.15.84:0/376848309 >> 172.21.15.84:0/376848309 conn(0x7f22dc0ce9b0 msgr2=0x7f22dc0b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:34.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.900+0000 7f22e150f640 1 -- 172.21.15.84:0/376848309 shutdown_connections 2023-10-16T10:08:34.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:34.900+0000 7f22e150f640 1 -- 172.21.15.84:0/376848309 wait complete. 2023-10-16T10:08:34.920 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T10:08:35.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.123+0000 7fb1656a2640 1 Processor -- start 2023-10-16T10:08:35.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.124+0000 7fb1656a2640 1 -- start start 2023-10-16T10:08:35.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.125+0000 7fb1656a2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb1600addb0 0x7fb1600caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:35.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.125+0000 7fb1656a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1600c9000 0x7fb160069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:35.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.125+0000 7fb1656a2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1600cb140 con 0x7fb16006a2e0 2023-10-16T10:08:35.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.126+0000 7fb1656a2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb160161a50 con 0x7fb1600addb0 2023-10-16T10:08:35.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.126+0000 7fb1656a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb16006a750 con 0x7fb1600c9000 2023-10-16T10:08:35.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.126+0000 7fb15f7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb16006a2e0 0x7fb1600cc580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55492/0 (socket says 172.21.15.84:55492) 2023-10-16T10:08:35.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.126+0000 7fb15f7fe640 1 -- 172.21.15.84:0/2663181439 learned_addr learned my addr 172.21.15.84:0/2663181439 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:35.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.126+0000 7fb1567fc640 1 --2- 172.21.15.84:0/2663181439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1600c9000 0x7fb160069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:35.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.126+0000 7fb15effd640 1 --2- 172.21.15.84:0/2663181439 >> v2:172.21.15.157:3300/0 conn(0x7fb1600addb0 0x7fb1600caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:35.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.126+0000 7fb15e7fc640 1 -- 172.21.15.84:0/2663181439 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2066530077 0 0) 0x7fb1600cb140 con 0x7fb16006a2e0 2023-10-16T10:08:35.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.126+0000 7fb15e7fc640 1 -- 172.21.15.84:0/2663181439 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb14c003520 con 0x7fb16006a2e0 2023-10-16T10:08:35.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.127+0000 7fb15e7fc640 1 -- 172.21.15.84:0/2663181439 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 615254221 0 0) 0x7fb14c003520 con 0x7fb16006a2e0 2023-10-16T10:08:35.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.127+0000 7fb15e7fc640 1 -- 172.21.15.84:0/2663181439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1600c9000 msgr2=0x7fb160069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:35.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.127+0000 7fb15e7fc640 1 --2- 172.21.15.84:0/2663181439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1600c9000 0x7fb160069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:35.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.127+0000 7fb15e7fc640 1 -- 172.21.15.84:0/2663181439 >> v2:172.21.15.157:3300/0 conn(0x7fb1600addb0 msgr2=0x7fb1600caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:35.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.127+0000 7fb15e7fc640 1 --2- 172.21.15.84:0/2663181439 >> v2:172.21.15.157:3300/0 conn(0x7fb1600addb0 0x7fb1600caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:35.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.127+0000 7fb15e7fc640 1 -- 172.21.15.84:0/2663181439 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16006b030 con 0x7fb16006a2e0 2023-10-16T10:08:35.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.127+0000 7fb15e7fc640 1 -- 172.21.15.84:0/2663181439 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fb1580019f0 con 0x7fb16006a2e0 2023-10-16T10:08:35.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.127+0000 7fb15effd640 1 --2- 172.21.15.84:0/2663181439 >> v2:172.21.15.157:3300/0 conn(0x7fb1600addb0 0x7fb1600caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:08:35.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.127+0000 7fb15e7fc640 1 -- 172.21.15.84:0/2663181439 >> v1:172.21.15.84:6789/0 conn(0x7fb16006a2e0 legacy=0x7fb1600cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:35.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.127+0000 7fb15e7fc640 1 --2- 172.21.15.84:0/2663181439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1600c9000 0x7fb14c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:35.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.127+0000 7fb15e7fc640 1 --2- 172.21.15.84:0/2663181439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb14c003fd0 0x7fb14c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:35.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.127+0000 7fb15e7fc640 1 --2- 172.21.15.84:0/2663181439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb14c004d00 0x7fb14c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:35.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.127+0000 7fb15e7fc640 1 -- 172.21.15.84:0/2663181439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb160161a50 con 0x7fb14c004d00 2023-10-16T10:08:35.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.127+0000 7fb15e7fc640 1 -- 172.21.15.84:0/2663181439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16006a750 con 0x7fb14c003fd0 2023-10-16T10:08:35.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.127+0000 7fb15e7fc640 1 -- 172.21.15.84:0/2663181439 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c00bab0 con 0x7fb1600c9000 2023-10-16T10:08:35.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.127+0000 7fb15f7fe640 1 --2- 172.21.15.84:0/2663181439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb14c004d00 0x7fb14c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:35.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.128+0000 7fb1567fc640 1 --2- 172.21.15.84:0/2663181439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb14c003fd0 0x7fb14c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:35.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.128+0000 7fb15effd640 1 --2- 172.21.15.84:0/2663181439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1600c9000 0x7fb14c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:35.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.128+0000 7fb15f7fe640 1 -- 172.21.15.84:0/2663181439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1600c9000 msgr2=0x7fb14c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:35.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.128+0000 7fb15f7fe640 1 --2- 172.21.15.84:0/2663181439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1600c9000 0x7fb14c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:35.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.128+0000 7fb15f7fe640 1 -- 172.21.15.84:0/2663181439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb14c003fd0 msgr2=0x7fb14c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:35.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.128+0000 7fb15f7fe640 1 --2- 172.21.15.84:0/2663181439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb14c003fd0 0x7fb14c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:35.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.128+0000 7fb15f7fe640 1 -- 172.21.15.84:0/2663181439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c003520 con 0x7fb14c004d00 2023-10-16T10:08:35.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.128+0000 7fb15effd640 1 --2- 172.21.15.84:0/2663181439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1600c9000 0x7fb14c0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:35.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.129+0000 7fb15f7fe640 1 --2- 172.21.15.84:0/2663181439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb14c004d00 0x7fb14c005180 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fb15800a830 tx=0x7fb14c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e38d0306eff2f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:35.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.129+0000 7fb15e7fc640 1 -- 172.21.15.84:0/2663181439 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1580072d0 con 0x7fb14c004d00 2023-10-16T10:08:35.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.129+0000 7fb15e7fc640 1 -- 172.21.15.84:0/2663181439 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb15802f070 con 0x7fb14c004d00 2023-10-16T10:08:35.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.129+0000 7fb15e7fc640 1 -- 172.21.15.84:0/2663181439 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1580076f0 con 0x7fb14c004d00 2023-10-16T10:08:35.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.130+0000 7fb1656a2640 1 -- 172.21.15.84:0/2663181439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb14c004d00 msgr2=0x7fb14c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:35.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.130+0000 7fb1656a2640 1 --2- 172.21.15.84:0/2663181439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb14c004d00 0x7fb14c005180 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fb15800a830 tx=0x7fb14c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:35.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.130+0000 7fb15f7fe640 1 -- 172.21.15.84:0/2663181439 reap_dead start 2023-10-16T10:08:35.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.130+0000 7fb1656a2640 1 -- 172.21.15.84:0/2663181439 shutdown_connections 2023-10-16T10:08:35.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.130+0000 7fb1656a2640 1 -- 172.21.15.84:0/2663181439 >> 172.21.15.84:0/2663181439 conn(0x7fb1600ce9b0 msgr2=0x7fb1600cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:35.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.130+0000 7fb1656a2640 1 -- 172.21.15.84:0/2663181439 shutdown_connections 2023-10-16T10:08:35.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.130+0000 7fb1656a2640 1 -- 172.21.15.84:0/2663181439 wait complete. 2023-10-16T10:08:35.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.132+0000 7fb1656a2640 1 Processor -- start 2023-10-16T10:08:35.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.132+0000 7fb1656a2640 1 -- start start 2023-10-16T10:08:35.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.133+0000 7fb1656a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb14c004d00 0x7fb1600c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:35.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.133+0000 7fb1656a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1600c5250 0x7fb1600c2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:35.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.133+0000 7fb1656a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1600c2d80 0x7fb1600c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:35.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.133+0000 7fb1656a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb160161d70 con 0x7fb1600c2d80 2023-10-16T10:08:35.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.133+0000 7fb1656a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1600c4e10 con 0x7fb14c004d00 2023-10-16T10:08:35.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.133+0000 7fb1656a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb160162010 con 0x7fb1600c5250 2023-10-16T10:08:35.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.133+0000 7fb15f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1600c2d80 0x7fb1600c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:35.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.133+0000 7fb15f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1600c2d80 0x7fb1600c74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52180/0 (socket says 172.21.15.84:52180) 2023-10-16T10:08:35.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.133+0000 7fb15f7fe640 1 -- 172.21.15.84:0/3749066856 learned_addr learned my addr 172.21.15.84:0/3749066856 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:35.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.133+0000 7fb15effd640 1 --2- 172.21.15.84:0/3749066856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb14c004d00 0x7fb1600c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:35.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.133+0000 7fb1567fc640 1 --2- 172.21.15.84:0/3749066856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1600c5250 0x7fb1600c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:35.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.133+0000 7fb15f7fe640 1 -- 172.21.15.84:0/3749066856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1600c5250 msgr2=0x7fb1600c2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:35.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.133+0000 7fb15f7fe640 1 --2- 172.21.15.84:0/3749066856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1600c5250 0x7fb1600c2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:35.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.133+0000 7fb15f7fe640 1 -- 172.21.15.84:0/3749066856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb14c004d00 msgr2=0x7fb1600c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:35.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.133+0000 7fb15f7fe640 1 --2- 172.21.15.84:0/3749066856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb14c004d00 0x7fb1600c48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:35.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.133+0000 7fb15f7fe640 1 -- 172.21.15.84:0/3749066856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1601622f0 con 0x7fb1600c2d80 2023-10-16T10:08:35.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.134+0000 7fb15f7fe640 1 --2- 172.21.15.84:0/3749066856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1600c2d80 0x7fb1600c74c0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fb158001cb0 tx=0x7fb158001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:35.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.134+0000 7fb157fff640 1 -- 172.21.15.84:0/3749066856 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158004d60 con 0x7fb1600c2d80 2023-10-16T10:08:35.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.134+0000 7fb157fff640 1 -- 172.21.15.84:0/3749066856 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb158001250 con 0x7fb1600c2d80 2023-10-16T10:08:35.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.134+0000 7fb157fff640 1 -- 172.21.15.84:0/3749066856 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1580072f0 con 0x7fb1600c2d80 2023-10-16T10:08:35.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.134+0000 7fb1656a2640 1 -- 172.21.15.84:0/3749066856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1601625d0 con 0x7fb1600c2d80 2023-10-16T10:08:35.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.134+0000 7fb1656a2640 1 -- 172.21.15.84:0/3749066856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1600c56d0 con 0x7fb1600c2d80 2023-10-16T10:08:35.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.135+0000 7fb1656a2640 1 -- 172.21.15.84:0/3749066856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb16004e160 con 0x7fb1600c2d80 2023-10-16T10:08:35.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.136+0000 7fb157fff640 1 -- 172.21.15.84:0/3749066856 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 70) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb158031030 con 0x7fb1600c2d80 2023-10-16T10:08:35.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.138+0000 7fb157fff640 1 --2- 172.21.15.84:0/3749066856 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb14006af90 0x7fb14006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:35.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.138+0000 7fb157fff640 1 -- 172.21.15.84:0/3749066856 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb1580ab570 con 0x7fb1600c2d80 2023-10-16T10:08:35.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.138+0000 7fb15effd640 1 --2- 172.21.15.84:0/3749066856 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb14006af90 0x7fb14006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:35.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.139+0000 7fb15effd640 1 --2- 172.21.15.84:0/3749066856 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb14006af90 0x7fb14006d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb150006fd0 tx=0x7fb150008040 comp rx=0 tx=0).ready entity=mgr.8322 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:35.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.142+0000 7fb157fff640 1 -- 172.21.15.84:0/3749066856 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb158076340 con 0x7fb1600c2d80 2023-10-16T10:08:35.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:35.382+0000 7fb1656a2640 1 -- 172.21.15.84:0/3749066856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fb16004e160 con 0x7fb1600c2d80 2023-10-16T10:08:36.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.027+0000 7fb157fff640 1 -- 172.21.15.84:0/3749066856 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v71) v1 ==== 92+0+0 (secure 0 0 0) 0x7fb158079960 con 0x7fb1600c2d80 2023-10-16T10:08:36.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.031+0000 7fb1656a2640 1 -- 172.21.15.84:0/3749066856 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb14006af90 msgr2=0x7fb14006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:36.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.031+0000 7fb1656a2640 1 --2- 172.21.15.84:0/3749066856 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb14006af90 0x7fb14006d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb150006fd0 tx=0x7fb150008040 comp rx=0 tx=0).stop 2023-10-16T10:08:36.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.031+0000 7fb1656a2640 1 -- 172.21.15.84:0/3749066856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1600c2d80 msgr2=0x7fb1600c74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:36.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.031+0000 7fb1656a2640 1 --2- 172.21.15.84:0/3749066856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1600c2d80 0x7fb1600c74c0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fb158001cb0 tx=0x7fb158001d80 comp rx=0 tx=0).stop 2023-10-16T10:08:36.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.032+0000 7fb1656a2640 1 -- 172.21.15.84:0/3749066856 shutdown_connections 2023-10-16T10:08:36.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.032+0000 7fb1656a2640 1 --2- 172.21.15.84:0/3749066856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1600c2d80 0x7fb1600c74c0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.032+0000 7fb1656a2640 1 --2- 172.21.15.84:0/3749066856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1600c5250 0x7fb1600c2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.032+0000 7fb1656a2640 1 --2- 172.21.15.84:0/3749066856 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fb14006af90 0x7fb14006d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.032+0000 7fb1656a2640 1 --2- 172.21.15.84:0/3749066856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb14c004d00 0x7fb1600c48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.032+0000 7fb1656a2640 1 -- 172.21.15.84:0/3749066856 >> 172.21.15.84:0/3749066856 conn(0x7fb1600ce9b0 msgr2=0x7fb1600af1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:36.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.035+0000 7fb1656a2640 1 -- 172.21.15.84:0/3749066856 shutdown_connections 2023-10-16T10:08:36.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.035+0000 7fb1656a2640 1 -- 172.21.15.84:0/3749066856 wait complete. 2023-10-16T10:08:36.059 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T10:08:36.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.290+0000 7f14d2bd2640 1 Processor -- start 2023-10-16T10:08:36.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.291+0000 7f14d2bd2640 1 -- start start 2023-10-16T10:08:36.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.292+0000 7f14d2bd2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14cc160240 0x7f14cc0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:36.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.292+0000 7f14d2bd2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f14cc0b1ca0 0x7f14cc0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:36.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.292+0000 7f14d2bd2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14cc06afe0 con 0x7f14cc160620 2023-10-16T10:08:36.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.292+0000 7f14d2bd2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14cc0cf540 con 0x7f14cc0b1ca0 2023-10-16T10:08:36.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.292+0000 7f14d2bd2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc0cf820 con 0x7f14cc160240 2023-10-16T10:08:36.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.293+0000 7f14d23d1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f14cc0b1ca0 0x7f14cc0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:36.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.293+0000 7f14d1bd0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14cc160240 0x7f14cc0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:36.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.293+0000 7f14d23d1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f14cc0b1ca0 0x7f14cc0cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:59654/0 (socket says 172.21.15.84:59654) 2023-10-16T10:08:36.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.293+0000 7f14d1bd0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14cc160240 0x7f14cc0caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:47398/0 (socket says 172.21.15.84:47398) 2023-10-16T10:08:36.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.293+0000 7f14d23d1640 1 -- 172.21.15.84:0/366802704 learned_addr learned my addr 172.21.15.84:0/366802704 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:36.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.294+0000 7f14d23d1640 1 -- 172.21.15.84:0/366802704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14cc160240 msgr2=0x7f14cc0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:36.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.294+0000 7f14d23d1640 1 --2- 172.21.15.84:0/366802704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14cc160240 0x7f14cc0caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.294+0000 7f14d23d1640 1 -- 172.21.15.84:0/366802704 >> v1:172.21.15.157:6789/0 conn(0x7f14cc160620 legacy=0x7f14cc0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:36.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.294+0000 7f14d23d1640 1 -- 172.21.15.84:0/366802704 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14cc0cfb00 con 0x7f14cc0b1ca0 2023-10-16T10:08:36.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.295+0000 7f14d23d1640 1 --2- 172.21.15.84:0/366802704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14cc0b1ca0 0x7f14cc0cd220 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009e50 tx=0x7f14c402bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e1ebc960f64fe8f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:36.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.295+0000 7f14d0bce640 1 -- 172.21.15.84:0/366802704 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c402ca70 con 0x7f14cc0b1ca0 2023-10-16T10:08:36.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.295+0000 7f14d0bce640 1 -- 172.21.15.84:0/366802704 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c402cbd0 con 0x7f14cc0b1ca0 2023-10-16T10:08:36.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.295+0000 7f14d0bce640 1 -- 172.21.15.84:0/366802704 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c40343f0 con 0x7f14cc0b1ca0 2023-10-16T10:08:36.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.296+0000 7f14d2bd2640 1 -- 172.21.15.84:0/366802704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14cc0b1ca0 msgr2=0x7f14cc0cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:36.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.296+0000 7f14d2bd2640 1 --2- 172.21.15.84:0/366802704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14cc0b1ca0 0x7f14cc0cd220 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009e50 tx=0x7f14c402bed0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.296+0000 7f14d2bd2640 1 -- 172.21.15.84:0/366802704 shutdown_connections 2023-10-16T10:08:36.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.296+0000 7f14d2bd2640 1 --2- 172.21.15.84:0/366802704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14cc0b1ca0 0x7f14cc0cd220 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.296+0000 7f14d2bd2640 1 --2- 172.21.15.84:0/366802704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14cc160240 0x7f14cc0caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.296+0000 7f14d2bd2640 1 -- 172.21.15.84:0/366802704 >> 172.21.15.84:0/366802704 conn(0x7f14cc099a10 msgr2=0x7f14cc0d0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:36.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.296+0000 7f14d2bd2640 1 -- 172.21.15.84:0/366802704 shutdown_connections 2023-10-16T10:08:36.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.296+0000 7f14d2bd2640 1 -- 172.21.15.84:0/366802704 wait complete. 2023-10-16T10:08:36.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.298+0000 7f14d2bd2640 1 Processor -- start 2023-10-16T10:08:36.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.298+0000 7f14d2bd2640 1 -- start start 2023-10-16T10:08:36.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.299+0000 7f14d2bd2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14cc0b1ca0 0x7f14cc0c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:36.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.299+0000 7f14d2bd2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14cc160240 0x7f14cc0c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:36.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.299+0000 7f14d2bd2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14cc160620 0x7f14cc0b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:36.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.299+0000 7f14d2bd2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc0b1310 con 0x7f14cc160620 2023-10-16T10:08:36.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.299+0000 7f14d2bd2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc15d930 con 0x7f14cc160240 2023-10-16T10:08:36.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.299+0000 7f14d2bd2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc0ca5d0 con 0x7f14cc0b1ca0 2023-10-16T10:08:36.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.299+0000 7f14d23d1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14cc160620 0x7f14cc0b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:36.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.299+0000 7f14d23d1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14cc160620 0x7f14cc0b0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53048/0 (socket says 172.21.15.84:53048) 2023-10-16T10:08:36.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.299+0000 7f14d23d1640 1 -- 172.21.15.84:0/1250531448 learned_addr learned my addr 172.21.15.84:0/1250531448 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:36.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.300+0000 7f14d1bd0640 1 --2- 172.21.15.84:0/1250531448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14cc0b1ca0 0x7f14cc0c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:36.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.300+0000 7f14d13cf640 1 --2- 172.21.15.84:0/1250531448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14cc160240 0x7f14cc0c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:36.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.300+0000 7f14d23d1640 1 -- 172.21.15.84:0/1250531448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14cc0b1ca0 msgr2=0x7f14cc0c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:36.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.300+0000 7f14d23d1640 1 --2- 172.21.15.84:0/1250531448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14cc0b1ca0 0x7f14cc0c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.300+0000 7f14d23d1640 1 -- 172.21.15.84:0/1250531448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14cc160240 msgr2=0x7f14cc0c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:36.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.300+0000 7f14d23d1640 1 --2- 172.21.15.84:0/1250531448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14cc160240 0x7f14cc0c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.300+0000 7f14d23d1640 1 -- 172.21.15.84:0/1250531448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c4009b30 con 0x7f14cc160620 2023-10-16T10:08:36.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.300+0000 7f14d13cf640 1 --2- 172.21.15.84:0/1250531448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14cc160240 0x7f14cc0c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:36.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.300+0000 7f14d23d1640 1 --2- 172.21.15.84:0/1250531448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14cc160620 0x7f14cc0b0dd0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009e50 tx=0x7f14c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:36.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.300+0000 7f14c27fc640 1 -- 172.21.15.84:0/1250531448 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c4009510 con 0x7f14cc160620 2023-10-16T10:08:36.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.301+0000 7f14c27fc640 1 -- 172.21.15.84:0/1250531448 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c403e450 con 0x7f14cc160620 2023-10-16T10:08:36.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.301+0000 7f14c27fc640 1 -- 172.21.15.84:0/1250531448 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c403e800 con 0x7f14cc160620 2023-10-16T10:08:36.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.301+0000 7f14d2bd2640 1 -- 172.21.15.84:0/1250531448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14cc0c9940 con 0x7f14cc160620 2023-10-16T10:08:36.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.301+0000 7f14d2bd2640 1 -- 172.21.15.84:0/1250531448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14cc0c9380 con 0x7f14cc160620 2023-10-16T10:08:36.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.302+0000 7f14d2bd2640 1 -- 172.21.15.84:0/1250531448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14cc04e9b0 con 0x7f14cc160620 2023-10-16T10:08:36.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.303+0000 7f14c27fc640 1 -- 172.21.15.84:0/1250531448 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 71) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f14c403e960 con 0x7f14cc160620 2023-10-16T10:08:36.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.304+0000 7f14c27fc640 1 --2- 172.21.15.84:0/1250531448 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f14b006afe0 0x7f14b006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:36.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.304+0000 7f14d1bd0640 1 -- 172.21.15.84:0/1250531448 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f14b006afe0 msgr2=0x7f14b006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2784642648 2023-10-16T10:08:36.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.304+0000 7f14c27fc640 1 -- 172.21.15.84:0/1250531448 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f14c40ac4d0 con 0x7f14cc160620 2023-10-16T10:08:36.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.304+0000 7f14d1bd0640 1 --2- 172.21.15.84:0/1250531448 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f14b006afe0 0x7f14b006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:36.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.308+0000 7f14c27fc640 1 -- 172.21.15.84:0/1250531448 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f14c40774c0 con 0x7f14cc160620 2023-10-16T10:08:36.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.502+0000 7f14d2bd2640 1 -- 172.21.15.84:0/1250531448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f14cc072140 con 0x7f14cc160620 2023-10-16T10:08:36.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.505+0000 7f14d1bd0640 1 -- 172.21.15.84:0/1250531448 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f14b006afe0 msgr2=0x7f14b006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2784642648 2023-10-16T10:08:36.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.505+0000 7f14d1bd0640 1 --2- 172.21.15.84:0/1250531448 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f14b006afe0 0x7f14b006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:08:36.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.507+0000 7f14c27fc640 1 -- 172.21.15.84:0/1250531448 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v47)=0 v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7f14c407aae0 con 0x7f14cc160620 2023-10-16T10:08:36.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.510+0000 7f14d2bd2640 1 -- 172.21.15.84:0/1250531448 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f14b006afe0 msgr2=0x7f14b006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:36.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.510+0000 7f14d2bd2640 1 --2- 172.21.15.84:0/1250531448 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f14b006afe0 0x7f14b006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.510+0000 7f14d2bd2640 1 -- 172.21.15.84:0/1250531448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14cc160620 msgr2=0x7f14cc0b0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:36.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.510+0000 7f14d2bd2640 1 --2- 172.21.15.84:0/1250531448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14cc160620 0x7f14cc0b0dd0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009e50 tx=0x7f14c4004a40 comp rx=0 tx=0).stop 2023-10-16T10:08:36.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.510+0000 7f14d2bd2640 1 -- 172.21.15.84:0/1250531448 shutdown_connections 2023-10-16T10:08:36.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.510+0000 7f14d2bd2640 1 --2- 172.21.15.84:0/1250531448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14cc160620 0x7f14cc0b0dd0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.510+0000 7f14d2bd2640 1 --2- 172.21.15.84:0/1250531448 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f14b006afe0 0x7f14b006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.510+0000 7f14d2bd2640 1 --2- 172.21.15.84:0/1250531448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14cc160240 0x7f14cc0c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.510+0000 7f14d2bd2640 1 --2- 172.21.15.84:0/1250531448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14cc0b1ca0 0x7f14cc0c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.510+0000 7f14d2bd2640 1 -- 172.21.15.84:0/1250531448 >> 172.21.15.84:0/1250531448 conn(0x7f14cc099a10 msgr2=0x7f14cc06c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:36.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.510+0000 7f14d2bd2640 1 -- 172.21.15.84:0/1250531448 shutdown_connections 2023-10-16T10:08:36.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.510+0000 7f14d2bd2640 1 -- 172.21.15.84:0/1250531448 wait complete. 2023-10-16T10:08:36.528 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T10:08:36.542 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:36.541+0000 7eff92443040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:08:36.598 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:36.596+0000 7eff92443040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:08:36.606 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:36.604+0000 7fb2f7e32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:08:36.663 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:36.661+0000 7fb2f7e32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:08:36.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.734+0000 7f041b335640 1 Processor -- start 2023-10-16T10:08:36.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.735+0000 7f041b335640 1 -- start start 2023-10-16T10:08:36.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.736+0000 7f041b335640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f041415cd70 0x7f04140d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:36.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.736+0000 7f041b335640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f04140b8480 0x7f04140cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:36.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.736+0000 7f041b335640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f041415d150 con 0x7f04140d0be0 2023-10-16T10:08:36.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.736+0000 7f041b335640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04140d1180 con 0x7f041415cd70 2023-10-16T10:08:36.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.736+0000 7f041b335640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04140d1460 con 0x7f04140b8480 2023-10-16T10:08:36.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.736+0000 7f041a333640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f041415cd70 0x7f04140d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:36.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.736+0000 7f041a333640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f041415cd70 0x7f04140d2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53054/0 (socket says 172.21.15.84:53054) 2023-10-16T10:08:36.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.736+0000 7f041a333640 1 -- 172.21.15.84:0/307877918 learned_addr learned my addr 172.21.15.84:0/307877918 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:36.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.737+0000 7f0419b32640 1 --2- 172.21.15.84:0/307877918 >> v2:172.21.15.157:3300/0 conn(0x7f04140b8480 0x7f04140cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:36.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.737+0000 7f041a333640 1 -- 172.21.15.84:0/307877918 >> v2:172.21.15.157:3300/0 conn(0x7f04140b8480 msgr2=0x7f04140cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:36.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.737+0000 7f041a333640 1 --2- 172.21.15.84:0/307877918 >> v2:172.21.15.157:3300/0 conn(0x7f04140b8480 0x7f04140cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.737+0000 7f041a333640 1 -- 172.21.15.84:0/307877918 >> v1:172.21.15.157:6789/0 conn(0x7f04140d0be0 legacy=0x7f04140b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:36.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.737+0000 7f041a333640 1 -- 172.21.15.84:0/307877918 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04140d1740 con 0x7f041415cd70 2023-10-16T10:08:36.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.738+0000 7f041a333640 1 --2- 172.21.15.84:0/307877918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041415cd70 0x7f04140d2220 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f0414066a00 tx=0x7f041002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371d32615f393084 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:36.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.738+0000 7f0419331640 1 -- 172.21.15.84:0/307877918 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410031870 con 0x7f041415cd70 2023-10-16T10:08:36.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.738+0000 7f0419331640 1 -- 172.21.15.84:0/307877918 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04100319d0 con 0x7f041415cd70 2023-10-16T10:08:36.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.738+0000 7f0419331640 1 -- 172.21.15.84:0/307877918 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410031d60 con 0x7f041415cd70 2023-10-16T10:08:36.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.739+0000 7f041b335640 1 -- 172.21.15.84:0/307877918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041415cd70 msgr2=0x7f04140d2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:36.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.739+0000 7f041b335640 1 --2- 172.21.15.84:0/307877918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041415cd70 0x7f04140d2220 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f0414066a00 tx=0x7f041002b910 comp rx=0 tx=0).stop 2023-10-16T10:08:36.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.739+0000 7f041b335640 1 -- 172.21.15.84:0/307877918 shutdown_connections 2023-10-16T10:08:36.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.739+0000 7f041b335640 1 --2- 172.21.15.84:0/307877918 >> v2:172.21.15.157:3300/0 conn(0x7f04140b8480 0x7f04140cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.739+0000 7f041b335640 1 --2- 172.21.15.84:0/307877918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041415cd70 0x7f04140d2220 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.739+0000 7f041b335640 1 -- 172.21.15.84:0/307877918 >> 172.21.15.84:0/307877918 conn(0x7f04140cf570 msgr2=0x7f04140cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:36.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.739+0000 7f041b335640 1 -- 172.21.15.84:0/307877918 shutdown_connections 2023-10-16T10:08:36.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.739+0000 7f041b335640 1 -- 172.21.15.84:0/307877918 wait complete. 2023-10-16T10:08:36.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.741+0000 7f041b335640 1 Processor -- start 2023-10-16T10:08:36.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.741+0000 7f041b335640 1 -- start start 2023-10-16T10:08:36.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.742+0000 7f041b335640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04140b8480 0x7f04140cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:36.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.742+0000 7f041b335640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04140d0be0 0x7f041415d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:36.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.742+0000 7f041b335640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041415cd70 0x7f041415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:36.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.742+0000 7f041b335640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04140cbfa0 con 0x7f04140d0be0 2023-10-16T10:08:36.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.742+0000 7f041b335640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0414154d50 con 0x7f041415cd70 2023-10-16T10:08:36.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.742+0000 7f041b335640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0414155030 con 0x7f04140b8480 2023-10-16T10:08:36.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.742+0000 7f0419b32640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04140d0be0 0x7f041415d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:36.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.742+0000 7f0419b32640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04140d0be0 0x7f041415d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53062/0 (socket says 172.21.15.84:53062) 2023-10-16T10:08:36.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.742+0000 7f0419b32640 1 -- 172.21.15.84:0/4066247599 learned_addr learned my addr 172.21.15.84:0/4066247599 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:36.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.742+0000 7f041ab34640 1 --2- 172.21.15.84:0/4066247599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041415cd70 0x7f041415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:36.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.742+0000 7f041a333640 1 --2- 172.21.15.84:0/4066247599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04140b8480 0x7f04140cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:36.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.743+0000 7f0419b32640 1 -- 172.21.15.84:0/4066247599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04140b8480 msgr2=0x7f04140cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:36.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.743+0000 7f0419b32640 1 --2- 172.21.15.84:0/4066247599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04140b8480 0x7f04140cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.743+0000 7f0419b32640 1 -- 172.21.15.84:0/4066247599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041415cd70 msgr2=0x7f041415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:36.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.743+0000 7f0419b32640 1 --2- 172.21.15.84:0/4066247599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041415cd70 0x7f041415dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.743+0000 7f0419b32640 1 -- 172.21.15.84:0/4066247599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04100094d0 con 0x7f04140d0be0 2023-10-16T10:08:36.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.743+0000 7f0419b32640 1 --2- 172.21.15.84:0/4066247599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04140d0be0 0x7f041415d760 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f040400d5f0 tx=0x7f040400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:36.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.743+0000 7f040affd640 1 -- 172.21.15.84:0/4066247599 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04040125f0 con 0x7f04140d0be0 2023-10-16T10:08:36.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.743+0000 7f040affd640 1 -- 172.21.15.84:0/4066247599 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0404012750 con 0x7f04140d0be0 2023-10-16T10:08:36.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.743+0000 7f041b335640 1 -- 172.21.15.84:0/4066247599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0414155310 con 0x7f04140d0be0 2023-10-16T10:08:36.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.743+0000 7f040affd640 1 -- 172.21.15.84:0/4066247599 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404012b00 con 0x7f04140d0be0 2023-10-16T10:08:36.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.743+0000 7f041b335640 1 -- 172.21.15.84:0/4066247599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04141558d0 con 0x7f04140d0be0 2023-10-16T10:08:36.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.745+0000 7f041b335640 1 -- 172.21.15.84:0/4066247599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041404e9b0 con 0x7f04140d0be0 2023-10-16T10:08:36.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.747+0000 7f040affd640 1 -- 172.21.15.84:0/4066247599 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 71) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f0404012c60 con 0x7f04140d0be0 2023-10-16T10:08:36.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.748+0000 7f040affd640 1 --2- 172.21.15.84:0/4066247599 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f03f806afe0 0x7f03f806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:36.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.748+0000 7f041a333640 1 -- 172.21.15.84:0/4066247599 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f03f806afe0 msgr2=0x7f03f806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2784642648 2023-10-16T10:08:36.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.748+0000 7f040affd640 1 -- 172.21.15.84:0/4066247599 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0404089d70 con 0x7f04140d0be0 2023-10-16T10:08:36.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.748+0000 7f041a333640 1 --2- 172.21.15.84:0/4066247599 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f03f806afe0 0x7f03f806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:36.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.752+0000 7f040affd640 1 -- 172.21.15.84:0/4066247599 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0404054c50 con 0x7f04140d0be0 2023-10-16T10:08:36.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.946+0000 7f041b335640 1 -- 172.21.15.84:0/4066247599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f041415f590 con 0x7f04140d0be0 2023-10-16T10:08:36.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.949+0000 7f041a333640 1 -- 172.21.15.84:0/4066247599 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f03f806afe0 msgr2=0x7f03f806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2784642648 2023-10-16T10:08:36.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.949+0000 7f041a333640 1 --2- 172.21.15.84:0/4066247599 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f03f806afe0 0x7f03f806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:08:36.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.954+0000 7f040affd640 1 -- 172.21.15.84:0/4066247599 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v48)=0 v48) v1 ==== 133+0+0 (secure 0 0 0) 0x7f0404058270 con 0x7f04140d0be0 2023-10-16T10:08:36.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.956+0000 7f041b335640 1 -- 172.21.15.84:0/4066247599 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f03f806afe0 msgr2=0x7f03f806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:36.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.956+0000 7f041b335640 1 --2- 172.21.15.84:0/4066247599 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f03f806afe0 0x7f03f806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.956+0000 7f041b335640 1 -- 172.21.15.84:0/4066247599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04140d0be0 msgr2=0x7f041415d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:36.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.956+0000 7f041b335640 1 --2- 172.21.15.84:0/4066247599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04140d0be0 0x7f041415d760 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f040400d5f0 tx=0x7f040400dab0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.957+0000 7f041b335640 1 -- 172.21.15.84:0/4066247599 shutdown_connections 2023-10-16T10:08:36.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.957+0000 7f041b335640 1 --2- 172.21.15.84:0/4066247599 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7f03f806afe0 0x7f03f806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.957+0000 7f041b335640 1 --2- 172.21.15.84:0/4066247599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041415cd70 0x7f041415dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.957+0000 7f041b335640 1 --2- 172.21.15.84:0/4066247599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04140d0be0 0x7f041415d760 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.957+0000 7f041b335640 1 --2- 172.21.15.84:0/4066247599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04140b8480 0x7f04140cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:36.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.957+0000 7f041b335640 1 -- 172.21.15.84:0/4066247599 >> 172.21.15.84:0/4066247599 conn(0x7f04140cf570 msgr2=0x7f041415fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:36.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.957+0000 7f041b335640 1 -- 172.21.15.84:0/4066247599 shutdown_connections 2023-10-16T10:08:36.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:36.957+0000 7f041b335640 1 -- 172.21.15.84:0/4066247599 wait complete. 2023-10-16T10:08:36.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:36.975 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/minutes) 2023-10-16T10:08:37.030 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/minutes 1m 2023-10-16T10:08:37.049 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:37.048+0000 7eff92443040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:08:37.102 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:37.100+0000 7eff92443040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:08:37.137 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:37.135+0000 7fb2f7e32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:08:37.152 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:37.151+0000 7eff92443040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:08:37.192 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:37.190+0000 7fb2f7e32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:08:37.213 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:37.212+0000 7eff92443040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:08:37.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.226+0000 7fc22f444640 1 Processor -- start 2023-10-16T10:08:37.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.227+0000 7fc22f444640 1 -- start start 2023-10-16T10:08:37.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.228+0000 7fc22f444640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc22815ad70 0x7fc228069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:37.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.228+0000 7fc22f444640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc2280caac0 0x7fc22806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:37.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.228+0000 7fc22f444640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2280cb030 con 0x7fc2280c9ca0 2023-10-16T10:08:37.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.228+0000 7fc22f444640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2280c91c0 con 0x7fc2280caac0 2023-10-16T10:08:37.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.228+0000 7fc22f444640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2280c94a0 con 0x7fc22815ad70 2023-10-16T10:08:37.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.228+0000 7fc22c9df640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc2280c9ca0 0x7fc2280ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55642/0 (socket says 172.21.15.84:55642) 2023-10-16T10:08:37.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.228+0000 7fc22c9df640 1 -- 172.21.15.84:0/2718564533 learned_addr learned my addr 172.21.15.84:0/2718564533 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:37.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.229+0000 7fc22d1e0640 1 --2- 172.21.15.84:0/2718564533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc22815ad70 0x7fc228069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:37.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.229+0000 7fc22d9e1640 1 --2- 172.21.15.84:0/2718564533 >> v2:172.21.15.157:3300/0 conn(0x7fc2280caac0 0x7fc22806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:37.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.229+0000 7fc21ffff640 1 -- 172.21.15.84:0/2718564533 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 444648183 0 0) 0x7fc2280cb030 con 0x7fc2280c9ca0 2023-10-16T10:08:37.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.229+0000 7fc21ffff640 1 -- 172.21.15.84:0/2718564533 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc214003520 con 0x7fc2280c9ca0 2023-10-16T10:08:37.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.229+0000 7fc21ffff640 1 -- 172.21.15.84:0/2718564533 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2070432850 0 0) 0x7fc214003520 con 0x7fc2280c9ca0 2023-10-16T10:08:37.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.229+0000 7fc21ffff640 1 -- 172.21.15.84:0/2718564533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc22815ad70 msgr2=0x7fc228069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:37.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.229+0000 7fc21ffff640 1 --2- 172.21.15.84:0/2718564533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc22815ad70 0x7fc228069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:37.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.229+0000 7fc21ffff640 1 -- 172.21.15.84:0/2718564533 >> v2:172.21.15.157:3300/0 conn(0x7fc2280caac0 msgr2=0x7fc22806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:37.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.229+0000 7fc21ffff640 1 --2- 172.21.15.84:0/2718564533 >> v2:172.21.15.157:3300/0 conn(0x7fc2280caac0 0x7fc22806a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:37.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.229+0000 7fc21ffff640 1 -- 172.21.15.84:0/2718564533 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2280c9780 con 0x7fc2280c9ca0 2023-10-16T10:08:37.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.229+0000 7fc21ffff640 1 -- 172.21.15.84:0/2718564533 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc218001670 con 0x7fc2280c9ca0 2023-10-16T10:08:37.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.229+0000 7fc21ffff640 1 -- 172.21.15.84:0/2718564533 >> v1:172.21.15.84:6789/0 conn(0x7fc2280c9ca0 legacy=0x7fc2280ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:37.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.230+0000 7fc21ffff640 1 --2- 172.21.15.84:0/2718564533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc214003970 0x7fc214003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:37.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.230+0000 7fc21ffff640 1 --2- 172.21.15.84:0/2718564533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2140047c0 0x7fc214004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:37.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.230+0000 7fc21ffff640 1 --2- 172.21.15.84:0/2718564533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc22815ad70 0x7fc214005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:37.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.230+0000 7fc21ffff640 1 -- 172.21.15.84:0/2718564533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2280c91c0 con 0x7fc2140047c0 2023-10-16T10:08:37.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.230+0000 7fc21ffff640 1 -- 172.21.15.84:0/2718564533 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2280c94a0 con 0x7fc214003970 2023-10-16T10:08:37.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.230+0000 7fc21ffff640 1 -- 172.21.15.84:0/2718564533 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21400b980 con 0x7fc22815ad70 2023-10-16T10:08:37.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.230+0000 7fc22c9df640 1 --2- 172.21.15.84:0/2718564533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2140047c0 0x7fc214004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:37.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.230+0000 7fc22c9df640 1 -- 172.21.15.84:0/2718564533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc22815ad70 msgr2=0x7fc214005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:37.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.230+0000 7fc22d9e1640 1 --2- 172.21.15.84:0/2718564533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc22815ad70 0x7fc214005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:37.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.230+0000 7fc22c9df640 1 --2- 172.21.15.84:0/2718564533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc22815ad70 0x7fc214005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:37.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.230+0000 7fc22c9df640 1 -- 172.21.15.84:0/2718564533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc214003970 msgr2=0x7fc214003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:37.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.230+0000 7fc22c9df640 1 --2- 172.21.15.84:0/2718564533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc214003970 0x7fc214003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:37.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.230+0000 7fc22c9df640 1 -- 172.21.15.84:0/2718564533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc214003520 con 0x7fc2140047c0 2023-10-16T10:08:37.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.231+0000 7fc22c9df640 1 --2- 172.21.15.84:0/2718564533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2140047c0 0x7fc214004c40 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fc21800c290 tx=0x7fc2140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4d21cb7a99b7b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:37.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.231+0000 7fc21ffff640 1 -- 172.21.15.84:0/2718564533 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218005ad0 con 0x7fc2140047c0 2023-10-16T10:08:37.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.232+0000 7fc21ffff640 1 -- 172.21.15.84:0/2718564533 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc218005c30 con 0x7fc2140047c0 2023-10-16T10:08:37.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.232+0000 7fc21ffff640 1 -- 172.21.15.84:0/2718564533 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21802c8d0 con 0x7fc2140047c0 2023-10-16T10:08:37.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.232+0000 7fc22f444640 1 -- 172.21.15.84:0/2718564533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2140047c0 msgr2=0x7fc214004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:37.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.232+0000 7fc22f444640 1 --2- 172.21.15.84:0/2718564533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2140047c0 0x7fc214004c40 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fc21800c290 tx=0x7fc2140057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:37.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.232+0000 7fc22d9e1640 1 -- 172.21.15.84:0/2718564533 reap_dead start 2023-10-16T10:08:37.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.232+0000 7fc22f444640 1 -- 172.21.15.84:0/2718564533 shutdown_connections 2023-10-16T10:08:37.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.232+0000 7fc22f444640 1 -- 172.21.15.84:0/2718564533 >> 172.21.15.84:0/2718564533 conn(0x7fc2280ce9b0 msgr2=0x7fc2280cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:37.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.233+0000 7fc22f444640 1 -- 172.21.15.84:0/2718564533 shutdown_connections 2023-10-16T10:08:37.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.233+0000 7fc22f444640 1 -- 172.21.15.84:0/2718564533 wait complete. 2023-10-16T10:08:37.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.234+0000 7fc22f444640 1 Processor -- start 2023-10-16T10:08:37.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.234+0000 7fc22f444640 1 -- start start 2023-10-16T10:08:37.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.235+0000 7fc22f444640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2140047c0 0x7fc22814a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:37.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.235+0000 7fc22f444640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2280a8c30 0x7fc2280b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:37.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.235+0000 7fc22f444640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc22814f0c0 0x7fc22814f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:37.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.235+0000 7fc22f444640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22814a6b0 con 0x7fc22814f0c0 2023-10-16T10:08:37.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.235+0000 7fc22f444640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22814a990 con 0x7fc2280a8c30 2023-10-16T10:08:37.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.235+0000 7fc22f444640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22814ac70 con 0x7fc2140047c0 2023-10-16T10:08:37.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.236+0000 7fc22d9e1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc22814f0c0 0x7fc22814f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:37.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.236+0000 7fc22d9e1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc22814f0c0 0x7fc22814f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53082/0 (socket says 172.21.15.84:53082) 2023-10-16T10:08:37.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.236+0000 7fc22d9e1640 1 -- 172.21.15.84:0/3674293092 learned_addr learned my addr 172.21.15.84:0/3674293092 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:37.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.236+0000 7fc22d9e1640 1 -- 172.21.15.84:0/3674293092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2140047c0 msgr2=0x7fc22814a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:37.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.236+0000 7fc22d9e1640 1 --2- 172.21.15.84:0/3674293092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2140047c0 0x7fc22814a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:37.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.236+0000 7fc22d9e1640 1 -- 172.21.15.84:0/3674293092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2280a8c30 msgr2=0x7fc2280b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:37.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.236+0000 7fc22d9e1640 1 --2- 172.21.15.84:0/3674293092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2280a8c30 0x7fc2280b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:37.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.236+0000 7fc22d9e1640 1 -- 172.21.15.84:0/3674293092 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22814af50 con 0x7fc22814f0c0 2023-10-16T10:08:37.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.236+0000 7fc22d9e1640 1 --2- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc22814f0c0 0x7fc22814f540 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fc2280cf780 tx=0x7fc22000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:37.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.236+0000 7fc21dffb640 1 -- 172.21.15.84:0/3674293092 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc220017400 con 0x7fc22814f0c0 2023-10-16T10:08:37.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.236+0000 7fc21dffb640 1 -- 172.21.15.84:0/3674293092 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc220011040 con 0x7fc22814f0c0 2023-10-16T10:08:37.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.237+0000 7fc21dffb640 1 -- 172.21.15.84:0/3674293092 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2200177c0 con 0x7fc22814f0c0 2023-10-16T10:08:37.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.237+0000 7fc22f444640 1 -- 172.21.15.84:0/3674293092 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22814b230 con 0x7fc22814f0c0 2023-10-16T10:08:37.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.237+0000 7fc22f444640 1 -- 172.21.15.84:0/3674293092 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2280b3d80 con 0x7fc22814f0c0 2023-10-16T10:08:37.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.238+0000 7fc22f444640 1 -- 172.21.15.84:0/3674293092 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22804e160 con 0x7fc22814f0c0 2023-10-16T10:08:37.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.239+0000 7fc21dffb640 1 -- 172.21.15.84:0/3674293092 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 71) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fc220017920 con 0x7fc22814f0c0 2023-10-16T10:08:37.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.240+0000 7fc21dffb640 1 --2- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fc20c06afe0 0x7fc20c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:37.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.240+0000 7fc21dffb640 1 -- 172.21.15.84:0/3674293092 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc220089e60 con 0x7fc22814f0c0 2023-10-16T10:08:37.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.240+0000 7fc22d1e0640 1 -- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fc20c06afe0 msgr2=0x7fc20c06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2784642648 2023-10-16T10:08:37.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.240+0000 7fc22d1e0640 1 --2- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fc20c06afe0 0x7fc20c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:37.250 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:37.244+0000 7fb2f7e32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:08:37.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.244+0000 7fc21dffb640 1 -- 172.21.15.84:0/3674293092 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc220054c30 con 0x7fc22814f0c0 2023-10-16T10:08:37.263 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:37.262+0000 7eff92443040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:08:37.308 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:37.307+0000 7fb2f7e32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:08:37.359 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:37.358+0000 7fb2f7e32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:08:37.374 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:37.372+0000 7eff92443040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:08:37.436 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:37.434+0000 7eff92443040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:08:37.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.441+0000 7fc22d1e0640 1 -- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fc20c06afe0 msgr2=0x7fc20c06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2784642648 2023-10-16T10:08:37.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.441+0000 7fc22d1e0640 1 --2- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fc20c06afe0 0x7fc20c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:08:37.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.460+0000 7fc22f444640 1 -- 172.21.15.84:0/3674293092 --> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/minutes", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fc228069da0 con 0x7fc20c06afe0 2023-10-16T10:08:37.474 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:37.473+0000 7fb2f7e32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:08:37.485 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:37.484+0000 7eff92443040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:08:37.537 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:37.535+0000 7fb2f7e32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:08:37.543 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:37.541+0000 7eff92443040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:08:37.586 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:37.584+0000 7fb2f7e32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:08:37.593 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:37.591+0000 7eff92443040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:08:37.642 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:37.640+0000 7fb2f7e32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:08:37.691 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:37.690+0000 7fb2f7e32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:08:37.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.842+0000 7fc22d1e0640 1 -- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fc20c06afe0 msgr2=0x7fc20c06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2784642648 2023-10-16T10:08:37.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:37.842+0000 7fc22d1e0640 1 --2- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fc20c06afe0 0x7fc20c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T10:08:37.974 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:37.972+0000 7eff92443040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:08:38.070 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:38.069+0000 7fb2f7e32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:08:38.133 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:38.132+0000 7eff92443040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:08:38.229 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:38.227+0000 7fb2f7e32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:08:38.241 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:38.240+0000 7eff92443040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:08:38.291 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:38.289+0000 7eff92443040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:08:38.336 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:38.334+0000 7fb2f7e32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:08:38.385 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:38.383+0000 7fb2f7e32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:08:38.397 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:38.396+0000 7eff92443040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:08:38.493 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:38.491+0000 7fb2f7e32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:08:38.504 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:38.503+0000 7eff92443040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:08:38.600 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:38.598+0000 7fb2f7e32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:08:38.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:38.643+0000 7fc22d1e0640 1 -- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fc20c06afe0 msgr2=0x7fc20c06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2784642648 2023-10-16T10:08:38.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:38.643+0000 7fc22d1e0640 1 --2- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fc20c06afe0 0x7fc20c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T10:08:38.662 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:38.660+0000 7eff92443040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:08:38.711 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:38.709+0000 7eff92443040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:08:38.755 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:38.753+0000 7fb2f7e32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:08:38.805 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:38.802+0000 7fb2f7e32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:08:38.829 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:38.827+0000 7eff92443040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:08:38.921 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:38.920+0000 7fb2f7e32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:08:39.632 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:39.630+0000 7eff92443040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:08:39.740 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:39.738+0000 7fb2f7e32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:08:39.926 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:39.925+0000 7eff92443040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:08:39.983 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:39.981+0000 7eff92443040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:08:40.034 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:40.030+0000 7fb2f7e32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:08:40.087 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:40.086+0000 7fb2f7e32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:08:40.096 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:40.094+0000 7eff92443040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:08:40.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:40.162+0000 7fc21dffb640 1 -- 172.21.15.84:0/3674293092 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 72) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fc220055000 con 0x7fc22814f0c0 2023-10-16T10:08:40.206 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:40.205+0000 7fb2f7e32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:08:40.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:40.220+0000 7fc21dffb640 1 -- 172.21.15.84:0/3674293092 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 73) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fc220057de0 con 0x7fc22814f0c0 2023-10-16T10:08:40.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:40.221+0000 7fc21dffb640 1 -- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fc20c06afe0 msgr2=0x7fc20c06d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:40.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:40.221+0000 7fc21dffb640 1 --2- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:6832/2784642648,v1:172.21.15.84:6833/2784642648] conn(0x7fc20c06afe0 0x7fc20c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:40.579 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:40.578+0000 7fb2761cd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:40.580 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:40.578+0000 7fb2761cd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:40.580 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:40.578+0000 7fb2761cd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:40.580 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:40.578+0000 7fb2761cd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:40.580 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:40.578+0000 7fb2761cd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:40.581 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:40.579+0000 7fb274189640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:40.581 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:40.579+0000 7fb274189640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:40.581 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:40.579+0000 7fb274189640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:40.582 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:40.579+0000 7fb274189640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:40.582 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:40.579+0000 7fb274189640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:41.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.232+0000 7fc21dffb640 1 -- 172.21.15.84:0/3674293092 <== mon.0 v2:172.21.15.84:3300/0 9 ==== mgrmap(e 74) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fc220081c80 con 0x7fc22814f0c0 2023-10-16T10:08:41.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.233+0000 7fc21dffb640 1 --2- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fc20c01d1c0 0x7fc20c074f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:41.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.233+0000 7fc21dffb640 1 -- 172.21.15.84:0/3674293092 --> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/minutes", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fc20c01d5a0 con 0x7fc20c01d1c0 2023-10-16T10:08:41.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.241+0000 7fc22d1e0640 1 --2- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fc20c01d1c0 0x7fc20c074f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:41.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.263+0000 7fc22d1e0640 1 --2- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fc20c01d1c0 0x7fc20c074f00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc224006e80 tx=0x7fc224009300 comp rx=0 tx=0).ready entity=mgr.8754 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:41.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.466+0000 7fc21dffb640 1 -- 172.21.15.84:0/3674293092 <== mgr.8754 v2:172.21.15.84:6832/1203729023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+45 (secure 0 0 0) 0x7fc20c01d5a0 con 0x7fc20c01d1c0 2023-10-16T10:08:41.469 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path snap_test_dir1/minutes 2023-10-16T10:08:41.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.470+0000 7fc22f444640 1 -- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fc20c01d1c0 msgr2=0x7fc20c074f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:41.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.470+0000 7fc22f444640 1 --2- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fc20c01d1c0 0x7fc20c074f00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc224006e80 tx=0x7fc224009300 comp rx=0 tx=0).stop 2023-10-16T10:08:41.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.470+0000 7fc22f444640 1 -- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc22814f0c0 msgr2=0x7fc22814f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:41.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.470+0000 7fc22f444640 1 --2- 172.21.15.84:0/3674293092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc22814f0c0 0x7fc22814f540 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fc2280cf780 tx=0x7fc22000f4a0 comp rx=0 tx=0).stop 2023-10-16T10:08:41.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.470+0000 7fc22d9e1640 1 -- 172.21.15.84:0/3674293092 reap_dead start 2023-10-16T10:08:41.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.470+0000 7fc22f444640 1 -- 172.21.15.84:0/3674293092 shutdown_connections 2023-10-16T10:08:41.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.470+0000 7fc22f444640 1 -- 172.21.15.84:0/3674293092 >> 172.21.15.84:0/3674293092 conn(0x7fc2280ce9b0 msgr2=0x7fc22815d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:41.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.470+0000 7fc22f444640 1 -- 172.21.15.84:0/3674293092 shutdown_connections 2023-10-16T10:08:41.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.470+0000 7fc22f444640 1 -- 172.21.15.84:0/3674293092 wait complete. 2023-10-16T10:08:41.491 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/minutes 2023-10-16T10:08:41.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:41.492 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/hourly) 2023-10-16T10:08:41.549 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/hourly 1h 2023-10-16T10:08:41.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.760+0000 7f62b71bf640 1 Processor -- start 2023-10-16T10:08:41.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.760+0000 7f62b71bf640 1 -- start start 2023-10-16T10:08:41.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.762+0000 7f62b71bf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62b015dde0 0x7f62b00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:41.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.762+0000 7f62b71bf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b0160a90 con 0x7f62b015c9f0 2023-10-16T10:08:41.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.762+0000 7f62b71bf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b00cd900 con 0x7f62b0070f20 2023-10-16T10:08:41.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.762+0000 7f62b71bf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b00cdec0 con 0x7f62b015dde0 2023-10-16T10:08:41.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.762+0000 7f62b61bd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f62b015c9f0 0x7f62b00c8cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55646/0 (socket says 172.21.15.84:55646) 2023-10-16T10:08:41.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.762+0000 7f62b61bd640 1 -- 172.21.15.84:0/184111850 learned_addr learned my addr 172.21.15.84:0/184111850 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:41.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.763+0000 7f62b59bc640 1 --2- 172.21.15.84:0/184111850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62b015dde0 0x7f62b00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:41.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.763+0000 7f62b51bb640 1 -- 172.21.15.84:0/184111850 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146582770 0 0) 0x7f62b0160a90 con 0x7f62b015c9f0 2023-10-16T10:08:41.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.763+0000 7f62b51bb640 1 -- 172.21.15.84:0/184111850 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f629c003520 con 0x7f62b015c9f0 2023-10-16T10:08:41.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.763+0000 7f62b51bb640 1 -- 172.21.15.84:0/184111850 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 519597270 0 0) 0x7f629c003520 con 0x7f62b015c9f0 2023-10-16T10:08:41.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.763+0000 7f62b51bb640 1 -- 172.21.15.84:0/184111850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62b015dde0 msgr2=0x7f62b00709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:41.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.763+0000 7f62b51bb640 1 --2- 172.21.15.84:0/184111850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62b015dde0 0x7f62b00709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:41.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.763+0000 7f62b51bb640 1 -- 172.21.15.84:0/184111850 >> v1:172.21.15.157:6789/0 conn(0x7f62b0070f20 legacy=0x7f62b00ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:41.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.763+0000 7f62b51bb640 1 -- 172.21.15.84:0/184111850 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b00cdbe0 con 0x7f62b015c9f0 2023-10-16T10:08:41.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.763+0000 7f62b51bb640 1 -- 172.21.15.84:0/184111850 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f62ac001670 con 0x7f62b015c9f0 2023-10-16T10:08:41.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.763+0000 7f62b59bc640 1 --2- 172.21.15.84:0/184111850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62b015dde0 0x7f62b00709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:08:41.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.763+0000 7f62b51bb640 1 -- 172.21.15.84:0/184111850 >> v1:172.21.15.84:6789/0 conn(0x7f62b015c9f0 legacy=0x7f62b00c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:41.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.763+0000 7f62b51bb640 1 --2- 172.21.15.84:0/184111850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f629c0039a0 0x7f629c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:41.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.763+0000 7f62b51bb640 1 --2- 172.21.15.84:0/184111850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62b015dde0 0x7f629c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:41.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.763+0000 7f62b51bb640 1 --2- 172.21.15.84:0/184111850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f629c004d00 0x7f629c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:41.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.763+0000 7f62b51bb640 1 -- 172.21.15.84:0/184111850 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c00ba60 con 0x7f629c004d00 2023-10-16T10:08:41.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.763+0000 7f62b51bb640 1 -- 172.21.15.84:0/184111850 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c00bbd0 con 0x7f629c0039a0 2023-10-16T10:08:41.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.763+0000 7f62b51bb640 1 -- 172.21.15.84:0/184111850 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629c00bd90 con 0x7f62b015dde0 2023-10-16T10:08:41.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.764+0000 7f62b69be640 1 --2- 172.21.15.84:0/184111850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f629c004d00 0x7f629c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:41.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.764+0000 7f62b61bd640 1 --2- 172.21.15.84:0/184111850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f629c0039a0 0x7f629c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:41.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.764+0000 7f62b59bc640 1 --2- 172.21.15.84:0/184111850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62b015dde0 0x7f629c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:41.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.764+0000 7f62b69be640 1 -- 172.21.15.84:0/184111850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62b015dde0 msgr2=0x7f629c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:41.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.764+0000 7f62b69be640 1 --2- 172.21.15.84:0/184111850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62b015dde0 0x7f629c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:41.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.764+0000 7f62b69be640 1 -- 172.21.15.84:0/184111850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f629c0039a0 msgr2=0x7f629c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:41.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.764+0000 7f62b69be640 1 --2- 172.21.15.84:0/184111850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f629c0039a0 0x7f629c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:41.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.764+0000 7f62b69be640 1 -- 172.21.15.84:0/184111850 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c003520 con 0x7f629c004d00 2023-10-16T10:08:41.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.765+0000 7f62b69be640 1 --2- 172.21.15.84:0/184111850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f629c004d00 0x7f629c005180 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f62a8001e20 tx=0x7f629c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d061e4c1d45ecc51 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:41.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.766+0000 7f62b51bb640 1 -- 172.21.15.84:0/184111850 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a8003b70 con 0x7f629c004d00 2023-10-16T10:08:41.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.766+0000 7f62b51bb640 1 -- 172.21.15.84:0/184111850 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a8003cd0 con 0x7f629c004d00 2023-10-16T10:08:41.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.766+0000 7f62b51bb640 1 -- 172.21.15.84:0/184111850 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a8005310 con 0x7f629c004d00 2023-10-16T10:08:41.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.766+0000 7f62b71bf640 1 -- 172.21.15.84:0/184111850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f629c004d00 msgr2=0x7f629c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:41.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.766+0000 7f62b71bf640 1 --2- 172.21.15.84:0/184111850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f629c004d00 0x7f629c005180 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f62a8001e20 tx=0x7f629c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:41.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.766+0000 7f62b69be640 1 -- 172.21.15.84:0/184111850 reap_dead start 2023-10-16T10:08:41.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.766+0000 7f62b71bf640 1 -- 172.21.15.84:0/184111850 shutdown_connections 2023-10-16T10:08:41.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.766+0000 7f62b71bf640 1 -- 172.21.15.84:0/184111850 >> 172.21.15.84:0/184111850 conn(0x7f62b00ce6a0 msgr2=0x7f62b00ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:41.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.766+0000 7f62b71bf640 1 -- 172.21.15.84:0/184111850 shutdown_connections 2023-10-16T10:08:41.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.767+0000 7f62b71bf640 1 -- 172.21.15.84:0/184111850 wait complete. 2023-10-16T10:08:41.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.768+0000 7f62b71bf640 1 Processor -- start 2023-10-16T10:08:41.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.768+0000 7f62b71bf640 1 -- start start 2023-10-16T10:08:41.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.769+0000 7f62b71bf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f629c004d00 0x7f62b00c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:41.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.769+0000 7f62b71bf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62b0156e70 0x7f62b00c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:41.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.769+0000 7f62b71bf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62b015f370 0x7f62b015f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:41.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.769+0000 7f62b71bf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b015fd30 con 0x7f629c004d00 2023-10-16T10:08:41.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.769+0000 7f62b71bf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b00c5cb0 con 0x7f62b015f370 2023-10-16T10:08:41.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.769+0000 7f62b71bf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b0069da0 con 0x7f62b0156e70 2023-10-16T10:08:41.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.770+0000 7f62b61bd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f629c004d00 0x7f62b00c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:41.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.770+0000 7f62b61bd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f629c004d00 0x7f62b00c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53184/0 (socket says 172.21.15.84:53184) 2023-10-16T10:08:41.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.770+0000 7f62b61bd640 1 -- 172.21.15.84:0/489546781 learned_addr learned my addr 172.21.15.84:0/489546781 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:41.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.770+0000 7f62b69be640 1 --2- 172.21.15.84:0/489546781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62b015f370 0x7f62b015f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:41.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.770+0000 7f62b59bc640 1 --2- 172.21.15.84:0/489546781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62b0156e70 0x7f62b00c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:41.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.770+0000 7f62b61bd640 1 -- 172.21.15.84:0/489546781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62b0156e70 msgr2=0x7f62b00c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:41.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.770+0000 7f62b61bd640 1 --2- 172.21.15.84:0/489546781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62b0156e70 0x7f62b00c5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:41.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.770+0000 7f62b61bd640 1 -- 172.21.15.84:0/489546781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62b015f370 msgr2=0x7f62b015f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:41.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.770+0000 7f62b61bd640 1 --2- 172.21.15.84:0/489546781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62b015f370 0x7f62b015f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:41.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.770+0000 7f62b61bd640 1 -- 172.21.15.84:0/489546781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b00c5f60 con 0x7f629c004d00 2023-10-16T10:08:41.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.771+0000 7f62b61bd640 1 --2- 172.21.15.84:0/489546781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f629c004d00 0x7f62b00c80d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00e7f0 tx=0x7f62ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:41.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.771+0000 7f62a2ffd640 1 -- 172.21.15.84:0/489546781 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac00c8d0 con 0x7f629c004d00 2023-10-16T10:08:41.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.771+0000 7f62a2ffd640 1 -- 172.21.15.84:0/489546781 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac00ca30 con 0x7f629c004d00 2023-10-16T10:08:41.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.771+0000 7f62b71bf640 1 -- 172.21.15.84:0/489546781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b00c6240 con 0x7f629c004d00 2023-10-16T10:08:41.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.771+0000 7f62a2ffd640 1 -- 172.21.15.84:0/489546781 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac00cdc0 con 0x7f629c004d00 2023-10-16T10:08:41.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.771+0000 7f62b71bf640 1 -- 172.21.15.84:0/489546781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b00c6800 con 0x7f629c004d00 2023-10-16T10:08:41.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.772+0000 7f62b71bf640 1 -- 172.21.15.84:0/489546781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b0069fc0 con 0x7f629c004d00 2023-10-16T10:08:41.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.773+0000 7f62a2ffd640 1 -- 172.21.15.84:0/489546781 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 74) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f62ac01d460 con 0x7f629c004d00 2023-10-16T10:08:41.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.774+0000 7f62a2ffd640 1 --2- 172.21.15.84:0/489546781 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f629406ad10 0x7f629406d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:41.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.774+0000 7f62a2ffd640 1 -- 172.21.15.84:0/489546781 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f62ac089440 con 0x7f629c004d00 2023-10-16T10:08:41.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.774+0000 7f62b59bc640 1 --2- 172.21.15.84:0/489546781 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f629406ad10 0x7f629406d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:41.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.775+0000 7f62b59bc640 1 --2- 172.21.15.84:0/489546781 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f629406ad10 0x7f629406d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f62b00c73d0 tx=0x7f62a4008040 comp rx=0 tx=0).ready entity=mgr.8754 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:41.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.780+0000 7f62a2ffd640 1 -- 172.21.15.84:0/489546781 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f62b0069fc0 con 0x7f629c004d00 2023-10-16T10:08:41.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:41.996+0000 7f62b71bf640 1 -- 172.21.15.84:0/489546781 --> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/hourly", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f62b00704c0 con 0x7f629406ad10 2023-10-16T10:08:42.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.181+0000 7f62a2ffd640 1 -- 172.21.15.84:0/489546781 <== mgr.8754 v2:172.21.15.84:6832/1203729023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+44 (secure 0 0 0) 0x7f62a400c040 con 0x7f629406ad10 2023-10-16T10:08:42.183 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path snap_test_dir1/hourly 2023-10-16T10:08:42.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.184+0000 7f62b71bf640 1 -- 172.21.15.84:0/489546781 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f629406ad10 msgr2=0x7f629406d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:42.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.184+0000 7f62b71bf640 1 --2- 172.21.15.84:0/489546781 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f629406ad10 0x7f629406d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f62b00c73d0 tx=0x7f62a4008040 comp rx=0 tx=0).stop 2023-10-16T10:08:42.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.184+0000 7f62b71bf640 1 -- 172.21.15.84:0/489546781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f629c004d00 msgr2=0x7f62b00c80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:42.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.184+0000 7f62b71bf640 1 --2- 172.21.15.84:0/489546781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f629c004d00 0x7f62b00c80d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00e7f0 tx=0x7f62ac00ecb0 comp rx=0 tx=0).stop 2023-10-16T10:08:42.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.184+0000 7f62b71bf640 1 -- 172.21.15.84:0/489546781 shutdown_connections 2023-10-16T10:08:42.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.184+0000 7f62b71bf640 1 --2- 172.21.15.84:0/489546781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62b015f370 0x7f62b015f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:42.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.184+0000 7f62b71bf640 1 --2- 172.21.15.84:0/489546781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62b0156e70 0x7f62b00c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:42.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.184+0000 7f62b71bf640 1 --2- 172.21.15.84:0/489546781 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f629406ad10 0x7f629406d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:42.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.184+0000 7f62b71bf640 1 --2- 172.21.15.84:0/489546781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f629c004d00 0x7f62b00c80d0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:42.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.184+0000 7f62b71bf640 1 -- 172.21.15.84:0/489546781 >> 172.21.15.84:0/489546781 conn(0x7f62b00ce6a0 msgr2=0x7f62b015b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:42.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.184+0000 7f62b71bf640 1 -- 172.21.15.84:0/489546781 shutdown_connections 2023-10-16T10:08:42.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.184+0000 7f62b71bf640 1 -- 172.21.15.84:0/489546781 wait complete. 2023-10-16T10:08:42.206 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/hourly 2023-10-16T10:08:42.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:42.206 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/daily) 2023-10-16T10:08:42.262 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/daily 1d 2023-10-16T10:08:42.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.458+0000 7f73dea7c640 1 Processor -- start 2023-10-16T10:08:42.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.459+0000 7f73dea7c640 1 -- start start 2023-10-16T10:08:42.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.460+0000 7f73dea7c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f73d806a4b0 0x7f73d80ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:42.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.460+0000 7f73dea7c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d80cb030 con 0x7f73d80c9ca0 2023-10-16T10:08:42.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.460+0000 7f73dea7c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d80c91c0 con 0x7f73d815ad70 2023-10-16T10:08:42.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.460+0000 7f73dea7c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d80c94a0 con 0x7f73d806a4b0 2023-10-16T10:08:42.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.460+0000 7f73d7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f73d806a4b0 0x7f73d80ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:42.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.460+0000 7f73dd019640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f73d80c9ca0 0x7f73d80ad4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55652/0 (socket says 172.21.15.84:55652) 2023-10-16T10:08:42.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.461+0000 7f73dd019640 1 -- 172.21.15.84:0/2917433427 learned_addr learned my addr 172.21.15.84:0/2917433427 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:42.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.461+0000 7f73d7fff640 1 -- 172.21.15.84:0/2917433427 >> v1:172.21.15.157:6789/0 conn(0x7f73d815ad70 legacy=0x7f73d8069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:42.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.461+0000 7f73d7fff640 1 -- 172.21.15.84:0/2917433427 >> v1:172.21.15.84:6789/0 conn(0x7f73d80c9ca0 legacy=0x7f73d80ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:42.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.461+0000 7f73d7fff640 1 -- 172.21.15.84:0/2917433427 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d80c9780 con 0x7f73d806a4b0 2023-10-16T10:08:42.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.462+0000 7f73d7fff640 1 --2- 172.21.15.84:0/2917433427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d806a4b0 0x7f73d80ca3b0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f73c40097a0 tx=0x7f73c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d8b4af8b1dc61b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:42.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.462+0000 7f73d77fe640 1 -- 172.21.15.84:0/2917433427 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4031830 con 0x7f73d806a4b0 2023-10-16T10:08:42.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.462+0000 7f73d77fe640 1 -- 172.21.15.84:0/2917433427 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c4031990 con 0x7f73d806a4b0 2023-10-16T10:08:42.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.462+0000 7f73d77fe640 1 -- 172.21.15.84:0/2917433427 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4031d20 con 0x7f73d806a4b0 2023-10-16T10:08:42.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.463+0000 7f73dea7c640 1 -- 172.21.15.84:0/2917433427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d806a4b0 msgr2=0x7f73d80ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:42.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.463+0000 7f73dea7c640 1 --2- 172.21.15.84:0/2917433427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d806a4b0 0x7f73d80ca3b0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f73c40097a0 tx=0x7f73c402b8d0 comp rx=0 tx=0).stop 2023-10-16T10:08:42.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.463+0000 7f73dea7c640 1 -- 172.21.15.84:0/2917433427 shutdown_connections 2023-10-16T10:08:42.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.463+0000 7f73dea7c640 1 --2- 172.21.15.84:0/2917433427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d806a4b0 0x7f73d80ca3b0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:42.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.463+0000 7f73dea7c640 1 -- 172.21.15.84:0/2917433427 >> 172.21.15.84:0/2917433427 conn(0x7f73d80ce9b0 msgr2=0x7f73d80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:42.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.463+0000 7f73dea7c640 1 -- 172.21.15.84:0/2917433427 shutdown_connections 2023-10-16T10:08:42.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.463+0000 7f73dea7c640 1 -- 172.21.15.84:0/2917433427 wait complete. 2023-10-16T10:08:42.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.465+0000 7f73dea7c640 1 Processor -- start 2023-10-16T10:08:42.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.465+0000 7f73dea7c640 1 -- start start 2023-10-16T10:08:42.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.466+0000 7f73dea7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73d806a4b0 0x7f73d81f3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:42.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.466+0000 7f73dea7c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73d80c9ca0 0x7f73d80c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:42.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.466+0000 7f73dea7c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d815ad70 0x7f73d80c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:42.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.466+0000 7f73dea7c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80aac80 con 0x7f73d815ad70 2023-10-16T10:08:42.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.466+0000 7f73dea7c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80aa9a0 con 0x7f73d806a4b0 2023-10-16T10:08:42.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.466+0000 7f73dea7c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d81f3870 con 0x7f73d80c9ca0 2023-10-16T10:08:42.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.466+0000 7f73dd019640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d815ad70 0x7f73d80c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:42.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.466+0000 7f73dd019640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d815ad70 0x7f73d80c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53210/0 (socket says 172.21.15.84:53210) 2023-10-16T10:08:42.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.466+0000 7f73dd019640 1 -- 172.21.15.84:0/1089497493 learned_addr learned my addr 172.21.15.84:0/1089497493 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:42.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.467+0000 7f73dc818640 1 --2- 172.21.15.84:0/1089497493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73d806a4b0 0x7f73d81f3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:42.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.467+0000 7f73dd019640 1 -- 172.21.15.84:0/1089497493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73d80c9ca0 msgr2=0x7f73d80c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:42.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.467+0000 7f73dd019640 1 --2- 172.21.15.84:0/1089497493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73d80c9ca0 0x7f73d80c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:42.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.467+0000 7f73dd019640 1 -- 172.21.15.84:0/1089497493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73d806a4b0 msgr2=0x7f73d81f3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:42.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.467+0000 7f73dd019640 1 --2- 172.21.15.84:0/1089497493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73d806a4b0 0x7f73d81f3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:42.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.467+0000 7f73dd019640 1 -- 172.21.15.84:0/1089497493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c4009400 con 0x7f73d815ad70 2023-10-16T10:08:42.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.467+0000 7f73dd019640 1 --2- 172.21.15.84:0/1089497493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d815ad70 0x7f73d80c8150 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f73d000cbf0 tx=0x7f73d000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:42.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.467+0000 7f73d57fa640 1 -- 172.21.15.84:0/1089497493 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d000ad40 con 0x7f73d815ad70 2023-10-16T10:08:42.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.467+0000 7f73d57fa640 1 -- 172.21.15.84:0/1089497493 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d0012400 con 0x7f73d815ad70 2023-10-16T10:08:42.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.467+0000 7f73dea7c640 1 -- 172.21.15.84:0/1089497493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d80a9d20 con 0x7f73d815ad70 2023-10-16T10:08:42.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.467+0000 7f73d57fa640 1 -- 172.21.15.84:0/1089497493 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d000aea0 con 0x7f73d815ad70 2023-10-16T10:08:42.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.467+0000 7f73dea7c640 1 -- 172.21.15.84:0/1089497493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d80aa2e0 con 0x7f73d815ad70 2023-10-16T10:08:42.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.468+0000 7f73dea7c640 1 -- 172.21.15.84:0/1089497493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d80aaae0 con 0x7f73d815ad70 2023-10-16T10:08:42.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.469+0000 7f73d57fa640 1 -- 172.21.15.84:0/1089497493 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 75) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f73d0007070 con 0x7f73d815ad70 2023-10-16T10:08:42.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.470+0000 7f73d57fa640 1 --2- 172.21.15.84:0/1089497493 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f73bc06afe0 0x7f73bc06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:42.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.470+0000 7f73d57fa640 1 -- 172.21.15.84:0/1089497493 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f73d0089820 con 0x7f73d815ad70 2023-10-16T10:08:42.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.471+0000 7f73dc818640 1 --2- 172.21.15.84:0/1089497493 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f73bc06afe0 0x7f73bc06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:42.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.471+0000 7f73dc818640 1 --2- 172.21.15.84:0/1089497493 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f73bc06afe0 0x7f73bc06d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00a750 tx=0x7f73cc008040 comp rx=0 tx=0).ready entity=mgr.8754 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:42.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.475+0000 7f73d57fa640 1 -- 172.21.15.84:0/1089497493 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f73d80aaae0 con 0x7f73d815ad70 2023-10-16T10:08:42.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.689+0000 7f73dea7c640 1 -- 172.21.15.84:0/1089497493 --> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/daily", "snap_schedule": "1d", "target": ["mon-mgr", ""]}) v1 -- 0x7f73d80ca8f0 con 0x7f73bc06afe0 2023-10-16T10:08:42.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.877+0000 7f73d57fa640 1 -- 172.21.15.84:0/1089497493 <== mgr.8754 v2:172.21.15.84:6832/1203729023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f73cc00a380 con 0x7f73bc06afe0 2023-10-16T10:08:42.879 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path snap_test_dir1/daily 2023-10-16T10:08:42.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.880+0000 7f73dea7c640 1 -- 172.21.15.84:0/1089497493 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f73bc06afe0 msgr2=0x7f73bc06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:42.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.880+0000 7f73dea7c640 1 --2- 172.21.15.84:0/1089497493 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f73bc06afe0 0x7f73bc06d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00a750 tx=0x7f73cc008040 comp rx=0 tx=0).stop 2023-10-16T10:08:42.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.880+0000 7f73dea7c640 1 -- 172.21.15.84:0/1089497493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d815ad70 msgr2=0x7f73d80c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:42.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.881+0000 7f73dea7c640 1 --2- 172.21.15.84:0/1089497493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d815ad70 0x7f73d80c8150 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f73d000cbf0 tx=0x7f73d000a430 comp rx=0 tx=0).stop 2023-10-16T10:08:42.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.881+0000 7f73dea7c640 1 -- 172.21.15.84:0/1089497493 shutdown_connections 2023-10-16T10:08:42.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.881+0000 7f73dea7c640 1 --2- 172.21.15.84:0/1089497493 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f73bc06afe0 0x7f73bc06d4a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:42.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.881+0000 7f73dea7c640 1 --2- 172.21.15.84:0/1089497493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d815ad70 0x7f73d80c8150 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:42.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.881+0000 7f73dea7c640 1 --2- 172.21.15.84:0/1089497493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73d80c9ca0 0x7f73d80c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:42.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.881+0000 7f73dea7c640 1 --2- 172.21.15.84:0/1089497493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73d806a4b0 0x7f73d81f3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:42.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.881+0000 7f73dea7c640 1 -- 172.21.15.84:0/1089497493 >> 172.21.15.84:0/1089497493 conn(0x7f73d80ce9b0 msgr2=0x7f73d80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:42.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.881+0000 7f73dea7c640 1 -- 172.21.15.84:0/1089497493 shutdown_connections 2023-10-16T10:08:42.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:42.881+0000 7f73dea7c640 1 -- 172.21.15.84:0/1089497493 wait complete. 2023-10-16T10:08:42.903 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/daily 2023-10-16T10:08:42.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:42.904 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/weekly) 2023-10-16T10:08:42.956 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/weekly 1w 2023-10-16T10:08:43.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.165+0000 7fc7ce090640 1 Processor -- start 2023-10-16T10:08:43.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.166+0000 7fc7ce090640 1 -- start start 2023-10-16T10:08:43.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.167+0000 7fc7ce090640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc7c8160ad0 0x7fc7c80ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:43.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.167+0000 7fc7ce090640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c80ce800 con 0x7fc7c80ae260 2023-10-16T10:08:43.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.167+0000 7fc7ce090640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c80cdec0 con 0x7fc7c80ce350 2023-10-16T10:08:43.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.167+0000 7fc7ce090640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c80acdd0 con 0x7fc7c8160ad0 2023-10-16T10:08:43.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.167+0000 7fc7c77fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc7c8160ad0 0x7fc7c80ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:43.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.167+0000 7fc7c7fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc7c80ae260 0x7fc7c806a780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55654/0 (socket says 172.21.15.84:55654) 2023-10-16T10:08:43.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.167+0000 7fc7c7fff640 1 -- 172.21.15.84:0/3023792770 learned_addr learned my addr 172.21.15.84:0/3023792770 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:43.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.168+0000 7fc7c77fe640 1 -- 172.21.15.84:0/3023792770 >> v1:172.21.15.157:6789/0 conn(0x7fc7c80ce350 legacy=0x7fc7c8160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:43.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.168+0000 7fc7c77fe640 1 -- 172.21.15.84:0/3023792770 >> v1:172.21.15.84:6789/0 conn(0x7fc7c80ae260 legacy=0x7fc7c806a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:43.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.168+0000 7fc7c77fe640 1 -- 172.21.15.84:0/3023792770 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80ae740 con 0x7fc7c8160ad0 2023-10-16T10:08:43.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.168+0000 7fc7c67fc640 1 -- 172.21.15.84:0/3023792770 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134716179 0 0) 0x7fc7c80ce800 con 0x7fc7c80ae260 2023-10-16T10:08:43.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.169+0000 7fc7c77fe640 1 --2- 172.21.15.84:0/3023792770 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7c8160ad0 0x7fc7c80ac890 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00097a0 tx=0x7fc7b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f091ff9c401e668 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:43.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.169+0000 7fc7c67fc640 1 -- 172.21.15.84:0/3023792770 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0031830 con 0x7fc7c8160ad0 2023-10-16T10:08:43.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.169+0000 7fc7c67fc640 1 -- 172.21.15.84:0/3023792770 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b0031990 con 0x7fc7c8160ad0 2023-10-16T10:08:43.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.169+0000 7fc7c67fc640 1 -- 172.21.15.84:0/3023792770 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0031d20 con 0x7fc7c8160ad0 2023-10-16T10:08:43.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.170+0000 7fc7ce090640 1 -- 172.21.15.84:0/3023792770 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7c8160ad0 msgr2=0x7fc7c80ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:43.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.170+0000 7fc7ce090640 1 --2- 172.21.15.84:0/3023792770 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7c8160ad0 0x7fc7c80ac890 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00097a0 tx=0x7fc7b002b8d0 comp rx=0 tx=0).stop 2023-10-16T10:08:43.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.170+0000 7fc7ce090640 1 -- 172.21.15.84:0/3023792770 shutdown_connections 2023-10-16T10:08:43.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.170+0000 7fc7ce090640 1 --2- 172.21.15.84:0/3023792770 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7c8160ad0 0x7fc7c80ac890 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:43.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.170+0000 7fc7ce090640 1 -- 172.21.15.84:0/3023792770 >> 172.21.15.84:0/3023792770 conn(0x7fc7c806bc10 msgr2=0x7fc7c806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:43.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.170+0000 7fc7ce090640 1 -- 172.21.15.84:0/3023792770 shutdown_connections 2023-10-16T10:08:43.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.170+0000 7fc7ce090640 1 -- 172.21.15.84:0/3023792770 wait complete. 2023-10-16T10:08:43.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.172+0000 7fc7ce090640 1 Processor -- start 2023-10-16T10:08:43.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.172+0000 7fc7ce090640 1 -- start start 2023-10-16T10:08:43.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.173+0000 7fc7ce090640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7c80ae260 0x7fc7c815f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:43.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.173+0000 7fc7ce090640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80ce350 0x7fc7c815b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:43.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.173+0000 7fc7ce090640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7c8160ad0 0x7fc7c80aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:43.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.173+0000 7fc7ce090640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c815e0d0 con 0x7fc7c8160ad0 2023-10-16T10:08:43.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.173+0000 7fc7ce090640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c815e3b0 con 0x7fc7c80ce350 2023-10-16T10:08:43.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.173+0000 7fc7ce090640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c815e690 con 0x7fc7c80ae260 2023-10-16T10:08:43.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.173+0000 7fc7c7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7c8160ad0 0x7fc7c80aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:43.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.173+0000 7fc7c7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7c8160ad0 0x7fc7c80aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53228/0 (socket says 172.21.15.84:53228) 2023-10-16T10:08:43.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.173+0000 7fc7c7fff640 1 -- 172.21.15.84:0/3588034536 learned_addr learned my addr 172.21.15.84:0/3588034536 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:43.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.173+0000 7fc7c77fe640 1 --2- 172.21.15.84:0/3588034536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7c80ae260 0x7fc7c815f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:43.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.173+0000 7fc7c6ffd640 1 --2- 172.21.15.84:0/3588034536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80ce350 0x7fc7c815b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:43.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.173+0000 7fc7c7fff640 1 -- 172.21.15.84:0/3588034536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7c80ae260 msgr2=0x7fc7c815f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:43.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.173+0000 7fc7c7fff640 1 --2- 172.21.15.84:0/3588034536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7c80ae260 0x7fc7c815f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:43.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.173+0000 7fc7c7fff640 1 -- 172.21.15.84:0/3588034536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80ce350 msgr2=0x7fc7c815b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:43.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.173+0000 7fc7c7fff640 1 --2- 172.21.15.84:0/3588034536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80ce350 0x7fc7c815b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:43.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.174+0000 7fc7c7fff640 1 -- 172.21.15.84:0/3588034536 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b0009400 con 0x7fc7c8160ad0 2023-10-16T10:08:43.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.174+0000 7fc7c77fe640 1 --2- 172.21.15.84:0/3588034536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7c80ae260 0x7fc7c815f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:43.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.174+0000 7fc7c7fff640 1 --2- 172.21.15.84:0/3588034536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7c8160ad0 0x7fc7c80aeb10 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c450 tx=0x7fc7c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:43.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.174+0000 7fc7bffff640 1 -- 172.21.15.84:0/3588034536 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c00074e0 con 0x7fc7c8160ad0 2023-10-16T10:08:43.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.174+0000 7fc7bffff640 1 -- 172.21.15.84:0/3588034536 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c0007660 con 0x7fc7c8160ad0 2023-10-16T10:08:43.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.174+0000 7fc7bffff640 1 -- 172.21.15.84:0/3588034536 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0004a20 con 0x7fc7c8160ad0 2023-10-16T10:08:43.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.174+0000 7fc7ce090640 1 -- 172.21.15.84:0/3588034536 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c815e970 con 0x7fc7c8160ad0 2023-10-16T10:08:43.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.174+0000 7fc7ce090640 1 -- 172.21.15.84:0/3588034536 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c815be70 con 0x7fc7c8160ad0 2023-10-16T10:08:43.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.175+0000 7fc7ce090640 1 -- 172.21.15.84:0/3588034536 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c815ee90 con 0x7fc7c8160ad0 2023-10-16T10:08:43.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.176+0000 7fc7bffff640 1 -- 172.21.15.84:0/3588034536 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 75) v1 ==== 89349+0+0 (secure 0 0 0) 0x7fc7c0004b80 con 0x7fc7c8160ad0 2023-10-16T10:08:43.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.178+0000 7fc7bffff640 1 --2- 172.21.15.84:0/3588034536 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fc7a806b010 0x7fc7a806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:43.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.178+0000 7fc7bffff640 1 -- 172.21.15.84:0/3588034536 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7fc7c00889d0 con 0x7fc7c8160ad0 2023-10-16T10:08:43.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.178+0000 7fc7c77fe640 1 --2- 172.21.15.84:0/3588034536 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fc7a806b010 0x7fc7a806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:43.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.179+0000 7fc7c77fe640 1 --2- 172.21.15.84:0/3588034536 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fc7a806b010 0x7fc7a806d4d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0036000 tx=0x7fc7b0008030 comp rx=0 tx=0).ready entity=mgr.8754 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:43.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.182+0000 7fc7bffff640 1 -- 172.21.15.84:0/3588034536 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc7c815ee90 con 0x7fc7c8160ad0 2023-10-16T10:08:43.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.404+0000 7fc7ce090640 1 -- 172.21.15.84:0/3588034536 --> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/weekly", "snap_schedule": "1w", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7c80ced30 con 0x7fc7a806b010 2023-10-16T10:08:43.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.597+0000 7fc7bffff640 1 -- 172.21.15.84:0/3588034536 <== mgr.8754 v2:172.21.15.84:6832/1203729023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+44 (secure 0 0 0) 0x7fc7b0031b80 con 0x7fc7a806b010 2023-10-16T10:08:43.599 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path snap_test_dir1/weekly 2023-10-16T10:08:43.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.602+0000 7fc7ce090640 1 -- 172.21.15.84:0/3588034536 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fc7a806b010 msgr2=0x7fc7a806d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:43.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.602+0000 7fc7ce090640 1 --2- 172.21.15.84:0/3588034536 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fc7a806b010 0x7fc7a806d4d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0036000 tx=0x7fc7b0008030 comp rx=0 tx=0).stop 2023-10-16T10:08:43.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.602+0000 7fc7ce090640 1 -- 172.21.15.84:0/3588034536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7c8160ad0 msgr2=0x7fc7c80aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:43.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.602+0000 7fc7ce090640 1 --2- 172.21.15.84:0/3588034536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7c8160ad0 0x7fc7c80aeb10 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c450 tx=0x7fc7c000c910 comp rx=0 tx=0).stop 2023-10-16T10:08:43.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.602+0000 7fc7ce090640 1 -- 172.21.15.84:0/3588034536 shutdown_connections 2023-10-16T10:08:43.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.602+0000 7fc7ce090640 1 --2- 172.21.15.84:0/3588034536 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fc7a806b010 0x7fc7a806d4d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:43.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.602+0000 7fc7ce090640 1 --2- 172.21.15.84:0/3588034536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7c8160ad0 0x7fc7c80aeb10 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:43.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.602+0000 7fc7ce090640 1 --2- 172.21.15.84:0/3588034536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80ce350 0x7fc7c815b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:43.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.602+0000 7fc7ce090640 1 --2- 172.21.15.84:0/3588034536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7c80ae260 0x7fc7c815f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:43.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.602+0000 7fc7ce090640 1 -- 172.21.15.84:0/3588034536 >> 172.21.15.84:0/3588034536 conn(0x7fc7c806bc10 msgr2=0x7fc7c80afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:43.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.602+0000 7fc7ce090640 1 -- 172.21.15.84:0/3588034536 shutdown_connections 2023-10-16T10:08:43.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.603+0000 7fc7ce090640 1 -- 172.21.15.84:0/3588034536 wait complete. 2023-10-16T10:08:43.628 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/weekly 2023-10-16T10:08:43.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:43.629 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/monthly) 2023-10-16T10:08:43.686 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/monthly 1M 2023-10-16T10:08:43.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.904+0000 7f760c1d5640 1 Processor -- start 2023-10-16T10:08:43.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.904+0000 7f760c1d5640 1 -- start start 2023-10-16T10:08:43.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.905+0000 7f760c1d5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f76040caac0 0x7f76040ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:43.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.905+0000 7f760c1d5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76040cb030 con 0x7f760415ad70 2023-10-16T10:08:43.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.905+0000 7f760c1d5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76040c91c0 con 0x7f760406a4b0 2023-10-16T10:08:43.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.905+0000 7f760c1d5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76040c94a0 con 0x7f76040caac0 2023-10-16T10:08:43.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.906+0000 7f7609f71640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f760415ad70 0x7f7604069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55670/0 (socket says 172.21.15.84:55670) 2023-10-16T10:08:43.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.906+0000 7f7609f71640 1 -- 172.21.15.84:0/702202807 learned_addr learned my addr 172.21.15.84:0/702202807 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:43.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.906+0000 7f7608f6f640 1 -- 172.21.15.84:0/702202807 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875833233 0 0) 0x7f76040cb030 con 0x7f760415ad70 2023-10-16T10:08:43.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.906+0000 7f760a772640 1 --2- 172.21.15.84:0/702202807 >> v2:172.21.15.157:3300/0 conn(0x7f76040caac0 0x7f76040ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:43.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.906+0000 7f7608f6f640 1 -- 172.21.15.84:0/702202807 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75f0003520 con 0x7f760415ad70 2023-10-16T10:08:43.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.906+0000 7f7608f6f640 1 -- 172.21.15.84:0/702202807 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 855784186 0 0) 0x7f75f0003520 con 0x7f760415ad70 2023-10-16T10:08:43.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.906+0000 7f7608f6f640 1 -- 172.21.15.84:0/702202807 >> v2:172.21.15.157:3300/0 conn(0x7f76040caac0 msgr2=0x7f76040ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:43.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.906+0000 7f7608f6f640 1 --2- 172.21.15.84:0/702202807 >> v2:172.21.15.157:3300/0 conn(0x7f76040caac0 0x7f76040ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:43.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.906+0000 7f7608f6f640 1 -- 172.21.15.84:0/702202807 >> v1:172.21.15.157:6789/0 conn(0x7f760406a4b0 legacy=0x7f76040ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:43.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.907+0000 7f7608f6f640 1 -- 172.21.15.84:0/702202807 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76040c9780 con 0x7f760415ad70 2023-10-16T10:08:43.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.907+0000 7f7608f6f640 1 -- 172.21.15.84:0/702202807 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f7600001670 con 0x7f760415ad70 2023-10-16T10:08:43.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.907+0000 7f7608f6f640 1 -- 172.21.15.84:0/702202807 >> v1:172.21.15.84:6789/0 conn(0x7f760415ad70 legacy=0x7f7604069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:43.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.907+0000 7f7608f6f640 1 --2- 172.21.15.84:0/702202807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f00039c0 0x7f75f0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:43.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.907+0000 7f7608f6f640 1 --2- 172.21.15.84:0/702202807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f00047c0 0x7f75f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:43.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.907+0000 7f7608f6f640 1 --2- 172.21.15.84:0/702202807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75f00051d0 0x7f75f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:43.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.907+0000 7f7608f6f640 1 -- 172.21.15.84:0/702202807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76040c94a0 con 0x7f75f00039c0 2023-10-16T10:08:43.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.907+0000 7f7608f6f640 1 -- 172.21.15.84:0/702202807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f000c100 con 0x7f75f00047c0 2023-10-16T10:08:43.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.907+0000 7f7608f6f640 1 -- 172.21.15.84:0/702202807 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f000c2c0 con 0x7f75f00051d0 2023-10-16T10:08:43.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.907+0000 7f7609f71640 1 --2- 172.21.15.84:0/702202807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f00039c0 0x7f75f0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:43.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.907+0000 7f7609f71640 1 -- 172.21.15.84:0/702202807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75f00051d0 msgr2=0x7f75f000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:43.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.907+0000 7f760a772640 1 --2- 172.21.15.84:0/702202807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75f00051d0 0x7f75f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:43.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.907+0000 7f7609770640 1 --2- 172.21.15.84:0/702202807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f00047c0 0x7f75f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:43.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.907+0000 7f7609f71640 1 --2- 172.21.15.84:0/702202807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75f00051d0 0x7f75f000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:43.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.907+0000 7f7609f71640 1 -- 172.21.15.84:0/702202807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f00047c0 msgr2=0x7f75f0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:43.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.907+0000 7f7609f71640 1 --2- 172.21.15.84:0/702202807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f00047c0 0x7f75f0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:43.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.907+0000 7f7609f71640 1 -- 172.21.15.84:0/702202807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f0003520 con 0x7f75f00039c0 2023-10-16T10:08:43.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.907+0000 7f760a772640 1 -- 172.21.15.84:0/702202807 reap_dead start 2023-10-16T10:08:43.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.908+0000 7f7609f71640 1 --2- 172.21.15.84:0/702202807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f00039c0 0x7f75f0003dc0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f760000c160 tx=0x7f75f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873032813dc8f280 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:43.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.909+0000 7f7608f6f640 1 -- 172.21.15.84:0/702202807 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7600005ad0 con 0x7f75f00039c0 2023-10-16T10:08:43.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.909+0000 7f7608f6f640 1 -- 172.21.15.84:0/702202807 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7600005c30 con 0x7f75f00039c0 2023-10-16T10:08:43.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.909+0000 7f7608f6f640 1 -- 172.21.15.84:0/702202807 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760002c790 con 0x7f75f00039c0 2023-10-16T10:08:43.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.909+0000 7f760c1d5640 1 -- 172.21.15.84:0/702202807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f00039c0 msgr2=0x7f75f0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:43.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.909+0000 7f760c1d5640 1 --2- 172.21.15.84:0/702202807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f00039c0 0x7f75f0003dc0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f760000c160 tx=0x7f75f00057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:43.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.909+0000 7f760c1d5640 1 -- 172.21.15.84:0/702202807 shutdown_connections 2023-10-16T10:08:43.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.909+0000 7f760c1d5640 1 --2- 172.21.15.84:0/702202807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f00039c0 0x7f75f0003dc0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:43.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.909+0000 7f760c1d5640 1 -- 172.21.15.84:0/702202807 >> 172.21.15.84:0/702202807 conn(0x7f76040ce9b0 msgr2=0x7f76040cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:43.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.910+0000 7f760c1d5640 1 -- 172.21.15.84:0/702202807 shutdown_connections 2023-10-16T10:08:43.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.910+0000 7f760c1d5640 1 -- 172.21.15.84:0/702202807 wait complete. 2023-10-16T10:08:43.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.911+0000 7f760c1d5640 1 Processor -- start 2023-10-16T10:08:43.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.911+0000 7f760c1d5640 1 -- start start 2023-10-16T10:08:43.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.912+0000 7f760c1d5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f00039c0 0x7f760414f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:43.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.912+0000 7f760c1d5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76040c74e0 0x7f760415bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:43.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.912+0000 7f760c1d5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76040ced90 0x7f760415c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:43.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.912+0000 7f760c1d5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76040c8af0 con 0x7f75f00039c0 2023-10-16T10:08:43.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.912+0000 7f760c1d5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76040c8660 con 0x7f76040c74e0 2023-10-16T10:08:43.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.912+0000 7f760c1d5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760414f620 con 0x7f76040ced90 2023-10-16T10:08:43.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.913+0000 7f7609f71640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f00039c0 0x7f760414f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:43.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.913+0000 7f7609f71640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f00039c0 0x7f760414f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53256/0 (socket says 172.21.15.84:53256) 2023-10-16T10:08:43.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.913+0000 7f7609f71640 1 -- 172.21.15.84:0/3020698924 learned_addr learned my addr 172.21.15.84:0/3020698924 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:43.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.913+0000 7f7609f71640 1 -- 172.21.15.84:0/3020698924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76040ced90 msgr2=0x7f760415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:43.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.913+0000 7f760a772640 1 --2- 172.21.15.84:0/3020698924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76040ced90 0x7f760415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:43.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.913+0000 7f7609770640 1 --2- 172.21.15.84:0/3020698924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76040c74e0 0x7f760415bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:43.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.913+0000 7f7609f71640 1 --2- 172.21.15.84:0/3020698924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76040ced90 0x7f760415c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:43.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.913+0000 7f7609f71640 1 -- 172.21.15.84:0/3020698924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76040c74e0 msgr2=0x7f760415bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:43.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.913+0000 7f7609f71640 1 --2- 172.21.15.84:0/3020698924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76040c74e0 0x7f760415bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:43.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.913+0000 7f7609f71640 1 -- 172.21.15.84:0/3020698924 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76040c7a50 con 0x7f75f00039c0 2023-10-16T10:08:43.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.913+0000 7f7609f71640 1 --2- 172.21.15.84:0/3020698924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f00039c0 0x7f760414f0e0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f760000c280 tx=0x7f760002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:43.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.914+0000 7f75fa7fc640 1 -- 172.21.15.84:0/3020698924 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760003c040 con 0x7f75f00039c0 2023-10-16T10:08:43.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.914+0000 7f75fa7fc640 1 -- 172.21.15.84:0/3020698924 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760002cc60 con 0x7f75f00039c0 2023-10-16T10:08:43.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.914+0000 7f75fa7fc640 1 -- 172.21.15.84:0/3020698924 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7600007070 con 0x7f75f00039c0 2023-10-16T10:08:43.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.914+0000 7f760c1d5640 1 -- 172.21.15.84:0/3020698924 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76040c7d30 con 0x7f75f00039c0 2023-10-16T10:08:43.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.914+0000 7f760c1d5640 1 -- 172.21.15.84:0/3020698924 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76040c82f0 con 0x7f75f00039c0 2023-10-16T10:08:43.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.915+0000 7f760c1d5640 1 -- 172.21.15.84:0/3020698924 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76040c84b0 con 0x7f75f00039c0 2023-10-16T10:08:43.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.916+0000 7f75fa7fc640 1 -- 172.21.15.84:0/3020698924 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 75) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f7600032070 con 0x7f75f00039c0 2023-10-16T10:08:43.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.917+0000 7f75fa7fc640 1 --2- 172.21.15.84:0/3020698924 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f75e806b010 0x7f75e806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:43.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.917+0000 7f75fa7fc640 1 -- 172.21.15.84:0/3020698924 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f76000aae30 con 0x7f75f00039c0 2023-10-16T10:08:43.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.917+0000 7f7609770640 1 --2- 172.21.15.84:0/3020698924 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f75e806b010 0x7f75e806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:43.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.918+0000 7f7609770640 1 --2- 172.21.15.84:0/3020698924 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f75e806b010 0x7f75e806d4d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f760415c8d0 tx=0x7f75f4009040 comp rx=0 tx=0).ready entity=mgr.8754 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:43.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:43.921+0000 7f75fa7fc640 1 -- 172.21.15.84:0/3020698924 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7600075d00 con 0x7f75f00039c0 2023-10-16T10:08:44.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.134+0000 7f760c1d5640 1 -- 172.21.15.84:0/3020698924 --> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/monthly", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f76040c9ca0 con 0x7f75e806b010 2023-10-16T10:08:44.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.284+0000 7f75fa7fc640 1 -- 172.21.15.84:0/3020698924 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 76) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f76000a91d0 con 0x7f75f00039c0 2023-10-16T10:08:44.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.331+0000 7f75fa7fc640 1 -- 172.21.15.84:0/3020698924 <== mgr.8754 v2:172.21.15.84:6832/1203729023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+45 (secure 0 0 0) 0x7f75f400d040 con 0x7f75e806b010 2023-10-16T10:08:44.333 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path snap_test_dir1/monthly 2023-10-16T10:08:44.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.334+0000 7f760c1d5640 1 -- 172.21.15.84:0/3020698924 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f75e806b010 msgr2=0x7f75e806d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:44.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.334+0000 7f760c1d5640 1 --2- 172.21.15.84:0/3020698924 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f75e806b010 0x7f75e806d4d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f760415c8d0 tx=0x7f75f4009040 comp rx=0 tx=0).stop 2023-10-16T10:08:44.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.334+0000 7f760c1d5640 1 -- 172.21.15.84:0/3020698924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f00039c0 msgr2=0x7f760414f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:44.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.334+0000 7f760c1d5640 1 --2- 172.21.15.84:0/3020698924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f00039c0 0x7f760414f0e0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f760000c280 tx=0x7f760002e070 comp rx=0 tx=0).stop 2023-10-16T10:08:44.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.334+0000 7f760c1d5640 1 -- 172.21.15.84:0/3020698924 shutdown_connections 2023-10-16T10:08:44.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.334+0000 7f760c1d5640 1 --2- 172.21.15.84:0/3020698924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76040ced90 0x7f760415c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:44.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.334+0000 7f760c1d5640 1 --2- 172.21.15.84:0/3020698924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76040c74e0 0x7f760415bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:44.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.334+0000 7f760c1d5640 1 --2- 172.21.15.84:0/3020698924 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f75e806b010 0x7f75e806d4d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:44.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.334+0000 7f760c1d5640 1 --2- 172.21.15.84:0/3020698924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75f00039c0 0x7f760414f0e0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:44.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.334+0000 7f760c1d5640 1 -- 172.21.15.84:0/3020698924 >> 172.21.15.84:0/3020698924 conn(0x7f76040ce9b0 msgr2=0x7f7604069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:44.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.334+0000 7f760c1d5640 1 -- 172.21.15.84:0/3020698924 shutdown_connections 2023-10-16T10:08:44.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.334+0000 7f760c1d5640 1 -- 172.21.15.84:0/3020698924 wait complete. 2023-10-16T10:08:44.356 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/monthly 2023-10-16T10:08:44.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:44.357 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/yearly) 2023-10-16T10:08:44.413 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/yearly 1Y 2023-10-16T10:08:44.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.621+0000 7f0cf45e9640 1 Processor -- start 2023-10-16T10:08:44.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.621+0000 7f0cf45e9640 1 -- start start 2023-10-16T10:08:44.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.623+0000 7f0cf45e9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0cec0caa80 0x7f0cec069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:44.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.623+0000 7f0cf45e9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cec06a2e0 0x7f0cec1650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:44.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.623+0000 7f0cf45e9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cec06a730 con 0x7f0cec0addb0 2023-10-16T10:08:44.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.623+0000 7f0cf45e9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cec161a50 con 0x7f0cec0caa80 2023-10-16T10:08:44.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.623+0000 7f0cf45e9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec0caef0 con 0x7f0cec06a2e0 2023-10-16T10:08:44.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.623+0000 7f0cf2385640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0cec0addb0 0x7f0cec0c9000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55682/0 (socket says 172.21.15.84:55682) 2023-10-16T10:08:44.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.623+0000 7f0cf2385640 1 -- 172.21.15.84:0/1545351683 learned_addr learned my addr 172.21.15.84:0/1545351683 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:44.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.623+0000 7f0cf2b86640 1 --2- 172.21.15.84:0/1545351683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cec06a2e0 0x7f0cec1650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:44.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.623+0000 7f0cf1b84640 1 --2- 172.21.15.84:0/1545351683 >> v2:172.21.15.157:3300/0 conn(0x7f0cec0caa80 0x7f0cec069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:44.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.623+0000 7f0cf1383640 1 -- 172.21.15.84:0/1545351683 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1874330375 0 0) 0x7f0ce8001110 con 0x7f0cec0addb0 2023-10-16T10:08:44.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.624+0000 7f0cf1383640 1 -- 172.21.15.84:0/1545351683 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cd8003520 con 0x7f0cec0addb0 2023-10-16T10:08:44.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.624+0000 7f0cf1383640 1 -- 172.21.15.84:0/1545351683 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 630616739 0 0) 0x7f0cd8003520 con 0x7f0cec0addb0 2023-10-16T10:08:44.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.624+0000 7f0cf1383640 1 -- 172.21.15.84:0/1545351683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cec06a2e0 msgr2=0x7f0cec1650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:44.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.624+0000 7f0cf1383640 1 --2- 172.21.15.84:0/1545351683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cec06a2e0 0x7f0cec1650d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:44.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.624+0000 7f0cf1383640 1 -- 172.21.15.84:0/1545351683 >> v2:172.21.15.157:3300/0 conn(0x7f0cec0caa80 msgr2=0x7f0cec069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:44.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.624+0000 7f0cf1383640 1 --2- 172.21.15.84:0/1545351683 >> v2:172.21.15.157:3300/0 conn(0x7f0cec0caa80 0x7f0cec069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:44.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.624+0000 7f0cf1383640 1 -- 172.21.15.84:0/1545351683 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cec0cb140 con 0x7f0cec0addb0 2023-10-16T10:08:44.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.624+0000 7f0cf1383640 1 -- 172.21.15.84:0/1545351683 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f0ce8001800 con 0x7f0cec0addb0 2023-10-16T10:08:44.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.624+0000 7f0cf1383640 1 -- 172.21.15.84:0/1545351683 >> v1:172.21.15.84:6789/0 conn(0x7f0cec0addb0 legacy=0x7f0cec0c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:44.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.624+0000 7f0cf1383640 1 --2- 172.21.15.84:0/1545351683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cd8003970 0x7f0cd8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:44.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.624+0000 7f0cf1383640 1 --2- 172.21.15.84:0/1545351683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cd80047c0 0x7f0cd8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:44.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.624+0000 7f0cf1383640 1 --2- 172.21.15.84:0/1545351683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cec06a2e0 0x7f0cd8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:44.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.624+0000 7f0cf1383640 1 -- 172.21.15.84:0/1545351683 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec161a50 con 0x7f0cd80047c0 2023-10-16T10:08:44.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.624+0000 7f0cf1383640 1 -- 172.21.15.84:0/1545351683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec0caef0 con 0x7f0cd8003970 2023-10-16T10:08:44.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.624+0000 7f0cf1383640 1 -- 172.21.15.84:0/1545351683 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd800b980 con 0x7f0cec06a2e0 2023-10-16T10:08:44.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.624+0000 7f0cf1b84640 1 --2- 172.21.15.84:0/1545351683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cd80047c0 0x7f0cd8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:44.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.625+0000 7f0cf2385640 1 --2- 172.21.15.84:0/1545351683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cd8003970 0x7f0cd8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.625+0000 7f0cf1b84640 1 -- 172.21.15.84:0/1545351683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cec06a2e0 msgr2=0x7f0cd8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.625+0000 7f0cf1b84640 1 --2- 172.21.15.84:0/1545351683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cec06a2e0 0x7f0cd8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.625+0000 7f0cf1b84640 1 -- 172.21.15.84:0/1545351683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cd8003970 msgr2=0x7f0cd8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:44.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.625+0000 7f0cf1b84640 1 --2- 172.21.15.84:0/1545351683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cd8003970 0x7f0cd8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:44.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.625+0000 7f0cf1b84640 1 -- 172.21.15.84:0/1545351683 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd8003520 con 0x7f0cd80047c0 2023-10-16T10:08:44.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.625+0000 7f0cf2385640 1 --2- 172.21.15.84:0/1545351683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cd8003970 0x7f0cd8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:44.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.626+0000 7f0cf1b84640 1 --2- 172.21.15.84:0/1545351683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cd80047c0 0x7f0cd8004c40 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009ac0 tx=0x7f0cd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431b27adcb288c15 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:44.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.626+0000 7f0cf1383640 1 -- 172.21.15.84:0/1545351683 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc02c6d0 con 0x7f0cd80047c0 2023-10-16T10:08:44.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.626+0000 7f0cf1383640 1 -- 172.21.15.84:0/1545351683 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc02c830 con 0x7f0cd80047c0 2023-10-16T10:08:44.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.626+0000 7f0cf1383640 1 -- 172.21.15.84:0/1545351683 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc02cbc0 con 0x7f0cd80047c0 2023-10-16T10:08:44.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.627+0000 7f0cf45e9640 1 -- 172.21.15.84:0/1545351683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cd80047c0 msgr2=0x7f0cd8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:44.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.627+0000 7f0cf45e9640 1 --2- 172.21.15.84:0/1545351683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cd80047c0 0x7f0cd8004c40 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009ac0 tx=0x7f0cd80057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:44.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.627+0000 7f0cf2b86640 1 -- 172.21.15.84:0/1545351683 reap_dead start 2023-10-16T10:08:44.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.627+0000 7f0cf45e9640 1 -- 172.21.15.84:0/1545351683 shutdown_connections 2023-10-16T10:08:44.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.627+0000 7f0cf45e9640 1 -- 172.21.15.84:0/1545351683 >> 172.21.15.84:0/1545351683 conn(0x7f0cec0ce9b0 msgr2=0x7f0cec0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:44.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.627+0000 7f0cf45e9640 1 -- 172.21.15.84:0/1545351683 shutdown_connections 2023-10-16T10:08:44.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.627+0000 7f0cf45e9640 1 -- 172.21.15.84:0/1545351683 wait complete. 2023-10-16T10:08:44.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.629+0000 7f0cf45e9640 1 Processor -- start 2023-10-16T10:08:44.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.629+0000 7f0cf45e9640 1 -- start start 2023-10-16T10:08:44.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.630+0000 7f0cf45e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cd80047c0 0x7f0cec0c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:44.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.630+0000 7f0cf45e9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cec070590 0x7f0cec070a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:44.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.630+0000 7f0cf45e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cec0c5ed0 0x7f0cec0c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:44.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.630+0000 7f0cf45e9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec0c89a0 con 0x7f0cd80047c0 2023-10-16T10:08:44.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.630+0000 7f0cf45e9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec070f50 con 0x7f0cec0c5ed0 2023-10-16T10:08:44.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.630+0000 7f0cf45e9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec0c7d20 con 0x7f0cec070590 2023-10-16T10:08:44.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.630+0000 7f0cf2385640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cd80047c0 0x7f0cec0c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:44.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.630+0000 7f0cf2385640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cd80047c0 0x7f0cec0c7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53276/0 (socket says 172.21.15.84:53276) 2023-10-16T10:08:44.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.630+0000 7f0cf2385640 1 -- 172.21.15.84:0/542852965 learned_addr learned my addr 172.21.15.84:0/542852965 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:44.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.630+0000 7f0cf2385640 1 -- 172.21.15.84:0/542852965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cec070590 msgr2=0x7f0cec070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:44.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.630+0000 7f0cf2385640 1 --2- 172.21.15.84:0/542852965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cec070590 0x7f0cec070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:44.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.630+0000 7f0cf2385640 1 -- 172.21.15.84:0/542852965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cec0c5ed0 msgr2=0x7f0cec0c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:44.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.630+0000 7f0cf2385640 1 --2- 172.21.15.84:0/542852965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cec0c5ed0 0x7f0cec0c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:44.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.631+0000 7f0cf2385640 1 -- 172.21.15.84:0/542852965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cec0c8000 con 0x7f0cd80047c0 2023-10-16T10:08:44.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.631+0000 7f0cf2385640 1 --2- 172.21.15.84:0/542852965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cd80047c0 0x7f0cec0c7640 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800d7c0 tx=0x7f0ce800dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:44.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.631+0000 7f0ce2ffd640 1 -- 172.21.15.84:0/542852965 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce80074e0 con 0x7f0cd80047c0 2023-10-16T10:08:44.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.631+0000 7f0ce2ffd640 1 -- 172.21.15.84:0/542852965 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce8007640 con 0x7f0cd80047c0 2023-10-16T10:08:44.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.631+0000 7f0ce2ffd640 1 -- 172.21.15.84:0/542852965 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce80049e0 con 0x7f0cd80047c0 2023-10-16T10:08:44.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.631+0000 7f0cf45e9640 1 -- 172.21.15.84:0/542852965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cec0c82e0 con 0x7f0cd80047c0 2023-10-16T10:08:44.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.631+0000 7f0cf45e9640 1 -- 172.21.15.84:0/542852965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cec0c6890 con 0x7f0cd80047c0 2023-10-16T10:08:44.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.632+0000 7f0cf45e9640 1 -- 172.21.15.84:0/542852965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cec0c8c10 con 0x7f0cd80047c0 2023-10-16T10:08:44.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.633+0000 7f0ce2ffd640 1 -- 172.21.15.84:0/542852965 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 76) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f0ce8004b40 con 0x7f0cd80047c0 2023-10-16T10:08:44.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.634+0000 7f0ce2ffd640 1 --2- 172.21.15.84:0/542852965 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f0cd006b070 0x7f0cd006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:44.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.634+0000 7f0ce2ffd640 1 -- 172.21.15.84:0/542852965 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f0ce8089360 con 0x7f0cd80047c0 2023-10-16T10:08:44.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.634+0000 7f0cf1b84640 1 --2- 172.21.15.84:0/542852965 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f0cd006b070 0x7f0cd006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:44.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.635+0000 7f0cf1b84640 1 --2- 172.21.15.84:0/542852965 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f0cd006b070 0x7f0cd006d530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc036000 tx=0x7f0cdc02e000 comp rx=0 tx=0).ready entity=mgr.8754 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:44.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.638+0000 7f0ce2ffd640 1 -- 172.21.15.84:0/542852965 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0ce8053f90 con 0x7f0cd80047c0 2023-10-16T10:08:44.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:44.852+0000 7f0cf45e9640 1 -- 172.21.15.84:0/542852965 --> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/yearly", "snap_schedule": "1Y", "target": ["mon-mgr", ""]}) v1 -- 0x7f0cec06b310 con 0x7f0cd006b070 2023-10-16T10:08:45.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.040+0000 7f0ce2ffd640 1 -- 172.21.15.84:0/542852965 <== mgr.8754 v2:172.21.15.84:6832/1203729023 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+44 (secure 0 0 0) 0x7f0cdc004040 con 0x7f0cd006b070 2023-10-16T10:08:45.042 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path snap_test_dir1/yearly 2023-10-16T10:08:45.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.043+0000 7f0cf45e9640 1 -- 172.21.15.84:0/542852965 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f0cd006b070 msgr2=0x7f0cd006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:45.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.043+0000 7f0cf45e9640 1 --2- 172.21.15.84:0/542852965 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f0cd006b070 0x7f0cd006d530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc036000 tx=0x7f0cdc02e000 comp rx=0 tx=0).stop 2023-10-16T10:08:45.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.043+0000 7f0cf45e9640 1 -- 172.21.15.84:0/542852965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cd80047c0 msgr2=0x7f0cec0c7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:45.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.043+0000 7f0cf45e9640 1 --2- 172.21.15.84:0/542852965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cd80047c0 0x7f0cec0c7640 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800d7c0 tx=0x7f0ce800dc80 comp rx=0 tx=0).stop 2023-10-16T10:08:45.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.044+0000 7f0cf45e9640 1 -- 172.21.15.84:0/542852965 shutdown_connections 2023-10-16T10:08:45.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.044+0000 7f0cf45e9640 1 --2- 172.21.15.84:0/542852965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cec0c5ed0 0x7f0cec0c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:45.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.044+0000 7f0cf45e9640 1 --2- 172.21.15.84:0/542852965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0cec070590 0x7f0cec070a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:45.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.044+0000 7f0cf45e9640 1 --2- 172.21.15.84:0/542852965 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f0cd006b070 0x7f0cd006d530 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:45.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.044+0000 7f0cf45e9640 1 --2- 172.21.15.84:0/542852965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cd80047c0 0x7f0cec0c7640 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:45.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.044+0000 7f0cf45e9640 1 -- 172.21.15.84:0/542852965 >> 172.21.15.84:0/542852965 conn(0x7f0cec0ce9b0 msgr2=0x7f0cec0cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:45.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.044+0000 7f0cf45e9640 1 -- 172.21.15.84:0/542852965 shutdown_connections 2023-10-16T10:08:45.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.044+0000 7f0cf45e9640 1 -- 172.21.15.84:0/542852965 wait complete. 2023-10-16T10:08:45.065 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/yearly 2023-10-16T10:08:45.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:45.066 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/bad_period_spec) 2023-10-16T10:08:45.117 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec 1X 2023-10-16T10:08:45.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.326+0000 7f1f4426f640 1 Processor -- start 2023-10-16T10:08:45.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.327+0000 7f1f4426f640 1 -- start start 2023-10-16T10:08:45.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.328+0000 7f1f4426f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c15d150 0x7f1f3c0ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:45.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.328+0000 7f1f4426f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1f3c160b90 0x7f1f3c160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:45.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.328+0000 7f1f4426f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1f3c069da0 0x7f1f3c06a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:45.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.328+0000 7f1f4426f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f3c15d530 con 0x7f1f3c069da0 2023-10-16T10:08:45.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.328+0000 7f1f4426f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f3c06a6f0 con 0x7f1f3c160b90 2023-10-16T10:08:45.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.328+0000 7f1f4426f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c0ac890 con 0x7f1f3c15d150 2023-10-16T10:08:45.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.329+0000 7f1f4280c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1f3c069da0 0x7f1f3c06a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:45.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.329+0000 7f1f4280c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1f3c069da0 0x7f1f3c06a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53288/0 (socket says 172.21.15.84:53288) 2023-10-16T10:08:45.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.329+0000 7f1f4280c640 1 -- 172.21.15.84:0/1207415886 learned_addr learned my addr 172.21.15.84:0/1207415886 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:45.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.329+0000 7f1f4200b640 1 --2- 172.21.15.84:0/1207415886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c15d150 0x7f1f3c0ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:45.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.329+0000 7f1f4180a640 1 --2- 172.21.15.84:0/1207415886 >> v2:172.21.15.157:3300/0 conn(0x7f1f3c160b90 0x7f1f3c160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:45.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.329+0000 7f1f4280c640 1 -- 172.21.15.84:0/1207415886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c15d150 msgr2=0x7f1f3c0ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:45.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.329+0000 7f1f4280c640 1 --2- 172.21.15.84:0/1207415886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c15d150 0x7f1f3c0ad350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:45.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.329+0000 7f1f4280c640 1 -- 172.21.15.84:0/1207415886 >> v2:172.21.15.157:3300/0 conn(0x7f1f3c160b90 msgr2=0x7f1f3c160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:45.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.329+0000 7f1f4280c640 1 --2- 172.21.15.84:0/1207415886 >> v2:172.21.15.157:3300/0 conn(0x7f1f3c160b90 0x7f1f3c160fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:45.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.329+0000 7f1f4280c640 1 -- 172.21.15.84:0/1207415886 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3c161650 con 0x7f1f3c069da0 2023-10-16T10:08:45.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.330+0000 7f1f4280c640 1 --2- 172.21.15.84:0/1207415886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c069da0 0x7f1f3c06a1b0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0cf820 tx=0x7f1f3402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47eb325424f63cc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:45.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.330+0000 7f1f41009640 1 -- 172.21.15.84:0/1207415886 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34031870 con 0x7f1f3c069da0 2023-10-16T10:08:45.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.331+0000 7f1f41009640 1 -- 172.21.15.84:0/1207415886 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f340319d0 con 0x7f1f3c069da0 2023-10-16T10:08:45.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.331+0000 7f1f41009640 1 -- 172.21.15.84:0/1207415886 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34031d60 con 0x7f1f3c069da0 2023-10-16T10:08:45.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.331+0000 7f1f4426f640 1 -- 172.21.15.84:0/1207415886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c069da0 msgr2=0x7f1f3c06a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:45.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.331+0000 7f1f4426f640 1 --2- 172.21.15.84:0/1207415886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c069da0 0x7f1f3c06a1b0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0cf820 tx=0x7f1f3402b910 comp rx=0 tx=0).stop 2023-10-16T10:08:45.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.331+0000 7f1f4426f640 1 -- 172.21.15.84:0/1207415886 shutdown_connections 2023-10-16T10:08:45.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.331+0000 7f1f4426f640 1 --2- 172.21.15.84:0/1207415886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c069da0 0x7f1f3c06a1b0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:45.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.331+0000 7f1f4426f640 1 --2- 172.21.15.84:0/1207415886 >> v2:172.21.15.157:3300/0 conn(0x7f1f3c160b90 0x7f1f3c160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:45.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.331+0000 7f1f4426f640 1 --2- 172.21.15.84:0/1207415886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c15d150 0x7f1f3c0ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:45.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.331+0000 7f1f4426f640 1 -- 172.21.15.84:0/1207415886 >> 172.21.15.84:0/1207415886 conn(0x7f1f3c0ce9b0 msgr2=0x7f1f3c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:45.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.332+0000 7f1f4426f640 1 -- 172.21.15.84:0/1207415886 shutdown_connections 2023-10-16T10:08:45.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.332+0000 7f1f4426f640 1 -- 172.21.15.84:0/1207415886 wait complete. 2023-10-16T10:08:45.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.333+0000 7f1f4426f640 1 Processor -- start 2023-10-16T10:08:45.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.334+0000 7f1f4426f640 1 -- start start 2023-10-16T10:08:45.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.334+0000 7f1f4426f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f3c069da0 0x7f1f3c15a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:45.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.334+0000 7f1f4426f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c15d150 0x7f1f3c0ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:45.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.334+0000 7f1f4426f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c160b90 0x7f1f3c0ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:45.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.335+0000 7f1f4426f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c15e9b0 con 0x7f1f3c160b90 2023-10-16T10:08:45.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.335+0000 7f1f4426f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c15ec90 con 0x7f1f3c069da0 2023-10-16T10:08:45.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.335+0000 7f1f4426f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c15ef70 con 0x7f1f3c15d150 2023-10-16T10:08:45.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.335+0000 7f1f4280c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c160b90 0x7f1f3c0ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:45.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.335+0000 7f1f4280c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c160b90 0x7f1f3c0ae780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53290/0 (socket says 172.21.15.84:53290) 2023-10-16T10:08:45.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.335+0000 7f1f4280c640 1 -- 172.21.15.84:0/1736941011 learned_addr learned my addr 172.21.15.84:0/1736941011 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:45.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.335+0000 7f1f4200b640 1 --2- 172.21.15.84:0/1736941011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f3c069da0 0x7f1f3c15a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:45.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.335+0000 7f1f4180a640 1 --2- 172.21.15.84:0/1736941011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c15d150 0x7f1f3c0ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:45.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.335+0000 7f1f4280c640 1 -- 172.21.15.84:0/1736941011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c15d150 msgr2=0x7f1f3c0ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:45.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.335+0000 7f1f4280c640 1 --2- 172.21.15.84:0/1736941011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c15d150 0x7f1f3c0ae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:45.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.335+0000 7f1f4280c640 1 -- 172.21.15.84:0/1736941011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f3c069da0 msgr2=0x7f1f3c15a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:45.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.335+0000 7f1f4280c640 1 --2- 172.21.15.84:0/1736941011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f3c069da0 0x7f1f3c15a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:45.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.335+0000 7f1f4280c640 1 -- 172.21.15.84:0/1736941011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f340094d0 con 0x7f1f3c160b90 2023-10-16T10:08:45.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.335+0000 7f1f4180a640 1 --2- 172.21.15.84:0/1736941011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f3c15d150 0x7f1f3c0ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:45.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.336+0000 7f1f4280c640 1 --2- 172.21.15.84:0/1736941011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f3c160b90 0x7f1f3c0ae780 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f1f340090b0 tx=0x7f1f340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:45.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.336+0000 7f1f3affd640 1 -- 172.21.15.84:0/1736941011 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34031800 con 0x7f1f3c160b90 2023-10-16T10:08:45.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.336+0000 7f1f3affd640 1 -- 172.21.15.84:0/1736941011 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f340443f0 con 0x7f1f3c160b90 2023-10-16T10:08:45.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.336+0000 7f1f4426f640 1 -- 172.21.15.84:0/1736941011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3c15f250 con 0x7f1f3c160b90 2023-10-16T10:08:45.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.336+0000 7f1f3affd640 1 -- 172.21.15.84:0/1736941011 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f340447a0 con 0x7f1f3c160b90 2023-10-16T10:08:45.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.336+0000 7f1f4426f640 1 -- 172.21.15.84:0/1736941011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3c15e4f0 con 0x7f1f3c160b90 2023-10-16T10:08:45.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.337+0000 7f1f4426f640 1 -- 172.21.15.84:0/1736941011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3c15f770 con 0x7f1f3c160b90 2023-10-16T10:08:45.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.338+0000 7f1f3affd640 1 -- 172.21.15.84:0/1736941011 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 76) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f1f3404d450 con 0x7f1f3c160b90 2023-10-16T10:08:45.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.340+0000 7f1f3affd640 1 --2- 172.21.15.84:0/1736941011 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f1f2806b070 0x7f1f2806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:45.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.340+0000 7f1f3affd640 1 -- 172.21.15.84:0/1736941011 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f1f3402d040 con 0x7f1f3c160b90 2023-10-16T10:08:45.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.340+0000 7f1f4200b640 1 --2- 172.21.15.84:0/1736941011 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f1f2806b070 0x7f1f2806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:45.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.341+0000 7f1f4200b640 1 --2- 172.21.15.84:0/1736941011 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f1f2806b070 0x7f1f2806d530 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c007fd0 tx=0x7f1f2c007560 comp rx=0 tx=0).ready entity=mgr.8754 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:45.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.344+0000 7f1f3affd640 1 -- 172.21.15.84:0/1736941011 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1f3c15f770 con 0x7f1f3c160b90 2023-10-16T10:08:45.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.557+0000 7f1f4426f640 1 -- 172.21.15.84:0/1736941011 --> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "1X", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f3c0acb50 con 0x7f1f2806b070 2023-10-16T10:08:45.560 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:45.559+0000 7fb27e1dd640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - multiplier should be one of h,d,w,M,Y 2023-10-16T10:08:45.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.559+0000 7f1f3affd640 1 -- 172.21.15.84:0/1736941011 <== mgr.8754 v2:172.21.15.84:6832/1203729023 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - multiplier should be one of h,d,w,M,Y) v1 ==== 74+0+0 (secure 0 0 0) 0x7f1f2c013070 con 0x7f1f2806b070 2023-10-16T10:08:45.561 INFO:teuthology.orchestra.run.smithi084.stderr:Error ENOENT: invalid schedule specified - multiplier should be one of h,d,w,M,Y 2023-10-16T10:08:45.576 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T10:08:45.577 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec 1MM 2023-10-16T10:08:45.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.789+0000 7f419595a640 1 Processor -- start 2023-10-16T10:08:45.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.789+0000 7f419595a640 1 -- start start 2023-10-16T10:08:45.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.791+0000 7f419595a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f419015ad70 0x7f4190069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:45.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.791+0000 7f419595a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41900c9ca0 0x7f419006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:45.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.791+0000 7f419595a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f41900ca3b0 0x7f41900ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:45.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.791+0000 7f419595a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41900cad80 con 0x7f419015ad70 2023-10-16T10:08:45.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.791+0000 7f419595a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41900cb030 con 0x7f41900ca3b0 2023-10-16T10:08:45.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.791+0000 7f419595a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41900c91c0 con 0x7f41900c9ca0 2023-10-16T10:08:45.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.791+0000 7f418effd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f419015ad70 0x7f4190069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:45.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.791+0000 7f418effd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f419015ad70 0x7f4190069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53052/0 (socket says 172.21.15.84:53052) 2023-10-16T10:08:45.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.791+0000 7f418effd640 1 -- 172.21.15.84:0/3215277357 learned_addr learned my addr 172.21.15.84:0/3215277357 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:45.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.792+0000 7f418f7fe640 1 --2- 172.21.15.84:0/3215277357 >> v2:172.21.15.157:3300/0 conn(0x7f41900ca3b0 0x7f41900ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:45.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.792+0000 7f41867fc640 1 --2- 172.21.15.84:0/3215277357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41900c9ca0 0x7f419006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:45.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.792+0000 7f418effd640 1 -- 172.21.15.84:0/3215277357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41900c9ca0 msgr2=0x7f419006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:45.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.792+0000 7f418effd640 1 --2- 172.21.15.84:0/3215277357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41900c9ca0 0x7f419006a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:45.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.792+0000 7f418effd640 1 -- 172.21.15.84:0/3215277357 >> v2:172.21.15.157:3300/0 conn(0x7f41900ca3b0 msgr2=0x7f41900ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:45.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.792+0000 7f418effd640 1 --2- 172.21.15.84:0/3215277357 >> v2:172.21.15.157:3300/0 conn(0x7f41900ca3b0 0x7f41900ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:45.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.792+0000 7f418effd640 1 -- 172.21.15.84:0/3215277357 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41900c94a0 con 0x7f419015ad70 2023-10-16T10:08:45.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.793+0000 7f418effd640 1 --2- 172.21.15.84:0/3215277357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f419015ad70 0x7f4190069da0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f41900cf820 tx=0x7f418002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4255a145fe48bae0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:45.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.793+0000 7f418e7fc640 1 -- 172.21.15.84:0/3215277357 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180031870 con 0x7f419015ad70 2023-10-16T10:08:45.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.793+0000 7f418e7fc640 1 -- 172.21.15.84:0/3215277357 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41800319d0 con 0x7f419015ad70 2023-10-16T10:08:45.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.793+0000 7f418e7fc640 1 -- 172.21.15.84:0/3215277357 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180031d60 con 0x7f419015ad70 2023-10-16T10:08:45.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.794+0000 7f419595a640 1 -- 172.21.15.84:0/3215277357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f419015ad70 msgr2=0x7f4190069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:45.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.794+0000 7f419595a640 1 --2- 172.21.15.84:0/3215277357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f419015ad70 0x7f4190069da0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f41900cf820 tx=0x7f418002b910 comp rx=0 tx=0).stop 2023-10-16T10:08:45.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.794+0000 7f419595a640 1 -- 172.21.15.84:0/3215277357 shutdown_connections 2023-10-16T10:08:45.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.794+0000 7f419595a640 1 --2- 172.21.15.84:0/3215277357 >> v2:172.21.15.157:3300/0 conn(0x7f41900ca3b0 0x7f41900ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:45.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.794+0000 7f419595a640 1 --2- 172.21.15.84:0/3215277357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41900c9ca0 0x7f419006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:45.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.794+0000 7f419595a640 1 --2- 172.21.15.84:0/3215277357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f419015ad70 0x7f4190069da0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:45.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.794+0000 7f419595a640 1 -- 172.21.15.84:0/3215277357 >> 172.21.15.84:0/3215277357 conn(0x7f41900ce9b0 msgr2=0x7f41900cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:45.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.794+0000 7f419595a640 1 -- 172.21.15.84:0/3215277357 shutdown_connections 2023-10-16T10:08:45.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.794+0000 7f419595a640 1 -- 172.21.15.84:0/3215277357 wait complete. 2023-10-16T10:08:45.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.796+0000 7f419595a640 1 Processor -- start 2023-10-16T10:08:45.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.796+0000 7f419595a640 1 -- start start 2023-10-16T10:08:45.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.797+0000 7f419595a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41900c9ca0 0x7f41901517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:45.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.797+0000 7f419595a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41900ca3b0 0x7f41900b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:45.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.797+0000 7f419595a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f419015ad70 0x7f41900b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:45.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.797+0000 7f419595a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4190152370 con 0x7f419015ad70 2023-10-16T10:08:45.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.797+0000 7f419595a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41900cd7c0 con 0x7f41900ca3b0 2023-10-16T10:08:45.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.797+0000 7f419595a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41900cd200 con 0x7f41900c9ca0 2023-10-16T10:08:45.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.797+0000 7f418f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f419015ad70 0x7f41900b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:45.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.797+0000 7f418f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f419015ad70 0x7f41900b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53068/0 (socket says 172.21.15.84:53068) 2023-10-16T10:08:45.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.797+0000 7f418f7fe640 1 -- 172.21.15.84:0/4280241181 learned_addr learned my addr 172.21.15.84:0/4280241181 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:45.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.798+0000 7f418f7fe640 1 -- 172.21.15.84:0/4280241181 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41900c9ca0 msgr2=0x7f41901517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:45.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.798+0000 7f418f7fe640 1 --2- 172.21.15.84:0/4280241181 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41900c9ca0 0x7f41901517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:45.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.798+0000 7f418f7fe640 1 -- 172.21.15.84:0/4280241181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41900ca3b0 msgr2=0x7f41900b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:45.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.798+0000 7f41867fc640 1 --2- 172.21.15.84:0/4280241181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41900ca3b0 0x7f41900b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:45.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.798+0000 7f418f7fe640 1 --2- 172.21.15.84:0/4280241181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41900ca3b0 0x7f41900b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:45.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.798+0000 7f418f7fe640 1 -- 172.21.15.84:0/4280241181 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41800094d0 con 0x7f419015ad70 2023-10-16T10:08:45.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.798+0000 7f418f7fe640 1 --2- 172.21.15.84:0/4280241181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f419015ad70 0x7f41900b2b90 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f418800cd70 tx=0x7f418800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:45.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.798+0000 7f418cff9640 1 -- 172.21.15.84:0/4280241181 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188003b90 con 0x7f419015ad70 2023-10-16T10:08:45.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.798+0000 7f418cff9640 1 -- 172.21.15.84:0/4280241181 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4188003cf0 con 0x7f419015ad70 2023-10-16T10:08:45.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.798+0000 7f419595a640 1 -- 172.21.15.84:0/4280241181 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41900cd4e0 con 0x7f419015ad70 2023-10-16T10:08:45.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.798+0000 7f418cff9640 1 -- 172.21.15.84:0/4280241181 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188004240 con 0x7f419015ad70 2023-10-16T10:08:45.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.799+0000 7f419595a640 1 -- 172.21.15.84:0/4280241181 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41900c5e00 con 0x7f419015ad70 2023-10-16T10:08:45.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.800+0000 7f419595a640 1 -- 172.21.15.84:0/4280241181 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41900c6020 con 0x7f419015ad70 2023-10-16T10:08:45.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.801+0000 7f418cff9640 1 -- 172.21.15.84:0/4280241181 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 76) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f41880043a0 con 0x7f419015ad70 2023-10-16T10:08:45.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.802+0000 7f418cff9640 1 --2- 172.21.15.84:0/4280241181 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f417006b070 0x7f417006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:45.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.802+0000 7f418cff9640 1 -- 172.21.15.84:0/4280241181 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f4188012070 con 0x7f419015ad70 2023-10-16T10:08:45.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.802+0000 7f418effd640 1 --2- 172.21.15.84:0/4280241181 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f417006b070 0x7f417006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:45.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.803+0000 7f418effd640 1 --2- 172.21.15.84:0/4280241181 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f417006b070 0x7f417006d530 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4180036000 tx=0x7f4180030000 comp rx=0 tx=0).ready entity=mgr.8754 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:45.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:45.806+0000 7f418cff9640 1 -- 172.21.15.84:0/4280241181 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4188055160 con 0x7f419015ad70 2023-10-16T10:08:46.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.021+0000 7f419595a640 1 -- 172.21.15.84:0/4280241181 --> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "1MM", "target": ["mon-mgr", ""]}) v1 -- 0x7f41900c9780 con 0x7f417006b070 2023-10-16T10:08:46.024 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:46.023+0000 7fb27e1dd640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-16T10:08:46.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.023+0000 7f418cff9640 1 -- 172.21.15.84:0/4280241181 <== mgr.8754 v2:172.21.15.84:6832/1203729023 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4180031d30 con 0x7f417006b070 2023-10-16T10:08:46.025 INFO:teuthology.orchestra.run.smithi084.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-16T10:08:46.037 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T10:08:46.038 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec 1 2023-10-16T10:08:46.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.253+0000 7fcad518a640 1 Processor -- start 2023-10-16T10:08:46.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.254+0000 7fcad518a640 1 -- start start 2023-10-16T10:08:46.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.255+0000 7fcad518a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcad015ad70 0x7fcad0069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:46.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.255+0000 7fcad518a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcad00caac0 0x7fcad006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:46.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.255+0000 7fcad518a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad00cb030 con 0x7fcad00c9ca0 2023-10-16T10:08:46.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.255+0000 7fcad518a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad00c91c0 con 0x7fcad00caac0 2023-10-16T10:08:46.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.255+0000 7fcad518a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad00c94a0 con 0x7fcad015ad70 2023-10-16T10:08:46.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.255+0000 7fcac7fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fcad00c9ca0 0x7fcad00ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46834/0 (socket says 172.21.15.84:46834) 2023-10-16T10:08:46.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.255+0000 7fcac7fff640 1 -- 172.21.15.84:0/1057523601 learned_addr learned my addr 172.21.15.84:0/1057523601 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:46.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.255+0000 7fcacf59e640 1 --2- 172.21.15.84:0/1057523601 >> v2:172.21.15.84:3300/0 conn(0x7fcad00caac0 0x7fcad006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:46.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.255+0000 7fcaced9d640 1 --2- 172.21.15.84:0/1057523601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcad015ad70 0x7fcad0069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:46.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.256+0000 7fcace59c640 1 -- 172.21.15.84:0/1057523601 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320554557 0 0) 0x7fcad00cb030 con 0x7fcad00c9ca0 2023-10-16T10:08:46.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.256+0000 7fcace59c640 1 -- 172.21.15.84:0/1057523601 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcabc003520 con 0x7fcad00c9ca0 2023-10-16T10:08:46.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.256+0000 7fcacf59e640 1 -- 172.21.15.84:0/1057523601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcad015ad70 msgr2=0x7fcad0069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:46.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.256+0000 7fcacf59e640 1 --2- 172.21.15.84:0/1057523601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcad015ad70 0x7fcad0069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:46.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.256+0000 7fcacf59e640 1 -- 172.21.15.84:0/1057523601 >> v1:172.21.15.84:6789/0 conn(0x7fcad00c9ca0 legacy=0x7fcad00ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:46.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.256+0000 7fcacf59e640 1 -- 172.21.15.84:0/1057523601 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad00c9780 con 0x7fcad00caac0 2023-10-16T10:08:46.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.257+0000 7fcacf59e640 1 --2- 172.21.15.84:0/1057523601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcad00caac0 0x7fcad006a2e0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009d90 tx=0x7fcac802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b3438fa9659a3c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:46.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.257+0000 7fcace59c640 1 -- 172.21.15.84:0/1057523601 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac802cc50 con 0x7fcad00caac0 2023-10-16T10:08:46.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.257+0000 7fcace59c640 1 -- 172.21.15.84:0/1057523601 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac802cdb0 con 0x7fcad00caac0 2023-10-16T10:08:46.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.257+0000 7fcace59c640 1 -- 172.21.15.84:0/1057523601 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac8034660 con 0x7fcad00caac0 2023-10-16T10:08:46.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.258+0000 7fcad518a640 1 -- 172.21.15.84:0/1057523601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcad00caac0 msgr2=0x7fcad006a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:46.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.258+0000 7fcad518a640 1 --2- 172.21.15.84:0/1057523601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcad00caac0 0x7fcad006a2e0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009d90 tx=0x7fcac802bea0 comp rx=0 tx=0).stop 2023-10-16T10:08:46.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.258+0000 7fcad518a640 1 -- 172.21.15.84:0/1057523601 shutdown_connections 2023-10-16T10:08:46.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.258+0000 7fcad518a640 1 --2- 172.21.15.84:0/1057523601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcad00caac0 0x7fcad006a2e0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:46.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.258+0000 7fcad518a640 1 --2- 172.21.15.84:0/1057523601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcad015ad70 0x7fcad0069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:46.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.258+0000 7fcad518a640 1 -- 172.21.15.84:0/1057523601 >> 172.21.15.84:0/1057523601 conn(0x7fcad00ce9b0 msgr2=0x7fcad00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:46.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.258+0000 7fcad518a640 1 -- 172.21.15.84:0/1057523601 shutdown_connections 2023-10-16T10:08:46.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.258+0000 7fcad518a640 1 -- 172.21.15.84:0/1057523601 wait complete. 2023-10-16T10:08:46.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.260+0000 7fcad518a640 1 Processor -- start 2023-10-16T10:08:46.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.260+0000 7fcad518a640 1 -- start start 2023-10-16T10:08:46.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.261+0000 7fcad518a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcad00c9ca0 0x7fcad00ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:46.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.261+0000 7fcad518a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcad00caac0 0x7fcad00b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:46.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.261+0000 7fcad518a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcad015ad70 0x7fcad00af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:46.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.261+0000 7fcad518a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad00aefb0 con 0x7fcad00caac0 2023-10-16T10:08:46.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.261+0000 7fcad518a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad00b0eb0 con 0x7fcad015ad70 2023-10-16T10:08:46.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.261+0000 7fcad518a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad00b1190 con 0x7fcad00c9ca0 2023-10-16T10:08:46.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.261+0000 7fcac7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcad00caac0 0x7fcad00b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:46.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.261+0000 7fcac7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcad00caac0 0x7fcad00b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53084/0 (socket says 172.21.15.84:53084) 2023-10-16T10:08:46.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.261+0000 7fcac7fff640 1 -- 172.21.15.84:0/1040305840 learned_addr learned my addr 172.21.15.84:0/1040305840 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:46.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.262+0000 7fcaced9d640 1 --2- 172.21.15.84:0/1040305840 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcad00c9ca0 0x7fcad00ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:46.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.262+0000 7fcacf59e640 1 --2- 172.21.15.84:0/1040305840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcad015ad70 0x7fcad00af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:46.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.262+0000 7fcac7fff640 1 -- 172.21.15.84:0/1040305840 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcad00c9ca0 msgr2=0x7fcad00ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:46.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.262+0000 7fcac7fff640 1 --2- 172.21.15.84:0/1040305840 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcad00c9ca0 0x7fcad00ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:46.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.262+0000 7fcac7fff640 1 -- 172.21.15.84:0/1040305840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcad015ad70 msgr2=0x7fcad00af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:46.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.262+0000 7fcac7fff640 1 --2- 172.21.15.84:0/1040305840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcad015ad70 0x7fcad00af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:46.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.262+0000 7fcac7fff640 1 -- 172.21.15.84:0/1040305840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac8009a70 con 0x7fcad00caac0 2023-10-16T10:08:46.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.262+0000 7fcaced9d640 1 --2- 172.21.15.84:0/1040305840 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcad00c9ca0 0x7fcad00ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:46.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.262+0000 7fcac7fff640 1 --2- 172.21.15.84:0/1040305840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcad00caac0 0x7fcad00b0480 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fcab800c500 tx=0x7fcab800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:46.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.262+0000 7fcac77fe640 1 -- 172.21.15.84:0/1040305840 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8005ce0 con 0x7fcad00caac0 2023-10-16T10:08:46.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.262+0000 7fcac77fe640 1 -- 172.21.15.84:0/1040305840 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab8005e40 con 0x7fcad00caac0 2023-10-16T10:08:46.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.262+0000 7fcad518a640 1 -- 172.21.15.84:0/1040305840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad00b1470 con 0x7fcad00caac0 2023-10-16T10:08:46.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.263+0000 7fcac77fe640 1 -- 172.21.15.84:0/1040305840 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8007330 con 0x7fcad00caac0 2023-10-16T10:08:46.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.263+0000 7fcad518a640 1 -- 172.21.15.84:0/1040305840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad00b09c0 con 0x7fcad00caac0 2023-10-16T10:08:46.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.264+0000 7fcad518a640 1 -- 172.21.15.84:0/1040305840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad00ad4b0 con 0x7fcad00caac0 2023-10-16T10:08:46.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.265+0000 7fcac77fe640 1 -- 172.21.15.84:0/1040305840 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 76) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fcab8007550 con 0x7fcad00caac0 2023-10-16T10:08:46.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.266+0000 7fcac77fe640 1 --2- 172.21.15.84:0/1040305840 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fcab006b070 0x7fcab006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:46.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.266+0000 7fcac77fe640 1 -- 172.21.15.84:0/1040305840 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7fcab8089190 con 0x7fcad00caac0 2023-10-16T10:08:46.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.266+0000 7fcaced9d640 1 --2- 172.21.15.84:0/1040305840 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fcab006b070 0x7fcab006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:46.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.267+0000 7fcaced9d640 1 --2- 172.21.15.84:0/1040305840 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fcab006b070 0x7fcab006d530 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcac0006fd0 tx=0x7fcac0008040 comp rx=0 tx=0).ready entity=mgr.8754 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:46.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.270+0000 7fcac77fe640 1 -- 172.21.15.84:0/1040305840 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcad00ad4b0 con 0x7fcad00caac0 2023-10-16T10:08:46.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.484+0000 7fcad518a640 1 -- 172.21.15.84:0/1040305840 --> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fcad00b1300 con 0x7fcab006b070 2023-10-16T10:08:46.488 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:46.487+0000 7fb27e1dd640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-16T10:08:46.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.487+0000 7fcac77fe640 1 -- 172.21.15.84:0/1040305840 <== mgr.8754 v2:172.21.15.84:6832/1203729023 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcad00b1300 con 0x7fcab006b070 2023-10-16T10:08:46.489 INFO:teuthology.orchestra.run.smithi084.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-16T10:08:46.502 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T10:08:46.503 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec M 2023-10-16T10:08:46.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.723+0000 7f389b59e640 1 Processor -- start 2023-10-16T10:08:46.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.723+0000 7f389b59e640 1 -- start start 2023-10-16T10:08:46.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.725+0000 7f389b59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f389c0b8480 0x7f389c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:46.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.725+0000 7f389b59e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f389c15d150 con 0x7f389c0d0be0 2023-10-16T10:08:46.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.725+0000 7f389b59e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f389c0d1180 con 0x7f389c15cd70 2023-10-16T10:08:46.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.725+0000 7f389b59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0d1460 con 0x7f389c0b8480 2023-10-16T10:08:46.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.725+0000 7f389ad9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f389c0d0be0 0x7f389c0b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46838/0 (socket says 172.21.15.84:46838) 2023-10-16T10:08:46.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.725+0000 7f389ad9d640 1 -- 172.21.15.84:0/2557004204 learned_addr learned my addr 172.21.15.84:0/2557004204 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:46.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.725+0000 7f3891d9b640 1 --2- 172.21.15.84:0/2557004204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f389c0b8480 0x7f389c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:46.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.726+0000 7f3899d9b640 1 -- 172.21.15.84:0/2557004204 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171812633 0 0) 0x7f389c15d150 con 0x7f389c0d0be0 2023-10-16T10:08:46.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.726+0000 7f3899d9b640 1 -- 172.21.15.84:0/2557004204 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3888003520 con 0x7f389c0d0be0 2023-10-16T10:08:46.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.726+0000 7f3899d9b640 1 -- 172.21.15.84:0/2557004204 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2346223889 0 0) 0x7f3888003520 con 0x7f389c0d0be0 2023-10-16T10:08:46.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.726+0000 7f3899d9b640 1 -- 172.21.15.84:0/2557004204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f389c0b8480 msgr2=0x7f389c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:46.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.726+0000 7f3899d9b640 1 --2- 172.21.15.84:0/2557004204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f389c0b8480 0x7f389c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:46.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.726+0000 7f3899d9b640 1 -- 172.21.15.84:0/2557004204 >> v1:172.21.15.157:6789/0 conn(0x7f389c15cd70 legacy=0x7f389c0d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:46.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.726+0000 7f3899d9b640 1 -- 172.21.15.84:0/2557004204 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c0d1740 con 0x7f389c0d0be0 2023-10-16T10:08:46.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.726+0000 7f3899d9b640 1 -- 172.21.15.84:0/2557004204 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f3894001670 con 0x7f389c0d0be0 2023-10-16T10:08:46.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.726+0000 7f3899d9b640 1 -- 172.21.15.84:0/2557004204 >> v1:172.21.15.84:6789/0 conn(0x7f389c0d0be0 legacy=0x7f389c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:46.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.726+0000 7f3899d9b640 1 --2- 172.21.15.84:0/2557004204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38880039a0 0x7f3888003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:46.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.726+0000 7f3899d9b640 1 --2- 172.21.15.84:0/2557004204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38880047c0 0x7f3888004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:46.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.726+0000 7f3899d9b640 1 --2- 172.21.15.84:0/2557004204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f389c0b8480 0x7f3888005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:46.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.726+0000 7f3899d9b640 1 -- 172.21.15.84:0/2557004204 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0d1460 con 0x7f38880039a0 2023-10-16T10:08:46.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.726+0000 7f3899d9b640 1 -- 172.21.15.84:0/2557004204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388800ba60 con 0x7f38880047c0 2023-10-16T10:08:46.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.726+0000 7f3899d9b640 1 -- 172.21.15.84:0/2557004204 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388800bc20 con 0x7f389c0b8480 2023-10-16T10:08:46.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.727+0000 7f389a59c640 1 --2- 172.21.15.84:0/2557004204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38880039a0 0x7f3888003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:46.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.727+0000 7f389a59c640 1 -- 172.21.15.84:0/2557004204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f389c0b8480 msgr2=0x7f3888005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:46.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.727+0000 7f389a59c640 1 --2- 172.21.15.84:0/2557004204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f389c0b8480 0x7f3888005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:46.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.727+0000 7f389a59c640 1 -- 172.21.15.84:0/2557004204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38880047c0 msgr2=0x7f3888004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:46.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.727+0000 7f389a59c640 1 --2- 172.21.15.84:0/2557004204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38880047c0 0x7f3888004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:46.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.727+0000 7f389a59c640 1 -- 172.21.15.84:0/2557004204 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3888003520 con 0x7f38880039a0 2023-10-16T10:08:46.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.728+0000 7f389a59c640 1 --2- 172.21.15.84:0/2557004204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38880039a0 0x7f3888003dc0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f388c001720 tx=0x7f38880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c01b29176df7788 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:46.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.729+0000 7f3899d9b640 1 -- 172.21.15.84:0/2557004204 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c0052d0 con 0x7f38880039a0 2023-10-16T10:08:46.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.729+0000 7f3899d9b640 1 -- 172.21.15.84:0/2557004204 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c02e050 con 0x7f38880039a0 2023-10-16T10:08:46.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.729+0000 7f3899d9b640 1 -- 172.21.15.84:0/2557004204 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c0035f0 con 0x7f38880039a0 2023-10-16T10:08:46.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.729+0000 7f389b59e640 1 -- 172.21.15.84:0/2557004204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38880039a0 msgr2=0x7f3888003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:46.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.729+0000 7f389b59e640 1 --2- 172.21.15.84:0/2557004204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38880039a0 0x7f3888003dc0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f388c001720 tx=0x7f38880057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:46.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.729+0000 7f389ad9d640 1 -- 172.21.15.84:0/2557004204 reap_dead start 2023-10-16T10:08:46.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.729+0000 7f389b59e640 1 -- 172.21.15.84:0/2557004204 shutdown_connections 2023-10-16T10:08:46.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.729+0000 7f389b59e640 1 -- 172.21.15.84:0/2557004204 >> 172.21.15.84:0/2557004204 conn(0x7f389c0cf570 msgr2=0x7f389c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:46.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.729+0000 7f389b59e640 1 -- 172.21.15.84:0/2557004204 shutdown_connections 2023-10-16T10:08:46.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.729+0000 7f389b59e640 1 -- 172.21.15.84:0/2557004204 wait complete. 2023-10-16T10:08:46.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.731+0000 7f389b59e640 1 Processor -- start 2023-10-16T10:08:46.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.731+0000 7f389b59e640 1 -- start start 2023-10-16T10:08:46.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.732+0000 7f389b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38880039a0 0x7f389c0c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:46.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.732+0000 7f389b59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f389c1529f0 0x7f389c152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:46.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.732+0000 7f389b59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f389c144860 0x7f389c144ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:46.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.732+0000 7f389b59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c145220 con 0x7f38880039a0 2023-10-16T10:08:46.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.732+0000 7f389b59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0c91d0 con 0x7f389c1529f0 2023-10-16T10:08:46.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.732+0000 7f389b59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389c15f160 con 0x7f389c144860 2023-10-16T10:08:46.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.733+0000 7f389a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38880039a0 0x7f389c0c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:46.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.733+0000 7f389a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38880039a0 0x7f389c0c8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53094/0 (socket says 172.21.15.84:53094) 2023-10-16T10:08:46.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.733+0000 7f389a59c640 1 -- 172.21.15.84:0/323536685 learned_addr learned my addr 172.21.15.84:0/323536685 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:46.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.733+0000 7f389a59c640 1 -- 172.21.15.84:0/323536685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f389c144860 msgr2=0x7f389c144ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:46.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.733+0000 7f389a59c640 1 --2- 172.21.15.84:0/323536685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f389c144860 0x7f389c144ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:46.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.733+0000 7f389a59c640 1 -- 172.21.15.84:0/323536685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f389c1529f0 msgr2=0x7f389c152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:46.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.733+0000 7f389a59c640 1 --2- 172.21.15.84:0/323536685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f389c1529f0 0x7f389c152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:46.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.733+0000 7f389a59c640 1 -- 172.21.15.84:0/323536685 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c15f440 con 0x7f38880039a0 2023-10-16T10:08:46.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.733+0000 7f389a59c640 1 --2- 172.21.15.84:0/323536685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38880039a0 0x7f389c0c8c90 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f388c008fd0 tx=0x7f388c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:46.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.734+0000 7f38937fe640 1 -- 172.21.15.84:0/323536685 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c007640 con 0x7f38880039a0 2023-10-16T10:08:46.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.734+0000 7f38937fe640 1 -- 172.21.15.84:0/323536685 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c0069e0 con 0x7f38880039a0 2023-10-16T10:08:46.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.734+0000 7f38937fe640 1 -- 172.21.15.84:0/323536685 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c006d50 con 0x7f38880039a0 2023-10-16T10:08:46.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.734+0000 7f389b59e640 1 -- 172.21.15.84:0/323536685 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f389c15f720 con 0x7f38880039a0 2023-10-16T10:08:46.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.734+0000 7f389b59e640 1 -- 172.21.15.84:0/323536685 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389c15fce0 con 0x7f38880039a0 2023-10-16T10:08:46.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.735+0000 7f389b59e640 1 -- 172.21.15.84:0/323536685 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f389c04e9b0 con 0x7f38880039a0 2023-10-16T10:08:46.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.736+0000 7f38937fe640 1 -- 172.21.15.84:0/323536685 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 76) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f388c0035f0 con 0x7f38880039a0 2023-10-16T10:08:46.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.737+0000 7f38937fe640 1 --2- 172.21.15.84:0/323536685 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f387c06b070 0x7f387c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:46.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.737+0000 7f38937fe640 1 -- 172.21.15.84:0/323536685 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f388c0abcb0 con 0x7f38880039a0 2023-10-16T10:08:46.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.737+0000 7f3891d9b640 1 --2- 172.21.15.84:0/323536685 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f387c06b070 0x7f387c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:46.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.738+0000 7f3891d9b640 1 --2- 172.21.15.84:0/323536685 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f387c06b070 0x7f387c06d530 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f38840070b0 tx=0x7f388400d040 comp rx=0 tx=0).ready entity=mgr.8754 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:46.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.741+0000 7f38937fe640 1 -- 172.21.15.84:0/323536685 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f388c076970 con 0x7f38880039a0 2023-10-16T10:08:46.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.953+0000 7f389b59e640 1 -- 172.21.15.84:0/323536685 --> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "M", "target": ["mon-mgr", ""]}) v1 -- 0x7f389c145220 con 0x7f387c06b070 2023-10-16T10:08:46.956 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:46.955+0000 7fb27e1dd640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-16T10:08:46.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:46.955+0000 7f38937fe640 1 -- 172.21.15.84:0/323536685 <== mgr.8754 v2:172.21.15.84:6832/1203729023 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7f389c145220 con 0x7f387c06b070 2023-10-16T10:08:46.957 INFO:teuthology.orchestra.run.smithi084.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-16T10:08:46.970 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T10:08:46.971 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec -1m 2023-10-16T10:08:47.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.191+0000 7ff4f6e51640 1 Processor -- start 2023-10-16T10:08:47.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.191+0000 7ff4f6e51640 1 -- start start 2023-10-16T10:08:47.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.192+0000 7ff4f6e51640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f015ad70 0x7ff4f0069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:47.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.192+0000 7ff4f6e51640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff4f00caac0 0x7ff4f006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:47.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.192+0000 7ff4f6e51640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f00cb030 con 0x7ff4f00c9ca0 2023-10-16T10:08:47.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.192+0000 7ff4f6e51640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f00c91c0 con 0x7ff4f00caac0 2023-10-16T10:08:47.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.192+0000 7ff4f6e51640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f00c94a0 con 0x7ff4f015ad70 2023-10-16T10:08:47.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.193+0000 7ff4effff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff4f00c9ca0 0x7ff4f00ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46852/0 (socket says 172.21.15.84:46852) 2023-10-16T10:08:47.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.193+0000 7ff4effff640 1 -- 172.21.15.84:0/3084373314 learned_addr learned my addr 172.21.15.84:0/3084373314 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:47.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.193+0000 7ff4ef7fe640 1 -- 172.21.15.84:0/3084373314 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005427531 0 0) 0x7ff4f00cb030 con 0x7ff4f00c9ca0 2023-10-16T10:08:47.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.193+0000 7ff4ef7fe640 1 -- 172.21.15.84:0/3084373314 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4e0003520 con 0x7ff4f00c9ca0 2023-10-16T10:08:47.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.193+0000 7ff4f4bed640 1 --2- 172.21.15.84:0/3084373314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f015ad70 0x7ff4f0069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:47.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.193+0000 7ff4f53ee640 1 --2- 172.21.15.84:0/3084373314 >> v2:172.21.15.157:3300/0 conn(0x7ff4f00caac0 0x7ff4f006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:47.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4ef7fe640 1 -- 172.21.15.84:0/3084373314 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2414168593 0 0) 0x7ff4e0003520 con 0x7ff4f00c9ca0 2023-10-16T10:08:47.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4ef7fe640 1 -- 172.21.15.84:0/3084373314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f015ad70 msgr2=0x7ff4f0069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:47.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4ef7fe640 1 --2- 172.21.15.84:0/3084373314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f015ad70 0x7ff4f0069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:47.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4ef7fe640 1 -- 172.21.15.84:0/3084373314 >> v2:172.21.15.157:3300/0 conn(0x7ff4f00caac0 msgr2=0x7ff4f006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:47.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4ef7fe640 1 --2- 172.21.15.84:0/3084373314 >> v2:172.21.15.157:3300/0 conn(0x7ff4f00caac0 0x7ff4f006a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:47.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4ef7fe640 1 -- 172.21.15.84:0/3084373314 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f00c9780 con 0x7ff4f00c9ca0 2023-10-16T10:08:47.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4ef7fe640 1 -- 172.21.15.84:0/3084373314 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ff4dc001670 con 0x7ff4f00c9ca0 2023-10-16T10:08:47.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4ef7fe640 1 -- 172.21.15.84:0/3084373314 >> v1:172.21.15.84:6789/0 conn(0x7ff4f00c9ca0 legacy=0x7ff4f00ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:47.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4ef7fe640 1 --2- 172.21.15.84:0/3084373314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4e0003970 0x7ff4e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:47.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4ef7fe640 1 --2- 172.21.15.84:0/3084373314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f015ad70 0x7ff4e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:47.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4ef7fe640 1 --2- 172.21.15.84:0/3084373314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4e0004d00 0x7ff4e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:47.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4ef7fe640 1 -- 172.21.15.84:0/3084373314 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f00c91c0 con 0x7ff4e0004d00 2023-10-16T10:08:47.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4ef7fe640 1 -- 172.21.15.84:0/3084373314 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f00c94a0 con 0x7ff4e0003970 2023-10-16T10:08:47.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4ef7fe640 1 -- 172.21.15.84:0/3084373314 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e000bab0 con 0x7ff4f015ad70 2023-10-16T10:08:47.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4f53ee640 1 --2- 172.21.15.84:0/3084373314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4e0004d00 0x7ff4e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:47.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4f4bed640 1 --2- 172.21.15.84:0/3084373314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4e0003970 0x7ff4e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:47.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4f53ee640 1 -- 172.21.15.84:0/3084373314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f015ad70 msgr2=0x7ff4e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:47.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4f53ee640 1 --2- 172.21.15.84:0/3084373314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f015ad70 0x7ff4e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:47.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4f53ee640 1 -- 172.21.15.84:0/3084373314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4e0003970 msgr2=0x7ff4e0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:47.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4f53ee640 1 --2- 172.21.15.84:0/3084373314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4e0003970 0x7ff4e0003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:47.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.194+0000 7ff4f53ee640 1 -- 172.21.15.84:0/3084373314 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e0003520 con 0x7ff4e0004d00 2023-10-16T10:08:47.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.195+0000 7ff4f53ee640 1 --2- 172.21.15.84:0/3084373314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4e0004d00 0x7ff4e0005180 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8007360 tx=0x7ff4e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab24dbcf55499db server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:47.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.196+0000 7ff4ef7fe640 1 -- 172.21.15.84:0/3084373314 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e802ccf0 con 0x7ff4e0004d00 2023-10-16T10:08:47.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.196+0000 7ff4ef7fe640 1 -- 172.21.15.84:0/3084373314 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e802ce50 con 0x7ff4e0004d00 2023-10-16T10:08:47.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.196+0000 7ff4ef7fe640 1 -- 172.21.15.84:0/3084373314 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8034660 con 0x7ff4e0004d00 2023-10-16T10:08:47.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.196+0000 7ff4f6e51640 1 -- 172.21.15.84:0/3084373314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4e0004d00 msgr2=0x7ff4e0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:47.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.196+0000 7ff4f6e51640 1 --2- 172.21.15.84:0/3084373314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4e0004d00 0x7ff4e0005180 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8007360 tx=0x7ff4e00057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:47.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.196+0000 7ff4f53ee640 1 -- 172.21.15.84:0/3084373314 reap_dead start 2023-10-16T10:08:47.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.196+0000 7ff4f6e51640 1 -- 172.21.15.84:0/3084373314 shutdown_connections 2023-10-16T10:08:47.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.196+0000 7ff4f6e51640 1 -- 172.21.15.84:0/3084373314 >> 172.21.15.84:0/3084373314 conn(0x7ff4f00ce9b0 msgr2=0x7ff4f00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:47.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.196+0000 7ff4f6e51640 1 -- 172.21.15.84:0/3084373314 shutdown_connections 2023-10-16T10:08:47.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.197+0000 7ff4f6e51640 1 -- 172.21.15.84:0/3084373314 wait complete. 2023-10-16T10:08:47.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.198+0000 7ff4f6e51640 1 Processor -- start 2023-10-16T10:08:47.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.198+0000 7ff4f6e51640 1 -- start start 2023-10-16T10:08:47.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.199+0000 7ff4f6e51640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4e0004d00 0x7ff4f01f3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:47.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.199+0000 7ff4f6e51640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4f015b450 0x7ff4f006b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:47.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.199+0000 7ff4f6e51640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f006b750 0x7ff4f00c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:47.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.199+0000 7ff4f6e51640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f015b950 con 0x7ff4f015b450 2023-10-16T10:08:47.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.199+0000 7ff4f6e51640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f015bc30 con 0x7ff4e0004d00 2023-10-16T10:08:47.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.199+0000 7ff4f6e51640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f015bf10 con 0x7ff4f006b750 2023-10-16T10:08:47.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.200+0000 7ff4effff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4f015b450 0x7ff4f006b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:47.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.200+0000 7ff4effff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4f015b450 0x7ff4f006b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53120/0 (socket says 172.21.15.84:53120) 2023-10-16T10:08:47.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.200+0000 7ff4effff640 1 -- 172.21.15.84:0/2397145543 learned_addr learned my addr 172.21.15.84:0/2397145543 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:47.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.200+0000 7ff4f4bed640 1 --2- 172.21.15.84:0/2397145543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4e0004d00 0x7ff4f01f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:47.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.200+0000 7ff4f53ee640 1 --2- 172.21.15.84:0/2397145543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f006b750 0x7ff4f00c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:47.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.200+0000 7ff4effff640 1 -- 172.21.15.84:0/2397145543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f006b750 msgr2=0x7ff4f00c3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:47.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.200+0000 7ff4effff640 1 --2- 172.21.15.84:0/2397145543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4f006b750 0x7ff4f00c3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:47.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.200+0000 7ff4effff640 1 -- 172.21.15.84:0/2397145543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4e0004d00 msgr2=0x7ff4f01f3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:47.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.200+0000 7ff4effff640 1 --2- 172.21.15.84:0/2397145543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4e0004d00 0x7ff4f01f3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:47.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.200+0000 7ff4effff640 1 -- 172.21.15.84:0/2397145543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f015c1f0 con 0x7ff4f015b450 2023-10-16T10:08:47.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.200+0000 7ff4effff640 1 --2- 172.21.15.84:0/2397145543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4f015b450 0x7ff4f006b210 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00c730 tx=0x7ff4dc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:47.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.201+0000 7ff4ed7fa640 1 -- 172.21.15.84:0/2397145543 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc0074e0 con 0x7ff4f015b450 2023-10-16T10:08:47.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.201+0000 7ff4ed7fa640 1 -- 172.21.15.84:0/2397145543 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4dc007640 con 0x7ff4f015b450 2023-10-16T10:08:47.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.201+0000 7ff4f6e51640 1 -- 172.21.15.84:0/2397145543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f015c4d0 con 0x7ff4f015b450 2023-10-16T10:08:47.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.201+0000 7ff4ed7fa640 1 -- 172.21.15.84:0/2397145543 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc004a00 con 0x7ff4f015b450 2023-10-16T10:08:47.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.201+0000 7ff4f6e51640 1 -- 172.21.15.84:0/2397145543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f00c5140 con 0x7ff4f015b450 2023-10-16T10:08:47.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.202+0000 7ff4f6e51640 1 -- 172.21.15.84:0/2397145543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f015c710 con 0x7ff4f015b450 2023-10-16T10:08:47.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.203+0000 7ff4ed7fa640 1 -- 172.21.15.84:0/2397145543 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 76) v1 ==== 89405+0+0 (secure 0 0 0) 0x7ff4dc004b60 con 0x7ff4f015b450 2023-10-16T10:08:47.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.204+0000 7ff4ed7fa640 1 --2- 172.21.15.84:0/2397145543 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7ff4d406b070 0x7ff4d406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:47.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.204+0000 7ff4ed7fa640 1 -- 172.21.15.84:0/2397145543 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7ff4dc089380 con 0x7ff4f015b450 2023-10-16T10:08:47.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.204+0000 7ff4f4bed640 1 --2- 172.21.15.84:0/2397145543 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7ff4d406b070 0x7ff4d406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:47.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.205+0000 7ff4f4bed640 1 --2- 172.21.15.84:0/2397145543 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7ff4d406b070 0x7ff4d406d530 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00b2490 tx=0x7ff4e4008040 comp rx=0 tx=0).ready entity=mgr.8754 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:47.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.208+0000 7ff4ed7fa640 1 -- 172.21.15.84:0/2397145543 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff4f015c710 con 0x7ff4f015b450 2023-10-16T10:08:47.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.423+0000 7ff4f6e51640 1 -- 172.21.15.84:0/2397145543 --> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "-1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4f0069da0 con 0x7ff4d406b070 2023-10-16T10:08:47.427 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:47.426+0000 7fb27e1dd640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period must be a non-zero positive value e.g. 1h or 4d etc. 2023-10-16T10:08:47.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.426+0000 7ff4ed7fa640 1 -- 172.21.15.84:0/2397145543 <== mgr.8754 v2:172.21.15.84:6832/1203729023 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period must be a non-zero positive value e.g. 1h or 4d etc.) v1 ==== 96+0+0 (secure 0 0 0) 0x7ff4e4009520 con 0x7ff4d406b070 2023-10-16T10:08:47.428 INFO:teuthology.orchestra.run.smithi084.stderr:Error ENOENT: invalid schedule specified - period must be a non-zero positive value e.g. 1h or 4d etc. 2023-10-16T10:08:47.441 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T10:08:47.442 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec '' 2023-10-16T10:08:47.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.660+0000 7f33f3402640 1 Processor -- start 2023-10-16T10:08:47.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.661+0000 7f33f3402640 1 -- start start 2023-10-16T10:08:47.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.662+0000 7f33f3402640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f33ec0caac0 0x7f33ec0ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:47.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.662+0000 7f33f3402640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ec0cb030 con 0x7f33ec15ad70 2023-10-16T10:08:47.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.662+0000 7f33f3402640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ec0c91c0 con 0x7f33ec06a4b0 2023-10-16T10:08:47.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.662+0000 7f33f3402640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33ec0c94a0 con 0x7f33ec0caac0 2023-10-16T10:08:47.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.662+0000 7f33f119e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f33ec15ad70 0x7f33ec069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46866/0 (socket says 172.21.15.84:46866) 2023-10-16T10:08:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.662+0000 7f33f119e640 1 -- 172.21.15.84:0/1468085720 learned_addr learned my addr 172.21.15.84:0/1468085720 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:47.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.663+0000 7f33f199f640 1 --2- 172.21.15.84:0/1468085720 >> v2:172.21.15.157:3300/0 conn(0x7f33ec0caac0 0x7f33ec0ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:47.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.663+0000 7f33e3fff640 1 -- 172.21.15.84:0/1468085720 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3959010379 0 0) 0x7f33ec0cb030 con 0x7f33ec15ad70 2023-10-16T10:08:47.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.663+0000 7f33e3fff640 1 -- 172.21.15.84:0/1468085720 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33d8003520 con 0x7f33ec15ad70 2023-10-16T10:08:47.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.663+0000 7f33e3fff640 1 -- 172.21.15.84:0/1468085720 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1991496537 0 0) 0x7f33d8003520 con 0x7f33ec15ad70 2023-10-16T10:08:47.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.663+0000 7f33e3fff640 1 -- 172.21.15.84:0/1468085720 >> v2:172.21.15.157:3300/0 conn(0x7f33ec0caac0 msgr2=0x7f33ec0ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:47.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.663+0000 7f33e3fff640 1 --2- 172.21.15.84:0/1468085720 >> v2:172.21.15.157:3300/0 conn(0x7f33ec0caac0 0x7f33ec0ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:47.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.663+0000 7f33e3fff640 1 -- 172.21.15.84:0/1468085720 >> v1:172.21.15.157:6789/0 conn(0x7f33ec06a4b0 legacy=0x7f33ec0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:47.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.663+0000 7f33e3fff640 1 -- 172.21.15.84:0/1468085720 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec0c9780 con 0x7f33ec15ad70 2023-10-16T10:08:47.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.663+0000 7f33e3fff640 1 -- 172.21.15.84:0/1468085720 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f33e8001670 con 0x7f33ec15ad70 2023-10-16T10:08:47.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.663+0000 7f33e3fff640 1 -- 172.21.15.84:0/1468085720 >> v1:172.21.15.84:6789/0 conn(0x7f33ec15ad70 legacy=0x7f33ec069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:47.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.663+0000 7f33e3fff640 1 --2- 172.21.15.84:0/1468085720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33d80039c0 0x7f33d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:47.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.663+0000 7f33e3fff640 1 --2- 172.21.15.84:0/1468085720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33d80047c0 0x7f33d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:47.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.663+0000 7f33e3fff640 1 --2- 172.21.15.84:0/1468085720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33d8005180 0x7f33d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:47.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.663+0000 7f33e3fff640 1 -- 172.21.15.84:0/1468085720 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0c94a0 con 0x7f33d80039c0 2023-10-16T10:08:47.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.663+0000 7f33e3fff640 1 -- 172.21.15.84:0/1468085720 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d800c080 con 0x7f33d80047c0 2023-10-16T10:08:47.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.663+0000 7f33e3fff640 1 -- 172.21.15.84:0/1468085720 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d800c240 con 0x7f33d8005180 2023-10-16T10:08:47.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.664+0000 7f33f119e640 1 --2- 172.21.15.84:0/1468085720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33d80039c0 0x7f33d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:47.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.664+0000 7f33f099d640 1 --2- 172.21.15.84:0/1468085720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33d80047c0 0x7f33d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:47.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.664+0000 7f33f119e640 1 -- 172.21.15.84:0/1468085720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33d8005180 msgr2=0x7f33d800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:47.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.664+0000 7f33f119e640 1 --2- 172.21.15.84:0/1468085720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33d8005180 0x7f33d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:47.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.664+0000 7f33f119e640 1 -- 172.21.15.84:0/1468085720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33d80047c0 msgr2=0x7f33d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:47.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.664+0000 7f33f119e640 1 --2- 172.21.15.84:0/1468085720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33d80047c0 0x7f33d8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:47.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.664+0000 7f33f199f640 1 -- 172.21.15.84:0/1468085720 reap_dead start 2023-10-16T10:08:47.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.664+0000 7f33f119e640 1 -- 172.21.15.84:0/1468085720 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d8003520 con 0x7f33d80039c0 2023-10-16T10:08:47.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.665+0000 7f33f119e640 1 --2- 172.21.15.84:0/1468085720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33d80039c0 0x7f33d8003de0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f33e800c0e0 tx=0x7f33d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e1de2f860059a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:47.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.665+0000 7f33e3fff640 1 -- 172.21.15.84:0/1468085720 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e8005ad0 con 0x7f33d80039c0 2023-10-16T10:08:47.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.666+0000 7f33e3fff640 1 -- 172.21.15.84:0/1468085720 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e8005c30 con 0x7f33d80039c0 2023-10-16T10:08:47.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.666+0000 7f33e3fff640 1 -- 172.21.15.84:0/1468085720 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e802ca00 con 0x7f33d80039c0 2023-10-16T10:08:47.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.666+0000 7f33f3402640 1 -- 172.21.15.84:0/1468085720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33d80039c0 msgr2=0x7f33d8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:47.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.666+0000 7f33f3402640 1 --2- 172.21.15.84:0/1468085720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33d80039c0 0x7f33d8003de0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f33e800c0e0 tx=0x7f33d80057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:47.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.666+0000 7f33f3402640 1 -- 172.21.15.84:0/1468085720 shutdown_connections 2023-10-16T10:08:47.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.666+0000 7f33f3402640 1 --2- 172.21.15.84:0/1468085720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33d80039c0 0x7f33d8003de0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:47.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.666+0000 7f33f3402640 1 -- 172.21.15.84:0/1468085720 >> 172.21.15.84:0/1468085720 conn(0x7f33ec0ce9b0 msgr2=0x7f33ec0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:47.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.666+0000 7f33f3402640 1 -- 172.21.15.84:0/1468085720 shutdown_connections 2023-10-16T10:08:47.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.666+0000 7f33f3402640 1 -- 172.21.15.84:0/1468085720 wait complete. 2023-10-16T10:08:47.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.668+0000 7f33f3402640 1 Processor -- start 2023-10-16T10:08:47.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.668+0000 7f33f3402640 1 -- start start 2023-10-16T10:08:47.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.669+0000 7f33f3402640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33d80039c0 0x7f33ec157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:47.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.669+0000 7f33f3402640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33ec0c6490 0x7f33ec0a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:47.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.669+0000 7f33f3402640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33ec1efd80 0x7f33ec0b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:47.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.669+0000 7f33f3402640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0a9fb0 con 0x7f33ec1efd80 2023-10-16T10:08:47.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.669+0000 7f33f3402640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0c6910 con 0x7f33ec0c6490 2023-10-16T10:08:47.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.669+0000 7f33f3402640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0c6b90 con 0x7f33d80039c0 2023-10-16T10:08:47.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.669+0000 7f33f199f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33ec1efd80 0x7f33ec0b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:47.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.669+0000 7f33f199f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33ec1efd80 0x7f33ec0b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53136/0 (socket says 172.21.15.84:53136) 2023-10-16T10:08:47.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.669+0000 7f33f199f640 1 -- 172.21.15.84:0/4111301855 learned_addr learned my addr 172.21.15.84:0/4111301855 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:47.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.670+0000 7f33f119e640 1 --2- 172.21.15.84:0/4111301855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33d80039c0 0x7f33ec157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:47.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.670+0000 7f33f099d640 1 --2- 172.21.15.84:0/4111301855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33ec0c6490 0x7f33ec0a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:47.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.670+0000 7f33f199f640 1 -- 172.21.15.84:0/4111301855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33d80039c0 msgr2=0x7f33ec157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:47.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.670+0000 7f33f199f640 1 --2- 172.21.15.84:0/4111301855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33d80039c0 0x7f33ec157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:47.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.670+0000 7f33f199f640 1 -- 172.21.15.84:0/4111301855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33ec0c6490 msgr2=0x7f33ec0a9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:47.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.670+0000 7f33f199f640 1 --2- 172.21.15.84:0/4111301855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33ec0c6490 0x7f33ec0a9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:47.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.670+0000 7f33f199f640 1 -- 172.21.15.84:0/4111301855 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec0c7880 con 0x7f33ec1efd80 2023-10-16T10:08:47.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.670+0000 7f33f099d640 1 --2- 172.21.15.84:0/4111301855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33ec0c6490 0x7f33ec0a9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:47.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.670+0000 7f33f199f640 1 --2- 172.21.15.84:0/4111301855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33ec1efd80 0x7f33ec0b2650 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f33ec06d860 tx=0x7f33e400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:47.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.670+0000 7f33e1ffb640 1 -- 172.21.15.84:0/4111301855 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4010040 con 0x7f33ec1efd80 2023-10-16T10:08:47.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.670+0000 7f33e1ffb640 1 -- 172.21.15.84:0/4111301855 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e400bc80 con 0x7f33ec1efd80 2023-10-16T10:08:47.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.670+0000 7f33e1ffb640 1 -- 172.21.15.84:0/4111301855 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4014400 con 0x7f33ec1efd80 2023-10-16T10:08:47.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.670+0000 7f33f3402640 1 -- 172.21.15.84:0/4111301855 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ec0c7b60 con 0x7f33ec1efd80 2023-10-16T10:08:47.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.671+0000 7f33f3402640 1 -- 172.21.15.84:0/4111301855 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ec1f0230 con 0x7f33ec1efd80 2023-10-16T10:08:47.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.672+0000 7f33f3402640 1 -- 172.21.15.84:0/4111301855 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ec0c76e0 con 0x7f33ec1efd80 2023-10-16T10:08:47.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.673+0000 7f33e1ffb640 1 -- 172.21.15.84:0/4111301855 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 76) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f33e4014560 con 0x7f33ec1efd80 2023-10-16T10:08:47.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.674+0000 7f33e1ffb640 1 --2- 172.21.15.84:0/4111301855 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f33d006b070 0x7f33d006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:47.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.674+0000 7f33e1ffb640 1 -- 172.21.15.84:0/4111301855 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f33e408a190 con 0x7f33ec1efd80 2023-10-16T10:08:47.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.674+0000 7f33f119e640 1 --2- 172.21.15.84:0/4111301855 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f33d006b070 0x7f33d006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:47.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.675+0000 7f33f119e640 1 --2- 172.21.15.84:0/4111301855 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f33d006b070 0x7f33d006d530 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f33e8034000 tx=0x7f33e8009030 comp rx=0 tx=0).ready entity=mgr.8754 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:47.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.678+0000 7f33e1ffb640 1 -- 172.21.15.84:0/4111301855 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f33ec0c76e0 con 0x7f33ec1efd80 2023-10-16T10:08:47.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.892+0000 7f33f3402640 1 -- 172.21.15.84:0/4111301855 --> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f33ec0c9ca0 con 0x7f33d006b070 2023-10-16T10:08:47.896 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:47.895+0000 7fb27e1dd640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-16T10:08:47.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:47.895+0000 7f33e1ffb640 1 -- 172.21.15.84:0/4111301855 <== mgr.8754 v2:172.21.15.84:6832/1203729023 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7f33e80015f0 con 0x7f33d006b070 2023-10-16T10:08:47.897 INFO:teuthology.orchestra.run.smithi084.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-16T10:08:47.911 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T10:08:47.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:47.912 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/minutes) 2023-10-16T10:08:47.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:47.968 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/hourly) 2023-10-16T10:08:48.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:48.027 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/daily) 2023-10-16T10:08:48.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:48.083 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/weekly) 2023-10-16T10:08:48.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:48.137 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/monthly) 2023-10-16T10:08:48.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:48.194 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/yearly) 2023-10-16T10:08:48.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:48.250 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/bad_period_spec) 2023-10-16T10:08:48.305 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T10:08:48.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.518+0000 7f02c7fff640 1 Processor -- start 2023-10-16T10:08:48.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.518+0000 7f02c7fff640 1 -- start start 2023-10-16T10:08:48.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.519+0000 7f02c7fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f02c815cd70 0x7f02c80d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:48.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.519+0000 7f02c7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f02c80b8480 0x7f02c80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:48.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.519+0000 7f02c7fff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c815d150 con 0x7f02c80d0be0 2023-10-16T10:08:48.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.519+0000 7f02c7fff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02c80d1180 con 0x7f02c80b8480 2023-10-16T10:08:48.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.519+0000 7f02c7fff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02c80d1460 con 0x7f02c815cd70 2023-10-16T10:08:48.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.520+0000 7f02be7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f02c80b8480 0x7f02c80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:48.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.520+0000 7f02be7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f02c80b8480 0x7f02c80cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53144/0 (socket says 172.21.15.84:53144) 2023-10-16T10:08:48.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.520+0000 7f02be7fc640 1 -- 172.21.15.84:0/550154809 learned_addr learned my addr 172.21.15.84:0/550154809 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:48.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.520+0000 7f02c6ffd640 1 --2- 172.21.15.84:0/550154809 >> v2:172.21.15.157:3300/0 conn(0x7f02c815cd70 0x7f02c80d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:48.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.520+0000 7f02be7fc640 1 -- 172.21.15.84:0/550154809 >> v2:172.21.15.157:3300/0 conn(0x7f02c815cd70 msgr2=0x7f02c80d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:48.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.520+0000 7f02be7fc640 1 --2- 172.21.15.84:0/550154809 >> v2:172.21.15.157:3300/0 conn(0x7f02c815cd70 0x7f02c80d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:48.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.520+0000 7f02be7fc640 1 -- 172.21.15.84:0/550154809 >> v1:172.21.15.157:6789/0 conn(0x7f02c80d0be0 legacy=0x7f02c80b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:48.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.520+0000 7f02be7fc640 1 -- 172.21.15.84:0/550154809 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c80d1740 con 0x7f02c80b8480 2023-10-16T10:08:48.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.522+0000 7f02be7fc640 1 --2- 172.21.15.84:0/550154809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02c80b8480 0x7f02c80cd7c0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f02c80b56a0 tx=0x7f02b002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9598638e392c2ca server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:48.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.522+0000 7f02c67fc640 1 -- 172.21.15.84:0/550154809 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b00317e0 con 0x7f02c80b8480 2023-10-16T10:08:48.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.522+0000 7f02c67fc640 1 -- 172.21.15.84:0/550154809 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b0031940 con 0x7f02c80b8480 2023-10-16T10:08:48.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.522+0000 7f02c67fc640 1 -- 172.21.15.84:0/550154809 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0031cd0 con 0x7f02c80b8480 2023-10-16T10:08:48.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.522+0000 7f02c7fff640 1 -- 172.21.15.84:0/550154809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02c80b8480 msgr2=0x7f02c80cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:48.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.522+0000 7f02c7fff640 1 --2- 172.21.15.84:0/550154809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02c80b8480 0x7f02c80cd7c0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f02c80b56a0 tx=0x7f02b002b880 comp rx=0 tx=0).stop 2023-10-16T10:08:48.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.523+0000 7f02c7fff640 1 -- 172.21.15.84:0/550154809 shutdown_connections 2023-10-16T10:08:48.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.523+0000 7f02c7fff640 1 --2- 172.21.15.84:0/550154809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02c80b8480 0x7f02c80cd7c0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:48.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.523+0000 7f02c7fff640 1 --2- 172.21.15.84:0/550154809 >> v2:172.21.15.157:3300/0 conn(0x7f02c815cd70 0x7f02c80d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:48.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.523+0000 7f02c7fff640 1 -- 172.21.15.84:0/550154809 >> 172.21.15.84:0/550154809 conn(0x7f02c80cf570 msgr2=0x7f02c80cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:48.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.523+0000 7f02c7fff640 1 -- 172.21.15.84:0/550154809 shutdown_connections 2023-10-16T10:08:48.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.523+0000 7f02c7fff640 1 -- 172.21.15.84:0/550154809 wait complete. 2023-10-16T10:08:48.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.524+0000 7f02c7fff640 1 Processor -- start 2023-10-16T10:08:48.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.525+0000 7f02c7fff640 1 -- start start 2023-10-16T10:08:48.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.525+0000 7f02c7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c80b8480 0x7f02c8158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:48.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.525+0000 7f02c7fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c80d0be0 0x7f02c80a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:48.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.525+0000 7f02c7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02c815cd70 0x7f02c80a9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:48.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.525+0000 7f02c7fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c80cb550 con 0x7f02c815cd70 2023-10-16T10:08:48.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.525+0000 7f02c7fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c80ca8d0 con 0x7f02c80b8480 2023-10-16T10:08:48.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.525+0000 7f02c7fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c80cabb0 con 0x7f02c80d0be0 2023-10-16T10:08:48.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.526+0000 7f02c77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02c815cd70 0x7f02c80a9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:48.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.526+0000 7f02c77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02c815cd70 0x7f02c80a9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53160/0 (socket says 172.21.15.84:53160) 2023-10-16T10:08:48.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.526+0000 7f02c77fe640 1 -- 172.21.15.84:0/2577799265 learned_addr learned my addr 172.21.15.84:0/2577799265 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:48.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.526+0000 7f02be7fc640 1 --2- 172.21.15.84:0/2577799265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c80d0be0 0x7f02c80a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:48.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.526+0000 7f02c77fe640 1 -- 172.21.15.84:0/2577799265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c80d0be0 msgr2=0x7f02c80a94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:48.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.526+0000 7f02c77fe640 1 --2- 172.21.15.84:0/2577799265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c80d0be0 0x7f02c80a94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:48.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.526+0000 7f02c77fe640 1 -- 172.21.15.84:0/2577799265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c80b8480 msgr2=0x7f02c8158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:48.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.526+0000 7f02c6ffd640 1 --2- 172.21.15.84:0/2577799265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c80b8480 0x7f02c8158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:48.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.526+0000 7f02c77fe640 1 --2- 172.21.15.84:0/2577799265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c80b8480 0x7f02c8158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:48.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.526+0000 7f02c77fe640 1 -- 172.21.15.84:0/2577799265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b00094d0 con 0x7f02c815cd70 2023-10-16T10:08:48.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.526+0000 7f02be7fc640 1 --2- 172.21.15.84:0/2577799265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c80d0be0 0x7f02c80a94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:48.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.527+0000 7f02c77fe640 1 --2- 172.21.15.84:0/2577799265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02c815cd70 0x7f02c80a9a30 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f02c000c780 tx=0x7f02c000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:48.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.527+0000 7f02c4ff9640 1 -- 172.21.15.84:0/2577799265 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c0004b10 con 0x7f02c815cd70 2023-10-16T10:08:48.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.527+0000 7f02c4ff9640 1 -- 172.21.15.84:0/2577799265 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c0004c70 con 0x7f02c815cd70 2023-10-16T10:08:48.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.527+0000 7f02c7fff640 1 -- 172.21.15.84:0/2577799265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c80cae90 con 0x7f02c815cd70 2023-10-16T10:08:48.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.527+0000 7f02c4ff9640 1 -- 172.21.15.84:0/2577799265 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c000ce30 con 0x7f02c815cd70 2023-10-16T10:08:48.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.527+0000 7f02c7fff640 1 -- 172.21.15.84:0/2577799265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c8158640 con 0x7f02c815cd70 2023-10-16T10:08:48.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.528+0000 7f02c7fff640 1 -- 172.21.15.84:0/2577799265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c80d28f0 con 0x7f02c815cd70 2023-10-16T10:08:48.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.529+0000 7f02c4ff9640 1 -- 172.21.15.84:0/2577799265 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 76) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f02c000a850 con 0x7f02c815cd70 2023-10-16T10:08:48.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.530+0000 7f02c4ff9640 1 --2- 172.21.15.84:0/2577799265 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f02a806b070 0x7f02a806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:48.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.530+0000 7f02c4ff9640 1 -- 172.21.15.84:0/2577799265 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f02c0089c90 con 0x7f02c815cd70 2023-10-16T10:08:48.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.530+0000 7f02c6ffd640 1 --2- 172.21.15.84:0/2577799265 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f02a806b070 0x7f02a806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:48.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.531+0000 7f02c6ffd640 1 --2- 172.21.15.84:0/2577799265 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f02a806b070 0x7f02a806d530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f02b8006fd0 tx=0x7f02b8008040 comp rx=0 tx=0).ready entity=mgr.8754 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:48.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.535+0000 7f02c4ff9640 1 -- 172.21.15.84:0/2577799265 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f02c00548a0 con 0x7f02c815cd70 2023-10-16T10:08:48.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:48.779+0000 7f02c7fff640 1 -- 172.21.15.84:0/2577799265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f02c80cb690 con 0x7f02c815cd70 2023-10-16T10:08:49.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.035+0000 7f02c4ff9640 1 -- 172.21.15.84:0/2577799265 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v77) v1 ==== 93+0+0 (secure 0 0 0) 0x7f02c80cb690 con 0x7f02c815cd70 2023-10-16T10:08:49.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.038+0000 7f02c7fff640 1 -- 172.21.15.84:0/2577799265 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f02a806b070 msgr2=0x7f02a806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:49.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.038+0000 7f02c7fff640 1 --2- 172.21.15.84:0/2577799265 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f02a806b070 0x7f02a806d530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f02b8006fd0 tx=0x7f02b8008040 comp rx=0 tx=0).stop 2023-10-16T10:08:49.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.038+0000 7f02c7fff640 1 -- 172.21.15.84:0/2577799265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02c815cd70 msgr2=0x7f02c80a9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:49.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.038+0000 7f02c7fff640 1 --2- 172.21.15.84:0/2577799265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02c815cd70 0x7f02c80a9a30 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f02c000c780 tx=0x7f02c000cc40 comp rx=0 tx=0).stop 2023-10-16T10:08:49.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.039+0000 7f02c7fff640 1 -- 172.21.15.84:0/2577799265 shutdown_connections 2023-10-16T10:08:49.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.039+0000 7f02c7fff640 1 --2- 172.21.15.84:0/2577799265 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f02a806b070 0x7f02a806d530 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:49.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.039+0000 7f02c7fff640 1 --2- 172.21.15.84:0/2577799265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02c815cd70 0x7f02c80a9a30 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:49.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.039+0000 7f02c7fff640 1 --2- 172.21.15.84:0/2577799265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c80d0be0 0x7f02c80a94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:49.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.039+0000 7f02c7fff640 1 --2- 172.21.15.84:0/2577799265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c80b8480 0x7f02c8158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:49.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.039+0000 7f02c7fff640 1 -- 172.21.15.84:0/2577799265 >> 172.21.15.84:0/2577799265 conn(0x7f02c80cf570 msgr2=0x7f02c80cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:49.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.039+0000 7f02c7fff640 1 -- 172.21.15.84:0/2577799265 shutdown_connections 2023-10-16T10:08:49.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.039+0000 7f02c7fff640 1 -- 172.21.15.84:0/2577799265 wait complete. 2023-10-16T10:08:49.071 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:08:49.071 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:08:49.178 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:08:49.270 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:08:49.271 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:08:49.271 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:08:49.321 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:08:49.321 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:08:49.415 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:08:49.553 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:49.551+0000 7f5f914da040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:08:49.565 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:49.563+0000 7f099b258040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:08:49.606 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:49.605+0000 7f5f914da040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:08:49.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.613+0000 7f6f6d12c640 1 Processor -- start 2023-10-16T10:08:49.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.613+0000 7f6f6d12c640 1 -- start start 2023-10-16T10:08:49.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.614+0000 7f6f6d12c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6f6815cbe0 0x7f6f680d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:49.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.614+0000 7f6f6d12c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6f680b4c80 0x7f6f680b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:49.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.614+0000 7f6f6d12c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f680d15a0 con 0x7f6f6815cfc0 2023-10-16T10:08:49.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.614+0000 7f6f6d12c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f680d1880 con 0x7f6f680b4c80 2023-10-16T10:08:49.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.614+0000 7f6f6d12c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f680d1b60 con 0x7f6f6815cbe0 2023-10-16T10:08:49.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.615+0000 7f6f6c92b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6f680b4c80 0x7f6f680b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:49.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.615+0000 7f6f677fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6f6815cfc0 0x7f6f680b4570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46898/0 (socket says 172.21.15.84:46898) 2023-10-16T10:08:49.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.615+0000 7f6f677fe640 1 -- 172.21.15.84:0/3929047187 learned_addr learned my addr 172.21.15.84:0/3929047187 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:49.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.615+0000 7f6f6c92b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6f680b4c80 0x7f6f680b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53172/0 (socket says 172.21.15.84:53172) 2023-10-16T10:08:49.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.615+0000 7f6f67fff640 1 --2- 172.21.15.84:0/3929047187 >> v2:172.21.15.157:3300/0 conn(0x7f6f6815cbe0 0x7f6f680d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:49.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.615+0000 7f6f6c92b640 1 -- 172.21.15.84:0/3929047187 >> v2:172.21.15.157:3300/0 conn(0x7f6f6815cbe0 msgr2=0x7f6f680d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:49.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.615+0000 7f6f6c92b640 1 --2- 172.21.15.84:0/3929047187 >> v2:172.21.15.157:3300/0 conn(0x7f6f6815cbe0 0x7f6f680d0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:49.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.615+0000 7f6f6c92b640 1 -- 172.21.15.84:0/3929047187 >> v1:172.21.15.84:6789/0 conn(0x7f6f6815cfc0 legacy=0x7f6f680b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:49.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.615+0000 7f6f6c92b640 1 -- 172.21.15.84:0/3929047187 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f680d1e40 con 0x7f6f680b4c80 2023-10-16T10:08:49.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.615+0000 7f6f66ffd640 1 -- 172.21.15.84:0/3929047187 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824985989 0 0) 0x7f6f680d15a0 con 0x7f6f6815cfc0 2023-10-16T10:08:49.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.617+0000 7f6f6c92b640 1 --2- 172.21.15.84:0/3929047187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f680b4c80 0x7f6f680b5c20 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f6f680b53f0 tx=0x7f6f6002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02f9188e53c4094 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:49.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.617+0000 7f6f66ffd640 1 -- 172.21.15.84:0/3929047187 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6002cc80 con 0x7f6f680b4c80 2023-10-16T10:08:49.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.617+0000 7f6f66ffd640 1 -- 172.21.15.84:0/3929047187 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6002cde0 con 0x7f6f680b4c80 2023-10-16T10:08:49.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.617+0000 7f6f66ffd640 1 -- 172.21.15.84:0/3929047187 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f60034660 con 0x7f6f680b4c80 2023-10-16T10:08:49.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.618+0000 7f6f6d12c640 1 -- 172.21.15.84:0/3929047187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f680b4c80 msgr2=0x7f6f680b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:49.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.618+0000 7f6f6d12c640 1 --2- 172.21.15.84:0/3929047187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f680b4c80 0x7f6f680b5c20 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f6f680b53f0 tx=0x7f6f6002bed0 comp rx=0 tx=0).stop 2023-10-16T10:08:49.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.618+0000 7f6f6d12c640 1 -- 172.21.15.84:0/3929047187 shutdown_connections 2023-10-16T10:08:49.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.618+0000 7f6f6d12c640 1 --2- 172.21.15.84:0/3929047187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f680b4c80 0x7f6f680b5c20 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:49.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.618+0000 7f6f6d12c640 1 --2- 172.21.15.84:0/3929047187 >> v2:172.21.15.157:3300/0 conn(0x7f6f6815cbe0 0x7f6f680d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:49.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.618+0000 7f6f6d12c640 1 -- 172.21.15.84:0/3929047187 >> 172.21.15.84:0/3929047187 conn(0x7f6f680cf570 msgr2=0x7f6f680cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:49.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.618+0000 7f6f6d12c640 1 -- 172.21.15.84:0/3929047187 shutdown_connections 2023-10-16T10:08:49.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.618+0000 7f6f6d12c640 1 -- 172.21.15.84:0/3929047187 wait complete. 2023-10-16T10:08:49.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.620+0000 7f6f6d12c640 1 Processor -- start 2023-10-16T10:08:49.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.620+0000 7f6f6d12c640 1 -- start start 2023-10-16T10:08:49.624 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:49.620+0000 7f099b258040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:08:49.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.621+0000 7f6f6d12c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f680b4c80 0x7f6f68157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:49.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.621+0000 7f6f6d12c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f6815cbe0 0x7f6f68155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:49.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.621+0000 7f6f6d12c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f6815cfc0 0x7f6f681f5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:49.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.621+0000 7f6f6d12c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f681f61b0 con 0x7f6f680b4c80 2023-10-16T10:08:49.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.621+0000 7f6f6d12c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680b0f80 con 0x7f6f6815cbe0 2023-10-16T10:08:49.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.621+0000 7f6f6d12c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f681f5ed0 con 0x7f6f6815cfc0 2023-10-16T10:08:49.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.621+0000 7f6f67fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f680b4c80 0x7f6f68157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:49.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.621+0000 7f6f67fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f680b4c80 0x7f6f68157480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53182/0 (socket says 172.21.15.84:53182) 2023-10-16T10:08:49.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.621+0000 7f6f67fff640 1 -- 172.21.15.84:0/684068323 learned_addr learned my addr 172.21.15.84:0/684068323 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:49.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.622+0000 7f6f677fe640 1 --2- 172.21.15.84:0/684068323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f6815cbe0 0x7f6f68155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:49.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.622+0000 7f6f67fff640 1 -- 172.21.15.84:0/684068323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f6815cfc0 msgr2=0x7f6f681f5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:49.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.622+0000 7f6f67fff640 1 --2- 172.21.15.84:0/684068323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f6815cfc0 0x7f6f681f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:49.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.622+0000 7f6f67fff640 1 -- 172.21.15.84:0/684068323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f6815cbe0 msgr2=0x7f6f68155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:49.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.622+0000 7f6f67fff640 1 --2- 172.21.15.84:0/684068323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f6815cbe0 0x7f6f68155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:49.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.622+0000 7f6f67fff640 1 -- 172.21.15.84:0/684068323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f60009a70 con 0x7f6f680b4c80 2023-10-16T10:08:49.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.622+0000 7f6f67fff640 1 --2- 172.21.15.84:0/684068323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f680b4c80 0x7f6f68157480 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800e9b0 tx=0x7f6f5800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:49.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.622+0000 7f6f64ff9640 1 -- 172.21.15.84:0/684068323 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f58009670 con 0x7f6f680b4c80 2023-10-16T10:08:49.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.622+0000 7f6f64ff9640 1 -- 172.21.15.84:0/684068323 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5800c9a0 con 0x7f6f680b4c80 2023-10-16T10:08:49.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.622+0000 7f6f64ff9640 1 -- 172.21.15.84:0/684068323 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800cd50 con 0x7f6f680b4c80 2023-10-16T10:08:49.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.623+0000 7f6f6d12c640 1 -- 172.21.15.84:0/684068323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f681569a0 con 0x7f6f680b4c80 2023-10-16T10:08:49.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.623+0000 7f6f6d12c640 1 -- 172.21.15.84:0/684068323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f68156f60 con 0x7f6f680b4c80 2023-10-16T10:08:49.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.624+0000 7f6f6d12c640 1 -- 172.21.15.84:0/684068323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f6804e9b0 con 0x7f6f680b4c80 2023-10-16T10:08:49.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.625+0000 7f6f64ff9640 1 -- 172.21.15.84:0/684068323 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 77) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f6f58013470 con 0x7f6f680b4c80 2023-10-16T10:08:49.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.626+0000 7f6f64ff9640 1 --2- 172.21.15.84:0/684068323 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f6f5406b020 0x7f6f5406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:49.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.626+0000 7f6f64ff9640 1 -- 172.21.15.84:0/684068323 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f6f58089cc0 con 0x7f6f680b4c80 2023-10-16T10:08:49.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.626+0000 7f6f677fe640 1 -- 172.21.15.84:0/684068323 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f6f5406b020 msgr2=0x7f6f5406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1203729023 2023-10-16T10:08:49.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.626+0000 7f6f677fe640 1 --2- 172.21.15.84:0/684068323 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f6f5406b020 0x7f6f5406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:49.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.634+0000 7f6f64ff9640 1 -- 172.21.15.84:0/684068323 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6f58054970 con 0x7f6f680b4c80 2023-10-16T10:08:49.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.827+0000 7f6f677fe640 1 -- 172.21.15.84:0/684068323 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f6f5406b020 msgr2=0x7f6f5406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1203729023 2023-10-16T10:08:49.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.827+0000 7f6f677fe640 1 --2- 172.21.15.84:0/684068323 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f6f5406b020 0x7f6f5406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:08:49.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.828+0000 7f6f6d12c640 1 -- 172.21.15.84:0/684068323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f680b40f0 con 0x7f6f680b4c80 2023-10-16T10:08:49.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.830+0000 7f6f64ff9640 1 -- 172.21.15.84:0/684068323 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+17047 (secure 0 0 0) 0x7f6f58017090 con 0x7f6f680b4c80 2023-10-16T10:08:49.832 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:49.833 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":139,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:08:40.209311+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:08:23.558506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-16T10:08:24.709464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2708741108":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/914481139":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/4141270981":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/4132563801":"2023-10-17T10:08:40.209237+0000","172.21.15.84:6833/2784642648":"2023-10-17T10:08:40.209237+0000","172.21.15.84:6832/2784642648":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/2180503356":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/2933995314":"2023-10-17T10:08:40.209237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:08:49.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.833+0000 7f6f6d12c640 1 -- 172.21.15.84:0/684068323 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f6f5406b020 msgr2=0x7f6f5406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:49.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.833+0000 7f6f6d12c640 1 --2- 172.21.15.84:0/684068323 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f6f5406b020 0x7f6f5406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:49.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.833+0000 7f6f6d12c640 1 -- 172.21.15.84:0/684068323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f680b4c80 msgr2=0x7f6f68157480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:49.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.833+0000 7f6f6d12c640 1 --2- 172.21.15.84:0/684068323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f680b4c80 0x7f6f68157480 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800e9b0 tx=0x7f6f5800ee70 comp rx=0 tx=0).stop 2023-10-16T10:08:49.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.833+0000 7f6f6d12c640 1 -- 172.21.15.84:0/684068323 shutdown_connections 2023-10-16T10:08:49.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.833+0000 7f6f6d12c640 1 --2- 172.21.15.84:0/684068323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f6815cfc0 0x7f6f681f5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:49.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.833+0000 7f6f6d12c640 1 --2- 172.21.15.84:0/684068323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f6815cbe0 0x7f6f68155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:49.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.833+0000 7f6f6d12c640 1 --2- 172.21.15.84:0/684068323 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f6f5406b020 0x7f6f5406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:49.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.833+0000 7f6f6d12c640 1 --2- 172.21.15.84:0/684068323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f680b4c80 0x7f6f68157480 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:49.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.833+0000 7f6f6d12c640 1 -- 172.21.15.84:0/684068323 >> 172.21.15.84:0/684068323 conn(0x7f6f680cf570 msgr2=0x7f6f680b43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:49.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.833+0000 7f6f6d12c640 1 -- 172.21.15.84:0/684068323 shutdown_connections 2023-10-16T10:08:49.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:49.833+0000 7f6f6d12c640 1 -- 172.21.15.84:0/684068323 wait complete. 2023-10-16T10:08:49.853 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:08:23.558506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:08:24.709464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:08:49.853 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:08:50.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.049+0000 7f4b40a14640 1 Processor -- start 2023-10-16T10:08:50.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.049+0000 7f4b40a14640 1 -- start start 2023-10-16T10:08:50.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.051+0000 7f4b40a14640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4b3c0ce250 0x7f4b3c15e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.051+0000 7f4b40a14640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4b3c070970 0x7f4b3c15eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.051+0000 7f4b40a14640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b3c06fbc0 con 0x7f4b3c15e350 2023-10-16T10:08:50.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.051+0000 7f4b40a14640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b3c15e730 con 0x7f4b3c070970 2023-10-16T10:08:50.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.051+0000 7f4b40a14640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b3c0c9920 con 0x7f4b3c0ce250 2023-10-16T10:08:50.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.051+0000 7f4b3b59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4b3c070970 0x7f4b3c15eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:50.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.051+0000 7f4b3b59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4b3c070970 0x7f4b3c15eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53190/0 (socket says 172.21.15.84:53190) 2023-10-16T10:08:50.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.051+0000 7f4b3b59e640 1 -- 172.21.15.84:0/3807858276 learned_addr learned my addr 172.21.15.84:0/3807858276 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:50.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.051+0000 7f4b3259c640 1 --2- 172.21.15.84:0/3807858276 >> v2:172.21.15.157:3300/0 conn(0x7f4b3c0ce250 0x7f4b3c15e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:50.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.052+0000 7f4b3b59e640 1 -- 172.21.15.84:0/3807858276 >> v2:172.21.15.157:3300/0 conn(0x7f4b3c0ce250 msgr2=0x7f4b3c15e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:50.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.052+0000 7f4b3b59e640 1 --2- 172.21.15.84:0/3807858276 >> v2:172.21.15.157:3300/0 conn(0x7f4b3c0ce250 0x7f4b3c15e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.052+0000 7f4b3b59e640 1 -- 172.21.15.84:0/3807858276 >> v1:172.21.15.157:6789/0 conn(0x7f4b3c15e350 legacy=0x7f4b3c0cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:50.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.052+0000 7f4b3b59e640 1 -- 172.21.15.84:0/3807858276 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c15f400 con 0x7f4b3c070970 2023-10-16T10:08:50.057 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:50.051+0000 7f5f914da040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:08:50.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.053+0000 7f4b3b59e640 1 --2- 172.21.15.84:0/3807858276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b3c070970 0x7f4b3c15eec0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0671f0 tx=0x7f4b3402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5993aacbf5172573 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:50.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.054+0000 7f4b3a59c640 1 -- 172.21.15.84:0/3807858276 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3402c9f0 con 0x7f4b3c070970 2023-10-16T10:08:50.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.054+0000 7f4b3a59c640 1 -- 172.21.15.84:0/3807858276 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3402cb50 con 0x7f4b3c070970 2023-10-16T10:08:50.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.054+0000 7f4b3a59c640 1 -- 172.21.15.84:0/3807858276 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b340343f0 con 0x7f4b3c070970 2023-10-16T10:08:50.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.054+0000 7f4b40a14640 1 -- 172.21.15.84:0/3807858276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b3c070970 msgr2=0x7f4b3c15eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:50.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.054+0000 7f4b40a14640 1 --2- 172.21.15.84:0/3807858276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b3c070970 0x7f4b3c15eec0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0671f0 tx=0x7f4b3402be50 comp rx=0 tx=0).stop 2023-10-16T10:08:50.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.055+0000 7f4b40a14640 1 -- 172.21.15.84:0/3807858276 shutdown_connections 2023-10-16T10:08:50.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.055+0000 7f4b40a14640 1 --2- 172.21.15.84:0/3807858276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b3c070970 0x7f4b3c15eec0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.055+0000 7f4b40a14640 1 --2- 172.21.15.84:0/3807858276 >> v2:172.21.15.157:3300/0 conn(0x7f4b3c0ce250 0x7f4b3c15e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.055+0000 7f4b40a14640 1 -- 172.21.15.84:0/3807858276 >> 172.21.15.84:0/3807858276 conn(0x7f4b3c0b0a50 msgr2=0x7f4b3c0b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:50.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.055+0000 7f4b40a14640 1 -- 172.21.15.84:0/3807858276 shutdown_connections 2023-10-16T10:08:50.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.055+0000 7f4b40a14640 1 -- 172.21.15.84:0/3807858276 wait complete. 2023-10-16T10:08:50.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.056+0000 7f4b40a14640 1 Processor -- start 2023-10-16T10:08:50.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.057+0000 7f4b40a14640 1 -- start start 2023-10-16T10:08:50.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.058+0000 7f4b40a14640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b3c070970 0x7f4b3c09c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.058+0000 7f4b40a14640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b3c0ce250 0x7f4b3c14bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.058+0000 7f4b40a14640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b3c15e350 0x7f4b3c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.058+0000 7f4b40a14640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c09c900 con 0x7f4b3c0ce250 2023-10-16T10:08:50.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.058+0000 7f4b40a14640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c14c1b0 con 0x7f4b3c15e350 2023-10-16T10:08:50.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.058+0000 7f4b40a14640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c09b320 con 0x7f4b3c070970 2023-10-16T10:08:50.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.058+0000 7f4b3259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b3c0ce250 0x7f4b3c14bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:50.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.058+0000 7f4b3259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b3c0ce250 0x7f4b3c14bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53196/0 (socket says 172.21.15.84:53196) 2023-10-16T10:08:50.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.058+0000 7f4b3259c640 1 -- 172.21.15.84:0/425943109 learned_addr learned my addr 172.21.15.84:0/425943109 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:50.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.058+0000 7f4b3ad9d640 1 --2- 172.21.15.84:0/425943109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b3c070970 0x7f4b3c09c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:50.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.058+0000 7f4b3259c640 1 -- 172.21.15.84:0/425943109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b3c070970 msgr2=0x7f4b3c09c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:50.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.058+0000 7f4b3259c640 1 --2- 172.21.15.84:0/425943109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b3c070970 0x7f4b3c09c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.058+0000 7f4b3b59e640 1 --2- 172.21.15.84:0/425943109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b3c15e350 0x7f4b3c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:50.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.058+0000 7f4b3259c640 1 -- 172.21.15.84:0/425943109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b3c15e350 msgr2=0x7f4b3c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:50.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.058+0000 7f4b3259c640 1 --2- 172.21.15.84:0/425943109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b3c15e350 0x7f4b3c0afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.058+0000 7f4b3259c640 1 -- 172.21.15.84:0/425943109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b34009b30 con 0x7f4b3c0ce250 2023-10-16T10:08:50.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.058+0000 7f4b3ad9d640 1 --2- 172.21.15.84:0/425943109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b3c070970 0x7f4b3c09c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:50.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.059+0000 7f4b3259c640 1 --2- 172.21.15.84:0/425943109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b3c0ce250 0x7f4b3c14bc70 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400d5f0 tx=0x7f4b2400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:50.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.059+0000 7f4b33fff640 1 -- 172.21.15.84:0/425943109 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b240125f0 con 0x7f4b3c0ce250 2023-10-16T10:08:50.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.059+0000 7f4b33fff640 1 -- 172.21.15.84:0/425943109 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b24012750 con 0x7f4b3c0ce250 2023-10-16T10:08:50.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.059+0000 7f4b33fff640 1 -- 172.21.15.84:0/425943109 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b24012b00 con 0x7f4b3c0ce250 2023-10-16T10:08:50.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.059+0000 7f4b40a14640 1 -- 172.21.15.84:0/425943109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b3c09b600 con 0x7f4b3c0ce250 2023-10-16T10:08:50.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.059+0000 7f4b40a14640 1 -- 172.21.15.84:0/425943109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3c09bbc0 con 0x7f4b3c0ce250 2023-10-16T10:08:50.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.060+0000 7f4b40a14640 1 -- 172.21.15.84:0/425943109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b3c06e500 con 0x7f4b3c0ce250 2023-10-16T10:08:50.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.061+0000 7f4b33fff640 1 -- 172.21.15.84:0/425943109 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 77) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f4b24012c60 con 0x7f4b3c0ce250 2023-10-16T10:08:50.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.062+0000 7f4b33fff640 1 --2- 172.21.15.84:0/425943109 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f4b1c06b020 0x7f4b1c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.062+0000 7f4b3ad9d640 1 -- 172.21.15.84:0/425943109 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f4b1c06b020 msgr2=0x7f4b1c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1203729023 2023-10-16T10:08:50.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.062+0000 7f4b33fff640 1 -- 172.21.15.84:0/425943109 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f4b2408a0d0 con 0x7f4b3c0ce250 2023-10-16T10:08:50.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.062+0000 7f4b3ad9d640 1 --2- 172.21.15.84:0/425943109 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f4b1c06b020 0x7f4b1c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:50.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.070+0000 7f4b33fff640 1 -- 172.21.15.84:0/425943109 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4b24054c50 con 0x7f4b3c0ce250 2023-10-16T10:08:50.095 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:50.093+0000 7f099b258040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:08:50.105 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:50.103+0000 7f5f914da040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:08:50.149 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:50.147+0000 7f099b258040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:08:50.155 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:50.154+0000 7f5f914da040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:08:50.200 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:50.198+0000 7f099b258040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:08:50.216 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:50.215+0000 7f5f914da040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:08:50.263 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:50.261+0000 7f099b258040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:08:50.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.263+0000 7f4b3ad9d640 1 -- 172.21.15.84:0/425943109 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f4b1c06b020 msgr2=0x7f4b1c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1203729023 2023-10-16T10:08:50.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.263+0000 7f4b3ad9d640 1 --2- 172.21.15.84:0/425943109 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f4b1c06b020 0x7f4b1c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:08:50.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.264+0000 7f4b40a14640 1 -- 172.21.15.84:0/425943109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b3c0716e0 con 0x7f4b3c0ce250 2023-10-16T10:08:50.266 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:50.264+0000 7f5f914da040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:08:50.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.264+0000 7f4b33fff640 1 -- 172.21.15.84:0/425943109 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b24058270 con 0x7f4b3c0ce250 2023-10-16T10:08:50.267 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:08:50.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.267+0000 7f4b40a14640 1 -- 172.21.15.84:0/425943109 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f4b1c06b020 msgr2=0x7f4b1c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:50.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.267+0000 7f4b40a14640 1 --2- 172.21.15.84:0/425943109 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f4b1c06b020 0x7f4b1c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.267+0000 7f4b40a14640 1 -- 172.21.15.84:0/425943109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b3c0ce250 msgr2=0x7f4b3c14bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:50.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.267+0000 7f4b40a14640 1 --2- 172.21.15.84:0/425943109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b3c0ce250 0x7f4b3c14bc70 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400d5f0 tx=0x7f4b2400dab0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.267+0000 7f4b40a14640 1 -- 172.21.15.84:0/425943109 shutdown_connections 2023-10-16T10:08:50.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.267+0000 7f4b40a14640 1 --2- 172.21.15.84:0/425943109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b3c15e350 0x7f4b3c0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.267+0000 7f4b40a14640 1 --2- 172.21.15.84:0/425943109 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f4b1c06b020 0x7f4b1c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.267+0000 7f4b40a14640 1 --2- 172.21.15.84:0/425943109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b3c0ce250 0x7f4b3c14bc70 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.267+0000 7f4b40a14640 1 --2- 172.21.15.84:0/425943109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b3c070970 0x7f4b3c09c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.267+0000 7f4b40a14640 1 -- 172.21.15.84:0/425943109 >> 172.21.15.84:0/425943109 conn(0x7f4b3c0b0a50 msgr2=0x7f4b3c159d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:50.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.267+0000 7f4b40a14640 1 -- 172.21.15.84:0/425943109 shutdown_connections 2023-10-16T10:08:50.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.267+0000 7f4b40a14640 1 -- 172.21.15.84:0/425943109 wait complete. 2023-10-16T10:08:50.288 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:08:50.318 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:50.317+0000 7f099b258040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:08:50.380 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:50.378+0000 7f5f914da040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:08:50.434 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:50.433+0000 7f099b258040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:08:50.442 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:50.441+0000 7f5f914da040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:08:50.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.487+0000 7f76688a2640 1 Processor -- start 2023-10-16T10:08:50.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.487+0000 7f76688a2640 1 -- start start 2023-10-16T10:08:50.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.489+0000 7f76688a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f766415d330 0x7f76641618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.489+0000 7f76688a2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76640d0890 con 0x7f766406c7d0 2023-10-16T10:08:50.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.489+0000 7f76688a2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76640d05b0 con 0x7f76640661c0 2023-10-16T10:08:50.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.489+0000 7f76688a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766406b1e0 con 0x7f766415d330 2023-10-16T10:08:50.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.489+0000 7f765bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f766406c7d0 0x7f7664065ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46906/0 (socket says 172.21.15.84:46906) 2023-10-16T10:08:50.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.489+0000 7f765bfff640 1 -- 172.21.15.84:0/380841532 learned_addr learned my addr 172.21.15.84:0/380841532 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:50.492 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:50.490+0000 7f5f914da040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:08:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.489+0000 7f766259c640 1 --2- 172.21.15.84:0/380841532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f766415d330 0x7f76641618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.490+0000 7f7661d9b640 1 -- 172.21.15.84:0/380841532 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480550624 0 0) 0x7f76640d0890 con 0x7f766406c7d0 2023-10-16T10:08:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.490+0000 7f7661d9b640 1 -- 172.21.15.84:0/380841532 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7650003520 con 0x7f766406c7d0 2023-10-16T10:08:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.490+0000 7f7661d9b640 1 -- 172.21.15.84:0/380841532 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1919718585 0 0) 0x7f7650003520 con 0x7f766406c7d0 2023-10-16T10:08:50.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.490+0000 7f7661d9b640 1 -- 172.21.15.84:0/380841532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f766415d330 msgr2=0x7f76641618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:50.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.490+0000 7f7661d9b640 1 --2- 172.21.15.84:0/380841532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f766415d330 0x7f76641618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.490+0000 7f7661d9b640 1 -- 172.21.15.84:0/380841532 >> v1:172.21.15.157:6789/0 conn(0x7f76640661c0 legacy=0x7f76640ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:50.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.490+0000 7f7661d9b640 1 -- 172.21.15.84:0/380841532 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7664160dd0 con 0x7f766406c7d0 2023-10-16T10:08:50.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.490+0000 7f7661d9b640 1 -- 172.21.15.84:0/380841532 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f764c001670 con 0x7f766406c7d0 2023-10-16T10:08:50.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.490+0000 7f7661d9b640 1 -- 172.21.15.84:0/380841532 >> v1:172.21.15.84:6789/0 conn(0x7f766406c7d0 legacy=0x7f7664065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:50.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.490+0000 7f7661d9b640 1 --2- 172.21.15.84:0/380841532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76500039a0 0x7f7650003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.490+0000 7f7661d9b640 1 --2- 172.21.15.84:0/380841532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76500047c0 0x7f7650004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.490+0000 7f7661d9b640 1 --2- 172.21.15.84:0/380841532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f766415d330 0x7f76500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.490+0000 7f7661d9b640 1 -- 172.21.15.84:0/380841532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766406b1e0 con 0x7f76500047c0 2023-10-16T10:08:50.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.490+0000 7f7661d9b640 1 -- 172.21.15.84:0/380841532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765000bab0 con 0x7f76500039a0 2023-10-16T10:08:50.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.490+0000 7f7661d9b640 1 -- 172.21.15.84:0/380841532 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765000bc70 con 0x7f766415d330 2023-10-16T10:08:50.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.491+0000 7f765bfff640 1 --2- 172.21.15.84:0/380841532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76500047c0 0x7f7650004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:50.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.491+0000 7f766259c640 1 --2- 172.21.15.84:0/380841532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76500039a0 0x7f7650003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:50.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.491+0000 7f7662d9d640 1 --2- 172.21.15.84:0/380841532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f766415d330 0x7f76500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:50.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.491+0000 7f765bfff640 1 -- 172.21.15.84:0/380841532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f766415d330 msgr2=0x7f76500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:50.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.491+0000 7f765bfff640 1 --2- 172.21.15.84:0/380841532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f766415d330 0x7f76500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.491+0000 7f765bfff640 1 -- 172.21.15.84:0/380841532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76500039a0 msgr2=0x7f7650003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:50.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.491+0000 7f765bfff640 1 --2- 172.21.15.84:0/380841532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76500039a0 0x7f7650003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.491+0000 7f765bfff640 1 -- 172.21.15.84:0/380841532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7650003520 con 0x7f76500047c0 2023-10-16T10:08:50.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.492+0000 7f765bfff640 1 --2- 172.21.15.84:0/380841532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76500047c0 0x7f7650004c40 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f764c00c240 tx=0x7f76500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c213537cfd5270b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:50.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.493+0000 7f7661d9b640 1 -- 172.21.15.84:0/380841532 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c005ad0 con 0x7f76500047c0 2023-10-16T10:08:50.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.493+0000 7f7661d9b640 1 -- 172.21.15.84:0/380841532 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c005c50 con 0x7f76500047c0 2023-10-16T10:08:50.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.493+0000 7f7661d9b640 1 -- 172.21.15.84:0/380841532 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c02c8b0 con 0x7f76500047c0 2023-10-16T10:08:50.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.493+0000 7f76688a2640 1 -- 172.21.15.84:0/380841532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76500047c0 msgr2=0x7f7650004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:50.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.493+0000 7f76688a2640 1 --2- 172.21.15.84:0/380841532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76500047c0 0x7f7650004c40 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f764c00c240 tx=0x7f76500057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.493+0000 7f7662d9d640 1 -- 172.21.15.84:0/380841532 reap_dead start 2023-10-16T10:08:50.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.493+0000 7f76688a2640 1 -- 172.21.15.84:0/380841532 shutdown_connections 2023-10-16T10:08:50.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.493+0000 7f76688a2640 1 -- 172.21.15.84:0/380841532 >> 172.21.15.84:0/380841532 conn(0x7f76640d0ab0 msgr2=0x7f76640d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:50.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.493+0000 7f76688a2640 1 -- 172.21.15.84:0/380841532 shutdown_connections 2023-10-16T10:08:50.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.494+0000 7f76688a2640 1 -- 172.21.15.84:0/380841532 wait complete. 2023-10-16T10:08:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.495+0000 7f76688a2640 1 Processor -- start 2023-10-16T10:08:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.495+0000 7f76688a2640 1 -- start start 2023-10-16T10:08:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.496+0000 7f76688a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76500047c0 0x7f76640b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.496+0000 7f76688a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7664154930 0x7f76640cec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.496+0000 7f76688a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76640b4cb0 0x7f7664153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.496+0000 7f76688a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76640b3c10 con 0x7f76640b4cb0 2023-10-16T10:08:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.496+0000 7f76688a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76640b3ef0 con 0x7f76500047c0 2023-10-16T10:08:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.496+0000 7f76688a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76640b41d0 con 0x7f7664154930 2023-10-16T10:08:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.497+0000 7f7662d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76640b4cb0 0x7f7664153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.497+0000 7f7662d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76640b4cb0 0x7f7664153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53226/0 (socket says 172.21.15.84:53226) 2023-10-16T10:08:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.497+0000 7f7662d9d640 1 -- 172.21.15.84:0/1962017402 learned_addr learned my addr 172.21.15.84:0/1962017402 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.497+0000 7f766259c640 1 --2- 172.21.15.84:0/1962017402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76500047c0 0x7f76640b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.497+0000 7f765bfff640 1 --2- 172.21.15.84:0/1962017402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7664154930 0x7f76640cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.497+0000 7f7662d9d640 1 -- 172.21.15.84:0/1962017402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7664154930 msgr2=0x7f76640cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.497+0000 7f7662d9d640 1 --2- 172.21.15.84:0/1962017402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7664154930 0x7f76640cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.497+0000 7f7662d9d640 1 -- 172.21.15.84:0/1962017402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76500047c0 msgr2=0x7f76640b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.497+0000 7f7662d9d640 1 --2- 172.21.15.84:0/1962017402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76500047c0 0x7f76640b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.497+0000 7f7662d9d640 1 -- 172.21.15.84:0/1962017402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76640b44b0 con 0x7f76640b4cb0 2023-10-16T10:08:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.498+0000 7f7662d9d640 1 --2- 172.21.15.84:0/1962017402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76640b4cb0 0x7f7664153d60 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f766406f340 tx=0x7f765c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.498+0000 7f765affd640 1 -- 172.21.15.84:0/1962017402 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c003970 con 0x7f76640b4cb0 2023-10-16T10:08:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.498+0000 7f765affd640 1 -- 172.21.15.84:0/1962017402 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765c003ad0 con 0x7f76640b4cb0 2023-10-16T10:08:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.498+0000 7f76688a2640 1 -- 172.21.15.84:0/1962017402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76640b4790 con 0x7f76640b4cb0 2023-10-16T10:08:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.498+0000 7f765affd640 1 -- 172.21.15.84:0/1962017402 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c003e80 con 0x7f76640b4cb0 2023-10-16T10:08:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.498+0000 7f76688a2640 1 -- 172.21.15.84:0/1962017402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76640cf160 con 0x7f76640b4cb0 2023-10-16T10:08:50.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.499+0000 7f76688a2640 1 -- 172.21.15.84:0/1962017402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76640cf350 con 0x7f76640b4cb0 2023-10-16T10:08:50.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.500+0000 7f765affd640 1 -- 172.21.15.84:0/1962017402 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 77) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f765c0050d0 con 0x7f76640b4cb0 2023-10-16T10:08:50.510 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:50.500+0000 7f099b258040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:08:50.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.501+0000 7f765affd640 1 --2- 172.21.15.84:0/1962017402 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f764406b050 0x7f764406d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.501+0000 7f765affd640 1 -- 172.21.15.84:0/1962017402 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f765c089880 con 0x7f76640b4cb0 2023-10-16T10:08:50.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.501+0000 7f766259c640 1 -- 172.21.15.84:0/1962017402 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f764406b050 msgr2=0x7f764406d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1203729023 2023-10-16T10:08:50.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.501+0000 7f766259c640 1 --2- 172.21.15.84:0/1962017402 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f764406b050 0x7f764406d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:50.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.506+0000 7f765affd640 1 -- 172.21.15.84:0/1962017402 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f765c054520 con 0x7f76640b4cb0 2023-10-16T10:08:50.550 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:50.549+0000 7f5f914da040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:08:50.554 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:50.553+0000 7f099b258040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:08:50.600 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:50.599+0000 7f5f914da040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:08:50.613 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:50.611+0000 7f099b258040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:08:50.665 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:50.663+0000 7f099b258040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:08:50.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.699+0000 7f76688a2640 1 -- 172.21.15.84:0/1962017402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f766404e160 con 0x7f76640b4cb0 2023-10-16T10:08:50.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.700+0000 7f765affd640 1 -- 172.21.15.84:0/1962017402 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v139) v1 ==== 104+0+11 (secure 0 0 0) 0x7f765c057b40 con 0x7f76640b4cb0 2023-10-16T10:08:50.702 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T10:08:50.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.702+0000 7f766259c640 1 -- 172.21.15.84:0/1962017402 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f764406b050 msgr2=0x7f764406d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1203729023 2023-10-16T10:08:50.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.702+0000 7f766259c640 1 --2- 172.21.15.84:0/1962017402 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f764406b050 0x7f764406d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:08:50.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.702+0000 7f76688a2640 1 -- 172.21.15.84:0/1962017402 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f764406b050 msgr2=0x7f764406d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:50.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.702+0000 7f76688a2640 1 --2- 172.21.15.84:0/1962017402 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f764406b050 0x7f764406d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.702+0000 7f76688a2640 1 -- 172.21.15.84:0/1962017402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76640b4cb0 msgr2=0x7f7664153d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:50.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.702+0000 7f76688a2640 1 --2- 172.21.15.84:0/1962017402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76640b4cb0 0x7f7664153d60 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f766406f340 tx=0x7f765c00ec10 comp rx=0 tx=0).stop 2023-10-16T10:08:50.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.702+0000 7f76688a2640 1 -- 172.21.15.84:0/1962017402 shutdown_connections 2023-10-16T10:08:50.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.702+0000 7f76688a2640 1 --2- 172.21.15.84:0/1962017402 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7f764406b050 0x7f764406d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.702+0000 7f76688a2640 1 --2- 172.21.15.84:0/1962017402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76640b4cb0 0x7f7664153d60 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.702+0000 7f76688a2640 1 --2- 172.21.15.84:0/1962017402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7664154930 0x7f76640cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.702+0000 7f76688a2640 1 --2- 172.21.15.84:0/1962017402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76500047c0 0x7f76640b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.702+0000 7f76688a2640 1 -- 172.21.15.84:0/1962017402 >> 172.21.15.84:0/1962017402 conn(0x7f76640d0ab0 msgr2=0x7f766415ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:50.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.703+0000 7f76688a2640 1 -- 172.21.15.84:0/1962017402 shutdown_connections 2023-10-16T10:08:50.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.703+0000 7f76688a2640 1 -- 172.21.15.84:0/1962017402 wait complete. 2023-10-16T10:08:50.721 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:08:50.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.969+0000 7fcefa84b640 1 Processor -- start 2023-10-16T10:08:50.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.970+0000 7fcefa84b640 1 -- start start 2023-10-16T10:08:50.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.971+0000 7fcefa84b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcef4065ab0 0x7fcef4161100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.971+0000 7fcefa84b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcef4069fa0 0x7fcef406a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.971+0000 7fcefa84b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef40ac890 con 0x7fcef4160530 2023-10-16T10:08:50.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.971+0000 7fcefa84b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef40ae800 con 0x7fcef4069fa0 2023-10-16T10:08:50.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.971+0000 7fcefa84b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef40ae1f0 con 0x7fcef4065ab0 2023-10-16T10:08:50.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.971+0000 7fcef8de8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcef4069fa0 0x7fcef406a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:50.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.971+0000 7fcef8de8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcef4069fa0 0x7fcef406a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53240/0 (socket says 172.21.15.84:53240) 2023-10-16T10:08:50.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.971+0000 7fcef8de8640 1 -- 172.21.15.84:0/1013824312 learned_addr learned my addr 172.21.15.84:0/1013824312 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:50.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.972+0000 7fcef8de8640 1 -- 172.21.15.84:0/1013824312 >> v2:172.21.15.157:3300/0 conn(0x7fcef4065ab0 msgr2=0x7fcef4161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:50.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.972+0000 7fcef8de8640 1 --2- 172.21.15.84:0/1013824312 >> v2:172.21.15.157:3300/0 conn(0x7fcef4065ab0 0x7fcef4161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.972+0000 7fcef8de8640 1 -- 172.21.15.84:0/1013824312 >> v1:172.21.15.157:6789/0 conn(0x7fcef4160530 legacy=0x7fcef41609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:50.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.972+0000 7fcef8de8640 1 -- 172.21.15.84:0/1013824312 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef40acae0 con 0x7fcef4069fa0 2023-10-16T10:08:50.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.973+0000 7fcef8de8640 1 --2- 172.21.15.84:0/1013824312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef4069fa0 0x7fcef406a430 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fcef406f640 tx=0x7fceec02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677e334efebdf43d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:50.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.973+0000 7fcef37fe640 1 -- 172.21.15.84:0/1013824312 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec02c9f0 con 0x7fcef4069fa0 2023-10-16T10:08:50.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.973+0000 7fcef37fe640 1 -- 172.21.15.84:0/1013824312 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec02cb50 con 0x7fcef4069fa0 2023-10-16T10:08:50.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.973+0000 7fcef37fe640 1 -- 172.21.15.84:0/1013824312 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0343f0 con 0x7fcef4069fa0 2023-10-16T10:08:50.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.974+0000 7fcefa84b640 1 -- 172.21.15.84:0/1013824312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef4069fa0 msgr2=0x7fcef406a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:50.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.974+0000 7fcefa84b640 1 --2- 172.21.15.84:0/1013824312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef4069fa0 0x7fcef406a430 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fcef406f640 tx=0x7fceec02be50 comp rx=0 tx=0).stop 2023-10-16T10:08:50.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.974+0000 7fcefa84b640 1 -- 172.21.15.84:0/1013824312 shutdown_connections 2023-10-16T10:08:50.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.974+0000 7fcefa84b640 1 --2- 172.21.15.84:0/1013824312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef4069fa0 0x7fcef406a430 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.974+0000 7fcefa84b640 1 --2- 172.21.15.84:0/1013824312 >> v2:172.21.15.157:3300/0 conn(0x7fcef4065ab0 0x7fcef4161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.974+0000 7fcefa84b640 1 -- 172.21.15.84:0/1013824312 >> 172.21.15.84:0/1013824312 conn(0x7fcef406bc10 msgr2=0x7fcef406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:50.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.974+0000 7fcefa84b640 1 -- 172.21.15.84:0/1013824312 shutdown_connections 2023-10-16T10:08:50.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.974+0000 7fcefa84b640 1 -- 172.21.15.84:0/1013824312 wait complete. 2023-10-16T10:08:50.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.976+0000 7fcefa84b640 1 Processor -- start 2023-10-16T10:08:50.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.976+0000 7fcefa84b640 1 -- start start 2023-10-16T10:08:50.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.977+0000 7fcefa84b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcef4065ab0 0x7fcef414f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.977+0000 7fcefa84b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef4069fa0 0x7fcef40c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.977+0000 7fcefa84b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcef4160530 0x7fcef40c3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.977+0000 7fcefa84b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef40b4810 con 0x7fcef4069fa0 2023-10-16T10:08:50.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.977+0000 7fcefa84b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef40b2160 con 0x7fcef4160530 2023-10-16T10:08:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.977+0000 7fcefa84b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef40b4a60 con 0x7fcef4065ab0 2023-10-16T10:08:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.977+0000 7fceebfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef4069fa0 0x7fcef40c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.977+0000 7fceebfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef4069fa0 0x7fcef40c2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53242/0 (socket says 172.21.15.84:53242) 2023-10-16T10:08:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.977+0000 7fceebfff640 1 -- 172.21.15.84:0/1301932045 learned_addr learned my addr 172.21.15.84:0/1301932045 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.978+0000 7fcef3fff640 1 --2- 172.21.15.84:0/1301932045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcef4065ab0 0x7fcef414f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:50.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.978+0000 7fcef8de8640 1 --2- 172.21.15.84:0/1301932045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcef4160530 0x7fcef40c3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:50.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.978+0000 7fceebfff640 1 -- 172.21.15.84:0/1301932045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcef4065ab0 msgr2=0x7fcef414f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:50.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.978+0000 7fceebfff640 1 --2- 172.21.15.84:0/1301932045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcef4065ab0 0x7fcef414f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.978+0000 7fceebfff640 1 -- 172.21.15.84:0/1301932045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcef4160530 msgr2=0x7fcef40c3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:50.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.978+0000 7fceebfff640 1 --2- 172.21.15.84:0/1301932045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcef4160530 0x7fcef40c3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:50.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.978+0000 7fceebfff640 1 -- 172.21.15.84:0/1301932045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec009b30 con 0x7fcef4069fa0 2023-10-16T10:08:50.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.978+0000 7fceebfff640 1 --2- 172.21.15.84:0/1301932045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef4069fa0 0x7fcef40c2fc0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00b440 tx=0x7fcedc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:50.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.978+0000 7fcef17fa640 1 -- 172.21.15.84:0/1301932045 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc011890 con 0x7fcef4069fa0 2023-10-16T10:08:50.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.978+0000 7fcef17fa640 1 -- 172.21.15.84:0/1301932045 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcedc0119f0 con 0x7fcef4069fa0 2023-10-16T10:08:50.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.978+0000 7fcefa84b640 1 -- 172.21.15.84:0/1301932045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef40b4d40 con 0x7fcef4069fa0 2023-10-16T10:08:50.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.978+0000 7fcef17fa640 1 -- 172.21.15.84:0/1301932045 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc011da0 con 0x7fcef4069fa0 2023-10-16T10:08:50.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.978+0000 7fcefa84b640 1 -- 172.21.15.84:0/1301932045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef40b2690 con 0x7fcef4069fa0 2023-10-16T10:08:50.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.980+0000 7fcefa84b640 1 -- 172.21.15.84:0/1301932045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef40b4f30 con 0x7fcef4069fa0 2023-10-16T10:08:50.987 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:50.979+0000 7f5f914da040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:08:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.980+0000 7fcef17fa640 1 -- 172.21.15.84:0/1301932045 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 77) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fcedc01a400 con 0x7fcef4069fa0 2023-10-16T10:08:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.981+0000 7fcef17fa640 1 --2- 172.21.15.84:0/1301932045 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fced406b020 0x7fced406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.981+0000 7fcef3fff640 1 -- 172.21.15.84:0/1301932045 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fced406b020 msgr2=0x7fced406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1203729023 2023-10-16T10:08:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.981+0000 7fcef17fa640 1 -- 172.21.15.84:0/1301932045 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7fcedc089f20 con 0x7fcef4069fa0 2023-10-16T10:08:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.981+0000 7fcef3fff640 1 --2- 172.21.15.84:0/1301932045 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fced406b020 0x7fced406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:50.985+0000 7fcef17fa640 1 -- 172.21.15.84:0/1301932045 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcedc054b30 con 0x7fcef4069fa0 2023-10-16T10:08:51.067 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:51.065+0000 7f099b258040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:08:51.140 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:51.138+0000 7f5f914da040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:08:51.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.169+0000 7fcefa84b640 1 -- 172.21.15.84:0/1301932045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcef404e160 con 0x7fcef4069fa0 2023-10-16T10:08:51.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.170+0000 7fcef17fa640 1 -- 172.21.15.84:0/1301932045 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v139) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcedc015020 con 0x7fcef4069fa0 2023-10-16T10:08:51.172 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T10:08:51.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.173+0000 7fcefa84b640 1 -- 172.21.15.84:0/1301932045 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fced406b020 msgr2=0x7fced406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:51.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.173+0000 7fcefa84b640 1 --2- 172.21.15.84:0/1301932045 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fced406b020 0x7fced406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:51.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.173+0000 7fcefa84b640 1 -- 172.21.15.84:0/1301932045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef4069fa0 msgr2=0x7fcef40c2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:51.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.173+0000 7fcefa84b640 1 --2- 172.21.15.84:0/1301932045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef4069fa0 0x7fcef40c2fc0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00b440 tx=0x7fcedc00b900 comp rx=0 tx=0).stop 2023-10-16T10:08:51.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.173+0000 7fcefa84b640 1 -- 172.21.15.84:0/1301932045 shutdown_connections 2023-10-16T10:08:51.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.173+0000 7fcefa84b640 1 --2- 172.21.15.84:0/1301932045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcef4160530 0x7fcef40c3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:51.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.173+0000 7fcefa84b640 1 --2- 172.21.15.84:0/1301932045 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fced406b020 0x7fced406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:51.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.173+0000 7fcefa84b640 1 --2- 172.21.15.84:0/1301932045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef4069fa0 0x7fcef40c2fc0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:51.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.173+0000 7fcefa84b640 1 --2- 172.21.15.84:0/1301932045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcef4065ab0 0x7fcef414f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:51.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.173+0000 7fcefa84b640 1 -- 172.21.15.84:0/1301932045 >> 172.21.15.84:0/1301932045 conn(0x7fcef406bc10 msgr2=0x7fcef415f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:51.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.173+0000 7fcefa84b640 1 -- 172.21.15.84:0/1301932045 shutdown_connections 2023-10-16T10:08:51.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.173+0000 7fcefa84b640 1 -- 172.21.15.84:0/1301932045 wait complete. 2023-10-16T10:08:51.193 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T10:08:51.233 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:51.232+0000 7f099b258040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:08:51.248 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:51.246+0000 7f5f914da040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:08:51.298 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:51.296+0000 7f5f914da040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:08:51.346 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:51.344+0000 7f099b258040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:08:51.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.384+0000 7efdc759e640 1 Processor -- start 2023-10-16T10:08:51.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.384+0000 7efdc759e640 1 -- start start 2023-10-16T10:08:51.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.386+0000 7efdc759e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdc80c9060 0x7efdc806f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:51.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.386+0000 7efdc759e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efdc806c9e0 0x7efdc806f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:51.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.386+0000 7efdc759e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdc80ca560 con 0x7efdc80c9440 2023-10-16T10:08:51.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.386+0000 7efdc759e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc80c9950 con 0x7efdc806c9e0 2023-10-16T10:08:51.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.386+0000 7efdc759e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc80c9c30 con 0x7efdc80c9060 2023-10-16T10:08:51.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.386+0000 7efdbffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efdc806c9e0 0x7efdc806f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:51.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.386+0000 7efdbffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efdc806c9e0 0x7efdc806f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53246/0 (socket says 172.21.15.84:53246) 2023-10-16T10:08:51.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.387+0000 7efdbffff640 1 -- 172.21.15.84:0/1942101619 learned_addr learned my addr 172.21.15.84:0/1942101619 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:51.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.387+0000 7efdc659c640 1 --2- 172.21.15.84:0/1942101619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdc80c9060 0x7efdc806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:51.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.387+0000 7efdbffff640 1 -- 172.21.15.84:0/1942101619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdc80c9060 msgr2=0x7efdc806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:51.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.387+0000 7efdbffff640 1 --2- 172.21.15.84:0/1942101619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdc80c9060 0x7efdc806f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:51.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.387+0000 7efdbffff640 1 -- 172.21.15.84:0/1942101619 >> v1:172.21.15.157:6789/0 conn(0x7efdc80c9440 legacy=0x7efdc8164d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:51.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.387+0000 7efdbffff640 1 -- 172.21.15.84:0/1942101619 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc80c9f10 con 0x7efdc806c9e0 2023-10-16T10:08:51.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.388+0000 7efdbffff640 1 --2- 172.21.15.84:0/1942101619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdc806c9e0 0x7efdc806f980 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7efdb0009870 tx=0x7efdb002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e712897347fff4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:51.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.389+0000 7efdc5d9b640 1 -- 172.21.15.84:0/1942101619 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb0030840 con 0x7efdc806c9e0 2023-10-16T10:08:51.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.389+0000 7efdc5d9b640 1 -- 172.21.15.84:0/1942101619 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb00309a0 con 0x7efdc806c9e0 2023-10-16T10:08:51.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.389+0000 7efdc5d9b640 1 -- 172.21.15.84:0/1942101619 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb0030d30 con 0x7efdc806c9e0 2023-10-16T10:08:51.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.389+0000 7efdc759e640 1 -- 172.21.15.84:0/1942101619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdc806c9e0 msgr2=0x7efdc806f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:51.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.389+0000 7efdc759e640 1 --2- 172.21.15.84:0/1942101619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdc806c9e0 0x7efdc806f980 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7efdb0009870 tx=0x7efdb002b980 comp rx=0 tx=0).stop 2023-10-16T10:08:51.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.390+0000 7efdc759e640 1 -- 172.21.15.84:0/1942101619 shutdown_connections 2023-10-16T10:08:51.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.390+0000 7efdc759e640 1 --2- 172.21.15.84:0/1942101619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdc806c9e0 0x7efdc806f980 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:51.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.390+0000 7efdc759e640 1 --2- 172.21.15.84:0/1942101619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdc80c9060 0x7efdc806f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:51.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.390+0000 7efdc759e640 1 -- 172.21.15.84:0/1942101619 >> 172.21.15.84:0/1942101619 conn(0x7efdc80ce6a0 msgr2=0x7efdc80ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:51.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.390+0000 7efdc759e640 1 -- 172.21.15.84:0/1942101619 shutdown_connections 2023-10-16T10:08:51.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.390+0000 7efdc759e640 1 -- 172.21.15.84:0/1942101619 wait complete. 2023-10-16T10:08:51.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.391+0000 7efdc759e640 1 Processor -- start 2023-10-16T10:08:51.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.392+0000 7efdc759e640 1 -- start start 2023-10-16T10:08:51.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.393+0000 7efdc759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdc806c9e0 0x7efdc8164420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:51.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.393+0000 7efdc759e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdc80c9060 0x7efdc815fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:51.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.393+0000 7efdc759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdc80c9440 0x7efdc8161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:51.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.393+0000 7efdc759e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc80c35d0 con 0x7efdc80c9440 2023-10-16T10:08:51.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.393+0000 7efdc759e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc80c3760 con 0x7efdc806c9e0 2023-10-16T10:08:51.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.393+0000 7efdc759e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc8160100 con 0x7efdc80c9060 2023-10-16T10:08:51.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.393+0000 7efdc6d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdc80c9440 0x7efdc8161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:51.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.393+0000 7efdc6d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdc80c9440 0x7efdc8161fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53254/0 (socket says 172.21.15.84:53254) 2023-10-16T10:08:51.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.393+0000 7efdc6d9d640 1 -- 172.21.15.84:0/4255273540 learned_addr learned my addr 172.21.15.84:0/4255273540 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:51.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.393+0000 7efdbffff640 1 --2- 172.21.15.84:0/4255273540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdc80c9060 0x7efdc815fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:51.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.393+0000 7efdc659c640 1 --2- 172.21.15.84:0/4255273540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdc806c9e0 0x7efdc8164420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:51.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.394+0000 7efdc6d9d640 1 -- 172.21.15.84:0/4255273540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdc80c9060 msgr2=0x7efdc815fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:51.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.394+0000 7efdc6d9d640 1 --2- 172.21.15.84:0/4255273540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdc80c9060 0x7efdc815fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:51.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.394+0000 7efdc6d9d640 1 -- 172.21.15.84:0/4255273540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdc806c9e0 msgr2=0x7efdc8164420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:51.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.394+0000 7efdc6d9d640 1 --2- 172.21.15.84:0/4255273540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdc806c9e0 0x7efdc8164420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:51.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.394+0000 7efdc6d9d640 1 -- 172.21.15.84:0/4255273540 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb00094d0 con 0x7efdc80c9440 2023-10-16T10:08:51.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.394+0000 7efdbffff640 1 --2- 172.21.15.84:0/4255273540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdc80c9060 0x7efdc815fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:08:51.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.394+0000 7efdc6d9d640 1 --2- 172.21.15.84:0/4255273540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdc80c9440 0x7efdc8161fa0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7efdc000c6b0 tx=0x7efdc000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:51.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.394+0000 7efdbeffd640 1 -- 172.21.15.84:0/4255273540 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc0003b70 con 0x7efdc80c9440 2023-10-16T10:08:51.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.394+0000 7efdbeffd640 1 -- 172.21.15.84:0/4255273540 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc0003cd0 con 0x7efdc80c9440 2023-10-16T10:08:51.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.394+0000 7efdc759e640 1 -- 172.21.15.84:0/4255273540 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc81624e0 con 0x7efdc80c9440 2023-10-16T10:08:51.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.394+0000 7efdbeffd640 1 -- 172.21.15.84:0/4255273540 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc0005310 con 0x7efdc80c9440 2023-10-16T10:08:51.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.394+0000 7efdc759e640 1 -- 172.21.15.84:0/4255273540 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc8162a20 con 0x7efdc80c9440 2023-10-16T10:08:51.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.396+0000 7efdc759e640 1 -- 172.21.15.84:0/4255273540 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc80c3910 con 0x7efdc80c9440 2023-10-16T10:08:51.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.396+0000 7efdbeffd640 1 -- 172.21.15.84:0/4255273540 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 77) v1 ==== 89388+0+0 (secure 0 0 0) 0x7efdc0005470 con 0x7efdc80c9440 2023-10-16T10:08:51.403 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:51.397+0000 7f099b258040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:08:51.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.397+0000 7efdbeffd640 1 --2- 172.21.15.84:0/4255273540 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7efda806b020 0x7efda806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:51.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.398+0000 7efdbeffd640 1 -- 172.21.15.84:0/4255273540 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7efdc0089850 con 0x7efdc80c9440 2023-10-16T10:08:51.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.398+0000 7efdc659c640 1 -- 172.21.15.84:0/4255273540 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7efda806b020 msgr2=0x7efda806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1203729023 2023-10-16T10:08:51.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.398+0000 7efdc659c640 1 --2- 172.21.15.84:0/4255273540 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7efda806b020 0x7efda806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:51.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.401+0000 7efdbeffd640 1 -- 172.21.15.84:0/4255273540 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efdc80c3910 con 0x7efdc80c9440 2023-10-16T10:08:51.407 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:51.406+0000 7f5f914da040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:08:51.511 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:51.510+0000 7f099b258040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:08:51.514 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:51.512+0000 7f5f914da040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:08:51.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.599+0000 7efdc659c640 1 -- 172.21.15.84:0/4255273540 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7efda806b020 msgr2=0x7efda806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1203729023 2023-10-16T10:08:51.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.599+0000 7efdc659c640 1 --2- 172.21.15.84:0/4255273540 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7efda806b020 0x7efda806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:08:51.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.606+0000 7efdc759e640 1 -- 172.21.15.84:0/4255273540 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efdc80c3910 con 0x7efdc80c9440 2023-10-16T10:08:51.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.607+0000 7efdbeffd640 1 -- 172.21.15.84:0/4255273540 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v139) v1 ==== 80+0+469 (secure 0 0 0) 0x7efdc80c3910 con 0x7efdc80c9440 2023-10-16T10:08:51.609 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2708741108 2023-10-17T10:08:40.209237+0000 2023-10-16T10:08:51.610 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/914481139 2023-10-17T10:08:40.209237+0000 2023-10-16T10:08:51.610 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/4141270981 2023-10-17T10:08:40.209237+0000 2023-10-16T10:08:51.610 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/4132563801 2023-10-17T10:08:40.209237+0000 2023-10-16T10:08:51.610 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6833/2784642648 2023-10-17T10:08:40.209237+0000 2023-10-16T10:08:51.610 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6832/2784642648 2023-10-17T10:08:40.209237+0000 2023-10-16T10:08:51.611 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2180503356 2023-10-17T10:08:40.209237+0000 2023-10-16T10:08:51.611 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2933995314 2023-10-17T10:08:40.209237+0000 2023-10-16T10:08:51.612 INFO:teuthology.orchestra.run.smithi084.stderr:listed 8 entries 2023-10-16T10:08:51.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.610+0000 7efdc759e640 1 -- 172.21.15.84:0/4255273540 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7efda806b020 msgr2=0x7efda806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:51.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.610+0000 7efdc759e640 1 --2- 172.21.15.84:0/4255273540 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7efda806b020 0x7efda806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:51.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.610+0000 7efdc759e640 1 -- 172.21.15.84:0/4255273540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdc80c9440 msgr2=0x7efdc8161fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:51.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.610+0000 7efdc759e640 1 --2- 172.21.15.84:0/4255273540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdc80c9440 0x7efdc8161fa0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7efdc000c6b0 tx=0x7efdc000cb70 comp rx=0 tx=0).stop 2023-10-16T10:08:51.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.610+0000 7efdc759e640 1 -- 172.21.15.84:0/4255273540 shutdown_connections 2023-10-16T10:08:51.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.610+0000 7efdc759e640 1 --2- 172.21.15.84:0/4255273540 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7efda806b020 0x7efda806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:51.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.610+0000 7efdc759e640 1 --2- 172.21.15.84:0/4255273540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdc80c9440 0x7efdc8161fa0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:51.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.610+0000 7efdc759e640 1 --2- 172.21.15.84:0/4255273540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efdc80c9060 0x7efdc815fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:51.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.610+0000 7efdc759e640 1 --2- 172.21.15.84:0/4255273540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdc806c9e0 0x7efdc8164420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:51.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.610+0000 7efdc759e640 1 -- 172.21.15.84:0/4255273540 >> 172.21.15.84:0/4255273540 conn(0x7efdc80ce6a0 msgr2=0x7efdc80b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:51.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.610+0000 7efdc759e640 1 -- 172.21.15.84:0/4255273540 shutdown_connections 2023-10-16T10:08:51.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:51.610+0000 7efdc759e640 1 -- 172.21.15.84:0/4255273540 wait complete. 2023-10-16T10:08:51.624 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:51.622+0000 7f099b258040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:08:51.629 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T10:08:51.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:51.630 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T10:08:51.671 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:51.668+0000 7f5f914da040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:08:51.719 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:51.717+0000 7f5f914da040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:08:51.767 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T10:08:51.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:08:51.768 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T10:08:51.792 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:51.791+0000 7f099b258040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:08:51.815 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:08:51.815 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:08:51.839 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:51.838+0000 7f5f914da040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:08:51.844 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:51.843+0000 7f099b258040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:08:51.870 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:08:51.870 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:08:51.871 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:08:51.923 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:08:51.924 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:08:51.924 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:08:51.968 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:51.966+0000 7f099b258040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:08:51.980 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:08:51.981 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:08:52.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.183+0000 7ff762015640 1 Processor -- start 2023-10-16T10:08:52.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.184+0000 7ff762015640 1 -- start start 2023-10-16T10:08:52.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.185+0000 7ff762015640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff75c159b20 0x7ff75c06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:52.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.185+0000 7ff762015640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff75c159f00 0x7ff75c0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:52.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.185+0000 7ff762015640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c06f7f0 con 0x7ff75c06c120 2023-10-16T10:08:52.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.185+0000 7ff762015640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75c06c5e0 con 0x7ff75c159b20 2023-10-16T10:08:52.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.185+0000 7ff762015640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c06c8c0 con 0x7ff75c159f00 2023-10-16T10:08:52.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.185+0000 7ff75bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff75c06c120 0x7ff75c0cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46908/0 (socket says 172.21.15.84:46908) 2023-10-16T10:08:52.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.185+0000 7ff75bfff640 1 -- 172.21.15.84:0/4142751404 learned_addr learned my addr 172.21.15.84:0/4142751404 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:52.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.186+0000 7ff753fff640 1 --2- 172.21.15.84:0/4142751404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff75c159f00 0x7ff75c0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:52.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.186+0000 7ff75b7fe640 1 --2- 172.21.15.84:0/4142751404 >> v2:172.21.15.157:3300/0 conn(0x7ff75c159b20 0x7ff75c06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:52.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.186+0000 7ff75affd640 1 -- 172.21.15.84:0/4142751404 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3294838501 0 0) 0x7ff75c06f7f0 con 0x7ff75c06c120 2023-10-16T10:08:52.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.186+0000 7ff75affd640 1 -- 172.21.15.84:0/4142751404 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff748003520 con 0x7ff75c06c120 2023-10-16T10:08:52.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.186+0000 7ff75affd640 1 -- 172.21.15.84:0/4142751404 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 761620955 0 0) 0x7ff748003520 con 0x7ff75c06c120 2023-10-16T10:08:52.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.186+0000 7ff75affd640 1 -- 172.21.15.84:0/4142751404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff75c159f00 msgr2=0x7ff75c0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:52.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.186+0000 7ff75affd640 1 --2- 172.21.15.84:0/4142751404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff75c159f00 0x7ff75c0b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.186+0000 7ff75affd640 1 -- 172.21.15.84:0/4142751404 >> v2:172.21.15.157:3300/0 conn(0x7ff75c159b20 msgr2=0x7ff75c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:52.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.186+0000 7ff75affd640 1 --2- 172.21.15.84:0/4142751404 >> v2:172.21.15.157:3300/0 conn(0x7ff75c159b20 0x7ff75c06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.186+0000 7ff75affd640 1 -- 172.21.15.84:0/4142751404 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c06cba0 con 0x7ff75c06c120 2023-10-16T10:08:52.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.186+0000 7ff75affd640 1 -- 172.21.15.84:0/4142751404 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ff7540019f0 con 0x7ff75c06c120 2023-10-16T10:08:52.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.186+0000 7ff75affd640 1 -- 172.21.15.84:0/4142751404 >> v1:172.21.15.84:6789/0 conn(0x7ff75c06c120 legacy=0x7ff75c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:52.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.186+0000 7ff75affd640 1 --2- 172.21.15.84:0/4142751404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff75c159f00 0x7ff748003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:52.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.186+0000 7ff75affd640 1 --2- 172.21.15.84:0/4142751404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff748003ef0 0x7ff7480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:52.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.186+0000 7ff75affd640 1 --2- 172.21.15.84:0/4142751404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff748004d00 0x7ff748005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:52.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.186+0000 7ff75affd640 1 -- 172.21.15.84:0/4142751404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c06c5e0 con 0x7ff748004d00 2023-10-16T10:08:52.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.186+0000 7ff75affd640 1 -- 172.21.15.84:0/4142751404 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c06c8c0 con 0x7ff748003ef0 2023-10-16T10:08:52.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.186+0000 7ff75affd640 1 -- 172.21.15.84:0/4142751404 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74800b980 con 0x7ff75c159f00 2023-10-16T10:08:52.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.187+0000 7ff75bfff640 1 --2- 172.21.15.84:0/4142751404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff748004d00 0x7ff748005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:52.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.187+0000 7ff753fff640 1 --2- 172.21.15.84:0/4142751404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff748003ef0 0x7ff7480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:52.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.187+0000 7ff75bfff640 1 -- 172.21.15.84:0/4142751404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff75c159f00 msgr2=0x7ff748003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:52.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.187+0000 7ff75bfff640 1 --2- 172.21.15.84:0/4142751404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff75c159f00 0x7ff748003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.187+0000 7ff75bfff640 1 -- 172.21.15.84:0/4142751404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff748003ef0 msgr2=0x7ff7480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:52.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.187+0000 7ff75bfff640 1 --2- 172.21.15.84:0/4142751404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff748003ef0 0x7ff7480047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.187+0000 7ff75bfff640 1 -- 172.21.15.84:0/4142751404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff748003520 con 0x7ff748004d00 2023-10-16T10:08:52.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.187+0000 7ff753fff640 1 --2- 172.21.15.84:0/4142751404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff748003ef0 0x7ff7480047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:52.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.189+0000 7ff75bfff640 1 --2- 172.21.15.84:0/4142751404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff748004d00 0x7ff748005180 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0c5650 tx=0x7ff7480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40442e447763742c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:52.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.189+0000 7ff75affd640 1 -- 172.21.15.84:0/4142751404 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754034630 con 0x7ff748004d00 2023-10-16T10:08:52.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.189+0000 7ff75affd640 1 -- 172.21.15.84:0/4142751404 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75402f070 con 0x7ff748004d00 2023-10-16T10:08:52.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.189+0000 7ff75affd640 1 -- 172.21.15.84:0/4142751404 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754034a30 con 0x7ff748004d00 2023-10-16T10:08:52.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.190+0000 7ff762015640 1 -- 172.21.15.84:0/4142751404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff748004d00 msgr2=0x7ff748005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:52.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.190+0000 7ff762015640 1 --2- 172.21.15.84:0/4142751404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff748004d00 0x7ff748005180 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0c5650 tx=0x7ff7480057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.190+0000 7ff75bfff640 1 -- 172.21.15.84:0/4142751404 reap_dead start 2023-10-16T10:08:52.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.190+0000 7ff762015640 1 -- 172.21.15.84:0/4142751404 shutdown_connections 2023-10-16T10:08:52.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.190+0000 7ff762015640 1 -- 172.21.15.84:0/4142751404 >> 172.21.15.84:0/4142751404 conn(0x7ff75c0cb410 msgr2=0x7ff75c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:52.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.190+0000 7ff762015640 1 -- 172.21.15.84:0/4142751404 shutdown_connections 2023-10-16T10:08:52.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.190+0000 7ff762015640 1 -- 172.21.15.84:0/4142751404 wait complete. 2023-10-16T10:08:52.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.192+0000 7ff762015640 1 Processor -- start 2023-10-16T10:08:52.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.192+0000 7ff762015640 1 -- start start 2023-10-16T10:08:52.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.193+0000 7ff762015640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff75c155e30 0x7ff75c15cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:52.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.193+0000 7ff762015640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff75c156210 0x7ff75c15d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:52.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.193+0000 7ff762015640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff75c15c520 0x7ff75c0c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.193+0000 7ff762015640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c15b4f0 con 0x7ff75c155e30 2023-10-16T10:08:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.193+0000 7ff762015640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c15b760 con 0x7ff75c156210 2023-10-16T10:08:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.193+0000 7ff762015640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c15ba40 con 0x7ff75c15c520 2023-10-16T10:08:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.193+0000 7ff75b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff75c155e30 0x7ff75c15cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.193+0000 7ff75b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff75c155e30 0x7ff75c15cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53272/0 (socket says 172.21.15.84:53272) 2023-10-16T10:08:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.193+0000 7ff75b7fe640 1 -- 172.21.15.84:0/2847740270 learned_addr learned my addr 172.21.15.84:0/2847740270 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.193+0000 7ff75b7fe640 1 -- 172.21.15.84:0/2847740270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff75c15c520 msgr2=0x7ff75c0c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.193+0000 7ff75b7fe640 1 --2- 172.21.15.84:0/2847740270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff75c15c520 0x7ff75c0c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.193+0000 7ff75b7fe640 1 -- 172.21.15.84:0/2847740270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff75c156210 msgr2=0x7ff75c15d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.193+0000 7ff75b7fe640 1 --2- 172.21.15.84:0/2847740270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff75c156210 0x7ff75c15d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.193+0000 7ff75b7fe640 1 -- 172.21.15.84:0/2847740270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c15bd20 con 0x7ff75c155e30 2023-10-16T10:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.194+0000 7ff75b7fe640 1 --2- 172.21.15.84:0/2847740270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff75c155e30 0x7ff75c15cbc0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00e7c0 tx=0x7ff74c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.194+0000 7ff758ff9640 1 -- 172.21.15.84:0/2847740270 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c00eed0 con 0x7ff75c155e30 2023-10-16T10:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.194+0000 7ff758ff9640 1 -- 172.21.15.84:0/2847740270 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74c00c8c0 con 0x7ff75c155e30 2023-10-16T10:08:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.194+0000 7ff758ff9640 1 -- 172.21.15.84:0/2847740270 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c00cc70 con 0x7ff75c155e30 2023-10-16T10:08:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.194+0000 7ff762015640 1 -- 172.21.15.84:0/2847740270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c15c000 con 0x7ff75c155e30 2023-10-16T10:08:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.194+0000 7ff762015640 1 -- 172.21.15.84:0/2847740270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c0c4cc0 con 0x7ff75c155e30 2023-10-16T10:08:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.196+0000 7ff762015640 1 -- 172.21.15.84:0/2847740270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75c04e160 con 0x7ff75c155e30 2023-10-16T10:08:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.196+0000 7ff758ff9640 1 -- 172.21.15.84:0/2847740270 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 77) v1 ==== 89388+0+0 (secure 0 0 0) 0x7ff74c013470 con 0x7ff75c155e30 2023-10-16T10:08:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.197+0000 7ff758ff9640 1 --2- 172.21.15.84:0/2847740270 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7ff73c06b050 0x7ff73c06d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.198+0000 7ff753fff640 1 -- 172.21.15.84:0/2847740270 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7ff73c06b050 msgr2=0x7ff73c06d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1203729023 2023-10-16T10:08:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.198+0000 7ff758ff9640 1 -- 172.21.15.84:0/2847740270 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7ff74c089ae0 con 0x7ff75c155e30 2023-10-16T10:08:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.198+0000 7ff753fff640 1 --2- 172.21.15.84:0/2847740270 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7ff73c06b050 0x7ff73c06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.202+0000 7ff758ff9640 1 -- 172.21.15.84:0/2847740270 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff74c054790 con 0x7ff75c155e30 2023-10-16T10:08:52.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.399+0000 7ff753fff640 1 -- 172.21.15.84:0/2847740270 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7ff73c06b050 msgr2=0x7ff73c06d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1203729023 2023-10-16T10:08:52.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.399+0000 7ff753fff640 1 --2- 172.21.15.84:0/2847740270 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7ff73c06b050 0x7ff73c06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:08:52.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.446+0000 7ff762015640 1 -- 172.21.15.84:0/2847740270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff75c0b10b0 con 0x7ff75c155e30 2023-10-16T10:08:52.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.447+0000 7ff758ff9640 1 -- 172.21.15.84:0/2847740270 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4918 (secure 0 0 0) 0x7ff74c017090 con 0x7ff75c155e30 2023-10-16T10:08:52.449 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:52.449 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4118285132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":4118285132},{"type":"v1","addr":"172.21.15.84:6837","nonce":4118285132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3743702675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3743702675},{"type":"v1","addr":"172.21.15.157:6834","nonce":3743702675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2895418246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2895418246},{"type":"v1","addr":"172.21.15.157:6835","nonce":2895418246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:08:25.705299+0000","modified":"2023-10-16T10:08:28.762900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8574":{"gid":8574,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/791907708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":791907708},{"type":"v1","addr":"172.21.15.84:6836","nonce":791907708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-16T10:08:52.449 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 71 2023-10-16T10:08:52.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.449+0000 7ff762015640 1 -- 172.21.15.84:0/2847740270 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7ff73c06b050 msgr2=0x7ff73c06d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:52.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.449+0000 7ff762015640 1 --2- 172.21.15.84:0/2847740270 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7ff73c06b050 0x7ff73c06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.449+0000 7ff762015640 1 -- 172.21.15.84:0/2847740270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff75c155e30 msgr2=0x7ff75c15cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:52.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.449+0000 7ff762015640 1 --2- 172.21.15.84:0/2847740270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff75c155e30 0x7ff75c15cbc0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00e7c0 tx=0x7ff74c00ec80 comp rx=0 tx=0).stop 2023-10-16T10:08:52.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.449+0000 7ff762015640 1 -- 172.21.15.84:0/2847740270 shutdown_connections 2023-10-16T10:08:52.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.449+0000 7ff762015640 1 --2- 172.21.15.84:0/2847740270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff75c15c520 0x7ff75c0c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.449+0000 7ff762015640 1 --2- 172.21.15.84:0/2847740270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff75c156210 0x7ff75c15d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.449+0000 7ff762015640 1 --2- 172.21.15.84:0/2847740270 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7ff73c06b050 0x7ff73c06d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.449+0000 7ff762015640 1 --2- 172.21.15.84:0/2847740270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff75c155e30 0x7ff75c15cbc0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.449+0000 7ff762015640 1 -- 172.21.15.84:0/2847740270 >> 172.21.15.84:0/2847740270 conn(0x7ff75c0cb410 msgr2=0x7ff75c0cf130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:52.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.450+0000 7ff762015640 1 -- 172.21.15.84:0/2847740270 shutdown_connections 2023-10-16T10:08:52.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.450+0000 7ff762015640 1 -- 172.21.15.84:0/2847740270 wait complete. 2023-10-16T10:08:52.469 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:08:52.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.655+0000 7fd4b73ee640 1 Processor -- start 2023-10-16T10:08:52.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.655+0000 7fd4b73ee640 1 -- start start 2023-10-16T10:08:52.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.656+0000 7fd4b73ee640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd4b01593b0 0x7fd4b006f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:52.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.656+0000 7fd4b73ee640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd4b00caa10 0x7fd4b006fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:52.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.656+0000 7fd4b73ee640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4b006ffd0 0x7fd4b015ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:52.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.656+0000 7fd4b73ee640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b0159790 con 0x7fd4b00caa10 2023-10-16T10:08:52.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.656+0000 7fd4b73ee640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b00c9b20 con 0x7fd4b01593b0 2023-10-16T10:08:52.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.656+0000 7fd4b73ee640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b00c9e00 con 0x7fd4b006ffd0 2023-10-16T10:08:52.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.657+0000 7fd4b4989640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd4b00caa10 0x7fd4b006fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:52.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.657+0000 7fd4b4989640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd4b00caa10 0x7fd4b006fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53280/0 (socket says 172.21.15.84:53280) 2023-10-16T10:08:52.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.657+0000 7fd4b4989640 1 -- 172.21.15.84:0/4135350404 learned_addr learned my addr 172.21.15.84:0/4135350404 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:52.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.657+0000 7fd4b4989640 1 -- 172.21.15.84:0/4135350404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4b006ffd0 msgr2=0x7fd4b015ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:52.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.657+0000 7fd4b4989640 1 --2- 172.21.15.84:0/4135350404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4b006ffd0 0x7fd4b015ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.657+0000 7fd4b4989640 1 -- 172.21.15.84:0/4135350404 >> v2:172.21.15.157:3300/0 conn(0x7fd4b01593b0 msgr2=0x7fd4b006f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:52.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.657+0000 7fd4b4989640 1 --2- 172.21.15.84:0/4135350404 >> v2:172.21.15.157:3300/0 conn(0x7fd4b01593b0 0x7fd4b006f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.657+0000 7fd4b4989640 1 -- 172.21.15.84:0/4135350404 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b00ca0e0 con 0x7fd4b00caa10 2023-10-16T10:08:52.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.658+0000 7fd4b4989640 1 --2- 172.21.15.84:0/4135350404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4b00caa10 0x7fd4b006fa90 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fd4b00ce8f0 tx=0x7fd4a002dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee88ece504d4423 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:52.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.659+0000 7fd4a7fff640 1 -- 172.21.15.84:0/4135350404 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a002eb30 con 0x7fd4b00caa10 2023-10-16T10:08:52.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.659+0000 7fd4a7fff640 1 -- 172.21.15.84:0/4135350404 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a0032070 con 0x7fd4b00caa10 2023-10-16T10:08:52.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.659+0000 7fd4a7fff640 1 -- 172.21.15.84:0/4135350404 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0038400 con 0x7fd4b00caa10 2023-10-16T10:08:52.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.659+0000 7fd4b73ee640 1 -- 172.21.15.84:0/4135350404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4b00caa10 msgr2=0x7fd4b006fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:52.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.659+0000 7fd4b73ee640 1 --2- 172.21.15.84:0/4135350404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4b00caa10 0x7fd4b006fa90 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fd4b00ce8f0 tx=0x7fd4a002dd80 comp rx=0 tx=0).stop 2023-10-16T10:08:52.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.659+0000 7fd4b73ee640 1 -- 172.21.15.84:0/4135350404 shutdown_connections 2023-10-16T10:08:52.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.659+0000 7fd4b73ee640 1 --2- 172.21.15.84:0/4135350404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4b00caa10 0x7fd4b006fa90 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.659+0000 7fd4b73ee640 1 --2- 172.21.15.84:0/4135350404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4b006ffd0 0x7fd4b015ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.659+0000 7fd4b73ee640 1 --2- 172.21.15.84:0/4135350404 >> v2:172.21.15.157:3300/0 conn(0x7fd4b01593b0 0x7fd4b006f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.659+0000 7fd4b73ee640 1 -- 172.21.15.84:0/4135350404 >> 172.21.15.84:0/4135350404 conn(0x7fd4b00cb410 msgr2=0x7fd4b00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:52.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.659+0000 7fd4b73ee640 1 -- 172.21.15.84:0/4135350404 shutdown_connections 2023-10-16T10:08:52.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.659+0000 7fd4b73ee640 1 -- 172.21.15.84:0/4135350404 wait complete. 2023-10-16T10:08:52.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.661+0000 7fd4b73ee640 1 Processor -- start 2023-10-16T10:08:52.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.661+0000 7fd4b73ee640 1 -- start start 2023-10-16T10:08:52.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.662+0000 7fd4b73ee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4b006ffd0 0x7fd4b0155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:52.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.662+0000 7fd4b73ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4b00caa10 0x7fd4b0157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:52.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.662+0000 7fd4b73ee640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4b01593b0 0x7fd4b014c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:52.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.662+0000 7fd4b73ee640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b00c72d0 con 0x7fd4b00caa10 2023-10-16T10:08:52.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.662+0000 7fd4b73ee640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b0155290 con 0x7fd4b006ffd0 2023-10-16T10:08:52.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.662+0000 7fd4b73ee640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b0155570 con 0x7fd4b01593b0 2023-10-16T10:08:52.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.662+0000 7fd4b4989640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4b00caa10 0x7fd4b0157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:52.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.662+0000 7fd4b4989640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4b00caa10 0x7fd4b0157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53292/0 (socket says 172.21.15.84:53292) 2023-10-16T10:08:52.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.662+0000 7fd4b4989640 1 -- 172.21.15.84:0/2966447014 learned_addr learned my addr 172.21.15.84:0/2966447014 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:52.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.663+0000 7fd4b4989640 1 -- 172.21.15.84:0/2966447014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4b01593b0 msgr2=0x7fd4b014c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:52.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.663+0000 7fd4b4989640 1 --2- 172.21.15.84:0/2966447014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4b01593b0 0x7fd4b014c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.663+0000 7fd4b4989640 1 -- 172.21.15.84:0/2966447014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4b006ffd0 msgr2=0x7fd4b0155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:52.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.663+0000 7fd4b4989640 1 --2- 172.21.15.84:0/2966447014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4b006ffd0 0x7fd4b0155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.663+0000 7fd4b4989640 1 -- 172.21.15.84:0/2966447014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a000b760 con 0x7fd4b00caa10 2023-10-16T10:08:52.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.663+0000 7fd4b4989640 1 --2- 172.21.15.84:0/2966447014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4b00caa10 0x7fd4b0157ac0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0000c00 tx=0x7fd4a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:52.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.663+0000 7fd4a5ffb640 1 -- 172.21.15.84:0/2966447014 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0043470 con 0x7fd4b00caa10 2023-10-16T10:08:52.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.663+0000 7fd4a5ffb640 1 -- 172.21.15.84:0/2966447014 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a00435f0 con 0x7fd4b00caa10 2023-10-16T10:08:52.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.663+0000 7fd4a5ffb640 1 -- 172.21.15.84:0/2966447014 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a00439a0 con 0x7fd4b00caa10 2023-10-16T10:08:52.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.663+0000 7fd4b73ee640 1 -- 172.21.15.84:0/2966447014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b00c6ef0 con 0x7fd4b00caa10 2023-10-16T10:08:52.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.663+0000 7fd4b73ee640 1 -- 172.21.15.84:0/2966447014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b01563f0 con 0x7fd4b00caa10 2023-10-16T10:08:52.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.664+0000 7fd4b73ee640 1 -- 172.21.15.84:0/2966447014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b00c6d30 con 0x7fd4b00caa10 2023-10-16T10:08:52.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.665+0000 7fd4a5ffb640 1 -- 172.21.15.84:0/2966447014 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 77) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fd4a0043b00 con 0x7fd4b00caa10 2023-10-16T10:08:52.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.666+0000 7fd4a5ffb640 1 --2- 172.21.15.84:0/2966447014 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fd49406b020 0x7fd49406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:52.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.666+0000 7fd4b518a640 1 -- 172.21.15.84:0/2966447014 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fd49406b020 msgr2=0x7fd49406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1203729023 2023-10-16T10:08:52.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.666+0000 7fd4a5ffb640 1 -- 172.21.15.84:0/2966447014 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7fd4a00ae7a0 con 0x7fd4b00caa10 2023-10-16T10:08:52.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.666+0000 7fd4b518a640 1 --2- 172.21.15.84:0/2966447014 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fd49406b020 0x7fd49406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:52.673 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:52.672+0000 7f5f914da040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:08:52.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.675+0000 7fd4a5ffb640 1 -- 172.21.15.84:0/2966447014 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd4a00793b0 con 0x7fd4b00caa10 2023-10-16T10:08:52.842 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:52.840+0000 7f099b258040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:08:52.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.867+0000 7fd4b518a640 1 -- 172.21.15.84:0/2966447014 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fd49406b020 msgr2=0x7fd49406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1203729023 2023-10-16T10:08:52.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.867+0000 7fd4b518a640 1 --2- 172.21.15.84:0/2966447014 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fd49406b020 0x7fd49406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:08:52.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.868+0000 7fd4b73ee640 1 -- 172.21.15.84:0/2966447014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4b01563f0 con 0x7fd4b00caa10 2023-10-16T10:08:52.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.870+0000 7fd4a5ffb640 1 -- 172.21.15.84:0/2966447014 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+17047 (secure 0 0 0) 0x7fd4b01563f0 con 0x7fd4b00caa10 2023-10-16T10:08:52.872 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:52.872 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":139,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:08:40.209311+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:08:23.558506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-16T10:08:24.709464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2708741108":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/914481139":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/4141270981":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/4132563801":"2023-10-17T10:08:40.209237+0000","172.21.15.84:6833/2784642648":"2023-10-17T10:08:40.209237+0000","172.21.15.84:6832/2784642648":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/2180503356":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/2933995314":"2023-10-17T10:08:40.209237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:08:52.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.872+0000 7fd4b73ee640 1 -- 172.21.15.84:0/2966447014 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fd49406b020 msgr2=0x7fd49406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:52.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.873+0000 7fd4b73ee640 1 --2- 172.21.15.84:0/2966447014 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fd49406b020 0x7fd49406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.873+0000 7fd4b73ee640 1 -- 172.21.15.84:0/2966447014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4b00caa10 msgr2=0x7fd4b0157ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:52.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.873+0000 7fd4b73ee640 1 --2- 172.21.15.84:0/2966447014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4b00caa10 0x7fd4b0157ac0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0000c00 tx=0x7fd4a00046c0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.873+0000 7fd4b73ee640 1 -- 172.21.15.84:0/2966447014 shutdown_connections 2023-10-16T10:08:52.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.873+0000 7fd4b73ee640 1 --2- 172.21.15.84:0/2966447014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4b01593b0 0x7fd4b014c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.873+0000 7fd4b73ee640 1 --2- 172.21.15.84:0/2966447014 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fd49406b020 0x7fd49406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.873+0000 7fd4b73ee640 1 --2- 172.21.15.84:0/2966447014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4b00caa10 0x7fd4b0157ac0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.873+0000 7fd4b73ee640 1 --2- 172.21.15.84:0/2966447014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4b006ffd0 0x7fd4b0155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:52.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.873+0000 7fd4b73ee640 1 -- 172.21.15.84:0/2966447014 >> 172.21.15.84:0/2966447014 conn(0x7fd4b00cb410 msgr2=0x7fd4b00c5050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:52.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.873+0000 7fd4b73ee640 1 -- 172.21.15.84:0/2966447014 shutdown_connections 2023-10-16T10:08:52.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:52.873+0000 7fd4b73ee640 1 -- 172.21.15.84:0/2966447014 wait complete. 2023-10-16T10:08:52.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:08:23.558506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:08:24.709464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:08:52.893 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:08:52.968 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:52.966+0000 7f5f914da040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:08:53.026 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:53.024+0000 7f5f914da040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:08:53.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.094+0000 7fd0a1f6c640 1 Processor -- start 2023-10-16T10:08:53.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.094+0000 7fd0a1f6c640 1 -- start start 2023-10-16T10:08:53.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.096+0000 7fd0a1f6c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd09c15d330 0x7fd09c1618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:53.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.096+0000 7fd0a1f6c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd09c0661c0 0x7fd09c0ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:53.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.096+0000 7fd0a1f6c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd09c0d0890 con 0x7fd09c06c7d0 2023-10-16T10:08:53.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.096+0000 7fd0a1f6c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd09c06b1e0 con 0x7fd09c0661c0 2023-10-16T10:08:53.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.096+0000 7fd0a1f6c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c0d05b0 con 0x7fd09c15d330 2023-10-16T10:08:53.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.096+0000 7fd093fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd09c06c7d0 0x7fd09c065ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46916/0 (socket says 172.21.15.84:46916) 2023-10-16T10:08:53.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.096+0000 7fd093fff640 1 -- 172.21.15.84:0/443618198 learned_addr learned my addr 172.21.15.84:0/443618198 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:53.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.097+0000 7fd09bfff640 1 --2- 172.21.15.84:0/443618198 >> v2:172.21.15.157:3300/0 conn(0x7fd09c0661c0 0x7fd09c0ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:53.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.097+0000 7fd09b7fe640 1 --2- 172.21.15.84:0/443618198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd09c15d330 0x7fd09c1618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:53.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.097+0000 7fd09affd640 1 -- 172.21.15.84:0/443618198 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3207192595 0 0) 0x7fd09c0d0890 con 0x7fd09c06c7d0 2023-10-16T10:08:53.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.097+0000 7fd09affd640 1 -- 172.21.15.84:0/443618198 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd088003520 con 0x7fd09c06c7d0 2023-10-16T10:08:53.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.097+0000 7fd09affd640 1 -- 172.21.15.84:0/443618198 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2389954390 0 0) 0x7fd088003520 con 0x7fd09c06c7d0 2023-10-16T10:08:53.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.097+0000 7fd09affd640 1 -- 172.21.15.84:0/443618198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd09c15d330 msgr2=0x7fd09c1618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:53.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.097+0000 7fd09affd640 1 --2- 172.21.15.84:0/443618198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd09c15d330 0x7fd09c1618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.097+0000 7fd09affd640 1 -- 172.21.15.84:0/443618198 >> v2:172.21.15.157:3300/0 conn(0x7fd09c0661c0 msgr2=0x7fd09c0ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:53.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.097+0000 7fd09affd640 1 --2- 172.21.15.84:0/443618198 >> v2:172.21.15.157:3300/0 conn(0x7fd09c0661c0 0x7fd09c0ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.097+0000 7fd09affd640 1 -- 172.21.15.84:0/443618198 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09c160dd0 con 0x7fd09c06c7d0 2023-10-16T10:08:53.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.097+0000 7fd09affd640 1 -- 172.21.15.84:0/443618198 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd084001670 con 0x7fd09c06c7d0 2023-10-16T10:08:53.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.097+0000 7fd09affd640 1 -- 172.21.15.84:0/443618198 >> v1:172.21.15.84:6789/0 conn(0x7fd09c06c7d0 legacy=0x7fd09c065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:53.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.098+0000 7fd09affd640 1 --2- 172.21.15.84:0/443618198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd088003970 0x7fd088003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:53.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.098+0000 7fd09affd640 1 --2- 172.21.15.84:0/443618198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0880047c0 0x7fd088004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:53.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.098+0000 7fd09affd640 1 --2- 172.21.15.84:0/443618198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd09c15d330 0x7fd088005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:53.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.098+0000 7fd09affd640 1 -- 172.21.15.84:0/443618198 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c06b1e0 con 0x7fd088003970 2023-10-16T10:08:53.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.098+0000 7fd09affd640 1 -- 172.21.15.84:0/443618198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c0d05b0 con 0x7fd0880047c0 2023-10-16T10:08:53.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.098+0000 7fd09affd640 1 -- 172.21.15.84:0/443618198 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08800b980 con 0x7fd09c15d330 2023-10-16T10:08:53.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.098+0000 7fd09b7fe640 1 --2- 172.21.15.84:0/443618198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd088003970 0x7fd088003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:53.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.098+0000 7fd09bfff640 1 --2- 172.21.15.84:0/443618198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd09c15d330 0x7fd088005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:53.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.098+0000 7fd09b7fe640 1 -- 172.21.15.84:0/443618198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd09c15d330 msgr2=0x7fd088005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:53.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.098+0000 7fd09b7fe640 1 --2- 172.21.15.84:0/443618198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd09c15d330 0x7fd088005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.098+0000 7fd09b7fe640 1 -- 172.21.15.84:0/443618198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0880047c0 msgr2=0x7fd088004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:53.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.098+0000 7fd09b7fe640 1 --2- 172.21.15.84:0/443618198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0880047c0 0x7fd088004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.098+0000 7fd09b7fe640 1 -- 172.21.15.84:0/443618198 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd088003520 con 0x7fd088003970 2023-10-16T10:08:53.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.099+0000 7fd09b7fe640 1 --2- 172.21.15.84:0/443618198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd088003970 0x7fd088003d90 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009af0 tx=0x7fd0880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddc61f267d3c7e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:53.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.100+0000 7fd09affd640 1 -- 172.21.15.84:0/443618198 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c02c6b0 con 0x7fd088003970 2023-10-16T10:08:53.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.100+0000 7fd09affd640 1 -- 172.21.15.84:0/443618198 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c02c810 con 0x7fd088003970 2023-10-16T10:08:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.100+0000 7fd09affd640 1 -- 172.21.15.84:0/443618198 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c02cba0 con 0x7fd088003970 2023-10-16T10:08:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.100+0000 7fd0a1f6c640 1 -- 172.21.15.84:0/443618198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd088003970 msgr2=0x7fd088003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.100+0000 7fd0a1f6c640 1 --2- 172.21.15.84:0/443618198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd088003970 0x7fd088003d90 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009af0 tx=0x7fd0880057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.100+0000 7fd09bfff640 1 -- 172.21.15.84:0/443618198 reap_dead start 2023-10-16T10:08:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.100+0000 7fd0a1f6c640 1 -- 172.21.15.84:0/443618198 shutdown_connections 2023-10-16T10:08:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.100+0000 7fd0a1f6c640 1 -- 172.21.15.84:0/443618198 >> 172.21.15.84:0/443618198 conn(0x7fd09c0d0ab0 msgr2=0x7fd09c0d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.100+0000 7fd0a1f6c640 1 -- 172.21.15.84:0/443618198 shutdown_connections 2023-10-16T10:08:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.100+0000 7fd0a1f6c640 1 -- 172.21.15.84:0/443618198 wait complete. 2023-10-16T10:08:53.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.102+0000 7fd0a1f6c640 1 Processor -- start 2023-10-16T10:08:53.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.102+0000 7fd0a1f6c640 1 -- start start 2023-10-16T10:08:53.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.103+0000 7fd0a1f6c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd088003970 0x7fd09c0c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:53.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.103+0000 7fd0a1f6c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd09c1e7960 0x7fd09c154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:53.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.103+0000 7fd0a1f6c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd09c0b49e0 0x7fd09c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:53.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.103+0000 7fd0a1f6c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c0c8d90 con 0x7fd09c0b49e0 2023-10-16T10:08:53.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.103+0000 7fd0a1f6c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c0c8180 con 0x7fd09c1e7960 2023-10-16T10:08:53.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.103+0000 7fd0a1f6c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c0c8460 con 0x7fd088003970 2023-10-16T10:08:53.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.103+0000 7fd09bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd09c0b49e0 0x7fd09c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:53.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.103+0000 7fd09bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd09c0b49e0 0x7fd09c0b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53306/0 (socket says 172.21.15.84:53306) 2023-10-16T10:08:53.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.103+0000 7fd09bfff640 1 -- 172.21.15.84:0/3979062863 learned_addr learned my addr 172.21.15.84:0/3979062863 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:53.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.103+0000 7fd093fff640 1 --2- 172.21.15.84:0/3979062863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd09c1e7960 0x7fd09c154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:53.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.104+0000 7fd09bfff640 1 -- 172.21.15.84:0/3979062863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd088003970 msgr2=0x7fd09c0c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:53.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.104+0000 7fd09b7fe640 1 --2- 172.21.15.84:0/3979062863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd088003970 0x7fd09c0c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:53.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.104+0000 7fd09bfff640 1 --2- 172.21.15.84:0/3979062863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd088003970 0x7fd09c0c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.104+0000 7fd09bfff640 1 -- 172.21.15.84:0/3979062863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd09c1e7960 msgr2=0x7fd09c154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:53.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.104+0000 7fd09bfff640 1 --2- 172.21.15.84:0/3979062863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd09c1e7960 0x7fd09c154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.104+0000 7fd09bfff640 1 -- 172.21.15.84:0/3979062863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09c0c8740 con 0x7fd09c0b49e0 2023-10-16T10:08:53.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.104+0000 7fd093fff640 1 --2- 172.21.15.84:0/3979062863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd09c1e7960 0x7fd09c154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:53.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.104+0000 7fd09bfff640 1 --2- 172.21.15.84:0/3979062863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd09c0b49e0 0x7fd09c0b4e60 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fd09c06f340 tx=0x7fd09400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:53.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.104+0000 7fd098ff9640 1 -- 172.21.15.84:0/3979062863 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd094017400 con 0x7fd09c0b49e0 2023-10-16T10:08:53.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.104+0000 7fd098ff9640 1 -- 172.21.15.84:0/3979062863 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd094011040 con 0x7fd09c0b49e0 2023-10-16T10:08:53.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.104+0000 7fd0a1f6c640 1 -- 172.21.15.84:0/3979062863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd09c0c8a20 con 0x7fd09c0b49e0 2023-10-16T10:08:53.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.104+0000 7fd098ff9640 1 -- 172.21.15.84:0/3979062863 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0940177e0 con 0x7fd09c0b49e0 2023-10-16T10:08:53.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.104+0000 7fd0a1f6c640 1 -- 172.21.15.84:0/3979062863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09c1e7dc0 con 0x7fd09c0b49e0 2023-10-16T10:08:53.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.105+0000 7fd0a1f6c640 1 -- 172.21.15.84:0/3979062863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd09c161090 con 0x7fd09c0b49e0 2023-10-16T10:08:53.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.106+0000 7fd098ff9640 1 -- 172.21.15.84:0/3979062863 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 77) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fd094017940 con 0x7fd09c0b49e0 2023-10-16T10:08:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.107+0000 7fd098ff9640 1 --2- 172.21.15.84:0/3979062863 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fd07c06b020 0x7fd07c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.107+0000 7fd098ff9640 1 -- 172.21.15.84:0/3979062863 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7fd094089fb0 con 0x7fd09c0b49e0 2023-10-16T10:08:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.107+0000 7fd09b7fe640 1 -- 172.21.15.84:0/3979062863 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fd07c06b020 msgr2=0x7fd07c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1203729023 2023-10-16T10:08:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.107+0000 7fd09b7fe640 1 --2- 172.21.15.84:0/3979062863 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fd07c06b020 0x7fd07c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:08:53.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.112+0000 7fd098ff9640 1 -- 172.21.15.84:0/3979062863 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd09c161090 con 0x7fd09c0b49e0 2023-10-16T10:08:53.144 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:08:53.143+0000 7f5f914da040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:08:53.165 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:53.164+0000 7f099b258040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:08:53.225 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:53.223+0000 7f099b258040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:08:53.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.302+0000 7fd0a1f6c640 1 -- 172.21.15.84:0/3979062863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd09c0c8bb0 con 0x7fd09c0b49e0 2023-10-16T10:08:53.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.303+0000 7fd098ff9640 1 -- 172.21.15.84:0/3979062863 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd09401c0a0 con 0x7fd09c0b49e0 2023-10-16T10:08:53.305 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:08:53.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.306+0000 7fd0a1f6c640 1 -- 172.21.15.84:0/3979062863 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fd07c06b020 msgr2=0x7fd07c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:08:53.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.306+0000 7fd0a1f6c640 1 --2- 172.21.15.84:0/3979062863 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fd07c06b020 0x7fd07c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.306+0000 7fd0a1f6c640 1 -- 172.21.15.84:0/3979062863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd09c0b49e0 msgr2=0x7fd09c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:53.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.306+0000 7fd0a1f6c640 1 --2- 172.21.15.84:0/3979062863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd09c0b49e0 0x7fd09c0b4e60 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fd09c06f340 tx=0x7fd09400f4a0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.306+0000 7fd0a1f6c640 1 -- 172.21.15.84:0/3979062863 shutdown_connections 2023-10-16T10:08:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.306+0000 7fd0a1f6c640 1 --2- 172.21.15.84:0/3979062863 >> [v2:172.21.15.84:6832/1203729023,v1:172.21.15.84:6833/1203729023] conn(0x7fd07c06b020 0x7fd07c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.306+0000 7fd0a1f6c640 1 --2- 172.21.15.84:0/3979062863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd09c0b49e0 0x7fd09c0b4e60 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.306+0000 7fd0a1f6c640 1 --2- 172.21.15.84:0/3979062863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd09c1e7960 0x7fd09c154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.306+0000 7fd0a1f6c640 1 --2- 172.21.15.84:0/3979062863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd088003970 0x7fd09c0c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.306+0000 7fd0a1f6c640 1 -- 172.21.15.84:0/3979062863 >> 172.21.15.84:0/3979062863 conn(0x7fd09c0d0ab0 msgr2=0x7fd09c157c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.306+0000 7fd0a1f6c640 1 -- 172.21.15.84:0/3979062863 shutdown_connections 2023-10-16T10:08:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.306+0000 7fd0a1f6c640 1 -- 172.21.15.84:0/3979062863 wait complete. 2023-10-16T10:08:53.326 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:08:53.344 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:53.342+0000 7f099b258040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:08:53.422 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:53.420+0000 7f091dce6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:53.422 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:53.420+0000 7f091dce6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:53.422 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:53.420+0000 7f091dce6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:53.422 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:53.420+0000 7f091dce6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:53.423 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:53.420+0000 7f091dce6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:53.423 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:53.420+0000 7f0920cec640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:53.423 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:53.420+0000 7f0920cec640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:53.423 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:53.420+0000 7f0920cec640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:53.423 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:53.420+0000 7f0920cec640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:53.424 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:08:53.420+0000 7f0920cec640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:08:53.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.522+0000 7f5e39d1d640 1 Processor -- start 2023-10-16T10:08:53.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.522+0000 7f5e39d1d640 1 -- start start 2023-10-16T10:08:53.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.523+0000 7f5e39d1d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5e3415d150 0x7f5e340ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:53.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.523+0000 7f5e39d1d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e3406a4b0 0x7f5e34160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:53.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.523+0000 7f5e39d1d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e340ac890 con 0x7f5e34160b90 2023-10-16T10:08:53.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.523+0000 7f5e39d1d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3415d530 con 0x7f5e3415d150 2023-10-16T10:08:53.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.523+0000 7f5e39d1d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e34161650 con 0x7f5e3406a4b0 2023-10-16T10:08:53.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.524+0000 7f5e2bfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5e34160b90 0x7f5e34069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:60878/0 (socket says 172.21.15.84:60878) 2023-10-16T10:08:53.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.524+0000 7f5e2bfff640 1 -- 172.21.15.84:0/2470823904 learned_addr learned my addr 172.21.15.84:0/2470823904 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:53.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.524+0000 7f5e33fff640 1 --2- 172.21.15.84:0/2470823904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e3406a4b0 0x7f5e34160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:53.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.524+0000 7f5e337fe640 1 --2- 172.21.15.84:0/2470823904 >> v2:172.21.15.157:3300/0 conn(0x7f5e3415d150 0x7f5e340ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:53.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.525+0000 7f5e32ffd640 1 -- 172.21.15.84:0/2470823904 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039544019 0 0) 0x7f5e340ac890 con 0x7f5e34160b90 2023-10-16T10:08:53.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.525+0000 7f5e32ffd640 1 -- 172.21.15.84:0/2470823904 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e20003520 con 0x7f5e34160b90 2023-10-16T10:08:53.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.525+0000 7f5e337fe640 1 -- 172.21.15.84:0/2470823904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e3406a4b0 msgr2=0x7f5e34160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:53.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.525+0000 7f5e337fe640 1 --2- 172.21.15.84:0/2470823904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e3406a4b0 0x7f5e34160fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.525+0000 7f5e337fe640 1 -- 172.21.15.84:0/2470823904 >> v1:172.21.15.157:6789/0 conn(0x7f5e34160b90 legacy=0x7f5e34069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:53.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.525+0000 7f5e337fe640 1 -- 172.21.15.84:0/2470823904 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e34161930 con 0x7f5e3415d150 2023-10-16T10:08:53.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.525+0000 7f5e33fff640 1 --2- 172.21.15.84:0/2470823904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e3406a4b0 0x7f5e34160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:08:53.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.527+0000 7f5e337fe640 1 --2- 172.21.15.84:0/2470823904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3415d150 0x7f5e340ad350 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5e240097a0 tx=0x7f5e2402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=680e6edd76e9179 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:53.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.527+0000 7f5e32ffd640 1 -- 172.21.15.84:0/2470823904 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24031780 con 0x7f5e3415d150 2023-10-16T10:08:53.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.527+0000 7f5e32ffd640 1 -- 172.21.15.84:0/2470823904 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e240318e0 con 0x7f5e3415d150 2023-10-16T10:08:53.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.527+0000 7f5e32ffd640 1 -- 172.21.15.84:0/2470823904 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24031c50 con 0x7f5e3415d150 2023-10-16T10:08:53.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.528+0000 7f5e39d1d640 1 -- 172.21.15.84:0/2470823904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3415d150 msgr2=0x7f5e340ad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:53.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.528+0000 7f5e39d1d640 1 --2- 172.21.15.84:0/2470823904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3415d150 0x7f5e340ad350 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5e240097a0 tx=0x7f5e2402b820 comp rx=0 tx=0).stop 2023-10-16T10:08:53.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.528+0000 7f5e39d1d640 1 -- 172.21.15.84:0/2470823904 shutdown_connections 2023-10-16T10:08:53.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.528+0000 7f5e39d1d640 1 --2- 172.21.15.84:0/2470823904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e3406a4b0 0x7f5e34160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.528+0000 7f5e39d1d640 1 --2- 172.21.15.84:0/2470823904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3415d150 0x7f5e340ad350 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.528+0000 7f5e39d1d640 1 -- 172.21.15.84:0/2470823904 >> 172.21.15.84:0/2470823904 conn(0x7f5e340ce9b0 msgr2=0x7f5e340cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:53.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.528+0000 7f5e39d1d640 1 -- 172.21.15.84:0/2470823904 shutdown_connections 2023-10-16T10:08:53.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.528+0000 7f5e39d1d640 1 -- 172.21.15.84:0/2470823904 wait complete. 2023-10-16T10:08:53.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.530+0000 7f5e39d1d640 1 Processor -- start 2023-10-16T10:08:53.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.530+0000 7f5e39d1d640 1 -- start start 2023-10-16T10:08:53.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.531+0000 7f5e39d1d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e3406a4b0 0x7f5e340c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:53.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.531+0000 7f5e39d1d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3415d150 0x7f5e340c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:53.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.531+0000 7f5e39d1d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e34160b90 0x7f5e3414bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:53.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.531+0000 7f5e39d1d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340c39d0 con 0x7f5e34160b90 2023-10-16T10:08:53.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.531+0000 7f5e39d1d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340c2d50 con 0x7f5e3415d150 2023-10-16T10:08:53.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.531+0000 7f5e39d1d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340c3030 con 0x7f5e3406a4b0 2023-10-16T10:08:53.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.531+0000 7f5e33fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e34160b90 0x7f5e3414bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:53.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.531+0000 7f5e33fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e34160b90 0x7f5e3414bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53390/0 (socket says 172.21.15.84:53390) 2023-10-16T10:08:53.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.531+0000 7f5e33fff640 1 -- 172.21.15.84:0/1674763646 learned_addr learned my addr 172.21.15.84:0/1674763646 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:53.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.532+0000 7f5e337fe640 1 --2- 172.21.15.84:0/1674763646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e3406a4b0 0x7f5e340c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:53.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.532+0000 7f5e2bfff640 1 --2- 172.21.15.84:0/1674763646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3415d150 0x7f5e340c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:53.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.532+0000 7f5e33fff640 1 -- 172.21.15.84:0/1674763646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e3406a4b0 msgr2=0x7f5e340c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:53.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.532+0000 7f5e33fff640 1 --2- 172.21.15.84:0/1674763646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e3406a4b0 0x7f5e340c3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.532+0000 7f5e33fff640 1 -- 172.21.15.84:0/1674763646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3415d150 msgr2=0x7f5e340c24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:53.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.532+0000 7f5e33fff640 1 --2- 172.21.15.84:0/1674763646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3415d150 0x7f5e340c24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.532+0000 7f5e33fff640 1 -- 172.21.15.84:0/1674763646 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c009a70 con 0x7f5e34160b90 2023-10-16T10:08:53.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.532+0000 7f5e33fff640 1 --2- 172.21.15.84:0/1674763646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e34160b90 0x7f5e3414bb60 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00efc0 tx=0x7f5e2c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:53.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.532+0000 7f5e30ff9640 1 -- 172.21.15.84:0/1674763646 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c009cd0 con 0x7f5e34160b90 2023-10-16T10:08:53.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.532+0000 7f5e30ff9640 1 -- 172.21.15.84:0/1674763646 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c00f040 con 0x7f5e34160b90 2023-10-16T10:08:53.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.532+0000 7f5e39d1d640 1 -- 172.21.15.84:0/1674763646 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e24009400 con 0x7f5e34160b90 2023-10-16T10:08:53.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.532+0000 7f5e30ff9640 1 -- 172.21.15.84:0/1674763646 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c003990 con 0x7f5e34160b90 2023-10-16T10:08:53.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.532+0000 7f5e39d1d640 1 -- 172.21.15.84:0/1674763646 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e340c35f0 con 0x7f5e34160b90 2023-10-16T10:08:53.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.533+0000 7f5e39d1d640 1 -- 172.21.15.84:0/1674763646 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e340c3b10 con 0x7f5e34160b90 2023-10-16T10:08:53.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.534+0000 7f5e30ff9640 1 -- 172.21.15.84:0/1674763646 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 78) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f5e2c014480 con 0x7f5e34160b90 2023-10-16T10:08:53.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.535+0000 7f5e30ff9640 1 -- 172.21.15.84:0/1674763646 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f5e2c08a100 con 0x7f5e34160b90 2023-10-16T10:08:53.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.539+0000 7f5e30ff9640 1 -- 172.21.15.84:0/1674763646 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5e340c3b10 con 0x7f5e34160b90 2023-10-16T10:08:53.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.715+0000 7f5e39d1d640 1 -- 172.21.15.84:0/1674763646 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5e340c3b10 con 0x7f5e34160b90 2023-10-16T10:08:53.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.716+0000 7f5e30ff9640 1 -- 172.21.15.84:0/1674763646 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v140) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5e340c3b10 con 0x7f5e34160b90 2023-10-16T10:08:53.718 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T10:08:53.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.718+0000 7f5e39d1d640 1 -- 172.21.15.84:0/1674763646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e34160b90 msgr2=0x7f5e3414bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:53.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.718+0000 7f5e39d1d640 1 --2- 172.21.15.84:0/1674763646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e34160b90 0x7f5e3414bb60 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00efc0 tx=0x7f5e2c00c490 comp rx=0 tx=0).stop 2023-10-16T10:08:53.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.719+0000 7f5e39d1d640 1 -- 172.21.15.84:0/1674763646 shutdown_connections 2023-10-16T10:08:53.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.719+0000 7f5e39d1d640 1 --2- 172.21.15.84:0/1674763646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e34160b90 0x7f5e3414bb60 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.719+0000 7f5e39d1d640 1 --2- 172.21.15.84:0/1674763646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3415d150 0x7f5e340c24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.719+0000 7f5e39d1d640 1 --2- 172.21.15.84:0/1674763646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e3406a4b0 0x7f5e340c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:53.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.719+0000 7f5e39d1d640 1 -- 172.21.15.84:0/1674763646 >> 172.21.15.84:0/1674763646 conn(0x7f5e340ce9b0 msgr2=0x7f5e340cac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:53.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.719+0000 7f5e39d1d640 1 -- 172.21.15.84:0/1674763646 shutdown_connections 2023-10-16T10:08:53.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.719+0000 7f5e39d1d640 1 -- 172.21.15.84:0/1674763646 wait complete. 2023-10-16T10:08:53.736 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:08:53.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.993+0000 7ffa5ffff640 1 Processor -- start 2023-10-16T10:08:53.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.993+0000 7ffa5ffff640 1 -- start start 2023-10-16T10:08:53.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.995+0000 7ffa5ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa600b8480 0x7ffa600cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.995+0000 7ffa5ffff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa6015d150 con 0x7ffa600d0be0 2023-10-16T10:08:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.995+0000 7ffa5ffff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa600d1180 con 0x7ffa6015cd70 2023-10-16T10:08:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.995+0000 7ffa5ffff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa600d1460 con 0x7ffa600b8480 2023-10-16T10:08:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.995+0000 7ffa5f7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ffa600d0be0 0x7ffa600b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46924/0 (socket says 172.21.15.84:46924) 2023-10-16T10:08:53.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.995+0000 7ffa5f7fe640 1 -- 172.21.15.84:0/1507889867 learned_addr learned my addr 172.21.15.84:0/1507889867 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:54.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.996+0000 7ffa5dffb640 1 -- 172.21.15.84:0/1507889867 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195160646 0 0) 0x7ffa6015d150 con 0x7ffa600d0be0 2023-10-16T10:08:54.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.996+0000 7ffa5dffb640 1 -- 172.21.15.84:0/1507889867 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa4c003520 con 0x7ffa600d0be0 2023-10-16T10:08:54.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.996+0000 7ffa5e7fc640 1 --2- 172.21.15.84:0/1507889867 >> v2:172.21.15.157:3300/0 conn(0x7ffa600b8480 0x7ffa600cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:54.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.996+0000 7ffa5dffb640 1 -- 172.21.15.84:0/1507889867 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2036515134 0 0) 0x7ffa4c003520 con 0x7ffa600d0be0 2023-10-16T10:08:54.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.996+0000 7ffa5dffb640 1 -- 172.21.15.84:0/1507889867 >> v2:172.21.15.157:3300/0 conn(0x7ffa600b8480 msgr2=0x7ffa600cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:54.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.996+0000 7ffa5dffb640 1 --2- 172.21.15.84:0/1507889867 >> v2:172.21.15.157:3300/0 conn(0x7ffa600b8480 0x7ffa600cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.996+0000 7ffa5dffb640 1 -- 172.21.15.84:0/1507889867 >> v1:172.21.15.157:6789/0 conn(0x7ffa6015cd70 legacy=0x7ffa600d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:54.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.996+0000 7ffa5dffb640 1 -- 172.21.15.84:0/1507889867 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa600d1740 con 0x7ffa600d0be0 2023-10-16T10:08:54.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.996+0000 7ffa5dffb640 1 -- 172.21.15.84:0/1507889867 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ffa58001670 con 0x7ffa600d0be0 2023-10-16T10:08:54.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.996+0000 7ffa5dffb640 1 -- 172.21.15.84:0/1507889867 >> v1:172.21.15.84:6789/0 conn(0x7ffa600d0be0 legacy=0x7ffa600b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:54.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.996+0000 7ffa5dffb640 1 --2- 172.21.15.84:0/1507889867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa4c0039c0 0x7ffa4c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.996+0000 7ffa5dffb640 1 --2- 172.21.15.84:0/1507889867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa4c0047c0 0x7ffa4c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.996+0000 7ffa5dffb640 1 --2- 172.21.15.84:0/1507889867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa4c005250 0x7ffa4c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.996+0000 7ffa5dffb640 1 -- 172.21.15.84:0/1507889867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa600d1460 con 0x7ffa4c005250 2023-10-16T10:08:54.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.996+0000 7ffa5dffb640 1 -- 172.21.15.84:0/1507889867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c00c1a0 con 0x7ffa4c0039c0 2023-10-16T10:08:54.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.996+0000 7ffa5dffb640 1 -- 172.21.15.84:0/1507889867 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c00c360 con 0x7ffa4c0047c0 2023-10-16T10:08:54.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.996+0000 7ffa5f7fe640 1 --2- 172.21.15.84:0/1507889867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa4c005250 0x7ffa4c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:54.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.997+0000 7ffa5effd640 1 --2- 172.21.15.84:0/1507889867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa4c0039c0 0x7ffa4c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:54.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.997+0000 7ffa5f7fe640 1 -- 172.21.15.84:0/1507889867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa4c0047c0 msgr2=0x7ffa4c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:54.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.997+0000 7ffa5f7fe640 1 --2- 172.21.15.84:0/1507889867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa4c0047c0 0x7ffa4c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.997+0000 7ffa5f7fe640 1 -- 172.21.15.84:0/1507889867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa4c0039c0 msgr2=0x7ffa4c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:54.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.997+0000 7ffa5f7fe640 1 --2- 172.21.15.84:0/1507889867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa4c0039c0 0x7ffa4c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.997+0000 7ffa5f7fe640 1 -- 172.21.15.84:0/1507889867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c003520 con 0x7ffa4c005250 2023-10-16T10:08:54.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.997+0000 7ffa5effd640 1 --2- 172.21.15.84:0/1507889867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa4c0039c0 0x7ffa4c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:54.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.998+0000 7ffa5f7fe640 1 -- 172.21.15.84:0/1507889867 reap_dead start 2023-10-16T10:08:54.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.998+0000 7ffa5f7fe640 1 --2- 172.21.15.84:0/1507889867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa4c005250 0x7ffa4c00b980 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800c190 tx=0x7ffa4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bd20bd526fe858 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:54.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.998+0000 7ffa5dffb640 1 -- 172.21.15.84:0/1507889867 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58005ad0 con 0x7ffa4c005250 2023-10-16T10:08:54.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.998+0000 7ffa5dffb640 1 -- 172.21.15.84:0/1507889867 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa58005c70 con 0x7ffa4c005250 2023-10-16T10:08:54.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.998+0000 7ffa5dffb640 1 -- 172.21.15.84:0/1507889867 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5802c7f0 con 0x7ffa4c005250 2023-10-16T10:08:54.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.999+0000 7ffa5ffff640 1 -- 172.21.15.84:0/1507889867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa4c005250 msgr2=0x7ffa4c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:54.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.999+0000 7ffa5ffff640 1 --2- 172.21.15.84:0/1507889867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa4c005250 0x7ffa4c00b980 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800c190 tx=0x7ffa4c0057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.999+0000 7ffa5ffff640 1 -- 172.21.15.84:0/1507889867 shutdown_connections 2023-10-16T10:08:54.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.999+0000 7ffa5ffff640 1 --2- 172.21.15.84:0/1507889867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa4c005250 0x7ffa4c00b980 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.999+0000 7ffa5ffff640 1 -- 172.21.15.84:0/1507889867 >> 172.21.15.84:0/1507889867 conn(0x7ffa600cf570 msgr2=0x7ffa600cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:54.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.999+0000 7ffa5ffff640 1 -- 172.21.15.84:0/1507889867 shutdown_connections 2023-10-16T10:08:54.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:53.999+0000 7ffa5ffff640 1 -- 172.21.15.84:0/1507889867 wait complete. 2023-10-16T10:08:54.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.000+0000 7ffa5ffff640 1 Processor -- start 2023-10-16T10:08:54.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.001+0000 7ffa5ffff640 1 -- start start 2023-10-16T10:08:54.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.001+0000 7ffa5ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa4c005250 0x7ffa600c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.001+0000 7ffa5ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa60155680 0x7ffa6015f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.002+0000 7ffa5ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa60152be0 0x7ffa60150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.002+0000 7ffa5ffff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa60155b00 con 0x7ffa60152be0 2023-10-16T10:08:54.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.002+0000 7ffa5ffff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa600ca250 con 0x7ffa4c005250 2023-10-16T10:08:54.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.002+0000 7ffa5ffff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa600ca530 con 0x7ffa60155680 2023-10-16T10:08:54.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.002+0000 7ffa5f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa60152be0 0x7ffa60150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:54.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.002+0000 7ffa5f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa60152be0 0x7ffa60150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53396/0 (socket says 172.21.15.84:53396) 2023-10-16T10:08:54.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.002+0000 7ffa5f7fe640 1 -- 172.21.15.84:0/548744074 learned_addr learned my addr 172.21.15.84:0/548744074 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:54.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.002+0000 7ffa5e7fc640 1 --2- 172.21.15.84:0/548744074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa60155680 0x7ffa6015f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:54.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.002+0000 7ffa5effd640 1 --2- 172.21.15.84:0/548744074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa4c005250 0x7ffa600c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:54.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.002+0000 7ffa5f7fe640 1 -- 172.21.15.84:0/548744074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa60155680 msgr2=0x7ffa6015f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:54.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.002+0000 7ffa5f7fe640 1 --2- 172.21.15.84:0/548744074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa60155680 0x7ffa6015f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.002+0000 7ffa5f7fe640 1 -- 172.21.15.84:0/548744074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa4c005250 msgr2=0x7ffa600c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:54.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.002+0000 7ffa5f7fe640 1 --2- 172.21.15.84:0/548744074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa4c005250 0x7ffa600c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.002+0000 7ffa5f7fe640 1 -- 172.21.15.84:0/548744074 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa600ca810 con 0x7ffa60152be0 2023-10-16T10:08:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.003+0000 7ffa5f7fe640 1 --2- 172.21.15.84:0/548744074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa60152be0 0x7ffa60150ab0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7ffa5802c460 tx=0x7ffa58001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.003+0000 7ffa577fe640 1 -- 172.21.15.84:0/548744074 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa580049e0 con 0x7ffa60152be0 2023-10-16T10:08:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.003+0000 7ffa577fe640 1 -- 172.21.15.84:0/548744074 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa58004b40 con 0x7ffa60152be0 2023-10-16T10:08:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.003+0000 7ffa577fe640 1 -- 172.21.15.84:0/548744074 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5802de20 con 0x7ffa60152be0 2023-10-16T10:08:54.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.003+0000 7ffa5ffff640 1 -- 172.21.15.84:0/548744074 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa600caaf0 con 0x7ffa60152be0 2023-10-16T10:08:54.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.003+0000 7ffa5ffff640 1 -- 172.21.15.84:0/548744074 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa6015f970 con 0x7ffa60152be0 2023-10-16T10:08:54.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.004+0000 7ffa5ffff640 1 -- 172.21.15.84:0/548744074 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa6015fb00 con 0x7ffa60152be0 2023-10-16T10:08:54.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.005+0000 7ffa577fe640 1 -- 172.21.15.84:0/548744074 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 78) v1 ==== 88940+0+0 (secure 0 0 0) 0x7ffa58032070 con 0x7ffa60152be0 2023-10-16T10:08:54.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.006+0000 7ffa577fe640 1 -- 172.21.15.84:0/548744074 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6468+0+0 (secure 0 0 0) 0x7ffa5803c070 con 0x7ffa60152be0 2023-10-16T10:08:54.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.010+0000 7ffa577fe640 1 -- 172.21.15.84:0/548744074 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffa58075210 con 0x7ffa60152be0 2023-10-16T10:08:54.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.191+0000 7ffa5ffff640 1 -- 172.21.15.84:0/548744074 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffa6004e9b0 con 0x7ffa60152be0 2023-10-16T10:08:54.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.192+0000 7ffa577fe640 1 -- 172.21.15.84:0/548744074 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v140) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffa58078830 con 0x7ffa60152be0 2023-10-16T10:08:54.194 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T10:08:54.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.195+0000 7ffa5ffff640 1 -- 172.21.15.84:0/548744074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa60152be0 msgr2=0x7ffa60150ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:54.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.195+0000 7ffa5ffff640 1 --2- 172.21.15.84:0/548744074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa60152be0 0x7ffa60150ab0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7ffa5802c460 tx=0x7ffa58001930 comp rx=0 tx=0).stop 2023-10-16T10:08:54.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.195+0000 7ffa5ffff640 1 -- 172.21.15.84:0/548744074 shutdown_connections 2023-10-16T10:08:54.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.195+0000 7ffa5ffff640 1 --2- 172.21.15.84:0/548744074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa60152be0 0x7ffa60150ab0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.195+0000 7ffa5ffff640 1 --2- 172.21.15.84:0/548744074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa60155680 0x7ffa6015f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.195+0000 7ffa5ffff640 1 --2- 172.21.15.84:0/548744074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa4c005250 0x7ffa600c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.195+0000 7ffa5ffff640 1 -- 172.21.15.84:0/548744074 >> 172.21.15.84:0/548744074 conn(0x7ffa600cf570 msgr2=0x7ffa600b7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:54.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.195+0000 7ffa5ffff640 1 -- 172.21.15.84:0/548744074 shutdown_connections 2023-10-16T10:08:54.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.195+0000 7ffa5ffff640 1 -- 172.21.15.84:0/548744074 wait complete. 2023-10-16T10:08:54.214 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:08:54.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.429+0000 7f656a01f640 1 Processor -- start 2023-10-16T10:08:54.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.430+0000 7f656a01f640 1 -- start start 2023-10-16T10:08:54.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.431+0000 7f656a01f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f656415aaa0 0x7f656415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.431+0000 7f656a01f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656415ed60 con 0x7f656415ae80 2023-10-16T10:08:54.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.431+0000 7f656a01f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65640707e0 con 0x7f65640ce390 2023-10-16T10:08:54.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.431+0000 7f656a01f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f656406b810 con 0x7f656415aaa0 2023-10-16T10:08:54.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.431+0000 7f655affd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f656415ae80 0x7f65640cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46930/0 (socket says 172.21.15.84:46930) 2023-10-16T10:08:54.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.431+0000 7f655affd640 1 -- 172.21.15.84:0/164870431 learned_addr learned my addr 172.21.15.84:0/164870431 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:54.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.432+0000 7f6562ffd640 1 -- 172.21.15.84:0/164870431 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297539895 0 0) 0x7f656415ed60 con 0x7f656415ae80 2023-10-16T10:08:54.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.432+0000 7f6562ffd640 1 -- 172.21.15.84:0/164870431 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6550003520 con 0x7f656415ae80 2023-10-16T10:08:54.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.432+0000 7f65637fe640 1 --2- 172.21.15.84:0/164870431 >> v2:172.21.15.157:3300/0 conn(0x7f656415aaa0 0x7f656415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:54.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.432+0000 7f6562ffd640 1 -- 172.21.15.84:0/164870431 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1228346721 0 0) 0x7f6550003520 con 0x7f656415ae80 2023-10-16T10:08:54.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.432+0000 7f6562ffd640 1 -- 172.21.15.84:0/164870431 >> v2:172.21.15.157:3300/0 conn(0x7f656415aaa0 msgr2=0x7f656415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:54.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.432+0000 7f6562ffd640 1 --2- 172.21.15.84:0/164870431 >> v2:172.21.15.157:3300/0 conn(0x7f656415aaa0 0x7f656415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.432+0000 7f6562ffd640 1 -- 172.21.15.84:0/164870431 >> v1:172.21.15.157:6789/0 conn(0x7f65640ce390 legacy=0x7f6564070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:54.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.432+0000 7f6562ffd640 1 -- 172.21.15.84:0/164870431 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656415ef90 con 0x7f656415ae80 2023-10-16T10:08:54.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.432+0000 7f6562ffd640 1 -- 172.21.15.84:0/164870431 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f654c001670 con 0x7f656415ae80 2023-10-16T10:08:54.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.432+0000 7f6562ffd640 1 -- 172.21.15.84:0/164870431 >> v1:172.21.15.84:6789/0 conn(0x7f656415ae80 legacy=0x7f65640cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:54.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.432+0000 7f6562ffd640 1 --2- 172.21.15.84:0/164870431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65500039c0 0x7f6550003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.432+0000 7f6562ffd640 1 --2- 172.21.15.84:0/164870431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65500047c0 0x7f6550004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.432+0000 7f6562ffd640 1 --2- 172.21.15.84:0/164870431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65500051d0 0x7f655000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.432+0000 7f6562ffd640 1 -- 172.21.15.84:0/164870431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656406b810 con 0x7f65500051d0 2023-10-16T10:08:54.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.432+0000 7f6562ffd640 1 -- 172.21.15.84:0/164870431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655000c120 con 0x7f65500039c0 2023-10-16T10:08:54.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.432+0000 7f6562ffd640 1 -- 172.21.15.84:0/164870431 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655000c2e0 con 0x7f65500047c0 2023-10-16T10:08:54.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.432+0000 7f6563fff640 1 --2- 172.21.15.84:0/164870431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65500051d0 0x7f655000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:54.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.433+0000 7f65637fe640 1 --2- 172.21.15.84:0/164870431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65500039c0 0x7f6550003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:54.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.433+0000 7f6563fff640 1 -- 172.21.15.84:0/164870431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65500047c0 msgr2=0x7f6550004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:54.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.433+0000 7f6563fff640 1 --2- 172.21.15.84:0/164870431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65500047c0 0x7f6550004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.433+0000 7f6563fff640 1 -- 172.21.15.84:0/164870431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65500039c0 msgr2=0x7f6550003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:54.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.433+0000 7f6563fff640 1 --2- 172.21.15.84:0/164870431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65500039c0 0x7f6550003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.433+0000 7f6563fff640 1 -- 172.21.15.84:0/164870431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6550003520 con 0x7f65500051d0 2023-10-16T10:08:54.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.433+0000 7f65637fe640 1 --2- 172.21.15.84:0/164870431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65500039c0 0x7f6550003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:54.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.434+0000 7f6563fff640 1 -- 172.21.15.84:0/164870431 reap_dead start 2023-10-16T10:08:54.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.434+0000 7f6563fff640 1 --2- 172.21.15.84:0/164870431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65500051d0 0x7f655000b980 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f655c001750 tx=0x7f65500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f350abe69dbc3f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:54.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.434+0000 7f6562ffd640 1 -- 172.21.15.84:0/164870431 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c0052d0 con 0x7f65500051d0 2023-10-16T10:08:54.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.434+0000 7f6562ffd640 1 -- 172.21.15.84:0/164870431 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c02e050 con 0x7f65500051d0 2023-10-16T10:08:54.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.434+0000 7f6562ffd640 1 -- 172.21.15.84:0/164870431 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c003830 con 0x7f65500051d0 2023-10-16T10:08:54.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.435+0000 7f656a01f640 1 -- 172.21.15.84:0/164870431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65500051d0 msgr2=0x7f655000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:54.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.435+0000 7f656a01f640 1 --2- 172.21.15.84:0/164870431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65500051d0 0x7f655000b980 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f655c001750 tx=0x7f65500057f0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.435+0000 7f656a01f640 1 -- 172.21.15.84:0/164870431 shutdown_connections 2023-10-16T10:08:54.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.435+0000 7f656a01f640 1 --2- 172.21.15.84:0/164870431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65500051d0 0x7f655000b980 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.435+0000 7f656a01f640 1 -- 172.21.15.84:0/164870431 >> 172.21.15.84:0/164870431 conn(0x7f65640cb410 msgr2=0x7f65640cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:54.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.435+0000 7f656a01f640 1 -- 172.21.15.84:0/164870431 shutdown_connections 2023-10-16T10:08:54.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.435+0000 7f656a01f640 1 -- 172.21.15.84:0/164870431 wait complete. 2023-10-16T10:08:54.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.437+0000 7f656a01f640 1 Processor -- start 2023-10-16T10:08:54.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.437+0000 7f656a01f640 1 -- start start 2023-10-16T10:08:54.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.437+0000 7f656a01f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65500051d0 0x7f6564158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.438+0000 7f656a01f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f656415bd30 0x7f656415cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.438+0000 7f656a01f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f656415d4e0 0x7f65640c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.438+0000 7f656a01f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65640c8e10 con 0x7f656415bd30 2023-10-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.438+0000 7f656a01f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656415c1c0 con 0x7f656415d4e0 2023-10-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.438+0000 7f656a01f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656415c4a0 con 0x7f65500051d0 2023-10-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.438+0000 7f655affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f656415bd30 0x7f656415cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.438+0000 7f655affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f656415bd30 0x7f656415cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53422/0 (socket says 172.21.15.84:53422) 2023-10-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.438+0000 7f655affd640 1 -- 172.21.15.84:0/3393315734 learned_addr learned my addr 172.21.15.84:0/3393315734 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.438+0000 7f655affd640 1 -- 172.21.15.84:0/3393315734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65500051d0 msgr2=0x7f6564158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.438+0000 7f655affd640 1 --2- 172.21.15.84:0/3393315734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65500051d0 0x7f6564158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.438+0000 7f655affd640 1 -- 172.21.15.84:0/3393315734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f656415d4e0 msgr2=0x7f65640c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:54.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.438+0000 7f655affd640 1 --2- 172.21.15.84:0/3393315734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f656415d4e0 0x7f65640c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.438+0000 7f655affd640 1 -- 172.21.15.84:0/3393315734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656415c780 con 0x7f656415bd30 2023-10-16T10:08:54.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.439+0000 7f655affd640 1 --2- 172.21.15.84:0/3393315734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f656415bd30 0x7f656415cfa0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f654c00c700 tx=0x7f654c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:54.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.439+0000 7f6560ff9640 1 -- 172.21.15.84:0/3393315734 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c0072d0 con 0x7f656415bd30 2023-10-16T10:08:54.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.439+0000 7f6560ff9640 1 -- 172.21.15.84:0/3393315734 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654c007430 con 0x7f656415bd30 2023-10-16T10:08:54.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.439+0000 7f6560ff9640 1 -- 172.21.15.84:0/3393315734 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c0047e0 con 0x7f656415bd30 2023-10-16T10:08:54.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.439+0000 7f656a01f640 1 -- 172.21.15.84:0/3393315734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656415ca60 con 0x7f656415bd30 2023-10-16T10:08:54.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.439+0000 7f656a01f640 1 -- 172.21.15.84:0/3393315734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65640c72b0 con 0x7f656415bd30 2023-10-16T10:08:54.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.440+0000 7f656a01f640 1 -- 172.21.15.84:0/3393315734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656404e160 con 0x7f656415bd30 2023-10-16T10:08:54.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.441+0000 7f6560ff9640 1 -- 172.21.15.84:0/3393315734 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 79) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f654c004940 con 0x7f656415bd30 2023-10-16T10:08:54.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.442+0000 7f6560ff9640 1 --2- 172.21.15.84:0/3393315734 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f654406acc0 0x7f654406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.442+0000 7f6560ff9640 1 -- 172.21.15.84:0/3393315734 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f654c0889c0 con 0x7f656415bd30 2023-10-16T10:08:54.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.442+0000 7f65637fe640 1 --2- 172.21.15.84:0/3393315734 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f654406acc0 0x7f654406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:54.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.443+0000 7f65637fe640 1 --2- 172.21.15.84:0/3393315734 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f654406acc0 0x7f654406d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6554009540 tx=0x7f655400b040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:54.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.446+0000 7f6560ff9640 1 -- 172.21.15.84:0/3393315734 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f654c0539e0 con 0x7f656415bd30 2023-10-16T10:08:54.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.677+0000 7f656a01f640 1 -- 172.21.15.84:0/3393315734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65640c9010 con 0x7f656415bd30 2023-10-16T10:08:54.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.678+0000 7f6560ff9640 1 -- 172.21.15.84:0/3393315734 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f65640c9010 con 0x7f656415bd30 2023-10-16T10:08:54.680 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:54.681 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4118285132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":4118285132},{"type":"v1","addr":"172.21.15.84:6837","nonce":4118285132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3743702675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3743702675},{"type":"v1","addr":"172.21.15.157:6834","nonce":3743702675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2895418246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2895418246},{"type":"v1","addr":"172.21.15.157:6835","nonce":2895418246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:08:25.705299+0000","modified":"2023-10-16T10:08:28.762900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8574":{"gid":8574,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/791907708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":791907708},{"type":"v1","addr":"172.21.15.84:6836","nonce":791907708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-16T10:08:54.681 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 71 2023-10-16T10:08:54.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.680+0000 7f656a01f640 1 -- 172.21.15.84:0/3393315734 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f654406acc0 msgr2=0x7f654406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:54.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.680+0000 7f656a01f640 1 --2- 172.21.15.84:0/3393315734 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f654406acc0 0x7f654406d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6554009540 tx=0x7f655400b040 comp rx=0 tx=0).stop 2023-10-16T10:08:54.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.680+0000 7f656a01f640 1 -- 172.21.15.84:0/3393315734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f656415bd30 msgr2=0x7f656415cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:54.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.680+0000 7f656a01f640 1 --2- 172.21.15.84:0/3393315734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f656415bd30 0x7f656415cfa0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f654c00c700 tx=0x7f654c00cbc0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.681+0000 7f656a01f640 1 -- 172.21.15.84:0/3393315734 shutdown_connections 2023-10-16T10:08:54.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.681+0000 7f656a01f640 1 --2- 172.21.15.84:0/3393315734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f656415d4e0 0x7f65640c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.681+0000 7f656a01f640 1 --2- 172.21.15.84:0/3393315734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f656415bd30 0x7f656415cfa0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.681+0000 7f656a01f640 1 --2- 172.21.15.84:0/3393315734 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f654406acc0 0x7f654406d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.681+0000 7f656a01f640 1 --2- 172.21.15.84:0/3393315734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65500051d0 0x7f6564158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.681+0000 7f656a01f640 1 -- 172.21.15.84:0/3393315734 >> 172.21.15.84:0/3393315734 conn(0x7f65640cb410 msgr2=0x7f65640b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:54.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.681+0000 7f656a01f640 1 -- 172.21.15.84:0/3393315734 shutdown_connections 2023-10-16T10:08:54.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.681+0000 7f656a01f640 1 -- 172.21.15.84:0/3393315734 wait complete. 2023-10-16T10:08:54.700 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:08:54.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.906+0000 7fd86034e640 1 Processor -- start 2023-10-16T10:08:54.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.906+0000 7fd86034e640 1 -- start start 2023-10-16T10:08:54.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.908+0000 7fd86034e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd85815aaa0 0x7fd85815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.908+0000 7fd86034e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8580ce390 0x7fd858070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.908+0000 7fd86034e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85815ed60 con 0x7fd85815ae80 2023-10-16T10:08:54.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.908+0000 7fd86034e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85806b810 con 0x7fd85815aaa0 2023-10-16T10:08:54.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.908+0000 7fd86034e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8580707e0 con 0x7fd8580ce390 2023-10-16T10:08:54.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.908+0000 7fd85e0ea640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd85815aaa0 0x7fd85815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:54.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.908+0000 7fd85d8e9640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd85815ae80 0x7fd8580cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46942/0 (socket says 172.21.15.84:46942) 2023-10-16T10:08:54.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.908+0000 7fd85d8e9640 1 -- 172.21.15.84:0/440795821 learned_addr learned my addr 172.21.15.84:0/440795821 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:54.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.909+0000 7fd85e0ea640 1 -- 172.21.15.84:0/440795821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8580ce390 msgr2=0x7fd858070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:54.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.909+0000 7fd85e0ea640 1 --2- 172.21.15.84:0/440795821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8580ce390 0x7fd858070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.909+0000 7fd85e0ea640 1 -- 172.21.15.84:0/440795821 >> v1:172.21.15.84:6789/0 conn(0x7fd85815ae80 legacy=0x7fd8580cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:54.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.909+0000 7fd85e0ea640 1 -- 172.21.15.84:0/440795821 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85815ef90 con 0x7fd85815aaa0 2023-10-16T10:08:54.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.909+0000 7fd85d0e8640 1 -- 172.21.15.84:0/440795821 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 769865012 0 0) 0x7fd85815ed60 con 0x7fd85815ae80 2023-10-16T10:08:54.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.910+0000 7fd85e0ea640 1 --2- 172.21.15.84:0/440795821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd85815aaa0 0x7fd85815e820 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fd8540097a0 tx=0x7fd85402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c116844c59ce03 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:54.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.910+0000 7fd85d0e8640 1 -- 172.21.15.84:0/440795821 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd854031830 con 0x7fd85815aaa0 2023-10-16T10:08:54.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.910+0000 7fd85d0e8640 1 -- 172.21.15.84:0/440795821 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd854031990 con 0x7fd85815aaa0 2023-10-16T10:08:54.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.910+0000 7fd85d0e8640 1 -- 172.21.15.84:0/440795821 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd854031d20 con 0x7fd85815aaa0 2023-10-16T10:08:54.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.911+0000 7fd86034e640 1 -- 172.21.15.84:0/440795821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd85815aaa0 msgr2=0x7fd85815e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:54.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.911+0000 7fd86034e640 1 --2- 172.21.15.84:0/440795821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd85815aaa0 0x7fd85815e820 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fd8540097a0 tx=0x7fd85402b8d0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.911+0000 7fd86034e640 1 -- 172.21.15.84:0/440795821 shutdown_connections 2023-10-16T10:08:54.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.911+0000 7fd86034e640 1 --2- 172.21.15.84:0/440795821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8580ce390 0x7fd858070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.911+0000 7fd86034e640 1 --2- 172.21.15.84:0/440795821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd85815aaa0 0x7fd85815e820 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.911+0000 7fd86034e640 1 -- 172.21.15.84:0/440795821 >> 172.21.15.84:0/440795821 conn(0x7fd8580cb410 msgr2=0x7fd8580cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:54.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.911+0000 7fd86034e640 1 -- 172.21.15.84:0/440795821 shutdown_connections 2023-10-16T10:08:54.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.911+0000 7fd86034e640 1 -- 172.21.15.84:0/440795821 wait complete. 2023-10-16T10:08:54.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.913+0000 7fd86034e640 1 Processor -- start 2023-10-16T10:08:54.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.913+0000 7fd86034e640 1 -- start start 2023-10-16T10:08:54.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.914+0000 7fd86034e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8580ce390 0x7fd8581f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.914+0000 7fd86034e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85815aaa0 0x7fd858159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.914+0000 7fd86034e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd85815ae80 0x7fd8580c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.914+0000 7fd86034e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8580c25f0 con 0x7fd85815ae80 2023-10-16T10:08:54.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.914+0000 7fd86034e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85815a160 con 0x7fd8580ce390 2023-10-16T10:08:54.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.914+0000 7fd86034e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8580c9810 con 0x7fd85815aaa0 2023-10-16T10:08:54.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.914+0000 7fd85e8eb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd85815ae80 0x7fd8580c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:54.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.914+0000 7fd85e8eb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd85815ae80 0x7fd8580c8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53446/0 (socket says 172.21.15.84:53446) 2023-10-16T10:08:54.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.914+0000 7fd85e8eb640 1 -- 172.21.15.84:0/1702688194 learned_addr learned my addr 172.21.15.84:0/1702688194 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:54.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.914+0000 7fd85e8eb640 1 -- 172.21.15.84:0/1702688194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85815aaa0 msgr2=0x7fd858159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:54.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.914+0000 7fd85d8e9640 1 --2- 172.21.15.84:0/1702688194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85815aaa0 0x7fd858159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:54.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.914+0000 7fd85e8eb640 1 --2- 172.21.15.84:0/1702688194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85815aaa0 0x7fd858159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.915+0000 7fd85e8eb640 1 -- 172.21.15.84:0/1702688194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8580ce390 msgr2=0x7fd8581f66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:54.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.915+0000 7fd85e8eb640 1 --2- 172.21.15.84:0/1702688194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8580ce390 0x7fd8581f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:54.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.915+0000 7fd85e8eb640 1 -- 172.21.15.84:0/1702688194 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd854009400 con 0x7fd85815ae80 2023-10-16T10:08:54.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.915+0000 7fd85e8eb640 1 --2- 172.21.15.84:0/1702688194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd85815ae80 0x7fd8580c8fe0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fd85000b9a0 tx=0x7fd85000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:54.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.915+0000 7fd84effd640 1 -- 172.21.15.84:0/1702688194 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000cba0 con 0x7fd85815ae80 2023-10-16T10:08:54.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.915+0000 7fd84effd640 1 -- 172.21.15.84:0/1702688194 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85000cd20 con 0x7fd85815ae80 2023-10-16T10:08:54.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.915+0000 7fd84effd640 1 -- 172.21.15.84:0/1702688194 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850014680 con 0x7fd85815ae80 2023-10-16T10:08:54.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.915+0000 7fd86034e640 1 -- 172.21.15.84:0/1702688194 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8580c9af0 con 0x7fd85815ae80 2023-10-16T10:08:54.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.915+0000 7fd86034e640 1 -- 172.21.15.84:0/1702688194 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8580c9520 con 0x7fd85815ae80 2023-10-16T10:08:54.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.916+0000 7fd86034e640 1 -- 172.21.15.84:0/1702688194 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85804e160 con 0x7fd85815ae80 2023-10-16T10:08:54.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.917+0000 7fd84effd640 1 -- 172.21.15.84:0/1702688194 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 79) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fd85001d3b0 con 0x7fd85815ae80 2023-10-16T10:08:54.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.918+0000 7fd84effd640 1 --2- 172.21.15.84:0/1702688194 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd83c06acc0 0x7fd83c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:54.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.918+0000 7fd84effd640 1 -- 172.21.15.84:0/1702688194 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6468+0+0 (secure 0 0 0) 0x7fd8500895a0 con 0x7fd85815ae80 2023-10-16T10:08:54.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.918+0000 7fd85e0ea640 1 --2- 172.21.15.84:0/1702688194 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd83c06acc0 0x7fd83c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:54.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.919+0000 7fd85e0ea640 1 --2- 172.21.15.84:0/1702688194 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd83c06acc0 0x7fd83c06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd854036000 tx=0x7fd854030000 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:54.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:54.922+0000 7fd84effd640 1 -- 172.21.15.84:0/1702688194 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd8500545c0 con 0x7fd85815ae80 2023-10-16T10:08:55.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.119+0000 7fd86034e640 1 -- 172.21.15.84:0/1702688194 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd85815f520 con 0x7fd85815ae80 2023-10-16T10:08:55.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.122+0000 7fd84effd640 1 -- 172.21.15.84:0/1702688194 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+17611 (secure 0 0 0) 0x7fd850019090 con 0x7fd85815ae80 2023-10-16T10:08:55.124 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:55.124 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":140,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:08:53.345815+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:08:23.558506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-16T10:08:24.709464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/3145965786":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/3955589838":"2023-10-17T10:08:53.345773+0000","172.21.15.84:6833/1203729023":"2023-10-17T10:08:53.345773+0000","172.21.15.84:6832/1203729023":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/1578467419":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/2605161255":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/3702054673":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/2708741108":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/914481139":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/4141270981":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/2356933958":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/4132563801":"2023-10-17T10:08:40.209237+0000","172.21.15.84:6833/2784642648":"2023-10-17T10:08:40.209237+0000","172.21.15.84:6832/2784642648":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/3016819584":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/2180503356":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/2933995314":"2023-10-17T10:08:40.209237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:08:55.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.124+0000 7fd86034e640 1 -- 172.21.15.84:0/1702688194 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd83c06acc0 msgr2=0x7fd83c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:55.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.124+0000 7fd86034e640 1 --2- 172.21.15.84:0/1702688194 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd83c06acc0 0x7fd83c06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd854036000 tx=0x7fd854030000 comp rx=0 tx=0).stop 2023-10-16T10:08:55.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.125+0000 7fd86034e640 1 -- 172.21.15.84:0/1702688194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd85815ae80 msgr2=0x7fd8580c8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:55.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.125+0000 7fd86034e640 1 --2- 172.21.15.84:0/1702688194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd85815ae80 0x7fd8580c8fe0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fd85000b9a0 tx=0x7fd85000be60 comp rx=0 tx=0).stop 2023-10-16T10:08:55.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.125+0000 7fd86034e640 1 -- 172.21.15.84:0/1702688194 shutdown_connections 2023-10-16T10:08:55.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.125+0000 7fd86034e640 1 --2- 172.21.15.84:0/1702688194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd85815ae80 0x7fd8580c8fe0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.125+0000 7fd86034e640 1 --2- 172.21.15.84:0/1702688194 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd83c06acc0 0x7fd83c06d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.125+0000 7fd86034e640 1 --2- 172.21.15.84:0/1702688194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd85815aaa0 0x7fd858159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.125+0000 7fd86034e640 1 --2- 172.21.15.84:0/1702688194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8580ce390 0x7fd8581f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.125+0000 7fd86034e640 1 -- 172.21.15.84:0/1702688194 >> 172.21.15.84:0/1702688194 conn(0x7fd8580cb410 msgr2=0x7fd85806fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:55.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.125+0000 7fd86034e640 1 -- 172.21.15.84:0/1702688194 shutdown_connections 2023-10-16T10:08:55.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.125+0000 7fd86034e640 1 -- 172.21.15.84:0/1702688194 wait complete. 2023-10-16T10:08:55.146 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T10:08:55.146 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:08:55.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.357+0000 7fa821ca2640 1 Processor -- start 2023-10-16T10:08:55.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.357+0000 7fa821ca2640 1 -- start start 2023-10-16T10:08:55.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.358+0000 7fa821ca2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa81c15aaa0 0x7fa81c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:55.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.358+0000 7fa821ca2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa81c15ae80 0x7fa81c0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:55.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.359+0000 7fa821ca2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81c15ed60 con 0x7fa81c0ccaa0 2023-10-16T10:08:55.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.359+0000 7fa821ca2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81c06b810 con 0x7fa81c15ae80 2023-10-16T10:08:55.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.359+0000 7fa821ca2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81c0707e0 con 0x7fa81c15aaa0 2023-10-16T10:08:55.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.359+0000 7fa81affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa81c15ae80 0x7fa81c0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:55.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.359+0000 7fa81bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa81c0ccaa0 0x7fa81c070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46950/0 (socket says 172.21.15.84:46950) 2023-10-16T10:08:55.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.359+0000 7fa81bfff640 1 -- 172.21.15.84:0/3610225030 learned_addr learned my addr 172.21.15.84:0/3610225030 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:55.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.359+0000 7fa81b7fe640 1 --2- 172.21.15.84:0/3610225030 >> v2:172.21.15.157:3300/0 conn(0x7fa81c15aaa0 0x7fa81c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:55.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.359+0000 7fa81affd640 1 -- 172.21.15.84:0/3610225030 >> v2:172.21.15.157:3300/0 conn(0x7fa81c15aaa0 msgr2=0x7fa81c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:55.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.359+0000 7fa81affd640 1 --2- 172.21.15.84:0/3610225030 >> v2:172.21.15.157:3300/0 conn(0x7fa81c15aaa0 0x7fa81c15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.359+0000 7fa81affd640 1 -- 172.21.15.84:0/3610225030 >> v1:172.21.15.84:6789/0 conn(0x7fa81c0ccaa0 legacy=0x7fa81c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:55.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.360+0000 7fa81affd640 1 -- 172.21.15.84:0/3610225030 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c15ef90 con 0x7fa81c15ae80 2023-10-16T10:08:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.360+0000 7fa81a7fc640 1 -- 172.21.15.84:0/3610225030 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3042979405 0 0) 0x7fa81c15ed60 con 0x7fa81c0ccaa0 2023-10-16T10:08:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.361+0000 7fa81affd640 1 --2- 172.21.15.84:0/3610225030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa81c15ae80 0x7fa81c0cc560 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0cd990 tx=0x7fa81002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a75d0394fb58e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.361+0000 7fa81a7fc640 1 -- 172.21.15.84:0/3610225030 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810031830 con 0x7fa81c15ae80 2023-10-16T10:08:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.361+0000 7fa81a7fc640 1 -- 172.21.15.84:0/3610225030 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa810031990 con 0x7fa81c15ae80 2023-10-16T10:08:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.361+0000 7fa81a7fc640 1 -- 172.21.15.84:0/3610225030 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810031d20 con 0x7fa81c15ae80 2023-10-16T10:08:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.362+0000 7fa821ca2640 1 -- 172.21.15.84:0/3610225030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa81c15ae80 msgr2=0x7fa81c0cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.362+0000 7fa821ca2640 1 --2- 172.21.15.84:0/3610225030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa81c15ae80 0x7fa81c0cc560 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0cd990 tx=0x7fa81002b8d0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.362+0000 7fa821ca2640 1 -- 172.21.15.84:0/3610225030 shutdown_connections 2023-10-16T10:08:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.362+0000 7fa821ca2640 1 --2- 172.21.15.84:0/3610225030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa81c15ae80 0x7fa81c0cc560 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.362+0000 7fa821ca2640 1 --2- 172.21.15.84:0/3610225030 >> v2:172.21.15.157:3300/0 conn(0x7fa81c15aaa0 0x7fa81c15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.362+0000 7fa821ca2640 1 -- 172.21.15.84:0/3610225030 >> 172.21.15.84:0/3610225030 conn(0x7fa81c0cb410 msgr2=0x7fa81c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:55.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.362+0000 7fa821ca2640 1 -- 172.21.15.84:0/3610225030 shutdown_connections 2023-10-16T10:08:55.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.362+0000 7fa821ca2640 1 -- 172.21.15.84:0/3610225030 wait complete. 2023-10-16T10:08:55.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.364+0000 7fa821ca2640 1 Processor -- start 2023-10-16T10:08:55.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.364+0000 7fa821ca2640 1 -- start start 2023-10-16T10:08:55.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.365+0000 7fa821ca2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa81c0ccaa0 0x7fa81c15cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:55.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.365+0000 7fa821ca2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa81c15aaa0 0x7fa81c149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:55.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.365+0000 7fa821ca2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa81c15ae80 0x7fa81c14c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:55.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.365+0000 7fa821ca2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c15e420 con 0x7fa81c0ccaa0 2023-10-16T10:08:55.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.365+0000 7fa821ca2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c1496a0 con 0x7fa81c15aaa0 2023-10-16T10:08:55.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.365+0000 7fa821ca2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c14c8f0 con 0x7fa81c15ae80 2023-10-16T10:08:55.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.365+0000 7fa81b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa81c0ccaa0 0x7fa81c15cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:55.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.365+0000 7fa81b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa81c0ccaa0 0x7fa81c15cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53462/0 (socket says 172.21.15.84:53462) 2023-10-16T10:08:55.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.365+0000 7fa81b7fe640 1 -- 172.21.15.84:0/3603617055 learned_addr learned my addr 172.21.15.84:0/3603617055 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:55.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.366+0000 7fa81b7fe640 1 -- 172.21.15.84:0/3603617055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa81c15ae80 msgr2=0x7fa81c14c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:55.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.366+0000 7fa81b7fe640 1 --2- 172.21.15.84:0/3603617055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa81c15ae80 0x7fa81c14c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.366+0000 7fa81b7fe640 1 -- 172.21.15.84:0/3603617055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa81c15aaa0 msgr2=0x7fa81c149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:55.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.366+0000 7fa81b7fe640 1 --2- 172.21.15.84:0/3603617055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa81c15aaa0 0x7fa81c149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.366+0000 7fa81b7fe640 1 -- 172.21.15.84:0/3603617055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa810009400 con 0x7fa81c0ccaa0 2023-10-16T10:08:55.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.366+0000 7fa81b7fe640 1 --2- 172.21.15.84:0/3603617055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa81c0ccaa0 0x7fa81c15cd10 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00d830 tx=0x7fa80c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:55.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.366+0000 7fa803fff640 1 -- 172.21.15.84:0/3603617055 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c00b660 con 0x7fa81c0ccaa0 2023-10-16T10:08:55.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.366+0000 7fa803fff640 1 -- 172.21.15.84:0/3603617055 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80c00b7c0 con 0x7fa81c0ccaa0 2023-10-16T10:08:55.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.366+0000 7fa821ca2640 1 -- 172.21.15.84:0/3603617055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa81c15d380 con 0x7fa81c0ccaa0 2023-10-16T10:08:55.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.366+0000 7fa803fff640 1 -- 172.21.15.84:0/3603617055 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c00bb50 con 0x7fa81c0ccaa0 2023-10-16T10:08:55.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.366+0000 7fa821ca2640 1 -- 172.21.15.84:0/3603617055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81c15d940 con 0x7fa81c0ccaa0 2023-10-16T10:08:55.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.367+0000 7fa821ca2640 1 -- 172.21.15.84:0/3603617055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa81c04e160 con 0x7fa81c0ccaa0 2023-10-16T10:08:55.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.368+0000 7fa803fff640 1 -- 172.21.15.84:0/3603617055 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 79) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fa80c01e460 con 0x7fa81c0ccaa0 2023-10-16T10:08:55.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.369+0000 7fa803fff640 1 --2- 172.21.15.84:0/3603617055 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fa80806acc0 0x7fa80806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:55.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.369+0000 7fa803fff640 1 -- 172.21.15.84:0/3603617055 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6468+0+0 (secure 0 0 0) 0x7fa80c089d20 con 0x7fa81c0ccaa0 2023-10-16T10:08:55.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.370+0000 7fa81affd640 1 --2- 172.21.15.84:0/3603617055 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fa80806acc0 0x7fa80806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:55.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.370+0000 7fa81affd640 1 --2- 172.21.15.84:0/3603617055 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fa80806acc0 0x7fa80806d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa810036000 tx=0x7fa810030000 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:55.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.374+0000 7fa803fff640 1 -- 172.21.15.84:0/3603617055 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa80c054d40 con 0x7fa81c0ccaa0 2023-10-16T10:08:55.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.611+0000 7fa821ca2640 1 -- 172.21.15.84:0/3603617055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa81c0cdc60 con 0x7fa81c0ccaa0 2023-10-16T10:08:55.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.612+0000 7fa803fff640 1 -- 172.21.15.84:0/3603617055 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fa80c058360 con 0x7fa81c0ccaa0 2023-10-16T10:08:55.614 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:55.614 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4118285132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":4118285132},{"type":"v1","addr":"172.21.15.84:6837","nonce":4118285132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3743702675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3743702675},{"type":"v1","addr":"172.21.15.157:6834","nonce":3743702675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2895418246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2895418246},{"type":"v1","addr":"172.21.15.157:6835","nonce":2895418246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:08:25.705299+0000","modified":"2023-10-16T10:08:28.762900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8574":{"gid":8574,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/791907708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":791907708},{"type":"v1","addr":"172.21.15.84:6836","nonce":791907708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-16T10:08:55.614 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 71 2023-10-16T10:08:55.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.614+0000 7fa821ca2640 1 -- 172.21.15.84:0/3603617055 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fa80806acc0 msgr2=0x7fa80806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:55.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.614+0000 7fa821ca2640 1 --2- 172.21.15.84:0/3603617055 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fa80806acc0 0x7fa80806d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa810036000 tx=0x7fa810030000 comp rx=0 tx=0).stop 2023-10-16T10:08:55.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.614+0000 7fa821ca2640 1 -- 172.21.15.84:0/3603617055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa81c0ccaa0 msgr2=0x7fa81c15cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:55.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.614+0000 7fa821ca2640 1 --2- 172.21.15.84:0/3603617055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa81c0ccaa0 0x7fa81c15cd10 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00d830 tx=0x7fa80c00dcf0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.614+0000 7fa821ca2640 1 -- 172.21.15.84:0/3603617055 shutdown_connections 2023-10-16T10:08:55.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.614+0000 7fa821ca2640 1 --2- 172.21.15.84:0/3603617055 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fa80806acc0 0x7fa80806d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.614+0000 7fa821ca2640 1 --2- 172.21.15.84:0/3603617055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa81c15ae80 0x7fa81c14c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.614+0000 7fa821ca2640 1 --2- 172.21.15.84:0/3603617055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa81c15aaa0 0x7fa81c149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.614+0000 7fa821ca2640 1 --2- 172.21.15.84:0/3603617055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa81c0ccaa0 0x7fa81c15cd10 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.614+0000 7fa821ca2640 1 -- 172.21.15.84:0/3603617055 >> 172.21.15.84:0/3603617055 conn(0x7fa81c0cb410 msgr2=0x7fa81c0b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:55.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.614+0000 7fa821ca2640 1 -- 172.21.15.84:0/3603617055 shutdown_connections 2023-10-16T10:08:55.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.614+0000 7fa821ca2640 1 -- 172.21.15.84:0/3603617055 wait complete. 2023-10-16T10:08:55.636 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:08:55.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.848+0000 7f9821c5d640 1 Processor -- start 2023-10-16T10:08:55.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.848+0000 7f9821c5d640 1 -- start start 2023-10-16T10:08:55.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.850+0000 7f9821c5d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f981c15ae80 0x7f981c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:55.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.850+0000 7f9821c5d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f981c0ce390 0x7f981c070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:55.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.850+0000 7f9821c5d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981c15ed60 con 0x7f981c15aaa0 2023-10-16T10:08:55.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.850+0000 7f9821c5d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981c06b810 con 0x7f981c0ce390 2023-10-16T10:08:55.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.850+0000 7f9821c5d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0707e0 con 0x7f981c15ae80 2023-10-16T10:08:55.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.851+0000 7f9812ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f981c15ae80 0x7f981c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:55.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.851+0000 7f981bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f981c0ce390 0x7f981c070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:55.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.851+0000 7f981b7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f981c15aaa0 0x7f981c0cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:57220/0 (socket says 172.21.15.84:57220) 2023-10-16T10:08:55.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.851+0000 7f9812ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f981c15ae80 0x7f981c15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:39560/0 (socket says 172.21.15.84:39560) 2023-10-16T10:08:55.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.851+0000 7f981b7fe640 1 -- 172.21.15.84:0/2094165084 learned_addr learned my addr 172.21.15.84:0/2094165084 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:55.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.851+0000 7f981affd640 1 -- 172.21.15.84:0/2094165084 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176908344 0 0) 0x7f981c15ed60 con 0x7f981c15aaa0 2023-10-16T10:08:55.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.851+0000 7f981affd640 1 -- 172.21.15.84:0/2094165084 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9808003520 con 0x7f981c15aaa0 2023-10-16T10:08:55.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.851+0000 7f9812ffd640 1 -- 172.21.15.84:0/2094165084 >> v2:172.21.15.157:3300/0 conn(0x7f981c0ce390 msgr2=0x7f981c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:55.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.852+0000 7f9812ffd640 1 --2- 172.21.15.84:0/2094165084 >> v2:172.21.15.157:3300/0 conn(0x7f981c0ce390 0x7f981c070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.852+0000 7f9812ffd640 1 -- 172.21.15.84:0/2094165084 >> v1:172.21.15.157:6789/0 conn(0x7f981c15aaa0 legacy=0x7f981c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:55.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.852+0000 7f9812ffd640 1 -- 172.21.15.84:0/2094165084 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c15ef90 con 0x7f981c15ae80 2023-10-16T10:08:55.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.852+0000 7f981bfff640 1 --2- 172.21.15.84:0/2094165084 >> v2:172.21.15.157:3300/0 conn(0x7f981c0ce390 0x7f981c070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:08:55.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.853+0000 7f9812ffd640 1 --2- 172.21.15.84:0/2094165084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f981c15ae80 0x7f981c15e820 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f98040097a0 tx=0x7f980402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=988bc2f01d299164 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:55.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.853+0000 7f981affd640 1 -- 172.21.15.84:0/2094165084 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980402d040 con 0x7f981c15ae80 2023-10-16T10:08:55.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.853+0000 7f981affd640 1 -- 172.21.15.84:0/2094165084 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9804031810 con 0x7f981c15ae80 2023-10-16T10:08:55.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.853+0000 7f981affd640 1 -- 172.21.15.84:0/2094165084 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804031b80 con 0x7f981c15ae80 2023-10-16T10:08:55.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.854+0000 7f9821c5d640 1 -- 172.21.15.84:0/2094165084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f981c15ae80 msgr2=0x7f981c15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:55.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.854+0000 7f9821c5d640 1 --2- 172.21.15.84:0/2094165084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f981c15ae80 0x7f981c15e820 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f98040097a0 tx=0x7f980402b8b0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.854+0000 7f9821c5d640 1 -- 172.21.15.84:0/2094165084 shutdown_connections 2023-10-16T10:08:55.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.854+0000 7f9821c5d640 1 --2- 172.21.15.84:0/2094165084 >> v2:172.21.15.157:3300/0 conn(0x7f981c0ce390 0x7f981c070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.854+0000 7f9821c5d640 1 --2- 172.21.15.84:0/2094165084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f981c15ae80 0x7f981c15e820 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.854+0000 7f9821c5d640 1 -- 172.21.15.84:0/2094165084 >> 172.21.15.84:0/2094165084 conn(0x7f981c0cb410 msgr2=0x7f981c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:55.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.854+0000 7f9821c5d640 1 -- 172.21.15.84:0/2094165084 shutdown_connections 2023-10-16T10:08:55.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.854+0000 7f9821c5d640 1 -- 172.21.15.84:0/2094165084 wait complete. 2023-10-16T10:08:55.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.856+0000 7f9821c5d640 1 Processor -- start 2023-10-16T10:08:55.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.857+0000 7f9821c5d640 1 -- start start 2023-10-16T10:08:55.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.858+0000 7f9821c5d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f981c0ce390 0x7f981c0c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:55.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.858+0000 7f9821c5d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f981c15aaa0 0x7f981c1f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:55.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.858+0000 7f9821c5d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f981c15ae80 0x7f981c1f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:55.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.858+0000 7f9821c5d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0b0130 con 0x7f981c15ae80 2023-10-16T10:08:55.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.858+0000 7f9821c5d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0af4b0 con 0x7f981c15aaa0 2023-10-16T10:08:55.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.858+0000 7f9821c5d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0af790 con 0x7f981c0ce390 2023-10-16T10:08:55.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.858+0000 7f981bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f981c15ae80 0x7f981c1f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:55.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.858+0000 7f981bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f981c15ae80 0x7f981c1f1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51598/0 (socket says 172.21.15.84:51598) 2023-10-16T10:08:55.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.858+0000 7f981bfff640 1 -- 172.21.15.84:0/2491147443 learned_addr learned my addr 172.21.15.84:0/2491147443 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:55.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.858+0000 7f981b7fe640 1 --2- 172.21.15.84:0/2491147443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f981c0ce390 0x7f981c0c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:55.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.858+0000 7f9812ffd640 1 --2- 172.21.15.84:0/2491147443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f981c15aaa0 0x7f981c1f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:55.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.859+0000 7f981bfff640 1 -- 172.21.15.84:0/2491147443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f981c0ce390 msgr2=0x7f981c0c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:55.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.859+0000 7f981bfff640 1 --2- 172.21.15.84:0/2491147443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f981c0ce390 0x7f981c0c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.859+0000 7f981bfff640 1 -- 172.21.15.84:0/2491147443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f981c15aaa0 msgr2=0x7f981c1f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:55.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.859+0000 7f981bfff640 1 --2- 172.21.15.84:0/2491147443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f981c15aaa0 0x7f981c1f17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:55.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.859+0000 7f981bfff640 1 -- 172.21.15.84:0/2491147443 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9814009400 con 0x7f981c15ae80 2023-10-16T10:08:55.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.859+0000 7f981b7fe640 1 --2- 172.21.15.84:0/2491147443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f981c0ce390 0x7f981c0c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:55.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.859+0000 7f981bfff640 1 --2- 172.21.15.84:0/2491147443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f981c15ae80 0x7f981c1f1d10 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f981400e880 tx=0x7f981400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:55.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.859+0000 7f9818ff9640 1 -- 172.21.15.84:0/2491147443 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9814009660 con 0x7f981c15ae80 2023-10-16T10:08:55.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.859+0000 7f9818ff9640 1 -- 172.21.15.84:0/2491147443 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9814004020 con 0x7f981c15ae80 2023-10-16T10:08:55.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.859+0000 7f9818ff9640 1 -- 172.21.15.84:0/2491147443 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98140043b0 con 0x7f981c15ae80 2023-10-16T10:08:55.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.859+0000 7f9821c5d640 1 -- 172.21.15.84:0/2491147443 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9804009400 con 0x7f981c15ae80 2023-10-16T10:08:55.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.859+0000 7f9821c5d640 1 -- 172.21.15.84:0/2491147443 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981c0afd50 con 0x7f981c15ae80 2023-10-16T10:08:55.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.861+0000 7f9821c5d640 1 -- 172.21.15.84:0/2491147443 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981c0b0270 con 0x7f981c15ae80 2023-10-16T10:08:55.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.862+0000 7f9818ff9640 1 -- 172.21.15.84:0/2491147443 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 79) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f981400c940 con 0x7f981c15ae80 2023-10-16T10:08:55.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.862+0000 7f9818ff9640 1 --2- 172.21.15.84:0/2491147443 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f97fc06acc0 0x7f97fc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:55.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.863+0000 7f9818ff9640 1 -- 172.21.15.84:0/2491147443 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f9814089360 con 0x7f981c15ae80 2023-10-16T10:08:55.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.863+0000 7f981b7fe640 1 --2- 172.21.15.84:0/2491147443 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f97fc06acc0 0x7f97fc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:55.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.864+0000 7f981b7fe640 1 --2- 172.21.15.84:0/2491147443 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f97fc06acc0 0x7f97fc06d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f980c005c00 tx=0x7f980c009040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:55.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:55.869+0000 7f9818ff9640 1 -- 172.21.15.84:0/2491147443 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f981c0b0270 con 0x7f981c15ae80 2023-10-16T10:08:56.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.071+0000 7f9818ff9640 1 -- 172.21.15.84:0/2491147443 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 80) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f9814054820 con 0x7f981c15ae80 2023-10-16T10:08:56.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.106+0000 7f9821c5d640 1 -- 172.21.15.84:0/2491147443 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f981c0b0270 con 0x7f981c15ae80 2023-10-16T10:08:56.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.106+0000 7f9818ff9640 1 -- 172.21.15.84:0/2491147443 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f981c0b0270 con 0x7f981c15ae80 2023-10-16T10:08:56.108 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:56.109 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4118285132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":4118285132},{"type":"v1","addr":"172.21.15.84:6837","nonce":4118285132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3743702675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3743702675},{"type":"v1","addr":"172.21.15.157:6834","nonce":3743702675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2895418246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2895418246},{"type":"v1","addr":"172.21.15.157:6835","nonce":2895418246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:08:25.705299+0000","modified":"2023-10-16T10:08:28.762900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8574":{"gid":8574,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/791907708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":791907708},{"type":"v1","addr":"172.21.15.84:6836","nonce":791907708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-16T10:08:56.109 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 71 2023-10-16T10:08:56.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.109+0000 7f9821c5d640 1 -- 172.21.15.84:0/2491147443 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f97fc06acc0 msgr2=0x7f97fc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:56.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.109+0000 7f9821c5d640 1 --2- 172.21.15.84:0/2491147443 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f97fc06acc0 0x7f97fc06d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f980c005c00 tx=0x7f980c009040 comp rx=0 tx=0).stop 2023-10-16T10:08:56.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.109+0000 7f9821c5d640 1 -- 172.21.15.84:0/2491147443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f981c15ae80 msgr2=0x7f981c1f1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:56.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.109+0000 7f9821c5d640 1 --2- 172.21.15.84:0/2491147443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f981c15ae80 0x7f981c1f1d10 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f981400e880 tx=0x7f981400ed40 comp rx=0 tx=0).stop 2023-10-16T10:08:56.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.109+0000 7f9821c5d640 1 -- 172.21.15.84:0/2491147443 shutdown_connections 2023-10-16T10:08:56.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.109+0000 7f9821c5d640 1 --2- 172.21.15.84:0/2491147443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f981c15ae80 0x7f981c1f1d10 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:56.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.109+0000 7f9821c5d640 1 --2- 172.21.15.84:0/2491147443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f981c15aaa0 0x7f981c1f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:56.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.109+0000 7f9821c5d640 1 --2- 172.21.15.84:0/2491147443 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f97fc06acc0 0x7f97fc06d180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:56.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.109+0000 7f9821c5d640 1 --2- 172.21.15.84:0/2491147443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f981c0ce390 0x7f981c0c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:56.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.109+0000 7f9821c5d640 1 -- 172.21.15.84:0/2491147443 >> 172.21.15.84:0/2491147443 conn(0x7f981c0cb410 msgr2=0x7f981c0b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:56.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.109+0000 7f9821c5d640 1 -- 172.21.15.84:0/2491147443 shutdown_connections 2023-10-16T10:08:56.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.109+0000 7f9821c5d640 1 -- 172.21.15.84:0/2491147443 wait complete. 2023-10-16T10:08:56.128 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:08:56.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.335+0000 7f70b3bb0640 1 Processor -- start 2023-10-16T10:08:56.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.336+0000 7f70b3bb0640 1 -- start start 2023-10-16T10:08:56.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.337+0000 7f70b3bb0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f70ac160a40 0x7f70ac0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:56.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.337+0000 7f70b3bb0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f70ac15d080 0x7f70ac0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:56.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.337+0000 7f70b3bb0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70ac0cd060 con 0x7f70ac06c800 2023-10-16T10:08:56.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.337+0000 7f70b3bb0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70ac0cdbe0 con 0x7f70ac160a40 2023-10-16T10:08:56.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.337+0000 7f70b3bb0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70ac0cd2b0 con 0x7f70ac15d080 2023-10-16T10:08:56.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.338+0000 7f70b194c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f70ac06c800 0x7f70ac15c970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34142/0 (socket says 172.21.15.84:34142) 2023-10-16T10:08:56.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.338+0000 7f70b194c640 1 -- 172.21.15.84:0/307204107 learned_addr learned my addr 172.21.15.84:0/307204107 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:56.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.338+0000 7f70b114b640 1 --2- 172.21.15.84:0/307204107 >> v2:172.21.15.84:3300/0 conn(0x7f70ac160a40 0x7f70ac0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:56.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.338+0000 7f70b094a640 1 -- 172.21.15.84:0/307204107 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311621704 0 0) 0x7f70ac0cd060 con 0x7f70ac06c800 2023-10-16T10:08:56.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.338+0000 7f70b094a640 1 -- 172.21.15.84:0/307204107 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7094003520 con 0x7f70ac06c800 2023-10-16T10:08:56.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.338+0000 7f70b214d640 1 --2- 172.21.15.84:0/307204107 >> v2:172.21.15.157:3300/0 conn(0x7f70ac15d080 0x7f70ac0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:56.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.338+0000 7f70b114b640 1 -- 172.21.15.84:0/307204107 >> v2:172.21.15.157:3300/0 conn(0x7f70ac15d080 msgr2=0x7f70ac0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:56.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.338+0000 7f70b114b640 1 --2- 172.21.15.84:0/307204107 >> v2:172.21.15.157:3300/0 conn(0x7f70ac15d080 0x7f70ac0cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:56.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.338+0000 7f70b114b640 1 -- 172.21.15.84:0/307204107 >> v1:172.21.15.84:6789/0 conn(0x7f70ac06c800 legacy=0x7f70ac15c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:56.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.338+0000 7f70b114b640 1 -- 172.21.15.84:0/307204107 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ac0cd590 con 0x7f70ac160a40 2023-10-16T10:08:56.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.339+0000 7f70b114b640 1 --2- 172.21.15.84:0/307204107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70ac160a40 0x7f70ac0cc130 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f70ac161ac0 tx=0x7f70a002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9ae8f860024524 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:56.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.340+0000 7f70b094a640 1 -- 172.21.15.84:0/307204107 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0031830 con 0x7f70ac160a40 2023-10-16T10:08:56.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.340+0000 7f70b094a640 1 -- 172.21.15.84:0/307204107 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a0031990 con 0x7f70ac160a40 2023-10-16T10:08:56.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.340+0000 7f70b094a640 1 -- 172.21.15.84:0/307204107 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0031d00 con 0x7f70ac160a40 2023-10-16T10:08:56.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.340+0000 7f70b3bb0640 1 -- 172.21.15.84:0/307204107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70ac160a40 msgr2=0x7f70ac0cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:56.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.340+0000 7f70b3bb0640 1 --2- 172.21.15.84:0/307204107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70ac160a40 0x7f70ac0cc130 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f70ac161ac0 tx=0x7f70a002b8d0 comp rx=0 tx=0).stop 2023-10-16T10:08:56.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.341+0000 7f70b3bb0640 1 -- 172.21.15.84:0/307204107 shutdown_connections 2023-10-16T10:08:56.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.341+0000 7f70b3bb0640 1 --2- 172.21.15.84:0/307204107 >> v2:172.21.15.157:3300/0 conn(0x7f70ac15d080 0x7f70ac0cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:56.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.341+0000 7f70b3bb0640 1 --2- 172.21.15.84:0/307204107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70ac160a40 0x7f70ac0cc130 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:56.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.341+0000 7f70b3bb0640 1 -- 172.21.15.84:0/307204107 >> 172.21.15.84:0/307204107 conn(0x7f70ac06d700 msgr2=0x7f70ac06db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:56.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.341+0000 7f70b3bb0640 1 -- 172.21.15.84:0/307204107 shutdown_connections 2023-10-16T10:08:56.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.341+0000 7f70b3bb0640 1 -- 172.21.15.84:0/307204107 wait complete. 2023-10-16T10:08:56.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.342+0000 7f70b3bb0640 1 Processor -- start 2023-10-16T10:08:56.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.342+0000 7f70b3bb0640 1 -- start start 2023-10-16T10:08:56.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.343+0000 7f70b3bb0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70ac06c800 0x7f70ac0aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:56.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.343+0000 7f70b3bb0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70ac15d080 0x7f70ac0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:56.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.343+0000 7f70b3bb0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70ac160a40 0x7f70ac155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:56.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.343+0000 7f70b3bb0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac0af7f0 con 0x7f70ac160a40 2023-10-16T10:08:56.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.343+0000 7f70b3bb0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac0acdd0 con 0x7f70ac06c800 2023-10-16T10:08:56.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.343+0000 7f70b3bb0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac1560b0 con 0x7f70ac15d080 2023-10-16T10:08:56.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.344+0000 7f70b214d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70ac160a40 0x7f70ac155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:56.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.344+0000 7f70b214d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70ac160a40 0x7f70ac155b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51610/0 (socket says 172.21.15.84:51610) 2023-10-16T10:08:56.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.344+0000 7f70b214d640 1 -- 172.21.15.84:0/915908820 learned_addr learned my addr 172.21.15.84:0/915908820 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:56.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.344+0000 7f70b194c640 1 --2- 172.21.15.84:0/915908820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70ac06c800 0x7f70ac0aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:56.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.344+0000 7f70b214d640 1 -- 172.21.15.84:0/915908820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70ac15d080 msgr2=0x7f70ac0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:56.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.344+0000 7f70b114b640 1 --2- 172.21.15.84:0/915908820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70ac15d080 0x7f70ac0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:56.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.344+0000 7f70b214d640 1 --2- 172.21.15.84:0/915908820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70ac15d080 0x7f70ac0ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:56.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.344+0000 7f70b214d640 1 -- 172.21.15.84:0/915908820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70ac06c800 msgr2=0x7f70ac0aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:56.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.344+0000 7f70b214d640 1 --2- 172.21.15.84:0/915908820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70ac06c800 0x7f70ac0aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:56.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.344+0000 7f70b214d640 1 -- 172.21.15.84:0/915908820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a0009400 con 0x7f70ac160a40 2023-10-16T10:08:56.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.344+0000 7f70b194c640 1 --2- 172.21.15.84:0/915908820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70ac06c800 0x7f70ac0aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:56.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.344+0000 7f70b214d640 1 --2- 172.21.15.84:0/915908820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70ac160a40 0x7f70ac155b70 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f70a400d5d0 tx=0x7f70a400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:56.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.345+0000 7f70aa7fc640 1 -- 172.21.15.84:0/915908820 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a40125f0 con 0x7f70ac160a40 2023-10-16T10:08:56.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.345+0000 7f70aa7fc640 1 -- 172.21.15.84:0/915908820 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a4012750 con 0x7f70ac160a40 2023-10-16T10:08:56.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.345+0000 7f70b3bb0640 1 -- 172.21.15.84:0/915908820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ac156570 con 0x7f70ac160a40 2023-10-16T10:08:56.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.345+0000 7f70aa7fc640 1 -- 172.21.15.84:0/915908820 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a4012b00 con 0x7f70ac160a40 2023-10-16T10:08:56.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.345+0000 7f70b3bb0640 1 -- 172.21.15.84:0/915908820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ac0af510 con 0x7f70ac160a40 2023-10-16T10:08:56.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.346+0000 7f70b3bb0640 1 -- 172.21.15.84:0/915908820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ac06fce0 con 0x7f70ac160a40 2023-10-16T10:08:56.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.347+0000 7f70aa7fc640 1 -- 172.21.15.84:0/915908820 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 80) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f70a4012c60 con 0x7f70ac160a40 2023-10-16T10:08:56.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.348+0000 7f70aa7fc640 1 --2- 172.21.15.84:0/915908820 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f709806af00 0x7f709806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:56.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.348+0000 7f70aa7fc640 1 -- 172.21.15.84:0/915908820 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f70a4089ea0 con 0x7f70ac160a40 2023-10-16T10:08:56.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.348+0000 7f70b194c640 1 --2- 172.21.15.84:0/915908820 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f709806af00 0x7f709806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:56.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.349+0000 7f70b194c640 1 --2- 172.21.15.84:0/915908820 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f709806af00 0x7f709806d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f709c005c00 tx=0x7f709c009040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:56.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.352+0000 7f70aa7fc640 1 -- 172.21.15.84:0/915908820 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f70a4054b60 con 0x7f70ac160a40 2023-10-16T10:08:56.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.543+0000 7f70b3bb0640 1 -- 172.21.15.84:0/915908820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70ac04e160 con 0x7f70ac160a40 2023-10-16T10:08:56.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.545+0000 7f70aa7fc640 1 -- 172.21.15.84:0/915908820 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+17611 (secure 0 0 0) 0x7f70a4058180 con 0x7f70ac160a40 2023-10-16T10:08:56.547 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:08:56.547 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":140,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:08:53.345815+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:08:23.558506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-16T10:08:24.709464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/3145965786":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/3955589838":"2023-10-17T10:08:53.345773+0000","172.21.15.84:6833/1203729023":"2023-10-17T10:08:53.345773+0000","172.21.15.84:6832/1203729023":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/1578467419":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/2605161255":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/3702054673":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/2708741108":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/914481139":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/4141270981":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/2356933958":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/4132563801":"2023-10-17T10:08:40.209237+0000","172.21.15.84:6833/2784642648":"2023-10-17T10:08:40.209237+0000","172.21.15.84:6832/2784642648":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/3016819584":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/2180503356":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/2933995314":"2023-10-17T10:08:40.209237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:08:56.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.548+0000 7f70b3bb0640 1 -- 172.21.15.84:0/915908820 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f709806af00 msgr2=0x7f709806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:56.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.548+0000 7f70b3bb0640 1 --2- 172.21.15.84:0/915908820 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f709806af00 0x7f709806d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f709c005c00 tx=0x7f709c009040 comp rx=0 tx=0).stop 2023-10-16T10:08:56.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.548+0000 7f70b3bb0640 1 -- 172.21.15.84:0/915908820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70ac160a40 msgr2=0x7f70ac155b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:56.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.548+0000 7f70b3bb0640 1 --2- 172.21.15.84:0/915908820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70ac160a40 0x7f70ac155b70 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f70a400d5d0 tx=0x7f70a400da90 comp rx=0 tx=0).stop 2023-10-16T10:08:56.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.548+0000 7f70b3bb0640 1 -- 172.21.15.84:0/915908820 shutdown_connections 2023-10-16T10:08:56.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.548+0000 7f70b3bb0640 1 --2- 172.21.15.84:0/915908820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70ac160a40 0x7f70ac155b70 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:56.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.548+0000 7f70b3bb0640 1 --2- 172.21.15.84:0/915908820 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f709806af00 0x7f709806d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:56.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.548+0000 7f70b3bb0640 1 --2- 172.21.15.84:0/915908820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70ac15d080 0x7f70ac0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:56.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.548+0000 7f70b3bb0640 1 --2- 172.21.15.84:0/915908820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70ac06c800 0x7f70ac0aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:56.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.548+0000 7f70b3bb0640 1 -- 172.21.15.84:0/915908820 >> 172.21.15.84:0/915908820 conn(0x7f70ac06d700 msgr2=0x7f70ac06ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:56.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.549+0000 7f70b3bb0640 1 -- 172.21.15.84:0/915908820 shutdown_connections 2023-10-16T10:08:56.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.549+0000 7f70b3bb0640 1 -- 172.21.15.84:0/915908820 wait complete. 2023-10-16T10:08:56.570 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T10:08:56.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.781+0000 7f784bc02640 1 Processor -- start 2023-10-16T10:08:56.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.782+0000 7f784bc02640 1 -- start start 2023-10-16T10:08:56.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.783+0000 7f784bc02640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f78440b3420 0x7f78440b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:56.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.783+0000 7f784bc02640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78440d3980 con 0x7f78440cf710 2023-10-16T10:08:56.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.783+0000 7f784bc02640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78440ce630 con 0x7f78440cfaf0 2023-10-16T10:08:56.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.783+0000 7f784bc02640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78440d3c60 con 0x7f78440b3420 2023-10-16T10:08:56.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.784+0000 7f784999e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f78440cf710 0x7f78440b2d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34154/0 (socket says 172.21.15.84:34154) 2023-10-16T10:08:56.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.784+0000 7f784999e640 1 -- 172.21.15.84:0/2742125524 learned_addr learned my addr 172.21.15.84:0/2742125524 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:08:56.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.784+0000 7f784919d640 1 --2- 172.21.15.84:0/2742125524 >> v2:172.21.15.84:3300/0 conn(0x7f78440b3420 0x7f78440b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:56.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.784+0000 7f784919d640 1 -- 172.21.15.84:0/2742125524 >> v1:172.21.15.157:6789/0 conn(0x7f78440cfaf0 legacy=0x7f78440b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:56.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.784+0000 7f784919d640 1 -- 172.21.15.84:0/2742125524 >> v1:172.21.15.84:6789/0 conn(0x7f78440cf710 legacy=0x7f78440b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:56.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.784+0000 7f784919d640 1 -- 172.21.15.84:0/2742125524 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78440ce910 con 0x7f78440b3420 2023-10-16T10:08:56.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.784+0000 7f784899c640 1 -- 172.21.15.84:0/2742125524 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600443140 0 0) 0x7f78440d3980 con 0x7f78440cf710 2023-10-16T10:08:56.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.785+0000 7f784919d640 1 --2- 172.21.15.84:0/2742125524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78440b3420 0x7f78440b5cc0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f78380097a0 tx=0x7f783802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1910125e7b8d0d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:56.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.785+0000 7f784899c640 1 -- 172.21.15.84:0/2742125524 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783802e070 con 0x7f78440b3420 2023-10-16T10:08:56.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.786+0000 7f784899c640 1 -- 172.21.15.84:0/2742125524 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78380315a0 con 0x7f78440b3420 2023-10-16T10:08:56.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.786+0000 7f784899c640 1 -- 172.21.15.84:0/2742125524 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7838031930 con 0x7f78440b3420 2023-10-16T10:08:56.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.786+0000 7f784bc02640 1 -- 172.21.15.84:0/2742125524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78440b3420 msgr2=0x7f78440b5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:56.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.786+0000 7f784bc02640 1 --2- 172.21.15.84:0/2742125524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78440b3420 0x7f78440b5cc0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f78380097a0 tx=0x7f783802b840 comp rx=0 tx=0).stop 2023-10-16T10:08:56.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.786+0000 7f784bc02640 1 -- 172.21.15.84:0/2742125524 shutdown_connections 2023-10-16T10:08:56.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.786+0000 7f784bc02640 1 --2- 172.21.15.84:0/2742125524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78440b3420 0x7f78440b5cc0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:56.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.786+0000 7f784bc02640 1 -- 172.21.15.84:0/2742125524 >> 172.21.15.84:0/2742125524 conn(0x7f78440d2cc0 msgr2=0x7f78440d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:56.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.786+0000 7f784bc02640 1 -- 172.21.15.84:0/2742125524 shutdown_connections 2023-10-16T10:08:56.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.786+0000 7f784bc02640 1 -- 172.21.15.84:0/2742125524 wait complete. 2023-10-16T10:08:56.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.788+0000 7f784bc02640 1 Processor -- start 2023-10-16T10:08:56.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.788+0000 7f784bc02640 1 -- start start 2023-10-16T10:08:56.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.789+0000 7f784bc02640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78440b3420 0x7f78441501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:56.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.789+0000 7f784bc02640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78440cf710 0x7f78440b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:56.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.789+0000 7f784bc02640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78440cfaf0 0x7f78440b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:56.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.789+0000 7f784bc02640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78441506e0 con 0x7f78440cfaf0 2023-10-16T10:08:56.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.789+0000 7f784bc02640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784409fe70 con 0x7f78440b3420 2023-10-16T10:08:56.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.789+0000 7f784bc02640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78440a0150 con 0x7f78440cf710 2023-10-16T10:08:56.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.789+0000 7f784a19f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78440cfaf0 0x7f78440b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:56.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.789+0000 7f784a19f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78440cfaf0 0x7f78440b7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51628/0 (socket says 172.21.15.84:51628) 2023-10-16T10:08:56.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.789+0000 7f784a19f640 1 -- 172.21.15.84:0/2230855470 learned_addr learned my addr 172.21.15.84:0/2230855470 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:56.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.790+0000 7f784a19f640 1 -- 172.21.15.84:0/2230855470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78440cf710 msgr2=0x7f78440b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:56.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.790+0000 7f784a19f640 1 --2- 172.21.15.84:0/2230855470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78440cf710 0x7f78440b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:56.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.790+0000 7f784a19f640 1 -- 172.21.15.84:0/2230855470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78440b3420 msgr2=0x7f78441501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:56.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.790+0000 7f784a19f640 1 --2- 172.21.15.84:0/2230855470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78440b3420 0x7f78441501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:56.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.790+0000 7f784a19f640 1 -- 172.21.15.84:0/2230855470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7838009400 con 0x7f78440cfaf0 2023-10-16T10:08:56.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.790+0000 7f784a19f640 1 --2- 172.21.15.84:0/2230855470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78440cfaf0 0x7f78440b7d20 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f783c00b410 tx=0x7f783c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:56.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.790+0000 7f78427fc640 1 -- 172.21.15.84:0/2230855470 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c004020 con 0x7f78440cfaf0 2023-10-16T10:08:56.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.790+0000 7f78427fc640 1 -- 172.21.15.84:0/2230855470 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783c004180 con 0x7f78440cfaf0 2023-10-16T10:08:56.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.790+0000 7f784bc02640 1 -- 172.21.15.84:0/2230855470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78440a0430 con 0x7f78440cfaf0 2023-10-16T10:08:56.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.790+0000 7f78427fc640 1 -- 172.21.15.84:0/2230855470 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c011630 con 0x7f78440cfaf0 2023-10-16T10:08:56.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.791+0000 7f784bc02640 1 -- 172.21.15.84:0/2230855470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784409c9a0 con 0x7f78440cfaf0 2023-10-16T10:08:56.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.792+0000 7f784bc02640 1 -- 172.21.15.84:0/2230855470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784404e160 con 0x7f78440cfaf0 2023-10-16T10:08:56.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.792+0000 7f78427fc640 1 -- 172.21.15.84:0/2230855470 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 80) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f783c011790 con 0x7f78440cfaf0 2023-10-16T10:08:56.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.793+0000 7f78427fc640 1 --2- 172.21.15.84:0/2230855470 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f783006af30 0x7f783006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:56.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.794+0000 7f78427fc640 1 -- 172.21.15.84:0/2230855470 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f783c0896f0 con 0x7f78440cfaf0 2023-10-16T10:08:56.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.794+0000 7f784999e640 1 --2- 172.21.15.84:0/2230855470 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f783006af30 0x7f783006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:56.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.794+0000 7f784999e640 1 --2- 172.21.15.84:0/2230855470 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f783006af30 0x7f783006d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7834001460 tx=0x7f7834007b40 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:56.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:56.798+0000 7f78427fc640 1 -- 172.21.15.84:0/2230855470 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f783c0543b0 con 0x7f78440cfaf0 2023-10-16T10:08:57.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.027+0000 7f784bc02640 1 -- 172.21.15.84:0/2230855470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f78440ccff0 con 0x7f78440cfaf0 2023-10-16T10:08:57.030 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-16T10:08:57.028+0000 7fca0c7ff640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T10:08:57.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.032+0000 7f78427fc640 1 -- 172.21.15.84:0/2230855470 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) v1 ==== 161+0+0 (secure 0 0 0) 0x7f783c0579d0 con 0x7f78440cfaf0 2023-10-16T10:08:57.034 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T10:08:57.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.036+0000 7f784bc02640 1 -- 172.21.15.84:0/2230855470 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f783006af30 msgr2=0x7f783006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:57.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.036+0000 7f784bc02640 1 --2- 172.21.15.84:0/2230855470 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f783006af30 0x7f783006d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7834001460 tx=0x7f7834007b40 comp rx=0 tx=0).stop 2023-10-16T10:08:57.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.036+0000 7f784bc02640 1 -- 172.21.15.84:0/2230855470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78440cfaf0 msgr2=0x7f78440b7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:57.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.036+0000 7f784bc02640 1 --2- 172.21.15.84:0/2230855470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78440cfaf0 0x7f78440b7d20 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f783c00b410 tx=0x7f783c00b8d0 comp rx=0 tx=0).stop 2023-10-16T10:08:57.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.036+0000 7f784bc02640 1 -- 172.21.15.84:0/2230855470 shutdown_connections 2023-10-16T10:08:57.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.036+0000 7f784bc02640 1 --2- 172.21.15.84:0/2230855470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78440cfaf0 0x7f78440b7d20 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:57.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.036+0000 7f784bc02640 1 --2- 172.21.15.84:0/2230855470 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f783006af30 0x7f783006d3f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:57.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.036+0000 7f784bc02640 1 --2- 172.21.15.84:0/2230855470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78440cf710 0x7f78440b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:57.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.036+0000 7f784bc02640 1 --2- 172.21.15.84:0/2230855470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78440b3420 0x7f78441501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:57.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.036+0000 7f784bc02640 1 -- 172.21.15.84:0/2230855470 >> 172.21.15.84:0/2230855470 conn(0x7f78440d2cc0 msgr2=0x7f78440d1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:57.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.036+0000 7f784bc02640 1 -- 172.21.15.84:0/2230855470 shutdown_connections 2023-10-16T10:08:57.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.036+0000 7f784bc02640 1 -- 172.21.15.84:0/2230855470 wait complete. 2023-10-16T10:08:57.059 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T10:08:57.068 INFO:tasks.ceph.mds.a.smithi084.stderr: -8531> 2023-10-16T10:08:31.169+0000 7fc32dd55640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:08:57.136 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T10:08:57.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.253+0000 7f07a8a40640 1 Processor -- start 2023-10-16T10:08:57.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.254+0000 7f07a8a40640 1 -- start start 2023-10-16T10:08:57.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.255+0000 7f07a8a40640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a406c040 0x7f07a406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:57.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.255+0000 7f07a8a40640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f07a40ad970 0x7f07a415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:57.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.255+0000 7f07a8a40640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f07a415ebc0 0x7f07a415f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:57.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.255+0000 7f07a8a40640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a406c960 con 0x7f07a40ad970 2023-10-16T10:08:57.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.255+0000 7f07a8a40640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a415b9d0 con 0x7f07a415ebc0 2023-10-16T10:08:57.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.255+0000 7f07a8a40640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a415bcb0 con 0x7f07a406c040 2023-10-16T10:08:57.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.255+0000 7f079bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f07a40ad970 0x7f07a415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:57.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.256+0000 7f079bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f07a40ad970 0x7f07a415e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51636/0 (socket says 172.21.15.84:51636) 2023-10-16T10:08:57.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.256+0000 7f079bfff640 1 -- 172.21.15.84:0/2245908762 learned_addr learned my addr 172.21.15.84:0/2245908762 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:57.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.256+0000 7f07a37fe640 1 --2- 172.21.15.84:0/2245908762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a406c040 0x7f07a406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:57.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.256+0000 7f07a3fff640 1 --2- 172.21.15.84:0/2245908762 >> v2:172.21.15.157:3300/0 conn(0x7f07a415ebc0 0x7f07a415f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:57.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.256+0000 7f079bfff640 1 -- 172.21.15.84:0/2245908762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a406c040 msgr2=0x7f07a406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:57.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.256+0000 7f079bfff640 1 --2- 172.21.15.84:0/2245908762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a406c040 0x7f07a406c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:57.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.256+0000 7f079bfff640 1 -- 172.21.15.84:0/2245908762 >> v2:172.21.15.157:3300/0 conn(0x7f07a415ebc0 msgr2=0x7f07a415f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:57.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.256+0000 7f079bfff640 1 --2- 172.21.15.84:0/2245908762 >> v2:172.21.15.157:3300/0 conn(0x7f07a415ebc0 0x7f07a415f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:57.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.256+0000 7f079bfff640 1 -- 172.21.15.84:0/2245908762 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a415bf90 con 0x7f07a40ad970 2023-10-16T10:08:57.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.256+0000 7f07a3fff640 1 --2- 172.21.15.84:0/2245908762 >> v2:172.21.15.157:3300/0 conn(0x7f07a415ebc0 0x7f07a415f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:57.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.257+0000 7f079bfff640 1 --2- 172.21.15.84:0/2245908762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07a40ad970 0x7f07a415e680 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f078c009870 tx=0x7f078c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59598b9526513bf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:57.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.258+0000 7f07a2ffd640 1 -- 172.21.15.84:0/2245908762 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c031640 con 0x7f07a40ad970 2023-10-16T10:08:57.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.258+0000 7f07a2ffd640 1 -- 172.21.15.84:0/2245908762 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c0317a0 con 0x7f07a40ad970 2023-10-16T10:08:57.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.258+0000 7f07a2ffd640 1 -- 172.21.15.84:0/2245908762 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c031b30 con 0x7f07a40ad970 2023-10-16T10:08:57.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.258+0000 7f07a8a40640 1 -- 172.21.15.84:0/2245908762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07a40ad970 msgr2=0x7f07a415e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:57.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.258+0000 7f07a8a40640 1 --2- 172.21.15.84:0/2245908762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07a40ad970 0x7f07a415e680 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f078c009870 tx=0x7f078c02b8f0 comp rx=0 tx=0).stop 2023-10-16T10:08:57.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.258+0000 7f07a8a40640 1 -- 172.21.15.84:0/2245908762 shutdown_connections 2023-10-16T10:08:57.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.258+0000 7f07a8a40640 1 --2- 172.21.15.84:0/2245908762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07a40ad970 0x7f07a415e680 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:57.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.258+0000 7f07a8a40640 1 --2- 172.21.15.84:0/2245908762 >> v2:172.21.15.157:3300/0 conn(0x7f07a415ebc0 0x7f07a415f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:57.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.258+0000 7f07a8a40640 1 --2- 172.21.15.84:0/2245908762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a406c040 0x7f07a406c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:57.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.258+0000 7f07a8a40640 1 -- 172.21.15.84:0/2245908762 >> 172.21.15.84:0/2245908762 conn(0x7f07a40ce6a0 msgr2=0x7f07a40ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:57.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.258+0000 7f07a8a40640 1 -- 172.21.15.84:0/2245908762 shutdown_connections 2023-10-16T10:08:57.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.258+0000 7f07a8a40640 1 -- 172.21.15.84:0/2245908762 wait complete. 2023-10-16T10:08:57.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.260+0000 7f07a8a40640 1 Processor -- start 2023-10-16T10:08:57.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.260+0000 7f07a8a40640 1 -- start start 2023-10-16T10:08:57.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.261+0000 7f07a8a40640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a406c040 0x7f07a40c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:57.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.261+0000 7f07a8a40640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07a40ad970 0x7f07a4152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:57.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.261+0000 7f07a8a40640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a415ebc0 0x7f07a40b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:57.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.261+0000 7f07a8a40640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a40c83b0 con 0x7f07a40ad970 2023-10-16T10:08:57.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.261+0000 7f07a8a40640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a41581b0 con 0x7f07a415ebc0 2023-10-16T10:08:57.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.261+0000 7f07a8a40640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a4153250 con 0x7f07a406c040 2023-10-16T10:08:57.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.262+0000 7f079bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07a40ad970 0x7f07a4152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:57.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.262+0000 7f079bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07a40ad970 0x7f07a4152d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51652/0 (socket says 172.21.15.84:51652) 2023-10-16T10:08:57.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.262+0000 7f079bfff640 1 -- 172.21.15.84:0/901942390 learned_addr learned my addr 172.21.15.84:0/901942390 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:57.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.262+0000 7f079bfff640 1 -- 172.21.15.84:0/901942390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a406c040 msgr2=0x7f07a40c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:57.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.262+0000 7f079bfff640 1 --2- 172.21.15.84:0/901942390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a406c040 0x7f07a40c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:57.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.262+0000 7f079bfff640 1 -- 172.21.15.84:0/901942390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a415ebc0 msgr2=0x7f07a40b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:57.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.262+0000 7f079bfff640 1 --2- 172.21.15.84:0/901942390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a415ebc0 0x7f07a40b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:57.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.262+0000 7f079bfff640 1 -- 172.21.15.84:0/901942390 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c0094d0 con 0x7f07a40ad970 2023-10-16T10:08:57.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.262+0000 7f079bfff640 1 --2- 172.21.15.84:0/901942390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07a40ad970 0x7f07a4152d10 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f078c000c00 tx=0x7f078c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:57.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.263+0000 7f07a0ff9640 1 -- 172.21.15.84:0/901942390 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c03ac60 con 0x7f07a40ad970 2023-10-16T10:08:57.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.263+0000 7f07a0ff9640 1 -- 172.21.15.84:0/901942390 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c03ade0 con 0x7f07a40ad970 2023-10-16T10:08:57.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.263+0000 7f07a8a40640 1 -- 172.21.15.84:0/901942390 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a40aff10 con 0x7f07a40ad970 2023-10-16T10:08:57.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.263+0000 7f07a0ff9640 1 -- 172.21.15.84:0/901942390 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c044630 con 0x7f07a40ad970 2023-10-16T10:08:57.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.263+0000 7f07a8a40640 1 -- 172.21.15.84:0/901942390 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a40c80d0 con 0x7f07a40ad970 2023-10-16T10:08:57.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.264+0000 7f07a8a40640 1 -- 172.21.15.84:0/901942390 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a406b330 con 0x7f07a40ad970 2023-10-16T10:08:57.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.265+0000 7f07a0ff9640 1 -- 172.21.15.84:0/901942390 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 80) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f078c02f050 con 0x7f07a40ad970 2023-10-16T10:08:57.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.266+0000 7f07a0ff9640 1 --2- 172.21.15.84:0/901942390 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f078406af00 0x7f078406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:57.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.266+0000 7f07a0ff9640 1 -- 172.21.15.84:0/901942390 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6554+0+0 (secure 0 0 0) 0x7f078c0abbb0 con 0x7f07a40ad970 2023-10-16T10:08:57.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.266+0000 7f07a37fe640 1 --2- 172.21.15.84:0/901942390 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f078406af00 0x7f078406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:57.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.267+0000 7f07a37fe640 1 --2- 172.21.15.84:0/901942390 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f078406af00 0x7f078406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f07a40b1190 tx=0x7f0794008040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:57.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.270+0000 7f07a0ff9640 1 -- 172.21.15.84:0/901942390 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f078c0b2080 con 0x7f07a40ad970 2023-10-16T10:08:57.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:57.501+0000 7f07a8a40640 1 -- 172.21.15.84:0/901942390 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f07a40b00a0 con 0x7f07a40ad970 2023-10-16T10:08:58.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.068+0000 7f07a0ff9640 1 -- 172.21.15.84:0/901942390 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7f078c076810 con 0x7f07a40ad970 2023-10-16T10:08:58.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.071+0000 7f07a8a40640 1 -- 172.21.15.84:0/901942390 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f078406af00 msgr2=0x7f078406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:58.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.071+0000 7f07a8a40640 1 --2- 172.21.15.84:0/901942390 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f078406af00 0x7f078406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f07a40b1190 tx=0x7f0794008040 comp rx=0 tx=0).stop 2023-10-16T10:08:58.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.071+0000 7f07a8a40640 1 -- 172.21.15.84:0/901942390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07a40ad970 msgr2=0x7f07a4152d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:58.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.071+0000 7f07a8a40640 1 --2- 172.21.15.84:0/901942390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07a40ad970 0x7f07a4152d10 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f078c000c00 tx=0x7f078c0046c0 comp rx=0 tx=0).stop 2023-10-16T10:08:58.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.071+0000 7f07a8a40640 1 -- 172.21.15.84:0/901942390 shutdown_connections 2023-10-16T10:08:58.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.071+0000 7f07a8a40640 1 --2- 172.21.15.84:0/901942390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a415ebc0 0x7f07a40b03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:58.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.071+0000 7f07a8a40640 1 --2- 172.21.15.84:0/901942390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07a40ad970 0x7f07a4152d10 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:58.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.071+0000 7f07a8a40640 1 --2- 172.21.15.84:0/901942390 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f078406af00 0x7f078406d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:58.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.071+0000 7f07a8a40640 1 --2- 172.21.15.84:0/901942390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a406c040 0x7f07a40c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:58.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.071+0000 7f07a8a40640 1 -- 172.21.15.84:0/901942390 >> 172.21.15.84:0/901942390 conn(0x7f07a40ce6a0 msgr2=0x7f07a4069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:58.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.072+0000 7f07a8a40640 1 -- 172.21.15.84:0/901942390 shutdown_connections 2023-10-16T10:08:58.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.072+0000 7f07a8a40640 1 -- 172.21.15.84:0/901942390 wait complete. 2023-10-16T10:08:58.098 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T10:08:58.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.362+0000 7fda9791d640 1 Processor -- start 2023-10-16T10:08:58.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.363+0000 7fda9791d640 1 -- start start 2023-10-16T10:08:58.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.364+0000 7fda9791d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fda900c8fa0 0x7fda90165aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:58.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.364+0000 7fda9791d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda900b0360 0x7fda900b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:58.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.364+0000 7fda9791d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda90165610 con 0x7fda900c1d30 2023-10-16T10:08:58.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.364+0000 7fda9791d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda900b0cb0 con 0x7fda900c8fa0 2023-10-16T10:08:58.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.364+0000 7fda9791d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda900ce370 con 0x7fda900b0360 2023-10-16T10:08:58.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.365+0000 7fda95eba640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda900b0360 0x7fda900b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:58.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.365+0000 7fda95eba640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda900b0360 0x7fda900b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:39650/0 (socket says 172.21.15.84:39650) 2023-10-16T10:08:58.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.365+0000 7fda95eba640 1 -- 172.21.15.84:0/2576001167 learned_addr learned my addr 172.21.15.84:0/2576001167 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:58.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.365+0000 7fda94eb8640 1 --2- 172.21.15.84:0/2576001167 >> v2:172.21.15.157:3300/0 conn(0x7fda900c8fa0 0x7fda90165aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:58.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.366+0000 7fda95eba640 1 -- 172.21.15.84:0/2576001167 >> v2:172.21.15.157:3300/0 conn(0x7fda900c8fa0 msgr2=0x7fda90165aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:58.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.366+0000 7fda95eba640 1 --2- 172.21.15.84:0/2576001167 >> v2:172.21.15.157:3300/0 conn(0x7fda900c8fa0 0x7fda90165aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:58.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.366+0000 7fda95eba640 1 -- 172.21.15.84:0/2576001167 >> v1:172.21.15.157:6789/0 conn(0x7fda900c1d30 legacy=0x7fda900cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:58.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.366+0000 7fda95eba640 1 -- 172.21.15.84:0/2576001167 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda900ce650 con 0x7fda900b0360 2023-10-16T10:08:58.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.366+0000 7fda83fff640 1 -- 172.21.15.84:0/2576001167 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882966424 0 0) 0x7fda90165610 con 0x7fda900c1d30 2023-10-16T10:08:58.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.368+0000 7fda95eba640 1 --2- 172.21.15.84:0/2576001167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda900b0360 0x7fda900b0770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fda88009e50 tx=0x7fda8802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=203d08de8d2f102f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:58.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.368+0000 7fda83fff640 1 -- 172.21.15.84:0/2576001167 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8802cb00 con 0x7fda900b0360 2023-10-16T10:08:58.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.368+0000 7fda83fff640 1 -- 172.21.15.84:0/2576001167 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8802cc60 con 0x7fda900b0360 2023-10-16T10:08:58.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.368+0000 7fda83fff640 1 -- 172.21.15.84:0/2576001167 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda880343f0 con 0x7fda900b0360 2023-10-16T10:08:58.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.369+0000 7fda9791d640 1 -- 172.21.15.84:0/2576001167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda900b0360 msgr2=0x7fda900b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:58.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.369+0000 7fda9791d640 1 --2- 172.21.15.84:0/2576001167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda900b0360 0x7fda900b0770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fda88009e50 tx=0x7fda8802bf60 comp rx=0 tx=0).stop 2023-10-16T10:08:58.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.369+0000 7fda9791d640 1 -- 172.21.15.84:0/2576001167 shutdown_connections 2023-10-16T10:08:58.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.369+0000 7fda9791d640 1 --2- 172.21.15.84:0/2576001167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda900b0360 0x7fda900b0770 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:58.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.369+0000 7fda9791d640 1 --2- 172.21.15.84:0/2576001167 >> v2:172.21.15.157:3300/0 conn(0x7fda900c8fa0 0x7fda90165aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:58.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.369+0000 7fda9791d640 1 -- 172.21.15.84:0/2576001167 >> 172.21.15.84:0/2576001167 conn(0x7fda900cb360 msgr2=0x7fda900cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:58.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.369+0000 7fda9791d640 1 -- 172.21.15.84:0/2576001167 shutdown_connections 2023-10-16T10:08:58.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.369+0000 7fda9791d640 1 -- 172.21.15.84:0/2576001167 wait complete. 2023-10-16T10:08:58.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.371+0000 7fda9791d640 1 Processor -- start 2023-10-16T10:08:58.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.371+0000 7fda9791d640 1 -- start start 2023-10-16T10:08:58.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.371+0000 7fda9791d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda900b0360 0x7fda90156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:58.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.371+0000 7fda9791d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda900c1d30 0x7fda90163cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:58.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.372+0000 7fda9791d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda900c8fa0 0x7fda901ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:58.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.372+0000 7fda9791d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda90163100 con 0x7fda900b0360 2023-10-16T10:08:58.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.372+0000 7fda9791d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda90156650 con 0x7fda900c1d30 2023-10-16T10:08:58.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.372+0000 7fda9791d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda901624c0 con 0x7fda900c8fa0 2023-10-16T10:08:58.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.372+0000 7fda956b9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda900b0360 0x7fda90156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:58.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.372+0000 7fda956b9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda900b0360 0x7fda90156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51666/0 (socket says 172.21.15.84:51666) 2023-10-16T10:08:58.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.372+0000 7fda956b9640 1 -- 172.21.15.84:0/250204498 learned_addr learned my addr 172.21.15.84:0/250204498 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:58.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.372+0000 7fda94eb8640 1 --2- 172.21.15.84:0/250204498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda900c1d30 0x7fda90163cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:58.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.372+0000 7fda95eba640 1 --2- 172.21.15.84:0/250204498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda900c8fa0 0x7fda901ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:58.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.372+0000 7fda956b9640 1 -- 172.21.15.84:0/250204498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda900c8fa0 msgr2=0x7fda901ebe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:58.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.372+0000 7fda956b9640 1 --2- 172.21.15.84:0/250204498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda900c8fa0 0x7fda901ebe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:58.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.372+0000 7fda956b9640 1 -- 172.21.15.84:0/250204498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda900c1d30 msgr2=0x7fda90163cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:58.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.372+0000 7fda956b9640 1 --2- 172.21.15.84:0/250204498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda900c1d30 0x7fda90163cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:58.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.372+0000 7fda956b9640 1 -- 172.21.15.84:0/250204498 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda88009b30 con 0x7fda900b0360 2023-10-16T10:08:58.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.372+0000 7fda94eb8640 1 --2- 172.21.15.84:0/250204498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda900c1d30 0x7fda90163cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:58.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.373+0000 7fda956b9640 1 --2- 172.21.15.84:0/250204498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda900b0360 0x7fda90156110 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00c410 tx=0x7fda8c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:58.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.373+0000 7fda81ffb640 1 -- 172.21.15.84:0/250204498 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c0035f0 con 0x7fda900b0360 2023-10-16T10:08:58.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.373+0000 7fda81ffb640 1 -- 172.21.15.84:0/250204498 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8c003770 con 0x7fda900b0360 2023-10-16T10:08:58.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.373+0000 7fda81ffb640 1 -- 172.21.15.84:0/250204498 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c003b20 con 0x7fda900b0360 2023-10-16T10:08:58.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.373+0000 7fda9791d640 1 -- 172.21.15.84:0/250204498 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda90162760 con 0x7fda900b0360 2023-10-16T10:08:58.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.373+0000 7fda9791d640 1 -- 172.21.15.84:0/250204498 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda90162d20 con 0x7fda900b0360 2023-10-16T10:08:58.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.374+0000 7fda9791d640 1 -- 172.21.15.84:0/250204498 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda900ce0f0 con 0x7fda900b0360 2023-10-16T10:08:58.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.375+0000 7fda81ffb640 1 -- 172.21.15.84:0/250204498 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fda8c003c80 con 0x7fda900b0360 2023-10-16T10:08:58.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.376+0000 7fda81ffb640 1 --2- 172.21.15.84:0/250204498 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fda7406af90 0x7fda7406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:58.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.376+0000 7fda81ffb640 1 -- 172.21.15.84:0/250204498 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6554+0+0 (secure 0 0 0) 0x7fda8c089070 con 0x7fda900b0360 2023-10-16T10:08:58.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.376+0000 7fda94eb8640 1 --2- 172.21.15.84:0/250204498 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fda7406af90 0x7fda7406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:58.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.377+0000 7fda94eb8640 1 --2- 172.21.15.84:0/250204498 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fda7406af90 0x7fda7406d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fda9015ec10 tx=0x7fda84008040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:58.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.381+0000 7fda81ffb640 1 -- 172.21.15.84:0/250204498 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fda8c053d50 con 0x7fda900b0360 2023-10-16T10:08:58.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:58.558+0000 7fda9791d640 1 -- 172.21.15.84:0/250204498 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fda9004e160 con 0x7fda900b0360 2023-10-16T10:08:59.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.091+0000 7fda81ffb640 1 -- 172.21.15.84:0/250204498 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v142) v1 ==== 181+0+0 (secure 0 0 0) 0x7fda8c0170d0 con 0x7fda900b0360 2023-10-16T10:08:59.094 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-16T10:08:59.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.096+0000 7fda9791d640 1 -- 172.21.15.84:0/250204498 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fda7406af90 msgr2=0x7fda7406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:59.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.096+0000 7fda9791d640 1 --2- 172.21.15.84:0/250204498 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fda7406af90 0x7fda7406d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fda9015ec10 tx=0x7fda84008040 comp rx=0 tx=0).stop 2023-10-16T10:08:59.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.096+0000 7fda9791d640 1 -- 172.21.15.84:0/250204498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda900b0360 msgr2=0x7fda90156110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:59.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.096+0000 7fda9791d640 1 --2- 172.21.15.84:0/250204498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda900b0360 0x7fda90156110 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00c410 tx=0x7fda8c00c8d0 comp rx=0 tx=0).stop 2023-10-16T10:08:59.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.096+0000 7fda9791d640 1 -- 172.21.15.84:0/250204498 shutdown_connections 2023-10-16T10:08:59.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.096+0000 7fda9791d640 1 --2- 172.21.15.84:0/250204498 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fda7406af90 0x7fda7406d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:59.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.096+0000 7fda9791d640 1 --2- 172.21.15.84:0/250204498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda900c8fa0 0x7fda901ebe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:59.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.096+0000 7fda9791d640 1 --2- 172.21.15.84:0/250204498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda900c1d30 0x7fda90163cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:59.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.096+0000 7fda9791d640 1 --2- 172.21.15.84:0/250204498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda900b0360 0x7fda90156110 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:59.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.096+0000 7fda9791d640 1 -- 172.21.15.84:0/250204498 >> 172.21.15.84:0/250204498 conn(0x7fda900cb360 msgr2=0x7fda900aedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:59.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.096+0000 7fda9791d640 1 -- 172.21.15.84:0/250204498 shutdown_connections 2023-10-16T10:08:59.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.097+0000 7fda9791d640 1 -- 172.21.15.84:0/250204498 wait complete. 2023-10-16T10:08:59.124 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T10:08:59.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.325+0000 7fe9fe5cb640 1 Processor -- start 2023-10-16T10:08:59.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.325+0000 7fe9fe5cb640 1 -- start start 2023-10-16T10:08:59.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.326+0000 7fe9fe5cb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe9f8159950 0x7fe9f815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:59.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.326+0000 7fe9fe5cb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9f806c810 0x7fe9f80708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:59.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.327+0000 7fe9fe5cb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f815aef0 con 0x7fe9f8070450 2023-10-16T10:08:59.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.327+0000 7fe9fe5cb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f8070e20 con 0x7fe9f8159950 2023-10-16T10:08:59.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.327+0000 7fe9fe5cb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f815e490 con 0x7fe9f806c810 2023-10-16T10:08:59.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.327+0000 7fe9fd5c9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe9f8159950 0x7fe9f815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:59.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.327+0000 7fe9fd5c9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe9f8159950 0x7fe9f815a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51680/0 (socket says 172.21.15.84:51680) 2023-10-16T10:08:59.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.327+0000 7fe9fd5c9640 1 -- 172.21.15.84:0/2492015296 learned_addr learned my addr 172.21.15.84:0/2492015296 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:59.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.327+0000 7fe9fcdc8640 1 --1- 172.21.15.84:0/2492015296 >> v1:172.21.15.84:6789/0 conn(0x7fe9f8070450 0x7fe9f806c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34178/0 (socket says 172.21.15.84:34178) 2023-10-16T10:08:59.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.327+0000 7fe9fddca640 1 --2- 172.21.15.84:0/2492015296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9f806c810 0x7fe9f80708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:59.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.327+0000 7fe9fd5c9640 1 -- 172.21.15.84:0/2492015296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9f806c810 msgr2=0x7fe9f80708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:08:59.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.327+0000 7fe9fd5c9640 1 --2- 172.21.15.84:0/2492015296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9f806c810 0x7fe9f80708e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:59.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.327+0000 7fe9fd5c9640 1 -- 172.21.15.84:0/2492015296 >> v1:172.21.15.84:6789/0 conn(0x7fe9f8070450 legacy=0x7fe9f806c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:59.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.327+0000 7fe9fd5c9640 1 -- 172.21.15.84:0/2492015296 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f806cc60 con 0x7fe9f8159950 2023-10-16T10:08:59.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.328+0000 7fe9fd5c9640 1 --2- 172.21.15.84:0/2492015296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9f8159950 0x7fe9f815a9b0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009870 tx=0x7fe9f402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c2a3ee2e28d4a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:59.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.329+0000 7fe9effff640 1 -- 172.21.15.84:0/2492015296 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4031640 con 0x7fe9f8159950 2023-10-16T10:08:59.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.329+0000 7fe9effff640 1 -- 172.21.15.84:0/2492015296 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f40317a0 con 0x7fe9f8159950 2023-10-16T10:08:59.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.329+0000 7fe9effff640 1 -- 172.21.15.84:0/2492015296 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4031b30 con 0x7fe9f8159950 2023-10-16T10:08:59.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.329+0000 7fe9fe5cb640 1 -- 172.21.15.84:0/2492015296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9f8159950 msgr2=0x7fe9f815a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:59.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.329+0000 7fe9fe5cb640 1 --2- 172.21.15.84:0/2492015296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9f8159950 0x7fe9f815a9b0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009870 tx=0x7fe9f402b8f0 comp rx=0 tx=0).stop 2023-10-16T10:08:59.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.329+0000 7fe9fe5cb640 1 -- 172.21.15.84:0/2492015296 shutdown_connections 2023-10-16T10:08:59.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.329+0000 7fe9fe5cb640 1 --2- 172.21.15.84:0/2492015296 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9f806c810 0x7fe9f80708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:59.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.329+0000 7fe9fe5cb640 1 --2- 172.21.15.84:0/2492015296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9f8159950 0x7fe9f815a9b0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:59.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.329+0000 7fe9fe5cb640 1 -- 172.21.15.84:0/2492015296 >> 172.21.15.84:0/2492015296 conn(0x7fe9f80ce6a0 msgr2=0x7fe9f80ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:08:59.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.330+0000 7fe9fe5cb640 1 -- 172.21.15.84:0/2492015296 shutdown_connections 2023-10-16T10:08:59.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.330+0000 7fe9fe5cb640 1 -- 172.21.15.84:0/2492015296 wait complete. 2023-10-16T10:08:59.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.331+0000 7fe9fe5cb640 1 Processor -- start 2023-10-16T10:08:59.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.331+0000 7fe9fe5cb640 1 -- start start 2023-10-16T10:08:59.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.332+0000 7fe9fe5cb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9f806c810 0x7fe9f815b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:59.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.332+0000 7fe9fe5cb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9f8070450 0x7fe9f813b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:59.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.332+0000 7fe9fe5cb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9f8159950 0x7fe9f815bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:59.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.332+0000 7fe9fe5cb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f815bb30 con 0x7fe9f8070450 2023-10-16T10:08:59.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.332+0000 7fe9fe5cb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f8155d20 con 0x7fe9f806c810 2023-10-16T10:08:59.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.332+0000 7fe9fe5cb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f81529b0 con 0x7fe9f8159950 2023-10-16T10:08:59.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.333+0000 7fe9fcdc8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9f8070450 0x7fe9f813b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:59.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.333+0000 7fe9fcdc8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9f8070450 0x7fe9f813b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51694/0 (socket says 172.21.15.84:51694) 2023-10-16T10:08:59.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.333+0000 7fe9fcdc8640 1 -- 172.21.15.84:0/772651398 learned_addr learned my addr 172.21.15.84:0/772651398 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:08:59.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.333+0000 7fe9fddca640 1 --2- 172.21.15.84:0/772651398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9f8159950 0x7fe9f815bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:59.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.333+0000 7fe9fd5c9640 1 --2- 172.21.15.84:0/772651398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9f806c810 0x7fe9f815b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:59.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.333+0000 7fe9fcdc8640 1 -- 172.21.15.84:0/772651398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9f8159950 msgr2=0x7fe9f815bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:59.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.333+0000 7fe9fcdc8640 1 --2- 172.21.15.84:0/772651398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9f8159950 0x7fe9f815bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:59.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.333+0000 7fe9fcdc8640 1 -- 172.21.15.84:0/772651398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9f806c810 msgr2=0x7fe9f815b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:08:59.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.333+0000 7fe9fcdc8640 1 --2- 172.21.15.84:0/772651398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9f806c810 0x7fe9f815b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:08:59.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.333+0000 7fe9fcdc8640 1 -- 172.21.15.84:0/772651398 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f40094d0 con 0x7fe9f8070450 2023-10-16T10:08:59.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.333+0000 7fe9fddca640 1 --2- 172.21.15.84:0/772651398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9f8159950 0x7fe9f815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:08:59.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.333+0000 7fe9fcdc8640 1 --2- 172.21.15.84:0/772651398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9f8070450 0x7fe9f813b910 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800c4d0 tx=0x7fe9e800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:59.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.334+0000 7fe9edffb640 1 -- 172.21.15.84:0/772651398 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8007480 con 0x7fe9f8070450 2023-10-16T10:08:59.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.334+0000 7fe9edffb640 1 -- 172.21.15.84:0/772651398 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e80075e0 con 0x7fe9f8070450 2023-10-16T10:08:59.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.334+0000 7fe9edffb640 1 -- 172.21.15.84:0/772651398 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e80049e0 con 0x7fe9f8070450 2023-10-16T10:08:59.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.334+0000 7fe9fe5cb640 1 -- 172.21.15.84:0/772651398 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f8152c90 con 0x7fe9f8070450 2023-10-16T10:08:59.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.334+0000 7fe9fe5cb640 1 -- 172.21.15.84:0/772651398 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f815b800 con 0x7fe9f8070450 2023-10-16T10:08:59.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.335+0000 7fe9fe5cb640 1 -- 172.21.15.84:0/772651398 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f815bce0 con 0x7fe9f8070450 2023-10-16T10:08:59.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.336+0000 7fe9edffb640 1 -- 172.21.15.84:0/772651398 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe9e8004b40 con 0x7fe9f8070450 2023-10-16T10:08:59.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.337+0000 7fe9edffb640 1 --2- 172.21.15.84:0/772651398 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fe9dc06af90 0x7fe9dc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:08:59.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.337+0000 7fe9edffb640 1 -- 172.21.15.84:0/772651398 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fe9e8088870 con 0x7fe9f8070450 2023-10-16T10:08:59.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.337+0000 7fe9fd5c9640 1 --2- 172.21.15.84:0/772651398 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fe9dc06af90 0x7fe9dc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:08:59.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.338+0000 7fe9fd5c9640 1 --2- 172.21.15.84:0/772651398 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fe9dc06af90 0x7fe9dc06d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4036000 tx=0x7fe9f403c040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:08:59.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.341+0000 7fe9edffb640 1 -- 172.21.15.84:0/772651398 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe9f815bce0 con 0x7fe9f8070450 2023-10-16T10:08:59.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:08:59.535+0000 7fe9fe5cb640 1 -- 172.21.15.84:0/772651398 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe9f815aef0 con 0x7fe9f8070450 2023-10-16T10:09:00.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.102+0000 7fe9edffb640 1 -- 172.21.15.84:0/772651398 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v143) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe9f815aef0 con 0x7fe9f8070450 2023-10-16T10:09:00.104 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-16T10:09:00.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.106+0000 7fe9fe5cb640 1 -- 172.21.15.84:0/772651398 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fe9dc06af90 msgr2=0x7fe9dc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:00.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.106+0000 7fe9fe5cb640 1 --2- 172.21.15.84:0/772651398 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fe9dc06af90 0x7fe9dc06d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4036000 tx=0x7fe9f403c040 comp rx=0 tx=0).stop 2023-10-16T10:09:00.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.107+0000 7fe9fe5cb640 1 -- 172.21.15.84:0/772651398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9f8070450 msgr2=0x7fe9f813b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:00.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.107+0000 7fe9fe5cb640 1 --2- 172.21.15.84:0/772651398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9f8070450 0x7fe9f813b910 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800c4d0 tx=0x7fe9e800c990 comp rx=0 tx=0).stop 2023-10-16T10:09:00.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.108+0000 7fe9fe5cb640 1 -- 172.21.15.84:0/772651398 shutdown_connections 2023-10-16T10:09:00.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.108+0000 7fe9fe5cb640 1 --2- 172.21.15.84:0/772651398 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fe9dc06af90 0x7fe9dc06d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.108+0000 7fe9fe5cb640 1 --2- 172.21.15.84:0/772651398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9f8159950 0x7fe9f815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.108+0000 7fe9fe5cb640 1 --2- 172.21.15.84:0/772651398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9f8070450 0x7fe9f813b910 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.108+0000 7fe9fe5cb640 1 --2- 172.21.15.84:0/772651398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9f806c810 0x7fe9f815b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.108+0000 7fe9fe5cb640 1 -- 172.21.15.84:0/772651398 >> 172.21.15.84:0/772651398 conn(0x7fe9f80ce6a0 msgr2=0x7fe9f80af960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:00.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.112+0000 7fe9fe5cb640 1 -- 172.21.15.84:0/772651398 shutdown_connections 2023-10-16T10:09:00.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.112+0000 7fe9fe5cb640 1 -- 172.21.15.84:0/772651398 wait complete. 2023-10-16T10:09:00.138 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T10:09:00.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.328+0000 7f883900e640 1 Processor -- start 2023-10-16T10:09:00.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.328+0000 7f883900e640 1 -- start start 2023-10-16T10:09:00.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.330+0000 7f883900e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8834159ab0 0x7f88340ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:00.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.330+0000 7f883900e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8834159e90 0x7f883406fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:00.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.330+0000 7f883900e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f883406c140 0x7f883406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:00.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.330+0000 7f883900e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f883406cb10 con 0x7f883406c140 2023-10-16T10:09:00.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.330+0000 7f883900e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f883415d890 con 0x7f8834159e90 2023-10-16T10:09:00.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.330+0000 7f883900e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88340cd1f0 con 0x7f8834159ab0 2023-10-16T10:09:00.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.330+0000 7f883359e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f883406c140 0x7f883406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:00.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.330+0000 7f883359e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f883406c140 0x7f883406c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51706/0 (socket says 172.21.15.84:51706) 2023-10-16T10:09:00.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.330+0000 7f883359e640 1 -- 172.21.15.84:0/2248482620 learned_addr learned my addr 172.21.15.84:0/2248482620 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:00.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.331+0000 7f882bfff640 1 --2- 172.21.15.84:0/2248482620 >> v2:172.21.15.157:3300/0 conn(0x7f8834159e90 0x7f883406fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:00.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.331+0000 7f8832d9d640 1 --2- 172.21.15.84:0/2248482620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8834159ab0 0x7f88340ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:00.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.331+0000 7f883359e640 1 -- 172.21.15.84:0/2248482620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8834159ab0 msgr2=0x7f88340ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:00.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.331+0000 7f883359e640 1 --2- 172.21.15.84:0/2248482620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8834159ab0 0x7f88340ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.331+0000 7f883359e640 1 -- 172.21.15.84:0/2248482620 >> v2:172.21.15.157:3300/0 conn(0x7f8834159e90 msgr2=0x7f883406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:00.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.331+0000 7f883359e640 1 --2- 172.21.15.84:0/2248482620 >> v2:172.21.15.157:3300/0 conn(0x7f8834159e90 0x7f883406fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.331+0000 7f883359e640 1 -- 172.21.15.84:0/2248482620 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88340cd4d0 con 0x7f883406c140 2023-10-16T10:09:00.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.331+0000 7f882bfff640 1 --2- 172.21.15.84:0/2248482620 >> v2:172.21.15.157:3300/0 conn(0x7f8834159e90 0x7f883406fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:00.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.332+0000 7f883359e640 1 --2- 172.21.15.84:0/2248482620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f883406c140 0x7f883406c5d0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f882c009e50 tx=0x7f882c02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c21936ad026c9a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:00.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.332+0000 7f883259c640 1 -- 172.21.15.84:0/2248482620 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c02c950 con 0x7f883406c140 2023-10-16T10:09:00.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.332+0000 7f883259c640 1 -- 172.21.15.84:0/2248482620 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882c02cab0 con 0x7f883406c140 2023-10-16T10:09:00.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.332+0000 7f883259c640 1 -- 172.21.15.84:0/2248482620 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c02ce40 con 0x7f883406c140 2023-10-16T10:09:00.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.333+0000 7f883900e640 1 -- 172.21.15.84:0/2248482620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f883406c140 msgr2=0x7f883406c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:00.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.333+0000 7f883900e640 1 --2- 172.21.15.84:0/2248482620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f883406c140 0x7f883406c5d0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f882c009e50 tx=0x7f882c02bdb0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.333+0000 7f883900e640 1 -- 172.21.15.84:0/2248482620 shutdown_connections 2023-10-16T10:09:00.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.333+0000 7f883900e640 1 --2- 172.21.15.84:0/2248482620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f883406c140 0x7f883406c5d0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.333+0000 7f883900e640 1 --2- 172.21.15.84:0/2248482620 >> v2:172.21.15.157:3300/0 conn(0x7f8834159e90 0x7f883406fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.333+0000 7f883900e640 1 --2- 172.21.15.84:0/2248482620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8834159ab0 0x7f88340ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.333+0000 7f883900e640 1 -- 172.21.15.84:0/2248482620 >> 172.21.15.84:0/2248482620 conn(0x7f88340ce9b0 msgr2=0x7f88340cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:00.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.333+0000 7f883900e640 1 -- 172.21.15.84:0/2248482620 shutdown_connections 2023-10-16T10:09:00.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.333+0000 7f883900e640 1 -- 172.21.15.84:0/2248482620 wait complete. 2023-10-16T10:09:00.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.335+0000 7f883900e640 1 Processor -- start 2023-10-16T10:09:00.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.335+0000 7f883900e640 1 -- start start 2023-10-16T10:09:00.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.336+0000 7f883900e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883406c140 0x7f88340acc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:00.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.336+0000 7f883900e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8834159ab0 0x7f8834154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:00.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.336+0000 7f883900e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8834159e90 0x7f88340af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:00.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.336+0000 7f883900e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88340c3980 con 0x7f8834159e90 2023-10-16T10:09:00.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.336+0000 7f883900e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88340b15d0 con 0x7f883406c140 2023-10-16T10:09:00.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.336+0000 7f883900e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88340b1870 con 0x7f8834159ab0 2023-10-16T10:09:00.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.336+0000 7f883359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8834159e90 0x7f88340af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:00.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.336+0000 7f883359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8834159e90 0x7f88340af870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51722/0 (socket says 172.21.15.84:51722) 2023-10-16T10:09:00.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.336+0000 7f883359e640 1 -- 172.21.15.84:0/208962201 learned_addr learned my addr 172.21.15.84:0/208962201 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:00.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.337+0000 7f8832d9d640 1 --2- 172.21.15.84:0/208962201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883406c140 0x7f88340acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:00.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.337+0000 7f882bfff640 1 --2- 172.21.15.84:0/208962201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8834159ab0 0x7f8834154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:00.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.337+0000 7f883359e640 1 -- 172.21.15.84:0/208962201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8834159ab0 msgr2=0x7f8834154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:00.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.337+0000 7f883359e640 1 --2- 172.21.15.84:0/208962201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8834159ab0 0x7f8834154fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.337+0000 7f883359e640 1 -- 172.21.15.84:0/208962201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883406c140 msgr2=0x7f88340acc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:00.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.337+0000 7f883359e640 1 --2- 172.21.15.84:0/208962201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883406c140 0x7f88340acc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.337+0000 7f883359e640 1 -- 172.21.15.84:0/208962201 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c009b30 con 0x7f8834159e90 2023-10-16T10:09:00.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.337+0000 7f8832d9d640 1 --2- 172.21.15.84:0/208962201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883406c140 0x7f88340acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:00.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.337+0000 7f883359e640 1 --2- 172.21.15.84:0/208962201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8834159e90 0x7f88340af870 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f882c036040 tx=0x7f882c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:00.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.337+0000 7f882b7fe640 1 -- 172.21.15.84:0/208962201 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c03e450 con 0x7f8834159e90 2023-10-16T10:09:00.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.337+0000 7f882b7fe640 1 -- 172.21.15.84:0/208962201 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882c03e5d0 con 0x7f8834159e90 2023-10-16T10:09:00.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.337+0000 7f882b7fe640 1 -- 172.21.15.84:0/208962201 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c03e980 con 0x7f8834159e90 2023-10-16T10:09:00.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.337+0000 7f883900e640 1 -- 172.21.15.84:0/208962201 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88340b1b50 con 0x7f8834159e90 2023-10-16T10:09:00.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.338+0000 7f883900e640 1 -- 172.21.15.84:0/208962201 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88340ad1a0 con 0x7f8834159e90 2023-10-16T10:09:00.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.339+0000 7f883900e640 1 -- 172.21.15.84:0/208962201 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88340cda70 con 0x7f8834159e90 2023-10-16T10:09:00.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.340+0000 7f882b7fe640 1 -- 172.21.15.84:0/208962201 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f882c03eae0 con 0x7f8834159e90 2023-10-16T10:09:00.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.340+0000 7f882b7fe640 1 --2- 172.21.15.84:0/208962201 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f881406af90 0x7f881406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:00.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.341+0000 7f882b7fe640 1 -- 172.21.15.84:0/208962201 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f882c0ac670 con 0x7f8834159e90 2023-10-16T10:09:00.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.341+0000 7f8832d9d640 1 --2- 172.21.15.84:0/208962201 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f881406af90 0x7f881406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:00.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.341+0000 7f8832d9d640 1 --2- 172.21.15.84:0/208962201 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f881406af90 0x7f881406d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8824006fd0 tx=0x7f8824008040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:00.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.345+0000 7f882b7fe640 1 -- 172.21.15.84:0/208962201 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f88340cda70 con 0x7f8834159e90 2023-10-16T10:09:00.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.526+0000 7f883900e640 1 -- 172.21.15.84:0/208962201 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f88340c3ac0 con 0x7f8834159e90 2023-10-16T10:09:00.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.533+0000 7f882b7fe640 1 -- 172.21.15.84:0/208962201 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f88340c3ac0 con 0x7f8834159e90 2023-10-16T10:09:00.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.535+0000 7f883900e640 1 -- 172.21.15.84:0/208962201 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f881406af90 msgr2=0x7f881406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:00.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.535+0000 7f883900e640 1 --2- 172.21.15.84:0/208962201 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f881406af90 0x7f881406d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8824006fd0 tx=0x7f8824008040 comp rx=0 tx=0).stop 2023-10-16T10:09:00.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.535+0000 7f883900e640 1 -- 172.21.15.84:0/208962201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8834159e90 msgr2=0x7f88340af870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:00.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.535+0000 7f883900e640 1 --2- 172.21.15.84:0/208962201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8834159e90 0x7f88340af870 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f882c036040 tx=0x7f882c004a40 comp rx=0 tx=0).stop 2023-10-16T10:09:00.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.535+0000 7f883900e640 1 -- 172.21.15.84:0/208962201 shutdown_connections 2023-10-16T10:09:00.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.535+0000 7f883900e640 1 --2- 172.21.15.84:0/208962201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8834159e90 0x7f88340af870 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.535+0000 7f883900e640 1 --2- 172.21.15.84:0/208962201 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f881406af90 0x7f881406d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.536+0000 7f883900e640 1 --2- 172.21.15.84:0/208962201 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8834159ab0 0x7f8834154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.536+0000 7f883900e640 1 --2- 172.21.15.84:0/208962201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883406c140 0x7f88340acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.536+0000 7f883900e640 1 -- 172.21.15.84:0/208962201 >> 172.21.15.84:0/208962201 conn(0x7f88340ce9b0 msgr2=0x7f8834158be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:00.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.536+0000 7f883900e640 1 -- 172.21.15.84:0/208962201 shutdown_connections 2023-10-16T10:09:00.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.536+0000 7f883900e640 1 -- 172.21.15.84:0/208962201 wait complete. 2023-10-16T10:09:00.554 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T10:09:00.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.757+0000 7feeee133640 1 Processor -- start 2023-10-16T10:09:00.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.758+0000 7feeee133640 1 -- start start 2023-10-16T10:09:00.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.759+0000 7feeee133640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feee80caa80 0x7feee8069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:00.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.759+0000 7feeee133640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee80caef0 con 0x7feee80addb0 2023-10-16T10:09:00.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.759+0000 7feeee133640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee806a6f0 con 0x7feee806a2e0 2023-10-16T10:09:00.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.759+0000 7feeee133640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee8161a50 con 0x7feee80caa80 2023-10-16T10:09:00.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.760+0000 7feee77fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7feee80addb0 0x7feee80c9000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34180/0 (socket says 172.21.15.84:34180) 2023-10-16T10:09:00.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.760+0000 7feee77fe640 1 -- 172.21.15.84:0/2469810074 learned_addr learned my addr 172.21.15.84:0/2469810074 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:09:00.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.760+0000 7feee6ffd640 1 -- 172.21.15.84:0/2469810074 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169205028 0 0) 0x7feee80caef0 con 0x7feee80addb0 2023-10-16T10:09:00.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.760+0000 7feee6ffd640 1 -- 172.21.15.84:0/2469810074 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feed4003520 con 0x7feee80addb0 2023-10-16T10:09:00.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.760+0000 7feedffff640 1 --2- 172.21.15.84:0/2469810074 >> v2:172.21.15.157:3300/0 conn(0x7feee80caa80 0x7feee8069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:00.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.760+0000 7feee6ffd640 1 -- 172.21.15.84:0/2469810074 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1081047048 0 0) 0x7feed4003520 con 0x7feee80addb0 2023-10-16T10:09:00.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.760+0000 7feee6ffd640 1 -- 172.21.15.84:0/2469810074 >> v2:172.21.15.157:3300/0 conn(0x7feee80caa80 msgr2=0x7feee8069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:00.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.760+0000 7feee6ffd640 1 --2- 172.21.15.84:0/2469810074 >> v2:172.21.15.157:3300/0 conn(0x7feee80caa80 0x7feee8069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.760+0000 7feee6ffd640 1 -- 172.21.15.84:0/2469810074 >> v1:172.21.15.157:6789/0 conn(0x7feee806a2e0 legacy=0x7feee81650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:00.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.760+0000 7feee6ffd640 1 -- 172.21.15.84:0/2469810074 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee80cb140 con 0x7feee80addb0 2023-10-16T10:09:00.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.760+0000 7feee6ffd640 1 -- 172.21.15.84:0/2469810074 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7feed8001670 con 0x7feee80addb0 2023-10-16T10:09:00.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.760+0000 7feee6ffd640 1 -- 172.21.15.84:0/2469810074 >> v1:172.21.15.84:6789/0 conn(0x7feee80addb0 legacy=0x7feee80c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:00.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.761+0000 7feee6ffd640 1 --2- 172.21.15.84:0/2469810074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed40039c0 0x7feed4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:00.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.761+0000 7feee6ffd640 1 --2- 172.21.15.84:0/2469810074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed40047c0 0x7feed4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:00.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.761+0000 7feee6ffd640 1 --2- 172.21.15.84:0/2469810074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feed40051d0 0x7feed400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:00.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.761+0000 7feee6ffd640 1 -- 172.21.15.84:0/2469810074 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee8161a50 con 0x7feed40047c0 2023-10-16T10:09:00.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.761+0000 7feee6ffd640 1 -- 172.21.15.84:0/2469810074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed400c100 con 0x7feed40039c0 2023-10-16T10:09:00.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.761+0000 7feee6ffd640 1 -- 172.21.15.84:0/2469810074 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed400c2c0 con 0x7feed40051d0 2023-10-16T10:09:00.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.761+0000 7feedffff640 1 --2- 172.21.15.84:0/2469810074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed40047c0 0x7feed4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:00.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.761+0000 7feedffff640 1 -- 172.21.15.84:0/2469810074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feed40051d0 msgr2=0x7feed400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:00.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.761+0000 7feedffff640 1 --2- 172.21.15.84:0/2469810074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feed40051d0 0x7feed400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.761+0000 7feedffff640 1 -- 172.21.15.84:0/2469810074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed40039c0 msgr2=0x7feed4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:00.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.761+0000 7feedffff640 1 --2- 172.21.15.84:0/2469810074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed40039c0 0x7feed4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.761+0000 7feedffff640 1 -- 172.21.15.84:0/2469810074 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed4003520 con 0x7feed40047c0 2023-10-16T10:09:00.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.761+0000 7feee7fff640 1 -- 172.21.15.84:0/2469810074 reap_dead start 2023-10-16T10:09:00.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.762+0000 7feedffff640 1 --2- 172.21.15.84:0/2469810074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed40047c0 0x7feed4004c40 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7feed000ba00 tx=0x7feed40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61595b065d6a0dd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:00.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.763+0000 7feee6ffd640 1 -- 172.21.15.84:0/2469810074 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed0032620 con 0x7feed40047c0 2023-10-16T10:09:00.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.763+0000 7feee6ffd640 1 -- 172.21.15.84:0/2469810074 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed0032780 con 0x7feed40047c0 2023-10-16T10:09:00.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.763+0000 7feee6ffd640 1 -- 172.21.15.84:0/2469810074 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed0032b10 con 0x7feed40047c0 2023-10-16T10:09:00.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.763+0000 7feeee133640 1 -- 172.21.15.84:0/2469810074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed40047c0 msgr2=0x7feed4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:00.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.763+0000 7feeee133640 1 --2- 172.21.15.84:0/2469810074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed40047c0 0x7feed4004c40 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7feed000ba00 tx=0x7feed40057f0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.763+0000 7feeee133640 1 -- 172.21.15.84:0/2469810074 shutdown_connections 2023-10-16T10:09:00.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.763+0000 7feeee133640 1 --2- 172.21.15.84:0/2469810074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed40047c0 0x7feed4004c40 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.763+0000 7feeee133640 1 -- 172.21.15.84:0/2469810074 >> 172.21.15.84:0/2469810074 conn(0x7feee80ce9b0 msgr2=0x7feee80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:00.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.763+0000 7feeee133640 1 -- 172.21.15.84:0/2469810074 shutdown_connections 2023-10-16T10:09:00.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.763+0000 7feeee133640 1 -- 172.21.15.84:0/2469810074 wait complete. 2023-10-16T10:09:00.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.765+0000 7feeee133640 1 Processor -- start 2023-10-16T10:09:00.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.765+0000 7feeee133640 1 -- start start 2023-10-16T10:09:00.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.766+0000 7feeee133640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed40047c0 0x7feee815cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:00.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.766+0000 7feeee133640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee815fbf0 0x7feee81f9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:00.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.766+0000 7feeee133640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feee8163590 0x7feee8163a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:00.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.766+0000 7feeee133640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee8163f50 con 0x7feed40047c0 2023-10-16T10:09:00.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.766+0000 7feeee133640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee80c8c30 con 0x7feee8163590 2023-10-16T10:09:00.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.766+0000 7feeee133640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee80c5a80 con 0x7feee815fbf0 2023-10-16T10:09:00.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.766+0000 7feee77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed40047c0 0x7feee815cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:00.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.766+0000 7feee77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed40047c0 0x7feee815cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51746/0 (socket says 172.21.15.84:51746) 2023-10-16T10:09:00.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.766+0000 7feee77fe640 1 -- 172.21.15.84:0/2559106710 learned_addr learned my addr 172.21.15.84:0/2559106710 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:00.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.767+0000 7feee77fe640 1 -- 172.21.15.84:0/2559106710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee815fbf0 msgr2=0x7feee81f9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:00.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.767+0000 7feee77fe640 1 --2- 172.21.15.84:0/2559106710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee815fbf0 0x7feee81f9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.767+0000 7feee77fe640 1 -- 172.21.15.84:0/2559106710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feee8163590 msgr2=0x7feee8163a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:00.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.767+0000 7feee77fe640 1 --2- 172.21.15.84:0/2559106710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feee8163590 0x7feee8163a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.767+0000 7feee77fe640 1 -- 172.21.15.84:0/2559106710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee81fa240 con 0x7feed40047c0 2023-10-16T10:09:00.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.767+0000 7feee77fe640 1 --2- 172.21.15.84:0/2559106710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed40047c0 0x7feee815cae0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7feed800c700 tx=0x7feed800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:00.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.767+0000 7feee4ff9640 1 -- 172.21.15.84:0/2559106710 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed80072d0 con 0x7feed40047c0 2023-10-16T10:09:00.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.767+0000 7feee4ff9640 1 -- 172.21.15.84:0/2559106710 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed8007430 con 0x7feed40047c0 2023-10-16T10:09:00.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.767+0000 7feeee133640 1 -- 172.21.15.84:0/2559106710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee81fa4d0 con 0x7feed40047c0 2023-10-16T10:09:00.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.767+0000 7feee4ff9640 1 -- 172.21.15.84:0/2559106710 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed80047e0 con 0x7feed40047c0 2023-10-16T10:09:00.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.767+0000 7feeee133640 1 -- 172.21.15.84:0/2559106710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee81faa90 con 0x7feed40047c0 2023-10-16T10:09:00.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.769+0000 7feeee133640 1 -- 172.21.15.84:0/2559106710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee804e160 con 0x7feed40047c0 2023-10-16T10:09:00.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.770+0000 7feee4ff9640 1 -- 172.21.15.84:0/2559106710 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7feed8004940 con 0x7feed40047c0 2023-10-16T10:09:00.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.770+0000 7feee4ff9640 1 --2- 172.21.15.84:0/2559106710 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7feec806af90 0x7feec806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:00.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.771+0000 7feee4ff9640 1 -- 172.21.15.84:0/2559106710 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5702+0+0 (secure 0 0 0) 0x7feed8088fb0 con 0x7feed40047c0 2023-10-16T10:09:00.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.771+0000 7feedffff640 1 --2- 172.21.15.84:0/2559106710 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7feec806af90 0x7feec806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:00.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.771+0000 7feedffff640 1 --2- 172.21.15.84:0/2559106710 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7feec806af90 0x7feec806d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7feed40057a0 tx=0x7feed0032000 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:00.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.775+0000 7feee4ff9640 1 -- 172.21.15.84:0/2559106710 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7feed8053f40 con 0x7feed40047c0 2023-10-16T10:09:00.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.967+0000 7feeee133640 1 -- 172.21.15.84:0/2559106710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7feee806c5b0 con 0x7feed40047c0 2023-10-16T10:09:00.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.972+0000 7feee4ff9640 1 -- 172.21.15.84:0/2559106710 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v50) v1 ==== 120+0+0 (secure 0 0 0) 0x7feed80180b0 con 0x7feed40047c0 2023-10-16T10:09:00.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.975+0000 7feeee133640 1 -- 172.21.15.84:0/2559106710 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7feec806af90 msgr2=0x7feec806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:00.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.975+0000 7feeee133640 1 --2- 172.21.15.84:0/2559106710 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7feec806af90 0x7feec806d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7feed40057a0 tx=0x7feed0032000 comp rx=0 tx=0).stop 2023-10-16T10:09:00.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.975+0000 7feeee133640 1 -- 172.21.15.84:0/2559106710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed40047c0 msgr2=0x7feee815cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:00.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.975+0000 7feeee133640 1 --2- 172.21.15.84:0/2559106710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed40047c0 0x7feee815cae0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7feed800c700 tx=0x7feed800cbc0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.975+0000 7feeee133640 1 -- 172.21.15.84:0/2559106710 shutdown_connections 2023-10-16T10:09:00.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.975+0000 7feeee133640 1 --2- 172.21.15.84:0/2559106710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feee8163590 0x7feee8163a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.975+0000 7feeee133640 1 --2- 172.21.15.84:0/2559106710 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7feec806af90 0x7feec806d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.975+0000 7feeee133640 1 --2- 172.21.15.84:0/2559106710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee815fbf0 0x7feee81f9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.975+0000 7feeee133640 1 --2- 172.21.15.84:0/2559106710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed40047c0 0x7feee815cae0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:00.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.975+0000 7feeee133640 1 -- 172.21.15.84:0/2559106710 >> 172.21.15.84:0/2559106710 conn(0x7feee80ce9b0 msgr2=0x7feee80c9070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:00.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.975+0000 7feeee133640 1 -- 172.21.15.84:0/2559106710 shutdown_connections 2023-10-16T10:09:00.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:00.975+0000 7feeee133640 1 -- 172.21.15.84:0/2559106710 wait complete. 2023-10-16T10:09:00.997 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T10:09:01.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.213+0000 7f57d5c15640 1 Processor -- start 2023-10-16T10:09:01.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.213+0000 7f57d5c15640 1 -- start start 2023-10-16T10:09:01.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.214+0000 7f57d5c15640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d00caa10 0x7f57d01593b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:01.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.214+0000 7f57d5c15640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57d00b0480 0x7f57d006f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:01.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.214+0000 7f57d5c15640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d00ca5d0 con 0x7f57d00b08e0 2023-10-16T10:09:01.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.214+0000 7f57d5c15640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57d00c9a90 con 0x7f57d00b0480 2023-10-16T10:09:01.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.214+0000 7f57d5c15640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d00c9d30 con 0x7f57d00caa10 2023-10-16T10:09:01.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.215+0000 7f57cf7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d00caa10 0x7f57d01593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:01.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.215+0000 7f57cf7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d00caa10 0x7f57d01593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:39712/0 (socket says 172.21.15.84:39712) 2023-10-16T10:09:01.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.215+0000 7f57cf7fe640 1 -- 172.21.15.84:0/108855160 learned_addr learned my addr 172.21.15.84:0/108855160 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:01.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.215+0000 7f57c7fff640 1 --2- 172.21.15.84:0/108855160 >> v2:172.21.15.157:3300/0 conn(0x7f57d00b0480 0x7f57d006f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:01.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.216+0000 7f57cf7fe640 1 -- 172.21.15.84:0/108855160 >> v2:172.21.15.157:3300/0 conn(0x7f57d00b0480 msgr2=0x7f57d006f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:01.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.216+0000 7f57cf7fe640 1 --2- 172.21.15.84:0/108855160 >> v2:172.21.15.157:3300/0 conn(0x7f57d00b0480 0x7f57d006f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:01.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.216+0000 7f57cf7fe640 1 -- 172.21.15.84:0/108855160 >> v1:172.21.15.157:6789/0 conn(0x7f57d00b08e0 legacy=0x7f57d006fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:01.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.216+0000 7f57cf7fe640 1 -- 172.21.15.84:0/108855160 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d00ca010 con 0x7f57d00caa10 2023-10-16T10:09:01.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.216+0000 7f57ceffd640 1 -- 172.21.15.84:0/108855160 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4095594852 0 0) 0x7f57d00ca5d0 con 0x7f57d00b08e0 2023-10-16T10:09:01.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.218+0000 7f57cf7fe640 1 --2- 172.21.15.84:0/108855160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d00caa10 0x7f57d01593b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009870 tx=0x7f57c002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=733c413424ca337a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:01.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.218+0000 7f57ceffd640 1 -- 172.21.15.84:0/108855160 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0031640 con 0x7f57d00caa10 2023-10-16T10:09:01.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.218+0000 7f57ceffd640 1 -- 172.21.15.84:0/108855160 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c00317a0 con 0x7f57d00caa10 2023-10-16T10:09:01.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.218+0000 7f57ceffd640 1 -- 172.21.15.84:0/108855160 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0031b30 con 0x7f57d00caa10 2023-10-16T10:09:01.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.219+0000 7f57d5c15640 1 -- 172.21.15.84:0/108855160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d00caa10 msgr2=0x7f57d01593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:01.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.219+0000 7f57d5c15640 1 --2- 172.21.15.84:0/108855160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d00caa10 0x7f57d01593b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009870 tx=0x7f57c002b8f0 comp rx=0 tx=0).stop 2023-10-16T10:09:01.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.219+0000 7f57d5c15640 1 -- 172.21.15.84:0/108855160 shutdown_connections 2023-10-16T10:09:01.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.219+0000 7f57d5c15640 1 --2- 172.21.15.84:0/108855160 >> v2:172.21.15.157:3300/0 conn(0x7f57d00b0480 0x7f57d006f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:01.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.219+0000 7f57d5c15640 1 --2- 172.21.15.84:0/108855160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d00caa10 0x7f57d01593b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:01.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.219+0000 7f57d5c15640 1 -- 172.21.15.84:0/108855160 >> 172.21.15.84:0/108855160 conn(0x7f57d00cb410 msgr2=0x7f57d00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:01.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.219+0000 7f57d5c15640 1 -- 172.21.15.84:0/108855160 shutdown_connections 2023-10-16T10:09:01.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.219+0000 7f57d5c15640 1 -- 172.21.15.84:0/108855160 wait complete. 2023-10-16T10:09:01.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.221+0000 7f57d5c15640 1 Processor -- start 2023-10-16T10:09:01.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.221+0000 7f57d5c15640 1 -- start start 2023-10-16T10:09:01.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.223+0000 7f57d5c15640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57d00b0480 0x7f57d00c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:01.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.223+0000 7f57d5c15640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d00b08e0 0x7f57d014f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:01.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.223+0000 7f57d5c15640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57d00caa10 0x7f57d00ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:01.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.223+0000 7f57d5c15640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d014e1a0 con 0x7f57d00caa10 2023-10-16T10:09:01.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.223+0000 7f57d5c15640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d014e480 con 0x7f57d00b0480 2023-10-16T10:09:01.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.223+0000 7f57d5c15640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d014e760 con 0x7f57d00b08e0 2023-10-16T10:09:01.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.223+0000 7f57cffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57d00caa10 0x7f57d00ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:01.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.223+0000 7f57cffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57d00caa10 0x7f57d00ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51754/0 (socket says 172.21.15.84:51754) 2023-10-16T10:09:01.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.223+0000 7f57cffff640 1 -- 172.21.15.84:0/3148995471 learned_addr learned my addr 172.21.15.84:0/3148995471 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:01.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.223+0000 7f57cf7fe640 1 --2- 172.21.15.84:0/3148995471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57d00b0480 0x7f57d00c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:01.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.223+0000 7f57c7fff640 1 --2- 172.21.15.84:0/3148995471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d00b08e0 0x7f57d014f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:01.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.223+0000 7f57cffff640 1 -- 172.21.15.84:0/3148995471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d00b08e0 msgr2=0x7f57d014f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:01.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.223+0000 7f57cffff640 1 --2- 172.21.15.84:0/3148995471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d00b08e0 0x7f57d014f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:01.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.223+0000 7f57cffff640 1 -- 172.21.15.84:0/3148995471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57d00b0480 msgr2=0x7f57d00c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:01.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.223+0000 7f57cffff640 1 --2- 172.21.15.84:0/3148995471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57d00b0480 0x7f57d00c3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:01.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.224+0000 7f57cffff640 1 -- 172.21.15.84:0/3148995471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c00094d0 con 0x7f57d00caa10 2023-10-16T10:09:01.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.224+0000 7f57cffff640 1 --2- 172.21.15.84:0/3148995471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57d00caa10 0x7f57d00ac890 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f57c800c760 tx=0x7f57c800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:01.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.224+0000 7f57ccff9640 1 -- 172.21.15.84:0/3148995471 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8003da0 con 0x7f57d00caa10 2023-10-16T10:09:01.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.224+0000 7f57ccff9640 1 -- 172.21.15.84:0/3148995471 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c800ce70 con 0x7f57d00caa10 2023-10-16T10:09:01.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.224+0000 7f57d5c15640 1 -- 172.21.15.84:0/3148995471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d014ea40 con 0x7f57d00caa10 2023-10-16T10:09:01.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.224+0000 7f57ccff9640 1 -- 172.21.15.84:0/3148995471 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8005290 con 0x7f57d00caa10 2023-10-16T10:09:01.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.224+0000 7f57d5c15640 1 -- 172.21.15.84:0/3148995471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d014f780 con 0x7f57d00caa10 2023-10-16T10:09:01.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.225+0000 7f57d5c15640 1 -- 172.21.15.84:0/3148995471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d014ef60 con 0x7f57d00caa10 2023-10-16T10:09:01.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.226+0000 7f57ccff9640 1 -- 172.21.15.84:0/3148995471 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f57c80053f0 con 0x7f57d00caa10 2023-10-16T10:09:01.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.227+0000 7f57ccff9640 1 --2- 172.21.15.84:0/3148995471 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f57b006af90 0x7f57b006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:01.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.227+0000 7f57ccff9640 1 -- 172.21.15.84:0/3148995471 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f57c80895b0 con 0x7f57d00caa10 2023-10-16T10:09:01.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.227+0000 7f57cf7fe640 1 --2- 172.21.15.84:0/3148995471 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f57b006af90 0x7f57b006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:01.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.228+0000 7f57cf7fe640 1 --2- 172.21.15.84:0/3148995471 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f57b006af90 0x7f57b006d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f57c0036000 tx=0x7f57c0008030 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:01.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.232+0000 7f57ccff9640 1 -- 172.21.15.84:0/3148995471 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f57d014ef60 con 0x7f57d00caa10 2023-10-16T10:09:01.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.423+0000 7f57d5c15640 1 -- 172.21.15.84:0/3148995471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f57d014ef60 con 0x7f57d00caa10 2023-10-16T10:09:01.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.430+0000 7f57ccff9640 1 -- 172.21.15.84:0/3148995471 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v51) v1 ==== 115+0+0 (secure 0 0 0) 0x7f57d014ef60 con 0x7f57d00caa10 2023-10-16T10:09:01.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.433+0000 7f57d5c15640 1 -- 172.21.15.84:0/3148995471 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f57b006af90 msgr2=0x7f57b006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:01.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.433+0000 7f57d5c15640 1 --2- 172.21.15.84:0/3148995471 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f57b006af90 0x7f57b006d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f57c0036000 tx=0x7f57c0008030 comp rx=0 tx=0).stop 2023-10-16T10:09:01.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.433+0000 7f57d5c15640 1 -- 172.21.15.84:0/3148995471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57d00caa10 msgr2=0x7f57d00ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:01.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.433+0000 7f57d5c15640 1 --2- 172.21.15.84:0/3148995471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57d00caa10 0x7f57d00ac890 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f57c800c760 tx=0x7f57c800cc20 comp rx=0 tx=0).stop 2023-10-16T10:09:01.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.434+0000 7f57d5c15640 1 -- 172.21.15.84:0/3148995471 shutdown_connections 2023-10-16T10:09:01.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.434+0000 7f57d5c15640 1 --2- 172.21.15.84:0/3148995471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57d00caa10 0x7f57d00ac890 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:01.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.434+0000 7f57d5c15640 1 --2- 172.21.15.84:0/3148995471 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f57b006af90 0x7f57b006d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:01.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.434+0000 7f57d5c15640 1 --2- 172.21.15.84:0/3148995471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57d00b08e0 0x7f57d014f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:01.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.434+0000 7f57d5c15640 1 --2- 172.21.15.84:0/3148995471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57d00b0480 0x7f57d00c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:01.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.434+0000 7f57d5c15640 1 -- 172.21.15.84:0/3148995471 >> 172.21.15.84:0/3148995471 conn(0x7f57d00cb410 msgr2=0x7f57d00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:01.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.434+0000 7f57d5c15640 1 -- 172.21.15.84:0/3148995471 shutdown_connections 2023-10-16T10:09:01.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.434+0000 7f57d5c15640 1 -- 172.21.15.84:0/3148995471 wait complete. 2023-10-16T10:09:01.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods' 2023-10-16T10:09:01.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.654+0000 7f1304578640 1 Processor -- start 2023-10-16T10:09:01.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.654+0000 7f1304578640 1 -- start start 2023-10-16T10:09:01.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.655+0000 7f1304578640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f12fc15aaa0 0x7f12fc15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:01.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.655+0000 7f1304578640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f12fc15ae80 0x7f12fc0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:01.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.655+0000 7f1304578640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12fc0ccaa0 0x7f12fc070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:01.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.655+0000 7f1304578640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12fc06b810 con 0x7f12fc15aaa0 2023-10-16T10:09:01.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.655+0000 7f1304578640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12fc15ed60 con 0x7f12fc15ae80 2023-10-16T10:09:01.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.655+0000 7f1304578640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc0707e0 con 0x7f12fc0ccaa0 2023-10-16T10:09:01.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.656+0000 7f1302314640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f12fc15aaa0 0x7f12fc15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:01.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.656+0000 7f1302314640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f12fc15aaa0 0x7f12fc15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51768/0 (socket says 172.21.15.84:51768) 2023-10-16T10:09:01.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.656+0000 7f1302314640 1 -- 172.21.15.84:0/2070466458 learned_addr learned my addr 172.21.15.84:0/2070466458 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:01.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.656+0000 7f1302314640 1 -- 172.21.15.84:0/2070466458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12fc0ccaa0 msgr2=0x7f12fc070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:01.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.656+0000 7f1302314640 1 --2- 172.21.15.84:0/2070466458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12fc0ccaa0 0x7f12fc070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:01.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.656+0000 7f1302314640 1 -- 172.21.15.84:0/2070466458 >> v2:172.21.15.157:3300/0 conn(0x7f12fc15ae80 msgr2=0x7f12fc0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:01.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.656+0000 7f1302314640 1 --2- 172.21.15.84:0/2070466458 >> v2:172.21.15.157:3300/0 conn(0x7f12fc15ae80 0x7f12fc0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:01.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.656+0000 7f1302314640 1 -- 172.21.15.84:0/2070466458 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc15ef90 con 0x7f12fc15aaa0 2023-10-16T10:09:01.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.657+0000 7f1302314640 1 --2- 172.21.15.84:0/2070466458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12fc15aaa0 0x7f12fc15e820 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0c59c0 tx=0x7f12f802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f15b6cd98d3a9ae server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:01.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.658+0000 7f1301312640 1 -- 172.21.15.84:0/2070466458 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8031880 con 0x7f12fc15aaa0 2023-10-16T10:09:01.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.658+0000 7f1301312640 1 -- 172.21.15.84:0/2070466458 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f80319e0 con 0x7f12fc15aaa0 2023-10-16T10:09:01.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.658+0000 7f1301312640 1 -- 172.21.15.84:0/2070466458 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8031d70 con 0x7f12fc15aaa0 2023-10-16T10:09:01.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.658+0000 7f1304578640 1 -- 172.21.15.84:0/2070466458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12fc15aaa0 msgr2=0x7f12fc15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:01.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.658+0000 7f1304578640 1 --2- 172.21.15.84:0/2070466458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12fc15aaa0 0x7f12fc15e820 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0c59c0 tx=0x7f12f802b920 comp rx=0 tx=0).stop 2023-10-16T10:09:01.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.658+0000 7f1304578640 1 -- 172.21.15.84:0/2070466458 shutdown_connections 2023-10-16T10:09:01.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.658+0000 7f1304578640 1 --2- 172.21.15.84:0/2070466458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12fc0ccaa0 0x7f12fc070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:01.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.658+0000 7f1304578640 1 --2- 172.21.15.84:0/2070466458 >> v2:172.21.15.157:3300/0 conn(0x7f12fc15ae80 0x7f12fc0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:01.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.658+0000 7f1304578640 1 --2- 172.21.15.84:0/2070466458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12fc15aaa0 0x7f12fc15e820 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:01.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.658+0000 7f1304578640 1 -- 172.21.15.84:0/2070466458 >> 172.21.15.84:0/2070466458 conn(0x7f12fc0cb410 msgr2=0x7f12fc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:01.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.658+0000 7f1304578640 1 -- 172.21.15.84:0/2070466458 shutdown_connections 2023-10-16T10:09:01.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.658+0000 7f1304578640 1 -- 172.21.15.84:0/2070466458 wait complete. 2023-10-16T10:09:01.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.660+0000 7f1304578640 1 Processor -- start 2023-10-16T10:09:01.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.660+0000 7f1304578640 1 -- start start 2023-10-16T10:09:01.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.661+0000 7f1304578640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12fc0ccaa0 0x7f12fc0c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:01.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.661+0000 7f1304578640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12fc15aaa0 0x7f12fc144cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:01.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.661+0000 7f1304578640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12fc15ae80 0x7f12fc0b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:01.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.661+0000 7f1304578640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc142c20 con 0x7f12fc0ccaa0 2023-10-16T10:09:01.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.661+0000 7f1304578640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc0c4600 con 0x7f12fc15ae80 2023-10-16T10:09:01.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.661+0000 7f1304578640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc145730 con 0x7f12fc15aaa0 2023-10-16T10:09:01.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.661+0000 7f1302314640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12fc0ccaa0 0x7f12fc0c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:01.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.662+0000 7f1302314640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12fc0ccaa0 0x7f12fc0c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51782/0 (socket says 172.21.15.84:51782) 2023-10-16T10:09:01.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.662+0000 7f1302314640 1 -- 172.21.15.84:0/3869469131 learned_addr learned my addr 172.21.15.84:0/3869469131 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:01.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.662+0000 7f1301b13640 1 --2- 172.21.15.84:0/3869469131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12fc15aaa0 0x7f12fc144cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:01.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.662+0000 7f1302314640 1 -- 172.21.15.84:0/3869469131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12fc15aaa0 msgr2=0x7f12fc144cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:01.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.662+0000 7f1302314640 1 --2- 172.21.15.84:0/3869469131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12fc15aaa0 0x7f12fc144cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:01.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.662+0000 7f1302b15640 1 --2- 172.21.15.84:0/3869469131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12fc15ae80 0x7f12fc0b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:01.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.662+0000 7f1302314640 1 -- 172.21.15.84:0/3869469131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12fc15ae80 msgr2=0x7f12fc0b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:01.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.662+0000 7f1302314640 1 --2- 172.21.15.84:0/3869469131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12fc15ae80 0x7f12fc0b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:01.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.662+0000 7f1302314640 1 -- 172.21.15.84:0/3869469131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f80094d0 con 0x7f12fc0ccaa0 2023-10-16T10:09:01.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.662+0000 7f1301b13640 1 --2- 172.21.15.84:0/3869469131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12fc15aaa0 0x7f12fc144cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:01.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.662+0000 7f1302314640 1 --2- 172.21.15.84:0/3869469131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12fc0ccaa0 0x7f12fc0c40c0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f12f802b960 tx=0x7f12f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:01.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.663+0000 7f12f2ffd640 1 -- 172.21.15.84:0/3869469131 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f803ad30 con 0x7f12fc0ccaa0 2023-10-16T10:09:01.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.663+0000 7f12f2ffd640 1 -- 172.21.15.84:0/3869469131 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f803ae90 con 0x7f12fc0ccaa0 2023-10-16T10:09:01.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.663+0000 7f12f2ffd640 1 -- 172.21.15.84:0/3869469131 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8044630 con 0x7f12fc0ccaa0 2023-10-16T10:09:01.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.663+0000 7f1304578640 1 -- 172.21.15.84:0/3869469131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc145a10 con 0x7f12fc0ccaa0 2023-10-16T10:09:01.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.663+0000 7f1304578640 1 -- 172.21.15.84:0/3869469131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12fc143150 con 0x7f12fc0ccaa0 2023-10-16T10:09:01.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.664+0000 7f1304578640 1 -- 172.21.15.84:0/3869469131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12fc143340 con 0x7f12fc0ccaa0 2023-10-16T10:09:01.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.665+0000 7f12f2ffd640 1 -- 172.21.15.84:0/3869469131 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f12f8044790 con 0x7f12fc0ccaa0 2023-10-16T10:09:01.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.666+0000 7f12f2ffd640 1 --2- 172.21.15.84:0/3869469131 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f12e006af90 0x7f12e006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:01.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.666+0000 7f12f2ffd640 1 -- 172.21.15.84:0/3869469131 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f12f80ac020 con 0x7f12fc0ccaa0 2023-10-16T10:09:01.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.666+0000 7f1301b13640 1 --2- 172.21.15.84:0/3869469131 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f12e006af90 0x7f12e006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:01.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.667+0000 7f1301b13640 1 --2- 172.21.15.84:0/3869469131 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f12e006af90 0x7f12e006d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f12ec006fd0 tx=0x7f12ec008040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:01.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.670+0000 7f12f2ffd640 1 -- 172.21.15.84:0/3869469131 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f12f8077060 con 0x7f12fc0ccaa0 2023-10-16T10:09:01.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:01.932+0000 7f1304578640 1 -- 172.21.15.84:0/3869469131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]} v 0) v1 -- 0x7f12fc0b4010 con 0x7f12fc0ccaa0 2023-10-16T10:09:02.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.116+0000 7f12f2ffd640 1 -- 172.21.15.84:0/3869469131 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]}]=0 v777) v1 ==== 160+0+0 (secure 0 0 0) 0x7f12f807a680 con 0x7f12fc0ccaa0 2023-10-16T10:09:02.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.119+0000 7f1304578640 1 -- 172.21.15.84:0/3869469131 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f12e006af90 msgr2=0x7f12e006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:02.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.119+0000 7f1304578640 1 --2- 172.21.15.84:0/3869469131 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f12e006af90 0x7f12e006d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f12ec006fd0 tx=0x7f12ec008040 comp rx=0 tx=0).stop 2023-10-16T10:09:02.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.119+0000 7f1304578640 1 -- 172.21.15.84:0/3869469131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12fc0ccaa0 msgr2=0x7f12fc0c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:02.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.119+0000 7f1304578640 1 --2- 172.21.15.84:0/3869469131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12fc0ccaa0 0x7f12fc0c40c0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f12f802b960 tx=0x7f12f80046c0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.119+0000 7f1304578640 1 -- 172.21.15.84:0/3869469131 shutdown_connections 2023-10-16T10:09:02.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.119+0000 7f1304578640 1 --2- 172.21.15.84:0/3869469131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12fc15ae80 0x7f12fc0b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.119+0000 7f1304578640 1 --2- 172.21.15.84:0/3869469131 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f12e006af90 0x7f12e006d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.119+0000 7f1304578640 1 --2- 172.21.15.84:0/3869469131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12fc15aaa0 0x7f12fc144cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.119+0000 7f1304578640 1 --2- 172.21.15.84:0/3869469131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12fc0ccaa0 0x7f12fc0c40c0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.119+0000 7f1304578640 1 -- 172.21.15.84:0/3869469131 >> 172.21.15.84:0/3869469131 conn(0x7f12fc0cb410 msgr2=0x7f12fc0b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:02.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.119+0000 7f1304578640 1 -- 172.21.15.84:0/3869469131 shutdown_connections 2023-10-16T10:09:02.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.119+0000 7f1304578640 1 -- 172.21.15.84:0/3869469131 wait complete. 2023-10-16T10:09:02.140 INFO:tasks.cephfs_test_runner:test_snap_schedule_all_periods (tasks.cephfs.test_snap_schedules.TestSnapSchedules) ... ok 2023-10-16T10:09:02.140 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that number of snaps retained are as per user spec. 2023-10-16T10:09:02.141 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T10:09:02.141 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:09:02.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.349+0000 7f7545fc9640 1 Processor -- start 2023-10-16T10:09:02.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.349+0000 7f7545fc9640 1 -- start start 2023-10-16T10:09:02.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.350+0000 7f7545fc9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7540159f00 0x7f75400b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:02.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.350+0000 7f7545fc9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f754015d880 0x7f75400b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:02.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.350+0000 7f7545fc9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754006f7f0 con 0x7f7540159b20 2023-10-16T10:09:02.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.350+0000 7f7545fc9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f754006c5e0 con 0x7f7540159f00 2023-10-16T10:09:02.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.350+0000 7f7545fc9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754006c8c0 con 0x7f754015d880 2023-10-16T10:09:02.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.351+0000 7f7537fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7540159f00 0x7f75400b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:02.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.351+0000 7f7537fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7540159f00 0x7f75400b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51790/0 (socket says 172.21.15.84:51790) 2023-10-16T10:09:02.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.351+0000 7f7537fff640 1 -- 172.21.15.84:0/2054097508 learned_addr learned my addr 172.21.15.84:0/2054097508 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:02.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.351+0000 7f7537fff640 1 -- 172.21.15.84:0/2054097508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f754015d880 msgr2=0x7f75400b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:02.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.351+0000 7f7537fff640 1 --2- 172.21.15.84:0/2054097508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f754015d880 0x7f75400b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.351+0000 7f7537fff640 1 -- 172.21.15.84:0/2054097508 >> v1:172.21.15.157:6789/0 conn(0x7f7540159b20 legacy=0x7f754006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:02.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.351+0000 7f7537fff640 1 -- 172.21.15.84:0/2054097508 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754006cba0 con 0x7f7540159f00 2023-10-16T10:09:02.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.352+0000 7f7537fff640 1 --2- 172.21.15.84:0/2054097508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7540159f00 0x7f75400b0410 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f7528009870 tx=0x7f752802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1817c58921304f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:02.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.352+0000 7f753effd640 1 -- 172.21.15.84:0/2054097508 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7528031640 con 0x7f7540159f00 2023-10-16T10:09:02.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.352+0000 7f753effd640 1 -- 172.21.15.84:0/2054097508 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75280317a0 con 0x7f7540159f00 2023-10-16T10:09:02.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.352+0000 7f753effd640 1 -- 172.21.15.84:0/2054097508 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7528031b30 con 0x7f7540159f00 2023-10-16T10:09:02.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.353+0000 7f7545fc9640 1 -- 172.21.15.84:0/2054097508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7540159f00 msgr2=0x7f75400b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:02.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.353+0000 7f7545fc9640 1 --2- 172.21.15.84:0/2054097508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7540159f00 0x7f75400b0410 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f7528009870 tx=0x7f752802b8f0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.353+0000 7f7545fc9640 1 -- 172.21.15.84:0/2054097508 shutdown_connections 2023-10-16T10:09:02.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.353+0000 7f7545fc9640 1 --2- 172.21.15.84:0/2054097508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f754015d880 0x7f75400b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.353+0000 7f7545fc9640 1 --2- 172.21.15.84:0/2054097508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7540159f00 0x7f75400b0410 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.353+0000 7f7545fc9640 1 -- 172.21.15.84:0/2054097508 >> 172.21.15.84:0/2054097508 conn(0x7f75400cb410 msgr2=0x7f75400cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:02.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.353+0000 7f7545fc9640 1 -- 172.21.15.84:0/2054097508 shutdown_connections 2023-10-16T10:09:02.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.353+0000 7f7545fc9640 1 -- 172.21.15.84:0/2054097508 wait complete. 2023-10-16T10:09:02.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.355+0000 7f7545fc9640 1 Processor -- start 2023-10-16T10:09:02.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.355+0000 7f7545fc9640 1 -- start start 2023-10-16T10:09:02.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.356+0000 7f7545fc9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7540159b20 0x7f75400cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:02.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.356+0000 7f7545fc9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7540159f00 0x7f754015a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:02.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.356+0000 7f7545fc9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f754015d880 0x7f7540151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:02.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.356+0000 7f7545fc9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754015c980 con 0x7f754015d880 2023-10-16T10:09:02.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.356+0000 7f7545fc9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754015cc20 con 0x7f7540159b20 2023-10-16T10:09:02.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.356+0000 7f7545fc9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754015cf00 con 0x7f7540159f00 2023-10-16T10:09:02.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.356+0000 7f753ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f754015d880 0x7f7540151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:02.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.356+0000 7f753ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f754015d880 0x7f7540151dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51798/0 (socket says 172.21.15.84:51798) 2023-10-16T10:09:02.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.356+0000 7f753ffff640 1 -- 172.21.15.84:0/39245293 learned_addr learned my addr 172.21.15.84:0/39245293 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:02.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.357+0000 7f753ffff640 1 -- 172.21.15.84:0/39245293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7540159f00 msgr2=0x7f754015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:02.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.357+0000 7f753ffff640 1 --2- 172.21.15.84:0/39245293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7540159f00 0x7f754015a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.357+0000 7f753ffff640 1 -- 172.21.15.84:0/39245293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7540159b20 msgr2=0x7f75400cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:02.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.357+0000 7f753ffff640 1 --2- 172.21.15.84:0/39245293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7540159b20 0x7f75400cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.357+0000 7f753ffff640 1 -- 172.21.15.84:0/39245293 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75280094d0 con 0x7f754015d880 2023-10-16T10:09:02.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.357+0000 7f753ffff640 1 --2- 172.21.15.84:0/39245293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f754015d880 0x7f7540151dc0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f753800b970 tx=0x7f753800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:02.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.357+0000 7f753cff9640 1 -- 172.21.15.84:0/39245293 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753800cb40 con 0x7f754015d880 2023-10-16T10:09:02.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.357+0000 7f753cff9640 1 -- 172.21.15.84:0/39245293 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753800ccc0 con 0x7f754015d880 2023-10-16T10:09:02.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.357+0000 7f753cff9640 1 -- 172.21.15.84:0/39245293 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7538014680 con 0x7f754015d880 2023-10-16T10:09:02.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.357+0000 7f7545fc9640 1 -- 172.21.15.84:0/39245293 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75400ce360 con 0x7f754015d880 2023-10-16T10:09:02.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.357+0000 7f7545fc9640 1 -- 172.21.15.84:0/39245293 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7540152300 con 0x7f754015d880 2023-10-16T10:09:02.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.358+0000 7f7545fc9640 1 -- 172.21.15.84:0/39245293 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754004e160 con 0x7f754015d880 2023-10-16T10:09:02.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.359+0000 7f753cff9640 1 -- 172.21.15.84:0/39245293 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f753801d3b0 con 0x7f754015d880 2023-10-16T10:09:02.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.360+0000 7f753cff9640 1 --2- 172.21.15.84:0/39245293 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f752006af90 0x7f752006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:02.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.360+0000 7f753cff9640 1 -- 172.21.15.84:0/39245293 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f7538089750 con 0x7f754015d880 2023-10-16T10:09:02.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.361+0000 7f753f7fe640 1 --2- 172.21.15.84:0/39245293 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f752006af90 0x7f752006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:02.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.361+0000 7f753f7fe640 1 --2- 172.21.15.84:0/39245293 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f752006af90 0x7f752006d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7530009650 tx=0x7f7530006ba0 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:02.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.364+0000 7f753cff9640 1 -- 172.21.15.84:0/39245293 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f75380546e0 con 0x7f754015d880 2023-10-16T10:09:02.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.557+0000 7f7545fc9640 1 -- 172.21.15.84:0/39245293 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7540152550 con 0x7f754015d880 2023-10-16T10:09:02.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.559+0000 7f753cff9640 1 -- 172.21.15.84:0/39245293 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+14079 (secure 0 0 0) 0x7f7538057d00 con 0x7f754015d880 2023-10-16T10:09:02.561 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:09:02.561 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":143,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:09:00.093384+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6836/791907708":"2023-10-17T10:08:57.029460+0000","172.21.15.84:6834/791907708":"2023-10-17T10:08:57.029460+0000","172.21.15.84:0/3145965786":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/3955589838":"2023-10-17T10:08:53.345773+0000","172.21.15.84:6833/1203729023":"2023-10-17T10:08:53.345773+0000","172.21.15.84:6832/1203729023":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/1578467419":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/2605161255":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/3702054673":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/2708741108":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/914481139":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/4141270981":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/2356933958":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/4132563801":"2023-10-17T10:08:40.209237+0000","172.21.15.84:6833/2784642648":"2023-10-17T10:08:40.209237+0000","172.21.15.84:6832/2784642648":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/3016819584":"2023-10-17T10:08:53.345773+0000","172.21.15.84:0/2180503356":"2023-10-17T10:08:40.209237+0000","172.21.15.84:0/2933995314":"2023-10-17T10:08:40.209237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:09:02.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.561+0000 7f7545fc9640 1 -- 172.21.15.84:0/39245293 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f752006af90 msgr2=0x7f752006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:02.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.562+0000 7f7545fc9640 1 --2- 172.21.15.84:0/39245293 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f752006af90 0x7f752006d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7530009650 tx=0x7f7530006ba0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.562+0000 7f7545fc9640 1 -- 172.21.15.84:0/39245293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f754015d880 msgr2=0x7f7540151dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:02.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.562+0000 7f7545fc9640 1 --2- 172.21.15.84:0/39245293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f754015d880 0x7f7540151dc0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f753800b970 tx=0x7f753800be30 comp rx=0 tx=0).stop 2023-10-16T10:09:02.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.562+0000 7f7545fc9640 1 -- 172.21.15.84:0/39245293 shutdown_connections 2023-10-16T10:09:02.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.562+0000 7f7545fc9640 1 --2- 172.21.15.84:0/39245293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f754015d880 0x7f7540151dc0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.562+0000 7f7545fc9640 1 --2- 172.21.15.84:0/39245293 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f752006af90 0x7f752006d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.562+0000 7f7545fc9640 1 --2- 172.21.15.84:0/39245293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7540159f00 0x7f754015a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.562+0000 7f7545fc9640 1 --2- 172.21.15.84:0/39245293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7540159b20 0x7f75400cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.562+0000 7f7545fc9640 1 -- 172.21.15.84:0/39245293 >> 172.21.15.84:0/39245293 conn(0x7f75400cb410 msgr2=0x7f75400cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:02.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.562+0000 7f7545fc9640 1 -- 172.21.15.84:0/39245293 shutdown_connections 2023-10-16T10:09:02.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.562+0000 7f7545fc9640 1 -- 172.21.15.84:0/39245293 wait complete. 2023-10-16T10:09:02.582 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:09:02.583 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:09:02.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.802+0000 7fc04a55a640 1 Processor -- start 2023-10-16T10:09:02.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.802+0000 7fc04a55a640 1 -- start start 2023-10-16T10:09:02.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.804+0000 7fc04a55a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc044070830 0x7fc04406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:02.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.804+0000 7fc04a55a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc044161740 con 0x7fc0440cf5f0 2023-10-16T10:09:02.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.804+0000 7fc04a55a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04406b0f0 con 0x7fc044070c10 2023-10-16T10:09:02.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.804+0000 7fc04a55a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0441619b0 con 0x7fc044070830 2023-10-16T10:09:02.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.804+0000 7fc043fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc044070830 0x7fc04406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:02.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.804+0000 7fc03bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc0440cf5f0 0x7fc04406c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34192/0 (socket says 172.21.15.84:34192) 2023-10-16T10:09:02.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.804+0000 7fc03bfff640 1 -- 172.21.15.84:0/3739912781 learned_addr learned my addr 172.21.15.84:0/3739912781 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:09:02.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.804+0000 7fc043fff640 1 -- 172.21.15.84:0/3739912781 >> v1:172.21.15.157:6789/0 conn(0x7fc044070c10 legacy=0x7fc0440cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:02.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.804+0000 7fc043fff640 1 -- 172.21.15.84:0/3739912781 >> v1:172.21.15.84:6789/0 conn(0x7fc0440cf5f0 legacy=0x7fc04406c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:02.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.805+0000 7fc043fff640 1 -- 172.21.15.84:0/3739912781 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04406b3d0 con 0x7fc044070830 2023-10-16T10:09:02.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.806+0000 7fc043fff640 1 --2- 172.21.15.84:0/3739912781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc044070830 0x7fc04406d3c0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fc0340097a0 tx=0x7fc03402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab7caea85fa7a98 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:02.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.806+0000 7fc0437fe640 1 -- 172.21.15.84:0/3739912781 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03402f050 con 0x7fc044070830 2023-10-16T10:09:02.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.806+0000 7fc0437fe640 1 -- 172.21.15.84:0/3739912781 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc034031700 con 0x7fc044070830 2023-10-16T10:09:02.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.806+0000 7fc0437fe640 1 -- 172.21.15.84:0/3739912781 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034031a90 con 0x7fc044070830 2023-10-16T10:09:02.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.806+0000 7fc04a55a640 1 -- 172.21.15.84:0/3739912781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc044070830 msgr2=0x7fc04406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:02.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.806+0000 7fc04a55a640 1 --2- 172.21.15.84:0/3739912781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc044070830 0x7fc04406d3c0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fc0340097a0 tx=0x7fc03402b7a0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.807+0000 7fc04a55a640 1 -- 172.21.15.84:0/3739912781 shutdown_connections 2023-10-16T10:09:02.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.807+0000 7fc04a55a640 1 --2- 172.21.15.84:0/3739912781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc044070830 0x7fc04406d3c0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.807+0000 7fc04a55a640 1 -- 172.21.15.84:0/3739912781 >> 172.21.15.84:0/3739912781 conn(0x7fc0440cfac0 msgr2=0x7fc0440cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:02.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.807+0000 7fc04a55a640 1 -- 172.21.15.84:0/3739912781 shutdown_connections 2023-10-16T10:09:02.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.807+0000 7fc04a55a640 1 -- 172.21.15.84:0/3739912781 wait complete. 2023-10-16T10:09:02.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.808+0000 7fc04a55a640 1 Processor -- start 2023-10-16T10:09:02.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.809+0000 7fc04a55a640 1 -- start start 2023-10-16T10:09:02.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.809+0000 7fc04a55a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc044070830 0x7fc04414a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:02.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.809+0000 7fc04a55a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc044070c10 0x7fc0440b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:02.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.809+0000 7fc04a55a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0440cf5f0 0x7fc0440b4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:02.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.809+0000 7fc04a55a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440b5780 con 0x7fc044070c10 2023-10-16T10:09:02.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.809+0000 7fc04a55a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440b4b00 con 0x7fc044070830 2023-10-16T10:09:02.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.809+0000 7fc04a55a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440b4de0 con 0x7fc0440cf5f0 2023-10-16T10:09:02.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.810+0000 7fc03bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc044070c10 0x7fc0440b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:02.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.810+0000 7fc03bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc044070c10 0x7fc0440b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51816/0 (socket says 172.21.15.84:51816) 2023-10-16T10:09:02.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.810+0000 7fc03bfff640 1 -- 172.21.15.84:0/1292810443 learned_addr learned my addr 172.21.15.84:0/1292810443 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:02.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.810+0000 7fc03bfff640 1 -- 172.21.15.84:0/1292810443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0440cf5f0 msgr2=0x7fc0440b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:02.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.810+0000 7fc03bfff640 1 --2- 172.21.15.84:0/1292810443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0440cf5f0 0x7fc0440b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.810+0000 7fc03bfff640 1 -- 172.21.15.84:0/1292810443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc044070830 msgr2=0x7fc04414a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:02.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.810+0000 7fc043fff640 1 --2- 172.21.15.84:0/1292810443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc044070830 0x7fc04414a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:02.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.810+0000 7fc03bfff640 1 --2- 172.21.15.84:0/1292810443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc044070830 0x7fc04414a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.810+0000 7fc03bfff640 1 -- 172.21.15.84:0/1292810443 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc034009400 con 0x7fc044070c10 2023-10-16T10:09:02.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.811+0000 7fc03bfff640 1 --2- 172.21.15.84:0/1292810443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc044070c10 0x7fc0440b3f40 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00c390 tx=0x7fc02c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:02.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.811+0000 7fc0417fa640 1 -- 172.21.15.84:0/1292810443 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c005ce0 con 0x7fc044070c10 2023-10-16T10:09:02.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.811+0000 7fc0417fa640 1 -- 172.21.15.84:0/1292810443 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c005e40 con 0x7fc044070c10 2023-10-16T10:09:02.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.811+0000 7fc0417fa640 1 -- 172.21.15.84:0/1292810443 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0072d0 con 0x7fc044070c10 2023-10-16T10:09:02.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.811+0000 7fc04a55a640 1 -- 172.21.15.84:0/1292810443 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0440b50c0 con 0x7fc044070c10 2023-10-16T10:09:02.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.811+0000 7fc04a55a640 1 -- 172.21.15.84:0/1292810443 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04414abb0 con 0x7fc044070c10 2023-10-16T10:09:02.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.812+0000 7fc04a55a640 1 -- 172.21.15.84:0/1292810443 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc04404e160 con 0x7fc044070c10 2023-10-16T10:09:02.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.813+0000 7fc0417fa640 1 -- 172.21.15.84:0/1292810443 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc02c007430 con 0x7fc044070c10 2023-10-16T10:09:02.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.814+0000 7fc0417fa640 1 --2- 172.21.15.84:0/1292810443 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fc02406af90 0x7fc02406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:02.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.814+0000 7fc0417fa640 1 -- 172.21.15.84:0/1292810443 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fc02c088b50 con 0x7fc044070c10 2023-10-16T10:09:02.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.814+0000 7fc043fff640 1 --2- 172.21.15.84:0/1292810443 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fc02406af90 0x7fc02406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:02.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.815+0000 7fc043fff640 1 --2- 172.21.15.84:0/1292810443 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fc02406af90 0x7fc02406d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc034035000 tx=0x7fc034008030 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:02.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.818+0000 7fc0417fa640 1 -- 172.21.15.84:0/1292810443 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc02c053c30 con 0x7fc044070c10 2023-10-16T10:09:02.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.993+0000 7fc04a55a640 1 -- 172.21.15.84:0/1292810443 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0440b5a40 con 0x7fc044070c10 2023-10-16T10:09:02.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.993+0000 7fc0417fa640 1 -- 172.21.15.84:0/1292810443 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc02c0170a0 con 0x7fc044070c10 2023-10-16T10:09:02.995 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:09:02.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.996+0000 7fc04a55a640 1 -- 172.21.15.84:0/1292810443 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fc02406af90 msgr2=0x7fc02406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:02.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.996+0000 7fc04a55a640 1 --2- 172.21.15.84:0/1292810443 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fc02406af90 0x7fc02406d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc034035000 tx=0x7fc034008030 comp rx=0 tx=0).stop 2023-10-16T10:09:02.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.996+0000 7fc04a55a640 1 -- 172.21.15.84:0/1292810443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc044070c10 msgr2=0x7fc0440b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:02.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.996+0000 7fc04a55a640 1 --2- 172.21.15.84:0/1292810443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc044070c10 0x7fc0440b3f40 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00c390 tx=0x7fc02c00c850 comp rx=0 tx=0).stop 2023-10-16T10:09:02.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.996+0000 7fc04a55a640 1 -- 172.21.15.84:0/1292810443 shutdown_connections 2023-10-16T10:09:02.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.996+0000 7fc04a55a640 1 --2- 172.21.15.84:0/1292810443 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fc02406af90 0x7fc02406d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.996+0000 7fc04a55a640 1 --2- 172.21.15.84:0/1292810443 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0440cf5f0 0x7fc0440b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.996+0000 7fc04a55a640 1 --2- 172.21.15.84:0/1292810443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc044070c10 0x7fc0440b3f40 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:02.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.996+0000 7fc04a55a640 1 --2- 172.21.15.84:0/1292810443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc044070830 0x7fc04414a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:03.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.996+0000 7fc04a55a640 1 -- 172.21.15.84:0/1292810443 >> 172.21.15.84:0/1292810443 conn(0x7fc0440cfac0 msgr2=0x7fc0440ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:03.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.996+0000 7fc04a55a640 1 -- 172.21.15.84:0/1292810443 shutdown_connections 2023-10-16T10:09:03.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:02.996+0000 7fc04a55a640 1 -- 172.21.15.84:0/1292810443 wait complete. 2023-10-16T10:09:03.015 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-10-16T10:09:03.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.214+0000 7f6848e2d640 1 Processor -- start 2023-10-16T10:09:03.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.215+0000 7f6848e2d640 1 -- start start 2023-10-16T10:09:03.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.216+0000 7f6848e2d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6844070fd0 0x7f68440b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:03.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.216+0000 7f6848e2d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6844071450 con 0x7f68440adb50 2023-10-16T10:09:03.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.216+0000 7f6848e2d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68440cc600 con 0x7f68440adf30 2023-10-16T10:09:03.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.216+0000 7f6848e2d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68440cc8e0 con 0x7f6844070fd0 2023-10-16T10:09:03.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.216+0000 7f684259c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f68440adb50 0x7f684406f550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34198/0 (socket says 172.21.15.84:34198) 2023-10-16T10:09:03.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.217+0000 7f684259c640 1 -- 172.21.15.84:0/3916412516 learned_addr learned my addr 172.21.15.84:0/3916412516 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:09:03.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.217+0000 7f684159a640 1 -- 172.21.15.84:0/3916412516 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327947296 0 0) 0x7f6844071450 con 0x7f68440adb50 2023-10-16T10:09:03.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.217+0000 7f6841d9b640 1 --2- 172.21.15.84:0/3916412516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6844070fd0 0x7f68440b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:03.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.217+0000 7f684159a640 1 -- 172.21.15.84:0/3916412516 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6830003520 con 0x7f68440adb50 2023-10-16T10:09:03.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.217+0000 7f684159a640 1 -- 172.21.15.84:0/3916412516 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 970947492 0 0) 0x7f6830003520 con 0x7f68440adb50 2023-10-16T10:09:03.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.217+0000 7f684159a640 1 -- 172.21.15.84:0/3916412516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6844070fd0 msgr2=0x7f68440b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:03.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.217+0000 7f684159a640 1 --2- 172.21.15.84:0/3916412516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6844070fd0 0x7f68440b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:03.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.217+0000 7f684159a640 1 -- 172.21.15.84:0/3916412516 >> v1:172.21.15.157:6789/0 conn(0x7f68440adf30 legacy=0x7f6844164f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:03.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.217+0000 7f684159a640 1 -- 172.21.15.84:0/3916412516 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68440ccbc0 con 0x7f68440adb50 2023-10-16T10:09:03.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.217+0000 7f684159a640 1 -- 172.21.15.84:0/3916412516 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f68340019f0 con 0x7f68440adb50 2023-10-16T10:09:03.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.217+0000 7f6841d9b640 1 --2- 172.21.15.84:0/3916412516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6844070fd0 0x7f68440b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:09:03.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.218+0000 7f684159a640 1 -- 172.21.15.84:0/3916412516 >> v1:172.21.15.84:6789/0 conn(0x7f68440adb50 legacy=0x7f684406f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:03.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.218+0000 7f684159a640 1 --2- 172.21.15.84:0/3916412516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6844070fd0 0x7f68300039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:03.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.218+0000 7f684159a640 1 --2- 172.21.15.84:0/3916412516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6830003f90 0x7f68300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:03.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.218+0000 7f684159a640 1 --2- 172.21.15.84:0/3916412516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6830004d00 0x7f6830005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:03.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.218+0000 7f684159a640 1 -- 172.21.15.84:0/3916412516 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68440cc8e0 con 0x7f6830004d00 2023-10-16T10:09:03.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.218+0000 7f684159a640 1 -- 172.21.15.84:0/3916412516 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683000bae0 con 0x7f6830003f90 2023-10-16T10:09:03.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.218+0000 7f684159a640 1 -- 172.21.15.84:0/3916412516 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683000bca0 con 0x7f6844070fd0 2023-10-16T10:09:03.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.218+0000 7f6842d9d640 1 --2- 172.21.15.84:0/3916412516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6830004d00 0x7f6830005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:03.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.218+0000 7f6842d9d640 1 -- 172.21.15.84:0/3916412516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6844070fd0 msgr2=0x7f68300039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:03.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.218+0000 7f6842d9d640 1 --2- 172.21.15.84:0/3916412516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6844070fd0 0x7f68300039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:03.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.218+0000 7f6842d9d640 1 -- 172.21.15.84:0/3916412516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6830003f90 msgr2=0x7f68300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:03.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.218+0000 7f6842d9d640 1 --2- 172.21.15.84:0/3916412516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6830003f90 0x7f68300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:03.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.218+0000 7f6842d9d640 1 -- 172.21.15.84:0/3916412516 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6830003520 con 0x7f6830004d00 2023-10-16T10:09:03.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.219+0000 7f6842d9d640 1 --2- 172.21.15.84:0/3916412516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6830004d00 0x7f6830005180 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f683c001750 tx=0x7f68300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e4c155710ef632 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:03.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.220+0000 7f684159a640 1 -- 172.21.15.84:0/3916412516 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c0052d0 con 0x7f6830004d00 2023-10-16T10:09:03.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.220+0000 7f684159a640 1 -- 172.21.15.84:0/3916412516 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683c02e050 con 0x7f6830004d00 2023-10-16T10:09:03.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.220+0000 7f684159a640 1 -- 172.21.15.84:0/3916412516 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c003830 con 0x7f6830004d00 2023-10-16T10:09:03.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.220+0000 7f6848e2d640 1 -- 172.21.15.84:0/3916412516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6830004d00 msgr2=0x7f6830005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:03.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.220+0000 7f6848e2d640 1 --2- 172.21.15.84:0/3916412516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6830004d00 0x7f6830005180 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f683c001750 tx=0x7f68300057f0 comp rx=0 tx=0).stop 2023-10-16T10:09:03.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.220+0000 7f6842d9d640 1 -- 172.21.15.84:0/3916412516 reap_dead start 2023-10-16T10:09:03.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.220+0000 7f6848e2d640 1 -- 172.21.15.84:0/3916412516 shutdown_connections 2023-10-16T10:09:03.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.220+0000 7f6848e2d640 1 -- 172.21.15.84:0/3916412516 >> 172.21.15.84:0/3916412516 conn(0x7f68440cb410 msgr2=0x7f68440cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:03.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.220+0000 7f6848e2d640 1 -- 172.21.15.84:0/3916412516 shutdown_connections 2023-10-16T10:09:03.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.220+0000 7f6848e2d640 1 -- 172.21.15.84:0/3916412516 wait complete. 2023-10-16T10:09:03.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.222+0000 7f6848e2d640 1 Processor -- start 2023-10-16T10:09:03.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.222+0000 7f6848e2d640 1 -- start start 2023-10-16T10:09:03.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.223+0000 7f6848e2d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6830004d00 0x7f68441f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:03.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.223+0000 7f6848e2d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68440b1590 0x7f68441f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:03.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.223+0000 7f6848e2d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68440b3860 0x7f68440b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:03.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.223+0000 7f6848e2d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68441f7cf0 con 0x7f68440b3860 2023-10-16T10:09:03.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.223+0000 7f6848e2d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6844164740 con 0x7f68440b1590 2023-10-16T10:09:03.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.223+0000 7f6848e2d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68440b26f0 con 0x7f6830004d00 2023-10-16T10:09:03.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.223+0000 7f6842d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68440b3860 0x7f68440b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:03.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.223+0000 7f6842d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68440b3860 0x7f68440b3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51830/0 (socket says 172.21.15.84:51830) 2023-10-16T10:09:03.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.223+0000 7f6842d9d640 1 -- 172.21.15.84:0/1431858111 learned_addr learned my addr 172.21.15.84:0/1431858111 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:03.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.224+0000 7f6842d9d640 1 -- 172.21.15.84:0/1431858111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6830004d00 msgr2=0x7f68441f77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:03.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.224+0000 7f6842d9d640 1 --2- 172.21.15.84:0/1431858111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6830004d00 0x7f68441f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:03.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.224+0000 7f6842d9d640 1 -- 172.21.15.84:0/1431858111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68440b1590 msgr2=0x7f68441f4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:03.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.224+0000 7f6842d9d640 1 --2- 172.21.15.84:0/1431858111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68440b1590 0x7f68441f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:03.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.224+0000 7f6842d9d640 1 -- 172.21.15.84:0/1431858111 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68440b4220 con 0x7f68440b3860 2023-10-16T10:09:03.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.224+0000 7f6842d9d640 1 --2- 172.21.15.84:0/1431858111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68440b3860 0x7f68440b3ce0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f683c001870 tx=0x7f683c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:03.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.224+0000 7f683affd640 1 -- 172.21.15.84:0/1431858111 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c043070 con 0x7f68440b3860 2023-10-16T10:09:03.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.224+0000 7f683affd640 1 -- 172.21.15.84:0/1431858111 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683c007a30 con 0x7f68440b3860 2023-10-16T10:09:03.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.224+0000 7f683affd640 1 -- 172.21.15.84:0/1431858111 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c0067d0 con 0x7f68440b3860 2023-10-16T10:09:03.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.224+0000 7f6848e2d640 1 -- 172.21.15.84:0/1431858111 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68440b1ae0 con 0x7f68440b3860 2023-10-16T10:09:03.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.225+0000 7f6848e2d640 1 -- 172.21.15.84:0/1431858111 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68440b20a0 con 0x7f68440b3860 2023-10-16T10:09:03.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.226+0000 7f6848e2d640 1 -- 172.21.15.84:0/1431858111 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684404e160 con 0x7f68440b3860 2023-10-16T10:09:03.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.227+0000 7f683affd640 1 -- 172.21.15.84:0/1431858111 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f683c006950 con 0x7f68440b3860 2023-10-16T10:09:03.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.228+0000 7f683affd640 1 --2- 172.21.15.84:0/1431858111 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f682406afc0 0x7f682406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:03.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.228+0000 7f683affd640 1 -- 172.21.15.84:0/1431858111 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f683c0aaaf0 con 0x7f68440b3860 2023-10-16T10:09:03.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.228+0000 7f684259c640 1 --2- 172.21.15.84:0/1431858111 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f682406afc0 0x7f682406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:03.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.229+0000 7f684259c640 1 --2- 172.21.15.84:0/1431858111 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f682406afc0 0x7f682406d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6834001560 tx=0x7f68340018a0 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:03.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.233+0000 7f683affd640 1 -- 172.21.15.84:0/1431858111 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f683c0758e0 con 0x7f68440b3860 2023-10-16T10:09:03.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:03.487+0000 7f6848e2d640 1 -- 172.21.15.84:0/1431858111 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f68440b2860 con 0x7f68440b3860 2023-10-16T10:09:04.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.121+0000 7f683affd640 1 -- 172.21.15.84:0/1431858111 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v779) v1 ==== 181+0+0 (secure 0 0 0) 0x7f683c078f00 con 0x7f68440b3860 2023-10-16T10:09:04.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.125+0000 7f6848e2d640 1 -- 172.21.15.84:0/1431858111 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f682406afc0 msgr2=0x7f682406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:04.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.125+0000 7f6848e2d640 1 --2- 172.21.15.84:0/1431858111 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f682406afc0 0x7f682406d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6834001560 tx=0x7f68340018a0 comp rx=0 tx=0).stop 2023-10-16T10:09:04.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.125+0000 7f6848e2d640 1 -- 172.21.15.84:0/1431858111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68440b3860 msgr2=0x7f68440b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:04.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.125+0000 7f6848e2d640 1 --2- 172.21.15.84:0/1431858111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68440b3860 0x7f68440b3ce0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f683c001870 tx=0x7f683c001120 comp rx=0 tx=0).stop 2023-10-16T10:09:04.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.125+0000 7f6848e2d640 1 -- 172.21.15.84:0/1431858111 shutdown_connections 2023-10-16T10:09:04.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.125+0000 7f6848e2d640 1 --2- 172.21.15.84:0/1431858111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68440b3860 0x7f68440b3ce0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:04.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.125+0000 7f6848e2d640 1 --2- 172.21.15.84:0/1431858111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68440b1590 0x7f68441f4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:04.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.125+0000 7f6848e2d640 1 --2- 172.21.15.84:0/1431858111 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f682406afc0 0x7f682406d480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:04.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.125+0000 7f6848e2d640 1 --2- 172.21.15.84:0/1431858111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6830004d00 0x7f68441f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:04.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.125+0000 7f6848e2d640 1 -- 172.21.15.84:0/1431858111 >> 172.21.15.84:0/1431858111 conn(0x7f68440cb410 msgr2=0x7f68440ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:04.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.126+0000 7f6848e2d640 1 -- 172.21.15.84:0/1431858111 shutdown_connections 2023-10-16T10:09:04.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.126+0000 7f6848e2d640 1 -- 172.21.15.84:0/1431858111 wait complete. 2023-10-16T10:09:04.150 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T10:09:04.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.363+0000 7f46d7844640 1 Processor -- start 2023-10-16T10:09:04.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.363+0000 7f46d7844640 1 -- start start 2023-10-16T10:09:04.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.364+0000 7f46d7844640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f46d015ad70 0x7f46d0069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:04.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.365+0000 7f46d7844640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46d00caac0 0x7f46d006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:04.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.365+0000 7f46d7844640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d00cb030 con 0x7f46d00c9ca0 2023-10-16T10:09:04.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.365+0000 7f46d7844640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d00c91c0 con 0x7f46d015ad70 2023-10-16T10:09:04.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.365+0000 7f46d7844640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d00c94a0 con 0x7f46d00caac0 2023-10-16T10:09:04.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.365+0000 7f46d55e0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f46d015ad70 0x7f46d0069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:04.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.365+0000 7f46d55e0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f46d015ad70 0x7f46d0069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51832/0 (socket says 172.21.15.84:51832) 2023-10-16T10:09:04.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.365+0000 7f46d55e0640 1 -- 172.21.15.84:0/2779505385 learned_addr learned my addr 172.21.15.84:0/2779505385 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:04.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.365+0000 7f46d55e0640 1 -- 172.21.15.84:0/2779505385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46d00caac0 msgr2=0x7f46d006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:04.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.365+0000 7f46d55e0640 1 --2- 172.21.15.84:0/2779505385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46d00caac0 0x7f46d006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:04.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.365+0000 7f46d55e0640 1 -- 172.21.15.84:0/2779505385 >> v1:172.21.15.157:6789/0 conn(0x7f46d00c9ca0 legacy=0x7f46d00ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:04.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.365+0000 7f46d55e0640 1 -- 172.21.15.84:0/2779505385 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d00c9780 con 0x7f46d015ad70 2023-10-16T10:09:04.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.366+0000 7f46d55e0640 1 --2- 172.21.15.84:0/2779505385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46d015ad70 0x7f46d0069da0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009870 tx=0x7f46cc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f229eb3373a292a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:04.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.367+0000 7f46c7fff640 1 -- 172.21.15.84:0/2779505385 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc031640 con 0x7f46d015ad70 2023-10-16T10:09:04.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.367+0000 7f46c7fff640 1 -- 172.21.15.84:0/2779505385 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc0317a0 con 0x7f46d015ad70 2023-10-16T10:09:04.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.367+0000 7f46c7fff640 1 -- 172.21.15.84:0/2779505385 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc031b30 con 0x7f46d015ad70 2023-10-16T10:09:04.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.367+0000 7f46d7844640 1 -- 172.21.15.84:0/2779505385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46d015ad70 msgr2=0x7f46d0069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:04.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.367+0000 7f46d7844640 1 --2- 172.21.15.84:0/2779505385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46d015ad70 0x7f46d0069da0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009870 tx=0x7f46cc02b8f0 comp rx=0 tx=0).stop 2023-10-16T10:09:04.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.368+0000 7f46d7844640 1 -- 172.21.15.84:0/2779505385 shutdown_connections 2023-10-16T10:09:04.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.368+0000 7f46d7844640 1 --2- 172.21.15.84:0/2779505385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46d00caac0 0x7f46d006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:04.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.368+0000 7f46d7844640 1 --2- 172.21.15.84:0/2779505385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46d015ad70 0x7f46d0069da0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:04.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.368+0000 7f46d7844640 1 -- 172.21.15.84:0/2779505385 >> 172.21.15.84:0/2779505385 conn(0x7f46d00ce9b0 msgr2=0x7f46d00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:04.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.368+0000 7f46d7844640 1 -- 172.21.15.84:0/2779505385 shutdown_connections 2023-10-16T10:09:04.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.368+0000 7f46d7844640 1 -- 172.21.15.84:0/2779505385 wait complete. 2023-10-16T10:09:04.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.369+0000 7f46d7844640 1 Processor -- start 2023-10-16T10:09:04.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.369+0000 7f46d7844640 1 -- start start 2023-10-16T10:09:04.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.370+0000 7f46d7844640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46d00c9ca0 0x7f46d015c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:04.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.370+0000 7f46d7844640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46d00caac0 0x7f46d0159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:04.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.370+0000 7f46d7844640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46d015ad70 0x7f46d01f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:04.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.370+0000 7f46d7844640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d01f6b90 con 0x7f46d00c9ca0 2023-10-16T10:09:04.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.370+0000 7f46d7844640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d015e3b0 con 0x7f46d00caac0 2023-10-16T10:09:04.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.370+0000 7f46d7844640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d015db10 con 0x7f46d015ad70 2023-10-16T10:09:04.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.370+0000 7f46d55e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46d00c9ca0 0x7f46d015c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:04.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.371+0000 7f46d55e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46d00c9ca0 0x7f46d015c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51848/0 (socket says 172.21.15.84:51848) 2023-10-16T10:09:04.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.371+0000 7f46d55e0640 1 -- 172.21.15.84:0/3621769015 learned_addr learned my addr 172.21.15.84:0/3621769015 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:04.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.371+0000 7f46d55e0640 1 -- 172.21.15.84:0/3621769015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46d015ad70 msgr2=0x7f46d01f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:04.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.371+0000 7f46d55e0640 1 --2- 172.21.15.84:0/3621769015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46d015ad70 0x7f46d01f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:04.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.371+0000 7f46d55e0640 1 -- 172.21.15.84:0/3621769015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46d00caac0 msgr2=0x7f46d0159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:04.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.371+0000 7f46d55e0640 1 --2- 172.21.15.84:0/3621769015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46d00caac0 0x7f46d0159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:04.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.371+0000 7f46d55e0640 1 -- 172.21.15.84:0/3621769015 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc0094d0 con 0x7f46d00c9ca0 2023-10-16T10:09:04.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.371+0000 7f46d55e0640 1 --2- 172.21.15.84:0/3621769015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46d00c9ca0 0x7f46d015c800 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f46cc000c00 tx=0x7f46cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:04.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.371+0000 7f46c5ffb640 1 -- 172.21.15.84:0/3621769015 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc03ac60 con 0x7f46d00c9ca0 2023-10-16T10:09:04.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.371+0000 7f46c5ffb640 1 -- 172.21.15.84:0/3621769015 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc03adc0 con 0x7f46d00c9ca0 2023-10-16T10:09:04.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.371+0000 7f46c5ffb640 1 -- 172.21.15.84:0/3621769015 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc044630 con 0x7f46d00c9ca0 2023-10-16T10:09:04.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.371+0000 7f46d7844640 1 -- 172.21.15.84:0/3621769015 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d015ddf0 con 0x7f46d00c9ca0 2023-10-16T10:09:04.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.372+0000 7f46d7844640 1 -- 172.21.15.84:0/3621769015 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d015c000 con 0x7f46d00c9ca0 2023-10-16T10:09:04.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.373+0000 7f46d7844640 1 -- 172.21.15.84:0/3621769015 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d015be30 con 0x7f46d00c9ca0 2023-10-16T10:09:04.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.374+0000 7f46c5ffb640 1 -- 172.21.15.84:0/3621769015 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f46cc02f050 con 0x7f46d00c9ca0 2023-10-16T10:09:04.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.374+0000 7f46c5ffb640 1 --2- 172.21.15.84:0/3621769015 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f46b406af90 0x7f46b406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:04.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.374+0000 7f46c5ffb640 1 -- 172.21.15.84:0/3621769015 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f46cc0ab820 con 0x7f46d00c9ca0 2023-10-16T10:09:04.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.375+0000 7f46d4ddf640 1 --2- 172.21.15.84:0/3621769015 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f46b406af90 0x7f46b406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:04.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.375+0000 7f46d4ddf640 1 --2- 172.21.15.84:0/3621769015 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f46b406af90 0x7f46b406d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f46d00c88b0 tx=0x7f46c0007990 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:04.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.378+0000 7f46c5ffb640 1 -- 172.21.15.84:0/3621769015 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f46cc0766b0 con 0x7f46d00c9ca0 2023-10-16T10:09:04.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.557+0000 7f46d7844640 1 -- 172.21.15.84:0/3621769015 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f46d0066de0 con 0x7f46d00c9ca0 2023-10-16T10:09:04.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.563+0000 7f46c5ffb640 1 -- 172.21.15.84:0/3621769015 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7f46cc0ab020 con 0x7f46d00c9ca0 2023-10-16T10:09:04.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.565+0000 7f46d7844640 1 -- 172.21.15.84:0/3621769015 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f46b406af90 msgr2=0x7f46b406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:04.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.565+0000 7f46d7844640 1 --2- 172.21.15.84:0/3621769015 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f46b406af90 0x7f46b406d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f46d00c88b0 tx=0x7f46c0007990 comp rx=0 tx=0).stop 2023-10-16T10:09:04.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.565+0000 7f46d7844640 1 -- 172.21.15.84:0/3621769015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46d00c9ca0 msgr2=0x7f46d015c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:04.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.565+0000 7f46d7844640 1 --2- 172.21.15.84:0/3621769015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46d00c9ca0 0x7f46d015c800 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f46cc000c00 tx=0x7f46cc0046c0 comp rx=0 tx=0).stop 2023-10-16T10:09:04.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.566+0000 7f46d7844640 1 -- 172.21.15.84:0/3621769015 shutdown_connections 2023-10-16T10:09:04.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.566+0000 7f46d7844640 1 --2- 172.21.15.84:0/3621769015 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f46b406af90 0x7f46b406d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:04.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.566+0000 7f46d7844640 1 --2- 172.21.15.84:0/3621769015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46d015ad70 0x7f46d01f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:04.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.566+0000 7f46d7844640 1 --2- 172.21.15.84:0/3621769015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46d00caac0 0x7f46d0159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:04.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.566+0000 7f46d7844640 1 --2- 172.21.15.84:0/3621769015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46d00c9ca0 0x7f46d015c800 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:04.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.566+0000 7f46d7844640 1 -- 172.21.15.84:0/3621769015 >> 172.21.15.84:0/3621769015 conn(0x7f46d00ce9b0 msgr2=0x7f46d00b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:04.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.566+0000 7f46d7844640 1 -- 172.21.15.84:0/3621769015 shutdown_connections 2023-10-16T10:09:04.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:04.566+0000 7f46d7844640 1 -- 172.21.15.84:0/3621769015 wait complete. 2023-10-16T10:09:04.586 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:09:04.586 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:09:04.703 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:09:04.822 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:09:04.822 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:09:04.823 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:09:04.883 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:09:04.884 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:09:04.884 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:09:04.944 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:09:04.944 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:09:04.945 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:09:04.987 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:09:05.000 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:09:05.006 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T10:09:05.007 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T10:09:05.007 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:09:05.095 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T10:09:05.096 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T10:09:05.096 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:09:05.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.313+0000 7f3ee1075640 1 Processor -- start 2023-10-16T10:09:05.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.313+0000 7f3ee1075640 1 -- start start 2023-10-16T10:09:05.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.314+0000 7f3ee1075640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3edc15aaa0 0x7f3edc15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:05.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.315+0000 7f3ee1075640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3edc0ce390 0x7f3edc070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:05.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.315+0000 7f3ee1075640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3edc15ed60 con 0x7f3edc15ae80 2023-10-16T10:09:05.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.315+0000 7f3ee1075640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3edc06b810 con 0x7f3edc0ce390 2023-10-16T10:09:05.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.315+0000 7f3ee1075640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc0707e0 con 0x7f3edc15aaa0 2023-10-16T10:09:05.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.315+0000 7f3edad9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3edc15aaa0 0x7f3edc15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:05.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.315+0000 7f3ed3fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3edc15ae80 0x7f3edc0cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:57292/0 (socket says 172.21.15.84:57292) 2023-10-16T10:09:05.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.315+0000 7f3ed3fff640 1 -- 172.21.15.84:0/2140246012 learned_addr learned my addr 172.21.15.84:0/2140246012 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:09:05.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.315+0000 7f3edb59e640 1 --2- 172.21.15.84:0/2140246012 >> v2:172.21.15.157:3300/0 conn(0x7f3edc0ce390 0x7f3edc070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:05.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.316+0000 7f3edad9d640 1 -- 172.21.15.84:0/2140246012 >> v2:172.21.15.157:3300/0 conn(0x7f3edc0ce390 msgr2=0x7f3edc070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:05.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.316+0000 7f3edad9d640 1 --2- 172.21.15.84:0/2140246012 >> v2:172.21.15.157:3300/0 conn(0x7f3edc0ce390 0x7f3edc070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:05.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.316+0000 7f3edad9d640 1 -- 172.21.15.84:0/2140246012 >> v1:172.21.15.157:6789/0 conn(0x7f3edc15ae80 legacy=0x7f3edc0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:05.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.316+0000 7f3edad9d640 1 -- 172.21.15.84:0/2140246012 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3edc15ef90 con 0x7f3edc15aaa0 2023-10-16T10:09:05.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.317+0000 7f3edad9d640 1 --2- 172.21.15.84:0/2140246012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3edc15aaa0 0x7f3edc15e820 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0097a0 tx=0x7f3ecc02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5148731f91dbbf94 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:05.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.318+0000 7f3eda59c640 1 -- 172.21.15.84:0/2140246012 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc031830 con 0x7f3edc15aaa0 2023-10-16T10:09:05.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.318+0000 7f3eda59c640 1 -- 172.21.15.84:0/2140246012 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc031990 con 0x7f3edc15aaa0 2023-10-16T10:09:05.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.318+0000 7f3eda59c640 1 -- 172.21.15.84:0/2140246012 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc031d20 con 0x7f3edc15aaa0 2023-10-16T10:09:05.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.319+0000 7f3ee1075640 1 -- 172.21.15.84:0/2140246012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3edc15aaa0 msgr2=0x7f3edc15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:05.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.319+0000 7f3ee1075640 1 --2- 172.21.15.84:0/2140246012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3edc15aaa0 0x7f3edc15e820 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0097a0 tx=0x7f3ecc02b8d0 comp rx=0 tx=0).stop 2023-10-16T10:09:05.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.319+0000 7f3ee1075640 1 -- 172.21.15.84:0/2140246012 shutdown_connections 2023-10-16T10:09:05.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.319+0000 7f3ee1075640 1 --2- 172.21.15.84:0/2140246012 >> v2:172.21.15.157:3300/0 conn(0x7f3edc0ce390 0x7f3edc070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:05.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.319+0000 7f3ee1075640 1 --2- 172.21.15.84:0/2140246012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3edc15aaa0 0x7f3edc15e820 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:05.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.319+0000 7f3ee1075640 1 -- 172.21.15.84:0/2140246012 >> 172.21.15.84:0/2140246012 conn(0x7f3edc0cb410 msgr2=0x7f3edc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:05.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.319+0000 7f3ee1075640 1 -- 172.21.15.84:0/2140246012 shutdown_connections 2023-10-16T10:09:05.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.319+0000 7f3ee1075640 1 -- 172.21.15.84:0/2140246012 wait complete. 2023-10-16T10:09:05.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.321+0000 7f3ee1075640 1 Processor -- start 2023-10-16T10:09:05.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.321+0000 7f3ee1075640 1 -- start start 2023-10-16T10:09:05.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.322+0000 7f3ee1075640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3edc0ce390 0x7f3edc155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:05.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.322+0000 7f3ee1075640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3edc15aaa0 0x7f3edc0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:05.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.322+0000 7f3ee1075640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3edc15ae80 0x7f3edc0acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:05.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.322+0000 7f3ee1075640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc1e5310 con 0x7f3edc15aaa0 2023-10-16T10:09:05.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.322+0000 7f3ee1075640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc1e55f0 con 0x7f3edc0ce390 2023-10-16T10:09:05.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.322+0000 7f3ee1075640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc1e58d0 con 0x7f3edc15ae80 2023-10-16T10:09:05.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.322+0000 7f3ed3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3edc15aaa0 0x7f3edc0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:05.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.322+0000 7f3ed3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3edc15aaa0 0x7f3edc0ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51860/0 (socket says 172.21.15.84:51860) 2023-10-16T10:09:05.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.322+0000 7f3ed3fff640 1 -- 172.21.15.84:0/1282800781 learned_addr learned my addr 172.21.15.84:0/1282800781 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:05.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.322+0000 7f3edad9d640 1 --2- 172.21.15.84:0/1282800781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3edc0ce390 0x7f3edc155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:05.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.322+0000 7f3edb59e640 1 --2- 172.21.15.84:0/1282800781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3edc15ae80 0x7f3edc0acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:05.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.322+0000 7f3ed3fff640 1 -- 172.21.15.84:0/1282800781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3edc15ae80 msgr2=0x7f3edc0acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:05.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.322+0000 7f3ed3fff640 1 --2- 172.21.15.84:0/1282800781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3edc15ae80 0x7f3edc0acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:05.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.322+0000 7f3ed3fff640 1 -- 172.21.15.84:0/1282800781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3edc0ce390 msgr2=0x7f3edc155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:05.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.322+0000 7f3ed3fff640 1 --2- 172.21.15.84:0/1282800781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3edc0ce390 0x7f3edc155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:05.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.323+0000 7f3ed3fff640 1 -- 172.21.15.84:0/1282800781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc009400 con 0x7f3edc15aaa0 2023-10-16T10:09:05.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.323+0000 7f3ed3fff640 1 --2- 172.21.15.84:0/1282800781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3edc15aaa0 0x7f3edc0ac890 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400c4a0 tx=0x7f3ec400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:05.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.323+0000 7f3ed37fe640 1 -- 172.21.15.84:0/1282800781 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4005ce0 con 0x7f3edc15aaa0 2023-10-16T10:09:05.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.323+0000 7f3ed37fe640 1 -- 172.21.15.84:0/1282800781 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec4005e40 con 0x7f3edc15aaa0 2023-10-16T10:09:05.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.323+0000 7f3ed37fe640 1 -- 172.21.15.84:0/1282800781 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4007270 con 0x7f3edc15aaa0 2023-10-16T10:09:05.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.323+0000 7f3ee1075640 1 -- 172.21.15.84:0/1282800781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3edc1e5bb0 con 0x7f3edc15aaa0 2023-10-16T10:09:05.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.323+0000 7f3ee1075640 1 -- 172.21.15.84:0/1282800781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3edc1e4fc0 con 0x7f3edc15aaa0 2023-10-16T10:09:05.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.324+0000 7f3ee1075640 1 -- 172.21.15.84:0/1282800781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3edc04e160 con 0x7f3edc15aaa0 2023-10-16T10:09:05.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.325+0000 7f3ed37fe640 1 -- 172.21.15.84:0/1282800781 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3ec4007490 con 0x7f3edc15aaa0 2023-10-16T10:09:05.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.326+0000 7f3ed37fe640 1 --2- 172.21.15.84:0/1282800781 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f3ebc06af90 0x7f3ebc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:05.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.326+0000 7f3ed37fe640 1 -- 172.21.15.84:0/1282800781 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f3ec4088c70 con 0x7f3edc15aaa0 2023-10-16T10:09:05.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.327+0000 7f3edad9d640 1 --2- 172.21.15.84:0/1282800781 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f3ebc06af90 0x7f3ebc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:05.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.327+0000 7f3edad9d640 1 --2- 172.21.15.84:0/1282800781 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f3ebc06af90 0x7f3ebc06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc036000 tx=0x7f3ecc03c040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:05.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.331+0000 7f3ed37fe640 1 -- 172.21.15.84:0/1282800781 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3ec4053c20 con 0x7f3edc15aaa0 2023-10-16T10:09:05.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.561+0000 7f3ee1075640 1 -- 172.21.15.84:0/1282800781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3edc0cdc60 con 0x7f3edc15aaa0 2023-10-16T10:09:05.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.562+0000 7f3ed37fe640 1 -- 172.21.15.84:0/1282800781 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f3ec4017020 con 0x7f3edc15aaa0 2023-10-16T10:09:05.564 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:09:05.564 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":73,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4118285132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":4118285132},{"type":"v1","addr":"172.21.15.84:6837","nonce":4118285132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3743702675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3743702675},{"type":"v1","addr":"172.21.15.157:6834","nonce":3743702675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2895418246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2895418246},{"type":"v1","addr":"172.21.15.157:6835","nonce":2895418246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":9051,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/3216113668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3216113668},{"type":"v1","addr":"172.21.15.84:6836","nonce":3216113668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[]} 2023-10-16T10:09:05.564 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 73 2023-10-16T10:09:05.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.565+0000 7f3ee1075640 1 -- 172.21.15.84:0/1282800781 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f3ebc06af90 msgr2=0x7f3ebc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:05.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.565+0000 7f3ee1075640 1 --2- 172.21.15.84:0/1282800781 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f3ebc06af90 0x7f3ebc06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc036000 tx=0x7f3ecc03c040 comp rx=0 tx=0).stop 2023-10-16T10:09:05.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.565+0000 7f3ee1075640 1 -- 172.21.15.84:0/1282800781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3edc15aaa0 msgr2=0x7f3edc0ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:05.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.565+0000 7f3ee1075640 1 --2- 172.21.15.84:0/1282800781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3edc15aaa0 0x7f3edc0ac890 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400c4a0 tx=0x7f3ec400c960 comp rx=0 tx=0).stop 2023-10-16T10:09:05.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.565+0000 7f3ee1075640 1 -- 172.21.15.84:0/1282800781 shutdown_connections 2023-10-16T10:09:05.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.565+0000 7f3ee1075640 1 --2- 172.21.15.84:0/1282800781 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f3ebc06af90 0x7f3ebc06d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:05.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.565+0000 7f3ee1075640 1 --2- 172.21.15.84:0/1282800781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3edc15ae80 0x7f3edc0acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:05.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.565+0000 7f3ee1075640 1 --2- 172.21.15.84:0/1282800781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3edc15aaa0 0x7f3edc0ac890 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:05.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.565+0000 7f3ee1075640 1 --2- 172.21.15.84:0/1282800781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3edc0ce390 0x7f3edc155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:05.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.565+0000 7f3ee1075640 1 -- 172.21.15.84:0/1282800781 >> 172.21.15.84:0/1282800781 conn(0x7f3edc0cb410 msgr2=0x7f3edc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:05.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.565+0000 7f3ee1075640 1 -- 172.21.15.84:0/1282800781 shutdown_connections 2023-10-16T10:09:05.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.565+0000 7f3ee1075640 1 -- 172.21.15.84:0/1282800781 wait complete. 2023-10-16T10:09:05.586 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T10:09:05.587 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T10:09:05.587 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T10:09:05.587 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:09:05.587 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T10:09:05.588 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T10:09:05.588 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T10:09:05.588 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:09:05.589 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T10:09:05.589 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T10:09:05.589 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T10:09:05.589 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:09:05.590 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T10:09:05.590 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T10:09:05.590 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T10:09:05.590 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:09:05.603 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T10:09:05.603 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T10:09:05.604 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T10:09:05.605 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T10:09:05.612 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T10:09:05.612 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T10:09:05.613 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T10:09:05.613 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T10:09:05.652 INFO:tasks.ceph.mds.a:Started 2023-10-16T10:09:05.653 DEBUG:teuthology.parallel:result is None 2023-10-16T10:09:05.660 INFO:tasks.ceph.mds.c:Started 2023-10-16T10:09:05.660 DEBUG:teuthology.parallel:result is None 2023-10-16T10:09:05.661 INFO:tasks.ceph.mds.d:Started 2023-10-16T10:09:05.662 DEBUG:teuthology.parallel:result is None 2023-10-16T10:09:05.668 INFO:tasks.ceph.mds.b:Started 2023-10-16T10:09:05.669 DEBUG:teuthology.parallel:result is None 2023-10-16T10:09:05.669 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T10:09:05.818 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T10:09:05.818 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T10:09:05.830 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T10:09:05.832 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-10-16T10:09:05.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.858+0000 7fbc104d4640 1 Processor -- start 2023-10-16T10:09:05.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.858+0000 7fbc104d4640 1 -- start start 2023-10-16T10:09:05.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.860+0000 7fbc104d4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbc08159ab0 0x7fbc080ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:05.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.860+0000 7fbc104d4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc08070360 con 0x7fbc0806c140 2023-10-16T10:09:05.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.860+0000 7fbc104d4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc080cd1f0 con 0x7fbc08159e90 2023-10-16T10:09:05.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.860+0000 7fbc104d4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc0815d890 con 0x7fbc08159ab0 2023-10-16T10:09:05.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.860+0000 7fbc0e270640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbc08159ab0 0x7fbc080ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:05.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.860+0000 7fbc0ea71640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbc0806c140 0x7fbc0815dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55910/0 (socket says 172.21.15.84:55910) 2023-10-16T10:09:05.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.860+0000 7fbc0e270640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbc08159ab0 0x7fbc080ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51200/0 (socket says 172.21.15.84:51200) 2023-10-16T10:09:05.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.860+0000 7fbc0ea71640 1 -- 172.21.15.84:0/3739615308 learned_addr learned my addr 172.21.15.84:0/3739615308 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:09:05.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.860+0000 7fbc0e270640 1 -- 172.21.15.84:0/3739615308 >> v1:172.21.15.157:6789/0 conn(0x7fbc08159e90 legacy=0x7fbc0806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:05.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.860+0000 7fbc0e270640 1 -- 172.21.15.84:0/3739615308 >> v1:172.21.15.84:6789/0 conn(0x7fbc0806c140 legacy=0x7fbc0815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:05.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.860+0000 7fbc0e270640 1 -- 172.21.15.84:0/3739615308 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc080cd4d0 con 0x7fbc08159ab0 2023-10-16T10:09:05.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.861+0000 7fbc0e270640 1 --2- 172.21.15.84:0/3739615308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08159ab0 0x7fbc080ce370 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fbc040097a0 tx=0x7fbc0402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c37bfa9cf35ee9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:05.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.862+0000 7fbc0d26e640 1 -- 172.21.15.84:0/3739615308 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0402e070 con 0x7fbc08159ab0 2023-10-16T10:09:05.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.862+0000 7fbc0d26e640 1 -- 172.21.15.84:0/3739615308 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc040315a0 con 0x7fbc08159ab0 2023-10-16T10:09:05.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.862+0000 7fbc0d26e640 1 -- 172.21.15.84:0/3739615308 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc04031930 con 0x7fbc08159ab0 2023-10-16T10:09:05.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.862+0000 7fbc104d4640 1 -- 172.21.15.84:0/3739615308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08159ab0 msgr2=0x7fbc080ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:05.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.862+0000 7fbc104d4640 1 --2- 172.21.15.84:0/3739615308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08159ab0 0x7fbc080ce370 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fbc040097a0 tx=0x7fbc0402b840 comp rx=0 tx=0).stop 2023-10-16T10:09:05.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.862+0000 7fbc104d4640 1 -- 172.21.15.84:0/3739615308 shutdown_connections 2023-10-16T10:09:05.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.862+0000 7fbc104d4640 1 --2- 172.21.15.84:0/3739615308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08159ab0 0x7fbc080ce370 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:05.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.862+0000 7fbc104d4640 1 -- 172.21.15.84:0/3739615308 >> 172.21.15.84:0/3739615308 conn(0x7fbc080ce9b0 msgr2=0x7fbc080cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:05.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.862+0000 7fbc104d4640 1 -- 172.21.15.84:0/3739615308 shutdown_connections 2023-10-16T10:09:05.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.862+0000 7fbc104d4640 1 -- 172.21.15.84:0/3739615308 wait complete. 2023-10-16T10:09:05.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.864+0000 7fbc104d4640 1 Processor -- start 2023-10-16T10:09:05.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.864+0000 7fbc104d4640 1 -- start start 2023-10-16T10:09:05.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.865+0000 7fbc104d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc0806c140 0x7fbc0815bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:05.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.865+0000 7fbc104d4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08159ab0 0x7fbc08148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:05.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.865+0000 7fbc104d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc08159e90 0x7fbc08148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:05.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.865+0000 7fbc104d4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc081575c0 con 0x7fbc08159ab0 2023-10-16T10:09:05.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.865+0000 7fbc104d4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc081569b0 con 0x7fbc0806c140 2023-10-16T10:09:05.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.865+0000 7fbc104d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc08156c90 con 0x7fbc08159e90 2023-10-16T10:09:05.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.865+0000 7fbc0da6f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08159ab0 0x7fbc08148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:05.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.865+0000 7fbc0da6f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08159ab0 0x7fbc08148850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51208/0 (socket says 172.21.15.84:51208) 2023-10-16T10:09:05.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.865+0000 7fbc0da6f640 1 -- 172.21.15.84:0/1065902167 learned_addr learned my addr 172.21.15.84:0/1065902167 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:05.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.865+0000 7fbc0e270640 1 --2- 172.21.15.84:0/1065902167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc0806c140 0x7fbc0815bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:05.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.866+0000 7fbc0ea71640 1 --2- 172.21.15.84:0/1065902167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc08159e90 0x7fbc08148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:05.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.866+0000 7fbc0da6f640 1 -- 172.21.15.84:0/1065902167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc08159e90 msgr2=0x7fbc08148d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:05.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.866+0000 7fbc0da6f640 1 --2- 172.21.15.84:0/1065902167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc08159e90 0x7fbc08148d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:05.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.866+0000 7fbc0da6f640 1 -- 172.21.15.84:0/1065902167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc0806c140 msgr2=0x7fbc0815bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:05.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.866+0000 7fbc0da6f640 1 --2- 172.21.15.84:0/1065902167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc0806c140 0x7fbc0815bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:05.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.866+0000 7fbc0da6f640 1 -- 172.21.15.84:0/1065902167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc04009400 con 0x7fbc08159ab0 2023-10-16T10:09:05.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.866+0000 7fbc0da6f640 1 --2- 172.21.15.84:0/1065902167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08159ab0 0x7fbc08148850 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800c450 tx=0x7fbbf800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:05.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.866+0000 7fbbfeffd640 1 -- 172.21.15.84:0/1065902167 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80052d0 con 0x7fbc08159ab0 2023-10-16T10:09:05.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.866+0000 7fbbfeffd640 1 -- 172.21.15.84:0/1065902167 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf8005430 con 0x7fbc08159ab0 2023-10-16T10:09:05.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.866+0000 7fbbfeffd640 1 -- 172.21.15.84:0/1065902167 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8012870 con 0x7fbc08159ab0 2023-10-16T10:09:05.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.866+0000 7fbc104d4640 1 -- 172.21.15.84:0/1065902167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc08156f70 con 0x7fbc08159ab0 2023-10-16T10:09:05.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.867+0000 7fbc104d4640 1 -- 172.21.15.84:0/1065902167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc0815c4e0 con 0x7fbc08159ab0 2023-10-16T10:09:05.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.868+0000 7fbc104d4640 1 -- 172.21.15.84:0/1065902167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc0804e160 con 0x7fbc08159ab0 2023-10-16T10:09:05.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.869+0000 7fbbfeffd640 1 -- 172.21.15.84:0/1065902167 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbbf80129d0 con 0x7fbc08159ab0 2023-10-16T10:09:05.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.870+0000 7fbbfeffd640 1 --2- 172.21.15.84:0/1065902167 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fbbec06af90 0x7fbbec06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:05.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.870+0000 7fbbfeffd640 1 -- 172.21.15.84:0/1065902167 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fbbf8089270 con 0x7fbc08159ab0 2023-10-16T10:09:05.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.870+0000 7fbc0e270640 1 --2- 172.21.15.84:0/1065902167 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fbbec06af90 0x7fbbec06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:05.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.871+0000 7fbc0e270640 1 --2- 172.21.15.84:0/1065902167 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fbbec06af90 0x7fbbec06d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbc04036000 tx=0x7fbc0403d040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:05.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:05.874+0000 7fbbfeffd640 1 -- 172.21.15.84:0/1065902167 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbbf8054220 con 0x7fbc08159ab0 2023-10-16T10:09:06.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.075+0000 7fbc104d4640 1 -- 172.21.15.84:0/1065902167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbc080ccb60 con 0x7fbc08159ab0 2023-10-16T10:09:06.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.142+0000 7fbbfeffd640 1 -- 172.21.15.84:0/1065902167 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v145) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbbf8057840 con 0x7fbc08159ab0 2023-10-16T10:09:06.144 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-10-16T10:09:06.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.145+0000 7fbc104d4640 1 -- 172.21.15.84:0/1065902167 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fbbec06af90 msgr2=0x7fbbec06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:06.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.145+0000 7fbc104d4640 1 --2- 172.21.15.84:0/1065902167 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fbbec06af90 0x7fbbec06d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbc04036000 tx=0x7fbc0403d040 comp rx=0 tx=0).stop 2023-10-16T10:09:06.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.146+0000 7fbc104d4640 1 -- 172.21.15.84:0/1065902167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08159ab0 msgr2=0x7fbc08148850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:06.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.146+0000 7fbc104d4640 1 --2- 172.21.15.84:0/1065902167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08159ab0 0x7fbc08148850 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800c450 tx=0x7fbbf800c910 comp rx=0 tx=0).stop 2023-10-16T10:09:06.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.146+0000 7fbc104d4640 1 -- 172.21.15.84:0/1065902167 shutdown_connections 2023-10-16T10:09:06.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.146+0000 7fbc104d4640 1 --2- 172.21.15.84:0/1065902167 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fbbec06af90 0x7fbbec06d450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.146+0000 7fbc104d4640 1 --2- 172.21.15.84:0/1065902167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc08159e90 0x7fbc08148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.146+0000 7fbc104d4640 1 --2- 172.21.15.84:0/1065902167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08159ab0 0x7fbc08148850 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.146+0000 7fbc104d4640 1 --2- 172.21.15.84:0/1065902167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc0806c140 0x7fbc0815bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.146+0000 7fbc104d4640 1 -- 172.21.15.84:0/1065902167 >> 172.21.15.84:0/1065902167 conn(0x7fbc080ce9b0 msgr2=0x7fbc08159130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:06.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.146+0000 7fbc104d4640 1 -- 172.21.15.84:0/1065902167 shutdown_connections 2023-10-16T10:09:06.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.146+0000 7fbc104d4640 1 -- 172.21.15.84:0/1065902167 wait complete. 2023-10-16T10:09:06.169 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T10:09:06.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.370+0000 7ff83bfff640 1 Processor -- start 2023-10-16T10:09:06.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.370+0000 7ff83bfff640 1 -- start start 2023-10-16T10:09:06.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.370+0000 7ff83bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff83c0ad970 0x7ff83c0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:06.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.370+0000 7ff83bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff83c0ccfb0 0x7ff83c0cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:06.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.370+0000 7ff83bfff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff83c15b9d0 con 0x7ff83c06c040 2023-10-16T10:09:06.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.370+0000 7ff83bfff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff83c15bcb0 con 0x7ff83c0ccfb0 2023-10-16T10:09:06.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.370+0000 7ff83bfff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff83c15bf90 con 0x7ff83c0ad970 2023-10-16T10:09:06.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.371+0000 7ff83b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff83c0ccfb0 0x7ff83c0cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:06.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.371+0000 7ff83affd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff83c06c040 0x7ff83c06c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55922/0 (socket says 172.21.15.84:55922) 2023-10-16T10:09:06.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.371+0000 7ff83affd640 1 -- 172.21.15.84:0/3062765370 learned_addr learned my addr 172.21.15.84:0/3062765370 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:09:06.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.371+0000 7ff83b7fe640 1 -- 172.21.15.84:0/3062765370 >> v2:172.21.15.157:3300/0 conn(0x7ff83c0ad970 msgr2=0x7ff83c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:06.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.371+0000 7ff83b7fe640 1 --2- 172.21.15.84:0/3062765370 >> v2:172.21.15.157:3300/0 conn(0x7ff83c0ad970 0x7ff83c0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.371+0000 7ff83b7fe640 1 -- 172.21.15.84:0/3062765370 >> v1:172.21.15.84:6789/0 conn(0x7ff83c06c040 legacy=0x7ff83c06c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:06.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.371+0000 7ff83b7fe640 1 -- 172.21.15.84:0/3062765370 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83c0cc270 con 0x7ff83c0ccfb0 2023-10-16T10:09:06.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.371+0000 7ff839ffb640 1 -- 172.21.15.84:0/3062765370 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223903193 0 0) 0x7ff83c15b9d0 con 0x7ff83c06c040 2023-10-16T10:09:06.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.372+0000 7ff83b7fe640 1 --2- 172.21.15.84:0/3062765370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff83c0ccfb0 0x7ff83c0cd460 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7ff834009d90 tx=0x7ff83402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac70cdb9920b5e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:06.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.372+0000 7ff839ffb640 1 -- 172.21.15.84:0/3062765370 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83402cc60 con 0x7ff83c0ccfb0 2023-10-16T10:09:06.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.372+0000 7ff839ffb640 1 -- 172.21.15.84:0/3062765370 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83402cdc0 con 0x7ff83c0ccfb0 2023-10-16T10:09:06.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.372+0000 7ff839ffb640 1 -- 172.21.15.84:0/3062765370 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff834034660 con 0x7ff83c0ccfb0 2023-10-16T10:09:06.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.373+0000 7ff83bfff640 1 -- 172.21.15.84:0/3062765370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff83c0ccfb0 msgr2=0x7ff83c0cd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:06.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.373+0000 7ff83bfff640 1 --2- 172.21.15.84:0/3062765370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff83c0ccfb0 0x7ff83c0cd460 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7ff834009d90 tx=0x7ff83402beb0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.373+0000 7ff83bfff640 1 -- 172.21.15.84:0/3062765370 shutdown_connections 2023-10-16T10:09:06.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.373+0000 7ff83bfff640 1 --2- 172.21.15.84:0/3062765370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff83c0ccfb0 0x7ff83c0cd460 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.373+0000 7ff83bfff640 1 --2- 172.21.15.84:0/3062765370 >> v2:172.21.15.157:3300/0 conn(0x7ff83c0ad970 0x7ff83c0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.373+0000 7ff83bfff640 1 -- 172.21.15.84:0/3062765370 >> 172.21.15.84:0/3062765370 conn(0x7ff83c0ce6a0 msgr2=0x7ff83c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:06.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.373+0000 7ff83bfff640 1 -- 172.21.15.84:0/3062765370 shutdown_connections 2023-10-16T10:09:06.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.373+0000 7ff83bfff640 1 -- 172.21.15.84:0/3062765370 wait complete. 2023-10-16T10:09:06.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.375+0000 7ff83bfff640 1 Processor -- start 2023-10-16T10:09:06.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.375+0000 7ff83bfff640 1 -- start start 2023-10-16T10:09:06.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.376+0000 7ff83bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff83c06c040 0x7ff83c153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:06.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.376+0000 7ff83bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff83c0ad970 0x7ff83c0c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:06.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.376+0000 7ff83bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff83c0ccfb0 0x7ff83c0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:06.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.376+0000 7ff83bfff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c154120 con 0x7ff83c0ccfb0 2023-10-16T10:09:06.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.376+0000 7ff83bfff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c1e45b0 con 0x7ff83c06c040 2023-10-16T10:09:06.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.376+0000 7ff83bfff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c0c9740 con 0x7ff83c0ad970 2023-10-16T10:09:06.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.376+0000 7ff83b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff83c0ccfb0 0x7ff83c0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:06.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.376+0000 7ff83b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff83c0ccfb0 0x7ff83c0ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51234/0 (socket says 172.21.15.84:51234) 2023-10-16T10:09:06.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.376+0000 7ff83b7fe640 1 -- 172.21.15.84:0/347457664 learned_addr learned my addr 172.21.15.84:0/347457664 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:06.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.377+0000 7ff83b7fe640 1 -- 172.21.15.84:0/347457664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff83c0ad970 msgr2=0x7ff83c0c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:06.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.377+0000 7ff83b7fe640 1 --2- 172.21.15.84:0/347457664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff83c0ad970 0x7ff83c0c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.377+0000 7ff83b7fe640 1 -- 172.21.15.84:0/347457664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff83c06c040 msgr2=0x7ff83c153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:06.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.377+0000 7ff83b7fe640 1 --2- 172.21.15.84:0/347457664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff83c06c040 0x7ff83c153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.377+0000 7ff83b7fe640 1 -- 172.21.15.84:0/347457664 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff834009a70 con 0x7ff83c0ccfb0 2023-10-16T10:09:06.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.377+0000 7ff83b7fe640 1 --2- 172.21.15.84:0/347457664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff83c0ccfb0 0x7ff83c0ac890 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7ff834009d90 tx=0x7ff834004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:06.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.377+0000 7ff823fff640 1 -- 172.21.15.84:0/347457664 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff834040450 con 0x7ff83c0ccfb0 2023-10-16T10:09:06.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.377+0000 7ff823fff640 1 -- 172.21.15.84:0/347457664 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8340405d0 con 0x7ff83c0ccfb0 2023-10-16T10:09:06.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.377+0000 7ff823fff640 1 -- 172.21.15.84:0/347457664 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff834040980 con 0x7ff83c0ccfb0 2023-10-16T10:09:06.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.377+0000 7ff83bfff640 1 -- 172.21.15.84:0/347457664 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff83c0c9a20 con 0x7ff83c0ccfb0 2023-10-16T10:09:06.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.377+0000 7ff83bfff640 1 -- 172.21.15.84:0/347457664 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83c0c9fe0 con 0x7ff83c0ccfb0 2023-10-16T10:09:06.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.379+0000 7ff83bfff640 1 -- 172.21.15.84:0/347457664 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff83c06b330 con 0x7ff83c0ccfb0 2023-10-16T10:09:06.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.379+0000 7ff823fff640 1 -- 172.21.15.84:0/347457664 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff834040ae0 con 0x7ff83c0ccfb0 2023-10-16T10:09:06.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.380+0000 7ff823fff640 1 --2- 172.21.15.84:0/347457664 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7ff82806af90 0x7ff82806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:06.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.380+0000 7ff823fff640 1 -- 172.21.15.84:0/347457664 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4885+0+0 (secure 0 0 0) 0x7ff8340ac0c0 con 0x7ff83c0ccfb0 2023-10-16T10:09:06.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.381+0000 7ff83affd640 1 --2- 172.21.15.84:0/347457664 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7ff82806af90 0x7ff82806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:06.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.381+0000 7ff83affd640 1 --2- 172.21.15.84:0/347457664 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7ff82806af90 0x7ff82806d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff82c007fd0 tx=0x7ff82c007920 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:06.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.385+0000 7ff823fff640 1 -- 172.21.15.84:0/347457664 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff8340772f0 con 0x7ff83c0ccfb0 2023-10-16T10:09:06.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.625+0000 7ff83bfff640 1 -- 172.21.15.84:0/347457664 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff83c1e4720 con 0x7ff83c0ccfb0 2023-10-16T10:09:06.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.626+0000 7ff823fff640 1 -- 172.21.15.84:0/347457664 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff83407a910 con 0x7ff83c0ccfb0 2023-10-16T10:09:06.628 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:09:06.628 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T10:09:06.628 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-10-16T10:09:06.629 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.629 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-10-16T10:09:06.629 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l6dhjKRAA0pMvY6AK6qtFXDprNhwsqg==", 2023-10-16T10:09:06.629 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.629 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:09:06.630 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:09:06.630 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:09:06.630 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:09:06.631 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.631 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:09:06.631 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.631 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-10-16T10:09:06.631 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lFpMxEBAA0TLPRt5q3FwwBvMG019cjQ==", 2023-10-16T10:09:06.632 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.632 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:09:06.632 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:09:06.632 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:09:06.632 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:09:06.633 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.633 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:09:06.633 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.633 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-10-16T10:09:06.633 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l73RBOhAAOMEPxfUAMHhn5Blyvd5oMw==", 2023-10-16T10:09:06.634 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.634 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:09:06.634 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:09:06.634 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:09:06.634 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:09:06.635 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.635 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:09:06.635 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.635 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-10-16T10:09:06.635 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lcTIRIhAA0+FWm4b4yw77LD3+WKbk7w==", 2023-10-16T10:09:06.636 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.636 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:09:06.636 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:09:06.636 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:09:06.636 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:09:06.637 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.637 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:09:06.637 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.637 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-10-16T10:09:06.637 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDYBi1l1zBLERAA3YLW3xIYQoRVw7KSkwN3Og==", 2023-10-16T10:09:06.637 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.638 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:09:06.638 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:09:06.638 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:09:06.638 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.639 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:09:06.639 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.639 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-10-16T10:09:06.639 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDbBi1lFUUxKRAAQbus8SmYprJRmWTjoCznNA==", 2023-10-16T10:09:06.639 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.641 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:09:06.641 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:09:06.641 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:09:06.642 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.642 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:09:06.642 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.642 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-10-16T10:09:06.642 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDeBi1l+9lBMxAArl6sb/h15uBG/a9lVH40Hg==", 2023-10-16T10:09:06.643 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.643 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:09:06.643 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:09:06.643 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:09:06.643 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.644 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:09:06.644 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.644 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-10-16T10:09:06.644 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDiBi1l2rnEERAA3oRD71xa0fEOynM8zKuBEg==", 2023-10-16T10:09:06.644 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.645 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:09:06.645 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:09:06.645 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:09:06.645 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.645 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:09:06.646 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.646 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-10-16T10:09:06.646 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDnBi1lR7oLChAAZto8fDovDbpOd3Fcnc7C6w==", 2023-10-16T10:09:06.646 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.646 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:09:06.647 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:09:06.647 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:09:06.647 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.647 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:09:06.648 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.648 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-10-16T10:09:06.648 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDqBi1lmjg/JBAAlINKRGBc0/UWJJHMVFw57w==", 2023-10-16T10:09:06.648 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.648 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:09:06.648 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:09:06.649 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:09:06.649 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.649 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:09:06.649 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.649 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-10-16T10:09:06.650 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDuBi1lu0kMARAAtuSeM8F3yr8CVrc3p1biaQ==", 2023-10-16T10:09:06.650 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.650 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:09:06.650 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:09:06.650 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:09:06.651 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.651 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:09:06.651 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.651 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-10-16T10:09:06.652 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDxBi1lf6mNGBAAHGbqzJ0fmDxQIsasboHMfg==", 2023-10-16T10:09:06.652 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.652 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T10:09:06.652 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T10:09:06.652 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:09:06.653 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.653 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:09:06.653 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.653 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-10-16T10:09:06.654 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ==", 2023-10-16T10:09:06.654 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.654 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:09:06.654 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow r", 2023-10-16T10:09:06.654 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T10:09:06.655 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.655 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:09:06.655 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.655 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-10-16T10:09:06.656 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1lHU+4BxAAd8ET8hQ8Y+EUwdTJr9/+Ig==", 2023-10-16T10:09:06.656 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.656 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:09:06.656 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:09:06.656 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:09:06.657 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:09:06.657 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.657 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:09:06.657 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.657 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-10-16T10:09:06.658 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lA1BOORAAFNyfqgEF34MVR9YX3KBKkQ==", 2023-10-16T10:09:06.658 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.658 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:09:06.658 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:09:06.659 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:09:06.659 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:09:06.659 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.659 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:09:06.659 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.660 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-10-16T10:09:06.660 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1l848oERAANMfrZz9JrlgNVh7ZFbtnaA==", 2023-10-16T10:09:06.660 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.660 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T10:09:06.661 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T10:09:06.661 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T10:09:06.661 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T10:09:06.661 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.661 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:09:06.662 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.662 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-10-16T10:09:06.662 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDRBi1lTAxTGRAAMptxnxy3Qit/cTCXie6CrQ==", 2023-10-16T10:09:06.662 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.662 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:09:06.663 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T10:09:06.663 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T10:09:06.663 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:09:06.663 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.663 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:09:06.664 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.664 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-10-16T10:09:06.664 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lPuJxHRAABAnGPVCrkq0JR4Y6iFGZlw==", 2023-10-16T10:09:06.664 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.664 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:09:06.665 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T10:09:06.665 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:09:06.665 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.665 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T10:09:06.665 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:06.665 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-10-16T10:09:06.666 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lLxnKEBAA+a/pOHUp7Sg3m4weAust5w==", 2023-10-16T10:09:06.666 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T10:09:06.666 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T10:09:06.666 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T10:09:06.667 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T10:09:06.667 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.667 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:06.667 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-10-16T10:09:06.667 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T10:09:06.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.628+0000 7ff83bfff640 1 -- 172.21.15.84:0/347457664 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7ff82806af90 msgr2=0x7ff82806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:06.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.628+0000 7ff83bfff640 1 --2- 172.21.15.84:0/347457664 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7ff82806af90 0x7ff82806d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff82c007fd0 tx=0x7ff82c007920 comp rx=0 tx=0).stop 2023-10-16T10:09:06.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.628+0000 7ff83bfff640 1 -- 172.21.15.84:0/347457664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff83c0ccfb0 msgr2=0x7ff83c0ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:06.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.628+0000 7ff83bfff640 1 --2- 172.21.15.84:0/347457664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff83c0ccfb0 0x7ff83c0ac890 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7ff834009d90 tx=0x7ff834004a40 comp rx=0 tx=0).stop 2023-10-16T10:09:06.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.629+0000 7ff83bfff640 1 -- 172.21.15.84:0/347457664 shutdown_connections 2023-10-16T10:09:06.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.629+0000 7ff83bfff640 1 --2- 172.21.15.84:0/347457664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff83c0ccfb0 0x7ff83c0ac890 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.629+0000 7ff83bfff640 1 --2- 172.21.15.84:0/347457664 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7ff82806af90 0x7ff82806d450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.629+0000 7ff83bfff640 1 --2- 172.21.15.84:0/347457664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff83c0ad970 0x7ff83c0c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.629+0000 7ff83bfff640 1 --2- 172.21.15.84:0/347457664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff83c06c040 0x7ff83c153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.629+0000 7ff83bfff640 1 -- 172.21.15.84:0/347457664 >> 172.21.15.84:0/347457664 conn(0x7ff83c0ce6a0 msgr2=0x7ff83c0c5770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:06.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.629+0000 7ff83bfff640 1 -- 172.21.15.84:0/347457664 shutdown_connections 2023-10-16T10:09:06.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.629+0000 7ff83bfff640 1 -- 172.21.15.84:0/347457664 wait complete. 2023-10-16T10:09:06.673 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:09:06.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.886+0000 7f38cae39640 1 Processor -- start 2023-10-16T10:09:06.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.886+0000 7f38cae39640 1 -- start start 2023-10-16T10:09:06.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.888+0000 7f38cae39640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f38c415aaa0 0x7f38c415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:06.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.888+0000 7f38cae39640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f38c40ce390 0x7f38c4070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:06.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.888+0000 7f38cae39640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c415ed60 con 0x7f38c415ae80 2023-10-16T10:09:06.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.888+0000 7f38cae39640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c406b810 con 0x7f38c40ce390 2023-10-16T10:09:06.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.888+0000 7f38cae39640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c40707e0 con 0x7f38c415aaa0 2023-10-16T10:09:06.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.888+0000 7f38c93d6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f38c40ce390 0x7f38c4070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:06.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.888+0000 7f38c93d6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f38c40ce390 0x7f38c4070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51242/0 (socket says 172.21.15.84:51242) 2023-10-16T10:09:06.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.888+0000 7f38c93d6640 1 -- 172.21.15.84:0/4125066418 learned_addr learned my addr 172.21.15.84:0/4125066418 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:06.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.888+0000 7f38c8bd5640 1 --2- 172.21.15.84:0/4125066418 >> v2:172.21.15.157:3300/0 conn(0x7f38c415aaa0 0x7f38c415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:06.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.888+0000 7f38c93d6640 1 -- 172.21.15.84:0/4125066418 >> v2:172.21.15.157:3300/0 conn(0x7f38c415aaa0 msgr2=0x7f38c415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:06.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.888+0000 7f38c93d6640 1 --2- 172.21.15.84:0/4125066418 >> v2:172.21.15.157:3300/0 conn(0x7f38c415aaa0 0x7f38c415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.888+0000 7f38c93d6640 1 -- 172.21.15.84:0/4125066418 >> v1:172.21.15.157:6789/0 conn(0x7f38c415ae80 legacy=0x7f38c40cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:06.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.888+0000 7f38c93d6640 1 -- 172.21.15.84:0/4125066418 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c415ef90 con 0x7f38c40ce390 2023-10-16T10:09:06.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.889+0000 7f38c93d6640 1 --2- 172.21.15.84:0/4125066418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38c40ce390 0x7f38c4070f50 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f38c40cd990 tx=0x7f38bc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5df0ef39755641 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:06.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.890+0000 7f38c37fe640 1 -- 172.21.15.84:0/4125066418 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc02ca80 con 0x7f38c40ce390 2023-10-16T10:09:06.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.890+0000 7f38c37fe640 1 -- 172.21.15.84:0/4125066418 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc02cbe0 con 0x7f38c40ce390 2023-10-16T10:09:06.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.890+0000 7f38c37fe640 1 -- 172.21.15.84:0/4125066418 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc0343f0 con 0x7f38c40ce390 2023-10-16T10:09:06.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.890+0000 7f38cae39640 1 -- 172.21.15.84:0/4125066418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38c40ce390 msgr2=0x7f38c4070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:06.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.890+0000 7f38cae39640 1 --2- 172.21.15.84:0/4125066418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38c40ce390 0x7f38c4070f50 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f38c40cd990 tx=0x7f38bc02bee0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.890+0000 7f38cae39640 1 -- 172.21.15.84:0/4125066418 shutdown_connections 2023-10-16T10:09:06.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.890+0000 7f38cae39640 1 --2- 172.21.15.84:0/4125066418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38c40ce390 0x7f38c4070f50 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.890+0000 7f38cae39640 1 --2- 172.21.15.84:0/4125066418 >> v2:172.21.15.157:3300/0 conn(0x7f38c415aaa0 0x7f38c415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.890+0000 7f38cae39640 1 -- 172.21.15.84:0/4125066418 >> 172.21.15.84:0/4125066418 conn(0x7f38c40cb410 msgr2=0x7f38c40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:06.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.890+0000 7f38cae39640 1 -- 172.21.15.84:0/4125066418 shutdown_connections 2023-10-16T10:09:06.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.890+0000 7f38cae39640 1 -- 172.21.15.84:0/4125066418 wait complete. 2023-10-16T10:09:06.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.892+0000 7f38cae39640 1 Processor -- start 2023-10-16T10:09:06.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.892+0000 7f38cae39640 1 -- start start 2023-10-16T10:09:06.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.893+0000 7f38cae39640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38c40ce390 0x7f38c414ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:06.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.893+0000 7f38cae39640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c415aaa0 0x7f38c40c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:06.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.893+0000 7f38cae39640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38c415ae80 0x7f38c40ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:06.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.893+0000 7f38cae39640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c40af320 con 0x7f38c415ae80 2023-10-16T10:09:06.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.893+0000 7f38cae39640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c40af600 con 0x7f38c415aaa0 2023-10-16T10:09:06.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.893+0000 7f38cae39640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c40af8e0 con 0x7f38c40ce390 2023-10-16T10:09:06.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.894+0000 7f38c93d6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38c415ae80 0x7f38c40ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:06.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.894+0000 7f38c93d6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38c415ae80 0x7f38c40ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51250/0 (socket says 172.21.15.84:51250) 2023-10-16T10:09:06.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.894+0000 7f38c93d6640 1 -- 172.21.15.84:0/1134952695 learned_addr learned my addr 172.21.15.84:0/1134952695 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:06.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.894+0000 7f38c3fff640 1 --2- 172.21.15.84:0/1134952695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c415aaa0 0x7f38c40c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:06.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.894+0000 7f38c8bd5640 1 --2- 172.21.15.84:0/1134952695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38c40ce390 0x7f38c414ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:06.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.894+0000 7f38c93d6640 1 -- 172.21.15.84:0/1134952695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38c40ce390 msgr2=0x7f38c414ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:06.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.894+0000 7f38c93d6640 1 --2- 172.21.15.84:0/1134952695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38c40ce390 0x7f38c414ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.894+0000 7f38c93d6640 1 -- 172.21.15.84:0/1134952695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c415aaa0 msgr2=0x7f38c40c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:06.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.894+0000 7f38c93d6640 1 --2- 172.21.15.84:0/1134952695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c415aaa0 0x7f38c40c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:06.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.894+0000 7f38c93d6640 1 -- 172.21.15.84:0/1134952695 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc009b30 con 0x7f38c415ae80 2023-10-16T10:09:06.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.894+0000 7f38c3fff640 1 --2- 172.21.15.84:0/1134952695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c415aaa0 0x7f38c40c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:06.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.894+0000 7f38c93d6640 1 --2- 172.21.15.84:0/1134952695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38c415ae80 0x7f38c40ac890 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f38bc036040 tx=0x7f38bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:06.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.895+0000 7f38c17fa640 1 -- 172.21.15.84:0/1134952695 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc009510 con 0x7f38c415ae80 2023-10-16T10:09:06.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.895+0000 7f38c17fa640 1 -- 172.21.15.84:0/1134952695 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc03e450 con 0x7f38c415ae80 2023-10-16T10:09:06.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.895+0000 7f38c17fa640 1 -- 172.21.15.84:0/1134952695 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc03e7e0 con 0x7f38c415ae80 2023-10-16T10:09:06.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.895+0000 7f38cae39640 1 -- 172.21.15.84:0/1134952695 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c40afbc0 con 0x7f38c415ae80 2023-10-16T10:09:06.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.895+0000 7f38cae39640 1 -- 172.21.15.84:0/1134952695 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c40b00e0 con 0x7f38c415ae80 2023-10-16T10:09:06.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.896+0000 7f38cae39640 1 -- 172.21.15.84:0/1134952695 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c40afde0 con 0x7f38c415ae80 2023-10-16T10:09:06.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.897+0000 7f38c17fa640 1 -- 172.21.15.84:0/1134952695 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f38bc02cd50 con 0x7f38c415ae80 2023-10-16T10:09:06.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.898+0000 7f38c17fa640 1 --2- 172.21.15.84:0/1134952695 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f38a806afc0 0x7f38a806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:06.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.898+0000 7f38c17fa640 1 -- 172.21.15.84:0/1134952695 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f38bc0abc40 con 0x7f38c415ae80 2023-10-16T10:09:06.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.898+0000 7f38c8bd5640 1 --2- 172.21.15.84:0/1134952695 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f38a806afc0 0x7f38a806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:06.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.899+0000 7f38c8bd5640 1 --2- 172.21.15.84:0/1134952695 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f38a806afc0 0x7f38a806d480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f38b8006fd0 tx=0x7f38b8008040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:06.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:06.902+0000 7f38c17fa640 1 -- 172.21.15.84:0/1134952695 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f38bc076fa0 con 0x7f38c415ae80 2023-10-16T10:09:07.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.098+0000 7f38cae39640 1 -- 172.21.15.84:0/1134952695 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38c40afde0 con 0x7f38c415ae80 2023-10-16T10:09:07.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.100+0000 7f38c17fa640 1 -- 172.21.15.84:0/1134952695 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+12887 (secure 0 0 0) 0x7f38bc033e80 con 0x7f38c415ae80 2023-10-16T10:09:07.101 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:09:07.101 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":147,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:09:06.154037+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:09:07.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.103+0000 7f38cae39640 1 -- 172.21.15.84:0/1134952695 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f38a806afc0 msgr2=0x7f38a806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:07.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.103+0000 7f38cae39640 1 --2- 172.21.15.84:0/1134952695 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f38a806afc0 0x7f38a806d480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f38b8006fd0 tx=0x7f38b8008040 comp rx=0 tx=0).stop 2023-10-16T10:09:07.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.103+0000 7f38cae39640 1 -- 172.21.15.84:0/1134952695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38c415ae80 msgr2=0x7f38c40ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:07.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.103+0000 7f38cae39640 1 --2- 172.21.15.84:0/1134952695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38c415ae80 0x7f38c40ac890 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f38bc036040 tx=0x7f38bc004a40 comp rx=0 tx=0).stop 2023-10-16T10:09:07.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.103+0000 7f38cae39640 1 -- 172.21.15.84:0/1134952695 shutdown_connections 2023-10-16T10:09:07.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.103+0000 7f38cae39640 1 --2- 172.21.15.84:0/1134952695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38c415ae80 0x7f38c40ac890 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.103+0000 7f38cae39640 1 --2- 172.21.15.84:0/1134952695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c415aaa0 0x7f38c40c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.103+0000 7f38cae39640 1 --2- 172.21.15.84:0/1134952695 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f38a806afc0 0x7f38a806d480 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.103+0000 7f38cae39640 1 --2- 172.21.15.84:0/1134952695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38c40ce390 0x7f38c414ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.103+0000 7f38cae39640 1 -- 172.21.15.84:0/1134952695 >> 172.21.15.84:0/1134952695 conn(0x7f38c40cb410 msgr2=0x7f38c4158ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:07.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.103+0000 7f38cae39640 1 -- 172.21.15.84:0/1134952695 shutdown_connections 2023-10-16T10:09:07.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.103+0000 7f38cae39640 1 -- 172.21.15.84:0/1134952695 wait complete. 2023-10-16T10:09:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:09:07.122 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:09:07.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.330+0000 7fd781161640 1 Processor -- start 2023-10-16T10:09:07.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.330+0000 7fd781161640 1 -- start start 2023-10-16T10:09:07.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.332+0000 7fd781161640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd77c0cbf20 0x7fd77c0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:07.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.332+0000 7fd781161640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd77c06c140 con 0x7fd77c15e9b0 2023-10-16T10:09:07.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.332+0000 7fd781161640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd77c15b2c0 con 0x7fd77c15aee0 2023-10-16T10:09:07.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.332+0000 7fd781161640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd77c06b270 con 0x7fd77c0cbf20 2023-10-16T10:09:07.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.332+0000 7fd773fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd77c0cbf20 0x7fd77c0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:07.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.332+0000 7fd77b59e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd77c15e9b0 0x7fd77c15ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55930/0 (socket says 172.21.15.84:55930) 2023-10-16T10:09:07.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.332+0000 7fd77b59e640 1 -- 172.21.15.84:0/3008569430 learned_addr learned my addr 172.21.15.84:0/3008569430 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:09:07.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.333+0000 7fd773fff640 1 -- 172.21.15.84:0/3008569430 >> v1:172.21.15.157:6789/0 conn(0x7fd77c15aee0 legacy=0x7fd77c0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:07.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.333+0000 7fd773fff640 1 -- 172.21.15.84:0/3008569430 >> v1:172.21.15.84:6789/0 conn(0x7fd77c15e9b0 legacy=0x7fd77c15ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:07.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.333+0000 7fd773fff640 1 -- 172.21.15.84:0/3008569430 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c06daf0 con 0x7fd77c0cbf20 2023-10-16T10:09:07.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.334+0000 7fd773fff640 1 --2- 172.21.15.84:0/3008569430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd77c0cbf20 0x7fd77c0aeef0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fd7640097a0 tx=0x7fd76402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fd5456a9683c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:07.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.334+0000 7fd77a59c640 1 -- 172.21.15.84:0/3008569430 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76402e070 con 0x7fd77c0cbf20 2023-10-16T10:09:07.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.334+0000 7fd77a59c640 1 -- 172.21.15.84:0/3008569430 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7640315a0 con 0x7fd77c0cbf20 2023-10-16T10:09:07.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.334+0000 7fd77a59c640 1 -- 172.21.15.84:0/3008569430 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764031930 con 0x7fd77c0cbf20 2023-10-16T10:09:07.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.335+0000 7fd781161640 1 -- 172.21.15.84:0/3008569430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd77c0cbf20 msgr2=0x7fd77c0aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:07.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.335+0000 7fd781161640 1 --2- 172.21.15.84:0/3008569430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd77c0cbf20 0x7fd77c0aeef0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fd7640097a0 tx=0x7fd76402b840 comp rx=0 tx=0).stop 2023-10-16T10:09:07.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.335+0000 7fd781161640 1 -- 172.21.15.84:0/3008569430 shutdown_connections 2023-10-16T10:09:07.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.335+0000 7fd781161640 1 --2- 172.21.15.84:0/3008569430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd77c0cbf20 0x7fd77c0aeef0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.335+0000 7fd781161640 1 -- 172.21.15.84:0/3008569430 >> 172.21.15.84:0/3008569430 conn(0x7fd77c0cfac0 msgr2=0x7fd77c0cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:07.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.335+0000 7fd781161640 1 -- 172.21.15.84:0/3008569430 shutdown_connections 2023-10-16T10:09:07.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.335+0000 7fd781161640 1 -- 172.21.15.84:0/3008569430 wait complete. 2023-10-16T10:09:07.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.337+0000 7fd781161640 1 Processor -- start 2023-10-16T10:09:07.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.337+0000 7fd781161640 1 -- start start 2023-10-16T10:09:07.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.338+0000 7fd781161640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd77c0cbf20 0x7fd77c0c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:07.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.338+0000 7fd781161640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd77c15aee0 0x7fd77c14e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:07.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.338+0000 7fd781161640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd77c15e9b0 0x7fd77c158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:07.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.338+0000 7fd781161640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c14fb30 con 0x7fd77c15aee0 2023-10-16T10:09:07.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.338+0000 7fd781161640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c14f6e0 con 0x7fd77c0cbf20 2023-10-16T10:09:07.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.338+0000 7fd781161640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c0c7fa0 con 0x7fd77c15e9b0 2023-10-16T10:09:07.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.338+0000 7fd773fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd77c15aee0 0x7fd77c14e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:07.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.338+0000 7fd773fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd77c15aee0 0x7fd77c14e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51280/0 (socket says 172.21.15.84:51280) 2023-10-16T10:09:07.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.338+0000 7fd773fff640 1 -- 172.21.15.84:0/4129187905 learned_addr learned my addr 172.21.15.84:0/4129187905 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:07.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.339+0000 7fd773fff640 1 -- 172.21.15.84:0/4129187905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd77c15e9b0 msgr2=0x7fd77c158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:07.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.339+0000 7fd773fff640 1 --2- 172.21.15.84:0/4129187905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd77c15e9b0 0x7fd77c158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.339+0000 7fd773fff640 1 -- 172.21.15.84:0/4129187905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd77c0cbf20 msgr2=0x7fd77c0c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:07.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.339+0000 7fd77ad9d640 1 --2- 172.21.15.84:0/4129187905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd77c0cbf20 0x7fd77c0c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:07.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.339+0000 7fd773fff640 1 --2- 172.21.15.84:0/4129187905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd77c0cbf20 0x7fd77c0c7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.339+0000 7fd773fff640 1 -- 172.21.15.84:0/4129187905 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd764009400 con 0x7fd77c15aee0 2023-10-16T10:09:07.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.339+0000 7fd773fff640 1 --2- 172.21.15.84:0/4129187905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd77c15aee0 0x7fd77c14e3f0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fd764034040 tx=0x7fd7640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:07.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.339+0000 7fd7737fe640 1 -- 172.21.15.84:0/4129187905 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764032900 con 0x7fd77c15aee0 2023-10-16T10:09:07.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.339+0000 7fd7737fe640 1 -- 172.21.15.84:0/4129187905 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd764032a80 con 0x7fd77c15aee0 2023-10-16T10:09:07.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.339+0000 7fd7737fe640 1 -- 172.21.15.84:0/4129187905 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764032e30 con 0x7fd77c15aee0 2023-10-16T10:09:07.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.339+0000 7fd781161640 1 -- 172.21.15.84:0/4129187905 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77c14ee40 con 0x7fd77c15aee0 2023-10-16T10:09:07.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.339+0000 7fd781161640 1 -- 172.21.15.84:0/4129187905 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77c14f400 con 0x7fd77c15aee0 2023-10-16T10:09:07.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.341+0000 7fd781161640 1 -- 172.21.15.84:0/4129187905 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd77c14f890 con 0x7fd77c15aee0 2023-10-16T10:09:07.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.341+0000 7fd7737fe640 1 -- 172.21.15.84:0/4129187905 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd76403b3f0 con 0x7fd77c15aee0 2023-10-16T10:09:07.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.342+0000 7fd7737fe640 1 --2- 172.21.15.84:0/4129187905 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd75c06afc0 0x7fd75c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:07.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.342+0000 7fd7737fe640 1 -- 172.21.15.84:0/4129187905 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fd7640ab620 con 0x7fd77c15aee0 2023-10-16T10:09:07.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.343+0000 7fd77ad9d640 1 --2- 172.21.15.84:0/4129187905 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd75c06afc0 0x7fd75c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:07.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.343+0000 7fd77ad9d640 1 --2- 172.21.15.84:0/4129187905 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd75c06afc0 0x7fd75c06d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd76c003d90 tx=0x7fd76c009040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:07.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.347+0000 7fd7737fe640 1 -- 172.21.15.84:0/4129187905 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd77c14f890 con 0x7fd77c15aee0 2023-10-16T10:09:07.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.541+0000 7fd781161640 1 -- 172.21.15.84:0/4129187905 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd77c14f890 con 0x7fd77c15aee0 2023-10-16T10:09:07.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.541+0000 7fd7737fe640 1 -- 172.21.15.84:0/4129187905 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v147) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd77c14f890 con 0x7fd77c15aee0 2023-10-16T10:09:07.542 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T10:09:07.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.544+0000 7fd781161640 1 -- 172.21.15.84:0/4129187905 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd75c06afc0 msgr2=0x7fd75c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:07.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.544+0000 7fd781161640 1 --2- 172.21.15.84:0/4129187905 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd75c06afc0 0x7fd75c06d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd76c003d90 tx=0x7fd76c009040 comp rx=0 tx=0).stop 2023-10-16T10:09:07.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.544+0000 7fd781161640 1 -- 172.21.15.84:0/4129187905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd77c15aee0 msgr2=0x7fd77c14e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:07.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.544+0000 7fd781161640 1 --2- 172.21.15.84:0/4129187905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd77c15aee0 0x7fd77c14e3f0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fd764034040 tx=0x7fd7640046c0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.544+0000 7fd781161640 1 -- 172.21.15.84:0/4129187905 shutdown_connections 2023-10-16T10:09:07.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.544+0000 7fd781161640 1 --2- 172.21.15.84:0/4129187905 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd75c06afc0 0x7fd75c06d480 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.544+0000 7fd781161640 1 --2- 172.21.15.84:0/4129187905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd77c15e9b0 0x7fd77c158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.544+0000 7fd781161640 1 --2- 172.21.15.84:0/4129187905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd77c15aee0 0x7fd77c14e3f0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.544+0000 7fd781161640 1 --2- 172.21.15.84:0/4129187905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd77c0cbf20 0x7fd77c0c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.544+0000 7fd781161640 1 -- 172.21.15.84:0/4129187905 >> 172.21.15.84:0/4129187905 conn(0x7fd77c0cfac0 msgr2=0x7fd77c1568e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:07.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.544+0000 7fd781161640 1 -- 172.21.15.84:0/4129187905 shutdown_connections 2023-10-16T10:09:07.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.544+0000 7fd781161640 1 -- 172.21.15.84:0/4129187905 wait complete. 2023-10-16T10:09:07.564 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T10:09:07.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.772+0000 7ff9da27c640 1 Processor -- start 2023-10-16T10:09:07.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.773+0000 7ff9da27c640 1 -- start start 2023-10-16T10:09:07.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.774+0000 7ff9da27c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9d40b0b20 0x7ff9d41615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:07.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.774+0000 7ff9da27c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d415e3e0 con 0x7ff9d40cd0c0 2023-10-16T10:09:07.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.774+0000 7ff9da27c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d40cd4a0 con 0x7ff9d415d3f0 2023-10-16T10:09:07.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.774+0000 7ff9da27c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d415e100 con 0x7ff9d40b0b20 2023-10-16T10:09:07.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.774+0000 7ff9d3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff9d40cd0c0 0x7ff9d40b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55934/0 (socket says 172.21.15.84:55934) 2023-10-16T10:09:07.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.774+0000 7ff9d3fff640 1 -- 172.21.15.84:0/971390137 learned_addr learned my addr 172.21.15.84:0/971390137 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:09:07.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.775+0000 7ff9d37fe640 1 -- 172.21.15.84:0/971390137 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 399250235 0 0) 0x7ff9d415e3e0 con 0x7ff9d40cd0c0 2023-10-16T10:09:07.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.775+0000 7ff9d37fe640 1 -- 172.21.15.84:0/971390137 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9c0003520 con 0x7ff9d40cd0c0 2023-10-16T10:09:07.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.775+0000 7ff9cbfff640 1 --2- 172.21.15.84:0/971390137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9d40b0b20 0x7ff9d41615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:07.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.775+0000 7ff9d37fe640 1 -- 172.21.15.84:0/971390137 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1291537251 0 0) 0x7ff9c0003520 con 0x7ff9d40cd0c0 2023-10-16T10:09:07.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.775+0000 7ff9d37fe640 1 -- 172.21.15.84:0/971390137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9d40b0b20 msgr2=0x7ff9d41615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:07.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.775+0000 7ff9d37fe640 1 --2- 172.21.15.84:0/971390137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9d40b0b20 0x7ff9d41615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.775+0000 7ff9d37fe640 1 -- 172.21.15.84:0/971390137 >> v1:172.21.15.157:6789/0 conn(0x7ff9d415d3f0 legacy=0x7ff9d415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:07.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.775+0000 7ff9d37fe640 1 -- 172.21.15.84:0/971390137 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d4160b10 con 0x7ff9d40cd0c0 2023-10-16T10:09:07.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.775+0000 7ff9d37fe640 1 -- 172.21.15.84:0/971390137 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ff9c4001670 con 0x7ff9d40cd0c0 2023-10-16T10:09:07.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.775+0000 7ff9d37fe640 1 -- 172.21.15.84:0/971390137 >> v1:172.21.15.84:6789/0 conn(0x7ff9d40cd0c0 legacy=0x7ff9d40b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:07.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.775+0000 7ff9d37fe640 1 --2- 172.21.15.84:0/971390137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c00039a0 0x7ff9c0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:07.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.775+0000 7ff9d37fe640 1 --2- 172.21.15.84:0/971390137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9c00047c0 0x7ff9c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:07.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.775+0000 7ff9d37fe640 1 --2- 172.21.15.84:0/971390137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9d40b0b20 0x7ff9c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:07.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.775+0000 7ff9d37fe640 1 -- 172.21.15.84:0/971390137 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d415e100 con 0x7ff9c00039a0 2023-10-16T10:09:07.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.776+0000 7ff9d37fe640 1 -- 172.21.15.84:0/971390137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c00054e0 con 0x7ff9c00047c0 2023-10-16T10:09:07.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.776+0000 7ff9d37fe640 1 -- 172.21.15.84:0/971390137 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c000bec0 con 0x7ff9d40b0b20 2023-10-16T10:09:07.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.776+0000 7ff9d3fff640 1 --2- 172.21.15.84:0/971390137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c00039a0 0x7ff9c0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:07.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.776+0000 7ff9cbfff640 1 --2- 172.21.15.84:0/971390137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9c00047c0 0x7ff9c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:07.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.776+0000 7ff9d3fff640 1 -- 172.21.15.84:0/971390137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9d40b0b20 msgr2=0x7ff9c000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:07.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.776+0000 7ff9d8819640 1 --2- 172.21.15.84:0/971390137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9d40b0b20 0x7ff9c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:07.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.776+0000 7ff9d3fff640 1 --2- 172.21.15.84:0/971390137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9d40b0b20 0x7ff9c000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.776+0000 7ff9d3fff640 1 -- 172.21.15.84:0/971390137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9c00047c0 msgr2=0x7ff9c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:07.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.776+0000 7ff9d3fff640 1 --2- 172.21.15.84:0/971390137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9c00047c0 0x7ff9c0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.776+0000 7ff9d3fff640 1 -- 172.21.15.84:0/971390137 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c0003520 con 0x7ff9c00039a0 2023-10-16T10:09:07.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.776+0000 7ff9d8819640 1 --2- 172.21.15.84:0/971390137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9d40b0b20 0x7ff9c000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:07.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.776+0000 7ff9cbfff640 1 --2- 172.21.15.84:0/971390137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9c00047c0 0x7ff9c0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:07.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.777+0000 7ff9d3fff640 1 --2- 172.21.15.84:0/971390137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c00039a0 0x7ff9c0003dc0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c0e0 tx=0x7ff9c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7bd8ff85d82a4f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:07.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.778+0000 7ff9d37fe640 1 -- 172.21.15.84:0/971390137 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4005ad0 con 0x7ff9c00039a0 2023-10-16T10:09:07.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.778+0000 7ff9d37fe640 1 -- 172.21.15.84:0/971390137 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c4005c50 con 0x7ff9c00039a0 2023-10-16T10:09:07.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.778+0000 7ff9d37fe640 1 -- 172.21.15.84:0/971390137 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c402c7f0 con 0x7ff9c00039a0 2023-10-16T10:09:07.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.778+0000 7ff9da27c640 1 -- 172.21.15.84:0/971390137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c00039a0 msgr2=0x7ff9c0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:07.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.778+0000 7ff9da27c640 1 --2- 172.21.15.84:0/971390137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c00039a0 0x7ff9c0003dc0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c0e0 tx=0x7ff9c00057f0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.778+0000 7ff9d8819640 1 -- 172.21.15.84:0/971390137 reap_dead start 2023-10-16T10:09:07.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.778+0000 7ff9da27c640 1 -- 172.21.15.84:0/971390137 shutdown_connections 2023-10-16T10:09:07.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.778+0000 7ff9da27c640 1 -- 172.21.15.84:0/971390137 >> 172.21.15.84:0/971390137 conn(0x7ff9d40cb410 msgr2=0x7ff9d40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:07.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.778+0000 7ff9da27c640 1 -- 172.21.15.84:0/971390137 shutdown_connections 2023-10-16T10:09:07.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.778+0000 7ff9da27c640 1 -- 172.21.15.84:0/971390137 wait complete. 2023-10-16T10:09:07.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.780+0000 7ff9da27c640 1 Processor -- start 2023-10-16T10:09:07.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.780+0000 7ff9da27c640 1 -- start start 2023-10-16T10:09:07.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.781+0000 7ff9da27c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c00039a0 0x7ff9d40c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:07.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.781+0000 7ff9da27c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9d40b39a0 0x7ff9d4156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:07.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.781+0000 7ff9da27c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d40c5150 0x7ff9d40aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:07.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.781+0000 7ff9da27c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d4157250 con 0x7ff9c00039a0 2023-10-16T10:09:07.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.781+0000 7ff9da27c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d40c34c0 con 0x7ff9d40c5150 2023-10-16T10:09:07.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.781+0000 7ff9da27c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d40b44f0 con 0x7ff9d40b39a0 2023-10-16T10:09:07.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.781+0000 7ff9d3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c00039a0 0x7ff9d40c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:07.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.782+0000 7ff9d3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c00039a0 0x7ff9d40c5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51296/0 (socket says 172.21.15.84:51296) 2023-10-16T10:09:07.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.782+0000 7ff9d3fff640 1 -- 172.21.15.84:0/2005216781 learned_addr learned my addr 172.21.15.84:0/2005216781 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:07.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.782+0000 7ff9d3fff640 1 -- 172.21.15.84:0/2005216781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9d40b39a0 msgr2=0x7ff9d4156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:07.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.782+0000 7ff9d3fff640 1 --2- 172.21.15.84:0/2005216781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9d40b39a0 0x7ff9d4156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.782+0000 7ff9d8819640 1 --2- 172.21.15.84:0/2005216781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d40c5150 0x7ff9d40aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:07.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.782+0000 7ff9d3fff640 1 -- 172.21.15.84:0/2005216781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d40c5150 msgr2=0x7ff9d40aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:07.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.782+0000 7ff9d3fff640 1 --2- 172.21.15.84:0/2005216781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d40c5150 0x7ff9d40aeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.782+0000 7ff9d3fff640 1 -- 172.21.15.84:0/2005216781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d40b3f30 con 0x7ff9c00039a0 2023-10-16T10:09:07.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.782+0000 7ff9d3fff640 1 --2- 172.21.15.84:0/2005216781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c00039a0 0x7ff9d40c5a60 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7ff9c402c390 tx=0x7ff9c4001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:07.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.783+0000 7ff9d17fa640 1 -- 172.21.15.84:0/2005216781 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4039070 con 0x7ff9c00039a0 2023-10-16T10:09:07.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.783+0000 7ff9d17fa640 1 -- 172.21.15.84:0/2005216781 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c402cc00 con 0x7ff9c00039a0 2023-10-16T10:09:07.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.783+0000 7ff9d17fa640 1 -- 172.21.15.84:0/2005216781 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c403c040 con 0x7ff9c00039a0 2023-10-16T10:09:07.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.783+0000 7ff9da27c640 1 -- 172.21.15.84:0/2005216781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d40b4210 con 0x7ff9c00039a0 2023-10-16T10:09:07.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.783+0000 7ff9da27c640 1 -- 172.21.15.84:0/2005216781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d40c55e0 con 0x7ff9c00039a0 2023-10-16T10:09:07.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.784+0000 7ff9da27c640 1 -- 172.21.15.84:0/2005216781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d404e160 con 0x7ff9c00039a0 2023-10-16T10:09:07.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.785+0000 7ff9d17fa640 1 -- 172.21.15.84:0/2005216781 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff9c4032070 con 0x7ff9c00039a0 2023-10-16T10:09:07.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.786+0000 7ff9d17fa640 1 --2- 172.21.15.84:0/2005216781 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7ff9b406af90 0x7ff9b406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:07.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.786+0000 7ff9d17fa640 1 -- 172.21.15.84:0/2005216781 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4885+0+0 (secure 0 0 0) 0x7ff9c40aa7b0 con 0x7ff9c00039a0 2023-10-16T10:09:07.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.786+0000 7ff9cbfff640 1 --2- 172.21.15.84:0/2005216781 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7ff9b406af90 0x7ff9b406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:07.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.787+0000 7ff9cbfff640 1 --2- 172.21.15.84:0/2005216781 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7ff9b406af90 0x7ff9b406d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc006f00 tx=0x7ff9bc006e30 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:07.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.790+0000 7ff9d17fa640 1 -- 172.21.15.84:0/2005216781 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff9c4075b10 con 0x7ff9c00039a0 2023-10-16T10:09:07.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.982+0000 7ff9da27c640 1 -- 172.21.15.84:0/2005216781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff9d40c8cc0 con 0x7ff9c00039a0 2023-10-16T10:09:07.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.983+0000 7ff9d17fa640 1 -- 172.21.15.84:0/2005216781 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v147) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff9c4079130 con 0x7ff9c00039a0 2023-10-16T10:09:07.984 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:09:07.984 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T10:09:07.984 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:07.984 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-16T10:09:07.984 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-16T10:09:07.985 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:07.985 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T10:09:07.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.985+0000 7ff9da27c640 1 -- 172.21.15.84:0/2005216781 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7ff9b406af90 msgr2=0x7ff9b406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.985+0000 7ff9da27c640 1 --2- 172.21.15.84:0/2005216781 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7ff9b406af90 0x7ff9b406d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc006f00 tx=0x7ff9bc006e30 comp rx=0 tx=0).stop 2023-10-16T10:09:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.985+0000 7ff9da27c640 1 -- 172.21.15.84:0/2005216781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c00039a0 msgr2=0x7ff9d40c5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.985+0000 7ff9da27c640 1 --2- 172.21.15.84:0/2005216781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c00039a0 0x7ff9d40c5a60 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7ff9c402c390 tx=0x7ff9c4001930 comp rx=0 tx=0).stop 2023-10-16T10:09:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.985+0000 7ff9da27c640 1 -- 172.21.15.84:0/2005216781 shutdown_connections 2023-10-16T10:09:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.985+0000 7ff9da27c640 1 --2- 172.21.15.84:0/2005216781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d40c5150 0x7ff9d40aeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.985+0000 7ff9da27c640 1 --2- 172.21.15.84:0/2005216781 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7ff9b406af90 0x7ff9b406d450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.985+0000 7ff9da27c640 1 --2- 172.21.15.84:0/2005216781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9d40b39a0 0x7ff9d4156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.985+0000 7ff9da27c640 1 --2- 172.21.15.84:0/2005216781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c00039a0 0x7ff9d40c5a60 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.985+0000 7ff9da27c640 1 -- 172.21.15.84:0/2005216781 >> 172.21.15.84:0/2005216781 conn(0x7ff9d40cb410 msgr2=0x7ff9d40c90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.986+0000 7ff9da27c640 1 -- 172.21.15.84:0/2005216781 shutdown_connections 2023-10-16T10:09:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:07.986+0000 7ff9da27c640 1 -- 172.21.15.84:0/2005216781 wait complete. 2023-10-16T10:09:08.004 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T10:09:08.004 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T10:09:08.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.225+0000 7fcf7cb90640 1 Processor -- start 2023-10-16T10:09:08.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.226+0000 7fcf7cb90640 1 -- start start 2023-10-16T10:09:08.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.227+0000 7fcf7cb90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf78159da0 0x7fcf780c9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:08.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.227+0000 7fcf7cb90640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf7815dfb0 con 0x7fcf780cce20 2023-10-16T10:09:08.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.227+0000 7fcf7cb90640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf7815e290 con 0x7fcf781599c0 2023-10-16T10:09:08.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.227+0000 7fcf7cb90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7815e570 con 0x7fcf78159da0 2023-10-16T10:09:08.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.227+0000 7fcf76d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fcf780cce20 0x7fcf7806c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55946/0 (socket says 172.21.15.84:55946) 2023-10-16T10:09:08.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.227+0000 7fcf76d9d640 1 -- 172.21.15.84:0/3183083462 learned_addr learned my addr 172.21.15.84:0/3183083462 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:09:08.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.228+0000 7fcf75d9b640 1 --2- 172.21.15.84:0/3183083462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf78159da0 0x7fcf780c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:08.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.228+0000 7fcf7559a640 1 -- 172.21.15.84:0/3183083462 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 466835596 0 0) 0x7fcf7815dfb0 con 0x7fcf780cce20 2023-10-16T10:09:08.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.228+0000 7fcf7559a640 1 -- 172.21.15.84:0/3183083462 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf64003520 con 0x7fcf780cce20 2023-10-16T10:09:08.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.228+0000 7fcf7559a640 1 -- 172.21.15.84:0/3183083462 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4177085339 0 0) 0x7fcf64003520 con 0x7fcf780cce20 2023-10-16T10:09:08.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.228+0000 7fcf7559a640 1 -- 172.21.15.84:0/3183083462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf78159da0 msgr2=0x7fcf780c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:08.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.228+0000 7fcf7559a640 1 --2- 172.21.15.84:0/3183083462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf78159da0 0x7fcf780c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:08.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.228+0000 7fcf7559a640 1 -- 172.21.15.84:0/3183083462 >> v1:172.21.15.157:6789/0 conn(0x7fcf781599c0 legacy=0x7fcf7815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:08.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.228+0000 7fcf7559a640 1 -- 172.21.15.84:0/3183083462 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7806fda0 con 0x7fcf780cce20 2023-10-16T10:09:08.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.228+0000 7fcf7559a640 1 -- 172.21.15.84:0/3183083462 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fcf70001a10 con 0x7fcf780cce20 2023-10-16T10:09:08.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.228+0000 7fcf7559a640 1 -- 172.21.15.84:0/3183083462 >> v1:172.21.15.84:6789/0 conn(0x7fcf780cce20 legacy=0x7fcf7806c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:08.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.228+0000 7fcf7559a640 1 --2- 172.21.15.84:0/3183083462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf64003a10 0x7fcf64003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:08.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.228+0000 7fcf7559a640 1 --2- 172.21.15.84:0/3183083462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf640047c0 0x7fcf64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:08.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.228+0000 7fcf7559a640 1 --2- 172.21.15.84:0/3183083462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf78159da0 0x7fcf64005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:08.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.228+0000 7fcf7559a640 1 -- 172.21.15.84:0/3183083462 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7815e570 con 0x7fcf64003a10 2023-10-16T10:09:08.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.228+0000 7fcf7559a640 1 -- 172.21.15.84:0/3183083462 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6400bb40 con 0x7fcf640047c0 2023-10-16T10:09:08.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.228+0000 7fcf7559a640 1 -- 172.21.15.84:0/3183083462 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6400df10 con 0x7fcf78159da0 2023-10-16T10:09:08.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.229+0000 7fcf7659c640 1 --2- 172.21.15.84:0/3183083462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf64003a10 0x7fcf64003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:08.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.229+0000 7fcf75d9b640 1 --2- 172.21.15.84:0/3183083462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf640047c0 0x7fcf64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:08.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.229+0000 7fcf7659c640 1 -- 172.21.15.84:0/3183083462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf78159da0 msgr2=0x7fcf64005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:08.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.229+0000 7fcf7659c640 1 --2- 172.21.15.84:0/3183083462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf78159da0 0x7fcf64005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:08.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.229+0000 7fcf7659c640 1 -- 172.21.15.84:0/3183083462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf640047c0 msgr2=0x7fcf64004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:08.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.229+0000 7fcf7659c640 1 --2- 172.21.15.84:0/3183083462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf640047c0 0x7fcf64004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:08.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.229+0000 7fcf7659c640 1 -- 172.21.15.84:0/3183083462 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf64003520 con 0x7fcf64003a10 2023-10-16T10:09:08.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.229+0000 7fcf75d9b640 1 --2- 172.21.15.84:0/3183083462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf640047c0 0x7fcf64004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:08.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.230+0000 7fcf7659c640 1 --2- 172.21.15.84:0/3183083462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf64003a10 0x7fcf64003e30 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fcf68001750 tx=0x7fcf640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b8e2f5e1962a2e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:08.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.230+0000 7fcf7559a640 1 -- 172.21.15.84:0/3183083462 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf680052d0 con 0x7fcf64003a10 2023-10-16T10:09:08.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.230+0000 7fcf7559a640 1 -- 172.21.15.84:0/3183083462 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6802e050 con 0x7fcf64003a10 2023-10-16T10:09:08.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.230+0000 7fcf7559a640 1 -- 172.21.15.84:0/3183083462 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf680035f0 con 0x7fcf64003a10 2023-10-16T10:09:08.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.231+0000 7fcf7cb90640 1 -- 172.21.15.84:0/3183083462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf64003a10 msgr2=0x7fcf64003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:08.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.231+0000 7fcf7cb90640 1 --2- 172.21.15.84:0/3183083462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf64003a10 0x7fcf64003e30 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fcf68001750 tx=0x7fcf640057f0 comp rx=0 tx=0).stop 2023-10-16T10:09:08.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.231+0000 7fcf76d9d640 1 -- 172.21.15.84:0/3183083462 reap_dead start 2023-10-16T10:09:08.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.231+0000 7fcf7cb90640 1 -- 172.21.15.84:0/3183083462 shutdown_connections 2023-10-16T10:09:08.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.231+0000 7fcf7cb90640 1 -- 172.21.15.84:0/3183083462 >> 172.21.15.84:0/3183083462 conn(0x7fcf780cb360 msgr2=0x7fcf780cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:08.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.231+0000 7fcf7cb90640 1 -- 172.21.15.84:0/3183083462 shutdown_connections 2023-10-16T10:09:08.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.231+0000 7fcf7cb90640 1 -- 172.21.15.84:0/3183083462 wait complete. 2023-10-16T10:09:08.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.233+0000 7fcf7cb90640 1 Processor -- start 2023-10-16T10:09:08.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.233+0000 7fcf7cb90640 1 -- start start 2023-10-16T10:09:08.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.234+0000 7fcf7cb90640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf64003a10 0x7fcf7814d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:08.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.234+0000 7fcf7cb90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7814d930 0x7fcf780c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:08.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.234+0000 7fcf7cb90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf780ac890 0x7fcf780acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:08.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.234+0000 7fcf7cb90640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf780ad250 con 0x7fcf64003a10 2023-10-16T10:09:08.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.234+0000 7fcf7cb90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7814d760 con 0x7fcf7814d930 2023-10-16T10:09:08.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.234+0000 7fcf7cb90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf780c59b0 con 0x7fcf780ac890 2023-10-16T10:09:08.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.234+0000 7fcf7659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf64003a10 0x7fcf7814d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:08.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.234+0000 7fcf7659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf64003a10 0x7fcf7814d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51322/0 (socket says 172.21.15.84:51322) 2023-10-16T10:09:08.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.234+0000 7fcf7659c640 1 -- 172.21.15.84:0/1312919627 learned_addr learned my addr 172.21.15.84:0/1312919627 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:08.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.235+0000 7fcf75d9b640 1 --2- 172.21.15.84:0/1312919627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7814d930 0x7fcf780c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:08.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.235+0000 7fcf7659c640 1 -- 172.21.15.84:0/1312919627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf780ac890 msgr2=0x7fcf780acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:08.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.235+0000 7fcf7659c640 1 --2- 172.21.15.84:0/1312919627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf780ac890 0x7fcf780acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:08.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.235+0000 7fcf7659c640 1 -- 172.21.15.84:0/1312919627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7814d930 msgr2=0x7fcf780c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:08.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.235+0000 7fcf7659c640 1 --2- 172.21.15.84:0/1312919627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7814d930 0x7fcf780c81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:08.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.235+0000 7fcf7659c640 1 -- 172.21.15.84:0/1312919627 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf780cd410 con 0x7fcf64003a10 2023-10-16T10:09:08.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.235+0000 7fcf7659c640 1 --2- 172.21.15.84:0/1312919627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf64003a10 0x7fcf7814d220 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fcf68008fd0 tx=0x7fcf68001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:08.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.235+0000 7fcf6effd640 1 -- 172.21.15.84:0/1312919627 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf680076c0 con 0x7fcf64003a10 2023-10-16T10:09:08.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.235+0000 7fcf6effd640 1 -- 172.21.15.84:0/1312919627 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf68007a30 con 0x7fcf64003a10 2023-10-16T10:09:08.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.235+0000 7fcf6effd640 1 -- 172.21.15.84:0/1312919627 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68006c00 con 0x7fcf64003a10 2023-10-16T10:09:08.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.235+0000 7fcf7cb90640 1 -- 172.21.15.84:0/1312919627 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf780c5c50 con 0x7fcf64003a10 2023-10-16T10:09:08.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.236+0000 7fcf7cb90640 1 -- 172.21.15.84:0/1312919627 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf780c6210 con 0x7fcf64003a10 2023-10-16T10:09:08.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.237+0000 7fcf6cff9640 1 -- 172.21.15.84:0/1312919627 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf50005250 con 0x7fcf64003a10 2023-10-16T10:09:08.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.237+0000 7fcf6effd640 1 -- 172.21.15.84:0/1312919627 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcf680035f0 con 0x7fcf64003a10 2023-10-16T10:09:08.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.238+0000 7fcf6effd640 1 --2- 172.21.15.84:0/1312919627 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fcf5806af90 0x7fcf5806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:08.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.238+0000 7fcf6effd640 1 -- 172.21.15.84:0/1312919627 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fcf680ab760 con 0x7fcf64003a10 2023-10-16T10:09:08.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.239+0000 7fcf75d9b640 1 --2- 172.21.15.84:0/1312919627 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fcf5806af90 0x7fcf5806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:08.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.239+0000 7fcf75d9b640 1 --2- 172.21.15.84:0/1312919627 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fcf5806af90 0x7fcf5806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcf780ca2f0 tx=0x7fcf60008040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:08.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.243+0000 7fcf6effd640 1 -- 172.21.15.84:0/1312919627 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcf6807a4c0 con 0x7fcf64003a10 2023-10-16T10:09:08.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:08.435+0000 7fcf6cff9640 1 -- 172.21.15.84:0/1312919627 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcf500055a0 con 0x7fcf64003a10 2023-10-16T10:09:09.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.165+0000 7fcf6effd640 1 -- 172.21.15.84:0/1312919627 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v148) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcf6807a050 con 0x7fcf64003a10 2023-10-16T10:09:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-16T10:09:09.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.169+0000 7fcf6cff9640 1 -- 172.21.15.84:0/1312919627 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fcf5806af90 msgr2=0x7fcf5806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:09.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.169+0000 7fcf6cff9640 1 --2- 172.21.15.84:0/1312919627 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fcf5806af90 0x7fcf5806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcf780ca2f0 tx=0x7fcf60008040 comp rx=0 tx=0).stop 2023-10-16T10:09:09.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.169+0000 7fcf6cff9640 1 -- 172.21.15.84:0/1312919627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf64003a10 msgr2=0x7fcf7814d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:09.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.169+0000 7fcf6cff9640 1 --2- 172.21.15.84:0/1312919627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf64003a10 0x7fcf7814d220 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fcf68008fd0 tx=0x7fcf68001120 comp rx=0 tx=0).stop 2023-10-16T10:09:09.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.169+0000 7fcf6cff9640 1 -- 172.21.15.84:0/1312919627 shutdown_connections 2023-10-16T10:09:09.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.169+0000 7fcf6cff9640 1 --2- 172.21.15.84:0/1312919627 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fcf5806af90 0x7fcf5806d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:09.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.169+0000 7fcf6cff9640 1 --2- 172.21.15.84:0/1312919627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf780ac890 0x7fcf780acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:09.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.169+0000 7fcf6cff9640 1 --2- 172.21.15.84:0/1312919627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf7814d930 0x7fcf780c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:09.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.169+0000 7fcf6cff9640 1 --2- 172.21.15.84:0/1312919627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf64003a10 0x7fcf7814d220 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:09.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.169+0000 7fcf6cff9640 1 -- 172.21.15.84:0/1312919627 >> 172.21.15.84:0/1312919627 conn(0x7fcf780cb360 msgr2=0x7fcf7806b230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:09.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.170+0000 7fcf6cff9640 1 -- 172.21.15.84:0/1312919627 shutdown_connections 2023-10-16T10:09:09.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.170+0000 7fcf6cff9640 1 -- 172.21.15.84:0/1312919627 wait complete. 2023-10-16T10:09:09.198 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T10:09:09.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.400+0000 7f79c11a9640 1 Processor -- start 2023-10-16T10:09:09.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.400+0000 7f79c11a9640 1 -- start start 2023-10-16T10:09:09.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.401+0000 7f79c11a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f79bc15aaa0 0x7f79bc15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:09.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.401+0000 7f79c11a9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f79bc15ae80 0x7f79bc0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:09.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.401+0000 7f79c11a9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79bc0707e0 con 0x7f79bc0ccaa0 2023-10-16T10:09:09.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.401+0000 7f79c11a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79bc06b7a0 con 0x7f79bc15aaa0 2023-10-16T10:09:09.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.401+0000 7f79c11a9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79bc15ef20 con 0x7f79bc15ae80 2023-10-16T10:09:09.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.402+0000 7f79bad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f79bc15aaa0 0x7f79bc15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:09.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.402+0000 7f79bb59e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f79bc0ccaa0 0x7f79bc070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55962/0 (socket says 172.21.15.84:55962) 2023-10-16T10:09:09.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.402+0000 7f79bb59e640 1 -- 172.21.15.84:0/1107727753 learned_addr learned my addr 172.21.15.84:0/1107727753 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:09:09.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.402+0000 7f79b3fff640 1 --2- 172.21.15.84:0/1107727753 >> v2:172.21.15.157:3300/0 conn(0x7f79bc15ae80 0x7f79bc0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:09.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.402+0000 7f79bad9d640 1 -- 172.21.15.84:0/1107727753 >> v2:172.21.15.157:3300/0 conn(0x7f79bc15ae80 msgr2=0x7f79bc0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:09.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.402+0000 7f79bad9d640 1 --2- 172.21.15.84:0/1107727753 >> v2:172.21.15.157:3300/0 conn(0x7f79bc15ae80 0x7f79bc0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:09.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.402+0000 7f79bad9d640 1 -- 172.21.15.84:0/1107727753 >> v1:172.21.15.84:6789/0 conn(0x7f79bc0ccaa0 legacy=0x7f79bc070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:09.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.402+0000 7f79bad9d640 1 -- 172.21.15.84:0/1107727753 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc15f200 con 0x7f79bc15aaa0 2023-10-16T10:09:09.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.402+0000 7f79ba59c640 1 -- 172.21.15.84:0/1107727753 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2232595478 0 0) 0x7f79bc0707e0 con 0x7f79bc0ccaa0 2023-10-16T10:09:09.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.403+0000 7f79bad9d640 1 --2- 172.21.15.84:0/1107727753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79bc15aaa0 0x7f79bc15e820 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0b45e0 tx=0x7f79ac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69be35f6e69ab31c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:09.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.404+0000 7f79ba59c640 1 -- 172.21.15.84:0/1107727753 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac02e070 con 0x7f79bc15aaa0 2023-10-16T10:09:09.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.404+0000 7f79ba59c640 1 -- 172.21.15.84:0/1107727753 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ac0315a0 con 0x7f79bc15aaa0 2023-10-16T10:09:09.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.404+0000 7f79ba59c640 1 -- 172.21.15.84:0/1107727753 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac031930 con 0x7f79bc15aaa0 2023-10-16T10:09:09.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.404+0000 7f79c11a9640 1 -- 172.21.15.84:0/1107727753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79bc15aaa0 msgr2=0x7f79bc15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:09.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.404+0000 7f79c11a9640 1 --2- 172.21.15.84:0/1107727753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79bc15aaa0 0x7f79bc15e820 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0b45e0 tx=0x7f79ac02b840 comp rx=0 tx=0).stop 2023-10-16T10:09:09.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.404+0000 7f79c11a9640 1 -- 172.21.15.84:0/1107727753 shutdown_connections 2023-10-16T10:09:09.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.404+0000 7f79c11a9640 1 --2- 172.21.15.84:0/1107727753 >> v2:172.21.15.157:3300/0 conn(0x7f79bc15ae80 0x7f79bc0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:09.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.404+0000 7f79c11a9640 1 --2- 172.21.15.84:0/1107727753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79bc15aaa0 0x7f79bc15e820 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:09.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.404+0000 7f79c11a9640 1 -- 172.21.15.84:0/1107727753 >> 172.21.15.84:0/1107727753 conn(0x7f79bc0cb410 msgr2=0x7f79bc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:09.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.405+0000 7f79c11a9640 1 -- 172.21.15.84:0/1107727753 shutdown_connections 2023-10-16T10:09:09.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.405+0000 7f79c11a9640 1 -- 172.21.15.84:0/1107727753 wait complete. 2023-10-16T10:09:09.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.406+0000 7f79c11a9640 1 Processor -- start 2023-10-16T10:09:09.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.406+0000 7f79c11a9640 1 -- start start 2023-10-16T10:09:09.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.407+0000 7f79c11a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79bc0ccaa0 0x7f79bc0c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:09.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.407+0000 7f79c11a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79bc15aaa0 0x7f79bc0b3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:09.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.407+0000 7f79c11a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79bc15ae80 0x7f79bc150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:09.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.407+0000 7f79c11a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc0c8e60 con 0x7f79bc0ccaa0 2023-10-16T10:09:09.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.407+0000 7f79c11a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc1f23f0 con 0x7f79bc15ae80 2023-10-16T10:09:09.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.407+0000 7f79c11a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc1513b0 con 0x7f79bc15aaa0 2023-10-16T10:09:09.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.408+0000 7f79bad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79bc0ccaa0 0x7f79bc0c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:09.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.408+0000 7f79bad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79bc0ccaa0 0x7f79bc0c7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51352/0 (socket says 172.21.15.84:51352) 2023-10-16T10:09:09.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.408+0000 7f79bad9d640 1 -- 172.21.15.84:0/2477964135 learned_addr learned my addr 172.21.15.84:0/2477964135 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:09.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.408+0000 7f79bb59e640 1 --2- 172.21.15.84:0/2477964135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79bc15ae80 0x7f79bc150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:09.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.408+0000 7f79bad9d640 1 -- 172.21.15.84:0/2477964135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79bc15aaa0 msgr2=0x7f79bc0b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:09.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.408+0000 7f79bad9d640 1 --2- 172.21.15.84:0/2477964135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79bc15aaa0 0x7f79bc0b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:09.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.408+0000 7f79bad9d640 1 -- 172.21.15.84:0/2477964135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79bc15ae80 msgr2=0x7f79bc150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:09.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.408+0000 7f79bad9d640 1 --2- 172.21.15.84:0/2477964135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79bc15ae80 0x7f79bc150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:09.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.408+0000 7f79bad9d640 1 -- 172.21.15.84:0/2477964135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac009400 con 0x7f79bc0ccaa0 2023-10-16T10:09:09.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.408+0000 7f79bb59e640 1 --2- 172.21.15.84:0/2477964135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79bc15ae80 0x7f79bc150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:09.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.408+0000 7f79bad9d640 1 --2- 172.21.15.84:0/2477964135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79bc0ccaa0 0x7f79bc0c7b00 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009890 tx=0x7f79ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:09.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.409+0000 7f79b37fe640 1 -- 172.21.15.84:0/2477964135 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac032840 con 0x7f79bc0ccaa0 2023-10-16T10:09:09.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.409+0000 7f79b37fe640 1 -- 172.21.15.84:0/2477964135 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ac0329a0 con 0x7f79bc0ccaa0 2023-10-16T10:09:09.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.409+0000 7f79b37fe640 1 -- 172.21.15.84:0/2477964135 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac032d50 con 0x7f79bc0ccaa0 2023-10-16T10:09:09.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.409+0000 7f79c11a9640 1 -- 172.21.15.84:0/2477964135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79bc0c81e0 con 0x7f79bc0ccaa0 2023-10-16T10:09:09.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.409+0000 7f79c11a9640 1 -- 172.21.15.84:0/2477964135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79bc0c87a0 con 0x7f79bc0ccaa0 2023-10-16T10:09:09.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.410+0000 7f79c11a9640 1 -- 172.21.15.84:0/2477964135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79bc04e160 con 0x7f79bc0ccaa0 2023-10-16T10:09:09.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.411+0000 7f79b37fe640 1 -- 172.21.15.84:0/2477964135 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f79ac0453f0 con 0x7f79bc0ccaa0 2023-10-16T10:09:09.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.412+0000 7f79b37fe640 1 --2- 172.21.15.84:0/2477964135 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f799c06af90 0x7f799c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:09.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.412+0000 7f79b37fe640 1 -- 172.21.15.84:0/2477964135 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5255+0+0 (secure 0 0 0) 0x7f79ac0ab970 con 0x7f79bc0ccaa0 2023-10-16T10:09:09.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.412+0000 7f79b3fff640 1 --2- 172.21.15.84:0/2477964135 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f799c06af90 0x7f799c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:09.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.413+0000 7f79b3fff640 1 --2- 172.21.15.84:0/2477964135 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f799c06af90 0x7f799c06d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f79a4006fd0 tx=0x7f79a4008040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:09.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.417+0000 7f79b37fe640 1 -- 172.21.15.84:0/2477964135 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f79ac076ae0 con 0x7f79bc0ccaa0 2023-10-16T10:09:09.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:09.610+0000 7f79c11a9640 1 -- 172.21.15.84:0/2477964135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f79bc0cdc60 con 0x7f79bc0ccaa0 2023-10-16T10:09:10.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.171+0000 7f79b37fe640 1 -- 172.21.15.84:0/2477964135 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v149) v1 ==== 170+0+0 (secure 0 0 0) 0x7f79ac07a100 con 0x7f79bc0ccaa0 2023-10-16T10:09:10.173 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-16T10:09:10.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.176+0000 7f79c11a9640 1 -- 172.21.15.84:0/2477964135 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f799c06af90 msgr2=0x7f799c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:10.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.176+0000 7f79c11a9640 1 --2- 172.21.15.84:0/2477964135 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f799c06af90 0x7f799c06d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f79a4006fd0 tx=0x7f79a4008040 comp rx=0 tx=0).stop 2023-10-16T10:09:10.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.176+0000 7f79c11a9640 1 -- 172.21.15.84:0/2477964135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79bc0ccaa0 msgr2=0x7f79bc0c7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:10.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.176+0000 7f79c11a9640 1 --2- 172.21.15.84:0/2477964135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79bc0ccaa0 0x7f79bc0c7b00 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009890 tx=0x7f79ac0046c0 comp rx=0 tx=0).stop 2023-10-16T10:09:10.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.177+0000 7f79c11a9640 1 -- 172.21.15.84:0/2477964135 shutdown_connections 2023-10-16T10:09:10.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.177+0000 7f79c11a9640 1 --2- 172.21.15.84:0/2477964135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79bc15ae80 0x7f79bc150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:10.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.177+0000 7f79c11a9640 1 --2- 172.21.15.84:0/2477964135 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f799c06af90 0x7f799c06d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:10.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.177+0000 7f79c11a9640 1 --2- 172.21.15.84:0/2477964135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79bc15aaa0 0x7f79bc0b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:10.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.177+0000 7f79c11a9640 1 --2- 172.21.15.84:0/2477964135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79bc0ccaa0 0x7f79bc0c7b00 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:10.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.177+0000 7f79c11a9640 1 -- 172.21.15.84:0/2477964135 >> 172.21.15.84:0/2477964135 conn(0x7f79bc0cb410 msgr2=0x7f79bc0b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:10.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.180+0000 7f79c11a9640 1 -- 172.21.15.84:0/2477964135 shutdown_connections 2023-10-16T10:09:10.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.181+0000 7f79c11a9640 1 -- 172.21.15.84:0/2477964135 wait complete. 2023-10-16T10:09:10.216 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T10:09:10.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.402+0000 7f7b21767640 1 Processor -- start 2023-10-16T10:09:10.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.402+0000 7f7b21767640 1 -- start start 2023-10-16T10:09:10.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.404+0000 7f7b21767640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7b1c0b8480 0x7f7b1c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:10.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.404+0000 7f7b21767640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1c15d150 con 0x7f7b1c0d0be0 2023-10-16T10:09:10.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.404+0000 7f7b21767640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1c0d1180 con 0x7f7b1c15cd70 2023-10-16T10:09:10.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.404+0000 7f7b21767640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b1c0d1460 con 0x7f7b1c0b8480 2023-10-16T10:09:10.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.404+0000 7f7b20f66640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7b1c0d0be0 0x7f7b1c0b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55976/0 (socket says 172.21.15.84:55976) 2023-10-16T10:09:10.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.404+0000 7f7b20f66640 1 -- 172.21.15.84:0/3547359681 learned_addr learned my addr 172.21.15.84:0/3547359681 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:09:10.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.404+0000 7f7b13fff640 1 --2- 172.21.15.84:0/3547359681 >> v2:172.21.15.157:3300/0 conn(0x7f7b1c0b8480 0x7f7b1c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:10.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.404+0000 7f7b1b7fe640 1 -- 172.21.15.84:0/3547359681 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3655015587 0 0) 0x7f7b1c15d150 con 0x7f7b1c0d0be0 2023-10-16T10:09:10.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.405+0000 7f7b1b7fe640 1 -- 172.21.15.84:0/3547359681 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b08003520 con 0x7f7b1c0d0be0 2023-10-16T10:09:10.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.405+0000 7f7b1b7fe640 1 -- 172.21.15.84:0/3547359681 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3729292147 0 0) 0x7f7b08003520 con 0x7f7b1c0d0be0 2023-10-16T10:09:10.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.405+0000 7f7b1b7fe640 1 -- 172.21.15.84:0/3547359681 >> v2:172.21.15.157:3300/0 conn(0x7f7b1c0b8480 msgr2=0x7f7b1c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:10.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.405+0000 7f7b1b7fe640 1 --2- 172.21.15.84:0/3547359681 >> v2:172.21.15.157:3300/0 conn(0x7f7b1c0b8480 0x7f7b1c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:10.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.405+0000 7f7b1b7fe640 1 -- 172.21.15.84:0/3547359681 >> v1:172.21.15.157:6789/0 conn(0x7f7b1c15cd70 legacy=0x7f7b1c0d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:10.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.405+0000 7f7b1b7fe640 1 -- 172.21.15.84:0/3547359681 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c0d1740 con 0x7f7b1c0d0be0 2023-10-16T10:09:10.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.405+0000 7f7b1b7fe640 1 -- 172.21.15.84:0/3547359681 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f7b140019f0 con 0x7f7b1c0d0be0 2023-10-16T10:09:10.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.405+0000 7f7b1b7fe640 1 -- 172.21.15.84:0/3547359681 >> v1:172.21.15.84:6789/0 conn(0x7f7b1c0d0be0 legacy=0x7f7b1c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:10.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.405+0000 7f7b1b7fe640 1 --2- 172.21.15.84:0/3547359681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b08003970 0x7f7b08003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:10.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.405+0000 7f7b1b7fe640 1 --2- 172.21.15.84:0/3547359681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b080047c0 0x7f7b08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:10.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.405+0000 7f7b1b7fe640 1 --2- 172.21.15.84:0/3547359681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b08005180 0x7f7b0800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:10.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.405+0000 7f7b1b7fe640 1 -- 172.21.15.84:0/3547359681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c0d1460 con 0x7f7b08005180 2023-10-16T10:09:10.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.405+0000 7f7b1b7fe640 1 -- 172.21.15.84:0/3547359681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0800c120 con 0x7f7b08003970 2023-10-16T10:09:10.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.405+0000 7f7b1b7fe640 1 -- 172.21.15.84:0/3547359681 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0800c2e0 con 0x7f7b080047c0 2023-10-16T10:09:10.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.405+0000 7f7b20f66640 1 --2- 172.21.15.84:0/3547359681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b08005180 0x7f7b0800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:10.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.406+0000 7f7b1bfff640 1 --2- 172.21.15.84:0/3547359681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b08003970 0x7f7b08003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:10.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.406+0000 7f7b20f66640 1 -- 172.21.15.84:0/3547359681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b080047c0 msgr2=0x7f7b08004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:10.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.406+0000 7f7b20f66640 1 --2- 172.21.15.84:0/3547359681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b080047c0 0x7f7b08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:10.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.406+0000 7f7b20f66640 1 -- 172.21.15.84:0/3547359681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b08003970 msgr2=0x7f7b08003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:10.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.406+0000 7f7b20f66640 1 --2- 172.21.15.84:0/3547359681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b08003970 0x7f7b08003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:10.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.406+0000 7f7b20f66640 1 -- 172.21.15.84:0/3547359681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b08003520 con 0x7f7b08005180 2023-10-16T10:09:10.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.407+0000 7f7b20f66640 1 -- 172.21.15.84:0/3547359681 reap_dead start 2023-10-16T10:09:10.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.407+0000 7f7b20f66640 1 --2- 172.21.15.84:0/3547359681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b08005180 0x7f7b0800b980 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400a830 tx=0x7f7b080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06343cc2865aead server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:10.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.407+0000 7f7b1b7fe640 1 -- 172.21.15.84:0/3547359681 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b140072d0 con 0x7f7b08005180 2023-10-16T10:09:10.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.407+0000 7f7b1b7fe640 1 -- 172.21.15.84:0/3547359681 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1402f070 con 0x7f7b08005180 2023-10-16T10:09:10.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.407+0000 7f7b1b7fe640 1 -- 172.21.15.84:0/3547359681 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b140076b0 con 0x7f7b08005180 2023-10-16T10:09:10.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.408+0000 7f7b21767640 1 -- 172.21.15.84:0/3547359681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b08005180 msgr2=0x7f7b0800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:10.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.408+0000 7f7b21767640 1 --2- 172.21.15.84:0/3547359681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b08005180 0x7f7b0800b980 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400a830 tx=0x7f7b080057f0 comp rx=0 tx=0).stop 2023-10-16T10:09:10.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.408+0000 7f7b21767640 1 -- 172.21.15.84:0/3547359681 shutdown_connections 2023-10-16T10:09:10.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.408+0000 7f7b21767640 1 --2- 172.21.15.84:0/3547359681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b08005180 0x7f7b0800b980 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:10.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.408+0000 7f7b21767640 1 -- 172.21.15.84:0/3547359681 >> 172.21.15.84:0/3547359681 conn(0x7f7b1c0cf570 msgr2=0x7f7b1c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:10.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.408+0000 7f7b21767640 1 -- 172.21.15.84:0/3547359681 shutdown_connections 2023-10-16T10:09:10.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.408+0000 7f7b21767640 1 -- 172.21.15.84:0/3547359681 wait complete. 2023-10-16T10:09:10.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.410+0000 7f7b21767640 1 Processor -- start 2023-10-16T10:09:10.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.410+0000 7f7b21767640 1 -- start start 2023-10-16T10:09:10.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.411+0000 7f7b21767640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b08005180 0x7f7b1c0b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:10.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.411+0000 7f7b21767640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b1c15dcd0 0x7f7b1c1560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:10.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.411+0000 7f7b21767640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b1c158a40 0x7f7b1c156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:10.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.411+0000 7f7b21767640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c15e4a0 con 0x7f7b1c15dcd0 2023-10-16T10:09:10.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.411+0000 7f7b21767640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c159c50 con 0x7f7b1c158a40 2023-10-16T10:09:10.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.411+0000 7f7b21767640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c1593b0 con 0x7f7b08005180 2023-10-16T10:09:10.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.411+0000 7f7b13fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b1c15dcd0 0x7f7b1c1560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:10.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.411+0000 7f7b13fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b1c15dcd0 0x7f7b1c1560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51380/0 (socket says 172.21.15.84:51380) 2023-10-16T10:09:10.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.411+0000 7f7b13fff640 1 -- 172.21.15.84:0/1781140276 learned_addr learned my addr 172.21.15.84:0/1781140276 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:10.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.412+0000 7f7b13fff640 1 -- 172.21.15.84:0/1781140276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b08005180 msgr2=0x7f7b1c0b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:10.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.412+0000 7f7b13fff640 1 --2- 172.21.15.84:0/1781140276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b08005180 0x7f7b1c0b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:10.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.412+0000 7f7b20f66640 1 --2- 172.21.15.84:0/1781140276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b1c158a40 0x7f7b1c156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:10.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.412+0000 7f7b13fff640 1 -- 172.21.15.84:0/1781140276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b1c158a40 msgr2=0x7f7b1c156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:10.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.412+0000 7f7b13fff640 1 --2- 172.21.15.84:0/1781140276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b1c158a40 0x7f7b1c156a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:10.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.412+0000 7f7b13fff640 1 -- 172.21.15.84:0/1781140276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c159690 con 0x7f7b1c15dcd0 2023-10-16T10:09:10.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.412+0000 7f7b13fff640 1 --2- 172.21.15.84:0/1781140276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b1c15dcd0 0x7f7b1c1560c0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400e7a0 tx=0x7f7b0400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:10.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.412+0000 7f7b197fa640 1 -- 172.21.15.84:0/1781140276 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0400eeb0 con 0x7f7b1c15dcd0 2023-10-16T10:09:10.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.412+0000 7f7b197fa640 1 -- 172.21.15.84:0/1781140276 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0400c8e0 con 0x7f7b1c15dcd0 2023-10-16T10:09:10.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.412+0000 7f7b197fa640 1 -- 172.21.15.84:0/1781140276 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0400ccb0 con 0x7f7b1c15dcd0 2023-10-16T10:09:10.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.412+0000 7f7b21767640 1 -- 172.21.15.84:0/1781140276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1c159970 con 0x7f7b1c15dcd0 2023-10-16T10:09:10.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.412+0000 7f7b21767640 1 -- 172.21.15.84:0/1781140276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1c158ec0 con 0x7f7b1c15dcd0 2023-10-16T10:09:10.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.413+0000 7f7b21767640 1 -- 172.21.15.84:0/1781140276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b1c15e710 con 0x7f7b1c15dcd0 2023-10-16T10:09:10.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.415+0000 7f7b197fa640 1 -- 172.21.15.84:0/1781140276 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7b04013470 con 0x7f7b1c15dcd0 2023-10-16T10:09:10.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.416+0000 7f7b197fa640 1 --2- 172.21.15.84:0/1781140276 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f7afc06af90 0x7f7afc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:10.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.416+0000 7f7b197fa640 1 -- 172.21.15.84:0/1781140276 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5637+0+0 (secure 0 0 0) 0x7f7b04089ff0 con 0x7f7b1c15dcd0 2023-10-16T10:09:10.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.416+0000 7f7b1bfff640 1 --2- 172.21.15.84:0/1781140276 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f7afc06af90 0x7f7afc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:10.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.417+0000 7f7b1bfff640 1 --2- 172.21.15.84:0/1781140276 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f7afc06af90 0x7f7afc06d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0014f0 tx=0x7f7b0c009040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:10.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.420+0000 7f7b197fa640 1 -- 172.21.15.84:0/1781140276 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7b04054fd0 con 0x7f7b1c15dcd0 2023-10-16T10:09:10.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:10.650+0000 7f7b21767640 1 -- 172.21.15.84:0/1781140276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7b1c04e9b0 con 0x7f7b1c15dcd0 2023-10-16T10:09:11.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.217+0000 7f7b197fa640 1 -- 172.21.15.84:0/1781140276 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v76) v1 ==== 174+0+0 (secure 0 0 0) 0x7f7b040585f0 con 0x7f7b1c15dcd0 2023-10-16T10:09:11.222 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 18 and data pool 19 2023-10-16T10:09:11.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.228+0000 7f7b21767640 1 -- 172.21.15.84:0/1781140276 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f7afc06af90 msgr2=0x7f7afc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:11.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.228+0000 7f7b21767640 1 --2- 172.21.15.84:0/1781140276 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f7afc06af90 0x7f7afc06d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0014f0 tx=0x7f7b0c009040 comp rx=0 tx=0).stop 2023-10-16T10:09:11.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.228+0000 7f7b21767640 1 -- 172.21.15.84:0/1781140276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b1c15dcd0 msgr2=0x7f7b1c1560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:11.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.228+0000 7f7b21767640 1 --2- 172.21.15.84:0/1781140276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b1c15dcd0 0x7f7b1c1560c0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400e7a0 tx=0x7f7b0400ec60 comp rx=0 tx=0).stop 2023-10-16T10:09:11.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.230+0000 7f7b21767640 1 -- 172.21.15.84:0/1781140276 shutdown_connections 2023-10-16T10:09:11.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.230+0000 7f7b21767640 1 --2- 172.21.15.84:0/1781140276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b1c158a40 0x7f7b1c156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.230+0000 7f7b21767640 1 --2- 172.21.15.84:0/1781140276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b1c15dcd0 0x7f7b1c1560c0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.230+0000 7f7b21767640 1 --2- 172.21.15.84:0/1781140276 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f7afc06af90 0x7f7afc06d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.230+0000 7f7b21767640 1 --2- 172.21.15.84:0/1781140276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b08005180 0x7f7b1c0b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.230+0000 7f7b21767640 1 -- 172.21.15.84:0/1781140276 >> 172.21.15.84:0/1781140276 conn(0x7f7b1c0cf570 msgr2=0x7f7b1c0d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:11.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.230+0000 7f7b21767640 1 -- 172.21.15.84:0/1781140276 shutdown_connections 2023-10-16T10:09:11.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.231+0000 7f7b21767640 1 -- 172.21.15.84:0/1781140276 wait complete. 2023-10-16T10:09:11.264 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:09:11.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.439+0000 7ffa53316640 1 Processor -- start 2023-10-16T10:09:11.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.440+0000 7ffa53316640 1 -- start start 2023-10-16T10:09:11.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.441+0000 7ffa53316640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa4c159b20 0x7ffa4c06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:11.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.441+0000 7ffa53316640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa4c0b0b20 0x7ffa4c06fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:11.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.441+0000 7ffa53316640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa4c06f7f0 con 0x7ffa4c159f00 2023-10-16T10:09:11.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.441+0000 7ffa53316640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa4c06c5e0 con 0x7ffa4c159b20 2023-10-16T10:09:11.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.441+0000 7ffa53316640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c06c8c0 con 0x7ffa4c0b0b20 2023-10-16T10:09:11.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.441+0000 7ffa508b1640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ffa4c159f00 0x7ffa4c0b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55988/0 (socket says 172.21.15.84:55988) 2023-10-16T10:09:11.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.441+0000 7ffa508b1640 1 -- 172.21.15.84:0/2312179521 learned_addr learned my addr 172.21.15.84:0/2312179521 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:09:11.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa518b3640 1 --2- 172.21.15.84:0/2312179521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa4c0b0b20 0x7ffa4c06fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:11.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa510b2640 1 --2- 172.21.15.84:0/2312179521 >> v2:172.21.15.157:3300/0 conn(0x7ffa4c159b20 0x7ffa4c06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:11.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa4bfff640 1 -- 172.21.15.84:0/2312179521 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955481890 0 0) 0x7ffa4c06f7f0 con 0x7ffa4c159f00 2023-10-16T10:09:11.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa4bfff640 1 -- 172.21.15.84:0/2312179521 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa34003520 con 0x7ffa4c159f00 2023-10-16T10:09:11.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa4bfff640 1 -- 172.21.15.84:0/2312179521 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1998015222 0 0) 0x7ffa34003520 con 0x7ffa4c159f00 2023-10-16T10:09:11.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa4bfff640 1 -- 172.21.15.84:0/2312179521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa4c0b0b20 msgr2=0x7ffa4c06fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:11.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa4bfff640 1 --2- 172.21.15.84:0/2312179521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa4c0b0b20 0x7ffa4c06fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa4bfff640 1 -- 172.21.15.84:0/2312179521 >> v2:172.21.15.157:3300/0 conn(0x7ffa4c159b20 msgr2=0x7ffa4c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:11.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa4bfff640 1 --2- 172.21.15.84:0/2312179521 >> v2:172.21.15.157:3300/0 conn(0x7ffa4c159b20 0x7ffa4c06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa4bfff640 1 -- 172.21.15.84:0/2312179521 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c06cba0 con 0x7ffa4c159f00 2023-10-16T10:09:11.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa4bfff640 1 -- 172.21.15.84:0/2312179521 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ffa40001670 con 0x7ffa4c159f00 2023-10-16T10:09:11.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa4bfff640 1 -- 172.21.15.84:0/2312179521 >> v1:172.21.15.84:6789/0 conn(0x7ffa4c159f00 legacy=0x7ffa4c0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:11.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa4bfff640 1 --2- 172.21.15.84:0/2312179521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa34003970 0x7ffa34003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:11.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa4bfff640 1 --2- 172.21.15.84:0/2312179521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa340047c0 0x7ffa34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:11.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa4bfff640 1 --2- 172.21.15.84:0/2312179521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa34005180 0x7ffa3400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:11.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa4bfff640 1 -- 172.21.15.84:0/2312179521 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c06c5e0 con 0x7ffa340047c0 2023-10-16T10:09:11.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa4bfff640 1 -- 172.21.15.84:0/2312179521 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c06c8c0 con 0x7ffa34005180 2023-10-16T10:09:11.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa4bfff640 1 -- 172.21.15.84:0/2312179521 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3400e230 con 0x7ffa34003970 2023-10-16T10:09:11.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa518b3640 1 --2- 172.21.15.84:0/2312179521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa4c0b0b20 0x7ffa4c06fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:09:11.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa510b2640 1 --2- 172.21.15.84:0/2312179521 >> v2:172.21.15.157:3300/0 conn(0x7ffa4c159b20 0x7ffa4c06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:09:11.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.442+0000 7ffa508b1640 1 --2- 172.21.15.84:0/2312179521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa340047c0 0x7ffa34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:11.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.443+0000 7ffa508b1640 1 -- 172.21.15.84:0/2312179521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa34003970 msgr2=0x7ffa34003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:11.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.443+0000 7ffa508b1640 1 --2- 172.21.15.84:0/2312179521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa34003970 0x7ffa34003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.443+0000 7ffa508b1640 1 -- 172.21.15.84:0/2312179521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa34005180 msgr2=0x7ffa3400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:11.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.443+0000 7ffa508b1640 1 --2- 172.21.15.84:0/2312179521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa34005180 0x7ffa3400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.443+0000 7ffa508b1640 1 -- 172.21.15.84:0/2312179521 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa34003520 con 0x7ffa340047c0 2023-10-16T10:09:11.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.444+0000 7ffa508b1640 1 --2- 172.21.15.84:0/2312179521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa340047c0 0x7ffa34004c40 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000c0e0 tx=0x7ffa340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab44be53af626f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:11.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.444+0000 7ffa4bfff640 1 -- 172.21.15.84:0/2312179521 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa40005ad0 con 0x7ffa340047c0 2023-10-16T10:09:11.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.444+0000 7ffa4bfff640 1 -- 172.21.15.84:0/2312179521 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa40005c50 con 0x7ffa340047c0 2023-10-16T10:09:11.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.444+0000 7ffa4bfff640 1 -- 172.21.15.84:0/2312179521 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4002c810 con 0x7ffa340047c0 2023-10-16T10:09:11.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.445+0000 7ffa53316640 1 -- 172.21.15.84:0/2312179521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa340047c0 msgr2=0x7ffa34004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:11.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.445+0000 7ffa53316640 1 --2- 172.21.15.84:0/2312179521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa340047c0 0x7ffa34004c40 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000c0e0 tx=0x7ffa340057f0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.445+0000 7ffa518b3640 1 -- 172.21.15.84:0/2312179521 reap_dead start 2023-10-16T10:09:11.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.445+0000 7ffa53316640 1 -- 172.21.15.84:0/2312179521 shutdown_connections 2023-10-16T10:09:11.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.445+0000 7ffa53316640 1 -- 172.21.15.84:0/2312179521 >> 172.21.15.84:0/2312179521 conn(0x7ffa4c0cb410 msgr2=0x7ffa4c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:11.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.445+0000 7ffa53316640 1 -- 172.21.15.84:0/2312179521 shutdown_connections 2023-10-16T10:09:11.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.445+0000 7ffa53316640 1 -- 172.21.15.84:0/2312179521 wait complete. 2023-10-16T10:09:11.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.447+0000 7ffa53316640 1 Processor -- start 2023-10-16T10:09:11.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.447+0000 7ffa53316640 1 -- start start 2023-10-16T10:09:11.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.448+0000 7ffa53316640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa340047c0 0x7ffa4c15ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:11.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.448+0000 7ffa53316640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa4c150880 0x7ffa4c0cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:11.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.448+0000 7ffa53316640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa4c14fc60 0x7ffa4c1500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:11.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.448+0000 7ffa53316640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c1510f0 con 0x7ffa340047c0 2023-10-16T10:09:11.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.448+0000 7ffa53316640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c0c5c70 con 0x7ffa4c14fc60 2023-10-16T10:09:11.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.448+0000 7ffa53316640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c0c5f10 con 0x7ffa4c150880 2023-10-16T10:09:11.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.448+0000 7ffa510b2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa340047c0 0x7ffa4c15ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:11.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.448+0000 7ffa510b2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa340047c0 0x7ffa4c15ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51394/0 (socket says 172.21.15.84:51394) 2023-10-16T10:09:11.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.448+0000 7ffa510b2640 1 -- 172.21.15.84:0/3444312020 learned_addr learned my addr 172.21.15.84:0/3444312020 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:11.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.448+0000 7ffa510b2640 1 -- 172.21.15.84:0/3444312020 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa4c150880 msgr2=0x7ffa4c0cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:11.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.448+0000 7ffa510b2640 1 --2- 172.21.15.84:0/3444312020 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa4c150880 0x7ffa4c0cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.448+0000 7ffa510b2640 1 -- 172.21.15.84:0/3444312020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa4c14fc60 msgr2=0x7ffa4c1500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:11.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.448+0000 7ffa510b2640 1 --2- 172.21.15.84:0/3444312020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa4c14fc60 0x7ffa4c1500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.448+0000 7ffa510b2640 1 -- 172.21.15.84:0/3444312020 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c0c61f0 con 0x7ffa340047c0 2023-10-16T10:09:11.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.449+0000 7ffa510b2640 1 --2- 172.21.15.84:0/3444312020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa340047c0 0x7ffa4c15ccd0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c007f70 tx=0x7ffa3c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:11.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.449+0000 7ffa49ffb640 1 -- 172.21.15.84:0/3444312020 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c003b90 con 0x7ffa340047c0 2023-10-16T10:09:11.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.449+0000 7ffa49ffb640 1 -- 172.21.15.84:0/3444312020 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3c00f040 con 0x7ffa340047c0 2023-10-16T10:09:11.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.449+0000 7ffa53316640 1 -- 172.21.15.84:0/3444312020 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c0c64d0 con 0x7ffa340047c0 2023-10-16T10:09:11.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.449+0000 7ffa49ffb640 1 -- 172.21.15.84:0/3444312020 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c004030 con 0x7ffa340047c0 2023-10-16T10:09:11.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.449+0000 7ffa53316640 1 -- 172.21.15.84:0/3444312020 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c150620 con 0x7ffa340047c0 2023-10-16T10:09:11.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.450+0000 7ffa53316640 1 -- 172.21.15.84:0/3444312020 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c06ce60 con 0x7ffa340047c0 2023-10-16T10:09:11.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.451+0000 7ffa49ffb640 1 -- 172.21.15.84:0/3444312020 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ffa3c004190 con 0x7ffa340047c0 2023-10-16T10:09:11.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.452+0000 7ffa49ffb640 1 --2- 172.21.15.84:0/3444312020 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7ffa3806afc0 0x7ffa3806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:11.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.452+0000 7ffa49ffb640 1 -- 172.21.15.84:0/3444312020 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ffa3c089cc0 con 0x7ffa340047c0 2023-10-16T10:09:11.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.452+0000 7ffa508b1640 1 --2- 172.21.15.84:0/3444312020 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7ffa3806afc0 0x7ffa3806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:11.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.453+0000 7ffa508b1640 1 --2- 172.21.15.84:0/3444312020 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7ffa3806afc0 0x7ffa3806d480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffa40034000 tx=0x7ffa4002e000 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:11.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.456+0000 7ffa49ffb640 1 -- 172.21.15.84:0/3444312020 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffa4c06ce60 con 0x7ffa340047c0 2023-10-16T10:09:11.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.649+0000 7ffa53316640 1 -- 172.21.15.84:0/3444312020 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa4c06ce60 con 0x7ffa340047c0 2023-10-16T10:09:11.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.652+0000 7ffa49ffb640 1 -- 172.21.15.84:0/3444312020 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16612 (secure 0 0 0) 0x7ffa4c06ce60 con 0x7ffa340047c0 2023-10-16T10:09:11.653 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:09:11.654 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":150,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:09:10.651585+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:09:08.490425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-16T10:09:09.666051+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:09:11.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.655+0000 7ffa53316640 1 -- 172.21.15.84:0/3444312020 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7ffa3806afc0 msgr2=0x7ffa3806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:11.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.655+0000 7ffa53316640 1 --2- 172.21.15.84:0/3444312020 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7ffa3806afc0 0x7ffa3806d480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffa40034000 tx=0x7ffa4002e000 comp rx=0 tx=0).stop 2023-10-16T10:09:11.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.655+0000 7ffa53316640 1 -- 172.21.15.84:0/3444312020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa340047c0 msgr2=0x7ffa4c15ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:11.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.655+0000 7ffa53316640 1 --2- 172.21.15.84:0/3444312020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa340047c0 0x7ffa4c15ccd0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c007f70 tx=0x7ffa3c00d560 comp rx=0 tx=0).stop 2023-10-16T10:09:11.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.655+0000 7ffa53316640 1 -- 172.21.15.84:0/3444312020 shutdown_connections 2023-10-16T10:09:11.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.655+0000 7ffa53316640 1 --2- 172.21.15.84:0/3444312020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa4c14fc60 0x7ffa4c1500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.655+0000 7ffa53316640 1 --2- 172.21.15.84:0/3444312020 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7ffa3806afc0 0x7ffa3806d480 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.655+0000 7ffa53316640 1 --2- 172.21.15.84:0/3444312020 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa4c150880 0x7ffa4c0cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.655+0000 7ffa53316640 1 --2- 172.21.15.84:0/3444312020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa340047c0 0x7ffa4c15ccd0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.655+0000 7ffa53316640 1 -- 172.21.15.84:0/3444312020 >> 172.21.15.84:0/3444312020 conn(0x7ffa4c0cb410 msgr2=0x7ffa4c0ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:11.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.655+0000 7ffa53316640 1 -- 172.21.15.84:0/3444312020 shutdown_connections 2023-10-16T10:09:11.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.655+0000 7ffa53316640 1 -- 172.21.15.84:0/3444312020 wait complete. 2023-10-16T10:09:11.675 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:09:11.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.892+0000 7f59cfb89640 1 Processor -- start 2023-10-16T10:09:11.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.892+0000 7f59cfb89640 1 -- start start 2023-10-16T10:09:11.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.894+0000 7f59cfb89640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f59c80cd0c0 0x7f59c81615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:11.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.894+0000 7f59cfb89640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f59c815db00 0x7f59c80b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:11.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.894+0000 7f59cfb89640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c815e3e0 con 0x7f59c80b0410 2023-10-16T10:09:11.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.894+0000 7f59cfb89640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c80b0de0 con 0x7f59c815db00 2023-10-16T10:09:11.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.894+0000 7f59cfb89640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c80cd4a0 con 0x7f59c80cd0c0 2023-10-16T10:09:11.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.894+0000 7f59ce126640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f59c815db00 0x7f59c80b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:11.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.894+0000 7f59ce126640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f59c815db00 0x7f59c80b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51396/0 (socket says 172.21.15.84:51396) 2023-10-16T10:09:11.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.894+0000 7f59ce126640 1 -- 172.21.15.84:0/2546836954 learned_addr learned my addr 172.21.15.84:0/2546836954 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:11.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.895+0000 7f59cd925640 1 --2- 172.21.15.84:0/2546836954 >> v2:172.21.15.157:3300/0 conn(0x7f59c80cd0c0 0x7f59c81615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:11.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.895+0000 7f59ce126640 1 -- 172.21.15.84:0/2546836954 >> v2:172.21.15.157:3300/0 conn(0x7f59c80cd0c0 msgr2=0x7f59c81615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:11.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.895+0000 7f59ce126640 1 --2- 172.21.15.84:0/2546836954 >> v2:172.21.15.157:3300/0 conn(0x7f59c80cd0c0 0x7f59c81615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.895+0000 7f59ce126640 1 -- 172.21.15.84:0/2546836954 >> v1:172.21.15.157:6789/0 conn(0x7f59c80b0410 legacy=0x7f59c815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:11.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.895+0000 7f59ce126640 1 -- 172.21.15.84:0/2546836954 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c815e100 con 0x7f59c815db00 2023-10-16T10:09:11.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.896+0000 7f59ce126640 1 --2- 172.21.15.84:0/2546836954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59c815db00 0x7f59c80b08a0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f59c80cc4e0 tx=0x7f59c002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3c661467b2af13 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:11.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.896+0000 7f59cc923640 1 -- 172.21.15.84:0/2546836954 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c002c9f0 con 0x7f59c815db00 2023-10-16T10:09:11.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.896+0000 7f59cc923640 1 -- 172.21.15.84:0/2546836954 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c002cb50 con 0x7f59c815db00 2023-10-16T10:09:11.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.896+0000 7f59cc923640 1 -- 172.21.15.84:0/2546836954 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c00343f0 con 0x7f59c815db00 2023-10-16T10:09:11.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.897+0000 7f59cfb89640 1 -- 172.21.15.84:0/2546836954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59c815db00 msgr2=0x7f59c80b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:11.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.897+0000 7f59cfb89640 1 --2- 172.21.15.84:0/2546836954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59c815db00 0x7f59c80b08a0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f59c80cc4e0 tx=0x7f59c002be50 comp rx=0 tx=0).stop 2023-10-16T10:09:11.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.897+0000 7f59cfb89640 1 -- 172.21.15.84:0/2546836954 shutdown_connections 2023-10-16T10:09:11.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.897+0000 7f59cfb89640 1 --2- 172.21.15.84:0/2546836954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59c815db00 0x7f59c80b08a0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.897+0000 7f59cfb89640 1 --2- 172.21.15.84:0/2546836954 >> v2:172.21.15.157:3300/0 conn(0x7f59c80cd0c0 0x7f59c81615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.897+0000 7f59cfb89640 1 -- 172.21.15.84:0/2546836954 >> 172.21.15.84:0/2546836954 conn(0x7f59c80cb410 msgr2=0x7f59c80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:11.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.897+0000 7f59cfb89640 1 -- 172.21.15.84:0/2546836954 shutdown_connections 2023-10-16T10:09:11.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.897+0000 7f59cfb89640 1 -- 172.21.15.84:0/2546836954 wait complete. 2023-10-16T10:09:11.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.898+0000 7f59cfb89640 1 Processor -- start 2023-10-16T10:09:11.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.899+0000 7f59cfb89640 1 -- start start 2023-10-16T10:09:11.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.899+0000 7f59cfb89640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59c80b0410 0x7f59c8150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:11.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.899+0000 7f59cfb89640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59c80cd0c0 0x7f59c81e70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:11.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.899+0000 7f59cfb89640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59c815db00 0x7f59c81562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:11.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.899+0000 7f59cfb89640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c8151420 con 0x7f59c80cd0c0 2023-10-16T10:09:11.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.899+0000 7f59cfb89640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c80c3bc0 con 0x7f59c80b0410 2023-10-16T10:09:11.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.899+0000 7f59cfb89640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c80c3ea0 con 0x7f59c815db00 2023-10-16T10:09:11.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.900+0000 7f59cd124640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59c80cd0c0 0x7f59c81e70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:11.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.900+0000 7f59cd124640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59c80cd0c0 0x7f59c81e70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51408/0 (socket says 172.21.15.84:51408) 2023-10-16T10:09:11.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.900+0000 7f59cd124640 1 -- 172.21.15.84:0/1919267290 learned_addr learned my addr 172.21.15.84:0/1919267290 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:11.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.900+0000 7f59ce126640 1 --2- 172.21.15.84:0/1919267290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59c815db00 0x7f59c81562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:11.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.900+0000 7f59cd925640 1 --2- 172.21.15.84:0/1919267290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59c80b0410 0x7f59c8150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:11.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.900+0000 7f59cd124640 1 -- 172.21.15.84:0/1919267290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59c815db00 msgr2=0x7f59c81562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:11.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.900+0000 7f59cd124640 1 --2- 172.21.15.84:0/1919267290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59c815db00 0x7f59c81562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.900+0000 7f59cd124640 1 -- 172.21.15.84:0/1919267290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59c80b0410 msgr2=0x7f59c8150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:11.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.900+0000 7f59cd124640 1 --2- 172.21.15.84:0/1919267290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59c80b0410 0x7f59c8150ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:11.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.900+0000 7f59cd124640 1 -- 172.21.15.84:0/1919267290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c0009b30 con 0x7f59c80cd0c0 2023-10-16T10:09:11.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.901+0000 7f59cd124640 1 --2- 172.21.15.84:0/1919267290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59c80cd0c0 0x7f59c81e70b0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f59b800b3f0 tx=0x7f59b800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.901+0000 7f59be7fc640 1 -- 172.21.15.84:0/1919267290 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8011830 con 0x7f59c80cd0c0 2023-10-16T10:09:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.901+0000 7f59be7fc640 1 -- 172.21.15.84:0/1919267290 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b8011990 con 0x7f59c80cd0c0 2023-10-16T10:09:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.901+0000 7f59be7fc640 1 -- 172.21.15.84:0/1919267290 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8011d40 con 0x7f59c80cd0c0 2023-10-16T10:09:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.901+0000 7f59cfb89640 1 -- 172.21.15.84:0/1919267290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c80c4180 con 0x7f59c80cd0c0 2023-10-16T10:09:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.901+0000 7f59cfb89640 1 -- 172.21.15.84:0/1919267290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c80c4740 con 0x7f59c80cd0c0 2023-10-16T10:09:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.902+0000 7f59cfb89640 1 -- 172.21.15.84:0/1919267290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59c80ce260 con 0x7f59c80cd0c0 2023-10-16T10:09:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.903+0000 7f59be7fc640 1 -- 172.21.15.84:0/1919267290 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f59b801a400 con 0x7f59c80cd0c0 2023-10-16T10:09:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.904+0000 7f59be7fc640 1 --2- 172.21.15.84:0/1919267290 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f59ac06af90 0x7f59ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.904+0000 7f59be7fc640 1 -- 172.21.15.84:0/1919267290 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f59b8089600 con 0x7f59c80cd0c0 2023-10-16T10:09:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.904+0000 7f59cd925640 1 --2- 172.21.15.84:0/1919267290 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f59ac06af90 0x7f59ac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.905+0000 7f59cd925640 1 --2- 172.21.15.84:0/1919267290 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f59ac06af90 0x7f59ac06d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f59c4006fd0 tx=0x7f59c4008040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:11.908+0000 7f59be7fc640 1 -- 172.21.15.84:0/1919267290 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f59b8054570 con 0x7f59c80cd0c0 2023-10-16T10:09:12.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.099+0000 7f59cfb89640 1 -- 172.21.15.84:0/1919267290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59c806d600 con 0x7f59c80cd0c0 2023-10-16T10:09:12.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.102+0000 7f59be7fc640 1 -- 172.21.15.84:0/1919267290 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16612 (secure 0 0 0) 0x7f59b8057b90 con 0x7f59c80cd0c0 2023-10-16T10:09:12.103 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:09:12.103 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":150,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:09:10.651585+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:09:08.490425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-16T10:09:09.666051+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:09:12.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.105+0000 7f59cfb89640 1 -- 172.21.15.84:0/1919267290 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f59ac06af90 msgr2=0x7f59ac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:12.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.105+0000 7f59cfb89640 1 --2- 172.21.15.84:0/1919267290 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f59ac06af90 0x7f59ac06d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f59c4006fd0 tx=0x7f59c4008040 comp rx=0 tx=0).stop 2023-10-16T10:09:12.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.105+0000 7f59cfb89640 1 -- 172.21.15.84:0/1919267290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59c80cd0c0 msgr2=0x7f59c81e70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:12.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.105+0000 7f59cfb89640 1 --2- 172.21.15.84:0/1919267290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59c80cd0c0 0x7f59c81e70b0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f59b800b3f0 tx=0x7f59b800b8b0 comp rx=0 tx=0).stop 2023-10-16T10:09:12.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.105+0000 7f59cfb89640 1 -- 172.21.15.84:0/1919267290 shutdown_connections 2023-10-16T10:09:12.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.105+0000 7f59cfb89640 1 --2- 172.21.15.84:0/1919267290 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f59ac06af90 0x7f59ac06d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:12.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.105+0000 7f59cfb89640 1 --2- 172.21.15.84:0/1919267290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59c815db00 0x7f59c81562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:12.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.105+0000 7f59cfb89640 1 --2- 172.21.15.84:0/1919267290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59c80cd0c0 0x7f59c81e70b0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:12.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.105+0000 7f59cfb89640 1 --2- 172.21.15.84:0/1919267290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59c80b0410 0x7f59c8150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:12.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.105+0000 7f59cfb89640 1 -- 172.21.15.84:0/1919267290 >> 172.21.15.84:0/1919267290 conn(0x7f59c80cb410 msgr2=0x7f59c80ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:12.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.105+0000 7f59cfb89640 1 -- 172.21.15.84:0/1919267290 shutdown_connections 2023-10-16T10:09:12.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.105+0000 7f59cfb89640 1 -- 172.21.15.84:0/1919267290 wait complete. 2023-10-16T10:09:12.124 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T10:09:12.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.343+0000 7f08ccfc0640 1 Processor -- start 2023-10-16T10:09:12.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.343+0000 7f08ccfc0640 1 -- start start 2023-10-16T10:09:12.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.344+0000 7f08ccfc0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f08c80b3420 0x7f08c80b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:12.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.344+0000 7f08ccfc0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08c80cfaf0 0x7f08c80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:12.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.344+0000 7f08ccfc0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c80d3980 con 0x7f08c80cf710 2023-10-16T10:09:12.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.344+0000 7f08ccfc0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c80d3c60 con 0x7f08c80b3420 2023-10-16T10:09:12.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.344+0000 7f08ccfc0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c80ce630 con 0x7f08c80cfaf0 2023-10-16T10:09:12.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.345+0000 7f08c5d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f08c80b3420 0x7f08c80b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:12.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.345+0000 7f08c5d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f08c80b3420 0x7f08c80b5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51420/0 (socket says 172.21.15.84:51420) 2023-10-16T10:09:12.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.345+0000 7f08c5d9b640 1 -- 172.21.15.84:0/870230087 learned_addr learned my addr 172.21.15.84:0/870230087 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:12.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.345+0000 7f08c6d9d640 1 --2- 172.21.15.84:0/870230087 >> v2:172.21.15.157:3300/0 conn(0x7f08c80cfaf0 0x7f08c80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:12.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.345+0000 7f08c5d9b640 1 -- 172.21.15.84:0/870230087 >> v2:172.21.15.157:3300/0 conn(0x7f08c80cfaf0 msgr2=0x7f08c80b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:12.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.345+0000 7f08c5d9b640 1 --2- 172.21.15.84:0/870230087 >> v2:172.21.15.157:3300/0 conn(0x7f08c80cfaf0 0x7f08c80b6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:12.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.345+0000 7f08c5d9b640 1 -- 172.21.15.84:0/870230087 >> v1:172.21.15.84:6789/0 conn(0x7f08c80cf710 legacy=0x7f08c80b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:12.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.345+0000 7f08c559a640 1 -- 172.21.15.84:0/870230087 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804889456 0 0) 0x7f08c80d3980 con 0x7f08c80cf710 2023-10-16T10:09:12.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.345+0000 7f08c5d9b640 1 -- 172.21.15.84:0/870230087 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c80ce910 con 0x7f08c80b3420 2023-10-16T10:09:12.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.346+0000 7f08c5d9b640 1 --2- 172.21.15.84:0/870230087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c80b3420 0x7f08c80b5cc0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f08c80d07a0 tx=0x7f08b002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b44545a38ecca69 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:12.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.347+0000 7f08c559a640 1 -- 172.21.15.84:0/870230087 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b0031870 con 0x7f08c80b3420 2023-10-16T10:09:12.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.347+0000 7f08c559a640 1 -- 172.21.15.84:0/870230087 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b00319d0 con 0x7f08c80b3420 2023-10-16T10:09:12.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.347+0000 7f08c559a640 1 -- 172.21.15.84:0/870230087 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b0031d60 con 0x7f08c80b3420 2023-10-16T10:09:12.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.347+0000 7f08ccfc0640 1 -- 172.21.15.84:0/870230087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c80b3420 msgr2=0x7f08c80b5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:12.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.347+0000 7f08ccfc0640 1 --2- 172.21.15.84:0/870230087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c80b3420 0x7f08c80b5cc0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f08c80d07a0 tx=0x7f08b002b910 comp rx=0 tx=0).stop 2023-10-16T10:09:12.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.348+0000 7f08ccfc0640 1 -- 172.21.15.84:0/870230087 shutdown_connections 2023-10-16T10:09:12.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.348+0000 7f08ccfc0640 1 --2- 172.21.15.84:0/870230087 >> v2:172.21.15.157:3300/0 conn(0x7f08c80cfaf0 0x7f08c80b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:12.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.348+0000 7f08ccfc0640 1 --2- 172.21.15.84:0/870230087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c80b3420 0x7f08c80b5cc0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:12.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.348+0000 7f08ccfc0640 1 -- 172.21.15.84:0/870230087 >> 172.21.15.84:0/870230087 conn(0x7f08c80d2cc0 msgr2=0x7f08c80d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:12.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.348+0000 7f08ccfc0640 1 -- 172.21.15.84:0/870230087 shutdown_connections 2023-10-16T10:09:12.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.348+0000 7f08ccfc0640 1 -- 172.21.15.84:0/870230087 wait complete. 2023-10-16T10:09:12.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.349+0000 7f08ccfc0640 1 Processor -- start 2023-10-16T10:09:12.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.350+0000 7f08ccfc0640 1 -- start start 2023-10-16T10:09:12.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.350+0000 7f08ccfc0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c80b3420 0x7f08c815d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:12.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.351+0000 7f08ccfc0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08c80cf710 0x7f08c80cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:12.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.351+0000 7f08ccfc0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08c80cfaf0 0x7f08c80b7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:12.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.351+0000 7f08ccfc0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c80c7790 con 0x7f08c80b3420 2023-10-16T10:09:12.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.351+0000 7f08ccfc0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c80c9b90 con 0x7f08c80cf710 2023-10-16T10:09:12.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.351+0000 7f08ccfc0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c80c9010 con 0x7f08c80cfaf0 2023-10-16T10:09:12.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.351+0000 7f08c659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c80b3420 0x7f08c815d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:12.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.351+0000 7f08c659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c80b3420 0x7f08c815d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51434/0 (socket says 172.21.15.84:51434) 2023-10-16T10:09:12.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.351+0000 7f08c659c640 1 -- 172.21.15.84:0/3780078354 learned_addr learned my addr 172.21.15.84:0/3780078354 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:12.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.351+0000 7f08c5d9b640 1 --2- 172.21.15.84:0/3780078354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08c80cf710 0x7f08c80cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:12.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.351+0000 7f08c659c640 1 -- 172.21.15.84:0/3780078354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08c80cfaf0 msgr2=0x7f08c80b7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:12.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.351+0000 7f08c659c640 1 --2- 172.21.15.84:0/3780078354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08c80cfaf0 0x7f08c80b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:12.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.351+0000 7f08c659c640 1 -- 172.21.15.84:0/3780078354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08c80cf710 msgr2=0x7f08c80cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:12.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.351+0000 7f08c659c640 1 --2- 172.21.15.84:0/3780078354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08c80cf710 0x7f08c80cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:12.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.351+0000 7f08c659c640 1 -- 172.21.15.84:0/3780078354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b00094d0 con 0x7f08c80b3420 2023-10-16T10:09:12.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.352+0000 7f08c659c640 1 --2- 172.21.15.84:0/3780078354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c80b3420 0x7f08c815d8e0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f08b800c780 tx=0x7f08b800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:12.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.352+0000 7f08beffd640 1 -- 172.21.15.84:0/3780078354 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8003da0 con 0x7f08c80b3420 2023-10-16T10:09:12.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.352+0000 7f08beffd640 1 -- 172.21.15.84:0/3780078354 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b800ce90 con 0x7f08c80b3420 2023-10-16T10:09:12.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.352+0000 7f08beffd640 1 -- 172.21.15.84:0/3780078354 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b80052b0 con 0x7f08c80b3420 2023-10-16T10:09:12.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.352+0000 7f08ccfc0640 1 -- 172.21.15.84:0/3780078354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c80c92f0 con 0x7f08c80b3420 2023-10-16T10:09:12.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.352+0000 7f08ccfc0640 1 -- 172.21.15.84:0/3780078354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c80c98b0 con 0x7f08c80b3420 2023-10-16T10:09:12.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.353+0000 7f08ccfc0640 1 -- 172.21.15.84:0/3780078354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c80c78d0 con 0x7f08c80b3420 2023-10-16T10:09:12.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.354+0000 7f08beffd640 1 -- 172.21.15.84:0/3780078354 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f08b8005410 con 0x7f08c80b3420 2023-10-16T10:09:12.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.356+0000 7f08beffd640 1 --2- 172.21.15.84:0/3780078354 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f08a806af90 0x7f08a806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:12.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.356+0000 7f08beffd640 1 -- 172.21.15.84:0/3780078354 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f08b8089030 con 0x7f08c80b3420 2023-10-16T10:09:12.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.356+0000 7f08c5d9b640 1 --2- 172.21.15.84:0/3780078354 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f08a806af90 0x7f08a806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:12.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.357+0000 7f08c5d9b640 1 --2- 172.21.15.84:0/3780078354 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f08a806af90 0x7f08a806d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f08b0036000 tx=0x7f08b003c040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:12.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.360+0000 7f08beffd640 1 -- 172.21.15.84:0/3780078354 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f08b8053fb0 con 0x7f08c80b3420 2023-10-16T10:09:12.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:12.590+0000 7f08ccfc0640 1 -- 172.21.15.84:0/3780078354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f08c804e160 con 0x7f08c80b3420 2023-10-16T10:09:13.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.229+0000 7f08beffd640 1 -- 172.21.15.84:0/3780078354 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v79) v1 ==== 118+0+0 (secure 0 0 0) 0x7f08b80170b0 con 0x7f08c80b3420 2023-10-16T10:09:13.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.232+0000 7f08ccfc0640 1 -- 172.21.15.84:0/3780078354 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f08a806af90 msgr2=0x7f08a806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:13.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.232+0000 7f08ccfc0640 1 --2- 172.21.15.84:0/3780078354 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f08a806af90 0x7f08a806d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f08b0036000 tx=0x7f08b003c040 comp rx=0 tx=0).stop 2023-10-16T10:09:13.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.232+0000 7f08ccfc0640 1 -- 172.21.15.84:0/3780078354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c80b3420 msgr2=0x7f08c815d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:13.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.232+0000 7f08ccfc0640 1 --2- 172.21.15.84:0/3780078354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c80b3420 0x7f08c815d8e0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f08b800c780 tx=0x7f08b800cc40 comp rx=0 tx=0).stop 2023-10-16T10:09:13.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.233+0000 7f08ccfc0640 1 -- 172.21.15.84:0/3780078354 shutdown_connections 2023-10-16T10:09:13.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.233+0000 7f08ccfc0640 1 --2- 172.21.15.84:0/3780078354 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f08a806af90 0x7f08a806d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:13.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.233+0000 7f08ccfc0640 1 --2- 172.21.15.84:0/3780078354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08c80cfaf0 0x7f08c80b7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:13.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.233+0000 7f08ccfc0640 1 --2- 172.21.15.84:0/3780078354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08c80cf710 0x7f08c80cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:13.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.233+0000 7f08ccfc0640 1 --2- 172.21.15.84:0/3780078354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08c80b3420 0x7f08c815d8e0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:13.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.233+0000 7f08ccfc0640 1 -- 172.21.15.84:0/3780078354 >> 172.21.15.84:0/3780078354 conn(0x7f08c80d2cc0 msgr2=0x7f08c81601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:13.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.233+0000 7f08ccfc0640 1 -- 172.21.15.84:0/3780078354 shutdown_connections 2023-10-16T10:09:13.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.233+0000 7f08ccfc0640 1 -- 172.21.15.84:0/3780078354 wait complete. 2023-10-16T10:09:13.257 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T10:09:13.257 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T10:09:13.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.479+0000 7f867e0bb640 1 Processor -- start 2023-10-16T10:09:13.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.479+0000 7f867e0bb640 1 -- start start 2023-10-16T10:09:13.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.481+0000 7f867e0bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f867815ad70 0x7f8678069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:13.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.481+0000 7f867e0bb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f86780caac0 0x7f867806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:13.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.481+0000 7f867e0bb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86780cb030 con 0x7f86780c9ca0 2023-10-16T10:09:13.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.481+0000 7f867e0bb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86780c91c0 con 0x7f86780caac0 2023-10-16T10:09:13.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.481+0000 7f867e0bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86780c94a0 con 0x7f867815ad70 2023-10-16T10:09:13.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.481+0000 7f866ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f86780c9ca0 0x7f86780ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56008/0 (socket says 172.21.15.84:56008) 2023-10-16T10:09:13.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.481+0000 7f866ffff640 1 -- 172.21.15.84:0/179788615 learned_addr learned my addr 172.21.15.84:0/179788615 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:09:13.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.482+0000 7f86777fe640 1 --2- 172.21.15.84:0/179788615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f867815ad70 0x7f8678069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:13.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.482+0000 7f8677fff640 1 --2- 172.21.15.84:0/179788615 >> v2:172.21.15.157:3300/0 conn(0x7f86780caac0 0x7f867806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:13.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.482+0000 7f8676ffd640 1 -- 172.21.15.84:0/179788615 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897443208 0 0) 0x7f86780cb030 con 0x7f86780c9ca0 2023-10-16T10:09:13.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.482+0000 7f8676ffd640 1 -- 172.21.15.84:0/179788615 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8664003520 con 0x7f86780c9ca0 2023-10-16T10:09:13.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.482+0000 7f8676ffd640 1 -- 172.21.15.84:0/179788615 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 669146254 0 0) 0x7f8664003520 con 0x7f86780c9ca0 2023-10-16T10:09:13.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.482+0000 7f8676ffd640 1 -- 172.21.15.84:0/179788615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f867815ad70 msgr2=0x7f8678069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:13.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.482+0000 7f8676ffd640 1 --2- 172.21.15.84:0/179788615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f867815ad70 0x7f8678069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:13.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.482+0000 7f8676ffd640 1 -- 172.21.15.84:0/179788615 >> v2:172.21.15.157:3300/0 conn(0x7f86780caac0 msgr2=0x7f867806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:13.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.482+0000 7f8676ffd640 1 --2- 172.21.15.84:0/179788615 >> v2:172.21.15.157:3300/0 conn(0x7f86780caac0 0x7f867806a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:13.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.482+0000 7f8676ffd640 1 -- 172.21.15.84:0/179788615 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86780c9780 con 0x7f86780c9ca0 2023-10-16T10:09:13.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.482+0000 7f8676ffd640 1 -- 172.21.15.84:0/179788615 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f8660001670 con 0x7f86780c9ca0 2023-10-16T10:09:13.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.482+0000 7f8676ffd640 1 -- 172.21.15.84:0/179788615 >> v1:172.21.15.84:6789/0 conn(0x7f86780c9ca0 legacy=0x7f86780ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:13.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.482+0000 7f8676ffd640 1 --2- 172.21.15.84:0/179788615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f867815ad70 0x7f8664003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:13.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.482+0000 7f8676ffd640 1 --2- 172.21.15.84:0/179788615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8664003ef0 0x7f86640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:13.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.482+0000 7f8676ffd640 1 --2- 172.21.15.84:0/179788615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8664004d00 0x7f8664005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:13.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.482+0000 7f8676ffd640 1 -- 172.21.15.84:0/179788615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86780c91c0 con 0x7f8664004d00 2023-10-16T10:09:13.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.482+0000 7f8676ffd640 1 -- 172.21.15.84:0/179788615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86780c94a0 con 0x7f8664003ef0 2023-10-16T10:09:13.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.482+0000 7f8676ffd640 1 -- 172.21.15.84:0/179788615 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866400b980 con 0x7f867815ad70 2023-10-16T10:09:13.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.483+0000 7f8677fff640 1 --2- 172.21.15.84:0/179788615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8664004d00 0x7f8664005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:13.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.483+0000 7f86777fe640 1 --2- 172.21.15.84:0/179788615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f867815ad70 0x7f8664003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:13.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.483+0000 7f8677fff640 1 -- 172.21.15.84:0/179788615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f867815ad70 msgr2=0x7f8664003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:13.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.483+0000 7f8677fff640 1 --2- 172.21.15.84:0/179788615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f867815ad70 0x7f8664003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:13.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.483+0000 7f8677fff640 1 -- 172.21.15.84:0/179788615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8664003ef0 msgr2=0x7f86640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:13.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.483+0000 7f866ffff640 1 --2- 172.21.15.84:0/179788615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8664003ef0 0x7f86640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:13.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.483+0000 7f8677fff640 1 --2- 172.21.15.84:0/179788615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8664003ef0 0x7f86640047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:13.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.483+0000 7f8677fff640 1 -- 172.21.15.84:0/179788615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8664003520 con 0x7f8664004d00 2023-10-16T10:09:13.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.484+0000 7f8677fff640 1 --2- 172.21.15.84:0/179788615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8664004d00 0x7f8664005180 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f8670009fc0 tx=0x7f86640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79db3c9e6969fdb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:13.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.484+0000 7f8676ffd640 1 -- 172.21.15.84:0/179788615 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867002cc40 con 0x7f8664004d00 2023-10-16T10:09:13.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.484+0000 7f8676ffd640 1 -- 172.21.15.84:0/179788615 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867000f040 con 0x7f8664004d00 2023-10-16T10:09:13.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.484+0000 7f8676ffd640 1 -- 172.21.15.84:0/179788615 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8670034440 con 0x7f8664004d00 2023-10-16T10:09:13.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.485+0000 7f867e0bb640 1 -- 172.21.15.84:0/179788615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8664004d00 msgr2=0x7f8664005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:13.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.485+0000 7f867e0bb640 1 --2- 172.21.15.84:0/179788615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8664004d00 0x7f8664005180 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f8670009fc0 tx=0x7f86640057f0 comp rx=0 tx=0).stop 2023-10-16T10:09:13.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.485+0000 7f8677fff640 1 -- 172.21.15.84:0/179788615 reap_dead start 2023-10-16T10:09:13.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.485+0000 7f867e0bb640 1 -- 172.21.15.84:0/179788615 shutdown_connections 2023-10-16T10:09:13.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.485+0000 7f867e0bb640 1 -- 172.21.15.84:0/179788615 >> 172.21.15.84:0/179788615 conn(0x7f86780ce9b0 msgr2=0x7f86780cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:13.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.485+0000 7f867e0bb640 1 -- 172.21.15.84:0/179788615 shutdown_connections 2023-10-16T10:09:13.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.485+0000 7f867e0bb640 1 -- 172.21.15.84:0/179788615 wait complete. 2023-10-16T10:09:13.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.487+0000 7f867e0bb640 1 Processor -- start 2023-10-16T10:09:13.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.487+0000 7f867e0bb640 1 -- start start 2023-10-16T10:09:13.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.488+0000 7f867e0bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8664004d00 0x7f86781502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:13.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.488+0000 7f867e0bb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86780ccf90 0x7f867814fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:13.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.488+0000 7f867e0bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86780ac890 0x7f86780acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:13.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.488+0000 7f867e0bb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86780ad250 con 0x7f86780ccf90 2023-10-16T10:09:13.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.488+0000 7f867e0bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86780ce5a0 con 0x7f8664004d00 2023-10-16T10:09:13.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.488+0000 7f867e0bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86780cd550 con 0x7f86780ac890 2023-10-16T10:09:13.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.489+0000 7f866ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86780ccf90 0x7f867814fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:13.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.489+0000 7f866ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86780ccf90 0x7f867814fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51444/0 (socket says 172.21.15.84:51444) 2023-10-16T10:09:13.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.489+0000 7f866ffff640 1 -- 172.21.15.84:0/1162228032 learned_addr learned my addr 172.21.15.84:0/1162228032 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:13.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.489+0000 7f8677fff640 1 --2- 172.21.15.84:0/1162228032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86780ac890 0x7f86780acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:13.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.489+0000 7f866ffff640 1 -- 172.21.15.84:0/1162228032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86780ac890 msgr2=0x7f86780acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:13.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.489+0000 7f866ffff640 1 --2- 172.21.15.84:0/1162228032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86780ac890 0x7f86780acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:13.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.489+0000 7f866ffff640 1 -- 172.21.15.84:0/1162228032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8664004d00 msgr2=0x7f86781502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:13.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.489+0000 7f86777fe640 1 --2- 172.21.15.84:0/1162228032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8664004d00 0x7f86781502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:13.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.489+0000 7f866ffff640 1 --2- 172.21.15.84:0/1162228032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8664004d00 0x7f86781502a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:13.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.489+0000 7f866ffff640 1 -- 172.21.15.84:0/1162228032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86780cd830 con 0x7f86780ccf90 2023-10-16T10:09:13.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.489+0000 7f866ffff640 1 --2- 172.21.15.84:0/1162228032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86780ccf90 0x7f867814fa90 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f866000d8c0 tx=0x7f866000dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:13.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.490+0000 7f8674ff9640 1 -- 172.21.15.84:0/1162228032 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86600074e0 con 0x7f86780ccf90 2023-10-16T10:09:13.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.490+0000 7f8674ff9640 1 -- 172.21.15.84:0/1162228032 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8660007640 con 0x7f86780ccf90 2023-10-16T10:09:13.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.490+0000 7f8674ff9640 1 -- 172.21.15.84:0/1162228032 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86600049e0 con 0x7f86780ccf90 2023-10-16T10:09:13.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.490+0000 7f867e0bb640 1 -- 172.21.15.84:0/1162228032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86780cdb10 con 0x7f86780ccf90 2023-10-16T10:09:13.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.490+0000 7f867e0bb640 1 -- 172.21.15.84:0/1162228032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867814ffd0 con 0x7f86780ccf90 2023-10-16T10:09:13.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.491+0000 7f867e0bb640 1 -- 172.21.15.84:0/1162228032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867804e160 con 0x7f86780ccf90 2023-10-16T10:09:13.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.492+0000 7f8674ff9640 1 -- 172.21.15.84:0/1162228032 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8660004b40 con 0x7f86780ccf90 2023-10-16T10:09:13.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.493+0000 7f8674ff9640 1 --2- 172.21.15.84:0/1162228032 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f865806af90 0x7f865806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:13.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.493+0000 7f8674ff9640 1 -- 172.21.15.84:0/1162228032 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f86600893e0 con 0x7f86780ccf90 2023-10-16T10:09:13.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.493+0000 7f86777fe640 1 --2- 172.21.15.84:0/1162228032 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f865806af90 0x7f865806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:13.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.494+0000 7f86777fe640 1 --2- 172.21.15.84:0/1162228032 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f865806af90 0x7f865806d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8668009910 tx=0x7f8668008040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:13.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.498+0000 7f8674ff9640 1 -- 172.21.15.84:0/1162228032 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f86600541b0 con 0x7f86780ccf90 2023-10-16T10:09:13.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:13.729+0000 7f867e0bb640 1 -- 172.21.15.84:0/1162228032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f86780afdc0 con 0x7f86780ccf90 2023-10-16T10:09:14.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.239+0000 7f8674ff9640 1 -- 172.21.15.84:0/1162228032 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v80) v1 ==== 122+0+0 (secure 0 0 0) 0x7f86600577d0 con 0x7f86780ccf90 2023-10-16T10:09:14.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.243+0000 7f867e0bb640 1 -- 172.21.15.84:0/1162228032 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f865806af90 msgr2=0x7f865806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:14.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.243+0000 7f867e0bb640 1 --2- 172.21.15.84:0/1162228032 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f865806af90 0x7f865806d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8668009910 tx=0x7f8668008040 comp rx=0 tx=0).stop 2023-10-16T10:09:14.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.243+0000 7f867e0bb640 1 -- 172.21.15.84:0/1162228032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86780ccf90 msgr2=0x7f867814fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:14.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.243+0000 7f867e0bb640 1 --2- 172.21.15.84:0/1162228032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86780ccf90 0x7f867814fa90 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f866000d8c0 tx=0x7f866000dd80 comp rx=0 tx=0).stop 2023-10-16T10:09:14.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.243+0000 7f867e0bb640 1 -- 172.21.15.84:0/1162228032 shutdown_connections 2023-10-16T10:09:14.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.243+0000 7f867e0bb640 1 --2- 172.21.15.84:0/1162228032 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f865806af90 0x7f865806d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.243+0000 7f867e0bb640 1 --2- 172.21.15.84:0/1162228032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86780ac890 0x7f86780acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.243+0000 7f867e0bb640 1 --2- 172.21.15.84:0/1162228032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86780ccf90 0x7f867814fa90 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.243+0000 7f867e0bb640 1 --2- 172.21.15.84:0/1162228032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8664004d00 0x7f86781502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.243+0000 7f867e0bb640 1 -- 172.21.15.84:0/1162228032 >> 172.21.15.84:0/1162228032 conn(0x7f86780ce9b0 msgr2=0x7f86780b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:14.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.243+0000 7f867e0bb640 1 -- 172.21.15.84:0/1162228032 shutdown_connections 2023-10-16T10:09:14.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.243+0000 7f867e0bb640 1 -- 172.21.15.84:0/1162228032 wait complete. 2023-10-16T10:09:14.263 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T10:09:14.264 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:09:14.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.476+0000 7f1803e26640 1 Processor -- start 2023-10-16T10:09:14.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.476+0000 7f1803e26640 1 -- start start 2023-10-16T10:09:14.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.477+0000 7f1803e26640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f17fc15aaa0 0x7f17fc15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:14.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.477+0000 7f1803e26640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f17fc15ae80 0x7f17fc0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:14.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.478+0000 7f1803e26640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17fc15ed60 con 0x7f17fc0ccaa0 2023-10-16T10:09:14.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.478+0000 7f1803e26640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17fc06b810 con 0x7f17fc15ae80 2023-10-16T10:09:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.478+0000 7f1803e26640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17fc0707e0 con 0x7f17fc15aaa0 2023-10-16T10:09:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.478+0000 7f18013c1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f17fc15ae80 0x7f17fc0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.478+0000 7f18013c1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f17fc15ae80 0x7f17fc0cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51460/0 (socket says 172.21.15.84:51460) 2023-10-16T10:09:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.478+0000 7f18013c1640 1 -- 172.21.15.84:0/2670286894 learned_addr learned my addr 172.21.15.84:0/2670286894 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.478+0000 7f1801bc2640 1 --2- 172.21.15.84:0/2670286894 >> v2:172.21.15.157:3300/0 conn(0x7f17fc15aaa0 0x7f17fc15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:14.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.478+0000 7f18013c1640 1 -- 172.21.15.84:0/2670286894 >> v2:172.21.15.157:3300/0 conn(0x7f17fc15aaa0 msgr2=0x7f17fc15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:14.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.478+0000 7f18013c1640 1 --2- 172.21.15.84:0/2670286894 >> v2:172.21.15.157:3300/0 conn(0x7f17fc15aaa0 0x7f17fc15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.478+0000 7f18013c1640 1 -- 172.21.15.84:0/2670286894 >> v1:172.21.15.157:6789/0 conn(0x7f17fc0ccaa0 legacy=0x7f17fc070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:14.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.478+0000 7f18013c1640 1 -- 172.21.15.84:0/2670286894 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc15ef90 con 0x7f17fc15ae80 2023-10-16T10:09:14.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.479+0000 7f18013c1640 1 --2- 172.21.15.84:0/2670286894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17fc15ae80 0x7f17fc0cc560 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0cd990 tx=0x7f17ec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5b555a1916168e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:14.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.480+0000 7f1800bc0640 1 -- 172.21.15.84:0/2670286894 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec031870 con 0x7f17fc15ae80 2023-10-16T10:09:14.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.480+0000 7f1800bc0640 1 -- 172.21.15.84:0/2670286894 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ec0319d0 con 0x7f17fc15ae80 2023-10-16T10:09:14.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.480+0000 7f1800bc0640 1 -- 172.21.15.84:0/2670286894 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec031d60 con 0x7f17fc15ae80 2023-10-16T10:09:14.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.480+0000 7f1803e26640 1 -- 172.21.15.84:0/2670286894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17fc15ae80 msgr2=0x7f17fc0cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:14.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.480+0000 7f1803e26640 1 --2- 172.21.15.84:0/2670286894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17fc15ae80 0x7f17fc0cc560 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0cd990 tx=0x7f17ec02b910 comp rx=0 tx=0).stop 2023-10-16T10:09:14.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.480+0000 7f1803e26640 1 -- 172.21.15.84:0/2670286894 shutdown_connections 2023-10-16T10:09:14.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.480+0000 7f1803e26640 1 --2- 172.21.15.84:0/2670286894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17fc15ae80 0x7f17fc0cc560 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.480+0000 7f1803e26640 1 --2- 172.21.15.84:0/2670286894 >> v2:172.21.15.157:3300/0 conn(0x7f17fc15aaa0 0x7f17fc15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.480+0000 7f1803e26640 1 -- 172.21.15.84:0/2670286894 >> 172.21.15.84:0/2670286894 conn(0x7f17fc0cb410 msgr2=0x7f17fc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:14.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.481+0000 7f1803e26640 1 -- 172.21.15.84:0/2670286894 shutdown_connections 2023-10-16T10:09:14.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.481+0000 7f1803e26640 1 -- 172.21.15.84:0/2670286894 wait complete. 2023-10-16T10:09:14.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.482+0000 7f1803e26640 1 Processor -- start 2023-10-16T10:09:14.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.482+0000 7f1803e26640 1 -- start start 2023-10-16T10:09:14.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.483+0000 7f1803e26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17fc0ccaa0 0x7f17fc1f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:14.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.483+0000 7f1803e26640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17fc15aaa0 0x7f17fc0c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:14.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.483+0000 7f1803e26640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17fc15ae80 0x7f17fc0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:14.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.483+0000 7f1803e26640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc0c6630 con 0x7f17fc15ae80 2023-10-16T10:09:14.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.483+0000 7f1803e26640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc0c6910 con 0x7f17fc0ccaa0 2023-10-16T10:09:14.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.483+0000 7f1803e26640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc0c6bf0 con 0x7f17fc15aaa0 2023-10-16T10:09:14.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.484+0000 7f18023c3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17fc15ae80 0x7f17fc0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:14.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.484+0000 7f18023c3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17fc15ae80 0x7f17fc0ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51466/0 (socket says 172.21.15.84:51466) 2023-10-16T10:09:14.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.484+0000 7f18023c3640 1 -- 172.21.15.84:0/1202515059 learned_addr learned my addr 172.21.15.84:0/1202515059 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:14.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.484+0000 7f1801bc2640 1 --2- 172.21.15.84:0/1202515059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17fc0ccaa0 0x7f17fc1f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:14.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.484+0000 7f18013c1640 1 --2- 172.21.15.84:0/1202515059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17fc15aaa0 0x7f17fc0c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:14.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.484+0000 7f18023c3640 1 -- 172.21.15.84:0/1202515059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17fc15aaa0 msgr2=0x7f17fc0c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:14.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.484+0000 7f18023c3640 1 --2- 172.21.15.84:0/1202515059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17fc15aaa0 0x7f17fc0c4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.484+0000 7f18023c3640 1 -- 172.21.15.84:0/1202515059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17fc0ccaa0 msgr2=0x7f17fc1f2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:14.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.484+0000 7f18023c3640 1 --2- 172.21.15.84:0/1202515059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17fc0ccaa0 0x7f17fc1f2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.484+0000 7f18023c3640 1 -- 172.21.15.84:0/1202515059 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ec0094d0 con 0x7f17fc15ae80 2023-10-16T10:09:14.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.485+0000 7f18023c3640 1 --2- 172.21.15.84:0/1202515059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17fc15ae80 0x7f17fc0ac890 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f17f400c780 tx=0x7f17f400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:14.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.485+0000 7f17f27fc640 1 -- 172.21.15.84:0/1202515059 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f4004b60 con 0x7f17fc15ae80 2023-10-16T10:09:14.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.485+0000 7f17f27fc640 1 -- 172.21.15.84:0/1202515059 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f400ce90 con 0x7f17fc15ae80 2023-10-16T10:09:14.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.485+0000 7f1803e26640 1 -- 172.21.15.84:0/1202515059 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17fc0c6ed0 con 0x7f17fc15ae80 2023-10-16T10:09:14.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.485+0000 7f17f27fc640 1 -- 172.21.15.84:0/1202515059 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f400a850 con 0x7f17fc15ae80 2023-10-16T10:09:14.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.485+0000 7f1803e26640 1 -- 172.21.15.84:0/1202515059 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17fc156250 con 0x7f17fc15ae80 2023-10-16T10:09:14.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.486+0000 7f1803e26640 1 -- 172.21.15.84:0/1202515059 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17fc0af820 con 0x7f17fc15ae80 2023-10-16T10:09:14.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.487+0000 7f17f27fc640 1 -- 172.21.15.84:0/1202515059 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f17f400a9b0 con 0x7f17fc15ae80 2023-10-16T10:09:14.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.488+0000 7f17f27fc640 1 --2- 172.21.15.84:0/1202515059 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f17e006af90 0x7f17e006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:14.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.488+0000 7f17f27fc640 1 -- 172.21.15.84:0/1202515059 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f17f4089970 con 0x7f17fc15ae80 2023-10-16T10:09:14.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.488+0000 7f1801bc2640 1 --2- 172.21.15.84:0/1202515059 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f17e006af90 0x7f17e006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:14.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.489+0000 7f1801bc2640 1 --2- 172.21.15.84:0/1202515059 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f17e006af90 0x7f17e006d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f17f8006fd0 tx=0x7f17f8008040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:14.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.493+0000 7f17f27fc640 1 -- 172.21.15.84:0/1202515059 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f17fc0af820 con 0x7f17fc15ae80 2023-10-16T10:09:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.725+0000 7f1803e26640 1 -- 172.21.15.84:0/1202515059 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17fc0afc00 con 0x7f17fc15ae80 2023-10-16T10:09:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.726+0000 7f17f27fc640 1 -- 172.21.15.84:0/1202515059 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f17fc0afc00 con 0x7f17fc15ae80 2023-10-16T10:09:14.727 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:09:14.727 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6517,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1733540616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1733540616},{"type":"v1","addr":"172.21.15.157:6833","nonce":1733540616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3815048417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3815048417},{"type":"v1","addr":"172.21.15.157:6835","nonce":3815048417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9135,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1038548935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1038548935},{"type":"v1","addr":"172.21.15.84:6836","nonce":1038548935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:09:10.652604+0000","modified":"2023-10-16T10:09:13.730838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9138},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9138":{"gid":9138,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2797149326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2797149326},{"type":"v1","addr":"172.21.15.84:6837","nonce":2797149326}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-16T10:09:14.727 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 80 2023-10-16T10:09:14.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.729+0000 7f1803e26640 1 -- 172.21.15.84:0/1202515059 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f17e006af90 msgr2=0x7f17e006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:14.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.729+0000 7f1803e26640 1 --2- 172.21.15.84:0/1202515059 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f17e006af90 0x7f17e006d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f17f8006fd0 tx=0x7f17f8008040 comp rx=0 tx=0).stop 2023-10-16T10:09:14.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.729+0000 7f1803e26640 1 -- 172.21.15.84:0/1202515059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17fc15ae80 msgr2=0x7f17fc0ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:14.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.729+0000 7f1803e26640 1 --2- 172.21.15.84:0/1202515059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17fc15ae80 0x7f17fc0ac890 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f17f400c780 tx=0x7f17f400cc40 comp rx=0 tx=0).stop 2023-10-16T10:09:14.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.729+0000 7f1803e26640 1 -- 172.21.15.84:0/1202515059 shutdown_connections 2023-10-16T10:09:14.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.729+0000 7f1803e26640 1 --2- 172.21.15.84:0/1202515059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17fc15ae80 0x7f17fc0ac890 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.729+0000 7f1803e26640 1 --2- 172.21.15.84:0/1202515059 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f17e006af90 0x7f17e006d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.729+0000 7f1803e26640 1 --2- 172.21.15.84:0/1202515059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17fc15aaa0 0x7f17fc0c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.729+0000 7f1803e26640 1 --2- 172.21.15.84:0/1202515059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17fc0ccaa0 0x7f17fc1f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.729+0000 7f1803e26640 1 -- 172.21.15.84:0/1202515059 >> 172.21.15.84:0/1202515059 conn(0x7f17fc0cb410 msgr2=0x7f17fc0b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:14.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.729+0000 7f1803e26640 1 -- 172.21.15.84:0/1202515059 shutdown_connections 2023-10-16T10:09:14.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.729+0000 7f1803e26640 1 -- 172.21.15.84:0/1202515059 wait complete. 2023-10-16T10:09:14.747 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:09:14.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.961+0000 7f510099e640 1 Processor -- start 2023-10-16T10:09:14.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.961+0000 7f510099e640 1 -- start start 2023-10-16T10:09:14.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.963+0000 7f510099e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f50fc15aaa0 0x7f50fc15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:14.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.963+0000 7f510099e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50fc0ce390 0x7f50fc070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:14.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.963+0000 7f510099e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50fc15ed60 con 0x7f50fc15ae80 2023-10-16T10:09:14.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.963+0000 7f510099e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50fc06b810 con 0x7f50fc15aaa0 2023-10-16T10:09:14.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.963+0000 7f510099e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc0707e0 con 0x7f50fc0ce390 2023-10-16T10:09:14.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.963+0000 7f50fa59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f50fc15aaa0 0x7f50fc15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:14.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.963+0000 7f50fa59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f50fc15aaa0 0x7f50fc15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51476/0 (socket says 172.21.15.84:51476) 2023-10-16T10:09:14.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.963+0000 7f50fa59c640 1 -- 172.21.15.84:0/3338773817 learned_addr learned my addr 172.21.15.84:0/3338773817 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:14.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.964+0000 7f50fa59c640 1 -- 172.21.15.84:0/3338773817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50fc0ce390 msgr2=0x7f50fc070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:14.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.964+0000 7f50fa59c640 1 --2- 172.21.15.84:0/3338773817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50fc0ce390 0x7f50fc070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.964+0000 7f50fa59c640 1 -- 172.21.15.84:0/3338773817 >> v1:172.21.15.157:6789/0 conn(0x7f50fc15ae80 legacy=0x7f50fc0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:14.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.964+0000 7f50fa59c640 1 -- 172.21.15.84:0/3338773817 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50fc15ef90 con 0x7f50fc15aaa0 2023-10-16T10:09:14.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.965+0000 7f50fa59c640 1 --2- 172.21.15.84:0/3338773817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50fc15aaa0 0x7f50fc15e820 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0cd990 tx=0x7f50ec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ecc866d701177d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:14.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.965+0000 7f50f9d9b640 1 -- 172.21.15.84:0/3338773817 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec031640 con 0x7f50fc15aaa0 2023-10-16T10:09:14.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.965+0000 7f50f9d9b640 1 -- 172.21.15.84:0/3338773817 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ec0317a0 con 0x7f50fc15aaa0 2023-10-16T10:09:14.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.965+0000 7f50f9d9b640 1 -- 172.21.15.84:0/3338773817 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec031b30 con 0x7f50fc15aaa0 2023-10-16T10:09:14.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.966+0000 7f510099e640 1 -- 172.21.15.84:0/3338773817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50fc15aaa0 msgr2=0x7f50fc15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:14.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.966+0000 7f510099e640 1 --2- 172.21.15.84:0/3338773817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50fc15aaa0 0x7f50fc15e820 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0cd990 tx=0x7f50ec02b8f0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.966+0000 7f510099e640 1 -- 172.21.15.84:0/3338773817 shutdown_connections 2023-10-16T10:09:14.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.966+0000 7f510099e640 1 --2- 172.21.15.84:0/3338773817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50fc0ce390 0x7f50fc070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.966+0000 7f510099e640 1 --2- 172.21.15.84:0/3338773817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50fc15aaa0 0x7f50fc15e820 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.966+0000 7f510099e640 1 -- 172.21.15.84:0/3338773817 >> 172.21.15.84:0/3338773817 conn(0x7f50fc0cb410 msgr2=0x7f50fc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:14.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.966+0000 7f510099e640 1 -- 172.21.15.84:0/3338773817 shutdown_connections 2023-10-16T10:09:14.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.966+0000 7f510099e640 1 -- 172.21.15.84:0/3338773817 wait complete. 2023-10-16T10:09:14.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.968+0000 7f510099e640 1 Processor -- start 2023-10-16T10:09:14.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.968+0000 7f510099e640 1 -- start start 2023-10-16T10:09:14.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.969+0000 7f510099e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50fc0ce390 0x7f50fc147200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:14.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.969+0000 7f510099e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50fc15aaa0 0x7f50fc0c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:14.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.969+0000 7f510099e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50fc15ae80 0x7f50fc0c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:14.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.969+0000 7f510099e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc147740 con 0x7f50fc15aaa0 2023-10-16T10:09:14.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.969+0000 7f510099e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc0b3850 con 0x7f50fc15ae80 2023-10-16T10:09:14.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.969+0000 7f510099e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc15cec0 con 0x7f50fc0ce390 2023-10-16T10:09:14.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.969+0000 7f50f3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50fc15aaa0 0x7f50fc0c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:14.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.969+0000 7f50f3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50fc15aaa0 0x7f50fc0c74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51482/0 (socket says 172.21.15.84:51482) 2023-10-16T10:09:14.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.969+0000 7f50f3fff640 1 -- 172.21.15.84:0/1424559148 learned_addr learned my addr 172.21.15.84:0/1424559148 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:14.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.969+0000 7f50fa59c640 1 --2- 172.21.15.84:0/1424559148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50fc0ce390 0x7f50fc147200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:14.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.969+0000 7f50fad9d640 1 --2- 172.21.15.84:0/1424559148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50fc15ae80 0x7f50fc0c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:14.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.969+0000 7f50f3fff640 1 -- 172.21.15.84:0/1424559148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50fc0ce390 msgr2=0x7f50fc147200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:14.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.969+0000 7f50f3fff640 1 --2- 172.21.15.84:0/1424559148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50fc0ce390 0x7f50fc147200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.969+0000 7f50f3fff640 1 -- 172.21.15.84:0/1424559148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50fc15ae80 msgr2=0x7f50fc0c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:14.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.969+0000 7f50f3fff640 1 --2- 172.21.15.84:0/1424559148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50fc15ae80 0x7f50fc0c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:14.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.969+0000 7f50f3fff640 1 -- 172.21.15.84:0/1424559148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec0094d0 con 0x7f50fc15aaa0 2023-10-16T10:09:14.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.969+0000 7f50fad9d640 1 --2- 172.21.15.84:0/1424559148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50fc15ae80 0x7f50fc0c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:14.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.970+0000 7f50f3fff640 1 --2- 172.21.15.84:0/1424559148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50fc15aaa0 0x7f50fc0c74c0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f50e400b3c0 tx=0x7f50e400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:14.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.970+0000 7f50f2ffd640 1 -- 172.21.15.84:0/1424559148 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4011830 con 0x7f50fc15aaa0 2023-10-16T10:09:14.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.970+0000 7f50f2ffd640 1 -- 172.21.15.84:0/1424559148 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e4011990 con 0x7f50fc15aaa0 2023-10-16T10:09:14.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.970+0000 7f50f2ffd640 1 -- 172.21.15.84:0/1424559148 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4011d60 con 0x7f50fc15aaa0 2023-10-16T10:09:14.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.970+0000 7f510099e640 1 -- 172.21.15.84:0/1424559148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50fc0c8080 con 0x7f50fc15aaa0 2023-10-16T10:09:14.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.970+0000 7f510099e640 1 -- 172.21.15.84:0/1424559148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50fc15c870 con 0x7f50fc15aaa0 2023-10-16T10:09:14.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.972+0000 7f510099e640 1 -- 172.21.15.84:0/1424559148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50fc15c6d0 con 0x7f50fc15aaa0 2023-10-16T10:09:14.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.972+0000 7f50f2ffd640 1 -- 172.21.15.84:0/1424559148 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f50e401b460 con 0x7f50fc15aaa0 2023-10-16T10:09:14.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.973+0000 7f50f2ffd640 1 --2- 172.21.15.84:0/1424559148 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f50dc06af90 0x7f50dc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:14.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.973+0000 7f50f2ffd640 1 -- 172.21.15.84:0/1424559148 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f50e40895c0 con 0x7f50fc15aaa0 2023-10-16T10:09:14.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.974+0000 7f50fa59c640 1 --2- 172.21.15.84:0/1424559148 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f50dc06af90 0x7f50dc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:14.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.974+0000 7f50fa59c640 1 --2- 172.21.15.84:0/1424559148 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f50dc06af90 0x7f50dc06d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f50ec036000 tx=0x7f50ec03c040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:14.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:14.978+0000 7f50f2ffd640 1 -- 172.21.15.84:0/1424559148 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f50fc15c6d0 con 0x7f50fc15aaa0 2023-10-16T10:09:15.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.171+0000 7f510099e640 1 -- 172.21.15.84:0/1424559148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50fc15c6d0 con 0x7f50fc15aaa0 2023-10-16T10:09:15.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.173+0000 7f50f2ffd640 1 -- 172.21.15.84:0/1424559148 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16599 (secure 0 0 0) 0x7f50fc15c6d0 con 0x7f50fc15aaa0 2023-10-16T10:09:15.174 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:09:15.174 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":151,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T10:09:12.192240+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:09:08.490425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-16T10:09:09.666051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:09:15.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.176+0000 7f510099e640 1 -- 172.21.15.84:0/1424559148 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f50dc06af90 msgr2=0x7f50dc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:15.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.176+0000 7f510099e640 1 --2- 172.21.15.84:0/1424559148 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f50dc06af90 0x7f50dc06d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f50ec036000 tx=0x7f50ec03c040 comp rx=0 tx=0).stop 2023-10-16T10:09:15.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.176+0000 7f510099e640 1 -- 172.21.15.84:0/1424559148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50fc15aaa0 msgr2=0x7f50fc0c74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:15.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.176+0000 7f510099e640 1 --2- 172.21.15.84:0/1424559148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50fc15aaa0 0x7f50fc0c74c0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f50e400b3c0 tx=0x7f50e400b880 comp rx=0 tx=0).stop 2023-10-16T10:09:15.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.176+0000 7f510099e640 1 -- 172.21.15.84:0/1424559148 shutdown_connections 2023-10-16T10:09:15.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.176+0000 7f510099e640 1 --2- 172.21.15.84:0/1424559148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50fc15ae80 0x7f50fc0c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:15.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.176+0000 7f510099e640 1 --2- 172.21.15.84:0/1424559148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50fc15aaa0 0x7f50fc0c74c0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:15.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.176+0000 7f510099e640 1 --2- 172.21.15.84:0/1424559148 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f50dc06af90 0x7f50dc06d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:15.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.176+0000 7f510099e640 1 --2- 172.21.15.84:0/1424559148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50fc0ce390 0x7f50fc147200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:15.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.176+0000 7f510099e640 1 -- 172.21.15.84:0/1424559148 >> 172.21.15.84:0/1424559148 conn(0x7f50fc0cb410 msgr2=0x7f50fc06ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:15.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.176+0000 7f510099e640 1 -- 172.21.15.84:0/1424559148 shutdown_connections 2023-10-16T10:09:15.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.176+0000 7f510099e640 1 -- 172.21.15.84:0/1424559148 wait complete. 2023-10-16T10:09:15.194 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T10:09:15.195 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:09:15.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.411+0000 7fb372651640 1 Processor -- start 2023-10-16T10:09:15.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.411+0000 7fb372651640 1 -- start start 2023-10-16T10:09:15.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.413+0000 7fb372651640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb36c065ce0 0x7fb36c0710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:15.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.413+0000 7fb372651640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb36c06adb0 0x7fb36c06b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:15.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.413+0000 7fb372651640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36c0715e0 con 0x7fb36c070cc0 2023-10-16T10:09:15.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.413+0000 7fb372651640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36c06b780 con 0x7fb36c065ce0 2023-10-16T10:09:15.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.413+0000 7fb372651640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36c066140 con 0x7fb36c06adb0 2023-10-16T10:09:15.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.413+0000 7fb363fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb36c065ce0 0x7fb36c0710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:15.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.413+0000 7fb363fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb36c065ce0 0x7fb36c0710a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51490/0 (socket says 172.21.15.84:51490) 2023-10-16T10:09:15.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.413+0000 7fb363fff640 1 -- 172.21.15.84:0/2318553217 learned_addr learned my addr 172.21.15.84:0/2318553217 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:15.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.414+0000 7fb363fff640 1 -- 172.21.15.84:0/2318553217 >> v2:172.21.15.157:3300/0 conn(0x7fb36c06adb0 msgr2=0x7fb36c06b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:15.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.414+0000 7fb363fff640 1 --2- 172.21.15.84:0/2318553217 >> v2:172.21.15.157:3300/0 conn(0x7fb36c06adb0 0x7fb36c06b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:15.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.414+0000 7fb363fff640 1 -- 172.21.15.84:0/2318553217 >> v1:172.21.15.157:6789/0 conn(0x7fb36c070cc0 legacy=0x7fb36c0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:15.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.414+0000 7fb363fff640 1 -- 172.21.15.84:0/2318553217 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c0cd450 con 0x7fb36c065ce0 2023-10-16T10:09:15.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.415+0000 7fb363fff640 1 --2- 172.21.15.84:0/2318553217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb36c065ce0 0x7fb36c0710a0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fb36c06fce0 tx=0x7fb35402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6d31f84e260ce6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:15.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.415+0000 7fb36b7fe640 1 -- 172.21.15.84:0/2318553217 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb354031870 con 0x7fb36c065ce0 2023-10-16T10:09:15.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.415+0000 7fb36b7fe640 1 -- 172.21.15.84:0/2318553217 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3540319d0 con 0x7fb36c065ce0 2023-10-16T10:09:15.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.415+0000 7fb36b7fe640 1 -- 172.21.15.84:0/2318553217 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb354031d60 con 0x7fb36c065ce0 2023-10-16T10:09:15.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.416+0000 7fb372651640 1 -- 172.21.15.84:0/2318553217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb36c065ce0 msgr2=0x7fb36c0710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:15.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.416+0000 7fb372651640 1 --2- 172.21.15.84:0/2318553217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb36c065ce0 0x7fb36c0710a0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fb36c06fce0 tx=0x7fb35402b910 comp rx=0 tx=0).stop 2023-10-16T10:09:15.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.416+0000 7fb372651640 1 -- 172.21.15.84:0/2318553217 shutdown_connections 2023-10-16T10:09:15.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.416+0000 7fb372651640 1 --2- 172.21.15.84:0/2318553217 >> v2:172.21.15.157:3300/0 conn(0x7fb36c06adb0 0x7fb36c06b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:15.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.416+0000 7fb372651640 1 --2- 172.21.15.84:0/2318553217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb36c065ce0 0x7fb36c0710a0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:15.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.416+0000 7fb372651640 1 -- 172.21.15.84:0/2318553217 >> 172.21.15.84:0/2318553217 conn(0x7fb36c0b0c80 msgr2=0x7fb36c0b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:15.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.416+0000 7fb372651640 1 -- 172.21.15.84:0/2318553217 shutdown_connections 2023-10-16T10:09:15.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.416+0000 7fb372651640 1 -- 172.21.15.84:0/2318553217 wait complete. 2023-10-16T10:09:15.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.419+0000 7fb372651640 1 Processor -- start 2023-10-16T10:09:15.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.419+0000 7fb372651640 1 -- start start 2023-10-16T10:09:15.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.420+0000 7fb372651640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb36c065ce0 0x7fb36c06bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:15.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.420+0000 7fb372651640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb36c06adb0 0x7fb36c1f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:15.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.420+0000 7fb372651640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb36c070cc0 0x7fb36c1f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:15.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.420+0000 7fb372651640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c1f8180 con 0x7fb36c070cc0 2023-10-16T10:09:15.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.420+0000 7fb372651640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c1f4be0 con 0x7fb36c06adb0 2023-10-16T10:09:15.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.420+0000 7fb372651640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c06c0f0 con 0x7fb36c065ce0 2023-10-16T10:09:15.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.420+0000 7fb370bee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb36c070cc0 0x7fb36c1f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:15.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.421+0000 7fb370bee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb36c070cc0 0x7fb36c1f46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51496/0 (socket says 172.21.15.84:51496) 2023-10-16T10:09:15.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.421+0000 7fb370bee640 1 -- 172.21.15.84:0/2675672822 learned_addr learned my addr 172.21.15.84:0/2675672822 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:15.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.421+0000 7fb36bfff640 1 --2- 172.21.15.84:0/2675672822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb36c065ce0 0x7fb36c06bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:15.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.421+0000 7fb363fff640 1 --2- 172.21.15.84:0/2675672822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb36c06adb0 0x7fb36c1f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:15.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.421+0000 7fb370bee640 1 -- 172.21.15.84:0/2675672822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb36c065ce0 msgr2=0x7fb36c06bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:15.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.421+0000 7fb370bee640 1 --2- 172.21.15.84:0/2675672822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb36c065ce0 0x7fb36c06bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:15.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.421+0000 7fb370bee640 1 -- 172.21.15.84:0/2675672822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb36c06adb0 msgr2=0x7fb36c1f7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:15.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.421+0000 7fb370bee640 1 --2- 172.21.15.84:0/2675672822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb36c06adb0 0x7fb36c1f7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:15.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.421+0000 7fb370bee640 1 -- 172.21.15.84:0/2675672822 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3540094d0 con 0x7fb36c070cc0 2023-10-16T10:09:15.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.421+0000 7fb370bee640 1 --2- 172.21.15.84:0/2675672822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb36c070cc0 0x7fb36c1f46a0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fb36400b970 tx=0x7fb36400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:15.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.422+0000 7fb3697fa640 1 -- 172.21.15.84:0/2675672822 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36400cb40 con 0x7fb36c070cc0 2023-10-16T10:09:15.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.422+0000 7fb3697fa640 1 -- 172.21.15.84:0/2675672822 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36400ccc0 con 0x7fb36c070cc0 2023-10-16T10:09:15.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.422+0000 7fb3697fa640 1 -- 172.21.15.84:0/2675672822 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb364014680 con 0x7fb36c070cc0 2023-10-16T10:09:15.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.422+0000 7fb372651640 1 -- 172.21.15.84:0/2675672822 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36c06c3d0 con 0x7fb36c070cc0 2023-10-16T10:09:15.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.422+0000 7fb372651640 1 -- 172.21.15.84:0/2675672822 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36c06c990 con 0x7fb36c070cc0 2023-10-16T10:09:15.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.423+0000 7fb372651640 1 -- 172.21.15.84:0/2675672822 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36c04e160 con 0x7fb36c070cc0 2023-10-16T10:09:15.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.424+0000 7fb3697fa640 1 -- 172.21.15.84:0/2675672822 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb36401d3b0 con 0x7fb36c070cc0 2023-10-16T10:09:15.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.425+0000 7fb3697fa640 1 --2- 172.21.15.84:0/2675672822 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fb34c06af90 0x7fb34c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:15.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.425+0000 7fb3697fa640 1 -- 172.21.15.84:0/2675672822 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb364089770 con 0x7fb36c070cc0 2023-10-16T10:09:15.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.425+0000 7fb36bfff640 1 --2- 172.21.15.84:0/2675672822 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fb34c06af90 0x7fb34c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:15.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.426+0000 7fb36bfff640 1 --2- 172.21.15.84:0/2675672822 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fb34c06af90 0x7fb34c06d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb35c007fd0 tx=0x7fb35c009040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:15.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.429+0000 7fb3697fa640 1 -- 172.21.15.84:0/2675672822 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb3640546d0 con 0x7fb36c070cc0 2023-10-16T10:09:15.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.617+0000 7fb372651640 1 -- 172.21.15.84:0/2675672822 --> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb36c163520 con 0x7fb34c06af90 2023-10-16T10:09:15.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.636+0000 7fb3697fa640 1 -- 172.21.15.84:0/2675672822 <== mgr.8904 v2:172.21.15.84:6832/205480762 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287270 (secure 0 0 0) 0x7fb36c163520 con 0x7fb34c06af90 2023-10-16T10:09:15.641 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:09:15.642 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T10:09:15.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.641+0000 7fb372651640 1 -- 172.21.15.84:0/2675672822 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fb34c06af90 msgr2=0x7fb34c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:15.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.641+0000 7fb372651640 1 --2- 172.21.15.84:0/2675672822 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fb34c06af90 0x7fb34c06d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb35c007fd0 tx=0x7fb35c009040 comp rx=0 tx=0).stop 2023-10-16T10:09:15.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.641+0000 7fb372651640 1 -- 172.21.15.84:0/2675672822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb36c070cc0 msgr2=0x7fb36c1f46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:15.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.641+0000 7fb372651640 1 --2- 172.21.15.84:0/2675672822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb36c070cc0 0x7fb36c1f46a0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fb36400b970 tx=0x7fb36400be30 comp rx=0 tx=0).stop 2023-10-16T10:09:15.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.642+0000 7fb372651640 1 -- 172.21.15.84:0/2675672822 shutdown_connections 2023-10-16T10:09:15.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.642+0000 7fb372651640 1 --2- 172.21.15.84:0/2675672822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb36c070cc0 0x7fb36c1f46a0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:15.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.642+0000 7fb372651640 1 --2- 172.21.15.84:0/2675672822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb36c06adb0 0x7fb36c1f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:15.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.642+0000 7fb372651640 1 --2- 172.21.15.84:0/2675672822 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fb34c06af90 0x7fb34c06d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:15.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.642+0000 7fb372651640 1 --2- 172.21.15.84:0/2675672822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb36c065ce0 0x7fb36c06bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:15.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.642+0000 7fb372651640 1 -- 172.21.15.84:0/2675672822 >> 172.21.15.84:0/2675672822 conn(0x7fb36c0b0c80 msgr2=0x7fb36c1601e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:15.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.642+0000 7fb372651640 1 -- 172.21.15.84:0/2675672822 shutdown_connections 2023-10-16T10:09:15.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.642+0000 7fb372651640 1 -- 172.21.15.84:0/2675672822 wait complete. 2023-10-16T10:09:15.662 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-16T10:09:15.371025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1312,"num_read_kb":2614,"num_write":445,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2706,"ondisk_log_size":2706,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":366692,"kb_used_data":15912,"kb_used_omap":12,"kb_used_meta":350643,"kb_avail":754608028,"statfs":{"total":773094113280,"available":772718620672,"internally_reserved":0,"allocated":16293888,"data_stored":8387756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":359059001},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001776"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.207685+0000","last_change":"2023-10-16T10:09:11.213237+0000","last_active":"2023-10-16T10:09:12.207685+0000","last_peered":"2023-10-16T10:09:12.207685+0000","last_clean":"2023-10-16T10:09:12.207685+0000","last_became_active":"2023-10-16T10:09:11.212365+0000","last_became_peered":"2023-10-16T10:09:11.212365+0000","last_unstale":"2023-10-16T10:09:12.207685+0000","last_undegraded":"2023-10-16T10:09:12.207685+0000","last_fullsized":"2023-10-16T10:09:12.207685+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:28:35.977172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208544+0000","last_change":"2023-10-16T10:09:11.241999+0000","last_active":"2023-10-16T10:09:12.208544+0000","last_peered":"2023-10-16T10:09:12.208544+0000","last_clean":"2023-10-16T10:09:12.208544+0000","last_became_active":"2023-10-16T10:09:11.240315+0000","last_became_peered":"2023-10-16T10:09:11.240315+0000","last_unstale":"2023-10-16T10:09:12.208544+0000","last_undegraded":"2023-10-16T10:09:12.208544+0000","last_fullsized":"2023-10-16T10:09:12.208544+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:25:59.127401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208822+0000","last_change":"2023-10-16T10:09:11.225266+0000","last_active":"2023-10-16T10:09:12.208822+0000","last_peered":"2023-10-16T10:09:12.208822+0000","last_clean":"2023-10-16T10:09:12.208822+0000","last_became_active":"2023-10-16T10:09:11.224853+0000","last_became_peered":"2023-10-16T10:09:11.224853+0000","last_unstale":"2023-10-16T10:09:12.208822+0000","last_undegraded":"2023-10-16T10:09:12.208822+0000","last_fullsized":"2023-10-16T10:09:12.208822+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:10.978671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.206562+0000","last_change":"2023-10-16T10:09:11.226367+0000","last_active":"2023-10-16T10:09:12.206562+0000","last_peered":"2023-10-16T10:09:12.206562+0000","last_clean":"2023-10-16T10:09:12.206562+0000","last_became_active":"2023-10-16T10:09:11.226091+0000","last_became_peered":"2023-10-16T10:09:11.226091+0000","last_unstale":"2023-10-16T10:09:12.206562+0000","last_undegraded":"2023-10-16T10:09:12.206562+0000","last_fullsized":"2023-10-16T10:09:12.206562+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:18:30.005002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.050395+0000","last_change":"2023-10-16T10:09:11.227803+0000","last_active":"2023-10-16T10:09:13.050395+0000","last_peered":"2023-10-16T10:09:13.050395+0000","last_clean":"2023-10-16T10:09:13.050395+0000","last_became_active":"2023-10-16T10:09:11.227144+0000","last_became_peered":"2023-10-16T10:09:11.227144+0000","last_unstale":"2023-10-16T10:09:13.050395+0000","last_undegraded":"2023-10-16T10:09:13.050395+0000","last_fullsized":"2023-10-16T10:09:13.050395+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:45:42.610399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.048944+0000","last_change":"2023-10-16T10:09:11.208736+0000","last_active":"2023-10-16T10:09:13.048944+0000","last_peered":"2023-10-16T10:09:13.048944+0000","last_clean":"2023-10-16T10:09:13.048944+0000","last_became_active":"2023-10-16T10:09:11.206827+0000","last_became_peered":"2023-10-16T10:09:11.206827+0000","last_unstale":"2023-10-16T10:09:13.048944+0000","last_undegraded":"2023-10-16T10:09:13.048944+0000","last_fullsized":"2023-10-16T10:09:13.048944+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:48:31.852953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.048019+0000","last_change":"2023-10-16T10:09:11.214970+0000","last_active":"2023-10-16T10:09:13.048019+0000","last_peered":"2023-10-16T10:09:13.048019+0000","last_clean":"2023-10-16T10:09:13.048019+0000","last_became_active":"2023-10-16T10:09:11.214358+0000","last_became_peered":"2023-10-16T10:09:11.214358+0000","last_unstale":"2023-10-16T10:09:13.048019+0000","last_undegraded":"2023-10-16T10:09:13.048019+0000","last_fullsized":"2023-10-16T10:09:13.048019+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:31:23.775489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208389+0000","last_change":"2023-10-16T10:09:11.239754+0000","last_active":"2023-10-16T10:09:12.208389+0000","last_peered":"2023-10-16T10:09:12.208389+0000","last_clean":"2023-10-16T10:09:12.208389+0000","last_became_active":"2023-10-16T10:09:11.238775+0000","last_became_peered":"2023-10-16T10:09:11.238775+0000","last_unstale":"2023-10-16T10:09:12.208389+0000","last_undegraded":"2023-10-16T10:09:12.208389+0000","last_fullsized":"2023-10-16T10:09:12.208389+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:08:55.397866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.209411+0000","last_change":"2023-10-16T10:09:11.227057+0000","last_active":"2023-10-16T10:09:12.209411+0000","last_peered":"2023-10-16T10:09:12.209411+0000","last_clean":"2023-10-16T10:09:12.209411+0000","last_became_active":"2023-10-16T10:09:11.224735+0000","last_became_peered":"2023-10-16T10:09:11.224735+0000","last_unstale":"2023-10-16T10:09:12.209411+0000","last_undegraded":"2023-10-16T10:09:12.209411+0000","last_fullsized":"2023-10-16T10:09:12.209411+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:26.175578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208630+0000","last_change":"2023-10-16T10:09:11.234870+0000","last_active":"2023-10-16T10:09:12.208630+0000","last_peered":"2023-10-16T10:09:12.208630+0000","last_clean":"2023-10-16T10:09:12.208630+0000","last_became_active":"2023-10-16T10:09:11.234328+0000","last_became_peered":"2023-10-16T10:09:11.234328+0000","last_unstale":"2023-10-16T10:09:12.208630+0000","last_undegraded":"2023-10-16T10:09:12.208630+0000","last_fullsized":"2023-10-16T10:09:12.208630+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:32.525637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.336471+0000","last_change":"2023-10-16T10:09:11.243784+0000","last_active":"2023-10-16T10:09:12.336471+0000","last_peered":"2023-10-16T10:09:12.336471+0000","last_clean":"2023-10-16T10:09:12.336471+0000","last_became_active":"2023-10-16T10:09:11.240172+0000","last_became_peered":"2023-10-16T10:09:11.240172+0000","last_unstale":"2023-10-16T10:09:12.336471+0000","last_undegraded":"2023-10-16T10:09:12.336471+0000","last_fullsized":"2023-10-16T10:09:12.336471+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:54:52.623627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208639+0000","last_change":"2023-10-16T10:09:11.222714+0000","last_active":"2023-10-16T10:09:12.208639+0000","last_peered":"2023-10-16T10:09:12.208639+0000","last_clean":"2023-10-16T10:09:12.208639+0000","last_became_active":"2023-10-16T10:09:11.222327+0000","last_became_peered":"2023-10-16T10:09:11.222327+0000","last_unstale":"2023-10-16T10:09:12.208639+0000","last_undegraded":"2023-10-16T10:09:12.208639+0000","last_fullsized":"2023-10-16T10:09:12.208639+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:21:57.195467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.206896+0000","last_change":"2023-10-16T10:09:11.224307+0000","last_active":"2023-10-16T10:09:12.206896+0000","last_peered":"2023-10-16T10:09:12.206896+0000","last_clean":"2023-10-16T10:09:12.206896+0000","last_became_active":"2023-10-16T10:09:11.221606+0000","last_became_peered":"2023-10-16T10:09:11.221606+0000","last_unstale":"2023-10-16T10:09:12.206896+0000","last_undegraded":"2023-10-16T10:09:12.206896+0000","last_fullsized":"2023-10-16T10:09:12.206896+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:59:30.219320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208133+0000","last_change":"2023-10-16T10:09:11.242341+0000","last_active":"2023-10-16T10:09:12.208133+0000","last_peered":"2023-10-16T10:09:12.208133+0000","last_clean":"2023-10-16T10:09:12.208133+0000","last_became_active":"2023-10-16T10:09:11.241853+0000","last_became_peered":"2023-10-16T10:09:11.241853+0000","last_unstale":"2023-10-16T10:09:12.208133+0000","last_undegraded":"2023-10-16T10:09:12.208133+0000","last_fullsized":"2023-10-16T10:09:12.208133+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:36:07.019847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.206349+0000","last_change":"2023-10-16T10:09:11.225717+0000","last_active":"2023-10-16T10:09:12.206349+0000","last_peered":"2023-10-16T10:09:12.206349+0000","last_clean":"2023-10-16T10:09:12.206349+0000","last_became_active":"2023-10-16T10:09:11.222562+0000","last_became_peered":"2023-10-16T10:09:11.222562+0000","last_unstale":"2023-10-16T10:09:12.206349+0000","last_undegraded":"2023-10-16T10:09:12.206349+0000","last_fullsized":"2023-10-16T10:09:12.206349+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:10.978671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.047709+0000","last_change":"2023-10-16T10:09:11.247506+0000","last_active":"2023-10-16T10:09:13.047709+0000","last_peered":"2023-10-16T10:09:13.047709+0000","last_clean":"2023-10-16T10:09:13.047709+0000","last_became_active":"2023-10-16T10:09:11.246798+0000","last_became_peered":"2023-10-16T10:09:11.246798+0000","last_unstale":"2023-10-16T10:09:13.047709+0000","last_undegraded":"2023-10-16T10:09:13.047709+0000","last_fullsized":"2023-10-16T10:09:13.047709+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:36:39.209812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.050704+0000","last_change":"2023-10-16T10:09:11.228421+0000","last_active":"2023-10-16T10:09:13.050704+0000","last_peered":"2023-10-16T10:09:13.050704+0000","last_clean":"2023-10-16T10:09:13.050704+0000","last_became_active":"2023-10-16T10:09:11.228126+0000","last_became_peered":"2023-10-16T10:09:11.228126+0000","last_unstale":"2023-10-16T10:09:13.050704+0000","last_undegraded":"2023-10-16T10:09:13.050704+0000","last_fullsized":"2023-10-16T10:09:13.050704+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:07.335691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.047399+0000","last_change":"2023-10-16T10:09:11.235464+0000","last_active":"2023-10-16T10:09:13.047399+0000","last_peered":"2023-10-16T10:09:13.047399+0000","last_clean":"2023-10-16T10:09:13.047399+0000","last_became_active":"2023-10-16T10:09:11.234994+0000","last_became_peered":"2023-10-16T10:09:11.234994+0000","last_unstale":"2023-10-16T10:09:13.047399+0000","last_undegraded":"2023-10-16T10:09:13.047399+0000","last_fullsized":"2023-10-16T10:09:13.047399+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:03.475900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.336626+0000","last_change":"2023-10-16T10:09:11.236149+0000","last_active":"2023-10-16T10:09:12.336626+0000","last_peered":"2023-10-16T10:09:12.336626+0000","last_clean":"2023-10-16T10:09:12.336626+0000","last_became_active":"2023-10-16T10:09:11.235887+0000","last_became_peered":"2023-10-16T10:09:11.235887+0000","last_unstale":"2023-10-16T10:09:12.336626+0000","last_undegraded":"2023-10-16T10:09:12.336626+0000","last_fullsized":"2023-10-16T10:09:12.336626+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:18.434293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.047634+0000","last_change":"2023-10-16T10:09:11.241595+0000","last_active":"2023-10-16T10:09:13.047634+0000","last_peered":"2023-10-16T10:09:13.047634+0000","last_clean":"2023-10-16T10:09:13.047634+0000","last_became_active":"2023-10-16T10:09:11.241300+0000","last_became_peered":"2023-10-16T10:09:11.241300+0000","last_unstale":"2023-10-16T10:09:13.047634+0000","last_undegraded":"2023-10-16T10:09:13.047634+0000","last_fullsized":"2023-10-16T10:09:13.047634+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:48:01.795616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.050102+0000","last_change":"2023-10-16T10:09:11.227980+0000","last_active":"2023-10-16T10:09:13.050102+0000","last_peered":"2023-10-16T10:09:13.050102+0000","last_clean":"2023-10-16T10:09:13.050102+0000","last_became_active":"2023-10-16T10:09:11.227533+0000","last_became_peered":"2023-10-16T10:09:11.227533+0000","last_unstale":"2023-10-16T10:09:13.050102+0000","last_undegraded":"2023-10-16T10:09:13.050102+0000","last_fullsized":"2023-10-16T10:09:13.050102+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:22:45.857135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.336353+0000","last_change":"2023-10-16T10:09:11.234617+0000","last_active":"2023-10-16T10:09:12.336353+0000","last_peered":"2023-10-16T10:09:12.336353+0000","last_clean":"2023-10-16T10:09:12.336353+0000","last_became_active":"2023-10-16T10:09:11.234291+0000","last_became_peered":"2023-10-16T10:09:11.234291+0000","last_unstale":"2023-10-16T10:09:12.336353+0000","last_undegraded":"2023-10-16T10:09:12.336353+0000","last_fullsized":"2023-10-16T10:09:12.336353+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:54:20.233069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.209261+0000","last_change":"2023-10-16T10:09:11.224584+0000","last_active":"2023-10-16T10:09:12.209261+0000","last_peered":"2023-10-16T10:09:12.209261+0000","last_clean":"2023-10-16T10:09:12.209261+0000","last_became_active":"2023-10-16T10:09:11.224283+0000","last_became_peered":"2023-10-16T10:09:11.224283+0000","last_unstale":"2023-10-16T10:09:12.209261+0000","last_undegraded":"2023-10-16T10:09:12.209261+0000","last_fullsized":"2023-10-16T10:09:12.209261+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:18:30.005002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.335934+0000","last_change":"2023-10-16T10:09:11.227974+0000","last_active":"2023-10-16T10:09:12.335934+0000","last_peered":"2023-10-16T10:09:12.335934+0000","last_clean":"2023-10-16T10:09:12.335934+0000","last_became_active":"2023-10-16T10:09:11.226044+0000","last_became_peered":"2023-10-16T10:09:11.226044+0000","last_unstale":"2023-10-16T10:09:12.335934+0000","last_undegraded":"2023-10-16T10:09:12.335934+0000","last_fullsized":"2023-10-16T10:09:12.335934+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:32.525637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208719+0000","last_change":"2023-10-16T10:09:11.242688+0000","last_active":"2023-10-16T10:09:12.208719+0000","last_peered":"2023-10-16T10:09:12.208719+0000","last_clean":"2023-10-16T10:09:12.208719+0000","last_became_active":"2023-10-16T10:09:11.240760+0000","last_became_peered":"2023-10-16T10:09:11.240760+0000","last_unstale":"2023-10-16T10:09:12.208719+0000","last_undegraded":"2023-10-16T10:09:12.208719+0000","last_fullsized":"2023-10-16T10:09:12.208719+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:54:20.233069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.047545+0000","last_change":"2023-10-16T10:09:11.244664+0000","last_active":"2023-10-16T10:09:13.047545+0000","last_peered":"2023-10-16T10:09:13.047545+0000","last_clean":"2023-10-16T10:09:13.047545+0000","last_became_active":"2023-10-16T10:09:11.244407+0000","last_became_peered":"2023-10-16T10:09:11.244407+0000","last_unstale":"2023-10-16T10:09:13.047545+0000","last_undegraded":"2023-10-16T10:09:13.047545+0000","last_fullsized":"2023-10-16T10:09:13.047545+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:01.612542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.206787+0000","last_change":"2023-10-16T10:09:11.215670+0000","last_active":"2023-10-16T10:09:12.206787+0000","last_peered":"2023-10-16T10:09:12.206787+0000","last_clean":"2023-10-16T10:09:12.206787+0000","last_became_active":"2023-10-16T10:09:11.213422+0000","last_became_peered":"2023-10-16T10:09:11.213422+0000","last_unstale":"2023-10-16T10:09:12.206787+0000","last_undegraded":"2023-10-16T10:09:12.206787+0000","last_fullsized":"2023-10-16T10:09:12.206787+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:02:23.577704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.049216+0000","last_change":"2023-10-16T10:09:11.232744+0000","last_active":"2023-10-16T10:09:13.049216+0000","last_peered":"2023-10-16T10:09:13.049216+0000","last_clean":"2023-10-16T10:09:13.049216+0000","last_became_active":"2023-10-16T10:09:11.232438+0000","last_became_peered":"2023-10-16T10:09:11.232438+0000","last_unstale":"2023-10-16T10:09:13.049216+0000","last_undegraded":"2023-10-16T10:09:13.049216+0000","last_fullsized":"2023-10-16T10:09:13.049216+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:33.177507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.206979+0000","last_change":"2023-10-16T10:09:11.233847+0000","last_active":"2023-10-16T10:09:12.206979+0000","last_peered":"2023-10-16T10:09:12.206979+0000","last_clean":"2023-10-16T10:09:12.206979+0000","last_became_active":"2023-10-16T10:09:11.223153+0000","last_became_peered":"2023-10-16T10:09:11.223153+0000","last_unstale":"2023-10-16T10:09:12.206979+0000","last_undegraded":"2023-10-16T10:09:12.206979+0000","last_fullsized":"2023-10-16T10:09:12.206979+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:54.541150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.206677+0000","last_change":"2023-10-16T10:09:11.221428+0000","last_active":"2023-10-16T10:09:12.206677+0000","last_peered":"2023-10-16T10:09:12.206677+0000","last_clean":"2023-10-16T10:09:12.206677+0000","last_became_active":"2023-10-16T10:09:11.220977+0000","last_became_peered":"2023-10-16T10:09:11.220977+0000","last_unstale":"2023-10-16T10:09:12.206677+0000","last_undegraded":"2023-10-16T10:09:12.206677+0000","last_fullsized":"2023-10-16T10:09:12.206677+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:26.175578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.050476+0000","last_change":"2023-10-16T10:09:11.232879+0000","last_active":"2023-10-16T10:09:13.050476+0000","last_peered":"2023-10-16T10:09:13.050476+0000","last_clean":"2023-10-16T10:09:13.050476+0000","last_became_active":"2023-10-16T10:09:11.232131+0000","last_became_peered":"2023-10-16T10:09:11.232131+0000","last_unstale":"2023-10-16T10:09:13.050476+0000","last_undegraded":"2023-10-16T10:09:13.050476+0000","last_fullsized":"2023-10-16T10:09:13.050476+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:50.625153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.047508+0000","last_change":"2023-10-16T10:09:11.242790+0000","last_active":"2023-10-16T10:09:13.047508+0000","last_peered":"2023-10-16T10:09:13.047508+0000","last_clean":"2023-10-16T10:09:13.047508+0000","last_became_active":"2023-10-16T10:09:11.242375+0000","last_became_peered":"2023-10-16T10:09:11.242375+0000","last_unstale":"2023-10-16T10:09:13.047508+0000","last_undegraded":"2023-10-16T10:09:13.047508+0000","last_fullsized":"2023-10-16T10:09:13.047508+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:54.351202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.047751+0000","last_change":"2023-10-16T10:09:11.235262+0000","last_active":"2023-10-16T10:09:13.047751+0000","last_peered":"2023-10-16T10:09:13.047751+0000","last_clean":"2023-10-16T10:09:13.047751+0000","last_became_active":"2023-10-16T10:09:11.234718+0000","last_became_peered":"2023-10-16T10:09:11.234718+0000","last_unstale":"2023-10-16T10:09:13.047751+0000","last_undegraded":"2023-10-16T10:09:13.047751+0000","last_fullsized":"2023-10-16T10:09:13.047751+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:31.713636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"150'4","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.249089+0000","last_change":"2023-10-16T10:09:10.208679+0000","last_active":"2023-10-16T10:09:11.249089+0000","last_peered":"2023-10-16T10:09:11.249089+0000","last_clean":"2023-10-16T10:09:11.249089+0000","last_became_active":"2023-10-16T10:09:10.208427+0000","last_became_peered":"2023-10-16T10:09:10.208427+0000","last_unstale":"2023-10-16T10:09:11.249089+0000","last_undegraded":"2023-10-16T10:09:11.249089+0000","last_fullsized":"2023-10-16T10:09:11.249089+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:45:32.302211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.209495+0000","last_change":"2023-10-16T10:09:11.227653+0000","last_active":"2023-10-16T10:09:12.209495+0000","last_peered":"2023-10-16T10:09:12.209495+0000","last_clean":"2023-10-16T10:09:12.209495+0000","last_became_active":"2023-10-16T10:09:11.227391+0000","last_became_peered":"2023-10-16T10:09:11.227391+0000","last_unstale":"2023-10-16T10:09:12.209495+0000","last_undegraded":"2023-10-16T10:09:12.209495+0000","last_fullsized":"2023-10-16T10:09:12.209495+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:02:23.577704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.247997+0000","last_change":"2023-10-16T10:09:10.201030+0000","last_active":"2023-10-16T10:09:11.247997+0000","last_peered":"2023-10-16T10:09:11.247997+0000","last_clean":"2023-10-16T10:09:11.247997+0000","last_became_active":"2023-10-16T10:09:10.199995+0000","last_became_peered":"2023-10-16T10:09:10.199995+0000","last_unstale":"2023-10-16T10:09:11.247997+0000","last_undegraded":"2023-10-16T10:09:11.247997+0000","last_fullsized":"2023-10-16T10:09:11.247997+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:05:24.008125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.051518+0000","last_change":"2023-10-16T10:09:11.231021+0000","last_active":"2023-10-16T10:09:13.051518+0000","last_peered":"2023-10-16T10:09:13.051518+0000","last_clean":"2023-10-16T10:09:13.051518+0000","last_became_active":"2023-10-16T10:09:11.230314+0000","last_became_peered":"2023-10-16T10:09:11.230314+0000","last_unstale":"2023-10-16T10:09:13.051518+0000","last_undegraded":"2023-10-16T10:09:13.051518+0000","last_fullsized":"2023-10-16T10:09:13.051518+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:57.434101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.251607+0000","last_change":"2023-10-16T10:09:10.194076+0000","last_active":"2023-10-16T10:09:11.251607+0000","last_peered":"2023-10-16T10:09:11.251607+0000","last_clean":"2023-10-16T10:09:11.251607+0000","last_became_active":"2023-10-16T10:09:10.193820+0000","last_became_peered":"2023-10-16T10:09:10.193820+0000","last_unstale":"2023-10-16T10:09:11.251607+0000","last_undegraded":"2023-10-16T10:09:11.251607+0000","last_fullsized":"2023-10-16T10:09:11.251607+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:49:33.976254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.051911+0000","last_change":"2023-10-16T10:09:11.244395+0000","last_active":"2023-10-16T10:09:13.051911+0000","last_peered":"2023-10-16T10:09:13.051911+0000","last_clean":"2023-10-16T10:09:13.051911+0000","last_became_active":"2023-10-16T10:09:11.243852+0000","last_became_peered":"2023-10-16T10:09:11.243852+0000","last_unstale":"2023-10-16T10:09:13.051911+0000","last_undegraded":"2023-10-16T10:09:13.051911+0000","last_fullsized":"2023-10-16T10:09:13.051911+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:44.116225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.249665+0000","last_change":"2023-10-16T10:09:10.200360+0000","last_active":"2023-10-16T10:09:11.249665+0000","last_peered":"2023-10-16T10:09:11.249665+0000","last_clean":"2023-10-16T10:09:11.249665+0000","last_became_active":"2023-10-16T10:09:10.199787+0000","last_became_peered":"2023-10-16T10:09:10.199787+0000","last_unstale":"2023-10-16T10:09:11.249665+0000","last_undegraded":"2023-10-16T10:09:11.249665+0000","last_fullsized":"2023-10-16T10:09:11.249665+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:02:22.570336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.046976+0000","last_change":"2023-10-16T10:09:11.237863+0000","last_active":"2023-10-16T10:09:13.046976+0000","last_peered":"2023-10-16T10:09:13.046976+0000","last_clean":"2023-10-16T10:09:13.046976+0000","last_became_active":"2023-10-16T10:09:11.237575+0000","last_became_peered":"2023-10-16T10:09:11.237575+0000","last_unstale":"2023-10-16T10:09:13.046976+0000","last_undegraded":"2023-10-16T10:09:13.046976+0000","last_fullsized":"2023-10-16T10:09:13.046976+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:57:46.806651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.246082+0000","last_change":"2023-10-16T10:09:10.202300+0000","last_active":"2023-10-16T10:09:11.246082+0000","last_peered":"2023-10-16T10:09:11.246082+0000","last_clean":"2023-10-16T10:09:11.246082+0000","last_became_active":"2023-10-16T10:09:10.201978+0000","last_became_peered":"2023-10-16T10:09:10.201978+0000","last_unstale":"2023-10-16T10:09:11.246082+0000","last_undegraded":"2023-10-16T10:09:11.246082+0000","last_fullsized":"2023-10-16T10:09:11.246082+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:05.877227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.048048+0000","last_change":"2023-10-16T10:09:11.228714+0000","last_active":"2023-10-16T10:09:13.048048+0000","last_peered":"2023-10-16T10:09:13.048048+0000","last_clean":"2023-10-16T10:09:13.048048+0000","last_became_active":"2023-10-16T10:09:11.228372+0000","last_became_peered":"2023-10-16T10:09:11.228372+0000","last_unstale":"2023-10-16T10:09:13.048048+0000","last_undegraded":"2023-10-16T10:09:13.048048+0000","last_fullsized":"2023-10-16T10:09:13.048048+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:26.298785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.250384+0000","last_change":"2023-10-16T10:09:10.197219+0000","last_active":"2023-10-16T10:09:11.250384+0000","last_peered":"2023-10-16T10:09:11.250384+0000","last_clean":"2023-10-16T10:09:11.250384+0000","last_became_active":"2023-10-16T10:09:10.196896+0000","last_became_peered":"2023-10-16T10:09:10.196896+0000","last_unstale":"2023-10-16T10:09:11.250384+0000","last_undegraded":"2023-10-16T10:09:11.250384+0000","last_fullsized":"2023-10-16T10:09:11.250384+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:11:30.069197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208440+0000","last_change":"2023-10-16T10:09:11.239535+0000","last_active":"2023-10-16T10:09:12.208440+0000","last_peered":"2023-10-16T10:09:12.208440+0000","last_clean":"2023-10-16T10:09:12.208440+0000","last_became_active":"2023-10-16T10:09:11.238438+0000","last_became_peered":"2023-10-16T10:09:11.238438+0000","last_unstale":"2023-10-16T10:09:12.208440+0000","last_undegraded":"2023-10-16T10:09:12.208440+0000","last_fullsized":"2023-10-16T10:09:12.208440+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:35:25.459805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.203187+0000","last_change":"2023-10-16T10:09:10.205455+0000","last_active":"2023-10-16T10:09:11.203187+0000","last_peered":"2023-10-16T10:09:11.203187+0000","last_clean":"2023-10-16T10:09:11.203187+0000","last_became_active":"2023-10-16T10:09:10.204515+0000","last_became_peered":"2023-10-16T10:09:10.204515+0000","last_unstale":"2023-10-16T10:09:11.203187+0000","last_undegraded":"2023-10-16T10:09:11.203187+0000","last_fullsized":"2023-10-16T10:09:11.203187+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:08:18.685850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.207902+0000","last_change":"2023-10-16T10:09:11.242837+0000","last_active":"2023-10-16T10:09:12.207902+0000","last_peered":"2023-10-16T10:09:12.207902+0000","last_clean":"2023-10-16T10:09:12.207902+0000","last_became_active":"2023-10-16T10:09:11.242506+0000","last_became_peered":"2023-10-16T10:09:11.242506+0000","last_unstale":"2023-10-16T10:09:12.207902+0000","last_undegraded":"2023-10-16T10:09:12.207902+0000","last_fullsized":"2023-10-16T10:09:12.207902+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:53:04.619879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.217590+0000","last_change":"2023-10-16T10:09:10.211411+0000","last_active":"2023-10-16T10:09:11.217590+0000","last_peered":"2023-10-16T10:09:11.217590+0000","last_clean":"2023-10-16T10:09:11.217590+0000","last_became_active":"2023-10-16T10:09:10.210108+0000","last_became_peered":"2023-10-16T10:09:10.210108+0000","last_unstale":"2023-10-16T10:09:11.217590+0000","last_undegraded":"2023-10-16T10:09:11.217590+0000","last_fullsized":"2023-10-16T10:09:11.217590+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:53.533782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.207114+0000","last_change":"2023-10-16T10:09:11.227560+0000","last_active":"2023-10-16T10:09:12.207114+0000","last_peered":"2023-10-16T10:09:12.207114+0000","last_clean":"2023-10-16T10:09:12.207114+0000","last_became_active":"2023-10-16T10:09:11.227297+0000","last_became_peered":"2023-10-16T10:09:11.227297+0000","last_unstale":"2023-10-16T10:09:12.207114+0000","last_undegraded":"2023-10-16T10:09:12.207114+0000","last_fullsized":"2023-10-16T10:09:12.207114+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:06.884595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.225428+0000","last_change":"2023-10-16T10:09:10.204028+0000","last_active":"2023-10-16T10:09:11.225428+0000","last_peered":"2023-10-16T10:09:11.225428+0000","last_clean":"2023-10-16T10:09:11.225428+0000","last_became_active":"2023-10-16T10:09:10.203742+0000","last_became_peered":"2023-10-16T10:09:10.203742+0000","last_unstale":"2023-10-16T10:09:11.225428+0000","last_undegraded":"2023-10-16T10:09:11.225428+0000","last_fullsized":"2023-10-16T10:09:11.225428+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:22:12.115124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.050069+0000","last_change":"2023-10-16T10:09:11.232432+0000","last_active":"2023-10-16T10:09:13.050069+0000","last_peered":"2023-10-16T10:09:13.050069+0000","last_clean":"2023-10-16T10:09:13.050069+0000","last_became_active":"2023-10-16T10:09:11.232157+0000","last_became_peered":"2023-10-16T10:09:11.232157+0000","last_unstale":"2023-10-16T10:09:13.050069+0000","last_undegraded":"2023-10-16T10:09:13.050069+0000","last_fullsized":"2023-10-16T10:09:13.050069+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:26:04.070083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.231203+0000","last_change":"2023-10-16T10:09:10.209334+0000","last_active":"2023-10-16T10:09:11.231203+0000","last_peered":"2023-10-16T10:09:11.231203+0000","last_clean":"2023-10-16T10:09:11.231203+0000","last_became_active":"2023-10-16T10:09:10.209055+0000","last_became_peered":"2023-10-16T10:09:10.209055+0000","last_unstale":"2023-10-16T10:09:11.231203+0000","last_undegraded":"2023-10-16T10:09:11.231203+0000","last_fullsized":"2023-10-16T10:09:11.231203+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:24:59.824573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.051749+0000","last_change":"2023-10-16T10:09:11.233258+0000","last_active":"2023-10-16T10:09:13.051749+0000","last_peered":"2023-10-16T10:09:13.051749+0000","last_clean":"2023-10-16T10:09:13.051749+0000","last_became_active":"2023-10-16T10:09:11.232650+0000","last_became_peered":"2023-10-16T10:09:11.232650+0000","last_unstale":"2023-10-16T10:09:13.051749+0000","last_undegraded":"2023-10-16T10:09:13.051749+0000","last_fullsized":"2023-10-16T10:09:13.051749+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:31.100635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"150'3","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.250243+0000","last_change":"2023-10-16T10:09:10.205537+0000","last_active":"2023-10-16T10:09:11.250243+0000","last_peered":"2023-10-16T10:09:11.250243+0000","last_clean":"2023-10-16T10:09:11.250243+0000","last_became_active":"2023-10-16T10:09:10.205274+0000","last_became_peered":"2023-10-16T10:09:10.205274+0000","last_unstale":"2023-10-16T10:09:11.250243+0000","last_undegraded":"2023-10-16T10:09:11.250243+0000","last_fullsized":"2023-10-16T10:09:11.250243+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:04.167427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.051781+0000","last_change":"2023-10-16T10:09:11.229835+0000","last_active":"2023-10-16T10:09:13.051781+0000","last_peered":"2023-10-16T10:09:13.051781+0000","last_clean":"2023-10-16T10:09:13.051781+0000","last_became_active":"2023-10-16T10:09:11.229562+0000","last_became_peered":"2023-10-16T10:09:11.229562+0000","last_unstale":"2023-10-16T10:09:13.051781+0000","last_undegraded":"2023-10-16T10:09:13.051781+0000","last_fullsized":"2023-10-16T10:09:13.051781+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:04:11.537413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"150'6","reported_seq":18,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.248864+0000","last_change":"2023-10-16T10:09:10.211025+0000","last_active":"2023-10-16T10:09:11.248864+0000","last_peered":"2023-10-16T10:09:11.248864+0000","last_clean":"2023-10-16T10:09:11.248864+0000","last_became_active":"2023-10-16T10:09:10.209558+0000","last_became_peered":"2023-10-16T10:09:10.209558+0000","last_unstale":"2023-10-16T10:09:11.248864+0000","last_undegraded":"2023-10-16T10:09:11.248864+0000","last_fullsized":"2023-10-16T10:09:11.248864+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:55.065613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.207134+0000","last_change":"2023-10-16T10:09:11.227859+0000","last_active":"2023-10-16T10:09:12.207134+0000","last_peered":"2023-10-16T10:09:12.207134+0000","last_clean":"2023-10-16T10:09:12.207134+0000","last_became_active":"2023-10-16T10:09:11.226595+0000","last_became_peered":"2023-10-16T10:09:11.226595+0000","last_unstale":"2023-10-16T10:09:12.207134+0000","last_undegraded":"2023-10-16T10:09:12.207134+0000","last_fullsized":"2023-10-16T10:09:12.207134+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:11:31.338000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.197612+0000","last_change":"2023-10-16T10:09:10.199960+0000","last_active":"2023-10-16T10:09:11.197612+0000","last_peered":"2023-10-16T10:09:11.197612+0000","last_clean":"2023-10-16T10:09:11.197612+0000","last_became_active":"2023-10-16T10:09:10.199682+0000","last_became_peered":"2023-10-16T10:09:10.199682+0000","last_unstale":"2023-10-16T10:09:11.197612+0000","last_undegraded":"2023-10-16T10:09:11.197612+0000","last_fullsized":"2023-10-16T10:09:11.197612+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:52:42.889786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.207762+0000","last_change":"2023-10-16T10:09:11.222052+0000","last_active":"2023-10-16T10:09:12.207762+0000","last_peered":"2023-10-16T10:09:12.207762+0000","last_clean":"2023-10-16T10:09:12.207762+0000","last_became_active":"2023-10-16T10:09:11.221700+0000","last_became_peered":"2023-10-16T10:09:11.221700+0000","last_unstale":"2023-10-16T10:09:12.207762+0000","last_undegraded":"2023-10-16T10:09:12.207762+0000","last_fullsized":"2023-10-16T10:09:12.207762+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:53:25.682713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.224594+0000","last_change":"2023-10-16T10:09:10.208071+0000","last_active":"2023-10-16T10:09:11.224594+0000","last_peered":"2023-10-16T10:09:11.224594+0000","last_clean":"2023-10-16T10:09:11.224594+0000","last_became_active":"2023-10-16T10:09:10.207730+0000","last_became_peered":"2023-10-16T10:09:10.207730+0000","last_unstale":"2023-10-16T10:09:11.224594+0000","last_undegraded":"2023-10-16T10:09:11.224594+0000","last_fullsized":"2023-10-16T10:09:11.224594+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:09:14.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208722+0000","last_change":"2023-10-16T10:09:11.223324+0000","last_active":"2023-10-16T10:09:12.208722+0000","last_peered":"2023-10-16T10:09:12.208722+0000","last_clean":"2023-10-16T10:09:12.208722+0000","last_became_active":"2023-10-16T10:09:11.223039+0000","last_became_peered":"2023-10-16T10:09:11.223039+0000","last_unstale":"2023-10-16T10:09:12.208722+0000","last_undegraded":"2023-10-16T10:09:12.208722+0000","last_fullsized":"2023-10-16T10:09:12.208722+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:59.356399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"150'8","reported_seq":20,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.250551+0000","last_change":"2023-10-16T10:09:10.208706+0000","last_active":"2023-10-16T10:09:11.250551+0000","last_peered":"2023-10-16T10:09:11.250551+0000","last_clean":"2023-10-16T10:09:11.250551+0000","last_became_active":"2023-10-16T10:09:10.208358+0000","last_became_peered":"2023-10-16T10:09:10.208358+0000","last_unstale":"2023-10-16T10:09:11.250551+0000","last_undegraded":"2023-10-16T10:09:11.250551+0000","last_fullsized":"2023-10-16T10:09:11.250551+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:38:02.653183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208652+0000","last_change":"2023-10-16T10:09:11.221285+0000","last_active":"2023-10-16T10:09:12.208652+0000","last_peered":"2023-10-16T10:09:12.208652+0000","last_clean":"2023-10-16T10:09:12.208652+0000","last_became_active":"2023-10-16T10:09:11.221000+0000","last_became_peered":"2023-10-16T10:09:11.221000+0000","last_unstale":"2023-10-16T10:09:12.208652+0000","last_undegraded":"2023-10-16T10:09:12.208652+0000","last_fullsized":"2023-10-16T10:09:12.208652+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:09.872490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.245689+0000","last_change":"2023-10-16T10:09:10.205199+0000","last_active":"2023-10-16T10:09:11.245689+0000","last_peered":"2023-10-16T10:09:11.245689+0000","last_clean":"2023-10-16T10:09:11.245689+0000","last_became_active":"2023-10-16T10:09:10.204886+0000","last_became_peered":"2023-10-16T10:09:10.204886+0000","last_unstale":"2023-10-16T10:09:11.245689+0000","last_undegraded":"2023-10-16T10:09:11.245689+0000","last_fullsized":"2023-10-16T10:09:11.245689+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:08:18.685850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.050416+0000","last_change":"2023-10-16T10:09:11.225800+0000","last_active":"2023-10-16T10:09:13.050416+0000","last_peered":"2023-10-16T10:09:13.050416+0000","last_clean":"2023-10-16T10:09:13.050416+0000","last_became_active":"2023-10-16T10:09:11.225009+0000","last_became_peered":"2023-10-16T10:09:11.225009+0000","last_unstale":"2023-10-16T10:09:13.050416+0000","last_undegraded":"2023-10-16T10:09:13.050416+0000","last_fullsized":"2023-10-16T10:09:13.050416+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:00.485898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.248877+0000","last_change":"2023-10-16T10:09:10.211247+0000","last_active":"2023-10-16T10:09:11.248877+0000","last_peered":"2023-10-16T10:09:11.248877+0000","last_clean":"2023-10-16T10:09:11.248877+0000","last_became_active":"2023-10-16T10:09:10.210943+0000","last_became_peered":"2023-10-16T10:09:10.210943+0000","last_unstale":"2023-10-16T10:09:11.248877+0000","last_undegraded":"2023-10-16T10:09:11.248877+0000","last_fullsized":"2023-10-16T10:09:11.248877+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:36.396211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.047737+0000","last_change":"2023-10-16T10:09:11.241701+0000","last_active":"2023-10-16T10:09:13.047737+0000","last_peered":"2023-10-16T10:09:13.047737+0000","last_clean":"2023-10-16T10:09:13.047737+0000","last_became_active":"2023-10-16T10:09:11.241413+0000","last_became_peered":"2023-10-16T10:09:11.241413+0000","last_unstale":"2023-10-16T10:09:13.047737+0000","last_undegraded":"2023-10-16T10:09:13.047737+0000","last_fullsized":"2023-10-16T10:09:13.047737+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:44:14.173255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.249871+0000","last_change":"2023-10-16T10:09:10.199150+0000","last_active":"2023-10-16T10:09:11.249871+0000","last_peered":"2023-10-16T10:09:11.249871+0000","last_clean":"2023-10-16T10:09:11.249871+0000","last_became_active":"2023-10-16T10:09:10.197993+0000","last_became_peered":"2023-10-16T10:09:10.197993+0000","last_unstale":"2023-10-16T10:09:11.249871+0000","last_undegraded":"2023-10-16T10:09:11.249871+0000","last_fullsized":"2023-10-16T10:09:11.249871+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:26:03.062715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208271+0000","last_change":"2023-10-16T10:09:11.240689+0000","last_active":"2023-10-16T10:09:12.208271+0000","last_peered":"2023-10-16T10:09:12.208271+0000","last_clean":"2023-10-16T10:09:12.208271+0000","last_became_active":"2023-10-16T10:09:11.240387+0000","last_became_peered":"2023-10-16T10:09:11.240387+0000","last_unstale":"2023-10-16T10:09:12.208271+0000","last_undegraded":"2023-10-16T10:09:12.208271+0000","last_fullsized":"2023-10-16T10:09:12.208271+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:49:34.983622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.224288+0000","last_change":"2023-10-16T10:09:10.206139+0000","last_active":"2023-10-16T10:09:11.224288+0000","last_peered":"2023-10-16T10:09:11.224288+0000","last_clean":"2023-10-16T10:09:11.224288+0000","last_became_active":"2023-10-16T10:09:10.205808+0000","last_became_peered":"2023-10-16T10:09:10.205808+0000","last_unstale":"2023-10-16T10:09:11.224288+0000","last_undegraded":"2023-10-16T10:09:11.224288+0000","last_fullsized":"2023-10-16T10:09:11.224288+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:39:56.103852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.206845+0000","last_change":"2023-10-16T10:09:11.218098+0000","last_active":"2023-10-16T10:09:12.206845+0000","last_peered":"2023-10-16T10:09:12.206845+0000","last_clean":"2023-10-16T10:09:12.206845+0000","last_became_active":"2023-10-16T10:09:11.214187+0000","last_became_peered":"2023-10-16T10:09:11.214187+0000","last_unstale":"2023-10-16T10:09:12.206845+0000","last_undegraded":"2023-10-16T10:09:12.206845+0000","last_fullsized":"2023-10-16T10:09:12.206845+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:50:48.914441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.249524+0000","last_change":"2023-10-16T10:09:10.203911+0000","last_active":"2023-10-16T10:09:11.249524+0000","last_peered":"2023-10-16T10:09:11.249524+0000","last_clean":"2023-10-16T10:09:11.249524+0000","last_became_active":"2023-10-16T10:09:10.203003+0000","last_became_peered":"2023-10-16T10:09:10.203003+0000","last_unstale":"2023-10-16T10:09:11.249524+0000","last_undegraded":"2023-10-16T10:09:11.249524+0000","last_fullsized":"2023-10-16T10:09:11.249524+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:15:05.272452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.048408+0000","last_change":"2023-10-16T10:09:11.215534+0000","last_active":"2023-10-16T10:09:13.048408+0000","last_peered":"2023-10-16T10:09:13.048408+0000","last_clean":"2023-10-16T10:09:13.048408+0000","last_became_active":"2023-10-16T10:09:11.215024+0000","last_became_peered":"2023-10-16T10:09:11.215024+0000","last_unstale":"2023-10-16T10:09:13.048408+0000","last_undegraded":"2023-10-16T10:09:13.048408+0000","last_fullsized":"2023-10-16T10:09:13.048408+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:51.817283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.229528+0000","last_change":"2023-10-16T10:09:10.209580+0000","last_active":"2023-10-16T10:09:11.229528+0000","last_peered":"2023-10-16T10:09:11.229528+0000","last_clean":"2023-10-16T10:09:11.229528+0000","last_became_active":"2023-10-16T10:09:10.209024+0000","last_became_peered":"2023-10-16T10:09:10.209024+0000","last_unstale":"2023-10-16T10:09:11.229528+0000","last_undegraded":"2023-10-16T10:09:11.229528+0000","last_fullsized":"2023-10-16T10:09:11.229528+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:26:55.839042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208360+0000","last_change":"2023-10-16T10:09:11.236746+0000","last_active":"2023-10-16T10:09:12.208360+0000","last_peered":"2023-10-16T10:09:12.208360+0000","last_clean":"2023-10-16T10:09:12.208360+0000","last_became_active":"2023-10-16T10:09:11.232883+0000","last_became_peered":"2023-10-16T10:09:11.232883+0000","last_unstale":"2023-10-16T10:09:12.208360+0000","last_undegraded":"2023-10-16T10:09:12.208360+0000","last_fullsized":"2023-10-16T10:09:12.208360+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:10:46.500241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.230045+0000","last_change":"2023-10-16T10:09:10.199961+0000","last_active":"2023-10-16T10:09:11.230045+0000","last_peered":"2023-10-16T10:09:11.230045+0000","last_clean":"2023-10-16T10:09:11.230045+0000","last_became_active":"2023-10-16T10:09:10.199696+0000","last_became_peered":"2023-10-16T10:09:10.199696+0000","last_unstale":"2023-10-16T10:09:11.230045+0000","last_undegraded":"2023-10-16T10:09:11.230045+0000","last_fullsized":"2023-10-16T10:09:11.230045+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:49:51.674081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.050198+0000","last_change":"2023-10-16T10:09:11.245494+0000","last_active":"2023-10-16T10:09:13.050198+0000","last_peered":"2023-10-16T10:09:13.050198+0000","last_clean":"2023-10-16T10:09:13.050198+0000","last_became_active":"2023-10-16T10:09:11.244970+0000","last_became_peered":"2023-10-16T10:09:11.244970+0000","last_unstale":"2023-10-16T10:09:13.050198+0000","last_undegraded":"2023-10-16T10:09:13.050198+0000","last_fullsized":"2023-10-16T10:09:13.050198+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:49.918171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.223555+0000","last_change":"2023-10-16T10:09:10.209122+0000","last_active":"2023-10-16T10:09:11.223555+0000","last_peered":"2023-10-16T10:09:11.223555+0000","last_clean":"2023-10-16T10:09:11.223555+0000","last_became_active":"2023-10-16T10:09:10.208536+0000","last_became_peered":"2023-10-16T10:09:10.208536+0000","last_unstale":"2023-10-16T10:09:11.223555+0000","last_undegraded":"2023-10-16T10:09:11.223555+0000","last_fullsized":"2023-10-16T10:09:11.223555+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:26.460188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.051193+0000","last_change":"2023-10-16T10:09:11.225647+0000","last_active":"2023-10-16T10:09:13.051193+0000","last_peered":"2023-10-16T10:09:13.051193+0000","last_clean":"2023-10-16T10:09:13.051193+0000","last_became_active":"2023-10-16T10:09:11.225343+0000","last_became_peered":"2023-10-16T10:09:11.225343+0000","last_unstale":"2023-10-16T10:09:13.051193+0000","last_undegraded":"2023-10-16T10:09:13.051193+0000","last_fullsized":"2023-10-16T10:09:13.051193+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:32:26.308547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.249103+0000","last_change":"2023-10-16T10:09:10.207380+0000","last_active":"2023-10-16T10:09:11.249103+0000","last_peered":"2023-10-16T10:09:11.249103+0000","last_clean":"2023-10-16T10:09:11.249103+0000","last_became_active":"2023-10-16T10:09:10.207009+0000","last_became_peered":"2023-10-16T10:09:10.207009+0000","last_unstale":"2023-10-16T10:09:11.249103+0000","last_undegraded":"2023-10-16T10:09:11.249103+0000","last_fullsized":"2023-10-16T10:09:11.249103+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:58.349031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.207579+0000","last_change":"2023-10-16T10:09:11.232563+0000","last_active":"2023-10-16T10:09:12.207579+0000","last_peered":"2023-10-16T10:09:12.207579+0000","last_clean":"2023-10-16T10:09:12.207579+0000","last_became_active":"2023-10-16T10:09:11.228149+0000","last_became_peered":"2023-10-16T10:09:11.228149+0000","last_unstale":"2023-10-16T10:09:12.207579+0000","last_undegraded":"2023-10-16T10:09:12.207579+0000","last_fullsized":"2023-10-16T10:09:12.207579+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:14:29.869731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.217587+0000","last_change":"2023-10-16T10:09:10.210075+0000","last_active":"2023-10-16T10:09:11.217587+0000","last_peered":"2023-10-16T10:09:11.217587+0000","last_clean":"2023-10-16T10:09:11.217587+0000","last_became_active":"2023-10-16T10:09:10.207818+0000","last_became_peered":"2023-10-16T10:09:10.207818+0000","last_unstale":"2023-10-16T10:09:11.217587+0000","last_undegraded":"2023-10-16T10:09:11.217587+0000","last_fullsized":"2023-10-16T10:09:11.217587+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:57:15.139270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208062+0000","last_change":"2023-10-16T10:09:11.239886+0000","last_active":"2023-10-16T10:09:12.208062+0000","last_peered":"2023-10-16T10:09:12.208062+0000","last_clean":"2023-10-16T10:09:12.208062+0000","last_became_active":"2023-10-16T10:09:11.237590+0000","last_became_peered":"2023-10-16T10:09:11.237590+0000","last_unstale":"2023-10-16T10:09:12.208062+0000","last_undegraded":"2023-10-16T10:09:12.208062+0000","last_fullsized":"2023-10-16T10:09:12.208062+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:27.467556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.223559+0000","last_change":"2023-10-16T10:09:10.211471+0000","last_active":"2023-10-16T10:09:11.223559+0000","last_peered":"2023-10-16T10:09:11.223559+0000","last_clean":"2023-10-16T10:09:11.223559+0000","last_became_active":"2023-10-16T10:09:10.209919+0000","last_became_peered":"2023-10-16T10:09:10.209919+0000","last_unstale":"2023-10-16T10:09:11.223559+0000","last_undegraded":"2023-10-16T10:09:11.223559+0000","last_fullsized":"2023-10-16T10:09:11.223559+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:28:34.969804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.049166+0000","last_change":"2023-10-16T10:09:11.210524+0000","last_active":"2023-10-16T10:09:13.049166+0000","last_peered":"2023-10-16T10:09:13.049166+0000","last_clean":"2023-10-16T10:09:13.049166+0000","last_became_active":"2023-10-16T10:09:11.209945+0000","last_became_peered":"2023-10-16T10:09:11.209945+0000","last_unstale":"2023-10-16T10:09:13.049166+0000","last_undegraded":"2023-10-16T10:09:13.049166+0000","last_fullsized":"2023-10-16T10:09:13.049166+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:08:32.027249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.226366+0000","last_change":"2023-10-16T10:09:10.201750+0000","last_active":"2023-10-16T10:09:11.226366+0000","last_peered":"2023-10-16T10:09:11.226366+0000","last_clean":"2023-10-16T10:09:11.226366+0000","last_became_active":"2023-10-16T10:09:10.201454+0000","last_became_peered":"2023-10-16T10:09:10.201454+0000","last_unstale":"2023-10-16T10:09:11.226366+0000","last_undegraded":"2023-10-16T10:09:11.226366+0000","last_fullsized":"2023-10-16T10:09:11.226366+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:10:45.492873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.049956+0000","last_change":"2023-10-16T10:09:11.233467+0000","last_active":"2023-10-16T10:09:13.049956+0000","last_peered":"2023-10-16T10:09:13.049956+0000","last_clean":"2023-10-16T10:09:13.049956+0000","last_became_active":"2023-10-16T10:09:11.232187+0000","last_became_peered":"2023-10-16T10:09:11.232187+0000","last_unstale":"2023-10-16T10:09:13.049956+0000","last_undegraded":"2023-10-16T10:09:13.049956+0000","last_fullsized":"2023-10-16T10:09:13.049956+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:35:51.291548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.212388+0000","last_change":"2023-10-16T10:09:10.210941+0000","last_active":"2023-10-16T10:09:11.212388+0000","last_peered":"2023-10-16T10:09:11.212388+0000","last_clean":"2023-10-16T10:09:11.212388+0000","last_became_active":"2023-10-16T10:09:10.209812+0000","last_became_peered":"2023-10-16T10:09:10.209812+0000","last_unstale":"2023-10-16T10:09:11.212388+0000","last_undegraded":"2023-10-16T10:09:11.212388+0000","last_fullsized":"2023-10-16T10:09:11.212388+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:18:28.997634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.048333+0000","last_change":"2023-10-16T10:09:11.245636+0000","last_active":"2023-10-16T10:09:13.048333+0000","last_peered":"2023-10-16T10:09:13.048333+0000","last_clean":"2023-10-16T10:09:13.048333+0000","last_became_active":"2023-10-16T10:09:11.245304+0000","last_became_peered":"2023-10-16T10:09:11.245304+0000","last_unstale":"2023-10-16T10:09:13.048333+0000","last_undegraded":"2023-10-16T10:09:13.048333+0000","last_fullsized":"2023-10-16T10:09:13.048333+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:47:03.765628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.214997+0000","last_change":"2023-10-16T10:09:10.206311+0000","last_active":"2023-10-16T10:09:11.214997+0000","last_peered":"2023-10-16T10:09:11.214997+0000","last_clean":"2023-10-16T10:09:11.214997+0000","last_became_active":"2023-10-16T10:09:10.206018+0000","last_became_peered":"2023-10-16T10:09:10.206018+0000","last_unstale":"2023-10-16T10:09:11.214997+0000","last_undegraded":"2023-10-16T10:09:11.214997+0000","last_fullsized":"2023-10-16T10:09:11.214997+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:36:38.202444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.047343+0000","last_change":"2023-10-16T10:09:11.234973+0000","last_active":"2023-10-16T10:09:13.047343+0000","last_peered":"2023-10-16T10:09:13.047343+0000","last_clean":"2023-10-16T10:09:13.047343+0000","last_became_active":"2023-10-16T10:09:11.234617+0000","last_became_peered":"2023-10-16T10:09:11.234617+0000","last_unstale":"2023-10-16T10:09:13.047343+0000","last_undegraded":"2023-10-16T10:09:13.047343+0000","last_fullsized":"2023-10-16T10:09:13.047343+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:39:27.178785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.230771+0000","last_change":"2023-10-16T10:09:10.208508+0000","last_active":"2023-10-16T10:09:11.230771+0000","last_peered":"2023-10-16T10:09:11.230771+0000","last_clean":"2023-10-16T10:09:11.230771+0000","last_became_active":"2023-10-16T10:09:10.208224+0000","last_became_peered":"2023-10-16T10:09:10.208224+0000","last_unstale":"2023-10-16T10:09:11.230771+0000","last_undegraded":"2023-10-16T10:09:11.230771+0000","last_fullsized":"2023-10-16T10:09:11.230771+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:13.452692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208208+0000","last_change":"2023-10-16T10:09:11.229079+0000","last_active":"2023-10-16T10:09:12.208208+0000","last_peered":"2023-10-16T10:09:12.208208+0000","last_clean":"2023-10-16T10:09:12.208208+0000","last_became_active":"2023-10-16T10:09:11.228591+0000","last_became_peered":"2023-10-16T10:09:11.228591+0000","last_unstale":"2023-10-16T10:09:12.208208+0000","last_undegraded":"2023-10-16T10:09:12.208208+0000","last_fullsized":"2023-10-16T10:09:12.208208+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:37.403579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.223744+0000","last_change":"2023-10-16T10:09:10.200708+0000","last_active":"2023-10-16T10:09:11.223744+0000","last_peered":"2023-10-16T10:09:11.223744+0000","last_clean":"2023-10-16T10:09:11.223744+0000","last_became_active":"2023-10-16T10:09:10.200402+0000","last_became_peered":"2023-10-16T10:09:10.200402+0000","last_unstale":"2023-10-16T10:09:11.223744+0000","last_undegraded":"2023-10-16T10:09:11.223744+0000","last_fullsized":"2023-10-16T10:09:11.223744+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:36:06.012479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.049733+0000","last_change":"2023-10-16T10:09:11.245035+0000","last_active":"2023-10-16T10:09:13.049733+0000","last_peered":"2023-10-16T10:09:13.049733+0000","last_clean":"2023-10-16T10:09:13.049733+0000","last_became_active":"2023-10-16T10:09:11.244687+0000","last_became_peered":"2023-10-16T10:09:11.244687+0000","last_unstale":"2023-10-16T10:09:13.049733+0000","last_undegraded":"2023-10-16T10:09:13.049733+0000","last_fullsized":"2023-10-16T10:09:13.049733+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:31:04.265205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.249525+0000","last_change":"2023-10-16T10:09:10.210460+0000","last_active":"2023-10-16T10:09:11.249525+0000","last_peered":"2023-10-16T10:09:11.249525+0000","last_clean":"2023-10-16T10:09:11.249525+0000","last_became_active":"2023-10-16T10:09:10.209167+0000","last_became_peered":"2023-10-16T10:09:10.209167+0000","last_unstale":"2023-10-16T10:09:11.249525+0000","last_undegraded":"2023-10-16T10:09:11.249525+0000","last_fullsized":"2023-10-16T10:09:11.249525+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:40.618287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"140'2668","reported_seq":3209,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.213773+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T10:09:11.213773+0000","last_peered":"2023-10-16T10:09:11.213773+0000","last_clean":"2023-10-16T10:09:11.213773+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T10:09:11.213773+0000","last_undegraded":"2023-10-16T10:09:11.213773+0000","last_fullsized":"2023-10-16T10:09:11.213773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2668,"log_dups_size":0,"ondisk_log_size":2668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1312,"num_read_kb":2614,"num_write":401,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1312,"num_read_kb":2614,"num_write":401,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2668,"ondisk_log_size":2668,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575087,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33624,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338216,"statfs":{"total":96636764160,"available":96602333184,"internally_reserved":0,"allocated":2498560,"data_stored":1492893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.1100000000000001,"15min":1.1599999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.5960000000000001,"5min":1.6599999999999999,"15min":1.925},"last":1.425},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.1279999999999999,"15min":1.177},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.33600000000000002},"max":{"1min":1.5109999999999999,"5min":1.722,"15min":2.016},"last":1.2809999999999999}]},{"osd":1,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.177,"15min":1.2190000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.7749999999999999,"5min":1.788,"15min":1.923},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1439999999999999,"15min":1.2090000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.92600000000000005}]},{"osd":2,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1930000000000001,"15min":1.232},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.72,"5min":1.8109999999999999,"15min":1.8740000000000001},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.2589999999999999,"15min":1.28},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.64500000000000002},"max":{"1min":1.7709999999999999,"5min":1.8169999999999999,"15min":1.8620000000000001},"last":1.3740000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3089999999999999,"15min":1.335},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.67700000000000005},"max":{"1min":1.8700000000000001,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.302,"15min":1.351},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.603,"5min":1.8540000000000001,"15min":2.004},"last":0.84099999999999997}]},{"osd":4,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.3779999999999999,"15min":1.3620000000000001},"min":{"1min":0.88900000000000001,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.7569999999999999,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3169999999999999,"15min":1.3120000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.8360000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.5649999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.492,"15min":1.5109999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.8360000000000001,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.125},{"interface":"front","average":{"1min":1.411,"5min":1.4159999999999999,"15min":1.4239999999999999},"min":{"1min":0.84299999999999997,"5min":0.80400000000000005,"15min":0.45400000000000001},"max":{"1min":1.7649999999999999,"5min":1.9630000000000001,"15min":2.2170000000000001},"last":1.6659999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.4350000000000001,"15min":1.4039999999999999},"min":{"1min":0.85299999999999998,"5min":0.83599999999999997,"15min":0.54100000000000004},"max":{"1min":1.9810000000000001,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.605},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.615,"15min":1.611},"min":{"1min":0.80900000000000005,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0230000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.53}]}]},{"osd":6,"up_from":13,"seq":55834575087,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41356,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330484,"statfs":{"total":96636764160,"available":96594415616,"internally_reserved":0,"allocated":1892352,"data_stored":901887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.131,"15min":1.143},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.619,"5min":1.736,"15min":1.7689999999999999},"last":1.8009999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.216,"15min":1.228},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5329999999999999,"5min":1.7010000000000001,"15min":1.847},"last":1.1850000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1459999999999999,"15min":1.1659999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.52},{"interface":"front","average":{"1min":1.171,"5min":1.278,"15min":1.286},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6539999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.3560000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.1879999999999999,"15min":1.1990000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45100000000000001},"max":{"1min":1.4710000000000001,"5min":1.851,"15min":2.036},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.248,"15min":1.296},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.58999999999999997},"max":{"1min":1.6859999999999999,"5min":1.925,"15min":1.925},"last":1.7230000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1579999999999999,"15min":1.202},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":2.8239999999999998},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.1759999999999999,"15min":1.218},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5049999999999999,"5min":1.6719999999999999,"15min":1.865},"last":1.125}]},{"osd":4,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.28,"15min":1.2490000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.46000000000000002},"max":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":2.1629999999999998},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.423,"15min":1.4239999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.9730000000000001,"5min":2,"15min":3.6880000000000002},"last":1.476}]},{"osd":5,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3080000000000001,"15min":1.3160000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.38900000000000001},"max":{"1min":1.647,"5min":2.0790000000000002,"15min":2.5649999999999999},"last":1.268},{"interface":"front","average":{"1min":1.45,"5min":1.5,"15min":1.502},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.495},"max":{"1min":1.8440000000000001,"5min":1.996,"15min":2.3849999999999998},"last":1.865}]},{"osd":7,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.478,"15min":1.47},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":3.5739999999999998},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.512,"15min":1.488},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.4249999999999998},"last":1.679}]}]},{"osd":5,"up_from":13,"seq":55834575087,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41344,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330496,"statfs":{"total":96636764160,"available":96594427904,"internally_reserved":0,"allocated":1888256,"data_stored":900387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1830000000000001,"15min":1.1970000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.496},"max":{"1min":1.5149999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.036},{"interface":"front","average":{"1min":1.23,"5min":1.226,"15min":1.276},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.6830000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.98299999999999998}]},{"osd":1,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.095,"15min":1.147},"min":{"1min":0.63600000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3500000000000001,"5min":1.7310000000000001,"15min":1.7729999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.105,"5min":1.143,"15min":1.1399999999999999},"min":{"1min":0.66000000000000003,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.1850000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.161,"15min":1.163},"min":{"1min":0.78700000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.218},{"interface":"front","average":{"1min":1.236,"5min":1.214,"15min":1.238},"min":{"1min":0.82199999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.653,"5min":1.6739999999999999,"15min":1.8109999999999999},"last":1.3500000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.2370000000000001,"15min":1.2629999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5840000000000001,"5min":1.915,"15min":1.97},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.3049999999999999,"15min":1.3440000000000001},"min":{"1min":0.88100000000000001,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6850000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.5089999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.276,"15min":1.2709999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.282},{"interface":"front","average":{"1min":1.468,"5min":1.5,"15min":1.498},"min":{"1min":0.878,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.9490000000000001,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.3939999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.335,"15min":1.359},"min":{"1min":0.96099999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.7649999999999999,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.405,"15min":1.3839999999999999},"min":{"1min":0.83599999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.4410000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.474,"15min":1.4670000000000001},"min":{"1min":1.0029999999999999,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.849,"5min":2.1400000000000001,"15min":2.1989999999999998},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.581,"5min":1.5509999999999999,"15min":1.5669999999999999},"min":{"1min":1.032,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.0099999999999998,"5min":2.234,"15min":2.234},"last":1.7789999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575088,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46908,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324932,"statfs":{"total":96636764160,"available":96588730368,"internally_reserved":0,"allocated":1884160,"data_stored":900345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.093,"15min":1.139},"min":{"1min":0.69399999999999995,"5min":0.64300000000000002,"15min":0.38500000000000001},"max":{"1min":1.792,"5min":1.792,"15min":1.827},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.2150000000000001,"15min":1.264},"min":{"1min":0.64700000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.712,"5min":1.9359999999999999,"15min":11.974},"last":0.79000000000000004}]},{"osd":1,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.23,"15min":1.2909999999999999},"min":{"1min":0.752,"5min":0.626,"15min":0.626},"max":{"1min":1.526,"5min":1.8939999999999999,"15min":9.9939999999999998},"last":1.28},{"interface":"front","average":{"1min":1.113,"5min":1.1359999999999999,"15min":1.216},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.655,"5min":1.655,"15min":5.9420000000000002},"last":1.4410000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.244,"15min":1.25},"min":{"1min":0.78100000000000003,"5min":0.61799999999999999,"15min":0.59399999999999997},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.304,"15min":1.371},"min":{"1min":0.76000000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.4650000000000001,"5min":2.1219999999999999,"15min":5.0549999999999997},"last":1.121}]},{"osd":3,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.2569999999999999,"15min":1.2989999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.849,"5min":1.849,"15min":6.8079999999999998},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.361,"15min":1.3759999999999999},"min":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.67700000000000005},"max":{"1min":1.6890000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.643}]},{"osd":5,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.401,"15min":1.431},"min":{"1min":0.91300000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.647,"5min":2.1539999999999999,"15min":8.702},"last":1.375},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.357,"15min":1.3680000000000001},"min":{"1min":0.90500000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.675,"5min":2.331,"15min":3.0430000000000001},"last":1.5549999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.431,"15min":1.425},"min":{"1min":1.1080000000000001,"5min":0.84699999999999998,"15min":0.79000000000000004},"max":{"1min":1.8029999999999999,"5min":2.121,"15min":8.0579999999999998},"last":1.738},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.5429999999999999,"15min":1.5489999999999999},"min":{"1min":1.093,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.0609999999999999,"5min":2.2549999999999999,"15min":6.4089999999999998},"last":1.601}]},{"osd":7,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.488,"15min":1.4890000000000001},"min":{"1min":1.0649999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.742,"5min":2.3410000000000002,"15min":2.9399999999999999},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.554,"5min":1.5680000000000001,"15min":1.5700000000000001},"min":{"1min":1.1879999999999999,"5min":1.071,"15min":0.96799999999999997},"max":{"1min":2.004,"5min":2.2690000000000001,"15min":2.6120000000000001},"last":1.804}]}]},{"osd":3,"up_from":13,"seq":55834575087,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41344,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330496,"statfs":{"total":96636764160,"available":96594427904,"internally_reserved":0,"allocated":1888256,"data_stored":900435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:08:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81000000000000005,"15min":0.84399999999999997},"min":{"1min":0.434,"5min":0.30299999999999999,"15min":0.24399999999999999},"max":{"1min":1.6040000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.90000000000000002,"15min":0.98499999999999999},"min":{"1min":0.35299999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.82799999999999996}]},{"osd":1,"last update":"Mon Oct 16 10:08:35 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.83999999999999997,"15min":0.91300000000000003},"min":{"1min":0.48899999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.282,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.93000000000000005,"15min":1.0229999999999999},"min":{"1min":0.41599999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.77,"5min":3.363,"15min":3.363},"last":0.54000000000000004}]},{"osd":2,"last update":"Mon Oct 16 10:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.98099999999999998,"15min":1.042},"min":{"1min":0.432,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.907,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.04,"5min":1.036,"15min":1.141},"min":{"1min":0.58299999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.512,"5min":4.1230000000000002,"15min":4.1230000000000002},"last":0.88900000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.2030000000000001,"15min":1.2549999999999999},"min":{"1min":0.64500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.6739999999999999,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.256,"5min":1.2649999999999999,"15min":1.3080000000000001},"min":{"1min":0.72799999999999998,"5min":0.56499999999999995,"15min":0.54700000000000004},"max":{"1min":2.4249999999999998,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":1.024}]},{"osd":5,"last update":"Mon Oct 16 10:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.2450000000000001,"15min":1.2989999999999999},"min":{"1min":0.69799999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.7809999999999999,"5min":4.1829999999999998,"15min":4.1829999999999998},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2390000000000001,"15min":1.3109999999999999},"min":{"1min":0.67600000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.198,"5min":2.198,"15min":2.3140000000000001},"last":1.4430000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.359,"15min":1.4039999999999999},"min":{"1min":0.874,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.9199999999999999,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":0.998},{"interface":"front","average":{"1min":1.375,"5min":1.353,"15min":1.4179999999999999},"min":{"1min":0.81000000000000005,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.1030000000000002,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.401}]},{"osd":7,"last update":"Mon Oct 16 10:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.375,"15min":1.419},"min":{"1min":0.92100000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.7269999999999999,"5min":3.649,"15min":3.649},"last":1.147},{"interface":"front","average":{"1min":1.464,"5min":1.474,"15min":1.5329999999999999},"min":{"1min":0.88100000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":3.0710000000000002,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":1.3149999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575086,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59396,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312444,"statfs":{"total":96636764160,"available":96575942656,"internally_reserved":0,"allocated":1888256,"data_stored":900891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:08:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.875,"15min":0.874},"min":{"1min":0.432,"5min":0.35299999999999998,"15min":0.26500000000000001},"max":{"1min":1.1930000000000001,"5min":5.1829999999999998,"15min":5.1829999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.93899999999999995,"15min":1.0049999999999999},"min":{"1min":0.38900000000000001,"5min":0.35999999999999999,"15min":0.34999999999999998},"max":{"1min":1.3959999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.476}]},{"osd":1,"last update":"Mon Oct 16 10:08:38 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.024,"15min":1.0349999999999999},"min":{"1min":0.48999999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.407,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.997,"15min":1.077},"min":{"1min":0.441,"5min":0.38300000000000001,"15min":0.35499999999999998},"max":{"1min":1.331,"5min":1.4770000000000001,"15min":1.792},"last":1.6220000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:08:38 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.91200000000000003,"15min":0.99399999999999999},"min":{"1min":0.48399999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.776},"last":1.411},{"interface":"front","average":{"1min":1.077,"5min":1.139,"15min":1.175},"min":{"1min":0.46200000000000002,"5min":0.45500000000000002,"15min":0.41899999999999998},"max":{"1min":1.6910000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":1.288}]},{"osd":4,"last update":"Mon Oct 16 10:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3089999999999999,"15min":1.329},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.40300000000000002},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.153,"5min":1.238,"15min":1.29},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.46000000000000002},"max":{"1min":1.4650000000000001,"5min":3.262,"15min":3.262},"last":1.5629999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.387,"15min":1.381},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.59399999999999997},"max":{"1min":2.2919999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.661},{"interface":"front","average":{"1min":1.258,"5min":1.319,"15min":1.3759999999999999},"min":{"1min":0.76300000000000001,"5min":0.72699999999999998,"15min":0.60999999999999999},"max":{"1min":1.823,"5min":1.885,"15min":1.956},"last":1.821}]},{"osd":6,"last update":"Mon Oct 16 10:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2809999999999999,"15min":1.3260000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.7290000000000001,"5min":3.169,"15min":3.169},"last":1.7569999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.4139999999999999,"15min":1.4339999999999999},"min":{"1min":0.69599999999999995,"5min":0.63,"15min":0.63},"max":{"1min":2.177,"5min":3.8340000000000001,"15min":3.8340000000000001},"last":1.593}]},{"osd":7,"last update":"Mon Oct 16 10:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.401,"15min":1.4530000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.67000000000000004},"max":{"1min":1.6479999999999999,"5min":2.012,"15min":2.262},"last":1.9570000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3879999999999999,"15min":1.4199999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.948,"5min":3.3780000000000001,"15min":3.3780000000000001},"last":1.879}]}]},{"osd":1,"up_from":13,"seq":55834575086,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55228,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316612,"statfs":{"total":96636764160,"available":96580210688,"internally_reserved":0,"allocated":1880064,"data_stored":900275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.77500000000000002,"15min":0.84399999999999997},"min":{"1min":0.34399999999999997,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":1.0309999999999999,"5min":1.4690000000000001,"15min":1.631},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.92500000000000004,"15min":0.97899999999999998},"min":{"1min":0.38200000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.5509999999999999,"5min":1.6200000000000001,"15min":1.798},"last":0.60999999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.90100000000000002,"15min":0.96299999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2529999999999999,"5min":1.5069999999999999,"15min":1.673},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.95199999999999996,"5min":1.0640000000000001,"15min":1.1930000000000001},"min":{"1min":0.56000000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3919999999999999,"5min":1.609,"15min":1.7669999999999999},"last":0.64400000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.0009999999999999,"15min":1.097},"min":{"1min":0.51800000000000002,"5min":0.42999999999999999,"15min":0.42599999999999999},"max":{"1min":1.4550000000000001,"5min":1.798,"15min":1.798},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0229999999999999,"15min":1.0920000000000001},"min":{"1min":0.47599999999999998,"5min":0.44,"15min":0.40699999999999997},"max":{"1min":1.5489999999999999,"5min":1.6879999999999999,"15min":1.8280000000000001},"last":0.91800000000000004}]},{"osd":4,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1599999999999999,"15min":1.212},"min":{"1min":0.69099999999999995,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5860000000000001,"5min":1.7450000000000001,"15min":1.885},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.2689999999999999,"15min":1.341},"min":{"1min":0.64700000000000002,"5min":0.63200000000000001,"15min":0.60999999999999999},"max":{"1min":1.605,"5min":1.8680000000000001,"15min":2.1320000000000001},"last":1.2909999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2589999999999999,"15min":1.3180000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.528},{"interface":"front","average":{"1min":1.232,"5min":1.238,"15min":1.302},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.637,"5min":1.7290000000000001,"15min":2.0470000000000002},"last":1.369}]},{"osd":6,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.3169999999999999,"15min":1.381},"min":{"1min":0.72099999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.349},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.3089999999999999,"15min":1.369},"min":{"1min":0.68600000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.762,"5min":1.8779999999999999,"15min":2.1680000000000001},"last":1.4039999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.331,"15min":1.4330000000000001},"min":{"1min":0.73799999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.6950000000000001,"5min":2.0739999999999998,"15min":2.1699999999999999},"last":1.498},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4319999999999999,"15min":1.484},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.2709999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575087,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47492,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324348,"statfs":{"total":96636764160,"available":96588132352,"internally_reserved":0,"allocated":2473984,"data_stored":1490643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.80800000000000005,"15min":0.84799999999999998},"min":{"1min":0.35599999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.532,"5min":1.7589999999999999,"15min":2.766},"last":1.01},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.92900000000000005,"15min":1.0129999999999999},"min":{"1min":0.51600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.228,"5min":1.581,"15min":3.96},"last":0.73999999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.97299999999999998,"15min":1.028},"min":{"1min":0.54200000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.278,"5min":2.0449999999999999,"15min":2.9260000000000002},"last":0.87},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.98999999999999999,"15min":1.0660000000000001},"min":{"1min":0.47899999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.52,"5min":1.7789999999999999,"15min":4.4290000000000003},"last":1.141}]},{"osd":3,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.91100000000000003,"15min":1.012},"min":{"1min":0.5,"5min":0.41999999999999998,"15min":0.38},"max":{"1min":1.585,"5min":1.5920000000000001,"15min":4.7380000000000004},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.103,"15min":1.1859999999999999},"min":{"1min":0.51200000000000001,"5min":0.46800000000000003,"15min":0.44},"max":{"1min":1.3959999999999999,"5min":2.1859999999999999,"15min":5.1500000000000004},"last":1.3089999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2609999999999999,"15min":1.3200000000000001},"min":{"1min":0.83199999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5760000000000001,"5min":1.859,"15min":4.1970000000000001},"last":1.464},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1910000000000001,"15min":1.276},"min":{"1min":0.83899999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.3999999999999999,"5min":1.74,"15min":4.9630000000000001},"last":1.3660000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.202,"15min":1.296},"min":{"1min":0.877,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6799999999999999,"5min":1.6890000000000001,"15min":4.8600000000000003},"last":1.569},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3300000000000001,"15min":1.385},"min":{"1min":0.76500000000000001,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6299999999999999,"5min":2.1269999999999998,"15min":4.6779999999999999},"last":1.202}]},{"osd":6,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.407,"15min":1.4330000000000001},"min":{"1min":0.93000000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.7390000000000001,"5min":2.1030000000000002,"15min":3.8250000000000002},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.339,"15min":1.4139999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.782,"5min":2.02,"15min":4.0979999999999999},"last":1.252}]},{"osd":7,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.4179999999999999,"15min":1.4710000000000001},"min":{"1min":0.79900000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.8600000000000001,"5min":2.0750000000000002,"15min":4.3369999999999997},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.3089999999999999,"15min":1.417},"min":{"1min":0.86299999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4650000000000001,"5min":1.9670000000000001,"15min":5.0570000000000004},"last":1.5049999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:09:15.669 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:09:15.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.923+0000 7f03eda56640 1 Processor -- start 2023-10-16T10:09:15.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.923+0000 7f03eda56640 1 -- start start 2023-10-16T10:09:15.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.924+0000 7f03eda56640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03e81593b0 0x7f03e806f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:15.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.924+0000 7f03eda56640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f03e80caa10 0x7f03e806fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:15.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.924+0000 7f03eda56640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e8159790 con 0x7f03e806ffd0 2023-10-16T10:09:15.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.924+0000 7f03eda56640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e80c9b20 con 0x7f03e80caa10 2023-10-16T10:09:15.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.924+0000 7f03eda56640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e80c9e00 con 0x7f03e81593b0 2023-10-16T10:09:15.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.925+0000 7f03e77fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f03e806ffd0 0x7f03e815ceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35382/0 (socket says 172.21.15.84:35382) 2023-10-16T10:09:15.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.925+0000 7f03e67fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f03e80caa10 0x7f03e806fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:15.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.925+0000 7f03e77fe640 1 -- 172.21.15.84:0/3739702744 learned_addr learned my addr 172.21.15.84:0/3739702744 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:09:15.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.925+0000 7f03e6ffd640 1 --2- 172.21.15.84:0/3739702744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03e81593b0 0x7f03e806f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:15.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.925+0000 7f03e5ffb640 1 -- 172.21.15.84:0/3739702744 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2265063315 0 0) 0x7f03e8159790 con 0x7f03e806ffd0 2023-10-16T10:09:15.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.926+0000 7f03e5ffb640 1 -- 172.21.15.84:0/3739702744 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03d4005730 con 0x7f03e806ffd0 2023-10-16T10:09:15.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.926+0000 7f03e67fc640 1 -- 172.21.15.84:0/3739702744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03e81593b0 msgr2=0x7f03e806f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.926+0000 7f03e67fc640 1 --2- 172.21.15.84:0/3739702744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03e81593b0 0x7f03e806f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.926+0000 7f03e67fc640 1 -- 172.21.15.84:0/3739702744 >> v1:172.21.15.84:6789/0 conn(0x7f03e806ffd0 legacy=0x7f03e815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.926+0000 7f03e67fc640 1 -- 172.21.15.84:0/3739702744 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e80ca0e0 con 0x7f03e80caa10 2023-10-16T10:09:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.927+0000 7f03e67fc640 1 --2- 172.21.15.84:0/3739702744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e80caa10 0x7f03e806fa90 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f03d800aea0 tx=0x7f03d802bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90152b667b565e77 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.927+0000 7f03e5ffb640 1 -- 172.21.15.84:0/3739702744 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d802c910 con 0x7f03e80caa10 2023-10-16T10:09:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.927+0000 7f03e5ffb640 1 -- 172.21.15.84:0/3739702744 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d802ca70 con 0x7f03e80caa10 2023-10-16T10:09:15.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.927+0000 7f03e5ffb640 1 -- 172.21.15.84:0/3739702744 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d802ce00 con 0x7f03e80caa10 2023-10-16T10:09:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.927+0000 7f03eda56640 1 -- 172.21.15.84:0/3739702744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e80caa10 msgr2=0x7f03e806fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.927+0000 7f03eda56640 1 --2- 172.21.15.84:0/3739702744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e80caa10 0x7f03e806fa90 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f03d800aea0 tx=0x7f03d802bd70 comp rx=0 tx=0).stop 2023-10-16T10:09:15.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.928+0000 7f03eda56640 1 -- 172.21.15.84:0/3739702744 shutdown_connections 2023-10-16T10:09:15.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.928+0000 7f03eda56640 1 --2- 172.21.15.84:0/3739702744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e80caa10 0x7f03e806fa90 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:15.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.928+0000 7f03eda56640 1 --2- 172.21.15.84:0/3739702744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03e81593b0 0x7f03e806f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:15.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.928+0000 7f03eda56640 1 -- 172.21.15.84:0/3739702744 >> 172.21.15.84:0/3739702744 conn(0x7f03e80cb410 msgr2=0x7f03e80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:15.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.928+0000 7f03eda56640 1 -- 172.21.15.84:0/3739702744 shutdown_connections 2023-10-16T10:09:15.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.928+0000 7f03eda56640 1 -- 172.21.15.84:0/3739702744 wait complete. 2023-10-16T10:09:15.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.929+0000 7f03eda56640 1 Processor -- start 2023-10-16T10:09:15.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.929+0000 7f03eda56640 1 -- start start 2023-10-16T10:09:15.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.930+0000 7f03eda56640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03e806ffd0 0x7f03e814d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:15.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.930+0000 7f03eda56640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e80caa10 0x7f03e8153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:15.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.930+0000 7f03eda56640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e81593b0 0x7f03e80ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:15.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.930+0000 7f03eda56640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e8159ce0 con 0x7f03e81593b0 2023-10-16T10:09:15.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.930+0000 7f03eda56640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e814dac0 con 0x7f03e80caa10 2023-10-16T10:09:15.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.930+0000 7f03eda56640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e8159990 con 0x7f03e806ffd0 2023-10-16T10:09:15.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.931+0000 7f03e77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e81593b0 0x7f03e80ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:15.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.931+0000 7f03e77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e81593b0 0x7f03e80ae620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39910/0 (socket says 172.21.15.84:39910) 2023-10-16T10:09:15.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.931+0000 7f03e77fe640 1 -- 172.21.15.84:0/3134660484 learned_addr learned my addr 172.21.15.84:0/3134660484 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:15.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.931+0000 7f03e6ffd640 1 --2- 172.21.15.84:0/3134660484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03e806ffd0 0x7f03e814d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:15.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.931+0000 7f03e67fc640 1 --2- 172.21.15.84:0/3134660484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e80caa10 0x7f03e8153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:15.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.931+0000 7f03e77fe640 1 -- 172.21.15.84:0/3134660484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03e806ffd0 msgr2=0x7f03e814d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.931+0000 7f03e77fe640 1 --2- 172.21.15.84:0/3134660484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03e806ffd0 0x7f03e814d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.931+0000 7f03e77fe640 1 -- 172.21.15.84:0/3134660484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e80caa10 msgr2=0x7f03e8153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.931+0000 7f03e77fe640 1 --2- 172.21.15.84:0/3134660484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e80caa10 0x7f03e8153bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.931+0000 7f03e77fe640 1 -- 172.21.15.84:0/3134660484 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d800ab80 con 0x7f03e81593b0 2023-10-16T10:09:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.931+0000 7f03e6ffd640 1 --2- 172.21.15.84:0/3134660484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03e806ffd0 0x7f03e814d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.931+0000 7f03e77fe640 1 --2- 172.21.15.84:0/3134660484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e81593b0 0x7f03e80ae620 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f03e000c390 tx=0x7f03e000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.932+0000 7f03dffff640 1 -- 172.21.15.84:0/3134660484 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0005ce0 con 0x7f03e81593b0 2023-10-16T10:09:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.932+0000 7f03dffff640 1 -- 172.21.15.84:0/3134660484 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e0005e40 con 0x7f03e81593b0 2023-10-16T10:09:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.932+0000 7f03dffff640 1 -- 172.21.15.84:0/3134660484 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0007310 con 0x7f03e81593b0 2023-10-16T10:09:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.932+0000 7f03eda56640 1 -- 172.21.15.84:0/3134660484 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e80c7890 con 0x7f03e81593b0 2023-10-16T10:09:15.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.932+0000 7f03eda56640 1 -- 172.21.15.84:0/3134660484 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e80c7e50 con 0x7f03e81593b0 2023-10-16T10:09:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.933+0000 7f03eda56640 1 -- 172.21.15.84:0/3134660484 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e8159e20 con 0x7f03e81593b0 2023-10-16T10:09:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.934+0000 7f03dffff640 1 -- 172.21.15.84:0/3134660484 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f03e0007470 con 0x7f03e81593b0 2023-10-16T10:09:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.935+0000 7f03dffff640 1 --2- 172.21.15.84:0/3134660484 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f03c806af90 0x7f03c806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.935+0000 7f03dffff640 1 -- 172.21.15.84:0/3134660484 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f03e0088de0 con 0x7f03e81593b0 2023-10-16T10:09:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.935+0000 7f03e6ffd640 1 --2- 172.21.15.84:0/3134660484 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f03c806af90 0x7f03c806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:15.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.936+0000 7f03e6ffd640 1 --2- 172.21.15.84:0/3134660484 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f03c806af90 0x7f03c806d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f03d0006fd0 tx=0x7f03d0008040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:15.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:15.939+0000 7f03dffff640 1 -- 172.21.15.84:0/3134660484 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f03e8159e20 con 0x7f03e81593b0 2023-10-16T10:09:16.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.125+0000 7f03eda56640 1 -- 172.21.15.84:0/3134660484 --> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f03e80ca3c0 con 0x7f03c806af90 2023-10-16T10:09:16.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.140+0000 7f03dffff640 1 -- 172.21.15.84:0/3134660484 <== mgr.8904 v2:172.21.15.84:6832/205480762 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287270 (secure 0 0 0) 0x7f03d0009e10 con 0x7f03c806af90 2023-10-16T10:09:16.144 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:09:16.147 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T10:09:16.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.146+0000 7f03eda56640 1 -- 172.21.15.84:0/3134660484 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f03c806af90 msgr2=0x7f03c806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:16.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.146+0000 7f03eda56640 1 --2- 172.21.15.84:0/3134660484 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f03c806af90 0x7f03c806d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f03d0006fd0 tx=0x7f03d0008040 comp rx=0 tx=0).stop 2023-10-16T10:09:16.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.146+0000 7f03eda56640 1 -- 172.21.15.84:0/3134660484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e81593b0 msgr2=0x7f03e80ae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:16.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.146+0000 7f03eda56640 1 --2- 172.21.15.84:0/3134660484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e81593b0 0x7f03e80ae620 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f03e000c390 tx=0x7f03e000c850 comp rx=0 tx=0).stop 2023-10-16T10:09:16.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.146+0000 7f03eda56640 1 -- 172.21.15.84:0/3134660484 shutdown_connections 2023-10-16T10:09:16.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.146+0000 7f03eda56640 1 --2- 172.21.15.84:0/3134660484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e81593b0 0x7f03e80ae620 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.146+0000 7f03eda56640 1 --2- 172.21.15.84:0/3134660484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e80caa10 0x7f03e8153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.146+0000 7f03eda56640 1 --2- 172.21.15.84:0/3134660484 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f03c806af90 0x7f03c806d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.146+0000 7f03eda56640 1 --2- 172.21.15.84:0/3134660484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03e806ffd0 0x7f03e814d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.146+0000 7f03eda56640 1 -- 172.21.15.84:0/3134660484 >> 172.21.15.84:0/3134660484 conn(0x7f03e80cb410 msgr2=0x7f03e80c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:16.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.146+0000 7f03eda56640 1 -- 172.21.15.84:0/3134660484 shutdown_connections 2023-10-16T10:09:16.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.146+0000 7f03eda56640 1 -- 172.21.15.84:0/3134660484 wait complete. 2023-10-16T10:09:16.166 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-16T10:09:15.371025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1312,"num_read_kb":2614,"num_write":445,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2706,"ondisk_log_size":2706,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":366692,"kb_used_data":15912,"kb_used_omap":12,"kb_used_meta":350643,"kb_avail":754608028,"statfs":{"total":773094113280,"available":772718620672,"internally_reserved":0,"allocated":16293888,"data_stored":8387756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":359059001},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001776"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.207685+0000","last_change":"2023-10-16T10:09:11.213237+0000","last_active":"2023-10-16T10:09:12.207685+0000","last_peered":"2023-10-16T10:09:12.207685+0000","last_clean":"2023-10-16T10:09:12.207685+0000","last_became_active":"2023-10-16T10:09:11.212365+0000","last_became_peered":"2023-10-16T10:09:11.212365+0000","last_unstale":"2023-10-16T10:09:12.207685+0000","last_undegraded":"2023-10-16T10:09:12.207685+0000","last_fullsized":"2023-10-16T10:09:12.207685+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:28:35.977172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208544+0000","last_change":"2023-10-16T10:09:11.241999+0000","last_active":"2023-10-16T10:09:12.208544+0000","last_peered":"2023-10-16T10:09:12.208544+0000","last_clean":"2023-10-16T10:09:12.208544+0000","last_became_active":"2023-10-16T10:09:11.240315+0000","last_became_peered":"2023-10-16T10:09:11.240315+0000","last_unstale":"2023-10-16T10:09:12.208544+0000","last_undegraded":"2023-10-16T10:09:12.208544+0000","last_fullsized":"2023-10-16T10:09:12.208544+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:25:59.127401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208822+0000","last_change":"2023-10-16T10:09:11.225266+0000","last_active":"2023-10-16T10:09:12.208822+0000","last_peered":"2023-10-16T10:09:12.208822+0000","last_clean":"2023-10-16T10:09:12.208822+0000","last_became_active":"2023-10-16T10:09:11.224853+0000","last_became_peered":"2023-10-16T10:09:11.224853+0000","last_unstale":"2023-10-16T10:09:12.208822+0000","last_undegraded":"2023-10-16T10:09:12.208822+0000","last_fullsized":"2023-10-16T10:09:12.208822+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:10.978671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.206562+0000","last_change":"2023-10-16T10:09:11.226367+0000","last_active":"2023-10-16T10:09:12.206562+0000","last_peered":"2023-10-16T10:09:12.206562+0000","last_clean":"2023-10-16T10:09:12.206562+0000","last_became_active":"2023-10-16T10:09:11.226091+0000","last_became_peered":"2023-10-16T10:09:11.226091+0000","last_unstale":"2023-10-16T10:09:12.206562+0000","last_undegraded":"2023-10-16T10:09:12.206562+0000","last_fullsized":"2023-10-16T10:09:12.206562+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:18:30.005002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.050395+0000","last_change":"2023-10-16T10:09:11.227803+0000","last_active":"2023-10-16T10:09:13.050395+0000","last_peered":"2023-10-16T10:09:13.050395+0000","last_clean":"2023-10-16T10:09:13.050395+0000","last_became_active":"2023-10-16T10:09:11.227144+0000","last_became_peered":"2023-10-16T10:09:11.227144+0000","last_unstale":"2023-10-16T10:09:13.050395+0000","last_undegraded":"2023-10-16T10:09:13.050395+0000","last_fullsized":"2023-10-16T10:09:13.050395+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:45:42.610399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.048944+0000","last_change":"2023-10-16T10:09:11.208736+0000","last_active":"2023-10-16T10:09:13.048944+0000","last_peered":"2023-10-16T10:09:13.048944+0000","last_clean":"2023-10-16T10:09:13.048944+0000","last_became_active":"2023-10-16T10:09:11.206827+0000","last_became_peered":"2023-10-16T10:09:11.206827+0000","last_unstale":"2023-10-16T10:09:13.048944+0000","last_undegraded":"2023-10-16T10:09:13.048944+0000","last_fullsized":"2023-10-16T10:09:13.048944+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:48:31.852953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.048019+0000","last_change":"2023-10-16T10:09:11.214970+0000","last_active":"2023-10-16T10:09:13.048019+0000","last_peered":"2023-10-16T10:09:13.048019+0000","last_clean":"2023-10-16T10:09:13.048019+0000","last_became_active":"2023-10-16T10:09:11.214358+0000","last_became_peered":"2023-10-16T10:09:11.214358+0000","last_unstale":"2023-10-16T10:09:13.048019+0000","last_undegraded":"2023-10-16T10:09:13.048019+0000","last_fullsized":"2023-10-16T10:09:13.048019+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:31:23.775489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208389+0000","last_change":"2023-10-16T10:09:11.239754+0000","last_active":"2023-10-16T10:09:12.208389+0000","last_peered":"2023-10-16T10:09:12.208389+0000","last_clean":"2023-10-16T10:09:12.208389+0000","last_became_active":"2023-10-16T10:09:11.238775+0000","last_became_peered":"2023-10-16T10:09:11.238775+0000","last_unstale":"2023-10-16T10:09:12.208389+0000","last_undegraded":"2023-10-16T10:09:12.208389+0000","last_fullsized":"2023-10-16T10:09:12.208389+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:08:55.397866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.209411+0000","last_change":"2023-10-16T10:09:11.227057+0000","last_active":"2023-10-16T10:09:12.209411+0000","last_peered":"2023-10-16T10:09:12.209411+0000","last_clean":"2023-10-16T10:09:12.209411+0000","last_became_active":"2023-10-16T10:09:11.224735+0000","last_became_peered":"2023-10-16T10:09:11.224735+0000","last_unstale":"2023-10-16T10:09:12.209411+0000","last_undegraded":"2023-10-16T10:09:12.209411+0000","last_fullsized":"2023-10-16T10:09:12.209411+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:26.175578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208630+0000","last_change":"2023-10-16T10:09:11.234870+0000","last_active":"2023-10-16T10:09:12.208630+0000","last_peered":"2023-10-16T10:09:12.208630+0000","last_clean":"2023-10-16T10:09:12.208630+0000","last_became_active":"2023-10-16T10:09:11.234328+0000","last_became_peered":"2023-10-16T10:09:11.234328+0000","last_unstale":"2023-10-16T10:09:12.208630+0000","last_undegraded":"2023-10-16T10:09:12.208630+0000","last_fullsized":"2023-10-16T10:09:12.208630+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:32.525637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.336471+0000","last_change":"2023-10-16T10:09:11.243784+0000","last_active":"2023-10-16T10:09:12.336471+0000","last_peered":"2023-10-16T10:09:12.336471+0000","last_clean":"2023-10-16T10:09:12.336471+0000","last_became_active":"2023-10-16T10:09:11.240172+0000","last_became_peered":"2023-10-16T10:09:11.240172+0000","last_unstale":"2023-10-16T10:09:12.336471+0000","last_undegraded":"2023-10-16T10:09:12.336471+0000","last_fullsized":"2023-10-16T10:09:12.336471+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:54:52.623627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208639+0000","last_change":"2023-10-16T10:09:11.222714+0000","last_active":"2023-10-16T10:09:12.208639+0000","last_peered":"2023-10-16T10:09:12.208639+0000","last_clean":"2023-10-16T10:09:12.208639+0000","last_became_active":"2023-10-16T10:09:11.222327+0000","last_became_peered":"2023-10-16T10:09:11.222327+0000","last_unstale":"2023-10-16T10:09:12.208639+0000","last_undegraded":"2023-10-16T10:09:12.208639+0000","last_fullsized":"2023-10-16T10:09:12.208639+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:21:57.195467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.206896+0000","last_change":"2023-10-16T10:09:11.224307+0000","last_active":"2023-10-16T10:09:12.206896+0000","last_peered":"2023-10-16T10:09:12.206896+0000","last_clean":"2023-10-16T10:09:12.206896+0000","last_became_active":"2023-10-16T10:09:11.221606+0000","last_became_peered":"2023-10-16T10:09:11.221606+0000","last_unstale":"2023-10-16T10:09:12.206896+0000","last_undegraded":"2023-10-16T10:09:12.206896+0000","last_fullsized":"2023-10-16T10:09:12.206896+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:59:30.219320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208133+0000","last_change":"2023-10-16T10:09:11.242341+0000","last_active":"2023-10-16T10:09:12.208133+0000","last_peered":"2023-10-16T10:09:12.208133+0000","last_clean":"2023-10-16T10:09:12.208133+0000","last_became_active":"2023-10-16T10:09:11.241853+0000","last_became_peered":"2023-10-16T10:09:11.241853+0000","last_unstale":"2023-10-16T10:09:12.208133+0000","last_undegraded":"2023-10-16T10:09:12.208133+0000","last_fullsized":"2023-10-16T10:09:12.208133+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:36:07.019847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.206349+0000","last_change":"2023-10-16T10:09:11.225717+0000","last_active":"2023-10-16T10:09:12.206349+0000","last_peered":"2023-10-16T10:09:12.206349+0000","last_clean":"2023-10-16T10:09:12.206349+0000","last_became_active":"2023-10-16T10:09:11.222562+0000","last_became_peered":"2023-10-16T10:09:11.222562+0000","last_unstale":"2023-10-16T10:09:12.206349+0000","last_undegraded":"2023-10-16T10:09:12.206349+0000","last_fullsized":"2023-10-16T10:09:12.206349+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:10.978671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.047709+0000","last_change":"2023-10-16T10:09:11.247506+0000","last_active":"2023-10-16T10:09:13.047709+0000","last_peered":"2023-10-16T10:09:13.047709+0000","last_clean":"2023-10-16T10:09:13.047709+0000","last_became_active":"2023-10-16T10:09:11.246798+0000","last_became_peered":"2023-10-16T10:09:11.246798+0000","last_unstale":"2023-10-16T10:09:13.047709+0000","last_undegraded":"2023-10-16T10:09:13.047709+0000","last_fullsized":"2023-10-16T10:09:13.047709+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:36:39.209812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.050704+0000","last_change":"2023-10-16T10:09:11.228421+0000","last_active":"2023-10-16T10:09:13.050704+0000","last_peered":"2023-10-16T10:09:13.050704+0000","last_clean":"2023-10-16T10:09:13.050704+0000","last_became_active":"2023-10-16T10:09:11.228126+0000","last_became_peered":"2023-10-16T10:09:11.228126+0000","last_unstale":"2023-10-16T10:09:13.050704+0000","last_undegraded":"2023-10-16T10:09:13.050704+0000","last_fullsized":"2023-10-16T10:09:13.050704+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:07.335691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.047399+0000","last_change":"2023-10-16T10:09:11.235464+0000","last_active":"2023-10-16T10:09:13.047399+0000","last_peered":"2023-10-16T10:09:13.047399+0000","last_clean":"2023-10-16T10:09:13.047399+0000","last_became_active":"2023-10-16T10:09:11.234994+0000","last_became_peered":"2023-10-16T10:09:11.234994+0000","last_unstale":"2023-10-16T10:09:13.047399+0000","last_undegraded":"2023-10-16T10:09:13.047399+0000","last_fullsized":"2023-10-16T10:09:13.047399+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:03.475900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.336626+0000","last_change":"2023-10-16T10:09:11.236149+0000","last_active":"2023-10-16T10:09:12.336626+0000","last_peered":"2023-10-16T10:09:12.336626+0000","last_clean":"2023-10-16T10:09:12.336626+0000","last_became_active":"2023-10-16T10:09:11.235887+0000","last_became_peered":"2023-10-16T10:09:11.235887+0000","last_unstale":"2023-10-16T10:09:12.336626+0000","last_undegraded":"2023-10-16T10:09:12.336626+0000","last_fullsized":"2023-10-16T10:09:12.336626+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:18.434293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.047634+0000","last_change":"2023-10-16T10:09:11.241595+0000","last_active":"2023-10-16T10:09:13.047634+0000","last_peered":"2023-10-16T10:09:13.047634+0000","last_clean":"2023-10-16T10:09:13.047634+0000","last_became_active":"2023-10-16T10:09:11.241300+0000","last_became_peered":"2023-10-16T10:09:11.241300+0000","last_unstale":"2023-10-16T10:09:13.047634+0000","last_undegraded":"2023-10-16T10:09:13.047634+0000","last_fullsized":"2023-10-16T10:09:13.047634+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:48:01.795616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.050102+0000","last_change":"2023-10-16T10:09:11.227980+0000","last_active":"2023-10-16T10:09:13.050102+0000","last_peered":"2023-10-16T10:09:13.050102+0000","last_clean":"2023-10-16T10:09:13.050102+0000","last_became_active":"2023-10-16T10:09:11.227533+0000","last_became_peered":"2023-10-16T10:09:11.227533+0000","last_unstale":"2023-10-16T10:09:13.050102+0000","last_undegraded":"2023-10-16T10:09:13.050102+0000","last_fullsized":"2023-10-16T10:09:13.050102+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:22:45.857135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.336353+0000","last_change":"2023-10-16T10:09:11.234617+0000","last_active":"2023-10-16T10:09:12.336353+0000","last_peered":"2023-10-16T10:09:12.336353+0000","last_clean":"2023-10-16T10:09:12.336353+0000","last_became_active":"2023-10-16T10:09:11.234291+0000","last_became_peered":"2023-10-16T10:09:11.234291+0000","last_unstale":"2023-10-16T10:09:12.336353+0000","last_undegraded":"2023-10-16T10:09:12.336353+0000","last_fullsized":"2023-10-16T10:09:12.336353+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:54:20.233069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.209261+0000","last_change":"2023-10-16T10:09:11.224584+0000","last_active":"2023-10-16T10:09:12.209261+0000","last_peered":"2023-10-16T10:09:12.209261+0000","last_clean":"2023-10-16T10:09:12.209261+0000","last_became_active":"2023-10-16T10:09:11.224283+0000","last_became_peered":"2023-10-16T10:09:11.224283+0000","last_unstale":"2023-10-16T10:09:12.209261+0000","last_undegraded":"2023-10-16T10:09:12.209261+0000","last_fullsized":"2023-10-16T10:09:12.209261+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:18:30.005002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.335934+0000","last_change":"2023-10-16T10:09:11.227974+0000","last_active":"2023-10-16T10:09:12.335934+0000","last_peered":"2023-10-16T10:09:12.335934+0000","last_clean":"2023-10-16T10:09:12.335934+0000","last_became_active":"2023-10-16T10:09:11.226044+0000","last_became_peered":"2023-10-16T10:09:11.226044+0000","last_unstale":"2023-10-16T10:09:12.335934+0000","last_undegraded":"2023-10-16T10:09:12.335934+0000","last_fullsized":"2023-10-16T10:09:12.335934+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:32.525637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208719+0000","last_change":"2023-10-16T10:09:11.242688+0000","last_active":"2023-10-16T10:09:12.208719+0000","last_peered":"2023-10-16T10:09:12.208719+0000","last_clean":"2023-10-16T10:09:12.208719+0000","last_became_active":"2023-10-16T10:09:11.240760+0000","last_became_peered":"2023-10-16T10:09:11.240760+0000","last_unstale":"2023-10-16T10:09:12.208719+0000","last_undegraded":"2023-10-16T10:09:12.208719+0000","last_fullsized":"2023-10-16T10:09:12.208719+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:54:20.233069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.047545+0000","last_change":"2023-10-16T10:09:11.244664+0000","last_active":"2023-10-16T10:09:13.047545+0000","last_peered":"2023-10-16T10:09:13.047545+0000","last_clean":"2023-10-16T10:09:13.047545+0000","last_became_active":"2023-10-16T10:09:11.244407+0000","last_became_peered":"2023-10-16T10:09:11.244407+0000","last_unstale":"2023-10-16T10:09:13.047545+0000","last_undegraded":"2023-10-16T10:09:13.047545+0000","last_fullsized":"2023-10-16T10:09:13.047545+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:01.612542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.206787+0000","last_change":"2023-10-16T10:09:11.215670+0000","last_active":"2023-10-16T10:09:12.206787+0000","last_peered":"2023-10-16T10:09:12.206787+0000","last_clean":"2023-10-16T10:09:12.206787+0000","last_became_active":"2023-10-16T10:09:11.213422+0000","last_became_peered":"2023-10-16T10:09:11.213422+0000","last_unstale":"2023-10-16T10:09:12.206787+0000","last_undegraded":"2023-10-16T10:09:12.206787+0000","last_fullsized":"2023-10-16T10:09:12.206787+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:02:23.577704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.049216+0000","last_change":"2023-10-16T10:09:11.232744+0000","last_active":"2023-10-16T10:09:13.049216+0000","last_peered":"2023-10-16T10:09:13.049216+0000","last_clean":"2023-10-16T10:09:13.049216+0000","last_became_active":"2023-10-16T10:09:11.232438+0000","last_became_peered":"2023-10-16T10:09:11.232438+0000","last_unstale":"2023-10-16T10:09:13.049216+0000","last_undegraded":"2023-10-16T10:09:13.049216+0000","last_fullsized":"2023-10-16T10:09:13.049216+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:33.177507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.206979+0000","last_change":"2023-10-16T10:09:11.233847+0000","last_active":"2023-10-16T10:09:12.206979+0000","last_peered":"2023-10-16T10:09:12.206979+0000","last_clean":"2023-10-16T10:09:12.206979+0000","last_became_active":"2023-10-16T10:09:11.223153+0000","last_became_peered":"2023-10-16T10:09:11.223153+0000","last_unstale":"2023-10-16T10:09:12.206979+0000","last_undegraded":"2023-10-16T10:09:12.206979+0000","last_fullsized":"2023-10-16T10:09:12.206979+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:54.541150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.206677+0000","last_change":"2023-10-16T10:09:11.221428+0000","last_active":"2023-10-16T10:09:12.206677+0000","last_peered":"2023-10-16T10:09:12.206677+0000","last_clean":"2023-10-16T10:09:12.206677+0000","last_became_active":"2023-10-16T10:09:11.220977+0000","last_became_peered":"2023-10-16T10:09:11.220977+0000","last_unstale":"2023-10-16T10:09:12.206677+0000","last_undegraded":"2023-10-16T10:09:12.206677+0000","last_fullsized":"2023-10-16T10:09:12.206677+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:26.175578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.050476+0000","last_change":"2023-10-16T10:09:11.232879+0000","last_active":"2023-10-16T10:09:13.050476+0000","last_peered":"2023-10-16T10:09:13.050476+0000","last_clean":"2023-10-16T10:09:13.050476+0000","last_became_active":"2023-10-16T10:09:11.232131+0000","last_became_peered":"2023-10-16T10:09:11.232131+0000","last_unstale":"2023-10-16T10:09:13.050476+0000","last_undegraded":"2023-10-16T10:09:13.050476+0000","last_fullsized":"2023-10-16T10:09:13.050476+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:50.625153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.047508+0000","last_change":"2023-10-16T10:09:11.242790+0000","last_active":"2023-10-16T10:09:13.047508+0000","last_peered":"2023-10-16T10:09:13.047508+0000","last_clean":"2023-10-16T10:09:13.047508+0000","last_became_active":"2023-10-16T10:09:11.242375+0000","last_became_peered":"2023-10-16T10:09:11.242375+0000","last_unstale":"2023-10-16T10:09:13.047508+0000","last_undegraded":"2023-10-16T10:09:13.047508+0000","last_fullsized":"2023-10-16T10:09:13.047508+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:54.351202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.047751+0000","last_change":"2023-10-16T10:09:11.235262+0000","last_active":"2023-10-16T10:09:13.047751+0000","last_peered":"2023-10-16T10:09:13.047751+0000","last_clean":"2023-10-16T10:09:13.047751+0000","last_became_active":"2023-10-16T10:09:11.234718+0000","last_became_peered":"2023-10-16T10:09:11.234718+0000","last_unstale":"2023-10-16T10:09:13.047751+0000","last_undegraded":"2023-10-16T10:09:13.047751+0000","last_fullsized":"2023-10-16T10:09:13.047751+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:31.713636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"150'4","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.249089+0000","last_change":"2023-10-16T10:09:10.208679+0000","last_active":"2023-10-16T10:09:11.249089+0000","last_peered":"2023-10-16T10:09:11.249089+0000","last_clean":"2023-10-16T10:09:11.249089+0000","last_became_active":"2023-10-16T10:09:10.208427+0000","last_became_peered":"2023-10-16T10:09:10.208427+0000","last_unstale":"2023-10-16T10:09:11.249089+0000","last_undegraded":"2023-10-16T10:09:11.249089+0000","last_fullsized":"2023-10-16T10:09:11.249089+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:45:32.302211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.209495+0000","last_change":"2023-10-16T10:09:11.227653+0000","last_active":"2023-10-16T10:09:12.209495+0000","last_peered":"2023-10-16T10:09:12.209495+0000","last_clean":"2023-10-16T10:09:12.209495+0000","last_became_active":"2023-10-16T10:09:11.227391+0000","last_became_peered":"2023-10-16T10:09:11.227391+0000","last_unstale":"2023-10-16T10:09:12.209495+0000","last_undegraded":"2023-10-16T10:09:12.209495+0000","last_fullsized":"2023-10-16T10:09:12.209495+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:02:23.577704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.247997+0000","last_change":"2023-10-16T10:09:10.201030+0000","last_active":"2023-10-16T10:09:11.247997+0000","last_peered":"2023-10-16T10:09:11.247997+0000","last_clean":"2023-10-16T10:09:11.247997+0000","last_became_active":"2023-10-16T10:09:10.199995+0000","last_became_peered":"2023-10-16T10:09:10.199995+0000","last_unstale":"2023-10-16T10:09:11.247997+0000","last_undegraded":"2023-10-16T10:09:11.247997+0000","last_fullsized":"2023-10-16T10:09:11.247997+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:05:24.008125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.051518+0000","last_change":"2023-10-16T10:09:11.231021+0000","last_active":"2023-10-16T10:09:13.051518+0000","last_peered":"2023-10-16T10:09:13.051518+0000","last_clean":"2023-10-16T10:09:13.051518+0000","last_became_active":"2023-10-16T10:09:11.230314+0000","last_became_peered":"2023-10-16T10:09:11.230314+0000","last_unstale":"2023-10-16T10:09:13.051518+0000","last_undegraded":"2023-10-16T10:09:13.051518+0000","last_fullsized":"2023-10-16T10:09:13.051518+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:57.434101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.251607+0000","last_change":"2023-10-16T10:09:10.194076+0000","last_active":"2023-10-16T10:09:11.251607+0000","last_peered":"2023-10-16T10:09:11.251607+0000","last_clean":"2023-10-16T10:09:11.251607+0000","last_became_active":"2023-10-16T10:09:10.193820+0000","last_became_peered":"2023-10-16T10:09:10.193820+0000","last_unstale":"2023-10-16T10:09:11.251607+0000","last_undegraded":"2023-10-16T10:09:11.251607+0000","last_fullsized":"2023-10-16T10:09:11.251607+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:49:33.976254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.051911+0000","last_change":"2023-10-16T10:09:11.244395+0000","last_active":"2023-10-16T10:09:13.051911+0000","last_peered":"2023-10-16T10:09:13.051911+0000","last_clean":"2023-10-16T10:09:13.051911+0000","last_became_active":"2023-10-16T10:09:11.243852+0000","last_became_peered":"2023-10-16T10:09:11.243852+0000","last_unstale":"2023-10-16T10:09:13.051911+0000","last_undegraded":"2023-10-16T10:09:13.051911+0000","last_fullsized":"2023-10-16T10:09:13.051911+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:44.116225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.249665+0000","last_change":"2023-10-16T10:09:10.200360+0000","last_active":"2023-10-16T10:09:11.249665+0000","last_peered":"2023-10-16T10:09:11.249665+0000","last_clean":"2023-10-16T10:09:11.249665+0000","last_became_active":"2023-10-16T10:09:10.199787+0000","last_became_peered":"2023-10-16T10:09:10.199787+0000","last_unstale":"2023-10-16T10:09:11.249665+0000","last_undegraded":"2023-10-16T10:09:11.249665+0000","last_fullsized":"2023-10-16T10:09:11.249665+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:02:22.570336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.046976+0000","last_change":"2023-10-16T10:09:11.237863+0000","last_active":"2023-10-16T10:09:13.046976+0000","last_peered":"2023-10-16T10:09:13.046976+0000","last_clean":"2023-10-16T10:09:13.046976+0000","last_became_active":"2023-10-16T10:09:11.237575+0000","last_became_peered":"2023-10-16T10:09:11.237575+0000","last_unstale":"2023-10-16T10:09:13.046976+0000","last_undegraded":"2023-10-16T10:09:13.046976+0000","last_fullsized":"2023-10-16T10:09:13.046976+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:57:46.806651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.246082+0000","last_change":"2023-10-16T10:09:10.202300+0000","last_active":"2023-10-16T10:09:11.246082+0000","last_peered":"2023-10-16T10:09:11.246082+0000","last_clean":"2023-10-16T10:09:11.246082+0000","last_became_active":"2023-10-16T10:09:10.201978+0000","last_became_peered":"2023-10-16T10:09:10.201978+0000","last_unstale":"2023-10-16T10:09:11.246082+0000","last_undegraded":"2023-10-16T10:09:11.246082+0000","last_fullsized":"2023-10-16T10:09:11.246082+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:05.877227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.048048+0000","last_change":"2023-10-16T10:09:11.228714+0000","last_active":"2023-10-16T10:09:13.048048+0000","last_peered":"2023-10-16T10:09:13.048048+0000","last_clean":"2023-10-16T10:09:13.048048+0000","last_became_active":"2023-10-16T10:09:11.228372+0000","last_became_peered":"2023-10-16T10:09:11.228372+0000","last_unstale":"2023-10-16T10:09:13.048048+0000","last_undegraded":"2023-10-16T10:09:13.048048+0000","last_fullsized":"2023-10-16T10:09:13.048048+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:26.298785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.250384+0000","last_change":"2023-10-16T10:09:10.197219+0000","last_active":"2023-10-16T10:09:11.250384+0000","last_peered":"2023-10-16T10:09:11.250384+0000","last_clean":"2023-10-16T10:09:11.250384+0000","last_became_active":"2023-10-16T10:09:10.196896+0000","last_became_peered":"2023-10-16T10:09:10.196896+0000","last_unstale":"2023-10-16T10:09:11.250384+0000","last_undegraded":"2023-10-16T10:09:11.250384+0000","last_fullsized":"2023-10-16T10:09:11.250384+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:11:30.069197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208440+0000","last_change":"2023-10-16T10:09:11.239535+0000","last_active":"2023-10-16T10:09:12.208440+0000","last_peered":"2023-10-16T10:09:12.208440+0000","last_clean":"2023-10-16T10:09:12.208440+0000","last_became_active":"2023-10-16T10:09:11.238438+0000","last_became_peered":"2023-10-16T10:09:11.238438+0000","last_unstale":"2023-10-16T10:09:12.208440+0000","last_undegraded":"2023-10-16T10:09:12.208440+0000","last_fullsized":"2023-10-16T10:09:12.208440+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:35:25.459805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.203187+0000","last_change":"2023-10-16T10:09:10.205455+0000","last_active":"2023-10-16T10:09:11.203187+0000","last_peered":"2023-10-16T10:09:11.203187+0000","last_clean":"2023-10-16T10:09:11.203187+0000","last_became_active":"2023-10-16T10:09:10.204515+0000","last_became_peered":"2023-10-16T10:09:10.204515+0000","last_unstale":"2023-10-16T10:09:11.203187+0000","last_undegraded":"2023-10-16T10:09:11.203187+0000","last_fullsized":"2023-10-16T10:09:11.203187+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:08:18.685850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.207902+0000","last_change":"2023-10-16T10:09:11.242837+0000","last_active":"2023-10-16T10:09:12.207902+0000","last_peered":"2023-10-16T10:09:12.207902+0000","last_clean":"2023-10-16T10:09:12.207902+0000","last_became_active":"2023-10-16T10:09:11.242506+0000","last_became_peered":"2023-10-16T10:09:11.242506+0000","last_unstale":"2023-10-16T10:09:12.207902+0000","last_undegraded":"2023-10-16T10:09:12.207902+0000","last_fullsized":"2023-10-16T10:09:12.207902+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:53:04.619879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.217590+0000","last_change":"2023-10-16T10:09:10.211411+0000","last_active":"2023-10-16T10:09:11.217590+0000","last_peered":"2023-10-16T10:09:11.217590+0000","last_clean":"2023-10-16T10:09:11.217590+0000","last_became_active":"2023-10-16T10:09:10.210108+0000","last_became_peered":"2023-10-16T10:09:10.210108+0000","last_unstale":"2023-10-16T10:09:11.217590+0000","last_undegraded":"2023-10-16T10:09:11.217590+0000","last_fullsized":"2023-10-16T10:09:11.217590+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:53.533782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.207114+0000","last_change":"2023-10-16T10:09:11.227560+0000","last_active":"2023-10-16T10:09:12.207114+0000","last_peered":"2023-10-16T10:09:12.207114+0000","last_clean":"2023-10-16T10:09:12.207114+0000","last_became_active":"2023-10-16T10:09:11.227297+0000","last_became_peered":"2023-10-16T10:09:11.227297+0000","last_unstale":"2023-10-16T10:09:12.207114+0000","last_undegraded":"2023-10-16T10:09:12.207114+0000","last_fullsized":"2023-10-16T10:09:12.207114+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:06.884595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.225428+0000","last_change":"2023-10-16T10:09:10.204028+0000","last_active":"2023-10-16T10:09:11.225428+0000","last_peered":"2023-10-16T10:09:11.225428+0000","last_clean":"2023-10-16T10:09:11.225428+0000","last_became_active":"2023-10-16T10:09:10.203742+0000","last_became_peered":"2023-10-16T10:09:10.203742+0000","last_unstale":"2023-10-16T10:09:11.225428+0000","last_undegraded":"2023-10-16T10:09:11.225428+0000","last_fullsized":"2023-10-16T10:09:11.225428+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:22:12.115124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.050069+0000","last_change":"2023-10-16T10:09:11.232432+0000","last_active":"2023-10-16T10:09:13.050069+0000","last_peered":"2023-10-16T10:09:13.050069+0000","last_clean":"2023-10-16T10:09:13.050069+0000","last_became_active":"2023-10-16T10:09:11.232157+0000","last_became_peered":"2023-10-16T10:09:11.232157+0000","last_unstale":"2023-10-16T10:09:13.050069+0000","last_undegraded":"2023-10-16T10:09:13.050069+0000","last_fullsized":"2023-10-16T10:09:13.050069+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:26:04.070083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.231203+0000","last_change":"2023-10-16T10:09:10.209334+0000","last_active":"2023-10-16T10:09:11.231203+0000","last_peered":"2023-10-16T10:09:11.231203+0000","last_clean":"2023-10-16T10:09:11.231203+0000","last_became_active":"2023-10-16T10:09:10.209055+0000","last_became_peered":"2023-10-16T10:09:10.209055+0000","last_unstale":"2023-10-16T10:09:11.231203+0000","last_undegraded":"2023-10-16T10:09:11.231203+0000","last_fullsized":"2023-10-16T10:09:11.231203+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:24:59.824573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.051749+0000","last_change":"2023-10-16T10:09:11.233258+0000","last_active":"2023-10-16T10:09:13.051749+0000","last_peered":"2023-10-16T10:09:13.051749+0000","last_clean":"2023-10-16T10:09:13.051749+0000","last_became_active":"2023-10-16T10:09:11.232650+0000","last_became_peered":"2023-10-16T10:09:11.232650+0000","last_unstale":"2023-10-16T10:09:13.051749+0000","last_undegraded":"2023-10-16T10:09:13.051749+0000","last_fullsized":"2023-10-16T10:09:13.051749+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:31.100635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"150'3","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.250243+0000","last_change":"2023-10-16T10:09:10.205537+0000","last_active":"2023-10-16T10:09:11.250243+0000","last_peered":"2023-10-16T10:09:11.250243+0000","last_clean":"2023-10-16T10:09:11.250243+0000","last_became_active":"2023-10-16T10:09:10.205274+0000","last_became_peered":"2023-10-16T10:09:10.205274+0000","last_unstale":"2023-10-16T10:09:11.250243+0000","last_undegraded":"2023-10-16T10:09:11.250243+0000","last_fullsized":"2023-10-16T10:09:11.250243+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:04.167427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.051781+0000","last_change":"2023-10-16T10:09:11.229835+0000","last_active":"2023-10-16T10:09:13.051781+0000","last_peered":"2023-10-16T10:09:13.051781+0000","last_clean":"2023-10-16T10:09:13.051781+0000","last_became_active":"2023-10-16T10:09:11.229562+0000","last_became_peered":"2023-10-16T10:09:11.229562+0000","last_unstale":"2023-10-16T10:09:13.051781+0000","last_undegraded":"2023-10-16T10:09:13.051781+0000","last_fullsized":"2023-10-16T10:09:13.051781+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:04:11.537413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"150'6","reported_seq":18,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.248864+0000","last_change":"2023-10-16T10:09:10.211025+0000","last_active":"2023-10-16T10:09:11.248864+0000","last_peered":"2023-10-16T10:09:11.248864+0000","last_clean":"2023-10-16T10:09:11.248864+0000","last_became_active":"2023-10-16T10:09:10.209558+0000","last_became_peered":"2023-10-16T10:09:10.209558+0000","last_unstale":"2023-10-16T10:09:11.248864+0000","last_undegraded":"2023-10-16T10:09:11.248864+0000","last_fullsized":"2023-10-16T10:09:11.248864+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:55.065613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.207134+0000","last_change":"2023-10-16T10:09:11.227859+0000","last_active":"2023-10-16T10:09:12.207134+0000","last_peered":"2023-10-16T10:09:12.207134+0000","last_clean":"2023-10-16T10:09:12.207134+0000","last_became_active":"2023-10-16T10:09:11.226595+0000","last_became_peered":"2023-10-16T10:09:11.226595+0000","last_unstale":"2023-10-16T10:09:12.207134+0000","last_undegraded":"2023-10-16T10:09:12.207134+0000","last_fullsized":"2023-10-16T10:09:12.207134+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:11:31.338000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.197612+0000","last_change":"2023-10-16T10:09:10.199960+0000","last_active":"2023-10-16T10:09:11.197612+0000","last_peered":"2023-10-16T10:09:11.197612+0000","last_clean":"2023-10-16T10:09:11.197612+0000","last_became_active":"2023-10-16T10:09:10.199682+0000","last_became_peered":"2023-10-16T10:09:10.199682+0000","last_unstale":"2023-10-16T10:09:11.197612+0000","last_undegraded":"2023-10-16T10:09:11.197612+0000","last_fullsized":"2023-10-16T10:09:11.197612+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:52:42.889786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.207762+0000","last_change":"2023-10-16T10:09:11.222052+0000","last_active":"2023-10-16T10:09:12.207762+0000","last_peered":"2023-10-16T10:09:12.207762+0000","last_clean":"2023-10-16T10:09:12.207762+0000","last_became_active":"2023-10-16T10:09:11.221700+0000","last_became_peered":"2023-10-16T10:09:11.221700+0000","last_unstale":"2023-10-16T10:09:12.207762+0000","last_undegraded":"2023-10-16T10:09:12.207762+0000","last_fullsized":"2023-10-16T10:09:12.207762+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:53:25.682713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.224594+0000","last_change":"2023-10-16T10:09:10.208071+0000","last_active":"2023-10-16T10:09:11.224594+0000","last_peered":"2023-10-16T10:09:11.224594+0000","last_clean":"2023-10-16T10:09:11.224594+0000","last_became_active":"2023-10-16T10:09:10.207730+0000","last_became_peered":"2023-10-16T10:09:10.207730+0000","last_unstale":"2023-10-16T10:09:11.224594+0000","last_undegraded":"2023-10-16T10:09:11.224594+0000","last_fullsized":"2023-10-16T10:09:11.224594+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:09:14.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208722+0000","last_change":"2023-10-16T10:09:11.223324+0000","last_active":"2023-10-16T10:09:12.208722+0000","last_peered":"2023-10-16T10:09:12.208722+0000","last_clean":"2023-10-16T10:09:12.208722+0000","last_became_active":"2023-10-16T10:09:11.223039+0000","last_became_peered":"2023-10-16T10:09:11.223039+0000","last_unstale":"2023-10-16T10:09:12.208722+0000","last_undegraded":"2023-10-16T10:09:12.208722+0000","last_fullsized":"2023-10-16T10:09:12.208722+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:59.356399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"150'8","reported_seq":20,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.250551+0000","last_change":"2023-10-16T10:09:10.208706+0000","last_active":"2023-10-16T10:09:11.250551+0000","last_peered":"2023-10-16T10:09:11.250551+0000","last_clean":"2023-10-16T10:09:11.250551+0000","last_became_active":"2023-10-16T10:09:10.208358+0000","last_became_peered":"2023-10-16T10:09:10.208358+0000","last_unstale":"2023-10-16T10:09:11.250551+0000","last_undegraded":"2023-10-16T10:09:11.250551+0000","last_fullsized":"2023-10-16T10:09:11.250551+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:38:02.653183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208652+0000","last_change":"2023-10-16T10:09:11.221285+0000","last_active":"2023-10-16T10:09:12.208652+0000","last_peered":"2023-10-16T10:09:12.208652+0000","last_clean":"2023-10-16T10:09:12.208652+0000","last_became_active":"2023-10-16T10:09:11.221000+0000","last_became_peered":"2023-10-16T10:09:11.221000+0000","last_unstale":"2023-10-16T10:09:12.208652+0000","last_undegraded":"2023-10-16T10:09:12.208652+0000","last_fullsized":"2023-10-16T10:09:12.208652+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:09.872490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.245689+0000","last_change":"2023-10-16T10:09:10.205199+0000","last_active":"2023-10-16T10:09:11.245689+0000","last_peered":"2023-10-16T10:09:11.245689+0000","last_clean":"2023-10-16T10:09:11.245689+0000","last_became_active":"2023-10-16T10:09:10.204886+0000","last_became_peered":"2023-10-16T10:09:10.204886+0000","last_unstale":"2023-10-16T10:09:11.245689+0000","last_undegraded":"2023-10-16T10:09:11.245689+0000","last_fullsized":"2023-10-16T10:09:11.245689+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:08:18.685850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.050416+0000","last_change":"2023-10-16T10:09:11.225800+0000","last_active":"2023-10-16T10:09:13.050416+0000","last_peered":"2023-10-16T10:09:13.050416+0000","last_clean":"2023-10-16T10:09:13.050416+0000","last_became_active":"2023-10-16T10:09:11.225009+0000","last_became_peered":"2023-10-16T10:09:11.225009+0000","last_unstale":"2023-10-16T10:09:13.050416+0000","last_undegraded":"2023-10-16T10:09:13.050416+0000","last_fullsized":"2023-10-16T10:09:13.050416+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:00.485898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.248877+0000","last_change":"2023-10-16T10:09:10.211247+0000","last_active":"2023-10-16T10:09:11.248877+0000","last_peered":"2023-10-16T10:09:11.248877+0000","last_clean":"2023-10-16T10:09:11.248877+0000","last_became_active":"2023-10-16T10:09:10.210943+0000","last_became_peered":"2023-10-16T10:09:10.210943+0000","last_unstale":"2023-10-16T10:09:11.248877+0000","last_undegraded":"2023-10-16T10:09:11.248877+0000","last_fullsized":"2023-10-16T10:09:11.248877+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:36.396211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.047737+0000","last_change":"2023-10-16T10:09:11.241701+0000","last_active":"2023-10-16T10:09:13.047737+0000","last_peered":"2023-10-16T10:09:13.047737+0000","last_clean":"2023-10-16T10:09:13.047737+0000","last_became_active":"2023-10-16T10:09:11.241413+0000","last_became_peered":"2023-10-16T10:09:11.241413+0000","last_unstale":"2023-10-16T10:09:13.047737+0000","last_undegraded":"2023-10-16T10:09:13.047737+0000","last_fullsized":"2023-10-16T10:09:13.047737+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:44:14.173255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.249871+0000","last_change":"2023-10-16T10:09:10.199150+0000","last_active":"2023-10-16T10:09:11.249871+0000","last_peered":"2023-10-16T10:09:11.249871+0000","last_clean":"2023-10-16T10:09:11.249871+0000","last_became_active":"2023-10-16T10:09:10.197993+0000","last_became_peered":"2023-10-16T10:09:10.197993+0000","last_unstale":"2023-10-16T10:09:11.249871+0000","last_undegraded":"2023-10-16T10:09:11.249871+0000","last_fullsized":"2023-10-16T10:09:11.249871+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:26:03.062715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208271+0000","last_change":"2023-10-16T10:09:11.240689+0000","last_active":"2023-10-16T10:09:12.208271+0000","last_peered":"2023-10-16T10:09:12.208271+0000","last_clean":"2023-10-16T10:09:12.208271+0000","last_became_active":"2023-10-16T10:09:11.240387+0000","last_became_peered":"2023-10-16T10:09:11.240387+0000","last_unstale":"2023-10-16T10:09:12.208271+0000","last_undegraded":"2023-10-16T10:09:12.208271+0000","last_fullsized":"2023-10-16T10:09:12.208271+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:49:34.983622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.224288+0000","last_change":"2023-10-16T10:09:10.206139+0000","last_active":"2023-10-16T10:09:11.224288+0000","last_peered":"2023-10-16T10:09:11.224288+0000","last_clean":"2023-10-16T10:09:11.224288+0000","last_became_active":"2023-10-16T10:09:10.205808+0000","last_became_peered":"2023-10-16T10:09:10.205808+0000","last_unstale":"2023-10-16T10:09:11.224288+0000","last_undegraded":"2023-10-16T10:09:11.224288+0000","last_fullsized":"2023-10-16T10:09:11.224288+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:39:56.103852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.206845+0000","last_change":"2023-10-16T10:09:11.218098+0000","last_active":"2023-10-16T10:09:12.206845+0000","last_peered":"2023-10-16T10:09:12.206845+0000","last_clean":"2023-10-16T10:09:12.206845+0000","last_became_active":"2023-10-16T10:09:11.214187+0000","last_became_peered":"2023-10-16T10:09:11.214187+0000","last_unstale":"2023-10-16T10:09:12.206845+0000","last_undegraded":"2023-10-16T10:09:12.206845+0000","last_fullsized":"2023-10-16T10:09:12.206845+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:50:48.914441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.249524+0000","last_change":"2023-10-16T10:09:10.203911+0000","last_active":"2023-10-16T10:09:11.249524+0000","last_peered":"2023-10-16T10:09:11.249524+0000","last_clean":"2023-10-16T10:09:11.249524+0000","last_became_active":"2023-10-16T10:09:10.203003+0000","last_became_peered":"2023-10-16T10:09:10.203003+0000","last_unstale":"2023-10-16T10:09:11.249524+0000","last_undegraded":"2023-10-16T10:09:11.249524+0000","last_fullsized":"2023-10-16T10:09:11.249524+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:15:05.272452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.048408+0000","last_change":"2023-10-16T10:09:11.215534+0000","last_active":"2023-10-16T10:09:13.048408+0000","last_peered":"2023-10-16T10:09:13.048408+0000","last_clean":"2023-10-16T10:09:13.048408+0000","last_became_active":"2023-10-16T10:09:11.215024+0000","last_became_peered":"2023-10-16T10:09:11.215024+0000","last_unstale":"2023-10-16T10:09:13.048408+0000","last_undegraded":"2023-10-16T10:09:13.048408+0000","last_fullsized":"2023-10-16T10:09:13.048408+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:51.817283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.229528+0000","last_change":"2023-10-16T10:09:10.209580+0000","last_active":"2023-10-16T10:09:11.229528+0000","last_peered":"2023-10-16T10:09:11.229528+0000","last_clean":"2023-10-16T10:09:11.229528+0000","last_became_active":"2023-10-16T10:09:10.209024+0000","last_became_peered":"2023-10-16T10:09:10.209024+0000","last_unstale":"2023-10-16T10:09:11.229528+0000","last_undegraded":"2023-10-16T10:09:11.229528+0000","last_fullsized":"2023-10-16T10:09:11.229528+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:26:55.839042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208360+0000","last_change":"2023-10-16T10:09:11.236746+0000","last_active":"2023-10-16T10:09:12.208360+0000","last_peered":"2023-10-16T10:09:12.208360+0000","last_clean":"2023-10-16T10:09:12.208360+0000","last_became_active":"2023-10-16T10:09:11.232883+0000","last_became_peered":"2023-10-16T10:09:11.232883+0000","last_unstale":"2023-10-16T10:09:12.208360+0000","last_undegraded":"2023-10-16T10:09:12.208360+0000","last_fullsized":"2023-10-16T10:09:12.208360+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:10:46.500241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.230045+0000","last_change":"2023-10-16T10:09:10.199961+0000","last_active":"2023-10-16T10:09:11.230045+0000","last_peered":"2023-10-16T10:09:11.230045+0000","last_clean":"2023-10-16T10:09:11.230045+0000","last_became_active":"2023-10-16T10:09:10.199696+0000","last_became_peered":"2023-10-16T10:09:10.199696+0000","last_unstale":"2023-10-16T10:09:11.230045+0000","last_undegraded":"2023-10-16T10:09:11.230045+0000","last_fullsized":"2023-10-16T10:09:11.230045+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:49:51.674081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.050198+0000","last_change":"2023-10-16T10:09:11.245494+0000","last_active":"2023-10-16T10:09:13.050198+0000","last_peered":"2023-10-16T10:09:13.050198+0000","last_clean":"2023-10-16T10:09:13.050198+0000","last_became_active":"2023-10-16T10:09:11.244970+0000","last_became_peered":"2023-10-16T10:09:11.244970+0000","last_unstale":"2023-10-16T10:09:13.050198+0000","last_undegraded":"2023-10-16T10:09:13.050198+0000","last_fullsized":"2023-10-16T10:09:13.050198+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:49.918171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.223555+0000","last_change":"2023-10-16T10:09:10.209122+0000","last_active":"2023-10-16T10:09:11.223555+0000","last_peered":"2023-10-16T10:09:11.223555+0000","last_clean":"2023-10-16T10:09:11.223555+0000","last_became_active":"2023-10-16T10:09:10.208536+0000","last_became_peered":"2023-10-16T10:09:10.208536+0000","last_unstale":"2023-10-16T10:09:11.223555+0000","last_undegraded":"2023-10-16T10:09:11.223555+0000","last_fullsized":"2023-10-16T10:09:11.223555+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:26.460188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.051193+0000","last_change":"2023-10-16T10:09:11.225647+0000","last_active":"2023-10-16T10:09:13.051193+0000","last_peered":"2023-10-16T10:09:13.051193+0000","last_clean":"2023-10-16T10:09:13.051193+0000","last_became_active":"2023-10-16T10:09:11.225343+0000","last_became_peered":"2023-10-16T10:09:11.225343+0000","last_unstale":"2023-10-16T10:09:13.051193+0000","last_undegraded":"2023-10-16T10:09:13.051193+0000","last_fullsized":"2023-10-16T10:09:13.051193+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:32:26.308547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.249103+0000","last_change":"2023-10-16T10:09:10.207380+0000","last_active":"2023-10-16T10:09:11.249103+0000","last_peered":"2023-10-16T10:09:11.249103+0000","last_clean":"2023-10-16T10:09:11.249103+0000","last_became_active":"2023-10-16T10:09:10.207009+0000","last_became_peered":"2023-10-16T10:09:10.207009+0000","last_unstale":"2023-10-16T10:09:11.249103+0000","last_undegraded":"2023-10-16T10:09:11.249103+0000","last_fullsized":"2023-10-16T10:09:11.249103+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:58.349031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.207579+0000","last_change":"2023-10-16T10:09:11.232563+0000","last_active":"2023-10-16T10:09:12.207579+0000","last_peered":"2023-10-16T10:09:12.207579+0000","last_clean":"2023-10-16T10:09:12.207579+0000","last_became_active":"2023-10-16T10:09:11.228149+0000","last_became_peered":"2023-10-16T10:09:11.228149+0000","last_unstale":"2023-10-16T10:09:12.207579+0000","last_undegraded":"2023-10-16T10:09:12.207579+0000","last_fullsized":"2023-10-16T10:09:12.207579+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:14:29.869731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.217587+0000","last_change":"2023-10-16T10:09:10.210075+0000","last_active":"2023-10-16T10:09:11.217587+0000","last_peered":"2023-10-16T10:09:11.217587+0000","last_clean":"2023-10-16T10:09:11.217587+0000","last_became_active":"2023-10-16T10:09:10.207818+0000","last_became_peered":"2023-10-16T10:09:10.207818+0000","last_unstale":"2023-10-16T10:09:11.217587+0000","last_undegraded":"2023-10-16T10:09:11.217587+0000","last_fullsized":"2023-10-16T10:09:11.217587+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:57:15.139270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208062+0000","last_change":"2023-10-16T10:09:11.239886+0000","last_active":"2023-10-16T10:09:12.208062+0000","last_peered":"2023-10-16T10:09:12.208062+0000","last_clean":"2023-10-16T10:09:12.208062+0000","last_became_active":"2023-10-16T10:09:11.237590+0000","last_became_peered":"2023-10-16T10:09:11.237590+0000","last_unstale":"2023-10-16T10:09:12.208062+0000","last_undegraded":"2023-10-16T10:09:12.208062+0000","last_fullsized":"2023-10-16T10:09:12.208062+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:27.467556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.223559+0000","last_change":"2023-10-16T10:09:10.211471+0000","last_active":"2023-10-16T10:09:11.223559+0000","last_peered":"2023-10-16T10:09:11.223559+0000","last_clean":"2023-10-16T10:09:11.223559+0000","last_became_active":"2023-10-16T10:09:10.209919+0000","last_became_peered":"2023-10-16T10:09:10.209919+0000","last_unstale":"2023-10-16T10:09:11.223559+0000","last_undegraded":"2023-10-16T10:09:11.223559+0000","last_fullsized":"2023-10-16T10:09:11.223559+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:28:34.969804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.049166+0000","last_change":"2023-10-16T10:09:11.210524+0000","last_active":"2023-10-16T10:09:13.049166+0000","last_peered":"2023-10-16T10:09:13.049166+0000","last_clean":"2023-10-16T10:09:13.049166+0000","last_became_active":"2023-10-16T10:09:11.209945+0000","last_became_peered":"2023-10-16T10:09:11.209945+0000","last_unstale":"2023-10-16T10:09:13.049166+0000","last_undegraded":"2023-10-16T10:09:13.049166+0000","last_fullsized":"2023-10-16T10:09:13.049166+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:08:32.027249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.226366+0000","last_change":"2023-10-16T10:09:10.201750+0000","last_active":"2023-10-16T10:09:11.226366+0000","last_peered":"2023-10-16T10:09:11.226366+0000","last_clean":"2023-10-16T10:09:11.226366+0000","last_became_active":"2023-10-16T10:09:10.201454+0000","last_became_peered":"2023-10-16T10:09:10.201454+0000","last_unstale":"2023-10-16T10:09:11.226366+0000","last_undegraded":"2023-10-16T10:09:11.226366+0000","last_fullsized":"2023-10-16T10:09:11.226366+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:10:45.492873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.049956+0000","last_change":"2023-10-16T10:09:11.233467+0000","last_active":"2023-10-16T10:09:13.049956+0000","last_peered":"2023-10-16T10:09:13.049956+0000","last_clean":"2023-10-16T10:09:13.049956+0000","last_became_active":"2023-10-16T10:09:11.232187+0000","last_became_peered":"2023-10-16T10:09:11.232187+0000","last_unstale":"2023-10-16T10:09:13.049956+0000","last_undegraded":"2023-10-16T10:09:13.049956+0000","last_fullsized":"2023-10-16T10:09:13.049956+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:35:51.291548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.212388+0000","last_change":"2023-10-16T10:09:10.210941+0000","last_active":"2023-10-16T10:09:11.212388+0000","last_peered":"2023-10-16T10:09:11.212388+0000","last_clean":"2023-10-16T10:09:11.212388+0000","last_became_active":"2023-10-16T10:09:10.209812+0000","last_became_peered":"2023-10-16T10:09:10.209812+0000","last_unstale":"2023-10-16T10:09:11.212388+0000","last_undegraded":"2023-10-16T10:09:11.212388+0000","last_fullsized":"2023-10-16T10:09:11.212388+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:18:28.997634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.048333+0000","last_change":"2023-10-16T10:09:11.245636+0000","last_active":"2023-10-16T10:09:13.048333+0000","last_peered":"2023-10-16T10:09:13.048333+0000","last_clean":"2023-10-16T10:09:13.048333+0000","last_became_active":"2023-10-16T10:09:11.245304+0000","last_became_peered":"2023-10-16T10:09:11.245304+0000","last_unstale":"2023-10-16T10:09:13.048333+0000","last_undegraded":"2023-10-16T10:09:13.048333+0000","last_fullsized":"2023-10-16T10:09:13.048333+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:47:03.765628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.214997+0000","last_change":"2023-10-16T10:09:10.206311+0000","last_active":"2023-10-16T10:09:11.214997+0000","last_peered":"2023-10-16T10:09:11.214997+0000","last_clean":"2023-10-16T10:09:11.214997+0000","last_became_active":"2023-10-16T10:09:10.206018+0000","last_became_peered":"2023-10-16T10:09:10.206018+0000","last_unstale":"2023-10-16T10:09:11.214997+0000","last_undegraded":"2023-10-16T10:09:11.214997+0000","last_fullsized":"2023-10-16T10:09:11.214997+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:36:38.202444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.047343+0000","last_change":"2023-10-16T10:09:11.234973+0000","last_active":"2023-10-16T10:09:13.047343+0000","last_peered":"2023-10-16T10:09:13.047343+0000","last_clean":"2023-10-16T10:09:13.047343+0000","last_became_active":"2023-10-16T10:09:11.234617+0000","last_became_peered":"2023-10-16T10:09:11.234617+0000","last_unstale":"2023-10-16T10:09:13.047343+0000","last_undegraded":"2023-10-16T10:09:13.047343+0000","last_fullsized":"2023-10-16T10:09:13.047343+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:39:27.178785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.230771+0000","last_change":"2023-10-16T10:09:10.208508+0000","last_active":"2023-10-16T10:09:11.230771+0000","last_peered":"2023-10-16T10:09:11.230771+0000","last_clean":"2023-10-16T10:09:11.230771+0000","last_became_active":"2023-10-16T10:09:10.208224+0000","last_became_peered":"2023-10-16T10:09:10.208224+0000","last_unstale":"2023-10-16T10:09:11.230771+0000","last_undegraded":"2023-10-16T10:09:11.230771+0000","last_fullsized":"2023-10-16T10:09:11.230771+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:13.452692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:12.208208+0000","last_change":"2023-10-16T10:09:11.229079+0000","last_active":"2023-10-16T10:09:12.208208+0000","last_peered":"2023-10-16T10:09:12.208208+0000","last_clean":"2023-10-16T10:09:12.208208+0000","last_became_active":"2023-10-16T10:09:11.228591+0000","last_became_peered":"2023-10-16T10:09:11.228591+0000","last_unstale":"2023-10-16T10:09:12.208208+0000","last_undegraded":"2023-10-16T10:09:12.208208+0000","last_fullsized":"2023-10-16T10:09:12.208208+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:37.403579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.223744+0000","last_change":"2023-10-16T10:09:10.200708+0000","last_active":"2023-10-16T10:09:11.223744+0000","last_peered":"2023-10-16T10:09:11.223744+0000","last_clean":"2023-10-16T10:09:11.223744+0000","last_became_active":"2023-10-16T10:09:10.200402+0000","last_became_peered":"2023-10-16T10:09:10.200402+0000","last_unstale":"2023-10-16T10:09:11.223744+0000","last_undegraded":"2023-10-16T10:09:11.223744+0000","last_fullsized":"2023-10-16T10:09:11.223744+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:36:06.012479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-16T10:09:13.049733+0000","last_change":"2023-10-16T10:09:11.245035+0000","last_active":"2023-10-16T10:09:13.049733+0000","last_peered":"2023-10-16T10:09:13.049733+0000","last_clean":"2023-10-16T10:09:13.049733+0000","last_became_active":"2023-10-16T10:09:11.244687+0000","last_became_peered":"2023-10-16T10:09:11.244687+0000","last_unstale":"2023-10-16T10:09:13.049733+0000","last_undegraded":"2023-10-16T10:09:13.049733+0000","last_fullsized":"2023-10-16T10:09:13.049733+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:10.164305+0000","last_clean_scrub_stamp":"2023-10-16T10:09:10.164305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:31:04.265205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.249525+0000","last_change":"2023-10-16T10:09:10.210460+0000","last_active":"2023-10-16T10:09:11.249525+0000","last_peered":"2023-10-16T10:09:11.249525+0000","last_clean":"2023-10-16T10:09:11.249525+0000","last_became_active":"2023-10-16T10:09:10.209167+0000","last_became_peered":"2023-10-16T10:09:10.209167+0000","last_unstale":"2023-10-16T10:09:11.249525+0000","last_undegraded":"2023-10-16T10:09:11.249525+0000","last_fullsized":"2023-10-16T10:09:11.249525+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:09:09.156937+0000","last_clean_scrub_stamp":"2023-10-16T10:09:09.156937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:40.618287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"140'2668","reported_seq":3209,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-16T10:09:11.213773+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T10:09:11.213773+0000","last_peered":"2023-10-16T10:09:11.213773+0000","last_clean":"2023-10-16T10:09:11.213773+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T10:09:11.213773+0000","last_undegraded":"2023-10-16T10:09:11.213773+0000","last_fullsized":"2023-10-16T10:09:11.213773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2668,"log_dups_size":0,"ondisk_log_size":2668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1312,"num_read_kb":2614,"num_write":401,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1312,"num_read_kb":2614,"num_write":401,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2668,"ondisk_log_size":2668,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575087,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33624,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338216,"statfs":{"total":96636764160,"available":96602333184,"internally_reserved":0,"allocated":2498560,"data_stored":1492893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.1100000000000001,"15min":1.1599999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.5960000000000001,"5min":1.6599999999999999,"15min":1.925},"last":1.425},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.1279999999999999,"15min":1.177},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.33600000000000002},"max":{"1min":1.5109999999999999,"5min":1.722,"15min":2.016},"last":1.2809999999999999}]},{"osd":1,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.177,"15min":1.2190000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.7749999999999999,"5min":1.788,"15min":1.923},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1439999999999999,"15min":1.2090000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.92600000000000005}]},{"osd":2,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1930000000000001,"15min":1.232},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.72,"5min":1.8109999999999999,"15min":1.8740000000000001},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.2589999999999999,"15min":1.28},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.64500000000000002},"max":{"1min":1.7709999999999999,"5min":1.8169999999999999,"15min":1.8620000000000001},"last":1.3740000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3089999999999999,"15min":1.335},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.67700000000000005},"max":{"1min":1.8700000000000001,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.302,"15min":1.351},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.603,"5min":1.8540000000000001,"15min":2.004},"last":0.84099999999999997}]},{"osd":4,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.3779999999999999,"15min":1.3620000000000001},"min":{"1min":0.88900000000000001,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.7569999999999999,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3169999999999999,"15min":1.3120000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.8360000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.5649999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.492,"15min":1.5109999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.8360000000000001,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.125},{"interface":"front","average":{"1min":1.411,"5min":1.4159999999999999,"15min":1.4239999999999999},"min":{"1min":0.84299999999999997,"5min":0.80400000000000005,"15min":0.45400000000000001},"max":{"1min":1.7649999999999999,"5min":1.9630000000000001,"15min":2.2170000000000001},"last":1.6659999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.4350000000000001,"15min":1.4039999999999999},"min":{"1min":0.85299999999999998,"5min":0.83599999999999997,"15min":0.54100000000000004},"max":{"1min":1.9810000000000001,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.605},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.615,"15min":1.611},"min":{"1min":0.80900000000000005,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0230000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.53}]}]},{"osd":6,"up_from":13,"seq":55834575087,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41356,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330484,"statfs":{"total":96636764160,"available":96594415616,"internally_reserved":0,"allocated":1892352,"data_stored":901887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.131,"15min":1.143},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.619,"5min":1.736,"15min":1.7689999999999999},"last":1.8009999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.216,"15min":1.228},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5329999999999999,"5min":1.7010000000000001,"15min":1.847},"last":1.1850000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1459999999999999,"15min":1.1659999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.52},{"interface":"front","average":{"1min":1.171,"5min":1.278,"15min":1.286},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6539999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.3560000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.1879999999999999,"15min":1.1990000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45100000000000001},"max":{"1min":1.4710000000000001,"5min":1.851,"15min":2.036},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.248,"15min":1.296},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.58999999999999997},"max":{"1min":1.6859999999999999,"5min":1.925,"15min":1.925},"last":1.7230000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1579999999999999,"15min":1.202},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":2.8239999999999998},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.1759999999999999,"15min":1.218},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5049999999999999,"5min":1.6719999999999999,"15min":1.865},"last":1.125}]},{"osd":4,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.28,"15min":1.2490000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.46000000000000002},"max":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":2.1629999999999998},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.423,"15min":1.4239999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.9730000000000001,"5min":2,"15min":3.6880000000000002},"last":1.476}]},{"osd":5,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3080000000000001,"15min":1.3160000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.38900000000000001},"max":{"1min":1.647,"5min":2.0790000000000002,"15min":2.5649999999999999},"last":1.268},{"interface":"front","average":{"1min":1.45,"5min":1.5,"15min":1.502},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.495},"max":{"1min":1.8440000000000001,"5min":1.996,"15min":2.3849999999999998},"last":1.865}]},{"osd":7,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.478,"15min":1.47},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":3.5739999999999998},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.512,"15min":1.488},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.4249999999999998},"last":1.679}]}]},{"osd":5,"up_from":13,"seq":55834575087,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41344,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330496,"statfs":{"total":96636764160,"available":96594427904,"internally_reserved":0,"allocated":1888256,"data_stored":900387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1830000000000001,"15min":1.1970000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.496},"max":{"1min":1.5149999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.036},{"interface":"front","average":{"1min":1.23,"5min":1.226,"15min":1.276},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.6830000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.98299999999999998}]},{"osd":1,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.095,"15min":1.147},"min":{"1min":0.63600000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3500000000000001,"5min":1.7310000000000001,"15min":1.7729999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.105,"5min":1.143,"15min":1.1399999999999999},"min":{"1min":0.66000000000000003,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.1850000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.161,"15min":1.163},"min":{"1min":0.78700000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.218},{"interface":"front","average":{"1min":1.236,"5min":1.214,"15min":1.238},"min":{"1min":0.82199999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.653,"5min":1.6739999999999999,"15min":1.8109999999999999},"last":1.3500000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.2370000000000001,"15min":1.2629999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5840000000000001,"5min":1.915,"15min":1.97},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.3049999999999999,"15min":1.3440000000000001},"min":{"1min":0.88100000000000001,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6850000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.5089999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.276,"15min":1.2709999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.282},{"interface":"front","average":{"1min":1.468,"5min":1.5,"15min":1.498},"min":{"1min":0.878,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.9490000000000001,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.3939999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.335,"15min":1.359},"min":{"1min":0.96099999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.7649999999999999,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.405,"15min":1.3839999999999999},"min":{"1min":0.83599999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.4410000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.474,"15min":1.4670000000000001},"min":{"1min":1.0029999999999999,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.849,"5min":2.1400000000000001,"15min":2.1989999999999998},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.581,"5min":1.5509999999999999,"15min":1.5669999999999999},"min":{"1min":1.032,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.0099999999999998,"5min":2.234,"15min":2.234},"last":1.7789999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575088,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46908,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324932,"statfs":{"total":96636764160,"available":96588730368,"internally_reserved":0,"allocated":1884160,"data_stored":900345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.093,"15min":1.139},"min":{"1min":0.69399999999999995,"5min":0.64300000000000002,"15min":0.38500000000000001},"max":{"1min":1.792,"5min":1.792,"15min":1.827},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.2150000000000001,"15min":1.264},"min":{"1min":0.64700000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.712,"5min":1.9359999999999999,"15min":11.974},"last":0.79000000000000004}]},{"osd":1,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.23,"15min":1.2909999999999999},"min":{"1min":0.752,"5min":0.626,"15min":0.626},"max":{"1min":1.526,"5min":1.8939999999999999,"15min":9.9939999999999998},"last":1.28},{"interface":"front","average":{"1min":1.113,"5min":1.1359999999999999,"15min":1.216},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.655,"5min":1.655,"15min":5.9420000000000002},"last":1.4410000000000001}]},{"osd":2,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.244,"15min":1.25},"min":{"1min":0.78100000000000003,"5min":0.61799999999999999,"15min":0.59399999999999997},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.304,"15min":1.371},"min":{"1min":0.76000000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.4650000000000001,"5min":2.1219999999999999,"15min":5.0549999999999997},"last":1.121}]},{"osd":3,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.2569999999999999,"15min":1.2989999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.849,"5min":1.849,"15min":6.8079999999999998},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.361,"15min":1.3759999999999999},"min":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.67700000000000005},"max":{"1min":1.6890000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.643}]},{"osd":5,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.401,"15min":1.431},"min":{"1min":0.91300000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.647,"5min":2.1539999999999999,"15min":8.702},"last":1.375},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.357,"15min":1.3680000000000001},"min":{"1min":0.90500000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.675,"5min":2.331,"15min":3.0430000000000001},"last":1.5549999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.431,"15min":1.425},"min":{"1min":1.1080000000000001,"5min":0.84699999999999998,"15min":0.79000000000000004},"max":{"1min":1.8029999999999999,"5min":2.121,"15min":8.0579999999999998},"last":1.738},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.5429999999999999,"15min":1.5489999999999999},"min":{"1min":1.093,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.0609999999999999,"5min":2.2549999999999999,"15min":6.4089999999999998},"last":1.601}]},{"osd":7,"last update":"Mon Oct 16 10:08:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.488,"15min":1.4890000000000001},"min":{"1min":1.0649999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.742,"5min":2.3410000000000002,"15min":2.9399999999999999},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.554,"5min":1.5680000000000001,"15min":1.5700000000000001},"min":{"1min":1.1879999999999999,"5min":1.071,"15min":0.96799999999999997},"max":{"1min":2.004,"5min":2.2690000000000001,"15min":2.6120000000000001},"last":1.804}]}]},{"osd":3,"up_from":13,"seq":55834575087,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41344,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330496,"statfs":{"total":96636764160,"available":96594427904,"internally_reserved":0,"allocated":1888256,"data_stored":900435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:08:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81000000000000005,"15min":0.84399999999999997},"min":{"1min":0.434,"5min":0.30299999999999999,"15min":0.24399999999999999},"max":{"1min":1.6040000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.90000000000000002,"15min":0.98499999999999999},"min":{"1min":0.35299999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.82799999999999996}]},{"osd":1,"last update":"Mon Oct 16 10:08:35 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.83999999999999997,"15min":0.91300000000000003},"min":{"1min":0.48899999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.282,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.93000000000000005,"15min":1.0229999999999999},"min":{"1min":0.41599999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.77,"5min":3.363,"15min":3.363},"last":0.54000000000000004}]},{"osd":2,"last update":"Mon Oct 16 10:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.98099999999999998,"15min":1.042},"min":{"1min":0.432,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.907,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.04,"5min":1.036,"15min":1.141},"min":{"1min":0.58299999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.512,"5min":4.1230000000000002,"15min":4.1230000000000002},"last":0.88900000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.2030000000000001,"15min":1.2549999999999999},"min":{"1min":0.64500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.6739999999999999,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.256,"5min":1.2649999999999999,"15min":1.3080000000000001},"min":{"1min":0.72799999999999998,"5min":0.56499999999999995,"15min":0.54700000000000004},"max":{"1min":2.4249999999999998,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":1.024}]},{"osd":5,"last update":"Mon Oct 16 10:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.2450000000000001,"15min":1.2989999999999999},"min":{"1min":0.69799999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.7809999999999999,"5min":4.1829999999999998,"15min":4.1829999999999998},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2390000000000001,"15min":1.3109999999999999},"min":{"1min":0.67600000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.198,"5min":2.198,"15min":2.3140000000000001},"last":1.4430000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.359,"15min":1.4039999999999999},"min":{"1min":0.874,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.9199999999999999,"5min":4.0069999999999997,"15min":4.0069999999999997},"last":0.998},{"interface":"front","average":{"1min":1.375,"5min":1.353,"15min":1.4179999999999999},"min":{"1min":0.81000000000000005,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.1030000000000002,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.401}]},{"osd":7,"last update":"Mon Oct 16 10:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.375,"15min":1.419},"min":{"1min":0.92100000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.7269999999999999,"5min":3.649,"15min":3.649},"last":1.147},{"interface":"front","average":{"1min":1.464,"5min":1.474,"15min":1.5329999999999999},"min":{"1min":0.88100000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":3.0710000000000002,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":1.3149999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575086,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59396,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312444,"statfs":{"total":96636764160,"available":96575942656,"internally_reserved":0,"allocated":1888256,"data_stored":900891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:08:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.875,"15min":0.874},"min":{"1min":0.432,"5min":0.35299999999999998,"15min":0.26500000000000001},"max":{"1min":1.1930000000000001,"5min":5.1829999999999998,"15min":5.1829999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.93899999999999995,"15min":1.0049999999999999},"min":{"1min":0.38900000000000001,"5min":0.35999999999999999,"15min":0.34999999999999998},"max":{"1min":1.3959999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.476}]},{"osd":1,"last update":"Mon Oct 16 10:08:38 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.024,"15min":1.0349999999999999},"min":{"1min":0.48999999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.407,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.997,"15min":1.077},"min":{"1min":0.441,"5min":0.38300000000000001,"15min":0.35499999999999998},"max":{"1min":1.331,"5min":1.4770000000000001,"15min":1.792},"last":1.6220000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:08:38 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.91200000000000003,"15min":0.99399999999999999},"min":{"1min":0.48399999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.776},"last":1.411},{"interface":"front","average":{"1min":1.077,"5min":1.139,"15min":1.175},"min":{"1min":0.46200000000000002,"5min":0.45500000000000002,"15min":0.41899999999999998},"max":{"1min":1.6910000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":1.288}]},{"osd":4,"last update":"Mon Oct 16 10:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3089999999999999,"15min":1.329},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.40300000000000002},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.153,"5min":1.238,"15min":1.29},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.46000000000000002},"max":{"1min":1.4650000000000001,"5min":3.262,"15min":3.262},"last":1.5629999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.387,"15min":1.381},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.59399999999999997},"max":{"1min":2.2919999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.661},{"interface":"front","average":{"1min":1.258,"5min":1.319,"15min":1.3759999999999999},"min":{"1min":0.76300000000000001,"5min":0.72699999999999998,"15min":0.60999999999999999},"max":{"1min":1.823,"5min":1.885,"15min":1.956},"last":1.821}]},{"osd":6,"last update":"Mon Oct 16 10:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2809999999999999,"15min":1.3260000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.7290000000000001,"5min":3.169,"15min":3.169},"last":1.7569999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.4139999999999999,"15min":1.4339999999999999},"min":{"1min":0.69599999999999995,"5min":0.63,"15min":0.63},"max":{"1min":2.177,"5min":3.8340000000000001,"15min":3.8340000000000001},"last":1.593}]},{"osd":7,"last update":"Mon Oct 16 10:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.401,"15min":1.4530000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.67000000000000004},"max":{"1min":1.6479999999999999,"5min":2.012,"15min":2.262},"last":1.9570000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3879999999999999,"15min":1.4199999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.948,"5min":3.3780000000000001,"15min":3.3780000000000001},"last":1.879}]}]},{"osd":1,"up_from":13,"seq":55834575086,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55228,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316612,"statfs":{"total":96636764160,"available":96580210688,"internally_reserved":0,"allocated":1880064,"data_stored":900275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.77500000000000002,"15min":0.84399999999999997},"min":{"1min":0.34399999999999997,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":1.0309999999999999,"5min":1.4690000000000001,"15min":1.631},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.92500000000000004,"15min":0.97899999999999998},"min":{"1min":0.38200000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.5509999999999999,"5min":1.6200000000000001,"15min":1.798},"last":0.60999999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.90100000000000002,"15min":0.96299999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2529999999999999,"5min":1.5069999999999999,"15min":1.673},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.95199999999999996,"5min":1.0640000000000001,"15min":1.1930000000000001},"min":{"1min":0.56000000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3919999999999999,"5min":1.609,"15min":1.7669999999999999},"last":0.64400000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.0009999999999999,"15min":1.097},"min":{"1min":0.51800000000000002,"5min":0.42999999999999999,"15min":0.42599999999999999},"max":{"1min":1.4550000000000001,"5min":1.798,"15min":1.798},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0229999999999999,"15min":1.0920000000000001},"min":{"1min":0.47599999999999998,"5min":0.44,"15min":0.40699999999999997},"max":{"1min":1.5489999999999999,"5min":1.6879999999999999,"15min":1.8280000000000001},"last":0.91800000000000004}]},{"osd":4,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1599999999999999,"15min":1.212},"min":{"1min":0.69099999999999995,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5860000000000001,"5min":1.7450000000000001,"15min":1.885},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.2689999999999999,"15min":1.341},"min":{"1min":0.64700000000000002,"5min":0.63200000000000001,"15min":0.60999999999999999},"max":{"1min":1.605,"5min":1.8680000000000001,"15min":2.1320000000000001},"last":1.2909999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2589999999999999,"15min":1.3180000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.528},{"interface":"front","average":{"1min":1.232,"5min":1.238,"15min":1.302},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.637,"5min":1.7290000000000001,"15min":2.0470000000000002},"last":1.369}]},{"osd":6,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.3169999999999999,"15min":1.381},"min":{"1min":0.72099999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.349},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.3089999999999999,"15min":1.369},"min":{"1min":0.68600000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.762,"5min":1.8779999999999999,"15min":2.1680000000000001},"last":1.4039999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:08:26 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.331,"15min":1.4330000000000001},"min":{"1min":0.73799999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.6950000000000001,"5min":2.0739999999999998,"15min":2.1699999999999999},"last":1.498},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4319999999999999,"15min":1.484},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.2709999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575087,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47492,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324348,"statfs":{"total":96636764160,"available":96588132352,"internally_reserved":0,"allocated":2473984,"data_stored":1490643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.80800000000000005,"15min":0.84799999999999998},"min":{"1min":0.35599999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.532,"5min":1.7589999999999999,"15min":2.766},"last":1.01},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.92900000000000005,"15min":1.0129999999999999},"min":{"1min":0.51600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.228,"5min":1.581,"15min":3.96},"last":0.73999999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.97299999999999998,"15min":1.028},"min":{"1min":0.54200000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.278,"5min":2.0449999999999999,"15min":2.9260000000000002},"last":0.87},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.98999999999999999,"15min":1.0660000000000001},"min":{"1min":0.47899999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.52,"5min":1.7789999999999999,"15min":4.4290000000000003},"last":1.141}]},{"osd":3,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.91100000000000003,"15min":1.012},"min":{"1min":0.5,"5min":0.41999999999999998,"15min":0.38},"max":{"1min":1.585,"5min":1.5920000000000001,"15min":4.7380000000000004},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.103,"15min":1.1859999999999999},"min":{"1min":0.51200000000000001,"5min":0.46800000000000003,"15min":0.44},"max":{"1min":1.3959999999999999,"5min":2.1859999999999999,"15min":5.1500000000000004},"last":1.3089999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2609999999999999,"15min":1.3200000000000001},"min":{"1min":0.83199999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5760000000000001,"5min":1.859,"15min":4.1970000000000001},"last":1.464},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1910000000000001,"15min":1.276},"min":{"1min":0.83899999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.3999999999999999,"5min":1.74,"15min":4.9630000000000001},"last":1.3660000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.202,"15min":1.296},"min":{"1min":0.877,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6799999999999999,"5min":1.6890000000000001,"15min":4.8600000000000003},"last":1.569},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3300000000000001,"15min":1.385},"min":{"1min":0.76500000000000001,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6299999999999999,"5min":2.1269999999999998,"15min":4.6779999999999999},"last":1.202}]},{"osd":6,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.407,"15min":1.4330000000000001},"min":{"1min":0.93000000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.7390000000000001,"5min":2.1030000000000002,"15min":3.8250000000000002},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.339,"15min":1.4139999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.782,"5min":2.02,"15min":4.0979999999999999},"last":1.252}]},{"osd":7,"last update":"Mon Oct 16 10:08:29 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.4179999999999999,"15min":1.4710000000000001},"min":{"1min":0.79900000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.8600000000000001,"5min":2.0750000000000002,"15min":4.3369999999999997},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.3089999999999999,"15min":1.417},"min":{"1min":0.86299999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4650000000000001,"5min":1.9670000000000001,"15min":5.0570000000000004},"last":1.5049999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:09:16.172 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T10:09:16.173 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T10:09:16.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.436+0000 7f4dad64d640 1 Processor -- start 2023-10-16T10:09:16.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.436+0000 7f4dad64d640 1 -- start start 2023-10-16T10:09:16.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.437+0000 7f4dad64d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4da815cba0 0x7f4da8071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:16.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.437+0000 7f4dad64d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4da80720d0 0x7f4da80cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:16.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.438+0000 7f4dad64d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da80d0500 con 0x7f4da80cdd40 2023-10-16T10:09:16.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.438+0000 7f4dad64d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da80ce1b0 con 0x7f4da815cba0 2023-10-16T10:09:16.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.438+0000 7f4dad64d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da80d07c0 con 0x7f4da80720d0 2023-10-16T10:09:16.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.438+0000 7f4da7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4da815cba0 0x7f4da8071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:16.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.438+0000 7f4dace4c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4da80cdd40 0x7f4da81607e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35386/0 (socket says 172.21.15.84:35386) 2023-10-16T10:09:16.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.438+0000 7f4dace4c640 1 -- 172.21.15.84:0/2110990595 learned_addr learned my addr 172.21.15.84:0/2110990595 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:09:16.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.438+0000 7f4da77fe640 1 --2- 172.21.15.84:0/2110990595 >> v2:172.21.15.157:3300/0 conn(0x7f4da80720d0 0x7f4da80cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:16.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.438+0000 7f4da7fff640 1 -- 172.21.15.84:0/2110990595 >> v2:172.21.15.157:3300/0 conn(0x7f4da80720d0 msgr2=0x7f4da80cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:16.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.438+0000 7f4da7fff640 1 --2- 172.21.15.84:0/2110990595 >> v2:172.21.15.157:3300/0 conn(0x7f4da80720d0 0x7f4da80cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.438+0000 7f4da7fff640 1 -- 172.21.15.84:0/2110990595 >> v1:172.21.15.84:6789/0 conn(0x7f4da80cdd40 legacy=0x7f4da81607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:16.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.438+0000 7f4da7fff640 1 -- 172.21.15.84:0/2110990595 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da80d0aa0 con 0x7f4da815cba0 2023-10-16T10:09:16.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.438+0000 7f4da77fe640 1 --2- 172.21.15.84:0/2110990595 >> v2:172.21.15.157:3300/0 conn(0x7f4da80720d0 0x7f4da80cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:16.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.439+0000 7f4da7fff640 1 --2- 172.21.15.84:0/2110990595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da815cba0 0x7f4da8071b90 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f4da80ce3f0 tx=0x7f4d9802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab30e7ec82d3cca7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:16.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.440+0000 7f4da6ffd640 1 -- 172.21.15.84:0/2110990595 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9802cc70 con 0x7f4da815cba0 2023-10-16T10:09:16.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.440+0000 7f4da6ffd640 1 -- 172.21.15.84:0/2110990595 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9802cdd0 con 0x7f4da815cba0 2023-10-16T10:09:16.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.440+0000 7f4da6ffd640 1 -- 172.21.15.84:0/2110990595 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98034660 con 0x7f4da815cba0 2023-10-16T10:09:16.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.440+0000 7f4dad64d640 1 -- 172.21.15.84:0/2110990595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da815cba0 msgr2=0x7f4da8071b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:16.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.440+0000 7f4dad64d640 1 --2- 172.21.15.84:0/2110990595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da815cba0 0x7f4da8071b90 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f4da80ce3f0 tx=0x7f4d9802bec0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.441+0000 7f4dad64d640 1 -- 172.21.15.84:0/2110990595 shutdown_connections 2023-10-16T10:09:16.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.441+0000 7f4dad64d640 1 --2- 172.21.15.84:0/2110990595 >> v2:172.21.15.157:3300/0 conn(0x7f4da80720d0 0x7f4da80cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.441+0000 7f4dad64d640 1 --2- 172.21.15.84:0/2110990595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da815cba0 0x7f4da8071b90 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.441+0000 7f4dad64d640 1 -- 172.21.15.84:0/2110990595 >> 172.21.15.84:0/2110990595 conn(0x7f4da8065ed0 msgr2=0x7f4da80662e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:16.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.441+0000 7f4dad64d640 1 -- 172.21.15.84:0/2110990595 shutdown_connections 2023-10-16T10:09:16.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.441+0000 7f4dad64d640 1 -- 172.21.15.84:0/2110990595 wait complete. 2023-10-16T10:09:16.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.442+0000 7f4dad64d640 1 Processor -- start 2023-10-16T10:09:16.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.443+0000 7f4dad64d640 1 -- start start 2023-10-16T10:09:16.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.443+0000 7f4dad64d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da80720d0 0x7f4da80b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:16.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.443+0000 7f4dad64d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da80cdd40 0x7f4da8158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:16.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.444+0000 7f4dad64d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da815cba0 0x7f4da80b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:16.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.444+0000 7f4dad64d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80b5510 con 0x7f4da80720d0 2023-10-16T10:09:16.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.444+0000 7f4dad64d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da8159720 con 0x7f4da80cdd40 2023-10-16T10:09:16.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.444+0000 7f4dad64d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da8159180 con 0x7f4da815cba0 2023-10-16T10:09:16.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.444+0000 7f4da7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da80720d0 0x7f4da80b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:16.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.444+0000 7f4da7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da80720d0 0x7f4da80b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39934/0 (socket says 172.21.15.84:39934) 2023-10-16T10:09:16.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.444+0000 7f4da7fff640 1 -- 172.21.15.84:0/374626925 learned_addr learned my addr 172.21.15.84:0/374626925 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:16.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.444+0000 7f4da77fe640 1 --2- 172.21.15.84:0/374626925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da80cdd40 0x7f4da8158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:16.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.444+0000 7f4da7fff640 1 -- 172.21.15.84:0/374626925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da815cba0 msgr2=0x7f4da80b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.444+0000 7f4da7fff640 1 --2- 172.21.15.84:0/374626925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da815cba0 0x7f4da80b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.444+0000 7f4da7fff640 1 -- 172.21.15.84:0/374626925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da80cdd40 msgr2=0x7f4da8158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.444+0000 7f4da7fff640 1 --2- 172.21.15.84:0/374626925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da80cdd40 0x7f4da8158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.444+0000 7f4da7fff640 1 -- 172.21.15.84:0/374626925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d98009a70 con 0x7f4da80720d0 2023-10-16T10:09:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.445+0000 7f4da7fff640 1 --2- 172.21.15.84:0/374626925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da80720d0 0x7f4da80b4fd0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f4d98037040 tx=0x7f4d98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.445+0000 7f4da4ff9640 1 -- 172.21.15.84:0/374626925 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98040650 con 0x7f4da80720d0 2023-10-16T10:09:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.445+0000 7f4da4ff9640 1 -- 172.21.15.84:0/374626925 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d980407b0 con 0x7f4da80720d0 2023-10-16T10:09:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.445+0000 7f4dad64d640 1 -- 172.21.15.84:0/374626925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da815e4e0 con 0x7f4da80720d0 2023-10-16T10:09:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.445+0000 7f4da4ff9640 1 -- 172.21.15.84:0/374626925 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98040b60 con 0x7f4da80720d0 2023-10-16T10:09:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.445+0000 7f4dad64d640 1 -- 172.21.15.84:0/374626925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da815eaa0 con 0x7f4da80720d0 2023-10-16T10:09:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.446+0000 7f4dad64d640 1 -- 172.21.15.84:0/374626925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da804e9b0 con 0x7f4da80720d0 2023-10-16T10:09:16.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.447+0000 7f4da4ff9640 1 -- 172.21.15.84:0/374626925 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4d98040cc0 con 0x7f4da80720d0 2023-10-16T10:09:16.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.448+0000 7f4da4ff9640 1 --2- 172.21.15.84:0/374626925 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f4d8806af90 0x7f4d8806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.448+0000 7f4da4ff9640 1 -- 172.21.15.84:0/374626925 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4d980ac680 con 0x7f4da80720d0 2023-10-16T10:09:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.448+0000 7f4da77fe640 1 --2- 172.21.15.84:0/374626925 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f4d8806af90 0x7f4d8806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.449+0000 7f4da77fe640 1 --2- 172.21.15.84:0/374626925 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f4d8806af90 0x7f4d8806d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4da81f5830 tx=0x7f4d90008040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.452+0000 7f4da4ff9640 1 -- 172.21.15.84:0/374626925 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4d98077600 con 0x7f4da80720d0 2023-10-16T10:09:16.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.711+0000 7f4dad64d640 1 -- 172.21.15.84:0/374626925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4da81612a0 con 0x7f4da80720d0 2023-10-16T10:09:16.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.715+0000 7f4da4ff9640 1 -- 172.21.15.84:0/374626925 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) v1 ==== 190+0+147 (secure 0 0 0) 0x7f4d9807ac20 con 0x7f4da80720d0 2023-10-16T10:09:16.717 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2023-10-16T10:09:16.717 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ== 2023-10-16T10:09:16.717 INFO:teuthology.orchestra.run.smithi084.stdout: caps mds = "allow" 2023-10-16T10:09:16.717 INFO:teuthology.orchestra.run.smithi084.stdout: caps mon = "allow r" 2023-10-16T10:09:16.717 INFO:teuthology.orchestra.run.smithi084.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T10:09:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-16T10:09:16.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.718+0000 7f4dad64d640 1 -- 172.21.15.84:0/374626925 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f4d8806af90 msgr2=0x7f4d8806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:16.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.718+0000 7f4dad64d640 1 --2- 172.21.15.84:0/374626925 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f4d8806af90 0x7f4d8806d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4da81f5830 tx=0x7f4d90008040 comp rx=0 tx=0).stop 2023-10-16T10:09:16.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.718+0000 7f4dad64d640 1 -- 172.21.15.84:0/374626925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da80720d0 msgr2=0x7f4da80b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:16.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.718+0000 7f4dad64d640 1 --2- 172.21.15.84:0/374626925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da80720d0 0x7f4da80b4fd0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f4d98037040 tx=0x7f4d98004a40 comp rx=0 tx=0).stop 2023-10-16T10:09:16.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.718+0000 7f4dad64d640 1 -- 172.21.15.84:0/374626925 shutdown_connections 2023-10-16T10:09:16.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.718+0000 7f4dad64d640 1 --2- 172.21.15.84:0/374626925 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f4d8806af90 0x7f4d8806d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.718+0000 7f4dad64d640 1 --2- 172.21.15.84:0/374626925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da815cba0 0x7f4da80b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.718+0000 7f4dad64d640 1 --2- 172.21.15.84:0/374626925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da80cdd40 0x7f4da8158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.718+0000 7f4dad64d640 1 --2- 172.21.15.84:0/374626925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da80720d0 0x7f4da80b4fd0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.718+0000 7f4dad64d640 1 -- 172.21.15.84:0/374626925 >> 172.21.15.84:0/374626925 conn(0x7f4da8065ed0 msgr2=0x7f4da806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:16.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.718+0000 7f4dad64d640 1 -- 172.21.15.84:0/374626925 shutdown_connections 2023-10-16T10:09:16.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.718+0000 7f4dad64d640 1 -- 172.21.15.84:0/374626925 wait complete. 2023-10-16T10:09:16.738 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:09:16.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.940+0000 7f05a371d640 1 Processor -- start 2023-10-16T10:09:16.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.940+0000 7f05a371d640 1 -- start start 2023-10-16T10:09:16.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.942+0000 7f05a371d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f059c159f00 0x7f059c0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:16.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.942+0000 7f05a371d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f059c15d880 0x7f059c0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:16.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.942+0000 7f05a371d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059c06f7f0 con 0x7f059c159b20 2023-10-16T10:09:16.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.942+0000 7f05a371d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059c06c5e0 con 0x7f059c159f00 2023-10-16T10:09:16.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.942+0000 7f05a371d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059c06c8c0 con 0x7f059c15d880 2023-10-16T10:09:16.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.942+0000 7f05a0cb8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f059c159f00 0x7f059c0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:16.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.942+0000 7f05a0cb8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f059c159f00 0x7f059c0b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39946/0 (socket says 172.21.15.84:39946) 2023-10-16T10:09:16.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.942+0000 7f05a0cb8640 1 -- 172.21.15.84:0/1601917006 learned_addr learned my addr 172.21.15.84:0/1601917006 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:16.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.942+0000 7f05a0cb8640 1 -- 172.21.15.84:0/1601917006 >> v2:172.21.15.157:3300/0 conn(0x7f059c15d880 msgr2=0x7f059c0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:16.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.942+0000 7f05a1cba640 1 --2- 172.21.15.84:0/1601917006 >> v2:172.21.15.157:3300/0 conn(0x7f059c15d880 0x7f059c0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:16.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.942+0000 7f05a0cb8640 1 --2- 172.21.15.84:0/1601917006 >> v2:172.21.15.157:3300/0 conn(0x7f059c15d880 0x7f059c0b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.942+0000 7f05a0cb8640 1 -- 172.21.15.84:0/1601917006 >> v1:172.21.15.157:6789/0 conn(0x7f059c159b20 legacy=0x7f059c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:16.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.942+0000 7f05a0cb8640 1 -- 172.21.15.84:0/1601917006 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c06cba0 con 0x7f059c159f00 2023-10-16T10:09:16.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.943+0000 7f05a0cb8640 1 --2- 172.21.15.84:0/1601917006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059c159f00 0x7f059c0b0410 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f059c0c8bb0 tx=0x7f058c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e58dec5bfb22534 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:16.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.944+0000 7f0593fff640 1 -- 172.21.15.84:0/1601917006 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c031870 con 0x7f059c159f00 2023-10-16T10:09:16.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.944+0000 7f0593fff640 1 -- 172.21.15.84:0/1601917006 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c0319d0 con 0x7f059c159f00 2023-10-16T10:09:16.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.944+0000 7f0593fff640 1 -- 172.21.15.84:0/1601917006 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c031d60 con 0x7f059c159f00 2023-10-16T10:09:16.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.944+0000 7f05a371d640 1 -- 172.21.15.84:0/1601917006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059c159f00 msgr2=0x7f059c0b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:16.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.944+0000 7f05a371d640 1 --2- 172.21.15.84:0/1601917006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059c159f00 0x7f059c0b0410 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f059c0c8bb0 tx=0x7f058c02b910 comp rx=0 tx=0).stop 2023-10-16T10:09:16.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.944+0000 7f05a371d640 1 -- 172.21.15.84:0/1601917006 shutdown_connections 2023-10-16T10:09:16.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.944+0000 7f05a371d640 1 --2- 172.21.15.84:0/1601917006 >> v2:172.21.15.157:3300/0 conn(0x7f059c15d880 0x7f059c0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.944+0000 7f05a371d640 1 --2- 172.21.15.84:0/1601917006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059c159f00 0x7f059c0b0410 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.944+0000 7f05a371d640 1 -- 172.21.15.84:0/1601917006 >> 172.21.15.84:0/1601917006 conn(0x7f059c0cb410 msgr2=0x7f059c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:16.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.944+0000 7f05a371d640 1 -- 172.21.15.84:0/1601917006 shutdown_connections 2023-10-16T10:09:16.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.944+0000 7f05a371d640 1 -- 172.21.15.84:0/1601917006 wait complete. 2023-10-16T10:09:16.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.946+0000 7f05a371d640 1 Processor -- start 2023-10-16T10:09:16.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.946+0000 7f05a371d640 1 -- start start 2023-10-16T10:09:16.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.947+0000 7f05a371d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f059c159b20 0x7f059c151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:16.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.947+0000 7f05a371d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f059c159f00 0x7f059c15be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:16.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.947+0000 7f05a371d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059c15d880 0x7f059c15a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:16.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.947+0000 7f05a371d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c15b9b0 con 0x7f059c15d880 2023-10-16T10:09:16.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.947+0000 7f05a371d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c15ada0 con 0x7f059c159f00 2023-10-16T10:09:16.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.947+0000 7f05a371d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c15b080 con 0x7f059c159b20 2023-10-16T10:09:16.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.948+0000 7f05a1cba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059c15d880 0x7f059c15a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:16.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.948+0000 7f05a1cba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059c15d880 0x7f059c15a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39960/0 (socket says 172.21.15.84:39960) 2023-10-16T10:09:16.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.948+0000 7f05a1cba640 1 -- 172.21.15.84:0/2537288470 learned_addr learned my addr 172.21.15.84:0/2537288470 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:16.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.948+0000 7f05a0cb8640 1 --2- 172.21.15.84:0/2537288470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f059c159f00 0x7f059c15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:16.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.948+0000 7f05a14b9640 1 --2- 172.21.15.84:0/2537288470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f059c159b20 0x7f059c151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:16.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.948+0000 7f05a1cba640 1 -- 172.21.15.84:0/2537288470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f059c159b20 msgr2=0x7f059c151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:16.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.948+0000 7f05a1cba640 1 --2- 172.21.15.84:0/2537288470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f059c159b20 0x7f059c151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.948+0000 7f05a1cba640 1 -- 172.21.15.84:0/2537288470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f059c159f00 msgr2=0x7f059c15be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:16.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.948+0000 7f05a1cba640 1 --2- 172.21.15.84:0/2537288470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f059c159f00 0x7f059c15be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:16.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.948+0000 7f05a1cba640 1 -- 172.21.15.84:0/2537288470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c0094d0 con 0x7f059c15d880 2023-10-16T10:09:16.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.949+0000 7f05a1cba640 1 --2- 172.21.15.84:0/2537288470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059c15d880 0x7f059c15a2e0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f059400cd70 tx=0x7f059400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:16.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.949+0000 7f0591ffb640 1 -- 172.21.15.84:0/2537288470 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594003b90 con 0x7f059c15d880 2023-10-16T10:09:16.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.949+0000 7f0591ffb640 1 -- 172.21.15.84:0/2537288470 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0594003cf0 con 0x7f059c15d880 2023-10-16T10:09:16.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.949+0000 7f0591ffb640 1 -- 172.21.15.84:0/2537288470 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594004240 con 0x7f059c15d880 2023-10-16T10:09:16.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.949+0000 7f05a371d640 1 -- 172.21.15.84:0/2537288470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059c15b360 con 0x7f059c15d880 2023-10-16T10:09:16.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.949+0000 7f05a371d640 1 -- 172.21.15.84:0/2537288470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059c152a20 con 0x7f059c15d880 2023-10-16T10:09:16.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.950+0000 7f05a371d640 1 -- 172.21.15.84:0/2537288470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059c15bb20 con 0x7f059c15d880 2023-10-16T10:09:16.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.951+0000 7f0591ffb640 1 -- 172.21.15.84:0/2537288470 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f05940043a0 con 0x7f059c15d880 2023-10-16T10:09:16.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.952+0000 7f0591ffb640 1 --2- 172.21.15.84:0/2537288470 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f058006af90 0x7f058006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:16.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.952+0000 7f0591ffb640 1 -- 172.21.15.84:0/2537288470 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0594012070 con 0x7f059c15d880 2023-10-16T10:09:16.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.952+0000 7f05a14b9640 1 --2- 172.21.15.84:0/2537288470 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f058006af90 0x7f058006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:16.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.953+0000 7f05a14b9640 1 --2- 172.21.15.84:0/2537288470 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f058006af90 0x7f058006d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0598006fd0 tx=0x7f0598008040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:16.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:16.956+0000 7f0591ffb640 1 -- 172.21.15.84:0/2537288470 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0594055120 con 0x7f059c15d880 2023-10-16T10:09:17.157 INFO:tasks.ceph.mds.c.smithi084.stderr:2023-10-16T10:09:17.155+0000 7fd16363c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:09:17.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.169+0000 7f05a371d640 1 -- 172.21.15.84:0/2537288470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f059c04e160 con 0x7f059c15d880 2023-10-16T10:09:17.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.170+0000 7f0591ffb640 1 -- 172.21.15.84:0/2537288470 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f0594058740 con 0x7f059c15d880 2023-10-16T10:09:17.172 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:09:17.173 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6517,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1733540616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1733540616},{"type":"v1","addr":"172.21.15.157:6833","nonce":1733540616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3815048417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3815048417},{"type":"v1","addr":"172.21.15.157:6835","nonce":3815048417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9135,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1038548935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1038548935},{"type":"v1","addr":"172.21.15.84:6836","nonce":1038548935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:09:10.652604+0000","modified":"2023-10-16T10:09:13.730838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9138},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9138":{"gid":9138,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2797149326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2797149326},{"type":"v1","addr":"172.21.15.84:6837","nonce":2797149326}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-16T10:09:17.173 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 80 2023-10-16T10:09:17.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.173+0000 7f05a371d640 1 -- 172.21.15.84:0/2537288470 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f058006af90 msgr2=0x7f058006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:17.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.173+0000 7f05a371d640 1 --2- 172.21.15.84:0/2537288470 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f058006af90 0x7f058006d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0598006fd0 tx=0x7f0598008040 comp rx=0 tx=0).stop 2023-10-16T10:09:17.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.173+0000 7f05a371d640 1 -- 172.21.15.84:0/2537288470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059c15d880 msgr2=0x7f059c15a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:17.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.173+0000 7f05a371d640 1 --2- 172.21.15.84:0/2537288470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059c15d880 0x7f059c15a2e0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f059400cd70 tx=0x7f059400a660 comp rx=0 tx=0).stop 2023-10-16T10:09:17.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.173+0000 7f05a371d640 1 -- 172.21.15.84:0/2537288470 shutdown_connections 2023-10-16T10:09:17.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.173+0000 7f05a371d640 1 --2- 172.21.15.84:0/2537288470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059c15d880 0x7f059c15a2e0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.173+0000 7f05a371d640 1 --2- 172.21.15.84:0/2537288470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f059c159f00 0x7f059c15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.173+0000 7f05a371d640 1 --2- 172.21.15.84:0/2537288470 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f058006af90 0x7f058006d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.173+0000 7f05a371d640 1 --2- 172.21.15.84:0/2537288470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f059c159b20 0x7f059c151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.173+0000 7f05a371d640 1 -- 172.21.15.84:0/2537288470 >> 172.21.15.84:0/2537288470 conn(0x7f059c0cb410 msgr2=0x7f059c0acf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:17.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.174+0000 7f05a371d640 1 -- 172.21.15.84:0/2537288470 shutdown_connections 2023-10-16T10:09:17.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.174+0000 7f05a371d640 1 -- 172.21.15.84:0/2537288470 wait complete. 2023-10-16T10:09:17.594 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T10:09:10.652604+0000', 'modified': '2023-10-16T10:09:13.730838+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9138}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9138': {'gid': 9138, 'name': 'c', 'rank': 0, 'incarnation': 76, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.84:6837/2797149326', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6835', 'nonce': 2797149326}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 2797149326}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T10:09:17.595 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T10:09:17.595 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-16T10:09:17.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.815+0000 7f4a4aae7640 1 Processor -- start 2023-10-16T10:09:17.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.815+0000 7f4a4aae7640 1 -- start start 2023-10-16T10:09:17.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.816+0000 7f4a4aae7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4a4415ad70 0x7f4a44069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.817+0000 7f4a4aae7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a440c9ca0 0x7f4a4406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.817+0000 7f4a4aae7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a440cb030 con 0x7f4a440ca3b0 2023-10-16T10:09:17.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.817+0000 7f4a4aae7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a440c91c0 con 0x7f4a4415ad70 2023-10-16T10:09:17.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.817+0000 7f4a4aae7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440c94a0 con 0x7f4a440c9ca0 2023-10-16T10:09:17.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.817+0000 7f4a48883640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4a4415ad70 0x7f4a44069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:17.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.817+0000 7f4a48883640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4a4415ad70 0x7f4a44069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39962/0 (socket says 172.21.15.84:39962) 2023-10-16T10:09:17.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.817+0000 7f4a48883640 1 -- 172.21.15.84:0/1837919632 learned_addr learned my addr 172.21.15.84:0/1837919632 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:17.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.817+0000 7f4a43fff640 1 --2- 172.21.15.84:0/1837919632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a440c9ca0 0x7f4a4406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:17.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.818+0000 7f4a48883640 1 -- 172.21.15.84:0/1837919632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a440c9ca0 msgr2=0x7f4a4406a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:17.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.818+0000 7f4a48883640 1 --2- 172.21.15.84:0/1837919632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a440c9ca0 0x7f4a4406a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.818+0000 7f4a48883640 1 -- 172.21.15.84:0/1837919632 >> v1:172.21.15.157:6789/0 conn(0x7f4a440ca3b0 legacy=0x7f4a440ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:17.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.818+0000 7f4a48883640 1 -- 172.21.15.84:0/1837919632 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a440c9780 con 0x7f4a4415ad70 2023-10-16T10:09:17.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.819+0000 7f4a48883640 1 --2- 172.21.15.84:0/1837919632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a4415ad70 0x7f4a44069da0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009870 tx=0x7f4a3802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e317fc5eeb8bfa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:17.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.819+0000 7f4a437fe640 1 -- 172.21.15.84:0/1837919632 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a38031640 con 0x7f4a4415ad70 2023-10-16T10:09:17.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.819+0000 7f4a437fe640 1 -- 172.21.15.84:0/1837919632 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a380317a0 con 0x7f4a4415ad70 2023-10-16T10:09:17.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.819+0000 7f4a437fe640 1 -- 172.21.15.84:0/1837919632 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a38031b30 con 0x7f4a4415ad70 2023-10-16T10:09:17.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.820+0000 7f4a4aae7640 1 -- 172.21.15.84:0/1837919632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a4415ad70 msgr2=0x7f4a44069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:17.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.820+0000 7f4a4aae7640 1 --2- 172.21.15.84:0/1837919632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a4415ad70 0x7f4a44069da0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009870 tx=0x7f4a3802b8f0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.820+0000 7f4a4aae7640 1 -- 172.21.15.84:0/1837919632 shutdown_connections 2023-10-16T10:09:17.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.820+0000 7f4a4aae7640 1 --2- 172.21.15.84:0/1837919632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a440c9ca0 0x7f4a4406a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.820+0000 7f4a4aae7640 1 --2- 172.21.15.84:0/1837919632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a4415ad70 0x7f4a44069da0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.820+0000 7f4a4aae7640 1 -- 172.21.15.84:0/1837919632 >> 172.21.15.84:0/1837919632 conn(0x7f4a440ce9b0 msgr2=0x7f4a440cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:17.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.820+0000 7f4a4aae7640 1 -- 172.21.15.84:0/1837919632 shutdown_connections 2023-10-16T10:09:17.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.820+0000 7f4a4aae7640 1 -- 172.21.15.84:0/1837919632 wait complete. 2023-10-16T10:09:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.822+0000 7f4a4aae7640 1 Processor -- start 2023-10-16T10:09:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.822+0000 7f4a4aae7640 1 -- start start 2023-10-16T10:09:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.823+0000 7f4a4aae7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a440c9ca0 0x7f4a4414aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.823+0000 7f4a4aae7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a440ca3b0 0x7f4a441efb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.823+0000 7f4a4aae7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a4415ad70 0x7f4a4415c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.823+0000 7f4a4aae7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4414aa20 con 0x7f4a440ca3b0 2023-10-16T10:09:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.823+0000 7f4a4aae7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a44147b00 con 0x7f4a440c9ca0 2023-10-16T10:09:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.823+0000 7f4a4aae7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a44147de0 con 0x7f4a4415ad70 2023-10-16T10:09:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.823+0000 7f4a43fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a440ca3b0 0x7f4a441efb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.823+0000 7f4a43fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a440ca3b0 0x7f4a441efb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39968/0 (socket says 172.21.15.84:39968) 2023-10-16T10:09:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.823+0000 7f4a43fff640 1 -- 172.21.15.84:0/2946040628 learned_addr learned my addr 172.21.15.84:0/2946040628 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.823+0000 7f4a48883640 1 --2- 172.21.15.84:0/2946040628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a440c9ca0 0x7f4a4414aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.823+0000 7f4a43fff640 1 -- 172.21.15.84:0/2946040628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a4415ad70 msgr2=0x7f4a4415c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.823+0000 7f4a49084640 1 --2- 172.21.15.84:0/2946040628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a4415ad70 0x7f4a4415c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.823+0000 7f4a43fff640 1 --2- 172.21.15.84:0/2946040628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a4415ad70 0x7f4a4415c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.823+0000 7f4a43fff640 1 -- 172.21.15.84:0/2946040628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a440c9ca0 msgr2=0x7f4a4414aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.823+0000 7f4a43fff640 1 --2- 172.21.15.84:0/2946040628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a440c9ca0 0x7f4a4414aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.823+0000 7f4a43fff640 1 -- 172.21.15.84:0/2946040628 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a380094d0 con 0x7f4a440ca3b0 2023-10-16T10:09:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.823+0000 7f4a48883640 1 --2- 172.21.15.84:0/2946040628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a440c9ca0 0x7f4a4414aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.824+0000 7f4a43fff640 1 --2- 172.21.15.84:0/2946040628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a440ca3b0 0x7f4a441efb60 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000d800 tx=0x7f4a3000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.824+0000 7f4a417fa640 1 -- 172.21.15.84:0/2946040628 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000b6c0 con 0x7f4a440ca3b0 2023-10-16T10:09:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.824+0000 7f4a417fa640 1 -- 172.21.15.84:0/2946040628 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3000b820 con 0x7f4a440ca3b0 2023-10-16T10:09:17.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.824+0000 7f4a4aae7640 1 -- 172.21.15.84:0/2946040628 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a441480c0 con 0x7f4a440ca3b0 2023-10-16T10:09:17.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.824+0000 7f4a417fa640 1 -- 172.21.15.84:0/2946040628 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000bbb0 con 0x7f4a440ca3b0 2023-10-16T10:09:17.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.824+0000 7f4a4aae7640 1 -- 172.21.15.84:0/2946040628 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a44148680 con 0x7f4a440ca3b0 2023-10-16T10:09:17.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.826+0000 7f4a417fa640 1 -- 172.21.15.84:0/2946040628 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4a3000bd10 con 0x7f4a440ca3b0 2023-10-16T10:09:17.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.828+0000 7f4a417fa640 1 --2- 172.21.15.84:0/2946040628 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f4a2806af90 0x7f4a2806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.828+0000 7f4a417fa640 1 -- 172.21.15.84:0/2946040628 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4a30089d20 con 0x7f4a440ca3b0 2023-10-16T10:09:17.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.828+0000 7f4a48883640 1 --2- 172.21.15.84:0/2946040628 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f4a2806af90 0x7f4a2806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:17.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.828+0000 7f4a4d680740 1 Processor -- start 2023-10-16T10:09:17.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.829+0000 7f4a4d680740 1 -- start start 2023-10-16T10:09:17.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.829+0000 7f4a48883640 1 --2- 172.21.15.84:0/2946040628 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f4a2806af90 0x7f4a2806d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4a38036000 tx=0x7f4a3803c040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:17.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.829+0000 7f4a4d680740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb91f80 0x55574fb64e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.830+0000 7f4a4d680740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb92360 0x55574fb94d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.830+0000 7f4a4d680740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb653c0 0x55574fb95250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.830+0000 7f4a4d680740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574fa857a0 con 0x55574fb92360 2023-10-16T10:09:17.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.830+0000 7f4a4d680740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574fad7740 con 0x55574fb653c0 2023-10-16T10:09:17.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.830+0000 7f4a4d680740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55574fad4e30 con 0x55574fb91f80 2023-10-16T10:09:17.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.830+0000 7f4a43fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb92360 0x55574fb94d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:17.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.830+0000 7f4a43fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb92360 0x55574fb94d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39984/0 (socket says 172.21.15.84:39984) 2023-10-16T10:09:17.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.830+0000 7f4a43fff640 1 -- 172.21.15.84:0/3179945467 learned_addr learned my addr 172.21.15.84:0/3179945467 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:17.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.830+0000 7f4a49084640 1 --2- 172.21.15.84:0/3179945467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb653c0 0x55574fb95250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:17.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.830+0000 7f4a48883640 1 --2- 172.21.15.84:0/3179945467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb91f80 0x55574fb64e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:17.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.830+0000 7f4a43fff640 1 -- 172.21.15.84:0/3179945467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb91f80 msgr2=0x55574fb64e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:17.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.830+0000 7f4a43fff640 1 --2- 172.21.15.84:0/3179945467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb91f80 0x55574fb64e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.830+0000 7f4a43fff640 1 -- 172.21.15.84:0/3179945467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb653c0 msgr2=0x55574fb95250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:17.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.830+0000 7f4a43fff640 1 --2- 172.21.15.84:0/3179945467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb653c0 0x55574fb95250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.830+0000 7f4a43fff640 1 -- 172.21.15.84:0/3179945467 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55574fa94910 con 0x55574fb92360 2023-10-16T10:09:17.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.830+0000 7f4a48883640 1 --2- 172.21.15.84:0/3179945467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb91f80 0x55574fb64e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:17.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.830+0000 7f4a49084640 1 --2- 172.21.15.84:0/3179945467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb653c0 0x55574fb95250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:17.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.831+0000 7f4a43fff640 1 --2- 172.21.15.84:0/3179945467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb92360 0x55574fb94d10 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f4a30088100 tx=0x7f4a30059190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5ddcef0f333c31 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:17.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.831+0000 7f4a2e7fc640 1 -- 172.21.15.84:0/3179945467 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30085020 con 0x55574fb92360 2023-10-16T10:09:17.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.831+0000 7f4a2e7fc640 1 -- 172.21.15.84:0/3179945467 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a30060c50 con 0x55574fb92360 2023-10-16T10:09:17.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.831+0000 7f4a2e7fc640 1 -- 172.21.15.84:0/3179945467 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30088180 con 0x55574fb92360 2023-10-16T10:09:17.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.831+0000 7f4a4d680740 1 -- 172.21.15.84:0/3179945467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb92360 msgr2=0x55574fb94d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:17.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.831+0000 7f4a4d680740 1 --2- 172.21.15.84:0/3179945467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb92360 0x55574fb94d10 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f4a30088100 tx=0x7f4a30059190 comp rx=0 tx=0).stop 2023-10-16T10:09:17.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.831+0000 7f4a4d680740 1 -- 172.21.15.84:0/3179945467 shutdown_connections 2023-10-16T10:09:17.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.831+0000 7f4a4d680740 1 --2- 172.21.15.84:0/3179945467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb653c0 0x55574fb95250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.831+0000 7f4a4d680740 1 --2- 172.21.15.84:0/3179945467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb92360 0x55574fb94d10 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.831+0000 7f4a4d680740 1 --2- 172.21.15.84:0/3179945467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb91f80 0x55574fb64e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.831+0000 7f4a4d680740 1 -- 172.21.15.84:0/3179945467 >> 172.21.15.84:0/3179945467 conn(0x55574fb67190 msgr2=0x55574fbafd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:17.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.832+0000 7f4a4d680740 1 -- 172.21.15.84:0/3179945467 shutdown_connections 2023-10-16T10:09:17.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.832+0000 7f4a4d680740 1 -- 172.21.15.84:0/3179945467 wait complete. 2023-10-16T10:09:17.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.832+0000 7f4a4d680740 1 -- start start 2023-10-16T10:09:17.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.833+0000 7f4a4d680740 1 Processor -- start 2023-10-16T10:09:17.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.833+0000 7f4a4d680740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb91f80 0x55574fb932a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.834+0000 7f4a4d680740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb92360 0x55574fb937e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.834+0000 7f4a4d680740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb8d1e0 0x55574fb8d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.834+0000 7f4a4d680740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574fad0720 con 0x55574fb8d1e0 2023-10-16T10:09:17.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.834+0000 7f4a4d680740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574fa6e970 con 0x55574fb91f80 2023-10-16T10:09:17.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.834+0000 7f4a4d680740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55574fa6e610 con 0x55574fb92360 2023-10-16T10:09:17.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.834+0000 7f4a4d680740 10 client.0 _ll_register_callbacks cb 0x55574fa909d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:09:17.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.834+0000 7f4a49084640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb8d1e0 0x55574fb8d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:17.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.834+0000 7f4a49084640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb8d1e0 0x55574fb8d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39996/0 (socket says 172.21.15.84:39996) 2023-10-16T10:09:17.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.834+0000 7f4a49084640 1 -- 172.21.15.84:0/2432560197 learned_addr learned my addr 172.21.15.84:0/2432560197 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:17.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.834+0000 7f4a43fff640 1 --2- 172.21.15.84:0/2432560197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb92360 0x55574fb937e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:17.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.834+0000 7f4a48883640 1 --2- 172.21.15.84:0/2432560197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb91f80 0x55574fb932a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:17.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.834+0000 7f4a49084640 1 -- 172.21.15.84:0/2432560197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb92360 msgr2=0x55574fb937e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:17.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.834+0000 7f4a49084640 1 --2- 172.21.15.84:0/2432560197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb92360 0x55574fb937e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.834+0000 7f4a49084640 1 -- 172.21.15.84:0/2432560197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb91f80 msgr2=0x55574fb932a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:17.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.834+0000 7f4a49084640 1 --2- 172.21.15.84:0/2432560197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb91f80 0x55574fb932a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.834+0000 7f4a49084640 1 -- 172.21.15.84:0/2432560197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a44148680 con 0x55574fb8d1e0 2023-10-16T10:09:17.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.835+0000 7f4a49084640 1 --2- 172.21.15.84:0/2432560197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb8d1e0 0x55574fb8d660 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00e4f0 tx=0x7f4a3c00e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1308da61084f1837 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:17.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.835+0000 7f4a237fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T10:09:17.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.835+0000 7f4a237fe640 1 -- 172.21.15.84:0/2432560197 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c013870 con 0x55574fb8d1e0 2023-10-16T10:09:17.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.835+0000 7f4a4d680740 1 -- 172.21.15.84:0/2432560197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55574faf7160 con 0x55574fb8d1e0 2023-10-16T10:09:17.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.835+0000 7f4a237fe640 1 -- 172.21.15.84:0/2432560197 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4a3c0139d0 con 0x55574fb8d1e0 2023-10-16T10:09:17.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.835+0000 7f4a237fe640 1 -- 172.21.15.84:0/2432560197 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a3c013be0 con 0x55574fb8d1e0 2023-10-16T10:09:17.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.835+0000 7f4a4d680740 10 client.9288 fetch_fsmap learned FSMap version 80 2023-10-16T10:09:17.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.835+0000 7f4a4d680740 1 -- 172.21.15.84:0/2432560197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x55574faceda0 con 0x55574fb8d1e0 2023-10-16T10:09:17.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.835+0000 7f4a237fe640 1 -- 172.21.15.84:0/2432560197 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55574faf7160 con 0x55574fb8d1e0 2023-10-16T10:09:17.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.836+0000 7f4a237fe640 1 -- 172.21.15.84:0/2432560197 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4a3c01c3f0 con 0x55574fb8d1e0 2023-10-16T10:09:17.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.836+0000 7f4a4d680740 10 client.9288 fetch_fsmap finished waiting for FSMap version 80 2023-10-16T10:09:17.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.836+0000 7f4a4d680740 10 client.9288 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T10:09:17.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.836+0000 7f4a4d680740 20 client.9288 populate_metadata read hostname 'smithi084' 2023-10-16T10:09:17.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.836+0000 7f4a4d680740 1 --2- 172.21.15.84:0/2432560197 >> [v2:172.21.15.84:6835/2797149326,v1:172.21.15.84:6837/2797149326] conn(0x55574fb8ad70 0x55574fb8b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.836+0000 7f4a4d680740 4 client.9288 mds_command: new command op to 9138 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T10:09:17.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.836+0000 7f4a4d680740 1 -- 172.21.15.84:0/2432560197 --> [v2:172.21.15.84:6835/2797149326,v1:172.21.15.84:6837/2797149326] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55574faa2cc0 con 0x55574fb8ad70 2023-10-16T10:09:17.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.836+0000 7f4a48883640 1 --2- 172.21.15.84:0/2432560197 >> [v2:172.21.15.84:6835/2797149326,v1:172.21.15.84:6837/2797149326] conn(0x55574fb8ad70 0x55574fb8b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:17.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.837+0000 7f4a48883640 1 --2- 172.21.15.84:0/2432560197 >> [v2:172.21.15.84:6835/2797149326,v1:172.21.15.84:6837/2797149326] conn(0x55574fb8ad70 0x55574fb8b150 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a5791eb2cfd7650 server_cookie=2ab10931eee1ed8b in_seq=0 out_seq=0 2023-10-16T10:09:17.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.837+0000 7f4a237fe640 10 client.9288 ms_handle_connect on v2:172.21.15.84:6835/2797149326 2023-10-16T10:09:17.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.838+0000 7f4a237fe640 1 -- 172.21.15.84:0/2432560197 <== mds.0 v2:172.21.15.84:6835/2797149326 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55574faa2cc0 con 0x55574fb8ad70 2023-10-16T10:09:17.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.838+0000 7f4a237fe640 10 client.9288 handle_command_reply: tid=0 2023-10-16T10:09:17.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.838+0000 7f4a4d680740 1 client.9288 shutdown 2023-10-16T10:09:17.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.838+0000 7f4a4d680740 1 -- 172.21.15.84:0/2432560197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb8d1e0 msgr2=0x55574fb8d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:17.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.838+0000 7f4a4d680740 1 --2- 172.21.15.84:0/2432560197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb8d1e0 0x55574fb8d660 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00e4f0 tx=0x7f4a3c00e9b0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.838+0000 7f4a4d680740 1 -- 172.21.15.84:0/2432560197 shutdown_connections 2023-10-16T10:09:17.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.838+0000 7f4a4d680740 1 --2- 172.21.15.84:0/2432560197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb8d1e0 0x55574fb8d660 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.838+0000 7f4a4d680740 1 --2- 172.21.15.84:0/2432560197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb92360 0x55574fb937e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.838+0000 7f4a4d680740 1 --2- 172.21.15.84:0/2432560197 >> [v2:172.21.15.84:6835/2797149326,v1:172.21.15.84:6837/2797149326] conn(0x55574fb8ad70 0x55574fb8b150 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.838+0000 7f4a4d680740 1 --2- 172.21.15.84:0/2432560197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb91f80 0x55574fb932a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.838+0000 7f4a237fe640 0 client.9288 ms_handle_reset on v2:172.21.15.84:6835/2797149326 2023-10-16T10:09:17.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.838+0000 7f4a4d680740 1 -- 172.21.15.84:0/2432560197 >> 172.21.15.84:0/2432560197 conn(0x55574fb653c0 msgr2=0x55574fb68f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:17.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.838+0000 7f4a4d680740 1 -- 172.21.15.84:0/2432560197 shutdown_connections 2023-10-16T10:09:17.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.838+0000 7f4a4d680740 1 -- 172.21.15.84:0/2432560197 wait complete. 2023-10-16T10:09:17.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.839+0000 7f4a4d680740 20 client.9288 trim_cache size 0 max 16384 2023-10-16T10:09:17.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.855+0000 7f4a4d680740 1 Processor -- start 2023-10-16T10:09:17.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.855+0000 7f4a4d680740 1 -- start start 2023-10-16T10:09:17.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.856+0000 7f4a4d680740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb8d1e0 0x55574fbaf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.856+0000 7f4a4d680740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb91f80 0x55574fb68a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.856+0000 7f4a4d680740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb92360 0x55574fb68fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.856+0000 7f4a4d680740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574f834f20 con 0x55574fb91f80 2023-10-16T10:09:17.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.856+0000 7f4a4d680740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574fb91990 con 0x55574fb8d1e0 2023-10-16T10:09:17.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.856+0000 7f4a4d680740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55574fbd3f40 con 0x55574fb92360 2023-10-16T10:09:17.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.856+0000 7f4a43fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb91f80 0x55574fb68a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:17.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.856+0000 7f4a43fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb91f80 0x55574fb68a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40012/0 (socket says 172.21.15.84:40012) 2023-10-16T10:09:17.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.856+0000 7f4a43fff640 1 -- 172.21.15.84:0/2461408256 learned_addr learned my addr 172.21.15.84:0/2461408256 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:17.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.857+0000 7f4a48883640 1 --2- 172.21.15.84:0/2461408256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb8d1e0 0x55574fbaf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:17.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.857+0000 7f4a49084640 1 --2- 172.21.15.84:0/2461408256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb92360 0x55574fb68fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:17.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.857+0000 7f4a43fff640 1 -- 172.21.15.84:0/2461408256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb92360 msgr2=0x55574fb68fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:17.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.857+0000 7f4a43fff640 1 --2- 172.21.15.84:0/2461408256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb92360 0x55574fb68fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.857+0000 7f4a43fff640 1 -- 172.21.15.84:0/2461408256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb8d1e0 msgr2=0x55574fbaf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:17.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.857+0000 7f4a43fff640 1 --2- 172.21.15.84:0/2461408256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb8d1e0 0x55574fbaf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.857+0000 7f4a43fff640 1 -- 172.21.15.84:0/2461408256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55574fa6db80 con 0x55574fb91f80 2023-10-16T10:09:17.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.857+0000 7f4a43fff640 1 --2- 172.21.15.84:0/2461408256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb91f80 0x55574fb68a80 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f4a30058600 tx=0x7f4a3005b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681c754ea842c73a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:17.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.857+0000 7f4a2effd640 1 -- 172.21.15.84:0/2461408256 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30074070 con 0x55574fb91f80 2023-10-16T10:09:17.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.857+0000 7f4a2effd640 1 -- 172.21.15.84:0/2461408256 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a30056cf0 con 0x55574fb91f80 2023-10-16T10:09:17.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.857+0000 7f4a2effd640 1 -- 172.21.15.84:0/2461408256 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30085040 con 0x55574fb91f80 2023-10-16T10:09:17.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.858+0000 7f4a4d680740 1 -- 172.21.15.84:0/2461408256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb91f80 msgr2=0x55574fb68a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:17.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.858+0000 7f4a4d680740 1 --2- 172.21.15.84:0/2461408256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb91f80 0x55574fb68a80 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f4a30058600 tx=0x7f4a3005b000 comp rx=0 tx=0).stop 2023-10-16T10:09:17.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.858+0000 7f4a4d680740 1 -- 172.21.15.84:0/2461408256 shutdown_connections 2023-10-16T10:09:17.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.858+0000 7f4a4d680740 1 --2- 172.21.15.84:0/2461408256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb92360 0x55574fb68fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.858+0000 7f4a4d680740 1 --2- 172.21.15.84:0/2461408256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb91f80 0x55574fb68a80 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.858+0000 7f4a4d680740 1 --2- 172.21.15.84:0/2461408256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb8d1e0 0x55574fbaf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.858+0000 7f4a4d680740 1 -- 172.21.15.84:0/2461408256 >> 172.21.15.84:0/2461408256 conn(0x55574fb653c0 msgr2=0x55574fb63e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:17.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.858+0000 7f4a4d680740 1 -- 172.21.15.84:0/2461408256 shutdown_connections 2023-10-16T10:09:17.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.858+0000 7f4a4d680740 1 -- 172.21.15.84:0/2461408256 wait complete. 2023-10-16T10:09:17.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.859+0000 7f4a4d680740 1 -- start start 2023-10-16T10:09:17.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.859+0000 7f4a4d680740 1 Processor -- start 2023-10-16T10:09:17.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.860+0000 7f4a4d680740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb8d1e0 0x55574fc1e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.860+0000 7f4a4d680740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb91f80 0x55574fb68a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.860+0000 7f4a4d680740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb92360 0x55574fb68fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.860+0000 7f4a4d680740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574fbd3ab0 con 0x55574fb8d1e0 2023-10-16T10:09:17.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.860+0000 7f4a4d680740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55574fbd38a0 con 0x55574fb92360 2023-10-16T10:09:17.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.860+0000 7f4a4d680740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55574fc21a90 con 0x55574fb91f80 2023-10-16T10:09:17.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.860+0000 7f4a4d680740 10 client.0 _ll_register_callbacks cb 0x7f4a3c003be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:09:17.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.860+0000 7f4a48883640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb8d1e0 0x55574fc1e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:17.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.860+0000 7f4a48883640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb8d1e0 0x55574fc1e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40014/0 (socket says 172.21.15.84:40014) 2023-10-16T10:09:17.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.860+0000 7f4a48883640 1 -- 172.21.15.84:0/1658590983 learned_addr learned my addr 172.21.15.84:0/1658590983 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:17.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.860+0000 7f4a48883640 1 -- 172.21.15.84:0/1658590983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb91f80 msgr2=0x55574fb68a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:17.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.860+0000 7f4a48883640 1 --2- 172.21.15.84:0/1658590983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb91f80 0x55574fb68a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.860+0000 7f4a48883640 1 -- 172.21.15.84:0/1658590983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb92360 msgr2=0x55574fb68fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:17.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.860+0000 7f4a48883640 1 --2- 172.21.15.84:0/1658590983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb92360 0x55574fb68fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.860+0000 7f4a48883640 1 -- 172.21.15.84:0/1658590983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55574fa94910 con 0x55574fb8d1e0 2023-10-16T10:09:17.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.861+0000 7f4a48883640 1 --2- 172.21.15.84:0/1658590983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb8d1e0 0x55574fc1e740 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f4a38034040 tx=0x7f4a38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50c630dac4792f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:17.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.861+0000 7f4a237fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T10:09:17.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.861+0000 7f4a237fe640 1 -- 172.21.15.84:0/1658590983 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a38049040 con 0x55574fb8d1e0 2023-10-16T10:09:17.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.861+0000 7f4a237fe640 1 -- 172.21.15.84:0/1658590983 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4a38050910 con 0x55574fb8d1e0 2023-10-16T10:09:17.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.861+0000 7f4a4d680740 1 -- 172.21.15.84:0/1658590983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a30060c50 con 0x55574fb8d1e0 2023-10-16T10:09:17.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.861+0000 7f4a237fe640 1 -- 172.21.15.84:0/1658590983 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a38050ad0 con 0x55574fb8d1e0 2023-10-16T10:09:17.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.861+0000 7f4a4d680740 10 client.9294 fetch_fsmap learned FSMap version 80 2023-10-16T10:09:17.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.861+0000 7f4a4d680740 1 -- 172.21.15.84:0/1658590983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x55574fa93170 con 0x55574fb8d1e0 2023-10-16T10:09:17.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.862+0000 7f4a237fe640 1 -- 172.21.15.84:0/1658590983 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a30060c50 con 0x55574fb8d1e0 2023-10-16T10:09:17.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.862+0000 7f4a237fe640 1 -- 172.21.15.84:0/1658590983 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4a38050c30 con 0x55574fb8d1e0 2023-10-16T10:09:17.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.862+0000 7f4a4d680740 10 client.9294 fetch_fsmap finished waiting for FSMap version 80 2023-10-16T10:09:17.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.862+0000 7f4a4d680740 10 client.9294 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T10:09:17.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.862+0000 7f4a4d680740 20 client.9294 populate_metadata read hostname 'smithi084' 2023-10-16T10:09:17.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.862+0000 7f4a4d680740 1 --2- 172.21.15.84:0/1658590983 >> [v2:172.21.15.84:6835/2797149326,v1:172.21.15.84:6837/2797149326] conn(0x55574fbd1d50 0x55574fbd2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:17.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.862+0000 7f4a4d680740 4 client.9294 mds_command: new command op to 9138 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T10:09:17.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.862+0000 7f4a4d680740 1 -- 172.21.15.84:0/1658590983 --> [v2:172.21.15.84:6835/2797149326,v1:172.21.15.84:6837/2797149326] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55574fac9700 con 0x55574fbd1d50 2023-10-16T10:09:17.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.862+0000 7f4a43fff640 1 --2- 172.21.15.84:0/1658590983 >> [v2:172.21.15.84:6835/2797149326,v1:172.21.15.84:6837/2797149326] conn(0x55574fbd1d50 0x55574fbd2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:17.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.862+0000 7f4a43fff640 1 --2- 172.21.15.84:0/1658590983 >> [v2:172.21.15.84:6835/2797149326,v1:172.21.15.84:6837/2797149326] conn(0x55574fbd1d50 0x55574fbd2130 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa839ae7a251f983 server_cookie=f5f619db88629a61 in_seq=0 out_seq=0 2023-10-16T10:09:17.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.862+0000 7f4a237fe640 10 client.9294 ms_handle_connect on v2:172.21.15.84:6835/2797149326 2023-10-16T10:09:17.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.863+0000 7f4a237fe640 1 -- 172.21.15.84:0/1658590983 <== mds.0 v2:172.21.15.84:6835/2797149326 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55574fac9700 con 0x55574fbd1d50 2023-10-16T10:09:17.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.863+0000 7f4a237fe640 10 client.9294 handle_command_reply: tid=0 2023-10-16T10:09:17.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.863+0000 7f4a4d680740 1 client.9294 shutdown 2023-10-16T10:09:17.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.863+0000 7f4a4d680740 1 -- 172.21.15.84:0/1658590983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb8d1e0 msgr2=0x55574fc1e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:17.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.863+0000 7f4a4d680740 1 --2- 172.21.15.84:0/1658590983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb8d1e0 0x55574fc1e740 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f4a38034040 tx=0x7f4a38004730 comp rx=0 tx=0).stop 2023-10-16T10:09:17.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.863+0000 7f4a4d680740 1 -- 172.21.15.84:0/1658590983 shutdown_connections 2023-10-16T10:09:17.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.863+0000 7f4a4d680740 1 --2- 172.21.15.84:0/1658590983 >> [v2:172.21.15.84:6835/2797149326,v1:172.21.15.84:6837/2797149326] conn(0x55574fbd1d50 0x55574fbd2130 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.863+0000 7f4a4d680740 1 --2- 172.21.15.84:0/1658590983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55574fb92360 0x55574fb68fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.863+0000 7f4a4d680740 1 --2- 172.21.15.84:0/1658590983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55574fb91f80 0x55574fb68a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.863+0000 7f4a4d680740 1 --2- 172.21.15.84:0/1658590983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55574fb8d1e0 0x55574fc1e740 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.863+0000 7f4a237fe640 0 client.9294 ms_handle_reset on v2:172.21.15.84:6835/2797149326 2023-10-16T10:09:17.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.863+0000 7f4a4d680740 1 -- 172.21.15.84:0/1658590983 >> 172.21.15.84:0/1658590983 conn(0x55574fb653c0 msgr2=0x55574fbb3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:17.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.863+0000 7f4a4d680740 1 -- 172.21.15.84:0/1658590983 shutdown_connections 2023-10-16T10:09:17.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.863+0000 7f4a4d680740 1 -- 172.21.15.84:0/1658590983 wait complete. 2023-10-16T10:09:17.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.863+0000 7f4a4d680740 20 client.9294 trim_cache size 0 max 16384 2023-10-16T10:09:17.881 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T10:09:17.881 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "63fb4d68-f881-4694-b880-d10369105c0e", 2023-10-16T10:09:17.881 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-16T10:09:17.881 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 9138, 2023-10-16T10:09:17.882 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-16T10:09:17.882 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-16T10:09:17.882 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-16T10:09:17.882 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 6.6438382520000001, 2023-10-16T10:09:17.882 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 80, 2023-10-16T10:09:17.883 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 151, 2023-10-16T10:09:17.883 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 151, 2023-10-16T10:09:17.883 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 12.042022595000001 2023-10-16T10:09:17.883 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T10:09:17.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.864+0000 7f4a4aae7640 1 -- 172.21.15.84:0/2946040628 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f4a2806af90 msgr2=0x7f4a2806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:17.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.864+0000 7f4a4aae7640 1 --2- 172.21.15.84:0/2946040628 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f4a2806af90 0x7f4a2806d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4a38036000 tx=0x7f4a3803c040 comp rx=0 tx=0).stop 2023-10-16T10:09:17.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.864+0000 7f4a4aae7640 1 -- 172.21.15.84:0/2946040628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a440ca3b0 msgr2=0x7f4a441efb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:17.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.864+0000 7f4a4aae7640 1 --2- 172.21.15.84:0/2946040628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a440ca3b0 0x7f4a441efb60 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000d800 tx=0x7f4a3000dcc0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.864+0000 7f4a4aae7640 1 -- 172.21.15.84:0/2946040628 shutdown_connections 2023-10-16T10:09:17.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.864+0000 7f4a4aae7640 1 --2- 172.21.15.84:0/2946040628 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f4a2806af90 0x7f4a2806d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.864+0000 7f4a4aae7640 1 --2- 172.21.15.84:0/2946040628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a4415ad70 0x7f4a4415c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.864+0000 7f4a4aae7640 1 --2- 172.21.15.84:0/2946040628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a440ca3b0 0x7f4a441efb60 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.864+0000 7f4a4aae7640 1 --2- 172.21.15.84:0/2946040628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a440c9ca0 0x7f4a4414aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:17.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.864+0000 7f4a4aae7640 1 -- 172.21.15.84:0/2946040628 >> 172.21.15.84:0/2946040628 conn(0x7f4a440ce9b0 msgr2=0x7f4a440ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:17.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.864+0000 7f4a4aae7640 1 -- 172.21.15.84:0/2946040628 shutdown_connections 2023-10-16T10:09:17.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:17.864+0000 7f4a4aae7640 1 -- 172.21.15.84:0/2946040628 wait complete. 2023-10-16T10:09:17.887 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T10:09:18.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.093+0000 7fad30bc9640 1 Processor -- start 2023-10-16T10:09:18.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.094+0000 7fad30bc9640 1 -- start start 2023-10-16T10:09:18.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.095+0000 7fad30bc9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fad2c0b1d00 0x7fad2c0d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:18.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.095+0000 7fad30bc9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2c0cfb60 0x7fad2c161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:18.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.095+0000 7fad30bc9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2c0d0010 con 0x7fad2c0cf780 2023-10-16T10:09:18.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.095+0000 7fad30bc9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2c0d14a0 con 0x7fad2c0b1d00 2023-10-16T10:09:18.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.095+0000 7fad30bc9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c0d0290 con 0x7fad2c0cfb60 2023-10-16T10:09:18.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.095+0000 7fad21d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fad2c0b1d00 0x7fad2c0d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:18.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.096+0000 7fad2a59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fad2c0cf780 0x7fad2c160f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35400/0 (socket says 172.21.15.84:35400) 2023-10-16T10:09:18.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.096+0000 7fad2a59c640 1 -- 172.21.15.84:0/273650909 learned_addr learned my addr 172.21.15.84:0/273650909 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:09:18.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.096+0000 7fad2ad9d640 1 --2- 172.21.15.84:0/273650909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2c0cfb60 0x7fad2c161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:18.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.096+0000 7fad21d9b640 1 -- 172.21.15.84:0/273650909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2c0cfb60 msgr2=0x7fad2c161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:18.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.096+0000 7fad21d9b640 1 --2- 172.21.15.84:0/273650909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2c0cfb60 0x7fad2c161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:18.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.096+0000 7fad21d9b640 1 -- 172.21.15.84:0/273650909 >> v1:172.21.15.84:6789/0 conn(0x7fad2c0cf780 legacy=0x7fad2c160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:18.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.096+0000 7fad21d9b640 1 -- 172.21.15.84:0/273650909 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c0d16b0 con 0x7fad2c0b1d00 2023-10-16T10:09:18.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.096+0000 7fad2ad9d640 1 --2- 172.21.15.84:0/273650909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2c0cfb60 0x7fad2c161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:18.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.097+0000 7fad21d9b640 1 --2- 172.21.15.84:0/273650909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad2c0b1d00 0x7fad2c0d0a90 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fad140097a0 tx=0x7fad1402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12d5b02ecb19acc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:18.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.097+0000 7fad29d9b640 1 -- 172.21.15.84:0/273650909 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1402d040 con 0x7fad2c0b1d00 2023-10-16T10:09:18.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.097+0000 7fad29d9b640 1 -- 172.21.15.84:0/273650909 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad14031810 con 0x7fad2c0b1d00 2023-10-16T10:09:18.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.097+0000 7fad29d9b640 1 -- 172.21.15.84:0/273650909 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad14031ba0 con 0x7fad2c0b1d00 2023-10-16T10:09:18.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.098+0000 7fad30bc9640 1 -- 172.21.15.84:0/273650909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad2c0b1d00 msgr2=0x7fad2c0d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:18.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.098+0000 7fad30bc9640 1 --2- 172.21.15.84:0/273650909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad2c0b1d00 0x7fad2c0d0a90 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fad140097a0 tx=0x7fad1402b8b0 comp rx=0 tx=0).stop 2023-10-16T10:09:18.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.098+0000 7fad30bc9640 1 -- 172.21.15.84:0/273650909 shutdown_connections 2023-10-16T10:09:18.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.098+0000 7fad30bc9640 1 --2- 172.21.15.84:0/273650909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2c0cfb60 0x7fad2c161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:18.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.098+0000 7fad30bc9640 1 --2- 172.21.15.84:0/273650909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad2c0b1d00 0x7fad2c0d0a90 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:18.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.098+0000 7fad30bc9640 1 -- 172.21.15.84:0/273650909 >> 172.21.15.84:0/273650909 conn(0x7fad2c09c1f0 msgr2=0x7fad2c0d2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:18.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.098+0000 7fad30bc9640 1 -- 172.21.15.84:0/273650909 shutdown_connections 2023-10-16T10:09:18.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.098+0000 7fad30bc9640 1 -- 172.21.15.84:0/273650909 wait complete. 2023-10-16T10:09:18.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.101+0000 7fad30bc9640 1 Processor -- start 2023-10-16T10:09:18.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.101+0000 7fad30bc9640 1 -- start start 2023-10-16T10:09:18.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.102+0000 7fad30bc9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad2c0b1d00 0x7fad2c159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:18.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.102+0000 7fad30bc9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2c0cf780 0x7fad2c0b7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:18.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.102+0000 7fad30bc9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad2c0cfb60 0x7fad2c0b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:18.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.102+0000 7fad30bc9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c147500 con 0x7fad2c0cfb60 2023-10-16T10:09:18.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.102+0000 7fad30bc9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c147ac0 con 0x7fad2c0b1d00 2023-10-16T10:09:18.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.102+0000 7fad30bc9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c15be40 con 0x7fad2c0cf780 2023-10-16T10:09:18.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.102+0000 7fad2ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad2c0cfb60 0x7fad2c0b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:18.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.102+0000 7fad2ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad2c0cfb60 0x7fad2c0b81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40042/0 (socket says 172.21.15.84:40042) 2023-10-16T10:09:18.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.102+0000 7fad2ad9d640 1 -- 172.21.15.84:0/3667591756 learned_addr learned my addr 172.21.15.84:0/3667591756 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:18.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.103+0000 7fad2a59c640 1 --2- 172.21.15.84:0/3667591756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad2c0b1d00 0x7fad2c159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:18.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.103+0000 7fad21d9b640 1 --2- 172.21.15.84:0/3667591756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2c0cf780 0x7fad2c0b7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:18.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.103+0000 7fad2ad9d640 1 -- 172.21.15.84:0/3667591756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2c0cf780 msgr2=0x7fad2c0b7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:18.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.103+0000 7fad2ad9d640 1 --2- 172.21.15.84:0/3667591756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2c0cf780 0x7fad2c0b7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:18.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.103+0000 7fad2ad9d640 1 -- 172.21.15.84:0/3667591756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad2c0b1d00 msgr2=0x7fad2c159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:18.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.103+0000 7fad2ad9d640 1 --2- 172.21.15.84:0/3667591756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad2c0b1d00 0x7fad2c159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:18.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.103+0000 7fad2ad9d640 1 -- 172.21.15.84:0/3667591756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad14009400 con 0x7fad2c0cfb60 2023-10-16T10:09:18.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.103+0000 7fad21d9b640 1 --2- 172.21.15.84:0/3667591756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2c0cf780 0x7fad2c0b7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:18.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.103+0000 7fad2ad9d640 1 --2- 172.21.15.84:0/3667591756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad2c0cfb60 0x7fad2c0b81f0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fad2400cef0 tx=0x7fad2400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:18.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.103+0000 7fad237fe640 1 -- 172.21.15.84:0/3667591756 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2400d840 con 0x7fad2c0cfb60 2023-10-16T10:09:18.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.103+0000 7fad237fe640 1 -- 172.21.15.84:0/3667591756 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2400d9c0 con 0x7fad2c0cfb60 2023-10-16T10:09:18.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.103+0000 7fad237fe640 1 -- 172.21.15.84:0/3667591756 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2400dd70 con 0x7fad2c0cfb60 2023-10-16T10:09:18.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.104+0000 7fad30bc9640 1 -- 172.21.15.84:0/3667591756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2c15c120 con 0x7fad2c0cfb60 2023-10-16T10:09:18.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.104+0000 7fad30bc9640 1 -- 172.21.15.84:0/3667591756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2c15b0a0 con 0x7fad2c0cfb60 2023-10-16T10:09:18.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.105+0000 7fad30bc9640 1 -- 172.21.15.84:0/3667591756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2c147c30 con 0x7fad2c0cfb60 2023-10-16T10:09:18.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.106+0000 7fad237fe640 1 -- 172.21.15.84:0/3667591756 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fad2401e3b0 con 0x7fad2c0cfb60 2023-10-16T10:09:18.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.107+0000 7fad237fe640 1 --2- 172.21.15.84:0/3667591756 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fad0c06af90 0x7fad0c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:18.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.107+0000 7fad237fe640 1 -- 172.21.15.84:0/3667591756 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fad2408a290 con 0x7fad2c0cfb60 2023-10-16T10:09:18.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.107+0000 7fad2a59c640 1 --2- 172.21.15.84:0/3667591756 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fad0c06af90 0x7fad0c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:18.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.108+0000 7fad2a59c640 1 --2- 172.21.15.84:0/3667591756 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fad0c06af90 0x7fad0c06d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fad1c0010e0 tx=0x7fad1c009040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:18.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.111+0000 7fad237fe640 1 -- 172.21.15.84:0/3667591756 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fad240552d0 con 0x7fad2c0cfb60 2023-10-16T10:09:18.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.348+0000 7fad30bc9640 1 -- 172.21.15.84:0/3667591756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fad2c04e160 con 0x7fad2c0cfb60 2023-10-16T10:09:18.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.349+0000 7fad237fe640 1 -- 172.21.15.84:0/3667591756 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v80) v1 ==== 53+0+73 (secure 0 0 0) 0x7fad240588f0 con 0x7fad2c0cfb60 2023-10-16T10:09:18.350 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T10:09:18.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.351+0000 7fad30bc9640 1 -- 172.21.15.84:0/3667591756 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fad0c06af90 msgr2=0x7fad0c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:18.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.351+0000 7fad30bc9640 1 --2- 172.21.15.84:0/3667591756 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fad0c06af90 0x7fad0c06d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fad1c0010e0 tx=0x7fad1c009040 comp rx=0 tx=0).stop 2023-10-16T10:09:18.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.351+0000 7fad30bc9640 1 -- 172.21.15.84:0/3667591756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad2c0cfb60 msgr2=0x7fad2c0b81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:18.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.351+0000 7fad30bc9640 1 --2- 172.21.15.84:0/3667591756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad2c0cfb60 0x7fad2c0b81f0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fad2400cef0 tx=0x7fad2400a710 comp rx=0 tx=0).stop 2023-10-16T10:09:18.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.351+0000 7fad30bc9640 1 -- 172.21.15.84:0/3667591756 shutdown_connections 2023-10-16T10:09:18.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.351+0000 7fad30bc9640 1 --2- 172.21.15.84:0/3667591756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad2c0cfb60 0x7fad2c0b81f0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:18.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.351+0000 7fad30bc9640 1 --2- 172.21.15.84:0/3667591756 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fad0c06af90 0x7fad0c06d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:18.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.351+0000 7fad30bc9640 1 --2- 172.21.15.84:0/3667591756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad2c0cf780 0x7fad2c0b7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:18.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.351+0000 7fad30bc9640 1 --2- 172.21.15.84:0/3667591756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad2c0b1d00 0x7fad2c159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:18.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.352+0000 7fad30bc9640 1 -- 172.21.15.84:0/3667591756 >> 172.21.15.84:0/3667591756 conn(0x7fad2c09c1f0 msgr2=0x7fad2c0b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:18.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.352+0000 7fad30bc9640 1 -- 172.21.15.84:0/3667591756 shutdown_connections 2023-10-16T10:09:18.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:18.352+0000 7fad30bc9640 1 -- 172.21.15.84:0/3667591756 wait complete. 2023-10-16T10:09:18.371 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T10:09:18.372 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T10:09:18.372 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-16T10:09:18.372 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T10:09:18.373 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T10:09:18.373 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T10:09:18.373 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T10:09:18.374 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T10:09:18.374 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:09:18.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:09:18.375 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-16T10:09:18.428 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T10:09:18.428 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T10:09:18.428 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T10:09:18.429 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:09:18.429 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T10:09:18.429 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:09:18.429 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:09:18.429 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:09:18.430 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-16T10:09:18.430 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:09:18.430 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:09:18.430 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-16T10:09:18.430 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:09:18.431 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:09:18.431 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T10:09:18.431 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:09:18.431 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T10:09:18.431 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 41193sec preferred_lft 41193sec 2023-10-16T10:09:18.432 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-16T10:09:18.432 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:09:18.432 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T10:09:18.432 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:09:18.432 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T10:09:18.433 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:09:18.433 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::20a8:eaff:fe9d:5cea/64 scope link 2023-10-16T10:09:18.433 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:09:18.433 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:09:18.433 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T10:09:18.434 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1854:13ff:fe47:be92/64 scope link 2023-10-16T10:09:18.434 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:09:18.434 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:09:18.434 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether ee:d5:41:a3:a5:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T10:09:18.434 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ecd5:41ff:fea3:a570/64 scope link 2023-10-16T10:09:18.435 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:09:18.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:09:18.436 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T10:09:18.530 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-16T10:09:18.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:09:18.534 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-16T10:09:18.589 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-16T10:09:18.589 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T10:09:18.589 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T10:09:18.590 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T10:09:18.590 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T10:09:18.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:09:18.591 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T10:09:18.591 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T10:09:18.591 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T10:09:18.592 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T10:09:18.592 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T10:09:18.593 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T10:09:18.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:09:18.769 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T10:09:18.822 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T10:09:18.822 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T10:09:18.823 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:09:18.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:09:18.824 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-16T10:09:18.918 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:09:18.919 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T10:09:18.976 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:09:18.976 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:09:18.977 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:09:19.032 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-16T10:09:19.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:09:19.033 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:09:19.170 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:09:19.170 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T10:09:19.206 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T10:09:19.206 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T10:09:19.207 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T10:09:19.207 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with new device syntax: 0@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T10:09:19.207 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T10:09:19.208 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-16T10:09:19.209 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2023-10-16T10:09:19.209 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T10:09:19.209 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T10:09:19.209 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T10:09:19.214 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T10:09:19.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:09:19.215 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:09:19.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:09:19.309 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T10:09:19.376 INFO:teuthology.orchestra.run.smithi084.stdout:client6014 2023-10-16T10:09:19.377 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T10:09:19.377 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:09:19.433 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T10:09:19.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.644+0000 7fbc64ba9640 1 Processor -- start 2023-10-16T10:09:19.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.645+0000 7fbc64ba9640 1 -- start start 2023-10-16T10:09:19.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.646+0000 7fbc64ba9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbc60159ab0 0x7fbc600ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:19.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.646+0000 7fbc64ba9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbc6006c140 0x7fbc6006c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:19.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.646+0000 7fbc64ba9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc6006cad0 con 0x7fbc60159e90 2023-10-16T10:09:19.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.646+0000 7fbc64ba9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc6015d890 con 0x7fbc6006c140 2023-10-16T10:09:19.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.646+0000 7fbc64ba9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc60070360 con 0x7fbc60159ab0 2023-10-16T10:09:19.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.646+0000 7fbc5ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbc6006c140 0x7fbc6006c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:19.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.647+0000 7fbc5ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbc6006c140 0x7fbc6006c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40052/0 (socket says 172.21.15.84:40052) 2023-10-16T10:09:19.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.647+0000 7fbc5ed9d640 1 -- 172.21.15.84:0/1754151090 learned_addr learned my addr 172.21.15.84:0/1754151090 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:19.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.647+0000 7fbc5e59c640 1 --2- 172.21.15.84:0/1754151090 >> v2:172.21.15.157:3300/0 conn(0x7fbc60159ab0 0x7fbc600ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:19.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.647+0000 7fbc5ed9d640 1 -- 172.21.15.84:0/1754151090 >> v2:172.21.15.157:3300/0 conn(0x7fbc60159ab0 msgr2=0x7fbc600ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:19.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.647+0000 7fbc5ed9d640 1 --2- 172.21.15.84:0/1754151090 >> v2:172.21.15.157:3300/0 conn(0x7fbc60159ab0 0x7fbc600ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:19.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.647+0000 7fbc5ed9d640 1 -- 172.21.15.84:0/1754151090 >> v1:172.21.15.157:6789/0 conn(0x7fbc60159e90 legacy=0x7fbc6006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:19.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.647+0000 7fbc5ed9d640 1 -- 172.21.15.84:0/1754151090 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc600cd1f0 con 0x7fbc6006c140 2023-10-16T10:09:19.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.647+0000 7fbc5e59c640 1 --2- 172.21.15.84:0/1754151090 >> v2:172.21.15.157:3300/0 conn(0x7fbc60159ab0 0x7fbc600ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:19.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.648+0000 7fbc5ed9d640 1 --2- 172.21.15.84:0/1754151090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc6006c140 0x7fbc6006c590 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fbc600cfa00 tx=0x7fbc5802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6f1dc67e5ce760 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:19.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.648+0000 7fbc5d59a640 1 -- 172.21.15.84:0/1754151090 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58031750 con 0x7fbc6006c140 2023-10-16T10:09:19.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.648+0000 7fbc5d59a640 1 -- 172.21.15.84:0/1754151090 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc580318b0 con 0x7fbc6006c140 2023-10-16T10:09:19.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.648+0000 7fbc5d59a640 1 -- 172.21.15.84:0/1754151090 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58031c40 con 0x7fbc6006c140 2023-10-16T10:09:19.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.649+0000 7fbc64ba9640 1 -- 172.21.15.84:0/1754151090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc6006c140 msgr2=0x7fbc6006c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:19.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.649+0000 7fbc64ba9640 1 --2- 172.21.15.84:0/1754151090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc6006c140 0x7fbc6006c590 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fbc600cfa00 tx=0x7fbc5802b7f0 comp rx=0 tx=0).stop 2023-10-16T10:09:19.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.649+0000 7fbc64ba9640 1 -- 172.21.15.84:0/1754151090 shutdown_connections 2023-10-16T10:09:19.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.649+0000 7fbc64ba9640 1 --2- 172.21.15.84:0/1754151090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc6006c140 0x7fbc6006c590 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:19.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.649+0000 7fbc64ba9640 1 --2- 172.21.15.84:0/1754151090 >> v2:172.21.15.157:3300/0 conn(0x7fbc60159ab0 0x7fbc600ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:19.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.649+0000 7fbc64ba9640 1 -- 172.21.15.84:0/1754151090 >> 172.21.15.84:0/1754151090 conn(0x7fbc600ce9b0 msgr2=0x7fbc600cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:19.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.649+0000 7fbc64ba9640 1 -- 172.21.15.84:0/1754151090 shutdown_connections 2023-10-16T10:09:19.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.649+0000 7fbc64ba9640 1 -- 172.21.15.84:0/1754151090 wait complete. 2023-10-16T10:09:19.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.651+0000 7fbc64ba9640 1 Processor -- start 2023-10-16T10:09:19.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.651+0000 7fbc64ba9640 1 -- start start 2023-10-16T10:09:19.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.652+0000 7fbc64ba9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc6006c140 0x7fbc6015ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:19.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.652+0000 7fbc64ba9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc60159ab0 0x7fbc601f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:19.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.652+0000 7fbc64ba9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc60159e90 0x7fbc601f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:19.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.652+0000 7fbc64ba9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6015b9f0 con 0x7fbc60159ab0 2023-10-16T10:09:19.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.652+0000 7fbc64ba9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6015ade0 con 0x7fbc60159e90 2023-10-16T10:09:19.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.652+0000 7fbc64ba9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6015b0c0 con 0x7fbc6006c140 2023-10-16T10:09:19.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.652+0000 7fbc5dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc60159ab0 0x7fbc601f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:19.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.652+0000 7fbc5dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc60159ab0 0x7fbc601f3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40054/0 (socket says 172.21.15.84:40054) 2023-10-16T10:09:19.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.652+0000 7fbc5dd9b640 1 -- 172.21.15.84:0/2109184377 learned_addr learned my addr 172.21.15.84:0/2109184377 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:19.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.653+0000 7fbc5e59c640 1 --2- 172.21.15.84:0/2109184377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc6006c140 0x7fbc6015ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:19.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.653+0000 7fbc5ed9d640 1 --2- 172.21.15.84:0/2109184377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc60159e90 0x7fbc601f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:19.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.653+0000 7fbc5dd9b640 1 -- 172.21.15.84:0/2109184377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc6006c140 msgr2=0x7fbc6015ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:19.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.653+0000 7fbc5dd9b640 1 --2- 172.21.15.84:0/2109184377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc6006c140 0x7fbc6015ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:19.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.653+0000 7fbc5dd9b640 1 -- 172.21.15.84:0/2109184377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc60159e90 msgr2=0x7fbc601f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:19.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.653+0000 7fbc5dd9b640 1 --2- 172.21.15.84:0/2109184377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc60159e90 0x7fbc601f4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:19.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.653+0000 7fbc5dd9b640 1 -- 172.21.15.84:0/2109184377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc580094d0 con 0x7fbc60159ab0 2023-10-16T10:09:19.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.653+0000 7fbc5ed9d640 1 --2- 172.21.15.84:0/2109184377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc60159e90 0x7fbc601f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:19.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.653+0000 7fbc5dd9b640 1 --2- 172.21.15.84:0/2109184377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc60159ab0 0x7fbc601f3bc0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800c430 tx=0x7fbc4800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:19.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.653+0000 7fbc56ffd640 1 -- 172.21.15.84:0/2109184377 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc48011790 con 0x7fbc60159ab0 2023-10-16T10:09:19.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.653+0000 7fbc56ffd640 1 -- 172.21.15.84:0/2109184377 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc480118f0 con 0x7fbc60159ab0 2023-10-16T10:09:19.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.653+0000 7fbc56ffd640 1 -- 172.21.15.84:0/2109184377 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc48011ce0 con 0x7fbc60159ab0 2023-10-16T10:09:19.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.653+0000 7fbc64ba9640 1 -- 172.21.15.84:0/2109184377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6015b3a0 con 0x7fbc60159ab0 2023-10-16T10:09:19.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.654+0000 7fbc64ba9640 1 -- 172.21.15.84:0/2109184377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6015cf70 con 0x7fbc60159ab0 2023-10-16T10:09:19.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.655+0000 7fbc64ba9640 1 -- 172.21.15.84:0/2109184377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6015bb60 con 0x7fbc60159ab0 2023-10-16T10:09:19.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.656+0000 7fbc56ffd640 1 -- 172.21.15.84:0/2109184377 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbc4801c460 con 0x7fbc60159ab0 2023-10-16T10:09:19.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.657+0000 7fbc56ffd640 1 --2- 172.21.15.84:0/2109184377 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fbc4006af90 0x7fbc4006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:19.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.657+0000 7fbc56ffd640 1 -- 172.21.15.84:0/2109184377 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fbc48089810 con 0x7fbc60159ab0 2023-10-16T10:09:19.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.657+0000 7fbc5e59c640 1 --2- 172.21.15.84:0/2109184377 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fbc4006af90 0x7fbc4006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:19.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.658+0000 7fbc5e59c640 1 --2- 172.21.15.84:0/2109184377 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fbc4006af90 0x7fbc4006d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbc600c4230 tx=0x7fbc500084f0 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:19.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.661+0000 7fbc56ffd640 1 -- 172.21.15.84:0/2109184377 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbc480546f0 con 0x7fbc60159ab0 2023-10-16T10:09:19.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.876+0000 7fbc64ba9640 1 -- 172.21.15.84:0/2109184377 --> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc600cd7b0 con 0x7fbc4006af90 2023-10-16T10:09:19.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.880+0000 7fbc56ffd640 1 -- 172.21.15.84:0/2109184377 <== mgr.8904 v2:172.21.15.84:6832/205480762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fbc5000b420 con 0x7fbc4006af90 2023-10-16T10:09:19.881 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T10:09:19.882 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T10:09:19.882 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephfs" 2023-10-16T10:09:19.882 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T10:09:19.882 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T10:09:19.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.882+0000 7fbc64ba9640 1 -- 172.21.15.84:0/2109184377 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fbc4006af90 msgr2=0x7fbc4006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:19.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.882+0000 7fbc64ba9640 1 --2- 172.21.15.84:0/2109184377 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fbc4006af90 0x7fbc4006d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbc600c4230 tx=0x7fbc500084f0 comp rx=0 tx=0).stop 2023-10-16T10:09:19.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.882+0000 7fbc64ba9640 1 -- 172.21.15.84:0/2109184377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc60159ab0 msgr2=0x7fbc601f3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:19.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.882+0000 7fbc64ba9640 1 --2- 172.21.15.84:0/2109184377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc60159ab0 0x7fbc601f3bc0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800c430 tx=0x7fbc4800c8f0 comp rx=0 tx=0).stop 2023-10-16T10:09:19.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.882+0000 7fbc64ba9640 1 -- 172.21.15.84:0/2109184377 shutdown_connections 2023-10-16T10:09:19.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.882+0000 7fbc64ba9640 1 --2- 172.21.15.84:0/2109184377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc60159e90 0x7fbc601f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:19.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.882+0000 7fbc64ba9640 1 --2- 172.21.15.84:0/2109184377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc60159ab0 0x7fbc601f3bc0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:19.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.882+0000 7fbc64ba9640 1 --2- 172.21.15.84:0/2109184377 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fbc4006af90 0x7fbc4006d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:19.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.882+0000 7fbc64ba9640 1 --2- 172.21.15.84:0/2109184377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc6006c140 0x7fbc6015ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:19.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.883+0000 7fbc64ba9640 1 -- 172.21.15.84:0/2109184377 >> 172.21.15.84:0/2109184377 conn(0x7fbc600ce9b0 msgr2=0x7fbc600ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:19.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.883+0000 7fbc64ba9640 1 -- 172.21.15.84:0/2109184377 shutdown_connections 2023-10-16T10:09:19.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:19.883+0000 7fbc64ba9640 1 -- 172.21.15.84:0/2109184377 wait complete. 2023-10-16T10:09:19.903 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T10:09:20.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.118+0000 7fb8c610f640 1 Processor -- start 2023-10-16T10:09:20.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.118+0000 7fb8c610f640 1 -- start start 2023-10-16T10:09:20.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.119+0000 7fb8c610f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb8c0159ab0 0x7fb8c00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:20.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.119+0000 7fb8c610f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c0159e90 0x7fb8c006fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:20.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.119+0000 7fb8c610f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb8c006c140 0x7fb8c006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:20.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.119+0000 7fb8c610f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c006cb10 con 0x7fb8c006c140 2023-10-16T10:09:20.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.119+0000 7fb8c610f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c015d890 con 0x7fb8c0159ab0 2023-10-16T10:09:20.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.119+0000 7fb8c610f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c00cd1f0 con 0x7fb8c0159e90 2023-10-16T10:09:20.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.120+0000 7fb8bffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb8c006c140 0x7fb8c006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:20.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.120+0000 7fb8bffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb8c006c140 0x7fb8c006c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40064/0 (socket says 172.21.15.84:40064) 2023-10-16T10:09:20.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.120+0000 7fb8bffff640 1 -- 172.21.15.84:0/918207480 learned_addr learned my addr 172.21.15.84:0/918207480 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:20.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.120+0000 7fb8b7fff640 1 --2- 172.21.15.84:0/918207480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c0159e90 0x7fb8c006fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:20.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.120+0000 7fb8bf7fe640 1 --2- 172.21.15.84:0/918207480 >> v2:172.21.15.157:3300/0 conn(0x7fb8c0159ab0 0x7fb8c00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:20.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.120+0000 7fb8bffff640 1 -- 172.21.15.84:0/918207480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c0159e90 msgr2=0x7fb8c006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:20.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.120+0000 7fb8bffff640 1 --2- 172.21.15.84:0/918207480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c0159e90 0x7fb8c006fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:20.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.120+0000 7fb8bffff640 1 -- 172.21.15.84:0/918207480 >> v2:172.21.15.157:3300/0 conn(0x7fb8c0159ab0 msgr2=0x7fb8c00ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:20.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.120+0000 7fb8bffff640 1 --2- 172.21.15.84:0/918207480 >> v2:172.21.15.157:3300/0 conn(0x7fb8c0159ab0 0x7fb8c00ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:20.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.120+0000 7fb8bffff640 1 -- 172.21.15.84:0/918207480 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c00cd4d0 con 0x7fb8c006c140 2023-10-16T10:09:20.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.121+0000 7fb8bffff640 1 --2- 172.21.15.84:0/918207480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c006c140 0x7fb8c006c5d0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009e50 tx=0x7fb8b802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18534f02298378d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:20.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.122+0000 7fb8beffd640 1 -- 172.21.15.84:0/918207480 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b802c950 con 0x7fb8c006c140 2023-10-16T10:09:20.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.122+0000 7fb8beffd640 1 -- 172.21.15.84:0/918207480 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b802cab0 con 0x7fb8c006c140 2023-10-16T10:09:20.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.122+0000 7fb8beffd640 1 -- 172.21.15.84:0/918207480 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b802ce40 con 0x7fb8c006c140 2023-10-16T10:09:20.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.122+0000 7fb8c610f640 1 -- 172.21.15.84:0/918207480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c006c140 msgr2=0x7fb8c006c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:20.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.122+0000 7fb8c610f640 1 --2- 172.21.15.84:0/918207480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c006c140 0x7fb8c006c5d0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009e50 tx=0x7fb8b802bdb0 comp rx=0 tx=0).stop 2023-10-16T10:09:20.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.122+0000 7fb8c610f640 1 -- 172.21.15.84:0/918207480 shutdown_connections 2023-10-16T10:09:20.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.122+0000 7fb8c610f640 1 --2- 172.21.15.84:0/918207480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c006c140 0x7fb8c006c5d0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:20.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.122+0000 7fb8c610f640 1 --2- 172.21.15.84:0/918207480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c0159e90 0x7fb8c006fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:20.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.122+0000 7fb8c610f640 1 --2- 172.21.15.84:0/918207480 >> v2:172.21.15.157:3300/0 conn(0x7fb8c0159ab0 0x7fb8c00ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:20.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.122+0000 7fb8c610f640 1 -- 172.21.15.84:0/918207480 >> 172.21.15.84:0/918207480 conn(0x7fb8c00ce9b0 msgr2=0x7fb8c00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:20.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.122+0000 7fb8c610f640 1 -- 172.21.15.84:0/918207480 shutdown_connections 2023-10-16T10:09:20.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.122+0000 7fb8c610f640 1 -- 172.21.15.84:0/918207480 wait complete. 2023-10-16T10:09:20.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.124+0000 7fb8c610f640 1 Processor -- start 2023-10-16T10:09:20.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.124+0000 7fb8c610f640 1 -- start start 2023-10-16T10:09:20.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.125+0000 7fb8c610f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c006c140 0x7fb8c014fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:20.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.125+0000 7fb8c610f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8c0159ab0 0x7fb8c00c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:20.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.125+0000 7fb8c610f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c0159e90 0x7fb8c00c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:20.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.125+0000 7fb8c610f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c00c24e0 con 0x7fb8c0159e90 2023-10-16T10:09:20.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.125+0000 7fb8c610f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c00c2730 con 0x7fb8c0159ab0 2023-10-16T10:09:20.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.125+0000 7fb8c610f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c00c2a10 con 0x7fb8c006c140 2023-10-16T10:09:20.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.125+0000 7fb8bffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c0159e90 0x7fb8c00c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:20.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.125+0000 7fb8bffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c0159e90 0x7fb8c00c34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40068/0 (socket says 172.21.15.84:40068) 2023-10-16T10:09:20.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.125+0000 7fb8bffff640 1 -- 172.21.15.84:0/25091654 learned_addr learned my addr 172.21.15.84:0/25091654 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:20.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.126+0000 7fb8bf7fe640 1 --2- 172.21.15.84:0/25091654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c006c140 0x7fb8c014fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:20.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.126+0000 7fb8b7fff640 1 --2- 172.21.15.84:0/25091654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8c0159ab0 0x7fb8c00c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:20.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.126+0000 7fb8bffff640 1 -- 172.21.15.84:0/25091654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c006c140 msgr2=0x7fb8c014fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:20.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.126+0000 7fb8bffff640 1 --2- 172.21.15.84:0/25091654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c006c140 0x7fb8c014fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:20.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.126+0000 7fb8bffff640 1 -- 172.21.15.84:0/25091654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8c0159ab0 msgr2=0x7fb8c00c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:20.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.126+0000 7fb8bffff640 1 --2- 172.21.15.84:0/25091654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8c0159ab0 0x7fb8c00c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:20.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.126+0000 7fb8bffff640 1 -- 172.21.15.84:0/25091654 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b8009b30 con 0x7fb8c0159e90 2023-10-16T10:09:20.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.126+0000 7fb8b7fff640 1 --2- 172.21.15.84:0/25091654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8c0159ab0 0x7fb8c00c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:20.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.126+0000 7fb8bffff640 1 --2- 172.21.15.84:0/25091654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c0159e90 0x7fb8c00c34f0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8036040 tx=0x7fb8b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:20.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.126+0000 7fb8bcff9640 1 -- 172.21.15.84:0/25091654 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8009510 con 0x7fb8c0159e90 2023-10-16T10:09:20.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.126+0000 7fb8bcff9640 1 -- 172.21.15.84:0/25091654 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b803e450 con 0x7fb8c0159e90 2023-10-16T10:09:20.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.126+0000 7fb8bcff9640 1 -- 172.21.15.84:0/25091654 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b803e800 con 0x7fb8c0159e90 2023-10-16T10:09:20.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.127+0000 7fb8c610f640 1 -- 172.21.15.84:0/25091654 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c00c2cf0 con 0x7fb8c0159e90 2023-10-16T10:09:20.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.127+0000 7fb8c610f640 1 -- 172.21.15.84:0/25091654 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c00c53c0 con 0x7fb8c0159e90 2023-10-16T10:09:20.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.128+0000 7fb8c610f640 1 -- 172.21.15.84:0/25091654 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c004e160 con 0x7fb8c0159e90 2023-10-16T10:09:20.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.129+0000 7fb8bcff9640 1 -- 172.21.15.84:0/25091654 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb8b803e960 con 0x7fb8c0159e90 2023-10-16T10:09:20.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.129+0000 7fb8bcff9640 1 --2- 172.21.15.84:0/25091654 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fb8a006af90 0x7fb8a006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:20.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.130+0000 7fb8bcff9640 1 -- 172.21.15.84:0/25091654 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb8b80ac4f0 con 0x7fb8c0159e90 2023-10-16T10:09:20.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.130+0000 7fb8bf7fe640 1 --2- 172.21.15.84:0/25091654 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fb8a006af90 0x7fb8a006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:20.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.130+0000 7fb8bf7fe640 1 --2- 172.21.15.84:0/25091654 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fb8a006af90 0x7fb8a006d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0006fd0 tx=0x7fb8b0008040 comp rx=0 tx=0).ready entity=mgr.8904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:20.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.134+0000 7fb8bcff9640 1 -- 172.21.15.84:0/25091654 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb8b8077460 con 0x7fb8c0159e90 2023-10-16T10:09:20.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:20.374+0000 7fb8c610f640 1 -- 172.21.15.84:0/25091654 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fb8c00ccb60 con 0x7fb8c0159e90 2023-10-16T10:09:21.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.037+0000 7fb8bcff9640 1 -- 172.21.15.84:0/25091654 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v82) v1 ==== 92+0+0 (secure 0 0 0) 0x7fb8b807aa80 con 0x7fb8c0159e90 2023-10-16T10:09:21.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.041+0000 7fb8c610f640 1 -- 172.21.15.84:0/25091654 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fb8a006af90 msgr2=0x7fb8a006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:21.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.041+0000 7fb8c610f640 1 --2- 172.21.15.84:0/25091654 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fb8a006af90 0x7fb8a006d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0006fd0 tx=0x7fb8b0008040 comp rx=0 tx=0).stop 2023-10-16T10:09:21.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.041+0000 7fb8c610f640 1 -- 172.21.15.84:0/25091654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c0159e90 msgr2=0x7fb8c00c34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:21.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.041+0000 7fb8c610f640 1 --2- 172.21.15.84:0/25091654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c0159e90 0x7fb8c00c34f0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8036040 tx=0x7fb8b8004a40 comp rx=0 tx=0).stop 2023-10-16T10:09:21.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.041+0000 7fb8c610f640 1 -- 172.21.15.84:0/25091654 shutdown_connections 2023-10-16T10:09:21.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.041+0000 7fb8c610f640 1 --2- 172.21.15.84:0/25091654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c0159e90 0x7fb8c00c34f0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.041+0000 7fb8c610f640 1 --2- 172.21.15.84:0/25091654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8c0159ab0 0x7fb8c00c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.041+0000 7fb8c610f640 1 --2- 172.21.15.84:0/25091654 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fb8a006af90 0x7fb8a006d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.041+0000 7fb8c610f640 1 --2- 172.21.15.84:0/25091654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c006c140 0x7fb8c014fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.041+0000 7fb8c610f640 1 -- 172.21.15.84:0/25091654 >> 172.21.15.84:0/25091654 conn(0x7fb8c00ce9b0 msgr2=0x7fb8c0153c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:21.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.042+0000 7fb8c610f640 1 -- 172.21.15.84:0/25091654 shutdown_connections 2023-10-16T10:09:21.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.042+0000 7fb8c610f640 1 -- 172.21.15.84:0/25091654 wait complete. 2023-10-16T10:09:21.070 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T10:09:21.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.274+0000 7fd1ff59e640 1 Processor -- start 2023-10-16T10:09:21.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.274+0000 7fd1ff59e640 1 -- start start 2023-10-16T10:09:21.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.275+0000 7fd1ff59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20015cd70 0x7fd2000d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:21.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.276+0000 7fd1ff59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd2000b8480 0x7fd2000cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:21.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.276+0000 7fd1ff59e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd20015d150 con 0x7fd2000d0be0 2023-10-16T10:09:21.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.276+0000 7fd1ff59e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2000d1180 con 0x7fd2000b8480 2023-10-16T10:09:21.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.276+0000 7fd1ff59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000d1460 con 0x7fd20015cd70 2023-10-16T10:09:21.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.276+0000 7fd1f7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd2000b8480 0x7fd2000cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:21.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.276+0000 7fd1f7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd2000b8480 0x7fd2000cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40088/0 (socket says 172.21.15.84:40088) 2023-10-16T10:09:21.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.276+0000 7fd1f7fff640 1 -- 172.21.15.84:0/3184082885 learned_addr learned my addr 172.21.15.84:0/3184082885 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:21.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.276+0000 7fd1fe59c640 1 --2- 172.21.15.84:0/3184082885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20015cd70 0x7fd2000d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:21.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.276+0000 7fd1f7fff640 1 -- 172.21.15.84:0/3184082885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20015cd70 msgr2=0x7fd2000d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:21.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.276+0000 7fd1f7fff640 1 --2- 172.21.15.84:0/3184082885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20015cd70 0x7fd2000d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.276+0000 7fd1f7fff640 1 -- 172.21.15.84:0/3184082885 >> v1:172.21.15.157:6789/0 conn(0x7fd2000d0be0 legacy=0x7fd2000b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:21.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.276+0000 7fd1f7fff640 1 -- 172.21.15.84:0/3184082885 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2000d1740 con 0x7fd2000b8480 2023-10-16T10:09:21.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.276+0000 7fd1fe59c640 1 --2- 172.21.15.84:0/3184082885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20015cd70 0x7fd2000d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:21.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.278+0000 7fd1f7fff640 1 --2- 172.21.15.84:0/3184082885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2000b8480 0x7fd2000cd7c0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009870 tx=0x7fd1e802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd4ad2d541ab4ae server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:21.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.278+0000 7fd1fdd9b640 1 -- 172.21.15.84:0/3184082885 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8030840 con 0x7fd2000b8480 2023-10-16T10:09:21.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.278+0000 7fd1fdd9b640 1 -- 172.21.15.84:0/3184082885 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e80309a0 con 0x7fd2000b8480 2023-10-16T10:09:21.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.278+0000 7fd1fdd9b640 1 -- 172.21.15.84:0/3184082885 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8030d30 con 0x7fd2000b8480 2023-10-16T10:09:21.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.279+0000 7fd1ff59e640 1 -- 172.21.15.84:0/3184082885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2000b8480 msgr2=0x7fd2000cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:21.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.279+0000 7fd1ff59e640 1 --2- 172.21.15.84:0/3184082885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2000b8480 0x7fd2000cd7c0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009870 tx=0x7fd1e802b980 comp rx=0 tx=0).stop 2023-10-16T10:09:21.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.279+0000 7fd1ff59e640 1 -- 172.21.15.84:0/3184082885 shutdown_connections 2023-10-16T10:09:21.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.279+0000 7fd1ff59e640 1 --2- 172.21.15.84:0/3184082885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2000b8480 0x7fd2000cd7c0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.279+0000 7fd1ff59e640 1 --2- 172.21.15.84:0/3184082885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20015cd70 0x7fd2000d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.279+0000 7fd1ff59e640 1 -- 172.21.15.84:0/3184082885 >> 172.21.15.84:0/3184082885 conn(0x7fd2000cf570 msgr2=0x7fd2000cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:21.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.279+0000 7fd1ff59e640 1 -- 172.21.15.84:0/3184082885 shutdown_connections 2023-10-16T10:09:21.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.279+0000 7fd1ff59e640 1 -- 172.21.15.84:0/3184082885 wait complete. 2023-10-16T10:09:21.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.282+0000 7fd1ff59e640 1 Processor -- start 2023-10-16T10:09:21.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.282+0000 7fd1ff59e640 1 -- start start 2023-10-16T10:09:21.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.283+0000 7fd1ff59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2000b8480 0x7fd2001f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:21.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.283+0000 7fd1ff59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2000d0be0 0x7fd200145b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:21.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.283+0000 7fd1ff59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd20015cd70 0x7fd2001f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:21.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.283+0000 7fd1ff59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000c7f50 con 0x7fd20015cd70 2023-10-16T10:09:21.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.283+0000 7fd1ff59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000c7620 con 0x7fd2000b8480 2023-10-16T10:09:21.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.283+0000 7fd1ff59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000c7900 con 0x7fd2000d0be0 2023-10-16T10:09:21.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.284+0000 7fd1fed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd20015cd70 0x7fd2001f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:21.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.284+0000 7fd1fed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd20015cd70 0x7fd2001f58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40094/0 (socket says 172.21.15.84:40094) 2023-10-16T10:09:21.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.284+0000 7fd1fed9d640 1 -- 172.21.15.84:0/3564957522 learned_addr learned my addr 172.21.15.84:0/3564957522 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:21.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.284+0000 7fd1fe59c640 1 --2- 172.21.15.84:0/3564957522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2000b8480 0x7fd2001f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:21.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.284+0000 7fd1f7fff640 1 --2- 172.21.15.84:0/3564957522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2000d0be0 0x7fd200145b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:21.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.284+0000 7fd1fed9d640 1 -- 172.21.15.84:0/3564957522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2000d0be0 msgr2=0x7fd200145b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:21.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.284+0000 7fd1fed9d640 1 --2- 172.21.15.84:0/3564957522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2000d0be0 0x7fd200145b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.284+0000 7fd1fed9d640 1 -- 172.21.15.84:0/3564957522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2000b8480 msgr2=0x7fd2001f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:21.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.284+0000 7fd1fed9d640 1 --2- 172.21.15.84:0/3564957522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2000b8480 0x7fd2001f4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.284+0000 7fd1fed9d640 1 -- 172.21.15.84:0/3564957522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e80094d0 con 0x7fd20015cd70 2023-10-16T10:09:21.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.285+0000 7fd1fed9d640 1 --2- 172.21.15.84:0/3564957522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd20015cd70 0x7fd2001f58f0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800c730 tx=0x7fd1f800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:21.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.285+0000 7fd1f6ffd640 1 -- 172.21.15.84:0/3564957522 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8003b70 con 0x7fd20015cd70 2023-10-16T10:09:21.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.285+0000 7fd1f6ffd640 1 -- 172.21.15.84:0/3564957522 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f8003cf0 con 0x7fd20015cd70 2023-10-16T10:09:21.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.285+0000 7fd1ff59e640 1 -- 172.21.15.84:0/3564957522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2000c7be0 con 0x7fd20015cd70 2023-10-16T10:09:21.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.285+0000 7fd1f6ffd640 1 -- 172.21.15.84:0/3564957522 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8003ea0 con 0x7fd20015cd70 2023-10-16T10:09:21.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.285+0000 7fd1ff59e640 1 -- 172.21.15.84:0/3564957522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2001f5520 con 0x7fd20015cd70 2023-10-16T10:09:21.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.286+0000 7fd1ff59e640 1 -- 172.21.15.84:0/3564957522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd20004e9b0 con 0x7fd20015cd70 2023-10-16T10:09:21.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.287+0000 7fd1f6ffd640 1 -- 172.21.15.84:0/3564957522 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 82) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fd1f8005520 con 0x7fd20015cd70 2023-10-16T10:09:21.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.289+0000 7fd1f6ffd640 1 --2- 172.21.15.84:0/3564957522 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd1e006afe0 0x7fd1e006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:21.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.289+0000 7fd1fe59c640 1 -- 172.21.15.84:0/3564957522 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd1e006afe0 msgr2=0x7fd1e006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/205480762 2023-10-16T10:09:21.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.289+0000 7fd1f6ffd640 1 -- 172.21.15.84:0/3564957522 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd1f80896b0 con 0x7fd20015cd70 2023-10-16T10:09:21.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.289+0000 7fd1fe59c640 1 --2- 172.21.15.84:0/3564957522 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd1e006afe0 0x7fd1e006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:09:21.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.293+0000 7fd1f6ffd640 1 -- 172.21.15.84:0/3564957522 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd1f8054640 con 0x7fd20015cd70 2023-10-16T10:09:21.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.488+0000 7fd1ff59e640 1 -- 172.21.15.84:0/3564957522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fd20015c020 con 0x7fd20015cd70 2023-10-16T10:09:21.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.490+0000 7fd1fe59c640 1 -- 172.21.15.84:0/3564957522 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd1e006afe0 msgr2=0x7fd1e006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/205480762 2023-10-16T10:09:21.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.490+0000 7fd1fe59c640 1 --2- 172.21.15.84:0/3564957522 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd1e006afe0 0x7fd1e006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:09:21.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.493+0000 7fd1f6ffd640 1 -- 172.21.15.84:0/3564957522 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v53)=0 v53) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd1f8057c60 con 0x7fd20015cd70 2023-10-16T10:09:21.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.495+0000 7fd1ff59e640 1 -- 172.21.15.84:0/3564957522 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd1e006afe0 msgr2=0x7fd1e006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:09:21.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.495+0000 7fd1ff59e640 1 --2- 172.21.15.84:0/3564957522 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd1e006afe0 0x7fd1e006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.495+0000 7fd1ff59e640 1 -- 172.21.15.84:0/3564957522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd20015cd70 msgr2=0x7fd2001f58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:21.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.495+0000 7fd1ff59e640 1 --2- 172.21.15.84:0/3564957522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd20015cd70 0x7fd2001f58f0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800c730 tx=0x7fd1f800cbf0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.496+0000 7fd1ff59e640 1 -- 172.21.15.84:0/3564957522 shutdown_connections 2023-10-16T10:09:21.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.496+0000 7fd1ff59e640 1 --2- 172.21.15.84:0/3564957522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd20015cd70 0x7fd2001f58f0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.496+0000 7fd1ff59e640 1 --2- 172.21.15.84:0/3564957522 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7fd1e006afe0 0x7fd1e006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.496+0000 7fd1ff59e640 1 --2- 172.21.15.84:0/3564957522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2000d0be0 0x7fd200145b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.496+0000 7fd1ff59e640 1 --2- 172.21.15.84:0/3564957522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2000b8480 0x7fd2001f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.496+0000 7fd1ff59e640 1 -- 172.21.15.84:0/3564957522 >> 172.21.15.84:0/3564957522 conn(0x7fd2000cf570 msgr2=0x7fd2000b6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:21.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.496+0000 7fd1ff59e640 1 -- 172.21.15.84:0/3564957522 shutdown_connections 2023-10-16T10:09:21.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.496+0000 7fd1ff59e640 1 -- 172.21.15.84:0/3564957522 wait complete. 2023-10-16T10:09:21.516 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T10:09:21.586 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:21.586+0000 7fbb8a832040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:09:21.589 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:21.588+0000 7fbe4ce4b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:09:21.641 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:21.641+0000 7fbb8a832040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:09:21.646 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:21.644+0000 7fbe4ce4b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:09:21.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.725+0000 7f91ec2e7640 1 Processor -- start 2023-10-16T10:09:21.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.726+0000 7f91ec2e7640 1 -- start start 2023-10-16T10:09:21.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.727+0000 7f91ec2e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f91e4159ab0 0x7f91e40ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:21.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.727+0000 7f91ec2e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91e406c140 0x7f91e406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:21.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.727+0000 7f91ec2e7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e4070360 con 0x7f91e4159e90 2023-10-16T10:09:21.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.727+0000 7f91ec2e7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e406cb10 con 0x7f91e4159ab0 2023-10-16T10:09:21.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.727+0000 7f91ec2e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e415d890 con 0x7f91e406c140 2023-10-16T10:09:21.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.727+0000 7f91ea083640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f91e4159ab0 0x7f91e40ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:21.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.727+0000 7f91ea083640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f91e4159ab0 0x7f91e40ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:39928/0 (socket says 172.21.15.84:39928) 2023-10-16T10:09:21.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.727+0000 7f91ea083640 1 -- 172.21.15.84:0/3956717758 learned_addr learned my addr 172.21.15.84:0/3956717758 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:21.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.728+0000 7f91ea884640 1 --2- 172.21.15.84:0/3956717758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91e406c140 0x7f91e406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:21.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.728+0000 7f91ea083640 1 -- 172.21.15.84:0/3956717758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91e406c140 msgr2=0x7f91e406c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:21.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.728+0000 7f91ea083640 1 --2- 172.21.15.84:0/3956717758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91e406c140 0x7f91e406c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.728+0000 7f91ea083640 1 -- 172.21.15.84:0/3956717758 >> v1:172.21.15.157:6789/0 conn(0x7f91e4159e90 legacy=0x7f91e406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:21.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.728+0000 7f91ea083640 1 -- 172.21.15.84:0/3956717758 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e40cd1f0 con 0x7f91e4159ab0 2023-10-16T10:09:21.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.729+0000 7f91ea083640 1 --2- 172.21.15.84:0/3956717758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e4159ab0 0x7f91e40ce370 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009870 tx=0x7f91e002b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e98dde771bef2a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:21.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.729+0000 7f91e9081640 1 -- 172.21.15.84:0/3956717758 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e00317c0 con 0x7f91e4159ab0 2023-10-16T10:09:21.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.730+0000 7f91e9081640 1 -- 172.21.15.84:0/3956717758 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e0031920 con 0x7f91e4159ab0 2023-10-16T10:09:21.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.730+0000 7f91e9081640 1 -- 172.21.15.84:0/3956717758 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0031cb0 con 0x7f91e4159ab0 2023-10-16T10:09:21.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.730+0000 7f91ec2e7640 1 -- 172.21.15.84:0/3956717758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e4159ab0 msgr2=0x7f91e40ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:21.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.730+0000 7f91ec2e7640 1 --2- 172.21.15.84:0/3956717758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e4159ab0 0x7f91e40ce370 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009870 tx=0x7f91e002b860 comp rx=0 tx=0).stop 2023-10-16T10:09:21.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.730+0000 7f91ec2e7640 1 -- 172.21.15.84:0/3956717758 shutdown_connections 2023-10-16T10:09:21.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.730+0000 7f91ec2e7640 1 --2- 172.21.15.84:0/3956717758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91e406c140 0x7f91e406c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.730+0000 7f91ec2e7640 1 --2- 172.21.15.84:0/3956717758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e4159ab0 0x7f91e40ce370 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.730+0000 7f91ec2e7640 1 -- 172.21.15.84:0/3956717758 >> 172.21.15.84:0/3956717758 conn(0x7f91e40ce9b0 msgr2=0x7f91e40cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:21.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.730+0000 7f91ec2e7640 1 -- 172.21.15.84:0/3956717758 shutdown_connections 2023-10-16T10:09:21.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.730+0000 7f91ec2e7640 1 -- 172.21.15.84:0/3956717758 wait complete. 2023-10-16T10:09:21.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.732+0000 7f91ec2e7640 1 Processor -- start 2023-10-16T10:09:21.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.732+0000 7f91ec2e7640 1 -- start start 2023-10-16T10:09:21.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.733+0000 7f91ec2e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91e406c140 0x7f91e40af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:21.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.733+0000 7f91ec2e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e4159ab0 0x7f91e40af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:21.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.733+0000 7f91ec2e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e4159e90 0x7f91e414f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:21.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.733+0000 7f91ec2e7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e415ccc0 con 0x7f91e4159ab0 2023-10-16T10:09:21.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.733+0000 7f91ec2e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e415c0b0 con 0x7f91e4159e90 2023-10-16T10:09:21.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.733+0000 7f91ec2e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e415c390 con 0x7f91e406c140 2023-10-16T10:09:21.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.733+0000 7f91e9882640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e4159ab0 0x7f91e40af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:21.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.733+0000 7f91e9882640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e4159ab0 0x7f91e40af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40110/0 (socket says 172.21.15.84:40110) 2023-10-16T10:09:21.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.733+0000 7f91e9882640 1 -- 172.21.15.84:0/121850305 learned_addr learned my addr 172.21.15.84:0/121850305 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:21.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.733+0000 7f91ea884640 1 --2- 172.21.15.84:0/121850305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e4159e90 0x7f91e414f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:21.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.734+0000 7f91ea083640 1 --2- 172.21.15.84:0/121850305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91e406c140 0x7f91e40af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:21.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.734+0000 7f91e9882640 1 -- 172.21.15.84:0/121850305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91e406c140 msgr2=0x7f91e40af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:21.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.734+0000 7f91e9882640 1 --2- 172.21.15.84:0/121850305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91e406c140 0x7f91e40af250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.734+0000 7f91e9882640 1 -- 172.21.15.84:0/121850305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e4159e90 msgr2=0x7f91e414f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:21.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.734+0000 7f91e9882640 1 --2- 172.21.15.84:0/121850305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e4159e90 0x7f91e414f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.734+0000 7f91e9882640 1 -- 172.21.15.84:0/121850305 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e00094d0 con 0x7f91e4159ab0 2023-10-16T10:09:21.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.734+0000 7f91ea083640 1 --2- 172.21.15.84:0/121850305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91e406c140 0x7f91e40af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:21.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.734+0000 7f91e9882640 1 --2- 172.21.15.84:0/121850305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e4159ab0 0x7f91e40af790 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f91d400c400 tx=0x7f91d400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:21.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.734+0000 7f91daffd640 1 -- 172.21.15.84:0/121850305 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d40052d0 con 0x7f91e4159ab0 2023-10-16T10:09:21.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.734+0000 7f91daffd640 1 -- 172.21.15.84:0/121850305 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d4005450 con 0x7f91e4159ab0 2023-10-16T10:09:21.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.734+0000 7f91daffd640 1 -- 172.21.15.84:0/121850305 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4007260 con 0x7f91e4159ab0 2023-10-16T10:09:21.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.734+0000 7f91ec2e7640 1 -- 172.21.15.84:0/121850305 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e415c670 con 0x7f91e4159ab0 2023-10-16T10:09:21.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.734+0000 7f91ec2e7640 1 -- 172.21.15.84:0/121850305 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e41675a0 con 0x7f91e4159ab0 2023-10-16T10:09:21.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.736+0000 7f91ec2e7640 1 -- 172.21.15.84:0/121850305 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e4167760 con 0x7f91e4159ab0 2023-10-16T10:09:21.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.737+0000 7f91daffd640 1 -- 172.21.15.84:0/121850305 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 82) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f91d40073c0 con 0x7f91e4159ab0 2023-10-16T10:09:21.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.738+0000 7f91daffd640 1 --2- 172.21.15.84:0/121850305 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f91c806afe0 0x7f91c806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:21.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.738+0000 7f91daffd640 1 -- 172.21.15.84:0/121850305 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f91d4088b60 con 0x7f91e4159ab0 2023-10-16T10:09:21.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.738+0000 7f91ea083640 1 -- 172.21.15.84:0/121850305 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f91c806afe0 msgr2=0x7f91c806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/205480762 2023-10-16T10:09:21.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.738+0000 7f91ea083640 1 --2- 172.21.15.84:0/121850305 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f91c806afe0 0x7f91c806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:09:21.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.746+0000 7f91daffd640 1 -- 172.21.15.84:0/121850305 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f91d4053b90 con 0x7f91e4159ab0 2023-10-16T10:09:21.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.939+0000 7f91ea083640 1 -- 172.21.15.84:0/121850305 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f91c806afe0 msgr2=0x7f91c806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/205480762 2023-10-16T10:09:21.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.939+0000 7f91ea083640 1 --2- 172.21.15.84:0/121850305 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f91c806afe0 0x7f91c806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:09:21.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.940+0000 7f91ec2e7640 1 -- 172.21.15.84:0/121850305 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f91e404e160 con 0x7f91e4159ab0 2023-10-16T10:09:21.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.947+0000 7f91daffd640 1 -- 172.21.15.84:0/121850305 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v54)=0 v54) v1 ==== 133+0+0 (secure 0 0 0) 0x7f91d40571b0 con 0x7f91e4159ab0 2023-10-16T10:09:21.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.949+0000 7f91ec2e7640 1 -- 172.21.15.84:0/121850305 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f91c806afe0 msgr2=0x7f91c806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:09:21.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.949+0000 7f91ec2e7640 1 --2- 172.21.15.84:0/121850305 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f91c806afe0 0x7f91c806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.949+0000 7f91ec2e7640 1 -- 172.21.15.84:0/121850305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e4159ab0 msgr2=0x7f91e40af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:21.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.949+0000 7f91ec2e7640 1 --2- 172.21.15.84:0/121850305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e4159ab0 0x7f91e40af790 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f91d400c400 tx=0x7f91d400c8c0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.949+0000 7f91ec2e7640 1 -- 172.21.15.84:0/121850305 shutdown_connections 2023-10-16T10:09:21.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.949+0000 7f91ec2e7640 1 --2- 172.21.15.84:0/121850305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e4159e90 0x7f91e414f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.949+0000 7f91ec2e7640 1 --2- 172.21.15.84:0/121850305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e4159ab0 0x7f91e40af790 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.949+0000 7f91ec2e7640 1 --2- 172.21.15.84:0/121850305 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f91c806afe0 0x7f91c806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.949+0000 7f91ec2e7640 1 --2- 172.21.15.84:0/121850305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91e406c140 0x7f91e40af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:21.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.949+0000 7f91ec2e7640 1 -- 172.21.15.84:0/121850305 >> 172.21.15.84:0/121850305 conn(0x7f91e40ce9b0 msgr2=0x7f91e406fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:21.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.949+0000 7f91ec2e7640 1 -- 172.21.15.84:0/121850305 shutdown_connections 2023-10-16T10:09:21.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:21.949+0000 7f91ec2e7640 1 -- 172.21.15.84:0/121850305 wait complete. 2023-10-16T10:09:21.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:09:21.970 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-16T10:09:22.030 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1 1m 2023-10-16T10:09:22.096 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:22.096+0000 7fbb8a832040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:09:22.117 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:22.115+0000 7fbe4ce4b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:09:22.149 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:22.149+0000 7fbb8a832040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:09:22.171 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:22.169+0000 7fbe4ce4b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:09:22.200 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:22.200+0000 7fbb8a832040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:09:22.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.221+0000 7f1c874df640 1 Processor -- start 2023-10-16T10:09:22.223 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:22.221+0000 7fbe4ce4b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:09:22.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.221+0000 7f1c874df640 1 -- start start 2023-10-16T10:09:22.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.223+0000 7f1c874df640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8015ce30 0x7f1c800caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:22.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.223+0000 7f1c874df640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1c800cdb70 0x7f1c800ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:22.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.223+0000 7f1c874df640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c800cb410 con 0x7f1c80070260 2023-10-16T10:09:22.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.223+0000 7f1c874df640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c800ce540 con 0x7f1c800cdb70 2023-10-16T10:09:22.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.223+0000 7f1c874df640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c800c9e30 con 0x7f1c8015ce30 2023-10-16T10:09:22.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.223+0000 7f1c86cde640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1c800cdb70 0x7f1c800ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:22.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.223+0000 7f1c86cde640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1c800cdb70 0x7f1c800ce000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40118/0 (socket says 172.21.15.84:40118) 2023-10-16T10:09:22.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.223+0000 7f1c86cde640 1 -- 172.21.15.84:0/2162631246 learned_addr learned my addr 172.21.15.84:0/2162631246 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:22.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.224+0000 7f1c864dd640 1 --2- 172.21.15.84:0/2162631246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8015ce30 0x7f1c800caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:22.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.224+0000 7f1c86cde640 1 -- 172.21.15.84:0/2162631246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8015ce30 msgr2=0x7f1c800caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:22.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.224+0000 7f1c86cde640 1 --2- 172.21.15.84:0/2162631246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8015ce30 0x7f1c800caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:22.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.224+0000 7f1c86cde640 1 -- 172.21.15.84:0/2162631246 >> v1:172.21.15.157:6789/0 conn(0x7f1c80070260 legacy=0x7f1c800c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:22.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.224+0000 7f1c86cde640 1 -- 172.21.15.84:0/2162631246 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c800ca110 con 0x7f1c800cdb70 2023-10-16T10:09:22.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.226+0000 7f1c86cde640 1 --2- 172.21.15.84:0/2162631246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c800cdb70 0x7f1c800ce000 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009e50 tx=0x7f1c7802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590187b602e4be81 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:22.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.226+0000 7f1c854db640 1 -- 172.21.15.84:0/2162631246 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7802ca70 con 0x7f1c800cdb70 2023-10-16T10:09:22.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.226+0000 7f1c854db640 1 -- 172.21.15.84:0/2162631246 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7802cbd0 con 0x7f1c800cdb70 2023-10-16T10:09:22.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.226+0000 7f1c854db640 1 -- 172.21.15.84:0/2162631246 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c780343f0 con 0x7f1c800cdb70 2023-10-16T10:09:22.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.227+0000 7f1c874df640 1 -- 172.21.15.84:0/2162631246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c800cdb70 msgr2=0x7f1c800ce000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:22.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.227+0000 7f1c874df640 1 --2- 172.21.15.84:0/2162631246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c800cdb70 0x7f1c800ce000 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009e50 tx=0x7f1c7802bed0 comp rx=0 tx=0).stop 2023-10-16T10:09:22.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.227+0000 7f1c874df640 1 -- 172.21.15.84:0/2162631246 shutdown_connections 2023-10-16T10:09:22.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.227+0000 7f1c874df640 1 --2- 172.21.15.84:0/2162631246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c800cdb70 0x7f1c800ce000 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:22.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.227+0000 7f1c874df640 1 --2- 172.21.15.84:0/2162631246 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8015ce30 0x7f1c800caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:22.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.227+0000 7f1c874df640 1 -- 172.21.15.84:0/2162631246 >> 172.21.15.84:0/2162631246 conn(0x7f1c800b0a80 msgr2=0x7f1c800b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:22.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.227+0000 7f1c874df640 1 -- 172.21.15.84:0/2162631246 shutdown_connections 2023-10-16T10:09:22.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.227+0000 7f1c874df640 1 -- 172.21.15.84:0/2162631246 wait complete. 2023-10-16T10:09:22.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.230+0000 7f1c874df640 1 Processor -- start 2023-10-16T10:09:22.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.230+0000 7f1c874df640 1 -- start start 2023-10-16T10:09:22.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.231+0000 7f1c874df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c80070260 0x7f1c8006c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:22.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.231+0000 7f1c874df640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c800cdb70 0x7f1c800c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:22.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.231+0000 7f1c874df640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8015ce30 0x7f1c8014b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:22.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.232+0000 7f1c874df640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8006b910 con 0x7f1c800cdb70 2023-10-16T10:09:22.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.232+0000 7f1c874df640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c800c7bb0 con 0x7f1c80070260 2023-10-16T10:09:22.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.232+0000 7f1c874df640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8015beb0 con 0x7f1c8015ce30 2023-10-16T10:09:22.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.232+0000 7f1c85cdc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c800cdb70 0x7f1c800c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:22.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.232+0000 7f1c85cdc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c800cdb70 0x7f1c800c62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40130/0 (socket says 172.21.15.84:40130) 2023-10-16T10:09:22.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.232+0000 7f1c85cdc640 1 -- 172.21.15.84:0/2740304916 learned_addr learned my addr 172.21.15.84:0/2740304916 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:22.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.232+0000 7f1c864dd640 1 --2- 172.21.15.84:0/2740304916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c80070260 0x7f1c8006c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:22.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.232+0000 7f1c86cde640 1 --2- 172.21.15.84:0/2740304916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8015ce30 0x7f1c8014b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:22.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.232+0000 7f1c85cdc640 1 -- 172.21.15.84:0/2740304916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8015ce30 msgr2=0x7f1c8014b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:22.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.232+0000 7f1c85cdc640 1 --2- 172.21.15.84:0/2740304916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8015ce30 0x7f1c8014b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:22.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.232+0000 7f1c85cdc640 1 -- 172.21.15.84:0/2740304916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c80070260 msgr2=0x7f1c8006c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:22.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.232+0000 7f1c85cdc640 1 --2- 172.21.15.84:0/2740304916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c80070260 0x7f1c8006c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:22.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.232+0000 7f1c85cdc640 1 -- 172.21.15.84:0/2740304916 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c78009b30 con 0x7f1c800cdb70 2023-10-16T10:09:22.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.232+0000 7f1c86cde640 1 --2- 172.21.15.84:0/2740304916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8015ce30 0x7f1c8014b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:22.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.233+0000 7f1c85cdc640 1 --2- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c800cdb70 0x7f1c800c62d0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000c430 tx=0x7f1c7000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:22.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.233+0000 7f1c76ffd640 1 -- 172.21.15.84:0/2740304916 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c700052d0 con 0x7f1c800cdb70 2023-10-16T10:09:22.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.233+0000 7f1c76ffd640 1 -- 172.21.15.84:0/2740304916 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c70005430 con 0x7f1c800cdb70 2023-10-16T10:09:22.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.233+0000 7f1c874df640 1 -- 172.21.15.84:0/2740304916 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8006baf0 con 0x7f1c800cdb70 2023-10-16T10:09:22.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.233+0000 7f1c76ffd640 1 -- 172.21.15.84:0/2740304916 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70003830 con 0x7f1c800cdb70 2023-10-16T10:09:22.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.233+0000 7f1c874df640 1 -- 172.21.15.84:0/2740304916 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8006c0b0 con 0x7f1c800cdb70 2023-10-16T10:09:22.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.234+0000 7f1c874df640 1 -- 172.21.15.84:0/2740304916 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c8004e9b0 con 0x7f1c800cdb70 2023-10-16T10:09:22.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.235+0000 7f1c76ffd640 1 -- 172.21.15.84:0/2740304916 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 82) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f1c70003990 con 0x7f1c800cdb70 2023-10-16T10:09:22.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.236+0000 7f1c76ffd640 1 --2- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f1c6406afe0 0x7f1c6406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:22.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.237+0000 7f1c76ffd640 1 -- 172.21.15.84:0/2740304916 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1c70088e10 con 0x7f1c800cdb70 2023-10-16T10:09:22.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.237+0000 7f1c864dd640 1 -- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f1c6406afe0 msgr2=0x7f1c6406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/205480762 2023-10-16T10:09:22.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.237+0000 7f1c864dd640 1 --2- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f1c6406afe0 0x7f1c6406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:09:22.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.240+0000 7f1c76ffd640 1 -- 172.21.15.84:0/2740304916 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1c7008c030 con 0x7f1c800cdb70 2023-10-16T10:09:22.263 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:22.263+0000 7fbb8a832040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:09:22.291 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:22.289+0000 7fbe4ce4b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:09:22.313 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:22.313+0000 7fbb8a832040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:09:22.342 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:22.341+0000 7fbe4ce4b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:09:22.425 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:22.424+0000 7fbb8a832040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:09:22.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.438+0000 7f1c864dd640 1 -- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f1c6406afe0 msgr2=0x7f1c6406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/205480762 2023-10-16T10:09:22.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.438+0000 7f1c864dd640 1 --2- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f1c6406afe0 0x7f1c6406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:09:22.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.455+0000 7f1c874df640 1 -- 172.21.15.84:0/2740304916 --> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c800ca6d0 con 0x7f1c6406afe0 2023-10-16T10:09:22.457 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:22.456+0000 7fbe4ce4b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:09:22.488 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:22.487+0000 7fbb8a832040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:09:22.520 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:22.518+0000 7fbe4ce4b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:09:22.538 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:22.537+0000 7fbb8a832040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:09:22.570 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:22.568+0000 7fbe4ce4b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:09:22.595 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:22.595+0000 7fbb8a832040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:09:22.627 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:22.625+0000 7fbe4ce4b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:09:22.646 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:22.645+0000 7fbb8a832040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:09:22.676 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:22.675+0000 7fbe4ce4b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:09:22.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.839+0000 7f1c864dd640 1 -- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f1c6406afe0 msgr2=0x7f1c6406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/205480762 2023-10-16T10:09:22.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:22.839+0000 7f1c864dd640 1 --2- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f1c6406afe0 0x7f1c6406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T10:09:23.031 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:23.030+0000 7fbb8a832040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:09:23.058 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:23.057+0000 7fbe4ce4b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:09:23.191 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:23.189+0000 7fbb8a832040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:09:23.218 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:23.217+0000 7fbe4ce4b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:09:23.299 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:23.297+0000 7fbb8a832040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:09:23.327 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:23.325+0000 7fbe4ce4b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:09:23.349 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:23.348+0000 7fbb8a832040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:09:23.377 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:23.375+0000 7fbe4ce4b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:09:23.457 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:23.456+0000 7fbb8a832040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:09:23.484 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:23.483+0000 7fbe4ce4b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:09:23.564 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:23.563+0000 7fbb8a832040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:09:23.592 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:23.590+0000 7fbe4ce4b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:09:23.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:23.640+0000 7f1c864dd640 1 -- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f1c6406afe0 msgr2=0x7f1c6406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/205480762 2023-10-16T10:09:23.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:23.640+0000 7f1c864dd640 1 --2- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f1c6406afe0 0x7f1c6406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T10:09:23.722 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:23.720+0000 7fbb8a832040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:09:23.749 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:23.747+0000 7fbe4ce4b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:09:23.771 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:23.770+0000 7fbb8a832040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:09:23.797 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:23.795+0000 7fbe4ce4b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:09:23.892 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:23.891+0000 7fbb8a832040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:09:23.916 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:23.914+0000 7fbe4ce4b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:09:24.693 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:24.691+0000 7fbb8a832040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:09:24.741 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:24.739+0000 7fbe4ce4b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:09:24.991 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:24.990+0000 7fbb8a832040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:09:25.036 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:25.034+0000 7fbe4ce4b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:09:25.049 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:25.047+0000 7fbb8a832040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:09:25.092 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:25.091+0000 7fbe4ce4b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:09:25.164 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T10:09:25.162+0000 7fbb8a832040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:09:25.206 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:25.204+0000 7fbe4ce4b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:09:25.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:25.221+0000 7f1c76ffd640 1 -- 172.21.15.84:0/2740304916 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 83) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f1c700540c0 con 0x7f1c800cdb70 2023-10-16T10:09:25.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:25.222+0000 7f1c76ffd640 1 -- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f1c6406afe0 msgr2=0x7f1c6406d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:09:25.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:25.222+0000 7f1c76ffd640 1 --2- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:6832/205480762,v1:172.21.15.84:6833/205480762] conn(0x7f1c6406afe0 0x7f1c6406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:25.589 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:25.587+0000 7fbdcc99d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:09:25.589 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:25.587+0000 7fbdcc99d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:09:25.590 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:25.587+0000 7fbdcc99d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:09:25.590 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:25.587+0000 7fbdcc99d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:09:25.590 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:25.587+0000 7fbdcc99d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:09:25.591 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:25.588+0000 7fbdc9957640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:09:25.591 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:25.588+0000 7fbdc9957640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:09:25.591 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:25.588+0000 7fbdc9957640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:09:25.592 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:25.588+0000 7fbdc9957640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:09:25.592 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:09:25.588+0000 7fbdc9957640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:09:26.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.257+0000 7f1c76ffd640 1 -- 172.21.15.84:0/2740304916 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 84) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f1c70087270 con 0x7f1c800cdb70 2023-10-16T10:09:26.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.258+0000 7f1c76ffd640 1 --2- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f1c640355a0 0x7f1c64035980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:26.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.258+0000 7f1c76ffd640 1 -- 172.21.15.84:0/2740304916 --> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c64035ec0 con 0x7f1c640355a0 2023-10-16T10:09:26.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.268+0000 7f1c864dd640 1 --2- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f1c640355a0 0x7f1c64035980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:26.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.279+0000 7f1c864dd640 1 --2- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f1c640355a0 0x7f1c64035980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1c800ce4e0 tx=0x7f1c7c008040 comp rx=0 tx=0).ready entity=mgr.9327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:26.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.468+0000 7f1c76ffd640 1 -- 172.21.15.84:0/2740304916 <== mgr.9327 v2:172.21.15.84:6832/3746729142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f1c64035ec0 con 0x7f1c640355a0 2023-10-16T10:09:26.470 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path /snap_test_dir1 2023-10-16T10:09:26.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.472+0000 7f1c874df640 1 -- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f1c640355a0 msgr2=0x7f1c64035980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:26.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.472+0000 7f1c874df640 1 --2- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f1c640355a0 0x7f1c64035980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1c800ce4e0 tx=0x7f1c7c008040 comp rx=0 tx=0).stop 2023-10-16T10:09:26.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.472+0000 7f1c874df640 1 -- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c800cdb70 msgr2=0x7f1c800c62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:26.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.472+0000 7f1c874df640 1 --2- 172.21.15.84:0/2740304916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c800cdb70 0x7f1c800c62d0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000c430 tx=0x7f1c7000c8f0 comp rx=0 tx=0).stop 2023-10-16T10:09:26.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.472+0000 7f1c86cde640 1 -- 172.21.15.84:0/2740304916 reap_dead start 2023-10-16T10:09:26.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.472+0000 7f1c874df640 1 -- 172.21.15.84:0/2740304916 shutdown_connections 2023-10-16T10:09:26.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.472+0000 7f1c874df640 1 -- 172.21.15.84:0/2740304916 >> 172.21.15.84:0/2740304916 conn(0x7f1c800b0a80 msgr2=0x7f1c800ac4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:26.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.472+0000 7f1c874df640 1 -- 172.21.15.84:0/2740304916 shutdown_connections 2023-10-16T10:09:26.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.472+0000 7f1c874df640 1 -- 172.21.15.84:0/2740304916 wait complete. 2023-10-16T10:09:26.493 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1 2023-10-16T10:09:26.494 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add /snap_test_dir1 55n 2023-10-16T10:09:26.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.705+0000 7f3ea8dfb640 1 Processor -- start 2023-10-16T10:09:26.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.705+0000 7f3ea8dfb640 1 -- start start 2023-10-16T10:09:26.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.707+0000 7f3ea8dfb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3ea415ac10 0x7f3ea406c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:26.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.707+0000 7f3ea8dfb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea415b310 con 0x7f3ea40b0360 2023-10-16T10:09:26.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.707+0000 7f3ea8dfb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea406f640 con 0x7f3ea40cac20 2023-10-16T10:09:26.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.707+0000 7f3ea8dfb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea415b5b0 con 0x7f3ea415ac10 2023-10-16T10:09:26.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.707+0000 7f3ea259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3ea415ac10 0x7f3ea406c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:26.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.707+0000 7f3e99d9b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3ea40b0360 0x7f3ea40ca510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33186/0 (socket says 172.21.15.84:33186) 2023-10-16T10:09:26.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.707+0000 7f3e99d9b640 1 -- 172.21.15.84:0/3242861969 learned_addr learned my addr 172.21.15.84:0/3242861969 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:09:26.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.708+0000 7f3ea259c640 1 -- 172.21.15.84:0/3242861969 >> v1:172.21.15.157:6789/0 conn(0x7f3ea40cac20 legacy=0x7f3ea415e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:26.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.708+0000 7f3ea259c640 1 -- 172.21.15.84:0/3242861969 >> v1:172.21.15.84:6789/0 conn(0x7f3ea40b0360 legacy=0x7f3ea40ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:26.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.708+0000 7f3ea259c640 1 -- 172.21.15.84:0/3242861969 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea415b890 con 0x7f3ea415ac10 2023-10-16T10:09:26.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.708+0000 7f3ea1d9b640 1 -- 172.21.15.84:0/3242861969 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922326512 0 0) 0x7f3ea415b310 con 0x7f3ea40b0360 2023-10-16T10:09:26.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.709+0000 7f3ea259c640 1 --2- 172.21.15.84:0/3242861969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea415ac10 0x7f3ea406c730 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f3e940097a0 tx=0x7f3e9402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2244ddf4a57ceca0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:26.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.710+0000 7f3ea1d9b640 1 -- 172.21.15.84:0/3242861969 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9402e070 con 0x7f3ea415ac10 2023-10-16T10:09:26.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.710+0000 7f3ea1d9b640 1 -- 172.21.15.84:0/3242861969 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e940315a0 con 0x7f3ea415ac10 2023-10-16T10:09:26.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.710+0000 7f3ea1d9b640 1 -- 172.21.15.84:0/3242861969 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94031930 con 0x7f3ea415ac10 2023-10-16T10:09:26.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.710+0000 7f3ea8dfb640 1 -- 172.21.15.84:0/3242861969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea415ac10 msgr2=0x7f3ea406c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:26.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.710+0000 7f3ea8dfb640 1 --2- 172.21.15.84:0/3242861969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea415ac10 0x7f3ea406c730 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f3e940097a0 tx=0x7f3e9402b840 comp rx=0 tx=0).stop 2023-10-16T10:09:26.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.711+0000 7f3ea8dfb640 1 -- 172.21.15.84:0/3242861969 shutdown_connections 2023-10-16T10:09:26.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.711+0000 7f3ea8dfb640 1 --2- 172.21.15.84:0/3242861969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea415ac10 0x7f3ea406c730 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:26.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.711+0000 7f3ea8dfb640 1 -- 172.21.15.84:0/3242861969 >> 172.21.15.84:0/3242861969 conn(0x7f3ea40cb360 msgr2=0x7f3ea40cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:26.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.711+0000 7f3ea8dfb640 1 -- 172.21.15.84:0/3242861969 shutdown_connections 2023-10-16T10:09:26.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.711+0000 7f3ea8dfb640 1 -- 172.21.15.84:0/3242861969 wait complete. 2023-10-16T10:09:26.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.713+0000 7f3ea8dfb640 1 Processor -- start 2023-10-16T10:09:26.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.713+0000 7f3ea8dfb640 1 -- start start 2023-10-16T10:09:26.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.714+0000 7f3ea8dfb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea40b0360 0x7f3ea4145090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:26.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.714+0000 7f3ea8dfb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea40cac20 0x7f3ea40b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:26.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.714+0000 7f3ea8dfb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea415ac10 0x7f3ea40c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:26.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.714+0000 7f3ea8dfb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea40b1a40 con 0x7f3ea415ac10 2023-10-16T10:09:26.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.714+0000 7f3ea8dfb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea40b1ca0 con 0x7f3ea40cac20 2023-10-16T10:09:26.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.714+0000 7f3ea8dfb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea40b1f80 con 0x7f3ea40b0360 2023-10-16T10:09:26.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.715+0000 7f3ea2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea415ac10 0x7f3ea40c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:26.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.715+0000 7f3ea2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea415ac10 0x7f3ea40c4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34362/0 (socket says 172.21.15.84:34362) 2023-10-16T10:09:26.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.715+0000 7f3ea2d9d640 1 -- 172.21.15.84:0/2929622177 learned_addr learned my addr 172.21.15.84:0/2929622177 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:26.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.715+0000 7f3ea2d9d640 1 -- 172.21.15.84:0/2929622177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea40b0360 msgr2=0x7f3ea4145090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:26.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.715+0000 7f3ea2d9d640 1 --2- 172.21.15.84:0/2929622177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea40b0360 0x7f3ea4145090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:26.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.715+0000 7f3ea2d9d640 1 -- 172.21.15.84:0/2929622177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea40cac20 msgr2=0x7f3ea40b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:26.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.715+0000 7f3ea2d9d640 1 --2- 172.21.15.84:0/2929622177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea40cac20 0x7f3ea40b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:26.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.715+0000 7f3ea2d9d640 1 -- 172.21.15.84:0/2929622177 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e94009400 con 0x7f3ea415ac10 2023-10-16T10:09:26.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.715+0000 7f3ea2d9d640 1 --2- 172.21.15.84:0/2929622177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea415ac10 0x7f3ea40c4510 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00c780 tx=0x7f3e9c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:26.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.715+0000 7f3e9b7fe640 1 -- 172.21.15.84:0/2929622177 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c004b60 con 0x7f3ea415ac10 2023-10-16T10:09:26.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.716+0000 7f3e9b7fe640 1 -- 172.21.15.84:0/2929622177 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9c00ce90 con 0x7f3ea415ac10 2023-10-16T10:09:26.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.716+0000 7f3e9b7fe640 1 -- 172.21.15.84:0/2929622177 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c00a850 con 0x7f3ea415ac10 2023-10-16T10:09:26.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.716+0000 7f3ea8dfb640 1 -- 172.21.15.84:0/2929622177 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea40b2260 con 0x7f3ea415ac10 2023-10-16T10:09:26.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.716+0000 7f3ea8dfb640 1 -- 172.21.15.84:0/2929622177 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea40b2820 con 0x7f3ea415ac10 2023-10-16T10:09:26.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.717+0000 7f3ea8dfb640 1 -- 172.21.15.84:0/2929622177 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea404e160 con 0x7f3ea415ac10 2023-10-16T10:09:26.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.718+0000 7f3e9b7fe640 1 -- 172.21.15.84:0/2929622177 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 84) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f3e9c00a9b0 con 0x7f3ea415ac10 2023-10-16T10:09:26.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.719+0000 7f3e9b7fe640 1 --2- 172.21.15.84:0/2929622177 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f3e8406ad10 0x7f3e8406d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:26.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.719+0000 7f3e9b7fe640 1 -- 172.21.15.84:0/2929622177 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f3e9c0895d0 con 0x7f3ea415ac10 2023-10-16T10:09:26.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.720+0000 7f3ea259c640 1 --2- 172.21.15.84:0/2929622177 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f3e8406ad10 0x7f3e8406d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:26.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.720+0000 7f3ea259c640 1 --2- 172.21.15.84:0/2929622177 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f3e8406ad10 0x7f3e8406d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3e94036000 tx=0x7f3e94030000 comp rx=0 tx=0).ready entity=mgr.9327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:26.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.724+0000 7f3e9b7fe640 1 -- 172.21.15.84:0/2929622177 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3e9c054840 con 0x7f3ea415ac10 2023-10-16T10:09:26.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:26.938+0000 7f3ea8dfb640 1 -- 172.21.15.84:0/2929622177 --> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "/snap_test_dir1", "retention_spec_or_period": "55n", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ea415be50 con 0x7f3e8406ad10 2023-10-16T10:09:27.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.111+0000 7f3e9b7fe640 1 -- 172.21.15.84:0/2929622177 <== mgr.9327 v2:172.21.15.84:6832/3746729142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+40 (secure 0 0 0) 0x7f3e94031a50 con 0x7f3e8406ad10 2023-10-16T10:09:27.113 INFO:teuthology.orchestra.run.smithi084.stdout:Retention added to path /snap_test_dir1 2023-10-16T10:09:27.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.114+0000 7f3ea8dfb640 1 -- 172.21.15.84:0/2929622177 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f3e8406ad10 msgr2=0x7f3e8406d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:27.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.114+0000 7f3ea8dfb640 1 --2- 172.21.15.84:0/2929622177 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f3e8406ad10 0x7f3e8406d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3e94036000 tx=0x7f3e94030000 comp rx=0 tx=0).stop 2023-10-16T10:09:27.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.114+0000 7f3ea8dfb640 1 -- 172.21.15.84:0/2929622177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea415ac10 msgr2=0x7f3ea40c4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:27.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.114+0000 7f3ea8dfb640 1 --2- 172.21.15.84:0/2929622177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea415ac10 0x7f3ea40c4510 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00c780 tx=0x7f3e9c00cc40 comp rx=0 tx=0).stop 2023-10-16T10:09:27.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.114+0000 7f3ea8dfb640 1 -- 172.21.15.84:0/2929622177 shutdown_connections 2023-10-16T10:09:27.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.114+0000 7f3ea8dfb640 1 --2- 172.21.15.84:0/2929622177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ea415ac10 0x7f3ea40c4510 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:27.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.114+0000 7f3ea8dfb640 1 --2- 172.21.15.84:0/2929622177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea40cac20 0x7f3ea40b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:27.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.114+0000 7f3ea8dfb640 1 --2- 172.21.15.84:0/2929622177 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f3e8406ad10 0x7f3e8406d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:27.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.114+0000 7f3ea8dfb640 1 --2- 172.21.15.84:0/2929622177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ea40b0360 0x7f3ea4145090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:27.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.114+0000 7f3ea8dfb640 1 -- 172.21.15.84:0/2929622177 >> 172.21.15.84:0/2929622177 conn(0x7f3ea40cb360 msgr2=0x7f3ea40ae390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:27.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.114+0000 7f3ea8dfb640 1 -- 172.21.15.84:0/2929622177 shutdown_connections 2023-10-16T10:09:27.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.114+0000 7f3ea8dfb640 1 -- 172.21.15.84:0/2929622177 wait complete. 2023-10-16T10:09:27.137 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path /snap_test_dir1 2023-10-16T10:09:27.138 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [] 2023-10-16T10:09:27.138 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json /snap_test_dir1 2023-10-16T10:09:27.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.361+0000 7f63c2d47640 1 Processor -- start 2023-10-16T10:09:27.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.362+0000 7f63c2d47640 1 -- start start 2023-10-16T10:09:27.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.363+0000 7f63c2d47640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15ad70 0x7f63bc069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:27.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.363+0000 7f63c2d47640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f63bc0caac0 0x7f63bc06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:27.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.363+0000 7f63c2d47640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63bc0cb030 con 0x7f63bc0c9ca0 2023-10-16T10:09:27.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.363+0000 7f63c2d47640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63bc0c91c0 con 0x7f63bc0caac0 2023-10-16T10:09:27.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.363+0000 7f63c2d47640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0c94a0 con 0x7f63bc15ad70 2023-10-16T10:09:27.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.364+0000 7f63c0ae3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15ad70 0x7f63bc069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:27.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.364+0000 7f63bbfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f63bc0c9ca0 0x7f63bc0ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:52246/0 (socket says 172.21.15.84:52246) 2023-10-16T10:09:27.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.364+0000 7f63bbfff640 1 -- 172.21.15.84:0/2312206916 learned_addr learned my addr 172.21.15.84:0/2312206916 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T10:09:27.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.364+0000 7f63c12e4640 1 --2- 172.21.15.84:0/2312206916 >> v2:172.21.15.157:3300/0 conn(0x7f63bc0caac0 0x7f63bc06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:27.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.364+0000 7f63c0ae3640 1 -- 172.21.15.84:0/2312206916 >> v2:172.21.15.157:3300/0 conn(0x7f63bc0caac0 msgr2=0x7f63bc06a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:09:27.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.364+0000 7f63c0ae3640 1 --2- 172.21.15.84:0/2312206916 >> v2:172.21.15.157:3300/0 conn(0x7f63bc0caac0 0x7f63bc06a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:27.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.364+0000 7f63c0ae3640 1 -- 172.21.15.84:0/2312206916 >> v1:172.21.15.157:6789/0 conn(0x7f63bc0c9ca0 legacy=0x7f63bc0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:27.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.364+0000 7f63c0ae3640 1 -- 172.21.15.84:0/2312206916 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc0c9780 con 0x7f63bc15ad70 2023-10-16T10:09:27.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.366+0000 7f63c0ae3640 1 --2- 172.21.15.84:0/2312206916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15ad70 0x7f63bc069da0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009d90 tx=0x7f63b002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a19bcbd3705e3b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:27.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.367+0000 7f63bb7fe640 1 -- 172.21.15.84:0/2312206916 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b002cc50 con 0x7f63bc15ad70 2023-10-16T10:09:27.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.367+0000 7f63bb7fe640 1 -- 172.21.15.84:0/2312206916 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b002cdb0 con 0x7f63bc15ad70 2023-10-16T10:09:27.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.367+0000 7f63bb7fe640 1 -- 172.21.15.84:0/2312206916 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0034660 con 0x7f63bc15ad70 2023-10-16T10:09:27.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.367+0000 7f63c2d47640 1 -- 172.21.15.84:0/2312206916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15ad70 msgr2=0x7f63bc069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:27.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.367+0000 7f63c2d47640 1 --2- 172.21.15.84:0/2312206916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15ad70 0x7f63bc069da0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009d90 tx=0x7f63b002bea0 comp rx=0 tx=0).stop 2023-10-16T10:09:27.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.367+0000 7f63c2d47640 1 -- 172.21.15.84:0/2312206916 shutdown_connections 2023-10-16T10:09:27.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.367+0000 7f63c2d47640 1 --2- 172.21.15.84:0/2312206916 >> v2:172.21.15.157:3300/0 conn(0x7f63bc0caac0 0x7f63bc06a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:27.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.367+0000 7f63c2d47640 1 --2- 172.21.15.84:0/2312206916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15ad70 0x7f63bc069da0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:27.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.367+0000 7f63c2d47640 1 -- 172.21.15.84:0/2312206916 >> 172.21.15.84:0/2312206916 conn(0x7f63bc0ce9b0 msgr2=0x7f63bc0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:27.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.368+0000 7f63c2d47640 1 -- 172.21.15.84:0/2312206916 shutdown_connections 2023-10-16T10:09:27.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.368+0000 7f63c2d47640 1 -- 172.21.15.84:0/2312206916 wait complete. 2023-10-16T10:09:27.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.369+0000 7f63c2d47640 1 Processor -- start 2023-10-16T10:09:27.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.369+0000 7f63c2d47640 1 -- start start 2023-10-16T10:09:27.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.370+0000 7f63c2d47640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63bc0c9ca0 0x7f63bc1efd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:27.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.370+0000 7f63c2d47640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63bc0caac0 0x7f63bc0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:27.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.370+0000 7f63c2d47640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15ad70 0x7f63bc0acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:27.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.370+0000 7f63c2d47640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0cd4e0 con 0x7f63bc0caac0 2023-10-16T10:09:27.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.370+0000 7f63c2d47640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0cd7c0 con 0x7f63bc0c9ca0 2023-10-16T10:09:27.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.370+0000 7f63c2d47640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0cdaa0 con 0x7f63bc15ad70 2023-10-16T10:09:27.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.371+0000 7f63bbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63bc0caac0 0x7f63bc0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:27.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.371+0000 7f63bbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63bc0caac0 0x7f63bc0ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34396/0 (socket says 172.21.15.84:34396) 2023-10-16T10:09:27.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.371+0000 7f63bbfff640 1 -- 172.21.15.84:0/407401083 learned_addr learned my addr 172.21.15.84:0/407401083 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T10:09:27.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.371+0000 7f63c0ae3640 1 --2- 172.21.15.84:0/407401083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63bc0c9ca0 0x7f63bc1efd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:27.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.371+0000 7f63c12e4640 1 --2- 172.21.15.84:0/407401083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15ad70 0x7f63bc0acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:27.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.371+0000 7f63bbfff640 1 -- 172.21.15.84:0/407401083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15ad70 msgr2=0x7f63bc0acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:27.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.371+0000 7f63bbfff640 1 --2- 172.21.15.84:0/407401083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15ad70 0x7f63bc0acdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:27.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.371+0000 7f63bbfff640 1 -- 172.21.15.84:0/407401083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63bc0c9ca0 msgr2=0x7f63bc1efd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:27.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.371+0000 7f63bbfff640 1 --2- 172.21.15.84:0/407401083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63bc0c9ca0 0x7f63bc1efd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:27.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.371+0000 7f63bbfff640 1 -- 172.21.15.84:0/407401083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b0009a70 con 0x7f63bc0caac0 2023-10-16T10:09:27.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.371+0000 7f63c0ae3640 1 --2- 172.21.15.84:0/407401083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63bc0c9ca0 0x7f63bc1efd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:27.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.371+0000 7f63c12e4640 1 --2- 172.21.15.84:0/407401083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15ad70 0x7f63bc0acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:09:27.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.371+0000 7f63bbfff640 1 --2- 172.21.15.84:0/407401083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63bc0caac0 0x7f63bc0ac890 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f63a800c4a0 tx=0x7f63a800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:27.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.372+0000 7f63b97fa640 1 -- 172.21.15.84:0/407401083 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8005ce0 con 0x7f63bc0caac0 2023-10-16T10:09:27.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.372+0000 7f63b97fa640 1 -- 172.21.15.84:0/407401083 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a8005e40 con 0x7f63bc0caac0 2023-10-16T10:09:27.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.372+0000 7f63c2d47640 1 -- 172.21.15.84:0/407401083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63bc0cdd80 con 0x7f63bc0caac0 2023-10-16T10:09:27.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.372+0000 7f63b97fa640 1 -- 172.21.15.84:0/407401083 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8007270 con 0x7f63bc0caac0 2023-10-16T10:09:27.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.372+0000 7f63c2d47640 1 -- 172.21.15.84:0/407401083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63bc0cc930 con 0x7f63bc0caac0 2023-10-16T10:09:27.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.373+0000 7f63c2d47640 1 -- 172.21.15.84:0/407401083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63bc0cdfc0 con 0x7f63bc0caac0 2023-10-16T10:09:27.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.374+0000 7f63b97fa640 1 -- 172.21.15.84:0/407401083 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 85) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f63a8007490 con 0x7f63bc0caac0 2023-10-16T10:09:27.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.375+0000 7f63b97fa640 1 --2- 172.21.15.84:0/407401083 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f63a006b030 0x7f63a006d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:09:27.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.375+0000 7f63b97fa640 1 -- 172.21.15.84:0/407401083 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f63a8088e00 con 0x7f63bc0caac0 2023-10-16T10:09:27.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.375+0000 7f63c0ae3640 1 --2- 172.21.15.84:0/407401083 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f63a006b030 0x7f63a006d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:09:27.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.376+0000 7f63c0ae3640 1 --2- 172.21.15.84:0/407401083 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f63a006b030 0x7f63a006d4f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f63b0036000 tx=0x7f63b0030000 comp rx=0 tx=0).ready entity=mgr.9327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:09:27.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.379+0000 7f63b97fa640 1 -- 172.21.15.84:0/407401083 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f63bc0cdfc0 con 0x7f63bc0caac0 2023-10-16T10:09:27.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.592+0000 7f63c2d47640 1 -- 172.21.15.84:0/407401083 --> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63bc0ad4d0 con 0x7f63a006b030 2023-10-16T10:09:27.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.608+0000 7f63b97fa640 1 -- 172.21.15.84:0/407401083 <== mgr.9327 v2:172.21.15.84:6832/3746729142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+74 (secure 0 0 0) 0x7f63b00440c0 con 0x7f63a006b030 2023-10-16T10:09:27.610 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T10:09:27.611 INFO:teuthology.orchestra.run.smithi084.stdout:{"path": "/snap_test_dir1", "schedule": ["1m"], "retention": [{"n": 55}]} 2023-10-16T10:09:27.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.611+0000 7f63c2d47640 1 -- 172.21.15.84:0/407401083 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f63a006b030 msgr2=0x7f63a006d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:27.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.611+0000 7f63c2d47640 1 --2- 172.21.15.84:0/407401083 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f63a006b030 0x7f63a006d4f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f63b0036000 tx=0x7f63b0030000 comp rx=0 tx=0).stop 2023-10-16T10:09:27.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.611+0000 7f63c2d47640 1 -- 172.21.15.84:0/407401083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63bc0caac0 msgr2=0x7f63bc0ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:09:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.611+0000 7f63c2d47640 1 --2- 172.21.15.84:0/407401083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63bc0caac0 0x7f63bc0ac890 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f63a800c4a0 tx=0x7f63a800c960 comp rx=0 tx=0).stop 2023-10-16T10:09:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.611+0000 7f63c2d47640 1 -- 172.21.15.84:0/407401083 shutdown_connections 2023-10-16T10:09:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.611+0000 7f63c2d47640 1 --2- 172.21.15.84:0/407401083 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f63a006b030 0x7f63a006d4f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.611+0000 7f63c2d47640 1 --2- 172.21.15.84:0/407401083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f63bc15ad70 0x7f63bc0acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:27.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.611+0000 7f63c2d47640 1 --2- 172.21.15.84:0/407401083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63bc0caac0 0x7f63bc0ac890 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:27.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.611+0000 7f63c2d47640 1 --2- 172.21.15.84:0/407401083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63bc0c9ca0 0x7f63bc1efd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:09:27.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.611+0000 7f63c2d47640 1 -- 172.21.15.84:0/407401083 >> 172.21.15.84:0/407401083 conn(0x7f63bc0ce9b0 msgr2=0x7f63bc0b3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:09:27.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.611+0000 7f63c2d47640 1 -- 172.21.15.84:0/407401083 shutdown_connections 2023-10-16T10:09:27.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T10:09:27.611+0000 7f63c2d47640 1 -- 172.21.15.84:0/407401083 wait complete. 2023-10-16T10:09:27.631 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "/snap_test_dir1", "schedule": ["1m"], "retention": [{"n": 55}]} 2023-10-16T10:09:27.631 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': '/snap_test_dir1', 'schedule': ['1m'], 'retention': [{'n': 55}]} 2023-10-16T10:09:35.126 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:09:35.136 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:10:00.136 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:10:00.134+0000 7fbdbb93b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:10:00.137 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:10:00.134+0000 7fbdbb93b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:10:00.137 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:10:00.134+0000 7fbdbb93b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:10:00.137 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:10:00.134+0000 7fbdbb93b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:10:00.137 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T10:10:00.134+0000 7fbdbb93b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:10:05.283 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:10:05.293 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:10:35.428 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:10:35.437 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:11:05.576 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:11:05.586 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:11:35.895 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:11:35.905 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:12:06.039 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:12:06.049 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:12:36.427 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:12:36.437 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:13:06.605 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:13:06.614 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:13:36.767 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:13:36.777 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:14:07.002 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:14:07.013 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:14:37.170 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:14:37.181 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:15:07.316 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:15:07.326 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:15:37.454 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:15:37.464 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:16:07.606 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:16:07.616 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:16:37.750 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:16:37.760 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:17:07.893 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:17:07.903 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:17:38.293 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:17:38.303 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:18:08.460 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:18:08.471 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:18:38.615 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:18:38.686 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:19:08.867 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:19:08.877 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:19:39.270 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:19:39.283 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:20:09.421 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:20:09.430 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:20:39.568 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:20:39.578 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:21:09.722 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:21:09.732 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:21:39.887 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:21:39.897 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:22:10.057 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:22:10.068 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:22:40.221 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:22:40.231 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:23:10.376 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:23:10.386 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:23:40.521 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:23:40.531 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:24:10.694 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:24:10.706 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:24:40.866 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:24:40.876 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:25:11.020 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:25:11.030 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:25:41.326 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:25:41.335 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:26:11.472 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:26:11.574 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:26:41.744 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:26:41.754 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:27:12.117 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:27:12.127 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:27:42.277 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:27:42.287 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:28:12.424 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:28:12.434 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:28:42.569 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:28:42.579 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:29:12.717 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:29:12.727 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:29:43.010 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:29:43.020 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:30:13.169 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:30:13.179 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:30:43.331 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:30:43.341 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:31:13.470 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:31:13.480 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:31:43.849 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:31:43.860 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:32:14.052 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:32:14.062 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:32:44.218 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:32:44.228 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:33:14.366 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:33:14.376 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:33:44.713 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:33:44.722 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:34:14.858 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:34:14.868 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:34:45.028 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:34:45.037 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:35:15.314 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:35:15.324 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:35:45.464 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:35:45.474 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:36:15.696 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:36:15.705 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:36:45.839 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:36:45.850 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:37:16.086 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:37:16.096 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:37:46.388 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:37:46.398 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:38:16.537 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:38:16.548 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:38:46.695 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:38:46.705 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:39:16.851 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:39:16.861 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:39:46.994 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:39:47.005 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:40:17.141 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:40:17.151 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:40:47.305 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:40:47.319 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:41:17.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:41:17.714 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:41:47.856 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:41:47.866 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:42:18.011 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:42:18.021 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:42:48.158 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:42:48.167 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:43:18.304 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:43:18.314 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:43:48.468 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:43:48.478 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:44:18.639 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:44:18.649 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:44:48.818 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:44:48.828 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:45:19.154 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:45:19.164 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:45:49.324 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:45:49.334 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:46:19.494 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:46:19.503 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:46:49.941 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:46:49.951 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:47:20.088 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:47:20.098 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:47:50.243 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:47:50.252 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:48:20.414 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:48:20.424 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:48:50.577 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:48:50.587 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:49:20.732 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:49:20.742 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:49:50.869 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:49:50.880 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:50:21.213 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:50:21.223 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:50:51.356 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:50:51.366 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:51:21.501 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:51:21.511 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:51:51.656 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:51:51.666 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:52:21.802 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:52:21.815 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:52:51.975 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:52:51.985 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:53:22.121 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:53:22.131 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:53:52.276 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:53:52.286 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:54:22.646 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:54:22.656 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:54:52.794 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:54:52.804 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:55:22.957 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:55:22.966 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:55:53.129 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:55:53.139 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:56:23.526 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:56:23.536 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:56:53.672 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:56:53.681 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:57:23.998 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:57:24.008 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:57:54.185 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:57:54.195 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:58:24.347 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:58:24.356 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:58:54.500 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:58:54.509 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:59:24.663 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:59:24.673 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:59:54.816 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:59:54.826 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:00:24.955 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:00:24.966 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:00:55.445 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:00:55.454 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:01:25.582 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:01:25.592 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:01:55.737 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:01:55.748 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:02:25.985 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:02:25.995 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:02:56.139 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:02:56.149 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:03:26.286 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:03:26.295 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:03:56.425 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:03:56.435 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:04:26.729 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:04:26.738 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:04:56.874 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:04:56.884 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:05:15.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:15.633 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:05:15.703 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_11_00_UTC 2023-10-16T11:05:15.703 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_12_00_UTC 2023-10-16T11:05:15.703 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_13_00_UTC 2023-10-16T11:05:15.703 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_14_00_UTC 2023-10-16T11:05:15.704 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_15_00_UTC 2023-10-16T11:05:15.704 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_16_00_UTC 2023-10-16T11:05:15.704 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_17_00_UTC 2023-10-16T11:05:15.704 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_18_00_UTC 2023-10-16T11:05:15.704 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_19_00_UTC 2023-10-16T11:05:15.705 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_20_00_UTC 2023-10-16T11:05:15.705 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_21_00_UTC 2023-10-16T11:05:15.705 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_22_00_UTC 2023-10-16T11:05:15.705 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_23_00_UTC 2023-10-16T11:05:15.705 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_24_00_UTC 2023-10-16T11:05:15.706 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_25_00_UTC 2023-10-16T11:05:15.706 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_26_00_UTC 2023-10-16T11:05:15.706 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_27_00_UTC 2023-10-16T11:05:15.706 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_28_00_UTC 2023-10-16T11:05:15.707 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_29_00_UTC 2023-10-16T11:05:15.707 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_30_00_UTC 2023-10-16T11:05:15.707 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_31_00_UTC 2023-10-16T11:05:15.707 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_32_00_UTC 2023-10-16T11:05:15.707 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_33_00_UTC 2023-10-16T11:05:15.708 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_34_00_UTC 2023-10-16T11:05:15.708 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_35_00_UTC 2023-10-16T11:05:15.708 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_36_00_UTC 2023-10-16T11:05:15.708 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_37_00_UTC 2023-10-16T11:05:15.708 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_38_00_UTC 2023-10-16T11:05:15.708 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_39_00_UTC 2023-10-16T11:05:15.709 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_40_00_UTC 2023-10-16T11:05:15.709 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_41_00_UTC 2023-10-16T11:05:15.709 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_42_00_UTC 2023-10-16T11:05:15.709 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_43_00_UTC 2023-10-16T11:05:15.709 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_44_00_UTC 2023-10-16T11:05:15.710 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_45_00_UTC 2023-10-16T11:05:15.710 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_46_00_UTC 2023-10-16T11:05:15.710 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_47_00_UTC 2023-10-16T11:05:15.710 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_48_00_UTC 2023-10-16T11:05:15.710 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_49_01_UTC 2023-10-16T11:05:15.711 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_50_00_UTC 2023-10-16T11:05:15.711 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_51_00_UTC 2023-10-16T11:05:15.711 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_52_00_UTC 2023-10-16T11:05:15.711 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_53_00_UTC 2023-10-16T11:05:15.711 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_54_00_UTC 2023-10-16T11:05:15.712 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_55_00_UTC 2023-10-16T11:05:15.712 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_56_00_UTC 2023-10-16T11:05:15.712 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_57_00_UTC 2023-10-16T11:05:15.712 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_58_00_UTC 2023-10-16T11:05:15.712 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_59_00_UTC 2023-10-16T11:05:15.713 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_00_00_UTC 2023-10-16T11:05:15.713 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_01_00_UTC 2023-10-16T11:05:15.713 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_02_00_UTC 2023-10-16T11:05:15.713 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_03_00_UTC 2023-10-16T11:05:15.713 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_04_00_UTC 2023-10-16T11:05:15.714 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_05_00_UTC 2023-10-16T11:05:15.715 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-10_11_00_UTC', 'scheduled-2023-10-16-10_12_00_UTC', 'scheduled-2023-10-16-10_13_00_UTC', 'scheduled-2023-10-16-10_14_00_UTC', 'scheduled-2023-10-16-10_15_00_UTC', 'scheduled-2023-10-16-10_16_00_UTC', 'scheduled-2023-10-16-10_17_00_UTC', 'scheduled-2023-10-16-10_18_00_UTC', 'scheduled-2023-10-16-10_19_00_UTC', 'scheduled-2023-10-16-10_20_00_UTC', 'scheduled-2023-10-16-10_21_00_UTC', 'scheduled-2023-10-16-10_22_00_UTC', 'scheduled-2023-10-16-10_23_00_UTC', 'scheduled-2023-10-16-10_24_00_UTC', 'scheduled-2023-10-16-10_25_00_UTC', 'scheduled-2023-10-16-10_26_00_UTC', 'scheduled-2023-10-16-10_27_00_UTC', 'scheduled-2023-10-16-10_28_00_UTC', 'scheduled-2023-10-16-10_29_00_UTC', 'scheduled-2023-10-16-10_30_00_UTC', 'scheduled-2023-10-16-10_31_00_UTC', 'scheduled-2023-10-16-10_32_00_UTC', 'scheduled-2023-10-16-10_33_00_UTC', 'scheduled-2023-10-16-10_34_00_UTC', 'scheduled-2023-10-16-10_35_00_UTC', 'scheduled-2023-10-16-10_36_00_UTC', 'scheduled-2023-10-16-10_37_00_UTC', 'scheduled-2023-10-16-10_38_00_UTC', 'scheduled-2023-10-16-10_39_00_UTC', 'scheduled-2023-10-16-10_40_00_UTC', 'scheduled-2023-10-16-10_41_00_UTC', 'scheduled-2023-10-16-10_42_00_UTC', 'scheduled-2023-10-16-10_43_00_UTC', 'scheduled-2023-10-16-10_44_00_UTC', 'scheduled-2023-10-16-10_45_00_UTC', 'scheduled-2023-10-16-10_46_00_UTC', 'scheduled-2023-10-16-10_47_00_UTC', 'scheduled-2023-10-16-10_48_00_UTC', 'scheduled-2023-10-16-10_49_01_UTC', 'scheduled-2023-10-16-10_50_00_UTC', 'scheduled-2023-10-16-10_51_00_UTC', 'scheduled-2023-10-16-10_52_00_UTC', 'scheduled-2023-10-16-10_53_00_UTC', 'scheduled-2023-10-16-10_54_00_UTC', 'scheduled-2023-10-16-10_55_00_UTC', 'scheduled-2023-10-16-10_56_00_UTC', 'scheduled-2023-10-16-10_57_00_UTC', 'scheduled-2023-10-16-10_58_00_UTC', 'scheduled-2023-10-16-10_59_00_UTC', 'scheduled-2023-10-16-11_00_00_UTC', 'scheduled-2023-10-16-11_01_00_UTC', 'scheduled-2023-10-16-11_02_00_UTC', 'scheduled-2023-10-16-11_03_00_UTC', 'scheduled-2023-10-16-11_04_00_UTC', 'scheduled-2023-10-16-11_05_00_UTC'] 2023-10-16T11:05:15.715 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1 2023-10-16T11:05:15.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.925+0000 7fd2036a9640 1 Processor -- start 2023-10-16T11:05:15.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.925+0000 7fd2036a9640 1 -- start start 2023-10-16T11:05:15.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.926+0000 7fd2036a9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd1fc06a4b0 0x7fd1fc0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:15.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.926+0000 7fd2036a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd1fc0c9ca0 0x7fd1fc0ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.926+0000 7fd2036a9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1fc0cb030 con 0x7fd1fc15ad70 2023-10-16T11:05:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.926+0000 7fd2036a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1fc0cae30 con 0x7fd1fc0c9ca0 2023-10-16T11:05:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.926+0000 7fd2036a9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1fc0c91c0 con 0x7fd1fc06a4b0 2023-10-16T11:05:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.927+0000 7fd201c46640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd1fc0c9ca0 0x7fd1fc0ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.927+0000 7fd201445640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd1fc15ad70 0x7fd1fc069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37114/0 (socket says 172.21.15.84:37114) 2023-10-16T11:05:15.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.927+0000 7fd201445640 1 -- 172.21.15.84:0/1905523164 learned_addr learned my addr 172.21.15.84:0/1905523164 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:15.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.927+0000 7fd200c44640 1 --2- 172.21.15.84:0/1905523164 >> v2:172.21.15.157:3300/0 conn(0x7fd1fc06a4b0 0x7fd1fc0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.927+0000 7fd201c46640 1 -- 172.21.15.84:0/1905523164 >> v2:172.21.15.157:3300/0 conn(0x7fd1fc06a4b0 msgr2=0x7fd1fc0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.927+0000 7fd201c46640 1 --2- 172.21.15.84:0/1905523164 >> v2:172.21.15.157:3300/0 conn(0x7fd1fc06a4b0 0x7fd1fc0ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.927+0000 7fd201c46640 1 -- 172.21.15.84:0/1905523164 >> v1:172.21.15.84:6789/0 conn(0x7fd1fc15ad70 legacy=0x7fd1fc069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.927+0000 7fd201c46640 1 -- 172.21.15.84:0/1905523164 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1fc0c94a0 con 0x7fd1fc0c9ca0 2023-10-16T11:05:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.927+0000 7fd1f3fff640 1 -- 172.21.15.84:0/1905523164 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514336504 0 0) 0x7fd1fc0cb030 con 0x7fd1fc15ad70 2023-10-16T11:05:15.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.928+0000 7fd201c46640 1 --2- 172.21.15.84:0/1905523164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1fc0c9ca0 0x7fd1fc0ca8f0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0ce870 tx=0x7fd1f402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5850cf3ae168793e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:15.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.929+0000 7fd1f3fff640 1 -- 172.21.15.84:0/1905523164 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f402cc80 con 0x7fd1fc0c9ca0 2023-10-16T11:05:15.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.929+0000 7fd1f3fff640 1 -- 172.21.15.84:0/1905523164 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f402cde0 con 0x7fd1fc0c9ca0 2023-10-16T11:05:15.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.929+0000 7fd1f3fff640 1 -- 172.21.15.84:0/1905523164 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4034660 con 0x7fd1fc0c9ca0 2023-10-16T11:05:15.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.929+0000 7fd2036a9640 1 -- 172.21.15.84:0/1905523164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1fc0c9ca0 msgr2=0x7fd1fc0ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:15.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.929+0000 7fd2036a9640 1 --2- 172.21.15.84:0/1905523164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1fc0c9ca0 0x7fd1fc0ca8f0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0ce870 tx=0x7fd1f402bed0 comp rx=0 tx=0).stop 2023-10-16T11:05:15.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.929+0000 7fd2036a9640 1 -- 172.21.15.84:0/1905523164 shutdown_connections 2023-10-16T11:05:15.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.929+0000 7fd2036a9640 1 --2- 172.21.15.84:0/1905523164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1fc0c9ca0 0x7fd1fc0ca8f0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:15.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.929+0000 7fd2036a9640 1 --2- 172.21.15.84:0/1905523164 >> v2:172.21.15.157:3300/0 conn(0x7fd1fc06a4b0 0x7fd1fc0ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:15.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.929+0000 7fd2036a9640 1 -- 172.21.15.84:0/1905523164 >> 172.21.15.84:0/1905523164 conn(0x7fd1fc0ce9b0 msgr2=0x7fd1fc0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:15.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.930+0000 7fd2036a9640 1 -- 172.21.15.84:0/1905523164 shutdown_connections 2023-10-16T11:05:15.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.930+0000 7fd2036a9640 1 -- 172.21.15.84:0/1905523164 wait complete. 2023-10-16T11:05:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.931+0000 7fd2036a9640 1 Processor -- start 2023-10-16T11:05:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.932+0000 7fd2036a9640 1 -- start start 2023-10-16T11:05:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.932+0000 7fd2036a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1fc06a4b0 0x7fd1fc0c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.932+0000 7fd2036a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1fc0c9ca0 0x7fd1fc1efbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.932+0000 7fd2036a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1fc15ad70 0x7fd1fc0c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.933+0000 7fd2036a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc0c8300 con 0x7fd1fc0c9ca0 2023-10-16T11:05:15.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.933+0000 7fd2036a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc0c85d0 con 0x7fd1fc15ad70 2023-10-16T11:05:15.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.933+0000 7fd2036a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc0c88b0 con 0x7fd1fc06a4b0 2023-10-16T11:05:15.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.933+0000 7fd200c44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1fc0c9ca0 0x7fd1fc1efbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:15.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.933+0000 7fd200c44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1fc0c9ca0 0x7fd1fc1efbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36322/0 (socket says 172.21.15.84:36322) 2023-10-16T11:05:15.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.933+0000 7fd200c44640 1 -- 172.21.15.84:0/1612088508 learned_addr learned my addr 172.21.15.84:0/1612088508 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.933+0000 7fd201445640 1 --2- 172.21.15.84:0/1612088508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1fc06a4b0 0x7fd1fc0c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.933+0000 7fd201c46640 1 --2- 172.21.15.84:0/1612088508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1fc15ad70 0x7fd1fc0c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.933+0000 7fd200c44640 1 -- 172.21.15.84:0/1612088508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1fc06a4b0 msgr2=0x7fd1fc0c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.933+0000 7fd200c44640 1 --2- 172.21.15.84:0/1612088508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1fc06a4b0 0x7fd1fc0c7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.933+0000 7fd200c44640 1 -- 172.21.15.84:0/1612088508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1fc15ad70 msgr2=0x7fd1fc0c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.933+0000 7fd200c44640 1 --2- 172.21.15.84:0/1612088508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1fc15ad70 0x7fd1fc0c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:15.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.933+0000 7fd200c44640 1 -- 172.21.15.84:0/1612088508 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f4009a70 con 0x7fd1fc0c9ca0 2023-10-16T11:05:15.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.934+0000 7fd200c44640 1 --2- 172.21.15.84:0/1612088508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1fc0c9ca0 0x7fd1fc1efbf0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00d800 tx=0x7fd1ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:15.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.934+0000 7fd1f1ffb640 1 -- 172.21.15.84:0/1612088508 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec00b6c0 con 0x7fd1fc0c9ca0 2023-10-16T11:05:15.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.934+0000 7fd1f1ffb640 1 -- 172.21.15.84:0/1612088508 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec00b820 con 0x7fd1fc0c9ca0 2023-10-16T11:05:15.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.934+0000 7fd1f1ffb640 1 -- 172.21.15.84:0/1612088508 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec00bbb0 con 0x7fd1fc0c9ca0 2023-10-16T11:05:15.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.934+0000 7fd2036a9640 1 -- 172.21.15.84:0/1612088508 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1fc0c8b90 con 0x7fd1fc0c9ca0 2023-10-16T11:05:15.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.934+0000 7fd2036a9640 1 -- 172.21.15.84:0/1612088508 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1fc1f0130 con 0x7fd1fc0c9ca0 2023-10-16T11:05:15.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.935+0000 7fd2036a9640 1 -- 172.21.15.84:0/1612088508 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1fc04e160 con 0x7fd1fc0c9ca0 2023-10-16T11:05:15.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.936+0000 7fd1f1ffb640 1 -- 172.21.15.84:0/1612088508 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 88) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fd1ec00bd10 con 0x7fd1fc0c9ca0 2023-10-16T11:05:15.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.937+0000 7fd1f1ffb640 1 --2- 172.21.15.84:0/1612088508 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fd1e006b010 0x7fd1e006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:15.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.937+0000 7fd1f1ffb640 1 -- 172.21.15.84:0/1612088508 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fd1ec089f20 con 0x7fd1fc0c9ca0 2023-10-16T11:05:15.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.937+0000 7fd201445640 1 --2- 172.21.15.84:0/1612088508 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fd1e006b010 0x7fd1e006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:15.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.938+0000 7fd201445640 1 --2- 172.21.15.84:0/1612088508 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fd1e006b010 0x7fd1e006d4d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8000f60 tx=0x7fd1f8009040 comp rx=0 tx=0).ready entity=mgr.9327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:15.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:15.942+0000 7fd1f1ffb640 1 -- 172.21.15.84:0/1612088508 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd1ec054cd0 con 0x7fd1fc0c9ca0 2023-10-16T11:05:16.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.154+0000 7fd2036a9640 1 -- 172.21.15.84:0/1612088508 --> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd1fc0c9780 con 0x7fd1e006b010 2023-10-16T11:05:16.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.172+0000 7fd1f1ffb640 1 -- 172.21.15.84:0/1612088508 <== mgr.9327 v2:172.21.15.84:6832/3746729142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+373 (secure 0 0 0) 0x7fd1f8001270 con 0x7fd1e006b010 2023-10-16T11:05:16.174 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:16.174 INFO:teuthology.orchestra.run.smithi084.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1m", "retention": {"n": 55}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T10:09:26", "first": "2023-10-16T10:10:00", "last": "2023-10-16T11:05:00", "last_pruned": "2023-10-16T11:05:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-10-16T11:05:16.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.174+0000 7fd2036a9640 1 -- 172.21.15.84:0/1612088508 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fd1e006b010 msgr2=0x7fd1e006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:16.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.174+0000 7fd2036a9640 1 --2- 172.21.15.84:0/1612088508 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fd1e006b010 0x7fd1e006d4d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8000f60 tx=0x7fd1f8009040 comp rx=0 tx=0).stop 2023-10-16T11:05:16.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.174+0000 7fd2036a9640 1 -- 172.21.15.84:0/1612088508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1fc0c9ca0 msgr2=0x7fd1fc1efbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:16.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.174+0000 7fd2036a9640 1 --2- 172.21.15.84:0/1612088508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1fc0c9ca0 0x7fd1fc1efbf0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00d800 tx=0x7fd1ec00dcc0 comp rx=0 tx=0).stop 2023-10-16T11:05:16.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.175+0000 7fd2036a9640 1 -- 172.21.15.84:0/1612088508 shutdown_connections 2023-10-16T11:05:16.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.175+0000 7fd2036a9640 1 --2- 172.21.15.84:0/1612088508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1fc15ad70 0x7fd1fc0c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:16.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.175+0000 7fd2036a9640 1 --2- 172.21.15.84:0/1612088508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1fc0c9ca0 0x7fd1fc1efbf0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:16.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.175+0000 7fd2036a9640 1 --2- 172.21.15.84:0/1612088508 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fd1e006b010 0x7fd1e006d4d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:16.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.175+0000 7fd2036a9640 1 --2- 172.21.15.84:0/1612088508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1fc06a4b0 0x7fd1fc0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:16.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.175+0000 7fd2036a9640 1 -- 172.21.15.84:0/1612088508 >> 172.21.15.84:0/1612088508 conn(0x7fd1fc0ce9b0 msgr2=0x7fd1fc15d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:16.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.175+0000 7fd2036a9640 1 -- 172.21.15.84:0/1612088508 shutdown_connections 2023-10-16T11:05:16.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.175+0000 7fd2036a9640 1 -- 172.21.15.84:0/1612088508 wait complete. 2023-10-16T11:05:16.194 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1m", "retention": {"n": 55}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T10:09:26", "first": "2023-10-16T10:10:00", "last": "2023-10-16T11:05:00", "last_pruned": "2023-10-16T11:05:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-10-16T11:05:16.194 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1', 'rel_path': '/snap_test_dir1', 'schedule': '1m', 'retention': {'n': 55}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T10:09:26', 'first': '2023-10-16T10:10:00', 'last': '2023-10-16T11:05:00', 'last_pruned': '2023-10-16T11:05:00', 'created_count': 56, 'pruned_count': 1, 'active': True} 2023-10-16T11:05:16.194 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 55 2023-10-16T11:05:16.195 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 56 2023-10-16T11:05:16.195 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1 2023-10-16T11:05:16.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.401+0000 7f7d4d62b640 1 Processor -- start 2023-10-16T11:05:16.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.401+0000 7f7d4d62b640 1 -- start start 2023-10-16T11:05:16.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.402+0000 7f7d4d62b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d480661c0 0x7f7d480ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:16.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.402+0000 7f7d4d62b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d4806cee0 con 0x7f7d4815d330 2023-10-16T11:05:16.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.402+0000 7f7d4d62b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d480d0890 con 0x7f7d481618b0 2023-10-16T11:05:16.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.402+0000 7f7d4d62b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d4806b1e0 con 0x7f7d480661c0 2023-10-16T11:05:16.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.403+0000 7f7d46ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7d4815d330 0x7f7d4806c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37126/0 (socket says 172.21.15.84:37126) 2023-10-16T11:05:16.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.403+0000 7f7d46ffd640 1 -- 172.21.15.84:0/2743884947 learned_addr learned my addr 172.21.15.84:0/2743884947 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:16.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.403+0000 7f7d45ffb640 1 -- 172.21.15.84:0/2743884947 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1921079680 0 0) 0x7f7d4806cee0 con 0x7f7d4815d330 2023-10-16T11:05:16.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.403+0000 7f7d45ffb640 1 -- 172.21.15.84:0/2743884947 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d34003520 con 0x7f7d4815d330 2023-10-16T11:05:16.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.403+0000 7f7d477fe640 1 --2- 172.21.15.84:0/2743884947 >> v2:172.21.15.157:3300/0 conn(0x7f7d480661c0 0x7f7d480ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:16.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.403+0000 7f7d45ffb640 1 -- 172.21.15.84:0/2743884947 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1209554633 0 0) 0x7f7d34003520 con 0x7f7d4815d330 2023-10-16T11:05:16.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.403+0000 7f7d45ffb640 1 -- 172.21.15.84:0/2743884947 >> v2:172.21.15.157:3300/0 conn(0x7f7d480661c0 msgr2=0x7f7d480ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:16.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.403+0000 7f7d45ffb640 1 --2- 172.21.15.84:0/2743884947 >> v2:172.21.15.157:3300/0 conn(0x7f7d480661c0 0x7f7d480ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:16.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.403+0000 7f7d45ffb640 1 -- 172.21.15.84:0/2743884947 >> v1:172.21.15.157:6789/0 conn(0x7f7d481618b0 legacy=0x7f7d48065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:16.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.403+0000 7f7d45ffb640 1 -- 172.21.15.84:0/2743884947 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d480d05b0 con 0x7f7d4815d330 2023-10-16T11:05:16.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.403+0000 7f7d45ffb640 1 -- 172.21.15.84:0/2743884947 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f7d380019f0 con 0x7f7d4815d330 2023-10-16T11:05:16.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.404+0000 7f7d45ffb640 1 -- 172.21.15.84:0/2743884947 >> v1:172.21.15.84:6789/0 conn(0x7f7d4815d330 legacy=0x7f7d4806c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:16.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.404+0000 7f7d45ffb640 1 --2- 172.21.15.84:0/2743884947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d34003970 0x7f7d34003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:16.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.404+0000 7f7d45ffb640 1 --2- 172.21.15.84:0/2743884947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d340047c0 0x7f7d34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:16.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.404+0000 7f7d45ffb640 1 --2- 172.21.15.84:0/2743884947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d340052a0 0x7f7d3400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:16.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.404+0000 7f7d45ffb640 1 -- 172.21.15.84:0/2743884947 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4806b1e0 con 0x7f7d340047c0 2023-10-16T11:05:16.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.404+0000 7f7d45ffb640 1 -- 172.21.15.84:0/2743884947 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3400c170 con 0x7f7d34003970 2023-10-16T11:05:16.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.404+0000 7f7d45ffb640 1 -- 172.21.15.84:0/2743884947 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3400c330 con 0x7f7d340052a0 2023-10-16T11:05:16.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.404+0000 7f7d467fc640 1 --2- 172.21.15.84:0/2743884947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d340047c0 0x7f7d34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:16.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.404+0000 7f7d467fc640 1 -- 172.21.15.84:0/2743884947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d340052a0 msgr2=0x7f7d3400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:16.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.404+0000 7f7d467fc640 1 --2- 172.21.15.84:0/2743884947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d340052a0 0x7f7d3400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:16.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.404+0000 7f7d467fc640 1 -- 172.21.15.84:0/2743884947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d34003970 msgr2=0x7f7d34003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:16.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.404+0000 7f7d467fc640 1 --2- 172.21.15.84:0/2743884947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d34003970 0x7f7d34003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:16.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.404+0000 7f7d467fc640 1 -- 172.21.15.84:0/2743884947 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d34003520 con 0x7f7d340047c0 2023-10-16T11:05:16.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.404+0000 7f7d477fe640 1 -- 172.21.15.84:0/2743884947 reap_dead start 2023-10-16T11:05:16.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.405+0000 7f7d467fc640 1 --2- 172.21.15.84:0/2743884947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d340047c0 0x7f7d34004c40 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f7d30001720 tx=0x7f7d340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2842d23754e3957e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:16.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.405+0000 7f7d45ffb640 1 -- 172.21.15.84:0/2743884947 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d300052d0 con 0x7f7d340047c0 2023-10-16T11:05:16.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.406+0000 7f7d45ffb640 1 -- 172.21.15.84:0/2743884947 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3002e050 con 0x7f7d340047c0 2023-10-16T11:05:16.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.406+0000 7f7d45ffb640 1 -- 172.21.15.84:0/2743884947 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d30003830 con 0x7f7d340047c0 2023-10-16T11:05:16.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.406+0000 7f7d4d62b640 1 -- 172.21.15.84:0/2743884947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d340047c0 msgr2=0x7f7d34004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:16.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.406+0000 7f7d4d62b640 1 --2- 172.21.15.84:0/2743884947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d340047c0 0x7f7d34004c40 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f7d30001720 tx=0x7f7d340057f0 comp rx=0 tx=0).stop 2023-10-16T11:05:16.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.406+0000 7f7d4d62b640 1 -- 172.21.15.84:0/2743884947 shutdown_connections 2023-10-16T11:05:16.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.406+0000 7f7d4d62b640 1 --2- 172.21.15.84:0/2743884947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d340047c0 0x7f7d34004c40 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:16.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.406+0000 7f7d4d62b640 1 -- 172.21.15.84:0/2743884947 >> 172.21.15.84:0/2743884947 conn(0x7f7d480d0ab0 msgr2=0x7f7d480d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:16.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.406+0000 7f7d4d62b640 1 -- 172.21.15.84:0/2743884947 shutdown_connections 2023-10-16T11:05:16.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.406+0000 7f7d4d62b640 1 -- 172.21.15.84:0/2743884947 wait complete. 2023-10-16T11:05:16.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.408+0000 7f7d4d62b640 1 Processor -- start 2023-10-16T11:05:16.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.408+0000 7f7d4d62b640 1 -- start start 2023-10-16T11:05:16.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.409+0000 7f7d4d62b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d340047c0 0x7f7d4814d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:16.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.409+0000 7f7d4d62b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d480b04d0 0x7f7d481562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:16.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.409+0000 7f7d4d62b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d4814d930 0x7f7d481567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:16.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.409+0000 7f7d4d62b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480b1650 con 0x7f7d340047c0 2023-10-16T11:05:16.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.409+0000 7f7d4d62b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4814d770 con 0x7f7d480b04d0 2023-10-16T11:05:16.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.409+0000 7f7d4d62b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4815d980 con 0x7f7d4814d930 2023-10-16T11:05:16.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.409+0000 7f7d46ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d340047c0 0x7f7d4814d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:16.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.409+0000 7f7d46ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d340047c0 0x7f7d4814d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36352/0 (socket says 172.21.15.84:36352) 2023-10-16T11:05:16.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.409+0000 7f7d46ffd640 1 -- 172.21.15.84:0/4006264238 learned_addr learned my addr 172.21.15.84:0/4006264238 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:16.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.409+0000 7f7d46ffd640 1 -- 172.21.15.84:0/4006264238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d4814d930 msgr2=0x7f7d481567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:16.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.409+0000 7f7d46ffd640 1 --2- 172.21.15.84:0/4006264238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d4814d930 0x7f7d481567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:16.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.409+0000 7f7d46ffd640 1 -- 172.21.15.84:0/4006264238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d480b04d0 msgr2=0x7f7d481562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:16.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.409+0000 7f7d46ffd640 1 --2- 172.21.15.84:0/4006264238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d480b04d0 0x7f7d481562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:16.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.410+0000 7f7d46ffd640 1 -- 172.21.15.84:0/4006264238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d480b0a40 con 0x7f7d340047c0 2023-10-16T11:05:16.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.410+0000 7f7d46ffd640 1 --2- 172.21.15.84:0/4006264238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d340047c0 0x7f7d4814d230 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800caa0 tx=0x7f7d3800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:16.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.410+0000 7f7d3f7fe640 1 -- 172.21.15.84:0/4006264238 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800a7f0 con 0x7f7d340047c0 2023-10-16T11:05:16.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.410+0000 7f7d3f7fe640 1 -- 172.21.15.84:0/4006264238 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3800a950 con 0x7f7d340047c0 2023-10-16T11:05:16.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.410+0000 7f7d3f7fe640 1 -- 172.21.15.84:0/4006264238 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800ad40 con 0x7f7d340047c0 2023-10-16T11:05:16.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.410+0000 7f7d4d62b640 1 -- 172.21.15.84:0/4006264238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d480b0d20 con 0x7f7d340047c0 2023-10-16T11:05:16.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.410+0000 7f7d4d62b640 1 -- 172.21.15.84:0/4006264238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d480b12e0 con 0x7f7d340047c0 2023-10-16T11:05:16.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.411+0000 7f7d4d62b640 1 -- 172.21.15.84:0/4006264238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d4804e160 con 0x7f7d340047c0 2023-10-16T11:05:16.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.412+0000 7f7d3f7fe640 1 -- 172.21.15.84:0/4006264238 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 88) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f7d380070d0 con 0x7f7d340047c0 2023-10-16T11:05:16.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.413+0000 7f7d3f7fe640 1 --2- 172.21.15.84:0/4006264238 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f7d2806b010 0x7f7d2806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:16.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.413+0000 7f7d3f7fe640 1 -- 172.21.15.84:0/4006264238 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f7d38089120 con 0x7f7d340047c0 2023-10-16T11:05:16.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.414+0000 7f7d467fc640 1 --2- 172.21.15.84:0/4006264238 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f7d2806b010 0x7f7d2806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:16.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.414+0000 7f7d467fc640 1 --2- 172.21.15.84:0/4006264238 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f7d2806b010 0x7f7d2806d4d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7d30033000 tx=0x7f7d3002f000 comp rx=0 tx=0).ready entity=mgr.9327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:16.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.417+0000 7f7d3f7fe640 1 -- 172.21.15.84:0/4006264238 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7d38053ed0 con 0x7f7d340047c0 2023-10-16T11:05:16.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.633+0000 7f7d4d62b640 1 -- 172.21.15.84:0/4006264238 --> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d481610b0 con 0x7f7d2806b010 2023-10-16T11:05:16.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.852+0000 7f7d3f7fe640 1 -- 172.21.15.84:0/4006264238 <== mgr.9327 v2:172.21.15.84:6832/3746729142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+42 (secure 0 0 0) 0x7f7d300051d0 con 0x7f7d2806b010 2023-10-16T11:05:16.854 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path /snap_test_dir1 2023-10-16T11:05:16.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.855+0000 7f7d4d62b640 1 -- 172.21.15.84:0/4006264238 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f7d2806b010 msgr2=0x7f7d2806d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:16.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.855+0000 7f7d4d62b640 1 --2- 172.21.15.84:0/4006264238 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f7d2806b010 0x7f7d2806d4d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7d30033000 tx=0x7f7d3002f000 comp rx=0 tx=0).stop 2023-10-16T11:05:16.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.855+0000 7f7d4d62b640 1 -- 172.21.15.84:0/4006264238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d340047c0 msgr2=0x7f7d4814d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:16.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.855+0000 7f7d4d62b640 1 --2- 172.21.15.84:0/4006264238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d340047c0 0x7f7d4814d230 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800caa0 tx=0x7f7d3800cf60 comp rx=0 tx=0).stop 2023-10-16T11:05:16.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.855+0000 7f7d4d62b640 1 -- 172.21.15.84:0/4006264238 shutdown_connections 2023-10-16T11:05:16.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.855+0000 7f7d4d62b640 1 --2- 172.21.15.84:0/4006264238 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f7d2806b010 0x7f7d2806d4d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:16.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.855+0000 7f7d4d62b640 1 --2- 172.21.15.84:0/4006264238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7d4814d930 0x7f7d481567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:16.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.855+0000 7f7d4d62b640 1 --2- 172.21.15.84:0/4006264238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d480b04d0 0x7f7d481562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:16.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.855+0000 7f7d4d62b640 1 --2- 172.21.15.84:0/4006264238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d340047c0 0x7f7d4814d230 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:16.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.855+0000 7f7d4d62b640 1 -- 172.21.15.84:0/4006264238 >> 172.21.15.84:0/4006264238 conn(0x7f7d480d0ab0 msgr2=0x7f7d4806c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:16.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.855+0000 7f7d4d62b640 1 -- 172.21.15.84:0/4006264238 shutdown_connections 2023-10-16T11:05:16.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:16.856+0000 7f7d4d62b640 1 -- 172.21.15.84:0/4006264238 wait complete. 2023-10-16T11:05:16.876 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1 2023-10-16T11:05:16.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:16.877 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:05:16.935 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_11_00_UTC 2023-10-16T11:05:16.936 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_12_00_UTC 2023-10-16T11:05:16.936 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_13_00_UTC 2023-10-16T11:05:16.936 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_14_00_UTC 2023-10-16T11:05:16.936 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_15_00_UTC 2023-10-16T11:05:16.937 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_16_00_UTC 2023-10-16T11:05:16.937 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_17_00_UTC 2023-10-16T11:05:16.937 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_18_00_UTC 2023-10-16T11:05:16.937 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_19_00_UTC 2023-10-16T11:05:16.938 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_20_00_UTC 2023-10-16T11:05:16.938 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_21_00_UTC 2023-10-16T11:05:16.938 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_22_00_UTC 2023-10-16T11:05:16.938 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_23_00_UTC 2023-10-16T11:05:16.938 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_24_00_UTC 2023-10-16T11:05:16.939 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_25_00_UTC 2023-10-16T11:05:16.939 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_26_00_UTC 2023-10-16T11:05:16.939 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_27_00_UTC 2023-10-16T11:05:16.939 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_28_00_UTC 2023-10-16T11:05:16.939 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_29_00_UTC 2023-10-16T11:05:16.940 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_30_00_UTC 2023-10-16T11:05:16.940 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_31_00_UTC 2023-10-16T11:05:16.940 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_32_00_UTC 2023-10-16T11:05:16.940 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_33_00_UTC 2023-10-16T11:05:16.940 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_34_00_UTC 2023-10-16T11:05:16.940 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_35_00_UTC 2023-10-16T11:05:16.941 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_36_00_UTC 2023-10-16T11:05:16.941 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_37_00_UTC 2023-10-16T11:05:16.941 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_38_00_UTC 2023-10-16T11:05:16.941 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_39_00_UTC 2023-10-16T11:05:16.941 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_40_00_UTC 2023-10-16T11:05:16.942 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_41_00_UTC 2023-10-16T11:05:16.942 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_42_00_UTC 2023-10-16T11:05:16.942 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_43_00_UTC 2023-10-16T11:05:16.942 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_44_00_UTC 2023-10-16T11:05:16.942 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_45_00_UTC 2023-10-16T11:05:16.943 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_46_00_UTC 2023-10-16T11:05:16.943 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_47_00_UTC 2023-10-16T11:05:16.943 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_48_00_UTC 2023-10-16T11:05:16.943 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_49_01_UTC 2023-10-16T11:05:16.943 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_50_00_UTC 2023-10-16T11:05:16.944 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_51_00_UTC 2023-10-16T11:05:16.944 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_52_00_UTC 2023-10-16T11:05:16.944 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_53_00_UTC 2023-10-16T11:05:16.944 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_54_00_UTC 2023-10-16T11:05:16.944 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_55_00_UTC 2023-10-16T11:05:16.945 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_56_00_UTC 2023-10-16T11:05:16.945 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_57_00_UTC 2023-10-16T11:05:16.945 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_58_00_UTC 2023-10-16T11:05:16.945 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-10_59_00_UTC 2023-10-16T11:05:16.945 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_00_00_UTC 2023-10-16T11:05:16.946 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_01_00_UTC 2023-10-16T11:05:16.946 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_02_00_UTC 2023-10-16T11:05:16.946 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_03_00_UTC 2023-10-16T11:05:16.946 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_04_00_UTC 2023-10-16T11:05:16.946 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_05_00_UTC 2023-10-16T11:05:16.947 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_11_00_UTC 2023-10-16T11:05:16.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:16.948 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_11_00_UTC) 2023-10-16T11:05:17.020 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_12_00_UTC 2023-10-16T11:05:17.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:17.020 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_12_00_UTC) 2023-10-16T11:05:17.092 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_13_00_UTC 2023-10-16T11:05:17.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:17.093 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_13_00_UTC) 2023-10-16T11:05:17.160 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_14_00_UTC 2023-10-16T11:05:17.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:17.161 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_14_00_UTC) 2023-10-16T11:05:17.230 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_15_00_UTC 2023-10-16T11:05:17.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:17.230 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_15_00_UTC) 2023-10-16T11:05:17.299 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_16_00_UTC 2023-10-16T11:05:17.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:17.300 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_16_00_UTC) 2023-10-16T11:05:17.370 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_17_00_UTC 2023-10-16T11:05:17.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:17.371 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_17_00_UTC) 2023-10-16T11:05:17.436 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_18_00_UTC 2023-10-16T11:05:17.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:17.437 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_18_00_UTC) 2023-10-16T11:05:17.507 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_19_00_UTC 2023-10-16T11:05:17.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:17.509 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_19_00_UTC) 2023-10-16T11:05:17.584 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_20_00_UTC 2023-10-16T11:05:17.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:17.585 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_20_00_UTC) 2023-10-16T11:05:17.653 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_21_00_UTC 2023-10-16T11:05:17.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:17.654 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_21_00_UTC) 2023-10-16T11:05:17.728 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_22_00_UTC 2023-10-16T11:05:17.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:17.729 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_22_00_UTC) 2023-10-16T11:05:17.801 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_23_00_UTC 2023-10-16T11:05:17.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:17.802 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_23_00_UTC) 2023-10-16T11:05:17.870 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_24_00_UTC 2023-10-16T11:05:17.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:17.871 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_24_00_UTC) 2023-10-16T11:05:17.943 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_25_00_UTC 2023-10-16T11:05:17.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:17.944 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_25_00_UTC) 2023-10-16T11:05:18.013 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_26_00_UTC 2023-10-16T11:05:18.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:18.014 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_26_00_UTC) 2023-10-16T11:05:18.082 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_27_00_UTC 2023-10-16T11:05:18.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:18.083 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_27_00_UTC) 2023-10-16T11:05:18.156 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_28_00_UTC 2023-10-16T11:05:18.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:18.157 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_28_00_UTC) 2023-10-16T11:05:18.229 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_29_00_UTC 2023-10-16T11:05:18.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:18.230 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_29_00_UTC) 2023-10-16T11:05:18.302 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_30_00_UTC 2023-10-16T11:05:18.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:18.303 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_30_00_UTC) 2023-10-16T11:05:18.375 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_31_00_UTC 2023-10-16T11:05:18.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:18.376 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_31_00_UTC) 2023-10-16T11:05:18.447 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_32_00_UTC 2023-10-16T11:05:18.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:18.448 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_32_00_UTC) 2023-10-16T11:05:18.520 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_33_00_UTC 2023-10-16T11:05:18.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:18.521 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_33_00_UTC) 2023-10-16T11:05:18.593 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_34_00_UTC 2023-10-16T11:05:18.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:18.594 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_34_00_UTC) 2023-10-16T11:05:18.667 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_35_00_UTC 2023-10-16T11:05:18.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:18.668 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_35_00_UTC) 2023-10-16T11:05:18.734 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_36_00_UTC 2023-10-16T11:05:18.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:18.735 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_36_00_UTC) 2023-10-16T11:05:18.800 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_37_00_UTC 2023-10-16T11:05:18.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:18.801 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_37_00_UTC) 2023-10-16T11:05:18.870 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_38_00_UTC 2023-10-16T11:05:18.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:18.871 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_38_00_UTC) 2023-10-16T11:05:18.939 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_39_00_UTC 2023-10-16T11:05:18.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:18.940 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_39_00_UTC) 2023-10-16T11:05:19.015 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_40_00_UTC 2023-10-16T11:05:19.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:19.016 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_40_00_UTC) 2023-10-16T11:05:19.088 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_41_00_UTC 2023-10-16T11:05:19.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:19.088 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_41_00_UTC) 2023-10-16T11:05:19.162 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_42_00_UTC 2023-10-16T11:05:19.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:19.163 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_42_00_UTC) 2023-10-16T11:05:19.230 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_43_00_UTC 2023-10-16T11:05:19.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:19.231 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_43_00_UTC) 2023-10-16T11:05:19.303 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_44_00_UTC 2023-10-16T11:05:19.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:19.304 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_44_00_UTC) 2023-10-16T11:05:19.373 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_45_00_UTC 2023-10-16T11:05:19.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:19.374 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_45_00_UTC) 2023-10-16T11:05:19.444 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_46_00_UTC 2023-10-16T11:05:19.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:19.445 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_46_00_UTC) 2023-10-16T11:05:19.510 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_47_00_UTC 2023-10-16T11:05:19.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:19.511 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_47_00_UTC) 2023-10-16T11:05:19.574 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_48_00_UTC 2023-10-16T11:05:19.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:19.575 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_48_00_UTC) 2023-10-16T11:05:19.640 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_49_01_UTC 2023-10-16T11:05:19.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:19.641 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_49_01_UTC) 2023-10-16T11:05:19.707 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_50_00_UTC 2023-10-16T11:05:19.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:19.708 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_50_00_UTC) 2023-10-16T11:05:19.773 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_51_00_UTC 2023-10-16T11:05:19.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:19.774 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_51_00_UTC) 2023-10-16T11:05:19.834 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_52_00_UTC 2023-10-16T11:05:19.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:19.835 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_52_00_UTC) 2023-10-16T11:05:19.897 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_53_00_UTC 2023-10-16T11:05:19.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:19.898 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_53_00_UTC) 2023-10-16T11:05:19.960 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_54_00_UTC 2023-10-16T11:05:19.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:19.962 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_54_00_UTC) 2023-10-16T11:05:20.030 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_55_00_UTC 2023-10-16T11:05:20.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:20.031 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_55_00_UTC) 2023-10-16T11:05:20.091 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_56_00_UTC 2023-10-16T11:05:20.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:20.092 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_56_00_UTC) 2023-10-16T11:05:20.152 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_57_00_UTC 2023-10-16T11:05:20.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:20.153 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_57_00_UTC) 2023-10-16T11:05:20.218 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_58_00_UTC 2023-10-16T11:05:20.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:20.219 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_58_00_UTC) 2023-10-16T11:05:20.278 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_59_00_UTC 2023-10-16T11:05:20.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:20.279 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_59_00_UTC) 2023-10-16T11:05:20.338 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-11_00_00_UTC 2023-10-16T11:05:20.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:20.339 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-11_00_00_UTC) 2023-10-16T11:05:20.399 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-11_01_00_UTC 2023-10-16T11:05:20.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:20.400 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-11_01_00_UTC) 2023-10-16T11:05:20.462 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-11_02_00_UTC 2023-10-16T11:05:20.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:20.463 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-11_02_00_UTC) 2023-10-16T11:05:20.522 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-11_03_00_UTC 2023-10-16T11:05:20.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:20.523 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-11_03_00_UTC) 2023-10-16T11:05:20.585 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-11_04_00_UTC 2023-10-16T11:05:20.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:20.586 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-11_04_00_UTC) 2023-10-16T11:05:20.644 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-11_05_00_UTC 2023-10-16T11:05:20.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:20.645 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-11_05_00_UTC) 2023-10-16T11:05:20.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:20.708 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-16T11:05:20.763 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T11:05:20.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.978+0000 7fb805800640 1 Processor -- start 2023-10-16T11:05:20.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.978+0000 7fb805800640 1 -- start start 2023-10-16T11:05:20.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.980+0000 7fb805800640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb80015ad70 0x7fb800069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:20.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.980+0000 7fb805800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb8000caac0 0x7fb80006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:20.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.980+0000 7fb805800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8000cb030 con 0x7fb8000c9ca0 2023-10-16T11:05:20.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.980+0000 7fb805800640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8000c91c0 con 0x7fb80015ad70 2023-10-16T11:05:20.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.980+0000 7fb805800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8000c94a0 con 0x7fb8000caac0 2023-10-16T11:05:20.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.980+0000 7fb7feffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb80015ad70 0x7fb800069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:20.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.980+0000 7fb7feffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb80015ad70 0x7fb800069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36360/0 (socket says 172.21.15.84:36360) 2023-10-16T11:05:20.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.980+0000 7fb7feffd640 1 -- 172.21.15.84:0/451894454 learned_addr learned my addr 172.21.15.84:0/451894454 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:20.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.981+0000 7fb7feffd640 1 -- 172.21.15.84:0/451894454 >> v2:172.21.15.157:3300/0 conn(0x7fb8000caac0 msgr2=0x7fb80006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:20.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.981+0000 7fb7feffd640 1 --2- 172.21.15.84:0/451894454 >> v2:172.21.15.157:3300/0 conn(0x7fb8000caac0 0x7fb80006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:20.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.981+0000 7fb7feffd640 1 -- 172.21.15.84:0/451894454 >> v1:172.21.15.157:6789/0 conn(0x7fb8000c9ca0 legacy=0x7fb8000ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:20.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.981+0000 7fb7feffd640 1 -- 172.21.15.84:0/451894454 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8000c9780 con 0x7fb80015ad70 2023-10-16T11:05:20.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.982+0000 7fb7feffd640 1 --2- 172.21.15.84:0/451894454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb80015ad70 0x7fb800069da0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fb80006d090 tx=0x7fb7f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e54eb04b8d790b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:20.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.982+0000 7fb7fdffb640 1 -- 172.21.15.84:0/451894454 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0031870 con 0x7fb80015ad70 2023-10-16T11:05:20.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.982+0000 7fb7fdffb640 1 -- 172.21.15.84:0/451894454 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f00319d0 con 0x7fb80015ad70 2023-10-16T11:05:20.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.982+0000 7fb7fdffb640 1 -- 172.21.15.84:0/451894454 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0031d60 con 0x7fb80015ad70 2023-10-16T11:05:20.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.983+0000 7fb805800640 1 -- 172.21.15.84:0/451894454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb80015ad70 msgr2=0x7fb800069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:20.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.983+0000 7fb805800640 1 --2- 172.21.15.84:0/451894454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb80015ad70 0x7fb800069da0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fb80006d090 tx=0x7fb7f002b910 comp rx=0 tx=0).stop 2023-10-16T11:05:20.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.983+0000 7fb805800640 1 -- 172.21.15.84:0/451894454 shutdown_connections 2023-10-16T11:05:20.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.983+0000 7fb805800640 1 --2- 172.21.15.84:0/451894454 >> v2:172.21.15.157:3300/0 conn(0x7fb8000caac0 0x7fb80006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:20.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.983+0000 7fb805800640 1 --2- 172.21.15.84:0/451894454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb80015ad70 0x7fb800069da0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:20.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.983+0000 7fb805800640 1 -- 172.21.15.84:0/451894454 >> 172.21.15.84:0/451894454 conn(0x7fb8000ce9b0 msgr2=0x7fb8000cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:20.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.983+0000 7fb805800640 1 -- 172.21.15.84:0/451894454 shutdown_connections 2023-10-16T11:05:20.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.983+0000 7fb805800640 1 -- 172.21.15.84:0/451894454 wait complete. 2023-10-16T11:05:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.985+0000 7fb805800640 1 Processor -- start 2023-10-16T11:05:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.985+0000 7fb805800640 1 -- start start 2023-10-16T11:05:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.986+0000 7fb805800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8000c9ca0 0x7fb80015d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.986+0000 7fb805800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8000caac0 0x7fb8000ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.986+0000 7fb805800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb80015ad70 0x7fb8000acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:20.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.986+0000 7fb805800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80015e3c0 con 0x7fb80015ad70 2023-10-16T11:05:20.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.986+0000 7fb805800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80015de00 con 0x7fb8000caac0 2023-10-16T11:05:20.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.986+0000 7fb805800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80015e0e0 con 0x7fb8000c9ca0 2023-10-16T11:05:20.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.986+0000 7fb7ff7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb80015ad70 0x7fb8000acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:20.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.987+0000 7fb7ff7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb80015ad70 0x7fb8000acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36370/0 (socket says 172.21.15.84:36370) 2023-10-16T11:05:20.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.987+0000 7fb7ff7fe640 1 -- 172.21.15.84:0/3725437490 learned_addr learned my addr 172.21.15.84:0/3725437490 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:20.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.987+0000 7fb7ff7fe640 1 -- 172.21.15.84:0/3725437490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8000c9ca0 msgr2=0x7fb80015d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:20.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.987+0000 7fb7ff7fe640 1 --2- 172.21.15.84:0/3725437490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8000c9ca0 0x7fb80015d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:20.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.987+0000 7fb7ff7fe640 1 -- 172.21.15.84:0/3725437490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8000caac0 msgr2=0x7fb8000ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:20.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.987+0000 7fb7ff7fe640 1 --2- 172.21.15.84:0/3725437490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8000caac0 0x7fb8000ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:20.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.987+0000 7fb7ff7fe640 1 -- 172.21.15.84:0/3725437490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f00094d0 con 0x7fb80015ad70 2023-10-16T11:05:20.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.987+0000 7fb7ff7fe640 1 --2- 172.21.15.84:0/3725437490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb80015ad70 0x7fb8000acdd0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800b970 tx=0x7fb7f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:20.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.988+0000 7fb7e77fe640 1 -- 172.21.15.84:0/3725437490 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f800cb40 con 0x7fb80015ad70 2023-10-16T11:05:20.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.988+0000 7fb7e77fe640 1 -- 172.21.15.84:0/3725437490 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f800ccc0 con 0x7fb80015ad70 2023-10-16T11:05:20.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.988+0000 7fb7e77fe640 1 -- 172.21.15.84:0/3725437490 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8014680 con 0x7fb80015ad70 2023-10-16T11:05:20.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.988+0000 7fb805800640 1 -- 172.21.15.84:0/3725437490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8000cc720 con 0x7fb80015ad70 2023-10-16T11:05:20.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.988+0000 7fb805800640 1 -- 172.21.15.84:0/3725437490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80015daf0 con 0x7fb80015ad70 2023-10-16T11:05:20.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.989+0000 7fb805800640 1 -- 172.21.15.84:0/3725437490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8000cc580 con 0x7fb80015ad70 2023-10-16T11:05:20.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.990+0000 7fb7e77fe640 1 -- 172.21.15.84:0/3725437490 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 88) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fb7f801d3b0 con 0x7fb80015ad70 2023-10-16T11:05:20.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.991+0000 7fb7e77fe640 1 --2- 172.21.15.84:0/3725437490 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fb7ec06afe0 0x7fb7ec06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:20.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.991+0000 7fb7e77fe640 1 -- 172.21.15.84:0/3725437490 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fb7f8089910 con 0x7fb80015ad70 2023-10-16T11:05:20.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.991+0000 7fb7feffd640 1 --2- 172.21.15.84:0/3725437490 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fb7ec06afe0 0x7fb7ec06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:20.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.992+0000 7fb7feffd640 1 --2- 172.21.15.84:0/3725437490 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fb7ec06afe0 0x7fb7ec06d4a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0036000 tx=0x7fb7f0030000 comp rx=0 tx=0).ready entity=mgr.9327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:20.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:20.995+0000 7fb7e77fe640 1 -- 172.21.15.84:0/3725437490 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb8000cc580 con 0x7fb80015ad70 2023-10-16T11:05:21.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:21.238+0000 7fb805800640 1 -- 172.21.15.84:0/3725437490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fb8000ccc40 con 0x7fb80015ad70 2023-10-16T11:05:21.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:21.586+0000 7fb7e77fe640 1 -- 172.21.15.84:0/3725437490 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v89) v1 ==== 93+0+0 (secure 0 0 0) 0x7fb8000ccc40 con 0x7fb80015ad70 2023-10-16T11:05:21.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:21.589+0000 7fb805800640 1 -- 172.21.15.84:0/3725437490 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fb7ec06afe0 msgr2=0x7fb7ec06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:21.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:21.589+0000 7fb805800640 1 --2- 172.21.15.84:0/3725437490 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fb7ec06afe0 0x7fb7ec06d4a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0036000 tx=0x7fb7f0030000 comp rx=0 tx=0).stop 2023-10-16T11:05:21.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:21.589+0000 7fb805800640 1 -- 172.21.15.84:0/3725437490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb80015ad70 msgr2=0x7fb8000acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:21.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:21.589+0000 7fb805800640 1 --2- 172.21.15.84:0/3725437490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb80015ad70 0x7fb8000acdd0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800b970 tx=0x7fb7f800be30 comp rx=0 tx=0).stop 2023-10-16T11:05:21.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:21.590+0000 7fb805800640 1 -- 172.21.15.84:0/3725437490 shutdown_connections 2023-10-16T11:05:21.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:21.590+0000 7fb805800640 1 --2- 172.21.15.84:0/3725437490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb80015ad70 0x7fb8000acdd0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:21.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:21.590+0000 7fb805800640 1 --2- 172.21.15.84:0/3725437490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8000caac0 0x7fb8000ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:21.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:21.590+0000 7fb805800640 1 --2- 172.21.15.84:0/3725437490 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fb7ec06afe0 0x7fb7ec06d4a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:21.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:21.590+0000 7fb805800640 1 --2- 172.21.15.84:0/3725437490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8000c9ca0 0x7fb80015d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:21.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:21.590+0000 7fb805800640 1 -- 172.21.15.84:0/3725437490 >> 172.21.15.84:0/3725437490 conn(0x7fb8000ce9b0 msgr2=0x7fb800159fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:21.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:21.591+0000 7fb805800640 1 -- 172.21.15.84:0/3725437490 shutdown_connections 2023-10-16T11:05:21.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:21.591+0000 7fb805800640 1 -- 172.21.15.84:0/3725437490 wait complete. 2023-10-16T11:05:21.622 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:05:21.623 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:05:21.779 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:05:21.964 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:05:21.965 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:05:21.965 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:05:22.051 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:05:22.052 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:05:22.131 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:05:22.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.379+0000 7f79409c0640 1 Processor -- start 2023-10-16T11:05:22.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.379+0000 7f79409c0640 1 -- start start 2023-10-16T11:05:22.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.380+0000 7f79409c0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f793c15aaa0 0x7f793c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:22.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.380+0000 7f79409c0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f793c15ae80 0x7f793c0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:22.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.380+0000 7f79409c0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793c15ed60 con 0x7f793c0ccaa0 2023-10-16T11:05:22.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.380+0000 7f79409c0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f793c06b810 con 0x7f793c15aaa0 2023-10-16T11:05:22.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.380+0000 7f79409c0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f793c0707e0 con 0x7f793c15ae80 2023-10-16T11:05:22.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.381+0000 7f793a59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f793c15aaa0 0x7f793c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:22.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.381+0000 7f793a59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f793c15aaa0 0x7f793c15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36408/0 (socket says 172.21.15.84:36408) 2023-10-16T11:05:22.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.381+0000 7f793a59c640 1 -- 172.21.15.84:0/2319735849 learned_addr learned my addr 172.21.15.84:0/2319735849 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:22.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.381+0000 7f793ad9d640 1 --1- 172.21.15.84:0/2319735849 >> v1:172.21.15.84:6789/0 conn(0x7f793c0ccaa0 0x7f793c070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37150/0 (socket says 172.21.15.84:37150) 2023-10-16T11:05:22.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.381+0000 7f7933fff640 1 --2- 172.21.15.84:0/2319735849 >> v2:172.21.15.157:3300/0 conn(0x7f793c15ae80 0x7f793c0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:22.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.381+0000 7f793a59c640 1 -- 172.21.15.84:0/2319735849 >> v2:172.21.15.157:3300/0 conn(0x7f793c15ae80 msgr2=0x7f793c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:22.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.381+0000 7f793a59c640 1 --2- 172.21.15.84:0/2319735849 >> v2:172.21.15.157:3300/0 conn(0x7f793c15ae80 0x7f793c0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:22.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.381+0000 7f793a59c640 1 -- 172.21.15.84:0/2319735849 >> v1:172.21.15.84:6789/0 conn(0x7f793c0ccaa0 legacy=0x7f793c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:22.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.381+0000 7f793a59c640 1 -- 172.21.15.84:0/2319735849 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c15ef90 con 0x7f793c15aaa0 2023-10-16T11:05:22.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.381+0000 7f7933fff640 1 --2- 172.21.15.84:0/2319735849 >> v2:172.21.15.157:3300/0 conn(0x7f793c15ae80 0x7f793c0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:22.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.382+0000 7f793a59c640 1 --2- 172.21.15.84:0/2319735849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f793c15aaa0 0x7f793c15e820 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f793c0c59c0 tx=0x7f792c02b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9d5d66604be7f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:22.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.382+0000 7f7939d9b640 1 -- 172.21.15.84:0/2319735849 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c030880 con 0x7f793c15aaa0 2023-10-16T11:05:22.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.383+0000 7f7939d9b640 1 -- 172.21.15.84:0/2319735849 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792c0309e0 con 0x7f793c15aaa0 2023-10-16T11:05:22.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.383+0000 7f7939d9b640 1 -- 172.21.15.84:0/2319735849 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c030d70 con 0x7f793c15aaa0 2023-10-16T11:05:22.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.383+0000 7f79409c0640 1 -- 172.21.15.84:0/2319735849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f793c15aaa0 msgr2=0x7f793c15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:22.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.383+0000 7f79409c0640 1 --2- 172.21.15.84:0/2319735849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f793c15aaa0 0x7f793c15e820 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f793c0c59c0 tx=0x7f792c02b9c0 comp rx=0 tx=0).stop 2023-10-16T11:05:22.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.383+0000 7f79409c0640 1 -- 172.21.15.84:0/2319735849 shutdown_connections 2023-10-16T11:05:22.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.383+0000 7f79409c0640 1 --2- 172.21.15.84:0/2319735849 >> v2:172.21.15.157:3300/0 conn(0x7f793c15ae80 0x7f793c0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:22.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.383+0000 7f79409c0640 1 --2- 172.21.15.84:0/2319735849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f793c15aaa0 0x7f793c15e820 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:22.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.383+0000 7f79409c0640 1 -- 172.21.15.84:0/2319735849 >> 172.21.15.84:0/2319735849 conn(0x7f793c0cb410 msgr2=0x7f793c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:22.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.383+0000 7f79409c0640 1 -- 172.21.15.84:0/2319735849 shutdown_connections 2023-10-16T11:05:22.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.383+0000 7f79409c0640 1 -- 172.21.15.84:0/2319735849 wait complete. 2023-10-16T11:05:22.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.385+0000 7f79409c0640 1 Processor -- start 2023-10-16T11:05:22.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.385+0000 7f79409c0640 1 -- start start 2023-10-16T11:05:22.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.386+0000 7f79409c0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f793c0ccaa0 0x7f793c0af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:22.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.386+0000 7f79409c0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f793c15aaa0 0x7f793c0ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:22.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.386+0000 7f79409c0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f793c15ae80 0x7f793c15d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:22.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.386+0000 7f79409c0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c15dc50 con 0x7f793c0ccaa0 2023-10-16T11:05:22.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.386+0000 7f79409c0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0ae930 con 0x7f793c15ae80 2023-10-16T11:05:22.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.386+0000 7f79409c0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0aec10 con 0x7f793c15aaa0 2023-10-16T11:05:22.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.386+0000 7f793a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f793c0ccaa0 0x7f793c0af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:22.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.386+0000 7f793a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f793c0ccaa0 0x7f793c0af9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36424/0 (socket says 172.21.15.84:36424) 2023-10-16T11:05:22.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.386+0000 7f793a59c640 1 -- 172.21.15.84:0/599911173 learned_addr learned my addr 172.21.15.84:0/599911173 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:22.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.387+0000 7f793a59c640 1 -- 172.21.15.84:0/599911173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f793c15aaa0 msgr2=0x7f793c0ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:22.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.387+0000 7f793a59c640 1 --2- 172.21.15.84:0/599911173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f793c15aaa0 0x7f793c0ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:22.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.387+0000 7f793a59c640 1 -- 172.21.15.84:0/599911173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f793c15ae80 msgr2=0x7f793c15d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:22.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.387+0000 7f793a59c640 1 --2- 172.21.15.84:0/599911173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f793c15ae80 0x7f793c15d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:22.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.387+0000 7f793a59c640 1 -- 172.21.15.84:0/599911173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792c0094d0 con 0x7f793c0ccaa0 2023-10-16T11:05:22.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.387+0000 7f793a59c640 1 --2- 172.21.15.84:0/599911173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f793c0ccaa0 0x7f793c0af9d0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f792c02ba00 tx=0x7f792c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:22.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.387+0000 7f7932ffd640 1 -- 172.21.15.84:0/599911173 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c030870 con 0x7f793c0ccaa0 2023-10-16T11:05:22.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.387+0000 7f7932ffd640 1 -- 172.21.15.84:0/599911173 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792c043400 con 0x7f793c0ccaa0 2023-10-16T11:05:22.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.387+0000 7f7932ffd640 1 -- 172.21.15.84:0/599911173 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c0437b0 con 0x7f793c0ccaa0 2023-10-16T11:05:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.387+0000 7f79409c0640 1 -- 172.21.15.84:0/599911173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793c0aeef0 con 0x7f793c0ccaa0 2023-10-16T11:05:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.387+0000 7f79409c0640 1 -- 172.21.15.84:0/599911173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793c0af4b0 con 0x7f793c0ccaa0 2023-10-16T11:05:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.389+0000 7f79409c0640 1 -- 172.21.15.84:0/599911173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793c04e160 con 0x7f793c0ccaa0 2023-10-16T11:05:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.390+0000 7f7932ffd640 1 -- 172.21.15.84:0/599911173 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 89) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f792c043910 con 0x7f793c0ccaa0 2023-10-16T11:05:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.391+0000 7f7932ffd640 1 --2- 172.21.15.84:0/599911173 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f791c06af90 0x7f791c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.391+0000 7f7932ffd640 1 -- 172.21.15.84:0/599911173 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6227+0+0 (secure 0 0 0) 0x7f792c03e070 con 0x7f793c0ccaa0 2023-10-16T11:05:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.391+0000 7f7933fff640 1 -- 172.21.15.84:0/599911173 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f791c06af90 msgr2=0x7f791c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3746729142 2023-10-16T11:05:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.391+0000 7f7933fff640 1 --2- 172.21.15.84:0/599911173 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f791c06af90 0x7f791c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:05:22.401 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:22.393+0000 7fcf18430040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:05:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.396+0000 7f7932ffd640 1 -- 172.21.15.84:0/599911173 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f792c077ca0 con 0x7f793c0ccaa0 2023-10-16T11:05:22.404 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:22.403+0000 7f34c443e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:05:22.451 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:22.450+0000 7fcf18430040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:05:22.459 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:22.457+0000 7f34c443e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:05:22.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.581+0000 7f79409c0640 1 -- 172.21.15.84:0/599911173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f793c0cdc60 con 0x7f793c0ccaa0 2023-10-16T11:05:22.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.583+0000 7f7932ffd640 1 -- 172.21.15.84:0/599911173 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+17303 (secure 0 0 0) 0x7f792c07b2c0 con 0x7f793c0ccaa0 2023-10-16T11:05:22.586 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:22.586 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":157,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:05:21.582853+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:09:08.490425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-16T10:09:09.666051+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":112,"snap_epoch":157,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.26","mappings":[{"from":2,"to":0}]},{"pgid":"19.3c","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2439552742":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/384925658":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/3023009161":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/870751622":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/300305981":"2023-10-17T10:09:25.208422+0000","172.21.15.84:6833/205480762":"2023-10-17T10:09:25.208422+0000","172.21.15.84:6832/205480762":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/1358444053":"2023-10-17T10:09:25.208422+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":3,"length":54},{"begin":59,"length":54}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":3,"length":54},{"begin":59,"length":54}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:05:22.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.585+0000 7f79409c0640 1 -- 172.21.15.84:0/599911173 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f791c06af90 msgr2=0x7f791c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:05:22.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.586+0000 7f79409c0640 1 --2- 172.21.15.84:0/599911173 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f791c06af90 0x7f791c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:22.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.586+0000 7f79409c0640 1 -- 172.21.15.84:0/599911173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f793c0ccaa0 msgr2=0x7f793c0af9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:22.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.586+0000 7f79409c0640 1 --2- 172.21.15.84:0/599911173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f793c0ccaa0 0x7f793c0af9d0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f792c02ba00 tx=0x7f792c004730 comp rx=0 tx=0).stop 2023-10-16T11:05:22.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.586+0000 7f79409c0640 1 -- 172.21.15.84:0/599911173 shutdown_connections 2023-10-16T11:05:22.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.586+0000 7f79409c0640 1 --2- 172.21.15.84:0/599911173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f793c15ae80 0x7f793c15d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:22.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.586+0000 7f79409c0640 1 --2- 172.21.15.84:0/599911173 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f791c06af90 0x7f791c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:22.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.586+0000 7f79409c0640 1 --2- 172.21.15.84:0/599911173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f793c15aaa0 0x7f793c0ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:22.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.586+0000 7f79409c0640 1 --2- 172.21.15.84:0/599911173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f793c0ccaa0 0x7f793c0af9d0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:22.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.586+0000 7f79409c0640 1 -- 172.21.15.84:0/599911173 >> 172.21.15.84:0/599911173 conn(0x7f793c0cb410 msgr2=0x7f793c0cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:22.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.586+0000 7f79409c0640 1 -- 172.21.15.84:0/599911173 shutdown_connections 2023-10-16T11:05:22.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.586+0000 7f79409c0640 1 -- 172.21.15.84:0/599911173 wait complete. 2023-10-16T11:05:22.606 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:09:08.490425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:09:09.666051+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '157', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 112, 'snap_epoch': 157, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:05:22.607 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:05:22.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.795+0000 7f2794ad1640 1 Processor -- start 2023-10-16T11:05:22.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.796+0000 7f2794ad1640 1 -- start start 2023-10-16T11:05:22.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.797+0000 7f2794ad1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27900b2080 0x7f27900d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:22.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.797+0000 7f2794ad1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f27900cfc40 0x7f27900b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:22.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.797+0000 7f2794ad1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27900d0970 con 0x7f27900b1ca0 2023-10-16T11:05:22.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.797+0000 7f2794ad1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27900d0ea0 con 0x7f27900cfc40 2023-10-16T11:05:22.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.797+0000 7f2794ad1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27900d0bc0 con 0x7f27900b2080 2023-10-16T11:05:22.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.797+0000 7f278ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f27900cfc40 0x7f27900b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:22.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.797+0000 7f278ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f27900cfc40 0x7f27900b24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36432/0 (socket says 172.21.15.84:36432) 2023-10-16T11:05:22.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.797+0000 7f278ffff640 1 -- 172.21.15.84:0/216796198 learned_addr learned my addr 172.21.15.84:0/216796198 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:22.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.797+0000 7f2787fff640 1 --2- 172.21.15.84:0/216796198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27900b2080 0x7f27900d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:22.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.797+0000 7f278ffff640 1 -- 172.21.15.84:0/216796198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27900b2080 msgr2=0x7f27900d36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:22.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.797+0000 7f278ffff640 1 --2- 172.21.15.84:0/216796198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27900b2080 0x7f27900d36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:22.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.797+0000 7f278ffff640 1 -- 172.21.15.84:0/216796198 >> v1:172.21.15.157:6789/0 conn(0x7f27900b1ca0 legacy=0x7f27900ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:22.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.797+0000 7f278ffff640 1 -- 172.21.15.84:0/216796198 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27900cd940 con 0x7f27900cfc40 2023-10-16T11:05:22.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.797+0000 7f2787fff640 1 --2- 172.21.15.84:0/216796198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27900b2080 0x7f27900d36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.798+0000 7f278ffff640 1 --2- 172.21.15.84:0/216796198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27900cfc40 0x7f27900b24c0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f2788009e50 tx=0x7f278802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22ae75534cac5a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.799+0000 7f278effd640 1 -- 172.21.15.84:0/216796198 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278802cb00 con 0x7f27900cfc40 2023-10-16T11:05:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.799+0000 7f278effd640 1 -- 172.21.15.84:0/216796198 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278802cc60 con 0x7f27900cfc40 2023-10-16T11:05:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.799+0000 7f278effd640 1 -- 172.21.15.84:0/216796198 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27880343f0 con 0x7f27900cfc40 2023-10-16T11:05:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.799+0000 7f2794ad1640 1 -- 172.21.15.84:0/216796198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27900cfc40 msgr2=0x7f27900b24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.799+0000 7f2794ad1640 1 --2- 172.21.15.84:0/216796198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27900cfc40 0x7f27900b24c0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f2788009e50 tx=0x7f278802bf60 comp rx=0 tx=0).stop 2023-10-16T11:05:22.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.799+0000 7f2794ad1640 1 -- 172.21.15.84:0/216796198 shutdown_connections 2023-10-16T11:05:22.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.799+0000 7f2794ad1640 1 --2- 172.21.15.84:0/216796198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27900cfc40 0x7f27900b24c0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:22.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.800+0000 7f2794ad1640 1 --2- 172.21.15.84:0/216796198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27900b2080 0x7f27900d36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:22.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.800+0000 7f2794ad1640 1 -- 172.21.15.84:0/216796198 >> 172.21.15.84:0/216796198 conn(0x7f27900d2b90 msgr2=0x7f27900d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:22.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.800+0000 7f2794ad1640 1 -- 172.21.15.84:0/216796198 shutdown_connections 2023-10-16T11:05:22.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.800+0000 7f2794ad1640 1 -- 172.21.15.84:0/216796198 wait complete. 2023-10-16T11:05:22.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.801+0000 7f2794ad1640 1 Processor -- start 2023-10-16T11:05:22.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.801+0000 7f2794ad1640 1 -- start start 2023-10-16T11:05:22.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.802+0000 7f2794ad1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27900b1ca0 0x7f279015d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:22.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.802+0000 7f2794ad1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27900b2080 0x7f279009d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:22.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.802+0000 7f2794ad1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27900cfc40 0x7f279009fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:22.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.802+0000 7f2794ad1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279009e2b0 con 0x7f27900b1ca0 2023-10-16T11:05:22.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.802+0000 7f2794ad1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279009e590 con 0x7f27900b2080 2023-10-16T11:05:22.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.802+0000 7f2794ad1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279009e870 con 0x7f27900cfc40 2023-10-16T11:05:22.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.803+0000 7f278f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27900b1ca0 0x7f279015d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:22.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.803+0000 7f278f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27900b1ca0 0x7f279015d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36438/0 (socket says 172.21.15.84:36438) 2023-10-16T11:05:22.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.803+0000 7f278f7fe640 1 -- 172.21.15.84:0/3187713983 learned_addr learned my addr 172.21.15.84:0/3187713983 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:22.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.803+0000 7f2787fff640 1 --2- 172.21.15.84:0/3187713983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27900b2080 0x7f279009d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:22.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.803+0000 7f278f7fe640 1 -- 172.21.15.84:0/3187713983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27900cfc40 msgr2=0x7f279009fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:22.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.803+0000 7f278f7fe640 1 --2- 172.21.15.84:0/3187713983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27900cfc40 0x7f279009fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:22.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.803+0000 7f278f7fe640 1 -- 172.21.15.84:0/3187713983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27900b2080 msgr2=0x7f279009d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:22.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.803+0000 7f278f7fe640 1 --2- 172.21.15.84:0/3187713983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27900b2080 0x7f279009d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:22.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.803+0000 7f278f7fe640 1 -- 172.21.15.84:0/3187713983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2788009b30 con 0x7f27900b1ca0 2023-10-16T11:05:22.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.803+0000 7f278f7fe640 1 --2- 172.21.15.84:0/3187713983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27900b1ca0 0x7f279015d0b0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f278000b3e0 tx=0x7f278000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:22.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.804+0000 7f278cff9640 1 -- 172.21.15.84:0/3187713983 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780011830 con 0x7f27900b1ca0 2023-10-16T11:05:22.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.804+0000 7f278cff9640 1 -- 172.21.15.84:0/3187713983 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27800119b0 con 0x7f27900b1ca0 2023-10-16T11:05:22.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.804+0000 7f2794ad1640 1 -- 172.21.15.84:0/3187713983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27900c9e00 con 0x7f27900b1ca0 2023-10-16T11:05:22.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.804+0000 7f278cff9640 1 -- 172.21.15.84:0/3187713983 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780011d60 con 0x7f27900b1ca0 2023-10-16T11:05:22.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.804+0000 7f2794ad1640 1 -- 172.21.15.84:0/3187713983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f279009caf0 con 0x7f27900b1ca0 2023-10-16T11:05:22.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.805+0000 7f2794ad1640 1 -- 172.21.15.84:0/3187713983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27900ca320 con 0x7f27900b1ca0 2023-10-16T11:05:22.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.806+0000 7f278cff9640 1 -- 172.21.15.84:0/3187713983 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 89) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f278001a400 con 0x7f27900b1ca0 2023-10-16T11:05:22.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.807+0000 7f278cff9640 1 --2- 172.21.15.84:0/3187713983 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f277006af90 0x7f277006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:22.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.807+0000 7f2787fff640 1 -- 172.21.15.84:0/3187713983 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f277006af90 msgr2=0x7f277006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3746729142 2023-10-16T11:05:22.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.807+0000 7f278cff9640 1 -- 172.21.15.84:0/3187713983 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6227+0+0 (secure 0 0 0) 0x7f2780089bb0 con 0x7f27900b1ca0 2023-10-16T11:05:22.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.807+0000 7f2787fff640 1 --2- 172.21.15.84:0/3187713983 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f277006af90 0x7f277006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:05:22.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.811+0000 7f278cff9640 1 -- 172.21.15.84:0/3187713983 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f27900ca320 con 0x7f27900b1ca0 2023-10-16T11:05:22.921 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:22.919+0000 7f34c443e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:05:22.927 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:22.926+0000 7fcf18430040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:05:22.973 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:22.971+0000 7f34c443e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:05:22.980 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:22.979+0000 7fcf18430040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:05:22.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.995+0000 7f2794ad1640 1 -- 172.21.15.84:0/3187713983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27900ca320 con 0x7f27900b1ca0 2023-10-16T11:05:22.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.996+0000 7f278cff9640 1 -- 172.21.15.84:0/3187713983 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27900ca320 con 0x7f27900b1ca0 2023-10-16T11:05:22.998 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:05:23.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.998+0000 7f2794ad1640 1 -- 172.21.15.84:0/3187713983 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f277006af90 msgr2=0x7f277006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:05:23.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.998+0000 7f2794ad1640 1 --2- 172.21.15.84:0/3187713983 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f277006af90 0x7f277006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.998+0000 7f2794ad1640 1 -- 172.21.15.84:0/3187713983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27900b1ca0 msgr2=0x7f279015d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:23.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.998+0000 7f2794ad1640 1 --2- 172.21.15.84:0/3187713983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27900b1ca0 0x7f279015d0b0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f278000b3e0 tx=0x7f278000b8a0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.998+0000 7f2794ad1640 1 -- 172.21.15.84:0/3187713983 shutdown_connections 2023-10-16T11:05:23.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.998+0000 7f2794ad1640 1 --2- 172.21.15.84:0/3187713983 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f277006af90 0x7f277006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.998+0000 7f2794ad1640 1 --2- 172.21.15.84:0/3187713983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27900cfc40 0x7f279009fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.998+0000 7f2794ad1640 1 --2- 172.21.15.84:0/3187713983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27900b2080 0x7f279009d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.998+0000 7f2794ad1640 1 --2- 172.21.15.84:0/3187713983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27900b1ca0 0x7f279015d0b0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.998+0000 7f2794ad1640 1 -- 172.21.15.84:0/3187713983 >> 172.21.15.84:0/3187713983 conn(0x7f27900d2b90 msgr2=0x7f27900c86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:23.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.998+0000 7f2794ad1640 1 -- 172.21.15.84:0/3187713983 shutdown_connections 2023-10-16T11:05:23.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:22.998+0000 7f2794ad1640 1 -- 172.21.15.84:0/3187713983 wait complete. 2023-10-16T11:05:23.018 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:05:23.023 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:23.021+0000 7f34c443e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:05:23.035 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:23.034+0000 7fcf18430040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:05:23.084 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:23.082+0000 7f34c443e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:05:23.103 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:23.101+0000 7fcf18430040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:05:23.135 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:23.133+0000 7f34c443e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:05:23.154 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:23.152+0000 7fcf18430040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:05:23.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.228+0000 7fd20da59640 1 Processor -- start 2023-10-16T11:05:23.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.228+0000 7fd20da59640 1 -- start start 2023-10-16T11:05:23.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.230+0000 7fd20da59640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd208160c80 0x7fd208161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:23.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.230+0000 7fd20da59640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd208070950 0x7fd208140400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:23.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.230+0000 7fd20da59640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2080ae1a0 con 0x7fd2080ad7a0 2023-10-16T11:05:23.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.230+0000 7fd20da59640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2080ae480 con 0x7fd208070950 2023-10-16T11:05:23.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.230+0000 7fd20da59640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2080ae760 con 0x7fd208160c80 2023-10-16T11:05:23.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.231+0000 7fd207fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd2080ad7a0 0x7fd208070240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37154/0 (socket says 172.21.15.84:37154) 2023-10-16T11:05:23.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.231+0000 7fd207fff640 1 -- 172.21.15.84:0/1204594505 learned_addr learned my addr 172.21.15.84:0/1204594505 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:23.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.231+0000 7fd20ca57640 1 --2- 172.21.15.84:0/1204594505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd208160c80 0x7fd208161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:23.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.232+0000 7fd20d258640 1 --2- 172.21.15.84:0/1204594505 >> v2:172.21.15.157:3300/0 conn(0x7fd208070950 0x7fd208140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:23.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.232+0000 7fd2077fe640 1 -- 172.21.15.84:0/1204594505 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675821036 0 0) 0x7fd2080ae1a0 con 0x7fd2080ad7a0 2023-10-16T11:05:23.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.232+0000 7fd2077fe640 1 -- 172.21.15.84:0/1204594505 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1f8003520 con 0x7fd2080ad7a0 2023-10-16T11:05:23.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.233+0000 7fd20ca57640 1 -- 172.21.15.84:0/1204594505 >> v2:172.21.15.157:3300/0 conn(0x7fd208070950 msgr2=0x7fd208140400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:23.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.233+0000 7fd20ca57640 1 --2- 172.21.15.84:0/1204594505 >> v2:172.21.15.157:3300/0 conn(0x7fd208070950 0x7fd208140400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.233+0000 7fd20ca57640 1 -- 172.21.15.84:0/1204594505 >> v1:172.21.15.84:6789/0 conn(0x7fd2080ad7a0 legacy=0x7fd208070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:23.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.233+0000 7fd20ca57640 1 -- 172.21.15.84:0/1204594505 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2080aea40 con 0x7fd208160c80 2023-10-16T11:05:23.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.235+0000 7fd20ca57640 1 --2- 172.21.15.84:0/1204594505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd208160c80 0x7fd208161060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0097a0 tx=0x7fd1fc02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44c64bb35a0fc7e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:23.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.236+0000 7fd2077fe640 1 -- 172.21.15.84:0/1204594505 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc02d040 con 0x7fd208160c80 2023-10-16T11:05:23.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.237+0000 7fd2077fe640 1 -- 172.21.15.84:0/1204594505 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1fc031810 con 0x7fd208160c80 2023-10-16T11:05:23.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.237+0000 7fd2077fe640 1 -- 172.21.15.84:0/1204594505 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc031ba0 con 0x7fd208160c80 2023-10-16T11:05:23.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.238+0000 7fd20da59640 1 -- 172.21.15.84:0/1204594505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd208160c80 msgr2=0x7fd208161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:23.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.238+0000 7fd20da59640 1 --2- 172.21.15.84:0/1204594505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd208160c80 0x7fd208161060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0097a0 tx=0x7fd1fc02b8b0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.238+0000 7fd20da59640 1 -- 172.21.15.84:0/1204594505 shutdown_connections 2023-10-16T11:05:23.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.238+0000 7fd20da59640 1 --2- 172.21.15.84:0/1204594505 >> v2:172.21.15.157:3300/0 conn(0x7fd208070950 0x7fd208140400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.238+0000 7fd20da59640 1 --2- 172.21.15.84:0/1204594505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd208160c80 0x7fd208161060 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.238+0000 7fd20da59640 1 -- 172.21.15.84:0/1204594505 >> 172.21.15.84:0/1204594505 conn(0x7fd2080cc230 msgr2=0x7fd2080cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:23.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.238+0000 7fd20da59640 1 -- 172.21.15.84:0/1204594505 shutdown_connections 2023-10-16T11:05:23.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.238+0000 7fd20da59640 1 -- 172.21.15.84:0/1204594505 wait complete. 2023-10-16T11:05:23.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.240+0000 7fd20da59640 1 Processor -- start 2023-10-16T11:05:23.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.240+0000 7fd20da59640 1 -- start start 2023-10-16T11:05:23.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.241+0000 7fd20da59640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd208070950 0x7fd20806c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:23.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.241+0000 7fd20da59640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2080ad7a0 0x7fd2080c2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:23.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.241+0000 7fd20da59640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd208160c80 0x7fd208069da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:23.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.241+0000 7fd20da59640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2081f2670 con 0x7fd208160c80 2023-10-16T11:05:23.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.241+0000 7fd20da59640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20814f5c0 con 0x7fd208070950 2023-10-16T11:05:23.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.241+0000 7fd20da59640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20806ce50 con 0x7fd2080ad7a0 2023-10-16T11:05:23.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.241+0000 7fd20d258640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd208160c80 0x7fd208069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:23.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.241+0000 7fd20d258640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd208160c80 0x7fd208069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36452/0 (socket says 172.21.15.84:36452) 2023-10-16T11:05:23.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.241+0000 7fd20d258640 1 -- 172.21.15.84:0/1044572669 learned_addr learned my addr 172.21.15.84:0/1044572669 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:23.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.242+0000 7fd20ca57640 1 --2- 172.21.15.84:0/1044572669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd208070950 0x7fd20806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:23.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.242+0000 7fd207fff640 1 --2- 172.21.15.84:0/1044572669 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2080ad7a0 0x7fd2080c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:23.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.242+0000 7fd20d258640 1 -- 172.21.15.84:0/1044572669 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2080ad7a0 msgr2=0x7fd2080c2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:23.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.242+0000 7fd20d258640 1 --2- 172.21.15.84:0/1044572669 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2080ad7a0 0x7fd2080c2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.242+0000 7fd20d258640 1 -- 172.21.15.84:0/1044572669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd208070950 msgr2=0x7fd20806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:23.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.242+0000 7fd20d258640 1 --2- 172.21.15.84:0/1044572669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd208070950 0x7fd20806c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.242+0000 7fd20d258640 1 -- 172.21.15.84:0/1044572669 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1fc009400 con 0x7fd208160c80 2023-10-16T11:05:23.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.242+0000 7fd20d258640 1 --2- 172.21.15.84:0/1044572669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd208160c80 0x7fd208069da0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fd20000ef90 tx=0x7fd20000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:23.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.242+0000 7fd2057fa640 1 -- 172.21.15.84:0/1044572669 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd200009ca0 con 0x7fd208160c80 2023-10-16T11:05:23.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.242+0000 7fd2057fa640 1 -- 172.21.15.84:0/1044572669 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20000f040 con 0x7fd208160c80 2023-10-16T11:05:23.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.242+0000 7fd2057fa640 1 -- 172.21.15.84:0/1044572669 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2000146c0 con 0x7fd208160c80 2023-10-16T11:05:23.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.242+0000 7fd20da59640 1 -- 172.21.15.84:0/1044572669 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd20806d130 con 0x7fd208160c80 2023-10-16T11:05:23.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.243+0000 7fd20da59640 1 -- 172.21.15.84:0/1044572669 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2081f2880 con 0x7fd208160c80 2023-10-16T11:05:23.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.244+0000 7fd20da59640 1 -- 172.21.15.84:0/1044572669 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd20806d650 con 0x7fd208160c80 2023-10-16T11:05:23.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.245+0000 7fd2057fa640 1 -- 172.21.15.84:0/1044572669 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 89) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fd200014820 con 0x7fd208160c80 2023-10-16T11:05:23.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.246+0000 7fd2057fa640 1 --2- 172.21.15.84:0/1044572669 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fd1ec06af90 0x7fd1ec06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:23.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.246+0000 7fd2057fa640 1 -- 172.21.15.84:0/1044572669 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6227+0+0 (secure 0 0 0) 0x7fd20008ae70 con 0x7fd208160c80 2023-10-16T11:05:23.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.246+0000 7fd20ca57640 1 -- 172.21.15.84:0/1044572669 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fd1ec06af90 msgr2=0x7fd1ec06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3746729142 2023-10-16T11:05:23.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.246+0000 7fd20ca57640 1 --2- 172.21.15.84:0/1044572669 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fd1ec06af90 0x7fd1ec06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:05:23.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.249+0000 7fd2057fa640 1 -- 172.21.15.84:0/1044572669 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd200055ad0 con 0x7fd208160c80 2023-10-16T11:05:23.254 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:23.252+0000 7f34c443e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:05:23.278 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:23.276+0000 7fcf18430040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:05:23.318 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:23.317+0000 7f34c443e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:05:23.343 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:23.342+0000 7fcf18430040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:05:23.368 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:23.367+0000 7f34c443e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:05:23.394 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:23.393+0000 7fcf18430040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:05:23.427 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:23.425+0000 7f34c443e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:05:23.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.434+0000 7fd20da59640 1 -- 172.21.15.84:0/1044572669 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd20806d650 con 0x7fd208160c80 2023-10-16T11:05:23.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.435+0000 7fd2057fa640 1 -- 172.21.15.84:0/1044572669 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v158) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd20806d650 con 0x7fd208160c80 2023-10-16T11:05:23.436 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:05:23.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.437+0000 7fd20da59640 1 -- 172.21.15.84:0/1044572669 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fd1ec06af90 msgr2=0x7fd1ec06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:05:23.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.437+0000 7fd20da59640 1 --2- 172.21.15.84:0/1044572669 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fd1ec06af90 0x7fd1ec06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.437+0000 7fd20da59640 1 -- 172.21.15.84:0/1044572669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd208160c80 msgr2=0x7fd208069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:23.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.437+0000 7fd20da59640 1 --2- 172.21.15.84:0/1044572669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd208160c80 0x7fd208069da0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fd20000ef90 tx=0x7fd20000c490 comp rx=0 tx=0).stop 2023-10-16T11:05:23.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.438+0000 7fd20da59640 1 -- 172.21.15.84:0/1044572669 shutdown_connections 2023-10-16T11:05:23.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.438+0000 7fd20da59640 1 --2- 172.21.15.84:0/1044572669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd208160c80 0x7fd208069da0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.438+0000 7fd20da59640 1 --2- 172.21.15.84:0/1044572669 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fd1ec06af90 0x7fd1ec06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.438+0000 7fd20da59640 1 --2- 172.21.15.84:0/1044572669 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2080ad7a0 0x7fd2080c2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.438+0000 7fd20da59640 1 --2- 172.21.15.84:0/1044572669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd208070950 0x7fd20806c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.438+0000 7fd20da59640 1 -- 172.21.15.84:0/1044572669 >> 172.21.15.84:0/1044572669 conn(0x7fd2080cc230 msgr2=0x7fd2080ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:23.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.438+0000 7fd20da59640 1 -- 172.21.15.84:0/1044572669 shutdown_connections 2023-10-16T11:05:23.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.438+0000 7fd20da59640 1 -- 172.21.15.84:0/1044572669 wait complete. 2023-10-16T11:05:23.454 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:23.453+0000 7fcf18430040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:05:23.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:05:23.476 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:23.476+0000 7f34c443e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:05:23.507 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:23.506+0000 7fcf18430040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:05:23.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.701+0000 7fb27826f640 1 Processor -- start 2023-10-16T11:05:23.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.702+0000 7fb27826f640 1 -- start start 2023-10-16T11:05:23.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.703+0000 7fb27826f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2700661c0 0x7fb2700ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:23.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.703+0000 7fb27826f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb27006cee0 con 0x7fb27015d330 2023-10-16T11:05:23.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.703+0000 7fb27826f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2700d0890 con 0x7fb2701618b0 2023-10-16T11:05:23.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.703+0000 7fb27826f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27006b1e0 con 0x7fb2700661c0 2023-10-16T11:05:23.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.703+0000 7fb27600b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb27015d330 0x7fb27006c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37162/0 (socket says 172.21.15.84:37162) 2023-10-16T11:05:23.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.703+0000 7fb27600b640 1 -- 172.21.15.84:0/3001620662 learned_addr learned my addr 172.21.15.84:0/3001620662 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:23.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.704+0000 7fb27680c640 1 --2- 172.21.15.84:0/3001620662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2700661c0 0x7fb2700ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:23.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.704+0000 7fb275009640 1 -- 172.21.15.84:0/3001620662 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895387632 0 0) 0x7fb27006cee0 con 0x7fb27015d330 2023-10-16T11:05:23.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.704+0000 7fb275009640 1 -- 172.21.15.84:0/3001620662 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb25c003520 con 0x7fb27015d330 2023-10-16T11:05:23.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.704+0000 7fb275009640 1 -- 172.21.15.84:0/3001620662 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2708362077 0 0) 0x7fb25c003520 con 0x7fb27015d330 2023-10-16T11:05:23.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.704+0000 7fb275009640 1 -- 172.21.15.84:0/3001620662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2700661c0 msgr2=0x7fb2700ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:23.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.704+0000 7fb275009640 1 --2- 172.21.15.84:0/3001620662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2700661c0 0x7fb2700ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.704+0000 7fb275009640 1 -- 172.21.15.84:0/3001620662 >> v1:172.21.15.157:6789/0 conn(0x7fb2701618b0 legacy=0x7fb270065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:23.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.704+0000 7fb275009640 1 -- 172.21.15.84:0/3001620662 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2700d05b0 con 0x7fb27015d330 2023-10-16T11:05:23.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.704+0000 7fb275009640 1 -- 172.21.15.84:0/3001620662 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fb26c001670 con 0x7fb27015d330 2023-10-16T11:05:23.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.704+0000 7fb275009640 1 -- 172.21.15.84:0/3001620662 >> v1:172.21.15.84:6789/0 conn(0x7fb27015d330 legacy=0x7fb27006c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:23.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.704+0000 7fb275009640 1 --2- 172.21.15.84:0/3001620662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb25c0039a0 0x7fb25c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:23.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.705+0000 7fb275009640 1 --2- 172.21.15.84:0/3001620662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2700661c0 0x7fb25c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:23.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.705+0000 7fb275009640 1 --2- 172.21.15.84:0/3001620662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c004d00 0x7fb25c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:23.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.705+0000 7fb275009640 1 -- 172.21.15.84:0/3001620662 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27006b1e0 con 0x7fb25c0039a0 2023-10-16T11:05:23.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.705+0000 7fb275009640 1 -- 172.21.15.84:0/3001620662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c00bae0 con 0x7fb25c004d00 2023-10-16T11:05:23.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.705+0000 7fb275009640 1 -- 172.21.15.84:0/3001620662 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c00bca0 con 0x7fb2700661c0 2023-10-16T11:05:23.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.705+0000 7fb27600b640 1 --2- 172.21.15.84:0/3001620662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb25c0039a0 0x7fb25c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:23.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.705+0000 7fb27600b640 1 -- 172.21.15.84:0/3001620662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2700661c0 msgr2=0x7fb25c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:23.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.705+0000 7fb27680c640 1 --2- 172.21.15.84:0/3001620662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c004d00 0x7fb25c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:23.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.705+0000 7fb27580a640 1 --2- 172.21.15.84:0/3001620662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2700661c0 0x7fb25c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:23.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.705+0000 7fb27600b640 1 --2- 172.21.15.84:0/3001620662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2700661c0 0x7fb25c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.705+0000 7fb27600b640 1 -- 172.21.15.84:0/3001620662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c004d00 msgr2=0x7fb25c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:23.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.705+0000 7fb27600b640 1 --2- 172.21.15.84:0/3001620662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c004d00 0x7fb25c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.705+0000 7fb27600b640 1 -- 172.21.15.84:0/3001620662 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c003520 con 0x7fb25c0039a0 2023-10-16T11:05:23.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.705+0000 7fb27680c640 1 --2- 172.21.15.84:0/3001620662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c004d00 0x7fb25c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:23.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.705+0000 7fb27580a640 1 --2- 172.21.15.84:0/3001620662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2700661c0 0x7fb25c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:23.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.706+0000 7fb27600b640 1 --2- 172.21.15.84:0/3001620662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb25c0039a0 0x7fb25c003dc0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00c240 tx=0x7fb25c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763e09646cbee615 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:23.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.706+0000 7fb275009640 1 -- 172.21.15.84:0/3001620662 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c005ad0 con 0x7fb25c0039a0 2023-10-16T11:05:23.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.706+0000 7fb275009640 1 -- 172.21.15.84:0/3001620662 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c005c30 con 0x7fb25c0039a0 2023-10-16T11:05:23.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.707+0000 7fb275009640 1 -- 172.21.15.84:0/3001620662 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c02c7f0 con 0x7fb25c0039a0 2023-10-16T11:05:23.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.707+0000 7fb27826f640 1 -- 172.21.15.84:0/3001620662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb25c0039a0 msgr2=0x7fb25c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:23.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.707+0000 7fb27826f640 1 --2- 172.21.15.84:0/3001620662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb25c0039a0 0x7fb25c003dc0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00c240 tx=0x7fb25c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.707+0000 7fb27680c640 1 -- 172.21.15.84:0/3001620662 reap_dead start 2023-10-16T11:05:23.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.707+0000 7fb27826f640 1 -- 172.21.15.84:0/3001620662 shutdown_connections 2023-10-16T11:05:23.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.707+0000 7fb27826f640 1 -- 172.21.15.84:0/3001620662 >> 172.21.15.84:0/3001620662 conn(0x7fb2700d0ab0 msgr2=0x7fb2700d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:23.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.707+0000 7fb27826f640 1 -- 172.21.15.84:0/3001620662 shutdown_connections 2023-10-16T11:05:23.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.707+0000 7fb27826f640 1 -- 172.21.15.84:0/3001620662 wait complete. 2023-10-16T11:05:23.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.709+0000 7fb27826f640 1 Processor -- start 2023-10-16T11:05:23.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.709+0000 7fb27826f640 1 -- start start 2023-10-16T11:05:23.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.710+0000 7fb27826f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c0039a0 0x7fb2700b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:23.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.710+0000 7fb27826f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2700b38b0 0x7fb270155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:23.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.710+0000 7fb27826f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2700b03b0 0x7fb2700b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:23.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.710+0000 7fb27826f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2700ca3f0 con 0x7fb2700b03b0 2023-10-16T11:05:23.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.710+0000 7fb27826f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2700b4620 con 0x7fb25c0039a0 2023-10-16T11:05:23.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.710+0000 7fb27826f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2700c9da0 con 0x7fb2700b38b0 2023-10-16T11:05:23.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.710+0000 7fb27680c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2700b03b0 0x7fb2700b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:23.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.710+0000 7fb27680c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2700b03b0 0x7fb2700b0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36462/0 (socket says 172.21.15.84:36462) 2023-10-16T11:05:23.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.710+0000 7fb27680c640 1 -- 172.21.15.84:0/800874725 learned_addr learned my addr 172.21.15.84:0/800874725 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:23.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.710+0000 7fb27580a640 1 --2- 172.21.15.84:0/800874725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2700b38b0 0x7fb270155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:23.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.710+0000 7fb27600b640 1 --2- 172.21.15.84:0/800874725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c0039a0 0x7fb2700b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:23.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.711+0000 7fb27680c640 1 -- 172.21.15.84:0/800874725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2700b38b0 msgr2=0x7fb270155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:23.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.711+0000 7fb27680c640 1 --2- 172.21.15.84:0/800874725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2700b38b0 0x7fb270155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.711+0000 7fb27680c640 1 -- 172.21.15.84:0/800874725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c0039a0 msgr2=0x7fb2700b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:23.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.711+0000 7fb27680c640 1 --2- 172.21.15.84:0/800874725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c0039a0 0x7fb2700b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.711+0000 7fb27680c640 1 -- 172.21.15.84:0/800874725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2700ca080 con 0x7fb2700b03b0 2023-10-16T11:05:23.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.711+0000 7fb27580a640 1 --2- 172.21.15.84:0/800874725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2700b38b0 0x7fb270155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:23.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.711+0000 7fb27600b640 1 --2- 172.21.15.84:0/800874725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c0039a0 0x7fb2700b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:23.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.711+0000 7fb27680c640 1 --2- 172.21.15.84:0/800874725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2700b03b0 0x7fb2700b0810 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fb2700afa70 tx=0x7fb26800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:23.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.711+0000 7fb266ffd640 1 -- 172.21.15.84:0/800874725 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268017400 con 0x7fb2700b03b0 2023-10-16T11:05:23.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.711+0000 7fb266ffd640 1 -- 172.21.15.84:0/800874725 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb268011040 con 0x7fb2700b03b0 2023-10-16T11:05:23.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.711+0000 7fb27826f640 1 -- 172.21.15.84:0/800874725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2700b3d30 con 0x7fb2700b03b0 2023-10-16T11:05:23.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.711+0000 7fb266ffd640 1 -- 172.21.15.84:0/800874725 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2680177c0 con 0x7fb2700b03b0 2023-10-16T11:05:23.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.711+0000 7fb27826f640 1 -- 172.21.15.84:0/800874725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2700ae610 con 0x7fb2700b03b0 2023-10-16T11:05:23.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.712+0000 7fb27826f640 1 -- 172.21.15.84:0/800874725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2700ca5a0 con 0x7fb2700b03b0 2023-10-16T11:05:23.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.714+0000 7fb266ffd640 1 -- 172.21.15.84:0/800874725 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 89) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fb2680203b0 con 0x7fb2700b03b0 2023-10-16T11:05:23.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.715+0000 7fb266ffd640 1 --2- 172.21.15.84:0/800874725 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fb25406afc0 0x7fb25406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:23.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.715+0000 7fb266ffd640 1 -- 172.21.15.84:0/800874725 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6227+0+0 (secure 0 0 0) 0x7fb268089f00 con 0x7fb2700b03b0 2023-10-16T11:05:23.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.715+0000 7fb27600b640 1 -- 172.21.15.84:0/800874725 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fb25406afc0 msgr2=0x7fb25406d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3746729142 2023-10-16T11:05:23.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.715+0000 7fb27600b640 1 --2- 172.21.15.84:0/800874725 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fb25406afc0 0x7fb25406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:05:23.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.722+0000 7fb266ffd640 1 -- 172.21.15.84:0/800874725 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb268054b60 con 0x7fb2700b03b0 2023-10-16T11:05:23.864 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:23.864+0000 7f34c443e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:05:23.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.906+0000 7fb27826f640 1 -- 172.21.15.84:0/800874725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb2700ca210 con 0x7fb2700b03b0 2023-10-16T11:05:23.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.906+0000 7fb266ffd640 1 -- 172.21.15.84:0/800874725 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v158) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb26801c020 con 0x7fb2700b03b0 2023-10-16T11:05:23.908 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:05:23.910 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:23.908+0000 7fcf18430040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:05:23.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.909+0000 7fb27826f640 1 -- 172.21.15.84:0/800874725 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fb25406afc0 msgr2=0x7fb25406d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:05:23.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.909+0000 7fb27826f640 1 --2- 172.21.15.84:0/800874725 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fb25406afc0 0x7fb25406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.909+0000 7fb27826f640 1 -- 172.21.15.84:0/800874725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2700b03b0 msgr2=0x7fb2700b0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:23.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.909+0000 7fb27826f640 1 --2- 172.21.15.84:0/800874725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2700b03b0 0x7fb2700b0810 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fb2700afa70 tx=0x7fb26800f4a0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.909+0000 7fb27826f640 1 -- 172.21.15.84:0/800874725 shutdown_connections 2023-10-16T11:05:23.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.909+0000 7fb27826f640 1 --2- 172.21.15.84:0/800874725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2700b03b0 0x7fb2700b0810 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.909+0000 7fb27826f640 1 --2- 172.21.15.84:0/800874725 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fb25406afc0 0x7fb25406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.909+0000 7fb27826f640 1 --2- 172.21.15.84:0/800874725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2700b38b0 0x7fb270155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.909+0000 7fb27826f640 1 --2- 172.21.15.84:0/800874725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c0039a0 0x7fb2700b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:23.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.909+0000 7fb27826f640 1 -- 172.21.15.84:0/800874725 >> 172.21.15.84:0/800874725 conn(0x7fb2700d0ab0 msgr2=0x7fb27015d710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:23.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.909+0000 7fb27826f640 1 -- 172.21.15.84:0/800874725 shutdown_connections 2023-10-16T11:05:23.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:23.909+0000 7fb27826f640 1 -- 172.21.15.84:0/800874725 wait complete. 2023-10-16T11:05:23.929 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T11:05:24.025 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:24.024+0000 7f34c443e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:05:24.076 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:24.074+0000 7fcf18430040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:05:24.132 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:24.132+0000 7f34c443e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:05:24.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.174+0000 7f6af088a640 1 Processor -- start 2023-10-16T11:05:24.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.174+0000 7f6af088a640 1 -- start start 2023-10-16T11:05:24.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.175+0000 7f6af088a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6aec159950 0x7f6aec15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:24.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.175+0000 7f6af088a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6aec06c810 0x7f6aec0708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:24.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.175+0000 7f6af088a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aec070df0 con 0x7f6aec070450 2023-10-16T11:05:24.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.175+0000 7f6af088a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aec15e490 con 0x7f6aec06c810 2023-10-16T11:05:24.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.175+0000 7f6af088a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec15aef0 con 0x7f6aec159950 2023-10-16T11:05:24.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.176+0000 7f6aebfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6aec06c810 0x7f6aec0708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:24.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.176+0000 7f6aeaffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6aec070450 0x7f6aec06c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37176/0 (socket says 172.21.15.84:37176) 2023-10-16T11:05:24.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.176+0000 7f6aeaffd640 1 -- 172.21.15.84:0/3671608863 learned_addr learned my addr 172.21.15.84:0/3671608863 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:24.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.176+0000 7f6aeb7fe640 1 --2- 172.21.15.84:0/3671608863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6aec159950 0x7f6aec15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:24.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.176+0000 7f6aea7fc640 1 -- 172.21.15.84:0/3671608863 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978762624 0 0) 0x7f6aec070df0 con 0x7f6aec070450 2023-10-16T11:05:24.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.176+0000 7f6aea7fc640 1 -- 172.21.15.84:0/3671608863 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ad4005730 con 0x7f6aec070450 2023-10-16T11:05:24.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.176+0000 7f6aebfff640 1 -- 172.21.15.84:0/3671608863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6aec159950 msgr2=0x7f6aec15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:24.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.176+0000 7f6aebfff640 1 --2- 172.21.15.84:0/3671608863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6aec159950 0x7f6aec15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:24.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.176+0000 7f6aebfff640 1 -- 172.21.15.84:0/3671608863 >> v1:172.21.15.84:6789/0 conn(0x7f6aec070450 legacy=0x7f6aec06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:24.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.176+0000 7f6aebfff640 1 -- 172.21.15.84:0/3671608863 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec06cc80 con 0x7f6aec06c810 2023-10-16T11:05:24.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.178+0000 7f6aebfff640 1 --2- 172.21.15.84:0/3671608863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6aec06c810 0x7f6aec0708b0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009d90 tx=0x7f6ae402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c482f0efd18065 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:24.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.178+0000 7f6aea7fc640 1 -- 172.21.15.84:0/3671608863 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae402cbc0 con 0x7f6aec06c810 2023-10-16T11:05:24.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.178+0000 7f6aea7fc640 1 -- 172.21.15.84:0/3671608863 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae402cd20 con 0x7f6aec06c810 2023-10-16T11:05:24.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.178+0000 7f6aea7fc640 1 -- 172.21.15.84:0/3671608863 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4034660 con 0x7f6aec06c810 2023-10-16T11:05:24.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.179+0000 7f6af088a640 1 -- 172.21.15.84:0/3671608863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6aec06c810 msgr2=0x7f6aec0708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:24.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.179+0000 7f6af088a640 1 --2- 172.21.15.84:0/3671608863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6aec06c810 0x7f6aec0708b0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009d90 tx=0x7f6ae402be10 comp rx=0 tx=0).stop 2023-10-16T11:05:24.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.179+0000 7f6af088a640 1 -- 172.21.15.84:0/3671608863 shutdown_connections 2023-10-16T11:05:24.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.179+0000 7f6af088a640 1 --2- 172.21.15.84:0/3671608863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6aec06c810 0x7f6aec0708b0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:24.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.179+0000 7f6af088a640 1 --2- 172.21.15.84:0/3671608863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6aec159950 0x7f6aec15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:24.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.179+0000 7f6af088a640 1 -- 172.21.15.84:0/3671608863 >> 172.21.15.84:0/3671608863 conn(0x7f6aec0ce6a0 msgr2=0x7f6aec0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:24.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.179+0000 7f6af088a640 1 -- 172.21.15.84:0/3671608863 shutdown_connections 2023-10-16T11:05:24.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.179+0000 7f6af088a640 1 -- 172.21.15.84:0/3671608863 wait complete. 2023-10-16T11:05:24.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.181+0000 7f6af088a640 1 Processor -- start 2023-10-16T11:05:24.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.181+0000 7f6af088a640 1 -- start start 2023-10-16T11:05:24.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.182+0000 7f6af088a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6aec06c810 0x7f6aec0b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:24.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.182+0000 7f6af088a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6aec070450 0x7f6aec0c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:24.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.182+0000 7f6af088a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aec159950 0x7f6aec0c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:24.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.182+0000 7f6af088a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0c50d0 con 0x7f6aec070450 2023-10-16T11:05:24.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.182+0000 7f6af088a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec156740 con 0x7f6aec159950 2023-10-16T11:05:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.182+0000 7f6af088a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec156a20 con 0x7f6aec06c810 2023-10-16T11:05:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.182+0000 7f6aeaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6aec070450 0x7f6aec0c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.182+0000 7f6aeaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6aec070450 0x7f6aec0c4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36472/0 (socket says 172.21.15.84:36472) 2023-10-16T11:05:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.182+0000 7f6aeaffd640 1 -- 172.21.15.84:0/1217998573 learned_addr learned my addr 172.21.15.84:0/1217998573 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:24.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.182+0000 7f6aeb7fe640 1 --2- 172.21.15.84:0/1217998573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6aec06c810 0x7f6aec0b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.182+0000 7f6aebfff640 1 --2- 172.21.15.84:0/1217998573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aec159950 0x7f6aec0c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.183+0000 7f6aeaffd640 1 -- 172.21.15.84:0/1217998573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6aec06c810 msgr2=0x7f6aec0b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.183+0000 7f6aeaffd640 1 --2- 172.21.15.84:0/1217998573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6aec06c810 0x7f6aec0b08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.183+0000 7f6aeaffd640 1 -- 172.21.15.84:0/1217998573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aec159950 msgr2=0x7f6aec0c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.183+0000 7f6aeaffd640 1 --2- 172.21.15.84:0/1217998573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aec159950 0x7f6aec0c21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.183+0000 7f6aeaffd640 1 -- 172.21.15.84:0/1217998573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae4009a70 con 0x7f6aec070450 2023-10-16T11:05:24.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.183+0000 7f6aebfff640 1 --2- 172.21.15.84:0/1217998573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aec159950 0x7f6aec0c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:24.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.183+0000 7f6aeaffd640 1 --2- 172.21.15.84:0/1217998573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6aec070450 0x7f6aec0c4b90 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c500 tx=0x7f6ae000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:24.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.183+0000 7f6ad3fff640 1 -- 172.21.15.84:0/1217998573 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0005ce0 con 0x7f6aec070450 2023-10-16T11:05:24.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.183+0000 7f6ad3fff640 1 -- 172.21.15.84:0/1217998573 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae0005e40 con 0x7f6aec070450 2023-10-16T11:05:24.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.183+0000 7f6ad3fff640 1 -- 172.21.15.84:0/1217998573 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0007330 con 0x7f6aec070450 2023-10-16T11:05:24.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.183+0000 7f6af088a640 1 -- 172.21.15.84:0/1217998573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aec156d00 con 0x7f6aec070450 2023-10-16T11:05:24.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.183+0000 7f6af088a640 1 -- 172.21.15.84:0/1217998573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec1572c0 con 0x7f6aec070450 2023-10-16T11:05:24.192 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:24.182+0000 7f34c443e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:05:24.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.185+0000 7f6af088a640 1 -- 172.21.15.84:0/1217998573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aec04e9b0 con 0x7f6aec070450 2023-10-16T11:05:24.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.185+0000 7f6ad3fff640 1 -- 172.21.15.84:0/1217998573 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 89) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f6ae0007550 con 0x7f6aec070450 2023-10-16T11:05:24.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.187+0000 7f6ad3fff640 1 --2- 172.21.15.84:0/1217998573 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f6ad806af90 0x7f6ad806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:24.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.187+0000 7f6aeb7fe640 1 -- 172.21.15.84:0/1217998573 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f6ad806af90 msgr2=0x7f6ad806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3746729142 2023-10-16T11:05:24.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.187+0000 7f6ad3fff640 1 -- 172.21.15.84:0/1217998573 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6227+0+0 (secure 0 0 0) 0x7f6ae0088ff0 con 0x7f6aec070450 2023-10-16T11:05:24.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.187+0000 7f6aeb7fe640 1 --2- 172.21.15.84:0/1217998573 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f6ad806af90 0x7f6ad806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:05:24.194 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:24.190+0000 7fcf18430040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:05:24.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.191+0000 7f6ad3fff640 1 -- 172.21.15.84:0/1217998573 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6ae0053e90 con 0x7f6aec070450 2023-10-16T11:05:24.265 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:24.263+0000 7fcf18430040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:05:24.294 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:24.293+0000 7f34c443e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:05:24.381 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:24.379+0000 7fcf18430040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:05:24.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.388+0000 7f6aeb7fe640 1 -- 172.21.15.84:0/1217998573 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f6ad806af90 msgr2=0x7f6ad806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3746729142 2023-10-16T11:05:24.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.388+0000 7f6aeb7fe640 1 --2- 172.21.15.84:0/1217998573 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f6ad806af90 0x7f6ad806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:05:24.406 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:24.405+0000 7f34c443e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:05:24.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.411+0000 7f6af088a640 1 -- 172.21.15.84:0/1217998573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6aec0b4620 con 0x7f6aec070450 2023-10-16T11:05:24.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.412+0000 7f6ad3fff640 1 -- 172.21.15.84:0/1217998573 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v158) v1 ==== 80+0+465 (secure 0 0 0) 0x7f6ae0017080 con 0x7f6aec070450 2023-10-16T11:05:24.414 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2439552742 2023-10-17T10:09:25.208422+0000 2023-10-16T11:05:24.414 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/384925658 2023-10-17T10:09:25.208422+0000 2023-10-16T11:05:24.414 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3023009161 2023-10-17T10:09:25.208422+0000 2023-10-16T11:05:24.414 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/870751622 2023-10-17T10:09:25.208422+0000 2023-10-16T11:05:24.415 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/300305981 2023-10-17T10:09:25.208422+0000 2023-10-16T11:05:24.415 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6833/205480762 2023-10-17T10:09:25.208422+0000 2023-10-16T11:05:24.415 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6832/205480762 2023-10-17T10:09:25.208422+0000 2023-10-16T11:05:24.415 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1358444053 2023-10-17T10:09:25.208422+0000 2023-10-16T11:05:24.418 INFO:teuthology.orchestra.run.smithi084.stderr:listed 8 entries 2023-10-16T11:05:24.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.414+0000 7f6af088a640 1 -- 172.21.15.84:0/1217998573 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f6ad806af90 msgr2=0x7f6ad806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:05:24.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.414+0000 7f6af088a640 1 --2- 172.21.15.84:0/1217998573 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f6ad806af90 0x7f6ad806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:24.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.414+0000 7f6af088a640 1 -- 172.21.15.84:0/1217998573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6aec070450 msgr2=0x7f6aec0c4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:24.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.414+0000 7f6af088a640 1 --2- 172.21.15.84:0/1217998573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6aec070450 0x7f6aec0c4b90 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c500 tx=0x7f6ae000c9c0 comp rx=0 tx=0).stop 2023-10-16T11:05:24.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.414+0000 7f6af088a640 1 -- 172.21.15.84:0/1217998573 shutdown_connections 2023-10-16T11:05:24.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.414+0000 7f6af088a640 1 --2- 172.21.15.84:0/1217998573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aec159950 0x7f6aec0c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:24.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.414+0000 7f6af088a640 1 --2- 172.21.15.84:0/1217998573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6aec070450 0x7f6aec0c4b90 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:24.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.414+0000 7f6af088a640 1 --2- 172.21.15.84:0/1217998573 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f6ad806af90 0x7f6ad806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:24.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.415+0000 7f6af088a640 1 --2- 172.21.15.84:0/1217998573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6aec06c810 0x7f6aec0b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:24.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.415+0000 7f6af088a640 1 -- 172.21.15.84:0/1217998573 >> 172.21.15.84:0/1217998573 conn(0x7f6aec0ce6a0 msgr2=0x7f6aec070890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:24.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.415+0000 7f6af088a640 1 -- 172.21.15.84:0/1217998573 shutdown_connections 2023-10-16T11:05:24.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.415+0000 7f6af088a640 1 -- 172.21.15.84:0/1217998573 wait complete. 2023-10-16T11:05:24.432 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T11:05:24.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:24.433 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T11:05:24.498 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:24.496+0000 7fcf18430040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:05:24.516 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T11:05:24.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:24.517 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T11:05:24.564 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:05:24.565 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:05:24.565 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:24.563+0000 7f34c443e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:05:24.617 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:24.616+0000 7f34c443e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:05:24.621 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:05:24.622 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:05:24.622 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:05:24.665 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:24.663+0000 7fcf18430040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:05:24.674 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:05:24.674 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:05:24.675 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:05:24.716 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:24.714+0000 7fcf18430040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:05:24.728 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:05:24.728 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:05:24.744 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:24.744+0000 7f34c443e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:05:24.841 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:24.840+0000 7fcf18430040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:05:24.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.925+0000 7f8550051640 1 Processor -- start 2023-10-16T11:05:24.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.925+0000 7f8550051640 1 -- start start 2023-10-16T11:05:24.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.926+0000 7f8550051640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8548159b20 0x7f854806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:24.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.926+0000 7f8550051640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85480b0b20 0x7f854806fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:24.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.926+0000 7f8550051640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854806f7f0 con 0x7f8548159f00 2023-10-16T11:05:24.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.926+0000 7f8550051640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f854806c5e0 con 0x7f8548159b20 2023-10-16T11:05:24.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.926+0000 7f8550051640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f854806c8c0 con 0x7f85480b0b20 2023-10-16T11:05:24.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.927+0000 7f854dded640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8548159b20 0x7f854806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:24.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.927+0000 7f854dded640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8548159b20 0x7f854806bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36486/0 (socket says 172.21.15.84:36486) 2023-10-16T11:05:24.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.927+0000 7f854dded640 1 -- 172.21.15.84:0/514868338 learned_addr learned my addr 172.21.15.84:0/514868338 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:24.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.927+0000 7f854e5ee640 1 --2- 172.21.15.84:0/514868338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85480b0b20 0x7f854806fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:24.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.927+0000 7f854dded640 1 -- 172.21.15.84:0/514868338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85480b0b20 msgr2=0x7f854806fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:24.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.927+0000 7f854dded640 1 --2- 172.21.15.84:0/514868338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85480b0b20 0x7f854806fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:24.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.927+0000 7f854dded640 1 -- 172.21.15.84:0/514868338 >> v1:172.21.15.84:6789/0 conn(0x7f8548159f00 legacy=0x7f85480b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:24.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.927+0000 7f854dded640 1 -- 172.21.15.84:0/514868338 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854806cba0 con 0x7f8548159b20 2023-10-16T11:05:24.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.929+0000 7f854dded640 1 --2- 172.21.15.84:0/514868338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8548159b20 0x7f854806bbe0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f8544009870 tx=0x7f854402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2417c65e99625b23 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:24.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.929+0000 7f854cdeb640 1 -- 172.21.15.84:0/514868338 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544030840 con 0x7f8548159b20 2023-10-16T11:05:24.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.929+0000 7f854cdeb640 1 -- 172.21.15.84:0/514868338 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85440309a0 con 0x7f8548159b20 2023-10-16T11:05:24.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.929+0000 7f854cdeb640 1 -- 172.21.15.84:0/514868338 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544030d30 con 0x7f8548159b20 2023-10-16T11:05:24.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.929+0000 7f8550051640 1 -- 172.21.15.84:0/514868338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8548159b20 msgr2=0x7f854806bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:24.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.929+0000 7f8550051640 1 --2- 172.21.15.84:0/514868338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8548159b20 0x7f854806bbe0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f8544009870 tx=0x7f854402b980 comp rx=0 tx=0).stop 2023-10-16T11:05:24.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.930+0000 7f8550051640 1 -- 172.21.15.84:0/514868338 shutdown_connections 2023-10-16T11:05:24.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.930+0000 7f8550051640 1 --2- 172.21.15.84:0/514868338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85480b0b20 0x7f854806fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:24.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.930+0000 7f8550051640 1 --2- 172.21.15.84:0/514868338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8548159b20 0x7f854806bbe0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:24.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.930+0000 7f8550051640 1 -- 172.21.15.84:0/514868338 >> 172.21.15.84:0/514868338 conn(0x7f85480cb410 msgr2=0x7f85480cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:24.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.930+0000 7f8550051640 1 -- 172.21.15.84:0/514868338 shutdown_connections 2023-10-16T11:05:24.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.930+0000 7f8550051640 1 -- 172.21.15.84:0/514868338 wait complete. 2023-10-16T11:05:24.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.931+0000 7f8550051640 1 Processor -- start 2023-10-16T11:05:24.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.931+0000 7f8550051640 1 -- start start 2023-10-16T11:05:24.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.932+0000 7f8550051640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85480b0b20 0x7f85481565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:24.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.932+0000 7f8550051640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8548159b20 0x7f854815c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:24.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.932+0000 7f8550051640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8548159f00 0x7f854815cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:24.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.932+0000 7f8550051640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854815d3c0 con 0x7f8548159b20 2023-10-16T11:05:24.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.932+0000 7f8550051640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854815b460 con 0x7f85480b0b20 2023-10-16T11:05:24.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.932+0000 7f8550051640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85480ad170 con 0x7f8548159f00 2023-10-16T11:05:24.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.933+0000 7f854d5ec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8548159b20 0x7f854815c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:24.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.933+0000 7f854d5ec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8548159b20 0x7f854815c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36492/0 (socket says 172.21.15.84:36492) 2023-10-16T11:05:24.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.933+0000 7f854d5ec640 1 -- 172.21.15.84:0/4104486848 learned_addr learned my addr 172.21.15.84:0/4104486848 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:24.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.933+0000 7f854e5ee640 1 --2- 172.21.15.84:0/4104486848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8548159f00 0x7f854815cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:24.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.933+0000 7f854dded640 1 --2- 172.21.15.84:0/4104486848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85480b0b20 0x7f85481565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:24.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.933+0000 7f854d5ec640 1 -- 172.21.15.84:0/4104486848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8548159f00 msgr2=0x7f854815cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:24.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.933+0000 7f854d5ec640 1 --2- 172.21.15.84:0/4104486848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8548159f00 0x7f854815cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:24.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.933+0000 7f854d5ec640 1 -- 172.21.15.84:0/4104486848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85480b0b20 msgr2=0x7f85481565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:24.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.933+0000 7f854d5ec640 1 --2- 172.21.15.84:0/4104486848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85480b0b20 0x7f85481565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:24.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.933+0000 7f854d5ec640 1 -- 172.21.15.84:0/4104486848 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85440094d0 con 0x7f8548159b20 2023-10-16T11:05:24.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.934+0000 7f854d5ec640 1 --2- 172.21.15.84:0/4104486848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8548159b20 0x7f854815c860 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f853800c400 tx=0x7f853800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:24.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.934+0000 7f853e7fc640 1 -- 172.21.15.84:0/4104486848 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85380052d0 con 0x7f8548159b20 2023-10-16T11:05:24.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.934+0000 7f853e7fc640 1 -- 172.21.15.84:0/4104486848 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8538005430 con 0x7f8548159b20 2023-10-16T11:05:24.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.934+0000 7f853e7fc640 1 -- 172.21.15.84:0/4104486848 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538003830 con 0x7f8548159b20 2023-10-16T11:05:24.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.934+0000 7f8550051640 1 -- 172.21.15.84:0/4104486848 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85480ad450 con 0x7f8548159b20 2023-10-16T11:05:24.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.934+0000 7f8550051640 1 -- 172.21.15.84:0/4104486848 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85481571f0 con 0x7f8548159b20 2023-10-16T11:05:24.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.935+0000 7f8550051640 1 -- 172.21.15.84:0/4104486848 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f854804e160 con 0x7f8548159b20 2023-10-16T11:05:24.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.937+0000 7f853e7fc640 1 -- 172.21.15.84:0/4104486848 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 89) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f8538003990 con 0x7f8548159b20 2023-10-16T11:05:24.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.938+0000 7f853e7fc640 1 --2- 172.21.15.84:0/4104486848 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f852c06af90 0x7f852c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:24.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.938+0000 7f853e7fc640 1 -- 172.21.15.84:0/4104486848 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6227+0+0 (secure 0 0 0) 0x7f8538088f70 con 0x7f8548159b20 2023-10-16T11:05:24.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.938+0000 7f854dded640 1 -- 172.21.15.84:0/4104486848 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f852c06af90 msgr2=0x7f852c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3746729142 2023-10-16T11:05:24.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.938+0000 7f854dded640 1 --2- 172.21.15.84:0/4104486848 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f852c06af90 0x7f852c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:05:24.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:24.942+0000 7f853e7fc640 1 -- 172.21.15.84:0/4104486848 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f85380bbfe0 con 0x7f8548159b20 2023-10-16T11:05:25.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.139+0000 7f854dded640 1 -- 172.21.15.84:0/4104486848 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f852c06af90 msgr2=0x7f852c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3746729142 2023-10-16T11:05:25.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.139+0000 7f854dded640 1 --2- 172.21.15.84:0/4104486848 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f852c06af90 0x7f852c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:05:25.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.158+0000 7f8550051640 1 -- 172.21.15.84:0/4104486848 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85480c9610 con 0x7f8548159b20 2023-10-16T11:05:25.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.159+0000 7f853e7fc640 1 -- 172.21.15.84:0/4104486848 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f8538017090 con 0x7f8548159b20 2023-10-16T11:05:25.161 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:25.161 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6517,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1733540616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1733540616},{"type":"v1","addr":"172.21.15.157:6833","nonce":1733540616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3815048417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3815048417},{"type":"v1","addr":"172.21.15.157:6835","nonce":3815048417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9135,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1038548935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1038548935},{"type":"v1","addr":"172.21.15.84:6836","nonce":1038548935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:09:10.652604+0000","modified":"2023-10-16T10:09:13.730838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9138},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9138":{"gid":9138,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2797149326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2797149326},{"type":"v1","addr":"172.21.15.84:6837","nonce":2797149326}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-16T11:05:25.161 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 80 2023-10-16T11:05:25.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.162+0000 7f8550051640 1 -- 172.21.15.84:0/4104486848 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f852c06af90 msgr2=0x7f852c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:05:25.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.162+0000 7f8550051640 1 --2- 172.21.15.84:0/4104486848 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f852c06af90 0x7f852c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.162+0000 7f8550051640 1 -- 172.21.15.84:0/4104486848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8548159b20 msgr2=0x7f854815c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:25.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.162+0000 7f8550051640 1 --2- 172.21.15.84:0/4104486848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8548159b20 0x7f854815c860 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f853800c400 tx=0x7f853800c8c0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.162+0000 7f8550051640 1 -- 172.21.15.84:0/4104486848 shutdown_connections 2023-10-16T11:05:25.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.162+0000 7f8550051640 1 --2- 172.21.15.84:0/4104486848 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f852c06af90 0x7f852c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.162+0000 7f8550051640 1 --2- 172.21.15.84:0/4104486848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8548159f00 0x7f854815cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.162+0000 7f8550051640 1 --2- 172.21.15.84:0/4104486848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8548159b20 0x7f854815c860 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.162+0000 7f8550051640 1 --2- 172.21.15.84:0/4104486848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85480b0b20 0x7f85481565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.162+0000 7f8550051640 1 -- 172.21.15.84:0/4104486848 >> 172.21.15.84:0/4104486848 conn(0x7f85480cb410 msgr2=0x7f85480ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:25.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.162+0000 7f8550051640 1 -- 172.21.15.84:0/4104486848 shutdown_connections 2023-10-16T11:05:25.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.162+0000 7f8550051640 1 -- 172.21.15.84:0/4104486848 wait complete. 2023-10-16T11:05:25.181 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:05:25.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.369+0000 7f61f2341640 1 Processor -- start 2023-10-16T11:05:25.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.369+0000 7f61f2341640 1 -- start start 2023-10-16T11:05:25.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.370+0000 7f61f2341640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61ec159b20 0x7f61ec06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:25.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.370+0000 7f61f2341640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ec06f7f0 con 0x7f61ec0b0b20 2023-10-16T11:05:25.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.371+0000 7f61f2341640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ec06c5e0 con 0x7f61ec159f00 2023-10-16T11:05:25.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.371+0000 7f61f2341640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec06c8c0 con 0x7f61ec159b20 2023-10-16T11:05:25.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.371+0000 7f61f08de640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f61ec0b0b20 0x7f61ec06fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37184/0 (socket says 172.21.15.84:37184) 2023-10-16T11:05:25.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.371+0000 7f61f08de640 1 -- 172.21.15.84:0/642175910 learned_addr learned my addr 172.21.15.84:0/642175910 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:25.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.371+0000 7f61ebfff640 1 --2- 172.21.15.84:0/642175910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61ec159b20 0x7f61ec06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:25.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.371+0000 7f61eaffd640 1 -- 172.21.15.84:0/642175910 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369980322 0 0) 0x7f61ec06f7f0 con 0x7f61ec0b0b20 2023-10-16T11:05:25.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.371+0000 7f61eaffd640 1 -- 172.21.15.84:0/642175910 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61d4003520 con 0x7f61ec0b0b20 2023-10-16T11:05:25.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.372+0000 7f61eaffd640 1 -- 172.21.15.84:0/642175910 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2210483999 0 0) 0x7f61d4003520 con 0x7f61ec0b0b20 2023-10-16T11:05:25.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.372+0000 7f61eaffd640 1 -- 172.21.15.84:0/642175910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61ec159b20 msgr2=0x7f61ec06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:25.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.372+0000 7f61eaffd640 1 --2- 172.21.15.84:0/642175910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61ec159b20 0x7f61ec06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.372+0000 7f61eaffd640 1 -- 172.21.15.84:0/642175910 >> v1:172.21.15.157:6789/0 conn(0x7f61ec159f00 legacy=0x7f61ec0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:25.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.372+0000 7f61eaffd640 1 -- 172.21.15.84:0/642175910 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec06cba0 con 0x7f61ec0b0b20 2023-10-16T11:05:25.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.372+0000 7f61eaffd640 1 -- 172.21.15.84:0/642175910 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f61e40019f0 con 0x7f61ec0b0b20 2023-10-16T11:05:25.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.372+0000 7f61eaffd640 1 -- 172.21.15.84:0/642175910 >> v1:172.21.15.84:6789/0 conn(0x7f61ec0b0b20 legacy=0x7f61ec06fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:25.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.372+0000 7f61eaffd640 1 --2- 172.21.15.84:0/642175910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61d40039a0 0x7f61d4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:25.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.372+0000 7f61eaffd640 1 --2- 172.21.15.84:0/642175910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61ec159b20 0x7f61d40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:25.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.372+0000 7f61eaffd640 1 --2- 172.21.15.84:0/642175910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61d4004d00 0x7f61d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:25.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.372+0000 7f61eaffd640 1 -- 172.21.15.84:0/642175910 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec06c8c0 con 0x7f61d4004d00 2023-10-16T11:05:25.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.372+0000 7f61eaffd640 1 -- 172.21.15.84:0/642175910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d400dc70 con 0x7f61d40039a0 2023-10-16T11:05:25.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.372+0000 7f61eaffd640 1 -- 172.21.15.84:0/642175910 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d400de30 con 0x7f61ec159b20 2023-10-16T11:05:25.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.372+0000 7f61f08de640 1 --2- 172.21.15.84:0/642175910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61d4004d00 0x7f61d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:25.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.372+0000 7f61ebfff640 1 --2- 172.21.15.84:0/642175910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61d40039a0 0x7f61d4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:25.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.373+0000 7f61eb7fe640 1 --2- 172.21.15.84:0/642175910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61ec159b20 0x7f61d40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:25.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.373+0000 7f61f08de640 1 -- 172.21.15.84:0/642175910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61ec159b20 msgr2=0x7f61d40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:25.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.373+0000 7f61f08de640 1 --2- 172.21.15.84:0/642175910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61ec159b20 0x7f61d40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.373+0000 7f61f08de640 1 -- 172.21.15.84:0/642175910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61d40039a0 msgr2=0x7f61d4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:25.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.373+0000 7f61f08de640 1 --2- 172.21.15.84:0/642175910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61d40039a0 0x7f61d4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.373+0000 7f61f08de640 1 -- 172.21.15.84:0/642175910 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d4003520 con 0x7f61d4004d00 2023-10-16T11:05:25.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.374+0000 7f61f08de640 1 --2- 172.21.15.84:0/642175910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61d4004d00 0x7f61d4005180 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f61e4002230 tx=0x7f61d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193dfd8c84aa8ba8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:25.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.374+0000 7f61eaffd640 1 -- 172.21.15.84:0/642175910 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e4034630 con 0x7f61d4004d00 2023-10-16T11:05:25.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.374+0000 7f61eaffd640 1 -- 172.21.15.84:0/642175910 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e402f070 con 0x7f61d4004d00 2023-10-16T11:05:25.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.374+0000 7f61eaffd640 1 -- 172.21.15.84:0/642175910 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e4034a10 con 0x7f61d4004d00 2023-10-16T11:05:25.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.375+0000 7f61f2341640 1 -- 172.21.15.84:0/642175910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61d4004d00 msgr2=0x7f61d4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:25.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.375+0000 7f61f2341640 1 --2- 172.21.15.84:0/642175910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61d4004d00 0x7f61d4005180 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f61e4002230 tx=0x7f61d40057f0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.375+0000 7f61f08de640 1 -- 172.21.15.84:0/642175910 reap_dead start 2023-10-16T11:05:25.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.375+0000 7f61f2341640 1 -- 172.21.15.84:0/642175910 shutdown_connections 2023-10-16T11:05:25.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.375+0000 7f61f2341640 1 -- 172.21.15.84:0/642175910 >> 172.21.15.84:0/642175910 conn(0x7f61ec0cb410 msgr2=0x7f61ec0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:25.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.375+0000 7f61f2341640 1 -- 172.21.15.84:0/642175910 shutdown_connections 2023-10-16T11:05:25.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.375+0000 7f61f2341640 1 -- 172.21.15.84:0/642175910 wait complete. 2023-10-16T11:05:25.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.377+0000 7f61f2341640 1 Processor -- start 2023-10-16T11:05:25.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.377+0000 7f61f2341640 1 -- start start 2023-10-16T11:05:25.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.378+0000 7f61f2341640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61d4004d00 0x7f61ec0b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:25.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.378+0000 7f61f2341640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61ec0c5a60 0x7f61ec0c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:25.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.378+0000 7f61f2341640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61ec0c7c40 0x7f61ec0c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:25.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.378+0000 7f61f2341640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec151080 con 0x7f61ec0c7c40 2023-10-16T11:05:25.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.378+0000 7f61f2341640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec15bd70 con 0x7f61d4004d00 2023-10-16T11:05:25.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.378+0000 7f61f2341640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec0c8600 con 0x7f61ec0c5a60 2023-10-16T11:05:25.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.378+0000 7f61f08de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61ec0c7c40 0x7f61ec0c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:25.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.378+0000 7f61f08de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61ec0c7c40 0x7f61ec0c80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36500/0 (socket says 172.21.15.84:36500) 2023-10-16T11:05:25.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.378+0000 7f61f08de640 1 -- 172.21.15.84:0/101702317 learned_addr learned my addr 172.21.15.84:0/101702317 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:25.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.378+0000 7f61ebfff640 1 --2- 172.21.15.84:0/101702317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61d4004d00 0x7f61ec0b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:25.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.378+0000 7f61eb7fe640 1 --2- 172.21.15.84:0/101702317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61ec0c5a60 0x7f61ec0c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:25.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.378+0000 7f61f08de640 1 -- 172.21.15.84:0/101702317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61ec0c5a60 msgr2=0x7f61ec0c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:25.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.378+0000 7f61f08de640 1 --2- 172.21.15.84:0/101702317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61ec0c5a60 0x7f61ec0c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.378+0000 7f61f08de640 1 -- 172.21.15.84:0/101702317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61d4004d00 msgr2=0x7f61ec0b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:25.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.378+0000 7f61f08de640 1 --2- 172.21.15.84:0/101702317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61d4004d00 0x7f61ec0b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.378+0000 7f61f08de640 1 -- 172.21.15.84:0/101702317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec1f29a0 con 0x7f61ec0c7c40 2023-10-16T11:05:25.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.379+0000 7f61f08de640 1 --2- 172.21.15.84:0/101702317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61ec0c7c40 0x7f61ec0c80c0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f61e402ca10 tx=0x7f61e4001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:25.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.379+0000 7f61e8ff9640 1 -- 172.21.15.84:0/101702317 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e4001270 con 0x7f61ec0c7c40 2023-10-16T11:05:25.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.379+0000 7f61e8ff9640 1 -- 172.21.15.84:0/101702317 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e4034640 con 0x7f61ec0c7c40 2023-10-16T11:05:25.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.379+0000 7f61f2341640 1 -- 172.21.15.84:0/101702317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ec0c6020 con 0x7f61ec0c7c40 2023-10-16T11:05:25.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.379+0000 7f61e8ff9640 1 -- 172.21.15.84:0/101702317 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e40349b0 con 0x7f61ec0c7c40 2023-10-16T11:05:25.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.379+0000 7f61f2341640 1 -- 172.21.15.84:0/101702317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ec15c2a0 con 0x7f61ec0c7c40 2023-10-16T11:05:25.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.380+0000 7f61f2341640 1 -- 172.21.15.84:0/101702317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ec06ce60 con 0x7f61ec0c7c40 2023-10-16T11:05:25.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.381+0000 7f61e8ff9640 1 -- 172.21.15.84:0/101702317 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 89) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f61e4031030 con 0x7f61ec0c7c40 2023-10-16T11:05:25.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.382+0000 7f61e8ff9640 1 --2- 172.21.15.84:0/101702317 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f61d806afc0 0x7f61d806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:25.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.382+0000 7f61e8ff9640 1 -- 172.21.15.84:0/101702317 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6227+0+0 (secure 0 0 0) 0x7f61e40ac0c0 con 0x7f61ec0c7c40 2023-10-16T11:05:25.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.382+0000 7f61ebfff640 1 -- 172.21.15.84:0/101702317 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f61d806afc0 msgr2=0x7f61d806d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3746729142 2023-10-16T11:05:25.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.382+0000 7f61ebfff640 1 --2- 172.21.15.84:0/101702317 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f61d806afc0 0x7f61d806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:05:25.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.390+0000 7f61e8ff9640 1 -- 172.21.15.84:0/101702317 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f61ec06ce60 con 0x7f61ec0c7c40 2023-10-16T11:05:25.561 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:25.560+0000 7f34c443e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:05:25.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.583+0000 7f61ebfff640 1 -- 172.21.15.84:0/101702317 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f61d806afc0 msgr2=0x7f61d806d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3746729142 2023-10-16T11:05:25.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.583+0000 7f61ebfff640 1 --2- 172.21.15.84:0/101702317 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f61d806afc0 0x7f61d806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:05:25.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.584+0000 7f61f2341640 1 -- 172.21.15.84:0/101702317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61ec06ce60 con 0x7f61ec0c7c40 2023-10-16T11:05:25.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.587+0000 7f61e8ff9640 1 -- 172.21.15.84:0/101702317 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+17303 (secure 0 0 0) 0x7f61ec06ce60 con 0x7f61ec0c7c40 2023-10-16T11:05:25.589 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:25.590 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":158,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:05:22.602329+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:09:08.490425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-16T10:09:09.666051+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":158,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.26","mappings":[{"from":2,"to":0}]},{"pgid":"19.3c","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2439552742":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/384925658":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/3023009161":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/870751622":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/300305981":"2023-10-17T10:09:25.208422+0000","172.21.15.84:6833/205480762":"2023-10-17T10:09:25.208422+0000","172.21.15.84:6832/205480762":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/1358444053":"2023-10-17T10:09:25.208422+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:05:25.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.590+0000 7f61f2341640 1 -- 172.21.15.84:0/101702317 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f61d806afc0 msgr2=0x7f61d806d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:05:25.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.590+0000 7f61f2341640 1 --2- 172.21.15.84:0/101702317 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f61d806afc0 0x7f61d806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.590+0000 7f61f2341640 1 -- 172.21.15.84:0/101702317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61ec0c7c40 msgr2=0x7f61ec0c80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:25.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.590+0000 7f61f2341640 1 --2- 172.21.15.84:0/101702317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61ec0c7c40 0x7f61ec0c80c0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f61e402ca10 tx=0x7f61e4001cb0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.590+0000 7f61f2341640 1 -- 172.21.15.84:0/101702317 shutdown_connections 2023-10-16T11:05:25.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.590+0000 7f61f2341640 1 --2- 172.21.15.84:0/101702317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61ec0c7c40 0x7f61ec0c80c0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.590+0000 7f61f2341640 1 --2- 172.21.15.84:0/101702317 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7f61d806afc0 0x7f61d806d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.590+0000 7f61f2341640 1 --2- 172.21.15.84:0/101702317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61ec0c5a60 0x7f61ec0c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.590+0000 7f61f2341640 1 --2- 172.21.15.84:0/101702317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61d4004d00 0x7f61ec0b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.590+0000 7f61f2341640 1 -- 172.21.15.84:0/101702317 >> 172.21.15.84:0/101702317 conn(0x7f61ec0cb410 msgr2=0x7f61ec0cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:25.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.590+0000 7f61f2341640 1 -- 172.21.15.84:0/101702317 shutdown_connections 2023-10-16T11:05:25.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.590+0000 7f61f2341640 1 -- 172.21.15.84:0/101702317 wait complete. 2023-10-16T11:05:25.608 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:09:08.490425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:09:09.666051+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 113, 'snap_epoch': 158, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:05:25.609 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:05:25.656 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:25.655+0000 7fcf18430040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:05:25.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.794+0000 7fb52ede9640 1 Processor -- start 2023-10-16T11:05:25.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.795+0000 7fb52ede9640 1 -- start start 2023-10-16T11:05:25.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.796+0000 7fb52ede9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb5280b2080 0x7fb5280d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:25.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.796+0000 7fb52ede9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5280d0970 con 0x7fb5280cfc40 2023-10-16T11:05:25.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.796+0000 7fb52ede9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5280d0ea0 con 0x7fb5280b1ca0 2023-10-16T11:05:25.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.796+0000 7fb52ede9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5280d0bc0 con 0x7fb5280b2080 2023-10-16T11:05:25.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.796+0000 7fb52d5e6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb5280b2080 0x7fb5280d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:25.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.796+0000 7fb52d5e6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb5280b2080 0x7fb5280d36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48466/0 (socket says 172.21.15.84:48466) 2023-10-16T11:05:25.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.796+0000 7fb52e5e8640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb5280cfc40 0x7fb5280b24c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37790/0 (socket says 172.21.15.84:37790) 2023-10-16T11:05:25.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.796+0000 7fb52d5e6640 1 -- 172.21.15.84:0/2523058440 learned_addr learned my addr 172.21.15.84:0/2523058440 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:25.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.797+0000 7fb52d5e6640 1 -- 172.21.15.84:0/2523058440 >> v1:172.21.15.157:6789/0 conn(0x7fb5280b1ca0 legacy=0x7fb5280ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:25.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.797+0000 7fb52d5e6640 1 -- 172.21.15.84:0/2523058440 >> v1:172.21.15.84:6789/0 conn(0x7fb5280cfc40 legacy=0x7fb5280b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:25.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.797+0000 7fb52d5e6640 1 -- 172.21.15.84:0/2523058440 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5280cd940 con 0x7fb5280b2080 2023-10-16T11:05:25.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.798+0000 7fb52d5e6640 1 --2- 172.21.15.84:0/2523058440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5280b2080 0x7fb5280d36b0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fb518009870 tx=0x7fb51802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6569bc486beb36ba server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:25.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.798+0000 7fb52cde5640 1 -- 172.21.15.84:0/2523058440 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51802f850 con 0x7fb5280b2080 2023-10-16T11:05:25.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.798+0000 7fb52cde5640 1 -- 172.21.15.84:0/2523058440 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51802f9b0 con 0x7fb5280b2080 2023-10-16T11:05:25.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.798+0000 7fb52cde5640 1 -- 172.21.15.84:0/2523058440 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51802fd40 con 0x7fb5280b2080 2023-10-16T11:05:25.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.799+0000 7fb52ede9640 1 -- 172.21.15.84:0/2523058440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5280b2080 msgr2=0x7fb5280d36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:25.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.799+0000 7fb52ede9640 1 --2- 172.21.15.84:0/2523058440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5280b2080 0x7fb5280d36b0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fb518009870 tx=0x7fb51802ba30 comp rx=0 tx=0).stop 2023-10-16T11:05:25.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.799+0000 7fb52ede9640 1 -- 172.21.15.84:0/2523058440 shutdown_connections 2023-10-16T11:05:25.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.799+0000 7fb52ede9640 1 --2- 172.21.15.84:0/2523058440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5280b2080 0x7fb5280d36b0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.799+0000 7fb52ede9640 1 -- 172.21.15.84:0/2523058440 >> 172.21.15.84:0/2523058440 conn(0x7fb5280d2b90 msgr2=0x7fb5280d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:25.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.799+0000 7fb52ede9640 1 -- 172.21.15.84:0/2523058440 shutdown_connections 2023-10-16T11:05:25.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.799+0000 7fb52ede9640 1 -- 172.21.15.84:0/2523058440 wait complete. 2023-10-16T11:05:25.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.801+0000 7fb52ede9640 1 Processor -- start 2023-10-16T11:05:25.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.801+0000 7fb52ede9640 1 -- start start 2023-10-16T11:05:25.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.802+0000 7fb52ede9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5280b1ca0 0x7fb528155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:25.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.802+0000 7fb52ede9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5280b2080 0x7fb5280c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:25.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.802+0000 7fb52ede9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5280cfc40 0x7fb5280c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:25.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.802+0000 7fb52ede9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb528155840 con 0x7fb5280b1ca0 2023-10-16T11:05:25.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.802+0000 7fb52ede9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5280cac70 con 0x7fb5280b2080 2023-10-16T11:05:25.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.802+0000 7fb52ede9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5280caf50 con 0x7fb5280cfc40 2023-10-16T11:05:25.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.802+0000 7fb52dde7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5280b1ca0 0x7fb528155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:25.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.802+0000 7fb52dde7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5280b1ca0 0x7fb528155300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48472/0 (socket says 172.21.15.84:48472) 2023-10-16T11:05:25.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.802+0000 7fb52dde7640 1 -- 172.21.15.84:0/2017487828 learned_addr learned my addr 172.21.15.84:0/2017487828 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:25.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.802+0000 7fb52d5e6640 1 --2- 172.21.15.84:0/2017487828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5280b2080 0x7fb5280c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:25.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.802+0000 7fb52dde7640 1 -- 172.21.15.84:0/2017487828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5280cfc40 msgr2=0x7fb5280c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:25.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.802+0000 7fb52dde7640 1 --2- 172.21.15.84:0/2017487828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5280cfc40 0x7fb5280c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.803+0000 7fb52dde7640 1 -- 172.21.15.84:0/2017487828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5280b2080 msgr2=0x7fb5280c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:25.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.803+0000 7fb52dde7640 1 --2- 172.21.15.84:0/2017487828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5280b2080 0x7fb5280c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.803+0000 7fb52dde7640 1 -- 172.21.15.84:0/2017487828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5180094d0 con 0x7fb5280b1ca0 2023-10-16T11:05:25.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.803+0000 7fb52d5e6640 1 --2- 172.21.15.84:0/2017487828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5280b2080 0x7fb5280c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:25.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.803+0000 7fb52dde7640 1 --2- 172.21.15.84:0/2017487828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5280b1ca0 0x7fb528155300 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fb52400b3f0 tx=0x7fb52400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:25.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.803+0000 7fb51e7fc640 1 -- 172.21.15.84:0/2017487828 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb524011830 con 0x7fb5280b1ca0 2023-10-16T11:05:25.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.803+0000 7fb51e7fc640 1 -- 172.21.15.84:0/2017487828 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb524011990 con 0x7fb5280b1ca0 2023-10-16T11:05:25.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.803+0000 7fb51e7fc640 1 -- 172.21.15.84:0/2017487828 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb524011d40 con 0x7fb5280b1ca0 2023-10-16T11:05:25.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.803+0000 7fb52ede9640 1 -- 172.21.15.84:0/2017487828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5280cb230 con 0x7fb5280b1ca0 2023-10-16T11:05:25.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.803+0000 7fb52ede9640 1 -- 172.21.15.84:0/2017487828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5280c9f00 con 0x7fb5280b1ca0 2023-10-16T11:05:25.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.804+0000 7fb52ede9640 1 -- 172.21.15.84:0/2017487828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52804e9b0 con 0x7fb5280b1ca0 2023-10-16T11:05:25.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.806+0000 7fb51e7fc640 1 -- 172.21.15.84:0/2017487828 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 89) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fb52401a400 con 0x7fb5280b1ca0 2023-10-16T11:05:25.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.808+0000 7fb51e7fc640 1 --2- 172.21.15.84:0/2017487828 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fb50c06af90 0x7fb50c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:25.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.808+0000 7fb52d5e6640 1 -- 172.21.15.84:0/2017487828 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fb50c06af90 msgr2=0x7fb50c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3746729142 2023-10-16T11:05:25.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.808+0000 7fb51e7fc640 1 -- 172.21.15.84:0/2017487828 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6227+0+0 (secure 0 0 0) 0x7fb524089750 con 0x7fb5280b1ca0 2023-10-16T11:05:25.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.808+0000 7fb52d5e6640 1 --2- 172.21.15.84:0/2017487828 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fb50c06af90 0x7fb50c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:05:25.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.812+0000 7fb51e7fc640 1 -- 172.21.15.84:0/2017487828 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb524054570 con 0x7fb5280b1ca0 2023-10-16T11:05:25.860 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:25.860+0000 7f34c443e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:05:25.917 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:25.916+0000 7f34c443e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:05:25.963 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:25.961+0000 7fcf18430040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:05:25.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.992+0000 7fb52ede9640 1 -- 172.21.15.84:0/2017487828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb528073a40 con 0x7fb5280b1ca0 2023-10-16T11:05:25.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.993+0000 7fb51e7fc640 1 -- 172.21.15.84:0/2017487828 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb524057b90 con 0x7fb5280b1ca0 2023-10-16T11:05:25.995 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:05:25.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.996+0000 7fb52ede9640 1 -- 172.21.15.84:0/2017487828 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fb50c06af90 msgr2=0x7fb50c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:05:25.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.996+0000 7fb52ede9640 1 --2- 172.21.15.84:0/2017487828 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fb50c06af90 0x7fb50c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.996+0000 7fb52ede9640 1 -- 172.21.15.84:0/2017487828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5280b1ca0 msgr2=0x7fb528155300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:25.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.996+0000 7fb52ede9640 1 --2- 172.21.15.84:0/2017487828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5280b1ca0 0x7fb528155300 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fb52400b3f0 tx=0x7fb52400b8b0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.996+0000 7fb52ede9640 1 -- 172.21.15.84:0/2017487828 shutdown_connections 2023-10-16T11:05:25.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.996+0000 7fb52ede9640 1 --2- 172.21.15.84:0/2017487828 >> [v2:172.21.15.84:6832/3746729142,v1:172.21.15.84:6833/3746729142] conn(0x7fb50c06af90 0x7fb50c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.996+0000 7fb52ede9640 1 --2- 172.21.15.84:0/2017487828 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5280cfc40 0x7fb5280c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.996+0000 7fb52ede9640 1 --2- 172.21.15.84:0/2017487828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5280b2080 0x7fb5280c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.996+0000 7fb52ede9640 1 --2- 172.21.15.84:0/2017487828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5280b1ca0 0x7fb528155300 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:25.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.996+0000 7fb52ede9640 1 -- 172.21.15.84:0/2017487828 >> 172.21.15.84:0/2017487828 conn(0x7fb5280d2b90 msgr2=0x7fb5280d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:26.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.996+0000 7fb52ede9640 1 -- 172.21.15.84:0/2017487828 shutdown_connections 2023-10-16T11:05:26.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:25.996+0000 7fb52ede9640 1 -- 172.21.15.84:0/2017487828 wait complete. 2023-10-16T11:05:26.014 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:05:26.022 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:26.020+0000 7fcf18430040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:05:26.031 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:26.030+0000 7f34c443e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:05:26.142 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:26.140+0000 7fcf18430040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:05:26.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.210+0000 7fe6db897640 1 Processor -- start 2023-10-16T11:05:26.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.210+0000 7fe6db897640 1 -- start start 2023-10-16T11:05:26.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.211+0000 7fe6db897640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe6d40cf890 0x7fe6d40c6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:26.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.212+0000 7fe6db897640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe6d40cfc70 0x7fe6d40ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:26.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.212+0000 7fe6db897640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d40b3e80 con 0x7fe6d40c5cf0 2023-10-16T11:05:26.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.212+0000 7fe6db897640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6d40b4630 con 0x7fe6d40cfc70 2023-10-16T11:05:26.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.212+0000 7fe6db897640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6d40cf000 con 0x7fe6d40cf890 2023-10-16T11:05:26.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.212+0000 7fe6d9e34640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe6d40cfc70 0x7fe6d40ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:26.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.212+0000 7fe6d9e34640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe6d40cfc70 0x7fe6d40ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48538/0 (socket says 172.21.15.84:48538) 2023-10-16T11:05:26.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.212+0000 7fe6d9e34640 1 -- 172.21.15.84:0/1393058213 learned_addr learned my addr 172.21.15.84:0/1393058213 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:26.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.212+0000 7fe6d9e34640 1 -- 172.21.15.84:0/1393058213 >> v2:172.21.15.157:3300/0 conn(0x7fe6d40cf890 msgr2=0x7fe6d40c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:26.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.212+0000 7fe6d9e34640 1 --2- 172.21.15.84:0/1393058213 >> v2:172.21.15.157:3300/0 conn(0x7fe6d40cf890 0x7fe6d40c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:26.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.212+0000 7fe6d9e34640 1 -- 172.21.15.84:0/1393058213 >> v1:172.21.15.157:6789/0 conn(0x7fe6d40c5cf0 legacy=0x7fe6d40b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:26.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.212+0000 7fe6d9e34640 1 -- 172.21.15.84:0/1393058213 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d40ce630 con 0x7fe6d40cfc70 2023-10-16T11:05:26.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.214+0000 7fe6d9e34640 1 --2- 172.21.15.84:0/1393058213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6d40cfc70 0x7fe6d40ceac0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40d2c20 tx=0x7fe6cc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9956b063af860ff server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:26.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.214+0000 7fe6cbfff640 1 -- 172.21.15.84:0/1393058213 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc02c9f0 con 0x7fe6d40cfc70 2023-10-16T11:05:26.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.214+0000 7fe6cbfff640 1 -- 172.21.15.84:0/1393058213 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6cc02cb50 con 0x7fe6d40cfc70 2023-10-16T11:05:26.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.214+0000 7fe6cbfff640 1 -- 172.21.15.84:0/1393058213 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc0343f0 con 0x7fe6d40cfc70 2023-10-16T11:05:26.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.215+0000 7fe6db897640 1 -- 172.21.15.84:0/1393058213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6d40cfc70 msgr2=0x7fe6d40ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:26.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.215+0000 7fe6db897640 1 --2- 172.21.15.84:0/1393058213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6d40cfc70 0x7fe6d40ceac0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40d2c20 tx=0x7fe6cc02be50 comp rx=0 tx=0).stop 2023-10-16T11:05:26.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.215+0000 7fe6db897640 1 -- 172.21.15.84:0/1393058213 shutdown_connections 2023-10-16T11:05:26.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.215+0000 7fe6db897640 1 --2- 172.21.15.84:0/1393058213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6d40cfc70 0x7fe6d40ceac0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:26.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.215+0000 7fe6db897640 1 --2- 172.21.15.84:0/1393058213 >> v2:172.21.15.157:3300/0 conn(0x7fe6d40cf890 0x7fe6d40c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:26.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.215+0000 7fe6db897640 1 -- 172.21.15.84:0/1393058213 >> 172.21.15.84:0/1393058213 conn(0x7fe6d40d2cc0 msgr2=0x7fe6d40d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:26.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.215+0000 7fe6db897640 1 -- 172.21.15.84:0/1393058213 shutdown_connections 2023-10-16T11:05:26.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.215+0000 7fe6db897640 1 -- 172.21.15.84:0/1393058213 wait complete. 2023-10-16T11:05:26.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.217+0000 7fe6db897640 1 Processor -- start 2023-10-16T11:05:26.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.217+0000 7fe6db897640 1 -- start start 2023-10-16T11:05:26.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.218+0000 7fe6db897640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6d40c5cf0 0x7fe6d415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:26.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.218+0000 7fe6db897640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6d40cf890 0x7fe6d406b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:26.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.218+0000 7fe6db897640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6d40cfc70 0x7fe6d406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:26.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.218+0000 7fe6db897640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d4155a30 con 0x7fe6d40cf890 2023-10-16T11:05:26.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.218+0000 7fe6db897640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d40c84e0 con 0x7fe6d40c5cf0 2023-10-16T11:05:26.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.218+0000 7fe6db897640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d40c7440 con 0x7fe6d40cfc70 2023-10-16T11:05:26.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.219+0000 7fe6d8e32640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6d40cf890 0x7fe6d406b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:26.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.219+0000 7fe6d8e32640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6d40cf890 0x7fe6d406b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48562/0 (socket says 172.21.15.84:48562) 2023-10-16T11:05:26.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.219+0000 7fe6d8e32640 1 -- 172.21.15.84:0/2918626951 learned_addr learned my addr 172.21.15.84:0/2918626951 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:26.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.219+0000 7fe6d9e34640 1 --2- 172.21.15.84:0/2918626951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6d40cfc70 0x7fe6d406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:26.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.219+0000 7fe6d8e32640 1 -- 172.21.15.84:0/2918626951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6d40cfc70 msgr2=0x7fe6d406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:26.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.219+0000 7fe6d8e32640 1 --2- 172.21.15.84:0/2918626951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6d40cfc70 0x7fe6d406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:26.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.219+0000 7fe6d8e32640 1 -- 172.21.15.84:0/2918626951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6d40c5cf0 msgr2=0x7fe6d415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:26.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.219+0000 7fe6d8e32640 1 --2- 172.21.15.84:0/2918626951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6d40c5cf0 0x7fe6d415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:26.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.219+0000 7fe6d8e32640 1 -- 172.21.15.84:0/2918626951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc009b30 con 0x7fe6d40cf890 2023-10-16T11:05:26.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.220+0000 7fe6d8e32640 1 --2- 172.21.15.84:0/2918626951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6d40cf890 0x7fe6d406b460 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400b3f0 tx=0x7fe6c400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:26.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.220+0000 7fe6c9ffb640 1 -- 172.21.15.84:0/2918626951 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4011830 con 0x7fe6d40cf890 2023-10-16T11:05:26.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.220+0000 7fe6c9ffb640 1 -- 172.21.15.84:0/2918626951 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c4011990 con 0x7fe6d40cf890 2023-10-16T11:05:26.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.220+0000 7fe6c9ffb640 1 -- 172.21.15.84:0/2918626951 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4011d40 con 0x7fe6d40cf890 2023-10-16T11:05:26.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.220+0000 7fe6db897640 1 -- 172.21.15.84:0/2918626951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d40c7720 con 0x7fe6d40cf890 2023-10-16T11:05:26.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.220+0000 7fe6db897640 1 -- 172.21.15.84:0/2918626951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d40c7ce0 con 0x7fe6d40cf890 2023-10-16T11:05:26.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.221+0000 7fe6db897640 1 -- 172.21.15.84:0/2918626951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d40cd2a0 con 0x7fe6d40cf890 2023-10-16T11:05:26.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.223+0000 7fe6c9ffb640 1 -- 172.21.15.84:0/2918626951 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 90) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe6c4004020 con 0x7fe6d40cf890 2023-10-16T11:05:26.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.224+0000 7fe6c9ffb640 1 -- 172.21.15.84:0/2918626951 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6527+0+0 (secure 0 0 0) 0x7fe6c4089a60 con 0x7fe6d40cf890 2023-10-16T11:05:26.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.228+0000 7fe6c9ffb640 1 -- 172.21.15.84:0/2918626951 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe6c4054bc0 con 0x7fe6d40cf890 2023-10-16T11:05:26.232 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:26.231+0000 7fce99fdb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:26.233 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:26.231+0000 7fce99fdb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:26.233 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:26.231+0000 7fce99fdb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:26.233 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:26.231+0000 7fce99fdb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:26.233 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:26.231+0000 7fce99fdb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:26.234 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:26.231+0000 7fce9cfe1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:26.234 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:26.231+0000 7fce9cfe1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:26.234 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:26.231+0000 7fce9cfe1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:26.234 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:26.231+0000 7fce9cfe1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:26.234 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:26.231+0000 7fce9cfe1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:26.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.425+0000 7fe6db897640 1 -- 172.21.15.84:0/2918626951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe6d404e160 con 0x7fe6d40cf890 2023-10-16T11:05:26.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.426+0000 7fe6c9ffb640 1 -- 172.21.15.84:0/2918626951 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v159) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe6c40150a0 con 0x7fe6d40cf890 2023-10-16T11:05:26.428 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:05:26.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.429+0000 7fe6db897640 1 -- 172.21.15.84:0/2918626951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6d40cf890 msgr2=0x7fe6d406b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:26.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.429+0000 7fe6db897640 1 --2- 172.21.15.84:0/2918626951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6d40cf890 0x7fe6d406b460 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400b3f0 tx=0x7fe6c400b8b0 comp rx=0 tx=0).stop 2023-10-16T11:05:26.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.429+0000 7fe6db897640 1 -- 172.21.15.84:0/2918626951 shutdown_connections 2023-10-16T11:05:26.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.429+0000 7fe6db897640 1 --2- 172.21.15.84:0/2918626951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6d40cfc70 0x7fe6d406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:26.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.429+0000 7fe6db897640 1 --2- 172.21.15.84:0/2918626951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6d40cf890 0x7fe6d406b460 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:26.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.429+0000 7fe6db897640 1 --2- 172.21.15.84:0/2918626951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6d40c5cf0 0x7fe6d415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:26.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.429+0000 7fe6db897640 1 -- 172.21.15.84:0/2918626951 >> 172.21.15.84:0/2918626951 conn(0x7fe6d40d2cc0 msgr2=0x7fe6d41611a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:26.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.429+0000 7fe6db897640 1 -- 172.21.15.84:0/2918626951 shutdown_connections 2023-10-16T11:05:26.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.429+0000 7fe6db897640 1 -- 172.21.15.84:0/2918626951 wait complete. 2023-10-16T11:05:26.451 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:05:26.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.702+0000 7f58d7fff640 1 Processor -- start 2023-10-16T11:05:26.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.702+0000 7f58d7fff640 1 -- start start 2023-10-16T11:05:26.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.703+0000 7f58d7fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58d8070b60 0x7f58d815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:26.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.703+0000 7f58d7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f58d806c100 0x7f58d806c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:26.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.703+0000 7f58d7fff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d815aef0 con 0x7f58d8159950 2023-10-16T11:05:26.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.704+0000 7f58d7fff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d815e490 con 0x7f58d806c100 2023-10-16T11:05:26.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.704+0000 7f58d7fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d815d910 con 0x7f58d8070b60 2023-10-16T11:05:26.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.704+0000 7f58d77fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f58d806c100 0x7f58d806c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:26.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.704+0000 7f58d77fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f58d806c100 0x7f58d806c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48604/0 (socket says 172.21.15.84:48604) 2023-10-16T11:05:26.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.704+0000 7f58d77fe640 1 -- 172.21.15.84:0/1941404578 learned_addr learned my addr 172.21.15.84:0/1941404578 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:26.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.704+0000 7f58d67fc640 1 --2- 172.21.15.84:0/1941404578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58d8070b60 0x7f58d815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:26.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.704+0000 7f58d77fe640 1 -- 172.21.15.84:0/1941404578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58d8070b60 msgr2=0x7f58d815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:26.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.704+0000 7f58d77fe640 1 --2- 172.21.15.84:0/1941404578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58d8070b60 0x7f58d815a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:26.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.704+0000 7f58d77fe640 1 -- 172.21.15.84:0/1941404578 >> v1:172.21.15.157:6789/0 conn(0x7f58d8159950 legacy=0x7f58d8070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:26.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.704+0000 7f58d77fe640 1 -- 172.21.15.84:0/1941404578 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d815dbf0 con 0x7f58d806c100 2023-10-16T11:05:26.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.705+0000 7f58d77fe640 1 --2- 172.21.15.84:0/1941404578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d806c100 0x7f58d806c550 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009e50 tx=0x7f58d002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b348f022260b051 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:26.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.706+0000 7f58d5ffb640 1 -- 172.21.15.84:0/1941404578 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d002c9e0 con 0x7f58d806c100 2023-10-16T11:05:26.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.706+0000 7f58d5ffb640 1 -- 172.21.15.84:0/1941404578 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d002cb40 con 0x7f58d806c100 2023-10-16T11:05:26.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.706+0000 7f58d5ffb640 1 -- 172.21.15.84:0/1941404578 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d00343f0 con 0x7f58d806c100 2023-10-16T11:05:26.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.706+0000 7f58d7fff640 1 -- 172.21.15.84:0/1941404578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d806c100 msgr2=0x7f58d806c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:26.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.706+0000 7f58d7fff640 1 --2- 172.21.15.84:0/1941404578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d806c100 0x7f58d806c550 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009e50 tx=0x7f58d002be40 comp rx=0 tx=0).stop 2023-10-16T11:05:26.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.706+0000 7f58d7fff640 1 -- 172.21.15.84:0/1941404578 shutdown_connections 2023-10-16T11:05:26.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.706+0000 7f58d7fff640 1 --2- 172.21.15.84:0/1941404578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d806c100 0x7f58d806c550 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:26.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.706+0000 7f58d7fff640 1 --2- 172.21.15.84:0/1941404578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58d8070b60 0x7f58d815a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:26.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.706+0000 7f58d7fff640 1 -- 172.21.15.84:0/1941404578 >> 172.21.15.84:0/1941404578 conn(0x7f58d80ce6a0 msgr2=0x7f58d80ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:26.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.706+0000 7f58d7fff640 1 -- 172.21.15.84:0/1941404578 shutdown_connections 2023-10-16T11:05:26.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.706+0000 7f58d7fff640 1 -- 172.21.15.84:0/1941404578 wait complete. 2023-10-16T11:05:26.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.708+0000 7f58d7fff640 1 Processor -- start 2023-10-16T11:05:26.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.708+0000 7f58d7fff640 1 -- start start 2023-10-16T11:05:26.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.709+0000 7f58d7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d806c100 0x7f58d813e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:26.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.709+0000 7f58d7fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58d8070b60 0x7f58d81f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:26.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.709+0000 7f58d7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58d8159950 0x7f58d8140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:26.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.709+0000 7f58d7fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d8154540 con 0x7f58d806c100 2023-10-16T11:05:26.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.709+0000 7f58d7fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d81541a0 con 0x7f58d8159950 2023-10-16T11:05:26.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.709+0000 7f58d7fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d81f5b90 con 0x7f58d8070b60 2023-10-16T11:05:26.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.709+0000 7f58d6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d806c100 0x7f58d813e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:26.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.709+0000 7f58d6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d806c100 0x7f58d813e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48620/0 (socket says 172.21.15.84:48620) 2023-10-16T11:05:26.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.709+0000 7f58d6ffd640 1 -- 172.21.15.84:0/2489775088 learned_addr learned my addr 172.21.15.84:0/2489775088 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:26.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.710+0000 7f58d6ffd640 1 -- 172.21.15.84:0/2489775088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58d8070b60 msgr2=0x7f58d81f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:26.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.710+0000 7f58d67fc640 1 --2- 172.21.15.84:0/2489775088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58d8070b60 0x7f58d81f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:26.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.710+0000 7f58d6ffd640 1 --2- 172.21.15.84:0/2489775088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58d8070b60 0x7f58d81f5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:26.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.710+0000 7f58d6ffd640 1 -- 172.21.15.84:0/2489775088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58d8159950 msgr2=0x7f58d8140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:26.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.710+0000 7f58d77fe640 1 --2- 172.21.15.84:0/2489775088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58d8159950 0x7f58d8140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:26.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.710+0000 7f58d6ffd640 1 --2- 172.21.15.84:0/2489775088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58d8159950 0x7f58d8140fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:26.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.710+0000 7f58d6ffd640 1 -- 172.21.15.84:0/2489775088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d0009b30 con 0x7f58d806c100 2023-10-16T11:05:26.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.710+0000 7f58d6ffd640 1 --2- 172.21.15.84:0/2489775088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d806c100 0x7f58d813e5a0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f58c800b3f0 tx=0x7f58c800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:26.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.710+0000 7f58bf7fe640 1 -- 172.21.15.84:0/2489775088 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8011830 con 0x7f58d806c100 2023-10-16T11:05:26.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.710+0000 7f58bf7fe640 1 -- 172.21.15.84:0/2489775088 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c8011990 con 0x7f58d806c100 2023-10-16T11:05:26.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.710+0000 7f58d7fff640 1 -- 172.21.15.84:0/2489775088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d81524c0 con 0x7f58d806c100 2023-10-16T11:05:26.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.710+0000 7f58bf7fe640 1 -- 172.21.15.84:0/2489775088 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8011d40 con 0x7f58d806c100 2023-10-16T11:05:26.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.711+0000 7f58d7fff640 1 -- 172.21.15.84:0/2489775088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d8141500 con 0x7f58d806c100 2023-10-16T11:05:26.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.712+0000 7f58d7fff640 1 -- 172.21.15.84:0/2489775088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d8154000 con 0x7f58d806c100 2023-10-16T11:05:26.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.713+0000 7f58bf7fe640 1 -- 172.21.15.84:0/2489775088 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 90) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f58c801a400 con 0x7f58d806c100 2023-10-16T11:05:26.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.714+0000 7f58bf7fe640 1 -- 172.21.15.84:0/2489775088 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f58c8089310 con 0x7f58d806c100 2023-10-16T11:05:26.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.718+0000 7f58bf7fe640 1 -- 172.21.15.84:0/2489775088 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f58d8154000 con 0x7f58d806c100 2023-10-16T11:05:26.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.909+0000 7f58d7fff640 1 -- 172.21.15.84:0/2489775088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f58d8154000 con 0x7f58d806c100 2023-10-16T11:05:26.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.910+0000 7f58bf7fe640 1 -- 172.21.15.84:0/2489775088 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v159) v1 ==== 100+0+11 (secure 0 0 0) 0x7f58d8154000 con 0x7f58d806c100 2023-10-16T11:05:26.912 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:05:26.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.913+0000 7f58d7fff640 1 -- 172.21.15.84:0/2489775088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d806c100 msgr2=0x7f58d813e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:26.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.913+0000 7f58d7fff640 1 --2- 172.21.15.84:0/2489775088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d806c100 0x7f58d813e5a0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f58c800b3f0 tx=0x7f58c800b8b0 comp rx=0 tx=0).stop 2023-10-16T11:05:26.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.913+0000 7f58d7fff640 1 -- 172.21.15.84:0/2489775088 shutdown_connections 2023-10-16T11:05:26.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.913+0000 7f58d7fff640 1 --2- 172.21.15.84:0/2489775088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58d8159950 0x7f58d8140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:26.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.913+0000 7f58d7fff640 1 --2- 172.21.15.84:0/2489775088 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58d8070b60 0x7f58d81f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:26.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.913+0000 7f58d7fff640 1 --2- 172.21.15.84:0/2489775088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58d806c100 0x7f58d813e5a0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:26.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.913+0000 7f58d7fff640 1 -- 172.21.15.84:0/2489775088 >> 172.21.15.84:0/2489775088 conn(0x7f58d80ce6a0 msgr2=0x7f58d8070450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:26.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.913+0000 7f58d7fff640 1 -- 172.21.15.84:0/2489775088 shutdown_connections 2023-10-16T11:05:26.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:26.913+0000 7f58d7fff640 1 -- 172.21.15.84:0/2489775088 wait complete. 2023-10-16T11:05:26.932 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:05:27.022 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:05:27.031 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:05:27.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.136+0000 7fc898821640 1 Processor -- start 2023-10-16T11:05:27.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.136+0000 7fc898821640 1 -- start start 2023-10-16T11:05:27.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.137+0000 7fc898821640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc894159b20 0x7fc89406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:27.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.138+0000 7fc898821640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc8940b0b20 0x7fc89406fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:27.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.138+0000 7fc898821640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89406f7f0 con 0x7fc894159f00 2023-10-16T11:05:27.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.138+0000 7fc898821640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89406c5e0 con 0x7fc8940b0b20 2023-10-16T11:05:27.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.138+0000 7fc898821640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89406c8c0 con 0x7fc894159b20 2023-10-16T11:05:27.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.138+0000 7fc88bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc894159f00 0x7fc8940b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37792/0 (socket says 172.21.15.84:37792) 2023-10-16T11:05:27.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.138+0000 7fc88bfff640 1 -- 172.21.15.84:0/3861097714 learned_addr learned my addr 172.21.15.84:0/3861097714 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:27.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.138+0000 7fc89259c640 1 --2- 172.21.15.84:0/3861097714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc894159b20 0x7fc89406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:27.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.138+0000 7fc892d9d640 1 --2- 172.21.15.84:0/3861097714 >> v2:172.21.15.157:3300/0 conn(0x7fc8940b0b20 0x7fc89406fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:27.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.138+0000 7fc891d9b640 1 -- 172.21.15.84:0/3861097714 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214038170 0 0) 0x7fc89406f7f0 con 0x7fc894159f00 2023-10-16T11:05:27.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.138+0000 7fc891d9b640 1 -- 172.21.15.84:0/3861097714 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc880003520 con 0x7fc894159f00 2023-10-16T11:05:27.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.139+0000 7fc89259c640 1 -- 172.21.15.84:0/3861097714 >> v2:172.21.15.157:3300/0 conn(0x7fc8940b0b20 msgr2=0x7fc89406fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:27.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.139+0000 7fc89259c640 1 --2- 172.21.15.84:0/3861097714 >> v2:172.21.15.157:3300/0 conn(0x7fc8940b0b20 0x7fc89406fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.139+0000 7fc89259c640 1 -- 172.21.15.84:0/3861097714 >> v1:172.21.15.84:6789/0 conn(0x7fc894159f00 legacy=0x7fc8940b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:27.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.139+0000 7fc89259c640 1 -- 172.21.15.84:0/3861097714 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89406cba0 con 0x7fc894159b20 2023-10-16T11:05:27.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.140+0000 7fc89259c640 1 --2- 172.21.15.84:0/3861097714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc894159b20 0x7fc89406bbe0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc8840097a0 tx=0x7fc88402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c80859f4187fff7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:27.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.140+0000 7fc891d9b640 1 -- 172.21.15.84:0/3861097714 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88402d040 con 0x7fc894159b20 2023-10-16T11:05:27.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.141+0000 7fc891d9b640 1 -- 172.21.15.84:0/3861097714 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc884031810 con 0x7fc894159b20 2023-10-16T11:05:27.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.141+0000 7fc891d9b640 1 -- 172.21.15.84:0/3861097714 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc884031b80 con 0x7fc894159b20 2023-10-16T11:05:27.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.141+0000 7fc898821640 1 -- 172.21.15.84:0/3861097714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc894159b20 msgr2=0x7fc89406bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:27.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.141+0000 7fc898821640 1 --2- 172.21.15.84:0/3861097714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc894159b20 0x7fc89406bbe0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc8840097a0 tx=0x7fc88402b8b0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.141+0000 7fc898821640 1 -- 172.21.15.84:0/3861097714 shutdown_connections 2023-10-16T11:05:27.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.141+0000 7fc898821640 1 --2- 172.21.15.84:0/3861097714 >> v2:172.21.15.157:3300/0 conn(0x7fc8940b0b20 0x7fc89406fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.141+0000 7fc898821640 1 --2- 172.21.15.84:0/3861097714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc894159b20 0x7fc89406bbe0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.141+0000 7fc898821640 1 -- 172.21.15.84:0/3861097714 >> 172.21.15.84:0/3861097714 conn(0x7fc8940cb410 msgr2=0x7fc8940cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:27.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.141+0000 7fc898821640 1 -- 172.21.15.84:0/3861097714 shutdown_connections 2023-10-16T11:05:27.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.141+0000 7fc898821640 1 -- 172.21.15.84:0/3861097714 wait complete. 2023-10-16T11:05:27.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.143+0000 7fc898821640 1 Processor -- start 2023-10-16T11:05:27.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.143+0000 7fc898821640 1 -- start start 2023-10-16T11:05:27.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.144+0000 7fc898821640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8940b0b20 0x7fc8941581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:27.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.144+0000 7fc898821640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc894159b20 0x7fc89414acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:27.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.144+0000 7fc898821640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc894159f00 0x7fc8940c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:27.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.144+0000 7fc898821640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8940c4960 con 0x7fc8940b0b20 2023-10-16T11:05:27.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.144+0000 7fc898821640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89414b200 con 0x7fc894159f00 2023-10-16T11:05:27.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.144+0000 7fc898821640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8940c3d50 con 0x7fc894159b20 2023-10-16T11:05:27.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.144+0000 7fc89259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8940b0b20 0x7fc8941581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:27.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.144+0000 7fc89259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8940b0b20 0x7fc8941581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48626/0 (socket says 172.21.15.84:48626) 2023-10-16T11:05:27.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.144+0000 7fc89259c640 1 -- 172.21.15.84:0/4084122439 learned_addr learned my addr 172.21.15.84:0/4084122439 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:27.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.145+0000 7fc88bfff640 1 --2- 172.21.15.84:0/4084122439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc894159b20 0x7fc89414acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:27.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.145+0000 7fc892d9d640 1 --2- 172.21.15.84:0/4084122439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc894159f00 0x7fc8940c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:27.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.145+0000 7fc89259c640 1 -- 172.21.15.84:0/4084122439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc894159b20 msgr2=0x7fc89414acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:27.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.145+0000 7fc89259c640 1 --2- 172.21.15.84:0/4084122439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc894159b20 0x7fc89414acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.145+0000 7fc89259c640 1 -- 172.21.15.84:0/4084122439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc894159f00 msgr2=0x7fc8940c4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:27.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.145+0000 7fc89259c640 1 --2- 172.21.15.84:0/4084122439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc894159f00 0x7fc8940c4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.145+0000 7fc89259c640 1 -- 172.21.15.84:0/4084122439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc884009400 con 0x7fc8940b0b20 2023-10-16T11:05:27.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.145+0000 7fc89259c640 1 --2- 172.21.15.84:0/4084122439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8940b0b20 0x7fc8941581f0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fc88402b8f0 tx=0x7fc8840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:27.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.145+0000 7fc88affd640 1 -- 172.21.15.84:0/4084122439 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88403ad60 con 0x7fc8940b0b20 2023-10-16T11:05:27.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.145+0000 7fc88affd640 1 -- 172.21.15.84:0/4084122439 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8840443f0 con 0x7fc8940b0b20 2023-10-16T11:05:27.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.145+0000 7fc88affd640 1 -- 172.21.15.84:0/4084122439 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc884044580 con 0x7fc8940b0b20 2023-10-16T11:05:27.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.145+0000 7fc898821640 1 -- 172.21.15.84:0/4084122439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8940c4030 con 0x7fc8940b0b20 2023-10-16T11:05:27.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.146+0000 7fc898821640 1 -- 172.21.15.84:0/4084122439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8940c45f0 con 0x7fc8940b0b20 2023-10-16T11:05:27.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.147+0000 7fc898821640 1 -- 172.21.15.84:0/4084122439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89404e160 con 0x7fc8940b0b20 2023-10-16T11:05:27.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.148+0000 7fc88affd640 1 -- 172.21.15.84:0/4084122439 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 90) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fc88404d450 con 0x7fc8940b0b20 2023-10-16T11:05:27.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.149+0000 7fc88affd640 1 -- 172.21.15.84:0/4084122439 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6527+0+0 (secure 0 0 0) 0x7fc8840ab720 con 0x7fc8940b0b20 2023-10-16T11:05:27.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.153+0000 7fc88affd640 1 -- 172.21.15.84:0/4084122439 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc884076890 con 0x7fc8940b0b20 2023-10-16T11:05:27.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.164+0000 7fc88affd640 1 -- 172.21.15.84:0/4084122439 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 91) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fc884076340 con 0x7fc8940b0b20 2023-10-16T11:05:27.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.164+0000 7fc88affd640 1 --2- 172.21.15.84:0/4084122439 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc874070740 0x7fc874072b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:27.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.168+0000 7fc88bfff640 1 --2- 172.21.15.84:0/4084122439 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc874070740 0x7fc874072b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:27.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.177+0000 7fc88bfff640 1 --2- 172.21.15.84:0/4084122439 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc874070740 0x7fc874072b30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc8940c8420 tx=0x7fc87c009040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:27.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.397+0000 7fc898821640 1 -- 172.21.15.84:0/4084122439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc89406ce50 con 0x7fc8940b0b20 2023-10-16T11:05:27.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.398+0000 7fc88affd640 1 -- 172.21.15.84:0/4084122439 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fc884035090 con 0x7fc8940b0b20 2023-10-16T11:05:27.400 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:27.400 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6517,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1733540616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1733540616},{"type":"v1","addr":"172.21.15.157:6833","nonce":1733540616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3815048417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3815048417},{"type":"v1","addr":"172.21.15.157:6835","nonce":3815048417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9135,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1038548935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1038548935},{"type":"v1","addr":"172.21.15.84:6836","nonce":1038548935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:09:10.652604+0000","modified":"2023-10-16T10:09:13.730838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9138},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9138":{"gid":9138,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2797149326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2797149326},{"type":"v1","addr":"172.21.15.84:6837","nonce":2797149326}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-16T11:05:27.401 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 80 2023-10-16T11:05:27.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.402+0000 7fc898821640 1 -- 172.21.15.84:0/4084122439 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc874070740 msgr2=0x7fc874072b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:27.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.402+0000 7fc898821640 1 --2- 172.21.15.84:0/4084122439 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc874070740 0x7fc874072b30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc8940c8420 tx=0x7fc87c009040 comp rx=0 tx=0).stop 2023-10-16T11:05:27.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.402+0000 7fc898821640 1 -- 172.21.15.84:0/4084122439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8940b0b20 msgr2=0x7fc8941581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:27.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.402+0000 7fc898821640 1 --2- 172.21.15.84:0/4084122439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8940b0b20 0x7fc8941581f0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fc88402b8f0 tx=0x7fc8840046c0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.402+0000 7fc898821640 1 -- 172.21.15.84:0/4084122439 shutdown_connections 2023-10-16T11:05:27.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.402+0000 7fc898821640 1 --2- 172.21.15.84:0/4084122439 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc874070740 0x7fc874072b30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.402+0000 7fc898821640 1 --2- 172.21.15.84:0/4084122439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc894159f00 0x7fc8940c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.402+0000 7fc898821640 1 --2- 172.21.15.84:0/4084122439 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc894159b20 0x7fc89414acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.402+0000 7fc898821640 1 --2- 172.21.15.84:0/4084122439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8940b0b20 0x7fc8941581f0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.402+0000 7fc898821640 1 -- 172.21.15.84:0/4084122439 >> 172.21.15.84:0/4084122439 conn(0x7fc8940cb410 msgr2=0x7fc8940acf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:27.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.402+0000 7fc898821640 1 -- 172.21.15.84:0/4084122439 shutdown_connections 2023-10-16T11:05:27.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.402+0000 7fc898821640 1 -- 172.21.15.84:0/4084122439 wait complete. 2023-10-16T11:05:27.422 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:05:27.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.677+0000 7fc1923fb640 1 Processor -- start 2023-10-16T11:05:27.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.677+0000 7fc1923fb640 1 -- start start 2023-10-16T11:05:27.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.678+0000 7fc1923fb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc18c159b20 0x7fc18c06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:27.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.678+0000 7fc1923fb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc18c06f7f0 con 0x7fc18c0b0b20 2023-10-16T11:05:27.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.678+0000 7fc1923fb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc18c06c5e0 con 0x7fc18c159f00 2023-10-16T11:05:27.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.678+0000 7fc1923fb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc18c06c8c0 con 0x7fc18c159b20 2023-10-16T11:05:27.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.679+0000 7fc18bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc18c159b20 0x7fc18c06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:27.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.679+0000 7fc190998640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc18c0b0b20 0x7fc18c06fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37802/0 (socket says 172.21.15.84:37802) 2023-10-16T11:05:27.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.679+0000 7fc18bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc18c159b20 0x7fc18c06bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48642/0 (socket says 172.21.15.84:48642) 2023-10-16T11:05:27.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.679+0000 7fc190998640 1 -- 172.21.15.84:0/1884005889 learned_addr learned my addr 172.21.15.84:0/1884005889 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:27.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.679+0000 7fc18bfff640 1 -- 172.21.15.84:0/1884005889 >> v1:172.21.15.157:6789/0 conn(0x7fc18c159f00 legacy=0x7fc18c0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:27.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.679+0000 7fc18bfff640 1 -- 172.21.15.84:0/1884005889 >> v1:172.21.15.84:6789/0 conn(0x7fc18c0b0b20 legacy=0x7fc18c06fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:27.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.679+0000 7fc18bfff640 1 -- 172.21.15.84:0/1884005889 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c06cba0 con 0x7fc18c159b20 2023-10-16T11:05:27.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.680+0000 7fc18bfff640 1 --2- 172.21.15.84:0/1884005889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc18c159b20 0x7fc18c06bbe0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0097a0 tx=0x7fc17c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8501daa3656cf017 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:27.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.680+0000 7fc18b7fe640 1 -- 172.21.15.84:0/1884005889 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c031830 con 0x7fc18c159b20 2023-10-16T11:05:27.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.680+0000 7fc18b7fe640 1 -- 172.21.15.84:0/1884005889 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c031990 con 0x7fc18c159b20 2023-10-16T11:05:27.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.680+0000 7fc18b7fe640 1 -- 172.21.15.84:0/1884005889 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c031d20 con 0x7fc18c159b20 2023-10-16T11:05:27.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.681+0000 7fc1923fb640 1 -- 172.21.15.84:0/1884005889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc18c159b20 msgr2=0x7fc18c06bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:27.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.681+0000 7fc1923fb640 1 --2- 172.21.15.84:0/1884005889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc18c159b20 0x7fc18c06bbe0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0097a0 tx=0x7fc17c02b8d0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.681+0000 7fc1923fb640 1 -- 172.21.15.84:0/1884005889 shutdown_connections 2023-10-16T11:05:27.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.681+0000 7fc1923fb640 1 --2- 172.21.15.84:0/1884005889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc18c159b20 0x7fc18c06bbe0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.681+0000 7fc1923fb640 1 -- 172.21.15.84:0/1884005889 >> 172.21.15.84:0/1884005889 conn(0x7fc18c0cb410 msgr2=0x7fc18c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:27.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.681+0000 7fc1923fb640 1 -- 172.21.15.84:0/1884005889 shutdown_connections 2023-10-16T11:05:27.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.681+0000 7fc1923fb640 1 -- 172.21.15.84:0/1884005889 wait complete. 2023-10-16T11:05:27.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.683+0000 7fc1923fb640 1 Processor -- start 2023-10-16T11:05:27.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.683+0000 7fc1923fb640 1 -- start start 2023-10-16T11:05:27.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.684+0000 7fc1923fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18c0b0b20 0x7fc18c0c3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:27.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.684+0000 7fc1923fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc18c159b20 0x7fc18c15d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:27.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.684+0000 7fc1923fb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc18c159f00 0x7fc18c0c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:27.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.684+0000 7fc1923fb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c0c2f90 con 0x7fc18c159f00 2023-10-16T11:05:27.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.684+0000 7fc1923fb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c0c2660 con 0x7fc18c0b0b20 2023-10-16T11:05:27.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.684+0000 7fc1923fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c0c2940 con 0x7fc18c159b20 2023-10-16T11:05:27.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.685+0000 7fc190998640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc18c159f00 0x7fc18c0c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:27.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.685+0000 7fc190998640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc18c159f00 0x7fc18c0c43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48646/0 (socket says 172.21.15.84:48646) 2023-10-16T11:05:27.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.685+0000 7fc190998640 1 -- 172.21.15.84:0/566880632 learned_addr learned my addr 172.21.15.84:0/566880632 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:27.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.685+0000 7fc18bfff640 1 --2- 172.21.15.84:0/566880632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18c0b0b20 0x7fc18c0c3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:27.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.685+0000 7fc190998640 1 -- 172.21.15.84:0/566880632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc18c159b20 msgr2=0x7fc18c15d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:27.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.685+0000 7fc190998640 1 --2- 172.21.15.84:0/566880632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc18c159b20 0x7fc18c15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.685+0000 7fc190998640 1 -- 172.21.15.84:0/566880632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18c0b0b20 msgr2=0x7fc18c0c3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:27.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.685+0000 7fc190998640 1 --2- 172.21.15.84:0/566880632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18c0b0b20 0x7fc18c0c3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.685+0000 7fc190998640 1 -- 172.21.15.84:0/566880632 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c009400 con 0x7fc18c159f00 2023-10-16T11:05:27.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.685+0000 7fc190998640 1 --2- 172.21.15.84:0/566880632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc18c159f00 0x7fc18c0c43c0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fc18400ca80 tx=0x7fc18400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:27.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.685+0000 7fc1897fa640 1 -- 172.21.15.84:0/566880632 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18400ac30 con 0x7fc18c159f00 2023-10-16T11:05:27.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.686+0000 7fc1897fa640 1 -- 172.21.15.84:0/566880632 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18400ad90 con 0x7fc18c159f00 2023-10-16T11:05:27.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.686+0000 7fc1923fb640 1 -- 172.21.15.84:0/566880632 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18c0c2c20 con 0x7fc18c159f00 2023-10-16T11:05:27.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.686+0000 7fc1897fa640 1 -- 172.21.15.84:0/566880632 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184012600 con 0x7fc18c159f00 2023-10-16T11:05:27.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.686+0000 7fc1923fb640 1 -- 172.21.15.84:0/566880632 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18c15e020 con 0x7fc18c159f00 2023-10-16T11:05:27.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.687+0000 7fc1923fb640 1 -- 172.21.15.84:0/566880632 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc18c06ce60 con 0x7fc18c159f00 2023-10-16T11:05:27.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.688+0000 7fc1897fa640 1 -- 172.21.15.84:0/566880632 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 91) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fc184007070 con 0x7fc18c159f00 2023-10-16T11:05:27.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.689+0000 7fc1897fa640 1 --2- 172.21.15.84:0/566880632 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc16c06acc0 0x7fc16c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:27.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.689+0000 7fc1897fa640 1 -- 172.21.15.84:0/566880632 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6527+0+0 (secure 0 0 0) 0x7fc1840892d0 con 0x7fc18c159f00 2023-10-16T11:05:27.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.689+0000 7fc18bfff640 1 --2- 172.21.15.84:0/566880632 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc16c06acc0 0x7fc16c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:27.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.689+0000 7fc18bfff640 1 --2- 172.21.15.84:0/566880632 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc16c06acc0 0x7fc16c06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc17c036000 tx=0x7fc17c008030 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:27.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.693+0000 7fc1897fa640 1 -- 172.21.15.84:0/566880632 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc18c06ce60 con 0x7fc18c159f00 2023-10-16T11:05:27.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.886+0000 7fc1923fb640 1 -- 172.21.15.84:0/566880632 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc18c06ce60 con 0x7fc18c159f00 2023-10-16T11:05:27.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.889+0000 7fc1897fa640 1 -- 172.21.15.84:0/566880632 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+17734 (secure 0 0 0) 0x7fc18c06ce60 con 0x7fc18c159f00 2023-10-16T11:05:27.891 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:27.891 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":159,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:05:26.145788+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:09:08.490425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-16T10:09:09.666051+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":158,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.26","mappings":[{"from":2,"to":0}]},{"pgid":"19.3c","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1606872290":"2023-10-17T11:05:26.145693+0000","172.21.15.84:6833/3746729142":"2023-10-17T11:05:26.145693+0000","172.21.15.84:0/3209795865":"2023-10-17T11:05:26.145693+0000","172.21.15.84:0/2439552742":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/384925658":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/3023009161":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/683905531":"2023-10-17T11:05:26.145693+0000","172.21.15.84:0/3754993953":"2023-10-17T11:05:26.145693+0000","172.21.15.84:0/3629412882":"2023-10-17T11:05:26.145693+0000","172.21.15.84:0/1762476558":"2023-10-17T11:05:26.145693+0000","172.21.15.84:0/870751622":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/300305981":"2023-10-17T10:09:25.208422+0000","172.21.15.84:6832/3746729142":"2023-10-17T11:05:26.145693+0000","172.21.15.84:6833/205480762":"2023-10-17T10:09:25.208422+0000","172.21.15.84:6832/205480762":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/1358444053":"2023-10-17T10:09:25.208422+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:05:27.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.892+0000 7fc1923fb640 1 -- 172.21.15.84:0/566880632 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc16c06acc0 msgr2=0x7fc16c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:27.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.892+0000 7fc1923fb640 1 --2- 172.21.15.84:0/566880632 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc16c06acc0 0x7fc16c06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc17c036000 tx=0x7fc17c008030 comp rx=0 tx=0).stop 2023-10-16T11:05:27.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.892+0000 7fc1923fb640 1 -- 172.21.15.84:0/566880632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc18c159f00 msgr2=0x7fc18c0c43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:27.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.892+0000 7fc1923fb640 1 --2- 172.21.15.84:0/566880632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc18c159f00 0x7fc18c0c43c0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fc18400ca80 tx=0x7fc18400cf40 comp rx=0 tx=0).stop 2023-10-16T11:05:27.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.892+0000 7fc1923fb640 1 -- 172.21.15.84:0/566880632 shutdown_connections 2023-10-16T11:05:27.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.892+0000 7fc1923fb640 1 --2- 172.21.15.84:0/566880632 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc16c06acc0 0x7fc16c06d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.892+0000 7fc1923fb640 1 --2- 172.21.15.84:0/566880632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc18c159f00 0x7fc18c0c43c0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.892+0000 7fc1923fb640 1 --2- 172.21.15.84:0/566880632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc18c159b20 0x7fc18c15d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.892+0000 7fc1923fb640 1 --2- 172.21.15.84:0/566880632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18c0b0b20 0x7fc18c0c3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:27.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.892+0000 7fc1923fb640 1 -- 172.21.15.84:0/566880632 >> 172.21.15.84:0/566880632 conn(0x7fc18c0cb410 msgr2=0x7fc18c0c7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:27.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.892+0000 7fc1923fb640 1 -- 172.21.15.84:0/566880632 shutdown_connections 2023-10-16T11:05:27.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:27.892+0000 7fc1923fb640 1 -- 172.21.15.84:0/566880632 wait complete. 2023-10-16T11:05:27.912 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T11:05:27.913 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:05:28.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.126+0000 7fbf1759e640 1 Processor -- start 2023-10-16T11:05:28.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.126+0000 7fbf1759e640 1 -- start start 2023-10-16T11:05:28.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.127+0000 7fbf1759e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf18070b60 0x7fbf1815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:28.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.127+0000 7fbf1759e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf1806c100 0x7fbf1806c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:28.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.127+0000 7fbf1759e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1815aef0 con 0x7fbf18159950 2023-10-16T11:05:28.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.127+0000 7fbf1759e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1815e490 con 0x7fbf1806c100 2023-10-16T11:05:28.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.128+0000 7fbf1759e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1815d910 con 0x7fbf18070b60 2023-10-16T11:05:28.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.128+0000 7fbf1659c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbf18159950 0x7fbf18070450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37818/0 (socket says 172.21.15.84:37818) 2023-10-16T11:05:28.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.128+0000 7fbf1659c640 1 -- 172.21.15.84:0/908558786 learned_addr learned my addr 172.21.15.84:0/908558786 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:28.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.128+0000 7fbf16d9d640 1 --2- 172.21.15.84:0/908558786 >> v2:172.21.15.157:3300/0 conn(0x7fbf1806c100 0x7fbf1806c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:28.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.128+0000 7fbf15d9b640 1 -- 172.21.15.84:0/908558786 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2553664788 0 0) 0x7fbf08001110 con 0x7fbf18159950 2023-10-16T11:05:28.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.128+0000 7fbf0ffff640 1 --2- 172.21.15.84:0/908558786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf18070b60 0x7fbf1815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:28.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.128+0000 7fbf15d9b640 1 -- 172.21.15.84:0/908558786 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf04003520 con 0x7fbf18159950 2023-10-16T11:05:28.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf15d9b640 1 -- 172.21.15.84:0/908558786 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1665221349 0 0) 0x7fbf04003520 con 0x7fbf18159950 2023-10-16T11:05:28.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf15d9b640 1 -- 172.21.15.84:0/908558786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf18070b60 msgr2=0x7fbf1815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:28.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf15d9b640 1 --2- 172.21.15.84:0/908558786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf18070b60 0x7fbf1815a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf15d9b640 1 -- 172.21.15.84:0/908558786 >> v2:172.21.15.157:3300/0 conn(0x7fbf1806c100 msgr2=0x7fbf1806c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:28.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf15d9b640 1 --2- 172.21.15.84:0/908558786 >> v2:172.21.15.157:3300/0 conn(0x7fbf1806c100 0x7fbf1806c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf15d9b640 1 -- 172.21.15.84:0/908558786 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1815dbf0 con 0x7fbf18159950 2023-10-16T11:05:28.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf15d9b640 1 -- 172.21.15.84:0/908558786 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fbf08001800 con 0x7fbf18159950 2023-10-16T11:05:28.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf15d9b640 1 -- 172.21.15.84:0/908558786 >> v1:172.21.15.84:6789/0 conn(0x7fbf18159950 legacy=0x7fbf18070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:28.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf15d9b640 1 --2- 172.21.15.84:0/908558786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf04003970 0x7fbf04003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:28.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf15d9b640 1 --2- 172.21.15.84:0/908558786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf040047c0 0x7fbf04004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:28.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf15d9b640 1 --2- 172.21.15.84:0/908558786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf18070b60 0x7fbf04005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:28.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf15d9b640 1 -- 172.21.15.84:0/908558786 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1815e490 con 0x7fbf04003970 2023-10-16T11:05:28.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf15d9b640 1 -- 172.21.15.84:0/908558786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1815d910 con 0x7fbf040047c0 2023-10-16T11:05:28.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf15d9b640 1 -- 172.21.15.84:0/908558786 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0400b9b0 con 0x7fbf18070b60 2023-10-16T11:05:28.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf1659c640 1 --2- 172.21.15.84:0/908558786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf04003970 0x7fbf04003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:28.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf1659c640 1 -- 172.21.15.84:0/908558786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf18070b60 msgr2=0x7fbf04005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:28.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf1659c640 1 --2- 172.21.15.84:0/908558786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf18070b60 0x7fbf04005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf1659c640 1 -- 172.21.15.84:0/908558786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf040047c0 msgr2=0x7fbf04004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:28.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf1659c640 1 --2- 172.21.15.84:0/908558786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf040047c0 0x7fbf04004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.129+0000 7fbf1659c640 1 -- 172.21.15.84:0/908558786 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf04003520 con 0x7fbf04003970 2023-10-16T11:05:28.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.131+0000 7fbf1659c640 1 --2- 172.21.15.84:0/908558786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf04003970 0x7fbf04003d90 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800c210 tx=0x7fbf040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d51a66f01ebe0f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:28.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.131+0000 7fbf15d9b640 1 -- 172.21.15.84:0/908558786 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf08005ad0 con 0x7fbf04003970 2023-10-16T11:05:28.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.131+0000 7fbf15d9b640 1 -- 172.21.15.84:0/908558786 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf08005c30 con 0x7fbf04003970 2023-10-16T11:05:28.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.131+0000 7fbf15d9b640 1 -- 172.21.15.84:0/908558786 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0802c8c0 con 0x7fbf04003970 2023-10-16T11:05:28.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.131+0000 7fbf1759e640 1 -- 172.21.15.84:0/908558786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf04003970 msgr2=0x7fbf04003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:28.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.131+0000 7fbf1759e640 1 --2- 172.21.15.84:0/908558786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf04003970 0x7fbf04003d90 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800c210 tx=0x7fbf040057f0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.131+0000 7fbf16d9d640 1 -- 172.21.15.84:0/908558786 reap_dead start 2023-10-16T11:05:28.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.132+0000 7fbf1759e640 1 -- 172.21.15.84:0/908558786 shutdown_connections 2023-10-16T11:05:28.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.132+0000 7fbf1759e640 1 -- 172.21.15.84:0/908558786 >> 172.21.15.84:0/908558786 conn(0x7fbf180ce6a0 msgr2=0x7fbf180ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:28.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.132+0000 7fbf1759e640 1 -- 172.21.15.84:0/908558786 shutdown_connections 2023-10-16T11:05:28.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.132+0000 7fbf1759e640 1 -- 172.21.15.84:0/908558786 wait complete. 2023-10-16T11:05:28.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.133+0000 7fbf1759e640 1 Processor -- start 2023-10-16T11:05:28.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.134+0000 7fbf1759e640 1 -- start start 2023-10-16T11:05:28.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.134+0000 7fbf1759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf04003970 0x7fbf1814ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:28.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.134+0000 7fbf1759e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf18157130 0x7fbf1815c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:28.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.134+0000 7fbf1759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf1815b300 0x7fbf1814ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:28.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.134+0000 7fbf1759e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1815d030 con 0x7fbf04003970 2023-10-16T11:05:28.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.134+0000 7fbf1759e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf18139660 con 0x7fbf1815b300 2023-10-16T11:05:28.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.134+0000 7fbf1759e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf18139940 con 0x7fbf18157130 2023-10-16T11:05:28.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.135+0000 7fbf1659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf04003970 0x7fbf1814ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:28.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.135+0000 7fbf1659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf04003970 0x7fbf1814ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48664/0 (socket says 172.21.15.84:48664) 2023-10-16T11:05:28.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.135+0000 7fbf1659c640 1 -- 172.21.15.84:0/1492557639 learned_addr learned my addr 172.21.15.84:0/1492557639 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:28.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.135+0000 7fbf16d9d640 1 --2- 172.21.15.84:0/1492557639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf1815b300 0x7fbf1814ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:28.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.135+0000 7fbf1659c640 1 -- 172.21.15.84:0/1492557639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf18157130 msgr2=0x7fbf1815c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:28.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.135+0000 7fbf1659c640 1 --2- 172.21.15.84:0/1492557639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf18157130 0x7fbf1815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.135+0000 7fbf1659c640 1 -- 172.21.15.84:0/1492557639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf1815b300 msgr2=0x7fbf1814ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:28.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.135+0000 7fbf1659c640 1 --2- 172.21.15.84:0/1492557639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf1815b300 0x7fbf1814ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.135+0000 7fbf1659c640 1 -- 172.21.15.84:0/1492557639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf18139c20 con 0x7fbf04003970 2023-10-16T11:05:28.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.135+0000 7fbf1659c640 1 --2- 172.21.15.84:0/1492557639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf04003970 0x7fbf1814ff50 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fbf08001ac0 tx=0x7fbf08000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:28.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.136+0000 7fbf0effd640 1 -- 172.21.15.84:0/1492557639 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf08039070 con 0x7fbf04003970 2023-10-16T11:05:28.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.136+0000 7fbf0effd640 1 -- 172.21.15.84:0/1492557639 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0802dda0 con 0x7fbf04003970 2023-10-16T11:05:28.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.136+0000 7fbf0effd640 1 -- 172.21.15.84:0/1492557639 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf080072b0 con 0x7fbf04003970 2023-10-16T11:05:28.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.136+0000 7fbf1759e640 1 -- 172.21.15.84:0/1492557639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf18139f00 con 0x7fbf04003970 2023-10-16T11:05:28.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.136+0000 7fbf1759e640 1 -- 172.21.15.84:0/1492557639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1815b7b0 con 0x7fbf04003970 2023-10-16T11:05:28.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.137+0000 7fbf1759e640 1 -- 172.21.15.84:0/1492557639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf1804e9b0 con 0x7fbf04003970 2023-10-16T11:05:28.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.138+0000 7fbf0effd640 1 -- 172.21.15.84:0/1492557639 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 91) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fbf08032070 con 0x7fbf04003970 2023-10-16T11:05:28.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.139+0000 7fbf0effd640 1 --2- 172.21.15.84:0/1492557639 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fbef806acc0 0x7fbef806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:28.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.139+0000 7fbf0effd640 1 -- 172.21.15.84:0/1492557639 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6527+0+0 (secure 0 0 0) 0x7fbf0803c040 con 0x7fbf04003970 2023-10-16T11:05:28.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.139+0000 7fbf0ffff640 1 --2- 172.21.15.84:0/1492557639 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fbef806acc0 0x7fbef806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:28.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.140+0000 7fbf0ffff640 1 --2- 172.21.15.84:0/1492557639 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fbef806acc0 0x7fbef806d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbf00006a90 tx=0x7fbf00006a20 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:28.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.143+0000 7fbf0effd640 1 -- 172.21.15.84:0/1492557639 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbf08075940 con 0x7fbf04003970 2023-10-16T11:05:28.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.384+0000 7fbf1759e640 1 -- 172.21.15.84:0/1492557639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf1815aef0 con 0x7fbf04003970 2023-10-16T11:05:28.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.384+0000 7fbf0effd640 1 -- 172.21.15.84:0/1492557639 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fbf1815aef0 con 0x7fbf04003970 2023-10-16T11:05:28.387 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:28.387 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6517,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1733540616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1733540616},{"type":"v1","addr":"172.21.15.157:6833","nonce":1733540616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3815048417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3815048417},{"type":"v1","addr":"172.21.15.157:6835","nonce":3815048417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9135,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1038548935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1038548935},{"type":"v1","addr":"172.21.15.84:6836","nonce":1038548935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:09:10.652604+0000","modified":"2023-10-16T10:09:13.730838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9138},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9138":{"gid":9138,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2797149326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2797149326},{"type":"v1","addr":"172.21.15.84:6837","nonce":2797149326}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-16T11:05:28.387 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 80 2023-10-16T11:05:28.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.388+0000 7fbf1759e640 1 -- 172.21.15.84:0/1492557639 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fbef806acc0 msgr2=0x7fbef806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:28.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.388+0000 7fbf1759e640 1 --2- 172.21.15.84:0/1492557639 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fbef806acc0 0x7fbef806d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbf00006a90 tx=0x7fbf00006a20 comp rx=0 tx=0).stop 2023-10-16T11:05:28.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.388+0000 7fbf1759e640 1 -- 172.21.15.84:0/1492557639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf04003970 msgr2=0x7fbf1814ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:28.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.388+0000 7fbf1759e640 1 --2- 172.21.15.84:0/1492557639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf04003970 0x7fbf1814ff50 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fbf08001ac0 tx=0x7fbf08000fe0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.388+0000 7fbf1759e640 1 -- 172.21.15.84:0/1492557639 shutdown_connections 2023-10-16T11:05:28.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.388+0000 7fbf1759e640 1 --2- 172.21.15.84:0/1492557639 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fbef806acc0 0x7fbef806d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.388+0000 7fbf1759e640 1 --2- 172.21.15.84:0/1492557639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf1815b300 0x7fbf1814ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.388+0000 7fbf1759e640 1 --2- 172.21.15.84:0/1492557639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf18157130 0x7fbf1815c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.388+0000 7fbf1759e640 1 --2- 172.21.15.84:0/1492557639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf04003970 0x7fbf1814ff50 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.388+0000 7fbf1759e640 1 -- 172.21.15.84:0/1492557639 >> 172.21.15.84:0/1492557639 conn(0x7fbf180ce6a0 msgr2=0x7fbf180c93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:28.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.388+0000 7fbf1759e640 1 -- 172.21.15.84:0/1492557639 shutdown_connections 2023-10-16T11:05:28.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.388+0000 7fbf1759e640 1 -- 172.21.15.84:0/1492557639 wait complete. 2023-10-16T11:05:28.407 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:05:28.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.608+0000 7f3482997640 1 Processor -- start 2023-10-16T11:05:28.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.609+0000 7f3482997640 1 -- start start 2023-10-16T11:05:28.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.610+0000 7f3482997640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f347c15aaa0 0x7f347c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:28.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.610+0000 7f3482997640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f347c0ce390 0x7f347c070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:28.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.610+0000 7f3482997640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f347c15ed60 con 0x7f347c15ae80 2023-10-16T11:05:28.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.610+0000 7f3482997640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f347c06b810 con 0x7f347c0ce390 2023-10-16T11:05:28.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.610+0000 7f3482997640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f347c0707e0 con 0x7f347c15aaa0 2023-10-16T11:05:28.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.610+0000 7f3480f34640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f347c0ce390 0x7f347c070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:28.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.611+0000 7f347b7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f347c15ae80 0x7f347c0cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37822/0 (socket says 172.21.15.84:37822) 2023-10-16T11:05:28.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.611+0000 7f347b7fe640 1 -- 172.21.15.84:0/786042854 learned_addr learned my addr 172.21.15.84:0/786042854 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:28.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.611+0000 7f3480f34640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f347c0ce390 0x7f347c070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48684/0 (socket says 172.21.15.84:48684) 2023-10-16T11:05:28.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.611+0000 7f347bfff640 1 --2- 172.21.15.84:0/786042854 >> v2:172.21.15.157:3300/0 conn(0x7f347c15aaa0 0x7f347c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:28.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.611+0000 7f3480f34640 1 -- 172.21.15.84:0/786042854 >> v2:172.21.15.157:3300/0 conn(0x7f347c15aaa0 msgr2=0x7f347c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:28.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.611+0000 7f3480f34640 1 --2- 172.21.15.84:0/786042854 >> v2:172.21.15.157:3300/0 conn(0x7f347c15aaa0 0x7f347c15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.611+0000 7f3480f34640 1 -- 172.21.15.84:0/786042854 >> v1:172.21.15.84:6789/0 conn(0x7f347c15ae80 legacy=0x7f347c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:28.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.611+0000 7f3480f34640 1 -- 172.21.15.84:0/786042854 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c15ef90 con 0x7f347c0ce390 2023-10-16T11:05:28.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.611+0000 7f347affd640 1 -- 172.21.15.84:0/786042854 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3002054629 0 0) 0x7f347c15ed60 con 0x7f347c15ae80 2023-10-16T11:05:28.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.611+0000 7f347bfff640 1 --2- 172.21.15.84:0/786042854 >> v2:172.21.15.157:3300/0 conn(0x7f347c15aaa0 0x7f347c15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:28.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.612+0000 7f3480f34640 1 --2- 172.21.15.84:0/786042854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f347c0ce390 0x7f347c070f50 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f347c0cd990 tx=0x7f347402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a865286a70eee835 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:28.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.612+0000 7f347affd640 1 -- 172.21.15.84:0/786042854 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474031830 con 0x7f347c0ce390 2023-10-16T11:05:28.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.612+0000 7f347affd640 1 -- 172.21.15.84:0/786042854 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3474031990 con 0x7f347c0ce390 2023-10-16T11:05:28.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.612+0000 7f347affd640 1 -- 172.21.15.84:0/786042854 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474031d20 con 0x7f347c0ce390 2023-10-16T11:05:28.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.613+0000 7f3482997640 1 -- 172.21.15.84:0/786042854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f347c0ce390 msgr2=0x7f347c070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:28.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.613+0000 7f3482997640 1 --2- 172.21.15.84:0/786042854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f347c0ce390 0x7f347c070f50 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f347c0cd990 tx=0x7f347402b8d0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.613+0000 7f3482997640 1 -- 172.21.15.84:0/786042854 shutdown_connections 2023-10-16T11:05:28.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.613+0000 7f3482997640 1 --2- 172.21.15.84:0/786042854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f347c0ce390 0x7f347c070f50 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.613+0000 7f3482997640 1 --2- 172.21.15.84:0/786042854 >> v2:172.21.15.157:3300/0 conn(0x7f347c15aaa0 0x7f347c15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.613+0000 7f3482997640 1 -- 172.21.15.84:0/786042854 >> 172.21.15.84:0/786042854 conn(0x7f347c0cb410 msgr2=0x7f347c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:28.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.613+0000 7f3482997640 1 -- 172.21.15.84:0/786042854 shutdown_connections 2023-10-16T11:05:28.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.613+0000 7f3482997640 1 -- 172.21.15.84:0/786042854 wait complete. 2023-10-16T11:05:28.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.615+0000 7f3482997640 1 Processor -- start 2023-10-16T11:05:28.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.615+0000 7f3482997640 1 -- start start 2023-10-16T11:05:28.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.616+0000 7f3482997640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f347c0ce390 0x7f347c147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:28.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.616+0000 7f3482997640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f347c15aaa0 0x7f347c0c2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:28.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.616+0000 7f3482997640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f347c15ae80 0x7f347c0c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:28.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.616+0000 7f3482997640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c159510 con 0x7f347c15aaa0 2023-10-16T11:05:28.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.616+0000 7f3482997640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c158f50 con 0x7f347c0ce390 2023-10-16T11:05:28.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.616+0000 7f3482997640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347c159230 con 0x7f347c15ae80 2023-10-16T11:05:28.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.616+0000 7f347b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f347c15aaa0 0x7f347c0c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:28.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.616+0000 7f347b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f347c15aaa0 0x7f347c0c2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48690/0 (socket says 172.21.15.84:48690) 2023-10-16T11:05:28.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.617+0000 7f347b7fe640 1 -- 172.21.15.84:0/810243344 learned_addr learned my addr 172.21.15.84:0/810243344 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:28.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.617+0000 7f347b7fe640 1 -- 172.21.15.84:0/810243344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f347c15ae80 msgr2=0x7f347c0c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:28.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.617+0000 7f347b7fe640 1 --2- 172.21.15.84:0/810243344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f347c15ae80 0x7f347c0c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.617+0000 7f347b7fe640 1 -- 172.21.15.84:0/810243344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f347c0ce390 msgr2=0x7f347c147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:28.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.617+0000 7f347b7fe640 1 --2- 172.21.15.84:0/810243344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f347c0ce390 0x7f347c147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.617+0000 7f347b7fe640 1 -- 172.21.15.84:0/810243344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3474009400 con 0x7f347c15aaa0 2023-10-16T11:05:28.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.617+0000 7f347b7fe640 1 --2- 172.21.15.84:0/810243344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f347c15aaa0 0x7f347c0c2490 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f346400c410 tx=0x7f346400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:28.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.617+0000 7f3478ff9640 1 -- 172.21.15.84:0/810243344 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34640074e0 con 0x7f347c15aaa0 2023-10-16T11:05:28.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.617+0000 7f3478ff9640 1 -- 172.21.15.84:0/810243344 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3464007660 con 0x7f347c15aaa0 2023-10-16T11:05:28.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.617+0000 7f3478ff9640 1 -- 172.21.15.84:0/810243344 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3464004a20 con 0x7f347c15aaa0 2023-10-16T11:05:28.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.617+0000 7f3482997640 1 -- 172.21.15.84:0/810243344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347c1567f0 con 0x7f347c15aaa0 2023-10-16T11:05:28.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.618+0000 7f3482997640 1 -- 172.21.15.84:0/810243344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347c0c5170 con 0x7f347c15aaa0 2023-10-16T11:05:28.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.619+0000 7f3482997640 1 -- 172.21.15.84:0/810243344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347c156d10 con 0x7f347c15aaa0 2023-10-16T11:05:28.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.620+0000 7f3478ff9640 1 -- 172.21.15.84:0/810243344 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 91) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f3464004b80 con 0x7f347c15aaa0 2023-10-16T11:05:28.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.621+0000 7f3478ff9640 1 --2- 172.21.15.84:0/810243344 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f345c06acc0 0x7f345c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:28.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.621+0000 7f3478ff9640 1 -- 172.21.15.84:0/810243344 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f3464089430 con 0x7f347c15aaa0 2023-10-16T11:05:28.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.621+0000 7f347bfff640 1 --2- 172.21.15.84:0/810243344 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f345c06acc0 0x7f345c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:28.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.621+0000 7f347bfff640 1 --2- 172.21.15.84:0/810243344 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f345c06acc0 0x7f345c06d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f346c008ea0 tx=0x7f346c0083d0 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:28.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.625+0000 7f3478ff9640 1 -- 172.21.15.84:0/810243344 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f347c156d10 con 0x7f347c15aaa0 2023-10-16T11:05:28.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.635+0000 7f3478ff9640 1 -- 172.21.15.84:0/810243344 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 92) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f346401ad90 con 0x7f347c15aaa0 2023-10-16T11:05:28.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.856+0000 7f3482997640 1 -- 172.21.15.84:0/810243344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f347c156d10 con 0x7f347c15aaa0 2023-10-16T11:05:28.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.857+0000 7f3478ff9640 1 -- 172.21.15.84:0/810243344 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f347c156d10 con 0x7f347c15aaa0 2023-10-16T11:05:28.859 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:28.859 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6517,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1733540616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1733540616},{"type":"v1","addr":"172.21.15.157:6833","nonce":1733540616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3815048417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3815048417},{"type":"v1","addr":"172.21.15.157:6835","nonce":3815048417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9135,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1038548935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1038548935},{"type":"v1","addr":"172.21.15.84:6836","nonce":1038548935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:09:10.652604+0000","modified":"2023-10-16T10:09:13.730838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9138},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9138":{"gid":9138,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2797149326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2797149326},{"type":"v1","addr":"172.21.15.84:6837","nonce":2797149326}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-16T11:05:28.859 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 80 2023-10-16T11:05:28.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.859+0000 7f3482997640 1 -- 172.21.15.84:0/810243344 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f345c06acc0 msgr2=0x7f345c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:28.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.859+0000 7f3482997640 1 --2- 172.21.15.84:0/810243344 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f345c06acc0 0x7f345c06d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f346c008ea0 tx=0x7f346c0083d0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.859+0000 7f3482997640 1 -- 172.21.15.84:0/810243344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f347c15aaa0 msgr2=0x7f347c0c2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:28.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.859+0000 7f3482997640 1 --2- 172.21.15.84:0/810243344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f347c15aaa0 0x7f347c0c2490 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f346400c410 tx=0x7f346400c8d0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.860+0000 7f3482997640 1 -- 172.21.15.84:0/810243344 shutdown_connections 2023-10-16T11:05:28.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.860+0000 7f3482997640 1 --2- 172.21.15.84:0/810243344 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f345c06acc0 0x7f345c06d180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.860+0000 7f3482997640 1 --2- 172.21.15.84:0/810243344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f347c15ae80 0x7f347c0c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.860+0000 7f3482997640 1 --2- 172.21.15.84:0/810243344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f347c15aaa0 0x7f347c0c2490 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.860+0000 7f3482997640 1 --2- 172.21.15.84:0/810243344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f347c0ce390 0x7f347c147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:28.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.860+0000 7f3482997640 1 -- 172.21.15.84:0/810243344 >> 172.21.15.84:0/810243344 conn(0x7f347c0cb410 msgr2=0x7f347c06f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:28.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.860+0000 7f3482997640 1 -- 172.21.15.84:0/810243344 shutdown_connections 2023-10-16T11:05:28.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:28.860+0000 7f3482997640 1 -- 172.21.15.84:0/810243344 wait complete. 2023-10-16T11:05:28.880 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:05:29.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.075+0000 7fecfce51640 1 Processor -- start 2023-10-16T11:05:29.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.076+0000 7fecfce51640 1 -- start start 2023-10-16T11:05:29.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.077+0000 7fecfce51640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf80adb50 0x7fecf806f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:29.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.077+0000 7fecfce51640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fecf8070fd0 0x7fecf80b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:29.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.077+0000 7fecfce51640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fecf80adf30 0x7fecf80b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:29.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.077+0000 7fecfce51640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf8071450 con 0x7fecf8070fd0 2023-10-16T11:05:29.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.077+0000 7fecfce51640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf80cc600 con 0x7fecf80adf30 2023-10-16T11:05:29.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.077+0000 7fecfce51640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf80cc8e0 con 0x7fecf80adb50 2023-10-16T11:05:29.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.078+0000 7feceffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fecf8070fd0 0x7fecf80b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:29.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.078+0000 7feceffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fecf8070fd0 0x7fecf80b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48700/0 (socket says 172.21.15.84:48700) 2023-10-16T11:05:29.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.078+0000 7feceffff640 1 -- 172.21.15.84:0/1982769707 learned_addr learned my addr 172.21.15.84:0/1982769707 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:29.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.078+0000 7feceffff640 1 -- 172.21.15.84:0/1982769707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf80adb50 msgr2=0x7fecf806f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:29.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.078+0000 7feceffff640 1 --2- 172.21.15.84:0/1982769707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf80adb50 0x7fecf806f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.078+0000 7feceffff640 1 -- 172.21.15.84:0/1982769707 >> v2:172.21.15.157:3300/0 conn(0x7fecf80adf30 msgr2=0x7fecf80b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:29.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.078+0000 7feceffff640 1 --2- 172.21.15.84:0/1982769707 >> v2:172.21.15.157:3300/0 conn(0x7fecf80adf30 0x7fecf80b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.078+0000 7feceffff640 1 -- 172.21.15.84:0/1982769707 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf80ccbc0 con 0x7fecf8070fd0 2023-10-16T11:05:29.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.079+0000 7feceffff640 1 --2- 172.21.15.84:0/1982769707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8070fd0 0x7fecf80b0410 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fecf806d840 tx=0x7fece002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c766f92d568975 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:29.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.079+0000 7fecf5d9b640 1 -- 172.21.15.84:0/1982769707 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0031880 con 0x7fecf8070fd0 2023-10-16T11:05:29.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.079+0000 7fecf5d9b640 1 -- 172.21.15.84:0/1982769707 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece00319e0 con 0x7fecf8070fd0 2023-10-16T11:05:29.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.079+0000 7fecf5d9b640 1 -- 172.21.15.84:0/1982769707 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0031d70 con 0x7fecf8070fd0 2023-10-16T11:05:29.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.080+0000 7fecfce51640 1 -- 172.21.15.84:0/1982769707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8070fd0 msgr2=0x7fecf80b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:29.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.080+0000 7fecfce51640 1 --2- 172.21.15.84:0/1982769707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8070fd0 0x7fecf80b0410 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fecf806d840 tx=0x7fece002b920 comp rx=0 tx=0).stop 2023-10-16T11:05:29.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.080+0000 7fecfce51640 1 -- 172.21.15.84:0/1982769707 shutdown_connections 2023-10-16T11:05:29.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.080+0000 7fecfce51640 1 --2- 172.21.15.84:0/1982769707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8070fd0 0x7fecf80b0410 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.080+0000 7fecfce51640 1 --2- 172.21.15.84:0/1982769707 >> v2:172.21.15.157:3300/0 conn(0x7fecf80adf30 0x7fecf80b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.080+0000 7fecfce51640 1 --2- 172.21.15.84:0/1982769707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf80adb50 0x7fecf806f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.080+0000 7fecfce51640 1 -- 172.21.15.84:0/1982769707 >> 172.21.15.84:0/1982769707 conn(0x7fecf80cb410 msgr2=0x7fecf80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:29.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.080+0000 7fecfce51640 1 -- 172.21.15.84:0/1982769707 shutdown_connections 2023-10-16T11:05:29.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.080+0000 7fecfce51640 1 -- 172.21.15.84:0/1982769707 wait complete. 2023-10-16T11:05:29.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.082+0000 7fecfce51640 1 Processor -- start 2023-10-16T11:05:29.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.082+0000 7fecfce51640 1 -- start start 2023-10-16T11:05:29.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.083+0000 7fecfce51640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8070fd0 0x7fecf80b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:29.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.083+0000 7fecfce51640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf80adb50 0x7fecf80b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:29.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.083+0000 7fecfce51640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf80adf30 0x7fecf815df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:29.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.083+0000 7fecfce51640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf80b1a90 con 0x7fecf8070fd0 2023-10-16T11:05:29.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.083+0000 7fecfce51640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf80b1d70 con 0x7fecf80adb50 2023-10-16T11:05:29.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.083+0000 7fecfce51640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf80b2050 con 0x7fecf80adf30 2023-10-16T11:05:29.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.083+0000 7fecf659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8070fd0 0x7fecf80b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:29.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.083+0000 7fecf659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8070fd0 0x7fecf80b2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48706/0 (socket says 172.21.15.84:48706) 2023-10-16T11:05:29.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.083+0000 7fecf659c640 1 -- 172.21.15.84:0/3785272146 learned_addr learned my addr 172.21.15.84:0/3785272146 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:29.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.083+0000 7fecf659c640 1 -- 172.21.15.84:0/3785272146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf80adf30 msgr2=0x7fecf815df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:29.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.083+0000 7fecf659c640 1 --2- 172.21.15.84:0/3785272146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf80adf30 0x7fecf815df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.083+0000 7fecf659c640 1 -- 172.21.15.84:0/3785272146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf80adb50 msgr2=0x7fecf80b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:29.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.083+0000 7fecf659c640 1 --2- 172.21.15.84:0/3785272146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf80adb50 0x7fecf80b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.083+0000 7fecf659c640 1 -- 172.21.15.84:0/3785272146 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece00094d0 con 0x7fecf8070fd0 2023-10-16T11:05:29.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.084+0000 7fecf659c640 1 --2- 172.21.15.84:0/3785272146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8070fd0 0x7fecf80b2b30 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fece800b4a0 tx=0x7fece800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:29.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.084+0000 7feceeffd640 1 -- 172.21.15.84:0/3785272146 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece8010610 con 0x7fecf8070fd0 2023-10-16T11:05:29.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.084+0000 7feceeffd640 1 -- 172.21.15.84:0/3785272146 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece8010770 con 0x7fecf8070fd0 2023-10-16T11:05:29.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.084+0000 7feceeffd640 1 -- 172.21.15.84:0/3785272146 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece8010b20 con 0x7fecf8070fd0 2023-10-16T11:05:29.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.084+0000 7fecfce51640 1 -- 172.21.15.84:0/3785272146 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf80b2330 con 0x7fecf8070fd0 2023-10-16T11:05:29.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.084+0000 7fecfce51640 1 -- 172.21.15.84:0/3785272146 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf815c330 con 0x7fecf8070fd0 2023-10-16T11:05:29.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.085+0000 7fecfce51640 1 -- 172.21.15.84:0/3785272146 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf80b2850 con 0x7fecf8070fd0 2023-10-16T11:05:29.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.086+0000 7feceeffd640 1 -- 172.21.15.84:0/3785272146 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 92) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fece8010c80 con 0x7fecf8070fd0 2023-10-16T11:05:29.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.087+0000 7feceeffd640 1 --2- 172.21.15.84:0/3785272146 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fecd806af00 0x7fecd806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:29.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.087+0000 7feceeffd640 1 -- 172.21.15.84:0/3785272146 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6527+0+0 (secure 0 0 0) 0x7fece80895f0 con 0x7fecf8070fd0 2023-10-16T11:05:29.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.087+0000 7feceffff640 1 --2- 172.21.15.84:0/3785272146 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fecd806af00 0x7fecd806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:29.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.088+0000 7feceffff640 1 --2- 172.21.15.84:0/3785272146 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fecd806af00 0x7fecd806d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fece0036000 tx=0x7fece0030000 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:29.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.092+0000 7feceeffd640 1 -- 172.21.15.84:0/3785272146 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fecf80b2850 con 0x7fecf8070fd0 2023-10-16T11:05:29.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.288+0000 7fecfce51640 1 -- 172.21.15.84:0/3785272146 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fecf80b2850 con 0x7fecf8070fd0 2023-10-16T11:05:29.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.291+0000 7feceeffd640 1 -- 172.21.15.84:0/3785272146 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+17734 (secure 0 0 0) 0x7fece805ce60 con 0x7fecf8070fd0 2023-10-16T11:05:29.293 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:29.293 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":159,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:05:26.145788+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:09:08.490425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-16T10:09:09.666051+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":158,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.26","mappings":[{"from":2,"to":0}]},{"pgid":"19.3c","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1606872290":"2023-10-17T11:05:26.145693+0000","172.21.15.84:6833/3746729142":"2023-10-17T11:05:26.145693+0000","172.21.15.84:0/3209795865":"2023-10-17T11:05:26.145693+0000","172.21.15.84:0/2439552742":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/384925658":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/3023009161":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/683905531":"2023-10-17T11:05:26.145693+0000","172.21.15.84:0/3754993953":"2023-10-17T11:05:26.145693+0000","172.21.15.84:0/3629412882":"2023-10-17T11:05:26.145693+0000","172.21.15.84:0/1762476558":"2023-10-17T11:05:26.145693+0000","172.21.15.84:0/870751622":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/300305981":"2023-10-17T10:09:25.208422+0000","172.21.15.84:6832/3746729142":"2023-10-17T11:05:26.145693+0000","172.21.15.84:6833/205480762":"2023-10-17T10:09:25.208422+0000","172.21.15.84:6832/205480762":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/1358444053":"2023-10-17T10:09:25.208422+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:05:29.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.294+0000 7fecfce51640 1 -- 172.21.15.84:0/3785272146 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fecd806af00 msgr2=0x7fecd806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:29.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.294+0000 7fecfce51640 1 --2- 172.21.15.84:0/3785272146 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fecd806af00 0x7fecd806d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fece0036000 tx=0x7fece0030000 comp rx=0 tx=0).stop 2023-10-16T11:05:29.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.294+0000 7fecfce51640 1 -- 172.21.15.84:0/3785272146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8070fd0 msgr2=0x7fecf80b2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:29.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.294+0000 7fecfce51640 1 --2- 172.21.15.84:0/3785272146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8070fd0 0x7fecf80b2b30 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fece800b4a0 tx=0x7fece800b960 comp rx=0 tx=0).stop 2023-10-16T11:05:29.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.294+0000 7fecfce51640 1 -- 172.21.15.84:0/3785272146 shutdown_connections 2023-10-16T11:05:29.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.294+0000 7fecfce51640 1 --2- 172.21.15.84:0/3785272146 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fecd806af00 0x7fecd806d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.294+0000 7fecfce51640 1 --2- 172.21.15.84:0/3785272146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf80adf30 0x7fecf815df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.294+0000 7fecfce51640 1 --2- 172.21.15.84:0/3785272146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf80adb50 0x7fecf80b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.294+0000 7fecfce51640 1 --2- 172.21.15.84:0/3785272146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf8070fd0 0x7fecf80b2b30 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.294+0000 7fecfce51640 1 -- 172.21.15.84:0/3785272146 >> 172.21.15.84:0/3785272146 conn(0x7fecf80cb410 msgr2=0x7fecf8163dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:29.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.294+0000 7fecfce51640 1 -- 172.21.15.84:0/3785272146 shutdown_connections 2023-10-16T11:05:29.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.294+0000 7fecfce51640 1 -- 172.21.15.84:0/3785272146 wait complete. 2023-10-16T11:05:29.314 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T11:05:29.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.517+0000 7f2d0e10f640 1 Processor -- start 2023-10-16T11:05:29.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.518+0000 7f2d0e10f640 1 -- start start 2023-10-16T11:05:29.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.519+0000 7f2d0e10f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d080caa80 0x7f2d08069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:29.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.519+0000 7f2d0e10f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2d0806a2e0 0x7f2d081650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:29.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.519+0000 7f2d0e10f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d080caec0 con 0x7f2d080addb0 2023-10-16T11:05:29.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.519+0000 7f2d0e10f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d08161a50 con 0x7f2d0806a2e0 2023-10-16T11:05:29.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.519+0000 7f2d0e10f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0806a750 con 0x7f2d080caa80 2023-10-16T11:05:29.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.519+0000 7f2d077fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2d080addb0 0x7f2d080c9000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37838/0 (socket says 172.21.15.84:37838) 2023-10-16T11:05:29.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.519+0000 7f2d077fe640 1 -- 172.21.15.84:0/405468815 learned_addr learned my addr 172.21.15.84:0/405468815 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:29.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.520+0000 7f2cfffff640 1 --2- 172.21.15.84:0/405468815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d080caa80 0x7f2d08069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:29.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.520+0000 7f2d07fff640 1 --2- 172.21.15.84:0/405468815 >> v2:172.21.15.157:3300/0 conn(0x7f2d0806a2e0 0x7f2d081650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:29.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.520+0000 7f2d06ffd640 1 -- 172.21.15.84:0/405468815 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827144438 0 0) 0x7f2d080caec0 con 0x7f2d080addb0 2023-10-16T11:05:29.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.520+0000 7f2d06ffd640 1 -- 172.21.15.84:0/405468815 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cf4003520 con 0x7f2d080addb0 2023-10-16T11:05:29.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.520+0000 7f2d06ffd640 1 -- 172.21.15.84:0/405468815 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3351770729 0 0) 0x7f2cf4003520 con 0x7f2d080addb0 2023-10-16T11:05:29.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.520+0000 7f2d06ffd640 1 -- 172.21.15.84:0/405468815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d080caa80 msgr2=0x7f2d08069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:29.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.520+0000 7f2d06ffd640 1 --2- 172.21.15.84:0/405468815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d080caa80 0x7f2d08069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.520+0000 7f2d06ffd640 1 -- 172.21.15.84:0/405468815 >> v2:172.21.15.157:3300/0 conn(0x7f2d0806a2e0 msgr2=0x7f2d081650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:29.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.520+0000 7f2d06ffd640 1 --2- 172.21.15.84:0/405468815 >> v2:172.21.15.157:3300/0 conn(0x7f2d0806a2e0 0x7f2d081650d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.520+0000 7f2d06ffd640 1 -- 172.21.15.84:0/405468815 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d080cb140 con 0x7f2d080addb0 2023-10-16T11:05:29.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.520+0000 7f2d06ffd640 1 -- 172.21.15.84:0/405468815 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f2cf8001670 con 0x7f2d080addb0 2023-10-16T11:05:29.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.520+0000 7f2d06ffd640 1 -- 172.21.15.84:0/405468815 >> v1:172.21.15.84:6789/0 conn(0x7f2d080addb0 legacy=0x7f2d080c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:29.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.521+0000 7f2d06ffd640 1 --2- 172.21.15.84:0/405468815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2cf40039a0 0x7f2cf4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:29.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.521+0000 7f2d06ffd640 1 --2- 172.21.15.84:0/405468815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf40047c0 0x7f2cf4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:29.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.521+0000 7f2d06ffd640 1 --2- 172.21.15.84:0/405468815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d080caa80 0x7f2cf4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:29.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.521+0000 7f2d06ffd640 1 -- 172.21.15.84:0/405468815 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d08161a50 con 0x7f2cf40039a0 2023-10-16T11:05:29.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.521+0000 7f2d06ffd640 1 -- 172.21.15.84:0/405468815 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0806a750 con 0x7f2cf40047c0 2023-10-16T11:05:29.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.521+0000 7f2d06ffd640 1 -- 172.21.15.84:0/405468815 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf400b980 con 0x7f2d080caa80 2023-10-16T11:05:29.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.521+0000 7f2d077fe640 1 --2- 172.21.15.84:0/405468815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2cf40039a0 0x7f2cf4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:29.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.521+0000 7f2cfffff640 1 --2- 172.21.15.84:0/405468815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf40047c0 0x7f2cf4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:29.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.521+0000 7f2d077fe640 1 -- 172.21.15.84:0/405468815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d080caa80 msgr2=0x7f2cf4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:29.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.521+0000 7f2d077fe640 1 --2- 172.21.15.84:0/405468815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d080caa80 0x7f2cf4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.521+0000 7f2d077fe640 1 -- 172.21.15.84:0/405468815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf40047c0 msgr2=0x7f2cf4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:29.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.521+0000 7f2d077fe640 1 --2- 172.21.15.84:0/405468815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf40047c0 0x7f2cf4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.521+0000 7f2d077fe640 1 -- 172.21.15.84:0/405468815 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf4003520 con 0x7f2cf40039a0 2023-10-16T11:05:29.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.522+0000 7f2d077fe640 1 --2- 172.21.15.84:0/405468815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2cf40039a0 0x7f2cf4003dc0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800c240 tx=0x7f2cf40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caab0be01c886f20 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:29.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.523+0000 7f2d06ffd640 1 -- 172.21.15.84:0/405468815 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8005ad0 con 0x7f2cf40039a0 2023-10-16T11:05:29.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.523+0000 7f2d06ffd640 1 -- 172.21.15.84:0/405468815 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf8005c50 con 0x7f2cf40039a0 2023-10-16T11:05:29.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.523+0000 7f2d06ffd640 1 -- 172.21.15.84:0/405468815 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf802c8b0 con 0x7f2cf40039a0 2023-10-16T11:05:29.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.523+0000 7f2d0e10f640 1 -- 172.21.15.84:0/405468815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2cf40039a0 msgr2=0x7f2cf4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:29.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.523+0000 7f2d0e10f640 1 --2- 172.21.15.84:0/405468815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2cf40039a0 0x7f2cf4003dc0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800c240 tx=0x7f2cf40057f0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.523+0000 7f2d07fff640 1 -- 172.21.15.84:0/405468815 reap_dead start 2023-10-16T11:05:29.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.523+0000 7f2d0e10f640 1 -- 172.21.15.84:0/405468815 shutdown_connections 2023-10-16T11:05:29.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.523+0000 7f2d0e10f640 1 -- 172.21.15.84:0/405468815 >> 172.21.15.84:0/405468815 conn(0x7f2d080ce9b0 msgr2=0x7f2d080cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:29.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.523+0000 7f2d0e10f640 1 -- 172.21.15.84:0/405468815 shutdown_connections 2023-10-16T11:05:29.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.523+0000 7f2d0e10f640 1 -- 172.21.15.84:0/405468815 wait complete. 2023-10-16T11:05:29.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.525+0000 7f2d0e10f640 1 Processor -- start 2023-10-16T11:05:29.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.525+0000 7f2d0e10f640 1 -- start start 2023-10-16T11:05:29.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.526+0000 7f2d0e10f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf40039a0 0x7f2d080c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:29.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.526+0000 7f2d0e10f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d080b15a0 0x7f2d081f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:29.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.526+0000 7f2d0e10f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d0815e580 0x7f2d08163da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:29.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.526+0000 7f2d0e10f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0815c380 con 0x7f2d080b15a0 2023-10-16T11:05:29.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.526+0000 7f2d0e10f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0815c660 con 0x7f2cf40039a0 2023-10-16T11:05:29.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.526+0000 7f2d0e10f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0815c940 con 0x7f2d0815e580 2023-10-16T11:05:29.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.526+0000 7f2cfffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d080b15a0 0x7f2d081f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:29.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.526+0000 7f2cfffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d080b15a0 0x7f2d081f63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48730/0 (socket says 172.21.15.84:48730) 2023-10-16T11:05:29.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.526+0000 7f2cfffff640 1 -- 172.21.15.84:0/1633160748 learned_addr learned my addr 172.21.15.84:0/1633160748 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:29.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.527+0000 7f2cfffff640 1 -- 172.21.15.84:0/1633160748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d0815e580 msgr2=0x7f2d08163da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:29.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.527+0000 7f2d07fff640 1 --2- 172.21.15.84:0/1633160748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d0815e580 0x7f2d08163da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:29.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.527+0000 7f2d077fe640 1 --2- 172.21.15.84:0/1633160748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf40039a0 0x7f2d080c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:29.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.527+0000 7f2cfffff640 1 --2- 172.21.15.84:0/1633160748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d0815e580 0x7f2d08163da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.527+0000 7f2cfffff640 1 -- 172.21.15.84:0/1633160748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf40039a0 msgr2=0x7f2d080c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:29.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.527+0000 7f2cfffff640 1 --2- 172.21.15.84:0/1633160748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf40039a0 0x7f2d080c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.527+0000 7f2cfffff640 1 -- 172.21.15.84:0/1633160748 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0815cc20 con 0x7f2d080b15a0 2023-10-16T11:05:29.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.527+0000 7f2d07fff640 1 --2- 172.21.15.84:0/1633160748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d0815e580 0x7f2d08163da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:29.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.527+0000 7f2cfffff640 1 --2- 172.21.15.84:0/1633160748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d080b15a0 0x7f2d081f63d0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0011540 tx=0x7f2cf0011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.527+0000 7f2d04ff9640 1 -- 172.21.15.84:0/1633160748 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0015930 con 0x7f2d080b15a0 2023-10-16T11:05:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.527+0000 7f2d04ff9640 1 -- 172.21.15.84:0/1633160748 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf0015a90 con 0x7f2d080b15a0 2023-10-16T11:05:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.527+0000 7f2d0e10f640 1 -- 172.21.15.84:0/1633160748 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0815cf00 con 0x7f2d080b15a0 2023-10-16T11:05:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.527+0000 7f2d04ff9640 1 -- 172.21.15.84:0/1633160748 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0015e60 con 0x7f2d080b15a0 2023-10-16T11:05:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.528+0000 7f2d0e10f640 1 -- 172.21.15.84:0/1633160748 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0815bfa0 con 0x7f2d080b15a0 2023-10-16T11:05:29.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.529+0000 7f2d0e10f640 1 -- 172.21.15.84:0/1633160748 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0804e160 con 0x7f2d080b15a0 2023-10-16T11:05:29.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.530+0000 7f2d04ff9640 1 -- 172.21.15.84:0/1633160748 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 92) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f2cf000f460 con 0x7f2d080b15a0 2023-10-16T11:05:29.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.531+0000 7f2d04ff9640 1 --2- 172.21.15.84:0/1633160748 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f2ce806af00 0x7f2ce806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:29.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.531+0000 7f2d04ff9640 1 -- 172.21.15.84:0/1633160748 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f2cf008a270 con 0x7f2d080b15a0 2023-10-16T11:05:29.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.531+0000 7f2d077fe640 1 --2- 172.21.15.84:0/1633160748 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f2ce806af00 0x7f2ce806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:29.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.532+0000 7f2d077fe640 1 --2- 172.21.15.84:0/1633160748 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f2ce806af00 0x7f2ce806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8034000 tx=0x7f2cf8009030 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:29.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.536+0000 7f2d04ff9640 1 -- 172.21.15.84:0/1633160748 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2cf0054f00 con 0x7f2d080b15a0 2023-10-16T11:05:29.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.770+0000 7f2d0e10f640 1 -- 172.21.15.84:0/1633160748 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2d0806c5b0 con 0x7f2d080b15a0 2023-10-16T11:05:29.773 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-16T11:05:29.771+0000 7fca0c7ff640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:05:29.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.774+0000 7f2d04ff9640 1 -- 172.21.15.84:0/1633160748 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2cf0058520 con 0x7f2d080b15a0 2023-10-16T11:05:29.776 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:05:29.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.778+0000 7f2d0e10f640 1 -- 172.21.15.84:0/1633160748 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f2ce806af00 msgr2=0x7f2ce806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:29.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.778+0000 7f2d0e10f640 1 --2- 172.21.15.84:0/1633160748 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f2ce806af00 0x7f2ce806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8034000 tx=0x7f2cf8009030 comp rx=0 tx=0).stop 2023-10-16T11:05:29.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.778+0000 7f2d0e10f640 1 -- 172.21.15.84:0/1633160748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d080b15a0 msgr2=0x7f2d081f63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:29.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.778+0000 7f2d0e10f640 1 --2- 172.21.15.84:0/1633160748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d080b15a0 0x7f2d081f63d0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0011540 tx=0x7f2cf0011a00 comp rx=0 tx=0).stop 2023-10-16T11:05:29.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.780+0000 7f2d0e10f640 1 -- 172.21.15.84:0/1633160748 shutdown_connections 2023-10-16T11:05:29.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.780+0000 7f2d0e10f640 1 --2- 172.21.15.84:0/1633160748 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f2ce806af00 0x7f2ce806d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.780+0000 7f2d0e10f640 1 --2- 172.21.15.84:0/1633160748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d0815e580 0x7f2d08163da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.780+0000 7f2d0e10f640 1 --2- 172.21.15.84:0/1633160748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d080b15a0 0x7f2d081f63d0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.780+0000 7f2d0e10f640 1 --2- 172.21.15.84:0/1633160748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf40039a0 0x7f2d080c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:29.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.780+0000 7f2d0e10f640 1 -- 172.21.15.84:0/1633160748 >> 172.21.15.84:0/1633160748 conn(0x7f2d080ce9b0 msgr2=0x7f2d080cd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:29.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.781+0000 7f2d0e10f640 1 -- 172.21.15.84:0/1633160748 shutdown_connections 2023-10-16T11:05:29.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.781+0000 7f2d0e10f640 1 -- 172.21.15.84:0/1633160748 wait complete. 2023-10-16T11:05:29.801 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T11:05:29.854 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T11:05:30.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:29.999+0000 7fe676f78640 1 Processor -- start 2023-10-16T11:05:30.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.000+0000 7fe676f78640 1 -- start start 2023-10-16T11:05:30.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.001+0000 7fe676f78640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe67006a4b0 0x7fe6700ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:30.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.001+0000 7fe676f78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6700c9ca0 0x7fe6700ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:30.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.001+0000 7fe676f78640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6700cb030 con 0x7fe67015ad70 2023-10-16T11:05:30.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.001+0000 7fe676f78640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6700cae30 con 0x7fe67006a4b0 2023-10-16T11:05:30.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.001+0000 7fe676f78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6700c91c0 con 0x7fe6700c9ca0 2023-10-16T11:05:30.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.001+0000 7fe66ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe67006a4b0 0x7fe6700ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:30.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.002+0000 7fe66ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe67006a4b0 0x7fe6700ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48768/0 (socket says 172.21.15.84:48768) 2023-10-16T11:05:30.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.002+0000 7fe66ffff640 1 -- 172.21.15.84:0/490988061 learned_addr learned my addr 172.21.15.84:0/490988061 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:30.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.002+0000 7fe675515640 1 --2- 172.21.15.84:0/490988061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6700c9ca0 0x7fe6700ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:30.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.002+0000 7fe66ffff640 1 -- 172.21.15.84:0/490988061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6700c9ca0 msgr2=0x7fe6700ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:30.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.002+0000 7fe66ffff640 1 --2- 172.21.15.84:0/490988061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6700c9ca0 0x7fe6700ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:30.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.002+0000 7fe66ffff640 1 -- 172.21.15.84:0/490988061 >> v1:172.21.15.157:6789/0 conn(0x7fe67015ad70 legacy=0x7fe670069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:30.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.002+0000 7fe66ffff640 1 -- 172.21.15.84:0/490988061 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6700c94a0 con 0x7fe67006a4b0 2023-10-16T11:05:30.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.002+0000 7fe675515640 1 --2- 172.21.15.84:0/490988061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6700c9ca0 0x7fe6700ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:30.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.003+0000 7fe66ffff640 1 --2- 172.21.15.84:0/490988061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe67006a4b0 0x7fe6700ca3b0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009870 tx=0x7fe65c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa89b74ae4b9a03 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:30.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.003+0000 7fe66f7fe640 1 -- 172.21.15.84:0/490988061 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c031640 con 0x7fe67006a4b0 2023-10-16T11:05:30.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.004+0000 7fe66f7fe640 1 -- 172.21.15.84:0/490988061 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65c0317a0 con 0x7fe67006a4b0 2023-10-16T11:05:30.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.004+0000 7fe66f7fe640 1 -- 172.21.15.84:0/490988061 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c031b30 con 0x7fe67006a4b0 2023-10-16T11:05:30.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.004+0000 7fe676f78640 1 -- 172.21.15.84:0/490988061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe67006a4b0 msgr2=0x7fe6700ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:30.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.004+0000 7fe676f78640 1 --2- 172.21.15.84:0/490988061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe67006a4b0 0x7fe6700ca3b0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009870 tx=0x7fe65c02b8f0 comp rx=0 tx=0).stop 2023-10-16T11:05:30.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.004+0000 7fe676f78640 1 -- 172.21.15.84:0/490988061 shutdown_connections 2023-10-16T11:05:30.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.004+0000 7fe676f78640 1 --2- 172.21.15.84:0/490988061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6700c9ca0 0x7fe6700ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:30.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.004+0000 7fe676f78640 1 --2- 172.21.15.84:0/490988061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe67006a4b0 0x7fe6700ca3b0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:30.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.004+0000 7fe676f78640 1 -- 172.21.15.84:0/490988061 >> 172.21.15.84:0/490988061 conn(0x7fe6700ce9b0 msgr2=0x7fe6700cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:30.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.004+0000 7fe676f78640 1 -- 172.21.15.84:0/490988061 shutdown_connections 2023-10-16T11:05:30.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.004+0000 7fe676f78640 1 -- 172.21.15.84:0/490988061 wait complete. 2023-10-16T11:05:30.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.006+0000 7fe676f78640 1 Processor -- start 2023-10-16T11:05:30.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.006+0000 7fe676f78640 1 -- start start 2023-10-16T11:05:30.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.007+0000 7fe676f78640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe67006a4b0 0x7fe670159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:30.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.007+0000 7fe676f78640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6700c9ca0 0x7fe6700c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:30.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.007+0000 7fe676f78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe67015ad70 0x7fe6700c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:30.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.007+0000 7fe676f78640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6700af1c0 con 0x7fe67006a4b0 2023-10-16T11:05:30.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.007+0000 7fe676f78640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6700ae2f0 con 0x7fe6700c9ca0 2023-10-16T11:05:30.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.007+0000 7fe676f78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6700c5320 con 0x7fe67015ad70 2023-10-16T11:05:30.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.007+0000 7fe674d14640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe67006a4b0 0x7fe670159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:30.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.007+0000 7fe674d14640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe67006a4b0 0x7fe670159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48784/0 (socket says 172.21.15.84:48784) 2023-10-16T11:05:30.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.007+0000 7fe674d14640 1 -- 172.21.15.84:0/1146015936 learned_addr learned my addr 172.21.15.84:0/1146015936 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:30.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.008+0000 7fe675515640 1 --2- 172.21.15.84:0/1146015936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe67015ad70 0x7fe6700c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:30.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.008+0000 7fe674d14640 1 -- 172.21.15.84:0/1146015936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe67015ad70 msgr2=0x7fe6700c4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.008+0000 7fe66ffff640 1 --2- 172.21.15.84:0/1146015936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6700c9ca0 0x7fe6700c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.008+0000 7fe674d14640 1 --2- 172.21.15.84:0/1146015936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe67015ad70 0x7fe6700c4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.008+0000 7fe674d14640 1 -- 172.21.15.84:0/1146015936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6700c9ca0 msgr2=0x7fe6700c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.008+0000 7fe674d14640 1 --2- 172.21.15.84:0/1146015936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6700c9ca0 0x7fe6700c3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.008+0000 7fe674d14640 1 -- 172.21.15.84:0/1146015936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c0094d0 con 0x7fe67006a4b0 2023-10-16T11:05:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.008+0000 7fe674d14640 1 --2- 172.21.15.84:0/1146015936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe67006a4b0 0x7fe670159dd0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fe66400b3f0 tx=0x7fe66400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.008+0000 7fe66d7fa640 1 -- 172.21.15.84:0/1146015936 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe664011830 con 0x7fe67006a4b0 2023-10-16T11:05:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.008+0000 7fe66d7fa640 1 -- 172.21.15.84:0/1146015936 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe664011990 con 0x7fe67006a4b0 2023-10-16T11:05:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.008+0000 7fe66d7fa640 1 -- 172.21.15.84:0/1146015936 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe664011d40 con 0x7fe67006a4b0 2023-10-16T11:05:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.008+0000 7fe676f78640 1 -- 172.21.15.84:0/1146015936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6700ae540 con 0x7fe67006a4b0 2023-10-16T11:05:30.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.009+0000 7fe676f78640 1 -- 172.21.15.84:0/1146015936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6700aeb00 con 0x7fe67006a4b0 2023-10-16T11:05:30.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.010+0000 7fe676f78640 1 -- 172.21.15.84:0/1146015936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67004e160 con 0x7fe67006a4b0 2023-10-16T11:05:30.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.011+0000 7fe66d7fa640 1 -- 172.21.15.84:0/1146015936 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 92) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fe66401a400 con 0x7fe67006a4b0 2023-10-16T11:05:30.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.012+0000 7fe66d7fa640 1 --2- 172.21.15.84:0/1146015936 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fe65406af00 0x7fe65406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:30.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.012+0000 7fe66d7fa640 1 -- 172.21.15.84:0/1146015936 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6569+0+0 (secure 0 0 0) 0x7fe664089960 con 0x7fe67006a4b0 2023-10-16T11:05:30.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.012+0000 7fe66ffff640 1 --2- 172.21.15.84:0/1146015936 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fe65406af00 0x7fe65406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:30.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.013+0000 7fe66ffff640 1 --2- 172.21.15.84:0/1146015936 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fe65406af00 0x7fe65406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe65c036000 tx=0x7fe65c03c040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:30.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.016+0000 7fe66d7fa640 1 -- 172.21.15.84:0/1146015936 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe664054530 con 0x7fe67006a4b0 2023-10-16T11:05:30.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.246+0000 7fe676f78640 1 -- 172.21.15.84:0/1146015936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe6700736e0 con 0x7fe67006a4b0 2023-10-16T11:05:30.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.777+0000 7fe66d7fa640 1 -- 172.21.15.84:0/1146015936 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v82) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe664057b50 con 0x7fe67006a4b0 2023-10-16T11:05:30.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.781+0000 7fe676f78640 1 -- 172.21.15.84:0/1146015936 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fe65406af00 msgr2=0x7fe65406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:30.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.781+0000 7fe676f78640 1 --2- 172.21.15.84:0/1146015936 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fe65406af00 0x7fe65406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe65c036000 tx=0x7fe65c03c040 comp rx=0 tx=0).stop 2023-10-16T11:05:30.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.781+0000 7fe676f78640 1 -- 172.21.15.84:0/1146015936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe67006a4b0 msgr2=0x7fe670159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:30.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.781+0000 7fe676f78640 1 --2- 172.21.15.84:0/1146015936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe67006a4b0 0x7fe670159dd0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fe66400b3f0 tx=0x7fe66400b8b0 comp rx=0 tx=0).stop 2023-10-16T11:05:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.781+0000 7fe676f78640 1 -- 172.21.15.84:0/1146015936 shutdown_connections 2023-10-16T11:05:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.781+0000 7fe676f78640 1 --2- 172.21.15.84:0/1146015936 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fe65406af00 0x7fe65406d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.781+0000 7fe676f78640 1 --2- 172.21.15.84:0/1146015936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe67015ad70 0x7fe6700c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.781+0000 7fe676f78640 1 --2- 172.21.15.84:0/1146015936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6700c9ca0 0x7fe6700c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.781+0000 7fe676f78640 1 --2- 172.21.15.84:0/1146015936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe67006a4b0 0x7fe670159dd0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:30.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.781+0000 7fe676f78640 1 -- 172.21.15.84:0/1146015936 >> 172.21.15.84:0/1146015936 conn(0x7fe6700ce9b0 msgr2=0x7fe670069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:30.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.781+0000 7fe676f78640 1 -- 172.21.15.84:0/1146015936 shutdown_connections 2023-10-16T11:05:30.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:30.781+0000 7fe676f78640 1 -- 172.21.15.84:0/1146015936 wait complete. 2023-10-16T11:05:30.808 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T11:05:31.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.019+0000 7f61c4232640 1 Processor -- start 2023-10-16T11:05:31.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.019+0000 7f61c4232640 1 -- start start 2023-10-16T11:05:31.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.020+0000 7f61c4232640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61bc0b0360 0x7f61bc0b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:31.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.020+0000 7f61c4232640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f61bc0ca510 0x7f61bc0ca920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:31.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.020+0000 7f61c4232640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61bc15b310 con 0x7f61bc15ac10 2023-10-16T11:05:31.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.020+0000 7f61c4232640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61bc15f800 con 0x7f61bc0ca510 2023-10-16T11:05:31.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.020+0000 7f61c4232640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61bc0b0d30 con 0x7f61bc0b0360 2023-10-16T11:05:31.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.021+0000 7f61c1fce640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f61bc15ac10 0x7f61bc06c730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37856/0 (socket says 172.21.15.84:37856) 2023-10-16T11:05:31.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.021+0000 7f61c1fce640 1 -- 172.21.15.84:0/964299347 learned_addr learned my addr 172.21.15.84:0/964299347 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:31.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.021+0000 7f61c27cf640 1 --2- 172.21.15.84:0/964299347 >> v2:172.21.15.84:3300/0 conn(0x7f61bc0ca510 0x7f61bc0ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:31.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.021+0000 7f61c0fcc640 1 -- 172.21.15.84:0/964299347 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141324262 0 0) 0x7f61bc15b310 con 0x7f61bc15ac10 2023-10-16T11:05:31.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.021+0000 7f61c17cd640 1 --2- 172.21.15.84:0/964299347 >> v2:172.21.15.157:3300/0 conn(0x7f61bc0b0360 0x7f61bc0b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:31.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.021+0000 7f61c0fcc640 1 -- 172.21.15.84:0/964299347 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61a8003520 con 0x7f61bc15ac10 2023-10-16T11:05:31.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.021+0000 7f61c27cf640 1 -- 172.21.15.84:0/964299347 >> v2:172.21.15.157:3300/0 conn(0x7f61bc0b0360 msgr2=0x7f61bc0b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:31.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.021+0000 7f61c27cf640 1 --2- 172.21.15.84:0/964299347 >> v2:172.21.15.157:3300/0 conn(0x7f61bc0b0360 0x7f61bc0b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:31.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.021+0000 7f61c27cf640 1 -- 172.21.15.84:0/964299347 >> v1:172.21.15.84:6789/0 conn(0x7f61bc15ac10 legacy=0x7f61bc06c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:31.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.021+0000 7f61c27cf640 1 -- 172.21.15.84:0/964299347 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc06f640 con 0x7f61bc0ca510 2023-10-16T11:05:31.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.023+0000 7f61c27cf640 1 --2- 172.21.15.84:0/964299347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc0ca510 0x7f61bc0ca920 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f61bc0ae2b0 tx=0x7f61b402bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec2e0593d241e95 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:31.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.023+0000 7f61c0fcc640 1 -- 172.21.15.84:0/964299347 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b8001670 con 0x7f61bc0ca510 2023-10-16T11:05:31.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.023+0000 7f61c0fcc640 1 -- 172.21.15.84:0/964299347 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b402c750 con 0x7f61bc0ca510 2023-10-16T11:05:31.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.023+0000 7f61c0fcc640 1 -- 172.21.15.84:0/964299347 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b402cae0 con 0x7f61bc0ca510 2023-10-16T11:05:31.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.024+0000 7f61c4232640 1 -- 172.21.15.84:0/964299347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc0ca510 msgr2=0x7f61bc0ca920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:31.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.024+0000 7f61c4232640 1 --2- 172.21.15.84:0/964299347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc0ca510 0x7f61bc0ca920 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f61bc0ae2b0 tx=0x7f61b402bc20 comp rx=0 tx=0).stop 2023-10-16T11:05:31.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.024+0000 7f61c4232640 1 -- 172.21.15.84:0/964299347 shutdown_connections 2023-10-16T11:05:31.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.024+0000 7f61c4232640 1 --2- 172.21.15.84:0/964299347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc0ca510 0x7f61bc0ca920 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:31.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.024+0000 7f61c4232640 1 --2- 172.21.15.84:0/964299347 >> v2:172.21.15.157:3300/0 conn(0x7f61bc0b0360 0x7f61bc0b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:31.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.024+0000 7f61c4232640 1 -- 172.21.15.84:0/964299347 >> 172.21.15.84:0/964299347 conn(0x7f61bc0cb360 msgr2=0x7f61bc0cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:31.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.024+0000 7f61c4232640 1 -- 172.21.15.84:0/964299347 shutdown_connections 2023-10-16T11:05:31.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.024+0000 7f61c4232640 1 -- 172.21.15.84:0/964299347 wait complete. 2023-10-16T11:05:31.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.026+0000 7f61c4232640 1 Processor -- start 2023-10-16T11:05:31.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.026+0000 7f61c4232640 1 -- start start 2023-10-16T11:05:31.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.027+0000 7f61c4232640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61bc0b0360 0x7f61bc0c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:31.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.027+0000 7f61c4232640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61bc0ca510 0x7f61bc1f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:31.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.027+0000 7f61c4232640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc15ac10 0x7f61bc0cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:31.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.027+0000 7f61c4232640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc0c7a20 con 0x7f61bc15ac10 2023-10-16T11:05:31.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.027+0000 7f61c4232640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc0cd1b0 con 0x7f61bc0b0360 2023-10-16T11:05:31.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.027+0000 7f61c4232640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc1f3ab0 con 0x7f61bc0ca510 2023-10-16T11:05:31.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.028+0000 7f61c27cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc15ac10 0x7f61bc0cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:31.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.028+0000 7f61c27cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc15ac10 0x7f61bc0cdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48800/0 (socket says 172.21.15.84:48800) 2023-10-16T11:05:31.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.028+0000 7f61c27cf640 1 -- 172.21.15.84:0/2681698588 learned_addr learned my addr 172.21.15.84:0/2681698588 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:31.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.028+0000 7f61c1fce640 1 --2- 172.21.15.84:0/2681698588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61bc0b0360 0x7f61bc0c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:31.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.028+0000 7f61c27cf640 1 -- 172.21.15.84:0/2681698588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61bc0ca510 msgr2=0x7f61bc1f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:31.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.028+0000 7f61c27cf640 1 --2- 172.21.15.84:0/2681698588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61bc0ca510 0x7f61bc1f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:31.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.028+0000 7f61c27cf640 1 -- 172.21.15.84:0/2681698588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61bc0b0360 msgr2=0x7f61bc0c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:31.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.028+0000 7f61c27cf640 1 --2- 172.21.15.84:0/2681698588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61bc0b0360 0x7f61bc0c6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:31.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.028+0000 7f61c27cf640 1 -- 172.21.15.84:0/2681698588 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b4009a70 con 0x7f61bc15ac10 2023-10-16T11:05:31.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.028+0000 7f61c1fce640 1 --2- 172.21.15.84:0/2681698588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61bc0b0360 0x7f61bc0c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:31.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.029+0000 7f61c27cf640 1 --2- 172.21.15.84:0/2681698588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc15ac10 0x7f61bc0cdd40 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009e80 tx=0x7f61b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:31.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.029+0000 7f61b27fc640 1 -- 172.21.15.84:0/2681698588 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4034c00 con 0x7f61bc15ac10 2023-10-16T11:05:31.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.029+0000 7f61b27fc640 1 -- 172.21.15.84:0/2681698588 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b4034d60 con 0x7f61bc15ac10 2023-10-16T11:05:31.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.029+0000 7f61c4232640 1 -- 172.21.15.84:0/2681698588 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61bc0c6e10 con 0x7f61bc15ac10 2023-10-16T11:05:31.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.029+0000 7f61b27fc640 1 -- 172.21.15.84:0/2681698588 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4040650 con 0x7f61bc15ac10 2023-10-16T11:05:31.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.029+0000 7f61c4232640 1 -- 172.21.15.84:0/2681698588 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61bc0c73d0 con 0x7f61bc15ac10 2023-10-16T11:05:31.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.030+0000 7f61c4232640 1 -- 172.21.15.84:0/2681698588 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61bc0c7bd0 con 0x7f61bc15ac10 2023-10-16T11:05:31.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.031+0000 7f61b27fc640 1 -- 172.21.15.84:0/2681698588 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f61b40407b0 con 0x7f61bc15ac10 2023-10-16T11:05:31.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.032+0000 7f61b27fc640 1 --2- 172.21.15.84:0/2681698588 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f61a006afc0 0x7f61a006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:31.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.032+0000 7f61b27fc640 1 -- 172.21.15.84:0/2681698588 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6569+0+0 (secure 0 0 0) 0x7f61b40ac580 con 0x7f61bc15ac10 2023-10-16T11:05:31.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.032+0000 7f61c1fce640 1 --2- 172.21.15.84:0/2681698588 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f61a006afc0 0x7f61a006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:31.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.033+0000 7f61c1fce640 1 --2- 172.21.15.84:0/2681698588 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f61a006afc0 0x7f61a006d480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f61b8007bc0 tx=0x7f61b8009040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:31.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.037+0000 7f61b27fc640 1 -- 172.21.15.84:0/2681698588 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f61bc0c7bd0 con 0x7f61bc15ac10 2023-10-16T11:05:31.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.228+0000 7f61c4232640 1 -- 172.21.15.84:0/2681698588 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f61bc0c7bd0 con 0x7f61bc15ac10 2023-10-16T11:05:31.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.791+0000 7f61b27fc640 1 -- 172.21.15.84:0/2681698588 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v162) v1 ==== 181+0+0 (secure 0 0 0) 0x7f61bc0c7bd0 con 0x7f61bc15ac10 2023-10-16T11:05:31.793 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-16T11:05:31.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.795+0000 7f61c4232640 1 -- 172.21.15.84:0/2681698588 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f61a006afc0 msgr2=0x7f61a006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:31.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.795+0000 7f61c4232640 1 --2- 172.21.15.84:0/2681698588 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f61a006afc0 0x7f61a006d480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f61b8007bc0 tx=0x7f61b8009040 comp rx=0 tx=0).stop 2023-10-16T11:05:31.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.795+0000 7f61c4232640 1 -- 172.21.15.84:0/2681698588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc15ac10 msgr2=0x7f61bc0cdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:31.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.795+0000 7f61c4232640 1 --2- 172.21.15.84:0/2681698588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc15ac10 0x7f61bc0cdd40 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009e80 tx=0x7f61b4004a40 comp rx=0 tx=0).stop 2023-10-16T11:05:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.797+0000 7f61c4232640 1 -- 172.21.15.84:0/2681698588 shutdown_connections 2023-10-16T11:05:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.797+0000 7f61c4232640 1 --2- 172.21.15.84:0/2681698588 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f61a006afc0 0x7f61a006d480 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:31.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.797+0000 7f61c4232640 1 --2- 172.21.15.84:0/2681698588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc15ac10 0x7f61bc0cdd40 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:31.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.797+0000 7f61c4232640 1 --2- 172.21.15.84:0/2681698588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61bc0ca510 0x7f61bc1f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:31.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.797+0000 7f61c4232640 1 --2- 172.21.15.84:0/2681698588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61bc0b0360 0x7f61bc0c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:31.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.797+0000 7f61c4232640 1 -- 172.21.15.84:0/2681698588 >> 172.21.15.84:0/2681698588 conn(0x7f61bc0cb360 msgr2=0x7f61bc06c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:31.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.798+0000 7f61c4232640 1 -- 172.21.15.84:0/2681698588 shutdown_connections 2023-10-16T11:05:31.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:31.798+0000 7f61c4232640 1 -- 172.21.15.84:0/2681698588 wait complete. 2023-10-16T11:05:31.826 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T11:05:32.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.029+0000 7fc417851640 1 Processor -- start 2023-10-16T11:05:32.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.029+0000 7fc417851640 1 -- start start 2023-10-16T11:05:32.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.031+0000 7fc417851640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc41015ac10 0x7fc41006c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:32.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.031+0000 7fc417851640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4100b0360 0x7fc4100b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:32.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.031+0000 7fc417851640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc4100ca510 0x7fc4100ca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:32.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.031+0000 7fc417851640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4100b0d00 con 0x7fc4100ca510 2023-10-16T11:05:32.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.031+0000 7fc417851640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41015b310 con 0x7fc41015ac10 2023-10-16T11:05:32.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.031+0000 7fc417851640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41006f640 con 0x7fc4100b0360 2023-10-16T11:05:32.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.031+0000 7fc415dee640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc4100ca510 0x7fc4100ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:32.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.031+0000 7fc415dee640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc4100ca510 0x7fc4100ca940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48816/0 (socket says 172.21.15.84:48816) 2023-10-16T11:05:32.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.031+0000 7fc415dee640 1 -- 172.21.15.84:0/2550358866 learned_addr learned my addr 172.21.15.84:0/2550358866 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:32.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.032+0000 7fc414dec640 1 --2- 172.21.15.84:0/2550358866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4100b0360 0x7fc4100b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:32.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.032+0000 7fc4155ed640 1 --2- 172.21.15.84:0/2550358866 >> v2:172.21.15.157:3300/0 conn(0x7fc41015ac10 0x7fc41006c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:32.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.032+0000 7fc415dee640 1 -- 172.21.15.84:0/2550358866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4100b0360 msgr2=0x7fc4100b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:32.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.032+0000 7fc415dee640 1 --2- 172.21.15.84:0/2550358866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4100b0360 0x7fc4100b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:32.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.032+0000 7fc415dee640 1 -- 172.21.15.84:0/2550358866 >> v2:172.21.15.157:3300/0 conn(0x7fc41015ac10 msgr2=0x7fc41006c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:32.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.032+0000 7fc415dee640 1 --2- 172.21.15.84:0/2550358866 >> v2:172.21.15.157:3300/0 conn(0x7fc41015ac10 0x7fc41006c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:32.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.032+0000 7fc415dee640 1 -- 172.21.15.84:0/2550358866 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41015b5b0 con 0x7fc4100ca510 2023-10-16T11:05:32.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.032+0000 7fc4155ed640 1 --2- 172.21.15.84:0/2550358866 >> v2:172.21.15.157:3300/0 conn(0x7fc41015ac10 0x7fc41006c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:32.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.033+0000 7fc415dee640 1 --2- 172.21.15.84:0/2550358866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4100ca510 0x7fc4100ca940 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fc41006d540 tx=0x7fc40802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e8a3c5b7112487 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:32.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.034+0000 7fc403fff640 1 -- 172.21.15.84:0/2550358866 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40802c9e0 con 0x7fc4100ca510 2023-10-16T11:05:32.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.034+0000 7fc403fff640 1 -- 172.21.15.84:0/2550358866 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40802cb40 con 0x7fc4100ca510 2023-10-16T11:05:32.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.034+0000 7fc403fff640 1 -- 172.21.15.84:0/2550358866 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4080343f0 con 0x7fc4100ca510 2023-10-16T11:05:32.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.034+0000 7fc417851640 1 -- 172.21.15.84:0/2550358866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4100ca510 msgr2=0x7fc4100ca940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:32.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.034+0000 7fc417851640 1 --2- 172.21.15.84:0/2550358866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4100ca510 0x7fc4100ca940 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fc41006d540 tx=0x7fc40802be40 comp rx=0 tx=0).stop 2023-10-16T11:05:32.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.035+0000 7fc417851640 1 -- 172.21.15.84:0/2550358866 shutdown_connections 2023-10-16T11:05:32.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.035+0000 7fc417851640 1 --2- 172.21.15.84:0/2550358866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4100ca510 0x7fc4100ca940 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:32.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.035+0000 7fc417851640 1 --2- 172.21.15.84:0/2550358866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4100b0360 0x7fc4100b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:32.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.035+0000 7fc417851640 1 --2- 172.21.15.84:0/2550358866 >> v2:172.21.15.157:3300/0 conn(0x7fc41015ac10 0x7fc41006c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:32.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.035+0000 7fc417851640 1 -- 172.21.15.84:0/2550358866 >> 172.21.15.84:0/2550358866 conn(0x7fc4100cb360 msgr2=0x7fc4100cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:32.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.035+0000 7fc417851640 1 -- 172.21.15.84:0/2550358866 shutdown_connections 2023-10-16T11:05:32.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.035+0000 7fc417851640 1 -- 172.21.15.84:0/2550358866 wait complete. 2023-10-16T11:05:32.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.036+0000 7fc417851640 1 Processor -- start 2023-10-16T11:05:32.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.037+0000 7fc417851640 1 -- start start 2023-10-16T11:05:32.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.037+0000 7fc417851640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4100b0360 0x7fc410154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:32.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.037+0000 7fc417851640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4100ca510 0x7fc4100c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:32.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.037+0000 7fc417851640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc41015ac10 0x7fc4100c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:32.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.037+0000 7fc417851640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41015d7c0 con 0x7fc4100b0360 2023-10-16T11:05:32.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.037+0000 7fc417851640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100c2ee0 con 0x7fc4100ca510 2023-10-16T11:05:32.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.037+0000 7fc417851640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100c5190 con 0x7fc41015ac10 2023-10-16T11:05:32.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.038+0000 7fc4155ed640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4100b0360 0x7fc410154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:32.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.038+0000 7fc4155ed640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4100b0360 0x7fc410154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48828/0 (socket says 172.21.15.84:48828) 2023-10-16T11:05:32.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.038+0000 7fc4155ed640 1 -- 172.21.15.84:0/168395071 learned_addr learned my addr 172.21.15.84:0/168395071 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:32.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.038+0000 7fc414dec640 1 --2- 172.21.15.84:0/168395071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4100ca510 0x7fc4100c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:32.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.038+0000 7fc415dee640 1 --2- 172.21.15.84:0/168395071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc41015ac10 0x7fc4100c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:32.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.038+0000 7fc4155ed640 1 -- 172.21.15.84:0/168395071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc41015ac10 msgr2=0x7fc4100c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:32.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.038+0000 7fc4155ed640 1 --2- 172.21.15.84:0/168395071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc41015ac10 0x7fc4100c4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:32.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.038+0000 7fc4155ed640 1 -- 172.21.15.84:0/168395071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4100ca510 msgr2=0x7fc4100c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:32.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.038+0000 7fc4155ed640 1 --2- 172.21.15.84:0/168395071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4100ca510 0x7fc4100c4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:32.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.038+0000 7fc4155ed640 1 -- 172.21.15.84:0/168395071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc408009b30 con 0x7fc4100b0360 2023-10-16T11:05:32.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.039+0000 7fc4155ed640 1 --2- 172.21.15.84:0/168395071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4100b0360 0x7fc410154040 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00c7f0 tx=0x7fc40c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:32.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.039+0000 7fc401ffb640 1 -- 172.21.15.84:0/168395071 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c007820 con 0x7fc4100b0360 2023-10-16T11:05:32.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.039+0000 7fc401ffb640 1 -- 172.21.15.84:0/168395071 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40c007980 con 0x7fc4100b0360 2023-10-16T11:05:32.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.039+0000 7fc401ffb640 1 -- 172.21.15.84:0/168395071 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c007cf0 con 0x7fc4100b0360 2023-10-16T11:05:32.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.039+0000 7fc417851640 1 -- 172.21.15.84:0/168395071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4100c25b0 con 0x7fc4100b0360 2023-10-16T11:05:32.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.039+0000 7fc417851640 1 -- 172.21.15.84:0/168395071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4100c2b70 con 0x7fc4100b0360 2023-10-16T11:05:32.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.040+0000 7fc417851640 1 -- 172.21.15.84:0/168395071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4100cb120 con 0x7fc4100b0360 2023-10-16T11:05:32.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.041+0000 7fc401ffb640 1 -- 172.21.15.84:0/168395071 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc40c0113f0 con 0x7fc4100b0360 2023-10-16T11:05:32.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.042+0000 7fc401ffb640 1 --2- 172.21.15.84:0/168395071 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc3f406afc0 0x7fc3f406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:32.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.042+0000 7fc401ffb640 1 -- 172.21.15.84:0/168395071 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6131+0+0 (secure 0 0 0) 0x7fc40c089d00 con 0x7fc4100b0360 2023-10-16T11:05:32.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.042+0000 7fc414dec640 1 --2- 172.21.15.84:0/168395071 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc3f406afc0 0x7fc3f406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:32.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.043+0000 7fc414dec640 1 --2- 172.21.15.84:0/168395071 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc3f406afc0 0x7fc3f406d480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc404006fd0 tx=0x7fc404008040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:32.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.046+0000 7fc401ffb640 1 -- 172.21.15.84:0/168395071 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc40c0549d0 con 0x7fc4100b0360 2023-10-16T11:05:32.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.239+0000 7fc417851640 1 -- 172.21.15.84:0/168395071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc41004e160 con 0x7fc4100b0360 2023-10-16T11:05:32.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.801+0000 7fc401ffb640 1 -- 172.21.15.84:0/168395071 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v163) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc40c01f0d0 con 0x7fc4100b0360 2023-10-16T11:05:32.804 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-16T11:05:32.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.805+0000 7fc417851640 1 -- 172.21.15.84:0/168395071 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc3f406afc0 msgr2=0x7fc3f406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:32.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.805+0000 7fc417851640 1 --2- 172.21.15.84:0/168395071 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc3f406afc0 0x7fc3f406d480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc404006fd0 tx=0x7fc404008040 comp rx=0 tx=0).stop 2023-10-16T11:05:32.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.805+0000 7fc417851640 1 -- 172.21.15.84:0/168395071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4100b0360 msgr2=0x7fc410154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:32.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.805+0000 7fc417851640 1 --2- 172.21.15.84:0/168395071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4100b0360 0x7fc410154040 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00c7f0 tx=0x7fc40c00ccb0 comp rx=0 tx=0).stop 2023-10-16T11:05:32.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.806+0000 7fc417851640 1 -- 172.21.15.84:0/168395071 shutdown_connections 2023-10-16T11:05:32.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.806+0000 7fc417851640 1 --2- 172.21.15.84:0/168395071 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc3f406afc0 0x7fc3f406d480 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:32.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.806+0000 7fc417851640 1 --2- 172.21.15.84:0/168395071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc41015ac10 0x7fc4100c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:32.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.806+0000 7fc417851640 1 --2- 172.21.15.84:0/168395071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4100ca510 0x7fc4100c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:32.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.806+0000 7fc417851640 1 --2- 172.21.15.84:0/168395071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4100b0360 0x7fc410154040 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:32.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.806+0000 7fc417851640 1 -- 172.21.15.84:0/168395071 >> 172.21.15.84:0/168395071 conn(0x7fc4100cb360 msgr2=0x7fc4100c9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:32.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.806+0000 7fc417851640 1 -- 172.21.15.84:0/168395071 shutdown_connections 2023-10-16T11:05:32.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:32.806+0000 7fc417851640 1 -- 172.21.15.84:0/168395071 wait complete. 2023-10-16T11:05:32.833 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T11:05:33.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.021+0000 7fa69cf7d640 1 Processor -- start 2023-10-16T11:05:33.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.021+0000 7fa69cf7d640 1 -- start start 2023-10-16T11:05:33.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.023+0000 7fa69cf7d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa69815ad70 0x7fa698069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.023+0000 7fa69cf7d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa6980caac0 0x7fa69806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.023+0000 7fa69cf7d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6980cb030 con 0x7fa6980c9ca0 2023-10-16T11:05:33.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.023+0000 7fa69cf7d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6980c91c0 con 0x7fa6980caac0 2023-10-16T11:05:33.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.023+0000 7fa69cf7d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6980c94a0 con 0x7fa69815ad70 2023-10-16T11:05:33.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.023+0000 7fa696d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa6980caac0 0x7fa69806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:33.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.023+0000 7fa696d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa6980caac0 0x7fa69806a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48830/0 (socket says 172.21.15.84:48830) 2023-10-16T11:05:33.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.023+0000 7fa696d9d640 1 -- 172.21.15.84:0/2931880347 learned_addr learned my addr 172.21.15.84:0/2931880347 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:33.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.023+0000 7fa695d9b640 1 --1- 172.21.15.84:0/2931880347 >> v1:172.21.15.84:6789/0 conn(0x7fa6980c9ca0 0x7fa6980ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37864/0 (socket says 172.21.15.84:37864) 2023-10-16T11:05:33.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.023+0000 7fa69659c640 1 --2- 172.21.15.84:0/2931880347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa69815ad70 0x7fa698069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:33.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.024+0000 7fa696d9d640 1 -- 172.21.15.84:0/2931880347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa69815ad70 msgr2=0x7fa698069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:33.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.024+0000 7fa696d9d640 1 --2- 172.21.15.84:0/2931880347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa69815ad70 0x7fa698069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.024+0000 7fa696d9d640 1 -- 172.21.15.84:0/2931880347 >> v1:172.21.15.84:6789/0 conn(0x7fa6980c9ca0 legacy=0x7fa6980ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:33.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.024+0000 7fa696d9d640 1 -- 172.21.15.84:0/2931880347 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6980c9780 con 0x7fa6980caac0 2023-10-16T11:05:33.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.024+0000 7fa69559a640 1 -- 172.21.15.84:0/2931880347 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1279280830 0 0) 0x7fa6980cb030 con 0x7fa6980c9ca0 2023-10-16T11:05:33.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.025+0000 7fa696d9d640 1 --2- 172.21.15.84:0/2931880347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6980caac0 0x7fa69806a2e0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fa690009e50 tx=0x7fa69002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd90ee2ff5ca3983 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:33.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.025+0000 7fa69559a640 1 -- 172.21.15.84:0/2931880347 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69002cb00 con 0x7fa6980caac0 2023-10-16T11:05:33.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.025+0000 7fa69559a640 1 -- 172.21.15.84:0/2931880347 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69002cc60 con 0x7fa6980caac0 2023-10-16T11:05:33.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.025+0000 7fa69559a640 1 -- 172.21.15.84:0/2931880347 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6900343f0 con 0x7fa6980caac0 2023-10-16T11:05:33.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.025+0000 7fa69cf7d640 1 -- 172.21.15.84:0/2931880347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6980caac0 msgr2=0x7fa69806a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:33.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.025+0000 7fa69cf7d640 1 --2- 172.21.15.84:0/2931880347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6980caac0 0x7fa69806a2e0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fa690009e50 tx=0x7fa69002bf60 comp rx=0 tx=0).stop 2023-10-16T11:05:33.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.026+0000 7fa69cf7d640 1 -- 172.21.15.84:0/2931880347 shutdown_connections 2023-10-16T11:05:33.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.026+0000 7fa69cf7d640 1 --2- 172.21.15.84:0/2931880347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6980caac0 0x7fa69806a2e0 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.026+0000 7fa69cf7d640 1 --2- 172.21.15.84:0/2931880347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa69815ad70 0x7fa698069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.026+0000 7fa69cf7d640 1 -- 172.21.15.84:0/2931880347 >> 172.21.15.84:0/2931880347 conn(0x7fa6980ce9b0 msgr2=0x7fa6980cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:33.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.026+0000 7fa69cf7d640 1 -- 172.21.15.84:0/2931880347 shutdown_connections 2023-10-16T11:05:33.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.026+0000 7fa69cf7d640 1 -- 172.21.15.84:0/2931880347 wait complete. 2023-10-16T11:05:33.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.027+0000 7fa69cf7d640 1 Processor -- start 2023-10-16T11:05:33.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.028+0000 7fa69cf7d640 1 -- start start 2023-10-16T11:05:33.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.028+0000 7fa69cf7d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6980c9ca0 0x7fa6980b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.029+0000 7fa69cf7d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6980caac0 0x7fa698157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.029+0000 7fa69cf7d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa69815ad70 0x7fa6980ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.029+0000 7fa69cf7d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69815e2e0 con 0x7fa6980caac0 2023-10-16T11:05:33.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.029+0000 7fa69cf7d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6980c4150 con 0x7fa6980c9ca0 2023-10-16T11:05:33.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.029+0000 7fa69cf7d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6980b2c90 con 0x7fa69815ad70 2023-10-16T11:05:33.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.029+0000 7fa695d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6980caac0 0x7fa698157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:33.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.029+0000 7fa695d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6980caac0 0x7fa698157e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48832/0 (socket says 172.21.15.84:48832) 2023-10-16T11:05:33.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.029+0000 7fa695d9b640 1 -- 172.21.15.84:0/2501499378 learned_addr learned my addr 172.21.15.84:0/2501499378 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:33.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.029+0000 7fa696d9d640 1 --2- 172.21.15.84:0/2501499378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa69815ad70 0x7fa6980ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:33.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.029+0000 7fa695d9b640 1 -- 172.21.15.84:0/2501499378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa69815ad70 msgr2=0x7fa6980ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:33.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.029+0000 7fa695d9b640 1 --2- 172.21.15.84:0/2501499378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa69815ad70 0x7fa6980ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.029+0000 7fa69659c640 1 --2- 172.21.15.84:0/2501499378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6980c9ca0 0x7fa6980b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:33.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.029+0000 7fa695d9b640 1 -- 172.21.15.84:0/2501499378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6980c9ca0 msgr2=0x7fa6980b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:33.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.029+0000 7fa695d9b640 1 --2- 172.21.15.84:0/2501499378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6980c9ca0 0x7fa6980b2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.029+0000 7fa695d9b640 1 -- 172.21.15.84:0/2501499378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa690009b30 con 0x7fa6980caac0 2023-10-16T11:05:33.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.030+0000 7fa695d9b640 1 --2- 172.21.15.84:0/2501499378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6980caac0 0x7fa698157e50 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00c4d0 tx=0x7fa68c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:33.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.030+0000 7fa686ffd640 1 -- 172.21.15.84:0/2501499378 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c0074e0 con 0x7fa6980caac0 2023-10-16T11:05:33.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.030+0000 7fa686ffd640 1 -- 172.21.15.84:0/2501499378 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c007660 con 0x7fa6980caac0 2023-10-16T11:05:33.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.030+0000 7fa69cf7d640 1 -- 172.21.15.84:0/2501499378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6980c4410 con 0x7fa6980caac0 2023-10-16T11:05:33.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.030+0000 7fa686ffd640 1 -- 172.21.15.84:0/2501499378 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c004a20 con 0x7fa6980caac0 2023-10-16T11:05:33.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.030+0000 7fa69cf7d640 1 -- 172.21.15.84:0/2501499378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa69815b730 con 0x7fa6980caac0 2023-10-16T11:05:33.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.031+0000 7fa69cf7d640 1 -- 172.21.15.84:0/2501499378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6980b2510 con 0x7fa6980caac0 2023-10-16T11:05:33.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.032+0000 7fa686ffd640 1 -- 172.21.15.84:0/2501499378 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa68c004b80 con 0x7fa6980caac0 2023-10-16T11:05:33.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.033+0000 7fa686ffd640 1 --2- 172.21.15.84:0/2501499378 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa67806afc0 0x7fa67806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.033+0000 7fa686ffd640 1 -- 172.21.15.84:0/2501499378 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fa68c0889b0 con 0x7fa6980caac0 2023-10-16T11:05:33.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.033+0000 7fa69659c640 1 --2- 172.21.15.84:0/2501499378 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa67806afc0 0x7fa67806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:33.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.034+0000 7fa69659c640 1 --2- 172.21.15.84:0/2501499378 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa67806afc0 0x7fa67806d480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa68800b7c0 tx=0x7fa688008040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:33.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.037+0000 7fa686ffd640 1 -- 172.21.15.84:0/2501499378 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa68c0538e0 con 0x7fa6980caac0 2023-10-16T11:05:33.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.225+0000 7fa69cf7d640 1 -- 172.21.15.84:0/2501499378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa698066de0 con 0x7fa6980caac0 2023-10-16T11:05:33.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.230+0000 7fa686ffd640 1 -- 172.21.15.84:0/2501499378 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa68c0180a0 con 0x7fa6980caac0 2023-10-16T11:05:33.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.233+0000 7fa69cf7d640 1 -- 172.21.15.84:0/2501499378 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa67806afc0 msgr2=0x7fa67806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:33.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.233+0000 7fa69cf7d640 1 --2- 172.21.15.84:0/2501499378 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa67806afc0 0x7fa67806d480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa68800b7c0 tx=0x7fa688008040 comp rx=0 tx=0).stop 2023-10-16T11:05:33.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.233+0000 7fa69cf7d640 1 -- 172.21.15.84:0/2501499378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6980caac0 msgr2=0x7fa698157e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:33.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.233+0000 7fa69cf7d640 1 --2- 172.21.15.84:0/2501499378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6980caac0 0x7fa698157e50 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00c4d0 tx=0x7fa68c00c990 comp rx=0 tx=0).stop 2023-10-16T11:05:33.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.233+0000 7fa69cf7d640 1 -- 172.21.15.84:0/2501499378 shutdown_connections 2023-10-16T11:05:33.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.233+0000 7fa69cf7d640 1 --2- 172.21.15.84:0/2501499378 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa67806afc0 0x7fa67806d480 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.233+0000 7fa69cf7d640 1 --2- 172.21.15.84:0/2501499378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa69815ad70 0x7fa6980ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.233+0000 7fa69cf7d640 1 --2- 172.21.15.84:0/2501499378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6980caac0 0x7fa698157e50 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.233+0000 7fa69cf7d640 1 --2- 172.21.15.84:0/2501499378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6980c9ca0 0x7fa6980b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.233+0000 7fa69cf7d640 1 -- 172.21.15.84:0/2501499378 >> 172.21.15.84:0/2501499378 conn(0x7fa6980ce9b0 msgr2=0x7fa698159d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:33.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.233+0000 7fa69cf7d640 1 -- 172.21.15.84:0/2501499378 shutdown_connections 2023-10-16T11:05:33.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.233+0000 7fa69cf7d640 1 -- 172.21.15.84:0/2501499378 wait complete. 2023-10-16T11:05:33.251 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T11:05:33.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.463+0000 7fa15f178640 1 Processor -- start 2023-10-16T11:05:33.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.463+0000 7fa15f178640 1 -- start start 2023-10-16T11:05:33.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.465+0000 7fa15f178640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15806c850 0x7fa1580ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.465+0000 7fa15f178640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa158070360 con 0x7fa158159e90 2023-10-16T11:05:33.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.465+0000 7fa15f178640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1580cd1f0 con 0x7fa158159ab0 2023-10-16T11:05:33.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.465+0000 7fa15f178640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15815d890 con 0x7fa15806c850 2023-10-16T11:05:33.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.466+0000 7fa157fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa158159e90 0x7fa15806c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37870/0 (socket says 172.21.15.84:37870) 2023-10-16T11:05:33.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.466+0000 7fa157fff640 1 -- 172.21.15.84:0/1330403044 learned_addr learned my addr 172.21.15.84:0/1330403044 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:33.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.466+0000 7fa15d715640 1 --2- 172.21.15.84:0/1330403044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15806c850 0x7fa1580ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:33.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.466+0000 7fa1577fe640 1 -- 172.21.15.84:0/1330403044 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 689473841 0 0) 0x7fa158070360 con 0x7fa158159e90 2023-10-16T11:05:33.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.466+0000 7fa1577fe640 1 -- 172.21.15.84:0/1330403044 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa140005730 con 0x7fa158159e90 2023-10-16T11:05:33.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.466+0000 7fa1577fe640 1 -- 172.21.15.84:0/1330403044 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1719834221 0 0) 0x7fa140005730 con 0x7fa158159e90 2023-10-16T11:05:33.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.466+0000 7fa1577fe640 1 -- 172.21.15.84:0/1330403044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15806c850 msgr2=0x7fa1580ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:33.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.466+0000 7fa1577fe640 1 --2- 172.21.15.84:0/1330403044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15806c850 0x7fa1580ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.466+0000 7fa1577fe640 1 -- 172.21.15.84:0/1330403044 >> v1:172.21.15.157:6789/0 conn(0x7fa158159ab0 legacy=0x7fa15806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:33.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.466+0000 7fa1577fe640 1 -- 172.21.15.84:0/1330403044 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1580cd4d0 con 0x7fa158159e90 2023-10-16T11:05:33.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.466+0000 7fa1577fe640 1 -- 172.21.15.84:0/1330403044 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fa148001670 con 0x7fa158159e90 2023-10-16T11:05:33.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.467+0000 7fa1577fe640 1 -- 172.21.15.84:0/1330403044 >> v1:172.21.15.84:6789/0 conn(0x7fa158159e90 legacy=0x7fa15806c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:33.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.467+0000 7fa1577fe640 1 --2- 172.21.15.84:0/1330403044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa140005bb0 0x7fa140005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.467+0000 7fa1577fe640 1 --2- 172.21.15.84:0/1330403044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1400069d0 0x7fa140006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.467+0000 7fa1577fe640 1 --2- 172.21.15.84:0/1330403044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15806c850 0x7fa14000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.467+0000 7fa1577fe640 1 -- 172.21.15.84:0/1330403044 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15815d890 con 0x7fa1400069d0 2023-10-16T11:05:33.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.467+0000 7fa1577fe640 1 -- 172.21.15.84:0/1330403044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa140007720 con 0x7fa140005bb0 2023-10-16T11:05:33.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.467+0000 7fa1577fe640 1 -- 172.21.15.84:0/1330403044 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14000e0d0 con 0x7fa15806c850 2023-10-16T11:05:33.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.467+0000 7fa157fff640 1 --2- 172.21.15.84:0/1330403044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1400069d0 0x7fa140006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:33.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.467+0000 7fa15cf14640 1 --2- 172.21.15.84:0/1330403044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa140005bb0 0x7fa140005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:33.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.467+0000 7fa157fff640 1 -- 172.21.15.84:0/1330403044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15806c850 msgr2=0x7fa14000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:33.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.467+0000 7fa157fff640 1 --2- 172.21.15.84:0/1330403044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15806c850 0x7fa14000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.467+0000 7fa157fff640 1 -- 172.21.15.84:0/1330403044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa140005bb0 msgr2=0x7fa140005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:33.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.467+0000 7fa157fff640 1 --2- 172.21.15.84:0/1330403044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa140005bb0 0x7fa140005fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.467+0000 7fa157fff640 1 -- 172.21.15.84:0/1330403044 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa140005730 con 0x7fa1400069d0 2023-10-16T11:05:33.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.468+0000 7fa157fff640 1 --2- 172.21.15.84:0/1330403044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1400069d0 0x7fa140006e50 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fa14800c0e0 tx=0x7fa140007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37417b74fb1b4acc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:33.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.469+0000 7fa1577fe640 1 -- 172.21.15.84:0/1330403044 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148005ad0 con 0x7fa1400069d0 2023-10-16T11:05:33.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.469+0000 7fa1577fe640 1 -- 172.21.15.84:0/1330403044 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa148005c30 con 0x7fa1400069d0 2023-10-16T11:05:33.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.469+0000 7fa1577fe640 1 -- 172.21.15.84:0/1330403044 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14802c810 con 0x7fa1400069d0 2023-10-16T11:05:33.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.469+0000 7fa15f178640 1 -- 172.21.15.84:0/1330403044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1400069d0 msgr2=0x7fa140006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:33.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.469+0000 7fa15f178640 1 --2- 172.21.15.84:0/1330403044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1400069d0 0x7fa140006e50 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fa14800c0e0 tx=0x7fa140007a00 comp rx=0 tx=0).stop 2023-10-16T11:05:33.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.469+0000 7fa15d715640 1 -- 172.21.15.84:0/1330403044 reap_dead start 2023-10-16T11:05:33.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.469+0000 7fa15f178640 1 -- 172.21.15.84:0/1330403044 shutdown_connections 2023-10-16T11:05:33.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.469+0000 7fa15f178640 1 -- 172.21.15.84:0/1330403044 >> 172.21.15.84:0/1330403044 conn(0x7fa1580ce9b0 msgr2=0x7fa1580cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:33.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.469+0000 7fa15f178640 1 -- 172.21.15.84:0/1330403044 shutdown_connections 2023-10-16T11:05:33.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.469+0000 7fa15f178640 1 -- 172.21.15.84:0/1330403044 wait complete. 2023-10-16T11:05:33.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.471+0000 7fa15f178640 1 Processor -- start 2023-10-16T11:05:33.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.471+0000 7fa15f178640 1 -- start start 2023-10-16T11:05:33.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.472+0000 7fa15f178640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1400069d0 0x7fa1581572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.472+0000 7fa15f178640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1580af300 0x7fa1580ad040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.472+0000 7fa15f178640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15815ca10 0x7fa1580700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.472+0000 7fa15f178640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580afb20 con 0x7fa1580af300 2023-10-16T11:05:33.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.472+0000 7fa15f178640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa158157830 con 0x7fa1400069d0 2023-10-16T11:05:33.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.472+0000 7fa15f178640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580ad580 con 0x7fa15815ca10 2023-10-16T11:05:33.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.472+0000 7fa157fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1580af300 0x7fa1580ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:33.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.472+0000 7fa157fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1580af300 0x7fa1580ad040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48856/0 (socket says 172.21.15.84:48856) 2023-10-16T11:05:33.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.472+0000 7fa157fff640 1 -- 172.21.15.84:0/953605149 learned_addr learned my addr 172.21.15.84:0/953605149 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:33.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.473+0000 7fa157fff640 1 -- 172.21.15.84:0/953605149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15815ca10 msgr2=0x7fa1580700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:33.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.473+0000 7fa157fff640 1 --2- 172.21.15.84:0/953605149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15815ca10 0x7fa1580700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.473+0000 7fa157fff640 1 -- 172.21.15.84:0/953605149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1400069d0 msgr2=0x7fa1581572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:33.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.473+0000 7fa157fff640 1 --2- 172.21.15.84:0/953605149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1400069d0 0x7fa1581572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.473+0000 7fa157fff640 1 -- 172.21.15.84:0/953605149 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15815cf80 con 0x7fa1580af300 2023-10-16T11:05:33.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.473+0000 7fa157fff640 1 --2- 172.21.15.84:0/953605149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1580af300 0x7fa1580ad040 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fa14802c480 tx=0x7fa148001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:33.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.473+0000 7fa1557fa640 1 -- 172.21.15.84:0/953605149 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14803c040 con 0x7fa1580af300 2023-10-16T11:05:33.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.473+0000 7fa1557fa640 1 -- 172.21.15.84:0/953605149 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14802cc20 con 0x7fa1580af300 2023-10-16T11:05:33.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.473+0000 7fa15f178640 1 -- 172.21.15.84:0/953605149 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1580c2cd0 con 0x7fa1580af300 2023-10-16T11:05:33.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.473+0000 7fa1557fa640 1 -- 172.21.15.84:0/953605149 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148007070 con 0x7fa1580af300 2023-10-16T11:05:33.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.473+0000 7fa15f178640 1 -- 172.21.15.84:0/953605149 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1580c3290 con 0x7fa1580af300 2023-10-16T11:05:33.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.474+0000 7fa15f178640 1 -- 172.21.15.84:0/953605149 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1580c3450 con 0x7fa1580af300 2023-10-16T11:05:33.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.475+0000 7fa1557fa640 1 -- 172.21.15.84:0/953605149 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa148032070 con 0x7fa1580af300 2023-10-16T11:05:33.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.476+0000 7fa1557fa640 1 --2- 172.21.15.84:0/953605149 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa14406afc0 0x7fa14406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.476+0000 7fa1557fa640 1 -- 172.21.15.84:0/953605149 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fa1480aa740 con 0x7fa1580af300 2023-10-16T11:05:33.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.477+0000 7fa15cf14640 1 --2- 172.21.15.84:0/953605149 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa14406afc0 0x7fa14406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:33.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.477+0000 7fa15cf14640 1 --2- 172.21.15.84:0/953605149 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa14406afc0 0x7fa14406d480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa14c007fd0 tx=0x7fa14c009040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:33.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.481+0000 7fa1557fa640 1 -- 172.21.15.84:0/953605149 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa1480757a0 con 0x7fa1580af300 2023-10-16T11:05:33.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.674+0000 7fa15f178640 1 -- 172.21.15.84:0/953605149 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fa15804e160 con 0x7fa1580af300 2023-10-16T11:05:33.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.679+0000 7fa1557fa640 1 -- 172.21.15.84:0/953605149 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v56) v1 ==== 120+0+0 (secure 0 0 0) 0x7fa1480460b0 con 0x7fa1580af300 2023-10-16T11:05:33.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.682+0000 7fa15f178640 1 -- 172.21.15.84:0/953605149 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa14406afc0 msgr2=0x7fa14406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:33.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.682+0000 7fa15f178640 1 --2- 172.21.15.84:0/953605149 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa14406afc0 0x7fa14406d480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa14c007fd0 tx=0x7fa14c009040 comp rx=0 tx=0).stop 2023-10-16T11:05:33.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.682+0000 7fa15f178640 1 -- 172.21.15.84:0/953605149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1580af300 msgr2=0x7fa1580ad040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:33.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.682+0000 7fa15f178640 1 --2- 172.21.15.84:0/953605149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1580af300 0x7fa1580ad040 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fa14802c480 tx=0x7fa148001930 comp rx=0 tx=0).stop 2023-10-16T11:05:33.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.682+0000 7fa15f178640 1 -- 172.21.15.84:0/953605149 shutdown_connections 2023-10-16T11:05:33.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.682+0000 7fa15f178640 1 --2- 172.21.15.84:0/953605149 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa14406afc0 0x7fa14406d480 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.682+0000 7fa15f178640 1 --2- 172.21.15.84:0/953605149 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15815ca10 0x7fa1580700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.682+0000 7fa15f178640 1 --2- 172.21.15.84:0/953605149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1580af300 0x7fa1580ad040 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.682+0000 7fa15f178640 1 --2- 172.21.15.84:0/953605149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1400069d0 0x7fa1581572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.682+0000 7fa15f178640 1 -- 172.21.15.84:0/953605149 >> 172.21.15.84:0/953605149 conn(0x7fa1580ce9b0 msgr2=0x7fa1580ae410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:33.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.682+0000 7fa15f178640 1 -- 172.21.15.84:0/953605149 shutdown_connections 2023-10-16T11:05:33.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.682+0000 7fa15f178640 1 -- 172.21.15.84:0/953605149 wait complete. 2023-10-16T11:05:33.702 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T11:05:33.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.908+0000 7f92e830f640 1 Processor -- start 2023-10-16T11:05:33.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.908+0000 7f92e830f640 1 -- start start 2023-10-16T11:05:33.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.909+0000 7f92e830f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92e015aaa0 0x7f92e015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.909+0000 7f92e830f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f92e015ae80 0x7f92e00cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.909+0000 7f92e830f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e00707e0 con 0x7f92e00ccaa0 2023-10-16T11:05:33.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.909+0000 7f92e830f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e006b7a0 con 0x7f92e015ae80 2023-10-16T11:05:33.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.909+0000 7f92e830f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e015ef20 con 0x7f92e015aaa0 2023-10-16T11:05:33.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.910+0000 7f92e60ab640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92e015aaa0 0x7f92e015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:33.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.910+0000 7f92e68ac640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f92e00ccaa0 0x7f92e0070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:38332/0 (socket says 172.21.15.84:38332) 2023-10-16T11:05:33.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.910+0000 7f92e58aa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f92e015ae80 0x7f92e00cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:33.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.910+0000 7f92e68ac640 1 -- 172.21.15.84:0/313536591 learned_addr learned my addr 172.21.15.84:0/313536591 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:33.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.911+0000 7f92e60ab640 1 -- 172.21.15.84:0/313536591 >> v2:172.21.15.157:3300/0 conn(0x7f92e015ae80 msgr2=0x7f92e00cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:33.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.911+0000 7f92e60ab640 1 --2- 172.21.15.84:0/313536591 >> v2:172.21.15.157:3300/0 conn(0x7f92e015ae80 0x7f92e00cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.911+0000 7f92e60ab640 1 -- 172.21.15.84:0/313536591 >> v1:172.21.15.157:6789/0 conn(0x7f92e00ccaa0 legacy=0x7f92e0070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:33.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.911+0000 7f92e60ab640 1 -- 172.21.15.84:0/313536591 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e015f200 con 0x7f92e015aaa0 2023-10-16T11:05:33.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.911+0000 7f92e58aa640 1 --2- 172.21.15.84:0/313536591 >> v2:172.21.15.157:3300/0 conn(0x7f92e015ae80 0x7f92e00cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:05:33.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.912+0000 7f92e60ab640 1 --2- 172.21.15.84:0/313536591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92e015aaa0 0x7f92e015e820 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0097a0 tx=0x7f92dc02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5158b28b670258dd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:33.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.913+0000 7f92e50a9640 1 -- 172.21.15.84:0/313536591 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc031780 con 0x7f92e015aaa0 2023-10-16T11:05:33.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.913+0000 7f92e50a9640 1 -- 172.21.15.84:0/313536591 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc0318e0 con 0x7f92e015aaa0 2023-10-16T11:05:33.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.913+0000 7f92e50a9640 1 -- 172.21.15.84:0/313536591 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc031c70 con 0x7f92e015aaa0 2023-10-16T11:05:33.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.914+0000 7f92e830f640 1 -- 172.21.15.84:0/313536591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92e015aaa0 msgr2=0x7f92e015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:33.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.914+0000 7f92e830f640 1 --2- 172.21.15.84:0/313536591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92e015aaa0 0x7f92e015e820 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0097a0 tx=0x7f92dc02b820 comp rx=0 tx=0).stop 2023-10-16T11:05:33.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.914+0000 7f92e830f640 1 -- 172.21.15.84:0/313536591 shutdown_connections 2023-10-16T11:05:33.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.914+0000 7f92e830f640 1 --2- 172.21.15.84:0/313536591 >> v2:172.21.15.157:3300/0 conn(0x7f92e015ae80 0x7f92e00cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.914+0000 7f92e830f640 1 --2- 172.21.15.84:0/313536591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92e015aaa0 0x7f92e015e820 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.914+0000 7f92e830f640 1 -- 172.21.15.84:0/313536591 >> 172.21.15.84:0/313536591 conn(0x7f92e00cb410 msgr2=0x7f92e00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:33.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.914+0000 7f92e830f640 1 -- 172.21.15.84:0/313536591 shutdown_connections 2023-10-16T11:05:33.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.914+0000 7f92e830f640 1 -- 172.21.15.84:0/313536591 wait complete. 2023-10-16T11:05:33.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.916+0000 7f92e830f640 1 Processor -- start 2023-10-16T11:05:33.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.916+0000 7f92e830f640 1 -- start start 2023-10-16T11:05:33.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.917+0000 7f92e830f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92e00ccaa0 0x7f92e014c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.917+0000 7f92e830f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92e015aaa0 0x7f92e01596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.917+0000 7f92e830f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92e015ae80 0x7f92e00c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.917+0000 7f92e830f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e00c6590 con 0x7f92e015ae80 2023-10-16T11:05:33.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.917+0000 7f92e830f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e00c62b0 con 0x7f92e00ccaa0 2023-10-16T11:05:33.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.917+0000 7f92e830f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e014b790 con 0x7f92e015aaa0 2023-10-16T11:05:33.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.917+0000 7f92e68ac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92e015ae80 0x7f92e00c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:33.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.917+0000 7f92e68ac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92e015ae80 0x7f92e00c5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48872/0 (socket says 172.21.15.84:48872) 2023-10-16T11:05:33.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.917+0000 7f92e68ac640 1 -- 172.21.15.84:0/754360994 learned_addr learned my addr 172.21.15.84:0/754360994 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:33.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.917+0000 7f92e60ab640 1 --2- 172.21.15.84:0/754360994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92e00ccaa0 0x7f92e014c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:33.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.917+0000 7f92e58aa640 1 --2- 172.21.15.84:0/754360994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92e015aaa0 0x7f92e01596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:33.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.917+0000 7f92e68ac640 1 -- 172.21.15.84:0/754360994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92e015aaa0 msgr2=0x7f92e01596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:33.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.917+0000 7f92e68ac640 1 --2- 172.21.15.84:0/754360994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92e015aaa0 0x7f92e01596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.917+0000 7f92e68ac640 1 -- 172.21.15.84:0/754360994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92e00ccaa0 msgr2=0x7f92e014c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:33.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.917+0000 7f92e68ac640 1 --2- 172.21.15.84:0/754360994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92e00ccaa0 0x7f92e014c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:33.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.918+0000 7f92e68ac640 1 -- 172.21.15.84:0/754360994 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d0009400 con 0x7f92e015ae80 2023-10-16T11:05:33.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.918+0000 7f92e68ac640 1 --2- 172.21.15.84:0/754360994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92e015ae80 0x7f92e00c5a60 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f92d800cbc0 tx=0x7f92d800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:33.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.918+0000 7f92d6ffd640 1 -- 172.21.15.84:0/754360994 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d800ad40 con 0x7f92e015ae80 2023-10-16T11:05:33.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.918+0000 7f92d6ffd640 1 -- 172.21.15.84:0/754360994 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d8012400 con 0x7f92e015ae80 2023-10-16T11:05:33.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.918+0000 7f92e830f640 1 -- 172.21.15.84:0/754360994 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92dc009400 con 0x7f92e015ae80 2023-10-16T11:05:33.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.918+0000 7f92d6ffd640 1 -- 172.21.15.84:0/754360994 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d800aea0 con 0x7f92e015ae80 2023-10-16T11:05:33.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.918+0000 7f92e830f640 1 -- 172.21.15.84:0/754360994 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e014bd50 con 0x7f92e015ae80 2023-10-16T11:05:33.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.920+0000 7f92e830f640 1 -- 172.21.15.84:0/754360994 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e0071640 con 0x7f92e015ae80 2023-10-16T11:05:33.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.921+0000 7f92d6ffd640 1 -- 172.21.15.84:0/754360994 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f92d8007070 con 0x7f92e015ae80 2023-10-16T11:05:33.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.922+0000 7f92d6ffd640 1 --2- 172.21.15.84:0/754360994 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f92c406af90 0x7f92c406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:33.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.922+0000 7f92d6ffd640 1 -- 172.21.15.84:0/754360994 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f92d80895c0 con 0x7f92e015ae80 2023-10-16T11:05:33.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.922+0000 7f92e60ab640 1 --2- 172.21.15.84:0/754360994 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f92c406af90 0x7f92c406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:33.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.923+0000 7f92e60ab640 1 --2- 172.21.15.84:0/754360994 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f92c406af90 0x7f92c406d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f92dc036000 tx=0x7f92dc03d040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:33.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:33.927+0000 7f92d6ffd640 1 -- 172.21.15.84:0/754360994 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f92e0071640 con 0x7f92e015ae80 2023-10-16T11:05:34.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.117+0000 7f92e830f640 1 -- 172.21.15.84:0/754360994 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f92e0071640 con 0x7f92e015ae80 2023-10-16T11:05:34.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.122+0000 7f92d6ffd640 1 -- 172.21.15.84:0/754360994 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v57) v1 ==== 115+0+0 (secure 0 0 0) 0x7f92e0071640 con 0x7f92e015ae80 2023-10-16T11:05:34.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.125+0000 7f92e830f640 1 -- 172.21.15.84:0/754360994 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f92c406af90 msgr2=0x7f92c406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:34.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.125+0000 7f92e830f640 1 --2- 172.21.15.84:0/754360994 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f92c406af90 0x7f92c406d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f92dc036000 tx=0x7f92dc03d040 comp rx=0 tx=0).stop 2023-10-16T11:05:34.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.125+0000 7f92e830f640 1 -- 172.21.15.84:0/754360994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92e015ae80 msgr2=0x7f92e00c5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:34.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.125+0000 7f92e830f640 1 --2- 172.21.15.84:0/754360994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92e015ae80 0x7f92e00c5a60 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f92d800cbc0 tx=0x7f92d800a430 comp rx=0 tx=0).stop 2023-10-16T11:05:34.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.125+0000 7f92e830f640 1 -- 172.21.15.84:0/754360994 shutdown_connections 2023-10-16T11:05:34.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.125+0000 7f92e830f640 1 --2- 172.21.15.84:0/754360994 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f92c406af90 0x7f92c406d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:34.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.125+0000 7f92e830f640 1 --2- 172.21.15.84:0/754360994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92e015ae80 0x7f92e00c5a60 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:34.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.125+0000 7f92e830f640 1 --2- 172.21.15.84:0/754360994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92e015aaa0 0x7f92e01596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:34.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.125+0000 7f92e830f640 1 --2- 172.21.15.84:0/754360994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92e00ccaa0 0x7f92e014c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:34.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.125+0000 7f92e830f640 1 -- 172.21.15.84:0/754360994 >> 172.21.15.84:0/754360994 conn(0x7f92e00cb410 msgr2=0x7f92e00b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:34.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.125+0000 7f92e830f640 1 -- 172.21.15.84:0/754360994 shutdown_connections 2023-10-16T11:05:34.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.125+0000 7f92e830f640 1 -- 172.21.15.84:0/754360994 wait complete. 2023-10-16T11:05:34.146 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-10-16T11:05:34.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.348+0000 7f65f21a9640 1 Processor -- start 2023-10-16T11:05:34.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.348+0000 7f65f21a9640 1 -- start start 2023-10-16T11:05:34.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.350+0000 7f65f21a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f65ec15ae80 0x7f65ec15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:34.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.350+0000 7f65f21a9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65ec0ce390 0x7f65ec070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:34.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.350+0000 7f65f21a9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65ec15ed60 con 0x7f65ec15aaa0 2023-10-16T11:05:34.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.350+0000 7f65f21a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65ec06b810 con 0x7f65ec15ae80 2023-10-16T11:05:34.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.350+0000 7f65f21a9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65ec0707e0 con 0x7f65ec0ce390 2023-10-16T11:05:34.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.350+0000 7f65e2ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f65ec15ae80 0x7f65ec15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:34.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.350+0000 7f65e2ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f65ec15ae80 0x7f65ec15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48888/0 (socket says 172.21.15.84:48888) 2023-10-16T11:05:34.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.350+0000 7f65e2ffd640 1 -- 172.21.15.84:0/14002753 learned_addr learned my addr 172.21.15.84:0/14002753 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:34.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.350+0000 7f65ebfff640 1 --2- 172.21.15.84:0/14002753 >> v2:172.21.15.157:3300/0 conn(0x7f65ec0ce390 0x7f65ec070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:34.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.350+0000 7f65e2ffd640 1 -- 172.21.15.84:0/14002753 >> v2:172.21.15.157:3300/0 conn(0x7f65ec0ce390 msgr2=0x7f65ec070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:34.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.350+0000 7f65e2ffd640 1 --2- 172.21.15.84:0/14002753 >> v2:172.21.15.157:3300/0 conn(0x7f65ec0ce390 0x7f65ec070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:34.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.350+0000 7f65e2ffd640 1 -- 172.21.15.84:0/14002753 >> v1:172.21.15.157:6789/0 conn(0x7f65ec15aaa0 legacy=0x7f65ec0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:34.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.350+0000 7f65e2ffd640 1 -- 172.21.15.84:0/14002753 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec15ef90 con 0x7f65ec15ae80 2023-10-16T11:05:34.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.351+0000 7f65e2ffd640 1 --2- 172.21.15.84:0/14002753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ec15ae80 0x7f65ec15e820 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0c2490 tx=0x7f65d402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db42058a29ff962 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:34.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.351+0000 7f65eaffd640 1 -- 172.21.15.84:0/14002753 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d402ca80 con 0x7f65ec15ae80 2023-10-16T11:05:34.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.351+0000 7f65eaffd640 1 -- 172.21.15.84:0/14002753 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d402cbe0 con 0x7f65ec15ae80 2023-10-16T11:05:34.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.351+0000 7f65eaffd640 1 -- 172.21.15.84:0/14002753 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d40343f0 con 0x7f65ec15ae80 2023-10-16T11:05:34.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.352+0000 7f65f21a9640 1 -- 172.21.15.84:0/14002753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ec15ae80 msgr2=0x7f65ec15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:34.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.352+0000 7f65f21a9640 1 --2- 172.21.15.84:0/14002753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ec15ae80 0x7f65ec15e820 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0c2490 tx=0x7f65d402bee0 comp rx=0 tx=0).stop 2023-10-16T11:05:34.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.352+0000 7f65f21a9640 1 -- 172.21.15.84:0/14002753 shutdown_connections 2023-10-16T11:05:34.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.352+0000 7f65f21a9640 1 --2- 172.21.15.84:0/14002753 >> v2:172.21.15.157:3300/0 conn(0x7f65ec0ce390 0x7f65ec070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:34.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.352+0000 7f65f21a9640 1 --2- 172.21.15.84:0/14002753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ec15ae80 0x7f65ec15e820 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:34.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.352+0000 7f65f21a9640 1 -- 172.21.15.84:0/14002753 >> 172.21.15.84:0/14002753 conn(0x7f65ec0cb410 msgr2=0x7f65ec0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:34.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.352+0000 7f65f21a9640 1 -- 172.21.15.84:0/14002753 shutdown_connections 2023-10-16T11:05:34.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.352+0000 7f65f21a9640 1 -- 172.21.15.84:0/14002753 wait complete. 2023-10-16T11:05:34.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.354+0000 7f65f21a9640 1 Processor -- start 2023-10-16T11:05:34.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.354+0000 7f65f21a9640 1 -- start start 2023-10-16T11:05:34.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.355+0000 7f65f21a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65ec0ce390 0x7f65ec15ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:34.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.355+0000 7f65f21a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ec15aaa0 0x7f65ec0caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:34.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.355+0000 7f65f21a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ec15ae80 0x7f65ec152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:34.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.355+0000 7f65f21a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec15e300 con 0x7f65ec15ae80 2023-10-16T11:05:34.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.355+0000 7f65f21a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec0caff0 con 0x7f65ec15aaa0 2023-10-16T11:05:34.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.355+0000 7f65f21a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec15cd20 con 0x7f65ec0ce390 2023-10-16T11:05:34.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.355+0000 7f65ebfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ec15ae80 0x7f65ec152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:34.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.355+0000 7f65ebfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ec15ae80 0x7f65ec152f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48902/0 (socket says 172.21.15.84:48902) 2023-10-16T11:05:34.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.355+0000 7f65ebfff640 1 -- 172.21.15.84:0/4014319715 learned_addr learned my addr 172.21.15.84:0/4014319715 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:34.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.356+0000 7f65eb7fe640 1 --2- 172.21.15.84:0/4014319715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65ec0ce390 0x7f65ec15ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:34.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.356+0000 7f65ebfff640 1 -- 172.21.15.84:0/4014319715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65ec0ce390 msgr2=0x7f65ec15ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:34.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.356+0000 7f65ebfff640 1 --2- 172.21.15.84:0/4014319715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65ec0ce390 0x7f65ec15ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:34.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.356+0000 7f65ebfff640 1 -- 172.21.15.84:0/4014319715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ec15aaa0 msgr2=0x7f65ec0caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:34.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.356+0000 7f65ebfff640 1 --2- 172.21.15.84:0/4014319715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ec15aaa0 0x7f65ec0caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:34.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.356+0000 7f65ebfff640 1 -- 172.21.15.84:0/4014319715 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d4009b30 con 0x7f65ec15ae80 2023-10-16T11:05:34.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.356+0000 7f65ebfff640 1 --2- 172.21.15.84:0/4014319715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ec15ae80 0x7f65ec152f70 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f65e400d800 tx=0x7f65e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:34.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.356+0000 7f65e8ff9640 1 -- 172.21.15.84:0/4014319715 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e400b6c0 con 0x7f65ec15ae80 2023-10-16T11:05:34.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.356+0000 7f65e8ff9640 1 -- 172.21.15.84:0/4014319715 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e400b840 con 0x7f65ec15ae80 2023-10-16T11:05:34.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.356+0000 7f65e8ff9640 1 -- 172.21.15.84:0/4014319715 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e400bbf0 con 0x7f65ec15ae80 2023-10-16T11:05:34.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.356+0000 7f65f21a9640 1 -- 172.21.15.84:0/4014319715 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ec15d000 con 0x7f65ec15ae80 2023-10-16T11:05:34.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.356+0000 7f65f21a9640 1 -- 172.21.15.84:0/4014319715 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ec15d5c0 con 0x7f65ec15ae80 2023-10-16T11:05:34.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.358+0000 7f65f21a9640 1 -- 172.21.15.84:0/4014319715 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ec04e160 con 0x7f65ec15ae80 2023-10-16T11:05:34.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.358+0000 7f65e8ff9640 1 -- 172.21.15.84:0/4014319715 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f65e400bd50 con 0x7f65ec15ae80 2023-10-16T11:05:34.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.359+0000 7f65e8ff9640 1 --2- 172.21.15.84:0/4014319715 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f65cc06af90 0x7f65cc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:34.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.359+0000 7f65e8ff9640 1 -- 172.21.15.84:0/4014319715 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f65e4089d90 con 0x7f65ec15ae80 2023-10-16T11:05:34.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.360+0000 7f65eb7fe640 1 --2- 172.21.15.84:0/4014319715 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f65cc06af90 0x7f65cc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:34.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.360+0000 7f65eb7fe640 1 --2- 172.21.15.84:0/4014319715 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f65cc06af90 0x7f65cc06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f65dc007fd0 tx=0x7f65dc009040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:34.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.364+0000 7f65e8ff9640 1 -- 172.21.15.84:0/4014319715 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f65e4054cc0 con 0x7f65ec15ae80 2023-10-16T11:05:34.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.636+0000 7f65f21a9640 1 -- 172.21.15.84:0/4014319715 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f65ec0cdc60 con 0x7f65ec15ae80 2023-10-16T11:05:34.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.807+0000 7f65e8ff9640 1 -- 172.21.15.84:0/4014319715 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v2495) v1 ==== 178+0+0 (secure 0 0 0) 0x7f65e40582e0 con 0x7f65ec15ae80 2023-10-16T11:05:34.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.811+0000 7f65f21a9640 1 -- 172.21.15.84:0/4014319715 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f65cc06af90 msgr2=0x7f65cc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:34.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.811+0000 7f65f21a9640 1 --2- 172.21.15.84:0/4014319715 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f65cc06af90 0x7f65cc06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f65dc007fd0 tx=0x7f65dc009040 comp rx=0 tx=0).stop 2023-10-16T11:05:34.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.811+0000 7f65f21a9640 1 -- 172.21.15.84:0/4014319715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ec15ae80 msgr2=0x7f65ec152f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:34.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.811+0000 7f65f21a9640 1 --2- 172.21.15.84:0/4014319715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ec15ae80 0x7f65ec152f70 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f65e400d800 tx=0x7f65e400dcc0 comp rx=0 tx=0).stop 2023-10-16T11:05:34.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.811+0000 7f65f21a9640 1 -- 172.21.15.84:0/4014319715 shutdown_connections 2023-10-16T11:05:34.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.811+0000 7f65f21a9640 1 --2- 172.21.15.84:0/4014319715 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f65cc06af90 0x7f65cc06d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:34.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.811+0000 7f65f21a9640 1 --2- 172.21.15.84:0/4014319715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ec15ae80 0x7f65ec152f70 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:34.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.811+0000 7f65f21a9640 1 --2- 172.21.15.84:0/4014319715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65ec15aaa0 0x7f65ec0caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:34.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.811+0000 7f65f21a9640 1 --2- 172.21.15.84:0/4014319715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65ec0ce390 0x7f65ec15ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:34.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.811+0000 7f65f21a9640 1 -- 172.21.15.84:0/4014319715 >> 172.21.15.84:0/4014319715 conn(0x7f65ec0cb410 msgr2=0x7f65ec06f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:34.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.811+0000 7f65f21a9640 1 -- 172.21.15.84:0/4014319715 shutdown_connections 2023-10-16T11:05:34.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:34.811+0000 7f65f21a9640 1 -- 172.21.15.84:0/4014319715 wait complete. 2023-10-16T11:05:34.830 INFO:tasks.cephfs_test_runner:Test that number of snaps retained are as per user spec. ... ok 2023-10-16T11:05:34.830 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2023-10-16T11:05:34.831 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T11:05:34.831 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:05:35.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.038+0000 7fdc2f10f640 1 Processor -- start 2023-10-16T11:05:35.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.038+0000 7fdc2f10f640 1 -- start start 2023-10-16T11:05:35.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.039+0000 7fdc2f10f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc281615f0 0x7fdc2815db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.039+0000 7fdc2f10f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2815e3e0 con 0x7fdc280cd0c0 2023-10-16T11:05:35.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.039+0000 7fdc2f10f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc280cd4a0 con 0x7fdc280b0b20 2023-10-16T11:05:35.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.039+0000 7fdc2f10f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2815e100 con 0x7fdc281615f0 2023-10-16T11:05:35.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.040+0000 7fdc2ceab640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdc280cd0c0 0x7fdc280b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37874/0 (socket says 172.21.15.84:37874) 2023-10-16T11:05:35.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.040+0000 7fdc2ceab640 1 -- 172.21.15.84:0/4029305685 learned_addr learned my addr 172.21.15.84:0/4029305685 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:35.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.040+0000 7fdc2d6ac640 1 --2- 172.21.15.84:0/4029305685 >> v2:172.21.15.157:3300/0 conn(0x7fdc281615f0 0x7fdc2815db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:35.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.040+0000 7fdc277fe640 1 -- 172.21.15.84:0/4029305685 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855498480 0 0) 0x7fdc2815e3e0 con 0x7fdc280cd0c0 2023-10-16T11:05:35.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.040+0000 7fdc277fe640 1 -- 172.21.15.84:0/4029305685 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc18003520 con 0x7fdc280cd0c0 2023-10-16T11:05:35.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc277fe640 1 -- 172.21.15.84:0/4029305685 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 722949765 0 0) 0x7fdc18003520 con 0x7fdc280cd0c0 2023-10-16T11:05:35.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc277fe640 1 -- 172.21.15.84:0/4029305685 >> v2:172.21.15.157:3300/0 conn(0x7fdc281615f0 msgr2=0x7fdc2815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:35.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc277fe640 1 --2- 172.21.15.84:0/4029305685 >> v2:172.21.15.157:3300/0 conn(0x7fdc281615f0 0x7fdc2815db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc277fe640 1 -- 172.21.15.84:0/4029305685 >> v1:172.21.15.157:6789/0 conn(0x7fdc280b0b20 legacy=0x7fdc2815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:35.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc277fe640 1 -- 172.21.15.84:0/4029305685 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc28160b10 con 0x7fdc280cd0c0 2023-10-16T11:05:35.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc277fe640 1 -- 172.21.15.84:0/4029305685 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fdc1c001670 con 0x7fdc280cd0c0 2023-10-16T11:05:35.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc277fe640 1 -- 172.21.15.84:0/4029305685 >> v1:172.21.15.84:6789/0 conn(0x7fdc280cd0c0 legacy=0x7fdc280b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:35.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc277fe640 1 --2- 172.21.15.84:0/4029305685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc18003970 0x7fdc18003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc277fe640 1 --2- 172.21.15.84:0/4029305685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc180047c0 0x7fdc18004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc277fe640 1 --2- 172.21.15.84:0/4029305685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc18005180 0x7fdc1800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc277fe640 1 -- 172.21.15.84:0/4029305685 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2815e100 con 0x7fdc18005180 2023-10-16T11:05:35.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc277fe640 1 -- 172.21.15.84:0/4029305685 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1800c0b0 con 0x7fdc18003970 2023-10-16T11:05:35.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc277fe640 1 -- 172.21.15.84:0/4029305685 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1800c270 con 0x7fdc180047c0 2023-10-16T11:05:35.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc2d6ac640 1 --2- 172.21.15.84:0/4029305685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc18005180 0x7fdc1800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:35.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc2d6ac640 1 -- 172.21.15.84:0/4029305685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc180047c0 msgr2=0x7fdc18004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:35.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc2d6ac640 1 --2- 172.21.15.84:0/4029305685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc180047c0 0x7fdc18004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc2d6ac640 1 -- 172.21.15.84:0/4029305685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc18003970 msgr2=0x7fdc18003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:35.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc2d6ac640 1 --2- 172.21.15.84:0/4029305685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc18003970 0x7fdc18003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.041+0000 7fdc2d6ac640 1 -- 172.21.15.84:0/4029305685 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc18003520 con 0x7fdc18005180 2023-10-16T11:05:35.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.042+0000 7fdc2d6ac640 1 -- 172.21.15.84:0/4029305685 reap_dead start 2023-10-16T11:05:35.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.042+0000 7fdc2d6ac640 1 --2- 172.21.15.84:0/4029305685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc18005180 0x7fdc1800b980 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009fc0 tx=0x7fdc180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d34aa0d8466c34 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:35.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.043+0000 7fdc277fe640 1 -- 172.21.15.84:0/4029305685 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2002cc60 con 0x7fdc18005180 2023-10-16T11:05:35.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.043+0000 7fdc277fe640 1 -- 172.21.15.84:0/4029305685 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2000f040 con 0x7fdc18005180 2023-10-16T11:05:35.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.043+0000 7fdc277fe640 1 -- 172.21.15.84:0/4029305685 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc20003990 con 0x7fdc18005180 2023-10-16T11:05:35.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.043+0000 7fdc2f10f640 1 -- 172.21.15.84:0/4029305685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc18005180 msgr2=0x7fdc1800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:35.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.043+0000 7fdc2f10f640 1 --2- 172.21.15.84:0/4029305685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc18005180 0x7fdc1800b980 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009fc0 tx=0x7fdc180057f0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.043+0000 7fdc2f10f640 1 -- 172.21.15.84:0/4029305685 shutdown_connections 2023-10-16T11:05:35.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.043+0000 7fdc2f10f640 1 --2- 172.21.15.84:0/4029305685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc18005180 0x7fdc1800b980 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.043+0000 7fdc2f10f640 1 -- 172.21.15.84:0/4029305685 >> 172.21.15.84:0/4029305685 conn(0x7fdc280cb410 msgr2=0x7fdc280cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:35.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.043+0000 7fdc2f10f640 1 -- 172.21.15.84:0/4029305685 shutdown_connections 2023-10-16T11:05:35.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.043+0000 7fdc2f10f640 1 -- 172.21.15.84:0/4029305685 wait complete. 2023-10-16T11:05:35.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.045+0000 7fdc2f10f640 1 Processor -- start 2023-10-16T11:05:35.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.045+0000 7fdc2f10f640 1 -- start start 2023-10-16T11:05:35.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.046+0000 7fdc2f10f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc18005180 0x7fdc280c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.046+0000 7fdc2f10f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc281604d0 0x7fdc280b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.046+0000 7fdc2f10f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc28151d30 0x7fdc280c2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.046+0000 7fdc2f10f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc281528a0 con 0x7fdc18005180 2023-10-16T11:05:35.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.046+0000 7fdc2f10f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280c34e0 con 0x7fdc28151d30 2023-10-16T11:05:35.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.046+0000 7fdc2f10f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc28152af0 con 0x7fdc281604d0 2023-10-16T11:05:35.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.046+0000 7fdc2ceab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc18005180 0x7fdc280c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:35.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.046+0000 7fdc2ceab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc18005180 0x7fdc280c4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48914/0 (socket says 172.21.15.84:48914) 2023-10-16T11:05:35.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.046+0000 7fdc2ceab640 1 -- 172.21.15.84:0/1063196764 learned_addr learned my addr 172.21.15.84:0/1063196764 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:35.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.047+0000 7fdc2ceab640 1 -- 172.21.15.84:0/1063196764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc281604d0 msgr2=0x7fdc280b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:35.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.047+0000 7fdc2ceab640 1 --2- 172.21.15.84:0/1063196764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc281604d0 0x7fdc280b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.047+0000 7fdc2ceab640 1 -- 172.21.15.84:0/1063196764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc28151d30 msgr2=0x7fdc280c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:35.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.047+0000 7fdc2ceab640 1 --2- 172.21.15.84:0/1063196764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc28151d30 0x7fdc280c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.047+0000 7fdc2ceab640 1 -- 172.21.15.84:0/1063196764 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc28152dd0 con 0x7fdc18005180 2023-10-16T11:05:35.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.047+0000 7fdc2ceab640 1 --2- 172.21.15.84:0/1063196764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc18005180 0x7fdc280c4990 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00c770 tx=0x7fdc1c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:35.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.047+0000 7fdc257fa640 1 -- 172.21.15.84:0/1063196764 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0074e0 con 0x7fdc18005180 2023-10-16T11:05:35.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.047+0000 7fdc257fa640 1 -- 172.21.15.84:0/1063196764 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c007640 con 0x7fdc18005180 2023-10-16T11:05:35.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.047+0000 7fdc257fa640 1 -- 172.21.15.84:0/1063196764 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c004a20 con 0x7fdc18005180 2023-10-16T11:05:35.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.047+0000 7fdc2f10f640 1 -- 172.21.15.84:0/1063196764 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc281530b0 con 0x7fdc18005180 2023-10-16T11:05:35.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.047+0000 7fdc2f10f640 1 -- 172.21.15.84:0/1063196764 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc28153670 con 0x7fdc18005180 2023-10-16T11:05:35.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.048+0000 7fdc2f10f640 1 -- 172.21.15.84:0/1063196764 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2804e160 con 0x7fdc18005180 2023-10-16T11:05:35.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.049+0000 7fdc257fa640 1 -- 172.21.15.84:0/1063196764 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdc1c004b80 con 0x7fdc18005180 2023-10-16T11:05:35.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.050+0000 7fdc257fa640 1 --2- 172.21.15.84:0/1063196764 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fdc0c06af90 0x7fdc0c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.050+0000 7fdc257fa640 1 -- 172.21.15.84:0/1063196764 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fdc1c088ca0 con 0x7fdc18005180 2023-10-16T11:05:35.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.051+0000 7fdc27fff640 1 --2- 172.21.15.84:0/1063196764 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fdc0c06af90 0x7fdc0c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:35.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.051+0000 7fdc27fff640 1 --2- 172.21.15.84:0/1063196764 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fdc0c06af90 0x7fdc0c06d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdc280c4780 tx=0x7fdc140079b0 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:35.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.054+0000 7fdc257fa640 1 -- 172.21.15.84:0/1063196764 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdc1c053c60 con 0x7fdc18005180 2023-10-16T11:05:35.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.247+0000 7fdc2f10f640 1 -- 172.21.15.84:0/1063196764 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc28160dc0 con 0x7fdc18005180 2023-10-16T11:05:35.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.250+0000 7fdc257fa640 1 -- 172.21.15.84:0/1063196764 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+14014 (secure 0 0 0) 0x7fdc1c018020 con 0x7fdc18005180 2023-10-16T11:05:35.250 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:35.251 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":163,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:05:32.793008+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1606872290":"2023-10-17T11:05:26.145693+0000","172.21.15.84:6833/3746729142":"2023-10-17T11:05:26.145693+0000","172.21.15.84:0/3209795865":"2023-10-17T11:05:26.145693+0000","172.21.15.84:0/2439552742":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/384925658":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/3023009161":"2023-10-17T10:09:25.208422+0000","172.21.15.84:6837/2797149326":"2023-10-17T11:05:29.772436+0000","172.21.15.84:0/683905531":"2023-10-17T11:05:26.145693+0000","172.21.15.84:0/3754993953":"2023-10-17T11:05:26.145693+0000","172.21.15.84:0/3629412882":"2023-10-17T11:05:26.145693+0000","172.21.15.84:0/1762476558":"2023-10-17T11:05:26.145693+0000","172.21.15.84:0/870751622":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/300305981":"2023-10-17T10:09:25.208422+0000","172.21.15.84:6832/3746729142":"2023-10-17T11:05:26.145693+0000","172.21.15.84:6833/205480762":"2023-10-17T10:09:25.208422+0000","172.21.15.84:6835/2797149326":"2023-10-17T11:05:29.772436+0000","172.21.15.84:6832/205480762":"2023-10-17T10:09:25.208422+0000","172.21.15.84:0/1358444053":"2023-10-17T10:09:25.208422+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:05:35.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.251+0000 7fdc2f10f640 1 -- 172.21.15.84:0/1063196764 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fdc0c06af90 msgr2=0x7fdc0c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:35.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.251+0000 7fdc2f10f640 1 --2- 172.21.15.84:0/1063196764 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fdc0c06af90 0x7fdc0c06d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdc280c4780 tx=0x7fdc140079b0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.251+0000 7fdc2f10f640 1 -- 172.21.15.84:0/1063196764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc18005180 msgr2=0x7fdc280c4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:35.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.251+0000 7fdc2f10f640 1 --2- 172.21.15.84:0/1063196764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc18005180 0x7fdc280c4990 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00c770 tx=0x7fdc1c00cc30 comp rx=0 tx=0).stop 2023-10-16T11:05:35.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.251+0000 7fdc2f10f640 1 -- 172.21.15.84:0/1063196764 shutdown_connections 2023-10-16T11:05:35.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.251+0000 7fdc2f10f640 1 --2- 172.21.15.84:0/1063196764 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fdc0c06af90 0x7fdc0c06d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.251+0000 7fdc2f10f640 1 --2- 172.21.15.84:0/1063196764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc28151d30 0x7fdc280c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.251+0000 7fdc2f10f640 1 --2- 172.21.15.84:0/1063196764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc281604d0 0x7fdc280b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.251+0000 7fdc2f10f640 1 --2- 172.21.15.84:0/1063196764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc18005180 0x7fdc280c4990 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.251+0000 7fdc2f10f640 1 -- 172.21.15.84:0/1063196764 >> 172.21.15.84:0/1063196764 conn(0x7fdc280cb410 msgr2=0x7fdc280c9030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:35.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.251+0000 7fdc2f10f640 1 -- 172.21.15.84:0/1063196764 shutdown_connections 2023-10-16T11:05:35.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.252+0000 7fdc2f10f640 1 -- 172.21.15.84:0/1063196764 wait complete. 2023-10-16T11:05:35.273 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:05:35.273 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:05:35.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.489+0000 7f7e5b77c640 1 Processor -- start 2023-10-16T11:05:35.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.489+0000 7f7e5b77c640 1 -- start start 2023-10-16T11:05:35.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.490+0000 7f7e5b77c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7e54159ab0 0x7f7e540ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.490+0000 7f7e5b77c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e54159e90 0x7f7e5406fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.490+0000 7f7e5b77c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e540cd1f0 con 0x7f7e5406c140 2023-10-16T11:05:35.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.490+0000 7f7e5b77c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e5415d890 con 0x7f7e54159ab0 2023-10-16T11:05:35.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.490+0000 7f7e5b77c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540cd4d0 con 0x7f7e54159e90 2023-10-16T11:05:35.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.491+0000 7f7e59518640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7e54159ab0 0x7f7e540ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:35.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.491+0000 7f7e59d19640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7e5406c140 0x7f7e5415dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37882/0 (socket says 172.21.15.84:37882) 2023-10-16T11:05:35.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.491+0000 7f7e59d19640 1 -- 172.21.15.84:0/3372301498 learned_addr learned my addr 172.21.15.84:0/3372301498 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:35.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.491+0000 7f7e58d17640 1 --2- 172.21.15.84:0/3372301498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e54159e90 0x7f7e5406fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:35.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.491+0000 7f7e59518640 1 -- 172.21.15.84:0/3372301498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e54159e90 msgr2=0x7f7e5406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:35.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.491+0000 7f7e59518640 1 --2- 172.21.15.84:0/3372301498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e54159e90 0x7f7e5406fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.491+0000 7f7e59518640 1 -- 172.21.15.84:0/3372301498 >> v1:172.21.15.84:6789/0 conn(0x7f7e5406c140 legacy=0x7f7e5415dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:35.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.491+0000 7f7e47fff640 1 -- 172.21.15.84:0/3372301498 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515437036 0 0) 0x7f7e540cd1f0 con 0x7f7e5406c140 2023-10-16T11:05:35.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.491+0000 7f7e59518640 1 -- 172.21.15.84:0/3372301498 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e540cd7b0 con 0x7f7e54159ab0 2023-10-16T11:05:35.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.491+0000 7f7e58d17640 1 --2- 172.21.15.84:0/3372301498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e54159e90 0x7f7e5406fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:35.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.492+0000 7f7e59518640 1 --2- 172.21.15.84:0/3372301498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e54159ab0 0x7f7e540ce370 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000ba70 tx=0x7f7e5002df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc3ebef30619a38 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:35.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.493+0000 7f7e47fff640 1 -- 172.21.15.84:0/3372301498 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5002ed00 con 0x7f7e54159ab0 2023-10-16T11:05:35.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.493+0000 7f7e47fff640 1 -- 172.21.15.84:0/3372301498 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5002ee60 con 0x7f7e54159ab0 2023-10-16T11:05:35.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.493+0000 7f7e47fff640 1 -- 172.21.15.84:0/3372301498 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e50036660 con 0x7f7e54159ab0 2023-10-16T11:05:35.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.493+0000 7f7e5b77c640 1 -- 172.21.15.84:0/3372301498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e54159ab0 msgr2=0x7f7e540ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:35.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.493+0000 7f7e5b77c640 1 --2- 172.21.15.84:0/3372301498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e54159ab0 0x7f7e540ce370 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000ba70 tx=0x7f7e5002df50 comp rx=0 tx=0).stop 2023-10-16T11:05:35.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.493+0000 7f7e5b77c640 1 -- 172.21.15.84:0/3372301498 shutdown_connections 2023-10-16T11:05:35.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.493+0000 7f7e5b77c640 1 --2- 172.21.15.84:0/3372301498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e54159e90 0x7f7e5406fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.493+0000 7f7e5b77c640 1 --2- 172.21.15.84:0/3372301498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e54159ab0 0x7f7e540ce370 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.493+0000 7f7e5b77c640 1 -- 172.21.15.84:0/3372301498 >> 172.21.15.84:0/3372301498 conn(0x7f7e540ce9b0 msgr2=0x7f7e540cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:35.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.493+0000 7f7e5b77c640 1 -- 172.21.15.84:0/3372301498 shutdown_connections 2023-10-16T11:05:35.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.493+0000 7f7e5b77c640 1 -- 172.21.15.84:0/3372301498 wait complete. 2023-10-16T11:05:35.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.495+0000 7f7e5b77c640 1 Processor -- start 2023-10-16T11:05:35.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.495+0000 7f7e5b77c640 1 -- start start 2023-10-16T11:05:35.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.496+0000 7f7e5b77c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e5406c140 0x7f7e54155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.496+0000 7f7e5b77c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e54159ab0 0x7f7e540c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.496+0000 7f7e5b77c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e54159e90 0x7f7e540c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.496+0000 7f7e5b77c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540c26f0 con 0x7f7e54159e90 2023-10-16T11:05:35.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.496+0000 7f7e5b77c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e54155d60 con 0x7f7e5406c140 2023-10-16T11:05:35.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.496+0000 7f7e5b77c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540c5a80 con 0x7f7e54159ab0 2023-10-16T11:05:35.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.496+0000 7f7e59d19640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e54159e90 0x7f7e540c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:35.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.496+0000 7f7e59d19640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e54159e90 0x7f7e540c38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48936/0 (socket says 172.21.15.84:48936) 2023-10-16T11:05:35.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.496+0000 7f7e59d19640 1 -- 172.21.15.84:0/3534427260 learned_addr learned my addr 172.21.15.84:0/3534427260 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:35.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.497+0000 7f7e59518640 1 --2- 172.21.15.84:0/3534427260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e5406c140 0x7f7e54155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:35.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.497+0000 7f7e59d19640 1 -- 172.21.15.84:0/3534427260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e54159ab0 msgr2=0x7f7e540c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:35.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.497+0000 7f7e59d19640 1 --2- 172.21.15.84:0/3534427260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e54159ab0 0x7f7e540c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.497+0000 7f7e59d19640 1 -- 172.21.15.84:0/3534427260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e5406c140 msgr2=0x7f7e54155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:35.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.497+0000 7f7e59d19640 1 --2- 172.21.15.84:0/3534427260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e5406c140 0x7f7e54155820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.497+0000 7f7e59d19640 1 -- 172.21.15.84:0/3534427260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5000b6a0 con 0x7f7e54159e90 2023-10-16T11:05:35.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.497+0000 7f7e59d19640 1 --2- 172.21.15.84:0/3534427260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e54159e90 0x7f7e540c38e0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00c410 tx=0x7f7e4c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:35.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.497+0000 7f7e45ffb640 1 -- 172.21.15.84:0/3534427260 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0074e0 con 0x7f7e54159e90 2023-10-16T11:05:35.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.497+0000 7f7e45ffb640 1 -- 172.21.15.84:0/3534427260 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4c007640 con 0x7f7e54159e90 2023-10-16T11:05:35.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.497+0000 7f7e45ffb640 1 -- 172.21.15.84:0/3534427260 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c004a20 con 0x7f7e54159e90 2023-10-16T11:05:35.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.497+0000 7f7e5b77c640 1 -- 172.21.15.84:0/3534427260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e540c5d60 con 0x7f7e54159e90 2023-10-16T11:05:35.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.498+0000 7f7e5b77c640 1 -- 172.21.15.84:0/3534427260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e540c8d80 con 0x7f7e54159e90 2023-10-16T11:05:35.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.499+0000 7f7e5b77c640 1 -- 172.21.15.84:0/3534427260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e540c6280 con 0x7f7e54159e90 2023-10-16T11:05:35.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.500+0000 7f7e45ffb640 1 -- 172.21.15.84:0/3534427260 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7e4c004b80 con 0x7f7e54159e90 2023-10-16T11:05:35.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.501+0000 7f7e45ffb640 1 --2- 172.21.15.84:0/3534427260 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f7e3806af90 0x7f7e3806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.501+0000 7f7e45ffb640 1 -- 172.21.15.84:0/3534427260 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f7e4c089540 con 0x7f7e54159e90 2023-10-16T11:05:35.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.501+0000 7f7e59518640 1 --2- 172.21.15.84:0/3534427260 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f7e3806af90 0x7f7e3806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:35.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.502+0000 7f7e59518640 1 --2- 172.21.15.84:0/3534427260 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f7e3806af90 0x7f7e3806d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e50038000 tx=0x7f7e50032000 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:35.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.507+0000 7f7e45ffb640 1 -- 172.21.15.84:0/3534427260 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7e4c054500 con 0x7f7e54159e90 2023-10-16T11:05:35.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.698+0000 7f7e5b77c640 1 -- 172.21.15.84:0/3534427260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e5404e160 con 0x7f7e54159e90 2023-10-16T11:05:35.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.699+0000 7f7e45ffb640 1 -- 172.21.15.84:0/3534427260 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v163) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7e4c001060 con 0x7f7e54159e90 2023-10-16T11:05:35.700 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:05:35.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.701+0000 7f7e5b77c640 1 -- 172.21.15.84:0/3534427260 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f7e3806af90 msgr2=0x7f7e3806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:35.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.701+0000 7f7e5b77c640 1 --2- 172.21.15.84:0/3534427260 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f7e3806af90 0x7f7e3806d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e50038000 tx=0x7f7e50032000 comp rx=0 tx=0).stop 2023-10-16T11:05:35.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.701+0000 7f7e5b77c640 1 -- 172.21.15.84:0/3534427260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e54159e90 msgr2=0x7f7e540c38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:35.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.701+0000 7f7e5b77c640 1 --2- 172.21.15.84:0/3534427260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e54159e90 0x7f7e540c38e0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00c410 tx=0x7f7e4c00c8d0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.701+0000 7f7e5b77c640 1 -- 172.21.15.84:0/3534427260 shutdown_connections 2023-10-16T11:05:35.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.701+0000 7f7e5b77c640 1 --2- 172.21.15.84:0/3534427260 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f7e3806af90 0x7f7e3806d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.701+0000 7f7e5b77c640 1 --2- 172.21.15.84:0/3534427260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e54159e90 0x7f7e540c38e0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.701+0000 7f7e5b77c640 1 --2- 172.21.15.84:0/3534427260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e54159ab0 0x7f7e540c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.701+0000 7f7e5b77c640 1 --2- 172.21.15.84:0/3534427260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e5406c140 0x7f7e54155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.701+0000 7f7e5b77c640 1 -- 172.21.15.84:0/3534427260 >> 172.21.15.84:0/3534427260 conn(0x7f7e540ce9b0 msgr2=0x7f7e540ae310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:35.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.701+0000 7f7e5b77c640 1 -- 172.21.15.84:0/3534427260 shutdown_connections 2023-10-16T11:05:35.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.702+0000 7f7e5b77c640 1 -- 172.21.15.84:0/3534427260 wait complete. 2023-10-16T11:05:35.719 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-10-16T11:05:35.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.937+0000 7fc735ed7640 1 Processor -- start 2023-10-16T11:05:35.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.937+0000 7fc735ed7640 1 -- start start 2023-10-16T11:05:35.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.939+0000 7fc735ed7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc73015aaa0 0x7fc73015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.939+0000 7fc735ed7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73015ae80 0x7fc7300cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.939+0000 7fc735ed7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7300ccaa0 0x7fc730070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.939+0000 7fc735ed7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73006b810 con 0x7fc73015aaa0 2023-10-16T11:05:35.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.939+0000 7fc735ed7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73015ed60 con 0x7fc7300ccaa0 2023-10-16T11:05:35.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.939+0000 7fc735ed7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300707e0 con 0x7fc73015ae80 2023-10-16T11:05:35.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.939+0000 7fc72f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc73015aaa0 0x7fc73015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:35.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.939+0000 7fc72f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc73015aaa0 0x7fc73015e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58946/0 (socket says 172.21.15.84:58946) 2023-10-16T11:05:35.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.939+0000 7fc72f7fe640 1 -- 172.21.15.84:0/307387758 learned_addr learned my addr 172.21.15.84:0/307387758 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:35.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.939+0000 7fc72f7fe640 1 -- 172.21.15.84:0/307387758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73015ae80 msgr2=0x7fc7300cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:35.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.939+0000 7fc72effd640 1 --2- 172.21.15.84:0/307387758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73015ae80 0x7fc7300cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:35.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.939+0000 7fc72ffff640 1 --2- 172.21.15.84:0/307387758 >> v2:172.21.15.157:3300/0 conn(0x7fc7300ccaa0 0x7fc730070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:35.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.939+0000 7fc72f7fe640 1 --2- 172.21.15.84:0/307387758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73015ae80 0x7fc7300cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.939+0000 7fc72f7fe640 1 -- 172.21.15.84:0/307387758 >> v2:172.21.15.157:3300/0 conn(0x7fc7300ccaa0 msgr2=0x7fc730070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:35.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.939+0000 7fc72f7fe640 1 --2- 172.21.15.84:0/307387758 >> v2:172.21.15.157:3300/0 conn(0x7fc7300ccaa0 0x7fc730070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.940+0000 7fc72f7fe640 1 -- 172.21.15.84:0/307387758 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73015ef90 con 0x7fc73015aaa0 2023-10-16T11:05:35.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.940+0000 7fc72effd640 1 --2- 172.21.15.84:0/307387758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73015ae80 0x7fc7300cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:35.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.941+0000 7fc72f7fe640 1 --2- 172.21.15.84:0/307387758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73015aaa0 0x7fc73015e820 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fc7300ae3c0 tx=0x7fc72002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606fd11e9856b97b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.941+0000 7fc72e7fc640 1 -- 172.21.15.84:0/307387758 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720031880 con 0x7fc73015aaa0 2023-10-16T11:05:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.941+0000 7fc72e7fc640 1 -- 172.21.15.84:0/307387758 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7200319e0 con 0x7fc73015aaa0 2023-10-16T11:05:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.941+0000 7fc72e7fc640 1 -- 172.21.15.84:0/307387758 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720031d70 con 0x7fc73015aaa0 2023-10-16T11:05:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.941+0000 7fc735ed7640 1 -- 172.21.15.84:0/307387758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73015aaa0 msgr2=0x7fc73015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.941+0000 7fc735ed7640 1 --2- 172.21.15.84:0/307387758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73015aaa0 0x7fc73015e820 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fc7300ae3c0 tx=0x7fc72002b920 comp rx=0 tx=0).stop 2023-10-16T11:05:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.942+0000 7fc735ed7640 1 -- 172.21.15.84:0/307387758 shutdown_connections 2023-10-16T11:05:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.942+0000 7fc735ed7640 1 --2- 172.21.15.84:0/307387758 >> v2:172.21.15.157:3300/0 conn(0x7fc7300ccaa0 0x7fc730070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.942+0000 7fc735ed7640 1 --2- 172.21.15.84:0/307387758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73015ae80 0x7fc7300cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.942+0000 7fc735ed7640 1 --2- 172.21.15.84:0/307387758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73015aaa0 0x7fc73015e820 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.942+0000 7fc735ed7640 1 -- 172.21.15.84:0/307387758 >> 172.21.15.84:0/307387758 conn(0x7fc7300cb410 msgr2=0x7fc7300cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:35.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.942+0000 7fc735ed7640 1 -- 172.21.15.84:0/307387758 shutdown_connections 2023-10-16T11:05:35.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.942+0000 7fc735ed7640 1 -- 172.21.15.84:0/307387758 wait complete. 2023-10-16T11:05:35.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.943+0000 7fc735ed7640 1 Processor -- start 2023-10-16T11:05:35.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.943+0000 7fc735ed7640 1 -- start start 2023-10-16T11:05:35.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.944+0000 7fc735ed7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7300ccaa0 0x7fc730158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.944+0000 7fc735ed7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73015aaa0 0x7fc730159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.944+0000 7fc735ed7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73015ae80 0x7fc7300c3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.944+0000 7fc735ed7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73015c0f0 con 0x7fc73015ae80 2023-10-16T11:05:35.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.944+0000 7fc735ed7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc730159890 con 0x7fc73015aaa0 2023-10-16T11:05:35.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.944+0000 7fc735ed7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc730156e10 con 0x7fc7300ccaa0 2023-10-16T11:05:35.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.945+0000 7fc72ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73015ae80 0x7fc7300c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:35.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.945+0000 7fc72ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73015ae80 0x7fc7300c3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58948/0 (socket says 172.21.15.84:58948) 2023-10-16T11:05:35.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.945+0000 7fc72ffff640 1 -- 172.21.15.84:0/2982354363 learned_addr learned my addr 172.21.15.84:0/2982354363 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:35.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.945+0000 7fc72f7fe640 1 --2- 172.21.15.84:0/2982354363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7300ccaa0 0x7fc730158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:35.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.945+0000 7fc72effd640 1 --2- 172.21.15.84:0/2982354363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73015aaa0 0x7fc730159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:35.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.945+0000 7fc72ffff640 1 -- 172.21.15.84:0/2982354363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7300ccaa0 msgr2=0x7fc730158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:35.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.945+0000 7fc72ffff640 1 --2- 172.21.15.84:0/2982354363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7300ccaa0 0x7fc730158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.945+0000 7fc72ffff640 1 -- 172.21.15.84:0/2982354363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73015aaa0 msgr2=0x7fc730159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:35.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.945+0000 7fc72ffff640 1 --2- 172.21.15.84:0/2982354363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73015aaa0 0x7fc730159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:35.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.945+0000 7fc72ffff640 1 -- 172.21.15.84:0/2982354363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7200094d0 con 0x7fc73015ae80 2023-10-16T11:05:35.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.945+0000 7fc72effd640 1 --2- 172.21.15.84:0/2982354363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73015aaa0 0x7fc730159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:35.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.946+0000 7fc72ffff640 1 --2- 172.21.15.84:0/2982354363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73015ae80 0x7fc7300c3510 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fc72800cb80 tx=0x7fc72800a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:35.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.946+0000 7fc717fff640 1 -- 172.21.15.84:0/2982354363 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72800ce70 con 0x7fc73015ae80 2023-10-16T11:05:35.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.946+0000 7fc717fff640 1 -- 172.21.15.84:0/2982354363 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72800abd0 con 0x7fc73015ae80 2023-10-16T11:05:35.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.946+0000 7fc717fff640 1 -- 172.21.15.84:0/2982354363 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728011410 con 0x7fc73015ae80 2023-10-16T11:05:35.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.946+0000 7fc735ed7640 1 -- 172.21.15.84:0/2982354363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc730157290 con 0x7fc73015ae80 2023-10-16T11:05:35.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.946+0000 7fc735ed7640 1 -- 172.21.15.84:0/2982354363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc730157850 con 0x7fc73015ae80 2023-10-16T11:05:35.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.947+0000 7fc735ed7640 1 -- 172.21.15.84:0/2982354363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc73015c2a0 con 0x7fc73015ae80 2023-10-16T11:05:35.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.948+0000 7fc717fff640 1 -- 172.21.15.84:0/2982354363 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc728011570 con 0x7fc73015ae80 2023-10-16T11:05:35.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.949+0000 7fc717fff640 1 --2- 172.21.15.84:0/2982354363 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc71c06af90 0x7fc71c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:35.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.949+0000 7fc717fff640 1 -- 172.21.15.84:0/2982354363 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fc728089640 con 0x7fc73015ae80 2023-10-16T11:05:35.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.949+0000 7fc72f7fe640 1 --2- 172.21.15.84:0/2982354363 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc71c06af90 0x7fc71c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:35.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.950+0000 7fc72f7fe640 1 --2- 172.21.15.84:0/2982354363 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc71c06af90 0x7fc71c06d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc720036000 tx=0x7fc72003c040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:35.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:35.953+0000 7fc717fff640 1 -- 172.21.15.84:0/2982354363 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc73015c2a0 con 0x7fc73015ae80 2023-10-16T11:05:36.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:36.215+0000 7fc735ed7640 1 -- 172.21.15.84:0/2982354363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7fc73004e160 con 0x7fc73015ae80 2023-10-16T11:05:36.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:36.821+0000 7fc717fff640 1 -- 172.21.15.84:0/2982354363 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2497) v1 ==== 177+0+0 (secure 0 0 0) 0x7fc728054590 con 0x7fc73015ae80 2023-10-16T11:05:36.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:36.824+0000 7fc735ed7640 1 -- 172.21.15.84:0/2982354363 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc71c06af90 msgr2=0x7fc71c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:36.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:36.824+0000 7fc735ed7640 1 --2- 172.21.15.84:0/2982354363 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc71c06af90 0x7fc71c06d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc720036000 tx=0x7fc72003c040 comp rx=0 tx=0).stop 2023-10-16T11:05:36.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:36.824+0000 7fc735ed7640 1 -- 172.21.15.84:0/2982354363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73015ae80 msgr2=0x7fc7300c3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:36.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:36.824+0000 7fc735ed7640 1 --2- 172.21.15.84:0/2982354363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73015ae80 0x7fc7300c3510 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fc72800cb80 tx=0x7fc72800a4c0 comp rx=0 tx=0).stop 2023-10-16T11:05:36.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:36.824+0000 7fc735ed7640 1 -- 172.21.15.84:0/2982354363 shutdown_connections 2023-10-16T11:05:36.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:36.824+0000 7fc735ed7640 1 --2- 172.21.15.84:0/2982354363 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fc71c06af90 0x7fc71c06d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:36.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:36.824+0000 7fc735ed7640 1 --2- 172.21.15.84:0/2982354363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73015ae80 0x7fc7300c3510 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:36.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:36.824+0000 7fc735ed7640 1 --2- 172.21.15.84:0/2982354363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73015aaa0 0x7fc730159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:36.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:36.824+0000 7fc735ed7640 1 --2- 172.21.15.84:0/2982354363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7300ccaa0 0x7fc730158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:36.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:36.824+0000 7fc735ed7640 1 -- 172.21.15.84:0/2982354363 >> 172.21.15.84:0/2982354363 conn(0x7fc7300cb410 msgr2=0x7fc7300cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:36.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:36.824+0000 7fc735ed7640 1 -- 172.21.15.84:0/2982354363 shutdown_connections 2023-10-16T11:05:36.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:36.824+0000 7fc735ed7640 1 -- 172.21.15.84:0/2982354363 wait complete. 2023-10-16T11:05:36.847 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T11:05:37.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.056+0000 7f21416b4640 1 Processor -- start 2023-10-16T11:05:37.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.056+0000 7f21416b4640 1 -- start start 2023-10-16T11:05:37.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.058+0000 7f21416b4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f213c159ab0 0x7f213c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:37.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.058+0000 7f21416b4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f213c06c140 0x7f213c06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:37.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.058+0000 7f21416b4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f213c070360 con 0x7f213c159e90 2023-10-16T11:05:37.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.058+0000 7f21416b4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f213c06cb10 con 0x7f213c159ab0 2023-10-16T11:05:37.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.058+0000 7f21416b4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213c15d890 con 0x7f213c06c140 2023-10-16T11:05:37.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.058+0000 7f2133fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f213c159e90 0x7f213c06fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42428/0 (socket says 172.21.15.84:42428) 2023-10-16T11:05:37.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.058+0000 7f2133fff640 1 -- 172.21.15.84:0/505176944 learned_addr learned my addr 172.21.15.84:0/505176944 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:37.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213affd640 1 --2- 172.21.15.84:0/505176944 >> v2:172.21.15.157:3300/0 conn(0x7f213c159ab0 0x7f213c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:37.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213b7fe640 1 --2- 172.21.15.84:0/505176944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f213c06c140 0x7f213c06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:37.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213a7fc640 1 -- 172.21.15.84:0/505176944 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940153791 0 0) 0x7f213c070360 con 0x7f213c159e90 2023-10-16T11:05:37.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213a7fc640 1 -- 172.21.15.84:0/505176944 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2128003520 con 0x7f213c159e90 2023-10-16T11:05:37.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213a7fc640 1 -- 172.21.15.84:0/505176944 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1235428264 0 0) 0x7f2128003520 con 0x7f213c159e90 2023-10-16T11:05:37.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213a7fc640 1 -- 172.21.15.84:0/505176944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f213c06c140 msgr2=0x7f213c06c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:37.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213a7fc640 1 --2- 172.21.15.84:0/505176944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f213c06c140 0x7f213c06c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:37.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213a7fc640 1 -- 172.21.15.84:0/505176944 >> v2:172.21.15.157:3300/0 conn(0x7f213c159ab0 msgr2=0x7f213c0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:37.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213a7fc640 1 --2- 172.21.15.84:0/505176944 >> v2:172.21.15.157:3300/0 conn(0x7f213c159ab0 0x7f213c0ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:37.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213a7fc640 1 -- 172.21.15.84:0/505176944 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c0cd1f0 con 0x7f213c159e90 2023-10-16T11:05:37.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213a7fc640 1 -- 172.21.15.84:0/505176944 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f2124001670 con 0x7f213c159e90 2023-10-16T11:05:37.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213affd640 1 --2- 172.21.15.84:0/505176944 >> v2:172.21.15.157:3300/0 conn(0x7f213c159ab0 0x7f213c0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:05:37.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213a7fc640 1 -- 172.21.15.84:0/505176944 >> v1:172.21.15.84:6789/0 conn(0x7f213c159e90 legacy=0x7f213c06fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:37.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213a7fc640 1 --2- 172.21.15.84:0/505176944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2128003970 0x7f2128003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:37.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213a7fc640 1 --2- 172.21.15.84:0/505176944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21280047c0 0x7f2128004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:37.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213a7fc640 1 --2- 172.21.15.84:0/505176944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f213c06c140 0x7f2128005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:37.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213a7fc640 1 -- 172.21.15.84:0/505176944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c06cb10 con 0x7f2128003970 2023-10-16T11:05:37.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213a7fc640 1 -- 172.21.15.84:0/505176944 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c15d890 con 0x7f21280047c0 2023-10-16T11:05:37.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.059+0000 7f213a7fc640 1 -- 172.21.15.84:0/505176944 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212800bb60 con 0x7f213c06c140 2023-10-16T11:05:37.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.060+0000 7f213affd640 1 --2- 172.21.15.84:0/505176944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2128003970 0x7f2128003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:37.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.060+0000 7f2133fff640 1 --2- 172.21.15.84:0/505176944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21280047c0 0x7f2128004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:37.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.060+0000 7f213affd640 1 -- 172.21.15.84:0/505176944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f213c06c140 msgr2=0x7f2128005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:37.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.060+0000 7f213affd640 1 --2- 172.21.15.84:0/505176944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f213c06c140 0x7f2128005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:37.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.060+0000 7f213affd640 1 -- 172.21.15.84:0/505176944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21280047c0 msgr2=0x7f2128004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:37.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.060+0000 7f213affd640 1 --2- 172.21.15.84:0/505176944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21280047c0 0x7f2128004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:37.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.060+0000 7f213affd640 1 -- 172.21.15.84:0/505176944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2128003520 con 0x7f2128003970 2023-10-16T11:05:37.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.060+0000 7f2133fff640 1 --2- 172.21.15.84:0/505176944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21280047c0 0x7f2128004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:37.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.061+0000 7f213affd640 1 --2- 172.21.15.84:0/505176944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2128003970 0x7f2128003d90 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f212c0098a0 tx=0x7f21280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594000fd8d0363ec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:37.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.061+0000 7f213a7fc640 1 -- 172.21.15.84:0/505176944 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c004230 con 0x7f2128003970 2023-10-16T11:05:37.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.061+0000 7f213a7fc640 1 -- 172.21.15.84:0/505176944 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212c004390 con 0x7f2128003970 2023-10-16T11:05:37.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.061+0000 7f213a7fc640 1 -- 172.21.15.84:0/505176944 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c003830 con 0x7f2128003970 2023-10-16T11:05:37.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.062+0000 7f21416b4640 1 -- 172.21.15.84:0/505176944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2128003970 msgr2=0x7f2128003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:37.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.062+0000 7f21416b4640 1 --2- 172.21.15.84:0/505176944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2128003970 0x7f2128003d90 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f212c0098a0 tx=0x7f21280057f0 comp rx=0 tx=0).stop 2023-10-16T11:05:37.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.062+0000 7f213b7fe640 1 -- 172.21.15.84:0/505176944 reap_dead start 2023-10-16T11:05:37.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.062+0000 7f21416b4640 1 -- 172.21.15.84:0/505176944 shutdown_connections 2023-10-16T11:05:37.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.062+0000 7f21416b4640 1 -- 172.21.15.84:0/505176944 >> 172.21.15.84:0/505176944 conn(0x7f213c0ce9b0 msgr2=0x7f213c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:37.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.062+0000 7f21416b4640 1 -- 172.21.15.84:0/505176944 shutdown_connections 2023-10-16T11:05:37.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.062+0000 7f21416b4640 1 -- 172.21.15.84:0/505176944 wait complete. 2023-10-16T11:05:37.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.064+0000 7f21416b4640 1 Processor -- start 2023-10-16T11:05:37.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.064+0000 7f21416b4640 1 -- start start 2023-10-16T11:05:37.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.065+0000 7f21416b4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2128003970 0x7f213c0b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:37.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.065+0000 7f21416b4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f213c154780 0x7f213c0af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:37.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.065+0000 7f21416b4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f213c0afb00 0x7f213c1e4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:37.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.065+0000 7f21416b4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c0b1f90 con 0x7f213c154780 2023-10-16T11:05:37.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.065+0000 7f21416b4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c0aff80 con 0x7f2128003970 2023-10-16T11:05:37.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.065+0000 7f21416b4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213c0b3760 con 0x7f213c0afb00 2023-10-16T11:05:37.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.065+0000 7f2133fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f213c154780 0x7f213c0af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:37.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.065+0000 7f2133fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f213c154780 0x7f213c0af5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58972/0 (socket says 172.21.15.84:58972) 2023-10-16T11:05:37.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.065+0000 7f2133fff640 1 -- 172.21.15.84:0/3680256701 learned_addr learned my addr 172.21.15.84:0/3680256701 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:37.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.066+0000 7f213affd640 1 --2- 172.21.15.84:0/3680256701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2128003970 0x7f213c0b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:37.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.066+0000 7f2133fff640 1 -- 172.21.15.84:0/3680256701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f213c0afb00 msgr2=0x7f213c1e4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:37.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.066+0000 7f2133fff640 1 --2- 172.21.15.84:0/3680256701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f213c0afb00 0x7f213c1e4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:37.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.066+0000 7f2133fff640 1 -- 172.21.15.84:0/3680256701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2128003970 msgr2=0x7f213c0b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:37.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.066+0000 7f2133fff640 1 --2- 172.21.15.84:0/3680256701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2128003970 0x7f213c0b3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:37.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.066+0000 7f2133fff640 1 -- 172.21.15.84:0/3680256701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c0b2d90 con 0x7f213c154780 2023-10-16T11:05:37.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.066+0000 7f2133fff640 1 --2- 172.21.15.84:0/3680256701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f213c154780 0x7f213c0af5c0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f2124000ff0 tx=0x7f212400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:37.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.066+0000 7f21337fe640 1 -- 172.21.15.84:0/3680256701 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21240074e0 con 0x7f213c154780 2023-10-16T11:05:37.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.066+0000 7f21337fe640 1 -- 172.21.15.84:0/3680256701 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2124007640 con 0x7f213c154780 2023-10-16T11:05:37.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.066+0000 7f21416b4640 1 -- 172.21.15.84:0/3680256701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213c0b2180 con 0x7f213c154780 2023-10-16T11:05:37.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.066+0000 7f21337fe640 1 -- 172.21.15.84:0/3680256701 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124004a20 con 0x7f213c154780 2023-10-16T11:05:37.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.067+0000 7f21416b4640 1 -- 172.21.15.84:0/3680256701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213c0b2740 con 0x7f213c154780 2023-10-16T11:05:37.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.068+0000 7f21416b4640 1 -- 172.21.15.84:0/3680256701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f213c0b2f20 con 0x7f213c154780 2023-10-16T11:05:37.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.069+0000 7f21337fe640 1 -- 172.21.15.84:0/3680256701 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2124004b80 con 0x7f213c154780 2023-10-16T11:05:37.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.069+0000 7f21337fe640 1 --2- 172.21.15.84:0/3680256701 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f211c06af90 0x7f211c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:37.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.070+0000 7f21337fe640 1 -- 172.21.15.84:0/3680256701 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f2124089530 con 0x7f213c154780 2023-10-16T11:05:37.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.070+0000 7f213affd640 1 --2- 172.21.15.84:0/3680256701 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f211c06af90 0x7f211c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:37.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.071+0000 7f213affd640 1 --2- 172.21.15.84:0/3680256701 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f211c06af90 0x7f211c06d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f212c035000 tx=0x7f212c039040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:37.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.074+0000 7f21337fe640 1 -- 172.21.15.84:0/3680256701 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f21240543f0 con 0x7f213c154780 2023-10-16T11:05:37.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.269+0000 7f21416b4640 1 -- 172.21.15.84:0/3680256701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f213c04e160 con 0x7f213c154780 2023-10-16T11:05:37.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.274+0000 7f21337fe640 1 -- 172.21.15.84:0/3680256701 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v58)=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2124057a10 con 0x7f213c154780 2023-10-16T11:05:37.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.277+0000 7f21416b4640 1 -- 172.21.15.84:0/3680256701 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f211c06af90 msgr2=0x7f211c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:37.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.277+0000 7f21416b4640 1 --2- 172.21.15.84:0/3680256701 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f211c06af90 0x7f211c06d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f212c035000 tx=0x7f212c039040 comp rx=0 tx=0).stop 2023-10-16T11:05:37.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.277+0000 7f21416b4640 1 -- 172.21.15.84:0/3680256701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f213c154780 msgr2=0x7f213c0af5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:37.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.277+0000 7f21416b4640 1 --2- 172.21.15.84:0/3680256701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f213c154780 0x7f213c0af5c0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f2124000ff0 tx=0x7f212400dd40 comp rx=0 tx=0).stop 2023-10-16T11:05:37.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.277+0000 7f21416b4640 1 -- 172.21.15.84:0/3680256701 shutdown_connections 2023-10-16T11:05:37.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.277+0000 7f21416b4640 1 --2- 172.21.15.84:0/3680256701 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f211c06af90 0x7f211c06d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:37.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.277+0000 7f21416b4640 1 --2- 172.21.15.84:0/3680256701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f213c0afb00 0x7f213c1e4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:37.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.277+0000 7f21416b4640 1 --2- 172.21.15.84:0/3680256701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f213c154780 0x7f213c0af5c0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:37.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.277+0000 7f21416b4640 1 --2- 172.21.15.84:0/3680256701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2128003970 0x7f213c0b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:37.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.277+0000 7f21416b4640 1 -- 172.21.15.84:0/3680256701 >> 172.21.15.84:0/3680256701 conn(0x7f213c0ce9b0 msgr2=0x7f213c159010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:37.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.277+0000 7f21416b4640 1 -- 172.21.15.84:0/3680256701 shutdown_connections 2023-10-16T11:05:37.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:37.277+0000 7f21416b4640 1 -- 172.21.15.84:0/3680256701 wait complete. 2023-10-16T11:05:37.296 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:05:37.297 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:05:37.406 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:05:37.534 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:05:37.535 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:05:37.535 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:05:37.595 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:05:37.596 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:05:37.596 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:05:37.655 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:05:37.656 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:05:37.656 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:05:37.716 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:05:37.716 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:05:37.717 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:05:37.778 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:05:37.778 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T11:05:37.779 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:05:38.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.000+0000 7faa15dff640 1 Processor -- start 2023-10-16T11:05:38.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.001+0000 7faa15dff640 1 -- start start 2023-10-16T11:05:38.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.002+0000 7faa15dff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faa100ad970 0x7faa100cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:38.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.002+0000 7faa15dff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa100ccfb0 0x7faa100cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:38.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.002+0000 7faa15dff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1015b9d0 con 0x7faa1006c040 2023-10-16T11:05:38.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.002+0000 7faa15dff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1015bcb0 con 0x7faa100ad970 2023-10-16T11:05:38.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.002+0000 7faa15dff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1015bf90 con 0x7faa100ccfb0 2023-10-16T11:05:38.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.002+0000 7faa0ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faa100ad970 0x7faa100cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:38.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.002+0000 7faa0ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faa100ad970 0x7faa100cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58978/0 (socket says 172.21.15.84:58978) 2023-10-16T11:05:38.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.002+0000 7faa0ffff640 1 -- 172.21.15.84:0/1092751554 learned_addr learned my addr 172.21.15.84:0/1092751554 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:38.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.002+0000 7faa155fe640 1 --2- 172.21.15.84:0/1092751554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa100ccfb0 0x7faa100cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:38.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.002+0000 7faa0ffff640 1 -- 172.21.15.84:0/1092751554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa100ccfb0 msgr2=0x7faa100cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:38.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.003+0000 7faa0ffff640 1 --2- 172.21.15.84:0/1092751554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa100ccfb0 0x7faa100cd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.003+0000 7faa0ffff640 1 -- 172.21.15.84:0/1092751554 >> v1:172.21.15.157:6789/0 conn(0x7faa1006c040 legacy=0x7faa1006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:38.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.003+0000 7faa0ffff640 1 -- 172.21.15.84:0/1092751554 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa100cc270 con 0x7faa100ad970 2023-10-16T11:05:38.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.004+0000 7faa0ffff640 1 --2- 172.21.15.84:0/1092751554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa100ad970 0x7faa100cca70 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7faa00009e50 tx=0x7faa0002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5c00bd04fa45a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:38.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.004+0000 7faa0f7fe640 1 -- 172.21.15.84:0/1092751554 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0002ca70 con 0x7faa100ad970 2023-10-16T11:05:38.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.004+0000 7faa0f7fe640 1 -- 172.21.15.84:0/1092751554 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0002cbd0 con 0x7faa100ad970 2023-10-16T11:05:38.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.004+0000 7faa0f7fe640 1 -- 172.21.15.84:0/1092751554 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa000343f0 con 0x7faa100ad970 2023-10-16T11:05:38.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.004+0000 7faa15dff640 1 -- 172.21.15.84:0/1092751554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa100ad970 msgr2=0x7faa100cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:38.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.004+0000 7faa15dff640 1 --2- 172.21.15.84:0/1092751554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa100ad970 0x7faa100cca70 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7faa00009e50 tx=0x7faa0002bed0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.005+0000 7faa15dff640 1 -- 172.21.15.84:0/1092751554 shutdown_connections 2023-10-16T11:05:38.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.005+0000 7faa15dff640 1 --2- 172.21.15.84:0/1092751554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa100ccfb0 0x7faa100cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.005+0000 7faa15dff640 1 --2- 172.21.15.84:0/1092751554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa100ad970 0x7faa100cca70 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.005+0000 7faa15dff640 1 -- 172.21.15.84:0/1092751554 >> 172.21.15.84:0/1092751554 conn(0x7faa100ce6a0 msgr2=0x7faa100ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:38.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.005+0000 7faa15dff640 1 -- 172.21.15.84:0/1092751554 shutdown_connections 2023-10-16T11:05:38.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.005+0000 7faa15dff640 1 -- 172.21.15.84:0/1092751554 wait complete. 2023-10-16T11:05:38.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.006+0000 7faa15dff640 1 Processor -- start 2023-10-16T11:05:38.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.007+0000 7faa15dff640 1 -- start start 2023-10-16T11:05:38.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.007+0000 7faa15dff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa1006c040 0x7faa100c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:38.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.007+0000 7faa15dff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa100ad970 0x7faa100ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:38.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.007+0000 7faa15dff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa100ccfb0 0x7faa10158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:38.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.007+0000 7faa15dff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa100ad080 con 0x7faa1006c040 2023-10-16T11:05:38.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.007+0000 7faa15dff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa100aeb00 con 0x7faa100ad970 2023-10-16T11:05:38.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.007+0000 7faa15dff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa100af920 con 0x7faa100ccfb0 2023-10-16T11:05:38.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.008+0000 7faa14dfd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa1006c040 0x7faa100c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:38.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.008+0000 7faa14dfd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa1006c040 0x7faa100c2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58990/0 (socket says 172.21.15.84:58990) 2023-10-16T11:05:38.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.008+0000 7faa14dfd640 1 -- 172.21.15.84:0/3281532418 learned_addr learned my addr 172.21.15.84:0/3281532418 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:38.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.008+0000 7faa155fe640 1 --2- 172.21.15.84:0/3281532418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa100ccfb0 0x7faa10158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:38.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.008+0000 7faa14dfd640 1 -- 172.21.15.84:0/3281532418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa100ccfb0 msgr2=0x7faa10158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:38.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.008+0000 7faa0ffff640 1 --2- 172.21.15.84:0/3281532418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa100ad970 0x7faa100ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:38.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.008+0000 7faa14dfd640 1 --2- 172.21.15.84:0/3281532418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa100ccfb0 0x7faa10158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.008+0000 7faa14dfd640 1 -- 172.21.15.84:0/3281532418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa100ad970 msgr2=0x7faa100ae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:38.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.008+0000 7faa14dfd640 1 --2- 172.21.15.84:0/3281532418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa100ad970 0x7faa100ae5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.008+0000 7faa14dfd640 1 -- 172.21.15.84:0/3281532418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa00009b30 con 0x7faa1006c040 2023-10-16T11:05:38.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.008+0000 7faa155fe640 1 --2- 172.21.15.84:0/3281532418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa100ccfb0 0x7faa10158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:38.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.008+0000 7faa14dfd640 1 --2- 172.21.15.84:0/3281532418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa1006c040 0x7faa100c2ce0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800c410 tx=0x7fa9f800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:38.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.009+0000 7faa0d7fa640 1 -- 172.21.15.84:0/3281532418 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80035f0 con 0x7faa1006c040 2023-10-16T11:05:38.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.009+0000 7faa0d7fa640 1 -- 172.21.15.84:0/3281532418 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f8003750 con 0x7faa1006c040 2023-10-16T11:05:38.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.009+0000 7faa0d7fa640 1 -- 172.21.15.84:0/3281532418 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8003b40 con 0x7faa1006c040 2023-10-16T11:05:38.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.009+0000 7faa15dff640 1 -- 172.21.15.84:0/3281532418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa10158cd0 con 0x7faa1006c040 2023-10-16T11:05:38.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.009+0000 7faa15dff640 1 -- 172.21.15.84:0/3281532418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa100aeff0 con 0x7faa1006c040 2023-10-16T11:05:38.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.010+0000 7faa15dff640 1 -- 172.21.15.84:0/3281532418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1006b330 con 0x7faa1006c040 2023-10-16T11:05:38.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.011+0000 7faa0d7fa640 1 -- 172.21.15.84:0/3281532418 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa9f8003ca0 con 0x7faa1006c040 2023-10-16T11:05:38.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.012+0000 7faa0d7fa640 1 --2- 172.21.15.84:0/3281532418 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa9f006af90 0x7fa9f006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:38.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.012+0000 7faa0d7fa640 1 -- 172.21.15.84:0/3281532418 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fa9f8088970 con 0x7faa1006c040 2023-10-16T11:05:38.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.012+0000 7faa0ffff640 1 --2- 172.21.15.84:0/3281532418 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa9f006af90 0x7fa9f006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:38.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.013+0000 7faa0ffff640 1 --2- 172.21.15.84:0/3281532418 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa9f006af90 0x7fa9f006d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faa00035000 tx=0x7faa00030000 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:38.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.016+0000 7faa0d7fa640 1 -- 172.21.15.84:0/3281532418 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa9f8053940 con 0x7faa1006c040 2023-10-16T11:05:38.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.245+0000 7faa15dff640 1 -- 172.21.15.84:0/3281532418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa100afc00 con 0x7faa1006c040 2023-10-16T11:05:38.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.247+0000 7faa0d7fa640 1 -- 172.21.15.84:0/3281532418 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fa9f8017e90 con 0x7faa1006c040 2023-10-16T11:05:38.248 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:38.248 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":82,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6517,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1733540616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1733540616},{"type":"v1","addr":"172.21.15.157:6833","nonce":1733540616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3815048417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3815048417},{"type":"v1","addr":"172.21.15.157:6835","nonce":3815048417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9135,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1038548935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1038548935},{"type":"v1","addr":"172.21.15.84:6836","nonce":1038548935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9597,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/584870642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":584870642},{"type":"v1","addr":"172.21.15.84:6837","nonce":584870642}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82}],"filesystems":[]} 2023-10-16T11:05:38.249 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 82 2023-10-16T11:05:38.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.249+0000 7faa15dff640 1 -- 172.21.15.84:0/3281532418 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa9f006af90 msgr2=0x7fa9f006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:38.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.249+0000 7faa15dff640 1 --2- 172.21.15.84:0/3281532418 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa9f006af90 0x7fa9f006d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faa00035000 tx=0x7faa00030000 comp rx=0 tx=0).stop 2023-10-16T11:05:38.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.249+0000 7faa15dff640 1 -- 172.21.15.84:0/3281532418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa1006c040 msgr2=0x7faa100c2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:38.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.249+0000 7faa15dff640 1 --2- 172.21.15.84:0/3281532418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa1006c040 0x7faa100c2ce0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800c410 tx=0x7fa9f800c8d0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.250+0000 7faa15dff640 1 -- 172.21.15.84:0/3281532418 shutdown_connections 2023-10-16T11:05:38.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.250+0000 7faa15dff640 1 --2- 172.21.15.84:0/3281532418 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa9f006af90 0x7fa9f006d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.250+0000 7faa15dff640 1 --2- 172.21.15.84:0/3281532418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa100ccfb0 0x7faa10158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.250+0000 7faa15dff640 1 --2- 172.21.15.84:0/3281532418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa100ad970 0x7faa100ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.250+0000 7faa15dff640 1 --2- 172.21.15.84:0/3281532418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa1006c040 0x7faa100c2ce0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.250+0000 7faa15dff640 1 -- 172.21.15.84:0/3281532418 >> 172.21.15.84:0/3281532418 conn(0x7faa100ce6a0 msgr2=0x7faa1015c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:38.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.250+0000 7faa15dff640 1 -- 172.21.15.84:0/3281532418 shutdown_connections 2023-10-16T11:05:38.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.250+0000 7faa15dff640 1 -- 172.21.15.84:0/3281532418 wait complete. 2023-10-16T11:05:38.268 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T11:05:38.268 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T11:05:38.268 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T11:05:38.268 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:05:38.269 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T11:05:38.269 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T11:05:38.269 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T11:05:38.270 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:05:38.270 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T11:05:38.270 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T11:05:38.271 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T11:05:38.271 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:05:38.271 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T11:05:38.271 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T11:05:38.272 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T11:05:38.272 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:05:38.288 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T11:05:38.288 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T11:05:38.289 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T11:05:38.290 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T11:05:38.293 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T11:05:38.293 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T11:05:38.294 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T11:05:38.294 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T11:05:38.336 INFO:tasks.ceph.mds.a:Started 2023-10-16T11:05:38.337 DEBUG:teuthology.parallel:result is None 2023-10-16T11:05:38.340 INFO:tasks.ceph.mds.b:Started 2023-10-16T11:05:38.341 DEBUG:teuthology.parallel:result is None 2023-10-16T11:05:38.344 INFO:tasks.ceph.mds.c:Started 2023-10-16T11:05:38.344 DEBUG:teuthology.parallel:result is None 2023-10-16T11:05:38.348 INFO:tasks.ceph.mds.d:Started 2023-10-16T11:05:38.348 DEBUG:teuthology.parallel:result is None 2023-10-16T11:05:38.349 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T11:05:38.491 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T11:05:38.498 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T11:05:38.512 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-10-16T11:05:38.514 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T11:05:38.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.529+0000 7fd37ffff640 1 Processor -- start 2023-10-16T11:05:38.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.529+0000 7fd37ffff640 1 -- start start 2023-10-16T11:05:38.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.530+0000 7fd37ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd3800cdf30 0x7fd380071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:38.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.530+0000 7fd37ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd380071fa0 0x7fd38006aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:38.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.530+0000 7fd37ffff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3800d12f0 con 0x7fd38015cc80 2023-10-16T11:05:38.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.530+0000 7fd37ffff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3800d0500 con 0x7fd3800cdf30 2023-10-16T11:05:38.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.530+0000 7fd37ffff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3800723d0 con 0x7fd380071fa0 2023-10-16T11:05:38.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.531+0000 7fd37effd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd38015cc80 0x7fd3800cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42458/0 (socket says 172.21.15.84:42458) 2023-10-16T11:05:38.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.531+0000 7fd37effd640 1 -- 172.21.15.84:0/2868987110 learned_addr learned my addr 172.21.15.84:0/2868987110 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:38.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.531+0000 7fd3767fc640 1 --2- 172.21.15.84:0/2868987110 >> v2:172.21.15.157:3300/0 conn(0x7fd3800cdf30 0x7fd380071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:38.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.531+0000 7fd37f7fe640 1 --2- 172.21.15.84:0/2868987110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd380071fa0 0x7fd38006aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:38.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.531+0000 7fd37e7fc640 1 -- 172.21.15.84:0/2868987110 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758298683 0 0) 0x7fd3800d12f0 con 0x7fd38015cc80 2023-10-16T11:05:38.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.531+0000 7fd37e7fc640 1 -- 172.21.15.84:0/2868987110 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd36c003520 con 0x7fd38015cc80 2023-10-16T11:05:38.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.531+0000 7fd3767fc640 1 -- 172.21.15.84:0/2868987110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd380071fa0 msgr2=0x7fd38006aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:38.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.531+0000 7fd3767fc640 1 --2- 172.21.15.84:0/2868987110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd380071fa0 0x7fd38006aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.531+0000 7fd3767fc640 1 -- 172.21.15.84:0/2868987110 >> v1:172.21.15.84:6789/0 conn(0x7fd38015cc80 legacy=0x7fd3800cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:38.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.531+0000 7fd3767fc640 1 -- 172.21.15.84:0/2868987110 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3800d06e0 con 0x7fd3800cdf30 2023-10-16T11:05:38.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.533+0000 7fd3767fc640 1 --2- 172.21.15.84:0/2868987110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3800cdf30 0x7fd380071a60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd3680097a0 tx=0x7fd36802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f40dd6aba0f2a3c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:38.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.533+0000 7fd37e7fc640 1 -- 172.21.15.84:0/2868987110 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd368031830 con 0x7fd3800cdf30 2023-10-16T11:05:38.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.533+0000 7fd37e7fc640 1 -- 172.21.15.84:0/2868987110 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd368031990 con 0x7fd3800cdf30 2023-10-16T11:05:38.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.533+0000 7fd37e7fc640 1 -- 172.21.15.84:0/2868987110 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd368031d20 con 0x7fd3800cdf30 2023-10-16T11:05:38.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.533+0000 7fd37ffff640 1 -- 172.21.15.84:0/2868987110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3800cdf30 msgr2=0x7fd380071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:38.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.533+0000 7fd37ffff640 1 --2- 172.21.15.84:0/2868987110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3800cdf30 0x7fd380071a60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd3680097a0 tx=0x7fd36802b8d0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.533+0000 7fd37ffff640 1 -- 172.21.15.84:0/2868987110 shutdown_connections 2023-10-16T11:05:38.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.533+0000 7fd37ffff640 1 --2- 172.21.15.84:0/2868987110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd380071fa0 0x7fd38006aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.533+0000 7fd37ffff640 1 --2- 172.21.15.84:0/2868987110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3800cdf30 0x7fd380071a60 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.533+0000 7fd37ffff640 1 -- 172.21.15.84:0/2868987110 >> 172.21.15.84:0/2868987110 conn(0x7fd380065ed0 msgr2=0x7fd3800662e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:38.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.533+0000 7fd37ffff640 1 -- 172.21.15.84:0/2868987110 shutdown_connections 2023-10-16T11:05:38.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.534+0000 7fd37ffff640 1 -- 172.21.15.84:0/2868987110 wait complete. 2023-10-16T11:05:38.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.535+0000 7fd37ffff640 1 Processor -- start 2023-10-16T11:05:38.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.535+0000 7fd37ffff640 1 -- start start 2023-10-16T11:05:38.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.536+0000 7fd37ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd380071fa0 0x7fd3800c5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:38.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.536+0000 7fd37ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3800cdf30 0x7fd3800b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:38.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.536+0000 7fd37ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd38015cc80 0x7fd3800b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:38.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.536+0000 7fd37ffff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3800b4f50 con 0x7fd3800cdf30 2023-10-16T11:05:38.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.536+0000 7fd37ffff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3800c5ab0 con 0x7fd380071fa0 2023-10-16T11:05:38.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.536+0000 7fd37ffff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3800c5d90 con 0x7fd38015cc80 2023-10-16T11:05:38.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.536+0000 7fd3767fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3800cdf30 0x7fd3800b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:38.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.536+0000 7fd3767fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3800cdf30 0x7fd3800b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59020/0 (socket says 172.21.15.84:59020) 2023-10-16T11:05:38.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.536+0000 7fd3767fc640 1 -- 172.21.15.84:0/2820266002 learned_addr learned my addr 172.21.15.84:0/2820266002 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:38.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.537+0000 7fd37effd640 1 --2- 172.21.15.84:0/2820266002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd380071fa0 0x7fd3800c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:38.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.537+0000 7fd37f7fe640 1 --2- 172.21.15.84:0/2820266002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd38015cc80 0x7fd3800b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:38.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.537+0000 7fd3767fc640 1 -- 172.21.15.84:0/2820266002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd38015cc80 msgr2=0x7fd3800b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:38.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.537+0000 7fd3767fc640 1 --2- 172.21.15.84:0/2820266002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd38015cc80 0x7fd3800b5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.537+0000 7fd3767fc640 1 -- 172.21.15.84:0/2820266002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd380071fa0 msgr2=0x7fd3800c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:38.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.537+0000 7fd3767fc640 1 --2- 172.21.15.84:0/2820266002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd380071fa0 0x7fd3800c5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.537+0000 7fd3767fc640 1 -- 172.21.15.84:0/2820266002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd368009400 con 0x7fd3800cdf30 2023-10-16T11:05:38.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.537+0000 7fd3767fc640 1 --2- 172.21.15.84:0/2820266002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3800cdf30 0x7fd3800b4a10 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fd36802bee0 tx=0x7fd3680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:38.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.537+0000 7fd377fff640 1 -- 172.21.15.84:0/2820266002 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36803a630 con 0x7fd3800cdf30 2023-10-16T11:05:38.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.538+0000 7fd377fff640 1 -- 172.21.15.84:0/2820266002 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36803a7b0 con 0x7fd3800cdf30 2023-10-16T11:05:38.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.538+0000 7fd37ffff640 1 -- 172.21.15.84:0/2820266002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3800c6070 con 0x7fd3800cdf30 2023-10-16T11:05:38.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.538+0000 7fd377fff640 1 -- 172.21.15.84:0/2820266002 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36803ab60 con 0x7fd3800cdf30 2023-10-16T11:05:38.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.538+0000 7fd37ffff640 1 -- 172.21.15.84:0/2820266002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3800b3630 con 0x7fd3800cdf30 2023-10-16T11:05:38.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.539+0000 7fd37ffff640 1 -- 172.21.15.84:0/2820266002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38004e9b0 con 0x7fd3800cdf30 2023-10-16T11:05:38.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.540+0000 7fd377fff640 1 -- 172.21.15.84:0/2820266002 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd36803acc0 con 0x7fd3800cdf30 2023-10-16T11:05:38.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.540+0000 7fd377fff640 1 --2- 172.21.15.84:0/2820266002 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd36006af90 0x7fd36006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:38.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.541+0000 7fd377fff640 1 -- 172.21.15.84:0/2820266002 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fd3680ab400 con 0x7fd3800cdf30 2023-10-16T11:05:38.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.541+0000 7fd37effd640 1 --2- 172.21.15.84:0/2820266002 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd36006af90 0x7fd36006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:38.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.541+0000 7fd37effd640 1 --2- 172.21.15.84:0/2820266002 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd36006af90 0x7fd36006d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd370001560 tx=0x7fd370001a90 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:38.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.545+0000 7fd377fff640 1 -- 172.21.15.84:0/2820266002 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd368076360 con 0x7fd3800cdf30 2023-10-16T11:05:38.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.750+0000 7fd37ffff640 1 -- 172.21.15.84:0/2820266002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd3800b6860 con 0x7fd3800cdf30 2023-10-16T11:05:38.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.846+0000 7fd377fff640 1 -- 172.21.15.84:0/2820266002 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v165) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd368079980 con 0x7fd3800cdf30 2023-10-16T11:05:38.847 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-10-16T11:05:38.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.849+0000 7fd37ffff640 1 -- 172.21.15.84:0/2820266002 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd36006af90 msgr2=0x7fd36006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:38.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.849+0000 7fd37ffff640 1 --2- 172.21.15.84:0/2820266002 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd36006af90 0x7fd36006d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd370001560 tx=0x7fd370001a90 comp rx=0 tx=0).stop 2023-10-16T11:05:38.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.849+0000 7fd37ffff640 1 -- 172.21.15.84:0/2820266002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3800cdf30 msgr2=0x7fd3800b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:38.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.849+0000 7fd37ffff640 1 --2- 172.21.15.84:0/2820266002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3800cdf30 0x7fd3800b4a10 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fd36802bee0 tx=0x7fd3680046c0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.849+0000 7fd37ffff640 1 -- 172.21.15.84:0/2820266002 shutdown_connections 2023-10-16T11:05:38.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.849+0000 7fd37ffff640 1 --2- 172.21.15.84:0/2820266002 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd36006af90 0x7fd36006d450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.849+0000 7fd37ffff640 1 --2- 172.21.15.84:0/2820266002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd38015cc80 0x7fd3800b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.849+0000 7fd37ffff640 1 --2- 172.21.15.84:0/2820266002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3800cdf30 0x7fd3800b4a10 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.849+0000 7fd37ffff640 1 --2- 172.21.15.84:0/2820266002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd380071fa0 0x7fd3800c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:38.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.849+0000 7fd37ffff640 1 -- 172.21.15.84:0/2820266002 >> 172.21.15.84:0/2820266002 conn(0x7fd380065ed0 msgr2=0x7fd38015d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:38.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.849+0000 7fd37ffff640 1 -- 172.21.15.84:0/2820266002 shutdown_connections 2023-10-16T11:05:38.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:38.849+0000 7fd37ffff640 1 -- 172.21.15.84:0/2820266002 wait complete. 2023-10-16T11:05:38.871 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T11:05:39.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.074+0000 7f99c8461640 1 Processor -- start 2023-10-16T11:05:39.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.074+0000 7f99c8461640 1 -- start start 2023-10-16T11:05:39.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.076+0000 7f99c8461640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f99c00b0b20 0x7f99c0164b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:39.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.076+0000 7f99c8461640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c006f7f0 con 0x7f99c0159f00 2023-10-16T11:05:39.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.076+0000 7f99c8461640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c006c5e0 con 0x7f99c0159b20 2023-10-16T11:05:39.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.076+0000 7f99c8461640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c006c8c0 con 0x7f99c00b0b20 2023-10-16T11:05:39.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.076+0000 7f99c69fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f99c00b0b20 0x7f99c0164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:39.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.076+0000 7f99c59fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f99c0159f00 0x7f99c00b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42466/0 (socket says 172.21.15.84:42466) 2023-10-16T11:05:39.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.076+0000 7f99c59fc640 1 -- 172.21.15.84:0/4183237106 learned_addr learned my addr 172.21.15.84:0/4183237106 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:39.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.076+0000 7f99c69fe640 1 -- 172.21.15.84:0/4183237106 >> v1:172.21.15.157:6789/0 conn(0x7f99c0159b20 legacy=0x7f99c006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:39.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.076+0000 7f99c69fe640 1 -- 172.21.15.84:0/4183237106 >> v1:172.21.15.84:6789/0 conn(0x7f99c0159f00 legacy=0x7f99c00b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:39.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.076+0000 7f99c69fe640 1 -- 172.21.15.84:0/4183237106 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c006cba0 con 0x7f99c00b0b20 2023-10-16T11:05:39.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.077+0000 7f99c69fe640 1 --2- 172.21.15.84:0/4183237106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c00b0b20 0x7f99c0164b40 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009d90 tx=0x7f99b802e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fb2dd24f0e4c7f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:39.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.078+0000 7f99c51fb640 1 -- 172.21.15.84:0/4183237106 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b802ee70 con 0x7f99c00b0b20 2023-10-16T11:05:39.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.078+0000 7f99c51fb640 1 -- 172.21.15.84:0/4183237106 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b8036440 con 0x7f99c00b0b20 2023-10-16T11:05:39.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.078+0000 7f99c51fb640 1 -- 172.21.15.84:0/4183237106 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b80367d0 con 0x7f99c00b0b20 2023-10-16T11:05:39.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.078+0000 7f99c8461640 1 -- 172.21.15.84:0/4183237106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c00b0b20 msgr2=0x7f99c0164b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:39.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.078+0000 7f99c8461640 1 --2- 172.21.15.84:0/4183237106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c00b0b20 0x7f99c0164b40 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009d90 tx=0x7f99b802e0c0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.079+0000 7f99c8461640 1 -- 172.21.15.84:0/4183237106 shutdown_connections 2023-10-16T11:05:39.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.079+0000 7f99c8461640 1 --2- 172.21.15.84:0/4183237106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c00b0b20 0x7f99c0164b40 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.079+0000 7f99c8461640 1 -- 172.21.15.84:0/4183237106 >> 172.21.15.84:0/4183237106 conn(0x7f99c00cb410 msgr2=0x7f99c00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:39.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.079+0000 7f99c8461640 1 -- 172.21.15.84:0/4183237106 shutdown_connections 2023-10-16T11:05:39.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.079+0000 7f99c8461640 1 -- 172.21.15.84:0/4183237106 wait complete. 2023-10-16T11:05:39.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.080+0000 7f99c8461640 1 Processor -- start 2023-10-16T11:05:39.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.080+0000 7f99c8461640 1 -- start start 2023-10-16T11:05:39.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.081+0000 7f99c8461640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c00b0b20 0x7f99c01543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:39.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.081+0000 7f99c8461640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99c0159b20 0x7f99c00c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:39.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.081+0000 7f99c8461640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c0159f00 0x7f99c0157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:39.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.081+0000 7f99c8461640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c0154910 con 0x7f99c0159f00 2023-10-16T11:05:39.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.081+0000 7f99c8461640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00c5a60 con 0x7f99c00b0b20 2023-10-16T11:05:39.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.081+0000 7f99c8461640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00c8560 con 0x7f99c0159b20 2023-10-16T11:05:39.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.082+0000 7f99c69fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c0159f00 0x7f99c0157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:39.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.082+0000 7f99c69fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c0159f00 0x7f99c0157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59040/0 (socket says 172.21.15.84:59040) 2023-10-16T11:05:39.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.082+0000 7f99c69fe640 1 -- 172.21.15.84:0/2737885053 learned_addr learned my addr 172.21.15.84:0/2737885053 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:39.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.082+0000 7f99c61fd640 1 --2- 172.21.15.84:0/2737885053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c00b0b20 0x7f99c01543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:39.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.082+0000 7f99c69fe640 1 -- 172.21.15.84:0/2737885053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99c0159b20 msgr2=0x7f99c00c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:39.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.082+0000 7f99c69fe640 1 --2- 172.21.15.84:0/2737885053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99c0159b20 0x7f99c00c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.082+0000 7f99c69fe640 1 -- 172.21.15.84:0/2737885053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c00b0b20 msgr2=0x7f99c01543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:39.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.082+0000 7f99c69fe640 1 --2- 172.21.15.84:0/2737885053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c00b0b20 0x7f99c01543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.082+0000 7f99c69fe640 1 -- 172.21.15.84:0/2737885053 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b8009a70 con 0x7f99c0159f00 2023-10-16T11:05:39.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.082+0000 7f99c61fd640 1 --2- 172.21.15.84:0/2737885053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c00b0b20 0x7f99c01543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:39.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.082+0000 7f99c69fe640 1 --2- 172.21.15.84:0/2737885053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c0159f00 0x7f99c0157c40 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009d90 tx=0x7f99b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:39.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.083+0000 7f99b6ffd640 1 -- 172.21.15.84:0/2737885053 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8034850 con 0x7f99c0159f00 2023-10-16T11:05:39.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.083+0000 7f99b6ffd640 1 -- 172.21.15.84:0/2737885053 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b80349d0 con 0x7f99c0159f00 2023-10-16T11:05:39.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.083+0000 7f99c8461640 1 -- 172.21.15.84:0/2737885053 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c00c78e0 con 0x7f99c0159f00 2023-10-16T11:05:39.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.083+0000 7f99b6ffd640 1 -- 172.21.15.84:0/2737885053 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8034d80 con 0x7f99c0159f00 2023-10-16T11:05:39.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.083+0000 7f99c8461640 1 -- 172.21.15.84:0/2737885053 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c00c7ea0 con 0x7f99c0159f00 2023-10-16T11:05:39.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.084+0000 7f99c8461640 1 -- 172.21.15.84:0/2737885053 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c00716e0 con 0x7f99c0159f00 2023-10-16T11:05:39.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.085+0000 7f99b6ffd640 1 -- 172.21.15.84:0/2737885053 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f99b803d050 con 0x7f99c0159f00 2023-10-16T11:05:39.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.086+0000 7f99b6ffd640 1 --2- 172.21.15.84:0/2737885053 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f99a406af90 0x7f99a406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:39.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.086+0000 7f99b6ffd640 1 -- 172.21.15.84:0/2737885053 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f99b80adff0 con 0x7f99c0159f00 2023-10-16T11:05:39.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.086+0000 7f99c61fd640 1 --2- 172.21.15.84:0/2737885053 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f99a406af90 0x7f99a406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:39.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.086+0000 7f99c61fd640 1 --2- 172.21.15.84:0/2737885053 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f99a406af90 0x7f99a406d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f99c015d630 tx=0x7f99bc009040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:39.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.090+0000 7f99b6ffd640 1 -- 172.21.15.84:0/2737885053 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f99b80793d0 con 0x7f99c0159f00 2023-10-16T11:05:39.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.332+0000 7f99c8461640 1 -- 172.21.15.84:0/2737885053 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f99c004e160 con 0x7f99c0159f00 2023-10-16T11:05:39.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.333+0000 7f99b6ffd640 1 -- 172.21.15.84:0/2737885053 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v45) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f99b807c9f0 con 0x7f99c0159f00 2023-10-16T11:05:39.334 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:39.335 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:05:39.335 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-10-16T11:05:39.335 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.336 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-10-16T11:05:39.336 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l6dhjKRAA0pMvY6AK6qtFXDprNhwsqg==", 2023-10-16T11:05:39.336 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.336 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:05:39.336 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:05:39.337 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:05:39.337 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:05:39.337 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.337 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:05:39.338 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.338 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-10-16T11:05:39.338 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lFpMxEBAA0TLPRt5q3FwwBvMG019cjQ==", 2023-10-16T11:05:39.338 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.338 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:05:39.339 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:05:39.339 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:05:39.339 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:05:39.339 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.340 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:05:39.340 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.340 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-10-16T11:05:39.340 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l73RBOhAAOMEPxfUAMHhn5Blyvd5oMw==", 2023-10-16T11:05:39.340 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.341 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:05:39.341 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:05:39.341 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:05:39.341 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:05:39.342 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.342 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:05:39.342 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.342 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-10-16T11:05:39.342 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lcTIRIhAA0+FWm4b4yw77LD3+WKbk7w==", 2023-10-16T11:05:39.343 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.343 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:05:39.343 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:05:39.343 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:05:39.344 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:05:39.344 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.344 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:05:39.344 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.345 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-10-16T11:05:39.345 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDYBi1l1zBLERAA3YLW3xIYQoRVw7KSkwN3Og==", 2023-10-16T11:05:39.345 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.345 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:05:39.346 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:05:39.346 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:05:39.346 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.347 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:05:39.347 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.347 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-10-16T11:05:39.348 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDbBi1lFUUxKRAAQbus8SmYprJRmWTjoCznNA==", 2023-10-16T11:05:39.348 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.348 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:05:39.348 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:05:39.349 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:05:39.349 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.349 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:05:39.349 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.350 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-10-16T11:05:39.350 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDeBi1l+9lBMxAArl6sb/h15uBG/a9lVH40Hg==", 2023-10-16T11:05:39.350 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.350 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:05:39.351 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:05:39.351 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:05:39.351 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.351 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:05:39.351 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.352 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-10-16T11:05:39.352 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDiBi1l2rnEERAA3oRD71xa0fEOynM8zKuBEg==", 2023-10-16T11:05:39.352 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.352 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:05:39.353 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:05:39.353 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:05:39.353 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.353 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:05:39.353 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.354 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-10-16T11:05:39.354 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDnBi1lR7oLChAAZto8fDovDbpOd3Fcnc7C6w==", 2023-10-16T11:05:39.354 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.355 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:05:39.355 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:05:39.355 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:05:39.355 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.356 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:05:39.356 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.356 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-10-16T11:05:39.356 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDqBi1lmjg/JBAAlINKRGBc0/UWJJHMVFw57w==", 2023-10-16T11:05:39.357 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.357 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:05:39.357 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:05:39.357 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:05:39.358 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.358 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:05:39.358 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.358 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-10-16T11:05:39.358 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDuBi1lu0kMARAAtuSeM8F3yr8CVrc3p1biaQ==", 2023-10-16T11:05:39.359 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.359 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:05:39.359 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:05:39.359 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:05:39.359 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.360 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:05:39.360 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.360 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-10-16T11:05:39.360 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDxBi1lf6mNGBAAHGbqzJ0fmDxQIsasboHMfg==", 2023-10-16T11:05:39.361 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.361 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:05:39.361 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:05:39.361 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:05:39.361 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.362 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:05:39.362 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.362 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-10-16T11:05:39.362 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ==", 2023-10-16T11:05:39.362 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.363 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:05:39.363 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow r", 2023-10-16T11:05:39.363 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T11:05:39.363 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.363 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:05:39.364 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.364 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-10-16T11:05:39.364 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1lHU+4BxAAd8ET8hQ8Y+EUwdTJr9/+Ig==", 2023-10-16T11:05:39.364 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.365 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:05:39.365 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:05:39.365 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:05:39.365 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:05:39.366 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.366 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:05:39.366 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.366 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-10-16T11:05:39.367 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lA1BOORAAFNyfqgEF34MVR9YX3KBKkQ==", 2023-10-16T11:05:39.367 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.367 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:05:39.367 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:05:39.367 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:05:39.368 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:05:39.368 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.368 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:05:39.369 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.369 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-10-16T11:05:39.369 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1l848oERAANMfrZz9JrlgNVh7ZFbtnaA==", 2023-10-16T11:05:39.370 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.370 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:05:39.370 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:05:39.371 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:05:39.371 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:05:39.371 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.372 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:05:39.372 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.373 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-10-16T11:05:39.373 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDRBi1lTAxTGRAAMptxnxy3Qit/cTCXie6CrQ==", 2023-10-16T11:05:39.373 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.374 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:05:39.374 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:05:39.375 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:05:39.375 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:05:39.375 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.376 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:05:39.376 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.377 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-10-16T11:05:39.377 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lPuJxHRAABAnGPVCrkq0JR4Y6iFGZlw==", 2023-10-16T11:05:39.377 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.378 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:05:39.378 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T11:05:39.379 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:05:39.379 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.379 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:05:39.379 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:39.380 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-10-16T11:05:39.380 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lLxnKEBAA+a/pOHUp7Sg3m4weAust5w==", 2023-10-16T11:05:39.380 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:05:39.381 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:05:39.381 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T11:05:39.381 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:05:39.381 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.381 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:39.382 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-10-16T11:05:39.382 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:05:39.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.336+0000 7f99c8461640 1 -- 172.21.15.84:0/2737885053 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f99a406af90 msgr2=0x7f99a406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:39.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.336+0000 7f99c8461640 1 --2- 172.21.15.84:0/2737885053 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f99a406af90 0x7f99a406d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f99c015d630 tx=0x7f99bc009040 comp rx=0 tx=0).stop 2023-10-16T11:05:39.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.336+0000 7f99c8461640 1 -- 172.21.15.84:0/2737885053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c0159f00 msgr2=0x7f99c0157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:39.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.336+0000 7f99c8461640 1 --2- 172.21.15.84:0/2737885053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c0159f00 0x7f99c0157c40 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009d90 tx=0x7f99b8004a40 comp rx=0 tx=0).stop 2023-10-16T11:05:39.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.337+0000 7f99c8461640 1 -- 172.21.15.84:0/2737885053 shutdown_connections 2023-10-16T11:05:39.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.337+0000 7f99c8461640 1 --2- 172.21.15.84:0/2737885053 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f99a406af90 0x7f99a406d450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.337+0000 7f99c8461640 1 --2- 172.21.15.84:0/2737885053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c0159f00 0x7f99c0157c40 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.337+0000 7f99c8461640 1 --2- 172.21.15.84:0/2737885053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99c0159b20 0x7f99c00c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.337+0000 7f99c8461640 1 --2- 172.21.15.84:0/2737885053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c00b0b20 0x7f99c01543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.337+0000 7f99c8461640 1 -- 172.21.15.84:0/2737885053 >> 172.21.15.84:0/2737885053 conn(0x7f99c00cb410 msgr2=0x7f99c00ad9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:39.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.337+0000 7f99c8461640 1 -- 172.21.15.84:0/2737885053 shutdown_connections 2023-10-16T11:05:39.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.337+0000 7f99c8461640 1 -- 172.21.15.84:0/2737885053 wait complete. 2023-10-16T11:05:39.387 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:05:39.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.600+0000 7fd29fccf640 1 Processor -- start 2023-10-16T11:05:39.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.600+0000 7fd29fccf640 1 -- start start 2023-10-16T11:05:39.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.602+0000 7fd29fccf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd2980b0b20 0x7fd2981615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:39.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.602+0000 7fd29fccf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd29815d3f0 0x7fd29815d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:39.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.602+0000 7fd29fccf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29815e3e0 con 0x7fd2980cd0c0 2023-10-16T11:05:39.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.602+0000 7fd29fccf640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2980cd4a0 con 0x7fd2980b0b20 2023-10-16T11:05:39.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.602+0000 7fd29fccf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29815e100 con 0x7fd29815d3f0 2023-10-16T11:05:39.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.602+0000 7fd29d26a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd2980b0b20 0x7fd2981615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:39.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.602+0000 7fd29d26a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd2980b0b20 0x7fd2981615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59042/0 (socket says 172.21.15.84:59042) 2023-10-16T11:05:39.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.602+0000 7fd29d26a640 1 -- 172.21.15.84:0/2855993977 learned_addr learned my addr 172.21.15.84:0/2855993977 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:39.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.602+0000 7fd29da6b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd2980cd0c0 0x7fd2980b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42474/0 (socket says 172.21.15.84:42474) 2023-10-16T11:05:39.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.603+0000 7fd29d26a640 1 -- 172.21.15.84:0/2855993977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd29815d3f0 msgr2=0x7fd29815d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:39.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.603+0000 7fd29d26a640 1 --2- 172.21.15.84:0/2855993977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd29815d3f0 0x7fd29815d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.603+0000 7fd29d26a640 1 -- 172.21.15.84:0/2855993977 >> v1:172.21.15.84:6789/0 conn(0x7fd2980cd0c0 legacy=0x7fd2980b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:39.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.603+0000 7fd29d26a640 1 -- 172.21.15.84:0/2855993977 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd298160b10 con 0x7fd2980b0b20 2023-10-16T11:05:39.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.603+0000 7fd29ca69640 1 -- 172.21.15.84:0/2855993977 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002168666 0 0) 0x7fd29815e3e0 con 0x7fd2980cd0c0 2023-10-16T11:05:39.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.604+0000 7fd29d26a640 1 --2- 172.21.15.84:0/2855993977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2980b0b20 0x7fd2981615f0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fd288009870 tx=0x7fd28802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3449c3a95c73046e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:39.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.604+0000 7fd29ca69640 1 -- 172.21.15.84:0/2855993977 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288031640 con 0x7fd2980b0b20 2023-10-16T11:05:39.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.604+0000 7fd29ca69640 1 -- 172.21.15.84:0/2855993977 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2880317a0 con 0x7fd2980b0b20 2023-10-16T11:05:39.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.604+0000 7fd29ca69640 1 -- 172.21.15.84:0/2855993977 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288031b30 con 0x7fd2980b0b20 2023-10-16T11:05:39.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.605+0000 7fd29fccf640 1 -- 172.21.15.84:0/2855993977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2980b0b20 msgr2=0x7fd2981615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:39.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.605+0000 7fd29fccf640 1 --2- 172.21.15.84:0/2855993977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2980b0b20 0x7fd2981615f0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fd288009870 tx=0x7fd28802b8f0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.605+0000 7fd29fccf640 1 -- 172.21.15.84:0/2855993977 shutdown_connections 2023-10-16T11:05:39.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.605+0000 7fd29fccf640 1 --2- 172.21.15.84:0/2855993977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd29815d3f0 0x7fd29815d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.605+0000 7fd29fccf640 1 --2- 172.21.15.84:0/2855993977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2980b0b20 0x7fd2981615f0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.605+0000 7fd29fccf640 1 -- 172.21.15.84:0/2855993977 >> 172.21.15.84:0/2855993977 conn(0x7fd2980cb410 msgr2=0x7fd2980cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:39.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.605+0000 7fd29fccf640 1 -- 172.21.15.84:0/2855993977 shutdown_connections 2023-10-16T11:05:39.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.605+0000 7fd29fccf640 1 -- 172.21.15.84:0/2855993977 wait complete. 2023-10-16T11:05:39.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.607+0000 7fd29fccf640 1 Processor -- start 2023-10-16T11:05:39.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.607+0000 7fd29fccf640 1 -- start start 2023-10-16T11:05:39.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.608+0000 7fd29fccf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2980b0b20 0x7fd2981e89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:39.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.608+0000 7fd29fccf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2980cd0c0 0x7fd2980b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:39.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.608+0000 7fd29fccf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd29815d3f0 0x7fd298143490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:39.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.608+0000 7fd29fccf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2981e8f20 con 0x7fd2980cd0c0 2023-10-16T11:05:39.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.608+0000 7fd29fccf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd298157330 con 0x7fd29815d3f0 2023-10-16T11:05:39.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.608+0000 7fd29fccf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2981e7940 con 0x7fd2980b0b20 2023-10-16T11:05:39.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.609+0000 7fd29d26a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2980cd0c0 0x7fd2980b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:39.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.609+0000 7fd29d26a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2980cd0c0 0x7fd2980b3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59054/0 (socket says 172.21.15.84:59054) 2023-10-16T11:05:39.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.609+0000 7fd29d26a640 1 -- 172.21.15.84:0/1812646901 learned_addr learned my addr 172.21.15.84:0/1812646901 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:39.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.609+0000 7fd29d26a640 1 -- 172.21.15.84:0/1812646901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2980b0b20 msgr2=0x7fd2981e89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:39.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.609+0000 7fd29d26a640 1 --2- 172.21.15.84:0/1812646901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2980b0b20 0x7fd2981e89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.609+0000 7fd29d26a640 1 -- 172.21.15.84:0/1812646901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd29815d3f0 msgr2=0x7fd298143490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:39.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.609+0000 7fd29e26c640 1 --2- 172.21.15.84:0/1812646901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd29815d3f0 0x7fd298143490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:39.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.609+0000 7fd29d26a640 1 --2- 172.21.15.84:0/1812646901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd29815d3f0 0x7fd298143490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.609+0000 7fd29d26a640 1 -- 172.21.15.84:0/1812646901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2880094d0 con 0x7fd2980cd0c0 2023-10-16T11:05:39.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.609+0000 7fd29d26a640 1 --2- 172.21.15.84:0/1812646901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2980cd0c0 0x7fd2980b3e70 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fd288009870 tx=0x7fd2880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:39.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.610+0000 7fd28e7fc640 1 -- 172.21.15.84:0/1812646901 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28803ac60 con 0x7fd2980cd0c0 2023-10-16T11:05:39.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.610+0000 7fd28e7fc640 1 -- 172.21.15.84:0/1812646901 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28803adc0 con 0x7fd2980cd0c0 2023-10-16T11:05:39.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.610+0000 7fd29fccf640 1 -- 172.21.15.84:0/1812646901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2981e7c20 con 0x7fd2980cd0c0 2023-10-16T11:05:39.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.610+0000 7fd28e7fc640 1 -- 172.21.15.84:0/1812646901 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288044630 con 0x7fd2980cd0c0 2023-10-16T11:05:39.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.610+0000 7fd29fccf640 1 -- 172.21.15.84:0/1812646901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2981e81e0 con 0x7fd2980cd0c0 2023-10-16T11:05:39.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.611+0000 7fd29fccf640 1 -- 172.21.15.84:0/1812646901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29804e160 con 0x7fd2980cd0c0 2023-10-16T11:05:39.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.612+0000 7fd28e7fc640 1 -- 172.21.15.84:0/1812646901 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd28802f050 con 0x7fd2980cd0c0 2023-10-16T11:05:39.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.614+0000 7fd28e7fc640 1 --2- 172.21.15.84:0/1812646901 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd27c06af90 0x7fd27c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:39.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.614+0000 7fd28e7fc640 1 -- 172.21.15.84:0/1812646901 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fd2880ab560 con 0x7fd2980cd0c0 2023-10-16T11:05:39.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.614+0000 7fd29da6b640 1 --2- 172.21.15.84:0/1812646901 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd27c06af90 0x7fd27c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:39.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.615+0000 7fd29da6b640 1 --2- 172.21.15.84:0/1812646901 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd27c06af90 0x7fd27c06d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd294007fd0 tx=0x7fd294007960 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:39.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.619+0000 7fd28e7fc640 1 -- 172.21.15.84:0/1812646901 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd288076710 con 0x7fd2980cd0c0 2023-10-16T11:05:39.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.810+0000 7fd29fccf640 1 -- 172.21.15.84:0/1812646901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2980b4910 con 0x7fd2980cd0c0 2023-10-16T11:05:39.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.812+0000 7fd28e7fc640 1 -- 172.21.15.84:0/1812646901 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+12887 (secure 0 0 0) 0x7fd288079d30 con 0x7fd2980cd0c0 2023-10-16T11:05:39.813 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:39.813 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":167,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:05:38.860913+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:05:39.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.814+0000 7fd29fccf640 1 -- 172.21.15.84:0/1812646901 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd27c06af90 msgr2=0x7fd27c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:39.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.814+0000 7fd29fccf640 1 --2- 172.21.15.84:0/1812646901 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd27c06af90 0x7fd27c06d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd294007fd0 tx=0x7fd294007960 comp rx=0 tx=0).stop 2023-10-16T11:05:39.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.814+0000 7fd29fccf640 1 -- 172.21.15.84:0/1812646901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2980cd0c0 msgr2=0x7fd2980b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:39.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.814+0000 7fd29fccf640 1 --2- 172.21.15.84:0/1812646901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2980cd0c0 0x7fd2980b3e70 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fd288009870 tx=0x7fd2880046c0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.815+0000 7fd29fccf640 1 -- 172.21.15.84:0/1812646901 shutdown_connections 2023-10-16T11:05:39.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.815+0000 7fd29fccf640 1 --2- 172.21.15.84:0/1812646901 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd27c06af90 0x7fd27c06d450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.815+0000 7fd29fccf640 1 --2- 172.21.15.84:0/1812646901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd29815d3f0 0x7fd298143490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.815+0000 7fd29fccf640 1 --2- 172.21.15.84:0/1812646901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2980cd0c0 0x7fd2980b3e70 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.815+0000 7fd29fccf640 1 --2- 172.21.15.84:0/1812646901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2980b0b20 0x7fd2981e89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:39.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.815+0000 7fd29fccf640 1 -- 172.21.15.84:0/1812646901 >> 172.21.15.84:0/1812646901 conn(0x7fd2980cb410 msgr2=0x7fd29815cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:39.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.815+0000 7fd29fccf640 1 -- 172.21.15.84:0/1812646901 shutdown_connections 2023-10-16T11:05:39.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:39.815+0000 7fd29fccf640 1 -- 172.21.15.84:0/1812646901 wait complete. 2023-10-16T11:05:39.835 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:05:39.836 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:05:40.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.050+0000 7f086cf9e640 1 Processor -- start 2023-10-16T11:05:40.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.051+0000 7f086cf9e640 1 -- start start 2023-10-16T11:05:40.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.052+0000 7f086cf9e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f086806a4b0 0x7f08680ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.052+0000 7f086cf9e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f08680c9ca0 0x7f08680ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.052+0000 7f086cf9e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08680cb030 con 0x7f086815ad70 2023-10-16T11:05:40.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.052+0000 7f086cf9e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08680cae30 con 0x7f08680c9ca0 2023-10-16T11:05:40.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.052+0000 7f086cf9e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08680c91c0 con 0x7f086806a4b0 2023-10-16T11:05:40.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.053+0000 7f086659c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f086815ad70 0x7f0868069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42488/0 (socket says 172.21.15.84:42488) 2023-10-16T11:05:40.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.053+0000 7f086659c640 1 -- 172.21.15.84:0/3886591796 learned_addr learned my addr 172.21.15.84:0/3886591796 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:40.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.053+0000 7f0866d9d640 1 --2- 172.21.15.84:0/3886591796 >> v2:172.21.15.84:3300/0 conn(0x7f08680c9ca0 0x7f08680ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:40.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.053+0000 7f085ffff640 1 --2- 172.21.15.84:0/3886591796 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f086806a4b0 0x7f08680ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:40.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.053+0000 7f0865d9b640 1 -- 172.21.15.84:0/3886591796 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 869378153 0 0) 0x7f08680cb030 con 0x7f086815ad70 2023-10-16T11:05:40.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.053+0000 7f0865d9b640 1 -- 172.21.15.84:0/3886591796 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0854003520 con 0x7f086815ad70 2023-10-16T11:05:40.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.053+0000 7f0866d9d640 1 -- 172.21.15.84:0/3886591796 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f086806a4b0 msgr2=0x7f08680ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:40.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.053+0000 7f0866d9d640 1 --2- 172.21.15.84:0/3886591796 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f086806a4b0 0x7f08680ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.053+0000 7f0866d9d640 1 -- 172.21.15.84:0/3886591796 >> v1:172.21.15.84:6789/0 conn(0x7f086815ad70 legacy=0x7f0868069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:40.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.053+0000 7f0866d9d640 1 -- 172.21.15.84:0/3886591796 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08680c94a0 con 0x7f08680c9ca0 2023-10-16T11:05:40.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.055+0000 7f0866d9d640 1 --2- 172.21.15.84:0/3886591796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08680c9ca0 0x7f08680ca8f0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f0860009d90 tx=0x7f086002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1948874aec32d844 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:40.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.055+0000 7f0865d9b640 1 -- 172.21.15.84:0/3886591796 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086002cc50 con 0x7f08680c9ca0 2023-10-16T11:05:40.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.055+0000 7f0865d9b640 1 -- 172.21.15.84:0/3886591796 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086002cdb0 con 0x7f08680c9ca0 2023-10-16T11:05:40.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.055+0000 7f0865d9b640 1 -- 172.21.15.84:0/3886591796 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860034660 con 0x7f08680c9ca0 2023-10-16T11:05:40.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.055+0000 7f086cf9e640 1 -- 172.21.15.84:0/3886591796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08680c9ca0 msgr2=0x7f08680ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:40.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.055+0000 7f086cf9e640 1 --2- 172.21.15.84:0/3886591796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08680c9ca0 0x7f08680ca8f0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f0860009d90 tx=0x7f086002bea0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.056+0000 7f086cf9e640 1 -- 172.21.15.84:0/3886591796 shutdown_connections 2023-10-16T11:05:40.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.056+0000 7f086cf9e640 1 --2- 172.21.15.84:0/3886591796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08680c9ca0 0x7f08680ca8f0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.056+0000 7f086cf9e640 1 --2- 172.21.15.84:0/3886591796 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f086806a4b0 0x7f08680ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.056+0000 7f086cf9e640 1 -- 172.21.15.84:0/3886591796 >> 172.21.15.84:0/3886591796 conn(0x7f08680ce9b0 msgr2=0x7f08680cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:40.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.056+0000 7f086cf9e640 1 -- 172.21.15.84:0/3886591796 shutdown_connections 2023-10-16T11:05:40.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.056+0000 7f086cf9e640 1 -- 172.21.15.84:0/3886591796 wait complete. 2023-10-16T11:05:40.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.057+0000 7f086cf9e640 1 Processor -- start 2023-10-16T11:05:40.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.057+0000 7f086cf9e640 1 -- start start 2023-10-16T11:05:40.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.058+0000 7f086cf9e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f086806a4b0 0x7f08680ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.058+0000 7f086cf9e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08680c9ca0 0x7f08681efc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.058+0000 7f086cf9e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f086815ad70 0x7f08680ab640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.058+0000 7f086cf9e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680c6f30 con 0x7f08680c9ca0 2023-10-16T11:05:40.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.058+0000 7f086cf9e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680a8e00 con 0x7f086806a4b0 2023-10-16T11:05:40.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.058+0000 7f086cf9e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08680a90a0 con 0x7f086815ad70 2023-10-16T11:05:40.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.059+0000 7f085ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08680c9ca0 0x7f08681efc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:40.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.059+0000 7f085ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08680c9ca0 0x7f08681efc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59070/0 (socket says 172.21.15.84:59070) 2023-10-16T11:05:40.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.059+0000 7f085ffff640 1 -- 172.21.15.84:0/3653083649 learned_addr learned my addr 172.21.15.84:0/3653083649 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:40.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.059+0000 7f085ffff640 1 -- 172.21.15.84:0/3653083649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f086815ad70 msgr2=0x7f08680ab640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:40.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.059+0000 7f085ffff640 1 --2- 172.21.15.84:0/3653083649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f086815ad70 0x7f08680ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.059+0000 7f085ffff640 1 -- 172.21.15.84:0/3653083649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f086806a4b0 msgr2=0x7f08680ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:40.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.059+0000 7f085ffff640 1 --2- 172.21.15.84:0/3653083649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f086806a4b0 0x7f08680ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.059+0000 7f085ffff640 1 -- 172.21.15.84:0/3653083649 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0860009a70 con 0x7f08680c9ca0 2023-10-16T11:05:40.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.059+0000 7f085ffff640 1 --2- 172.21.15.84:0/3653083649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08680c9ca0 0x7f08681efc10 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f085000e9b0 tx=0x7f085000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:40.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.060+0000 7f085effd640 1 -- 172.21.15.84:0/3653083649 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850009670 con 0x7f08680c9ca0 2023-10-16T11:05:40.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.060+0000 7f085effd640 1 -- 172.21.15.84:0/3653083649 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085000c9a0 con 0x7f08680c9ca0 2023-10-16T11:05:40.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.060+0000 7f085effd640 1 -- 172.21.15.84:0/3653083649 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085000cd50 con 0x7f08680c9ca0 2023-10-16T11:05:40.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.060+0000 7f086cf9e640 1 -- 172.21.15.84:0/3653083649 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08680a9380 con 0x7f08680c9ca0 2023-10-16T11:05:40.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.060+0000 7f086cf9e640 1 -- 172.21.15.84:0/3653083649 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08680a9940 con 0x7f08680c9ca0 2023-10-16T11:05:40.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.061+0000 7f086cf9e640 1 -- 172.21.15.84:0/3653083649 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f086804e160 con 0x7f08680c9ca0 2023-10-16T11:05:40.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.062+0000 7f085effd640 1 -- 172.21.15.84:0/3653083649 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0850013470 con 0x7f08680c9ca0 2023-10-16T11:05:40.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.063+0000 7f085effd640 1 --2- 172.21.15.84:0/3653083649 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f084806af90 0x7f084806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.063+0000 7f085effd640 1 -- 172.21.15.84:0/3653083649 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f0850089720 con 0x7f08680c9ca0 2023-10-16T11:05:40.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.063+0000 7f086659c640 1 --2- 172.21.15.84:0/3653083649 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f084806af90 0x7f084806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:40.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.064+0000 7f086659c640 1 --2- 172.21.15.84:0/3653083649 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f084806af90 0x7f084806d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0858001570 tx=0x7f0858007930 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:40.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.067+0000 7f085effd640 1 -- 172.21.15.84:0/3653083649 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f08500549e0 con 0x7f08680c9ca0 2023-10-16T11:05:40.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.259+0000 7f086cf9e640 1 -- 172.21.15.84:0/3653083649 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08680b1dc0 con 0x7f08680c9ca0 2023-10-16T11:05:40.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.260+0000 7f085effd640 1 -- 172.21.15.84:0/3653083649 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v167) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0850058000 con 0x7f08680c9ca0 2023-10-16T11:05:40.261 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:05:40.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.262+0000 7f086cf9e640 1 -- 172.21.15.84:0/3653083649 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f084806af90 msgr2=0x7f084806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:40.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.262+0000 7f086cf9e640 1 --2- 172.21.15.84:0/3653083649 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f084806af90 0x7f084806d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0858001570 tx=0x7f0858007930 comp rx=0 tx=0).stop 2023-10-16T11:05:40.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.262+0000 7f086cf9e640 1 -- 172.21.15.84:0/3653083649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08680c9ca0 msgr2=0x7f08681efc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:40.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.263+0000 7f086cf9e640 1 --2- 172.21.15.84:0/3653083649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08680c9ca0 0x7f08681efc10 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f085000e9b0 tx=0x7f085000ee70 comp rx=0 tx=0).stop 2023-10-16T11:05:40.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.263+0000 7f086cf9e640 1 -- 172.21.15.84:0/3653083649 shutdown_connections 2023-10-16T11:05:40.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.263+0000 7f086cf9e640 1 --2- 172.21.15.84:0/3653083649 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f084806af90 0x7f084806d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.263+0000 7f086cf9e640 1 --2- 172.21.15.84:0/3653083649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f086815ad70 0x7f08680ab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.263+0000 7f086cf9e640 1 --2- 172.21.15.84:0/3653083649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08680c9ca0 0x7f08681efc10 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.263+0000 7f086cf9e640 1 --2- 172.21.15.84:0/3653083649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f086806a4b0 0x7f08680ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.263+0000 7f086cf9e640 1 -- 172.21.15.84:0/3653083649 >> 172.21.15.84:0/3653083649 conn(0x7f08680ce9b0 msgr2=0x7f08680ca080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:40.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.263+0000 7f086cf9e640 1 -- 172.21.15.84:0/3653083649 shutdown_connections 2023-10-16T11:05:40.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.263+0000 7f086cf9e640 1 -- 172.21.15.84:0/3653083649 wait complete. 2023-10-16T11:05:40.282 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T11:05:40.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.493+0000 7f73ec232640 1 Processor -- start 2023-10-16T11:05:40.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.493+0000 7f73ec232640 1 -- start start 2023-10-16T11:05:40.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.495+0000 7f73ec232640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73e415aaa0 0x7f73e415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.495+0000 7f73ec232640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f73e40ce390 0x7f73e4070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.495+0000 7f73ec232640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e415ed60 con 0x7f73e415ae80 2023-10-16T11:05:40.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.495+0000 7f73ec232640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e406b810 con 0x7f73e40ce390 2023-10-16T11:05:40.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.495+0000 7f73ec232640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e40707e0 con 0x7f73e415aaa0 2023-10-16T11:05:40.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.496+0000 7f73e97cd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f73e415ae80 0x7f73e40cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42490/0 (socket says 172.21.15.84:42490) 2023-10-16T11:05:40.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.496+0000 7f73e97cd640 1 -- 172.21.15.84:0/903452195 learned_addr learned my addr 172.21.15.84:0/903452195 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:40.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.496+0000 7f73e9fce640 1 --2- 172.21.15.84:0/903452195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73e415aaa0 0x7f73e415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:40.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.496+0000 7f73ea7cf640 1 --2- 172.21.15.84:0/903452195 >> v2:172.21.15.157:3300/0 conn(0x7f73e40ce390 0x7f73e4070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:40.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.496+0000 7f73e8fcc640 1 -- 172.21.15.84:0/903452195 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74698344 0 0) 0x7f73e415ed60 con 0x7f73e415ae80 2023-10-16T11:05:40.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.496+0000 7f73e8fcc640 1 -- 172.21.15.84:0/903452195 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73cc003520 con 0x7f73e415ae80 2023-10-16T11:05:40.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.496+0000 7f73e9fce640 1 -- 172.21.15.84:0/903452195 >> v2:172.21.15.157:3300/0 conn(0x7f73e40ce390 msgr2=0x7f73e4070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:40.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.496+0000 7f73e9fce640 1 --2- 172.21.15.84:0/903452195 >> v2:172.21.15.157:3300/0 conn(0x7f73e40ce390 0x7f73e4070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.496+0000 7f73e9fce640 1 -- 172.21.15.84:0/903452195 >> v1:172.21.15.84:6789/0 conn(0x7f73e415ae80 legacy=0x7f73e40cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:40.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.496+0000 7f73e9fce640 1 -- 172.21.15.84:0/903452195 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e415ef90 con 0x7f73e415aaa0 2023-10-16T11:05:40.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.498+0000 7f73e9fce640 1 --2- 172.21.15.84:0/903452195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73e415aaa0 0x7f73e415e820 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f73d40097a0 tx=0x7f73d402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1274f7e1b0a86044 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:40.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.498+0000 7f73e8fcc640 1 -- 172.21.15.84:0/903452195 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d402d040 con 0x7f73e415aaa0 2023-10-16T11:05:40.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.498+0000 7f73e8fcc640 1 -- 172.21.15.84:0/903452195 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d4031810 con 0x7f73e415aaa0 2023-10-16T11:05:40.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.498+0000 7f73e8fcc640 1 -- 172.21.15.84:0/903452195 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d4031ba0 con 0x7f73e415aaa0 2023-10-16T11:05:40.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.499+0000 7f73ec232640 1 -- 172.21.15.84:0/903452195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73e415aaa0 msgr2=0x7f73e415e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:40.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.499+0000 7f73ec232640 1 --2- 172.21.15.84:0/903452195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73e415aaa0 0x7f73e415e820 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f73d40097a0 tx=0x7f73d402b8b0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.499+0000 7f73ec232640 1 -- 172.21.15.84:0/903452195 shutdown_connections 2023-10-16T11:05:40.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.499+0000 7f73ec232640 1 --2- 172.21.15.84:0/903452195 >> v2:172.21.15.157:3300/0 conn(0x7f73e40ce390 0x7f73e4070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.499+0000 7f73ec232640 1 --2- 172.21.15.84:0/903452195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73e415aaa0 0x7f73e415e820 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.499+0000 7f73ec232640 1 -- 172.21.15.84:0/903452195 >> 172.21.15.84:0/903452195 conn(0x7f73e40cb410 msgr2=0x7f73e40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:40.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.499+0000 7f73ec232640 1 -- 172.21.15.84:0/903452195 shutdown_connections 2023-10-16T11:05:40.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.499+0000 7f73ec232640 1 -- 172.21.15.84:0/903452195 wait complete. 2023-10-16T11:05:40.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.501+0000 7f73ec232640 1 Processor -- start 2023-10-16T11:05:40.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.501+0000 7f73ec232640 1 -- start start 2023-10-16T11:05:40.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.502+0000 7f73ec232640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73e40ce390 0x7f73e41590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.502+0000 7f73ec232640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73e415aaa0 0x7f73e415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.502+0000 7f73ec232640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73e415ae80 0x7f73e415c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.502+0000 7f73ec232640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e40c2e60 con 0x7f73e415aaa0 2023-10-16T11:05:40.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.502+0000 7f73ec232640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e40c86f0 con 0x7f73e40ce390 2023-10-16T11:05:40.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.502+0000 7f73ec232640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e40c3100 con 0x7f73e415ae80 2023-10-16T11:05:40.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.502+0000 7f73e97cd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73e415aaa0 0x7f73e415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:40.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.502+0000 7f73e97cd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73e415aaa0 0x7f73e415be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59078/0 (socket says 172.21.15.84:59078) 2023-10-16T11:05:40.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.502+0000 7f73e97cd640 1 -- 172.21.15.84:0/2914463996 learned_addr learned my addr 172.21.15.84:0/2914463996 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:40.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.503+0000 7f73e9fce640 1 --2- 172.21.15.84:0/2914463996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73e40ce390 0x7f73e41590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:40.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.503+0000 7f73e97cd640 1 -- 172.21.15.84:0/2914463996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73e415ae80 msgr2=0x7f73e415c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:40.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.503+0000 7f73e97cd640 1 --2- 172.21.15.84:0/2914463996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73e415ae80 0x7f73e415c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.503+0000 7f73e97cd640 1 -- 172.21.15.84:0/2914463996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73e40ce390 msgr2=0x7f73e41590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:40.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.503+0000 7f73e97cd640 1 --2- 172.21.15.84:0/2914463996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73e40ce390 0x7f73e41590d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.503+0000 7f73e97cd640 1 -- 172.21.15.84:0/2914463996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d4009400 con 0x7f73e415aaa0 2023-10-16T11:05:40.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.503+0000 7f73e97cd640 1 --2- 172.21.15.84:0/2914463996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73e415aaa0 0x7f73e415be20 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f73e000c480 tx=0x7f73e000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:40.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.503+0000 7f73da7fc640 1 -- 172.21.15.84:0/2914463996 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e0005ce0 con 0x7f73e415aaa0 2023-10-16T11:05:40.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.503+0000 7f73da7fc640 1 -- 172.21.15.84:0/2914463996 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e0005e40 con 0x7f73e415aaa0 2023-10-16T11:05:40.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.503+0000 7f73da7fc640 1 -- 172.21.15.84:0/2914463996 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e00072b0 con 0x7f73e415aaa0 2023-10-16T11:05:40.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.503+0000 7f73ec232640 1 -- 172.21.15.84:0/2914463996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e40c33e0 con 0x7f73e415aaa0 2023-10-16T11:05:40.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.504+0000 7f73ec232640 1 -- 172.21.15.84:0/2914463996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e40c8260 con 0x7f73e415aaa0 2023-10-16T11:05:40.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.505+0000 7f73ec232640 1 -- 172.21.15.84:0/2914463996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e40c84b0 con 0x7f73e415aaa0 2023-10-16T11:05:40.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.506+0000 7f73da7fc640 1 -- 172.21.15.84:0/2914463996 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f73e0007410 con 0x7f73e415aaa0 2023-10-16T11:05:40.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.506+0000 7f73da7fc640 1 --2- 172.21.15.84:0/2914463996 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f73d006afc0 0x7f73d006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.507+0000 7f73da7fc640 1 -- 172.21.15.84:0/2914463996 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f73e00889e0 con 0x7f73e415aaa0 2023-10-16T11:05:40.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.507+0000 7f73e9fce640 1 --2- 172.21.15.84:0/2914463996 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f73d006afc0 0x7f73d006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:40.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.507+0000 7f73e9fce640 1 --2- 172.21.15.84:0/2914463996 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f73d006afc0 0x7f73d006d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f73d4036000 tx=0x7f73d403c040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:40.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.511+0000 7f73da7fc640 1 -- 172.21.15.84:0/2914463996 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f73e40c84b0 con 0x7f73e415aaa0 2023-10-16T11:05:40.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.704+0000 7f73ec232640 1 -- 172.21.15.84:0/2914463996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f73e40c3900 con 0x7f73e415aaa0 2023-10-16T11:05:40.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.705+0000 7f73da7fc640 1 -- 172.21.15.84:0/2914463996 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v167) v1 ==== 84+0+65 (secure 0 0 0) 0x7f73e40c3900 con 0x7f73e415aaa0 2023-10-16T11:05:40.706 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:40.706 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T11:05:40.706 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:40.706 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-16T11:05:40.707 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-16T11:05:40.707 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:40.707 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T11:05:40.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.708+0000 7f73ec232640 1 -- 172.21.15.84:0/2914463996 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f73d006afc0 msgr2=0x7f73d006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:40.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.708+0000 7f73ec232640 1 --2- 172.21.15.84:0/2914463996 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f73d006afc0 0x7f73d006d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f73d4036000 tx=0x7f73d403c040 comp rx=0 tx=0).stop 2023-10-16T11:05:40.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.708+0000 7f73ec232640 1 -- 172.21.15.84:0/2914463996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73e415aaa0 msgr2=0x7f73e415be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:40.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.708+0000 7f73ec232640 1 --2- 172.21.15.84:0/2914463996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73e415aaa0 0x7f73e415be20 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f73e000c480 tx=0x7f73e000c940 comp rx=0 tx=0).stop 2023-10-16T11:05:40.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.708+0000 7f73ec232640 1 -- 172.21.15.84:0/2914463996 shutdown_connections 2023-10-16T11:05:40.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.708+0000 7f73ec232640 1 --2- 172.21.15.84:0/2914463996 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f73d006afc0 0x7f73d006d480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.708+0000 7f73ec232640 1 --2- 172.21.15.84:0/2914463996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73e415ae80 0x7f73e415c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.708+0000 7f73ec232640 1 --2- 172.21.15.84:0/2914463996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73e415aaa0 0x7f73e415be20 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.708+0000 7f73ec232640 1 --2- 172.21.15.84:0/2914463996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73e40ce390 0x7f73e41590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.708+0000 7f73ec232640 1 -- 172.21.15.84:0/2914463996 >> 172.21.15.84:0/2914463996 conn(0x7f73e40cb410 msgr2=0x7f73e406fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:40.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.708+0000 7f73ec232640 1 -- 172.21.15.84:0/2914463996 shutdown_connections 2023-10-16T11:05:40.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.708+0000 7f73ec232640 1 -- 172.21.15.84:0/2914463996 wait complete. 2023-10-16T11:05:40.728 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T11:05:40.728 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T11:05:40.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.949+0000 7f2b1a446640 1 Processor -- start 2023-10-16T11:05:40.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.949+0000 7f2b1a446640 1 -- start start 2023-10-16T11:05:40.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.950+0000 7f2b1a446640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b141576b0 0x7f2b140ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.950+0000 7f2b1a446640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2b1415e2c0 0x7f2b1406b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.950+0000 7f2b1a446640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b140ae2a0 con 0x7f2b1406bad0 2023-10-16T11:05:40.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.950+0000 7f2b1a446640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b1406bf20 con 0x7f2b1415e2c0 2023-10-16T11:05:40.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.950+0000 7f2b1a446640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b14157a90 con 0x7f2b141576b0 2023-10-16T11:05:40.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.950+0000 7f2b189e3640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2b1406bad0 0x7f2b140ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42496/0 (socket says 172.21.15.84:42496) 2023-10-16T11:05:40.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.950+0000 7f2b189e3640 1 -- 172.21.15.84:0/3785885283 learned_addr learned my addr 172.21.15.84:0/3785885283 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:40.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.951+0000 7f2b12ffd640 1 -- 172.21.15.84:0/3785885283 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462898356 0 0) 0x7f2b0c001110 con 0x7f2b1406bad0 2023-10-16T11:05:40.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.951+0000 7f2b12ffd640 1 -- 172.21.15.84:0/3785885283 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2afc003520 con 0x7f2b1406bad0 2023-10-16T11:05:40.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.951+0000 7f2b137fe640 1 --2- 172.21.15.84:0/3785885283 >> v2:172.21.15.157:3300/0 conn(0x7f2b1415e2c0 0x7f2b1406b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:40.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.951+0000 7f2b13fff640 1 --2- 172.21.15.84:0/3785885283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b141576b0 0x7f2b140ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:40.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.951+0000 7f2b12ffd640 1 -- 172.21.15.84:0/3785885283 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2713224857 0 0) 0x7f2afc003520 con 0x7f2b1406bad0 2023-10-16T11:05:40.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.951+0000 7f2b12ffd640 1 -- 172.21.15.84:0/3785885283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b141576b0 msgr2=0x7f2b140ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:40.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.951+0000 7f2b12ffd640 1 --2- 172.21.15.84:0/3785885283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b141576b0 0x7f2b140ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.951+0000 7f2b12ffd640 1 -- 172.21.15.84:0/3785885283 >> v2:172.21.15.157:3300/0 conn(0x7f2b1415e2c0 msgr2=0x7f2b1406b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:40.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.951+0000 7f2b12ffd640 1 --2- 172.21.15.84:0/3785885283 >> v2:172.21.15.157:3300/0 conn(0x7f2b1415e2c0 0x7f2b1406b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.951+0000 7f2b12ffd640 1 -- 172.21.15.84:0/3785885283 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b140ade10 con 0x7f2b1406bad0 2023-10-16T11:05:40.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.951+0000 7f2b12ffd640 1 -- 172.21.15.84:0/3785885283 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f2b0c001800 con 0x7f2b1406bad0 2023-10-16T11:05:40.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.951+0000 7f2b12ffd640 1 -- 172.21.15.84:0/3785885283 >> v1:172.21.15.84:6789/0 conn(0x7f2b1406bad0 legacy=0x7f2b140ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:40.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.951+0000 7f2b12ffd640 1 --2- 172.21.15.84:0/3785885283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b141576b0 0x7f2afc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.951+0000 7f2b12ffd640 1 --2- 172.21.15.84:0/3785885283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2afc004010 0x7f2afc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.951+0000 7f2b12ffd640 1 --2- 172.21.15.84:0/3785885283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2afc004d00 0x7f2afc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.951+0000 7f2b12ffd640 1 -- 172.21.15.84:0/3785885283 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1406bf20 con 0x7f2afc004d00 2023-10-16T11:05:40.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.951+0000 7f2b12ffd640 1 -- 172.21.15.84:0/3785885283 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b14157a90 con 0x7f2afc004010 2023-10-16T11:05:40.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.951+0000 7f2b12ffd640 1 -- 172.21.15.84:0/3785885283 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc00bb10 con 0x7f2b141576b0 2023-10-16T11:05:40.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.952+0000 7f2b189e3640 1 --2- 172.21.15.84:0/3785885283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2afc004d00 0x7f2afc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:40.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.952+0000 7f2b137fe640 1 --2- 172.21.15.84:0/3785885283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2afc004010 0x7f2afc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:40.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.952+0000 7f2b13fff640 1 --2- 172.21.15.84:0/3785885283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b141576b0 0x7f2afc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:40.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.952+0000 7f2b189e3640 1 -- 172.21.15.84:0/3785885283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b141576b0 msgr2=0x7f2afc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:40.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.952+0000 7f2b189e3640 1 --2- 172.21.15.84:0/3785885283 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b141576b0 0x7f2afc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.952+0000 7f2b189e3640 1 -- 172.21.15.84:0/3785885283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2afc004010 msgr2=0x7f2afc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:40.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.952+0000 7f2b189e3640 1 --2- 172.21.15.84:0/3785885283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2afc004010 0x7f2afc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.952+0000 7f2b189e3640 1 -- 172.21.15.84:0/3785885283 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc003520 con 0x7f2afc004d00 2023-10-16T11:05:40.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.953+0000 7f2b189e3640 1 --2- 172.21.15.84:0/3785885283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2afc004d00 0x7f2afc005180 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c001eb0 tx=0x7f2afc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b7df1f3e55b0aa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:40.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.954+0000 7f2b12ffd640 1 -- 172.21.15.84:0/3785885283 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c005ad0 con 0x7f2afc004d00 2023-10-16T11:05:40.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.954+0000 7f2b12ffd640 1 -- 172.21.15.84:0/3785885283 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0c005c50 con 0x7f2afc004d00 2023-10-16T11:05:40.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.954+0000 7f2b12ffd640 1 -- 172.21.15.84:0/3785885283 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c02c8b0 con 0x7f2afc004d00 2023-10-16T11:05:40.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.954+0000 7f2b1a446640 1 -- 172.21.15.84:0/3785885283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2afc004d00 msgr2=0x7f2afc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:40.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.954+0000 7f2b1a446640 1 --2- 172.21.15.84:0/3785885283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2afc004d00 0x7f2afc005180 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c001eb0 tx=0x7f2afc0057f0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.954+0000 7f2b189e3640 1 -- 172.21.15.84:0/3785885283 reap_dead start 2023-10-16T11:05:40.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.954+0000 7f2b1a446640 1 -- 172.21.15.84:0/3785885283 shutdown_connections 2023-10-16T11:05:40.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.954+0000 7f2b1a446640 1 -- 172.21.15.84:0/3785885283 >> 172.21.15.84:0/3785885283 conn(0x7f2b140b0860 msgr2=0x7f2b140b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:40.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.954+0000 7f2b1a446640 1 -- 172.21.15.84:0/3785885283 shutdown_connections 2023-10-16T11:05:40.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.954+0000 7f2b1a446640 1 -- 172.21.15.84:0/3785885283 wait complete. 2023-10-16T11:05:40.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.956+0000 7f2b1a446640 1 Processor -- start 2023-10-16T11:05:40.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.956+0000 7f2b1a446640 1 -- start start 2023-10-16T11:05:40.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.957+0000 7f2b1a446640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2afc004d00 0x7f2b141f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.957+0000 7f2b1a446640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b14156210 0x7f2b140b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.957+0000 7f2b1a446640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b140b38a0 0x7f2b140af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.957+0000 7f2b1a446640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1415e740 con 0x7f2b14156210 2023-10-16T11:05:40.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.957+0000 7f2b1a446640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b14157ca0 con 0x7f2b140b38a0 2023-10-16T11:05:40.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.957+0000 7f2b1a446640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b14157f10 con 0x7f2afc004d00 2023-10-16T11:05:40.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.957+0000 7f2b137fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b14156210 0x7f2b140b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:40.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.958+0000 7f2b137fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b14156210 0x7f2b140b3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59108/0 (socket says 172.21.15.84:59108) 2023-10-16T11:05:40.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.958+0000 7f2b137fe640 1 -- 172.21.15.84:0/1118876818 learned_addr learned my addr 172.21.15.84:0/1118876818 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:40.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.958+0000 7f2b189e3640 1 --2- 172.21.15.84:0/1118876818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b140b38a0 0x7f2b140af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:40.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.958+0000 7f2b13fff640 1 --2- 172.21.15.84:0/1118876818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2afc004d00 0x7f2b141f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:40.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.958+0000 7f2b137fe640 1 -- 172.21.15.84:0/1118876818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2afc004d00 msgr2=0x7f2b141f3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:40.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.958+0000 7f2b137fe640 1 --2- 172.21.15.84:0/1118876818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2afc004d00 0x7f2b141f3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.958+0000 7f2b137fe640 1 -- 172.21.15.84:0/1118876818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b140b38a0 msgr2=0x7f2b140af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:40.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.958+0000 7f2b137fe640 1 --2- 172.21.15.84:0/1118876818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b140b38a0 0x7f2b140af9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:40.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.958+0000 7f2b137fe640 1 -- 172.21.15.84:0/1118876818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b141581f0 con 0x7f2b14156210 2023-10-16T11:05:40.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.958+0000 7f2b189e3640 1 --2- 172.21.15.84:0/1118876818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b140b38a0 0x7f2b140af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:40.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.958+0000 7f2b137fe640 1 --2- 172.21.15.84:0/1118876818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b14156210 0x7f2b140b3360 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f2b0800f600 tx=0x7f2b0800fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:40.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.959+0000 7f2b10ff9640 1 -- 172.21.15.84:0/1118876818 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b08014870 con 0x7f2b14156210 2023-10-16T11:05:40.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.959+0000 7f2b10ff9640 1 -- 172.21.15.84:0/1118876818 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b080149d0 con 0x7f2b14156210 2023-10-16T11:05:40.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.959+0000 7f2b10ff9640 1 -- 172.21.15.84:0/1118876818 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b08014d40 con 0x7f2b14156210 2023-10-16T11:05:40.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.959+0000 7f2b1a446640 1 -- 172.21.15.84:0/1118876818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b141584d0 con 0x7f2b14156210 2023-10-16T11:05:40.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.959+0000 7f2b1a446640 1 -- 172.21.15.84:0/1118876818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b14156690 con 0x7f2b14156210 2023-10-16T11:05:40.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.960+0000 7f2b1a446640 1 -- 172.21.15.84:0/1118876818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b1404e160 con 0x7f2b14156210 2023-10-16T11:05:40.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.961+0000 7f2b10ff9640 1 -- 172.21.15.84:0/1118876818 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2b0800d490 con 0x7f2b14156210 2023-10-16T11:05:40.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.962+0000 7f2b10ff9640 1 --2- 172.21.15.84:0/1118876818 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f2b0006afc0 0x7f2b0006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:40.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.962+0000 7f2b10ff9640 1 -- 172.21.15.84:0/1118876818 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f2b08089f50 con 0x7f2b14156210 2023-10-16T11:05:40.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.962+0000 7f2b13fff640 1 --2- 172.21.15.84:0/1118876818 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f2b0006afc0 0x7f2b0006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:40.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.963+0000 7f2b13fff640 1 --2- 172.21.15.84:0/1118876818 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f2b0006afc0 0x7f2b0006d480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2b141f5e20 tx=0x7f2b0400b040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:40.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:40.966+0000 7f2b10ff9640 1 -- 172.21.15.84:0/1118876818 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2b08055190 con 0x7f2b14156210 2023-10-16T11:05:41.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:41.158+0000 7f2b1a446640 1 -- 172.21.15.84:0/1118876818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2b14074140 con 0x7f2b14156210 2023-10-16T11:05:41.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:41.874+0000 7f2b10ff9640 1 -- 172.21.15.84:0/1118876818 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v168) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2b080587b0 con 0x7f2b14156210 2023-10-16T11:05:41.876 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-16T11:05:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:41.878+0000 7f2b1a446640 1 -- 172.21.15.84:0/1118876818 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f2b0006afc0 msgr2=0x7f2b0006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:41.878+0000 7f2b1a446640 1 --2- 172.21.15.84:0/1118876818 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f2b0006afc0 0x7f2b0006d480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2b141f5e20 tx=0x7f2b0400b040 comp rx=0 tx=0).stop 2023-10-16T11:05:41.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:41.878+0000 7f2b1a446640 1 -- 172.21.15.84:0/1118876818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b14156210 msgr2=0x7f2b140b3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:41.878+0000 7f2b1a446640 1 --2- 172.21.15.84:0/1118876818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b14156210 0x7f2b140b3360 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f2b0800f600 tx=0x7f2b0800fac0 comp rx=0 tx=0).stop 2023-10-16T11:05:41.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:41.879+0000 7f2b1a446640 1 -- 172.21.15.84:0/1118876818 shutdown_connections 2023-10-16T11:05:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:41.879+0000 7f2b1a446640 1 --2- 172.21.15.84:0/1118876818 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f2b0006afc0 0x7f2b0006d480 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:41.879+0000 7f2b1a446640 1 --2- 172.21.15.84:0/1118876818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b140b38a0 0x7f2b140af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:41.879+0000 7f2b1a446640 1 --2- 172.21.15.84:0/1118876818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b14156210 0x7f2b140b3360 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:41.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:41.879+0000 7f2b1a446640 1 --2- 172.21.15.84:0/1118876818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2afc004d00 0x7f2b141f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:41.879+0000 7f2b1a446640 1 -- 172.21.15.84:0/1118876818 >> 172.21.15.84:0/1118876818 conn(0x7f2b140b0860 msgr2=0x7f2b140cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:41.880+0000 7f2b1a446640 1 -- 172.21.15.84:0/1118876818 shutdown_connections 2023-10-16T11:05:41.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:41.880+0000 7f2b1a446640 1 -- 172.21.15.84:0/1118876818 wait complete. 2023-10-16T11:05:41.906 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T11:05:42.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.114+0000 7f38a007c640 1 Processor -- start 2023-10-16T11:05:42.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.115+0000 7f38a007c640 1 -- start start 2023-10-16T11:05:42.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.116+0000 7f38a007c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f389815aaa0 0x7f389815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:42.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.116+0000 7f38a007c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f389815ae80 0x7f38980cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:42.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.116+0000 7f38a007c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38980707e0 con 0x7f38980ccaa0 2023-10-16T11:05:42.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.116+0000 7f38a007c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f389806b7a0 con 0x7f389815ae80 2023-10-16T11:05:42.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.116+0000 7f38a007c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389815ef20 con 0x7f389815aaa0 2023-10-16T11:05:42.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.117+0000 7f389d617640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f389815ae80 0x7f38980cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:42.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.117+0000 7f389d617640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f389815ae80 0x7f38980cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59112/0 (socket says 172.21.15.84:59112) 2023-10-16T11:05:42.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.117+0000 7f389d617640 1 -- 172.21.15.84:0/1553427153 learned_addr learned my addr 172.21.15.84:0/1553427153 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:42.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.117+0000 7f389de18640 1 --2- 172.21.15.84:0/1553427153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f389815aaa0 0x7f389815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:42.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.117+0000 7f389d617640 1 -- 172.21.15.84:0/1553427153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f389815aaa0 msgr2=0x7f389815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:42.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.117+0000 7f389d617640 1 --2- 172.21.15.84:0/1553427153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f389815aaa0 0x7f389815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:42.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.117+0000 7f389d617640 1 -- 172.21.15.84:0/1553427153 >> v1:172.21.15.157:6789/0 conn(0x7f38980ccaa0 legacy=0x7f3898070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:42.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.117+0000 7f389d617640 1 -- 172.21.15.84:0/1553427153 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389815f200 con 0x7f389815ae80 2023-10-16T11:05:42.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.118+0000 7f389d617640 1 --2- 172.21.15.84:0/1553427153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f389815ae80 0x7f38980cc560 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f388c009870 tx=0x7f388c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e6a53629371911 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:42.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.119+0000 7f389ce16640 1 -- 172.21.15.84:0/1553427153 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c0317c0 con 0x7f389815ae80 2023-10-16T11:05:42.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.119+0000 7f389ce16640 1 -- 172.21.15.84:0/1553427153 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c031920 con 0x7f389815ae80 2023-10-16T11:05:42.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.119+0000 7f389ce16640 1 -- 172.21.15.84:0/1553427153 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c031cb0 con 0x7f389815ae80 2023-10-16T11:05:42.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.119+0000 7f38a007c640 1 -- 172.21.15.84:0/1553427153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f389815ae80 msgr2=0x7f38980cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:42.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.119+0000 7f38a007c640 1 --2- 172.21.15.84:0/1553427153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f389815ae80 0x7f38980cc560 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f388c009870 tx=0x7f388c02b860 comp rx=0 tx=0).stop 2023-10-16T11:05:42.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.120+0000 7f38a007c640 1 -- 172.21.15.84:0/1553427153 shutdown_connections 2023-10-16T11:05:42.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.120+0000 7f38a007c640 1 --2- 172.21.15.84:0/1553427153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f389815ae80 0x7f38980cc560 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:42.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.120+0000 7f38a007c640 1 --2- 172.21.15.84:0/1553427153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f389815aaa0 0x7f389815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:42.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.120+0000 7f38a007c640 1 -- 172.21.15.84:0/1553427153 >> 172.21.15.84:0/1553427153 conn(0x7f38980cb410 msgr2=0x7f38980cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:42.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.120+0000 7f38a007c640 1 -- 172.21.15.84:0/1553427153 shutdown_connections 2023-10-16T11:05:42.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.120+0000 7f38a007c640 1 -- 172.21.15.84:0/1553427153 wait complete. 2023-10-16T11:05:42.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.121+0000 7f38a007c640 1 Processor -- start 2023-10-16T11:05:42.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.122+0000 7f38a007c640 1 -- start start 2023-10-16T11:05:42.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.123+0000 7f38a007c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38980ccaa0 0x7f38980c3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:42.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.123+0000 7f38a007c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f389815aaa0 0x7f38981504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:42.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.123+0000 7f38a007c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f389815ae80 0x7f389814e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:42.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.123+0000 7f38a007c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38980c2a50 con 0x7f389815ae80 2023-10-16T11:05:42.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.123+0000 7f38a007c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38980c2d30 con 0x7f389815aaa0 2023-10-16T11:05:42.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.123+0000 7f38a007c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38980c3010 con 0x7f38980ccaa0 2023-10-16T11:05:42.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.123+0000 7f389e619640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f389815ae80 0x7f389814e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:42.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.123+0000 7f389e619640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f389815ae80 0x7f389814e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59114/0 (socket says 172.21.15.84:59114) 2023-10-16T11:05:42.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.123+0000 7f389e619640 1 -- 172.21.15.84:0/205064204 learned_addr learned my addr 172.21.15.84:0/205064204 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:42.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.123+0000 7f389de18640 1 --2- 172.21.15.84:0/205064204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38980ccaa0 0x7f38980c3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:42.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.123+0000 7f389d617640 1 --2- 172.21.15.84:0/205064204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f389815aaa0 0x7f38981504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:42.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.123+0000 7f389e619640 1 -- 172.21.15.84:0/205064204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38980ccaa0 msgr2=0x7f38980c3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:42.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.123+0000 7f389e619640 1 --2- 172.21.15.84:0/205064204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38980ccaa0 0x7f38980c3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:42.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.123+0000 7f389e619640 1 -- 172.21.15.84:0/205064204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f389815aaa0 msgr2=0x7f38981504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:42.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.123+0000 7f389e619640 1 --2- 172.21.15.84:0/205064204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f389815aaa0 0x7f38981504b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:42.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.123+0000 7f389e619640 1 -- 172.21.15.84:0/205064204 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c0094d0 con 0x7f389815ae80 2023-10-16T11:05:42.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.123+0000 7f389de18640 1 --2- 172.21.15.84:0/205064204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38980ccaa0 0x7f38980c3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:42.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.123+0000 7f389d617640 1 --2- 172.21.15.84:0/205064204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f389815aaa0 0x7f38981504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:42.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.124+0000 7f389e619640 1 --2- 172.21.15.84:0/205064204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f389815ae80 0x7f389814e040 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f389000c780 tx=0x7f389000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:42.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.124+0000 7f388a7fc640 1 -- 172.21.15.84:0/205064204 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3890004b60 con 0x7f389815ae80 2023-10-16T11:05:42.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.124+0000 7f388a7fc640 1 -- 172.21.15.84:0/205064204 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389000ce90 con 0x7f389815ae80 2023-10-16T11:05:42.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.124+0000 7f388a7fc640 1 -- 172.21.15.84:0/205064204 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389000a850 con 0x7f389815ae80 2023-10-16T11:05:42.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.124+0000 7f38a007c640 1 -- 172.21.15.84:0/205064204 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38980c32f0 con 0x7f389815ae80 2023-10-16T11:05:42.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.124+0000 7f38a007c640 1 -- 172.21.15.84:0/205064204 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38981509f0 con 0x7f389815ae80 2023-10-16T11:05:42.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.125+0000 7f38a007c640 1 -- 172.21.15.84:0/205064204 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38980b2aa0 con 0x7f389815ae80 2023-10-16T11:05:42.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.126+0000 7f388a7fc640 1 -- 172.21.15.84:0/205064204 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f389000a9b0 con 0x7f389815ae80 2023-10-16T11:05:42.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.127+0000 7f388a7fc640 1 --2- 172.21.15.84:0/205064204 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f387c06af90 0x7f387c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:42.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.128+0000 7f388a7fc640 1 -- 172.21.15.84:0/205064204 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5255+0+0 (secure 0 0 0) 0x7f38900897a0 con 0x7f389815ae80 2023-10-16T11:05:42.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.128+0000 7f389de18640 1 --2- 172.21.15.84:0/205064204 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f387c06af90 0x7f387c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:42.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.128+0000 7f389de18640 1 --2- 172.21.15.84:0/205064204 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f387c06af90 0x7f387c06d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3894006fd0 tx=0x7f3894008040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:42.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.132+0000 7f388a7fc640 1 -- 172.21.15.84:0/205064204 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f38900548f0 con 0x7f389815ae80 2023-10-16T11:05:42.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.328+0000 7f38a007c640 1 -- 172.21.15.84:0/205064204 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f389804e160 con 0x7f389815ae80 2023-10-16T11:05:42.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.881+0000 7f388a7fc640 1 -- 172.21.15.84:0/205064204 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v169) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3890057f10 con 0x7f389815ae80 2023-10-16T11:05:42.882 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-16T11:05:42.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.886+0000 7f38a007c640 1 -- 172.21.15.84:0/205064204 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f387c06af90 msgr2=0x7f387c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:42.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.886+0000 7f38a007c640 1 --2- 172.21.15.84:0/205064204 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f387c06af90 0x7f387c06d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3894006fd0 tx=0x7f3894008040 comp rx=0 tx=0).stop 2023-10-16T11:05:42.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.886+0000 7f38a007c640 1 -- 172.21.15.84:0/205064204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f389815ae80 msgr2=0x7f389814e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:42.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.886+0000 7f38a007c640 1 --2- 172.21.15.84:0/205064204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f389815ae80 0x7f389814e040 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f389000c780 tx=0x7f389000cc40 comp rx=0 tx=0).stop 2023-10-16T11:05:42.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.887+0000 7f38a007c640 1 -- 172.21.15.84:0/205064204 shutdown_connections 2023-10-16T11:05:42.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.887+0000 7f38a007c640 1 --2- 172.21.15.84:0/205064204 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f387c06af90 0x7f387c06d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:42.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.887+0000 7f38a007c640 1 --2- 172.21.15.84:0/205064204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f389815ae80 0x7f389814e040 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:42.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.887+0000 7f38a007c640 1 --2- 172.21.15.84:0/205064204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f389815aaa0 0x7f38981504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:42.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.887+0000 7f38a007c640 1 --2- 172.21.15.84:0/205064204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38980ccaa0 0x7f38980c3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:42.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.887+0000 7f38a007c640 1 -- 172.21.15.84:0/205064204 >> 172.21.15.84:0/205064204 conn(0x7f38980cb410 msgr2=0x7f389806bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:42.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.888+0000 7f38a007c640 1 -- 172.21.15.84:0/205064204 shutdown_connections 2023-10-16T11:05:42.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:42.888+0000 7f38a007c640 1 -- 172.21.15.84:0/205064204 wait complete. 2023-10-16T11:05:42.919 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T11:05:43.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.104+0000 7fd5dbe09640 1 Processor -- start 2023-10-16T11:05:43.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.104+0000 7fd5dbe09640 1 -- start start 2023-10-16T11:05:43.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.105+0000 7fd5dbe09640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd5d40cca00 0x7fd5d40b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:43.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.106+0000 7fd5dbe09640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d41544c0 con 0x7fd5d40b0df0 2023-10-16T11:05:43.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.106+0000 7fd5dbe09640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d40ccf00 con 0x7fd5d41540e0 2023-10-16T11:05:43.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.106+0000 7fd5dbe09640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5d40b12a0 con 0x7fd5d40cca00 2023-10-16T11:05:43.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.106+0000 7fd5da3a6640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd5d40b0df0 0x7fd5d415d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42498/0 (socket says 172.21.15.84:42498) 2023-10-16T11:05:43.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.106+0000 7fd5da3a6640 1 -- 172.21.15.84:0/1429463316 learned_addr learned my addr 172.21.15.84:0/1429463316 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:43.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.106+0000 7fd5d93a4640 1 --2- 172.21.15.84:0/1429463316 >> v2:172.21.15.157:3300/0 conn(0x7fd5d40cca00 0x7fd5d40b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:43.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.106+0000 7fd5d8ba3640 1 -- 172.21.15.84:0/1429463316 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555165292 0 0) 0x7fd5d41544c0 con 0x7fd5d40b0df0 2023-10-16T11:05:43.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.106+0000 7fd5d8ba3640 1 -- 172.21.15.84:0/1429463316 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5c0003520 con 0x7fd5d40b0df0 2023-10-16T11:05:43.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.107+0000 7fd5d8ba3640 1 -- 172.21.15.84:0/1429463316 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1832269684 0 0) 0x7fd5c0003520 con 0x7fd5d40b0df0 2023-10-16T11:05:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.107+0000 7fd5d8ba3640 1 -- 172.21.15.84:0/1429463316 >> v2:172.21.15.157:3300/0 conn(0x7fd5d40cca00 msgr2=0x7fd5d40b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.107+0000 7fd5d8ba3640 1 --2- 172.21.15.84:0/1429463316 >> v2:172.21.15.157:3300/0 conn(0x7fd5d40cca00 0x7fd5d40b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.107+0000 7fd5d8ba3640 1 -- 172.21.15.84:0/1429463316 >> v1:172.21.15.157:6789/0 conn(0x7fd5d41540e0 legacy=0x7fd5d40adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.107+0000 7fd5d8ba3640 1 -- 172.21.15.84:0/1429463316 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d40cd1e0 con 0x7fd5d40b0df0 2023-10-16T11:05:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.107+0000 7fd5d8ba3640 1 -- 172.21.15.84:0/1429463316 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd5cc0019f0 con 0x7fd5d40b0df0 2023-10-16T11:05:43.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.107+0000 7fd5d8ba3640 1 -- 172.21.15.84:0/1429463316 >> v1:172.21.15.84:6789/0 conn(0x7fd5d40b0df0 legacy=0x7fd5d415d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:43.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.107+0000 7fd5d8ba3640 1 --2- 172.21.15.84:0/1429463316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5c0003970 0x7fd5c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:43.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.107+0000 7fd5d8ba3640 1 --2- 172.21.15.84:0/1429463316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5c00047c0 0x7fd5c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:43.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.107+0000 7fd5d8ba3640 1 --2- 172.21.15.84:0/1429463316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5c0005180 0x7fd5c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:43.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.107+0000 7fd5d8ba3640 1 -- 172.21.15.84:0/1429463316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40b12a0 con 0x7fd5c00047c0 2023-10-16T11:05:43.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.107+0000 7fd5d8ba3640 1 -- 172.21.15.84:0/1429463316 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c000c020 con 0x7fd5c0005180 2023-10-16T11:05:43.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.107+0000 7fd5d8ba3640 1 -- 172.21.15.84:0/1429463316 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c000c1e0 con 0x7fd5c0003970 2023-10-16T11:05:43.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.107+0000 7fd5d93a4640 1 --2- 172.21.15.84:0/1429463316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5c00047c0 0x7fd5c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:43.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.107+0000 7fd5da3a6640 1 --2- 172.21.15.84:0/1429463316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5c0005180 0x7fd5c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:43.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.108+0000 7fd5d93a4640 1 -- 172.21.15.84:0/1429463316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5c0003970 msgr2=0x7fd5c0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:43.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.108+0000 7fd5d93a4640 1 --2- 172.21.15.84:0/1429463316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5c0003970 0x7fd5c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:43.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.108+0000 7fd5d93a4640 1 -- 172.21.15.84:0/1429463316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5c0005180 msgr2=0x7fd5c000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:43.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.108+0000 7fd5d93a4640 1 --2- 172.21.15.84:0/1429463316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5c0005180 0x7fd5c000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:43.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.108+0000 7fd5d93a4640 1 -- 172.21.15.84:0/1429463316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c0003520 con 0x7fd5c00047c0 2023-10-16T11:05:43.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.108+0000 7fd5da3a6640 1 --2- 172.21.15.84:0/1429463316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5c0005180 0x7fd5c000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:43.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.108+0000 7fd5da3a6640 1 -- 172.21.15.84:0/1429463316 reap_dead start 2023-10-16T11:05:43.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.109+0000 7fd5d93a4640 1 --2- 172.21.15.84:0/1429463316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5c00047c0 0x7fd5c0004c40 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40cce30 tx=0x7fd5c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a67769e93cf90 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:43.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.109+0000 7fd5d8ba3640 1 -- 172.21.15.84:0/1429463316 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d002c660 con 0x7fd5c00047c0 2023-10-16T11:05:43.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.109+0000 7fd5d8ba3640 1 -- 172.21.15.84:0/1429463316 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d002c7c0 con 0x7fd5c00047c0 2023-10-16T11:05:43.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.109+0000 7fd5d8ba3640 1 -- 172.21.15.84:0/1429463316 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d002cb50 con 0x7fd5c00047c0 2023-10-16T11:05:43.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.109+0000 7fd5dbe09640 1 -- 172.21.15.84:0/1429463316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5c00047c0 msgr2=0x7fd5c0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:43.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.109+0000 7fd5dbe09640 1 --2- 172.21.15.84:0/1429463316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5c00047c0 0x7fd5c0004c40 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40cce30 tx=0x7fd5c00057f0 comp rx=0 tx=0).stop 2023-10-16T11:05:43.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.110+0000 7fd5dbe09640 1 -- 172.21.15.84:0/1429463316 shutdown_connections 2023-10-16T11:05:43.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.110+0000 7fd5dbe09640 1 --2- 172.21.15.84:0/1429463316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5c00047c0 0x7fd5c0004c40 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:43.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.110+0000 7fd5dbe09640 1 -- 172.21.15.84:0/1429463316 >> 172.21.15.84:0/1429463316 conn(0x7fd5d40cb8b0 msgr2=0x7fd5d40cbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:43.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.110+0000 7fd5dbe09640 1 -- 172.21.15.84:0/1429463316 shutdown_connections 2023-10-16T11:05:43.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.110+0000 7fd5dbe09640 1 -- 172.21.15.84:0/1429463316 wait complete. 2023-10-16T11:05:43.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.111+0000 7fd5dbe09640 1 Processor -- start 2023-10-16T11:05:43.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.112+0000 7fd5dbe09640 1 -- start start 2023-10-16T11:05:43.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.112+0000 7fd5dbe09640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5c00047c0 0x7fd5d40c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:43.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.112+0000 7fd5dbe09640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5d40c51b0 0x7fd5d40c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:43.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.113+0000 7fd5dbe09640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5d4146e30 0x7fd5d41472b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:43.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.113+0000 7fd5dbe09640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40c2a60 con 0x7fd5d40c51b0 2023-10-16T11:05:43.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.113+0000 7fd5dbe09640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40c84c0 con 0x7fd5d4146e30 2023-10-16T11:05:43.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.113+0000 7fd5dbe09640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d41477f0 con 0x7fd5c00047c0 2023-10-16T11:05:43.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.113+0000 7fd5d93a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5d40c51b0 0x7fd5d40c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:43.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.113+0000 7fd5d93a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5d40c51b0 0x7fd5d40c7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59140/0 (socket says 172.21.15.84:59140) 2023-10-16T11:05:43.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.113+0000 7fd5d93a4640 1 -- 172.21.15.84:0/3595175362 learned_addr learned my addr 172.21.15.84:0/3595175362 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:43.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.113+0000 7fd5da3a6640 1 --2- 172.21.15.84:0/3595175362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5d4146e30 0x7fd5d41472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:43.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.113+0000 7fd5d93a4640 1 -- 172.21.15.84:0/3595175362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5c00047c0 msgr2=0x7fd5d40c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:43.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.113+0000 7fd5d9ba5640 1 --2- 172.21.15.84:0/3595175362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5c00047c0 0x7fd5d40c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:43.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.113+0000 7fd5d93a4640 1 --2- 172.21.15.84:0/3595175362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5c00047c0 0x7fd5d40c5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:43.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.113+0000 7fd5d93a4640 1 -- 172.21.15.84:0/3595175362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5d4146e30 msgr2=0x7fd5d41472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:43.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.113+0000 7fd5d93a4640 1 --2- 172.21.15.84:0/3595175362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5d4146e30 0x7fd5d41472b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:43.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.113+0000 7fd5d93a4640 1 -- 172.21.15.84:0/3595175362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d40c8730 con 0x7fd5d40c51b0 2023-10-16T11:05:43.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.113+0000 7fd5da3a6640 1 --2- 172.21.15.84:0/3595175362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5d4146e30 0x7fd5d41472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:43.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.114+0000 7fd5d93a4640 1 --2- 172.21.15.84:0/3595175362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5d40c51b0 0x7fd5d40c7b60 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009b80 tx=0x7fd5d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:43.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.114+0000 7fd5c67fc640 1 -- 172.21.15.84:0/3595175362 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0003810 con 0x7fd5d40c51b0 2023-10-16T11:05:43.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.114+0000 7fd5c67fc640 1 -- 172.21.15.84:0/3595175362 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d0003970 con 0x7fd5d40c51b0 2023-10-16T11:05:43.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.114+0000 7fd5dbe09640 1 -- 172.21.15.84:0/3595175362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d40c8a10 con 0x7fd5d40c51b0 2023-10-16T11:05:43.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.114+0000 7fd5c67fc640 1 -- 172.21.15.84:0/3595175362 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0003d00 con 0x7fd5d40c51b0 2023-10-16T11:05:43.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.114+0000 7fd5dbe09640 1 -- 172.21.15.84:0/3595175362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d40c5630 con 0x7fd5d40c51b0 2023-10-16T11:05:43.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.115+0000 7fd5dbe09640 1 -- 172.21.15.84:0/3595175362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5d40c2ba0 con 0x7fd5d40c51b0 2023-10-16T11:05:43.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.117+0000 7fd5c67fc640 1 -- 172.21.15.84:0/3595175362 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd5d0040450 con 0x7fd5d40c51b0 2023-10-16T11:05:43.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.118+0000 7fd5c67fc640 1 --2- 172.21.15.84:0/3595175362 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd5b806af90 0x7fd5b806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:43.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.118+0000 7fd5c67fc640 1 -- 172.21.15.84:0/3595175362 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5637+0+0 (secure 0 0 0) 0x7fd5d00abc30 con 0x7fd5d40c51b0 2023-10-16T11:05:43.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.118+0000 7fd5d9ba5640 1 --2- 172.21.15.84:0/3595175362 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd5b806af90 0x7fd5b806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:43.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.119+0000 7fd5d9ba5640 1 --2- 172.21.15.84:0/3595175362 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd5b806af90 0x7fd5b806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80015d0 tx=0x7fd5c8009040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:43.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.122+0000 7fd5c67fc640 1 -- 172.21.15.84:0/3595175362 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd5d0076cf0 con 0x7fd5d40c51b0 2023-10-16T11:05:43.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.340+0000 7fd5dbe09640 1 -- 172.21.15.84:0/3595175362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd5d40c2ba0 con 0x7fd5d40c51b0 2023-10-16T11:05:43.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.925+0000 7fd5c67fc640 1 -- 172.21.15.84:0/3595175362 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v85) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd5d40c2ba0 con 0x7fd5d40c51b0 2023-10-16T11:05:43.927 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 20 and data pool 21 2023-10-16T11:05:43.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.935+0000 7fd5dbe09640 1 -- 172.21.15.84:0/3595175362 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd5b806af90 msgr2=0x7fd5b806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:43.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.935+0000 7fd5dbe09640 1 --2- 172.21.15.84:0/3595175362 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd5b806af90 0x7fd5b806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80015d0 tx=0x7fd5c8009040 comp rx=0 tx=0).stop 2023-10-16T11:05:43.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.935+0000 7fd5dbe09640 1 -- 172.21.15.84:0/3595175362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5d40c51b0 msgr2=0x7fd5d40c7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:43.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.935+0000 7fd5dbe09640 1 --2- 172.21.15.84:0/3595175362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5d40c51b0 0x7fd5d40c7b60 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009b80 tx=0x7fd5d0004730 comp rx=0 tx=0).stop 2023-10-16T11:05:43.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.936+0000 7fd5dbe09640 1 -- 172.21.15.84:0/3595175362 shutdown_connections 2023-10-16T11:05:43.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.936+0000 7fd5dbe09640 1 --2- 172.21.15.84:0/3595175362 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd5b806af90 0x7fd5b806d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:43.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.936+0000 7fd5dbe09640 1 --2- 172.21.15.84:0/3595175362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5d4146e30 0x7fd5d41472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:43.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.936+0000 7fd5dbe09640 1 --2- 172.21.15.84:0/3595175362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5d40c51b0 0x7fd5d40c7b60 secure :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009b80 tx=0x7fd5d0004730 comp rx=0 tx=0).stop 2023-10-16T11:05:43.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.936+0000 7fd5dbe09640 1 --2- 172.21.15.84:0/3595175362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5c00047c0 0x7fd5d40c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:43.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.936+0000 7fd5dbe09640 1 -- 172.21.15.84:0/3595175362 >> 172.21.15.84:0/3595175362 conn(0x7fd5d40cb8b0 msgr2=0x7fd5d415b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:43.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.937+0000 7fd5dbe09640 1 -- 172.21.15.84:0/3595175362 shutdown_connections 2023-10-16T11:05:43.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:43.939+0000 7fd5dbe09640 1 -- 172.21.15.84:0/3595175362 wait complete. 2023-10-16T11:05:43.966 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:05:44.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.147+0000 7f842afc7640 1 Processor -- start 2023-10-16T11:05:44.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.147+0000 7f842afc7640 1 -- start start 2023-10-16T11:05:44.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.149+0000 7f842afc7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f84240cd0c0 0x7f84241615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:44.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.149+0000 7f842afc7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f84240b0410 0x7f84240b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:44.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.149+0000 7f842afc7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f842415d3f0 0x7f842415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:44.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.149+0000 7f842afc7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84240b0de0 con 0x7f84240cd0c0 2023-10-16T11:05:44.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.149+0000 7f842afc7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f842415e3e0 con 0x7f84240b0410 2023-10-16T11:05:44.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.149+0000 7f842afc7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84240cd4a0 con 0x7f842415d3f0 2023-10-16T11:05:44.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.149+0000 7f8428d63640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f84240cd0c0 0x7f84241615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:44.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.149+0000 7f8428d63640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f84240cd0c0 0x7f84241615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59156/0 (socket says 172.21.15.84:59156) 2023-10-16T11:05:44.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.149+0000 7f8428d63640 1 -- 172.21.15.84:0/945245451 learned_addr learned my addr 172.21.15.84:0/945245451 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:44.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.149+0000 7f8429564640 1 --2- 172.21.15.84:0/945245451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f842415d3f0 0x7f842415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:44.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.149+0000 7f8423fff640 1 --2- 172.21.15.84:0/945245451 >> v2:172.21.15.157:3300/0 conn(0x7f84240b0410 0x7f84240b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:44.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.150+0000 7f8428d63640 1 -- 172.21.15.84:0/945245451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f842415d3f0 msgr2=0x7f842415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:44.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.150+0000 7f8428d63640 1 --2- 172.21.15.84:0/945245451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f842415d3f0 0x7f842415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.150+0000 7f8428d63640 1 -- 172.21.15.84:0/945245451 >> v2:172.21.15.157:3300/0 conn(0x7f84240b0410 msgr2=0x7f84240b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:44.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.150+0000 7f8428d63640 1 --2- 172.21.15.84:0/945245451 >> v2:172.21.15.157:3300/0 conn(0x7f84240b0410 0x7f84240b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.150+0000 7f8428d63640 1 -- 172.21.15.84:0/945245451 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842415e100 con 0x7f84240cd0c0 2023-10-16T11:05:44.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.150+0000 7f8423fff640 1 --2- 172.21.15.84:0/945245451 >> v2:172.21.15.157:3300/0 conn(0x7f84240b0410 0x7f84240b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:44.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.151+0000 7f8428d63640 1 --2- 172.21.15.84:0/945245451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84240cd0c0 0x7f84241615f0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f84240cdad0 tx=0x7f841802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8859aedb5136ba server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:44.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.151+0000 7f84237fe640 1 -- 172.21.15.84:0/945245451 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84180317d0 con 0x7f84240cd0c0 2023-10-16T11:05:44.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.151+0000 7f84237fe640 1 -- 172.21.15.84:0/945245451 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8418031930 con 0x7f84240cd0c0 2023-10-16T11:05:44.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.151+0000 7f84237fe640 1 -- 172.21.15.84:0/945245451 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418031cc0 con 0x7f84240cd0c0 2023-10-16T11:05:44.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.152+0000 7f842afc7640 1 -- 172.21.15.84:0/945245451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84240cd0c0 msgr2=0x7f84241615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:44.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.152+0000 7f842afc7640 1 --2- 172.21.15.84:0/945245451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84240cd0c0 0x7f84241615f0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f84240cdad0 tx=0x7f841802b870 comp rx=0 tx=0).stop 2023-10-16T11:05:44.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.152+0000 7f842afc7640 1 -- 172.21.15.84:0/945245451 shutdown_connections 2023-10-16T11:05:44.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.152+0000 7f842afc7640 1 --2- 172.21.15.84:0/945245451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f842415d3f0 0x7f842415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.152+0000 7f842afc7640 1 --2- 172.21.15.84:0/945245451 >> v2:172.21.15.157:3300/0 conn(0x7f84240b0410 0x7f84240b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.152+0000 7f842afc7640 1 --2- 172.21.15.84:0/945245451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84240cd0c0 0x7f84241615f0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.152+0000 7f842afc7640 1 -- 172.21.15.84:0/945245451 >> 172.21.15.84:0/945245451 conn(0x7f84240cb410 msgr2=0x7f84240cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:44.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.152+0000 7f842afc7640 1 -- 172.21.15.84:0/945245451 shutdown_connections 2023-10-16T11:05:44.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.152+0000 7f842afc7640 1 -- 172.21.15.84:0/945245451 wait complete. 2023-10-16T11:05:44.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.154+0000 7f842afc7640 1 Processor -- start 2023-10-16T11:05:44.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.154+0000 7f842afc7640 1 -- start start 2023-10-16T11:05:44.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.155+0000 7f842afc7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84240b0410 0x7f84240c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:44.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.155+0000 7f842afc7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84240cd0c0 0x7f84240c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:44.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.155+0000 7f842afc7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f842415d3f0 0x7f842415c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:44.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.155+0000 7f842afc7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84240c68b0 con 0x7f842415d3f0 2023-10-16T11:05:44.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.155+0000 7f842afc7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84241546a0 con 0x7f84240cd0c0 2023-10-16T11:05:44.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.155+0000 7f842afc7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84240c62f0 con 0x7f84240b0410 2023-10-16T11:05:44.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.155+0000 7f8429564640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f842415d3f0 0x7f842415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:44.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.155+0000 7f8429564640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f842415d3f0 0x7f842415c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59162/0 (socket says 172.21.15.84:59162) 2023-10-16T11:05:44.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.155+0000 7f8429564640 1 -- 172.21.15.84:0/1623060838 learned_addr learned my addr 172.21.15.84:0/1623060838 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:44.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.156+0000 7f8428d63640 1 --2- 172.21.15.84:0/1623060838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84240b0410 0x7f84240c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:44.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.156+0000 7f8423fff640 1 --2- 172.21.15.84:0/1623060838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84240cd0c0 0x7f84240c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:44.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.156+0000 7f8429564640 1 -- 172.21.15.84:0/1623060838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84240b0410 msgr2=0x7f84240c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:44.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.156+0000 7f8429564640 1 --2- 172.21.15.84:0/1623060838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84240b0410 0x7f84240c5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.156+0000 7f8429564640 1 -- 172.21.15.84:0/1623060838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84240cd0c0 msgr2=0x7f84240c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:44.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.156+0000 7f8429564640 1 --2- 172.21.15.84:0/1623060838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84240cd0c0 0x7f84240c8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.156+0000 7f8429564640 1 -- 172.21.15.84:0/1623060838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84180094d0 con 0x7f842415d3f0 2023-10-16T11:05:44.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.156+0000 7f8429564640 1 --2- 172.21.15.84:0/1623060838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f842415d3f0 0x7f842415c710 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f841c007e40 tx=0x7f841c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:44.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.156+0000 7f84217fa640 1 -- 172.21.15.84:0/1623060838 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c014620 con 0x7f842415d3f0 2023-10-16T11:05:44.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.156+0000 7f84217fa640 1 -- 172.21.15.84:0/1623060838 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841c00ee90 con 0x7f842415d3f0 2023-10-16T11:05:44.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.156+0000 7f84217fa640 1 -- 172.21.15.84:0/1623060838 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c0149e0 con 0x7f842415d3f0 2023-10-16T11:05:44.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.156+0000 7f842afc7640 1 -- 172.21.15.84:0/1623060838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84240c65d0 con 0x7f842415d3f0 2023-10-16T11:05:44.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.157+0000 7f842afc7640 1 -- 172.21.15.84:0/1623060838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8424154bb0 con 0x7f842415d3f0 2023-10-16T11:05:44.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.158+0000 7f842afc7640 1 -- 172.21.15.84:0/1623060838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f842404e160 con 0x7f842415d3f0 2023-10-16T11:05:44.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.159+0000 7f84217fa640 1 -- 172.21.15.84:0/1623060838 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f841c014b40 con 0x7f842415d3f0 2023-10-16T11:05:44.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.159+0000 7f84217fa640 1 --2- 172.21.15.84:0/1623060838 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f840806afc0 0x7f840806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:44.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.160+0000 7f84217fa640 1 -- 172.21.15.84:0/1623060838 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f841c08a200 con 0x7f842415d3f0 2023-10-16T11:05:44.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.160+0000 7f8428d63640 1 --2- 172.21.15.84:0/1623060838 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f840806afc0 0x7f840806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:44.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.160+0000 7f8428d63640 1 --2- 172.21.15.84:0/1623060838 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f840806afc0 0x7f840806d480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8418036000 tx=0x7f841803d040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:44.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.165+0000 7f84217fa640 1 -- 172.21.15.84:0/1623060838 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f841c0550e0 con 0x7f842415d3f0 2023-10-16T11:05:44.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.350+0000 7f842afc7640 1 -- 172.21.15.84:0/1623060838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8424070800 con 0x7f842415d3f0 2023-10-16T11:05:44.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.352+0000 7f84217fa640 1 -- 172.21.15.84:0/1623060838 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+16612 (secure 0 0 0) 0x7f841c058700 con 0x7f842415d3f0 2023-10-16T11:05:44.354 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:44.355 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":170,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:05:43.341065+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:05:41.217474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-16T11:05:42.387743+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:05:44.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.355+0000 7f842afc7640 1 -- 172.21.15.84:0/1623060838 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f840806afc0 msgr2=0x7f840806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:44.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.355+0000 7f842afc7640 1 --2- 172.21.15.84:0/1623060838 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f840806afc0 0x7f840806d480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8418036000 tx=0x7f841803d040 comp rx=0 tx=0).stop 2023-10-16T11:05:44.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.355+0000 7f842afc7640 1 -- 172.21.15.84:0/1623060838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f842415d3f0 msgr2=0x7f842415c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:44.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.355+0000 7f842afc7640 1 --2- 172.21.15.84:0/1623060838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f842415d3f0 0x7f842415c710 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f841c007e40 tx=0x7f841c00e4b0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.355+0000 7f842afc7640 1 -- 172.21.15.84:0/1623060838 shutdown_connections 2023-10-16T11:05:44.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.355+0000 7f842afc7640 1 --2- 172.21.15.84:0/1623060838 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f840806afc0 0x7f840806d480 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.355+0000 7f842afc7640 1 --2- 172.21.15.84:0/1623060838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f842415d3f0 0x7f842415c710 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.355+0000 7f842afc7640 1 --2- 172.21.15.84:0/1623060838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84240cd0c0 0x7f84240c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.355+0000 7f842afc7640 1 --2- 172.21.15.84:0/1623060838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84240b0410 0x7f84240c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.355+0000 7f842afc7640 1 -- 172.21.15.84:0/1623060838 >> 172.21.15.84:0/1623060838 conn(0x7f84240cb410 msgr2=0x7f842415b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:44.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.355+0000 7f842afc7640 1 -- 172.21.15.84:0/1623060838 shutdown_connections 2023-10-16T11:05:44.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.355+0000 7f842afc7640 1 -- 172.21.15.84:0/1623060838 wait complete. 2023-10-16T11:05:44.375 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:05:44.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.573+0000 7fd92c9cc640 1 Processor -- start 2023-10-16T11:05:44.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.573+0000 7fd92c9cc640 1 -- start start 2023-10-16T11:05:44.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.574+0000 7fd92c9cc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd92815aaa0 0x7fd92815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:44.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.575+0000 7fd92c9cc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd9280ce390 0x7fd928070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:44.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.575+0000 7fd92c9cc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92815ed60 con 0x7fd92815ae80 2023-10-16T11:05:44.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.575+0000 7fd92c9cc640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd92806b810 con 0x7fd9280ce390 2023-10-16T11:05:44.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.575+0000 7fd92c9cc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280707e0 con 0x7fd92815aaa0 2023-10-16T11:05:44.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.575+0000 7fd926d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd9280ce390 0x7fd928070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:44.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.575+0000 7fd926d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd9280ce390 0x7fd928070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59178/0 (socket says 172.21.15.84:59178) 2023-10-16T11:05:44.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.575+0000 7fd926d9d640 1 -- 172.21.15.84:0/596128982 learned_addr learned my addr 172.21.15.84:0/596128982 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:44.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.575+0000 7fd92659c640 1 --2- 172.21.15.84:0/596128982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd92815aaa0 0x7fd92815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:44.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.575+0000 7fd926d9d640 1 -- 172.21.15.84:0/596128982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd92815aaa0 msgr2=0x7fd92815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:44.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.575+0000 7fd926d9d640 1 --2- 172.21.15.84:0/596128982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd92815aaa0 0x7fd92815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.575+0000 7fd926d9d640 1 -- 172.21.15.84:0/596128982 >> v1:172.21.15.157:6789/0 conn(0x7fd92815ae80 legacy=0x7fd9280cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:44.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.575+0000 7fd926d9d640 1 -- 172.21.15.84:0/596128982 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92815ef90 con 0x7fd9280ce390 2023-10-16T11:05:44.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.576+0000 7fd926d9d640 1 --2- 172.21.15.84:0/596128982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9280ce390 0x7fd928070f50 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fd920009e50 tx=0x7fd92002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965ce792815d4da0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:44.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.577+0000 7fd92559a640 1 -- 172.21.15.84:0/596128982 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92002ca70 con 0x7fd9280ce390 2023-10-16T11:05:44.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.577+0000 7fd92559a640 1 -- 172.21.15.84:0/596128982 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92002cbd0 con 0x7fd9280ce390 2023-10-16T11:05:44.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.577+0000 7fd92559a640 1 -- 172.21.15.84:0/596128982 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9200343f0 con 0x7fd9280ce390 2023-10-16T11:05:44.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.577+0000 7fd92c9cc640 1 -- 172.21.15.84:0/596128982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9280ce390 msgr2=0x7fd928070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:44.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.577+0000 7fd92c9cc640 1 --2- 172.21.15.84:0/596128982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9280ce390 0x7fd928070f50 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fd920009e50 tx=0x7fd92002bed0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.577+0000 7fd92c9cc640 1 -- 172.21.15.84:0/596128982 shutdown_connections 2023-10-16T11:05:44.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.577+0000 7fd92c9cc640 1 --2- 172.21.15.84:0/596128982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9280ce390 0x7fd928070f50 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.577+0000 7fd92c9cc640 1 --2- 172.21.15.84:0/596128982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd92815aaa0 0x7fd92815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.577+0000 7fd92c9cc640 1 -- 172.21.15.84:0/596128982 >> 172.21.15.84:0/596128982 conn(0x7fd9280cb410 msgr2=0x7fd9280cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:44.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.578+0000 7fd92c9cc640 1 -- 172.21.15.84:0/596128982 shutdown_connections 2023-10-16T11:05:44.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.578+0000 7fd92c9cc640 1 -- 172.21.15.84:0/596128982 wait complete. 2023-10-16T11:05:44.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.580+0000 7fd92c9cc640 1 Processor -- start 2023-10-16T11:05:44.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.580+0000 7fd92c9cc640 1 -- start start 2023-10-16T11:05:44.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.581+0000 7fd92c9cc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9280ce390 0x7fd92814f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:44.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.581+0000 7fd92c9cc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd92815aaa0 0x7fd9280c1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:44.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.581+0000 7fd92c9cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd92815ae80 0x7fd9280ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:44.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.581+0000 7fd92c9cc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280c9070 con 0x7fd92815aaa0 2023-10-16T11:05:44.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.581+0000 7fd92c9cc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280c8200 con 0x7fd92815ae80 2023-10-16T11:05:44.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.581+0000 7fd92c9cc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280c1e70 con 0x7fd9280ce390 2023-10-16T11:05:44.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.581+0000 7fd925d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd92815aaa0 0x7fd9280c1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:44.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.581+0000 7fd925d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd92815aaa0 0x7fd9280c1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59194/0 (socket says 172.21.15.84:59194) 2023-10-16T11:05:44.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.581+0000 7fd925d9b640 1 -- 172.21.15.84:0/2678107962 learned_addr learned my addr 172.21.15.84:0/2678107962 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:44.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.582+0000 7fd925d9b640 1 -- 172.21.15.84:0/2678107962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9280ce390 msgr2=0x7fd92814f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:44.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.582+0000 7fd925d9b640 1 --2- 172.21.15.84:0/2678107962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9280ce390 0x7fd92814f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.582+0000 7fd926d9d640 1 --2- 172.21.15.84:0/2678107962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd92815ae80 0x7fd9280ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:44.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.582+0000 7fd925d9b640 1 -- 172.21.15.84:0/2678107962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd92815ae80 msgr2=0x7fd9280ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:44.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.582+0000 7fd925d9b640 1 --2- 172.21.15.84:0/2678107962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd92815ae80 0x7fd9280ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.582+0000 7fd925d9b640 1 -- 172.21.15.84:0/2678107962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd920009b30 con 0x7fd92815aaa0 2023-10-16T11:05:44.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.582+0000 7fd925d9b640 1 --2- 172.21.15.84:0/2678107962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd92815aaa0 0x7fd9280c1930 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00c410 tx=0x7fd91c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:44.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.582+0000 7fd916ffd640 1 -- 172.21.15.84:0/2678107962 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c0035f0 con 0x7fd92815aaa0 2023-10-16T11:05:44.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.582+0000 7fd916ffd640 1 -- 172.21.15.84:0/2678107962 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91c003750 con 0x7fd92815aaa0 2023-10-16T11:05:44.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.582+0000 7fd916ffd640 1 -- 172.21.15.84:0/2678107962 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c003b40 con 0x7fd92815aaa0 2023-10-16T11:05:44.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.582+0000 7fd92c9cc640 1 -- 172.21.15.84:0/2678107962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92815d840 con 0x7fd92815aaa0 2023-10-16T11:05:44.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.582+0000 7fd92c9cc640 1 -- 172.21.15.84:0/2678107962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9280c8780 con 0x7fd92815aaa0 2023-10-16T11:05:44.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.584+0000 7fd92c9cc640 1 -- 172.21.15.84:0/2678107962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92804e160 con 0x7fd92815aaa0 2023-10-16T11:05:44.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.584+0000 7fd916ffd640 1 -- 172.21.15.84:0/2678107962 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd91c003ca0 con 0x7fd92815aaa0 2023-10-16T11:05:44.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.585+0000 7fd916ffd640 1 --2- 172.21.15.84:0/2678107962 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd90806af90 0x7fd90806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:44.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.585+0000 7fd916ffd640 1 -- 172.21.15.84:0/2678107962 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd91c0889b0 con 0x7fd92815aaa0 2023-10-16T11:05:44.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.586+0000 7fd92659c640 1 --2- 172.21.15.84:0/2678107962 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd90806af90 0x7fd90806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:44.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.586+0000 7fd92659c640 1 --2- 172.21.15.84:0/2678107962 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd90806af90 0x7fd90806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd9180098f0 tx=0x7fd918008040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:44.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.590+0000 7fd916ffd640 1 -- 172.21.15.84:0/2678107962 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd91c053930 con 0x7fd92815aaa0 2023-10-16T11:05:44.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.768+0000 7fd92c9cc640 1 -- 172.21.15.84:0/2678107962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd928071280 con 0x7fd92815aaa0 2023-10-16T11:05:44.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.770+0000 7fd916ffd640 1 -- 172.21.15.84:0/2678107962 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+16612 (secure 0 0 0) 0x7fd91c056f50 con 0x7fd92815aaa0 2023-10-16T11:05:44.772 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:44.773 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":170,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:05:43.341065+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:05:41.217474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-16T11:05:42.387743+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:05:44.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.773+0000 7fd92c9cc640 1 -- 172.21.15.84:0/2678107962 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd90806af90 msgr2=0x7fd90806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:44.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.773+0000 7fd92c9cc640 1 --2- 172.21.15.84:0/2678107962 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd90806af90 0x7fd90806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd9180098f0 tx=0x7fd918008040 comp rx=0 tx=0).stop 2023-10-16T11:05:44.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.773+0000 7fd92c9cc640 1 -- 172.21.15.84:0/2678107962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd92815aaa0 msgr2=0x7fd9280c1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:44.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.773+0000 7fd92c9cc640 1 --2- 172.21.15.84:0/2678107962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd92815aaa0 0x7fd9280c1930 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00c410 tx=0x7fd91c00c8d0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.773+0000 7fd92c9cc640 1 -- 172.21.15.84:0/2678107962 shutdown_connections 2023-10-16T11:05:44.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.773+0000 7fd92c9cc640 1 --2- 172.21.15.84:0/2678107962 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd90806af90 0x7fd90806d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.773+0000 7fd92c9cc640 1 --2- 172.21.15.84:0/2678107962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd92815ae80 0x7fd9280ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.773+0000 7fd92c9cc640 1 --2- 172.21.15.84:0/2678107962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd92815aaa0 0x7fd9280c1930 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.773+0000 7fd92c9cc640 1 --2- 172.21.15.84:0/2678107962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9280ce390 0x7fd92814f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:44.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.773+0000 7fd92c9cc640 1 -- 172.21.15.84:0/2678107962 >> 172.21.15.84:0/2678107962 conn(0x7fd9280cb410 msgr2=0x7fd928070010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:44.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.773+0000 7fd92c9cc640 1 -- 172.21.15.84:0/2678107962 shutdown_connections 2023-10-16T11:05:44.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.773+0000 7fd92c9cc640 1 -- 172.21.15.84:0/2678107962 wait complete. 2023-10-16T11:05:44.791 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T11:05:44.856 INFO:tasks.ceph.mds.c.smithi084.stderr:2023-10-16T11:05:44.855+0000 7f95f422c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:05:44.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.994+0000 7f24ff051640 1 Processor -- start 2023-10-16T11:05:44.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.994+0000 7f24ff051640 1 -- start start 2023-10-16T11:05:44.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.996+0000 7f24ff051640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f806a4b0 0x7f24f80ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:44.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.996+0000 7f24ff051640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f24f80c9ca0 0x7f24f80ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:44.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.996+0000 7f24ff051640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f80cb030 con 0x7f24f815ad70 2023-10-16T11:05:44.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.996+0000 7f24ff051640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f80cae30 con 0x7f24f80c9ca0 2023-10-16T11:05:44.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.996+0000 7f24ff051640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f80c91c0 con 0x7f24f806a4b0 2023-10-16T11:05:44.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.996+0000 7f24fcded640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f24f815ad70 0x7f24f8069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42500/0 (socket says 172.21.15.84:42500) 2023-10-16T11:05:44.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.996+0000 7f24fcded640 1 -- 172.21.15.84:0/1727099182 learned_addr learned my addr 172.21.15.84:0/1727099182 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:45.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.996+0000 7f24fd5ee640 1 --2- 172.21.15.84:0/1727099182 >> v2:172.21.15.84:3300/0 conn(0x7f24f80c9ca0 0x7f24f80ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:45.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.996+0000 7f24f7fff640 1 --2- 172.21.15.84:0/1727099182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f806a4b0 0x7f24f80ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:45.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.996+0000 7f24f77fe640 1 -- 172.21.15.84:0/1727099182 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1778827237 0 0) 0x7f24f80cb030 con 0x7f24f815ad70 2023-10-16T11:05:45.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.996+0000 7f24fd5ee640 1 -- 172.21.15.84:0/1727099182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f806a4b0 msgr2=0x7f24f80ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:45.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.996+0000 7f24fd5ee640 1 --2- 172.21.15.84:0/1727099182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f806a4b0 0x7f24f80ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:45.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.996+0000 7f24fd5ee640 1 -- 172.21.15.84:0/1727099182 >> v1:172.21.15.84:6789/0 conn(0x7f24f815ad70 legacy=0x7f24f8069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:45.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.997+0000 7f24fd5ee640 1 -- 172.21.15.84:0/1727099182 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f80c94a0 con 0x7f24f80c9ca0 2023-10-16T11:05:45.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.998+0000 7f24fd5ee640 1 --2- 172.21.15.84:0/1727099182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24f80c9ca0 0x7f24f80ca8f0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009d90 tx=0x7f24f002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fad1eb71ed8208 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:45.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.998+0000 7f24f77fe640 1 -- 172.21.15.84:0/1727099182 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f002cc50 con 0x7f24f80c9ca0 2023-10-16T11:05:45.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.998+0000 7f24f77fe640 1 -- 172.21.15.84:0/1727099182 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f002cdb0 con 0x7f24f80c9ca0 2023-10-16T11:05:45.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.998+0000 7f24f77fe640 1 -- 172.21.15.84:0/1727099182 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f0034660 con 0x7f24f80c9ca0 2023-10-16T11:05:45.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.998+0000 7f24ff051640 1 -- 172.21.15.84:0/1727099182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24f80c9ca0 msgr2=0x7f24f80ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:45.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.998+0000 7f24ff051640 1 --2- 172.21.15.84:0/1727099182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24f80c9ca0 0x7f24f80ca8f0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009d90 tx=0x7f24f002bea0 comp rx=0 tx=0).stop 2023-10-16T11:05:45.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.999+0000 7f24ff051640 1 -- 172.21.15.84:0/1727099182 shutdown_connections 2023-10-16T11:05:45.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.999+0000 7f24ff051640 1 --2- 172.21.15.84:0/1727099182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24f80c9ca0 0x7f24f80ca8f0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:45.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.999+0000 7f24ff051640 1 --2- 172.21.15.84:0/1727099182 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f806a4b0 0x7f24f80ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:45.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.999+0000 7f24ff051640 1 -- 172.21.15.84:0/1727099182 >> 172.21.15.84:0/1727099182 conn(0x7f24f80ce9b0 msgr2=0x7f24f80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:45.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.999+0000 7f24ff051640 1 -- 172.21.15.84:0/1727099182 shutdown_connections 2023-10-16T11:05:45.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:44.999+0000 7f24ff051640 1 -- 172.21.15.84:0/1727099182 wait complete. 2023-10-16T11:05:45.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.000+0000 7f24ff051640 1 Processor -- start 2023-10-16T11:05:45.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.000+0000 7f24ff051640 1 -- start start 2023-10-16T11:05:45.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.001+0000 7f24ff051640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f806a4b0 0x7f24f8157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:45.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.001+0000 7f24ff051640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f80c9ca0 0x7f24f80a9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:45.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.001+0000 7f24ff051640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24f815ad70 0x7f24f80aa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:45.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.001+0000 7f24ff051640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f80af210 con 0x7f24f815ad70 2023-10-16T11:05:45.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.001+0000 7f24ff051640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f80a9100 con 0x7f24f80c9ca0 2023-10-16T11:05:45.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.001+0000 7f24ff051640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f80a93e0 con 0x7f24f806a4b0 2023-10-16T11:05:45.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.002+0000 7f24fd5ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24f815ad70 0x7f24f80aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:45.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.002+0000 7f24fd5ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24f815ad70 0x7f24f80aa500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59206/0 (socket says 172.21.15.84:59206) 2023-10-16T11:05:45.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.002+0000 7f24fd5ee640 1 -- 172.21.15.84:0/3724496502 learned_addr learned my addr 172.21.15.84:0/3724496502 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:45.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.002+0000 7f24fcded640 1 --2- 172.21.15.84:0/3724496502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f806a4b0 0x7f24f8157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:45.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.002+0000 7f24fd5ee640 1 -- 172.21.15.84:0/3724496502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f806a4b0 msgr2=0x7f24f8157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:45.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.002+0000 7f24f7fff640 1 --2- 172.21.15.84:0/3724496502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f80c9ca0 0x7f24f80a9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:45.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.002+0000 7f24fd5ee640 1 --2- 172.21.15.84:0/3724496502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f806a4b0 0x7f24f8157db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:45.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.002+0000 7f24fd5ee640 1 -- 172.21.15.84:0/3724496502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f80c9ca0 msgr2=0x7f24f80a9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:45.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.002+0000 7f24fd5ee640 1 --2- 172.21.15.84:0/3724496502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f80c9ca0 0x7f24f80a9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:45.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.002+0000 7f24fd5ee640 1 -- 172.21.15.84:0/3724496502 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f0009a70 con 0x7f24f815ad70 2023-10-16T11:05:45.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.002+0000 7f24fd5ee640 1 --2- 172.21.15.84:0/3724496502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24f815ad70 0x7f24f80aa500 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009d90 tx=0x7f24f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:45.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.003+0000 7f24f57fa640 1 -- 172.21.15.84:0/3724496502 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f003e610 con 0x7f24f815ad70 2023-10-16T11:05:45.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.003+0000 7f24f57fa640 1 -- 172.21.15.84:0/3724496502 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f003e790 con 0x7f24f815ad70 2023-10-16T11:05:45.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.003+0000 7f24f57fa640 1 -- 172.21.15.84:0/3724496502 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f003eb40 con 0x7f24f815ad70 2023-10-16T11:05:45.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.003+0000 7f24ff051640 1 -- 172.21.15.84:0/3724496502 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f80a96c0 con 0x7f24f815ad70 2023-10-16T11:05:45.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.003+0000 7f24ff051640 1 -- 172.21.15.84:0/3724496502 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f81582f0 con 0x7f24f815ad70 2023-10-16T11:05:45.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.004+0000 7f24ff051640 1 -- 172.21.15.84:0/3724496502 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f80c5af0 con 0x7f24f815ad70 2023-10-16T11:05:45.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.005+0000 7f24f57fa640 1 -- 172.21.15.84:0/3724496502 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f24f003b050 con 0x7f24f815ad70 2023-10-16T11:05:45.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.006+0000 7f24f57fa640 1 --2- 172.21.15.84:0/3724496502 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f24dc06af90 0x7f24dc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:45.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.007+0000 7f24f57fa640 1 -- 172.21.15.84:0/3724496502 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f24f00abe20 con 0x7f24f815ad70 2023-10-16T11:05:45.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.007+0000 7f24fcded640 1 --2- 172.21.15.84:0/3724496502 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f24dc06af90 0x7f24dc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:45.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.007+0000 7f24fcded640 1 --2- 172.21.15.84:0/3724496502 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f24dc06af90 0x7f24dc06d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f24ec000f60 tx=0x7f24ec009040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:45.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.011+0000 7f24f57fa640 1 -- 172.21.15.84:0/3724496502 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f24f80c5af0 con 0x7f24f815ad70 2023-10-16T11:05:45.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.223+0000 7f24ff051640 1 -- 172.21.15.84:0/3724496502 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f24f80c5af0 con 0x7f24f815ad70 2023-10-16T11:05:45.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.930+0000 7f24f57fa640 1 -- 172.21.15.84:0/3724496502 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v87) v1 ==== 118+0+0 (secure 0 0 0) 0x7f24f80c5af0 con 0x7f24f815ad70 2023-10-16T11:05:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.934+0000 7f24ff051640 1 -- 172.21.15.84:0/3724496502 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f24dc06af90 msgr2=0x7f24dc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.934+0000 7f24ff051640 1 --2- 172.21.15.84:0/3724496502 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f24dc06af90 0x7f24dc06d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f24ec000f60 tx=0x7f24ec009040 comp rx=0 tx=0).stop 2023-10-16T11:05:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.934+0000 7f24ff051640 1 -- 172.21.15.84:0/3724496502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24f815ad70 msgr2=0x7f24f80aa500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.934+0000 7f24ff051640 1 --2- 172.21.15.84:0/3724496502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24f815ad70 0x7f24f80aa500 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009d90 tx=0x7f24f0004a40 comp rx=0 tx=0).stop 2023-10-16T11:05:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.934+0000 7f24ff051640 1 -- 172.21.15.84:0/3724496502 shutdown_connections 2023-10-16T11:05:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.934+0000 7f24ff051640 1 --2- 172.21.15.84:0/3724496502 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f24dc06af90 0x7f24dc06d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.934+0000 7f24ff051640 1 --2- 172.21.15.84:0/3724496502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24f815ad70 0x7f24f80aa500 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.934+0000 7f24ff051640 1 --2- 172.21.15.84:0/3724496502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f80c9ca0 0x7f24f80a9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:45.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.934+0000 7f24ff051640 1 --2- 172.21.15.84:0/3724496502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24f806a4b0 0x7f24f8157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:45.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.934+0000 7f24ff051640 1 -- 172.21.15.84:0/3724496502 >> 172.21.15.84:0/3724496502 conn(0x7f24f80ce9b0 msgr2=0x7f24f815dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:45.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.934+0000 7f24ff051640 1 -- 172.21.15.84:0/3724496502 shutdown_connections 2023-10-16T11:05:45.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:45.934+0000 7f24ff051640 1 -- 172.21.15.84:0/3724496502 wait complete. 2023-10-16T11:05:45.955 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T11:05:45.955 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T11:05:46.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.174+0000 7f86f4cbb640 1 Processor -- start 2023-10-16T11:05:46.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.174+0000 7f86f4cbb640 1 -- start start 2023-10-16T11:05:46.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.176+0000 7f86f4cbb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f86f015b320 0x7f86f00cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:46.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.176+0000 7f86f4cbb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f015b9d0 con 0x7f86f006c040 2023-10-16T11:05:46.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.176+0000 7f86f4cbb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f015bcb0 con 0x7f86f00ad970 2023-10-16T11:05:46.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.176+0000 7f86f4cbb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f015b7d0 con 0x7f86f015b320 2023-10-16T11:05:46.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.176+0000 7f86ef7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f86f006c040 0x7f86f006c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34284/0 (socket says 172.21.15.84:34284) 2023-10-16T11:05:46.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.176+0000 7f86ef7fe640 1 -- 172.21.15.84:0/3449907712 learned_addr learned my addr 172.21.15.84:0/3449907712 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:46.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.177+0000 7f86effff640 1 --2- 172.21.15.84:0/3449907712 >> v2:172.21.15.157:3300/0 conn(0x7f86f015b320 0x7f86f00cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:46.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.177+0000 7f86eeffd640 1 -- 172.21.15.84:0/3449907712 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3262799339 0 0) 0x7f86f015b9d0 con 0x7f86f006c040 2023-10-16T11:05:46.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.177+0000 7f86eeffd640 1 -- 172.21.15.84:0/3449907712 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86dc003520 con 0x7f86f006c040 2023-10-16T11:05:46.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.177+0000 7f86eeffd640 1 -- 172.21.15.84:0/3449907712 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3860991627 0 0) 0x7f86dc003520 con 0x7f86f006c040 2023-10-16T11:05:46.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.177+0000 7f86eeffd640 1 -- 172.21.15.84:0/3449907712 >> v2:172.21.15.157:3300/0 conn(0x7f86f015b320 msgr2=0x7f86f00cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:46.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.177+0000 7f86eeffd640 1 --2- 172.21.15.84:0/3449907712 >> v2:172.21.15.157:3300/0 conn(0x7f86f015b320 0x7f86f00cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:46.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.177+0000 7f86eeffd640 1 -- 172.21.15.84:0/3449907712 >> v1:172.21.15.157:6789/0 conn(0x7f86f00ad970 legacy=0x7f86f00cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:46.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.177+0000 7f86eeffd640 1 -- 172.21.15.84:0/3449907712 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f015bf90 con 0x7f86f006c040 2023-10-16T11:05:46.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.177+0000 7f86eeffd640 1 -- 172.21.15.84:0/3449907712 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f86e0001670 con 0x7f86f006c040 2023-10-16T11:05:46.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.177+0000 7f86eeffd640 1 -- 172.21.15.84:0/3449907712 >> v1:172.21.15.84:6789/0 conn(0x7f86f006c040 legacy=0x7f86f006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:46.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.177+0000 7f86eeffd640 1 --2- 172.21.15.84:0/3449907712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86dc0039f0 0x7f86dc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:46.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.177+0000 7f86eeffd640 1 --2- 172.21.15.84:0/3449907712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86dc0047c0 0x7f86dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:46.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.177+0000 7f86eeffd640 1 --2- 172.21.15.84:0/3449907712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86dc005180 0x7f86dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:46.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.177+0000 7f86eeffd640 1 -- 172.21.15.84:0/3449907712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f015b7d0 con 0x7f86dc005180 2023-10-16T11:05:46.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.177+0000 7f86eeffd640 1 -- 172.21.15.84:0/3449907712 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc00c0b0 con 0x7f86dc0039f0 2023-10-16T11:05:46.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.177+0000 7f86eeffd640 1 -- 172.21.15.84:0/3449907712 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc00c270 con 0x7f86dc0047c0 2023-10-16T11:05:46.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.178+0000 7f86effff640 1 --2- 172.21.15.84:0/3449907712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86dc005180 0x7f86dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:46.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.178+0000 7f86ef7fe640 1 --2- 172.21.15.84:0/3449907712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86dc0039f0 0x7f86dc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:46.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.178+0000 7f86effff640 1 -- 172.21.15.84:0/3449907712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86dc0047c0 msgr2=0x7f86dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:46.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.178+0000 7f86effff640 1 --2- 172.21.15.84:0/3449907712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86dc0047c0 0x7f86dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:46.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.178+0000 7f86effff640 1 -- 172.21.15.84:0/3449907712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86dc0039f0 msgr2=0x7f86dc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:46.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.178+0000 7f86effff640 1 --2- 172.21.15.84:0/3449907712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86dc0039f0 0x7f86dc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:46.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.178+0000 7f86effff640 1 -- 172.21.15.84:0/3449907712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc003520 con 0x7f86dc005180 2023-10-16T11:05:46.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.178+0000 7f86ef7fe640 1 --2- 172.21.15.84:0/3449907712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86dc0039f0 0x7f86dc003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:46.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.179+0000 7f86effff640 1 -- 172.21.15.84:0/3449907712 reap_dead start 2023-10-16T11:05:46.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.179+0000 7f86effff640 1 --2- 172.21.15.84:0/3449907712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86dc005180 0x7f86dc00b980 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009fc0 tx=0x7f86dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeeb9bc9a0d6ebf4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:46.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.179+0000 7f86eeffd640 1 -- 172.21.15.84:0/3449907712 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e802cc60 con 0x7f86dc005180 2023-10-16T11:05:46.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.179+0000 7f86eeffd640 1 -- 172.21.15.84:0/3449907712 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e800f040 con 0x7f86dc005180 2023-10-16T11:05:46.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.179+0000 7f86eeffd640 1 -- 172.21.15.84:0/3449907712 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8003990 con 0x7f86dc005180 2023-10-16T11:05:46.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.180+0000 7f86f4cbb640 1 -- 172.21.15.84:0/3449907712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86dc005180 msgr2=0x7f86dc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:46.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.180+0000 7f86f4cbb640 1 --2- 172.21.15.84:0/3449907712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86dc005180 0x7f86dc00b980 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009fc0 tx=0x7f86dc0057f0 comp rx=0 tx=0).stop 2023-10-16T11:05:46.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.180+0000 7f86f4cbb640 1 -- 172.21.15.84:0/3449907712 shutdown_connections 2023-10-16T11:05:46.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.180+0000 7f86f4cbb640 1 --2- 172.21.15.84:0/3449907712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86dc005180 0x7f86dc00b980 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:46.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.180+0000 7f86f4cbb640 1 -- 172.21.15.84:0/3449907712 >> 172.21.15.84:0/3449907712 conn(0x7f86f00ce6a0 msgr2=0x7f86f00ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:46.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.180+0000 7f86f4cbb640 1 -- 172.21.15.84:0/3449907712 shutdown_connections 2023-10-16T11:05:46.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.180+0000 7f86f4cbb640 1 -- 172.21.15.84:0/3449907712 wait complete. 2023-10-16T11:05:46.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.182+0000 7f86f4cbb640 1 Processor -- start 2023-10-16T11:05:46.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.182+0000 7f86f4cbb640 1 -- start start 2023-10-16T11:05:46.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.183+0000 7f86f4cbb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86dc005180 0x7f86f01529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:46.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.183+0000 7f86f4cbb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86f00b3ed0 0x7f86f00b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:46.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.183+0000 7f86f4cbb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f00b26e0 0x7f86f0159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:46.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.183+0000 7f86f4cbb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f0152f20 con 0x7f86dc005180 2023-10-16T11:05:46.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.183+0000 7f86f4cbb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f00b3110 con 0x7f86f00b26e0 2023-10-16T11:05:46.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.183+0000 7f86f4cbb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f00b33f0 con 0x7f86f00b3ed0 2023-10-16T11:05:46.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.183+0000 7f86ef7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86dc005180 0x7f86f01529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:46.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.183+0000 7f86ef7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86dc005180 0x7f86f01529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54662/0 (socket says 172.21.15.84:54662) 2023-10-16T11:05:46.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.183+0000 7f86ef7fe640 1 -- 172.21.15.84:0/432988609 learned_addr learned my addr 172.21.15.84:0/432988609 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:46.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.184+0000 7f86ef7fe640 1 -- 172.21.15.84:0/432988609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86f00b3ed0 msgr2=0x7f86f00b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:46.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.184+0000 7f86ef7fe640 1 --2- 172.21.15.84:0/432988609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86f00b3ed0 0x7f86f00b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:46.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.184+0000 7f86ef7fe640 1 -- 172.21.15.84:0/432988609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f00b26e0 msgr2=0x7f86f0159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:46.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.184+0000 7f86effff640 1 --2- 172.21.15.84:0/432988609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f00b26e0 0x7f86f0159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:46.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.184+0000 7f86ef7fe640 1 --2- 172.21.15.84:0/432988609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f00b26e0 0x7f86f0159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:46.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.184+0000 7f86ef7fe640 1 -- 172.21.15.84:0/432988609 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f00b36d0 con 0x7f86dc005180 2023-10-16T11:05:46.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.184+0000 7f86ef7fe640 1 --2- 172.21.15.84:0/432988609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86dc005180 0x7f86f01529e0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f86e000d7a0 tx=0x7f86e000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:46.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.184+0000 7f86ecff9640 1 -- 172.21.15.84:0/432988609 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e00074e0 con 0x7f86dc005180 2023-10-16T11:05:46.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.184+0000 7f86ecff9640 1 -- 172.21.15.84:0/432988609 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e0007640 con 0x7f86dc005180 2023-10-16T11:05:46.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.184+0000 7f86ecff9640 1 -- 172.21.15.84:0/432988609 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e00049e0 con 0x7f86dc005180 2023-10-16T11:05:46.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.184+0000 7f86f4cbb640 1 -- 172.21.15.84:0/432988609 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f00b39b0 con 0x7f86dc005180 2023-10-16T11:05:46.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.185+0000 7f86f4cbb640 1 -- 172.21.15.84:0/432988609 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f00b2ca0 con 0x7f86dc005180 2023-10-16T11:05:46.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.186+0000 7f86f4cbb640 1 -- 172.21.15.84:0/432988609 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f006b330 con 0x7f86dc005180 2023-10-16T11:05:46.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.187+0000 7f86ecff9640 1 -- 172.21.15.84:0/432988609 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f86e0004b40 con 0x7f86dc005180 2023-10-16T11:05:46.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.188+0000 7f86ecff9640 1 --2- 172.21.15.84:0/432988609 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f86d006af90 0x7f86d006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:46.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.188+0000 7f86ecff9640 1 -- 172.21.15.84:0/432988609 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f86e00891f0 con 0x7f86dc005180 2023-10-16T11:05:46.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.188+0000 7f86e7fff640 1 --2- 172.21.15.84:0/432988609 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f86d006af90 0x7f86d006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:46.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.188+0000 7f86e7fff640 1 --2- 172.21.15.84:0/432988609 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f86d006af90 0x7f86d006d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f86f00c4f60 tx=0x7f86d80079b0 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:46.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.192+0000 7f86ecff9640 1 -- 172.21.15.84:0/432988609 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f86e0054040 con 0x7f86dc005180 2023-10-16T11:05:46.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.426+0000 7f86f4cbb640 1 -- 172.21.15.84:0/432988609 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f86f00b3b40 con 0x7f86dc005180 2023-10-16T11:05:46.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.941+0000 7f86ecff9640 1 -- 172.21.15.84:0/432988609 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v88) v1 ==== 122+0+0 (secure 0 0 0) 0x7f86e0057660 con 0x7f86dc005180 2023-10-16T11:05:46.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.945+0000 7f86f4cbb640 1 -- 172.21.15.84:0/432988609 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f86d006af90 msgr2=0x7f86d006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:46.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.945+0000 7f86f4cbb640 1 --2- 172.21.15.84:0/432988609 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f86d006af90 0x7f86d006d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f86f00c4f60 tx=0x7f86d80079b0 comp rx=0 tx=0).stop 2023-10-16T11:05:46.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.945+0000 7f86f4cbb640 1 -- 172.21.15.84:0/432988609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86dc005180 msgr2=0x7f86f01529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:46.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.945+0000 7f86f4cbb640 1 --2- 172.21.15.84:0/432988609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86dc005180 0x7f86f01529e0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f86e000d7a0 tx=0x7f86e000dc60 comp rx=0 tx=0).stop 2023-10-16T11:05:46.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.945+0000 7f86f4cbb640 1 -- 172.21.15.84:0/432988609 shutdown_connections 2023-10-16T11:05:46.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.945+0000 7f86f4cbb640 1 --2- 172.21.15.84:0/432988609 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f86d006af90 0x7f86d006d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:46.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.945+0000 7f86f4cbb640 1 --2- 172.21.15.84:0/432988609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f00b26e0 0x7f86f0159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:46.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.945+0000 7f86f4cbb640 1 --2- 172.21.15.84:0/432988609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86f00b3ed0 0x7f86f00b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:46.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.945+0000 7f86f4cbb640 1 --2- 172.21.15.84:0/432988609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86dc005180 0x7f86f01529e0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:46.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.945+0000 7f86f4cbb640 1 -- 172.21.15.84:0/432988609 >> 172.21.15.84:0/432988609 conn(0x7f86f00ce6a0 msgr2=0x7f86f00cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:46.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.946+0000 7f86f4cbb640 1 -- 172.21.15.84:0/432988609 shutdown_connections 2023-10-16T11:05:46.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:46.946+0000 7f86f4cbb640 1 -- 172.21.15.84:0/432988609 wait complete. 2023-10-16T11:05:46.968 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T11:05:46.968 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:05:47.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.184+0000 7fdb789c4640 1 Processor -- start 2023-10-16T11:05:47.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.185+0000 7fdb789c4640 1 -- start start 2023-10-16T11:05:47.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.186+0000 7fdb789c4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdb7406c040 0x7fdb7406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:47.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.186+0000 7fdb789c4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb740cda80 0x7fdb7415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:47.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.186+0000 7fdb789c4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7406c960 con 0x7fdb740ad970 2023-10-16T11:05:47.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.186+0000 7fdb789c4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb7415b9d0 con 0x7fdb7406c040 2023-10-16T11:05:47.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.186+0000 7fdb789c4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7415bcb0 con 0x7fdb740cda80 2023-10-16T11:05:47.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.186+0000 7fdb6bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdb740ad970 0x7fdb7415e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34294/0 (socket says 172.21.15.84:34294) 2023-10-16T11:05:47.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.186+0000 7fdb6bfff640 1 -- 172.21.15.84:0/2990633304 learned_addr learned my addr 172.21.15.84:0/2990633304 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:47.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.186+0000 7fdb737fe640 1 --2- 172.21.15.84:0/2990633304 >> v2:172.21.15.157:3300/0 conn(0x7fdb7406c040 0x7fdb7406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:47.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.186+0000 7fdb73fff640 1 --2- 172.21.15.84:0/2990633304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb740cda80 0x7fdb7415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:47.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.187+0000 7fdb72ffd640 1 -- 172.21.15.84:0/2990633304 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1976578494 0 0) 0x7fdb7406c960 con 0x7fdb740ad970 2023-10-16T11:05:47.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.187+0000 7fdb72ffd640 1 -- 172.21.15.84:0/2990633304 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb60003520 con 0x7fdb740ad970 2023-10-16T11:05:47.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.187+0000 7fdb72ffd640 1 -- 172.21.15.84:0/2990633304 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1444786368 0 0) 0x7fdb60003520 con 0x7fdb740ad970 2023-10-16T11:05:47.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.187+0000 7fdb72ffd640 1 -- 172.21.15.84:0/2990633304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb740cda80 msgr2=0x7fdb7415ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:47.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.187+0000 7fdb72ffd640 1 --2- 172.21.15.84:0/2990633304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb740cda80 0x7fdb7415ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.187+0000 7fdb72ffd640 1 -- 172.21.15.84:0/2990633304 >> v2:172.21.15.157:3300/0 conn(0x7fdb7406c040 msgr2=0x7fdb7406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:47.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.187+0000 7fdb72ffd640 1 --2- 172.21.15.84:0/2990633304 >> v2:172.21.15.157:3300/0 conn(0x7fdb7406c040 0x7fdb7406c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.187+0000 7fdb72ffd640 1 -- 172.21.15.84:0/2990633304 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7415bf90 con 0x7fdb740ad970 2023-10-16T11:05:47.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.187+0000 7fdb72ffd640 1 -- 172.21.15.84:0/2990633304 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fdb5c001670 con 0x7fdb740ad970 2023-10-16T11:05:47.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.187+0000 7fdb72ffd640 1 -- 172.21.15.84:0/2990633304 >> v1:172.21.15.84:6789/0 conn(0x7fdb740ad970 legacy=0x7fdb7415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:47.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.187+0000 7fdb72ffd640 1 --2- 172.21.15.84:0/2990633304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb60003970 0x7fdb60003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:47.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.187+0000 7fdb72ffd640 1 --2- 172.21.15.84:0/2990633304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb740cda80 0x7fdb600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:47.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.187+0000 7fdb72ffd640 1 --2- 172.21.15.84:0/2990633304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb60004d00 0x7fdb60005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:47.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.187+0000 7fdb72ffd640 1 -- 172.21.15.84:0/2990633304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7415b9d0 con 0x7fdb60003970 2023-10-16T11:05:47.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.187+0000 7fdb72ffd640 1 -- 172.21.15.84:0/2990633304 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7415bcb0 con 0x7fdb60004d00 2023-10-16T11:05:47.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.187+0000 7fdb72ffd640 1 -- 172.21.15.84:0/2990633304 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6000b980 con 0x7fdb740cda80 2023-10-16T11:05:47.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.187+0000 7fdb737fe640 1 --2- 172.21.15.84:0/2990633304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb60003970 0x7fdb60003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:47.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.188+0000 7fdb737fe640 1 -- 172.21.15.84:0/2990633304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb740cda80 msgr2=0x7fdb600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:47.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.188+0000 7fdb737fe640 1 --2- 172.21.15.84:0/2990633304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb740cda80 0x7fdb600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.188+0000 7fdb73fff640 1 --2- 172.21.15.84:0/2990633304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb60004d00 0x7fdb60005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:47.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.188+0000 7fdb737fe640 1 -- 172.21.15.84:0/2990633304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb60004d00 msgr2=0x7fdb60005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:47.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.188+0000 7fdb737fe640 1 --2- 172.21.15.84:0/2990633304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb60004d00 0x7fdb60005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.188+0000 7fdb737fe640 1 -- 172.21.15.84:0/2990633304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb60003520 con 0x7fdb60003970 2023-10-16T11:05:47.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.188+0000 7fdb73fff640 1 --2- 172.21.15.84:0/2990633304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb60004d00 0x7fdb60005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:47.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.189+0000 7fdb737fe640 1 --2- 172.21.15.84:0/2990633304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb60003970 0x7fdb60003d90 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009ac0 tx=0x7fdb600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98db8d6208d26caa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:47.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.189+0000 7fdb72ffd640 1 -- 172.21.15.84:0/2990633304 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6402c6d0 con 0x7fdb60003970 2023-10-16T11:05:47.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.189+0000 7fdb72ffd640 1 -- 172.21.15.84:0/2990633304 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6402c830 con 0x7fdb60003970 2023-10-16T11:05:47.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.189+0000 7fdb72ffd640 1 -- 172.21.15.84:0/2990633304 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6402cbc0 con 0x7fdb60003970 2023-10-16T11:05:47.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.190+0000 7fdb789c4640 1 -- 172.21.15.84:0/2990633304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb60003970 msgr2=0x7fdb60003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:47.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.190+0000 7fdb789c4640 1 --2- 172.21.15.84:0/2990633304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb60003970 0x7fdb60003d90 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009ac0 tx=0x7fdb600057f0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.190+0000 7fdb73fff640 1 -- 172.21.15.84:0/2990633304 reap_dead start 2023-10-16T11:05:47.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.190+0000 7fdb789c4640 1 -- 172.21.15.84:0/2990633304 shutdown_connections 2023-10-16T11:05:47.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.190+0000 7fdb789c4640 1 -- 172.21.15.84:0/2990633304 >> 172.21.15.84:0/2990633304 conn(0x7fdb740ce6a0 msgr2=0x7fdb740ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:47.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.190+0000 7fdb789c4640 1 -- 172.21.15.84:0/2990633304 shutdown_connections 2023-10-16T11:05:47.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.190+0000 7fdb789c4640 1 -- 172.21.15.84:0/2990633304 wait complete. 2023-10-16T11:05:47.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.192+0000 7fdb789c4640 1 Processor -- start 2023-10-16T11:05:47.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.192+0000 7fdb789c4640 1 -- start start 2023-10-16T11:05:47.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.193+0000 7fdb789c4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb60003970 0x7fdb741ee0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:47.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.193+0000 7fdb789c4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb74158790 0x7fdb740ac900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:47.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.193+0000 7fdb789c4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb740ace40 0x7fdb7415a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:47.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.193+0000 7fdb789c4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb740ad3c0 con 0x7fdb74158790 2023-10-16T11:05:47.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.193+0000 7fdb789c4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb740c6c50 con 0x7fdb740ace40 2023-10-16T11:05:47.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.193+0000 7fdb789c4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb741ee5e0 con 0x7fdb60003970 2023-10-16T11:05:47.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.193+0000 7fdb6bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb74158790 0x7fdb740ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:47.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.193+0000 7fdb6bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb74158790 0x7fdb740ac900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54690/0 (socket says 172.21.15.84:54690) 2023-10-16T11:05:47.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.193+0000 7fdb6bfff640 1 -- 172.21.15.84:0/3048081544 learned_addr learned my addr 172.21.15.84:0/3048081544 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:47.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.194+0000 7fdb73fff640 1 --2- 172.21.15.84:0/3048081544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb740ace40 0x7fdb7415a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:47.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.194+0000 7fdb737fe640 1 --2- 172.21.15.84:0/3048081544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb60003970 0x7fdb741ee0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:47.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.194+0000 7fdb6bfff640 1 -- 172.21.15.84:0/3048081544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb60003970 msgr2=0x7fdb741ee0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:47.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.194+0000 7fdb6bfff640 1 --2- 172.21.15.84:0/3048081544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb60003970 0x7fdb741ee0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.194+0000 7fdb6bfff640 1 -- 172.21.15.84:0/3048081544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb740ace40 msgr2=0x7fdb7415a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:47.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.194+0000 7fdb6bfff640 1 --2- 172.21.15.84:0/3048081544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb740ace40 0x7fdb7415a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.194+0000 7fdb6bfff640 1 -- 172.21.15.84:0/3048081544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb740c6ef0 con 0x7fdb74158790 2023-10-16T11:05:47.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.194+0000 7fdb73fff640 1 --2- 172.21.15.84:0/3048081544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb740ace40 0x7fdb7415a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:47.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.194+0000 7fdb6bfff640 1 --2- 172.21.15.84:0/3048081544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb74158790 0x7fdb740ac900 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00c690 tx=0x7fdb5c00cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:47.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.194+0000 7fdb70ff9640 1 -- 172.21.15.84:0/3048081544 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c0072d0 con 0x7fdb74158790 2023-10-16T11:05:47.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.194+0000 7fdb70ff9640 1 -- 172.21.15.84:0/3048081544 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5c007450 con 0x7fdb74158790 2023-10-16T11:05:47.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.194+0000 7fdb70ff9640 1 -- 172.21.15.84:0/3048081544 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c0047e0 con 0x7fdb74158790 2023-10-16T11:05:47.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.194+0000 7fdb789c4640 1 -- 172.21.15.84:0/3048081544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb740c71d0 con 0x7fdb74158790 2023-10-16T11:05:47.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.195+0000 7fdb789c4640 1 -- 172.21.15.84:0/3048081544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb740b0bc0 con 0x7fdb74158790 2023-10-16T11:05:47.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.196+0000 7fdb789c4640 1 -- 172.21.15.84:0/3048081544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb7406b330 con 0x7fdb74158790 2023-10-16T11:05:47.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.197+0000 7fdb70ff9640 1 -- 172.21.15.84:0/3048081544 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdb5c004960 con 0x7fdb74158790 2023-10-16T11:05:47.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.197+0000 7fdb70ff9640 1 --2- 172.21.15.84:0/3048081544 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fdb5406afc0 0x7fdb5406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:47.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.198+0000 7fdb70ff9640 1 -- 172.21.15.84:0/3048081544 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fdb5c088b10 con 0x7fdb74158790 2023-10-16T11:05:47.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.198+0000 7fdb737fe640 1 --2- 172.21.15.84:0/3048081544 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fdb5406afc0 0x7fdb5406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:47.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.198+0000 7fdb737fe640 1 --2- 172.21.15.84:0/3048081544 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fdb5406afc0 0x7fdb5406d480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdb64036000 tx=0x7fdb64030000 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:47.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.202+0000 7fdb70ff9640 1 -- 172.21.15.84:0/3048081544 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdb5c053a80 con 0x7fdb74158790 2023-10-16T11:05:47.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.431+0000 7fdb789c4640 1 -- 172.21.15.84:0/3048081544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb7404e9b0 con 0x7fdb74158790 2023-10-16T11:05:47.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.432+0000 7fdb70ff9640 1 -- 172.21.15.84:0/3048081544 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fdb5c018020 con 0x7fdb74158790 2023-10-16T11:05:47.434 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:47.434 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":88,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6718,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3060266169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3060266169},{"type":"v1","addr":"172.21.15.157:6834","nonce":3060266169}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":6721,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2965939103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2965939103},{"type":"v1","addr":"172.21.15.157:6835","nonce":2965939103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9684,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3677909298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3677909298},{"type":"v1","addr":"172.21.15.84:6835","nonce":3677909298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:05:43.341956+0000","modified":"2023-10-16T11:05:46.428445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9687},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9687":{"gid":9687,"name":"c","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/424056375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":424056375},{"type":"v1","addr":"172.21.15.84:6837","nonce":424056375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-16T11:05:47.434 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 88 2023-10-16T11:05:47.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.434+0000 7fdb789c4640 1 -- 172.21.15.84:0/3048081544 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fdb5406afc0 msgr2=0x7fdb5406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:47.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.434+0000 7fdb789c4640 1 --2- 172.21.15.84:0/3048081544 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fdb5406afc0 0x7fdb5406d480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdb64036000 tx=0x7fdb64030000 comp rx=0 tx=0).stop 2023-10-16T11:05:47.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.434+0000 7fdb789c4640 1 -- 172.21.15.84:0/3048081544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb74158790 msgr2=0x7fdb740ac900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:47.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.434+0000 7fdb789c4640 1 --2- 172.21.15.84:0/3048081544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb74158790 0x7fdb740ac900 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00c690 tx=0x7fdb5c00cb50 comp rx=0 tx=0).stop 2023-10-16T11:05:47.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.434+0000 7fdb789c4640 1 -- 172.21.15.84:0/3048081544 shutdown_connections 2023-10-16T11:05:47.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.434+0000 7fdb789c4640 1 --2- 172.21.15.84:0/3048081544 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fdb5406afc0 0x7fdb5406d480 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.434+0000 7fdb789c4640 1 --2- 172.21.15.84:0/3048081544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb740ace40 0x7fdb7415a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.434+0000 7fdb789c4640 1 --2- 172.21.15.84:0/3048081544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb74158790 0x7fdb740ac900 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.434+0000 7fdb789c4640 1 --2- 172.21.15.84:0/3048081544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb60003970 0x7fdb741ee0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.434+0000 7fdb789c4640 1 -- 172.21.15.84:0/3048081544 >> 172.21.15.84:0/3048081544 conn(0x7fdb740ce6a0 msgr2=0x7fdb7415dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:47.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.434+0000 7fdb789c4640 1 -- 172.21.15.84:0/3048081544 shutdown_connections 2023-10-16T11:05:47.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.435+0000 7fdb789c4640 1 -- 172.21.15.84:0/3048081544 wait complete. 2023-10-16T11:05:47.455 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:05:47.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.650+0000 7f197b378640 1 Processor -- start 2023-10-16T11:05:47.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.650+0000 7f197b378640 1 -- start start 2023-10-16T11:05:47.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.651+0000 7f197b378640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1974159b20 0x7f197406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:47.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.652+0000 7f197b378640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197406f7f0 con 0x7f19740b0b20 2023-10-16T11:05:47.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.652+0000 7f197b378640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197406c5e0 con 0x7f1974159f00 2023-10-16T11:05:47.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.652+0000 7f197b378640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197406c8c0 con 0x7f1974159b20 2023-10-16T11:05:47.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.652+0000 7f1979915640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f19740b0b20 0x7f197406fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34298/0 (socket says 172.21.15.84:34298) 2023-10-16T11:05:47.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.652+0000 7f1979915640 1 -- 172.21.15.84:0/4140855546 learned_addr learned my addr 172.21.15.84:0/4140855546 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:47.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.652+0000 7f1979114640 1 --2- 172.21.15.84:0/4140855546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1974159b20 0x7f197406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:47.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.653+0000 7f196bfff640 1 -- 172.21.15.84:0/4140855546 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966538160 0 0) 0x7f197406f7f0 con 0x7f19740b0b20 2023-10-16T11:05:47.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.653+0000 7f196bfff640 1 -- 172.21.15.84:0/4140855546 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1960003520 con 0x7f19740b0b20 2023-10-16T11:05:47.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.653+0000 7f1979114640 1 -- 172.21.15.84:0/4140855546 >> v1:172.21.15.157:6789/0 conn(0x7f1974159f00 legacy=0x7f19740b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:47.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.653+0000 7f1979114640 1 -- 172.21.15.84:0/4140855546 >> v1:172.21.15.84:6789/0 conn(0x7f19740b0b20 legacy=0x7f197406fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:47.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.653+0000 7f196bfff640 1 -- 172.21.15.84:0/4140855546 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2823253740 0 0) 0x7f1960003520 con 0x7f19740b0b20 2023-10-16T11:05:47.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.653+0000 7f1979114640 1 -- 172.21.15.84:0/4140855546 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197406cba0 con 0x7f1974159b20 2023-10-16T11:05:47.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.654+0000 7f1979114640 1 --2- 172.21.15.84:0/4140855546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1974159b20 0x7f197406bbe0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1970009dc0 tx=0x7f197002bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45c1e82de3ddb3cd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:47.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.655+0000 7f196bfff640 1 -- 172.21.15.84:0/4140855546 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c001670 con 0x7f1974159b20 2023-10-16T11:05:47.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.655+0000 7f196bfff640 1 -- 172.21.15.84:0/4140855546 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197002c910 con 0x7f1974159b20 2023-10-16T11:05:47.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.655+0000 7f196bfff640 1 -- 172.21.15.84:0/4140855546 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197002cca0 con 0x7f1974159b20 2023-10-16T11:05:47.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.655+0000 7f197b378640 1 -- 172.21.15.84:0/4140855546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1974159b20 msgr2=0x7f197406bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:47.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.655+0000 7f197b378640 1 --2- 172.21.15.84:0/4140855546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1974159b20 0x7f197406bbe0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1970009dc0 tx=0x7f197002bde0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.656+0000 7f197b378640 1 -- 172.21.15.84:0/4140855546 shutdown_connections 2023-10-16T11:05:47.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.656+0000 7f197b378640 1 --2- 172.21.15.84:0/4140855546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1974159b20 0x7f197406bbe0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.656+0000 7f197b378640 1 -- 172.21.15.84:0/4140855546 >> 172.21.15.84:0/4140855546 conn(0x7f19740cb410 msgr2=0x7f19740cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:47.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.656+0000 7f197b378640 1 -- 172.21.15.84:0/4140855546 shutdown_connections 2023-10-16T11:05:47.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.656+0000 7f197b378640 1 -- 172.21.15.84:0/4140855546 wait complete. 2023-10-16T11:05:47.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.657+0000 7f197b378640 1 Processor -- start 2023-10-16T11:05:47.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.657+0000 7f197b378640 1 -- start start 2023-10-16T11:05:47.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.658+0000 7f197b378640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19740b0b20 0x7f197415a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:47.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.658+0000 7f197b378640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1974159b20 0x7f19741f1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:47.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.658+0000 7f197b378640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1974159f00 0x7f197415bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:47.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.658+0000 7f197b378640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19741f1a60 con 0x7f1974159f00 2023-10-16T11:05:47.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.658+0000 7f197b378640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197415b5b0 con 0x7f19740b0b20 2023-10-16T11:05:47.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.658+0000 7f197b378640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197415a9a0 con 0x7f1974159b20 2023-10-16T11:05:47.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.659+0000 7f1979915640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1974159f00 0x7f197415bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:47.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.659+0000 7f1979915640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1974159f00 0x7f197415bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54694/0 (socket says 172.21.15.84:54694) 2023-10-16T11:05:47.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.659+0000 7f1979915640 1 -- 172.21.15.84:0/1999643071 learned_addr learned my addr 172.21.15.84:0/1999643071 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:47.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.659+0000 7f1979114640 1 --2- 172.21.15.84:0/1999643071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19740b0b20 0x7f197415a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:47.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.659+0000 7f1978913640 1 --2- 172.21.15.84:0/1999643071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1974159b20 0x7f19741f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:47.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.659+0000 7f1979915640 1 -- 172.21.15.84:0/1999643071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1974159b20 msgr2=0x7f19741f1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.659+0000 7f1979915640 1 --2- 172.21.15.84:0/1999643071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1974159b20 0x7f19741f1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.659+0000 7f1979915640 1 -- 172.21.15.84:0/1999643071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19740b0b20 msgr2=0x7f197415a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.659+0000 7f1979915640 1 --2- 172.21.15.84:0/1999643071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19740b0b20 0x7f197415a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.659+0000 7f1979915640 1 -- 172.21.15.84:0/1999643071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1970009a70 con 0x7f1974159f00 2023-10-16T11:05:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.659+0000 7f1978913640 1 --2- 172.21.15.84:0/1999643071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1974159b20 0x7f19741f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.659+0000 7f1979915640 1 --2- 172.21.15.84:0/1999643071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1974159f00 0x7f197415bef0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f196c00c580 tx=0x7f196c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:47.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.660+0000 7f1969ffb640 1 -- 172.21.15.84:0/1999643071 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c005ce0 con 0x7f1974159f00 2023-10-16T11:05:47.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.660+0000 7f1969ffb640 1 -- 172.21.15.84:0/1999643071 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196c005e60 con 0x7f1974159f00 2023-10-16T11:05:47.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.660+0000 7f1969ffb640 1 -- 172.21.15.84:0/1999643071 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c0072b0 con 0x7f1974159f00 2023-10-16T11:05:47.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.660+0000 7f197b378640 1 -- 172.21.15.84:0/1999643071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197415ac80 con 0x7f1974159f00 2023-10-16T11:05:47.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.660+0000 7f197b378640 1 -- 172.21.15.84:0/1999643071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197415b240 con 0x7f1974159f00 2023-10-16T11:05:47.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.661+0000 7f197b378640 1 -- 172.21.15.84:0/1999643071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197415b760 con 0x7f1974159f00 2023-10-16T11:05:47.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.662+0000 7f1969ffb640 1 -- 172.21.15.84:0/1999643071 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f196c007410 con 0x7f1974159f00 2023-10-16T11:05:47.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.663+0000 7f1969ffb640 1 --2- 172.21.15.84:0/1999643071 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f195806af90 0x7f195806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:47.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.663+0000 7f1969ffb640 1 -- 172.21.15.84:0/1999643071 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f196c089470 con 0x7f1974159f00 2023-10-16T11:05:47.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.663+0000 7f1979114640 1 --2- 172.21.15.84:0/1999643071 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f195806af90 0x7f195806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:47.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.664+0000 7f1979114640 1 --2- 172.21.15.84:0/1999643071 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f195806af90 0x7f195806d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1970036000 tx=0x7f1970030000 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:47.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.667+0000 7f1969ffb640 1 -- 172.21.15.84:0/1999643071 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f197415b760 con 0x7f1974159f00 2023-10-16T11:05:47.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.862+0000 7f197b378640 1 -- 172.21.15.84:0/1999643071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f197415b720 con 0x7f1974159f00 2023-10-16T11:05:47.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.863+0000 7f1969ffb640 1 -- 172.21.15.84:0/1999643071 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16599 (secure 0 0 0) 0x7f196c0543e0 con 0x7f1974159f00 2023-10-16T11:05:47.865 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:47.866 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":171,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:05:44.900505+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:05:41.217474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-16T11:05:42.387743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:05:47.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.866+0000 7f197b378640 1 -- 172.21.15.84:0/1999643071 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f195806af90 msgr2=0x7f195806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:47.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.866+0000 7f197b378640 1 --2- 172.21.15.84:0/1999643071 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f195806af90 0x7f195806d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1970036000 tx=0x7f1970030000 comp rx=0 tx=0).stop 2023-10-16T11:05:47.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.866+0000 7f197b378640 1 -- 172.21.15.84:0/1999643071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1974159f00 msgr2=0x7f197415bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:47.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.866+0000 7f197b378640 1 --2- 172.21.15.84:0/1999643071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1974159f00 0x7f197415bef0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f196c00c580 tx=0x7f196c00ca40 comp rx=0 tx=0).stop 2023-10-16T11:05:47.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.866+0000 7f197b378640 1 -- 172.21.15.84:0/1999643071 shutdown_connections 2023-10-16T11:05:47.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.866+0000 7f197b378640 1 --2- 172.21.15.84:0/1999643071 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f195806af90 0x7f195806d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.866+0000 7f197b378640 1 --2- 172.21.15.84:0/1999643071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1974159f00 0x7f197415bef0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.866+0000 7f197b378640 1 --2- 172.21.15.84:0/1999643071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1974159b20 0x7f19741f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.866+0000 7f197b378640 1 --2- 172.21.15.84:0/1999643071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19740b0b20 0x7f197415a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:47.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.866+0000 7f197b378640 1 -- 172.21.15.84:0/1999643071 >> 172.21.15.84:0/1999643071 conn(0x7f19740cb410 msgr2=0x7f19740cc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:47.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.866+0000 7f197b378640 1 -- 172.21.15.84:0/1999643071 shutdown_connections 2023-10-16T11:05:47.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:47.866+0000 7f197b378640 1 -- 172.21.15.84:0/1999643071 wait complete. 2023-10-16T11:05:47.887 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T11:05:47.887 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:05:48.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.089+0000 7fd278178640 1 Processor -- start 2023-10-16T11:05:48.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.089+0000 7fd278178640 1 -- start start 2023-10-16T11:05:48.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.091+0000 7fd278178640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd27015aaa0 0x7fd27015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:48.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.091+0000 7fd278178640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd27015ae80 0x7fd2700cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:48.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.091+0000 7fd278178640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd27015ed60 con 0x7fd2700ccaa0 2023-10-16T11:05:48.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.091+0000 7fd278178640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd27006b810 con 0x7fd27015ae80 2023-10-16T11:05:48.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.091+0000 7fd278178640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2700707e0 con 0x7fd27015aaa0 2023-10-16T11:05:48.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.091+0000 7fd275713640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd27015ae80 0x7fd2700cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:48.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.091+0000 7fd275713640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd27015ae80 0x7fd2700cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54698/0 (socket says 172.21.15.84:54698) 2023-10-16T11:05:48.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.091+0000 7fd275713640 1 -- 172.21.15.84:0/57964229 learned_addr learned my addr 172.21.15.84:0/57964229 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:48.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.091+0000 7fd276715640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd2700ccaa0 0x7fd270070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34306/0 (socket says 172.21.15.84:34306) 2023-10-16T11:05:48.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.091+0000 7fd275f14640 1 --2- 172.21.15.84:0/57964229 >> v2:172.21.15.157:3300/0 conn(0x7fd27015aaa0 0x7fd27015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:48.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.091+0000 7fd275713640 1 -- 172.21.15.84:0/57964229 >> v2:172.21.15.157:3300/0 conn(0x7fd27015aaa0 msgr2=0x7fd27015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:48.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.092+0000 7fd275713640 1 --2- 172.21.15.84:0/57964229 >> v2:172.21.15.157:3300/0 conn(0x7fd27015aaa0 0x7fd27015e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.092+0000 7fd275713640 1 -- 172.21.15.84:0/57964229 >> v1:172.21.15.84:6789/0 conn(0x7fd2700ccaa0 legacy=0x7fd270070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:48.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.092+0000 7fd274f12640 1 -- 172.21.15.84:0/57964229 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3819600184 0 0) 0x7fd27015ed60 con 0x7fd2700ccaa0 2023-10-16T11:05:48.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.092+0000 7fd275713640 1 -- 172.21.15.84:0/57964229 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27015ef90 con 0x7fd27015ae80 2023-10-16T11:05:48.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.093+0000 7fd275713640 1 --2- 172.21.15.84:0/57964229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd27015ae80 0x7fd2700cc560 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fd2700cd990 tx=0x7fd26002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86c694cc93b1e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:48.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.093+0000 7fd274f12640 1 -- 172.21.15.84:0/57964229 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd260030860 con 0x7fd27015ae80 2023-10-16T11:05:48.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.093+0000 7fd274f12640 1 -- 172.21.15.84:0/57964229 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2600309c0 con 0x7fd27015ae80 2023-10-16T11:05:48.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.093+0000 7fd274f12640 1 -- 172.21.15.84:0/57964229 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd260030d50 con 0x7fd27015ae80 2023-10-16T11:05:48.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.093+0000 7fd278178640 1 -- 172.21.15.84:0/57964229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd27015ae80 msgr2=0x7fd2700cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:48.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.093+0000 7fd278178640 1 --2- 172.21.15.84:0/57964229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd27015ae80 0x7fd2700cc560 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fd2700cd990 tx=0x7fd26002b9a0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.093+0000 7fd278178640 1 -- 172.21.15.84:0/57964229 shutdown_connections 2023-10-16T11:05:48.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.093+0000 7fd278178640 1 --2- 172.21.15.84:0/57964229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd27015ae80 0x7fd2700cc560 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.093+0000 7fd278178640 1 --2- 172.21.15.84:0/57964229 >> v2:172.21.15.157:3300/0 conn(0x7fd27015aaa0 0x7fd27015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.093+0000 7fd278178640 1 -- 172.21.15.84:0/57964229 >> 172.21.15.84:0/57964229 conn(0x7fd2700cb410 msgr2=0x7fd2700cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:48.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.093+0000 7fd278178640 1 -- 172.21.15.84:0/57964229 shutdown_connections 2023-10-16T11:05:48.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.093+0000 7fd278178640 1 -- 172.21.15.84:0/57964229 wait complete. 2023-10-16T11:05:48.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.095+0000 7fd278178640 1 Processor -- start 2023-10-16T11:05:48.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.095+0000 7fd278178640 1 -- start start 2023-10-16T11:05:48.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.096+0000 7fd278178640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2700ccaa0 0x7fd270159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:48.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.096+0000 7fd278178640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd27015aaa0 0x7fd27015b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:48.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.096+0000 7fd278178640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd27015ae80 0x7fd2700c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:48.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.096+0000 7fd278178640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27015d0f0 con 0x7fd27015aaa0 2023-10-16T11:05:48.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.096+0000 7fd278178640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27015d3d0 con 0x7fd2700ccaa0 2023-10-16T11:05:48.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.096+0000 7fd278178640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27015a520 con 0x7fd27015ae80 2023-10-16T11:05:48.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.096+0000 7fd275713640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd27015aaa0 0x7fd27015b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:48.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.096+0000 7fd275713640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd27015aaa0 0x7fd27015b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54712/0 (socket says 172.21.15.84:54712) 2023-10-16T11:05:48.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.096+0000 7fd275713640 1 -- 172.21.15.84:0/2335435445 learned_addr learned my addr 172.21.15.84:0/2335435445 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:48.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.097+0000 7fd275713640 1 -- 172.21.15.84:0/2335435445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd27015ae80 msgr2=0x7fd2700c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:48.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.097+0000 7fd275713640 1 --2- 172.21.15.84:0/2335435445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd27015ae80 0x7fd2700c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.097+0000 7fd275f14640 1 --2- 172.21.15.84:0/2335435445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2700ccaa0 0x7fd270159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:48.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.097+0000 7fd275713640 1 -- 172.21.15.84:0/2335435445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2700ccaa0 msgr2=0x7fd270159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:48.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.097+0000 7fd275713640 1 --2- 172.21.15.84:0/2335435445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2700ccaa0 0x7fd270159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.097+0000 7fd275713640 1 -- 172.21.15.84:0/2335435445 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2600094d0 con 0x7fd27015aaa0 2023-10-16T11:05:48.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.097+0000 7fd275713640 1 --2- 172.21.15.84:0/2335435445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd27015aaa0 0x7fd27015b2b0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fd260000c00 tx=0x7fd2600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:48.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.097+0000 7fd2667fc640 1 -- 172.21.15.84:0/2335435445 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26003e070 con 0x7fd27015aaa0 2023-10-16T11:05:48.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.097+0000 7fd2667fc640 1 -- 172.21.15.84:0/2335435445 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd260004030 con 0x7fd27015aaa0 2023-10-16T11:05:48.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.097+0000 7fd2667fc640 1 -- 172.21.15.84:0/2335435445 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2600043c0 con 0x7fd27015aaa0 2023-10-16T11:05:48.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.097+0000 7fd278178640 1 -- 172.21.15.84:0/2335435445 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27015b7f0 con 0x7fd27015aaa0 2023-10-16T11:05:48.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.098+0000 7fd278178640 1 -- 172.21.15.84:0/2335435445 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27015bd70 con 0x7fd27015aaa0 2023-10-16T11:05:48.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.099+0000 7fd278178640 1 -- 172.21.15.84:0/2335435445 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2700734d0 con 0x7fd27015aaa0 2023-10-16T11:05:48.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.099+0000 7fd2667fc640 1 -- 172.21.15.84:0/2335435445 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd260042070 con 0x7fd27015aaa0 2023-10-16T11:05:48.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.100+0000 7fd2667fc640 1 --2- 172.21.15.84:0/2335435445 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd25406afc0 0x7fd25406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:48.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.100+0000 7fd2667fc640 1 -- 172.21.15.84:0/2335435445 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd2600ab870 con 0x7fd27015aaa0 2023-10-16T11:05:48.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.101+0000 7fd275f14640 1 --2- 172.21.15.84:0/2335435445 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd25406afc0 0x7fd25406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:48.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.101+0000 7fd275f14640 1 --2- 172.21.15.84:0/2335435445 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd25406afc0 0x7fd25406d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd26c006fd0 tx=0x7fd26c008040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:48.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.104+0000 7fd2667fc640 1 -- 172.21.15.84:0/2335435445 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd260076640 con 0x7fd27015aaa0 2023-10-16T11:05:48.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.280+0000 7fd278178640 1 -- 172.21.15.84:0/2335435445 --> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd27015f550 con 0x7fd25406afc0 2023-10-16T11:05:48.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.296+0000 7fd2667fc640 1 -- 172.21.15.84:0/2335435445 <== mgr.6631 v2:172.21.15.84:6832/2442400184 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286866 (secure 0 0 0) 0x7fd26c009d60 con 0x7fd25406afc0 2023-10-16T11:05:48.302 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:48.303 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:05:48.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.302+0000 7fd278178640 1 -- 172.21.15.84:0/2335435445 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd25406afc0 msgr2=0x7fd25406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:48.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.302+0000 7fd278178640 1 --2- 172.21.15.84:0/2335435445 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd25406afc0 0x7fd25406d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd26c006fd0 tx=0x7fd26c008040 comp rx=0 tx=0).stop 2023-10-16T11:05:48.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.302+0000 7fd278178640 1 -- 172.21.15.84:0/2335435445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd27015aaa0 msgr2=0x7fd27015b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:48.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.302+0000 7fd278178640 1 --2- 172.21.15.84:0/2335435445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd27015aaa0 0x7fd27015b2b0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fd260000c00 tx=0x7fd2600046c0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.302+0000 7fd278178640 1 -- 172.21.15.84:0/2335435445 shutdown_connections 2023-10-16T11:05:48.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.302+0000 7fd278178640 1 --2- 172.21.15.84:0/2335435445 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fd25406afc0 0x7fd25406d480 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.302+0000 7fd278178640 1 --2- 172.21.15.84:0/2335435445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd27015ae80 0x7fd2700c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.302+0000 7fd278178640 1 --2- 172.21.15.84:0/2335435445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd27015aaa0 0x7fd27015b2b0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.302+0000 7fd278178640 1 --2- 172.21.15.84:0/2335435445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2700ccaa0 0x7fd270159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.302+0000 7fd278178640 1 -- 172.21.15.84:0/2335435445 >> 172.21.15.84:0/2335435445 conn(0x7fd2700cb410 msgr2=0x7fd2700cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:48.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.303+0000 7fd278178640 1 -- 172.21.15.84:0/2335435445 shutdown_connections 2023-10-16T11:05:48.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.303+0000 7fd278178640 1 -- 172.21.15.84:0/2335435445 wait complete. 2023-10-16T11:05:48.322 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-16T11:05:48.183429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1504,"num_read_kb":3026,"num_write":485,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2728,"ondisk_log_size":2728,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":404776,"kb_used_data":17896,"kb_used_omap":12,"kb_used_meta":386739,"kb_avail":754569944,"statfs":{"total":773094113280,"available":772679622656,"internally_reserved":0,"allocated":18325504,"data_stored":9409548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":396021305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001343"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.345286+0000","last_change":"2023-10-16T11:05:43.939537+0000","last_active":"2023-10-16T11:05:45.345286+0000","last_peered":"2023-10-16T11:05:45.345286+0000","last_clean":"2023-10-16T11:05:45.345286+0000","last_became_active":"2023-10-16T11:05:43.939272+0000","last_became_peered":"2023-10-16T11:05:43.939272+0000","last_unstale":"2023-10-16T11:05:45.345286+0000","last_undegraded":"2023-10-16T11:05:45.345286+0000","last_fullsized":"2023-10-16T11:05:45.345286+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:55.821423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344015+0000","last_change":"2023-10-16T11:05:43.933909+0000","last_active":"2023-10-16T11:05:45.344015+0000","last_peered":"2023-10-16T11:05:45.344015+0000","last_clean":"2023-10-16T11:05:45.344015+0000","last_became_active":"2023-10-16T11:05:43.933264+0000","last_became_peered":"2023-10-16T11:05:43.933264+0000","last_unstale":"2023-10-16T11:05:45.344015+0000","last_undegraded":"2023-10-16T11:05:45.344015+0000","last_fullsized":"2023-10-16T11:05:45.344015+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:41.082759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.347016+0000","last_change":"2023-10-16T11:05:43.930945+0000","last_active":"2023-10-16T11:05:45.347016+0000","last_peered":"2023-10-16T11:05:45.347016+0000","last_clean":"2023-10-16T11:05:45.347016+0000","last_became_active":"2023-10-16T11:05:43.928911+0000","last_became_peered":"2023-10-16T11:05:43.928911+0000","last_unstale":"2023-10-16T11:05:45.347016+0000","last_undegraded":"2023-10-16T11:05:45.347016+0000","last_fullsized":"2023-10-16T11:05:45.347016+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:17:27.802465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.919486+0000","last_change":"2023-10-16T11:05:43.948973+0000","last_active":"2023-10-16T11:05:44.919486+0000","last_peered":"2023-10-16T11:05:44.919486+0000","last_clean":"2023-10-16T11:05:44.919486+0000","last_became_active":"2023-10-16T11:05:43.948685+0000","last_became_peered":"2023-10-16T11:05:43.948685+0000","last_unstale":"2023-10-16T11:05:44.919486+0000","last_undegraded":"2023-10-16T11:05:44.919486+0000","last_fullsized":"2023-10-16T11:05:44.919486+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:22:12.554802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343973+0000","last_change":"2023-10-16T11:05:43.944009+0000","last_active":"2023-10-16T11:05:45.343973+0000","last_peered":"2023-10-16T11:05:45.343973+0000","last_clean":"2023-10-16T11:05:45.343973+0000","last_became_active":"2023-10-16T11:05:43.942783+0000","last_became_peered":"2023-10-16T11:05:43.942783+0000","last_unstale":"2023-10-16T11:05:45.343973+0000","last_undegraded":"2023-10-16T11:05:45.343973+0000","last_fullsized":"2023-10-16T11:05:45.343973+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:24.594901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343340+0000","last_change":"2023-10-16T11:05:43.916350+0000","last_active":"2023-10-16T11:05:45.343340+0000","last_peered":"2023-10-16T11:05:45.343340+0000","last_clean":"2023-10-16T11:05:45.343340+0000","last_became_active":"2023-10-16T11:05:43.915708+0000","last_became_peered":"2023-10-16T11:05:43.915708+0000","last_unstale":"2023-10-16T11:05:45.343340+0000","last_undegraded":"2023-10-16T11:05:45.343340+0000","last_fullsized":"2023-10-16T11:05:45.343340+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:17.978137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.920347+0000","last_change":"2023-10-16T11:05:43.937982+0000","last_active":"2023-10-16T11:05:44.920347+0000","last_peered":"2023-10-16T11:05:44.920347+0000","last_clean":"2023-10-16T11:05:44.920347+0000","last_became_active":"2023-10-16T11:05:43.936784+0000","last_became_peered":"2023-10-16T11:05:43.936784+0000","last_unstale":"2023-10-16T11:05:44.920347+0000","last_undegraded":"2023-10-16T11:05:44.920347+0000","last_fullsized":"2023-10-16T11:05:44.920347+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:55:56.309108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343054+0000","last_change":"2023-10-16T11:05:43.937581+0000","last_active":"2023-10-16T11:05:45.343054+0000","last_peered":"2023-10-16T11:05:45.343054+0000","last_clean":"2023-10-16T11:05:45.343054+0000","last_became_active":"2023-10-16T11:05:43.937324+0000","last_became_peered":"2023-10-16T11:05:43.937324+0000","last_unstale":"2023-10-16T11:05:45.343054+0000","last_undegraded":"2023-10-16T11:05:45.343054+0000","last_fullsized":"2023-10-16T11:05:45.343054+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:43:05.123820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343345+0000","last_change":"2023-10-16T11:05:43.932617+0000","last_active":"2023-10-16T11:05:45.343345+0000","last_peered":"2023-10-16T11:05:45.343345+0000","last_clean":"2023-10-16T11:05:45.343345+0000","last_became_active":"2023-10-16T11:05:43.932064+0000","last_became_peered":"2023-10-16T11:05:43.932064+0000","last_unstale":"2023-10-16T11:05:45.343345+0000","last_undegraded":"2023-10-16T11:05:45.343345+0000","last_fullsized":"2023-10-16T11:05:45.343345+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:28.431551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343948+0000","last_change":"2023-10-16T11:05:43.941441+0000","last_active":"2023-10-16T11:05:45.343948+0000","last_peered":"2023-10-16T11:05:45.343948+0000","last_clean":"2023-10-16T11:05:45.343948+0000","last_became_active":"2023-10-16T11:05:43.939827+0000","last_became_peered":"2023-10-16T11:05:43.939827+0000","last_unstale":"2023-10-16T11:05:45.343948+0000","last_undegraded":"2023-10-16T11:05:45.343948+0000","last_fullsized":"2023-10-16T11:05:45.343948+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:28.255181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343897+0000","last_change":"2023-10-16T11:05:43.927590+0000","last_active":"2023-10-16T11:05:45.343897+0000","last_peered":"2023-10-16T11:05:45.343897+0000","last_clean":"2023-10-16T11:05:45.343897+0000","last_became_active":"2023-10-16T11:05:43.927181+0000","last_became_peered":"2023-10-16T11:05:43.927181+0000","last_unstale":"2023-10-16T11:05:45.343897+0000","last_undegraded":"2023-10-16T11:05:45.343897+0000","last_fullsized":"2023-10-16T11:05:45.343897+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:35.114622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.345612+0000","last_change":"2023-10-16T11:05:43.942366+0000","last_active":"2023-10-16T11:05:45.345612+0000","last_peered":"2023-10-16T11:05:45.345612+0000","last_clean":"2023-10-16T11:05:45.345612+0000","last_became_active":"2023-10-16T11:05:43.940902+0000","last_became_peered":"2023-10-16T11:05:43.940902+0000","last_unstale":"2023-10-16T11:05:45.345612+0000","last_undegraded":"2023-10-16T11:05:45.345612+0000","last_fullsized":"2023-10-16T11:05:45.345612+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:01.682559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344375+0000","last_change":"2023-10-16T11:05:43.945639+0000","last_active":"2023-10-16T11:05:45.344375+0000","last_peered":"2023-10-16T11:05:45.344375+0000","last_clean":"2023-10-16T11:05:45.344375+0000","last_became_active":"2023-10-16T11:05:43.941960+0000","last_became_peered":"2023-10-16T11:05:43.941960+0000","last_unstale":"2023-10-16T11:05:45.344375+0000","last_undegraded":"2023-10-16T11:05:45.344375+0000","last_fullsized":"2023-10-16T11:05:45.344375+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:21:09.268813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.919247+0000","last_change":"2023-10-16T11:05:43.948919+0000","last_active":"2023-10-16T11:05:44.919247+0000","last_peered":"2023-10-16T11:05:44.919247+0000","last_clean":"2023-10-16T11:05:44.919247+0000","last_became_active":"2023-10-16T11:05:43.948601+0000","last_became_peered":"2023-10-16T11:05:43.948601+0000","last_unstale":"2023-10-16T11:05:44.919247+0000","last_undegraded":"2023-10-16T11:05:44.919247+0000","last_fullsized":"2023-10-16T11:05:44.919247+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:05.610634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343134+0000","last_change":"2023-10-16T11:05:43.945422+0000","last_active":"2023-10-16T11:05:45.343134+0000","last_peered":"2023-10-16T11:05:45.343134+0000","last_clean":"2023-10-16T11:05:45.343134+0000","last_became_active":"2023-10-16T11:05:43.944371+0000","last_became_peered":"2023-10-16T11:05:43.944371+0000","last_unstale":"2023-10-16T11:05:45.343134+0000","last_undegraded":"2023-10-16T11:05:45.343134+0000","last_fullsized":"2023-10-16T11:05:45.343134+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:04:03.115562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.919222+0000","last_change":"2023-10-16T11:05:43.928395+0000","last_active":"2023-10-16T11:05:44.919222+0000","last_peered":"2023-10-16T11:05:44.919222+0000","last_clean":"2023-10-16T11:05:44.919222+0000","last_became_active":"2023-10-16T11:05:43.928096+0000","last_became_peered":"2023-10-16T11:05:43.928096+0000","last_unstale":"2023-10-16T11:05:44.919222+0000","last_undegraded":"2023-10-16T11:05:44.919222+0000","last_fullsized":"2023-10-16T11:05:44.919222+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:54:46.053648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.346435+0000","last_change":"2023-10-16T11:05:43.944326+0000","last_active":"2023-10-16T11:05:45.346435+0000","last_peered":"2023-10-16T11:05:45.346435+0000","last_clean":"2023-10-16T11:05:45.346435+0000","last_became_active":"2023-10-16T11:05:43.944055+0000","last_became_peered":"2023-10-16T11:05:43.944055+0000","last_unstale":"2023-10-16T11:05:45.346435+0000","last_undegraded":"2023-10-16T11:05:45.346435+0000","last_fullsized":"2023-10-16T11:05:45.346435+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:09.705333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.347507+0000","last_change":"2023-10-16T11:05:43.927446+0000","last_active":"2023-10-16T11:05:45.347507+0000","last_peered":"2023-10-16T11:05:45.347507+0000","last_clean":"2023-10-16T11:05:45.347507+0000","last_became_active":"2023-10-16T11:05:43.927120+0000","last_became_peered":"2023-10-16T11:05:43.927120+0000","last_unstale":"2023-10-16T11:05:45.347507+0000","last_undegraded":"2023-10-16T11:05:45.347507+0000","last_fullsized":"2023-10-16T11:05:45.347507+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:42:43.713631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344809+0000","last_change":"2023-10-16T11:05:43.932863+0000","last_active":"2023-10-16T11:05:45.344809+0000","last_peered":"2023-10-16T11:05:45.344809+0000","last_clean":"2023-10-16T11:05:45.344809+0000","last_became_active":"2023-10-16T11:05:43.932595+0000","last_became_peered":"2023-10-16T11:05:43.932595+0000","last_unstale":"2023-10-16T11:05:45.344809+0000","last_undegraded":"2023-10-16T11:05:45.344809+0000","last_fullsized":"2023-10-16T11:05:45.344809+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:19:35.567531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.920369+0000","last_change":"2023-10-16T11:05:43.939877+0000","last_active":"2023-10-16T11:05:44.920369+0000","last_peered":"2023-10-16T11:05:44.920369+0000","last_clean":"2023-10-16T11:05:44.920369+0000","last_became_active":"2023-10-16T11:05:43.936107+0000","last_became_peered":"2023-10-16T11:05:43.936107+0000","last_unstale":"2023-10-16T11:05:44.920369+0000","last_undegraded":"2023-10-16T11:05:44.920369+0000","last_fullsized":"2023-10-16T11:05:44.920369+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:31:27.542416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344655+0000","last_change":"2023-10-16T11:05:43.939808+0000","last_active":"2023-10-16T11:05:45.344655+0000","last_peered":"2023-10-16T11:05:45.344655+0000","last_clean":"2023-10-16T11:05:45.344655+0000","last_became_active":"2023-10-16T11:05:43.939504+0000","last_became_peered":"2023-10-16T11:05:43.939504+0000","last_unstale":"2023-10-16T11:05:45.344655+0000","last_undegraded":"2023-10-16T11:05:45.344655+0000","last_fullsized":"2023-10-16T11:05:45.344655+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:53:46.539221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.918878+0000","last_change":"2023-10-16T11:05:43.939090+0000","last_active":"2023-10-16T11:05:44.918878+0000","last_peered":"2023-10-16T11:05:44.918878+0000","last_clean":"2023-10-16T11:05:44.918878+0000","last_became_active":"2023-10-16T11:05:43.938744+0000","last_became_peered":"2023-10-16T11:05:43.938744+0000","last_unstale":"2023-10-16T11:05:44.918878+0000","last_undegraded":"2023-10-16T11:05:44.918878+0000","last_fullsized":"2023-10-16T11:05:44.918878+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:58:58.490380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.918633+0000","last_change":"2023-10-16T11:05:43.939354+0000","last_active":"2023-10-16T11:05:44.918633+0000","last_peered":"2023-10-16T11:05:44.918633+0000","last_clean":"2023-10-16T11:05:44.918633+0000","last_became_active":"2023-10-16T11:05:43.936886+0000","last_became_peered":"2023-10-16T11:05:43.936886+0000","last_unstale":"2023-10-16T11:05:44.918633+0000","last_undegraded":"2023-10-16T11:05:44.918633+0000","last_fullsized":"2023-10-16T11:05:44.918633+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:43.016362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.342966+0000","last_change":"2023-10-16T11:05:43.938894+0000","last_active":"2023-10-16T11:05:45.342966+0000","last_peered":"2023-10-16T11:05:45.342966+0000","last_clean":"2023-10-16T11:05:45.342966+0000","last_became_active":"2023-10-16T11:05:43.938181+0000","last_became_peered":"2023-10-16T11:05:43.938181+0000","last_unstale":"2023-10-16T11:05:45.342966+0000","last_undegraded":"2023-10-16T11:05:45.342966+0000","last_fullsized":"2023-10-16T11:05:45.342966+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:42.494223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.920701+0000","last_change":"2023-10-16T11:05:43.935650+0000","last_active":"2023-10-16T11:05:44.920701+0000","last_peered":"2023-10-16T11:05:44.920701+0000","last_clean":"2023-10-16T11:05:44.920701+0000","last_became_active":"2023-10-16T11:05:43.935268+0000","last_became_peered":"2023-10-16T11:05:43.935268+0000","last_unstale":"2023-10-16T11:05:44.920701+0000","last_undegraded":"2023-10-16T11:05:44.920701+0000","last_fullsized":"2023-10-16T11:05:44.920701+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:48:33.468683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344977+0000","last_change":"2023-10-16T11:05:43.932243+0000","last_active":"2023-10-16T11:05:45.344977+0000","last_peered":"2023-10-16T11:05:45.344977+0000","last_clean":"2023-10-16T11:05:45.344977+0000","last_became_active":"2023-10-16T11:05:43.931928+0000","last_became_peered":"2023-10-16T11:05:43.931928+0000","last_unstale":"2023-10-16T11:05:45.344977+0000","last_undegraded":"2023-10-16T11:05:45.344977+0000","last_fullsized":"2023-10-16T11:05:45.344977+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:42.729654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.920794+0000","last_change":"2023-10-16T11:05:43.943642+0000","last_active":"2023-10-16T11:05:44.920794+0000","last_peered":"2023-10-16T11:05:44.920794+0000","last_clean":"2023-10-16T11:05:44.920794+0000","last_became_active":"2023-10-16T11:05:43.943372+0000","last_became_peered":"2023-10-16T11:05:43.943372+0000","last_unstale":"2023-10-16T11:05:44.920794+0000","last_undegraded":"2023-10-16T11:05:44.920794+0000","last_fullsized":"2023-10-16T11:05:44.920794+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:24.594901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.918402+0000","last_change":"2023-10-16T11:05:43.955065+0000","last_active":"2023-10-16T11:05:44.918402+0000","last_peered":"2023-10-16T11:05:44.918402+0000","last_clean":"2023-10-16T11:05:44.918402+0000","last_became_active":"2023-10-16T11:05:43.954642+0000","last_became_peered":"2023-10-16T11:05:43.954642+0000","last_unstale":"2023-10-16T11:05:44.918402+0000","last_undegraded":"2023-10-16T11:05:44.918402+0000","last_fullsized":"2023-10-16T11:05:44.918402+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:52:05.468611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.342839+0000","last_change":"2023-10-16T11:05:43.917547+0000","last_active":"2023-10-16T11:05:45.342839+0000","last_peered":"2023-10-16T11:05:45.342839+0000","last_clean":"2023-10-16T11:05:45.342839+0000","last_became_active":"2023-10-16T11:05:43.914894+0000","last_became_peered":"2023-10-16T11:05:43.914894+0000","last_unstale":"2023-10-16T11:05:45.342839+0000","last_undegraded":"2023-10-16T11:05:45.342839+0000","last_fullsized":"2023-10-16T11:05:45.342839+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:14:22.417740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.919280+0000","last_change":"2023-10-16T11:05:43.934761+0000","last_active":"2023-10-16T11:05:44.919280+0000","last_peered":"2023-10-16T11:05:44.919280+0000","last_clean":"2023-10-16T11:05:44.919280+0000","last_became_active":"2023-10-16T11:05:43.934455+0000","last_became_peered":"2023-10-16T11:05:43.934455+0000","last_unstale":"2023-10-16T11:05:44.919280+0000","last_undegraded":"2023-10-16T11:05:44.919280+0000","last_fullsized":"2023-10-16T11:05:44.919280+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:44:02.763410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343261+0000","last_change":"2023-10-16T11:05:43.947823+0000","last_active":"2023-10-16T11:05:45.343261+0000","last_peered":"2023-10-16T11:05:45.343261+0000","last_clean":"2023-10-16T11:05:45.343261+0000","last_became_active":"2023-10-16T11:05:43.947537+0000","last_became_peered":"2023-10-16T11:05:43.947537+0000","last_unstale":"2023-10-16T11:05:45.343261+0000","last_undegraded":"2023-10-16T11:05:45.343261+0000","last_fullsized":"2023-10-16T11:05:45.343261+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:42:08.013977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.919882+0000","last_change":"2023-10-16T11:05:43.941896+0000","last_active":"2023-10-16T11:05:44.919882+0000","last_peered":"2023-10-16T11:05:44.919882+0000","last_clean":"2023-10-16T11:05:44.919882+0000","last_became_active":"2023-10-16T11:05:43.941643+0000","last_became_peered":"2023-10-16T11:05:43.941643+0000","last_unstale":"2023-10-16T11:05:44.919882+0000","last_undegraded":"2023-10-16T11:05:44.919882+0000","last_fullsized":"2023-10-16T11:05:44.919882+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:55:56.309108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344304+0000","last_change":"2023-10-16T11:05:43.941645+0000","last_active":"2023-10-16T11:05:45.344304+0000","last_peered":"2023-10-16T11:05:45.344304+0000","last_clean":"2023-10-16T11:05:45.344304+0000","last_became_active":"2023-10-16T11:05:43.939871+0000","last_became_peered":"2023-10-16T11:05:43.939871+0000","last_unstale":"2023-10-16T11:05:45.344304+0000","last_undegraded":"2023-10-16T11:05:45.344304+0000","last_fullsized":"2023-10-16T11:05:45.344304+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:13:03.474138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"170'4","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.950861+0000","last_change":"2023-10-16T11:05:42.918870+0000","last_active":"2023-10-16T11:05:43.950861+0000","last_peered":"2023-10-16T11:05:43.950861+0000","last_clean":"2023-10-16T11:05:43.950861+0000","last_became_active":"2023-10-16T11:05:42.918555+0000","last_became_peered":"2023-10-16T11:05:42.918555+0000","last_unstale":"2023-10-16T11:05:43.950861+0000","last_undegraded":"2023-10-16T11:05:43.950861+0000","last_fullsized":"2023-10-16T11:05:43.950861+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:07.474892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344027+0000","last_change":"2023-10-16T11:05:43.935112+0000","last_active":"2023-10-16T11:05:45.344027+0000","last_peered":"2023-10-16T11:05:45.344027+0000","last_clean":"2023-10-16T11:05:45.344027+0000","last_became_active":"2023-10-16T11:05:43.934858+0000","last_became_peered":"2023-10-16T11:05:43.934858+0000","last_unstale":"2023-10-16T11:05:45.344027+0000","last_undegraded":"2023-10-16T11:05:45.344027+0000","last_fullsized":"2023-10-16T11:05:45.344027+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:56.538637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.950534+0000","last_change":"2023-10-16T11:05:42.915253+0000","last_active":"2023-10-16T11:05:43.950534+0000","last_peered":"2023-10-16T11:05:43.950534+0000","last_clean":"2023-10-16T11:05:43.950534+0000","last_became_active":"2023-10-16T11:05:42.914890+0000","last_became_peered":"2023-10-16T11:05:42.914890+0000","last_unstale":"2023-10-16T11:05:43.950534+0000","last_undegraded":"2023-10-16T11:05:43.950534+0000","last_fullsized":"2023-10-16T11:05:43.950534+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:37.839005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343949+0000","last_change":"2023-10-16T11:05:43.942571+0000","last_active":"2023-10-16T11:05:45.343949+0000","last_peered":"2023-10-16T11:05:45.343949+0000","last_clean":"2023-10-16T11:05:45.343949+0000","last_became_active":"2023-10-16T11:05:43.942310+0000","last_became_peered":"2023-10-16T11:05:43.942310+0000","last_unstale":"2023-10-16T11:05:45.343949+0000","last_undegraded":"2023-10-16T11:05:45.343949+0000","last_fullsized":"2023-10-16T11:05:45.343949+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:34.728238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.955427+0000","last_change":"2023-10-16T11:05:42.905210+0000","last_active":"2023-10-16T11:05:43.955427+0000","last_peered":"2023-10-16T11:05:43.955427+0000","last_clean":"2023-10-16T11:05:43.955427+0000","last_became_active":"2023-10-16T11:05:42.904920+0000","last_became_peered":"2023-10-16T11:05:42.904920+0000","last_unstale":"2023-10-16T11:05:43.955427+0000","last_undegraded":"2023-10-16T11:05:43.955427+0000","last_fullsized":"2023-10-16T11:05:43.955427+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:03.159020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343606+0000","last_change":"2023-10-16T11:05:43.938972+0000","last_active":"2023-10-16T11:05:45.343606+0000","last_peered":"2023-10-16T11:05:45.343606+0000","last_clean":"2023-10-16T11:05:45.343606+0000","last_became_active":"2023-10-16T11:05:43.938616+0000","last_became_peered":"2023-10-16T11:05:43.938616+0000","last_unstale":"2023-10-16T11:05:45.343606+0000","last_undegraded":"2023-10-16T11:05:45.343606+0000","last_fullsized":"2023-10-16T11:05:45.343606+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:21:57.920762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.953885+0000","last_change":"2023-10-16T11:05:42.910283+0000","last_active":"2023-10-16T11:05:43.953885+0000","last_peered":"2023-10-16T11:05:43.953885+0000","last_clean":"2023-10-16T11:05:43.953885+0000","last_became_active":"2023-10-16T11:05:42.908996+0000","last_became_peered":"2023-10-16T11:05:42.908996+0000","last_unstale":"2023-10-16T11:05:43.953885+0000","last_undegraded":"2023-10-16T11:05:43.953885+0000","last_fullsized":"2023-10-16T11:05:43.953885+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:33:45.172836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343648+0000","last_change":"2023-10-16T11:05:43.948056+0000","last_active":"2023-10-16T11:05:45.343648+0000","last_peered":"2023-10-16T11:05:45.343648+0000","last_clean":"2023-10-16T11:05:45.343648+0000","last_became_active":"2023-10-16T11:05:43.947753+0000","last_became_peered":"2023-10-16T11:05:43.947753+0000","last_unstale":"2023-10-16T11:05:45.343648+0000","last_undegraded":"2023-10-16T11:05:45.343648+0000","last_fullsized":"2023-10-16T11:05:45.343648+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:42.329874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.952209+0000","last_change":"2023-10-16T11:05:42.913476+0000","last_active":"2023-10-16T11:05:43.952209+0000","last_peered":"2023-10-16T11:05:43.952209+0000","last_clean":"2023-10-16T11:05:43.952209+0000","last_became_active":"2023-10-16T11:05:42.913176+0000","last_became_peered":"2023-10-16T11:05:42.913176+0000","last_unstale":"2023-10-16T11:05:43.952209+0000","last_undegraded":"2023-10-16T11:05:43.952209+0000","last_fullsized":"2023-10-16T11:05:43.952209+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:05:11.461717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.919099+0000","last_change":"2023-10-16T11:05:43.948302+0000","last_active":"2023-10-16T11:05:44.919099+0000","last_peered":"2023-10-16T11:05:44.919099+0000","last_clean":"2023-10-16T11:05:44.919099+0000","last_became_active":"2023-10-16T11:05:43.948011+0000","last_became_peered":"2023-10-16T11:05:43.948011+0000","last_unstale":"2023-10-16T11:05:44.919099+0000","last_undegraded":"2023-10-16T11:05:44.919099+0000","last_fullsized":"2023-10-16T11:05:44.919099+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:33:09.375472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.951363+0000","last_change":"2023-10-16T11:05:42.915535+0000","last_active":"2023-10-16T11:05:43.951363+0000","last_peered":"2023-10-16T11:05:43.951363+0000","last_clean":"2023-10-16T11:05:43.951363+0000","last_became_active":"2023-10-16T11:05:42.915273+0000","last_became_peered":"2023-10-16T11:05:42.915273+0000","last_unstale":"2023-10-16T11:05:43.951363+0000","last_undegraded":"2023-10-16T11:05:43.951363+0000","last_fullsized":"2023-10-16T11:05:43.951363+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:33:10.465828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343151+0000","last_change":"2023-10-16T11:05:43.940800+0000","last_active":"2023-10-16T11:05:45.343151+0000","last_peered":"2023-10-16T11:05:45.343151+0000","last_clean":"2023-10-16T11:05:45.343151+0000","last_became_active":"2023-10-16T11:05:43.940519+0000","last_became_peered":"2023-10-16T11:05:43.940519+0000","last_unstale":"2023-10-16T11:05:45.343151+0000","last_undegraded":"2023-10-16T11:05:45.343151+0000","last_fullsized":"2023-10-16T11:05:45.343151+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:08:58.300877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.909282+0000","last_change":"2023-10-16T11:05:42.914001+0000","last_active":"2023-10-16T11:05:43.909282+0000","last_peered":"2023-10-16T11:05:43.909282+0000","last_clean":"2023-10-16T11:05:43.909282+0000","last_became_active":"2023-10-16T11:05:42.913713+0000","last_became_peered":"2023-10-16T11:05:42.913713+0000","last_unstale":"2023-10-16T11:05:43.909282+0000","last_undegraded":"2023-10-16T11:05:43.909282+0000","last_fullsized":"2023-10-16T11:05:43.909282+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:57:14.976665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.342994+0000","last_change":"2023-10-16T11:05:43.932283+0000","last_active":"2023-10-16T11:05:45.342994+0000","last_peered":"2023-10-16T11:05:45.342994+0000","last_clean":"2023-10-16T11:05:45.342994+0000","last_became_active":"2023-10-16T11:05:43.929314+0000","last_became_peered":"2023-10-16T11:05:43.929314+0000","last_unstale":"2023-10-16T11:05:45.342994+0000","last_undegraded":"2023-10-16T11:05:45.342994+0000","last_fullsized":"2023-10-16T11:05:45.342994+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:55:56.309108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.927424+0000","last_change":"2023-10-16T11:05:42.913285+0000","last_active":"2023-10-16T11:05:43.927424+0000","last_peered":"2023-10-16T11:05:43.927424+0000","last_clean":"2023-10-16T11:05:43.927424+0000","last_became_active":"2023-10-16T11:05:42.912925+0000","last_became_peered":"2023-10-16T11:05:42.912925+0000","last_unstale":"2023-10-16T11:05:43.927424+0000","last_undegraded":"2023-10-16T11:05:43.927424+0000","last_fullsized":"2023-10-16T11:05:43.927424+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:54.804384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.345683+0000","last_change":"2023-10-16T11:05:43.935465+0000","last_active":"2023-10-16T11:05:45.345683+0000","last_peered":"2023-10-16T11:05:45.345683+0000","last_clean":"2023-10-16T11:05:45.345683+0000","last_became_active":"2023-10-16T11:05:43.934444+0000","last_became_peered":"2023-10-16T11:05:43.934444+0000","last_unstale":"2023-10-16T11:05:45.345683+0000","last_undegraded":"2023-10-16T11:05:45.345683+0000","last_fullsized":"2023-10-16T11:05:45.345683+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:40:13.279771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.924030+0000","last_change":"2023-10-16T11:05:42.907377+0000","last_active":"2023-10-16T11:05:43.924030+0000","last_peered":"2023-10-16T11:05:43.924030+0000","last_clean":"2023-10-16T11:05:43.924030+0000","last_became_active":"2023-10-16T11:05:42.907101+0000","last_became_peered":"2023-10-16T11:05:42.907101+0000","last_unstale":"2023-10-16T11:05:43.924030+0000","last_undegraded":"2023-10-16T11:05:43.924030+0000","last_fullsized":"2023-10-16T11:05:43.924030+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:24:26.136046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.345898+0000","last_change":"2023-10-16T11:05:43.944147+0000","last_active":"2023-10-16T11:05:45.345898+0000","last_peered":"2023-10-16T11:05:45.345898+0000","last_clean":"2023-10-16T11:05:45.345898+0000","last_became_active":"2023-10-16T11:05:43.943883+0000","last_became_peered":"2023-10-16T11:05:43.943883+0000","last_unstale":"2023-10-16T11:05:45.345898+0000","last_undegraded":"2023-10-16T11:05:45.345898+0000","last_fullsized":"2023-10-16T11:05:45.345898+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:31:00.203499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.930738+0000","last_change":"2023-10-16T11:05:42.919730+0000","last_active":"2023-10-16T11:05:43.930738+0000","last_peered":"2023-10-16T11:05:43.930738+0000","last_clean":"2023-10-16T11:05:43.930738+0000","last_became_active":"2023-10-16T11:05:42.919466+0000","last_became_peered":"2023-10-16T11:05:42.919466+0000","last_unstale":"2023-10-16T11:05:43.930738+0000","last_undegraded":"2023-10-16T11:05:43.930738+0000","last_fullsized":"2023-10-16T11:05:43.930738+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:02:11.493187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.921053+0000","last_change":"2023-10-16T11:05:43.931226+0000","last_active":"2023-10-16T11:05:44.921053+0000","last_peered":"2023-10-16T11:05:44.921053+0000","last_clean":"2023-10-16T11:05:44.921053+0000","last_became_active":"2023-10-16T11:05:43.930927+0000","last_became_peered":"2023-10-16T11:05:43.930927+0000","last_unstale":"2023-10-16T11:05:44.921053+0000","last_undegraded":"2023-10-16T11:05:44.921053+0000","last_fullsized":"2023-10-16T11:05:44.921053+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:29.299026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"170'6","reported_seq":18,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.957237+0000","last_change":"2023-10-16T11:05:42.907360+0000","last_active":"2023-10-16T11:05:43.957237+0000","last_peered":"2023-10-16T11:05:43.957237+0000","last_clean":"2023-10-16T11:05:43.957237+0000","last_became_active":"2023-10-16T11:05:42.905385+0000","last_became_peered":"2023-10-16T11:05:42.905385+0000","last_unstale":"2023-10-16T11:05:43.957237+0000","last_undegraded":"2023-10-16T11:05:43.957237+0000","last_fullsized":"2023-10-16T11:05:43.957237+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:30:54.157804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.346662+0000","last_change":"2023-10-16T11:05:43.936449+0000","last_active":"2023-10-16T11:05:45.346662+0000","last_peered":"2023-10-16T11:05:45.346662+0000","last_clean":"2023-10-16T11:05:45.346662+0000","last_became_active":"2023-10-16T11:05:43.936085+0000","last_became_peered":"2023-10-16T11:05:43.936085+0000","last_unstale":"2023-10-16T11:05:45.346662+0000","last_undegraded":"2023-10-16T11:05:45.346662+0000","last_fullsized":"2023-10-16T11:05:45.346662+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:05:12.478756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.956239+0000","last_change":"2023-10-16T11:05:42.910401+0000","last_active":"2023-10-16T11:05:43.956239+0000","last_peered":"2023-10-16T11:05:43.956239+0000","last_clean":"2023-10-16T11:05:43.956239+0000","last_became_active":"2023-10-16T11:05:42.909868+0000","last_became_peered":"2023-10-16T11:05:42.909868+0000","last_unstale":"2023-10-16T11:05:43.956239+0000","last_undegraded":"2023-10-16T11:05:43.956239+0000","last_fullsized":"2023-10-16T11:05:43.956239+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:35:04.022368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343622+0000","last_change":"2023-10-16T11:05:43.937515+0000","last_active":"2023-10-16T11:05:45.343622+0000","last_peered":"2023-10-16T11:05:45.343622+0000","last_clean":"2023-10-16T11:05:45.343622+0000","last_became_active":"2023-10-16T11:05:43.936358+0000","last_became_peered":"2023-10-16T11:05:43.936358+0000","last_unstale":"2023-10-16T11:05:45.343622+0000","last_undegraded":"2023-10-16T11:05:45.343622+0000","last_fullsized":"2023-10-16T11:05:45.343622+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:31.492817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.908532+0000","last_change":"2023-10-16T11:05:42.908928+0000","last_active":"2023-10-16T11:05:43.908532+0000","last_peered":"2023-10-16T11:05:43.908532+0000","last_clean":"2023-10-16T11:05:43.908532+0000","last_became_active":"2023-10-16T11:05:42.908594+0000","last_became_peered":"2023-10-16T11:05:42.908594+0000","last_unstale":"2023-10-16T11:05:43.908532+0000","last_undegraded":"2023-10-16T11:05:43.908532+0000","last_fullsized":"2023-10-16T11:05:43.908532+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:28.830200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343221+0000","last_change":"2023-10-16T11:05:43.942743+0000","last_active":"2023-10-16T11:05:45.343221+0000","last_peered":"2023-10-16T11:05:45.343221+0000","last_clean":"2023-10-16T11:05:45.343221+0000","last_became_active":"2023-10-16T11:05:43.942382+0000","last_became_peered":"2023-10-16T11:05:43.942382+0000","last_unstale":"2023-10-16T11:05:45.343221+0000","last_undegraded":"2023-10-16T11:05:45.343221+0000","last_fullsized":"2023-10-16T11:05:45.343221+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:31:27.542416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.906948+0000","last_change":"2023-10-16T11:05:42.905562+0000","last_active":"2023-10-16T11:05:43.906948+0000","last_peered":"2023-10-16T11:05:43.906948+0000","last_clean":"2023-10-16T11:05:43.906948+0000","last_became_active":"2023-10-16T11:05:42.905127+0000","last_became_peered":"2023-10-16T11:05:42.905127+0000","last_unstale":"2023-10-16T11:05:43.906948+0000","last_undegraded":"2023-10-16T11:05:43.906948+0000","last_fullsized":"2023-10-16T11:05:43.906948+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:46.130329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.346162+0000","last_change":"2023-10-16T11:05:43.924551+0000","last_active":"2023-10-16T11:05:45.346162+0000","last_peered":"2023-10-16T11:05:45.346162+0000","last_clean":"2023-10-16T11:05:45.346162+0000","last_became_active":"2023-10-16T11:05:43.923368+0000","last_became_peered":"2023-10-16T11:05:43.923368+0000","last_unstale":"2023-10-16T11:05:45.346162+0000","last_undegraded":"2023-10-16T11:05:45.346162+0000","last_fullsized":"2023-10-16T11:05:45.346162+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:55.571221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"170'8","reported_seq":20,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.956853+0000","last_change":"2023-10-16T11:05:42.899908+0000","last_active":"2023-10-16T11:05:43.956853+0000","last_peered":"2023-10-16T11:05:43.956853+0000","last_clean":"2023-10-16T11:05:43.956853+0000","last_became_active":"2023-10-16T11:05:42.899636+0000","last_became_peered":"2023-10-16T11:05:42.899636+0000","last_unstale":"2023-10-16T11:05:43.956853+0000","last_undegraded":"2023-10-16T11:05:43.956853+0000","last_fullsized":"2023-10-16T11:05:43.956853+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:17.635993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344846+0000","last_change":"2023-10-16T11:05:43.942344+0000","last_active":"2023-10-16T11:05:45.344846+0000","last_peered":"2023-10-16T11:05:45.344846+0000","last_clean":"2023-10-16T11:05:45.344846+0000","last_became_active":"2023-10-16T11:05:43.942027+0000","last_became_peered":"2023-10-16T11:05:43.942027+0000","last_unstale":"2023-10-16T11:05:45.344846+0000","last_undegraded":"2023-10-16T11:05:45.344846+0000","last_fullsized":"2023-10-16T11:05:45.344846+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:36:27.710717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.953486+0000","last_change":"2023-10-16T11:05:42.914096+0000","last_active":"2023-10-16T11:05:43.953486+0000","last_peered":"2023-10-16T11:05:43.953486+0000","last_clean":"2023-10-16T11:05:43.953486+0000","last_became_active":"2023-10-16T11:05:42.913780+0000","last_became_peered":"2023-10-16T11:05:42.913780+0000","last_unstale":"2023-10-16T11:05:43.953486+0000","last_undegraded":"2023-10-16T11:05:43.953486+0000","last_fullsized":"2023-10-16T11:05:43.953486+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:55:45.317520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.920922+0000","last_change":"2023-10-16T11:05:43.946265+0000","last_active":"2023-10-16T11:05:44.920922+0000","last_peered":"2023-10-16T11:05:44.920922+0000","last_clean":"2023-10-16T11:05:44.920922+0000","last_became_active":"2023-10-16T11:05:43.945202+0000","last_became_peered":"2023-10-16T11:05:43.945202+0000","last_unstale":"2023-10-16T11:05:44.920922+0000","last_undegraded":"2023-10-16T11:05:44.920922+0000","last_fullsized":"2023-10-16T11:05:44.920922+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:21:09.268813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.954360+0000","last_change":"2023-10-16T11:05:42.910188+0000","last_active":"2023-10-16T11:05:43.954360+0000","last_peered":"2023-10-16T11:05:43.954360+0000","last_clean":"2023-10-16T11:05:43.954360+0000","last_became_active":"2023-10-16T11:05:42.909869+0000","last_became_peered":"2023-10-16T11:05:42.909869+0000","last_unstale":"2023-10-16T11:05:43.954360+0000","last_undegraded":"2023-10-16T11:05:43.954360+0000","last_fullsized":"2023-10-16T11:05:43.954360+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:15:38.581080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.346675+0000","last_change":"2023-10-16T11:05:43.939770+0000","last_active":"2023-10-16T11:05:45.346675+0000","last_peered":"2023-10-16T11:05:45.346675+0000","last_clean":"2023-10-16T11:05:45.346675+0000","last_became_active":"2023-10-16T11:05:43.937396+0000","last_became_peered":"2023-10-16T11:05:43.937396+0000","last_unstale":"2023-10-16T11:05:45.346675+0000","last_undegraded":"2023-10-16T11:05:45.346675+0000","last_fullsized":"2023-10-16T11:05:45.346675+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:57:15.993704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.951769+0000","last_change":"2023-10-16T11:05:42.911198+0000","last_active":"2023-10-16T11:05:43.951769+0000","last_peered":"2023-10-16T11:05:43.951769+0000","last_clean":"2023-10-16T11:05:43.951769+0000","last_became_active":"2023-10-16T11:05:42.910923+0000","last_became_peered":"2023-10-16T11:05:42.910923+0000","last_unstale":"2023-10-16T11:05:43.951769+0000","last_undegraded":"2023-10-16T11:05:43.951769+0000","last_fullsized":"2023-10-16T11:05:43.951769+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:22:20.697602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344408+0000","last_change":"2023-10-16T11:05:43.918404+0000","last_active":"2023-10-16T11:05:45.344408+0000","last_peered":"2023-10-16T11:05:45.344408+0000","last_clean":"2023-10-16T11:05:45.344408+0000","last_became_active":"2023-10-16T11:05:43.917782+0000","last_became_peered":"2023-10-16T11:05:43.917782+0000","last_unstale":"2023-10-16T11:05:45.344408+0000","last_undegraded":"2023-10-16T11:05:45.344408+0000","last_fullsized":"2023-10-16T11:05:45.344408+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:54:51.097482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.927489+0000","last_change":"2023-10-16T11:05:42.905615+0000","last_active":"2023-10-16T11:05:43.927489+0000","last_peered":"2023-10-16T11:05:43.927489+0000","last_clean":"2023-10-16T11:05:43.927489+0000","last_became_active":"2023-10-16T11:05:42.904492+0000","last_became_peered":"2023-10-16T11:05:42.904492+0000","last_unstale":"2023-10-16T11:05:43.927489+0000","last_undegraded":"2023-10-16T11:05:43.927489+0000","last_fullsized":"2023-10-16T11:05:43.927489+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:46:07.003568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.919628+0000","last_change":"2023-10-16T11:05:43.933380+0000","last_active":"2023-10-16T11:05:44.919628+0000","last_peered":"2023-10-16T11:05:44.919628+0000","last_clean":"2023-10-16T11:05:44.919628+0000","last_became_active":"2023-10-16T11:05:43.932936+0000","last_became_peered":"2023-10-16T11:05:43.932936+0000","last_unstale":"2023-10-16T11:05:44.919628+0000","last_undegraded":"2023-10-16T11:05:44.919628+0000","last_fullsized":"2023-10-16T11:05:44.919628+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:19.045384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.954625+0000","last_change":"2023-10-16T11:05:42.912394+0000","last_active":"2023-10-16T11:05:43.954625+0000","last_peered":"2023-10-16T11:05:43.954625+0000","last_clean":"2023-10-16T11:05:43.954625+0000","last_became_active":"2023-10-16T11:05:42.912102+0000","last_became_peered":"2023-10-16T11:05:42.912102+0000","last_unstale":"2023-10-16T11:05:43.954625+0000","last_undegraded":"2023-10-16T11:05:43.954625+0000","last_fullsized":"2023-10-16T11:05:43.954625+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:17.635993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.918678+0000","last_change":"2023-10-16T11:05:43.939980+0000","last_active":"2023-10-16T11:05:44.918678+0000","last_peered":"2023-10-16T11:05:44.918678+0000","last_clean":"2023-10-16T11:05:44.918678+0000","last_became_active":"2023-10-16T11:05:43.937118+0000","last_became_peered":"2023-10-16T11:05:43.937118+0000","last_unstale":"2023-10-16T11:05:44.918678+0000","last_undegraded":"2023-10-16T11:05:44.918678+0000","last_fullsized":"2023-10-16T11:05:44.918678+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:47.147368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.930738+0000","last_change":"2023-10-16T11:05:42.918022+0000","last_active":"2023-10-16T11:05:43.930738+0000","last_peered":"2023-10-16T11:05:43.930738+0000","last_clean":"2023-10-16T11:05:43.930738+0000","last_became_active":"2023-10-16T11:05:42.917723+0000","last_became_peered":"2023-10-16T11:05:42.917723+0000","last_unstale":"2023-10-16T11:05:43.930738+0000","last_undegraded":"2023-10-16T11:05:43.930738+0000","last_fullsized":"2023-10-16T11:05:43.930738+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:56:40.269324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.918317+0000","last_change":"2023-10-16T11:05:43.940159+0000","last_active":"2023-10-16T11:05:44.918317+0000","last_peered":"2023-10-16T11:05:44.918317+0000","last_clean":"2023-10-16T11:05:44.918317+0000","last_became_active":"2023-10-16T11:05:43.936836+0000","last_became_peered":"2023-10-16T11:05:43.936836+0000","last_unstale":"2023-10-16T11:05:44.918317+0000","last_undegraded":"2023-10-16T11:05:44.918317+0000","last_fullsized":"2023-10-16T11:05:44.918317+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:52.352439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.923355+0000","last_change":"2023-10-16T11:05:42.916214+0000","last_active":"2023-10-16T11:05:43.923355+0000","last_peered":"2023-10-16T11:05:43.923355+0000","last_clean":"2023-10-16T11:05:43.923355+0000","last_became_active":"2023-10-16T11:05:42.914986+0000","last_became_peered":"2023-10-16T11:05:42.914986+0000","last_unstale":"2023-10-16T11:05:43.923355+0000","last_undegraded":"2023-10-16T11:05:43.923355+0000","last_fullsized":"2023-10-16T11:05:43.923355+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:19:53.282121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.918971+0000","last_change":"2023-10-16T11:05:43.939592+0000","last_active":"2023-10-16T11:05:44.918971+0000","last_peered":"2023-10-16T11:05:44.918971+0000","last_clean":"2023-10-16T11:05:44.918971+0000","last_became_active":"2023-10-16T11:05:43.939217+0000","last_became_peered":"2023-10-16T11:05:43.939217+0000","last_unstale":"2023-10-16T11:05:44.918971+0000","last_undegraded":"2023-10-16T11:05:44.918971+0000","last_fullsized":"2023-10-16T11:05:44.918971+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:09:59.897371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.908081+0000","last_change":"2023-10-16T11:05:42.905792+0000","last_active":"2023-10-16T11:05:43.908081+0000","last_peered":"2023-10-16T11:05:43.908081+0000","last_clean":"2023-10-16T11:05:43.908081+0000","last_became_active":"2023-10-16T11:05:42.905502+0000","last_became_peered":"2023-10-16T11:05:42.905502+0000","last_unstale":"2023-10-16T11:05:43.908081+0000","last_undegraded":"2023-10-16T11:05:43.908081+0000","last_fullsized":"2023-10-16T11:05:43.908081+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:06.715283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.346884+0000","last_change":"2023-10-16T11:05:43.945177+0000","last_active":"2023-10-16T11:05:45.346884+0000","last_peered":"2023-10-16T11:05:45.346884+0000","last_clean":"2023-10-16T11:05:45.346884+0000","last_became_active":"2023-10-16T11:05:43.943724+0000","last_became_peered":"2023-10-16T11:05:43.943724+0000","last_unstale":"2023-10-16T11:05:45.346884+0000","last_undegraded":"2023-10-16T11:05:45.346884+0000","last_fullsized":"2023-10-16T11:05:45.346884+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:48:19.683014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.953621+0000","last_change":"2023-10-16T11:05:42.918054+0000","last_active":"2023-10-16T11:05:43.953621+0000","last_peered":"2023-10-16T11:05:43.953621+0000","last_clean":"2023-10-16T11:05:43.953621+0000","last_became_active":"2023-10-16T11:05:42.917765+0000","last_became_peered":"2023-10-16T11:05:42.917765+0000","last_unstale":"2023-10-16T11:05:43.953621+0000","last_undegraded":"2023-10-16T11:05:43.953621+0000","last_fullsized":"2023-10-16T11:05:43.953621+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:21:51.753638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.348058+0000","last_change":"2023-10-16T11:05:43.936490+0000","last_active":"2023-10-16T11:05:45.348058+0000","last_peered":"2023-10-16T11:05:45.348058+0000","last_clean":"2023-10-16T11:05:45.348058+0000","last_became_active":"2023-10-16T11:05:43.936191+0000","last_became_peered":"2023-10-16T11:05:43.936191+0000","last_unstale":"2023-10-16T11:05:45.348058+0000","last_undegraded":"2023-10-16T11:05:45.348058+0000","last_fullsized":"2023-10-16T11:05:45.348058+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:54:34.506511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.910783+0000","last_change":"2023-10-16T11:05:42.917331+0000","last_active":"2023-10-16T11:05:43.910783+0000","last_peered":"2023-10-16T11:05:43.910783+0000","last_clean":"2023-10-16T11:05:43.910783+0000","last_became_active":"2023-10-16T11:05:42.916992+0000","last_became_peered":"2023-10-16T11:05:42.916992+0000","last_unstale":"2023-10-16T11:05:43.910783+0000","last_undegraded":"2023-10-16T11:05:43.910783+0000","last_fullsized":"2023-10-16T11:05:43.910783+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:48:18.665975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344217+0000","last_change":"2023-10-16T11:05:43.951151+0000","last_active":"2023-10-16T11:05:45.344217+0000","last_peered":"2023-10-16T11:05:45.344217+0000","last_clean":"2023-10-16T11:05:45.344217+0000","last_became_active":"2023-10-16T11:05:43.949664+0000","last_became_peered":"2023-10-16T11:05:43.949664+0000","last_unstale":"2023-10-16T11:05:45.344217+0000","last_undegraded":"2023-10-16T11:05:45.344217+0000","last_fullsized":"2023-10-16T11:05:45.344217+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:10:24.868935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.914278+0000","last_change":"2023-10-16T11:05:42.905603+0000","last_active":"2023-10-16T11:05:43.914278+0000","last_peered":"2023-10-16T11:05:43.914278+0000","last_clean":"2023-10-16T11:05:43.914278+0000","last_became_active":"2023-10-16T11:05:42.904897+0000","last_became_peered":"2023-10-16T11:05:42.904897+0000","last_unstale":"2023-10-16T11:05:43.914278+0000","last_undegraded":"2023-10-16T11:05:43.914278+0000","last_fullsized":"2023-10-16T11:05:43.914278+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:54:45.039479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344408+0000","last_change":"2023-10-16T11:05:43.929675+0000","last_active":"2023-10-16T11:05:45.344408+0000","last_peered":"2023-10-16T11:05:45.344408+0000","last_clean":"2023-10-16T11:05:45.344408+0000","last_became_active":"2023-10-16T11:05:43.928934+0000","last_became_peered":"2023-10-16T11:05:43.928934+0000","last_unstale":"2023-10-16T11:05:45.344408+0000","last_undegraded":"2023-10-16T11:05:45.344408+0000","last_fullsized":"2023-10-16T11:05:45.344408+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:07:20.042750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.908337+0000","last_change":"2023-10-16T11:05:42.907090+0000","last_active":"2023-10-16T11:05:43.908337+0000","last_peered":"2023-10-16T11:05:43.908337+0000","last_clean":"2023-10-16T11:05:43.908337+0000","last_became_active":"2023-10-16T11:05:42.906778+0000","last_became_peered":"2023-10-16T11:05:42.906778+0000","last_unstale":"2023-10-16T11:05:43.908337+0000","last_undegraded":"2023-10-16T11:05:43.908337+0000","last_fullsized":"2023-10-16T11:05:43.908337+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:28.281987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.919114+0000","last_change":"2023-10-16T11:05:43.933007+0000","last_active":"2023-10-16T11:05:44.919114+0000","last_peered":"2023-10-16T11:05:44.919114+0000","last_clean":"2023-10-16T11:05:44.919114+0000","last_became_active":"2023-10-16T11:05:43.930134+0000","last_became_peered":"2023-10-16T11:05:43.930134+0000","last_unstale":"2023-10-16T11:05:44.919114+0000","last_undegraded":"2023-10-16T11:05:44.919114+0000","last_fullsized":"2023-10-16T11:05:44.919114+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:47.147368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.926123+0000","last_change":"2023-10-16T11:05:42.908481+0000","last_active":"2023-10-16T11:05:43.926123+0000","last_peered":"2023-10-16T11:05:43.926123+0000","last_clean":"2023-10-16T11:05:43.926123+0000","last_became_active":"2023-10-16T11:05:42.908207+0000","last_became_peered":"2023-10-16T11:05:42.908207+0000","last_unstale":"2023-10-16T11:05:43.926123+0000","last_undegraded":"2023-10-16T11:05:43.926123+0000","last_fullsized":"2023-10-16T11:05:43.926123+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:33:10.465828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.345073+0000","last_change":"2023-10-16T11:05:43.949130+0000","last_active":"2023-10-16T11:05:45.345073+0000","last_peered":"2023-10-16T11:05:45.345073+0000","last_clean":"2023-10-16T11:05:45.345073+0000","last_became_active":"2023-10-16T11:05:43.948805+0000","last_became_peered":"2023-10-16T11:05:43.948805+0000","last_unstale":"2023-10-16T11:05:45.345073+0000","last_undegraded":"2023-10-16T11:05:45.345073+0000","last_fullsized":"2023-10-16T11:05:45.345073+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:29.299026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.930867+0000","last_change":"2023-10-16T11:05:42.917987+0000","last_active":"2023-10-16T11:05:43.930867+0000","last_peered":"2023-10-16T11:05:43.930867+0000","last_clean":"2023-10-16T11:05:43.930867+0000","last_became_active":"2023-10-16T11:05:42.917681+0000","last_became_peered":"2023-10-16T11:05:42.917681+0000","last_unstale":"2023-10-16T11:05:43.930867+0000","last_undegraded":"2023-10-16T11:05:43.930867+0000","last_fullsized":"2023-10-16T11:05:43.930867+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:16:00.770358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343147+0000","last_change":"2023-10-16T11:05:43.941693+0000","last_active":"2023-10-16T11:05:45.343147+0000","last_peered":"2023-10-16T11:05:45.343147+0000","last_clean":"2023-10-16T11:05:45.343147+0000","last_became_active":"2023-10-16T11:05:43.941098+0000","last_became_peered":"2023-10-16T11:05:43.941098+0000","last_unstale":"2023-10-16T11:05:45.343147+0000","last_undegraded":"2023-10-16T11:05:45.343147+0000","last_fullsized":"2023-10-16T11:05:45.343147+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:02:17.916946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.907635+0000","last_change":"2023-10-16T11:05:42.907004+0000","last_active":"2023-10-16T11:05:43.907635+0000","last_peered":"2023-10-16T11:05:43.907635+0000","last_clean":"2023-10-16T11:05:43.907635+0000","last_became_active":"2023-10-16T11:05:42.906721+0000","last_became_peered":"2023-10-16T11:05:42.906721+0000","last_unstale":"2023-10-16T11:05:43.907635+0000","last_undegraded":"2023-10-16T11:05:43.907635+0000","last_fullsized":"2023-10-16T11:05:43.907635+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:39.259972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343687+0000","last_change":"2023-10-16T11:05:43.932513+0000","last_active":"2023-10-16T11:05:45.343687+0000","last_peered":"2023-10-16T11:05:45.343687+0000","last_clean":"2023-10-16T11:05:45.343687+0000","last_became_active":"2023-10-16T11:05:43.931958+0000","last_became_peered":"2023-10-16T11:05:43.931958+0000","last_unstale":"2023-10-16T11:05:45.343687+0000","last_undegraded":"2023-10-16T11:05:45.343687+0000","last_fullsized":"2023-10-16T11:05:45.343687+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:48:33.468683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.921145+0000","last_change":"2023-10-16T11:05:42.912613+0000","last_active":"2023-10-16T11:05:43.921145+0000","last_peered":"2023-10-16T11:05:43.921145+0000","last_clean":"2023-10-16T11:05:43.921145+0000","last_became_active":"2023-10-16T11:05:42.912139+0000","last_became_peered":"2023-10-16T11:05:42.912139+0000","last_unstale":"2023-10-16T11:05:43.921145+0000","last_undegraded":"2023-10-16T11:05:43.921145+0000","last_fullsized":"2023-10-16T11:05:43.921145+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:23:19.947376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.918701+0000","last_change":"2023-10-16T11:05:43.953051+0000","last_active":"2023-10-16T11:05:44.918701+0000","last_peered":"2023-10-16T11:05:44.918701+0000","last_clean":"2023-10-16T11:05:44.918701+0000","last_became_active":"2023-10-16T11:05:43.952652+0000","last_became_peered":"2023-10-16T11:05:43.952652+0000","last_unstale":"2023-10-16T11:05:44.918701+0000","last_undegraded":"2023-10-16T11:05:44.918701+0000","last_fullsized":"2023-10-16T11:05:44.918701+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:54:46.053648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"170'3","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.956300+0000","last_change":"2023-10-16T11:05:42.905639+0000","last_active":"2023-10-16T11:05:43.956300+0000","last_peered":"2023-10-16T11:05:43.956300+0000","last_clean":"2023-10-16T11:05:43.956300+0000","last_became_active":"2023-10-16T11:05:42.905299+0000","last_became_peered":"2023-10-16T11:05:42.905299+0000","last_unstale":"2023-10-16T11:05:43.956300+0000","last_undegraded":"2023-10-16T11:05:43.956300+0000","last_fullsized":"2023-10-16T11:05:43.956300+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:00.924601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"159'2690","reported_seq":3331,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.927088+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:05:43.927088+0000","last_peered":"2023-10-16T11:05:43.927088+0000","last_clean":"2023-10-16T11:05:43.927088+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:05:43.927088+0000","last_undegraded":"2023-10-16T11:05:43.927088+0000","last_fullsized":"2023-10-16T11:05:43.927088+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2690,"log_dups_size":0,"ondisk_log_size":2690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1504,"num_read_kb":3026,"num_write":441,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1504,"num_read_kb":3026,"num_write":441,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2690,"ondisk_log_size":2690,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575765,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64384,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307456,"statfs":{"total":96636764160,"available":96570834944,"internally_reserved":0,"allocated":2736128,"data_stored":1619019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":63175115},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:05:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.1739999999999999,"15min":1.2090000000000001},"min":{"1min":0.86699999999999999,"5min":0.76600000000000001,"15min":0.69499999999999995},"max":{"1min":1.577,"5min":1.7230000000000001,"15min":1.853},"last":1.165},{"interface":"front","average":{"1min":1.282,"5min":1.2390000000000001,"15min":1.2370000000000001},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.67300000000000004},"max":{"1min":1.6459999999999999,"5min":1.8029999999999999,"15min":1.9710000000000001},"last":1.365}]},{"osd":1,"last update":"Mon Oct 16 11:05:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2549999999999999,"15min":1.2629999999999999},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.78200000000000003},"max":{"1min":1.796,"5min":1.796,"15min":1.907},"last":1.226},{"interface":"front","average":{"1min":1.325,"5min":1.2869999999999999,"15min":1.28},"min":{"1min":1.0700000000000001,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.613,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.444}]},{"osd":2,"last update":"Mon Oct 16 11:05:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.367,"15min":1.3169999999999999},"min":{"1min":0.97299999999999998,"5min":0.93000000000000005,"15min":0.81899999999999995},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.125},"last":1.494},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.379,"15min":1.351},"min":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.79100000000000004},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":1.0149999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:05:12 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.395,"15min":1.3839999999999999},"min":{"1min":1.109,"5min":0.98899999999999999,"15min":0.96599999999999997},"max":{"1min":1.7529999999999999,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.518,"15min":1.458},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.8879999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.397}]},{"osd":4,"last update":"Mon Oct 16 11:05:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.446,"15min":1.401},"min":{"1min":0.998,"5min":0.85999999999999999,"15min":0.79700000000000004},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3999999999999999,"15min":1.359},"min":{"1min":0.91900000000000004,"5min":0.88200000000000001,"15min":0.74299999999999999},"max":{"1min":1.909,"5min":1.909,"15min":2.101},"last":1.123}]},{"osd":5,"last update":"Mon Oct 16 11:05:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.627,"15min":1.5600000000000001},"min":{"1min":1.0169999999999999,"5min":0.99199999999999999,"15min":0.98799999999999999},"max":{"1min":1.9610000000000001,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.244},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.5429999999999999,"15min":1.496},"min":{"1min":1.1279999999999999,"5min":1.125,"15min":0.89100000000000001},"max":{"1min":2.008,"5min":2.0249999999999999,"15min":2.395},"last":1.2789999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:05:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5529999999999999,"15min":1.4910000000000001},"min":{"1min":1.004,"5min":0.89200000000000002,"15min":0.88500000000000001},"max":{"1min":1.8520000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.7170000000000001,"5min":1.754,"15min":1.6830000000000001},"min":{"1min":1.266,"5min":1.093,"15min":1.093},"max":{"1min":2.1240000000000001,"5min":2.2400000000000002,"15min":2.2690000000000001},"last":1.5600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575765,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29124,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342716,"statfs":{"total":96636764160,"available":96606941184,"internally_reserved":0,"allocated":2146304,"data_stored":1028145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:05:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.256,"15min":1.2070000000000001},"min":{"1min":0.85499999999999998,"5min":0.75,"15min":0.65300000000000002},"max":{"1min":1.5840000000000001,"5min":1.665,"15min":1.665},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2889999999999999,"15min":1.272},"min":{"1min":0.96399999999999997,"5min":0.89100000000000001,"15min":0.874},"max":{"1min":1.544,"5min":1.784,"15min":1.784},"last":0.84399999999999997}]},{"osd":1,"last update":"Mon Oct 16 11:05:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.288,"15min":1.2370000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.373,"5min":1.3839999999999999,"15min":1.347},"min":{"1min":1.052,"5min":0.96399999999999997,"15min":0.56200000000000006},"max":{"1min":1.7110000000000001,"5min":1.8220000000000001,"15min":2.1440000000000001},"last":0.70399999999999996}]},{"osd":2,"last update":"Mon Oct 16 11:05:04 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.262},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.69999999999999996},"max":{"1min":1.5660000000000001,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4059999999999999,"15min":1.3979999999999999},"min":{"1min":0.84499999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.778,"5min":1.988,"15min":2.4129999999999998},"last":1.3740000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:05:04 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.274,"15min":1.272},"min":{"1min":0.96199999999999997,"5min":0.83399999999999996,"15min":0.746},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.9299999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.246,"15min":1.262},"min":{"1min":1.008,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.669,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.319}]},{"osd":4,"last update":"Mon Oct 16 11:05:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3240000000000001,"15min":1.3320000000000001},"min":{"1min":0.80900000000000005,"5min":0.78300000000000003,"15min":0.75600000000000001},"max":{"1min":1.8380000000000001,"5min":1.9159999999999999,"15min":2.2130000000000001},"last":1.472},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.518,"15min":1.4990000000000001},"min":{"1min":1.024,"5min":0.82399999999999995,"15min":0.79800000000000004},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.2490000000000001},"last":0.97699999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:05:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.3460000000000001,"15min":1.347},"min":{"1min":0.89300000000000002,"5min":0.83699999999999997,"15min":0.77000000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":2.1869999999999998},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.603,"5min":1.591,"15min":1.569},"min":{"1min":1.1240000000000001,"5min":1.0289999999999999,"15min":0.71699999999999997},"max":{"1min":2.0169999999999999,"5min":2.056,"15min":2.54},"last":1.5740000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:05:04 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.5069999999999999,"15min":1.49},"min":{"1min":0.89900000000000002,"5min":0.81899999999999995,"15min":0.621},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":2.1360000000000001},"last":1.423},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.5629999999999999,"15min":1.5429999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.89000000000000001},"max":{"1min":1.966,"5min":1.966,"15min":2.2050000000000001},"last":1.601}]}]},{"osd":5,"up_from":13,"seq":55834575765,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41596,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330244,"statfs":{"total":96636764160,"available":96594169856,"internally_reserved":0,"allocated":2138112,"data_stored":1028021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:05:18 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2749999999999999,"15min":1.2769999999999999},"min":{"1min":0.91600000000000004,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.6850000000000001,"5min":1.927,"15min":1.927},"last":1.617},{"interface":"front","average":{"1min":1.448,"5min":1.389,"15min":1.357},"min":{"1min":1.272,"5min":0.996,"15min":0.94899999999999995},"max":{"1min":1.714,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.5649999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:05:18 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.1970000000000001,"15min":1.194},"min":{"1min":0.82299999999999995,"5min":0.79300000000000004,"15min":0.76300000000000001},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2350000000000001,"15min":1.2230000000000001},"min":{"1min":1.016,"5min":0.83499999999999996,"15min":0.79400000000000004},"max":{"1min":1.431,"5min":1.821,"15min":1.877},"last":1.3919999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:05:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2270000000000001,"15min":1.22},"min":{"1min":0.92300000000000004,"5min":0.86299999999999999,"15min":0.81899999999999995},"max":{"1min":1.552,"5min":1.7709999999999999,"15min":1.9079999999999999},"last":1.165},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3169999999999999,"15min":1.3100000000000001},"min":{"1min":1.048,"5min":0.92300000000000004,"15min":0.83399999999999996},"max":{"1min":1.712,"5min":1.782,"15min":1.9570000000000001},"last":1.1220000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:05:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3200000000000001,"15min":1.3089999999999999},"min":{"1min":1.004,"5min":0.92800000000000005,"15min":0.88500000000000001},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":2.274},"last":1.333},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.423,"15min":1.4279999999999999},"min":{"1min":1.1000000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":1.956,"5min":1.956,"15min":2.1440000000000001},"last":1.468}]},{"osd":4,"last update":"Mon Oct 16 11:05:18 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.2609999999999999,"15min":1.28},"min":{"1min":0.79700000000000004,"5min":0.76300000000000001,"15min":0.70299999999999996},"max":{"1min":1.853,"5min":1.853,"15min":1.9870000000000001},"last":1.778},{"interface":"front","average":{"1min":1.587,"5min":1.5609999999999999,"15min":1.556},"min":{"1min":1.345,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.9650000000000001,"5min":2.0539999999999998,"15min":2.4620000000000002},"last":1.867}]},{"osd":6,"last update":"Mon Oct 16 11:05:18 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.45,"15min":1.419},"min":{"1min":0.97599999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.663},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.456,"15min":1.4419999999999999},"min":{"1min":0.98799999999999999,"5min":0.89100000000000001,"15min":0.86399999999999999},"max":{"1min":1.994,"5min":1.994,"15min":2.1099999999999999},"last":1.956}]},{"osd":7,"last update":"Mon Oct 16 11:05:18 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.5609999999999999,"15min":1.55},"min":{"1min":1.214,"5min":0.91400000000000003,"15min":0.85399999999999998},"max":{"1min":2.069,"5min":2.069,"15min":2.5270000000000001},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.599,"5min":1.659,"15min":1.6439999999999999},"min":{"1min":1.1539999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.6549999999999998},"last":1.823}]}]},{"osd":4,"up_from":13,"seq":55834575766,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59632,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312208,"statfs":{"total":96636764160,"available":96575700992,"internally_reserved":0,"allocated":2134016,"data_stored":1027999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:05:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.173,"15min":1.218},"min":{"1min":0.84499999999999997,"5min":0.74199999999999999,"15min":0.52600000000000002},"max":{"1min":1.6299999999999999,"5min":1.7190000000000001,"15min":1.8169999999999999},"last":1.837},{"interface":"front","average":{"1min":1.179,"5min":1.238,"15min":1.2529999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.78400000000000003},"max":{"1min":1.5760000000000001,"5min":1.7070000000000001,"15min":1.9790000000000001},"last":0.99199999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:05:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.351,"15min":1.3380000000000001},"min":{"1min":1.026,"5min":0.85999999999999999,"15min":0.82899999999999996},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.1680000000000001},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.2370000000000001,"15min":1.26},"min":{"1min":0.90900000000000003,"5min":0.88500000000000001,"15min":0.83999999999999997},"max":{"1min":1.671,"5min":1.7789999999999999,"15min":1.8080000000000001},"last":1.4019999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:05:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3,"15min":1.2809999999999999},"min":{"1min":0.93999999999999995,"5min":0.81200000000000006,"15min":0.67200000000000004},"max":{"1min":1.7330000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.546},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.387,"15min":1.419},"min":{"1min":1.03,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":1.8859999999999999,"5min":2.234,"15min":2.234},"last":1.3120000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:05:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.349,"15min":1.323},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.92400000000000004},"max":{"1min":1.772,"5min":1.899,"15min":1.899},"last":1.089},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.448,"15min":1.4299999999999999},"min":{"1min":1.036,"5min":0.97299999999999998,"15min":0.75800000000000001},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.9470000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:05:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.413,"15min":1.413},"min":{"1min":0.98399999999999999,"5min":0.83399999999999996,"15min":0.81799999999999995},"max":{"1min":1.7,"5min":1.9079999999999999,"15min":1.9119999999999999},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.429,"5min":1.4319999999999999,"15min":1.401},"min":{"1min":0.96199999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.766,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.6719999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:05:09 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.464,"15min":1.446},"min":{"1min":0.97599999999999998,"5min":0.92700000000000005,"15min":0.85699999999999998},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.3490000000000002},"last":2.2679999999999998},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.6020000000000001,"15min":1.583},"min":{"1min":1.109,"5min":1.109,"15min":0.75800000000000001},"max":{"1min":1.889,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.73}]},{"osd":7,"last update":"Mon Oct 16 11:05:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.5509999999999999,"15min":1.534},"min":{"1min":1.107,"5min":1.0860000000000001,"15min":1.04},"max":{"1min":1.774,"5min":1.9890000000000001,"15min":2.117},"last":1.9930000000000001},{"interface":"front","average":{"1min":1.657,"5min":1.639,"15min":1.625},"min":{"1min":1.3999999999999999,"5min":1.119,"15min":0.874},"max":{"1min":2.161,"5min":2.1829999999999998,"15min":2.2690000000000001},"last":2.7229999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575765,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47156,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324684,"statfs":{"total":96636764160,"available":96588476416,"internally_reserved":0,"allocated":2138112,"data_stored":1028089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.90100000000000002,"15min":0.90200000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.968},"last":1.071},{"interface":"front","average":{"1min":0.90000000000000002,"5min":1.0249999999999999,"15min":1.048},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0920000000000001,"5min":1.5800000000000001,"15min":2.1040000000000001},"last":1.0960000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.0129999999999999,"15min":0.998},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.351,"5min":1.5509999999999999,"15min":2.6269999999999998},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1200000000000001,"15min":1.1339999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.6299999999999999,"5min":1.651,"15min":2.2970000000000002},"last":0.84399999999999997}]},{"osd":2,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.101,"15min":1.1140000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3879999999999999,"5min":1.641,"15min":2.3359999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.198,"5min":1.208,"15min":1.22},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5740000000000001,"5min":1.623,"15min":1.74},"last":1.0209999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.367,"15min":1.343},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.724,"5min":1.74,"15min":2.1459999999999999},"last":1.139},{"interface":"front","average":{"1min":1.347,"5min":1.3540000000000001,"15min":1.347},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6319999999999999,"5min":1.7110000000000001,"15min":1.7889999999999999},"last":1.3799999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.379,"15min":1.385},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.97299999999999998},"max":{"1min":1.6699999999999999,"5min":1.897,"15min":2.4590000000000001},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.419,"15min":1.421},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8220000000000001,"5min":1.9910000000000001,"15min":2.089},"last":1.2390000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.5109999999999999,"15min":1.502},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.9510000000000001,"5min":2.3279999999999998,"15min":2.5760000000000001},"last":1.196},{"interface":"front","average":{"1min":1.486,"5min":1.522,"15min":1.522},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.0219999999999998,"5min":2.1509999999999998,"15min":2.4169999999999998},"last":1.3500000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.522,"15min":1.5269999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":2.29,"5min":2.29,"15min":2.746},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.635,"15min":1.643},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.8029999999999999},"last":1.8560000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575764,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51332,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320508,"statfs":{"total":96636764160,"available":96584200192,"internally_reserved":0,"allocated":2150400,"data_stored":1029681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.95599999999999996,"15min":0.93100000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.548,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.026,"15min":1.0680000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3540000000000001,"5min":1.5409999999999999,"15min":2.3010000000000002},"last":0.73299999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.1399999999999999,"15min":1.1080000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.657,"5min":1.7290000000000001,"15min":1.7749999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.121,"15min":1.1559999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.546,"5min":1.661,"15min":2.6120000000000001},"last":0.96799999999999997}]},{"osd":3,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.109,"15min":1.085},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.3600000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.159,"5min":1.2849999999999999,"15min":1.266},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6910000000000001,"5min":1.899,"15min":1.99},"last":1.2310000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.351,"15min":1.3899999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.728,"5min":1.8340000000000001,"15min":2.5190000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.3580000000000001,"15min":1.341},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.4729999999999999,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":0.92900000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.476,"15min":1.4490000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.2559999999999998,"5min":2.2570000000000001,"15min":2.282},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4610000000000001,"15min":1.4710000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.2120000000000002,"5min":2.3450000000000002,"15min":2.9649999999999999},"last":1.46}]},{"osd":6,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.4239999999999999,"15min":1.4039999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.929,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.569,"15min":1.5269999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.0819999999999999,"5min":2.3420000000000001,"15min":2.5139999999999998},"last":1.661}]},{"osd":7,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.538,"15min":1.538},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.1389999999999998,"5min":2.141,"15min":3.1669999999999998},"last":1.399},{"interface":"front","average":{"1min":1.47,"5min":1.5169999999999999,"15min":1.514},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.2469999999999999,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":1.486}]}]},{"osd":1,"up_from":13,"seq":55834575764,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59636,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312204,"statfs":{"total":96636764160,"available":96575696896,"internally_reserved":0,"allocated":2138112,"data_stored":1028581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:05:15 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.92600000000000005,"15min":0.92300000000000004},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5609999999999999,"5min":1.581,"15min":1.6819999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0509999999999999,"15min":1.0549999999999999},"min":{"1min":0.748,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.589,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.873}]},{"osd":2,"last update":"Mon Oct 16 11:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.97799999999999998,"15min":1.0089999999999999},"min":{"1min":0.74399999999999999,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.359,"5min":1.5349999999999999,"15min":1.8069999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.2709999999999999,"15min":1.2789999999999999},"min":{"1min":0.92600000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.5629999999999999,"5min":1.77,"15min":1.77},"last":1.046}]},{"osd":3,"last update":"Mon Oct 16 11:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2150000000000001,"15min":1.216},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.659,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1639999999999999,"15min":1.149},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5740000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.282}]},{"osd":4,"last update":"Mon Oct 16 11:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.266,"15min":1.298},"min":{"1min":1.0549999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.536,"5min":1.659,"15min":2.1030000000000002},"last":1.873},{"interface":"front","average":{"1min":1.427,"5min":1.4299999999999999,"15min":1.4139999999999999},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.93899999999999995},"max":{"1min":1.839,"5min":1.974,"15min":2.0110000000000001},"last":1.6140000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.415,"15min":1.3999999999999999},"min":{"1min":1.032,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.827,"5min":1.841,"15min":2.0720000000000001},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.393,"5min":1.421,"15min":1.395},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.92800000000000005},"max":{"1min":1.6759999999999999,"5min":2.8690000000000002,"15min":2.8690000000000002},"last":1.3660000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4830000000000001,"15min":1.4610000000000001},"min":{"1min":1.169,"5min":1.0980000000000001,"15min":1.0549999999999999},"max":{"1min":1.8859999999999999,"5min":2.0609999999999999,"15min":2.149},"last":1.8009999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.419,"15min":1.4410000000000001},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.6719999999999999,"5min":1.867,"15min":2.077},"last":1.5249999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.5209999999999999,"15min":1.534},"min":{"1min":1.147,"5min":1.147,"15min":1.105},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.494},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.5589999999999999,"15min":1.569},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":1.7649999999999999,"5min":2.3340000000000001,"15min":2.4289999999999998},"last":1.74}]}]},{"osd":0,"up_from":13,"seq":55834575765,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51916,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319924,"statfs":{"total":96636764160,"available":96583602176,"internally_reserved":0,"allocated":2744320,"data_stored":1620013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:05:08 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.95999999999999996,"15min":0.90900000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.077,"15min":1.0720000000000001},"min":{"1min":0.78900000000000003,"5min":0.68899999999999995,"15min":0.41199999999999998},"max":{"1min":1.663,"5min":1.714,"15min":1.714},"last":0.72499999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:05:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.1080000000000001,"15min":1.0940000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.749},"max":{"1min":1.504,"5min":1.625,"15min":1.6299999999999999},"last":1.351},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.159,"15min":1.1499999999999999},"min":{"1min":0.91800000000000004,"5min":0.746,"15min":0.47799999999999998},"max":{"1min":1.3520000000000001,"5min":1.823,"15min":1.823},"last":1.0049999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:05:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1319999999999999,"15min":1.077},"min":{"1min":0.88,"5min":0.74399999999999999,"15min":0.64300000000000002},"max":{"1min":1.484,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.246,"5min":1.3080000000000001,"15min":1.264},"min":{"1min":0.95199999999999996,"5min":0.873,"15min":0.82699999999999996},"max":{"1min":1.544,"5min":1.7689999999999999,"15min":1.8480000000000001},"last":1.4019999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:05:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.385,"15min":1.373},"min":{"1min":1.151,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.623,"5min":1.7010000000000001,"15min":1.7410000000000001},"last":1.2},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.3360000000000001,"15min":1.325},"min":{"1min":1.0509999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":1.7330000000000001,"5min":1.8169999999999999,"15min":1.97},"last":1.1220000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:05:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.393,"15min":1.3759999999999999},"min":{"1min":1.01,"5min":0.96399999999999997,"15min":0.95699999999999996},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":2.0259999999999998},"last":1.337},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.506,"15min":1.462},"min":{"1min":1.1519999999999999,"5min":1.1319999999999999,"15min":1.0669999999999999},"max":{"1min":1.7809999999999999,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.4610000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:05:08 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.5640000000000001,"15min":1.51},"min":{"1min":1.196,"5min":1.077,"15min":1.077},"max":{"1min":1.8620000000000001,"5min":2.286,"15min":2.286},"last":1.51},{"interface":"front","average":{"1min":1.494,"5min":1.5369999999999999,"15min":1.47},"min":{"1min":1.095,"5min":1.0580000000000001,"15min":1.038},"max":{"1min":2.0099999999999998,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.4199999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:05:08 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.589,"15min":1.5349999999999999},"min":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.0900000000000001},"max":{"1min":1.8999999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.49},{"interface":"front","average":{"1min":1.486,"5min":1.5369999999999999,"15min":1.482},"min":{"1min":1.196,"5min":1.1200000000000001,"15min":0.997},"max":{"1min":1.875,"5min":2.0259999999999998,"15min":2.0289999999999999},"last":1.175}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:05:48.328 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:05:48.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.569+0000 7f374bca2640 1 Processor -- start 2023-10-16T11:05:48.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.570+0000 7f374bca2640 1 -- start start 2023-10-16T11:05:48.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.571+0000 7f374bca2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f374415ae80 0x7f374415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:48.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.571+0000 7f374bca2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37440ce390 0x7f3744070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:48.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.571+0000 7f374bca2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f374415ed60 con 0x7f374415aaa0 2023-10-16T11:05:48.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.571+0000 7f374bca2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f374406b810 con 0x7f374415ae80 2023-10-16T11:05:48.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.571+0000 7f374bca2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37440707e0 con 0x7f37440ce390 2023-10-16T11:05:48.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.572+0000 7f3749a3e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f374415aaa0 0x7f37440cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34310/0 (socket says 172.21.15.84:34310) 2023-10-16T11:05:48.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.572+0000 7f3749a3e640 1 -- 172.21.15.84:0/2904088764 learned_addr learned my addr 172.21.15.84:0/2904088764 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:48.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.572+0000 7f374923d640 1 --2- 172.21.15.84:0/2904088764 >> v2:172.21.15.157:3300/0 conn(0x7f374415ae80 0x7f374415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:48.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.572+0000 7f3748a3c640 1 -- 172.21.15.84:0/2904088764 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956233492 0 0) 0x7f374415ed60 con 0x7f374415aaa0 2023-10-16T11:05:48.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.572+0000 7f3748a3c640 1 -- 172.21.15.84:0/2904088764 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3730003520 con 0x7f374415aaa0 2023-10-16T11:05:48.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.572+0000 7f374a23f640 1 --2- 172.21.15.84:0/2904088764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37440ce390 0x7f3744070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:48.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.572+0000 7f3748a3c640 1 -- 172.21.15.84:0/2904088764 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3389160459 0 0) 0x7f3730003520 con 0x7f374415aaa0 2023-10-16T11:05:48.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.572+0000 7f3748a3c640 1 -- 172.21.15.84:0/2904088764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37440ce390 msgr2=0x7f3744070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:48.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.572+0000 7f3748a3c640 1 --2- 172.21.15.84:0/2904088764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37440ce390 0x7f3744070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.572+0000 7f3748a3c640 1 -- 172.21.15.84:0/2904088764 >> v2:172.21.15.157:3300/0 conn(0x7f374415ae80 msgr2=0x7f374415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:48.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.572+0000 7f3748a3c640 1 --2- 172.21.15.84:0/2904088764 >> v2:172.21.15.157:3300/0 conn(0x7f374415ae80 0x7f374415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.572+0000 7f3748a3c640 1 -- 172.21.15.84:0/2904088764 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374415ef90 con 0x7f374415aaa0 2023-10-16T11:05:48.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.572+0000 7f3748a3c640 1 -- 172.21.15.84:0/2904088764 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f3740001690 con 0x7f374415aaa0 2023-10-16T11:05:48.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.573+0000 7f3748a3c640 1 -- 172.21.15.84:0/2904088764 >> v1:172.21.15.84:6789/0 conn(0x7f374415aaa0 legacy=0x7f37440cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:48.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.573+0000 7f3748a3c640 1 --2- 172.21.15.84:0/2904088764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3730003970 0x7f3730003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:48.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.573+0000 7f3748a3c640 1 --2- 172.21.15.84:0/2904088764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37300047c0 0x7f3730004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:48.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.573+0000 7f3748a3c640 1 --2- 172.21.15.84:0/2904088764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3730005180 0x7f373000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:48.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.573+0000 7f3748a3c640 1 -- 172.21.15.84:0/2904088764 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374406b810 con 0x7f3730003970 2023-10-16T11:05:48.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.573+0000 7f3748a3c640 1 -- 172.21.15.84:0/2904088764 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37440707e0 con 0x7f37300047c0 2023-10-16T11:05:48.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.573+0000 7f3748a3c640 1 -- 172.21.15.84:0/2904088764 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373000bef0 con 0x7f3730005180 2023-10-16T11:05:48.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.573+0000 7f374a23f640 1 --2- 172.21.15.84:0/2904088764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37440ce390 0x7f3744070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:05:48.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.573+0000 7f3749a3e640 1 --2- 172.21.15.84:0/2904088764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3730003970 0x7f3730003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:48.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.573+0000 7f374923d640 1 --2- 172.21.15.84:0/2904088764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37300047c0 0x7f3730004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:48.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.573+0000 7f3749a3e640 1 -- 172.21.15.84:0/2904088764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3730005180 msgr2=0x7f373000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:48.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.573+0000 7f3749a3e640 1 --2- 172.21.15.84:0/2904088764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3730005180 0x7f373000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.573+0000 7f3749a3e640 1 -- 172.21.15.84:0/2904088764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37300047c0 msgr2=0x7f3730004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:48.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.573+0000 7f3749a3e640 1 --2- 172.21.15.84:0/2904088764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37300047c0 0x7f3730004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.573+0000 7f3749a3e640 1 -- 172.21.15.84:0/2904088764 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3730003520 con 0x7f3730003970 2023-10-16T11:05:48.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.573+0000 7f374923d640 1 --2- 172.21.15.84:0/2904088764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37300047c0 0x7f3730004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:48.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.574+0000 7f3749a3e640 1 --2- 172.21.15.84:0/2904088764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3730003970 0x7f3730003d90 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f374000c210 tx=0x7f37300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e3a23743d2666c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:48.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.575+0000 7f3748a3c640 1 -- 172.21.15.84:0/2904088764 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740005ad0 con 0x7f3730003970 2023-10-16T11:05:48.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.575+0000 7f3748a3c640 1 -- 172.21.15.84:0/2904088764 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3740005c30 con 0x7f3730003970 2023-10-16T11:05:48.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.575+0000 7f3748a3c640 1 -- 172.21.15.84:0/2904088764 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374002c8a0 con 0x7f3730003970 2023-10-16T11:05:48.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.575+0000 7f374bca2640 1 -- 172.21.15.84:0/2904088764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3730003970 msgr2=0x7f3730003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:48.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.575+0000 7f374bca2640 1 --2- 172.21.15.84:0/2904088764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3730003970 0x7f3730003d90 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f374000c210 tx=0x7f37300057f0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.575+0000 7f374a23f640 1 -- 172.21.15.84:0/2904088764 reap_dead start 2023-10-16T11:05:48.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.575+0000 7f374bca2640 1 -- 172.21.15.84:0/2904088764 shutdown_connections 2023-10-16T11:05:48.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.575+0000 7f374bca2640 1 -- 172.21.15.84:0/2904088764 >> 172.21.15.84:0/2904088764 conn(0x7f37440cb410 msgr2=0x7f37440cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:48.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.575+0000 7f374bca2640 1 -- 172.21.15.84:0/2904088764 shutdown_connections 2023-10-16T11:05:48.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.575+0000 7f374bca2640 1 -- 172.21.15.84:0/2904088764 wait complete. 2023-10-16T11:05:48.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.577+0000 7f374bca2640 1 Processor -- start 2023-10-16T11:05:48.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.577+0000 7f374bca2640 1 -- start start 2023-10-16T11:05:48.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.578+0000 7f374bca2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3730003970 0x7f37441504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:48.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.578+0000 7f374bca2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37440c3ba0 0x7f37440c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:48.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.578+0000 7f374bca2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37440c4c20 0x7f3744152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:48.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.578+0000 7f374bca2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374415b320 con 0x7f37440c4c20 2023-10-16T11:05:48.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.578+0000 7f374bca2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37440c4020 con 0x7f3730003970 2023-10-16T11:05:48.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.578+0000 7f374bca2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374415b570 con 0x7f37440c3ba0 2023-10-16T11:05:48.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.579+0000 7f374a23f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37440c4c20 0x7f3744152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:48.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.579+0000 7f374a23f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37440c4c20 0x7f3744152710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54716/0 (socket says 172.21.15.84:54716) 2023-10-16T11:05:48.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.579+0000 7f374a23f640 1 -- 172.21.15.84:0/4066340325 learned_addr learned my addr 172.21.15.84:0/4066340325 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:48.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.579+0000 7f3749a3e640 1 --2- 172.21.15.84:0/4066340325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3730003970 0x7f37441504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:48.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.579+0000 7f374923d640 1 --2- 172.21.15.84:0/4066340325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37440c3ba0 0x7f37440c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:48.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.579+0000 7f374a23f640 1 -- 172.21.15.84:0/4066340325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37440c3ba0 msgr2=0x7f37440c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:48.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.579+0000 7f374a23f640 1 --2- 172.21.15.84:0/4066340325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37440c3ba0 0x7f37440c46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.579+0000 7f374a23f640 1 -- 172.21.15.84:0/4066340325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3730003970 msgr2=0x7f37441504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:48.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.579+0000 7f374a23f640 1 --2- 172.21.15.84:0/4066340325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3730003970 0x7f37441504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.579+0000 7f374a23f640 1 -- 172.21.15.84:0/4066340325 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374415b850 con 0x7f37440c4c20 2023-10-16T11:05:48.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.579+0000 7f3749a3e640 1 --2- 172.21.15.84:0/4066340325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3730003970 0x7f37441504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:48.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.579+0000 7f374923d640 1 --2- 172.21.15.84:0/4066340325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37440c3ba0 0x7f37440c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:48.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.579+0000 7f374a23f640 1 --2- 172.21.15.84:0/4066340325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37440c4c20 0x7f3744152710 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f37440ce840 tx=0x7f373c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:48.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.580+0000 7f373a7fc640 1 -- 172.21.15.84:0/4066340325 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c00dc90 con 0x7f37440c4c20 2023-10-16T11:05:48.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.580+0000 7f373a7fc640 1 -- 172.21.15.84:0/4066340325 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373c00f040 con 0x7f37440c4c20 2023-10-16T11:05:48.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.580+0000 7f373a7fc640 1 -- 172.21.15.84:0/4066340325 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c003990 con 0x7f37440c4c20 2023-10-16T11:05:48.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.580+0000 7f374bca2640 1 -- 172.21.15.84:0/4066340325 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37440c30e0 con 0x7f37440c4c20 2023-10-16T11:05:48.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.580+0000 7f374bca2640 1 -- 172.21.15.84:0/4066340325 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37440c36a0 con 0x7f37440c4c20 2023-10-16T11:05:48.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.581+0000 7f374bca2640 1 -- 172.21.15.84:0/4066340325 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f374404e160 con 0x7f37440c4c20 2023-10-16T11:05:48.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.582+0000 7f373a7fc640 1 -- 172.21.15.84:0/4066340325 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f373c003af0 con 0x7f37440c4c20 2023-10-16T11:05:48.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.583+0000 7f373a7fc640 1 --2- 172.21.15.84:0/4066340325 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f372806af90 0x7f372806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:48.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.584+0000 7f373a7fc640 1 -- 172.21.15.84:0/4066340325 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f373c089820 con 0x7f37440c4c20 2023-10-16T11:05:48.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.584+0000 7f3749a3e640 1 --2- 172.21.15.84:0/4066340325 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f372806af90 0x7f372806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:48.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.584+0000 7f3749a3e640 1 --2- 172.21.15.84:0/4066340325 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f372806af90 0x7f372806d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3740034000 tx=0x7f3740037040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:48.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.588+0000 7f373a7fc640 1 -- 172.21.15.84:0/4066340325 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f373c054680 con 0x7f37440c4c20 2023-10-16T11:05:48.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.775+0000 7f374bca2640 1 -- 172.21.15.84:0/4066340325 --> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f374415f550 con 0x7f372806af90 2023-10-16T11:05:48.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.798+0000 7f373a7fc640 1 -- 172.21.15.84:0/4066340325 <== mgr.6631 v2:172.21.15.84:6832/2442400184 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286866 (secure 0 0 0) 0x7f3740001680 con 0x7f372806af90 2023-10-16T11:05:48.803 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:48.805 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:05:48.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.805+0000 7f374bca2640 1 -- 172.21.15.84:0/4066340325 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f372806af90 msgr2=0x7f372806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:48.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.805+0000 7f374bca2640 1 --2- 172.21.15.84:0/4066340325 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f372806af90 0x7f372806d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3740034000 tx=0x7f3740037040 comp rx=0 tx=0).stop 2023-10-16T11:05:48.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.805+0000 7f374bca2640 1 -- 172.21.15.84:0/4066340325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37440c4c20 msgr2=0x7f3744152710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:48.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.805+0000 7f374bca2640 1 --2- 172.21.15.84:0/4066340325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37440c4c20 0x7f3744152710 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f37440ce840 tx=0x7f373c00d4a0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.805+0000 7f374bca2640 1 -- 172.21.15.84:0/4066340325 shutdown_connections 2023-10-16T11:05:48.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.805+0000 7f374bca2640 1 --2- 172.21.15.84:0/4066340325 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f372806af90 0x7f372806d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.805+0000 7f374bca2640 1 --2- 172.21.15.84:0/4066340325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37440c4c20 0x7f3744152710 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.805+0000 7f374bca2640 1 --2- 172.21.15.84:0/4066340325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37440c3ba0 0x7f37440c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.805+0000 7f374bca2640 1 --2- 172.21.15.84:0/4066340325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3730003970 0x7f37441504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:48.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.805+0000 7f374bca2640 1 -- 172.21.15.84:0/4066340325 >> 172.21.15.84:0/4066340325 conn(0x7f37440cb410 msgr2=0x7f3744154b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:48.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.805+0000 7f374bca2640 1 -- 172.21.15.84:0/4066340325 shutdown_connections 2023-10-16T11:05:48.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:48.805+0000 7f374bca2640 1 -- 172.21.15.84:0/4066340325 wait complete. 2023-10-16T11:05:48.827 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-16T11:05:48.183429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1504,"num_read_kb":3026,"num_write":485,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2728,"ondisk_log_size":2728,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":404776,"kb_used_data":17896,"kb_used_omap":12,"kb_used_meta":386739,"kb_avail":754569944,"statfs":{"total":773094113280,"available":772679622656,"internally_reserved":0,"allocated":18325504,"data_stored":9409548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":396021305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001343"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.345286+0000","last_change":"2023-10-16T11:05:43.939537+0000","last_active":"2023-10-16T11:05:45.345286+0000","last_peered":"2023-10-16T11:05:45.345286+0000","last_clean":"2023-10-16T11:05:45.345286+0000","last_became_active":"2023-10-16T11:05:43.939272+0000","last_became_peered":"2023-10-16T11:05:43.939272+0000","last_unstale":"2023-10-16T11:05:45.345286+0000","last_undegraded":"2023-10-16T11:05:45.345286+0000","last_fullsized":"2023-10-16T11:05:45.345286+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:55.821423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344015+0000","last_change":"2023-10-16T11:05:43.933909+0000","last_active":"2023-10-16T11:05:45.344015+0000","last_peered":"2023-10-16T11:05:45.344015+0000","last_clean":"2023-10-16T11:05:45.344015+0000","last_became_active":"2023-10-16T11:05:43.933264+0000","last_became_peered":"2023-10-16T11:05:43.933264+0000","last_unstale":"2023-10-16T11:05:45.344015+0000","last_undegraded":"2023-10-16T11:05:45.344015+0000","last_fullsized":"2023-10-16T11:05:45.344015+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:41.082759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.347016+0000","last_change":"2023-10-16T11:05:43.930945+0000","last_active":"2023-10-16T11:05:45.347016+0000","last_peered":"2023-10-16T11:05:45.347016+0000","last_clean":"2023-10-16T11:05:45.347016+0000","last_became_active":"2023-10-16T11:05:43.928911+0000","last_became_peered":"2023-10-16T11:05:43.928911+0000","last_unstale":"2023-10-16T11:05:45.347016+0000","last_undegraded":"2023-10-16T11:05:45.347016+0000","last_fullsized":"2023-10-16T11:05:45.347016+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:17:27.802465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.919486+0000","last_change":"2023-10-16T11:05:43.948973+0000","last_active":"2023-10-16T11:05:44.919486+0000","last_peered":"2023-10-16T11:05:44.919486+0000","last_clean":"2023-10-16T11:05:44.919486+0000","last_became_active":"2023-10-16T11:05:43.948685+0000","last_became_peered":"2023-10-16T11:05:43.948685+0000","last_unstale":"2023-10-16T11:05:44.919486+0000","last_undegraded":"2023-10-16T11:05:44.919486+0000","last_fullsized":"2023-10-16T11:05:44.919486+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:22:12.554802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343973+0000","last_change":"2023-10-16T11:05:43.944009+0000","last_active":"2023-10-16T11:05:45.343973+0000","last_peered":"2023-10-16T11:05:45.343973+0000","last_clean":"2023-10-16T11:05:45.343973+0000","last_became_active":"2023-10-16T11:05:43.942783+0000","last_became_peered":"2023-10-16T11:05:43.942783+0000","last_unstale":"2023-10-16T11:05:45.343973+0000","last_undegraded":"2023-10-16T11:05:45.343973+0000","last_fullsized":"2023-10-16T11:05:45.343973+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:24.594901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343340+0000","last_change":"2023-10-16T11:05:43.916350+0000","last_active":"2023-10-16T11:05:45.343340+0000","last_peered":"2023-10-16T11:05:45.343340+0000","last_clean":"2023-10-16T11:05:45.343340+0000","last_became_active":"2023-10-16T11:05:43.915708+0000","last_became_peered":"2023-10-16T11:05:43.915708+0000","last_unstale":"2023-10-16T11:05:45.343340+0000","last_undegraded":"2023-10-16T11:05:45.343340+0000","last_fullsized":"2023-10-16T11:05:45.343340+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:17.978137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.920347+0000","last_change":"2023-10-16T11:05:43.937982+0000","last_active":"2023-10-16T11:05:44.920347+0000","last_peered":"2023-10-16T11:05:44.920347+0000","last_clean":"2023-10-16T11:05:44.920347+0000","last_became_active":"2023-10-16T11:05:43.936784+0000","last_became_peered":"2023-10-16T11:05:43.936784+0000","last_unstale":"2023-10-16T11:05:44.920347+0000","last_undegraded":"2023-10-16T11:05:44.920347+0000","last_fullsized":"2023-10-16T11:05:44.920347+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:55:56.309108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343054+0000","last_change":"2023-10-16T11:05:43.937581+0000","last_active":"2023-10-16T11:05:45.343054+0000","last_peered":"2023-10-16T11:05:45.343054+0000","last_clean":"2023-10-16T11:05:45.343054+0000","last_became_active":"2023-10-16T11:05:43.937324+0000","last_became_peered":"2023-10-16T11:05:43.937324+0000","last_unstale":"2023-10-16T11:05:45.343054+0000","last_undegraded":"2023-10-16T11:05:45.343054+0000","last_fullsized":"2023-10-16T11:05:45.343054+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:43:05.123820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343345+0000","last_change":"2023-10-16T11:05:43.932617+0000","last_active":"2023-10-16T11:05:45.343345+0000","last_peered":"2023-10-16T11:05:45.343345+0000","last_clean":"2023-10-16T11:05:45.343345+0000","last_became_active":"2023-10-16T11:05:43.932064+0000","last_became_peered":"2023-10-16T11:05:43.932064+0000","last_unstale":"2023-10-16T11:05:45.343345+0000","last_undegraded":"2023-10-16T11:05:45.343345+0000","last_fullsized":"2023-10-16T11:05:45.343345+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:28.431551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343948+0000","last_change":"2023-10-16T11:05:43.941441+0000","last_active":"2023-10-16T11:05:45.343948+0000","last_peered":"2023-10-16T11:05:45.343948+0000","last_clean":"2023-10-16T11:05:45.343948+0000","last_became_active":"2023-10-16T11:05:43.939827+0000","last_became_peered":"2023-10-16T11:05:43.939827+0000","last_unstale":"2023-10-16T11:05:45.343948+0000","last_undegraded":"2023-10-16T11:05:45.343948+0000","last_fullsized":"2023-10-16T11:05:45.343948+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:28.255181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343897+0000","last_change":"2023-10-16T11:05:43.927590+0000","last_active":"2023-10-16T11:05:45.343897+0000","last_peered":"2023-10-16T11:05:45.343897+0000","last_clean":"2023-10-16T11:05:45.343897+0000","last_became_active":"2023-10-16T11:05:43.927181+0000","last_became_peered":"2023-10-16T11:05:43.927181+0000","last_unstale":"2023-10-16T11:05:45.343897+0000","last_undegraded":"2023-10-16T11:05:45.343897+0000","last_fullsized":"2023-10-16T11:05:45.343897+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:35.114622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.345612+0000","last_change":"2023-10-16T11:05:43.942366+0000","last_active":"2023-10-16T11:05:45.345612+0000","last_peered":"2023-10-16T11:05:45.345612+0000","last_clean":"2023-10-16T11:05:45.345612+0000","last_became_active":"2023-10-16T11:05:43.940902+0000","last_became_peered":"2023-10-16T11:05:43.940902+0000","last_unstale":"2023-10-16T11:05:45.345612+0000","last_undegraded":"2023-10-16T11:05:45.345612+0000","last_fullsized":"2023-10-16T11:05:45.345612+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:01.682559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344375+0000","last_change":"2023-10-16T11:05:43.945639+0000","last_active":"2023-10-16T11:05:45.344375+0000","last_peered":"2023-10-16T11:05:45.344375+0000","last_clean":"2023-10-16T11:05:45.344375+0000","last_became_active":"2023-10-16T11:05:43.941960+0000","last_became_peered":"2023-10-16T11:05:43.941960+0000","last_unstale":"2023-10-16T11:05:45.344375+0000","last_undegraded":"2023-10-16T11:05:45.344375+0000","last_fullsized":"2023-10-16T11:05:45.344375+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:21:09.268813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.919247+0000","last_change":"2023-10-16T11:05:43.948919+0000","last_active":"2023-10-16T11:05:44.919247+0000","last_peered":"2023-10-16T11:05:44.919247+0000","last_clean":"2023-10-16T11:05:44.919247+0000","last_became_active":"2023-10-16T11:05:43.948601+0000","last_became_peered":"2023-10-16T11:05:43.948601+0000","last_unstale":"2023-10-16T11:05:44.919247+0000","last_undegraded":"2023-10-16T11:05:44.919247+0000","last_fullsized":"2023-10-16T11:05:44.919247+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:05.610634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343134+0000","last_change":"2023-10-16T11:05:43.945422+0000","last_active":"2023-10-16T11:05:45.343134+0000","last_peered":"2023-10-16T11:05:45.343134+0000","last_clean":"2023-10-16T11:05:45.343134+0000","last_became_active":"2023-10-16T11:05:43.944371+0000","last_became_peered":"2023-10-16T11:05:43.944371+0000","last_unstale":"2023-10-16T11:05:45.343134+0000","last_undegraded":"2023-10-16T11:05:45.343134+0000","last_fullsized":"2023-10-16T11:05:45.343134+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:04:03.115562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.919222+0000","last_change":"2023-10-16T11:05:43.928395+0000","last_active":"2023-10-16T11:05:44.919222+0000","last_peered":"2023-10-16T11:05:44.919222+0000","last_clean":"2023-10-16T11:05:44.919222+0000","last_became_active":"2023-10-16T11:05:43.928096+0000","last_became_peered":"2023-10-16T11:05:43.928096+0000","last_unstale":"2023-10-16T11:05:44.919222+0000","last_undegraded":"2023-10-16T11:05:44.919222+0000","last_fullsized":"2023-10-16T11:05:44.919222+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:54:46.053648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.346435+0000","last_change":"2023-10-16T11:05:43.944326+0000","last_active":"2023-10-16T11:05:45.346435+0000","last_peered":"2023-10-16T11:05:45.346435+0000","last_clean":"2023-10-16T11:05:45.346435+0000","last_became_active":"2023-10-16T11:05:43.944055+0000","last_became_peered":"2023-10-16T11:05:43.944055+0000","last_unstale":"2023-10-16T11:05:45.346435+0000","last_undegraded":"2023-10-16T11:05:45.346435+0000","last_fullsized":"2023-10-16T11:05:45.346435+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:09.705333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.347507+0000","last_change":"2023-10-16T11:05:43.927446+0000","last_active":"2023-10-16T11:05:45.347507+0000","last_peered":"2023-10-16T11:05:45.347507+0000","last_clean":"2023-10-16T11:05:45.347507+0000","last_became_active":"2023-10-16T11:05:43.927120+0000","last_became_peered":"2023-10-16T11:05:43.927120+0000","last_unstale":"2023-10-16T11:05:45.347507+0000","last_undegraded":"2023-10-16T11:05:45.347507+0000","last_fullsized":"2023-10-16T11:05:45.347507+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:42:43.713631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344809+0000","last_change":"2023-10-16T11:05:43.932863+0000","last_active":"2023-10-16T11:05:45.344809+0000","last_peered":"2023-10-16T11:05:45.344809+0000","last_clean":"2023-10-16T11:05:45.344809+0000","last_became_active":"2023-10-16T11:05:43.932595+0000","last_became_peered":"2023-10-16T11:05:43.932595+0000","last_unstale":"2023-10-16T11:05:45.344809+0000","last_undegraded":"2023-10-16T11:05:45.344809+0000","last_fullsized":"2023-10-16T11:05:45.344809+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:19:35.567531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.920369+0000","last_change":"2023-10-16T11:05:43.939877+0000","last_active":"2023-10-16T11:05:44.920369+0000","last_peered":"2023-10-16T11:05:44.920369+0000","last_clean":"2023-10-16T11:05:44.920369+0000","last_became_active":"2023-10-16T11:05:43.936107+0000","last_became_peered":"2023-10-16T11:05:43.936107+0000","last_unstale":"2023-10-16T11:05:44.920369+0000","last_undegraded":"2023-10-16T11:05:44.920369+0000","last_fullsized":"2023-10-16T11:05:44.920369+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:31:27.542416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344655+0000","last_change":"2023-10-16T11:05:43.939808+0000","last_active":"2023-10-16T11:05:45.344655+0000","last_peered":"2023-10-16T11:05:45.344655+0000","last_clean":"2023-10-16T11:05:45.344655+0000","last_became_active":"2023-10-16T11:05:43.939504+0000","last_became_peered":"2023-10-16T11:05:43.939504+0000","last_unstale":"2023-10-16T11:05:45.344655+0000","last_undegraded":"2023-10-16T11:05:45.344655+0000","last_fullsized":"2023-10-16T11:05:45.344655+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:53:46.539221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.918878+0000","last_change":"2023-10-16T11:05:43.939090+0000","last_active":"2023-10-16T11:05:44.918878+0000","last_peered":"2023-10-16T11:05:44.918878+0000","last_clean":"2023-10-16T11:05:44.918878+0000","last_became_active":"2023-10-16T11:05:43.938744+0000","last_became_peered":"2023-10-16T11:05:43.938744+0000","last_unstale":"2023-10-16T11:05:44.918878+0000","last_undegraded":"2023-10-16T11:05:44.918878+0000","last_fullsized":"2023-10-16T11:05:44.918878+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:58:58.490380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.918633+0000","last_change":"2023-10-16T11:05:43.939354+0000","last_active":"2023-10-16T11:05:44.918633+0000","last_peered":"2023-10-16T11:05:44.918633+0000","last_clean":"2023-10-16T11:05:44.918633+0000","last_became_active":"2023-10-16T11:05:43.936886+0000","last_became_peered":"2023-10-16T11:05:43.936886+0000","last_unstale":"2023-10-16T11:05:44.918633+0000","last_undegraded":"2023-10-16T11:05:44.918633+0000","last_fullsized":"2023-10-16T11:05:44.918633+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:43.016362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.342966+0000","last_change":"2023-10-16T11:05:43.938894+0000","last_active":"2023-10-16T11:05:45.342966+0000","last_peered":"2023-10-16T11:05:45.342966+0000","last_clean":"2023-10-16T11:05:45.342966+0000","last_became_active":"2023-10-16T11:05:43.938181+0000","last_became_peered":"2023-10-16T11:05:43.938181+0000","last_unstale":"2023-10-16T11:05:45.342966+0000","last_undegraded":"2023-10-16T11:05:45.342966+0000","last_fullsized":"2023-10-16T11:05:45.342966+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:42.494223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.920701+0000","last_change":"2023-10-16T11:05:43.935650+0000","last_active":"2023-10-16T11:05:44.920701+0000","last_peered":"2023-10-16T11:05:44.920701+0000","last_clean":"2023-10-16T11:05:44.920701+0000","last_became_active":"2023-10-16T11:05:43.935268+0000","last_became_peered":"2023-10-16T11:05:43.935268+0000","last_unstale":"2023-10-16T11:05:44.920701+0000","last_undegraded":"2023-10-16T11:05:44.920701+0000","last_fullsized":"2023-10-16T11:05:44.920701+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:48:33.468683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344977+0000","last_change":"2023-10-16T11:05:43.932243+0000","last_active":"2023-10-16T11:05:45.344977+0000","last_peered":"2023-10-16T11:05:45.344977+0000","last_clean":"2023-10-16T11:05:45.344977+0000","last_became_active":"2023-10-16T11:05:43.931928+0000","last_became_peered":"2023-10-16T11:05:43.931928+0000","last_unstale":"2023-10-16T11:05:45.344977+0000","last_undegraded":"2023-10-16T11:05:45.344977+0000","last_fullsized":"2023-10-16T11:05:45.344977+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:42.729654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.920794+0000","last_change":"2023-10-16T11:05:43.943642+0000","last_active":"2023-10-16T11:05:44.920794+0000","last_peered":"2023-10-16T11:05:44.920794+0000","last_clean":"2023-10-16T11:05:44.920794+0000","last_became_active":"2023-10-16T11:05:43.943372+0000","last_became_peered":"2023-10-16T11:05:43.943372+0000","last_unstale":"2023-10-16T11:05:44.920794+0000","last_undegraded":"2023-10-16T11:05:44.920794+0000","last_fullsized":"2023-10-16T11:05:44.920794+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:24.594901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.918402+0000","last_change":"2023-10-16T11:05:43.955065+0000","last_active":"2023-10-16T11:05:44.918402+0000","last_peered":"2023-10-16T11:05:44.918402+0000","last_clean":"2023-10-16T11:05:44.918402+0000","last_became_active":"2023-10-16T11:05:43.954642+0000","last_became_peered":"2023-10-16T11:05:43.954642+0000","last_unstale":"2023-10-16T11:05:44.918402+0000","last_undegraded":"2023-10-16T11:05:44.918402+0000","last_fullsized":"2023-10-16T11:05:44.918402+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:52:05.468611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.342839+0000","last_change":"2023-10-16T11:05:43.917547+0000","last_active":"2023-10-16T11:05:45.342839+0000","last_peered":"2023-10-16T11:05:45.342839+0000","last_clean":"2023-10-16T11:05:45.342839+0000","last_became_active":"2023-10-16T11:05:43.914894+0000","last_became_peered":"2023-10-16T11:05:43.914894+0000","last_unstale":"2023-10-16T11:05:45.342839+0000","last_undegraded":"2023-10-16T11:05:45.342839+0000","last_fullsized":"2023-10-16T11:05:45.342839+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:14:22.417740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.919280+0000","last_change":"2023-10-16T11:05:43.934761+0000","last_active":"2023-10-16T11:05:44.919280+0000","last_peered":"2023-10-16T11:05:44.919280+0000","last_clean":"2023-10-16T11:05:44.919280+0000","last_became_active":"2023-10-16T11:05:43.934455+0000","last_became_peered":"2023-10-16T11:05:43.934455+0000","last_unstale":"2023-10-16T11:05:44.919280+0000","last_undegraded":"2023-10-16T11:05:44.919280+0000","last_fullsized":"2023-10-16T11:05:44.919280+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:44:02.763410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343261+0000","last_change":"2023-10-16T11:05:43.947823+0000","last_active":"2023-10-16T11:05:45.343261+0000","last_peered":"2023-10-16T11:05:45.343261+0000","last_clean":"2023-10-16T11:05:45.343261+0000","last_became_active":"2023-10-16T11:05:43.947537+0000","last_became_peered":"2023-10-16T11:05:43.947537+0000","last_unstale":"2023-10-16T11:05:45.343261+0000","last_undegraded":"2023-10-16T11:05:45.343261+0000","last_fullsized":"2023-10-16T11:05:45.343261+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:42:08.013977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.919882+0000","last_change":"2023-10-16T11:05:43.941896+0000","last_active":"2023-10-16T11:05:44.919882+0000","last_peered":"2023-10-16T11:05:44.919882+0000","last_clean":"2023-10-16T11:05:44.919882+0000","last_became_active":"2023-10-16T11:05:43.941643+0000","last_became_peered":"2023-10-16T11:05:43.941643+0000","last_unstale":"2023-10-16T11:05:44.919882+0000","last_undegraded":"2023-10-16T11:05:44.919882+0000","last_fullsized":"2023-10-16T11:05:44.919882+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:55:56.309108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344304+0000","last_change":"2023-10-16T11:05:43.941645+0000","last_active":"2023-10-16T11:05:45.344304+0000","last_peered":"2023-10-16T11:05:45.344304+0000","last_clean":"2023-10-16T11:05:45.344304+0000","last_became_active":"2023-10-16T11:05:43.939871+0000","last_became_peered":"2023-10-16T11:05:43.939871+0000","last_unstale":"2023-10-16T11:05:45.344304+0000","last_undegraded":"2023-10-16T11:05:45.344304+0000","last_fullsized":"2023-10-16T11:05:45.344304+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:13:03.474138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"170'4","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.950861+0000","last_change":"2023-10-16T11:05:42.918870+0000","last_active":"2023-10-16T11:05:43.950861+0000","last_peered":"2023-10-16T11:05:43.950861+0000","last_clean":"2023-10-16T11:05:43.950861+0000","last_became_active":"2023-10-16T11:05:42.918555+0000","last_became_peered":"2023-10-16T11:05:42.918555+0000","last_unstale":"2023-10-16T11:05:43.950861+0000","last_undegraded":"2023-10-16T11:05:43.950861+0000","last_fullsized":"2023-10-16T11:05:43.950861+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:07.474892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344027+0000","last_change":"2023-10-16T11:05:43.935112+0000","last_active":"2023-10-16T11:05:45.344027+0000","last_peered":"2023-10-16T11:05:45.344027+0000","last_clean":"2023-10-16T11:05:45.344027+0000","last_became_active":"2023-10-16T11:05:43.934858+0000","last_became_peered":"2023-10-16T11:05:43.934858+0000","last_unstale":"2023-10-16T11:05:45.344027+0000","last_undegraded":"2023-10-16T11:05:45.344027+0000","last_fullsized":"2023-10-16T11:05:45.344027+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:56.538637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.950534+0000","last_change":"2023-10-16T11:05:42.915253+0000","last_active":"2023-10-16T11:05:43.950534+0000","last_peered":"2023-10-16T11:05:43.950534+0000","last_clean":"2023-10-16T11:05:43.950534+0000","last_became_active":"2023-10-16T11:05:42.914890+0000","last_became_peered":"2023-10-16T11:05:42.914890+0000","last_unstale":"2023-10-16T11:05:43.950534+0000","last_undegraded":"2023-10-16T11:05:43.950534+0000","last_fullsized":"2023-10-16T11:05:43.950534+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:37.839005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343949+0000","last_change":"2023-10-16T11:05:43.942571+0000","last_active":"2023-10-16T11:05:45.343949+0000","last_peered":"2023-10-16T11:05:45.343949+0000","last_clean":"2023-10-16T11:05:45.343949+0000","last_became_active":"2023-10-16T11:05:43.942310+0000","last_became_peered":"2023-10-16T11:05:43.942310+0000","last_unstale":"2023-10-16T11:05:45.343949+0000","last_undegraded":"2023-10-16T11:05:45.343949+0000","last_fullsized":"2023-10-16T11:05:45.343949+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:34.728238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.955427+0000","last_change":"2023-10-16T11:05:42.905210+0000","last_active":"2023-10-16T11:05:43.955427+0000","last_peered":"2023-10-16T11:05:43.955427+0000","last_clean":"2023-10-16T11:05:43.955427+0000","last_became_active":"2023-10-16T11:05:42.904920+0000","last_became_peered":"2023-10-16T11:05:42.904920+0000","last_unstale":"2023-10-16T11:05:43.955427+0000","last_undegraded":"2023-10-16T11:05:43.955427+0000","last_fullsized":"2023-10-16T11:05:43.955427+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:03.159020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343606+0000","last_change":"2023-10-16T11:05:43.938972+0000","last_active":"2023-10-16T11:05:45.343606+0000","last_peered":"2023-10-16T11:05:45.343606+0000","last_clean":"2023-10-16T11:05:45.343606+0000","last_became_active":"2023-10-16T11:05:43.938616+0000","last_became_peered":"2023-10-16T11:05:43.938616+0000","last_unstale":"2023-10-16T11:05:45.343606+0000","last_undegraded":"2023-10-16T11:05:45.343606+0000","last_fullsized":"2023-10-16T11:05:45.343606+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:21:57.920762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.953885+0000","last_change":"2023-10-16T11:05:42.910283+0000","last_active":"2023-10-16T11:05:43.953885+0000","last_peered":"2023-10-16T11:05:43.953885+0000","last_clean":"2023-10-16T11:05:43.953885+0000","last_became_active":"2023-10-16T11:05:42.908996+0000","last_became_peered":"2023-10-16T11:05:42.908996+0000","last_unstale":"2023-10-16T11:05:43.953885+0000","last_undegraded":"2023-10-16T11:05:43.953885+0000","last_fullsized":"2023-10-16T11:05:43.953885+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:33:45.172836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343648+0000","last_change":"2023-10-16T11:05:43.948056+0000","last_active":"2023-10-16T11:05:45.343648+0000","last_peered":"2023-10-16T11:05:45.343648+0000","last_clean":"2023-10-16T11:05:45.343648+0000","last_became_active":"2023-10-16T11:05:43.947753+0000","last_became_peered":"2023-10-16T11:05:43.947753+0000","last_unstale":"2023-10-16T11:05:45.343648+0000","last_undegraded":"2023-10-16T11:05:45.343648+0000","last_fullsized":"2023-10-16T11:05:45.343648+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:42.329874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.952209+0000","last_change":"2023-10-16T11:05:42.913476+0000","last_active":"2023-10-16T11:05:43.952209+0000","last_peered":"2023-10-16T11:05:43.952209+0000","last_clean":"2023-10-16T11:05:43.952209+0000","last_became_active":"2023-10-16T11:05:42.913176+0000","last_became_peered":"2023-10-16T11:05:42.913176+0000","last_unstale":"2023-10-16T11:05:43.952209+0000","last_undegraded":"2023-10-16T11:05:43.952209+0000","last_fullsized":"2023-10-16T11:05:43.952209+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:05:11.461717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.919099+0000","last_change":"2023-10-16T11:05:43.948302+0000","last_active":"2023-10-16T11:05:44.919099+0000","last_peered":"2023-10-16T11:05:44.919099+0000","last_clean":"2023-10-16T11:05:44.919099+0000","last_became_active":"2023-10-16T11:05:43.948011+0000","last_became_peered":"2023-10-16T11:05:43.948011+0000","last_unstale":"2023-10-16T11:05:44.919099+0000","last_undegraded":"2023-10-16T11:05:44.919099+0000","last_fullsized":"2023-10-16T11:05:44.919099+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:33:09.375472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.951363+0000","last_change":"2023-10-16T11:05:42.915535+0000","last_active":"2023-10-16T11:05:43.951363+0000","last_peered":"2023-10-16T11:05:43.951363+0000","last_clean":"2023-10-16T11:05:43.951363+0000","last_became_active":"2023-10-16T11:05:42.915273+0000","last_became_peered":"2023-10-16T11:05:42.915273+0000","last_unstale":"2023-10-16T11:05:43.951363+0000","last_undegraded":"2023-10-16T11:05:43.951363+0000","last_fullsized":"2023-10-16T11:05:43.951363+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:33:10.465828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343151+0000","last_change":"2023-10-16T11:05:43.940800+0000","last_active":"2023-10-16T11:05:45.343151+0000","last_peered":"2023-10-16T11:05:45.343151+0000","last_clean":"2023-10-16T11:05:45.343151+0000","last_became_active":"2023-10-16T11:05:43.940519+0000","last_became_peered":"2023-10-16T11:05:43.940519+0000","last_unstale":"2023-10-16T11:05:45.343151+0000","last_undegraded":"2023-10-16T11:05:45.343151+0000","last_fullsized":"2023-10-16T11:05:45.343151+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:08:58.300877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.909282+0000","last_change":"2023-10-16T11:05:42.914001+0000","last_active":"2023-10-16T11:05:43.909282+0000","last_peered":"2023-10-16T11:05:43.909282+0000","last_clean":"2023-10-16T11:05:43.909282+0000","last_became_active":"2023-10-16T11:05:42.913713+0000","last_became_peered":"2023-10-16T11:05:42.913713+0000","last_unstale":"2023-10-16T11:05:43.909282+0000","last_undegraded":"2023-10-16T11:05:43.909282+0000","last_fullsized":"2023-10-16T11:05:43.909282+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:57:14.976665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.342994+0000","last_change":"2023-10-16T11:05:43.932283+0000","last_active":"2023-10-16T11:05:45.342994+0000","last_peered":"2023-10-16T11:05:45.342994+0000","last_clean":"2023-10-16T11:05:45.342994+0000","last_became_active":"2023-10-16T11:05:43.929314+0000","last_became_peered":"2023-10-16T11:05:43.929314+0000","last_unstale":"2023-10-16T11:05:45.342994+0000","last_undegraded":"2023-10-16T11:05:45.342994+0000","last_fullsized":"2023-10-16T11:05:45.342994+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:55:56.309108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.927424+0000","last_change":"2023-10-16T11:05:42.913285+0000","last_active":"2023-10-16T11:05:43.927424+0000","last_peered":"2023-10-16T11:05:43.927424+0000","last_clean":"2023-10-16T11:05:43.927424+0000","last_became_active":"2023-10-16T11:05:42.912925+0000","last_became_peered":"2023-10-16T11:05:42.912925+0000","last_unstale":"2023-10-16T11:05:43.927424+0000","last_undegraded":"2023-10-16T11:05:43.927424+0000","last_fullsized":"2023-10-16T11:05:43.927424+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:54.804384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.345683+0000","last_change":"2023-10-16T11:05:43.935465+0000","last_active":"2023-10-16T11:05:45.345683+0000","last_peered":"2023-10-16T11:05:45.345683+0000","last_clean":"2023-10-16T11:05:45.345683+0000","last_became_active":"2023-10-16T11:05:43.934444+0000","last_became_peered":"2023-10-16T11:05:43.934444+0000","last_unstale":"2023-10-16T11:05:45.345683+0000","last_undegraded":"2023-10-16T11:05:45.345683+0000","last_fullsized":"2023-10-16T11:05:45.345683+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:40:13.279771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.924030+0000","last_change":"2023-10-16T11:05:42.907377+0000","last_active":"2023-10-16T11:05:43.924030+0000","last_peered":"2023-10-16T11:05:43.924030+0000","last_clean":"2023-10-16T11:05:43.924030+0000","last_became_active":"2023-10-16T11:05:42.907101+0000","last_became_peered":"2023-10-16T11:05:42.907101+0000","last_unstale":"2023-10-16T11:05:43.924030+0000","last_undegraded":"2023-10-16T11:05:43.924030+0000","last_fullsized":"2023-10-16T11:05:43.924030+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:24:26.136046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.345898+0000","last_change":"2023-10-16T11:05:43.944147+0000","last_active":"2023-10-16T11:05:45.345898+0000","last_peered":"2023-10-16T11:05:45.345898+0000","last_clean":"2023-10-16T11:05:45.345898+0000","last_became_active":"2023-10-16T11:05:43.943883+0000","last_became_peered":"2023-10-16T11:05:43.943883+0000","last_unstale":"2023-10-16T11:05:45.345898+0000","last_undegraded":"2023-10-16T11:05:45.345898+0000","last_fullsized":"2023-10-16T11:05:45.345898+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:31:00.203499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.930738+0000","last_change":"2023-10-16T11:05:42.919730+0000","last_active":"2023-10-16T11:05:43.930738+0000","last_peered":"2023-10-16T11:05:43.930738+0000","last_clean":"2023-10-16T11:05:43.930738+0000","last_became_active":"2023-10-16T11:05:42.919466+0000","last_became_peered":"2023-10-16T11:05:42.919466+0000","last_unstale":"2023-10-16T11:05:43.930738+0000","last_undegraded":"2023-10-16T11:05:43.930738+0000","last_fullsized":"2023-10-16T11:05:43.930738+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:02:11.493187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.921053+0000","last_change":"2023-10-16T11:05:43.931226+0000","last_active":"2023-10-16T11:05:44.921053+0000","last_peered":"2023-10-16T11:05:44.921053+0000","last_clean":"2023-10-16T11:05:44.921053+0000","last_became_active":"2023-10-16T11:05:43.930927+0000","last_became_peered":"2023-10-16T11:05:43.930927+0000","last_unstale":"2023-10-16T11:05:44.921053+0000","last_undegraded":"2023-10-16T11:05:44.921053+0000","last_fullsized":"2023-10-16T11:05:44.921053+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:29.299026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"170'6","reported_seq":18,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.957237+0000","last_change":"2023-10-16T11:05:42.907360+0000","last_active":"2023-10-16T11:05:43.957237+0000","last_peered":"2023-10-16T11:05:43.957237+0000","last_clean":"2023-10-16T11:05:43.957237+0000","last_became_active":"2023-10-16T11:05:42.905385+0000","last_became_peered":"2023-10-16T11:05:42.905385+0000","last_unstale":"2023-10-16T11:05:43.957237+0000","last_undegraded":"2023-10-16T11:05:43.957237+0000","last_fullsized":"2023-10-16T11:05:43.957237+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:30:54.157804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.346662+0000","last_change":"2023-10-16T11:05:43.936449+0000","last_active":"2023-10-16T11:05:45.346662+0000","last_peered":"2023-10-16T11:05:45.346662+0000","last_clean":"2023-10-16T11:05:45.346662+0000","last_became_active":"2023-10-16T11:05:43.936085+0000","last_became_peered":"2023-10-16T11:05:43.936085+0000","last_unstale":"2023-10-16T11:05:45.346662+0000","last_undegraded":"2023-10-16T11:05:45.346662+0000","last_fullsized":"2023-10-16T11:05:45.346662+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:05:12.478756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.956239+0000","last_change":"2023-10-16T11:05:42.910401+0000","last_active":"2023-10-16T11:05:43.956239+0000","last_peered":"2023-10-16T11:05:43.956239+0000","last_clean":"2023-10-16T11:05:43.956239+0000","last_became_active":"2023-10-16T11:05:42.909868+0000","last_became_peered":"2023-10-16T11:05:42.909868+0000","last_unstale":"2023-10-16T11:05:43.956239+0000","last_undegraded":"2023-10-16T11:05:43.956239+0000","last_fullsized":"2023-10-16T11:05:43.956239+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:35:04.022368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343622+0000","last_change":"2023-10-16T11:05:43.937515+0000","last_active":"2023-10-16T11:05:45.343622+0000","last_peered":"2023-10-16T11:05:45.343622+0000","last_clean":"2023-10-16T11:05:45.343622+0000","last_became_active":"2023-10-16T11:05:43.936358+0000","last_became_peered":"2023-10-16T11:05:43.936358+0000","last_unstale":"2023-10-16T11:05:45.343622+0000","last_undegraded":"2023-10-16T11:05:45.343622+0000","last_fullsized":"2023-10-16T11:05:45.343622+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:31.492817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.908532+0000","last_change":"2023-10-16T11:05:42.908928+0000","last_active":"2023-10-16T11:05:43.908532+0000","last_peered":"2023-10-16T11:05:43.908532+0000","last_clean":"2023-10-16T11:05:43.908532+0000","last_became_active":"2023-10-16T11:05:42.908594+0000","last_became_peered":"2023-10-16T11:05:42.908594+0000","last_unstale":"2023-10-16T11:05:43.908532+0000","last_undegraded":"2023-10-16T11:05:43.908532+0000","last_fullsized":"2023-10-16T11:05:43.908532+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:28.830200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343221+0000","last_change":"2023-10-16T11:05:43.942743+0000","last_active":"2023-10-16T11:05:45.343221+0000","last_peered":"2023-10-16T11:05:45.343221+0000","last_clean":"2023-10-16T11:05:45.343221+0000","last_became_active":"2023-10-16T11:05:43.942382+0000","last_became_peered":"2023-10-16T11:05:43.942382+0000","last_unstale":"2023-10-16T11:05:45.343221+0000","last_undegraded":"2023-10-16T11:05:45.343221+0000","last_fullsized":"2023-10-16T11:05:45.343221+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:31:27.542416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.906948+0000","last_change":"2023-10-16T11:05:42.905562+0000","last_active":"2023-10-16T11:05:43.906948+0000","last_peered":"2023-10-16T11:05:43.906948+0000","last_clean":"2023-10-16T11:05:43.906948+0000","last_became_active":"2023-10-16T11:05:42.905127+0000","last_became_peered":"2023-10-16T11:05:42.905127+0000","last_unstale":"2023-10-16T11:05:43.906948+0000","last_undegraded":"2023-10-16T11:05:43.906948+0000","last_fullsized":"2023-10-16T11:05:43.906948+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:46.130329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.346162+0000","last_change":"2023-10-16T11:05:43.924551+0000","last_active":"2023-10-16T11:05:45.346162+0000","last_peered":"2023-10-16T11:05:45.346162+0000","last_clean":"2023-10-16T11:05:45.346162+0000","last_became_active":"2023-10-16T11:05:43.923368+0000","last_became_peered":"2023-10-16T11:05:43.923368+0000","last_unstale":"2023-10-16T11:05:45.346162+0000","last_undegraded":"2023-10-16T11:05:45.346162+0000","last_fullsized":"2023-10-16T11:05:45.346162+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:55.571221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"170'8","reported_seq":20,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.956853+0000","last_change":"2023-10-16T11:05:42.899908+0000","last_active":"2023-10-16T11:05:43.956853+0000","last_peered":"2023-10-16T11:05:43.956853+0000","last_clean":"2023-10-16T11:05:43.956853+0000","last_became_active":"2023-10-16T11:05:42.899636+0000","last_became_peered":"2023-10-16T11:05:42.899636+0000","last_unstale":"2023-10-16T11:05:43.956853+0000","last_undegraded":"2023-10-16T11:05:43.956853+0000","last_fullsized":"2023-10-16T11:05:43.956853+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:17.635993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344846+0000","last_change":"2023-10-16T11:05:43.942344+0000","last_active":"2023-10-16T11:05:45.344846+0000","last_peered":"2023-10-16T11:05:45.344846+0000","last_clean":"2023-10-16T11:05:45.344846+0000","last_became_active":"2023-10-16T11:05:43.942027+0000","last_became_peered":"2023-10-16T11:05:43.942027+0000","last_unstale":"2023-10-16T11:05:45.344846+0000","last_undegraded":"2023-10-16T11:05:45.344846+0000","last_fullsized":"2023-10-16T11:05:45.344846+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:36:27.710717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.953486+0000","last_change":"2023-10-16T11:05:42.914096+0000","last_active":"2023-10-16T11:05:43.953486+0000","last_peered":"2023-10-16T11:05:43.953486+0000","last_clean":"2023-10-16T11:05:43.953486+0000","last_became_active":"2023-10-16T11:05:42.913780+0000","last_became_peered":"2023-10-16T11:05:42.913780+0000","last_unstale":"2023-10-16T11:05:43.953486+0000","last_undegraded":"2023-10-16T11:05:43.953486+0000","last_fullsized":"2023-10-16T11:05:43.953486+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:55:45.317520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.920922+0000","last_change":"2023-10-16T11:05:43.946265+0000","last_active":"2023-10-16T11:05:44.920922+0000","last_peered":"2023-10-16T11:05:44.920922+0000","last_clean":"2023-10-16T11:05:44.920922+0000","last_became_active":"2023-10-16T11:05:43.945202+0000","last_became_peered":"2023-10-16T11:05:43.945202+0000","last_unstale":"2023-10-16T11:05:44.920922+0000","last_undegraded":"2023-10-16T11:05:44.920922+0000","last_fullsized":"2023-10-16T11:05:44.920922+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:21:09.268813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.954360+0000","last_change":"2023-10-16T11:05:42.910188+0000","last_active":"2023-10-16T11:05:43.954360+0000","last_peered":"2023-10-16T11:05:43.954360+0000","last_clean":"2023-10-16T11:05:43.954360+0000","last_became_active":"2023-10-16T11:05:42.909869+0000","last_became_peered":"2023-10-16T11:05:42.909869+0000","last_unstale":"2023-10-16T11:05:43.954360+0000","last_undegraded":"2023-10-16T11:05:43.954360+0000","last_fullsized":"2023-10-16T11:05:43.954360+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:15:38.581080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.346675+0000","last_change":"2023-10-16T11:05:43.939770+0000","last_active":"2023-10-16T11:05:45.346675+0000","last_peered":"2023-10-16T11:05:45.346675+0000","last_clean":"2023-10-16T11:05:45.346675+0000","last_became_active":"2023-10-16T11:05:43.937396+0000","last_became_peered":"2023-10-16T11:05:43.937396+0000","last_unstale":"2023-10-16T11:05:45.346675+0000","last_undegraded":"2023-10-16T11:05:45.346675+0000","last_fullsized":"2023-10-16T11:05:45.346675+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:57:15.993704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.951769+0000","last_change":"2023-10-16T11:05:42.911198+0000","last_active":"2023-10-16T11:05:43.951769+0000","last_peered":"2023-10-16T11:05:43.951769+0000","last_clean":"2023-10-16T11:05:43.951769+0000","last_became_active":"2023-10-16T11:05:42.910923+0000","last_became_peered":"2023-10-16T11:05:42.910923+0000","last_unstale":"2023-10-16T11:05:43.951769+0000","last_undegraded":"2023-10-16T11:05:43.951769+0000","last_fullsized":"2023-10-16T11:05:43.951769+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:22:20.697602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344408+0000","last_change":"2023-10-16T11:05:43.918404+0000","last_active":"2023-10-16T11:05:45.344408+0000","last_peered":"2023-10-16T11:05:45.344408+0000","last_clean":"2023-10-16T11:05:45.344408+0000","last_became_active":"2023-10-16T11:05:43.917782+0000","last_became_peered":"2023-10-16T11:05:43.917782+0000","last_unstale":"2023-10-16T11:05:45.344408+0000","last_undegraded":"2023-10-16T11:05:45.344408+0000","last_fullsized":"2023-10-16T11:05:45.344408+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:54:51.097482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.927489+0000","last_change":"2023-10-16T11:05:42.905615+0000","last_active":"2023-10-16T11:05:43.927489+0000","last_peered":"2023-10-16T11:05:43.927489+0000","last_clean":"2023-10-16T11:05:43.927489+0000","last_became_active":"2023-10-16T11:05:42.904492+0000","last_became_peered":"2023-10-16T11:05:42.904492+0000","last_unstale":"2023-10-16T11:05:43.927489+0000","last_undegraded":"2023-10-16T11:05:43.927489+0000","last_fullsized":"2023-10-16T11:05:43.927489+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:46:07.003568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.919628+0000","last_change":"2023-10-16T11:05:43.933380+0000","last_active":"2023-10-16T11:05:44.919628+0000","last_peered":"2023-10-16T11:05:44.919628+0000","last_clean":"2023-10-16T11:05:44.919628+0000","last_became_active":"2023-10-16T11:05:43.932936+0000","last_became_peered":"2023-10-16T11:05:43.932936+0000","last_unstale":"2023-10-16T11:05:44.919628+0000","last_undegraded":"2023-10-16T11:05:44.919628+0000","last_fullsized":"2023-10-16T11:05:44.919628+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:19.045384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.954625+0000","last_change":"2023-10-16T11:05:42.912394+0000","last_active":"2023-10-16T11:05:43.954625+0000","last_peered":"2023-10-16T11:05:43.954625+0000","last_clean":"2023-10-16T11:05:43.954625+0000","last_became_active":"2023-10-16T11:05:42.912102+0000","last_became_peered":"2023-10-16T11:05:42.912102+0000","last_unstale":"2023-10-16T11:05:43.954625+0000","last_undegraded":"2023-10-16T11:05:43.954625+0000","last_fullsized":"2023-10-16T11:05:43.954625+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:17.635993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.918678+0000","last_change":"2023-10-16T11:05:43.939980+0000","last_active":"2023-10-16T11:05:44.918678+0000","last_peered":"2023-10-16T11:05:44.918678+0000","last_clean":"2023-10-16T11:05:44.918678+0000","last_became_active":"2023-10-16T11:05:43.937118+0000","last_became_peered":"2023-10-16T11:05:43.937118+0000","last_unstale":"2023-10-16T11:05:44.918678+0000","last_undegraded":"2023-10-16T11:05:44.918678+0000","last_fullsized":"2023-10-16T11:05:44.918678+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:47.147368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.930738+0000","last_change":"2023-10-16T11:05:42.918022+0000","last_active":"2023-10-16T11:05:43.930738+0000","last_peered":"2023-10-16T11:05:43.930738+0000","last_clean":"2023-10-16T11:05:43.930738+0000","last_became_active":"2023-10-16T11:05:42.917723+0000","last_became_peered":"2023-10-16T11:05:42.917723+0000","last_unstale":"2023-10-16T11:05:43.930738+0000","last_undegraded":"2023-10-16T11:05:43.930738+0000","last_fullsized":"2023-10-16T11:05:43.930738+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:56:40.269324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.918317+0000","last_change":"2023-10-16T11:05:43.940159+0000","last_active":"2023-10-16T11:05:44.918317+0000","last_peered":"2023-10-16T11:05:44.918317+0000","last_clean":"2023-10-16T11:05:44.918317+0000","last_became_active":"2023-10-16T11:05:43.936836+0000","last_became_peered":"2023-10-16T11:05:43.936836+0000","last_unstale":"2023-10-16T11:05:44.918317+0000","last_undegraded":"2023-10-16T11:05:44.918317+0000","last_fullsized":"2023-10-16T11:05:44.918317+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:52.352439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.923355+0000","last_change":"2023-10-16T11:05:42.916214+0000","last_active":"2023-10-16T11:05:43.923355+0000","last_peered":"2023-10-16T11:05:43.923355+0000","last_clean":"2023-10-16T11:05:43.923355+0000","last_became_active":"2023-10-16T11:05:42.914986+0000","last_became_peered":"2023-10-16T11:05:42.914986+0000","last_unstale":"2023-10-16T11:05:43.923355+0000","last_undegraded":"2023-10-16T11:05:43.923355+0000","last_fullsized":"2023-10-16T11:05:43.923355+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:19:53.282121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.918971+0000","last_change":"2023-10-16T11:05:43.939592+0000","last_active":"2023-10-16T11:05:44.918971+0000","last_peered":"2023-10-16T11:05:44.918971+0000","last_clean":"2023-10-16T11:05:44.918971+0000","last_became_active":"2023-10-16T11:05:43.939217+0000","last_became_peered":"2023-10-16T11:05:43.939217+0000","last_unstale":"2023-10-16T11:05:44.918971+0000","last_undegraded":"2023-10-16T11:05:44.918971+0000","last_fullsized":"2023-10-16T11:05:44.918971+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:09:59.897371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.908081+0000","last_change":"2023-10-16T11:05:42.905792+0000","last_active":"2023-10-16T11:05:43.908081+0000","last_peered":"2023-10-16T11:05:43.908081+0000","last_clean":"2023-10-16T11:05:43.908081+0000","last_became_active":"2023-10-16T11:05:42.905502+0000","last_became_peered":"2023-10-16T11:05:42.905502+0000","last_unstale":"2023-10-16T11:05:43.908081+0000","last_undegraded":"2023-10-16T11:05:43.908081+0000","last_fullsized":"2023-10-16T11:05:43.908081+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:06.715283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.346884+0000","last_change":"2023-10-16T11:05:43.945177+0000","last_active":"2023-10-16T11:05:45.346884+0000","last_peered":"2023-10-16T11:05:45.346884+0000","last_clean":"2023-10-16T11:05:45.346884+0000","last_became_active":"2023-10-16T11:05:43.943724+0000","last_became_peered":"2023-10-16T11:05:43.943724+0000","last_unstale":"2023-10-16T11:05:45.346884+0000","last_undegraded":"2023-10-16T11:05:45.346884+0000","last_fullsized":"2023-10-16T11:05:45.346884+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:48:19.683014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.953621+0000","last_change":"2023-10-16T11:05:42.918054+0000","last_active":"2023-10-16T11:05:43.953621+0000","last_peered":"2023-10-16T11:05:43.953621+0000","last_clean":"2023-10-16T11:05:43.953621+0000","last_became_active":"2023-10-16T11:05:42.917765+0000","last_became_peered":"2023-10-16T11:05:42.917765+0000","last_unstale":"2023-10-16T11:05:43.953621+0000","last_undegraded":"2023-10-16T11:05:43.953621+0000","last_fullsized":"2023-10-16T11:05:43.953621+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:21:51.753638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.348058+0000","last_change":"2023-10-16T11:05:43.936490+0000","last_active":"2023-10-16T11:05:45.348058+0000","last_peered":"2023-10-16T11:05:45.348058+0000","last_clean":"2023-10-16T11:05:45.348058+0000","last_became_active":"2023-10-16T11:05:43.936191+0000","last_became_peered":"2023-10-16T11:05:43.936191+0000","last_unstale":"2023-10-16T11:05:45.348058+0000","last_undegraded":"2023-10-16T11:05:45.348058+0000","last_fullsized":"2023-10-16T11:05:45.348058+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:54:34.506511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.910783+0000","last_change":"2023-10-16T11:05:42.917331+0000","last_active":"2023-10-16T11:05:43.910783+0000","last_peered":"2023-10-16T11:05:43.910783+0000","last_clean":"2023-10-16T11:05:43.910783+0000","last_became_active":"2023-10-16T11:05:42.916992+0000","last_became_peered":"2023-10-16T11:05:42.916992+0000","last_unstale":"2023-10-16T11:05:43.910783+0000","last_undegraded":"2023-10-16T11:05:43.910783+0000","last_fullsized":"2023-10-16T11:05:43.910783+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:48:18.665975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344217+0000","last_change":"2023-10-16T11:05:43.951151+0000","last_active":"2023-10-16T11:05:45.344217+0000","last_peered":"2023-10-16T11:05:45.344217+0000","last_clean":"2023-10-16T11:05:45.344217+0000","last_became_active":"2023-10-16T11:05:43.949664+0000","last_became_peered":"2023-10-16T11:05:43.949664+0000","last_unstale":"2023-10-16T11:05:45.344217+0000","last_undegraded":"2023-10-16T11:05:45.344217+0000","last_fullsized":"2023-10-16T11:05:45.344217+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:10:24.868935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.914278+0000","last_change":"2023-10-16T11:05:42.905603+0000","last_active":"2023-10-16T11:05:43.914278+0000","last_peered":"2023-10-16T11:05:43.914278+0000","last_clean":"2023-10-16T11:05:43.914278+0000","last_became_active":"2023-10-16T11:05:42.904897+0000","last_became_peered":"2023-10-16T11:05:42.904897+0000","last_unstale":"2023-10-16T11:05:43.914278+0000","last_undegraded":"2023-10-16T11:05:43.914278+0000","last_fullsized":"2023-10-16T11:05:43.914278+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:54:45.039479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.344408+0000","last_change":"2023-10-16T11:05:43.929675+0000","last_active":"2023-10-16T11:05:45.344408+0000","last_peered":"2023-10-16T11:05:45.344408+0000","last_clean":"2023-10-16T11:05:45.344408+0000","last_became_active":"2023-10-16T11:05:43.928934+0000","last_became_peered":"2023-10-16T11:05:43.928934+0000","last_unstale":"2023-10-16T11:05:45.344408+0000","last_undegraded":"2023-10-16T11:05:45.344408+0000","last_fullsized":"2023-10-16T11:05:45.344408+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:07:20.042750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.908337+0000","last_change":"2023-10-16T11:05:42.907090+0000","last_active":"2023-10-16T11:05:43.908337+0000","last_peered":"2023-10-16T11:05:43.908337+0000","last_clean":"2023-10-16T11:05:43.908337+0000","last_became_active":"2023-10-16T11:05:42.906778+0000","last_became_peered":"2023-10-16T11:05:42.906778+0000","last_unstale":"2023-10-16T11:05:43.908337+0000","last_undegraded":"2023-10-16T11:05:43.908337+0000","last_fullsized":"2023-10-16T11:05:43.908337+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:28.281987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.919114+0000","last_change":"2023-10-16T11:05:43.933007+0000","last_active":"2023-10-16T11:05:44.919114+0000","last_peered":"2023-10-16T11:05:44.919114+0000","last_clean":"2023-10-16T11:05:44.919114+0000","last_became_active":"2023-10-16T11:05:43.930134+0000","last_became_peered":"2023-10-16T11:05:43.930134+0000","last_unstale":"2023-10-16T11:05:44.919114+0000","last_undegraded":"2023-10-16T11:05:44.919114+0000","last_fullsized":"2023-10-16T11:05:44.919114+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:47.147368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.926123+0000","last_change":"2023-10-16T11:05:42.908481+0000","last_active":"2023-10-16T11:05:43.926123+0000","last_peered":"2023-10-16T11:05:43.926123+0000","last_clean":"2023-10-16T11:05:43.926123+0000","last_became_active":"2023-10-16T11:05:42.908207+0000","last_became_peered":"2023-10-16T11:05:42.908207+0000","last_unstale":"2023-10-16T11:05:43.926123+0000","last_undegraded":"2023-10-16T11:05:43.926123+0000","last_fullsized":"2023-10-16T11:05:43.926123+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:33:10.465828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.345073+0000","last_change":"2023-10-16T11:05:43.949130+0000","last_active":"2023-10-16T11:05:45.345073+0000","last_peered":"2023-10-16T11:05:45.345073+0000","last_clean":"2023-10-16T11:05:45.345073+0000","last_became_active":"2023-10-16T11:05:43.948805+0000","last_became_peered":"2023-10-16T11:05:43.948805+0000","last_unstale":"2023-10-16T11:05:45.345073+0000","last_undegraded":"2023-10-16T11:05:45.345073+0000","last_fullsized":"2023-10-16T11:05:45.345073+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:29.299026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.930867+0000","last_change":"2023-10-16T11:05:42.917987+0000","last_active":"2023-10-16T11:05:43.930867+0000","last_peered":"2023-10-16T11:05:43.930867+0000","last_clean":"2023-10-16T11:05:43.930867+0000","last_became_active":"2023-10-16T11:05:42.917681+0000","last_became_peered":"2023-10-16T11:05:42.917681+0000","last_unstale":"2023-10-16T11:05:43.930867+0000","last_undegraded":"2023-10-16T11:05:43.930867+0000","last_fullsized":"2023-10-16T11:05:43.930867+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:16:00.770358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343147+0000","last_change":"2023-10-16T11:05:43.941693+0000","last_active":"2023-10-16T11:05:45.343147+0000","last_peered":"2023-10-16T11:05:45.343147+0000","last_clean":"2023-10-16T11:05:45.343147+0000","last_became_active":"2023-10-16T11:05:43.941098+0000","last_became_peered":"2023-10-16T11:05:43.941098+0000","last_unstale":"2023-10-16T11:05:45.343147+0000","last_undegraded":"2023-10-16T11:05:45.343147+0000","last_fullsized":"2023-10-16T11:05:45.343147+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:02:17.916946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.907635+0000","last_change":"2023-10-16T11:05:42.907004+0000","last_active":"2023-10-16T11:05:43.907635+0000","last_peered":"2023-10-16T11:05:43.907635+0000","last_clean":"2023-10-16T11:05:43.907635+0000","last_became_active":"2023-10-16T11:05:42.906721+0000","last_became_peered":"2023-10-16T11:05:42.906721+0000","last_unstale":"2023-10-16T11:05:43.907635+0000","last_undegraded":"2023-10-16T11:05:43.907635+0000","last_fullsized":"2023-10-16T11:05:43.907635+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:39.259972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:45.343687+0000","last_change":"2023-10-16T11:05:43.932513+0000","last_active":"2023-10-16T11:05:45.343687+0000","last_peered":"2023-10-16T11:05:45.343687+0000","last_clean":"2023-10-16T11:05:45.343687+0000","last_became_active":"2023-10-16T11:05:43.931958+0000","last_became_peered":"2023-10-16T11:05:43.931958+0000","last_unstale":"2023-10-16T11:05:45.343687+0000","last_undegraded":"2023-10-16T11:05:45.343687+0000","last_fullsized":"2023-10-16T11:05:45.343687+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:48:33.468683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.921145+0000","last_change":"2023-10-16T11:05:42.912613+0000","last_active":"2023-10-16T11:05:43.921145+0000","last_peered":"2023-10-16T11:05:43.921145+0000","last_clean":"2023-10-16T11:05:43.921145+0000","last_became_active":"2023-10-16T11:05:42.912139+0000","last_became_peered":"2023-10-16T11:05:42.912139+0000","last_unstale":"2023-10-16T11:05:43.921145+0000","last_undegraded":"2023-10-16T11:05:43.921145+0000","last_fullsized":"2023-10-16T11:05:43.921145+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:23:19.947376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-16T11:05:44.918701+0000","last_change":"2023-10-16T11:05:43.953051+0000","last_active":"2023-10-16T11:05:44.918701+0000","last_peered":"2023-10-16T11:05:44.918701+0000","last_clean":"2023-10-16T11:05:44.918701+0000","last_became_active":"2023-10-16T11:05:43.952652+0000","last_became_peered":"2023-10-16T11:05:43.952652+0000","last_unstale":"2023-10-16T11:05:44.918701+0000","last_undegraded":"2023-10-16T11:05:44.918701+0000","last_fullsized":"2023-10-16T11:05:44.918701+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:42.874338+0000","last_clean_scrub_stamp":"2023-10-16T11:05:42.874338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:54:46.053648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"170'3","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.956300+0000","last_change":"2023-10-16T11:05:42.905639+0000","last_active":"2023-10-16T11:05:43.956300+0000","last_peered":"2023-10-16T11:05:43.956300+0000","last_clean":"2023-10-16T11:05:43.956300+0000","last_became_active":"2023-10-16T11:05:42.905299+0000","last_became_peered":"2023-10-16T11:05:42.905299+0000","last_unstale":"2023-10-16T11:05:43.956300+0000","last_undegraded":"2023-10-16T11:05:43.956300+0000","last_fullsized":"2023-10-16T11:05:43.956300+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:05:41.857299+0000","last_clean_scrub_stamp":"2023-10-16T11:05:41.857299+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:00.924601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"159'2690","reported_seq":3331,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-16T11:05:43.927088+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:05:43.927088+0000","last_peered":"2023-10-16T11:05:43.927088+0000","last_clean":"2023-10-16T11:05:43.927088+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:05:43.927088+0000","last_undegraded":"2023-10-16T11:05:43.927088+0000","last_fullsized":"2023-10-16T11:05:43.927088+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2690,"log_dups_size":0,"ondisk_log_size":2690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1504,"num_read_kb":3026,"num_write":441,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1504,"num_read_kb":3026,"num_write":441,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2690,"ondisk_log_size":2690,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575765,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64384,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307456,"statfs":{"total":96636764160,"available":96570834944,"internally_reserved":0,"allocated":2736128,"data_stored":1619019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":63175115},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:05:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.1739999999999999,"15min":1.2090000000000001},"min":{"1min":0.86699999999999999,"5min":0.76600000000000001,"15min":0.69499999999999995},"max":{"1min":1.577,"5min":1.7230000000000001,"15min":1.853},"last":1.165},{"interface":"front","average":{"1min":1.282,"5min":1.2390000000000001,"15min":1.2370000000000001},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.67300000000000004},"max":{"1min":1.6459999999999999,"5min":1.8029999999999999,"15min":1.9710000000000001},"last":1.365}]},{"osd":1,"last update":"Mon Oct 16 11:05:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2549999999999999,"15min":1.2629999999999999},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.78200000000000003},"max":{"1min":1.796,"5min":1.796,"15min":1.907},"last":1.226},{"interface":"front","average":{"1min":1.325,"5min":1.2869999999999999,"15min":1.28},"min":{"1min":1.0700000000000001,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.613,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.444}]},{"osd":2,"last update":"Mon Oct 16 11:05:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.367,"15min":1.3169999999999999},"min":{"1min":0.97299999999999998,"5min":0.93000000000000005,"15min":0.81899999999999995},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.125},"last":1.494},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.379,"15min":1.351},"min":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.79100000000000004},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":1.0149999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:05:12 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.395,"15min":1.3839999999999999},"min":{"1min":1.109,"5min":0.98899999999999999,"15min":0.96599999999999997},"max":{"1min":1.7529999999999999,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.518,"15min":1.458},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.8879999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.397}]},{"osd":4,"last update":"Mon Oct 16 11:05:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.446,"15min":1.401},"min":{"1min":0.998,"5min":0.85999999999999999,"15min":0.79700000000000004},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3999999999999999,"15min":1.359},"min":{"1min":0.91900000000000004,"5min":0.88200000000000001,"15min":0.74299999999999999},"max":{"1min":1.909,"5min":1.909,"15min":2.101},"last":1.123}]},{"osd":5,"last update":"Mon Oct 16 11:05:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.627,"15min":1.5600000000000001},"min":{"1min":1.0169999999999999,"5min":0.99199999999999999,"15min":0.98799999999999999},"max":{"1min":1.9610000000000001,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.244},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.5429999999999999,"15min":1.496},"min":{"1min":1.1279999999999999,"5min":1.125,"15min":0.89100000000000001},"max":{"1min":2.008,"5min":2.0249999999999999,"15min":2.395},"last":1.2789999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:05:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5529999999999999,"15min":1.4910000000000001},"min":{"1min":1.004,"5min":0.89200000000000002,"15min":0.88500000000000001},"max":{"1min":1.8520000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.7170000000000001,"5min":1.754,"15min":1.6830000000000001},"min":{"1min":1.266,"5min":1.093,"15min":1.093},"max":{"1min":2.1240000000000001,"5min":2.2400000000000002,"15min":2.2690000000000001},"last":1.5600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575765,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29124,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342716,"statfs":{"total":96636764160,"available":96606941184,"internally_reserved":0,"allocated":2146304,"data_stored":1028145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:05:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.256,"15min":1.2070000000000001},"min":{"1min":0.85499999999999998,"5min":0.75,"15min":0.65300000000000002},"max":{"1min":1.5840000000000001,"5min":1.665,"15min":1.665},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2889999999999999,"15min":1.272},"min":{"1min":0.96399999999999997,"5min":0.89100000000000001,"15min":0.874},"max":{"1min":1.544,"5min":1.784,"15min":1.784},"last":0.84399999999999997}]},{"osd":1,"last update":"Mon Oct 16 11:05:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.288,"15min":1.2370000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.373,"5min":1.3839999999999999,"15min":1.347},"min":{"1min":1.052,"5min":0.96399999999999997,"15min":0.56200000000000006},"max":{"1min":1.7110000000000001,"5min":1.8220000000000001,"15min":2.1440000000000001},"last":0.70399999999999996}]},{"osd":2,"last update":"Mon Oct 16 11:05:04 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.262},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.69999999999999996},"max":{"1min":1.5660000000000001,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4059999999999999,"15min":1.3979999999999999},"min":{"1min":0.84499999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.778,"5min":1.988,"15min":2.4129999999999998},"last":1.3740000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:05:04 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.274,"15min":1.272},"min":{"1min":0.96199999999999997,"5min":0.83399999999999996,"15min":0.746},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.9299999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.246,"15min":1.262},"min":{"1min":1.008,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.669,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.319}]},{"osd":4,"last update":"Mon Oct 16 11:05:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3240000000000001,"15min":1.3320000000000001},"min":{"1min":0.80900000000000005,"5min":0.78300000000000003,"15min":0.75600000000000001},"max":{"1min":1.8380000000000001,"5min":1.9159999999999999,"15min":2.2130000000000001},"last":1.472},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.518,"15min":1.4990000000000001},"min":{"1min":1.024,"5min":0.82399999999999995,"15min":0.79800000000000004},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.2490000000000001},"last":0.97699999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:05:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.3460000000000001,"15min":1.347},"min":{"1min":0.89300000000000002,"5min":0.83699999999999997,"15min":0.77000000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":2.1869999999999998},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.603,"5min":1.591,"15min":1.569},"min":{"1min":1.1240000000000001,"5min":1.0289999999999999,"15min":0.71699999999999997},"max":{"1min":2.0169999999999999,"5min":2.056,"15min":2.54},"last":1.5740000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:05:04 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.5069999999999999,"15min":1.49},"min":{"1min":0.89900000000000002,"5min":0.81899999999999995,"15min":0.621},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":2.1360000000000001},"last":1.423},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.5629999999999999,"15min":1.5429999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.89000000000000001},"max":{"1min":1.966,"5min":1.966,"15min":2.2050000000000001},"last":1.601}]}]},{"osd":5,"up_from":13,"seq":55834575765,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41596,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330244,"statfs":{"total":96636764160,"available":96594169856,"internally_reserved":0,"allocated":2138112,"data_stored":1028021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:05:18 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2749999999999999,"15min":1.2769999999999999},"min":{"1min":0.91600000000000004,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.6850000000000001,"5min":1.927,"15min":1.927},"last":1.617},{"interface":"front","average":{"1min":1.448,"5min":1.389,"15min":1.357},"min":{"1min":1.272,"5min":0.996,"15min":0.94899999999999995},"max":{"1min":1.714,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.5649999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:05:18 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.1970000000000001,"15min":1.194},"min":{"1min":0.82299999999999995,"5min":0.79300000000000004,"15min":0.76300000000000001},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2350000000000001,"15min":1.2230000000000001},"min":{"1min":1.016,"5min":0.83499999999999996,"15min":0.79400000000000004},"max":{"1min":1.431,"5min":1.821,"15min":1.877},"last":1.3919999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:05:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2270000000000001,"15min":1.22},"min":{"1min":0.92300000000000004,"5min":0.86299999999999999,"15min":0.81899999999999995},"max":{"1min":1.552,"5min":1.7709999999999999,"15min":1.9079999999999999},"last":1.165},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3169999999999999,"15min":1.3100000000000001},"min":{"1min":1.048,"5min":0.92300000000000004,"15min":0.83399999999999996},"max":{"1min":1.712,"5min":1.782,"15min":1.9570000000000001},"last":1.1220000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:05:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3200000000000001,"15min":1.3089999999999999},"min":{"1min":1.004,"5min":0.92800000000000005,"15min":0.88500000000000001},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":2.274},"last":1.333},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.423,"15min":1.4279999999999999},"min":{"1min":1.1000000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":1.956,"5min":1.956,"15min":2.1440000000000001},"last":1.468}]},{"osd":4,"last update":"Mon Oct 16 11:05:18 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.2609999999999999,"15min":1.28},"min":{"1min":0.79700000000000004,"5min":0.76300000000000001,"15min":0.70299999999999996},"max":{"1min":1.853,"5min":1.853,"15min":1.9870000000000001},"last":1.778},{"interface":"front","average":{"1min":1.587,"5min":1.5609999999999999,"15min":1.556},"min":{"1min":1.345,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.9650000000000001,"5min":2.0539999999999998,"15min":2.4620000000000002},"last":1.867}]},{"osd":6,"last update":"Mon Oct 16 11:05:18 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.45,"15min":1.419},"min":{"1min":0.97599999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.663},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.456,"15min":1.4419999999999999},"min":{"1min":0.98799999999999999,"5min":0.89100000000000001,"15min":0.86399999999999999},"max":{"1min":1.994,"5min":1.994,"15min":2.1099999999999999},"last":1.956}]},{"osd":7,"last update":"Mon Oct 16 11:05:18 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.5609999999999999,"15min":1.55},"min":{"1min":1.214,"5min":0.91400000000000003,"15min":0.85399999999999998},"max":{"1min":2.069,"5min":2.069,"15min":2.5270000000000001},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.599,"5min":1.659,"15min":1.6439999999999999},"min":{"1min":1.1539999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.6549999999999998},"last":1.823}]}]},{"osd":4,"up_from":13,"seq":55834575766,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59632,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312208,"statfs":{"total":96636764160,"available":96575700992,"internally_reserved":0,"allocated":2134016,"data_stored":1027999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:05:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.173,"15min":1.218},"min":{"1min":0.84499999999999997,"5min":0.74199999999999999,"15min":0.52600000000000002},"max":{"1min":1.6299999999999999,"5min":1.7190000000000001,"15min":1.8169999999999999},"last":1.837},{"interface":"front","average":{"1min":1.179,"5min":1.238,"15min":1.2529999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.78400000000000003},"max":{"1min":1.5760000000000001,"5min":1.7070000000000001,"15min":1.9790000000000001},"last":0.99199999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:05:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.351,"15min":1.3380000000000001},"min":{"1min":1.026,"5min":0.85999999999999999,"15min":0.82899999999999996},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.1680000000000001},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.2370000000000001,"15min":1.26},"min":{"1min":0.90900000000000003,"5min":0.88500000000000001,"15min":0.83999999999999997},"max":{"1min":1.671,"5min":1.7789999999999999,"15min":1.8080000000000001},"last":1.4019999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:05:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3,"15min":1.2809999999999999},"min":{"1min":0.93999999999999995,"5min":0.81200000000000006,"15min":0.67200000000000004},"max":{"1min":1.7330000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.546},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.387,"15min":1.419},"min":{"1min":1.03,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":1.8859999999999999,"5min":2.234,"15min":2.234},"last":1.3120000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:05:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.349,"15min":1.323},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.92400000000000004},"max":{"1min":1.772,"5min":1.899,"15min":1.899},"last":1.089},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.448,"15min":1.4299999999999999},"min":{"1min":1.036,"5min":0.97299999999999998,"15min":0.75800000000000001},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.9470000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:05:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.413,"15min":1.413},"min":{"1min":0.98399999999999999,"5min":0.83399999999999996,"15min":0.81799999999999995},"max":{"1min":1.7,"5min":1.9079999999999999,"15min":1.9119999999999999},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.429,"5min":1.4319999999999999,"15min":1.401},"min":{"1min":0.96199999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.766,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.6719999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:05:09 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.464,"15min":1.446},"min":{"1min":0.97599999999999998,"5min":0.92700000000000005,"15min":0.85699999999999998},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.3490000000000002},"last":2.2679999999999998},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.6020000000000001,"15min":1.583},"min":{"1min":1.109,"5min":1.109,"15min":0.75800000000000001},"max":{"1min":1.889,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.73}]},{"osd":7,"last update":"Mon Oct 16 11:05:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.5509999999999999,"15min":1.534},"min":{"1min":1.107,"5min":1.0860000000000001,"15min":1.04},"max":{"1min":1.774,"5min":1.9890000000000001,"15min":2.117},"last":1.9930000000000001},{"interface":"front","average":{"1min":1.657,"5min":1.639,"15min":1.625},"min":{"1min":1.3999999999999999,"5min":1.119,"15min":0.874},"max":{"1min":2.161,"5min":2.1829999999999998,"15min":2.2690000000000001},"last":2.7229999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575765,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47156,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324684,"statfs":{"total":96636764160,"available":96588476416,"internally_reserved":0,"allocated":2138112,"data_stored":1028089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.90100000000000002,"15min":0.90200000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.968},"last":1.071},{"interface":"front","average":{"1min":0.90000000000000002,"5min":1.0249999999999999,"15min":1.048},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0920000000000001,"5min":1.5800000000000001,"15min":2.1040000000000001},"last":1.0960000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.0129999999999999,"15min":0.998},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.351,"5min":1.5509999999999999,"15min":2.6269999999999998},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1200000000000001,"15min":1.1339999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.6299999999999999,"5min":1.651,"15min":2.2970000000000002},"last":0.84399999999999997}]},{"osd":2,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.101,"15min":1.1140000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3879999999999999,"5min":1.641,"15min":2.3359999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.198,"5min":1.208,"15min":1.22},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5740000000000001,"5min":1.623,"15min":1.74},"last":1.0209999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.367,"15min":1.343},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.724,"5min":1.74,"15min":2.1459999999999999},"last":1.139},{"interface":"front","average":{"1min":1.347,"5min":1.3540000000000001,"15min":1.347},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6319999999999999,"5min":1.7110000000000001,"15min":1.7889999999999999},"last":1.3799999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.379,"15min":1.385},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.97299999999999998},"max":{"1min":1.6699999999999999,"5min":1.897,"15min":2.4590000000000001},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.419,"15min":1.421},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8220000000000001,"5min":1.9910000000000001,"15min":2.089},"last":1.2390000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.5109999999999999,"15min":1.502},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.9510000000000001,"5min":2.3279999999999998,"15min":2.5760000000000001},"last":1.196},{"interface":"front","average":{"1min":1.486,"5min":1.522,"15min":1.522},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.0219999999999998,"5min":2.1509999999999998,"15min":2.4169999999999998},"last":1.3500000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.522,"15min":1.5269999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":2.29,"5min":2.29,"15min":2.746},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.635,"15min":1.643},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.8029999999999999},"last":1.8560000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575764,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51332,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320508,"statfs":{"total":96636764160,"available":96584200192,"internally_reserved":0,"allocated":2150400,"data_stored":1029681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.95599999999999996,"15min":0.93100000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.548,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.026,"15min":1.0680000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3540000000000001,"5min":1.5409999999999999,"15min":2.3010000000000002},"last":0.73299999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.1399999999999999,"15min":1.1080000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.657,"5min":1.7290000000000001,"15min":1.7749999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.121,"15min":1.1559999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.546,"5min":1.661,"15min":2.6120000000000001},"last":0.96799999999999997}]},{"osd":3,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.109,"15min":1.085},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.3600000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.159,"5min":1.2849999999999999,"15min":1.266},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6910000000000001,"5min":1.899,"15min":1.99},"last":1.2310000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.351,"15min":1.3899999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.728,"5min":1.8340000000000001,"15min":2.5190000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.3580000000000001,"15min":1.341},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.4729999999999999,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":0.92900000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.476,"15min":1.4490000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.2559999999999998,"5min":2.2570000000000001,"15min":2.282},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4610000000000001,"15min":1.4710000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.2120000000000002,"5min":2.3450000000000002,"15min":2.9649999999999999},"last":1.46}]},{"osd":6,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.4239999999999999,"15min":1.4039999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.929,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.569,"15min":1.5269999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.0819999999999999,"5min":2.3420000000000001,"15min":2.5139999999999998},"last":1.661}]},{"osd":7,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.538,"15min":1.538},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.1389999999999998,"5min":2.141,"15min":3.1669999999999998},"last":1.399},{"interface":"front","average":{"1min":1.47,"5min":1.5169999999999999,"15min":1.514},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.2469999999999999,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":1.486}]}]},{"osd":1,"up_from":13,"seq":55834575764,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59636,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312204,"statfs":{"total":96636764160,"available":96575696896,"internally_reserved":0,"allocated":2138112,"data_stored":1028581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:05:15 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.92600000000000005,"15min":0.92300000000000004},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5609999999999999,"5min":1.581,"15min":1.6819999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0509999999999999,"15min":1.0549999999999999},"min":{"1min":0.748,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.589,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.873}]},{"osd":2,"last update":"Mon Oct 16 11:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.97799999999999998,"15min":1.0089999999999999},"min":{"1min":0.74399999999999999,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.359,"5min":1.5349999999999999,"15min":1.8069999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.2709999999999999,"15min":1.2789999999999999},"min":{"1min":0.92600000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.5629999999999999,"5min":1.77,"15min":1.77},"last":1.046}]},{"osd":3,"last update":"Mon Oct 16 11:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2150000000000001,"15min":1.216},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.659,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1639999999999999,"15min":1.149},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5740000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.282}]},{"osd":4,"last update":"Mon Oct 16 11:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.266,"15min":1.298},"min":{"1min":1.0549999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.536,"5min":1.659,"15min":2.1030000000000002},"last":1.873},{"interface":"front","average":{"1min":1.427,"5min":1.4299999999999999,"15min":1.4139999999999999},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.93899999999999995},"max":{"1min":1.839,"5min":1.974,"15min":2.0110000000000001},"last":1.6140000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.415,"15min":1.3999999999999999},"min":{"1min":1.032,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.827,"5min":1.841,"15min":2.0720000000000001},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.393,"5min":1.421,"15min":1.395},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.92800000000000005},"max":{"1min":1.6759999999999999,"5min":2.8690000000000002,"15min":2.8690000000000002},"last":1.3660000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4830000000000001,"15min":1.4610000000000001},"min":{"1min":1.169,"5min":1.0980000000000001,"15min":1.0549999999999999},"max":{"1min":1.8859999999999999,"5min":2.0609999999999999,"15min":2.149},"last":1.8009999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.419,"15min":1.4410000000000001},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.6719999999999999,"5min":1.867,"15min":2.077},"last":1.5249999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:05:15 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.5209999999999999,"15min":1.534},"min":{"1min":1.147,"5min":1.147,"15min":1.105},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.494},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.5589999999999999,"15min":1.569},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":1.7649999999999999,"5min":2.3340000000000001,"15min":2.4289999999999998},"last":1.74}]}]},{"osd":0,"up_from":13,"seq":55834575765,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51916,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319924,"statfs":{"total":96636764160,"available":96583602176,"internally_reserved":0,"allocated":2744320,"data_stored":1620013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:05:08 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.95999999999999996,"15min":0.90900000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.077,"15min":1.0720000000000001},"min":{"1min":0.78900000000000003,"5min":0.68899999999999995,"15min":0.41199999999999998},"max":{"1min":1.663,"5min":1.714,"15min":1.714},"last":0.72499999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:05:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.1080000000000001,"15min":1.0940000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.749},"max":{"1min":1.504,"5min":1.625,"15min":1.6299999999999999},"last":1.351},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.159,"15min":1.1499999999999999},"min":{"1min":0.91800000000000004,"5min":0.746,"15min":0.47799999999999998},"max":{"1min":1.3520000000000001,"5min":1.823,"15min":1.823},"last":1.0049999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:05:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1319999999999999,"15min":1.077},"min":{"1min":0.88,"5min":0.74399999999999999,"15min":0.64300000000000002},"max":{"1min":1.484,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.246,"5min":1.3080000000000001,"15min":1.264},"min":{"1min":0.95199999999999996,"5min":0.873,"15min":0.82699999999999996},"max":{"1min":1.544,"5min":1.7689999999999999,"15min":1.8480000000000001},"last":1.4019999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:05:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.385,"15min":1.373},"min":{"1min":1.151,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.623,"5min":1.7010000000000001,"15min":1.7410000000000001},"last":1.2},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.3360000000000001,"15min":1.325},"min":{"1min":1.0509999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":1.7330000000000001,"5min":1.8169999999999999,"15min":1.97},"last":1.1220000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:05:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.393,"15min":1.3759999999999999},"min":{"1min":1.01,"5min":0.96399999999999997,"15min":0.95699999999999996},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":2.0259999999999998},"last":1.337},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.506,"15min":1.462},"min":{"1min":1.1519999999999999,"5min":1.1319999999999999,"15min":1.0669999999999999},"max":{"1min":1.7809999999999999,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.4610000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:05:08 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.5640000000000001,"15min":1.51},"min":{"1min":1.196,"5min":1.077,"15min":1.077},"max":{"1min":1.8620000000000001,"5min":2.286,"15min":2.286},"last":1.51},{"interface":"front","average":{"1min":1.494,"5min":1.5369999999999999,"15min":1.47},"min":{"1min":1.095,"5min":1.0580000000000001,"15min":1.038},"max":{"1min":2.0099999999999998,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.4199999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:05:08 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.589,"15min":1.5349999999999999},"min":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.0900000000000001},"max":{"1min":1.8999999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.49},{"interface":"front","average":{"1min":1.486,"5min":1.5369999999999999,"15min":1.482},"min":{"1min":1.196,"5min":1.1200000000000001,"15min":0.997},"max":{"1min":1.875,"5min":2.0259999999999998,"15min":2.0289999999999999},"last":1.175}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:05:48.833 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T11:05:48.833 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T11:05:49.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.097+0000 7f1687ca2640 1 Processor -- start 2023-10-16T11:05:49.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.097+0000 7f1687ca2640 1 -- start start 2023-10-16T11:05:49.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.099+0000 7f1687ca2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f168006b4c0 0x7f16800ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:49.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.099+0000 7f1687ca2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1680160800 0x7f16800cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:49.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.099+0000 7f1687ca2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f168006e170 con 0x7f1680160420 2023-10-16T11:05:49.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.099+0000 7f1687ca2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16800cde30 con 0x7f168006b4c0 2023-10-16T11:05:49.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.099+0000 7f1687ca2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16800cd460 con 0x7f1680160800 2023-10-16T11:05:49.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.099+0000 7f168523d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f168006b4c0 0x7f16800ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:49.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.099+0000 7f1685a3e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1680160420 0x7f168006adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34318/0 (socket says 172.21.15.84:34318) 2023-10-16T11:05:49.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.099+0000 7f168523d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f168006b4c0 0x7f16800ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54728/0 (socket says 172.21.15.84:54728) 2023-10-16T11:05:49.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.099+0000 7f1685a3e640 1 -- 172.21.15.84:0/2700096190 learned_addr learned my addr 172.21.15.84:0/2700096190 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:49.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.100+0000 7f168523d640 1 -- 172.21.15.84:0/2700096190 >> v2:172.21.15.157:3300/0 conn(0x7f1680160800 msgr2=0x7f16800cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:49.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.100+0000 7f1684a3c640 1 -- 172.21.15.84:0/2700096190 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808047663 0 0) 0x7f168006e170 con 0x7f1680160420 2023-10-16T11:05:49.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.100+0000 7f168623f640 1 --2- 172.21.15.84:0/2700096190 >> v2:172.21.15.157:3300/0 conn(0x7f1680160800 0x7f16800cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:49.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.100+0000 7f168523d640 1 --2- 172.21.15.84:0/2700096190 >> v2:172.21.15.157:3300/0 conn(0x7f1680160800 0x7f16800cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.100+0000 7f168523d640 1 -- 172.21.15.84:0/2700096190 >> v1:172.21.15.84:6789/0 conn(0x7f1680160420 legacy=0x7f168006adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:49.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.100+0000 7f168523d640 1 -- 172.21.15.84:0/2700096190 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168006e3c0 con 0x7f168006b4c0 2023-10-16T11:05:49.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.100+0000 7f168623f640 1 --2- 172.21.15.84:0/2700096190 >> v2:172.21.15.157:3300/0 conn(0x7f1680160800 0x7f16800cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:49.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.101+0000 7f168523d640 1 --2- 172.21.15.84:0/2700096190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f168006b4c0 0x7f16800ae170 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f168006fd40 tx=0x7f167002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7aae5cd0b011dd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:49.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.101+0000 7f1684a3c640 1 -- 172.21.15.84:0/2700096190 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670031830 con 0x7f168006b4c0 2023-10-16T11:05:49.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.101+0000 7f1684a3c640 1 -- 172.21.15.84:0/2700096190 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1670031990 con 0x7f168006b4c0 2023-10-16T11:05:49.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.101+0000 7f1684a3c640 1 -- 172.21.15.84:0/2700096190 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670031d20 con 0x7f168006b4c0 2023-10-16T11:05:49.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.102+0000 7f1687ca2640 1 -- 172.21.15.84:0/2700096190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f168006b4c0 msgr2=0x7f16800ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:49.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.102+0000 7f1687ca2640 1 --2- 172.21.15.84:0/2700096190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f168006b4c0 0x7f16800ae170 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f168006fd40 tx=0x7f167002b8d0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.102+0000 7f1687ca2640 1 -- 172.21.15.84:0/2700096190 shutdown_connections 2023-10-16T11:05:49.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.102+0000 7f1687ca2640 1 --2- 172.21.15.84:0/2700096190 >> v2:172.21.15.157:3300/0 conn(0x7f1680160800 0x7f16800cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.102+0000 7f1687ca2640 1 --2- 172.21.15.84:0/2700096190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f168006b4c0 0x7f16800ae170 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.102+0000 7f1687ca2640 1 -- 172.21.15.84:0/2700096190 >> 172.21.15.84:0/2700096190 conn(0x7f16800b0d30 msgr2=0x7f16800b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:49.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.102+0000 7f1687ca2640 1 -- 172.21.15.84:0/2700096190 shutdown_connections 2023-10-16T11:05:49.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.102+0000 7f1687ca2640 1 -- 172.21.15.84:0/2700096190 wait complete. 2023-10-16T11:05:49.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.104+0000 7f1687ca2640 1 Processor -- start 2023-10-16T11:05:49.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.104+0000 7f1687ca2640 1 -- start start 2023-10-16T11:05:49.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.105+0000 7f1687ca2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f168006b4c0 0x7f168014d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:49.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.105+0000 7f1687ca2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1680160420 0x7f16800c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:49.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.105+0000 7f1687ca2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1680160800 0x7f16800c3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:49.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.105+0000 7f1687ca2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168014eb70 con 0x7f1680160420 2023-10-16T11:05:49.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.105+0000 7f1687ca2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16800c90a0 con 0x7f168006b4c0 2023-10-16T11:05:49.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.105+0000 7f1687ca2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168014def0 con 0x7f1680160800 2023-10-16T11:05:49.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.105+0000 7f168523d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1680160420 0x7f16800c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:49.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.105+0000 7f168523d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1680160420 0x7f16800c2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54742/0 (socket says 172.21.15.84:54742) 2023-10-16T11:05:49.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.105+0000 7f168523d640 1 -- 172.21.15.84:0/576495905 learned_addr learned my addr 172.21.15.84:0/576495905 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:49.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.106+0000 7f1685a3e640 1 --2- 172.21.15.84:0/576495905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f168006b4c0 0x7f168014d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:49.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.106+0000 7f168523d640 1 -- 172.21.15.84:0/576495905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1680160800 msgr2=0x7f16800c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:49.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.106+0000 7f168523d640 1 --2- 172.21.15.84:0/576495905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1680160800 0x7f16800c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.106+0000 7f168523d640 1 -- 172.21.15.84:0/576495905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f168006b4c0 msgr2=0x7f168014d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:49.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.106+0000 7f168523d640 1 --2- 172.21.15.84:0/576495905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f168006b4c0 0x7f168014d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.106+0000 7f168523d640 1 -- 172.21.15.84:0/576495905 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1670009400 con 0x7f1680160420 2023-10-16T11:05:49.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.106+0000 7f168523d640 1 --2- 172.21.15.84:0/576495905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1680160420 0x7f16800c2c10 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f1670000c00 tx=0x7f16700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:49.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.106+0000 7f16767fc640 1 -- 172.21.15.84:0/576495905 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670004230 con 0x7f1680160420 2023-10-16T11:05:49.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.106+0000 7f16767fc640 1 -- 172.21.15.84:0/576495905 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1670004390 con 0x7f1680160420 2023-10-16T11:05:49.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.106+0000 7f16767fc640 1 -- 172.21.15.84:0/576495905 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167003a920 con 0x7f1680160420 2023-10-16T11:05:49.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.106+0000 7f1687ca2640 1 -- 172.21.15.84:0/576495905 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168014e1d0 con 0x7f1680160420 2023-10-16T11:05:49.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.106+0000 7f1687ca2640 1 -- 172.21.15.84:0/576495905 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f168014e790 con 0x7f1680160420 2023-10-16T11:05:49.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.107+0000 7f1687ca2640 1 -- 172.21.15.84:0/576495905 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f168014ecb0 con 0x7f1680160420 2023-10-16T11:05:49.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.108+0000 7f16767fc640 1 -- 172.21.15.84:0/576495905 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f167003aa80 con 0x7f1680160420 2023-10-16T11:05:49.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.109+0000 7f16767fc640 1 --2- 172.21.15.84:0/576495905 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f166406af90 0x7f166406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:49.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.109+0000 7f16767fc640 1 -- 172.21.15.84:0/576495905 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f16700ac0d0 con 0x7f1680160420 2023-10-16T11:05:49.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.110+0000 7f1685a3e640 1 --2- 172.21.15.84:0/576495905 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f166406af90 0x7f166406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:49.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.110+0000 7f1685a3e640 1 --2- 172.21.15.84:0/576495905 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f166406af90 0x7f166406d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f167c007fd0 tx=0x7f167c009040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:49.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.114+0000 7f16767fc640 1 -- 172.21.15.84:0/576495905 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f168014ecb0 con 0x7f1680160420 2023-10-16T11:05:49.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.378+0000 7f1687ca2640 1 -- 172.21.15.84:0/576495905 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f168014ecb0 con 0x7f1680160420 2023-10-16T11:05:49.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.382+0000 7f16767fc640 1 -- 172.21.15.84:0/576495905 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v47) v1 ==== 190+0+147 (secure 0 0 0) 0x7f168014ecb0 con 0x7f1680160420 2023-10-16T11:05:49.383 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2023-10-16T11:05:49.384 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ== 2023-10-16T11:05:49.384 INFO:teuthology.orchestra.run.smithi084.stdout: caps mds = "allow" 2023-10-16T11:05:49.384 INFO:teuthology.orchestra.run.smithi084.stdout: caps mon = "allow r" 2023-10-16T11:05:49.384 INFO:teuthology.orchestra.run.smithi084.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T11:05:49.384 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-16T11:05:49.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.385+0000 7f1687ca2640 1 -- 172.21.15.84:0/576495905 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f166406af90 msgr2=0x7f166406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:49.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.385+0000 7f1687ca2640 1 --2- 172.21.15.84:0/576495905 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f166406af90 0x7f166406d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f167c007fd0 tx=0x7f167c009040 comp rx=0 tx=0).stop 2023-10-16T11:05:49.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.385+0000 7f1687ca2640 1 -- 172.21.15.84:0/576495905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1680160420 msgr2=0x7f16800c2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:49.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.385+0000 7f1687ca2640 1 --2- 172.21.15.84:0/576495905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1680160420 0x7f16800c2c10 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f1670000c00 tx=0x7f16700046c0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.385+0000 7f1687ca2640 1 -- 172.21.15.84:0/576495905 shutdown_connections 2023-10-16T11:05:49.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.385+0000 7f1687ca2640 1 --2- 172.21.15.84:0/576495905 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f166406af90 0x7f166406d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.385+0000 7f1687ca2640 1 --2- 172.21.15.84:0/576495905 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1680160800 0x7f16800c3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.385+0000 7f1687ca2640 1 --2- 172.21.15.84:0/576495905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1680160420 0x7f16800c2c10 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.385+0000 7f1687ca2640 1 --2- 172.21.15.84:0/576495905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f168006b4c0 0x7f168014d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.385+0000 7f1687ca2640 1 -- 172.21.15.84:0/576495905 >> 172.21.15.84:0/576495905 conn(0x7f16800b0d30 msgr2=0x7f16800afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:49.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.385+0000 7f1687ca2640 1 -- 172.21.15.84:0/576495905 shutdown_connections 2023-10-16T11:05:49.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.386+0000 7f1687ca2640 1 -- 172.21.15.84:0/576495905 wait complete. 2023-10-16T11:05:49.406 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:05:49.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.601+0000 7fb50103f640 1 Processor -- start 2023-10-16T11:05:49.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.601+0000 7fb50103f640 1 -- start start 2023-10-16T11:05:49.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.602+0000 7fb50103f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb4fc15aaa0 0x7fb4fc15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:49.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.602+0000 7fb50103f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb4fc15ae80 0x7fb4fc0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:49.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.602+0000 7fb50103f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4fc15ed60 con 0x7fb4fc0ccaa0 2023-10-16T11:05:49.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.603+0000 7fb50103f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4fc06b810 con 0x7fb4fc15aaa0 2023-10-16T11:05:49.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.603+0000 7fb50103f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4fc0707e0 con 0x7fb4fc15ae80 2023-10-16T11:05:49.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.603+0000 7fb4fad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb4fc15aaa0 0x7fb4fc15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:49.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.603+0000 7fb4fad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb4fc15aaa0 0x7fb4fc15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54752/0 (socket says 172.21.15.84:54752) 2023-10-16T11:05:49.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.603+0000 7fb4fad9d640 1 -- 172.21.15.84:0/1752537643 learned_addr learned my addr 172.21.15.84:0/1752537643 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:49.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.603+0000 7fb4fad9d640 1 -- 172.21.15.84:0/1752537643 >> v2:172.21.15.157:3300/0 conn(0x7fb4fc15ae80 msgr2=0x7fb4fc0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:49.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.603+0000 7fb4fad9d640 1 --2- 172.21.15.84:0/1752537643 >> v2:172.21.15.157:3300/0 conn(0x7fb4fc15ae80 0x7fb4fc0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.603+0000 7fb4fad9d640 1 -- 172.21.15.84:0/1752537643 >> v1:172.21.15.84:6789/0 conn(0x7fb4fc0ccaa0 legacy=0x7fb4fc070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:49.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.603+0000 7fb4fa59c640 1 -- 172.21.15.84:0/1752537643 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895381598 0 0) 0x7fb4fc15ed60 con 0x7fb4fc0ccaa0 2023-10-16T11:05:49.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.603+0000 7fb4fad9d640 1 -- 172.21.15.84:0/1752537643 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc15ef90 con 0x7fb4fc15aaa0 2023-10-16T11:05:49.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.604+0000 7fb4fad9d640 1 --2- 172.21.15.84:0/1752537643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4fc15aaa0 0x7fb4fc15e820 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0c59c0 tx=0x7fb4ec02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60fb884a3f74ef1d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:49.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.605+0000 7fb4fa59c640 1 -- 172.21.15.84:0/1752537643 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec030860 con 0x7fb4fc15aaa0 2023-10-16T11:05:49.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.605+0000 7fb4fa59c640 1 -- 172.21.15.84:0/1752537643 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec0309c0 con 0x7fb4fc15aaa0 2023-10-16T11:05:49.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.605+0000 7fb4fa59c640 1 -- 172.21.15.84:0/1752537643 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec030d50 con 0x7fb4fc15aaa0 2023-10-16T11:05:49.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.605+0000 7fb50103f640 1 -- 172.21.15.84:0/1752537643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4fc15aaa0 msgr2=0x7fb4fc15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:49.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.605+0000 7fb50103f640 1 --2- 172.21.15.84:0/1752537643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4fc15aaa0 0x7fb4fc15e820 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0c59c0 tx=0x7fb4ec02b9a0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.605+0000 7fb50103f640 1 -- 172.21.15.84:0/1752537643 shutdown_connections 2023-10-16T11:05:49.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.605+0000 7fb50103f640 1 --2- 172.21.15.84:0/1752537643 >> v2:172.21.15.157:3300/0 conn(0x7fb4fc15ae80 0x7fb4fc0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.605+0000 7fb50103f640 1 --2- 172.21.15.84:0/1752537643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4fc15aaa0 0x7fb4fc15e820 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.605+0000 7fb50103f640 1 -- 172.21.15.84:0/1752537643 >> 172.21.15.84:0/1752537643 conn(0x7fb4fc0cb410 msgr2=0x7fb4fc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:49.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.606+0000 7fb50103f640 1 -- 172.21.15.84:0/1752537643 shutdown_connections 2023-10-16T11:05:49.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.606+0000 7fb50103f640 1 -- 172.21.15.84:0/1752537643 wait complete. 2023-10-16T11:05:49.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.607+0000 7fb50103f640 1 Processor -- start 2023-10-16T11:05:49.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.607+0000 7fb50103f640 1 -- start start 2023-10-16T11:05:49.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.608+0000 7fb50103f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4fc0ccaa0 0x7fb4fc15bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:49.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.608+0000 7fb50103f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4fc15aaa0 0x7fb4fc0acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:49.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.608+0000 7fb50103f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4fc15ae80 0x7fb4fc0c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:49.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.608+0000 7fb50103f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0c5a60 con 0x7fb4fc0ccaa0 2023-10-16T11:05:49.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.608+0000 7fb50103f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0c5cb0 con 0x7fb4fc15aaa0 2023-10-16T11:05:49.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.608+0000 7fb50103f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0c5f90 con 0x7fb4fc15ae80 2023-10-16T11:05:49.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.609+0000 7fb4fad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4fc0ccaa0 0x7fb4fc15bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:49.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.609+0000 7fb4fad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4fc0ccaa0 0x7fb4fc15bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54762/0 (socket says 172.21.15.84:54762) 2023-10-16T11:05:49.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.609+0000 7fb4fad9d640 1 -- 172.21.15.84:0/137136772 learned_addr learned my addr 172.21.15.84:0/137136772 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:49.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.609+0000 7fb4fad9d640 1 -- 172.21.15.84:0/137136772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4fc15ae80 msgr2=0x7fb4fc0c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:49.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.609+0000 7fb4fad9d640 1 --2- 172.21.15.84:0/137136772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4fc15ae80 0x7fb4fc0c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.609+0000 7fb4fad9d640 1 -- 172.21.15.84:0/137136772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4fc15aaa0 msgr2=0x7fb4fc0acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:49.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.609+0000 7fb4fad9d640 1 --2- 172.21.15.84:0/137136772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4fc15aaa0 0x7fb4fc0acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.609+0000 7fb4fad9d640 1 -- 172.21.15.84:0/137136772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ec0094d0 con 0x7fb4fc0ccaa0 2023-10-16T11:05:49.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.609+0000 7fb4fad9d640 1 --2- 172.21.15.84:0/137136772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4fc0ccaa0 0x7fb4fc15bd30 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec000c00 tx=0x7fb4ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:49.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.610+0000 7fb4f37fe640 1 -- 172.21.15.84:0/137136772 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec030830 con 0x7fb4fc0ccaa0 2023-10-16T11:05:49.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.610+0000 7fb4f37fe640 1 -- 172.21.15.84:0/137136772 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec043400 con 0x7fb4fc0ccaa0 2023-10-16T11:05:49.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.610+0000 7fb50103f640 1 -- 172.21.15.84:0/137136772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4fc0c6270 con 0x7fb4fc0ccaa0 2023-10-16T11:05:49.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.610+0000 7fb4f37fe640 1 -- 172.21.15.84:0/137136772 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec0437b0 con 0x7fb4fc0ccaa0 2023-10-16T11:05:49.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.610+0000 7fb50103f640 1 -- 172.21.15.84:0/137136772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4fc0c6830 con 0x7fb4fc0ccaa0 2023-10-16T11:05:49.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.611+0000 7fb50103f640 1 -- 172.21.15.84:0/137136772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4fc0c6a50 con 0x7fb4fc0ccaa0 2023-10-16T11:05:49.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.612+0000 7fb4f37fe640 1 -- 172.21.15.84:0/137136772 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb4ec043910 con 0x7fb4fc0ccaa0 2023-10-16T11:05:49.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.613+0000 7fb4f37fe640 1 --2- 172.21.15.84:0/137136772 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fb4dc06af90 0x7fb4dc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:49.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.613+0000 7fb4f37fe640 1 -- 172.21.15.84:0/137136772 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb4ec03e070 con 0x7fb4fc0ccaa0 2023-10-16T11:05:49.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.613+0000 7fb4f3fff640 1 --2- 172.21.15.84:0/137136772 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fb4dc06af90 0x7fb4dc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:49.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.614+0000 7fb4f3fff640 1 --2- 172.21.15.84:0/137136772 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fb4dc06af90 0x7fb4dc06d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc1f5cd0 tx=0x7fb4e4006bf0 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:49.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.617+0000 7fb4f37fe640 1 -- 172.21.15.84:0/137136772 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb4ec077cf0 con 0x7fb4fc0ccaa0 2023-10-16T11:05:49.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.846+0000 7fb50103f640 1 -- 172.21.15.84:0/137136772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4fc04e160 con 0x7fb4fc0ccaa0 2023-10-16T11:05:49.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.847+0000 7fb4f37fe640 1 -- 172.21.15.84:0/137136772 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fb4ec07b310 con 0x7fb4fc0ccaa0 2023-10-16T11:05:49.848 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:05:49.849 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":88,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6718,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3060266169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3060266169},{"type":"v1","addr":"172.21.15.157:6834","nonce":3060266169}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":6721,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2965939103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2965939103},{"type":"v1","addr":"172.21.15.157:6835","nonce":2965939103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9684,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3677909298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3677909298},{"type":"v1","addr":"172.21.15.84:6835","nonce":3677909298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:05:43.341956+0000","modified":"2023-10-16T11:05:46.428445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9687},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9687":{"gid":9687,"name":"c","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/424056375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":424056375},{"type":"v1","addr":"172.21.15.84:6837","nonce":424056375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-16T11:05:49.849 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 88 2023-10-16T11:05:49.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.849+0000 7fb50103f640 1 -- 172.21.15.84:0/137136772 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fb4dc06af90 msgr2=0x7fb4dc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:49.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.849+0000 7fb50103f640 1 --2- 172.21.15.84:0/137136772 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fb4dc06af90 0x7fb4dc06d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc1f5cd0 tx=0x7fb4e4006bf0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.849+0000 7fb50103f640 1 -- 172.21.15.84:0/137136772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4fc0ccaa0 msgr2=0x7fb4fc15bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:49.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.849+0000 7fb50103f640 1 --2- 172.21.15.84:0/137136772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4fc0ccaa0 0x7fb4fc15bd30 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec000c00 tx=0x7fb4ec004730 comp rx=0 tx=0).stop 2023-10-16T11:05:49.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.849+0000 7fb50103f640 1 -- 172.21.15.84:0/137136772 shutdown_connections 2023-10-16T11:05:49.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.849+0000 7fb50103f640 1 --2- 172.21.15.84:0/137136772 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fb4dc06af90 0x7fb4dc06d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.849+0000 7fb50103f640 1 --2- 172.21.15.84:0/137136772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4fc15ae80 0x7fb4fc0c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.849+0000 7fb50103f640 1 --2- 172.21.15.84:0/137136772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4fc15aaa0 0x7fb4fc0acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.849+0000 7fb50103f640 1 --2- 172.21.15.84:0/137136772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4fc0ccaa0 0x7fb4fc15bd30 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:49.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.849+0000 7fb50103f640 1 -- 172.21.15.84:0/137136772 >> 172.21.15.84:0/137136772 conn(0x7fb4fc0cb410 msgr2=0x7fb4fc06cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:49.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.849+0000 7fb50103f640 1 -- 172.21.15.84:0/137136772 shutdown_connections 2023-10-16T11:05:49.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:49.849+0000 7fb50103f640 1 -- 172.21.15.84:0/137136772 wait complete. 2023-10-16T11:05:50.272 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T11:05:43.341956+0000', 'modified': '2023-10-16T11:05:46.428445+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9687}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9687': {'gid': 9687, 'name': 'c', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.84:6837/424056375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6836', 'nonce': 424056375}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 424056375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T11:05:50.273 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T11:05:50.273 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-16T11:05:50.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.499+0000 7effe3f33640 1 Processor -- start 2023-10-16T11:05:50.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.499+0000 7effe3f33640 1 -- start start 2023-10-16T11:05:50.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.500+0000 7effe3f33640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7effdc159e90 0x7effdc0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.501+0000 7effe3f33640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7effdc06c140 0x7effdc06c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.501+0000 7effe3f33640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effdc06cad0 con 0x7effdc159ab0 2023-10-16T11:05:50.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.501+0000 7effe3f33640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effdc15d890 con 0x7effdc159e90 2023-10-16T11:05:50.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.501+0000 7effe3f33640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effdc070360 con 0x7effdc06c140 2023-10-16T11:05:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.501+0000 7effe14ce640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7effdc159e90 0x7effdc0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.501+0000 7effe14ce640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7effdc159e90 0x7effdc0ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54770/0 (socket says 172.21.15.84:54770) 2023-10-16T11:05:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.501+0000 7effe14ce640 1 -- 172.21.15.84:0/1692415014 learned_addr learned my addr 172.21.15.84:0/1692415014 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.501+0000 7effe24d0640 1 --2- 172.21.15.84:0/1692415014 >> v2:172.21.15.157:3300/0 conn(0x7effdc06c140 0x7effdc06c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.501+0000 7effe14ce640 1 -- 172.21.15.84:0/1692415014 >> v2:172.21.15.157:3300/0 conn(0x7effdc06c140 msgr2=0x7effdc06c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.501+0000 7effe14ce640 1 --2- 172.21.15.84:0/1692415014 >> v2:172.21.15.157:3300/0 conn(0x7effdc06c140 0x7effdc06c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.501+0000 7effe14ce640 1 -- 172.21.15.84:0/1692415014 >> v1:172.21.15.157:6789/0 conn(0x7effdc159ab0 legacy=0x7effdc06fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.501+0000 7effe14ce640 1 -- 172.21.15.84:0/1692415014 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc0cd1f0 con 0x7effdc159e90 2023-10-16T11:05:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.503+0000 7effe14ce640 1 --2- 172.21.15.84:0/1692415014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effdc159e90 0x7effdc0ce370 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7effdc0adcd0 tx=0x7effcc02dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5759986e1ab865 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.503+0000 7effe0ccd640 1 -- 172.21.15.84:0/1692415014 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc02ed70 con 0x7effdc159e90 2023-10-16T11:05:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.503+0000 7effe0ccd640 1 -- 172.21.15.84:0/1692415014 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effcc036440 con 0x7effdc159e90 2023-10-16T11:05:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.503+0000 7effe0ccd640 1 -- 172.21.15.84:0/1692415014 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc0367d0 con 0x7effdc159e90 2023-10-16T11:05:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.503+0000 7effe3f33640 1 -- 172.21.15.84:0/1692415014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effdc159e90 msgr2=0x7effdc0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.503+0000 7effe3f33640 1 --2- 172.21.15.84:0/1692415014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effdc159e90 0x7effdc0ce370 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7effdc0adcd0 tx=0x7effcc02dfc0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.504+0000 7effe3f33640 1 -- 172.21.15.84:0/1692415014 shutdown_connections 2023-10-16T11:05:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.504+0000 7effe3f33640 1 --2- 172.21.15.84:0/1692415014 >> v2:172.21.15.157:3300/0 conn(0x7effdc06c140 0x7effdc06c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.504+0000 7effe3f33640 1 --2- 172.21.15.84:0/1692415014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effdc159e90 0x7effdc0ce370 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.504+0000 7effe3f33640 1 -- 172.21.15.84:0/1692415014 >> 172.21.15.84:0/1692415014 conn(0x7effdc0ce9b0 msgr2=0x7effdc0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.504+0000 7effe3f33640 1 -- 172.21.15.84:0/1692415014 shutdown_connections 2023-10-16T11:05:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.504+0000 7effe3f33640 1 -- 172.21.15.84:0/1692415014 wait complete. 2023-10-16T11:05:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.505+0000 7effe3f33640 1 Processor -- start 2023-10-16T11:05:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.505+0000 7effe3f33640 1 -- start start 2023-10-16T11:05:50.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.506+0000 7effe3f33640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effdc06c140 0x7effdc157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.506+0000 7effe3f33640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effdc159ab0 0x7effdc15d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.506+0000 7effe3f33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effdc159e90 0x7effdc0b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.506+0000 7effe3f33640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc15bd30 con 0x7effdc06c140 2023-10-16T11:05:50.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.506+0000 7effe3f33640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc15bf80 con 0x7effdc159e90 2023-10-16T11:05:50.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.506+0000 7effe3f33640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effdc15c260 con 0x7effdc159ab0 2023-10-16T11:05:50.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.507+0000 7effe1ccf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effdc06c140 0x7effdc157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:50.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.507+0000 7effe1ccf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effdc06c140 0x7effdc157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54782/0 (socket says 172.21.15.84:54782) 2023-10-16T11:05:50.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.507+0000 7effe1ccf640 1 -- 172.21.15.84:0/350372831 learned_addr learned my addr 172.21.15.84:0/350372831 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:50.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.507+0000 7effe1ccf640 1 -- 172.21.15.84:0/350372831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effdc159ab0 msgr2=0x7effdc15d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:50.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.507+0000 7effe1ccf640 1 --2- 172.21.15.84:0/350372831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effdc159ab0 0x7effdc15d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.507+0000 7effe1ccf640 1 -- 172.21.15.84:0/350372831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effdc159e90 msgr2=0x7effdc0b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:50.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.507+0000 7effe1ccf640 1 --2- 172.21.15.84:0/350372831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effdc159e90 0x7effdc0b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.507+0000 7effe1ccf640 1 -- 172.21.15.84:0/350372831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc00b770 con 0x7effdc06c140 2023-10-16T11:05:50.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.507+0000 7effe1ccf640 1 --2- 172.21.15.84:0/350372831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effdc06c140 0x7effdc157ee0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7effd800c410 tx=0x7effd800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:50.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.508+0000 7effd27fc640 1 -- 172.21.15.84:0/350372831 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd80035f0 con 0x7effdc06c140 2023-10-16T11:05:50.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.508+0000 7effd27fc640 1 -- 172.21.15.84:0/350372831 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd8003750 con 0x7effdc06c140 2023-10-16T11:05:50.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.508+0000 7effe3f33640 1 -- 172.21.15.84:0/350372831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effdc15c540 con 0x7effdc06c140 2023-10-16T11:05:50.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.508+0000 7effd27fc640 1 -- 172.21.15.84:0/350372831 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8003b40 con 0x7effdc06c140 2023-10-16T11:05:50.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.508+0000 7effe3f33640 1 -- 172.21.15.84:0/350372831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effdc15cb00 con 0x7effdc06c140 2023-10-16T11:05:50.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.510+0000 7effd27fc640 1 -- 172.21.15.84:0/350372831 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7effd8003ca0 con 0x7effdc06c140 2023-10-16T11:05:50.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.511+0000 7effd27fc640 1 --2- 172.21.15.84:0/350372831 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7effc006af90 0x7effc006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.511+0000 7effd27fc640 1 -- 172.21.15.84:0/350372831 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5737+0+0 (secure 0 0 0) 0x7effd8088aa0 con 0x7effdc06c140 2023-10-16T11:05:50.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.512+0000 7effe14ce640 1 --2- 172.21.15.84:0/350372831 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7effc006af90 0x7effc006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:50.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.512+0000 7effe6ba0740 1 Processor -- start 2023-10-16T11:05:50.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.512+0000 7effe6ba0740 1 -- start start 2023-10-16T11:05:50.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.512+0000 7effe14ce640 1 --2- 172.21.15.84:0/350372831 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7effc006af90 0x7effc006d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7effcc038000 tx=0x7effcc030000 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:50.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.513+0000 7effe6ba0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 0x55f3a8368e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.513+0000 7effe6ba0740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f3a8396360 0x55f3a8398d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.513+0000 7effe6ba0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a83693c0 0x55f3a8399250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.513+0000 7effe6ba0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a82897a0 con 0x55f3a8395f80 2023-10-16T11:05:50.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.513+0000 7effe6ba0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a82db740 con 0x55f3a83693c0 2023-10-16T11:05:50.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.513+0000 7effe6ba0740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3a82d8e30 con 0x55f3a8396360 2023-10-16T11:05:50.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.513+0000 7effe1ccf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 0x55f3a8368e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:50.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.513+0000 7effe1ccf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 0x55f3a8368e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54798/0 (socket says 172.21.15.84:54798) 2023-10-16T11:05:50.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.513+0000 7effe1ccf640 1 -- 172.21.15.84:0/1375593988 learned_addr learned my addr 172.21.15.84:0/1375593988 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:50.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.514+0000 7effe1ccf640 1 -- 172.21.15.84:0/1375593988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f3a8396360 msgr2=0x55f3a8398d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:50.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.514+0000 7effe1ccf640 1 --2- 172.21.15.84:0/1375593988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f3a8396360 0x55f3a8398d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.514+0000 7effe1ccf640 1 -- 172.21.15.84:0/1375593988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a83693c0 msgr2=0x55f3a8399250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:50.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.514+0000 7effe1ccf640 1 --2- 172.21.15.84:0/1375593988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a83693c0 0x55f3a8399250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.514+0000 7effe1ccf640 1 -- 172.21.15.84:0/1375593988 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3a8298910 con 0x55f3a8395f80 2023-10-16T11:05:50.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.514+0000 7effe1ccf640 1 --2- 172.21.15.84:0/1375593988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 0x55f3a8368e80 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7effd805f410 tx=0x7effd80572b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3e02e51969be85 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:50.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.514+0000 7effbf7fe640 1 -- 172.21.15.84:0/1375593988 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd805f940 con 0x55f3a8395f80 2023-10-16T11:05:50.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.514+0000 7effbf7fe640 1 -- 172.21.15.84:0/1375593988 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd8086f00 con 0x55f3a8395f80 2023-10-16T11:05:50.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.514+0000 7effbf7fe640 1 -- 172.21.15.84:0/1375593988 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8087070 con 0x55f3a8395f80 2023-10-16T11:05:50.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.515+0000 7effe6ba0740 1 -- 172.21.15.84:0/1375593988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 msgr2=0x55f3a8368e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:50.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.515+0000 7effe6ba0740 1 --2- 172.21.15.84:0/1375593988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 0x55f3a8368e80 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7effd805f410 tx=0x7effd80572b0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.515+0000 7effe6ba0740 1 -- 172.21.15.84:0/1375593988 shutdown_connections 2023-10-16T11:05:50.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.515+0000 7effe6ba0740 1 --2- 172.21.15.84:0/1375593988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a83693c0 0x55f3a8399250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.515+0000 7effe6ba0740 1 --2- 172.21.15.84:0/1375593988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f3a8396360 0x55f3a8398d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.515+0000 7effe6ba0740 1 --2- 172.21.15.84:0/1375593988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 0x55f3a8368e80 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.515+0000 7effe6ba0740 1 -- 172.21.15.84:0/1375593988 >> 172.21.15.84:0/1375593988 conn(0x55f3a836b190 msgr2=0x55f3a83b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:50.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.515+0000 7effe6ba0740 1 -- 172.21.15.84:0/1375593988 shutdown_connections 2023-10-16T11:05:50.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.515+0000 7effe6ba0740 1 -- 172.21.15.84:0/1375593988 wait complete. 2023-10-16T11:05:50.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.516+0000 7effe6ba0740 1 -- start start 2023-10-16T11:05:50.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.516+0000 7effe6ba0740 1 Processor -- start 2023-10-16T11:05:50.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.517+0000 7effe6ba0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 0x55f3a8390f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.517+0000 7effe6ba0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a8396360 0x55f3a8398580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.517+0000 7effe6ba0740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f3a8380360 0x55f3a839a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.517+0000 7effe6ba0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a82d4720 con 0x55f3a8395f80 2023-10-16T11:05:50.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.517+0000 7effe6ba0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a8272970 con 0x55f3a8396360 2023-10-16T11:05:50.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.517+0000 7effe6ba0740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3a8272610 con 0x55f3a8380360 2023-10-16T11:05:50.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.517+0000 7effe6ba0740 10 client.0 _ll_register_callbacks cb 0x55f3a82949d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:05:50.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.517+0000 7effe1ccf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 0x55f3a8390f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:50.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.517+0000 7effe1ccf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 0x55f3a8390f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54800/0 (socket says 172.21.15.84:54800) 2023-10-16T11:05:50.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.517+0000 7effe1ccf640 1 -- 172.21.15.84:0/382704870 learned_addr learned my addr 172.21.15.84:0/382704870 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:50.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.517+0000 7effe1ccf640 1 -- 172.21.15.84:0/382704870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f3a8380360 msgr2=0x55f3a839a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.517+0000 7effe1ccf640 1 --2- 172.21.15.84:0/382704870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f3a8380360 0x55f3a839a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.517+0000 7effe1ccf640 1 -- 172.21.15.84:0/382704870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a8396360 msgr2=0x55f3a8398580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.517+0000 7effe14ce640 1 --2- 172.21.15.84:0/382704870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a8396360 0x55f3a8398580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.517+0000 7effe1ccf640 1 --2- 172.21.15.84:0/382704870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a8396360 0x55f3a8398580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.517+0000 7effe1ccf640 1 -- 172.21.15.84:0/382704870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effdc15cb00 con 0x55f3a8395f80 2023-10-16T11:05:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.518+0000 7effe1ccf640 1 --2- 172.21.15.84:0/382704870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 0x55f3a8390f40 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7effd80050a0 tx=0x7effd8000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7bdfd1ab589c87 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.518+0000 7effbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T11:05:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.518+0000 7effbcff9640 1 -- 172.21.15.84:0/382704870 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd805f940 con 0x55f3a8395f80 2023-10-16T11:05:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.518+0000 7effbcff9640 1 -- 172.21.15.84:0/382704870 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 5737+0+0 (secure 0 0 0) 0x7effd805f6c0 con 0x55f3a8395f80 2023-10-16T11:05:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.518+0000 7effe6ba0740 1 -- 172.21.15.84:0/382704870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3a82fb160 con 0x55f3a8395f80 2023-10-16T11:05:50.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.518+0000 7effbcff9640 1 -- 172.21.15.84:0/382704870 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7effd8053430 con 0x55f3a8395f80 2023-10-16T11:05:50.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.518+0000 7effe6ba0740 10 client.9843 fetch_fsmap learned FSMap version 88 2023-10-16T11:05:50.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.518+0000 7effe6ba0740 1 -- 172.21.15.84:0/382704870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x55f3a82d2da0 con 0x55f3a8395f80 2023-10-16T11:05:50.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.518+0000 7effbcff9640 1 -- 172.21.15.84:0/382704870 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f3a82fb160 con 0x55f3a8395f80 2023-10-16T11:05:50.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.519+0000 7effbcff9640 1 -- 172.21.15.84:0/382704870 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 88) v1 ==== 3294+0+0 (secure 0 0 0) 0x7effd8073070 con 0x55f3a8395f80 2023-10-16T11:05:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.519+0000 7effe6ba0740 10 client.9843 fetch_fsmap finished waiting for FSMap version 88 2023-10-16T11:05:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.519+0000 7effe6ba0740 10 client.9843 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T11:05:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.519+0000 7effe6ba0740 20 client.9843 populate_metadata read hostname 'smithi084' 2023-10-16T11:05:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.519+0000 7effe6ba0740 1 --2- 172.21.15.84:0/382704870 >> [v2:172.21.15.84:6836/424056375,v1:172.21.15.84:6837/424056375] conn(0x55f3a838f0c0 0x55f3a83c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.519+0000 7effe6ba0740 4 client.9843 mds_command: new command op to 9687 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T11:05:50.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.519+0000 7effe6ba0740 1 -- 172.21.15.84:0/382704870 --> [v2:172.21.15.84:6836/424056375,v1:172.21.15.84:6837/424056375] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3a82a6cc0 con 0x55f3a838f0c0 2023-10-16T11:05:50.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.519+0000 7effe14ce640 1 --2- 172.21.15.84:0/382704870 >> [v2:172.21.15.84:6836/424056375,v1:172.21.15.84:6837/424056375] conn(0x55f3a838f0c0 0x55f3a83c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:50.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.520+0000 7effe14ce640 1 --2- 172.21.15.84:0/382704870 >> [v2:172.21.15.84:6836/424056375,v1:172.21.15.84:6837/424056375] conn(0x55f3a838f0c0 0x55f3a83c72c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbc177b44631cc0d server_cookie=1a568331e224df1f in_seq=0 out_seq=0 2023-10-16T11:05:50.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.520+0000 7effbcff9640 10 client.9843 ms_handle_connect on v2:172.21.15.84:6836/424056375 2023-10-16T11:05:50.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.520+0000 7effbcff9640 1 -- 172.21.15.84:0/382704870 <== mds.0 v2:172.21.15.84:6836/424056375 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f3a82a6cc0 con 0x55f3a838f0c0 2023-10-16T11:05:50.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.520+0000 7effbcff9640 10 client.9843 handle_command_reply: tid=0 2023-10-16T11:05:50.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.520+0000 7effe6ba0740 1 client.9843 shutdown 2023-10-16T11:05:50.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.521+0000 7effe6ba0740 1 -- 172.21.15.84:0/382704870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 msgr2=0x55f3a8390f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:50.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.521+0000 7effe6ba0740 1 --2- 172.21.15.84:0/382704870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 0x55f3a8390f40 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7effd80050a0 tx=0x7effd8000fb0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.521+0000 7effe6ba0740 1 -- 172.21.15.84:0/382704870 shutdown_connections 2023-10-16T11:05:50.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.521+0000 7effe6ba0740 1 --2- 172.21.15.84:0/382704870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f3a8380360 0x55f3a839a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.521+0000 7effe6ba0740 1 --2- 172.21.15.84:0/382704870 >> [v2:172.21.15.84:6836/424056375,v1:172.21.15.84:6837/424056375] conn(0x55f3a838f0c0 0x55f3a83c72c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.521+0000 7effe6ba0740 1 --2- 172.21.15.84:0/382704870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a8396360 0x55f3a8398580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.521+0000 7effe6ba0740 1 --2- 172.21.15.84:0/382704870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 0x55f3a8390f40 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.521+0000 7effbcff9640 0 client.9843 ms_handle_reset on v2:172.21.15.84:6836/424056375 2023-10-16T11:05:50.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.521+0000 7effe6ba0740 1 -- 172.21.15.84:0/382704870 >> 172.21.15.84:0/382704870 conn(0x55f3a83693c0 msgr2=0x55f3a83b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:50.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.521+0000 7effe6ba0740 1 -- 172.21.15.84:0/382704870 shutdown_connections 2023-10-16T11:05:50.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.521+0000 7effe6ba0740 1 -- 172.21.15.84:0/382704870 wait complete. 2023-10-16T11:05:50.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.521+0000 7effe6ba0740 20 client.9843 trim_cache size 0 max 16384 2023-10-16T11:05:50.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.537+0000 7effe6ba0740 1 Processor -- start 2023-10-16T11:05:50.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.537+0000 7effe6ba0740 1 -- start start 2023-10-16T11:05:50.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.538+0000 7effe6ba0740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f3a8380360 0x55f3a8399390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.538+0000 7effe6ba0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a8395f80 0x55f3a8398480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.538+0000 7effe6ba0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8396360 0x55f3a83916d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.538+0000 7effe6ba0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a8038f20 con 0x55f3a8396360 2023-10-16T11:05:50.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.538+0000 7effe6ba0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a839cc50 con 0x55f3a8395f80 2023-10-16T11:05:50.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.538+0000 7effe6ba0740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3a839ce20 con 0x55f3a8380360 2023-10-16T11:05:50.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.539+0000 7effe24d0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8396360 0x55f3a83916d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:50.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.539+0000 7effe24d0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8396360 0x55f3a83916d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54810/0 (socket says 172.21.15.84:54810) 2023-10-16T11:05:50.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.539+0000 7effe24d0640 1 -- 172.21.15.84:0/98580419 learned_addr learned my addr 172.21.15.84:0/98580419 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:50.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.539+0000 7effe24d0640 1 -- 172.21.15.84:0/98580419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f3a8380360 msgr2=0x55f3a8399390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:50.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.539+0000 7effe24d0640 1 --2- 172.21.15.84:0/98580419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f3a8380360 0x55f3a8399390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.539+0000 7effe24d0640 1 -- 172.21.15.84:0/98580419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a8395f80 msgr2=0x55f3a8398480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:50.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.539+0000 7effe24d0640 1 --2- 172.21.15.84:0/98580419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a8395f80 0x55f3a8398480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.539+0000 7effe24d0640 1 -- 172.21.15.84:0/98580419 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3a8271b80 con 0x55f3a8396360 2023-10-16T11:05:50.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.539+0000 7effe24d0640 1 --2- 172.21.15.84:0/98580419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8396360 0x55f3a83916d0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7effd4009dd0 tx=0x7effd4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c156d7a453769836 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:50.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.539+0000 7effbffff640 1 -- 172.21.15.84:0/98580419 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4003b90 con 0x55f3a8396360 2023-10-16T11:05:50.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.540+0000 7effbffff640 1 -- 172.21.15.84:0/98580419 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd4003d10 con 0x55f3a8396360 2023-10-16T11:05:50.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.540+0000 7effbffff640 1 -- 172.21.15.84:0/98580419 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4004260 con 0x55f3a8396360 2023-10-16T11:05:50.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.540+0000 7effe6ba0740 1 -- 172.21.15.84:0/98580419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8396360 msgr2=0x55f3a83916d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:50.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.540+0000 7effe6ba0740 1 --2- 172.21.15.84:0/98580419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8396360 0x55f3a83916d0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7effd4009dd0 tx=0x7effd4007700 comp rx=0 tx=0).stop 2023-10-16T11:05:50.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.540+0000 7effe6ba0740 1 -- 172.21.15.84:0/98580419 shutdown_connections 2023-10-16T11:05:50.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.540+0000 7effe6ba0740 1 --2- 172.21.15.84:0/98580419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8396360 0x55f3a83916d0 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.540+0000 7effe6ba0740 1 --2- 172.21.15.84:0/98580419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a8395f80 0x55f3a8398480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.540+0000 7effe6ba0740 1 --2- 172.21.15.84:0/98580419 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f3a8380360 0x55f3a8399390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.540+0000 7effe6ba0740 1 -- 172.21.15.84:0/98580419 >> 172.21.15.84:0/98580419 conn(0x55f3a83693c0 msgr2=0x55f3a839a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:50.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.540+0000 7effe6ba0740 1 -- 172.21.15.84:0/98580419 shutdown_connections 2023-10-16T11:05:50.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.540+0000 7effe6ba0740 1 -- 172.21.15.84:0/98580419 wait complete. 2023-10-16T11:05:50.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.541+0000 7effe6ba0740 1 -- start start 2023-10-16T11:05:50.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.541+0000 7effe6ba0740 1 Processor -- start 2023-10-16T11:05:50.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.542+0000 7effe6ba0740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f3a8380360 0x55f3a838d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.542+0000 7effe6ba0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 0x55f3a838d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.542+0000 7effe6ba0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a8396360 0x55f3a83cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.542+0000 7effe6ba0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a8413a90 con 0x55f3a8395f80 2023-10-16T11:05:50.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.542+0000 7effe6ba0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a8399810 con 0x55f3a8396360 2023-10-16T11:05:50.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.542+0000 7effe6ba0740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3a838cbb0 con 0x55f3a8380360 2023-10-16T11:05:50.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.542+0000 7effe6ba0740 10 client.0 _ll_register_callbacks cb 0x55f3a8369310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:05:50.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.542+0000 7effe14ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 0x55f3a838d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:50.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.543+0000 7effe14ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 0x55f3a838d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54826/0 (socket says 172.21.15.84:54826) 2023-10-16T11:05:50.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.543+0000 7effe14ce640 1 -- 172.21.15.84:0/1609336559 learned_addr learned my addr 172.21.15.84:0/1609336559 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:50.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.543+0000 7effe14ce640 1 -- 172.21.15.84:0/1609336559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f3a8380360 msgr2=0x55f3a838d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:50.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.543+0000 7effe14ce640 1 --2- 172.21.15.84:0/1609336559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f3a8380360 0x55f3a838d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.543+0000 7effe14ce640 1 -- 172.21.15.84:0/1609336559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a8396360 msgr2=0x55f3a83cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:50.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.543+0000 7effe14ce640 1 --2- 172.21.15.84:0/1609336559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a8396360 0x55f3a83cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.543+0000 7effe14ce640 1 -- 172.21.15.84:0/1609336559 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd4009a80 con 0x55f3a8395f80 2023-10-16T11:05:50.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.543+0000 7effe14ce640 1 --2- 172.21.15.84:0/1609336559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 0x55f3a838d580 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7effcc00b2c0 tx=0x7effcc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a428748fa759d58e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:50.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.543+0000 7effbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T11:05:50.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.543+0000 7effbcff9640 1 -- 172.21.15.84:0/1609336559 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc046040 con 0x55f3a8395f80 2023-10-16T11:05:50.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.543+0000 7effbcff9640 1 -- 172.21.15.84:0/1609336559 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 5737+0+0 (secure 0 0 0) 0x7effcc04e480 con 0x55f3a8395f80 2023-10-16T11:05:50.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.543+0000 7effe6ba0740 1 -- 172.21.15.84:0/1609336559 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effd8086f00 con 0x55f3a8395f80 2023-10-16T11:05:50.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.544+0000 7effbcff9640 1 -- 172.21.15.84:0/1609336559 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7effcc04ee50 con 0x55f3a8395f80 2023-10-16T11:05:50.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.544+0000 7effe6ba0740 10 client.9849 fetch_fsmap learned FSMap version 88 2023-10-16T11:05:50.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.544+0000 7effe6ba0740 1 -- 172.21.15.84:0/1609336559 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x55f3a8297170 con 0x55f3a8395f80 2023-10-16T11:05:50.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.544+0000 7effbcff9640 1 -- 172.21.15.84:0/1609336559 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd8086f00 con 0x55f3a8395f80 2023-10-16T11:05:50.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.544+0000 7effbcff9640 1 -- 172.21.15.84:0/1609336559 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 88) v1 ==== 3294+0+0 (secure 0 0 0) 0x7effcc036910 con 0x55f3a8395f80 2023-10-16T11:05:50.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.544+0000 7effe6ba0740 10 client.9849 fetch_fsmap finished waiting for FSMap version 88 2023-10-16T11:05:50.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.544+0000 7effe6ba0740 10 client.9849 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T11:05:50.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.544+0000 7effe6ba0740 20 client.9849 populate_metadata read hostname 'smithi084' 2023-10-16T11:05:50.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.544+0000 7effe6ba0740 1 --2- 172.21.15.84:0/1609336559 >> [v2:172.21.15.84:6836/424056375,v1:172.21.15.84:6837/424056375] conn(0x55f3a83cd9c0 0x55f3a83cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.544+0000 7effe6ba0740 4 client.9849 mds_command: new command op to 9687 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T11:05:50.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.544+0000 7effe6ba0740 1 -- 172.21.15.84:0/1609336559 --> [v2:172.21.15.84:6836/424056375,v1:172.21.15.84:6837/424056375] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f3a82cd700 con 0x55f3a83cd9c0 2023-10-16T11:05:50.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.544+0000 7effe1ccf640 1 --2- 172.21.15.84:0/1609336559 >> [v2:172.21.15.84:6836/424056375,v1:172.21.15.84:6837/424056375] conn(0x55f3a83cd9c0 0x55f3a83cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:50.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.545+0000 7effe1ccf640 1 --2- 172.21.15.84:0/1609336559 >> [v2:172.21.15.84:6836/424056375,v1:172.21.15.84:6837/424056375] conn(0x55f3a83cd9c0 0x55f3a83cdda0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3760a5e676e236 server_cookie=8c6a69b40adc059b in_seq=0 out_seq=0 2023-10-16T11:05:50.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.545+0000 7effbcff9640 10 client.9849 ms_handle_connect on v2:172.21.15.84:6836/424056375 2023-10-16T11:05:50.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.545+0000 7effbcff9640 1 -- 172.21.15.84:0/1609336559 <== mds.0 v2:172.21.15.84:6836/424056375 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55f3a82cd700 con 0x55f3a83cd9c0 2023-10-16T11:05:50.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.545+0000 7effbcff9640 10 client.9849 handle_command_reply: tid=0 2023-10-16T11:05:50.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.545+0000 7effe6ba0740 1 client.9849 shutdown 2023-10-16T11:05:50.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.545+0000 7effe6ba0740 1 -- 172.21.15.84:0/1609336559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 msgr2=0x55f3a838d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:50.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.545+0000 7effe6ba0740 1 --2- 172.21.15.84:0/1609336559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 0x55f3a838d580 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7effcc00b2c0 tx=0x7effcc004730 comp rx=0 tx=0).stop 2023-10-16T11:05:50.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.545+0000 7effe6ba0740 1 -- 172.21.15.84:0/1609336559 shutdown_connections 2023-10-16T11:05:50.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.545+0000 7effe6ba0740 1 --2- 172.21.15.84:0/1609336559 >> [v2:172.21.15.84:6836/424056375,v1:172.21.15.84:6837/424056375] conn(0x55f3a83cd9c0 0x55f3a83cdda0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.545+0000 7effe6ba0740 1 --2- 172.21.15.84:0/1609336559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a8396360 0x55f3a83cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.545+0000 7effe6ba0740 1 --2- 172.21.15.84:0/1609336559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3a8395f80 0x55f3a838d580 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.545+0000 7effbcff9640 0 client.9849 ms_handle_reset on v2:172.21.15.84:6836/424056375 2023-10-16T11:05:50.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.545+0000 7effe6ba0740 1 --2- 172.21.15.84:0/1609336559 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f3a8380360 0x55f3a838d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.545+0000 7effe6ba0740 1 -- 172.21.15.84:0/1609336559 >> 172.21.15.84:0/1609336559 conn(0x55f3a83693c0 msgr2=0x55f3a83916d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:50.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.545+0000 7effe6ba0740 1 -- 172.21.15.84:0/1609336559 shutdown_connections 2023-10-16T11:05:50.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.545+0000 7effe6ba0740 1 -- 172.21.15.84:0/1609336559 wait complete. 2023-10-16T11:05:50.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.546+0000 7effe6ba0740 20 client.9849 trim_cache size 0 max 16384 2023-10-16T11:05:50.560 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:05:50.560 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "63fb4d68-f881-4694-b880-d10369105c0e", 2023-10-16T11:05:50.560 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-16T11:05:50.560 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 9687, 2023-10-16T11:05:50.560 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-16T11:05:50.561 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-16T11:05:50.561 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-16T11:05:50.561 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 6.6191409109999997, 2023-10-16T11:05:50.561 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 88, 2023-10-16T11:05:50.561 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 171, 2023-10-16T11:05:50.562 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 171, 2023-10-16T11:05:50.562 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 12.04391744 2023-10-16T11:05:50.562 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:05:50.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.546+0000 7effe3f33640 1 -- 172.21.15.84:0/350372831 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7effc006af90 msgr2=0x7effc006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:50.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.546+0000 7effe3f33640 1 --2- 172.21.15.84:0/350372831 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7effc006af90 0x7effc006d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7effcc038000 tx=0x7effcc030000 comp rx=0 tx=0).stop 2023-10-16T11:05:50.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.546+0000 7effe3f33640 1 -- 172.21.15.84:0/350372831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effdc06c140 msgr2=0x7effdc157ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:50.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.546+0000 7effe3f33640 1 --2- 172.21.15.84:0/350372831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effdc06c140 0x7effdc157ee0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7effd800c410 tx=0x7effd800c8d0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.546+0000 7effe3f33640 1 -- 172.21.15.84:0/350372831 shutdown_connections 2023-10-16T11:05:50.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.546+0000 7effe3f33640 1 --2- 172.21.15.84:0/350372831 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7effc006af90 0x7effc006d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.546+0000 7effe3f33640 1 --2- 172.21.15.84:0/350372831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effdc159e90 0x7effdc0b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.546+0000 7effe3f33640 1 --2- 172.21.15.84:0/350372831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effdc159ab0 0x7effdc15d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.546+0000 7effe3f33640 1 --2- 172.21.15.84:0/350372831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effdc06c140 0x7effdc157ee0 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.546+0000 7effe3f33640 1 -- 172.21.15.84:0/350372831 >> 172.21.15.84:0/350372831 conn(0x7effdc0ce9b0 msgr2=0x7effdc0b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:50.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.546+0000 7effe3f33640 1 -- 172.21.15.84:0/350372831 shutdown_connections 2023-10-16T11:05:50.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.546+0000 7effe3f33640 1 -- 172.21.15.84:0/350372831 wait complete. 2023-10-16T11:05:50.566 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T11:05:50.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.776+0000 7f11fd1d2640 1 Processor -- start 2023-10-16T11:05:50.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.776+0000 7f11fd1d2640 1 -- start start 2023-10-16T11:05:50.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.778+0000 7f11fd1d2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f11f815a9d0 0x7f11f8071c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.778+0000 7f11fd1d2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f815e550 con 0x7f11f80b2360 2023-10-16T11:05:50.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.778+0000 7f11fd1d2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f8161950 con 0x7f11f80b27c0 2023-10-16T11:05:50.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.778+0000 7f11fd1d2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f80d0770 con 0x7f11f815a9d0 2023-10-16T11:05:50.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.778+0000 7f11f7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f11f815a9d0 0x7f11f8071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:50.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.778+0000 7f11effff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f11f80b2360 0x7f11f8160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34336/0 (socket says 172.21.15.84:34336) 2023-10-16T11:05:50.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.778+0000 7f11effff640 1 -- 172.21.15.84:0/2394518012 learned_addr learned my addr 172.21.15.84:0/2394518012 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:50.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.779+0000 7f11f7fff640 1 -- 172.21.15.84:0/2394518012 >> v1:172.21.15.157:6789/0 conn(0x7f11f80b27c0 legacy=0x7f11f8161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:50.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.779+0000 7f11f7fff640 1 -- 172.21.15.84:0/2394518012 >> v1:172.21.15.84:6789/0 conn(0x7f11f80b2360 legacy=0x7f11f8160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:50.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.779+0000 7f11f7fff640 1 -- 172.21.15.84:0/2394518012 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f8072170 con 0x7f11f815a9d0 2023-10-16T11:05:50.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.779+0000 7f11f77fe640 1 -- 172.21.15.84:0/2394518012 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3885211431 0 0) 0x7f11f815e550 con 0x7f11f80b2360 2023-10-16T11:05:50.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.780+0000 7f11f7fff640 1 --2- 172.21.15.84:0/2394518012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11f815a9d0 0x7f11f8071c30 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f11e80097a0 tx=0x7f11e802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf04b459ffdb33e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:50.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.781+0000 7f11f77fe640 1 -- 172.21.15.84:0/2394518012 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e802e070 con 0x7f11f815a9d0 2023-10-16T11:05:50.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.781+0000 7f11f77fe640 1 -- 172.21.15.84:0/2394518012 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e80315a0 con 0x7f11f815a9d0 2023-10-16T11:05:50.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.781+0000 7f11f77fe640 1 -- 172.21.15.84:0/2394518012 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8031930 con 0x7f11f815a9d0 2023-10-16T11:05:50.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.781+0000 7f11fd1d2640 1 -- 172.21.15.84:0/2394518012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11f815a9d0 msgr2=0x7f11f8071c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:50.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.781+0000 7f11fd1d2640 1 --2- 172.21.15.84:0/2394518012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11f815a9d0 0x7f11f8071c30 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f11e80097a0 tx=0x7f11e802b840 comp rx=0 tx=0).stop 2023-10-16T11:05:50.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.781+0000 7f11fd1d2640 1 -- 172.21.15.84:0/2394518012 shutdown_connections 2023-10-16T11:05:50.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.781+0000 7f11fd1d2640 1 --2- 172.21.15.84:0/2394518012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11f815a9d0 0x7f11f8071c30 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.781+0000 7f11fd1d2640 1 -- 172.21.15.84:0/2394518012 >> 172.21.15.84:0/2394518012 conn(0x7f11f806d560 msgr2=0x7f11f806d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:50.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.781+0000 7f11fd1d2640 1 -- 172.21.15.84:0/2394518012 shutdown_connections 2023-10-16T11:05:50.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.781+0000 7f11fd1d2640 1 -- 172.21.15.84:0/2394518012 wait complete. 2023-10-16T11:05:50.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.783+0000 7f11fd1d2640 1 Processor -- start 2023-10-16T11:05:50.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.783+0000 7f11fd1d2640 1 -- start start 2023-10-16T11:05:50.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.784+0000 7f11fd1d2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11f80b2360 0x7f11f80c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.784+0000 7f11fd1d2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11f80b27c0 0x7f11f815c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.784+0000 7f11fd1d2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11f815a9d0 0x7f11f80b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.784+0000 7f11fd1d2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80b0050 con 0x7f11f815a9d0 2023-10-16T11:05:50.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.784+0000 7f11fd1d2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80c7ef0 con 0x7f11f80b27c0 2023-10-16T11:05:50.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.784+0000 7f11fd1d2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80b5790 con 0x7f11f80b2360 2023-10-16T11:05:50.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.784+0000 7f11fc9d1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11f815a9d0 0x7f11f80b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:50.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.784+0000 7f11fc9d1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11f815a9d0 0x7f11f80b4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54842/0 (socket says 172.21.15.84:54842) 2023-10-16T11:05:50.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.784+0000 7f11fc9d1640 1 -- 172.21.15.84:0/2011438745 learned_addr learned my addr 172.21.15.84:0/2011438745 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:50.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.785+0000 7f11fc9d1640 1 -- 172.21.15.84:0/2011438745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11f80b2360 msgr2=0x7f11f80c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:50.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.785+0000 7f11fc9d1640 1 --2- 172.21.15.84:0/2011438745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11f80b2360 0x7f11f80c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.785+0000 7f11fc9d1640 1 -- 172.21.15.84:0/2011438745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11f80b27c0 msgr2=0x7f11f815c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:50.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.785+0000 7f11effff640 1 --2- 172.21.15.84:0/2011438745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11f80b27c0 0x7f11f815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:50.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.785+0000 7f11fc9d1640 1 --2- 172.21.15.84:0/2011438745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11f80b27c0 0x7f11f815c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:50.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.785+0000 7f11fc9d1640 1 -- 172.21.15.84:0/2011438745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e8009400 con 0x7f11f815a9d0 2023-10-16T11:05:50.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.785+0000 7f11fc9d1640 1 --2- 172.21.15.84:0/2011438745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11f815a9d0 0x7f11f80b4b10 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f11f000c780 tx=0x7f11f000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:50.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.785+0000 7f11f57fa640 1 -- 172.21.15.84:0/2011438745 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0004b60 con 0x7f11f815a9d0 2023-10-16T11:05:50.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.785+0000 7f11f57fa640 1 -- 172.21.15.84:0/2011438745 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f000ce90 con 0x7f11f815a9d0 2023-10-16T11:05:50.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.785+0000 7f11f57fa640 1 -- 172.21.15.84:0/2011438745 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f000a850 con 0x7f11f815a9d0 2023-10-16T11:05:50.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.786+0000 7f11fd1d2640 1 -- 172.21.15.84:0/2011438745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f80b5a70 con 0x7f11f815a9d0 2023-10-16T11:05:50.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.786+0000 7f11fd1d2640 1 -- 172.21.15.84:0/2011438745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f80b6030 con 0x7f11f815a9d0 2023-10-16T11:05:50.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.787+0000 7f11fd1d2640 1 -- 172.21.15.84:0/2011438745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f804e9b0 con 0x7f11f815a9d0 2023-10-16T11:05:50.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.788+0000 7f11f57fa640 1 -- 172.21.15.84:0/2011438745 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f11f000a9b0 con 0x7f11f815a9d0 2023-10-16T11:05:50.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.789+0000 7f11f57fa640 1 --2- 172.21.15.84:0/2011438745 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f11d806af90 0x7f11d806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:50.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.789+0000 7f11f57fa640 1 -- 172.21.15.84:0/2011438745 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f11f0089950 con 0x7f11f815a9d0 2023-10-16T11:05:50.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.789+0000 7f11f7fff640 1 --2- 172.21.15.84:0/2011438745 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f11d806af90 0x7f11d806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:50.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.790+0000 7f11f7fff640 1 --2- 172.21.15.84:0/2011438745 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f11d806af90 0x7f11d806d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f11e8036000 tx=0x7f11e8030000 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:50.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:50.793+0000 7f11f57fa640 1 -- 172.21.15.84:0/2011438745 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f11f00548d0 con 0x7f11f815a9d0 2023-10-16T11:05:51.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:51.028+0000 7f11fd1d2640 1 -- 172.21.15.84:0/2011438745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f11f80b34a0 con 0x7f11f815a9d0 2023-10-16T11:05:51.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:51.029+0000 7f11f57fa640 1 -- 172.21.15.84:0/2011438745 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v88) v1 ==== 53+0+73 (secure 0 0 0) 0x7f11f80b34a0 con 0x7f11f815a9d0 2023-10-16T11:05:51.031 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T11:05:51.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:51.031+0000 7f11fd1d2640 1 -- 172.21.15.84:0/2011438745 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f11d806af90 msgr2=0x7f11d806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:51.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:51.032+0000 7f11fd1d2640 1 --2- 172.21.15.84:0/2011438745 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f11d806af90 0x7f11d806d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f11e8036000 tx=0x7f11e8030000 comp rx=0 tx=0).stop 2023-10-16T11:05:51.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:51.032+0000 7f11fd1d2640 1 -- 172.21.15.84:0/2011438745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11f815a9d0 msgr2=0x7f11f80b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:51.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:51.032+0000 7f11fd1d2640 1 --2- 172.21.15.84:0/2011438745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11f815a9d0 0x7f11f80b4b10 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f11f000c780 tx=0x7f11f000cc40 comp rx=0 tx=0).stop 2023-10-16T11:05:51.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:51.032+0000 7f11fd1d2640 1 -- 172.21.15.84:0/2011438745 shutdown_connections 2023-10-16T11:05:51.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:51.032+0000 7f11fd1d2640 1 --2- 172.21.15.84:0/2011438745 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f11d806af90 0x7f11d806d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:51.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:51.032+0000 7f11fd1d2640 1 --2- 172.21.15.84:0/2011438745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11f815a9d0 0x7f11f80b4b10 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:51.032+0000 7f11fd1d2640 1 --2- 172.21.15.84:0/2011438745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11f80b27c0 0x7f11f815c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:51.032+0000 7f11fd1d2640 1 --2- 172.21.15.84:0/2011438745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11f80b2360 0x7f11f80c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:51.032+0000 7f11fd1d2640 1 -- 172.21.15.84:0/2011438745 >> 172.21.15.84:0/2011438745 conn(0x7f11f806d560 msgr2=0x7f11f8072e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:51.032+0000 7f11fd1d2640 1 -- 172.21.15.84:0/2011438745 shutdown_connections 2023-10-16T11:05:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:51.032+0000 7f11fd1d2640 1 -- 172.21.15.84:0/2011438745 wait complete. 2023-10-16T11:05:51.053 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T11:05:51.053 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T11:05:51.054 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-16T11:05:51.054 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T11:05:51.054 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T11:05:51.055 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T11:05:51.055 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T11:05:51.055 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T11:05:51.056 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:05:51.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:51.057 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-16T11:05:51.108 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T11:05:51.109 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T11:05:51.109 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T11:05:51.109 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:05:51.109 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T11:05:51.110 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:05:51.110 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:05:51.110 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:05:51.110 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-16T11:05:51.111 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:05:51.111 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:05:51.111 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-16T11:05:51.111 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:05:51.111 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:05:51.111 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T11:05:51.112 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:05:51.112 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T11:05:51.112 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 37800sec preferred_lft 37800sec 2023-10-16T11:05:51.112 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-16T11:05:51.112 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:05:51.113 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T11:05:51.113 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:05:51.113 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T11:05:51.113 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:05:51.113 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::20a8:eaff:fe9d:5cea/64 scope link 2023-10-16T11:05:51.114 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:05:51.114 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:05:51.114 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T11:05:51.114 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1854:13ff:fe47:be92/64 scope link 2023-10-16T11:05:51.114 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:05:51.115 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:05:51.115 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether ee:d5:41:a3:a5:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T11:05:51.115 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ecd5:41ff:fea3:a570/64 scope link 2023-10-16T11:05:51.115 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:05:51.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:51.117 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T11:05:51.209 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-16T11:05:51.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:51.214 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-16T11:05:51.270 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-16T11:05:51.271 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T11:05:51.271 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T11:05:51.271 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T11:05:51.271 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T11:05:51.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:51.272 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T11:05:51.273 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T11:05:51.273 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T11:05:51.273 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T11:05:51.274 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T11:05:51.274 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T11:05:51.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:51.463 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T11:05:51.518 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T11:05:51.518 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T11:05:51.519 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:05:51.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:51.520 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-16T11:05:51.616 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:05:51.616 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T11:05:51.673 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:05:51.673 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:05:51.673 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:05:51.727 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-16T11:05:51.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:51.728 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:05:51.869 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:05:51.869 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T11:05:51.901 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:05:51.901 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T11:05:51.902 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T11:05:51.902 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with new device syntax: 0@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T11:05:51.902 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T11:05:51.903 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-16T11:05:51.903 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2023-10-16T11:05:51.904 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T11:05:51.904 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T11:05:51.904 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T11:05:51.907 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T11:05:51.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:51.908 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:05:51.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:51.989 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T11:05:52.044 INFO:teuthology.orchestra.run.smithi084.stdout:client6191 2023-10-16T11:05:52.045 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:05:52.045 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:05:52.094 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T11:05:52.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.361+0000 7efc4eb01640 1 Processor -- start 2023-10-16T11:05:52.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.362+0000 7efc4eb01640 1 -- start start 2023-10-16T11:05:52.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.363+0000 7efc4eb01640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efc4815ad70 0x7efc48069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:52.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.363+0000 7efc4eb01640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efc480c9ca0 0x7efc4806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.363+0000 7efc4eb01640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc480cb030 con 0x7efc480ca3b0 2023-10-16T11:05:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.363+0000 7efc4eb01640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc480c91c0 con 0x7efc4815ad70 2023-10-16T11:05:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.363+0000 7efc4eb01640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc480c94a0 con 0x7efc480c9ca0 2023-10-16T11:05:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.363+0000 7efc4c89d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efc4815ad70 0x7efc48069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:52.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.364+0000 7efc4c89d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efc4815ad70 0x7efc48069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54856/0 (socket says 172.21.15.84:54856) 2023-10-16T11:05:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.364+0000 7efc4c89d640 1 -- 172.21.15.84:0/3700803880 learned_addr learned my addr 172.21.15.84:0/3700803880 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.364+0000 7efc47fff640 1 --2- 172.21.15.84:0/3700803880 >> v2:172.21.15.157:3300/0 conn(0x7efc480c9ca0 0x7efc4806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.364+0000 7efc4c89d640 1 -- 172.21.15.84:0/3700803880 >> v2:172.21.15.157:3300/0 conn(0x7efc480c9ca0 msgr2=0x7efc4806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.364+0000 7efc4c89d640 1 --2- 172.21.15.84:0/3700803880 >> v2:172.21.15.157:3300/0 conn(0x7efc480c9ca0 0x7efc4806a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:52.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.364+0000 7efc4c89d640 1 -- 172.21.15.84:0/3700803880 >> v1:172.21.15.84:6789/0 conn(0x7efc480ca3b0 legacy=0x7efc480ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:52.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.364+0000 7efc4c89d640 1 -- 172.21.15.84:0/3700803880 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc480c9780 con 0x7efc4815ad70 2023-10-16T11:05:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.365+0000 7efc4c89d640 1 --2- 172.21.15.84:0/3700803880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc4815ad70 0x7efc48069da0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7efc480ce870 tx=0x7efc3c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbad204917c6651 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.366+0000 7efc477fe640 1 -- 172.21.15.84:0/3700803880 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c031870 con 0x7efc4815ad70 2023-10-16T11:05:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.366+0000 7efc477fe640 1 -- 172.21.15.84:0/3700803880 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3c0319d0 con 0x7efc4815ad70 2023-10-16T11:05:52.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.366+0000 7efc477fe640 1 -- 172.21.15.84:0/3700803880 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c031d60 con 0x7efc4815ad70 2023-10-16T11:05:52.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.366+0000 7efc4eb01640 1 -- 172.21.15.84:0/3700803880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc4815ad70 msgr2=0x7efc48069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:52.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.366+0000 7efc4eb01640 1 --2- 172.21.15.84:0/3700803880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc4815ad70 0x7efc48069da0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7efc480ce870 tx=0x7efc3c02b910 comp rx=0 tx=0).stop 2023-10-16T11:05:52.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.366+0000 7efc4eb01640 1 -- 172.21.15.84:0/3700803880 shutdown_connections 2023-10-16T11:05:52.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.366+0000 7efc4eb01640 1 --2- 172.21.15.84:0/3700803880 >> v2:172.21.15.157:3300/0 conn(0x7efc480c9ca0 0x7efc4806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:52.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.366+0000 7efc4eb01640 1 --2- 172.21.15.84:0/3700803880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc4815ad70 0x7efc48069da0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:52.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.366+0000 7efc4eb01640 1 -- 172.21.15.84:0/3700803880 >> 172.21.15.84:0/3700803880 conn(0x7efc480ce9b0 msgr2=0x7efc480cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:52.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.367+0000 7efc4eb01640 1 -- 172.21.15.84:0/3700803880 shutdown_connections 2023-10-16T11:05:52.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.367+0000 7efc4eb01640 1 -- 172.21.15.84:0/3700803880 wait complete. 2023-10-16T11:05:52.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.368+0000 7efc4eb01640 1 Processor -- start 2023-10-16T11:05:52.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.369+0000 7efc4eb01640 1 -- start start 2023-10-16T11:05:52.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.369+0000 7efc4eb01640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc480c9ca0 0x7efc48150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:52.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.370+0000 7efc4eb01640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc480ca3b0 0x7efc480cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:52.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.370+0000 7efc4eb01640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc4815ad70 0x7efc480ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:52.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.370+0000 7efc4eb01640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc48151060 con 0x7efc4815ad70 2023-10-16T11:05:52.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.370+0000 7efc4eb01640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc480cc580 con 0x7efc480ca3b0 2023-10-16T11:05:52.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.370+0000 7efc4eb01640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc480cd100 con 0x7efc480c9ca0 2023-10-16T11:05:52.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.370+0000 7efc4d09e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc4815ad70 0x7efc480ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:52.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.370+0000 7efc4d09e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc4815ad70 0x7efc480ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54858/0 (socket says 172.21.15.84:54858) 2023-10-16T11:05:52.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.370+0000 7efc4d09e640 1 -- 172.21.15.84:0/178641675 learned_addr learned my addr 172.21.15.84:0/178641675 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:52.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.370+0000 7efc4c89d640 1 --2- 172.21.15.84:0/178641675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc480c9ca0 0x7efc48150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:52.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.370+0000 7efc47fff640 1 --2- 172.21.15.84:0/178641675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc480ca3b0 0x7efc480cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:52.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.370+0000 7efc4d09e640 1 -- 172.21.15.84:0/178641675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc480c9ca0 msgr2=0x7efc48150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:52.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.370+0000 7efc4d09e640 1 --2- 172.21.15.84:0/178641675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc480c9ca0 0x7efc48150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:52.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.370+0000 7efc4d09e640 1 -- 172.21.15.84:0/178641675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc480ca3b0 msgr2=0x7efc480cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:52.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.370+0000 7efc4d09e640 1 --2- 172.21.15.84:0/178641675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc480ca3b0 0x7efc480cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:52.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.370+0000 7efc4d09e640 1 -- 172.21.15.84:0/178641675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c0094d0 con 0x7efc4815ad70 2023-10-16T11:05:52.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.370+0000 7efc47fff640 1 --2- 172.21.15.84:0/178641675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc480ca3b0 0x7efc480cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:52.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.371+0000 7efc4d09e640 1 --2- 172.21.15.84:0/178641675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc4815ad70 0x7efc480ac890 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7efc4000c7b0 tx=0x7efc4000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:52.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.371+0000 7efc457fa640 1 -- 172.21.15.84:0/178641675 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc40004b60 con 0x7efc4815ad70 2023-10-16T11:05:52.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.371+0000 7efc457fa640 1 -- 172.21.15.84:0/178641675 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc40004cc0 con 0x7efc4815ad70 2023-10-16T11:05:52.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.371+0000 7efc4eb01640 1 -- 172.21.15.84:0/178641675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc480cc7d0 con 0x7efc4815ad70 2023-10-16T11:05:52.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.371+0000 7efc457fa640 1 -- 172.21.15.84:0/178641675 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4000aa90 con 0x7efc4815ad70 2023-10-16T11:05:52.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.371+0000 7efc4eb01640 1 -- 172.21.15.84:0/178641675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc480ccd90 con 0x7efc4815ad70 2023-10-16T11:05:52.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.373+0000 7efc4eb01640 1 -- 172.21.15.84:0/178641675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4804e160 con 0x7efc4815ad70 2023-10-16T11:05:52.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.373+0000 7efc457fa640 1 -- 172.21.15.84:0/178641675 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efc4000abf0 con 0x7efc4815ad70 2023-10-16T11:05:52.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.374+0000 7efc457fa640 1 --2- 172.21.15.84:0/178641675 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7efc2c06af90 0x7efc2c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:52.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.374+0000 7efc457fa640 1 -- 172.21.15.84:0/178641675 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5737+0+0 (secure 0 0 0) 0x7efc40089900 con 0x7efc4815ad70 2023-10-16T11:05:52.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.375+0000 7efc4c89d640 1 --2- 172.21.15.84:0/178641675 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7efc2c06af90 0x7efc2c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:52.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.375+0000 7efc4c89d640 1 --2- 172.21.15.84:0/178641675 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7efc2c06af90 0x7efc2c06d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efc3c036000 tx=0x7efc3c03c040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:52.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.379+0000 7efc457fa640 1 -- 172.21.15.84:0/178641675 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efc40054870 con 0x7efc4815ad70 2023-10-16T11:05:52.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.596+0000 7efc4eb01640 1 -- 172.21.15.84:0/178641675 --> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7efc480ad4d0 con 0x7efc2c06af90 2023-10-16T11:05:52.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.598+0000 7efc457fa640 1 -- 172.21.15.84:0/178641675 <== mgr.6631 v2:172.21.15.84:6832/2442400184 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7efc3c031830 con 0x7efc2c06af90 2023-10-16T11:05:52.600 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T11:05:52.600 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:05:52.600 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephfs" 2023-10-16T11:05:52.600 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:05:52.600 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T11:05:52.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.600+0000 7efc4eb01640 1 -- 172.21.15.84:0/178641675 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7efc2c06af90 msgr2=0x7efc2c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:52.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.600+0000 7efc4eb01640 1 --2- 172.21.15.84:0/178641675 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7efc2c06af90 0x7efc2c06d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efc3c036000 tx=0x7efc3c03c040 comp rx=0 tx=0).stop 2023-10-16T11:05:52.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.600+0000 7efc4eb01640 1 -- 172.21.15.84:0/178641675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc4815ad70 msgr2=0x7efc480ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:52.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.600+0000 7efc4eb01640 1 --2- 172.21.15.84:0/178641675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc4815ad70 0x7efc480ac890 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7efc4000c7b0 tx=0x7efc4000cc70 comp rx=0 tx=0).stop 2023-10-16T11:05:52.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.600+0000 7efc4eb01640 1 -- 172.21.15.84:0/178641675 shutdown_connections 2023-10-16T11:05:52.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.600+0000 7efc4eb01640 1 --2- 172.21.15.84:0/178641675 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7efc2c06af90 0x7efc2c06d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:52.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.600+0000 7efc4eb01640 1 --2- 172.21.15.84:0/178641675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc4815ad70 0x7efc480ac890 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:52.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.600+0000 7efc4eb01640 1 --2- 172.21.15.84:0/178641675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc480ca3b0 0x7efc480cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:52.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.600+0000 7efc4eb01640 1 --2- 172.21.15.84:0/178641675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc480c9ca0 0x7efc48150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:52.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.600+0000 7efc4eb01640 1 -- 172.21.15.84:0/178641675 >> 172.21.15.84:0/178641675 conn(0x7efc480ce9b0 msgr2=0x7efc48070d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:52.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.601+0000 7efc4eb01640 1 -- 172.21.15.84:0/178641675 shutdown_connections 2023-10-16T11:05:52.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.601+0000 7efc4eb01640 1 -- 172.21.15.84:0/178641675 wait complete. 2023-10-16T11:05:52.621 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T11:05:52.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.837+0000 7fa55d4a9640 1 Processor -- start 2023-10-16T11:05:52.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.837+0000 7fa55d4a9640 1 -- start start 2023-10-16T11:05:52.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.839+0000 7fa55d4a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa55815ad70 0x7fa558069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:52.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.839+0000 7fa55d4a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa5580c9ca0 0x7fa55806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:52.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.839+0000 7fa55d4a9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa5580ca3b0 0x7fa5580ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:52.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.839+0000 7fa55d4a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5580cad80 con 0x7fa5580c9ca0 2023-10-16T11:05:52.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.839+0000 7fa55d4a9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5580cb030 con 0x7fa5580ca3b0 2023-10-16T11:05:52.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.839+0000 7fa55d4a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5580c91c0 con 0x7fa55815ad70 2023-10-16T11:05:52.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.839+0000 7fa5567fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa5580c9ca0 0x7fa55806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:52.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.839+0000 7fa5567fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa5580c9ca0 0x7fa55806a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54866/0 (socket says 172.21.15.84:54866) 2023-10-16T11:05:52.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.839+0000 7fa5567fc640 1 -- 172.21.15.84:0/3538065152 learned_addr learned my addr 172.21.15.84:0/3538065152 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:52.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.840+0000 7fa556ffd640 1 --2- 172.21.15.84:0/3538065152 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa55815ad70 0x7fa558069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:52.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.840+0000 7fa5577fe640 1 --2- 172.21.15.84:0/3538065152 >> v2:172.21.15.157:3300/0 conn(0x7fa5580ca3b0 0x7fa5580ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:52.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.840+0000 7fa5567fc640 1 -- 172.21.15.84:0/3538065152 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa55815ad70 msgr2=0x7fa558069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:52.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.840+0000 7fa5567fc640 1 --2- 172.21.15.84:0/3538065152 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa55815ad70 0x7fa558069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:52.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.840+0000 7fa5567fc640 1 -- 172.21.15.84:0/3538065152 >> v2:172.21.15.157:3300/0 conn(0x7fa5580ca3b0 msgr2=0x7fa5580ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:52.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.840+0000 7fa5567fc640 1 --2- 172.21.15.84:0/3538065152 >> v2:172.21.15.157:3300/0 conn(0x7fa5580ca3b0 0x7fa5580ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:52.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.840+0000 7fa5567fc640 1 -- 172.21.15.84:0/3538065152 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5580c94a0 con 0x7fa5580c9ca0 2023-10-16T11:05:52.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.841+0000 7fa5567fc640 1 --2- 172.21.15.84:0/3538065152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5580c9ca0 0x7fa55806a2e0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fa5580cf820 tx=0x7fa54002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f3c996b0a0ff8a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:52.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.842+0000 7fa555ffb640 1 -- 172.21.15.84:0/3538065152 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540031870 con 0x7fa5580c9ca0 2023-10-16T11:05:52.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.842+0000 7fa555ffb640 1 -- 172.21.15.84:0/3538065152 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5400319d0 con 0x7fa5580c9ca0 2023-10-16T11:05:52.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.842+0000 7fa555ffb640 1 -- 172.21.15.84:0/3538065152 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540031d60 con 0x7fa5580c9ca0 2023-10-16T11:05:52.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.842+0000 7fa55d4a9640 1 -- 172.21.15.84:0/3538065152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5580c9ca0 msgr2=0x7fa55806a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:52.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.842+0000 7fa55d4a9640 1 --2- 172.21.15.84:0/3538065152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5580c9ca0 0x7fa55806a2e0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fa5580cf820 tx=0x7fa54002b910 comp rx=0 tx=0).stop 2023-10-16T11:05:52.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.842+0000 7fa55d4a9640 1 -- 172.21.15.84:0/3538065152 shutdown_connections 2023-10-16T11:05:52.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.842+0000 7fa55d4a9640 1 --2- 172.21.15.84:0/3538065152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5580c9ca0 0x7fa55806a2e0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:52.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.842+0000 7fa55d4a9640 1 --2- 172.21.15.84:0/3538065152 >> v2:172.21.15.157:3300/0 conn(0x7fa5580ca3b0 0x7fa5580ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:52.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.842+0000 7fa55d4a9640 1 --2- 172.21.15.84:0/3538065152 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa55815ad70 0x7fa558069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:52.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.842+0000 7fa55d4a9640 1 -- 172.21.15.84:0/3538065152 >> 172.21.15.84:0/3538065152 conn(0x7fa5580ce9b0 msgr2=0x7fa5580cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:52.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.843+0000 7fa55d4a9640 1 -- 172.21.15.84:0/3538065152 shutdown_connections 2023-10-16T11:05:52.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.843+0000 7fa55d4a9640 1 -- 172.21.15.84:0/3538065152 wait complete. 2023-10-16T11:05:52.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.845+0000 7fa55d4a9640 1 Processor -- start 2023-10-16T11:05:52.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.845+0000 7fa55d4a9640 1 -- start start 2023-10-16T11:05:52.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.846+0000 7fa55d4a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5580c9ca0 0x7fa5580c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:52.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.846+0000 7fa55d4a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5580ca3b0 0x7fa558158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:52.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.846+0000 7fa55d4a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa55815ad70 0x7fa5580ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:52.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.846+0000 7fa55d4a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5580c3800 con 0x7fa5580ca3b0 2023-10-16T11:05:52.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.846+0000 7fa55d4a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5580c3ae0 con 0x7fa5580c9ca0 2023-10-16T11:05:52.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.846+0000 7fa55d4a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5580c3dc0 con 0x7fa55815ad70 2023-10-16T11:05:52.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.847+0000 7fa5567fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5580ca3b0 0x7fa558158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:52.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.847+0000 7fa5567fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5580ca3b0 0x7fa558158070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54874/0 (socket says 172.21.15.84:54874) 2023-10-16T11:05:52.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.847+0000 7fa5567fc640 1 -- 172.21.15.84:0/2269278928 learned_addr learned my addr 172.21.15.84:0/2269278928 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:52.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.847+0000 7fa556ffd640 1 --2- 172.21.15.84:0/2269278928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5580c9ca0 0x7fa5580c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:52.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.847+0000 7fa5577fe640 1 --2- 172.21.15.84:0/2269278928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa55815ad70 0x7fa5580ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:52.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.847+0000 7fa5567fc640 1 -- 172.21.15.84:0/2269278928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa55815ad70 msgr2=0x7fa5580ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:52.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.847+0000 7fa5567fc640 1 --2- 172.21.15.84:0/2269278928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa55815ad70 0x7fa5580ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:52.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.847+0000 7fa5567fc640 1 -- 172.21.15.84:0/2269278928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5580c9ca0 msgr2=0x7fa5580c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:52.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.847+0000 7fa5567fc640 1 --2- 172.21.15.84:0/2269278928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5580c9ca0 0x7fa5580c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:52.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.847+0000 7fa5567fc640 1 -- 172.21.15.84:0/2269278928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5400094d0 con 0x7fa5580ca3b0 2023-10-16T11:05:52.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.847+0000 7fa556ffd640 1 --2- 172.21.15.84:0/2269278928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5580c9ca0 0x7fa5580c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:52.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.847+0000 7fa5577fe640 1 --2- 172.21.15.84:0/2269278928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa55815ad70 0x7fa5580ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:52.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.847+0000 7fa5567fc640 1 --2- 172.21.15.84:0/2269278928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5580ca3b0 0x7fa558158070 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fa540000c00 tx=0x7fa5400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:52.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.848+0000 7fa54f7fe640 1 -- 172.21.15.84:0/2269278928 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54003abe0 con 0x7fa5580ca3b0 2023-10-16T11:05:52.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.848+0000 7fa54f7fe640 1 -- 172.21.15.84:0/2269278928 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54003ad40 con 0x7fa5580ca3b0 2023-10-16T11:05:52.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.848+0000 7fa54f7fe640 1 -- 172.21.15.84:0/2269278928 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540044610 con 0x7fa5580ca3b0 2023-10-16T11:05:52.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.848+0000 7fa55d4a9640 1 -- 172.21.15.84:0/2269278928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5580c40a0 con 0x7fa5580ca3b0 2023-10-16T11:05:52.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.848+0000 7fa55d4a9640 1 -- 172.21.15.84:0/2269278928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5581585b0 con 0x7fa5580ca3b0 2023-10-16T11:05:52.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.849+0000 7fa55d4a9640 1 -- 172.21.15.84:0/2269278928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55804e160 con 0x7fa5580ca3b0 2023-10-16T11:05:52.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.850+0000 7fa54f7fe640 1 -- 172.21.15.84:0/2269278928 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa54004d450 con 0x7fa5580ca3b0 2023-10-16T11:05:52.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.851+0000 7fa54f7fe640 1 --2- 172.21.15.84:0/2269278928 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa53806af90 0x7fa53806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:52.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.851+0000 7fa54f7fe640 1 -- 172.21.15.84:0/2269278928 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa54002d040 con 0x7fa5580ca3b0 2023-10-16T11:05:52.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.851+0000 7fa556ffd640 1 --2- 172.21.15.84:0/2269278928 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa53806af90 0x7fa53806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:52.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.852+0000 7fa556ffd640 1 --2- 172.21.15.84:0/2269278928 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa53806af90 0x7fa53806d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa548006fd0 tx=0x7fa548008040 comp rx=0 tx=0).ready entity=mgr.6631 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:52.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:52.855+0000 7fa54f7fe640 1 -- 172.21.15.84:0/2269278928 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa540076da0 con 0x7fa5580ca3b0 2023-10-16T11:05:53.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.096+0000 7fa55d4a9640 1 -- 172.21.15.84:0/2269278928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fa5580ced90 con 0x7fa5580ca3b0 2023-10-16T11:05:53.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.395+0000 7fa54f7fe640 1 -- 172.21.15.84:0/2269278928 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v94) v1 ==== 92+0+0 (secure 0 0 0) 0x7fa54007a3c0 con 0x7fa5580ca3b0 2023-10-16T11:05:53.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.399+0000 7fa55d4a9640 1 -- 172.21.15.84:0/2269278928 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa53806af90 msgr2=0x7fa53806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:53.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.399+0000 7fa55d4a9640 1 --2- 172.21.15.84:0/2269278928 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa53806af90 0x7fa53806d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa548006fd0 tx=0x7fa548008040 comp rx=0 tx=0).stop 2023-10-16T11:05:53.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.400+0000 7fa55d4a9640 1 -- 172.21.15.84:0/2269278928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5580ca3b0 msgr2=0x7fa558158070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:53.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.400+0000 7fa55d4a9640 1 --2- 172.21.15.84:0/2269278928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5580ca3b0 0x7fa558158070 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fa540000c00 tx=0x7fa5400046c0 comp rx=0 tx=0).stop 2023-10-16T11:05:53.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.400+0000 7fa55d4a9640 1 -- 172.21.15.84:0/2269278928 shutdown_connections 2023-10-16T11:05:53.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.400+0000 7fa55d4a9640 1 --2- 172.21.15.84:0/2269278928 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7fa53806af90 0x7fa53806d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:53.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.400+0000 7fa55d4a9640 1 --2- 172.21.15.84:0/2269278928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa55815ad70 0x7fa5580ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:53.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.400+0000 7fa55d4a9640 1 --2- 172.21.15.84:0/2269278928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5580ca3b0 0x7fa558158070 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:53.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.400+0000 7fa55d4a9640 1 --2- 172.21.15.84:0/2269278928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5580c9ca0 0x7fa5580c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:53.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.400+0000 7fa55d4a9640 1 -- 172.21.15.84:0/2269278928 >> 172.21.15.84:0/2269278928 conn(0x7fa5580ce9b0 msgr2=0x7fa558069fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:53.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.400+0000 7fa55d4a9640 1 -- 172.21.15.84:0/2269278928 shutdown_connections 2023-10-16T11:05:53.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.400+0000 7fa55d4a9640 1 -- 172.21.15.84:0/2269278928 wait complete. 2023-10-16T11:05:53.433 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T11:05:53.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.630+0000 7eff924c9640 1 Processor -- start 2023-10-16T11:05:53.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.630+0000 7eff924c9640 1 -- start start 2023-10-16T11:05:53.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.632+0000 7eff924c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff8c15af80 0x7eff8c0cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:53.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.632+0000 7eff924c9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7eff8c065ab0 0x7eff8c065f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:53.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.632+0000 7eff924c9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff8c066450 con 0x7eff8c06faf0 2023-10-16T11:05:53.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.632+0000 7eff924c9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff8c0d0930 con 0x7eff8c065ab0 2023-10-16T11:05:53.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.632+0000 7eff924c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c15eff0 con 0x7eff8c15af80 2023-10-16T11:05:53.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.632+0000 7eff83fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7eff8c06faf0 0x7eff8c15f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34350/0 (socket says 172.21.15.84:34350) 2023-10-16T11:05:53.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.632+0000 7eff83fff640 1 -- 172.21.15.84:0/1408589727 learned_addr learned my addr 172.21.15.84:0/1408589727 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:53.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.632+0000 7eff8bfff640 1 --2- 172.21.15.84:0/1408589727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff8c15af80 0x7eff8c0cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:53.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.632+0000 7eff90a66640 1 --2- 172.21.15.84:0/1408589727 >> v2:172.21.15.157:3300/0 conn(0x7eff8c065ab0 0x7eff8c065f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:53.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.633+0000 7eff8b7fe640 1 -- 172.21.15.84:0/1408589727 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28969008 0 0) 0x7eff8c066450 con 0x7eff8c06faf0 2023-10-16T11:05:53.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.633+0000 7eff8b7fe640 1 -- 172.21.15.84:0/1408589727 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff78003520 con 0x7eff8c06faf0 2023-10-16T11:05:53.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.633+0000 7eff8b7fe640 1 -- 172.21.15.84:0/1408589727 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3401858651 0 0) 0x7eff78003520 con 0x7eff8c06faf0 2023-10-16T11:05:53.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.633+0000 7eff8b7fe640 1 -- 172.21.15.84:0/1408589727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff8c15af80 msgr2=0x7eff8c0cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:53.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.633+0000 7eff8b7fe640 1 --2- 172.21.15.84:0/1408589727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff8c15af80 0x7eff8c0cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:53.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.633+0000 7eff8b7fe640 1 -- 172.21.15.84:0/1408589727 >> v2:172.21.15.157:3300/0 conn(0x7eff8c065ab0 msgr2=0x7eff8c065f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:53.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.633+0000 7eff8b7fe640 1 --2- 172.21.15.84:0/1408589727 >> v2:172.21.15.157:3300/0 conn(0x7eff8c065ab0 0x7eff8c065f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:53.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.633+0000 7eff8b7fe640 1 -- 172.21.15.84:0/1408589727 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c15ecc0 con 0x7eff8c06faf0 2023-10-16T11:05:53.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.633+0000 7eff8b7fe640 1 -- 172.21.15.84:0/1408589727 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7eff74001670 con 0x7eff8c06faf0 2023-10-16T11:05:53.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.633+0000 7eff8b7fe640 1 -- 172.21.15.84:0/1408589727 >> v1:172.21.15.84:6789/0 conn(0x7eff8c06faf0 legacy=0x7eff8c15f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:53.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.633+0000 7eff8b7fe640 1 --2- 172.21.15.84:0/1408589727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff78003970 0x7eff78003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:53.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.633+0000 7eff8b7fe640 1 --2- 172.21.15.84:0/1408589727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff780047c0 0x7eff78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:53.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.633+0000 7eff8b7fe640 1 --2- 172.21.15.84:0/1408589727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff8c15af80 0x7eff78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:53.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.633+0000 7eff8b7fe640 1 -- 172.21.15.84:0/1408589727 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c0d0930 con 0x7eff780047c0 2023-10-16T11:05:53.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.633+0000 7eff8b7fe640 1 -- 172.21.15.84:0/1408589727 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c15eff0 con 0x7eff78003970 2023-10-16T11:05:53.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.633+0000 7eff8b7fe640 1 -- 172.21.15.84:0/1408589727 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7800b980 con 0x7eff8c15af80 2023-10-16T11:05:53.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.634+0000 7eff83fff640 1 --2- 172.21.15.84:0/1408589727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff780047c0 0x7eff78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:53.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.634+0000 7eff8bfff640 1 --2- 172.21.15.84:0/1408589727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff78003970 0x7eff78003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:53.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.634+0000 7eff90a66640 1 --2- 172.21.15.84:0/1408589727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff8c15af80 0x7eff78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:53.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.634+0000 7eff83fff640 1 -- 172.21.15.84:0/1408589727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff8c15af80 msgr2=0x7eff78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:53.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.634+0000 7eff83fff640 1 --2- 172.21.15.84:0/1408589727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff8c15af80 0x7eff78005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:53.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.634+0000 7eff83fff640 1 -- 172.21.15.84:0/1408589727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff78003970 msgr2=0x7eff78003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:53.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.634+0000 7eff83fff640 1 --2- 172.21.15.84:0/1408589727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff78003970 0x7eff78003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:53.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.634+0000 7eff83fff640 1 -- 172.21.15.84:0/1408589727 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff78003520 con 0x7eff780047c0 2023-10-16T11:05:53.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.635+0000 7eff83fff640 1 --2- 172.21.15.84:0/1408589727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff780047c0 0x7eff78004c40 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7eff7400c2c0 tx=0x7eff780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff4e8e9effd70d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:53.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.635+0000 7eff8b7fe640 1 -- 172.21.15.84:0/1408589727 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff74005ad0 con 0x7eff780047c0 2023-10-16T11:05:53.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.636+0000 7eff8b7fe640 1 -- 172.21.15.84:0/1408589727 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff74005c30 con 0x7eff780047c0 2023-10-16T11:05:53.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.636+0000 7eff8b7fe640 1 -- 172.21.15.84:0/1408589727 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7402c8b0 con 0x7eff780047c0 2023-10-16T11:05:53.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.636+0000 7eff924c9640 1 -- 172.21.15.84:0/1408589727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff780047c0 msgr2=0x7eff78004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:53.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.636+0000 7eff924c9640 1 --2- 172.21.15.84:0/1408589727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff780047c0 0x7eff78004c40 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7eff7400c2c0 tx=0x7eff780057f0 comp rx=0 tx=0).stop 2023-10-16T11:05:53.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.636+0000 7eff90a66640 1 -- 172.21.15.84:0/1408589727 reap_dead start 2023-10-16T11:05:53.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.636+0000 7eff924c9640 1 -- 172.21.15.84:0/1408589727 shutdown_connections 2023-10-16T11:05:53.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.636+0000 7eff924c9640 1 -- 172.21.15.84:0/1408589727 >> 172.21.15.84:0/1408589727 conn(0x7eff8c0d0ab0 msgr2=0x7eff8c0d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:53.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.636+0000 7eff924c9640 1 -- 172.21.15.84:0/1408589727 shutdown_connections 2023-10-16T11:05:53.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.636+0000 7eff924c9640 1 -- 172.21.15.84:0/1408589727 wait complete. 2023-10-16T11:05:53.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.638+0000 7eff924c9640 1 Processor -- start 2023-10-16T11:05:53.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.638+0000 7eff924c9640 1 -- start start 2023-10-16T11:05:53.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.639+0000 7eff924c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff780047c0 0x7eff8c1f1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:53.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.639+0000 7eff924c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff8c0ca500 0x7eff8c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:53.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.639+0000 7eff924c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff8c15ba40 0x7eff8c15bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:53.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.639+0000 7eff924c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c1f19b0 con 0x7eff8c15ba40 2023-10-16T11:05:53.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.639+0000 7eff924c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c15c930 con 0x7eff780047c0 2023-10-16T11:05:53.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.639+0000 7eff924c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c0ca980 con 0x7eff8c0ca500 2023-10-16T11:05:53.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.640+0000 7eff90a66640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff8c15ba40 0x7eff8c15bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:53.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.640+0000 7eff90a66640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff8c15ba40 0x7eff8c15bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54932/0 (socket says 172.21.15.84:54932) 2023-10-16T11:05:53.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.640+0000 7eff90a66640 1 -- 172.21.15.84:0/426899036 learned_addr learned my addr 172.21.15.84:0/426899036 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:53.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.640+0000 7eff83fff640 1 --2- 172.21.15.84:0/426899036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff8c0ca500 0x7eff8c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:53.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.640+0000 7eff8bfff640 1 --2- 172.21.15.84:0/426899036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff780047c0 0x7eff8c1f1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:53.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.640+0000 7eff90a66640 1 -- 172.21.15.84:0/426899036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff8c0ca500 msgr2=0x7eff8c0b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:53.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.640+0000 7eff90a66640 1 --2- 172.21.15.84:0/426899036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff8c0ca500 0x7eff8c0b6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:53.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.640+0000 7eff90a66640 1 -- 172.21.15.84:0/426899036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff780047c0 msgr2=0x7eff8c1f1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:53.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.640+0000 7eff90a66640 1 --2- 172.21.15.84:0/426899036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff780047c0 0x7eff8c1f1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:53.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.640+0000 7eff90a66640 1 -- 172.21.15.84:0/426899036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c15de30 con 0x7eff8c15ba40 2023-10-16T11:05:53.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.640+0000 7eff90a66640 1 --2- 172.21.15.84:0/426899036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff8c15ba40 0x7eff8c15bec0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0744f0 tx=0x7eff840115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:53.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.641+0000 7eff897fa640 1 -- 172.21.15.84:0/426899036 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8400e070 con 0x7eff8c15ba40 2023-10-16T11:05:53.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.641+0000 7eff897fa640 1 -- 172.21.15.84:0/426899036 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff84011ed0 con 0x7eff8c15ba40 2023-10-16T11:05:53.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.641+0000 7eff897fa640 1 -- 172.21.15.84:0/426899036 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff84003990 con 0x7eff8c15ba40 2023-10-16T11:05:53.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.641+0000 7eff924c9640 1 -- 172.21.15.84:0/426899036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8c15d500 con 0x7eff8c15ba40 2023-10-16T11:05:53.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.641+0000 7eff924c9640 1 -- 172.21.15.84:0/426899036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8c15dac0 con 0x7eff8c15ba40 2023-10-16T11:05:53.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.642+0000 7eff924c9640 1 -- 172.21.15.84:0/426899036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8c15dfc0 con 0x7eff8c15ba40 2023-10-16T11:05:53.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.643+0000 7eff897fa640 1 -- 172.21.15.84:0/426899036 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 94) v1 ==== 89348+0+0 (secure 0 0 0) 0x7eff84003af0 con 0x7eff8c15ba40 2023-10-16T11:05:53.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.644+0000 7eff897fa640 1 --2- 172.21.15.84:0/426899036 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7eff6c06b010 0x7eff6c06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:53.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.644+0000 7eff897fa640 1 -- 172.21.15.84:0/426899036 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5737+0+0 (secure 0 0 0) 0x7eff8408a1d0 con 0x7eff8c15ba40 2023-10-16T11:05:53.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.644+0000 7eff8bfff640 1 -- 172.21.15.84:0/426899036 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7eff6c06b010 msgr2=0x7eff6c06d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2442400184 2023-10-16T11:05:53.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.644+0000 7eff8bfff640 1 --2- 172.21.15.84:0/426899036 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7eff6c06b010 0x7eff6c06d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:05:53.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.653+0000 7eff897fa640 1 -- 172.21.15.84:0/426899036 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7eff84055040 con 0x7eff8c15ba40 2023-10-16T11:05:53.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.845+0000 7eff8bfff640 1 -- 172.21.15.84:0/426899036 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7eff6c06b010 msgr2=0x7eff6c06d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2442400184 2023-10-16T11:05:53.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.845+0000 7eff8bfff640 1 --2- 172.21.15.84:0/426899036 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7eff6c06b010 0x7eff6c06d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:05:53.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.847+0000 7eff924c9640 1 -- 172.21.15.84:0/426899036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7eff8c04e160 con 0x7eff8c15ba40 2023-10-16T11:05:53.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.851+0000 7eff897fa640 1 -- 172.21.15.84:0/426899036 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v59)=0 v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7eff84058660 con 0x7eff8c15ba40 2023-10-16T11:05:53.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.854+0000 7eff924c9640 1 -- 172.21.15.84:0/426899036 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7eff6c06b010 msgr2=0x7eff6c06d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:05:53.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.854+0000 7eff924c9640 1 --2- 172.21.15.84:0/426899036 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7eff6c06b010 0x7eff6c06d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:53.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.854+0000 7eff924c9640 1 -- 172.21.15.84:0/426899036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff8c15ba40 msgr2=0x7eff8c15bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:53.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.854+0000 7eff924c9640 1 --2- 172.21.15.84:0/426899036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff8c15ba40 0x7eff8c15bec0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0744f0 tx=0x7eff840115a0 comp rx=0 tx=0).stop 2023-10-16T11:05:53.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.854+0000 7eff924c9640 1 -- 172.21.15.84:0/426899036 shutdown_connections 2023-10-16T11:05:53.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.854+0000 7eff924c9640 1 --2- 172.21.15.84:0/426899036 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7eff6c06b010 0x7eff6c06d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:53.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.854+0000 7eff924c9640 1 --2- 172.21.15.84:0/426899036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff8c15ba40 0x7eff8c15bec0 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:53.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.854+0000 7eff924c9640 1 --2- 172.21.15.84:0/426899036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff8c0ca500 0x7eff8c0b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:53.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.854+0000 7eff924c9640 1 --2- 172.21.15.84:0/426899036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff780047c0 0x7eff8c1f1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:53.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.854+0000 7eff924c9640 1 -- 172.21.15.84:0/426899036 >> 172.21.15.84:0/426899036 conn(0x7eff8c0d0ab0 msgr2=0x7eff8c0c68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:53.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.854+0000 7eff924c9640 1 -- 172.21.15.84:0/426899036 shutdown_connections 2023-10-16T11:05:53.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:53.854+0000 7eff924c9640 1 -- 172.21.15.84:0/426899036 wait complete. 2023-10-16T11:05:53.875 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T11:05:53.947 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:53.945+0000 7fe43ec2e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:05:53.948 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:53.946+0000 7fac7d2b4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:05:54.004 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:54.002+0000 7fac7d2b4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:05:54.005 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:54.004+0000 7fe43ec2e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:05:54.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.080+0000 7f86470f1640 1 Processor -- start 2023-10-16T11:05:54.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.081+0000 7f86470f1640 1 -- start start 2023-10-16T11:05:54.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.082+0000 7f86470f1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f864015cd70 0x7f86400d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:54.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.082+0000 7f86470f1640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f864015d150 con 0x7f86400cded0 2023-10-16T11:05:54.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.082+0000 7f86470f1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86400d1180 con 0x7f86400b8480 2023-10-16T11:05:54.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.082+0000 7f86470f1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86400d1460 con 0x7f864015cd70 2023-10-16T11:05:54.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.082+0000 7f86468f0640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f86400cded0 0x7f86400b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34366/0 (socket says 172.21.15.84:34366) 2023-10-16T11:05:54.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.082+0000 7f86468f0640 1 -- 172.21.15.84:0/1426977488 learned_addr learned my addr 172.21.15.84:0/1426977488 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:54.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.083+0000 7f86460ef640 1 --2- 172.21.15.84:0/1426977488 >> v2:172.21.15.157:3300/0 conn(0x7f864015cd70 0x7f86400d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:54.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.083+0000 7f86450ed640 1 -- 172.21.15.84:0/1426977488 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1763085829 0 0) 0x7f864015d150 con 0x7f86400cded0 2023-10-16T11:05:54.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.083+0000 7f86450ed640 1 -- 172.21.15.84:0/1426977488 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f862c003520 con 0x7f86400cded0 2023-10-16T11:05:54.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.083+0000 7f86450ed640 1 -- 172.21.15.84:0/1426977488 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3342054900 0 0) 0x7f862c003520 con 0x7f86400cded0 2023-10-16T11:05:54.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.083+0000 7f86450ed640 1 -- 172.21.15.84:0/1426977488 >> v2:172.21.15.157:3300/0 conn(0x7f864015cd70 msgr2=0x7f86400d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:54.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.083+0000 7f86450ed640 1 --2- 172.21.15.84:0/1426977488 >> v2:172.21.15.157:3300/0 conn(0x7f864015cd70 0x7f86400d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:54.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.083+0000 7f86450ed640 1 -- 172.21.15.84:0/1426977488 >> v1:172.21.15.157:6789/0 conn(0x7f86400b8480 legacy=0x7f86400cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:54.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.083+0000 7f86450ed640 1 -- 172.21.15.84:0/1426977488 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86400d1740 con 0x7f86400cded0 2023-10-16T11:05:54.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.083+0000 7f86450ed640 1 -- 172.21.15.84:0/1426977488 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f86380019f0 con 0x7f86400cded0 2023-10-16T11:05:54.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.083+0000 7f86450ed640 1 -- 172.21.15.84:0/1426977488 >> v1:172.21.15.84:6789/0 conn(0x7f86400cded0 legacy=0x7f86400b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:54.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.083+0000 7f86450ed640 1 --2- 172.21.15.84:0/1426977488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f862c003970 0x7f862c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:54.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.083+0000 7f86450ed640 1 --2- 172.21.15.84:0/1426977488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f862c0047c0 0x7f862c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:54.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.083+0000 7f86450ed640 1 --2- 172.21.15.84:0/1426977488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c005180 0x7f862c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:54.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.083+0000 7f86450ed640 1 -- 172.21.15.84:0/1426977488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86400d1460 con 0x7f862c003970 2023-10-16T11:05:54.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.083+0000 7f86450ed640 1 -- 172.21.15.84:0/1426977488 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c00c020 con 0x7f862c005180 2023-10-16T11:05:54.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.083+0000 7f86450ed640 1 -- 172.21.15.84:0/1426977488 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862c00c1e0 con 0x7f862c0047c0 2023-10-16T11:05:54.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.084+0000 7f86460ef640 1 --2- 172.21.15.84:0/1426977488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f862c003970 0x7f862c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:54.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.084+0000 7f86458ee640 1 --2- 172.21.15.84:0/1426977488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f862c0047c0 0x7f862c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:54.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.084+0000 7f86468f0640 1 --2- 172.21.15.84:0/1426977488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c005180 0x7f862c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:54.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.084+0000 7f86460ef640 1 -- 172.21.15.84:0/1426977488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f862c0047c0 msgr2=0x7f862c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:54.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.084+0000 7f86460ef640 1 --2- 172.21.15.84:0/1426977488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f862c0047c0 0x7f862c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:54.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.084+0000 7f86460ef640 1 -- 172.21.15.84:0/1426977488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c005180 msgr2=0x7f862c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:54.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.084+0000 7f86460ef640 1 --2- 172.21.15.84:0/1426977488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c005180 0x7f862c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:54.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.084+0000 7f86460ef640 1 -- 172.21.15.84:0/1426977488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c003520 con 0x7f862c003970 2023-10-16T11:05:54.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.084+0000 7f86468f0640 1 --2- 172.21.15.84:0/1426977488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c005180 0x7f862c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:54.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.084+0000 7f86468f0640 1 -- 172.21.15.84:0/1426977488 reap_dead start 2023-10-16T11:05:54.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.086+0000 7f86460ef640 1 --2- 172.21.15.84:0/1426977488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f862c003970 0x7f862c003d90 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f863c009ac0 tx=0x7f862c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87275f63b733d370 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:54.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.086+0000 7f86450ed640 1 -- 172.21.15.84:0/1426977488 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c02c6b0 con 0x7f862c003970 2023-10-16T11:05:54.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.086+0000 7f86450ed640 1 -- 172.21.15.84:0/1426977488 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863c02c810 con 0x7f862c003970 2023-10-16T11:05:54.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.086+0000 7f86450ed640 1 -- 172.21.15.84:0/1426977488 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c02cba0 con 0x7f862c003970 2023-10-16T11:05:54.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.087+0000 7f86470f1640 1 -- 172.21.15.84:0/1426977488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f862c003970 msgr2=0x7f862c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:54.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.087+0000 7f86470f1640 1 --2- 172.21.15.84:0/1426977488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f862c003970 0x7f862c003d90 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f863c009ac0 tx=0x7f862c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:05:54.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.087+0000 7f86470f1640 1 -- 172.21.15.84:0/1426977488 shutdown_connections 2023-10-16T11:05:54.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.087+0000 7f86470f1640 1 --2- 172.21.15.84:0/1426977488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f862c003970 0x7f862c003d90 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:54.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.087+0000 7f86470f1640 1 -- 172.21.15.84:0/1426977488 >> 172.21.15.84:0/1426977488 conn(0x7f86400cf570 msgr2=0x7f86400cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:54.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.087+0000 7f86470f1640 1 -- 172.21.15.84:0/1426977488 shutdown_connections 2023-10-16T11:05:54.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.087+0000 7f86470f1640 1 -- 172.21.15.84:0/1426977488 wait complete. 2023-10-16T11:05:54.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.089+0000 7f86470f1640 1 Processor -- start 2023-10-16T11:05:54.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.089+0000 7f86470f1640 1 -- start start 2023-10-16T11:05:54.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.090+0000 7f86470f1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c003970 0x7f864015ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:54.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.090+0000 7f86470f1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86401f2b90 0x7f86400aad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:54.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.090+0000 7f86470f1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86400b7a40 0x7f8640159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:54.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.090+0000 7f86470f1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86401f1ed0 con 0x7f86401f2b90 2023-10-16T11:05:54.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.090+0000 7f86470f1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86400ab870 con 0x7f862c003970 2023-10-16T11:05:54.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.090+0000 7f86470f1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86401f2700 con 0x7f86400b7a40 2023-10-16T11:05:54.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.090+0000 7f86458ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86401f2b90 0x7f86400aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:54.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.090+0000 7f86458ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86401f2b90 0x7f86400aad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54956/0 (socket says 172.21.15.84:54956) 2023-10-16T11:05:54.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.090+0000 7f86458ee640 1 -- 172.21.15.84:0/587638755 learned_addr learned my addr 172.21.15.84:0/587638755 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:54.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.090+0000 7f86468f0640 1 --2- 172.21.15.84:0/587638755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86400b7a40 0x7f8640159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:54.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.090+0000 7f86460ef640 1 --2- 172.21.15.84:0/587638755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c003970 0x7f864015ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:54.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.090+0000 7f86458ee640 1 -- 172.21.15.84:0/587638755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86400b7a40 msgr2=0x7f8640159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:54.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.090+0000 7f86458ee640 1 --2- 172.21.15.84:0/587638755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86400b7a40 0x7f8640159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:54.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.090+0000 7f86458ee640 1 -- 172.21.15.84:0/587638755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c003970 msgr2=0x7f864015ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:54.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.090+0000 7f86458ee640 1 --2- 172.21.15.84:0/587638755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c003970 0x7f864015ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:54.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.090+0000 7f86458ee640 1 -- 172.21.15.84:0/587638755 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86400ab590 con 0x7f86401f2b90 2023-10-16T11:05:54.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.091+0000 7f86458ee640 1 --2- 172.21.15.84:0/587638755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86401f2b90 0x7f86400aad30 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f863000e550 tx=0x7f863000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:54.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.091+0000 7f8636ffd640 1 -- 172.21.15.84:0/587638755 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86300052e0 con 0x7f86401f2b90 2023-10-16T11:05:54.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.091+0000 7f8636ffd640 1 -- 172.21.15.84:0/587638755 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8630005440 con 0x7f86401f2b90 2023-10-16T11:05:54.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.091+0000 7f8636ffd640 1 -- 172.21.15.84:0/587638755 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8630003810 con 0x7f86401f2b90 2023-10-16T11:05:54.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.091+0000 7f86470f1640 1 -- 172.21.15.84:0/587638755 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86401f20b0 con 0x7f86401f2b90 2023-10-16T11:05:54.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.091+0000 7f86470f1640 1 -- 172.21.15.84:0/587638755 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86400ab270 con 0x7f86401f2b90 2023-10-16T11:05:54.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.092+0000 7f86470f1640 1 -- 172.21.15.84:0/587638755 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86401f28b0 con 0x7f86401f2b90 2023-10-16T11:05:54.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.094+0000 7f8636ffd640 1 -- 172.21.15.84:0/587638755 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 94) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f8630003970 con 0x7f86401f2b90 2023-10-16T11:05:54.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.094+0000 7f8636ffd640 1 --2- 172.21.15.84:0/587638755 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f862406b010 0x7f862406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:54.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.095+0000 7f8636ffd640 1 -- 172.21.15.84:0/587638755 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f8630089500 con 0x7f86401f2b90 2023-10-16T11:05:54.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.095+0000 7f86460ef640 1 -- 172.21.15.84:0/587638755 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f862406b010 msgr2=0x7f862406d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2442400184 2023-10-16T11:05:54.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.095+0000 7f86460ef640 1 --2- 172.21.15.84:0/587638755 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f862406b010 0x7f862406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:05:54.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.102+0000 7f8636ffd640 1 -- 172.21.15.84:0/587638755 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f86401f28b0 con 0x7f86401f2b90 2023-10-16T11:05:54.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.296+0000 7f86470f1640 1 -- 172.21.15.84:0/587638755 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f86401f2870 con 0x7f86401f2b90 2023-10-16T11:05:54.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.296+0000 7f86460ef640 1 -- 172.21.15.84:0/587638755 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f862406b010 msgr2=0x7f862406d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2442400184 2023-10-16T11:05:54.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.296+0000 7f86460ef640 1 --2- 172.21.15.84:0/587638755 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f862406b010 0x7f862406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:05:54.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.302+0000 7f8636ffd640 1 -- 172.21.15.84:0/587638755 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v60)=0 v60) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8630054470 con 0x7f86401f2b90 2023-10-16T11:05:54.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.304+0000 7f86470f1640 1 -- 172.21.15.84:0/587638755 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f862406b010 msgr2=0x7f862406d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:05:54.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.304+0000 7f86470f1640 1 --2- 172.21.15.84:0/587638755 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f862406b010 0x7f862406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:54.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.304+0000 7f86470f1640 1 -- 172.21.15.84:0/587638755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86401f2b90 msgr2=0x7f86400aad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:54.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.304+0000 7f86470f1640 1 --2- 172.21.15.84:0/587638755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86401f2b90 0x7f86400aad30 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f863000e550 tx=0x7f863000ea10 comp rx=0 tx=0).stop 2023-10-16T11:05:54.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.304+0000 7f86470f1640 1 -- 172.21.15.84:0/587638755 shutdown_connections 2023-10-16T11:05:54.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.304+0000 7f86470f1640 1 --2- 172.21.15.84:0/587638755 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f862406b010 0x7f862406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:54.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.304+0000 7f86470f1640 1 --2- 172.21.15.84:0/587638755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86400b7a40 0x7f8640159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:54.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.304+0000 7f86470f1640 1 --2- 172.21.15.84:0/587638755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86401f2b90 0x7f86400aad30 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:54.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.304+0000 7f86470f1640 1 --2- 172.21.15.84:0/587638755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f862c003970 0x7f864015ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:54.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.304+0000 7f86470f1640 1 -- 172.21.15.84:0/587638755 >> 172.21.15.84:0/587638755 conn(0x7f86400cf570 msgr2=0x7f86400b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:54.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.304+0000 7f86470f1640 1 -- 172.21.15.84:0/587638755 shutdown_connections 2023-10-16T11:05:54.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.304+0000 7f86470f1640 1 -- 172.21.15.84:0/587638755 wait complete. 2023-10-16T11:05:54.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:05:54.325 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-16T11:05:54.375 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1h 2023-10-16T11:05:54.469 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:54.467+0000 7fe43ec2e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:05:54.478 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:54.477+0000 7fac7d2b4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:05:54.520 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:54.518+0000 7fe43ec2e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:05:54.532 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:54.531+0000 7fac7d2b4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:05:54.570 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:54.569+0000 7fe43ec2e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:05:54.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.570+0000 7f6858533640 1 Processor -- start 2023-10-16T11:05:54.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.571+0000 7f6858533640 1 -- start start 2023-10-16T11:05:54.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.572+0000 7f6858533640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68500caf80 0x7f68500ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:54.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.572+0000 7f6858533640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6850065ab0 0x7f68500cb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:54.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.572+0000 7f6858533640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68500cb8a0 con 0x7f68500b4e50 2023-10-16T11:05:54.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.572+0000 7f6858533640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685015b1e0 con 0x7f6850065ab0 2023-10-16T11:05:54.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.572+0000 7f6858533640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68500c9930 con 0x7f68500caf80 2023-10-16T11:05:54.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.573+0000 7f6855ace640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6850065ab0 0x7f68500cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:54.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.573+0000 7f6855ace640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6850065ab0 0x7f68500cb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54964/0 (socket says 172.21.15.84:54964) 2023-10-16T11:05:54.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.573+0000 7f6855ace640 1 -- 172.21.15.84:0/2482825191 learned_addr learned my addr 172.21.15.84:0/2482825191 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:54.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.573+0000 7f68562cf640 1 --2- 172.21.15.84:0/2482825191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68500caf80 0x7f68500ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:54.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.573+0000 7f6855ace640 1 -- 172.21.15.84:0/2482825191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68500caf80 msgr2=0x7f68500ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:54.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.573+0000 7f6855ace640 1 --2- 172.21.15.84:0/2482825191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68500caf80 0x7f68500ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:54.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.573+0000 7f6855ace640 1 -- 172.21.15.84:0/2482825191 >> v1:172.21.15.157:6789/0 conn(0x7f68500b4e50 legacy=0x7f6850069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:54.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.573+0000 7f6855ace640 1 -- 172.21.15.84:0/2482825191 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68500c9c10 con 0x7f6850065ab0 2023-10-16T11:05:54.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.573+0000 7f68562cf640 1 --2- 172.21.15.84:0/2482825191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68500caf80 0x7f68500ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:54.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.574+0000 7f6855ace640 1 --2- 172.21.15.84:0/2482825191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6850065ab0 0x7f68500cb360 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f68400098a0 tx=0x7f684002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22f5d5ade79a355 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:54.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.575+0000 7f68552cd640 1 -- 172.21.15.84:0/2482825191 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400317c0 con 0x7f6850065ab0 2023-10-16T11:05:54.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.575+0000 7f68552cd640 1 -- 172.21.15.84:0/2482825191 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6840031920 con 0x7f6850065ab0 2023-10-16T11:05:54.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.575+0000 7f68552cd640 1 -- 172.21.15.84:0/2482825191 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840031cb0 con 0x7f6850065ab0 2023-10-16T11:05:54.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.575+0000 7f6858533640 1 -- 172.21.15.84:0/2482825191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6850065ab0 msgr2=0x7f68500cb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:54.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.575+0000 7f6858533640 1 --2- 172.21.15.84:0/2482825191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6850065ab0 0x7f68500cb360 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f68400098a0 tx=0x7f684002b860 comp rx=0 tx=0).stop 2023-10-16T11:05:54.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.575+0000 7f6858533640 1 -- 172.21.15.84:0/2482825191 shutdown_connections 2023-10-16T11:05:54.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.575+0000 7f6858533640 1 --2- 172.21.15.84:0/2482825191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6850065ab0 0x7f68500cb360 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:54.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.575+0000 7f6858533640 1 --2- 172.21.15.84:0/2482825191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68500caf80 0x7f68500ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:54.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.575+0000 7f6858533640 1 -- 172.21.15.84:0/2482825191 >> 172.21.15.84:0/2482825191 conn(0x7f685006bc10 msgr2=0x7f685006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:54.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.576+0000 7f6858533640 1 -- 172.21.15.84:0/2482825191 shutdown_connections 2023-10-16T11:05:54.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.576+0000 7f6858533640 1 -- 172.21.15.84:0/2482825191 wait complete. 2023-10-16T11:05:54.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.577+0000 7f6858533640 1 Processor -- start 2023-10-16T11:05:54.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.577+0000 7f6858533640 1 -- start start 2023-10-16T11:05:54.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.578+0000 7f6858533640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6850065ab0 0x7f68500c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:54.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.578+0000 7f6858533640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68500b4e50 0x7f68500c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:54.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.578+0000 7f6858533640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68500caf80 0x7f6850153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:54.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.578+0000 7f6858533640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500c3010 con 0x7f68500b4e50 2023-10-16T11:05:54.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.578+0000 7f6858533640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500c2cc0 con 0x7f6850065ab0 2023-10-16T11:05:54.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.578+0000 7f6858533640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68500c5370 con 0x7f68500caf80 2023-10-16T11:05:54.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.578+0000 7f6855ace640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68500b4e50 0x7f68500c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:54.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.579+0000 7f6855ace640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68500b4e50 0x7f68500c55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54968/0 (socket says 172.21.15.84:54968) 2023-10-16T11:05:54.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.579+0000 7f6855ace640 1 -- 172.21.15.84:0/2432420252 learned_addr learned my addr 172.21.15.84:0/2432420252 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:54.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.579+0000 7f6856ad0640 1 --2- 172.21.15.84:0/2432420252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68500caf80 0x7f6850153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:54.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.579+0000 7f6855ace640 1 -- 172.21.15.84:0/2432420252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68500caf80 msgr2=0x7f6850153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:54.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.579+0000 7f68562cf640 1 --2- 172.21.15.84:0/2432420252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6850065ab0 0x7f68500c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:54.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.579+0000 7f6855ace640 1 --2- 172.21.15.84:0/2432420252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68500caf80 0x7f6850153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:54.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.579+0000 7f6855ace640 1 -- 172.21.15.84:0/2432420252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6850065ab0 msgr2=0x7f68500c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:54.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.579+0000 7f6855ace640 1 --2- 172.21.15.84:0/2432420252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6850065ab0 0x7f68500c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:54.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.579+0000 7f6855ace640 1 -- 172.21.15.84:0/2432420252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68400094d0 con 0x7f68500b4e50 2023-10-16T11:05:54.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.579+0000 7f6856ad0640 1 --2- 172.21.15.84:0/2432420252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68500caf80 0x7f6850153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:54.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.579+0000 7f6855ace640 1 --2- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68500b4e50 0x7f68500c55c0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f6840000c00 tx=0x7f68400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:54.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.580+0000 7f6846ffd640 1 -- 172.21.15.84:0/2432420252 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400329a0 con 0x7f68500b4e50 2023-10-16T11:05:54.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.580+0000 7f6846ffd640 1 -- 172.21.15.84:0/2432420252 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6840032b20 con 0x7f68500b4e50 2023-10-16T11:05:54.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.580+0000 7f6858533640 1 -- 172.21.15.84:0/2432420252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68500c8850 con 0x7f68500b4e50 2023-10-16T11:05:54.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.580+0000 7f6846ffd640 1 -- 172.21.15.84:0/2432420252 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400453f0 con 0x7f68500b4e50 2023-10-16T11:05:54.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.580+0000 7f6858533640 1 -- 172.21.15.84:0/2432420252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68500c8e10 con 0x7f68500b4e50 2023-10-16T11:05:54.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.581+0000 7f6858533640 1 -- 172.21.15.84:0/2432420252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68500c6040 con 0x7f68500b4e50 2023-10-16T11:05:54.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.582+0000 7f6846ffd640 1 -- 172.21.15.84:0/2432420252 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 94) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f684004e450 con 0x7f68500b4e50 2023-10-16T11:05:54.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.583+0000 7f6846ffd640 1 --2- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f683406afe0 0x7f683406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:54.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.583+0000 7f6846ffd640 1 -- 172.21.15.84:0/2432420252 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f68400abd00 con 0x7f68500b4e50 2023-10-16T11:05:54.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.583+0000 7f68562cf640 1 -- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f683406afe0 msgr2=0x7f683406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2442400184 2023-10-16T11:05:54.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.583+0000 7f68562cf640 1 --2- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f683406afe0 0x7f683406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:05:54.591 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:54.584+0000 7fac7d2b4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:05:54.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.587+0000 7f6846ffd640 1 -- 172.21.15.84:0/2432420252 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6840076d90 con 0x7f68500b4e50 2023-10-16T11:05:54.632 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:54.630+0000 7fe43ec2e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:05:54.649 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:54.648+0000 7fac7d2b4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:05:54.681 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:54.679+0000 7fe43ec2e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:05:54.700 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:54.699+0000 7fac7d2b4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:05:54.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.784+0000 7f68562cf640 1 -- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f683406afe0 msgr2=0x7f683406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2442400184 2023-10-16T11:05:54.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.784+0000 7f68562cf640 1 --2- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f683406afe0 0x7f683406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:05:54.790 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:54.788+0000 7fe43ec2e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:05:54.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:54.803+0000 7f6858533640 1 -- 172.21.15.84:0/2432420252 --> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f68500ca1d0 con 0x7f683406afe0 2023-10-16T11:05:54.815 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:54.814+0000 7fac7d2b4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:05:54.852 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:54.851+0000 7fe43ec2e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:05:54.878 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:54.876+0000 7fac7d2b4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:05:54.902 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:54.901+0000 7fe43ec2e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:05:54.927 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:54.926+0000 7fac7d2b4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:05:54.958 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:54.957+0000 7fe43ec2e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:05:54.984 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:54.982+0000 7fac7d2b4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:05:55.008 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:55.006+0000 7fe43ec2e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:05:55.033 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:55.031+0000 7fac7d2b4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:05:55.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:55.185+0000 7f68562cf640 1 -- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f683406afe0 msgr2=0x7f683406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2442400184 2023-10-16T11:05:55.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:55.185+0000 7f68562cf640 1 --2- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f683406afe0 0x7f683406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T11:05:55.387 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:55.385+0000 7fe43ec2e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:05:55.410 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:55.409+0000 7fac7d2b4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:05:55.545 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:55.543+0000 7fe43ec2e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:05:55.568 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:55.566+0000 7fac7d2b4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:05:55.651 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:55.650+0000 7fe43ec2e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:05:55.676 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:55.674+0000 7fac7d2b4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:05:55.701 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:55.699+0000 7fe43ec2e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:05:55.726 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:55.724+0000 7fac7d2b4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:05:55.807 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:55.806+0000 7fe43ec2e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:05:55.832 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:55.830+0000 7fac7d2b4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:05:55.915 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:55.914+0000 7fe43ec2e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:05:55.937 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:55.935+0000 7fac7d2b4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:05:55.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:55.986+0000 7f68562cf640 1 -- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f683406afe0 msgr2=0x7f683406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2442400184 2023-10-16T11:05:55.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:55.986+0000 7f68562cf640 1 --2- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f683406afe0 0x7f683406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T11:05:56.071 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:56.069+0000 7fe43ec2e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:05:56.095 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:56.093+0000 7fac7d2b4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:05:56.120 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:56.118+0000 7fe43ec2e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:05:56.143 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:56.142+0000 7fac7d2b4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:05:56.238 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:56.235+0000 7fe43ec2e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:05:56.261 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:56.259+0000 7fac7d2b4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:05:57.053 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:57.051+0000 7fe43ec2e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:05:57.103 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:57.101+0000 7fac7d2b4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:05:57.168 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:05:57.177 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:05:57.353 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:57.351+0000 7fe43ec2e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:05:57.410 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:57.409+0000 7fe43ec2e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:05:57.420 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:57.419+0000 7fac7d2b4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:05:57.477 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:57.476+0000 7fac7d2b4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:05:57.525 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:05:57.522+0000 7fe43ec2e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:05:57.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:57.589+0000 7f68562cf640 1 -- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f683406afe0 msgr2=0x7f683406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2442400184 2023-10-16T11:05:57.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:57.589+0000 7f68562cf640 1 --2- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f683406afe0 0x7f683406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T11:05:57.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:57.590+0000 7f6846ffd640 1 -- 172.21.15.84:0/2432420252 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 95) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f684004ed40 con 0x7f68500b4e50 2023-10-16T11:05:57.594 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:57.593+0000 7fac7d2b4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:05:57.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:57.619+0000 7f6846ffd640 1 -- 172.21.15.84:0/2432420252 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 96) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f6840077160 con 0x7f68500b4e50 2023-10-16T11:05:57.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:57.619+0000 7f6846ffd640 1 -- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f683406afe0 msgr2=0x7f683406d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:05:57.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:57.619+0000 7f6846ffd640 1 --2- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:6832/2442400184,v1:172.21.15.84:6833/2442400184] conn(0x7f683406afe0 0x7f683406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:57.922 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:57.920+0000 7fabfa4f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:57.922 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:57.920+0000 7fabfa4f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:57.922 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:57.920+0000 7fabfa4f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:57.922 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:57.920+0000 7fabfa4f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:57.923 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:57.920+0000 7fabfa4f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:57.923 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:57.921+0000 7fabfd53e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:57.923 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:57.921+0000 7fabfd53e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:57.924 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:57.921+0000 7fabfd53e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:57.924 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:57.921+0000 7fabfd53e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:57.924 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:57.921+0000 7fabfd53e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:05:58.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:58.626+0000 7f6846ffd640 1 -- 172.21.15.84:0/2432420252 <== mon.0 v2:172.21.15.84:3300/0 9 ==== mgrmap(e 97) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f6840077a30 con 0x7f68500b4e50 2023-10-16T11:05:58.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:58.628+0000 7f6846ffd640 1 --2- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f683401d1c0 0x7f6834074f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:58.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:58.628+0000 7f6846ffd640 1 -- 172.21.15.84:0/2432420252 --> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f683401d5a0 con 0x7f683401d1c0 2023-10-16T11:05:58.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:58.636+0000 7f68562cf640 1 --2- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f683401d1c0 0x7f6834074f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:58.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:58.642+0000 7f68562cf640 1 --2- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f683401d1c0 0x7f6834074f00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f684c006fd0 tx=0x7f684c008040 comp rx=0 tx=0).ready entity=mgr.9882 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:58.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:58.840+0000 7f6846ffd640 1 -- 172.21.15.84:0/2432420252 <== mgr.9882 v2:172.21.15.84:6832/1697458478 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f683401d5a0 con 0x7f683401d1c0 2023-10-16T11:05:58.842 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path snap_test_dir1 2023-10-16T11:05:58.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:58.843+0000 7f6858533640 1 -- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f683401d1c0 msgr2=0x7f6834074f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:58.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:58.843+0000 7f6858533640 1 --2- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f683401d1c0 0x7f6834074f00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f684c006fd0 tx=0x7f684c008040 comp rx=0 tx=0).stop 2023-10-16T11:05:58.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:58.843+0000 7f6858533640 1 -- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68500b4e50 msgr2=0x7f68500c55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:58.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:58.843+0000 7f6858533640 1 --2- 172.21.15.84:0/2432420252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68500b4e50 0x7f68500c55c0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f6840000c00 tx=0x7f68400046c0 comp rx=0 tx=0).stop 2023-10-16T11:05:58.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:58.843+0000 7f6856ad0640 1 -- 172.21.15.84:0/2432420252 reap_dead start 2023-10-16T11:05:58.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:58.843+0000 7f6858533640 1 -- 172.21.15.84:0/2432420252 shutdown_connections 2023-10-16T11:05:58.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:58.843+0000 7f6858533640 1 -- 172.21.15.84:0/2432420252 >> 172.21.15.84:0/2432420252 conn(0x7f685006bc10 msgr2=0x7f68500ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:58.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:58.843+0000 7f6858533640 1 -- 172.21.15.84:0/2432420252 shutdown_connections 2023-10-16T11:05:58.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:58.843+0000 7f6858533640 1 -- 172.21.15.84:0/2432420252 wait complete. 2023-10-16T11:05:58.866 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-16T11:05:58.867 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-16T11:05:59.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.080+0000 7f47163a9640 1 Processor -- start 2023-10-16T11:05:59.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.080+0000 7f47163a9640 1 -- start start 2023-10-16T11:05:59.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.081+0000 7f47163a9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f471015ad70 0x7f4710069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:59.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.081+0000 7f47163a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100c9ca0 0x7f471006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:59.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.081+0000 7f47163a9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47100cb030 con 0x7f47100ca3b0 2023-10-16T11:05:59.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.082+0000 7f47163a9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47100c91c0 con 0x7f471015ad70 2023-10-16T11:05:59.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.082+0000 7f47163a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47100c94a0 con 0x7f47100c9ca0 2023-10-16T11:05:59.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.082+0000 7f470ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f471015ad70 0x7f4710069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:59.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.082+0000 7f4707fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100c9ca0 0x7f471006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:59.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.082+0000 7f470ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f471015ad70 0x7f4710069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:52510/0 (socket says 172.21.15.84:52510) 2023-10-16T11:05:59.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.082+0000 7f4707fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100c9ca0 0x7f471006a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:52982/0 (socket says 172.21.15.84:52982) 2023-10-16T11:05:59.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.082+0000 7f4707fff640 1 -- 172.21.15.84:0/3059507457 learned_addr learned my addr 172.21.15.84:0/3059507457 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:59.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.082+0000 7f4714946640 1 --1- 172.21.15.84:0/3059507457 >> v1:172.21.15.157:6789/0 conn(0x7f47100ca3b0 0x7f47100ca840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:49728/0 (socket says 172.21.15.84:49728) 2023-10-16T11:05:59.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.083+0000 7f4707fff640 1 -- 172.21.15.84:0/3059507457 >> v2:172.21.15.157:3300/0 conn(0x7f471015ad70 msgr2=0x7f4710069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:59.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.083+0000 7f4707fff640 1 --2- 172.21.15.84:0/3059507457 >> v2:172.21.15.157:3300/0 conn(0x7f471015ad70 0x7f4710069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:59.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.083+0000 7f4707fff640 1 -- 172.21.15.84:0/3059507457 >> v1:172.21.15.157:6789/0 conn(0x7f47100ca3b0 legacy=0x7f47100ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:59.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.083+0000 7f4707fff640 1 -- 172.21.15.84:0/3059507457 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47100c9780 con 0x7f47100c9ca0 2023-10-16T11:05:59.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.083+0000 7f470ffff640 1 --2- 172.21.15.84:0/3059507457 >> v2:172.21.15.157:3300/0 conn(0x7f471015ad70 0x7f4710069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:05:59.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.085+0000 7f4707fff640 1 --2- 172.21.15.84:0/3059507457 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100c9ca0 0x7f471006a2e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009870 tx=0x7f46f802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9993f08b98dd53e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:59.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.086+0000 7f470f7fe640 1 -- 172.21.15.84:0/3059507457 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8030840 con 0x7f47100c9ca0 2023-10-16T11:05:59.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.086+0000 7f470f7fe640 1 -- 172.21.15.84:0/3059507457 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f80309a0 con 0x7f47100c9ca0 2023-10-16T11:05:59.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.086+0000 7f470f7fe640 1 -- 172.21.15.84:0/3059507457 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8030d30 con 0x7f47100c9ca0 2023-10-16T11:05:59.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.086+0000 7f47163a9640 1 -- 172.21.15.84:0/3059507457 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100c9ca0 msgr2=0x7f471006a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:59.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.086+0000 7f47163a9640 1 --2- 172.21.15.84:0/3059507457 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100c9ca0 0x7f471006a2e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009870 tx=0x7f46f802b980 comp rx=0 tx=0).stop 2023-10-16T11:05:59.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.086+0000 7f47163a9640 1 -- 172.21.15.84:0/3059507457 shutdown_connections 2023-10-16T11:05:59.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.086+0000 7f47163a9640 1 --2- 172.21.15.84:0/3059507457 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100c9ca0 0x7f471006a2e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:59.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.086+0000 7f47163a9640 1 --2- 172.21.15.84:0/3059507457 >> v2:172.21.15.157:3300/0 conn(0x7f471015ad70 0x7f4710069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:59.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.086+0000 7f47163a9640 1 -- 172.21.15.84:0/3059507457 >> 172.21.15.84:0/3059507457 conn(0x7f47100ce9b0 msgr2=0x7f47100cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:59.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.086+0000 7f47163a9640 1 -- 172.21.15.84:0/3059507457 shutdown_connections 2023-10-16T11:05:59.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.087+0000 7f47163a9640 1 -- 172.21.15.84:0/3059507457 wait complete. 2023-10-16T11:05:59.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.088+0000 7f47163a9640 1 Processor -- start 2023-10-16T11:05:59.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.088+0000 7f47163a9640 1 -- start start 2023-10-16T11:05:59.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.089+0000 7f47163a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100c9ca0 0x7f471014bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:59.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.089+0000 7f47163a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47100ca3b0 0x7f47100b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:59.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.089+0000 7f47163a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f471015ad70 0x7f47100b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:59.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.089+0000 7f47163a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471014c440 con 0x7f471015ad70 2023-10-16T11:05:59.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.089+0000 7f47163a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47100c67c0 con 0x7f47100ca3b0 2023-10-16T11:05:59.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.089+0000 7f47163a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47100c5bb0 con 0x7f47100c9ca0 2023-10-16T11:05:59.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.090+0000 7f4714946640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f471015ad70 0x7f47100b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:59.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.090+0000 7f4714946640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f471015ad70 0x7f47100b2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46820/0 (socket says 172.21.15.84:46820) 2023-10-16T11:05:59.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.090+0000 7f4714946640 1 -- 172.21.15.84:0/2382674779 learned_addr learned my addr 172.21.15.84:0/2382674779 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:59.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.090+0000 7f470ffff640 1 --2- 172.21.15.84:0/2382674779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100c9ca0 0x7f471014bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:59.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.090+0000 7f4707fff640 1 --2- 172.21.15.84:0/2382674779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47100ca3b0 0x7f47100b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:59.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.090+0000 7f4714946640 1 -- 172.21.15.84:0/2382674779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100c9ca0 msgr2=0x7f471014bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:59.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.090+0000 7f4714946640 1 --2- 172.21.15.84:0/2382674779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100c9ca0 0x7f471014bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:59.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.090+0000 7f4714946640 1 -- 172.21.15.84:0/2382674779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47100ca3b0 msgr2=0x7f47100b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:59.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.090+0000 7f4714946640 1 --2- 172.21.15.84:0/2382674779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47100ca3b0 0x7f47100b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:59.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.090+0000 7f4714946640 1 -- 172.21.15.84:0/2382674779 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4700009400 con 0x7f471015ad70 2023-10-16T11:05:59.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.090+0000 7f4707fff640 1 --2- 172.21.15.84:0/2382674779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47100ca3b0 0x7f47100b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:05:59.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.090+0000 7f4714946640 1 --2- 172.21.15.84:0/2382674779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f471015ad70 0x7f47100b2820 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f470800c780 tx=0x7f470800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:59.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.091+0000 7f470d7fa640 1 -- 172.21.15.84:0/2382674779 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470800ce90 con 0x7f471015ad70 2023-10-16T11:05:59.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.091+0000 7f470d7fa640 1 -- 172.21.15.84:0/2382674779 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470800a640 con 0x7f471015ad70 2023-10-16T11:05:59.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.091+0000 7f470d7fa640 1 -- 172.21.15.84:0/2382674779 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470800a9f0 con 0x7f471015ad70 2023-10-16T11:05:59.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.091+0000 7f47163a9640 1 -- 172.21.15.84:0/2382674779 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f80094d0 con 0x7f471015ad70 2023-10-16T11:05:59.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.091+0000 7f47163a9640 1 -- 172.21.15.84:0/2382674779 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47100c6170 con 0x7f471015ad70 2023-10-16T11:05:59.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.092+0000 7f47163a9640 1 -- 172.21.15.84:0/2382674779 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47100c6930 con 0x7f471015ad70 2023-10-16T11:05:59.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.093+0000 7f470d7fa640 1 -- 172.21.15.84:0/2382674779 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 97) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f470800ab50 con 0x7f471015ad70 2023-10-16T11:05:59.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.094+0000 7f470d7fa640 1 --2- 172.21.15.84:0/2382674779 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f46f006ad40 0x7f46f006d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:59.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.094+0000 7f470d7fa640 1 -- 172.21.15.84:0/2382674779 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f4708088bd0 con 0x7f471015ad70 2023-10-16T11:05:59.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.094+0000 7f470ffff640 1 --2- 172.21.15.84:0/2382674779 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f46f006ad40 0x7f46f006d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:59.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.095+0000 7f470ffff640 1 --2- 172.21.15.84:0/2382674779 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f46f006ad40 0x7f46f006d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f47100b45a0 tx=0x7f4700008040 comp rx=0 tx=0).ready entity=mgr.9882 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:59.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.099+0000 7f470d7fa640 1 -- 172.21.15.84:0/2382674779 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4708053e20 con 0x7f471015ad70 2023-10-16T11:05:59.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.311+0000 7f47163a9640 1 -- 172.21.15.84:0/2382674779 --> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f47100ad4d0 con 0x7f46f006ad40 2023-10-16T11:05:59.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.472+0000 7f470d7fa640 1 -- 172.21.15.84:0/2382674779 <== mgr.9882 v2:172.21.15.84:6832/1697458478 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f4700010930 con 0x7f46f006ad40 2023-10-16T11:05:59.474 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path snap_test_dir1 2023-10-16T11:05:59.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.475+0000 7f47163a9640 1 -- 172.21.15.84:0/2382674779 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f46f006ad40 msgr2=0x7f46f006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:59.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.475+0000 7f47163a9640 1 --2- 172.21.15.84:0/2382674779 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f46f006ad40 0x7f46f006d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f47100b45a0 tx=0x7f4700008040 comp rx=0 tx=0).stop 2023-10-16T11:05:59.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.476+0000 7f47163a9640 1 -- 172.21.15.84:0/2382674779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f471015ad70 msgr2=0x7f47100b2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:59.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.476+0000 7f47163a9640 1 --2- 172.21.15.84:0/2382674779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f471015ad70 0x7f47100b2820 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f470800c780 tx=0x7f470800cc40 comp rx=0 tx=0).stop 2023-10-16T11:05:59.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.476+0000 7f47163a9640 1 -- 172.21.15.84:0/2382674779 shutdown_connections 2023-10-16T11:05:59.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.476+0000 7f47163a9640 1 --2- 172.21.15.84:0/2382674779 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f46f006ad40 0x7f46f006d200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:59.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.476+0000 7f47163a9640 1 --2- 172.21.15.84:0/2382674779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f471015ad70 0x7f47100b2820 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:59.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.476+0000 7f47163a9640 1 --2- 172.21.15.84:0/2382674779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47100ca3b0 0x7f47100b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:59.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.476+0000 7f47163a9640 1 --2- 172.21.15.84:0/2382674779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47100c9ca0 0x7f471014bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:59.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.476+0000 7f47163a9640 1 -- 172.21.15.84:0/2382674779 >> 172.21.15.84:0/2382674779 conn(0x7f47100ce9b0 msgr2=0x7f471006a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:59.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.476+0000 7f47163a9640 1 -- 172.21.15.84:0/2382674779 shutdown_connections 2023-10-16T11:05:59.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.476+0000 7f47163a9640 1 -- 172.21.15.84:0/2382674779 wait complete. 2023-10-16T11:05:59.499 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-16T11:05:59.500 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-10-16T11:05:59.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.718+0000 7f87926bb640 1 Processor -- start 2023-10-16T11:05:59.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.718+0000 7f87926bb640 1 -- start start 2023-10-16T11:05:59.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.720+0000 7f87926bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f878c15d410 0x7f878c0d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:59.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.720+0000 7f87926bb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f878c0b5d90 con 0x7f878c15d030 2023-10-16T11:05:59.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.720+0000 7f87926bb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f878c160fc0 con 0x7f878c0cd260 2023-10-16T11:05:59.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.720+0000 7f87926bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0d2b50 con 0x7f878c15d410 2023-10-16T11:05:59.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.720+0000 7f878bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f878c15d030 0x7f878c161450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52998/0 (socket says 172.21.15.84:52998) 2023-10-16T11:05:59.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.720+0000 7f878bfff640 1 -- 172.21.15.84:0/2821591098 learned_addr learned my addr 172.21.15.84:0/2821591098 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:05:59.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.720+0000 7f8783fff640 1 --2- 172.21.15.84:0/2821591098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f878c15d410 0x7f878c0d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:59.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.720+0000 7f878b7fe640 1 -- 172.21.15.84:0/2821591098 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2524453048 0 0) 0x7f878c0b5d90 con 0x7f878c15d030 2023-10-16T11:05:59.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.720+0000 7f878b7fe640 1 -- 172.21.15.84:0/2821591098 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8778003520 con 0x7f878c15d030 2023-10-16T11:05:59.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.721+0000 7f878b7fe640 1 -- 172.21.15.84:0/2821591098 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4242045356 0 0) 0x7f8778003520 con 0x7f878c15d030 2023-10-16T11:05:59.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.721+0000 7f878b7fe640 1 -- 172.21.15.84:0/2821591098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f878c15d410 msgr2=0x7f878c0d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:59.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.721+0000 7f878b7fe640 1 --2- 172.21.15.84:0/2821591098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f878c15d410 0x7f878c0d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:59.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.721+0000 7f878b7fe640 1 -- 172.21.15.84:0/2821591098 >> v1:172.21.15.157:6789/0 conn(0x7f878c0cd260 legacy=0x7f878c0b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:59.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.721+0000 7f878b7fe640 1 -- 172.21.15.84:0/2821591098 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c0d25d0 con 0x7f878c15d030 2023-10-16T11:05:59.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.721+0000 7f878b7fe640 1 -- 172.21.15.84:0/2821591098 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f877c001670 con 0x7f878c15d030 2023-10-16T11:05:59.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.721+0000 7f878b7fe640 1 -- 172.21.15.84:0/2821591098 >> v1:172.21.15.84:6789/0 conn(0x7f878c15d030 legacy=0x7f878c161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:59.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.721+0000 7f878b7fe640 1 --2- 172.21.15.84:0/2821591098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87780039a0 0x7f8778003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:59.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.721+0000 7f878b7fe640 1 --2- 172.21.15.84:0/2821591098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f878c15d410 0x7f87780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:59.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.721+0000 7f878b7fe640 1 --2- 172.21.15.84:0/2821591098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8778004d00 0x7f8778005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:59.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.721+0000 7f878b7fe640 1 -- 172.21.15.84:0/2821591098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877800baf0 con 0x7f87780039a0 2023-10-16T11:05:59.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.721+0000 7f878b7fe640 1 -- 172.21.15.84:0/2821591098 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877800bc60 con 0x7f8778004d00 2023-10-16T11:05:59.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.721+0000 7f878b7fe640 1 -- 172.21.15.84:0/2821591098 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877800be20 con 0x7f878c15d410 2023-10-16T11:05:59.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.721+0000 7f878bfff640 1 --2- 172.21.15.84:0/2821591098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87780039a0 0x7f8778003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:59.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.722+0000 7f8790c58640 1 --2- 172.21.15.84:0/2821591098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8778004d00 0x7f8778005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:59.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.722+0000 7f878bfff640 1 -- 172.21.15.84:0/2821591098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f878c15d410 msgr2=0x7f87780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:59.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.722+0000 7f878bfff640 1 --2- 172.21.15.84:0/2821591098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f878c15d410 0x7f87780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:59.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.722+0000 7f878bfff640 1 -- 172.21.15.84:0/2821591098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8778004d00 msgr2=0x7f8778005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:05:59.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.722+0000 7f878bfff640 1 --2- 172.21.15.84:0/2821591098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8778004d00 0x7f8778005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:59.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.722+0000 7f878bfff640 1 -- 172.21.15.84:0/2821591098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8778003520 con 0x7f87780039a0 2023-10-16T11:05:59.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.723+0000 7f878bfff640 1 --2- 172.21.15.84:0/2821591098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87780039a0 0x7f8778003dc0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f877c007fb0 tx=0x7f87780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e3edeeb8a47987 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:59.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.723+0000 7f878b7fe640 1 -- 172.21.15.84:0/2821591098 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c005ad0 con 0x7f87780039a0 2023-10-16T11:05:59.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.723+0000 7f878b7fe640 1 -- 172.21.15.84:0/2821591098 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877c005c30 con 0x7f87780039a0 2023-10-16T11:05:59.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.723+0000 7f878b7fe640 1 -- 172.21.15.84:0/2821591098 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c02c7c0 con 0x7f87780039a0 2023-10-16T11:05:59.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.724+0000 7f87926bb640 1 -- 172.21.15.84:0/2821591098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87780039a0 msgr2=0x7f8778003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:59.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.724+0000 7f87926bb640 1 --2- 172.21.15.84:0/2821591098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87780039a0 0x7f8778003dc0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f877c007fb0 tx=0x7f87780057f0 comp rx=0 tx=0).stop 2023-10-16T11:05:59.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.724+0000 7f8790c58640 1 -- 172.21.15.84:0/2821591098 reap_dead start 2023-10-16T11:05:59.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.724+0000 7f87926bb640 1 -- 172.21.15.84:0/2821591098 shutdown_connections 2023-10-16T11:05:59.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.724+0000 7f87926bb640 1 -- 172.21.15.84:0/2821591098 >> 172.21.15.84:0/2821591098 conn(0x7f878c0cf720 msgr2=0x7f878c0cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:05:59.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.724+0000 7f87926bb640 1 -- 172.21.15.84:0/2821591098 shutdown_connections 2023-10-16T11:05:59.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.724+0000 7f87926bb640 1 -- 172.21.15.84:0/2821591098 wait complete. 2023-10-16T11:05:59.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.726+0000 7f87926bb640 1 Processor -- start 2023-10-16T11:05:59.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.726+0000 7f87926bb640 1 -- start start 2023-10-16T11:05:59.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.727+0000 7f87926bb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87780039a0 0x7f878c1f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:59.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.727+0000 7f87926bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f878c15afc0 0x7f878c15b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:59.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.727+0000 7f87926bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f878c15a550 0x7f878c152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:59.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.727+0000 7f87926bb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c1f3cf0 con 0x7f87780039a0 2023-10-16T11:05:59.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.727+0000 7f87926bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c1594b0 con 0x7f878c15a550 2023-10-16T11:05:59.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.727+0000 7f87926bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878c159790 con 0x7f878c15afc0 2023-10-16T11:05:59.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.727+0000 7f878bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87780039a0 0x7f878c1f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:59.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.727+0000 7f878bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87780039a0 0x7f878c1f37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46852/0 (socket says 172.21.15.84:46852) 2023-10-16T11:05:59.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.728+0000 7f878bfff640 1 -- 172.21.15.84:0/3092616130 learned_addr learned my addr 172.21.15.84:0/3092616130 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:05:59.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.728+0000 7f8783fff640 1 --2- 172.21.15.84:0/3092616130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f878c15afc0 0x7f878c15b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:59.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.728+0000 7f878bfff640 1 -- 172.21.15.84:0/3092616130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f878c15afc0 msgr2=0x7f878c15b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:59.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.728+0000 7f878bfff640 1 --2- 172.21.15.84:0/3092616130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f878c15afc0 0x7f878c15b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:59.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.728+0000 7f8790c58640 1 --2- 172.21.15.84:0/3092616130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f878c15a550 0x7f878c152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:59.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.728+0000 7f878bfff640 1 -- 172.21.15.84:0/3092616130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f878c15a550 msgr2=0x7f878c152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:05:59.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.728+0000 7f878bfff640 1 --2- 172.21.15.84:0/3092616130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f878c15a550 0x7f878c152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:05:59.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.728+0000 7f878bfff640 1 -- 172.21.15.84:0/3092616130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c159a70 con 0x7f87780039a0 2023-10-16T11:05:59.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.728+0000 7f878bfff640 1 --2- 172.21.15.84:0/3092616130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87780039a0 0x7f878c1f37b0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f877c007fb0 tx=0x7f878c160f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:59.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.729+0000 7f87897fa640 1 -- 172.21.15.84:0/3092616130 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c02cbd0 con 0x7f87780039a0 2023-10-16T11:05:59.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.729+0000 7f87897fa640 1 -- 172.21.15.84:0/3092616130 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877c02cd30 con 0x7f87780039a0 2023-10-16T11:05:59.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.729+0000 7f87897fa640 1 -- 172.21.15.84:0/3092616130 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c02cea0 con 0x7f87780039a0 2023-10-16T11:05:59.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.729+0000 7f87926bb640 1 -- 172.21.15.84:0/3092616130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878c159d50 con 0x7f87780039a0 2023-10-16T11:05:59.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.729+0000 7f87926bb640 1 -- 172.21.15.84:0/3092616130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878c15bf30 con 0x7f87780039a0 2023-10-16T11:05:59.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.730+0000 7f87926bb640 1 -- 172.21.15.84:0/3092616130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878c04e160 con 0x7f87780039a0 2023-10-16T11:05:59.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.731+0000 7f87897fa640 1 -- 172.21.15.84:0/3092616130 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 97) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f877c032070 con 0x7f87780039a0 2023-10-16T11:05:59.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.732+0000 7f87897fa640 1 --2- 172.21.15.84:0/3092616130 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f876c06ad10 0x7f876c06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:05:59.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.732+0000 7f87897fa640 1 -- 172.21.15.84:0/3092616130 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f877c0aa3b0 con 0x7f87780039a0 2023-10-16T11:05:59.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.732+0000 7f8783fff640 1 --2- 172.21.15.84:0/3092616130 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f876c06ad10 0x7f876c06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:05:59.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.733+0000 7f8783fff640 1 --2- 172.21.15.84:0/3092616130 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f876c06ad10 0x7f876c06d1d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f878c0b6cd0 tx=0x7f8774008040 comp rx=0 tx=0).ready entity=mgr.9882 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:05:59.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.737+0000 7f87897fa640 1 -- 172.21.15.84:0/3092616130 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f877c0756d0 con 0x7f87780039a0 2023-10-16T11:05:59.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.967+0000 7f87926bb640 1 -- 172.21.15.84:0/3092616130 --> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f878c0d2870 con 0x7f876c06ad10 2023-10-16T11:05:59.985 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:05:59.983+0000 7fac0454c640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-16T11:05:59.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:05:59.983+0000 7f87897fa640 1 -- 172.21.15.84:0/3092616130 <== mgr.9882 v2:172.21.15.84:6832/1697458478 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f87740095c0 con 0x7f876c06ad10 2023-10-16T11:05:59.986 INFO:teuthology.orchestra.run.smithi084.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-16T11:06:00.002 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T11:06:00.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:06:00.004 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-16T11:06:00.064 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T11:06:00.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.282+0000 7f2c80db0640 1 Processor -- start 2023-10-16T11:06:00.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.282+0000 7f2c80db0640 1 -- start start 2023-10-16T11:06:00.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.284+0000 7f2c80db0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c15ad70 0x7f2c7c069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:00.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.284+0000 7f2c80db0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2c7c0c9ca0 0x7f2c7c06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:00.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.284+0000 7f2c80db0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7c0cb030 con 0x7f2c7c0ca3b0 2023-10-16T11:06:00.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.284+0000 7f2c80db0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c7c0c91c0 con 0x7f2c7c0c9ca0 2023-10-16T11:06:00.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.284+0000 7f2c80db0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c0c94a0 con 0x7f2c7c15ad70 2023-10-16T11:06:00.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.284+0000 7f2c73fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2c7c0c9ca0 0x7f2c7c06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:00.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.284+0000 7f2c7ad9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2c7c0ca3b0 0x7f2c7c0ca840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53010/0 (socket says 172.21.15.84:53010) 2023-10-16T11:06:00.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.284+0000 7f2c7ad9d640 1 -- 172.21.15.84:0/3603503430 learned_addr learned my addr 172.21.15.84:0/3603503430 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:00.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.285+0000 7f2c7a59c640 1 --2- 172.21.15.84:0/3603503430 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c15ad70 0x7f2c7c069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:00.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.285+0000 7f2c73fff640 1 -- 172.21.15.84:0/3603503430 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c15ad70 msgr2=0x7f2c7c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:00.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.285+0000 7f2c73fff640 1 --2- 172.21.15.84:0/3603503430 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c15ad70 0x7f2c7c069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:00.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.285+0000 7f2c73fff640 1 -- 172.21.15.84:0/3603503430 >> v1:172.21.15.84:6789/0 conn(0x7f2c7c0ca3b0 legacy=0x7f2c7c0ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:00.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.285+0000 7f2c73fff640 1 -- 172.21.15.84:0/3603503430 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7c0c9780 con 0x7f2c7c0c9ca0 2023-10-16T11:06:00.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.286+0000 7f2c73fff640 1 --2- 172.21.15.84:0/3603503430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c0c9ca0 0x7f2c7c06a2e0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f2c640097a0 tx=0x7f2c6402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69161e2cb1d7b5c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:00.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.286+0000 7f2c79d9b640 1 -- 172.21.15.84:0/3603503430 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6402d040 con 0x7f2c7c0c9ca0 2023-10-16T11:06:00.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.286+0000 7f2c79d9b640 1 -- 172.21.15.84:0/3603503430 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c64031810 con 0x7f2c7c0c9ca0 2023-10-16T11:06:00.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.286+0000 7f2c79d9b640 1 -- 172.21.15.84:0/3603503430 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64031ba0 con 0x7f2c7c0c9ca0 2023-10-16T11:06:00.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.287+0000 7f2c80db0640 1 -- 172.21.15.84:0/3603503430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c0c9ca0 msgr2=0x7f2c7c06a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:00.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.287+0000 7f2c80db0640 1 --2- 172.21.15.84:0/3603503430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c0c9ca0 0x7f2c7c06a2e0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f2c640097a0 tx=0x7f2c6402b8b0 comp rx=0 tx=0).stop 2023-10-16T11:06:00.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.287+0000 7f2c80db0640 1 -- 172.21.15.84:0/3603503430 shutdown_connections 2023-10-16T11:06:00.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.287+0000 7f2c80db0640 1 --2- 172.21.15.84:0/3603503430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c0c9ca0 0x7f2c7c06a2e0 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:00.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.287+0000 7f2c80db0640 1 --2- 172.21.15.84:0/3603503430 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c15ad70 0x7f2c7c069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:00.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.287+0000 7f2c80db0640 1 -- 172.21.15.84:0/3603503430 >> 172.21.15.84:0/3603503430 conn(0x7f2c7c0ce9b0 msgr2=0x7f2c7c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:00.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.287+0000 7f2c80db0640 1 -- 172.21.15.84:0/3603503430 shutdown_connections 2023-10-16T11:06:00.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.287+0000 7f2c80db0640 1 -- 172.21.15.84:0/3603503430 wait complete. 2023-10-16T11:06:00.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.289+0000 7f2c80db0640 1 Processor -- start 2023-10-16T11:06:00.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.289+0000 7f2c80db0640 1 -- start start 2023-10-16T11:06:00.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.290+0000 7f2c80db0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c7c0c9ca0 0x7f2c7c14bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:00.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.290+0000 7f2c80db0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c0ca3b0 0x7f2c7c14d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:00.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.290+0000 7f2c80db0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c15ad70 0x7f2c7c14e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:00.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.290+0000 7f2c80db0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c14c460 con 0x7f2c7c15ad70 2023-10-16T11:06:00.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.290+0000 7f2c80db0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c14db60 con 0x7f2c7c0c9ca0 2023-10-16T11:06:00.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.290+0000 7f2c80db0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c14de40 con 0x7f2c7c0ca3b0 2023-10-16T11:06:00.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.290+0000 7f2c7ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c15ad70 0x7f2c7c14e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:00.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.290+0000 7f2c7ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c15ad70 0x7f2c7c14e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46888/0 (socket says 172.21.15.84:46888) 2023-10-16T11:06:00.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.290+0000 7f2c7ad9d640 1 -- 172.21.15.84:0/1979931675 learned_addr learned my addr 172.21.15.84:0/1979931675 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:00.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.291+0000 7f2c7ad9d640 1 -- 172.21.15.84:0/1979931675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c0ca3b0 msgr2=0x7f2c7c14d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:00.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.291+0000 7f2c73fff640 1 --2- 172.21.15.84:0/1979931675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c0ca3b0 0x7f2c7c14d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:00.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.291+0000 7f2c7a59c640 1 --2- 172.21.15.84:0/1979931675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c7c0c9ca0 0x7f2c7c14bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:00.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.291+0000 7f2c7ad9d640 1 --2- 172.21.15.84:0/1979931675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c0ca3b0 0x7f2c7c14d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:00.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.291+0000 7f2c7ad9d640 1 -- 172.21.15.84:0/1979931675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c7c0c9ca0 msgr2=0x7f2c7c14bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:00.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.291+0000 7f2c7ad9d640 1 --2- 172.21.15.84:0/1979931675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c7c0c9ca0 0x7f2c7c14bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:00.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.291+0000 7f2c7ad9d640 1 -- 172.21.15.84:0/1979931675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c64009400 con 0x7f2c7c15ad70 2023-10-16T11:06:00.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.291+0000 7f2c7ad9d640 1 --2- 172.21.15.84:0/1979931675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c15ad70 0x7f2c7c14e6f0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f2c7400cbc0 tx=0x7f2c7400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:00.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.291+0000 7f2c72ffd640 1 -- 172.21.15.84:0/1979931675 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7400ad40 con 0x7f2c7c15ad70 2023-10-16T11:06:00.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.291+0000 7f2c72ffd640 1 -- 172.21.15.84:0/1979931675 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c74012400 con 0x7f2c7c15ad70 2023-10-16T11:06:00.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.291+0000 7f2c72ffd640 1 -- 172.21.15.84:0/1979931675 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7400aea0 con 0x7f2c7c15ad70 2023-10-16T11:06:00.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.291+0000 7f2c80db0640 1 -- 172.21.15.84:0/1979931675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7c14e120 con 0x7f2c7c15ad70 2023-10-16T11:06:00.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.291+0000 7f2c80db0640 1 -- 172.21.15.84:0/1979931675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7c14fbc0 con 0x7f2c7c15ad70 2023-10-16T11:06:00.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.293+0000 7f2c80db0640 1 -- 172.21.15.84:0/1979931675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c7c14fdb0 con 0x7f2c7c15ad70 2023-10-16T11:06:00.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.294+0000 7f2c72ffd640 1 -- 172.21.15.84:0/1979931675 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 97) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f2c74007070 con 0x7f2c7c15ad70 2023-10-16T11:06:00.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.294+0000 7f2c72ffd640 1 --2- 172.21.15.84:0/1979931675 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f2c5c06ace0 0x7f2c5c06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:00.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.295+0000 7f2c72ffd640 1 -- 172.21.15.84:0/1979931675 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f2c740892f0 con 0x7f2c7c15ad70 2023-10-16T11:06:00.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.295+0000 7f2c7a59c640 1 --2- 172.21.15.84:0/1979931675 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f2c5c06ace0 0x7f2c5c06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:00.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.296+0000 7f2c7a59c640 1 --2- 172.21.15.84:0/1979931675 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f2c5c06ace0 0x7f2c5c06d1a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c006fd0 tx=0x7f2c6c008040 comp rx=0 tx=0).ready entity=mgr.9882 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:00.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.299+0000 7f2c72ffd640 1 -- 172.21.15.84:0/1979931675 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2c74054610 con 0x7f2c7c15ad70 2023-10-16T11:06:00.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.382+0000 7f2c72ffd640 1 -- 172.21.15.84:0/1979931675 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 98) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f2c740549e0 con 0x7f2c7c15ad70 2023-10-16T11:06:00.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:00.546+0000 7f2c80db0640 1 -- 172.21.15.84:0/1979931675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f2c7c04e160 con 0x7f2c7c15ad70 2023-10-16T11:06:01.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.379+0000 7f2c72ffd640 1 -- 172.21.15.84:0/1979931675 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v99) v1 ==== 93+0+0 (secure 0 0 0) 0x7f2c74074910 con 0x7f2c7c15ad70 2023-10-16T11:06:01.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.383+0000 7f2c80db0640 1 -- 172.21.15.84:0/1979931675 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f2c5c06ace0 msgr2=0x7f2c5c06d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:01.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.383+0000 7f2c80db0640 1 --2- 172.21.15.84:0/1979931675 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f2c5c06ace0 0x7f2c5c06d1a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c006fd0 tx=0x7f2c6c008040 comp rx=0 tx=0).stop 2023-10-16T11:06:01.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.383+0000 7f2c80db0640 1 -- 172.21.15.84:0/1979931675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c15ad70 msgr2=0x7f2c7c14e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:01.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.383+0000 7f2c80db0640 1 --2- 172.21.15.84:0/1979931675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c15ad70 0x7f2c7c14e6f0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f2c7400cbc0 tx=0x7f2c7400a430 comp rx=0 tx=0).stop 2023-10-16T11:06:01.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.385+0000 7f2c80db0640 1 -- 172.21.15.84:0/1979931675 shutdown_connections 2023-10-16T11:06:01.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.385+0000 7f2c80db0640 1 --2- 172.21.15.84:0/1979931675 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f2c5c06ace0 0x7f2c5c06d1a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:01.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.385+0000 7f2c80db0640 1 --2- 172.21.15.84:0/1979931675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c15ad70 0x7f2c7c14e6f0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:01.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.385+0000 7f2c80db0640 1 --2- 172.21.15.84:0/1979931675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c0ca3b0 0x7f2c7c14d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:01.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.385+0000 7f2c80db0640 1 --2- 172.21.15.84:0/1979931675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c7c0c9ca0 0x7f2c7c14bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:01.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.385+0000 7f2c80db0640 1 -- 172.21.15.84:0/1979931675 >> 172.21.15.84:0/1979931675 conn(0x7f2c7c0ce9b0 msgr2=0x7f2c7c0ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:01.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.386+0000 7f2c80db0640 1 -- 172.21.15.84:0/1979931675 shutdown_connections 2023-10-16T11:06:01.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.389+0000 7f2c80db0640 1 -- 172.21.15.84:0/1979931675 wait complete. 2023-10-16T11:06:01.411 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:06:01.412 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:06:01.519 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:06:01.609 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:06:01.610 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:06:01.610 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:06:01.661 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:06:01.661 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:06:01.750 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:06:01.915 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:01.913+0000 7fbc4bc32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:06:01.935 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:01.934+0000 7fa2c2012040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:06:01.972 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:01.970+0000 7fbc4bc32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:06:01.994 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:01.993+0000 7fa2c2012040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:06:01.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.994+0000 7fb0e67f5640 1 Processor -- start 2023-10-16T11:06:01.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.994+0000 7fb0e67f5640 1 -- start start 2023-10-16T11:06:01.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.996+0000 7fb0e67f5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb0e0159b20 0x7fb0e006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:01.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.996+0000 7fb0e67f5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e006f7f0 con 0x7fb0e00b0b20 2023-10-16T11:06:01.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.996+0000 7fb0e67f5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e006c5e0 con 0x7fb0e0159f00 2023-10-16T11:06:01.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.996+0000 7fb0e67f5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e006c8c0 con 0x7fb0e0159b20 2023-10-16T11:06:01.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.996+0000 7fb0dffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb0e0159b20 0x7fb0e006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:01.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.996+0000 7fb0dffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb0e0159b20 0x7fb0e006bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46904/0 (socket says 172.21.15.84:46904) 2023-10-16T11:06:01.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.996+0000 7fb0dffff640 1 -- 172.21.15.84:0/1460064306 learned_addr learned my addr 172.21.15.84:0/1460064306 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:01.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.997+0000 7fb0dffff640 1 -- 172.21.15.84:0/1460064306 >> v1:172.21.15.157:6789/0 conn(0x7fb0e0159f00 legacy=0x7fb0e00b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:02.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.997+0000 7fb0dffff640 1 -- 172.21.15.84:0/1460064306 >> v1:172.21.15.84:6789/0 conn(0x7fb0e00b0b20 legacy=0x7fb0e006fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:02.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.997+0000 7fb0dffff640 1 -- 172.21.15.84:0/1460064306 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e006cba0 con 0x7fb0e0159b20 2023-10-16T11:06:02.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.998+0000 7fb0dffff640 1 --2- 172.21.15.84:0/1460064306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0e0159b20 0x7fb0e006bbe0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009870 tx=0x7fb0d002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8383d3d97985dbd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:02.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.998+0000 7fb0deffd640 1 -- 172.21.15.84:0/1460064306 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0030860 con 0x7fb0e0159b20 2023-10-16T11:06:02.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.998+0000 7fb0deffd640 1 -- 172.21.15.84:0/1460064306 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d00309c0 con 0x7fb0e0159b20 2023-10-16T11:06:02.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.998+0000 7fb0deffd640 1 -- 172.21.15.84:0/1460064306 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0030d50 con 0x7fb0e0159b20 2023-10-16T11:06:02.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.999+0000 7fb0e67f5640 1 -- 172.21.15.84:0/1460064306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0e0159b20 msgr2=0x7fb0e006bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:02.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.999+0000 7fb0e67f5640 1 --2- 172.21.15.84:0/1460064306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0e0159b20 0x7fb0e006bbe0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009870 tx=0x7fb0d002b9a0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.999+0000 7fb0e67f5640 1 -- 172.21.15.84:0/1460064306 shutdown_connections 2023-10-16T11:06:02.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.999+0000 7fb0e67f5640 1 --2- 172.21.15.84:0/1460064306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0e0159b20 0x7fb0e006bbe0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.999+0000 7fb0e67f5640 1 -- 172.21.15.84:0/1460064306 >> 172.21.15.84:0/1460064306 conn(0x7fb0e00cb410 msgr2=0x7fb0e00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:02.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.999+0000 7fb0e67f5640 1 -- 172.21.15.84:0/1460064306 shutdown_connections 2023-10-16T11:06:02.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:01.999+0000 7fb0e67f5640 1 -- 172.21.15.84:0/1460064306 wait complete. 2023-10-16T11:06:02.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.001+0000 7fb0e67f5640 1 Processor -- start 2023-10-16T11:06:02.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.001+0000 7fb0e67f5640 1 -- start start 2023-10-16T11:06:02.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.002+0000 7fb0e67f5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e00b0b20 0x7fb0e00c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:02.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.002+0000 7fb0e67f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0e0159b20 0x7fb0e0155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:02.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.002+0000 7fb0e67f5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0e0159f00 0x7fb0e00b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:02.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.002+0000 7fb0e67f5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e0155990 con 0x7fb0e0159b20 2023-10-16T11:06:02.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.002+0000 7fb0e67f5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e00c5be0 con 0x7fb0e00b0b20 2023-10-16T11:06:02.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.002+0000 7fb0e67f5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e00b36b0 con 0x7fb0e0159f00 2023-10-16T11:06:02.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.002+0000 7fb0df7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0e0159b20 0x7fb0e0155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:02.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.002+0000 7fb0df7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0e0159b20 0x7fb0e0155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46906/0 (socket says 172.21.15.84:46906) 2023-10-16T11:06:02.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.002+0000 7fb0df7fe640 1 -- 172.21.15.84:0/3923378869 learned_addr learned my addr 172.21.15.84:0/3923378869 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:02.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.003+0000 7fb0dffff640 1 --2- 172.21.15.84:0/3923378869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e00b0b20 0x7fb0e00c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:02.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.003+0000 7fb0e4d92640 1 --2- 172.21.15.84:0/3923378869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0e0159f00 0x7fb0e00b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:02.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.003+0000 7fb0df7fe640 1 -- 172.21.15.84:0/3923378869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0e0159f00 msgr2=0x7fb0e00b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:02.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.003+0000 7fb0df7fe640 1 --2- 172.21.15.84:0/3923378869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0e0159f00 0x7fb0e00b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.003+0000 7fb0df7fe640 1 -- 172.21.15.84:0/3923378869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e00b0b20 msgr2=0x7fb0e00c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:02.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.003+0000 7fb0df7fe640 1 --2- 172.21.15.84:0/3923378869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e00b0b20 0x7fb0e00c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.003+0000 7fb0df7fe640 1 -- 172.21.15.84:0/3923378869 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d00094d0 con 0x7fb0e0159b20 2023-10-16T11:06:02.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.003+0000 7fb0df7fe640 1 --2- 172.21.15.84:0/3923378869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0e0159b20 0x7fb0e0155450 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400c400 tx=0x7fb0d400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:02.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.003+0000 7fb0dcff9640 1 -- 172.21.15.84:0/3923378869 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4011790 con 0x7fb0e0159b20 2023-10-16T11:06:02.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.003+0000 7fb0dcff9640 1 -- 172.21.15.84:0/3923378869 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d4011930 con 0x7fb0e0159b20 2023-10-16T11:06:02.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.003+0000 7fb0dcff9640 1 -- 172.21.15.84:0/3923378869 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4011ce0 con 0x7fb0e0159b20 2023-10-16T11:06:02.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.003+0000 7fb0e67f5640 1 -- 172.21.15.84:0/3923378869 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e00c5e80 con 0x7fb0e0159b20 2023-10-16T11:06:02.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.004+0000 7fb0e67f5640 1 -- 172.21.15.84:0/3923378869 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e00b3990 con 0x7fb0e0159b20 2023-10-16T11:06:02.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.005+0000 7fb0e67f5640 1 -- 172.21.15.84:0/3923378869 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e004e160 con 0x7fb0e0159b20 2023-10-16T11:06:02.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.006+0000 7fb0dcff9640 1 -- 172.21.15.84:0/3923378869 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 99) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fb0d401c460 con 0x7fb0e0159b20 2023-10-16T11:06:02.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.007+0000 7fb0dcff9640 1 --2- 172.21.15.84:0/3923378869 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7fb0cc06af90 0x7fb0cc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:02.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.007+0000 7fb0dffff640 1 -- 172.21.15.84:0/3923378869 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7fb0cc06af90 msgr2=0x7fb0cc06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1697458478 2023-10-16T11:06:02.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.007+0000 7fb0dcff9640 1 -- 172.21.15.84:0/3923378869 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6081+0+0 (secure 0 0 0) 0x7fb0d4089830 con 0x7fb0e0159b20 2023-10-16T11:06:02.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.007+0000 7fb0dffff640 1 --2- 172.21.15.84:0/3923378869 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7fb0cc06af90 0x7fb0cc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:06:02.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.011+0000 7fb0dcff9640 1 -- 172.21.15.84:0/3923378869 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb0d4054650 con 0x7fb0e0159b20 2023-10-16T11:06:02.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.207+0000 7fb0e67f5640 1 -- 172.21.15.84:0/3923378869 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0e00c9610 con 0x7fb0e0159b20 2023-10-16T11:06:02.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.208+0000 7fb0dffff640 1 -- 172.21.15.84:0/3923378869 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7fb0cc06af90 msgr2=0x7fb0cc06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1697458478 2023-10-16T11:06:02.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.208+0000 7fb0dffff640 1 --2- 172.21.15.84:0/3923378869 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7fb0cc06af90 0x7fb0cc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:06:02.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.209+0000 7fb0dcff9640 1 -- 172.21.15.84:0/3923378869 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+17100 (secure 0 0 0) 0x7fb0d4057c70 con 0x7fb0e0159b20 2023-10-16T11:06:02.210 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:02.211 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":172,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:05:57.603936+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:05:41.217474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-16T11:05:42.387743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/3425147335":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/1161603770":"2023-10-17T11:05:57.603800+0000","172.21.15.84:6833/2442400184":"2023-10-17T11:05:57.603800+0000","172.21.15.84:6832/2442400184":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/3633178028":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/1914327915":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/2602045766":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/3065897528":"2023-10-17T11:05:57.603800+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:06:02.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.211+0000 7fb0e67f5640 1 -- 172.21.15.84:0/3923378869 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7fb0cc06af90 msgr2=0x7fb0cc06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:06:02.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.211+0000 7fb0e67f5640 1 --2- 172.21.15.84:0/3923378869 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7fb0cc06af90 0x7fb0cc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.211+0000 7fb0e67f5640 1 -- 172.21.15.84:0/3923378869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0e0159b20 msgr2=0x7fb0e0155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:02.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.211+0000 7fb0e67f5640 1 --2- 172.21.15.84:0/3923378869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0e0159b20 0x7fb0e0155450 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400c400 tx=0x7fb0d400c8c0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.211+0000 7fb0e67f5640 1 -- 172.21.15.84:0/3923378869 shutdown_connections 2023-10-16T11:06:02.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.211+0000 7fb0e67f5640 1 --2- 172.21.15.84:0/3923378869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0e0159f00 0x7fb0e00b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.211+0000 7fb0e67f5640 1 --2- 172.21.15.84:0/3923378869 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7fb0cc06af90 0x7fb0cc06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.211+0000 7fb0e67f5640 1 --2- 172.21.15.84:0/3923378869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0e0159b20 0x7fb0e0155450 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.211+0000 7fb0e67f5640 1 --2- 172.21.15.84:0/3923378869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e00b0b20 0x7fb0e00c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.211+0000 7fb0e67f5640 1 -- 172.21.15.84:0/3923378869 >> 172.21.15.84:0/3923378869 conn(0x7fb0e00cb410 msgr2=0x7fb0e015d690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:02.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.211+0000 7fb0e67f5640 1 -- 172.21.15.84:0/3923378869 shutdown_connections 2023-10-16T11:06:02.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.211+0000 7fb0e67f5640 1 -- 172.21.15.84:0/3923378869 wait complete. 2023-10-16T11:06:02.232 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:05:41.217474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:05:42.387743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:06:02.233 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:06:02.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.437+0000 7f2888d16640 1 Processor -- start 2023-10-16T11:06:02.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.437+0000 7f2888d16640 1 -- start start 2023-10-16T11:06:02.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.438+0000 7f2888d16640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28840b1ca0 0x7f28840d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:02.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.439+0000 7f2888d16640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f28840b2080 0x7f28840ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:02.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.439+0000 7f2888d16640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28840d0970 con 0x7f28840ce960 2023-10-16T11:06:02.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.439+0000 7f2888d16640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28840d0ea0 con 0x7f28840b2080 2023-10-16T11:06:02.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.439+0000 7f2888d16640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28840d0bc0 con 0x7f28840b1ca0 2023-10-16T11:06:02.442 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:02.439+0000 7fbc4bc32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:06:02.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.439+0000 7f28837fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28840b1ca0 0x7f28840d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.439+0000 7f287bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f28840b2080 0x7f28840ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.439+0000 7f287bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f28840b2080 0x7f28840ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:52596/0 (socket says 172.21.15.84:52596) 2023-10-16T11:06:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.439+0000 7f287bfff640 1 -- 172.21.15.84:0/3733315406 learned_addr learned my addr 172.21.15.84:0/3733315406 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.440+0000 7f28837fe640 1 -- 172.21.15.84:0/3733315406 >> v2:172.21.15.157:3300/0 conn(0x7f28840b2080 msgr2=0x7f28840ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.440+0000 7f28837fe640 1 --2- 172.21.15.84:0/3733315406 >> v2:172.21.15.157:3300/0 conn(0x7f28840b2080 0x7f28840ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.440+0000 7f28837fe640 1 -- 172.21.15.84:0/3733315406 >> v1:172.21.15.157:6789/0 conn(0x7f28840ce960 legacy=0x7f28840b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.440+0000 7f28837fe640 1 -- 172.21.15.84:0/3733315406 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28840cd940 con 0x7f28840b1ca0 2023-10-16T11:06:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.440+0000 7f287bfff640 1 --2- 172.21.15.84:0/3733315406 >> v2:172.21.15.157:3300/0 conn(0x7f28840b2080 0x7f28840ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:06:02.445 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:02.442+0000 7fa2c2012040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:06:02.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.442+0000 7f28837fe640 1 --2- 172.21.15.84:0/3733315406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28840b1ca0 0x7f28840d36b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f28740097a0 tx=0x7f287402b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27abee44f2a0daf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:02.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.442+0000 7f2882ffd640 1 -- 172.21.15.84:0/3733315406 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2874030800 con 0x7f28840b1ca0 2023-10-16T11:06:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.442+0000 7f2882ffd640 1 -- 172.21.15.84:0/3733315406 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2874030960 con 0x7f28840b1ca0 2023-10-16T11:06:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.442+0000 7f2882ffd640 1 -- 172.21.15.84:0/3733315406 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2874030cf0 con 0x7f28840b1ca0 2023-10-16T11:06:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.443+0000 7f2888d16640 1 -- 172.21.15.84:0/3733315406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28840b1ca0 msgr2=0x7f28840d36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.443+0000 7f2888d16640 1 --2- 172.21.15.84:0/3733315406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28840b1ca0 0x7f28840d36b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f28740097a0 tx=0x7f287402b940 comp rx=0 tx=0).stop 2023-10-16T11:06:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.443+0000 7f2888d16640 1 -- 172.21.15.84:0/3733315406 shutdown_connections 2023-10-16T11:06:02.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.443+0000 7f2888d16640 1 --2- 172.21.15.84:0/3733315406 >> v2:172.21.15.157:3300/0 conn(0x7f28840b2080 0x7f28840ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.443+0000 7f2888d16640 1 --2- 172.21.15.84:0/3733315406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28840b1ca0 0x7f28840d36b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.443+0000 7f2888d16640 1 -- 172.21.15.84:0/3733315406 >> 172.21.15.84:0/3733315406 conn(0x7f28840d2b90 msgr2=0x7f28840d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:02.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.443+0000 7f2888d16640 1 -- 172.21.15.84:0/3733315406 shutdown_connections 2023-10-16T11:06:02.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.443+0000 7f2888d16640 1 -- 172.21.15.84:0/3733315406 wait complete. 2023-10-16T11:06:02.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.445+0000 7f2888d16640 1 Processor -- start 2023-10-16T11:06:02.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.445+0000 7f2888d16640 1 -- start start 2023-10-16T11:06:02.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.446+0000 7f2888d16640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28840b1ca0 0x7f288409d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:02.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.446+0000 7f2888d16640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28840b2080 0x7f28840b7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:02.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.446+0000 7f2888d16640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28840ce960 0x7f28841f2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:02.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.446+0000 7f2888d16640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288409e000 con 0x7f28840ce960 2023-10-16T11:06:02.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.446+0000 7f2888d16640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28840c9d70 con 0x7f28840b1ca0 2023-10-16T11:06:02.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.446+0000 7f2888d16640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28840ca050 con 0x7f28840b2080 2023-10-16T11:06:02.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.446+0000 7f2883fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28840ce960 0x7f28841f2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:02.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.446+0000 7f2883fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28840ce960 0x7f28841f2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46914/0 (socket says 172.21.15.84:46914) 2023-10-16T11:06:02.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.446+0000 7f2883fff640 1 -- 172.21.15.84:0/3531771401 learned_addr learned my addr 172.21.15.84:0/3531771401 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:02.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.446+0000 7f28837fe640 1 --2- 172.21.15.84:0/3531771401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28840b1ca0 0x7f288409d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:02.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.447+0000 7f287bfff640 1 --2- 172.21.15.84:0/3531771401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28840b2080 0x7f28840b7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:02.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.447+0000 7f2883fff640 1 -- 172.21.15.84:0/3531771401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28840b2080 msgr2=0x7f28840b7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:02.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.447+0000 7f2883fff640 1 --2- 172.21.15.84:0/3531771401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28840b2080 0x7f28840b7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.447+0000 7f2883fff640 1 -- 172.21.15.84:0/3531771401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28840b1ca0 msgr2=0x7f288409d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:02.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.447+0000 7f2883fff640 1 --2- 172.21.15.84:0/3531771401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28840b1ca0 0x7f288409d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.447+0000 7f2883fff640 1 -- 172.21.15.84:0/3531771401 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286c0094d0 con 0x7f28840ce960 2023-10-16T11:06:02.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.447+0000 7f287bfff640 1 --2- 172.21.15.84:0/3531771401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28840b2080 0x7f28840b7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:02.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.447+0000 7f2883fff640 1 --2- 172.21.15.84:0/3531771401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28840ce960 0x7f28841f2850 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f287c00c6b0 tx=0x7f287c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:02.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.447+0000 7f2880ff9640 1 -- 172.21.15.84:0/3531771401 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c003b70 con 0x7f28840ce960 2023-10-16T11:06:02.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.447+0000 7f2880ff9640 1 -- 172.21.15.84:0/3531771401 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287c003cf0 con 0x7f28840ce960 2023-10-16T11:06:02.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.447+0000 7f2880ff9640 1 -- 172.21.15.84:0/3531771401 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c003ea0 con 0x7f28840ce960 2023-10-16T11:06:02.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.447+0000 7f2888d16640 1 -- 172.21.15.84:0/3531771401 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2874009400 con 0x7f28840ce960 2023-10-16T11:06:02.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.447+0000 7f2888d16640 1 -- 172.21.15.84:0/3531771401 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28840ca610 con 0x7f28840ce960 2023-10-16T11:06:02.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.449+0000 7f2888d16640 1 -- 172.21.15.84:0/3531771401 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f288404e9b0 con 0x7f28840ce960 2023-10-16T11:06:02.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.450+0000 7f2880ff9640 1 -- 172.21.15.84:0/3531771401 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 99) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f287c005310 con 0x7f28840ce960 2023-10-16T11:06:02.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.451+0000 7f2880ff9640 1 --2- 172.21.15.84:0/3531771401 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f286406af90 0x7f286406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:02.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.451+0000 7f2880ff9640 1 -- 172.21.15.84:0/3531771401 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f287c089750 con 0x7f28840ce960 2023-10-16T11:06:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.451+0000 7f28837fe640 1 -- 172.21.15.84:0/3531771401 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f286406af90 msgr2=0x7f286406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1697458478 2023-10-16T11:06:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.451+0000 7f28837fe640 1 --2- 172.21.15.84:0/3531771401 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f286406af90 0x7f286406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:06:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.456+0000 7f2880ff9640 1 -- 172.21.15.84:0/3531771401 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f287c0544e0 con 0x7f28840ce960 2023-10-16T11:06:02.497 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:02.495+0000 7fbc4bc32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:06:02.498 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:02.495+0000 7fa2c2012040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:06:02.548 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:02.546+0000 7fa2c2012040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:06:02.548 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:02.547+0000 7fbc4bc32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:06:02.608 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:02.607+0000 7fa2c2012040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:06:02.610 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:02.609+0000 7fbc4bc32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:06:02.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.646+0000 7f2888d16640 1 -- 172.21.15.84:0/3531771401 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f28840b8840 con 0x7f28840ce960 2023-10-16T11:06:02.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.647+0000 7f2880ff9640 1 -- 172.21.15.84:0/3531771401 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v172) v1 ==== 93+0+10 (secure 0 0 0) 0x7f287c057b00 con 0x7f28840ce960 2023-10-16T11:06:02.649 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:06:02.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.649+0000 7f2888d16640 1 -- 172.21.15.84:0/3531771401 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f286406af90 msgr2=0x7f286406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:06:02.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.649+0000 7f2888d16640 1 --2- 172.21.15.84:0/3531771401 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f286406af90 0x7f286406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.649+0000 7f2888d16640 1 -- 172.21.15.84:0/3531771401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28840ce960 msgr2=0x7f28841f2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:02.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.649+0000 7f2888d16640 1 --2- 172.21.15.84:0/3531771401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28840ce960 0x7f28841f2850 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f287c00c6b0 tx=0x7f287c00cb70 comp rx=0 tx=0).stop 2023-10-16T11:06:02.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.649+0000 7f2888d16640 1 -- 172.21.15.84:0/3531771401 shutdown_connections 2023-10-16T11:06:02.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.649+0000 7f2888d16640 1 --2- 172.21.15.84:0/3531771401 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f286406af90 0x7f286406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.649+0000 7f2888d16640 1 --2- 172.21.15.84:0/3531771401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28840ce960 0x7f28841f2850 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.649+0000 7f2888d16640 1 --2- 172.21.15.84:0/3531771401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28840b2080 0x7f28840b7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.649+0000 7f2888d16640 1 --2- 172.21.15.84:0/3531771401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28840b1ca0 0x7f288409d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.649+0000 7f2888d16640 1 -- 172.21.15.84:0/3531771401 >> 172.21.15.84:0/3531771401 conn(0x7f28840d2b90 msgr2=0x7f28840b1280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:02.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.650+0000 7f2888d16640 1 -- 172.21.15.84:0/3531771401 shutdown_connections 2023-10-16T11:06:02.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.650+0000 7f2888d16640 1 -- 172.21.15.84:0/3531771401 wait complete. 2023-10-16T11:06:02.656 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:02.655+0000 7fa2c2012040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:06:02.661 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:02.660+0000 7fbc4bc32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:06:02.668 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:06:02.766 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:02.765+0000 7fa2c2012040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:06:02.779 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:02.777+0000 7fbc4bc32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:06:02.828 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:02.827+0000 7fa2c2012040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:06:02.842 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:02.841+0000 7fbc4bc32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:06:02.878 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:02.877+0000 7fa2c2012040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:06:02.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.883+0000 7f4419d78640 1 Processor -- start 2023-10-16T11:06:02.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.883+0000 7f4419d78640 1 -- start start 2023-10-16T11:06:02.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.884+0000 7f4419d78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44140cfaf0 0x7f44140b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:02.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.884+0000 7f4419d78640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44140d3980 con 0x7f44140cf710 2023-10-16T11:06:02.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.884+0000 7f4419d78640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44140ce630 con 0x7f44140b3420 2023-10-16T11:06:02.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.884+0000 7f4419d78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44140d3c60 con 0x7f44140cfaf0 2023-10-16T11:06:02.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.885+0000 7f44137fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f44140cf710 0x7f44140b2d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53036/0 (socket says 172.21.15.84:53036) 2023-10-16T11:06:02.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.885+0000 7f44137fe640 1 -- 172.21.15.84:0/3706199648 learned_addr learned my addr 172.21.15.84:0/3706199648 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:02.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.885+0000 7f4413fff640 1 --2- 172.21.15.84:0/3706199648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44140cfaf0 0x7f44140b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:02.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.885+0000 7f4412ffd640 1 -- 172.21.15.84:0/3706199648 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163040268 0 0) 0x7f44140d3980 con 0x7f44140cf710 2023-10-16T11:06:02.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.885+0000 7f4412ffd640 1 -- 172.21.15.84:0/3706199648 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4400003520 con 0x7f44140cf710 2023-10-16T11:06:02.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.885+0000 7f4412ffd640 1 -- 172.21.15.84:0/3706199648 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 997396122 0 0) 0x7f4400003520 con 0x7f44140cf710 2023-10-16T11:06:02.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.885+0000 7f4412ffd640 1 -- 172.21.15.84:0/3706199648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44140cfaf0 msgr2=0x7f44140b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:02.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.885+0000 7f4412ffd640 1 --2- 172.21.15.84:0/3706199648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44140cfaf0 0x7f44140b63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.885+0000 7f4412ffd640 1 -- 172.21.15.84:0/3706199648 >> v1:172.21.15.157:6789/0 conn(0x7f44140b3420 legacy=0x7f44140b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:02.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.885+0000 7f4412ffd640 1 -- 172.21.15.84:0/3706199648 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44140ce910 con 0x7f44140cf710 2023-10-16T11:06:02.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.885+0000 7f4412ffd640 1 -- 172.21.15.84:0/3706199648 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f44040017f0 con 0x7f44140cf710 2023-10-16T11:06:02.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.886+0000 7f4412ffd640 1 -- 172.21.15.84:0/3706199648 >> v1:172.21.15.84:6789/0 conn(0x7f44140cf710 legacy=0x7f44140b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:02.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.886+0000 7f4412ffd640 1 --2- 172.21.15.84:0/3706199648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44000039a0 0x7f4400003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:02.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.886+0000 7f4412ffd640 1 --2- 172.21.15.84:0/3706199648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44140cfaf0 0x7f44000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:02.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.886+0000 7f4412ffd640 1 --2- 172.21.15.84:0/3706199648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4400004d00 0x7f4400005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:02.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.886+0000 7f4412ffd640 1 -- 172.21.15.84:0/3706199648 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44140d3c60 con 0x7f4400004d00 2023-10-16T11:06:02.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.886+0000 7f4412ffd640 1 -- 172.21.15.84:0/3706199648 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440000bb40 con 0x7f44000039a0 2023-10-16T11:06:02.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.886+0000 7f4412ffd640 1 -- 172.21.15.84:0/3706199648 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440000bd00 con 0x7f44140cfaf0 2023-10-16T11:06:02.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.886+0000 7f4413fff640 1 --2- 172.21.15.84:0/3706199648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4400004d00 0x7f4400005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:02.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.886+0000 7f44137fe640 1 --2- 172.21.15.84:0/3706199648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44000039a0 0x7f4400003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:02.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.886+0000 7f440bfff640 1 --2- 172.21.15.84:0/3706199648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44140cfaf0 0x7f44000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:02.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.886+0000 7f4413fff640 1 -- 172.21.15.84:0/3706199648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44140cfaf0 msgr2=0x7f44000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:02.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.886+0000 7f4413fff640 1 --2- 172.21.15.84:0/3706199648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44140cfaf0 0x7f44000047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.886+0000 7f4413fff640 1 -- 172.21.15.84:0/3706199648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44000039a0 msgr2=0x7f4400003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:02.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.886+0000 7f4413fff640 1 --2- 172.21.15.84:0/3706199648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44000039a0 0x7f4400003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.886+0000 7f4413fff640 1 -- 172.21.15.84:0/3706199648 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4400003520 con 0x7f4400004d00 2023-10-16T11:06:02.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.887+0000 7f4413fff640 1 --2- 172.21.15.84:0/3706199648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4400004d00 0x7f4400005180 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f440c009fc0 tx=0x7f44000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b82528e83ec1cbe server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:02.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.888+0000 7f4412ffd640 1 -- 172.21.15.84:0/3706199648 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c02cc20 con 0x7f4400004d00 2023-10-16T11:06:02.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.888+0000 7f4412ffd640 1 -- 172.21.15.84:0/3706199648 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440c00f040 con 0x7f4400004d00 2023-10-16T11:06:02.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.888+0000 7f4412ffd640 1 -- 172.21.15.84:0/3706199648 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c034440 con 0x7f4400004d00 2023-10-16T11:06:02.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.889+0000 7f4419d78640 1 -- 172.21.15.84:0/3706199648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4400004d00 msgr2=0x7f4400005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:02.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.889+0000 7f4419d78640 1 --2- 172.21.15.84:0/3706199648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4400004d00 0x7f4400005180 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f440c009fc0 tx=0x7f44000057f0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.889+0000 7f4413fff640 1 -- 172.21.15.84:0/3706199648 reap_dead start 2023-10-16T11:06:02.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.889+0000 7f4419d78640 1 -- 172.21.15.84:0/3706199648 shutdown_connections 2023-10-16T11:06:02.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.889+0000 7f4419d78640 1 -- 172.21.15.84:0/3706199648 >> 172.21.15.84:0/3706199648 conn(0x7f44140d2cc0 msgr2=0x7f44140d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:02.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.889+0000 7f4419d78640 1 -- 172.21.15.84:0/3706199648 shutdown_connections 2023-10-16T11:06:02.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.889+0000 7f4419d78640 1 -- 172.21.15.84:0/3706199648 wait complete. 2023-10-16T11:06:02.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.891+0000 7f4419d78640 1 Processor -- start 2023-10-16T11:06:02.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.891+0000 7f4419d78640 1 -- start start 2023-10-16T11:06:02.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.892+0000 7f4419d78640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4400004d00 0x7f4414151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:02.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.892+0000 7f4419d78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44140af890 0x7f441415e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:02.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.892+0000 7f4419d78640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44140b22a0 0x7f441415e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:02.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.892+0000 7f4419d78640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441415a5a0 con 0x7f4400004d00 2023-10-16T11:06:02.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.892+0000 7f4419d78640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441415a880 con 0x7f44140b22a0 2023-10-16T11:06:02.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.892+0000 7f4419d78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44140b6e70 con 0x7f44140af890 2023-10-16T11:06:02.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.892+0000 7f44137fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4400004d00 0x7f4414151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:02.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.892+0000 7f44137fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4400004d00 0x7f4414151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46936/0 (socket says 172.21.15.84:46936) 2023-10-16T11:06:02.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.892+0000 7f44137fe640 1 -- 172.21.15.84:0/1888374224 learned_addr learned my addr 172.21.15.84:0/1888374224 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:02.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.892+0000 7f4413fff640 1 --2- 172.21.15.84:0/1888374224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44140b22a0 0x7f441415e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:02.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.892+0000 7f440bfff640 1 --2- 172.21.15.84:0/1888374224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44140af890 0x7f441415e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:02.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.892+0000 7f44137fe640 1 -- 172.21.15.84:0/1888374224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44140af890 msgr2=0x7f441415e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:02.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.892+0000 7f44137fe640 1 --2- 172.21.15.84:0/1888374224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44140af890 0x7f441415e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.892+0000 7f44137fe640 1 -- 172.21.15.84:0/1888374224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44140b22a0 msgr2=0x7f441415e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:02.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.892+0000 7f44137fe640 1 --2- 172.21.15.84:0/1888374224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44140b22a0 0x7f441415e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:02.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.892+0000 7f44137fe640 1 -- 172.21.15.84:0/1888374224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44140b7150 con 0x7f4400004d00 2023-10-16T11:06:02.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.893+0000 7f44137fe640 1 --2- 172.21.15.84:0/1888374224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4400004d00 0x7f4414151140 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f440400f8e0 tx=0x7f440400fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:02.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.893+0000 7f4410ff9640 1 -- 172.21.15.84:0/1888374224 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44040072e0 con 0x7f4400004d00 2023-10-16T11:06:02.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.893+0000 7f4410ff9640 1 -- 172.21.15.84:0/1888374224 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4404007460 con 0x7f4400004d00 2023-10-16T11:06:02.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.893+0000 7f4410ff9640 1 -- 172.21.15.84:0/1888374224 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44040059e0 con 0x7f4400004d00 2023-10-16T11:06:02.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.893+0000 7f4419d78640 1 -- 172.21.15.84:0/1888374224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44140b7430 con 0x7f4400004d00 2023-10-16T11:06:02.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.893+0000 7f4419d78640 1 -- 172.21.15.84:0/1888374224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441415ee90 con 0x7f4400004d00 2023-10-16T11:06:02.904 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:02.894+0000 7fbc4bc32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:06:02.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.894+0000 7f4419d78640 1 -- 172.21.15.84:0/1888374224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44140c9b00 con 0x7f4400004d00 2023-10-16T11:06:02.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.896+0000 7f4410ff9640 1 -- 172.21.15.84:0/1888374224 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 99) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f4404005b40 con 0x7f4400004d00 2023-10-16T11:06:02.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.897+0000 7f4410ff9640 1 --2- 172.21.15.84:0/1888374224 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f43f406afc0 0x7f43f406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:02.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.897+0000 7f4410ff9640 1 -- 172.21.15.84:0/1888374224 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f4404089b60 con 0x7f4400004d00 2023-10-16T11:06:02.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.897+0000 7f440bfff640 1 -- 172.21.15.84:0/1888374224 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f43f406afc0 msgr2=0x7f43f406d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1697458478 2023-10-16T11:06:02.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.897+0000 7f440bfff640 1 --2- 172.21.15.84:0/1888374224 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f43f406afc0 0x7f43f406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:06:02.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:02.902+0000 7f4410ff9640 1 -- 172.21.15.84:0/1888374224 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4404058400 con 0x7f4400004d00 2023-10-16T11:06:02.936 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:02.935+0000 7fa2c2012040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:06:02.955 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:02.953+0000 7fbc4bc32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:06:02.986 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:02.985+0000 7fa2c2012040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:06:03.006 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:03.005+0000 7fbc4bc32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:06:03.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.094+0000 7f4419d78640 1 -- 172.21.15.84:0/1888374224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f441404e160 con 0x7f4400004d00 2023-10-16T11:06:03.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.094+0000 7f4410ff9640 1 -- 172.21.15.84:0/1888374224 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v172) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4404057f90 con 0x7f4400004d00 2023-10-16T11:06:03.096 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:06:03.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.097+0000 7f4419d78640 1 -- 172.21.15.84:0/1888374224 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f43f406afc0 msgr2=0x7f43f406d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:06:03.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.097+0000 7f4419d78640 1 --2- 172.21.15.84:0/1888374224 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f43f406afc0 0x7f43f406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.097+0000 7f4419d78640 1 -- 172.21.15.84:0/1888374224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4400004d00 msgr2=0x7f4414151140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:03.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.097+0000 7f4419d78640 1 --2- 172.21.15.84:0/1888374224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4400004d00 0x7f4414151140 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f440400f8e0 tx=0x7f440400fda0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.097+0000 7f4419d78640 1 -- 172.21.15.84:0/1888374224 shutdown_connections 2023-10-16T11:06:03.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.097+0000 7f4419d78640 1 --2- 172.21.15.84:0/1888374224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44140b22a0 0x7f441415e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.097+0000 7f4419d78640 1 --2- 172.21.15.84:0/1888374224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44140af890 0x7f441415e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.097+0000 7f4419d78640 1 --2- 172.21.15.84:0/1888374224 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f43f406afc0 0x7f43f406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.097+0000 7f4419d78640 1 --2- 172.21.15.84:0/1888374224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4400004d00 0x7f4414151140 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.097+0000 7f4419d78640 1 -- 172.21.15.84:0/1888374224 >> 172.21.15.84:0/1888374224 conn(0x7f44140d2cc0 msgr2=0x7f441406b590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:03.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.097+0000 7f4419d78640 1 -- 172.21.15.84:0/1888374224 shutdown_connections 2023-10-16T11:06:03.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.097+0000 7f4419d78640 1 -- 172.21.15.84:0/1888374224 wait complete. 2023-10-16T11:06:03.118 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:06:03.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.321+0000 7fe0fdffb640 1 Processor -- start 2023-10-16T11:06:03.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.322+0000 7fe0fdffb640 1 -- start start 2023-10-16T11:06:03.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.323+0000 7fe0fdffb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0f806a4b0 0x7fe0f80ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:03.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.323+0000 7fe0fdffb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f80cb030 con 0x7fe0f815ad70 2023-10-16T11:06:03.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.323+0000 7fe0fdffb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f80c91c0 con 0x7fe0f80c9ca0 2023-10-16T11:06:03.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.323+0000 7fe0fdffb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f80c94a0 con 0x7fe0f806a4b0 2023-10-16T11:06:03.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.323+0000 7fe0f77fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe0f815ad70 0x7fe0f8069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53038/0 (socket says 172.21.15.84:53038) 2023-10-16T11:06:03.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.323+0000 7fe0f77fe640 1 -- 172.21.15.84:0/378539398 learned_addr learned my addr 172.21.15.84:0/378539398 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:03.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.324+0000 7fe0f6ffd640 1 -- 172.21.15.84:0/378539398 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 588209780 0 0) 0x7fe0f80cb030 con 0x7fe0f815ad70 2023-10-16T11:06:03.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.324+0000 7fe0effff640 1 --2- 172.21.15.84:0/378539398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0f806a4b0 0x7fe0f80ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:03.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.324+0000 7fe0f6ffd640 1 -- 172.21.15.84:0/378539398 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0e4003520 con 0x7fe0f815ad70 2023-10-16T11:06:03.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.324+0000 7fe0f6ffd640 1 -- 172.21.15.84:0/378539398 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 922042958 0 0) 0x7fe0e4003520 con 0x7fe0f815ad70 2023-10-16T11:06:03.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.324+0000 7fe0f6ffd640 1 -- 172.21.15.84:0/378539398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0f806a4b0 msgr2=0x7fe0f80ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:03.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.324+0000 7fe0f6ffd640 1 --2- 172.21.15.84:0/378539398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0f806a4b0 0x7fe0f80ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.324+0000 7fe0f6ffd640 1 -- 172.21.15.84:0/378539398 >> v1:172.21.15.157:6789/0 conn(0x7fe0f80c9ca0 legacy=0x7fe0f80ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:03.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.324+0000 7fe0f6ffd640 1 -- 172.21.15.84:0/378539398 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f80c9780 con 0x7fe0f815ad70 2023-10-16T11:06:03.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.324+0000 7fe0f6ffd640 1 -- 172.21.15.84:0/378539398 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fe0e8001670 con 0x7fe0f815ad70 2023-10-16T11:06:03.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.324+0000 7fe0f6ffd640 1 -- 172.21.15.84:0/378539398 >> v1:172.21.15.84:6789/0 conn(0x7fe0f815ad70 legacy=0x7fe0f8069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:03.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.324+0000 7fe0f6ffd640 1 --2- 172.21.15.84:0/378539398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0e40039a0 0x7fe0e4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:03.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.324+0000 7fe0f6ffd640 1 --2- 172.21.15.84:0/378539398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0f806a4b0 0x7fe0e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:03.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.324+0000 7fe0f6ffd640 1 --2- 172.21.15.84:0/378539398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e4004d00 0x7fe0e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.324+0000 7fe0f6ffd640 1 -- 172.21.15.84:0/378539398 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f80c94a0 con 0x7fe0e40039a0 2023-10-16T11:06:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.324+0000 7fe0f6ffd640 1 -- 172.21.15.84:0/378539398 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e400bb60 con 0x7fe0e4004d00 2023-10-16T11:06:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.324+0000 7fe0f6ffd640 1 -- 172.21.15.84:0/378539398 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e400bd20 con 0x7fe0f806a4b0 2023-10-16T11:06:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.325+0000 7fe0f77fe640 1 --2- 172.21.15.84:0/378539398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0e40039a0 0x7fe0e4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.325+0000 7fe0f7fff640 1 --2- 172.21.15.84:0/378539398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e4004d00 0x7fe0e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:03.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.325+0000 7fe0effff640 1 --2- 172.21.15.84:0/378539398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0f806a4b0 0x7fe0e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:03.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.325+0000 7fe0f77fe640 1 -- 172.21.15.84:0/378539398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0f806a4b0 msgr2=0x7fe0e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:03.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.325+0000 7fe0f77fe640 1 --2- 172.21.15.84:0/378539398 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0f806a4b0 0x7fe0e40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.325+0000 7fe0f77fe640 1 -- 172.21.15.84:0/378539398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e4004d00 msgr2=0x7fe0e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:03.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.325+0000 7fe0f77fe640 1 --2- 172.21.15.84:0/378539398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e4004d00 0x7fe0e4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.325+0000 7fe0f77fe640 1 -- 172.21.15.84:0/378539398 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e4003520 con 0x7fe0e40039a0 2023-10-16T11:06:03.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.326+0000 7fe0f77fe640 1 --2- 172.21.15.84:0/378539398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0e40039a0 0x7fe0e4003dc0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800c0e0 tx=0x7fe0e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc11eea18a8c3a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:03.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.326+0000 7fe0f6ffd640 1 -- 172.21.15.84:0/378539398 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8005ad0 con 0x7fe0e40039a0 2023-10-16T11:06:03.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.326+0000 7fe0f6ffd640 1 -- 172.21.15.84:0/378539398 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e8005c30 con 0x7fe0e40039a0 2023-10-16T11:06:03.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.326+0000 7fe0f6ffd640 1 -- 172.21.15.84:0/378539398 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e802c7d0 con 0x7fe0e40039a0 2023-10-16T11:06:03.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.327+0000 7fe0fdffb640 1 -- 172.21.15.84:0/378539398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0e40039a0 msgr2=0x7fe0e4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:03.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.327+0000 7fe0fdffb640 1 --2- 172.21.15.84:0/378539398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0e40039a0 0x7fe0e4003dc0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800c0e0 tx=0x7fe0e40057f0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.327+0000 7fe0f7fff640 1 -- 172.21.15.84:0/378539398 reap_dead start 2023-10-16T11:06:03.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.327+0000 7fe0fdffb640 1 -- 172.21.15.84:0/378539398 shutdown_connections 2023-10-16T11:06:03.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.327+0000 7fe0fdffb640 1 -- 172.21.15.84:0/378539398 >> 172.21.15.84:0/378539398 conn(0x7fe0f80ce9b0 msgr2=0x7fe0f80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:03.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.327+0000 7fe0fdffb640 1 -- 172.21.15.84:0/378539398 shutdown_connections 2023-10-16T11:06:03.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.327+0000 7fe0fdffb640 1 -- 172.21.15.84:0/378539398 wait complete. 2023-10-16T11:06:03.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.329+0000 7fe0fdffb640 1 Processor -- start 2023-10-16T11:06:03.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.329+0000 7fe0fdffb640 1 -- start start 2023-10-16T11:06:03.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.330+0000 7fe0fdffb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e40039a0 0x7fe0f8158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:03.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.330+0000 7fe0fdffb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0f80b0230 0x7fe0f80b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:03.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.330+0000 7fe0fdffb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0f80ac890 0x7fe0f80acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:03.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.330+0000 7fe0fdffb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f80ad250 con 0x7fe0f80b0230 2023-10-16T11:06:03.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.330+0000 7fe0fdffb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f80cc990 con 0x7fe0e40039a0 2023-10-16T11:06:03.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.330+0000 7fe0fdffb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f81586b0 con 0x7fe0f80ac890 2023-10-16T11:06:03.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.330+0000 7fe0effff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0f80b0230 0x7fe0f80b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:03.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.330+0000 7fe0effff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0f80b0230 0x7fe0f80b06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46944/0 (socket says 172.21.15.84:46944) 2023-10-16T11:06:03.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.330+0000 7fe0effff640 1 -- 172.21.15.84:0/4222886432 learned_addr learned my addr 172.21.15.84:0/4222886432 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:03.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.330+0000 7fe0f77fe640 1 --2- 172.21.15.84:0/4222886432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e40039a0 0x7fe0f8158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:03.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.330+0000 7fe0f7fff640 1 --2- 172.21.15.84:0/4222886432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0f80ac890 0x7fe0f80acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:03.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.331+0000 7fe0effff640 1 -- 172.21.15.84:0/4222886432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0f80ac890 msgr2=0x7fe0f80acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:03.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.331+0000 7fe0effff640 1 --2- 172.21.15.84:0/4222886432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0f80ac890 0x7fe0f80acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.331+0000 7fe0effff640 1 -- 172.21.15.84:0/4222886432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e40039a0 msgr2=0x7fe0f8158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:03.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.331+0000 7fe0effff640 1 --2- 172.21.15.84:0/4222886432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e40039a0 0x7fe0f8158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.331+0000 7fe0effff640 1 -- 172.21.15.84:0/4222886432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f80b0d80 con 0x7fe0f80b0230 2023-10-16T11:06:03.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.331+0000 7fe0effff640 1 --2- 172.21.15.84:0/4222886432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0f80b0230 0x7fe0f80b06b0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000e840 tx=0x7fe0e000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:03.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.331+0000 7fe0f4ff9640 1 -- 172.21.15.84:0/4222886432 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e000c8d0 con 0x7fe0f80b0230 2023-10-16T11:06:03.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.331+0000 7fe0f4ff9640 1 -- 172.21.15.84:0/4222886432 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e000ca30 con 0x7fe0f80b0230 2023-10-16T11:06:03.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.331+0000 7fe0fdffb640 1 -- 172.21.15.84:0/4222886432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f80b1060 con 0x7fe0f80b0230 2023-10-16T11:06:03.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.331+0000 7fe0f4ff9640 1 -- 172.21.15.84:0/4222886432 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e000cdc0 con 0x7fe0f80b0230 2023-10-16T11:06:03.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.332+0000 7fe0fdffb640 1 -- 172.21.15.84:0/4222886432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f80b1340 con 0x7fe0f80b0230 2023-10-16T11:06:03.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.333+0000 7fe0fdffb640 1 -- 172.21.15.84:0/4222886432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f804e160 con 0x7fe0f80b0230 2023-10-16T11:06:03.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.334+0000 7fe0f4ff9640 1 -- 172.21.15.84:0/4222886432 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 99) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fe0e0013410 con 0x7fe0f80b0230 2023-10-16T11:06:03.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.335+0000 7fe0f4ff9640 1 --2- 172.21.15.84:0/4222886432 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7fe0d806afc0 0x7fe0d806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:03.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.335+0000 7fe0f4ff9640 1 -- 172.21.15.84:0/4222886432 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6081+0+0 (secure 0 0 0) 0x7fe0e00895a0 con 0x7fe0f80b0230 2023-10-16T11:06:03.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.335+0000 7fe0f77fe640 1 -- 172.21.15.84:0/4222886432 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7fe0d806afc0 msgr2=0x7fe0d806d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1697458478 2023-10-16T11:06:03.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.335+0000 7fe0f77fe640 1 --2- 172.21.15.84:0/4222886432 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7fe0d806afc0 0x7fe0d806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:06:03.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.340+0000 7fe0f4ff9640 1 -- 172.21.15.84:0/4222886432 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe0e0054360 con 0x7fe0f80b0230 2023-10-16T11:06:03.368 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:03.366+0000 7fa2c2012040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:06:03.407 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:03.405+0000 7fbc4bc32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:06:03.528 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:03.527+0000 7fa2c2012040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:06:03.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.533+0000 7fe0fdffb640 1 -- 172.21.15.84:0/4222886432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe0f80ccad0 con 0x7fe0f80b0230 2023-10-16T11:06:03.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.534+0000 7fe0f4ff9640 1 -- 172.21.15.84:0/4222886432 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v172) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe0e0009380 con 0x7fe0f80b0230 2023-10-16T11:06:03.536 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:06:03.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.536+0000 7fe0f77fe640 1 -- 172.21.15.84:0/4222886432 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7fe0d806afc0 msgr2=0x7fe0d806d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1697458478 2023-10-16T11:06:03.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.536+0000 7fe0f77fe640 1 --2- 172.21.15.84:0/4222886432 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7fe0d806afc0 0x7fe0d806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:06:03.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.536+0000 7fe0fdffb640 1 -- 172.21.15.84:0/4222886432 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7fe0d806afc0 msgr2=0x7fe0d806d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:06:03.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.536+0000 7fe0fdffb640 1 --2- 172.21.15.84:0/4222886432 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7fe0d806afc0 0x7fe0d806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.536+0000 7fe0fdffb640 1 -- 172.21.15.84:0/4222886432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0f80b0230 msgr2=0x7fe0f80b06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:03.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.536+0000 7fe0fdffb640 1 --2- 172.21.15.84:0/4222886432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0f80b0230 0x7fe0f80b06b0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000e840 tx=0x7fe0e000ed00 comp rx=0 tx=0).stop 2023-10-16T11:06:03.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.536+0000 7fe0fdffb640 1 -- 172.21.15.84:0/4222886432 shutdown_connections 2023-10-16T11:06:03.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.536+0000 7fe0fdffb640 1 --2- 172.21.15.84:0/4222886432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0f80ac890 0x7fe0f80acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.536+0000 7fe0fdffb640 1 --2- 172.21.15.84:0/4222886432 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7fe0d806afc0 0x7fe0d806d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.536+0000 7fe0fdffb640 1 --2- 172.21.15.84:0/4222886432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0f80b0230 0x7fe0f80b06b0 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.536+0000 7fe0fdffb640 1 --2- 172.21.15.84:0/4222886432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0e40039a0 0x7fe0f8158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.536+0000 7fe0fdffb640 1 -- 172.21.15.84:0/4222886432 >> 172.21.15.84:0/4222886432 conn(0x7fe0f80ce9b0 msgr2=0x7fe0f8069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:03.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.537+0000 7fe0fdffb640 1 -- 172.21.15.84:0/4222886432 shutdown_connections 2023-10-16T11:06:03.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.537+0000 7fe0fdffb640 1 -- 172.21.15.84:0/4222886432 wait complete. 2023-10-16T11:06:03.557 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T11:06:03.575 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:03.574+0000 7fbc4bc32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:06:03.636 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:03.635+0000 7fa2c2012040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:06:03.686 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:03.684+0000 7fa2c2012040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:06:03.690 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:03.688+0000 7fbc4bc32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:06:03.742 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:03.741+0000 7fbc4bc32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:06:03.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.756+0000 7f0d8f59e640 1 Processor -- start 2023-10-16T11:06:03.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.756+0000 7f0d8f59e640 1 -- start start 2023-10-16T11:06:03.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.758+0000 7f0d8f59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0d900cded0 0x7f0d900b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:03.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.758+0000 7f0d8f59e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d9015d150 con 0x7f0d900b8480 2023-10-16T11:06:03.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.758+0000 7f0d8f59e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d900d1180 con 0x7f0d9015cd70 2023-10-16T11:06:03.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.758+0000 7f0d8f59e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d900d1460 con 0x7f0d900cded0 2023-10-16T11:06:03.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.758+0000 7f0d8dd9b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0d900b8480 0x7f0d900cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53042/0 (socket says 172.21.15.84:53042) 2023-10-16T11:06:03.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.758+0000 7f0d8dd9b640 1 -- 172.21.15.84:0/2071023171 learned_addr learned my addr 172.21.15.84:0/2071023171 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:03.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.758+0000 7f0d8ed9d640 1 --2- 172.21.15.84:0/2071023171 >> v2:172.21.15.84:3300/0 conn(0x7f0d900cded0 0x7f0d900b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:03.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.759+0000 7f0d8ed9d640 1 -- 172.21.15.84:0/2071023171 >> v1:172.21.15.157:6789/0 conn(0x7f0d9015cd70 legacy=0x7f0d900d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:03.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.759+0000 7f0d8ed9d640 1 -- 172.21.15.84:0/2071023171 >> v1:172.21.15.84:6789/0 conn(0x7f0d900b8480 legacy=0x7f0d900cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:03.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.759+0000 7f0d8ed9d640 1 -- 172.21.15.84:0/2071023171 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d900d1740 con 0x7f0d900cded0 2023-10-16T11:06:03.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.760+0000 7f0d8ed9d640 1 --2- 172.21.15.84:0/2071023171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d900cded0 0x7f0d900b5c90 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009d90 tx=0x7f0d8802bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac348fbaacc69f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:03.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.761+0000 7f0d8d59a640 1 -- 172.21.15.84:0/2071023171 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8802ccf0 con 0x7f0d900cded0 2023-10-16T11:06:03.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.761+0000 7f0d8d59a640 1 -- 172.21.15.84:0/2071023171 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8802ce50 con 0x7f0d900cded0 2023-10-16T11:06:03.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.761+0000 7f0d8d59a640 1 -- 172.21.15.84:0/2071023171 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d88034660 con 0x7f0d900cded0 2023-10-16T11:06:03.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.761+0000 7f0d8f59e640 1 -- 172.21.15.84:0/2071023171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d900cded0 msgr2=0x7f0d900b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:03.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.761+0000 7f0d8f59e640 1 --2- 172.21.15.84:0/2071023171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d900cded0 0x7f0d900b5c90 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009d90 tx=0x7f0d8802bf40 comp rx=0 tx=0).stop 2023-10-16T11:06:03.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.762+0000 7f0d8f59e640 1 -- 172.21.15.84:0/2071023171 shutdown_connections 2023-10-16T11:06:03.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.762+0000 7f0d8f59e640 1 --2- 172.21.15.84:0/2071023171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d900cded0 0x7f0d900b5c90 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.762+0000 7f0d8f59e640 1 -- 172.21.15.84:0/2071023171 >> 172.21.15.84:0/2071023171 conn(0x7f0d900cf570 msgr2=0x7f0d900cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:03.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.762+0000 7f0d8f59e640 1 -- 172.21.15.84:0/2071023171 shutdown_connections 2023-10-16T11:06:03.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.762+0000 7f0d8f59e640 1 -- 172.21.15.84:0/2071023171 wait complete. 2023-10-16T11:06:03.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.763+0000 7f0d8f59e640 1 Processor -- start 2023-10-16T11:06:03.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.763+0000 7f0d8f59e640 1 -- start start 2023-10-16T11:06:03.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.764+0000 7f0d8f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d900b8480 0x7f0d9015ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:03.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.764+0000 7f0d8f59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d900cded0 0x7f0d901f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:03.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.764+0000 7f0d8f59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d9015cd70 0x7f0d900aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:03.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.764+0000 7f0d8f59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9015f3e0 con 0x7f0d900b8480 2023-10-16T11:06:03.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.764+0000 7f0d8f59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9015f6c0 con 0x7f0d9015cd70 2023-10-16T11:06:03.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.764+0000 7f0d8f59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9015f9a0 con 0x7f0d900cded0 2023-10-16T11:06:03.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.765+0000 7f0d8e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d900b8480 0x7f0d9015ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:03.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.765+0000 7f0d8e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d900b8480 0x7f0d9015ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46960/0 (socket says 172.21.15.84:46960) 2023-10-16T11:06:03.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.765+0000 7f0d8e59c640 1 -- 172.21.15.84:0/3953047480 learned_addr learned my addr 172.21.15.84:0/3953047480 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:03.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.765+0000 7f0d8dd9b640 1 --2- 172.21.15.84:0/3953047480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d900cded0 0x7f0d901f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:03.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.765+0000 7f0d8ed9d640 1 --2- 172.21.15.84:0/3953047480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d9015cd70 0x7f0d900aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:03.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.765+0000 7f0d8e59c640 1 -- 172.21.15.84:0/3953047480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d900cded0 msgr2=0x7f0d901f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:03.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.765+0000 7f0d8e59c640 1 --2- 172.21.15.84:0/3953047480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d900cded0 0x7f0d901f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.765+0000 7f0d8e59c640 1 -- 172.21.15.84:0/3953047480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d9015cd70 msgr2=0x7f0d900aa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:03.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.765+0000 7f0d8e59c640 1 --2- 172.21.15.84:0/3953047480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d9015cd70 0x7f0d900aa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.765+0000 7f0d8e59c640 1 -- 172.21.15.84:0/3953047480 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d88009a70 con 0x7f0d900b8480 2023-10-16T11:06:03.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.765+0000 7f0d8ed9d640 1 --2- 172.21.15.84:0/3953047480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d9015cd70 0x7f0d900aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:03.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.765+0000 7f0d8e59c640 1 --2- 172.21.15.84:0/3953047480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d900b8480 0x7f0d9015ed50 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000c400 tx=0x7f0d8000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:03.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.766+0000 7f0d7effd640 1 -- 172.21.15.84:0/3953047480 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d800052d0 con 0x7f0d900b8480 2023-10-16T11:06:03.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.766+0000 7f0d7effd640 1 -- 172.21.15.84:0/3953047480 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d80005430 con 0x7f0d900b8480 2023-10-16T11:06:03.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.766+0000 7f0d8f59e640 1 -- 172.21.15.84:0/3953047480 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d9015fc80 con 0x7f0d900b8480 2023-10-16T11:06:03.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.766+0000 7f0d7effd640 1 -- 172.21.15.84:0/3953047480 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80003830 con 0x7f0d900b8480 2023-10-16T11:06:03.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.766+0000 7f0d8f59e640 1 -- 172.21.15.84:0/3953047480 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d901f3390 con 0x7f0d900b8480 2023-10-16T11:06:03.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.767+0000 7f0d8f59e640 1 -- 172.21.15.84:0/3953047480 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d900cee50 con 0x7f0d900b8480 2023-10-16T11:06:03.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.768+0000 7f0d7effd640 1 -- 172.21.15.84:0/3953047480 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 99) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f0d80003990 con 0x7f0d900b8480 2023-10-16T11:06:03.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.769+0000 7f0d7effd640 1 --2- 172.21.15.84:0/3953047480 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f0d7006af90 0x7f0d7006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:03.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.769+0000 7f0d7effd640 1 -- 172.21.15.84:0/3953047480 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f0d80088e70 con 0x7f0d900b8480 2023-10-16T11:06:03.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.769+0000 7f0d8dd9b640 1 -- 172.21.15.84:0/3953047480 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f0d7006af90 msgr2=0x7f0d7006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1697458478 2023-10-16T11:06:03.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.769+0000 7f0d8dd9b640 1 --2- 172.21.15.84:0/3953047480 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f0d7006af90 0x7f0d7006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:06:03.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.773+0000 7f0d7effd640 1 -- 172.21.15.84:0/3953047480 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0d80053c90 con 0x7f0d900b8480 2023-10-16T11:06:03.794 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:03.793+0000 7fa2c2012040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:06:03.856 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:03.854+0000 7fbc4bc32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:06:03.901 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:03.900+0000 7fa2c2012040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:06:03.964 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:03.963+0000 7fbc4bc32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:06:03.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.970+0000 7f0d8dd9b640 1 -- 172.21.15.84:0/3953047480 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f0d7006af90 msgr2=0x7f0d7006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1697458478 2023-10-16T11:06:03.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.970+0000 7f0d8dd9b640 1 --2- 172.21.15.84:0/3953047480 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f0d7006af90 0x7f0d7006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:06:03.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.982+0000 7f0d8f59e640 1 -- 172.21.15.84:0/3953047480 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0d90160140 con 0x7f0d900b8480 2023-10-16T11:06:03.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.983+0000 7f0d7effd640 1 -- 172.21.15.84:0/3953047480 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v172) v1 ==== 80+0+470 (secure 0 0 0) 0x7f0d80017080 con 0x7f0d900b8480 2023-10-16T11:06:03.984 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3425147335 2023-10-17T11:05:57.603800+0000 2023-10-16T11:06:03.985 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1161603770 2023-10-17T11:05:57.603800+0000 2023-10-16T11:06:03.985 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6833/2442400184 2023-10-17T11:05:57.603800+0000 2023-10-16T11:06:03.985 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6832/2442400184 2023-10-17T11:05:57.603800+0000 2023-10-16T11:06:03.985 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3633178028 2023-10-17T11:05:57.603800+0000 2023-10-16T11:06:03.986 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1914327915 2023-10-17T11:05:57.603800+0000 2023-10-16T11:06:03.986 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2602045766 2023-10-17T11:05:57.603800+0000 2023-10-16T11:06:03.986 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3065897528 2023-10-17T11:05:57.603800+0000 2023-10-16T11:06:03.987 INFO:teuthology.orchestra.run.smithi084.stderr:listed 8 entries 2023-10-16T11:06:03.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.985+0000 7f0d8f59e640 1 -- 172.21.15.84:0/3953047480 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f0d7006af90 msgr2=0x7f0d7006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:06:03.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.985+0000 7f0d8f59e640 1 --2- 172.21.15.84:0/3953047480 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f0d7006af90 0x7f0d7006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.985+0000 7f0d8f59e640 1 -- 172.21.15.84:0/3953047480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d900b8480 msgr2=0x7f0d9015ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:03.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.985+0000 7f0d8f59e640 1 --2- 172.21.15.84:0/3953047480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d900b8480 0x7f0d9015ed50 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000c400 tx=0x7f0d8000c8c0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.985+0000 7f0d8f59e640 1 -- 172.21.15.84:0/3953047480 shutdown_connections 2023-10-16T11:06:03.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.985+0000 7f0d8f59e640 1 --2- 172.21.15.84:0/3953047480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d9015cd70 0x7f0d900aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.985+0000 7f0d8f59e640 1 --2- 172.21.15.84:0/3953047480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d900cded0 0x7f0d901f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.985+0000 7f0d8f59e640 1 --2- 172.21.15.84:0/3953047480 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f0d7006af90 0x7f0d7006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.985+0000 7f0d8f59e640 1 --2- 172.21.15.84:0/3953047480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d900b8480 0x7f0d9015ed50 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:03.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.985+0000 7f0d8f59e640 1 -- 172.21.15.84:0/3953047480 >> 172.21.15.84:0/3953047480 conn(0x7f0d900cf570 msgr2=0x7f0d900b76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:03.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.985+0000 7f0d8f59e640 1 -- 172.21.15.84:0/3953047480 shutdown_connections 2023-10-16T11:06:03.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:03.985+0000 7f0d8f59e640 1 -- 172.21.15.84:0/3953047480 wait complete. 2023-10-16T11:06:04.004 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T11:06:04.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:06:04.004 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T11:06:04.060 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:04.059+0000 7fa2c2012040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:06:04.109 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:04.108+0000 7fa2c2012040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:06:04.131 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:04.129+0000 7fbc4bc32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:06:04.142 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T11:06:04.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:06:04.142 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T11:06:04.186 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:04.185+0000 7fbc4bc32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:06:04.192 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:06:04.192 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:06:04.231 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:04.229+0000 7fa2c2012040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:06:04.247 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:06:04.247 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:06:04.247 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:06:04.299 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:06:04.300 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:06:04.300 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:06:04.315 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:04.313+0000 7fbc4bc32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:06:04.354 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:06:04.355 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:06:04.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.558+0000 7f0e197ff640 1 Processor -- start 2023-10-16T11:06:04.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.558+0000 7f0e197ff640 1 -- start start 2023-10-16T11:06:04.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.560+0000 7f0e197ff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1415c9f0 0x7f0e140c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:04.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.560+0000 7f0e197ff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0e1415dde0 0x7f0e140ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:04.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.560+0000 7f0e197ff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e14160a90 con 0x7f0e140c9230 2023-10-16T11:06:04.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.560+0000 7f0e197ff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e140cdec0 con 0x7f0e1415dde0 2023-10-16T11:06:04.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.560+0000 7f0e197ff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e140cd900 con 0x7f0e1415c9f0 2023-10-16T11:06:04.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.560+0000 7f0e0bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0e140c9230 0x7f0e140709e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53044/0 (socket says 172.21.15.84:53044) 2023-10-16T11:06:04.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.560+0000 7f0e0bfff640 1 -- 172.21.15.84:0/106413842 learned_addr learned my addr 172.21.15.84:0/106413842 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:04.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.560+0000 7f0e18ffe640 1 --2- 172.21.15.84:0/106413842 >> v2:172.21.15.157:3300/0 conn(0x7f0e1415dde0 0x7f0e140ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:04.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.560+0000 7f0e13fff640 1 --2- 172.21.15.84:0/106413842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1415c9f0 0x7f0e140c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:04.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.561+0000 7f0e137fe640 1 -- 172.21.15.84:0/106413842 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039063380 0 0) 0x7f0e14160a90 con 0x7f0e140c9230 2023-10-16T11:06:04.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.561+0000 7f0e137fe640 1 -- 172.21.15.84:0/106413842 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e00003520 con 0x7f0e140c9230 2023-10-16T11:06:04.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.561+0000 7f0e137fe640 1 -- 172.21.15.84:0/106413842 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2691092566 0 0) 0x7f0e00003520 con 0x7f0e140c9230 2023-10-16T11:06:04.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.561+0000 7f0e137fe640 1 -- 172.21.15.84:0/106413842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1415c9f0 msgr2=0x7f0e140c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:04.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.561+0000 7f0e137fe640 1 --2- 172.21.15.84:0/106413842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1415c9f0 0x7f0e140c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:04.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.561+0000 7f0e137fe640 1 -- 172.21.15.84:0/106413842 >> v2:172.21.15.157:3300/0 conn(0x7f0e1415dde0 msgr2=0x7f0e140ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:04.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.561+0000 7f0e137fe640 1 --2- 172.21.15.84:0/106413842 >> v2:172.21.15.157:3300/0 conn(0x7f0e1415dde0 0x7f0e140ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:04.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.561+0000 7f0e137fe640 1 -- 172.21.15.84:0/106413842 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e140cdbe0 con 0x7f0e140c9230 2023-10-16T11:06:04.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.561+0000 7f0e137fe640 1 -- 172.21.15.84:0/106413842 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f0dfc001670 con 0x7f0e140c9230 2023-10-16T11:06:04.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.561+0000 7f0e137fe640 1 -- 172.21.15.84:0/106413842 >> v1:172.21.15.84:6789/0 conn(0x7f0e140c9230 legacy=0x7f0e140709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:04.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.561+0000 7f0e137fe640 1 --2- 172.21.15.84:0/106413842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e00003970 0x7f0e00003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:04.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.561+0000 7f0e137fe640 1 --2- 172.21.15.84:0/106413842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1415c9f0 0x7f0e000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:04.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.561+0000 7f0e137fe640 1 --2- 172.21.15.84:0/106413842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e00004d00 0x7f0e00005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:04.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.561+0000 7f0e137fe640 1 -- 172.21.15.84:0/106413842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e140cd900 con 0x7f0e00003970 2023-10-16T11:06:04.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.561+0000 7f0e137fe640 1 -- 172.21.15.84:0/106413842 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0000b980 con 0x7f0e00004d00 2023-10-16T11:06:04.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.561+0000 7f0e137fe640 1 -- 172.21.15.84:0/106413842 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0000bac0 con 0x7f0e1415c9f0 2023-10-16T11:06:04.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.562+0000 7f0e13fff640 1 --2- 172.21.15.84:0/106413842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e00003970 0x7f0e00003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:04.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.562+0000 7f0e13fff640 1 -- 172.21.15.84:0/106413842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1415c9f0 msgr2=0x7f0e000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:04.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.562+0000 7f0e13fff640 1 --2- 172.21.15.84:0/106413842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1415c9f0 0x7f0e000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:04.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.562+0000 7f0e13fff640 1 -- 172.21.15.84:0/106413842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e00004d00 msgr2=0x7f0e00005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:04.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.562+0000 7f0e13fff640 1 --2- 172.21.15.84:0/106413842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e00004d00 0x7f0e00005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:04.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.562+0000 7f0e13fff640 1 -- 172.21.15.84:0/106413842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e00003520 con 0x7f0e00003970 2023-10-16T11:06:04.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.563+0000 7f0e13fff640 1 --2- 172.21.15.84:0/106413842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e00003970 0x7f0e00003d90 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009ac0 tx=0x7f0e000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef7e6efced5f2cb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:04.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.563+0000 7f0e137fe640 1 -- 172.21.15.84:0/106413842 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0402c710 con 0x7f0e00003970 2023-10-16T11:06:04.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.563+0000 7f0e137fe640 1 -- 172.21.15.84:0/106413842 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0402c870 con 0x7f0e00003970 2023-10-16T11:06:04.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.563+0000 7f0e137fe640 1 -- 172.21.15.84:0/106413842 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0402cc00 con 0x7f0e00003970 2023-10-16T11:06:04.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.564+0000 7f0e197ff640 1 -- 172.21.15.84:0/106413842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e00003970 msgr2=0x7f0e00003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:04.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.564+0000 7f0e197ff640 1 --2- 172.21.15.84:0/106413842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e00003970 0x7f0e00003d90 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009ac0 tx=0x7f0e000057f0 comp rx=0 tx=0).stop 2023-10-16T11:06:04.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.564+0000 7f0e18ffe640 1 -- 172.21.15.84:0/106413842 reap_dead start 2023-10-16T11:06:04.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.564+0000 7f0e197ff640 1 -- 172.21.15.84:0/106413842 shutdown_connections 2023-10-16T11:06:04.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.564+0000 7f0e197ff640 1 -- 172.21.15.84:0/106413842 >> 172.21.15.84:0/106413842 conn(0x7f0e140ce6a0 msgr2=0x7f0e140ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:04.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.564+0000 7f0e197ff640 1 -- 172.21.15.84:0/106413842 shutdown_connections 2023-10-16T11:06:04.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.564+0000 7f0e197ff640 1 -- 172.21.15.84:0/106413842 wait complete. 2023-10-16T11:06:04.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.566+0000 7f0e197ff640 1 Processor -- start 2023-10-16T11:06:04.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.566+0000 7f0e197ff640 1 -- start start 2023-10-16T11:06:04.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.567+0000 7f0e197ff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e00003970 0x7f0e140c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:04.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.567+0000 7f0e197ff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e14159740 0x7f0e14157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:04.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.567+0000 7f0e197ff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e14159bc0 0x7f0e14157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:04.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.567+0000 7f0e197ff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e14158290 con 0x7f0e00003970 2023-10-16T11:06:04.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.567+0000 7f0e197ff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e140c6560 con 0x7f0e14159bc0 2023-10-16T11:06:04.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.567+0000 7f0e197ff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e140c2270 con 0x7f0e14159740 2023-10-16T11:06:04.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.567+0000 7f0e13fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e00003970 0x7f0e140c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:04.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.567+0000 7f0e13fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e00003970 0x7f0e140c5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46964/0 (socket says 172.21.15.84:46964) 2023-10-16T11:06:04.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.567+0000 7f0e13fff640 1 -- 172.21.15.84:0/3371492333 learned_addr learned my addr 172.21.15.84:0/3371492333 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:04.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.567+0000 7f0e0bfff640 1 --2- 172.21.15.84:0/3371492333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e14159740 0x7f0e14157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:04.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.567+0000 7f0e18ffe640 1 --2- 172.21.15.84:0/3371492333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e14159bc0 0x7f0e14157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:04.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.567+0000 7f0e13fff640 1 -- 172.21.15.84:0/3371492333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e14159740 msgr2=0x7f0e14157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:04.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.567+0000 7f0e13fff640 1 --2- 172.21.15.84:0/3371492333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e14159740 0x7f0e14157810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:04.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.567+0000 7f0e13fff640 1 -- 172.21.15.84:0/3371492333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e14159bc0 msgr2=0x7f0e14157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:04.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.567+0000 7f0e13fff640 1 --2- 172.21.15.84:0/3371492333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e14159bc0 0x7f0e14157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:04.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.567+0000 7f0e13fff640 1 -- 172.21.15.84:0/3371492333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e140c2550 con 0x7f0e00003970 2023-10-16T11:06:04.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.568+0000 7f0e13fff640 1 --2- 172.21.15.84:0/3371492333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e00003970 0x7f0e140c5770 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009ac0 tx=0x7f0e04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:04.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.568+0000 7f0e117fa640 1 -- 172.21.15.84:0/3371492333 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e04003810 con 0x7f0e00003970 2023-10-16T11:06:04.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.568+0000 7f0e117fa640 1 -- 172.21.15.84:0/3371492333 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0402de90 con 0x7f0e00003970 2023-10-16T11:06:04.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.568+0000 7f0e197ff640 1 -- 172.21.15.84:0/3371492333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e140c2830 con 0x7f0e00003970 2023-10-16T11:06:04.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.568+0000 7f0e117fa640 1 -- 172.21.15.84:0/3371492333 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e04003b90 con 0x7f0e00003970 2023-10-16T11:06:04.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.568+0000 7f0e197ff640 1 -- 172.21.15.84:0/3371492333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e140c6180 con 0x7f0e00003970 2023-10-16T11:06:04.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.569+0000 7f0e197ff640 1 -- 172.21.15.84:0/3371492333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e14071580 con 0x7f0e00003970 2023-10-16T11:06:04.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.570+0000 7f0e117fa640 1 -- 172.21.15.84:0/3371492333 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 99) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f0e04003cf0 con 0x7f0e00003970 2023-10-16T11:06:04.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.572+0000 7f0e117fa640 1 --2- 172.21.15.84:0/3371492333 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f0df406af90 0x7f0df406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:04.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.572+0000 7f0e0bfff640 1 -- 172.21.15.84:0/3371492333 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f0df406af90 msgr2=0x7f0df406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1697458478 2023-10-16T11:06:04.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.572+0000 7f0e117fa640 1 -- 172.21.15.84:0/3371492333 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f0e040ab8e0 con 0x7f0e00003970 2023-10-16T11:06:04.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.572+0000 7f0e0bfff640 1 --2- 172.21.15.84:0/3371492333 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f0df406af90 0x7f0df406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:06:04.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.576+0000 7f0e117fa640 1 -- 172.21.15.84:0/3371492333 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0e04076700 con 0x7f0e00003970 2023-10-16T11:06:04.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.773+0000 7f0e0bfff640 1 -- 172.21.15.84:0/3371492333 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f0df406af90 msgr2=0x7f0df406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1697458478 2023-10-16T11:06:04.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.773+0000 7f0e0bfff640 1 --2- 172.21.15.84:0/3371492333 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f0df406af90 0x7f0df406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:06:04.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.804+0000 7f0e197ff640 1 -- 172.21.15.84:0/3371492333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e14161670 con 0x7f0e00003970 2023-10-16T11:06:04.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.805+0000 7f0e117fa640 1 -- 172.21.15.84:0/3371492333 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f0e04079d20 con 0x7f0e00003970 2023-10-16T11:06:04.806 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:04.807 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":88,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6718,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3060266169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3060266169},{"type":"v1","addr":"172.21.15.157:6834","nonce":3060266169}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":6721,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2965939103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2965939103},{"type":"v1","addr":"172.21.15.157:6835","nonce":2965939103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9684,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3677909298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3677909298},{"type":"v1","addr":"172.21.15.84:6835","nonce":3677909298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:05:43.341956+0000","modified":"2023-10-16T11:05:46.428445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9687},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9687":{"gid":9687,"name":"c","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/424056375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":424056375},{"type":"v1","addr":"172.21.15.84:6837","nonce":424056375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-16T11:06:04.807 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 88 2023-10-16T11:06:04.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.807+0000 7f0e197ff640 1 -- 172.21.15.84:0/3371492333 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f0df406af90 msgr2=0x7f0df406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:06:04.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.807+0000 7f0e197ff640 1 --2- 172.21.15.84:0/3371492333 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f0df406af90 0x7f0df406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:04.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.807+0000 7f0e197ff640 1 -- 172.21.15.84:0/3371492333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e00003970 msgr2=0x7f0e140c5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:04.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.807+0000 7f0e197ff640 1 --2- 172.21.15.84:0/3371492333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e00003970 0x7f0e140c5770 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009ac0 tx=0x7f0e04004730 comp rx=0 tx=0).stop 2023-10-16T11:06:04.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.807+0000 7f0e197ff640 1 -- 172.21.15.84:0/3371492333 shutdown_connections 2023-10-16T11:06:04.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.807+0000 7f0e197ff640 1 --2- 172.21.15.84:0/3371492333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e14159bc0 0x7f0e14157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:04.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.807+0000 7f0e197ff640 1 --2- 172.21.15.84:0/3371492333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e14159740 0x7f0e14157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:04.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.807+0000 7f0e197ff640 1 --2- 172.21.15.84:0/3371492333 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f0df406af90 0x7f0df406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:04.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.807+0000 7f0e197ff640 1 --2- 172.21.15.84:0/3371492333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e00003970 0x7f0e140c5770 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:04.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.807+0000 7f0e197ff640 1 -- 172.21.15.84:0/3371492333 >> 172.21.15.84:0/3371492333 conn(0x7f0e140ce6a0 msgr2=0x7f0e140cd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:04.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.807+0000 7f0e197ff640 1 -- 172.21.15.84:0/3371492333 shutdown_connections 2023-10-16T11:06:04.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:04.807+0000 7f0e197ff640 1 -- 172.21.15.84:0/3371492333 wait complete. 2023-10-16T11:06:04.827 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:06:05.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.008+0000 7f32cf0df640 1 Processor -- start 2023-10-16T11:06:05.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.009+0000 7f32cf0df640 1 -- start start 2023-10-16T11:06:05.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.010+0000 7f32cf0df640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32c8159950 0x7f32c815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.010+0000 7f32cf0df640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c815aef0 con 0x7f32c806c810 2023-10-16T11:06:05.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.010+0000 7f32cf0df640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c806cc80 con 0x7f32c8070450 2023-10-16T11:06:05.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.010+0000 7f32cf0df640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c815e490 con 0x7f32c8159950 2023-10-16T11:06:05.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.010+0000 7f32ce8de640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f32c806c810 0x7f32c80c9090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53050/0 (socket says 172.21.15.84:53050) 2023-10-16T11:06:05.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.010+0000 7f32ce8de640 1 -- 172.21.15.84:0/1673034241 learned_addr learned my addr 172.21.15.84:0/1673034241 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:05.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.011+0000 7f32ce0dd640 1 --2- 172.21.15.84:0/1673034241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32c8159950 0x7f32c815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:05.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.011+0000 7f32cd0db640 1 -- 172.21.15.84:0/1673034241 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645476311 0 0) 0x7f32c0001490 con 0x7f32c806c810 2023-10-16T11:06:05.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.011+0000 7f32cd0db640 1 -- 172.21.15.84:0/1673034241 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32b4003520 con 0x7f32c806c810 2023-10-16T11:06:05.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.011+0000 7f32cd0db640 1 -- 172.21.15.84:0/1673034241 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2130623357 0 0) 0x7f32b4003520 con 0x7f32c806c810 2023-10-16T11:06:05.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.011+0000 7f32cd0db640 1 -- 172.21.15.84:0/1673034241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32c8159950 msgr2=0x7f32c815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:05.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.011+0000 7f32cd0db640 1 --2- 172.21.15.84:0/1673034241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32c8159950 0x7f32c815a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.011+0000 7f32cd0db640 1 -- 172.21.15.84:0/1673034241 >> v1:172.21.15.157:6789/0 conn(0x7f32c8070450 legacy=0x7f32c806c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:05.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.011+0000 7f32cd0db640 1 -- 172.21.15.84:0/1673034241 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c815d910 con 0x7f32c806c810 2023-10-16T11:06:05.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.011+0000 7f32cd0db640 1 -- 172.21.15.84:0/1673034241 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f32c0001b80 con 0x7f32c806c810 2023-10-16T11:06:05.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.011+0000 7f32cd0db640 1 -- 172.21.15.84:0/1673034241 >> v1:172.21.15.84:6789/0 conn(0x7f32c806c810 legacy=0x7f32c80c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:05.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.012+0000 7f32cd0db640 1 --2- 172.21.15.84:0/1673034241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32c8159950 0x7f32b40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.012+0000 7f32cd0db640 1 --2- 172.21.15.84:0/1673034241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32b4003ee0 0x7f32b40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.012+0000 7f32cd0db640 1 --2- 172.21.15.84:0/1673034241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b4004d00 0x7f32b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.012+0000 7f32cd0db640 1 -- 172.21.15.84:0/1673034241 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c815e490 con 0x7f32b4003ee0 2023-10-16T11:06:05.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.012+0000 7f32cd0db640 1 -- 172.21.15.84:0/1673034241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b400bac0 con 0x7f32b4004d00 2023-10-16T11:06:05.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.012+0000 7f32cd0db640 1 -- 172.21.15.84:0/1673034241 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b400de90 con 0x7f32c8159950 2023-10-16T11:06:05.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.012+0000 7f32cd8dc640 1 --2- 172.21.15.84:0/1673034241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32b4003ee0 0x7f32b40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:05.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.012+0000 7f32ce0dd640 1 --2- 172.21.15.84:0/1673034241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32c8159950 0x7f32b40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:05.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.012+0000 7f32ce8de640 1 --2- 172.21.15.84:0/1673034241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b4004d00 0x7f32b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:05.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.012+0000 7f32cd8dc640 1 -- 172.21.15.84:0/1673034241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32c8159950 msgr2=0x7f32b40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:05.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.012+0000 7f32cd8dc640 1 --2- 172.21.15.84:0/1673034241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32c8159950 0x7f32b40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.012+0000 7f32cd8dc640 1 -- 172.21.15.84:0/1673034241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b4004d00 msgr2=0x7f32b4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:05.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.012+0000 7f32cd8dc640 1 --2- 172.21.15.84:0/1673034241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b4004d00 0x7f32b4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.012+0000 7f32cd8dc640 1 -- 172.21.15.84:0/1673034241 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b4003520 con 0x7f32b4003ee0 2023-10-16T11:06:05.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.013+0000 7f32cd8dc640 1 --2- 172.21.15.84:0/1673034241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32b4003ee0 0x7f32b40047c0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f32b8001860 tx=0x7f32b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932a634e3b1442b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:05.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.014+0000 7f32cd0db640 1 -- 172.21.15.84:0/1673034241 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8033410 con 0x7f32b4003ee0 2023-10-16T11:06:05.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.014+0000 7f32cd0db640 1 -- 172.21.15.84:0/1673034241 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b8033590 con 0x7f32b4003ee0 2023-10-16T11:06:05.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.014+0000 7f32cd0db640 1 -- 172.21.15.84:0/1673034241 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8033940 con 0x7f32b4003ee0 2023-10-16T11:06:05.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.014+0000 7f32cf0df640 1 -- 172.21.15.84:0/1673034241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32b4003ee0 msgr2=0x7f32b40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:05.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.014+0000 7f32cf0df640 1 --2- 172.21.15.84:0/1673034241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32b4003ee0 0x7f32b40047c0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f32b8001860 tx=0x7f32b40057f0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.014+0000 7f32ce8de640 1 -- 172.21.15.84:0/1673034241 reap_dead start 2023-10-16T11:06:05.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.014+0000 7f32cf0df640 1 -- 172.21.15.84:0/1673034241 shutdown_connections 2023-10-16T11:06:05.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.014+0000 7f32cf0df640 1 -- 172.21.15.84:0/1673034241 >> 172.21.15.84:0/1673034241 conn(0x7f32c80ce6a0 msgr2=0x7f32c80ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:05.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.014+0000 7f32cf0df640 1 -- 172.21.15.84:0/1673034241 shutdown_connections 2023-10-16T11:06:05.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.014+0000 7f32cf0df640 1 -- 172.21.15.84:0/1673034241 wait complete. 2023-10-16T11:06:05.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.016+0000 7f32cf0df640 1 Processor -- start 2023-10-16T11:06:05.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.016+0000 7f32cf0df640 1 -- start start 2023-10-16T11:06:05.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.017+0000 7f32cf0df640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b4003ee0 0x7f32c8157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.017+0000 7f32cf0df640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32c81ee650 0x7f32c80b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.017+0000 7f32cf0df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c8155cd0 0x7f32c80c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.017+0000 7f32cf0df640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c80c6820 con 0x7f32c81ee650 2023-10-16T11:06:05.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.017+0000 7f32cf0df640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c80c6a90 con 0x7f32c8155cd0 2023-10-16T11:06:05.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.017+0000 7f32cf0df640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c80c6d70 con 0x7f32b4003ee0 2023-10-16T11:06:05.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.017+0000 7f32cd8dc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32c81ee650 0x7f32c80b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:05.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.017+0000 7f32cd8dc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32c81ee650 0x7f32c80b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46984/0 (socket says 172.21.15.84:46984) 2023-10-16T11:06:05.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.017+0000 7f32cd8dc640 1 -- 172.21.15.84:0/2411440458 learned_addr learned my addr 172.21.15.84:0/2411440458 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:05.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.018+0000 7f32ce0dd640 1 --2- 172.21.15.84:0/2411440458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b4003ee0 0x7f32c8157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:05.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.018+0000 7f32ce8de640 1 --2- 172.21.15.84:0/2411440458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c8155cd0 0x7f32c80c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:05.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.018+0000 7f32cd8dc640 1 -- 172.21.15.84:0/2411440458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b4003ee0 msgr2=0x7f32c8157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:05.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.018+0000 7f32cd8dc640 1 --2- 172.21.15.84:0/2411440458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b4003ee0 0x7f32c8157700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.018+0000 7f32cd8dc640 1 -- 172.21.15.84:0/2411440458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c8155cd0 msgr2=0x7f32c80c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:05.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.018+0000 7f32cd8dc640 1 --2- 172.21.15.84:0/2411440458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c8155cd0 0x7f32c80c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.018+0000 7f32cd8dc640 1 -- 172.21.15.84:0/2411440458 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c80c7050 con 0x7f32c81ee650 2023-10-16T11:06:05.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.018+0000 7f32ce8de640 1 --2- 172.21.15.84:0/2411440458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c8155cd0 0x7f32c80c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:05.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.018+0000 7f32cd8dc640 1 --2- 172.21.15.84:0/2411440458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32c81ee650 0x7f32c80b1b20 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f32b802bd00 tx=0x7f32b8001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:05.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.018+0000 7f32beffd640 1 -- 172.21.15.84:0/2411440458 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8005270 con 0x7f32c81ee650 2023-10-16T11:06:05.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.018+0000 7f32beffd640 1 -- 172.21.15.84:0/2411440458 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b80053d0 con 0x7f32c81ee650 2023-10-16T11:06:05.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.018+0000 7f32beffd640 1 -- 172.21.15.84:0/2411440458 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8033920 con 0x7f32c81ee650 2023-10-16T11:06:05.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.018+0000 7f32cf0df640 1 -- 172.21.15.84:0/2411440458 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c80c7330 con 0x7f32c81ee650 2023-10-16T11:06:05.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.019+0000 7f32cf0df640 1 -- 172.21.15.84:0/2411440458 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c8157e20 con 0x7f32c81ee650 2023-10-16T11:06:05.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.020+0000 7f32cf0df640 1 -- 172.21.15.84:0/2411440458 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c804e9b0 con 0x7f32c81ee650 2023-10-16T11:06:05.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.021+0000 7f32beffd640 1 -- 172.21.15.84:0/2411440458 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 99) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f32b803a050 con 0x7f32c81ee650 2023-10-16T11:06:05.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.021+0000 7f32beffd640 1 --2- 172.21.15.84:0/2411440458 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f32ac06af90 0x7f32ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.022+0000 7f32beffd640 1 -- 172.21.15.84:0/2411440458 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f32b80aadf0 con 0x7f32c81ee650 2023-10-16T11:06:05.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.022+0000 7f32ce0dd640 1 -- 172.21.15.84:0/2411440458 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f32ac06af90 msgr2=0x7f32ac06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1697458478 2023-10-16T11:06:05.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.022+0000 7f32ce0dd640 1 --2- 172.21.15.84:0/2411440458 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f32ac06af90 0x7f32ac06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:06:05.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.029+0000 7f32beffd640 1 -- 172.21.15.84:0/2411440458 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f32b8075bb0 con 0x7f32c81ee650 2023-10-16T11:06:05.050 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:05.049+0000 7fa2c2012040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:06:05.149 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:05.147+0000 7fbc4bc32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:06:05.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.223+0000 7f32ce0dd640 1 -- 172.21.15.84:0/2411440458 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f32ac06af90 msgr2=0x7f32ac06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1697458478 2023-10-16T11:06:05.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.223+0000 7f32cf0df640 1 -- 172.21.15.84:0/2411440458 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32c8157e20 con 0x7f32c81ee650 2023-10-16T11:06:05.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.223+0000 7f32ce0dd640 1 --2- 172.21.15.84:0/2411440458 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f32ac06af90 0x7f32ac06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:06:05.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.225+0000 7f32beffd640 1 -- 172.21.15.84:0/2411440458 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+17100 (secure 0 0 0) 0x7f32c8157e20 con 0x7f32c81ee650 2023-10-16T11:06:05.227 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:05.227 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":172,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:05:57.603936+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:05:41.217474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-16T11:05:42.387743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/3425147335":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/1161603770":"2023-10-17T11:05:57.603800+0000","172.21.15.84:6833/2442400184":"2023-10-17T11:05:57.603800+0000","172.21.15.84:6832/2442400184":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/3633178028":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/1914327915":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/2602045766":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/3065897528":"2023-10-17T11:05:57.603800+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:06:05.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.227+0000 7f32cf0df640 1 -- 172.21.15.84:0/2411440458 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f32ac06af90 msgr2=0x7f32ac06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:06:05.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.227+0000 7f32cf0df640 1 --2- 172.21.15.84:0/2411440458 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f32ac06af90 0x7f32ac06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.227+0000 7f32cf0df640 1 -- 172.21.15.84:0/2411440458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32c81ee650 msgr2=0x7f32c80b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:05.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.227+0000 7f32cf0df640 1 --2- 172.21.15.84:0/2411440458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32c81ee650 0x7f32c80b1b20 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f32b802bd00 tx=0x7f32b8001230 comp rx=0 tx=0).stop 2023-10-16T11:06:05.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.227+0000 7f32cf0df640 1 -- 172.21.15.84:0/2411440458 shutdown_connections 2023-10-16T11:06:05.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.227+0000 7f32cf0df640 1 --2- 172.21.15.84:0/2411440458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c8155cd0 0x7f32c80c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.227+0000 7f32cf0df640 1 --2- 172.21.15.84:0/2411440458 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f32ac06af90 0x7f32ac06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.227+0000 7f32cf0df640 1 --2- 172.21.15.84:0/2411440458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32c81ee650 0x7f32c80b1b20 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.227+0000 7f32cf0df640 1 --2- 172.21.15.84:0/2411440458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32b4003ee0 0x7f32c8157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.227+0000 7f32cf0df640 1 -- 172.21.15.84:0/2411440458 >> 172.21.15.84:0/2411440458 conn(0x7f32c80ce6a0 msgr2=0x7f32c806aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:05.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.227+0000 7f32cf0df640 1 -- 172.21.15.84:0/2411440458 shutdown_connections 2023-10-16T11:06:05.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.227+0000 7f32cf0df640 1 -- 172.21.15.84:0/2411440458 wait complete. 2023-10-16T11:06:05.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:05:41.217474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:05:42.387743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:06:05.249 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:06:05.345 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:05.343+0000 7fa2c2012040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:06:05.402 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:05.401+0000 7fa2c2012040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:06:05.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.445+0000 7f83390cf640 1 Processor -- start 2023-10-16T11:06:05.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.445+0000 7f83390cf640 1 -- start start 2023-10-16T11:06:05.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.446+0000 7f83390cf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f833415d030 0x7f83340d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.446+0000 7f83390cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83340cd260 0x7f83340cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.446+0000 7f83390cf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83340b5d90 con 0x7f833415d410 2023-10-16T11:06:05.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.447+0000 7f83390cf640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83340d2b50 con 0x7f83340cd260 2023-10-16T11:06:05.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.447+0000 7f83390cf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83340cdbf0 con 0x7f833415d030 2023-10-16T11:06:05.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.447+0000 7f833359e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83340cd260 0x7f83340cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:05.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.447+0000 7f833359e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83340cd260 0x7f83340cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46990/0 (socket says 172.21.15.84:46990) 2023-10-16T11:06:05.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.447+0000 7f833359e640 1 -- 172.21.15.84:0/1400963080 learned_addr learned my addr 172.21.15.84:0/1400963080 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:05.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.447+0000 7f8332d9d640 1 --2- 172.21.15.84:0/1400963080 >> v2:172.21.15.157:3300/0 conn(0x7f833415d030 0x7f83340d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:05.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.447+0000 7f833359e640 1 -- 172.21.15.84:0/1400963080 >> v2:172.21.15.157:3300/0 conn(0x7f833415d030 msgr2=0x7f83340d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:05.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.447+0000 7f833359e640 1 --2- 172.21.15.84:0/1400963080 >> v2:172.21.15.157:3300/0 conn(0x7f833415d030 0x7f83340d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.447+0000 7f833359e640 1 -- 172.21.15.84:0/1400963080 >> v1:172.21.15.157:6789/0 conn(0x7f833415d410 legacy=0x7f8334161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:05.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.447+0000 7f833359e640 1 -- 172.21.15.84:0/1400963080 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8334160fc0 con 0x7f83340cd260 2023-10-16T11:06:05.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.448+0000 7f833359e640 1 --2- 172.21.15.84:0/1400963080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83340cd260 0x7f83340cd6b0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f833406cd10 tx=0x7f832c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d03d51bf96395 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:05.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.449+0000 7f8331d9b640 1 -- 172.21.15.84:0/1400963080 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c02cb20 con 0x7f83340cd260 2023-10-16T11:06:05.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.449+0000 7f8331d9b640 1 -- 172.21.15.84:0/1400963080 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c02cc80 con 0x7f83340cd260 2023-10-16T11:06:05.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.449+0000 7f8331d9b640 1 -- 172.21.15.84:0/1400963080 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c0343f0 con 0x7f83340cd260 2023-10-16T11:06:05.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.449+0000 7f83390cf640 1 -- 172.21.15.84:0/1400963080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83340cd260 msgr2=0x7f83340cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:05.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.449+0000 7f83390cf640 1 --2- 172.21.15.84:0/1400963080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83340cd260 0x7f83340cd6b0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f833406cd10 tx=0x7f832c02bf80 comp rx=0 tx=0).stop 2023-10-16T11:06:05.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.449+0000 7f83390cf640 1 -- 172.21.15.84:0/1400963080 shutdown_connections 2023-10-16T11:06:05.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.449+0000 7f83390cf640 1 --2- 172.21.15.84:0/1400963080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83340cd260 0x7f83340cd6b0 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.449+0000 7f83390cf640 1 --2- 172.21.15.84:0/1400963080 >> v2:172.21.15.157:3300/0 conn(0x7f833415d030 0x7f83340d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.449+0000 7f83390cf640 1 -- 172.21.15.84:0/1400963080 >> 172.21.15.84:0/1400963080 conn(0x7f83340cf720 msgr2=0x7f83340cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:05.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.450+0000 7f83390cf640 1 -- 172.21.15.84:0/1400963080 shutdown_connections 2023-10-16T11:06:05.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.450+0000 7f83390cf640 1 -- 172.21.15.84:0/1400963080 wait complete. 2023-10-16T11:06:05.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.451+0000 7f83390cf640 1 Processor -- start 2023-10-16T11:06:05.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.451+0000 7f83390cf640 1 -- start start 2023-10-16T11:06:05.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.452+0000 7f83390cf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83340cd260 0x7f833415d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.452+0000 7f83390cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f833415d030 0x7f8334150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.452+0000 7f83390cf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f833415d410 0x7f833414fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.452+0000 7f83390cf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8334151180 con 0x7f833415d030 2023-10-16T11:06:05.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.452+0000 7f83390cf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83341513f0 con 0x7f833415d410 2023-10-16T11:06:05.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.452+0000 7f83390cf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833415f4d0 con 0x7f83340cd260 2023-10-16T11:06:05.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.453+0000 7f833259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f833415d030 0x7f8334150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:05.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.453+0000 7f833259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f833415d030 0x7f8334150c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47004/0 (socket says 172.21.15.84:47004) 2023-10-16T11:06:05.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.453+0000 7f833259c640 1 -- 172.21.15.84:0/2533880692 learned_addr learned my addr 172.21.15.84:0/2533880692 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:05.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.453+0000 7f8332d9d640 1 --2- 172.21.15.84:0/2533880692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83340cd260 0x7f833415d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:05.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.453+0000 7f833359e640 1 --2- 172.21.15.84:0/2533880692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f833415d410 0x7f833414fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:05.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.453+0000 7f833259c640 1 -- 172.21.15.84:0/2533880692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83340cd260 msgr2=0x7f833415d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:05.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.453+0000 7f833259c640 1 --2- 172.21.15.84:0/2533880692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83340cd260 0x7f833415d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.453+0000 7f833259c640 1 -- 172.21.15.84:0/2533880692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f833415d410 msgr2=0x7f833414fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:05.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.453+0000 7f833259c640 1 --2- 172.21.15.84:0/2533880692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f833415d410 0x7f833414fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.453+0000 7f833259c640 1 -- 172.21.15.84:0/2533880692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c009b30 con 0x7f833415d030 2023-10-16T11:06:05.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.453+0000 7f833359e640 1 --2- 172.21.15.84:0/2533880692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f833415d410 0x7f833414fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:05.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.453+0000 7f833259c640 1 --2- 172.21.15.84:0/2533880692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f833415d030 0x7f8334150c40 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f832800b3c0 tx=0x7f832800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:05.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.453+0000 7f831b7fe640 1 -- 172.21.15.84:0/2533880692 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328011830 con 0x7f833415d030 2023-10-16T11:06:05.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.454+0000 7f831b7fe640 1 -- 172.21.15.84:0/2533880692 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8328011990 con 0x7f833415d030 2023-10-16T11:06:05.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.454+0000 7f831b7fe640 1 -- 172.21.15.84:0/2533880692 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328011d80 con 0x7f833415d030 2023-10-16T11:06:05.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.454+0000 7f83390cf640 1 -- 172.21.15.84:0/2533880692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833415f7b0 con 0x7f833415d030 2023-10-16T11:06:05.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.454+0000 7f83390cf640 1 -- 172.21.15.84:0/2533880692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833414ff70 con 0x7f833415d030 2023-10-16T11:06:05.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.455+0000 7f83390cf640 1 -- 172.21.15.84:0/2533880692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f833404e160 con 0x7f833415d030 2023-10-16T11:06:05.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.456+0000 7f831b7fe640 1 -- 172.21.15.84:0/2533880692 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 99) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f832801b460 con 0x7f833415d030 2023-10-16T11:06:05.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.456+0000 7f831b7fe640 1 --2- 172.21.15.84:0/2533880692 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f832006af90 0x7f832006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.457+0000 7f8332d9d640 1 -- 172.21.15.84:0/2533880692 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f832006af90 msgr2=0x7f832006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1697458478 2023-10-16T11:06:05.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.457+0000 7f831b7fe640 1 -- 172.21.15.84:0/2533880692 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f8328089470 con 0x7f833415d030 2023-10-16T11:06:05.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.457+0000 7f8332d9d640 1 --2- 172.21.15.84:0/2533880692 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f832006af90 0x7f832006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:06:05.466 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:05.457+0000 7fbc4bc32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:06:05.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.461+0000 7f831b7fe640 1 -- 172.21.15.84:0/2533880692 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8328054290 con 0x7f833415d030 2023-10-16T11:06:05.518 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:05.516+0000 7fa2c2012040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:06:05.519 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:05.517+0000 7fbc4bc32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:06:05.637 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:05.635+0000 7fbc4bc32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:06:05.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.657+0000 7f831b7fe640 1 -- 172.21.15.84:0/2533880692 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 100) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f8328087810 con 0x7f833415d030 2023-10-16T11:06:05.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.658+0000 7f8332d9d640 1 -- 172.21.15.84:0/2533880692 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f832006af90 msgr2=0x7f832006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1697458478 2023-10-16T11:06:05.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.658+0000 7f8332d9d640 1 --2- 172.21.15.84:0/2533880692 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f832006af90 0x7f832006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:06:05.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.658+0000 7f831b7fe640 1 -- 172.21.15.84:0/2533880692 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f832006af90 msgr2=0x7f832006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:06:05.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.658+0000 7f831b7fe640 1 --2- 172.21.15.84:0/2533880692 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f832006af90 0x7f832006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.659+0000 7f83390cf640 1 -- 172.21.15.84:0/2533880692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83340ce620 con 0x7f833415d030 2023-10-16T11:06:05.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.659+0000 7f831b7fe640 1 -- 172.21.15.84:0/2533880692 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v173) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83280724c0 con 0x7f833415d030 2023-10-16T11:06:05.662 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:06:05.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.663+0000 7f83390cf640 1 -- 172.21.15.84:0/2533880692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f833415d030 msgr2=0x7f8334150c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:05.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.663+0000 7f83390cf640 1 --2- 172.21.15.84:0/2533880692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f833415d030 0x7f8334150c40 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f832800b3c0 tx=0x7f832800b880 comp rx=0 tx=0).stop 2023-10-16T11:06:05.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.663+0000 7f83390cf640 1 -- 172.21.15.84:0/2533880692 shutdown_connections 2023-10-16T11:06:05.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.663+0000 7f83390cf640 1 --2- 172.21.15.84:0/2533880692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f833415d410 0x7f833414fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.663+0000 7f83390cf640 1 --2- 172.21.15.84:0/2533880692 >> [v2:172.21.15.84:6832/1697458478,v1:172.21.15.84:6833/1697458478] conn(0x7f832006af90 0x7f832006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.663+0000 7f83390cf640 1 --2- 172.21.15.84:0/2533880692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f833415d030 0x7f8334150c40 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.663+0000 7f83390cf640 1 --2- 172.21.15.84:0/2533880692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83340cd260 0x7f833415d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.663+0000 7f83390cf640 1 -- 172.21.15.84:0/2533880692 >> 172.21.15.84:0/2533880692 conn(0x7f83340cf720 msgr2=0x7f833415c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:05.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.663+0000 7f83390cf640 1 -- 172.21.15.84:0/2533880692 shutdown_connections 2023-10-16T11:06:05.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.663+0000 7f83390cf640 1 -- 172.21.15.84:0/2533880692 wait complete. 2023-10-16T11:06:05.682 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:06:05.726 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:05.725+0000 7fbbccfc2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:05.727 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:05.725+0000 7fbbccfc2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:05.727 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:05.725+0000 7fbbccfc2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:05.727 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:05.725+0000 7fbbccfc2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:05.727 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:05.725+0000 7fbbccfc2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:05.728 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:05.726+0000 7fbbd07c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:05.728 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:05.726+0000 7fbbd07c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:05.728 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:05.726+0000 7fbbd07c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:05.729 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:05.726+0000 7fbbd07c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:05.729 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:05.726+0000 7fbbd07c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:05.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.865+0000 7fe5893c0640 1 Processor -- start 2023-10-16T11:06:05.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.865+0000 7fe5893c0640 1 -- start start 2023-10-16T11:06:05.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.866+0000 7fe5893c0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe58415ad70 0x7fe584069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.866+0000 7fe5893c0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe5840c9ca0 0x7fe58406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.866+0000 7fe5893c0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5840cb030 con 0x7fe5840ca3b0 2023-10-16T11:06:05.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.866+0000 7fe5893c0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5840c91c0 con 0x7fe58415ad70 2023-10-16T11:06:05.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.866+0000 7fe5893c0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5840c94a0 con 0x7fe5840c9ca0 2023-10-16T11:06:05.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.867+0000 7fe582ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe58415ad70 0x7fe584069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:05.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.867+0000 7fe582ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe58415ad70 0x7fe584069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56212/0 (socket says 172.21.15.84:56212) 2023-10-16T11:06:05.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.867+0000 7fe582ffd640 1 -- 172.21.15.84:0/2891171853 learned_addr learned my addr 172.21.15.84:0/2891171853 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:05.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.867+0000 7fe582ffd640 1 -- 172.21.15.84:0/2891171853 >> v2:172.21.15.157:3300/0 conn(0x7fe5840c9ca0 msgr2=0x7fe58406a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:05.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.867+0000 7fe582ffd640 1 --2- 172.21.15.84:0/2891171853 >> v2:172.21.15.157:3300/0 conn(0x7fe5840c9ca0 0x7fe58406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.867+0000 7fe582ffd640 1 -- 172.21.15.84:0/2891171853 >> v1:172.21.15.157:6789/0 conn(0x7fe5840ca3b0 legacy=0x7fe5840ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:05.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.867+0000 7fe582ffd640 1 -- 172.21.15.84:0/2891171853 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5840c9780 con 0x7fe58415ad70 2023-10-16T11:06:05.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.869+0000 7fe582ffd640 1 --2- 172.21.15.84:0/2891171853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe58415ad70 0x7fe584069da0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7fe5840ce870 tx=0x7fe57402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f363c58c6c57a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:05.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.869+0000 7fe581ffb640 1 -- 172.21.15.84:0/2891171853 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574031870 con 0x7fe58415ad70 2023-10-16T11:06:05.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.869+0000 7fe581ffb640 1 -- 172.21.15.84:0/2891171853 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5740319d0 con 0x7fe58415ad70 2023-10-16T11:06:05.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.869+0000 7fe581ffb640 1 -- 172.21.15.84:0/2891171853 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574031d60 con 0x7fe58415ad70 2023-10-16T11:06:05.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.869+0000 7fe5893c0640 1 -- 172.21.15.84:0/2891171853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe58415ad70 msgr2=0x7fe584069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:05.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.869+0000 7fe5893c0640 1 --2- 172.21.15.84:0/2891171853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe58415ad70 0x7fe584069da0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7fe5840ce870 tx=0x7fe57402b910 comp rx=0 tx=0).stop 2023-10-16T11:06:05.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.870+0000 7fe5893c0640 1 -- 172.21.15.84:0/2891171853 shutdown_connections 2023-10-16T11:06:05.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.870+0000 7fe5893c0640 1 --2- 172.21.15.84:0/2891171853 >> v2:172.21.15.157:3300/0 conn(0x7fe5840c9ca0 0x7fe58406a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.870+0000 7fe5893c0640 1 --2- 172.21.15.84:0/2891171853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe58415ad70 0x7fe584069da0 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.870+0000 7fe5893c0640 1 -- 172.21.15.84:0/2891171853 >> 172.21.15.84:0/2891171853 conn(0x7fe5840ce9b0 msgr2=0x7fe5840cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:05.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.870+0000 7fe5893c0640 1 -- 172.21.15.84:0/2891171853 shutdown_connections 2023-10-16T11:06:05.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.870+0000 7fe5893c0640 1 -- 172.21.15.84:0/2891171853 wait complete. 2023-10-16T11:06:05.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.871+0000 7fe5893c0640 1 Processor -- start 2023-10-16T11:06:05.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.871+0000 7fe5893c0640 1 -- start start 2023-10-16T11:06:05.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.872+0000 7fe5893c0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5840c9ca0 0x7fe58414af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.872+0000 7fe5893c0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5840ca3b0 0x7fe5840ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.872+0000 7fe5893c0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe58415ad70 0x7fe5840acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:05.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.872+0000 7fe5893c0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58414e5a0 con 0x7fe58415ad70 2023-10-16T11:06:05.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.872+0000 7fe5893c0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58414d990 con 0x7fe5840ca3b0 2023-10-16T11:06:05.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.872+0000 7fe5893c0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58414dc70 con 0x7fe5840c9ca0 2023-10-16T11:06:05.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.873+0000 7fe5837fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe58415ad70 0x7fe5840acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:05.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.873+0000 7fe5837fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe58415ad70 0x7fe5840acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56216/0 (socket says 172.21.15.84:56216) 2023-10-16T11:06:05.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.873+0000 7fe5837fe640 1 -- 172.21.15.84:0/371936460 learned_addr learned my addr 172.21.15.84:0/371936460 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:05.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.873+0000 7fe582ffd640 1 --2- 172.21.15.84:0/371936460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5840c9ca0 0x7fe58414af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:05.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.873+0000 7fe5827fc640 1 --2- 172.21.15.84:0/371936460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5840ca3b0 0x7fe5840ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:05.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.873+0000 7fe5837fe640 1 -- 172.21.15.84:0/371936460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5840c9ca0 msgr2=0x7fe58414af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:05.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.873+0000 7fe5837fe640 1 --2- 172.21.15.84:0/371936460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5840c9ca0 0x7fe58414af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.873+0000 7fe5837fe640 1 -- 172.21.15.84:0/371936460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5840ca3b0 msgr2=0x7fe5840ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:05.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.873+0000 7fe5837fe640 1 --2- 172.21.15.84:0/371936460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5840ca3b0 0x7fe5840ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:05.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.873+0000 7fe5837fe640 1 -- 172.21.15.84:0/371936460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5740094d0 con 0x7fe58415ad70 2023-10-16T11:06:05.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.873+0000 7fe5837fe640 1 --2- 172.21.15.84:0/371936460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe58415ad70 0x7fe5840acdd0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00c780 tx=0x7fe57c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:05.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.874+0000 7fe56b7fe640 1 -- 172.21.15.84:0/371936460 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c003d80 con 0x7fe58415ad70 2023-10-16T11:06:05.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.874+0000 7fe56b7fe640 1 -- 172.21.15.84:0/371936460 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c00ce90 con 0x7fe58415ad70 2023-10-16T11:06:05.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.874+0000 7fe5893c0640 1 -- 172.21.15.84:0/371936460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58414df50 con 0x7fe58415ad70 2023-10-16T11:06:05.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.874+0000 7fe56b7fe640 1 -- 172.21.15.84:0/371936460 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0052b0 con 0x7fe58415ad70 2023-10-16T11:06:05.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.874+0000 7fe5893c0640 1 -- 172.21.15.84:0/371936460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58414b470 con 0x7fe58415ad70 2023-10-16T11:06:05.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.875+0000 7fe5893c0640 1 -- 172.21.15.84:0/371936460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe58414e750 con 0x7fe58415ad70 2023-10-16T11:06:05.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.876+0000 7fe56b7fe640 1 -- 172.21.15.84:0/371936460 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 100) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe57c005410 con 0x7fe58415ad70 2023-10-16T11:06:05.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.877+0000 7fe56b7fe640 1 -- 172.21.15.84:0/371936460 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6425+0+0 (secure 0 0 0) 0x7fe57c088c50 con 0x7fe58415ad70 2023-10-16T11:06:05.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:05.881+0000 7fe56b7fe640 1 -- 172.21.15.84:0/371936460 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe58414e750 con 0x7fe58415ad70 2023-10-16T11:06:06.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.057+0000 7fe5893c0640 1 -- 172.21.15.84:0/371936460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe58414e750 con 0x7fe58415ad70 2023-10-16T11:06:06.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.058+0000 7fe56b7fe640 1 -- 172.21.15.84:0/371936460 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v173) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe58414e750 con 0x7fe58415ad70 2023-10-16T11:06:06.060 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:06:06.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.060+0000 7fe5893c0640 1 -- 172.21.15.84:0/371936460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe58415ad70 msgr2=0x7fe5840acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:06.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.060+0000 7fe5893c0640 1 --2- 172.21.15.84:0/371936460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe58415ad70 0x7fe5840acdd0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00c780 tx=0x7fe57c00cc40 comp rx=0 tx=0).stop 2023-10-16T11:06:06.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.060+0000 7fe5893c0640 1 -- 172.21.15.84:0/371936460 shutdown_connections 2023-10-16T11:06:06.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.060+0000 7fe5893c0640 1 --2- 172.21.15.84:0/371936460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe58415ad70 0x7fe5840acdd0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:06.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.060+0000 7fe5893c0640 1 --2- 172.21.15.84:0/371936460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5840ca3b0 0x7fe5840ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:06.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.060+0000 7fe5893c0640 1 --2- 172.21.15.84:0/371936460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5840c9ca0 0x7fe58414af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:06.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.060+0000 7fe5893c0640 1 -- 172.21.15.84:0/371936460 >> 172.21.15.84:0/371936460 conn(0x7fe5840ce9b0 msgr2=0x7fe5840ca840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:06.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.061+0000 7fe5893c0640 1 -- 172.21.15.84:0/371936460 shutdown_connections 2023-10-16T11:06:06.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.061+0000 7fe5893c0640 1 -- 172.21.15.84:0/371936460 wait complete. 2023-10-16T11:06:06.078 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:06:06.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.331+0000 7f77d844b640 1 Processor -- start 2023-10-16T11:06:06.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.331+0000 7f77d844b640 1 -- start start 2023-10-16T11:06:06.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.333+0000 7f77d844b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77d0159ab0 0x7f77d00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:06.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.333+0000 7f77d844b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d0070360 con 0x7f77d0159e90 2023-10-16T11:06:06.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.333+0000 7f77d844b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d00cd1f0 con 0x7f77d006c140 2023-10-16T11:06:06.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.333+0000 7f77d844b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d015d890 con 0x7f77d0159ab0 2023-10-16T11:06:06.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.333+0000 7f77d59e6640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f77d0159e90 0x7f77d006fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55290/0 (socket says 172.21.15.84:55290) 2023-10-16T11:06:06.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.333+0000 7f77d59e6640 1 -- 172.21.15.84:0/342725428 learned_addr learned my addr 172.21.15.84:0/342725428 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:06.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.333+0000 7f77d51e5640 1 -- 172.21.15.84:0/342725428 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 911386881 0 0) 0x7f77d0070360 con 0x7f77d0159e90 2023-10-16T11:06:06.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.334+0000 7f77d61e7640 1 --2- 172.21.15.84:0/342725428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77d0159ab0 0x7f77d00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:06.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.334+0000 7f77d51e5640 1 -- 172.21.15.84:0/342725428 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77b8003520 con 0x7f77d0159e90 2023-10-16T11:06:06.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.334+0000 7f77d51e5640 1 -- 172.21.15.84:0/342725428 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2141692412 0 0) 0x7f77b8003520 con 0x7f77d0159e90 2023-10-16T11:06:06.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.334+0000 7f77d51e5640 1 -- 172.21.15.84:0/342725428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77d0159ab0 msgr2=0x7f77d00ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:06.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.334+0000 7f77d51e5640 1 --2- 172.21.15.84:0/342725428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77d0159ab0 0x7f77d00ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:06.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.334+0000 7f77d51e5640 1 -- 172.21.15.84:0/342725428 >> v1:172.21.15.157:6789/0 conn(0x7f77d006c140 legacy=0x7f77d015dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:06.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.334+0000 7f77d51e5640 1 -- 172.21.15.84:0/342725428 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d00cd4d0 con 0x7f77d0159e90 2023-10-16T11:06:06.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.334+0000 7f77d51e5640 1 -- 172.21.15.84:0/342725428 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f77cc001670 con 0x7f77d0159e90 2023-10-16T11:06:06.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.334+0000 7f77d51e5640 1 -- 172.21.15.84:0/342725428 >> v1:172.21.15.84:6789/0 conn(0x7f77d0159e90 legacy=0x7f77d006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:06.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.334+0000 7f77d51e5640 1 --2- 172.21.15.84:0/342725428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77d0159ab0 0x7f77b80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:06.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.334+0000 7f77d51e5640 1 --2- 172.21.15.84:0/342725428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b8003f20 0x7f77b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:06.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.334+0000 7f77d51e5640 1 --2- 172.21.15.84:0/342725428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77b8004d00 0x7f77b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:06.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.334+0000 7f77d51e5640 1 -- 172.21.15.84:0/342725428 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d015d890 con 0x7f77b8004d00 2023-10-16T11:06:06.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.334+0000 7f77d51e5640 1 -- 172.21.15.84:0/342725428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b800bbc0 con 0x7f77b8003f20 2023-10-16T11:06:06.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.334+0000 7f77d51e5640 1 -- 172.21.15.84:0/342725428 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b800bd80 con 0x7f77d0159ab0 2023-10-16T11:06:06.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.335+0000 7f77d69e8640 1 --2- 172.21.15.84:0/342725428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77b8004d00 0x7f77b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:06.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.335+0000 7f77d61e7640 1 --2- 172.21.15.84:0/342725428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77d0159ab0 0x7f77b80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:06.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.335+0000 7f77d59e6640 1 --2- 172.21.15.84:0/342725428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b8003f20 0x7f77b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:06.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.335+0000 7f77d69e8640 1 -- 172.21.15.84:0/342725428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77d0159ab0 msgr2=0x7f77b80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:06.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.335+0000 7f77d69e8640 1 --2- 172.21.15.84:0/342725428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77d0159ab0 0x7f77b80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:06.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.335+0000 7f77d69e8640 1 -- 172.21.15.84:0/342725428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b8003f20 msgr2=0x7f77b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:06.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.335+0000 7f77d69e8640 1 --2- 172.21.15.84:0/342725428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b8003f20 0x7f77b80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:06.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.335+0000 7f77d69e8640 1 -- 172.21.15.84:0/342725428 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b8003520 con 0x7f77b8004d00 2023-10-16T11:06:06.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.335+0000 7f77d59e6640 1 --2- 172.21.15.84:0/342725428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b8003f20 0x7f77b80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:06.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.336+0000 7f77d69e8640 1 --2- 172.21.15.84:0/342725428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77b8004d00 0x7f77b8005180 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f77c8001e50 tx=0x7f77b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b4a754add996ef server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:06.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.336+0000 7f77d51e5640 1 -- 172.21.15.84:0/342725428 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8003b70 con 0x7f77b8004d00 2023-10-16T11:06:06.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.336+0000 7f77d51e5640 1 -- 172.21.15.84:0/342725428 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c8003cd0 con 0x7f77b8004d00 2023-10-16T11:06:06.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.336+0000 7f77d51e5640 1 -- 172.21.15.84:0/342725428 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8005310 con 0x7f77b8004d00 2023-10-16T11:06:06.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.337+0000 7f77d844b640 1 -- 172.21.15.84:0/342725428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77b8004d00 msgr2=0x7f77b8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:06.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.337+0000 7f77d844b640 1 --2- 172.21.15.84:0/342725428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77b8004d00 0x7f77b8005180 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f77c8001e50 tx=0x7f77b80057f0 comp rx=0 tx=0).stop 2023-10-16T11:06:06.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.337+0000 7f77d69e8640 1 -- 172.21.15.84:0/342725428 reap_dead start 2023-10-16T11:06:06.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.337+0000 7f77d844b640 1 -- 172.21.15.84:0/342725428 shutdown_connections 2023-10-16T11:06:06.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.337+0000 7f77d844b640 1 -- 172.21.15.84:0/342725428 >> 172.21.15.84:0/342725428 conn(0x7f77d00ce9b0 msgr2=0x7f77d00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:06.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.337+0000 7f77d844b640 1 -- 172.21.15.84:0/342725428 shutdown_connections 2023-10-16T11:06:06.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.337+0000 7f77d844b640 1 -- 172.21.15.84:0/342725428 wait complete. 2023-10-16T11:06:06.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.339+0000 7f77d844b640 1 Processor -- start 2023-10-16T11:06:06.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.339+0000 7f77d844b640 1 -- start start 2023-10-16T11:06:06.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.340+0000 7f77d844b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77d014bd90 0x7f77d01e39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:06.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.340+0000 7f77d844b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77d00c8070 0x7f77d01e4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:06.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.340+0000 7f77d844b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77d01e4d50 0x7f77d01e2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:06.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.340+0000 7f77d844b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d01e51e0 con 0x7f77d00c8070 2023-10-16T11:06:06.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.340+0000 7f77d844b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d00c6610 con 0x7f77d01e4d50 2023-10-16T11:06:06.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.340+0000 7f77d844b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d00c68f0 con 0x7f77d014bd90 2023-10-16T11:06:06.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.340+0000 7f77d59e6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77d00c8070 0x7f77d01e4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:06.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.340+0000 7f77d59e6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77d00c8070 0x7f77d01e4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56234/0 (socket says 172.21.15.84:56234) 2023-10-16T11:06:06.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.340+0000 7f77d59e6640 1 -- 172.21.15.84:0/1868087771 learned_addr learned my addr 172.21.15.84:0/1868087771 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:06.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.340+0000 7f77d69e8640 1 --2- 172.21.15.84:0/1868087771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77d01e4d50 0x7f77d01e2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:06.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.340+0000 7f77d61e7640 1 --2- 172.21.15.84:0/1868087771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77d014bd90 0x7f77d01e39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:06.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.340+0000 7f77d59e6640 1 -- 172.21.15.84:0/1868087771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77d014bd90 msgr2=0x7f77d01e39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:06.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.340+0000 7f77d59e6640 1 --2- 172.21.15.84:0/1868087771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77d014bd90 0x7f77d01e39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:06.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.340+0000 7f77d59e6640 1 -- 172.21.15.84:0/1868087771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77d01e4d50 msgr2=0x7f77d01e2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:06.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.340+0000 7f77d59e6640 1 --2- 172.21.15.84:0/1868087771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77d01e4d50 0x7f77d01e2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:06.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.340+0000 7f77d59e6640 1 -- 172.21.15.84:0/1868087771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d00c6bd0 con 0x7f77d00c8070 2023-10-16T11:06:06.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.340+0000 7f77d69e8640 1 --2- 172.21.15.84:0/1868087771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77d01e4d50 0x7f77d01e2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:06.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.341+0000 7f77d59e6640 1 --2- 172.21.15.84:0/1868087771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77d00c8070 0x7f77d01e4810 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00db10 tx=0x7f77cc001e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:06.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.341+0000 7f77c6ffd640 1 -- 172.21.15.84:0/1868087771 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc004cb0 con 0x7f77d00c8070 2023-10-16T11:06:06.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.341+0000 7f77c6ffd640 1 -- 172.21.15.84:0/1868087771 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77cc004e10 con 0x7f77d00c8070 2023-10-16T11:06:06.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.341+0000 7f77c6ffd640 1 -- 172.21.15.84:0/1868087771 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc007310 con 0x7f77d00c8070 2023-10-16T11:06:06.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.341+0000 7f77d844b640 1 -- 172.21.15.84:0/1868087771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d00c6eb0 con 0x7f77d00c8070 2023-10-16T11:06:06.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.341+0000 7f77d844b640 1 -- 172.21.15.84:0/1868087771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d00c73d0 con 0x7f77d00c8070 2023-10-16T11:06:06.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.342+0000 7f77d844b640 1 -- 172.21.15.84:0/1868087771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d00cdda0 con 0x7f77d00c8070 2023-10-16T11:06:06.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.343+0000 7f77c6ffd640 1 -- 172.21.15.84:0/1868087771 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 100) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f77cc007470 con 0x7f77d00c8070 2023-10-16T11:06:06.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.344+0000 7f77c6ffd640 1 -- 172.21.15.84:0/1868087771 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f77cc088df0 con 0x7f77d00c8070 2023-10-16T11:06:06.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.349+0000 7f77c6ffd640 1 -- 172.21.15.84:0/1868087771 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f77cc054020 con 0x7f77d00c8070 2023-10-16T11:06:06.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.530+0000 7f77d844b640 1 -- 172.21.15.84:0/1868087771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77d004e160 con 0x7f77d00c8070 2023-10-16T11:06:06.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.531+0000 7f77c6ffd640 1 -- 172.21.15.84:0/1868087771 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v173) v1 ==== 100+0+11 (secure 0 0 0) 0x7f77cc0190a0 con 0x7f77d00c8070 2023-10-16T11:06:06.533 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:06:06.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.534+0000 7f77d844b640 1 -- 172.21.15.84:0/1868087771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77d00c8070 msgr2=0x7f77d01e4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:06.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.534+0000 7f77d844b640 1 --2- 172.21.15.84:0/1868087771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77d00c8070 0x7f77d01e4810 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00db10 tx=0x7f77cc001e00 comp rx=0 tx=0).stop 2023-10-16T11:06:06.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.534+0000 7f77d844b640 1 -- 172.21.15.84:0/1868087771 shutdown_connections 2023-10-16T11:06:06.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.534+0000 7f77d844b640 1 --2- 172.21.15.84:0/1868087771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77d01e4d50 0x7f77d01e2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:06.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.534+0000 7f77d844b640 1 --2- 172.21.15.84:0/1868087771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77d00c8070 0x7f77d01e4810 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:06.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.534+0000 7f77d844b640 1 --2- 172.21.15.84:0/1868087771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77d014bd90 0x7f77d01e39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:06.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.534+0000 7f77d844b640 1 -- 172.21.15.84:0/1868087771 >> 172.21.15.84:0/1868087771 conn(0x7f77d00ce9b0 msgr2=0x7f77d006b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:06.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.534+0000 7f77d844b640 1 -- 172.21.15.84:0/1868087771 shutdown_connections 2023-10-16T11:06:06.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.534+0000 7f77d844b640 1 -- 172.21.15.84:0/1868087771 wait complete. 2023-10-16T11:06:06.553 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:06:06.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.766+0000 7fa9ebf61640 1 Processor -- start 2023-10-16T11:06:06.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.766+0000 7fa9ebf61640 1 -- start start 2023-10-16T11:06:06.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.767+0000 7fa9ebf61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9e40ce390 0x7fa9e415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:06.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.767+0000 7fa9ebf61640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e415ed60 con 0x7fa9e415ae80 2023-10-16T11:06:06.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.767+0000 7fa9ebf61640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e40707e0 con 0x7fa9e415aaa0 2023-10-16T11:06:06.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.767+0000 7fa9ebf61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e406b810 con 0x7fa9e40ce390 2023-10-16T11:06:06.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.768+0000 7fa9e94fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa9e415ae80 0x7fa9e4070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55300/0 (socket says 172.21.15.84:55300) 2023-10-16T11:06:06.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.768+0000 7fa9e94fc640 1 -- 172.21.15.84:0/394175848 learned_addr learned my addr 172.21.15.84:0/394175848 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:06.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.768+0000 7fa9ea4fe640 1 --2- 172.21.15.84:0/394175848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9e40ce390 0x7fa9e415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:06.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.768+0000 7fa9e8cfb640 1 -- 172.21.15.84:0/394175848 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605439511 0 0) 0x7fa9e415ed60 con 0x7fa9e415ae80 2023-10-16T11:06:06.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.768+0000 7fa9e8cfb640 1 -- 172.21.15.84:0/394175848 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9d0003520 con 0x7fa9e415ae80 2023-10-16T11:06:06.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e8cfb640 1 -- 172.21.15.84:0/394175848 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 849272008 0 0) 0x7fa9d0003520 con 0x7fa9e415ae80 2023-10-16T11:06:06.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e8cfb640 1 -- 172.21.15.84:0/394175848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9e40ce390 msgr2=0x7fa9e415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:06.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e8cfb640 1 --2- 172.21.15.84:0/394175848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9e40ce390 0x7fa9e415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:06.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e8cfb640 1 -- 172.21.15.84:0/394175848 >> v1:172.21.15.157:6789/0 conn(0x7fa9e415aaa0 legacy=0x7fa9e40cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:06.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e8cfb640 1 -- 172.21.15.84:0/394175848 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e415ef90 con 0x7fa9e415ae80 2023-10-16T11:06:06.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e8cfb640 1 -- 172.21.15.84:0/394175848 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fa9d4001690 con 0x7fa9e415ae80 2023-10-16T11:06:06.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e8cfb640 1 -- 172.21.15.84:0/394175848 >> v1:172.21.15.84:6789/0 conn(0x7fa9e415ae80 legacy=0x7fa9e4070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:06.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e8cfb640 1 --2- 172.21.15.84:0/394175848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9d00039a0 0x7fa9d0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:06.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e8cfb640 1 --2- 172.21.15.84:0/394175848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9d00047c0 0x7fa9d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:06.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e8cfb640 1 --2- 172.21.15.84:0/394175848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9e40ce390 0x7fa9d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:06.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e8cfb640 1 -- 172.21.15.84:0/394175848 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e406b810 con 0x7fa9d00047c0 2023-10-16T11:06:06.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e8cfb640 1 -- 172.21.15.84:0/394175848 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d000bab0 con 0x7fa9d00039a0 2023-10-16T11:06:06.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e8cfb640 1 -- 172.21.15.84:0/394175848 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d000bc70 con 0x7fa9e40ce390 2023-10-16T11:06:06.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e94fc640 1 --2- 172.21.15.84:0/394175848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9d00047c0 0x7fa9d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:06.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e9cfd640 1 --2- 172.21.15.84:0/394175848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9d00039a0 0x7fa9d0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:06.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e94fc640 1 -- 172.21.15.84:0/394175848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9e40ce390 msgr2=0x7fa9d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:06.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e94fc640 1 --2- 172.21.15.84:0/394175848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9e40ce390 0x7fa9d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:06.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e94fc640 1 -- 172.21.15.84:0/394175848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9d00039a0 msgr2=0x7fa9d0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:06.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e94fc640 1 --2- 172.21.15.84:0/394175848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9d00039a0 0x7fa9d0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:06.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.769+0000 7fa9e94fc640 1 -- 172.21.15.84:0/394175848 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d0003520 con 0x7fa9d00047c0 2023-10-16T11:06:06.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.770+0000 7fa9e94fc640 1 --2- 172.21.15.84:0/394175848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9d00047c0 0x7fa9d0004c40 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fa9e415b2f0 tx=0x7fa9d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d100f7a9e91d09d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:06.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.771+0000 7fa9e8cfb640 1 -- 172.21.15.84:0/394175848 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4005ad0 con 0x7fa9d00047c0 2023-10-16T11:06:06.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.771+0000 7fa9e8cfb640 1 -- 172.21.15.84:0/394175848 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d4005c30 con 0x7fa9d00047c0 2023-10-16T11:06:06.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.771+0000 7fa9e8cfb640 1 -- 172.21.15.84:0/394175848 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d402c870 con 0x7fa9d00047c0 2023-10-16T11:06:06.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.771+0000 7fa9ebf61640 1 -- 172.21.15.84:0/394175848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9d00047c0 msgr2=0x7fa9d0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:06.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.771+0000 7fa9ebf61640 1 --2- 172.21.15.84:0/394175848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9d00047c0 0x7fa9d0004c40 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fa9e415b2f0 tx=0x7fa9d00057f0 comp rx=0 tx=0).stop 2023-10-16T11:06:06.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.771+0000 7fa9ea4fe640 1 -- 172.21.15.84:0/394175848 reap_dead start 2023-10-16T11:06:06.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.771+0000 7fa9ebf61640 1 -- 172.21.15.84:0/394175848 shutdown_connections 2023-10-16T11:06:06.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.771+0000 7fa9ebf61640 1 -- 172.21.15.84:0/394175848 >> 172.21.15.84:0/394175848 conn(0x7fa9e40cb410 msgr2=0x7fa9e40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:06.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.771+0000 7fa9ebf61640 1 -- 172.21.15.84:0/394175848 shutdown_connections 2023-10-16T11:06:06.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.771+0000 7fa9ebf61640 1 -- 172.21.15.84:0/394175848 wait complete. 2023-10-16T11:06:06.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.773+0000 7fa9ebf61640 1 Processor -- start 2023-10-16T11:06:06.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.773+0000 7fa9ebf61640 1 -- start start 2023-10-16T11:06:06.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.774+0000 7fa9ebf61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9d00047c0 0x7fa9e415d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:06.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.774+0000 7fa9ebf61640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9e415bc40 0x7fa9e40c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:06.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.774+0000 7fa9ebf61640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9e40b1720 0x7fa9e40c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:06.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.774+0000 7fa9ebf61640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e415cdd0 con 0x7fa9e415bc40 2023-10-16T11:06:06.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.774+0000 7fa9ebf61640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e41583a0 con 0x7fa9e40b1720 2023-10-16T11:06:06.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.774+0000 7fa9ebf61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e40c46f0 con 0x7fa9d00047c0 2023-10-16T11:06:06.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.774+0000 7fa9e94fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9e415bc40 0x7fa9e40c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:06.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.774+0000 7fa9e94fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9e415bc40 0x7fa9e40c41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56256/0 (socket says 172.21.15.84:56256) 2023-10-16T11:06:06.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.774+0000 7fa9e94fc640 1 -- 172.21.15.84:0/2353445616 learned_addr learned my addr 172.21.15.84:0/2353445616 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:06.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.775+0000 7fa9ea4fe640 1 --2- 172.21.15.84:0/2353445616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9e40b1720 0x7fa9e40c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:06.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.775+0000 7fa9e94fc640 1 -- 172.21.15.84:0/2353445616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9d00047c0 msgr2=0x7fa9e415d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:06.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.775+0000 7fa9e94fc640 1 --2- 172.21.15.84:0/2353445616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9d00047c0 0x7fa9e415d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:06.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.775+0000 7fa9e94fc640 1 -- 172.21.15.84:0/2353445616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9e40b1720 msgr2=0x7fa9e40c24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:06.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.775+0000 7fa9e94fc640 1 --2- 172.21.15.84:0/2353445616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9e40b1720 0x7fa9e40c24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:06.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.775+0000 7fa9e94fc640 1 -- 172.21.15.84:0/2353445616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e415c150 con 0x7fa9e415bc40 2023-10-16T11:06:06.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.775+0000 7fa9e94fc640 1 --2- 172.21.15.84:0/2353445616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9e415bc40 0x7fa9e40c41b0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fa9d402c410 tx=0x7fa9d4001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:06.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.775+0000 7fa9da7fc640 1 -- 172.21.15.84:0/2353445616 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4039070 con 0x7fa9e415bc40 2023-10-16T11:06:06.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.775+0000 7fa9da7fc640 1 -- 172.21.15.84:0/2353445616 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d402cc80 con 0x7fa9e415bc40 2023-10-16T11:06:06.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.776+0000 7fa9da7fc640 1 -- 172.21.15.84:0/2353445616 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d403c040 con 0x7fa9e415bc40 2023-10-16T11:06:06.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.776+0000 7fa9ebf61640 1 -- 172.21.15.84:0/2353445616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e415c430 con 0x7fa9e415bc40 2023-10-16T11:06:06.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.776+0000 7fa9ebf61640 1 -- 172.21.15.84:0/2353445616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e415c9f0 con 0x7fa9e415bc40 2023-10-16T11:06:06.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.777+0000 7fa9ebf61640 1 -- 172.21.15.84:0/2353445616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e404e160 con 0x7fa9e415bc40 2023-10-16T11:06:06.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.778+0000 7fa9da7fc640 1 -- 172.21.15.84:0/2353445616 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 101) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fa9d4032070 con 0x7fa9e415bc40 2023-10-16T11:06:06.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.779+0000 7fa9da7fc640 1 --2- 172.21.15.84:0/2353445616 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fa9c806acc0 0x7fa9c806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:06.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.779+0000 7fa9da7fc640 1 -- 172.21.15.84:0/2353445616 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6425+0+0 (secure 0 0 0) 0x7fa9d40aa950 con 0x7fa9e415bc40 2023-10-16T11:06:06.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.779+0000 7fa9e9cfd640 1 --2- 172.21.15.84:0/2353445616 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fa9c806acc0 0x7fa9c806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:06.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.779+0000 7fa9e9cfd640 1 --2- 172.21.15.84:0/2353445616 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fa9c806acc0 0x7fa9c806d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0007fd0 tx=0x7fa9e0009040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:06.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:06.783+0000 7fa9da7fc640 1 -- 172.21.15.84:0/2353445616 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa9d4075a70 con 0x7fa9e415bc40 2023-10-16T11:06:07.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.016+0000 7fa9ebf61640 1 -- 172.21.15.84:0/2353445616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9e40cdc60 con 0x7fa9e415bc40 2023-10-16T11:06:07.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.016+0000 7fa9da7fc640 1 -- 172.21.15.84:0/2353445616 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fa9d4079090 con 0x7fa9e415bc40 2023-10-16T11:06:07.018 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:07.019 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":88,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6718,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3060266169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3060266169},{"type":"v1","addr":"172.21.15.157:6834","nonce":3060266169}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":6721,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2965939103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2965939103},{"type":"v1","addr":"172.21.15.157:6835","nonce":2965939103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9684,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3677909298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3677909298},{"type":"v1","addr":"172.21.15.84:6835","nonce":3677909298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:05:43.341956+0000","modified":"2023-10-16T11:05:46.428445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9687},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9687":{"gid":9687,"name":"c","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/424056375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":424056375},{"type":"v1","addr":"172.21.15.84:6837","nonce":424056375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-16T11:06:07.019 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 88 2023-10-16T11:06:07.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.019+0000 7fa9ebf61640 1 -- 172.21.15.84:0/2353445616 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fa9c806acc0 msgr2=0x7fa9c806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:07.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.019+0000 7fa9ebf61640 1 --2- 172.21.15.84:0/2353445616 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fa9c806acc0 0x7fa9c806d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0007fd0 tx=0x7fa9e0009040 comp rx=0 tx=0).stop 2023-10-16T11:06:07.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.019+0000 7fa9ebf61640 1 -- 172.21.15.84:0/2353445616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9e415bc40 msgr2=0x7fa9e40c41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:07.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.019+0000 7fa9ebf61640 1 --2- 172.21.15.84:0/2353445616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9e415bc40 0x7fa9e40c41b0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fa9d402c410 tx=0x7fa9d4001a20 comp rx=0 tx=0).stop 2023-10-16T11:06:07.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.019+0000 7fa9ebf61640 1 -- 172.21.15.84:0/2353445616 shutdown_connections 2023-10-16T11:06:07.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.019+0000 7fa9ebf61640 1 --2- 172.21.15.84:0/2353445616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9e40b1720 0x7fa9e40c24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.019+0000 7fa9ebf61640 1 --2- 172.21.15.84:0/2353445616 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fa9c806acc0 0x7fa9c806d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.019+0000 7fa9ebf61640 1 --2- 172.21.15.84:0/2353445616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9e415bc40 0x7fa9e40c41b0 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.019+0000 7fa9ebf61640 1 --2- 172.21.15.84:0/2353445616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9d00047c0 0x7fa9e415d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.019+0000 7fa9ebf61640 1 -- 172.21.15.84:0/2353445616 >> 172.21.15.84:0/2353445616 conn(0x7fa9e40cb410 msgr2=0x7fa9e40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:07.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.020+0000 7fa9ebf61640 1 -- 172.21.15.84:0/2353445616 shutdown_connections 2023-10-16T11:06:07.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.020+0000 7fa9ebf61640 1 -- 172.21.15.84:0/2353445616 wait complete. 2023-10-16T11:06:07.038 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:06:07.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.245+0000 7f510f59e640 1 Processor -- start 2023-10-16T11:06:07.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.245+0000 7f510f59e640 1 -- start start 2023-10-16T11:06:07.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.246+0000 7f510f59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f511006c040 0x7f511006c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:07.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.247+0000 7f510f59e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511015f4a0 con 0x7f51100cda80 2023-10-16T11:06:07.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.247+0000 7f510f59e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511006c960 con 0x7f51100ad970 2023-10-16T11:06:07.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.247+0000 7f510f59e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f511015b9d0 con 0x7f511006c040 2023-10-16T11:06:07.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.247+0000 7f510ed9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f51100cda80 0x7f511015ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55310/0 (socket says 172.21.15.84:55310) 2023-10-16T11:06:07.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.247+0000 7f510ed9d640 1 -- 172.21.15.84:0/656638887 learned_addr learned my addr 172.21.15.84:0/656638887 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:07.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.247+0000 7f510e59c640 1 --2- 172.21.15.84:0/656638887 >> v2:172.21.15.157:3300/0 conn(0x7f511006c040 0x7f511006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:07.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.247+0000 7f510dd9b640 1 -- 172.21.15.84:0/656638887 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2089638707 0 0) 0x7f511015f4a0 con 0x7f51100cda80 2023-10-16T11:06:07.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.247+0000 7f510dd9b640 1 -- 172.21.15.84:0/656638887 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50f0003520 con 0x7f51100cda80 2023-10-16T11:06:07.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f510dd9b640 1 -- 172.21.15.84:0/656638887 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1627124782 0 0) 0x7f50f0003520 con 0x7f51100cda80 2023-10-16T11:06:07.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f510dd9b640 1 -- 172.21.15.84:0/656638887 >> v2:172.21.15.157:3300/0 conn(0x7f511006c040 msgr2=0x7f511006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:07.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f510dd9b640 1 --2- 172.21.15.84:0/656638887 >> v2:172.21.15.157:3300/0 conn(0x7f511006c040 0x7f511006c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f510dd9b640 1 -- 172.21.15.84:0/656638887 >> v1:172.21.15.157:6789/0 conn(0x7f51100ad970 legacy=0x7f511015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:07.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f510dd9b640 1 -- 172.21.15.84:0/656638887 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511015bcb0 con 0x7f51100cda80 2023-10-16T11:06:07.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f510dd9b640 1 -- 172.21.15.84:0/656638887 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f51080019f0 con 0x7f51100cda80 2023-10-16T11:06:07.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f510dd9b640 1 -- 172.21.15.84:0/656638887 >> v1:172.21.15.84:6789/0 conn(0x7f51100cda80 legacy=0x7f511015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:07.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f510dd9b640 1 --2- 172.21.15.84:0/656638887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50f00039c0 0x7f50f0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:07.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f510dd9b640 1 --2- 172.21.15.84:0/656638887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50f00047c0 0x7f50f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:07.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f510dd9b640 1 --2- 172.21.15.84:0/656638887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f00051d0 0x7f50f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:07.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f510dd9b640 1 -- 172.21.15.84:0/656638887 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511015b9d0 con 0x7f50f00047c0 2023-10-16T11:06:07.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f510dd9b640 1 -- 172.21.15.84:0/656638887 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f000c120 con 0x7f50f00039c0 2023-10-16T11:06:07.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f510dd9b640 1 -- 172.21.15.84:0/656638887 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f000c2e0 con 0x7f50f00051d0 2023-10-16T11:06:07.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f510e59c640 1 --2- 172.21.15.84:0/656638887 >> v2:172.21.15.157:3300/0 conn(0x7f511006c040 0x7f511006c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:06:07.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f5107fff640 1 --2- 172.21.15.84:0/656638887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50f00047c0 0x7f50f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:07.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f5107fff640 1 -- 172.21.15.84:0/656638887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f00051d0 msgr2=0x7f50f000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:07.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f5107fff640 1 --2- 172.21.15.84:0/656638887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f00051d0 0x7f50f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f5107fff640 1 -- 172.21.15.84:0/656638887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50f00039c0 msgr2=0x7f50f0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:07.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.248+0000 7f5107fff640 1 --2- 172.21.15.84:0/656638887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50f00039c0 0x7f50f0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.249+0000 7f5107fff640 1 -- 172.21.15.84:0/656638887 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f0003520 con 0x7f50f00047c0 2023-10-16T11:06:07.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.249+0000 7f510ed9d640 1 -- 172.21.15.84:0/656638887 reap_dead start 2023-10-16T11:06:07.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.250+0000 7f5107fff640 1 --2- 172.21.15.84:0/656638887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50f00047c0 0x7f50f0004c40 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f50f8001720 tx=0x7f50f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946d2ac34472cace server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:07.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.250+0000 7f510dd9b640 1 -- 172.21.15.84:0/656638887 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f80052d0 con 0x7f50f00047c0 2023-10-16T11:06:07.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.250+0000 7f510dd9b640 1 -- 172.21.15.84:0/656638887 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f802e050 con 0x7f50f00047c0 2023-10-16T11:06:07.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.250+0000 7f510dd9b640 1 -- 172.21.15.84:0/656638887 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8003830 con 0x7f50f00047c0 2023-10-16T11:06:07.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.250+0000 7f510f59e640 1 -- 172.21.15.84:0/656638887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50f00047c0 msgr2=0x7f50f0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:07.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.250+0000 7f510f59e640 1 --2- 172.21.15.84:0/656638887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50f00047c0 0x7f50f0004c40 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f50f8001720 tx=0x7f50f00057f0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.250+0000 7f510f59e640 1 -- 172.21.15.84:0/656638887 shutdown_connections 2023-10-16T11:06:07.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.250+0000 7f510f59e640 1 --2- 172.21.15.84:0/656638887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50f00047c0 0x7f50f0004c40 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.250+0000 7f510f59e640 1 -- 172.21.15.84:0/656638887 >> 172.21.15.84:0/656638887 conn(0x7f51100ce6a0 msgr2=0x7f51100ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:07.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.250+0000 7f510f59e640 1 -- 172.21.15.84:0/656638887 shutdown_connections 2023-10-16T11:06:07.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.250+0000 7f510f59e640 1 -- 172.21.15.84:0/656638887 wait complete. 2023-10-16T11:06:07.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.252+0000 7f510f59e640 1 Processor -- start 2023-10-16T11:06:07.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.252+0000 7f510f59e640 1 -- start start 2023-10-16T11:06:07.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.253+0000 7f510f59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f00047c0 0x7f51101e4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:07.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.253+0000 7f510f59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51101e5b70 0x7f51100ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:07.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.253+0000 7f510f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f511014fbc0 0x7f51100ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:07.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.253+0000 7f510f59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51101e5fd0 con 0x7f511014fbc0 2023-10-16T11:06:07.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.253+0000 7f510f59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51100cad00 con 0x7f51101e5b70 2023-10-16T11:06:07.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.253+0000 7f510f59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51101e4ad0 con 0x7f50f00047c0 2023-10-16T11:06:07.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.254+0000 7f510ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f511014fbc0 0x7f51100ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:07.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.254+0000 7f510ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f511014fbc0 0x7f51100ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56270/0 (socket says 172.21.15.84:56270) 2023-10-16T11:06:07.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.254+0000 7f510ed9d640 1 -- 172.21.15.84:0/1048370942 learned_addr learned my addr 172.21.15.84:0/1048370942 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:07.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.254+0000 7f510e59c640 1 --2- 172.21.15.84:0/1048370942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f00047c0 0x7f51101e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:07.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.254+0000 7f510ed9d640 1 -- 172.21.15.84:0/1048370942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f00047c0 msgr2=0x7f51101e4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:07.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.254+0000 7f510ed9d640 1 --2- 172.21.15.84:0/1048370942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f00047c0 0x7f51101e4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.254+0000 7f5107fff640 1 --2- 172.21.15.84:0/1048370942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51101e5b70 0x7f51100ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:07.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.254+0000 7f510ed9d640 1 -- 172.21.15.84:0/1048370942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51101e5b70 msgr2=0x7f51100ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:07.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.254+0000 7f510ed9d640 1 --2- 172.21.15.84:0/1048370942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51101e5b70 0x7f51100ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.254+0000 7f510ed9d640 1 -- 172.21.15.84:0/1048370942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51101e4db0 con 0x7f511014fbc0 2023-10-16T11:06:07.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.254+0000 7f510ed9d640 1 --2- 172.21.15.84:0/1048370942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f511014fbc0 0x7f51100ac890 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f511015f720 tx=0x7f510800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:07.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.254+0000 7f5106ffd640 1 -- 172.21.15.84:0/1048370942 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510800cea0 con 0x7f511014fbc0 2023-10-16T11:06:07.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.254+0000 7f5106ffd640 1 -- 172.21.15.84:0/1048370942 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510800a610 con 0x7f511014fbc0 2023-10-16T11:06:07.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.254+0000 7f5106ffd640 1 -- 172.21.15.84:0/1048370942 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510800a9e0 con 0x7f511014fbc0 2023-10-16T11:06:07.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.254+0000 7f510f59e640 1 -- 172.21.15.84:0/1048370942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51101e5090 con 0x7f511014fbc0 2023-10-16T11:06:07.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.255+0000 7f510f59e640 1 -- 172.21.15.84:0/1048370942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51101e5650 con 0x7f511014fbc0 2023-10-16T11:06:07.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.256+0000 7f510f59e640 1 -- 172.21.15.84:0/1048370942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511006b330 con 0x7f511014fbc0 2023-10-16T11:06:07.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.257+0000 7f5106ffd640 1 -- 172.21.15.84:0/1048370942 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 101) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f510800ab40 con 0x7f511014fbc0 2023-10-16T11:06:07.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.258+0000 7f5106ffd640 1 --2- 172.21.15.84:0/1048370942 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f50f406acc0 0x7f50f406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:07.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.258+0000 7f5106ffd640 1 -- 172.21.15.84:0/1048370942 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f5108088dc0 con 0x7f511014fbc0 2023-10-16T11:06:07.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.258+0000 7f510e59c640 1 --2- 172.21.15.84:0/1048370942 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f50f406acc0 0x7f50f406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:07.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.259+0000 7f510e59c640 1 --2- 172.21.15.84:0/1048370942 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f50f406acc0 0x7f50f406d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5110063ab0 tx=0x7f5100008040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:07.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.262+0000 7f5106ffd640 1 -- 172.21.15.84:0/1048370942 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5108053ee0 con 0x7f511014fbc0 2023-10-16T11:06:07.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.454+0000 7f510f59e640 1 -- 172.21.15.84:0/1048370942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f511004e9b0 con 0x7f511014fbc0 2023-10-16T11:06:07.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.457+0000 7f5106ffd640 1 -- 172.21.15.84:0/1048370942 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+17601 (secure 0 0 0) 0x7f510808a020 con 0x7f511014fbc0 2023-10-16T11:06:07.459 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:07.459 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":173,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:06:05.639308+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:05:41.217474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-16T11:05:42.387743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/849217271":"2023-10-17T11:06:05.639149+0000","172.21.15.84:0/1510301035":"2023-10-17T11:06:05.639149+0000","172.21.15.84:0/2220106058":"2023-10-17T11:06:05.639149+0000","172.21.15.84:6833/1697458478":"2023-10-17T11:06:05.639149+0000","172.21.15.84:0/1365551239":"2023-10-17T11:06:05.639149+0000","172.21.15.84:0/2246425640":"2023-10-17T11:06:05.639149+0000","172.21.15.84:0/3425147335":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/1161603770":"2023-10-17T11:05:57.603800+0000","172.21.15.84:6832/1697458478":"2023-10-17T11:06:05.639149+0000","172.21.15.84:6833/2442400184":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/3742714530":"2023-10-17T11:06:05.639149+0000","172.21.15.84:6832/2442400184":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/3633178028":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/1914327915":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/2602045766":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/3065897528":"2023-10-17T11:05:57.603800+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:06:07.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.459+0000 7f510f59e640 1 -- 172.21.15.84:0/1048370942 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f50f406acc0 msgr2=0x7f50f406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:07.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.459+0000 7f510f59e640 1 --2- 172.21.15.84:0/1048370942 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f50f406acc0 0x7f50f406d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5110063ab0 tx=0x7f5100008040 comp rx=0 tx=0).stop 2023-10-16T11:06:07.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.459+0000 7f510f59e640 1 -- 172.21.15.84:0/1048370942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f511014fbc0 msgr2=0x7f51100ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:07.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.459+0000 7f510f59e640 1 --2- 172.21.15.84:0/1048370942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f511014fbc0 0x7f51100ac890 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f511015f720 tx=0x7f510800cc50 comp rx=0 tx=0).stop 2023-10-16T11:06:07.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.460+0000 7f510f59e640 1 -- 172.21.15.84:0/1048370942 shutdown_connections 2023-10-16T11:06:07.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.460+0000 7f510f59e640 1 --2- 172.21.15.84:0/1048370942 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f50f406acc0 0x7f50f406d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.460+0000 7f510f59e640 1 --2- 172.21.15.84:0/1048370942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f511014fbc0 0x7f51100ac890 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.460+0000 7f510f59e640 1 --2- 172.21.15.84:0/1048370942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51101e5b70 0x7f51100ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.460+0000 7f510f59e640 1 --2- 172.21.15.84:0/1048370942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f00047c0 0x7f51101e4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.460+0000 7f510f59e640 1 -- 172.21.15.84:0/1048370942 >> 172.21.15.84:0/1048370942 conn(0x7f51100ce6a0 msgr2=0x7f511015c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:07.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.460+0000 7f510f59e640 1 -- 172.21.15.84:0/1048370942 shutdown_connections 2023-10-16T11:06:07.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.460+0000 7f510f59e640 1 -- 172.21.15.84:0/1048370942 wait complete. 2023-10-16T11:06:07.481 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T11:06:07.482 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:06:07.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.683+0000 7fabb0da5640 1 Processor -- start 2023-10-16T11:06:07.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.683+0000 7fabb0da5640 1 -- start start 2023-10-16T11:06:07.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.684+0000 7fabb0da5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabac15aaa0 0x7fabac15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:07.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.684+0000 7fabb0da5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fabac15ae80 0x7fabac0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:07.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.684+0000 7fabb0da5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabac15ed60 con 0x7fabac0ccaa0 2023-10-16T11:06:07.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.684+0000 7fabb0da5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabac06b810 con 0x7fabac15ae80 2023-10-16T11:06:07.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.684+0000 7fabb0da5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac0707e0 con 0x7fabac15aaa0 2023-10-16T11:06:07.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.685+0000 7fabaad9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fabac0ccaa0 0x7fabac070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55314/0 (socket says 172.21.15.84:55314) 2023-10-16T11:06:07.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.685+0000 7fabaad9d640 1 -- 172.21.15.84:0/2655119153 learned_addr learned my addr 172.21.15.84:0/2655119153 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:07.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.685+0000 7faba3fff640 1 --2- 172.21.15.84:0/2655119153 >> v2:172.21.15.157:3300/0 conn(0x7fabac15ae80 0x7fabac0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:07.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.685+0000 7fabaa59c640 1 --2- 172.21.15.84:0/2655119153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabac15aaa0 0x7fabac15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:07.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.685+0000 7faba9d9b640 1 -- 172.21.15.84:0/2655119153 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2740595518 0 0) 0x7fabac15ed60 con 0x7fabac0ccaa0 2023-10-16T11:06:07.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.685+0000 7faba9d9b640 1 -- 172.21.15.84:0/2655119153 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab98003520 con 0x7fabac0ccaa0 2023-10-16T11:06:07.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.686+0000 7faba9d9b640 1 -- 172.21.15.84:0/2655119153 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1369444856 0 0) 0x7fab98003520 con 0x7fabac0ccaa0 2023-10-16T11:06:07.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.686+0000 7faba9d9b640 1 -- 172.21.15.84:0/2655119153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabac15aaa0 msgr2=0x7fabac15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:07.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.686+0000 7faba9d9b640 1 --2- 172.21.15.84:0/2655119153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabac15aaa0 0x7fabac15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.686+0000 7faba9d9b640 1 -- 172.21.15.84:0/2655119153 >> v2:172.21.15.157:3300/0 conn(0x7fabac15ae80 msgr2=0x7fabac0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:07.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.686+0000 7faba9d9b640 1 --2- 172.21.15.84:0/2655119153 >> v2:172.21.15.157:3300/0 conn(0x7fabac15ae80 0x7fabac0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.686+0000 7faba9d9b640 1 -- 172.21.15.84:0/2655119153 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac15ef90 con 0x7fabac0ccaa0 2023-10-16T11:06:07.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.686+0000 7faba9d9b640 1 -- 172.21.15.84:0/2655119153 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7faba40019f0 con 0x7fabac0ccaa0 2023-10-16T11:06:07.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.686+0000 7faba9d9b640 1 -- 172.21.15.84:0/2655119153 >> v1:172.21.15.84:6789/0 conn(0x7fabac0ccaa0 legacy=0x7fabac070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:07.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.686+0000 7faba9d9b640 1 --2- 172.21.15.84:0/2655119153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabac15aaa0 0x7fab98003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:07.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.686+0000 7faba9d9b640 1 --2- 172.21.15.84:0/2655119153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab98003ef0 0x7fab980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:07.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.686+0000 7faba9d9b640 1 --2- 172.21.15.84:0/2655119153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab98004d00 0x7fab98005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:07.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.686+0000 7faba9d9b640 1 -- 172.21.15.84:0/2655119153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac06b810 con 0x7fab98003ef0 2023-10-16T11:06:07.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.686+0000 7faba9d9b640 1 -- 172.21.15.84:0/2655119153 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac0707e0 con 0x7fab98004d00 2023-10-16T11:06:07.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.686+0000 7faba9d9b640 1 -- 172.21.15.84:0/2655119153 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9800fde0 con 0x7fabac15aaa0 2023-10-16T11:06:07.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.686+0000 7faba3fff640 1 --2- 172.21.15.84:0/2655119153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab98003ef0 0x7fab980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:07.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.686+0000 7fabaa59c640 1 --2- 172.21.15.84:0/2655119153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabac15aaa0 0x7fab98003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:07.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.686+0000 7faba3fff640 1 -- 172.21.15.84:0/2655119153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabac15aaa0 msgr2=0x7fab98003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:07.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.686+0000 7fabaad9d640 1 --2- 172.21.15.84:0/2655119153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab98004d00 0x7fab98005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:07.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.687+0000 7faba3fff640 1 --2- 172.21.15.84:0/2655119153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabac15aaa0 0x7fab98003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.687+0000 7faba3fff640 1 -- 172.21.15.84:0/2655119153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab98004d00 msgr2=0x7fab98005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:07.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.687+0000 7faba3fff640 1 --2- 172.21.15.84:0/2655119153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab98004d00 0x7fab98005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.687+0000 7faba3fff640 1 -- 172.21.15.84:0/2655119153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab98003520 con 0x7fab98003ef0 2023-10-16T11:06:07.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.687+0000 7fabaa59c640 1 --2- 172.21.15.84:0/2655119153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabac15aaa0 0x7fab98003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:07.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.688+0000 7faba3fff640 1 --2- 172.21.15.84:0/2655119153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab98003ef0 0x7fab980047c0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fab94009ac0 tx=0x7fab980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db204f9ae02c4f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:07.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.688+0000 7faba9d9b640 1 -- 172.21.15.84:0/2655119153 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9402c710 con 0x7fab98003ef0 2023-10-16T11:06:07.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.688+0000 7faba9d9b640 1 -- 172.21.15.84:0/2655119153 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9402c870 con 0x7fab98003ef0 2023-10-16T11:06:07.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.688+0000 7faba9d9b640 1 -- 172.21.15.84:0/2655119153 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9402cc00 con 0x7fab98003ef0 2023-10-16T11:06:07.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.689+0000 7fabb0da5640 1 -- 172.21.15.84:0/2655119153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab98003ef0 msgr2=0x7fab980047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:07.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.689+0000 7fabb0da5640 1 --2- 172.21.15.84:0/2655119153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab98003ef0 0x7fab980047c0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fab94009ac0 tx=0x7fab980057f0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.689+0000 7fabaad9d640 1 -- 172.21.15.84:0/2655119153 reap_dead start 2023-10-16T11:06:07.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.689+0000 7fabb0da5640 1 -- 172.21.15.84:0/2655119153 shutdown_connections 2023-10-16T11:06:07.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.689+0000 7fabb0da5640 1 -- 172.21.15.84:0/2655119153 >> 172.21.15.84:0/2655119153 conn(0x7fabac0cb410 msgr2=0x7fabac0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:07.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.689+0000 7fabb0da5640 1 -- 172.21.15.84:0/2655119153 shutdown_connections 2023-10-16T11:06:07.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.689+0000 7fabb0da5640 1 -- 172.21.15.84:0/2655119153 wait complete. 2023-10-16T11:06:07.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.691+0000 7fabb0da5640 1 Processor -- start 2023-10-16T11:06:07.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.691+0000 7fabb0da5640 1 -- start start 2023-10-16T11:06:07.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.692+0000 7fabb0da5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab98003ef0 0x7fabac0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:07.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.692+0000 7fabb0da5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabac1eee40 0x7fabac0c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:07.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.692+0000 7fabb0da5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabac1ee150 0x7fabac1ee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:07.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.692+0000 7fabb0da5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac0c5a60 con 0x7fab98003ef0 2023-10-16T11:06:07.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.692+0000 7fabb0da5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac1eeb10 con 0x7fabac1ee150 2023-10-16T11:06:07.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.692+0000 7fabb0da5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac0c2fc0 con 0x7fabac1eee40 2023-10-16T11:06:07.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.692+0000 7fabaa59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab98003ef0 0x7fabac0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:07.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.692+0000 7fabaa59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab98003ef0 0x7fabac0ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56290/0 (socket says 172.21.15.84:56290) 2023-10-16T11:06:07.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.692+0000 7fabaa59c640 1 -- 172.21.15.84:0/3869664085 learned_addr learned my addr 172.21.15.84:0/3869664085 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:07.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.692+0000 7fabaad9d640 1 --2- 172.21.15.84:0/3869664085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabac1ee150 0x7fabac1ee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:07.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.692+0000 7fabaa59c640 1 -- 172.21.15.84:0/3869664085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabac1eee40 msgr2=0x7fabac0c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:07.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.692+0000 7fabaa59c640 1 --2- 172.21.15.84:0/3869664085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabac1eee40 0x7fabac0c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.692+0000 7fabaa59c640 1 -- 172.21.15.84:0/3869664085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabac1ee150 msgr2=0x7fabac1ee5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:07.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.692+0000 7fabaa59c640 1 --2- 172.21.15.84:0/3869664085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabac1ee150 0x7fabac1ee5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.692+0000 7fabaa59c640 1 -- 172.21.15.84:0/3869664085 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac0c32a0 con 0x7fab98003ef0 2023-10-16T11:06:07.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.693+0000 7fabaa59c640 1 --2- 172.21.15.84:0/3869664085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab98003ef0 0x7fabac0ca240 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00e7e0 tx=0x7fab9c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:07.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.693+0000 7faba2ffd640 1 -- 172.21.15.84:0/3869664085 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c00c6d0 con 0x7fab98003ef0 2023-10-16T11:06:07.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.693+0000 7faba2ffd640 1 -- 172.21.15.84:0/3869664085 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c00c830 con 0x7fab98003ef0 2023-10-16T11:06:07.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.693+0000 7fabb0da5640 1 -- 172.21.15.84:0/3869664085 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabac0c3580 con 0x7fab98003ef0 2023-10-16T11:06:07.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.693+0000 7faba2ffd640 1 -- 172.21.15.84:0/3869664085 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c00cba0 con 0x7fab98003ef0 2023-10-16T11:06:07.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.693+0000 7fabb0da5640 1 -- 172.21.15.84:0/3869664085 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabac0c3b40 con 0x7fab98003ef0 2023-10-16T11:06:07.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.694+0000 7fabb0da5640 1 -- 172.21.15.84:0/3869664085 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabac04e160 con 0x7fab98003ef0 2023-10-16T11:06:07.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.695+0000 7faba2ffd640 1 -- 172.21.15.84:0/3869664085 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 101) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fab9c00cd00 con 0x7fab98003ef0 2023-10-16T11:06:07.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.696+0000 7faba2ffd640 1 --2- 172.21.15.84:0/3869664085 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fab8c06acf0 0x7fab8c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:07.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.696+0000 7faba2ffd640 1 -- 172.21.15.84:0/3869664085 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6425+0+0 (secure 0 0 0) 0x7fab9c0891a0 con 0x7fab98003ef0 2023-10-16T11:06:07.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.696+0000 7faba3fff640 1 --2- 172.21.15.84:0/3869664085 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fab8c06acf0 0x7fab8c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:07.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.697+0000 7faba3fff640 1 --2- 172.21.15.84:0/3869664085 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fab8c06acf0 0x7fab8c06d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fab94036000 tx=0x7fab9402e000 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:07.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.701+0000 7faba2ffd640 1 -- 172.21.15.84:0/3869664085 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fab9c0542c0 con 0x7fab98003ef0 2023-10-16T11:06:07.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.952+0000 7fabb0da5640 1 -- 172.21.15.84:0/3869664085 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabac06d570 con 0x7fab98003ef0 2023-10-16T11:06:07.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.953+0000 7faba2ffd640 1 -- 172.21.15.84:0/3869664085 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fab9c0578e0 con 0x7fab98003ef0 2023-10-16T11:06:07.954 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:07.955 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":88,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6718,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3060266169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3060266169},{"type":"v1","addr":"172.21.15.157:6834","nonce":3060266169}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":6721,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2965939103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2965939103},{"type":"v1","addr":"172.21.15.157:6835","nonce":2965939103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9684,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3677909298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3677909298},{"type":"v1","addr":"172.21.15.84:6835","nonce":3677909298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:05:43.341956+0000","modified":"2023-10-16T11:05:46.428445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9687},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9687":{"gid":9687,"name":"c","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/424056375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":424056375},{"type":"v1","addr":"172.21.15.84:6837","nonce":424056375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-16T11:06:07.955 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 88 2023-10-16T11:06:07.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.955+0000 7fabb0da5640 1 -- 172.21.15.84:0/3869664085 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fab8c06acf0 msgr2=0x7fab8c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:07.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.955+0000 7fabb0da5640 1 --2- 172.21.15.84:0/3869664085 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fab8c06acf0 0x7fab8c06d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fab94036000 tx=0x7fab9402e000 comp rx=0 tx=0).stop 2023-10-16T11:06:07.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.955+0000 7fabb0da5640 1 -- 172.21.15.84:0/3869664085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab98003ef0 msgr2=0x7fabac0ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:07.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.955+0000 7fabb0da5640 1 --2- 172.21.15.84:0/3869664085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab98003ef0 0x7fabac0ca240 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00e7e0 tx=0x7fab9c00eca0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.955+0000 7fabb0da5640 1 -- 172.21.15.84:0/3869664085 shutdown_connections 2023-10-16T11:06:07.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.955+0000 7fabb0da5640 1 --2- 172.21.15.84:0/3869664085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabac1ee150 0x7fabac1ee5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.955+0000 7fabb0da5640 1 --2- 172.21.15.84:0/3869664085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabac1eee40 0x7fabac0c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.955+0000 7fabb0da5640 1 --2- 172.21.15.84:0/3869664085 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fab8c06acf0 0x7fab8c06d1b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.955+0000 7fabb0da5640 1 --2- 172.21.15.84:0/3869664085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab98003ef0 0x7fabac0ca240 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:07.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.955+0000 7fabb0da5640 1 -- 172.21.15.84:0/3869664085 >> 172.21.15.84:0/3869664085 conn(0x7fabac0cb410 msgr2=0x7fabac070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:07.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.956+0000 7fabb0da5640 1 -- 172.21.15.84:0/3869664085 shutdown_connections 2023-10-16T11:06:07.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:07.956+0000 7fabb0da5640 1 -- 172.21.15.84:0/3869664085 wait complete. 2023-10-16T11:06:07.975 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:06:08.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.189+0000 7f596d74e640 1 Processor -- start 2023-10-16T11:06:08.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.190+0000 7f596d74e640 1 -- start start 2023-10-16T11:06:08.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.191+0000 7f596d74e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f596806c040 0x7f596806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:08.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.191+0000 7f596d74e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f59680ad970 0x7f596815e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:08.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.191+0000 7f596d74e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f596815ebc0 0x7f596815f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:08.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.191+0000 7f596d74e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596806c960 con 0x7f59680ad970 2023-10-16T11:06:08.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.191+0000 7f596d74e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596815b9d0 con 0x7f596806c040 2023-10-16T11:06:08.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.191+0000 7f596d74e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596815bcb0 con 0x7f596815ebc0 2023-10-16T11:06:08.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.192+0000 7f595ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f59680ad970 0x7f596815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:08.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.192+0000 7f595ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f59680ad970 0x7f596815e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56302/0 (socket says 172.21.15.84:56302) 2023-10-16T11:06:08.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.192+0000 7f595ffff640 1 -- 172.21.15.84:0/2189089910 learned_addr learned my addr 172.21.15.84:0/2189089910 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:08.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.192+0000 7f5967fff640 1 --2- 172.21.15.84:0/2189089910 >> v2:172.21.15.157:3300/0 conn(0x7f596806c040 0x7f596806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:08.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.192+0000 7f595ffff640 1 -- 172.21.15.84:0/2189089910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f596815ebc0 msgr2=0x7f596815f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:08.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.192+0000 7f596cf4d640 1 --2- 172.21.15.84:0/2189089910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f596815ebc0 0x7f596815f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:08.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.192+0000 7f595ffff640 1 --2- 172.21.15.84:0/2189089910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f596815ebc0 0x7f596815f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.192+0000 7f595ffff640 1 -- 172.21.15.84:0/2189089910 >> v2:172.21.15.157:3300/0 conn(0x7f596806c040 msgr2=0x7f596806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:08.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.192+0000 7f595ffff640 1 --2- 172.21.15.84:0/2189089910 >> v2:172.21.15.157:3300/0 conn(0x7f596806c040 0x7f596806c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.192+0000 7f595ffff640 1 -- 172.21.15.84:0/2189089910 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596815bf90 con 0x7f59680ad970 2023-10-16T11:06:08.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.192+0000 7f5967fff640 1 --2- 172.21.15.84:0/2189089910 >> v2:172.21.15.157:3300/0 conn(0x7f596806c040 0x7f596806c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:08.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.193+0000 7f595ffff640 1 --2- 172.21.15.84:0/2189089910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59680ad970 0x7f596815e680 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f59680cf350 tx=0x7f595002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d27d1642dbbba7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:08.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.194+0000 7f59677fe640 1 -- 172.21.15.84:0/2189089910 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5950031870 con 0x7f59680ad970 2023-10-16T11:06:08.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.194+0000 7f59677fe640 1 -- 172.21.15.84:0/2189089910 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59500319d0 con 0x7f59680ad970 2023-10-16T11:06:08.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.194+0000 7f59677fe640 1 -- 172.21.15.84:0/2189089910 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5950031d60 con 0x7f59680ad970 2023-10-16T11:06:08.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.194+0000 7f596d74e640 1 -- 172.21.15.84:0/2189089910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59680ad970 msgr2=0x7f596815e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:08.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.194+0000 7f596d74e640 1 --2- 172.21.15.84:0/2189089910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59680ad970 0x7f596815e680 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f59680cf350 tx=0x7f595002b910 comp rx=0 tx=0).stop 2023-10-16T11:06:08.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.195+0000 7f596d74e640 1 -- 172.21.15.84:0/2189089910 shutdown_connections 2023-10-16T11:06:08.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.195+0000 7f596d74e640 1 --2- 172.21.15.84:0/2189089910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59680ad970 0x7f596815e680 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.195+0000 7f596d74e640 1 --2- 172.21.15.84:0/2189089910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f596815ebc0 0x7f596815f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.195+0000 7f596d74e640 1 --2- 172.21.15.84:0/2189089910 >> v2:172.21.15.157:3300/0 conn(0x7f596806c040 0x7f596806c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.195+0000 7f596d74e640 1 -- 172.21.15.84:0/2189089910 >> 172.21.15.84:0/2189089910 conn(0x7f59680ce6a0 msgr2=0x7f59680ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:08.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.195+0000 7f596d74e640 1 -- 172.21.15.84:0/2189089910 shutdown_connections 2023-10-16T11:06:08.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.195+0000 7f596d74e640 1 -- 172.21.15.84:0/2189089910 wait complete. 2023-10-16T11:06:08.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.196+0000 7f596d74e640 1 Processor -- start 2023-10-16T11:06:08.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.196+0000 7f596d74e640 1 -- start start 2023-10-16T11:06:08.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.197+0000 7f596d74e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f596806c040 0x7f59680c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:08.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.197+0000 7f596d74e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59680ad970 0x7f59680c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:08.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.197+0000 7f596d74e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596815ebc0 0x7f59680cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:08.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.197+0000 7f596d74e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59680c5d50 con 0x7f596806c040 2023-10-16T11:06:08.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.197+0000 7f596d74e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59680c6030 con 0x7f596815ebc0 2023-10-16T11:06:08.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.197+0000 7f596d74e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59680c6310 con 0x7f59680ad970 2023-10-16T11:06:08.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.198+0000 7f5967fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f596806c040 0x7f59680c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:08.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.198+0000 7f5967fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f596806c040 0x7f59680c5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56308/0 (socket says 172.21.15.84:56308) 2023-10-16T11:06:08.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.198+0000 7f5967fff640 1 -- 172.21.15.84:0/1057915577 learned_addr learned my addr 172.21.15.84:0/1057915577 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:08.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.198+0000 7f596cf4d640 1 --2- 172.21.15.84:0/1057915577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596815ebc0 0x7f59680cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:08.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.198+0000 7f5967fff640 1 -- 172.21.15.84:0/1057915577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59680ad970 msgr2=0x7f59680c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:08.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.198+0000 7f5967fff640 1 --2- 172.21.15.84:0/1057915577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59680ad970 0x7f59680c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.198+0000 7f5967fff640 1 -- 172.21.15.84:0/1057915577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596815ebc0 msgr2=0x7f59680cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:08.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.198+0000 7f5967fff640 1 --2- 172.21.15.84:0/1057915577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596815ebc0 0x7f59680cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.198+0000 7f5967fff640 1 -- 172.21.15.84:0/1057915577 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59500094d0 con 0x7f596806c040 2023-10-16T11:06:08.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.198+0000 7f596cf4d640 1 --2- 172.21.15.84:0/1057915577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596815ebc0 0x7f59680cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:08.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.198+0000 7f5967fff640 1 --2- 172.21.15.84:0/1057915577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f596806c040 0x7f59680c5770 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f595800c7f0 tx=0x7f595800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:08.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.199+0000 7f59657fa640 1 -- 172.21.15.84:0/1057915577 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5958007820 con 0x7f596806c040 2023-10-16T11:06:08.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.199+0000 7f59657fa640 1 -- 172.21.15.84:0/1057915577 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5958007980 con 0x7f596806c040 2023-10-16T11:06:08.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.199+0000 7f596d74e640 1 -- 172.21.15.84:0/1057915577 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59680c65f0 con 0x7f596806c040 2023-10-16T11:06:08.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.199+0000 7f59657fa640 1 -- 172.21.15.84:0/1057915577 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5958007d30 con 0x7f596806c040 2023-10-16T11:06:08.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.199+0000 7f596d74e640 1 -- 172.21.15.84:0/1057915577 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59680c7330 con 0x7f596806c040 2023-10-16T11:06:08.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.200+0000 7f596d74e640 1 -- 172.21.15.84:0/1057915577 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596806b330 con 0x7f596806c040 2023-10-16T11:06:08.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.201+0000 7f59657fa640 1 -- 172.21.15.84:0/1057915577 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 101) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f59580113f0 con 0x7f596806c040 2023-10-16T11:06:08.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.202+0000 7f59657fa640 1 --2- 172.21.15.84:0/1057915577 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f594806acf0 0x7f594806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:08.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.202+0000 7f59657fa640 1 -- 172.21.15.84:0/1057915577 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f5958089800 con 0x7f596806c040 2023-10-16T11:06:08.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.203+0000 7f595ffff640 1 --2- 172.21.15.84:0/1057915577 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f594806acf0 0x7f594806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:08.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.203+0000 7f595ffff640 1 --2- 172.21.15.84:0/1057915577 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f594806acf0 0x7f594806d1b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5950036000 tx=0x7f5950030000 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:08.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.207+0000 7f59657fa640 1 -- 172.21.15.84:0/1057915577 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f59580548a0 con 0x7f596806c040 2023-10-16T11:06:08.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.434+0000 7f59657fa640 1 -- 172.21.15.84:0/1057915577 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 102) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f5958054c70 con 0x7f596806c040 2023-10-16T11:06:08.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.446+0000 7f596d74e640 1 -- 172.21.15.84:0/1057915577 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59680c6b10 con 0x7f596806c040 2023-10-16T11:06:08.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.447+0000 7f59657fa640 1 -- 172.21.15.84:0/1057915577 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f59680c6b10 con 0x7f596806c040 2023-10-16T11:06:08.449 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:08.449 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":88,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6718,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3060266169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3060266169},{"type":"v1","addr":"172.21.15.157:6834","nonce":3060266169}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":6721,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2965939103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2965939103},{"type":"v1","addr":"172.21.15.157:6835","nonce":2965939103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9684,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3677909298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3677909298},{"type":"v1","addr":"172.21.15.84:6835","nonce":3677909298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:05:43.341956+0000","modified":"2023-10-16T11:05:46.428445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9687},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9687":{"gid":9687,"name":"c","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/424056375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":424056375},{"type":"v1","addr":"172.21.15.84:6837","nonce":424056375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-16T11:06:08.450 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 88 2023-10-16T11:06:08.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.450+0000 7f596d74e640 1 -- 172.21.15.84:0/1057915577 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f594806acf0 msgr2=0x7f594806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:08.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.450+0000 7f596d74e640 1 --2- 172.21.15.84:0/1057915577 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f594806acf0 0x7f594806d1b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5950036000 tx=0x7f5950030000 comp rx=0 tx=0).stop 2023-10-16T11:06:08.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.450+0000 7f596d74e640 1 -- 172.21.15.84:0/1057915577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f596806c040 msgr2=0x7f59680c5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:08.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.450+0000 7f596d74e640 1 --2- 172.21.15.84:0/1057915577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f596806c040 0x7f59680c5770 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f595800c7f0 tx=0x7f595800ccb0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.450+0000 7f596d74e640 1 -- 172.21.15.84:0/1057915577 shutdown_connections 2023-10-16T11:06:08.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.450+0000 7f596d74e640 1 --2- 172.21.15.84:0/1057915577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596815ebc0 0x7f59680cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.450+0000 7f596d74e640 1 --2- 172.21.15.84:0/1057915577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59680ad970 0x7f59680c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.450+0000 7f596d74e640 1 --2- 172.21.15.84:0/1057915577 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f594806acf0 0x7f594806d1b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.450+0000 7f596d74e640 1 --2- 172.21.15.84:0/1057915577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f596806c040 0x7f59680c5770 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.450+0000 7f596d74e640 1 -- 172.21.15.84:0/1057915577 >> 172.21.15.84:0/1057915577 conn(0x7f59680ce6a0 msgr2=0x7f5968159bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:08.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.450+0000 7f596d74e640 1 -- 172.21.15.84:0/1057915577 shutdown_connections 2023-10-16T11:06:08.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.451+0000 7f596d74e640 1 -- 172.21.15.84:0/1057915577 wait complete. 2023-10-16T11:06:08.471 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:06:08.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.673+0000 7f8b7d533640 1 Processor -- start 2023-10-16T11:06:08.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.674+0000 7f8b7d533640 1 -- start start 2023-10-16T11:06:08.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.675+0000 7f8b7d533640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b78159f00 0x7f8b780b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:08.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.675+0000 7f8b7d533640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b7815d880 0x7f8b780b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:08.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.675+0000 7f8b7d533640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7806f7f0 con 0x7f8b78159b20 2023-10-16T11:06:08.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.675+0000 7f8b7d533640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b7806c5e0 con 0x7f8b7815d880 2023-10-16T11:06:08.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.675+0000 7f8b7d533640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7806c8c0 con 0x7f8b78159f00 2023-10-16T11:06:08.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.676+0000 7f8b777fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b7815d880 0x7f8b780b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:08.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.676+0000 7f8b777fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b7815d880 0x7f8b780b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:42768/0 (socket says 172.21.15.84:42768) 2023-10-16T11:06:08.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.676+0000 7f8b777fe640 1 -- 172.21.15.84:0/1077143986 learned_addr learned my addr 172.21.15.84:0/1077143986 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:08.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.676+0000 7f8b76ffd640 1 --1- 172.21.15.84:0/1077143986 >> v1:172.21.15.157:6789/0 conn(0x7f8b78159b20 0x7f8b7806bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:36424/0 (socket says 172.21.15.84:36424) 2023-10-16T11:06:08.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.676+0000 7f8b6e7fc640 1 --2- 172.21.15.84:0/1077143986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b78159f00 0x7f8b780b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:08.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.677+0000 7f8b777fe640 1 -- 172.21.15.84:0/1077143986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b78159f00 msgr2=0x7f8b780b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:08.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.677+0000 7f8b777fe640 1 --2- 172.21.15.84:0/1077143986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b78159f00 0x7f8b780b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.677+0000 7f8b777fe640 1 -- 172.21.15.84:0/1077143986 >> v1:172.21.15.157:6789/0 conn(0x7f8b78159b20 legacy=0x7f8b7806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:08.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.677+0000 7f8b777fe640 1 -- 172.21.15.84:0/1077143986 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7806cba0 con 0x7f8b7815d880 2023-10-16T11:06:08.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.677+0000 7f8b767fc640 1 -- 172.21.15.84:0/1077143986 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4062835771 0 0) 0x7f8b7806f7f0 con 0x7f8b78159b20 2023-10-16T11:06:08.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.677+0000 7f8b6e7fc640 1 --2- 172.21.15.84:0/1077143986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b78159f00 0x7f8b780b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:06:08.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.679+0000 7f8b777fe640 1 --2- 172.21.15.84:0/1077143986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7815d880 0x7f8b780b0950 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009e50 tx=0x7f8b7002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c56c327e6f20af10 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:08.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.679+0000 7f8b767fc640 1 -- 172.21.15.84:0/1077143986 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7002cb00 con 0x7f8b7815d880 2023-10-16T11:06:08.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.679+0000 7f8b767fc640 1 -- 172.21.15.84:0/1077143986 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7002cc60 con 0x7f8b7815d880 2023-10-16T11:06:08.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.679+0000 7f8b767fc640 1 -- 172.21.15.84:0/1077143986 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b700343f0 con 0x7f8b7815d880 2023-10-16T11:06:08.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.680+0000 7f8b7d533640 1 -- 172.21.15.84:0/1077143986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7815d880 msgr2=0x7f8b780b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:08.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.680+0000 7f8b7d533640 1 --2- 172.21.15.84:0/1077143986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7815d880 0x7f8b780b0950 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009e50 tx=0x7f8b7002bf60 comp rx=0 tx=0).stop 2023-10-16T11:06:08.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.680+0000 7f8b7d533640 1 -- 172.21.15.84:0/1077143986 shutdown_connections 2023-10-16T11:06:08.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.680+0000 7f8b7d533640 1 --2- 172.21.15.84:0/1077143986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7815d880 0x7f8b780b0950 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.680+0000 7f8b7d533640 1 --2- 172.21.15.84:0/1077143986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b78159f00 0x7f8b780b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.680+0000 7f8b7d533640 1 -- 172.21.15.84:0/1077143986 >> 172.21.15.84:0/1077143986 conn(0x7f8b780cb410 msgr2=0x7f8b780cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:08.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.680+0000 7f8b7d533640 1 -- 172.21.15.84:0/1077143986 shutdown_connections 2023-10-16T11:06:08.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.680+0000 7f8b7d533640 1 -- 172.21.15.84:0/1077143986 wait complete. 2023-10-16T11:06:08.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.682+0000 7f8b7d533640 1 Processor -- start 2023-10-16T11:06:08.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.683+0000 7f8b7d533640 1 -- start start 2023-10-16T11:06:08.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.684+0000 7f8b7d533640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b78159b20 0x7f8b780ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:08.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.684+0000 7f8b7d533640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b78159f00 0x7f8b78150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:08.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.684+0000 7f8b7d533640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7815d880 0x7f8b781588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:08.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.684+0000 7f8b7d533640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b780c5a60 con 0x7f8b78159b20 2023-10-16T11:06:08.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.684+0000 7f8b7d533640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7815bb20 con 0x7f8b7815d880 2023-10-16T11:06:08.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.684+0000 7f8b7d533640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b78151490 con 0x7f8b78159f00 2023-10-16T11:06:08.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.684+0000 7f8b76ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b78159b20 0x7f8b780ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:08.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.684+0000 7f8b76ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b78159b20 0x7f8b780ae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56318/0 (socket says 172.21.15.84:56318) 2023-10-16T11:06:08.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.684+0000 7f8b76ffd640 1 -- 172.21.15.84:0/2320918117 learned_addr learned my addr 172.21.15.84:0/2320918117 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:08.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.684+0000 7f8b6e7fc640 1 --2- 172.21.15.84:0/2320918117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b78159f00 0x7f8b78150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:08.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.685+0000 7f8b76ffd640 1 -- 172.21.15.84:0/2320918117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b78159f00 msgr2=0x7f8b78150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:08.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.685+0000 7f8b76ffd640 1 --2- 172.21.15.84:0/2320918117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b78159f00 0x7f8b78150f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.685+0000 7f8b76ffd640 1 -- 172.21.15.84:0/2320918117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7815d880 msgr2=0x7f8b781588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:08.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.685+0000 7f8b76ffd640 1 --2- 172.21.15.84:0/2320918117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7815d880 0x7f8b781588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.685+0000 7f8b76ffd640 1 -- 172.21.15.84:0/2320918117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b60009400 con 0x7f8b78159b20 2023-10-16T11:06:08.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.685+0000 7f8b76ffd640 1 --2- 172.21.15.84:0/2320918117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b78159b20 0x7f8b780ae3d0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800c420 tx=0x7f8b6800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:08.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.685+0000 7f8b6ffff640 1 -- 172.21.15.84:0/2320918117 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b680074e0 con 0x7f8b78159b20 2023-10-16T11:06:08.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.685+0000 7f8b6ffff640 1 -- 172.21.15.84:0/2320918117 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b68007640 con 0x7f8b78159b20 2023-10-16T11:06:08.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.685+0000 7f8b7d533640 1 -- 172.21.15.84:0/2320918117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b70009b30 con 0x7f8b78159b20 2023-10-16T11:06:08.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.685+0000 7f8b6ffff640 1 -- 172.21.15.84:0/2320918117 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b68004a00 con 0x7f8b78159b20 2023-10-16T11:06:08.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.685+0000 7f8b7d533640 1 -- 172.21.15.84:0/2320918117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b780af370 con 0x7f8b78159b20 2023-10-16T11:06:08.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.687+0000 7f8b7d533640 1 -- 172.21.15.84:0/2320918117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7815b8f0 con 0x7f8b78159b20 2023-10-16T11:06:08.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.688+0000 7f8b6ffff640 1 -- 172.21.15.84:0/2320918117 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 102) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f8b68004b60 con 0x7f8b78159b20 2023-10-16T11:06:08.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.689+0000 7f8b6ffff640 1 --2- 172.21.15.84:0/2320918117 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f8b5c06af30 0x7f8b5c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:08.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.689+0000 7f8b6ffff640 1 -- 172.21.15.84:0/2320918117 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f8b680896b0 con 0x7f8b78159b20 2023-10-16T11:06:08.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.689+0000 7f8b6e7fc640 1 --2- 172.21.15.84:0/2320918117 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f8b5c06af30 0x7f8b5c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:08.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.690+0000 7f8b6e7fc640 1 --2- 172.21.15.84:0/2320918117 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f8b5c06af30 0x7f8b5c06d3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8b780cc800 tx=0x7f8b60008040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:08.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.693+0000 7f8b6ffff640 1 -- 172.21.15.84:0/2320918117 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8b7815b8f0 con 0x7f8b78159b20 2023-10-16T11:06:08.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.886+0000 7f8b7d533640 1 -- 172.21.15.84:0/2320918117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b7815b8f0 con 0x7f8b78159b20 2023-10-16T11:06:08.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.888+0000 7f8b6ffff640 1 -- 172.21.15.84:0/2320918117 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+17601 (secure 0 0 0) 0x7f8b7815b8f0 con 0x7f8b78159b20 2023-10-16T11:06:08.890 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:08.890 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":173,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:06:05.639308+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:05:41.217474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-16T11:05:42.387743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/849217271":"2023-10-17T11:06:05.639149+0000","172.21.15.84:0/1510301035":"2023-10-17T11:06:05.639149+0000","172.21.15.84:0/2220106058":"2023-10-17T11:06:05.639149+0000","172.21.15.84:6833/1697458478":"2023-10-17T11:06:05.639149+0000","172.21.15.84:0/1365551239":"2023-10-17T11:06:05.639149+0000","172.21.15.84:0/2246425640":"2023-10-17T11:06:05.639149+0000","172.21.15.84:0/3425147335":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/1161603770":"2023-10-17T11:05:57.603800+0000","172.21.15.84:6832/1697458478":"2023-10-17T11:06:05.639149+0000","172.21.15.84:6833/2442400184":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/3742714530":"2023-10-17T11:06:05.639149+0000","172.21.15.84:6832/2442400184":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/3633178028":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/1914327915":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/2602045766":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/3065897528":"2023-10-17T11:05:57.603800+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:06:08.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.890+0000 7f8b7d533640 1 -- 172.21.15.84:0/2320918117 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f8b5c06af30 msgr2=0x7f8b5c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:08.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.890+0000 7f8b7d533640 1 --2- 172.21.15.84:0/2320918117 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f8b5c06af30 0x7f8b5c06d3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8b780cc800 tx=0x7f8b60008040 comp rx=0 tx=0).stop 2023-10-16T11:06:08.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.890+0000 7f8b7d533640 1 -- 172.21.15.84:0/2320918117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b78159b20 msgr2=0x7f8b780ae3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:08.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.890+0000 7f8b7d533640 1 --2- 172.21.15.84:0/2320918117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b78159b20 0x7f8b780ae3d0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800c420 tx=0x7f8b6800c8e0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.891+0000 7f8b7d533640 1 -- 172.21.15.84:0/2320918117 shutdown_connections 2023-10-16T11:06:08.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.891+0000 7f8b7d533640 1 --2- 172.21.15.84:0/2320918117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7815d880 0x7f8b781588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.891+0000 7f8b7d533640 1 --2- 172.21.15.84:0/2320918117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b78159f00 0x7f8b78150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.891+0000 7f8b7d533640 1 --2- 172.21.15.84:0/2320918117 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f8b5c06af30 0x7f8b5c06d3f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.891+0000 7f8b7d533640 1 --2- 172.21.15.84:0/2320918117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b78159b20 0x7f8b780ae3d0 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:08.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.891+0000 7f8b7d533640 1 -- 172.21.15.84:0/2320918117 >> 172.21.15.84:0/2320918117 conn(0x7f8b780cb410 msgr2=0x7f8b780cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:08.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.891+0000 7f8b7d533640 1 -- 172.21.15.84:0/2320918117 shutdown_connections 2023-10-16T11:06:08.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:08.891+0000 7f8b7d533640 1 -- 172.21.15.84:0/2320918117 wait complete. 2023-10-16T11:06:08.912 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T11:06:09.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.126+0000 7fbccd748640 1 Processor -- start 2023-10-16T11:06:09.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.126+0000 7fbccd748640 1 -- start start 2023-10-16T11:06:09.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.127+0000 7fbccd748640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcc815d330 0x7fbcc81618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:09.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.127+0000 7fbccd748640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbcc80661c0 0x7fbcc80ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:09.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.127+0000 7fbccd748640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc80d0890 con 0x7fbcc806c7d0 2023-10-16T11:06:09.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.127+0000 7fbccd748640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc806b1e0 con 0x7fbcc80661c0 2023-10-16T11:06:09.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.127+0000 7fbccd748640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc80d05b0 con 0x7fbcc815d330 2023-10-16T11:06:09.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.128+0000 7fbcc77fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbcc80661c0 0x7fbcc80ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:09.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.128+0000 7fbcc77fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbcc80661c0 0x7fbcc80ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:42786/0 (socket says 172.21.15.84:42786) 2023-10-16T11:06:09.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.128+0000 7fbcc77fe640 1 -- 172.21.15.84:0/2554287948 learned_addr learned my addr 172.21.15.84:0/2554287948 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:09.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.128+0000 7fbcc67fc640 1 --1- 172.21.15.84:0/2554287948 >> v1:172.21.15.157:6789/0 conn(0x7fbcc806c7d0 0x7fbcc8065ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:36430/0 (socket says 172.21.15.84:36430) 2023-10-16T11:06:09.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.128+0000 7fbcc6ffd640 1 --2- 172.21.15.84:0/2554287948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcc815d330 0x7fbcc81618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:09.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.129+0000 7fbcc77fe640 1 -- 172.21.15.84:0/2554287948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcc815d330 msgr2=0x7fbcc81618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:09.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.129+0000 7fbcc77fe640 1 --2- 172.21.15.84:0/2554287948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcc815d330 0x7fbcc81618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:09.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.129+0000 7fbcc77fe640 1 -- 172.21.15.84:0/2554287948 >> v1:172.21.15.157:6789/0 conn(0x7fbcc806c7d0 legacy=0x7fbcc8065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:09.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.129+0000 7fbcc77fe640 1 -- 172.21.15.84:0/2554287948 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc8160dd0 con 0x7fbcc80661c0 2023-10-16T11:06:09.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.129+0000 7fbcc6ffd640 1 --2- 172.21.15.84:0/2554287948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcc815d330 0x7fbcc81618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:06:09.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.130+0000 7fbcc77fe640 1 --2- 172.21.15.84:0/2554287948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcc80661c0 0x7fbcc80ce9b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009870 tx=0x7fbcc002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e57bc107a9906032 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:09.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.131+0000 7fbcc5ffb640 1 -- 172.21.15.84:0/2554287948 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0031640 con 0x7fbcc80661c0 2023-10-16T11:06:09.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.131+0000 7fbcc5ffb640 1 -- 172.21.15.84:0/2554287948 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc00317a0 con 0x7fbcc80661c0 2023-10-16T11:06:09.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.131+0000 7fbcc5ffb640 1 -- 172.21.15.84:0/2554287948 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0031b30 con 0x7fbcc80661c0 2023-10-16T11:06:09.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.131+0000 7fbccd748640 1 -- 172.21.15.84:0/2554287948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcc80661c0 msgr2=0x7fbcc80ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:09.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.131+0000 7fbccd748640 1 --2- 172.21.15.84:0/2554287948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcc80661c0 0x7fbcc80ce9b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009870 tx=0x7fbcc002b8f0 comp rx=0 tx=0).stop 2023-10-16T11:06:09.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.131+0000 7fbccd748640 1 -- 172.21.15.84:0/2554287948 shutdown_connections 2023-10-16T11:06:09.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.131+0000 7fbccd748640 1 --2- 172.21.15.84:0/2554287948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcc80661c0 0x7fbcc80ce9b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:09.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.131+0000 7fbccd748640 1 --2- 172.21.15.84:0/2554287948 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcc815d330 0x7fbcc81618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:09.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.131+0000 7fbccd748640 1 -- 172.21.15.84:0/2554287948 >> 172.21.15.84:0/2554287948 conn(0x7fbcc80d0ab0 msgr2=0x7fbcc80d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:09.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.131+0000 7fbccd748640 1 -- 172.21.15.84:0/2554287948 shutdown_connections 2023-10-16T11:06:09.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.131+0000 7fbccd748640 1 -- 172.21.15.84:0/2554287948 wait complete. 2023-10-16T11:06:09.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.133+0000 7fbccd748640 1 Processor -- start 2023-10-16T11:06:09.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.133+0000 7fbccd748640 1 -- start start 2023-10-16T11:06:09.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.134+0000 7fbccd748640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcc80661c0 0x7fbcc8154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:09.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.134+0000 7fbccd748640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcc806c7d0 0x7fbcc815f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:09.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.134+0000 7fbccd748640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcc815d330 0x7fbcc81f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:09.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.134+0000 7fbccd748640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc80c7af0 con 0x7fbcc806c7d0 2023-10-16T11:06:09.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.134+0000 7fbccd748640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc815fed0 con 0x7fbcc80661c0 2023-10-16T11:06:09.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.134+0000 7fbccd748640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc80c7d40 con 0x7fbcc815d330 2023-10-16T11:06:09.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.135+0000 7fbcc67fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcc806c7d0 0x7fbcc815f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:09.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.135+0000 7fbcc67fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcc806c7d0 0x7fbcc815f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56330/0 (socket says 172.21.15.84:56330) 2023-10-16T11:06:09.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.135+0000 7fbcc67fc640 1 -- 172.21.15.84:0/1418658433 learned_addr learned my addr 172.21.15.84:0/1418658433 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:09.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.135+0000 7fbcc77fe640 1 --2- 172.21.15.84:0/1418658433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcc815d330 0x7fbcc81f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:09.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.135+0000 7fbcc6ffd640 1 --2- 172.21.15.84:0/1418658433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcc80661c0 0x7fbcc8154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:09.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.135+0000 7fbcc67fc640 1 -- 172.21.15.84:0/1418658433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcc815d330 msgr2=0x7fbcc81f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:09.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.135+0000 7fbcc67fc640 1 --2- 172.21.15.84:0/1418658433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcc815d330 0x7fbcc81f3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:09.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.135+0000 7fbcc67fc640 1 -- 172.21.15.84:0/1418658433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcc80661c0 msgr2=0x7fbcc8154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:09.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.135+0000 7fbcc67fc640 1 --2- 172.21.15.84:0/1418658433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcc80661c0 0x7fbcc8154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:09.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.135+0000 7fbcc67fc640 1 -- 172.21.15.84:0/1418658433 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb8009a70 con 0x7fbcc806c7d0 2023-10-16T11:06:09.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.135+0000 7fbcc6ffd640 1 --2- 172.21.15.84:0/1418658433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcc80661c0 0x7fbcc8154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:09.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.135+0000 7fbcc77fe640 1 --2- 172.21.15.84:0/1418658433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcc815d330 0x7fbcc81f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:09.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.135+0000 7fbcc67fc640 1 --2- 172.21.15.84:0/1418658433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcc806c7d0 0x7fbcc815f990 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000c3c0 tx=0x7fbcb000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:09.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.136+0000 7fbcbf7fe640 1 -- 172.21.15.84:0/1418658433 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00074e0 con 0x7fbcc806c7d0 2023-10-16T11:06:09.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.136+0000 7fbcbf7fe640 1 -- 172.21.15.84:0/1418658433 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb0007640 con 0x7fbcc806c7d0 2023-10-16T11:06:09.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.136+0000 7fbcbf7fe640 1 -- 172.21.15.84:0/1418658433 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0004a00 con 0x7fbcc806c7d0 2023-10-16T11:06:09.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.136+0000 7fbccd748640 1 -- 172.21.15.84:0/1418658433 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc00094d0 con 0x7fbcc806c7d0 2023-10-16T11:06:09.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.136+0000 7fbccd748640 1 -- 172.21.15.84:0/1418658433 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc80c8300 con 0x7fbcc806c7d0 2023-10-16T11:06:09.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.137+0000 7fbccd748640 1 -- 172.21.15.84:0/1418658433 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc804e160 con 0x7fbcc806c7d0 2023-10-16T11:06:09.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.138+0000 7fbcbf7fe640 1 -- 172.21.15.84:0/1418658433 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 102) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fbcb0004b60 con 0x7fbcc806c7d0 2023-10-16T11:06:09.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.139+0000 7fbcbf7fe640 1 --2- 172.21.15.84:0/1418658433 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fbca806af00 0x7fbca806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:09.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.139+0000 7fbcbf7fe640 1 -- 172.21.15.84:0/1418658433 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6425+0+0 (secure 0 0 0) 0x7fbcb0089390 con 0x7fbcc806c7d0 2023-10-16T11:06:09.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.139+0000 7fbcc6ffd640 1 --2- 172.21.15.84:0/1418658433 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fbca806af00 0x7fbca806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:09.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.140+0000 7fbcc6ffd640 1 --2- 172.21.15.84:0/1418658433 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fbca806af00 0x7fbca806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009fd0 tx=0x7fbcb800b040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:09.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.143+0000 7fbcbf7fe640 1 -- 172.21.15.84:0/1418658433 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbcb0054130 con 0x7fbcc806c7d0 2023-10-16T11:06:09.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.376+0000 7fbccd748640 1 -- 172.21.15.84:0/1418658433 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbcc806f3a0 con 0x7fbcc806c7d0 2023-10-16T11:06:09.378 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-16T11:06:09.377+0000 7fca0c7ff640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:06:09.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.381+0000 7fbcbf7fe640 1 -- 172.21.15.84:0/1418658433 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v89) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbcb0057750 con 0x7fbcc806c7d0 2023-10-16T11:06:09.382 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:06:09.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.385+0000 7fbccd748640 1 -- 172.21.15.84:0/1418658433 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fbca806af00 msgr2=0x7fbca806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:09.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.385+0000 7fbccd748640 1 --2- 172.21.15.84:0/1418658433 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fbca806af00 0x7fbca806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009fd0 tx=0x7fbcb800b040 comp rx=0 tx=0).stop 2023-10-16T11:06:09.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.385+0000 7fbccd748640 1 -- 172.21.15.84:0/1418658433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcc806c7d0 msgr2=0x7fbcc815f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:09.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.385+0000 7fbccd748640 1 --2- 172.21.15.84:0/1418658433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcc806c7d0 0x7fbcc815f990 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000c3c0 tx=0x7fbcb000c880 comp rx=0 tx=0).stop 2023-10-16T11:06:09.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.385+0000 7fbccd748640 1 -- 172.21.15.84:0/1418658433 shutdown_connections 2023-10-16T11:06:09.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.385+0000 7fbccd748640 1 --2- 172.21.15.84:0/1418658433 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcc815d330 0x7fbcc81f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:09.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.385+0000 7fbccd748640 1 --2- 172.21.15.84:0/1418658433 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fbca806af00 0x7fbca806d3c0 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009fd0 tx=0x7fbcb800b040 comp rx=0 tx=0).stop 2023-10-16T11:06:09.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.385+0000 7fbccd748640 1 --2- 172.21.15.84:0/1418658433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcc806c7d0 0x7fbcc815f990 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:09.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.385+0000 7fbccd748640 1 --2- 172.21.15.84:0/1418658433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcc80661c0 0x7fbcc8154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:09.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.385+0000 7fbccd748640 1 -- 172.21.15.84:0/1418658433 >> 172.21.15.84:0/1418658433 conn(0x7fbcc80d0ab0 msgr2=0x7fbcc8155260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:09.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.386+0000 7fbccd748640 1 -- 172.21.15.84:0/1418658433 shutdown_connections 2023-10-16T11:06:09.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.387+0000 7fbccd748640 1 -- 172.21.15.84:0/1418658433 wait complete. 2023-10-16T11:06:09.398 INFO:tasks.ceph.mds.c.smithi084.stderr: -3933> 2023-10-16T11:05:44.855+0000 7f95f422c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:06:09.415 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T11:06:09.458 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T11:06:09.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.608+0000 7f375d461640 1 Processor -- start 2023-10-16T11:06:09.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.608+0000 7f375d461640 1 -- start start 2023-10-16T11:06:09.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.610+0000 7f375d461640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3758159ab0 0x7f37580ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:09.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.610+0000 7f375d461640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f375806c140 0x7f375806c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:09.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.610+0000 7f375d461640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375806cad0 con 0x7f3758159e90 2023-10-16T11:06:09.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.610+0000 7f375d461640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375815d890 con 0x7f375806c140 2023-10-16T11:06:09.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.610+0000 7f375d461640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3758070360 con 0x7f3758159ab0 2023-10-16T11:06:09.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.610+0000 7f37577fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f375806c140 0x7f375806c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:09.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.610+0000 7f37577fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f375806c140 0x7f375806c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56358/0 (socket says 172.21.15.84:56358) 2023-10-16T11:06:09.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.610+0000 7f37577fe640 1 -- 172.21.15.84:0/3500517376 learned_addr learned my addr 172.21.15.84:0/3500517376 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:09.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.610+0000 7f3756ffd640 1 --2- 172.21.15.84:0/3500517376 >> v2:172.21.15.157:3300/0 conn(0x7f3758159ab0 0x7f37580ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:09.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.611+0000 7f37577fe640 1 -- 172.21.15.84:0/3500517376 >> v2:172.21.15.157:3300/0 conn(0x7f3758159ab0 msgr2=0x7f37580ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:09.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.611+0000 7f37577fe640 1 --2- 172.21.15.84:0/3500517376 >> v2:172.21.15.157:3300/0 conn(0x7f3758159ab0 0x7f37580ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:09.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.611+0000 7f37577fe640 1 -- 172.21.15.84:0/3500517376 >> v1:172.21.15.84:6789/0 conn(0x7f3758159e90 legacy=0x7f375806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:09.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.611+0000 7f37577fe640 1 -- 172.21.15.84:0/3500517376 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37580cd1f0 con 0x7f375806c140 2023-10-16T11:06:09.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.611+0000 7f3756ffd640 1 --2- 172.21.15.84:0/3500517376 >> v2:172.21.15.157:3300/0 conn(0x7f3758159ab0 0x7f37580ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:09.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.612+0000 7f37577fe640 1 --2- 172.21.15.84:0/3500517376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f375806c140 0x7f375806c590 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f37580cfa00 tx=0x7f375002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699b0bd4dde11f24 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:09.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.612+0000 7f37567fc640 1 -- 172.21.15.84:0/3500517376 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375002c9f0 con 0x7f375806c140 2023-10-16T11:06:09.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.612+0000 7f37567fc640 1 -- 172.21.15.84:0/3500517376 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375002cb50 con 0x7f375806c140 2023-10-16T11:06:09.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.612+0000 7f37567fc640 1 -- 172.21.15.84:0/3500517376 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37500343f0 con 0x7f375806c140 2023-10-16T11:06:09.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.613+0000 7f375d461640 1 -- 172.21.15.84:0/3500517376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f375806c140 msgr2=0x7f375806c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:09.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.613+0000 7f375d461640 1 --2- 172.21.15.84:0/3500517376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f375806c140 0x7f375806c590 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f37580cfa00 tx=0x7f375002be50 comp rx=0 tx=0).stop 2023-10-16T11:06:09.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.613+0000 7f375d461640 1 -- 172.21.15.84:0/3500517376 shutdown_connections 2023-10-16T11:06:09.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.613+0000 7f375d461640 1 --2- 172.21.15.84:0/3500517376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f375806c140 0x7f375806c590 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:09.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.613+0000 7f375d461640 1 --2- 172.21.15.84:0/3500517376 >> v2:172.21.15.157:3300/0 conn(0x7f3758159ab0 0x7f37580ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:09.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.613+0000 7f375d461640 1 -- 172.21.15.84:0/3500517376 >> 172.21.15.84:0/3500517376 conn(0x7f37580ce9b0 msgr2=0x7f37580cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:09.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.613+0000 7f375d461640 1 -- 172.21.15.84:0/3500517376 shutdown_connections 2023-10-16T11:06:09.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.613+0000 7f375d461640 1 -- 172.21.15.84:0/3500517376 wait complete. 2023-10-16T11:06:09.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.615+0000 7f375d461640 1 Processor -- start 2023-10-16T11:06:09.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.615+0000 7f375d461640 1 -- start start 2023-10-16T11:06:09.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.616+0000 7f375d461640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f375806c140 0x7f37580c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:09.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.616+0000 7f375d461640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3758159ab0 0x7f375806b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:09.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.616+0000 7f375d461640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3758159e90 0x7f375806b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:09.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.616+0000 7f375d461640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37580c5420 con 0x7f3758159e90 2023-10-16T11:06:09.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.616+0000 7f375d461640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37580c9930 con 0x7f375806c140 2023-10-16T11:06:09.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.617+0000 7f375d461640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37580c9650 con 0x7f3758159ab0 2023-10-16T11:06:09.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.617+0000 7f37577fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3758159e90 0x7f375806b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:09.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.617+0000 7f37577fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3758159e90 0x7f375806b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56364/0 (socket says 172.21.15.84:56364) 2023-10-16T11:06:09.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.617+0000 7f37577fe640 1 -- 172.21.15.84:0/2642653937 learned_addr learned my addr 172.21.15.84:0/2642653937 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:09.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.617+0000 7f37577fe640 1 -- 172.21.15.84:0/2642653937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3758159ab0 msgr2=0x7f375806b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:09.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.617+0000 7f37577fe640 1 --2- 172.21.15.84:0/2642653937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3758159ab0 0x7f375806b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:09.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.617+0000 7f37577fe640 1 -- 172.21.15.84:0/2642653937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f375806c140 msgr2=0x7f37580c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:09.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.617+0000 7f3756ffd640 1 --2- 172.21.15.84:0/2642653937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f375806c140 0x7f37580c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:09.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.617+0000 7f37577fe640 1 --2- 172.21.15.84:0/2642653937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f375806c140 0x7f37580c46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:09.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.617+0000 7f37577fe640 1 -- 172.21.15.84:0/2642653937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3750009b30 con 0x7f3758159e90 2023-10-16T11:06:09.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.617+0000 7f3756ffd640 1 --2- 172.21.15.84:0/2642653937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f375806c140 0x7f37580c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:09.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.618+0000 7f37577fe640 1 --2- 172.21.15.84:0/2642653937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3758159e90 0x7f375806b570 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f3750036040 tx=0x7f3750004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:09.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.618+0000 7f374f7fe640 1 -- 172.21.15.84:0/2642653937 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3750009510 con 0x7f3758159e90 2023-10-16T11:06:09.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.618+0000 7f374f7fe640 1 -- 172.21.15.84:0/2642653937 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375003e450 con 0x7f3758159e90 2023-10-16T11:06:09.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.618+0000 7f374f7fe640 1 -- 172.21.15.84:0/2642653937 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375003e800 con 0x7f3758159e90 2023-10-16T11:06:09.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.618+0000 7f375d461640 1 -- 172.21.15.84:0/2642653937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37580c4e60 con 0x7f3758159e90 2023-10-16T11:06:09.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.618+0000 7f375d461640 1 -- 172.21.15.84:0/2642653937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37580c92b0 con 0x7f3758159e90 2023-10-16T11:06:09.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.619+0000 7f375d461640 1 -- 172.21.15.84:0/2642653937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f375804e160 con 0x7f3758159e90 2023-10-16T11:06:09.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.620+0000 7f374f7fe640 1 -- 172.21.15.84:0/2642653937 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 102) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f375003e960 con 0x7f3758159e90 2023-10-16T11:06:09.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.621+0000 7f374f7fe640 1 --2- 172.21.15.84:0/2642653937 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f373806af00 0x7f373806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:09.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.621+0000 7f374f7fe640 1 -- 172.21.15.84:0/2642653937 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6511+0+0 (secure 0 0 0) 0x7f37500ac900 con 0x7f3758159e90 2023-10-16T11:06:09.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.621+0000 7f3756ffd640 1 --2- 172.21.15.84:0/2642653937 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f373806af00 0x7f373806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:09.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.622+0000 7f3756ffd640 1 --2- 172.21.15.84:0/2642653937 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f373806af00 0x7f373806d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3748006fd0 tx=0x7f3748008040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:09.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.626+0000 7f374f7fe640 1 -- 172.21.15.84:0/2642653937 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3750077770 con 0x7f3758159e90 2023-10-16T11:06:09.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:09.852+0000 7f375d461640 1 -- 172.21.15.84:0/2642653937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f37580ccb60 con 0x7f3758159e90 2023-10-16T11:06:10.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.428+0000 7f374f7fe640 1 -- 172.21.15.84:0/2642653937 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v90) v1 ==== 104+0+0 (secure 0 0 0) 0x7f375007ad90 con 0x7f3758159e90 2023-10-16T11:06:10.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.432+0000 7f375d461640 1 -- 172.21.15.84:0/2642653937 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f373806af00 msgr2=0x7f373806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:10.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.432+0000 7f375d461640 1 --2- 172.21.15.84:0/2642653937 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f373806af00 0x7f373806d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3748006fd0 tx=0x7f3748008040 comp rx=0 tx=0).stop 2023-10-16T11:06:10.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.432+0000 7f375d461640 1 -- 172.21.15.84:0/2642653937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3758159e90 msgr2=0x7f375806b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:10.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.432+0000 7f375d461640 1 --2- 172.21.15.84:0/2642653937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3758159e90 0x7f375806b570 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f3750036040 tx=0x7f3750004a40 comp rx=0 tx=0).stop 2023-10-16T11:06:10.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.432+0000 7f375d461640 1 -- 172.21.15.84:0/2642653937 shutdown_connections 2023-10-16T11:06:10.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.432+0000 7f375d461640 1 --2- 172.21.15.84:0/2642653937 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f373806af00 0x7f373806d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:10.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.432+0000 7f375d461640 1 --2- 172.21.15.84:0/2642653937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3758159e90 0x7f375806b570 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:10.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.432+0000 7f375d461640 1 --2- 172.21.15.84:0/2642653937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3758159ab0 0x7f375806b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:10.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.432+0000 7f375d461640 1 --2- 172.21.15.84:0/2642653937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f375806c140 0x7f37580c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:10.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.432+0000 7f375d461640 1 -- 172.21.15.84:0/2642653937 >> 172.21.15.84:0/2642653937 conn(0x7f37580ce9b0 msgr2=0x7f37580ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:10.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.432+0000 7f375d461640 1 -- 172.21.15.84:0/2642653937 shutdown_connections 2023-10-16T11:06:10.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.432+0000 7f375d461640 1 -- 172.21.15.84:0/2642653937 wait complete. 2023-10-16T11:06:10.458 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T11:06:10.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.717+0000 7f1b8302b640 1 Processor -- start 2023-10-16T11:06:10.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.718+0000 7f1b8302b640 1 -- start start 2023-10-16T11:06:10.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.719+0000 7f1b8302b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1b7c15cf60 0x7f1b7c06f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:10.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.719+0000 7f1b8302b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b7c0b0360 0x7f1b7c0b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:10.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.719+0000 7f1b8302b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b7c0c9620 con 0x7f1b7c06f9d0 2023-10-16T11:06:10.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.719+0000 7f1b8302b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b7c15d340 con 0x7f1b7c15cf60 2023-10-16T11:06:10.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.719+0000 7f1b8302b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c0cc860 con 0x7f1b7c0b0360 2023-10-16T11:06:10.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.719+0000 7f1b7bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1b7c06f9d0 0x7f1b7c0c9ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55334/0 (socket says 172.21.15.84:55334) 2023-10-16T11:06:10.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.720+0000 7f1b7bfff640 1 -- 172.21.15.84:0/884823977 learned_addr learned my addr 172.21.15.84:0/884823977 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:10.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.720+0000 7f1b80dc7640 1 --2- 172.21.15.84:0/884823977 >> v2:172.21.15.157:3300/0 conn(0x7f1b7c15cf60 0x7f1b7c06f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:10.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.720+0000 7f1b815c8640 1 --2- 172.21.15.84:0/884823977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b7c0b0360 0x7f1b7c0b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:10.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.720+0000 7f1b7b7fe640 1 -- 172.21.15.84:0/884823977 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523521350 0 0) 0x7f1b7c0c9620 con 0x7f1b7c06f9d0 2023-10-16T11:06:10.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.720+0000 7f1b7b7fe640 1 -- 172.21.15.84:0/884823977 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b64003520 con 0x7f1b7c06f9d0 2023-10-16T11:06:10.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.720+0000 7f1b7b7fe640 1 -- 172.21.15.84:0/884823977 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3483666872 0 0) 0x7f1b64003520 con 0x7f1b7c06f9d0 2023-10-16T11:06:10.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.720+0000 7f1b7b7fe640 1 -- 172.21.15.84:0/884823977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b7c0b0360 msgr2=0x7f1b7c0b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:10.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.720+0000 7f1b7b7fe640 1 --2- 172.21.15.84:0/884823977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b7c0b0360 0x7f1b7c0b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:10.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.720+0000 7f1b7b7fe640 1 -- 172.21.15.84:0/884823977 >> v2:172.21.15.157:3300/0 conn(0x7f1b7c15cf60 msgr2=0x7f1b7c06f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:10.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.720+0000 7f1b7b7fe640 1 --2- 172.21.15.84:0/884823977 >> v2:172.21.15.157:3300/0 conn(0x7f1b7c15cf60 0x7f1b7c06f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:10.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.720+0000 7f1b7b7fe640 1 -- 172.21.15.84:0/884823977 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7c0ad4a0 con 0x7f1b7c06f9d0 2023-10-16T11:06:10.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.720+0000 7f1b7b7fe640 1 -- 172.21.15.84:0/884823977 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f1b6c001670 con 0x7f1b7c06f9d0 2023-10-16T11:06:10.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.720+0000 7f1b7b7fe640 1 -- 172.21.15.84:0/884823977 >> v1:172.21.15.84:6789/0 conn(0x7f1b7c06f9d0 legacy=0x7f1b7c0c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:10.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.721+0000 7f1b7b7fe640 1 --2- 172.21.15.84:0/884823977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b640039e0 0x7f1b64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:10.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.721+0000 7f1b7b7fe640 1 --2- 172.21.15.84:0/884823977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b640047c0 0x7f1b64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:10.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.721+0000 7f1b7b7fe640 1 --2- 172.21.15.84:0/884823977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b7c0b0360 0x7f1b64005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:10.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.721+0000 7f1b7b7fe640 1 -- 172.21.15.84:0/884823977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c0cc860 con 0x7f1b640047c0 2023-10-16T11:06:10.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.721+0000 7f1b7b7fe640 1 -- 172.21.15.84:0/884823977 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6400b980 con 0x7f1b640039e0 2023-10-16T11:06:10.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.721+0000 7f1b7b7fe640 1 -- 172.21.15.84:0/884823977 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6400baf0 con 0x7f1b7c0b0360 2023-10-16T11:06:10.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.721+0000 7f1b7bfff640 1 --2- 172.21.15.84:0/884823977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b640047c0 0x7f1b64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:10.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.721+0000 7f1b80dc7640 1 --2- 172.21.15.84:0/884823977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b640039e0 0x7f1b64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:10.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.721+0000 7f1b7bfff640 1 -- 172.21.15.84:0/884823977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b7c0b0360 msgr2=0x7f1b64005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:10.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.721+0000 7f1b7bfff640 1 --2- 172.21.15.84:0/884823977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b7c0b0360 0x7f1b64005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:10.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.721+0000 7f1b7bfff640 1 -- 172.21.15.84:0/884823977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b640039e0 msgr2=0x7f1b64003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:10.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.721+0000 7f1b7bfff640 1 --2- 172.21.15.84:0/884823977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b640039e0 0x7f1b64003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:10.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.721+0000 7f1b7bfff640 1 -- 172.21.15.84:0/884823977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b64003520 con 0x7f1b640047c0 2023-10-16T11:06:10.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.722+0000 7f1b7bfff640 1 --2- 172.21.15.84:0/884823977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b640047c0 0x7f1b64004c40 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00c0e0 tx=0x7f1b640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7afe9779d51cfdac server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:10.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.723+0000 7f1b7b7fe640 1 -- 172.21.15.84:0/884823977 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c005ad0 con 0x7f1b640047c0 2023-10-16T11:06:10.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.723+0000 7f1b7b7fe640 1 -- 172.21.15.84:0/884823977 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6c005c30 con 0x7f1b640047c0 2023-10-16T11:06:10.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.723+0000 7f1b7b7fe640 1 -- 172.21.15.84:0/884823977 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c02c810 con 0x7f1b640047c0 2023-10-16T11:06:10.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.723+0000 7f1b8302b640 1 -- 172.21.15.84:0/884823977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b640047c0 msgr2=0x7f1b64004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:10.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.723+0000 7f1b8302b640 1 --2- 172.21.15.84:0/884823977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b640047c0 0x7f1b64004c40 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00c0e0 tx=0x7f1b640057f0 comp rx=0 tx=0).stop 2023-10-16T11:06:10.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.723+0000 7f1b815c8640 1 -- 172.21.15.84:0/884823977 reap_dead start 2023-10-16T11:06:10.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.723+0000 7f1b8302b640 1 -- 172.21.15.84:0/884823977 shutdown_connections 2023-10-16T11:06:10.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.723+0000 7f1b8302b640 1 -- 172.21.15.84:0/884823977 >> 172.21.15.84:0/884823977 conn(0x7f1b7c0cb360 msgr2=0x7f1b7c0cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:10.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.723+0000 7f1b8302b640 1 -- 172.21.15.84:0/884823977 shutdown_connections 2023-10-16T11:06:10.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.723+0000 7f1b8302b640 1 -- 172.21.15.84:0/884823977 wait complete. 2023-10-16T11:06:10.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.725+0000 7f1b8302b640 1 Processor -- start 2023-10-16T11:06:10.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.725+0000 7f1b8302b640 1 -- start start 2023-10-16T11:06:10.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.726+0000 7f1b8302b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b640047c0 0x7f1b7c148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:10.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.726+0000 7f1b8302b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c14b2b0 0x7f1b7c149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:10.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.726+0000 7f1b8302b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b7c149c90 0x7f1b7c15aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:10.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.726+0000 7f1b8302b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c14ae90 con 0x7f1b640047c0 2023-10-16T11:06:10.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.726+0000 7f1b8302b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c14a210 con 0x7f1b7c14b2b0 2023-10-16T11:06:10.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.726+0000 7f1b8302b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c14a4f0 con 0x7f1b7c149c90 2023-10-16T11:06:10.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.726+0000 7f1b80dc7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b640047c0 0x7f1b7c148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:10.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.727+0000 7f1b80dc7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b640047c0 0x7f1b7c148eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56380/0 (socket says 172.21.15.84:56380) 2023-10-16T11:06:10.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.727+0000 7f1b80dc7640 1 -- 172.21.15.84:0/3015295710 learned_addr learned my addr 172.21.15.84:0/3015295710 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:10.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.727+0000 7f1b7bfff640 1 --2- 172.21.15.84:0/3015295710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c14b2b0 0x7f1b7c149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:10.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.727+0000 7f1b80dc7640 1 -- 172.21.15.84:0/3015295710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b7c149c90 msgr2=0x7f1b7c15aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:10.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.727+0000 7f1b80dc7640 1 --2- 172.21.15.84:0/3015295710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b7c149c90 0x7f1b7c15aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:10.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.727+0000 7f1b80dc7640 1 -- 172.21.15.84:0/3015295710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c14b2b0 msgr2=0x7f1b7c149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:10.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.727+0000 7f1b80dc7640 1 --2- 172.21.15.84:0/3015295710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c14b2b0 0x7f1b7c149750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:10.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.727+0000 7f1b80dc7640 1 -- 172.21.15.84:0/3015295710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7c14a7d0 con 0x7f1b640047c0 2023-10-16T11:06:10.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.727+0000 7f1b80dc7640 1 --2- 172.21.15.84:0/3015295710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b640047c0 0x7f1b7c148eb0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f1b70011590 tx=0x7f1b70011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:10.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.727+0000 7f1b797fa640 1 -- 172.21.15.84:0/3015295710 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b70015870 con 0x7f1b640047c0 2023-10-16T11:06:10.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.727+0000 7f1b797fa640 1 -- 172.21.15.84:0/3015295710 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b700159d0 con 0x7f1b640047c0 2023-10-16T11:06:10.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.727+0000 7f1b797fa640 1 -- 172.21.15.84:0/3015295710 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b70015d60 con 0x7f1b640047c0 2023-10-16T11:06:10.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.727+0000 7f1b8302b640 1 -- 172.21.15.84:0/3015295710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7c14aab0 con 0x7f1b640047c0 2023-10-16T11:06:10.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.728+0000 7f1b8302b640 1 -- 172.21.15.84:0/3015295710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7c15d560 con 0x7f1b640047c0 2023-10-16T11:06:10.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.729+0000 7f1b8302b640 1 -- 172.21.15.84:0/3015295710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b7c14b100 con 0x7f1b640047c0 2023-10-16T11:06:10.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.730+0000 7f1b797fa640 1 -- 172.21.15.84:0/3015295710 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1b7001d400 con 0x7f1b640047c0 2023-10-16T11:06:10.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.731+0000 7f1b797fa640 1 --2- 172.21.15.84:0/3015295710 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f1b6806afc0 0x7f1b6806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:10.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.731+0000 7f1b797fa640 1 -- 172.21.15.84:0/3015295710 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6511+0+0 (secure 0 0 0) 0x7f1b7008a1b0 con 0x7f1b640047c0 2023-10-16T11:06:10.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.731+0000 7f1b7bfff640 1 --2- 172.21.15.84:0/3015295710 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f1b6806afc0 0x7f1b6806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:10.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.732+0000 7f1b7bfff640 1 --2- 172.21.15.84:0/3015295710 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f1b6806afc0 0x7f1b6806d480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c034000 tx=0x7f1b6c009030 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:10.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.736+0000 7f1b797fa640 1 -- 172.21.15.84:0/3015295710 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1b70054e30 con 0x7f1b640047c0 2023-10-16T11:06:10.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:10.928+0000 7f1b8302b640 1 -- 172.21.15.84:0/3015295710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1b7c04e160 con 0x7f1b640047c0 2023-10-16T11:06:11.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.442+0000 7f1b797fa640 1 -- 172.21.15.84:0/3015295710 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v175) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1b70058450 con 0x7f1b640047c0 2023-10-16T11:06:11.444 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-16T11:06:11.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.446+0000 7f1b8302b640 1 -- 172.21.15.84:0/3015295710 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f1b6806afc0 msgr2=0x7f1b6806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:11.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.446+0000 7f1b8302b640 1 --2- 172.21.15.84:0/3015295710 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f1b6806afc0 0x7f1b6806d480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c034000 tx=0x7f1b6c009030 comp rx=0 tx=0).stop 2023-10-16T11:06:11.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.446+0000 7f1b8302b640 1 -- 172.21.15.84:0/3015295710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b640047c0 msgr2=0x7f1b7c148eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:11.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.446+0000 7f1b8302b640 1 --2- 172.21.15.84:0/3015295710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b640047c0 0x7f1b7c148eb0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f1b70011590 tx=0x7f1b70011a50 comp rx=0 tx=0).stop 2023-10-16T11:06:11.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.447+0000 7f1b8302b640 1 -- 172.21.15.84:0/3015295710 shutdown_connections 2023-10-16T11:06:11.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.447+0000 7f1b8302b640 1 --2- 172.21.15.84:0/3015295710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b7c149c90 0x7f1b7c15aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:11.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.447+0000 7f1b8302b640 1 --2- 172.21.15.84:0/3015295710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c14b2b0 0x7f1b7c149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:11.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.447+0000 7f1b8302b640 1 --2- 172.21.15.84:0/3015295710 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f1b6806afc0 0x7f1b6806d480 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:11.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.447+0000 7f1b8302b640 1 --2- 172.21.15.84:0/3015295710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b640047c0 0x7f1b7c148eb0 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:11.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.447+0000 7f1b8302b640 1 -- 172.21.15.84:0/3015295710 >> 172.21.15.84:0/3015295710 conn(0x7f1b7c0cb360 msgr2=0x7f1b7c070750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:11.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.448+0000 7f1b8302b640 1 -- 172.21.15.84:0/3015295710 shutdown_connections 2023-10-16T11:06:11.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.448+0000 7f1b8302b640 1 -- 172.21.15.84:0/3015295710 wait complete. 2023-10-16T11:06:11.475 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T11:06:11.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.669+0000 7f2de73c9640 1 Processor -- start 2023-10-16T11:06:11.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.669+0000 7f2de73c9640 1 -- start start 2023-10-16T11:06:11.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.670+0000 7f2de73c9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2de00b0360 0x7f2de00b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:11.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.670+0000 7f2de73c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de00ca510 0x7f2de00ca920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:11.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.670+0000 7f2de73c9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de015b310 con 0x7f2de015ac10 2023-10-16T11:06:11.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.670+0000 7f2de73c9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de015f800 con 0x7f2de00b0360 2023-10-16T11:06:11.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.670+0000 7f2de73c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de00b0d30 con 0x7f2de00ca510 2023-10-16T11:06:11.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.671+0000 7f2de5165640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2de015ac10 0x7f2de006c730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55350/0 (socket says 172.21.15.84:55350) 2023-10-16T11:06:11.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.671+0000 7f2de5165640 1 -- 172.21.15.84:0/3492368372 learned_addr learned my addr 172.21.15.84:0/3492368372 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:11.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.671+0000 7f2de4964640 1 --2- 172.21.15.84:0/3492368372 >> v2:172.21.15.84:3300/0 conn(0x7f2de00b0360 0x7f2de00b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:11.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.671+0000 7f2de5966640 1 --2- 172.21.15.84:0/3492368372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de00ca510 0x7f2de00ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:11.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.671+0000 7f2dd3fff640 1 -- 172.21.15.84:0/3492368372 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951638192 0 0) 0x7f2de015b310 con 0x7f2de015ac10 2023-10-16T11:06:11.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.672+0000 7f2dd3fff640 1 -- 172.21.15.84:0/3492368372 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dcc003520 con 0x7f2de015ac10 2023-10-16T11:06:11.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.672+0000 7f2de4964640 1 -- 172.21.15.84:0/3492368372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de00ca510 msgr2=0x7f2de00ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:11.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.672+0000 7f2de4964640 1 --2- 172.21.15.84:0/3492368372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de00ca510 0x7f2de00ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:11.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.672+0000 7f2de4964640 1 -- 172.21.15.84:0/3492368372 >> v1:172.21.15.84:6789/0 conn(0x7f2de015ac10 legacy=0x7f2de006c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:11.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.672+0000 7f2de4964640 1 -- 172.21.15.84:0/3492368372 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de006f640 con 0x7f2de00b0360 2023-10-16T11:06:11.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.673+0000 7f2de4964640 1 --2- 172.21.15.84:0/3492368372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de00b0360 0x7f2de00b07f0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f2dd40097d0 tx=0x7f2dd402b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0d6d3986c9c156 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:11.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.673+0000 7f2dd3fff640 1 -- 172.21.15.84:0/3492368372 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4031770 con 0x7f2de00b0360 2023-10-16T11:06:11.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.673+0000 7f2dd3fff640 1 -- 172.21.15.84:0/3492368372 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd40318d0 con 0x7f2de00b0360 2023-10-16T11:06:11.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.673+0000 7f2dd3fff640 1 -- 172.21.15.84:0/3492368372 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4031c60 con 0x7f2de00b0360 2023-10-16T11:06:11.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.674+0000 7f2de73c9640 1 -- 172.21.15.84:0/3492368372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de00b0360 msgr2=0x7f2de00b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:11.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.674+0000 7f2de73c9640 1 --2- 172.21.15.84:0/3492368372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de00b0360 0x7f2de00b07f0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f2dd40097d0 tx=0x7f2dd402b810 comp rx=0 tx=0).stop 2023-10-16T11:06:11.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.674+0000 7f2de73c9640 1 -- 172.21.15.84:0/3492368372 shutdown_connections 2023-10-16T11:06:11.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.674+0000 7f2de73c9640 1 --2- 172.21.15.84:0/3492368372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de00ca510 0x7f2de00ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:11.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.674+0000 7f2de73c9640 1 --2- 172.21.15.84:0/3492368372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de00b0360 0x7f2de00b07f0 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:11.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.674+0000 7f2de73c9640 1 -- 172.21.15.84:0/3492368372 >> 172.21.15.84:0/3492368372 conn(0x7f2de00cb360 msgr2=0x7f2de00cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:11.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.674+0000 7f2de73c9640 1 -- 172.21.15.84:0/3492368372 shutdown_connections 2023-10-16T11:06:11.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.674+0000 7f2de73c9640 1 -- 172.21.15.84:0/3492368372 wait complete. 2023-10-16T11:06:11.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.676+0000 7f2de73c9640 1 Processor -- start 2023-10-16T11:06:11.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.676+0000 7f2de73c9640 1 -- start start 2023-10-16T11:06:11.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.677+0000 7f2de73c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de00b0360 0x7f2de00b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:11.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.677+0000 7f2de73c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de00ca510 0x7f2de0155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:11.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.677+0000 7f2de73c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de015ac10 0x7f2de00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:11.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.677+0000 7f2de73c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de01f6d90 con 0x7f2de00ca510 2023-10-16T11:06:11.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.677+0000 7f2de73c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de00c6b00 con 0x7f2de015ac10 2023-10-16T11:06:11.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.677+0000 7f2de73c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de00c5be0 con 0x7f2de00b0360 2023-10-16T11:06:11.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.677+0000 7f2de4964640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de00ca510 0x7f2de0155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:11.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.677+0000 7f2de4964640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de00ca510 0x7f2de0155200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56396/0 (socket says 172.21.15.84:56396) 2023-10-16T11:06:11.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.677+0000 7f2de4964640 1 -- 172.21.15.84:0/3087207499 learned_addr learned my addr 172.21.15.84:0/3087207499 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:11.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.678+0000 7f2de5165640 1 --2- 172.21.15.84:0/3087207499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de00b0360 0x7f2de00b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:11.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.678+0000 7f2de5966640 1 --2- 172.21.15.84:0/3087207499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de015ac10 0x7f2de00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:11.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.678+0000 7f2de4964640 1 -- 172.21.15.84:0/3087207499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de00b0360 msgr2=0x7f2de00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:11.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.678+0000 7f2de4964640 1 --2- 172.21.15.84:0/3087207499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de00b0360 0x7f2de00b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:11.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.678+0000 7f2de4964640 1 -- 172.21.15.84:0/3087207499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de015ac10 msgr2=0x7f2de00b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:11.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.678+0000 7f2de4964640 1 --2- 172.21.15.84:0/3087207499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de015ac10 0x7f2de00b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:11.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.678+0000 7f2de4964640 1 -- 172.21.15.84:0/3087207499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd4009400 con 0x7f2de00ca510 2023-10-16T11:06:11.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.678+0000 7f2de4964640 1 --2- 172.21.15.84:0/3087207499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de00ca510 0x7f2de0155200 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f2dd40097d0 tx=0x7f2dd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:11.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.678+0000 7f2dd1ffb640 1 -- 172.21.15.84:0/3087207499 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4032610 con 0x7f2de00ca510 2023-10-16T11:06:11.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.679+0000 7f2dd1ffb640 1 -- 172.21.15.84:0/3087207499 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd4032790 con 0x7f2de00ca510 2023-10-16T11:06:11.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.679+0000 7f2dd1ffb640 1 -- 172.21.15.84:0/3087207499 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4032b40 con 0x7f2de00ca510 2023-10-16T11:06:11.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.679+0000 7f2de73c9640 1 -- 172.21.15.84:0/3087207499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de00c6070 con 0x7f2de00ca510 2023-10-16T11:06:11.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.679+0000 7f2de73c9640 1 -- 172.21.15.84:0/3087207499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de0156240 con 0x7f2de00ca510 2023-10-16T11:06:11.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.680+0000 7f2de73c9640 1 -- 172.21.15.84:0/3087207499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de00cb160 con 0x7f2de00ca510 2023-10-16T11:06:11.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.681+0000 7f2dd1ffb640 1 -- 172.21.15.84:0/3087207499 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2dd4032ca0 con 0x7f2de00ca510 2023-10-16T11:06:11.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.682+0000 7f2dd1ffb640 1 --2- 172.21.15.84:0/3087207499 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f2dc406af90 0x7f2dc406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:11.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.682+0000 7f2dd1ffb640 1 -- 172.21.15.84:0/3087207499 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2dd40aba10 con 0x7f2de00ca510 2023-10-16T11:06:11.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.682+0000 7f2de5165640 1 --2- 172.21.15.84:0/3087207499 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f2dc406af90 0x7f2dc406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:11.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.683+0000 7f2de5165640 1 --2- 172.21.15.84:0/3087207499 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f2dc406af90 0x7f2dc406d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc0011e0 tx=0x7f2ddc009040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:11.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.686+0000 7f2dd1ffb640 1 -- 172.21.15.84:0/3087207499 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2dd4076740 con 0x7f2de00ca510 2023-10-16T11:06:11.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:11.872+0000 7f2de73c9640 1 -- 172.21.15.84:0/3087207499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2de0066de0 con 0x7f2de00ca510 2023-10-16T11:06:12.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.465+0000 7f2dd1ffb640 1 -- 172.21.15.84:0/3087207499 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v176) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2dd4079d60 con 0x7f2de00ca510 2023-10-16T11:06:12.467 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-16T11:06:12.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.470+0000 7f2de73c9640 1 -- 172.21.15.84:0/3087207499 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f2dc406af90 msgr2=0x7f2dc406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:12.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.470+0000 7f2de73c9640 1 --2- 172.21.15.84:0/3087207499 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f2dc406af90 0x7f2dc406d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc0011e0 tx=0x7f2ddc009040 comp rx=0 tx=0).stop 2023-10-16T11:06:12.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.470+0000 7f2de73c9640 1 -- 172.21.15.84:0/3087207499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de00ca510 msgr2=0x7f2de0155200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:12.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.470+0000 7f2de73c9640 1 --2- 172.21.15.84:0/3087207499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de00ca510 0x7f2de0155200 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f2dd40097d0 tx=0x7f2dd40046c0 comp rx=0 tx=0).stop 2023-10-16T11:06:12.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.473+0000 7f2de73c9640 1 -- 172.21.15.84:0/3087207499 shutdown_connections 2023-10-16T11:06:12.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.473+0000 7f2de73c9640 1 --2- 172.21.15.84:0/3087207499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de015ac10 0x7f2de00b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:12.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.473+0000 7f2de73c9640 1 --2- 172.21.15.84:0/3087207499 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f2dc406af90 0x7f2dc406d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:12.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.473+0000 7f2de73c9640 1 --2- 172.21.15.84:0/3087207499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de00ca510 0x7f2de0155200 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:12.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.473+0000 7f2de73c9640 1 --2- 172.21.15.84:0/3087207499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de00b0360 0x7f2de00b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:12.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.473+0000 7f2de73c9640 1 -- 172.21.15.84:0/3087207499 >> 172.21.15.84:0/3087207499 conn(0x7f2de00cb360 msgr2=0x7f2de00afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:12.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.473+0000 7f2de73c9640 1 -- 172.21.15.84:0/3087207499 shutdown_connections 2023-10-16T11:06:12.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.473+0000 7f2de73c9640 1 -- 172.21.15.84:0/3087207499 wait complete. 2023-10-16T11:06:12.506 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T11:06:12.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.687+0000 7fc0b9dce640 1 Processor -- start 2023-10-16T11:06:12.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.688+0000 7fc0b9dce640 1 -- start start 2023-10-16T11:06:12.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.689+0000 7fc0b9dce640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc0b415d150 0x7fc0b40ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:12.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.689+0000 7fc0b9dce640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc0b4160b90 0x7fc0b4160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:12.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.689+0000 7fc0b9dce640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b40ac890 con 0x7fc0b4069da0 2023-10-16T11:06:12.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.689+0000 7fc0b9dce640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b415d530 con 0x7fc0b4160b90 2023-10-16T11:06:12.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.689+0000 7fc0b9dce640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b4161650 con 0x7fc0b415d150 2023-10-16T11:06:12.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.689+0000 7fc0abfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc0b4160b90 0x7fc0b4160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:12.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.689+0000 7fc0abfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc0b4160b90 0x7fc0b4160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56400/0 (socket says 172.21.15.84:56400) 2023-10-16T11:06:12.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.689+0000 7fc0abfff640 1 -- 172.21.15.84:0/4190017753 learned_addr learned my addr 172.21.15.84:0/4190017753 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:12.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.689+0000 7fc0b3fff640 1 --1- 172.21.15.84:0/4190017753 >> v1:172.21.15.84:6789/0 conn(0x7fc0b4069da0 0x7fc0b40c9000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55354/0 (socket says 172.21.15.84:55354) 2023-10-16T11:06:12.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.690+0000 7fc0abfff640 1 -- 172.21.15.84:0/4190017753 >> v2:172.21.15.157:3300/0 conn(0x7fc0b415d150 msgr2=0x7fc0b40ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:12.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.690+0000 7fc0abfff640 1 --2- 172.21.15.84:0/4190017753 >> v2:172.21.15.157:3300/0 conn(0x7fc0b415d150 0x7fc0b40ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:12.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.690+0000 7fc0abfff640 1 -- 172.21.15.84:0/4190017753 >> v1:172.21.15.84:6789/0 conn(0x7fc0b4069da0 legacy=0x7fc0b40c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:12.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.690+0000 7fc0abfff640 1 -- 172.21.15.84:0/4190017753 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b4161930 con 0x7fc0b4160b90 2023-10-16T11:06:12.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.690+0000 7fc0b2ffd640 1 -- 172.21.15.84:0/4190017753 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732216552 0 0) 0x7fc0b40ac890 con 0x7fc0b4069da0 2023-10-16T11:06:12.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.691+0000 7fc0abfff640 1 --2- 172.21.15.84:0/4190017753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0b4160b90 0x7fc0b4160fe0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fc0b40b15a0 tx=0x7fc09c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0c17a8759d7306 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:12.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.691+0000 7fc0b2ffd640 1 -- 172.21.15.84:0/4190017753 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c031870 con 0x7fc0b4160b90 2023-10-16T11:06:12.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.691+0000 7fc0b2ffd640 1 -- 172.21.15.84:0/4190017753 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09c0319d0 con 0x7fc0b4160b90 2023-10-16T11:06:12.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.691+0000 7fc0b2ffd640 1 -- 172.21.15.84:0/4190017753 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c031d60 con 0x7fc0b4160b90 2023-10-16T11:06:12.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.692+0000 7fc0b9dce640 1 -- 172.21.15.84:0/4190017753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0b4160b90 msgr2=0x7fc0b4160fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:12.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.692+0000 7fc0b9dce640 1 --2- 172.21.15.84:0/4190017753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0b4160b90 0x7fc0b4160fe0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fc0b40b15a0 tx=0x7fc09c02b910 comp rx=0 tx=0).stop 2023-10-16T11:06:12.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.692+0000 7fc0b9dce640 1 -- 172.21.15.84:0/4190017753 shutdown_connections 2023-10-16T11:06:12.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.692+0000 7fc0b9dce640 1 --2- 172.21.15.84:0/4190017753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0b4160b90 0x7fc0b4160fe0 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:12.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.692+0000 7fc0b9dce640 1 --2- 172.21.15.84:0/4190017753 >> v2:172.21.15.157:3300/0 conn(0x7fc0b415d150 0x7fc0b40ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:12.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.692+0000 7fc0b9dce640 1 -- 172.21.15.84:0/4190017753 >> 172.21.15.84:0/4190017753 conn(0x7fc0b40ce9b0 msgr2=0x7fc0b40cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:12.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.692+0000 7fc0b9dce640 1 -- 172.21.15.84:0/4190017753 shutdown_connections 2023-10-16T11:06:12.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.692+0000 7fc0b9dce640 1 -- 172.21.15.84:0/4190017753 wait complete. 2023-10-16T11:06:12.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.694+0000 7fc0b9dce640 1 Processor -- start 2023-10-16T11:06:12.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.694+0000 7fc0b9dce640 1 -- start start 2023-10-16T11:06:12.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.695+0000 7fc0b9dce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0b4069da0 0x7fc0b4148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:12.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.695+0000 7fc0b9dce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0b415d150 0x7fc0b40c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:12.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.695+0000 7fc0b9dce640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0b4160b90 0x7fc0b414b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:12.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.695+0000 7fc0b9dce640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b40c3530 con 0x7fc0b4069da0 2023-10-16T11:06:12.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.695+0000 7fc0b9dce640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b40c3ab0 con 0x7fc0b415d150 2023-10-16T11:06:12.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.695+0000 7fc0b9dce640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b414c390 con 0x7fc0b4160b90 2023-10-16T11:06:12.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.695+0000 7fc0b37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0b4069da0 0x7fc0b4148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:12.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.695+0000 7fc0b37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0b4069da0 0x7fc0b4148ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56408/0 (socket says 172.21.15.84:56408) 2023-10-16T11:06:12.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.695+0000 7fc0b37fe640 1 -- 172.21.15.84:0/965248842 learned_addr learned my addr 172.21.15.84:0/965248842 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:12.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.696+0000 7fc0b37fe640 1 -- 172.21.15.84:0/965248842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0b4160b90 msgr2=0x7fc0b414b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:12.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.696+0000 7fc0b37fe640 1 --2- 172.21.15.84:0/965248842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0b4160b90 0x7fc0b414b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:12.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.696+0000 7fc0b37fe640 1 -- 172.21.15.84:0/965248842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0b415d150 msgr2=0x7fc0b40c3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:12.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.696+0000 7fc0b37fe640 1 --2- 172.21.15.84:0/965248842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0b415d150 0x7fc0b40c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:12.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.696+0000 7fc0b37fe640 1 -- 172.21.15.84:0/965248842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09c0094d0 con 0x7fc0b4069da0 2023-10-16T11:06:12.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.696+0000 7fc0b37fe640 1 --2- 172.21.15.84:0/965248842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0b4069da0 0x7fc0b4148ef0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400b410 tx=0x7fc0a400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:12.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.696+0000 7fc0b0ff9640 1 -- 172.21.15.84:0/965248842 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4011830 con 0x7fc0b4069da0 2023-10-16T11:06:12.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.696+0000 7fc0b0ff9640 1 -- 172.21.15.84:0/965248842 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a40119b0 con 0x7fc0b4069da0 2023-10-16T11:06:12.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.696+0000 7fc0b0ff9640 1 -- 172.21.15.84:0/965248842 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4011d60 con 0x7fc0b4069da0 2023-10-16T11:06:12.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.696+0000 7fc0b9dce640 1 -- 172.21.15.84:0/965248842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b414c670 con 0x7fc0b4069da0 2023-10-16T11:06:12.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.696+0000 7fc0b9dce640 1 -- 172.21.15.84:0/965248842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b414fab0 con 0x7fc0b4069da0 2023-10-16T11:06:12.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.698+0000 7fc0b9dce640 1 -- 172.21.15.84:0/965248842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b414c860 con 0x7fc0b4069da0 2023-10-16T11:06:12.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.699+0000 7fc0b0ff9640 1 -- 172.21.15.84:0/965248842 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc0a401a400 con 0x7fc0b4069da0 2023-10-16T11:06:12.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.699+0000 7fc0b0ff9640 1 --2- 172.21.15.84:0/965248842 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc09406af90 0x7fc09406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:12.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.700+0000 7fc0b0ff9640 1 -- 172.21.15.84:0/965248842 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fc0a4089980 con 0x7fc0b4069da0 2023-10-16T11:06:12.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.700+0000 7fc0abfff640 1 --2- 172.21.15.84:0/965248842 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc09406af90 0x7fc09406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:12.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.700+0000 7fc0abfff640 1 --2- 172.21.15.84:0/965248842 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc09406af90 0x7fc09406d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc09c036000 tx=0x7fc09c030000 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:12.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.704+0000 7fc0b0ff9640 1 -- 172.21.15.84:0/965248842 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc0a40548b0 con 0x7fc0b4069da0 2023-10-16T11:06:12.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.893+0000 7fc0b9dce640 1 -- 172.21.15.84:0/965248842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc0b404e160 con 0x7fc0b4069da0 2023-10-16T11:06:12.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.898+0000 7fc0b0ff9640 1 -- 172.21.15.84:0/965248842 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc0a40150a0 con 0x7fc0b4069da0 2023-10-16T11:06:12.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.901+0000 7fc0b9dce640 1 -- 172.21.15.84:0/965248842 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc09406af90 msgr2=0x7fc09406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:12.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.901+0000 7fc0b9dce640 1 --2- 172.21.15.84:0/965248842 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc09406af90 0x7fc09406d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc09c036000 tx=0x7fc09c030000 comp rx=0 tx=0).stop 2023-10-16T11:06:12.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.901+0000 7fc0b9dce640 1 -- 172.21.15.84:0/965248842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0b4069da0 msgr2=0x7fc0b4148ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:12.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.901+0000 7fc0b9dce640 1 --2- 172.21.15.84:0/965248842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0b4069da0 0x7fc0b4148ef0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400b410 tx=0x7fc0a400b8d0 comp rx=0 tx=0).stop 2023-10-16T11:06:12.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.901+0000 7fc0b9dce640 1 -- 172.21.15.84:0/965248842 shutdown_connections 2023-10-16T11:06:12.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.901+0000 7fc0b9dce640 1 --2- 172.21.15.84:0/965248842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0b4160b90 0x7fc0b414b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:12.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.901+0000 7fc0b9dce640 1 --2- 172.21.15.84:0/965248842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0b415d150 0x7fc0b40c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:12.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.901+0000 7fc0b9dce640 1 --2- 172.21.15.84:0/965248842 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc09406af90 0x7fc09406d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:12.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.901+0000 7fc0b9dce640 1 --2- 172.21.15.84:0/965248842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0b4069da0 0x7fc0b4148ef0 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:12.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.901+0000 7fc0b9dce640 1 -- 172.21.15.84:0/965248842 >> 172.21.15.84:0/965248842 conn(0x7fc0b40ce9b0 msgr2=0x7fc0b40c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:12.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.901+0000 7fc0b9dce640 1 -- 172.21.15.84:0/965248842 shutdown_connections 2023-10-16T11:06:12.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:12.901+0000 7fc0b9dce640 1 -- 172.21.15.84:0/965248842 wait complete. 2023-10-16T11:06:12.919 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T11:06:13.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.119+0000 7fc79510a640 1 Processor -- start 2023-10-16T11:06:13.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.119+0000 7fc79510a640 1 -- start start 2023-10-16T11:06:13.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.120+0000 7fc79510a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc79006c040 0x7fc79006c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:13.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.120+0000 7fc79510a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7900ad970 0x7fc79015e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:13.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.121+0000 7fc79510a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79006c960 con 0x7fc79015ebc0 2023-10-16T11:06:13.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.121+0000 7fc79510a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79015b9d0 con 0x7fc79006c040 2023-10-16T11:06:13.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.121+0000 7fc79510a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79015bcb0 con 0x7fc7900ad970 2023-10-16T11:06:13.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.121+0000 7fc78ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc79006c040 0x7fc79006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:13.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.121+0000 7fc78ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc79006c040 0x7fc79006c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56416/0 (socket says 172.21.15.84:56416) 2023-10-16T11:06:13.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.121+0000 7fc78ffff640 1 -- 172.21.15.84:0/1890246407 learned_addr learned my addr 172.21.15.84:0/1890246407 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:13.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.121+0000 7fc78ffff640 1 -- 172.21.15.84:0/1890246407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7900ad970 msgr2=0x7fc79015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:13.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.121+0000 7fc78ffff640 1 --2- 172.21.15.84:0/1890246407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7900ad970 0x7fc79015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.121+0000 7fc78ffff640 1 -- 172.21.15.84:0/1890246407 >> v1:172.21.15.84:6789/0 conn(0x7fc79015ebc0 legacy=0x7fc7900cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:13.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.121+0000 7fc78ffff640 1 -- 172.21.15.84:0/1890246407 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79015bf90 con 0x7fc79006c040 2023-10-16T11:06:13.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.122+0000 7fc78ffff640 1 --2- 172.21.15.84:0/1890246407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc79006c040 0x7fc79006c420 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fc7800098a0 tx=0x7fc78002b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9592753b2e55baab server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:13.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.123+0000 7fc78f7fe640 1 -- 172.21.15.84:0/1890246407 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780030830 con 0x7fc79006c040 2023-10-16T11:06:13.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.123+0000 7fc78f7fe640 1 -- 172.21.15.84:0/1890246407 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc780030990 con 0x7fc79006c040 2023-10-16T11:06:13.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.123+0000 7fc78f7fe640 1 -- 172.21.15.84:0/1890246407 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780030d20 con 0x7fc79006c040 2023-10-16T11:06:13.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.123+0000 7fc79510a640 1 -- 172.21.15.84:0/1890246407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc79006c040 msgr2=0x7fc79006c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:13.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.123+0000 7fc79510a640 1 --2- 172.21.15.84:0/1890246407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc79006c040 0x7fc79006c420 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fc7800098a0 tx=0x7fc78002b970 comp rx=0 tx=0).stop 2023-10-16T11:06:13.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.123+0000 7fc79510a640 1 -- 172.21.15.84:0/1890246407 shutdown_connections 2023-10-16T11:06:13.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.123+0000 7fc79510a640 1 --2- 172.21.15.84:0/1890246407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7900ad970 0x7fc79015e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.123+0000 7fc79510a640 1 --2- 172.21.15.84:0/1890246407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc79006c040 0x7fc79006c420 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.123+0000 7fc79510a640 1 -- 172.21.15.84:0/1890246407 >> 172.21.15.84:0/1890246407 conn(0x7fc7900ce6a0 msgr2=0x7fc7900ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:13.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.124+0000 7fc79510a640 1 -- 172.21.15.84:0/1890246407 shutdown_connections 2023-10-16T11:06:13.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.124+0000 7fc79510a640 1 -- 172.21.15.84:0/1890246407 wait complete. 2023-10-16T11:06:13.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.125+0000 7fc79510a640 1 Processor -- start 2023-10-16T11:06:13.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.125+0000 7fc79510a640 1 -- start start 2023-10-16T11:06:13.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.126+0000 7fc79510a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc79006c040 0x7fc7900c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:13.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.126+0000 7fc79510a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7900ad970 0x7fc7900c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:13.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.126+0000 7fc79510a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79015ebc0 0x7fc790153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:13.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.126+0000 7fc79510a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7900c5cb0 con 0x7fc7900ad970 2023-10-16T11:06:13.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.126+0000 7fc79510a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc790154560 con 0x7fc79006c040 2023-10-16T11:06:13.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.126+0000 7fc79510a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc790154160 con 0x7fc79015ebc0 2023-10-16T11:06:13.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.127+0000 7fc787fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7900ad970 0x7fc7900c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:13.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.127+0000 7fc787fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7900ad970 0x7fc7900c5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56424/0 (socket says 172.21.15.84:56424) 2023-10-16T11:06:13.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.127+0000 7fc787fff640 1 -- 172.21.15.84:0/2620218626 learned_addr learned my addr 172.21.15.84:0/2620218626 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:13.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.127+0000 7fc787fff640 1 -- 172.21.15.84:0/2620218626 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79015ebc0 msgr2=0x7fc790153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:13.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.127+0000 7fc787fff640 1 --2- 172.21.15.84:0/2620218626 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79015ebc0 0x7fc790153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.127+0000 7fc787fff640 1 -- 172.21.15.84:0/2620218626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc79006c040 msgr2=0x7fc7900c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:13.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.127+0000 7fc787fff640 1 --2- 172.21.15.84:0/2620218626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc79006c040 0x7fc7900c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.127+0000 7fc787fff640 1 -- 172.21.15.84:0/2620218626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7800094d0 con 0x7fc7900ad970 2023-10-16T11:06:13.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.127+0000 7fc787fff640 1 --2- 172.21.15.84:0/2620218626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7900ad970 0x7fc7900c5770 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fc77800b410 tx=0x7fc77800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:13.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.127+0000 7fc78d7fa640 1 -- 172.21.15.84:0/2620218626 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778011830 con 0x7fc7900ad970 2023-10-16T11:06:13.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.127+0000 7fc78d7fa640 1 -- 172.21.15.84:0/2620218626 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7780119b0 con 0x7fc7900ad970 2023-10-16T11:06:13.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.127+0000 7fc78d7fa640 1 -- 172.21.15.84:0/2620218626 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778011d60 con 0x7fc7900ad970 2023-10-16T11:06:13.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.127+0000 7fc79510a640 1 -- 172.21.15.84:0/2620218626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7900c5ec0 con 0x7fc7900ad970 2023-10-16T11:06:13.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.128+0000 7fc79510a640 1 -- 172.21.15.84:0/2620218626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7900c6480 con 0x7fc7900ad970 2023-10-16T11:06:13.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.129+0000 7fc79510a640 1 -- 172.21.15.84:0/2620218626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc79006b330 con 0x7fc7900ad970 2023-10-16T11:06:13.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.130+0000 7fc78d7fa640 1 -- 172.21.15.84:0/2620218626 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc77801a400 con 0x7fc7900ad970 2023-10-16T11:06:13.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.130+0000 7fc78d7fa640 1 --2- 172.21.15.84:0/2620218626 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc77006af90 0x7fc77006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:13.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.131+0000 7fc78d7fa640 1 -- 172.21.15.84:0/2620218626 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fc7780899b0 con 0x7fc7900ad970 2023-10-16T11:06:13.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.131+0000 7fc78ffff640 1 --2- 172.21.15.84:0/2620218626 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc77006af90 0x7fc77006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:13.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.131+0000 7fc78ffff640 1 --2- 172.21.15.84:0/2620218626 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc77006af90 0x7fc77006d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc780036000 tx=0x7fc78003b040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:13.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.136+0000 7fc78d7fa640 1 -- 172.21.15.84:0/2620218626 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc7780548e0 con 0x7fc7900ad970 2023-10-16T11:06:13.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.326+0000 7fc79510a640 1 -- 172.21.15.84:0/2620218626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fc79004e9b0 con 0x7fc7900ad970 2023-10-16T11:06:13.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.332+0000 7fc78d7fa640 1 -- 172.21.15.84:0/2620218626 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v62) v1 ==== 120+0+0 (secure 0 0 0) 0x7fc7780150b0 con 0x7fc7900ad970 2023-10-16T11:06:13.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.335+0000 7fc79510a640 1 -- 172.21.15.84:0/2620218626 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc77006af90 msgr2=0x7fc77006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:13.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.335+0000 7fc79510a640 1 --2- 172.21.15.84:0/2620218626 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc77006af90 0x7fc77006d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc780036000 tx=0x7fc78003b040 comp rx=0 tx=0).stop 2023-10-16T11:06:13.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.335+0000 7fc79510a640 1 -- 172.21.15.84:0/2620218626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7900ad970 msgr2=0x7fc7900c5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:13.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.335+0000 7fc79510a640 1 --2- 172.21.15.84:0/2620218626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7900ad970 0x7fc7900c5770 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fc77800b410 tx=0x7fc77800b8d0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.335+0000 7fc79510a640 1 -- 172.21.15.84:0/2620218626 shutdown_connections 2023-10-16T11:06:13.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.335+0000 7fc79510a640 1 --2- 172.21.15.84:0/2620218626 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc79015ebc0 0x7fc790153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.335+0000 7fc79510a640 1 --2- 172.21.15.84:0/2620218626 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc77006af90 0x7fc77006d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.335+0000 7fc79510a640 1 --2- 172.21.15.84:0/2620218626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7900ad970 0x7fc7900c5770 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.335+0000 7fc79510a640 1 --2- 172.21.15.84:0/2620218626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc79006c040 0x7fc7900c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.335+0000 7fc79510a640 1 -- 172.21.15.84:0/2620218626 >> 172.21.15.84:0/2620218626 conn(0x7fc7900ce6a0 msgr2=0x7fc79015dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:13.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.335+0000 7fc79510a640 1 -- 172.21.15.84:0/2620218626 shutdown_connections 2023-10-16T11:06:13.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.335+0000 7fc79510a640 1 -- 172.21.15.84:0/2620218626 wait complete. 2023-10-16T11:06:13.355 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T11:06:13.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.568+0000 7fc37bfd5640 1 Processor -- start 2023-10-16T11:06:13.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.569+0000 7fc37bfd5640 1 -- start start 2023-10-16T11:06:13.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.570+0000 7fc37bfd5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc374159f00 0x7fc3740b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:13.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.570+0000 7fc37bfd5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37406f890 con 0x7fc37415d8c0 2023-10-16T11:06:13.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.570+0000 7fc37bfd5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37406c650 con 0x7fc374159b20 2023-10-16T11:06:13.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.570+0000 7fc37bfd5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37406c930 con 0x7fc374159f00 2023-10-16T11:06:13.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.570+0000 7fc37a572640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc37415d8c0 0x7fc37406fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55362/0 (socket says 172.21.15.84:55362) 2023-10-16T11:06:13.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.570+0000 7fc37a572640 1 -- 172.21.15.84:0/1662038640 learned_addr learned my addr 172.21.15.84:0/1662038640 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:13.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.571+0000 7fc378d6f640 1 -- 172.21.15.84:0/1662038640 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 733369620 0 0) 0x7fc37406f890 con 0x7fc37415d8c0 2023-10-16T11:06:13.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.571+0000 7fc378d6f640 1 -- 172.21.15.84:0/1662038640 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc360003520 con 0x7fc37415d8c0 2023-10-16T11:06:13.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.571+0000 7fc379570640 1 --2- 172.21.15.84:0/1662038640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc374159f00 0x7fc3740b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:13.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.571+0000 7fc378d6f640 1 -- 172.21.15.84:0/1662038640 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 49281902 0 0) 0x7fc360003520 con 0x7fc37415d8c0 2023-10-16T11:06:13.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.571+0000 7fc378d6f640 1 -- 172.21.15.84:0/1662038640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc374159f00 msgr2=0x7fc3740b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:13.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.571+0000 7fc378d6f640 1 --2- 172.21.15.84:0/1662038640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc374159f00 0x7fc3740b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.571+0000 7fc378d6f640 1 -- 172.21.15.84:0/1662038640 >> v1:172.21.15.157:6789/0 conn(0x7fc374159b20 legacy=0x7fc37406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:13.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.571+0000 7fc378d6f640 1 -- 172.21.15.84:0/1662038640 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37406cc10 con 0x7fc37415d8c0 2023-10-16T11:06:13.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.571+0000 7fc378d6f640 1 -- 172.21.15.84:0/1662038640 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc36c0019f0 con 0x7fc37415d8c0 2023-10-16T11:06:13.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.571+0000 7fc378d6f640 1 -- 172.21.15.84:0/1662038640 >> v1:172.21.15.84:6789/0 conn(0x7fc37415d8c0 legacy=0x7fc37406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:13.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.571+0000 7fc378d6f640 1 --2- 172.21.15.84:0/1662038640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc374159f00 0x7fc3600039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:13.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.571+0000 7fc378d6f640 1 --2- 172.21.15.84:0/1662038640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc360004020 0x7fc3600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:13.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.571+0000 7fc378d6f640 1 --2- 172.21.15.84:0/1662038640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc360004d00 0x7fc360005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:13.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.571+0000 7fc378d6f640 1 -- 172.21.15.84:0/1662038640 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37406c930 con 0x7fc360004d00 2023-10-16T11:06:13.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.571+0000 7fc378d6f640 1 -- 172.21.15.84:0/1662038640 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36000ddd0 con 0x7fc360004020 2023-10-16T11:06:13.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.572+0000 7fc378d6f640 1 -- 172.21.15.84:0/1662038640 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36000df90 con 0x7fc374159f00 2023-10-16T11:06:13.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.572+0000 7fc37a572640 1 --2- 172.21.15.84:0/1662038640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc360004d00 0x7fc360005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:13.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.572+0000 7fc379570640 1 --2- 172.21.15.84:0/1662038640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc360004020 0x7fc3600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:13.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.572+0000 7fc37a572640 1 -- 172.21.15.84:0/1662038640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc374159f00 msgr2=0x7fc3600039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:13.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.572+0000 7fc37a572640 1 --2- 172.21.15.84:0/1662038640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc374159f00 0x7fc3600039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.572+0000 7fc37a572640 1 -- 172.21.15.84:0/1662038640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc360004020 msgr2=0x7fc3600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:13.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.572+0000 7fc37a572640 1 --2- 172.21.15.84:0/1662038640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc360004020 0x7fc3600047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.572+0000 7fc37a572640 1 -- 172.21.15.84:0/1662038640 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc360003520 con 0x7fc360004d00 2023-10-16T11:06:13.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.572+0000 7fc379570640 1 --2- 172.21.15.84:0/1662038640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc360004020 0x7fc3600047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:13.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.573+0000 7fc37a572640 1 --2- 172.21.15.84:0/1662038640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc360004d00 0x7fc360005180 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fc37415d770 tx=0x7fc3600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8363b1497e8c2e10 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:13.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.573+0000 7fc378d6f640 1 -- 172.21.15.84:0/1662038640 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c034630 con 0x7fc360004d00 2023-10-16T11:06:13.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.574+0000 7fc378d6f640 1 -- 172.21.15.84:0/1662038640 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36c02f070 con 0x7fc360004d00 2023-10-16T11:06:13.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.574+0000 7fc378d6f640 1 -- 172.21.15.84:0/1662038640 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c034a50 con 0x7fc360004d00 2023-10-16T11:06:13.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.574+0000 7fc37bfd5640 1 -- 172.21.15.84:0/1662038640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc360004d00 msgr2=0x7fc360005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:13.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.574+0000 7fc37bfd5640 1 --2- 172.21.15.84:0/1662038640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc360004d00 0x7fc360005180 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fc37415d770 tx=0x7fc3600057f0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.574+0000 7fc37a572640 1 -- 172.21.15.84:0/1662038640 reap_dead start 2023-10-16T11:06:13.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.574+0000 7fc37bfd5640 1 -- 172.21.15.84:0/1662038640 shutdown_connections 2023-10-16T11:06:13.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.574+0000 7fc37bfd5640 1 -- 172.21.15.84:0/1662038640 >> 172.21.15.84:0/1662038640 conn(0x7fc3740cb410 msgr2=0x7fc3740cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:13.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.574+0000 7fc37bfd5640 1 -- 172.21.15.84:0/1662038640 shutdown_connections 2023-10-16T11:06:13.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.574+0000 7fc37bfd5640 1 -- 172.21.15.84:0/1662038640 wait complete. 2023-10-16T11:06:13.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.576+0000 7fc37bfd5640 1 Processor -- start 2023-10-16T11:06:13.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.576+0000 7fc37bfd5640 1 -- start start 2023-10-16T11:06:13.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.577+0000 7fc37bfd5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc360004d00 0x7fc37415b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:13.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.577+0000 7fc37bfd5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc37415b8a0 0x7fc3740c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:13.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.577+0000 7fc37bfd5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3740c8570 0x7fc3741584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:13.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.577+0000 7fc37bfd5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37415ca30 con 0x7fc37415b8a0 2023-10-16T11:06:13.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.577+0000 7fc37bfd5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740c8a20 con 0x7fc3740c8570 2023-10-16T11:06:13.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.577+0000 7fc37bfd5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37415be20 con 0x7fc360004d00 2023-10-16T11:06:13.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.577+0000 7fc379570640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc37415b8a0 0x7fc3740c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:13.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.577+0000 7fc379570640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc37415b8a0 0x7fc3740c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56432/0 (socket says 172.21.15.84:56432) 2023-10-16T11:06:13.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.577+0000 7fc379570640 1 -- 172.21.15.84:0/1168580953 learned_addr learned my addr 172.21.15.84:0/1168580953 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:13.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.578+0000 7fc379570640 1 -- 172.21.15.84:0/1168580953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc360004d00 msgr2=0x7fc37415b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:13.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.578+0000 7fc379570640 1 --2- 172.21.15.84:0/1168580953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc360004d00 0x7fc37415b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.578+0000 7fc379570640 1 -- 172.21.15.84:0/1168580953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3740c8570 msgr2=0x7fc3741584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:13.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.578+0000 7fc379570640 1 --2- 172.21.15.84:0/1168580953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3740c8570 0x7fc3741584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.578+0000 7fc379570640 1 -- 172.21.15.84:0/1168580953 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37415c100 con 0x7fc37415b8a0 2023-10-16T11:06:13.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.578+0000 7fc379570640 1 --2- 172.21.15.84:0/1168580953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc37415b8a0 0x7fc3740c8030 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7fc36400e810 tx=0x7fc36400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:13.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.578+0000 7fc36a7fc640 1 -- 172.21.15.84:0/1168580953 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364004020 con 0x7fc37415b8a0 2023-10-16T11:06:13.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.578+0000 7fc36a7fc640 1 -- 172.21.15.84:0/1168580953 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc364004180 con 0x7fc37415b8a0 2023-10-16T11:06:13.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.578+0000 7fc37bfd5640 1 -- 172.21.15.84:0/1168580953 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37415c3e0 con 0x7fc37415b8a0 2023-10-16T11:06:13.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.578+0000 7fc36a7fc640 1 -- 172.21.15.84:0/1168580953 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3640044f0 con 0x7fc37415b8a0 2023-10-16T11:06:13.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.578+0000 7fc37bfd5640 1 -- 172.21.15.84:0/1168580953 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc374159350 con 0x7fc37415b8a0 2023-10-16T11:06:13.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.580+0000 7fc37bfd5640 1 -- 172.21.15.84:0/1168580953 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37406c270 con 0x7fc37415b8a0 2023-10-16T11:06:13.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.581+0000 7fc36a7fc640 1 -- 172.21.15.84:0/1168580953 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc36400c650 con 0x7fc37415b8a0 2023-10-16T11:06:13.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.582+0000 7fc36a7fc640 1 --2- 172.21.15.84:0/1168580953 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc35806af90 0x7fc35806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:13.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.582+0000 7fc36a7fc640 1 -- 172.21.15.84:0/1168580953 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fc3640892e0 con 0x7fc37415b8a0 2023-10-16T11:06:13.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.582+0000 7fc379d71640 1 --2- 172.21.15.84:0/1168580953 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc35806af90 0x7fc35806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:13.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.583+0000 7fc379d71640 1 --2- 172.21.15.84:0/1168580953 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc35806af90 0x7fc35806d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc3740b4760 tx=0x7fc3700079b0 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:13.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.586+0000 7fc36a7fc640 1 -- 172.21.15.84:0/1168580953 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc364054100 con 0x7fc37415b8a0 2023-10-16T11:06:13.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.779+0000 7fc37bfd5640 1 -- 172.21.15.84:0/1168580953 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fc37404e160 con 0x7fc37415b8a0 2023-10-16T11:06:13.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.786+0000 7fc36a7fc640 1 -- 172.21.15.84:0/1168580953 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v63) v1 ==== 115+0+0 (secure 0 0 0) 0x7fc364057720 con 0x7fc37415b8a0 2023-10-16T11:06:13.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.789+0000 7fc37bfd5640 1 -- 172.21.15.84:0/1168580953 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc35806af90 msgr2=0x7fc35806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:13.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.789+0000 7fc37bfd5640 1 --2- 172.21.15.84:0/1168580953 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc35806af90 0x7fc35806d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc3740b4760 tx=0x7fc3700079b0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.789+0000 7fc37bfd5640 1 -- 172.21.15.84:0/1168580953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc37415b8a0 msgr2=0x7fc3740c8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:13.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.789+0000 7fc37bfd5640 1 --2- 172.21.15.84:0/1168580953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc37415b8a0 0x7fc3740c8030 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7fc36400e810 tx=0x7fc36400ecd0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.789+0000 7fc37bfd5640 1 -- 172.21.15.84:0/1168580953 shutdown_connections 2023-10-16T11:06:13.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.789+0000 7fc37bfd5640 1 --2- 172.21.15.84:0/1168580953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3740c8570 0x7fc3741584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.789+0000 7fc37bfd5640 1 --2- 172.21.15.84:0/1168580953 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc35806af90 0x7fc35806d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.789+0000 7fc37bfd5640 1 --2- 172.21.15.84:0/1168580953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc37415b8a0 0x7fc3740c8030 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.789+0000 7fc37bfd5640 1 --2- 172.21.15.84:0/1168580953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc360004d00 0x7fc37415b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:13.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.789+0000 7fc37bfd5640 1 -- 172.21.15.84:0/1168580953 >> 172.21.15.84:0/1168580953 conn(0x7fc3740cb410 msgr2=0x7fc3740aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:13.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.789+0000 7fc37bfd5640 1 -- 172.21.15.84:0/1168580953 shutdown_connections 2023-10-16T11:06:13.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:13.789+0000 7fc37bfd5640 1 -- 172.21.15.84:0/1168580953 wait complete. 2023-10-16T11:06:13.811 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-10-16T11:06:14.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.013+0000 7fd145075640 1 Processor -- start 2023-10-16T11:06:14.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.013+0000 7fd145075640 1 -- start start 2023-10-16T11:06:14.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.014+0000 7fd145075640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd14015ae80 0x7fd14015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:14.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.014+0000 7fd145075640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1400ce390 0x7fd140070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:14.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.015+0000 7fd145075640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14015ed60 con 0x7fd14015aaa0 2023-10-16T11:06:14.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.015+0000 7fd145075640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14006b810 con 0x7fd14015ae80 2023-10-16T11:06:14.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.015+0000 7fd145075640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1400707e0 con 0x7fd1400ce390 2023-10-16T11:06:14.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.015+0000 7fd13e59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd14015ae80 0x7fd14015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:14.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.015+0000 7fd13e59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd14015ae80 0x7fd14015e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:42914/0 (socket says 172.21.15.84:42914) 2023-10-16T11:06:14.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.015+0000 7fd13e59c640 1 -- 172.21.15.84:0/3941259314 learned_addr learned my addr 172.21.15.84:0/3941259314 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:14.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.015+0000 7fd13f59e640 1 --2- 172.21.15.84:0/3941259314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1400ce390 0x7fd140070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:14.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.016+0000 7fd13f59e640 1 -- 172.21.15.84:0/3941259314 >> v2:172.21.15.157:3300/0 conn(0x7fd14015ae80 msgr2=0x7fd14015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:14.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.016+0000 7fd13f59e640 1 --2- 172.21.15.84:0/3941259314 >> v2:172.21.15.157:3300/0 conn(0x7fd14015ae80 0x7fd14015e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.016+0000 7fd13f59e640 1 -- 172.21.15.84:0/3941259314 >> v1:172.21.15.157:6789/0 conn(0x7fd14015aaa0 legacy=0x7fd1400cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:14.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.016+0000 7fd13f59e640 1 -- 172.21.15.84:0/3941259314 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14015ef90 con 0x7fd1400ce390 2023-10-16T11:06:14.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.016+0000 7fd13e59c640 1 --2- 172.21.15.84:0/3941259314 >> v2:172.21.15.157:3300/0 conn(0x7fd14015ae80 0x7fd14015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:06:14.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.017+0000 7fd13f59e640 1 --2- 172.21.15.84:0/3941259314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1400ce390 0x7fd140070f50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd138009d90 tx=0x7fd13802beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cfe4e2f03860c92 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:14.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.018+0000 7fd13dd9b640 1 -- 172.21.15.84:0/3941259314 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13802cc60 con 0x7fd1400ce390 2023-10-16T11:06:14.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.018+0000 7fd13dd9b640 1 -- 172.21.15.84:0/3941259314 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13802cdc0 con 0x7fd1400ce390 2023-10-16T11:06:14.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.018+0000 7fd13dd9b640 1 -- 172.21.15.84:0/3941259314 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138034660 con 0x7fd1400ce390 2023-10-16T11:06:14.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.019+0000 7fd145075640 1 -- 172.21.15.84:0/3941259314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1400ce390 msgr2=0x7fd140070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:14.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.019+0000 7fd145075640 1 --2- 172.21.15.84:0/3941259314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1400ce390 0x7fd140070f50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd138009d90 tx=0x7fd13802beb0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.019+0000 7fd145075640 1 -- 172.21.15.84:0/3941259314 shutdown_connections 2023-10-16T11:06:14.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.019+0000 7fd145075640 1 --2- 172.21.15.84:0/3941259314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1400ce390 0x7fd140070f50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.019+0000 7fd145075640 1 --2- 172.21.15.84:0/3941259314 >> v2:172.21.15.157:3300/0 conn(0x7fd14015ae80 0x7fd14015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.019+0000 7fd145075640 1 -- 172.21.15.84:0/3941259314 >> 172.21.15.84:0/3941259314 conn(0x7fd1400cb410 msgr2=0x7fd1400cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:14.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.019+0000 7fd145075640 1 -- 172.21.15.84:0/3941259314 shutdown_connections 2023-10-16T11:06:14.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.019+0000 7fd145075640 1 -- 172.21.15.84:0/3941259314 wait complete. 2023-10-16T11:06:14.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.021+0000 7fd145075640 1 Processor -- start 2023-10-16T11:06:14.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.021+0000 7fd145075640 1 -- start start 2023-10-16T11:06:14.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.022+0000 7fd145075640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1400ce390 0x7fd1401510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:14.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.022+0000 7fd145075640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd14015aaa0 0x7fd140147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:14.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.022+0000 7fd145075640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd14015ae80 0x7fd1400c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:14.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.022+0000 7fd145075640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14014e080 con 0x7fd1400ce390 2023-10-16T11:06:14.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.022+0000 7fd145075640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1400c3290 con 0x7fd14015aaa0 2023-10-16T11:06:14.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.022+0000 7fd145075640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1400c34e0 con 0x7fd14015ae80 2023-10-16T11:06:14.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.023+0000 7fd13ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1400ce390 0x7fd1401510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:14.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.023+0000 7fd13ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1400ce390 0x7fd1401510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56434/0 (socket says 172.21.15.84:56434) 2023-10-16T11:06:14.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.023+0000 7fd13ed9d640 1 -- 172.21.15.84:0/1116494740 learned_addr learned my addr 172.21.15.84:0/1116494740 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:14.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.023+0000 7fd13e59c640 1 --2- 172.21.15.84:0/1116494740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd14015aaa0 0x7fd140147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:14.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.023+0000 7fd13f59e640 1 --2- 172.21.15.84:0/1116494740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd14015ae80 0x7fd1400c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:14.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.023+0000 7fd13ed9d640 1 -- 172.21.15.84:0/1116494740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd14015ae80 msgr2=0x7fd1400c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:14.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.023+0000 7fd13ed9d640 1 --2- 172.21.15.84:0/1116494740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd14015ae80 0x7fd1400c4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.023+0000 7fd13ed9d640 1 -- 172.21.15.84:0/1116494740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd14015aaa0 msgr2=0x7fd140147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:14.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.023+0000 7fd13ed9d640 1 --2- 172.21.15.84:0/1116494740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd14015aaa0 0x7fd140147e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.023+0000 7fd13ed9d640 1 -- 172.21.15.84:0/1116494740 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1340094d0 con 0x7fd1400ce390 2023-10-16T11:06:14.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.023+0000 7fd13ed9d640 1 --2- 172.21.15.84:0/1116494740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1400ce390 0x7fd1401510b0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fd13000c4d0 tx=0x7fd13000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:14.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.024+0000 7fd1277fe640 1 -- 172.21.15.84:0/1116494740 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1300052d0 con 0x7fd1400ce390 2023-10-16T11:06:14.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.024+0000 7fd1277fe640 1 -- 172.21.15.84:0/1116494740 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd130005430 con 0x7fd1400ce390 2023-10-16T11:06:14.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.024+0000 7fd1277fe640 1 -- 172.21.15.84:0/1116494740 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130012850 con 0x7fd1400ce390 2023-10-16T11:06:14.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.024+0000 7fd145075640 1 -- 172.21.15.84:0/1116494740 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd138009a70 con 0x7fd1400ce390 2023-10-16T11:06:14.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.024+0000 7fd145075640 1 -- 172.21.15.84:0/1116494740 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd140151900 con 0x7fd1400ce390 2023-10-16T11:06:14.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.025+0000 7fd145075640 1 -- 172.21.15.84:0/1116494740 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14004e160 con 0x7fd1400ce390 2023-10-16T11:06:14.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.026+0000 7fd1277fe640 1 -- 172.21.15.84:0/1116494740 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd1300129b0 con 0x7fd1400ce390 2023-10-16T11:06:14.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.027+0000 7fd1277fe640 1 --2- 172.21.15.84:0/1116494740 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fd12c06af90 0x7fd12c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:14.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.027+0000 7fd1277fe640 1 -- 172.21.15.84:0/1116494740 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fd1300892d0 con 0x7fd1400ce390 2023-10-16T11:06:14.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.027+0000 7fd13e59c640 1 --2- 172.21.15.84:0/1116494740 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fd12c06af90 0x7fd12c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:14.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.028+0000 7fd13e59c640 1 --2- 172.21.15.84:0/1116494740 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fd12c06af90 0x7fd12c06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd14015a640 tx=0x7fd134008040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:14.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.031+0000 7fd1277fe640 1 -- 172.21.15.84:0/1116494740 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd130054200 con 0x7fd1400ce390 2023-10-16T11:06:14.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.294+0000 7fd145075640 1 -- 172.21.15.84:0/1116494740 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7fd1400cdc60 con 0x7fd1400ce390 2023-10-16T11:06:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.478+0000 7fd1277fe640 1 -- 172.21.15.84:0/1116494740 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2530) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd130057820 con 0x7fd1400ce390 2023-10-16T11:06:14.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.482+0000 7fd145075640 1 -- 172.21.15.84:0/1116494740 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fd12c06af90 msgr2=0x7fd12c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:14.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.482+0000 7fd145075640 1 --2- 172.21.15.84:0/1116494740 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fd12c06af90 0x7fd12c06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd14015a640 tx=0x7fd134008040 comp rx=0 tx=0).stop 2023-10-16T11:06:14.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.482+0000 7fd145075640 1 -- 172.21.15.84:0/1116494740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1400ce390 msgr2=0x7fd1401510b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:14.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.482+0000 7fd145075640 1 --2- 172.21.15.84:0/1116494740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1400ce390 0x7fd1401510b0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fd13000c4d0 tx=0x7fd13000c990 comp rx=0 tx=0).stop 2023-10-16T11:06:14.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.482+0000 7fd145075640 1 -- 172.21.15.84:0/1116494740 shutdown_connections 2023-10-16T11:06:14.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.482+0000 7fd145075640 1 --2- 172.21.15.84:0/1116494740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd14015ae80 0x7fd1400c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.482+0000 7fd145075640 1 --2- 172.21.15.84:0/1116494740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd14015aaa0 0x7fd140147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.482+0000 7fd145075640 1 --2- 172.21.15.84:0/1116494740 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fd12c06af90 0x7fd12c06d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.482+0000 7fd145075640 1 --2- 172.21.15.84:0/1116494740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1400ce390 0x7fd1401510b0 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.482+0000 7fd145075640 1 -- 172.21.15.84:0/1116494740 >> 172.21.15.84:0/1116494740 conn(0x7fd1400cb410 msgr2=0x7fd140152180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:14.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.482+0000 7fd145075640 1 -- 172.21.15.84:0/1116494740 shutdown_connections 2023-10-16T11:06:14.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.482+0000 7fd145075640 1 -- 172.21.15.84:0/1116494740 wait complete. 2023-10-16T11:06:14.505 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2023-10-16T11:06:14.506 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2023-10-16T11:06:14.506 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T11:06:14.507 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:06:14.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.709+0000 7f2379e2b640 1 Processor -- start 2023-10-16T11:06:14.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.709+0000 7f2379e2b640 1 -- start start 2023-10-16T11:06:14.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.710+0000 7f2379e2b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f23740b3e70 0x7f23740d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:14.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.710+0000 7f2379e2b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23740b3940 con 0x7f23740b3070 2023-10-16T11:06:14.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.710+0000 7f2379e2b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23740b45e0 con 0x7f23740cfc10 2023-10-16T11:06:14.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.710+0000 7f2379e2b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23740c7880 con 0x7f23740b3e70 2023-10-16T11:06:14.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.710+0000 7f23737fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f23740b3070 0x7f23740cf500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55372/0 (socket says 172.21.15.84:55372) 2023-10-16T11:06:14.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.711+0000 7f23737fe640 1 -- 172.21.15.84:0/1551590276 learned_addr learned my addr 172.21.15.84:0/1551590276 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:14.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.711+0000 7f2373fff640 1 --2- 172.21.15.84:0/1551590276 >> v2:172.21.15.157:3300/0 conn(0x7f23740b3e70 0x7f23740d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:14.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.711+0000 7f23727fc640 1 -- 172.21.15.84:0/1551590276 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573310475 0 0) 0x7f23740b3940 con 0x7f23740b3070 2023-10-16T11:06:14.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.711+0000 7f23727fc640 1 -- 172.21.15.84:0/1551590276 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2360003520 con 0x7f23740b3070 2023-10-16T11:06:14.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.711+0000 7f23727fc640 1 -- 172.21.15.84:0/1551590276 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4212273940 0 0) 0x7f2360003520 con 0x7f23740b3070 2023-10-16T11:06:14.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.711+0000 7f23727fc640 1 -- 172.21.15.84:0/1551590276 >> v2:172.21.15.157:3300/0 conn(0x7f23740b3e70 msgr2=0x7f23740d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:14.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.711+0000 7f23727fc640 1 --2- 172.21.15.84:0/1551590276 >> v2:172.21.15.157:3300/0 conn(0x7f23740b3e70 0x7f23740d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.711+0000 7f23727fc640 1 -- 172.21.15.84:0/1551590276 >> v1:172.21.15.157:6789/0 conn(0x7f23740cfc10 legacy=0x7f23740d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:14.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.711+0000 7f23727fc640 1 -- 172.21.15.84:0/1551590276 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740b3b90 con 0x7f23740b3070 2023-10-16T11:06:14.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.711+0000 7f23727fc640 1 -- 172.21.15.84:0/1551590276 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f2364001b70 con 0x7f23740b3070 2023-10-16T11:06:14.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.711+0000 7f2373fff640 1 --2- 172.21.15.84:0/1551590276 >> v2:172.21.15.157:3300/0 conn(0x7f23740b3e70 0x7f23740d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:06:14.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.711+0000 7f23727fc640 1 -- 172.21.15.84:0/1551590276 >> v1:172.21.15.84:6789/0 conn(0x7f23740b3070 legacy=0x7f23740cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:14.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.711+0000 7f23727fc640 1 --2- 172.21.15.84:0/1551590276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23600039c0 0x7f2360003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.712+0000 7f23727fc640 1 --2- 172.21.15.84:0/1551590276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23600047c0 0x7f2360004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.712+0000 7f23727fc640 1 --2- 172.21.15.84:0/1551590276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23600051d0 0x7f236000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.712+0000 7f23727fc640 1 -- 172.21.15.84:0/1551590276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740c7880 con 0x7f23600047c0 2023-10-16T11:06:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.712+0000 7f23727fc640 1 -- 172.21.15.84:0/1551590276 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236000c120 con 0x7f23600039c0 2023-10-16T11:06:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.712+0000 7f23727fc640 1 -- 172.21.15.84:0/1551590276 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236000c2e0 con 0x7f23600051d0 2023-10-16T11:06:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.712+0000 7f2372ffd640 1 --2- 172.21.15.84:0/1551590276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23600047c0 0x7f2360004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:14.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.712+0000 7f23737fe640 1 --2- 172.21.15.84:0/1551590276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23600039c0 0x7f2360003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:14.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.712+0000 7f2373fff640 1 --2- 172.21.15.84:0/1551590276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23600051d0 0x7f236000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:14.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.712+0000 7f2372ffd640 1 -- 172.21.15.84:0/1551590276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23600051d0 msgr2=0x7f236000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:14.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.712+0000 7f2372ffd640 1 --2- 172.21.15.84:0/1551590276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23600051d0 0x7f236000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.712+0000 7f2372ffd640 1 -- 172.21.15.84:0/1551590276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23600039c0 msgr2=0x7f2360003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.712+0000 7f2372ffd640 1 --2- 172.21.15.84:0/1551590276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23600039c0 0x7f2360003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.712+0000 7f2372ffd640 1 -- 172.21.15.84:0/1551590276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2360003520 con 0x7f23600047c0 2023-10-16T11:06:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.712+0000 7f2373fff640 1 --2- 172.21.15.84:0/1551590276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23600051d0 0x7f236000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.712+0000 7f2373fff640 1 -- 172.21.15.84:0/1551590276 reap_dead start 2023-10-16T11:06:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.713+0000 7f2372ffd640 1 --2- 172.21.15.84:0/1551590276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23600047c0 0x7f2360004c40 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f235c001750 tx=0x7f23600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3930c4807e0d8c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.714+0000 7f23727fc640 1 -- 172.21.15.84:0/1551590276 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c0052d0 con 0x7f23600047c0 2023-10-16T11:06:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.714+0000 7f23727fc640 1 -- 172.21.15.84:0/1551590276 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c02e050 con 0x7f23600047c0 2023-10-16T11:06:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.714+0000 7f23727fc640 1 -- 172.21.15.84:0/1551590276 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c003830 con 0x7f23600047c0 2023-10-16T11:06:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.714+0000 7f2379e2b640 1 -- 172.21.15.84:0/1551590276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23600047c0 msgr2=0x7f2360004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.714+0000 7f2379e2b640 1 --2- 172.21.15.84:0/1551590276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23600047c0 0x7f2360004c40 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f235c001750 tx=0x7f23600057f0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.714+0000 7f2379e2b640 1 -- 172.21.15.84:0/1551590276 shutdown_connections 2023-10-16T11:06:14.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.714+0000 7f2379e2b640 1 --2- 172.21.15.84:0/1551590276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23600047c0 0x7f2360004c40 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.714+0000 7f2379e2b640 1 -- 172.21.15.84:0/1551590276 >> 172.21.15.84:0/1551590276 conn(0x7f23740d2d20 msgr2=0x7f23740d3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:14.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.714+0000 7f2379e2b640 1 -- 172.21.15.84:0/1551590276 shutdown_connections 2023-10-16T11:06:14.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.714+0000 7f2379e2b640 1 -- 172.21.15.84:0/1551590276 wait complete. 2023-10-16T11:06:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.716+0000 7f2379e2b640 1 Processor -- start 2023-10-16T11:06:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.716+0000 7f2379e2b640 1 -- start start 2023-10-16T11:06:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.717+0000 7f2379e2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23600047c0 0x7f237415b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.717+0000 7f2379e2b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f237415c060 0x7f237415f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.717+0000 7f2379e2b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237415f650 0x7f23740caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.717+0000 7f2379e2b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237415ecf0 con 0x7f23600047c0 2023-10-16T11:06:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.717+0000 7f2379e2b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740c85b0 con 0x7f237415c060 2023-10-16T11:06:14.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.717+0000 7f2379e2b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23740c7ff0 con 0x7f237415f650 2023-10-16T11:06:14.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.717+0000 7f23737fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23600047c0 0x7f237415b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:14.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.717+0000 7f23737fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23600047c0 0x7f237415b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56454/0 (socket says 172.21.15.84:56454) 2023-10-16T11:06:14.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.717+0000 7f23737fe640 1 -- 172.21.15.84:0/4081185060 learned_addr learned my addr 172.21.15.84:0/4081185060 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:14.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.718+0000 7f23737fe640 1 -- 172.21.15.84:0/4081185060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237415f650 msgr2=0x7f23740caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:14.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.718+0000 7f23737fe640 1 --2- 172.21.15.84:0/4081185060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237415f650 0x7f23740caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.718+0000 7f23737fe640 1 -- 172.21.15.84:0/4081185060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f237415c060 msgr2=0x7f237415f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:14.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.718+0000 7f23737fe640 1 --2- 172.21.15.84:0/4081185060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f237415c060 0x7f237415f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.718+0000 7f23737fe640 1 -- 172.21.15.84:0/4081185060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740c82d0 con 0x7f23600047c0 2023-10-16T11:06:14.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.718+0000 7f23737fe640 1 --2- 172.21.15.84:0/4081185060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23600047c0 0x7f237415b890 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f2364002230 tx=0x7f236400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:14.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.718+0000 7f236bfff640 1 -- 172.21.15.84:0/4081185060 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364005d30 con 0x7f23600047c0 2023-10-16T11:06:14.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.718+0000 7f236bfff640 1 -- 172.21.15.84:0/4081185060 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236400c780 con 0x7f23600047c0 2023-10-16T11:06:14.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.718+0000 7f236bfff640 1 -- 172.21.15.84:0/4081185060 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236400cb30 con 0x7f23600047c0 2023-10-16T11:06:14.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.718+0000 7f2379e2b640 1 -- 172.21.15.84:0/4081185060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237415e630 con 0x7f23600047c0 2023-10-16T11:06:14.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.719+0000 7f2379e2b640 1 -- 172.21.15.84:0/4081185060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237415bdd0 con 0x7f23600047c0 2023-10-16T11:06:14.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.720+0000 7f2379e2b640 1 -- 172.21.15.84:0/4081185060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237415ef60 con 0x7f23600047c0 2023-10-16T11:06:14.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.721+0000 7f236bfff640 1 -- 172.21.15.84:0/4081185060 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f236400cc90 con 0x7f23600047c0 2023-10-16T11:06:14.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.722+0000 7f236bfff640 1 --2- 172.21.15.84:0/4081185060 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f235406afc0 0x7f235406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:14.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.722+0000 7f236bfff640 1 -- 172.21.15.84:0/4081185060 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f23640899f0 con 0x7f23600047c0 2023-10-16T11:06:14.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.722+0000 7f2372ffd640 1 --2- 172.21.15.84:0/4081185060 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f235406afc0 0x7f235406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:14.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.723+0000 7f2372ffd640 1 --2- 172.21.15.84:0/4081185060 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f235406afc0 0x7f235406d480 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f235c033000 tx=0x7f235c02f000 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:14.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.727+0000 7f236bfff640 1 -- 172.21.15.84:0/4081185060 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f23640549c0 con 0x7f23600047c0 2023-10-16T11:06:14.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.920+0000 7f2379e2b640 1 -- 172.21.15.84:0/4081185060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23740d37c0 con 0x7f23600047c0 2023-10-16T11:06:14.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.921+0000 7f236bfff640 1 -- 172.21.15.84:0/4081185060 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+14017 (secure 0 0 0) 0x7f2364057fe0 con 0x7f23600047c0 2023-10-16T11:06:14.923 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:14.923 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":176,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:06:12.456713+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6837/424056375":"2023-10-17T11:06:09.378226+0000","172.21.15.84:0/849217271":"2023-10-17T11:06:05.639149+0000","172.21.15.84:0/1510301035":"2023-10-17T11:06:05.639149+0000","172.21.15.84:0/2220106058":"2023-10-17T11:06:05.639149+0000","172.21.15.84:6833/1697458478":"2023-10-17T11:06:05.639149+0000","172.21.15.84:0/1365551239":"2023-10-17T11:06:05.639149+0000","172.21.15.84:0/2246425640":"2023-10-17T11:06:05.639149+0000","172.21.15.84:0/3425147335":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/1161603770":"2023-10-17T11:05:57.603800+0000","172.21.15.84:6832/1697458478":"2023-10-17T11:06:05.639149+0000","172.21.15.84:6833/2442400184":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/3742714530":"2023-10-17T11:06:05.639149+0000","172.21.15.84:6832/2442400184":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/3633178028":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/1914327915":"2023-10-17T11:05:57.603800+0000","172.21.15.84:6836/424056375":"2023-10-17T11:06:09.378226+0000","172.21.15.84:0/2602045766":"2023-10-17T11:05:57.603800+0000","172.21.15.84:0/3065897528":"2023-10-17T11:05:57.603800+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:06:14.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.924+0000 7f2379e2b640 1 -- 172.21.15.84:0/4081185060 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f235406afc0 msgr2=0x7f235406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:14.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.924+0000 7f2379e2b640 1 --2- 172.21.15.84:0/4081185060 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f235406afc0 0x7f235406d480 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f235c033000 tx=0x7f235c02f000 comp rx=0 tx=0).stop 2023-10-16T11:06:14.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.924+0000 7f2379e2b640 1 -- 172.21.15.84:0/4081185060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23600047c0 msgr2=0x7f237415b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:14.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.924+0000 7f2379e2b640 1 --2- 172.21.15.84:0/4081185060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23600047c0 0x7f237415b890 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f2364002230 tx=0x7f236400ef00 comp rx=0 tx=0).stop 2023-10-16T11:06:14.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.924+0000 7f2379e2b640 1 -- 172.21.15.84:0/4081185060 shutdown_connections 2023-10-16T11:06:14.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.924+0000 7f2379e2b640 1 --2- 172.21.15.84:0/4081185060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f237415f650 0x7f23740caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.924+0000 7f2379e2b640 1 --2- 172.21.15.84:0/4081185060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f237415c060 0x7f237415f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.924+0000 7f2379e2b640 1 --2- 172.21.15.84:0/4081185060 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f235406afc0 0x7f235406d480 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.924+0000 7f2379e2b640 1 --2- 172.21.15.84:0/4081185060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23600047c0 0x7f237415b890 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:14.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.924+0000 7f2379e2b640 1 -- 172.21.15.84:0/4081185060 >> 172.21.15.84:0/4081185060 conn(0x7f23740d2d20 msgr2=0x7f2374160640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:14.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.924+0000 7f2379e2b640 1 -- 172.21.15.84:0/4081185060 shutdown_connections 2023-10-16T11:06:14.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:14.924+0000 7f2379e2b640 1 -- 172.21.15.84:0/4081185060 wait complete. 2023-10-16T11:06:14.944 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:06:14.944 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:06:15.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.163+0000 7f214aa44640 1 Processor -- start 2023-10-16T11:06:15.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.163+0000 7f214aa44640 1 -- start start 2023-10-16T11:06:15.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.165+0000 7f214aa44640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f214415ad70 0x7f2144069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:15.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.165+0000 7f214aa44640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21440c9ca0 0x7f214406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:15.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.165+0000 7f214aa44640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f21440ca3b0 0x7f21440ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:15.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.165+0000 7f214aa44640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21440cad80 con 0x7f21440ca3b0 2023-10-16T11:06:15.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.165+0000 7f214aa44640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21440cb030 con 0x7f214415ad70 2023-10-16T11:06:15.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.165+0000 7f214aa44640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21440c91c0 con 0x7f21440c9ca0 2023-10-16T11:06:15.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.165+0000 7f2148fe1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f21440ca3b0 0x7f21440ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:15.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.165+0000 7f2148fe1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f21440ca3b0 0x7f21440ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56464/0 (socket says 172.21.15.84:56464) 2023-10-16T11:06:15.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.165+0000 7f2148fe1640 1 -- 172.21.15.84:0/1344624421 learned_addr learned my addr 172.21.15.84:0/1344624421 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:15.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.165+0000 7f2143fff640 1 --2- 172.21.15.84:0/1344624421 >> v2:172.21.15.157:3300/0 conn(0x7f214415ad70 0x7f2144069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:15.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.165+0000 7f213b7fe640 1 --2- 172.21.15.84:0/1344624421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21440c9ca0 0x7f214406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:15.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.165+0000 7f2148fe1640 1 -- 172.21.15.84:0/1344624421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21440c9ca0 msgr2=0x7f214406a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:15.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.166+0000 7f2148fe1640 1 --2- 172.21.15.84:0/1344624421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21440c9ca0 0x7f214406a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:15.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.166+0000 7f2148fe1640 1 -- 172.21.15.84:0/1344624421 >> v2:172.21.15.157:3300/0 conn(0x7f214415ad70 msgr2=0x7f2144069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:15.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.166+0000 7f2148fe1640 1 --2- 172.21.15.84:0/1344624421 >> v2:172.21.15.157:3300/0 conn(0x7f214415ad70 0x7f2144069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:15.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.166+0000 7f2148fe1640 1 -- 172.21.15.84:0/1344624421 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21440c94a0 con 0x7f21440ca3b0 2023-10-16T11:06:15.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.167+0000 7f2148fe1640 1 --2- 172.21.15.84:0/1344624421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21440ca3b0 0x7f21440ca840 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f21440cf780 tx=0x7f213c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1eeb86bf773b01 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:15.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.167+0000 7f21437fe640 1 -- 172.21.15.84:0/1344624421 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c031870 con 0x7f21440ca3b0 2023-10-16T11:06:15.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.167+0000 7f21437fe640 1 -- 172.21.15.84:0/1344624421 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c0319d0 con 0x7f21440ca3b0 2023-10-16T11:06:15.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.167+0000 7f21437fe640 1 -- 172.21.15.84:0/1344624421 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c031d60 con 0x7f21440ca3b0 2023-10-16T11:06:15.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.167+0000 7f214aa44640 1 -- 172.21.15.84:0/1344624421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21440ca3b0 msgr2=0x7f21440ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:15.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.167+0000 7f214aa44640 1 --2- 172.21.15.84:0/1344624421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21440ca3b0 0x7f21440ca840 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f21440cf780 tx=0x7f213c02b910 comp rx=0 tx=0).stop 2023-10-16T11:06:15.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.168+0000 7f214aa44640 1 -- 172.21.15.84:0/1344624421 shutdown_connections 2023-10-16T11:06:15.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.168+0000 7f214aa44640 1 --2- 172.21.15.84:0/1344624421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21440ca3b0 0x7f21440ca840 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:15.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.168+0000 7f214aa44640 1 --2- 172.21.15.84:0/1344624421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21440c9ca0 0x7f214406a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:15.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.168+0000 7f214aa44640 1 --2- 172.21.15.84:0/1344624421 >> v2:172.21.15.157:3300/0 conn(0x7f214415ad70 0x7f2144069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:15.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.168+0000 7f214aa44640 1 -- 172.21.15.84:0/1344624421 >> 172.21.15.84:0/1344624421 conn(0x7f21440ce9b0 msgr2=0x7f21440cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:15.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.168+0000 7f214aa44640 1 -- 172.21.15.84:0/1344624421 shutdown_connections 2023-10-16T11:06:15.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.168+0000 7f214aa44640 1 -- 172.21.15.84:0/1344624421 wait complete. 2023-10-16T11:06:15.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.169+0000 7f214aa44640 1 Processor -- start 2023-10-16T11:06:15.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.170+0000 7f214aa44640 1 -- start start 2023-10-16T11:06:15.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.170+0000 7f214aa44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21440c9ca0 0x7f21440c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:15.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.170+0000 7f214aa44640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21440ca3b0 0x7f21440ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:15.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.171+0000 7f214aa44640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f214415ad70 0x7f2144157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:15.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.171+0000 7f214aa44640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21440acf50 con 0x7f21440c9ca0 2023-10-16T11:06:15.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.171+0000 7f214aa44640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21440c6ad0 con 0x7f214415ad70 2023-10-16T11:06:15.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.171+0000 7f214aa44640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21440c7370 con 0x7f21440ca3b0 2023-10-16T11:06:15.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.171+0000 7f2143fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21440c9ca0 0x7f21440c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:15.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.171+0000 7f2143fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21440c9ca0 0x7f21440c7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56476/0 (socket says 172.21.15.84:56476) 2023-10-16T11:06:15.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.171+0000 7f2143fff640 1 -- 172.21.15.84:0/2873767858 learned_addr learned my addr 172.21.15.84:0/2873767858 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:15.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.171+0000 7f2143fff640 1 -- 172.21.15.84:0/2873767858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21440ca3b0 msgr2=0x7f21440ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:15.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.171+0000 7f2143fff640 1 --2- 172.21.15.84:0/2873767858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21440ca3b0 0x7f21440ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:15.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.171+0000 7f2143fff640 1 -- 172.21.15.84:0/2873767858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f214415ad70 msgr2=0x7f2144157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:15.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.171+0000 7f2143fff640 1 --2- 172.21.15.84:0/2873767858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f214415ad70 0x7f2144157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:15.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.171+0000 7f2143fff640 1 -- 172.21.15.84:0/2873767858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c0094d0 con 0x7f21440c9ca0 2023-10-16T11:06:15.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.171+0000 7f2143fff640 1 --2- 172.21.15.84:0/2873767858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21440c9ca0 0x7f21440c7800 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f213400cd00 tx=0x7f213400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:15.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.172+0000 7f21417fa640 1 -- 172.21.15.84:0/2873767858 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213400ae80 con 0x7f21440c9ca0 2023-10-16T11:06:15.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.172+0000 7f21417fa640 1 -- 172.21.15.84:0/2873767858 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2134003990 con 0x7f21440c9ca0 2023-10-16T11:06:15.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.172+0000 7f21417fa640 1 -- 172.21.15.84:0/2873767858 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134003d00 con 0x7f21440c9ca0 2023-10-16T11:06:15.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.172+0000 7f214aa44640 1 -- 172.21.15.84:0/2873767858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2144157fd0 con 0x7f21440c9ca0 2023-10-16T11:06:15.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.172+0000 7f214aa44640 1 -- 172.21.15.84:0/2873767858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21440c6d20 con 0x7f21440c9ca0 2023-10-16T11:06:15.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.173+0000 7f214aa44640 1 -- 172.21.15.84:0/2873767858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f214404e160 con 0x7f21440c9ca0 2023-10-16T11:06:15.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.174+0000 7f21417fa640 1 -- 172.21.15.84:0/2873767858 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2134004020 con 0x7f21440c9ca0 2023-10-16T11:06:15.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.175+0000 7f21417fa640 1 --2- 172.21.15.84:0/2873767858 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f212406af90 0x7f212406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:15.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.175+0000 7f21417fa640 1 -- 172.21.15.84:0/2873767858 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f2134089c20 con 0x7f21440c9ca0 2023-10-16T11:06:15.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.175+0000 7f213b7fe640 1 --2- 172.21.15.84:0/2873767858 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f212406af90 0x7f212406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:15.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.176+0000 7f213b7fe640 1 --2- 172.21.15.84:0/2873767858 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f212406af90 0x7f212406d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f212c006b10 tx=0x7f212c008040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:15.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.179+0000 7f21417fa640 1 -- 172.21.15.84:0/2873767858 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2134054ae0 con 0x7f21440c9ca0 2023-10-16T11:06:15.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.375+0000 7f214aa44640 1 -- 172.21.15.84:0/2873767858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f214415c8f0 con 0x7f21440c9ca0 2023-10-16T11:06:15.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.376+0000 7f21417fa640 1 -- 172.21.15.84:0/2873767858 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v176) v1 ==== 93+0+10 (secure 0 0 0) 0x7f213401d070 con 0x7f21440c9ca0 2023-10-16T11:06:15.377 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:06:15.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.378+0000 7f214aa44640 1 -- 172.21.15.84:0/2873767858 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f212406af90 msgr2=0x7f212406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:15.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.378+0000 7f214aa44640 1 --2- 172.21.15.84:0/2873767858 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f212406af90 0x7f212406d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f212c006b10 tx=0x7f212c008040 comp rx=0 tx=0).stop 2023-10-16T11:06:15.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.378+0000 7f214aa44640 1 -- 172.21.15.84:0/2873767858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21440c9ca0 msgr2=0x7f21440c7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:15.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.378+0000 7f214aa44640 1 --2- 172.21.15.84:0/2873767858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21440c9ca0 0x7f21440c7800 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f213400cd00 tx=0x7f213400a5c0 comp rx=0 tx=0).stop 2023-10-16T11:06:15.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.378+0000 7f214aa44640 1 -- 172.21.15.84:0/2873767858 shutdown_connections 2023-10-16T11:06:15.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.378+0000 7f214aa44640 1 --2- 172.21.15.84:0/2873767858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f214415ad70 0x7f2144157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:15.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.378+0000 7f214aa44640 1 --2- 172.21.15.84:0/2873767858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21440ca3b0 0x7f21440ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:15.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.378+0000 7f214aa44640 1 --2- 172.21.15.84:0/2873767858 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f212406af90 0x7f212406d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:15.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.378+0000 7f214aa44640 1 --2- 172.21.15.84:0/2873767858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21440c9ca0 0x7f21440c7800 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:15.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.379+0000 7f214aa44640 1 -- 172.21.15.84:0/2873767858 >> 172.21.15.84:0/2873767858 conn(0x7f21440ce9b0 msgr2=0x7f21440afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:15.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.379+0000 7f214aa44640 1 -- 172.21.15.84:0/2873767858 shutdown_connections 2023-10-16T11:06:15.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.379+0000 7f214aa44640 1 -- 172.21.15.84:0/2873767858 wait complete. 2023-10-16T11:06:15.398 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-10-16T11:06:15.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.607+0000 7f220ca2d640 1 Processor -- start 2023-10-16T11:06:15.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.607+0000 7f220ca2d640 1 -- start start 2023-10-16T11:06:15.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.608+0000 7f220ca2d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f220815aaa0 0x7f220815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:15.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.608+0000 7f220ca2d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f220815ae80 0x7f22080cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:15.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.609+0000 7f220ca2d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f220815ed60 con 0x7f22080ccaa0 2023-10-16T11:06:15.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.609+0000 7f220ca2d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220806b810 con 0x7f220815aaa0 2023-10-16T11:06:15.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.609+0000 7f220ca2d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22080707e0 con 0x7f220815ae80 2023-10-16T11:06:15.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.609+0000 7f220659c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f220815aaa0 0x7f220815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:15.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.609+0000 7f220659c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f220815aaa0 0x7f220815e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43464/0 (socket says 172.21.15.84:43464) 2023-10-16T11:06:15.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.609+0000 7f220659c640 1 -- 172.21.15.84:0/3598290953 learned_addr learned my addr 172.21.15.84:0/3598290953 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:15.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.609+0000 7f220659c640 1 -- 172.21.15.84:0/3598290953 >> v2:172.21.15.157:3300/0 conn(0x7f220815ae80 msgr2=0x7f22080cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:15.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.609+0000 7f21fffff640 1 --2- 172.21.15.84:0/3598290953 >> v2:172.21.15.157:3300/0 conn(0x7f220815ae80 0x7f22080cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:15.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.609+0000 7f220659c640 1 --2- 172.21.15.84:0/3598290953 >> v2:172.21.15.157:3300/0 conn(0x7f220815ae80 0x7f22080cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:15.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.609+0000 7f220659c640 1 -- 172.21.15.84:0/3598290953 >> v1:172.21.15.157:6789/0 conn(0x7f22080ccaa0 legacy=0x7f2208070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:15.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.609+0000 7f220659c640 1 -- 172.21.15.84:0/3598290953 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220815ef90 con 0x7f220815aaa0 2023-10-16T11:06:15.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.610+0000 7f220659c640 1 --2- 172.21.15.84:0/3598290953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220815aaa0 0x7f220815e820 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f22080c59c0 tx=0x7f21f802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d700a4ed7abd4df server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:15.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.611+0000 7f2205d9b640 1 -- 172.21.15.84:0/3598290953 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8031870 con 0x7f220815aaa0 2023-10-16T11:06:15.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.611+0000 7f2205d9b640 1 -- 172.21.15.84:0/3598290953 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f80319d0 con 0x7f220815aaa0 2023-10-16T11:06:15.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.611+0000 7f2205d9b640 1 -- 172.21.15.84:0/3598290953 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8031d60 con 0x7f220815aaa0 2023-10-16T11:06:15.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.612+0000 7f220ca2d640 1 -- 172.21.15.84:0/3598290953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220815aaa0 msgr2=0x7f220815e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:15.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.612+0000 7f220ca2d640 1 --2- 172.21.15.84:0/3598290953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220815aaa0 0x7f220815e820 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f22080c59c0 tx=0x7f21f802b910 comp rx=0 tx=0).stop 2023-10-16T11:06:15.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.612+0000 7f220ca2d640 1 -- 172.21.15.84:0/3598290953 shutdown_connections 2023-10-16T11:06:15.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.612+0000 7f220ca2d640 1 --2- 172.21.15.84:0/3598290953 >> v2:172.21.15.157:3300/0 conn(0x7f220815ae80 0x7f22080cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:15.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.612+0000 7f220ca2d640 1 --2- 172.21.15.84:0/3598290953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220815aaa0 0x7f220815e820 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:15.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.612+0000 7f220ca2d640 1 -- 172.21.15.84:0/3598290953 >> 172.21.15.84:0/3598290953 conn(0x7f22080cb410 msgr2=0x7f22080cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:15.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.612+0000 7f220ca2d640 1 -- 172.21.15.84:0/3598290953 shutdown_connections 2023-10-16T11:06:15.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.612+0000 7f220ca2d640 1 -- 172.21.15.84:0/3598290953 wait complete. 2023-10-16T11:06:15.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.613+0000 7f220ca2d640 1 Processor -- start 2023-10-16T11:06:15.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.614+0000 7f220ca2d640 1 -- start start 2023-10-16T11:06:15.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.614+0000 7f220ca2d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22080ccaa0 0x7f2208156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:15.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.614+0000 7f220ca2d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220815aaa0 0x7f22081585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:15.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.615+0000 7f220ca2d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f220815ae80 0x7f2208155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:15.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.615+0000 7f220ca2d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22081571e0 con 0x7f220815aaa0 2023-10-16T11:06:15.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.615+0000 7f220ca2d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2208158e30 con 0x7f22080ccaa0 2023-10-16T11:06:15.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.615+0000 7f220ca2d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2208159110 con 0x7f220815ae80 2023-10-16T11:06:15.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.615+0000 7f21fffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220815aaa0 0x7f22081585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:15.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.615+0000 7f21fffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220815aaa0 0x7f22081585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43478/0 (socket says 172.21.15.84:43478) 2023-10-16T11:06:15.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.615+0000 7f21fffff640 1 -- 172.21.15.84:0/4244008198 learned_addr learned my addr 172.21.15.84:0/4244008198 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:15.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.615+0000 7f220659c640 1 --2- 172.21.15.84:0/4244008198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22080ccaa0 0x7f2208156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:15.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.615+0000 7f2206d9d640 1 --2- 172.21.15.84:0/4244008198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f220815ae80 0x7f2208155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:15.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.615+0000 7f21fffff640 1 -- 172.21.15.84:0/4244008198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f220815ae80 msgr2=0x7f2208155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.615+0000 7f21fffff640 1 --2- 172.21.15.84:0/4244008198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f220815ae80 0x7f2208155880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.615+0000 7f21fffff640 1 -- 172.21.15.84:0/4244008198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22080ccaa0 msgr2=0x7f2208156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.615+0000 7f21fffff640 1 --2- 172.21.15.84:0/4244008198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22080ccaa0 0x7f2208156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.615+0000 7f21fffff640 1 -- 172.21.15.84:0/4244008198 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f80094d0 con 0x7f220815aaa0 2023-10-16T11:06:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.615+0000 7f220659c640 1 --2- 172.21.15.84:0/4244008198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22080ccaa0 0x7f2208156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.616+0000 7f21fffff640 1 --2- 172.21.15.84:0/4244008198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220815aaa0 0x7f22081585c0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f21f000c7c0 tx=0x7f21f000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.616+0000 7f21feffd640 1 -- 172.21.15.84:0/4244008198 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0007820 con 0x7f220815aaa0 2023-10-16T11:06:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.616+0000 7f21feffd640 1 -- 172.21.15.84:0/4244008198 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f0007980 con 0x7f220815aaa0 2023-10-16T11:06:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.616+0000 7f220ca2d640 1 -- 172.21.15.84:0/4244008198 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22081593f0 con 0x7f220815aaa0 2023-10-16T11:06:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.616+0000 7f21feffd640 1 -- 172.21.15.84:0/4244008198 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0007cf0 con 0x7f220815aaa0 2023-10-16T11:06:15.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.616+0000 7f220ca2d640 1 -- 172.21.15.84:0/4244008198 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22081599b0 con 0x7f220815aaa0 2023-10-16T11:06:15.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.617+0000 7f220ca2d640 1 -- 172.21.15.84:0/4244008198 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f220804e160 con 0x7f220815aaa0 2023-10-16T11:06:15.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.618+0000 7f21feffd640 1 -- 172.21.15.84:0/4244008198 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f21f00113f0 con 0x7f220815aaa0 2023-10-16T11:06:15.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.619+0000 7f21feffd640 1 --2- 172.21.15.84:0/4244008198 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f21e806af90 0x7f21e806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:15.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.620+0000 7f21feffd640 1 -- 172.21.15.84:0/4244008198 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f21f0089510 con 0x7f220815aaa0 2023-10-16T11:06:15.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.620+0000 7f220659c640 1 --2- 172.21.15.84:0/4244008198 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f21e806af90 0x7f21e806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:15.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.620+0000 7f220659c640 1 --2- 172.21.15.84:0/4244008198 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f21e806af90 0x7f21e806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f21f8036000 tx=0x7f21f803c040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:15.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.624+0000 7f21feffd640 1 -- 172.21.15.84:0/4244008198 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f21f0054460 con 0x7f220815aaa0 2023-10-16T11:06:15.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:15.874+0000 7f220ca2d640 1 -- 172.21.15.84:0/4244008198 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f22080cdc60 con 0x7f220815aaa0 2023-10-16T11:06:16.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.491+0000 7f21feffd640 1 -- 172.21.15.84:0/4244008198 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2532) v1 ==== 168+0+0 (secure 0 0 0) 0x7f21f0057a80 con 0x7f220815aaa0 2023-10-16T11:06:16.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.494+0000 7f220ca2d640 1 -- 172.21.15.84:0/4244008198 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f21e806af90 msgr2=0x7f21e806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:16.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.494+0000 7f220ca2d640 1 --2- 172.21.15.84:0/4244008198 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f21e806af90 0x7f21e806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f21f8036000 tx=0x7f21f803c040 comp rx=0 tx=0).stop 2023-10-16T11:06:16.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.494+0000 7f220ca2d640 1 -- 172.21.15.84:0/4244008198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220815aaa0 msgr2=0x7f22081585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:16.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.494+0000 7f220ca2d640 1 --2- 172.21.15.84:0/4244008198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220815aaa0 0x7f22081585c0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f21f000c7c0 tx=0x7f21f000cc80 comp rx=0 tx=0).stop 2023-10-16T11:06:16.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.495+0000 7f220ca2d640 1 -- 172.21.15.84:0/4244008198 shutdown_connections 2023-10-16T11:06:16.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.495+0000 7f220ca2d640 1 --2- 172.21.15.84:0/4244008198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f220815ae80 0x7f2208155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:16.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.495+0000 7f220ca2d640 1 --2- 172.21.15.84:0/4244008198 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f21e806af90 0x7f21e806d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:16.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.495+0000 7f220ca2d640 1 --2- 172.21.15.84:0/4244008198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220815aaa0 0x7f22081585c0 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:16.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.495+0000 7f220ca2d640 1 --2- 172.21.15.84:0/4244008198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22080ccaa0 0x7f2208156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:16.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.495+0000 7f220ca2d640 1 -- 172.21.15.84:0/4244008198 >> 172.21.15.84:0/4244008198 conn(0x7f22080cb410 msgr2=0x7f22080ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:16.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.495+0000 7f220ca2d640 1 -- 172.21.15.84:0/4244008198 shutdown_connections 2023-10-16T11:06:16.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.495+0000 7f220ca2d640 1 -- 172.21.15.84:0/4244008198 wait complete. 2023-10-16T11:06:16.516 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T11:06:16.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.731+0000 7fcb4dc94640 1 Processor -- start 2023-10-16T11:06:16.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.731+0000 7fcb4dc94640 1 -- start start 2023-10-16T11:06:16.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.732+0000 7fcb4dc94640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcb480caac0 0x7fcb480ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:16.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.732+0000 7fcb4dc94640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb480cb030 con 0x7fcb4806a4b0 2023-10-16T11:06:16.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.733+0000 7fcb4dc94640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb480c91c0 con 0x7fcb4815ad70 2023-10-16T11:06:16.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.733+0000 7fcb4dc94640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb480c94a0 con 0x7fcb480caac0 2023-10-16T11:06:16.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.733+0000 7fcb3ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fcb4806a4b0 0x7fcb480ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49728/0 (socket says 172.21.15.84:49728) 2023-10-16T11:06:16.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.733+0000 7fcb3ffff640 1 -- 172.21.15.84:0/3092333275 learned_addr learned my addr 172.21.15.84:0/3092333275 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:16.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.733+0000 7fcb47fff640 1 --2- 172.21.15.84:0/3092333275 >> v2:172.21.15.157:3300/0 conn(0x7fcb480caac0 0x7fcb480ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:16.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.733+0000 7fcb46ffd640 1 -- 172.21.15.84:0/3092333275 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 644139496 0 0) 0x7fcb480cb030 con 0x7fcb4806a4b0 2023-10-16T11:06:16.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.733+0000 7fcb46ffd640 1 -- 172.21.15.84:0/3092333275 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb34003520 con 0x7fcb4806a4b0 2023-10-16T11:06:16.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.734+0000 7fcb46ffd640 1 -- 172.21.15.84:0/3092333275 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 20122882 0 0) 0x7fcb34003520 con 0x7fcb4806a4b0 2023-10-16T11:06:16.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.734+0000 7fcb46ffd640 1 -- 172.21.15.84:0/3092333275 >> v2:172.21.15.157:3300/0 conn(0x7fcb480caac0 msgr2=0x7fcb480ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:16.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.734+0000 7fcb46ffd640 1 --2- 172.21.15.84:0/3092333275 >> v2:172.21.15.157:3300/0 conn(0x7fcb480caac0 0x7fcb480ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:16.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.734+0000 7fcb46ffd640 1 -- 172.21.15.84:0/3092333275 >> v1:172.21.15.157:6789/0 conn(0x7fcb4815ad70 legacy=0x7fcb48069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:16.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.734+0000 7fcb46ffd640 1 -- 172.21.15.84:0/3092333275 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb480c9780 con 0x7fcb4806a4b0 2023-10-16T11:06:16.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.734+0000 7fcb46ffd640 1 -- 172.21.15.84:0/3092333275 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fcb30001670 con 0x7fcb4806a4b0 2023-10-16T11:06:16.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.734+0000 7fcb46ffd640 1 -- 172.21.15.84:0/3092333275 >> v1:172.21.15.84:6789/0 conn(0x7fcb4806a4b0 legacy=0x7fcb480ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:16.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.734+0000 7fcb46ffd640 1 --2- 172.21.15.84:0/3092333275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb34003970 0x7fcb34003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:16.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.734+0000 7fcb46ffd640 1 --2- 172.21.15.84:0/3092333275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb340047c0 0x7fcb34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:16.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.734+0000 7fcb46ffd640 1 --2- 172.21.15.84:0/3092333275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb34005180 0x7fcb3400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:16.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.734+0000 7fcb46ffd640 1 -- 172.21.15.84:0/3092333275 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb480c94a0 con 0x7fcb34003970 2023-10-16T11:06:16.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.734+0000 7fcb46ffd640 1 -- 172.21.15.84:0/3092333275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3400c020 con 0x7fcb34005180 2023-10-16T11:06:16.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.734+0000 7fcb46ffd640 1 -- 172.21.15.84:0/3092333275 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3400c1e0 con 0x7fcb340047c0 2023-10-16T11:06:16.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.734+0000 7fcb477fe640 1 --2- 172.21.15.84:0/3092333275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb34003970 0x7fcb34003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:16.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.734+0000 7fcb47fff640 1 --2- 172.21.15.84:0/3092333275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb34005180 0x7fcb3400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:16.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.734+0000 7fcb3ffff640 1 --2- 172.21.15.84:0/3092333275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb340047c0 0x7fcb34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:16.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.734+0000 7fcb477fe640 1 -- 172.21.15.84:0/3092333275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb340047c0 msgr2=0x7fcb34004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:16.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.735+0000 7fcb477fe640 1 --2- 172.21.15.84:0/3092333275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb340047c0 0x7fcb34004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:16.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.735+0000 7fcb477fe640 1 -- 172.21.15.84:0/3092333275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb34005180 msgr2=0x7fcb3400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:16.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.735+0000 7fcb477fe640 1 --2- 172.21.15.84:0/3092333275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb34005180 0x7fcb3400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:16.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.735+0000 7fcb477fe640 1 -- 172.21.15.84:0/3092333275 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb34003520 con 0x7fcb34003970 2023-10-16T11:06:16.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.735+0000 7fcb47fff640 1 -- 172.21.15.84:0/3092333275 reap_dead start 2023-10-16T11:06:16.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.736+0000 7fcb477fe640 1 --2- 172.21.15.84:0/3092333275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb34003970 0x7fcb34003d90 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fcb38001750 tx=0x7fcb340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b0bca71da4112d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:16.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.736+0000 7fcb46ffd640 1 -- 172.21.15.84:0/3092333275 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb380052d0 con 0x7fcb34003970 2023-10-16T11:06:16.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.736+0000 7fcb46ffd640 1 -- 172.21.15.84:0/3092333275 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3802e050 con 0x7fcb34003970 2023-10-16T11:06:16.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.736+0000 7fcb46ffd640 1 -- 172.21.15.84:0/3092333275 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb38003830 con 0x7fcb34003970 2023-10-16T11:06:16.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.736+0000 7fcb4dc94640 1 -- 172.21.15.84:0/3092333275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb34003970 msgr2=0x7fcb34003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:16.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.736+0000 7fcb4dc94640 1 --2- 172.21.15.84:0/3092333275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb34003970 0x7fcb34003d90 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fcb38001750 tx=0x7fcb340057f0 comp rx=0 tx=0).stop 2023-10-16T11:06:16.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.736+0000 7fcb4dc94640 1 -- 172.21.15.84:0/3092333275 shutdown_connections 2023-10-16T11:06:16.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.736+0000 7fcb4dc94640 1 --2- 172.21.15.84:0/3092333275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb34003970 0x7fcb34003d90 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:16.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.736+0000 7fcb4dc94640 1 -- 172.21.15.84:0/3092333275 >> 172.21.15.84:0/3092333275 conn(0x7fcb480ce9b0 msgr2=0x7fcb480cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:16.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.737+0000 7fcb4dc94640 1 -- 172.21.15.84:0/3092333275 shutdown_connections 2023-10-16T11:06:16.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.737+0000 7fcb4dc94640 1 -- 172.21.15.84:0/3092333275 wait complete. 2023-10-16T11:06:16.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.738+0000 7fcb4dc94640 1 Processor -- start 2023-10-16T11:06:16.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.738+0000 7fcb4dc94640 1 -- start start 2023-10-16T11:06:16.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.739+0000 7fcb4dc94640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb34003970 0x7fcb4814ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:16.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.739+0000 7fcb4dc94640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb480cc580 0x7fcb480c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:16.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.739+0000 7fcb4dc94640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb480c29e0 0x7fcb480ad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:16.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.739+0000 7fcb4dc94640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4814f610 con 0x7fcb480cc580 2023-10-16T11:06:16.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.739+0000 7fcb4dc94640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb480c2e60 con 0x7fcb480c29e0 2023-10-16T11:06:16.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.739+0000 7fcb4dc94640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4814e9b0 con 0x7fcb34003970 2023-10-16T11:06:16.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.740+0000 7fcb3ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb480cc580 0x7fcb480c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:16.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.740+0000 7fcb3ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb480cc580 0x7fcb480c24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43488/0 (socket says 172.21.15.84:43488) 2023-10-16T11:06:16.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.740+0000 7fcb3ffff640 1 -- 172.21.15.84:0/2795078894 learned_addr learned my addr 172.21.15.84:0/2795078894 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:16.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.740+0000 7fcb477fe640 1 --2- 172.21.15.84:0/2795078894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb34003970 0x7fcb4814ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:16.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.740+0000 7fcb3ffff640 1 -- 172.21.15.84:0/2795078894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb34003970 msgr2=0x7fcb4814ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:16.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.740+0000 7fcb3ffff640 1 --2- 172.21.15.84:0/2795078894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb34003970 0x7fcb4814ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:16.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.740+0000 7fcb3ffff640 1 -- 172.21.15.84:0/2795078894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb480c29e0 msgr2=0x7fcb480ad3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:16.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.740+0000 7fcb3ffff640 1 --2- 172.21.15.84:0/2795078894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb480c29e0 0x7fcb480ad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:16.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.740+0000 7fcb3ffff640 1 -- 172.21.15.84:0/2795078894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4814ec70 con 0x7fcb480cc580 2023-10-16T11:06:16.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.740+0000 7fcb3ffff640 1 --2- 172.21.15.84:0/2795078894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb480cc580 0x7fcb480c24a0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000d720 tx=0x7fcb3000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:16.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.740+0000 7fcb44ff9640 1 -- 172.21.15.84:0/2795078894 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb300072d0 con 0x7fcb480cc580 2023-10-16T11:06:16.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.741+0000 7fcb44ff9640 1 -- 172.21.15.84:0/2795078894 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb30007430 con 0x7fcb480cc580 2023-10-16T11:06:16.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.741+0000 7fcb44ff9640 1 -- 172.21.15.84:0/2795078894 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb300047e0 con 0x7fcb480cc580 2023-10-16T11:06:16.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.741+0000 7fcb4dc94640 1 -- 172.21.15.84:0/2795078894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4814ef50 con 0x7fcb480cc580 2023-10-16T11:06:16.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.741+0000 7fcb4dc94640 1 -- 172.21.15.84:0/2795078894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4814e310 con 0x7fcb480cc580 2023-10-16T11:06:16.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.742+0000 7fcb4dc94640 1 -- 172.21.15.84:0/2795078894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb4814f750 con 0x7fcb480cc580 2023-10-16T11:06:16.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.743+0000 7fcb44ff9640 1 -- 172.21.15.84:0/2795078894 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcb30004940 con 0x7fcb480cc580 2023-10-16T11:06:16.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.743+0000 7fcb44ff9640 1 --2- 172.21.15.84:0/2795078894 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fcb2806afc0 0x7fcb2806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:16.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.744+0000 7fcb44ff9640 1 -- 172.21.15.84:0/2795078894 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fcb300892d0 con 0x7fcb480cc580 2023-10-16T11:06:16.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.744+0000 7fcb477fe640 1 --2- 172.21.15.84:0/2795078894 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fcb2806afc0 0x7fcb2806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:16.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.744+0000 7fcb477fe640 1 --2- 172.21.15.84:0/2795078894 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fcb2806afc0 0x7fcb2806d480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcb38033000 tx=0x7fcb38009030 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:16.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.748+0000 7fcb44ff9640 1 -- 172.21.15.84:0/2795078894 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcb4814f750 con 0x7fcb480cc580 2023-10-16T11:06:16.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.927+0000 7fcb4dc94640 1 -- 172.21.15.84:0/2795078894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcb4814f750 con 0x7fcb480cc580 2023-10-16T11:06:16.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.933+0000 7fcb44ff9640 1 -- 172.21.15.84:0/2795078894 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v64)=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcb300541a0 con 0x7fcb480cc580 2023-10-16T11:06:16.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.936+0000 7fcb4dc94640 1 -- 172.21.15.84:0/2795078894 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fcb2806afc0 msgr2=0x7fcb2806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:16.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.936+0000 7fcb4dc94640 1 --2- 172.21.15.84:0/2795078894 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fcb2806afc0 0x7fcb2806d480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcb38033000 tx=0x7fcb38009030 comp rx=0 tx=0).stop 2023-10-16T11:06:16.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.936+0000 7fcb4dc94640 1 -- 172.21.15.84:0/2795078894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb480cc580 msgr2=0x7fcb480c24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:16.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.936+0000 7fcb4dc94640 1 --2- 172.21.15.84:0/2795078894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb480cc580 0x7fcb480c24a0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000d720 tx=0x7fcb3000dbe0 comp rx=0 tx=0).stop 2023-10-16T11:06:16.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.936+0000 7fcb4dc94640 1 -- 172.21.15.84:0/2795078894 shutdown_connections 2023-10-16T11:06:16.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.936+0000 7fcb4dc94640 1 --2- 172.21.15.84:0/2795078894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb480c29e0 0x7fcb480ad3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:16.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.936+0000 7fcb4dc94640 1 --2- 172.21.15.84:0/2795078894 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fcb2806afc0 0x7fcb2806d480 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:16.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.936+0000 7fcb4dc94640 1 --2- 172.21.15.84:0/2795078894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb480cc580 0x7fcb480c24a0 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:16.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.936+0000 7fcb4dc94640 1 --2- 172.21.15.84:0/2795078894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb34003970 0x7fcb4814ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:16.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.936+0000 7fcb4dc94640 1 -- 172.21.15.84:0/2795078894 >> 172.21.15.84:0/2795078894 conn(0x7fcb480ce9b0 msgr2=0x7fcb4815a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:16.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.936+0000 7fcb4dc94640 1 -- 172.21.15.84:0/2795078894 shutdown_connections 2023-10-16T11:06:16.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:16.936+0000 7fcb4dc94640 1 -- 172.21.15.84:0/2795078894 wait complete. 2023-10-16T11:06:16.956 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:06:16.957 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:06:17.058 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:06:17.188 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:06:17.189 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:06:17.189 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:06:17.248 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:06:17.248 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:06:17.249 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:06:17.308 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:06:17.309 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:06:17.309 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:06:17.370 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:06:17.370 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:06:17.371 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:06:17.432 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:06:17.432 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T11:06:17.433 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:06:17.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.655+0000 7f6842068640 1 Processor -- start 2023-10-16T11:06:17.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.655+0000 7f6842068640 1 -- start start 2023-10-16T11:06:17.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.656+0000 7f6842068640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f683c159b20 0x7f683c06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:17.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.656+0000 7f6842068640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f683c159f00 0x7f683c0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:17.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.656+0000 7f6842068640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f683c06c120 0x7f683c0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:17.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.656+0000 7f6842068640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683c06f7f0 con 0x7f683c159b20 2023-10-16T11:06:17.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.656+0000 7f6842068640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683c06c5e0 con 0x7f683c159f00 2023-10-16T11:06:17.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.656+0000 7f6842068640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683c06c8c0 con 0x7f683c06c120 2023-10-16T11:06:17.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.656+0000 7f683b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f683c159b20 0x7f683c06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:17.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.657+0000 7f683b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f683c159b20 0x7f683c06bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43494/0 (socket says 172.21.15.84:43494) 2023-10-16T11:06:17.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.657+0000 7f683b7fe640 1 -- 172.21.15.84:0/2312431255 learned_addr learned my addr 172.21.15.84:0/2312431255 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:17.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.657+0000 7f683b7fe640 1 -- 172.21.15.84:0/2312431255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f683c06c120 msgr2=0x7f683c0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:17.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.657+0000 7f683b7fe640 1 --2- 172.21.15.84:0/2312431255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f683c06c120 0x7f683c0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:17.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.657+0000 7f683b7fe640 1 -- 172.21.15.84:0/2312431255 >> v2:172.21.15.157:3300/0 conn(0x7f683c159f00 msgr2=0x7f683c0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:17.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.657+0000 7f683b7fe640 1 --2- 172.21.15.84:0/2312431255 >> v2:172.21.15.157:3300/0 conn(0x7f683c159f00 0x7f683c0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:17.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.657+0000 7f683b7fe640 1 -- 172.21.15.84:0/2312431255 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c06cba0 con 0x7f683c159b20 2023-10-16T11:06:17.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.658+0000 7f683b7fe640 1 --2- 172.21.15.84:0/2312431255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f683c159b20 0x7f683c06bbe0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f683c15d770 tx=0x7f682c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debbc2e47704b882 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:17.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.658+0000 7f683affd640 1 -- 172.21.15.84:0/2312431255 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c031870 con 0x7f683c159b20 2023-10-16T11:06:17.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.658+0000 7f683affd640 1 -- 172.21.15.84:0/2312431255 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682c0319d0 con 0x7f683c159b20 2023-10-16T11:06:17.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.658+0000 7f683affd640 1 -- 172.21.15.84:0/2312431255 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c031d60 con 0x7f683c159b20 2023-10-16T11:06:17.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.659+0000 7f6842068640 1 -- 172.21.15.84:0/2312431255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f683c159b20 msgr2=0x7f683c06bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:17.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.659+0000 7f6842068640 1 --2- 172.21.15.84:0/2312431255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f683c159b20 0x7f683c06bbe0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f683c15d770 tx=0x7f682c02b910 comp rx=0 tx=0).stop 2023-10-16T11:06:17.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.659+0000 7f6842068640 1 -- 172.21.15.84:0/2312431255 shutdown_connections 2023-10-16T11:06:17.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.659+0000 7f6842068640 1 --2- 172.21.15.84:0/2312431255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f683c06c120 0x7f683c0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:17.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.659+0000 7f6842068640 1 --2- 172.21.15.84:0/2312431255 >> v2:172.21.15.157:3300/0 conn(0x7f683c159f00 0x7f683c0b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:17.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.659+0000 7f6842068640 1 --2- 172.21.15.84:0/2312431255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f683c159b20 0x7f683c06bbe0 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:17.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.659+0000 7f6842068640 1 -- 172.21.15.84:0/2312431255 >> 172.21.15.84:0/2312431255 conn(0x7f683c0cb410 msgr2=0x7f683c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:17.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.659+0000 7f6842068640 1 -- 172.21.15.84:0/2312431255 shutdown_connections 2023-10-16T11:06:17.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.659+0000 7f6842068640 1 -- 172.21.15.84:0/2312431255 wait complete. 2023-10-16T11:06:17.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.661+0000 7f6842068640 1 Processor -- start 2023-10-16T11:06:17.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.661+0000 7f6842068640 1 -- start start 2023-10-16T11:06:17.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.662+0000 7f6842068640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f683c06c120 0x7f683c0c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:17.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.662+0000 7f6842068640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f683c159b20 0x7f683c0c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:17.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.662+0000 7f6842068640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f683c159f00 0x7f683c1558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:17.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.662+0000 7f6842068640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c14f660 con 0x7f683c159f00 2023-10-16T11:06:17.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.662+0000 7f6842068640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c0c5450 con 0x7f683c159b20 2023-10-16T11:06:17.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.662+0000 7f6842068640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683c0c8400 con 0x7f683c06c120 2023-10-16T11:06:17.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.662+0000 7f683bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f683c159f00 0x7f683c1558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:17.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.662+0000 7f683bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f683c159f00 0x7f683c1558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43496/0 (socket says 172.21.15.84:43496) 2023-10-16T11:06:17.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.662+0000 7f683bfff640 1 -- 172.21.15.84:0/2526474784 learned_addr learned my addr 172.21.15.84:0/2526474784 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:17.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.663+0000 7f683b7fe640 1 --2- 172.21.15.84:0/2526474784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f683c06c120 0x7f683c0c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:17.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.663+0000 7f6833fff640 1 --2- 172.21.15.84:0/2526474784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f683c159b20 0x7f683c0c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:17.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.663+0000 7f683bfff640 1 -- 172.21.15.84:0/2526474784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f683c06c120 msgr2=0x7f683c0c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:17.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.663+0000 7f683bfff640 1 --2- 172.21.15.84:0/2526474784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f683c06c120 0x7f683c0c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:17.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.663+0000 7f683bfff640 1 -- 172.21.15.84:0/2526474784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f683c159b20 msgr2=0x7f683c0c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:17.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.663+0000 7f683bfff640 1 --2- 172.21.15.84:0/2526474784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f683c159b20 0x7f683c0c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:17.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.663+0000 7f683bfff640 1 -- 172.21.15.84:0/2526474784 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c0094d0 con 0x7f683c159f00 2023-10-16T11:06:17.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.663+0000 7f6833fff640 1 --2- 172.21.15.84:0/2526474784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f683c159b20 0x7f683c0c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.663+0000 7f683bfff640 1 --2- 172.21.15.84:0/2526474784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f683c159f00 0x7f683c1558a0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f683400b9e0 tx=0x7f683400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.663+0000 7f6838ff9640 1 -- 172.21.15.84:0/2526474784 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683400cb20 con 0x7f683c159f00 2023-10-16T11:06:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.663+0000 7f6838ff9640 1 -- 172.21.15.84:0/2526474784 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683400cca0 con 0x7f683c159f00 2023-10-16T11:06:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.663+0000 7f6838ff9640 1 -- 172.21.15.84:0/2526474784 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6834014680 con 0x7f683c159f00 2023-10-16T11:06:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.663+0000 7f6842068640 1 -- 172.21.15.84:0/2526474784 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683c0c8980 con 0x7f683c159f00 2023-10-16T11:06:17.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.664+0000 7f6842068640 1 -- 172.21.15.84:0/2526474784 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683c0c4710 con 0x7f683c159f00 2023-10-16T11:06:17.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.665+0000 7f6842068640 1 -- 172.21.15.84:0/2526474784 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683c0c8150 con 0x7f683c159f00 2023-10-16T11:06:17.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.666+0000 7f6838ff9640 1 -- 172.21.15.84:0/2526474784 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f683401d3b0 con 0x7f683c159f00 2023-10-16T11:06:17.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.666+0000 7f6838ff9640 1 --2- 172.21.15.84:0/2526474784 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f681c06af90 0x7f681c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:17.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.667+0000 7f6838ff9640 1 -- 172.21.15.84:0/2526474784 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f6834089730 con 0x7f683c159f00 2023-10-16T11:06:17.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.667+0000 7f683b7fe640 1 --2- 172.21.15.84:0/2526474784 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f681c06af90 0x7f681c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:17.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.668+0000 7f683b7fe640 1 --2- 172.21.15.84:0/2526474784 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f681c06af90 0x7f681c06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f682c036000 tx=0x7f682c03c040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:17.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.671+0000 7f6838ff9640 1 -- 172.21.15.84:0/2526474784 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6834054660 con 0x7f683c159f00 2023-10-16T11:06:17.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.903+0000 7f6842068640 1 -- 172.21.15.84:0/2526474784 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f683c04e160 con 0x7f683c159f00 2023-10-16T11:06:17.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.904+0000 7f6838ff9640 1 -- 172.21.15.84:0/2526474784 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+3706 (secure 0 0 0) 0x7f6834019090 con 0x7f683c159f00 2023-10-16T11:06:17.906 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:17.906 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":90,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6718,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3060266169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3060266169},{"type":"v1","addr":"172.21.15.157:6834","nonce":3060266169}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":6721,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2965939103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2965939103},{"type":"v1","addr":"172.21.15.157:6835","nonce":2965939103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9684,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3677909298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3677909298},{"type":"v1","addr":"172.21.15.84:6835","nonce":3677909298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":10107,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/1039005457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":1039005457},{"type":"v1","addr":"172.21.15.84:6837","nonce":1039005457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[]} 2023-10-16T11:06:17.907 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 90 2023-10-16T11:06:17.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.907+0000 7f6842068640 1 -- 172.21.15.84:0/2526474784 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f681c06af90 msgr2=0x7f681c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:17.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.907+0000 7f6842068640 1 --2- 172.21.15.84:0/2526474784 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f681c06af90 0x7f681c06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f682c036000 tx=0x7f682c03c040 comp rx=0 tx=0).stop 2023-10-16T11:06:17.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.907+0000 7f6842068640 1 -- 172.21.15.84:0/2526474784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f683c159f00 msgr2=0x7f683c1558a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:17.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.907+0000 7f6842068640 1 --2- 172.21.15.84:0/2526474784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f683c159f00 0x7f683c1558a0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f683400b9e0 tx=0x7f683400bea0 comp rx=0 tx=0).stop 2023-10-16T11:06:17.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.907+0000 7f6842068640 1 -- 172.21.15.84:0/2526474784 shutdown_connections 2023-10-16T11:06:17.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.907+0000 7f6842068640 1 --2- 172.21.15.84:0/2526474784 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f681c06af90 0x7f681c06d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:17.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.907+0000 7f6842068640 1 --2- 172.21.15.84:0/2526474784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f683c159f00 0x7f683c1558a0 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:17.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.907+0000 7f6842068640 1 --2- 172.21.15.84:0/2526474784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f683c159b20 0x7f683c0c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:17.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.907+0000 7f6842068640 1 --2- 172.21.15.84:0/2526474784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f683c06c120 0x7f683c0c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:17.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.907+0000 7f6842068640 1 -- 172.21.15.84:0/2526474784 >> 172.21.15.84:0/2526474784 conn(0x7f683c0cb410 msgr2=0x7f683c0cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:17.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.907+0000 7f6842068640 1 -- 172.21.15.84:0/2526474784 shutdown_connections 2023-10-16T11:06:17.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:17.907+0000 7f6842068640 1 -- 172.21.15.84:0/2526474784 wait complete. 2023-10-16T11:06:17.926 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T11:06:17.926 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T11:06:17.927 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T11:06:17.927 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:06:17.927 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T11:06:17.928 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T11:06:17.928 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T11:06:17.928 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:06:17.928 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T11:06:17.929 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T11:06:17.929 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T11:06:17.929 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:06:17.930 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T11:06:17.930 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T11:06:17.930 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T11:06:17.930 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:06:17.945 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T11:06:17.946 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T11:06:17.947 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T11:06:17.948 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T11:06:17.955 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T11:06:17.956 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T11:06:17.957 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T11:06:17.957 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T11:06:17.995 INFO:tasks.ceph.mds.a:Started 2023-10-16T11:06:17.996 DEBUG:teuthology.parallel:result is None 2023-10-16T11:06:18.003 INFO:tasks.ceph.mds.c:Started 2023-10-16T11:06:18.003 DEBUG:teuthology.parallel:result is None 2023-10-16T11:06:18.005 INFO:tasks.ceph.mds.d:Started 2023-10-16T11:06:18.005 DEBUG:teuthology.parallel:result is None 2023-10-16T11:06:18.011 INFO:tasks.ceph.mds.b:Started 2023-10-16T11:06:18.012 DEBUG:teuthology.parallel:result is None 2023-10-16T11:06:18.012 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T11:06:18.152 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T11:06:18.159 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T11:06:18.167 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T11:06:18.168 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-10-16T11:06:18.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.200+0000 7ff0befe7640 1 Processor -- start 2023-10-16T11:06:18.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.200+0000 7ff0befe7640 1 -- start start 2023-10-16T11:06:18.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.201+0000 7ff0befe7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff0b815ad70 0x7ff0b8069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:18.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.201+0000 7ff0befe7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff0b80c9ca0 0x7ff0b806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:18.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.201+0000 7ff0befe7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b80cb030 con 0x7ff0b80ca3b0 2023-10-16T11:06:18.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.201+0000 7ff0befe7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b80c91c0 con 0x7ff0b80c9ca0 2023-10-16T11:06:18.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.201+0000 7ff0befe7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b80c94a0 con 0x7ff0b815ad70 2023-10-16T11:06:18.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.201+0000 7ff0b7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff0b80c9ca0 0x7ff0b806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:18.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.201+0000 7ff0b7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff0b80c9ca0 0x7ff0b806a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43522/0 (socket says 172.21.15.84:43522) 2023-10-16T11:06:18.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.201+0000 7ff0b7fff640 1 -- 172.21.15.84:0/1058853476 learned_addr learned my addr 172.21.15.84:0/1058853476 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:18.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.201+0000 7ff0b7fff640 1 -- 172.21.15.84:0/1058853476 >> v2:172.21.15.157:3300/0 conn(0x7ff0b815ad70 msgr2=0x7ff0b8069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:18.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.201+0000 7ff0bcd83640 1 --2- 172.21.15.84:0/1058853476 >> v2:172.21.15.157:3300/0 conn(0x7ff0b815ad70 0x7ff0b8069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:18.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.202+0000 7ff0b7fff640 1 --2- 172.21.15.84:0/1058853476 >> v2:172.21.15.157:3300/0 conn(0x7ff0b815ad70 0x7ff0b8069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:18.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.202+0000 7ff0b7fff640 1 -- 172.21.15.84:0/1058853476 >> v1:172.21.15.157:6789/0 conn(0x7ff0b80ca3b0 legacy=0x7ff0b80ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:18.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.202+0000 7ff0b7fff640 1 -- 172.21.15.84:0/1058853476 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b80c9780 con 0x7ff0b80c9ca0 2023-10-16T11:06:18.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.202+0000 7ff0bcd83640 1 --2- 172.21.15.84:0/1058853476 >> v2:172.21.15.157:3300/0 conn(0x7ff0b815ad70 0x7ff0b8069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:18.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.203+0000 7ff0b7fff640 1 --2- 172.21.15.84:0/1058853476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0b80c9ca0 0x7ff0b806a2e0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7ff0b80cf780 tx=0x7ff0a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e73c3deaab8960 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:18.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.203+0000 7ff0b77fe640 1 -- 172.21.15.84:0/1058853476 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4031870 con 0x7ff0b80c9ca0 2023-10-16T11:06:18.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.203+0000 7ff0b77fe640 1 -- 172.21.15.84:0/1058853476 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a40319d0 con 0x7ff0b80c9ca0 2023-10-16T11:06:18.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.203+0000 7ff0b77fe640 1 -- 172.21.15.84:0/1058853476 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4031d60 con 0x7ff0b80c9ca0 2023-10-16T11:06:18.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.204+0000 7ff0befe7640 1 -- 172.21.15.84:0/1058853476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0b80c9ca0 msgr2=0x7ff0b806a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:18.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.204+0000 7ff0befe7640 1 --2- 172.21.15.84:0/1058853476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0b80c9ca0 0x7ff0b806a2e0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7ff0b80cf780 tx=0x7ff0a402b910 comp rx=0 tx=0).stop 2023-10-16T11:06:18.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.204+0000 7ff0befe7640 1 -- 172.21.15.84:0/1058853476 shutdown_connections 2023-10-16T11:06:18.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.204+0000 7ff0befe7640 1 --2- 172.21.15.84:0/1058853476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0b80c9ca0 0x7ff0b806a2e0 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:18.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.204+0000 7ff0befe7640 1 --2- 172.21.15.84:0/1058853476 >> v2:172.21.15.157:3300/0 conn(0x7ff0b815ad70 0x7ff0b8069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:18.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.204+0000 7ff0befe7640 1 -- 172.21.15.84:0/1058853476 >> 172.21.15.84:0/1058853476 conn(0x7ff0b80ce9b0 msgr2=0x7ff0b80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:18.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.204+0000 7ff0befe7640 1 -- 172.21.15.84:0/1058853476 shutdown_connections 2023-10-16T11:06:18.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.204+0000 7ff0befe7640 1 -- 172.21.15.84:0/1058853476 wait complete. 2023-10-16T11:06:18.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.205+0000 7ff0befe7640 1 Processor -- start 2023-10-16T11:06:18.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.206+0000 7ff0befe7640 1 -- start start 2023-10-16T11:06:18.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.206+0000 7ff0befe7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0b80c9ca0 0x7ff0b8150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:18.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.206+0000 7ff0befe7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b80ca3b0 0x7ff0b80ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:18.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.206+0000 7ff0befe7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0b815ad70 0x7ff0b80aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:18.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.206+0000 7ff0befe7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b80c6420 con 0x7ff0b815ad70 2023-10-16T11:06:18.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.206+0000 7ff0befe7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b815da80 con 0x7ff0b80ca3b0 2023-10-16T11:06:18.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.206+0000 7ff0befe7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b80c5a80 con 0x7ff0b80c9ca0 2023-10-16T11:06:18.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.207+0000 7ff0bd584640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0b815ad70 0x7ff0b80aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:18.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.207+0000 7ff0bd584640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0b815ad70 0x7ff0b80aeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43534/0 (socket says 172.21.15.84:43534) 2023-10-16T11:06:18.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.207+0000 7ff0bd584640 1 -- 172.21.15.84:0/940356220 learned_addr learned my addr 172.21.15.84:0/940356220 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:18.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.207+0000 7ff0bcd83640 1 --2- 172.21.15.84:0/940356220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0b80c9ca0 0x7ff0b8150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:18.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.207+0000 7ff0b7fff640 1 --2- 172.21.15.84:0/940356220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b80ca3b0 0x7ff0b80ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:18.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.207+0000 7ff0bd584640 1 -- 172.21.15.84:0/940356220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0b80c9ca0 msgr2=0x7ff0b8150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:18.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.207+0000 7ff0bd584640 1 --2- 172.21.15.84:0/940356220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0b80c9ca0 0x7ff0b8150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:18.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.207+0000 7ff0bd584640 1 -- 172.21.15.84:0/940356220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b80ca3b0 msgr2=0x7ff0b80ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:18.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.207+0000 7ff0bd584640 1 --2- 172.21.15.84:0/940356220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b80ca3b0 0x7ff0b80ae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:18.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.207+0000 7ff0bd584640 1 -- 172.21.15.84:0/940356220 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a40094d0 con 0x7ff0b815ad70 2023-10-16T11:06:18.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.207+0000 7ff0bd584640 1 --2- 172.21.15.84:0/940356220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0b815ad70 0x7ff0b80aeeb0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000c780 tx=0x7ff0b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:18.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.208+0000 7ff0b57fa640 1 -- 172.21.15.84:0/940356220 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0004b60 con 0x7ff0b815ad70 2023-10-16T11:06:18.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.208+0000 7ff0b57fa640 1 -- 172.21.15.84:0/940356220 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b000ce90 con 0x7ff0b815ad70 2023-10-16T11:06:18.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.208+0000 7ff0befe7640 1 -- 172.21.15.84:0/940356220 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b80c5d60 con 0x7ff0b815ad70 2023-10-16T11:06:18.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.208+0000 7ff0b57fa640 1 -- 172.21.15.84:0/940356220 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b000a850 con 0x7ff0b815ad70 2023-10-16T11:06:18.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.208+0000 7ff0befe7640 1 -- 172.21.15.84:0/940356220 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b8151300 con 0x7ff0b815ad70 2023-10-16T11:06:18.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.209+0000 7ff0befe7640 1 -- 172.21.15.84:0/940356220 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b80c6560 con 0x7ff0b815ad70 2023-10-16T11:06:18.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.210+0000 7ff0b57fa640 1 -- 172.21.15.84:0/940356220 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff0b000a9b0 con 0x7ff0b815ad70 2023-10-16T11:06:18.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.211+0000 7ff0b57fa640 1 --2- 172.21.15.84:0/940356220 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7ff09c06af90 0x7ff09c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:18.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.211+0000 7ff0b57fa640 1 -- 172.21.15.84:0/940356220 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5659+0+0 (secure 0 0 0) 0x7ff0b0089830 con 0x7ff0b815ad70 2023-10-16T11:06:18.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.211+0000 7ff0bcd83640 1 --2- 172.21.15.84:0/940356220 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7ff09c06af90 0x7ff09c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:18.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.212+0000 7ff0bcd83640 1 --2- 172.21.15.84:0/940356220 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7ff09c06af90 0x7ff09c06d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff0b80b31f0 tx=0x7ff0ac008040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:18.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.215+0000 7ff0b57fa640 1 -- 172.21.15.84:0/940356220 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff0b0054810 con 0x7ff0b815ad70 2023-10-16T11:06:18.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.419+0000 7ff0befe7640 1 -- 172.21.15.84:0/940356220 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff0b804e160 con 0x7ff0b815ad70 2023-10-16T11:06:18.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.516+0000 7ff0b57fa640 1 -- 172.21.15.84:0/940356220 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v178) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff0b0057e30 con 0x7ff0b815ad70 2023-10-16T11:06:18.518 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-10-16T11:06:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.519+0000 7ff0befe7640 1 -- 172.21.15.84:0/940356220 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7ff09c06af90 msgr2=0x7ff09c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.519+0000 7ff0befe7640 1 --2- 172.21.15.84:0/940356220 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7ff09c06af90 0x7ff09c06d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff0b80b31f0 tx=0x7ff0ac008040 comp rx=0 tx=0).stop 2023-10-16T11:06:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.519+0000 7ff0befe7640 1 -- 172.21.15.84:0/940356220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0b815ad70 msgr2=0x7ff0b80aeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.519+0000 7ff0befe7640 1 --2- 172.21.15.84:0/940356220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0b815ad70 0x7ff0b80aeeb0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000c780 tx=0x7ff0b000cc40 comp rx=0 tx=0).stop 2023-10-16T11:06:18.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.520+0000 7ff0befe7640 1 -- 172.21.15.84:0/940356220 shutdown_connections 2023-10-16T11:06:18.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.520+0000 7ff0befe7640 1 --2- 172.21.15.84:0/940356220 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7ff09c06af90 0x7ff09c06d450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:18.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.520+0000 7ff0befe7640 1 --2- 172.21.15.84:0/940356220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0b815ad70 0x7ff0b80aeeb0 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:18.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.520+0000 7ff0befe7640 1 --2- 172.21.15.84:0/940356220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b80ca3b0 0x7ff0b80ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:18.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.520+0000 7ff0befe7640 1 --2- 172.21.15.84:0/940356220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0b80c9ca0 0x7ff0b8150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:18.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.520+0000 7ff0befe7640 1 -- 172.21.15.84:0/940356220 >> 172.21.15.84:0/940356220 conn(0x7ff0b80ce9b0 msgr2=0x7ff0b8159f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:18.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.520+0000 7ff0befe7640 1 -- 172.21.15.84:0/940356220 shutdown_connections 2023-10-16T11:06:18.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.520+0000 7ff0befe7640 1 -- 172.21.15.84:0/940356220 wait complete. 2023-10-16T11:06:18.542 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T11:06:18.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.737+0000 7fc6c2d6c640 1 Processor -- start 2023-10-16T11:06:18.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.738+0000 7fc6c2d6c640 1 -- start start 2023-10-16T11:06:18.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.739+0000 7fc6c2d6c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6bc15ae80 0x7fc6bc15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:18.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.739+0000 7fc6c2d6c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6bc15ed60 con 0x7fc6bc0ce390 2023-10-16T11:06:18.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.739+0000 7fc6c2d6c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6bc0707e0 con 0x7fc6bc15aaa0 2023-10-16T11:06:18.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.739+0000 7fc6c2d6c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc06b810 con 0x7fc6bc15ae80 2023-10-16T11:06:18.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.739+0000 7fc6c1309640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc6bc0ce390 0x7fc6bc070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49744/0 (socket says 172.21.15.84:49744) 2023-10-16T11:06:18.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.739+0000 7fc6c1309640 1 -- 172.21.15.84:0/2336837759 learned_addr learned my addr 172.21.15.84:0/2336837759 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:18.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.740+0000 7fc6bbfff640 1 --2- 172.21.15.84:0/2336837759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6bc15ae80 0x7fc6bc15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:18.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.740+0000 7fc6bb7fe640 1 -- 172.21.15.84:0/2336837759 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4006255433 0 0) 0x7fc6bc15ed60 con 0x7fc6bc0ce390 2023-10-16T11:06:18.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.740+0000 7fc6bb7fe640 1 -- 172.21.15.84:0/2336837759 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6ac003520 con 0x7fc6bc0ce390 2023-10-16T11:06:18.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.740+0000 7fc6bb7fe640 1 -- 172.21.15.84:0/2336837759 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3960819146 0 0) 0x7fc6ac003520 con 0x7fc6bc0ce390 2023-10-16T11:06:18.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.740+0000 7fc6bb7fe640 1 -- 172.21.15.84:0/2336837759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6bc15ae80 msgr2=0x7fc6bc15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:18.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.740+0000 7fc6bb7fe640 1 --2- 172.21.15.84:0/2336837759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6bc15ae80 0x7fc6bc15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:18.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.740+0000 7fc6bb7fe640 1 -- 172.21.15.84:0/2336837759 >> v1:172.21.15.157:6789/0 conn(0x7fc6bc15aaa0 legacy=0x7fc6bc0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:18.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.740+0000 7fc6bb7fe640 1 -- 172.21.15.84:0/2336837759 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc15ef90 con 0x7fc6bc0ce390 2023-10-16T11:06:18.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.740+0000 7fc6bb7fe640 1 -- 172.21.15.84:0/2336837759 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc6b40019f0 con 0x7fc6bc0ce390 2023-10-16T11:06:18.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.740+0000 7fc6bb7fe640 1 -- 172.21.15.84:0/2336837759 >> v1:172.21.15.84:6789/0 conn(0x7fc6bc0ce390 legacy=0x7fc6bc070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:18.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.741+0000 7fc6bb7fe640 1 --2- 172.21.15.84:0/2336837759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6ac0039a0 0x7fc6ac003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:18.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.741+0000 7fc6bb7fe640 1 --2- 172.21.15.84:0/2336837759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6bc15ae80 0x7fc6ac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:18.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.741+0000 7fc6bb7fe640 1 --2- 172.21.15.84:0/2336837759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ac004d00 0x7fc6ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:18.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.741+0000 7fc6bb7fe640 1 -- 172.21.15.84:0/2336837759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc06b810 con 0x7fc6ac0039a0 2023-10-16T11:06:18.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.741+0000 7fc6bb7fe640 1 -- 172.21.15.84:0/2336837759 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac00ba60 con 0x7fc6ac004d00 2023-10-16T11:06:18.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.741+0000 7fc6bb7fe640 1 -- 172.21.15.84:0/2336837759 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac00bc20 con 0x7fc6bc15ae80 2023-10-16T11:06:18.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.741+0000 7fc6c0b08640 1 --2- 172.21.15.84:0/2336837759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6ac0039a0 0x7fc6ac003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:18.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.741+0000 7fc6c0b08640 1 -- 172.21.15.84:0/2336837759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6bc15ae80 msgr2=0x7fc6ac0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:18.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.741+0000 7fc6c1309640 1 --2- 172.21.15.84:0/2336837759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ac004d00 0x7fc6ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:18.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.741+0000 7fc6c0b08640 1 --2- 172.21.15.84:0/2336837759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6bc15ae80 0x7fc6ac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:18.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.741+0000 7fc6c0b08640 1 -- 172.21.15.84:0/2336837759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ac004d00 msgr2=0x7fc6ac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:18.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.741+0000 7fc6c0b08640 1 --2- 172.21.15.84:0/2336837759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ac004d00 0x7fc6ac005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:18.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.741+0000 7fc6c0b08640 1 -- 172.21.15.84:0/2336837759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac003520 con 0x7fc6ac0039a0 2023-10-16T11:06:18.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.742+0000 7fc6c0b08640 1 --2- 172.21.15.84:0/2336837759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6ac0039a0 0x7fc6ac003dc0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0001720 tx=0x7fc6ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a3054238d1f986 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:18.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.743+0000 7fc6bb7fe640 1 -- 172.21.15.84:0/2336837759 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b00052d0 con 0x7fc6ac0039a0 2023-10-16T11:06:18.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.743+0000 7fc6bb7fe640 1 -- 172.21.15.84:0/2336837759 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b002e050 con 0x7fc6ac0039a0 2023-10-16T11:06:18.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.743+0000 7fc6bb7fe640 1 -- 172.21.15.84:0/2336837759 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b00035f0 con 0x7fc6ac0039a0 2023-10-16T11:06:18.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.743+0000 7fc6c2d6c640 1 -- 172.21.15.84:0/2336837759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6ac0039a0 msgr2=0x7fc6ac003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:18.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.743+0000 7fc6c2d6c640 1 --2- 172.21.15.84:0/2336837759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6ac0039a0 0x7fc6ac003dc0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0001720 tx=0x7fc6ac0057f0 comp rx=0 tx=0).stop 2023-10-16T11:06:18.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.743+0000 7fc6c1309640 1 -- 172.21.15.84:0/2336837759 reap_dead start 2023-10-16T11:06:18.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.743+0000 7fc6c2d6c640 1 -- 172.21.15.84:0/2336837759 shutdown_connections 2023-10-16T11:06:18.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.743+0000 7fc6c2d6c640 1 -- 172.21.15.84:0/2336837759 >> 172.21.15.84:0/2336837759 conn(0x7fc6bc0cb410 msgr2=0x7fc6bc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:18.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.743+0000 7fc6c2d6c640 1 -- 172.21.15.84:0/2336837759 shutdown_connections 2023-10-16T11:06:18.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.743+0000 7fc6c2d6c640 1 -- 172.21.15.84:0/2336837759 wait complete. 2023-10-16T11:06:18.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.745+0000 7fc6c2d6c640 1 Processor -- start 2023-10-16T11:06:18.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.745+0000 7fc6c2d6c640 1 -- start start 2023-10-16T11:06:18.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.746+0000 7fc6c2d6c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ac0039a0 0x7fc6bc1edbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:18.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.746+0000 7fc6c2d6c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6bc15e160 0x7fc6bc0aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:18.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.746+0000 7fc6c2d6c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6bc1f19e0 0x7fc6bc1f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:18.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.746+0000 7fc6c2d6c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc1f23b0 con 0x7fc6bc1f19e0 2023-10-16T11:06:18.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.746+0000 7fc6c2d6c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc0c37d0 con 0x7fc6ac0039a0 2023-10-16T11:06:18.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.746+0000 7fc6c2d6c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc0af7f0 con 0x7fc6bc15e160 2023-10-16T11:06:18.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.746+0000 7fc6c1309640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6bc1f19e0 0x7fc6bc1f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:18.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.746+0000 7fc6c1309640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6bc1f19e0 0x7fc6bc1f1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43540/0 (socket says 172.21.15.84:43540) 2023-10-16T11:06:18.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.746+0000 7fc6c1309640 1 -- 172.21.15.84:0/606524270 learned_addr learned my addr 172.21.15.84:0/606524270 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:18.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.747+0000 7fc6c0b08640 1 --2- 172.21.15.84:0/606524270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ac0039a0 0x7fc6bc1edbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:18.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.747+0000 7fc6c1309640 1 -- 172.21.15.84:0/606524270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6bc15e160 msgr2=0x7fc6bc0aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:18.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.747+0000 7fc6c1309640 1 --2- 172.21.15.84:0/606524270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6bc15e160 0x7fc6bc0aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:18.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.747+0000 7fc6c1309640 1 -- 172.21.15.84:0/606524270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ac0039a0 msgr2=0x7fc6bc1edbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:18.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.747+0000 7fc6c1309640 1 --2- 172.21.15.84:0/606524270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ac0039a0 0x7fc6bc1edbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:18.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.747+0000 7fc6c1309640 1 -- 172.21.15.84:0/606524270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc0afad0 con 0x7fc6bc1f19e0 2023-10-16T11:06:18.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.747+0000 7fc6c1309640 1 --2- 172.21.15.84:0/606524270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6bc1f19e0 0x7fc6bc1f1e70 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0ce2e0 tx=0x7fc6b400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:18.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.747+0000 7fc6b97fa640 1 -- 172.21.15.84:0/606524270 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400bb80 con 0x7fc6bc1f19e0 2023-10-16T11:06:18.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.747+0000 7fc6b97fa640 1 -- 172.21.15.84:0/606524270 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b400bce0 con 0x7fc6bc1f19e0 2023-10-16T11:06:18.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.747+0000 7fc6b97fa640 1 -- 172.21.15.84:0/606524270 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4007290 con 0x7fc6bc1f19e0 2023-10-16T11:06:18.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.747+0000 7fc6c2d6c640 1 -- 172.21.15.84:0/606524270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6bc0afdb0 con 0x7fc6bc1f19e0 2023-10-16T11:06:18.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.747+0000 7fc6c2d6c640 1 -- 172.21.15.84:0/606524270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6bc1ee120 con 0x7fc6bc1f19e0 2023-10-16T11:06:18.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.749+0000 7fc6c2d6c640 1 -- 172.21.15.84:0/606524270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6bc0c3960 con 0x7fc6bc1f19e0 2023-10-16T11:06:18.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.750+0000 7fc6b97fa640 1 -- 172.21.15.84:0/606524270 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc6b40073f0 con 0x7fc6bc1f19e0 2023-10-16T11:06:18.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.751+0000 7fc6b97fa640 1 --2- 172.21.15.84:0/606524270 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc6a006af50 0x7fc6a006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:18.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.751+0000 7fc6b97fa640 1 -- 172.21.15.84:0/606524270 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fc6b4089780 con 0x7fc6bc1f19e0 2023-10-16T11:06:18.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.751+0000 7fc6c0b08640 1 --2- 172.21.15.84:0/606524270 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc6a006af50 0x7fc6a006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:18.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.752+0000 7fc6c0b08640 1 --2- 172.21.15.84:0/606524270 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc6a006af50 0x7fc6a006d410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0033000 tx=0x7fc6b0009030 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:18.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:18.755+0000 7fc6b97fa640 1 -- 172.21.15.84:0/606524270 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc6b40548a0 con 0x7fc6bc1f19e0 2023-10-16T11:06:19.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.014+0000 7fc6c2d6c640 1 -- 172.21.15.84:0/606524270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc6bc04e160 con 0x7fc6bc1f19e0 2023-10-16T11:06:19.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.015+0000 7fc6b97fa640 1 -- 172.21.15.84:0/606524270 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v47) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc6b4057ec0 con 0x7fc6bc1f19e0 2023-10-16T11:06:19.017 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:19.017 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:06:19.017 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-10-16T11:06:19.018 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.018 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-10-16T11:06:19.018 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l6dhjKRAA0pMvY6AK6qtFXDprNhwsqg==", 2023-10-16T11:06:19.018 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.019 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:06:19.019 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:06:19.019 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:06:19.019 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:06:19.020 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.020 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:06:19.020 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.020 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-10-16T11:06:19.020 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lFpMxEBAA0TLPRt5q3FwwBvMG019cjQ==", 2023-10-16T11:06:19.021 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.021 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:06:19.021 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:06:19.021 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:06:19.021 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:06:19.022 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.022 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:06:19.022 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.022 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-10-16T11:06:19.022 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l73RBOhAAOMEPxfUAMHhn5Blyvd5oMw==", 2023-10-16T11:06:19.023 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.023 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:06:19.023 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:06:19.023 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:06:19.023 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:06:19.024 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.024 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:06:19.024 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.024 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-10-16T11:06:19.024 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lcTIRIhAA0+FWm4b4yw77LD3+WKbk7w==", 2023-10-16T11:06:19.025 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.025 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:06:19.025 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:06:19.025 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:06:19.025 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:06:19.026 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.026 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:06:19.026 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.026 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-10-16T11:06:19.026 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDYBi1l1zBLERAA3YLW3xIYQoRVw7KSkwN3Og==", 2023-10-16T11:06:19.027 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.027 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:06:19.027 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:06:19.027 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:06:19.027 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.028 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:06:19.028 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.028 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-10-16T11:06:19.028 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDbBi1lFUUxKRAAQbus8SmYprJRmWTjoCznNA==", 2023-10-16T11:06:19.028 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.029 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:06:19.029 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:06:19.029 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:06:19.029 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.030 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:06:19.030 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.030 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-10-16T11:06:19.030 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDeBi1l+9lBMxAArl6sb/h15uBG/a9lVH40Hg==", 2023-10-16T11:06:19.030 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.031 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:06:19.031 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:06:19.031 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:06:19.031 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.031 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:06:19.032 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.032 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-10-16T11:06:19.032 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDiBi1l2rnEERAA3oRD71xa0fEOynM8zKuBEg==", 2023-10-16T11:06:19.032 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.032 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:06:19.032 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:06:19.033 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:06:19.033 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.033 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:06:19.033 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.033 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-10-16T11:06:19.034 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDnBi1lR7oLChAAZto8fDovDbpOd3Fcnc7C6w==", 2023-10-16T11:06:19.034 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.034 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:06:19.034 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:06:19.034 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:06:19.035 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.035 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:06:19.035 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.035 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-10-16T11:06:19.035 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDqBi1lmjg/JBAAlINKRGBc0/UWJJHMVFw57w==", 2023-10-16T11:06:19.036 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.036 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:06:19.036 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:06:19.036 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:06:19.036 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.037 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:06:19.037 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.037 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-10-16T11:06:19.037 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDuBi1lu0kMARAAtuSeM8F3yr8CVrc3p1biaQ==", 2023-10-16T11:06:19.038 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.038 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:06:19.038 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:06:19.038 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:06:19.038 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.039 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:06:19.039 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.039 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-10-16T11:06:19.039 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDxBi1lf6mNGBAAHGbqzJ0fmDxQIsasboHMfg==", 2023-10-16T11:06:19.039 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.040 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:06:19.040 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:06:19.040 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:06:19.040 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.040 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:06:19.040 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.041 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-10-16T11:06:19.041 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ==", 2023-10-16T11:06:19.041 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.041 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:06:19.042 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow r", 2023-10-16T11:06:19.042 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T11:06:19.042 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.042 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:06:19.043 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.043 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-10-16T11:06:19.043 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1lHU+4BxAAd8ET8hQ8Y+EUwdTJr9/+Ig==", 2023-10-16T11:06:19.043 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.043 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:06:19.043 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:06:19.044 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:06:19.044 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:06:19.044 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.044 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:06:19.044 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.045 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-10-16T11:06:19.045 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lA1BOORAAFNyfqgEF34MVR9YX3KBKkQ==", 2023-10-16T11:06:19.045 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.045 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:06:19.045 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:06:19.046 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:06:19.046 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:06:19.046 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.046 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:06:19.046 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.047 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-10-16T11:06:19.047 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1l848oERAANMfrZz9JrlgNVh7ZFbtnaA==", 2023-10-16T11:06:19.047 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.047 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:06:19.047 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:06:19.048 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:06:19.048 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:06:19.048 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.048 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:06:19.048 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.049 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-10-16T11:06:19.049 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDRBi1lTAxTGRAAMptxnxy3Qit/cTCXie6CrQ==", 2023-10-16T11:06:19.049 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.049 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:06:19.049 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:06:19.050 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:06:19.050 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:06:19.050 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.050 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:06:19.050 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.051 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-10-16T11:06:19.051 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lPuJxHRAABAnGPVCrkq0JR4Y6iFGZlw==", 2023-10-16T11:06:19.051 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.051 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:06:19.051 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T11:06:19.052 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:06:19.052 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.052 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:06:19.052 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:19.052 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-10-16T11:06:19.053 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lLxnKEBAA+a/pOHUp7Sg3m4weAust5w==", 2023-10-16T11:06:19.053 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:06:19.053 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:06:19.054 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T11:06:19.054 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:06:19.054 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.054 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:19.054 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-10-16T11:06:19.055 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:06:19.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.018+0000 7fc6c2d6c640 1 -- 172.21.15.84:0/606524270 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc6a006af50 msgr2=0x7fc6a006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:19.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.018+0000 7fc6c2d6c640 1 --2- 172.21.15.84:0/606524270 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc6a006af50 0x7fc6a006d410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0033000 tx=0x7fc6b0009030 comp rx=0 tx=0).stop 2023-10-16T11:06:19.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.018+0000 7fc6c2d6c640 1 -- 172.21.15.84:0/606524270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6bc1f19e0 msgr2=0x7fc6bc1f1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:19.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.018+0000 7fc6c2d6c640 1 --2- 172.21.15.84:0/606524270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6bc1f19e0 0x7fc6bc1f1e70 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0ce2e0 tx=0x7fc6b400b4a0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.018+0000 7fc6c2d6c640 1 -- 172.21.15.84:0/606524270 shutdown_connections 2023-10-16T11:06:19.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.018+0000 7fc6c2d6c640 1 --2- 172.21.15.84:0/606524270 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc6a006af50 0x7fc6a006d410 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.018+0000 7fc6c2d6c640 1 --2- 172.21.15.84:0/606524270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6bc1f19e0 0x7fc6bc1f1e70 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.018+0000 7fc6c2d6c640 1 --2- 172.21.15.84:0/606524270 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6bc15e160 0x7fc6bc0aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.018+0000 7fc6c2d6c640 1 --2- 172.21.15.84:0/606524270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6ac0039a0 0x7fc6bc1edbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.018+0000 7fc6c2d6c640 1 -- 172.21.15.84:0/606524270 >> 172.21.15.84:0/606524270 conn(0x7fc6bc0cb410 msgr2=0x7fc6bc06cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:19.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.018+0000 7fc6c2d6c640 1 -- 172.21.15.84:0/606524270 shutdown_connections 2023-10-16T11:06:19.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.018+0000 7fc6c2d6c640 1 -- 172.21.15.84:0/606524270 wait complete. 2023-10-16T11:06:19.060 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:06:19.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.278+0000 7f040902b640 1 Processor -- start 2023-10-16T11:06:19.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.278+0000 7f040902b640 1 -- start start 2023-10-16T11:06:19.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.279+0000 7f040902b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0404159b20 0x7f040406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:19.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.279+0000 7f040902b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04040b0b20 0x7f040406fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:19.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.279+0000 7f040902b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f040406f7f0 con 0x7f0404159f00 2023-10-16T11:06:19.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.279+0000 7f040902b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f040406c5e0 con 0x7f0404159b20 2023-10-16T11:06:19.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.279+0000 7f040902b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040406c8c0 con 0x7f04040b0b20 2023-10-16T11:06:19.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.280+0000 7f0402d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0404159b20 0x7f040406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:19.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.280+0000 7f040259c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0404159f00 0x7f04040b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49752/0 (socket says 172.21.15.84:49752) 2023-10-16T11:06:19.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.280+0000 7f040259c640 1 -- 172.21.15.84:0/2810863793 learned_addr learned my addr 172.21.15.84:0/2810863793 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:19.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.280+0000 7f0402d9d640 1 -- 172.21.15.84:0/2810863793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04040b0b20 msgr2=0x7f040406fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:19.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.280+0000 7f0402d9d640 1 --2- 172.21.15.84:0/2810863793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04040b0b20 0x7f040406fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.280+0000 7f0402d9d640 1 -- 172.21.15.84:0/2810863793 >> v1:172.21.15.84:6789/0 conn(0x7f0404159f00 legacy=0x7f04040b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:19.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.280+0000 7f0402d9d640 1 -- 172.21.15.84:0/2810863793 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040406cba0 con 0x7f0404159b20 2023-10-16T11:06:19.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.280+0000 7f0401d9b640 1 -- 172.21.15.84:0/2810863793 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154113324 0 0) 0x7f040406f7f0 con 0x7f0404159f00 2023-10-16T11:06:19.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.281+0000 7f0402d9d640 1 --2- 172.21.15.84:0/2810863793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0404159b20 0x7f040406bbe0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f03f40097a0 tx=0x7f03f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38167873b655b0d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:19.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.281+0000 7f0401d9b640 1 -- 172.21.15.84:0/2810863793 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4031830 con 0x7f0404159b20 2023-10-16T11:06:19.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.281+0000 7f0401d9b640 1 -- 172.21.15.84:0/2810863793 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f4031990 con 0x7f0404159b20 2023-10-16T11:06:19.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.281+0000 7f0401d9b640 1 -- 172.21.15.84:0/2810863793 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4031d20 con 0x7f0404159b20 2023-10-16T11:06:19.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.282+0000 7f040902b640 1 -- 172.21.15.84:0/2810863793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0404159b20 msgr2=0x7f040406bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:19.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.282+0000 7f040902b640 1 --2- 172.21.15.84:0/2810863793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0404159b20 0x7f040406bbe0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f03f40097a0 tx=0x7f03f402b8d0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.282+0000 7f040902b640 1 -- 172.21.15.84:0/2810863793 shutdown_connections 2023-10-16T11:06:19.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.282+0000 7f040902b640 1 --2- 172.21.15.84:0/2810863793 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04040b0b20 0x7f040406fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.282+0000 7f040902b640 1 --2- 172.21.15.84:0/2810863793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0404159b20 0x7f040406bbe0 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.282+0000 7f040902b640 1 -- 172.21.15.84:0/2810863793 >> 172.21.15.84:0/2810863793 conn(0x7f04040cb410 msgr2=0x7f04040cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:19.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.282+0000 7f040902b640 1 -- 172.21.15.84:0/2810863793 shutdown_connections 2023-10-16T11:06:19.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.282+0000 7f040902b640 1 -- 172.21.15.84:0/2810863793 wait complete. 2023-10-16T11:06:19.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.284+0000 7f040902b640 1 Processor -- start 2023-10-16T11:06:19.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.284+0000 7f040902b640 1 -- start start 2023-10-16T11:06:19.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.285+0000 7f040902b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04040b0b20 0x7f04040acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:19.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.285+0000 7f040902b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0404159b20 0x7f04040ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:19.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.285+0000 7f040902b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0404159f00 0x7f04040ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:19.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.285+0000 7f040902b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04041f2ad0 con 0x7f04040b0b20 2023-10-16T11:06:19.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.285+0000 7f040902b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04040aef90 con 0x7f0404159b20 2023-10-16T11:06:19.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.285+0000 7f040902b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04040af270 con 0x7f0404159f00 2023-10-16T11:06:19.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.285+0000 7f0402d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04040b0b20 0x7f04040acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:19.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.285+0000 7f0402d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04040b0b20 0x7f04040acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43556/0 (socket says 172.21.15.84:43556) 2023-10-16T11:06:19.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.286+0000 7f0402d9d640 1 -- 172.21.15.84:0/1480265247 learned_addr learned my addr 172.21.15.84:0/1480265247 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:19.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.286+0000 7f040259c640 1 --2- 172.21.15.84:0/1480265247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0404159b20 0x7f04040ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:19.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.286+0000 7f0402d9d640 1 -- 172.21.15.84:0/1480265247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0404159f00 msgr2=0x7f04040ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:19.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.286+0000 7f0402d9d640 1 --2- 172.21.15.84:0/1480265247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0404159f00 0x7f04040ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.286+0000 7f0402d9d640 1 -- 172.21.15.84:0/1480265247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0404159b20 msgr2=0x7f04040ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:19.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.286+0000 7f0402d9d640 1 --2- 172.21.15.84:0/1480265247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0404159b20 0x7f04040ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.286+0000 7f0402d9d640 1 -- 172.21.15.84:0/1480265247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f4009400 con 0x7f04040b0b20 2023-10-16T11:06:19.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.286+0000 7f0402d9d640 1 --2- 172.21.15.84:0/1480265247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04040b0b20 0x7f04040acb00 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f03f40098c0 tx=0x7f03f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:19.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.287+0000 7f03eb7fe640 1 -- 172.21.15.84:0/1480265247 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f403ba70 con 0x7f04040b0b20 2023-10-16T11:06:19.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.287+0000 7f03eb7fe640 1 -- 172.21.15.84:0/1480265247 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f403bbd0 con 0x7f04040b0b20 2023-10-16T11:06:19.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.287+0000 7f03eb7fe640 1 -- 172.21.15.84:0/1480265247 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f40453f0 con 0x7f04040b0b20 2023-10-16T11:06:19.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.287+0000 7f040902b640 1 -- 172.21.15.84:0/1480265247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04040af550 con 0x7f04040b0b20 2023-10-16T11:06:19.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.287+0000 7f040902b640 1 -- 172.21.15.84:0/1480265247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04040afb10 con 0x7f04040b0b20 2023-10-16T11:06:19.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.288+0000 7f040902b640 1 -- 172.21.15.84:0/1480265247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f040404e160 con 0x7f04040b0b20 2023-10-16T11:06:19.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.289+0000 7f03eb7fe640 1 -- 172.21.15.84:0/1480265247 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f03f404e450 con 0x7f04040b0b20 2023-10-16T11:06:19.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.290+0000 7f03eb7fe640 1 --2- 172.21.15.84:0/1480265247 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f03f006af90 0x7f03f006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:19.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.290+0000 7f03eb7fe640 1 -- 172.21.15.84:0/1480265247 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f03f40ab820 con 0x7f04040b0b20 2023-10-16T11:06:19.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.290+0000 7f040259c640 1 --2- 172.21.15.84:0/1480265247 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f03f006af90 0x7f03f006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:19.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.291+0000 7f040259c640 1 --2- 172.21.15.84:0/1480265247 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f03f006af90 0x7f03f006d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f03f8007fd0 tx=0x7f03f8009040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:19.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.294+0000 7f03eb7fe640 1 -- 172.21.15.84:0/1480265247 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f03f40769d0 con 0x7f04040b0b20 2023-10-16T11:06:19.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.488+0000 7f040902b640 1 -- 172.21.15.84:0/1480265247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04040c9610 con 0x7f04040b0b20 2023-10-16T11:06:19.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.490+0000 7f03eb7fe640 1 -- 172.21.15.84:0/1480265247 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+12887 (secure 0 0 0) 0x7f03f4079ff0 con 0x7f04040b0b20 2023-10-16T11:06:19.492 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:19.492 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":180,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:06:18.533405+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:06:19.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.492+0000 7f040902b640 1 -- 172.21.15.84:0/1480265247 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f03f006af90 msgr2=0x7f03f006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:19.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.492+0000 7f040902b640 1 --2- 172.21.15.84:0/1480265247 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f03f006af90 0x7f03f006d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f03f8007fd0 tx=0x7f03f8009040 comp rx=0 tx=0).stop 2023-10-16T11:06:19.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.493+0000 7f040902b640 1 -- 172.21.15.84:0/1480265247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04040b0b20 msgr2=0x7f04040acb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:19.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.493+0000 7f040902b640 1 --2- 172.21.15.84:0/1480265247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04040b0b20 0x7f04040acb00 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f03f40098c0 tx=0x7f03f40046c0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.493+0000 7f040902b640 1 -- 172.21.15.84:0/1480265247 shutdown_connections 2023-10-16T11:06:19.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.493+0000 7f040902b640 1 --2- 172.21.15.84:0/1480265247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0404159f00 0x7f04040ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.493+0000 7f040902b640 1 --2- 172.21.15.84:0/1480265247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0404159b20 0x7f04040ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.493+0000 7f040902b640 1 --2- 172.21.15.84:0/1480265247 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f03f006af90 0x7f03f006d450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.493+0000 7f040902b640 1 --2- 172.21.15.84:0/1480265247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04040b0b20 0x7f04040acb00 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.493+0000 7f040902b640 1 -- 172.21.15.84:0/1480265247 >> 172.21.15.84:0/1480265247 conn(0x7f04040cb410 msgr2=0x7f04040ceb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:19.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.493+0000 7f040902b640 1 -- 172.21.15.84:0/1480265247 shutdown_connections 2023-10-16T11:06:19.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.493+0000 7f040902b640 1 -- 172.21.15.84:0/1480265247 wait complete. 2023-10-16T11:06:19.513 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:06:19.514 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:06:19.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.728+0000 7fb0c5189640 1 Processor -- start 2023-10-16T11:06:19.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.729+0000 7fb0c5189640 1 -- start start 2023-10-16T11:06:19.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.730+0000 7fb0c5189640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c01428c0 0x7fb0c00c9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:19.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.730+0000 7fb0c5189640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb0c00c9690 0x7fb0c00ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:19.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.730+0000 7fb0c5189640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c00cc6b0 con 0x7fb0c006b660 2023-10-16T11:06:19.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.730+0000 7fb0c5189640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c00c9b20 con 0x7fb0c00c9690 2023-10-16T11:06:19.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.730+0000 7fb0c5189640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c01599b0 con 0x7fb0c01428c0 2023-10-16T11:06:19.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.731+0000 7fb0bf59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb0c00c9690 0x7fb0c00ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:19.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.731+0000 7fb0be59c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c01428c0 0x7fb0c00c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:19.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.731+0000 7fb0be59c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c01428c0 0x7fb0c00c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:37166/0 (socket says 172.21.15.84:37166) 2023-10-16T11:06:19.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.731+0000 7fb0be59c640 1 -- 172.21.15.84:0/60110757 learned_addr learned my addr 172.21.15.84:0/60110757 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:19.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.731+0000 7fb0bed9d640 1 --1- 172.21.15.84:0/60110757 >> v1:172.21.15.157:6789/0 conn(0x7fb0c006b660 0x7fb0c006ba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:48396/0 (socket says 172.21.15.84:48396) 2023-10-16T11:06:19.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.732+0000 7fb0bf59e640 1 -- 172.21.15.84:0/60110757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c01428c0 msgr2=0x7fb0c00c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:19.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.732+0000 7fb0bf59e640 1 --2- 172.21.15.84:0/60110757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c01428c0 0x7fb0c00c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.732+0000 7fb0bf59e640 1 -- 172.21.15.84:0/60110757 >> v1:172.21.15.157:6789/0 conn(0x7fb0c006b660 legacy=0x7fb0c006ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:19.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.732+0000 7fb0bf59e640 1 -- 172.21.15.84:0/60110757 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c006c290 con 0x7fb0c00c9690 2023-10-16T11:06:19.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.733+0000 7fb0bf59e640 1 --2- 172.21.15.84:0/60110757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c00c9690 0x7fb0c00ccb40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009d90 tx=0x7fb0b802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e162b874aafaea7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:19.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.733+0000 7fb0bdd9b640 1 -- 172.21.15.84:0/60110757 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b802cc50 con 0x7fb0c00c9690 2023-10-16T11:06:19.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.733+0000 7fb0bdd9b640 1 -- 172.21.15.84:0/60110757 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b802cdb0 con 0x7fb0c00c9690 2023-10-16T11:06:19.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.733+0000 7fb0bdd9b640 1 -- 172.21.15.84:0/60110757 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8034660 con 0x7fb0c00c9690 2023-10-16T11:06:19.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.734+0000 7fb0c5189640 1 -- 172.21.15.84:0/60110757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c00c9690 msgr2=0x7fb0c00ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:19.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.734+0000 7fb0c5189640 1 --2- 172.21.15.84:0/60110757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c00c9690 0x7fb0c00ccb40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009d90 tx=0x7fb0b802bea0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.734+0000 7fb0c5189640 1 -- 172.21.15.84:0/60110757 shutdown_connections 2023-10-16T11:06:19.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.734+0000 7fb0c5189640 1 --2- 172.21.15.84:0/60110757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c00c9690 0x7fb0c00ccb40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.734+0000 7fb0c5189640 1 --2- 172.21.15.84:0/60110757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c01428c0 0x7fb0c00c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.734+0000 7fb0c5189640 1 -- 172.21.15.84:0/60110757 >> 172.21.15.84:0/60110757 conn(0x7fb0c00ce9b0 msgr2=0x7fb0c00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:19.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.734+0000 7fb0c5189640 1 -- 172.21.15.84:0/60110757 shutdown_connections 2023-10-16T11:06:19.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.734+0000 7fb0c5189640 1 -- 172.21.15.84:0/60110757 wait complete. 2023-10-16T11:06:19.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.736+0000 7fb0c5189640 1 Processor -- start 2023-10-16T11:06:19.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.737+0000 7fb0c5189640 1 -- start start 2023-10-16T11:06:19.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.738+0000 7fb0c5189640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c006b660 0x7fb0c00af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:19.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.738+0000 7fb0c5189640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c00c9690 0x7fb0c00c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:19.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.738+0000 7fb0c5189640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0c01428c0 0x7fb0c0150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:19.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.738+0000 7fb0c5189640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c00c6ac0 con 0x7fb0c01428c0 2023-10-16T11:06:19.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.738+0000 7fb0c5189640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c00c6da0 con 0x7fb0c00c9690 2023-10-16T11:06:19.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.738+0000 7fb0c5189640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c00c37d0 con 0x7fb0c006b660 2023-10-16T11:06:19.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.738+0000 7fb0bf59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0c01428c0 0x7fb0c0150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:19.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.738+0000 7fb0bf59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0c01428c0 0x7fb0c0150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43570/0 (socket says 172.21.15.84:43570) 2023-10-16T11:06:19.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.738+0000 7fb0bf59e640 1 -- 172.21.15.84:0/2860228837 learned_addr learned my addr 172.21.15.84:0/2860228837 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:19.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.738+0000 7fb0be59c640 1 --2- 172.21.15.84:0/2860228837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c00c9690 0x7fb0c00c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:19.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.739+0000 7fb0bed9d640 1 --2- 172.21.15.84:0/2860228837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c006b660 0x7fb0c00af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:19.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.739+0000 7fb0bf59e640 1 -- 172.21.15.84:0/2860228837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c006b660 msgr2=0x7fb0c00af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:19.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.739+0000 7fb0bf59e640 1 --2- 172.21.15.84:0/2860228837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c006b660 0x7fb0c00af6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.739+0000 7fb0bf59e640 1 -- 172.21.15.84:0/2860228837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c00c9690 msgr2=0x7fb0c00c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:19.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.739+0000 7fb0bf59e640 1 --2- 172.21.15.84:0/2860228837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c00c9690 0x7fb0c00c6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.739+0000 7fb0bf59e640 1 -- 172.21.15.84:0/2860228837 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b8009a70 con 0x7fb0c01428c0 2023-10-16T11:06:19.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.739+0000 7fb0bed9d640 1 --2- 172.21.15.84:0/2860228837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c006b660 0x7fb0c00af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:06:19.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.739+0000 7fb0bf59e640 1 --2- 172.21.15.84:0/2860228837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0c01428c0 0x7fb0c0150ab0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009d90 tx=0x7fb0b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:19.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.739+0000 7fb0a77fe640 1 -- 172.21.15.84:0/2860228837 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b803e610 con 0x7fb0c01428c0 2023-10-16T11:06:19.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.740+0000 7fb0a77fe640 1 -- 172.21.15.84:0/2860228837 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b803e770 con 0x7fb0c01428c0 2023-10-16T11:06:19.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.740+0000 7fb0c5189640 1 -- 172.21.15.84:0/2860228837 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c00c3ab0 con 0x7fb0c01428c0 2023-10-16T11:06:19.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.740+0000 7fb0a77fe640 1 -- 172.21.15.84:0/2860228837 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b803eb20 con 0x7fb0c01428c0 2023-10-16T11:06:19.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.740+0000 7fb0c5189640 1 -- 172.21.15.84:0/2860228837 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c00afdd0 con 0x7fb0c01428c0 2023-10-16T11:06:19.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.741+0000 7fb0c5189640 1 -- 172.21.15.84:0/2860228837 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c004e160 con 0x7fb0c01428c0 2023-10-16T11:06:19.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.742+0000 7fb0a77fe640 1 -- 172.21.15.84:0/2860228837 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb0b803b050 con 0x7fb0c01428c0 2023-10-16T11:06:19.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.743+0000 7fb0a77fe640 1 --2- 172.21.15.84:0/2860228837 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fb0ac06af90 0x7fb0ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:19.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.743+0000 7fb0a77fe640 1 -- 172.21.15.84:0/2860228837 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fb0b80abc50 con 0x7fb0c01428c0 2023-10-16T11:06:19.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.744+0000 7fb0bed9d640 1 --2- 172.21.15.84:0/2860228837 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fb0ac06af90 0x7fb0ac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:19.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.744+0000 7fb0bed9d640 1 --2- 172.21.15.84:0/2860228837 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fb0ac06af90 0x7fb0ac06d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0007fd0 tx=0x7fb0b0007d40 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:19.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.748+0000 7fb0a77fe640 1 -- 172.21.15.84:0/2860228837 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb0b8077060 con 0x7fb0c01428c0 2023-10-16T11:06:19.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.937+0000 7fb0c5189640 1 -- 172.21.15.84:0/2860228837 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0c00afdd0 con 0x7fb0c01428c0 2023-10-16T11:06:19.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.938+0000 7fb0a77fe640 1 -- 172.21.15.84:0/2860228837 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v180) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0b804e020 con 0x7fb0c01428c0 2023-10-16T11:06:19.940 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:06:19.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.941+0000 7fb0c5189640 1 -- 172.21.15.84:0/2860228837 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fb0ac06af90 msgr2=0x7fb0ac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:19.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.941+0000 7fb0c5189640 1 --2- 172.21.15.84:0/2860228837 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fb0ac06af90 0x7fb0ac06d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0007fd0 tx=0x7fb0b0007d40 comp rx=0 tx=0).stop 2023-10-16T11:06:19.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.941+0000 7fb0c5189640 1 -- 172.21.15.84:0/2860228837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0c01428c0 msgr2=0x7fb0c0150ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:19.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.941+0000 7fb0c5189640 1 --2- 172.21.15.84:0/2860228837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0c01428c0 0x7fb0c0150ab0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009d90 tx=0x7fb0b8004a40 comp rx=0 tx=0).stop 2023-10-16T11:06:19.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.941+0000 7fb0c5189640 1 -- 172.21.15.84:0/2860228837 shutdown_connections 2023-10-16T11:06:19.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.941+0000 7fb0c5189640 1 --2- 172.21.15.84:0/2860228837 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fb0ac06af90 0x7fb0ac06d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.941+0000 7fb0c5189640 1 --2- 172.21.15.84:0/2860228837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0c01428c0 0x7fb0c0150ab0 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.941+0000 7fb0c5189640 1 --2- 172.21.15.84:0/2860228837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0c00c9690 0x7fb0c00c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.941+0000 7fb0c5189640 1 --2- 172.21.15.84:0/2860228837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0c006b660 0x7fb0c00af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:19.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.941+0000 7fb0c5189640 1 -- 172.21.15.84:0/2860228837 >> 172.21.15.84:0/2860228837 conn(0x7fb0c00ce9b0 msgr2=0x7fb0c0159b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:19.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.941+0000 7fb0c5189640 1 -- 172.21.15.84:0/2860228837 shutdown_connections 2023-10-16T11:06:19.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:19.941+0000 7fb0c5189640 1 -- 172.21.15.84:0/2860228837 wait complete. 2023-10-16T11:06:19.962 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T11:06:20.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.164+0000 7fc41100d640 1 Processor -- start 2023-10-16T11:06:20.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.164+0000 7fc41100d640 1 -- start start 2023-10-16T11:06:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.165+0000 7fc41100d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc40c06c040 0x7fc40c06c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.165+0000 7fc41100d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc40c0ad970 0x7fc40c15e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.165+0000 7fc41100d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc40c06c960 con 0x7fc40c15ebc0 2023-10-16T11:06:20.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.165+0000 7fc41100d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc40c15b9d0 con 0x7fc40c0ad970 2023-10-16T11:06:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.165+0000 7fc41100d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc40c15bcb0 con 0x7fc40c06c040 2023-10-16T11:06:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.166+0000 7fc403fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc40c0ad970 0x7fc40c15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.166+0000 7fc41080c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc40c15ebc0 0x7fc40c0cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49766/0 (socket says 172.21.15.84:49766) 2023-10-16T11:06:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.166+0000 7fc403fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc40c0ad970 0x7fc40c15e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43580/0 (socket says 172.21.15.84:43580) 2023-10-16T11:06:20.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.166+0000 7fc41080c640 1 -- 172.21.15.84:0/2757107282 learned_addr learned my addr 172.21.15.84:0/2757107282 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:20.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.166+0000 7fc40bfff640 1 --2- 172.21.15.84:0/2757107282 >> v2:172.21.15.157:3300/0 conn(0x7fc40c06c040 0x7fc40c06c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:20.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.166+0000 7fc403fff640 1 -- 172.21.15.84:0/2757107282 >> v2:172.21.15.157:3300/0 conn(0x7fc40c06c040 msgr2=0x7fc40c06c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:20.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.166+0000 7fc403fff640 1 --2- 172.21.15.84:0/2757107282 >> v2:172.21.15.157:3300/0 conn(0x7fc40c06c040 0x7fc40c06c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:20.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.166+0000 7fc403fff640 1 -- 172.21.15.84:0/2757107282 >> v1:172.21.15.84:6789/0 conn(0x7fc40c15ebc0 legacy=0x7fc40c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.166+0000 7fc403fff640 1 -- 172.21.15.84:0/2757107282 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40c15bf90 con 0x7fc40c0ad970 2023-10-16T11:06:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.166+0000 7fc40b7fe640 1 -- 172.21.15.84:0/2757107282 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4013994651 0 0) 0x7fc40c06c960 con 0x7fc40c15ebc0 2023-10-16T11:06:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.166+0000 7fc40bfff640 1 --2- 172.21.15.84:0/2757107282 >> v2:172.21.15.157:3300/0 conn(0x7fc40c06c040 0x7fc40c06c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.167+0000 7fc403fff640 1 --2- 172.21.15.84:0/2757107282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc40c0ad970 0x7fc40c15e680 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0cf350 tx=0x7fc3f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27db09429b42f2e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:20.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.167+0000 7fc40b7fe640 1 -- 172.21.15.84:0/2757107282 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4031830 con 0x7fc40c0ad970 2023-10-16T11:06:20.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.168+0000 7fc40b7fe640 1 -- 172.21.15.84:0/2757107282 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f4031990 con 0x7fc40c0ad970 2023-10-16T11:06:20.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.168+0000 7fc40b7fe640 1 -- 172.21.15.84:0/2757107282 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4031d20 con 0x7fc40c0ad970 2023-10-16T11:06:20.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.168+0000 7fc41100d640 1 -- 172.21.15.84:0/2757107282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc40c0ad970 msgr2=0x7fc40c15e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:20.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.168+0000 7fc41100d640 1 --2- 172.21.15.84:0/2757107282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc40c0ad970 0x7fc40c15e680 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0cf350 tx=0x7fc3f402b8d0 comp rx=0 tx=0).stop 2023-10-16T11:06:20.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.168+0000 7fc41100d640 1 -- 172.21.15.84:0/2757107282 shutdown_connections 2023-10-16T11:06:20.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.168+0000 7fc41100d640 1 --2- 172.21.15.84:0/2757107282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc40c0ad970 0x7fc40c15e680 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:20.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.168+0000 7fc41100d640 1 --2- 172.21.15.84:0/2757107282 >> v2:172.21.15.157:3300/0 conn(0x7fc40c06c040 0x7fc40c06c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:20.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.168+0000 7fc41100d640 1 -- 172.21.15.84:0/2757107282 >> 172.21.15.84:0/2757107282 conn(0x7fc40c0ce6a0 msgr2=0x7fc40c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:20.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.168+0000 7fc41100d640 1 -- 172.21.15.84:0/2757107282 shutdown_connections 2023-10-16T11:06:20.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.168+0000 7fc41100d640 1 -- 172.21.15.84:0/2757107282 wait complete. 2023-10-16T11:06:20.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.170+0000 7fc41100d640 1 Processor -- start 2023-10-16T11:06:20.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.170+0000 7fc41100d640 1 -- start start 2023-10-16T11:06:20.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.171+0000 7fc41100d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc40c06c040 0x7fc40c0b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:20.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.171+0000 7fc41100d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc40c0ad970 0x7fc40c0acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:20.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.171+0000 7fc41100d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc40c15ebc0 0x7fc40c1f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:20.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.171+0000 7fc41100d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c1e48a0 con 0x7fc40c15ebc0 2023-10-16T11:06:20.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.171+0000 7fc41100d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c0b45d0 con 0x7fc40c0ad970 2023-10-16T11:06:20.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.171+0000 7fc41100d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c0b42f0 con 0x7fc40c06c040 2023-10-16T11:06:20.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.171+0000 7fc41080c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc40c15ebc0 0x7fc40c1f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:20.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.171+0000 7fc41080c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc40c15ebc0 0x7fc40c1f29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43590/0 (socket says 172.21.15.84:43590) 2023-10-16T11:06:20.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.171+0000 7fc41080c640 1 -- 172.21.15.84:0/3357392748 learned_addr learned my addr 172.21.15.84:0/3357392748 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:20.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.171+0000 7fc403fff640 1 --2- 172.21.15.84:0/3357392748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc40c0ad970 0x7fc40c0acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:20.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.171+0000 7fc40bfff640 1 --2- 172.21.15.84:0/3357392748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc40c06c040 0x7fc40c0b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:20.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.172+0000 7fc41080c640 1 -- 172.21.15.84:0/3357392748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc40c06c040 msgr2=0x7fc40c0b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:20.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.172+0000 7fc41080c640 1 --2- 172.21.15.84:0/3357392748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc40c06c040 0x7fc40c0b2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:20.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.172+0000 7fc41080c640 1 -- 172.21.15.84:0/3357392748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc40c0ad970 msgr2=0x7fc40c0acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:20.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.172+0000 7fc41080c640 1 --2- 172.21.15.84:0/3357392748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc40c0ad970 0x7fc40c0acb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:20.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.172+0000 7fc41080c640 1 -- 172.21.15.84:0/3357392748 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f4009400 con 0x7fc40c15ebc0 2023-10-16T11:06:20.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.172+0000 7fc403fff640 1 --2- 172.21.15.84:0/3357392748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc40c0ad970 0x7fc40c0acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:20.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.172+0000 7fc41080c640 1 --2- 172.21.15.84:0/3357392748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc40c15ebc0 0x7fc40c1f29e0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7fc40400c810 tx=0x7fc40400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:20.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.172+0000 7fc4097fa640 1 -- 172.21.15.84:0/3357392748 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40400a640 con 0x7fc40c15ebc0 2023-10-16T11:06:20.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.172+0000 7fc4097fa640 1 -- 172.21.15.84:0/3357392748 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40400a7c0 con 0x7fc40c15ebc0 2023-10-16T11:06:20.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.172+0000 7fc4097fa640 1 -- 172.21.15.84:0/3357392748 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40400ab70 con 0x7fc40c15ebc0 2023-10-16T11:06:20.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.172+0000 7fc41100d640 1 -- 172.21.15.84:0/3357392748 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc40c0b23b0 con 0x7fc40c15ebc0 2023-10-16T11:06:20.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.172+0000 7fc41100d640 1 -- 172.21.15.84:0/3357392748 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc40c0b2970 con 0x7fc40c15ebc0 2023-10-16T11:06:20.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.173+0000 7fc41100d640 1 -- 172.21.15.84:0/3357392748 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc40c06b330 con 0x7fc40c15ebc0 2023-10-16T11:06:20.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.174+0000 7fc4097fa640 1 -- 172.21.15.84:0/3357392748 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc40400acd0 con 0x7fc40c15ebc0 2023-10-16T11:06:20.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.175+0000 7fc4097fa640 1 --2- 172.21.15.84:0/3357392748 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc3ec06af90 0x7fc3ec06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:20.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.175+0000 7fc4097fa640 1 -- 172.21.15.84:0/3357392748 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fc404088900 con 0x7fc40c15ebc0 2023-10-16T11:06:20.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.175+0000 7fc40bfff640 1 --2- 172.21.15.84:0/3357392748 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc3ec06af90 0x7fc3ec06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:20.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.176+0000 7fc40bfff640 1 --2- 172.21.15.84:0/3357392748 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc3ec06af90 0x7fc3ec06d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00b840 tx=0x7fc3fc008040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:20.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.180+0000 7fc4097fa640 1 -- 172.21.15.84:0/3357392748 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc404053bc0 con 0x7fc40c15ebc0 2023-10-16T11:06:20.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.371+0000 7fc41100d640 1 -- 172.21.15.84:0/3357392748 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc40c04e9b0 con 0x7fc40c15ebc0 2023-10-16T11:06:20.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.372+0000 7fc4097fa640 1 -- 172.21.15.84:0/3357392748 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v180) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc404018020 con 0x7fc40c15ebc0 2023-10-16T11:06:20.373 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:20.374 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T11:06:20.374 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:20.374 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-16T11:06:20.374 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-16T11:06:20.374 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:20.375 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T11:06:20.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.374+0000 7fc41100d640 1 -- 172.21.15.84:0/3357392748 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc3ec06af90 msgr2=0x7fc3ec06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:20.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.374+0000 7fc41100d640 1 --2- 172.21.15.84:0/3357392748 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc3ec06af90 0x7fc3ec06d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00b840 tx=0x7fc3fc008040 comp rx=0 tx=0).stop 2023-10-16T11:06:20.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.374+0000 7fc41100d640 1 -- 172.21.15.84:0/3357392748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc40c15ebc0 msgr2=0x7fc40c1f29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:20.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.374+0000 7fc41100d640 1 --2- 172.21.15.84:0/3357392748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc40c15ebc0 0x7fc40c1f29e0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7fc40400c810 tx=0x7fc40400ccd0 comp rx=0 tx=0).stop 2023-10-16T11:06:20.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.374+0000 7fc41100d640 1 -- 172.21.15.84:0/3357392748 shutdown_connections 2023-10-16T11:06:20.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.374+0000 7fc41100d640 1 --2- 172.21.15.84:0/3357392748 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc3ec06af90 0x7fc3ec06d450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:20.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.374+0000 7fc41100d640 1 --2- 172.21.15.84:0/3357392748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc40c15ebc0 0x7fc40c1f29e0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:20.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.374+0000 7fc41100d640 1 --2- 172.21.15.84:0/3357392748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc40c0ad970 0x7fc40c0acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:20.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.374+0000 7fc41100d640 1 --2- 172.21.15.84:0/3357392748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc40c06c040 0x7fc40c0b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:20.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.374+0000 7fc41100d640 1 -- 172.21.15.84:0/3357392748 >> 172.21.15.84:0/3357392748 conn(0x7fc40c0ce6a0 msgr2=0x7fc40c0c5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:20.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.375+0000 7fc41100d640 1 -- 172.21.15.84:0/3357392748 shutdown_connections 2023-10-16T11:06:20.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.375+0000 7fc41100d640 1 -- 172.21.15.84:0/3357392748 wait complete. 2023-10-16T11:06:20.393 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T11:06:20.394 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T11:06:20.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.607+0000 7f906d19e640 1 Processor -- start 2023-10-16T11:06:20.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.608+0000 7f906d19e640 1 -- start start 2023-10-16T11:06:20.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.609+0000 7f906d19e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f906815cf50 0x7f90680d2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:20.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.609+0000 7f906d19e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f90680b8a10 0x7f90680d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:20.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.609+0000 7f906d19e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f90680b4750 0x7f90680b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:20.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.609+0000 7f906d19e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90680b5120 con 0x7f90680b8a10 2023-10-16T11:06:20.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.609+0000 7f906d19e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f906815d330 con 0x7f90680b4750 2023-10-16T11:06:20.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.609+0000 7f906d19e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90680d1680 con 0x7f906815cf50 2023-10-16T11:06:20.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.609+0000 7f906659c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f90680b8a10 0x7f90680d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:20.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.610+0000 7f906659c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f90680b8a10 0x7f90680d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43592/0 (socket says 172.21.15.84:43592) 2023-10-16T11:06:20.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.610+0000 7f906659c640 1 -- 172.21.15.84:0/782823488 learned_addr learned my addr 172.21.15.84:0/782823488 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:20.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.610+0000 7f906659c640 1 -- 172.21.15.84:0/782823488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f906815cf50 msgr2=0x7f90680d2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:20.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.610+0000 7f906659c640 1 --2- 172.21.15.84:0/782823488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f906815cf50 0x7f90680d2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:20.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.610+0000 7f906659c640 1 -- 172.21.15.84:0/782823488 >> v2:172.21.15.157:3300/0 conn(0x7f90680b4750 msgr2=0x7f90680b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:20.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.610+0000 7f906759e640 1 --2- 172.21.15.84:0/782823488 >> v2:172.21.15.157:3300/0 conn(0x7f90680b4750 0x7f90680b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:20.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.610+0000 7f906659c640 1 --2- 172.21.15.84:0/782823488 >> v2:172.21.15.157:3300/0 conn(0x7f90680b4750 0x7f90680b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:20.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.610+0000 7f906659c640 1 -- 172.21.15.84:0/782823488 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90680d1960 con 0x7f90680b8a10 2023-10-16T11:06:20.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.611+0000 7f906659c640 1 --2- 172.21.15.84:0/782823488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90680b8a10 0x7f90680d0d10 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f90680b5850 tx=0x7f905002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840e47eefaf26b16 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:20.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.611+0000 7f9065d9b640 1 -- 172.21.15.84:0/782823488 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050031870 con 0x7f90680b8a10 2023-10-16T11:06:20.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.611+0000 7f9065d9b640 1 -- 172.21.15.84:0/782823488 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90500319d0 con 0x7f90680b8a10 2023-10-16T11:06:20.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.611+0000 7f9065d9b640 1 -- 172.21.15.84:0/782823488 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050031d60 con 0x7f90680b8a10 2023-10-16T11:06:20.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.612+0000 7f906d19e640 1 -- 172.21.15.84:0/782823488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90680b8a10 msgr2=0x7f90680d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:20.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.612+0000 7f906d19e640 1 --2- 172.21.15.84:0/782823488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90680b8a10 0x7f90680d0d10 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f90680b5850 tx=0x7f905002b910 comp rx=0 tx=0).stop 2023-10-16T11:06:20.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.612+0000 7f906d19e640 1 -- 172.21.15.84:0/782823488 shutdown_connections 2023-10-16T11:06:20.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.612+0000 7f906d19e640 1 --2- 172.21.15.84:0/782823488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90680b8a10 0x7f90680d0d10 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:20.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.612+0000 7f906d19e640 1 --2- 172.21.15.84:0/782823488 >> v2:172.21.15.157:3300/0 conn(0x7f90680b4750 0x7f90680b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:20.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.612+0000 7f906d19e640 1 --2- 172.21.15.84:0/782823488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f906815cf50 0x7f90680d2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:20.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.612+0000 7f906d19e640 1 -- 172.21.15.84:0/782823488 >> 172.21.15.84:0/782823488 conn(0x7f90680cf750 msgr2=0x7f90680cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:20.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.612+0000 7f906d19e640 1 -- 172.21.15.84:0/782823488 shutdown_connections 2023-10-16T11:06:20.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.612+0000 7f906d19e640 1 -- 172.21.15.84:0/782823488 wait complete. 2023-10-16T11:06:20.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.614+0000 7f906d19e640 1 Processor -- start 2023-10-16T11:06:20.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.614+0000 7f906d19e640 1 -- start start 2023-10-16T11:06:20.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.615+0000 7f906d19e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90680b4750 0x7f90680b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:20.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.615+0000 7f906d19e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90680b8a10 0x7f906814a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:20.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.615+0000 7f906d19e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f906815cf50 0x7f90680b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:20.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.615+0000 7f906d19e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90680b7480 con 0x7f90680b8a10 2023-10-16T11:06:20.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.615+0000 7f906d19e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90681f3b40 con 0x7f906815cf50 2023-10-16T11:06:20.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.615+0000 7f906d19e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90681552e0 con 0x7f90680b4750 2023-10-16T11:06:20.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.615+0000 7f906659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90680b8a10 0x7f906814a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:20.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.615+0000 7f906659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90680b8a10 0x7f906814a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43608/0 (socket says 172.21.15.84:43608) 2023-10-16T11:06:20.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.615+0000 7f906659c640 1 -- 172.21.15.84:0/2777996063 learned_addr learned my addr 172.21.15.84:0/2777996063 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:20.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.616+0000 7f906659c640 1 -- 172.21.15.84:0/2777996063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90680b4750 msgr2=0x7f90680b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:20.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.616+0000 7f906659c640 1 --2- 172.21.15.84:0/2777996063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90680b4750 0x7f90680b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:20.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.616+0000 7f906759e640 1 --2- 172.21.15.84:0/2777996063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f906815cf50 0x7f90680b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:20.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.616+0000 7f906659c640 1 -- 172.21.15.84:0/2777996063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f906815cf50 msgr2=0x7f90680b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:20.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.616+0000 7f906659c640 1 --2- 172.21.15.84:0/2777996063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f906815cf50 0x7f90680b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:20.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.616+0000 7f906659c640 1 -- 172.21.15.84:0/2777996063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90500094d0 con 0x7f90680b8a10 2023-10-16T11:06:20.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.616+0000 7f906759e640 1 --2- 172.21.15.84:0/2777996063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f906815cf50 0x7f90680b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:20.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.616+0000 7f906659c640 1 --2- 172.21.15.84:0/2777996063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90680b8a10 0x7f906814a500 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f905002b950 tx=0x7f90500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:20.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.616+0000 7f905f7fe640 1 -- 172.21.15.84:0/2777996063 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050031860 con 0x7f90680b8a10 2023-10-16T11:06:20.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.616+0000 7f905f7fe640 1 -- 172.21.15.84:0/2777996063 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90500443f0 con 0x7f90680b8a10 2023-10-16T11:06:20.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.616+0000 7f905f7fe640 1 -- 172.21.15.84:0/2777996063 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90500447a0 con 0x7f90680b8a10 2023-10-16T11:06:20.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.616+0000 7f906d19e640 1 -- 172.21.15.84:0/2777996063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90681555c0 con 0x7f90680b8a10 2023-10-16T11:06:20.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.617+0000 7f906d19e640 1 -- 172.21.15.84:0/2777996063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9068155b80 con 0x7f90680b8a10 2023-10-16T11:06:20.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.618+0000 7f906d19e640 1 -- 172.21.15.84:0/2777996063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9068072350 con 0x7f90680b8a10 2023-10-16T11:06:20.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.619+0000 7f905f7fe640 1 -- 172.21.15.84:0/2777996063 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f905004d450 con 0x7f90680b8a10 2023-10-16T11:06:20.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.619+0000 7f905f7fe640 1 --2- 172.21.15.84:0/2777996063 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f904806af90 0x7f904806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:20.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.620+0000 7f905f7fe640 1 -- 172.21.15.84:0/2777996063 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f905002d040 con 0x7f90680b8a10 2023-10-16T11:06:20.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.620+0000 7f9066d9d640 1 --2- 172.21.15.84:0/2777996063 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f904806af90 0x7f904806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:20.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.620+0000 7f9066d9d640 1 --2- 172.21.15.84:0/2777996063 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f904806af90 0x7f904806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f90681f3a10 tx=0x7f9058009250 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:20.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.624+0000 7f905f7fe640 1 -- 172.21.15.84:0/2777996063 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9050076e90 con 0x7f90680b8a10 2023-10-16T11:06:20.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:20.816+0000 7f906d19e640 1 -- 172.21.15.84:0/2777996063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9068155420 con 0x7f90680b8a10 2023-10-16T11:06:21.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.548+0000 7f905f7fe640 1 -- 172.21.15.84:0/2777996063 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v181) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9068155420 con 0x7f90680b8a10 2023-10-16T11:06:21.550 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-16T11:06:21.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.552+0000 7f906d19e640 1 -- 172.21.15.84:0/2777996063 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f904806af90 msgr2=0x7f904806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:21.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.552+0000 7f906d19e640 1 --2- 172.21.15.84:0/2777996063 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f904806af90 0x7f904806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f90681f3a10 tx=0x7f9058009250 comp rx=0 tx=0).stop 2023-10-16T11:06:21.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.552+0000 7f906d19e640 1 -- 172.21.15.84:0/2777996063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90680b8a10 msgr2=0x7f906814a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:21.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.552+0000 7f906d19e640 1 --2- 172.21.15.84:0/2777996063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90680b8a10 0x7f906814a500 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f905002b950 tx=0x7f90500046c0 comp rx=0 tx=0).stop 2023-10-16T11:06:21.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.552+0000 7f906d19e640 1 -- 172.21.15.84:0/2777996063 shutdown_connections 2023-10-16T11:06:21.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.552+0000 7f906d19e640 1 --2- 172.21.15.84:0/2777996063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f906815cf50 0x7f90680b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:21.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.552+0000 7f906d19e640 1 --2- 172.21.15.84:0/2777996063 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f904806af90 0x7f904806d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:21.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.552+0000 7f906d19e640 1 --2- 172.21.15.84:0/2777996063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90680b8a10 0x7f906814a500 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:21.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.552+0000 7f906d19e640 1 --2- 172.21.15.84:0/2777996063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90680b4750 0x7f90680b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:21.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.552+0000 7f906d19e640 1 -- 172.21.15.84:0/2777996063 >> 172.21.15.84:0/2777996063 conn(0x7f90680cf750 msgr2=0x7f906815a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:21.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.553+0000 7f906d19e640 1 -- 172.21.15.84:0/2777996063 shutdown_connections 2023-10-16T11:06:21.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.553+0000 7f906d19e640 1 -- 172.21.15.84:0/2777996063 wait complete. 2023-10-16T11:06:21.581 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T11:06:21.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.793+0000 7fc99a0df640 1 Processor -- start 2023-10-16T11:06:21.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.793+0000 7fc99a0df640 1 -- start start 2023-10-16T11:06:21.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.794+0000 7fc99a0df640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc994065ab0 0x7fc9940c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:21.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.795+0000 7fc99a0df640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9940acd70 con 0x7fc99406d220 2023-10-16T11:06:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.795+0000 7fc99a0df640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9940ac4a0 con 0x7fc9940ce280 2023-10-16T11:06:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.795+0000 7fc99a0df640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc994065f40 con 0x7fc994065ab0 2023-10-16T11:06:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.795+0000 7fc9990dd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc99406d220 0x7fc9940cdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49776/0 (socket says 172.21.15.84:49776) 2023-10-16T11:06:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.795+0000 7fc9990dd640 1 -- 172.21.15.84:0/26872951 learned_addr learned my addr 172.21.15.84:0/26872951 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:21.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.795+0000 7fc9988dc640 1 --2- 172.21.15.84:0/26872951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc994065ab0 0x7fc9940c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:21.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.795+0000 7fc98bfff640 1 -- 172.21.15.84:0/26872951 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340993680 0 0) 0x7fc9940acd70 con 0x7fc99406d220 2023-10-16T11:06:21.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.795+0000 7fc98bfff640 1 -- 172.21.15.84:0/26872951 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc980003520 con 0x7fc99406d220 2023-10-16T11:06:21.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.796+0000 7fc98bfff640 1 -- 172.21.15.84:0/26872951 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1187732702 0 0) 0x7fc980003520 con 0x7fc99406d220 2023-10-16T11:06:21.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.796+0000 7fc98bfff640 1 -- 172.21.15.84:0/26872951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc994065ab0 msgr2=0x7fc9940c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:21.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.796+0000 7fc98bfff640 1 --2- 172.21.15.84:0/26872951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc994065ab0 0x7fc9940c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:21.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.796+0000 7fc98bfff640 1 -- 172.21.15.84:0/26872951 >> v1:172.21.15.157:6789/0 conn(0x7fc9940ce280 legacy=0x7fc9940ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:21.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.796+0000 7fc98bfff640 1 -- 172.21.15.84:0/26872951 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9940ac720 con 0x7fc99406d220 2023-10-16T11:06:21.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.796+0000 7fc98bfff640 1 -- 172.21.15.84:0/26872951 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc990001670 con 0x7fc99406d220 2023-10-16T11:06:21.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.796+0000 7fc98bfff640 1 -- 172.21.15.84:0/26872951 >> v1:172.21.15.84:6789/0 conn(0x7fc99406d220 legacy=0x7fc9940cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:21.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.796+0000 7fc98bfff640 1 --2- 172.21.15.84:0/26872951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9800039a0 0x7fc980003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:21.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.796+0000 7fc98bfff640 1 --2- 172.21.15.84:0/26872951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc980004300 0x7fc9800047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:21.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.796+0000 7fc98bfff640 1 --2- 172.21.15.84:0/26872951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc980004fe0 0x7fc98000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:21.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.796+0000 7fc98bfff640 1 -- 172.21.15.84:0/26872951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994065f40 con 0x7fc980004300 2023-10-16T11:06:21.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.796+0000 7fc98bfff640 1 -- 172.21.15.84:0/26872951 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98000bec0 con 0x7fc980004fe0 2023-10-16T11:06:21.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.796+0000 7fc98bfff640 1 -- 172.21.15.84:0/26872951 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98000c030 con 0x7fc9800039a0 2023-10-16T11:06:21.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.796+0000 7fc9988dc640 1 --2- 172.21.15.84:0/26872951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc980004300 0x7fc9800047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:21.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.797+0000 7fc9990dd640 1 --2- 172.21.15.84:0/26872951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9800039a0 0x7fc980003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:21.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.797+0000 7fc9998de640 1 --2- 172.21.15.84:0/26872951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc980004fe0 0x7fc98000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:21.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.797+0000 7fc9988dc640 1 -- 172.21.15.84:0/26872951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9800039a0 msgr2=0x7fc980003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:21.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.797+0000 7fc9988dc640 1 --2- 172.21.15.84:0/26872951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9800039a0 0x7fc980003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:21.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.797+0000 7fc9988dc640 1 -- 172.21.15.84:0/26872951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc980004fe0 msgr2=0x7fc98000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:21.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.797+0000 7fc9988dc640 1 --2- 172.21.15.84:0/26872951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc980004fe0 0x7fc98000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:21.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.797+0000 7fc9988dc640 1 -- 172.21.15.84:0/26872951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc980003520 con 0x7fc980004300 2023-10-16T11:06:21.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.797+0000 7fc9990dd640 1 --2- 172.21.15.84:0/26872951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9800039a0 0x7fc980003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:21.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.797+0000 7fc9998de640 1 --2- 172.21.15.84:0/26872951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc980004fe0 0x7fc98000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:21.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.798+0000 7fc9988dc640 1 --2- 172.21.15.84:0/26872951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc980004300 0x7fc9800047e0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fc984009ac0 tx=0x7fc9800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e5edc391221f2d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:21.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.798+0000 7fc98bfff640 1 -- 172.21.15.84:0/26872951 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98402c6b0 con 0x7fc980004300 2023-10-16T11:06:21.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.798+0000 7fc98bfff640 1 -- 172.21.15.84:0/26872951 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98402c810 con 0x7fc980004300 2023-10-16T11:06:21.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.798+0000 7fc98bfff640 1 -- 172.21.15.84:0/26872951 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98402cba0 con 0x7fc980004300 2023-10-16T11:06:21.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.799+0000 7fc99a0df640 1 -- 172.21.15.84:0/26872951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc980004300 msgr2=0x7fc9800047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:21.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.799+0000 7fc99a0df640 1 --2- 172.21.15.84:0/26872951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc980004300 0x7fc9800047e0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fc984009ac0 tx=0x7fc9800057f0 comp rx=0 tx=0).stop 2023-10-16T11:06:21.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.799+0000 7fc9998de640 1 -- 172.21.15.84:0/26872951 reap_dead start 2023-10-16T11:06:21.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.799+0000 7fc99a0df640 1 -- 172.21.15.84:0/26872951 shutdown_connections 2023-10-16T11:06:21.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.799+0000 7fc99a0df640 1 -- 172.21.15.84:0/26872951 >> 172.21.15.84:0/26872951 conn(0x7fc9940b0a80 msgr2=0x7fc9940b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:21.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.799+0000 7fc99a0df640 1 -- 172.21.15.84:0/26872951 shutdown_connections 2023-10-16T11:06:21.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.799+0000 7fc99a0df640 1 -- 172.21.15.84:0/26872951 wait complete. 2023-10-16T11:06:21.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.801+0000 7fc99a0df640 1 Processor -- start 2023-10-16T11:06:21.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.801+0000 7fc99a0df640 1 -- start start 2023-10-16T11:06:21.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.802+0000 7fc99a0df640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc980004300 0x7fc9941450c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:21.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.802+0000 7fc99a0df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc994145600 0x7fc994141c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:21.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.802+0000 7fc99a0df640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9940af590 0x7fc9941421c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:21.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.802+0000 7fc99a0df640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994148ca0 con 0x7fc9940af590 2023-10-16T11:06:21.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.802+0000 7fc99a0df640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9940ae860 con 0x7fc994145600 2023-10-16T11:06:21.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.802+0000 7fc99a0df640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99406a310 con 0x7fc980004300 2023-10-16T11:06:21.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.802+0000 7fc9998de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9940af590 0x7fc9941421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:21.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.802+0000 7fc9998de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9940af590 0x7fc9941421c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43616/0 (socket says 172.21.15.84:43616) 2023-10-16T11:06:21.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.802+0000 7fc9998de640 1 -- 172.21.15.84:0/2143700733 learned_addr learned my addr 172.21.15.84:0/2143700733 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:21.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.803+0000 7fc9990dd640 1 --2- 172.21.15.84:0/2143700733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc980004300 0x7fc9941450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:21.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.803+0000 7fc9988dc640 1 --2- 172.21.15.84:0/2143700733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc994145600 0x7fc994141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:21.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.803+0000 7fc9998de640 1 -- 172.21.15.84:0/2143700733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc980004300 msgr2=0x7fc9941450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:21.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.803+0000 7fc9998de640 1 --2- 172.21.15.84:0/2143700733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc980004300 0x7fc9941450c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:21.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.803+0000 7fc9998de640 1 -- 172.21.15.84:0/2143700733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc994145600 msgr2=0x7fc994141c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:21.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.803+0000 7fc9998de640 1 --2- 172.21.15.84:0/2143700733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc994145600 0x7fc994141c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:21.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.803+0000 7fc9998de640 1 -- 172.21.15.84:0/2143700733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9940aeab0 con 0x7fc9940af590 2023-10-16T11:06:21.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.803+0000 7fc9998de640 1 --2- 172.21.15.84:0/2143700733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9940af590 0x7fc9941421c0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fc9940c6010 tx=0x7fc98c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:21.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.803+0000 7fc989ffb640 1 -- 172.21.15.84:0/2143700733 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c006d50 con 0x7fc9940af590 2023-10-16T11:06:21.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.803+0000 7fc989ffb640 1 -- 172.21.15.84:0/2143700733 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98c003970 con 0x7fc9940af590 2023-10-16T11:06:21.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.803+0000 7fc989ffb640 1 -- 172.21.15.84:0/2143700733 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c003d60 con 0x7fc9940af590 2023-10-16T11:06:21.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.803+0000 7fc99a0df640 1 -- 172.21.15.84:0/2143700733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9940aed90 con 0x7fc9940af590 2023-10-16T11:06:21.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.804+0000 7fc99a0df640 1 -- 172.21.15.84:0/2143700733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc994069da0 con 0x7fc9940af590 2023-10-16T11:06:21.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.805+0000 7fc99a0df640 1 -- 172.21.15.84:0/2143700733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9940af2b0 con 0x7fc9940af590 2023-10-16T11:06:21.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.806+0000 7fc989ffb640 1 -- 172.21.15.84:0/2143700733 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc98c005070 con 0x7fc9940af590 2023-10-16T11:06:21.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.808+0000 7fc989ffb640 1 --2- 172.21.15.84:0/2143700733 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc97806afc0 0x7fc97806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:21.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.808+0000 7fc989ffb640 1 -- 172.21.15.84:0/2143700733 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5255+0+0 (secure 0 0 0) 0x7fc98c089960 con 0x7fc9940af590 2023-10-16T11:06:21.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.808+0000 7fc9990dd640 1 --2- 172.21.15.84:0/2143700733 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc97806afc0 0x7fc97806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:21.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.809+0000 7fc9990dd640 1 --2- 172.21.15.84:0/2143700733 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc97806afc0 0x7fc97806d480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc990001eb0 tx=0x7fc990001550 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:21.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:21.812+0000 7fc989ffb640 1 -- 172.21.15.84:0/2143700733 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc98c054a20 con 0x7fc9940af590 2023-10-16T11:06:22.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.005+0000 7fc99a0df640 1 -- 172.21.15.84:0/2143700733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc9940af2b0 con 0x7fc9940af590 2023-10-16T11:06:22.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.566+0000 7fc989ffb640 1 -- 172.21.15.84:0/2143700733 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v182) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc9940af2b0 con 0x7fc9940af590 2023-10-16T11:06:22.568 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-16T11:06:22.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.569+0000 7fc99a0df640 1 -- 172.21.15.84:0/2143700733 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc97806afc0 msgr2=0x7fc97806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:22.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.569+0000 7fc99a0df640 1 --2- 172.21.15.84:0/2143700733 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc97806afc0 0x7fc97806d480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc990001eb0 tx=0x7fc990001550 comp rx=0 tx=0).stop 2023-10-16T11:06:22.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.569+0000 7fc99a0df640 1 -- 172.21.15.84:0/2143700733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9940af590 msgr2=0x7fc9941421c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:22.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.569+0000 7fc99a0df640 1 --2- 172.21.15.84:0/2143700733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9940af590 0x7fc9941421c0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fc9940c6010 tx=0x7fc98c00dc00 comp rx=0 tx=0).stop 2023-10-16T11:06:22.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.570+0000 7fc99a0df640 1 -- 172.21.15.84:0/2143700733 shutdown_connections 2023-10-16T11:06:22.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.570+0000 7fc99a0df640 1 --2- 172.21.15.84:0/2143700733 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fc97806afc0 0x7fc97806d480 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:22.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.570+0000 7fc99a0df640 1 --2- 172.21.15.84:0/2143700733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9940af590 0x7fc9941421c0 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:22.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.570+0000 7fc99a0df640 1 --2- 172.21.15.84:0/2143700733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc994145600 0x7fc994141c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:22.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.570+0000 7fc99a0df640 1 --2- 172.21.15.84:0/2143700733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc980004300 0x7fc9941450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:22.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.570+0000 7fc99a0df640 1 -- 172.21.15.84:0/2143700733 >> 172.21.15.84:0/2143700733 conn(0x7fc9940b0a80 msgr2=0x7fc99415a980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:22.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.570+0000 7fc99a0df640 1 -- 172.21.15.84:0/2143700733 shutdown_connections 2023-10-16T11:06:22.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.570+0000 7fc99a0df640 1 -- 172.21.15.84:0/2143700733 wait complete. 2023-10-16T11:06:22.605 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T11:06:22.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.790+0000 7f56ac9ac640 1 Processor -- start 2023-10-16T11:06:22.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.790+0000 7f56ac9ac640 1 -- start start 2023-10-16T11:06:22.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.792+0000 7f56ac9ac640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56a806c040 0x7f56a806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:22.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.792+0000 7f56ac9ac640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f56a80cda80 0x7f56a815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:22.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.792+0000 7f56ac9ac640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56a806c960 con 0x7f56a80ad970 2023-10-16T11:06:22.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.792+0000 7f56ac9ac640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56a815b9d0 con 0x7f56a80cda80 2023-10-16T11:06:22.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.792+0000 7f56ac9ac640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a815bcb0 con 0x7f56a806c040 2023-10-16T11:06:22.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.792+0000 7f56a7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f56a80cda80 0x7f56a815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:22.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.792+0000 7f56a7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f56a80cda80 0x7f56a815ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43620/0 (socket says 172.21.15.84:43620) 2023-10-16T11:06:22.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.792+0000 7f56a7fff640 1 -- 172.21.15.84:0/1931549844 learned_addr learned my addr 172.21.15.84:0/1931549844 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:22.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.792+0000 7f56a77fe640 1 --2- 172.21.15.84:0/1931549844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56a806c040 0x7f56a806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:22.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.793+0000 7f56a7fff640 1 -- 172.21.15.84:0/1931549844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56a806c040 msgr2=0x7f56a806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:22.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.793+0000 7f56a7fff640 1 --2- 172.21.15.84:0/1931549844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56a806c040 0x7f56a806c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:22.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.793+0000 7f56a7fff640 1 -- 172.21.15.84:0/1931549844 >> v1:172.21.15.157:6789/0 conn(0x7f56a80ad970 legacy=0x7f56a815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:22.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.793+0000 7f56a7fff640 1 -- 172.21.15.84:0/1931549844 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a815bf90 con 0x7f56a80cda80 2023-10-16T11:06:22.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.794+0000 7f56a7fff640 1 --2- 172.21.15.84:0/1931549844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a80cda80 0x7f56a815ed90 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009e50 tx=0x7f56a002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa59c6d6aeb5f96e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:22.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.794+0000 7f56a6ffd640 1 -- 172.21.15.84:0/1931549844 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a002ca70 con 0x7f56a80cda80 2023-10-16T11:06:22.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.794+0000 7f56a6ffd640 1 -- 172.21.15.84:0/1931549844 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a002cbd0 con 0x7f56a80cda80 2023-10-16T11:06:22.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.794+0000 7f56a6ffd640 1 -- 172.21.15.84:0/1931549844 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a00343f0 con 0x7f56a80cda80 2023-10-16T11:06:22.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.795+0000 7f56ac9ac640 1 -- 172.21.15.84:0/1931549844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a80cda80 msgr2=0x7f56a815ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:22.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.795+0000 7f56ac9ac640 1 --2- 172.21.15.84:0/1931549844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a80cda80 0x7f56a815ed90 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009e50 tx=0x7f56a002bed0 comp rx=0 tx=0).stop 2023-10-16T11:06:22.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.795+0000 7f56ac9ac640 1 -- 172.21.15.84:0/1931549844 shutdown_connections 2023-10-16T11:06:22.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.795+0000 7f56ac9ac640 1 --2- 172.21.15.84:0/1931549844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a80cda80 0x7f56a815ed90 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:22.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.795+0000 7f56ac9ac640 1 --2- 172.21.15.84:0/1931549844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56a806c040 0x7f56a806c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:22.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.795+0000 7f56ac9ac640 1 -- 172.21.15.84:0/1931549844 >> 172.21.15.84:0/1931549844 conn(0x7f56a80ce6a0 msgr2=0x7f56a80ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:22.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.795+0000 7f56ac9ac640 1 -- 172.21.15.84:0/1931549844 shutdown_connections 2023-10-16T11:06:22.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.795+0000 7f56ac9ac640 1 -- 172.21.15.84:0/1931549844 wait complete. 2023-10-16T11:06:22.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.797+0000 7f56ac9ac640 1 Processor -- start 2023-10-16T11:06:22.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.797+0000 7f56ac9ac640 1 -- start start 2023-10-16T11:06:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.798+0000 7f56ac9ac640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56a806c040 0x7f56a8157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.798+0000 7f56ac9ac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a80ad970 0x7f56a8154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.798+0000 7f56ac9ac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a80cda80 0x7f56a8154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.798+0000 7f56ac9ac640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a81f0e90 con 0x7f56a80cda80 2023-10-16T11:06:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.798+0000 7f56ac9ac640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a8155230 con 0x7f56a80ad970 2023-10-16T11:06:22.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.798+0000 7f56ac9ac640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a81f04f0 con 0x7f56a806c040 2023-10-16T11:06:22.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.798+0000 7f56a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a80cda80 0x7f56a8154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:22.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.798+0000 7f56a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a80cda80 0x7f56a8154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43626/0 (socket says 172.21.15.84:43626) 2023-10-16T11:06:22.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.798+0000 7f56a7fff640 1 -- 172.21.15.84:0/4031484670 learned_addr learned my addr 172.21.15.84:0/4031484670 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:22.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.798+0000 7f56a7fff640 1 -- 172.21.15.84:0/4031484670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56a806c040 msgr2=0x7f56a8157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.798+0000 7f56a77fe640 1 --2- 172.21.15.84:0/4031484670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56a806c040 0x7f56a8157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.799+0000 7f569ffff640 1 --2- 172.21.15.84:0/4031484670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a80ad970 0x7f56a8154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.799+0000 7f56a7fff640 1 --2- 172.21.15.84:0/4031484670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56a806c040 0x7f56a8157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.799+0000 7f56a7fff640 1 -- 172.21.15.84:0/4031484670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a80ad970 msgr2=0x7f56a8154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.799+0000 7f56a7fff640 1 --2- 172.21.15.84:0/4031484670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a80ad970 0x7f56a8154690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:22.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.799+0000 7f56a7fff640 1 -- 172.21.15.84:0/4031484670 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a0009b30 con 0x7f56a80cda80 2023-10-16T11:06:22.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.799+0000 7f56a7fff640 1 --2- 172.21.15.84:0/4031484670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a80cda80 0x7f56a8154cf0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009e50 tx=0x7f56a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:22.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.799+0000 7f56a4ff9640 1 -- 172.21.15.84:0/4031484670 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0040670 con 0x7f56a80cda80 2023-10-16T11:06:22.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.799+0000 7f56a4ff9640 1 -- 172.21.15.84:0/4031484670 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a00407f0 con 0x7f56a80cda80 2023-10-16T11:06:22.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.799+0000 7f56a4ff9640 1 -- 172.21.15.84:0/4031484670 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0040ba0 con 0x7f56a80cda80 2023-10-16T11:06:22.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.799+0000 7f56ac9ac640 1 -- 172.21.15.84:0/4031484670 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a81f07d0 con 0x7f56a80cda80 2023-10-16T11:06:22.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.799+0000 7f56ac9ac640 1 -- 172.21.15.84:0/4031484670 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a81f12b0 con 0x7f56a80cda80 2023-10-16T11:06:22.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.801+0000 7f56ac9ac640 1 -- 172.21.15.84:0/4031484670 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a806b330 con 0x7f56a80cda80 2023-10-16T11:06:22.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.802+0000 7f56a4ff9640 1 -- 172.21.15.84:0/4031484670 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f56a0040d00 con 0x7f56a80cda80 2023-10-16T11:06:22.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.803+0000 7f56a4ff9640 1 --2- 172.21.15.84:0/4031484670 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f568806af90 0x7f568806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:22.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.803+0000 7f56a4ff9640 1 -- 172.21.15.84:0/4031484670 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5637+0+0 (secure 0 0 0) 0x7f56a00acb40 con 0x7f56a80cda80 2023-10-16T11:06:22.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.803+0000 7f56a77fe640 1 --2- 172.21.15.84:0/4031484670 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f568806af90 0x7f568806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:22.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.804+0000 7f56a77fe640 1 --2- 172.21.15.84:0/4031484670 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f568806af90 0x7f568806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5698006fd0 tx=0x7f5698008040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:22.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:22.807+0000 7f56a4ff9640 1 -- 172.21.15.84:0/4031484670 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f56a0077c90 con 0x7f56a80cda80 2023-10-16T11:06:23.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.032+0000 7f56ac9ac640 1 -- 172.21.15.84:0/4031484670 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f56a81f0fd0 con 0x7f56a80cda80 2023-10-16T11:06:23.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.588+0000 7f56a4ff9640 1 -- 172.21.15.84:0/4031484670 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v93) v1 ==== 174+0+0 (secure 0 0 0) 0x7f56a0009510 con 0x7f56a80cda80 2023-10-16T11:06:23.592 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 22 and data pool 23 2023-10-16T11:06:23.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.594+0000 7f56ac9ac640 1 -- 172.21.15.84:0/4031484670 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f568806af90 msgr2=0x7f568806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:23.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.594+0000 7f56ac9ac640 1 --2- 172.21.15.84:0/4031484670 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f568806af90 0x7f568806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5698006fd0 tx=0x7f5698008040 comp rx=0 tx=0).stop 2023-10-16T11:06:23.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.594+0000 7f56ac9ac640 1 -- 172.21.15.84:0/4031484670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a80cda80 msgr2=0x7f56a8154cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:23.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.594+0000 7f56ac9ac640 1 --2- 172.21.15.84:0/4031484670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a80cda80 0x7f56a8154cf0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009e50 tx=0x7f56a0004a40 comp rx=0 tx=0).stop 2023-10-16T11:06:23.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.595+0000 7f56ac9ac640 1 -- 172.21.15.84:0/4031484670 shutdown_connections 2023-10-16T11:06:23.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.595+0000 7f56ac9ac640 1 --2- 172.21.15.84:0/4031484670 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f568806af90 0x7f568806d450 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5698006fd0 tx=0x7f5698008040 comp rx=0 tx=0).stop 2023-10-16T11:06:23.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.595+0000 7f56ac9ac640 1 --2- 172.21.15.84:0/4031484670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a80cda80 0x7f56a8154cf0 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:23.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.595+0000 7f56ac9ac640 1 --2- 172.21.15.84:0/4031484670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a80ad970 0x7f56a8154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:23.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.595+0000 7f56ac9ac640 1 --2- 172.21.15.84:0/4031484670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56a806c040 0x7f56a8157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:23.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.595+0000 7f56ac9ac640 1 -- 172.21.15.84:0/4031484670 >> 172.21.15.84:0/4031484670 conn(0x7f56a80ce6a0 msgr2=0x7f56a815da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:23.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.598+0000 7f56ac9ac640 1 -- 172.21.15.84:0/4031484670 shutdown_connections 2023-10-16T11:06:23.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.598+0000 7f56ac9ac640 1 -- 172.21.15.84:0/4031484670 wait complete. 2023-10-16T11:06:23.633 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:06:23.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.812+0000 7fb582a61640 1 Processor -- start 2023-10-16T11:06:23.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.812+0000 7fb582a61640 1 -- start start 2023-10-16T11:06:23.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.813+0000 7fb582a61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57c15aaa0 0x7fb57c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:23.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.813+0000 7fb582a61640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb57c0ce390 0x7fb57c070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:23.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.813+0000 7fb582a61640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb57c15ed60 con 0x7fb57c15ae80 2023-10-16T11:06:23.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.813+0000 7fb582a61640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb57c06b810 con 0x7fb57c0ce390 2023-10-16T11:06:23.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.813+0000 7fb582a61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c0707e0 con 0x7fb57c15aaa0 2023-10-16T11:06:23.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.814+0000 7fb580ffe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb57c0ce390 0x7fb57c070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:23.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.814+0000 7fb580ffe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb57c0ce390 0x7fb57c070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43632/0 (socket says 172.21.15.84:43632) 2023-10-16T11:06:23.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.814+0000 7fb580ffe640 1 -- 172.21.15.84:0/3276350663 learned_addr learned my addr 172.21.15.84:0/3276350663 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:23.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.814+0000 7fb580ffe640 1 -- 172.21.15.84:0/3276350663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57c15aaa0 msgr2=0x7fb57c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:23.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.814+0000 7fb580ffe640 1 --2- 172.21.15.84:0/3276350663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57c15aaa0 0x7fb57c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:23.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.814+0000 7fb580ffe640 1 -- 172.21.15.84:0/3276350663 >> v1:172.21.15.157:6789/0 conn(0x7fb57c15ae80 legacy=0x7fb57c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:23.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.814+0000 7fb580ffe640 1 -- 172.21.15.84:0/3276350663 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57c15ef90 con 0x7fb57c0ce390 2023-10-16T11:06:23.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.815+0000 7fb580ffe640 1 --2- 172.21.15.84:0/3276350663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb57c0ce390 0x7fb57c070f50 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7fb574009e50 tx=0x7fb57402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab75f1b46ac9f8be server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:23.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.816+0000 7fb57affd640 1 -- 172.21.15.84:0/3276350663 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57402ca80 con 0x7fb57c0ce390 2023-10-16T11:06:23.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.816+0000 7fb57affd640 1 -- 172.21.15.84:0/3276350663 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57402cbe0 con 0x7fb57c0ce390 2023-10-16T11:06:23.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.816+0000 7fb57affd640 1 -- 172.21.15.84:0/3276350663 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5740343f0 con 0x7fb57c0ce390 2023-10-16T11:06:23.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.816+0000 7fb582a61640 1 -- 172.21.15.84:0/3276350663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb57c0ce390 msgr2=0x7fb57c070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:23.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.816+0000 7fb582a61640 1 --2- 172.21.15.84:0/3276350663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb57c0ce390 0x7fb57c070f50 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7fb574009e50 tx=0x7fb57402bee0 comp rx=0 tx=0).stop 2023-10-16T11:06:23.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.816+0000 7fb582a61640 1 -- 172.21.15.84:0/3276350663 shutdown_connections 2023-10-16T11:06:23.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.816+0000 7fb582a61640 1 --2- 172.21.15.84:0/3276350663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb57c0ce390 0x7fb57c070f50 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:23.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.816+0000 7fb582a61640 1 --2- 172.21.15.84:0/3276350663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57c15aaa0 0x7fb57c15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:23.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.816+0000 7fb582a61640 1 -- 172.21.15.84:0/3276350663 >> 172.21.15.84:0/3276350663 conn(0x7fb57c0cb410 msgr2=0x7fb57c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:23.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.817+0000 7fb582a61640 1 -- 172.21.15.84:0/3276350663 shutdown_connections 2023-10-16T11:06:23.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.817+0000 7fb582a61640 1 -- 172.21.15.84:0/3276350663 wait complete. 2023-10-16T11:06:23.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.818+0000 7fb582a61640 1 Processor -- start 2023-10-16T11:06:23.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.819+0000 7fb582a61640 1 -- start start 2023-10-16T11:06:23.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.820+0000 7fb582a61640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb57c0ce390 0x7fb57c1585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.820+0000 7fb582a61640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb57c15aaa0 0x7fb57c158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.820+0000 7fb582a61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57c15ae80 0x7fb57c0c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.820+0000 7fb582a61640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c0c3fc0 con 0x7fb57c15aaa0 2023-10-16T11:06:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.820+0000 7fb582a61640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c15e200 con 0x7fb57c0ce390 2023-10-16T11:06:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.820+0000 7fb582a61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c1f2ae0 con 0x7fb57c15ae80 2023-10-16T11:06:23.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.820+0000 7fb57b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb57c15aaa0 0x7fb57c158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:23.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.820+0000 7fb57b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb57c15aaa0 0x7fb57c158b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43644/0 (socket says 172.21.15.84:43644) 2023-10-16T11:06:23.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.820+0000 7fb57b7fe640 1 -- 172.21.15.84:0/1731717493 learned_addr learned my addr 172.21.15.84:0/1731717493 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:23.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.820+0000 7fb57bfff640 1 --2- 172.21.15.84:0/1731717493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb57c0ce390 0x7fb57c1585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:23.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.820+0000 7fb57b7fe640 1 -- 172.21.15.84:0/1731717493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57c15ae80 msgr2=0x7fb57c0c43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.820+0000 7fb57b7fe640 1 --2- 172.21.15.84:0/1731717493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57c15ae80 0x7fb57c0c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.820+0000 7fb57b7fe640 1 -- 172.21.15.84:0/1731717493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb57c0ce390 msgr2=0x7fb57c1585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.820+0000 7fb57b7fe640 1 --2- 172.21.15.84:0/1731717493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb57c0ce390 0x7fb57c1585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.820+0000 7fb57b7fe640 1 -- 172.21.15.84:0/1731717493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb574009b30 con 0x7fb57c15aaa0 2023-10-16T11:06:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.821+0000 7fb57b7fe640 1 --2- 172.21.15.84:0/1731717493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb57c15aaa0 0x7fb57c158b20 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fb57000c410 tx=0x7fb57000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.821+0000 7fb578ff9640 1 -- 172.21.15.84:0/1731717493 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5700035f0 con 0x7fb57c15aaa0 2023-10-16T11:06:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.821+0000 7fb578ff9640 1 -- 172.21.15.84:0/1731717493 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb570003750 con 0x7fb57c15aaa0 2023-10-16T11:06:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.821+0000 7fb582a61640 1 -- 172.21.15.84:0/1731717493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb57c0c3620 con 0x7fb57c15aaa0 2023-10-16T11:06:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.821+0000 7fb578ff9640 1 -- 172.21.15.84:0/1731717493 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb570003b00 con 0x7fb57c15aaa0 2023-10-16T11:06:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.821+0000 7fb582a61640 1 -- 172.21.15.84:0/1731717493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb57c0c3be0 con 0x7fb57c15aaa0 2023-10-16T11:06:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.822+0000 7fb582a61640 1 -- 172.21.15.84:0/1731717493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb57c0c4100 con 0x7fb57c15aaa0 2023-10-16T11:06:23.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.823+0000 7fb578ff9640 1 -- 172.21.15.84:0/1731717493 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb570003c60 con 0x7fb57c15aaa0 2023-10-16T11:06:23.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.824+0000 7fb578ff9640 1 --2- 172.21.15.84:0/1731717493 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fb56806af90 0x7fb56806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:23.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.824+0000 7fb578ff9640 1 -- 172.21.15.84:0/1731717493 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb570088ef0 con 0x7fb57c15aaa0 2023-10-16T11:06:23.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.825+0000 7fb57bfff640 1 --2- 172.21.15.84:0/1731717493 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fb56806af90 0x7fb56806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:23.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.825+0000 7fb57bfff640 1 --2- 172.21.15.84:0/1731717493 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fb56806af90 0x7fb56806d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009890 tx=0x7fb56c008040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:23.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:23.829+0000 7fb578ff9640 1 -- 172.21.15.84:0/1731717493 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb57c0c4100 con 0x7fb57c15aaa0 2023-10-16T11:06:24.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.012+0000 7fb582a61640 1 -- 172.21.15.84:0/1731717493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb57c0c4100 con 0x7fb57c15aaa0 2023-10-16T11:06:24.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.014+0000 7fb578ff9640 1 -- 172.21.15.84:0/1731717493 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+16604 (secure 0 0 0) 0x7fb5700860f0 con 0x7fb57c15aaa0 2023-10-16T11:06:24.016 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:24.016 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":183,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:06:23.034022+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:06:20.873257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-16T11:06:22.065944+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:06:24.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.016+0000 7fb582a61640 1 -- 172.21.15.84:0/1731717493 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fb56806af90 msgr2=0x7fb56806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:24.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.016+0000 7fb582a61640 1 --2- 172.21.15.84:0/1731717493 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fb56806af90 0x7fb56806d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009890 tx=0x7fb56c008040 comp rx=0 tx=0).stop 2023-10-16T11:06:24.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.016+0000 7fb582a61640 1 -- 172.21.15.84:0/1731717493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb57c15aaa0 msgr2=0x7fb57c158b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:24.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.016+0000 7fb582a61640 1 --2- 172.21.15.84:0/1731717493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb57c15aaa0 0x7fb57c158b20 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fb57000c410 tx=0x7fb57000c8d0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.016+0000 7fb582a61640 1 -- 172.21.15.84:0/1731717493 shutdown_connections 2023-10-16T11:06:24.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.016+0000 7fb582a61640 1 --2- 172.21.15.84:0/1731717493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb57c15ae80 0x7fb57c0c43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.016+0000 7fb582a61640 1 --2- 172.21.15.84:0/1731717493 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fb56806af90 0x7fb56806d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.016+0000 7fb582a61640 1 --2- 172.21.15.84:0/1731717493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb57c15aaa0 0x7fb57c158b20 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.016+0000 7fb582a61640 1 --2- 172.21.15.84:0/1731717493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb57c0ce390 0x7fb57c1585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.016+0000 7fb582a61640 1 -- 172.21.15.84:0/1731717493 >> 172.21.15.84:0/1731717493 conn(0x7fb57c0cb410 msgr2=0x7fb57c0acaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:24.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.017+0000 7fb582a61640 1 -- 172.21.15.84:0/1731717493 shutdown_connections 2023-10-16T11:06:24.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.017+0000 7fb582a61640 1 -- 172.21.15.84:0/1731717493 wait complete. 2023-10-16T11:06:24.036 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:06:24.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.237+0000 7fce4ca90640 1 Processor -- start 2023-10-16T11:06:24.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.237+0000 7fce4ca90640 1 -- start start 2023-10-16T11:06:24.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.238+0000 7fce4ca90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce48159b20 0x7fce4806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:24.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.238+0000 7fce4ca90640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fce480b0b20 0x7fce4806fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:24.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.238+0000 7fce4ca90640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4806f7f0 con 0x7fce48159f00 2023-10-16T11:06:24.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.238+0000 7fce4ca90640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce4806c5e0 con 0x7fce480b0b20 2023-10-16T11:06:24.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.238+0000 7fce4ca90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4806c8c0 con 0x7fce48159b20 2023-10-16T11:06:24.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.238+0000 7fce46d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fce480b0b20 0x7fce4806fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:24.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.239+0000 7fce46d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fce480b0b20 0x7fce4806fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43658/0 (socket says 172.21.15.84:43658) 2023-10-16T11:06:24.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.239+0000 7fce46d9d640 1 -- 172.21.15.84:0/3345062043 learned_addr learned my addr 172.21.15.84:0/3345062043 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:24.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.239+0000 7fce4659c640 1 --2- 172.21.15.84:0/3345062043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce48159b20 0x7fce4806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:24.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.239+0000 7fce46d9d640 1 -- 172.21.15.84:0/3345062043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce48159b20 msgr2=0x7fce4806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:24.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.239+0000 7fce46d9d640 1 --2- 172.21.15.84:0/3345062043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce48159b20 0x7fce4806bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.239+0000 7fce46d9d640 1 -- 172.21.15.84:0/3345062043 >> v1:172.21.15.157:6789/0 conn(0x7fce48159f00 legacy=0x7fce480b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:24.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.239+0000 7fce46d9d640 1 -- 172.21.15.84:0/3345062043 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4806cba0 con 0x7fce480b0b20 2023-10-16T11:06:24.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.240+0000 7fce46d9d640 1 --2- 172.21.15.84:0/3345062043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce480b0b20 0x7fce4806fc80 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fce40009e50 tx=0x7fce4002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37eba961c1f6641 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:24.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.240+0000 7fce4559a640 1 -- 172.21.15.84:0/3345062043 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4002ca70 con 0x7fce480b0b20 2023-10-16T11:06:24.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.240+0000 7fce4559a640 1 -- 172.21.15.84:0/3345062043 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4002cbd0 con 0x7fce480b0b20 2023-10-16T11:06:24.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.240+0000 7fce4559a640 1 -- 172.21.15.84:0/3345062043 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce400343f0 con 0x7fce480b0b20 2023-10-16T11:06:24.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.241+0000 7fce4ca90640 1 -- 172.21.15.84:0/3345062043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce480b0b20 msgr2=0x7fce4806fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:24.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.241+0000 7fce4ca90640 1 --2- 172.21.15.84:0/3345062043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce480b0b20 0x7fce4806fc80 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fce40009e50 tx=0x7fce4002bed0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.241+0000 7fce4ca90640 1 -- 172.21.15.84:0/3345062043 shutdown_connections 2023-10-16T11:06:24.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.241+0000 7fce4ca90640 1 --2- 172.21.15.84:0/3345062043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce480b0b20 0x7fce4806fc80 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.241+0000 7fce4ca90640 1 --2- 172.21.15.84:0/3345062043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce48159b20 0x7fce4806bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.241+0000 7fce4ca90640 1 -- 172.21.15.84:0/3345062043 >> 172.21.15.84:0/3345062043 conn(0x7fce480cb410 msgr2=0x7fce480cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:24.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.241+0000 7fce4ca90640 1 -- 172.21.15.84:0/3345062043 shutdown_connections 2023-10-16T11:06:24.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.241+0000 7fce4ca90640 1 -- 172.21.15.84:0/3345062043 wait complete. 2023-10-16T11:06:24.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.243+0000 7fce4ca90640 1 Processor -- start 2023-10-16T11:06:24.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.243+0000 7fce4ca90640 1 -- start start 2023-10-16T11:06:24.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.244+0000 7fce4ca90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce480b0b20 0x7fce481487d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:24.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.244+0000 7fce4ca90640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce48159b20 0x7fce480c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:24.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.244+0000 7fce4ca90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce48159f00 0x7fce48149130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:24.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.244+0000 7fce4ca90640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce480c48a0 con 0x7fce48159b20 2023-10-16T11:06:24.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.244+0000 7fce4ca90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce480c3580 con 0x7fce480b0b20 2023-10-16T11:06:24.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.244+0000 7fce4ca90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce480c4220 con 0x7fce48159f00 2023-10-16T11:06:24.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.244+0000 7fce45d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce48159b20 0x7fce480c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:24.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.244+0000 7fce45d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce48159b20 0x7fce480c4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43670/0 (socket says 172.21.15.84:43670) 2023-10-16T11:06:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.244+0000 7fce45d9b640 1 -- 172.21.15.84:0/2544588983 learned_addr learned my addr 172.21.15.84:0/2544588983 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.244+0000 7fce4659c640 1 --2- 172.21.15.84:0/2544588983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce480b0b20 0x7fce481487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.244+0000 7fce46d9d640 1 --2- 172.21.15.84:0/2544588983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce48159f00 0x7fce48149130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.245+0000 7fce45d9b640 1 -- 172.21.15.84:0/2544588983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce48159f00 msgr2=0x7fce48149130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:24.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.245+0000 7fce45d9b640 1 --2- 172.21.15.84:0/2544588983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce48159f00 0x7fce48149130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.245+0000 7fce45d9b640 1 -- 172.21.15.84:0/2544588983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce480b0b20 msgr2=0x7fce481487d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:24.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.245+0000 7fce45d9b640 1 --2- 172.21.15.84:0/2544588983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce480b0b20 0x7fce481487d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.245+0000 7fce45d9b640 1 -- 172.21.15.84:0/2544588983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce40009b30 con 0x7fce48159b20 2023-10-16T11:06:24.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.245+0000 7fce45d9b640 1 --2- 172.21.15.84:0/2544588983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce48159b20 0x7fce480c4cc0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00c430 tx=0x7fce3c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:24.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.245+0000 7fce36ffd640 1 -- 172.21.15.84:0/2544588983 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c011790 con 0x7fce48159b20 2023-10-16T11:06:24.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.245+0000 7fce36ffd640 1 -- 172.21.15.84:0/2544588983 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c0118f0 con 0x7fce48159b20 2023-10-16T11:06:24.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.245+0000 7fce36ffd640 1 -- 172.21.15.84:0/2544588983 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c011ca0 con 0x7fce48159b20 2023-10-16T11:06:24.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.245+0000 7fce4ca90640 1 -- 172.21.15.84:0/2544588983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce480c32a0 con 0x7fce48159b20 2023-10-16T11:06:24.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.245+0000 7fce4ca90640 1 -- 172.21.15.84:0/2544588983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce48148d10 con 0x7fce48159b20 2023-10-16T11:06:24.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.247+0000 7fce4ca90640 1 -- 172.21.15.84:0/2544588983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce48148ea0 con 0x7fce48159b20 2023-10-16T11:06:24.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.247+0000 7fce36ffd640 1 -- 172.21.15.84:0/2544588983 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fce3c01c460 con 0x7fce48159b20 2023-10-16T11:06:24.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.248+0000 7fce36ffd640 1 --2- 172.21.15.84:0/2544588983 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fce2806af90 0x7fce2806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:24.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.248+0000 7fce36ffd640 1 -- 172.21.15.84:0/2544588983 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fce3c0896d0 con 0x7fce48159b20 2023-10-16T11:06:24.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.249+0000 7fce4659c640 1 --2- 172.21.15.84:0/2544588983 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fce2806af90 0x7fce2806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:24.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.249+0000 7fce4659c640 1 --2- 172.21.15.84:0/2544588983 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fce2806af90 0x7fce2806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fce38006fd0 tx=0x7fce38008040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:24.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.253+0000 7fce36ffd640 1 -- 172.21.15.84:0/2544588983 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fce3c0546e0 con 0x7fce48159b20 2023-10-16T11:06:24.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.432+0000 7fce4ca90640 1 -- 172.21.15.84:0/2544588983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce480c4080 con 0x7fce48159b20 2023-10-16T11:06:24.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.435+0000 7fce36ffd640 1 -- 172.21.15.84:0/2544588983 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+16604 (secure 0 0 0) 0x7fce480c4080 con 0x7fce48159b20 2023-10-16T11:06:24.437 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:24.437 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":183,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:06:23.034022+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:06:20.873257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-16T11:06:22.065944+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:06:24.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.437+0000 7fce4ca90640 1 -- 172.21.15.84:0/2544588983 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fce2806af90 msgr2=0x7fce2806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:24.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.437+0000 7fce4ca90640 1 --2- 172.21.15.84:0/2544588983 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fce2806af90 0x7fce2806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fce38006fd0 tx=0x7fce38008040 comp rx=0 tx=0).stop 2023-10-16T11:06:24.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.437+0000 7fce4ca90640 1 -- 172.21.15.84:0/2544588983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce48159b20 msgr2=0x7fce480c4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:24.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.437+0000 7fce4ca90640 1 --2- 172.21.15.84:0/2544588983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce48159b20 0x7fce480c4cc0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00c430 tx=0x7fce3c00c8f0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.438+0000 7fce4ca90640 1 -- 172.21.15.84:0/2544588983 shutdown_connections 2023-10-16T11:06:24.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.438+0000 7fce4ca90640 1 --2- 172.21.15.84:0/2544588983 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce48159f00 0x7fce48149130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.438+0000 7fce4ca90640 1 --2- 172.21.15.84:0/2544588983 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fce2806af90 0x7fce2806d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.438+0000 7fce4ca90640 1 --2- 172.21.15.84:0/2544588983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce48159b20 0x7fce480c4cc0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.438+0000 7fce4ca90640 1 --2- 172.21.15.84:0/2544588983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce480b0b20 0x7fce481487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.438+0000 7fce4ca90640 1 -- 172.21.15.84:0/2544588983 >> 172.21.15.84:0/2544588983 conn(0x7fce480cb410 msgr2=0x7fce4815d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:24.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.438+0000 7fce4ca90640 1 -- 172.21.15.84:0/2544588983 shutdown_connections 2023-10-16T11:06:24.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.438+0000 7fce4ca90640 1 -- 172.21.15.84:0/2544588983 wait complete. 2023-10-16T11:06:24.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T11:06:24.529 INFO:tasks.ceph.mds.c.smithi084.stderr:2023-10-16T11:06:24.527+0000 7faed3cdc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:06:24.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.676+0000 7fed87844640 1 Processor -- start 2023-10-16T11:06:24.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.677+0000 7fed87844640 1 -- start start 2023-10-16T11:06:24.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.678+0000 7fed87844640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fed8015ad70 0x7fed80069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:24.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.678+0000 7fed87844640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fed800caac0 0x7fed8006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:24.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.678+0000 7fed87844640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed800cb030 con 0x7fed800c9ca0 2023-10-16T11:06:24.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.678+0000 7fed87844640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed800c91c0 con 0x7fed8015ad70 2023-10-16T11:06:24.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.678+0000 7fed87844640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed800c94a0 con 0x7fed800caac0 2023-10-16T11:06:24.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.679+0000 7fed855e0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fed8015ad70 0x7fed80069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:24.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.679+0000 7fed855e0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fed8015ad70 0x7fed80069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43682/0 (socket says 172.21.15.84:43682) 2023-10-16T11:06:24.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.679+0000 7fed855e0640 1 -- 172.21.15.84:0/1982577570 learned_addr learned my addr 172.21.15.84:0/1982577570 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:24.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.679+0000 7fed85de1640 1 --2- 172.21.15.84:0/1982577570 >> v2:172.21.15.157:3300/0 conn(0x7fed800caac0 0x7fed8006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:24.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.679+0000 7fed855e0640 1 -- 172.21.15.84:0/1982577570 >> v2:172.21.15.157:3300/0 conn(0x7fed800caac0 msgr2=0x7fed8006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:24.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.679+0000 7fed855e0640 1 --2- 172.21.15.84:0/1982577570 >> v2:172.21.15.157:3300/0 conn(0x7fed800caac0 0x7fed8006a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.679+0000 7fed855e0640 1 -- 172.21.15.84:0/1982577570 >> v1:172.21.15.84:6789/0 conn(0x7fed800c9ca0 legacy=0x7fed800ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:24.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.679+0000 7fed7ffff640 1 -- 172.21.15.84:0/1982577570 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125368528 0 0) 0x7fed800cb030 con 0x7fed800c9ca0 2023-10-16T11:06:24.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.679+0000 7fed855e0640 1 -- 172.21.15.84:0/1982577570 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed800c9780 con 0x7fed8015ad70 2023-10-16T11:06:24.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.681+0000 7fed855e0640 1 --2- 172.21.15.84:0/1982577570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed8015ad70 0x7fed80069da0 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7fed8006d090 tx=0x7fed7002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677d476e05c5a232 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:24.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.681+0000 7fed7ffff640 1 -- 172.21.15.84:0/1982577570 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7002cb20 con 0x7fed8015ad70 2023-10-16T11:06:24.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.681+0000 7fed7ffff640 1 -- 172.21.15.84:0/1982577570 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7002cc80 con 0x7fed8015ad70 2023-10-16T11:06:24.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.681+0000 7fed7ffff640 1 -- 172.21.15.84:0/1982577570 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed700343f0 con 0x7fed8015ad70 2023-10-16T11:06:24.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.682+0000 7fed87844640 1 -- 172.21.15.84:0/1982577570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed8015ad70 msgr2=0x7fed80069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:24.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.682+0000 7fed87844640 1 --2- 172.21.15.84:0/1982577570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed8015ad70 0x7fed80069da0 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7fed8006d090 tx=0x7fed7002bf80 comp rx=0 tx=0).stop 2023-10-16T11:06:24.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.682+0000 7fed87844640 1 -- 172.21.15.84:0/1982577570 shutdown_connections 2023-10-16T11:06:24.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.682+0000 7fed87844640 1 --2- 172.21.15.84:0/1982577570 >> v2:172.21.15.157:3300/0 conn(0x7fed800caac0 0x7fed8006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.682+0000 7fed87844640 1 --2- 172.21.15.84:0/1982577570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed8015ad70 0x7fed80069da0 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.682+0000 7fed87844640 1 -- 172.21.15.84:0/1982577570 >> 172.21.15.84:0/1982577570 conn(0x7fed800ce9b0 msgr2=0x7fed800cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:24.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.682+0000 7fed87844640 1 -- 172.21.15.84:0/1982577570 shutdown_connections 2023-10-16T11:06:24.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.682+0000 7fed87844640 1 -- 172.21.15.84:0/1982577570 wait complete. 2023-10-16T11:06:24.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.684+0000 7fed87844640 1 Processor -- start 2023-10-16T11:06:24.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.684+0000 7fed87844640 1 -- start start 2023-10-16T11:06:24.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.685+0000 7fed87844640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed800c9ca0 0x7fed800c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:24.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.685+0000 7fed87844640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed800caac0 0x7fed801f2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:24.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.685+0000 7fed87844640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed8015ad70 0x7fed800c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:24.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.685+0000 7fed87844640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed801f3a30 con 0x7fed800caac0 2023-10-16T11:06:24.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.685+0000 7fed87844640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed801f2b50 con 0x7fed800c9ca0 2023-10-16T11:06:24.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.685+0000 7fed87844640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed800c6730 con 0x7fed8015ad70 2023-10-16T11:06:24.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.685+0000 7fed84ddf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed800caac0 0x7fed801f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:24.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.685+0000 7fed84ddf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed800caac0 0x7fed801f2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43694/0 (socket says 172.21.15.84:43694) 2023-10-16T11:06:24.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.685+0000 7fed84ddf640 1 -- 172.21.15.84:0/2847412630 learned_addr learned my addr 172.21.15.84:0/2847412630 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:24.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.686+0000 7fed855e0640 1 --2- 172.21.15.84:0/2847412630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed800c9ca0 0x7fed800c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:24.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.686+0000 7fed85de1640 1 --2- 172.21.15.84:0/2847412630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed8015ad70 0x7fed800c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:24.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.686+0000 7fed84ddf640 1 -- 172.21.15.84:0/2847412630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed8015ad70 msgr2=0x7fed800c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:24.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.686+0000 7fed84ddf640 1 --2- 172.21.15.84:0/2847412630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed8015ad70 0x7fed800c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.686+0000 7fed84ddf640 1 -- 172.21.15.84:0/2847412630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed800c9ca0 msgr2=0x7fed800c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:24.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.686+0000 7fed84ddf640 1 --2- 172.21.15.84:0/2847412630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed800c9ca0 0x7fed800c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:24.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.686+0000 7fed84ddf640 1 -- 172.21.15.84:0/2847412630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed70009b30 con 0x7fed800caac0 2023-10-16T11:06:24.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.686+0000 7fed855e0640 1 --2- 172.21.15.84:0/2847412630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed800c9ca0 0x7fed800c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:24.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.686+0000 7fed84ddf640 1 --2- 172.21.15.84:0/2847412630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed800caac0 0x7fed801f2610 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fed7400c4b0 tx=0x7fed7400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:24.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.686+0000 7fed7dffb640 1 -- 172.21.15.84:0/2847412630 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed74005480 con 0x7fed800caac0 2023-10-16T11:06:24.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.686+0000 7fed7dffb640 1 -- 172.21.15.84:0/2847412630 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed740035f0 con 0x7fed800caac0 2023-10-16T11:06:24.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.686+0000 7fed7dffb640 1 -- 172.21.15.84:0/2847412630 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed740039a0 con 0x7fed800caac0 2023-10-16T11:06:24.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.686+0000 7fed87844640 1 -- 172.21.15.84:0/2847412630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed801f35e0 con 0x7fed800caac0 2023-10-16T11:06:24.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.687+0000 7fed87844640 1 -- 172.21.15.84:0/2847412630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed801f3020 con 0x7fed800caac0 2023-10-16T11:06:24.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.688+0000 7fed87844640 1 -- 172.21.15.84:0/2847412630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed801f3b70 con 0x7fed800caac0 2023-10-16T11:06:24.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.689+0000 7fed7dffb640 1 -- 172.21.15.84:0/2847412630 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fed74003b00 con 0x7fed800caac0 2023-10-16T11:06:24.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.689+0000 7fed7dffb640 1 --2- 172.21.15.84:0/2847412630 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fed6c06af90 0x7fed6c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:24.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.690+0000 7fed7dffb640 1 -- 172.21.15.84:0/2847412630 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fed74088d10 con 0x7fed800caac0 2023-10-16T11:06:24.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.690+0000 7fed855e0640 1 --2- 172.21.15.84:0/2847412630 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fed6c06af90 0x7fed6c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:24.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.691+0000 7fed855e0640 1 --2- 172.21.15.84:0/2847412630 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fed6c06af90 0x7fed6c06d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fed70035000 tx=0x7fed70030000 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:24.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.695+0000 7fed7dffb640 1 -- 172.21.15.84:0/2847412630 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fed801f3b70 con 0x7fed800caac0 2023-10-16T11:06:24.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:24.926+0000 7fed87844640 1 -- 172.21.15.84:0/2847412630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fed801f3b70 con 0x7fed800caac0 2023-10-16T11:06:25.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.603+0000 7fed7dffb640 1 -- 172.21.15.84:0/2847412630 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v95) v1 ==== 118+0+0 (secure 0 0 0) 0x7fed801f3b70 con 0x7fed800caac0 2023-10-16T11:06:25.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.607+0000 7fed87844640 1 -- 172.21.15.84:0/2847412630 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fed6c06af90 msgr2=0x7fed6c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:25.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.607+0000 7fed87844640 1 --2- 172.21.15.84:0/2847412630 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fed6c06af90 0x7fed6c06d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fed70035000 tx=0x7fed70030000 comp rx=0 tx=0).stop 2023-10-16T11:06:25.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.607+0000 7fed87844640 1 -- 172.21.15.84:0/2847412630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed800caac0 msgr2=0x7fed801f2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:25.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.607+0000 7fed87844640 1 --2- 172.21.15.84:0/2847412630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed800caac0 0x7fed801f2610 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fed7400c4b0 tx=0x7fed7400c970 comp rx=0 tx=0).stop 2023-10-16T11:06:25.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.607+0000 7fed87844640 1 -- 172.21.15.84:0/2847412630 shutdown_connections 2023-10-16T11:06:25.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.607+0000 7fed87844640 1 --2- 172.21.15.84:0/2847412630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed8015ad70 0x7fed800c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:25.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.607+0000 7fed87844640 1 --2- 172.21.15.84:0/2847412630 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fed6c06af90 0x7fed6c06d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:25.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.607+0000 7fed87844640 1 --2- 172.21.15.84:0/2847412630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed800caac0 0x7fed801f2610 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:25.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.607+0000 7fed87844640 1 --2- 172.21.15.84:0/2847412630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed800c9ca0 0x7fed800c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:25.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.607+0000 7fed87844640 1 -- 172.21.15.84:0/2847412630 >> 172.21.15.84:0/2847412630 conn(0x7fed800ce9b0 msgr2=0x7fed80069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:25.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.608+0000 7fed87844640 1 -- 172.21.15.84:0/2847412630 shutdown_connections 2023-10-16T11:06:25.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.608+0000 7fed87844640 1 -- 172.21.15.84:0/2847412630 wait complete. 2023-10-16T11:06:25.631 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T11:06:25.632 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T11:06:25.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.852+0000 7faef8fc9640 1 Processor -- start 2023-10-16T11:06:25.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.852+0000 7faef8fc9640 1 -- start start 2023-10-16T11:06:25.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.853+0000 7faef8fc9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faef4159ab0 0x7faef40ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:25.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.853+0000 7faef8fc9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faef406c140 0x7faef406c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:25.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.853+0000 7faef8fc9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef406cad0 con 0x7faef4159e90 2023-10-16T11:06:25.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.853+0000 7faef8fc9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef415d890 con 0x7faef406c140 2023-10-16T11:06:25.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.853+0000 7faef8fc9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef4070360 con 0x7faef4159ab0 2023-10-16T11:06:25.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.854+0000 7faef2d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faef406c140 0x7faef406c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:25.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.854+0000 7faef2d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faef406c140 0x7faef406c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33224/0 (socket says 172.21.15.84:33224) 2023-10-16T11:06:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.854+0000 7faef2d9d640 1 -- 172.21.15.84:0/1074935018 learned_addr learned my addr 172.21.15.84:0/1074935018 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.854+0000 7faef259c640 1 --2- 172.21.15.84:0/1074935018 >> v2:172.21.15.157:3300/0 conn(0x7faef4159ab0 0x7faef40ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.854+0000 7faef2d9d640 1 -- 172.21.15.84:0/1074935018 >> v2:172.21.15.157:3300/0 conn(0x7faef4159ab0 msgr2=0x7faef40ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.854+0000 7faef2d9d640 1 --2- 172.21.15.84:0/1074935018 >> v2:172.21.15.157:3300/0 conn(0x7faef4159ab0 0x7faef40ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.854+0000 7faef2d9d640 1 -- 172.21.15.84:0/1074935018 >> v1:172.21.15.157:6789/0 conn(0x7faef4159e90 legacy=0x7faef406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.854+0000 7faef2d9d640 1 -- 172.21.15.84:0/1074935018 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef40cd1f0 con 0x7faef406c140 2023-10-16T11:06:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.856+0000 7faef2d9d640 1 --2- 172.21.15.84:0/1074935018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef406c140 0x7faef406c590 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7faef40cfa00 tx=0x7faeec02bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311a062e82b08c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.856+0000 7faef1d9b640 1 -- 172.21.15.84:0/1074935018 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec02c960 con 0x7faef406c140 2023-10-16T11:06:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.856+0000 7faef1d9b640 1 -- 172.21.15.84:0/1074935018 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeec02cac0 con 0x7faef406c140 2023-10-16T11:06:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.856+0000 7faef1d9b640 1 -- 172.21.15.84:0/1074935018 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec02ce50 con 0x7faef406c140 2023-10-16T11:06:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.857+0000 7faef8fc9640 1 -- 172.21.15.84:0/1074935018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef406c140 msgr2=0x7faef406c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:25.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.857+0000 7faef8fc9640 1 --2- 172.21.15.84:0/1074935018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef406c140 0x7faef406c590 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7faef40cfa00 tx=0x7faeec02bdc0 comp rx=0 tx=0).stop 2023-10-16T11:06:25.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.857+0000 7faef8fc9640 1 -- 172.21.15.84:0/1074935018 shutdown_connections 2023-10-16T11:06:25.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.857+0000 7faef8fc9640 1 --2- 172.21.15.84:0/1074935018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef406c140 0x7faef406c590 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:25.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.857+0000 7faef8fc9640 1 --2- 172.21.15.84:0/1074935018 >> v2:172.21.15.157:3300/0 conn(0x7faef4159ab0 0x7faef40ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:25.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.857+0000 7faef8fc9640 1 -- 172.21.15.84:0/1074935018 >> 172.21.15.84:0/1074935018 conn(0x7faef40ce9b0 msgr2=0x7faef40cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:25.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.857+0000 7faef8fc9640 1 -- 172.21.15.84:0/1074935018 shutdown_connections 2023-10-16T11:06:25.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.857+0000 7faef8fc9640 1 -- 172.21.15.84:0/1074935018 wait complete. 2023-10-16T11:06:25.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.858+0000 7faef8fc9640 1 Processor -- start 2023-10-16T11:06:25.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.859+0000 7faef8fc9640 1 -- start start 2023-10-16T11:06:25.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.859+0000 7faef8fc9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef406c140 0x7faef415c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:25.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.859+0000 7faef8fc9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4159ab0 0x7faef41665a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:25.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.860+0000 7faef8fc9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef4159e90 0x7faef415a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:25.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.860+0000 7faef8fc9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef415d430 con 0x7faef4159e90 2023-10-16T11:06:25.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.860+0000 7faef8fc9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef415c820 con 0x7faef406c140 2023-10-16T11:06:25.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.860+0000 7faef8fc9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef415cb00 con 0x7faef4159ab0 2023-10-16T11:06:25.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.860+0000 7faef2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef4159e90 0x7faef415a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:25.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.860+0000 7faef2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef4159e90 0x7faef415a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33240/0 (socket says 172.21.15.84:33240) 2023-10-16T11:06:25.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.860+0000 7faef2d9d640 1 -- 172.21.15.84:0/1748408051 learned_addr learned my addr 172.21.15.84:0/1748408051 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:25.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.860+0000 7faef259c640 1 --2- 172.21.15.84:0/1748408051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef406c140 0x7faef415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:25.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.860+0000 7faef2d9d640 1 -- 172.21.15.84:0/1748408051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4159ab0 msgr2=0x7faef41665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:25.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.860+0000 7faeebfff640 1 --2- 172.21.15.84:0/1748408051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4159ab0 0x7faef41665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:25.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.860+0000 7faef2d9d640 1 --2- 172.21.15.84:0/1748408051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4159ab0 0x7faef41665a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:25.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.860+0000 7faef2d9d640 1 -- 172.21.15.84:0/1748408051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef406c140 msgr2=0x7faef415c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:25.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.860+0000 7faef2d9d640 1 --2- 172.21.15.84:0/1748408051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef406c140 0x7faef415c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:25.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.860+0000 7faef2d9d640 1 -- 172.21.15.84:0/1748408051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec009b30 con 0x7faef4159e90 2023-10-16T11:06:25.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.861+0000 7faef2d9d640 1 --2- 172.21.15.84:0/1748408051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef4159e90 0x7faef415a510 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7faeec036040 tx=0x7faeec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:25.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.861+0000 7faeeaffd640 1 -- 172.21.15.84:0/1748408051 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec009510 con 0x7faef4159e90 2023-10-16T11:06:25.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.861+0000 7faeeaffd640 1 -- 172.21.15.84:0/1748408051 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeec03e450 con 0x7faef4159e90 2023-10-16T11:06:25.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.861+0000 7faeeaffd640 1 -- 172.21.15.84:0/1748408051 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec03e800 con 0x7faef4159e90 2023-10-16T11:06:25.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.861+0000 7faef8fc9640 1 -- 172.21.15.84:0/1748408051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef415cde0 con 0x7faef4159e90 2023-10-16T11:06:25.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.861+0000 7faef8fc9640 1 -- 172.21.15.84:0/1748408051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef41595f0 con 0x7faef4159e90 2023-10-16T11:06:25.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.862+0000 7faef8fc9640 1 -- 172.21.15.84:0/1748408051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef40cda70 con 0x7faef4159e90 2023-10-16T11:06:25.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.863+0000 7faeeaffd640 1 -- 172.21.15.84:0/1748408051 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7faeec03e960 con 0x7faef4159e90 2023-10-16T11:06:25.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.865+0000 7faeeaffd640 1 --2- 172.21.15.84:0/1748408051 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7faed406af90 0x7faed406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:25.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.865+0000 7faeeaffd640 1 -- 172.21.15.84:0/1748408051 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5737+0+0 (secure 0 0 0) 0x7faeec0ac800 con 0x7faef4159e90 2023-10-16T11:06:25.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.865+0000 7faef259c640 1 --2- 172.21.15.84:0/1748408051 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7faed406af90 0x7faed406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:25.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.866+0000 7faef259c640 1 --2- 172.21.15.84:0/1748408051 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7faed406af90 0x7faed406d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faee4006fd0 tx=0x7faee4008040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:25.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:25.869+0000 7faeeaffd640 1 -- 172.21.15.84:0/1748408051 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7faef40cda70 con 0x7faef4159e90 2023-10-16T11:06:26.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.104+0000 7faef8fc9640 1 -- 172.21.15.84:0/1748408051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7faef406cad0 con 0x7faef4159e90 2023-10-16T11:06:26.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.616+0000 7faeeaffd640 1 -- 172.21.15.84:0/1748408051 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v96) v1 ==== 122+0+0 (secure 0 0 0) 0x7faef406cad0 con 0x7faef4159e90 2023-10-16T11:06:26.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.620+0000 7faef8fc9640 1 -- 172.21.15.84:0/1748408051 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7faed406af90 msgr2=0x7faed406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:26.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.620+0000 7faef8fc9640 1 --2- 172.21.15.84:0/1748408051 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7faed406af90 0x7faed406d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faee4006fd0 tx=0x7faee4008040 comp rx=0 tx=0).stop 2023-10-16T11:06:26.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.620+0000 7faef8fc9640 1 -- 172.21.15.84:0/1748408051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef4159e90 msgr2=0x7faef415a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:26.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.620+0000 7faef8fc9640 1 --2- 172.21.15.84:0/1748408051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef4159e90 0x7faef415a510 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7faeec036040 tx=0x7faeec004a40 comp rx=0 tx=0).stop 2023-10-16T11:06:26.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.621+0000 7faef8fc9640 1 -- 172.21.15.84:0/1748408051 shutdown_connections 2023-10-16T11:06:26.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.621+0000 7faef8fc9640 1 --2- 172.21.15.84:0/1748408051 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7faed406af90 0x7faed406d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:26.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.621+0000 7faef8fc9640 1 --2- 172.21.15.84:0/1748408051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef4159e90 0x7faef415a510 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:26.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.621+0000 7faef8fc9640 1 --2- 172.21.15.84:0/1748408051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4159ab0 0x7faef41665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:26.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.621+0000 7faef8fc9640 1 --2- 172.21.15.84:0/1748408051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef406c140 0x7faef415c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:26.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.621+0000 7faef8fc9640 1 -- 172.21.15.84:0/1748408051 >> 172.21.15.84:0/1748408051 conn(0x7faef40ce9b0 msgr2=0x7faef4069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:26.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.621+0000 7faef8fc9640 1 -- 172.21.15.84:0/1748408051 shutdown_connections 2023-10-16T11:06:26.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.621+0000 7faef8fc9640 1 -- 172.21.15.84:0/1748408051 wait complete. 2023-10-16T11:06:26.655 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T11:06:26.656 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:06:26.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.869+0000 7fce120c9640 1 Processor -- start 2023-10-16T11:06:26.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.869+0000 7fce120c9640 1 -- start start 2023-10-16T11:06:26.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.870+0000 7fce120c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce0c0caa10 0x7fce0c06fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:26.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.870+0000 7fce120c9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce0c0cc920 0x7fce0c0ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:26.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.870+0000 7fce120c9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce0c159790 con 0x7fce0c1593b0 2023-10-16T11:06:26.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.870+0000 7fce120c9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce0c0c9b20 con 0x7fce0c0cc920 2023-10-16T11:06:26.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.870+0000 7fce120c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c0c9e00 con 0x7fce0c0caa10 2023-10-16T11:06:26.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.871+0000 7fce0affd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce0c0caa10 0x7fce0c06fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:26.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.871+0000 7fce0affd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce0c0caa10 0x7fce0c06fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:56182/0 (socket says 172.21.15.84:56182) 2023-10-16T11:06:26.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.871+0000 7fce0affd640 1 -- 172.21.15.84:0/385844668 learned_addr learned my addr 172.21.15.84:0/385844668 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:26.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.871+0000 7fce0bfff640 1 --2- 172.21.15.84:0/385844668 >> v2:172.21.15.157:3300/0 conn(0x7fce0c0cc920 0x7fce0c0ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:26.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.872+0000 7fce0affd640 1 -- 172.21.15.84:0/385844668 >> v2:172.21.15.157:3300/0 conn(0x7fce0c0cc920 msgr2=0x7fce0c0ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:26.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.872+0000 7fce0affd640 1 --2- 172.21.15.84:0/385844668 >> v2:172.21.15.157:3300/0 conn(0x7fce0c0cc920 0x7fce0c0ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:26.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.872+0000 7fce0affd640 1 -- 172.21.15.84:0/385844668 >> v1:172.21.15.157:6789/0 conn(0x7fce0c1593b0 legacy=0x7fce0c06f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:26.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.872+0000 7fce0affd640 1 -- 172.21.15.84:0/385844668 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0c0ca0e0 con 0x7fce0c0caa10 2023-10-16T11:06:26.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.872+0000 7fce0bfff640 1 --2- 172.21.15.84:0/385844668 >> v2:172.21.15.157:3300/0 conn(0x7fce0c0cc920 0x7fce0c0ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:06:26.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.873+0000 7fce0affd640 1 --2- 172.21.15.84:0/385844668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce0c0caa10 0x7fce0c06fc60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fce00009e50 tx=0x7fce0002be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32c9a41e5d45c3b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:26.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.873+0000 7fce0a7fc640 1 -- 172.21.15.84:0/385844668 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0002c9f0 con 0x7fce0c0caa10 2023-10-16T11:06:26.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.874+0000 7fce0a7fc640 1 -- 172.21.15.84:0/385844668 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0002cb50 con 0x7fce0c0caa10 2023-10-16T11:06:26.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.874+0000 7fce0a7fc640 1 -- 172.21.15.84:0/385844668 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce000343f0 con 0x7fce0c0caa10 2023-10-16T11:06:26.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.874+0000 7fce120c9640 1 -- 172.21.15.84:0/385844668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce0c0caa10 msgr2=0x7fce0c06fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:26.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.874+0000 7fce120c9640 1 --2- 172.21.15.84:0/385844668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce0c0caa10 0x7fce0c06fc60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fce00009e50 tx=0x7fce0002be50 comp rx=0 tx=0).stop 2023-10-16T11:06:26.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.874+0000 7fce120c9640 1 -- 172.21.15.84:0/385844668 shutdown_connections 2023-10-16T11:06:26.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.874+0000 7fce120c9640 1 --2- 172.21.15.84:0/385844668 >> v2:172.21.15.157:3300/0 conn(0x7fce0c0cc920 0x7fce0c0ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:26.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.874+0000 7fce120c9640 1 --2- 172.21.15.84:0/385844668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce0c0caa10 0x7fce0c06fc60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:26.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.874+0000 7fce120c9640 1 -- 172.21.15.84:0/385844668 >> 172.21.15.84:0/385844668 conn(0x7fce0c0cb410 msgr2=0x7fce0c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:26.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.874+0000 7fce120c9640 1 -- 172.21.15.84:0/385844668 shutdown_connections 2023-10-16T11:06:26.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.874+0000 7fce120c9640 1 -- 172.21.15.84:0/385844668 wait complete. 2023-10-16T11:06:26.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.876+0000 7fce120c9640 1 Processor -- start 2023-10-16T11:06:26.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.876+0000 7fce120c9640 1 -- start start 2023-10-16T11:06:26.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.877+0000 7fce120c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce0c0caa10 0x7fce0c15c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:26.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.877+0000 7fce120c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce0c0cc920 0x7fce0c152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:26.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.877+0000 7fce120c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce0c1593b0 0x7fce0c1f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:26.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.877+0000 7fce120c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c157570 con 0x7fce0c1593b0 2023-10-16T11:06:26.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.877+0000 7fce120c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c1577f0 con 0x7fce0c0cc920 2023-10-16T11:06:26.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.877+0000 7fce120c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c157ad0 con 0x7fce0c0caa10 2023-10-16T11:06:26.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.877+0000 7fce0bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce0c1593b0 0x7fce0c1f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:26.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.877+0000 7fce0bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce0c1593b0 0x7fce0c1f3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33252/0 (socket says 172.21.15.84:33252) 2023-10-16T11:06:26.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.877+0000 7fce0bfff640 1 -- 172.21.15.84:0/1988076601 learned_addr learned my addr 172.21.15.84:0/1988076601 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:26.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.878+0000 7fce0b7fe640 1 --2- 172.21.15.84:0/1988076601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce0c0caa10 0x7fce0c15c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:26.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.878+0000 7fce0bfff640 1 -- 172.21.15.84:0/1988076601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce0c0caa10 msgr2=0x7fce0c15c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:26.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.878+0000 7fce0affd640 1 --2- 172.21.15.84:0/1988076601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce0c0cc920 0x7fce0c152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:26.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.878+0000 7fce0bfff640 1 --2- 172.21.15.84:0/1988076601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce0c0caa10 0x7fce0c15c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:26.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.878+0000 7fce0bfff640 1 -- 172.21.15.84:0/1988076601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce0c0cc920 msgr2=0x7fce0c152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:26.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.878+0000 7fce0bfff640 1 --2- 172.21.15.84:0/1988076601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce0c0cc920 0x7fce0c152890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:26.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.878+0000 7fce0bfff640 1 -- 172.21.15.84:0/1988076601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0400a5a0 con 0x7fce0c1593b0 2023-10-16T11:06:26.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.878+0000 7fce0bfff640 1 --2- 172.21.15.84:0/1988076601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce0c1593b0 0x7fce0c1f3e00 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fce0400e890 tx=0x7fce0400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:26.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.878+0000 7fcdf3fff640 1 -- 172.21.15.84:0/1988076601 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0400cc40 con 0x7fce0c1593b0 2023-10-16T11:06:26.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.878+0000 7fcdf3fff640 1 -- 172.21.15.84:0/1988076601 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0400cda0 con 0x7fce0c1593b0 2023-10-16T11:06:26.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.878+0000 7fce120c9640 1 -- 172.21.15.84:0/1988076601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce00009b30 con 0x7fce0c1593b0 2023-10-16T11:06:26.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.879+0000 7fcdf3fff640 1 -- 172.21.15.84:0/1988076601 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce04004240 con 0x7fce0c1593b0 2023-10-16T11:06:26.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.879+0000 7fce120c9640 1 -- 172.21.15.84:0/1988076601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0c158090 con 0x7fce0c1593b0 2023-10-16T11:06:26.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.880+0000 7fce120c9640 1 -- 172.21.15.84:0/1988076601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0c04e160 con 0x7fce0c1593b0 2023-10-16T11:06:26.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.881+0000 7fcdf3fff640 1 -- 172.21.15.84:0/1988076601 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fce040043a0 con 0x7fce0c1593b0 2023-10-16T11:06:26.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.882+0000 7fcdf3fff640 1 --2- 172.21.15.84:0/1988076601 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fcdf806af90 0x7fcdf806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:26.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.882+0000 7fcdf3fff640 1 -- 172.21.15.84:0/1988076601 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fce040896b0 con 0x7fce0c1593b0 2023-10-16T11:06:26.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.882+0000 7fce0b7fe640 1 --2- 172.21.15.84:0/1988076601 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fcdf806af90 0x7fcdf806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:26.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.883+0000 7fce0b7fe640 1 --2- 172.21.15.84:0/1988076601 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fcdf806af90 0x7fcdf806d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc000f60 tx=0x7fcdfc009040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:26.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:26.886+0000 7fcdf3fff640 1 -- 172.21.15.84:0/1988076601 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fce04054520 con 0x7fce0c1593b0 2023-10-16T11:06:27.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.116+0000 7fce120c9640 1 -- 172.21.15.84:0/1988076601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce0c0b4910 con 0x7fce0c1593b0 2023-10-16T11:06:27.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.117+0000 7fcdf3fff640 1 -- 172.21.15.84:0/1988076601 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fce04057b40 con 0x7fce0c1593b0 2023-10-16T11:06:27.119 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:27.119 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6347,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1346424254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1346424254},{"type":"v1","addr":"172.21.15.157:6835","nonce":1346424254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3224476768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3224476768},{"type":"v1","addr":"172.21.15.157:6834","nonce":3224476768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":10197,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2490106026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2490106026},{"type":"v1","addr":"172.21.15.84:6835","nonce":2490106026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:06:23.034809+0000","modified":"2023-10-16T11:06:26.105994+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10206},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10206":{"gid":10206,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/1106170430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":1106170430},{"type":"v1","addr":"172.21.15.84:6837","nonce":1106170430}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-16T11:06:27.120 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 96 2023-10-16T11:06:27.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.119+0000 7fce120c9640 1 -- 172.21.15.84:0/1988076601 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fcdf806af90 msgr2=0x7fcdf806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:27.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.119+0000 7fce120c9640 1 --2- 172.21.15.84:0/1988076601 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fcdf806af90 0x7fcdf806d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc000f60 tx=0x7fcdfc009040 comp rx=0 tx=0).stop 2023-10-16T11:06:27.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.119+0000 7fce120c9640 1 -- 172.21.15.84:0/1988076601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce0c1593b0 msgr2=0x7fce0c1f3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:27.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.119+0000 7fce120c9640 1 --2- 172.21.15.84:0/1988076601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce0c1593b0 0x7fce0c1f3e00 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fce0400e890 tx=0x7fce0400ed50 comp rx=0 tx=0).stop 2023-10-16T11:06:27.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.120+0000 7fce120c9640 1 -- 172.21.15.84:0/1988076601 shutdown_connections 2023-10-16T11:06:27.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.120+0000 7fce120c9640 1 --2- 172.21.15.84:0/1988076601 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fcdf806af90 0x7fcdf806d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.120+0000 7fce120c9640 1 --2- 172.21.15.84:0/1988076601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce0c1593b0 0x7fce0c1f3e00 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.120+0000 7fce120c9640 1 --2- 172.21.15.84:0/1988076601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce0c0cc920 0x7fce0c152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.120+0000 7fce120c9640 1 --2- 172.21.15.84:0/1988076601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce0c0caa10 0x7fce0c15c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.120+0000 7fce120c9640 1 -- 172.21.15.84:0/1988076601 >> 172.21.15.84:0/1988076601 conn(0x7fce0c0cb410 msgr2=0x7fce0c0b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:27.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.120+0000 7fce120c9640 1 -- 172.21.15.84:0/1988076601 shutdown_connections 2023-10-16T11:06:27.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.120+0000 7fce120c9640 1 -- 172.21.15.84:0/1988076601 wait complete. 2023-10-16T11:06:27.141 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:06:27.307 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:06:27.316 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:06:27.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.354+0000 7f093882e640 1 Processor -- start 2023-10-16T11:06:27.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.354+0000 7f093882e640 1 -- start start 2023-10-16T11:06:27.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.355+0000 7f093882e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f093406c040 0x7f093406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:27.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.355+0000 7f093882e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09340ad970 0x7f093415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:27.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.355+0000 7f093882e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f093415ebc0 0x7f093415f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:27.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.355+0000 7f093882e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093406c960 con 0x7f093406c040 2023-10-16T11:06:27.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.355+0000 7f093882e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093415b9d0 con 0x7f093415ebc0 2023-10-16T11:06:27.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.355+0000 7f093882e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093415bcb0 con 0x7f09340ad970 2023-10-16T11:06:27.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.355+0000 7f09337fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f093406c040 0x7f093406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:27.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.356+0000 7f09337fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f093406c040 0x7f093406c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33256/0 (socket says 172.21.15.84:33256) 2023-10-16T11:06:27.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.356+0000 7f09337fe640 1 -- 172.21.15.84:0/1986458942 learned_addr learned my addr 172.21.15.84:0/1986458942 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:27.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.356+0000 7f0932ffd640 1 --2- 172.21.15.84:0/1986458942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09340ad970 0x7f093415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:27.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.356+0000 7f0933fff640 1 --2- 172.21.15.84:0/1986458942 >> v2:172.21.15.157:3300/0 conn(0x7f093415ebc0 0x7f093415f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:27.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.356+0000 7f09337fe640 1 -- 172.21.15.84:0/1986458942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09340ad970 msgr2=0x7f093415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:27.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.356+0000 7f09337fe640 1 --2- 172.21.15.84:0/1986458942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09340ad970 0x7f093415e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.356+0000 7f09337fe640 1 -- 172.21.15.84:0/1986458942 >> v2:172.21.15.157:3300/0 conn(0x7f093415ebc0 msgr2=0x7f093415f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:27.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.356+0000 7f09337fe640 1 --2- 172.21.15.84:0/1986458942 >> v2:172.21.15.157:3300/0 conn(0x7f093415ebc0 0x7f093415f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.356+0000 7f09337fe640 1 -- 172.21.15.84:0/1986458942 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093415bf90 con 0x7f093406c040 2023-10-16T11:06:27.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.356+0000 7f0932ffd640 1 --2- 172.21.15.84:0/1986458942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09340ad970 0x7f093415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:06:27.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.357+0000 7f09337fe640 1 --2- 172.21.15.84:0/1986458942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f093406c040 0x7f093406c420 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f0924009870 tx=0x7f092402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd551b99c7bd9e82 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:27.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.358+0000 7f09327fc640 1 -- 172.21.15.84:0/1986458942 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924031640 con 0x7f093406c040 2023-10-16T11:06:27.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.358+0000 7f09327fc640 1 -- 172.21.15.84:0/1986458942 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09240317a0 con 0x7f093406c040 2023-10-16T11:06:27.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.358+0000 7f09327fc640 1 -- 172.21.15.84:0/1986458942 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924031b30 con 0x7f093406c040 2023-10-16T11:06:27.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.358+0000 7f093882e640 1 -- 172.21.15.84:0/1986458942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f093406c040 msgr2=0x7f093406c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:27.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.358+0000 7f093882e640 1 --2- 172.21.15.84:0/1986458942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f093406c040 0x7f093406c420 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f0924009870 tx=0x7f092402b8f0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.358+0000 7f093882e640 1 -- 172.21.15.84:0/1986458942 shutdown_connections 2023-10-16T11:06:27.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.358+0000 7f093882e640 1 --2- 172.21.15.84:0/1986458942 >> v2:172.21.15.157:3300/0 conn(0x7f093415ebc0 0x7f093415f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.358+0000 7f093882e640 1 --2- 172.21.15.84:0/1986458942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09340ad970 0x7f093415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.358+0000 7f093882e640 1 --2- 172.21.15.84:0/1986458942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f093406c040 0x7f093406c420 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.358+0000 7f093882e640 1 -- 172.21.15.84:0/1986458942 >> 172.21.15.84:0/1986458942 conn(0x7f09340ce6a0 msgr2=0x7f09340ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:27.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.358+0000 7f093882e640 1 -- 172.21.15.84:0/1986458942 shutdown_connections 2023-10-16T11:06:27.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.358+0000 7f093882e640 1 -- 172.21.15.84:0/1986458942 wait complete. 2023-10-16T11:06:27.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.360+0000 7f093882e640 1 Processor -- start 2023-10-16T11:06:27.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.360+0000 7f093882e640 1 -- start start 2023-10-16T11:06:27.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.361+0000 7f093882e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f093406c040 0x7f0934149b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:27.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.361+0000 7f093882e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09340ad970 0x7f09341f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:27.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.361+0000 7f093882e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f093415ebc0 0x7f09340ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:27.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.361+0000 7f093882e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093414a0a0 con 0x7f09340ad970 2023-10-16T11:06:27.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.361+0000 7f093882e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09341f2c70 con 0x7f093406c040 2023-10-16T11:06:27.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.361+0000 7f093882e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09341f2f50 con 0x7f093415ebc0 2023-10-16T11:06:27.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.362+0000 7f0932ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09340ad970 0x7f09341f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:27.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.362+0000 7f0932ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09340ad970 0x7f09341f2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33270/0 (socket says 172.21.15.84:33270) 2023-10-16T11:06:27.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.362+0000 7f0932ffd640 1 -- 172.21.15.84:0/2976206568 learned_addr learned my addr 172.21.15.84:0/2976206568 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:27.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.362+0000 7f0933fff640 1 --2- 172.21.15.84:0/2976206568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f093415ebc0 0x7f09340ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:27.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.362+0000 7f09337fe640 1 --2- 172.21.15.84:0/2976206568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f093406c040 0x7f0934149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:27.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.362+0000 7f0932ffd640 1 -- 172.21.15.84:0/2976206568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f093415ebc0 msgr2=0x7f09340ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:27.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.362+0000 7f0932ffd640 1 --2- 172.21.15.84:0/2976206568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f093415ebc0 0x7f09340ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.362+0000 7f0932ffd640 1 -- 172.21.15.84:0/2976206568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f093406c040 msgr2=0x7f0934149b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:27.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.362+0000 7f0932ffd640 1 --2- 172.21.15.84:0/2976206568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f093406c040 0x7f0934149b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.362+0000 7f0932ffd640 1 -- 172.21.15.84:0/2976206568 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09240094d0 con 0x7f09340ad970 2023-10-16T11:06:27.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.363+0000 7f0932ffd640 1 --2- 172.21.15.84:0/2976206568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09340ad970 0x7f09341f2370 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f092800e810 tx=0x7f092800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:27.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.363+0000 7f091bfff640 1 -- 172.21.15.84:0/2976206568 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0928004030 con 0x7f09340ad970 2023-10-16T11:06:27.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.363+0000 7f091bfff640 1 -- 172.21.15.84:0/2976206568 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0928004190 con 0x7f09340ad970 2023-10-16T11:06:27.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.363+0000 7f091bfff640 1 -- 172.21.15.84:0/2976206568 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0928004520 con 0x7f09340ad970 2023-10-16T11:06:27.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.363+0000 7f093882e640 1 -- 172.21.15.84:0/2976206568 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09341f3230 con 0x7f09340ad970 2023-10-16T11:06:27.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.363+0000 7f093882e640 1 -- 172.21.15.84:0/2976206568 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09340ae3e0 con 0x7f09340ad970 2023-10-16T11:06:27.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.364+0000 7f093882e640 1 -- 172.21.15.84:0/2976206568 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093406b330 con 0x7f09340ad970 2023-10-16T11:06:27.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.365+0000 7f091bfff640 1 -- 172.21.15.84:0/2976206568 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f092800c6d0 con 0x7f09340ad970 2023-10-16T11:06:27.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.366+0000 7f091bfff640 1 --2- 172.21.15.84:0/2976206568 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f092006af90 0x7f092006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:27.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.366+0000 7f091bfff640 1 -- 172.21.15.84:0/2976206568 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f09280894a0 con 0x7f09340ad970 2023-10-16T11:06:27.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.367+0000 7f09337fe640 1 --2- 172.21.15.84:0/2976206568 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f092006af90 0x7f092006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:27.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.367+0000 7f09337fe640 1 --2- 172.21.15.84:0/2976206568 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f092006af90 0x7f092006d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0924036000 tx=0x7f092403c040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:27.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.371+0000 7f091bfff640 1 -- 172.21.15.84:0/2976206568 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0928054320 con 0x7f09340ad970 2023-10-16T11:06:27.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.566+0000 7f093882e640 1 -- 172.21.15.84:0/2976206568 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f093404e9b0 con 0x7f09340ad970 2023-10-16T11:06:27.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.568+0000 7f091bfff640 1 -- 172.21.15.84:0/2976206568 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+16591 (secure 0 0 0) 0x7f0928057940 con 0x7f09340ad970 2023-10-16T11:06:27.570 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:27.571 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":184,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:06:24.589906+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:06:20.873257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-16T11:06:22.065944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:06:27.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.571+0000 7f093882e640 1 -- 172.21.15.84:0/2976206568 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f092006af90 msgr2=0x7f092006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:27.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.571+0000 7f093882e640 1 --2- 172.21.15.84:0/2976206568 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f092006af90 0x7f092006d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0924036000 tx=0x7f092403c040 comp rx=0 tx=0).stop 2023-10-16T11:06:27.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.571+0000 7f093882e640 1 -- 172.21.15.84:0/2976206568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09340ad970 msgr2=0x7f09341f2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:27.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.571+0000 7f093882e640 1 --2- 172.21.15.84:0/2976206568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09340ad970 0x7f09341f2370 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f092800e810 tx=0x7f092800ecd0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.571+0000 7f093882e640 1 -- 172.21.15.84:0/2976206568 shutdown_connections 2023-10-16T11:06:27.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.571+0000 7f093882e640 1 --2- 172.21.15.84:0/2976206568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f093415ebc0 0x7f09340ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.571+0000 7f093882e640 1 --2- 172.21.15.84:0/2976206568 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f092006af90 0x7f092006d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.571+0000 7f093882e640 1 --2- 172.21.15.84:0/2976206568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09340ad970 0x7f09341f2370 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.571+0000 7f093882e640 1 --2- 172.21.15.84:0/2976206568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f093406c040 0x7f0934149b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.571+0000 7f093882e640 1 -- 172.21.15.84:0/2976206568 >> 172.21.15.84:0/2976206568 conn(0x7f09340ce6a0 msgr2=0x7f09340ca6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:27.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.571+0000 7f093882e640 1 -- 172.21.15.84:0/2976206568 shutdown_connections 2023-10-16T11:06:27.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.572+0000 7f093882e640 1 -- 172.21.15.84:0/2976206568 wait complete. 2023-10-16T11:06:27.591 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T11:06:27.592 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:06:27.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.795+0000 7f85f112c640 1 Processor -- start 2023-10-16T11:06:27.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.795+0000 7f85f112c640 1 -- start start 2023-10-16T11:06:27.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.796+0000 7f85f112c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f85ec0adb50 0x7f85ec06f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:27.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.796+0000 7f85f112c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f85ec070fd0 0x7f85ec0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:27.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.796+0000 7f85f112c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85ec071450 con 0x7f85ec0adf30 2023-10-16T11:06:27.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.797+0000 7f85f112c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85ec0cc600 con 0x7f85ec070fd0 2023-10-16T11:06:27.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.797+0000 7f85f112c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85ec0cc8e0 con 0x7f85ec0adb50 2023-10-16T11:06:27.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.797+0000 7f85e3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f85ec070fd0 0x7f85ec0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:27.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.797+0000 7f85e3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f85ec070fd0 0x7f85ec0b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33278/0 (socket says 172.21.15.84:33278) 2023-10-16T11:06:27.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.797+0000 7f85e3fff640 1 -- 172.21.15.84:0/1072059058 learned_addr learned my addr 172.21.15.84:0/1072059058 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:27.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.797+0000 7f85ead9d640 1 --2- 172.21.15.84:0/1072059058 >> v2:172.21.15.157:3300/0 conn(0x7f85ec0adb50 0x7f85ec06f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:27.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.797+0000 7f85e3fff640 1 -- 172.21.15.84:0/1072059058 >> v2:172.21.15.157:3300/0 conn(0x7f85ec0adb50 msgr2=0x7f85ec06f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:27.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.797+0000 7f85e3fff640 1 --2- 172.21.15.84:0/1072059058 >> v2:172.21.15.157:3300/0 conn(0x7f85ec0adb50 0x7f85ec06f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.797+0000 7f85e3fff640 1 -- 172.21.15.84:0/1072059058 >> v1:172.21.15.157:6789/0 conn(0x7f85ec0adf30 legacy=0x7f85ec164f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:27.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.797+0000 7f85e3fff640 1 -- 172.21.15.84:0/1072059058 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ec0ccbc0 con 0x7f85ec070fd0 2023-10-16T11:06:27.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.797+0000 7f85ead9d640 1 --2- 172.21.15.84:0/1072059058 >> v2:172.21.15.157:3300/0 conn(0x7f85ec0adb50 0x7f85ec06f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:27.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.798+0000 7f85e3fff640 1 --2- 172.21.15.84:0/1072059058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85ec070fd0 0x7f85ec0b0410 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f85ec071a00 tx=0x7f85d402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f8b84b34344291 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:27.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.799+0000 7f85ea59c640 1 -- 172.21.15.84:0/1072059058 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d4031890 con 0x7f85ec070fd0 2023-10-16T11:06:27.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.799+0000 7f85ea59c640 1 -- 172.21.15.84:0/1072059058 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d40319f0 con 0x7f85ec070fd0 2023-10-16T11:06:27.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.799+0000 7f85ea59c640 1 -- 172.21.15.84:0/1072059058 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d4031d80 con 0x7f85ec070fd0 2023-10-16T11:06:27.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.799+0000 7f85f112c640 1 -- 172.21.15.84:0/1072059058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85ec070fd0 msgr2=0x7f85ec0b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:27.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.799+0000 7f85f112c640 1 --2- 172.21.15.84:0/1072059058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85ec070fd0 0x7f85ec0b0410 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f85ec071a00 tx=0x7f85d402b930 comp rx=0 tx=0).stop 2023-10-16T11:06:27.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.799+0000 7f85f112c640 1 -- 172.21.15.84:0/1072059058 shutdown_connections 2023-10-16T11:06:27.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.799+0000 7f85f112c640 1 --2- 172.21.15.84:0/1072059058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85ec070fd0 0x7f85ec0b0410 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.799+0000 7f85f112c640 1 --2- 172.21.15.84:0/1072059058 >> v2:172.21.15.157:3300/0 conn(0x7f85ec0adb50 0x7f85ec06f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.799+0000 7f85f112c640 1 -- 172.21.15.84:0/1072059058 >> 172.21.15.84:0/1072059058 conn(0x7f85ec0cb410 msgr2=0x7f85ec0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:27.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.800+0000 7f85f112c640 1 -- 172.21.15.84:0/1072059058 shutdown_connections 2023-10-16T11:06:27.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.800+0000 7f85f112c640 1 -- 172.21.15.84:0/1072059058 wait complete. 2023-10-16T11:06:27.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.801+0000 7f85f112c640 1 Processor -- start 2023-10-16T11:06:27.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.801+0000 7f85f112c640 1 -- start start 2023-10-16T11:06:27.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.802+0000 7f85f112c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ec070fd0 0x7f85ec154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:27.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.802+0000 7f85f112c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85ec0adb50 0x7f85ec164430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:27.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.802+0000 7f85f112c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85ec0adf30 0x7f85ec0ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:27.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.802+0000 7f85f112c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec0c32c0 con 0x7f85ec0adb50 2023-10-16T11:06:27.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.802+0000 7f85f112c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec164970 con 0x7f85ec070fd0 2023-10-16T11:06:27.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.802+0000 7f85f112c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec0c26b0 con 0x7f85ec0adf30 2023-10-16T11:06:27.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.803+0000 7f85e3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85ec0adb50 0x7f85ec164430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:27.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.803+0000 7f85e3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85ec0adb50 0x7f85ec164430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33290/0 (socket says 172.21.15.84:33290) 2023-10-16T11:06:27.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.803+0000 7f85e3fff640 1 -- 172.21.15.84:0/807755780 learned_addr learned my addr 172.21.15.84:0/807755780 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:27.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.803+0000 7f85ead9d640 1 --2- 172.21.15.84:0/807755780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ec070fd0 0x7f85ec154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:27.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.803+0000 7f85e3fff640 1 -- 172.21.15.84:0/807755780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85ec0adf30 msgr2=0x7f85ec0ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:27.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.803+0000 7f85e3fff640 1 --2- 172.21.15.84:0/807755780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85ec0adf30 0x7f85ec0ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.803+0000 7f85e3fff640 1 -- 172.21.15.84:0/807755780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ec070fd0 msgr2=0x7f85ec154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:27.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.803+0000 7f85e3fff640 1 --2- 172.21.15.84:0/807755780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ec070fd0 0x7f85ec154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:27.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.803+0000 7f85e3fff640 1 -- 172.21.15.84:0/807755780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d40094d0 con 0x7f85ec0adb50 2023-10-16T11:06:27.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.803+0000 7f85ead9d640 1 --2- 172.21.15.84:0/807755780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ec070fd0 0x7f85ec154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:27.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.804+0000 7f85e3fff640 1 --2- 172.21.15.84:0/807755780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85ec0adb50 0x7f85ec164430 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f85d4000c00 tx=0x7f85d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:27.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.804+0000 7f85e37fe640 1 -- 172.21.15.84:0/807755780 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d4031840 con 0x7f85ec0adb50 2023-10-16T11:06:27.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.804+0000 7f85e37fe640 1 -- 172.21.15.84:0/807755780 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d40443f0 con 0x7f85ec0adb50 2023-10-16T11:06:27.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.804+0000 7f85e37fe640 1 -- 172.21.15.84:0/807755780 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d40447a0 con 0x7f85ec0adb50 2023-10-16T11:06:27.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.804+0000 7f85f112c640 1 -- 172.21.15.84:0/807755780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ec0c2990 con 0x7f85ec0adb50 2023-10-16T11:06:27.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.804+0000 7f85f112c640 1 -- 172.21.15.84:0/807755780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ec0c2f50 con 0x7f85ec0adb50 2023-10-16T11:06:27.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.805+0000 7f85f112c640 1 -- 172.21.15.84:0/807755780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85ec0ce230 con 0x7f85ec0adb50 2023-10-16T11:06:27.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.806+0000 7f85e37fe640 1 -- 172.21.15.84:0/807755780 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f85d404d450 con 0x7f85ec0adb50 2023-10-16T11:06:27.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.807+0000 7f85e37fe640 1 --2- 172.21.15.84:0/807755780 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f85cc06af90 0x7f85cc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:27.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.807+0000 7f85e37fe640 1 -- 172.21.15.84:0/807755780 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f85d40ac0e0 con 0x7f85ec0adb50 2023-10-16T11:06:27.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.807+0000 7f85ead9d640 1 --2- 172.21.15.84:0/807755780 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f85cc06af90 0x7f85cc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:27.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.808+0000 7f85ead9d640 1 --2- 172.21.15.84:0/807755780 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f85cc06af90 0x7f85cc06d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f85dc006fd0 tx=0x7f85dc008040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:27.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:27.811+0000 7f85e37fe640 1 -- 172.21.15.84:0/807755780 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f85d4076f50 con 0x7f85ec0adb50 2023-10-16T11:06:28.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.001+0000 7f85f112c640 1 -- 172.21.15.84:0/807755780 --> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f85ec071730 con 0x7f85cc06af90 2023-10-16T11:06:28.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.016+0000 7f85e37fe640 1 -- 172.21.15.84:0/807755780 <== mgr.9969 v2:172.21.15.84:6832/1441051579 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287359 (secure 0 0 0) 0x7f85dc00c040 con 0x7f85cc06af90 2023-10-16T11:06:28.021 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:28.023 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:06:28.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.021+0000 7f85f112c640 1 -- 172.21.15.84:0/807755780 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f85cc06af90 msgr2=0x7f85cc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:28.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.021+0000 7f85f112c640 1 --2- 172.21.15.84:0/807755780 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f85cc06af90 0x7f85cc06d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f85dc006fd0 tx=0x7f85dc008040 comp rx=0 tx=0).stop 2023-10-16T11:06:28.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.021+0000 7f85f112c640 1 -- 172.21.15.84:0/807755780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85ec0adb50 msgr2=0x7f85ec164430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:28.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.021+0000 7f85f112c640 1 --2- 172.21.15.84:0/807755780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85ec0adb50 0x7f85ec164430 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f85d4000c00 tx=0x7f85d40046c0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.021+0000 7f85f112c640 1 -- 172.21.15.84:0/807755780 shutdown_connections 2023-10-16T11:06:28.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.021+0000 7f85f112c640 1 --2- 172.21.15.84:0/807755780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85ec0adf30 0x7f85ec0ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.021+0000 7f85f112c640 1 --2- 172.21.15.84:0/807755780 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f85cc06af90 0x7f85cc06d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.022+0000 7f85f112c640 1 --2- 172.21.15.84:0/807755780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85ec0adb50 0x7f85ec164430 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.022+0000 7f85f112c640 1 --2- 172.21.15.84:0/807755780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ec070fd0 0x7f85ec154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.022+0000 7f85f112c640 1 -- 172.21.15.84:0/807755780 >> 172.21.15.84:0/807755780 conn(0x7f85ec0cb410 msgr2=0x7f85ec0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:28.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.022+0000 7f85f112c640 1 -- 172.21.15.84:0/807755780 shutdown_connections 2023-10-16T11:06:28.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.022+0000 7f85f112c640 1 -- 172.21.15.84:0/807755780 wait complete. 2023-10-16T11:06:28.042 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-16T11:06:27.678123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1696,"num_read_kb":3438,"num_write":525,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2750,"ondisk_log_size":2750,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":389420,"kb_used_data":19208,"kb_used_omap":12,"kb_used_meta":370099,"kb_avail":754585300,"statfs":{"total":773094113280,"available":772695347200,"internally_reserved":0,"allocated":19668992,"data_stored":10052860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":378981945},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001409"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.690053+0000","last_change":"2023-10-16T11:06:23.612356+0000","last_active":"2023-10-16T11:06:25.690053+0000","last_peered":"2023-10-16T11:06:25.690053+0000","last_clean":"2023-10-16T11:06:25.690053+0000","last_became_active":"2023-10-16T11:06:23.611936+0000","last_became_peered":"2023-10-16T11:06:23.611936+0000","last_unstale":"2023-10-16T11:06:25.690053+0000","last_undegraded":"2023-10-16T11:06:25.690053+0000","last_fullsized":"2023-10-16T11:06:25.690053+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:44:58.903083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.980986+0000","last_change":"2023-10-16T11:06:23.612251+0000","last_active":"2023-10-16T11:06:24.980986+0000","last_peered":"2023-10-16T11:06:24.980986+0000","last_clean":"2023-10-16T11:06:24.980986+0000","last_became_active":"2023-10-16T11:06:23.611968+0000","last_became_peered":"2023-10-16T11:06:23.611968+0000","last_unstale":"2023-10-16T11:06:24.980986+0000","last_undegraded":"2023-10-16T11:06:24.980986+0000","last_fullsized":"2023-10-16T11:06:24.980986+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:29:32.028128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.691944+0000","last_change":"2023-10-16T11:06:23.612260+0000","last_active":"2023-10-16T11:06:25.691944+0000","last_peered":"2023-10-16T11:06:25.691944+0000","last_clean":"2023-10-16T11:06:25.691944+0000","last_became_active":"2023-10-16T11:06:23.611970+0000","last_became_peered":"2023-10-16T11:06:23.611970+0000","last_unstale":"2023-10-16T11:06:25.691944+0000","last_undegraded":"2023-10-16T11:06:25.691944+0000","last_fullsized":"2023-10-16T11:06:25.691944+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:35.201235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.682638+0000","last_change":"2023-10-16T11:06:23.613121+0000","last_active":"2023-10-16T11:06:24.682638+0000","last_peered":"2023-10-16T11:06:24.682638+0000","last_clean":"2023-10-16T11:06:24.682638+0000","last_became_active":"2023-10-16T11:06:23.612829+0000","last_became_peered":"2023-10-16T11:06:23.612829+0000","last_unstale":"2023-10-16T11:06:24.682638+0000","last_undegraded":"2023-10-16T11:06:24.682638+0000","last_fullsized":"2023-10-16T11:06:24.682638+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:56:48.322204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.980352+0000","last_change":"2023-10-16T11:06:23.613177+0000","last_active":"2023-10-16T11:06:24.980352+0000","last_peered":"2023-10-16T11:06:24.980352+0000","last_clean":"2023-10-16T11:06:24.980352+0000","last_became_active":"2023-10-16T11:06:23.612910+0000","last_became_peered":"2023-10-16T11:06:23.612910+0000","last_unstale":"2023-10-16T11:06:24.980352+0000","last_undegraded":"2023-10-16T11:06:24.980352+0000","last_fullsized":"2023-10-16T11:06:24.980352+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:20:15.243219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.604960+0000","last_change":"2023-10-16T11:06:23.617312+0000","last_active":"2023-10-16T11:06:24.604960+0000","last_peered":"2023-10-16T11:06:24.604960+0000","last_clean":"2023-10-16T11:06:24.604960+0000","last_became_active":"2023-10-16T11:06:23.613671+0000","last_became_peered":"2023-10-16T11:06:23.613671+0000","last_unstale":"2023-10-16T11:06:24.604960+0000","last_undegraded":"2023-10-16T11:06:24.604960+0000","last_fullsized":"2023-10-16T11:06:24.604960+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:44:42.439099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.188401+0000","last_change":"2023-10-16T11:06:23.614759+0000","last_active":"2023-10-16T11:06:25.188401+0000","last_peered":"2023-10-16T11:06:25.188401+0000","last_clean":"2023-10-16T11:06:25.188401+0000","last_became_active":"2023-10-16T11:06:23.614461+0000","last_became_peered":"2023-10-16T11:06:23.614461+0000","last_unstale":"2023-10-16T11:06:25.188401+0000","last_undegraded":"2023-10-16T11:06:25.188401+0000","last_fullsized":"2023-10-16T11:06:25.188401+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:45.286323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.980191+0000","last_change":"2023-10-16T11:06:23.614582+0000","last_active":"2023-10-16T11:06:24.980191+0000","last_peered":"2023-10-16T11:06:24.980191+0000","last_clean":"2023-10-16T11:06:24.980191+0000","last_became_active":"2023-10-16T11:06:23.614021+0000","last_became_peered":"2023-10-16T11:06:23.614021+0000","last_unstale":"2023-10-16T11:06:24.980191+0000","last_undegraded":"2023-10-16T11:06:24.980191+0000","last_fullsized":"2023-10-16T11:06:24.980191+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:07:59.718438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.981340+0000","last_change":"2023-10-16T11:06:23.623414+0000","last_active":"2023-10-16T11:06:24.981340+0000","last_peered":"2023-10-16T11:06:24.981340+0000","last_clean":"2023-10-16T11:06:24.981340+0000","last_became_active":"2023-10-16T11:06:23.623103+0000","last_became_peered":"2023-10-16T11:06:23.623103+0000","last_unstale":"2023-10-16T11:06:24.981340+0000","last_undegraded":"2023-10-16T11:06:24.981340+0000","last_fullsized":"2023-10-16T11:06:24.981340+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:38.166069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.980993+0000","last_change":"2023-10-16T11:06:23.615412+0000","last_active":"2023-10-16T11:06:24.980993+0000","last_peered":"2023-10-16T11:06:24.980993+0000","last_clean":"2023-10-16T11:06:24.980993+0000","last_became_active":"2023-10-16T11:06:23.614420+0000","last_became_peered":"2023-10-16T11:06:23.614420+0000","last_unstale":"2023-10-16T11:06:24.980993+0000","last_undegraded":"2023-10-16T11:06:24.980993+0000","last_fullsized":"2023-10-16T11:06:24.980993+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:04:22.692050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.980958+0000","last_change":"2023-10-16T11:06:23.617173+0000","last_active":"2023-10-16T11:06:24.980958+0000","last_peered":"2023-10-16T11:06:24.980958+0000","last_clean":"2023-10-16T11:06:24.980958+0000","last_became_active":"2023-10-16T11:06:23.614887+0000","last_became_peered":"2023-10-16T11:06:23.614887+0000","last_unstale":"2023-10-16T11:06:24.980958+0000","last_undegraded":"2023-10-16T11:06:24.980958+0000","last_fullsized":"2023-10-16T11:06:24.980958+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:25.865564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.980421+0000","last_change":"2023-10-16T11:06:23.617560+0000","last_active":"2023-10-16T11:06:24.980421+0000","last_peered":"2023-10-16T11:06:24.980421+0000","last_clean":"2023-10-16T11:06:24.980421+0000","last_became_active":"2023-10-16T11:06:23.617265+0000","last_became_peered":"2023-10-16T11:06:23.617265+0000","last_unstale":"2023-10-16T11:06:24.980421+0000","last_undegraded":"2023-10-16T11:06:24.980421+0000","last_fullsized":"2023-10-16T11:06:24.980421+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:22.405343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.691675+0000","last_change":"2023-10-16T11:06:23.612310+0000","last_active":"2023-10-16T11:06:25.691675+0000","last_peered":"2023-10-16T11:06:25.691675+0000","last_clean":"2023-10-16T11:06:25.691675+0000","last_became_active":"2023-10-16T11:06:23.611942+0000","last_became_peered":"2023-10-16T11:06:23.611942+0000","last_unstale":"2023-10-16T11:06:25.691675+0000","last_undegraded":"2023-10-16T11:06:25.691675+0000","last_fullsized":"2023-10-16T11:06:25.691675+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:42.960951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.690961+0000","last_change":"2023-10-16T11:06:23.614568+0000","last_active":"2023-10-16T11:06:25.690961+0000","last_peered":"2023-10-16T11:06:25.690961+0000","last_clean":"2023-10-16T11:06:25.690961+0000","last_became_active":"2023-10-16T11:06:23.614245+0000","last_became_peered":"2023-10-16T11:06:23.614245+0000","last_unstale":"2023-10-16T11:06:25.690961+0000","last_undegraded":"2023-10-16T11:06:25.690961+0000","last_fullsized":"2023-10-16T11:06:25.690961+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:57:33.492506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.689077+0000","last_change":"2023-10-16T11:06:23.602913+0000","last_active":"2023-10-16T11:06:25.689077+0000","last_peered":"2023-10-16T11:06:25.689077+0000","last_clean":"2023-10-16T11:06:25.689077+0000","last_became_active":"2023-10-16T11:06:23.602483+0000","last_became_peered":"2023-10-16T11:06:23.602483+0000","last_unstale":"2023-10-16T11:06:25.689077+0000","last_undegraded":"2023-10-16T11:06:25.689077+0000","last_fullsized":"2023-10-16T11:06:25.689077+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:03.691297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.188075+0000","last_change":"2023-10-16T11:06:23.613782+0000","last_active":"2023-10-16T11:06:25.188075+0000","last_peered":"2023-10-16T11:06:25.188075+0000","last_clean":"2023-10-16T11:06:25.188075+0000","last_became_active":"2023-10-16T11:06:23.610093+0000","last_became_peered":"2023-10-16T11:06:23.610093+0000","last_unstale":"2023-10-16T11:06:25.188075+0000","last_undegraded":"2023-10-16T11:06:25.188075+0000","last_fullsized":"2023-10-16T11:06:25.188075+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:38.166069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.605025+0000","last_change":"2023-10-16T11:06:23.603606+0000","last_active":"2023-10-16T11:06:24.605025+0000","last_peered":"2023-10-16T11:06:24.605025+0000","last_clean":"2023-10-16T11:06:24.605025+0000","last_became_active":"2023-10-16T11:06:23.602978+0000","last_became_peered":"2023-10-16T11:06:23.602978+0000","last_unstale":"2023-10-16T11:06:24.605025+0000","last_undegraded":"2023-10-16T11:06:24.605025+0000","last_fullsized":"2023-10-16T11:06:24.605025+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:58.721072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.608311+0000","last_change":"2023-10-16T11:06:23.599561+0000","last_active":"2023-10-16T11:06:24.608311+0000","last_peered":"2023-10-16T11:06:24.608311+0000","last_clean":"2023-10-16T11:06:24.608311+0000","last_became_active":"2023-10-16T11:06:23.598538+0000","last_became_peered":"2023-10-16T11:06:23.598538+0000","last_unstale":"2023-10-16T11:06:24.608311+0000","last_undegraded":"2023-10-16T11:06:24.608311+0000","last_fullsized":"2023-10-16T11:06:24.608311+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:29.386268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.605897+0000","last_change":"2023-10-16T11:06:23.594334+0000","last_active":"2023-10-16T11:06:24.605897+0000","last_peered":"2023-10-16T11:06:24.605897+0000","last_clean":"2023-10-16T11:06:24.605897+0000","last_became_active":"2023-10-16T11:06:23.593916+0000","last_became_peered":"2023-10-16T11:06:23.593916+0000","last_unstale":"2023-10-16T11:06:24.605897+0000","last_undegraded":"2023-10-16T11:06:24.605897+0000","last_fullsized":"2023-10-16T11:06:24.605897+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:29.249502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.691725+0000","last_change":"2023-10-16T11:06:23.612406+0000","last_active":"2023-10-16T11:06:25.691725+0000","last_peered":"2023-10-16T11:06:25.691725+0000","last_clean":"2023-10-16T11:06:25.691725+0000","last_became_active":"2023-10-16T11:06:23.612146+0000","last_became_peered":"2023-10-16T11:06:23.612146+0000","last_unstale":"2023-10-16T11:06:25.691725+0000","last_undegraded":"2023-10-16T11:06:25.691725+0000","last_fullsized":"2023-10-16T11:06:25.691725+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:34:55.741968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.605998+0000","last_change":"2023-10-16T11:06:23.611256+0000","last_active":"2023-10-16T11:06:24.605998+0000","last_peered":"2023-10-16T11:06:24.605998+0000","last_clean":"2023-10-16T11:06:24.605998+0000","last_became_active":"2023-10-16T11:06:23.610895+0000","last_became_peered":"2023-10-16T11:06:23.610895+0000","last_unstale":"2023-10-16T11:06:24.605998+0000","last_undegraded":"2023-10-16T11:06:24.605998+0000","last_fullsized":"2023-10-16T11:06:24.605998+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:24:10.847068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.980524+0000","last_change":"2023-10-16T11:06:23.621851+0000","last_active":"2023-10-16T11:06:24.980524+0000","last_peered":"2023-10-16T11:06:24.980524+0000","last_clean":"2023-10-16T11:06:24.980524+0000","last_became_active":"2023-10-16T11:06:23.621568+0000","last_became_peered":"2023-10-16T11:06:23.621568+0000","last_unstale":"2023-10-16T11:06:24.980524+0000","last_undegraded":"2023-10-16T11:06:24.980524+0000","last_fullsized":"2023-10-16T11:06:24.980524+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:42:35.497112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.605427+0000","last_change":"2023-10-16T11:06:23.610819+0000","last_active":"2023-10-16T11:06:24.605427+0000","last_peered":"2023-10-16T11:06:24.605427+0000","last_clean":"2023-10-16T11:06:24.605427+0000","last_became_active":"2023-10-16T11:06:23.607043+0000","last_became_peered":"2023-10-16T11:06:23.607043+0000","last_unstale":"2023-10-16T11:06:24.605427+0000","last_undegraded":"2023-10-16T11:06:24.605427+0000","last_fullsized":"2023-10-16T11:06:24.605427+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:05:42.824835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.980631+0000","last_change":"2023-10-16T11:06:23.610604+0000","last_active":"2023-10-16T11:06:24.980631+0000","last_peered":"2023-10-16T11:06:24.980631+0000","last_clean":"2023-10-16T11:06:24.980631+0000","last_became_active":"2023-10-16T11:06:23.609531+0000","last_became_peered":"2023-10-16T11:06:23.609531+0000","last_unstale":"2023-10-16T11:06:24.980631+0000","last_undegraded":"2023-10-16T11:06:24.980631+0000","last_fullsized":"2023-10-16T11:06:24.980631+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:43.851748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.692726+0000","last_change":"2023-10-16T11:06:23.610559+0000","last_active":"2023-10-16T11:06:25.692726+0000","last_peered":"2023-10-16T11:06:25.692726+0000","last_clean":"2023-10-16T11:06:25.692726+0000","last_became_active":"2023-10-16T11:06:23.610262+0000","last_became_peered":"2023-10-16T11:06:23.610262+0000","last_unstale":"2023-10-16T11:06:25.692726+0000","last_undegraded":"2023-10-16T11:06:25.692726+0000","last_fullsized":"2023-10-16T11:06:25.692726+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:28:12.369626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.189094+0000","last_change":"2023-10-16T11:06:23.617205+0000","last_active":"2023-10-16T11:06:25.189094+0000","last_peered":"2023-10-16T11:06:25.189094+0000","last_clean":"2023-10-16T11:06:25.189094+0000","last_became_active":"2023-10-16T11:06:23.616911+0000","last_became_peered":"2023-10-16T11:06:23.616911+0000","last_unstale":"2023-10-16T11:06:25.189094+0000","last_undegraded":"2023-10-16T11:06:25.189094+0000","last_fullsized":"2023-10-16T11:06:25.189094+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:05:42.824835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.692194+0000","last_change":"2023-10-16T11:06:23.601068+0000","last_active":"2023-10-16T11:06:25.692194+0000","last_peered":"2023-10-16T11:06:25.692194+0000","last_clean":"2023-10-16T11:06:25.692194+0000","last_became_active":"2023-10-16T11:06:23.600673+0000","last_became_peered":"2023-10-16T11:06:23.600673+0000","last_unstale":"2023-10-16T11:06:25.692194+0000","last_undegraded":"2023-10-16T11:06:25.692194+0000","last_fullsized":"2023-10-16T11:06:25.692194+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:11:47.595867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.682587+0000","last_change":"2023-10-16T11:06:23.612760+0000","last_active":"2023-10-16T11:06:24.682587+0000","last_peered":"2023-10-16T11:06:24.682587+0000","last_clean":"2023-10-16T11:06:24.682587+0000","last_became_active":"2023-10-16T11:06:23.612421+0000","last_became_peered":"2023-10-16T11:06:23.612421+0000","last_unstale":"2023-10-16T11:06:24.682587+0000","last_undegraded":"2023-10-16T11:06:24.682587+0000","last_fullsized":"2023-10-16T11:06:24.682587+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:00.640104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.691784+0000","last_change":"2023-10-16T11:06:23.607530+0000","last_active":"2023-10-16T11:06:25.691784+0000","last_peered":"2023-10-16T11:06:25.691784+0000","last_clean":"2023-10-16T11:06:25.691784+0000","last_became_active":"2023-10-16T11:06:23.605401+0000","last_became_peered":"2023-10-16T11:06:23.605401+0000","last_unstale":"2023-10-16T11:06:25.691784+0000","last_undegraded":"2023-10-16T11:06:25.691784+0000","last_fullsized":"2023-10-16T11:06:25.691784+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:56.865944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.607245+0000","last_change":"2023-10-16T11:06:23.604972+0000","last_active":"2023-10-16T11:06:24.607245+0000","last_peered":"2023-10-16T11:06:24.607245+0000","last_clean":"2023-10-16T11:06:24.607245+0000","last_became_active":"2023-10-16T11:06:23.604600+0000","last_became_peered":"2023-10-16T11:06:23.604600+0000","last_unstale":"2023-10-16T11:06:24.607245+0000","last_undegraded":"2023-10-16T11:06:24.607245+0000","last_fullsized":"2023-10-16T11:06:24.607245+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:00.640104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.607200+0000","last_change":"2023-10-16T11:06:23.609657+0000","last_active":"2023-10-16T11:06:24.607200+0000","last_peered":"2023-10-16T11:06:24.607200+0000","last_clean":"2023-10-16T11:06:24.607200+0000","last_became_active":"2023-10-16T11:06:23.607692+0000","last_became_peered":"2023-10-16T11:06:23.607692+0000","last_unstale":"2023-10-16T11:06:24.607200+0000","last_undegraded":"2023-10-16T11:06:24.607200+0000","last_fullsized":"2023-10-16T11:06:24.607200+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:11:30.976449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.607408+0000","last_change":"2023-10-16T11:06:23.594020+0000","last_active":"2023-10-16T11:06:24.607408+0000","last_peered":"2023-10-16T11:06:24.607408+0000","last_clean":"2023-10-16T11:06:24.607408+0000","last_became_active":"2023-10-16T11:06:23.593700+0000","last_became_peered":"2023-10-16T11:06:23.593700+0000","last_unstale":"2023-10-16T11:06:24.607408+0000","last_undegraded":"2023-10-16T11:06:24.607408+0000","last_fullsized":"2023-10-16T11:06:24.607408+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:54:26.214909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.189313+0000","last_change":"2023-10-16T11:06:23.610321+0000","last_active":"2023-10-16T11:06:25.189313+0000","last_peered":"2023-10-16T11:06:25.189313+0000","last_clean":"2023-10-16T11:06:25.189313+0000","last_became_active":"2023-10-16T11:06:23.606913+0000","last_became_peered":"2023-10-16T11:06:23.606913+0000","last_unstale":"2023-10-16T11:06:25.189313+0000","last_undegraded":"2023-10-16T11:06:25.189313+0000","last_fullsized":"2023-10-16T11:06:25.189313+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:53.501651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"183'4","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.625395+0000","last_change":"2023-10-16T11:06:22.605732+0000","last_active":"2023-10-16T11:06:23.625395+0000","last_peered":"2023-10-16T11:06:23.625395+0000","last_clean":"2023-10-16T11:06:23.625395+0000","last_became_active":"2023-10-16T11:06:22.604612+0000","last_became_peered":"2023-10-16T11:06:22.604612+0000","last_unstale":"2023-10-16T11:06:23.625395+0000","last_undegraded":"2023-10-16T11:06:23.625395+0000","last_fullsized":"2023-10-16T11:06:23.625395+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:07.029335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.608917+0000","last_change":"2023-10-16T11:06:23.597349+0000","last_active":"2023-10-16T11:06:24.608917+0000","last_peered":"2023-10-16T11:06:24.608917+0000","last_clean":"2023-10-16T11:06:24.608917+0000","last_became_active":"2023-10-16T11:06:23.596394+0000","last_became_peered":"2023-10-16T11:06:23.596394+0000","last_unstale":"2023-10-16T11:06:24.608917+0000","last_undegraded":"2023-10-16T11:06:24.608917+0000","last_fullsized":"2023-10-16T11:06:24.608917+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:07:18.531732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"183'1","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.625495+0000","last_change":"2023-10-16T11:06:22.593488+0000","last_active":"2023-10-16T11:06:23.625495+0000","last_peered":"2023-10-16T11:06:23.625495+0000","last_clean":"2023-10-16T11:06:23.625495+0000","last_became_active":"2023-10-16T11:06:22.592029+0000","last_became_peered":"2023-10-16T11:06:22.592029+0000","last_unstale":"2023-10-16T11:06:23.625495+0000","last_undegraded":"2023-10-16T11:06:23.625495+0000","last_fullsized":"2023-10-16T11:06:23.625495+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:22.856408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.604999+0000","last_change":"2023-10-16T11:06:23.605466+0000","last_active":"2023-10-16T11:06:24.604999+0000","last_peered":"2023-10-16T11:06:24.604999+0000","last_clean":"2023-10-16T11:06:24.604999+0000","last_became_active":"2023-10-16T11:06:23.605192+0000","last_became_peered":"2023-10-16T11:06:23.605192+0000","last_unstale":"2023-10-16T11:06:24.604999+0000","last_undegraded":"2023-10-16T11:06:24.604999+0000","last_fullsized":"2023-10-16T11:06:24.604999+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:22:52.230491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.625035+0000","last_change":"2023-10-16T11:06:22.602261+0000","last_active":"2023-10-16T11:06:23.625035+0000","last_peered":"2023-10-16T11:06:23.625035+0000","last_clean":"2023-10-16T11:06:23.625035+0000","last_became_active":"2023-10-16T11:06:22.601894+0000","last_became_peered":"2023-10-16T11:06:22.601894+0000","last_unstale":"2023-10-16T11:06:23.625035+0000","last_undegraded":"2023-10-16T11:06:23.625035+0000","last_fullsized":"2023-10-16T11:06:23.625035+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:34:54.722413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.692312+0000","last_change":"2023-10-16T11:06:23.605548+0000","last_active":"2023-10-16T11:06:25.692312+0000","last_peered":"2023-10-16T11:06:25.692312+0000","last_clean":"2023-10-16T11:06:25.692312+0000","last_became_active":"2023-10-16T11:06:23.602818+0000","last_became_peered":"2023-10-16T11:06:23.602818+0000","last_unstale":"2023-10-16T11:06:25.692312+0000","last_undegraded":"2023-10-16T11:06:25.692312+0000","last_fullsized":"2023-10-16T11:06:25.692312+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:15:05.089108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.623948+0000","last_change":"2023-10-16T11:06:22.606664+0000","last_active":"2023-10-16T11:06:23.623948+0000","last_peered":"2023-10-16T11:06:23.623948+0000","last_clean":"2023-10-16T11:06:23.623948+0000","last_became_active":"2023-10-16T11:06:22.606461+0000","last_became_peered":"2023-10-16T11:06:22.606461+0000","last_unstale":"2023-10-16T11:06:23.623948+0000","last_undegraded":"2023-10-16T11:06:23.623948+0000","last_fullsized":"2023-10-16T11:06:23.623948+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:33.775278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.188835+0000","last_change":"2023-10-16T11:06:23.617883+0000","last_active":"2023-10-16T11:06:25.188835+0000","last_peered":"2023-10-16T11:06:25.188835+0000","last_clean":"2023-10-16T11:06:25.188835+0000","last_became_active":"2023-10-16T11:06:23.617478+0000","last_became_peered":"2023-10-16T11:06:23.617478+0000","last_unstale":"2023-10-16T11:06:25.188835+0000","last_undegraded":"2023-10-16T11:06:25.188835+0000","last_fullsized":"2023-10-16T11:06:25.188835+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:44:42.439099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"183'1","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.625827+0000","last_change":"2023-10-16T11:06:22.600437+0000","last_active":"2023-10-16T11:06:23.625827+0000","last_peered":"2023-10-16T11:06:23.625827+0000","last_clean":"2023-10-16T11:06:23.625827+0000","last_became_active":"2023-10-16T11:06:22.599713+0000","last_became_peered":"2023-10-16T11:06:22.599713+0000","last_unstale":"2023-10-16T11:06:23.625827+0000","last_undegraded":"2023-10-16T11:06:23.625827+0000","last_fullsized":"2023-10-16T11:06:23.625827+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:28.229948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.682430+0000","last_change":"2023-10-16T11:06:23.619789+0000","last_active":"2023-10-16T11:06:24.682430+0000","last_peered":"2023-10-16T11:06:24.682430+0000","last_clean":"2023-10-16T11:06:24.682430+0000","last_became_active":"2023-10-16T11:06:23.619515+0000","last_became_peered":"2023-10-16T11:06:23.619515+0000","last_unstale":"2023-10-16T11:06:24.682430+0000","last_undegraded":"2023-10-16T11:06:24.682430+0000","last_fullsized":"2023-10-16T11:06:24.682430+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:11:30.976449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"183'1","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.623390+0000","last_change":"2023-10-16T11:06:22.597888+0000","last_active":"2023-10-16T11:06:23.623390+0000","last_peered":"2023-10-16T11:06:23.623390+0000","last_clean":"2023-10-16T11:06:23.623390+0000","last_became_active":"2023-10-16T11:06:22.597350+0000","last_became_peered":"2023-10-16T11:06:22.597350+0000","last_unstale":"2023-10-16T11:06:23.623390+0000","last_undegraded":"2023-10-16T11:06:23.623390+0000","last_fullsized":"2023-10-16T11:06:23.623390+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:22.856408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.682110+0000","last_change":"2023-10-16T11:06:23.614177+0000","last_active":"2023-10-16T11:06:24.682110+0000","last_peered":"2023-10-16T11:06:24.682110+0000","last_clean":"2023-10-16T11:06:24.682110+0000","last_became_active":"2023-10-16T11:06:23.613292+0000","last_became_peered":"2023-10-16T11:06:23.613292+0000","last_unstale":"2023-10-16T11:06:24.682110+0000","last_undegraded":"2023-10-16T11:06:24.682110+0000","last_fullsized":"2023-10-16T11:06:24.682110+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:52:43.765054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.596043+0000","last_change":"2023-10-16T11:06:22.596029+0000","last_active":"2023-10-16T11:06:23.596043+0000","last_peered":"2023-10-16T11:06:23.596043+0000","last_clean":"2023-10-16T11:06:23.596043+0000","last_became_active":"2023-10-16T11:06:22.595616+0000","last_became_peered":"2023-10-16T11:06:22.595616+0000","last_unstale":"2023-10-16T11:06:23.596043+0000","last_undegraded":"2023-10-16T11:06:23.596043+0000","last_fullsized":"2023-10-16T11:06:23.596043+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:05.357813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.605484+0000","last_change":"2023-10-16T11:06:23.608846+0000","last_active":"2023-10-16T11:06:24.605484+0000","last_peered":"2023-10-16T11:06:24.605484+0000","last_clean":"2023-10-16T11:06:24.605484+0000","last_became_active":"2023-10-16T11:06:23.607714+0000","last_became_peered":"2023-10-16T11:06:23.607714+0000","last_unstale":"2023-10-16T11:06:24.605484+0000","last_undegraded":"2023-10-16T11:06:24.605484+0000","last_fullsized":"2023-10-16T11:06:24.605484+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:40.441635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.594690+0000","last_change":"2023-10-16T11:06:22.585438+0000","last_active":"2023-10-16T11:06:23.594690+0000","last_peered":"2023-10-16T11:06:23.594690+0000","last_clean":"2023-10-16T11:06:23.594690+0000","last_became_active":"2023-10-16T11:06:22.584253+0000","last_became_peered":"2023-10-16T11:06:22.584253+0000","last_unstale":"2023-10-16T11:06:23.594690+0000","last_undegraded":"2023-10-16T11:06:23.594690+0000","last_fullsized":"2023-10-16T11:06:23.594690+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:21.385789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.608422+0000","last_change":"2023-10-16T11:06:23.606712+0000","last_active":"2023-10-16T11:06:24.608422+0000","last_peered":"2023-10-16T11:06:24.608422+0000","last_clean":"2023-10-16T11:06:24.608422+0000","last_became_active":"2023-10-16T11:06:23.606253+0000","last_became_peered":"2023-10-16T11:06:23.606253+0000","last_unstale":"2023-10-16T11:06:24.608422+0000","last_undegraded":"2023-10-16T11:06:24.608422+0000","last_fullsized":"2023-10-16T11:06:24.608422+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:11.282478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.582956+0000","last_change":"2023-10-16T11:06:22.600873+0000","last_active":"2023-10-16T11:06:23.582956+0000","last_peered":"2023-10-16T11:06:23.582956+0000","last_clean":"2023-10-16T11:06:23.582956+0000","last_became_active":"2023-10-16T11:06:22.600547+0000","last_became_peered":"2023-10-16T11:06:22.600547+0000","last_unstale":"2023-10-16T11:06:23.582956+0000","last_undegraded":"2023-10-16T11:06:23.582956+0000","last_fullsized":"2023-10-16T11:06:23.582956+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:37.146515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.607971+0000","last_change":"2023-10-16T11:06:23.603842+0000","last_active":"2023-10-16T11:06:24.607971+0000","last_peered":"2023-10-16T11:06:24.607971+0000","last_clean":"2023-10-16T11:06:24.607971+0000","last_became_active":"2023-10-16T11:06:23.603588+0000","last_became_peered":"2023-10-16T11:06:23.603588+0000","last_unstale":"2023-10-16T11:06:24.607971+0000","last_undegraded":"2023-10-16T11:06:24.607971+0000","last_fullsized":"2023-10-16T11:06:24.607971+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:31:39.879188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"183'3","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.624770+0000","last_change":"2023-10-16T11:06:22.601095+0000","last_active":"2023-10-16T11:06:23.624770+0000","last_peered":"2023-10-16T11:06:23.624770+0000","last_clean":"2023-10-16T11:06:23.624770+0000","last_became_active":"2023-10-16T11:06:22.600809+0000","last_became_peered":"2023-10-16T11:06:22.600809+0000","last_unstale":"2023-10-16T11:06:23.624770+0000","last_undegraded":"2023-10-16T11:06:23.624770+0000","last_fullsized":"2023-10-16T11:06:23.624770+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:24.846010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.608336+0000","last_change":"2023-10-16T11:06:23.607578+0000","last_active":"2023-10-16T11:06:24.608336+0000","last_peered":"2023-10-16T11:06:24.608336+0000","last_clean":"2023-10-16T11:06:24.608336+0000","last_became_active":"2023-10-16T11:06:23.607238+0000","last_became_peered":"2023-10-16T11:06:23.607238+0000","last_unstale":"2023-10-16T11:06:24.608336+0000","last_undegraded":"2023-10-16T11:06:24.608336+0000","last_fullsized":"2023-10-16T11:06:24.608336+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:10:51.251237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"183'6","reported_seq":18,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.629331+0000","last_change":"2023-10-16T11:06:22.602204+0000","last_active":"2023-10-16T11:06:23.629331+0000","last_peered":"2023-10-16T11:06:23.629331+0000","last_clean":"2023-10-16T11:06:23.629331+0000","last_became_active":"2023-10-16T11:06:22.601239+0000","last_became_peered":"2023-10-16T11:06:22.601239+0000","last_unstale":"2023-10-16T11:06:23.629331+0000","last_undegraded":"2023-10-16T11:06:23.629331+0000","last_fullsized":"2023-10-16T11:06:23.629331+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:33.775278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.607564+0000","last_change":"2023-10-16T11:06:23.607824+0000","last_active":"2023-10-16T11:06:24.607564+0000","last_peered":"2023-10-16T11:06:24.607564+0000","last_clean":"2023-10-16T11:06:24.607564+0000","last_became_active":"2023-10-16T11:06:23.607557+0000","last_became_peered":"2023-10-16T11:06:23.607557+0000","last_unstale":"2023-10-16T11:06:24.607564+0000","last_undegraded":"2023-10-16T11:06:24.607564+0000","last_fullsized":"2023-10-16T11:06:24.607564+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:37:07.386405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.624019+0000","last_change":"2023-10-16T11:06:22.598426+0000","last_active":"2023-10-16T11:06:23.624019+0000","last_peered":"2023-10-16T11:06:23.624019+0000","last_clean":"2023-10-16T11:06:23.624019+0000","last_became_active":"2023-10-16T11:06:22.597766+0000","last_became_peered":"2023-10-16T11:06:22.597766+0000","last_unstale":"2023-10-16T11:06:23.624019+0000","last_undegraded":"2023-10-16T11:06:23.624019+0000","last_fullsized":"2023-10-16T11:06:23.624019+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:15:04.069554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.188867+0000","last_change":"2023-10-16T11:06:23.612172+0000","last_active":"2023-10-16T11:06:25.188867+0000","last_peered":"2023-10-16T11:06:25.188867+0000","last_clean":"2023-10-16T11:06:25.188867+0000","last_became_active":"2023-10-16T11:06:23.611776+0000","last_became_peered":"2023-10-16T11:06:23.611776+0000","last_unstale":"2023-10-16T11:06:25.188867+0000","last_undegraded":"2023-10-16T11:06:25.188867+0000","last_fullsized":"2023-10-16T11:06:25.188867+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:22:52.230491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.580924+0000","last_change":"2023-10-16T11:06:22.593856+0000","last_active":"2023-10-16T11:06:23.580924+0000","last_peered":"2023-10-16T11:06:23.580924+0000","last_clean":"2023-10-16T11:06:23.580924+0000","last_became_active":"2023-10-16T11:06:22.593580+0000","last_became_peered":"2023-10-16T11:06:22.593580+0000","last_unstale":"2023-10-16T11:06:23.580924+0000","last_undegraded":"2023-10-16T11:06:23.580924+0000","last_fullsized":"2023-10-16T11:06:23.580924+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:22:31.426812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.682111+0000","last_change":"2023-10-16T11:06:23.609053+0000","last_active":"2023-10-16T11:06:24.682111+0000","last_peered":"2023-10-16T11:06:24.682111+0000","last_clean":"2023-10-16T11:06:24.682111+0000","last_became_active":"2023-10-16T11:06:23.608765+0000","last_became_peered":"2023-10-16T11:06:23.608765+0000","last_unstale":"2023-10-16T11:06:24.682111+0000","last_undegraded":"2023-10-16T11:06:24.682111+0000","last_fullsized":"2023-10-16T11:06:24.682111+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:49:24.469079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.593027+0000","last_change":"2023-10-16T11:06:22.600102+0000","last_active":"2023-10-16T11:06:23.593027+0000","last_peered":"2023-10-16T11:06:23.593027+0000","last_clean":"2023-10-16T11:06:23.593027+0000","last_became_active":"2023-10-16T11:06:22.599762+0000","last_became_peered":"2023-10-16T11:06:22.599762+0000","last_unstale":"2023-10-16T11:06:23.593027+0000","last_undegraded":"2023-10-16T11:06:23.593027+0000","last_fullsized":"2023-10-16T11:06:23.593027+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:03.924849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.690834+0000","last_change":"2023-10-16T11:06:23.618703+0000","last_active":"2023-10-16T11:06:25.690834+0000","last_peered":"2023-10-16T11:06:25.690834+0000","last_clean":"2023-10-16T11:06:25.690834+0000","last_became_active":"2023-10-16T11:06:23.618407+0000","last_became_peered":"2023-10-16T11:06:23.618407+0000","last_unstale":"2023-10-16T11:06:25.690834+0000","last_undegraded":"2023-10-16T11:06:25.690834+0000","last_fullsized":"2023-10-16T11:06:25.690834+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:47:13.076842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"183'8","reported_seq":20,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.626698+0000","last_change":"2023-10-16T11:06:22.600038+0000","last_active":"2023-10-16T11:06:23.626698+0000","last_peered":"2023-10-16T11:06:23.626698+0000","last_clean":"2023-10-16T11:06:23.626698+0000","last_became_active":"2023-10-16T11:06:22.599724+0000","last_became_peered":"2023-10-16T11:06:22.599724+0000","last_unstale":"2023-10-16T11:06:23.626698+0000","last_undegraded":"2023-10-16T11:06:23.626698+0000","last_fullsized":"2023-10-16T11:06:23.626698+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:05:00.713229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.682741+0000","last_change":"2023-10-16T11:06:23.604069+0000","last_active":"2023-10-16T11:06:24.682741+0000","last_peered":"2023-10-16T11:06:24.682741+0000","last_clean":"2023-10-16T11:06:24.682741+0000","last_became_active":"2023-10-16T11:06:23.603632+0000","last_became_peered":"2023-10-16T11:06:23.603632+0000","last_unstale":"2023-10-16T11:06:24.682741+0000","last_undegraded":"2023-10-16T11:06:24.682741+0000","last_fullsized":"2023-10-16T11:06:24.682741+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:07:18.531732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"183'1","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.625377+0000","last_change":"2023-10-16T11:06:22.603319+0000","last_active":"2023-10-16T11:06:23.625377+0000","last_peered":"2023-10-16T11:06:23.625377+0000","last_clean":"2023-10-16T11:06:23.625377+0000","last_became_active":"2023-10-16T11:06:22.603029+0000","last_became_peered":"2023-10-16T11:06:22.603029+0000","last_unstale":"2023-10-16T11:06:23.625377+0000","last_undegraded":"2023-10-16T11:06:23.625377+0000","last_fullsized":"2023-10-16T11:06:23.625377+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:22:51.210936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.608424+0000","last_change":"2023-10-16T11:06:23.598934+0000","last_active":"2023-10-16T11:06:24.608424+0000","last_peered":"2023-10-16T11:06:24.608424+0000","last_clean":"2023-10-16T11:06:24.608424+0000","last_became_active":"2023-10-16T11:06:23.598635+0000","last_became_peered":"2023-10-16T11:06:23.598635+0000","last_unstale":"2023-10-16T11:06:24.608424+0000","last_undegraded":"2023-10-16T11:06:24.608424+0000","last_fullsized":"2023-10-16T11:06:24.608424+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:33:13.769975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.622238+0000","last_change":"2023-10-16T11:06:22.598212+0000","last_active":"2023-10-16T11:06:23.622238+0000","last_peered":"2023-10-16T11:06:23.622238+0000","last_clean":"2023-10-16T11:06:23.622238+0000","last_became_active":"2023-10-16T11:06:22.597864+0000","last_became_peered":"2023-10-16T11:06:22.597864+0000","last_unstale":"2023-10-16T11:06:23.622238+0000","last_undegraded":"2023-10-16T11:06:23.622238+0000","last_fullsized":"2023-10-16T11:06:23.622238+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:39.422081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.682729+0000","last_change":"2023-10-16T11:06:23.620695+0000","last_active":"2023-10-16T11:06:24.682729+0000","last_peered":"2023-10-16T11:06:24.682729+0000","last_clean":"2023-10-16T11:06:24.682729+0000","last_became_active":"2023-10-16T11:06:23.620431+0000","last_became_peered":"2023-10-16T11:06:23.620431+0000","last_unstale":"2023-10-16T11:06:24.682729+0000","last_undegraded":"2023-10-16T11:06:24.682729+0000","last_fullsized":"2023-10-16T11:06:24.682729+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:10:51.251237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"183'1","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.625933+0000","last_change":"2023-10-16T11:06:22.605078+0000","last_active":"2023-10-16T11:06:23.625933+0000","last_peered":"2023-10-16T11:06:23.625933+0000","last_clean":"2023-10-16T11:06:23.625933+0000","last_became_active":"2023-10-16T11:06:22.604777+0000","last_became_peered":"2023-10-16T11:06:22.604777+0000","last_unstale":"2023-10-16T11:06:23.625933+0000","last_undegraded":"2023-10-16T11:06:23.625933+0000","last_fullsized":"2023-10-16T11:06:23.625933+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:41:07.425104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.188987+0000","last_change":"2023-10-16T11:06:23.611858+0000","last_active":"2023-10-16T11:06:25.188987+0000","last_peered":"2023-10-16T11:06:25.188987+0000","last_clean":"2023-10-16T11:06:25.188987+0000","last_became_active":"2023-10-16T11:06:23.611532+0000","last_became_peered":"2023-10-16T11:06:23.611532+0000","last_unstale":"2023-10-16T11:06:25.188987+0000","last_undegraded":"2023-10-16T11:06:25.188987+0000","last_fullsized":"2023-10-16T11:06:25.188987+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:27.107177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.589557+0000","last_change":"2023-10-16T11:06:22.603376+0000","last_active":"2023-10-16T11:06:23.589557+0000","last_peered":"2023-10-16T11:06:23.589557+0000","last_clean":"2023-10-16T11:06:23.589557+0000","last_became_active":"2023-10-16T11:06:22.602995+0000","last_became_peered":"2023-10-16T11:06:22.602995+0000","last_unstale":"2023-10-16T11:06:23.589557+0000","last_undegraded":"2023-10-16T11:06:23.589557+0000","last_fullsized":"2023-10-16T11:06:23.589557+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:00.081712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.691324+0000","last_change":"2023-10-16T11:06:23.613081+0000","last_active":"2023-10-16T11:06:25.691324+0000","last_peered":"2023-10-16T11:06:25.691324+0000","last_clean":"2023-10-16T11:06:25.691324+0000","last_became_active":"2023-10-16T11:06:23.611374+0000","last_became_peered":"2023-10-16T11:06:23.611374+0000","last_unstale":"2023-10-16T11:06:25.691324+0000","last_undegraded":"2023-10-16T11:06:25.691324+0000","last_fullsized":"2023-10-16T11:06:25.691324+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:28:21.762166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.625321+0000","last_change":"2023-10-16T11:06:22.599579+0000","last_active":"2023-10-16T11:06:23.625321+0000","last_peered":"2023-10-16T11:06:23.625321+0000","last_clean":"2023-10-16T11:06:23.625321+0000","last_became_active":"2023-10-16T11:06:22.599013+0000","last_became_peered":"2023-10-16T11:06:22.599013+0000","last_unstale":"2023-10-16T11:06:23.625321+0000","last_undegraded":"2023-10-16T11:06:23.625321+0000","last_fullsized":"2023-10-16T11:06:23.625321+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:28:11.350072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.188995+0000","last_change":"2023-10-16T11:06:23.609436+0000","last_active":"2023-10-16T11:06:25.188995+0000","last_peered":"2023-10-16T11:06:25.188995+0000","last_clean":"2023-10-16T11:06:25.188995+0000","last_became_active":"2023-10-16T11:06:23.608957+0000","last_became_peered":"2023-10-16T11:06:23.608957+0000","last_unstale":"2023-10-16T11:06:25.188995+0000","last_undegraded":"2023-10-16T11:06:25.188995+0000","last_fullsized":"2023-10-16T11:06:25.188995+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:58.721072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.588628+0000","last_change":"2023-10-16T11:06:22.602334+0000","last_active":"2023-10-16T11:06:23.588628+0000","last_peered":"2023-10-16T11:06:23.588628+0000","last_clean":"2023-10-16T11:06:23.588628+0000","last_became_active":"2023-10-16T11:06:22.600968+0000","last_became_peered":"2023-10-16T11:06:22.600968+0000","last_unstale":"2023-10-16T11:06:23.588628+0000","last_undegraded":"2023-10-16T11:06:23.588628+0000","last_fullsized":"2023-10-16T11:06:23.588628+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:46:00.461827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.689432+0000","last_change":"2023-10-16T11:06:23.611058+0000","last_active":"2023-10-16T11:06:25.689432+0000","last_peered":"2023-10-16T11:06:25.689432+0000","last_clean":"2023-10-16T11:06:25.689432+0000","last_became_active":"2023-10-16T11:06:23.610730+0000","last_became_peered":"2023-10-16T11:06:23.610730+0000","last_unstale":"2023-10-16T11:06:25.689432+0000","last_undegraded":"2023-10-16T11:06:25.689432+0000","last_fullsized":"2023-10-16T11:06:25.689432+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:19.812532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.593339+0000","last_change":"2023-10-16T11:06:22.601790+0000","last_active":"2023-10-16T11:06:23.593339+0000","last_peered":"2023-10-16T11:06:23.593339+0000","last_clean":"2023-10-16T11:06:23.593339+0000","last_became_active":"2023-10-16T11:06:22.601481+0000","last_became_peered":"2023-10-16T11:06:22.601481+0000","last_unstale":"2023-10-16T11:06:23.593339+0000","last_undegraded":"2023-10-16T11:06:23.593339+0000","last_fullsized":"2023-10-16T11:06:23.593339+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:04:24.297177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.605763+0000","last_change":"2023-10-16T11:06:23.609513+0000","last_active":"2023-10-16T11:06:24.605763+0000","last_peered":"2023-10-16T11:06:24.605763+0000","last_clean":"2023-10-16T11:06:24.605763+0000","last_became_active":"2023-10-16T11:06:23.609147+0000","last_became_peered":"2023-10-16T11:06:23.609147+0000","last_unstale":"2023-10-16T11:06:24.605763+0000","last_undegraded":"2023-10-16T11:06:24.605763+0000","last_fullsized":"2023-10-16T11:06:24.605763+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:53.501651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.590572+0000","last_change":"2023-10-16T11:06:22.603971+0000","last_active":"2023-10-16T11:06:23.590572+0000","last_peered":"2023-10-16T11:06:23.590572+0000","last_clean":"2023-10-16T11:06:23.590572+0000","last_became_active":"2023-10-16T11:06:22.603652+0000","last_became_peered":"2023-10-16T11:06:22.603652+0000","last_unstale":"2023-10-16T11:06:23.590572+0000","last_undegraded":"2023-10-16T11:06:23.590572+0000","last_fullsized":"2023-10-16T11:06:23.590572+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:06:42.071943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.608691+0000","last_change":"2023-10-16T11:06:23.604780+0000","last_active":"2023-10-16T11:06:24.608691+0000","last_peered":"2023-10-16T11:06:24.608691+0000","last_clean":"2023-10-16T11:06:24.608691+0000","last_became_active":"2023-10-16T11:06:23.602656+0000","last_became_peered":"2023-10-16T11:06:23.602656+0000","last_unstale":"2023-10-16T11:06:24.608691+0000","last_undegraded":"2023-10-16T11:06:24.608691+0000","last_fullsized":"2023-10-16T11:06:24.608691+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:28:20.180992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.625450+0000","last_change":"2023-10-16T11:06:22.591878+0000","last_active":"2023-10-16T11:06:23.625450+0000","last_peered":"2023-10-16T11:06:23.625450+0000","last_clean":"2023-10-16T11:06:23.625450+0000","last_became_active":"2023-10-16T11:06:22.591618+0000","last_became_peered":"2023-10-16T11:06:22.591618+0000","last_unstale":"2023-10-16T11:06:23.625450+0000","last_undegraded":"2023-10-16T11:06:23.625450+0000","last_fullsized":"2023-10-16T11:06:23.625450+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:22:36.576896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.189313+0000","last_change":"2023-10-16T11:06:23.617435+0000","last_active":"2023-10-16T11:06:25.189313+0000","last_peered":"2023-10-16T11:06:25.189313+0000","last_clean":"2023-10-16T11:06:25.189313+0000","last_became_active":"2023-10-16T11:06:23.617176+0000","last_became_peered":"2023-10-16T11:06:23.617176+0000","last_unstale":"2023-10-16T11:06:25.189313+0000","last_undegraded":"2023-10-16T11:06:25.189313+0000","last_fullsized":"2023-10-16T11:06:25.189313+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:40.441635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.596694+0000","last_change":"2023-10-16T11:06:22.602848+0000","last_active":"2023-10-16T11:06:23.596694+0000","last_peered":"2023-10-16T11:06:23.596694+0000","last_clean":"2023-10-16T11:06:23.596694+0000","last_became_active":"2023-10-16T11:06:22.600563+0000","last_became_peered":"2023-10-16T11:06:22.600563+0000","last_unstale":"2023-10-16T11:06:23.596694+0000","last_undegraded":"2023-10-16T11:06:23.596694+0000","last_fullsized":"2023-10-16T11:06:23.596694+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:52.482097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.605364+0000","last_change":"2023-10-16T11:06:23.598565+0000","last_active":"2023-10-16T11:06:24.605364+0000","last_peered":"2023-10-16T11:06:24.605364+0000","last_clean":"2023-10-16T11:06:24.605364+0000","last_became_active":"2023-10-16T11:06:23.598298+0000","last_became_peered":"2023-10-16T11:06:23.598298+0000","last_unstale":"2023-10-16T11:06:24.605364+0000","last_undegraded":"2023-10-16T11:06:24.605364+0000","last_fullsized":"2023-10-16T11:06:24.605364+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:27.107177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.599169+0000","last_change":"2023-10-16T11:06:22.597633+0000","last_active":"2023-10-16T11:06:23.599169+0000","last_peered":"2023-10-16T11:06:23.599169+0000","last_clean":"2023-10-16T11:06:23.599169+0000","last_became_active":"2023-10-16T11:06:22.597321+0000","last_became_peered":"2023-10-16T11:06:22.597321+0000","last_unstale":"2023-10-16T11:06:23.599169+0000","last_undegraded":"2023-10-16T11:06:23.599169+0000","last_fullsized":"2023-10-16T11:06:23.599169+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:47:30.215796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.608073+0000","last_change":"2023-10-16T11:06:23.608326+0000","last_active":"2023-10-16T11:06:24.608073+0000","last_peered":"2023-10-16T11:06:24.608073+0000","last_clean":"2023-10-16T11:06:24.608073+0000","last_became_active":"2023-10-16T11:06:23.608006+0000","last_became_peered":"2023-10-16T11:06:23.608006+0000","last_unstale":"2023-10-16T11:06:24.608073+0000","last_undegraded":"2023-10-16T11:06:24.608073+0000","last_fullsized":"2023-10-16T11:06:24.608073+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:49.381022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.579231+0000","last_change":"2023-10-16T11:06:22.598914+0000","last_active":"2023-10-16T11:06:23.579231+0000","last_peered":"2023-10-16T11:06:23.579231+0000","last_clean":"2023-10-16T11:06:23.579231+0000","last_became_active":"2023-10-16T11:06:22.598150+0000","last_became_peered":"2023-10-16T11:06:22.598150+0000","last_unstale":"2023-10-16T11:06:23.579231+0000","last_undegraded":"2023-10-16T11:06:23.579231+0000","last_fullsized":"2023-10-16T11:06:23.579231+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:47.148066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.607659+0000","last_change":"2023-10-16T11:06:23.591515+0000","last_active":"2023-10-16T11:06:24.607659+0000","last_peered":"2023-10-16T11:06:24.607659+0000","last_clean":"2023-10-16T11:06:24.607659+0000","last_became_active":"2023-10-16T11:06:23.591080+0000","last_became_peered":"2023-10-16T11:06:23.591080+0000","last_unstale":"2023-10-16T11:06:24.607659+0000","last_undegraded":"2023-10-16T11:06:24.607659+0000","last_fullsized":"2023-10-16T11:06:24.607659+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:41.358248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.580150+0000","last_change":"2023-10-16T11:06:22.593798+0000","last_active":"2023-10-16T11:06:23.580150+0000","last_peered":"2023-10-16T11:06:23.580150+0000","last_clean":"2023-10-16T11:06:23.580150+0000","last_became_active":"2023-10-16T11:06:22.593451+0000","last_became_peered":"2023-10-16T11:06:22.593451+0000","last_unstale":"2023-10-16T11:06:23.580150+0000","last_undegraded":"2023-10-16T11:06:23.580150+0000","last_fullsized":"2023-10-16T11:06:23.580150+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:49:23.449525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.607670+0000","last_change":"2023-10-16T11:06:23.608011+0000","last_active":"2023-10-16T11:06:24.607670+0000","last_peered":"2023-10-16T11:06:24.607670+0000","last_clean":"2023-10-16T11:06:24.607670+0000","last_became_active":"2023-10-16T11:06:23.607460+0000","last_became_peered":"2023-10-16T11:06:23.607460+0000","last_unstale":"2023-10-16T11:06:24.607670+0000","last_undegraded":"2023-10-16T11:06:24.607670+0000","last_fullsized":"2023-10-16T11:06:24.607670+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:56:48.322204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.596786+0000","last_change":"2023-10-16T11:06:22.591391+0000","last_active":"2023-10-16T11:06:23.596786+0000","last_peered":"2023-10-16T11:06:23.596786+0000","last_clean":"2023-10-16T11:06:23.596786+0000","last_became_active":"2023-10-16T11:06:22.590826+0000","last_became_peered":"2023-10-16T11:06:22.590826+0000","last_unstale":"2023-10-16T11:06:23.596786+0000","last_undegraded":"2023-10-16T11:06:23.596786+0000","last_fullsized":"2023-10-16T11:06:23.596786+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:05:00.713229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.188098+0000","last_change":"2023-10-16T11:06:23.607961+0000","last_active":"2023-10-16T11:06:25.188098+0000","last_peered":"2023-10-16T11:06:25.188098+0000","last_clean":"2023-10-16T11:06:25.188098+0000","last_became_active":"2023-10-16T11:06:23.607661+0000","last_became_peered":"2023-10-16T11:06:23.607661+0000","last_unstale":"2023-10-16T11:06:25.188098+0000","last_undegraded":"2023-10-16T11:06:25.188098+0000","last_fullsized":"2023-10-16T11:06:25.188098+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:33:49.051160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.596232+0000","last_change":"2023-10-16T11:06:22.592760+0000","last_active":"2023-10-16T11:06:23.596232+0000","last_peered":"2023-10-16T11:06:23.596232+0000","last_clean":"2023-10-16T11:06:23.596232+0000","last_became_active":"2023-10-16T11:06:22.592476+0000","last_became_peered":"2023-10-16T11:06:22.592476+0000","last_unstale":"2023-10-16T11:06:23.596232+0000","last_undegraded":"2023-10-16T11:06:23.596232+0000","last_fullsized":"2023-10-16T11:06:23.596232+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:10:14.100974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.690857+0000","last_change":"2023-10-16T11:06:23.606624+0000","last_active":"2023-10-16T11:06:25.690857+0000","last_peered":"2023-10-16T11:06:25.690857+0000","last_clean":"2023-10-16T11:06:25.690857+0000","last_became_active":"2023-10-16T11:06:23.605844+0000","last_became_peered":"2023-10-16T11:06:23.605844+0000","last_unstale":"2023-10-16T11:06:25.690857+0000","last_undegraded":"2023-10-16T11:06:25.690857+0000","last_fullsized":"2023-10-16T11:06:25.690857+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:28:03.417569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.579944+0000","last_change":"2023-10-16T11:06:22.594879+0000","last_active":"2023-10-16T11:06:23.579944+0000","last_peered":"2023-10-16T11:06:23.579944+0000","last_clean":"2023-10-16T11:06:23.579944+0000","last_became_active":"2023-10-16T11:06:22.590604+0000","last_became_peered":"2023-10-16T11:06:22.590604+0000","last_unstale":"2023-10-16T11:06:23.579944+0000","last_undegraded":"2023-10-16T11:06:23.579944+0000","last_fullsized":"2023-10-16T11:06:23.579944+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:00.276431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.608692+0000","last_change":"2023-10-16T11:06:23.604954+0000","last_active":"2023-10-16T11:06:24.608692+0000","last_peered":"2023-10-16T11:06:24.608692+0000","last_clean":"2023-10-16T11:06:24.608692+0000","last_became_active":"2023-10-16T11:06:23.604416+0000","last_became_peered":"2023-10-16T11:06:23.604416+0000","last_unstale":"2023-10-16T11:06:24.608692+0000","last_undegraded":"2023-10-16T11:06:24.608692+0000","last_fullsized":"2023-10-16T11:06:24.608692+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:21.726501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.580612+0000","last_change":"2023-10-16T11:06:22.595710+0000","last_active":"2023-10-16T11:06:23.580612+0000","last_peered":"2023-10-16T11:06:23.580612+0000","last_clean":"2023-10-16T11:06:23.580612+0000","last_became_active":"2023-10-16T11:06:22.593479+0000","last_became_peered":"2023-10-16T11:06:22.593479+0000","last_unstale":"2023-10-16T11:06:23.580612+0000","last_undegraded":"2023-10-16T11:06:23.580612+0000","last_fullsized":"2023-10-16T11:06:23.580612+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:52:42.745500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"173'2712","reported_seq":3420,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.590797+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:06:23.590797+0000","last_peered":"2023-10-16T11:06:23.590797+0000","last_clean":"2023-10-16T11:06:23.590797+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:06:23.590797+0000","last_undegraded":"2023-10-16T11:06:23.590797+0000","last_fullsized":"2023-10-16T11:06:23.590797+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2712,"log_dups_size":0,"ondisk_log_size":2712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1696,"num_read_kb":3438,"num_write":481,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1696,"num_read_kb":3438,"num_write":481,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2712,"ondisk_log_size":2712,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575772,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47920,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323920,"statfs":{"total":96636764160,"available":96587694080,"internally_reserved":0,"allocated":2920448,"data_stored":1701135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1739999999999999,"15min":1.2030000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6479999999999999,"5min":1.7230000000000001,"15min":1.853},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.2090000000000001,"15min":1.226},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.67300000000000004},"max":{"1min":1.4730000000000001,"5min":1.8029999999999999,"15min":1.9710000000000001},"last":0.871}]},{"osd":1,"last update":"Mon Oct 16 11:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2370000000000001,"15min":1.26},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.4990000000000001,"5min":1.796,"15min":1.907},"last":1.802},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.2350000000000001,"15min":1.2589999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.444,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.95299999999999996}]},{"osd":2,"last update":"Mon Oct 16 11:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.3400000000000001,"15min":1.3129999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.6519999999999999,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.524},{"interface":"front","average":{"1min":1.232,"5min":1.3580000000000001,"15min":1.3420000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.79100000000000004},"max":{"1min":1.5640000000000001,"5min":1.869,"15min":1.869},"last":1.2709999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.357,"15min":1.3680000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.5069999999999999,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.478,"15min":1.456},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.873,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.645}]},{"osd":4,"last update":"Mon Oct 16 11:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":1.444,"15min":1.4099999999999999},"min":{"1min":1.036,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":2.016,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.7569999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.3620000000000001,"15min":1.3540000000000001},"min":{"1min":0.87,"5min":0.87,"15min":0.74299999999999999},"max":{"1min":1.75,"5min":1.909,"15min":2.101},"last":1.978}]},{"osd":5,"last update":"Mon Oct 16 11:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.609,"15min":1.5640000000000001},"min":{"1min":1.21,"5min":0.99199999999999999,"15min":0.98799999999999999},"max":{"1min":2.0409999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.897},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.498,"15min":1.486},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.9770000000000001,"5min":2.0249999999999999,"15min":2.395},"last":2.0819999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.512,"15min":1.4850000000000001},"min":{"1min":1.0920000000000001,"5min":0.89200000000000002,"15min":0.88500000000000001},"max":{"1min":1.71,"5min":2.0680000000000001,"15min":2.1459999999999999},"last":1.931},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.7290000000000001,"15min":1.6839999999999999},"min":{"1min":1.415,"5min":1.093,"15min":1.093},"max":{"1min":2.1059999999999999,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":2.0299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575772,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29272,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342568,"statfs":{"total":96636764160,"available":96606789632,"internally_reserved":0,"allocated":2306048,"data_stored":1108435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:06:05 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.222,"15min":1.202},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.5209999999999999,"5min":1.665,"15min":1.665},"last":1.212},{"interface":"front","average":{"1min":1.175,"5min":1.256,"15min":1.264},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.0469999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:06:05 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.25,"15min":1.234},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7390000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.7609999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.3540000000000001,"15min":1.343},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.754,"5min":1.8220000000000001,"15min":2.1440000000000001},"last":1.431}]},{"osd":2,"last update":"Mon Oct 16 11:06:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.262,"15min":1.2589999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.696,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3740000000000001,"15min":1.3879999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6559999999999999,"5min":1.988,"15min":2.4129999999999998},"last":1.9390000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:06:05 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.272,"15min":1.2649999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.536},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.254,"15min":1.26},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.8300000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:06:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.319,"15min":1.339},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.845,"5min":1.9159999999999999,"15min":2.2130000000000001},"last":2.145},{"interface":"front","average":{"1min":1.462,"5min":1.5189999999999999,"15min":1.492},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.2490000000000001},"last":1.704}]},{"osd":5,"last update":"Mon Oct 16 11:06:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.3899999999999999,"15min":1.355},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.016,"5min":2.016,"15min":2.1869999999999998},"last":1.9870000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5820000000000001,"15min":1.575},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.9810000000000001,"5min":2.056,"15min":2.54},"last":2.0510000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:06:05 2023","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.534,"15min":1.4910000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.897},{"interface":"front","average":{"1min":1.641,"5min":1.5920000000000001,"15min":1.5529999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.2050000000000001},"last":2.0870000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575772,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41756,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330084,"statfs":{"total":96636764160,"available":96594006016,"internally_reserved":0,"allocated":2310144,"data_stored":1109017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:06:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.2210000000000001,"15min":1.262},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6220000000000001,"5min":1.7949999999999999,"15min":1.927},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.258,"5min":1.367,"15min":1.3540000000000001},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.5649999999999999,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.4430000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:06:21 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1930000000000001,"15min":1.196},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.786,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.2110000000000001,"15min":1.2150000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.579,"5min":1.6259999999999999,"15min":1.877},"last":1.319}]},{"osd":2,"last update":"Mon Oct 16 11:06:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1990000000000001,"15min":1.2170000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.627,"5min":1.667,"15min":1.895},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.3180000000000001,"15min":1.3089999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.4950000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:06:21 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.333,"15min":1.3049999999999999},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.88500000000000001},"max":{"1min":1.728,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.244},{"interface":"front","average":{"1min":1.347,"5min":1.4099999999999999,"15min":1.4199999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":2.1440000000000001},"last":1.377}]},{"osd":4,"last update":"Mon Oct 16 11:06:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.3049999999999999,"15min":1.2909999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.70299999999999996},"max":{"1min":1.802,"5min":1.853,"15min":1.9670000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5620000000000001,"15min":1.5569999999999999},"min":{"1min":1.2370000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.867,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.6479999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:06:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.4630000000000001,"15min":1.4239999999999999},"min":{"1min":0.999,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.4650000000000001,"15min":1.4430000000000001},"min":{"1min":0.98499999999999999,"5min":0.89100000000000001,"15min":0.86399999999999999},"max":{"1min":1.956,"5min":1.994,"15min":1.994},"last":1.423}]},{"osd":7,"last update":"Mon Oct 16 11:06:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.5820000000000001,"15min":1.5509999999999999},"min":{"1min":1.1339999999999999,"5min":0.91400000000000003,"15min":0.85399999999999998},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.2290000000000001},"last":1.591},{"interface":"front","average":{"1min":1.641,"5min":1.661,"15min":1.647},"min":{"1min":1.099,"5min":1.099,"15min":1.0149999999999999},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.6549999999999998},"last":2.0550000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59812,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312028,"statfs":{"total":96636764160,"available":96575516672,"internally_reserved":0,"allocated":2318336,"data_stored":1110095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.157,"15min":1.206},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.52600000000000002},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.173},{"interface":"front","average":{"1min":1.165,"5min":1.214,"15min":1.246},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9790000000000001},"last":1.4199999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.3260000000000001,"15min":1.3380000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.82899999999999996},"max":{"1min":1.782,"5min":1.8240000000000001,"15min":2.1680000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.216,"15min":1.254},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.581,"5min":1.7270000000000001,"15min":1.8080000000000001},"last":1.327}]},{"osd":2,"last update":"Mon Oct 16 11:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2889999999999999,"15min":1.284},"min":{"1min":0.872,"5min":0.81200000000000006,"15min":0.67200000000000004},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8560000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.347,"15min":1.4079999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.845,"5min":1.8859999999999999,"15min":2.234},"last":0.77600000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.3260000000000001,"15min":1.325},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.671,"5min":1.772,"15min":1.899},"last":1.46},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.444,"15min":1.4299999999999999},"min":{"1min":1.0049999999999999,"5min":0.97299999999999998,"15min":0.75800000000000001},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.49}]},{"osd":5,"last update":"Mon Oct 16 11:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.421,"15min":1.421},"min":{"1min":1.054,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.232},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4490000000000001,"15min":1.4059999999999999},"min":{"1min":1.238,"5min":0.93000000000000005,"15min":0.84299999999999997},"max":{"1min":1.7170000000000001,"5min":1.821,"15min":2.0819999999999999},"last":1.5760000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.486,"15min":1.46},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85699999999999998},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.3490000000000002},"last":1.286},{"interface":"front","average":{"1min":1.635,"5min":1.611,"15min":1.593},"min":{"1min":1.1899999999999999,"5min":1.109,"15min":0.75800000000000001},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.3450000000000002},"last":1.538}]},{"osd":7,"last update":"Mon Oct 16 11:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5680000000000001,"15min":1.5429999999999999},"min":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.04},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":2.117},"last":1.522},{"interface":"front","average":{"1min":1.6990000000000001,"5min":1.647,"15min":1.6359999999999999},"min":{"1min":1.129,"5min":1.119,"15min":0.874},"max":{"1min":2.7229999999999999,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":1.611}]}]},{"osd":3,"up_from":13,"seq":55834575772,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47320,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324520,"statfs":{"total":96636764160,"available":96588308480,"internally_reserved":0,"allocated":2306048,"data_stored":1108447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.90100000000000002,"15min":0.90200000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.968},"last":1.2430000000000001},{"interface":"front","average":{"1min":0.90000000000000002,"5min":1.0249999999999999,"15min":1.048},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0920000000000001,"5min":1.5800000000000001,"15min":2.1040000000000001},"last":0.99199999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.0129999999999999,"15min":0.998},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.351,"5min":1.5509999999999999,"15min":2.6269999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1200000000000001,"15min":1.1339999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.6299999999999999,"5min":1.651,"15min":2.2970000000000002},"last":1.331}]},{"osd":2,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.101,"15min":1.1140000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3879999999999999,"5min":1.641,"15min":2.3359999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.208,"15min":1.22},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5740000000000001,"5min":1.623,"15min":1.74},"last":0.95299999999999996}]},{"osd":4,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.367,"15min":1.343},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.724,"5min":1.74,"15min":2.1459999999999999},"last":1.446},{"interface":"front","average":{"1min":1.347,"5min":1.3540000000000001,"15min":1.347},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6319999999999999,"5min":1.7110000000000001,"15min":1.7889999999999999},"last":1.208}]},{"osd":5,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.379,"15min":1.385},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.97299999999999998},"max":{"1min":1.6699999999999999,"5min":1.897,"15min":2.4590000000000001},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.419,"15min":1.421},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8220000000000001,"5min":1.9910000000000001,"15min":2.089},"last":1.222}]},{"osd":6,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.5109999999999999,"15min":1.502},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.9510000000000001,"5min":2.3279999999999998,"15min":2.5760000000000001},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.486,"5min":1.522,"15min":1.522},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.0219999999999998,"5min":2.1509999999999998,"15min":2.4169999999999998},"last":1.3009999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.522,"15min":1.5269999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":2.29,"5min":2.29,"15min":2.746},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.635,"15min":1.643},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.8029999999999999},"last":1.353}]}]},{"osd":2,"up_from":13,"seq":55834575771,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51484,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320356,"statfs":{"total":96636764160,"available":96584044544,"internally_reserved":0,"allocated":2306048,"data_stored":1108503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.95599999999999996,"15min":0.93100000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.548,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.026,"15min":1.0680000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3540000000000001,"5min":1.5409999999999999,"15min":2.3010000000000002},"last":0.73099999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.1399999999999999,"15min":1.1080000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.657,"5min":1.7290000000000001,"15min":1.7749999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.121,"15min":1.1559999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.546,"5min":1.661,"15min":2.6120000000000001},"last":1.2869999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.109,"15min":1.085},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.3600000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.2849999999999999,"15min":1.266},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6910000000000001,"5min":1.899,"15min":1.99},"last":1.095}]},{"osd":4,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.351,"15min":1.3899999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.728,"5min":1.8340000000000001,"15min":2.5190000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.3580000000000001,"15min":1.341},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.4729999999999999,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":1.1639999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.476,"15min":1.4490000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.2559999999999998,"5min":2.2570000000000001,"15min":2.282},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4610000000000001,"15min":1.4710000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.2120000000000002,"5min":2.3450000000000002,"15min":2.9649999999999999},"last":1.391}]},{"osd":6,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.4239999999999999,"15min":1.4039999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.929,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.569,"15min":1.5269999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.0819999999999999,"5min":2.3420000000000001,"15min":2.5139999999999998},"last":1.4670000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.538,"15min":1.538},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.1389999999999998,"5min":2.141,"15min":3.1669999999999998},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.5169999999999999,"15min":1.514},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.2469999999999999,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":1.341}]}]},{"osd":1,"up_from":13,"seq":55834575771,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59800,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312040,"statfs":{"total":96636764160,"available":96575528960,"internally_reserved":0,"allocated":2306048,"data_stored":1108447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.88600000000000001,"15min":0.90500000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.0269999999999999,"15min":1.0469999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":4.5110000000000001,"5min":4.5110000000000001,"15min":4.5110000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.93100000000000005,"15min":0.99099999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.4930000000000001,"5min":1.5349999999999999,"15min":1.8069999999999999},"last":1.411},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.2170000000000001,"15min":1.256},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.4410000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:06:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.196,"15min":1.2070000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":4.351,"5min":4.351,"15min":4.351},"last":1.248},{"interface":"front","average":{"1min":0.89300000000000002,"5min":1.0960000000000001,"15min":1.131},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6419999999999999},"last":1.304}]},{"osd":4,"last update":"Mon Oct 16 11:06:17 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.2529999999999999,"15min":1.29},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.873,"5min":1.873,"15min":2.1030000000000002},"last":1.194},{"interface":"front","average":{"1min":1.294,"5min":1.3979999999999999,"15min":1.407},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":1.3480000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:06:17 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3859999999999999,"15min":1.3999999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":4.4400000000000004,"5min":4.4400000000000004,"15min":4.4400000000000004},"last":1.605},{"interface":"front","average":{"1min":1.28,"5min":1.383,"15min":1.387},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.089,"5min":2.8690000000000002,"15min":2.8690000000000002},"last":1.579}]},{"osd":6,"last update":"Mon Oct 16 11:06:17 2023","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.4890000000000001,"15min":1.472},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":7.2370000000000001,"5min":7.2370000000000001,"15min":7.2370000000000001},"last":1.738},{"interface":"front","average":{"1min":1.6859999999999999,"5min":1.4650000000000001,"15min":1.4550000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":10.683999999999999,"5min":10.683999999999999,"15min":10.683999999999999},"last":1.504}]},{"osd":7,"last update":"Mon Oct 16 11:06:17 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.4930000000000001,"15min":1.5249999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.5249999999999999,"15min":1.5649999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":4.5549999999999997,"5min":4.5549999999999997,"15min":4.5549999999999997},"last":1.8220000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575772,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52056,"kb_used_data":2828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319784,"statfs":{"total":96636764160,"available":96583458816,"internally_reserved":0,"allocated":2895872,"data_stored":1698781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.93200000000000005,"15min":0.90700000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.383,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.0329999999999999,"15min":1.0549999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4219999999999999,"5min":1.663,"15min":1.714},"last":0.47999999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":1.0369999999999999,"15min":1.0760000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.421,"5min":1.625,"15min":1.625},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.123,"15min":1.1419999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6060000000000001,"5min":1.659,"15min":1.823},"last":0.54900000000000004}]},{"osd":3,"last update":"Mon Oct 16 11:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":1.0760000000000001,"15min":1.0680000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.575,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.054,"5min":1.232,"15min":1.2509999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.536,"5min":1.6559999999999999,"15min":1.8480000000000001},"last":0.70699999999999996}]},{"osd":4,"last update":"Mon Oct 16 11:06:10 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3480000000000001,"15min":1.365},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.6659999999999999,"5min":1.7010000000000001,"15min":1.7410000000000001},"last":1.127},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.294,"15min":1.319},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.883,"5min":1.883,"15min":1.97},"last":1.1439999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:06:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.365,"15min":1.367},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":2.0259999999999998},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.4550000000000001,"15min":1.4570000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.0920000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.0960000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:06:10 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.492,"15min":1.5},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.7410000000000001,"5min":2.286,"15min":2.286},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.4650000000000001,"15min":1.4570000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.8320000000000001,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.0649999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:06:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.526,"15min":1.5229999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.4850000000000001,"15min":1.4730000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.1819999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:06:28.049 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:06:28.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.299+0000 7fe800421640 1 Processor -- start 2023-10-16T11:06:28.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.299+0000 7fe800421640 1 -- start start 2023-10-16T11:06:28.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.301+0000 7fe800421640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ae80 0x7fe7f815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:28.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.301+0000 7fe800421640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe7f80ce390 0x7fe7f8070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:28.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.301+0000 7fe800421640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f815ed60 con 0x7fe7f815aaa0 2023-10-16T11:06:28.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.301+0000 7fe800421640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f806b810 con 0x7fe7f80ce390 2023-10-16T11:06:28.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.301+0000 7fe800421640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f80707e0 con 0x7fe7f815ae80 2023-10-16T11:06:28.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.301+0000 7fe7fe9be640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe7f80ce390 0x7fe7f8070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:28.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.301+0000 7fe7fe9be640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe7f80ce390 0x7fe7f8070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33306/0 (socket says 172.21.15.84:33306) 2023-10-16T11:06:28.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.301+0000 7fe7fe9be640 1 -- 172.21.15.84:0/1505175680 learned_addr learned my addr 172.21.15.84:0/1505175680 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:28.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.301+0000 7fe7fd9bc640 1 --2- 172.21.15.84:0/1505175680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ae80 0x7fe7f815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:28.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.302+0000 7fe7fe9be640 1 -- 172.21.15.84:0/1505175680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ae80 msgr2=0x7fe7f815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:28.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.302+0000 7fe7fe9be640 1 --2- 172.21.15.84:0/1505175680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ae80 0x7fe7f815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.302+0000 7fe7fe9be640 1 -- 172.21.15.84:0/1505175680 >> v1:172.21.15.157:6789/0 conn(0x7fe7f815aaa0 legacy=0x7fe7f80cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:28.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.302+0000 7fe7fe9be640 1 -- 172.21.15.84:0/1505175680 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f815ef90 con 0x7fe7f80ce390 2023-10-16T11:06:28.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.303+0000 7fe7fe9be640 1 --2- 172.21.15.84:0/1505175680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80ce390 0x7fe7f8070f50 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009e50 tx=0x7fe7f002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a98abbef040d15 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:28.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.303+0000 7fe7fd1bb640 1 -- 172.21.15.84:0/1505175680 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f002ca70 con 0x7fe7f80ce390 2023-10-16T11:06:28.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.303+0000 7fe7fd1bb640 1 -- 172.21.15.84:0/1505175680 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f002cbd0 con 0x7fe7f80ce390 2023-10-16T11:06:28.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.303+0000 7fe7fd1bb640 1 -- 172.21.15.84:0/1505175680 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f00343f0 con 0x7fe7f80ce390 2023-10-16T11:06:28.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.304+0000 7fe800421640 1 -- 172.21.15.84:0/1505175680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80ce390 msgr2=0x7fe7f8070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:28.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.304+0000 7fe800421640 1 --2- 172.21.15.84:0/1505175680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80ce390 0x7fe7f8070f50 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009e50 tx=0x7fe7f002bed0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.304+0000 7fe800421640 1 -- 172.21.15.84:0/1505175680 shutdown_connections 2023-10-16T11:06:28.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.304+0000 7fe800421640 1 --2- 172.21.15.84:0/1505175680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80ce390 0x7fe7f8070f50 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.304+0000 7fe800421640 1 --2- 172.21.15.84:0/1505175680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ae80 0x7fe7f815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.304+0000 7fe800421640 1 -- 172.21.15.84:0/1505175680 >> 172.21.15.84:0/1505175680 conn(0x7fe7f80cb410 msgr2=0x7fe7f80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:28.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.304+0000 7fe800421640 1 -- 172.21.15.84:0/1505175680 shutdown_connections 2023-10-16T11:06:28.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.304+0000 7fe800421640 1 -- 172.21.15.84:0/1505175680 wait complete. 2023-10-16T11:06:28.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.306+0000 7fe800421640 1 Processor -- start 2023-10-16T11:06:28.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.306+0000 7fe800421640 1 -- start start 2023-10-16T11:06:28.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.307+0000 7fe800421640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80ce390 0x7fe7f814eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:28.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.307+0000 7fe800421640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f815aaa0 0x7fe7f80c4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:28.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.307+0000 7fe800421640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ae80 0x7fe7f80ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:28.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.307+0000 7fe800421640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f80c52c0 con 0x7fe7f80ce390 2023-10-16T11:06:28.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.307+0000 7fe800421640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f80c4fe0 con 0x7fe7f815aaa0 2023-10-16T11:06:28.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.307+0000 7fe800421640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f80c9080 con 0x7fe7f815ae80 2023-10-16T11:06:28.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.307+0000 7fe7fe1bd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80ce390 0x7fe7f814eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:28.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.307+0000 7fe7fe1bd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80ce390 0x7fe7f814eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33308/0 (socket says 172.21.15.84:33308) 2023-10-16T11:06:28.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.307+0000 7fe7fe1bd640 1 -- 172.21.15.84:0/1768794337 learned_addr learned my addr 172.21.15.84:0/1768794337 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:28.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.307+0000 7fe7fe1bd640 1 -- 172.21.15.84:0/1768794337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ae80 msgr2=0x7fe7f80ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:28.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.307+0000 7fe7fd9bc640 1 --2- 172.21.15.84:0/1768794337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f815aaa0 0x7fe7f80c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:28.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.307+0000 7fe7fe9be640 1 --2- 172.21.15.84:0/1768794337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ae80 0x7fe7f80ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:28.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.308+0000 7fe7fe1bd640 1 --2- 172.21.15.84:0/1768794337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ae80 0x7fe7f80ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.308+0000 7fe7fe1bd640 1 -- 172.21.15.84:0/1768794337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f815aaa0 msgr2=0x7fe7f80c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:28.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.308+0000 7fe7fe1bd640 1 --2- 172.21.15.84:0/1768794337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f815aaa0 0x7fe7f80c4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.308+0000 7fe7fe1bd640 1 -- 172.21.15.84:0/1768794337 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f0009b30 con 0x7fe7f80ce390 2023-10-16T11:06:28.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.308+0000 7fe7fe1bd640 1 --2- 172.21.15.84:0/1768794337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80ce390 0x7fe7f814eec0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400c430 tx=0x7fe7f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:28.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.308+0000 7fe7eeffd640 1 -- 172.21.15.84:0/1768794337 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4011790 con 0x7fe7f80ce390 2023-10-16T11:06:28.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.308+0000 7fe7eeffd640 1 -- 172.21.15.84:0/1768794337 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f40118f0 con 0x7fe7f80ce390 2023-10-16T11:06:28.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.308+0000 7fe7eeffd640 1 -- 172.21.15.84:0/1768794337 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4011ce0 con 0x7fe7f80ce390 2023-10-16T11:06:28.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.308+0000 7fe800421640 1 -- 172.21.15.84:0/1768794337 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f80c9360 con 0x7fe7f80ce390 2023-10-16T11:06:28.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.308+0000 7fe800421640 1 -- 172.21.15.84:0/1768794337 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f814f400 con 0x7fe7f80ce390 2023-10-16T11:06:28.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.309+0000 7fe800421640 1 -- 172.21.15.84:0/1768794337 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f80c9880 con 0x7fe7f80ce390 2023-10-16T11:06:28.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.310+0000 7fe7eeffd640 1 -- 172.21.15.84:0/1768794337 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe7f401c460 con 0x7fe7f80ce390 2023-10-16T11:06:28.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.311+0000 7fe7eeffd640 1 --2- 172.21.15.84:0/1768794337 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fe7dc06af90 0x7fe7dc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:28.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.311+0000 7fe7eeffd640 1 -- 172.21.15.84:0/1768794337 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe7f40896c0 con 0x7fe7f80ce390 2023-10-16T11:06:28.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.311+0000 7fe7fd9bc640 1 --2- 172.21.15.84:0/1768794337 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fe7dc06af90 0x7fe7dc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:28.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.312+0000 7fe7fd9bc640 1 --2- 172.21.15.84:0/1768794337 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fe7dc06af90 0x7fe7dc06d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe7f815d850 tx=0x7fe7e8008040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:28.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.315+0000 7fe7eeffd640 1 -- 172.21.15.84:0/1768794337 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe7f80c9880 con 0x7fe7f80ce390 2023-10-16T11:06:28.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.504+0000 7fe800421640 1 -- 172.21.15.84:0/1768794337 --> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe7f815f550 con 0x7fe7dc06af90 2023-10-16T11:06:28.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.516+0000 7fe7eeffd640 1 -- 172.21.15.84:0/1768794337 <== mgr.9969 v2:172.21.15.84:6832/1441051579 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287359 (secure 0 0 0) 0x7fe7e8009d60 con 0x7fe7dc06af90 2023-10-16T11:06:28.521 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:28.523 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:06:28.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.522+0000 7fe800421640 1 -- 172.21.15.84:0/1768794337 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fe7dc06af90 msgr2=0x7fe7dc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:28.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.522+0000 7fe800421640 1 --2- 172.21.15.84:0/1768794337 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fe7dc06af90 0x7fe7dc06d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe7f815d850 tx=0x7fe7e8008040 comp rx=0 tx=0).stop 2023-10-16T11:06:28.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.522+0000 7fe800421640 1 -- 172.21.15.84:0/1768794337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80ce390 msgr2=0x7fe7f814eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:28.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.522+0000 7fe800421640 1 --2- 172.21.15.84:0/1768794337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80ce390 0x7fe7f814eec0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400c430 tx=0x7fe7f400c8f0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.522+0000 7fe800421640 1 -- 172.21.15.84:0/1768794337 shutdown_connections 2023-10-16T11:06:28.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.522+0000 7fe800421640 1 --2- 172.21.15.84:0/1768794337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7f815ae80 0x7fe7f80ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.522+0000 7fe800421640 1 --2- 172.21.15.84:0/1768794337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f815aaa0 0x7fe7f80c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.522+0000 7fe800421640 1 --2- 172.21.15.84:0/1768794337 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fe7dc06af90 0x7fe7dc06d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.522+0000 7fe800421640 1 --2- 172.21.15.84:0/1768794337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7f80ce390 0x7fe7f814eec0 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.522+0000 7fe800421640 1 -- 172.21.15.84:0/1768794337 >> 172.21.15.84:0/1768794337 conn(0x7fe7f80cb410 msgr2=0x7fe7f806ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:28.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.522+0000 7fe800421640 1 -- 172.21.15.84:0/1768794337 shutdown_connections 2023-10-16T11:06:28.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.522+0000 7fe800421640 1 -- 172.21.15.84:0/1768794337 wait complete. 2023-10-16T11:06:28.544 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-16T11:06:27.678123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1696,"num_read_kb":3438,"num_write":525,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2750,"ondisk_log_size":2750,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":389420,"kb_used_data":19208,"kb_used_omap":12,"kb_used_meta":370099,"kb_avail":754585300,"statfs":{"total":773094113280,"available":772695347200,"internally_reserved":0,"allocated":19668992,"data_stored":10052860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":378981945},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001409"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.690053+0000","last_change":"2023-10-16T11:06:23.612356+0000","last_active":"2023-10-16T11:06:25.690053+0000","last_peered":"2023-10-16T11:06:25.690053+0000","last_clean":"2023-10-16T11:06:25.690053+0000","last_became_active":"2023-10-16T11:06:23.611936+0000","last_became_peered":"2023-10-16T11:06:23.611936+0000","last_unstale":"2023-10-16T11:06:25.690053+0000","last_undegraded":"2023-10-16T11:06:25.690053+0000","last_fullsized":"2023-10-16T11:06:25.690053+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:44:58.903083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.980986+0000","last_change":"2023-10-16T11:06:23.612251+0000","last_active":"2023-10-16T11:06:24.980986+0000","last_peered":"2023-10-16T11:06:24.980986+0000","last_clean":"2023-10-16T11:06:24.980986+0000","last_became_active":"2023-10-16T11:06:23.611968+0000","last_became_peered":"2023-10-16T11:06:23.611968+0000","last_unstale":"2023-10-16T11:06:24.980986+0000","last_undegraded":"2023-10-16T11:06:24.980986+0000","last_fullsized":"2023-10-16T11:06:24.980986+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:29:32.028128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.691944+0000","last_change":"2023-10-16T11:06:23.612260+0000","last_active":"2023-10-16T11:06:25.691944+0000","last_peered":"2023-10-16T11:06:25.691944+0000","last_clean":"2023-10-16T11:06:25.691944+0000","last_became_active":"2023-10-16T11:06:23.611970+0000","last_became_peered":"2023-10-16T11:06:23.611970+0000","last_unstale":"2023-10-16T11:06:25.691944+0000","last_undegraded":"2023-10-16T11:06:25.691944+0000","last_fullsized":"2023-10-16T11:06:25.691944+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:35.201235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.682638+0000","last_change":"2023-10-16T11:06:23.613121+0000","last_active":"2023-10-16T11:06:24.682638+0000","last_peered":"2023-10-16T11:06:24.682638+0000","last_clean":"2023-10-16T11:06:24.682638+0000","last_became_active":"2023-10-16T11:06:23.612829+0000","last_became_peered":"2023-10-16T11:06:23.612829+0000","last_unstale":"2023-10-16T11:06:24.682638+0000","last_undegraded":"2023-10-16T11:06:24.682638+0000","last_fullsized":"2023-10-16T11:06:24.682638+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:56:48.322204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.980352+0000","last_change":"2023-10-16T11:06:23.613177+0000","last_active":"2023-10-16T11:06:24.980352+0000","last_peered":"2023-10-16T11:06:24.980352+0000","last_clean":"2023-10-16T11:06:24.980352+0000","last_became_active":"2023-10-16T11:06:23.612910+0000","last_became_peered":"2023-10-16T11:06:23.612910+0000","last_unstale":"2023-10-16T11:06:24.980352+0000","last_undegraded":"2023-10-16T11:06:24.980352+0000","last_fullsized":"2023-10-16T11:06:24.980352+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:20:15.243219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.604960+0000","last_change":"2023-10-16T11:06:23.617312+0000","last_active":"2023-10-16T11:06:24.604960+0000","last_peered":"2023-10-16T11:06:24.604960+0000","last_clean":"2023-10-16T11:06:24.604960+0000","last_became_active":"2023-10-16T11:06:23.613671+0000","last_became_peered":"2023-10-16T11:06:23.613671+0000","last_unstale":"2023-10-16T11:06:24.604960+0000","last_undegraded":"2023-10-16T11:06:24.604960+0000","last_fullsized":"2023-10-16T11:06:24.604960+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:44:42.439099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.188401+0000","last_change":"2023-10-16T11:06:23.614759+0000","last_active":"2023-10-16T11:06:25.188401+0000","last_peered":"2023-10-16T11:06:25.188401+0000","last_clean":"2023-10-16T11:06:25.188401+0000","last_became_active":"2023-10-16T11:06:23.614461+0000","last_became_peered":"2023-10-16T11:06:23.614461+0000","last_unstale":"2023-10-16T11:06:25.188401+0000","last_undegraded":"2023-10-16T11:06:25.188401+0000","last_fullsized":"2023-10-16T11:06:25.188401+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:45.286323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.980191+0000","last_change":"2023-10-16T11:06:23.614582+0000","last_active":"2023-10-16T11:06:24.980191+0000","last_peered":"2023-10-16T11:06:24.980191+0000","last_clean":"2023-10-16T11:06:24.980191+0000","last_became_active":"2023-10-16T11:06:23.614021+0000","last_became_peered":"2023-10-16T11:06:23.614021+0000","last_unstale":"2023-10-16T11:06:24.980191+0000","last_undegraded":"2023-10-16T11:06:24.980191+0000","last_fullsized":"2023-10-16T11:06:24.980191+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:07:59.718438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.981340+0000","last_change":"2023-10-16T11:06:23.623414+0000","last_active":"2023-10-16T11:06:24.981340+0000","last_peered":"2023-10-16T11:06:24.981340+0000","last_clean":"2023-10-16T11:06:24.981340+0000","last_became_active":"2023-10-16T11:06:23.623103+0000","last_became_peered":"2023-10-16T11:06:23.623103+0000","last_unstale":"2023-10-16T11:06:24.981340+0000","last_undegraded":"2023-10-16T11:06:24.981340+0000","last_fullsized":"2023-10-16T11:06:24.981340+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:38.166069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.980993+0000","last_change":"2023-10-16T11:06:23.615412+0000","last_active":"2023-10-16T11:06:24.980993+0000","last_peered":"2023-10-16T11:06:24.980993+0000","last_clean":"2023-10-16T11:06:24.980993+0000","last_became_active":"2023-10-16T11:06:23.614420+0000","last_became_peered":"2023-10-16T11:06:23.614420+0000","last_unstale":"2023-10-16T11:06:24.980993+0000","last_undegraded":"2023-10-16T11:06:24.980993+0000","last_fullsized":"2023-10-16T11:06:24.980993+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:04:22.692050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.980958+0000","last_change":"2023-10-16T11:06:23.617173+0000","last_active":"2023-10-16T11:06:24.980958+0000","last_peered":"2023-10-16T11:06:24.980958+0000","last_clean":"2023-10-16T11:06:24.980958+0000","last_became_active":"2023-10-16T11:06:23.614887+0000","last_became_peered":"2023-10-16T11:06:23.614887+0000","last_unstale":"2023-10-16T11:06:24.980958+0000","last_undegraded":"2023-10-16T11:06:24.980958+0000","last_fullsized":"2023-10-16T11:06:24.980958+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:25.865564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.980421+0000","last_change":"2023-10-16T11:06:23.617560+0000","last_active":"2023-10-16T11:06:24.980421+0000","last_peered":"2023-10-16T11:06:24.980421+0000","last_clean":"2023-10-16T11:06:24.980421+0000","last_became_active":"2023-10-16T11:06:23.617265+0000","last_became_peered":"2023-10-16T11:06:23.617265+0000","last_unstale":"2023-10-16T11:06:24.980421+0000","last_undegraded":"2023-10-16T11:06:24.980421+0000","last_fullsized":"2023-10-16T11:06:24.980421+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:22.405343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.691675+0000","last_change":"2023-10-16T11:06:23.612310+0000","last_active":"2023-10-16T11:06:25.691675+0000","last_peered":"2023-10-16T11:06:25.691675+0000","last_clean":"2023-10-16T11:06:25.691675+0000","last_became_active":"2023-10-16T11:06:23.611942+0000","last_became_peered":"2023-10-16T11:06:23.611942+0000","last_unstale":"2023-10-16T11:06:25.691675+0000","last_undegraded":"2023-10-16T11:06:25.691675+0000","last_fullsized":"2023-10-16T11:06:25.691675+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:42.960951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.690961+0000","last_change":"2023-10-16T11:06:23.614568+0000","last_active":"2023-10-16T11:06:25.690961+0000","last_peered":"2023-10-16T11:06:25.690961+0000","last_clean":"2023-10-16T11:06:25.690961+0000","last_became_active":"2023-10-16T11:06:23.614245+0000","last_became_peered":"2023-10-16T11:06:23.614245+0000","last_unstale":"2023-10-16T11:06:25.690961+0000","last_undegraded":"2023-10-16T11:06:25.690961+0000","last_fullsized":"2023-10-16T11:06:25.690961+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:57:33.492506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.689077+0000","last_change":"2023-10-16T11:06:23.602913+0000","last_active":"2023-10-16T11:06:25.689077+0000","last_peered":"2023-10-16T11:06:25.689077+0000","last_clean":"2023-10-16T11:06:25.689077+0000","last_became_active":"2023-10-16T11:06:23.602483+0000","last_became_peered":"2023-10-16T11:06:23.602483+0000","last_unstale":"2023-10-16T11:06:25.689077+0000","last_undegraded":"2023-10-16T11:06:25.689077+0000","last_fullsized":"2023-10-16T11:06:25.689077+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:03.691297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.188075+0000","last_change":"2023-10-16T11:06:23.613782+0000","last_active":"2023-10-16T11:06:25.188075+0000","last_peered":"2023-10-16T11:06:25.188075+0000","last_clean":"2023-10-16T11:06:25.188075+0000","last_became_active":"2023-10-16T11:06:23.610093+0000","last_became_peered":"2023-10-16T11:06:23.610093+0000","last_unstale":"2023-10-16T11:06:25.188075+0000","last_undegraded":"2023-10-16T11:06:25.188075+0000","last_fullsized":"2023-10-16T11:06:25.188075+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:38.166069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.605025+0000","last_change":"2023-10-16T11:06:23.603606+0000","last_active":"2023-10-16T11:06:24.605025+0000","last_peered":"2023-10-16T11:06:24.605025+0000","last_clean":"2023-10-16T11:06:24.605025+0000","last_became_active":"2023-10-16T11:06:23.602978+0000","last_became_peered":"2023-10-16T11:06:23.602978+0000","last_unstale":"2023-10-16T11:06:24.605025+0000","last_undegraded":"2023-10-16T11:06:24.605025+0000","last_fullsized":"2023-10-16T11:06:24.605025+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:58.721072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.608311+0000","last_change":"2023-10-16T11:06:23.599561+0000","last_active":"2023-10-16T11:06:24.608311+0000","last_peered":"2023-10-16T11:06:24.608311+0000","last_clean":"2023-10-16T11:06:24.608311+0000","last_became_active":"2023-10-16T11:06:23.598538+0000","last_became_peered":"2023-10-16T11:06:23.598538+0000","last_unstale":"2023-10-16T11:06:24.608311+0000","last_undegraded":"2023-10-16T11:06:24.608311+0000","last_fullsized":"2023-10-16T11:06:24.608311+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:29.386268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.605897+0000","last_change":"2023-10-16T11:06:23.594334+0000","last_active":"2023-10-16T11:06:24.605897+0000","last_peered":"2023-10-16T11:06:24.605897+0000","last_clean":"2023-10-16T11:06:24.605897+0000","last_became_active":"2023-10-16T11:06:23.593916+0000","last_became_peered":"2023-10-16T11:06:23.593916+0000","last_unstale":"2023-10-16T11:06:24.605897+0000","last_undegraded":"2023-10-16T11:06:24.605897+0000","last_fullsized":"2023-10-16T11:06:24.605897+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:29.249502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.691725+0000","last_change":"2023-10-16T11:06:23.612406+0000","last_active":"2023-10-16T11:06:25.691725+0000","last_peered":"2023-10-16T11:06:25.691725+0000","last_clean":"2023-10-16T11:06:25.691725+0000","last_became_active":"2023-10-16T11:06:23.612146+0000","last_became_peered":"2023-10-16T11:06:23.612146+0000","last_unstale":"2023-10-16T11:06:25.691725+0000","last_undegraded":"2023-10-16T11:06:25.691725+0000","last_fullsized":"2023-10-16T11:06:25.691725+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:34:55.741968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.605998+0000","last_change":"2023-10-16T11:06:23.611256+0000","last_active":"2023-10-16T11:06:24.605998+0000","last_peered":"2023-10-16T11:06:24.605998+0000","last_clean":"2023-10-16T11:06:24.605998+0000","last_became_active":"2023-10-16T11:06:23.610895+0000","last_became_peered":"2023-10-16T11:06:23.610895+0000","last_unstale":"2023-10-16T11:06:24.605998+0000","last_undegraded":"2023-10-16T11:06:24.605998+0000","last_fullsized":"2023-10-16T11:06:24.605998+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:24:10.847068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.980524+0000","last_change":"2023-10-16T11:06:23.621851+0000","last_active":"2023-10-16T11:06:24.980524+0000","last_peered":"2023-10-16T11:06:24.980524+0000","last_clean":"2023-10-16T11:06:24.980524+0000","last_became_active":"2023-10-16T11:06:23.621568+0000","last_became_peered":"2023-10-16T11:06:23.621568+0000","last_unstale":"2023-10-16T11:06:24.980524+0000","last_undegraded":"2023-10-16T11:06:24.980524+0000","last_fullsized":"2023-10-16T11:06:24.980524+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:42:35.497112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.605427+0000","last_change":"2023-10-16T11:06:23.610819+0000","last_active":"2023-10-16T11:06:24.605427+0000","last_peered":"2023-10-16T11:06:24.605427+0000","last_clean":"2023-10-16T11:06:24.605427+0000","last_became_active":"2023-10-16T11:06:23.607043+0000","last_became_peered":"2023-10-16T11:06:23.607043+0000","last_unstale":"2023-10-16T11:06:24.605427+0000","last_undegraded":"2023-10-16T11:06:24.605427+0000","last_fullsized":"2023-10-16T11:06:24.605427+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:05:42.824835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.980631+0000","last_change":"2023-10-16T11:06:23.610604+0000","last_active":"2023-10-16T11:06:24.980631+0000","last_peered":"2023-10-16T11:06:24.980631+0000","last_clean":"2023-10-16T11:06:24.980631+0000","last_became_active":"2023-10-16T11:06:23.609531+0000","last_became_peered":"2023-10-16T11:06:23.609531+0000","last_unstale":"2023-10-16T11:06:24.980631+0000","last_undegraded":"2023-10-16T11:06:24.980631+0000","last_fullsized":"2023-10-16T11:06:24.980631+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:43.851748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.692726+0000","last_change":"2023-10-16T11:06:23.610559+0000","last_active":"2023-10-16T11:06:25.692726+0000","last_peered":"2023-10-16T11:06:25.692726+0000","last_clean":"2023-10-16T11:06:25.692726+0000","last_became_active":"2023-10-16T11:06:23.610262+0000","last_became_peered":"2023-10-16T11:06:23.610262+0000","last_unstale":"2023-10-16T11:06:25.692726+0000","last_undegraded":"2023-10-16T11:06:25.692726+0000","last_fullsized":"2023-10-16T11:06:25.692726+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:28:12.369626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.189094+0000","last_change":"2023-10-16T11:06:23.617205+0000","last_active":"2023-10-16T11:06:25.189094+0000","last_peered":"2023-10-16T11:06:25.189094+0000","last_clean":"2023-10-16T11:06:25.189094+0000","last_became_active":"2023-10-16T11:06:23.616911+0000","last_became_peered":"2023-10-16T11:06:23.616911+0000","last_unstale":"2023-10-16T11:06:25.189094+0000","last_undegraded":"2023-10-16T11:06:25.189094+0000","last_fullsized":"2023-10-16T11:06:25.189094+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:05:42.824835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.692194+0000","last_change":"2023-10-16T11:06:23.601068+0000","last_active":"2023-10-16T11:06:25.692194+0000","last_peered":"2023-10-16T11:06:25.692194+0000","last_clean":"2023-10-16T11:06:25.692194+0000","last_became_active":"2023-10-16T11:06:23.600673+0000","last_became_peered":"2023-10-16T11:06:23.600673+0000","last_unstale":"2023-10-16T11:06:25.692194+0000","last_undegraded":"2023-10-16T11:06:25.692194+0000","last_fullsized":"2023-10-16T11:06:25.692194+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:11:47.595867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.682587+0000","last_change":"2023-10-16T11:06:23.612760+0000","last_active":"2023-10-16T11:06:24.682587+0000","last_peered":"2023-10-16T11:06:24.682587+0000","last_clean":"2023-10-16T11:06:24.682587+0000","last_became_active":"2023-10-16T11:06:23.612421+0000","last_became_peered":"2023-10-16T11:06:23.612421+0000","last_unstale":"2023-10-16T11:06:24.682587+0000","last_undegraded":"2023-10-16T11:06:24.682587+0000","last_fullsized":"2023-10-16T11:06:24.682587+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:00.640104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.691784+0000","last_change":"2023-10-16T11:06:23.607530+0000","last_active":"2023-10-16T11:06:25.691784+0000","last_peered":"2023-10-16T11:06:25.691784+0000","last_clean":"2023-10-16T11:06:25.691784+0000","last_became_active":"2023-10-16T11:06:23.605401+0000","last_became_peered":"2023-10-16T11:06:23.605401+0000","last_unstale":"2023-10-16T11:06:25.691784+0000","last_undegraded":"2023-10-16T11:06:25.691784+0000","last_fullsized":"2023-10-16T11:06:25.691784+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:56.865944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.607245+0000","last_change":"2023-10-16T11:06:23.604972+0000","last_active":"2023-10-16T11:06:24.607245+0000","last_peered":"2023-10-16T11:06:24.607245+0000","last_clean":"2023-10-16T11:06:24.607245+0000","last_became_active":"2023-10-16T11:06:23.604600+0000","last_became_peered":"2023-10-16T11:06:23.604600+0000","last_unstale":"2023-10-16T11:06:24.607245+0000","last_undegraded":"2023-10-16T11:06:24.607245+0000","last_fullsized":"2023-10-16T11:06:24.607245+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:00.640104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.607200+0000","last_change":"2023-10-16T11:06:23.609657+0000","last_active":"2023-10-16T11:06:24.607200+0000","last_peered":"2023-10-16T11:06:24.607200+0000","last_clean":"2023-10-16T11:06:24.607200+0000","last_became_active":"2023-10-16T11:06:23.607692+0000","last_became_peered":"2023-10-16T11:06:23.607692+0000","last_unstale":"2023-10-16T11:06:24.607200+0000","last_undegraded":"2023-10-16T11:06:24.607200+0000","last_fullsized":"2023-10-16T11:06:24.607200+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:11:30.976449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.607408+0000","last_change":"2023-10-16T11:06:23.594020+0000","last_active":"2023-10-16T11:06:24.607408+0000","last_peered":"2023-10-16T11:06:24.607408+0000","last_clean":"2023-10-16T11:06:24.607408+0000","last_became_active":"2023-10-16T11:06:23.593700+0000","last_became_peered":"2023-10-16T11:06:23.593700+0000","last_unstale":"2023-10-16T11:06:24.607408+0000","last_undegraded":"2023-10-16T11:06:24.607408+0000","last_fullsized":"2023-10-16T11:06:24.607408+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:54:26.214909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.189313+0000","last_change":"2023-10-16T11:06:23.610321+0000","last_active":"2023-10-16T11:06:25.189313+0000","last_peered":"2023-10-16T11:06:25.189313+0000","last_clean":"2023-10-16T11:06:25.189313+0000","last_became_active":"2023-10-16T11:06:23.606913+0000","last_became_peered":"2023-10-16T11:06:23.606913+0000","last_unstale":"2023-10-16T11:06:25.189313+0000","last_undegraded":"2023-10-16T11:06:25.189313+0000","last_fullsized":"2023-10-16T11:06:25.189313+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:53.501651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"183'4","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.625395+0000","last_change":"2023-10-16T11:06:22.605732+0000","last_active":"2023-10-16T11:06:23.625395+0000","last_peered":"2023-10-16T11:06:23.625395+0000","last_clean":"2023-10-16T11:06:23.625395+0000","last_became_active":"2023-10-16T11:06:22.604612+0000","last_became_peered":"2023-10-16T11:06:22.604612+0000","last_unstale":"2023-10-16T11:06:23.625395+0000","last_undegraded":"2023-10-16T11:06:23.625395+0000","last_fullsized":"2023-10-16T11:06:23.625395+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:07.029335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.608917+0000","last_change":"2023-10-16T11:06:23.597349+0000","last_active":"2023-10-16T11:06:24.608917+0000","last_peered":"2023-10-16T11:06:24.608917+0000","last_clean":"2023-10-16T11:06:24.608917+0000","last_became_active":"2023-10-16T11:06:23.596394+0000","last_became_peered":"2023-10-16T11:06:23.596394+0000","last_unstale":"2023-10-16T11:06:24.608917+0000","last_undegraded":"2023-10-16T11:06:24.608917+0000","last_fullsized":"2023-10-16T11:06:24.608917+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:07:18.531732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"183'1","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.625495+0000","last_change":"2023-10-16T11:06:22.593488+0000","last_active":"2023-10-16T11:06:23.625495+0000","last_peered":"2023-10-16T11:06:23.625495+0000","last_clean":"2023-10-16T11:06:23.625495+0000","last_became_active":"2023-10-16T11:06:22.592029+0000","last_became_peered":"2023-10-16T11:06:22.592029+0000","last_unstale":"2023-10-16T11:06:23.625495+0000","last_undegraded":"2023-10-16T11:06:23.625495+0000","last_fullsized":"2023-10-16T11:06:23.625495+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:22.856408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.604999+0000","last_change":"2023-10-16T11:06:23.605466+0000","last_active":"2023-10-16T11:06:24.604999+0000","last_peered":"2023-10-16T11:06:24.604999+0000","last_clean":"2023-10-16T11:06:24.604999+0000","last_became_active":"2023-10-16T11:06:23.605192+0000","last_became_peered":"2023-10-16T11:06:23.605192+0000","last_unstale":"2023-10-16T11:06:24.604999+0000","last_undegraded":"2023-10-16T11:06:24.604999+0000","last_fullsized":"2023-10-16T11:06:24.604999+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:22:52.230491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.625035+0000","last_change":"2023-10-16T11:06:22.602261+0000","last_active":"2023-10-16T11:06:23.625035+0000","last_peered":"2023-10-16T11:06:23.625035+0000","last_clean":"2023-10-16T11:06:23.625035+0000","last_became_active":"2023-10-16T11:06:22.601894+0000","last_became_peered":"2023-10-16T11:06:22.601894+0000","last_unstale":"2023-10-16T11:06:23.625035+0000","last_undegraded":"2023-10-16T11:06:23.625035+0000","last_fullsized":"2023-10-16T11:06:23.625035+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:34:54.722413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.692312+0000","last_change":"2023-10-16T11:06:23.605548+0000","last_active":"2023-10-16T11:06:25.692312+0000","last_peered":"2023-10-16T11:06:25.692312+0000","last_clean":"2023-10-16T11:06:25.692312+0000","last_became_active":"2023-10-16T11:06:23.602818+0000","last_became_peered":"2023-10-16T11:06:23.602818+0000","last_unstale":"2023-10-16T11:06:25.692312+0000","last_undegraded":"2023-10-16T11:06:25.692312+0000","last_fullsized":"2023-10-16T11:06:25.692312+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:15:05.089108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.623948+0000","last_change":"2023-10-16T11:06:22.606664+0000","last_active":"2023-10-16T11:06:23.623948+0000","last_peered":"2023-10-16T11:06:23.623948+0000","last_clean":"2023-10-16T11:06:23.623948+0000","last_became_active":"2023-10-16T11:06:22.606461+0000","last_became_peered":"2023-10-16T11:06:22.606461+0000","last_unstale":"2023-10-16T11:06:23.623948+0000","last_undegraded":"2023-10-16T11:06:23.623948+0000","last_fullsized":"2023-10-16T11:06:23.623948+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:33.775278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.188835+0000","last_change":"2023-10-16T11:06:23.617883+0000","last_active":"2023-10-16T11:06:25.188835+0000","last_peered":"2023-10-16T11:06:25.188835+0000","last_clean":"2023-10-16T11:06:25.188835+0000","last_became_active":"2023-10-16T11:06:23.617478+0000","last_became_peered":"2023-10-16T11:06:23.617478+0000","last_unstale":"2023-10-16T11:06:25.188835+0000","last_undegraded":"2023-10-16T11:06:25.188835+0000","last_fullsized":"2023-10-16T11:06:25.188835+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:44:42.439099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"183'1","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.625827+0000","last_change":"2023-10-16T11:06:22.600437+0000","last_active":"2023-10-16T11:06:23.625827+0000","last_peered":"2023-10-16T11:06:23.625827+0000","last_clean":"2023-10-16T11:06:23.625827+0000","last_became_active":"2023-10-16T11:06:22.599713+0000","last_became_peered":"2023-10-16T11:06:22.599713+0000","last_unstale":"2023-10-16T11:06:23.625827+0000","last_undegraded":"2023-10-16T11:06:23.625827+0000","last_fullsized":"2023-10-16T11:06:23.625827+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:28.229948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.682430+0000","last_change":"2023-10-16T11:06:23.619789+0000","last_active":"2023-10-16T11:06:24.682430+0000","last_peered":"2023-10-16T11:06:24.682430+0000","last_clean":"2023-10-16T11:06:24.682430+0000","last_became_active":"2023-10-16T11:06:23.619515+0000","last_became_peered":"2023-10-16T11:06:23.619515+0000","last_unstale":"2023-10-16T11:06:24.682430+0000","last_undegraded":"2023-10-16T11:06:24.682430+0000","last_fullsized":"2023-10-16T11:06:24.682430+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:11:30.976449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"183'1","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.623390+0000","last_change":"2023-10-16T11:06:22.597888+0000","last_active":"2023-10-16T11:06:23.623390+0000","last_peered":"2023-10-16T11:06:23.623390+0000","last_clean":"2023-10-16T11:06:23.623390+0000","last_became_active":"2023-10-16T11:06:22.597350+0000","last_became_peered":"2023-10-16T11:06:22.597350+0000","last_unstale":"2023-10-16T11:06:23.623390+0000","last_undegraded":"2023-10-16T11:06:23.623390+0000","last_fullsized":"2023-10-16T11:06:23.623390+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:22.856408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.682110+0000","last_change":"2023-10-16T11:06:23.614177+0000","last_active":"2023-10-16T11:06:24.682110+0000","last_peered":"2023-10-16T11:06:24.682110+0000","last_clean":"2023-10-16T11:06:24.682110+0000","last_became_active":"2023-10-16T11:06:23.613292+0000","last_became_peered":"2023-10-16T11:06:23.613292+0000","last_unstale":"2023-10-16T11:06:24.682110+0000","last_undegraded":"2023-10-16T11:06:24.682110+0000","last_fullsized":"2023-10-16T11:06:24.682110+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:52:43.765054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.596043+0000","last_change":"2023-10-16T11:06:22.596029+0000","last_active":"2023-10-16T11:06:23.596043+0000","last_peered":"2023-10-16T11:06:23.596043+0000","last_clean":"2023-10-16T11:06:23.596043+0000","last_became_active":"2023-10-16T11:06:22.595616+0000","last_became_peered":"2023-10-16T11:06:22.595616+0000","last_unstale":"2023-10-16T11:06:23.596043+0000","last_undegraded":"2023-10-16T11:06:23.596043+0000","last_fullsized":"2023-10-16T11:06:23.596043+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:05.357813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.605484+0000","last_change":"2023-10-16T11:06:23.608846+0000","last_active":"2023-10-16T11:06:24.605484+0000","last_peered":"2023-10-16T11:06:24.605484+0000","last_clean":"2023-10-16T11:06:24.605484+0000","last_became_active":"2023-10-16T11:06:23.607714+0000","last_became_peered":"2023-10-16T11:06:23.607714+0000","last_unstale":"2023-10-16T11:06:24.605484+0000","last_undegraded":"2023-10-16T11:06:24.605484+0000","last_fullsized":"2023-10-16T11:06:24.605484+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:40.441635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.594690+0000","last_change":"2023-10-16T11:06:22.585438+0000","last_active":"2023-10-16T11:06:23.594690+0000","last_peered":"2023-10-16T11:06:23.594690+0000","last_clean":"2023-10-16T11:06:23.594690+0000","last_became_active":"2023-10-16T11:06:22.584253+0000","last_became_peered":"2023-10-16T11:06:22.584253+0000","last_unstale":"2023-10-16T11:06:23.594690+0000","last_undegraded":"2023-10-16T11:06:23.594690+0000","last_fullsized":"2023-10-16T11:06:23.594690+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:21.385789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.608422+0000","last_change":"2023-10-16T11:06:23.606712+0000","last_active":"2023-10-16T11:06:24.608422+0000","last_peered":"2023-10-16T11:06:24.608422+0000","last_clean":"2023-10-16T11:06:24.608422+0000","last_became_active":"2023-10-16T11:06:23.606253+0000","last_became_peered":"2023-10-16T11:06:23.606253+0000","last_unstale":"2023-10-16T11:06:24.608422+0000","last_undegraded":"2023-10-16T11:06:24.608422+0000","last_fullsized":"2023-10-16T11:06:24.608422+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:11.282478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.582956+0000","last_change":"2023-10-16T11:06:22.600873+0000","last_active":"2023-10-16T11:06:23.582956+0000","last_peered":"2023-10-16T11:06:23.582956+0000","last_clean":"2023-10-16T11:06:23.582956+0000","last_became_active":"2023-10-16T11:06:22.600547+0000","last_became_peered":"2023-10-16T11:06:22.600547+0000","last_unstale":"2023-10-16T11:06:23.582956+0000","last_undegraded":"2023-10-16T11:06:23.582956+0000","last_fullsized":"2023-10-16T11:06:23.582956+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:37.146515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.607971+0000","last_change":"2023-10-16T11:06:23.603842+0000","last_active":"2023-10-16T11:06:24.607971+0000","last_peered":"2023-10-16T11:06:24.607971+0000","last_clean":"2023-10-16T11:06:24.607971+0000","last_became_active":"2023-10-16T11:06:23.603588+0000","last_became_peered":"2023-10-16T11:06:23.603588+0000","last_unstale":"2023-10-16T11:06:24.607971+0000","last_undegraded":"2023-10-16T11:06:24.607971+0000","last_fullsized":"2023-10-16T11:06:24.607971+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:31:39.879188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"183'3","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.624770+0000","last_change":"2023-10-16T11:06:22.601095+0000","last_active":"2023-10-16T11:06:23.624770+0000","last_peered":"2023-10-16T11:06:23.624770+0000","last_clean":"2023-10-16T11:06:23.624770+0000","last_became_active":"2023-10-16T11:06:22.600809+0000","last_became_peered":"2023-10-16T11:06:22.600809+0000","last_unstale":"2023-10-16T11:06:23.624770+0000","last_undegraded":"2023-10-16T11:06:23.624770+0000","last_fullsized":"2023-10-16T11:06:23.624770+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:24.846010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.608336+0000","last_change":"2023-10-16T11:06:23.607578+0000","last_active":"2023-10-16T11:06:24.608336+0000","last_peered":"2023-10-16T11:06:24.608336+0000","last_clean":"2023-10-16T11:06:24.608336+0000","last_became_active":"2023-10-16T11:06:23.607238+0000","last_became_peered":"2023-10-16T11:06:23.607238+0000","last_unstale":"2023-10-16T11:06:24.608336+0000","last_undegraded":"2023-10-16T11:06:24.608336+0000","last_fullsized":"2023-10-16T11:06:24.608336+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:10:51.251237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"183'6","reported_seq":18,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.629331+0000","last_change":"2023-10-16T11:06:22.602204+0000","last_active":"2023-10-16T11:06:23.629331+0000","last_peered":"2023-10-16T11:06:23.629331+0000","last_clean":"2023-10-16T11:06:23.629331+0000","last_became_active":"2023-10-16T11:06:22.601239+0000","last_became_peered":"2023-10-16T11:06:22.601239+0000","last_unstale":"2023-10-16T11:06:23.629331+0000","last_undegraded":"2023-10-16T11:06:23.629331+0000","last_fullsized":"2023-10-16T11:06:23.629331+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:33.775278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.607564+0000","last_change":"2023-10-16T11:06:23.607824+0000","last_active":"2023-10-16T11:06:24.607564+0000","last_peered":"2023-10-16T11:06:24.607564+0000","last_clean":"2023-10-16T11:06:24.607564+0000","last_became_active":"2023-10-16T11:06:23.607557+0000","last_became_peered":"2023-10-16T11:06:23.607557+0000","last_unstale":"2023-10-16T11:06:24.607564+0000","last_undegraded":"2023-10-16T11:06:24.607564+0000","last_fullsized":"2023-10-16T11:06:24.607564+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:37:07.386405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.624019+0000","last_change":"2023-10-16T11:06:22.598426+0000","last_active":"2023-10-16T11:06:23.624019+0000","last_peered":"2023-10-16T11:06:23.624019+0000","last_clean":"2023-10-16T11:06:23.624019+0000","last_became_active":"2023-10-16T11:06:22.597766+0000","last_became_peered":"2023-10-16T11:06:22.597766+0000","last_unstale":"2023-10-16T11:06:23.624019+0000","last_undegraded":"2023-10-16T11:06:23.624019+0000","last_fullsized":"2023-10-16T11:06:23.624019+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:15:04.069554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.188867+0000","last_change":"2023-10-16T11:06:23.612172+0000","last_active":"2023-10-16T11:06:25.188867+0000","last_peered":"2023-10-16T11:06:25.188867+0000","last_clean":"2023-10-16T11:06:25.188867+0000","last_became_active":"2023-10-16T11:06:23.611776+0000","last_became_peered":"2023-10-16T11:06:23.611776+0000","last_unstale":"2023-10-16T11:06:25.188867+0000","last_undegraded":"2023-10-16T11:06:25.188867+0000","last_fullsized":"2023-10-16T11:06:25.188867+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:22:52.230491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.580924+0000","last_change":"2023-10-16T11:06:22.593856+0000","last_active":"2023-10-16T11:06:23.580924+0000","last_peered":"2023-10-16T11:06:23.580924+0000","last_clean":"2023-10-16T11:06:23.580924+0000","last_became_active":"2023-10-16T11:06:22.593580+0000","last_became_peered":"2023-10-16T11:06:22.593580+0000","last_unstale":"2023-10-16T11:06:23.580924+0000","last_undegraded":"2023-10-16T11:06:23.580924+0000","last_fullsized":"2023-10-16T11:06:23.580924+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:22:31.426812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.682111+0000","last_change":"2023-10-16T11:06:23.609053+0000","last_active":"2023-10-16T11:06:24.682111+0000","last_peered":"2023-10-16T11:06:24.682111+0000","last_clean":"2023-10-16T11:06:24.682111+0000","last_became_active":"2023-10-16T11:06:23.608765+0000","last_became_peered":"2023-10-16T11:06:23.608765+0000","last_unstale":"2023-10-16T11:06:24.682111+0000","last_undegraded":"2023-10-16T11:06:24.682111+0000","last_fullsized":"2023-10-16T11:06:24.682111+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:49:24.469079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.593027+0000","last_change":"2023-10-16T11:06:22.600102+0000","last_active":"2023-10-16T11:06:23.593027+0000","last_peered":"2023-10-16T11:06:23.593027+0000","last_clean":"2023-10-16T11:06:23.593027+0000","last_became_active":"2023-10-16T11:06:22.599762+0000","last_became_peered":"2023-10-16T11:06:22.599762+0000","last_unstale":"2023-10-16T11:06:23.593027+0000","last_undegraded":"2023-10-16T11:06:23.593027+0000","last_fullsized":"2023-10-16T11:06:23.593027+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:03.924849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.690834+0000","last_change":"2023-10-16T11:06:23.618703+0000","last_active":"2023-10-16T11:06:25.690834+0000","last_peered":"2023-10-16T11:06:25.690834+0000","last_clean":"2023-10-16T11:06:25.690834+0000","last_became_active":"2023-10-16T11:06:23.618407+0000","last_became_peered":"2023-10-16T11:06:23.618407+0000","last_unstale":"2023-10-16T11:06:25.690834+0000","last_undegraded":"2023-10-16T11:06:25.690834+0000","last_fullsized":"2023-10-16T11:06:25.690834+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:47:13.076842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"183'8","reported_seq":20,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.626698+0000","last_change":"2023-10-16T11:06:22.600038+0000","last_active":"2023-10-16T11:06:23.626698+0000","last_peered":"2023-10-16T11:06:23.626698+0000","last_clean":"2023-10-16T11:06:23.626698+0000","last_became_active":"2023-10-16T11:06:22.599724+0000","last_became_peered":"2023-10-16T11:06:22.599724+0000","last_unstale":"2023-10-16T11:06:23.626698+0000","last_undegraded":"2023-10-16T11:06:23.626698+0000","last_fullsized":"2023-10-16T11:06:23.626698+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:05:00.713229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.682741+0000","last_change":"2023-10-16T11:06:23.604069+0000","last_active":"2023-10-16T11:06:24.682741+0000","last_peered":"2023-10-16T11:06:24.682741+0000","last_clean":"2023-10-16T11:06:24.682741+0000","last_became_active":"2023-10-16T11:06:23.603632+0000","last_became_peered":"2023-10-16T11:06:23.603632+0000","last_unstale":"2023-10-16T11:06:24.682741+0000","last_undegraded":"2023-10-16T11:06:24.682741+0000","last_fullsized":"2023-10-16T11:06:24.682741+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:07:18.531732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"183'1","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.625377+0000","last_change":"2023-10-16T11:06:22.603319+0000","last_active":"2023-10-16T11:06:23.625377+0000","last_peered":"2023-10-16T11:06:23.625377+0000","last_clean":"2023-10-16T11:06:23.625377+0000","last_became_active":"2023-10-16T11:06:22.603029+0000","last_became_peered":"2023-10-16T11:06:22.603029+0000","last_unstale":"2023-10-16T11:06:23.625377+0000","last_undegraded":"2023-10-16T11:06:23.625377+0000","last_fullsized":"2023-10-16T11:06:23.625377+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:22:51.210936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.608424+0000","last_change":"2023-10-16T11:06:23.598934+0000","last_active":"2023-10-16T11:06:24.608424+0000","last_peered":"2023-10-16T11:06:24.608424+0000","last_clean":"2023-10-16T11:06:24.608424+0000","last_became_active":"2023-10-16T11:06:23.598635+0000","last_became_peered":"2023-10-16T11:06:23.598635+0000","last_unstale":"2023-10-16T11:06:24.608424+0000","last_undegraded":"2023-10-16T11:06:24.608424+0000","last_fullsized":"2023-10-16T11:06:24.608424+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:33:13.769975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.622238+0000","last_change":"2023-10-16T11:06:22.598212+0000","last_active":"2023-10-16T11:06:23.622238+0000","last_peered":"2023-10-16T11:06:23.622238+0000","last_clean":"2023-10-16T11:06:23.622238+0000","last_became_active":"2023-10-16T11:06:22.597864+0000","last_became_peered":"2023-10-16T11:06:22.597864+0000","last_unstale":"2023-10-16T11:06:23.622238+0000","last_undegraded":"2023-10-16T11:06:23.622238+0000","last_fullsized":"2023-10-16T11:06:23.622238+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:39.422081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.682729+0000","last_change":"2023-10-16T11:06:23.620695+0000","last_active":"2023-10-16T11:06:24.682729+0000","last_peered":"2023-10-16T11:06:24.682729+0000","last_clean":"2023-10-16T11:06:24.682729+0000","last_became_active":"2023-10-16T11:06:23.620431+0000","last_became_peered":"2023-10-16T11:06:23.620431+0000","last_unstale":"2023-10-16T11:06:24.682729+0000","last_undegraded":"2023-10-16T11:06:24.682729+0000","last_fullsized":"2023-10-16T11:06:24.682729+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:10:51.251237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"183'1","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.625933+0000","last_change":"2023-10-16T11:06:22.605078+0000","last_active":"2023-10-16T11:06:23.625933+0000","last_peered":"2023-10-16T11:06:23.625933+0000","last_clean":"2023-10-16T11:06:23.625933+0000","last_became_active":"2023-10-16T11:06:22.604777+0000","last_became_peered":"2023-10-16T11:06:22.604777+0000","last_unstale":"2023-10-16T11:06:23.625933+0000","last_undegraded":"2023-10-16T11:06:23.625933+0000","last_fullsized":"2023-10-16T11:06:23.625933+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:41:07.425104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.188987+0000","last_change":"2023-10-16T11:06:23.611858+0000","last_active":"2023-10-16T11:06:25.188987+0000","last_peered":"2023-10-16T11:06:25.188987+0000","last_clean":"2023-10-16T11:06:25.188987+0000","last_became_active":"2023-10-16T11:06:23.611532+0000","last_became_peered":"2023-10-16T11:06:23.611532+0000","last_unstale":"2023-10-16T11:06:25.188987+0000","last_undegraded":"2023-10-16T11:06:25.188987+0000","last_fullsized":"2023-10-16T11:06:25.188987+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:27.107177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.589557+0000","last_change":"2023-10-16T11:06:22.603376+0000","last_active":"2023-10-16T11:06:23.589557+0000","last_peered":"2023-10-16T11:06:23.589557+0000","last_clean":"2023-10-16T11:06:23.589557+0000","last_became_active":"2023-10-16T11:06:22.602995+0000","last_became_peered":"2023-10-16T11:06:22.602995+0000","last_unstale":"2023-10-16T11:06:23.589557+0000","last_undegraded":"2023-10-16T11:06:23.589557+0000","last_fullsized":"2023-10-16T11:06:23.589557+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:00.081712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.691324+0000","last_change":"2023-10-16T11:06:23.613081+0000","last_active":"2023-10-16T11:06:25.691324+0000","last_peered":"2023-10-16T11:06:25.691324+0000","last_clean":"2023-10-16T11:06:25.691324+0000","last_became_active":"2023-10-16T11:06:23.611374+0000","last_became_peered":"2023-10-16T11:06:23.611374+0000","last_unstale":"2023-10-16T11:06:25.691324+0000","last_undegraded":"2023-10-16T11:06:25.691324+0000","last_fullsized":"2023-10-16T11:06:25.691324+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:28:21.762166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.625321+0000","last_change":"2023-10-16T11:06:22.599579+0000","last_active":"2023-10-16T11:06:23.625321+0000","last_peered":"2023-10-16T11:06:23.625321+0000","last_clean":"2023-10-16T11:06:23.625321+0000","last_became_active":"2023-10-16T11:06:22.599013+0000","last_became_peered":"2023-10-16T11:06:22.599013+0000","last_unstale":"2023-10-16T11:06:23.625321+0000","last_undegraded":"2023-10-16T11:06:23.625321+0000","last_fullsized":"2023-10-16T11:06:23.625321+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:28:11.350072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.188995+0000","last_change":"2023-10-16T11:06:23.609436+0000","last_active":"2023-10-16T11:06:25.188995+0000","last_peered":"2023-10-16T11:06:25.188995+0000","last_clean":"2023-10-16T11:06:25.188995+0000","last_became_active":"2023-10-16T11:06:23.608957+0000","last_became_peered":"2023-10-16T11:06:23.608957+0000","last_unstale":"2023-10-16T11:06:25.188995+0000","last_undegraded":"2023-10-16T11:06:25.188995+0000","last_fullsized":"2023-10-16T11:06:25.188995+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:58.721072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.588628+0000","last_change":"2023-10-16T11:06:22.602334+0000","last_active":"2023-10-16T11:06:23.588628+0000","last_peered":"2023-10-16T11:06:23.588628+0000","last_clean":"2023-10-16T11:06:23.588628+0000","last_became_active":"2023-10-16T11:06:22.600968+0000","last_became_peered":"2023-10-16T11:06:22.600968+0000","last_unstale":"2023-10-16T11:06:23.588628+0000","last_undegraded":"2023-10-16T11:06:23.588628+0000","last_fullsized":"2023-10-16T11:06:23.588628+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:46:00.461827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.689432+0000","last_change":"2023-10-16T11:06:23.611058+0000","last_active":"2023-10-16T11:06:25.689432+0000","last_peered":"2023-10-16T11:06:25.689432+0000","last_clean":"2023-10-16T11:06:25.689432+0000","last_became_active":"2023-10-16T11:06:23.610730+0000","last_became_peered":"2023-10-16T11:06:23.610730+0000","last_unstale":"2023-10-16T11:06:25.689432+0000","last_undegraded":"2023-10-16T11:06:25.689432+0000","last_fullsized":"2023-10-16T11:06:25.689432+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:19.812532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.593339+0000","last_change":"2023-10-16T11:06:22.601790+0000","last_active":"2023-10-16T11:06:23.593339+0000","last_peered":"2023-10-16T11:06:23.593339+0000","last_clean":"2023-10-16T11:06:23.593339+0000","last_became_active":"2023-10-16T11:06:22.601481+0000","last_became_peered":"2023-10-16T11:06:22.601481+0000","last_unstale":"2023-10-16T11:06:23.593339+0000","last_undegraded":"2023-10-16T11:06:23.593339+0000","last_fullsized":"2023-10-16T11:06:23.593339+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:04:24.297177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.605763+0000","last_change":"2023-10-16T11:06:23.609513+0000","last_active":"2023-10-16T11:06:24.605763+0000","last_peered":"2023-10-16T11:06:24.605763+0000","last_clean":"2023-10-16T11:06:24.605763+0000","last_became_active":"2023-10-16T11:06:23.609147+0000","last_became_peered":"2023-10-16T11:06:23.609147+0000","last_unstale":"2023-10-16T11:06:24.605763+0000","last_undegraded":"2023-10-16T11:06:24.605763+0000","last_fullsized":"2023-10-16T11:06:24.605763+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:53.501651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.590572+0000","last_change":"2023-10-16T11:06:22.603971+0000","last_active":"2023-10-16T11:06:23.590572+0000","last_peered":"2023-10-16T11:06:23.590572+0000","last_clean":"2023-10-16T11:06:23.590572+0000","last_became_active":"2023-10-16T11:06:22.603652+0000","last_became_peered":"2023-10-16T11:06:22.603652+0000","last_unstale":"2023-10-16T11:06:23.590572+0000","last_undegraded":"2023-10-16T11:06:23.590572+0000","last_fullsized":"2023-10-16T11:06:23.590572+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:06:42.071943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.608691+0000","last_change":"2023-10-16T11:06:23.604780+0000","last_active":"2023-10-16T11:06:24.608691+0000","last_peered":"2023-10-16T11:06:24.608691+0000","last_clean":"2023-10-16T11:06:24.608691+0000","last_became_active":"2023-10-16T11:06:23.602656+0000","last_became_peered":"2023-10-16T11:06:23.602656+0000","last_unstale":"2023-10-16T11:06:24.608691+0000","last_undegraded":"2023-10-16T11:06:24.608691+0000","last_fullsized":"2023-10-16T11:06:24.608691+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:28:20.180992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"183'2","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.625450+0000","last_change":"2023-10-16T11:06:22.591878+0000","last_active":"2023-10-16T11:06:23.625450+0000","last_peered":"2023-10-16T11:06:23.625450+0000","last_clean":"2023-10-16T11:06:23.625450+0000","last_became_active":"2023-10-16T11:06:22.591618+0000","last_became_peered":"2023-10-16T11:06:22.591618+0000","last_unstale":"2023-10-16T11:06:23.625450+0000","last_undegraded":"2023-10-16T11:06:23.625450+0000","last_fullsized":"2023-10-16T11:06:23.625450+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:22:36.576896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.189313+0000","last_change":"2023-10-16T11:06:23.617435+0000","last_active":"2023-10-16T11:06:25.189313+0000","last_peered":"2023-10-16T11:06:25.189313+0000","last_clean":"2023-10-16T11:06:25.189313+0000","last_became_active":"2023-10-16T11:06:23.617176+0000","last_became_peered":"2023-10-16T11:06:23.617176+0000","last_unstale":"2023-10-16T11:06:25.189313+0000","last_undegraded":"2023-10-16T11:06:25.189313+0000","last_fullsized":"2023-10-16T11:06:25.189313+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:40.441635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.596694+0000","last_change":"2023-10-16T11:06:22.602848+0000","last_active":"2023-10-16T11:06:23.596694+0000","last_peered":"2023-10-16T11:06:23.596694+0000","last_clean":"2023-10-16T11:06:23.596694+0000","last_became_active":"2023-10-16T11:06:22.600563+0000","last_became_peered":"2023-10-16T11:06:22.600563+0000","last_unstale":"2023-10-16T11:06:23.596694+0000","last_undegraded":"2023-10-16T11:06:23.596694+0000","last_fullsized":"2023-10-16T11:06:23.596694+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:52.482097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.605364+0000","last_change":"2023-10-16T11:06:23.598565+0000","last_active":"2023-10-16T11:06:24.605364+0000","last_peered":"2023-10-16T11:06:24.605364+0000","last_clean":"2023-10-16T11:06:24.605364+0000","last_became_active":"2023-10-16T11:06:23.598298+0000","last_became_peered":"2023-10-16T11:06:23.598298+0000","last_unstale":"2023-10-16T11:06:24.605364+0000","last_undegraded":"2023-10-16T11:06:24.605364+0000","last_fullsized":"2023-10-16T11:06:24.605364+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:27.107177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.599169+0000","last_change":"2023-10-16T11:06:22.597633+0000","last_active":"2023-10-16T11:06:23.599169+0000","last_peered":"2023-10-16T11:06:23.599169+0000","last_clean":"2023-10-16T11:06:23.599169+0000","last_became_active":"2023-10-16T11:06:22.597321+0000","last_became_peered":"2023-10-16T11:06:22.597321+0000","last_unstale":"2023-10-16T11:06:23.599169+0000","last_undegraded":"2023-10-16T11:06:23.599169+0000","last_fullsized":"2023-10-16T11:06:23.599169+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:47:30.215796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.608073+0000","last_change":"2023-10-16T11:06:23.608326+0000","last_active":"2023-10-16T11:06:24.608073+0000","last_peered":"2023-10-16T11:06:24.608073+0000","last_clean":"2023-10-16T11:06:24.608073+0000","last_became_active":"2023-10-16T11:06:23.608006+0000","last_became_peered":"2023-10-16T11:06:23.608006+0000","last_unstale":"2023-10-16T11:06:24.608073+0000","last_undegraded":"2023-10-16T11:06:24.608073+0000","last_fullsized":"2023-10-16T11:06:24.608073+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:49.381022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.579231+0000","last_change":"2023-10-16T11:06:22.598914+0000","last_active":"2023-10-16T11:06:23.579231+0000","last_peered":"2023-10-16T11:06:23.579231+0000","last_clean":"2023-10-16T11:06:23.579231+0000","last_became_active":"2023-10-16T11:06:22.598150+0000","last_became_peered":"2023-10-16T11:06:22.598150+0000","last_unstale":"2023-10-16T11:06:23.579231+0000","last_undegraded":"2023-10-16T11:06:23.579231+0000","last_fullsized":"2023-10-16T11:06:23.579231+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:47.148066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.607659+0000","last_change":"2023-10-16T11:06:23.591515+0000","last_active":"2023-10-16T11:06:24.607659+0000","last_peered":"2023-10-16T11:06:24.607659+0000","last_clean":"2023-10-16T11:06:24.607659+0000","last_became_active":"2023-10-16T11:06:23.591080+0000","last_became_peered":"2023-10-16T11:06:23.591080+0000","last_unstale":"2023-10-16T11:06:24.607659+0000","last_undegraded":"2023-10-16T11:06:24.607659+0000","last_fullsized":"2023-10-16T11:06:24.607659+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:41.358248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.580150+0000","last_change":"2023-10-16T11:06:22.593798+0000","last_active":"2023-10-16T11:06:23.580150+0000","last_peered":"2023-10-16T11:06:23.580150+0000","last_clean":"2023-10-16T11:06:23.580150+0000","last_became_active":"2023-10-16T11:06:22.593451+0000","last_became_peered":"2023-10-16T11:06:22.593451+0000","last_unstale":"2023-10-16T11:06:23.580150+0000","last_undegraded":"2023-10-16T11:06:23.580150+0000","last_fullsized":"2023-10-16T11:06:23.580150+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:49:23.449525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.607670+0000","last_change":"2023-10-16T11:06:23.608011+0000","last_active":"2023-10-16T11:06:24.607670+0000","last_peered":"2023-10-16T11:06:24.607670+0000","last_clean":"2023-10-16T11:06:24.607670+0000","last_became_active":"2023-10-16T11:06:23.607460+0000","last_became_peered":"2023-10-16T11:06:23.607460+0000","last_unstale":"2023-10-16T11:06:24.607670+0000","last_undegraded":"2023-10-16T11:06:24.607670+0000","last_fullsized":"2023-10-16T11:06:24.607670+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:56:48.322204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.596786+0000","last_change":"2023-10-16T11:06:22.591391+0000","last_active":"2023-10-16T11:06:23.596786+0000","last_peered":"2023-10-16T11:06:23.596786+0000","last_clean":"2023-10-16T11:06:23.596786+0000","last_became_active":"2023-10-16T11:06:22.590826+0000","last_became_peered":"2023-10-16T11:06:22.590826+0000","last_unstale":"2023-10-16T11:06:23.596786+0000","last_undegraded":"2023-10-16T11:06:23.596786+0000","last_fullsized":"2023-10-16T11:06:23.596786+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:05:00.713229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.188098+0000","last_change":"2023-10-16T11:06:23.607961+0000","last_active":"2023-10-16T11:06:25.188098+0000","last_peered":"2023-10-16T11:06:25.188098+0000","last_clean":"2023-10-16T11:06:25.188098+0000","last_became_active":"2023-10-16T11:06:23.607661+0000","last_became_peered":"2023-10-16T11:06:23.607661+0000","last_unstale":"2023-10-16T11:06:25.188098+0000","last_undegraded":"2023-10-16T11:06:25.188098+0000","last_fullsized":"2023-10-16T11:06:25.188098+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:33:49.051160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.596232+0000","last_change":"2023-10-16T11:06:22.592760+0000","last_active":"2023-10-16T11:06:23.596232+0000","last_peered":"2023-10-16T11:06:23.596232+0000","last_clean":"2023-10-16T11:06:23.596232+0000","last_became_active":"2023-10-16T11:06:22.592476+0000","last_became_peered":"2023-10-16T11:06:22.592476+0000","last_unstale":"2023-10-16T11:06:23.596232+0000","last_undegraded":"2023-10-16T11:06:23.596232+0000","last_fullsized":"2023-10-16T11:06:23.596232+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:10:14.100974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:25.690857+0000","last_change":"2023-10-16T11:06:23.606624+0000","last_active":"2023-10-16T11:06:25.690857+0000","last_peered":"2023-10-16T11:06:25.690857+0000","last_clean":"2023-10-16T11:06:25.690857+0000","last_became_active":"2023-10-16T11:06:23.605844+0000","last_became_peered":"2023-10-16T11:06:23.605844+0000","last_unstale":"2023-10-16T11:06:25.690857+0000","last_undegraded":"2023-10-16T11:06:25.690857+0000","last_fullsized":"2023-10-16T11:06:25.690857+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:28:03.417569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.579944+0000","last_change":"2023-10-16T11:06:22.594879+0000","last_active":"2023-10-16T11:06:23.579944+0000","last_peered":"2023-10-16T11:06:23.579944+0000","last_clean":"2023-10-16T11:06:23.579944+0000","last_became_active":"2023-10-16T11:06:22.590604+0000","last_became_peered":"2023-10-16T11:06:22.590604+0000","last_unstale":"2023-10-16T11:06:23.579944+0000","last_undegraded":"2023-10-16T11:06:23.579944+0000","last_fullsized":"2023-10-16T11:06:23.579944+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:00.276431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-16T11:06:24.608692+0000","last_change":"2023-10-16T11:06:23.604954+0000","last_active":"2023-10-16T11:06:24.608692+0000","last_peered":"2023-10-16T11:06:24.608692+0000","last_clean":"2023-10-16T11:06:24.608692+0000","last_became_active":"2023-10-16T11:06:23.604416+0000","last_became_peered":"2023-10-16T11:06:23.604416+0000","last_unstale":"2023-10-16T11:06:24.608692+0000","last_undegraded":"2023-10-16T11:06:24.608692+0000","last_fullsized":"2023-10-16T11:06:24.608692+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:22.550027+0000","last_clean_scrub_stamp":"2023-10-16T11:06:22.550027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:21.726501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.580612+0000","last_change":"2023-10-16T11:06:22.595710+0000","last_active":"2023-10-16T11:06:23.580612+0000","last_peered":"2023-10-16T11:06:23.580612+0000","last_clean":"2023-10-16T11:06:23.580612+0000","last_became_active":"2023-10-16T11:06:22.593479+0000","last_became_peered":"2023-10-16T11:06:22.593479+0000","last_unstale":"2023-10-16T11:06:23.580612+0000","last_undegraded":"2023-10-16T11:06:23.580612+0000","last_fullsized":"2023-10-16T11:06:23.580612+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:06:21.530473+0000","last_clean_scrub_stamp":"2023-10-16T11:06:21.530473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:52:42.745500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"173'2712","reported_seq":3420,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-16T11:06:23.590797+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:06:23.590797+0000","last_peered":"2023-10-16T11:06:23.590797+0000","last_clean":"2023-10-16T11:06:23.590797+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:06:23.590797+0000","last_undegraded":"2023-10-16T11:06:23.590797+0000","last_fullsized":"2023-10-16T11:06:23.590797+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2712,"log_dups_size":0,"ondisk_log_size":2712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1696,"num_read_kb":3438,"num_write":481,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1696,"num_read_kb":3438,"num_write":481,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2712,"ondisk_log_size":2712,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575772,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47920,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323920,"statfs":{"total":96636764160,"available":96587694080,"internally_reserved":0,"allocated":2920448,"data_stored":1701135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1739999999999999,"15min":1.2030000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6479999999999999,"5min":1.7230000000000001,"15min":1.853},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.2090000000000001,"15min":1.226},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.67300000000000004},"max":{"1min":1.4730000000000001,"5min":1.8029999999999999,"15min":1.9710000000000001},"last":0.871}]},{"osd":1,"last update":"Mon Oct 16 11:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2370000000000001,"15min":1.26},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.4990000000000001,"5min":1.796,"15min":1.907},"last":1.802},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.2350000000000001,"15min":1.2589999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.444,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.95299999999999996}]},{"osd":2,"last update":"Mon Oct 16 11:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.3400000000000001,"15min":1.3129999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.6519999999999999,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.524},{"interface":"front","average":{"1min":1.232,"5min":1.3580000000000001,"15min":1.3420000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.79100000000000004},"max":{"1min":1.5640000000000001,"5min":1.869,"15min":1.869},"last":1.2709999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.357,"15min":1.3680000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.5069999999999999,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.478,"15min":1.456},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.873,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.645}]},{"osd":4,"last update":"Mon Oct 16 11:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":1.444,"15min":1.4099999999999999},"min":{"1min":1.036,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":2.016,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.7569999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.3620000000000001,"15min":1.3540000000000001},"min":{"1min":0.87,"5min":0.87,"15min":0.74299999999999999},"max":{"1min":1.75,"5min":1.909,"15min":2.101},"last":1.978}]},{"osd":5,"last update":"Mon Oct 16 11:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.609,"15min":1.5640000000000001},"min":{"1min":1.21,"5min":0.99199999999999999,"15min":0.98799999999999999},"max":{"1min":2.0409999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.897},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.498,"15min":1.486},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.9770000000000001,"5min":2.0249999999999999,"15min":2.395},"last":2.0819999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.512,"15min":1.4850000000000001},"min":{"1min":1.0920000000000001,"5min":0.89200000000000002,"15min":0.88500000000000001},"max":{"1min":1.71,"5min":2.0680000000000001,"15min":2.1459999999999999},"last":1.931},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.7290000000000001,"15min":1.6839999999999999},"min":{"1min":1.415,"5min":1.093,"15min":1.093},"max":{"1min":2.1059999999999999,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":2.0299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575772,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29272,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342568,"statfs":{"total":96636764160,"available":96606789632,"internally_reserved":0,"allocated":2306048,"data_stored":1108435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:06:05 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.222,"15min":1.202},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.5209999999999999,"5min":1.665,"15min":1.665},"last":1.212},{"interface":"front","average":{"1min":1.175,"5min":1.256,"15min":1.264},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.0469999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:06:05 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.25,"15min":1.234},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7390000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.7609999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.3540000000000001,"15min":1.343},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.754,"5min":1.8220000000000001,"15min":2.1440000000000001},"last":1.431}]},{"osd":2,"last update":"Mon Oct 16 11:06:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.262,"15min":1.2589999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.696,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3740000000000001,"15min":1.3879999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6559999999999999,"5min":1.988,"15min":2.4129999999999998},"last":1.9390000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:06:05 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.272,"15min":1.2649999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.536},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.254,"15min":1.26},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.8300000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:06:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.319,"15min":1.339},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.845,"5min":1.9159999999999999,"15min":2.2130000000000001},"last":2.145},{"interface":"front","average":{"1min":1.462,"5min":1.5189999999999999,"15min":1.492},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.2490000000000001},"last":1.704}]},{"osd":5,"last update":"Mon Oct 16 11:06:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.3899999999999999,"15min":1.355},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.016,"5min":2.016,"15min":2.1869999999999998},"last":1.9870000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5820000000000001,"15min":1.575},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.9810000000000001,"5min":2.056,"15min":2.54},"last":2.0510000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:06:05 2023","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.534,"15min":1.4910000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.897},{"interface":"front","average":{"1min":1.641,"5min":1.5920000000000001,"15min":1.5529999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.2050000000000001},"last":2.0870000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575772,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41756,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330084,"statfs":{"total":96636764160,"available":96594006016,"internally_reserved":0,"allocated":2310144,"data_stored":1109017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:06:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.2210000000000001,"15min":1.262},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6220000000000001,"5min":1.7949999999999999,"15min":1.927},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.258,"5min":1.367,"15min":1.3540000000000001},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.5649999999999999,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.4430000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:06:21 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1930000000000001,"15min":1.196},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.786,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.2110000000000001,"15min":1.2150000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.579,"5min":1.6259999999999999,"15min":1.877},"last":1.319}]},{"osd":2,"last update":"Mon Oct 16 11:06:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1990000000000001,"15min":1.2170000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.627,"5min":1.667,"15min":1.895},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.3180000000000001,"15min":1.3089999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.4950000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:06:21 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.333,"15min":1.3049999999999999},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.88500000000000001},"max":{"1min":1.728,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.244},{"interface":"front","average":{"1min":1.347,"5min":1.4099999999999999,"15min":1.4199999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":2.1440000000000001},"last":1.377}]},{"osd":4,"last update":"Mon Oct 16 11:06:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.3049999999999999,"15min":1.2909999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.70299999999999996},"max":{"1min":1.802,"5min":1.853,"15min":1.9670000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5620000000000001,"15min":1.5569999999999999},"min":{"1min":1.2370000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.867,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.6479999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:06:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.4630000000000001,"15min":1.4239999999999999},"min":{"1min":0.999,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.4650000000000001,"15min":1.4430000000000001},"min":{"1min":0.98499999999999999,"5min":0.89100000000000001,"15min":0.86399999999999999},"max":{"1min":1.956,"5min":1.994,"15min":1.994},"last":1.423}]},{"osd":7,"last update":"Mon Oct 16 11:06:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.5820000000000001,"15min":1.5509999999999999},"min":{"1min":1.1339999999999999,"5min":0.91400000000000003,"15min":0.85399999999999998},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.2290000000000001},"last":1.591},{"interface":"front","average":{"1min":1.641,"5min":1.661,"15min":1.647},"min":{"1min":1.099,"5min":1.099,"15min":1.0149999999999999},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.6549999999999998},"last":2.0550000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59812,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312028,"statfs":{"total":96636764160,"available":96575516672,"internally_reserved":0,"allocated":2318336,"data_stored":1110095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.157,"15min":1.206},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.52600000000000002},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.173},{"interface":"front","average":{"1min":1.165,"5min":1.214,"15min":1.246},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9790000000000001},"last":1.4199999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.3260000000000001,"15min":1.3380000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.82899999999999996},"max":{"1min":1.782,"5min":1.8240000000000001,"15min":2.1680000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.216,"15min":1.254},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.581,"5min":1.7270000000000001,"15min":1.8080000000000001},"last":1.327}]},{"osd":2,"last update":"Mon Oct 16 11:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2889999999999999,"15min":1.284},"min":{"1min":0.872,"5min":0.81200000000000006,"15min":0.67200000000000004},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8560000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.347,"15min":1.4079999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.845,"5min":1.8859999999999999,"15min":2.234},"last":0.77600000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.3260000000000001,"15min":1.325},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.671,"5min":1.772,"15min":1.899},"last":1.46},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.444,"15min":1.4299999999999999},"min":{"1min":1.0049999999999999,"5min":0.97299999999999998,"15min":0.75800000000000001},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.49}]},{"osd":5,"last update":"Mon Oct 16 11:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.421,"15min":1.421},"min":{"1min":1.054,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.232},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4490000000000001,"15min":1.4059999999999999},"min":{"1min":1.238,"5min":0.93000000000000005,"15min":0.84299999999999997},"max":{"1min":1.7170000000000001,"5min":1.821,"15min":2.0819999999999999},"last":1.5760000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.486,"15min":1.46},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85699999999999998},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.3490000000000002},"last":1.286},{"interface":"front","average":{"1min":1.635,"5min":1.611,"15min":1.593},"min":{"1min":1.1899999999999999,"5min":1.109,"15min":0.75800000000000001},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.3450000000000002},"last":1.538}]},{"osd":7,"last update":"Mon Oct 16 11:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5680000000000001,"15min":1.5429999999999999},"min":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.04},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":2.117},"last":1.522},{"interface":"front","average":{"1min":1.6990000000000001,"5min":1.647,"15min":1.6359999999999999},"min":{"1min":1.129,"5min":1.119,"15min":0.874},"max":{"1min":2.7229999999999999,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":1.611}]}]},{"osd":3,"up_from":13,"seq":55834575772,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47320,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324520,"statfs":{"total":96636764160,"available":96588308480,"internally_reserved":0,"allocated":2306048,"data_stored":1108447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.90100000000000002,"15min":0.90200000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.968},"last":1.2430000000000001},{"interface":"front","average":{"1min":0.90000000000000002,"5min":1.0249999999999999,"15min":1.048},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0920000000000001,"5min":1.5800000000000001,"15min":2.1040000000000001},"last":0.99199999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.0129999999999999,"15min":0.998},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.351,"5min":1.5509999999999999,"15min":2.6269999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1200000000000001,"15min":1.1339999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.6299999999999999,"5min":1.651,"15min":2.2970000000000002},"last":1.331}]},{"osd":2,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.101,"15min":1.1140000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3879999999999999,"5min":1.641,"15min":2.3359999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.208,"15min":1.22},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5740000000000001,"5min":1.623,"15min":1.74},"last":0.95299999999999996}]},{"osd":4,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.367,"15min":1.343},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.724,"5min":1.74,"15min":2.1459999999999999},"last":1.446},{"interface":"front","average":{"1min":1.347,"5min":1.3540000000000001,"15min":1.347},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6319999999999999,"5min":1.7110000000000001,"15min":1.7889999999999999},"last":1.208}]},{"osd":5,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.379,"15min":1.385},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.97299999999999998},"max":{"1min":1.6699999999999999,"5min":1.897,"15min":2.4590000000000001},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.419,"15min":1.421},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8220000000000001,"5min":1.9910000000000001,"15min":2.089},"last":1.222}]},{"osd":6,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.5109999999999999,"15min":1.502},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.9510000000000001,"5min":2.3279999999999998,"15min":2.5760000000000001},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.486,"5min":1.522,"15min":1.522},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.0219999999999998,"5min":2.1509999999999998,"15min":2.4169999999999998},"last":1.3009999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:05:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.522,"15min":1.5269999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":2.29,"5min":2.29,"15min":2.746},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.635,"15min":1.643},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.8029999999999999},"last":1.353}]}]},{"osd":2,"up_from":13,"seq":55834575771,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51484,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320356,"statfs":{"total":96636764160,"available":96584044544,"internally_reserved":0,"allocated":2306048,"data_stored":1108503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.95599999999999996,"15min":0.93100000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.548,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.026,"15min":1.0680000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3540000000000001,"5min":1.5409999999999999,"15min":2.3010000000000002},"last":0.73099999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.1399999999999999,"15min":1.1080000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.657,"5min":1.7290000000000001,"15min":1.7749999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.121,"15min":1.1559999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.546,"5min":1.661,"15min":2.6120000000000001},"last":1.2869999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.109,"15min":1.085},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.3600000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.2849999999999999,"15min":1.266},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6910000000000001,"5min":1.899,"15min":1.99},"last":1.095}]},{"osd":4,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.351,"15min":1.3899999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.728,"5min":1.8340000000000001,"15min":2.5190000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.3580000000000001,"15min":1.341},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.4729999999999999,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":1.1639999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.476,"15min":1.4490000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.2559999999999998,"5min":2.2570000000000001,"15min":2.282},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4610000000000001,"15min":1.4710000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.2120000000000002,"5min":2.3450000000000002,"15min":2.9649999999999999},"last":1.391}]},{"osd":6,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.4239999999999999,"15min":1.4039999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.929,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.569,"15min":1.5269999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.0819999999999999,"5min":2.3420000000000001,"15min":2.5139999999999998},"last":1.4670000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:05:40 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.538,"15min":1.538},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.1389999999999998,"5min":2.141,"15min":3.1669999999999998},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.5169999999999999,"15min":1.514},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.2469999999999999,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":1.341}]}]},{"osd":1,"up_from":13,"seq":55834575771,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59800,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312040,"statfs":{"total":96636764160,"available":96575528960,"internally_reserved":0,"allocated":2306048,"data_stored":1108447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.88600000000000001,"15min":0.90500000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.0269999999999999,"15min":1.0469999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":4.5110000000000001,"5min":4.5110000000000001,"15min":4.5110000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.93100000000000005,"15min":0.99099999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.4930000000000001,"5min":1.5349999999999999,"15min":1.8069999999999999},"last":1.411},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.2170000000000001,"15min":1.256},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.4410000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:06:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.196,"15min":1.2070000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":4.351,"5min":4.351,"15min":4.351},"last":1.248},{"interface":"front","average":{"1min":0.89300000000000002,"5min":1.0960000000000001,"15min":1.131},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6419999999999999},"last":1.304}]},{"osd":4,"last update":"Mon Oct 16 11:06:17 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.2529999999999999,"15min":1.29},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.873,"5min":1.873,"15min":2.1030000000000002},"last":1.194},{"interface":"front","average":{"1min":1.294,"5min":1.3979999999999999,"15min":1.407},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":1.3480000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:06:17 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3859999999999999,"15min":1.3999999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":4.4400000000000004,"5min":4.4400000000000004,"15min":4.4400000000000004},"last":1.605},{"interface":"front","average":{"1min":1.28,"5min":1.383,"15min":1.387},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.089,"5min":2.8690000000000002,"15min":2.8690000000000002},"last":1.579}]},{"osd":6,"last update":"Mon Oct 16 11:06:17 2023","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.4890000000000001,"15min":1.472},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":7.2370000000000001,"5min":7.2370000000000001,"15min":7.2370000000000001},"last":1.738},{"interface":"front","average":{"1min":1.6859999999999999,"5min":1.4650000000000001,"15min":1.4550000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":10.683999999999999,"5min":10.683999999999999,"15min":10.683999999999999},"last":1.504}]},{"osd":7,"last update":"Mon Oct 16 11:06:17 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.4930000000000001,"15min":1.5249999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.5249999999999999,"15min":1.5649999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":4.5549999999999997,"5min":4.5549999999999997,"15min":4.5549999999999997},"last":1.8220000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575772,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52056,"kb_used_data":2828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319784,"statfs":{"total":96636764160,"available":96583458816,"internally_reserved":0,"allocated":2895872,"data_stored":1698781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.93200000000000005,"15min":0.90700000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.383,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.0329999999999999,"15min":1.0549999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4219999999999999,"5min":1.663,"15min":1.714},"last":0.47999999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":1.0369999999999999,"15min":1.0760000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.421,"5min":1.625,"15min":1.625},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.123,"15min":1.1419999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6060000000000001,"5min":1.659,"15min":1.823},"last":0.54900000000000004}]},{"osd":3,"last update":"Mon Oct 16 11:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":1.0760000000000001,"15min":1.0680000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.575,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.054,"5min":1.232,"15min":1.2509999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.536,"5min":1.6559999999999999,"15min":1.8480000000000001},"last":0.70699999999999996}]},{"osd":4,"last update":"Mon Oct 16 11:06:10 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3480000000000001,"15min":1.365},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.6659999999999999,"5min":1.7010000000000001,"15min":1.7410000000000001},"last":1.127},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.294,"15min":1.319},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.883,"5min":1.883,"15min":1.97},"last":1.1439999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:06:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.365,"15min":1.367},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":2.0259999999999998},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.4550000000000001,"15min":1.4570000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.0920000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.0960000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:06:10 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.492,"15min":1.5},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.7410000000000001,"5min":2.286,"15min":2.286},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.4650000000000001,"15min":1.4570000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.8320000000000001,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.0649999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:06:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.526,"15min":1.5229999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.4850000000000001,"15min":1.4730000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.1819999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:06:28.550 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T11:06:28.551 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T11:06:28.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.809+0000 7f698e533640 1 Processor -- start 2023-10-16T11:06:28.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.810+0000 7f698e533640 1 -- start start 2023-10-16T11:06:28.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.811+0000 7f698e533640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f69881615d0 0x7f698815db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:28.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.811+0000 7f698e533640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f698815e3c0 con 0x7f69880ccfc0 2023-10-16T11:06:28.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.811+0000 7f698e533640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f698815e0e0 con 0x7f69880b0b20 2023-10-16T11:06:28.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.811+0000 7f698e533640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6988160af0 con 0x7f69881615d0 2023-10-16T11:06:28.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.812+0000 7f6987fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f69880ccfc0 0x7f69880b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54640/0 (socket says 172.21.15.84:54640) 2023-10-16T11:06:28.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.812+0000 7f6987fff640 1 -- 172.21.15.84:0/1186937910 learned_addr learned my addr 172.21.15.84:0/1186937910 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:28.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.812+0000 7f698cad0640 1 --2- 172.21.15.84:0/1186937910 >> v2:172.21.15.157:3300/0 conn(0x7f69881615d0 0x7f698815db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:28.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.812+0000 7f69877fe640 1 -- 172.21.15.84:0/1186937910 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 913542381 0 0) 0x7f698815e3c0 con 0x7f69880ccfc0 2023-10-16T11:06:28.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.812+0000 7f69877fe640 1 -- 172.21.15.84:0/1186937910 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6974003520 con 0x7f69880ccfc0 2023-10-16T11:06:28.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.812+0000 7f69877fe640 1 -- 172.21.15.84:0/1186937910 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1114411108 0 0) 0x7f6974003520 con 0x7f69880ccfc0 2023-10-16T11:06:28.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.812+0000 7f69877fe640 1 -- 172.21.15.84:0/1186937910 >> v2:172.21.15.157:3300/0 conn(0x7f69881615d0 msgr2=0x7f698815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:28.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.812+0000 7f69877fe640 1 --2- 172.21.15.84:0/1186937910 >> v2:172.21.15.157:3300/0 conn(0x7f69881615d0 0x7f698815db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.813+0000 7f69877fe640 1 -- 172.21.15.84:0/1186937910 >> v1:172.21.15.157:6789/0 conn(0x7f69880b0b20 legacy=0x7f698815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:28.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.813+0000 7f69877fe640 1 -- 172.21.15.84:0/1186937910 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6988160dd0 con 0x7f69880ccfc0 2023-10-16T11:06:28.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.813+0000 7f69877fe640 1 -- 172.21.15.84:0/1186937910 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f6978001670 con 0x7f69880ccfc0 2023-10-16T11:06:28.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.813+0000 7f69877fe640 1 -- 172.21.15.84:0/1186937910 >> v1:172.21.15.84:6789/0 conn(0x7f69880ccfc0 legacy=0x7f69880b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:28.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.813+0000 7f69877fe640 1 --2- 172.21.15.84:0/1186937910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69740039c0 0x7f6974003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:28.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.813+0000 7f69877fe640 1 --2- 172.21.15.84:0/1186937910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69740047c0 0x7f6974004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:28.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.813+0000 7f69877fe640 1 --2- 172.21.15.84:0/1186937910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69740051d0 0x7f697400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:28.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.813+0000 7f69877fe640 1 -- 172.21.15.84:0/1186937910 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6988160af0 con 0x7f69740039c0 2023-10-16T11:06:28.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.813+0000 7f69877fe640 1 -- 172.21.15.84:0/1186937910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697400c120 con 0x7f69740051d0 2023-10-16T11:06:28.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.813+0000 7f69877fe640 1 -- 172.21.15.84:0/1186937910 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697400c2e0 con 0x7f69740047c0 2023-10-16T11:06:28.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.813+0000 7f6987fff640 1 --2- 172.21.15.84:0/1186937910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69740039c0 0x7f6974003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:28.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.813+0000 7f698cad0640 1 --2- 172.21.15.84:0/1186937910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69740051d0 0x7f697400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:28.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.813+0000 7f697ffff640 1 --2- 172.21.15.84:0/1186937910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69740047c0 0x7f6974004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:28.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.814+0000 7f6987fff640 1 -- 172.21.15.84:0/1186937910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69740047c0 msgr2=0x7f6974004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:28.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.814+0000 7f6987fff640 1 --2- 172.21.15.84:0/1186937910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69740047c0 0x7f6974004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.814+0000 7f6987fff640 1 -- 172.21.15.84:0/1186937910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69740051d0 msgr2=0x7f697400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:28.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.814+0000 7f6987fff640 1 --2- 172.21.15.84:0/1186937910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69740051d0 0x7f697400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.814+0000 7f6987fff640 1 -- 172.21.15.84:0/1186937910 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6974003520 con 0x7f69740039c0 2023-10-16T11:06:28.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.814+0000 7f698cad0640 1 -- 172.21.15.84:0/1186937910 reap_dead start 2023-10-16T11:06:28.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.815+0000 7f6987fff640 1 --2- 172.21.15.84:0/1186937910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69740039c0 0x7f6974003de0 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f6978001eb0 tx=0x7f69740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d52314ba8fc941c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:28.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.815+0000 7f69877fe640 1 -- 172.21.15.84:0/1186937910 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978005ad0 con 0x7f69740039c0 2023-10-16T11:06:28.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.815+0000 7f69877fe640 1 -- 172.21.15.84:0/1186937910 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6978005c30 con 0x7f69740039c0 2023-10-16T11:06:28.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.815+0000 7f69877fe640 1 -- 172.21.15.84:0/1186937910 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69780072b0 con 0x7f69740039c0 2023-10-16T11:06:28.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.815+0000 7f698e533640 1 -- 172.21.15.84:0/1186937910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69740039c0 msgr2=0x7f6974003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:28.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.815+0000 7f698e533640 1 --2- 172.21.15.84:0/1186937910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69740039c0 0x7f6974003de0 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f6978001eb0 tx=0x7f69740057f0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.816+0000 7f698e533640 1 -- 172.21.15.84:0/1186937910 shutdown_connections 2023-10-16T11:06:28.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.816+0000 7f698e533640 1 --2- 172.21.15.84:0/1186937910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69740039c0 0x7f6974003de0 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.816+0000 7f698e533640 1 -- 172.21.15.84:0/1186937910 >> 172.21.15.84:0/1186937910 conn(0x7f69880cb410 msgr2=0x7f69880cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:28.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.816+0000 7f698e533640 1 -- 172.21.15.84:0/1186937910 shutdown_connections 2023-10-16T11:06:28.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.816+0000 7f698e533640 1 -- 172.21.15.84:0/1186937910 wait complete. 2023-10-16T11:06:28.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.818+0000 7f698e533640 1 Processor -- start 2023-10-16T11:06:28.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.818+0000 7f698e533640 1 -- start start 2023-10-16T11:06:28.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.819+0000 7f698e533640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69740039c0 0x7f69881f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:28.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.819+0000 7f698e533640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69881f24a0 0x7f69880b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:28.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.819+0000 7f698e533640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69880b3d10 0x7f698806f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:28.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.819+0000 7f698e533640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69880b2980 con 0x7f69881f24a0 2023-10-16T11:06:28.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.819+0000 7f698e533640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69880b2c60 con 0x7f69880b3d10 2023-10-16T11:06:28.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.819+0000 7f698e533640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69880b2f40 con 0x7f69740039c0 2023-10-16T11:06:28.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.819+0000 7f697ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69881f24a0 0x7f69880b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:28.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.819+0000 7f697ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69881f24a0 0x7f69880b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33320/0 (socket says 172.21.15.84:33320) 2023-10-16T11:06:28.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.819+0000 7f697ffff640 1 -- 172.21.15.84:0/2909832041 learned_addr learned my addr 172.21.15.84:0/2909832041 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:28.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.819+0000 7f6987fff640 1 --2- 172.21.15.84:0/2909832041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69740039c0 0x7f69881f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:28.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.819+0000 7f698cad0640 1 --2- 172.21.15.84:0/2909832041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69880b3d10 0x7f698806f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:28.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.819+0000 7f697ffff640 1 -- 172.21.15.84:0/2909832041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69740039c0 msgr2=0x7f69881f1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:28.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.819+0000 7f697ffff640 1 --2- 172.21.15.84:0/2909832041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69740039c0 0x7f69881f1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.819+0000 7f697ffff640 1 -- 172.21.15.84:0/2909832041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69880b3d10 msgr2=0x7f698806f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:28.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.819+0000 7f697ffff640 1 --2- 172.21.15.84:0/2909832041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69880b3d10 0x7f698806f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:28.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.819+0000 7f697ffff640 1 -- 172.21.15.84:0/2909832041 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69880b3220 con 0x7f69881f24a0 2023-10-16T11:06:28.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.819+0000 7f698cad0640 1 --2- 172.21.15.84:0/2909832041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69880b3d10 0x7f698806f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:28.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.820+0000 7f697ffff640 1 --2- 172.21.15.84:0/2909832041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69881f24a0 0x7f69880b37d0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f697000e4d0 tx=0x7f697000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:28.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.820+0000 7f69857fa640 1 -- 172.21.15.84:0/2909832041 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69700053a0 con 0x7f69881f24a0 2023-10-16T11:06:28.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.820+0000 7f69857fa640 1 -- 172.21.15.84:0/2909832041 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69700035f0 con 0x7f69881f24a0 2023-10-16T11:06:28.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.820+0000 7f698e533640 1 -- 172.21.15.84:0/2909832041 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69880b25a0 con 0x7f69881f24a0 2023-10-16T11:06:28.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.820+0000 7f69857fa640 1 -- 172.21.15.84:0/2909832041 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6970003980 con 0x7f69881f24a0 2023-10-16T11:06:28.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.820+0000 7f698e533640 1 -- 172.21.15.84:0/2909832041 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698806fb70 con 0x7f69881f24a0 2023-10-16T11:06:28.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.821+0000 7f698e533640 1 -- 172.21.15.84:0/2909832041 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f698804e160 con 0x7f69881f24a0 2023-10-16T11:06:28.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.822+0000 7f69857fa640 1 -- 172.21.15.84:0/2909832041 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6970003ae0 con 0x7f69881f24a0 2023-10-16T11:06:28.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.823+0000 7f69857fa640 1 --2- 172.21.15.84:0/2909832041 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f696806af90 0x7f696806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:28.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.823+0000 7f69857fa640 1 -- 172.21.15.84:0/2909832041 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6970089580 con 0x7f69881f24a0 2023-10-16T11:06:28.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.823+0000 7f6987fff640 1 --2- 172.21.15.84:0/2909832041 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f696806af90 0x7f696806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:28.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.824+0000 7f6987fff640 1 --2- 172.21.15.84:0/2909832041 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f696806af90 0x7f696806d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6978034000 tx=0x7f6978009030 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:28.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:28.827+0000 7f69857fa640 1 -- 172.21.15.84:0/2909832041 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6970054500 con 0x7f69881f24a0 2023-10-16T11:06:29.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.090+0000 7f698e533640 1 -- 172.21.15.84:0/2909832041 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f69880b4910 con 0x7f69881f24a0 2023-10-16T11:06:29.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.094+0000 7f69857fa640 1 -- 172.21.15.84:0/2909832041 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v48) v1 ==== 190+0+147 (secure 0 0 0) 0x7f6970057b20 con 0x7f69881f24a0 2023-10-16T11:06:29.096 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2023-10-16T11:06:29.096 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ== 2023-10-16T11:06:29.096 INFO:teuthology.orchestra.run.smithi084.stdout: caps mds = "allow" 2023-10-16T11:06:29.096 INFO:teuthology.orchestra.run.smithi084.stdout: caps mon = "allow r" 2023-10-16T11:06:29.096 INFO:teuthology.orchestra.run.smithi084.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T11:06:29.097 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-16T11:06:29.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.097+0000 7f698e533640 1 -- 172.21.15.84:0/2909832041 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f696806af90 msgr2=0x7f696806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:29.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.097+0000 7f698e533640 1 --2- 172.21.15.84:0/2909832041 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f696806af90 0x7f696806d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6978034000 tx=0x7f6978009030 comp rx=0 tx=0).stop 2023-10-16T11:06:29.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.097+0000 7f698e533640 1 -- 172.21.15.84:0/2909832041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69881f24a0 msgr2=0x7f69880b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:29.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.097+0000 7f698e533640 1 --2- 172.21.15.84:0/2909832041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69881f24a0 0x7f69880b37d0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f697000e4d0 tx=0x7f697000e990 comp rx=0 tx=0).stop 2023-10-16T11:06:29.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.098+0000 7f698e533640 1 -- 172.21.15.84:0/2909832041 shutdown_connections 2023-10-16T11:06:29.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.098+0000 7f698e533640 1 --2- 172.21.15.84:0/2909832041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69880b3d10 0x7f698806f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:29.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.098+0000 7f698e533640 1 --2- 172.21.15.84:0/2909832041 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f696806af90 0x7f696806d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:29.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.098+0000 7f698e533640 1 --2- 172.21.15.84:0/2909832041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69881f24a0 0x7f69880b37d0 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:29.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.098+0000 7f698e533640 1 --2- 172.21.15.84:0/2909832041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69740039c0 0x7f69881f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:29.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.098+0000 7f698e533640 1 -- 172.21.15.84:0/2909832041 >> 172.21.15.84:0/2909832041 conn(0x7f69880cb410 msgr2=0x7f69880cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:29.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.098+0000 7f698e533640 1 -- 172.21.15.84:0/2909832041 shutdown_connections 2023-10-16T11:06:29.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.098+0000 7f698e533640 1 -- 172.21.15.84:0/2909832041 wait complete. 2023-10-16T11:06:29.119 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:06:29.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.325+0000 7fab9723f640 1 Processor -- start 2023-10-16T11:06:29.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.325+0000 7fab9723f640 1 -- start start 2023-10-16T11:06:29.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.326+0000 7fab9723f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab90159f00 0x7fab900b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:29.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.326+0000 7fab9723f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9006f7f0 con 0x7fab90159b20 2023-10-16T11:06:29.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.327+0000 7fab9723f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9006c5e0 con 0x7fab9015d880 2023-10-16T11:06:29.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.327+0000 7fab9723f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9006c8c0 con 0x7fab90159f00 2023-10-16T11:06:29.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.327+0000 7fab94fdb640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fab90159b20 0x7fab9006bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54646/0 (socket says 172.21.15.84:54646) 2023-10-16T11:06:29.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.327+0000 7fab94fdb640 1 -- 172.21.15.84:0/3366628802 learned_addr learned my addr 172.21.15.84:0/3366628802 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:29.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.327+0000 7fab8f7fe640 1 -- 172.21.15.84:0/3366628802 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895601470 0 0) 0x7fab9006f7f0 con 0x7fab90159b20 2023-10-16T11:06:29.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.327+0000 7fab8f7fe640 1 -- 172.21.15.84:0/3366628802 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab80003520 con 0x7fab90159b20 2023-10-16T11:06:29.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.327+0000 7fab8ffff640 1 --2- 172.21.15.84:0/3366628802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab90159f00 0x7fab900b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:29.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.328+0000 7fab8f7fe640 1 -- 172.21.15.84:0/3366628802 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1033678474 0 0) 0x7fab80003520 con 0x7fab90159b20 2023-10-16T11:06:29.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.328+0000 7fab8f7fe640 1 -- 172.21.15.84:0/3366628802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab90159f00 msgr2=0x7fab900b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:29.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.328+0000 7fab8f7fe640 1 --2- 172.21.15.84:0/3366628802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab90159f00 0x7fab900b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:29.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.328+0000 7fab8f7fe640 1 -- 172.21.15.84:0/3366628802 >> v1:172.21.15.157:6789/0 conn(0x7fab9015d880 legacy=0x7fab9006fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:29.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.328+0000 7fab8f7fe640 1 -- 172.21.15.84:0/3366628802 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9006cba0 con 0x7fab90159b20 2023-10-16T11:06:29.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.328+0000 7fab8f7fe640 1 -- 172.21.15.84:0/3366628802 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fab84001670 con 0x7fab90159b20 2023-10-16T11:06:29.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.328+0000 7fab8f7fe640 1 -- 172.21.15.84:0/3366628802 >> v1:172.21.15.84:6789/0 conn(0x7fab90159b20 legacy=0x7fab9006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:29.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.328+0000 7fab8f7fe640 1 --2- 172.21.15.84:0/3366628802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab90159f00 0x7fab800039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:29.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.328+0000 7fab8f7fe640 1 --2- 172.21.15.84:0/3366628802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab80003fd0 0x7fab800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:29.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.328+0000 7fab8f7fe640 1 --2- 172.21.15.84:0/3366628802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab80004d00 0x7fab80005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:29.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.328+0000 7fab8f7fe640 1 -- 172.21.15.84:0/3366628802 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9006c8c0 con 0x7fab80004d00 2023-10-16T11:06:29.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.328+0000 7fab8f7fe640 1 -- 172.21.15.84:0/3366628802 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8000bb40 con 0x7fab80003fd0 2023-10-16T11:06:29.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.328+0000 7fab8f7fe640 1 -- 172.21.15.84:0/3366628802 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8000bd00 con 0x7fab90159f00 2023-10-16T11:06:29.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.328+0000 7fab957dc640 1 --2- 172.21.15.84:0/3366628802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab80004d00 0x7fab80005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:29.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.328+0000 7fab94fdb640 1 --2- 172.21.15.84:0/3366628802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab90159f00 0x7fab800039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:29.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.328+0000 7fab8ffff640 1 --2- 172.21.15.84:0/3366628802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab80003fd0 0x7fab800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:29.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.329+0000 7fab957dc640 1 -- 172.21.15.84:0/3366628802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab90159f00 msgr2=0x7fab800039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:29.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.329+0000 7fab957dc640 1 --2- 172.21.15.84:0/3366628802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab90159f00 0x7fab800039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:29.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.329+0000 7fab957dc640 1 -- 172.21.15.84:0/3366628802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab80003fd0 msgr2=0x7fab800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:29.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.329+0000 7fab957dc640 1 --2- 172.21.15.84:0/3366628802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab80003fd0 0x7fab800047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:29.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.329+0000 7fab957dc640 1 -- 172.21.15.84:0/3366628802 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab80003520 con 0x7fab80004d00 2023-10-16T11:06:29.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.330+0000 7fab957dc640 1 --2- 172.21.15.84:0/3366628802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab80004d00 0x7fab80005180 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7fab88001e50 tx=0x7fab800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c6c98b849c1098 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:29.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.330+0000 7fab8f7fe640 1 -- 172.21.15.84:0/3366628802 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88003b70 con 0x7fab80004d00 2023-10-16T11:06:29.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.330+0000 7fab8f7fe640 1 -- 172.21.15.84:0/3366628802 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab88003cd0 con 0x7fab80004d00 2023-10-16T11:06:29.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.330+0000 7fab8f7fe640 1 -- 172.21.15.84:0/3366628802 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88005310 con 0x7fab80004d00 2023-10-16T11:06:29.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.330+0000 7fab9723f640 1 -- 172.21.15.84:0/3366628802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab80004d00 msgr2=0x7fab80005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:29.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.330+0000 7fab9723f640 1 --2- 172.21.15.84:0/3366628802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab80004d00 0x7fab80005180 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7fab88001e50 tx=0x7fab800057f0 comp rx=0 tx=0).stop 2023-10-16T11:06:29.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.330+0000 7fab957dc640 1 -- 172.21.15.84:0/3366628802 reap_dead start 2023-10-16T11:06:29.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.330+0000 7fab9723f640 1 -- 172.21.15.84:0/3366628802 shutdown_connections 2023-10-16T11:06:29.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.330+0000 7fab9723f640 1 -- 172.21.15.84:0/3366628802 >> 172.21.15.84:0/3366628802 conn(0x7fab900cb410 msgr2=0x7fab900cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:29.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.331+0000 7fab9723f640 1 -- 172.21.15.84:0/3366628802 shutdown_connections 2023-10-16T11:06:29.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.331+0000 7fab9723f640 1 -- 172.21.15.84:0/3366628802 wait complete. 2023-10-16T11:06:29.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.332+0000 7fab9723f640 1 Processor -- start 2023-10-16T11:06:29.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.332+0000 7fab9723f640 1 -- start start 2023-10-16T11:06:29.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.333+0000 7fab9723f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab80004d00 0x7fab900b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:29.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.333+0000 7fab9723f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab900c7490 0x7fab900b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:29.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.333+0000 7fab9723f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab900af6b0 0x7fab90151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:29.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.333+0000 7fab9723f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab900b2a90 con 0x7fab900c7490 2023-10-16T11:06:29.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.333+0000 7fab9723f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab900b2d70 con 0x7fab80004d00 2023-10-16T11:06:29.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.333+0000 7fab9723f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab900b3050 con 0x7fab900af6b0 2023-10-16T11:06:29.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.334+0000 7fab8ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab900c7490 0x7fab900b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:29.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.334+0000 7fab8ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab900c7490 0x7fab900b1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33332/0 (socket says 172.21.15.84:33332) 2023-10-16T11:06:29.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.334+0000 7fab8ffff640 1 -- 172.21.15.84:0/1291595778 learned_addr learned my addr 172.21.15.84:0/1291595778 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:29.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.334+0000 7fab957dc640 1 --2- 172.21.15.84:0/1291595778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab900af6b0 0x7fab90151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:29.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.334+0000 7fab94fdb640 1 --2- 172.21.15.84:0/1291595778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab80004d00 0x7fab900b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:29.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.334+0000 7fab8ffff640 1 -- 172.21.15.84:0/1291595778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab900af6b0 msgr2=0x7fab90151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:29.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.334+0000 7fab8ffff640 1 --2- 172.21.15.84:0/1291595778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab900af6b0 0x7fab90151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:29.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.334+0000 7fab8ffff640 1 -- 172.21.15.84:0/1291595778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab80004d00 msgr2=0x7fab900b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:29.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.334+0000 7fab8ffff640 1 --2- 172.21.15.84:0/1291595778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab80004d00 0x7fab900b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:29.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.334+0000 7fab8ffff640 1 -- 172.21.15.84:0/1291595778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab900b3330 con 0x7fab900c7490 2023-10-16T11:06:29.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.335+0000 7fab8ffff640 1 --2- 172.21.15.84:0/1291595778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab900c7490 0x7fab900b1c90 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00f550 tx=0x7fab7c00fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:29.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.335+0000 7fab8d7fa640 1 -- 172.21.15.84:0/1291595778 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c014870 con 0x7fab900c7490 2023-10-16T11:06:29.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.335+0000 7fab8d7fa640 1 -- 172.21.15.84:0/1291595778 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c0149d0 con 0x7fab900c7490 2023-10-16T11:06:29.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.335+0000 7fab8d7fa640 1 -- 172.21.15.84:0/1291595778 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c014d80 con 0x7fab900c7490 2023-10-16T11:06:29.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.335+0000 7fab9723f640 1 -- 172.21.15.84:0/1291595778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab900b3610 con 0x7fab900c7490 2023-10-16T11:06:29.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.335+0000 7fab9723f640 1 -- 172.21.15.84:0/1291595778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab900b21d0 con 0x7fab900c7490 2023-10-16T11:06:29.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.336+0000 7fab9723f640 1 -- 172.21.15.84:0/1291595778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab900b2450 con 0x7fab900c7490 2023-10-16T11:06:29.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.337+0000 7fab8d7fa640 1 -- 172.21.15.84:0/1291595778 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fab7c015410 con 0x7fab900c7490 2023-10-16T11:06:29.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.338+0000 7fab8d7fa640 1 --2- 172.21.15.84:0/1291595778 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fab7406afc0 0x7fab7406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:29.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.338+0000 7fab8d7fa640 1 -- 172.21.15.84:0/1291595778 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fab7c08a1d0 con 0x7fab900c7490 2023-10-16T11:06:29.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.338+0000 7fab94fdb640 1 --2- 172.21.15.84:0/1291595778 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fab7406afc0 0x7fab7406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:29.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.339+0000 7fab94fdb640 1 --2- 172.21.15.84:0/1291595778 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fab7406afc0 0x7fab7406d480 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fab84001050 tx=0x7fab84009040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:29.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.343+0000 7fab8d7fa640 1 -- 172.21.15.84:0/1291595778 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fab900b2450 con 0x7fab900c7490 2023-10-16T11:06:29.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.576+0000 7fab9723f640 1 -- 172.21.15.84:0/1291595778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab900b2450 con 0x7fab900c7490 2023-10-16T11:06:29.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.577+0000 7fab8d7fa640 1 -- 172.21.15.84:0/1291595778 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fab900b2450 con 0x7fab900c7490 2023-10-16T11:06:29.579 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:06:29.580 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6347,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1346424254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1346424254},{"type":"v1","addr":"172.21.15.157:6835","nonce":1346424254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3224476768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3224476768},{"type":"v1","addr":"172.21.15.157:6834","nonce":3224476768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":10197,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2490106026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2490106026},{"type":"v1","addr":"172.21.15.84:6835","nonce":2490106026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:06:23.034809+0000","modified":"2023-10-16T11:06:26.105994+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10206},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10206":{"gid":10206,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/1106170430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":1106170430},{"type":"v1","addr":"172.21.15.84:6837","nonce":1106170430}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-16T11:06:29.580 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 96 2023-10-16T11:06:29.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.580+0000 7fab9723f640 1 -- 172.21.15.84:0/1291595778 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fab7406afc0 msgr2=0x7fab7406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:29.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.580+0000 7fab9723f640 1 --2- 172.21.15.84:0/1291595778 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fab7406afc0 0x7fab7406d480 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fab84001050 tx=0x7fab84009040 comp rx=0 tx=0).stop 2023-10-16T11:06:29.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.580+0000 7fab9723f640 1 -- 172.21.15.84:0/1291595778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab900c7490 msgr2=0x7fab900b1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:29.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.580+0000 7fab9723f640 1 --2- 172.21.15.84:0/1291595778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab900c7490 0x7fab900b1c90 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00f550 tx=0x7fab7c00fa10 comp rx=0 tx=0).stop 2023-10-16T11:06:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.580+0000 7fab9723f640 1 -- 172.21.15.84:0/1291595778 shutdown_connections 2023-10-16T11:06:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.580+0000 7fab9723f640 1 --2- 172.21.15.84:0/1291595778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab900af6b0 0x7fab90151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.580+0000 7fab9723f640 1 --2- 172.21.15.84:0/1291595778 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fab7406afc0 0x7fab7406d480 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.580+0000 7fab9723f640 1 --2- 172.21.15.84:0/1291595778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab900c7490 0x7fab900b1c90 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.580+0000 7fab9723f640 1 --2- 172.21.15.84:0/1291595778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab80004d00 0x7fab900b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.580+0000 7fab9723f640 1 -- 172.21.15.84:0/1291595778 >> 172.21.15.84:0/1291595778 conn(0x7fab900cb410 msgr2=0x7fab900ce8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.580+0000 7fab9723f640 1 -- 172.21.15.84:0/1291595778 shutdown_connections 2023-10-16T11:06:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:29.580+0000 7fab9723f640 1 -- 172.21.15.84:0/1291595778 wait complete. 2023-10-16T11:06:30.002 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T11:06:23.034809+0000', 'modified': '2023-10-16T11:06:26.105994+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10206}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10206': {'gid': 10206, 'name': 'c', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.84:6837/1106170430', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6836', 'nonce': 1106170430}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 1106170430}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T11:06:30.002 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T11:06:30.003 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-16T11:06:30.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.225+0000 7efeb0bc0640 1 Processor -- start 2023-10-16T11:06:30.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.225+0000 7efeb0bc0640 1 -- start start 2023-10-16T11:06:30.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.227+0000 7efeb0bc0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efeac15af80 0x7efeac0cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.227+0000 7efeb0bc0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efeac065ab0 0x7efeac065f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.227+0000 7efeb0bc0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeac066450 con 0x7efeac06faf0 2023-10-16T11:06:30.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.227+0000 7efeb0bc0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeac0d0930 con 0x7efeac065ab0 2023-10-16T11:06:30.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.227+0000 7efeb0bc0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeac15eff0 con 0x7efeac15af80 2023-10-16T11:06:30.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.227+0000 7efea3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7efeac06faf0 0x7efeac15f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54658/0 (socket says 172.21.15.84:54658) 2023-10-16T11:06:30.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.227+0000 7efea3fff640 1 -- 172.21.15.84:0/3161649517 learned_addr learned my addr 172.21.15.84:0/3161649517 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:30.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.227+0000 7efeaad9d640 1 --2- 172.21.15.84:0/3161649517 >> v2:172.21.15.157:3300/0 conn(0x7efeac065ab0 0x7efeac065f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.227+0000 7efeaa59c640 1 --2- 172.21.15.84:0/3161649517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efeac15af80 0x7efeac0cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.227+0000 7efea9d9b640 1 -- 172.21.15.84:0/3161649517 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177359483 0 0) 0x7efeac066450 con 0x7efeac06faf0 2023-10-16T11:06:30.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.228+0000 7efea9d9b640 1 -- 172.21.15.84:0/3161649517 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe98003520 con 0x7efeac06faf0 2023-10-16T11:06:30.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.228+0000 7efea9d9b640 1 -- 172.21.15.84:0/3161649517 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 208075645 0 0) 0x7efe98003520 con 0x7efeac06faf0 2023-10-16T11:06:30.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.228+0000 7efea9d9b640 1 -- 172.21.15.84:0/3161649517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efeac15af80 msgr2=0x7efeac0cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:30.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.228+0000 7efea9d9b640 1 --2- 172.21.15.84:0/3161649517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efeac15af80 0x7efeac0cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.228+0000 7efea9d9b640 1 -- 172.21.15.84:0/3161649517 >> v2:172.21.15.157:3300/0 conn(0x7efeac065ab0 msgr2=0x7efeac065f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:30.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.228+0000 7efea9d9b640 1 --2- 172.21.15.84:0/3161649517 >> v2:172.21.15.157:3300/0 conn(0x7efeac065ab0 0x7efeac065f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.228+0000 7efea9d9b640 1 -- 172.21.15.84:0/3161649517 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeac15ecc0 con 0x7efeac06faf0 2023-10-16T11:06:30.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.228+0000 7efea9d9b640 1 -- 172.21.15.84:0/3161649517 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7efe94001670 con 0x7efeac06faf0 2023-10-16T11:06:30.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.228+0000 7efea9d9b640 1 -- 172.21.15.84:0/3161649517 >> v1:172.21.15.84:6789/0 conn(0x7efeac06faf0 legacy=0x7efeac15f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:30.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.228+0000 7efea9d9b640 1 --2- 172.21.15.84:0/3161649517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe98003970 0x7efe98003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.228+0000 7efea9d9b640 1 --2- 172.21.15.84:0/3161649517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efeac15af80 0x7efe980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.228+0000 7efea9d9b640 1 --2- 172.21.15.84:0/3161649517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe98004d00 0x7efe98005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.228+0000 7efea9d9b640 1 -- 172.21.15.84:0/3161649517 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac0d0930 con 0x7efe98004d00 2023-10-16T11:06:30.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.228+0000 7efea9d9b640 1 -- 172.21.15.84:0/3161649517 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac15eff0 con 0x7efe98003970 2023-10-16T11:06:30.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.228+0000 7efea9d9b640 1 -- 172.21.15.84:0/3161649517 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9800b9b0 con 0x7efeac15af80 2023-10-16T11:06:30.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.228+0000 7efeaad9d640 1 --2- 172.21.15.84:0/3161649517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe98004d00 0x7efe98005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.229+0000 7efea3fff640 1 --2- 172.21.15.84:0/3161649517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efeac15af80 0x7efe980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.229+0000 7efeaad9d640 1 -- 172.21.15.84:0/3161649517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efeac15af80 msgr2=0x7efe980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:30.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.229+0000 7efeaad9d640 1 --2- 172.21.15.84:0/3161649517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efeac15af80 0x7efe980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.229+0000 7efeaad9d640 1 -- 172.21.15.84:0/3161649517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe98003970 msgr2=0x7efe98003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:30.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.229+0000 7efeaad9d640 1 --2- 172.21.15.84:0/3161649517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe98003970 0x7efe98003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.229+0000 7efeaad9d640 1 -- 172.21.15.84:0/3161649517 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe98003520 con 0x7efe98004d00 2023-10-16T11:06:30.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.229+0000 7efea3fff640 1 --2- 172.21.15.84:0/3161649517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efeac15af80 0x7efe980047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:30.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.230+0000 7efeaad9d640 1 --2- 172.21.15.84:0/3161649517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe98004d00 0x7efe98005180 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7efea4009fc0 tx=0x7efe980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f507db31dd655a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:30.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.230+0000 7efea9d9b640 1 -- 172.21.15.84:0/3161649517 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea402cc60 con 0x7efe98004d00 2023-10-16T11:06:30.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.230+0000 7efea9d9b640 1 -- 172.21.15.84:0/3161649517 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea400f040 con 0x7efe98004d00 2023-10-16T11:06:30.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.230+0000 7efea9d9b640 1 -- 172.21.15.84:0/3161649517 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea4034440 con 0x7efe98004d00 2023-10-16T11:06:30.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.231+0000 7efeb0bc0640 1 -- 172.21.15.84:0/3161649517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe98004d00 msgr2=0x7efe98005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:30.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.231+0000 7efeb0bc0640 1 --2- 172.21.15.84:0/3161649517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe98004d00 0x7efe98005180 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7efea4009fc0 tx=0x7efe980057f0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.231+0000 7efeaad9d640 1 -- 172.21.15.84:0/3161649517 reap_dead start 2023-10-16T11:06:30.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.231+0000 7efeb0bc0640 1 -- 172.21.15.84:0/3161649517 shutdown_connections 2023-10-16T11:06:30.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.231+0000 7efeb0bc0640 1 -- 172.21.15.84:0/3161649517 >> 172.21.15.84:0/3161649517 conn(0x7efeac0d0ab0 msgr2=0x7efeac0d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:30.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.231+0000 7efeb0bc0640 1 -- 172.21.15.84:0/3161649517 shutdown_connections 2023-10-16T11:06:30.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.231+0000 7efeb0bc0640 1 -- 172.21.15.84:0/3161649517 wait complete. 2023-10-16T11:06:30.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.233+0000 7efeb0bc0640 1 Processor -- start 2023-10-16T11:06:30.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.233+0000 7efeb0bc0640 1 -- start start 2023-10-16T11:06:30.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.234+0000 7efeb0bc0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe98004d00 0x7efeac158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.234+0000 7efeb0bc0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efeac0c82e0 0x7efeac0c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.234+0000 7efeb0bc0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efeac06b870 0x7efeac06bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.234+0000 7efeb0bc0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac158550 con 0x7efe98004d00 2023-10-16T11:06:30.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.234+0000 7efeb0bc0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac0c8ca0 con 0x7efeac0c82e0 2023-10-16T11:06:30.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.234+0000 7efeb0bc0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeac0b3630 con 0x7efeac06b870 2023-10-16T11:06:30.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.234+0000 7efeaa59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe98004d00 0x7efeac158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.234+0000 7efeaa59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe98004d00 0x7efeac158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33346/0 (socket says 172.21.15.84:33346) 2023-10-16T11:06:30.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.234+0000 7efeaa59c640 1 -- 172.21.15.84:0/2967997795 learned_addr learned my addr 172.21.15.84:0/2967997795 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:30.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.234+0000 7efeaad9d640 1 --2- 172.21.15.84:0/2967997795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efeac06b870 0x7efeac06bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.234+0000 7efeaa59c640 1 -- 172.21.15.84:0/2967997795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efeac06b870 msgr2=0x7efeac06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:30.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.234+0000 7efeaa59c640 1 --2- 172.21.15.84:0/2967997795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efeac06b870 0x7efeac06bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.234+0000 7efeaa59c640 1 -- 172.21.15.84:0/2967997795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efeac0c82e0 msgr2=0x7efeac0c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:30.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.234+0000 7efea3fff640 1 --2- 172.21.15.84:0/2967997795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efeac0c82e0 0x7efeac0c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.234+0000 7efeaa59c640 1 --2- 172.21.15.84:0/2967997795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efeac0c82e0 0x7efeac0c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.234+0000 7efeaa59c640 1 -- 172.21.15.84:0/2967997795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeac0adc30 con 0x7efe98004d00 2023-10-16T11:06:30.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.235+0000 7efeaa59c640 1 --2- 172.21.15.84:0/2967997795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe98004d00 0x7efeac158010 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00e7f0 tx=0x7efe9c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:30.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.235+0000 7efea2ffd640 1 -- 172.21.15.84:0/2967997795 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c00c8c0 con 0x7efe98004d00 2023-10-16T11:06:30.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.235+0000 7efea2ffd640 1 -- 172.21.15.84:0/2967997795 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9c00ca20 con 0x7efe98004d00 2023-10-16T11:06:30.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.235+0000 7efeb0bc0640 1 -- 172.21.15.84:0/2967997795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeac06c5f0 con 0x7efe98004d00 2023-10-16T11:06:30.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.235+0000 7efea2ffd640 1 -- 172.21.15.84:0/2967997795 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c00cdd0 con 0x7efe98004d00 2023-10-16T11:06:30.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.235+0000 7efeb0bc0640 1 -- 172.21.15.84:0/2967997795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeac0ad8c0 con 0x7efe98004d00 2023-10-16T11:06:30.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.237+0000 7efea2ffd640 1 -- 172.21.15.84:0/2967997795 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efe9c013470 con 0x7efe98004d00 2023-10-16T11:06:30.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.238+0000 7efea2ffd640 1 --2- 172.21.15.84:0/2967997795 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7efe8c06af90 0x7efe8c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.238+0000 7efea2ffd640 1 -- 172.21.15.84:0/2967997795 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5737+0+0 (secure 0 0 0) 0x7efe9c089850 con 0x7efe98004d00 2023-10-16T11:06:30.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.238+0000 7efea3fff640 1 --2- 172.21.15.84:0/2967997795 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7efe8c06af90 0x7efe8c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.239+0000 7efeb382d740 1 Processor -- start 2023-10-16T11:06:30.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.239+0000 7efea3fff640 1 --2- 172.21.15.84:0/2967997795 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7efe8c06af90 0x7efe8c06d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efeac0c7fd0 tx=0x7efe94009040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:30.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.239+0000 7efeb382d740 1 -- start start 2023-10-16T11:06:30.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.240+0000 7efeb382d740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a3af80 0x557fc9a0de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.240+0000 7efeb382d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a3b360 0x557fc9a3dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.240+0000 7efeb382d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a0e3c0 0x557fc9a3e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.240+0000 7efeb382d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc992e7a0 con 0x557fc9a0e3c0 2023-10-16T11:06:30.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.240+0000 7efeb382d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc9980740 con 0x557fc9a3b360 2023-10-16T11:06:30.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.240+0000 7efeb382d740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fc997de30 con 0x557fc9a3af80 2023-10-16T11:06:30.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.240+0000 7efeaad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a0e3c0 0x557fc9a3e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.240+0000 7efeaad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a0e3c0 0x557fc9a3e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33356/0 (socket says 172.21.15.84:33356) 2023-10-16T11:06:30.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.240+0000 7efeaad9d640 1 -- 172.21.15.84:0/693738473 learned_addr learned my addr 172.21.15.84:0/693738473 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:30.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.241+0000 7efeaa59c640 1 --2- 172.21.15.84:0/693738473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a3af80 0x557fc9a0de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.241+0000 7efea3fff640 1 --2- 172.21.15.84:0/693738473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a3b360 0x557fc9a3dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.241+0000 7efeaad9d640 1 -- 172.21.15.84:0/693738473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a3af80 msgr2=0x557fc9a0de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:30.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.241+0000 7efeaad9d640 1 --2- 172.21.15.84:0/693738473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a3af80 0x557fc9a0de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.241+0000 7efeaad9d640 1 -- 172.21.15.84:0/693738473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a3b360 msgr2=0x557fc9a3dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:30.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.241+0000 7efeaad9d640 1 --2- 172.21.15.84:0/693738473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a3b360 0x557fc9a3dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.241+0000 7efeaad9d640 1 -- 172.21.15.84:0/693738473 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fc993d910 con 0x557fc9a0e3c0 2023-10-16T11:06:30.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.241+0000 7efea3fff640 1 --2- 172.21.15.84:0/693738473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a3b360 0x557fc9a3dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:30.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.241+0000 7efeaad9d640 1 --2- 172.21.15.84:0/693738473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a0e3c0 0x557fc9a3e250 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7efea402c6d0 tx=0x7efea402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490c79ab7fdf3f9a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:30.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.241+0000 7efe8bfff640 1 -- 172.21.15.84:0/693738473 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea402cc60 con 0x557fc9a0e3c0 2023-10-16T11:06:30.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.241+0000 7efe8bfff640 1 -- 172.21.15.84:0/693738473 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea402cdc0 con 0x557fc9a0e3c0 2023-10-16T11:06:30.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.241+0000 7efe8bfff640 1 -- 172.21.15.84:0/693738473 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea4003bd0 con 0x557fc9a0e3c0 2023-10-16T11:06:30.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.242+0000 7efeb382d740 1 -- 172.21.15.84:0/693738473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a0e3c0 msgr2=0x557fc9a3e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:30.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.242+0000 7efeb382d740 1 --2- 172.21.15.84:0/693738473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a0e3c0 0x557fc9a3e250 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7efea402c6d0 tx=0x7efea402e070 comp rx=0 tx=0).stop 2023-10-16T11:06:30.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.242+0000 7efeb382d740 1 -- 172.21.15.84:0/693738473 shutdown_connections 2023-10-16T11:06:30.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.242+0000 7efeb382d740 1 --2- 172.21.15.84:0/693738473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a0e3c0 0x557fc9a3e250 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.242+0000 7efeb382d740 1 --2- 172.21.15.84:0/693738473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a3b360 0x557fc9a3dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.242+0000 7efeb382d740 1 --2- 172.21.15.84:0/693738473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a3af80 0x557fc9a0de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.242+0000 7efeb382d740 1 -- 172.21.15.84:0/693738473 >> 172.21.15.84:0/693738473 conn(0x557fc9a10190 msgr2=0x557fc9a58d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:30.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.242+0000 7efeb382d740 1 -- 172.21.15.84:0/693738473 shutdown_connections 2023-10-16T11:06:30.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.242+0000 7efeb382d740 1 -- 172.21.15.84:0/693738473 wait complete. 2023-10-16T11:06:30.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.243+0000 7efeb382d740 1 -- start start 2023-10-16T11:06:30.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.243+0000 7efeb382d740 1 Processor -- start 2023-10-16T11:06:30.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.244+0000 7efeb382d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a3af80 0x557fc9a535e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.244+0000 7efeb382d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3b360 0x557fc9a6b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.244+0000 7efeb382d740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a50960 0x557fc9a5c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.244+0000 7efeb382d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc9979720 con 0x557fc9a3b360 2023-10-16T11:06:30.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.244+0000 7efeb382d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc9917970 con 0x557fc9a3af80 2023-10-16T11:06:30.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.244+0000 7efeb382d740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fc9917610 con 0x557fc9a50960 2023-10-16T11:06:30.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.244+0000 7efeb382d740 10 client.0 _ll_register_callbacks cb 0x557fc99399d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:06:30.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.244+0000 7efea3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3b360 0x557fc9a6b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.244+0000 7efea3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3b360 0x557fc9a6b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33372/0 (socket says 172.21.15.84:33372) 2023-10-16T11:06:30.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.244+0000 7efea3fff640 1 -- 172.21.15.84:0/1930683830 learned_addr learned my addr 172.21.15.84:0/1930683830 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:30.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.245+0000 7efeaad9d640 1 --2- 172.21.15.84:0/1930683830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a50960 0x557fc9a5c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.245+0000 7efea3fff640 1 -- 172.21.15.84:0/1930683830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a50960 msgr2=0x557fc9a5c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:30.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.245+0000 7efeaa59c640 1 --2- 172.21.15.84:0/1930683830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a3af80 0x557fc9a535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.245+0000 7efea3fff640 1 --2- 172.21.15.84:0/1930683830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a50960 0x557fc9a5c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.245+0000 7efea3fff640 1 -- 172.21.15.84:0/1930683830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a3af80 msgr2=0x557fc9a535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:30.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.245+0000 7efea3fff640 1 --2- 172.21.15.84:0/1930683830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a3af80 0x557fc9a535e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.245+0000 7efea3fff640 1 -- 172.21.15.84:0/1930683830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe98003520 con 0x557fc9a3b360 2023-10-16T11:06:30.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.245+0000 7efea3fff640 1 --2- 172.21.15.84:0/1930683830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3b360 0x557fc9a6b980 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7efe9400eb40 tx=0x7efe94001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375d95a0649f1dfc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:30.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.245+0000 7efe897fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T11:06:30.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.245+0000 7efe897fa640 1 -- 172.21.15.84:0/1930683830 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9400c070 con 0x557fc9a3b360 2023-10-16T11:06:30.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.245+0000 7efe897fa640 1 -- 172.21.15.84:0/1930683830 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 5737+0+0 (secure 0 0 0) 0x7efe94021aa0 con 0x557fc9a3b360 2023-10-16T11:06:30.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.245+0000 7efeb382d740 1 -- 172.21.15.84:0/1930683830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557fc99a0160 con 0x557fc9a3b360 2023-10-16T11:06:30.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.246+0000 7efe897fa640 1 -- 172.21.15.84:0/1930683830 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe94021c60 con 0x557fc9a3b360 2023-10-16T11:06:30.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.246+0000 7efeb382d740 10 client.10335 fetch_fsmap learned FSMap version 96 2023-10-16T11:06:30.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.246+0000 7efeb382d740 1 -- 172.21.15.84:0/1930683830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x557fc9977da0 con 0x557fc9a3b360 2023-10-16T11:06:30.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.246+0000 7efe897fa640 1 -- 172.21.15.84:0/1930683830 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557fc99a0160 con 0x557fc9a3b360 2023-10-16T11:06:30.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.246+0000 7efe897fa640 1 -- 172.21.15.84:0/1930683830 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 96) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efe94021420 con 0x557fc9a3b360 2023-10-16T11:06:30.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.246+0000 7efeb382d740 10 client.10335 fetch_fsmap finished waiting for FSMap version 96 2023-10-16T11:06:30.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.246+0000 7efeb382d740 10 client.10335 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T11:06:30.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.246+0000 7efeb382d740 20 client.10335 populate_metadata read hostname 'smithi084' 2023-10-16T11:06:30.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.246+0000 7efeb382d740 1 --2- 172.21.15.84:0/1930683830 >> [v2:172.21.15.84:6836/1106170430,v1:172.21.15.84:6837/1106170430] conn(0x557fc9a40800 0x557fc9a68b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.246+0000 7efeb382d740 4 client.10335 mds_command: new command op to 10206 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T11:06:30.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.246+0000 7efeb382d740 1 -- 172.21.15.84:0/1930683830 --> [v2:172.21.15.84:6836/1106170430,v1:172.21.15.84:6837/1106170430] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fc994bcc0 con 0x557fc9a40800 2023-10-16T11:06:30.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.246+0000 7efeaa59c640 1 --2- 172.21.15.84:0/1930683830 >> [v2:172.21.15.84:6836/1106170430,v1:172.21.15.84:6837/1106170430] conn(0x557fc9a40800 0x557fc9a68b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.247+0000 7efeaa59c640 1 --2- 172.21.15.84:0/1930683830 >> [v2:172.21.15.84:6836/1106170430,v1:172.21.15.84:6837/1106170430] conn(0x557fc9a40800 0x557fc9a68b70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=245d0f022195052d server_cookie=18342c4c31c04e18 in_seq=0 out_seq=0 2023-10-16T11:06:30.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.247+0000 7efe897fa640 10 client.10335 ms_handle_connect on v2:172.21.15.84:6836/1106170430 2023-10-16T11:06:30.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.248+0000 7efe897fa640 1 -- 172.21.15.84:0/1930683830 <== mds.0 v2:172.21.15.84:6836/1106170430 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557fc994bcc0 con 0x557fc9a40800 2023-10-16T11:06:30.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.248+0000 7efe897fa640 10 client.10335 handle_command_reply: tid=0 2023-10-16T11:06:30.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.248+0000 7efeb382d740 1 client.10335 shutdown 2023-10-16T11:06:30.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.248+0000 7efeb382d740 1 -- 172.21.15.84:0/1930683830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3b360 msgr2=0x557fc9a6b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:30.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.248+0000 7efeb382d740 1 --2- 172.21.15.84:0/1930683830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3b360 0x557fc9a6b980 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7efe9400eb40 tx=0x7efe94001930 comp rx=0 tx=0).stop 2023-10-16T11:06:30.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.248+0000 7efeb382d740 1 -- 172.21.15.84:0/1930683830 shutdown_connections 2023-10-16T11:06:30.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.248+0000 7efeb382d740 1 --2- 172.21.15.84:0/1930683830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a50960 0x557fc9a5c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.248+0000 7efeb382d740 1 --2- 172.21.15.84:0/1930683830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3b360 0x557fc9a6b980 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.248+0000 7efeb382d740 1 --2- 172.21.15.84:0/1930683830 >> [v2:172.21.15.84:6836/1106170430,v1:172.21.15.84:6837/1106170430] conn(0x557fc9a40800 0x557fc9a68b70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.248+0000 7efeb382d740 1 --2- 172.21.15.84:0/1930683830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a3af80 0x557fc9a535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.248+0000 7efeb382d740 1 -- 172.21.15.84:0/1930683830 >> 172.21.15.84:0/1930683830 conn(0x557fc9a0e3c0 msgr2=0x557fc9a3dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:30.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.248+0000 7efe897fa640 0 client.10335 ms_handle_reset on v2:172.21.15.84:6836/1106170430 2023-10-16T11:06:30.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.248+0000 7efeb382d740 1 -- 172.21.15.84:0/1930683830 shutdown_connections 2023-10-16T11:06:30.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.248+0000 7efeb382d740 1 -- 172.21.15.84:0/1930683830 wait complete. 2023-10-16T11:06:30.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.248+0000 7efeb382d740 20 client.10335 trim_cache size 0 max 16384 2023-10-16T11:06:30.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.264+0000 7efeb382d740 1 Processor -- start 2023-10-16T11:06:30.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.265+0000 7efeb382d740 1 -- start start 2023-10-16T11:06:30.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.265+0000 7efeb382d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3af80 0x557fc9a58fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.265+0000 7efeb382d740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a3b360 0x557fc9a12010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.265+0000 7efeb382d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a50960 0x557fc9a41d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.265+0000 7efeb382d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc96ddf20 con 0x557fc9a3af80 2023-10-16T11:06:30.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.265+0000 7efeb382d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc9a12550 con 0x557fc9a50960 2023-10-16T11:06:30.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.265+0000 7efeb382d740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fc9a114f0 con 0x557fc9a3b360 2023-10-16T11:06:30.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.266+0000 7efeaa59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3af80 0x557fc9a58fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.266+0000 7efeaa59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3af80 0x557fc9a58fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33384/0 (socket says 172.21.15.84:33384) 2023-10-16T11:06:30.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.266+0000 7efeaa59c640 1 -- 172.21.15.84:0/289166196 learned_addr learned my addr 172.21.15.84:0/289166196 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:30.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.266+0000 7efea3fff640 1 --2- 172.21.15.84:0/289166196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a3b360 0x557fc9a12010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.266+0000 7efeaa59c640 1 -- 172.21.15.84:0/289166196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a3b360 msgr2=0x557fc9a12010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:30.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.266+0000 7efeaa59c640 1 --2- 172.21.15.84:0/289166196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a3b360 0x557fc9a12010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.266+0000 7efeaa59c640 1 -- 172.21.15.84:0/289166196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a50960 msgr2=0x557fc9a41d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:30.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.266+0000 7efeaa59c640 1 --2- 172.21.15.84:0/289166196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a50960 0x557fc9a41d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.266+0000 7efeaa59c640 1 -- 172.21.15.84:0/289166196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fc9916b80 con 0x557fc9a3af80 2023-10-16T11:06:30.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.266+0000 7efeaa59c640 1 --2- 172.21.15.84:0/289166196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3af80 0x557fc9a58fb0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7efe9c087ba0 tx=0x7efe9c00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23a975ce067d06d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:30.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.267+0000 7efea0ff9640 1 -- 172.21.15.84:0/289166196 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c0606f0 con 0x557fc9a3af80 2023-10-16T11:06:30.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.267+0000 7efea0ff9640 1 -- 172.21.15.84:0/289166196 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9c06d070 con 0x557fc9a3af80 2023-10-16T11:06:30.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.267+0000 7efea0ff9640 1 -- 172.21.15.84:0/289166196 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c054b00 con 0x557fc9a3af80 2023-10-16T11:06:30.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.267+0000 7efeb382d740 1 -- 172.21.15.84:0/289166196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3af80 msgr2=0x557fc9a58fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:30.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.267+0000 7efeb382d740 1 --2- 172.21.15.84:0/289166196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3af80 0x557fc9a58fb0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7efe9c087ba0 tx=0x7efe9c00ef00 comp rx=0 tx=0).stop 2023-10-16T11:06:30.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.267+0000 7efeb382d740 1 -- 172.21.15.84:0/289166196 shutdown_connections 2023-10-16T11:06:30.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.267+0000 7efeb382d740 1 --2- 172.21.15.84:0/289166196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a50960 0x557fc9a41d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.267+0000 7efeb382d740 1 --2- 172.21.15.84:0/289166196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a3b360 0x557fc9a12010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.267+0000 7efeb382d740 1 --2- 172.21.15.84:0/289166196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3af80 0x557fc9a58fb0 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.267+0000 7efeb382d740 1 -- 172.21.15.84:0/289166196 >> 172.21.15.84:0/289166196 conn(0x557fc9a0e3c0 msgr2=0x557fc9a13d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:30.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.267+0000 7efeb382d740 1 -- 172.21.15.84:0/289166196 shutdown_connections 2023-10-16T11:06:30.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.267+0000 7efeb382d740 1 -- 172.21.15.84:0/289166196 wait complete. 2023-10-16T11:06:30.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.268+0000 7efeb382d740 1 -- start start 2023-10-16T11:06:30.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.268+0000 7efeb382d740 1 Processor -- start 2023-10-16T11:06:30.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.269+0000 7efeb382d740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a3af80 0x557fc9aaa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.269+0000 7efeb382d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3b360 0x557fc9a75420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.269+0000 7efeb382d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a50960 0x557fc9a12710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.269+0000 7efeb382d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc9a14290 con 0x557fc9a3b360 2023-10-16T11:06:30.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.269+0000 7efeb382d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc9aa88a0 con 0x557fc9a50960 2023-10-16T11:06:30.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.269+0000 7efeb382d740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fc9a71850 con 0x557fc9a3af80 2023-10-16T11:06:30.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.269+0000 7efeb382d740 10 client.0 _ll_register_callbacks cb 0x7efe94001de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:06:30.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.269+0000 7efea3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3b360 0x557fc9a75420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.269+0000 7efea3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3b360 0x557fc9a75420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33396/0 (socket says 172.21.15.84:33396) 2023-10-16T11:06:30.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.269+0000 7efea3fff640 1 -- 172.21.15.84:0/4168022691 learned_addr learned my addr 172.21.15.84:0/4168022691 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:30.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.270+0000 7efea3fff640 1 -- 172.21.15.84:0/4168022691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a3af80 msgr2=0x557fc9aaa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:30.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.270+0000 7efea3fff640 1 --2- 172.21.15.84:0/4168022691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a3af80 0x557fc9aaa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.270+0000 7efea3fff640 1 -- 172.21.15.84:0/4168022691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a50960 msgr2=0x557fc9a12710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:30.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.270+0000 7efea3fff640 1 --2- 172.21.15.84:0/4168022691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a50960 0x557fc9a12710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.270+0000 7efea3fff640 1 -- 172.21.15.84:0/4168022691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeac0ad8c0 con 0x557fc9a3b360 2023-10-16T11:06:30.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.270+0000 7efea3fff640 1 --2- 172.21.15.84:0/4168022691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3b360 0x557fc9a75420 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7efe940078c0 tx=0x7efe94012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea70dce505accbcf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:30.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.270+0000 7efe897fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T11:06:30.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.270+0000 7efe897fa640 1 -- 172.21.15.84:0/4168022691 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94005270 con 0x557fc9a3b360 2023-10-16T11:06:30.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.270+0000 7efe897fa640 1 -- 172.21.15.84:0/4168022691 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 5737+0+0 (secure 0 0 0) 0x7efe940053d0 con 0x557fc9a3b360 2023-10-16T11:06:30.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.270+0000 7efeb382d740 1 -- 172.21.15.84:0/4168022691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efea402cdc0 con 0x557fc9a3b360 2023-10-16T11:06:30.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.271+0000 7efe897fa640 1 -- 172.21.15.84:0/4168022691 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe94005590 con 0x557fc9a3b360 2023-10-16T11:06:30.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.271+0000 7efeb382d740 10 client.10341 fetch_fsmap learned FSMap version 96 2023-10-16T11:06:30.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.271+0000 7efeb382d740 1 -- 172.21.15.84:0/4168022691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x557fc993c170 con 0x557fc9a3b360 2023-10-16T11:06:30.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.271+0000 7efe897fa640 1 -- 172.21.15.84:0/4168022691 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea402cdc0 con 0x557fc9a3b360 2023-10-16T11:06:30.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.271+0000 7efe897fa640 1 -- 172.21.15.84:0/4168022691 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 96) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efe940047e0 con 0x557fc9a3b360 2023-10-16T11:06:30.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.271+0000 7efeb382d740 10 client.10341 fetch_fsmap finished waiting for FSMap version 96 2023-10-16T11:06:30.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.271+0000 7efeb382d740 10 client.10341 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T11:06:30.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.271+0000 7efeb382d740 20 client.10341 populate_metadata read hostname 'smithi084' 2023-10-16T11:06:30.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.271+0000 7efeb382d740 1 --2- 172.21.15.84:0/4168022691 >> [v2:172.21.15.84:6836/1106170430,v1:172.21.15.84:6837/1106170430] conn(0x557fc9aa7ab0 0x557fc9aa7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.271+0000 7efeb382d740 4 client.10341 mds_command: new command op to 10206 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T11:06:30.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.271+0000 7efeb382d740 1 -- 172.21.15.84:0/4168022691 --> [v2:172.21.15.84:6836/1106170430,v1:172.21.15.84:6837/1106170430] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557fc9972700 con 0x557fc9aa7ab0 2023-10-16T11:06:30.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.271+0000 7efeaa59c640 1 --2- 172.21.15.84:0/4168022691 >> [v2:172.21.15.84:6836/1106170430,v1:172.21.15.84:6837/1106170430] conn(0x557fc9aa7ab0 0x557fc9aa7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.272+0000 7efe897fa640 10 client.10341 ms_handle_connect on v2:172.21.15.84:6836/1106170430 2023-10-16T11:06:30.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.272+0000 7efeaa59c640 1 --2- 172.21.15.84:0/4168022691 >> [v2:172.21.15.84:6836/1106170430,v1:172.21.15.84:6837/1106170430] conn(0x557fc9aa7ab0 0x557fc9aa7e90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35ebcdef408fa92 server_cookie=493c9615ac51b46f in_seq=0 out_seq=0 2023-10-16T11:06:30.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.272+0000 7efe897fa640 1 -- 172.21.15.84:0/4168022691 <== mds.0 v2:172.21.15.84:6836/1106170430 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x557fc9972700 con 0x557fc9aa7ab0 2023-10-16T11:06:30.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.272+0000 7efe897fa640 10 client.10341 handle_command_reply: tid=0 2023-10-16T11:06:30.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.272+0000 7efeb382d740 1 client.10341 shutdown 2023-10-16T11:06:30.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.272+0000 7efeb382d740 1 -- 172.21.15.84:0/4168022691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3b360 msgr2=0x557fc9a75420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:30.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.272+0000 7efeb382d740 1 --2- 172.21.15.84:0/4168022691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3b360 0x557fc9a75420 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7efe940078c0 tx=0x7efe94012000 comp rx=0 tx=0).stop 2023-10-16T11:06:30.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.272+0000 7efeb382d740 1 -- 172.21.15.84:0/4168022691 shutdown_connections 2023-10-16T11:06:30.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.272+0000 7efeb382d740 1 --2- 172.21.15.84:0/4168022691 >> [v2:172.21.15.84:6836/1106170430,v1:172.21.15.84:6837/1106170430] conn(0x557fc9aa7ab0 0x557fc9aa7e90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.272+0000 7efeb382d740 1 --2- 172.21.15.84:0/4168022691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557fc9a50960 0x557fc9a12710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.272+0000 7efe897fa640 0 client.10341 ms_handle_reset on v2:172.21.15.84:6836/1106170430 2023-10-16T11:06:30.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.272+0000 7efeb382d740 1 --2- 172.21.15.84:0/4168022691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fc9a3b360 0x557fc9a75420 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.272+0000 7efeb382d740 1 --2- 172.21.15.84:0/4168022691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x557fc9a3af80 0x557fc9aaa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.272+0000 7efeb382d740 1 -- 172.21.15.84:0/4168022691 >> 172.21.15.84:0/4168022691 conn(0x557fc9a0e3c0 msgr2=0x557fc9a3f820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:30.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.273+0000 7efeb382d740 1 -- 172.21.15.84:0/4168022691 shutdown_connections 2023-10-16T11:06:30.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.273+0000 7efeb382d740 1 -- 172.21.15.84:0/4168022691 wait complete. 2023-10-16T11:06:30.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.273+0000 7efeb382d740 20 client.10341 trim_cache size 0 max 16384 2023-10-16T11:06:30.292 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:06:30.292 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "63fb4d68-f881-4694-b880-d10369105c0e", 2023-10-16T11:06:30.293 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-16T11:06:30.293 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 10206, 2023-10-16T11:06:30.293 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-16T11:06:30.293 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-16T11:06:30.294 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-16T11:06:30.294 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 6.6767581979999999, 2023-10-16T11:06:30.294 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 96, 2023-10-16T11:06:30.294 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 184, 2023-10-16T11:06:30.294 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 184, 2023-10-16T11:06:30.295 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 12.109821278 2023-10-16T11:06:30.295 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:06:30.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.273+0000 7efeb0bc0640 1 -- 172.21.15.84:0/2967997795 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7efe8c06af90 msgr2=0x7efe8c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:30.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.273+0000 7efeb0bc0640 1 --2- 172.21.15.84:0/2967997795 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7efe8c06af90 0x7efe8c06d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efeac0c7fd0 tx=0x7efe94009040 comp rx=0 tx=0).stop 2023-10-16T11:06:30.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.273+0000 7efeb0bc0640 1 -- 172.21.15.84:0/2967997795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe98004d00 msgr2=0x7efeac158010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:30.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.273+0000 7efeb0bc0640 1 --2- 172.21.15.84:0/2967997795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe98004d00 0x7efeac158010 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00e7f0 tx=0x7efe9c00ecb0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.273+0000 7efeb0bc0640 1 -- 172.21.15.84:0/2967997795 shutdown_connections 2023-10-16T11:06:30.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.273+0000 7efeb0bc0640 1 --2- 172.21.15.84:0/2967997795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efeac06b870 0x7efeac06bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.273+0000 7efeb0bc0640 1 --2- 172.21.15.84:0/2967997795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efeac0c82e0 0x7efeac0c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.273+0000 7efeb0bc0640 1 --2- 172.21.15.84:0/2967997795 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7efe8c06af90 0x7efe8c06d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.273+0000 7efeb0bc0640 1 --2- 172.21.15.84:0/2967997795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe98004d00 0x7efeac158010 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.273+0000 7efeb0bc0640 1 -- 172.21.15.84:0/2967997795 >> 172.21.15.84:0/2967997795 conn(0x7efeac0d0ab0 msgr2=0x7efeac159320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:30.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.273+0000 7efeb0bc0640 1 -- 172.21.15.84:0/2967997795 shutdown_connections 2023-10-16T11:06:30.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.274+0000 7efeb0bc0640 1 -- 172.21.15.84:0/2967997795 wait complete. 2023-10-16T11:06:30.299 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T11:06:30.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.495+0000 7f161b251640 1 Processor -- start 2023-10-16T11:06:30.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.495+0000 7f161b251640 1 -- start start 2023-10-16T11:06:30.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.497+0000 7f161b251640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f161415aaa0 0x7f161415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.497+0000 7f161b251640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f161415ae80 0x7f16140cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.497+0000 7f161b251640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f16140ccaa0 0x7f1614070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.497+0000 7f161b251640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161406b810 con 0x7f16140ccaa0 2023-10-16T11:06:30.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.497+0000 7f161b251640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161415ed60 con 0x7f161415ae80 2023-10-16T11:06:30.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.497+0000 7f161b251640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16140707e0 con 0x7f161415aaa0 2023-10-16T11:06:30.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.497+0000 7f16197ee640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f16140ccaa0 0x7f1614070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.497+0000 7f16197ee640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f16140ccaa0 0x7f1614070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33404/0 (socket says 172.21.15.84:33404) 2023-10-16T11:06:30.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.497+0000 7f16197ee640 1 -- 172.21.15.84:0/3133755459 learned_addr learned my addr 172.21.15.84:0/3133755459 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:30.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.497+0000 7f1613fff640 1 --2- 172.21.15.84:0/3133755459 >> v2:172.21.15.157:3300/0 conn(0x7f161415ae80 0x7f16140cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.497+0000 7f1618fed640 1 --2- 172.21.15.84:0/3133755459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f161415aaa0 0x7f161415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.497+0000 7f16197ee640 1 -- 172.21.15.84:0/3133755459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f161415aaa0 msgr2=0x7f161415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:30.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.497+0000 7f16197ee640 1 --2- 172.21.15.84:0/3133755459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f161415aaa0 0x7f161415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.497+0000 7f16197ee640 1 -- 172.21.15.84:0/3133755459 >> v2:172.21.15.157:3300/0 conn(0x7f161415ae80 msgr2=0x7f16140cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:30.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.497+0000 7f16197ee640 1 --2- 172.21.15.84:0/3133755459 >> v2:172.21.15.157:3300/0 conn(0x7f161415ae80 0x7f16140cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.497+0000 7f16197ee640 1 -- 172.21.15.84:0/3133755459 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161415ef90 con 0x7f16140ccaa0 2023-10-16T11:06:30.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.497+0000 7f1613fff640 1 --2- 172.21.15.84:0/3133755459 >> v2:172.21.15.157:3300/0 conn(0x7f161415ae80 0x7f16140cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:30.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.497+0000 7f1618fed640 1 --2- 172.21.15.84:0/3133755459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f161415aaa0 0x7f161415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:30.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.498+0000 7f16197ee640 1 --2- 172.21.15.84:0/3133755459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16140ccaa0 0x7f1614070f50 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f16140cb270 tx=0x7f160c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bcf1d9386037280 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:30.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.499+0000 7f16137fe640 1 -- 172.21.15.84:0/3133755459 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c02ca70 con 0x7f16140ccaa0 2023-10-16T11:06:30.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.499+0000 7f16137fe640 1 -- 172.21.15.84:0/3133755459 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160c02cbd0 con 0x7f16140ccaa0 2023-10-16T11:06:30.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.499+0000 7f16137fe640 1 -- 172.21.15.84:0/3133755459 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c0343f0 con 0x7f16140ccaa0 2023-10-16T11:06:30.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.499+0000 7f161b251640 1 -- 172.21.15.84:0/3133755459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16140ccaa0 msgr2=0x7f1614070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:30.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.499+0000 7f161b251640 1 --2- 172.21.15.84:0/3133755459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16140ccaa0 0x7f1614070f50 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f16140cb270 tx=0x7f160c02bed0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.499+0000 7f161b251640 1 -- 172.21.15.84:0/3133755459 shutdown_connections 2023-10-16T11:06:30.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.499+0000 7f161b251640 1 --2- 172.21.15.84:0/3133755459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16140ccaa0 0x7f1614070f50 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.499+0000 7f161b251640 1 --2- 172.21.15.84:0/3133755459 >> v2:172.21.15.157:3300/0 conn(0x7f161415ae80 0x7f16140cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.500+0000 7f161b251640 1 --2- 172.21.15.84:0/3133755459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f161415aaa0 0x7f161415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.500+0000 7f161b251640 1 -- 172.21.15.84:0/3133755459 >> 172.21.15.84:0/3133755459 conn(0x7f16140cb410 msgr2=0x7f16140cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:30.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.500+0000 7f161b251640 1 -- 172.21.15.84:0/3133755459 shutdown_connections 2023-10-16T11:06:30.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.500+0000 7f161b251640 1 -- 172.21.15.84:0/3133755459 wait complete. 2023-10-16T11:06:30.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.501+0000 7f161b251640 1 Processor -- start 2023-10-16T11:06:30.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.502+0000 7f161b251640 1 -- start start 2023-10-16T11:06:30.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.503+0000 7f161b251640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16140ccaa0 0x7f161415bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.503+0000 7f161b251640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f161415aaa0 0x7f16140c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.503+0000 7f161b251640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f161415ae80 0x7f16141f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.503+0000 7f161b251640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16140c5360 con 0x7f161415aaa0 2023-10-16T11:06:30.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.503+0000 7f161b251640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16141f3630 con 0x7f161415ae80 2023-10-16T11:06:30.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.503+0000 7f161b251640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161415c3a0 con 0x7f16140ccaa0 2023-10-16T11:06:30.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.503+0000 7f1613fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f161415aaa0 0x7f16140c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.503+0000 7f1613fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f161415aaa0 0x7f16140c4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33406/0 (socket says 172.21.15.84:33406) 2023-10-16T11:06:30.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.503+0000 7f1613fff640 1 -- 172.21.15.84:0/2600335067 learned_addr learned my addr 172.21.15.84:0/2600335067 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:30.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.503+0000 7f1618fed640 1 --2- 172.21.15.84:0/2600335067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16140ccaa0 0x7f161415bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.503+0000 7f16197ee640 1 --2- 172.21.15.84:0/2600335067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f161415ae80 0x7f16141f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.503+0000 7f1613fff640 1 -- 172.21.15.84:0/2600335067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16140ccaa0 msgr2=0x7f161415bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:30.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.503+0000 7f1613fff640 1 --2- 172.21.15.84:0/2600335067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16140ccaa0 0x7f161415bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.503+0000 7f1613fff640 1 -- 172.21.15.84:0/2600335067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f161415ae80 msgr2=0x7f16141f30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:30.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.503+0000 7f1613fff640 1 --2- 172.21.15.84:0/2600335067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f161415ae80 0x7f16141f30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.503+0000 7f1613fff640 1 -- 172.21.15.84:0/2600335067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160c009b30 con 0x7f161415aaa0 2023-10-16T11:06:30.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.504+0000 7f1613fff640 1 --2- 172.21.15.84:0/2600335067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f161415aaa0 0x7f16140c4e20 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f160000c7f0 tx=0x7f160000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:30.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.504+0000 7f16117fa640 1 -- 172.21.15.84:0/2600335067 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1600007a30 con 0x7f161415aaa0 2023-10-16T11:06:30.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.504+0000 7f16117fa640 1 -- 172.21.15.84:0/2600335067 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1600007b90 con 0x7f161415aaa0 2023-10-16T11:06:30.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.504+0000 7f16117fa640 1 -- 172.21.15.84:0/2600335067 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160000ce40 con 0x7f161415aaa0 2023-10-16T11:06:30.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.504+0000 7f161b251640 1 -- 172.21.15.84:0/2600335067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161415c680 con 0x7f161415aaa0 2023-10-16T11:06:30.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.504+0000 7f161b251640 1 -- 172.21.15.84:0/2600335067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16140c2840 con 0x7f161415aaa0 2023-10-16T11:06:30.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.505+0000 7f161b251640 1 -- 172.21.15.84:0/2600335067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16140c2a60 con 0x7f161415aaa0 2023-10-16T11:06:30.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.506+0000 7f16117fa640 1 -- 172.21.15.84:0/2600335067 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f16000113f0 con 0x7f161415aaa0 2023-10-16T11:06:30.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.507+0000 7f16117fa640 1 --2- 172.21.15.84:0/2600335067 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f15f806af90 0x7f15f806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:30.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.507+0000 7f16117fa640 1 -- 172.21.15.84:0/2600335067 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1600089aa0 con 0x7f161415aaa0 2023-10-16T11:06:30.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.507+0000 7f1618fed640 1 --2- 172.21.15.84:0/2600335067 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f15f806af90 0x7f15f806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:30.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.508+0000 7f1618fed640 1 --2- 172.21.15.84:0/2600335067 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f15f806af90 0x7f15f806d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f160800a980 tx=0x7f1608008040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:30.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.511+0000 7f16117fa640 1 -- 172.21.15.84:0/2600335067 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1600054900 con 0x7f161415aaa0 2023-10-16T11:06:30.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.745+0000 7f161b251640 1 -- 172.21.15.84:0/2600335067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f161404e160 con 0x7f161415aaa0 2023-10-16T11:06:30.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.746+0000 7f16117fa640 1 -- 172.21.15.84:0/2600335067 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v96) v1 ==== 53+0+73 (secure 0 0 0) 0x7f160001f070 con 0x7f161415aaa0 2023-10-16T11:06:30.748 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T11:06:30.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.749+0000 7f161b251640 1 -- 172.21.15.84:0/2600335067 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f15f806af90 msgr2=0x7f15f806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:30.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.749+0000 7f161b251640 1 --2- 172.21.15.84:0/2600335067 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f15f806af90 0x7f15f806d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f160800a980 tx=0x7f1608008040 comp rx=0 tx=0).stop 2023-10-16T11:06:30.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.749+0000 7f161b251640 1 -- 172.21.15.84:0/2600335067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f161415aaa0 msgr2=0x7f16140c4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:30.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.749+0000 7f161b251640 1 --2- 172.21.15.84:0/2600335067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f161415aaa0 0x7f16140c4e20 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f160000c7f0 tx=0x7f160000cbf0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.749+0000 7f161b251640 1 -- 172.21.15.84:0/2600335067 shutdown_connections 2023-10-16T11:06:30.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.749+0000 7f161b251640 1 --2- 172.21.15.84:0/2600335067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f161415ae80 0x7f16141f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.749+0000 7f161b251640 1 --2- 172.21.15.84:0/2600335067 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f15f806af90 0x7f15f806d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.749+0000 7f161b251640 1 --2- 172.21.15.84:0/2600335067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f161415aaa0 0x7f16140c4e20 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.749+0000 7f161b251640 1 --2- 172.21.15.84:0/2600335067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16140ccaa0 0x7f161415bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:30.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.749+0000 7f161b251640 1 -- 172.21.15.84:0/2600335067 >> 172.21.15.84:0/2600335067 conn(0x7f16140cb410 msgr2=0x7f1614070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:30.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.749+0000 7f161b251640 1 -- 172.21.15.84:0/2600335067 shutdown_connections 2023-10-16T11:06:30.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:30.749+0000 7f161b251640 1 -- 172.21.15.84:0/2600335067 wait complete. 2023-10-16T11:06:30.768 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T11:06:30.768 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T11:06:30.769 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-16T11:06:30.769 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T11:06:30.770 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T11:06:30.770 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T11:06:30.770 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T11:06:30.771 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T11:06:30.771 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:06:30.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:06:30.772 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-16T11:06:30.825 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T11:06:30.826 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T11:06:30.826 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T11:06:30.826 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:06:30.826 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T11:06:30.827 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:06:30.827 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:06:30.827 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:06:30.827 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-16T11:06:30.827 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:06:30.828 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:06:30.828 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-16T11:06:30.828 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:06:30.828 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:06:30.828 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T11:06:30.829 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:06:30.829 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T11:06:30.829 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 37760sec preferred_lft 37760sec 2023-10-16T11:06:30.829 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-16T11:06:30.829 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:06:30.830 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T11:06:30.830 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:06:30.830 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T11:06:30.830 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:06:30.830 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::20a8:eaff:fe9d:5cea/64 scope link 2023-10-16T11:06:30.831 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:06:30.831 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:06:30.831 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T11:06:30.831 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1854:13ff:fe47:be92/64 scope link 2023-10-16T11:06:30.831 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:06:30.832 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:06:30.832 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether ee:d5:41:a3:a5:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T11:06:30.832 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ecd5:41ff:fea3:a570/64 scope link 2023-10-16T11:06:30.832 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:06:30.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:06:30.833 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T11:06:30.920 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-16T11:06:30.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:06:30.924 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-16T11:06:30.975 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-16T11:06:30.975 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T11:06:30.975 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T11:06:30.975 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T11:06:30.976 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T11:06:30.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:06:30.976 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T11:06:30.977 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T11:06:30.977 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T11:06:30.977 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T11:06:30.978 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T11:06:30.978 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T11:06:31.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:06:31.165 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T11:06:31.217 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T11:06:31.217 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T11:06:31.218 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:06:31.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:06:31.219 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-16T11:06:31.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:06:31.319 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T11:06:31.375 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:06:31.375 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:06:31.376 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:06:31.432 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-16T11:06:31.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:06:31.432 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:06:31.570 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:06:31.571 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T11:06:31.604 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:06:31.605 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T11:06:31.605 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T11:06:31.605 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with new device syntax: 0@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T11:06:31.605 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T11:06:31.607 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-16T11:06:31.608 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2023-10-16T11:06:31.608 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T11:06:31.608 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T11:06:31.608 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T11:06:31.612 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T11:06:31.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:06:31.612 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:06:31.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:06:31.689 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T11:06:31.744 INFO:teuthology.orchestra.run.smithi084.stdout:client10350 2023-10-16T11:06:31.745 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:06:31.745 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:06:31.793 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T11:06:32.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.053+0000 7fecda5e2640 1 Processor -- start 2023-10-16T11:06:32.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.053+0000 7fecda5e2640 1 -- start start 2023-10-16T11:06:32.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.055+0000 7fecda5e2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fecd40ad970 0x7fecd40cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:32.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.055+0000 7fecda5e2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fecd40ccfb0 0x7fecd40cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:32.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.055+0000 7fecda5e2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd415b9d0 con 0x7fecd406c040 2023-10-16T11:06:32.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.055+0000 7fecda5e2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd415bcb0 con 0x7fecd40ccfb0 2023-10-16T11:06:32.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.055+0000 7fecda5e2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd415bf90 con 0x7fecd40ad970 2023-10-16T11:06:32.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.055+0000 7fecd9de1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fecd40ccfb0 0x7fecd40cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:32.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.055+0000 7fecd95e0640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fecd406c040 0x7fecd406c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54662/0 (socket says 172.21.15.84:54662) 2023-10-16T11:06:32.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.055+0000 7fecd95e0640 1 -- 172.21.15.84:0/1062510174 learned_addr learned my addr 172.21.15.84:0/1062510174 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:32.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.055+0000 7fecd8ddf640 1 --2- 172.21.15.84:0/1062510174 >> v2:172.21.15.157:3300/0 conn(0x7fecd40ad970 0x7fecd40cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:32.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.056+0000 7fecd9de1640 1 -- 172.21.15.84:0/1062510174 >> v2:172.21.15.157:3300/0 conn(0x7fecd40ad970 msgr2=0x7fecd40cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:32.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.056+0000 7fecd9de1640 1 --2- 172.21.15.84:0/1062510174 >> v2:172.21.15.157:3300/0 conn(0x7fecd40ad970 0x7fecd40cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:32.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.056+0000 7fecd9de1640 1 -- 172.21.15.84:0/1062510174 >> v1:172.21.15.84:6789/0 conn(0x7fecd406c040 legacy=0x7fecd406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:32.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.056+0000 7fecd9de1640 1 -- 172.21.15.84:0/1062510174 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd40cc270 con 0x7fecd40ccfb0 2023-10-16T11:06:32.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.056+0000 7feccbfff640 1 -- 172.21.15.84:0/1062510174 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449744827 0 0) 0x7fecd415b9d0 con 0x7fecd406c040 2023-10-16T11:06:32.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.057+0000 7fecd9de1640 1 --2- 172.21.15.84:0/1062510174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd40ccfb0 0x7fecd40cd460 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7feccc009d90 tx=0x7feccc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ededca79756324b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:32.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.057+0000 7feccbfff640 1 -- 172.21.15.84:0/1062510174 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc02cc60 con 0x7fecd40ccfb0 2023-10-16T11:06:32.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.057+0000 7feccbfff640 1 -- 172.21.15.84:0/1062510174 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feccc02cdc0 con 0x7fecd40ccfb0 2023-10-16T11:06:32.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.057+0000 7feccbfff640 1 -- 172.21.15.84:0/1062510174 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc034660 con 0x7fecd40ccfb0 2023-10-16T11:06:32.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.058+0000 7fecda5e2640 1 -- 172.21.15.84:0/1062510174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd40ccfb0 msgr2=0x7fecd40cd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:32.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.058+0000 7fecda5e2640 1 --2- 172.21.15.84:0/1062510174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd40ccfb0 0x7fecd40cd460 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7feccc009d90 tx=0x7feccc02beb0 comp rx=0 tx=0).stop 2023-10-16T11:06:32.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.058+0000 7fecda5e2640 1 -- 172.21.15.84:0/1062510174 shutdown_connections 2023-10-16T11:06:32.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.058+0000 7fecda5e2640 1 --2- 172.21.15.84:0/1062510174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd40ccfb0 0x7fecd40cd460 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:32.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.058+0000 7fecda5e2640 1 --2- 172.21.15.84:0/1062510174 >> v2:172.21.15.157:3300/0 conn(0x7fecd40ad970 0x7fecd40cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:32.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.058+0000 7fecda5e2640 1 -- 172.21.15.84:0/1062510174 >> 172.21.15.84:0/1062510174 conn(0x7fecd40ce6a0 msgr2=0x7fecd40ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:32.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.058+0000 7fecda5e2640 1 -- 172.21.15.84:0/1062510174 shutdown_connections 2023-10-16T11:06:32.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.058+0000 7fecda5e2640 1 -- 172.21.15.84:0/1062510174 wait complete. 2023-10-16T11:06:32.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.060+0000 7fecda5e2640 1 Processor -- start 2023-10-16T11:06:32.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.060+0000 7fecda5e2640 1 -- start start 2023-10-16T11:06:32.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.061+0000 7fecda5e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd406c040 0x7fecd40ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:32.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.061+0000 7fecda5e2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecd40ad970 0x7fecd40a9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:32.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.061+0000 7fecda5e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd40ccfb0 0x7fecd40c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:32.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.061+0000 7fecda5e2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40aae80 con 0x7fecd406c040 2023-10-16T11:06:32.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.061+0000 7fecda5e2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40aab70 con 0x7fecd40ccfb0 2023-10-16T11:06:32.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.061+0000 7fecda5e2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40b42b0 con 0x7fecd40ad970 2023-10-16T11:06:32.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.061+0000 7fecd95e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd406c040 0x7fecd40ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:32.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.061+0000 7fecd95e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd406c040 0x7fecd40ace40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33430/0 (socket says 172.21.15.84:33430) 2023-10-16T11:06:32.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.061+0000 7fecd95e0640 1 -- 172.21.15.84:0/1808881529 learned_addr learned my addr 172.21.15.84:0/1808881529 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:32.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.061+0000 7fecd95e0640 1 -- 172.21.15.84:0/1808881529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecd40ad970 msgr2=0x7fecd40a9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:32.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.061+0000 7fecd95e0640 1 --2- 172.21.15.84:0/1808881529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecd40ad970 0x7fecd40a9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:32.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.061+0000 7fecd95e0640 1 -- 172.21.15.84:0/1808881529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd40ccfb0 msgr2=0x7fecd40c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:32.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.061+0000 7fecd95e0640 1 --2- 172.21.15.84:0/1808881529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd40ccfb0 0x7fecd40c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:32.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.061+0000 7fecd95e0640 1 -- 172.21.15.84:0/1808881529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feccc009a70 con 0x7fecd406c040 2023-10-16T11:06:32.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.062+0000 7fecd95e0640 1 --2- 172.21.15.84:0/1808881529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd406c040 0x7fecd40ace40 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fecd000c4d0 tx=0x7fecd000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:32.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.062+0000 7fecc9ffb640 1 -- 172.21.15.84:0/1808881529 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd00074e0 con 0x7fecd406c040 2023-10-16T11:06:32.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.062+0000 7fecc9ffb640 1 -- 172.21.15.84:0/1808881529 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd0007660 con 0x7fecd406c040 2023-10-16T11:06:32.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.062+0000 7fecc9ffb640 1 -- 172.21.15.84:0/1808881529 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd0004a20 con 0x7fecd406c040 2023-10-16T11:06:32.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.062+0000 7fecda5e2640 1 -- 172.21.15.84:0/1808881529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd40b3a10 con 0x7fecd406c040 2023-10-16T11:06:32.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.062+0000 7fecda5e2640 1 -- 172.21.15.84:0/1808881529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd40b3fd0 con 0x7fecd406c040 2023-10-16T11:06:32.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.063+0000 7fecda5e2640 1 -- 172.21.15.84:0/1808881529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd406b330 con 0x7fecd406c040 2023-10-16T11:06:32.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.065+0000 7fecc9ffb640 1 -- 172.21.15.84:0/1808881529 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fecd0004b80 con 0x7fecd406c040 2023-10-16T11:06:32.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.066+0000 7fecc9ffb640 1 --2- 172.21.15.84:0/1808881529 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fecb806af90 0x7fecb806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:32.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.066+0000 7fecc9ffb640 1 -- 172.21.15.84:0/1808881529 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fecd0088910 con 0x7fecd406c040 2023-10-16T11:06:32.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.066+0000 7fecd8ddf640 1 --2- 172.21.15.84:0/1808881529 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fecb806af90 0x7fecb806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:32.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.067+0000 7fecd8ddf640 1 --2- 172.21.15.84:0/1808881529 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fecb806af90 0x7fecb806d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fecd40c7a30 tx=0x7fecc4008040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:32.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.070+0000 7fecc9ffb640 1 -- 172.21.15.84:0/1808881529 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fecd0053880 con 0x7fecd406c040 2023-10-16T11:06:32.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.287+0000 7fecda5e2640 1 -- 172.21.15.84:0/1808881529 --> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fecd40aafc0 con 0x7fecb806af90 2023-10-16T11:06:32.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.289+0000 7fecc9ffb640 1 -- 172.21.15.84:0/1808881529 <== mgr.9969 v2:172.21.15.84:6832/1441051579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fecd40aafc0 con 0x7fecb806af90 2023-10-16T11:06:32.290 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T11:06:32.290 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:06:32.291 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephfs" 2023-10-16T11:06:32.291 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:06:32.291 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T11:06:32.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.292+0000 7fecda5e2640 1 -- 172.21.15.84:0/1808881529 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fecb806af90 msgr2=0x7fecb806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:32.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.292+0000 7fecda5e2640 1 --2- 172.21.15.84:0/1808881529 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fecb806af90 0x7fecb806d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fecd40c7a30 tx=0x7fecc4008040 comp rx=0 tx=0).stop 2023-10-16T11:06:32.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.292+0000 7fecda5e2640 1 -- 172.21.15.84:0/1808881529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd406c040 msgr2=0x7fecd40ace40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:32.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.292+0000 7fecda5e2640 1 --2- 172.21.15.84:0/1808881529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd406c040 0x7fecd40ace40 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fecd000c4d0 tx=0x7fecd000c990 comp rx=0 tx=0).stop 2023-10-16T11:06:32.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.292+0000 7fecda5e2640 1 -- 172.21.15.84:0/1808881529 shutdown_connections 2023-10-16T11:06:32.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.292+0000 7fecda5e2640 1 --2- 172.21.15.84:0/1808881529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd40ccfb0 0x7fecd40c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:32.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.292+0000 7fecda5e2640 1 --2- 172.21.15.84:0/1808881529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecd40ad970 0x7fecd40a9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:32.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.292+0000 7fecda5e2640 1 --2- 172.21.15.84:0/1808881529 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fecb806af90 0x7fecb806d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:32.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.292+0000 7fecda5e2640 1 --2- 172.21.15.84:0/1808881529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd406c040 0x7fecd40ace40 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:32.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.292+0000 7fecda5e2640 1 -- 172.21.15.84:0/1808881529 >> 172.21.15.84:0/1808881529 conn(0x7fecd40ce6a0 msgr2=0x7fecd40c5770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:32.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.292+0000 7fecda5e2640 1 -- 172.21.15.84:0/1808881529 shutdown_connections 2023-10-16T11:06:32.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.292+0000 7fecda5e2640 1 -- 172.21.15.84:0/1808881529 wait complete. 2023-10-16T11:06:32.311 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T11:06:32.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.524+0000 7f75b5ed1640 1 Processor -- start 2023-10-16T11:06:32.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.524+0000 7f75b5ed1640 1 -- start start 2023-10-16T11:06:32.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.526+0000 7f75b5ed1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75b015c9f0 0x7f75b00c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:32.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.526+0000 7f75b5ed1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f75b00c9230 0x7f75b00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:32.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.526+0000 7f75b5ed1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b0160a90 con 0x7f75b0070f20 2023-10-16T11:06:32.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.526+0000 7f75b5ed1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b00cdec0 con 0x7f75b00c9230 2023-10-16T11:06:32.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.526+0000 7f75b5ed1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00cd900 con 0x7f75b015c9f0 2023-10-16T11:06:32.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.527+0000 7f75affff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f75b00c9230 0x7f75b00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:32.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.527+0000 7f75b4ecf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75b015c9f0 0x7f75b00c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:32.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.527+0000 7f75affff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f75b00c9230 0x7f75b00709e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:43490/0 (socket says 172.21.15.84:43490) 2023-10-16T11:06:32.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.527+0000 7f75affff640 1 -- 172.21.15.84:0/4149642524 learned_addr learned my addr 172.21.15.84:0/4149642524 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:32.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.527+0000 7f75b4ecf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75b015c9f0 0x7f75b00c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:56418/0 (socket says 172.21.15.84:56418) 2023-10-16T11:06:32.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.528+0000 7f75affff640 1 -- 172.21.15.84:0/4149642524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75b015c9f0 msgr2=0x7f75b00c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:32.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.528+0000 7f75affff640 1 --2- 172.21.15.84:0/4149642524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75b015c9f0 0x7f75b00c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:32.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.528+0000 7f75affff640 1 -- 172.21.15.84:0/4149642524 >> v1:172.21.15.157:6789/0 conn(0x7f75b0070f20 legacy=0x7f75b00ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:32.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.528+0000 7f75affff640 1 -- 172.21.15.84:0/4149642524 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b00cdbe0 con 0x7f75b00c9230 2023-10-16T11:06:32.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.529+0000 7f75affff640 1 --2- 172.21.15.84:0/4149642524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b00c9230 0x7f75b00709e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f759c009870 tx=0x7f759c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=570aef95c1c4633e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:32.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.530+0000 7f75af7fe640 1 -- 172.21.15.84:0/4149642524 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c030840 con 0x7f75b00c9230 2023-10-16T11:06:32.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.530+0000 7f75af7fe640 1 -- 172.21.15.84:0/4149642524 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c0309a0 con 0x7f75b00c9230 2023-10-16T11:06:32.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.530+0000 7f75af7fe640 1 -- 172.21.15.84:0/4149642524 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c030d30 con 0x7f75b00c9230 2023-10-16T11:06:32.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.530+0000 7f75b5ed1640 1 -- 172.21.15.84:0/4149642524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b00c9230 msgr2=0x7f75b00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:32.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.530+0000 7f75b5ed1640 1 --2- 172.21.15.84:0/4149642524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b00c9230 0x7f75b00709e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f759c009870 tx=0x7f759c02b980 comp rx=0 tx=0).stop 2023-10-16T11:06:32.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.531+0000 7f75b5ed1640 1 -- 172.21.15.84:0/4149642524 shutdown_connections 2023-10-16T11:06:32.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.531+0000 7f75b5ed1640 1 --2- 172.21.15.84:0/4149642524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b00c9230 0x7f75b00709e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:32.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.531+0000 7f75b5ed1640 1 --2- 172.21.15.84:0/4149642524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75b015c9f0 0x7f75b00c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:32.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.531+0000 7f75b5ed1640 1 -- 172.21.15.84:0/4149642524 >> 172.21.15.84:0/4149642524 conn(0x7f75b00ce6a0 msgr2=0x7f75b00ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:32.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.531+0000 7f75b5ed1640 1 -- 172.21.15.84:0/4149642524 shutdown_connections 2023-10-16T11:06:32.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.531+0000 7f75b5ed1640 1 -- 172.21.15.84:0/4149642524 wait complete. 2023-10-16T11:06:32.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.532+0000 7f75b5ed1640 1 Processor -- start 2023-10-16T11:06:32.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.533+0000 7f75b5ed1640 1 -- start start 2023-10-16T11:06:32.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.534+0000 7f75b5ed1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b0070f20 0x7f75b0159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:32.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.534+0000 7f75b5ed1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75b00c9230 0x7f75b00c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:32.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.534+0000 7f75b5ed1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b015c9f0 0x7f75b006bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:32.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.534+0000 7f75b5ed1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b006cc10 con 0x7f75b015c9f0 2023-10-16T11:06:32.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.534+0000 7f75b5ed1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b006c650 con 0x7f75b0070f20 2023-10-16T11:06:32.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.534+0000 7f75b5ed1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b006c930 con 0x7f75b00c9230 2023-10-16T11:06:32.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.534+0000 7f75b56d0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b015c9f0 0x7f75b006bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:32.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.534+0000 7f75b56d0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b015c9f0 0x7f75b006bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33434/0 (socket says 172.21.15.84:33434) 2023-10-16T11:06:32.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.534+0000 7f75b56d0640 1 -- 172.21.15.84:0/15356811 learned_addr learned my addr 172.21.15.84:0/15356811 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:32.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.534+0000 7f75b4ecf640 1 --2- 172.21.15.84:0/15356811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b0070f20 0x7f75b0159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:32.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.534+0000 7f75affff640 1 --2- 172.21.15.84:0/15356811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75b00c9230 0x7f75b00c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:32.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.534+0000 7f75b56d0640 1 -- 172.21.15.84:0/15356811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75b00c9230 msgr2=0x7f75b00c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:32.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.534+0000 7f75b56d0640 1 --2- 172.21.15.84:0/15356811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75b00c9230 0x7f75b00c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:32.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.534+0000 7f75b56d0640 1 -- 172.21.15.84:0/15356811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b0070f20 msgr2=0x7f75b0159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:32.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.534+0000 7f75b56d0640 1 --2- 172.21.15.84:0/15356811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b0070f20 0x7f75b0159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:32.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.534+0000 7f75b56d0640 1 -- 172.21.15.84:0/15356811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c0094d0 con 0x7f75b015c9f0 2023-10-16T11:06:32.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.535+0000 7f75b56d0640 1 --2- 172.21.15.84:0/15356811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b015c9f0 0x7f75b006bbf0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f75a800c7a0 tx=0x7f75a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:32.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.535+0000 7f75ad7fa640 1 -- 172.21.15.84:0/15356811 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8003b70 con 0x7f75b015c9f0 2023-10-16T11:06:32.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.535+0000 7f75ad7fa640 1 -- 172.21.15.84:0/15356811 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a8003cf0 con 0x7f75b015c9f0 2023-10-16T11:06:32.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.535+0000 7f75b5ed1640 1 -- 172.21.15.84:0/15356811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b00c7270 con 0x7f75b015c9f0 2023-10-16T11:06:32.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.535+0000 7f75ad7fa640 1 -- 172.21.15.84:0/15356811 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8003ea0 con 0x7f75b015c9f0 2023-10-16T11:06:32.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.535+0000 7f75b5ed1640 1 -- 172.21.15.84:0/15356811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b00c7fb0 con 0x7f75b015c9f0 2023-10-16T11:06:32.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.537+0000 7f75b5ed1640 1 -- 172.21.15.84:0/15356811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b00c7790 con 0x7f75b015c9f0 2023-10-16T11:06:32.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.538+0000 7f75ad7fa640 1 -- 172.21.15.84:0/15356811 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 103) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f75a800aa90 con 0x7f75b015c9f0 2023-10-16T11:06:32.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.538+0000 7f75ad7fa640 1 --2- 172.21.15.84:0/15356811 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f759406af90 0x7f759406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:32.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.539+0000 7f75ad7fa640 1 -- 172.21.15.84:0/15356811 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f75a8089610 con 0x7f75b015c9f0 2023-10-16T11:06:32.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.539+0000 7f75b4ecf640 1 --2- 172.21.15.84:0/15356811 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f759406af90 0x7f759406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:32.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.539+0000 7f75b4ecf640 1 --2- 172.21.15.84:0/15356811 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f759406af90 0x7f759406d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f75b00c1aa0 tx=0x7f75a4008040 comp rx=0 tx=0).ready entity=mgr.9969 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:32.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.543+0000 7f75ad7fa640 1 -- 172.21.15.84:0/15356811 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f75b00c7790 con 0x7f75b015c9f0 2023-10-16T11:06:32.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:32.788+0000 7f75b5ed1640 1 -- 172.21.15.84:0/15356811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f75b00c7790 con 0x7f75b015c9f0 2023-10-16T11:06:33.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.674+0000 7f75ad7fa640 1 -- 172.21.15.84:0/15356811 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v104) v1 ==== 92+0+0 (secure 0 0 0) 0x7f75b00c7790 con 0x7f75b015c9f0 2023-10-16T11:06:33.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.677+0000 7f75b5ed1640 1 -- 172.21.15.84:0/15356811 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f759406af90 msgr2=0x7f759406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:33.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.677+0000 7f75b5ed1640 1 --2- 172.21.15.84:0/15356811 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f759406af90 0x7f759406d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f75b00c1aa0 tx=0x7f75a4008040 comp rx=0 tx=0).stop 2023-10-16T11:06:33.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.677+0000 7f75b5ed1640 1 -- 172.21.15.84:0/15356811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b015c9f0 msgr2=0x7f75b006bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:33.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.677+0000 7f75b5ed1640 1 --2- 172.21.15.84:0/15356811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b015c9f0 0x7f75b006bbf0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f75a800c7a0 tx=0x7f75a800cc60 comp rx=0 tx=0).stop 2023-10-16T11:06:33.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.678+0000 7f75b5ed1640 1 -- 172.21.15.84:0/15356811 shutdown_connections 2023-10-16T11:06:33.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.678+0000 7f75b5ed1640 1 --2- 172.21.15.84:0/15356811 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f759406af90 0x7f759406d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:33.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.678+0000 7f75b5ed1640 1 --2- 172.21.15.84:0/15356811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b015c9f0 0x7f75b006bbf0 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:33.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.678+0000 7f75b5ed1640 1 --2- 172.21.15.84:0/15356811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75b00c9230 0x7f75b00c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:33.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.678+0000 7f75b5ed1640 1 --2- 172.21.15.84:0/15356811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b0070f20 0x7f75b0159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:33.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.678+0000 7f75b5ed1640 1 -- 172.21.15.84:0/15356811 >> 172.21.15.84:0/15356811 conn(0x7f75b00ce6a0 msgr2=0x7f75b015ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:33.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.679+0000 7f75b5ed1640 1 -- 172.21.15.84:0/15356811 shutdown_connections 2023-10-16T11:06:33.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.679+0000 7f75b5ed1640 1 -- 172.21.15.84:0/15356811 wait complete. 2023-10-16T11:06:33.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T11:06:33.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.905+0000 7fdd07f8e640 1 Processor -- start 2023-10-16T11:06:33.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.905+0000 7fdd07f8e640 1 -- start start 2023-10-16T11:06:33.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.907+0000 7fdd07f8e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdd000caac0 0x7fdd000ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:33.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.907+0000 7fdd07f8e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd000cb030 con 0x7fdd0006a4b0 2023-10-16T11:06:33.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.907+0000 7fdd07f8e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd000c91c0 con 0x7fdd0015ad70 2023-10-16T11:06:33.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.907+0000 7fdd07f8e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd000c94a0 con 0x7fdd000caac0 2023-10-16T11:06:33.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.907+0000 7fdd05529640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdd0006a4b0 0x7fdd000ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54676/0 (socket says 172.21.15.84:54676) 2023-10-16T11:06:33.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.907+0000 7fdd05529640 1 -- 172.21.15.84:0/1393918640 learned_addr learned my addr 172.21.15.84:0/1393918640 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:33.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.907+0000 7fdd0652b640 1 --2- 172.21.15.84:0/1393918640 >> v2:172.21.15.84:3300/0 conn(0x7fdd000caac0 0x7fdd000ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:33.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.907+0000 7fdd0652b640 1 -- 172.21.15.84:0/1393918640 >> v1:172.21.15.157:6789/0 conn(0x7fdd0015ad70 legacy=0x7fdd00069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:33.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.907+0000 7fdd0652b640 1 -- 172.21.15.84:0/1393918640 >> v1:172.21.15.84:6789/0 conn(0x7fdd0006a4b0 legacy=0x7fdd000ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:33.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.908+0000 7fdd0652b640 1 -- 172.21.15.84:0/1393918640 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd000c9780 con 0x7fdd000caac0 2023-10-16T11:06:33.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.907+0000 7fdd04d28640 1 -- 172.21.15.84:0/1393918640 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594612569 0 0) 0x7fdd000cb030 con 0x7fdd0006a4b0 2023-10-16T11:06:33.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.909+0000 7fdd0652b640 1 --2- 172.21.15.84:0/1393918640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd000caac0 0x7fdd000ad4d0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009d90 tx=0x7fdcf802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659b802071855b5d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:33.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.909+0000 7fdd04d28640 1 -- 172.21.15.84:0/1393918640 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf802cd00 con 0x7fdd000caac0 2023-10-16T11:06:33.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.909+0000 7fdd04d28640 1 -- 172.21.15.84:0/1393918640 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf802ce60 con 0x7fdd000caac0 2023-10-16T11:06:33.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.909+0000 7fdd04d28640 1 -- 172.21.15.84:0/1393918640 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8034660 con 0x7fdd000caac0 2023-10-16T11:06:33.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.910+0000 7fdd07f8e640 1 -- 172.21.15.84:0/1393918640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd000caac0 msgr2=0x7fdd000ad4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:33.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.910+0000 7fdd07f8e640 1 --2- 172.21.15.84:0/1393918640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd000caac0 0x7fdd000ad4d0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009d90 tx=0x7fdcf802bf50 comp rx=0 tx=0).stop 2023-10-16T11:06:33.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.910+0000 7fdd07f8e640 1 -- 172.21.15.84:0/1393918640 shutdown_connections 2023-10-16T11:06:33.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.910+0000 7fdd07f8e640 1 --2- 172.21.15.84:0/1393918640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd000caac0 0x7fdd000ad4d0 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:33.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.910+0000 7fdd07f8e640 1 -- 172.21.15.84:0/1393918640 >> 172.21.15.84:0/1393918640 conn(0x7fdd000ce9b0 msgr2=0x7fdd000cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:33.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.910+0000 7fdd07f8e640 1 -- 172.21.15.84:0/1393918640 shutdown_connections 2023-10-16T11:06:33.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.910+0000 7fdd07f8e640 1 -- 172.21.15.84:0/1393918640 wait complete. 2023-10-16T11:06:33.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.912+0000 7fdd07f8e640 1 Processor -- start 2023-10-16T11:06:33.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.912+0000 7fdd07f8e640 1 -- start start 2023-10-16T11:06:33.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.913+0000 7fdd07f8e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd0006a4b0 0x7fdd000c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:33.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.913+0000 7fdd07f8e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd000caac0 0x7fdd000c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:33.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.913+0000 7fdd07f8e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0015ad70 0x7fdd00149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:33.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.913+0000 7fdd07f8e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd001495c0 con 0x7fdd000caac0 2023-10-16T11:06:33.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.913+0000 7fdd07f8e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd000c7b10 con 0x7fdd0006a4b0 2023-10-16T11:06:33.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.913+0000 7fdd07f8e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd000c7df0 con 0x7fdd0015ad70 2023-10-16T11:06:33.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.913+0000 7fdd05529640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd000caac0 0x7fdd000c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:33.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.913+0000 7fdd05529640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd000caac0 0x7fdd000c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33468/0 (socket says 172.21.15.84:33468) 2023-10-16T11:06:33.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.913+0000 7fdd05529640 1 -- 172.21.15.84:0/1625178474 learned_addr learned my addr 172.21.15.84:0/1625178474 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:33.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.913+0000 7fdd05d2a640 1 --2- 172.21.15.84:0/1625178474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd0006a4b0 0x7fdd000c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:33.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.914+0000 7fdd0652b640 1 --2- 172.21.15.84:0/1625178474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0015ad70 0x7fdd00149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:33.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.914+0000 7fdd05529640 1 -- 172.21.15.84:0/1625178474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0015ad70 msgr2=0x7fdd00149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:33.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.914+0000 7fdd05529640 1 --2- 172.21.15.84:0/1625178474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0015ad70 0x7fdd00149080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:33.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.914+0000 7fdd05529640 1 -- 172.21.15.84:0/1625178474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd0006a4b0 msgr2=0x7fdd000c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:33.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.914+0000 7fdd05529640 1 --2- 172.21.15.84:0/1625178474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd0006a4b0 0x7fdd000c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:33.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.914+0000 7fdd05529640 1 -- 172.21.15.84:0/1625178474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf8009a70 con 0x7fdd000caac0 2023-10-16T11:06:33.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.914+0000 7fdd05529640 1 --2- 172.21.15.84:0/1625178474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd000caac0 0x7fdd000c8bb0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000c4a0 tx=0x7fdcf000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:33.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.914+0000 7fdcf67fc640 1 -- 172.21.15.84:0/1625178474 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf00074e0 con 0x7fdd000caac0 2023-10-16T11:06:33.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.914+0000 7fdcf67fc640 1 -- 172.21.15.84:0/1625178474 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf0007640 con 0x7fdd000caac0 2023-10-16T11:06:33.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.914+0000 7fdcf67fc640 1 -- 172.21.15.84:0/1625178474 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0004a00 con 0x7fdd000caac0 2023-10-16T11:06:33.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.914+0000 7fdd07f8e640 1 -- 172.21.15.84:0/1625178474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd000c80d0 con 0x7fdd000caac0 2023-10-16T11:06:33.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.915+0000 7fdd07f8e640 1 -- 172.21.15.84:0/1625178474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd000c8690 con 0x7fdd000caac0 2023-10-16T11:06:33.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.916+0000 7fdd07f8e640 1 -- 172.21.15.84:0/1625178474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd000c88b0 con 0x7fdd000caac0 2023-10-16T11:06:33.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.918+0000 7fdcf67fc640 1 -- 172.21.15.84:0/1625178474 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 104) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fdcf0004b60 con 0x7fdd000caac0 2023-10-16T11:06:33.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.918+0000 7fdcf67fc640 1 --2- 172.21.15.84:0/1625178474 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fdce406b010 0x7fdce406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:33.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.919+0000 7fdcf67fc640 1 -- 172.21.15.84:0/1625178474 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fdcf00887f0 con 0x7fdd000caac0 2023-10-16T11:06:33.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.919+0000 7fdd05d2a640 1 -- 172.21.15.84:0/1625178474 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fdce406b010 msgr2=0x7fdce406d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1441051579 2023-10-16T11:06:33.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.919+0000 7fdd05d2a640 1 --2- 172.21.15.84:0/1625178474 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fdce406b010 0x7fdce406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:06:33.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:33.923+0000 7fdcf67fc640 1 -- 172.21.15.84:0/1625178474 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdcf00538b0 con 0x7fdd000caac0 2023-10-16T11:06:34.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.116+0000 7fdd07f8e640 1 -- 172.21.15.84:0/1625178474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fdd0004e160 con 0x7fdd000caac0 2023-10-16T11:06:34.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.120+0000 7fdd05d2a640 1 -- 172.21.15.84:0/1625178474 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fdce406b010 msgr2=0x7fdce406d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1441051579 2023-10-16T11:06:34.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.120+0000 7fdd05d2a640 1 --2- 172.21.15.84:0/1625178474 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fdce406b010 0x7fdce406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:06:34.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.123+0000 7fdcf67fc640 1 -- 172.21.15.84:0/1625178474 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v65)=0 v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdcf0018020 con 0x7fdd000caac0 2023-10-16T11:06:34.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.127+0000 7fdd07f8e640 1 -- 172.21.15.84:0/1625178474 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fdce406b010 msgr2=0x7fdce406d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:06:34.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.127+0000 7fdd07f8e640 1 --2- 172.21.15.84:0/1625178474 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fdce406b010 0x7fdce406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.127+0000 7fdd07f8e640 1 -- 172.21.15.84:0/1625178474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd000caac0 msgr2=0x7fdd000c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:34.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.127+0000 7fdd07f8e640 1 --2- 172.21.15.84:0/1625178474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd000caac0 0x7fdd000c8bb0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000c4a0 tx=0x7fdcf000c960 comp rx=0 tx=0).stop 2023-10-16T11:06:34.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.127+0000 7fdd07f8e640 1 -- 172.21.15.84:0/1625178474 shutdown_connections 2023-10-16T11:06:34.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.127+0000 7fdd07f8e640 1 --2- 172.21.15.84:0/1625178474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0015ad70 0x7fdd00149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.127+0000 7fdd07f8e640 1 --2- 172.21.15.84:0/1625178474 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7fdce406b010 0x7fdce406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.127+0000 7fdd07f8e640 1 --2- 172.21.15.84:0/1625178474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd000caac0 0x7fdd000c8bb0 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.127+0000 7fdd07f8e640 1 --2- 172.21.15.84:0/1625178474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd0006a4b0 0x7fdd000c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.127+0000 7fdd07f8e640 1 -- 172.21.15.84:0/1625178474 >> 172.21.15.84:0/1625178474 conn(0x7fdd000ce9b0 msgr2=0x7fdd000afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:34.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.127+0000 7fdd07f8e640 1 -- 172.21.15.84:0/1625178474 shutdown_connections 2023-10-16T11:06:34.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.127+0000 7fdd07f8e640 1 -- 172.21.15.84:0/1625178474 wait complete. 2023-10-16T11:06:34.145 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T11:06:34.202 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:34.200+0000 7f9938852040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:06:34.207 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:34.206+0000 7fbf6a845040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:06:34.256 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:34.254+0000 7f9938852040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:06:34.263 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:34.263+0000 7fbf6a845040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:06:34.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.349+0000 7f291160e640 1 Processor -- start 2023-10-16T11:06:34.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.350+0000 7f291160e640 1 -- start start 2023-10-16T11:06:34.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.351+0000 7f291160e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f290c159ab0 0x7f290c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:34.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.351+0000 7f291160e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f290c06c140 0x7f290c06c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:34.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.351+0000 7f291160e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f290c06cad0 con 0x7f290c159e90 2023-10-16T11:06:34.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.352+0000 7f291160e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f290c15d890 con 0x7f290c159ab0 2023-10-16T11:06:34.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.352+0000 7f291160e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f290c070360 con 0x7f290c06c140 2023-10-16T11:06:34.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.352+0000 7f290affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f290c159ab0 0x7f290c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:34.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.352+0000 7f290affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f290c159ab0 0x7f290c0ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33470/0 (socket says 172.21.15.84:33470) 2023-10-16T11:06:34.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.352+0000 7f290affd640 1 -- 172.21.15.84:0/1117653296 learned_addr learned my addr 172.21.15.84:0/1117653296 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:34.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.352+0000 7f2903fff640 1 --1- 172.21.15.84:0/1117653296 >> v1:172.21.15.84:6789/0 conn(0x7f290c159e90 0x7f290c06fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54682/0 (socket says 172.21.15.84:54682) 2023-10-16T11:06:34.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.352+0000 7f290affd640 1 -- 172.21.15.84:0/1117653296 >> v2:172.21.15.157:3300/0 conn(0x7f290c06c140 msgr2=0x7f290c06c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:34.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.352+0000 7f290affd640 1 --2- 172.21.15.84:0/1117653296 >> v2:172.21.15.157:3300/0 conn(0x7f290c06c140 0x7f290c06c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.352+0000 7f290affd640 1 -- 172.21.15.84:0/1117653296 >> v1:172.21.15.84:6789/0 conn(0x7f290c159e90 legacy=0x7f290c06fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:34.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.353+0000 7f290affd640 1 -- 172.21.15.84:0/1117653296 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c0cd1f0 con 0x7f290c159ab0 2023-10-16T11:06:34.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.354+0000 7f290affd640 1 --2- 172.21.15.84:0/1117653296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c159ab0 0x7f290c0ce370 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f290c0adcd0 tx=0x7f28fc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0ae97b24a3cbcc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:34.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.354+0000 7f290a7fc640 1 -- 172.21.15.84:0/1117653296 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc0317e0 con 0x7f290c159ab0 2023-10-16T11:06:34.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.354+0000 7f290a7fc640 1 -- 172.21.15.84:0/1117653296 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28fc031940 con 0x7f290c159ab0 2023-10-16T11:06:34.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.354+0000 7f290a7fc640 1 -- 172.21.15.84:0/1117653296 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc031cd0 con 0x7f290c159ab0 2023-10-16T11:06:34.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.355+0000 7f291160e640 1 -- 172.21.15.84:0/1117653296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c159ab0 msgr2=0x7f290c0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:34.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.355+0000 7f291160e640 1 --2- 172.21.15.84:0/1117653296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c159ab0 0x7f290c0ce370 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f290c0adcd0 tx=0x7f28fc02b880 comp rx=0 tx=0).stop 2023-10-16T11:06:34.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.355+0000 7f291160e640 1 -- 172.21.15.84:0/1117653296 shutdown_connections 2023-10-16T11:06:34.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.355+0000 7f291160e640 1 --2- 172.21.15.84:0/1117653296 >> v2:172.21.15.157:3300/0 conn(0x7f290c06c140 0x7f290c06c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.355+0000 7f291160e640 1 --2- 172.21.15.84:0/1117653296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c159ab0 0x7f290c0ce370 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.355+0000 7f291160e640 1 -- 172.21.15.84:0/1117653296 >> 172.21.15.84:0/1117653296 conn(0x7f290c0ce9b0 msgr2=0x7f290c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:34.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.355+0000 7f291160e640 1 -- 172.21.15.84:0/1117653296 shutdown_connections 2023-10-16T11:06:34.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.355+0000 7f291160e640 1 -- 172.21.15.84:0/1117653296 wait complete. 2023-10-16T11:06:34.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.356+0000 7f291160e640 1 Processor -- start 2023-10-16T11:06:34.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.357+0000 7f291160e640 1 -- start start 2023-10-16T11:06:34.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.357+0000 7f291160e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c06c140 0x7f290c14deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:34.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.357+0000 7f291160e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f290c159ab0 0x7f290c0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:34.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.358+0000 7f291160e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f290c159e90 0x7f290c0acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:34.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.358+0000 7f291160e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c15cf50 con 0x7f290c06c140 2023-10-16T11:06:34.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.358+0000 7f291160e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c14e3f0 con 0x7f290c159e90 2023-10-16T11:06:34.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.358+0000 7f291160e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290c0c66d0 con 0x7f290c159ab0 2023-10-16T11:06:34.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.358+0000 7f290affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c06c140 0x7f290c14deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:34.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.358+0000 7f290affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c06c140 0x7f290c14deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33486/0 (socket says 172.21.15.84:33486) 2023-10-16T11:06:34.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.358+0000 7f290affd640 1 -- 172.21.15.84:0/2679944352 learned_addr learned my addr 172.21.15.84:0/2679944352 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:34.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.358+0000 7f2903fff640 1 --2- 172.21.15.84:0/2679944352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f290c159ab0 0x7f290c0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:34.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.358+0000 7f290b7fe640 1 --2- 172.21.15.84:0/2679944352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f290c159e90 0x7f290c0acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:34.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.358+0000 7f290affd640 1 -- 172.21.15.84:0/2679944352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f290c159ab0 msgr2=0x7f290c0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:34.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.358+0000 7f290affd640 1 --2- 172.21.15.84:0/2679944352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f290c159ab0 0x7f290c0ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.358+0000 7f290affd640 1 -- 172.21.15.84:0/2679944352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f290c159e90 msgr2=0x7f290c0acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:34.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.358+0000 7f290affd640 1 --2- 172.21.15.84:0/2679944352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f290c159e90 0x7f290c0acdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.358+0000 7f290affd640 1 -- 172.21.15.84:0/2679944352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc0094d0 con 0x7f290c06c140 2023-10-16T11:06:34.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.358+0000 7f2903fff640 1 --2- 172.21.15.84:0/2679944352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f290c159ab0 0x7f290c0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:06:34.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.359+0000 7f290affd640 1 --2- 172.21.15.84:0/2679944352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c06c140 0x7f290c14deb0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f28fc02b8c0 tx=0x7f28fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:34.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.359+0000 7f29037fe640 1 -- 172.21.15.84:0/2679944352 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc032980 con 0x7f290c06c140 2023-10-16T11:06:34.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.359+0000 7f29037fe640 1 -- 172.21.15.84:0/2679944352 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28fc032ae0 con 0x7f290c06c140 2023-10-16T11:06:34.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.359+0000 7f29037fe640 1 -- 172.21.15.84:0/2679944352 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc032e90 con 0x7f290c06c140 2023-10-16T11:06:34.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.359+0000 7f291160e640 1 -- 172.21.15.84:0/2679944352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f290c0c69b0 con 0x7f290c06c140 2023-10-16T11:06:34.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.359+0000 7f291160e640 1 -- 172.21.15.84:0/2679944352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290c0c6f70 con 0x7f290c06c140 2023-10-16T11:06:34.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.361+0000 7f291160e640 1 -- 172.21.15.84:0/2679944352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f290c04e160 con 0x7f290c06c140 2023-10-16T11:06:34.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.361+0000 7f29037fe640 1 -- 172.21.15.84:0/2679944352 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 104) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f28fc0453f0 con 0x7f290c06c140 2023-10-16T11:06:34.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.362+0000 7f29037fe640 1 --2- 172.21.15.84:0/2679944352 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f28ec06afe0 0x7f28ec06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:34.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.362+0000 7f2903fff640 1 -- 172.21.15.84:0/2679944352 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f28ec06afe0 msgr2=0x7f28ec06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1441051579 2023-10-16T11:06:34.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.362+0000 7f29037fe640 1 -- 172.21.15.84:0/2679944352 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f28fc0abc50 con 0x7f290c06c140 2023-10-16T11:06:34.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.362+0000 7f2903fff640 1 --2- 172.21.15.84:0/2679944352 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f28ec06afe0 0x7f28ec06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:06:34.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.370+0000 7f29037fe640 1 -- 172.21.15.84:0/2679944352 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f28fc076c00 con 0x7f290c06c140 2023-10-16T11:06:34.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.558+0000 7f291160e640 1 -- 172.21.15.84:0/2679944352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f290c0ae3b0 con 0x7f290c06c140 2023-10-16T11:06:34.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.563+0000 7f2903fff640 1 -- 172.21.15.84:0/2679944352 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f28ec06afe0 msgr2=0x7f28ec06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1441051579 2023-10-16T11:06:34.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.563+0000 7f2903fff640 1 --2- 172.21.15.84:0/2679944352 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f28ec06afe0 0x7f28ec06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:06:34.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.563+0000 7f29037fe640 1 -- 172.21.15.84:0/2679944352 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v66)=0 v66) v1 ==== 133+0+0 (secure 0 0 0) 0x7f28fc07a220 con 0x7f290c06c140 2023-10-16T11:06:34.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.566+0000 7f291160e640 1 -- 172.21.15.84:0/2679944352 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f28ec06afe0 msgr2=0x7f28ec06d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:06:34.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.566+0000 7f291160e640 1 --2- 172.21.15.84:0/2679944352 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f28ec06afe0 0x7f28ec06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.566+0000 7f291160e640 1 -- 172.21.15.84:0/2679944352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c06c140 msgr2=0x7f290c14deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:34.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.566+0000 7f291160e640 1 --2- 172.21.15.84:0/2679944352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c06c140 0x7f290c14deb0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f28fc02b8c0 tx=0x7f28fc0046c0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.566+0000 7f291160e640 1 -- 172.21.15.84:0/2679944352 shutdown_connections 2023-10-16T11:06:34.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.566+0000 7f291160e640 1 --2- 172.21.15.84:0/2679944352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f290c159e90 0x7f290c0acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.566+0000 7f291160e640 1 --2- 172.21.15.84:0/2679944352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f290c159ab0 0x7f290c0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.566+0000 7f291160e640 1 --2- 172.21.15.84:0/2679944352 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f28ec06afe0 0x7f28ec06d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.566+0000 7f291160e640 1 --2- 172.21.15.84:0/2679944352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c06c140 0x7f290c14deb0 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.566+0000 7f291160e640 1 -- 172.21.15.84:0/2679944352 >> 172.21.15.84:0/2679944352 conn(0x7f290c0ce9b0 msgr2=0x7f290c06b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:34.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.566+0000 7f291160e640 1 -- 172.21.15.84:0/2679944352 shutdown_connections 2023-10-16T11:06:34.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.566+0000 7f291160e640 1 -- 172.21.15.84:0/2679944352 wait complete. 2023-10-16T11:06:34.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:06:34.587 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-16T11:06:34.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:06:34.645 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2023-10-16T11:06:34.690 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/test_restart 1m 2023-10-16T11:06:34.709 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:34.708+0000 7f9938852040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:06:34.741 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:34.741+0000 7fbf6a845040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:06:34.762 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:34.760+0000 7f9938852040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:06:34.796 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:34.795+0000 7fbf6a845040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:06:34.812 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:34.811+0000 7f9938852040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:06:34.848 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:34.848+0000 7fbf6a845040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:06:34.873 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:34.872+0000 7f9938852040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:06:34.912 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:34.911+0000 7fbf6a845040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:06:34.922 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:34.921+0000 7f9938852040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:06:34.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.938+0000 7f6008a3e640 1 Processor -- start 2023-10-16T11:06:34.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.938+0000 7f6008a3e640 1 -- start start 2023-10-16T11:06:34.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.940+0000 7f6008a3e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f600415cd70 0x7f60040d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:34.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.940+0000 7f6008a3e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60040cded0 0x7f60040b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:34.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.940+0000 7f6008a3e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600415d150 con 0x7f60040b8480 2023-10-16T11:06:34.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.940+0000 7f6008a3e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60040d1180 con 0x7f600415cd70 2023-10-16T11:06:34.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.940+0000 7f6008a3e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60040d1460 con 0x7f60040cded0 2023-10-16T11:06:34.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.940+0000 7f5ffbfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f60040b8480 0x7f60040cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54698/0 (socket says 172.21.15.84:54698) 2023-10-16T11:06:34.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.940+0000 7f5ffbfff640 1 -- 172.21.15.84:0/4099059900 learned_addr learned my addr 172.21.15.84:0/4099059900 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:06:34.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.940+0000 7f6002d9d640 1 --2- 172.21.15.84:0/4099059900 >> v2:172.21.15.157:3300/0 conn(0x7f600415cd70 0x7f60040d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:34.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.940+0000 7f600359e640 1 --2- 172.21.15.84:0/4099059900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60040cded0 0x7f60040b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:34.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.940+0000 7f600259c640 1 -- 172.21.15.84:0/4099059900 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864462347 0 0) 0x7f600415d150 con 0x7f60040b8480 2023-10-16T11:06:34.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.941+0000 7f600259c640 1 -- 172.21.15.84:0/4099059900 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ff0003520 con 0x7f60040b8480 2023-10-16T11:06:34.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.941+0000 7f600259c640 1 -- 172.21.15.84:0/4099059900 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2889279662 0 0) 0x7f5ff0003520 con 0x7f60040b8480 2023-10-16T11:06:34.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.941+0000 7f600259c640 1 -- 172.21.15.84:0/4099059900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60040cded0 msgr2=0x7f60040b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:34.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.941+0000 7f600259c640 1 --2- 172.21.15.84:0/4099059900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60040cded0 0x7f60040b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.941+0000 7f600259c640 1 -- 172.21.15.84:0/4099059900 >> v2:172.21.15.157:3300/0 conn(0x7f600415cd70 msgr2=0x7f60040d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:34.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.941+0000 7f600259c640 1 --2- 172.21.15.84:0/4099059900 >> v2:172.21.15.157:3300/0 conn(0x7f600415cd70 0x7f60040d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.941+0000 7f600259c640 1 -- 172.21.15.84:0/4099059900 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60040d1740 con 0x7f60040b8480 2023-10-16T11:06:34.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.941+0000 7f600259c640 1 -- 172.21.15.84:0/4099059900 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f5fec001690 con 0x7f60040b8480 2023-10-16T11:06:34.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.941+0000 7f600259c640 1 -- 172.21.15.84:0/4099059900 >> v1:172.21.15.84:6789/0 conn(0x7f60040b8480 legacy=0x7f60040cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:34.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.941+0000 7f600259c640 1 --2- 172.21.15.84:0/4099059900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff0003970 0x7f5ff0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:34.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.941+0000 7f600259c640 1 --2- 172.21.15.84:0/4099059900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60040cded0 0x7f5ff00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:34.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.941+0000 7f600259c640 1 --2- 172.21.15.84:0/4099059900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ff0004d00 0x7f5ff0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:34.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.941+0000 7f600259c640 1 -- 172.21.15.84:0/4099059900 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60040d1180 con 0x7f5ff0004d00 2023-10-16T11:06:34.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.941+0000 7f600259c640 1 -- 172.21.15.84:0/4099059900 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60040d1460 con 0x7f5ff0003970 2023-10-16T11:06:34.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.941+0000 7f600259c640 1 -- 172.21.15.84:0/4099059900 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff000b980 con 0x7f60040cded0 2023-10-16T11:06:34.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.941+0000 7f6002d9d640 1 --2- 172.21.15.84:0/4099059900 >> v2:172.21.15.157:3300/0 conn(0x7f600415cd70 0x7f60040d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:06:34.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.942+0000 7f600359e640 1 --2- 172.21.15.84:0/4099059900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ff0004d00 0x7f5ff0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:34.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.942+0000 7f5ffbfff640 1 --2- 172.21.15.84:0/4099059900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60040cded0 0x7f5ff00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:34.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.942+0000 7f6002d9d640 1 --2- 172.21.15.84:0/4099059900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff0003970 0x7f5ff0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:34.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.942+0000 7f600359e640 1 -- 172.21.15.84:0/4099059900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60040cded0 msgr2=0x7f5ff00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:34.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.942+0000 7f600359e640 1 --2- 172.21.15.84:0/4099059900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60040cded0 0x7f5ff00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.942+0000 7f600359e640 1 -- 172.21.15.84:0/4099059900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff0003970 msgr2=0x7f5ff0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:34.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.942+0000 7f600359e640 1 --2- 172.21.15.84:0/4099059900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff0003970 0x7f5ff0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.942+0000 7f600359e640 1 -- 172.21.15.84:0/4099059900 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff0003520 con 0x7f5ff0004d00 2023-10-16T11:06:34.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.942+0000 7f6002d9d640 1 --2- 172.21.15.84:0/4099059900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff0003970 0x7f5ff0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:34.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.943+0000 7f600359e640 1 --2- 172.21.15.84:0/4099059900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ff0004d00 0x7f5ff0005180 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009f90 tx=0x7f5ff00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49870fca5e20820e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:34.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.943+0000 7f600259c640 1 -- 172.21.15.84:0/4099059900 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc02cbc0 con 0x7f5ff0004d00 2023-10-16T11:06:34.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.943+0000 7f600259c640 1 -- 172.21.15.84:0/4099059900 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ffc00f040 con 0x7f5ff0004d00 2023-10-16T11:06:34.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.943+0000 7f600259c640 1 -- 172.21.15.84:0/4099059900 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc034440 con 0x7f5ff0004d00 2023-10-16T11:06:34.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.944+0000 7f6008a3e640 1 -- 172.21.15.84:0/4099059900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ff0004d00 msgr2=0x7f5ff0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:34.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.944+0000 7f6008a3e640 1 --2- 172.21.15.84:0/4099059900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ff0004d00 0x7f5ff0005180 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009f90 tx=0x7f5ff00057f0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.944+0000 7f600359e640 1 -- 172.21.15.84:0/4099059900 reap_dead start 2023-10-16T11:06:34.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.944+0000 7f6008a3e640 1 -- 172.21.15.84:0/4099059900 shutdown_connections 2023-10-16T11:06:34.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.944+0000 7f6008a3e640 1 -- 172.21.15.84:0/4099059900 >> 172.21.15.84:0/4099059900 conn(0x7f60040cf570 msgr2=0x7f60040cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:34.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.944+0000 7f6008a3e640 1 -- 172.21.15.84:0/4099059900 shutdown_connections 2023-10-16T11:06:34.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.944+0000 7f6008a3e640 1 -- 172.21.15.84:0/4099059900 wait complete. 2023-10-16T11:06:34.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.946+0000 7f6008a3e640 1 Processor -- start 2023-10-16T11:06:34.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.946+0000 7f6008a3e640 1 -- start start 2023-10-16T11:06:34.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.947+0000 7f6008a3e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff0004d00 0x7f60041514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:34.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.947+0000 7f6008a3e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60041600e0 0x7f60040c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:34.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.947+0000 7f6008a3e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60040cb670 0x7f60040cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:34.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.947+0000 7f6008a3e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6004158b10 con 0x7f60041600e0 2023-10-16T11:06:34.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.947+0000 7f6008a3e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6004158810 con 0x7f5ff0004d00 2023-10-16T11:06:34.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.947+0000 7f6008a3e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600415ea60 con 0x7f60040cb670 2023-10-16T11:06:34.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.947+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60041600e0 0x7f60040c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:34.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.947+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60041600e0 0x7f60040c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33512/0 (socket says 172.21.15.84:33512) 2023-10-16T11:06:34.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.947+0000 7f5ffbfff640 1 -- 172.21.15.84:0/888670379 learned_addr learned my addr 172.21.15.84:0/888670379 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:34.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.947+0000 7f6002d9d640 1 --2- 172.21.15.84:0/888670379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff0004d00 0x7f60041514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:34.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.947+0000 7f5ffbfff640 1 -- 172.21.15.84:0/888670379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60040cb670 msgr2=0x7f60040cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:34.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.947+0000 7f5ffbfff640 1 --2- 172.21.15.84:0/888670379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60040cb670 0x7f60040cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.947+0000 7f5ffbfff640 1 -- 172.21.15.84:0/888670379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff0004d00 msgr2=0x7f60041514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:34.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.947+0000 7f5ffbfff640 1 --2- 172.21.15.84:0/888670379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff0004d00 0x7f60041514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:34.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.947+0000 7f5ffbfff640 1 -- 172.21.15.84:0/888670379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600415ed40 con 0x7f60041600e0 2023-10-16T11:06:34.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.948+0000 7f5ffbfff640 1 --2- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60041600e0 0x7f60040c9bc0 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00e820 tx=0x7f5fec00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:34.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.948+0000 7f5ffb7fe640 1 -- 172.21.15.84:0/888670379 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec00c7f0 con 0x7f60041600e0 2023-10-16T11:06:34.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.948+0000 7f5ffb7fe640 1 -- 172.21.15.84:0/888670379 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec00c950 con 0x7f60041600e0 2023-10-16T11:06:34.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.948+0000 7f5ffb7fe640 1 -- 172.21.15.84:0/888670379 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec00cd00 con 0x7f60041600e0 2023-10-16T11:06:34.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.948+0000 7f6008a3e640 1 -- 172.21.15.84:0/888670379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60040cc040 con 0x7f60041600e0 2023-10-16T11:06:34.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.948+0000 7f6008a3e640 1 -- 172.21.15.84:0/888670379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60040ca100 con 0x7f60041600e0 2023-10-16T11:06:34.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.949+0000 7f6008a3e640 1 -- 172.21.15.84:0/888670379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6004158c80 con 0x7f60041600e0 2023-10-16T11:06:34.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.950+0000 7f5ffb7fe640 1 -- 172.21.15.84:0/888670379 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 104) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f5fec01d460 con 0x7f60041600e0 2023-10-16T11:06:34.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.951+0000 7f5ffb7fe640 1 --2- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f5fe406b010 0x7f5fe406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:34.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.951+0000 7f5ffb7fe640 1 -- 172.21.15.84:0/888670379 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5fec0897c0 con 0x7f60041600e0 2023-10-16T11:06:34.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.951+0000 7f6002d9d640 1 -- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f5fe406b010 msgr2=0x7f5fe406d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1441051579 2023-10-16T11:06:34.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.951+0000 7f6002d9d640 1 --2- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f5fe406b010 0x7f5fe406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:06:34.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:34.960+0000 7f5ffb7fe640 1 -- 172.21.15.84:0/888670379 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5fec054740 con 0x7f60041600e0 2023-10-16T11:06:34.968 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:34.968+0000 7fbf6a845040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:06:35.034 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:35.033+0000 7f9938852040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:06:35.082 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:35.082+0000 7fbf6a845040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:06:35.096 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:35.094+0000 7f9938852040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:06:35.144 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:35.143+0000 7f9938852040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:06:35.145 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:35.144+0000 7fbf6a845040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:06:35.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:35.152+0000 7f6002d9d640 1 -- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f5fe406b010 msgr2=0x7f5fe406d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1441051579 2023-10-16T11:06:35.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:35.152+0000 7f6002d9d640 1 --2- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f5fe406b010 0x7f5fe406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:06:35.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:35.165+0000 7f6008a3e640 1 -- 172.21.15.84:0/888670379 --> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f60040d1d00 con 0x7f5fe406b010 2023-10-16T11:06:35.195 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:35.195+0000 7fbf6a845040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:06:35.201 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:35.200+0000 7f9938852040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:06:35.251 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:35.251+0000 7fbf6a845040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:06:35.252 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:35.250+0000 7f9938852040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:06:35.301 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:35.301+0000 7fbf6a845040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:06:35.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:35.553+0000 7f6002d9d640 1 -- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f5fe406b010 msgr2=0x7f5fe406d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1441051579 2023-10-16T11:06:35.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:35.553+0000 7f6002d9d640 1 --2- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f5fe406b010 0x7f5fe406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T11:06:35.631 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:35.630+0000 7f9938852040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:06:35.676 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:35.676+0000 7fbf6a845040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:06:35.790 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:35.788+0000 7f9938852040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:06:35.835 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:35.834+0000 7fbf6a845040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:06:35.897 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:35.895+0000 7f9938852040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:06:35.942 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:35.941+0000 7fbf6a845040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:06:35.947 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:35.945+0000 7f9938852040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:06:35.991 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:35.990+0000 7fbf6a845040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:06:36.053 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:36.051+0000 7f9938852040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:06:36.099 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:36.098+0000 7fbf6a845040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:06:36.159 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:36.157+0000 7f9938852040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:06:36.203 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:36.203+0000 7fbf6a845040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:06:36.316 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:36.313+0000 7f9938852040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:06:36.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:36.354+0000 7f6002d9d640 1 -- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f5fe406b010 msgr2=0x7f5fe406d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1441051579 2023-10-16T11:06:36.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:36.354+0000 7f6002d9d640 1 --2- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f5fe406b010 0x7f5fe406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T11:06:36.359 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:36.358+0000 7fbf6a845040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:06:36.364 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:36.363+0000 7f9938852040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:06:36.407 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:36.406+0000 7fbf6a845040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:06:36.484 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:36.483+0000 7f9938852040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:06:36.526 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:36.525+0000 7fbf6a845040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:06:37.299 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:37.297+0000 7f9938852040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:06:37.355 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:37.354+0000 7fbf6a845040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:06:37.592 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:37.590+0000 7f9938852040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:06:37.646 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:37.646+0000 7fbf6a845040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:06:37.648 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:37.647+0000 7f9938852040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:06:37.703 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:37.702+0000 7fbf6a845040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:06:37.761 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:06:37.759+0000 7f9938852040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:06:37.816 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:37.816+0000 7fbf6a845040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:06:37.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:37.827+0000 7f5ffb7fe640 1 -- 172.21.15.84:0/888670379 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 105) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f5fec087cc0 con 0x7f60041600e0 2023-10-16T11:06:37.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:37.849+0000 7f5ffb7fe640 1 -- 172.21.15.84:0/888670379 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 106) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f5fec0878d0 con 0x7f60041600e0 2023-10-16T11:06:37.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:37.850+0000 7f5ffb7fe640 1 -- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f5fe406b010 msgr2=0x7f5fe406d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:06:37.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:37.850+0000 7f5ffb7fe640 1 --2- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:6832/1441051579,v1:172.21.15.84:6833/1441051579] conn(0x7f5fe406b010 0x7f5fe406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:38.207 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:38.206+0000 7fbee734e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:38.207 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:38.206+0000 7fbee734e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:38.207 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:38.206+0000 7fbee734e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:38.207 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:38.206+0000 7fbee734e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:38.208 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:38.206+0000 7fbee734e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:38.208 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:38.206+0000 7fbeea394640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:38.208 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:38.207+0000 7fbeea394640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:38.209 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:38.207+0000 7fbeea394640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:38.209 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:38.207+0000 7fbeea394640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:38.209 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:06:38.207+0000 7fbeea394640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:06:38.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:38.857+0000 7f5ffb7fe640 1 -- 172.21.15.84:0/888670379 <== mon.0 v2:172.21.15.84:3300/0 9 ==== mgrmap(e 107) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f5fec0553e0 con 0x7f60041600e0 2023-10-16T11:06:38.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:38.859+0000 7f5ffb7fe640 1 --2- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7f5fe401d1f0 0x7f5fe4074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:38.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:38.859+0000 7f5ffb7fe640 1 -- 172.21.15.84:0/888670379 --> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fe400b680 con 0x7f5fe401d1f0 2023-10-16T11:06:38.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:38.866+0000 7f6002d9d640 1 --2- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7f5fe401d1f0 0x7f5fe4074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:38.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:38.873+0000 7f6002d9d640 1 --2- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7f5fe401d1f0 0x7f5fe4074f30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5ff40093d0 tx=0x7f5ff4008040 comp rx=0 tx=0).ready entity=mgr.10374 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:39.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.072+0000 7f5ffb7fe640 1 -- 172.21.15.84:0/888670379 <== mgr.10374 v2:172.21.15.84:6832/3993913200 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f5fe400b680 con 0x7f5fe401d1f0 2023-10-16T11:06:39.073 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path /snap_test_dir1/test_restart 2023-10-16T11:06:39.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.075+0000 7f6008a3e640 1 -- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7f5fe401d1f0 msgr2=0x7f5fe4074f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:39.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.075+0000 7f6008a3e640 1 --2- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7f5fe401d1f0 0x7f5fe4074f30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5ff40093d0 tx=0x7f5ff4008040 comp rx=0 tx=0).stop 2023-10-16T11:06:39.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.075+0000 7f6008a3e640 1 -- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60041600e0 msgr2=0x7f60040c9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:39.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.075+0000 7f6008a3e640 1 --2- 172.21.15.84:0/888670379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60041600e0 0x7f60040c9bc0 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00e820 tx=0x7f5fec00ece0 comp rx=0 tx=0).stop 2023-10-16T11:06:39.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.075+0000 7f600359e640 1 -- 172.21.15.84:0/888670379 reap_dead start 2023-10-16T11:06:39.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.075+0000 7f6008a3e640 1 -- 172.21.15.84:0/888670379 shutdown_connections 2023-10-16T11:06:39.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.075+0000 7f6008a3e640 1 -- 172.21.15.84:0/888670379 >> 172.21.15.84:0/888670379 conn(0x7f60040cf570 msgr2=0x7f600415c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:39.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.075+0000 7f6008a3e640 1 -- 172.21.15.84:0/888670379 shutdown_connections 2023-10-16T11:06:39.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.075+0000 7f6008a3e640 1 -- 172.21.15.84:0/888670379 wait complete. 2023-10-16T11:06:39.095 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2023-10-16T11:06:39.095 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/test_restart 1m 2023-10-16T11:06:39.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.360+0000 7efd1558e640 1 Processor -- start 2023-10-16T11:06:39.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.360+0000 7efd1558e640 1 -- start start 2023-10-16T11:06:39.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.361+0000 7efd1558e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd10159ab0 0x7efd100ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:39.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.361+0000 7efd1558e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efd1006c140 0x7efd1006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:39.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.361+0000 7efd1558e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd10070360 con 0x7efd10159e90 2023-10-16T11:06:39.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.361+0000 7efd1558e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd1006cb10 con 0x7efd1006c140 2023-10-16T11:06:39.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.361+0000 7efd1558e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1015d890 con 0x7efd10159ab0 2023-10-16T11:06:39.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.362+0000 7efd0f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efd1006c140 0x7efd1006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:39.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.362+0000 7efd0f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efd1006c140 0x7efd1006c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45852/0 (socket says 172.21.15.84:45852) 2023-10-16T11:06:39.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.362+0000 7efd0f7fe640 1 -- 172.21.15.84:0/2078296686 learned_addr learned my addr 172.21.15.84:0/2078296686 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:39.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.362+0000 7efd0effd640 1 --2- 172.21.15.84:0/2078296686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd10159ab0 0x7efd100ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:39.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.362+0000 7efd0f7fe640 1 -- 172.21.15.84:0/2078296686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd10159ab0 msgr2=0x7efd100ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:39.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.362+0000 7efd0f7fe640 1 --2- 172.21.15.84:0/2078296686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd10159ab0 0x7efd100ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:39.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.362+0000 7efd0f7fe640 1 -- 172.21.15.84:0/2078296686 >> v1:172.21.15.157:6789/0 conn(0x7efd10159e90 legacy=0x7efd1006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:06:39.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.362+0000 7efd0f7fe640 1 -- 172.21.15.84:0/2078296686 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd100cd1f0 con 0x7efd1006c140 2023-10-16T11:06:39.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.362+0000 7efd0effd640 1 --2- 172.21.15.84:0/2078296686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd10159ab0 0x7efd100ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:06:39.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.364+0000 7efd0f7fe640 1 --2- 172.21.15.84:0/2078296686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd1006c140 0x7efd1006c5d0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7efd0800bb40 tx=0x7efd0802df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea0d38ca6c4d67 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:39.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.364+0000 7efd0dffb640 1 -- 172.21.15.84:0/2078296686 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0802ed40 con 0x7efd1006c140 2023-10-16T11:06:39.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.364+0000 7efd0dffb640 1 -- 172.21.15.84:0/2078296686 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd08036440 con 0x7efd1006c140 2023-10-16T11:06:39.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.364+0000 7efd0dffb640 1 -- 172.21.15.84:0/2078296686 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0802eea0 con 0x7efd1006c140 2023-10-16T11:06:39.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.365+0000 7efd1558e640 1 -- 172.21.15.84:0/2078296686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd1006c140 msgr2=0x7efd1006c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:39.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.365+0000 7efd1558e640 1 --2- 172.21.15.84:0/2078296686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd1006c140 0x7efd1006c5d0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7efd0800bb40 tx=0x7efd0802df90 comp rx=0 tx=0).stop 2023-10-16T11:06:39.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.365+0000 7efd1558e640 1 -- 172.21.15.84:0/2078296686 shutdown_connections 2023-10-16T11:06:39.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.365+0000 7efd1558e640 1 --2- 172.21.15.84:0/2078296686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd1006c140 0x7efd1006c5d0 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:39.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.365+0000 7efd1558e640 1 --2- 172.21.15.84:0/2078296686 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd10159ab0 0x7efd100ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:39.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.365+0000 7efd1558e640 1 -- 172.21.15.84:0/2078296686 >> 172.21.15.84:0/2078296686 conn(0x7efd100ce9b0 msgr2=0x7efd100cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:39.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.365+0000 7efd1558e640 1 -- 172.21.15.84:0/2078296686 shutdown_connections 2023-10-16T11:06:39.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.365+0000 7efd1558e640 1 -- 172.21.15.84:0/2078296686 wait complete. 2023-10-16T11:06:39.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.367+0000 7efd1558e640 1 Processor -- start 2023-10-16T11:06:39.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.367+0000 7efd1558e640 1 -- start start 2023-10-16T11:06:39.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.368+0000 7efd1558e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd1006c140 0x7efd1006b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:39.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.368+0000 7efd1558e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd10159ab0 0x7efd100c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:39.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.368+0000 7efd1558e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd10159e90 0x7efd100c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:39.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.368+0000 7efd1558e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1015a3d0 con 0x7efd10159e90 2023-10-16T11:06:39.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.368+0000 7efd1558e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1015a670 con 0x7efd1006c140 2023-10-16T11:06:39.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.368+0000 7efd1558e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1015a950 con 0x7efd10159ab0 2023-10-16T11:06:39.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.368+0000 7efd0f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd10159e90 0x7efd100c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:39.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.368+0000 7efd0f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd10159e90 0x7efd100c4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45864/0 (socket says 172.21.15.84:45864) 2023-10-16T11:06:39.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.368+0000 7efd0f7fe640 1 -- 172.21.15.84:0/1616330520 learned_addr learned my addr 172.21.15.84:0/1616330520 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:06:39.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.369+0000 7efd0f7fe640 1 -- 172.21.15.84:0/1616330520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd10159ab0 msgr2=0x7efd100c3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:39.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.369+0000 7efd0f7fe640 1 --2- 172.21.15.84:0/1616330520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd10159ab0 0x7efd100c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:39.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.369+0000 7efd0f7fe640 1 -- 172.21.15.84:0/1616330520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd1006c140 msgr2=0x7efd1006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:39.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.369+0000 7efd0f7fe640 1 --2- 172.21.15.84:0/1616330520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd1006c140 0x7efd1006b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:39.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.369+0000 7efd0f7fe640 1 -- 172.21.15.84:0/1616330520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0800b770 con 0x7efd10159e90 2023-10-16T11:06:39.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.369+0000 7efd0f7fe640 1 --2- 172.21.15.84:0/1616330520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd10159e90 0x7efd100c4470 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7efd0802dfd0 tx=0x7efd080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:39.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.369+0000 7efd077fe640 1 -- 172.21.15.84:0/1616330520 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0803f900 con 0x7efd10159e90 2023-10-16T11:06:39.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.369+0000 7efd077fe640 1 -- 172.21.15.84:0/1616330520 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0803fa60 con 0x7efd10159e90 2023-10-16T11:06:39.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.369+0000 7efd077fe640 1 -- 172.21.15.84:0/1616330520 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0803fe10 con 0x7efd10159e90 2023-10-16T11:06:39.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.369+0000 7efd1558e640 1 -- 172.21.15.84:0/1616330520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1015ac30 con 0x7efd10159e90 2023-10-16T11:06:39.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.370+0000 7efd1558e640 1 -- 172.21.15.84:0/1616330520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd100c4b50 con 0x7efd10159e90 2023-10-16T11:06:39.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.371+0000 7efd1558e640 1 -- 172.21.15.84:0/1616330520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1015ae50 con 0x7efd10159e90 2023-10-16T11:06:39.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.372+0000 7efd077fe640 1 -- 172.21.15.84:0/1616330520 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 107) v1 ==== 89084+0+0 (secure 0 0 0) 0x7efd0803d040 con 0x7efd10159e90 2023-10-16T11:06:39.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.372+0000 7efd077fe640 1 --2- 172.21.15.84:0/1616330520 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7efcf006ad10 0x7efcf006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:06:39.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.373+0000 7efd077fe640 1 -- 172.21.15.84:0/1616330520 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6081+0+0 (secure 0 0 0) 0x7efd080adf30 con 0x7efd10159e90 2023-10-16T11:06:39.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.373+0000 7efd0effd640 1 --2- 172.21.15.84:0/1616330520 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7efcf006ad10 0x7efcf006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:06:39.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.373+0000 7efd0effd640 1 --2- 172.21.15.84:0/1616330520 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7efcf006ad10 0x7efcf006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efd00008e00 tx=0x7efd00008330 comp rx=0 tx=0).ready entity=mgr.10374 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:06:39.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.378+0000 7efd077fe640 1 -- 172.21.15.84:0/1616330520 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efd080791a0 con 0x7efd10159e90 2023-10-16T11:06:39.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.597+0000 7efd1558e640 1 -- 172.21.15.84:0/1616330520 --> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/test_restart", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7efd100cd7b0 con 0x7efcf006ad10 2023-10-16T11:06:39.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.754+0000 7efd077fe640 1 -- 172.21.15.84:0/1616330520 <== mgr.10374 v2:172.21.15.84:6832/3993913200 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7efd000089f0 con 0x7efcf006ad10 2023-10-16T11:06:39.756 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule activated for path /snap_test_dir1/test_restart 2023-10-16T11:06:39.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.758+0000 7efd1558e640 1 -- 172.21.15.84:0/1616330520 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7efcf006ad10 msgr2=0x7efcf006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:39.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.758+0000 7efd1558e640 1 --2- 172.21.15.84:0/1616330520 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7efcf006ad10 0x7efcf006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efd00008e00 tx=0x7efd00008330 comp rx=0 tx=0).stop 2023-10-16T11:06:39.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.758+0000 7efd1558e640 1 -- 172.21.15.84:0/1616330520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd10159e90 msgr2=0x7efd100c4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:06:39.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.758+0000 7efd1558e640 1 --2- 172.21.15.84:0/1616330520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd10159e90 0x7efd100c4470 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7efd0802dfd0 tx=0x7efd080046c0 comp rx=0 tx=0).stop 2023-10-16T11:06:39.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.758+0000 7efd1558e640 1 -- 172.21.15.84:0/1616330520 shutdown_connections 2023-10-16T11:06:39.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.759+0000 7efd1558e640 1 --2- 172.21.15.84:0/1616330520 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7efcf006ad10 0x7efcf006d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:39.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.759+0000 7efd1558e640 1 --2- 172.21.15.84:0/1616330520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd10159e90 0x7efd100c4470 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:39.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.759+0000 7efd1558e640 1 --2- 172.21.15.84:0/1616330520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd10159ab0 0x7efd100c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:39.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.759+0000 7efd1558e640 1 --2- 172.21.15.84:0/1616330520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd1006c140 0x7efd1006b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:06:39.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.759+0000 7efd1558e640 1 -- 172.21.15.84:0/1616330520 >> 172.21.15.84:0/1616330520 conn(0x7efd100ce9b0 msgr2=0x7efd100ae640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:06:39.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.759+0000 7efd1558e640 1 -- 172.21.15.84:0/1616330520 shutdown_connections 2023-10-16T11:06:39.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:06:39.759+0000 7efd1558e640 1 -- 172.21.15.84:0/1616330520 wait complete. 2023-10-16T11:06:39.781 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2023-10-16T11:06:57.454 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:06:57.464 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:07:00.777 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:07:00.776+0000 7fbed9332640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:07:00.778 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:07:00.776+0000 7fbed9332640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:07:00.778 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:07:00.776+0000 7fbed9332640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:07:00.778 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:07:00.776+0000 7fbed9332640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:07:00.778 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:07:00.776+0000 7fbed9332640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:07:27.763 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:07:27.773 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:07:57.932 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:07:57.942 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:08:28.079 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:08:28.089 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:08:58.509 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:08:58.519 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:09:28.659 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:09:28.670 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:09:59.104 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:09:59.114 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:10:29.256 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:10:29.265 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:10:59.462 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:10:59.472 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:11:29.707 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:11:29.717 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:11:59.855 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:11:59.865 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:12:30.010 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:12:30.020 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:13:00.160 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:13:00.170 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:13:30.311 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:13:30.321 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:14:00.458 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:14:00.467 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:14:30.604 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:14:30.614 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:15:00.751 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:15:00.760 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:15:30.891 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:15:30.901 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:16:01.045 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:16:01.054 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:16:31.215 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:16:31.225 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:17:01.372 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:17:01.382 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:17:15.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:15.783 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-16T11:17:15.848 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_07_00_UTC 2023-10-16T11:17:15.848 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_08_00_UTC 2023-10-16T11:17:15.848 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_09_00_UTC 2023-10-16T11:17:15.848 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_10_00_UTC 2023-10-16T11:17:15.849 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_11_00_UTC 2023-10-16T11:17:15.849 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_12_00_UTC 2023-10-16T11:17:15.849 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_13_00_UTC 2023-10-16T11:17:15.849 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_14_00_UTC 2023-10-16T11:17:15.849 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_15_00_UTC 2023-10-16T11:17:15.849 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_16_00_UTC 2023-10-16T11:17:15.850 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_17_00_UTC 2023-10-16T11:17:15.851 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-11_07_00_UTC', 'scheduled-2023-10-16-11_08_00_UTC', 'scheduled-2023-10-16-11_09_00_UTC', 'scheduled-2023-10-16-11_10_00_UTC', 'scheduled-2023-10-16-11_11_00_UTC', 'scheduled-2023-10-16-11_12_00_UTC', 'scheduled-2023-10-16-11_13_00_UTC', 'scheduled-2023-10-16-11_14_00_UTC', 'scheduled-2023-10-16-11_15_00_UTC', 'scheduled-2023-10-16-11_16_00_UTC', 'scheduled-2023-10-16-11_17_00_UTC'] 2023-10-16T11:17:15.851 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/test_restart 2023-10-16T11:17:16.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.065+0000 7f4e7debb640 1 Processor -- start 2023-10-16T11:17:16.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.065+0000 7f4e7debb640 1 -- start start 2023-10-16T11:17:16.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.066+0000 7f4e7debb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4e7815ad70 0x7f4e78069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.066+0000 7f4e7debb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e780cb030 con 0x7f4e780c9ca0 2023-10-16T11:17:16.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.066+0000 7f4e7debb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e780c91c0 con 0x7f4e780caac0 2023-10-16T11:17:16.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.066+0000 7f4e7debb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e780c94a0 con 0x7f4e7815ad70 2023-10-16T11:17:16.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.067+0000 7f4e6ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4e780c9ca0 0x7f4e780ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43982/0 (socket says 172.21.15.84:43982) 2023-10-16T11:17:16.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.067+0000 7f4e6ffff640 1 -- 172.21.15.84:0/1052445128 learned_addr learned my addr 172.21.15.84:0/1052445128 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:17:16.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.067+0000 7f4e76ffd640 1 -- 172.21.15.84:0/1052445128 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3834252976 0 0) 0x7f4e780cb030 con 0x7f4e780c9ca0 2023-10-16T11:17:16.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.067+0000 7f4e76ffd640 1 -- 172.21.15.84:0/1052445128 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e64003520 con 0x7f4e780c9ca0 2023-10-16T11:17:16.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.067+0000 7f4e777fe640 1 --2- 172.21.15.84:0/1052445128 >> v2:172.21.15.157:3300/0 conn(0x7f4e7815ad70 0x7f4e78069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:16.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.068+0000 7f4e76ffd640 1 -- 172.21.15.84:0/1052445128 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3874269776 0 0) 0x7f4e64003520 con 0x7f4e780c9ca0 2023-10-16T11:17:16.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.068+0000 7f4e76ffd640 1 -- 172.21.15.84:0/1052445128 >> v2:172.21.15.157:3300/0 conn(0x7f4e7815ad70 msgr2=0x7f4e78069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:16.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.068+0000 7f4e76ffd640 1 --2- 172.21.15.84:0/1052445128 >> v2:172.21.15.157:3300/0 conn(0x7f4e7815ad70 0x7f4e78069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.068+0000 7f4e76ffd640 1 -- 172.21.15.84:0/1052445128 >> v1:172.21.15.157:6789/0 conn(0x7f4e780caac0 legacy=0x7f4e780ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:16.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.068+0000 7f4e76ffd640 1 -- 172.21.15.84:0/1052445128 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e780c9780 con 0x7f4e780c9ca0 2023-10-16T11:17:16.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.068+0000 7f4e76ffd640 1 -- 172.21.15.84:0/1052445128 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f4e60001670 con 0x7f4e780c9ca0 2023-10-16T11:17:16.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.068+0000 7f4e76ffd640 1 -- 172.21.15.84:0/1052445128 >> v1:172.21.15.84:6789/0 conn(0x7f4e780c9ca0 legacy=0x7f4e780ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:16.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.068+0000 7f4e76ffd640 1 --2- 172.21.15.84:0/1052445128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e64003970 0x7f4e64003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.068+0000 7f4e76ffd640 1 --2- 172.21.15.84:0/1052445128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e640047c0 0x7f4e64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.068+0000 7f4e76ffd640 1 --2- 172.21.15.84:0/1052445128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e64005180 0x7f4e6400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.068+0000 7f4e76ffd640 1 -- 172.21.15.84:0/1052445128 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780c94a0 con 0x7f4e64005180 2023-10-16T11:17:16.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.068+0000 7f4e76ffd640 1 -- 172.21.15.84:0/1052445128 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6400c0b0 con 0x7f4e64003970 2023-10-16T11:17:16.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.068+0000 7f4e76ffd640 1 -- 172.21.15.84:0/1052445128 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6400c270 con 0x7f4e640047c0 2023-10-16T11:17:16.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.068+0000 7f4e77fff640 1 --2- 172.21.15.84:0/1052445128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e64005180 0x7f4e6400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:16.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.068+0000 7f4e777fe640 1 --2- 172.21.15.84:0/1052445128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e64003970 0x7f4e64003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:16.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.068+0000 7f4e6ffff640 1 --2- 172.21.15.84:0/1052445128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e640047c0 0x7f4e64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:16.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.069+0000 7f4e77fff640 1 -- 172.21.15.84:0/1052445128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e640047c0 msgr2=0x7f4e64004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:16.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.069+0000 7f4e77fff640 1 --2- 172.21.15.84:0/1052445128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e640047c0 0x7f4e64004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.069+0000 7f4e77fff640 1 -- 172.21.15.84:0/1052445128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e64003970 msgr2=0x7f4e64003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:16.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.069+0000 7f4e77fff640 1 --2- 172.21.15.84:0/1052445128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e64003970 0x7f4e64003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.069+0000 7f4e77fff640 1 -- 172.21.15.84:0/1052445128 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e64003520 con 0x7f4e64005180 2023-10-16T11:17:16.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.069+0000 7f4e77fff640 1 -- 172.21.15.84:0/1052445128 reap_dead start 2023-10-16T11:17:16.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.070+0000 7f4e77fff640 1 --2- 172.21.15.84:0/1052445128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e64005180 0x7f4e6400b980 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f4e70004480 tx=0x7f4e640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404eef357b3a8ce1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:16.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.070+0000 7f4e76ffd640 1 -- 172.21.15.84:0/1052445128 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70006b50 con 0x7f4e64005180 2023-10-16T11:17:16.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.070+0000 7f4e76ffd640 1 -- 172.21.15.84:0/1052445128 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e70006cd0 con 0x7f4e64005180 2023-10-16T11:17:16.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.070+0000 7f4e76ffd640 1 -- 172.21.15.84:0/1052445128 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70007260 con 0x7f4e64005180 2023-10-16T11:17:16.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.070+0000 7f4e7debb640 1 -- 172.21.15.84:0/1052445128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e64005180 msgr2=0x7f4e6400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:16.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.070+0000 7f4e7debb640 1 --2- 172.21.15.84:0/1052445128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e64005180 0x7f4e6400b980 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f4e70004480 tx=0x7f4e640057f0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.071+0000 7f4e7debb640 1 -- 172.21.15.84:0/1052445128 shutdown_connections 2023-10-16T11:17:16.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.071+0000 7f4e7debb640 1 --2- 172.21.15.84:0/1052445128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e64005180 0x7f4e6400b980 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.071+0000 7f4e7debb640 1 -- 172.21.15.84:0/1052445128 >> 172.21.15.84:0/1052445128 conn(0x7f4e780ce9b0 msgr2=0x7f4e780cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:16.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.071+0000 7f4e7debb640 1 -- 172.21.15.84:0/1052445128 shutdown_connections 2023-10-16T11:17:16.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.071+0000 7f4e7debb640 1 -- 172.21.15.84:0/1052445128 wait complete. 2023-10-16T11:17:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.072+0000 7f4e7debb640 1 Processor -- start 2023-10-16T11:17:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.073+0000 7f4e7debb640 1 -- start start 2023-10-16T11:17:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.074+0000 7f4e7debb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e64005180 0x7f4e7814bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.074+0000 7f4e7debb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e780af3d0 0x7f4e780b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.074+0000 7f4e7debb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e780b3790 0x7f4e780c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.074+0000 7f4e7debb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780b2e30 con 0x7f4e780af3d0 2023-10-16T11:17:16.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.074+0000 7f4e7debb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780af920 con 0x7f4e64005180 2023-10-16T11:17:16.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.074+0000 7f4e7debb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780b2490 con 0x7f4e780b3790 2023-10-16T11:17:16.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.074+0000 7f4e6ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e780af3d0 0x7f4e780b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:16.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.074+0000 7f4e6ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e780af3d0 0x7f4e780b3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53978/0 (socket says 172.21.15.84:53978) 2023-10-16T11:17:16.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.074+0000 7f4e6ffff640 1 -- 172.21.15.84:0/3272383571 learned_addr learned my addr 172.21.15.84:0/3272383571 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:17:16.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.074+0000 7f4e6ffff640 1 -- 172.21.15.84:0/3272383571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e780b3790 msgr2=0x7f4e780c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:16.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.074+0000 7f4e6ffff640 1 --2- 172.21.15.84:0/3272383571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e780b3790 0x7f4e780c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.074+0000 7f4e6ffff640 1 -- 172.21.15.84:0/3272383571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e64005180 msgr2=0x7f4e7814bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:16.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.074+0000 7f4e6ffff640 1 --2- 172.21.15.84:0/3272383571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e64005180 0x7f4e7814bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.074+0000 7f4e6ffff640 1 -- 172.21.15.84:0/3272383571 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e780b2770 con 0x7f4e780af3d0 2023-10-16T11:17:16.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.075+0000 7f4e6ffff640 1 --2- 172.21.15.84:0/3272383571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e780af3d0 0x7f4e780b3250 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f4e6000e7f0 tx=0x7f4e6000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:16.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.075+0000 7f4e74ff9640 1 -- 172.21.15.84:0/3272383571 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6000c870 con 0x7f4e780af3d0 2023-10-16T11:17:16.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.075+0000 7f4e74ff9640 1 -- 172.21.15.84:0/3272383571 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6000c9d0 con 0x7f4e780af3d0 2023-10-16T11:17:16.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.075+0000 7f4e7debb640 1 -- 172.21.15.84:0/3272383571 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e780b2a50 con 0x7f4e780af3d0 2023-10-16T11:17:16.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.075+0000 7f4e74ff9640 1 -- 172.21.15.84:0/3272383571 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6000cd60 con 0x7f4e780af3d0 2023-10-16T11:17:16.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.075+0000 7f4e7debb640 1 -- 172.21.15.84:0/3272383571 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e78157bc0 con 0x7f4e780af3d0 2023-10-16T11:17:16.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.076+0000 7f4e7debb640 1 -- 172.21.15.84:0/3272383571 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e7804e160 con 0x7f4e780af3d0 2023-10-16T11:17:16.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.077+0000 7f4e74ff9640 1 -- 172.21.15.84:0/3272383571 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 111) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f4e6001d460 con 0x7f4e780af3d0 2023-10-16T11:17:16.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.078+0000 7f4e74ff9640 1 --2- 172.21.15.84:0/3272383571 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7f4e5806afe0 0x7f4e5806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.078+0000 7f4e74ff9640 1 -- 172.21.15.84:0/3272383571 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f4e60089910 con 0x7f4e780af3d0 2023-10-16T11:17:16.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.078+0000 7f4e777fe640 1 --2- 172.21.15.84:0/3272383571 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7f4e5806afe0 0x7f4e5806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:16.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.079+0000 7f4e777fe640 1 --2- 172.21.15.84:0/3272383571 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7f4e5806afe0 0x7f4e5806d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4e68006fd0 tx=0x7f4e6800d280 comp rx=0 tx=0).ready entity=mgr.10374 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:16.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.085+0000 7f4e74ff9640 1 -- 172.21.15.84:0/3272383571 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4e60054690 con 0x7f4e780af3d0 2023-10-16T11:17:16.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.299+0000 7f4e7debb640 1 -- 172.21.15.84:0/3272383571 --> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4e780b2600 con 0x7f4e5806afe0 2023-10-16T11:17:16.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.316+0000 7f4e74ff9640 1 -- 172.21.15.84:0/3272383571 <== mgr.10374 v2:172.21.15.84:6832/3993913200 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+375 (secure 0 0 0) 0x7f4e780b2600 con 0x7f4e5806afe0 2023-10-16T11:17:16.318 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:17:16.318 INFO:teuthology.orchestra.run.smithi084.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T11:06:38", "first": "2023-10-16T11:07:00", "last": "2023-10-16T11:17:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-10-16T11:17:16.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.319+0000 7f4e7debb640 1 -- 172.21.15.84:0/3272383571 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7f4e5806afe0 msgr2=0x7f4e5806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:16.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.319+0000 7f4e7debb640 1 --2- 172.21.15.84:0/3272383571 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7f4e5806afe0 0x7f4e5806d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4e68006fd0 tx=0x7f4e6800d280 comp rx=0 tx=0).stop 2023-10-16T11:17:16.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.319+0000 7f4e7debb640 1 -- 172.21.15.84:0/3272383571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e780af3d0 msgr2=0x7f4e780b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:16.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.319+0000 7f4e7debb640 1 --2- 172.21.15.84:0/3272383571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e780af3d0 0x7f4e780b3250 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f4e6000e7f0 tx=0x7f4e6000ecb0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.319+0000 7f4e7debb640 1 -- 172.21.15.84:0/3272383571 shutdown_connections 2023-10-16T11:17:16.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.319+0000 7f4e7debb640 1 --2- 172.21.15.84:0/3272383571 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7f4e5806afe0 0x7f4e5806d4a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.319+0000 7f4e7debb640 1 --2- 172.21.15.84:0/3272383571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e780b3790 0x7f4e780c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.319+0000 7f4e7debb640 1 --2- 172.21.15.84:0/3272383571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e780af3d0 0x7f4e780b3250 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.319+0000 7f4e7debb640 1 --2- 172.21.15.84:0/3272383571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e64005180 0x7f4e7814bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.319+0000 7f4e7debb640 1 -- 172.21.15.84:0/3272383571 >> 172.21.15.84:0/3272383571 conn(0x7f4e780ce9b0 msgr2=0x7f4e78159d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:16.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.319+0000 7f4e7debb640 1 -- 172.21.15.84:0/3272383571 shutdown_connections 2023-10-16T11:17:16.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.319+0000 7f4e7debb640 1 -- 172.21.15.84:0/3272383571 wait complete. 2023-10-16T11:17:16.338 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T11:06:38", "first": "2023-10-16T11:07:00", "last": "2023-10-16T11:17:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-10-16T11:17:16.339 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T11:06:38', 'first': '2023-10-16T11:07:00', 'last': '2023-10-16T11:17:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2023-10-16T11:17:16.339 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T11:17:16.340 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T11:17:16.340 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-16T11:17:16.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.541+0000 7f3648f83640 1 Processor -- start 2023-10-16T11:17:16.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.541+0000 7f3648f83640 1 -- start start 2023-10-16T11:17:16.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.543+0000 7f3648f83640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f36440b0b20 0x7f36441615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.543+0000 7f3648f83640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f364415d3f0 0x7f364415d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.543+0000 7f3648f83640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364415e3e0 con 0x7f36440cd0c0 2023-10-16T11:17:16.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.543+0000 7f3648f83640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36440cd4a0 con 0x7f36440b0b20 2023-10-16T11:17:16.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.543+0000 7f3648f83640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364415e100 con 0x7f364415d3f0 2023-10-16T11:17:16.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.543+0000 7f363bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f36440b0b20 0x7f36441615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:16.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.543+0000 7f364259c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f36440cd0c0 0x7f36440b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43988/0 (socket says 172.21.15.84:43988) 2023-10-16T11:17:16.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.543+0000 7f364259c640 1 -- 172.21.15.84:0/994154676 learned_addr learned my addr 172.21.15.84:0/994154676 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:17:16.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.544+0000 7f3642d9d640 1 --2- 172.21.15.84:0/994154676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f364415d3f0 0x7f364415d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:16.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.544+0000 7f363bfff640 1 -- 172.21.15.84:0/994154676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f364415d3f0 msgr2=0x7f364415d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:16.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.544+0000 7f363bfff640 1 --2- 172.21.15.84:0/994154676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f364415d3f0 0x7f364415d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.544+0000 7f363bfff640 1 -- 172.21.15.84:0/994154676 >> v1:172.21.15.84:6789/0 conn(0x7f36440cd0c0 legacy=0x7f36440b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:16.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.544+0000 7f363bfff640 1 -- 172.21.15.84:0/994154676 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3644160b10 con 0x7f36440b0b20 2023-10-16T11:17:16.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.544+0000 7f3642d9d640 1 --2- 172.21.15.84:0/994154676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f364415d3f0 0x7f364415d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:17:16.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.545+0000 7f363bfff640 1 --2- 172.21.15.84:0/994154676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36440b0b20 0x7f36441615f0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f362c0097a0 tx=0x7f362c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687cefee22e596a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:16.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.545+0000 7f3641d9b640 1 -- 172.21.15.84:0/994154676 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c031780 con 0x7f36440b0b20 2023-10-16T11:17:16.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.545+0000 7f3641d9b640 1 -- 172.21.15.84:0/994154676 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c0318e0 con 0x7f36440b0b20 2023-10-16T11:17:16.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.545+0000 7f3641d9b640 1 -- 172.21.15.84:0/994154676 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c031c70 con 0x7f36440b0b20 2023-10-16T11:17:16.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.546+0000 7f3648f83640 1 -- 172.21.15.84:0/994154676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36440b0b20 msgr2=0x7f36441615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:16.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.546+0000 7f3648f83640 1 --2- 172.21.15.84:0/994154676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36440b0b20 0x7f36441615f0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f362c0097a0 tx=0x7f362c02b820 comp rx=0 tx=0).stop 2023-10-16T11:17:16.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.546+0000 7f3648f83640 1 -- 172.21.15.84:0/994154676 shutdown_connections 2023-10-16T11:17:16.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.546+0000 7f3648f83640 1 --2- 172.21.15.84:0/994154676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f364415d3f0 0x7f364415d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.546+0000 7f3648f83640 1 --2- 172.21.15.84:0/994154676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36440b0b20 0x7f36441615f0 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.546+0000 7f3648f83640 1 -- 172.21.15.84:0/994154676 >> 172.21.15.84:0/994154676 conn(0x7f36440cb410 msgr2=0x7f36440cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:16.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.546+0000 7f3648f83640 1 -- 172.21.15.84:0/994154676 shutdown_connections 2023-10-16T11:17:16.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.546+0000 7f3648f83640 1 -- 172.21.15.84:0/994154676 wait complete. 2023-10-16T11:17:16.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.548+0000 7f3648f83640 1 Processor -- start 2023-10-16T11:17:16.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.548+0000 7f3648f83640 1 -- start start 2023-10-16T11:17:16.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.549+0000 7f3648f83640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36440b0b20 0x7f36440b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.549+0000 7f3648f83640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36440cd0c0 0x7f36440b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.549+0000 7f3648f83640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f364415d3f0 0x7f3644154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.549+0000 7f3648f83640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36441e6510 con 0x7f36440cd0c0 2023-10-16T11:17:16.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.549+0000 7f3648f83640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36441e67d0 con 0x7f36440b0b20 2023-10-16T11:17:16.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.549+0000 7f3648f83640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36441e6ab0 con 0x7f364415d3f0 2023-10-16T11:17:16.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.549+0000 7f363bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36440cd0c0 0x7f36440b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:16.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.549+0000 7f363bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36440cd0c0 0x7f36440b16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54004/0 (socket says 172.21.15.84:54004) 2023-10-16T11:17:16.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.549+0000 7f363bfff640 1 -- 172.21.15.84:0/961663966 learned_addr learned my addr 172.21.15.84:0/961663966 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:17:16.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.550+0000 7f363bfff640 1 -- 172.21.15.84:0/961663966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f364415d3f0 msgr2=0x7f3644154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:16.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.550+0000 7f3642d9d640 1 --2- 172.21.15.84:0/961663966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f364415d3f0 0x7f3644154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:16.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.550+0000 7f364259c640 1 --2- 172.21.15.84:0/961663966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36440b0b20 0x7f36440b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:16.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.550+0000 7f363bfff640 1 --2- 172.21.15.84:0/961663966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f364415d3f0 0x7f3644154ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.550+0000 7f363bfff640 1 -- 172.21.15.84:0/961663966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36440b0b20 msgr2=0x7f36440b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:16.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.550+0000 7f363bfff640 1 --2- 172.21.15.84:0/961663966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36440b0b20 0x7f36440b03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.550+0000 7f363bfff640 1 -- 172.21.15.84:0/961663966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c009400 con 0x7f36440cd0c0 2023-10-16T11:17:16.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.550+0000 7f363bfff640 1 --2- 172.21.15.84:0/961663966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36440cd0c0 0x7f36440b16d0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f362c034040 tx=0x7f362c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:16.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.550+0000 7f363affd640 1 -- 172.21.15.84:0/961663966 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c032960 con 0x7f36440cd0c0 2023-10-16T11:17:16.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.550+0000 7f363affd640 1 -- 172.21.15.84:0/961663966 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c032ac0 con 0x7f36440cd0c0 2023-10-16T11:17:16.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.550+0000 7f363affd640 1 -- 172.21.15.84:0/961663966 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c032e70 con 0x7f36440cd0c0 2023-10-16T11:17:16.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.550+0000 7f3648f83640 1 -- 172.21.15.84:0/961663966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36441e6d90 con 0x7f36440cd0c0 2023-10-16T11:17:16.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.551+0000 7f3648f83640 1 -- 172.21.15.84:0/961663966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36441e7350 con 0x7f36440cd0c0 2023-10-16T11:17:16.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.552+0000 7f3648f83640 1 -- 172.21.15.84:0/961663966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364404e160 con 0x7f36440cd0c0 2023-10-16T11:17:16.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.553+0000 7f363affd640 1 -- 172.21.15.84:0/961663966 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 111) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f362c03b3f0 con 0x7f36440cd0c0 2023-10-16T11:17:16.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.553+0000 7f363affd640 1 --2- 172.21.15.84:0/961663966 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7f362406b010 0x7f362406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.554+0000 7f363affd640 1 -- 172.21.15.84:0/961663966 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f362c0abe40 con 0x7f36440cd0c0 2023-10-16T11:17:16.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.554+0000 7f364259c640 1 --2- 172.21.15.84:0/961663966 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7f362406b010 0x7f362406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:16.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.554+0000 7f364259c640 1 --2- 172.21.15.84:0/961663966 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7f362406b010 0x7f362406d4d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3634005880 tx=0x7f3634009040 comp rx=0 tx=0).ready entity=mgr.10374 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:16.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.558+0000 7f363affd640 1 -- 172.21.15.84:0/961663966 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f362c076be0 con 0x7f36440cd0c0 2023-10-16T11:17:16.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.799+0000 7f3648f83640 1 -- 172.21.15.84:0/961663966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f36440b4910 con 0x7f36440cd0c0 2023-10-16T11:17:16.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.808+0000 7f363affd640 1 -- 172.21.15.84:0/961663966 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v111) v1 ==== 74+0+171237 (secure 0 0 0) 0x7f362c07a200 con 0x7f36440cd0c0 2023-10-16T11:17:16.812 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:17:16.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.812+0000 7f3648f83640 1 -- 172.21.15.84:0/961663966 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7f362406b010 msgr2=0x7f362406d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:16.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.812+0000 7f3648f83640 1 --2- 172.21.15.84:0/961663966 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7f362406b010 0x7f362406d4d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3634005880 tx=0x7f3634009040 comp rx=0 tx=0).stop 2023-10-16T11:17:16.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.812+0000 7f3648f83640 1 -- 172.21.15.84:0/961663966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36440cd0c0 msgr2=0x7f36440b16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:16.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.812+0000 7f3648f83640 1 --2- 172.21.15.84:0/961663966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36440cd0c0 0x7f36440b16d0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f362c034040 tx=0x7f362c0046c0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.812+0000 7f3648f83640 1 -- 172.21.15.84:0/961663966 shutdown_connections 2023-10-16T11:17:16.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.812+0000 7f3648f83640 1 --2- 172.21.15.84:0/961663966 >> [v2:172.21.15.84:6832/3993913200,v1:172.21.15.84:6833/3993913200] conn(0x7f362406b010 0x7f362406d4d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.812+0000 7f3648f83640 1 --2- 172.21.15.84:0/961663966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f364415d3f0 0x7f3644154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.812+0000 7f3648f83640 1 --2- 172.21.15.84:0/961663966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36440cd0c0 0x7f36440b16d0 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.812+0000 7f3648f83640 1 --2- 172.21.15.84:0/961663966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36440b0b20 0x7f36440b03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.812+0000 7f3648f83640 1 -- 172.21.15.84:0/961663966 >> 172.21.15.84:0/961663966 conn(0x7f36440cb410 msgr2=0x7f364415e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:16.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.812+0000 7f3648f83640 1 -- 172.21.15.84:0/961663966 shutdown_connections 2023-10-16T11:17:16.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:17:16.812+0000 7f3648f83640 1 -- 172.21.15.84:0/961663966 wait complete. 2023-10-16T11:17:16.832 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":111,"active_gid":10374,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3993913200},{"type":"v1","addr":"172.21.15.84:6833","nonce":3993913200}]},"active_addr":"172.21.15.84:6833/3993913200","active_change":"2023-10-16T11:06:37.826153+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":6413,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":185,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":2013731252}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":542522615}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":4095400472}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":878936990}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":1064178200}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":2359294858}]}]} 2023-10-16T11:17:16.836 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: y 2023-10-16T11:17:16.837 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-16T11:17:16.837 INFO:tasks.ceph.mgr.y:Stopping old one... 2023-10-16T11:17:16.837 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-16T11:17:16.838 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:17:16.918 INFO:tasks.ceph.mgr.y:Stopped 2023-10-16T11:17:16.919 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-16T11:17:16.927 INFO:tasks.ceph.mgr.y:Started 2023-10-16T11:17:17.497 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:17.495+0000 7fb59ca45040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:17:17.551 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:17.549+0000 7fb59ca45040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:17:17.997 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:17.995+0000 7fb59ca45040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:17:18.048 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:18.046+0000 7fb59ca45040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:17:18.098 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:18.096+0000 7fb59ca45040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:17:18.158 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:18.156+0000 7fb59ca45040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:17:18.207 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:18.205+0000 7fb59ca45040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:17:18.318 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:18.316+0000 7fb59ca45040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:17:18.380 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:18.378+0000 7fb59ca45040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:17:18.429 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:18.427+0000 7fb59ca45040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:17:18.485 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:18.483+0000 7fb59ca45040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:17:18.535 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:18.533+0000 7fb59ca45040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:17:18.914 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:18.912+0000 7fb59ca45040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:17:19.071 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:19.069+0000 7fb59ca45040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:17:19.177 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:19.175+0000 7fb59ca45040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:17:19.227 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:19.225+0000 7fb59ca45040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:17:19.333 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:19.331+0000 7fb59ca45040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:17:19.439 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:19.437+0000 7fb59ca45040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:17:19.593 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:19.591+0000 7fb59ca45040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:17:19.648 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:19.646+0000 7fb59ca45040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:17:19.768 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:19.766+0000 7fb59ca45040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:17:20.598 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:20.596+0000 7fb59ca45040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:17:20.894 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:20.892+0000 7fb59ca45040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:17:20.950 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:20.948+0000 7fb59ca45040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:17:21.064 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:21.062+0000 7fb59ca45040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:17:21.458 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:21.456+0000 7fb51b552640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:21.459 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:21.456+0000 7fb51b552640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:21.459 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:21.456+0000 7fb51b552640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:21.459 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:21.456+0000 7fb51b552640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:21.459 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:21.456+0000 7fb51b552640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:21.459 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:21.456+0000 7fb518d4d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:21.460 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:21.456+0000 7fb518d4d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:21.460 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:21.456+0000 7fb518d4d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:21.460 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:21.456+0000 7fb518d4d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:21.460 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:17:21.456+0000 7fb518d4d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:31.518 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:17:31.528 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:18:00.456 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:18:00.454+0000 7fb523da3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:18:00.457 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:18:00.454+0000 7fb523da3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:18:00.457 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:18:00.454+0000 7fb523da3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:18:00.457 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:18:00.454+0000 7fb523da3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:18:00.458 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:18:00.454+0000 7fb523da3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:18:01.665 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:18:01.675 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:18:31.828 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:18:31.838 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:19:02.050 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:19:02.060 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:19:32.197 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:19:32.206 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:20:02.343 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:20:02.353 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:20:32.610 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:20:32.620 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:21:02.872 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:21:02.882 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:21:33.017 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:21:33.026 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:22:03.397 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:22:03.407 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:22:16.928 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/test_restart 1m 2023-10-16T11:22:17.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.155+0000 7f0a3ea75640 1 Processor -- start 2023-10-16T11:22:17.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.156+0000 7f0a3ea75640 1 -- start start 2023-10-16T11:22:17.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.157+0000 7f0a3ea75640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3815ad70 0x7f0a38069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.157+0000 7f0a3ea75640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0a380c9ca0 0x7f0a3806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.157+0000 7f0a3ea75640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a380cb030 con 0x7f0a380ca3b0 2023-10-16T11:22:17.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.157+0000 7f0a3ea75640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a380c91c0 con 0x7f0a380c9ca0 2023-10-16T11:22:17.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.157+0000 7f0a3ea75640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380c94a0 con 0x7f0a3815ad70 2023-10-16T11:22:17.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.158+0000 7f0a3c811640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3815ad70 0x7f0a38069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.158+0000 7f0a3d012640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0a380ca3b0 0x7f0a380ca840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:43998/0 (socket says 172.21.15.84:43998) 2023-10-16T11:22:17.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.158+0000 7f0a3d012640 1 -- 172.21.15.84:0/3918864918 learned_addr learned my addr 172.21.15.84:0/3918864918 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:17.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.158+0000 7f0a37fff640 1 --2- 172.21.15.84:0/3918864918 >> v2:172.21.15.157:3300/0 conn(0x7f0a380c9ca0 0x7f0a3806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.159+0000 7f0a3c811640 1 -- 172.21.15.84:0/3918864918 >> v2:172.21.15.157:3300/0 conn(0x7f0a380c9ca0 msgr2=0x7f0a3806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:17.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.159+0000 7f0a3c811640 1 --2- 172.21.15.84:0/3918864918 >> v2:172.21.15.157:3300/0 conn(0x7f0a380c9ca0 0x7f0a3806a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.159+0000 7f0a3c811640 1 -- 172.21.15.84:0/3918864918 >> v1:172.21.15.157:6789/0 conn(0x7f0a380ca3b0 legacy=0x7f0a380ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.159+0000 7f0a3c811640 1 -- 172.21.15.84:0/3918864918 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a380c9780 con 0x7f0a3815ad70 2023-10-16T11:22:17.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.161+0000 7f0a3c811640 1 --2- 172.21.15.84:0/3918864918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3815ad70 0x7f0a38069da0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0a280097a0 tx=0x7f0a2802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cbdd376921574f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:17.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.161+0000 7f0a377fe640 1 -- 172.21.15.84:0/3918864918 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28031830 con 0x7f0a3815ad70 2023-10-16T11:22:17.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.161+0000 7f0a377fe640 1 -- 172.21.15.84:0/3918864918 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a28031990 con 0x7f0a3815ad70 2023-10-16T11:22:17.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.161+0000 7f0a377fe640 1 -- 172.21.15.84:0/3918864918 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28031d20 con 0x7f0a3815ad70 2023-10-16T11:22:17.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.162+0000 7f0a3ea75640 1 -- 172.21.15.84:0/3918864918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3815ad70 msgr2=0x7f0a38069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.162+0000 7f0a3ea75640 1 --2- 172.21.15.84:0/3918864918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3815ad70 0x7f0a38069da0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0a280097a0 tx=0x7f0a2802b8d0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.162+0000 7f0a3ea75640 1 -- 172.21.15.84:0/3918864918 shutdown_connections 2023-10-16T11:22:17.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.162+0000 7f0a3ea75640 1 --2- 172.21.15.84:0/3918864918 >> v2:172.21.15.157:3300/0 conn(0x7f0a380c9ca0 0x7f0a3806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.162+0000 7f0a3ea75640 1 --2- 172.21.15.84:0/3918864918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3815ad70 0x7f0a38069da0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.162+0000 7f0a3ea75640 1 -- 172.21.15.84:0/3918864918 >> 172.21.15.84:0/3918864918 conn(0x7f0a380ce9b0 msgr2=0x7f0a380cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:17.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.162+0000 7f0a3ea75640 1 -- 172.21.15.84:0/3918864918 shutdown_connections 2023-10-16T11:22:17.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.162+0000 7f0a3ea75640 1 -- 172.21.15.84:0/3918864918 wait complete. 2023-10-16T11:22:17.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.164+0000 7f0a3ea75640 1 Processor -- start 2023-10-16T11:22:17.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.164+0000 7f0a3ea75640 1 -- start start 2023-10-16T11:22:17.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.165+0000 7f0a3ea75640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a380c9ca0 0x7f0a380b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.165+0000 7f0a3ea75640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a380ca3b0 0x7f0a380ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.165+0000 7f0a3ea75640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3815ad70 0x7f0a380abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.165+0000 7f0a3ea75640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380b1130 con 0x7f0a380ca3b0 2023-10-16T11:22:17.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.165+0000 7f0a3ea75640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380b0520 con 0x7f0a380c9ca0 2023-10-16T11:22:17.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.165+0000 7f0a3ea75640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380b0800 con 0x7f0a3815ad70 2023-10-16T11:22:17.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.165+0000 7f0a37fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a380ca3b0 0x7f0a380ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.166+0000 7f0a37fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a380ca3b0 0x7f0a380ab6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53818/0 (socket says 172.21.15.84:53818) 2023-10-16T11:22:17.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.166+0000 7f0a37fff640 1 -- 172.21.15.84:0/49946432 learned_addr learned my addr 172.21.15.84:0/49946432 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:17.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.166+0000 7f0a3d012640 1 --2- 172.21.15.84:0/49946432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3815ad70 0x7f0a380abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.166+0000 7f0a3c811640 1 --2- 172.21.15.84:0/49946432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a380c9ca0 0x7f0a380b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.166+0000 7f0a37fff640 1 -- 172.21.15.84:0/49946432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3815ad70 msgr2=0x7f0a380abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.166+0000 7f0a37fff640 1 --2- 172.21.15.84:0/49946432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3815ad70 0x7f0a380abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.166+0000 7f0a37fff640 1 -- 172.21.15.84:0/49946432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a380c9ca0 msgr2=0x7f0a380b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.166+0000 7f0a37fff640 1 --2- 172.21.15.84:0/49946432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a380c9ca0 0x7f0a380b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.166+0000 7f0a37fff640 1 -- 172.21.15.84:0/49946432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a28009400 con 0x7f0a380ca3b0 2023-10-16T11:22:17.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.166+0000 7f0a37fff640 1 --2- 172.21.15.84:0/49946432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a380ca3b0 0x7f0a380ab6b0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00e9b0 tx=0x7f0a2c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:17.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.166+0000 7f0a357fa640 1 -- 172.21.15.84:0/49946432 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c009670 con 0x7f0a380ca3b0 2023-10-16T11:22:17.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.167+0000 7f0a357fa640 1 -- 172.21.15.84:0/49946432 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2c00c9a0 con 0x7f0a380ca3b0 2023-10-16T11:22:17.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.167+0000 7f0a357fa640 1 -- 172.21.15.84:0/49946432 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c00cd50 con 0x7f0a380ca3b0 2023-10-16T11:22:17.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.167+0000 7f0a3ea75640 1 -- 172.21.15.84:0/49946432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a380b0ae0 con 0x7f0a380ca3b0 2023-10-16T11:22:17.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.167+0000 7f0a3ea75640 1 -- 172.21.15.84:0/49946432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a381499b0 con 0x7f0a380ca3b0 2023-10-16T11:22:17.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.168+0000 7f0a3ea75640 1 -- 172.21.15.84:0/49946432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a380b12e0 con 0x7f0a380ca3b0 2023-10-16T11:22:17.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.169+0000 7f0a357fa640 1 -- 172.21.15.84:0/49946432 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 117) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f0a2c013470 con 0x7f0a380ca3b0 2023-10-16T11:22:17.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.170+0000 7f0a357fa640 1 --2- 172.21.15.84:0/49946432 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f0a2406afe0 0x7f0a2406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.170+0000 7f0a357fa640 1 -- 172.21.15.84:0/49946432 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f0a2c089d30 con 0x7f0a380ca3b0 2023-10-16T11:22:17.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.170+0000 7f0a3c811640 1 --2- 172.21.15.84:0/49946432 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f0a2406afe0 0x7f0a2406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.171+0000 7f0a3c811640 1 --2- 172.21.15.84:0/49946432 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f0a2406afe0 0x7f0a2406d4a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0a28036000 tx=0x7f0a2803c040 comp rx=0 tx=0).ready entity=mgr.10476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:17.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.175+0000 7f0a357fa640 1 -- 172.21.15.84:0/49946432 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0a380b12e0 con 0x7f0a380ca3b0 2023-10-16T11:22:17.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.396+0000 7f0a3ea75640 1 -- 172.21.15.84:0/49946432 --> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/test_restart", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a380ad4d0 con 0x7f0a2406afe0 2023-10-16T11:22:17.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.583+0000 7f0a357fa640 1 -- 172.21.15.84:0/49946432 <== mgr.10476 v2:172.21.15.84:6832/285590107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f0a28031b00 con 0x7f0a2406afe0 2023-10-16T11:22:17.585 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2023-10-16T11:22:17.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.586+0000 7f0a3ea75640 1 -- 172.21.15.84:0/49946432 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f0a2406afe0 msgr2=0x7f0a2406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.586+0000 7f0a3ea75640 1 --2- 172.21.15.84:0/49946432 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f0a2406afe0 0x7f0a2406d4a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0a28036000 tx=0x7f0a2803c040 comp rx=0 tx=0).stop 2023-10-16T11:22:17.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.586+0000 7f0a3ea75640 1 -- 172.21.15.84:0/49946432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a380ca3b0 msgr2=0x7f0a380ab6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.586+0000 7f0a3ea75640 1 --2- 172.21.15.84:0/49946432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a380ca3b0 0x7f0a380ab6b0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00e9b0 tx=0x7f0a2c00ee70 comp rx=0 tx=0).stop 2023-10-16T11:22:17.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.586+0000 7f0a3ea75640 1 -- 172.21.15.84:0/49946432 shutdown_connections 2023-10-16T11:22:17.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.586+0000 7f0a3ea75640 1 --2- 172.21.15.84:0/49946432 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f0a2406afe0 0x7f0a2406d4a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.586+0000 7f0a3ea75640 1 --2- 172.21.15.84:0/49946432 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a3815ad70 0x7f0a380abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.586+0000 7f0a3ea75640 1 --2- 172.21.15.84:0/49946432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a380ca3b0 0x7f0a380ab6b0 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.586+0000 7f0a3ea75640 1 --2- 172.21.15.84:0/49946432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a380c9ca0 0x7f0a380b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.586+0000 7f0a3ea75640 1 -- 172.21.15.84:0/49946432 >> 172.21.15.84:0/49946432 conn(0x7f0a380ce9b0 msgr2=0x7f0a3815b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:17.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.586+0000 7f0a3ea75640 1 -- 172.21.15.84:0/49946432 shutdown_connections 2023-10-16T11:22:17.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.587+0000 7f0a3ea75640 1 -- 172.21.15.84:0/49946432 wait complete. 2023-10-16T11:22:17.610 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2023-10-16T11:22:17.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:17.612 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-16T11:22:17.672 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_07_00_UTC 2023-10-16T11:22:17.673 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_08_00_UTC 2023-10-16T11:22:17.673 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_09_00_UTC 2023-10-16T11:22:17.674 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_10_00_UTC 2023-10-16T11:22:17.674 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_11_00_UTC 2023-10-16T11:22:17.674 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_12_00_UTC 2023-10-16T11:22:17.674 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_13_00_UTC 2023-10-16T11:22:17.675 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_14_00_UTC 2023-10-16T11:22:17.675 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_15_00_UTC 2023-10-16T11:22:17.675 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_16_00_UTC 2023-10-16T11:22:17.675 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_17_00_UTC 2023-10-16T11:22:17.676 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:22:17.676 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_19_00_UTC 2023-10-16T11:22:17.676 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_20_00_UTC 2023-10-16T11:22:17.676 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_21_00_UTC 2023-10-16T11:22:17.677 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_22_00_UTC 2023-10-16T11:22:17.678 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-11_07_00_UTC', 'scheduled-2023-10-16-11_08_00_UTC', 'scheduled-2023-10-16-11_09_00_UTC', 'scheduled-2023-10-16-11_10_00_UTC', 'scheduled-2023-10-16-11_11_00_UTC', 'scheduled-2023-10-16-11_12_00_UTC', 'scheduled-2023-10-16-11_13_00_UTC', 'scheduled-2023-10-16-11_14_00_UTC', 'scheduled-2023-10-16-11_15_00_UTC', 'scheduled-2023-10-16-11_16_00_UTC', 'scheduled-2023-10-16-11_17_00_UTC', 'scheduled-2023-10-16-11_18_00_UTC', 'scheduled-2023-10-16-11_19_00_UTC', 'scheduled-2023-10-16-11_20_00_UTC', 'scheduled-2023-10-16-11_21_00_UTC', 'scheduled-2023-10-16-11_22_00_UTC'] 2023-10-16T11:22:17.679 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/test_restart 2023-10-16T11:22:17.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.896+0000 7f4703ccf640 1 Processor -- start 2023-10-16T11:22:17.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.896+0000 7f4703ccf640 1 -- start start 2023-10-16T11:22:17.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.898+0000 7f4703ccf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f46fc15ad70 0x7f46fc069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.898+0000 7f4703ccf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46fc0c9ca0 0x7f46fc06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.898+0000 7f4703ccf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46fc0cb030 con 0x7f46fc0ca3b0 2023-10-16T11:22:17.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.898+0000 7f4703ccf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46fc0c91c0 con 0x7f46fc15ad70 2023-10-16T11:22:17.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.898+0000 7f4703ccf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0c94a0 con 0x7f46fc0c9ca0 2023-10-16T11:22:17.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.898+0000 7f470226c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f46fc0ca3b0 0x7f46fc0ca840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40834/0 (socket says 172.21.15.84:40834) 2023-10-16T11:22:17.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.898+0000 7f470226c640 1 -- 172.21.15.84:0/4255493098 learned_addr learned my addr 172.21.15.84:0/4255493098 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:17.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.899+0000 7f470126a640 1 --2- 172.21.15.84:0/4255493098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46fc0c9ca0 0x7f46fc06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.899+0000 7f4701a6b640 1 --2- 172.21.15.84:0/4255493098 >> v2:172.21.15.157:3300/0 conn(0x7f46fc15ad70 0x7f46fc069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.899+0000 7f4700a69640 1 -- 172.21.15.84:0/4255493098 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3363367555 0 0) 0x7f46fc0cb030 con 0x7f46fc0ca3b0 2023-10-16T11:22:17.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.899+0000 7f4700a69640 1 -- 172.21.15.84:0/4255493098 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46e8003520 con 0x7f46fc0ca3b0 2023-10-16T11:22:17.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.899+0000 7f4700a69640 1 -- 172.21.15.84:0/4255493098 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2708016861 0 0) 0x7f46e8003520 con 0x7f46fc0ca3b0 2023-10-16T11:22:17.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.899+0000 7f4700a69640 1 -- 172.21.15.84:0/4255493098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46fc0c9ca0 msgr2=0x7f46fc06a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:17.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.899+0000 7f4700a69640 1 --2- 172.21.15.84:0/4255493098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46fc0c9ca0 0x7f46fc06a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.899+0000 7f4700a69640 1 -- 172.21.15.84:0/4255493098 >> v2:172.21.15.157:3300/0 conn(0x7f46fc15ad70 msgr2=0x7f46fc069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:17.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.899+0000 7f4700a69640 1 --2- 172.21.15.84:0/4255493098 >> v2:172.21.15.157:3300/0 conn(0x7f46fc15ad70 0x7f46fc069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.899+0000 7f4700a69640 1 -- 172.21.15.84:0/4255493098 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46fc0c9780 con 0x7f46fc0ca3b0 2023-10-16T11:22:17.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.899+0000 7f4700a69640 1 -- 172.21.15.84:0/4255493098 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f46f40019f0 con 0x7f46fc0ca3b0 2023-10-16T11:22:17.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.899+0000 7f4700a69640 1 -- 172.21.15.84:0/4255493098 >> v1:172.21.15.84:6789/0 conn(0x7f46fc0ca3b0 legacy=0x7f46fc0ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.899+0000 7f4700a69640 1 --2- 172.21.15.84:0/4255493098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46fc0c9ca0 0x7f46e8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.900+0000 7f4700a69640 1 --2- 172.21.15.84:0/4255493098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46e8003f40 0x7f46e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.900+0000 7f4700a69640 1 --2- 172.21.15.84:0/4255493098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46e8004d00 0x7f46e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.900+0000 7f4700a69640 1 -- 172.21.15.84:0/4255493098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0c91c0 con 0x7f46e8003f40 2023-10-16T11:22:17.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.900+0000 7f4700a69640 1 -- 172.21.15.84:0/4255493098 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0c94a0 con 0x7f46e8004d00 2023-10-16T11:22:17.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.900+0000 7f4700a69640 1 -- 172.21.15.84:0/4255493098 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e800b980 con 0x7f46fc0c9ca0 2023-10-16T11:22:17.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.900+0000 7f470126a640 1 --2- 172.21.15.84:0/4255493098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46e8003f40 0x7f46e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.900+0000 7f4701a6b640 1 --2- 172.21.15.84:0/4255493098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46fc0c9ca0 0x7f46e8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.900+0000 7f470126a640 1 -- 172.21.15.84:0/4255493098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46fc0c9ca0 msgr2=0x7f46e8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:17.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.900+0000 7f470126a640 1 --2- 172.21.15.84:0/4255493098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46fc0c9ca0 0x7f46e8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.900+0000 7f470226c640 1 --2- 172.21.15.84:0/4255493098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46e8004d00 0x7f46e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.900+0000 7f470126a640 1 -- 172.21.15.84:0/4255493098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46e8004d00 msgr2=0x7f46e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:17.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.900+0000 7f470126a640 1 --2- 172.21.15.84:0/4255493098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46e8004d00 0x7f46e8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.900+0000 7f470126a640 1 -- 172.21.15.84:0/4255493098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e8003520 con 0x7f46e8003f40 2023-10-16T11:22:17.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.900+0000 7f4701a6b640 1 --2- 172.21.15.84:0/4255493098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46fc0c9ca0 0x7f46e8003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:17.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.901+0000 7f470126a640 1 --2- 172.21.15.84:0/4255493098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46e8003f40 0x7f46e80047c0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009af0 tx=0x7f46e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9de5221be7b32a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:17.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.902+0000 7f4700a69640 1 -- 172.21.15.84:0/4255493098 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec02c6b0 con 0x7f46e8003f40 2023-10-16T11:22:17.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.902+0000 7f4700a69640 1 -- 172.21.15.84:0/4255493098 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec02c810 con 0x7f46e8003f40 2023-10-16T11:22:17.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.902+0000 7f4700a69640 1 -- 172.21.15.84:0/4255493098 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec02cba0 con 0x7f46e8003f40 2023-10-16T11:22:17.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.902+0000 7f4703ccf640 1 -- 172.21.15.84:0/4255493098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46e8003f40 msgr2=0x7f46e80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.902+0000 7f4703ccf640 1 --2- 172.21.15.84:0/4255493098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46e8003f40 0x7f46e80047c0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009af0 tx=0x7f46e80057f0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.902+0000 7f470226c640 1 -- 172.21.15.84:0/4255493098 reap_dead start 2023-10-16T11:22:17.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.902+0000 7f4703ccf640 1 -- 172.21.15.84:0/4255493098 shutdown_connections 2023-10-16T11:22:17.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.902+0000 7f4703ccf640 1 -- 172.21.15.84:0/4255493098 >> 172.21.15.84:0/4255493098 conn(0x7f46fc0ce9b0 msgr2=0x7f46fc0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:17.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.903+0000 7f4703ccf640 1 -- 172.21.15.84:0/4255493098 shutdown_connections 2023-10-16T11:22:17.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.903+0000 7f4703ccf640 1 -- 172.21.15.84:0/4255493098 wait complete. 2023-10-16T11:22:17.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.904+0000 7f4703ccf640 1 Processor -- start 2023-10-16T11:22:17.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.904+0000 7f4703ccf640 1 -- start start 2023-10-16T11:22:17.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.905+0000 7f4703ccf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46fc158c10 0x7f46fc0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.905+0000 7f4703ccf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46fc1f23b0 0x7f46fc0b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.905+0000 7f4703ccf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46fc1593c0 0x7f46fc0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.905+0000 7f4703ccf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0c7e80 con 0x7f46fc158c10 2023-10-16T11:22:17.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.905+0000 7f4703ccf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0c8160 con 0x7f46fc1593c0 2023-10-16T11:22:17.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.905+0000 7f4703ccf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0c8440 con 0x7f46fc1f23b0 2023-10-16T11:22:17.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.906+0000 7f4701a6b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46fc158c10 0x7f46fc0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.906+0000 7f4701a6b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46fc158c10 0x7f46fc0ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53826/0 (socket says 172.21.15.84:53826) 2023-10-16T11:22:17.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.906+0000 7f4701a6b640 1 -- 172.21.15.84:0/790676716 learned_addr learned my addr 172.21.15.84:0/790676716 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:17.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.906+0000 7f470226c640 1 --2- 172.21.15.84:0/790676716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46fc1593c0 0x7f46fc0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.906+0000 7f4701a6b640 1 -- 172.21.15.84:0/790676716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46fc1f23b0 msgr2=0x7f46fc0b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.906+0000 7f4701a6b640 1 --2- 172.21.15.84:0/790676716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46fc1f23b0 0x7f46fc0b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.906+0000 7f4701a6b640 1 -- 172.21.15.84:0/790676716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46fc1593c0 msgr2=0x7f46fc0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.906+0000 7f4701a6b640 1 --2- 172.21.15.84:0/790676716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46fc1593c0 0x7f46fc0b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.906+0000 7f4701a6b640 1 -- 172.21.15.84:0/790676716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46fc0c8720 con 0x7f46fc158c10 2023-10-16T11:22:17.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.906+0000 7f4701a6b640 1 --2- 172.21.15.84:0/790676716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46fc158c10 0x7f46fc0ac890 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f46f80109d0 tx=0x7f46f8010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:17.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.907+0000 7f46f27fc640 1 -- 172.21.15.84:0/790676716 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f800ec20 con 0x7f46fc158c10 2023-10-16T11:22:17.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.907+0000 7f46f27fc640 1 -- 172.21.15.84:0/790676716 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f800ed80 con 0x7f46fc158c10 2023-10-16T11:22:17.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.907+0000 7f46f27fc640 1 -- 172.21.15.84:0/790676716 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f801c620 con 0x7f46fc158c10 2023-10-16T11:22:17.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.907+0000 7f4703ccf640 1 -- 172.21.15.84:0/790676716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46fc0acdd0 con 0x7f46fc158c10 2023-10-16T11:22:17.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.907+0000 7f4703ccf640 1 -- 172.21.15.84:0/790676716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46fc1591d0 con 0x7f46fc158c10 2023-10-16T11:22:17.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.908+0000 7f4703ccf640 1 -- 172.21.15.84:0/790676716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46fc04e160 con 0x7f46fc158c10 2023-10-16T11:22:17.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.909+0000 7f46f27fc640 1 -- 172.21.15.84:0/790676716 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 117) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f46f801c780 con 0x7f46fc158c10 2023-10-16T11:22:17.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.910+0000 7f46f27fc640 1 --2- 172.21.15.84:0/790676716 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f46e006b010 0x7f46e006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.910+0000 7f46f27fc640 1 -- 172.21.15.84:0/790676716 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f46f808a100 con 0x7f46fc158c10 2023-10-16T11:22:17.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.910+0000 7f470126a640 1 --2- 172.21.15.84:0/790676716 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f46e006b010 0x7f46e006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.911+0000 7f470126a640 1 --2- 172.21.15.84:0/790676716 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f46e006b010 0x7f46e006d4d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f46ec036000 tx=0x7f46ec02e000 comp rx=0 tx=0).ready entity=mgr.10476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:17.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:17.914+0000 7f46f27fc640 1 -- 172.21.15.84:0/790676716 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f46f8054e70 con 0x7f46fc158c10 2023-10-16T11:22:18.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.130+0000 7f4703ccf640 1 -- 172.21.15.84:0/790676716 --> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f46fc06a2e0 con 0x7f46e006b010 2023-10-16T11:22:18.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.147+0000 7f46f27fc640 1 -- 172.21.15.84:0/790676716 <== mgr.10476 v2:172.21.15.84:6832/285590107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+376 (secure 0 0 0) 0x7f46ec004040 con 0x7f46e006b010 2023-10-16T11:22:18.150 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:18.150 INFO:teuthology.orchestra.run.smithi084.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T11:06:38", "first": "2023-10-16T11:07:00", "last": "2023-10-16T11:22:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-10-16T11:22:18.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.150+0000 7f4703ccf640 1 -- 172.21.15.84:0/790676716 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f46e006b010 msgr2=0x7f46e006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.150+0000 7f4703ccf640 1 --2- 172.21.15.84:0/790676716 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f46e006b010 0x7f46e006d4d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f46ec036000 tx=0x7f46ec02e000 comp rx=0 tx=0).stop 2023-10-16T11:22:18.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.150+0000 7f4703ccf640 1 -- 172.21.15.84:0/790676716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46fc158c10 msgr2=0x7f46fc0ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.150+0000 7f4703ccf640 1 --2- 172.21.15.84:0/790676716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46fc158c10 0x7f46fc0ac890 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f46f80109d0 tx=0x7f46f8010e90 comp rx=0 tx=0).stop 2023-10-16T11:22:18.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.150+0000 7f4703ccf640 1 -- 172.21.15.84:0/790676716 shutdown_connections 2023-10-16T11:22:18.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.150+0000 7f4703ccf640 1 --2- 172.21.15.84:0/790676716 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f46e006b010 0x7f46e006d4d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.150+0000 7f4703ccf640 1 --2- 172.21.15.84:0/790676716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46fc1593c0 0x7f46fc0b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.150+0000 7f4703ccf640 1 --2- 172.21.15.84:0/790676716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46fc1f23b0 0x7f46fc0b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.150+0000 7f4703ccf640 1 --2- 172.21.15.84:0/790676716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46fc158c10 0x7f46fc0ac890 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.150+0000 7f4703ccf640 1 -- 172.21.15.84:0/790676716 >> 172.21.15.84:0/790676716 conn(0x7f46fc0ce9b0 msgr2=0x7f46fc0afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:18.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.151+0000 7f4703ccf640 1 -- 172.21.15.84:0/790676716 shutdown_connections 2023-10-16T11:22:18.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.151+0000 7f4703ccf640 1 -- 172.21.15.84:0/790676716 wait complete. 2023-10-16T11:22:18.170 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T11:06:38", "first": "2023-10-16T11:07:00", "last": "2023-10-16T11:22:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-10-16T11:22:18.170 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T11:06:38', 'first': '2023-10-16T11:07:00', 'last': '2023-10-16T11:22:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2023-10-16T11:22:18.171 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 16 2023-10-16T11:22:18.172 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 16 2023-10-16T11:22:18.172 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/test_restart 1m 2023-10-16T11:22:18.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.387+0000 7f0710c83640 1 Processor -- start 2023-10-16T11:22:18.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.387+0000 7f0710c83640 1 -- start start 2023-10-16T11:22:18.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.389+0000 7f0710c83640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f070c159e90 0x7f070c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.389+0000 7f0710c83640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f070c06c140 0x7f070c06c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.389+0000 7f0710c83640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f070c06cad0 con 0x7f070c159ab0 2023-10-16T11:22:18.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.389+0000 7f0710c83640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f070c15d890 con 0x7f070c06c140 2023-10-16T11:22:18.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.389+0000 7f0710c83640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f070c070360 con 0x7f070c159e90 2023-10-16T11:22:18.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.389+0000 7f070a59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f070c159ab0 0x7f070c06fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40846/0 (socket says 172.21.15.84:40846) 2023-10-16T11:22:18.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.389+0000 7f070a59c640 1 -- 172.21.15.84:0/2333201814 learned_addr learned my addr 172.21.15.84:0/2333201814 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:18.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.389+0000 7f070ad9d640 1 --2- 172.21.15.84:0/2333201814 >> v2:172.21.15.84:3300/0 conn(0x7f070c06c140 0x7f070c06c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.389+0000 7f0703fff640 1 --2- 172.21.15.84:0/2333201814 >> v2:172.21.15.157:3300/0 conn(0x7f070c159e90 0x7f070c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.390+0000 7f0709d9b640 1 -- 172.21.15.84:0/2333201814 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203291796 0 0) 0x7f06fc001110 con 0x7f070c159ab0 2023-10-16T11:22:18.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.390+0000 7f0709d9b640 1 -- 172.21.15.84:0/2333201814 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06f8003520 con 0x7f070c159ab0 2023-10-16T11:22:18.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.390+0000 7f070ad9d640 1 -- 172.21.15.84:0/2333201814 >> v2:172.21.15.157:3300/0 conn(0x7f070c159e90 msgr2=0x7f070c0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:18.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.390+0000 7f070ad9d640 1 --2- 172.21.15.84:0/2333201814 >> v2:172.21.15.157:3300/0 conn(0x7f070c159e90 0x7f070c0ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.390+0000 7f070ad9d640 1 -- 172.21.15.84:0/2333201814 >> v1:172.21.15.84:6789/0 conn(0x7f070c159ab0 legacy=0x7f070c06fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.390+0000 7f070ad9d640 1 -- 172.21.15.84:0/2333201814 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c0cd1f0 con 0x7f070c06c140 2023-10-16T11:22:18.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.391+0000 7f070ad9d640 1 --2- 172.21.15.84:0/2333201814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f070c06c140 0x7f070c06c590 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f070c06d900 tx=0x7f070402bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe230169dfdb7d75 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:18.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.391+0000 7f0709d9b640 1 -- 172.21.15.84:0/2333201814 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070402cb50 con 0x7f070c06c140 2023-10-16T11:22:18.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.391+0000 7f0709d9b640 1 -- 172.21.15.84:0/2333201814 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070402ccb0 con 0x7f070c06c140 2023-10-16T11:22:18.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.391+0000 7f0709d9b640 1 -- 172.21.15.84:0/2333201814 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0704034660 con 0x7f070c06c140 2023-10-16T11:22:18.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.392+0000 7f0710c83640 1 -- 172.21.15.84:0/2333201814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f070c06c140 msgr2=0x7f070c06c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.392+0000 7f0710c83640 1 --2- 172.21.15.84:0/2333201814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f070c06c140 0x7f070c06c590 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f070c06d900 tx=0x7f070402bda0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.392+0000 7f0710c83640 1 -- 172.21.15.84:0/2333201814 shutdown_connections 2023-10-16T11:22:18.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.392+0000 7f0710c83640 1 --2- 172.21.15.84:0/2333201814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f070c06c140 0x7f070c06c590 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.392+0000 7f0710c83640 1 --2- 172.21.15.84:0/2333201814 >> v2:172.21.15.157:3300/0 conn(0x7f070c159e90 0x7f070c0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.392+0000 7f0710c83640 1 -- 172.21.15.84:0/2333201814 >> 172.21.15.84:0/2333201814 conn(0x7f070c0ce9b0 msgr2=0x7f070c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:18.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.392+0000 7f0710c83640 1 -- 172.21.15.84:0/2333201814 shutdown_connections 2023-10-16T11:22:18.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.392+0000 7f0710c83640 1 -- 172.21.15.84:0/2333201814 wait complete. 2023-10-16T11:22:18.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.394+0000 7f0710c83640 1 Processor -- start 2023-10-16T11:22:18.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.394+0000 7f0710c83640 1 -- start start 2023-10-16T11:22:18.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.395+0000 7f0710c83640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f070c06c140 0x7f070c155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.395+0000 7f0710c83640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f070c159ab0 0x7f070c070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.395+0000 7f0710c83640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f070c159e90 0x7f070c070a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.395+0000 7f0710c83640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c0c98c0 con 0x7f070c06c140 2023-10-16T11:22:18.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.395+0000 7f0710c83640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c0b0000 con 0x7f070c159e90 2023-10-16T11:22:18.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.395+0000 7f0710c83640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070c157620 con 0x7f070c159ab0 2023-10-16T11:22:18.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.395+0000 7f070a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f070c06c140 0x7f070c155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.395+0000 7f070a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f070c06c140 0x7f070c155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53840/0 (socket says 172.21.15.84:53840) 2023-10-16T11:22:18.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.395+0000 7f070a59c640 1 -- 172.21.15.84:0/315014426 learned_addr learned my addr 172.21.15.84:0/315014426 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:18.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.395+0000 7f070a59c640 1 -- 172.21.15.84:0/315014426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f070c159ab0 msgr2=0x7f070c070500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.395+0000 7f070a59c640 1 --2- 172.21.15.84:0/315014426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f070c159ab0 0x7f070c070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.395+0000 7f070a59c640 1 -- 172.21.15.84:0/315014426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f070c159e90 msgr2=0x7f070c070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.395+0000 7f070a59c640 1 --2- 172.21.15.84:0/315014426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f070c159e90 0x7f070c070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.395+0000 7f070a59c640 1 -- 172.21.15.84:0/315014426 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0704009a70 con 0x7f070c06c140 2023-10-16T11:22:18.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.396+0000 7f070a59c640 1 --2- 172.21.15.84:0/315014426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f070c06c140 0x7f070c155a70 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00c700 tx=0x7f06fc00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:18.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.396+0000 7f0702ffd640 1 -- 172.21.15.84:0/315014426 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc005ce0 con 0x7f070c06c140 2023-10-16T11:22:18.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.396+0000 7f0702ffd640 1 -- 172.21.15.84:0/315014426 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06fc005e40 con 0x7f070c06c140 2023-10-16T11:22:18.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.396+0000 7f0702ffd640 1 -- 172.21.15.84:0/315014426 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc007290 con 0x7f070c06c140 2023-10-16T11:22:18.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.396+0000 7f0710c83640 1 -- 172.21.15.84:0/315014426 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070c15a3d0 con 0x7f070c06c140 2023-10-16T11:22:18.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.396+0000 7f0710c83640 1 -- 172.21.15.84:0/315014426 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f070c15a990 con 0x7f070c06c140 2023-10-16T11:22:18.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.397+0000 7f0710c83640 1 -- 172.21.15.84:0/315014426 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070c0adf70 con 0x7f070c06c140 2023-10-16T11:22:18.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.398+0000 7f0702ffd640 1 -- 172.21.15.84:0/315014426 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 117) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f06fc0074b0 con 0x7f070c06c140 2023-10-16T11:22:18.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.399+0000 7f0702ffd640 1 --2- 172.21.15.84:0/315014426 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f06ec06afe0 0x7f06ec06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.399+0000 7f0702ffd640 1 -- 172.21.15.84:0/315014426 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f06fc088ff0 con 0x7f070c06c140 2023-10-16T11:22:18.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.399+0000 7f0703fff640 1 --2- 172.21.15.84:0/315014426 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f06ec06afe0 0x7f06ec06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.400+0000 7f0703fff640 1 --2- 172.21.15.84:0/315014426 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f06ec06afe0 0x7f06ec06d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f070c155fb0 tx=0x7f06f4008040 comp rx=0 tx=0).ready entity=mgr.10476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:18.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.404+0000 7f0702ffd640 1 -- 172.21.15.84:0/315014426 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f06fc057680 con 0x7f070c06c140 2023-10-16T11:22:18.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.617+0000 7f0710c83640 1 -- 172.21.15.84:0/315014426 --> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/test_restart", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f070c0cd7b0 con 0x7f06ec06afe0 2023-10-16T11:22:18.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.821+0000 7f0702ffd640 1 -- 172.21.15.84:0/315014426 <== mgr.10476 v2:172.21.15.84:6832/285590107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7f06f4009e40 con 0x7f06ec06afe0 2023-10-16T11:22:18.824 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path /snap_test_dir1/test_restart 2023-10-16T11:22:18.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.824+0000 7f0710c83640 1 -- 172.21.15.84:0/315014426 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f06ec06afe0 msgr2=0x7f06ec06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.824+0000 7f0710c83640 1 --2- 172.21.15.84:0/315014426 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f06ec06afe0 0x7f06ec06d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f070c155fb0 tx=0x7f06f4008040 comp rx=0 tx=0).stop 2023-10-16T11:22:18.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.824+0000 7f0710c83640 1 -- 172.21.15.84:0/315014426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f070c06c140 msgr2=0x7f070c155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.824+0000 7f0710c83640 1 --2- 172.21.15.84:0/315014426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f070c06c140 0x7f070c155a70 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00c700 tx=0x7f06fc00cb00 comp rx=0 tx=0).stop 2023-10-16T11:22:18.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.825+0000 7f0710c83640 1 -- 172.21.15.84:0/315014426 shutdown_connections 2023-10-16T11:22:18.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.825+0000 7f0710c83640 1 --2- 172.21.15.84:0/315014426 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f06ec06afe0 0x7f06ec06d4a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.825+0000 7f0710c83640 1 --2- 172.21.15.84:0/315014426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f070c159e90 0x7f070c070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.825+0000 7f0710c83640 1 --2- 172.21.15.84:0/315014426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f070c159ab0 0x7f070c070500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.825+0000 7f0710c83640 1 --2- 172.21.15.84:0/315014426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f070c06c140 0x7f070c155a70 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.825+0000 7f0710c83640 1 -- 172.21.15.84:0/315014426 >> 172.21.15.84:0/315014426 conn(0x7f070c0ce9b0 msgr2=0x7f070c158f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:18.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.825+0000 7f0710c83640 1 -- 172.21.15.84:0/315014426 shutdown_connections 2023-10-16T11:22:18.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:18.825+0000 7f0710c83640 1 -- 172.21.15.84:0/315014426 wait complete. 2023-10-16T11:22:18.845 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2023-10-16T11:22:18.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:18.846 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-16T11:22:18.902 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_07_00_UTC 2023-10-16T11:22:18.902 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_08_00_UTC 2023-10-16T11:22:18.902 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_09_00_UTC 2023-10-16T11:22:18.903 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_10_00_UTC 2023-10-16T11:22:18.903 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_11_00_UTC 2023-10-16T11:22:18.903 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_12_00_UTC 2023-10-16T11:22:18.903 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_13_00_UTC 2023-10-16T11:22:18.903 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_14_00_UTC 2023-10-16T11:22:18.904 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_15_00_UTC 2023-10-16T11:22:18.904 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_16_00_UTC 2023-10-16T11:22:18.904 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_17_00_UTC 2023-10-16T11:22:18.904 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:22:18.904 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_19_00_UTC 2023-10-16T11:22:18.905 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_20_00_UTC 2023-10-16T11:22:18.905 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_21_00_UTC 2023-10-16T11:22:18.905 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_22_00_UTC 2023-10-16T11:22:18.906 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_07_00_UTC 2023-10-16T11:22:18.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:18.907 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_07_00_UTC) 2023-10-16T11:22:18.971 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_08_00_UTC 2023-10-16T11:22:18.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:18.972 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_08_00_UTC) 2023-10-16T11:22:19.041 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_09_00_UTC 2023-10-16T11:22:19.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:19.041 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_09_00_UTC) 2023-10-16T11:22:19.112 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_10_00_UTC 2023-10-16T11:22:19.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:19.113 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_10_00_UTC) 2023-10-16T11:22:19.183 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_11_00_UTC 2023-10-16T11:22:19.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:19.183 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_11_00_UTC) 2023-10-16T11:22:19.255 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_12_00_UTC 2023-10-16T11:22:19.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:19.256 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_12_00_UTC) 2023-10-16T11:22:19.324 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_13_00_UTC 2023-10-16T11:22:19.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:19.325 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_13_00_UTC) 2023-10-16T11:22:19.396 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_14_00_UTC 2023-10-16T11:22:19.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:19.396 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_14_00_UTC) 2023-10-16T11:22:19.465 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_15_00_UTC 2023-10-16T11:22:19.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:19.466 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_15_00_UTC) 2023-10-16T11:22:19.534 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_16_00_UTC 2023-10-16T11:22:19.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:19.535 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_16_00_UTC) 2023-10-16T11:22:19.605 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_17_00_UTC 2023-10-16T11:22:19.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:19.606 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_17_00_UTC) 2023-10-16T11:22:19.672 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:22:19.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:19.673 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_18_00_UTC) 2023-10-16T11:22:19.745 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_19_00_UTC 2023-10-16T11:22:19.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:19.746 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_19_00_UTC) 2023-10-16T11:22:19.814 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_20_00_UTC 2023-10-16T11:22:19.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:19.815 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_20_00_UTC) 2023-10-16T11:22:19.882 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_21_00_UTC 2023-10-16T11:22:19.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:19.883 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_21_00_UTC) 2023-10-16T11:22:19.954 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_22_00_UTC 2023-10-16T11:22:19.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:19.955 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_22_00_UTC) 2023-10-16T11:22:20.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:20.023 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2023-10-16T11:22:20.084 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T11:22:20.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.302+0000 7f1241202640 1 Processor -- start 2023-10-16T11:22:20.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.302+0000 7f1241202640 1 -- start start 2023-10-16T11:22:20.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.303+0000 7f1241202640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123c06a4b0 0x7f123c0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:20.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.303+0000 7f1241202640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f123c0c9ca0 0x7f123c0ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:20.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.304+0000 7f1241202640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f123c0cb030 con 0x7f123c15ad70 2023-10-16T11:22:20.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.304+0000 7f1241202640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f123c0cae30 con 0x7f123c0c9ca0 2023-10-16T11:22:20.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.304+0000 7f1241202640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123c0c91c0 con 0x7f123c06a4b0 2023-10-16T11:22:20.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.304+0000 7f123ad9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f123c15ad70 0x7f123c069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40854/0 (socket says 172.21.15.84:40854) 2023-10-16T11:22:20.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.304+0000 7f123ad9d640 1 -- 172.21.15.84:0/438373758 learned_addr learned my addr 172.21.15.84:0/438373758 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:20.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.304+0000 7f1233fff640 1 --2- 172.21.15.84:0/438373758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123c06a4b0 0x7f123c0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:20.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.304+0000 7f123b59e640 1 --2- 172.21.15.84:0/438373758 >> v2:172.21.15.157:3300/0 conn(0x7f123c0c9ca0 0x7f123c0ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:20.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.304+0000 7f123a59c640 1 -- 172.21.15.84:0/438373758 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279949644 0 0) 0x7f123c0cb030 con 0x7f123c15ad70 2023-10-16T11:22:20.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.304+0000 7f123a59c640 1 -- 172.21.15.84:0/438373758 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1228003520 con 0x7f123c15ad70 2023-10-16T11:22:20.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.305+0000 7f123a59c640 1 -- 172.21.15.84:0/438373758 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2397304728 0 0) 0x7f1228003520 con 0x7f123c15ad70 2023-10-16T11:22:20.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.305+0000 7f123a59c640 1 -- 172.21.15.84:0/438373758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123c06a4b0 msgr2=0x7f123c0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:20.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.305+0000 7f123a59c640 1 --2- 172.21.15.84:0/438373758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123c06a4b0 0x7f123c0ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:20.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.305+0000 7f123a59c640 1 -- 172.21.15.84:0/438373758 >> v2:172.21.15.157:3300/0 conn(0x7f123c0c9ca0 msgr2=0x7f123c0ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:20.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.305+0000 7f123a59c640 1 --2- 172.21.15.84:0/438373758 >> v2:172.21.15.157:3300/0 conn(0x7f123c0c9ca0 0x7f123c0ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:20.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.305+0000 7f123a59c640 1 -- 172.21.15.84:0/438373758 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c0c94a0 con 0x7f123c15ad70 2023-10-16T11:22:20.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.305+0000 7f123a59c640 1 -- 172.21.15.84:0/438373758 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f122c001670 con 0x7f123c15ad70 2023-10-16T11:22:20.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.305+0000 7f123a59c640 1 -- 172.21.15.84:0/438373758 >> v1:172.21.15.84:6789/0 conn(0x7f123c15ad70 legacy=0x7f123c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:20.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.305+0000 7f123a59c640 1 --2- 172.21.15.84:0/438373758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1228003970 0x7f1228003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:20.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.305+0000 7f123a59c640 1 --2- 172.21.15.84:0/438373758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123c06a4b0 0x7f12280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:20.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.305+0000 7f123a59c640 1 --2- 172.21.15.84:0/438373758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1228004d00 0x7f1228005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:20.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.305+0000 7f123a59c640 1 -- 172.21.15.84:0/438373758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c0cae30 con 0x7f1228004d00 2023-10-16T11:22:20.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.305+0000 7f123a59c640 1 -- 172.21.15.84:0/438373758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c0c91c0 con 0x7f1228003970 2023-10-16T11:22:20.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.305+0000 7f123a59c640 1 -- 172.21.15.84:0/438373758 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122800b980 con 0x7f123c06a4b0 2023-10-16T11:22:20.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.305+0000 7f123b59e640 1 --2- 172.21.15.84:0/438373758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1228004d00 0x7f1228005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:20.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.305+0000 7f123ad9d640 1 --2- 172.21.15.84:0/438373758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1228003970 0x7f1228003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:20.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.305+0000 7f123b59e640 1 -- 172.21.15.84:0/438373758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123c06a4b0 msgr2=0x7f12280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:20.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.305+0000 7f123b59e640 1 --2- 172.21.15.84:0/438373758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123c06a4b0 0x7f12280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:20.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.306+0000 7f123b59e640 1 -- 172.21.15.84:0/438373758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1228003970 msgr2=0x7f1228003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:20.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.306+0000 7f123b59e640 1 --2- 172.21.15.84:0/438373758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1228003970 0x7f1228003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:20.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.306+0000 7f123b59e640 1 -- 172.21.15.84:0/438373758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1228003520 con 0x7f1228004d00 2023-10-16T11:22:20.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.306+0000 7f123ad9d640 1 --2- 172.21.15.84:0/438373758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1228003970 0x7f1228003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:20.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.307+0000 7f123b59e640 1 --2- 172.21.15.84:0/438373758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1228004d00 0x7f1228005180 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f1234009fc0 tx=0x7f12280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b127385a4750f49 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:20.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.307+0000 7f123a59c640 1 -- 172.21.15.84:0/438373758 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123402cc60 con 0x7f1228004d00 2023-10-16T11:22:20.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.307+0000 7f123a59c640 1 -- 172.21.15.84:0/438373758 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123400f040 con 0x7f1228004d00 2023-10-16T11:22:20.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.307+0000 7f123a59c640 1 -- 172.21.15.84:0/438373758 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1234034440 con 0x7f1228004d00 2023-10-16T11:22:20.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.307+0000 7f1241202640 1 -- 172.21.15.84:0/438373758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1228004d00 msgr2=0x7f1228005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:20.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.307+0000 7f1241202640 1 --2- 172.21.15.84:0/438373758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1228004d00 0x7f1228005180 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f1234009fc0 tx=0x7f12280057f0 comp rx=0 tx=0).stop 2023-10-16T11:22:20.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.307+0000 7f123b59e640 1 -- 172.21.15.84:0/438373758 reap_dead start 2023-10-16T11:22:20.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.308+0000 7f1241202640 1 -- 172.21.15.84:0/438373758 shutdown_connections 2023-10-16T11:22:20.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.308+0000 7f1241202640 1 -- 172.21.15.84:0/438373758 >> 172.21.15.84:0/438373758 conn(0x7f123c0ce9b0 msgr2=0x7f123c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:20.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.308+0000 7f1241202640 1 -- 172.21.15.84:0/438373758 shutdown_connections 2023-10-16T11:22:20.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.308+0000 7f1241202640 1 -- 172.21.15.84:0/438373758 wait complete. 2023-10-16T11:22:20.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.309+0000 7f1241202640 1 Processor -- start 2023-10-16T11:22:20.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.310+0000 7f1241202640 1 -- start start 2023-10-16T11:22:20.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.310+0000 7f1241202640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1228004d00 0x7f123c0c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:20.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.310+0000 7f1241202640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123c0cdbf0 0x7f123c146350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:20.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.310+0000 7f1241202640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f123c149230 0x7f123c1496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:20.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.310+0000 7f1241202640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c149bf0 con 0x7f123c149230 2023-10-16T11:22:20.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.310+0000 7f1241202640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c0c8930 con 0x7f1228004d00 2023-10-16T11:22:20.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.310+0000 7f1241202640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123c0c7f60 con 0x7f123c0cdbf0 2023-10-16T11:22:20.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.311+0000 7f123b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f123c149230 0x7f123c1496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:20.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.311+0000 7f123b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f123c149230 0x7f123c1496b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53848/0 (socket says 172.21.15.84:53848) 2023-10-16T11:22:20.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.311+0000 7f123b59e640 1 -- 172.21.15.84:0/3317667874 learned_addr learned my addr 172.21.15.84:0/3317667874 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:20.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.311+0000 7f123ad9d640 1 --2- 172.21.15.84:0/3317667874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1228004d00 0x7f123c0c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:20.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.311+0000 7f1233fff640 1 --2- 172.21.15.84:0/3317667874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123c0cdbf0 0x7f123c146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:20.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.311+0000 7f123b59e640 1 -- 172.21.15.84:0/3317667874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123c0cdbf0 msgr2=0x7f123c146350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:20.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.311+0000 7f123b59e640 1 --2- 172.21.15.84:0/3317667874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123c0cdbf0 0x7f123c146350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:20.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.311+0000 7f123b59e640 1 -- 172.21.15.84:0/3317667874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1228004d00 msgr2=0x7f123c0c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:20.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.311+0000 7f123b59e640 1 --2- 172.21.15.84:0/3317667874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1228004d00 0x7f123c0c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:20.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.311+0000 7f123b59e640 1 -- 172.21.15.84:0/3317667874 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c146890 con 0x7f123c149230 2023-10-16T11:22:20.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.311+0000 7f123ad9d640 1 --2- 172.21.15.84:0/3317667874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1228004d00 0x7f123c0c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:20.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.311+0000 7f123b59e640 1 --2- 172.21.15.84:0/3317667874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f123c149230 0x7f123c1496b0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f123402c6d0 tx=0x7f1234009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:20.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.312+0000 7f12337fe640 1 -- 172.21.15.84:0/3317667874 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1234043070 con 0x7f123c149230 2023-10-16T11:22:20.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.312+0000 7f12337fe640 1 -- 172.21.15.84:0/3317667874 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1234034b50 con 0x7f123c149230 2023-10-16T11:22:20.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.312+0000 7f12337fe640 1 -- 172.21.15.84:0/3317667874 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1234003990 con 0x7f123c149230 2023-10-16T11:22:20.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.312+0000 7f1241202640 1 -- 172.21.15.84:0/3317667874 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123c151820 con 0x7f123c149230 2023-10-16T11:22:20.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.312+0000 7f1241202640 1 -- 172.21.15.84:0/3317667874 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123c0c7910 con 0x7f123c149230 2023-10-16T11:22:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.313+0000 7f1241202640 1 -- 172.21.15.84:0/3317667874 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f123c06fe60 con 0x7f123c149230 2023-10-16T11:22:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.314+0000 7f12337fe640 1 -- 172.21.15.84:0/3317667874 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 117) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f123403e440 con 0x7f123c149230 2023-10-16T11:22:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.315+0000 7f12337fe640 1 --2- 172.21.15.84:0/3317667874 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f121c06afe0 0x7f121c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.315+0000 7f12337fe640 1 -- 172.21.15.84:0/3317667874 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f12340abb90 con 0x7f123c149230 2023-10-16T11:22:20.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.315+0000 7f123ad9d640 1 --2- 172.21.15.84:0/3317667874 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f121c06afe0 0x7f121c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:20.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.316+0000 7f123ad9d640 1 --2- 172.21.15.84:0/3317667874 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f121c06afe0 0x7f121c06d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f122c000c00 tx=0x7f122c009040 comp rx=0 tx=0).ready entity=mgr.10476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:20.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.319+0000 7f12337fe640 1 -- 172.21.15.84:0/3317667874 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f12340768e0 con 0x7f123c149230 2023-10-16T11:22:20.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.560+0000 7f1241202640 1 -- 172.21.15.84:0/3317667874 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f123c04e160 con 0x7f123c149230 2023-10-16T11:22:20.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.911+0000 7f12337fe640 1 -- 172.21.15.84:0/3317667874 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v118) v1 ==== 93+0+0 (secure 0 0 0) 0x7f1234039020 con 0x7f123c149230 2023-10-16T11:22:20.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.914+0000 7f1241202640 1 -- 172.21.15.84:0/3317667874 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f121c06afe0 msgr2=0x7f121c06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:20.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.914+0000 7f1241202640 1 --2- 172.21.15.84:0/3317667874 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f121c06afe0 0x7f121c06d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f122c000c00 tx=0x7f122c009040 comp rx=0 tx=0).stop 2023-10-16T11:22:20.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.915+0000 7f1241202640 1 -- 172.21.15.84:0/3317667874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f123c149230 msgr2=0x7f123c1496b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:20.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.915+0000 7f1241202640 1 --2- 172.21.15.84:0/3317667874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f123c149230 0x7f123c1496b0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f123402c6d0 tx=0x7f1234009750 comp rx=0 tx=0).stop 2023-10-16T11:22:20.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.915+0000 7f1241202640 1 -- 172.21.15.84:0/3317667874 shutdown_connections 2023-10-16T11:22:20.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.915+0000 7f1241202640 1 --2- 172.21.15.84:0/3317667874 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f121c06afe0 0x7f121c06d4a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:20.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.915+0000 7f1241202640 1 --2- 172.21.15.84:0/3317667874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f123c149230 0x7f123c1496b0 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:20.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.915+0000 7f1241202640 1 --2- 172.21.15.84:0/3317667874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f123c0cdbf0 0x7f123c146350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:20.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.915+0000 7f1241202640 1 --2- 172.21.15.84:0/3317667874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1228004d00 0x7f123c0c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:20.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.915+0000 7f1241202640 1 -- 172.21.15.84:0/3317667874 >> 172.21.15.84:0/3317667874 conn(0x7f123c0ce9b0 msgr2=0x7f123c0ca1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:20.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.915+0000 7f1241202640 1 -- 172.21.15.84:0/3317667874 shutdown_connections 2023-10-16T11:22:20.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:20.915+0000 7f1241202640 1 -- 172.21.15.84:0/3317667874 wait complete. 2023-10-16T11:22:20.942 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:22:20.943 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:22:21.040 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:22:21.132 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:22:21.133 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:22:21.133 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:21.183 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:22:21.183 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:21.276 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:22:21.445 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:21.443+0000 7f6ff463d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:22:21.459 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:21.457+0000 7fbdf223d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:22:21.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.480+0000 7fe5eb3c0640 1 Processor -- start 2023-10-16T11:22:21.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.480+0000 7fe5eb3c0640 1 -- start start 2023-10-16T11:22:21.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.481+0000 7fe5eb3c0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5e415ae80 0x7fe5e415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.482+0000 7fe5eb3c0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe5e40ce390 0x7fe5e4070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.482+0000 7fe5eb3c0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e415ed60 con 0x7fe5e415aaa0 2023-10-16T11:22:21.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.482+0000 7fe5eb3c0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e406b810 con 0x7fe5e40ce390 2023-10-16T11:22:21.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.482+0000 7fe5eb3c0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40707e0 con 0x7fe5e415ae80 2023-10-16T11:22:21.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.482+0000 7fe5e915c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe5e415aaa0 0x7fe5e40cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40864/0 (socket says 172.21.15.84:40864) 2023-10-16T11:22:21.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.482+0000 7fe5e915c640 1 -- 172.21.15.84:0/3894303238 learned_addr learned my addr 172.21.15.84:0/3894303238 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:21.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.482+0000 7fe5dbfff640 1 -- 172.21.15.84:0/3894303238 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125717258 0 0) 0x7fe5e415ed60 con 0x7fe5e415aaa0 2023-10-16T11:22:21.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.482+0000 7fe5e995d640 1 --2- 172.21.15.84:0/3894303238 >> v2:172.21.15.157:3300/0 conn(0x7fe5e40ce390 0x7fe5e4070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.482+0000 7fe5e895b640 1 --2- 172.21.15.84:0/3894303238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5e415ae80 0x7fe5e415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.482+0000 7fe5dbfff640 1 -- 172.21.15.84:0/3894303238 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5d0003520 con 0x7fe5e415aaa0 2023-10-16T11:22:21.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5dbfff640 1 -- 172.21.15.84:0/3894303238 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2219123478 0 0) 0x7fe5d0003520 con 0x7fe5e415aaa0 2023-10-16T11:22:21.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5dbfff640 1 -- 172.21.15.84:0/3894303238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5e415ae80 msgr2=0x7fe5e415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:21.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5dbfff640 1 --2- 172.21.15.84:0/3894303238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5e415ae80 0x7fe5e415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5dbfff640 1 -- 172.21.15.84:0/3894303238 >> v2:172.21.15.157:3300/0 conn(0x7fe5e40ce390 msgr2=0x7fe5e4070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:21.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5dbfff640 1 --2- 172.21.15.84:0/3894303238 >> v2:172.21.15.157:3300/0 conn(0x7fe5e40ce390 0x7fe5e4070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5dbfff640 1 -- 172.21.15.84:0/3894303238 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e415ef90 con 0x7fe5e415aaa0 2023-10-16T11:22:21.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5dbfff640 1 -- 172.21.15.84:0/3894303238 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fe5e0001670 con 0x7fe5e415aaa0 2023-10-16T11:22:21.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5dbfff640 1 -- 172.21.15.84:0/3894303238 >> v1:172.21.15.84:6789/0 conn(0x7fe5e415aaa0 legacy=0x7fe5e40cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:21.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5dbfff640 1 --2- 172.21.15.84:0/3894303238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d0003970 0x7fe5d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5dbfff640 1 --2- 172.21.15.84:0/3894303238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5d00047c0 0x7fe5d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5dbfff640 1 --2- 172.21.15.84:0/3894303238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5e415ae80 0x7fe5d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5dbfff640 1 -- 172.21.15.84:0/3894303238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e406b810 con 0x7fe5d00047c0 2023-10-16T11:22:21.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5dbfff640 1 -- 172.21.15.84:0/3894303238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40707e0 con 0x7fe5d0003970 2023-10-16T11:22:21.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5dbfff640 1 -- 172.21.15.84:0/3894303238 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d000bab0 con 0x7fe5e415ae80 2023-10-16T11:22:21.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5e895b640 1 --2- 172.21.15.84:0/3894303238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5d00047c0 0x7fe5d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5e895b640 1 -- 172.21.15.84:0/3894303238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5e415ae80 msgr2=0x7fe5d0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:21.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5e895b640 1 --2- 172.21.15.84:0/3894303238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5e415ae80 0x7fe5d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5e895b640 1 -- 172.21.15.84:0/3894303238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d0003970 msgr2=0x7fe5d0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:21.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5e915c640 1 --2- 172.21.15.84:0/3894303238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d0003970 0x7fe5d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5e895b640 1 --2- 172.21.15.84:0/3894303238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d0003970 0x7fe5d0003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.483+0000 7fe5e895b640 1 -- 172.21.15.84:0/3894303238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d0003520 con 0x7fe5d00047c0 2023-10-16T11:22:21.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.484+0000 7fe5e895b640 1 --2- 172.21.15.84:0/3894303238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5d00047c0 0x7fe5d0004c40 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400ba00 tx=0x7fe5d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df29d616eab151d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:21.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.485+0000 7fe5dbfff640 1 -- 172.21.15.84:0/3894303238 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4032630 con 0x7fe5d00047c0 2023-10-16T11:22:21.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.485+0000 7fe5dbfff640 1 -- 172.21.15.84:0/3894303238 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d4032790 con 0x7fe5d00047c0 2023-10-16T11:22:21.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.485+0000 7fe5dbfff640 1 -- 172.21.15.84:0/3894303238 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4032b20 con 0x7fe5d00047c0 2023-10-16T11:22:21.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.485+0000 7fe5eb3c0640 1 -- 172.21.15.84:0/3894303238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5d00047c0 msgr2=0x7fe5d0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:21.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.485+0000 7fe5eb3c0640 1 --2- 172.21.15.84:0/3894303238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5d00047c0 0x7fe5d0004c40 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400ba00 tx=0x7fe5d00057f0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.485+0000 7fe5e995d640 1 -- 172.21.15.84:0/3894303238 reap_dead start 2023-10-16T11:22:21.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.485+0000 7fe5eb3c0640 1 -- 172.21.15.84:0/3894303238 shutdown_connections 2023-10-16T11:22:21.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.485+0000 7fe5eb3c0640 1 -- 172.21.15.84:0/3894303238 >> 172.21.15.84:0/3894303238 conn(0x7fe5e40cb410 msgr2=0x7fe5e40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:21.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.485+0000 7fe5eb3c0640 1 -- 172.21.15.84:0/3894303238 shutdown_connections 2023-10-16T11:22:21.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.485+0000 7fe5eb3c0640 1 -- 172.21.15.84:0/3894303238 wait complete. 2023-10-16T11:22:21.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.487+0000 7fe5eb3c0640 1 Processor -- start 2023-10-16T11:22:21.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.487+0000 7fe5eb3c0640 1 -- start start 2023-10-16T11:22:21.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.488+0000 7fe5eb3c0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d00047c0 0x7fe5e40b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.488+0000 7fe5eb3c0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5e40ca1c0 0x7fe5e413d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.488+0000 7fe5eb3c0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5e40ca640 0x7fe5e413dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.488+0000 7fe5eb3c0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e414f4a0 con 0x7fe5e40ca640 2023-10-16T11:22:21.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.488+0000 7fe5eb3c0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e414ec90 con 0x7fe5d00047c0 2023-10-16T11:22:21.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.488+0000 7fe5eb3c0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e414eee0 con 0x7fe5e40ca1c0 2023-10-16T11:22:21.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.489+0000 7fe5e995d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5e40ca640 0x7fe5e413dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.489+0000 7fe5e995d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5e40ca640 0x7fe5e413dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53894/0 (socket says 172.21.15.84:53894) 2023-10-16T11:22:21.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.489+0000 7fe5e995d640 1 -- 172.21.15.84:0/3976559641 learned_addr learned my addr 172.21.15.84:0/3976559641 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:21.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.489+0000 7fe5e915c640 1 --2- 172.21.15.84:0/3976559641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d00047c0 0x7fe5e40b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.489+0000 7fe5e895b640 1 --2- 172.21.15.84:0/3976559641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5e40ca1c0 0x7fe5e413d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.489+0000 7fe5e995d640 1 -- 172.21.15.84:0/3976559641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5e40ca1c0 msgr2=0x7fe5e413d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:21.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.489+0000 7fe5e995d640 1 --2- 172.21.15.84:0/3976559641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5e40ca1c0 0x7fe5e413d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.489+0000 7fe5e995d640 1 -- 172.21.15.84:0/3976559641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d00047c0 msgr2=0x7fe5e40b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:21.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.489+0000 7fe5e995d640 1 --2- 172.21.15.84:0/3976559641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d00047c0 0x7fe5e40b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.489+0000 7fe5e995d640 1 -- 172.21.15.84:0/3976559641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e414f1c0 con 0x7fe5e40ca640 2023-10-16T11:22:21.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.489+0000 7fe5e915c640 1 --2- 172.21.15.84:0/3976559641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d00047c0 0x7fe5e40b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:21.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.489+0000 7fe5e995d640 1 --2- 172.21.15.84:0/3976559641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5e40ca640 0x7fe5e413dc10 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40ce2e0 tx=0x7fe5dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:21.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.490+0000 7fe5d9ffb640 1 -- 172.21.15.84:0/3976559641 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc00ed90 con 0x7fe5e40ca640 2023-10-16T11:22:21.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.490+0000 7fe5d9ffb640 1 -- 172.21.15.84:0/3976559641 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc014420 con 0x7fe5e40ca640 2023-10-16T11:22:21.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.490+0000 7fe5d9ffb640 1 -- 172.21.15.84:0/3976559641 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc0147d0 con 0x7fe5e40ca640 2023-10-16T11:22:21.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.490+0000 7fe5eb3c0640 1 -- 172.21.15.84:0/3976559641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e40c99e0 con 0x7fe5e40ca640 2023-10-16T11:22:21.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.490+0000 7fe5eb3c0640 1 -- 172.21.15.84:0/3976559641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e40c5a60 con 0x7fe5e40ca640 2023-10-16T11:22:21.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.491+0000 7fe5eb3c0640 1 -- 172.21.15.84:0/3976559641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e40c9840 con 0x7fe5e40ca640 2023-10-16T11:22:21.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.492+0000 7fe5d9ffb640 1 -- 172.21.15.84:0/3976559641 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 118) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fe5dc014930 con 0x7fe5e40ca640 2023-10-16T11:22:21.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.493+0000 7fe5d9ffb640 1 --2- 172.21.15.84:0/3976559641 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fe5c806af90 0x7fe5c806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.493+0000 7fe5d9ffb640 1 -- 172.21.15.84:0/3976559641 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6425+0+0 (secure 0 0 0) 0x7fe5dc08a070 con 0x7fe5e40ca640 2023-10-16T11:22:21.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.493+0000 7fe5e915c640 1 -- 172.21.15.84:0/3976559641 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fe5c806af90 msgr2=0x7fe5c806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/285590107 2023-10-16T11:22:21.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.493+0000 7fe5e915c640 1 --2- 172.21.15.84:0/3976559641 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fe5c806af90 0x7fe5c806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:21.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.497+0000 7fe5d9ffb640 1 -- 172.21.15.84:0/3976559641 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe5e40c9840 con 0x7fe5e40ca640 2023-10-16T11:22:21.508 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:21.501+0000 7f6ff463d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:22:21.514 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:21.513+0000 7fbdf223d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:22:21.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.691+0000 7fe5eb3c0640 1 -- 172.21.15.84:0/3976559641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5e40c9840 con 0x7fe5e40ca640 2023-10-16T11:22:21.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.693+0000 7fe5d9ffb640 1 -- 172.21.15.84:0/3976559641 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+17591 (secure 0 0 0) 0x7fe5e40c9840 con 0x7fe5e40ca640 2023-10-16T11:22:21.695 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:21.696 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":186,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:17:21.065708+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:06:20.873257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-16T11:06:22.065944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/542522615":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/878936990":"2023-10-17T11:17:21.065656+0000","172.21.15.84:6833/3993913200":"2023-10-17T11:17:21.065656+0000","172.21.15.84:6832/3993913200":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/1064178200":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/3548457413":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/2013731252":"2023-10-17T11:17:21.065656+0000","172.21.15.84:6833/1441051579":"2023-10-17T11:06:37.825968+0000","172.21.15.84:6832/1441051579":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/2359294858":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/2830443959":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/4150685824":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/4095400472":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/3348937827":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/2158345505":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/235070477":"2023-10-17T11:06:37.825968+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:22:21.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.694+0000 7fe5e915c640 1 -- 172.21.15.84:0/3976559641 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fe5c806af90 msgr2=0x7fe5c806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/285590107 2023-10-16T11:22:21.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.694+0000 7fe5e915c640 1 --2- 172.21.15.84:0/3976559641 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fe5c806af90 0x7fe5c806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:21.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.695+0000 7fe5eb3c0640 1 -- 172.21.15.84:0/3976559641 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fe5c806af90 msgr2=0x7fe5c806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:21.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.695+0000 7fe5eb3c0640 1 --2- 172.21.15.84:0/3976559641 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fe5c806af90 0x7fe5c806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.695+0000 7fe5eb3c0640 1 -- 172.21.15.84:0/3976559641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5e40ca640 msgr2=0x7fe5e413dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:21.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.695+0000 7fe5eb3c0640 1 --2- 172.21.15.84:0/3976559641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5e40ca640 0x7fe5e413dc10 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40ce2e0 tx=0x7fe5dc00e4b0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.695+0000 7fe5eb3c0640 1 -- 172.21.15.84:0/3976559641 shutdown_connections 2023-10-16T11:22:21.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.695+0000 7fe5eb3c0640 1 --2- 172.21.15.84:0/3976559641 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fe5c806af90 0x7fe5c806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.695+0000 7fe5eb3c0640 1 --2- 172.21.15.84:0/3976559641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5e40ca640 0x7fe5e413dc10 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.695+0000 7fe5eb3c0640 1 --2- 172.21.15.84:0/3976559641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5e40ca1c0 0x7fe5e413d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.695+0000 7fe5eb3c0640 1 --2- 172.21.15.84:0/3976559641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d00047c0 0x7fe5e40b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.695+0000 7fe5eb3c0640 1 -- 172.21.15.84:0/3976559641 >> 172.21.15.84:0/3976559641 conn(0x7fe5e40cb410 msgr2=0x7fe5e40ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:21.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.695+0000 7fe5eb3c0640 1 -- 172.21.15.84:0/3976559641 shutdown_connections 2023-10-16T11:22:21.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.695+0000 7fe5eb3c0640 1 -- 172.21.15.84:0/3976559641 wait complete. 2023-10-16T11:22:21.717 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:06:20.873257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:06:22.065944+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:22:21.718 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:22:21.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.915+0000 7fc921032640 1 Processor -- start 2023-10-16T11:22:21.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.915+0000 7fc921032640 1 -- start start 2023-10-16T11:22:21.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.916+0000 7fc921032640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc91c0b2080 0x7fc91c0d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.916+0000 7fc921032640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc91c0cfc40 0x7fc91c0b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.916+0000 7fc921032640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc91c0d0970 con 0x7fc91c0b1ca0 2023-10-16T11:22:21.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.916+0000 7fc921032640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc91c0d0ea0 con 0x7fc91c0b2080 2023-10-16T11:22:21.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.916+0000 7fc921032640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c0d0bc0 con 0x7fc91c0cfc40 2023-10-16T11:22:21.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.917+0000 7fc91bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc91c0b1ca0 0x7fc91c0ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40870/0 (socket says 172.21.15.84:40870) 2023-10-16T11:22:21.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.917+0000 7fc91bfff640 1 -- 172.21.15.84:0/3448766822 learned_addr learned my addr 172.21.15.84:0/3448766822 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:21.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.917+0000 7fc920831640 1 --2- 172.21.15.84:0/3448766822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc91c0cfc40 0x7fc91c0b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.917+0000 7fc91b7fe640 1 --2- 172.21.15.84:0/3448766822 >> v2:172.21.15.157:3300/0 conn(0x7fc91c0b2080 0x7fc91c0d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.917+0000 7fc91affd640 1 -- 172.21.15.84:0/3448766822 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190114366 0 0) 0x7fc91c0d0970 con 0x7fc91c0b1ca0 2023-10-16T11:22:21.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.917+0000 7fc91affd640 1 -- 172.21.15.84:0/3448766822 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc904003520 con 0x7fc91c0b1ca0 2023-10-16T11:22:21.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.917+0000 7fc91affd640 1 -- 172.21.15.84:0/3448766822 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 568635293 0 0) 0x7fc904003520 con 0x7fc91c0b1ca0 2023-10-16T11:22:21.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.917+0000 7fc91affd640 1 -- 172.21.15.84:0/3448766822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc91c0cfc40 msgr2=0x7fc91c0b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:21.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.917+0000 7fc91affd640 1 --2- 172.21.15.84:0/3448766822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc91c0cfc40 0x7fc91c0b24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.918+0000 7fc91affd640 1 -- 172.21.15.84:0/3448766822 >> v2:172.21.15.157:3300/0 conn(0x7fc91c0b2080 msgr2=0x7fc91c0d36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:21.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.918+0000 7fc91affd640 1 --2- 172.21.15.84:0/3448766822 >> v2:172.21.15.157:3300/0 conn(0x7fc91c0b2080 0x7fc91c0d36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.918+0000 7fc91affd640 1 -- 172.21.15.84:0/3448766822 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91c0cd940 con 0x7fc91c0b1ca0 2023-10-16T11:22:21.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.918+0000 7fc91affd640 1 -- 172.21.15.84:0/3448766822 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc90c001670 con 0x7fc91c0b1ca0 2023-10-16T11:22:21.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.918+0000 7fc91affd640 1 -- 172.21.15.84:0/3448766822 >> v1:172.21.15.84:6789/0 conn(0x7fc91c0b1ca0 legacy=0x7fc91c0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:21.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.918+0000 7fc91affd640 1 --2- 172.21.15.84:0/3448766822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc91c0cfc40 0x7fc904003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.918+0000 7fc91affd640 1 --2- 172.21.15.84:0/3448766822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc904003ef0 0x7fc9040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.918+0000 7fc91affd640 1 --2- 172.21.15.84:0/3448766822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc904004d00 0x7fc904005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.918+0000 7fc91affd640 1 -- 172.21.15.84:0/3448766822 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c0d0ea0 con 0x7fc904003ef0 2023-10-16T11:22:21.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.918+0000 7fc91affd640 1 -- 172.21.15.84:0/3448766822 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c0d0bc0 con 0x7fc904004d00 2023-10-16T11:22:21.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.918+0000 7fc91affd640 1 -- 172.21.15.84:0/3448766822 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90400b9b0 con 0x7fc91c0cfc40 2023-10-16T11:22:21.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.918+0000 7fc91b7fe640 1 --2- 172.21.15.84:0/3448766822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc904003ef0 0x7fc9040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.918+0000 7fc91bfff640 1 --2- 172.21.15.84:0/3448766822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc91c0cfc40 0x7fc904003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.918+0000 7fc920831640 1 --2- 172.21.15.84:0/3448766822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc904004d00 0x7fc904005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.919+0000 7fc91b7fe640 1 -- 172.21.15.84:0/3448766822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc91c0cfc40 msgr2=0x7fc904003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:21.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.919+0000 7fc91b7fe640 1 --2- 172.21.15.84:0/3448766822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc91c0cfc40 0x7fc904003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.919+0000 7fc91b7fe640 1 -- 172.21.15.84:0/3448766822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc904004d00 msgr2=0x7fc904005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:21.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.919+0000 7fc91b7fe640 1 --2- 172.21.15.84:0/3448766822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc904004d00 0x7fc904005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.919+0000 7fc91b7fe640 1 -- 172.21.15.84:0/3448766822 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc904003520 con 0x7fc904003ef0 2023-10-16T11:22:21.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.920+0000 7fc91b7fe640 1 --2- 172.21.15.84:0/3448766822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc904003ef0 0x7fc9040047c0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fc910009ac0 tx=0x7fc9040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803a846a82af155c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:21.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.920+0000 7fc91affd640 1 -- 172.21.15.84:0/3448766822 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91002c710 con 0x7fc904003ef0 2023-10-16T11:22:21.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.920+0000 7fc91affd640 1 -- 172.21.15.84:0/3448766822 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91002c870 con 0x7fc904003ef0 2023-10-16T11:22:21.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.920+0000 7fc91affd640 1 -- 172.21.15.84:0/3448766822 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91002cc00 con 0x7fc904003ef0 2023-10-16T11:22:21.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.921+0000 7fc921032640 1 -- 172.21.15.84:0/3448766822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc904003ef0 msgr2=0x7fc9040047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:21.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.921+0000 7fc921032640 1 --2- 172.21.15.84:0/3448766822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc904003ef0 0x7fc9040047c0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fc910009ac0 tx=0x7fc9040057f0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.921+0000 7fc920831640 1 -- 172.21.15.84:0/3448766822 reap_dead start 2023-10-16T11:22:21.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.921+0000 7fc921032640 1 -- 172.21.15.84:0/3448766822 shutdown_connections 2023-10-16T11:22:21.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.921+0000 7fc921032640 1 -- 172.21.15.84:0/3448766822 >> 172.21.15.84:0/3448766822 conn(0x7fc91c0d2b90 msgr2=0x7fc91c0d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:21.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.921+0000 7fc921032640 1 -- 172.21.15.84:0/3448766822 shutdown_connections 2023-10-16T11:22:21.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.921+0000 7fc921032640 1 -- 172.21.15.84:0/3448766822 wait complete. 2023-10-16T11:22:21.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.923+0000 7fc921032640 1 Processor -- start 2023-10-16T11:22:21.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.923+0000 7fc921032640 1 -- start start 2023-10-16T11:22:21.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.924+0000 7fc921032640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc904003ef0 0x7fc91c154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.924+0000 7fc921032640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc91c1e6aa0 0x7fc91c1e6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.924+0000 7fc921032640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc91c0c8cf0 0x7fc91c0c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.924+0000 7fc921032640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c1e7440 con 0x7fc904003ef0 2023-10-16T11:22:21.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.924+0000 7fc921032640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c0cc620 con 0x7fc91c1e6aa0 2023-10-16T11:22:21.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.924+0000 7fc921032640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c0c96b0 con 0x7fc91c0c8cf0 2023-10-16T11:22:21.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.924+0000 7fc91bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc904003ef0 0x7fc91c154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.924+0000 7fc91bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc904003ef0 0x7fc91c154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53912/0 (socket says 172.21.15.84:53912) 2023-10-16T11:22:21.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.924+0000 7fc91bfff640 1 -- 172.21.15.84:0/861553834 learned_addr learned my addr 172.21.15.84:0/861553834 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:21.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.924+0000 7fc920831640 1 --2- 172.21.15.84:0/861553834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc91c0c8cf0 0x7fc91c0c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.924+0000 7fc91b7fe640 1 --2- 172.21.15.84:0/861553834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc91c1e6aa0 0x7fc91c1e6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.924+0000 7fc91bfff640 1 -- 172.21.15.84:0/861553834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc91c0c8cf0 msgr2=0x7fc91c0c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:21.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.924+0000 7fc91bfff640 1 --2- 172.21.15.84:0/861553834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc91c0c8cf0 0x7fc91c0c9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.924+0000 7fc91bfff640 1 -- 172.21.15.84:0/861553834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc91c1e6aa0 msgr2=0x7fc91c1e6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:21.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.924+0000 7fc91bfff640 1 --2- 172.21.15.84:0/861553834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc91c1e6aa0 0x7fc91c1e6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.924+0000 7fc91bfff640 1 -- 172.21.15.84:0/861553834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91c0c6840 con 0x7fc904003ef0 2023-10-16T11:22:21.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.924+0000 7fc91b7fe640 1 --2- 172.21.15.84:0/861553834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc91c1e6aa0 0x7fc91c1e6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:21.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.925+0000 7fc91bfff640 1 --2- 172.21.15.84:0/861553834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc904003ef0 0x7fc91c154c60 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fc90c007340 tx=0x7fc90c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:21.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.925+0000 7fc918ff9640 1 -- 172.21.15.84:0/861553834 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c00f8c0 con 0x7fc904003ef0 2023-10-16T11:22:21.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.925+0000 7fc918ff9640 1 -- 172.21.15.84:0/861553834 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90c00fa60 con 0x7fc904003ef0 2023-10-16T11:22:21.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.925+0000 7fc921032640 1 -- 172.21.15.84:0/861553834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91c0c6b00 con 0x7fc904003ef0 2023-10-16T11:22:21.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.925+0000 7fc918ff9640 1 -- 172.21.15.84:0/861553834 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c00fe10 con 0x7fc904003ef0 2023-10-16T11:22:21.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.925+0000 7fc921032640 1 -- 172.21.15.84:0/861553834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc91c0c70c0 con 0x7fc904003ef0 2023-10-16T11:22:21.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.926+0000 7fc921032640 1 -- 172.21.15.84:0/861553834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc91c04e9b0 con 0x7fc904003ef0 2023-10-16T11:22:21.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.928+0000 7fc918ff9640 1 -- 172.21.15.84:0/861553834 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 118) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fc90c01d460 con 0x7fc904003ef0 2023-10-16T11:22:21.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.928+0000 7fc918ff9640 1 --2- 172.21.15.84:0/861553834 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fc90806af90 0x7fc90806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.929+0000 7fc918ff9640 1 -- 172.21.15.84:0/861553834 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6425+0+0 (secure 0 0 0) 0x7fc90c089cf0 con 0x7fc904003ef0 2023-10-16T11:22:21.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.929+0000 7fc91b7fe640 1 -- 172.21.15.84:0/861553834 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fc90806af90 msgr2=0x7fc90806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/285590107 2023-10-16T11:22:21.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.929+0000 7fc91b7fe640 1 --2- 172.21.15.84:0/861553834 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fc90806af90 0x7fc90806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:21.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:21.934+0000 7fc918ff9640 1 -- 172.21.15.84:0/861553834 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc90c054920 con 0x7fc904003ef0 2023-10-16T11:22:21.961 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:21.959+0000 7fbdf223d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:22:21.962 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:21.960+0000 7f6ff463d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:22:22.014 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:22.011+0000 7fbdf223d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:22:22.015 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:22.013+0000 7f6ff463d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:22:22.063 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:22.061+0000 7fbdf223d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:22:22.066 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:22.064+0000 7f6ff463d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:22:22.124 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:22.122+0000 7fbdf223d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:22:22.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.125+0000 7fc921032640 1 -- 172.21.15.84:0/861553834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc91c073a40 con 0x7fc904003ef0 2023-10-16T11:22:22.127 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:22.126+0000 7f6ff463d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:22:22.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.126+0000 7fc918ff9640 1 -- 172.21.15.84:0/861553834 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v186) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc90c057f40 con 0x7fc904003ef0 2023-10-16T11:22:22.128 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:22:22.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.128+0000 7fc921032640 1 -- 172.21.15.84:0/861553834 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fc90806af90 msgr2=0x7fc90806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:22.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.128+0000 7fc921032640 1 --2- 172.21.15.84:0/861553834 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fc90806af90 0x7fc90806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.129+0000 7fc921032640 1 -- 172.21.15.84:0/861553834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc904003ef0 msgr2=0x7fc91c154c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.129+0000 7fc921032640 1 --2- 172.21.15.84:0/861553834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc904003ef0 0x7fc91c154c60 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fc90c007340 tx=0x7fc90c007800 comp rx=0 tx=0).stop 2023-10-16T11:22:22.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.129+0000 7fc921032640 1 -- 172.21.15.84:0/861553834 shutdown_connections 2023-10-16T11:22:22.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.129+0000 7fc921032640 1 --2- 172.21.15.84:0/861553834 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fc90806af90 0x7fc90806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.129+0000 7fc921032640 1 --2- 172.21.15.84:0/861553834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc91c0c8cf0 0x7fc91c0c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.129+0000 7fc921032640 1 --2- 172.21.15.84:0/861553834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc91c1e6aa0 0x7fc91c1e6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.129+0000 7fc921032640 1 --2- 172.21.15.84:0/861553834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc904003ef0 0x7fc91c154c60 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.129+0000 7fc921032640 1 -- 172.21.15.84:0/861553834 >> 172.21.15.84:0/861553834 conn(0x7fc91c0d2b90 msgr2=0x7fc91c15ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:22.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.129+0000 7fc921032640 1 -- 172.21.15.84:0/861553834 shutdown_connections 2023-10-16T11:22:22.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.129+0000 7fc921032640 1 -- 172.21.15.84:0/861553834 wait complete. 2023-10-16T11:22:22.148 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:22:22.172 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:22.170+0000 7fbdf223d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:22:22.180 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:22.178+0000 7f6ff463d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:22:22.284 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:22.282+0000 7fbdf223d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:22:22.296 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:22.294+0000 7f6ff463d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:22:22.346 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:22.344+0000 7fbdf223d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:22:22.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.348+0000 7f2448cad640 1 Processor -- start 2023-10-16T11:22:22.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.348+0000 7f2448cad640 1 -- start start 2023-10-16T11:22:22.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.349+0000 7f2448cad640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24440b8480 0x7f24440cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.349+0000 7f2448cad640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f24440d0be0 0x7f24440cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.349+0000 7f2448cad640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f244415d150 con 0x7f244415cd70 2023-10-16T11:22:22.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.349+0000 7f2448cad640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24440d1180 con 0x7f24440d0be0 2023-10-16T11:22:22.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.349+0000 7f2448cad640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24440d1460 con 0x7f24440b8480 2023-10-16T11:22:22.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.350+0000 7f2443fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f24440d0be0 0x7f24440cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.350+0000 7f2443fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f24440d0be0 0x7f24440cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53914/0 (socket says 172.21.15.84:53914) 2023-10-16T11:22:22.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.350+0000 7f2443fff640 1 -- 172.21.15.84:0/1610186069 learned_addr learned my addr 172.21.15.84:0/1610186069 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:22.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.350+0000 7f2442ffd640 1 --2- 172.21.15.84:0/1610186069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24440b8480 0x7f24440cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.350+0000 7f2443fff640 1 -- 172.21.15.84:0/1610186069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24440b8480 msgr2=0x7f24440cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:22.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.350+0000 7f2443fff640 1 --2- 172.21.15.84:0/1610186069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24440b8480 0x7f24440cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.350+0000 7f2443fff640 1 -- 172.21.15.84:0/1610186069 >> v1:172.21.15.157:6789/0 conn(0x7f244415cd70 legacy=0x7f24440d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:22.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.350+0000 7f2443fff640 1 -- 172.21.15.84:0/1610186069 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24440d1740 con 0x7f24440d0be0 2023-10-16T11:22:22.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.350+0000 7f2442ffd640 1 --2- 172.21.15.84:0/1610186069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24440b8480 0x7f24440cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:22.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.352+0000 7f2443fff640 1 --2- 172.21.15.84:0/1610186069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24440d0be0 0x7f24440cdd00 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f243c009870 tx=0x7f243c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6933b3335da6db8b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:22.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.352+0000 7f24427fc640 1 -- 172.21.15.84:0/1610186069 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c030840 con 0x7f24440d0be0 2023-10-16T11:22:22.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.352+0000 7f24427fc640 1 -- 172.21.15.84:0/1610186069 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c0309a0 con 0x7f24440d0be0 2023-10-16T11:22:22.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.352+0000 7f24427fc640 1 -- 172.21.15.84:0/1610186069 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c030d30 con 0x7f24440d0be0 2023-10-16T11:22:22.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.353+0000 7f2448cad640 1 -- 172.21.15.84:0/1610186069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24440d0be0 msgr2=0x7f24440cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.353+0000 7f2448cad640 1 --2- 172.21.15.84:0/1610186069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24440d0be0 0x7f24440cdd00 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f243c009870 tx=0x7f243c02b980 comp rx=0 tx=0).stop 2023-10-16T11:22:22.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.353+0000 7f2448cad640 1 -- 172.21.15.84:0/1610186069 shutdown_connections 2023-10-16T11:22:22.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.353+0000 7f2448cad640 1 --2- 172.21.15.84:0/1610186069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24440d0be0 0x7f24440cdd00 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.353+0000 7f2448cad640 1 --2- 172.21.15.84:0/1610186069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24440b8480 0x7f24440cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.353+0000 7f2448cad640 1 -- 172.21.15.84:0/1610186069 >> 172.21.15.84:0/1610186069 conn(0x7f24440cf570 msgr2=0x7f24440cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:22.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.353+0000 7f2448cad640 1 -- 172.21.15.84:0/1610186069 shutdown_connections 2023-10-16T11:22:22.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.353+0000 7f2448cad640 1 -- 172.21.15.84:0/1610186069 wait complete. 2023-10-16T11:22:22.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.355+0000 7f2448cad640 1 Processor -- start 2023-10-16T11:22:22.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.355+0000 7f2448cad640 1 -- start start 2023-10-16T11:22:22.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.356+0000 7f2448cad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24440b8480 0x7f24440b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.356+0000 7f2448cad640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24440d0be0 0x7f24440cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.356+0000 7f2448cad640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f244415cd70 0x7f24441475f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.356+0000 7f2448cad640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440cb670 con 0x7f244415cd70 2023-10-16T11:22:22.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.356+0000 7f2448cad640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440cb950 con 0x7f24440b8480 2023-10-16T11:22:22.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.356+0000 7f2448cad640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24440cbc30 con 0x7f24440d0be0 2023-10-16T11:22:22.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.356+0000 7f2443fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f244415cd70 0x7f24441475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.356+0000 7f2443fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f244415cd70 0x7f24441475f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53922/0 (socket says 172.21.15.84:53922) 2023-10-16T11:22:22.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.356+0000 7f2443fff640 1 -- 172.21.15.84:0/320061951 learned_addr learned my addr 172.21.15.84:0/320061951 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:22.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.356+0000 7f2442ffd640 1 --2- 172.21.15.84:0/320061951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24440d0be0 0x7f24440cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.357+0000 7f24437fe640 1 --2- 172.21.15.84:0/320061951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24440b8480 0x7f24440b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.357+0000 7f2443fff640 1 -- 172.21.15.84:0/320061951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24440d0be0 msgr2=0x7f24440cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.357+0000 7f2443fff640 1 --2- 172.21.15.84:0/320061951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24440d0be0 0x7f24440cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.357+0000 7f2443fff640 1 -- 172.21.15.84:0/320061951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24440b8480 msgr2=0x7f24440b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.357+0000 7f2443fff640 1 --2- 172.21.15.84:0/320061951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24440b8480 0x7f24440b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.357+0000 7f2443fff640 1 -- 172.21.15.84:0/320061951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c0094d0 con 0x7f244415cd70 2023-10-16T11:22:22.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.357+0000 7f24437fe640 1 --2- 172.21.15.84:0/320061951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24440b8480 0x7f24440b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:22.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.357+0000 7f2442ffd640 1 --2- 172.21.15.84:0/320061951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24440d0be0 0x7f24440cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:22.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.357+0000 7f2443fff640 1 --2- 172.21.15.84:0/320061951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f244415cd70 0x7f24441475f0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f243c009870 tx=0x7f243c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:22.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.357+0000 7f243bfff640 1 -- 172.21.15.84:0/320061951 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c039c90 con 0x7f244415cd70 2023-10-16T11:22:22.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.357+0000 7f243bfff640 1 -- 172.21.15.84:0/320061951 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c039e10 con 0x7f244415cd70 2023-10-16T11:22:22.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.357+0000 7f243bfff640 1 -- 172.21.15.84:0/320061951 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c043640 con 0x7f244415cd70 2023-10-16T11:22:22.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.358+0000 7f2448cad640 1 -- 172.21.15.84:0/320061951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24440cbf10 con 0x7f244415cd70 2023-10-16T11:22:22.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.358+0000 7f2448cad640 1 -- 172.21.15.84:0/320061951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24440b4bf0 con 0x7f244415cd70 2023-10-16T11:22:22.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.359+0000 7f2448cad640 1 -- 172.21.15.84:0/320061951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24440cc430 con 0x7f244415cd70 2023-10-16T11:22:22.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.361+0000 7f243bfff640 1 -- 172.21.15.84:0/320061951 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 118) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f243c0437a0 con 0x7f244415cd70 2023-10-16T11:22:22.366 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:22.362+0000 7f6ff463d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:22:22.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.362+0000 7f243bfff640 1 --2- 172.21.15.84:0/320061951 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f242406af90 0x7f242406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.362+0000 7f243bfff640 1 -- 172.21.15.84:0/320061951 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f243c03e070 con 0x7f244415cd70 2023-10-16T11:22:22.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.362+0000 7f24437fe640 1 -- 172.21.15.84:0/320061951 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f242406af90 msgr2=0x7f242406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/285590107 2023-10-16T11:22:22.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.362+0000 7f24437fe640 1 --2- 172.21.15.84:0/320061951 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f242406af90 0x7f242406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:22.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.367+0000 7f243bfff640 1 -- 172.21.15.84:0/320061951 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f243c077760 con 0x7f244415cd70 2023-10-16T11:22:22.398 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:22.396+0000 7fbdf223d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:22:22.415 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:22.414+0000 7f6ff463d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:22:22.455 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:22.453+0000 7fbdf223d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:22:22.472 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:22.471+0000 7f6ff463d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:22:22.505 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:22.503+0000 7fbdf223d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:22:22.522 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:22.521+0000 7f6ff463d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:22:22.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.559+0000 7f2448cad640 1 -- 172.21.15.84:0/320061951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f244404e9b0 con 0x7f244415cd70 2023-10-16T11:22:22.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.560+0000 7f243bfff640 1 -- 172.21.15.84:0/320061951 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v186) v1 ==== 104+0+11 (secure 0 0 0) 0x7f243c034020 con 0x7f244415cd70 2023-10-16T11:22:22.561 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:22:22.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.563+0000 7f2448cad640 1 -- 172.21.15.84:0/320061951 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f242406af90 msgr2=0x7f242406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:22.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.563+0000 7f2448cad640 1 --2- 172.21.15.84:0/320061951 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f242406af90 0x7f242406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.563+0000 7f2448cad640 1 -- 172.21.15.84:0/320061951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f244415cd70 msgr2=0x7f24441475f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.563+0000 7f2448cad640 1 --2- 172.21.15.84:0/320061951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f244415cd70 0x7f24441475f0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f243c009870 tx=0x7f243c004730 comp rx=0 tx=0).stop 2023-10-16T11:22:22.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.563+0000 7f2448cad640 1 -- 172.21.15.84:0/320061951 shutdown_connections 2023-10-16T11:22:22.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.563+0000 7f2448cad640 1 --2- 172.21.15.84:0/320061951 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f242406af90 0x7f242406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.563+0000 7f2448cad640 1 --2- 172.21.15.84:0/320061951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f244415cd70 0x7f24441475f0 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.563+0000 7f2448cad640 1 --2- 172.21.15.84:0/320061951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24440d0be0 0x7f24440cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.563+0000 7f2448cad640 1 --2- 172.21.15.84:0/320061951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24440b8480 0x7f24440b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.563+0000 7f2448cad640 1 -- 172.21.15.84:0/320061951 >> 172.21.15.84:0/320061951 conn(0x7f24440cf570 msgr2=0x7f24440cd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:22.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.563+0000 7f2448cad640 1 -- 172.21.15.84:0/320061951 shutdown_connections 2023-10-16T11:22:22.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.563+0000 7f2448cad640 1 -- 172.21.15.84:0/320061951 wait complete. 2023-10-16T11:22:22.582 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:22:22.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.825+0000 7f20d1651640 1 Processor -- start 2023-10-16T11:22:22.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.825+0000 7f20d1651640 1 -- start start 2023-10-16T11:22:22.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.827+0000 7f20d1651640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f20cc0cf5f0 0x7f20cc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.827+0000 7f20d1651640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20cc070c10 0x7f20cc0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.827+0000 7f20d1651640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20cc161740 con 0x7f20cc070830 2023-10-16T11:22:22.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.827+0000 7f20d1651640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20cc0cdbd0 con 0x7f20cc0cf5f0 2023-10-16T11:22:22.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.827+0000 7f20d1651640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc06b0f0 con 0x7f20cc070c10 2023-10-16T11:22:22.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.827+0000 7f20caffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f20cc070830 0x7f20cc06c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40884/0 (socket says 172.21.15.84:40884) 2023-10-16T11:22:22.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.827+0000 7f20caffd640 1 -- 172.21.15.84:0/429214286 learned_addr learned my addr 172.21.15.84:0/429214286 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:22.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.827+0000 7f20ca7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f20cc0cf5f0 0x7f20cc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.828+0000 7f20cb7fe640 1 --2- 172.21.15.84:0/429214286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20cc070c10 0x7f20cc0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.828+0000 7f20c9ffb640 1 -- 172.21.15.84:0/429214286 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1328636106 0 0) 0x7f20cc161740 con 0x7f20cc070830 2023-10-16T11:22:22.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.828+0000 7f20c9ffb640 1 -- 172.21.15.84:0/429214286 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20b4003520 con 0x7f20cc070830 2023-10-16T11:22:22.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.828+0000 7f20ca7fc640 1 -- 172.21.15.84:0/429214286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20cc070c10 msgr2=0x7f20cc0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:22.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.828+0000 7f20ca7fc640 1 --2- 172.21.15.84:0/429214286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20cc070c10 0x7f20cc0cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.828+0000 7f20ca7fc640 1 -- 172.21.15.84:0/429214286 >> v1:172.21.15.84:6789/0 conn(0x7f20cc070830 legacy=0x7f20cc06c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.828+0000 7f20ca7fc640 1 -- 172.21.15.84:0/429214286 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc1619b0 con 0x7f20cc0cf5f0 2023-10-16T11:22:22.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.829+0000 7f20ca7fc640 1 --2- 172.21.15.84:0/429214286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20cc0cf5f0 0x7f20cc06d3c0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f20c00097a0 tx=0x7f20c002b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a363cb6b4c042f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:22.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.829+0000 7f20c9ffb640 1 -- 172.21.15.84:0/429214286 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c00316f0 con 0x7f20cc0cf5f0 2023-10-16T11:22:22.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.829+0000 7f20c9ffb640 1 -- 172.21.15.84:0/429214286 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c0031850 con 0x7f20cc0cf5f0 2023-10-16T11:22:22.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.829+0000 7f20c9ffb640 1 -- 172.21.15.84:0/429214286 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c0031bc0 con 0x7f20cc0cf5f0 2023-10-16T11:22:22.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.830+0000 7f20d1651640 1 -- 172.21.15.84:0/429214286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20cc0cf5f0 msgr2=0x7f20cc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.830+0000 7f20d1651640 1 --2- 172.21.15.84:0/429214286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20cc0cf5f0 0x7f20cc06d3c0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f20c00097a0 tx=0x7f20c002b790 comp rx=0 tx=0).stop 2023-10-16T11:22:22.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.830+0000 7f20d1651640 1 -- 172.21.15.84:0/429214286 shutdown_connections 2023-10-16T11:22:22.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.830+0000 7f20d1651640 1 --2- 172.21.15.84:0/429214286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20cc070c10 0x7f20cc0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.830+0000 7f20d1651640 1 --2- 172.21.15.84:0/429214286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20cc0cf5f0 0x7f20cc06d3c0 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.830+0000 7f20d1651640 1 -- 172.21.15.84:0/429214286 >> 172.21.15.84:0/429214286 conn(0x7f20cc0cfac0 msgr2=0x7f20cc0cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:22.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.830+0000 7f20d1651640 1 -- 172.21.15.84:0/429214286 shutdown_connections 2023-10-16T11:22:22.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.830+0000 7f20d1651640 1 -- 172.21.15.84:0/429214286 wait complete. 2023-10-16T11:22:22.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.832+0000 7f20d1651640 1 Processor -- start 2023-10-16T11:22:22.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.832+0000 7f20d1651640 1 -- start start 2023-10-16T11:22:22.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.833+0000 7f20d1651640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20cc070830 0x7f20cc0c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.833+0000 7f20d1651640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20cc070c10 0x7f20cc0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.833+0000 7f20d1651640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20cc0cf5f0 0x7f20cc0ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.833+0000 7f20d1651640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc0c7be0 con 0x7f20cc070c10 2023-10-16T11:22:22.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.833+0000 7f20d1651640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc14d150 con 0x7f20cc0cf5f0 2023-10-16T11:22:22.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.833+0000 7f20d1651640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc14d430 con 0x7f20cc070830 2023-10-16T11:22:22.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.833+0000 7f20ca7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20cc070c10 0x7f20cc0ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.834+0000 7f20ca7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20cc070c10 0x7f20cc0ad510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53940/0 (socket says 172.21.15.84:53940) 2023-10-16T11:22:22.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.834+0000 7f20ca7fc640 1 -- 172.21.15.84:0/660754773 learned_addr learned my addr 172.21.15.84:0/660754773 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:22.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.834+0000 7f20caffd640 1 --2- 172.21.15.84:0/660754773 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20cc070830 0x7f20cc0c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.834+0000 7f20cb7fe640 1 --2- 172.21.15.84:0/660754773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20cc0cf5f0 0x7f20cc0ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.834+0000 7f20ca7fc640 1 -- 172.21.15.84:0/660754773 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20cc070830 msgr2=0x7f20cc0c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.834+0000 7f20ca7fc640 1 --2- 172.21.15.84:0/660754773 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20cc070830 0x7f20cc0c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.834+0000 7f20ca7fc640 1 -- 172.21.15.84:0/660754773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20cc0cf5f0 msgr2=0x7f20cc0ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.834+0000 7f20ca7fc640 1 --2- 172.21.15.84:0/660754773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20cc0cf5f0 0x7f20cc0ada50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.834+0000 7f20ca7fc640 1 -- 172.21.15.84:0/660754773 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c0009400 con 0x7f20cc070c10 2023-10-16T11:22:22.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.834+0000 7f20ca7fc640 1 --2- 172.21.15.84:0/660754773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20cc070c10 0x7f20cc0ad510 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f20c002b7d0 tx=0x7f20c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:22.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.834+0000 7f20b37fe640 1 -- 172.21.15.84:0/660754773 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c0004230 con 0x7f20cc070c10 2023-10-16T11:22:22.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.835+0000 7f20b37fe640 1 -- 172.21.15.84:0/660754773 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c00043b0 con 0x7f20cc070c10 2023-10-16T11:22:22.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.835+0000 7f20b37fe640 1 -- 172.21.15.84:0/660754773 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c0004520 con 0x7f20cc070c10 2023-10-16T11:22:22.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.835+0000 7f20d1651640 1 -- 172.21.15.84:0/660754773 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20cc0c7df0 con 0x7f20cc070c10 2023-10-16T11:22:22.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.835+0000 7f20d1651640 1 -- 172.21.15.84:0/660754773 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20cc0c8b30 con 0x7f20cc070c10 2023-10-16T11:22:22.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.836+0000 7f20d1651640 1 -- 172.21.15.84:0/660754773 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20cc14d5f0 con 0x7f20cc070c10 2023-10-16T11:22:22.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.837+0000 7f20b37fe640 1 -- 172.21.15.84:0/660754773 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 118) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f20c003b050 con 0x7f20cc070c10 2023-10-16T11:22:22.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.838+0000 7f20b37fe640 1 --2- 172.21.15.84:0/660754773 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f20b806af90 0x7f20b806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.839+0000 7f20caffd640 1 -- 172.21.15.84:0/660754773 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f20b806af90 msgr2=0x7f20b806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/285590107 2023-10-16T11:22:22.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.839+0000 7f20b37fe640 1 -- 172.21.15.84:0/660754773 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f20c00ab640 con 0x7f20cc070c10 2023-10-16T11:22:22.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.839+0000 7f20caffd640 1 --2- 172.21.15.84:0/660754773 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f20b806af90 0x7f20b806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:22.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:22.843+0000 7f20b37fe640 1 -- 172.21.15.84:0/660754773 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f20c0076160 con 0x7f20cc070c10 2023-10-16T11:22:22.893 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:22.891+0000 7fbdf223d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:22:22.939 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:22.938+0000 7f6ff463d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:22:23.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.040+0000 7f20caffd640 1 -- 172.21.15.84:0/660754773 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f20b806af90 msgr2=0x7f20b806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/285590107 2023-10-16T11:22:23.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.040+0000 7f20caffd640 1 --2- 172.21.15.84:0/660754773 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f20b806af90 0x7f20b806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:23.052 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:23.050+0000 7fbdf223d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:22:23.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.058+0000 7f20d1651640 1 -- 172.21.15.84:0/660754773 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f20cc04e160 con 0x7f20cc070c10 2023-10-16T11:22:23.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.059+0000 7f20b37fe640 1 -- 172.21.15.84:0/660754773 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v186) v1 ==== 100+0+11 (secure 0 0 0) 0x7f20c0079780 con 0x7f20cc070c10 2023-10-16T11:22:23.060 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:22:23.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.061+0000 7f20d1651640 1 -- 172.21.15.84:0/660754773 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f20b806af90 msgr2=0x7f20b806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:23.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.061+0000 7f20d1651640 1 --2- 172.21.15.84:0/660754773 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f20b806af90 0x7f20b806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.061+0000 7f20d1651640 1 -- 172.21.15.84:0/660754773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20cc070c10 msgr2=0x7f20cc0ad510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:23.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.061+0000 7f20d1651640 1 --2- 172.21.15.84:0/660754773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20cc070c10 0x7f20cc0ad510 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f20c002b7d0 tx=0x7f20c00046c0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.061+0000 7f20d1651640 1 -- 172.21.15.84:0/660754773 shutdown_connections 2023-10-16T11:22:23.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.061+0000 7f20d1651640 1 --2- 172.21.15.84:0/660754773 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f20b806af90 0x7f20b806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.061+0000 7f20d1651640 1 --2- 172.21.15.84:0/660754773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20cc0cf5f0 0x7f20cc0ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.061+0000 7f20d1651640 1 --2- 172.21.15.84:0/660754773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20cc070c10 0x7f20cc0ad510 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.061+0000 7f20d1651640 1 --2- 172.21.15.84:0/660754773 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20cc070830 0x7f20cc0c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.061+0000 7f20d1651640 1 -- 172.21.15.84:0/660754773 >> 172.21.15.84:0/660754773 conn(0x7f20cc0cfac0 msgr2=0x7f20cc072e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:23.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.061+0000 7f20d1651640 1 -- 172.21.15.84:0/660754773 shutdown_connections 2023-10-16T11:22:23.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.062+0000 7f20d1651640 1 -- 172.21.15.84:0/660754773 wait complete. 2023-10-16T11:22:23.080 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T11:22:23.102 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:23.101+0000 7f6ff463d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:22:23.160 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:23.158+0000 7fbdf223d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:22:23.212 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:23.210+0000 7fbdf223d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:22:23.216 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:23.215+0000 7f6ff463d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:22:23.267 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:23.266+0000 7f6ff463d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:22:23.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.270+0000 7f08b759e640 1 Processor -- start 2023-10-16T11:22:23.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.271+0000 7f08b759e640 1 -- start start 2023-10-16T11:22:23.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.272+0000 7f08b759e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b815cd70 0x7f08b80d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.272+0000 7f08b759e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08b80cded0 0x7f08b80b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.272+0000 7f08b759e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b815d150 con 0x7f08b80b8480 2023-10-16T11:22:23.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.272+0000 7f08b759e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b80d1180 con 0x7f08b80cded0 2023-10-16T11:22:23.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.272+0000 7f08b759e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80d1460 con 0x7f08b815cd70 2023-10-16T11:22:23.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.272+0000 7f08affff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f08b80b8480 0x7f08b80cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40892/0 (socket says 172.21.15.84:40892) 2023-10-16T11:22:23.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.272+0000 7f08affff640 1 -- 172.21.15.84:0/3209887956 learned_addr learned my addr 172.21.15.84:0/3209887956 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:23.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.273+0000 7f08b659c640 1 --2- 172.21.15.84:0/3209887956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b815cd70 0x7f08b80d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:23.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.273+0000 7f08b6d9d640 1 --2- 172.21.15.84:0/3209887956 >> v2:172.21.15.157:3300/0 conn(0x7f08b80cded0 0x7f08b80b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:23.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.273+0000 7f08b5d9b640 1 -- 172.21.15.84:0/3209887956 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016057802 0 0) 0x7f08b815d150 con 0x7f08b80b8480 2023-10-16T11:22:23.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.273+0000 7f08b5d9b640 1 -- 172.21.15.84:0/3209887956 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08a4003520 con 0x7f08b80b8480 2023-10-16T11:22:23.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.273+0000 7f08b5d9b640 1 -- 172.21.15.84:0/3209887956 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1991542848 0 0) 0x7f08a4003520 con 0x7f08b80b8480 2023-10-16T11:22:23.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.273+0000 7f08b5d9b640 1 -- 172.21.15.84:0/3209887956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b815cd70 msgr2=0x7f08b80d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:23.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.273+0000 7f08b5d9b640 1 --2- 172.21.15.84:0/3209887956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b815cd70 0x7f08b80d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.273+0000 7f08b5d9b640 1 -- 172.21.15.84:0/3209887956 >> v2:172.21.15.157:3300/0 conn(0x7f08b80cded0 msgr2=0x7f08b80b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:23.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.273+0000 7f08b5d9b640 1 --2- 172.21.15.84:0/3209887956 >> v2:172.21.15.157:3300/0 conn(0x7f08b80cded0 0x7f08b80b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.273+0000 7f08b5d9b640 1 -- 172.21.15.84:0/3209887956 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b80d1740 con 0x7f08b80b8480 2023-10-16T11:22:23.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.273+0000 7f08b5d9b640 1 -- 172.21.15.84:0/3209887956 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f08a0001670 con 0x7f08b80b8480 2023-10-16T11:22:23.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.273+0000 7f08b5d9b640 1 -- 172.21.15.84:0/3209887956 >> v1:172.21.15.84:6789/0 conn(0x7f08b80b8480 legacy=0x7f08b80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:23.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.273+0000 7f08b5d9b640 1 --2- 172.21.15.84:0/3209887956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08a4003970 0x7f08a4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.273+0000 7f08b5d9b640 1 --2- 172.21.15.84:0/3209887956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b815cd70 0x7f08a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.273+0000 7f08b5d9b640 1 --2- 172.21.15.84:0/3209887956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a4004d00 0x7f08a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.273+0000 7f08b5d9b640 1 -- 172.21.15.84:0/3209887956 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80d1180 con 0x7f08a4003970 2023-10-16T11:22:23.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.273+0000 7f08b5d9b640 1 -- 172.21.15.84:0/3209887956 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80d1460 con 0x7f08a4004d00 2023-10-16T11:22:23.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.273+0000 7f08b5d9b640 1 -- 172.21.15.84:0/3209887956 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a400b980 con 0x7f08b815cd70 2023-10-16T11:22:23.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.274+0000 7f08b659c640 1 --2- 172.21.15.84:0/3209887956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08a4003970 0x7f08a4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:23.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.274+0000 7f08affff640 1 --2- 172.21.15.84:0/3209887956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b815cd70 0x7f08a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:23.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.274+0000 7f08b659c640 1 -- 172.21.15.84:0/3209887956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b815cd70 msgr2=0x7f08a40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:23.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.274+0000 7f08b6d9d640 1 --2- 172.21.15.84:0/3209887956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a4004d00 0x7f08a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:23.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.274+0000 7f08b659c640 1 --2- 172.21.15.84:0/3209887956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b815cd70 0x7f08a40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.274+0000 7f08b659c640 1 -- 172.21.15.84:0/3209887956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a4004d00 msgr2=0x7f08a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:23.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.274+0000 7f08b659c640 1 --2- 172.21.15.84:0/3209887956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a4004d00 0x7f08a4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.274+0000 7f08b659c640 1 -- 172.21.15.84:0/3209887956 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4003520 con 0x7f08a4003970 2023-10-16T11:22:23.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.275+0000 7f08b659c640 1 --2- 172.21.15.84:0/3209887956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08a4003970 0x7f08a4003d90 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009ac0 tx=0x7f08a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1d7f2c2e9c4ee5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:23.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.275+0000 7f08b5d9b640 1 -- 172.21.15.84:0/3209887956 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a802c6b0 con 0x7f08a4003970 2023-10-16T11:22:23.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.275+0000 7f08b5d9b640 1 -- 172.21.15.84:0/3209887956 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a802c810 con 0x7f08a4003970 2023-10-16T11:22:23.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.275+0000 7f08b5d9b640 1 -- 172.21.15.84:0/3209887956 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a802cba0 con 0x7f08a4003970 2023-10-16T11:22:23.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.276+0000 7f08b759e640 1 -- 172.21.15.84:0/3209887956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08a4003970 msgr2=0x7f08a4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:23.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.276+0000 7f08b759e640 1 --2- 172.21.15.84:0/3209887956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08a4003970 0x7f08a4003d90 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009ac0 tx=0x7f08a40057f0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.276+0000 7f08b6d9d640 1 -- 172.21.15.84:0/3209887956 reap_dead start 2023-10-16T11:22:23.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.276+0000 7f08b759e640 1 -- 172.21.15.84:0/3209887956 shutdown_connections 2023-10-16T11:22:23.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.276+0000 7f08b759e640 1 -- 172.21.15.84:0/3209887956 >> 172.21.15.84:0/3209887956 conn(0x7f08b80cf570 msgr2=0x7f08b80cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:23.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.276+0000 7f08b759e640 1 -- 172.21.15.84:0/3209887956 shutdown_connections 2023-10-16T11:22:23.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.276+0000 7f08b759e640 1 -- 172.21.15.84:0/3209887956 wait complete. 2023-10-16T11:22:23.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.278+0000 7f08b759e640 1 Processor -- start 2023-10-16T11:22:23.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.278+0000 7f08b759e640 1 -- start start 2023-10-16T11:22:23.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.279+0000 7f08b759e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08a4003970 0x7f08b80aba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.279+0000 7f08b759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08b80cc880 0x7f08b80c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.279+0000 7f08b759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b8159060 0x7f08b80b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.279+0000 7f08b759e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80cb040 con 0x7f08b80cc880 2023-10-16T11:22:23.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.279+0000 7f08b759e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80ca410 con 0x7f08b8159060 2023-10-16T11:22:23.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.279+0000 7f08b759e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80ca6f0 con 0x7f08a4003970 2023-10-16T11:22:23.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.279+0000 7f08affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08b80cc880 0x7f08b80c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:23.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.279+0000 7f08affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08b80cc880 0x7f08b80c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53964/0 (socket says 172.21.15.84:53964) 2023-10-16T11:22:23.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.279+0000 7f08affff640 1 -- 172.21.15.84:0/1914864130 learned_addr learned my addr 172.21.15.84:0/1914864130 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:23.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.279+0000 7f08b659c640 1 --2- 172.21.15.84:0/1914864130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08a4003970 0x7f08b80aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:23.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.279+0000 7f08affff640 1 -- 172.21.15.84:0/1914864130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08a4003970 msgr2=0x7f08b80aba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:23.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.279+0000 7f08affff640 1 --2- 172.21.15.84:0/1914864130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08a4003970 0x7f08b80aba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.279+0000 7f08affff640 1 -- 172.21.15.84:0/1914864130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b8159060 msgr2=0x7f08b80b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:23.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.279+0000 7f08affff640 1 --2- 172.21.15.84:0/1914864130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b8159060 0x7f08b80b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.279+0000 7f08affff640 1 -- 172.21.15.84:0/1914864130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b80ca9d0 con 0x7f08b80cc880 2023-10-16T11:22:23.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.280+0000 7f08affff640 1 --2- 172.21.15.84:0/1914864130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08b80cc880 0x7f08b80c9bc0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f08a000d700 tx=0x7f08a000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:23.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.280+0000 7f08aeffd640 1 -- 172.21.15.84:0/1914864130 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a00074e0 con 0x7f08b80cc880 2023-10-16T11:22:23.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.280+0000 7f08aeffd640 1 -- 172.21.15.84:0/1914864130 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a0007640 con 0x7f08b80cc880 2023-10-16T11:22:23.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.280+0000 7f08aeffd640 1 -- 172.21.15.84:0/1914864130 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0004a00 con 0x7f08b80cc880 2023-10-16T11:22:23.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.280+0000 7f08b759e640 1 -- 172.21.15.84:0/1914864130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b80cacb0 con 0x7f08b80cc880 2023-10-16T11:22:23.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.280+0000 7f08b759e640 1 -- 172.21.15.84:0/1914864130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b81594e0 con 0x7f08b80cc880 2023-10-16T11:22:23.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.281+0000 7f08b759e640 1 -- 172.21.15.84:0/1914864130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b804e9b0 con 0x7f08b80cc880 2023-10-16T11:22:23.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.282+0000 7f08aeffd640 1 -- 172.21.15.84:0/1914864130 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 118) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f08a0004b60 con 0x7f08b80cc880 2023-10-16T11:22:23.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.283+0000 7f08aeffd640 1 --2- 172.21.15.84:0/1914864130 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f089806af90 0x7f089806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.283+0000 7f08aeffd640 1 -- 172.21.15.84:0/1914864130 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f08a0089390 con 0x7f08b80cc880 2023-10-16T11:22:23.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.283+0000 7f08b659c640 1 -- 172.21.15.84:0/1914864130 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f089806af90 msgr2=0x7f089806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/285590107 2023-10-16T11:22:23.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.283+0000 7f08b659c640 1 --2- 172.21.15.84:0/1914864130 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f089806af90 0x7f089806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:23.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.287+0000 7f08aeffd640 1 -- 172.21.15.84:0/1914864130 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f08a0053f40 con 0x7f08b80cc880 2023-10-16T11:22:23.322 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:23.320+0000 7fbdf223d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:22:23.381 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:23.380+0000 7f6ff463d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:22:23.431 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:23.429+0000 7fbdf223d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:22:23.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.484+0000 7f08b659c640 1 -- 172.21.15.84:0/1914864130 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f089806af90 msgr2=0x7f089806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/285590107 2023-10-16T11:22:23.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.484+0000 7f08b659c640 1 --2- 172.21.15.84:0/1914864130 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f089806af90 0x7f089806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:23.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.486+0000 7f08b759e640 1 -- 172.21.15.84:0/1914864130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f08b80ced70 con 0x7f08b80cc880 2023-10-16T11:22:23.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.487+0000 7f08aeffd640 1 -- 172.21.15.84:0/1914864130 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v186) v1 ==== 81+0+937 (secure 0 0 0) 0x7f08a0057560 con 0x7f08b80cc880 2023-10-16T11:22:23.488 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/542522615 2023-10-17T11:17:21.065656+0000 2023-10-16T11:22:23.488 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/878936990 2023-10-17T11:17:21.065656+0000 2023-10-16T11:22:23.488 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6833/3993913200 2023-10-17T11:17:21.065656+0000 2023-10-16T11:22:23.488 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6832/3993913200 2023-10-17T11:17:21.065656+0000 2023-10-16T11:22:23.489 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1064178200 2023-10-17T11:17:21.065656+0000 2023-10-16T11:22:23.489 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3548457413 2023-10-17T11:06:37.825968+0000 2023-10-16T11:22:23.489 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2013731252 2023-10-17T11:17:21.065656+0000 2023-10-16T11:22:23.489 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6833/1441051579 2023-10-17T11:06:37.825968+0000 2023-10-16T11:22:23.489 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6832/1441051579 2023-10-17T11:06:37.825968+0000 2023-10-16T11:22:23.490 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2359294858 2023-10-17T11:17:21.065656+0000 2023-10-16T11:22:23.490 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2830443959 2023-10-17T11:06:37.825968+0000 2023-10-16T11:22:23.490 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/4150685824 2023-10-17T11:06:37.825968+0000 2023-10-16T11:22:23.490 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/4095400472 2023-10-17T11:17:21.065656+0000 2023-10-16T11:22:23.490 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3348937827 2023-10-17T11:06:37.825968+0000 2023-10-16T11:22:23.491 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2158345505 2023-10-17T11:06:37.825968+0000 2023-10-16T11:22:23.491 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/235070477 2023-10-17T11:06:37.825968+0000 2023-10-16T11:22:23.492 INFO:teuthology.orchestra.run.smithi084.stderr:listed 16 entries 2023-10-16T11:22:23.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.489+0000 7f08b759e640 1 -- 172.21.15.84:0/1914864130 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f089806af90 msgr2=0x7f089806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:23.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.489+0000 7f08b759e640 1 --2- 172.21.15.84:0/1914864130 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f089806af90 0x7f089806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.489+0000 7f08b759e640 1 -- 172.21.15.84:0/1914864130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08b80cc880 msgr2=0x7f08b80c9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:23.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.489+0000 7f08b759e640 1 --2- 172.21.15.84:0/1914864130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08b80cc880 0x7f08b80c9bc0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f08a000d700 tx=0x7f08a000dbc0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.489+0000 7f08b759e640 1 -- 172.21.15.84:0/1914864130 shutdown_connections 2023-10-16T11:22:23.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.489+0000 7f08b759e640 1 --2- 172.21.15.84:0/1914864130 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7f089806af90 0x7f089806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.489+0000 7f08b759e640 1 --2- 172.21.15.84:0/1914864130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b8159060 0x7f08b80b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.489+0000 7f08b759e640 1 --2- 172.21.15.84:0/1914864130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08b80cc880 0x7f08b80c9bc0 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.489+0000 7f08b759e640 1 --2- 172.21.15.84:0/1914864130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08a4003970 0x7f08b80aba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.489+0000 7f08b759e640 1 -- 172.21.15.84:0/1914864130 >> 172.21.15.84:0/1914864130 conn(0x7f08b80cf570 msgr2=0x7f08b80b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:23.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.489+0000 7f08b759e640 1 -- 172.21.15.84:0/1914864130 shutdown_connections 2023-10-16T11:22:23.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:23.489+0000 7f08b759e640 1 -- 172.21.15.84:0/1914864130 wait complete. 2023-10-16T11:22:23.495 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:23.490+0000 7f6ff463d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:22:23.507 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T11:22:23.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:23.508 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T11:22:23.587 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:23.584+0000 7fbdf223d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:22:23.636 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:23.634+0000 7fbdf223d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:22:23.646 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T11:22:23.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:23.647 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T11:22:23.660 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:23.660+0000 7f6ff463d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:22:23.693 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:22:23.693 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:23.712 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:23.712+0000 7f6ff463d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:22:23.747 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:22:23.748 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:22:23.748 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:23.765 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:23.763+0000 7fbdf223d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:22:23.800 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:22:23.801 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:22:23.801 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:23.840 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:23.839+0000 7f6ff463d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:22:23.857 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:22:23.857 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:22:24.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.088+0000 7ff48c301640 1 Processor -- start 2023-10-16T11:22:24.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.088+0000 7ff48c301640 1 -- start start 2023-10-16T11:22:24.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.089+0000 7ff48c301640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff48415ae80 0x7ff48415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:24.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.089+0000 7ff48c301640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff4840ce390 0x7ff484070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:24.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.089+0000 7ff48c301640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48415ed60 con 0x7ff48415aaa0 2023-10-16T11:22:24.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.089+0000 7ff48c301640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48406b810 con 0x7ff48415ae80 2023-10-16T11:22:24.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.089+0000 7ff48c301640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4840707e0 con 0x7ff4840ce390 2023-10-16T11:22:24.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.089+0000 7ff48a09d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff48415aaa0 0x7ff4840cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40896/0 (socket says 172.21.15.84:40896) 2023-10-16T11:22:24.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.089+0000 7ff48a09d640 1 -- 172.21.15.84:0/3041481419 learned_addr learned my addr 172.21.15.84:0/3041481419 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:24.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.089+0000 7ff48989c640 1 --2- 172.21.15.84:0/3041481419 >> v2:172.21.15.84:3300/0 conn(0x7ff48415ae80 0x7ff48415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:24.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.089+0000 7ff48a89e640 1 --2- 172.21.15.84:0/3041481419 >> v2:172.21.15.157:3300/0 conn(0x7ff4840ce390 0x7ff484070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:24.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.089+0000 7ff48909b640 1 -- 172.21.15.84:0/3041481419 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844506625 0 0) 0x7ff48415ed60 con 0x7ff48415aaa0 2023-10-16T11:22:24.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.090+0000 7ff48909b640 1 -- 172.21.15.84:0/3041481419 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff470003520 con 0x7ff48415aaa0 2023-10-16T11:22:24.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.090+0000 7ff48989c640 1 -- 172.21.15.84:0/3041481419 >> v2:172.21.15.157:3300/0 conn(0x7ff4840ce390 msgr2=0x7ff484070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:24.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.090+0000 7ff48989c640 1 --2- 172.21.15.84:0/3041481419 >> v2:172.21.15.157:3300/0 conn(0x7ff4840ce390 0x7ff484070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.090+0000 7ff48989c640 1 -- 172.21.15.84:0/3041481419 >> v1:172.21.15.84:6789/0 conn(0x7ff48415aaa0 legacy=0x7ff4840cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:24.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.090+0000 7ff48989c640 1 -- 172.21.15.84:0/3041481419 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48415ef90 con 0x7ff48415ae80 2023-10-16T11:22:24.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.090+0000 7ff48a89e640 1 --2- 172.21.15.84:0/3041481419 >> v2:172.21.15.157:3300/0 conn(0x7ff4840ce390 0x7ff484070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:24.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.091+0000 7ff48989c640 1 --2- 172.21.15.84:0/3041481419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48415ae80 0x7ff48415e820 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7ff4840c2490 tx=0x7ff47402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c299fb840c3a9842 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:24.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.091+0000 7ff48909b640 1 -- 172.21.15.84:0/3041481419 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff474031830 con 0x7ff48415ae80 2023-10-16T11:22:24.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.091+0000 7ff48909b640 1 -- 172.21.15.84:0/3041481419 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff474031990 con 0x7ff48415ae80 2023-10-16T11:22:24.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.091+0000 7ff48909b640 1 -- 172.21.15.84:0/3041481419 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff474031d20 con 0x7ff48415ae80 2023-10-16T11:22:24.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.091+0000 7ff48c301640 1 -- 172.21.15.84:0/3041481419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48415ae80 msgr2=0x7ff48415e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:24.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.091+0000 7ff48c301640 1 --2- 172.21.15.84:0/3041481419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48415ae80 0x7ff48415e820 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7ff4840c2490 tx=0x7ff47402b8d0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.091+0000 7ff48c301640 1 -- 172.21.15.84:0/3041481419 shutdown_connections 2023-10-16T11:22:24.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.091+0000 7ff48c301640 1 --2- 172.21.15.84:0/3041481419 >> v2:172.21.15.157:3300/0 conn(0x7ff4840ce390 0x7ff484070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.091+0000 7ff48c301640 1 --2- 172.21.15.84:0/3041481419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48415ae80 0x7ff48415e820 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.091+0000 7ff48c301640 1 -- 172.21.15.84:0/3041481419 >> 172.21.15.84:0/3041481419 conn(0x7ff4840cb410 msgr2=0x7ff4840cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:24.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.092+0000 7ff48c301640 1 -- 172.21.15.84:0/3041481419 shutdown_connections 2023-10-16T11:22:24.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.092+0000 7ff48c301640 1 -- 172.21.15.84:0/3041481419 wait complete. 2023-10-16T11:22:24.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.093+0000 7ff48c301640 1 Processor -- start 2023-10-16T11:22:24.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.093+0000 7ff48c301640 1 -- start start 2023-10-16T11:22:24.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.094+0000 7ff48c301640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4840ce390 0x7ff4840ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:24.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.094+0000 7ff48c301640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff48415aaa0 0x7ff48415ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:24.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.094+0000 7ff48c301640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48415ae80 0x7ff48414cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:24.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.094+0000 7ff48c301640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48414a8c0 con 0x7ff48415ae80 2023-10-16T11:22:24.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.094+0000 7ff48c301640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48414ad10 con 0x7ff4840ce390 2023-10-16T11:22:24.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.094+0000 7ff48c301640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48415de80 con 0x7ff48415aaa0 2023-10-16T11:22:24.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.094+0000 7ff48a89e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48415ae80 0x7ff48414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:24.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.095+0000 7ff48a89e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48415ae80 0x7ff48414cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53976/0 (socket says 172.21.15.84:53976) 2023-10-16T11:22:24.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.095+0000 7ff48a89e640 1 -- 172.21.15.84:0/1140916328 learned_addr learned my addr 172.21.15.84:0/1140916328 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:24.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.095+0000 7ff48a09d640 1 --2- 172.21.15.84:0/1140916328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4840ce390 0x7ff4840ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:24.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.095+0000 7ff48a89e640 1 -- 172.21.15.84:0/1140916328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff48415aaa0 msgr2=0x7ff48415ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:24.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.095+0000 7ff48a89e640 1 --2- 172.21.15.84:0/1140916328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff48415aaa0 0x7ff48415ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.095+0000 7ff48a89e640 1 -- 172.21.15.84:0/1140916328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4840ce390 msgr2=0x7ff4840ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:24.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.095+0000 7ff48a89e640 1 --2- 172.21.15.84:0/1140916328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4840ce390 0x7ff4840ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.095+0000 7ff48a89e640 1 -- 172.21.15.84:0/1140916328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff474009400 con 0x7ff48415ae80 2023-10-16T11:22:24.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.095+0000 7ff48a09d640 1 --2- 172.21.15.84:0/1140916328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4840ce390 0x7ff4840ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:24.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.095+0000 7ff48a89e640 1 --2- 172.21.15.84:0/1140916328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48415ae80 0x7ff48414cbc0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7ff47c00cf50 tx=0x7ff47c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:24.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.095+0000 7ff47affd640 1 -- 172.21.15.84:0/1140916328 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c00d840 con 0x7ff48415ae80 2023-10-16T11:22:24.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.095+0000 7ff47affd640 1 -- 172.21.15.84:0/1140916328 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c00d9a0 con 0x7ff48415ae80 2023-10-16T11:22:24.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.095+0000 7ff47affd640 1 -- 172.21.15.84:0/1140916328 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c00dd50 con 0x7ff48415ae80 2023-10-16T11:22:24.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.095+0000 7ff48c301640 1 -- 172.21.15.84:0/1140916328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48415d350 con 0x7ff48415ae80 2023-10-16T11:22:24.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.096+0000 7ff48c301640 1 -- 172.21.15.84:0/1140916328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48415d8c0 con 0x7ff48415ae80 2023-10-16T11:22:24.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.097+0000 7ff48c301640 1 -- 172.21.15.84:0/1140916328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48404e160 con 0x7ff48415ae80 2023-10-16T11:22:24.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.098+0000 7ff47affd640 1 -- 172.21.15.84:0/1140916328 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 118) v1 ==== 89343+0+0 (secure 0 0 0) 0x7ff47c01e3b0 con 0x7ff48415ae80 2023-10-16T11:22:24.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.099+0000 7ff47affd640 1 --2- 172.21.15.84:0/1140916328 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7ff46806af90 0x7ff46806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:24.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.099+0000 7ff47affd640 1 -- 172.21.15.84:0/1140916328 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6513+0+0 (secure 0 0 0) 0x7ff47c08a580 con 0x7ff48415ae80 2023-10-16T11:22:24.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.099+0000 7ff48a09d640 1 -- 172.21.15.84:0/1140916328 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7ff46806af90 msgr2=0x7ff46806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/285590107 2023-10-16T11:22:24.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.099+0000 7ff48a09d640 1 --2- 172.21.15.84:0/1140916328 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7ff46806af90 0x7ff46806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:24.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.103+0000 7ff47affd640 1 -- 172.21.15.84:0/1140916328 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff47c0551f0 con 0x7ff48415ae80 2023-10-16T11:22:24.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.300+0000 7ff48a09d640 1 -- 172.21.15.84:0/1140916328 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7ff46806af90 msgr2=0x7ff46806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/285590107 2023-10-16T11:22:24.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.300+0000 7ff48a09d640 1 --2- 172.21.15.84:0/1140916328 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7ff46806af90 0x7ff46806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:24.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.350+0000 7ff48c301640 1 -- 172.21.15.84:0/1140916328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff48414ae50 con 0x7ff48415ae80 2023-10-16T11:22:24.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.350+0000 7ff47affd640 1 -- 172.21.15.84:0/1140916328 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4927 (secure 0 0 0) 0x7ff47c058810 con 0x7ff48415ae80 2023-10-16T11:22:24.352 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:24.352 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6347,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1346424254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1346424254},{"type":"v1","addr":"172.21.15.157:6835","nonce":1346424254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3224476768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3224476768},{"type":"v1","addr":"172.21.15.157:6834","nonce":3224476768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":10197,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2490106026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2490106026},{"type":"v1","addr":"172.21.15.84:6835","nonce":2490106026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:06:23.034809+0000","modified":"2023-10-16T11:06:26.105994+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10206},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10206":{"gid":10206,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/1106170430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":1106170430},{"type":"v1","addr":"172.21.15.84:6837","nonce":1106170430}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-16T11:22:24.352 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 96 2023-10-16T11:22:24.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.353+0000 7ff48c301640 1 -- 172.21.15.84:0/1140916328 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7ff46806af90 msgr2=0x7ff46806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:24.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.353+0000 7ff48c301640 1 --2- 172.21.15.84:0/1140916328 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7ff46806af90 0x7ff46806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.353+0000 7ff48c301640 1 -- 172.21.15.84:0/1140916328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48415ae80 msgr2=0x7ff48414cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:24.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.353+0000 7ff48c301640 1 --2- 172.21.15.84:0/1140916328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48415ae80 0x7ff48414cbc0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7ff47c00cf50 tx=0x7ff47c00a7d0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.353+0000 7ff48c301640 1 -- 172.21.15.84:0/1140916328 shutdown_connections 2023-10-16T11:22:24.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.353+0000 7ff48c301640 1 --2- 172.21.15.84:0/1140916328 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7ff46806af90 0x7ff46806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.353+0000 7ff48c301640 1 --2- 172.21.15.84:0/1140916328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48415ae80 0x7ff48414cbc0 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.353+0000 7ff48c301640 1 --2- 172.21.15.84:0/1140916328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff48415aaa0 0x7ff48415ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.353+0000 7ff48c301640 1 --2- 172.21.15.84:0/1140916328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4840ce390 0x7ff4840ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.353+0000 7ff48c301640 1 -- 172.21.15.84:0/1140916328 >> 172.21.15.84:0/1140916328 conn(0x7ff4840cb410 msgr2=0x7ff4840b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:24.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.353+0000 7ff48c301640 1 -- 172.21.15.84:0/1140916328 shutdown_connections 2023-10-16T11:22:24.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.353+0000 7ff48c301640 1 -- 172.21.15.84:0/1140916328 wait complete. 2023-10-16T11:22:24.372 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:22:24.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.580+0000 7fa6d79e7640 1 Processor -- start 2023-10-16T11:22:24.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.581+0000 7fa6d79e7640 1 -- start start 2023-10-16T11:22:24.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.582+0000 7fa6d79e7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa6d0159b20 0x7fa6d006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.582+0000 7fa6d79e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6d0159f00 0x7fa6d00b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.582+0000 7fa6d79e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d006c120 0x7fa6d00b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.582+0000 7fa6d79e7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d006f7f0 con 0x7fa6d0159b20 2023-10-16T11:22:24.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.582+0000 7fa6d79e7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d006c5e0 con 0x7fa6d0159f00 2023-10-16T11:22:24.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.582+0000 7fa6d79e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d006c8c0 con 0x7fa6d006c120 2023-10-16T11:22:24.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.582+0000 7fa6d5783640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa6d0159b20 0x7fa6d006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:24.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.582+0000 7fa6d5783640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa6d0159b20 0x7fa6d006bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53990/0 (socket says 172.21.15.84:53990) 2023-10-16T11:22:24.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.582+0000 7fa6d5783640 1 -- 172.21.15.84:0/2522389346 learned_addr learned my addr 172.21.15.84:0/2522389346 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:24.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.582+0000 7fa6d4f82640 1 --2- 172.21.15.84:0/2522389346 >> v2:172.21.15.157:3300/0 conn(0x7fa6d0159f00 0x7fa6d00b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:24.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.582+0000 7fa6d5f84640 1 --2- 172.21.15.84:0/2522389346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d006c120 0x7fa6d00b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:24.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.582+0000 7fa6d5783640 1 -- 172.21.15.84:0/2522389346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d006c120 msgr2=0x7fa6d00b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:24.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.582+0000 7fa6d5783640 1 --2- 172.21.15.84:0/2522389346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d006c120 0x7fa6d00b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.582+0000 7fa6d5783640 1 -- 172.21.15.84:0/2522389346 >> v2:172.21.15.157:3300/0 conn(0x7fa6d0159f00 msgr2=0x7fa6d00b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:24.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.582+0000 7fa6d5783640 1 --2- 172.21.15.84:0/2522389346 >> v2:172.21.15.157:3300/0 conn(0x7fa6d0159f00 0x7fa6d00b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.582+0000 7fa6d5783640 1 -- 172.21.15.84:0/2522389346 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d006cba0 con 0x7fa6d0159b20 2023-10-16T11:22:24.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.583+0000 7fa6d5783640 1 --2- 172.21.15.84:0/2522389346 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6d0159b20 0x7fa6d006bbe0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fa6d015d770 tx=0x7fa6cc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1364649dfc1b4b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:24.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.584+0000 7fa6c7fff640 1 -- 172.21.15.84:0/2522389346 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc031870 con 0x7fa6d0159b20 2023-10-16T11:22:24.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.584+0000 7fa6c7fff640 1 -- 172.21.15.84:0/2522389346 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc0319d0 con 0x7fa6d0159b20 2023-10-16T11:22:24.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.584+0000 7fa6c7fff640 1 -- 172.21.15.84:0/2522389346 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc031d60 con 0x7fa6d0159b20 2023-10-16T11:22:24.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.585+0000 7fa6d79e7640 1 -- 172.21.15.84:0/2522389346 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6d0159b20 msgr2=0x7fa6d006bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:24.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.585+0000 7fa6d79e7640 1 --2- 172.21.15.84:0/2522389346 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6d0159b20 0x7fa6d006bbe0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fa6d015d770 tx=0x7fa6cc02b910 comp rx=0 tx=0).stop 2023-10-16T11:22:24.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.585+0000 7fa6d79e7640 1 -- 172.21.15.84:0/2522389346 shutdown_connections 2023-10-16T11:22:24.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.585+0000 7fa6d79e7640 1 --2- 172.21.15.84:0/2522389346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d006c120 0x7fa6d00b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.585+0000 7fa6d79e7640 1 --2- 172.21.15.84:0/2522389346 >> v2:172.21.15.157:3300/0 conn(0x7fa6d0159f00 0x7fa6d00b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.585+0000 7fa6d79e7640 1 --2- 172.21.15.84:0/2522389346 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6d0159b20 0x7fa6d006bbe0 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.585+0000 7fa6d79e7640 1 -- 172.21.15.84:0/2522389346 >> 172.21.15.84:0/2522389346 conn(0x7fa6d00cb410 msgr2=0x7fa6d00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:24.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.585+0000 7fa6d79e7640 1 -- 172.21.15.84:0/2522389346 shutdown_connections 2023-10-16T11:22:24.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.585+0000 7fa6d79e7640 1 -- 172.21.15.84:0/2522389346 wait complete. 2023-10-16T11:22:24.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.587+0000 7fa6d79e7640 1 Processor -- start 2023-10-16T11:22:24.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.588+0000 7fa6d79e7640 1 -- start start 2023-10-16T11:22:24.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.589+0000 7fa6d79e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d006c120 0x7fa6d00b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:24.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.589+0000 7fa6d79e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d0159b20 0x7fa6d0148d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:24.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.589+0000 7fa6d79e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6d0159f00 0x7fa6d00b2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:24.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.589+0000 7fa6d79e7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d00b1ca0 con 0x7fa6d0159f00 2023-10-16T11:22:24.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.589+0000 7fa6d79e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d00b40e0 con 0x7fa6d0159b20 2023-10-16T11:22:24.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.589+0000 7fa6d79e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d00b1370 con 0x7fa6d006c120 2023-10-16T11:22:24.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.589+0000 7fa6d5f84640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6d0159f00 0x7fa6d00b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:24.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.589+0000 7fa6d5f84640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6d0159f00 0x7fa6d00b2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53998/0 (socket says 172.21.15.84:53998) 2023-10-16T11:22:24.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.589+0000 7fa6d5f84640 1 -- 172.21.15.84:0/325492624 learned_addr learned my addr 172.21.15.84:0/325492624 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:24.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.590+0000 7fa6d4f82640 1 --2- 172.21.15.84:0/325492624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d0159b20 0x7fa6d0148d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:24.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.590+0000 7fa6d5783640 1 --2- 172.21.15.84:0/325492624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d006c120 0x7fa6d00b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:24.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.590+0000 7fa6d5f84640 1 -- 172.21.15.84:0/325492624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d006c120 msgr2=0x7fa6d00b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:24.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.590+0000 7fa6d5f84640 1 --2- 172.21.15.84:0/325492624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d006c120 0x7fa6d00b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.590+0000 7fa6d5f84640 1 -- 172.21.15.84:0/325492624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d0159b20 msgr2=0x7fa6d0148d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:24.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.590+0000 7fa6d5f84640 1 --2- 172.21.15.84:0/325492624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d0159b20 0x7fa6d0148d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.590+0000 7fa6d5f84640 1 -- 172.21.15.84:0/325492624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6cc0094d0 con 0x7fa6d0159f00 2023-10-16T11:22:24.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.590+0000 7fa6d4f82640 1 --2- 172.21.15.84:0/325492624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d0159b20 0x7fa6d0148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:24.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.590+0000 7fa6d5f84640 1 --2- 172.21.15.84:0/325492624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6d0159f00 0x7fa6d00b2130 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009fd0 tx=0x7fa6c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:24.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.590+0000 7fa6c5ffb640 1 -- 172.21.15.84:0/325492624 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8014420 con 0x7fa6d0159f00 2023-10-16T11:22:24.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.590+0000 7fa6c5ffb640 1 -- 172.21.15.84:0/325492624 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c8014580 con 0x7fa6d0159f00 2023-10-16T11:22:24.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.590+0000 7fa6c5ffb640 1 -- 172.21.15.84:0/325492624 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8014930 con 0x7fa6d0159f00 2023-10-16T11:22:24.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.590+0000 7fa6d79e7640 1 -- 172.21.15.84:0/325492624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d00b1650 con 0x7fa6d0159f00 2023-10-16T11:22:24.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.591+0000 7fa6d79e7640 1 -- 172.21.15.84:0/325492624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d00b2670 con 0x7fa6d0159f00 2023-10-16T11:22:24.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.592+0000 7fa6d79e7640 1 -- 172.21.15.84:0/325492624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d00b1e50 con 0x7fa6d0159f00 2023-10-16T11:22:24.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.593+0000 7fa6c5ffb640 1 -- 172.21.15.84:0/325492624 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 118) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fa6c8014a90 con 0x7fa6d0159f00 2023-10-16T11:22:24.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.593+0000 7fa6c5ffb640 1 --2- 172.21.15.84:0/325492624 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fa6b406af90 0x7fa6b406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:24.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.594+0000 7fa6c5ffb640 1 -- 172.21.15.84:0/325492624 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6513+0+0 (secure 0 0 0) 0x7fa6c808a470 con 0x7fa6d0159f00 2023-10-16T11:22:24.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.594+0000 7fa6d5783640 1 -- 172.21.15.84:0/325492624 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fa6b406af90 msgr2=0x7fa6b406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/285590107 2023-10-16T11:22:24.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.594+0000 7fa6d5783640 1 --2- 172.21.15.84:0/325492624 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fa6b406af90 0x7fa6b406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:24.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.598+0000 7fa6c5ffb640 1 -- 172.21.15.84:0/325492624 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa6c8055040 con 0x7fa6d0159f00 2023-10-16T11:22:24.675 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:24.667+0000 7fbdf223d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:22:24.748 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:24.747+0000 7f6ff463d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:22:24.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.785+0000 7fa6d79e7640 1 -- 172.21.15.84:0/325492624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6d004e160 con 0x7fa6d0159f00 2023-10-16T11:22:24.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.787+0000 7fa6c5ffb640 1 -- 172.21.15.84:0/325492624 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+17756 (secure 0 0 0) 0x7fa6c8058660 con 0x7fa6d0159f00 2023-10-16T11:22:24.789 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:24.789 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":187,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:22:23.931491+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:06:20.873257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-16T11:06:22.065944+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":32,"snap_epoch":187,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/542522615":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/878936990":"2023-10-17T11:17:21.065656+0000","172.21.15.84:6833/3993913200":"2023-10-17T11:17:21.065656+0000","172.21.15.84:6832/3993913200":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/1064178200":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/3548457413":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/2013731252":"2023-10-17T11:17:21.065656+0000","172.21.15.84:6833/1441051579":"2023-10-17T11:06:37.825968+0000","172.21.15.84:6832/1441051579":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/2359294858":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/2830443959":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/4150685824":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/4095400472":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/3348937827":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/2158345505":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/235070477":"2023-10-17T11:06:37.825968+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":23,"snaps":[{"begin":2,"length":15},{"begin":18,"length":15}]}],"new_removed_snaps":[{"pool":23,"snaps":[{"begin":2,"length":15},{"begin":18,"length":15}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:22:24.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.790+0000 7fa6d79e7640 1 -- 172.21.15.84:0/325492624 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fa6b406af90 msgr2=0x7fa6b406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:24.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.790+0000 7fa6d79e7640 1 --2- 172.21.15.84:0/325492624 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fa6b406af90 0x7fa6b406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.790+0000 7fa6d79e7640 1 -- 172.21.15.84:0/325492624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6d0159f00 msgr2=0x7fa6d00b2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:24.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.790+0000 7fa6d79e7640 1 --2- 172.21.15.84:0/325492624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6d0159f00 0x7fa6d00b2130 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009fd0 tx=0x7fa6c800e4b0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.790+0000 7fa6d79e7640 1 -- 172.21.15.84:0/325492624 shutdown_connections 2023-10-16T11:22:24.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.790+0000 7fa6d79e7640 1 --2- 172.21.15.84:0/325492624 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fa6b406af90 0x7fa6b406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.790+0000 7fa6d79e7640 1 --2- 172.21.15.84:0/325492624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6d0159f00 0x7fa6d00b2130 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.790+0000 7fa6d79e7640 1 --2- 172.21.15.84:0/325492624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d0159b20 0x7fa6d0148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.790+0000 7fa6d79e7640 1 --2- 172.21.15.84:0/325492624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6d006c120 0x7fa6d00b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.790+0000 7fa6d79e7640 1 -- 172.21.15.84:0/325492624 >> 172.21.15.84:0/325492624 conn(0x7fa6d00cb410 msgr2=0x7fa6d00c2700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.790+0000 7fa6d79e7640 1 -- 172.21.15.84:0/325492624 shutdown_connections 2023-10-16T11:22:24.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:24.790+0000 7fa6d79e7640 1 -- 172.21.15.84:0/325492624 wait complete. 2023-10-16T11:22:24.809 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:06:20.873257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:06:22.065944+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '187', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 32, 'snap_epoch': 187, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:22:24.810 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:22:24.971 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:24.970+0000 7fbdf223d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:22:25.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.007+0000 7fe94571d640 1 Processor -- start 2023-10-16T11:22:25.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.007+0000 7fe94571d640 1 -- start start 2023-10-16T11:22:25.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.008+0000 7fe94571d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe94006b660 0x7fe94006ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.008+0000 7fe94571d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe9401428c0 0x7fe9400c9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.008+0000 7fe94571d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9400c9690 0x7fe94015d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.008+0000 7fe94571d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9400c9b20 con 0x7fe9401428c0 2023-10-16T11:22:25.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.008+0000 7fe94571d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9400cc6b0 con 0x7fe9400c9690 2023-10-16T11:22:25.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.008+0000 7fe94571d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9401599b0 con 0x7fe94006b660 2023-10-16T11:22:25.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.008+0000 7fe93e7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe9401428c0 0x7fe9400c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.009+0000 7fe93e7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe9401428c0 0x7fe9400c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54004/0 (socket says 172.21.15.84:54004) 2023-10-16T11:22:25.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.009+0000 7fe93e7fc640 1 -- 172.21.15.84:0/1432740436 learned_addr learned my addr 172.21.15.84:0/1432740436 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:25.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.009+0000 7fe93effd640 1 --2- 172.21.15.84:0/1432740436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe94006b660 0x7fe94006ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.009+0000 7fe93f7fe640 1 --2- 172.21.15.84:0/1432740436 >> v2:172.21.15.157:3300/0 conn(0x7fe9400c9690 0x7fe94015d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.009+0000 7fe93e7fc640 1 -- 172.21.15.84:0/1432740436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe94006b660 msgr2=0x7fe94006ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:25.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.009+0000 7fe93e7fc640 1 --2- 172.21.15.84:0/1432740436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe94006b660 0x7fe94006ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.009+0000 7fe93e7fc640 1 -- 172.21.15.84:0/1432740436 >> v2:172.21.15.157:3300/0 conn(0x7fe9400c9690 msgr2=0x7fe94015d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:25.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.009+0000 7fe93e7fc640 1 --2- 172.21.15.84:0/1432740436 >> v2:172.21.15.157:3300/0 conn(0x7fe9400c9690 0x7fe94015d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.009+0000 7fe93e7fc640 1 -- 172.21.15.84:0/1432740436 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94006c290 con 0x7fe9401428c0 2023-10-16T11:22:25.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.009+0000 7fe93f7fe640 1 --2- 172.21.15.84:0/1432740436 >> v2:172.21.15.157:3300/0 conn(0x7fe9400c9690 0x7fe94015d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:25.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.010+0000 7fe93e7fc640 1 --2- 172.21.15.84:0/1432740436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9401428c0 0x7fe9400c9150 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7fe9400ae3a0 tx=0x7fe93402db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ae57e03b5e02c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:25.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.010+0000 7fe93dffb640 1 -- 172.21.15.84:0/1432740436 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93402f8c0 con 0x7fe9401428c0 2023-10-16T11:22:25.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.010+0000 7fe93dffb640 1 -- 172.21.15.84:0/1432740436 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93402fa20 con 0x7fe9401428c0 2023-10-16T11:22:25.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.010+0000 7fe93dffb640 1 -- 172.21.15.84:0/1432740436 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93402fdb0 con 0x7fe9401428c0 2023-10-16T11:22:25.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.011+0000 7fe94571d640 1 -- 172.21.15.84:0/1432740436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9401428c0 msgr2=0x7fe9400c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.011+0000 7fe94571d640 1 --2- 172.21.15.84:0/1432740436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9401428c0 0x7fe9400c9150 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7fe9400ae3a0 tx=0x7fe93402db00 comp rx=0 tx=0).stop 2023-10-16T11:22:25.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.011+0000 7fe94571d640 1 -- 172.21.15.84:0/1432740436 shutdown_connections 2023-10-16T11:22:25.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.011+0000 7fe94571d640 1 --2- 172.21.15.84:0/1432740436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9401428c0 0x7fe9400c9150 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.011+0000 7fe94571d640 1 --2- 172.21.15.84:0/1432740436 >> v2:172.21.15.157:3300/0 conn(0x7fe9400c9690 0x7fe94015d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.011+0000 7fe94571d640 1 --2- 172.21.15.84:0/1432740436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe94006b660 0x7fe94006ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.011+0000 7fe94571d640 1 -- 172.21.15.84:0/1432740436 >> 172.21.15.84:0/1432740436 conn(0x7fe9400ce9b0 msgr2=0x7fe9400cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:25.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.011+0000 7fe94571d640 1 -- 172.21.15.84:0/1432740436 shutdown_connections 2023-10-16T11:22:25.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.011+0000 7fe94571d640 1 -- 172.21.15.84:0/1432740436 wait complete. 2023-10-16T11:22:25.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.013+0000 7fe94571d640 1 Processor -- start 2023-10-16T11:22:25.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.013+0000 7fe94571d640 1 -- start start 2023-10-16T11:22:25.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.014+0000 7fe94571d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe94006b660 0x7fe9400c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.014+0000 7fe94571d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9400c9690 0x7fe9401542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.014+0000 7fe94571d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9401428c0 0x7fe9400c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.014+0000 7fe94571d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9400c6e50 con 0x7fe9401428c0 2023-10-16T11:22:25.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.014+0000 7fe94571d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9400c5fc0 con 0x7fe9400c9690 2023-10-16T11:22:25.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.014+0000 7fe94571d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9400c61d0 con 0x7fe94006b660 2023-10-16T11:22:25.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.014+0000 7fe93f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9401428c0 0x7fe9400c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.014+0000 7fe93f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9401428c0 0x7fe9400c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54018/0 (socket says 172.21.15.84:54018) 2023-10-16T11:22:25.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.014+0000 7fe93f7fe640 1 -- 172.21.15.84:0/926752058 learned_addr learned my addr 172.21.15.84:0/926752058 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:25.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.014+0000 7fe93effd640 1 --2- 172.21.15.84:0/926752058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe94006b660 0x7fe9400c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.015+0000 7fe93e7fc640 1 --2- 172.21.15.84:0/926752058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9400c9690 0x7fe9401542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.015+0000 7fe93f7fe640 1 -- 172.21.15.84:0/926752058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe94006b660 msgr2=0x7fe9400c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.015+0000 7fe93f7fe640 1 --2- 172.21.15.84:0/926752058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe94006b660 0x7fe9400c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.015+0000 7fe93f7fe640 1 -- 172.21.15.84:0/926752058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9400c9690 msgr2=0x7fe9401542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.015+0000 7fe93f7fe640 1 --2- 172.21.15.84:0/926752058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9400c9690 0x7fe9401542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.015+0000 7fe93f7fe640 1 -- 172.21.15.84:0/926752058 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9340094d0 con 0x7fe9401428c0 2023-10-16T11:22:25.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.015+0000 7fe93e7fc640 1 --2- 172.21.15.84:0/926752058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9400c9690 0x7fe9401542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:25.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.015+0000 7fe93effd640 1 --2- 172.21.15.84:0/926752058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe94006b660 0x7fe9400c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:25.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.015+0000 7fe93f7fe640 1 --2- 172.21.15.84:0/926752058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9401428c0 0x7fe9400c5a80 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7fe93800cd50 tx=0x7fe93800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:25.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.015+0000 7fe9277fe640 1 -- 172.21.15.84:0/926752058 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93800f680 con 0x7fe9401428c0 2023-10-16T11:22:25.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.015+0000 7fe9277fe640 1 -- 172.21.15.84:0/926752058 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93800f800 con 0x7fe9401428c0 2023-10-16T11:22:25.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.015+0000 7fe9277fe640 1 -- 172.21.15.84:0/926752058 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93800fbb0 con 0x7fe9401428c0 2023-10-16T11:22:25.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.015+0000 7fe94571d640 1 -- 172.21.15.84:0/926752058 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9400c64b0 con 0x7fe9401428c0 2023-10-16T11:22:25.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.016+0000 7fe94571d640 1 -- 172.21.15.84:0/926752058 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9400c6a70 con 0x7fe9401428c0 2023-10-16T11:22:25.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.016+0000 7fe94571d640 1 -- 172.21.15.84:0/926752058 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe940154930 con 0x7fe9401428c0 2023-10-16T11:22:25.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.018+0000 7fe9277fe640 1 -- 172.21.15.84:0/926752058 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 118) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fe93800b040 con 0x7fe9401428c0 2023-10-16T11:22:25.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.018+0000 7fe9277fe640 1 --2- 172.21.15.84:0/926752058 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fe92c06af90 0x7fe92c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.018+0000 7fe93effd640 1 -- 172.21.15.84:0/926752058 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fe92c06af90 msgr2=0x7fe92c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/285590107 2023-10-16T11:22:25.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.018+0000 7fe9277fe640 1 -- 172.21.15.84:0/926752058 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6497+0+0 (secure 0 0 0) 0x7fe93808a530 con 0x7fe9401428c0 2023-10-16T11:22:25.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.019+0000 7fe93effd640 1 --2- 172.21.15.84:0/926752058 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fe92c06af90 0x7fe92c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:25.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.023+0000 7fe9277fe640 1 -- 172.21.15.84:0/926752058 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe940154930 con 0x7fe9401428c0 2023-10-16T11:22:25.030 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:25.029+0000 7fbdf223d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:22:25.067 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:25.066+0000 7f6ff463d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:22:25.123 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:25.122+0000 7f6ff463d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:22:25.143 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:25.142+0000 7fbdf223d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:22:25.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.215+0000 7fe94571d640 1 -- 172.21.15.84:0/926752058 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe940154930 con 0x7fe9401428c0 2023-10-16T11:22:25.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.216+0000 7fe9277fe640 1 -- 172.21.15.84:0/926752058 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v188) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe940154930 con 0x7fe9401428c0 2023-10-16T11:22:25.217 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:22:25.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.218+0000 7fe94571d640 1 -- 172.21.15.84:0/926752058 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fe92c06af90 msgr2=0x7fe92c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:25.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.218+0000 7fe94571d640 1 --2- 172.21.15.84:0/926752058 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fe92c06af90 0x7fe92c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.218+0000 7fe94571d640 1 -- 172.21.15.84:0/926752058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9401428c0 msgr2=0x7fe9400c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.218+0000 7fe94571d640 1 --2- 172.21.15.84:0/926752058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9401428c0 0x7fe9400c5a80 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7fe93800cd50 tx=0x7fe93800a650 comp rx=0 tx=0).stop 2023-10-16T11:22:25.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.218+0000 7fe94571d640 1 -- 172.21.15.84:0/926752058 shutdown_connections 2023-10-16T11:22:25.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.218+0000 7fe94571d640 1 --2- 172.21.15.84:0/926752058 >> [v2:172.21.15.84:6832/285590107,v1:172.21.15.84:6833/285590107] conn(0x7fe92c06af90 0x7fe92c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.218+0000 7fe94571d640 1 --2- 172.21.15.84:0/926752058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9401428c0 0x7fe9400c5a80 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.218+0000 7fe94571d640 1 --2- 172.21.15.84:0/926752058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9400c9690 0x7fe9401542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.218+0000 7fe94571d640 1 --2- 172.21.15.84:0/926752058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe94006b660 0x7fe9400c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.218+0000 7fe94571d640 1 -- 172.21.15.84:0/926752058 >> 172.21.15.84:0/926752058 conn(0x7fe9400ce9b0 msgr2=0x7fe940070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:25.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.219+0000 7fe94571d640 1 -- 172.21.15.84:0/926752058 shutdown_connections 2023-10-16T11:22:25.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.219+0000 7fe94571d640 1 -- 172.21.15.84:0/926752058 wait complete. 2023-10-16T11:22:25.235 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:22:25.240 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:25.238+0000 7f6ff463d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:22:25.313 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:25.312+0000 7f6f788bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:25.313 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:25.312+0000 7f6f788bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:25.314 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:25.312+0000 7f6f788bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:25.314 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:25.312+0000 7f6f788bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:25.314 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:25.312+0000 7f6f788bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:25.315 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:25.314+0000 7f6f760ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:25.315 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:25.314+0000 7f6f760ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:25.315 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:25.314+0000 7f6f760ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:25.315 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:25.314+0000 7f6f760ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:25.315 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:25.314+0000 7f6f760ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:25.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.425+0000 7ff0da0c6640 1 Processor -- start 2023-10-16T11:22:25.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.425+0000 7ff0da0c6640 1 -- start start 2023-10-16T11:22:25.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.427+0000 7ff0da0c6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0d40cb1c0 0x7ff0d406faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.427+0000 7ff0da0c6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff0d4065ab0 0x7ff0d4065f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.427+0000 7ff0da0c6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d4066450 con 0x7ff0d415af80 2023-10-16T11:22:25.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.427+0000 7ff0da0c6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d40d0930 con 0x7ff0d4065ab0 2023-10-16T11:22:25.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.427+0000 7ff0da0c6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d4070030 con 0x7ff0d40cb1c0 2023-10-16T11:22:25.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.427+0000 7ff0d37fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff0d415af80 0x7ff0d415f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40908/0 (socket says 172.21.15.84:40908) 2023-10-16T11:22:25.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.427+0000 7ff0d37fe640 1 -- 172.21.15.84:0/4207300099 learned_addr learned my addr 172.21.15.84:0/4207300099 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:25.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.427+0000 7ff0d3fff640 1 --2- 172.21.15.84:0/4207300099 >> v2:172.21.15.84:3300/0 conn(0x7ff0d4065ab0 0x7ff0d4065f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.427+0000 7ff0d27fc640 1 -- 172.21.15.84:0/4207300099 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290198498 0 0) 0x7ff0d4066450 con 0x7ff0d415af80 2023-10-16T11:22:25.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.427+0000 7ff0d27fc640 1 -- 172.21.15.84:0/4207300099 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0bc003520 con 0x7ff0d415af80 2023-10-16T11:22:25.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.427+0000 7ff0d3fff640 1 -- 172.21.15.84:0/4207300099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0d40cb1c0 msgr2=0x7ff0d406faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:25.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.427+0000 7ff0d2ffd640 1 --2- 172.21.15.84:0/4207300099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0d40cb1c0 0x7ff0d406faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.427+0000 7ff0d3fff640 1 --2- 172.21.15.84:0/4207300099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0d40cb1c0 0x7ff0d406faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.427+0000 7ff0d3fff640 1 -- 172.21.15.84:0/4207300099 >> v1:172.21.15.84:6789/0 conn(0x7ff0d415af80 legacy=0x7ff0d415f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.427+0000 7ff0d3fff640 1 -- 172.21.15.84:0/4207300099 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d415eff0 con 0x7ff0d4065ab0 2023-10-16T11:22:25.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.427+0000 7ff0d2ffd640 1 --2- 172.21.15.84:0/4207300099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0d40cb1c0 0x7ff0d406faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:25.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.428+0000 7ff0d3fff640 1 --2- 172.21.15.84:0/4207300099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d4065ab0 0x7ff0d4065f10 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc009dc0 tx=0x7ff0cc02be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f601d9f00f129dda server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:25.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.429+0000 7ff0d27fc640 1 -- 172.21.15.84:0/4207300099 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc02cbb0 con 0x7ff0d4065ab0 2023-10-16T11:22:25.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.429+0000 7ff0d27fc640 1 -- 172.21.15.84:0/4207300099 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0cc02cd10 con 0x7ff0d4065ab0 2023-10-16T11:22:25.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.429+0000 7ff0d27fc640 1 -- 172.21.15.84:0/4207300099 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc034660 con 0x7ff0d4065ab0 2023-10-16T11:22:25.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.429+0000 7ff0da0c6640 1 -- 172.21.15.84:0/4207300099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d4065ab0 msgr2=0x7ff0d4065f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.429+0000 7ff0da0c6640 1 --2- 172.21.15.84:0/4207300099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d4065ab0 0x7ff0d4065f10 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc009dc0 tx=0x7ff0cc02be00 comp rx=0 tx=0).stop 2023-10-16T11:22:25.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.429+0000 7ff0da0c6640 1 -- 172.21.15.84:0/4207300099 shutdown_connections 2023-10-16T11:22:25.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.429+0000 7ff0da0c6640 1 --2- 172.21.15.84:0/4207300099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d4065ab0 0x7ff0d4065f10 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.429+0000 7ff0da0c6640 1 --2- 172.21.15.84:0/4207300099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0d40cb1c0 0x7ff0d406faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.429+0000 7ff0da0c6640 1 -- 172.21.15.84:0/4207300099 >> 172.21.15.84:0/4207300099 conn(0x7ff0d40d0ab0 msgr2=0x7ff0d40d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:25.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.429+0000 7ff0da0c6640 1 -- 172.21.15.84:0/4207300099 shutdown_connections 2023-10-16T11:22:25.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.430+0000 7ff0da0c6640 1 -- 172.21.15.84:0/4207300099 wait complete. 2023-10-16T11:22:25.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.431+0000 7ff0da0c6640 1 Processor -- start 2023-10-16T11:22:25.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.431+0000 7ff0da0c6640 1 -- start start 2023-10-16T11:22:25.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.432+0000 7ff0da0c6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0d4065ab0 0x7ff0d4158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.432+0000 7ff0da0c6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0d40cb1c0 0x7ff0d40c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.432+0000 7ff0da0c6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d415af80 0x7ff0d40c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.432+0000 7ff0da0c6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d40b3d50 con 0x7ff0d415af80 2023-10-16T11:22:25.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.432+0000 7ff0da0c6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d40c7af0 con 0x7ff0d4065ab0 2023-10-16T11:22:25.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.432+0000 7ff0da0c6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d40c7d00 con 0x7ff0d40cb1c0 2023-10-16T11:22:25.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.432+0000 7ff0d3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d415af80 0x7ff0d40c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.432+0000 7ff0d3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d415af80 0x7ff0d40c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54130/0 (socket says 172.21.15.84:54130) 2023-10-16T11:22:25.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.432+0000 7ff0d3fff640 1 -- 172.21.15.84:0/1163958388 learned_addr learned my addr 172.21.15.84:0/1163958388 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:25.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.433+0000 7ff0d3fff640 1 -- 172.21.15.84:0/1163958388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0d40cb1c0 msgr2=0x7ff0d40c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.433+0000 7ff0d3fff640 1 --2- 172.21.15.84:0/1163958388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0d40cb1c0 0x7ff0d40c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.433+0000 7ff0d3fff640 1 -- 172.21.15.84:0/1163958388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0d4065ab0 msgr2=0x7ff0d4158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.433+0000 7ff0d3fff640 1 --2- 172.21.15.84:0/1163958388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0d4065ab0 0x7ff0d4158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.433+0000 7ff0d3fff640 1 -- 172.21.15.84:0/1163958388 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc009a70 con 0x7ff0d415af80 2023-10-16T11:22:25.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.433+0000 7ff0d3fff640 1 --2- 172.21.15.84:0/1163958388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d415af80 0x7ff0d40c6fa0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc037040 tx=0x7ff0cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:25.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.433+0000 7ff0bbfff640 1 -- 172.21.15.84:0/1163958388 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc040650 con 0x7ff0d415af80 2023-10-16T11:22:25.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.433+0000 7ff0bbfff640 1 -- 172.21.15.84:0/1163958388 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0cc0407b0 con 0x7ff0d415af80 2023-10-16T11:22:25.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.433+0000 7ff0da0c6640 1 -- 172.21.15.84:0/1163958388 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d40c7fe0 con 0x7ff0d415af80 2023-10-16T11:22:25.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.433+0000 7ff0bbfff640 1 -- 172.21.15.84:0/1163958388 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc040b60 con 0x7ff0d415af80 2023-10-16T11:22:25.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.434+0000 7ff0da0c6640 1 -- 172.21.15.84:0/1163958388 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d40c85a0 con 0x7ff0d415af80 2023-10-16T11:22:25.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.435+0000 7ff0da0c6640 1 -- 172.21.15.84:0/1163958388 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d40c8790 con 0x7ff0d415af80 2023-10-16T11:22:25.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.435+0000 7ff0bbfff640 1 -- 172.21.15.84:0/1163958388 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 119) v1 ==== 88940+0+0 (secure 0 0 0) 0x7ff0cc040cc0 con 0x7ff0d415af80 2023-10-16T11:22:25.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.436+0000 7ff0bbfff640 1 -- 172.21.15.84:0/1163958388 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6797+0+0 (secure 0 0 0) 0x7ff0cc0ac3a0 con 0x7ff0d415af80 2023-10-16T11:22:25.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.441+0000 7ff0bbfff640 1 -- 172.21.15.84:0/1163958388 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff0cc077370 con 0x7ff0d415af80 2023-10-16T11:22:25.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.622+0000 7ff0da0c6640 1 -- 172.21.15.84:0/1163958388 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff0d404e160 con 0x7ff0d415af80 2023-10-16T11:22:25.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.623+0000 7ff0bbfff640 1 -- 172.21.15.84:0/1163958388 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v189) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff0cc07a990 con 0x7ff0d415af80 2023-10-16T11:22:25.624 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:22:25.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.626+0000 7ff0da0c6640 1 -- 172.21.15.84:0/1163958388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d415af80 msgr2=0x7ff0d40c6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.626+0000 7ff0da0c6640 1 --2- 172.21.15.84:0/1163958388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d415af80 0x7ff0d40c6fa0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc037040 tx=0x7ff0cc004a40 comp rx=0 tx=0).stop 2023-10-16T11:22:25.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.626+0000 7ff0da0c6640 1 -- 172.21.15.84:0/1163958388 shutdown_connections 2023-10-16T11:22:25.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.626+0000 7ff0da0c6640 1 --2- 172.21.15.84:0/1163958388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d415af80 0x7ff0d40c6fa0 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.626+0000 7ff0da0c6640 1 --2- 172.21.15.84:0/1163958388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0d40cb1c0 0x7ff0d40c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.626+0000 7ff0da0c6640 1 --2- 172.21.15.84:0/1163958388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0d4065ab0 0x7ff0d4158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.626+0000 7ff0da0c6640 1 -- 172.21.15.84:0/1163958388 >> 172.21.15.84:0/1163958388 conn(0x7ff0d40d0ab0 msgr2=0x7ff0d40acb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:25.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.626+0000 7ff0da0c6640 1 -- 172.21.15.84:0/1163958388 shutdown_connections 2023-10-16T11:22:25.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.626+0000 7ff0da0c6640 1 -- 172.21.15.84:0/1163958388 wait complete. 2023-10-16T11:22:25.644 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:22:25.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.832+0000 7f874ebfb640 1 Processor -- start 2023-10-16T11:22:25.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.833+0000 7f874ebfb640 1 -- start start 2023-10-16T11:22:25.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.834+0000 7f874ebfb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87480ad350 0x7f8748069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.834+0000 7f874ebfb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f874806a2e0 0x7f87480c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.834+0000 7f874ebfb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f874806a6f0 con 0x7f874815d150 2023-10-16T11:22:25.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.834+0000 7f874ebfb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874815d530 con 0x7f874806a2e0 2023-10-16T11:22:25.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.834+0000 7f874ebfb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87480ac890 con 0x7f87480ad350 2023-10-16T11:22:25.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.834+0000 7f874c997640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f874815d150 0x7f8748160b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60556/0 (socket says 172.21.15.84:60556) 2023-10-16T11:22:25.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.834+0000 7f874c997640 1 -- 172.21.15.84:0/2283545048 learned_addr learned my addr 172.21.15.84:0/2283545048 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:25.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.834+0000 7f87477fe640 1 -- 172.21.15.84:0/2283545048 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 588274584 0 0) 0x7f874806a6f0 con 0x7f874815d150 2023-10-16T11:22:25.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.835+0000 7f87477fe640 1 -- 172.21.15.84:0/2283545048 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8738003520 con 0x7f874815d150 2023-10-16T11:22:25.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.835+0000 7f8747fff640 1 --2- 172.21.15.84:0/2283545048 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87480ad350 0x7f8748069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.835+0000 7f874d198640 1 --2- 172.21.15.84:0/2283545048 >> v2:172.21.15.157:3300/0 conn(0x7f874806a2e0 0x7f87480c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.835+0000 7f87477fe640 1 -- 172.21.15.84:0/2283545048 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2957649447 0 0) 0x7f8738003520 con 0x7f874815d150 2023-10-16T11:22:25.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.835+0000 7f87477fe640 1 -- 172.21.15.84:0/2283545048 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87480ad350 msgr2=0x7f8748069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:25.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.835+0000 7f87477fe640 1 --2- 172.21.15.84:0/2283545048 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87480ad350 0x7f8748069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.835+0000 7f87477fe640 1 -- 172.21.15.84:0/2283545048 >> v2:172.21.15.157:3300/0 conn(0x7f874806a2e0 msgr2=0x7f87480c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:25.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.835+0000 7f87477fe640 1 --2- 172.21.15.84:0/2283545048 >> v2:172.21.15.157:3300/0 conn(0x7f874806a2e0 0x7f87480c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.835+0000 7f87477fe640 1 -- 172.21.15.84:0/2283545048 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8748161650 con 0x7f874815d150 2023-10-16T11:22:25.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.835+0000 7f87477fe640 1 -- 172.21.15.84:0/2283545048 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f873c001670 con 0x7f874815d150 2023-10-16T11:22:25.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.835+0000 7f87477fe640 1 -- 172.21.15.84:0/2283545048 >> v1:172.21.15.84:6789/0 conn(0x7f874815d150 legacy=0x7f8748160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.835+0000 7f87477fe640 1 --2- 172.21.15.84:0/2283545048 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87480ad350 0x7f8738003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.835+0000 7f87477fe640 1 --2- 172.21.15.84:0/2283545048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8738003ff0 0x7f87380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.835+0000 7f87477fe640 1 --2- 172.21.15.84:0/2283545048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8738004d00 0x7f8738005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.835+0000 7f87477fe640 1 -- 172.21.15.84:0/2283545048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874815d530 con 0x7f8738004d00 2023-10-16T11:22:25.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.835+0000 7f87477fe640 1 -- 172.21.15.84:0/2283545048 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87480ac890 con 0x7f8738003ff0 2023-10-16T11:22:25.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.835+0000 7f87477fe640 1 -- 172.21.15.84:0/2283545048 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873800bae0 con 0x7f87480ad350 2023-10-16T11:22:25.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.835+0000 7f874d198640 1 --2- 172.21.15.84:0/2283545048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8738004d00 0x7f8738005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.836+0000 7f874d198640 1 -- 172.21.15.84:0/2283545048 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87480ad350 msgr2=0x7f8738003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:25.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.836+0000 7f874c997640 1 --2- 172.21.15.84:0/2283545048 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87480ad350 0x7f8738003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.836+0000 7f8747fff640 1 --2- 172.21.15.84:0/2283545048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8738003ff0 0x7f87380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.836+0000 7f874d198640 1 --2- 172.21.15.84:0/2283545048 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87480ad350 0x7f8738003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.836+0000 7f874d198640 1 -- 172.21.15.84:0/2283545048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8738003ff0 msgr2=0x7f87380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:25.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.836+0000 7f874d198640 1 --2- 172.21.15.84:0/2283545048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8738003ff0 0x7f87380047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.836+0000 7f874d198640 1 -- 172.21.15.84:0/2283545048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8738003520 con 0x7f8738004d00 2023-10-16T11:22:25.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.837+0000 7f874d198640 1 --2- 172.21.15.84:0/2283545048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8738004d00 0x7f8738005180 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f8740007360 tx=0x7f87380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb5a445fd94b3c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:25.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.837+0000 7f87477fe640 1 -- 172.21.15.84:0/2283545048 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874002ccf0 con 0x7f8738004d00 2023-10-16T11:22:25.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.837+0000 7f87477fe640 1 -- 172.21.15.84:0/2283545048 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874002ce50 con 0x7f8738004d00 2023-10-16T11:22:25.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.837+0000 7f87477fe640 1 -- 172.21.15.84:0/2283545048 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740034660 con 0x7f8738004d00 2023-10-16T11:22:25.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.837+0000 7f874ebfb640 1 -- 172.21.15.84:0/2283545048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8738004d00 msgr2=0x7f8738005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.837+0000 7f874ebfb640 1 --2- 172.21.15.84:0/2283545048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8738004d00 0x7f8738005180 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f8740007360 tx=0x7f87380057f0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.837+0000 7f874d198640 1 -- 172.21.15.84:0/2283545048 reap_dead start 2023-10-16T11:22:25.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.838+0000 7f874ebfb640 1 -- 172.21.15.84:0/2283545048 shutdown_connections 2023-10-16T11:22:25.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.838+0000 7f874ebfb640 1 -- 172.21.15.84:0/2283545048 >> 172.21.15.84:0/2283545048 conn(0x7f87480ce9b0 msgr2=0x7f87480cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:25.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.838+0000 7f874ebfb640 1 -- 172.21.15.84:0/2283545048 shutdown_connections 2023-10-16T11:22:25.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.838+0000 7f874ebfb640 1 -- 172.21.15.84:0/2283545048 wait complete. 2023-10-16T11:22:25.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.839+0000 7f874ebfb640 1 Processor -- start 2023-10-16T11:22:25.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.839+0000 7f874ebfb640 1 -- start start 2023-10-16T11:22:25.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.840+0000 7f874ebfb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8738004d00 0x7f874814aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.840+0000 7f874ebfb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f874814a1b0 0x7f874815fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.840+0000 7f874ebfb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f874815ff80 0x7f87480c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.840+0000 7f874ebfb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87480c7740 con 0x7f874814a1b0 2023-10-16T11:22:25.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.840+0000 7f874ebfb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87480c6ac0 con 0x7f8738004d00 2023-10-16T11:22:25.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.840+0000 7f874ebfb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87480c6da0 con 0x7f874815ff80 2023-10-16T11:22:25.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.841+0000 7f8747fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f874814a1b0 0x7f874815fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.841+0000 7f8747fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f874814a1b0 0x7f874815fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49492/0 (socket says 172.21.15.84:49492) 2023-10-16T11:22:25.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.841+0000 7f8747fff640 1 -- 172.21.15.84:0/3117449112 learned_addr learned my addr 172.21.15.84:0/3117449112 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:25.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.841+0000 7f874d198640 1 --2- 172.21.15.84:0/3117449112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f874815ff80 0x7f87480c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.841+0000 7f8747fff640 1 -- 172.21.15.84:0/3117449112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f874815ff80 msgr2=0x7f87480c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.841+0000 7f8747fff640 1 --2- 172.21.15.84:0/3117449112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f874815ff80 0x7f87480c5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.841+0000 7f8747fff640 1 -- 172.21.15.84:0/3117449112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8738004d00 msgr2=0x7f874814aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.841+0000 7f8747fff640 1 --2- 172.21.15.84:0/3117449112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8738004d00 0x7f874814aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.841+0000 7f8747fff640 1 -- 172.21.15.84:0/3117449112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87480c7080 con 0x7f874814a1b0 2023-10-16T11:22:25.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.841+0000 7f8747fff640 1 --2- 172.21.15.84:0/3117449112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f874814a1b0 0x7f874815fa40 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f873400e7e0 tx=0x7f873400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:25.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.841+0000 7f87457fa640 1 -- 172.21.15.84:0/3117449112 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873400c6d0 con 0x7f874814a1b0 2023-10-16T11:22:25.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.842+0000 7f87457fa640 1 -- 172.21.15.84:0/3117449112 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873400c830 con 0x7f874814a1b0 2023-10-16T11:22:25.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.842+0000 7f87457fa640 1 -- 172.21.15.84:0/3117449112 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873400cba0 con 0x7f874814a1b0 2023-10-16T11:22:25.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.842+0000 7f874ebfb640 1 -- 172.21.15.84:0/3117449112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87480c7360 con 0x7f874814a1b0 2023-10-16T11:22:25.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.842+0000 7f874ebfb640 1 -- 172.21.15.84:0/3117449112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8748160400 con 0x7f874814a1b0 2023-10-16T11:22:25.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.843+0000 7f874ebfb640 1 -- 172.21.15.84:0/3117449112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874804e160 con 0x7f874814a1b0 2023-10-16T11:22:25.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.843+0000 7f87457fa640 1 -- 172.21.15.84:0/3117449112 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 119) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f873400cd00 con 0x7f874814a1b0 2023-10-16T11:22:25.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.844+0000 7f87457fa640 1 -- 172.21.15.84:0/3117449112 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6797+0+0 (secure 0 0 0) 0x7f8734089390 con 0x7f874814a1b0 2023-10-16T11:22:25.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:25.849+0000 7f87457fa640 1 -- 172.21.15.84:0/3117449112 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8734054360 con 0x7f874814a1b0 2023-10-16T11:22:26.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.041+0000 7f874ebfb640 1 -- 172.21.15.84:0/3117449112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f87480cd6f0 con 0x7f874814a1b0 2023-10-16T11:22:26.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.042+0000 7f87457fa640 1 -- 172.21.15.84:0/3117449112 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v189) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8734057980 con 0x7f874814a1b0 2023-10-16T11:22:26.043 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:22:26.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.045+0000 7f874ebfb640 1 -- 172.21.15.84:0/3117449112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f874814a1b0 msgr2=0x7f874815fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.045+0000 7f874ebfb640 1 --2- 172.21.15.84:0/3117449112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f874814a1b0 0x7f874815fa40 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f873400e7e0 tx=0x7f873400eca0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.045+0000 7f874ebfb640 1 -- 172.21.15.84:0/3117449112 shutdown_connections 2023-10-16T11:22:26.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.045+0000 7f874ebfb640 1 --2- 172.21.15.84:0/3117449112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f874815ff80 0x7f87480c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.045+0000 7f874ebfb640 1 --2- 172.21.15.84:0/3117449112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f874814a1b0 0x7f874815fa40 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.045+0000 7f874ebfb640 1 --2- 172.21.15.84:0/3117449112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8738004d00 0x7f874814aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.045+0000 7f874ebfb640 1 -- 172.21.15.84:0/3117449112 >> 172.21.15.84:0/3117449112 conn(0x7f87480ce9b0 msgr2=0x7f87480b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:26.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.045+0000 7f874ebfb640 1 -- 172.21.15.84:0/3117449112 shutdown_connections 2023-10-16T11:22:26.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.045+0000 7f874ebfb640 1 -- 172.21.15.84:0/3117449112 wait complete. 2023-10-16T11:22:26.062 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:22:26.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.268+0000 7f74e5148640 1 Processor -- start 2023-10-16T11:22:26.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.268+0000 7f74e5148640 1 -- start start 2023-10-16T11:22:26.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.270+0000 7f74e5148640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f74e006c3f0 0x7f74e006c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.270+0000 7f74e5148640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e015e0c0 0x7f74e00b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.270+0000 7f74e5148640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e00cd630 con 0x7f74e015e540 2023-10-16T11:22:26.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.270+0000 7f74e5148640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e00ce1b0 con 0x7f74e006c3f0 2023-10-16T11:22:26.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.270+0000 7f74e5148640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e006cd10 con 0x7f74e015e0c0 2023-10-16T11:22:26.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.270+0000 7f74ded9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f74e006c3f0 0x7f74e006c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.271+0000 7f74ded9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f74e006c3f0 0x7f74e006c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49506/0 (socket says 172.21.15.84:49506) 2023-10-16T11:22:26.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.271+0000 7f74ded9d640 1 -- 172.21.15.84:0/1021722347 learned_addr learned my addr 172.21.15.84:0/1021722347 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:26.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.271+0000 7f74ded9d640 1 -- 172.21.15.84:0/1021722347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e015e0c0 msgr2=0x7f74e00b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:26.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.271+0000 7f74ded9d640 1 --2- 172.21.15.84:0/1021722347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e015e0c0 0x7f74e00b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.271+0000 7f74ded9d640 1 -- 172.21.15.84:0/1021722347 >> v1:172.21.15.157:6789/0 conn(0x7f74e015e540 legacy=0x7f74e00b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:26.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.271+0000 7f74ded9d640 1 -- 172.21.15.84:0/1021722347 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e00d2170 con 0x7f74e006c3f0 2023-10-16T11:22:26.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.273+0000 7f74ded9d640 1 --2- 172.21.15.84:0/1021722347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e006c3f0 0x7f74e006c7d0 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009870 tx=0x7f74d002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba193ad2b467c613 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:26.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.273+0000 7f74de59c640 1 -- 172.21.15.84:0/1021722347 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d0031640 con 0x7f74e006c3f0 2023-10-16T11:22:26.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.273+0000 7f74de59c640 1 -- 172.21.15.84:0/1021722347 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d00317a0 con 0x7f74e006c3f0 2023-10-16T11:22:26.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.273+0000 7f74de59c640 1 -- 172.21.15.84:0/1021722347 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d0031b30 con 0x7f74e006c3f0 2023-10-16T11:22:26.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.274+0000 7f74e5148640 1 -- 172.21.15.84:0/1021722347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e006c3f0 msgr2=0x7f74e006c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.274+0000 7f74e5148640 1 --2- 172.21.15.84:0/1021722347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e006c3f0 0x7f74e006c7d0 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009870 tx=0x7f74d002b8f0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.274+0000 7f74e5148640 1 -- 172.21.15.84:0/1021722347 shutdown_connections 2023-10-16T11:22:26.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.274+0000 7f74e5148640 1 --2- 172.21.15.84:0/1021722347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e015e0c0 0x7f74e00b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.274+0000 7f74e5148640 1 --2- 172.21.15.84:0/1021722347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e006c3f0 0x7f74e006c7d0 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.274+0000 7f74e5148640 1 -- 172.21.15.84:0/1021722347 >> 172.21.15.84:0/1021722347 conn(0x7f74e00d2cb0 msgr2=0x7f74e00d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:26.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.274+0000 7f74e5148640 1 -- 172.21.15.84:0/1021722347 shutdown_connections 2023-10-16T11:22:26.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.274+0000 7f74e5148640 1 -- 172.21.15.84:0/1021722347 wait complete. 2023-10-16T11:22:26.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.276+0000 7f74e5148640 1 Processor -- start 2023-10-16T11:22:26.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.276+0000 7f74e5148640 1 -- start start 2023-10-16T11:22:26.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.277+0000 7f74e5148640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e006c3f0 0x7f74e014c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.277+0000 7f74e5148640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e015e0c0 0x7f74e015bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.277+0000 7f74e5148640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74e015e540 0x7f74e01f6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.277+0000 7f74e5148640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e0158360 con 0x7f74e015e0c0 2023-10-16T11:22:26.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.277+0000 7f74e5148640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e015c7a0 con 0x7f74e015e540 2023-10-16T11:22:26.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.277+0000 7f74e5148640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e015ca80 con 0x7f74e006c3f0 2023-10-16T11:22:26.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.277+0000 7f74d7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e015e0c0 0x7f74e015bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.277+0000 7f74d7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e015e0c0 0x7f74e015bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49520/0 (socket says 172.21.15.84:49520) 2023-10-16T11:22:26.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.277+0000 7f74d7fff640 1 -- 172.21.15.84:0/3624071720 learned_addr learned my addr 172.21.15.84:0/3624071720 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:26.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.278+0000 7f74ded9d640 1 --2- 172.21.15.84:0/3624071720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e006c3f0 0x7f74e014c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.278+0000 7f74d7fff640 1 -- 172.21.15.84:0/3624071720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e006c3f0 msgr2=0x7f74e014c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.278+0000 7f74d7fff640 1 --2- 172.21.15.84:0/3624071720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e006c3f0 0x7f74e014c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.278+0000 7f74d7fff640 1 -- 172.21.15.84:0/3624071720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74e015e540 msgr2=0x7f74e01f6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.278+0000 7f74df59e640 1 --2- 172.21.15.84:0/3624071720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74e015e540 0x7f74e01f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.278+0000 7f74d7fff640 1 --2- 172.21.15.84:0/3624071720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74e015e540 0x7f74e01f6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.278+0000 7f74d7fff640 1 -- 172.21.15.84:0/3624071720 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d00094d0 con 0x7f74e015e0c0 2023-10-16T11:22:26.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.278+0000 7f74ded9d640 1 --2- 172.21.15.84:0/3624071720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e006c3f0 0x7f74e014c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:26.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.278+0000 7f74df59e640 1 --2- 172.21.15.84:0/3624071720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74e015e540 0x7f74e01f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:26.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.278+0000 7f74d7fff640 1 --2- 172.21.15.84:0/3624071720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e015e0c0 0x7f74e015bce0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f74c800b4a0 tx=0x7f74c800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:26.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.278+0000 7f74d77fe640 1 -- 172.21.15.84:0/3624071720 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c8010610 con 0x7f74e015e0c0 2023-10-16T11:22:26.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.278+0000 7f74d77fe640 1 -- 172.21.15.84:0/3624071720 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c8010770 con 0x7f74e015e0c0 2023-10-16T11:22:26.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.279+0000 7f74e5148640 1 -- 172.21.15.84:0/3624071720 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e015cd60 con 0x7f74e015e0c0 2023-10-16T11:22:26.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.279+0000 7f74d77fe640 1 -- 172.21.15.84:0/3624071720 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c8010b20 con 0x7f74e015e0c0 2023-10-16T11:22:26.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.279+0000 7f74e5148640 1 -- 172.21.15.84:0/3624071720 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e01f6f50 con 0x7f74e015e0c0 2023-10-16T11:22:26.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.280+0000 7f74e5148640 1 -- 172.21.15.84:0/3624071720 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e01584a0 con 0x7f74e015e0c0 2023-10-16T11:22:26.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.281+0000 7f74d77fe640 1 -- 172.21.15.84:0/3624071720 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 120) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f74c8010c80 con 0x7f74e015e0c0 2023-10-16T11:22:26.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.281+0000 7f74d77fe640 1 --2- 172.21.15.84:0/3624071720 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f74c406acc0 0x7f74c406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.281+0000 7f74d77fe640 1 -- 172.21.15.84:0/3624071720 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6797+0+0 (secure 0 0 0) 0x7f74c80892e0 con 0x7f74e015e0c0 2023-10-16T11:22:26.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.282+0000 7f74ded9d640 1 --2- 172.21.15.84:0/3624071720 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f74c406acc0 0x7f74c406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.282+0000 7f74ded9d640 1 --2- 172.21.15.84:0/3624071720 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f74c406acc0 0x7f74c406d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f74d0036000 tx=0x7f74d003c040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:26.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.286+0000 7f74d77fe640 1 -- 172.21.15.84:0/3624071720 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f74e01584a0 con 0x7f74e015e0c0 2023-10-16T11:22:26.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.517+0000 7f74e5148640 1 -- 172.21.15.84:0/3624071720 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74e01584a0 con 0x7f74e015e0c0 2023-10-16T11:22:26.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.518+0000 7f74d77fe640 1 -- 172.21.15.84:0/3624071720 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f74e01584a0 con 0x7f74e015e0c0 2023-10-16T11:22:26.519 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:26.520 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6347,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1346424254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1346424254},{"type":"v1","addr":"172.21.15.157:6835","nonce":1346424254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3224476768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3224476768},{"type":"v1","addr":"172.21.15.157:6834","nonce":3224476768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":10197,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2490106026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2490106026},{"type":"v1","addr":"172.21.15.84:6835","nonce":2490106026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:06:23.034809+0000","modified":"2023-10-16T11:06:26.105994+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10206},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10206":{"gid":10206,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/1106170430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":1106170430},{"type":"v1","addr":"172.21.15.84:6837","nonce":1106170430}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-16T11:22:26.520 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 96 2023-10-16T11:22:26.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.521+0000 7f74e5148640 1 -- 172.21.15.84:0/3624071720 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f74c406acc0 msgr2=0x7f74c406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.521+0000 7f74e5148640 1 --2- 172.21.15.84:0/3624071720 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f74c406acc0 0x7f74c406d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f74d0036000 tx=0x7f74d003c040 comp rx=0 tx=0).stop 2023-10-16T11:22:26.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.521+0000 7f74e5148640 1 -- 172.21.15.84:0/3624071720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e015e0c0 msgr2=0x7f74e015bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.521+0000 7f74e5148640 1 --2- 172.21.15.84:0/3624071720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e015e0c0 0x7f74e015bce0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f74c800b4a0 tx=0x7f74c800b960 comp rx=0 tx=0).stop 2023-10-16T11:22:26.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.521+0000 7f74e5148640 1 -- 172.21.15.84:0/3624071720 shutdown_connections 2023-10-16T11:22:26.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.521+0000 7f74e5148640 1 --2- 172.21.15.84:0/3624071720 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f74c406acc0 0x7f74c406d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.521+0000 7f74e5148640 1 --2- 172.21.15.84:0/3624071720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74e015e540 0x7f74e01f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.521+0000 7f74e5148640 1 --2- 172.21.15.84:0/3624071720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e015e0c0 0x7f74e015bce0 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.521+0000 7f74e5148640 1 --2- 172.21.15.84:0/3624071720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e006c3f0 0x7f74e014c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.521+0000 7f74e5148640 1 -- 172.21.15.84:0/3624071720 >> 172.21.15.84:0/3624071720 conn(0x7f74e00d2cb0 msgr2=0x7f74e006ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:26.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.521+0000 7f74e5148640 1 -- 172.21.15.84:0/3624071720 shutdown_connections 2023-10-16T11:22:26.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.521+0000 7f74e5148640 1 -- 172.21.15.84:0/3624071720 wait complete. 2023-10-16T11:22:26.543 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:22:26.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.745+0000 7f307fab4640 1 Processor -- start 2023-10-16T11:22:26.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.745+0000 7f307fab4640 1 -- start start 2023-10-16T11:22:26.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.747+0000 7f307fab4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307806c3f0 0x7f307806c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.747+0000 7f307fab4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f307815e540 0x7f30780b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.747+0000 7f307fab4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30780cd630 con 0x7f307815e0c0 2023-10-16T11:22:26.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.747+0000 7f307fab4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30780ce1b0 con 0x7f307815e540 2023-10-16T11:22:26.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.747+0000 7f307fab4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307806cd10 con 0x7f307806c3f0 2023-10-16T11:22:26.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.747+0000 7f307e051640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f307815e540 0x7f30780b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.747+0000 7f307e051640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f307815e540 0x7f30780b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49522/0 (socket says 172.21.15.84:49522) 2023-10-16T11:22:26.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.747+0000 7f307e051640 1 -- 172.21.15.84:0/2738441431 learned_addr learned my addr 172.21.15.84:0/2738441431 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:26.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.748+0000 7f307d850640 1 --2- 172.21.15.84:0/2738441431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307806c3f0 0x7f307806c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.748+0000 7f307e051640 1 -- 172.21.15.84:0/2738441431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307806c3f0 msgr2=0x7f307806c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:26.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.748+0000 7f307e051640 1 --2- 172.21.15.84:0/2738441431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307806c3f0 0x7f307806c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.748+0000 7f307e051640 1 -- 172.21.15.84:0/2738441431 >> v1:172.21.15.157:6789/0 conn(0x7f307815e0c0 legacy=0x7f30780b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:26.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.748+0000 7f307e051640 1 -- 172.21.15.84:0/2738441431 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30780d2170 con 0x7f307815e540 2023-10-16T11:22:26.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.749+0000 7f307e051640 1 --2- 172.21.15.84:0/2738441431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307815e540 0x7f30780b1710 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f3070009870 tx=0x7f307002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70ad67f54e8bcd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:26.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.749+0000 7f307c84e640 1 -- 172.21.15.84:0/2738441431 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070031640 con 0x7f307815e540 2023-10-16T11:22:26.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.749+0000 7f307c84e640 1 -- 172.21.15.84:0/2738441431 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30700317a0 con 0x7f307815e540 2023-10-16T11:22:26.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.749+0000 7f307c84e640 1 -- 172.21.15.84:0/2738441431 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070031b30 con 0x7f307815e540 2023-10-16T11:22:26.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.750+0000 7f307fab4640 1 -- 172.21.15.84:0/2738441431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307815e540 msgr2=0x7f30780b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.750+0000 7f307fab4640 1 --2- 172.21.15.84:0/2738441431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307815e540 0x7f30780b1710 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f3070009870 tx=0x7f307002b8f0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.750+0000 7f307fab4640 1 -- 172.21.15.84:0/2738441431 shutdown_connections 2023-10-16T11:22:26.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.750+0000 7f307fab4640 1 --2- 172.21.15.84:0/2738441431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307815e540 0x7f30780b1710 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.750+0000 7f307fab4640 1 --2- 172.21.15.84:0/2738441431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307806c3f0 0x7f307806c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.750+0000 7f307fab4640 1 -- 172.21.15.84:0/2738441431 >> 172.21.15.84:0/2738441431 conn(0x7f30780d2cb0 msgr2=0x7f30780d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:26.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.750+0000 7f307fab4640 1 -- 172.21.15.84:0/2738441431 shutdown_connections 2023-10-16T11:22:26.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.750+0000 7f307fab4640 1 -- 172.21.15.84:0/2738441431 wait complete. 2023-10-16T11:22:26.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.752+0000 7f307fab4640 1 Processor -- start 2023-10-16T11:22:26.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.752+0000 7f307fab4640 1 -- start start 2023-10-16T11:22:26.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.753+0000 7f307fab4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307806c3f0 0x7f30780cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.753+0000 7f307fab4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307815e0c0 0x7f3078158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.753+0000 7f307fab4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f307815e540 0x7f30781568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.753+0000 7f307fab4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307815c4f0 con 0x7f307815e0c0 2023-10-16T11:22:26.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.753+0000 7f307fab4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3078158fc0 con 0x7f307815e540 2023-10-16T11:22:26.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.753+0000 7f307fab4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30780ccf00 con 0x7f307806c3f0 2023-10-16T11:22:26.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.753+0000 7f307d04f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307815e0c0 0x7f3078158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.753+0000 7f307d04f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307815e0c0 0x7f3078158a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49524/0 (socket says 172.21.15.84:49524) 2023-10-16T11:22:26.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.753+0000 7f307d04f640 1 -- 172.21.15.84:0/1012161253 learned_addr learned my addr 172.21.15.84:0/1012161253 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:26.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.754+0000 7f307d04f640 1 -- 172.21.15.84:0/1012161253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307806c3f0 msgr2=0x7f30780cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.754+0000 7f307d04f640 1 --2- 172.21.15.84:0/1012161253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307806c3f0 0x7f30780cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.754+0000 7f307d04f640 1 -- 172.21.15.84:0/1012161253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f307815e540 msgr2=0x7f30781568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.754+0000 7f307d04f640 1 --2- 172.21.15.84:0/1012161253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f307815e540 0x7f30781568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.754+0000 7f307d04f640 1 -- 172.21.15.84:0/1012161253 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30700094d0 con 0x7f307815e0c0 2023-10-16T11:22:26.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.754+0000 7f307d04f640 1 --2- 172.21.15.84:0/1012161253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307815e0c0 0x7f3078158a80 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f307400c7c0 tx=0x7f307400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:26.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.754+0000 7f306e7fc640 1 -- 172.21.15.84:0/1012161253 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307400aa50 con 0x7f307815e0c0 2023-10-16T11:22:26.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.754+0000 7f306e7fc640 1 -- 172.21.15.84:0/1012161253 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307400abb0 con 0x7f307815e0c0 2023-10-16T11:22:26.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.754+0000 7f306e7fc640 1 -- 172.21.15.84:0/1012161253 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074012420 con 0x7f307815e0c0 2023-10-16T11:22:26.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.754+0000 7f307fab4640 1 -- 172.21.15.84:0/1012161253 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307815bb90 con 0x7f307815e0c0 2023-10-16T11:22:26.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.755+0000 7f307fab4640 1 -- 172.21.15.84:0/1012161253 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307815be30 con 0x7f307815e0c0 2023-10-16T11:22:26.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.756+0000 7f307fab4640 1 -- 172.21.15.84:0/1012161253 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30780cdb40 con 0x7f307815e0c0 2023-10-16T11:22:26.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.757+0000 7f306e7fc640 1 -- 172.21.15.84:0/1012161253 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 120) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f3074012580 con 0x7f307815e0c0 2023-10-16T11:22:26.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.757+0000 7f306e7fc640 1 --2- 172.21.15.84:0/1012161253 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f306406acc0 0x7f306406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.757+0000 7f306e7fc640 1 -- 172.21.15.84:0/1012161253 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6797+0+0 (secure 0 0 0) 0x7f30740898d0 con 0x7f307815e0c0 2023-10-16T11:22:26.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.758+0000 7f307d850640 1 --2- 172.21.15.84:0/1012161253 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f306406acc0 0x7f306406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.758+0000 7f307d850640 1 --2- 172.21.15.84:0/1012161253 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f306406acc0 0x7f306406d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3068006fd0 tx=0x7f3068008040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:26.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.762+0000 7f306e7fc640 1 -- 172.21.15.84:0/1012161253 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f30780cdb40 con 0x7f307815e0c0 2023-10-16T11:22:26.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.955+0000 7f307fab4640 1 -- 172.21.15.84:0/1012161253 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30780cdb40 con 0x7f307815e0c0 2023-10-16T11:22:26.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.957+0000 7f306e7fc640 1 -- 172.21.15.84:0/1012161253 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+18159 (secure 0 0 0) 0x7f30780cdb40 con 0x7f307815e0c0 2023-10-16T11:22:26.959 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:26.959 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":189,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:22:25.240193+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:06:20.873257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-16T11:06:22.065944+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":188,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/207496168":"2023-10-17T11:22:25.240138+0000","172.21.15.84:0/2289397951":"2023-10-17T11:22:25.240138+0000","172.21.15.84:0/3792094619":"2023-10-17T11:22:25.240138+0000","172.21.15.84:6833/285590107":"2023-10-17T11:22:25.240138+0000","172.21.15.84:6832/285590107":"2023-10-17T11:22:25.240138+0000","172.21.15.84:0/2324168371":"2023-10-17T11:22:25.240138+0000","172.21.15.84:0/1509521423":"2023-10-17T11:22:25.240138+0000","172.21.15.84:0/542522615":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/878936990":"2023-10-17T11:17:21.065656+0000","172.21.15.84:6833/3993913200":"2023-10-17T11:17:21.065656+0000","172.21.15.84:6832/3993913200":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/1064178200":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/3548457413":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/2013731252":"2023-10-17T11:17:21.065656+0000","172.21.15.84:6833/1441051579":"2023-10-17T11:06:37.825968+0000","172.21.15.84:6832/1441051579":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/2359294858":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/2830443959":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/4150685824":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/4095400472":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/3348937827":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/2158345505":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/563672373":"2023-10-17T11:22:25.240138+0000","172.21.15.84:0/235070477":"2023-10-17T11:06:37.825968+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":23,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:22:26.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.960+0000 7f307fab4640 1 -- 172.21.15.84:0/1012161253 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f306406acc0 msgr2=0x7f306406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.960+0000 7f307fab4640 1 --2- 172.21.15.84:0/1012161253 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f306406acc0 0x7f306406d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3068006fd0 tx=0x7f3068008040 comp rx=0 tx=0).stop 2023-10-16T11:22:26.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.960+0000 7f307fab4640 1 -- 172.21.15.84:0/1012161253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307815e0c0 msgr2=0x7f3078158a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.960+0000 7f307fab4640 1 --2- 172.21.15.84:0/1012161253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307815e0c0 0x7f3078158a80 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f307400c7c0 tx=0x7f307400cc80 comp rx=0 tx=0).stop 2023-10-16T11:22:26.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.960+0000 7f307fab4640 1 -- 172.21.15.84:0/1012161253 shutdown_connections 2023-10-16T11:22:26.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.960+0000 7f307fab4640 1 --2- 172.21.15.84:0/1012161253 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f306406acc0 0x7f306406d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.960+0000 7f307fab4640 1 --2- 172.21.15.84:0/1012161253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f307815e540 0x7f30781568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.960+0000 7f307fab4640 1 --2- 172.21.15.84:0/1012161253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307815e0c0 0x7f3078158a80 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.960+0000 7f307fab4640 1 --2- 172.21.15.84:0/1012161253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f307806c3f0 0x7f30780cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.960+0000 7f307fab4640 1 -- 172.21.15.84:0/1012161253 >> 172.21.15.84:0/1012161253 conn(0x7f30780d2cb0 msgr2=0x7f30780b4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:26.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.961+0000 7f307fab4640 1 -- 172.21.15.84:0/1012161253 shutdown_connections 2023-10-16T11:22:26.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:26.961+0000 7f307fab4640 1 -- 172.21.15.84:0/1012161253 wait complete. 2023-10-16T11:22:26.980 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T11:22:26.980 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:22:27.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.186+0000 7f3e401e9640 1 Processor -- start 2023-10-16T11:22:27.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.186+0000 7f3e401e9640 1 -- start start 2023-10-16T11:22:27.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.188+0000 7f3e401e9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3e3806c3f0 0x7f3e3806c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:27.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.188+0000 7f3e401e9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e380cd630 con 0x7f3e3815e540 2023-10-16T11:22:27.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.188+0000 7f3e401e9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e380ce1b0 con 0x7f3e3815e0c0 2023-10-16T11:22:27.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.188+0000 7f3e401e9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e3806cd10 con 0x7f3e3806c3f0 2023-10-16T11:22:27.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.188+0000 7f3e3df85640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3e3806c3f0 0x7f3e3806c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:27.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.188+0000 7f3e3df85640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3e3806c3f0 0x7f3e3806c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49534/0 (socket says 172.21.15.84:49534) 2023-10-16T11:22:27.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.188+0000 7f3e3e786640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3e3815e540 0x7f3e380b1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60566/0 (socket says 172.21.15.84:60566) 2023-10-16T11:22:27.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.188+0000 7f3e3df85640 1 -- 172.21.15.84:0/271859479 learned_addr learned my addr 172.21.15.84:0/271859479 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:27.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.189+0000 7f3e3df85640 1 -- 172.21.15.84:0/271859479 >> v1:172.21.15.157:6789/0 conn(0x7f3e3815e0c0 legacy=0x7f3e380b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:27.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.189+0000 7f3e3df85640 1 -- 172.21.15.84:0/271859479 >> v1:172.21.15.84:6789/0 conn(0x7f3e3815e540 legacy=0x7f3e380b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:27.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.189+0000 7f3e3df85640 1 -- 172.21.15.84:0/271859479 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e380d2170 con 0x7f3e3806c3f0 2023-10-16T11:22:27.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.189+0000 7f3e3cf83640 1 -- 172.21.15.84:0/271859479 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1927017975 0 0) 0x7f3e380cd630 con 0x7f3e3815e540 2023-10-16T11:22:27.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.190+0000 7f3e3df85640 1 --2- 172.21.15.84:0/271859479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e3806c3f0 0x7f3e3806c7d0 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009870 tx=0x7f3e3402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f33a1abb0a98a71 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:27.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.190+0000 7f3e3cf83640 1 -- 172.21.15.84:0/271859479 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34030860 con 0x7f3e3806c3f0 2023-10-16T11:22:27.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.191+0000 7f3e3cf83640 1 -- 172.21.15.84:0/271859479 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e340309c0 con 0x7f3e3806c3f0 2023-10-16T11:22:27.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.191+0000 7f3e3cf83640 1 -- 172.21.15.84:0/271859479 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34030d50 con 0x7f3e3806c3f0 2023-10-16T11:22:27.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.191+0000 7f3e401e9640 1 -- 172.21.15.84:0/271859479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e3806c3f0 msgr2=0x7f3e3806c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:27.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.191+0000 7f3e401e9640 1 --2- 172.21.15.84:0/271859479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e3806c3f0 0x7f3e3806c7d0 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009870 tx=0x7f3e3402b9a0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.191+0000 7f3e401e9640 1 -- 172.21.15.84:0/271859479 shutdown_connections 2023-10-16T11:22:27.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.191+0000 7f3e401e9640 1 --2- 172.21.15.84:0/271859479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e3806c3f0 0x7f3e3806c7d0 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.191+0000 7f3e401e9640 1 -- 172.21.15.84:0/271859479 >> 172.21.15.84:0/271859479 conn(0x7f3e380d2cb0 msgr2=0x7f3e380d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:27.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.191+0000 7f3e401e9640 1 -- 172.21.15.84:0/271859479 shutdown_connections 2023-10-16T11:22:27.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.191+0000 7f3e401e9640 1 -- 172.21.15.84:0/271859479 wait complete. 2023-10-16T11:22:27.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.193+0000 7f3e401e9640 1 Processor -- start 2023-10-16T11:22:27.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.193+0000 7f3e401e9640 1 -- start start 2023-10-16T11:22:27.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.195+0000 7f3e401e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e3806c3f0 0x7f3e38154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:27.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.195+0000 7f3e401e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e3815e0c0 0x7f3e3815a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:27.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.195+0000 7f3e401e9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e3815e540 0x7f3e3815eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:27.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.195+0000 7f3e401e9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e380cace0 con 0x7f3e3815e0c0 2023-10-16T11:22:27.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.195+0000 7f3e401e9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e38155380 con 0x7f3e3806c3f0 2023-10-16T11:22:27.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.195+0000 7f3e401e9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e380ca060 con 0x7f3e3815e540 2023-10-16T11:22:27.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.195+0000 7f3e3d784640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e3815e0c0 0x7f3e3815a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:27.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.195+0000 7f3e3d784640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e3815e0c0 0x7f3e3815a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49536/0 (socket says 172.21.15.84:49536) 2023-10-16T11:22:27.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.195+0000 7f3e3d784640 1 -- 172.21.15.84:0/3576667625 learned_addr learned my addr 172.21.15.84:0/3576667625 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:27.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.195+0000 7f3e3df85640 1 --2- 172.21.15.84:0/3576667625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e3806c3f0 0x7f3e38154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:27.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.195+0000 7f3e3d784640 1 -- 172.21.15.84:0/3576667625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e3815e540 msgr2=0x7f3e3815eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:27.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.195+0000 7f3e3d784640 1 --2- 172.21.15.84:0/3576667625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e3815e540 0x7f3e3815eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.195+0000 7f3e3d784640 1 -- 172.21.15.84:0/3576667625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e3806c3f0 msgr2=0x7f3e38154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:27.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.195+0000 7f3e3d784640 1 --2- 172.21.15.84:0/3576667625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e3806c3f0 0x7f3e38154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.195+0000 7f3e3d784640 1 -- 172.21.15.84:0/3576667625 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e340094d0 con 0x7f3e3815e0c0 2023-10-16T11:22:27.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.196+0000 7f3e3d784640 1 --2- 172.21.15.84:0/3576667625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e3815e0c0 0x7f3e3815a5c0 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800b3f0 tx=0x7f3e2800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:27.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.196+0000 7f3e2e7fc640 1 -- 172.21.15.84:0/3576667625 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e28011830 con 0x7f3e3815e0c0 2023-10-16T11:22:27.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.196+0000 7f3e2e7fc640 1 -- 172.21.15.84:0/3576667625 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e28011990 con 0x7f3e3815e0c0 2023-10-16T11:22:27.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.196+0000 7f3e2e7fc640 1 -- 172.21.15.84:0/3576667625 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e28011d40 con 0x7f3e3815e0c0 2023-10-16T11:22:27.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.196+0000 7f3e401e9640 1 -- 172.21.15.84:0/3576667625 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e380ca340 con 0x7f3e3815e0c0 2023-10-16T11:22:27.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.196+0000 7f3e401e9640 1 -- 172.21.15.84:0/3576667625 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e380ca900 con 0x7f3e3815e0c0 2023-10-16T11:22:27.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.197+0000 7f3e401e9640 1 -- 172.21.15.84:0/3576667625 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e380d1f60 con 0x7f3e3815e0c0 2023-10-16T11:22:27.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.198+0000 7f3e2e7fc640 1 -- 172.21.15.84:0/3576667625 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 120) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f3e2801a400 con 0x7f3e3815e0c0 2023-10-16T11:22:27.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.199+0000 7f3e2e7fc640 1 --2- 172.21.15.84:0/3576667625 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f3e1c06ac90 0x7f3e1c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:27.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.199+0000 7f3e2e7fc640 1 -- 172.21.15.84:0/3576667625 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6797+0+0 (secure 0 0 0) 0x7f3e28089590 con 0x7f3e3815e0c0 2023-10-16T11:22:27.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.199+0000 7f3e3df85640 1 --2- 172.21.15.84:0/3576667625 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f3e1c06ac90 0x7f3e1c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:27.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.200+0000 7f3e3df85640 1 --2- 172.21.15.84:0/3576667625 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f3e1c06ac90 0x7f3e1c06d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3e34036000 tx=0x7f3e3403b040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:27.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.203+0000 7f3e2e7fc640 1 -- 172.21.15.84:0/3576667625 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3e28054470 con 0x7f3e3815e0c0 2023-10-16T11:22:27.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.441+0000 7f3e401e9640 1 -- 172.21.15.84:0/3576667625 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e381561a0 con 0x7f3e3815e0c0 2023-10-16T11:22:27.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.442+0000 7f3e2e7fc640 1 -- 172.21.15.84:0/3576667625 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f3e28057a90 con 0x7f3e3815e0c0 2023-10-16T11:22:27.443 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:27.443 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6347,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1346424254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1346424254},{"type":"v1","addr":"172.21.15.157:6835","nonce":1346424254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3224476768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3224476768},{"type":"v1","addr":"172.21.15.157:6834","nonce":3224476768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":10197,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2490106026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2490106026},{"type":"v1","addr":"172.21.15.84:6835","nonce":2490106026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:06:23.034809+0000","modified":"2023-10-16T11:06:26.105994+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10206},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10206":{"gid":10206,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/1106170430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":1106170430},{"type":"v1","addr":"172.21.15.84:6837","nonce":1106170430}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-16T11:22:27.444 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 96 2023-10-16T11:22:27.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.445+0000 7f3e401e9640 1 -- 172.21.15.84:0/3576667625 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f3e1c06ac90 msgr2=0x7f3e1c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:27.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.445+0000 7f3e401e9640 1 --2- 172.21.15.84:0/3576667625 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f3e1c06ac90 0x7f3e1c06d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3e34036000 tx=0x7f3e3403b040 comp rx=0 tx=0).stop 2023-10-16T11:22:27.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.445+0000 7f3e401e9640 1 -- 172.21.15.84:0/3576667625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e3815e0c0 msgr2=0x7f3e3815a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:27.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.445+0000 7f3e401e9640 1 --2- 172.21.15.84:0/3576667625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e3815e0c0 0x7f3e3815a5c0 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800b3f0 tx=0x7f3e2800b8b0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.445+0000 7f3e401e9640 1 -- 172.21.15.84:0/3576667625 shutdown_connections 2023-10-16T11:22:27.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.445+0000 7f3e401e9640 1 --2- 172.21.15.84:0/3576667625 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f3e1c06ac90 0x7f3e1c06d150 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.445+0000 7f3e401e9640 1 --2- 172.21.15.84:0/3576667625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e3815e540 0x7f3e3815eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.445+0000 7f3e401e9640 1 --2- 172.21.15.84:0/3576667625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e3815e0c0 0x7f3e3815a5c0 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.445+0000 7f3e401e9640 1 --2- 172.21.15.84:0/3576667625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e3806c3f0 0x7f3e38154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.445+0000 7f3e401e9640 1 -- 172.21.15.84:0/3576667625 >> 172.21.15.84:0/3576667625 conn(0x7f3e380d2cb0 msgr2=0x7f3e38155a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:27.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.445+0000 7f3e401e9640 1 -- 172.21.15.84:0/3576667625 shutdown_connections 2023-10-16T11:22:27.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.445+0000 7f3e401e9640 1 -- 172.21.15.84:0/3576667625 wait complete. 2023-10-16T11:22:27.464 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:22:27.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.654+0000 7f3b9f4c6640 1 Processor -- start 2023-10-16T11:22:27.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.654+0000 7f3b9f4c6640 1 -- start start 2023-10-16T11:22:27.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.655+0000 7f3b9f4c6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b980cd0c0 0x7f3b981615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:27.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.655+0000 7f3b9f4c6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3b980b0410 0x7f3b980b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:27.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.655+0000 7f3b9f4c6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b980b0db0 con 0x7f3b9815d3f0 2023-10-16T11:22:27.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.655+0000 7f3b9f4c6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b9815e3e0 con 0x7f3b980b0410 2023-10-16T11:22:27.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.655+0000 7f3b9f4c6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b980cd4a0 con 0x7f3b980cd0c0 2023-10-16T11:22:27.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.656+0000 7f3b9ca61640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3b980b0410 0x7f3b980b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:27.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.656+0000 7f3b9da63640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3b9815d3f0 0x7f3b9815d880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60570/0 (socket says 172.21.15.84:60570) 2023-10-16T11:22:27.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.656+0000 7f3b9da63640 1 -- 172.21.15.84:0/3388354802 learned_addr learned my addr 172.21.15.84:0/3388354802 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:27.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.656+0000 7f3b9ca61640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3b980b0410 0x7f3b980b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49564/0 (socket says 172.21.15.84:49564) 2023-10-16T11:22:27.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.656+0000 7f3b9d262640 1 --2- 172.21.15.84:0/3388354802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b980cd0c0 0x7f3b981615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:27.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.656+0000 7f3b9ca61640 1 -- 172.21.15.84:0/3388354802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b980cd0c0 msgr2=0x7f3b981615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:27.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.656+0000 7f3b9ca61640 1 --2- 172.21.15.84:0/3388354802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b980cd0c0 0x7f3b981615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.656+0000 7f3b9ca61640 1 -- 172.21.15.84:0/3388354802 >> v1:172.21.15.84:6789/0 conn(0x7f3b9815d3f0 legacy=0x7f3b9815d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:27.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.656+0000 7f3b9ca61640 1 -- 172.21.15.84:0/3388354802 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9815e100 con 0x7f3b980b0410 2023-10-16T11:22:27.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.657+0000 7f3b9ca61640 1 --2- 172.21.15.84:0/3388354802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b980b0410 0x7f3b980b0870 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f3b880097a0 tx=0x7f3b8802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee22359257772e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:27.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.658+0000 7f3b8ffff640 1 -- 172.21.15.84:0/3388354802 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88031780 con 0x7f3b980b0410 2023-10-16T11:22:27.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.658+0000 7f3b8ffff640 1 -- 172.21.15.84:0/3388354802 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b880318e0 con 0x7f3b980b0410 2023-10-16T11:22:27.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.658+0000 7f3b8ffff640 1 -- 172.21.15.84:0/3388354802 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88031c70 con 0x7f3b980b0410 2023-10-16T11:22:27.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.658+0000 7f3b9f4c6640 1 -- 172.21.15.84:0/3388354802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b980b0410 msgr2=0x7f3b980b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:27.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.658+0000 7f3b9f4c6640 1 --2- 172.21.15.84:0/3388354802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b980b0410 0x7f3b980b0870 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f3b880097a0 tx=0x7f3b8802b820 comp rx=0 tx=0).stop 2023-10-16T11:22:27.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.659+0000 7f3b9f4c6640 1 -- 172.21.15.84:0/3388354802 shutdown_connections 2023-10-16T11:22:27.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.659+0000 7f3b9f4c6640 1 --2- 172.21.15.84:0/3388354802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b980b0410 0x7f3b980b0870 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.659+0000 7f3b9f4c6640 1 --2- 172.21.15.84:0/3388354802 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b980cd0c0 0x7f3b981615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.659+0000 7f3b9f4c6640 1 -- 172.21.15.84:0/3388354802 >> 172.21.15.84:0/3388354802 conn(0x7f3b980cb410 msgr2=0x7f3b980cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:27.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.659+0000 7f3b9f4c6640 1 -- 172.21.15.84:0/3388354802 shutdown_connections 2023-10-16T11:22:27.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.659+0000 7f3b9f4c6640 1 -- 172.21.15.84:0/3388354802 wait complete. 2023-10-16T11:22:27.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.660+0000 7f3b9f4c6640 1 Processor -- start 2023-10-16T11:22:27.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.660+0000 7f3b9f4c6640 1 -- start start 2023-10-16T11:22:27.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.661+0000 7f3b9f4c6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b980b0410 0x7f3b98156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:27.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.661+0000 7f3b9f4c6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b980cd0c0 0x7f3b980b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:27.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.661+0000 7f3b9f4c6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9815d3f0 0x7f3b980b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:27.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.661+0000 7f3b9f4c6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b980c5290 con 0x7f3b980cd0c0 2023-10-16T11:22:27.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.661+0000 7f3b9f4c6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b980b1570 con 0x7f3b9815d3f0 2023-10-16T11:22:27.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.661+0000 7f3b9f4c6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b980b1850 con 0x7f3b980b0410 2023-10-16T11:22:27.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.662+0000 7f3b9ca61640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b980cd0c0 0x7f3b980b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:27.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.662+0000 7f3b9ca61640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b980cd0c0 0x7f3b980b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49580/0 (socket says 172.21.15.84:49580) 2023-10-16T11:22:27.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.662+0000 7f3b9ca61640 1 -- 172.21.15.84:0/1833686311 learned_addr learned my addr 172.21.15.84:0/1833686311 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:27.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.662+0000 7f3b9da63640 1 --2- 172.21.15.84:0/1833686311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9815d3f0 0x7f3b980b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:27.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.662+0000 7f3b9d262640 1 --2- 172.21.15.84:0/1833686311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b980b0410 0x7f3b98156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:27.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.662+0000 7f3b9ca61640 1 -- 172.21.15.84:0/1833686311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b980b0410 msgr2=0x7f3b98156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:27.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.662+0000 7f3b9ca61640 1 --2- 172.21.15.84:0/1833686311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b980b0410 0x7f3b98156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.662+0000 7f3b9ca61640 1 -- 172.21.15.84:0/1833686311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9815d3f0 msgr2=0x7f3b980b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:27.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.662+0000 7f3b9ca61640 1 --2- 172.21.15.84:0/1833686311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9815d3f0 0x7f3b980b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.662+0000 7f3b9ca61640 1 -- 172.21.15.84:0/1833686311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b88009400 con 0x7f3b980cd0c0 2023-10-16T11:22:27.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.662+0000 7f3b9d262640 1 --2- 172.21.15.84:0/1833686311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b980b0410 0x7f3b98156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:27.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.662+0000 7f3b9ca61640 1 --2- 172.21.15.84:0/1833686311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b980cd0c0 0x7f3b980b3060 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f3b88034040 tx=0x7f3b880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:27.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.663+0000 7f3b8dffb640 1 -- 172.21.15.84:0/1833686311 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88032820 con 0x7f3b980cd0c0 2023-10-16T11:22:27.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.663+0000 7f3b8dffb640 1 -- 172.21.15.84:0/1833686311 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b88032980 con 0x7f3b980cd0c0 2023-10-16T11:22:27.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.663+0000 7f3b8dffb640 1 -- 172.21.15.84:0/1833686311 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88032d30 con 0x7f3b980cd0c0 2023-10-16T11:22:27.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.663+0000 7f3b9f4c6640 1 -- 172.21.15.84:0/1833686311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b980c4cd0 con 0x7f3b980cd0c0 2023-10-16T11:22:27.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.663+0000 7f3b9f4c6640 1 -- 172.21.15.84:0/1833686311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b980b1b30 con 0x7f3b980cd0c0 2023-10-16T11:22:27.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.664+0000 7f3b9f4c6640 1 -- 172.21.15.84:0/1833686311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b980add80 con 0x7f3b980cd0c0 2023-10-16T11:22:27.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.665+0000 7f3b8dffb640 1 -- 172.21.15.84:0/1833686311 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 120) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f3b8804e440 con 0x7f3b980cd0c0 2023-10-16T11:22:27.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.666+0000 7f3b8dffb640 1 --2- 172.21.15.84:0/1833686311 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f3b7c06acc0 0x7f3b7c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:27.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.666+0000 7f3b8dffb640 1 -- 172.21.15.84:0/1833686311 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6797+0+0 (secure 0 0 0) 0x7f3b880ab8c0 con 0x7f3b980cd0c0 2023-10-16T11:22:27.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.666+0000 7f3b9d262640 1 --2- 172.21.15.84:0/1833686311 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f3b7c06acc0 0x7f3b7c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:27.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.667+0000 7f3b9d262640 1 --2- 172.21.15.84:0/1833686311 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f3b7c06acc0 0x7f3b7c06d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3b94006fd0 tx=0x7f3b94008040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:27.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.670+0000 7f3b8dffb640 1 -- 172.21.15.84:0/1833686311 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3b980add80 con 0x7f3b980cd0c0 2023-10-16T11:22:27.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.899+0000 7f3b9f4c6640 1 -- 172.21.15.84:0/1833686311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b980add80 con 0x7f3b980cd0c0 2023-10-16T11:22:27.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.900+0000 7f3b8dffb640 1 -- 172.21.15.84:0/1833686311 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f3b980add80 con 0x7f3b980cd0c0 2023-10-16T11:22:27.902 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:27.902 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6347,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1346424254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1346424254},{"type":"v1","addr":"172.21.15.157:6835","nonce":1346424254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3224476768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3224476768},{"type":"v1","addr":"172.21.15.157:6834","nonce":3224476768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":10197,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2490106026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2490106026},{"type":"v1","addr":"172.21.15.84:6835","nonce":2490106026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:06:23.034809+0000","modified":"2023-10-16T11:06:26.105994+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10206},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10206":{"gid":10206,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/1106170430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":1106170430},{"type":"v1","addr":"172.21.15.84:6837","nonce":1106170430}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-16T11:22:27.903 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 96 2023-10-16T11:22:27.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.903+0000 7f3b9f4c6640 1 -- 172.21.15.84:0/1833686311 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f3b7c06acc0 msgr2=0x7f3b7c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:27.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.903+0000 7f3b9f4c6640 1 --2- 172.21.15.84:0/1833686311 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f3b7c06acc0 0x7f3b7c06d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3b94006fd0 tx=0x7f3b94008040 comp rx=0 tx=0).stop 2023-10-16T11:22:27.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.903+0000 7f3b9f4c6640 1 -- 172.21.15.84:0/1833686311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b980cd0c0 msgr2=0x7f3b980b3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:27.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.903+0000 7f3b9f4c6640 1 --2- 172.21.15.84:0/1833686311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b980cd0c0 0x7f3b980b3060 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f3b88034040 tx=0x7f3b880046c0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.903+0000 7f3b9f4c6640 1 -- 172.21.15.84:0/1833686311 shutdown_connections 2023-10-16T11:22:27.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.903+0000 7f3b9f4c6640 1 --2- 172.21.15.84:0/1833686311 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f3b7c06acc0 0x7f3b7c06d180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.903+0000 7f3b9f4c6640 1 --2- 172.21.15.84:0/1833686311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9815d3f0 0x7f3b980b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.903+0000 7f3b9f4c6640 1 --2- 172.21.15.84:0/1833686311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b980cd0c0 0x7f3b980b3060 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.903+0000 7f3b9f4c6640 1 --2- 172.21.15.84:0/1833686311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b980b0410 0x7f3b98156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:27.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.903+0000 7f3b9f4c6640 1 -- 172.21.15.84:0/1833686311 >> 172.21.15.84:0/1833686311 conn(0x7f3b980cb410 msgr2=0x7f3b980c9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:27.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.903+0000 7f3b9f4c6640 1 -- 172.21.15.84:0/1833686311 shutdown_connections 2023-10-16T11:22:27.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:27.903+0000 7f3b9f4c6640 1 -- 172.21.15.84:0/1833686311 wait complete. 2023-10-16T11:22:27.923 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:22:28.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.128+0000 7fc8c32a2640 1 Processor -- start 2023-10-16T11:22:28.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.128+0000 7fc8c32a2640 1 -- start start 2023-10-16T11:22:28.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.129+0000 7fc8c32a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8bc15ae80 0x7fc8bc15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.129+0000 7fc8c32a2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc8bc0ce390 0x7fc8bc070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.129+0000 7fc8c32a2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8bc15ed60 con 0x7fc8bc15aaa0 2023-10-16T11:22:28.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.129+0000 7fc8c32a2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8bc06b810 con 0x7fc8bc0ce390 2023-10-16T11:22:28.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.129+0000 7fc8c32a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc0707e0 con 0x7fc8bc15ae80 2023-10-16T11:22:28.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.130+0000 7fc8c183f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc8bc0ce390 0x7fc8bc070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.130+0000 7fc8c183f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc8bc0ce390 0x7fc8bc070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49592/0 (socket says 172.21.15.84:49592) 2023-10-16T11:22:28.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.130+0000 7fc8c183f640 1 -- 172.21.15.84:0/1797503490 learned_addr learned my addr 172.21.15.84:0/1797503490 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:28.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.130+0000 7fc8c083d640 1 --2- 172.21.15.84:0/1797503490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8bc15ae80 0x7fc8bc15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.130+0000 7fc8c183f640 1 -- 172.21.15.84:0/1797503490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8bc15ae80 msgr2=0x7fc8bc15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:28.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.130+0000 7fc8c183f640 1 --2- 172.21.15.84:0/1797503490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8bc15ae80 0x7fc8bc15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.130+0000 7fc8c183f640 1 -- 172.21.15.84:0/1797503490 >> v1:172.21.15.157:6789/0 conn(0x7fc8bc15aaa0 legacy=0x7fc8bc0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:28.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.130+0000 7fc8c183f640 1 -- 172.21.15.84:0/1797503490 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8bc15ef90 con 0x7fc8bc0ce390 2023-10-16T11:22:28.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.130+0000 7fc8c083d640 1 --2- 172.21.15.84:0/1797503490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8bc15ae80 0x7fc8bc15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:28.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.131+0000 7fc8c183f640 1 --2- 172.21.15.84:0/1797503490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8bc0ce390 0x7fc8bc070f50 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4009e50 tx=0x7fc8b402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370f23a0a99c6b51 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.132+0000 7fc8b3fff640 1 -- 172.21.15.84:0/1797503490 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b402ca80 con 0x7fc8bc0ce390 2023-10-16T11:22:28.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.132+0000 7fc8b3fff640 1 -- 172.21.15.84:0/1797503490 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b402cbe0 con 0x7fc8bc0ce390 2023-10-16T11:22:28.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.132+0000 7fc8b3fff640 1 -- 172.21.15.84:0/1797503490 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b40343f0 con 0x7fc8bc0ce390 2023-10-16T11:22:28.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.132+0000 7fc8c32a2640 1 -- 172.21.15.84:0/1797503490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8bc0ce390 msgr2=0x7fc8bc070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.132+0000 7fc8c32a2640 1 --2- 172.21.15.84:0/1797503490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8bc0ce390 0x7fc8bc070f50 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4009e50 tx=0x7fc8b402bee0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.132+0000 7fc8c32a2640 1 -- 172.21.15.84:0/1797503490 shutdown_connections 2023-10-16T11:22:28.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.132+0000 7fc8c32a2640 1 --2- 172.21.15.84:0/1797503490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8bc0ce390 0x7fc8bc070f50 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.132+0000 7fc8c32a2640 1 --2- 172.21.15.84:0/1797503490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8bc15ae80 0x7fc8bc15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.132+0000 7fc8c32a2640 1 -- 172.21.15.84:0/1797503490 >> 172.21.15.84:0/1797503490 conn(0x7fc8bc0cb410 msgr2=0x7fc8bc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:28.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.132+0000 7fc8c32a2640 1 -- 172.21.15.84:0/1797503490 shutdown_connections 2023-10-16T11:22:28.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.132+0000 7fc8c32a2640 1 -- 172.21.15.84:0/1797503490 wait complete. 2023-10-16T11:22:28.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.134+0000 7fc8c32a2640 1 Processor -- start 2023-10-16T11:22:28.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.134+0000 7fc8c32a2640 1 -- start start 2023-10-16T11:22:28.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.135+0000 7fc8c32a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8bc0ce390 0x7fc8bc155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.135+0000 7fc8c32a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8bc15aaa0 0x7fc8bc155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.135+0000 7fc8c32a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8bc15ae80 0x7fc8bc0c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.135+0000 7fc8c32a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc155a80 con 0x7fc8bc0ce390 2023-10-16T11:22:28.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.135+0000 7fc8c32a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc15b480 con 0x7fc8bc15aaa0 2023-10-16T11:22:28.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.135+0000 7fc8c32a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc15b760 con 0x7fc8bc15ae80 2023-10-16T11:22:28.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.135+0000 7fc8c103e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8bc0ce390 0x7fc8bc155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.135+0000 7fc8c103e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8bc0ce390 0x7fc8bc155000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49604/0 (socket says 172.21.15.84:49604) 2023-10-16T11:22:28.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.135+0000 7fc8c103e640 1 -- 172.21.15.84:0/2304820136 learned_addr learned my addr 172.21.15.84:0/2304820136 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:28.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.136+0000 7fc8c183f640 1 --2- 172.21.15.84:0/2304820136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8bc15ae80 0x7fc8bc0c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.136+0000 7fc8c083d640 1 --2- 172.21.15.84:0/2304820136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8bc15aaa0 0x7fc8bc155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.136+0000 7fc8c103e640 1 -- 172.21.15.84:0/2304820136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8bc15ae80 msgr2=0x7fc8bc0c24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.136+0000 7fc8c103e640 1 --2- 172.21.15.84:0/2304820136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8bc15ae80 0x7fc8bc0c24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.136+0000 7fc8c103e640 1 -- 172.21.15.84:0/2304820136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8bc15aaa0 msgr2=0x7fc8bc155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.136+0000 7fc8c103e640 1 --2- 172.21.15.84:0/2304820136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8bc15aaa0 0x7fc8bc155540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.136+0000 7fc8c103e640 1 -- 172.21.15.84:0/2304820136 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b4009b30 con 0x7fc8bc0ce390 2023-10-16T11:22:28.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.136+0000 7fc8c103e640 1 --2- 172.21.15.84:0/2304820136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8bc0ce390 0x7fc8bc155000 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800c430 tx=0x7fc8b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.136+0000 7fc8b1ffb640 1 -- 172.21.15.84:0/2304820136 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8011790 con 0x7fc8bc0ce390 2023-10-16T11:22:28.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.136+0000 7fc8b1ffb640 1 -- 172.21.15.84:0/2304820136 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b80118f0 con 0x7fc8bc0ce390 2023-10-16T11:22:28.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.136+0000 7fc8c32a2640 1 -- 172.21.15.84:0/2304820136 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8bc15ba40 con 0x7fc8bc0ce390 2023-10-16T11:22:28.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.136+0000 7fc8b1ffb640 1 -- 172.21.15.84:0/2304820136 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8011ce0 con 0x7fc8bc0ce390 2023-10-16T11:22:28.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.137+0000 7fc8c32a2640 1 -- 172.21.15.84:0/2304820136 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8bc15c000 con 0x7fc8bc0ce390 2023-10-16T11:22:28.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.138+0000 7fc8c32a2640 1 -- 172.21.15.84:0/2304820136 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8bc04e160 con 0x7fc8bc0ce390 2023-10-16T11:22:28.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.138+0000 7fc8b1ffb640 1 -- 172.21.15.84:0/2304820136 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 121) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fc8b801c460 con 0x7fc8bc0ce390 2023-10-16T11:22:28.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.140+0000 7fc8b1ffb640 1 --2- 172.21.15.84:0/2304820136 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc8a006af00 0x7fc8a006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.140+0000 7fc8b1ffb640 1 -- 172.21.15.84:0/2304820136 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6797+0+0 (secure 0 0 0) 0x7fc8b8089980 con 0x7fc8bc0ce390 2023-10-16T11:22:28.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.140+0000 7fc8c083d640 1 --2- 172.21.15.84:0/2304820136 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc8a006af00 0x7fc8a006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.141+0000 7fc8c083d640 1 --2- 172.21.15.84:0/2304820136 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc8a006af00 0x7fc8a006d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc15a230 tx=0x7fc8ac008040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.145+0000 7fc8b1ffb640 1 -- 172.21.15.84:0/2304820136 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc8b8054690 con 0x7fc8bc0ce390 2023-10-16T11:22:28.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.337+0000 7fc8c32a2640 1 -- 172.21.15.84:0/2304820136 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8bc15f520 con 0x7fc8bc0ce390 2023-10-16T11:22:28.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.340+0000 7fc8b1ffb640 1 -- 172.21.15.84:0/2304820136 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+18159 (secure 0 0 0) 0x7fc8b8057cb0 con 0x7fc8bc0ce390 2023-10-16T11:22:28.342 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:28.342 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":189,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:22:25.240193+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:06:20.873257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-16T11:06:22.065944+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":188,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/207496168":"2023-10-17T11:22:25.240138+0000","172.21.15.84:0/2289397951":"2023-10-17T11:22:25.240138+0000","172.21.15.84:0/3792094619":"2023-10-17T11:22:25.240138+0000","172.21.15.84:6833/285590107":"2023-10-17T11:22:25.240138+0000","172.21.15.84:6832/285590107":"2023-10-17T11:22:25.240138+0000","172.21.15.84:0/2324168371":"2023-10-17T11:22:25.240138+0000","172.21.15.84:0/1509521423":"2023-10-17T11:22:25.240138+0000","172.21.15.84:0/542522615":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/878936990":"2023-10-17T11:17:21.065656+0000","172.21.15.84:6833/3993913200":"2023-10-17T11:17:21.065656+0000","172.21.15.84:6832/3993913200":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/1064178200":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/3548457413":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/2013731252":"2023-10-17T11:17:21.065656+0000","172.21.15.84:6833/1441051579":"2023-10-17T11:06:37.825968+0000","172.21.15.84:6832/1441051579":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/2359294858":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/2830443959":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/4150685824":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/4095400472":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/3348937827":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/2158345505":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/563672373":"2023-10-17T11:22:25.240138+0000","172.21.15.84:0/235070477":"2023-10-17T11:06:37.825968+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":23,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:22:28.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.343+0000 7fc8c32a2640 1 -- 172.21.15.84:0/2304820136 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc8a006af00 msgr2=0x7fc8a006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.343+0000 7fc8c32a2640 1 --2- 172.21.15.84:0/2304820136 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc8a006af00 0x7fc8a006d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc15a230 tx=0x7fc8ac008040 comp rx=0 tx=0).stop 2023-10-16T11:22:28.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.343+0000 7fc8c32a2640 1 -- 172.21.15.84:0/2304820136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8bc0ce390 msgr2=0x7fc8bc155000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.343+0000 7fc8c32a2640 1 --2- 172.21.15.84:0/2304820136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8bc0ce390 0x7fc8bc155000 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800c430 tx=0x7fc8b800c8f0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.343+0000 7fc8c32a2640 1 -- 172.21.15.84:0/2304820136 shutdown_connections 2023-10-16T11:22:28.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.343+0000 7fc8c32a2640 1 --2- 172.21.15.84:0/2304820136 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc8a006af00 0x7fc8a006d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.343+0000 7fc8c32a2640 1 --2- 172.21.15.84:0/2304820136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8bc15ae80 0x7fc8bc0c24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.343+0000 7fc8c32a2640 1 --2- 172.21.15.84:0/2304820136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8bc15aaa0 0x7fc8bc155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.343+0000 7fc8c32a2640 1 --2- 172.21.15.84:0/2304820136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8bc0ce390 0x7fc8bc155000 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.343+0000 7fc8c32a2640 1 -- 172.21.15.84:0/2304820136 >> 172.21.15.84:0/2304820136 conn(0x7fc8bc0cb410 msgr2=0x7fc8bc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:28.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.343+0000 7fc8c32a2640 1 -- 172.21.15.84:0/2304820136 shutdown_connections 2023-10-16T11:22:28.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.344+0000 7fc8c32a2640 1 -- 172.21.15.84:0/2304820136 wait complete. 2023-10-16T11:22:28.363 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T11:22:28.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.583+0000 7feeec42b640 1 Processor -- start 2023-10-16T11:22:28.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.583+0000 7feeec42b640 1 -- start start 2023-10-16T11:22:28.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.585+0000 7feeec42b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feee406c850 0x7feee40ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.585+0000 7feeec42b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee4070360 con 0x7feee4159ab0 2023-10-16T11:22:28.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.585+0000 7feeec42b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee40cd1f0 con 0x7feee4159e90 2023-10-16T11:22:28.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.585+0000 7feeec42b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee415d890 con 0x7feee406c850 2023-10-16T11:22:28.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.585+0000 7feeea1c7640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7feee4159ab0 0x7feee406fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60572/0 (socket says 172.21.15.84:60572) 2023-10-16T11:22:28.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.585+0000 7feeea1c7640 1 -- 172.21.15.84:0/3502022440 learned_addr learned my addr 172.21.15.84:0/3502022440 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:28.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.585+0000 7feeea9c8640 1 --2- 172.21.15.84:0/3502022440 >> v2:172.21.15.84:3300/0 conn(0x7feee406c850 0x7feee40ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.586+0000 7feeea9c8640 1 -- 172.21.15.84:0/3502022440 >> v1:172.21.15.157:6789/0 conn(0x7feee4159e90 legacy=0x7feee406c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:28.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.586+0000 7feeea9c8640 1 -- 172.21.15.84:0/3502022440 >> v1:172.21.15.84:6789/0 conn(0x7feee4159ab0 legacy=0x7feee406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.586+0000 7feeea9c8640 1 -- 172.21.15.84:0/3502022440 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee40cd4d0 con 0x7feee406c850 2023-10-16T11:22:28.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.587+0000 7feeea9c8640 1 --2- 172.21.15.84:0/3502022440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee406c850 0x7feee40ce370 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7feedc009d90 tx=0x7feedc02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11028da639393715 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.587+0000 7feee91c5640 1 -- 172.21.15.84:0/3502022440 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc02cb40 con 0x7feee406c850 2023-10-16T11:22:28.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.587+0000 7feee91c5640 1 -- 172.21.15.84:0/3502022440 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feedc02cca0 con 0x7feee406c850 2023-10-16T11:22:28.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.587+0000 7feee91c5640 1 -- 172.21.15.84:0/3502022440 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc034660 con 0x7feee406c850 2023-10-16T11:22:28.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.588+0000 7feeec42b640 1 -- 172.21.15.84:0/3502022440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee406c850 msgr2=0x7feee40ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.588+0000 7feeec42b640 1 --2- 172.21.15.84:0/3502022440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee406c850 0x7feee40ce370 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7feedc009d90 tx=0x7feedc02bd90 comp rx=0 tx=0).stop 2023-10-16T11:22:28.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.588+0000 7feeec42b640 1 -- 172.21.15.84:0/3502022440 shutdown_connections 2023-10-16T11:22:28.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.588+0000 7feeec42b640 1 --2- 172.21.15.84:0/3502022440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee406c850 0x7feee40ce370 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.588+0000 7feeec42b640 1 -- 172.21.15.84:0/3502022440 >> 172.21.15.84:0/3502022440 conn(0x7feee40ce9b0 msgr2=0x7feee40cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:28.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.588+0000 7feeec42b640 1 -- 172.21.15.84:0/3502022440 shutdown_connections 2023-10-16T11:22:28.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.588+0000 7feeec42b640 1 -- 172.21.15.84:0/3502022440 wait complete. 2023-10-16T11:22:28.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.590+0000 7feeec42b640 1 Processor -- start 2023-10-16T11:22:28.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.590+0000 7feeec42b640 1 -- start start 2023-10-16T11:22:28.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.591+0000 7feeec42b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee406c850 0x7feee40c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.591+0000 7feeec42b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feee4159ab0 0x7feee40c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.591+0000 7feeec42b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee4159e90 0x7feee4157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.591+0000 7feeec42b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee40c69d0 con 0x7feee4159e90 2023-10-16T11:22:28.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.591+0000 7feeec42b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee40c8460 con 0x7feee4159ab0 2023-10-16T11:22:28.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.591+0000 7feeec42b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee40c7fd0 con 0x7feee406c850 2023-10-16T11:22:28.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.591+0000 7feeea9c8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee4159e90 0x7feee4157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.591+0000 7feeea9c8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee4159e90 0x7feee4157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49612/0 (socket says 172.21.15.84:49612) 2023-10-16T11:22:28.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.591+0000 7feeea9c8640 1 -- 172.21.15.84:0/3289863156 learned_addr learned my addr 172.21.15.84:0/3289863156 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:28.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.592+0000 7feee99c6640 1 --2- 172.21.15.84:0/3289863156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feee4159ab0 0x7feee40c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.592+0000 7feeea1c7640 1 --2- 172.21.15.84:0/3289863156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee406c850 0x7feee40c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.592+0000 7feeea9c8640 1 -- 172.21.15.84:0/3289863156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee406c850 msgr2=0x7feee40c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.592+0000 7feeea9c8640 1 --2- 172.21.15.84:0/3289863156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee406c850 0x7feee40c6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.592+0000 7feeea9c8640 1 -- 172.21.15.84:0/3289863156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feee4159ab0 msgr2=0x7feee40c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.592+0000 7feeea9c8640 1 --2- 172.21.15.84:0/3289863156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feee4159ab0 0x7feee40c6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.592+0000 7feeea9c8640 1 -- 172.21.15.84:0/3289863156 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc009a70 con 0x7feee4159e90 2023-10-16T11:22:28.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.592+0000 7feee99c6640 1 --2- 172.21.15.84:0/3289863156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feee4159ab0 0x7feee40c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:28.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.592+0000 7feeea9c8640 1 --2- 172.21.15.84:0/3289863156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee4159e90 0x7feee4157c40 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7feedc037040 tx=0x7feedc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.592+0000 7feedaffd640 1 -- 172.21.15.84:0/3289863156 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc040650 con 0x7feee4159e90 2023-10-16T11:22:28.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.593+0000 7feedaffd640 1 -- 172.21.15.84:0/3289863156 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feedc0407b0 con 0x7feee4159e90 2023-10-16T11:22:28.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.593+0000 7feedaffd640 1 -- 172.21.15.84:0/3289863156 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc040b60 con 0x7feee4159e90 2023-10-16T11:22:28.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.593+0000 7feeec42b640 1 -- 172.21.15.84:0/3289863156 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee40c73c0 con 0x7feee4159e90 2023-10-16T11:22:28.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.593+0000 7feeec42b640 1 -- 172.21.15.84:0/3289863156 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee40c7980 con 0x7feee4159e90 2023-10-16T11:22:28.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.594+0000 7feeec42b640 1 -- 172.21.15.84:0/3289863156 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee40c8160 con 0x7feee4159e90 2023-10-16T11:22:28.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.595+0000 7feedaffd640 1 -- 172.21.15.84:0/3289863156 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 121) v1 ==== 89275+0+0 (secure 0 0 0) 0x7feedc040cc0 con 0x7feee4159e90 2023-10-16T11:22:28.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.596+0000 7feedaffd640 1 --2- 172.21.15.84:0/3289863156 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7feec806af00 0x7feec806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.596+0000 7feedaffd640 1 -- 172.21.15.84:0/3289863156 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6797+0+0 (secure 0 0 0) 0x7feedc0acb10 con 0x7feee4159e90 2023-10-16T11:22:28.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.596+0000 7feeea1c7640 1 --2- 172.21.15.84:0/3289863156 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7feec806af00 0x7feec806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.597+0000 7feeea1c7640 1 --2- 172.21.15.84:0/3289863156 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7feec806af00 0x7feec806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feee0000f60 tx=0x7feee0009040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.600+0000 7feedaffd640 1 -- 172.21.15.84:0/3289863156 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7feedc077690 con 0x7feee4159e90 2023-10-16T11:22:28.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.833+0000 7feeec42b640 1 -- 172.21.15.84:0/3289863156 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7feee40c8110 con 0x7feee4159e90 2023-10-16T11:22:28.835 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-16T11:22:28.834+0000 7fca0c7ff640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:22:28.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.837+0000 7feedaffd640 1 -- 172.21.15.84:0/3289863156 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v97) v1 ==== 161+0+0 (secure 0 0 0) 0x7feedc07acb0 con 0x7feee4159e90 2023-10-16T11:22:28.839 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:22:28.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.841+0000 7feeec42b640 1 -- 172.21.15.84:0/3289863156 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7feec806af00 msgr2=0x7feec806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.841+0000 7feeec42b640 1 --2- 172.21.15.84:0/3289863156 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7feec806af00 0x7feec806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feee0000f60 tx=0x7feee0009040 comp rx=0 tx=0).stop 2023-10-16T11:22:28.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.841+0000 7feeec42b640 1 -- 172.21.15.84:0/3289863156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee4159e90 msgr2=0x7feee4157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.841+0000 7feeec42b640 1 --2- 172.21.15.84:0/3289863156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee4159e90 0x7feee4157c40 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7feedc037040 tx=0x7feedc004a40 comp rx=0 tx=0).stop 2023-10-16T11:22:28.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.841+0000 7feeec42b640 1 -- 172.21.15.84:0/3289863156 shutdown_connections 2023-10-16T11:22:28.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.841+0000 7feeec42b640 1 --2- 172.21.15.84:0/3289863156 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7feec806af00 0x7feec806d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.841+0000 7feeec42b640 1 --2- 172.21.15.84:0/3289863156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee4159e90 0x7feee4157c40 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.841+0000 7feeec42b640 1 --2- 172.21.15.84:0/3289863156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feee4159ab0 0x7feee40c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.841+0000 7feeec42b640 1 --2- 172.21.15.84:0/3289863156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feee406c850 0x7feee40c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.841+0000 7feeec42b640 1 -- 172.21.15.84:0/3289863156 >> 172.21.15.84:0/3289863156 conn(0x7feee40ce9b0 msgr2=0x7feee406ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:28.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.841+0000 7feeec42b640 1 -- 172.21.15.84:0/3289863156 shutdown_connections 2023-10-16T11:22:28.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:28.842+0000 7feeec42b640 1 -- 172.21.15.84:0/3289863156 wait complete. 2023-10-16T11:22:28.861 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T11:22:28.956 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T11:22:29.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.060+0000 7fc5f82e7640 1 Processor -- start 2023-10-16T11:22:29.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.060+0000 7fc5f82e7640 1 -- start start 2023-10-16T11:22:29.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.062+0000 7fc5f82e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5f00addb0 0x7fc5f00caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.062+0000 7fc5f82e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc5f00c9000 0x7fc5f0069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.062+0000 7fc5f82e7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f00cb140 con 0x7fc5f006a2e0 2023-10-16T11:22:29.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.062+0000 7fc5f82e7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f0161a50 con 0x7fc5f00c9000 2023-10-16T11:22:29.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.062+0000 7fc5f82e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f006a750 con 0x7fc5f00addb0 2023-10-16T11:22:29.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.062+0000 7fc5f6884640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc5f006a2e0 0x7fc5f00cc580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60598/0 (socket says 172.21.15.84:60598) 2023-10-16T11:22:29.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.062+0000 7fc5f6884640 1 -- 172.21.15.84:0/239290575 learned_addr learned my addr 172.21.15.84:0/239290575 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:29.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.062+0000 7fc5f6083640 1 --2- 172.21.15.84:0/239290575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5f00addb0 0x7fc5f00caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.062+0000 7fc5f5882640 1 --2- 172.21.15.84:0/239290575 >> v2:172.21.15.157:3300/0 conn(0x7fc5f00c9000 0x7fc5f0069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.063+0000 7fc5f5081640 1 -- 172.21.15.84:0/239290575 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1397527273 0 0) 0x7fc5f00cb140 con 0x7fc5f006a2e0 2023-10-16T11:22:29.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.063+0000 7fc5f5081640 1 -- 172.21.15.84:0/239290575 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5dc003520 con 0x7fc5f006a2e0 2023-10-16T11:22:29.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.063+0000 7fc5f6083640 1 -- 172.21.15.84:0/239290575 >> v2:172.21.15.157:3300/0 conn(0x7fc5f00c9000 msgr2=0x7fc5f0069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:29.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.063+0000 7fc5f6083640 1 --2- 172.21.15.84:0/239290575 >> v2:172.21.15.157:3300/0 conn(0x7fc5f00c9000 0x7fc5f0069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.063+0000 7fc5f6083640 1 -- 172.21.15.84:0/239290575 >> v1:172.21.15.84:6789/0 conn(0x7fc5f006a2e0 legacy=0x7fc5f00cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.063+0000 7fc5f6083640 1 -- 172.21.15.84:0/239290575 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f006b030 con 0x7fc5f00addb0 2023-10-16T11:22:29.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.065+0000 7fc5f6083640 1 --2- 172.21.15.84:0/239290575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5f00addb0 0x7fc5f00caa80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0097a0 tx=0x7fc5ec02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=769e6b885ad3b0e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:29.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.065+0000 7fc5f5081640 1 -- 172.21.15.84:0/239290575 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec02d040 con 0x7fc5f00addb0 2023-10-16T11:22:29.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.065+0000 7fc5f5081640 1 -- 172.21.15.84:0/239290575 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec031810 con 0x7fc5f00addb0 2023-10-16T11:22:29.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.065+0000 7fc5f5081640 1 -- 172.21.15.84:0/239290575 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec031ba0 con 0x7fc5f00addb0 2023-10-16T11:22:29.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.066+0000 7fc5f82e7640 1 -- 172.21.15.84:0/239290575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5f00addb0 msgr2=0x7fc5f00caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.066+0000 7fc5f82e7640 1 --2- 172.21.15.84:0/239290575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5f00addb0 0x7fc5f00caa80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0097a0 tx=0x7fc5ec02b8b0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.066+0000 7fc5f82e7640 1 -- 172.21.15.84:0/239290575 shutdown_connections 2023-10-16T11:22:29.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.066+0000 7fc5f82e7640 1 --2- 172.21.15.84:0/239290575 >> v2:172.21.15.157:3300/0 conn(0x7fc5f00c9000 0x7fc5f0069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.066+0000 7fc5f82e7640 1 --2- 172.21.15.84:0/239290575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5f00addb0 0x7fc5f00caa80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.066+0000 7fc5f82e7640 1 -- 172.21.15.84:0/239290575 >> 172.21.15.84:0/239290575 conn(0x7fc5f00ce9b0 msgr2=0x7fc5f00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:29.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.066+0000 7fc5f82e7640 1 -- 172.21.15.84:0/239290575 shutdown_connections 2023-10-16T11:22:29.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.066+0000 7fc5f82e7640 1 -- 172.21.15.84:0/239290575 wait complete. 2023-10-16T11:22:29.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.068+0000 7fc5f82e7640 1 Processor -- start 2023-10-16T11:22:29.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.069+0000 7fc5f82e7640 1 -- start start 2023-10-16T11:22:29.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.070+0000 7fc5f82e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5f006a2e0 0x7fc5f0164160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.070+0000 7fc5f82e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5f00addb0 0x7fc5f00c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.070+0000 7fc5f82e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5f00c9000 0x7fc5f00c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.070+0000 7fc5f82e7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f01630c0 con 0x7fc5f00addb0 2023-10-16T11:22:29.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.070+0000 7fc5f82e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f01633a0 con 0x7fc5f00c9000 2023-10-16T11:22:29.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.070+0000 7fc5f82e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f0163680 con 0x7fc5f006a2e0 2023-10-16T11:22:29.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.070+0000 7fc5f5882640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5f00addb0 0x7fc5f00c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.070+0000 7fc5f5882640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5f00addb0 0x7fc5f00c4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49624/0 (socket says 172.21.15.84:49624) 2023-10-16T11:22:29.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.070+0000 7fc5f5882640 1 -- 172.21.15.84:0/2717650096 learned_addr learned my addr 172.21.15.84:0/2717650096 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:29.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.071+0000 7fc5f6083640 1 --2- 172.21.15.84:0/2717650096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5f006a2e0 0x7fc5f0164160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.071+0000 7fc5f5882640 1 -- 172.21.15.84:0/2717650096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5f006a2e0 msgr2=0x7fc5f0164160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.071+0000 7fc5f6884640 1 --2- 172.21.15.84:0/2717650096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5f00c9000 0x7fc5f00c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.071+0000 7fc5f5882640 1 --2- 172.21.15.84:0/2717650096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5f006a2e0 0x7fc5f0164160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.071+0000 7fc5f5882640 1 -- 172.21.15.84:0/2717650096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5f00c9000 msgr2=0x7fc5f00c4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.071+0000 7fc5f5882640 1 --2- 172.21.15.84:0/2717650096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5f00c9000 0x7fc5f00c4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.071+0000 7fc5f5882640 1 -- 172.21.15.84:0/2717650096 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec009400 con 0x7fc5f00addb0 2023-10-16T11:22:29.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.071+0000 7fc5f6083640 1 --2- 172.21.15.84:0/2717650096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5f006a2e0 0x7fc5f0164160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:29.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.071+0000 7fc5f5882640 1 --2- 172.21.15.84:0/2717650096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5f00addb0 0x7fc5f00c4960 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000e870 tx=0x7fc5e000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:29.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.071+0000 7fc5e6ffd640 1 -- 172.21.15.84:0/2717650096 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e00094f0 con 0x7fc5f00addb0 2023-10-16T11:22:29.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.071+0000 7fc5e6ffd640 1 -- 172.21.15.84:0/2717650096 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e000c940 con 0x7fc5f00addb0 2023-10-16T11:22:29.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.071+0000 7fc5e6ffd640 1 -- 172.21.15.84:0/2717650096 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e000ccf0 con 0x7fc5f00addb0 2023-10-16T11:22:29.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.071+0000 7fc5f82e7640 1 -- 172.21.15.84:0/2717650096 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f0163960 con 0x7fc5f00addb0 2023-10-16T11:22:29.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.072+0000 7fc5f82e7640 1 -- 172.21.15.84:0/2717650096 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f0162be0 con 0x7fc5f00addb0 2023-10-16T11:22:29.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.073+0000 7fc5f82e7640 1 -- 172.21.15.84:0/2717650096 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f004e160 con 0x7fc5f00addb0 2023-10-16T11:22:29.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.074+0000 7fc5e6ffd640 1 -- 172.21.15.84:0/2717650096 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 121) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fc5e0013470 con 0x7fc5f00addb0 2023-10-16T11:22:29.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.075+0000 7fc5e6ffd640 1 --2- 172.21.15.84:0/2717650096 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc5d406af00 0x7fc5d406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.075+0000 7fc5e6ffd640 1 -- 172.21.15.84:0/2717650096 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6883+0+0 (secure 0 0 0) 0x7fc5e0089c80 con 0x7fc5f00addb0 2023-10-16T11:22:29.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.075+0000 7fc5f6083640 1 --2- 172.21.15.84:0/2717650096 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc5d406af00 0x7fc5d406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.076+0000 7fc5f6083640 1 --2- 172.21.15.84:0/2717650096 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc5d406af00 0x7fc5d406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0097a0 tx=0x7fc5ec03c040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:29.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.079+0000 7fc5e6ffd640 1 -- 172.21.15.84:0/2717650096 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc5e00547a0 con 0x7fc5f00addb0 2023-10-16T11:22:29.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.311+0000 7fc5f82e7640 1 -- 172.21.15.84:0/2717650096 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc5f0162e00 con 0x7fc5f00addb0 2023-10-16T11:22:29.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.841+0000 7fc5e6ffd640 1 -- 172.21.15.84:0/2717650096 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v98) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc5e00170a0 con 0x7fc5f00addb0 2023-10-16T11:22:29.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.844+0000 7fc5f82e7640 1 -- 172.21.15.84:0/2717650096 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc5d406af00 msgr2=0x7fc5d406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.844+0000 7fc5f82e7640 1 --2- 172.21.15.84:0/2717650096 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc5d406af00 0x7fc5d406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0097a0 tx=0x7fc5ec03c040 comp rx=0 tx=0).stop 2023-10-16T11:22:29.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.844+0000 7fc5f82e7640 1 -- 172.21.15.84:0/2717650096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5f00addb0 msgr2=0x7fc5f00c4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.844+0000 7fc5f82e7640 1 --2- 172.21.15.84:0/2717650096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5f00addb0 0x7fc5f00c4960 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000e870 tx=0x7fc5e000ed30 comp rx=0 tx=0).stop 2023-10-16T11:22:29.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.844+0000 7fc5f82e7640 1 -- 172.21.15.84:0/2717650096 shutdown_connections 2023-10-16T11:22:29.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.844+0000 7fc5f82e7640 1 --2- 172.21.15.84:0/2717650096 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc5d406af00 0x7fc5d406d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.844+0000 7fc5f82e7640 1 --2- 172.21.15.84:0/2717650096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5f00c9000 0x7fc5f00c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.844+0000 7fc5f82e7640 1 --2- 172.21.15.84:0/2717650096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5f00addb0 0x7fc5f00c4960 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.844+0000 7fc5f82e7640 1 --2- 172.21.15.84:0/2717650096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5f006a2e0 0x7fc5f0164160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.844+0000 7fc5f82e7640 1 -- 172.21.15.84:0/2717650096 >> 172.21.15.84:0/2717650096 conn(0x7fc5f00ce9b0 msgr2=0x7fc5f00b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:29.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.845+0000 7fc5f82e7640 1 -- 172.21.15.84:0/2717650096 shutdown_connections 2023-10-16T11:22:29.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:29.845+0000 7fc5f82e7640 1 -- 172.21.15.84:0/2717650096 wait complete. 2023-10-16T11:22:29.867 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T11:22:30.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.079+0000 7faf7b3e7640 1 Processor -- start 2023-10-16T11:22:30.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.079+0000 7faf7b3e7640 1 -- start start 2023-10-16T11:22:30.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.080+0000 7faf7b3e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faf74159da0 0x7faf740c9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.080+0000 7faf7b3e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf740cce20 0x7faf7406c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.080+0000 7faf7b3e7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf7415dfb0 con 0x7faf741599c0 2023-10-16T11:22:30.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.080+0000 7faf7b3e7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf7415e290 con 0x7faf74159da0 2023-10-16T11:22:30.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.081+0000 7faf7b3e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7415e570 con 0x7faf740cce20 2023-10-16T11:22:30.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.081+0000 7faf78982640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faf74159da0 0x7faf740c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.081+0000 7faf79984640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf740cce20 0x7faf7406c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.081+0000 7faf79984640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf740cce20 0x7faf7406c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:54088/0 (socket says 172.21.15.84:54088) 2023-10-16T11:22:30.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.081+0000 7faf79984640 1 -- 172.21.15.84:0/2164348824 learned_addr learned my addr 172.21.15.84:0/2164348824 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:30.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.082+0000 7faf79984640 1 -- 172.21.15.84:0/2164348824 >> v2:172.21.15.157:3300/0 conn(0x7faf74159da0 msgr2=0x7faf740c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:30.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.082+0000 7faf79984640 1 --2- 172.21.15.84:0/2164348824 >> v2:172.21.15.157:3300/0 conn(0x7faf74159da0 0x7faf740c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.082+0000 7faf79984640 1 -- 172.21.15.84:0/2164348824 >> v1:172.21.15.157:6789/0 conn(0x7faf741599c0 legacy=0x7faf7415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.082+0000 7faf79984640 1 -- 172.21.15.84:0/2164348824 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7406fda0 con 0x7faf740cce20 2023-10-16T11:22:30.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.084+0000 7faf79984640 1 --2- 172.21.15.84:0/2164348824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf740cce20 0x7faf7406c0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009e50 tx=0x7faf6c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc5c64b53a40cb5e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:30.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.085+0000 7faf6bfff640 1 -- 172.21.15.84:0/2164348824 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c02cb00 con 0x7faf740cce20 2023-10-16T11:22:30.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.085+0000 7faf6bfff640 1 -- 172.21.15.84:0/2164348824 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6c02cc60 con 0x7faf740cce20 2023-10-16T11:22:30.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.085+0000 7faf6bfff640 1 -- 172.21.15.84:0/2164348824 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c0343f0 con 0x7faf740cce20 2023-10-16T11:22:30.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.086+0000 7faf7b3e7640 1 -- 172.21.15.84:0/2164348824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf740cce20 msgr2=0x7faf7406c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.086+0000 7faf7b3e7640 1 --2- 172.21.15.84:0/2164348824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf740cce20 0x7faf7406c0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009e50 tx=0x7faf6c02bf60 comp rx=0 tx=0).stop 2023-10-16T11:22:30.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.086+0000 7faf7b3e7640 1 -- 172.21.15.84:0/2164348824 shutdown_connections 2023-10-16T11:22:30.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.086+0000 7faf7b3e7640 1 --2- 172.21.15.84:0/2164348824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf740cce20 0x7faf7406c0d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.086+0000 7faf7b3e7640 1 --2- 172.21.15.84:0/2164348824 >> v2:172.21.15.157:3300/0 conn(0x7faf74159da0 0x7faf740c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.086+0000 7faf7b3e7640 1 -- 172.21.15.84:0/2164348824 >> 172.21.15.84:0/2164348824 conn(0x7faf740cb360 msgr2=0x7faf740cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:30.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.086+0000 7faf7b3e7640 1 -- 172.21.15.84:0/2164348824 shutdown_connections 2023-10-16T11:22:30.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.086+0000 7faf7b3e7640 1 -- 172.21.15.84:0/2164348824 wait complete. 2023-10-16T11:22:30.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.089+0000 7faf7b3e7640 1 Processor -- start 2023-10-16T11:22:30.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.089+0000 7faf7b3e7640 1 -- start start 2023-10-16T11:22:30.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.090+0000 7faf7b3e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf740cce20 0x7faf740ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.090+0000 7faf7b3e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf741599c0 0x7faf740c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.090+0000 7faf7b3e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf74159da0 0x7faf74149800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.090+0000 7faf7b3e7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf740aeb20 con 0x7faf741599c0 2023-10-16T11:22:30.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.090+0000 7faf7b3e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf740aedc0 con 0x7faf740cce20 2023-10-16T11:22:30.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.090+0000 7faf7b3e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf740af0a0 con 0x7faf74159da0 2023-10-16T11:22:30.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.090+0000 7faf78982640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf741599c0 0x7faf740c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.090+0000 7faf78982640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf741599c0 0x7faf740c3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49636/0 (socket says 172.21.15.84:49636) 2023-10-16T11:22:30.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.090+0000 7faf78982640 1 -- 172.21.15.84:0/4272956921 learned_addr learned my addr 172.21.15.84:0/4272956921 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:30.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.090+0000 7faf78982640 1 -- 172.21.15.84:0/4272956921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf74159da0 msgr2=0x7faf74149800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.090+0000 7faf78982640 1 --2- 172.21.15.84:0/4272956921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf74159da0 0x7faf74149800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.090+0000 7faf78982640 1 -- 172.21.15.84:0/4272956921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf740cce20 msgr2=0x7faf740ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.090+0000 7faf78982640 1 --2- 172.21.15.84:0/4272956921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf740cce20 0x7faf740ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.090+0000 7faf78982640 1 -- 172.21.15.84:0/4272956921 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c009b30 con 0x7faf741599c0 2023-10-16T11:22:30.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.091+0000 7faf78982640 1 --2- 172.21.15.84:0/4272956921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf741599c0 0x7faf740c3c90 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7faf6400e9b0 tx=0x7faf6400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:30.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.091+0000 7faf69ffb640 1 -- 172.21.15.84:0/4272956921 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64009670 con 0x7faf741599c0 2023-10-16T11:22:30.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.091+0000 7faf69ffb640 1 -- 172.21.15.84:0/4272956921 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6400c9a0 con 0x7faf741599c0 2023-10-16T11:22:30.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.091+0000 7faf69ffb640 1 -- 172.21.15.84:0/4272956921 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6400cd50 con 0x7faf741599c0 2023-10-16T11:22:30.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.091+0000 7faf7b3e7640 1 -- 172.21.15.84:0/4272956921 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7415a3c0 con 0x7faf741599c0 2023-10-16T11:22:30.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.091+0000 7faf7b3e7640 1 -- 172.21.15.84:0/4272956921 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf740af380 con 0x7faf741599c0 2023-10-16T11:22:30.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.092+0000 7faf7b3e7640 1 -- 172.21.15.84:0/4272956921 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf7404e160 con 0x7faf741599c0 2023-10-16T11:22:30.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.094+0000 7faf69ffb640 1 -- 172.21.15.84:0/4272956921 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7faf64013470 con 0x7faf741599c0 2023-10-16T11:22:30.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.095+0000 7faf69ffb640 1 --2- 172.21.15.84:0/4272956921 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7faf5806af90 0x7faf5806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.095+0000 7faf69ffb640 1 -- 172.21.15.84:0/4272956921 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6883+0+0 (secure 0 0 0) 0x7faf64089df0 con 0x7faf741599c0 2023-10-16T11:22:30.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.095+0000 7faf79183640 1 --2- 172.21.15.84:0/4272956921 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7faf5806af90 0x7faf5806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.096+0000 7faf79183640 1 --2- 172.21.15.84:0/4272956921 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7faf5806af90 0x7faf5806d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faf7406cbb0 tx=0x7faf70007970 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:30.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.100+0000 7faf69ffb640 1 -- 172.21.15.84:0/4272956921 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7faf64054980 con 0x7faf741599c0 2023-10-16T11:22:30.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:30.293+0000 7faf7b3e7640 1 -- 172.21.15.84:0/4272956921 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faf7415d2c0 con 0x7faf741599c0 2023-10-16T11:22:31.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.001+0000 7faf69ffb640 1 -- 172.21.15.84:0/4272956921 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v192) v1 ==== 181+0+0 (secure 0 0 0) 0x7faf640170d0 con 0x7faf741599c0 2023-10-16T11:22:31.002 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-16T11:22:31.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.009+0000 7faf7b3e7640 1 -- 172.21.15.84:0/4272956921 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7faf5806af90 msgr2=0x7faf5806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.009+0000 7faf7b3e7640 1 --2- 172.21.15.84:0/4272956921 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7faf5806af90 0x7faf5806d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faf7406cbb0 tx=0x7faf70007970 comp rx=0 tx=0).stop 2023-10-16T11:22:31.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.009+0000 7faf7b3e7640 1 -- 172.21.15.84:0/4272956921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf741599c0 msgr2=0x7faf740c3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.009+0000 7faf7b3e7640 1 --2- 172.21.15.84:0/4272956921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf741599c0 0x7faf740c3c90 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7faf6400e9b0 tx=0x7faf6400ee70 comp rx=0 tx=0).stop 2023-10-16T11:22:31.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.011+0000 7faf7b3e7640 1 -- 172.21.15.84:0/4272956921 shutdown_connections 2023-10-16T11:22:31.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.011+0000 7faf7b3e7640 1 --2- 172.21.15.84:0/4272956921 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7faf5806af90 0x7faf5806d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.011+0000 7faf7b3e7640 1 --2- 172.21.15.84:0/4272956921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf74159da0 0x7faf74149800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.011+0000 7faf7b3e7640 1 --2- 172.21.15.84:0/4272956921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf741599c0 0x7faf740c3c90 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.011+0000 7faf7b3e7640 1 --2- 172.21.15.84:0/4272956921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf740cce20 0x7faf740ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.011+0000 7faf7b3e7640 1 -- 172.21.15.84:0/4272956921 >> 172.21.15.84:0/4272956921 conn(0x7faf740cb360 msgr2=0x7faf7414fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:31.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.011+0000 7faf7b3e7640 1 -- 172.21.15.84:0/4272956921 shutdown_connections 2023-10-16T11:22:31.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.011+0000 7faf7b3e7640 1 -- 172.21.15.84:0/4272956921 wait complete. 2023-10-16T11:22:31.037 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T11:22:31.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.236+0000 7fb86e876640 1 Processor -- start 2023-10-16T11:22:31.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.236+0000 7fb86e876640 1 -- start start 2023-10-16T11:22:31.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.238+0000 7fb86e876640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb86815cf50 0x7fb8680d2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.238+0000 7fb86e876640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8680b4750 0x7fb8680b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.238+0000 7fb86e876640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8680d1420 con 0x7fb8680b8a10 2023-10-16T11:22:31.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.238+0000 7fb86e876640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8680b5100 con 0x7fb86815cf50 2023-10-16T11:22:31.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.238+0000 7fb86e876640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86815d330 con 0x7fb8680b4750 2023-10-16T11:22:31.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.239+0000 7fb8677fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb8680b8a10 0x7fb8680d0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60608/0 (socket says 172.21.15.84:60608) 2023-10-16T11:22:31.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.239+0000 7fb8677fe640 1 -- 172.21.15.84:0/3078012780 learned_addr learned my addr 172.21.15.84:0/3078012780 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:31.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.239+0000 7fb867fff640 1 --2- 172.21.15.84:0/3078012780 >> v2:172.21.15.157:3300/0 conn(0x7fb86815cf50 0x7fb8680d2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.239+0000 7fb866ffd640 1 -- 172.21.15.84:0/3078012780 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 298921760 0 0) 0x7fb8680d1420 con 0x7fb8680b8a10 2023-10-16T11:22:31.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.239+0000 7fb86ce13640 1 --2- 172.21.15.84:0/3078012780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8680b4750 0x7fb8680b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.239+0000 7fb866ffd640 1 -- 172.21.15.84:0/3078012780 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb850003520 con 0x7fb8680b8a10 2023-10-16T11:22:31.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.239+0000 7fb866ffd640 1 -- 172.21.15.84:0/3078012780 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 843830704 0 0) 0x7fb850003520 con 0x7fb8680b8a10 2023-10-16T11:22:31.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.239+0000 7fb866ffd640 1 -- 172.21.15.84:0/3078012780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8680b4750 msgr2=0x7fb8680b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:31.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.239+0000 7fb866ffd640 1 --2- 172.21.15.84:0/3078012780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8680b4750 0x7fb8680b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.239+0000 7fb866ffd640 1 -- 172.21.15.84:0/3078012780 >> v2:172.21.15.157:3300/0 conn(0x7fb86815cf50 msgr2=0x7fb8680d2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:31.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.239+0000 7fb866ffd640 1 --2- 172.21.15.84:0/3078012780 >> v2:172.21.15.157:3300/0 conn(0x7fb86815cf50 0x7fb8680d2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.239+0000 7fb866ffd640 1 -- 172.21.15.84:0/3078012780 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8680d1680 con 0x7fb8680b8a10 2023-10-16T11:22:31.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.239+0000 7fb866ffd640 1 -- 172.21.15.84:0/3078012780 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fb85c001690 con 0x7fb8680b8a10 2023-10-16T11:22:31.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.240+0000 7fb866ffd640 1 -- 172.21.15.84:0/3078012780 >> v1:172.21.15.84:6789/0 conn(0x7fb8680b8a10 legacy=0x7fb8680d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.240+0000 7fb866ffd640 1 --2- 172.21.15.84:0/3078012780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8500039e0 0x7fb850003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.240+0000 7fb866ffd640 1 --2- 172.21.15.84:0/3078012780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8680b4750 0x7fb8500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.240+0000 7fb866ffd640 1 --2- 172.21.15.84:0/3078012780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb850004d00 0x7fb850005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.240+0000 7fb866ffd640 1 -- 172.21.15.84:0/3078012780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8680b5100 con 0x7fb850004d00 2023-10-16T11:22:31.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.240+0000 7fb866ffd640 1 -- 172.21.15.84:0/3078012780 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86815d330 con 0x7fb8500039e0 2023-10-16T11:22:31.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.240+0000 7fb866ffd640 1 -- 172.21.15.84:0/3078012780 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85000ba00 con 0x7fb8680b4750 2023-10-16T11:22:31.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.240+0000 7fb86ce13640 1 --2- 172.21.15.84:0/3078012780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb850004d00 0x7fb850005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.240+0000 7fb8677fe640 1 --2- 172.21.15.84:0/3078012780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8680b4750 0x7fb8500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.240+0000 7fb86ce13640 1 -- 172.21.15.84:0/3078012780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8680b4750 msgr2=0x7fb8500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:31.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.240+0000 7fb867fff640 1 --2- 172.21.15.84:0/3078012780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8500039e0 0x7fb850003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.240+0000 7fb86ce13640 1 --2- 172.21.15.84:0/3078012780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8680b4750 0x7fb8500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.240+0000 7fb86ce13640 1 -- 172.21.15.84:0/3078012780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8500039e0 msgr2=0x7fb850003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:31.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.240+0000 7fb86ce13640 1 --2- 172.21.15.84:0/3078012780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8500039e0 0x7fb850003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.240+0000 7fb86ce13640 1 -- 172.21.15.84:0/3078012780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb850003520 con 0x7fb850004d00 2023-10-16T11:22:31.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.240+0000 7fb867fff640 1 --2- 172.21.15.84:0/3078012780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8500039e0 0x7fb850003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:31.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.240+0000 7fb8677fe640 1 --2- 172.21.15.84:0/3078012780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8680b4750 0x7fb8500047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:31.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.242+0000 7fb86ce13640 1 --2- 172.21.15.84:0/3078012780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb850004d00 0x7fb850005180 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fb860007360 tx=0x7fb8500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9020eb7482396a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:31.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.242+0000 7fb866ffd640 1 -- 172.21.15.84:0/3078012780 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86002ccb0 con 0x7fb850004d00 2023-10-16T11:22:31.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.242+0000 7fb866ffd640 1 -- 172.21.15.84:0/3078012780 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86002ce10 con 0x7fb850004d00 2023-10-16T11:22:31.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.242+0000 7fb866ffd640 1 -- 172.21.15.84:0/3078012780 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860034660 con 0x7fb850004d00 2023-10-16T11:22:31.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.243+0000 7fb86e876640 1 -- 172.21.15.84:0/3078012780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb850004d00 msgr2=0x7fb850005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.243+0000 7fb86e876640 1 --2- 172.21.15.84:0/3078012780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb850004d00 0x7fb850005180 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fb860007360 tx=0x7fb8500057f0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.243+0000 7fb86ce13640 1 -- 172.21.15.84:0/3078012780 reap_dead start 2023-10-16T11:22:31.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.243+0000 7fb86e876640 1 -- 172.21.15.84:0/3078012780 shutdown_connections 2023-10-16T11:22:31.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.243+0000 7fb86e876640 1 -- 172.21.15.84:0/3078012780 >> 172.21.15.84:0/3078012780 conn(0x7fb8680cf750 msgr2=0x7fb8680cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:31.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.243+0000 7fb86e876640 1 -- 172.21.15.84:0/3078012780 shutdown_connections 2023-10-16T11:22:31.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.243+0000 7fb86e876640 1 -- 172.21.15.84:0/3078012780 wait complete. 2023-10-16T11:22:31.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.245+0000 7fb86e876640 1 Processor -- start 2023-10-16T11:22:31.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.245+0000 7fb86e876640 1 -- start start 2023-10-16T11:22:31.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.246+0000 7fb86e876640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb850004d00 0x7fb8680b7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.246+0000 7fb86e876640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb868160120 0x7fb868152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.246+0000 7fb86e876640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb86815e1d0 0x7fb86806bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.246+0000 7fb86e876640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb868152950 con 0x7fb868160120 2023-10-16T11:22:31.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.246+0000 7fb86e876640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8681f30b0 con 0x7fb850004d00 2023-10-16T11:22:31.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.246+0000 7fb86e876640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8681f24a0 con 0x7fb86815e1d0 2023-10-16T11:22:31.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.246+0000 7fb8677fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb868160120 0x7fb868152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.246+0000 7fb8677fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb868160120 0x7fb868152410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49656/0 (socket says 172.21.15.84:49656) 2023-10-16T11:22:31.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.246+0000 7fb8677fe640 1 -- 172.21.15.84:0/183639843 learned_addr learned my addr 172.21.15.84:0/183639843 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:31.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.247+0000 7fb867fff640 1 --2- 172.21.15.84:0/183639843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb850004d00 0x7fb8680b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.247+0000 7fb8677fe640 1 -- 172.21.15.84:0/183639843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb86815e1d0 msgr2=0x7fb86806bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.247+0000 7fb8677fe640 1 --2- 172.21.15.84:0/183639843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb86815e1d0 0x7fb86806bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.247+0000 7fb8677fe640 1 -- 172.21.15.84:0/183639843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb850004d00 msgr2=0x7fb8680b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.247+0000 7fb8677fe640 1 --2- 172.21.15.84:0/183639843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb850004d00 0x7fb8680b7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.247+0000 7fb8677fe640 1 -- 172.21.15.84:0/183639843 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8681f2780 con 0x7fb868160120 2023-10-16T11:22:31.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.247+0000 7fb867fff640 1 --2- 172.21.15.84:0/183639843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb850004d00 0x7fb8680b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:31.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.247+0000 7fb8677fe640 1 --2- 172.21.15.84:0/183639843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb868160120 0x7fb868152410 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00d7c0 tx=0x7fb85c00dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:31.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.247+0000 7fb864ff9640 1 -- 172.21.15.84:0/183639843 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0074e0 con 0x7fb868160120 2023-10-16T11:22:31.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.248+0000 7fb864ff9640 1 -- 172.21.15.84:0/183639843 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85c007640 con 0x7fb868160120 2023-10-16T11:22:31.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.248+0000 7fb86e876640 1 -- 172.21.15.84:0/183639843 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8681f2a60 con 0x7fb868160120 2023-10-16T11:22:31.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.248+0000 7fb864ff9640 1 -- 172.21.15.84:0/183639843 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c004a20 con 0x7fb868160120 2023-10-16T11:22:31.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.248+0000 7fb86e876640 1 -- 172.21.15.84:0/183639843 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86815e650 con 0x7fb868160120 2023-10-16T11:22:31.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.249+0000 7fb86e876640 1 -- 172.21.15.84:0/183639843 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb868072350 con 0x7fb868160120 2023-10-16T11:22:31.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.250+0000 7fb864ff9640 1 -- 172.21.15.84:0/183639843 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb85c004b80 con 0x7fb868160120 2023-10-16T11:22:31.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.251+0000 7fb864ff9640 1 --2- 172.21.15.84:0/183639843 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fb85406af90 0x7fb85406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.251+0000 7fb864ff9640 1 -- 172.21.15.84:0/183639843 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fb85c0898e0 con 0x7fb868160120 2023-10-16T11:22:31.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.251+0000 7fb867fff640 1 --2- 172.21.15.84:0/183639843 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fb85406af90 0x7fb85406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.252+0000 7fb867fff640 1 --2- 172.21.15.84:0/183639843 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fb85406af90 0x7fb85406d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb8680b82d0 tx=0x7fb858008040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:31.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.255+0000 7fb864ff9640 1 -- 172.21.15.84:0/183639843 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb85c054400 con 0x7fb868160120 2023-10-16T11:22:31.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:31.449+0000 7fb86e876640 1 -- 172.21.15.84:0/183639843 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb86804e160 con 0x7fb868160120 2023-10-16T11:22:32.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.011+0000 7fb864ff9640 1 -- 172.21.15.84:0/183639843 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v193) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb85c057a20 con 0x7fb868160120 2023-10-16T11:22:32.013 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-16T11:22:32.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.015+0000 7fb86e876640 1 -- 172.21.15.84:0/183639843 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fb85406af90 msgr2=0x7fb85406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.015+0000 7fb86e876640 1 --2- 172.21.15.84:0/183639843 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fb85406af90 0x7fb85406d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb8680b82d0 tx=0x7fb858008040 comp rx=0 tx=0).stop 2023-10-16T11:22:32.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.015+0000 7fb86e876640 1 -- 172.21.15.84:0/183639843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb868160120 msgr2=0x7fb868152410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.015+0000 7fb86e876640 1 --2- 172.21.15.84:0/183639843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb868160120 0x7fb868152410 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00d7c0 tx=0x7fb85c00dc80 comp rx=0 tx=0).stop 2023-10-16T11:22:32.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.016+0000 7fb86e876640 1 -- 172.21.15.84:0/183639843 shutdown_connections 2023-10-16T11:22:32.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.016+0000 7fb86e876640 1 --2- 172.21.15.84:0/183639843 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fb85406af90 0x7fb85406d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.016+0000 7fb86e876640 1 --2- 172.21.15.84:0/183639843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb86815e1d0 0x7fb86806bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.016+0000 7fb86e876640 1 --2- 172.21.15.84:0/183639843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb868160120 0x7fb868152410 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.016+0000 7fb86e876640 1 --2- 172.21.15.84:0/183639843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb850004d00 0x7fb8680b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.016+0000 7fb86e876640 1 -- 172.21.15.84:0/183639843 >> 172.21.15.84:0/183639843 conn(0x7fb8680cf750 msgr2=0x7fb8680ce330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:32.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.017+0000 7fb86e876640 1 -- 172.21.15.84:0/183639843 shutdown_connections 2023-10-16T11:22:32.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.017+0000 7fb86e876640 1 -- 172.21.15.84:0/183639843 wait complete. 2023-10-16T11:22:32.041 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T11:22:32.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.250+0000 7fa9b4e51640 1 Processor -- start 2023-10-16T11:22:32.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.250+0000 7fa9b4e51640 1 -- start start 2023-10-16T11:22:32.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.251+0000 7fa9b4e51640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9b015ad70 0x7fa9b0069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:32.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.251+0000 7fa9b4e51640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa9b00caac0 0x7fa9b006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:32.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.251+0000 7fa9b4e51640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b00cb030 con 0x7fa9b00c9ca0 2023-10-16T11:22:32.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.252+0000 7fa9b4e51640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b00c91c0 con 0x7fa9b00caac0 2023-10-16T11:22:32.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.252+0000 7fa9b4e51640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b00c94a0 con 0x7fa9b015ad70 2023-10-16T11:22:32.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.252+0000 7fa9aed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa9b00caac0 0x7fa9b006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:32.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.252+0000 7fa9aed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa9b00caac0 0x7fa9b006a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49662/0 (socket says 172.21.15.84:49662) 2023-10-16T11:22:32.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.252+0000 7fa9aed9d640 1 -- 172.21.15.84:0/498824342 learned_addr learned my addr 172.21.15.84:0/498824342 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:32.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.252+0000 7fa9ae59c640 1 --2- 172.21.15.84:0/498824342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9b015ad70 0x7fa9b0069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:32.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.252+0000 7fa9aed9d640 1 -- 172.21.15.84:0/498824342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9b015ad70 msgr2=0x7fa9b0069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:32.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.252+0000 7fa9aed9d640 1 --2- 172.21.15.84:0/498824342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9b015ad70 0x7fa9b0069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.252+0000 7fa9aed9d640 1 -- 172.21.15.84:0/498824342 >> v1:172.21.15.157:6789/0 conn(0x7fa9b00c9ca0 legacy=0x7fa9b00ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:32.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.252+0000 7fa9aed9d640 1 -- 172.21.15.84:0/498824342 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b00c9780 con 0x7fa9b00caac0 2023-10-16T11:22:32.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.253+0000 7fa9aed9d640 1 --2- 172.21.15.84:0/498824342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9b00caac0 0x7fa9b006a2e0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009e50 tx=0x7fa9a802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1718478419c657a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:32.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.254+0000 7fa9add9b640 1 -- 172.21.15.84:0/498824342 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a802ca70 con 0x7fa9b00caac0 2023-10-16T11:22:32.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.254+0000 7fa9add9b640 1 -- 172.21.15.84:0/498824342 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a802cbd0 con 0x7fa9b00caac0 2023-10-16T11:22:32.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.254+0000 7fa9add9b640 1 -- 172.21.15.84:0/498824342 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a80343f0 con 0x7fa9b00caac0 2023-10-16T11:22:32.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.255+0000 7fa9b4e51640 1 -- 172.21.15.84:0/498824342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9b00caac0 msgr2=0x7fa9b006a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.255+0000 7fa9b4e51640 1 --2- 172.21.15.84:0/498824342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9b00caac0 0x7fa9b006a2e0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009e50 tx=0x7fa9a802bed0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.255+0000 7fa9b4e51640 1 -- 172.21.15.84:0/498824342 shutdown_connections 2023-10-16T11:22:32.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.255+0000 7fa9b4e51640 1 --2- 172.21.15.84:0/498824342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9b00caac0 0x7fa9b006a2e0 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.255+0000 7fa9b4e51640 1 --2- 172.21.15.84:0/498824342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9b015ad70 0x7fa9b0069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.255+0000 7fa9b4e51640 1 -- 172.21.15.84:0/498824342 >> 172.21.15.84:0/498824342 conn(0x7fa9b00ce9b0 msgr2=0x7fa9b00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:32.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.255+0000 7fa9b4e51640 1 -- 172.21.15.84:0/498824342 shutdown_connections 2023-10-16T11:22:32.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.255+0000 7fa9b4e51640 1 -- 172.21.15.84:0/498824342 wait complete. 2023-10-16T11:22:32.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.256+0000 7fa9b4e51640 1 Processor -- start 2023-10-16T11:22:32.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.257+0000 7fa9b4e51640 1 -- start start 2023-10-16T11:22:32.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.258+0000 7fa9b4e51640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b00c9ca0 0x7fa9b00c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:32.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.258+0000 7fa9b4e51640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9b00caac0 0x7fa9b0155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:32.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.258+0000 7fa9b4e51640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9b015ad70 0x7fa9b015dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:32.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.258+0000 7fa9b4e51640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b015bc10 con 0x7fa9b015ad70 2023-10-16T11:22:32.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.258+0000 7fa9b4e51640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b015c670 con 0x7fa9b00c9ca0 2023-10-16T11:22:32.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.258+0000 7fa9b4e51640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b015b930 con 0x7fa9b00caac0 2023-10-16T11:22:32.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.258+0000 7fa9aed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9b015ad70 0x7fa9b015dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:32.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.258+0000 7fa9aed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9b015ad70 0x7fa9b015dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49676/0 (socket says 172.21.15.84:49676) 2023-10-16T11:22:32.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.258+0000 7fa9aed9d640 1 -- 172.21.15.84:0/4080283630 learned_addr learned my addr 172.21.15.84:0/4080283630 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:32.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.258+0000 7fa9ae59c640 1 --2- 172.21.15.84:0/4080283630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b00c9ca0 0x7fa9b00c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:32.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.258+0000 7fa9aed9d640 1 -- 172.21.15.84:0/4080283630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9b00caac0 msgr2=0x7fa9b0155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.258+0000 7fa9a7fff640 1 --2- 172.21.15.84:0/4080283630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9b00caac0 0x7fa9b0155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:32.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.258+0000 7fa9aed9d640 1 --2- 172.21.15.84:0/4080283630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9b00caac0 0x7fa9b0155040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.258+0000 7fa9aed9d640 1 -- 172.21.15.84:0/4080283630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b00c9ca0 msgr2=0x7fa9b00c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.258+0000 7fa9aed9d640 1 --2- 172.21.15.84:0/4080283630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b00c9ca0 0x7fa9b00c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.258+0000 7fa9aed9d640 1 -- 172.21.15.84:0/4080283630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a8009b30 con 0x7fa9b015ad70 2023-10-16T11:22:32.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.258+0000 7fa9ae59c640 1 --2- 172.21.15.84:0/4080283630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b00c9ca0 0x7fa9b00c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:32.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.259+0000 7fa9aed9d640 1 --2- 172.21.15.84:0/4080283630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9b015ad70 0x7fa9b015dc40 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009e50 tx=0x7fa9a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:32.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.259+0000 7fa9a6ffd640 1 -- 172.21.15.84:0/4080283630 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8009510 con 0x7fa9b015ad70 2023-10-16T11:22:32.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.259+0000 7fa9a6ffd640 1 -- 172.21.15.84:0/4080283630 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a803e450 con 0x7fa9b015ad70 2023-10-16T11:22:32.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.259+0000 7fa9a6ffd640 1 -- 172.21.15.84:0/4080283630 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a803e800 con 0x7fa9b015ad70 2023-10-16T11:22:32.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.259+0000 7fa9b4e51640 1 -- 172.21.15.84:0/4080283630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b015bfb0 con 0x7fa9b015ad70 2023-10-16T11:22:32.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.259+0000 7fa9b4e51640 1 -- 172.21.15.84:0/4080283630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b0155580 con 0x7fa9b015ad70 2023-10-16T11:22:32.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.261+0000 7fa9b4e51640 1 -- 172.21.15.84:0/4080283630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b0155710 con 0x7fa9b015ad70 2023-10-16T11:22:32.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.261+0000 7fa9a6ffd640 1 -- 172.21.15.84:0/4080283630 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa9a803e960 con 0x7fa9b015ad70 2023-10-16T11:22:32.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.262+0000 7fa9a6ffd640 1 --2- 172.21.15.84:0/4080283630 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fa99006af90 0x7fa99006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:32.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.262+0000 7fa9a6ffd640 1 -- 172.21.15.84:0/4080283630 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6003+0+0 (secure 0 0 0) 0x7fa9a80ac7c0 con 0x7fa9b015ad70 2023-10-16T11:22:32.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.263+0000 7fa9ae59c640 1 --2- 172.21.15.84:0/4080283630 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fa99006af90 0x7fa99006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:32.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.263+0000 7fa9ae59c640 1 --2- 172.21.15.84:0/4080283630 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fa99006af90 0x7fa99006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000b820 tx=0x7fa9a0008040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:32.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.267+0000 7fa9a6ffd640 1 -- 172.21.15.84:0/4080283630 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa9a8077650 con 0x7fa9b015ad70 2023-10-16T11:22:32.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.463+0000 7fa9b4e51640 1 -- 172.21.15.84:0/4080283630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa9b015bde0 con 0x7fa9b015ad70 2023-10-16T11:22:32.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.468+0000 7fa9a6ffd640 1 -- 172.21.15.84:0/4080283630 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa9a807ac70 con 0x7fa9b015ad70 2023-10-16T11:22:32.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.472+0000 7fa9b4e51640 1 -- 172.21.15.84:0/4080283630 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fa99006af90 msgr2=0x7fa99006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.472+0000 7fa9b4e51640 1 --2- 172.21.15.84:0/4080283630 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fa99006af90 0x7fa99006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000b820 tx=0x7fa9a0008040 comp rx=0 tx=0).stop 2023-10-16T11:22:32.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.472+0000 7fa9b4e51640 1 -- 172.21.15.84:0/4080283630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9b015ad70 msgr2=0x7fa9b015dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.472+0000 7fa9b4e51640 1 --2- 172.21.15.84:0/4080283630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9b015ad70 0x7fa9b015dc40 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009e50 tx=0x7fa9a8004a40 comp rx=0 tx=0).stop 2023-10-16T11:22:32.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.472+0000 7fa9b4e51640 1 -- 172.21.15.84:0/4080283630 shutdown_connections 2023-10-16T11:22:32.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.472+0000 7fa9b4e51640 1 --2- 172.21.15.84:0/4080283630 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fa99006af90 0x7fa99006d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.472+0000 7fa9b4e51640 1 --2- 172.21.15.84:0/4080283630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9b015ad70 0x7fa9b015dc40 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.472+0000 7fa9b4e51640 1 --2- 172.21.15.84:0/4080283630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9b00caac0 0x7fa9b0155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.472+0000 7fa9b4e51640 1 --2- 172.21.15.84:0/4080283630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b00c9ca0 0x7fa9b00c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.472+0000 7fa9b4e51640 1 -- 172.21.15.84:0/4080283630 >> 172.21.15.84:0/4080283630 conn(0x7fa9b00ce9b0 msgr2=0x7fa9b006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:32.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.473+0000 7fa9b4e51640 1 -- 172.21.15.84:0/4080283630 shutdown_connections 2023-10-16T11:22:32.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.474+0000 7fa9b4e51640 1 -- 172.21.15.84:0/4080283630 wait complete. 2023-10-16T11:22:32.501 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T11:22:32.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.684+0000 7f10df74e640 1 Processor -- start 2023-10-16T11:22:32.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.684+0000 7f10df74e640 1 -- start start 2023-10-16T11:22:32.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.685+0000 7f10df74e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f10d815ad70 0x7f10d8069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:32.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.685+0000 7f10df74e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d80caac0 0x7f10d806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:32.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.685+0000 7f10df74e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d80cb030 con 0x7f10d80c9ca0 2023-10-16T11:22:32.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.685+0000 7f10df74e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d80c91c0 con 0x7f10d815ad70 2023-10-16T11:22:32.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.685+0000 7f10df74e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d80c94a0 con 0x7f10d80caac0 2023-10-16T11:22:32.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.686+0000 7f10dd4ea640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f10d815ad70 0x7f10d8069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:32.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.686+0000 7f10ddceb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d80caac0 0x7f10d806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:32.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.686+0000 7f10dd4ea640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f10d815ad70 0x7f10d8069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:50834/0 (socket says 172.21.15.84:50834) 2023-10-16T11:22:32.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.686+0000 7f10dcce9640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f10d80c9ca0 0x7f10d80ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:44324/0 (socket says 172.21.15.84:44324) 2023-10-16T11:22:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.686+0000 7f10dd4ea640 1 -- 172.21.15.84:0/2235732351 learned_addr learned my addr 172.21.15.84:0/2235732351 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.687+0000 7f10ddceb640 1 -- 172.21.15.84:0/2235732351 >> v2:172.21.15.157:3300/0 conn(0x7f10d815ad70 msgr2=0x7f10d8069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.687+0000 7f10ddceb640 1 --2- 172.21.15.84:0/2235732351 >> v2:172.21.15.157:3300/0 conn(0x7f10d815ad70 0x7f10d8069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.687+0000 7f10ddceb640 1 -- 172.21.15.84:0/2235732351 >> v1:172.21.15.157:6789/0 conn(0x7f10d80c9ca0 legacy=0x7f10d80ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.687+0000 7f10ddceb640 1 -- 172.21.15.84:0/2235732351 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d80c9780 con 0x7f10d80caac0 2023-10-16T11:22:32.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.687+0000 7f10d7fff640 1 -- 172.21.15.84:0/2235732351 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3520036712 0 0) 0x7f10d80cb030 con 0x7f10d80c9ca0 2023-10-16T11:22:32.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.689+0000 7f10ddceb640 1 --2- 172.21.15.84:0/2235732351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d80caac0 0x7f10d806a2e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f10d00097a0 tx=0x7f10d002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f1b1a4d5c4cab0e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:32.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.689+0000 7f10d7fff640 1 -- 172.21.15.84:0/2235732351 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d002d040 con 0x7f10d80caac0 2023-10-16T11:22:32.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.689+0000 7f10d7fff640 1 -- 172.21.15.84:0/2235732351 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d0031810 con 0x7f10d80caac0 2023-10-16T11:22:32.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.689+0000 7f10d7fff640 1 -- 172.21.15.84:0/2235732351 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d0031ba0 con 0x7f10d80caac0 2023-10-16T11:22:32.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.690+0000 7f10df74e640 1 -- 172.21.15.84:0/2235732351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d80caac0 msgr2=0x7f10d806a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.690+0000 7f10df74e640 1 --2- 172.21.15.84:0/2235732351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d80caac0 0x7f10d806a2e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f10d00097a0 tx=0x7f10d002b8b0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.690+0000 7f10df74e640 1 -- 172.21.15.84:0/2235732351 shutdown_connections 2023-10-16T11:22:32.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.690+0000 7f10df74e640 1 --2- 172.21.15.84:0/2235732351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d80caac0 0x7f10d806a2e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.690+0000 7f10df74e640 1 --2- 172.21.15.84:0/2235732351 >> v2:172.21.15.157:3300/0 conn(0x7f10d815ad70 0x7f10d8069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.690+0000 7f10df74e640 1 -- 172.21.15.84:0/2235732351 >> 172.21.15.84:0/2235732351 conn(0x7f10d80ce9b0 msgr2=0x7f10d80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:32.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.690+0000 7f10df74e640 1 -- 172.21.15.84:0/2235732351 shutdown_connections 2023-10-16T11:22:32.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.690+0000 7f10df74e640 1 -- 172.21.15.84:0/2235732351 wait complete. 2023-10-16T11:22:32.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.692+0000 7f10df74e640 1 Processor -- start 2023-10-16T11:22:32.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.692+0000 7f10df74e640 1 -- start start 2023-10-16T11:22:32.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.693+0000 7f10df74e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10d80c9ca0 0x7f10d80c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:32.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.693+0000 7f10df74e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10d80caac0 0x7f10d80b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:32.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.693+0000 7f10df74e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d815ad70 0x7f10d81f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:32.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.693+0000 7f10df74e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d80c3c10 con 0x7f10d80caac0 2023-10-16T11:22:32.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.693+0000 7f10df74e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d81f2790 con 0x7f10d80c9ca0 2023-10-16T11:22:32.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.693+0000 7f10df74e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d81f2a70 con 0x7f10d815ad70 2023-10-16T11:22:32.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.693+0000 7f10dcce9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10d80caac0 0x7f10d80b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:32.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.694+0000 7f10dcce9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10d80caac0 0x7f10d80b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49686/0 (socket says 172.21.15.84:49686) 2023-10-16T11:22:32.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.694+0000 7f10dcce9640 1 -- 172.21.15.84:0/2378457496 learned_addr learned my addr 172.21.15.84:0/2378457496 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:32.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.694+0000 7f10dd4ea640 1 --2- 172.21.15.84:0/2378457496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10d80c9ca0 0x7f10d80c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:32.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.694+0000 7f10ddceb640 1 --2- 172.21.15.84:0/2378457496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d815ad70 0x7f10d81f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:32.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.694+0000 7f10dcce9640 1 -- 172.21.15.84:0/2378457496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d815ad70 msgr2=0x7f10d81f3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.694+0000 7f10dcce9640 1 --2- 172.21.15.84:0/2378457496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d815ad70 0x7f10d81f3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.694+0000 7f10dcce9640 1 -- 172.21.15.84:0/2378457496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10d80c9ca0 msgr2=0x7f10d80c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.694+0000 7f10dcce9640 1 --2- 172.21.15.84:0/2378457496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10d80c9ca0 0x7f10d80c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.694+0000 7f10dcce9640 1 -- 172.21.15.84:0/2378457496 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d0009400 con 0x7f10d80caac0 2023-10-16T11:22:32.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.694+0000 7f10dd4ea640 1 --2- 172.21.15.84:0/2378457496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10d80c9ca0 0x7f10d80c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:32.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.694+0000 7f10dcce9640 1 --2- 172.21.15.84:0/2378457496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10d80caac0 0x7f10d80b2fa0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00c4d0 tx=0x7f10cc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:32.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.694+0000 7f10d5ffb640 1 -- 172.21.15.84:0/2378457496 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc0074e0 con 0x7f10d80caac0 2023-10-16T11:22:32.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.695+0000 7f10d5ffb640 1 -- 172.21.15.84:0/2378457496 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10cc007660 con 0x7f10d80caac0 2023-10-16T11:22:32.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.695+0000 7f10d5ffb640 1 -- 172.21.15.84:0/2378457496 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc004a20 con 0x7f10d80caac0 2023-10-16T11:22:32.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.695+0000 7f10df74e640 1 -- 172.21.15.84:0/2378457496 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d81f2d50 con 0x7f10d80caac0 2023-10-16T11:22:32.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.695+0000 7f10df74e640 1 -- 172.21.15.84:0/2378457496 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d81f3960 con 0x7f10d80caac0 2023-10-16T11:22:32.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.696+0000 7f10df74e640 1 -- 172.21.15.84:0/2378457496 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d804e160 con 0x7f10d80caac0 2023-10-16T11:22:32.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.697+0000 7f10d5ffb640 1 -- 172.21.15.84:0/2378457496 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f10cc004b80 con 0x7f10d80caac0 2023-10-16T11:22:32.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.698+0000 7f10d5ffb640 1 --2- 172.21.15.84:0/2378457496 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f10c406af90 0x7f10c406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:32.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.698+0000 7f10d5ffb640 1 -- 172.21.15.84:0/2378457496 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6003+0+0 (secure 0 0 0) 0x7f10cc088af0 con 0x7f10d80caac0 2023-10-16T11:22:32.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.698+0000 7f10dd4ea640 1 --2- 172.21.15.84:0/2378457496 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f10c406af90 0x7f10c406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:32.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.699+0000 7f10dd4ea640 1 --2- 172.21.15.84:0/2378457496 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f10c406af90 0x7f10c406d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009440 tx=0x7f10c800b040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:32.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.703+0000 7f10d5ffb640 1 -- 172.21.15.84:0/2378457496 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f10cc0538c0 con 0x7f10d80caac0 2023-10-16T11:22:32.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.889+0000 7f10df74e640 1 -- 172.21.15.84:0/2378457496 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f10d80ad480 con 0x7f10d80caac0 2023-10-16T11:22:32.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.894+0000 7f10d5ffb640 1 -- 172.21.15.84:0/2378457496 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v68) v1 ==== 120+0+0 (secure 0 0 0) 0x7f10cc0180b0 con 0x7f10d80caac0 2023-10-16T11:22:32.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.897+0000 7f10df74e640 1 -- 172.21.15.84:0/2378457496 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f10c406af90 msgr2=0x7f10c406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.897+0000 7f10df74e640 1 --2- 172.21.15.84:0/2378457496 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f10c406af90 0x7f10c406d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009440 tx=0x7f10c800b040 comp rx=0 tx=0).stop 2023-10-16T11:22:32.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.897+0000 7f10df74e640 1 -- 172.21.15.84:0/2378457496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10d80caac0 msgr2=0x7f10d80b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.897+0000 7f10df74e640 1 --2- 172.21.15.84:0/2378457496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10d80caac0 0x7f10d80b2fa0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00c4d0 tx=0x7f10cc00c990 comp rx=0 tx=0).stop 2023-10-16T11:22:32.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.897+0000 7f10df74e640 1 -- 172.21.15.84:0/2378457496 shutdown_connections 2023-10-16T11:22:32.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.897+0000 7f10df74e640 1 --2- 172.21.15.84:0/2378457496 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f10c406af90 0x7f10c406d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.897+0000 7f10df74e640 1 --2- 172.21.15.84:0/2378457496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10d815ad70 0x7f10d81f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.897+0000 7f10df74e640 1 --2- 172.21.15.84:0/2378457496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10d80caac0 0x7f10d80b2fa0 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.897+0000 7f10df74e640 1 --2- 172.21.15.84:0/2378457496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10d80c9ca0 0x7f10d80c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.897+0000 7f10df74e640 1 -- 172.21.15.84:0/2378457496 >> 172.21.15.84:0/2378457496 conn(0x7f10d80ce9b0 msgr2=0x7f10d80afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:32.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.897+0000 7f10df74e640 1 -- 172.21.15.84:0/2378457496 shutdown_connections 2023-10-16T11:22:32.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:32.898+0000 7f10df74e640 1 -- 172.21.15.84:0/2378457496 wait complete. 2023-10-16T11:22:32.918 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T11:22:33.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.117+0000 7efd0b316640 1 Processor -- start 2023-10-16T11:22:33.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.118+0000 7efd0b316640 1 -- start start 2023-10-16T11:22:33.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.119+0000 7efd0b316640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efd040cd970 0x7efd0415aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:33.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.119+0000 7efd0b316640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0406be30 con 0x7efd040b0ea0 2023-10-16T11:22:33.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.119+0000 7efd0b316640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd040ada10 con 0x7efd040b3ec0 2023-10-16T11:22:33.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.119+0000 7efd0b316640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd0406ca00 con 0x7efd040cd970 2023-10-16T11:22:33.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.119+0000 7efd0ab15640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efd040cd970 0x7efd0415aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:33.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.119+0000 7efd0ab15640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efd040cd970 0x7efd0415aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49692/0 (socket says 172.21.15.84:49692) 2023-10-16T11:22:33.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.119+0000 7efd0ab15640 1 -- 172.21.15.84:0/1897555105 learned_addr learned my addr 172.21.15.84:0/1897555105 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:33.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.119+0000 7efd09b13640 1 --1- 172.21.15.84:0/1897555105 >> v1:172.21.15.84:6789/0 conn(0x7efd040b0ea0 0x7efd0415a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60616/0 (socket says 172.21.15.84:60616) 2023-10-16T11:22:33.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.119+0000 7efd0ab15640 1 -- 172.21.15.84:0/1897555105 >> v1:172.21.15.157:6789/0 conn(0x7efd040b3ec0 legacy=0x7efd040cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:33.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.119+0000 7efd0ab15640 1 -- 172.21.15.84:0/1897555105 >> v1:172.21.15.84:6789/0 conn(0x7efd040b0ea0 legacy=0x7efd0415a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:33.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.119+0000 7efd0ab15640 1 -- 172.21.15.84:0/1897555105 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0406c060 con 0x7efd040cd970 2023-10-16T11:22:33.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.119+0000 7efd09312640 1 -- 172.21.15.84:0/1897555105 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771472655 0 0) 0x7efd0406be30 con 0x7efd040b0ea0 2023-10-16T11:22:33.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.120+0000 7efd0ab15640 1 --2- 172.21.15.84:0/1897555105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd040cd970 0x7efd0415aa00 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009e50 tx=0x7efcfc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9387ae211390d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:33.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.121+0000 7efd09312640 1 -- 172.21.15.84:0/1897555105 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc02ca80 con 0x7efd040cd970 2023-10-16T11:22:33.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.121+0000 7efd09312640 1 -- 172.21.15.84:0/1897555105 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcfc02cbe0 con 0x7efd040cd970 2023-10-16T11:22:33.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.121+0000 7efd09312640 1 -- 172.21.15.84:0/1897555105 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc0343f0 con 0x7efd040cd970 2023-10-16T11:22:33.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.121+0000 7efd0b316640 1 -- 172.21.15.84:0/1897555105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd040cd970 msgr2=0x7efd0415aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:33.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.121+0000 7efd0b316640 1 --2- 172.21.15.84:0/1897555105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd040cd970 0x7efd0415aa00 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009e50 tx=0x7efcfc02bee0 comp rx=0 tx=0).stop 2023-10-16T11:22:33.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.121+0000 7efd0b316640 1 -- 172.21.15.84:0/1897555105 shutdown_connections 2023-10-16T11:22:33.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.121+0000 7efd0b316640 1 --2- 172.21.15.84:0/1897555105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd040cd970 0x7efd0415aa00 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:33.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.121+0000 7efd0b316640 1 -- 172.21.15.84:0/1897555105 >> 172.21.15.84:0/1897555105 conn(0x7efd040b0170 msgr2=0x7efd040b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:33.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.121+0000 7efd0b316640 1 -- 172.21.15.84:0/1897555105 shutdown_connections 2023-10-16T11:22:33.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.121+0000 7efd0b316640 1 -- 172.21.15.84:0/1897555105 wait complete. 2023-10-16T11:22:33.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.123+0000 7efd0b316640 1 Processor -- start 2023-10-16T11:22:33.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.123+0000 7efd0b316640 1 -- start start 2023-10-16T11:22:33.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.124+0000 7efd0b316640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd040b0ea0 0x7efd041f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:33.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.124+0000 7efd0b316640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd040b3ec0 0x7efd04157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:33.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.124+0000 7efd0b316640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd040cd970 0x7efd040c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:33.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.124+0000 7efd0b316640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd04158600 con 0x7efd040b0ea0 2023-10-16T11:22:33.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.124+0000 7efd0b316640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd041f3f10 con 0x7efd040b3ec0 2023-10-16T11:22:33.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.124+0000 7efd0b316640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd041587e0 con 0x7efd040cd970 2023-10-16T11:22:33.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.124+0000 7efd0a314640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd040b0ea0 0x7efd041f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:33.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.124+0000 7efd0a314640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd040b0ea0 0x7efd041f4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49706/0 (socket says 172.21.15.84:49706) 2023-10-16T11:22:33.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.124+0000 7efd0a314640 1 -- 172.21.15.84:0/2640415209 learned_addr learned my addr 172.21.15.84:0/2640415209 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:33.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.125+0000 7efd0a314640 1 -- 172.21.15.84:0/2640415209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd040cd970 msgr2=0x7efd040c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:33.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.125+0000 7efd0a314640 1 --2- 172.21.15.84:0/2640415209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd040cd970 0x7efd040c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:33.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.125+0000 7efd0a314640 1 -- 172.21.15.84:0/2640415209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd040b3ec0 msgr2=0x7efd04157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:33.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.125+0000 7efd09b13640 1 --2- 172.21.15.84:0/2640415209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd040b3ec0 0x7efd04157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:33.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.125+0000 7efd0a314640 1 --2- 172.21.15.84:0/2640415209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd040b3ec0 0x7efd04157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:33.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.125+0000 7efd0a314640 1 -- 172.21.15.84:0/2640415209 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc009b30 con 0x7efd040b0ea0 2023-10-16T11:22:33.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.125+0000 7efd09b13640 1 --2- 172.21.15.84:0/2640415209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd040b3ec0 0x7efd04157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:33.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.125+0000 7efd0a314640 1 --2- 172.21.15.84:0/2640415209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd040b0ea0 0x7efd041f4330 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7efd0000b420 tx=0x7efd0000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:33.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.125+0000 7efcfaffd640 1 -- 172.21.15.84:0/2640415209 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd00011830 con 0x7efd040b0ea0 2023-10-16T11:22:33.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.125+0000 7efcfaffd640 1 -- 172.21.15.84:0/2640415209 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd00011990 con 0x7efd040b0ea0 2023-10-16T11:22:33.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.125+0000 7efcfaffd640 1 -- 172.21.15.84:0/2640415209 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd00011d40 con 0x7efd040b0ea0 2023-10-16T11:22:33.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.126+0000 7efd0b316640 1 -- 172.21.15.84:0/2640415209 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd04158ac0 con 0x7efd040b0ea0 2023-10-16T11:22:33.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.126+0000 7efd0b316640 1 -- 172.21.15.84:0/2640415209 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd04157770 con 0x7efd040b0ea0 2023-10-16T11:22:33.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.127+0000 7efd0b316640 1 -- 172.21.15.84:0/2640415209 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd0406a890 con 0x7efd040b0ea0 2023-10-16T11:22:33.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.128+0000 7efcfaffd640 1 -- 172.21.15.84:0/2640415209 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efd0001b460 con 0x7efd040b0ea0 2023-10-16T11:22:33.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.129+0000 7efcfaffd640 1 --2- 172.21.15.84:0/2640415209 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7efce806af90 0x7efce806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:33.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.129+0000 7efcfaffd640 1 -- 172.21.15.84:0/2640415209 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6003+0+0 (secure 0 0 0) 0x7efd000893c0 con 0x7efd040b0ea0 2023-10-16T11:22:33.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.129+0000 7efd09b13640 1 --2- 172.21.15.84:0/2640415209 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7efce806af90 0x7efce806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:33.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.129+0000 7efd09b13640 1 --2- 172.21.15.84:0/2640415209 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7efce806af90 0x7efce806d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efd040ad840 tx=0x7efcf4009040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:33.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.133+0000 7efcfaffd640 1 -- 172.21.15.84:0/2640415209 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efd0406a890 con 0x7efd040b0ea0 2023-10-16T11:22:33.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.325+0000 7efd0b316640 1 -- 172.21.15.84:0/2640415209 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7efd041f4050 con 0x7efd040b0ea0 2023-10-16T11:22:33.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.330+0000 7efcfaffd640 1 -- 172.21.15.84:0/2640415209 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v69) v1 ==== 115+0+0 (secure 0 0 0) 0x7efd041f4050 con 0x7efd040b0ea0 2023-10-16T11:22:33.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.334+0000 7efd0b316640 1 -- 172.21.15.84:0/2640415209 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7efce806af90 msgr2=0x7efce806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:33.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.334+0000 7efd0b316640 1 --2- 172.21.15.84:0/2640415209 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7efce806af90 0x7efce806d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efd040ad840 tx=0x7efcf4009040 comp rx=0 tx=0).stop 2023-10-16T11:22:33.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.334+0000 7efd0b316640 1 -- 172.21.15.84:0/2640415209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd040b0ea0 msgr2=0x7efd041f4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:33.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.334+0000 7efd0b316640 1 --2- 172.21.15.84:0/2640415209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd040b0ea0 0x7efd041f4330 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7efd0000b420 tx=0x7efd0000b8e0 comp rx=0 tx=0).stop 2023-10-16T11:22:33.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.334+0000 7efd0b316640 1 -- 172.21.15.84:0/2640415209 shutdown_connections 2023-10-16T11:22:33.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.334+0000 7efd0b316640 1 --2- 172.21.15.84:0/2640415209 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7efce806af90 0x7efce806d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:33.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.334+0000 7efd0b316640 1 --2- 172.21.15.84:0/2640415209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd040cd970 0x7efd040c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:33.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.334+0000 7efd0b316640 1 --2- 172.21.15.84:0/2640415209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd040b3ec0 0x7efd04157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:33.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.334+0000 7efd0b316640 1 --2- 172.21.15.84:0/2640415209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd040b0ea0 0x7efd041f4330 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:33.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.334+0000 7efd0b316640 1 -- 172.21.15.84:0/2640415209 >> 172.21.15.84:0/2640415209 conn(0x7efd040b0170 msgr2=0x7efd040b12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:33.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.334+0000 7efd0b316640 1 -- 172.21.15.84:0/2640415209 shutdown_connections 2023-10-16T11:22:33.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.334+0000 7efd0b316640 1 -- 172.21.15.84:0/2640415209 wait complete. 2023-10-16T11:22:33.356 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-10-16T11:22:33.552 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:22:33.561 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:22:33.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.555+0000 7fdb8ffff640 1 Processor -- start 2023-10-16T11:22:33.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.555+0000 7fdb8ffff640 1 -- start start 2023-10-16T11:22:33.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.557+0000 7fdb8ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdb90070b60 0x7fdb9015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:33.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.557+0000 7fdb8ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9006c100 0x7fdb9006c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:33.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.557+0000 7fdb8ffff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9015aef0 con 0x7fdb90159950 2023-10-16T11:22:33.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.557+0000 7fdb8ffff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb9015e490 con 0x7fdb90070b60 2023-10-16T11:22:33.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.557+0000 7fdb8ffff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9015d910 con 0x7fdb9006c100 2023-10-16T11:22:33.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.557+0000 7fdb8effd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdb90159950 0x7fdb90070450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60620/0 (socket says 172.21.15.84:60620) 2023-10-16T11:22:33.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.557+0000 7fdb8effd640 1 -- 172.21.15.84:0/954896755 learned_addr learned my addr 172.21.15.84:0/954896755 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:33.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8dffb640 1 -- 172.21.15.84:0/954896755 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930919360 0 0) 0x7fdb80001110 con 0x7fdb90159950 2023-10-16T11:22:33.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8e7fc640 1 --2- 172.21.15.84:0/954896755 >> v2:172.21.15.157:3300/0 conn(0x7fdb90070b60 0x7fdb9015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:33.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8dffb640 1 -- 172.21.15.84:0/954896755 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb78003520 con 0x7fdb90159950 2023-10-16T11:22:33.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8dffb640 1 -- 172.21.15.84:0/954896755 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2542904358 0 0) 0x7fdb78003520 con 0x7fdb90159950 2023-10-16T11:22:33.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8f7fe640 1 --2- 172.21.15.84:0/954896755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9006c100 0x7fdb9006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:33.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8dffb640 1 -- 172.21.15.84:0/954896755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9006c100 msgr2=0x7fdb9006c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:33.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8dffb640 1 --2- 172.21.15.84:0/954896755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9006c100 0x7fdb9006c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:33.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8dffb640 1 -- 172.21.15.84:0/954896755 >> v2:172.21.15.157:3300/0 conn(0x7fdb90070b60 msgr2=0x7fdb9015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:33.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8dffb640 1 --2- 172.21.15.84:0/954896755 >> v2:172.21.15.157:3300/0 conn(0x7fdb90070b60 0x7fdb9015a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:33.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8dffb640 1 -- 172.21.15.84:0/954896755 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9015dbf0 con 0x7fdb90159950 2023-10-16T11:22:33.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8dffb640 1 -- 172.21.15.84:0/954896755 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fdb80001800 con 0x7fdb90159950 2023-10-16T11:22:33.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8e7fc640 1 --2- 172.21.15.84:0/954896755 >> v2:172.21.15.157:3300/0 conn(0x7fdb90070b60 0x7fdb9015a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:33.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8dffb640 1 -- 172.21.15.84:0/954896755 >> v1:172.21.15.84:6789/0 conn(0x7fdb90159950 legacy=0x7fdb90070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:33.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8dffb640 1 --2- 172.21.15.84:0/954896755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb78003970 0x7fdb78003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:33.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8dffb640 1 --2- 172.21.15.84:0/954896755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb780047c0 0x7fdb78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:33.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8dffb640 1 --2- 172.21.15.84:0/954896755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb780051d0 0x7fdb7800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:33.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8dffb640 1 -- 172.21.15.84:0/954896755 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9015e490 con 0x7fdb780047c0 2023-10-16T11:22:33.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8dffb640 1 -- 172.21.15.84:0/954896755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9015d910 con 0x7fdb78003970 2023-10-16T11:22:33.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8dffb640 1 -- 172.21.15.84:0/954896755 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7800e340 con 0x7fdb780051d0 2023-10-16T11:22:33.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.558+0000 7fdb8f7fe640 1 --2- 172.21.15.84:0/954896755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9006c100 0x7fdb9006c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:33.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.559+0000 7fdb8e7fc640 1 --2- 172.21.15.84:0/954896755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb780047c0 0x7fdb78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:33.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.559+0000 7fdb8effd640 1 --2- 172.21.15.84:0/954896755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb78003970 0x7fdb78003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:33.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.559+0000 7fdb8e7fc640 1 -- 172.21.15.84:0/954896755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb780051d0 msgr2=0x7fdb7800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:33.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.559+0000 7fdb8e7fc640 1 --2- 172.21.15.84:0/954896755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb780051d0 0x7fdb7800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:33.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.559+0000 7fdb8e7fc640 1 -- 172.21.15.84:0/954896755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb78003970 msgr2=0x7fdb78003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:33.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.559+0000 7fdb8e7fc640 1 --2- 172.21.15.84:0/954896755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb78003970 0x7fdb78003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:33.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.559+0000 7fdb8e7fc640 1 -- 172.21.15.84:0/954896755 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb78003520 con 0x7fdb780047c0 2023-10-16T11:22:33.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.560+0000 7fdb8e7fc640 1 --2- 172.21.15.84:0/954896755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb780047c0 0x7fdb78004c40 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fdb840098a0 tx=0x7fdb780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d21246a4150b8a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:33.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.561+0000 7fdb8dffb640 1 -- 172.21.15.84:0/954896755 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84004230 con 0x7fdb780047c0 2023-10-16T11:22:33.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.561+0000 7fdb8dffb640 1 -- 172.21.15.84:0/954896755 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb84004390 con 0x7fdb780047c0 2023-10-16T11:22:33.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.561+0000 7fdb8dffb640 1 -- 172.21.15.84:0/954896755 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84003830 con 0x7fdb780047c0 2023-10-16T11:22:33.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.561+0000 7fdb8ffff640 1 -- 172.21.15.84:0/954896755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb780047c0 msgr2=0x7fdb78004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:33.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.561+0000 7fdb8ffff640 1 --2- 172.21.15.84:0/954896755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb780047c0 0x7fdb78004c40 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fdb840098a0 tx=0x7fdb780057f0 comp rx=0 tx=0).stop 2023-10-16T11:22:33.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.561+0000 7fdb8f7fe640 1 -- 172.21.15.84:0/954896755 reap_dead start 2023-10-16T11:22:33.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.561+0000 7fdb8ffff640 1 -- 172.21.15.84:0/954896755 shutdown_connections 2023-10-16T11:22:33.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.561+0000 7fdb8ffff640 1 -- 172.21.15.84:0/954896755 >> 172.21.15.84:0/954896755 conn(0x7fdb900ce6a0 msgr2=0x7fdb900ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:33.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.562+0000 7fdb8ffff640 1 -- 172.21.15.84:0/954896755 shutdown_connections 2023-10-16T11:22:33.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.562+0000 7fdb8ffff640 1 -- 172.21.15.84:0/954896755 wait complete. 2023-10-16T11:22:33.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.563+0000 7fdb8ffff640 1 Processor -- start 2023-10-16T11:22:33.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.564+0000 7fdb8ffff640 1 -- start start 2023-10-16T11:22:33.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.564+0000 7fdb8ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9006ada0 0x7fdb9006b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:33.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.564+0000 7fdb8ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb90155be0 0x7fdb900b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:33.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.564+0000 7fdb8ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb900b1c40 0x7fdb900c2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:33.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.564+0000 7fdb8ffff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb900b3860 con 0x7fdb90155be0 2023-10-16T11:22:33.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.564+0000 7fdb8ffff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9006b6c0 con 0x7fdb900b1c40 2023-10-16T11:22:33.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.564+0000 7fdb8ffff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb900b32a0 con 0x7fdb9006ada0 2023-10-16T11:22:33.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.565+0000 7fdb8e7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb90155be0 0x7fdb900b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:33.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.565+0000 7fdb8e7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb90155be0 0x7fdb900b2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49732/0 (socket says 172.21.15.84:49732) 2023-10-16T11:22:33.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.565+0000 7fdb8e7fc640 1 -- 172.21.15.84:0/3583534925 learned_addr learned my addr 172.21.15.84:0/3583534925 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:33.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.565+0000 7fdb8effd640 1 --2- 172.21.15.84:0/3583534925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9006ada0 0x7fdb9006b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:33.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.565+0000 7fdb8f7fe640 1 --2- 172.21.15.84:0/3583534925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb900b1c40 0x7fdb900c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:33.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.565+0000 7fdb8e7fc640 1 -- 172.21.15.84:0/3583534925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9006ada0 msgr2=0x7fdb9006b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:33.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.565+0000 7fdb8e7fc640 1 --2- 172.21.15.84:0/3583534925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9006ada0 0x7fdb9006b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:33.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.565+0000 7fdb8e7fc640 1 -- 172.21.15.84:0/3583534925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb900b1c40 msgr2=0x7fdb900c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:33.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.565+0000 7fdb8e7fc640 1 --2- 172.21.15.84:0/3583534925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb900b1c40 0x7fdb900c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:33.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.565+0000 7fdb8e7fc640 1 -- 172.21.15.84:0/3583534925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb900b3580 con 0x7fdb90155be0 2023-10-16T11:22:33.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.565+0000 7fdb8f7fe640 1 --2- 172.21.15.84:0/3583534925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb900b1c40 0x7fdb900c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:33.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.566+0000 7fdb8e7fc640 1 --2- 172.21.15.84:0/3583534925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb90155be0 0x7fdb900b2a50 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7fdb840099c0 tx=0x7fdb84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:33.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.566+0000 7fdb777fe640 1 -- 172.21.15.84:0/3583534925 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84042600 con 0x7fdb90155be0 2023-10-16T11:22:33.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.566+0000 7fdb777fe640 1 -- 172.21.15.84:0/3583534925 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb84042760 con 0x7fdb90155be0 2023-10-16T11:22:33.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.566+0000 7fdb777fe640 1 -- 172.21.15.84:0/3583534925 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84042b10 con 0x7fdb90155be0 2023-10-16T11:22:33.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.566+0000 7fdb8ffff640 1 -- 172.21.15.84:0/3583534925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb900b1440 con 0x7fdb90155be0 2023-10-16T11:22:33.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.566+0000 7fdb8ffff640 1 -- 172.21.15.84:0/3583534925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb900b2f90 con 0x7fdb90155be0 2023-10-16T11:22:33.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.567+0000 7fdb8ffff640 1 -- 172.21.15.84:0/3583534925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb9004e9b0 con 0x7fdb90155be0 2023-10-16T11:22:33.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.568+0000 7fdb777fe640 1 -- 172.21.15.84:0/3583534925 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdb840373f0 con 0x7fdb90155be0 2023-10-16T11:22:33.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.569+0000 7fdb777fe640 1 --2- 172.21.15.84:0/3583534925 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fdb7c06af90 0x7fdb7c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:33.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.569+0000 7fdb777fe640 1 -- 172.21.15.84:0/3583534925 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6003+0+0 (secure 0 0 0) 0x7fdb840abb10 con 0x7fdb90155be0 2023-10-16T11:22:33.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.570+0000 7fdb8effd640 1 --2- 172.21.15.84:0/3583534925 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fdb7c06af90 0x7fdb7c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:33.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.570+0000 7fdb8effd640 1 --2- 172.21.15.84:0/3583534925 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fdb7c06af90 0x7fdb7c06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdb80001050 tx=0x7fdb80009040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:33.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.574+0000 7fdb777fe640 1 -- 172.21.15.84:0/3583534925 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdb840768e0 con 0x7fdb90155be0 2023-10-16T11:22:33.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:33.821+0000 7fdb8ffff640 1 -- 172.21.15.84:0/3583534925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7fdb900b4620 con 0x7fdb90155be0 2023-10-16T11:22:34.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.024+0000 7fdb777fe640 1 -- 172.21.15.84:0/3583534925 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v3040) v1 ==== 165+0+0 (secure 0 0 0) 0x7fdb84047020 con 0x7fdb90155be0 2023-10-16T11:22:34.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.027+0000 7fdb8ffff640 1 -- 172.21.15.84:0/3583534925 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fdb7c06af90 msgr2=0x7fdb7c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:34.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.027+0000 7fdb8ffff640 1 --2- 172.21.15.84:0/3583534925 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fdb7c06af90 0x7fdb7c06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdb80001050 tx=0x7fdb80009040 comp rx=0 tx=0).stop 2023-10-16T11:22:34.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.027+0000 7fdb8ffff640 1 -- 172.21.15.84:0/3583534925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb90155be0 msgr2=0x7fdb900b2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:34.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.027+0000 7fdb8ffff640 1 --2- 172.21.15.84:0/3583534925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb90155be0 0x7fdb900b2a50 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7fdb840099c0 tx=0x7fdb84004730 comp rx=0 tx=0).stop 2023-10-16T11:22:34.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.027+0000 7fdb8ffff640 1 -- 172.21.15.84:0/3583534925 shutdown_connections 2023-10-16T11:22:34.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.027+0000 7fdb8ffff640 1 --2- 172.21.15.84:0/3583534925 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fdb7c06af90 0x7fdb7c06d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.027+0000 7fdb8ffff640 1 --2- 172.21.15.84:0/3583534925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb900b1c40 0x7fdb900c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.027+0000 7fdb8ffff640 1 --2- 172.21.15.84:0/3583534925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb90155be0 0x7fdb900b2a50 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.027+0000 7fdb8ffff640 1 --2- 172.21.15.84:0/3583534925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb9006ada0 0x7fdb9006b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.027+0000 7fdb8ffff640 1 -- 172.21.15.84:0/3583534925 >> 172.21.15.84:0/3583534925 conn(0x7fdb900ce6a0 msgr2=0x7fdb90070450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:34.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.027+0000 7fdb8ffff640 1 -- 172.21.15.84:0/3583534925 shutdown_connections 2023-10-16T11:22:34.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.027+0000 7fdb8ffff640 1 -- 172.21.15.84:0/3583534925 wait complete. 2023-10-16T11:22:34.047 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2023-10-16T11:22:34.048 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2023-10-16T11:22:34.048 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T11:22:34.049 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:22:34.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.251+0000 7fea23232640 1 Processor -- start 2023-10-16T11:22:34.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.251+0000 7fea23232640 1 -- start start 2023-10-16T11:22:34.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.253+0000 7fea23232640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fea1c06c040 0x7fea1c06c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:34.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.253+0000 7fea23232640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1c15f4a0 con 0x7fea1c0cda80 2023-10-16T11:22:34.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.253+0000 7fea23232640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1c06c960 con 0x7fea1c0ad970 2023-10-16T11:22:34.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.253+0000 7fea23232640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1c15b9d0 con 0x7fea1c06c040 2023-10-16T11:22:34.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.253+0000 7fea22a31640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fea1c0cda80 0x7fea1c15ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60628/0 (socket says 172.21.15.84:60628) 2023-10-16T11:22:34.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.253+0000 7fea22a31640 1 -- 172.21.15.84:0/2884108776 learned_addr learned my addr 172.21.15.84:0/2884108776 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:34.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.253+0000 7fea2122e640 1 -- 172.21.15.84:0/2884108776 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3998177093 0 0) 0x7fea1c15f4a0 con 0x7fea1c0cda80 2023-10-16T11:22:34.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.253+0000 7fea2122e640 1 -- 172.21.15.84:0/2884108776 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea08003520 con 0x7fea1c0cda80 2023-10-16T11:22:34.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.253+0000 7fea22230640 1 --2- 172.21.15.84:0/2884108776 >> v2:172.21.15.157:3300/0 conn(0x7fea1c06c040 0x7fea1c06c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:34.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea2122e640 1 -- 172.21.15.84:0/2884108776 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2048594854 0 0) 0x7fea08003520 con 0x7fea1c0cda80 2023-10-16T11:22:34.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea2122e640 1 -- 172.21.15.84:0/2884108776 >> v2:172.21.15.157:3300/0 conn(0x7fea1c06c040 msgr2=0x7fea1c06c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:34.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea2122e640 1 --2- 172.21.15.84:0/2884108776 >> v2:172.21.15.157:3300/0 conn(0x7fea1c06c040 0x7fea1c06c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea2122e640 1 -- 172.21.15.84:0/2884108776 >> v1:172.21.15.157:6789/0 conn(0x7fea1c0ad970 legacy=0x7fea1c15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:34.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea2122e640 1 -- 172.21.15.84:0/2884108776 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c15bcb0 con 0x7fea1c0cda80 2023-10-16T11:22:34.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea2122e640 1 -- 172.21.15.84:0/2884108776 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fea140019f0 con 0x7fea1c0cda80 2023-10-16T11:22:34.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea2122e640 1 -- 172.21.15.84:0/2884108776 >> v1:172.21.15.84:6789/0 conn(0x7fea1c0cda80 legacy=0x7fea1c15ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:34.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea2122e640 1 --2- 172.21.15.84:0/2884108776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea08003970 0x7fea08003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:34.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea2122e640 1 --2- 172.21.15.84:0/2884108776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea080047c0 0x7fea08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:34.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea2122e640 1 --2- 172.21.15.84:0/2884108776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea08005180 0x7fea0800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:34.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea2122e640 1 -- 172.21.15.84:0/2884108776 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c15b9d0 con 0x7fea080047c0 2023-10-16T11:22:34.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea2122e640 1 -- 172.21.15.84:0/2884108776 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0800c0b0 con 0x7fea08003970 2023-10-16T11:22:34.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea2122e640 1 -- 172.21.15.84:0/2884108776 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0800c270 con 0x7fea08005180 2023-10-16T11:22:34.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea21a2f640 1 --2- 172.21.15.84:0/2884108776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea080047c0 0x7fea08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:34.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea22230640 1 --2- 172.21.15.84:0/2884108776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea08003970 0x7fea08003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:34.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea21a2f640 1 -- 172.21.15.84:0/2884108776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea08005180 msgr2=0x7fea0800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:34.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea21a2f640 1 --2- 172.21.15.84:0/2884108776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea08005180 0x7fea0800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea21a2f640 1 -- 172.21.15.84:0/2884108776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea08003970 msgr2=0x7fea08003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:34.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.254+0000 7fea21a2f640 1 --2- 172.21.15.84:0/2884108776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea08003970 0x7fea08003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.255+0000 7fea21a2f640 1 -- 172.21.15.84:0/2884108776 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea08003520 con 0x7fea080047c0 2023-10-16T11:22:34.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.255+0000 7fea22a31640 1 -- 172.21.15.84:0/2884108776 reap_dead start 2023-10-16T11:22:34.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.256+0000 7fea21a2f640 1 --2- 172.21.15.84:0/2884108776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea080047c0 0x7fea08004c40 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0ade20 tx=0x7fea080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f905b25565eb1ec9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:34.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.256+0000 7fea2122e640 1 -- 172.21.15.84:0/2884108776 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c005270 con 0x7fea080047c0 2023-10-16T11:22:34.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.256+0000 7fea2122e640 1 -- 172.21.15.84:0/2884108776 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0c02e070 con 0x7fea080047c0 2023-10-16T11:22:34.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.256+0000 7fea2122e640 1 -- 172.21.15.84:0/2884108776 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c003610 con 0x7fea080047c0 2023-10-16T11:22:34.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.256+0000 7fea23232640 1 -- 172.21.15.84:0/2884108776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea080047c0 msgr2=0x7fea08004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:34.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.256+0000 7fea23232640 1 --2- 172.21.15.84:0/2884108776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea080047c0 0x7fea08004c40 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0ade20 tx=0x7fea080057f0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.256+0000 7fea23232640 1 -- 172.21.15.84:0/2884108776 shutdown_connections 2023-10-16T11:22:34.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.256+0000 7fea23232640 1 --2- 172.21.15.84:0/2884108776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea080047c0 0x7fea08004c40 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.256+0000 7fea23232640 1 -- 172.21.15.84:0/2884108776 >> 172.21.15.84:0/2884108776 conn(0x7fea1c0ce6a0 msgr2=0x7fea1c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:34.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.257+0000 7fea23232640 1 -- 172.21.15.84:0/2884108776 shutdown_connections 2023-10-16T11:22:34.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.257+0000 7fea23232640 1 -- 172.21.15.84:0/2884108776 wait complete. 2023-10-16T11:22:34.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.258+0000 7fea23232640 1 Processor -- start 2023-10-16T11:22:34.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.258+0000 7fea23232640 1 -- start start 2023-10-16T11:22:34.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.259+0000 7fea23232640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea080047c0 0x7fea1c0c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:34.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.259+0000 7fea23232640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea1c0c6ef0 0x7fea1c0c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:34.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.259+0000 7fea23232640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea1c15a700 0x7fea1c0c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:34.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.259+0000 7fea23232640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c0c6ad0 con 0x7fea1c0c6ef0 2023-10-16T11:22:34.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.259+0000 7fea23232640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c15abb0 con 0x7fea1c15a700 2023-10-16T11:22:34.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.259+0000 7fea23232640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c0c5e50 con 0x7fea080047c0 2023-10-16T11:22:34.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.260+0000 7fea21a2f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea1c0c6ef0 0x7fea1c0c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:34.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.260+0000 7fea21a2f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea1c0c6ef0 0x7fea1c0c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49750/0 (socket says 172.21.15.84:49750) 2023-10-16T11:22:34.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.260+0000 7fea21a2f640 1 -- 172.21.15.84:0/3486105492 learned_addr learned my addr 172.21.15.84:0/3486105492 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:34.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.260+0000 7fea21a2f640 1 -- 172.21.15.84:0/3486105492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea080047c0 msgr2=0x7fea1c0c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:34.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.260+0000 7fea21a2f640 1 --2- 172.21.15.84:0/3486105492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea080047c0 0x7fea1c0c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.260+0000 7fea22a31640 1 --2- 172.21.15.84:0/3486105492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea1c15a700 0x7fea1c0c33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:34.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.260+0000 7fea21a2f640 1 -- 172.21.15.84:0/3486105492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea1c15a700 msgr2=0x7fea1c0c33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:34.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.260+0000 7fea21a2f640 1 --2- 172.21.15.84:0/3486105492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea1c15a700 0x7fea1c0c33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.260+0000 7fea21a2f640 1 -- 172.21.15.84:0/3486105492 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c0c6130 con 0x7fea1c0c6ef0 2023-10-16T11:22:34.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.260+0000 7fea21a2f640 1 --2- 172.21.15.84:0/3486105492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea1c0c6ef0 0x7fea1c0c8150 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fea0c02baa0 tx=0x7fea0c000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:34.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.260+0000 7fea12ffd640 1 -- 172.21.15.84:0/3486105492 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c003b20 con 0x7fea1c0c6ef0 2023-10-16T11:22:34.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.261+0000 7fea12ffd640 1 -- 172.21.15.84:0/3486105492 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0c003c80 con 0x7fea1c0c6ef0 2023-10-16T11:22:34.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.261+0000 7fea12ffd640 1 -- 172.21.15.84:0/3486105492 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c007030 con 0x7fea1c0c6ef0 2023-10-16T11:22:34.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.261+0000 7fea23232640 1 -- 172.21.15.84:0/3486105492 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1c0c6410 con 0x7fea1c0c6ef0 2023-10-16T11:22:34.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.261+0000 7fea23232640 1 -- 172.21.15.84:0/3486105492 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1c0c3930 con 0x7fea1c0c6ef0 2023-10-16T11:22:34.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.262+0000 7fea23232640 1 -- 172.21.15.84:0/3486105492 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea1c06b330 con 0x7fea1c0c6ef0 2023-10-16T11:22:34.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.263+0000 7fea12ffd640 1 -- 172.21.15.84:0/3486105492 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fea0c030030 con 0x7fea1c0c6ef0 2023-10-16T11:22:34.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.264+0000 7fea12ffd640 1 --2- 172.21.15.84:0/3486105492 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fea0006af90 0x7fea0006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:34.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.264+0000 7fea12ffd640 1 -- 172.21.15.84:0/3486105492 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6003+0+0 (secure 0 0 0) 0x7fea0c0aaa00 con 0x7fea1c0c6ef0 2023-10-16T11:22:34.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.264+0000 7fea22230640 1 --2- 172.21.15.84:0/3486105492 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fea0006af90 0x7fea0006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:34.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.265+0000 7fea22230640 1 --2- 172.21.15.84:0/3486105492 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fea0006af90 0x7fea0006d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fea18006a90 tx=0x7fea18006a20 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:34.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.268+0000 7fea12ffd640 1 -- 172.21.15.84:0/3486105492 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fea0c0757f0 con 0x7fea1c0c6ef0 2023-10-16T11:22:34.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.460+0000 7fea23232640 1 -- 172.21.15.84:0/3486105492 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea1c04e9b0 con 0x7fea1c0c6ef0 2023-10-16T11:22:34.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.462+0000 7fea12ffd640 1 -- 172.21.15.84:0/3486105492 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+14515 (secure 0 0 0) 0x7fea0c045020 con 0x7fea1c0c6ef0 2023-10-16T11:22:34.464 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:34.464 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":193,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:22:32.002139+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6837/1106170430":"2023-10-17T11:22:28.834437+0000","172.21.15.84:0/207496168":"2023-10-17T11:22:25.240138+0000","172.21.15.84:0/2289397951":"2023-10-17T11:22:25.240138+0000","172.21.15.84:0/3792094619":"2023-10-17T11:22:25.240138+0000","172.21.15.84:6833/285590107":"2023-10-17T11:22:25.240138+0000","172.21.15.84:6832/285590107":"2023-10-17T11:22:25.240138+0000","172.21.15.84:0/2324168371":"2023-10-17T11:22:25.240138+0000","172.21.15.84:0/1509521423":"2023-10-17T11:22:25.240138+0000","172.21.15.84:0/542522615":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/878936990":"2023-10-17T11:17:21.065656+0000","172.21.15.84:6833/3993913200":"2023-10-17T11:17:21.065656+0000","172.21.15.84:6832/3993913200":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/1064178200":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/3548457413":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/2013731252":"2023-10-17T11:17:21.065656+0000","172.21.15.84:6833/1441051579":"2023-10-17T11:06:37.825968+0000","172.21.15.84:6832/1441051579":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/2359294858":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/2830443959":"2023-10-17T11:06:37.825968+0000","172.21.15.84:6836/1106170430":"2023-10-17T11:22:28.834437+0000","172.21.15.84:0/4150685824":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/4095400472":"2023-10-17T11:17:21.065656+0000","172.21.15.84:0/3348937827":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/2158345505":"2023-10-17T11:06:37.825968+0000","172.21.15.84:0/563672373":"2023-10-17T11:22:25.240138+0000","172.21.15.84:0/235070477":"2023-10-17T11:06:37.825968+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:22:34.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.464+0000 7fea23232640 1 -- 172.21.15.84:0/3486105492 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fea0006af90 msgr2=0x7fea0006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:34.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.464+0000 7fea23232640 1 --2- 172.21.15.84:0/3486105492 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fea0006af90 0x7fea0006d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fea18006a90 tx=0x7fea18006a20 comp rx=0 tx=0).stop 2023-10-16T11:22:34.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.465+0000 7fea23232640 1 -- 172.21.15.84:0/3486105492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea1c0c6ef0 msgr2=0x7fea1c0c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:34.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.465+0000 7fea23232640 1 --2- 172.21.15.84:0/3486105492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea1c0c6ef0 0x7fea1c0c8150 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fea0c02baa0 tx=0x7fea0c000fb0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.465+0000 7fea23232640 1 -- 172.21.15.84:0/3486105492 shutdown_connections 2023-10-16T11:22:34.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.465+0000 7fea23232640 1 --2- 172.21.15.84:0/3486105492 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fea0006af90 0x7fea0006d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.465+0000 7fea23232640 1 --2- 172.21.15.84:0/3486105492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea1c15a700 0x7fea1c0c33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.465+0000 7fea23232640 1 --2- 172.21.15.84:0/3486105492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea1c0c6ef0 0x7fea1c0c8150 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.465+0000 7fea23232640 1 --2- 172.21.15.84:0/3486105492 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea080047c0 0x7fea1c0c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.465+0000 7fea23232640 1 -- 172.21.15.84:0/3486105492 >> 172.21.15.84:0/3486105492 conn(0x7fea1c0ce6a0 msgr2=0x7fea1c0702e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:34.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.465+0000 7fea23232640 1 -- 172.21.15.84:0/3486105492 shutdown_connections 2023-10-16T11:22:34.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.465+0000 7fea23232640 1 -- 172.21.15.84:0/3486105492 wait complete. 2023-10-16T11:22:34.485 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:22:34.486 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:22:34.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.688+0000 7f1c93fd5640 1 Processor -- start 2023-10-16T11:22:34.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.688+0000 7f1c93fd5640 1 -- start start 2023-10-16T11:22:34.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.690+0000 7f1c93fd5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8c15ad70 0x7f1c8c069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:34.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.690+0000 7f1c93fd5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1c8c0c9ca0 0x7f1c8c06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:34.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.690+0000 7f1c93fd5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1c8c0ca3b0 0x7f1c8c0ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:34.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.690+0000 7f1c93fd5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c8c0cad80 con 0x7f1c8c0ca3b0 2023-10-16T11:22:34.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.690+0000 7f1c93fd5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c8c0cb030 con 0x7f1c8c0c9ca0 2023-10-16T11:22:34.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.690+0000 7f1c93fd5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c0c91c0 con 0x7f1c8c15ad70 2023-10-16T11:22:34.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.690+0000 7f1c92572640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1c8c0ca3b0 0x7f1c8c0ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:34.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.690+0000 7f1c92572640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1c8c0ca3b0 0x7f1c8c0ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49756/0 (socket says 172.21.15.84:49756) 2023-10-16T11:22:34.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.690+0000 7f1c92572640 1 -- 172.21.15.84:0/1527318918 learned_addr learned my addr 172.21.15.84:0/1527318918 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.690+0000 7f1c91d71640 1 --2- 172.21.15.84:0/1527318918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8c15ad70 0x7f1c8c069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.691+0000 7f1c92572640 1 -- 172.21.15.84:0/1527318918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8c15ad70 msgr2=0x7f1c8c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.691+0000 7f1c91570640 1 --2- 172.21.15.84:0/1527318918 >> v2:172.21.15.157:3300/0 conn(0x7f1c8c0c9ca0 0x7f1c8c06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.691+0000 7f1c92572640 1 --2- 172.21.15.84:0/1527318918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8c15ad70 0x7f1c8c069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.691+0000 7f1c92572640 1 -- 172.21.15.84:0/1527318918 >> v2:172.21.15.157:3300/0 conn(0x7f1c8c0c9ca0 msgr2=0x7f1c8c06a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.691+0000 7f1c92572640 1 --2- 172.21.15.84:0/1527318918 >> v2:172.21.15.157:3300/0 conn(0x7f1c8c0c9ca0 0x7f1c8c06a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.691+0000 7f1c92572640 1 -- 172.21.15.84:0/1527318918 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8c0c94a0 con 0x7f1c8c0ca3b0 2023-10-16T11:22:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.692+0000 7f1c92572640 1 --2- 172.21.15.84:0/1527318918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c8c0ca3b0 0x7f1c8c0ca840 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c06af00 tx=0x7f1c8402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ebbaab1eaa1451 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.692+0000 7f1c90d6f640 1 -- 172.21.15.84:0/1527318918 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8402ca70 con 0x7f1c8c0ca3b0 2023-10-16T11:22:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.692+0000 7f1c90d6f640 1 -- 172.21.15.84:0/1527318918 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8402cbd0 con 0x7f1c8c0ca3b0 2023-10-16T11:22:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.692+0000 7f1c90d6f640 1 -- 172.21.15.84:0/1527318918 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c840343f0 con 0x7f1c8c0ca3b0 2023-10-16T11:22:34.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.693+0000 7f1c93fd5640 1 -- 172.21.15.84:0/1527318918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c8c0ca3b0 msgr2=0x7f1c8c0ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:34.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.693+0000 7f1c93fd5640 1 --2- 172.21.15.84:0/1527318918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c8c0ca3b0 0x7f1c8c0ca840 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c06af00 tx=0x7f1c8402bed0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.693+0000 7f1c93fd5640 1 -- 172.21.15.84:0/1527318918 shutdown_connections 2023-10-16T11:22:34.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.693+0000 7f1c93fd5640 1 --2- 172.21.15.84:0/1527318918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c8c0ca3b0 0x7f1c8c0ca840 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.693+0000 7f1c93fd5640 1 --2- 172.21.15.84:0/1527318918 >> v2:172.21.15.157:3300/0 conn(0x7f1c8c0c9ca0 0x7f1c8c06a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.693+0000 7f1c93fd5640 1 --2- 172.21.15.84:0/1527318918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8c15ad70 0x7f1c8c069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.693+0000 7f1c93fd5640 1 -- 172.21.15.84:0/1527318918 >> 172.21.15.84:0/1527318918 conn(0x7f1c8c0ce9b0 msgr2=0x7f1c8c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.693+0000 7f1c93fd5640 1 -- 172.21.15.84:0/1527318918 shutdown_connections 2023-10-16T11:22:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.693+0000 7f1c93fd5640 1 -- 172.21.15.84:0/1527318918 wait complete. 2023-10-16T11:22:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.695+0000 7f1c93fd5640 1 Processor -- start 2023-10-16T11:22:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.696+0000 7f1c93fd5640 1 -- start start 2023-10-16T11:22:34.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.697+0000 7f1c93fd5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8c0c9ca0 0x7f1c8c159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:34.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.697+0000 7f1c93fd5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c8c0ca3b0 0x7f1c8c1f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:34.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.697+0000 7f1c93fd5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c8c15ad70 0x7f1c8c0c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:34.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.697+0000 7f1c93fd5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c0c8a70 con 0x7f1c8c0ca3b0 2023-10-16T11:22:34.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.697+0000 7f1c93fd5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c0c81d0 con 0x7f1c8c15ad70 2023-10-16T11:22:34.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.697+0000 7f1c93fd5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c0c84b0 con 0x7f1c8c0c9ca0 2023-10-16T11:22:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.697+0000 7f1c91570640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c8c0ca3b0 0x7f1c8c1f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.697+0000 7f1c91570640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c8c0ca3b0 0x7f1c8c1f14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49762/0 (socket says 172.21.15.84:49762) 2023-10-16T11:22:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.697+0000 7f1c91570640 1 -- 172.21.15.84:0/3476236770 learned_addr learned my addr 172.21.15.84:0/3476236770 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.697+0000 7f1c91d71640 1 --2- 172.21.15.84:0/3476236770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8c0c9ca0 0x7f1c8c159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:34.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.697+0000 7f1c92572640 1 --2- 172.21.15.84:0/3476236770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c8c15ad70 0x7f1c8c0c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:34.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.698+0000 7f1c91570640 1 -- 172.21.15.84:0/3476236770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8c0c9ca0 msgr2=0x7f1c8c159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:34.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.698+0000 7f1c91570640 1 --2- 172.21.15.84:0/3476236770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8c0c9ca0 0x7f1c8c159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.698+0000 7f1c91570640 1 -- 172.21.15.84:0/3476236770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c8c15ad70 msgr2=0x7f1c8c0c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:34.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.698+0000 7f1c91570640 1 --2- 172.21.15.84:0/3476236770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c8c15ad70 0x7f1c8c0c79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.698+0000 7f1c91570640 1 -- 172.21.15.84:0/3476236770 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c84009b30 con 0x7f1c8c0ca3b0 2023-10-16T11:22:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.698+0000 7f1c92572640 1 --2- 172.21.15.84:0/3476236770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c8c15ad70 0x7f1c8c0c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.698+0000 7f1c91570640 1 --2- 172.21.15.84:0/3476236770 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c8c0ca3b0 0x7f1c8c1f14a0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00c7f0 tx=0x7f1c7c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.698+0000 7f1c827fc640 1 -- 172.21.15.84:0/3476236770 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c004220 con 0x7f1c8c0ca3b0 2023-10-16T11:22:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.698+0000 7f1c827fc640 1 -- 172.21.15.84:0/3476236770 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7c004380 con 0x7f1c8c0ca3b0 2023-10-16T11:22:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.698+0000 7f1c827fc640 1 -- 172.21.15.84:0/3476236770 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c007860 con 0x7f1c8c0ca3b0 2023-10-16T11:22:34.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.698+0000 7f1c93fd5640 1 -- 172.21.15.84:0/3476236770 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8c0c8790 con 0x7f1c8c0ca3b0 2023-10-16T11:22:34.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.698+0000 7f1c93fd5640 1 -- 172.21.15.84:0/3476236770 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8c1f19e0 con 0x7f1c8c0ca3b0 2023-10-16T11:22:34.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.699+0000 7f1c93fd5640 1 -- 172.21.15.84:0/3476236770 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c8c0a8b50 con 0x7f1c8c0ca3b0 2023-10-16T11:22:34.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.700+0000 7f1c827fc640 1 -- 172.21.15.84:0/3476236770 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1c7c007bd0 con 0x7f1c8c0ca3b0 2023-10-16T11:22:34.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.702+0000 7f1c827fc640 1 --2- 172.21.15.84:0/3476236770 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f1c7006af90 0x7f1c7006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:34.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.702+0000 7f1c827fc640 1 -- 172.21.15.84:0/3476236770 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6003+0+0 (secure 0 0 0) 0x7f1c7c089900 con 0x7f1c8c0ca3b0 2023-10-16T11:22:34.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.702+0000 7f1c91d71640 1 --2- 172.21.15.84:0/3476236770 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f1c7006af90 0x7f1c7006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:34.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.703+0000 7f1c91d71640 1 --2- 172.21.15.84:0/3476236770 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f1c7006af90 0x7f1c7006d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800b7c0 tx=0x7f1c88008040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:34.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.706+0000 7f1c827fc640 1 -- 172.21.15.84:0/3476236770 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1c8c0a8b50 con 0x7f1c8c0ca3b0 2023-10-16T11:22:34.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.897+0000 7f1c93fd5640 1 -- 172.21.15.84:0/3476236770 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c8c0a8f30 con 0x7f1c8c0ca3b0 2023-10-16T11:22:34.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.898+0000 7f1c827fc640 1 -- 172.21.15.84:0/3476236770 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v193) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c8c0a8f30 con 0x7f1c8c0ca3b0 2023-10-16T11:22:34.900 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:22:34.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.901+0000 7f1c93fd5640 1 -- 172.21.15.84:0/3476236770 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f1c7006af90 msgr2=0x7f1c7006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:34.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.901+0000 7f1c93fd5640 1 --2- 172.21.15.84:0/3476236770 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f1c7006af90 0x7f1c7006d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800b7c0 tx=0x7f1c88008040 comp rx=0 tx=0).stop 2023-10-16T11:22:34.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.901+0000 7f1c93fd5640 1 -- 172.21.15.84:0/3476236770 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c8c0ca3b0 msgr2=0x7f1c8c1f14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:34.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.901+0000 7f1c93fd5640 1 --2- 172.21.15.84:0/3476236770 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c8c0ca3b0 0x7f1c8c1f14a0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00c7f0 tx=0x7f1c7c00ccb0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.901+0000 7f1c93fd5640 1 -- 172.21.15.84:0/3476236770 shutdown_connections 2023-10-16T11:22:34.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.901+0000 7f1c93fd5640 1 --2- 172.21.15.84:0/3476236770 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f1c7006af90 0x7f1c7006d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.901+0000 7f1c93fd5640 1 --2- 172.21.15.84:0/3476236770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c8c15ad70 0x7f1c8c0c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.901+0000 7f1c93fd5640 1 --2- 172.21.15.84:0/3476236770 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c8c0ca3b0 0x7f1c8c1f14a0 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.901+0000 7f1c93fd5640 1 --2- 172.21.15.84:0/3476236770 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c8c0c9ca0 0x7f1c8c159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:34.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.901+0000 7f1c93fd5640 1 -- 172.21.15.84:0/3476236770 >> 172.21.15.84:0/3476236770 conn(0x7f1c8c0ce9b0 msgr2=0x7f1c8c0b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:34.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.901+0000 7f1c93fd5640 1 -- 172.21.15.84:0/3476236770 shutdown_connections 2023-10-16T11:22:34.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:34.901+0000 7f1c93fd5640 1 -- 172.21.15.84:0/3476236770 wait complete. 2023-10-16T11:22:34.920 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-10-16T11:22:35.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.131+0000 7fc00f3f0640 1 Processor -- start 2023-10-16T11:22:35.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.131+0000 7fc00f3f0640 1 -- start start 2023-10-16T11:22:35.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.132+0000 7fc00f3f0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0080adb50 0x7fc00806f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:35.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.132+0000 7fc00f3f0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc008071450 con 0x7fc0080b0b20 2023-10-16T11:22:35.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.132+0000 7fc00f3f0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0080cc600 con 0x7fc008070fd0 2023-10-16T11:22:35.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.132+0000 7fc00f3f0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0080cc8e0 con 0x7fc0080adb50 2023-10-16T11:22:35.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.133+0000 7fc00d98d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc0080b0b20 0x7fc008164f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60644/0 (socket says 172.21.15.84:60644) 2023-10-16T11:22:35.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.133+0000 7fc00d98d640 1 -- 172.21.15.84:0/2299813315 learned_addr learned my addr 172.21.15.84:0/2299813315 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:35.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.133+0000 7fc00d18c640 1 --2- 172.21.15.84:0/2299813315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0080adb50 0x7fc00806f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:35.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.133+0000 7fc007fff640 1 -- 172.21.15.84:0/2299813315 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395584465 0 0) 0x7fc008071450 con 0x7fc0080b0b20 2023-10-16T11:22:35.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.133+0000 7fc007fff640 1 -- 172.21.15.84:0/2299813315 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbff0003520 con 0x7fc0080b0b20 2023-10-16T11:22:35.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.133+0000 7fc007fff640 1 -- 172.21.15.84:0/2299813315 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1290743656 0 0) 0x7fbff0003520 con 0x7fc0080b0b20 2023-10-16T11:22:35.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.133+0000 7fc007fff640 1 -- 172.21.15.84:0/2299813315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0080adb50 msgr2=0x7fc00806f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:35.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.133+0000 7fc007fff640 1 --2- 172.21.15.84:0/2299813315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0080adb50 0x7fc00806f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:35.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc007fff640 1 -- 172.21.15.84:0/2299813315 >> v1:172.21.15.157:6789/0 conn(0x7fc008070fd0 legacy=0x7fc0080b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:35.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc007fff640 1 -- 172.21.15.84:0/2299813315 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0080ccbc0 con 0x7fc0080b0b20 2023-10-16T11:22:35.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc007fff640 1 -- 172.21.15.84:0/2299813315 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc000001670 con 0x7fc0080b0b20 2023-10-16T11:22:35.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc007fff640 1 -- 172.21.15.84:0/2299813315 >> v1:172.21.15.84:6789/0 conn(0x7fc0080b0b20 legacy=0x7fc008164f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:35.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc007fff640 1 --2- 172.21.15.84:0/2299813315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0080adb50 0x7fbff00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:35.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc007fff640 1 --2- 172.21.15.84:0/2299813315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff0003f70 0x7fbff00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:35.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc007fff640 1 --2- 172.21.15.84:0/2299813315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbff0004d00 0x7fbff0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:35.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc007fff640 1 -- 172.21.15.84:0/2299813315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0080cc8e0 con 0x7fbff0004d00 2023-10-16T11:22:35.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc007fff640 1 -- 172.21.15.84:0/2299813315 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff000bac0 con 0x7fbff0003f70 2023-10-16T11:22:35.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc007fff640 1 -- 172.21.15.84:0/2299813315 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff000bc80 con 0x7fc0080adb50 2023-10-16T11:22:35.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc00d98d640 1 --2- 172.21.15.84:0/2299813315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbff0004d00 0x7fbff0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:35.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc00c98b640 1 --2- 172.21.15.84:0/2299813315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff0003f70 0x7fbff00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:35.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc00d18c640 1 --2- 172.21.15.84:0/2299813315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0080adb50 0x7fbff00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:35.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc00d98d640 1 -- 172.21.15.84:0/2299813315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0080adb50 msgr2=0x7fbff00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:35.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc00d98d640 1 --2- 172.21.15.84:0/2299813315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0080adb50 0x7fbff00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:35.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc00d98d640 1 -- 172.21.15.84:0/2299813315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff0003f70 msgr2=0x7fbff00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:35.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc00d98d640 1 --2- 172.21.15.84:0/2299813315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff0003f70 0x7fbff00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:35.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc00d98d640 1 -- 172.21.15.84:0/2299813315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff0003520 con 0x7fbff0004d00 2023-10-16T11:22:35.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc00c98b640 1 --2- 172.21.15.84:0/2299813315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff0003f70 0x7fbff00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:35.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.134+0000 7fc00d18c640 1 --2- 172.21.15.84:0/2299813315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0080adb50 0x7fbff00039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:35.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.135+0000 7fc00d98d640 1 --2- 172.21.15.84:0/2299813315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbff0004d00 0x7fbff0005180 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fc00000c0e0 tx=0x7fbff00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a439d54b10e562a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:35.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.136+0000 7fc007fff640 1 -- 172.21.15.84:0/2299813315 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000005ad0 con 0x7fbff0004d00 2023-10-16T11:22:35.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.136+0000 7fc007fff640 1 -- 172.21.15.84:0/2299813315 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc000005c30 con 0x7fbff0004d00 2023-10-16T11:22:35.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.136+0000 7fc007fff640 1 -- 172.21.15.84:0/2299813315 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00002ca50 con 0x7fbff0004d00 2023-10-16T11:22:35.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.136+0000 7fc00f3f0640 1 -- 172.21.15.84:0/2299813315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbff0004d00 msgr2=0x7fbff0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:35.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.136+0000 7fc00f3f0640 1 --2- 172.21.15.84:0/2299813315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbff0004d00 0x7fbff0005180 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fc00000c0e0 tx=0x7fbff00057f0 comp rx=0 tx=0).stop 2023-10-16T11:22:35.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.136+0000 7fc00d98d640 1 -- 172.21.15.84:0/2299813315 reap_dead start 2023-10-16T11:22:35.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.136+0000 7fc00f3f0640 1 -- 172.21.15.84:0/2299813315 shutdown_connections 2023-10-16T11:22:35.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.136+0000 7fc00f3f0640 1 -- 172.21.15.84:0/2299813315 >> 172.21.15.84:0/2299813315 conn(0x7fc0080cb410 msgr2=0x7fc0080cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:35.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.136+0000 7fc00f3f0640 1 -- 172.21.15.84:0/2299813315 shutdown_connections 2023-10-16T11:22:35.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.137+0000 7fc00f3f0640 1 -- 172.21.15.84:0/2299813315 wait complete. 2023-10-16T11:22:35.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.138+0000 7fc00f3f0640 1 Processor -- start 2023-10-16T11:22:35.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.138+0000 7fc00f3f0640 1 -- start start 2023-10-16T11:22:35.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.139+0000 7fc00f3f0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbff0004d00 0x7fc0080c1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:35.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.139+0000 7fc00f3f0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0080c3320 0x7fc008160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:35.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.139+0000 7fc00f3f0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0080af210 0x7fc0080af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:35.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.139+0000 7fc00f3f0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0080afc00 con 0x7fbff0004d00 2023-10-16T11:22:35.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.139+0000 7fc00f3f0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0080b15b0 con 0x7fc0080af210 2023-10-16T11:22:35.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.139+0000 7fc00f3f0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0080c1770 con 0x7fc0080c3320 2023-10-16T11:22:35.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.140+0000 7fc00d18c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbff0004d00 0x7fc0080c1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:35.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.140+0000 7fc00d18c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbff0004d00 0x7fc0080c1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49770/0 (socket says 172.21.15.84:49770) 2023-10-16T11:22:35.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.140+0000 7fc00d18c640 1 -- 172.21.15.84:0/1245036269 learned_addr learned my addr 172.21.15.84:0/1245036269 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:35.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.140+0000 7fc00d18c640 1 -- 172.21.15.84:0/1245036269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0080c3320 msgr2=0x7fc008160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:35.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.140+0000 7fc00c98b640 1 --2- 172.21.15.84:0/1245036269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0080c3320 0x7fc008160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:35.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.140+0000 7fc00d18c640 1 --2- 172.21.15.84:0/1245036269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0080c3320 0x7fc008160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:35.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.140+0000 7fc00d18c640 1 -- 172.21.15.84:0/1245036269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0080af210 msgr2=0x7fc0080af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:35.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.140+0000 7fc00d18c640 1 --2- 172.21.15.84:0/1245036269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0080af210 0x7fc0080af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:35.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.140+0000 7fc00d18c640 1 -- 172.21.15.84:0/1245036269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0080c38e0 con 0x7fbff0004d00 2023-10-16T11:22:35.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.140+0000 7fc00d18c640 1 --2- 172.21.15.84:0/1245036269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbff0004d00 0x7fc0080c1230 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7fbff800f350 tx=0x7fbff800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:35.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.140+0000 7fc005ffb640 1 -- 172.21.15.84:0/1245036269 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff8003b90 con 0x7fbff0004d00 2023-10-16T11:22:35.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.141+0000 7fc005ffb640 1 -- 172.21.15.84:0/1245036269 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff8011040 con 0x7fbff0004d00 2023-10-16T11:22:35.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.141+0000 7fc005ffb640 1 -- 172.21.15.84:0/1245036269 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff800fea0 con 0x7fbff0004d00 2023-10-16T11:22:35.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.141+0000 7fc00f3f0640 1 -- 172.21.15.84:0/1245036269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0080c3bc0 con 0x7fbff0004d00 2023-10-16T11:22:35.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.141+0000 7fc00f3f0640 1 -- 172.21.15.84:0/1245036269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0080c4180 con 0x7fbff0004d00 2023-10-16T11:22:35.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.142+0000 7fc00f3f0640 1 -- 172.21.15.84:0/1245036269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0080c5860 con 0x7fbff0004d00 2023-10-16T11:22:35.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.143+0000 7fc005ffb640 1 -- 172.21.15.84:0/1245036269 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbff8004020 con 0x7fbff0004d00 2023-10-16T11:22:35.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.144+0000 7fc005ffb640 1 --2- 172.21.15.84:0/1245036269 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fbff406af90 0x7fbff406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:35.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.144+0000 7fc005ffb640 1 -- 172.21.15.84:0/1245036269 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6003+0+0 (secure 0 0 0) 0x7fbff808a150 con 0x7fbff0004d00 2023-10-16T11:22:35.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.144+0000 7fc00c98b640 1 --2- 172.21.15.84:0/1245036269 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fbff406af90 0x7fbff406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:35.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.145+0000 7fc00c98b640 1 --2- 172.21.15.84:0/1245036269 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fbff406af90 0x7fbff406d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc0081fa040 tx=0x7fbffc009040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:35.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.148+0000 7fc005ffb640 1 -- 172.21.15.84:0/1245036269 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbff8054ea0 con 0x7fbff0004d00 2023-10-16T11:22:35.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:35.417+0000 7fc00f3f0640 1 -- 172.21.15.84:0/1245036269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7fc00804e160 con 0x7fbff0004d00 2023-10-16T11:22:36.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.031+0000 7fc005ffb640 1 -- 172.21.15.84:0/1245036269 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3042) v1 ==== 166+0+0 (secure 0 0 0) 0x7fbff80584c0 con 0x7fbff0004d00 2023-10-16T11:22:36.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.035+0000 7fc00f3f0640 1 -- 172.21.15.84:0/1245036269 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fbff406af90 msgr2=0x7fbff406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:36.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.035+0000 7fc00f3f0640 1 --2- 172.21.15.84:0/1245036269 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fbff406af90 0x7fbff406d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc0081fa040 tx=0x7fbffc009040 comp rx=0 tx=0).stop 2023-10-16T11:22:36.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.035+0000 7fc00f3f0640 1 -- 172.21.15.84:0/1245036269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbff0004d00 msgr2=0x7fc0080c1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:36.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.035+0000 7fc00f3f0640 1 --2- 172.21.15.84:0/1245036269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbff0004d00 0x7fc0080c1230 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7fbff800f350 tx=0x7fbff800f810 comp rx=0 tx=0).stop 2023-10-16T11:22:36.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.035+0000 7fc00f3f0640 1 -- 172.21.15.84:0/1245036269 shutdown_connections 2023-10-16T11:22:36.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.035+0000 7fc00f3f0640 1 --2- 172.21.15.84:0/1245036269 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fbff406af90 0x7fbff406d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:36.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.035+0000 7fc00f3f0640 1 --2- 172.21.15.84:0/1245036269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0080af210 0x7fc0080af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:36.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.035+0000 7fc00f3f0640 1 --2- 172.21.15.84:0/1245036269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0080c3320 0x7fc008160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:36.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.035+0000 7fc00f3f0640 1 --2- 172.21.15.84:0/1245036269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbff0004d00 0x7fc0080c1230 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:36.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.035+0000 7fc00f3f0640 1 -- 172.21.15.84:0/1245036269 >> 172.21.15.84:0/1245036269 conn(0x7fc0080cb410 msgr2=0x7fc008164130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:36.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.035+0000 7fc00f3f0640 1 -- 172.21.15.84:0/1245036269 shutdown_connections 2023-10-16T11:22:36.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.035+0000 7fc00f3f0640 1 -- 172.21.15.84:0/1245036269 wait complete. 2023-10-16T11:22:36.061 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T11:22:36.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.280+0000 7fcf9250f640 1 Processor -- start 2023-10-16T11:22:36.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.280+0000 7fcf9250f640 1 -- start start 2023-10-16T11:22:36.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.281+0000 7fcf9250f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcf8c0caac0 0x7fcf8c0ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:36.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.281+0000 7fcf9250f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf8c0cb030 con 0x7fcf8c15ad70 2023-10-16T11:22:36.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.281+0000 7fcf9250f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf8c0c91c0 con 0x7fcf8c06a4b0 2023-10-16T11:22:36.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.281+0000 7fcf9250f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf8c0c94a0 con 0x7fcf8c0caac0 2023-10-16T11:22:36.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.282+0000 7fcf8bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fcf8c15ad70 0x7fcf8c069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58798/0 (socket says 172.21.15.84:58798) 2023-10-16T11:22:36.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.282+0000 7fcf8bfff640 1 -- 172.21.15.84:0/2981544918 learned_addr learned my addr 172.21.15.84:0/2981544918 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:36.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.282+0000 7fcf90aac640 1 --2- 172.21.15.84:0/2981544918 >> v2:172.21.15.84:3300/0 conn(0x7fcf8c0caac0 0x7fcf8c0ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:36.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.282+0000 7fcf8b7fe640 1 -- 172.21.15.84:0/2981544918 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 523043677 0 0) 0x7fcf8c0cb030 con 0x7fcf8c15ad70 2023-10-16T11:22:36.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.282+0000 7fcf90aac640 1 -- 172.21.15.84:0/2981544918 >> v1:172.21.15.157:6789/0 conn(0x7fcf8c06a4b0 legacy=0x7fcf8c0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:36.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.282+0000 7fcf90aac640 1 -- 172.21.15.84:0/2981544918 >> v1:172.21.15.84:6789/0 conn(0x7fcf8c15ad70 legacy=0x7fcf8c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:36.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.282+0000 7fcf90aac640 1 -- 172.21.15.84:0/2981544918 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8c0c9780 con 0x7fcf8c0caac0 2023-10-16T11:22:36.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.283+0000 7fcf90aac640 1 --2- 172.21.15.84:0/2981544918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf8c0caac0 0x7fcf8c0ad4d0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009d90 tx=0x7fcf8402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85258dd1346f4588 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:36.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.284+0000 7fcf8b7fe640 1 -- 172.21.15.84:0/2981544918 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8402cc60 con 0x7fcf8c0caac0 2023-10-16T11:22:36.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.284+0000 7fcf8b7fe640 1 -- 172.21.15.84:0/2981544918 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8402cdc0 con 0x7fcf8c0caac0 2023-10-16T11:22:36.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.284+0000 7fcf8b7fe640 1 -- 172.21.15.84:0/2981544918 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf84034660 con 0x7fcf8c0caac0 2023-10-16T11:22:36.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.284+0000 7fcf9250f640 1 -- 172.21.15.84:0/2981544918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf8c0caac0 msgr2=0x7fcf8c0ad4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:36.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.284+0000 7fcf9250f640 1 --2- 172.21.15.84:0/2981544918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf8c0caac0 0x7fcf8c0ad4d0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009d90 tx=0x7fcf8402beb0 comp rx=0 tx=0).stop 2023-10-16T11:22:36.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.284+0000 7fcf9250f640 1 -- 172.21.15.84:0/2981544918 shutdown_connections 2023-10-16T11:22:36.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.284+0000 7fcf9250f640 1 --2- 172.21.15.84:0/2981544918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf8c0caac0 0x7fcf8c0ad4d0 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:36.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.284+0000 7fcf9250f640 1 -- 172.21.15.84:0/2981544918 >> 172.21.15.84:0/2981544918 conn(0x7fcf8c0ce9b0 msgr2=0x7fcf8c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:36.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.284+0000 7fcf9250f640 1 -- 172.21.15.84:0/2981544918 shutdown_connections 2023-10-16T11:22:36.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.284+0000 7fcf9250f640 1 -- 172.21.15.84:0/2981544918 wait complete. 2023-10-16T11:22:36.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.286+0000 7fcf9250f640 1 Processor -- start 2023-10-16T11:22:36.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.286+0000 7fcf9250f640 1 -- start start 2023-10-16T11:22:36.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.287+0000 7fcf9250f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf8c06a4b0 0x7fcf8c14ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:36.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.287+0000 7fcf9250f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf8c0caac0 0x7fcf8c0c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:36.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.287+0000 7fcf9250f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf8c15ad70 0x7fcf8c1590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:36.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.287+0000 7fcf9250f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c0addd0 con 0x7fcf8c0caac0 2023-10-16T11:22:36.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.287+0000 7fcf9250f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c0ae090 con 0x7fcf8c06a4b0 2023-10-16T11:22:36.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.287+0000 7fcf9250f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c0c4570 con 0x7fcf8c15ad70 2023-10-16T11:22:36.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.287+0000 7fcf83fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf8c0caac0 0x7fcf8c0c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:36.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.287+0000 7fcf83fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf8c0caac0 0x7fcf8c0c34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48202/0 (socket says 172.21.15.84:48202) 2023-10-16T11:22:36.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.287+0000 7fcf83fff640 1 -- 172.21.15.84:0/3562410831 learned_addr learned my addr 172.21.15.84:0/3562410831 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:36.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.288+0000 7fcf90aac640 1 --2- 172.21.15.84:0/3562410831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf8c15ad70 0x7fcf8c1590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:36.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.288+0000 7fcf83fff640 1 -- 172.21.15.84:0/3562410831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf8c15ad70 msgr2=0x7fcf8c1590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:36.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.288+0000 7fcf8bfff640 1 --2- 172.21.15.84:0/3562410831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf8c06a4b0 0x7fcf8c14ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:36.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.288+0000 7fcf83fff640 1 --2- 172.21.15.84:0/3562410831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf8c15ad70 0x7fcf8c1590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:36.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.288+0000 7fcf83fff640 1 -- 172.21.15.84:0/3562410831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf8c06a4b0 msgr2=0x7fcf8c14ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:36.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.288+0000 7fcf83fff640 1 --2- 172.21.15.84:0/3562410831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf8c06a4b0 0x7fcf8c14ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:36.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.288+0000 7fcf83fff640 1 -- 172.21.15.84:0/3562410831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf84009a70 con 0x7fcf8c0caac0 2023-10-16T11:22:36.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.288+0000 7fcf83fff640 1 --2- 172.21.15.84:0/3562410831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf8c0caac0 0x7fcf8c0c34f0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400c430 tx=0x7fcf7400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:36.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.288+0000 7fcf897fa640 1 -- 172.21.15.84:0/3562410831 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74011790 con 0x7fcf8c0caac0 2023-10-16T11:22:36.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.289+0000 7fcf897fa640 1 -- 172.21.15.84:0/3562410831 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf740118f0 con 0x7fcf8c0caac0 2023-10-16T11:22:36.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.289+0000 7fcf897fa640 1 -- 172.21.15.84:0/3562410831 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74011ca0 con 0x7fcf8c0caac0 2023-10-16T11:22:36.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.289+0000 7fcf9250f640 1 -- 172.21.15.84:0/3562410831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8c0c3a30 con 0x7fcf8c0caac0 2023-10-16T11:22:36.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.289+0000 7fcf9250f640 1 -- 172.21.15.84:0/3562410831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf8c0c3fb0 con 0x7fcf8c0caac0 2023-10-16T11:22:36.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.290+0000 7fcf9250f640 1 -- 172.21.15.84:0/3562410831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf8c06fe60 con 0x7fcf8c0caac0 2023-10-16T11:22:36.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.291+0000 7fcf897fa640 1 -- 172.21.15.84:0/3562410831 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcf7401c460 con 0x7fcf8c0caac0 2023-10-16T11:22:36.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.292+0000 7fcf897fa640 1 --2- 172.21.15.84:0/3562410831 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fcf6c06af90 0x7fcf6c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:36.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.292+0000 7fcf897fa640 1 -- 172.21.15.84:0/3562410831 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6003+0+0 (secure 0 0 0) 0x7fcf740898b0 con 0x7fcf8c0caac0 2023-10-16T11:22:36.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.292+0000 7fcf8bfff640 1 --2- 172.21.15.84:0/3562410831 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fcf6c06af90 0x7fcf6c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:36.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.293+0000 7fcf8bfff640 1 --2- 172.21.15.84:0/3562410831 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fcf6c06af90 0x7fcf6c06d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c000f60 tx=0x7fcf7c009040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:36.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.296+0000 7fcf897fa640 1 -- 172.21.15.84:0/3562410831 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcf74054680 con 0x7fcf8c0caac0 2023-10-16T11:22:36.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.488+0000 7fcf9250f640 1 -- 172.21.15.84:0/3562410831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcf8c0ae200 con 0x7fcf8c0caac0 2023-10-16T11:22:36.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.494+0000 7fcf897fa640 1 -- 172.21.15.84:0/3562410831 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v70)=0 v70) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcf74057ca0 con 0x7fcf8c0caac0 2023-10-16T11:22:36.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.496+0000 7fcf9250f640 1 -- 172.21.15.84:0/3562410831 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fcf6c06af90 msgr2=0x7fcf6c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:36.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.496+0000 7fcf9250f640 1 --2- 172.21.15.84:0/3562410831 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fcf6c06af90 0x7fcf6c06d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c000f60 tx=0x7fcf7c009040 comp rx=0 tx=0).stop 2023-10-16T11:22:36.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.497+0000 7fcf9250f640 1 -- 172.21.15.84:0/3562410831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf8c0caac0 msgr2=0x7fcf8c0c34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:36.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.497+0000 7fcf9250f640 1 --2- 172.21.15.84:0/3562410831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf8c0caac0 0x7fcf8c0c34f0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400c430 tx=0x7fcf7400c8f0 comp rx=0 tx=0).stop 2023-10-16T11:22:36.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.497+0000 7fcf9250f640 1 -- 172.21.15.84:0/3562410831 shutdown_connections 2023-10-16T11:22:36.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.497+0000 7fcf9250f640 1 --2- 172.21.15.84:0/3562410831 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fcf6c06af90 0x7fcf6c06d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:36.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.497+0000 7fcf9250f640 1 --2- 172.21.15.84:0/3562410831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf8c15ad70 0x7fcf8c1590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:36.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.497+0000 7fcf9250f640 1 --2- 172.21.15.84:0/3562410831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf8c0caac0 0x7fcf8c0c34f0 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:36.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.497+0000 7fcf9250f640 1 --2- 172.21.15.84:0/3562410831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf8c06a4b0 0x7fcf8c14ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:36.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.497+0000 7fcf9250f640 1 -- 172.21.15.84:0/3562410831 >> 172.21.15.84:0/3562410831 conn(0x7fcf8c0ce9b0 msgr2=0x7fcf8c069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:36.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.497+0000 7fcf9250f640 1 -- 172.21.15.84:0/3562410831 shutdown_connections 2023-10-16T11:22:36.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:36.497+0000 7fcf9250f640 1 -- 172.21.15.84:0/3562410831 wait complete. 2023-10-16T11:22:36.517 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:22:36.517 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:22:36.627 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:22:36.757 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:22:36.759 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:22:36.760 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:36.821 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:22:36.822 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:22:36.823 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:36.884 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:22:36.884 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:22:36.885 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:36.945 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:22:36.945 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:22:36.946 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:37.006 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:22:37.006 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T11:22:37.007 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:22:37.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.225+0000 7f1d3359e640 1 Processor -- start 2023-10-16T11:22:37.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.225+0000 7f1d3359e640 1 -- start start 2023-10-16T11:22:37.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.226+0000 7f1d3359e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1d34070b60 0x7f1d3415a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:37.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.227+0000 7f1d3359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d3406c100 0x7f1d3406c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:37.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.227+0000 7f1d3359e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d3415aef0 con 0x7f1d34159950 2023-10-16T11:22:37.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.227+0000 7f1d3359e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d3415e490 con 0x7f1d34070b60 2023-10-16T11:22:37.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.227+0000 7f1d3359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3415d910 con 0x7f1d3406c100 2023-10-16T11:22:37.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.227+0000 7f1d3259c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1d34159950 0x7f1d34070450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58800/0 (socket says 172.21.15.84:58800) 2023-10-16T11:22:37.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.227+0000 7f1d3259c640 1 -- 172.21.15.84:0/3135841950 learned_addr learned my addr 172.21.15.84:0/3135841950 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:37.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.227+0000 7f1d31d9b640 1 --2- 172.21.15.84:0/3135841950 >> v2:172.21.15.84:3300/0 conn(0x7f1d34070b60 0x7f1d3415a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:37.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.227+0000 7f1d32d9d640 1 --2- 172.21.15.84:0/3135841950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d3406c100 0x7f1d3406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:37.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.227+0000 7f1d3159a640 1 -- 172.21.15.84:0/3135841950 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022075382 0 0) 0x7f1d24001110 con 0x7f1d34159950 2023-10-16T11:22:37.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.227+0000 7f1d3159a640 1 -- 172.21.15.84:0/3135841950 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d1c003520 con 0x7f1d34159950 2023-10-16T11:22:37.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.227+0000 7f1d31d9b640 1 -- 172.21.15.84:0/3135841950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d3406c100 msgr2=0x7f1d3406c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:37.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.227+0000 7f1d31d9b640 1 --2- 172.21.15.84:0/3135841950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d3406c100 0x7f1d3406c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:37.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.227+0000 7f1d31d9b640 1 -- 172.21.15.84:0/3135841950 >> v1:172.21.15.84:6789/0 conn(0x7f1d34159950 legacy=0x7f1d34070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:37.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.227+0000 7f1d31d9b640 1 -- 172.21.15.84:0/3135841950 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3415dbf0 con 0x7f1d34070b60 2023-10-16T11:22:37.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.227+0000 7f1d32d9d640 1 --2- 172.21.15.84:0/3135841950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d3406c100 0x7f1d3406c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:37.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.228+0000 7f1d31d9b640 1 --2- 172.21.15.84:0/3135841950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d34070b60 0x7f1d3415a9b0 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f1d280097d0 tx=0x7f1d2802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3afe07570cff02 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:37.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.229+0000 7f1d3159a640 1 -- 172.21.15.84:0/3135841950 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d28031780 con 0x7f1d34070b60 2023-10-16T11:22:37.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.229+0000 7f1d3159a640 1 -- 172.21.15.84:0/3135841950 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d280318e0 con 0x7f1d34070b60 2023-10-16T11:22:37.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.229+0000 7f1d3159a640 1 -- 172.21.15.84:0/3135841950 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d28031c50 con 0x7f1d34070b60 2023-10-16T11:22:37.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.229+0000 7f1d3359e640 1 -- 172.21.15.84:0/3135841950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d34070b60 msgr2=0x7f1d3415a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:37.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.229+0000 7f1d3359e640 1 --2- 172.21.15.84:0/3135841950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d34070b60 0x7f1d3415a9b0 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f1d280097d0 tx=0x7f1d2802b820 comp rx=0 tx=0).stop 2023-10-16T11:22:37.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.229+0000 7f1d3359e640 1 -- 172.21.15.84:0/3135841950 shutdown_connections 2023-10-16T11:22:37.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.229+0000 7f1d3359e640 1 --2- 172.21.15.84:0/3135841950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d3406c100 0x7f1d3406c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:37.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.229+0000 7f1d3359e640 1 --2- 172.21.15.84:0/3135841950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d34070b60 0x7f1d3415a9b0 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:37.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.229+0000 7f1d3359e640 1 -- 172.21.15.84:0/3135841950 >> 172.21.15.84:0/3135841950 conn(0x7f1d340ce6a0 msgr2=0x7f1d340ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:37.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.229+0000 7f1d3359e640 1 -- 172.21.15.84:0/3135841950 shutdown_connections 2023-10-16T11:22:37.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.229+0000 7f1d3359e640 1 -- 172.21.15.84:0/3135841950 wait complete. 2023-10-16T11:22:37.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.231+0000 7f1d3359e640 1 Processor -- start 2023-10-16T11:22:37.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.231+0000 7f1d3359e640 1 -- start start 2023-10-16T11:22:37.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.232+0000 7f1d3359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d3406c100 0x7f1d3415cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:37.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.232+0000 7f1d3359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d34070b60 0x7f1d340c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:37.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.232+0000 7f1d3359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d34159950 0x7f1d340c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:37.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.232+0000 7f1d3359e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d340c5cb0 con 0x7f1d34070b60 2023-10-16T11:22:37.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.232+0000 7f1d3359e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d340c5ee0 con 0x7f1d34159950 2023-10-16T11:22:37.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.232+0000 7f1d3359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d340c61c0 con 0x7f1d3406c100 2023-10-16T11:22:37.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.232+0000 7f1d31d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d34070b60 0x7f1d340c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:37.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.232+0000 7f1d31d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d34070b60 0x7f1d340c5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48226/0 (socket says 172.21.15.84:48226) 2023-10-16T11:22:37.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.232+0000 7f1d31d9b640 1 -- 172.21.15.84:0/2444725210 learned_addr learned my addr 172.21.15.84:0/2444725210 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:37.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.233+0000 7f1d31d9b640 1 -- 172.21.15.84:0/2444725210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d3406c100 msgr2=0x7f1d3415cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:37.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.233+0000 7f1d31d9b640 1 --2- 172.21.15.84:0/2444725210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d3406c100 0x7f1d3415cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:37.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.233+0000 7f1d31d9b640 1 -- 172.21.15.84:0/2444725210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d34159950 msgr2=0x7f1d340c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:37.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.233+0000 7f1d31d9b640 1 --2- 172.21.15.84:0/2444725210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d34159950 0x7f1d340c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:37.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.233+0000 7f1d31d9b640 1 -- 172.21.15.84:0/2444725210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d28009400 con 0x7f1d34070b60 2023-10-16T11:22:37.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.233+0000 7f1d31d9b640 1 --2- 172.21.15.84:0/2444725210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d34070b60 0x7f1d340c5770 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f1d28034040 tx=0x7f1d280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:37.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.233+0000 7f1d22ffd640 1 -- 172.21.15.84:0/2444725210 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d28032610 con 0x7f1d34070b60 2023-10-16T11:22:37.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.233+0000 7f1d22ffd640 1 -- 172.21.15.84:0/2444725210 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d28032770 con 0x7f1d34070b60 2023-10-16T11:22:37.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.233+0000 7f1d22ffd640 1 -- 172.21.15.84:0/2444725210 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d28032b00 con 0x7f1d34070b60 2023-10-16T11:22:37.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.233+0000 7f1d3359e640 1 -- 172.21.15.84:0/2444725210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d340c64a0 con 0x7f1d34070b60 2023-10-16T11:22:37.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.234+0000 7f1d3359e640 1 -- 172.21.15.84:0/2444725210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d340c6a60 con 0x7f1d34070b60 2023-10-16T11:22:37.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.235+0000 7f1d3359e640 1 -- 172.21.15.84:0/2444725210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d340c6c50 con 0x7f1d34070b60 2023-10-16T11:22:37.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.235+0000 7f1d22ffd640 1 -- 172.21.15.84:0/2444725210 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1d28032c60 con 0x7f1d34070b60 2023-10-16T11:22:37.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.236+0000 7f1d22ffd640 1 --2- 172.21.15.84:0/2444725210 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f1d1406af90 0x7f1d1406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:37.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.236+0000 7f1d22ffd640 1 -- 172.21.15.84:0/2444725210 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6003+0+0 (secure 0 0 0) 0x7f1d280aba90 con 0x7f1d34070b60 2023-10-16T11:22:37.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.237+0000 7f1d3259c640 1 --2- 172.21.15.84:0/2444725210 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f1d1406af90 0x7f1d1406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:37.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.237+0000 7f1d3259c640 1 --2- 172.21.15.84:0/2444725210 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f1d1406af90 0x7f1d1406d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1d24000f60 tx=0x7f1d24001110 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:37.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.241+0000 7f1d22ffd640 1 -- 172.21.15.84:0/2444725210 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1d28076890 con 0x7f1d34070b60 2023-10-16T11:22:37.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.453+0000 7f1d3359e640 1 -- 172.21.15.84:0/2444725210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d3404e9b0 con 0x7f1d34070b60 2023-10-16T11:22:37.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.455+0000 7f1d22ffd640 1 -- 172.21.15.84:0/2444725210 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+3707 (secure 0 0 0) 0x7f1d28079eb0 con 0x7f1d34070b60 2023-10-16T11:22:37.457 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:37.457 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":98,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6347,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1346424254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1346424254},{"type":"v1","addr":"172.21.15.157:6835","nonce":1346424254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3224476768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3224476768},{"type":"v1","addr":"172.21.15.157:6834","nonce":3224476768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":10197,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2490106026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2490106026},{"type":"v1","addr":"172.21.15.84:6835","nonce":2490106026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91},{"gid":10722,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3867314623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3867314623},{"type":"v1","addr":"172.21.15.84:6837","nonce":3867314623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[]} 2023-10-16T11:22:37.457 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 98 2023-10-16T11:22:37.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.457+0000 7f1d3359e640 1 -- 172.21.15.84:0/2444725210 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f1d1406af90 msgr2=0x7f1d1406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:37.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.457+0000 7f1d3359e640 1 --2- 172.21.15.84:0/2444725210 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f1d1406af90 0x7f1d1406d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1d24000f60 tx=0x7f1d24001110 comp rx=0 tx=0).stop 2023-10-16T11:22:37.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.457+0000 7f1d3359e640 1 -- 172.21.15.84:0/2444725210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d34070b60 msgr2=0x7f1d340c5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:37.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.457+0000 7f1d3359e640 1 --2- 172.21.15.84:0/2444725210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d34070b60 0x7f1d340c5770 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f1d28034040 tx=0x7f1d280046c0 comp rx=0 tx=0).stop 2023-10-16T11:22:37.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.457+0000 7f1d3359e640 1 -- 172.21.15.84:0/2444725210 shutdown_connections 2023-10-16T11:22:37.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.457+0000 7f1d3359e640 1 --2- 172.21.15.84:0/2444725210 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f1d1406af90 0x7f1d1406d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:37.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.457+0000 7f1d3359e640 1 --2- 172.21.15.84:0/2444725210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d34159950 0x7f1d340c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:37.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.457+0000 7f1d3359e640 1 --2- 172.21.15.84:0/2444725210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d34070b60 0x7f1d340c5770 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:37.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.457+0000 7f1d3359e640 1 --2- 172.21.15.84:0/2444725210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d3406c100 0x7f1d3415cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:37.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.457+0000 7f1d3359e640 1 -- 172.21.15.84:0/2444725210 >> 172.21.15.84:0/2444725210 conn(0x7f1d340ce6a0 msgr2=0x7f1d34070450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:37.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.457+0000 7f1d3359e640 1 -- 172.21.15.84:0/2444725210 shutdown_connections 2023-10-16T11:22:37.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.458+0000 7f1d3359e640 1 -- 172.21.15.84:0/2444725210 wait complete. 2023-10-16T11:22:37.478 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T11:22:37.479 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T11:22:37.479 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T11:22:37.480 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:37.480 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T11:22:37.481 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T11:22:37.481 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T11:22:37.482 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:37.482 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T11:22:37.482 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T11:22:37.483 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T11:22:37.483 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:37.484 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T11:22:37.484 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T11:22:37.484 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T11:22:37.485 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:37.499 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T11:22:37.499 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T11:22:37.505 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T11:22:37.506 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T11:22:37.508 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T11:22:37.508 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T11:22:37.512 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T11:22:37.512 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T11:22:37.557 INFO:tasks.ceph.mds.c:Started 2023-10-16T11:22:37.557 DEBUG:teuthology.parallel:result is None 2023-10-16T11:22:37.559 INFO:tasks.ceph.mds.d:Started 2023-10-16T11:22:37.560 DEBUG:teuthology.parallel:result is None 2023-10-16T11:22:37.564 INFO:tasks.ceph.mds.a:Started 2023-10-16T11:22:37.564 DEBUG:teuthology.parallel:result is None 2023-10-16T11:22:37.567 INFO:tasks.ceph.mds.b:Started 2023-10-16T11:22:37.567 DEBUG:teuthology.parallel:result is None 2023-10-16T11:22:37.568 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T11:22:37.709 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T11:22:37.710 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T11:22:37.728 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T11:22:37.728 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-10-16T11:22:37.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.751+0000 7f1ebf3c9640 1 Processor -- start 2023-10-16T11:22:37.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.751+0000 7f1ebf3c9640 1 -- start start 2023-10-16T11:22:37.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.752+0000 7f1ebf3c9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1eb8065ab0 0x7f1eb8161100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:37.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.752+0000 7f1ebf3c9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb80ac890 con 0x7f1eb8069fa0 2023-10-16T11:22:37.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.752+0000 7f1ebf3c9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb80ae800 con 0x7f1eb8160530 2023-10-16T11:22:37.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.752+0000 7f1ebf3c9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb80ae1f0 con 0x7f1eb8065ab0 2023-10-16T11:22:37.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.753+0000 7f1ebd966640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1eb8069fa0 0x7f1eb806a430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58812/0 (socket says 172.21.15.84:58812) 2023-10-16T11:22:37.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.753+0000 7f1ebc964640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1eb8065ab0 0x7f1eb8161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:37.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.753+0000 7f1ebd966640 1 -- 172.21.15.84:0/2365562143 learned_addr learned my addr 172.21.15.84:0/2365562143 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:37.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.753+0000 7f1ebc964640 1 -- 172.21.15.84:0/2365562143 >> v1:172.21.15.157:6789/0 conn(0x7f1eb8160530 legacy=0x7f1eb81609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:37.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.753+0000 7f1ebc964640 1 -- 172.21.15.84:0/2365562143 >> v1:172.21.15.84:6789/0 conn(0x7f1eb8069fa0 legacy=0x7f1eb806a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:37.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.753+0000 7f1ebc964640 1 -- 172.21.15.84:0/2365562143 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb80acae0 con 0x7f1eb8065ab0 2023-10-16T11:22:37.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.753+0000 7f1eaffff640 1 -- 172.21.15.84:0/2365562143 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610990013 0 0) 0x7f1eb80ac890 con 0x7f1eb8069fa0 2023-10-16T11:22:37.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.754+0000 7f1ebc964640 1 --2- 172.21.15.84:0/2365562143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb8065ab0 0x7f1eb8161100 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80097a0 tx=0x7f1ea802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f65e7ef1924a1528 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:37.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.755+0000 7f1eaffff640 1 -- 172.21.15.84:0/2365562143 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8031830 con 0x7f1eb8065ab0 2023-10-16T11:22:37.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.755+0000 7f1eaffff640 1 -- 172.21.15.84:0/2365562143 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea8031990 con 0x7f1eb8065ab0 2023-10-16T11:22:37.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.755+0000 7f1eaffff640 1 -- 172.21.15.84:0/2365562143 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8031d20 con 0x7f1eb8065ab0 2023-10-16T11:22:37.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.755+0000 7f1ebf3c9640 1 -- 172.21.15.84:0/2365562143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb8065ab0 msgr2=0x7f1eb8161100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:37.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.755+0000 7f1ebf3c9640 1 --2- 172.21.15.84:0/2365562143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb8065ab0 0x7f1eb8161100 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80097a0 tx=0x7f1ea802b8d0 comp rx=0 tx=0).stop 2023-10-16T11:22:37.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.755+0000 7f1ebf3c9640 1 -- 172.21.15.84:0/2365562143 shutdown_connections 2023-10-16T11:22:37.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.755+0000 7f1ebf3c9640 1 --2- 172.21.15.84:0/2365562143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb8065ab0 0x7f1eb8161100 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:37.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.755+0000 7f1ebf3c9640 1 -- 172.21.15.84:0/2365562143 >> 172.21.15.84:0/2365562143 conn(0x7f1eb806bc10 msgr2=0x7f1eb806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:37.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.755+0000 7f1ebf3c9640 1 -- 172.21.15.84:0/2365562143 shutdown_connections 2023-10-16T11:22:37.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.755+0000 7f1ebf3c9640 1 -- 172.21.15.84:0/2365562143 wait complete. 2023-10-16T11:22:37.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.757+0000 7f1ebf3c9640 1 Processor -- start 2023-10-16T11:22:37.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.757+0000 7f1ebf3c9640 1 -- start start 2023-10-16T11:22:37.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.758+0000 7f1ebf3c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb8065ab0 0x7f1eb80b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:37.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.758+0000 7f1ebf3c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb8069fa0 0x7f1eb814e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:37.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.758+0000 7f1ebf3c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1eb8160530 0x7f1eb80afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:37.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.758+0000 7f1ebf3c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80c3250 con 0x7f1eb8160530 2023-10-16T11:22:37.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.758+0000 7f1ebf3c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80c3530 con 0x7f1eb8065ab0 2023-10-16T11:22:37.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.758+0000 7f1ebf3c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80c3810 con 0x7f1eb8069fa0 2023-10-16T11:22:37.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.758+0000 7f1ebd966640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1eb8160530 0x7f1eb80afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:37.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.758+0000 7f1ebd966640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1eb8160530 0x7f1eb80afa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48270/0 (socket says 172.21.15.84:48270) 2023-10-16T11:22:37.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.758+0000 7f1ebd966640 1 -- 172.21.15.84:0/1551938373 learned_addr learned my addr 172.21.15.84:0/1551938373 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:37.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.759+0000 7f1ebc964640 1 --2- 172.21.15.84:0/1551938373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb8069fa0 0x7f1eb814e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:37.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.759+0000 7f1ebd165640 1 --2- 172.21.15.84:0/1551938373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb8065ab0 0x7f1eb80b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:37.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.759+0000 7f1ebd966640 1 -- 172.21.15.84:0/1551938373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb8069fa0 msgr2=0x7f1eb814e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:37.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.759+0000 7f1ebd966640 1 --2- 172.21.15.84:0/1551938373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb8069fa0 0x7f1eb814e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:37.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.759+0000 7f1ebd966640 1 -- 172.21.15.84:0/1551938373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb8065ab0 msgr2=0x7f1eb80b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:37.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.759+0000 7f1ebd966640 1 --2- 172.21.15.84:0/1551938373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb8065ab0 0x7f1eb80b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:37.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.759+0000 7f1ebd966640 1 -- 172.21.15.84:0/1551938373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea8009400 con 0x7f1eb8160530 2023-10-16T11:22:37.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.759+0000 7f1ebd966640 1 --2- 172.21.15.84:0/1551938373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1eb8160530 0x7f1eb80afa70 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c7e0 tx=0x7f1eb000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:37.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.759+0000 7f1eadffb640 1 -- 172.21.15.84:0/1551938373 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000a640 con 0x7f1eb8160530 2023-10-16T11:22:37.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.759+0000 7f1eadffb640 1 -- 172.21.15.84:0/1551938373 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb000a7c0 con 0x7f1eb8160530 2023-10-16T11:22:37.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.759+0000 7f1eadffb640 1 -- 172.21.15.84:0/1551938373 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000ab70 con 0x7f1eb8160530 2023-10-16T11:22:37.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.759+0000 7f1ebf3c9640 1 -- 172.21.15.84:0/1551938373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb80c3af0 con 0x7f1eb8160530 2023-10-16T11:22:37.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.759+0000 7f1ebf3c9640 1 -- 172.21.15.84:0/1551938373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb80b4d80 con 0x7f1eb8160530 2023-10-16T11:22:37.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.761+0000 7f1ebf3c9640 1 -- 172.21.15.84:0/1551938373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb80c4010 con 0x7f1eb8160530 2023-10-16T11:22:37.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.761+0000 7f1eadffb640 1 -- 172.21.15.84:0/1551938373 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1eb000acd0 con 0x7f1eb8160530 2023-10-16T11:22:37.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.762+0000 7f1eadffb640 1 --2- 172.21.15.84:0/1551938373 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f1e9c06af90 0x7f1e9c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:37.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.762+0000 7f1eadffb640 1 -- 172.21.15.84:0/1551938373 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6003+0+0 (secure 0 0 0) 0x7f1eb0088ee0 con 0x7f1eb8160530 2023-10-16T11:22:37.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.763+0000 7f1ebd165640 1 --2- 172.21.15.84:0/1551938373 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f1e9c06af90 0x7f1e9c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:37.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.763+0000 7f1ebd165640 1 --2- 172.21.15.84:0/1551938373 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f1e9c06af90 0x7f1e9c06d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4000f60 tx=0x7f1eb4009040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:37.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.767+0000 7f1eadffb640 1 -- 172.21.15.84:0/1551938373 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1eb80c4010 con 0x7f1eb8160530 2023-10-16T11:22:37.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:37.966+0000 7f1ebf3c9640 1 -- 172.21.15.84:0/1551938373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1eb80c4010 con 0x7f1eb8160530 2023-10-16T11:22:38.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.053+0000 7f1eadffb640 1 -- 172.21.15.84:0/1551938373 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v195) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1eb80c4010 con 0x7f1eb8160530 2023-10-16T11:22:38.055 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-10-16T11:22:38.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.056+0000 7f1ebf3c9640 1 -- 172.21.15.84:0/1551938373 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f1e9c06af90 msgr2=0x7f1e9c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:38.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.056+0000 7f1ebf3c9640 1 --2- 172.21.15.84:0/1551938373 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f1e9c06af90 0x7f1e9c06d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4000f60 tx=0x7f1eb4009040 comp rx=0 tx=0).stop 2023-10-16T11:22:38.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.056+0000 7f1ebf3c9640 1 -- 172.21.15.84:0/1551938373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1eb8160530 msgr2=0x7f1eb80afa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:38.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.056+0000 7f1ebf3c9640 1 --2- 172.21.15.84:0/1551938373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1eb8160530 0x7f1eb80afa70 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c7e0 tx=0x7f1eb000cca0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.056+0000 7f1ebf3c9640 1 -- 172.21.15.84:0/1551938373 shutdown_connections 2023-10-16T11:22:38.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.056+0000 7f1ebf3c9640 1 --2- 172.21.15.84:0/1551938373 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f1e9c06af90 0x7f1e9c06d450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.056+0000 7f1ebf3c9640 1 --2- 172.21.15.84:0/1551938373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1eb8160530 0x7f1eb80afa70 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.056+0000 7f1ebf3c9640 1 --2- 172.21.15.84:0/1551938373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1eb8069fa0 0x7f1eb814e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.056+0000 7f1ebf3c9640 1 --2- 172.21.15.84:0/1551938373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1eb8065ab0 0x7f1eb80b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.056+0000 7f1ebf3c9640 1 -- 172.21.15.84:0/1551938373 >> 172.21.15.84:0/1551938373 conn(0x7f1eb806bc10 msgr2=0x7f1eb80aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:38.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.056+0000 7f1ebf3c9640 1 -- 172.21.15.84:0/1551938373 shutdown_connections 2023-10-16T11:22:38.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.056+0000 7f1ebf3c9640 1 -- 172.21.15.84:0/1551938373 wait complete. 2023-10-16T11:22:38.079 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T11:22:38.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.273+0000 7fce40397640 1 Processor -- start 2023-10-16T11:22:38.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.273+0000 7fce40397640 1 -- start start 2023-10-16T11:22:38.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.274+0000 7fce40397640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fce380ce390 0x7fce3815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:38.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.274+0000 7fce40397640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3815ed60 con 0x7fce3815aaa0 2023-10-16T11:22:38.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.274+0000 7fce40397640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce380707e0 con 0x7fce3815ae80 2023-10-16T11:22:38.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.274+0000 7fce40397640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce3806b810 con 0x7fce380ce390 2023-10-16T11:22:38.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.275+0000 7fce3e934640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fce380ce390 0x7fce3815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:38.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.275+0000 7fce3e934640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fce380ce390 0x7fce3815e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48284/0 (socket says 172.21.15.84:48284) 2023-10-16T11:22:38.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.275+0000 7fce3e934640 1 -- 172.21.15.84:0/3746649803 learned_addr learned my addr 172.21.15.84:0/3746649803 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:38.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.275+0000 7fce3e934640 1 -- 172.21.15.84:0/3746649803 >> v1:172.21.15.157:6789/0 conn(0x7fce3815ae80 legacy=0x7fce38070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:38.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.275+0000 7fce3e934640 1 -- 172.21.15.84:0/3746649803 >> v1:172.21.15.84:6789/0 conn(0x7fce3815aaa0 legacy=0x7fce380cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:38.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.275+0000 7fce3e934640 1 -- 172.21.15.84:0/3746649803 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3815ef90 con 0x7fce380ce390 2023-10-16T11:22:38.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.276+0000 7fce3e934640 1 --2- 172.21.15.84:0/3746649803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce380ce390 0x7fce3815e820 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7fce30009e50 tx=0x7fce3002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3358cee8498d80ad server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:38.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.277+0000 7fce3d131640 1 -- 172.21.15.84:0/3746649803 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3002cb10 con 0x7fce380ce390 2023-10-16T11:22:38.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.277+0000 7fce3d131640 1 -- 172.21.15.84:0/3746649803 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3002cc70 con 0x7fce380ce390 2023-10-16T11:22:38.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.277+0000 7fce3d131640 1 -- 172.21.15.84:0/3746649803 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce300343f0 con 0x7fce380ce390 2023-10-16T11:22:38.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.277+0000 7fce40397640 1 -- 172.21.15.84:0/3746649803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce380ce390 msgr2=0x7fce3815e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:38.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.277+0000 7fce40397640 1 --2- 172.21.15.84:0/3746649803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce380ce390 0x7fce3815e820 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7fce30009e50 tx=0x7fce3002bf70 comp rx=0 tx=0).stop 2023-10-16T11:22:38.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.277+0000 7fce40397640 1 -- 172.21.15.84:0/3746649803 shutdown_connections 2023-10-16T11:22:38.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.277+0000 7fce40397640 1 --2- 172.21.15.84:0/3746649803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce380ce390 0x7fce3815e820 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.277+0000 7fce40397640 1 -- 172.21.15.84:0/3746649803 >> 172.21.15.84:0/3746649803 conn(0x7fce380cb410 msgr2=0x7fce380cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:38.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.277+0000 7fce40397640 1 -- 172.21.15.84:0/3746649803 shutdown_connections 2023-10-16T11:22:38.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.277+0000 7fce40397640 1 -- 172.21.15.84:0/3746649803 wait complete. 2023-10-16T11:22:38.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.279+0000 7fce40397640 1 Processor -- start 2023-10-16T11:22:38.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.279+0000 7fce40397640 1 -- start start 2023-10-16T11:22:38.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.280+0000 7fce40397640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce380ce390 0x7fce381f0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:38.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.280+0000 7fce40397640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce3815aaa0 0x7fce381576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:38.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.280+0000 7fce40397640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce3815ae80 0x7fce38157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:38.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.280+0000 7fce40397640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3815d8e0 con 0x7fce3815ae80 2023-10-16T11:22:38.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.280+0000 7fce40397640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3815ca00 con 0x7fce380ce390 2023-10-16T11:22:38.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.280+0000 7fce40397640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce380c6b90 con 0x7fce3815aaa0 2023-10-16T11:22:38.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.280+0000 7fce3e934640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce3815ae80 0x7fce38157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:38.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.280+0000 7fce3e934640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce3815ae80 0x7fce38157be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48300/0 (socket says 172.21.15.84:48300) 2023-10-16T11:22:38.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.280+0000 7fce3e934640 1 -- 172.21.15.84:0/2606385733 learned_addr learned my addr 172.21.15.84:0/2606385733 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:38.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.281+0000 7fce3e934640 1 -- 172.21.15.84:0/2606385733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce3815aaa0 msgr2=0x7fce381576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:38.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.281+0000 7fce3e934640 1 --2- 172.21.15.84:0/2606385733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce3815aaa0 0x7fce381576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.281+0000 7fce3e934640 1 -- 172.21.15.84:0/2606385733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce380ce390 msgr2=0x7fce381f0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:38.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.281+0000 7fce3e934640 1 --2- 172.21.15.84:0/2606385733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce380ce390 0x7fce381f0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.281+0000 7fce3e934640 1 -- 172.21.15.84:0/2606385733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce30009b30 con 0x7fce3815ae80 2023-10-16T11:22:38.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.281+0000 7fce3e934640 1 --2- 172.21.15.84:0/2606385733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce3815ae80 0x7fce38157be0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fce30036040 tx=0x7fce30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:38.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.281+0000 7fce2effd640 1 -- 172.21.15.84:0/2606385733 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3003e450 con 0x7fce3815ae80 2023-10-16T11:22:38.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.281+0000 7fce2effd640 1 -- 172.21.15.84:0/2606385733 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3003e5d0 con 0x7fce3815ae80 2023-10-16T11:22:38.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.281+0000 7fce2effd640 1 -- 172.21.15.84:0/2606385733 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3003e980 con 0x7fce3815ae80 2023-10-16T11:22:38.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.281+0000 7fce40397640 1 -- 172.21.15.84:0/2606385733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce381f19e0 con 0x7fce3815ae80 2023-10-16T11:22:38.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.281+0000 7fce40397640 1 -- 172.21.15.84:0/2606385733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce380c7110 con 0x7fce3815ae80 2023-10-16T11:22:38.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.282+0000 7fce40397640 1 -- 172.21.15.84:0/2606385733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3804e160 con 0x7fce3815ae80 2023-10-16T11:22:38.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.283+0000 7fce2effd640 1 -- 172.21.15.84:0/2606385733 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fce3003eae0 con 0x7fce3815ae80 2023-10-16T11:22:38.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.284+0000 7fce2effd640 1 --2- 172.21.15.84:0/2606385733 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fce1c06af90 0x7fce1c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:38.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.284+0000 7fce2effd640 1 -- 172.21.15.84:0/2606385733 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fce300ac260 con 0x7fce3815ae80 2023-10-16T11:22:38.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.284+0000 7fce3e133640 1 --2- 172.21.15.84:0/2606385733 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fce1c06af90 0x7fce1c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:38.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.285+0000 7fce3e133640 1 --2- 172.21.15.84:0/2606385733 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fce1c06af90 0x7fce1c06d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fce34000f60 tx=0x7fce340079b0 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:38.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.288+0000 7fce2effd640 1 -- 172.21.15.84:0/2606385733 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fce30077670 con 0x7fce3815ae80 2023-10-16T11:22:38.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.526+0000 7fce40397640 1 -- 172.21.15.84:0/2606385733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fce380c72a0 con 0x7fce3815ae80 2023-10-16T11:22:38.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.528+0000 7fce2effd640 1 -- 172.21.15.84:0/2606385733 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v53) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fce3007ac90 con 0x7fce3815ae80 2023-10-16T11:22:38.529 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:38.530 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:22:38.530 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-10-16T11:22:38.530 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.530 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-10-16T11:22:38.531 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l6dhjKRAA0pMvY6AK6qtFXDprNhwsqg==", 2023-10-16T11:22:38.531 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.531 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:22:38.531 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:22:38.531 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:22:38.531 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:22:38.532 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.532 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:22:38.532 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.532 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-10-16T11:22:38.532 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lFpMxEBAA0TLPRt5q3FwwBvMG019cjQ==", 2023-10-16T11:22:38.532 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.533 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:22:38.533 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:22:38.533 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:22:38.533 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:22:38.533 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.533 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:22:38.534 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.534 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-10-16T11:22:38.534 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l73RBOhAAOMEPxfUAMHhn5Blyvd5oMw==", 2023-10-16T11:22:38.534 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.534 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:22:38.535 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:22:38.535 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:22:38.535 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:22:38.535 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.535 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:22:38.535 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.536 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-10-16T11:22:38.536 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lcTIRIhAA0+FWm4b4yw77LD3+WKbk7w==", 2023-10-16T11:22:38.536 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.536 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:22:38.536 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:22:38.536 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:22:38.537 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:22:38.537 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.537 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:22:38.537 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.537 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-10-16T11:22:38.538 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDYBi1l1zBLERAA3YLW3xIYQoRVw7KSkwN3Og==", 2023-10-16T11:22:38.538 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.538 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:22:38.538 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:22:38.538 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:22:38.539 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.539 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:22:38.539 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.539 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-10-16T11:22:38.539 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDbBi1lFUUxKRAAQbus8SmYprJRmWTjoCznNA==", 2023-10-16T11:22:38.540 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.540 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:22:38.540 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:22:38.540 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:22:38.540 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.540 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:22:38.541 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.541 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-10-16T11:22:38.541 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDeBi1l+9lBMxAArl6sb/h15uBG/a9lVH40Hg==", 2023-10-16T11:22:38.541 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.541 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:22:38.542 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:22:38.542 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:22:38.542 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.542 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:22:38.542 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.543 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-10-16T11:22:38.543 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDiBi1l2rnEERAA3oRD71xa0fEOynM8zKuBEg==", 2023-10-16T11:22:38.543 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.543 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:22:38.543 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:22:38.544 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:22:38.544 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.544 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:22:38.544 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.544 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-10-16T11:22:38.545 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDnBi1lR7oLChAAZto8fDovDbpOd3Fcnc7C6w==", 2023-10-16T11:22:38.545 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.545 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:22:38.545 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:22:38.545 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:22:38.546 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.546 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:22:38.546 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.546 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-10-16T11:22:38.546 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDqBi1lmjg/JBAAlINKRGBc0/UWJJHMVFw57w==", 2023-10-16T11:22:38.547 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.547 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:22:38.547 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:22:38.547 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:22:38.547 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.548 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:22:38.548 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.548 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-10-16T11:22:38.548 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDuBi1lu0kMARAAtuSeM8F3yr8CVrc3p1biaQ==", 2023-10-16T11:22:38.548 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.548 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:22:38.549 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:22:38.549 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:22:38.549 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.549 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:22:38.549 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.550 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-10-16T11:22:38.550 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDxBi1lf6mNGBAAHGbqzJ0fmDxQIsasboHMfg==", 2023-10-16T11:22:38.550 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.550 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:22:38.550 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:22:38.551 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:22:38.551 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.551 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:22:38.551 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.551 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-10-16T11:22:38.552 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ==", 2023-10-16T11:22:38.552 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.552 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:22:38.552 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow r", 2023-10-16T11:22:38.552 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T11:22:38.552 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.553 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:22:38.553 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.553 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-10-16T11:22:38.553 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1lHU+4BxAAd8ET8hQ8Y+EUwdTJr9/+Ig==", 2023-10-16T11:22:38.553 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.554 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:22:38.554 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:22:38.554 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:22:38.554 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:22:38.554 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.554 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:22:38.555 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.555 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-10-16T11:22:38.555 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lA1BOORAAFNyfqgEF34MVR9YX3KBKkQ==", 2023-10-16T11:22:38.555 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.555 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:22:38.556 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:22:38.556 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:22:38.556 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:22:38.556 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.556 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:22:38.557 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.557 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-10-16T11:22:38.557 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1l848oERAANMfrZz9JrlgNVh7ZFbtnaA==", 2023-10-16T11:22:38.557 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.557 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:22:38.557 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:22:38.558 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:22:38.558 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:22:38.558 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.558 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:22:38.558 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.559 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-10-16T11:22:38.559 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDRBi1lTAxTGRAAMptxnxy3Qit/cTCXie6CrQ==", 2023-10-16T11:22:38.559 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.559 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:22:38.559 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:22:38.560 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:22:38.560 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:22:38.560 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.560 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:22:38.560 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.561 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-10-16T11:22:38.561 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lPuJxHRAABAnGPVCrkq0JR4Y6iFGZlw==", 2023-10-16T11:22:38.561 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.561 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:22:38.561 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T11:22:38.561 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:22:38.562 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.562 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:22:38.562 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:38.562 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-10-16T11:22:38.562 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lLxnKEBAA+a/pOHUp7Sg3m4weAust5w==", 2023-10-16T11:22:38.563 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:22:38.563 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:22:38.563 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T11:22:38.563 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:22:38.563 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.563 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:38.564 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-10-16T11:22:38.564 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:22:38.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.530+0000 7fce40397640 1 -- 172.21.15.84:0/2606385733 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fce1c06af90 msgr2=0x7fce1c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:38.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.530+0000 7fce40397640 1 --2- 172.21.15.84:0/2606385733 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fce1c06af90 0x7fce1c06d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fce34000f60 tx=0x7fce340079b0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.530+0000 7fce40397640 1 -- 172.21.15.84:0/2606385733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce3815ae80 msgr2=0x7fce38157be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:38.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.530+0000 7fce40397640 1 --2- 172.21.15.84:0/2606385733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce3815ae80 0x7fce38157be0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fce30036040 tx=0x7fce30004a40 comp rx=0 tx=0).stop 2023-10-16T11:22:38.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.530+0000 7fce40397640 1 -- 172.21.15.84:0/2606385733 shutdown_connections 2023-10-16T11:22:38.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.530+0000 7fce40397640 1 --2- 172.21.15.84:0/2606385733 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fce1c06af90 0x7fce1c06d450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.530+0000 7fce40397640 1 --2- 172.21.15.84:0/2606385733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce3815ae80 0x7fce38157be0 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.530+0000 7fce40397640 1 --2- 172.21.15.84:0/2606385733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce3815aaa0 0x7fce381576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.531+0000 7fce40397640 1 --2- 172.21.15.84:0/2606385733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce380ce390 0x7fce381f0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.531+0000 7fce40397640 1 -- 172.21.15.84:0/2606385733 >> 172.21.15.84:0/2606385733 conn(0x7fce380cb410 msgr2=0x7fce3806ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:38.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.531+0000 7fce40397640 1 -- 172.21.15.84:0/2606385733 shutdown_connections 2023-10-16T11:22:38.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.531+0000 7fce40397640 1 -- 172.21.15.84:0/2606385733 wait complete. 2023-10-16T11:22:38.568 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:22:38.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.784+0000 7fc61dc7e640 1 Processor -- start 2023-10-16T11:22:38.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.784+0000 7fc61dc7e640 1 -- start start 2023-10-16T11:22:38.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.786+0000 7fc61dc7e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6180b8480 0x7fc6180cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:38.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.786+0000 7fc61dc7e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc6180d0be0 0x7fc6180cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:38.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.786+0000 7fc61dc7e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc61815d150 con 0x7fc61815cd70 2023-10-16T11:22:38.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.786+0000 7fc61dc7e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6180d1180 con 0x7fc6180d0be0 2023-10-16T11:22:38.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.786+0000 7fc61dc7e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6180d1460 con 0x7fc6180b8480 2023-10-16T11:22:38.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.786+0000 7fc61d47d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc6180d0be0 0x7fc6180cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:38.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.786+0000 7fc61d47d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc6180d0be0 0x7fc6180cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48312/0 (socket says 172.21.15.84:48312) 2023-10-16T11:22:38.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.787+0000 7fc61d47d640 1 -- 172.21.15.84:0/2702365764 learned_addr learned my addr 172.21.15.84:0/2702365764 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:38.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.787+0000 7fc617fff640 1 --2- 172.21.15.84:0/2702365764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6180b8480 0x7fc6180cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:38.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.787+0000 7fc61d47d640 1 -- 172.21.15.84:0/2702365764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6180b8480 msgr2=0x7fc6180cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:38.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.787+0000 7fc61d47d640 1 --2- 172.21.15.84:0/2702365764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6180b8480 0x7fc6180cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.787+0000 7fc61d47d640 1 -- 172.21.15.84:0/2702365764 >> v1:172.21.15.157:6789/0 conn(0x7fc61815cd70 legacy=0x7fc6180d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:38.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.787+0000 7fc61d47d640 1 -- 172.21.15.84:0/2702365764 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6180d1740 con 0x7fc6180d0be0 2023-10-16T11:22:38.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.787+0000 7fc617fff640 1 --2- 172.21.15.84:0/2702365764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6180b8480 0x7fc6180cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:38.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.788+0000 7fc61d47d640 1 --2- 172.21.15.84:0/2702365764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6180d0be0 0x7fc6180cdd00 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fc610009e50 tx=0x7fc61002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4a3ba86a01122 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:38.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.788+0000 7fc6177fe640 1 -- 172.21.15.84:0/2702365764 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61002ca80 con 0x7fc6180d0be0 2023-10-16T11:22:38.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.788+0000 7fc6177fe640 1 -- 172.21.15.84:0/2702365764 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61002cbe0 con 0x7fc6180d0be0 2023-10-16T11:22:38.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.788+0000 7fc6177fe640 1 -- 172.21.15.84:0/2702365764 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6100343f0 con 0x7fc6180d0be0 2023-10-16T11:22:38.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.789+0000 7fc61dc7e640 1 -- 172.21.15.84:0/2702365764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6180d0be0 msgr2=0x7fc6180cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:38.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.789+0000 7fc61dc7e640 1 --2- 172.21.15.84:0/2702365764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6180d0be0 0x7fc6180cdd00 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fc610009e50 tx=0x7fc61002bee0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.789+0000 7fc61dc7e640 1 -- 172.21.15.84:0/2702365764 shutdown_connections 2023-10-16T11:22:38.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.789+0000 7fc61dc7e640 1 --2- 172.21.15.84:0/2702365764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6180d0be0 0x7fc6180cdd00 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.789+0000 7fc61dc7e640 1 --2- 172.21.15.84:0/2702365764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6180b8480 0x7fc6180cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.789+0000 7fc61dc7e640 1 -- 172.21.15.84:0/2702365764 >> 172.21.15.84:0/2702365764 conn(0x7fc6180cf570 msgr2=0x7fc6180cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:38.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.789+0000 7fc61dc7e640 1 -- 172.21.15.84:0/2702365764 shutdown_connections 2023-10-16T11:22:38.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.789+0000 7fc61dc7e640 1 -- 172.21.15.84:0/2702365764 wait complete. 2023-10-16T11:22:38.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.791+0000 7fc61dc7e640 1 Processor -- start 2023-10-16T11:22:38.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.791+0000 7fc61dc7e640 1 -- start start 2023-10-16T11:22:38.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.792+0000 7fc61dc7e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6180b8480 0x7fc6180c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:38.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.792+0000 7fc61dc7e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6180d0be0 0x7fc61815dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:38.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.792+0000 7fc61dc7e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc61815cd70 0x7fc6180c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:38.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.792+0000 7fc61dc7e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6180cbb70 con 0x7fc61815cd70 2023-10-16T11:22:38.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.792+0000 7fc61dc7e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6180cbda0 con 0x7fc6180d0be0 2023-10-16T11:22:38.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.792+0000 7fc61dc7e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6180cc080 con 0x7fc6180b8480 2023-10-16T11:22:38.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.792+0000 7fc61d47d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc61815cd70 0x7fc6180c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:38.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.792+0000 7fc61d47d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc61815cd70 0x7fc6180c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48316/0 (socket says 172.21.15.84:48316) 2023-10-16T11:22:38.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.792+0000 7fc61d47d640 1 -- 172.21.15.84:0/2759165343 learned_addr learned my addr 172.21.15.84:0/2759165343 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:38.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.793+0000 7fc61d47d640 1 -- 172.21.15.84:0/2759165343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6180b8480 msgr2=0x7fc6180c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:38.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.793+0000 7fc61d47d640 1 --2- 172.21.15.84:0/2759165343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6180b8480 0x7fc6180c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.793+0000 7fc61d47d640 1 -- 172.21.15.84:0/2759165343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6180d0be0 msgr2=0x7fc61815dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:38.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.793+0000 7fc61d47d640 1 --2- 172.21.15.84:0/2759165343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6180d0be0 0x7fc61815dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:38.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.793+0000 7fc61d47d640 1 -- 172.21.15.84:0/2759165343 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc610009b30 con 0x7fc61815cd70 2023-10-16T11:22:38.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.793+0000 7fc61d47d640 1 --2- 172.21.15.84:0/2759165343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc61815cd70 0x7fc6180c9bc0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7fc610036040 tx=0x7fc610004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:38.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.793+0000 7fc6157fa640 1 -- 172.21.15.84:0/2759165343 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc610009510 con 0x7fc61815cd70 2023-10-16T11:22:38.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.793+0000 7fc6157fa640 1 -- 172.21.15.84:0/2759165343 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61003e450 con 0x7fc61815cd70 2023-10-16T11:22:38.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.793+0000 7fc61dc7e640 1 -- 172.21.15.84:0/2759165343 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6180cc360 con 0x7fc61815cd70 2023-10-16T11:22:38.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.793+0000 7fc6157fa640 1 -- 172.21.15.84:0/2759165343 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61003e800 con 0x7fc61815cd70 2023-10-16T11:22:38.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.793+0000 7fc61dc7e640 1 -- 172.21.15.84:0/2759165343 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6180cc920 con 0x7fc61815cd70 2023-10-16T11:22:38.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.794+0000 7fc61dc7e640 1 -- 172.21.15.84:0/2759165343 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc61804e9b0 con 0x7fc61815cd70 2023-10-16T11:22:38.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.795+0000 7fc6157fa640 1 -- 172.21.15.84:0/2759165343 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc61002cd50 con 0x7fc61815cd70 2023-10-16T11:22:38.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.796+0000 7fc6157fa640 1 --2- 172.21.15.84:0/2759165343 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc60406af90 0x7fc60406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:38.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.796+0000 7fc6157fa640 1 -- 172.21.15.84:0/2759165343 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fc6100abfe0 con 0x7fc61815cd70 2023-10-16T11:22:38.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.796+0000 7fc61cc7c640 1 --2- 172.21.15.84:0/2759165343 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc60406af90 0x7fc60406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:38.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.797+0000 7fc61cc7c640 1 --2- 172.21.15.84:0/2759165343 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc60406af90 0x7fc60406d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc61815d5e0 tx=0x7fc60c0079b0 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:38.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.801+0000 7fc6157fa640 1 -- 172.21.15.84:0/2759165343 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc6100772b0 con 0x7fc61815cd70 2023-10-16T11:22:38.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.993+0000 7fc61dc7e640 1 -- 172.21.15.84:0/2759165343 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6180b7280 con 0x7fc61815cd70 2023-10-16T11:22:38.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.995+0000 7fc6157fa640 1 -- 172.21.15.84:0/2759165343 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+12887 (secure 0 0 0) 0x7fc61007a8d0 con 0x7fc61815cd70 2023-10-16T11:22:38.998 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:38.998 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":197,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:22:38.065661+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:22:39.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.998+0000 7fc61dc7e640 1 -- 172.21.15.84:0/2759165343 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc60406af90 msgr2=0x7fc60406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:39.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.998+0000 7fc61dc7e640 1 --2- 172.21.15.84:0/2759165343 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc60406af90 0x7fc60406d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc61815d5e0 tx=0x7fc60c0079b0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.998+0000 7fc61dc7e640 1 -- 172.21.15.84:0/2759165343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc61815cd70 msgr2=0x7fc6180c9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:39.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.998+0000 7fc61dc7e640 1 --2- 172.21.15.84:0/2759165343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc61815cd70 0x7fc6180c9bc0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7fc610036040 tx=0x7fc610004a40 comp rx=0 tx=0).stop 2023-10-16T11:22:39.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.998+0000 7fc61dc7e640 1 -- 172.21.15.84:0/2759165343 shutdown_connections 2023-10-16T11:22:39.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.998+0000 7fc61dc7e640 1 --2- 172.21.15.84:0/2759165343 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc60406af90 0x7fc60406d450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.998+0000 7fc61dc7e640 1 --2- 172.21.15.84:0/2759165343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc61815cd70 0x7fc6180c9bc0 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.998+0000 7fc61dc7e640 1 --2- 172.21.15.84:0/2759165343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6180d0be0 0x7fc61815dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.998+0000 7fc61dc7e640 1 --2- 172.21.15.84:0/2759165343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6180b8480 0x7fc6180c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.998+0000 7fc61dc7e640 1 -- 172.21.15.84:0/2759165343 >> 172.21.15.84:0/2759165343 conn(0x7fc6180cf570 msgr2=0x7fc6180d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:39.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.998+0000 7fc61dc7e640 1 -- 172.21.15.84:0/2759165343 shutdown_connections 2023-10-16T11:22:39.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:38.998+0000 7fc61dc7e640 1 -- 172.21.15.84:0/2759165343 wait complete. 2023-10-16T11:22:39.019 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:22:39.020 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:22:39.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.227+0000 7f8d15076640 1 Processor -- start 2023-10-16T11:22:39.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.227+0000 7f8d15076640 1 -- start start 2023-10-16T11:22:39.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.229+0000 7f8d15076640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8d10159ab0 0x7f8d100ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:39.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.229+0000 7f8d15076640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8d1006c140 0x7f8d1006c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:39.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.229+0000 7f8d15076640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d1006cad0 con 0x7f8d10159e90 2023-10-16T11:22:39.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.229+0000 7f8d15076640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d1015d890 con 0x7f8d1006c140 2023-10-16T11:22:39.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.229+0000 7f8d15076640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d10070360 con 0x7f8d10159ab0 2023-10-16T11:22:39.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.229+0000 7f8d0f59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8d1006c140 0x7f8d1006c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:39.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.229+0000 7f8d0f59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8d1006c140 0x7f8d1006c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48324/0 (socket says 172.21.15.84:48324) 2023-10-16T11:22:39.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.229+0000 7f8d0f59e640 1 -- 172.21.15.84:0/2014857524 learned_addr learned my addr 172.21.15.84:0/2014857524 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:39.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.229+0000 7f8d0ed9d640 1 --2- 172.21.15.84:0/2014857524 >> v2:172.21.15.157:3300/0 conn(0x7f8d10159ab0 0x7f8d100ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:39.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.229+0000 7f8d0f59e640 1 -- 172.21.15.84:0/2014857524 >> v2:172.21.15.157:3300/0 conn(0x7f8d10159ab0 msgr2=0x7f8d100ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:39.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.229+0000 7f8d0f59e640 1 --2- 172.21.15.84:0/2014857524 >> v2:172.21.15.157:3300/0 conn(0x7f8d10159ab0 0x7f8d100ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.229+0000 7f8d0f59e640 1 -- 172.21.15.84:0/2014857524 >> v1:172.21.15.157:6789/0 conn(0x7f8d10159e90 legacy=0x7f8d1006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:39.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.230+0000 7f8d0f59e640 1 -- 172.21.15.84:0/2014857524 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d100cd1f0 con 0x7f8d1006c140 2023-10-16T11:22:39.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.231+0000 7f8d0f59e640 1 --2- 172.21.15.84:0/2014857524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d1006c140 0x7f8d1006c590 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f8d100cfa00 tx=0x7f8d0802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fef5e6d78841908 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:39.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.231+0000 7f8d0dd9b640 1 -- 172.21.15.84:0/2014857524 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d08031750 con 0x7f8d1006c140 2023-10-16T11:22:39.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.231+0000 7f8d0dd9b640 1 -- 172.21.15.84:0/2014857524 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d080318b0 con 0x7f8d1006c140 2023-10-16T11:22:39.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.231+0000 7f8d0dd9b640 1 -- 172.21.15.84:0/2014857524 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d08031c40 con 0x7f8d1006c140 2023-10-16T11:22:39.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.231+0000 7f8d15076640 1 -- 172.21.15.84:0/2014857524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d1006c140 msgr2=0x7f8d1006c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:39.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.231+0000 7f8d15076640 1 --2- 172.21.15.84:0/2014857524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d1006c140 0x7f8d1006c590 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f8d100cfa00 tx=0x7f8d0802b7f0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.232+0000 7f8d15076640 1 -- 172.21.15.84:0/2014857524 shutdown_connections 2023-10-16T11:22:39.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.232+0000 7f8d15076640 1 --2- 172.21.15.84:0/2014857524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d1006c140 0x7f8d1006c590 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.232+0000 7f8d15076640 1 --2- 172.21.15.84:0/2014857524 >> v2:172.21.15.157:3300/0 conn(0x7f8d10159ab0 0x7f8d100ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.232+0000 7f8d15076640 1 -- 172.21.15.84:0/2014857524 >> 172.21.15.84:0/2014857524 conn(0x7f8d100ce9b0 msgr2=0x7f8d100cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:39.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.232+0000 7f8d15076640 1 -- 172.21.15.84:0/2014857524 shutdown_connections 2023-10-16T11:22:39.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.232+0000 7f8d15076640 1 -- 172.21.15.84:0/2014857524 wait complete. 2023-10-16T11:22:39.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.233+0000 7f8d15076640 1 Processor -- start 2023-10-16T11:22:39.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.234+0000 7f8d15076640 1 -- start start 2023-10-16T11:22:39.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.235+0000 7f8d15076640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d1006c140 0x7f8d100c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:39.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.235+0000 7f8d15076640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d10159ab0 0x7f8d101e4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:39.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.235+0000 7f8d15076640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d10159e90 0x7f8d10147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:39.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.235+0000 7f8d15076640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101e5fe0 con 0x7f8d10159e90 2023-10-16T11:22:39.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.235+0000 7f8d15076640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101e53d0 con 0x7f8d10159ab0 2023-10-16T11:22:39.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.235+0000 7f8d15076640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101e56b0 con 0x7f8d1006c140 2023-10-16T11:22:39.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.235+0000 7f8d0f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d10159e90 0x7f8d10147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:39.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.235+0000 7f8d0f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d10159e90 0x7f8d10147d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48338/0 (socket says 172.21.15.84:48338) 2023-10-16T11:22:39.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.235+0000 7f8d0f59e640 1 -- 172.21.15.84:0/2288090060 learned_addr learned my addr 172.21.15.84:0/2288090060 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:39.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.235+0000 7f8d0ed9d640 1 --2- 172.21.15.84:0/2288090060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d1006c140 0x7f8d100c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:39.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.235+0000 7f8d0e59c640 1 --2- 172.21.15.84:0/2288090060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d10159ab0 0x7f8d101e4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:39.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.235+0000 7f8d0f59e640 1 -- 172.21.15.84:0/2288090060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d1006c140 msgr2=0x7f8d100c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:39.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.235+0000 7f8d0f59e640 1 --2- 172.21.15.84:0/2288090060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d1006c140 0x7f8d100c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.235+0000 7f8d0f59e640 1 -- 172.21.15.84:0/2288090060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d10159ab0 msgr2=0x7f8d101e4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:39.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.235+0000 7f8d0f59e640 1 --2- 172.21.15.84:0/2288090060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d10159ab0 0x7f8d101e4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.235+0000 7f8d0f59e640 1 -- 172.21.15.84:0/2288090060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d080094d0 con 0x7f8d10159e90 2023-10-16T11:22:39.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.236+0000 7f8d0f59e640 1 --2- 172.21.15.84:0/2288090060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d10159e90 0x7f8d10147d50 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f8d0802b830 tx=0x7f8d080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:39.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.236+0000 7f8d077fe640 1 -- 172.21.15.84:0/2288090060 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d08033c80 con 0x7f8d10159e90 2023-10-16T11:22:39.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.236+0000 7f8d077fe640 1 -- 172.21.15.84:0/2288090060 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d08033de0 con 0x7f8d10159e90 2023-10-16T11:22:39.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.236+0000 7f8d077fe640 1 -- 172.21.15.84:0/2288090060 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0803c640 con 0x7f8d10159e90 2023-10-16T11:22:39.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.236+0000 7f8d15076640 1 -- 172.21.15.84:0/2288090060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d101e5990 con 0x7f8d10159e90 2023-10-16T11:22:39.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.236+0000 7f8d15076640 1 -- 172.21.15.84:0/2288090060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d101e50c0 con 0x7f8d10159e90 2023-10-16T11:22:39.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.238+0000 7f8d15076640 1 -- 172.21.15.84:0/2288090060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d101e6190 con 0x7f8d10159e90 2023-10-16T11:22:39.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.238+0000 7f8d077fe640 1 -- 172.21.15.84:0/2288090060 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8d0803a040 con 0x7f8d10159e90 2023-10-16T11:22:39.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.239+0000 7f8d077fe640 1 --2- 172.21.15.84:0/2288090060 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f8cf006afc0 0x7f8cf006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:39.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.239+0000 7f8d077fe640 1 -- 172.21.15.84:0/2288090060 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f8d080aaf70 con 0x7f8d10159e90 2023-10-16T11:22:39.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.240+0000 7f8d0ed9d640 1 --2- 172.21.15.84:0/2288090060 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f8cf006afc0 0x7f8cf006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:39.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.240+0000 7f8d0ed9d640 1 --2- 172.21.15.84:0/2288090060 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f8cf006afc0 0x7f8cf006d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8d000070b0 tx=0x7f8d00007040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:39.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.244+0000 7f8d077fe640 1 -- 172.21.15.84:0/2288090060 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8d080ddfe0 con 0x7f8d10159e90 2023-10-16T11:22:39.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.434+0000 7f8d15076640 1 -- 172.21.15.84:0/2288090060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d1006cad0 con 0x7f8d10159e90 2023-10-16T11:22:39.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.435+0000 7f8d077fe640 1 -- 172.21.15.84:0/2288090060 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v197) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d1006cad0 con 0x7f8d10159e90 2023-10-16T11:22:39.438 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:22:39.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.438+0000 7f8d15076640 1 -- 172.21.15.84:0/2288090060 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f8cf006afc0 msgr2=0x7f8cf006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:39.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.438+0000 7f8d15076640 1 --2- 172.21.15.84:0/2288090060 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f8cf006afc0 0x7f8cf006d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8d000070b0 tx=0x7f8d00007040 comp rx=0 tx=0).stop 2023-10-16T11:22:39.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.438+0000 7f8d15076640 1 -- 172.21.15.84:0/2288090060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d10159e90 msgr2=0x7f8d10147d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:39.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.438+0000 7f8d15076640 1 --2- 172.21.15.84:0/2288090060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d10159e90 0x7f8d10147d50 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f8d0802b830 tx=0x7f8d080046c0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.438+0000 7f8d15076640 1 -- 172.21.15.84:0/2288090060 shutdown_connections 2023-10-16T11:22:39.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.438+0000 7f8d15076640 1 --2- 172.21.15.84:0/2288090060 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f8cf006afc0 0x7f8cf006d480 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.438+0000 7f8d15076640 1 --2- 172.21.15.84:0/2288090060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d10159e90 0x7f8d10147d50 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.438+0000 7f8d15076640 1 --2- 172.21.15.84:0/2288090060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d10159ab0 0x7f8d101e4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.438+0000 7f8d15076640 1 --2- 172.21.15.84:0/2288090060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d1006c140 0x7f8d100c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.438+0000 7f8d15076640 1 -- 172.21.15.84:0/2288090060 >> 172.21.15.84:0/2288090060 conn(0x7f8d100ce9b0 msgr2=0x7f8d100af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.438+0000 7f8d15076640 1 -- 172.21.15.84:0/2288090060 shutdown_connections 2023-10-16T11:22:39.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.438+0000 7f8d15076640 1 -- 172.21.15.84:0/2288090060 wait complete. 2023-10-16T11:22:39.459 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T11:22:39.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.665+0000 7f14ddf4e640 1 Processor -- start 2023-10-16T11:22:39.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.666+0000 7f14ddf4e640 1 -- start start 2023-10-16T11:22:39.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.667+0000 7f14ddf4e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d80b1d60 0x7f14d80b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:39.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.667+0000 7f14ddf4e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f14d80b4ad0 0x7f14d80d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:39.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.667+0000 7f14ddf4e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d80b4f20 con 0x7f14d815ce10 2023-10-16T11:22:39.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.667+0000 7f14ddf4e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d80b5c40 con 0x7f14d80b4ad0 2023-10-16T11:22:39.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.667+0000 7f14ddf4e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d80cdb50 con 0x7f14d80b1d60 2023-10-16T11:22:39.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.668+0000 7f14dcf4c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f14d815ce10 0x7f14d8161430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:37178/0 (socket says 172.21.15.84:37178) 2023-10-16T11:22:39.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.668+0000 7f14dcf4c640 1 -- 172.21.15.84:0/536756349 learned_addr learned my addr 172.21.15.84:0/536756349 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:39.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.668+0000 7f14dd74d640 1 --2- 172.21.15.84:0/536756349 >> v2:172.21.15.157:3300/0 conn(0x7f14d80b4ad0 0x7f14d80d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:39.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.668+0000 7f14d7fff640 1 --2- 172.21.15.84:0/536756349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d80b1d60 0x7f14d80b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:39.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.669+0000 7f14d77fe640 1 -- 172.21.15.84:0/536756349 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748039039 0 0) 0x7f14c8001490 con 0x7f14d815ce10 2023-10-16T11:22:39.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.669+0000 7f14d77fe640 1 -- 172.21.15.84:0/536756349 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14c0003520 con 0x7f14d815ce10 2023-10-16T11:22:39.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.669+0000 7f14dd74d640 1 -- 172.21.15.84:0/536756349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d80b1d60 msgr2=0x7f14d80b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:39.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.669+0000 7f14dd74d640 1 --2- 172.21.15.84:0/536756349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d80b1d60 0x7f14d80b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.669+0000 7f14dd74d640 1 -- 172.21.15.84:0/536756349 >> v1:172.21.15.157:6789/0 conn(0x7f14d815ce10 legacy=0x7f14d8161430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:39.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.669+0000 7f14dd74d640 1 -- 172.21.15.84:0/536756349 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d80cddf0 con 0x7f14d80b4ad0 2023-10-16T11:22:39.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.670+0000 7f14dd74d640 1 --2- 172.21.15.84:0/536756349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14d80b4ad0 0x7f14d80d1740 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f14d00097a0 tx=0x7f14d002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63cfe521bc5a0ecf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:39.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.671+0000 7f14d77fe640 1 -- 172.21.15.84:0/536756349 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0031830 con 0x7f14d80b4ad0 2023-10-16T11:22:39.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.671+0000 7f14d77fe640 1 -- 172.21.15.84:0/536756349 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d0031990 con 0x7f14d80b4ad0 2023-10-16T11:22:39.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.671+0000 7f14d77fe640 1 -- 172.21.15.84:0/536756349 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0031d00 con 0x7f14d80b4ad0 2023-10-16T11:22:39.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.671+0000 7f14ddf4e640 1 -- 172.21.15.84:0/536756349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14d80b4ad0 msgr2=0x7f14d80d1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:39.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.671+0000 7f14ddf4e640 1 --2- 172.21.15.84:0/536756349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14d80b4ad0 0x7f14d80d1740 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f14d00097a0 tx=0x7f14d002b8d0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.672+0000 7f14ddf4e640 1 -- 172.21.15.84:0/536756349 shutdown_connections 2023-10-16T11:22:39.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.672+0000 7f14ddf4e640 1 --2- 172.21.15.84:0/536756349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14d80b4ad0 0x7f14d80d1740 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.672+0000 7f14ddf4e640 1 --2- 172.21.15.84:0/536756349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d80b1d60 0x7f14d80b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.672+0000 7f14ddf4e640 1 -- 172.21.15.84:0/536756349 >> 172.21.15.84:0/536756349 conn(0x7f14d80cf590 msgr2=0x7f14d80cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:39.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.672+0000 7f14ddf4e640 1 -- 172.21.15.84:0/536756349 shutdown_connections 2023-10-16T11:22:39.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.672+0000 7f14ddf4e640 1 -- 172.21.15.84:0/536756349 wait complete. 2023-10-16T11:22:39.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.673+0000 7f14ddf4e640 1 Processor -- start 2023-10-16T11:22:39.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.674+0000 7f14ddf4e640 1 -- start start 2023-10-16T11:22:39.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.674+0000 7f14ddf4e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14d80b1d60 0x7f14d81f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:39.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.675+0000 7f14ddf4e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d80b4ad0 0x7f14d80c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:39.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.675+0000 7f14ddf4e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d815ce10 0x7f14d815b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:39.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.675+0000 7f14ddf4e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d81f6f80 con 0x7f14d815ce10 2023-10-16T11:22:39.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.675+0000 7f14ddf4e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d8155a90 con 0x7f14d80b1d60 2023-10-16T11:22:39.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.675+0000 7f14ddf4e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d81f6340 con 0x7f14d80b4ad0 2023-10-16T11:22:39.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.675+0000 7f14dd74d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d815ce10 0x7f14d815b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:39.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.675+0000 7f14dd74d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d815ce10 0x7f14d815b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48354/0 (socket says 172.21.15.84:48354) 2023-10-16T11:22:39.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.675+0000 7f14dd74d640 1 -- 172.21.15.84:0/3730474265 learned_addr learned my addr 172.21.15.84:0/3730474265 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:39.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.675+0000 7f14dcf4c640 1 --2- 172.21.15.84:0/3730474265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14d80b1d60 0x7f14d81f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:39.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.675+0000 7f14d7fff640 1 --2- 172.21.15.84:0/3730474265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d80b4ad0 0x7f14d80c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:39.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.675+0000 7f14dd74d640 1 -- 172.21.15.84:0/3730474265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d80b4ad0 msgr2=0x7f14d80c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:39.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.675+0000 7f14dd74d640 1 --2- 172.21.15.84:0/3730474265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d80b4ad0 0x7f14d80c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.675+0000 7f14dd74d640 1 -- 172.21.15.84:0/3730474265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14d80b1d60 msgr2=0x7f14d81f35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:39.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.675+0000 7f14dd74d640 1 --2- 172.21.15.84:0/3730474265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14d80b1d60 0x7f14d81f35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.675+0000 7f14dd74d640 1 -- 172.21.15.84:0/3730474265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d0009400 con 0x7f14d815ce10 2023-10-16T11:22:39.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.676+0000 7f14dd74d640 1 --2- 172.21.15.84:0/3730474265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d815ce10 0x7f14d815b720 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f14d0034040 tx=0x7f14d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:39.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.676+0000 7f14d57fa640 1 -- 172.21.15.84:0/3730474265 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d003a830 con 0x7f14d815ce10 2023-10-16T11:22:39.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.676+0000 7f14d57fa640 1 -- 172.21.15.84:0/3730474265 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d003a990 con 0x7f14d815ce10 2023-10-16T11:22:39.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.676+0000 7f14d57fa640 1 -- 172.21.15.84:0/3730474265 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d003ad40 con 0x7f14d815ce10 2023-10-16T11:22:39.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.676+0000 7f14ddf4e640 1 -- 172.21.15.84:0/3730474265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d81f65e0 con 0x7f14d815ce10 2023-10-16T11:22:39.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.676+0000 7f14ddf4e640 1 -- 172.21.15.84:0/3730474265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d81f6ba0 con 0x7f14d815ce10 2023-10-16T11:22:39.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.677+0000 7f14ddf4e640 1 -- 172.21.15.84:0/3730474265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d81f70c0 con 0x7f14d815ce10 2023-10-16T11:22:39.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.678+0000 7f14d57fa640 1 -- 172.21.15.84:0/3730474265 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f14d004d440 con 0x7f14d815ce10 2023-10-16T11:22:39.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.680+0000 7f14d57fa640 1 --2- 172.21.15.84:0/3730474265 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f14c406af90 0x7f14c406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:39.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.680+0000 7f14d57fa640 1 -- 172.21.15.84:0/3730474265 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f14d00ab5e0 con 0x7f14d815ce10 2023-10-16T11:22:39.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.680+0000 7f14dcf4c640 1 --2- 172.21.15.84:0/3730474265 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f14c406af90 0x7f14c406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:39.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.681+0000 7f14dcf4c640 1 --2- 172.21.15.84:0/3730474265 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f14c406af90 0x7f14c406d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f14c80015f0 tx=0x7f14c8009040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:39.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.684+0000 7f14d57fa640 1 -- 172.21.15.84:0/3730474265 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f14d81f70c0 con 0x7f14d815ce10 2023-10-16T11:22:39.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.879+0000 7f14ddf4e640 1 -- 172.21.15.84:0/3730474265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f14d81f70c0 con 0x7f14d815ce10 2023-10-16T11:22:39.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.879+0000 7f14d57fa640 1 -- 172.21.15.84:0/3730474265 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v197) v1 ==== 84+0+65 (secure 0 0 0) 0x7f14d81f70c0 con 0x7f14d815ce10 2023-10-16T11:22:39.882 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:39.882 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T11:22:39.882 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:39.882 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-16T11:22:39.882 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-16T11:22:39.883 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:39.883 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T11:22:39.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.882+0000 7f14ddf4e640 1 -- 172.21.15.84:0/3730474265 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f14c406af90 msgr2=0x7f14c406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:39.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.882+0000 7f14ddf4e640 1 --2- 172.21.15.84:0/3730474265 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f14c406af90 0x7f14c406d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f14c80015f0 tx=0x7f14c8009040 comp rx=0 tx=0).stop 2023-10-16T11:22:39.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.882+0000 7f14ddf4e640 1 -- 172.21.15.84:0/3730474265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d815ce10 msgr2=0x7f14d815b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:39.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.882+0000 7f14ddf4e640 1 --2- 172.21.15.84:0/3730474265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d815ce10 0x7f14d815b720 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f14d0034040 tx=0x7f14d00046c0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.882+0000 7f14ddf4e640 1 -- 172.21.15.84:0/3730474265 shutdown_connections 2023-10-16T11:22:39.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.882+0000 7f14ddf4e640 1 --2- 172.21.15.84:0/3730474265 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f14c406af90 0x7f14c406d450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.882+0000 7f14ddf4e640 1 --2- 172.21.15.84:0/3730474265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14d815ce10 0x7f14d815b720 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.882+0000 7f14ddf4e640 1 --2- 172.21.15.84:0/3730474265 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14d80b4ad0 0x7f14d80c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.882+0000 7f14ddf4e640 1 --2- 172.21.15.84:0/3730474265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14d80b1d60 0x7f14d81f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:39.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.882+0000 7f14ddf4e640 1 -- 172.21.15.84:0/3730474265 >> 172.21.15.84:0/3730474265 conn(0x7f14d80cf590 msgr2=0x7f14d80b7230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:39.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.882+0000 7f14ddf4e640 1 -- 172.21.15.84:0/3730474265 shutdown_connections 2023-10-16T11:22:39.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:39.882+0000 7f14ddf4e640 1 -- 172.21.15.84:0/3730474265 wait complete. 2023-10-16T11:22:39.902 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T11:22:39.902 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T11:22:40.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.116+0000 7f152b9a9640 1 Processor -- start 2023-10-16T11:22:40.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.117+0000 7f152b9a9640 1 -- start start 2023-10-16T11:22:40.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.118+0000 7f152b9a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f152415cf50 0x7f15240d2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:40.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.118+0000 7f152b9a9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f15240b8a10 0x7f15240d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:40.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.118+0000 7f152b9a9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f152415d330 con 0x7f15240b4750 2023-10-16T11:22:40.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.118+0000 7f152b9a9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15240d1680 con 0x7f15240b8a10 2023-10-16T11:22:40.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.118+0000 7f152b9a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15240d1960 con 0x7f152415cf50 2023-10-16T11:22:40.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.118+0000 7f1529f46640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f15240b4750 0x7f152406bd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58828/0 (socket says 172.21.15.84:58828) 2023-10-16T11:22:40.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.118+0000 7f1529f46640 1 -- 172.21.15.84:0/3747484508 learned_addr learned my addr 172.21.15.84:0/3747484508 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:40.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.118+0000 7f1528f44640 1 --2- 172.21.15.84:0/3747484508 >> v2:172.21.15.157:3300/0 conn(0x7f15240b8a10 0x7f15240d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.118+0000 7f1529745640 1 --2- 172.21.15.84:0/3747484508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f152415cf50 0x7f15240d2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.118+0000 7f151bfff640 1 -- 172.21.15.84:0/3747484508 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 170406590 0 0) 0x7f152415d330 con 0x7f15240b4750 2023-10-16T11:22:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.119+0000 7f151bfff640 1 -- 172.21.15.84:0/3747484508 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1510003520 con 0x7f15240b4750 2023-10-16T11:22:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.119+0000 7f151bfff640 1 -- 172.21.15.84:0/3747484508 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1074532945 0 0) 0x7f1510003520 con 0x7f15240b4750 2023-10-16T11:22:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.119+0000 7f151bfff640 1 -- 172.21.15.84:0/3747484508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f152415cf50 msgr2=0x7f15240d2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.119+0000 7f151bfff640 1 --2- 172.21.15.84:0/3747484508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f152415cf50 0x7f15240d2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.119+0000 7f151bfff640 1 -- 172.21.15.84:0/3747484508 >> v2:172.21.15.157:3300/0 conn(0x7f15240b8a10 msgr2=0x7f15240d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.119+0000 7f151bfff640 1 --2- 172.21.15.84:0/3747484508 >> v2:172.21.15.157:3300/0 conn(0x7f15240b8a10 0x7f15240d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.119+0000 7f151bfff640 1 -- 172.21.15.84:0/3747484508 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15240d1c40 con 0x7f15240b4750 2023-10-16T11:22:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.119+0000 7f151bfff640 1 -- 172.21.15.84:0/3747484508 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f151c0019f0 con 0x7f15240b4750 2023-10-16T11:22:40.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.119+0000 7f1529745640 1 --2- 172.21.15.84:0/3747484508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f152415cf50 0x7f15240d2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:40.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.119+0000 7f151bfff640 1 -- 172.21.15.84:0/3747484508 >> v1:172.21.15.84:6789/0 conn(0x7f15240b4750 legacy=0x7f152406bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:40.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.119+0000 7f151bfff640 1 --2- 172.21.15.84:0/3747484508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15100039e0 0x7f1510003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:40.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.119+0000 7f151bfff640 1 --2- 172.21.15.84:0/3747484508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15100047c0 0x7f1510004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:40.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.119+0000 7f151bfff640 1 --2- 172.21.15.84:0/3747484508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f152415cf50 0x7f1510005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:40.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.119+0000 7f151bfff640 1 -- 172.21.15.84:0/3747484508 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15240d1680 con 0x7f15100039e0 2023-10-16T11:22:40.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.119+0000 7f151bfff640 1 -- 172.21.15.84:0/3747484508 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15240d1960 con 0x7f15100047c0 2023-10-16T11:22:40.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.119+0000 7f151bfff640 1 -- 172.21.15.84:0/3747484508 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151000bab0 con 0x7f152415cf50 2023-10-16T11:22:40.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.119+0000 7f1529745640 1 --2- 172.21.15.84:0/3747484508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15100039e0 0x7f1510003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:40.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.120+0000 7f1529745640 1 -- 172.21.15.84:0/3747484508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f152415cf50 msgr2=0x7f1510005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:40.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.120+0000 7f1529745640 1 --2- 172.21.15.84:0/3747484508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f152415cf50 0x7f1510005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:40.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.120+0000 7f1529745640 1 -- 172.21.15.84:0/3747484508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15100047c0 msgr2=0x7f1510004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:40.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.120+0000 7f1529745640 1 --2- 172.21.15.84:0/3747484508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15100047c0 0x7f1510004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:40.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.120+0000 7f1529745640 1 -- 172.21.15.84:0/3747484508 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1510003520 con 0x7f15100039e0 2023-10-16T11:22:40.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.121+0000 7f1529745640 1 --2- 172.21.15.84:0/3747484508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15100039e0 0x7f1510003e00 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f1520006fd0 tx=0x7f15100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e704007fd5c097 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:40.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.121+0000 7f151bfff640 1 -- 172.21.15.84:0/3747484508 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520004230 con 0x7f15100039e0 2023-10-16T11:22:40.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.121+0000 7f151bfff640 1 -- 172.21.15.84:0/3747484508 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1520004390 con 0x7f15100039e0 2023-10-16T11:22:40.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.121+0000 7f151bfff640 1 -- 172.21.15.84:0/3747484508 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520003830 con 0x7f15100039e0 2023-10-16T11:22:40.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.121+0000 7f152b9a9640 1 -- 172.21.15.84:0/3747484508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15100039e0 msgr2=0x7f1510003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:40.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.121+0000 7f152b9a9640 1 --2- 172.21.15.84:0/3747484508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15100039e0 0x7f1510003e00 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f1520006fd0 tx=0x7f15100057f0 comp rx=0 tx=0).stop 2023-10-16T11:22:40.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.121+0000 7f1529f46640 1 -- 172.21.15.84:0/3747484508 reap_dead start 2023-10-16T11:22:40.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.121+0000 7f152b9a9640 1 -- 172.21.15.84:0/3747484508 shutdown_connections 2023-10-16T11:22:40.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.121+0000 7f152b9a9640 1 -- 172.21.15.84:0/3747484508 >> 172.21.15.84:0/3747484508 conn(0x7f15240cf750 msgr2=0x7f15240cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:40.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.122+0000 7f152b9a9640 1 -- 172.21.15.84:0/3747484508 shutdown_connections 2023-10-16T11:22:40.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.122+0000 7f152b9a9640 1 -- 172.21.15.84:0/3747484508 wait complete. 2023-10-16T11:22:40.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.123+0000 7f152b9a9640 1 Processor -- start 2023-10-16T11:22:40.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.123+0000 7f152b9a9640 1 -- start start 2023-10-16T11:22:40.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.124+0000 7f152b9a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15100039e0 0x7f15241f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:40.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.124+0000 7f152b9a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f152415ef40 0x7f15240c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:40.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.124+0000 7f152b9a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15240cac20 0x7f15240cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:40.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.124+0000 7f152b9a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152415ffa0 con 0x7f152415ef40 2023-10-16T11:22:40.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.124+0000 7f152b9a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15240cb610 con 0x7f15240cac20 2023-10-16T11:22:40.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.124+0000 7f152b9a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152414d940 con 0x7f15100039e0 2023-10-16T11:22:40.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.124+0000 7f1528f44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f152415ef40 0x7f15240c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:40.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.125+0000 7f1528f44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f152415ef40 0x7f15240c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48362/0 (socket says 172.21.15.84:48362) 2023-10-16T11:22:40.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.125+0000 7f1528f44640 1 -- 172.21.15.84:0/3863319475 learned_addr learned my addr 172.21.15.84:0/3863319475 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.125+0000 7f1528f44640 1 -- 172.21.15.84:0/3863319475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15100039e0 msgr2=0x7f15241f5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.125+0000 7f1528f44640 1 --2- 172.21.15.84:0/3863319475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15100039e0 0x7f15241f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.125+0000 7f1528f44640 1 -- 172.21.15.84:0/3863319475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15240cac20 msgr2=0x7f15240cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.125+0000 7f1528f44640 1 --2- 172.21.15.84:0/3863319475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15240cac20 0x7f15240cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.125+0000 7f1528f44640 1 -- 172.21.15.84:0/3863319475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152415f420 con 0x7f152415ef40 2023-10-16T11:22:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.125+0000 7f1528f44640 1 --2- 172.21.15.84:0/3863319475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f152415ef40 0x7f15240c9da0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f151400d830 tx=0x7f151400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.125+0000 7f1519ffb640 1 -- 172.21.15.84:0/3863319475 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151400b660 con 0x7f152415ef40 2023-10-16T11:22:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.125+0000 7f1519ffb640 1 -- 172.21.15.84:0/3863319475 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151400b7c0 con 0x7f152415ef40 2023-10-16T11:22:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.125+0000 7f152b9a9640 1 -- 172.21.15.84:0/3863319475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152415f700 con 0x7f152415ef40 2023-10-16T11:22:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.125+0000 7f1519ffb640 1 -- 172.21.15.84:0/3863319475 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151400bb70 con 0x7f152415ef40 2023-10-16T11:22:40.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.126+0000 7f152b9a9640 1 -- 172.21.15.84:0/3863319475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152415fcc0 con 0x7f152415ef40 2023-10-16T11:22:40.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.127+0000 7f152b9a9640 1 -- 172.21.15.84:0/3863319475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1524072350 con 0x7f152415ef40 2023-10-16T11:22:40.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.127+0000 7f1519ffb640 1 -- 172.21.15.84:0/3863319475 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f151401e460 con 0x7f152415ef40 2023-10-16T11:22:40.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.129+0000 7f1519ffb640 1 --2- 172.21.15.84:0/3863319475 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f150806afc0 0x7f150806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:40.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.129+0000 7f1519ffb640 1 -- 172.21.15.84:0/3863319475 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f1514089b00 con 0x7f152415ef40 2023-10-16T11:22:40.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.129+0000 7f1529745640 1 --2- 172.21.15.84:0/3863319475 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f150806afc0 0x7f150806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:40.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.130+0000 7f1529745640 1 --2- 172.21.15.84:0/3863319475 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f150806afc0 0x7f150806d480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1520035000 tx=0x7f1520030000 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:40.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.134+0000 7f1519ffb640 1 -- 172.21.15.84:0/3863319475 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f151408be60 con 0x7f152415ef40 2023-10-16T11:22:40.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:40.323+0000 7f152b9a9640 1 -- 172.21.15.84:0/3863319475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f152404e160 con 0x7f152415ef40 2023-10-16T11:22:41.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.084+0000 7f1519ffb640 1 -- 172.21.15.84:0/3863319475 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v198) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1514054d30 con 0x7f152415ef40 2023-10-16T11:22:41.087 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-16T11:22:41.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.088+0000 7f152b9a9640 1 -- 172.21.15.84:0/3863319475 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f150806afc0 msgr2=0x7f150806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:41.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.088+0000 7f152b9a9640 1 --2- 172.21.15.84:0/3863319475 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f150806afc0 0x7f150806d480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1520035000 tx=0x7f1520030000 comp rx=0 tx=0).stop 2023-10-16T11:22:41.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.089+0000 7f152b9a9640 1 -- 172.21.15.84:0/3863319475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f152415ef40 msgr2=0x7f15240c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:41.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.089+0000 7f152b9a9640 1 --2- 172.21.15.84:0/3863319475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f152415ef40 0x7f15240c9da0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f151400d830 tx=0x7f151400dcf0 comp rx=0 tx=0).stop 2023-10-16T11:22:41.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.089+0000 7f152b9a9640 1 -- 172.21.15.84:0/3863319475 shutdown_connections 2023-10-16T11:22:41.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.089+0000 7f152b9a9640 1 --2- 172.21.15.84:0/3863319475 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f150806afc0 0x7f150806d480 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:41.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.089+0000 7f152b9a9640 1 --2- 172.21.15.84:0/3863319475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15240cac20 0x7f15240cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:41.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.089+0000 7f152b9a9640 1 --2- 172.21.15.84:0/3863319475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f152415ef40 0x7f15240c9da0 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:41.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.089+0000 7f152b9a9640 1 --2- 172.21.15.84:0/3863319475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15100039e0 0x7f15241f5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:41.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.089+0000 7f152b9a9640 1 -- 172.21.15.84:0/3863319475 >> 172.21.15.84:0/3863319475 conn(0x7f15240cf750 msgr2=0x7f15240cd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:41.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.089+0000 7f152b9a9640 1 -- 172.21.15.84:0/3863319475 shutdown_connections 2023-10-16T11:22:41.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.089+0000 7f152b9a9640 1 -- 172.21.15.84:0/3863319475 wait complete. 2023-10-16T11:22:41.115 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T11:22:41.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.320+0000 7f2419c0e640 1 Processor -- start 2023-10-16T11:22:41.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.321+0000 7f2419c0e640 1 -- start start 2023-10-16T11:22:41.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.322+0000 7f2419c0e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2414160420 0x7f24140ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:41.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.322+0000 7f2419c0e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f241406e170 con 0x7f241406b4c0 2023-10-16T11:22:41.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.322+0000 7f2419c0e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24140cd460 con 0x7f2414160800 2023-10-16T11:22:41.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.322+0000 7f2419c0e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241406e3c0 con 0x7f2414160420 2023-10-16T11:22:41.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.322+0000 7f2413fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f241406b4c0 0x7f2414070cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58830/0 (socket says 172.21.15.84:58830) 2023-10-16T11:22:41.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.322+0000 7f2413fff640 1 -- 172.21.15.84:0/3845378908 learned_addr learned my addr 172.21.15.84:0/3845378908 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:41.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.323+0000 7f24137fe640 1 --2- 172.21.15.84:0/3845378908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2414160420 0x7f24140ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:41.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.323+0000 7f24127fc640 1 -- 172.21.15.84:0/3845378908 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841257061 0 0) 0x7f241406e170 con 0x7f241406b4c0 2023-10-16T11:22:41.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.323+0000 7f24127fc640 1 -- 172.21.15.84:0/3845378908 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2400003520 con 0x7f241406b4c0 2023-10-16T11:22:41.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.323+0000 7f24127fc640 1 -- 172.21.15.84:0/3845378908 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1264152563 0 0) 0x7f2400003520 con 0x7f241406b4c0 2023-10-16T11:22:41.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.323+0000 7f24127fc640 1 -- 172.21.15.84:0/3845378908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2414160420 msgr2=0x7f24140ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:41.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.323+0000 7f24127fc640 1 --2- 172.21.15.84:0/3845378908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2414160420 0x7f24140ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:41.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.323+0000 7f24127fc640 1 -- 172.21.15.84:0/3845378908 >> v1:172.21.15.157:6789/0 conn(0x7f2414160800 legacy=0x7f241406adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:41.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.323+0000 7f24127fc640 1 -- 172.21.15.84:0/3845378908 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241406e6a0 con 0x7f241406b4c0 2023-10-16T11:22:41.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.323+0000 7f24127fc640 1 -- 172.21.15.84:0/3845378908 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f240c001670 con 0x7f241406b4c0 2023-10-16T11:22:41.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.323+0000 7f24127fc640 1 -- 172.21.15.84:0/3845378908 >> v1:172.21.15.84:6789/0 conn(0x7f241406b4c0 legacy=0x7f2414070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:41.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.324+0000 7f24127fc640 1 --2- 172.21.15.84:0/3845378908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2414160420 0x7f24000039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:41.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.324+0000 7f24127fc640 1 --2- 172.21.15.84:0/3845378908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2400003fd0 0x7f24000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:41.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.324+0000 7f24127fc640 1 --2- 172.21.15.84:0/3845378908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2400004d00 0x7f2400005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:41.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.324+0000 7f24127fc640 1 -- 172.21.15.84:0/3845378908 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241406e3c0 con 0x7f2400004d00 2023-10-16T11:22:41.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.324+0000 7f24127fc640 1 -- 172.21.15.84:0/3845378908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240000bb40 con 0x7f2400003fd0 2023-10-16T11:22:41.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.324+0000 7f24127fc640 1 -- 172.21.15.84:0/3845378908 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240000bd00 con 0x7f2414160420 2023-10-16T11:22:41.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.324+0000 7f2413fff640 1 --2- 172.21.15.84:0/3845378908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2400004d00 0x7f2400005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:41.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.324+0000 7f24137fe640 1 --2- 172.21.15.84:0/3845378908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2414160420 0x7f24000039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:41.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.324+0000 7f2412ffd640 1 --2- 172.21.15.84:0/3845378908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2400003fd0 0x7f24000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:41.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.324+0000 7f2413fff640 1 -- 172.21.15.84:0/3845378908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2414160420 msgr2=0x7f24000039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:41.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.324+0000 7f2413fff640 1 --2- 172.21.15.84:0/3845378908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2414160420 0x7f24000039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:41.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.324+0000 7f2413fff640 1 -- 172.21.15.84:0/3845378908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2400003fd0 msgr2=0x7f24000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:41.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.324+0000 7f2413fff640 1 --2- 172.21.15.84:0/3845378908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2400003fd0 0x7f24000047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:41.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.324+0000 7f2413fff640 1 -- 172.21.15.84:0/3845378908 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2400003520 con 0x7f2400004d00 2023-10-16T11:22:41.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.325+0000 7f2413fff640 1 --2- 172.21.15.84:0/3845378908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2400004d00 0x7f2400005180 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f240c00c210 tx=0x7f24000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26d6a6d3119e26e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:41.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.326+0000 7f24127fc640 1 -- 172.21.15.84:0/3845378908 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c005ad0 con 0x7f2400004d00 2023-10-16T11:22:41.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.326+0000 7f24127fc640 1 -- 172.21.15.84:0/3845378908 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240c005c30 con 0x7f2400004d00 2023-10-16T11:22:41.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.326+0000 7f24127fc640 1 -- 172.21.15.84:0/3845378908 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c02cad0 con 0x7f2400004d00 2023-10-16T11:22:41.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.326+0000 7f2419c0e640 1 -- 172.21.15.84:0/3845378908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2400004d00 msgr2=0x7f2400005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:41.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.326+0000 7f2419c0e640 1 --2- 172.21.15.84:0/3845378908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2400004d00 0x7f2400005180 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f240c00c210 tx=0x7f24000057f0 comp rx=0 tx=0).stop 2023-10-16T11:22:41.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.326+0000 7f2413fff640 1 -- 172.21.15.84:0/3845378908 reap_dead start 2023-10-16T11:22:41.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.326+0000 7f2419c0e640 1 -- 172.21.15.84:0/3845378908 shutdown_connections 2023-10-16T11:22:41.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.326+0000 7f2419c0e640 1 -- 172.21.15.84:0/3845378908 >> 172.21.15.84:0/3845378908 conn(0x7f24140b0d30 msgr2=0x7f24140b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:41.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.327+0000 7f2419c0e640 1 -- 172.21.15.84:0/3845378908 shutdown_connections 2023-10-16T11:22:41.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.327+0000 7f2419c0e640 1 -- 172.21.15.84:0/3845378908 wait complete. 2023-10-16T11:22:41.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.328+0000 7f2419c0e640 1 Processor -- start 2023-10-16T11:22:41.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.329+0000 7f2419c0e640 1 -- start start 2023-10-16T11:22:41.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.329+0000 7f2419c0e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2400004d00 0x7f241409a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:41.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.329+0000 7f2419c0e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24141f62b0 0x7f241415aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.329+0000 7f2419c0e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241415b430 0x7f24140c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.329+0000 7f2419c0e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241409bab0 con 0x7f2400004d00 2023-10-16T11:22:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.329+0000 7f2419c0e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241409ae30 con 0x7f241415b430 2023-10-16T11:22:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.329+0000 7f2419c0e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241409b110 con 0x7f24141f62b0 2023-10-16T11:22:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.330+0000 7f24137fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2400004d00 0x7f241409a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.330+0000 7f24137fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2400004d00 0x7f241409a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48386/0 (socket says 172.21.15.84:48386) 2023-10-16T11:22:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.330+0000 7f24137fe640 1 -- 172.21.15.84:0/2641650538 learned_addr learned my addr 172.21.15.84:0/2641650538 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.330+0000 7f2412ffd640 1 --2- 172.21.15.84:0/2641650538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24141f62b0 0x7f241415aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.330+0000 7f24137fe640 1 -- 172.21.15.84:0/2641650538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24141f62b0 msgr2=0x7f241415aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:41.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.330+0000 7f24137fe640 1 --2- 172.21.15.84:0/2641650538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24141f62b0 0x7f241415aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:41.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.330+0000 7f24137fe640 1 -- 172.21.15.84:0/2641650538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241415b430 msgr2=0x7f24140c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:41.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.330+0000 7f2413fff640 1 --2- 172.21.15.84:0/2641650538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241415b430 0x7f24140c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:41.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.330+0000 7f24137fe640 1 --2- 172.21.15.84:0/2641650538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241415b430 0x7f24140c28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:41.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.330+0000 7f24137fe640 1 -- 172.21.15.84:0/2641650538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241409b3f0 con 0x7f2400004d00 2023-10-16T11:22:41.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.331+0000 7f24137fe640 1 --2- 172.21.15.84:0/2641650538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2400004d00 0x7f241409a4c0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f240400f580 tx=0x7f240400fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:41.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.331+0000 7f240bfff640 1 -- 172.21.15.84:0/2641650538 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404014870 con 0x7f2400004d00 2023-10-16T11:22:41.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.331+0000 7f240bfff640 1 -- 172.21.15.84:0/2641650538 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24040149d0 con 0x7f2400004d00 2023-10-16T11:22:41.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.331+0000 7f240bfff640 1 -- 172.21.15.84:0/2641650538 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404014d80 con 0x7f2400004d00 2023-10-16T11:22:41.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.331+0000 7f2419c0e640 1 -- 172.21.15.84:0/2641650538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241409b6d0 con 0x7f2400004d00 2023-10-16T11:22:41.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.331+0000 7f2419c0e640 1 -- 172.21.15.84:0/2641650538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241409abe0 con 0x7f2400004d00 2023-10-16T11:22:41.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.332+0000 7f2419c0e640 1 -- 172.21.15.84:0/2641650538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f241404e160 con 0x7f2400004d00 2023-10-16T11:22:41.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.334+0000 7f240bfff640 1 -- 172.21.15.84:0/2641650538 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2404015410 con 0x7f2400004d00 2023-10-16T11:22:41.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.335+0000 7f240bfff640 1 --2- 172.21.15.84:0/2641650538 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f23f406af90 0x7f23f406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:41.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.335+0000 7f240bfff640 1 -- 172.21.15.84:0/2641650538 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5255+0+0 (secure 0 0 0) 0x7f240408a640 con 0x7f2400004d00 2023-10-16T11:22:41.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.335+0000 7f2412ffd640 1 --2- 172.21.15.84:0/2641650538 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f23f406af90 0x7f23f406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:41.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.336+0000 7f2412ffd640 1 --2- 172.21.15.84:0/2641650538 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f23f406af90 0x7f23f406d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23fc007fd0 tx=0x7f23fc009040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:41.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.339+0000 7f240bfff640 1 -- 172.21.15.84:0/2641650538 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2404055790 con 0x7f2400004d00 2023-10-16T11:22:41.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:41.530+0000 7f2419c0e640 1 -- 172.21.15.84:0/2641650538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f24140af1d0 con 0x7f2400004d00 2023-10-16T11:22:42.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.104+0000 7f240bfff640 1 -- 172.21.15.84:0/2641650538 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v199) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2404058db0 con 0x7f2400004d00 2023-10-16T11:22:42.106 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-16T11:22:42.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.108+0000 7f2419c0e640 1 -- 172.21.15.84:0/2641650538 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f23f406af90 msgr2=0x7f23f406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:42.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.108+0000 7f2419c0e640 1 --2- 172.21.15.84:0/2641650538 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f23f406af90 0x7f23f406d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23fc007fd0 tx=0x7f23fc009040 comp rx=0 tx=0).stop 2023-10-16T11:22:42.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.108+0000 7f2419c0e640 1 -- 172.21.15.84:0/2641650538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2400004d00 msgr2=0x7f241409a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:42.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.108+0000 7f2419c0e640 1 --2- 172.21.15.84:0/2641650538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2400004d00 0x7f241409a4c0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f240400f580 tx=0x7f240400fa40 comp rx=0 tx=0).stop 2023-10-16T11:22:42.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.110+0000 7f2419c0e640 1 -- 172.21.15.84:0/2641650538 shutdown_connections 2023-10-16T11:22:42.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.110+0000 7f2419c0e640 1 --2- 172.21.15.84:0/2641650538 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f23f406af90 0x7f23f406d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:42.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.110+0000 7f2419c0e640 1 --2- 172.21.15.84:0/2641650538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241415b430 0x7f24140c28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:42.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.110+0000 7f2419c0e640 1 --2- 172.21.15.84:0/2641650538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24141f62b0 0x7f241415aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:42.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.110+0000 7f2419c0e640 1 --2- 172.21.15.84:0/2641650538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2400004d00 0x7f241409a4c0 secure :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f240400f580 tx=0x7f240400fa40 comp rx=0 tx=0).stop 2023-10-16T11:22:42.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.110+0000 7f2419c0e640 1 -- 172.21.15.84:0/2641650538 >> 172.21.15.84:0/2641650538 conn(0x7f24140b0d30 msgr2=0x7f241406a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:42.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.112+0000 7f2419c0e640 1 -- 172.21.15.84:0/2641650538 shutdown_connections 2023-10-16T11:22:42.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.112+0000 7f2419c0e640 1 -- 172.21.15.84:0/2641650538 wait complete. 2023-10-16T11:22:42.147 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T11:22:42.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.331+0000 7f694e9a9640 1 Processor -- start 2023-10-16T11:22:42.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.332+0000 7f694e9a9640 1 -- start start 2023-10-16T11:22:42.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.333+0000 7f694e9a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f694815ad70 0x7f6948069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:42.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.333+0000 7f694e9a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69480c9ca0 0x7f694806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:42.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.333+0000 7f694e9a9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69480cb030 con 0x7f69480ca3b0 2023-10-16T11:22:42.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.333+0000 7f694e9a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69480c91c0 con 0x7f694815ad70 2023-10-16T11:22:42.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.333+0000 7f694e9a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69480c94a0 con 0x7f69480c9ca0 2023-10-16T11:22:42.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.333+0000 7f6947fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f694815ad70 0x7f6948069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:42.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.333+0000 7f6947fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f694815ad70 0x7f6948069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48400/0 (socket says 172.21.15.84:48400) 2023-10-16T11:22:42.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.333+0000 7f6947fff640 1 -- 172.21.15.84:0/4200383632 learned_addr learned my addr 172.21.15.84:0/4200383632 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:42.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.334+0000 7f6947fff640 1 -- 172.21.15.84:0/4200383632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69480c9ca0 msgr2=0x7f694806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:42.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.334+0000 7f6947fff640 1 --2- 172.21.15.84:0/4200383632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69480c9ca0 0x7f694806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:42.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.334+0000 7f6947fff640 1 -- 172.21.15.84:0/4200383632 >> v1:172.21.15.157:6789/0 conn(0x7f69480ca3b0 legacy=0x7f69480ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:42.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.334+0000 7f6947fff640 1 -- 172.21.15.84:0/4200383632 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69480c9780 con 0x7f694815ad70 2023-10-16T11:22:42.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.335+0000 7f6947fff640 1 --2- 172.21.15.84:0/4200383632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f694815ad70 0x7f6948069da0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f6938009870 tx=0x7f693802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4eeb968f5ed0d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:42.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.335+0000 7f6946ffd640 1 -- 172.21.15.84:0/4200383632 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6938031640 con 0x7f694815ad70 2023-10-16T11:22:42.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.335+0000 7f6946ffd640 1 -- 172.21.15.84:0/4200383632 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69380317a0 con 0x7f694815ad70 2023-10-16T11:22:42.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.335+0000 7f6946ffd640 1 -- 172.21.15.84:0/4200383632 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6938031b30 con 0x7f694815ad70 2023-10-16T11:22:42.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.336+0000 7f694e9a9640 1 -- 172.21.15.84:0/4200383632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f694815ad70 msgr2=0x7f6948069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:42.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.336+0000 7f694e9a9640 1 --2- 172.21.15.84:0/4200383632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f694815ad70 0x7f6948069da0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f6938009870 tx=0x7f693802b8f0 comp rx=0 tx=0).stop 2023-10-16T11:22:42.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.336+0000 7f694e9a9640 1 -- 172.21.15.84:0/4200383632 shutdown_connections 2023-10-16T11:22:42.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.336+0000 7f694e9a9640 1 --2- 172.21.15.84:0/4200383632 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69480c9ca0 0x7f694806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:42.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.336+0000 7f694e9a9640 1 --2- 172.21.15.84:0/4200383632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f694815ad70 0x7f6948069da0 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:42.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.336+0000 7f694e9a9640 1 -- 172.21.15.84:0/4200383632 >> 172.21.15.84:0/4200383632 conn(0x7f69480ce9b0 msgr2=0x7f69480cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:42.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.336+0000 7f694e9a9640 1 -- 172.21.15.84:0/4200383632 shutdown_connections 2023-10-16T11:22:42.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.336+0000 7f694e9a9640 1 -- 172.21.15.84:0/4200383632 wait complete. 2023-10-16T11:22:42.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.338+0000 7f694e9a9640 1 Processor -- start 2023-10-16T11:22:42.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.338+0000 7f694e9a9640 1 -- start start 2023-10-16T11:22:42.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.339+0000 7f694e9a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69480c9ca0 0x7f6948159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:42.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.339+0000 7f694e9a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69480ca3b0 0x7f69480b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:42.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.339+0000 7f694e9a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f694815ad70 0x7f69480a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:42.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.339+0000 7f694e9a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69480b3030 con 0x7f69480ca3b0 2023-10-16T11:22:42.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.339+0000 7f694e9a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69480b3310 con 0x7f694815ad70 2023-10-16T11:22:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.339+0000 7f694e9a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69480b35f0 con 0x7f69480c9ca0 2023-10-16T11:22:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.339+0000 7f69477fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69480ca3b0 0x7f69480b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.339+0000 7f69477fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69480ca3b0 0x7f69480b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48402/0 (socket says 172.21.15.84:48402) 2023-10-16T11:22:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.339+0000 7f69477fe640 1 -- 172.21.15.84:0/201073029 learned_addr learned my addr 172.21.15.84:0/201073029 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.340+0000 7f6947fff640 1 --2- 172.21.15.84:0/201073029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69480c9ca0 0x7f6948159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.340+0000 7f694cf46640 1 --2- 172.21.15.84:0/201073029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f694815ad70 0x7f69480a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.340+0000 7f69477fe640 1 -- 172.21.15.84:0/201073029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69480c9ca0 msgr2=0x7f6948159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.340+0000 7f69477fe640 1 --2- 172.21.15.84:0/201073029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69480c9ca0 0x7f6948159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.340+0000 7f69477fe640 1 -- 172.21.15.84:0/201073029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f694815ad70 msgr2=0x7f69480a90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.340+0000 7f69477fe640 1 --2- 172.21.15.84:0/201073029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f694815ad70 0x7f69480a90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.340+0000 7f69477fe640 1 -- 172.21.15.84:0/201073029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69380094d0 con 0x7f69480ca3b0 2023-10-16T11:22:42.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.340+0000 7f69477fe640 1 --2- 172.21.15.84:0/201073029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69480ca3b0 0x7f69480b2650 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f693000ba50 tx=0x7f693000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:42.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.340+0000 7f6944ff9640 1 -- 172.21.15.84:0/201073029 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693000cbe0 con 0x7f69480ca3b0 2023-10-16T11:22:42.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.340+0000 7f6944ff9640 1 -- 172.21.15.84:0/201073029 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693000cd60 con 0x7f69480ca3b0 2023-10-16T11:22:42.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.340+0000 7f6944ff9640 1 -- 172.21.15.84:0/201073029 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6930014680 con 0x7f69480ca3b0 2023-10-16T11:22:42.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.340+0000 7f694e9a9640 1 -- 172.21.15.84:0/201073029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69480b38d0 con 0x7f69480ca3b0 2023-10-16T11:22:42.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.340+0000 7f694e9a9640 1 -- 172.21.15.84:0/201073029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6948159a60 con 0x7f69480ca3b0 2023-10-16T11:22:42.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.341+0000 7f694e9a9640 1 -- 172.21.15.84:0/201073029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69480b3df0 con 0x7f69480ca3b0 2023-10-16T11:22:42.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.344+0000 7f6944ff9640 1 -- 172.21.15.84:0/201073029 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f693001d3b0 con 0x7f69480ca3b0 2023-10-16T11:22:42.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.344+0000 7f6944ff9640 1 --2- 172.21.15.84:0/201073029 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f692806afc0 0x7f692806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:42.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.345+0000 7f6944ff9640 1 -- 172.21.15.84:0/201073029 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5637+0+0 (secure 0 0 0) 0x7f6930089c10 con 0x7f69480ca3b0 2023-10-16T11:22:42.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.345+0000 7f6947fff640 1 --2- 172.21.15.84:0/201073029 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f692806afc0 0x7f692806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:42.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.345+0000 7f6947fff640 1 --2- 172.21.15.84:0/201073029 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f692806afc0 0x7f692806d480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6938036000 tx=0x7f693803c040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:42.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.349+0000 7f6944ff9640 1 -- 172.21.15.84:0/201073029 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6930054cd0 con 0x7f69480ca3b0 2023-10-16T11:22:42.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:42.576+0000 7f694e9a9640 1 -- 172.21.15.84:0/201073029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f69480b3df0 con 0x7f69480ca3b0 2023-10-16T11:22:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.108+0000 7f6944ff9640 1 -- 172.21.15.84:0/201073029 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v101) v1 ==== 174+0+0 (secure 0 0 0) 0x7f69480b3df0 con 0x7f69480ca3b0 2023-10-16T11:22:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 24 and data pool 25 2023-10-16T11:22:43.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.113+0000 7f694e9a9640 1 -- 172.21.15.84:0/201073029 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f692806afc0 msgr2=0x7f692806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:43.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.113+0000 7f694e9a9640 1 --2- 172.21.15.84:0/201073029 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f692806afc0 0x7f692806d480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6938036000 tx=0x7f693803c040 comp rx=0 tx=0).stop 2023-10-16T11:22:43.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.113+0000 7f694e9a9640 1 -- 172.21.15.84:0/201073029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69480ca3b0 msgr2=0x7f69480b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:43.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.113+0000 7f694e9a9640 1 --2- 172.21.15.84:0/201073029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69480ca3b0 0x7f69480b2650 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f693000ba50 tx=0x7f693000bf10 comp rx=0 tx=0).stop 2023-10-16T11:22:43.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.117+0000 7f694e9a9640 1 -- 172.21.15.84:0/201073029 shutdown_connections 2023-10-16T11:22:43.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.117+0000 7f694e9a9640 1 --2- 172.21.15.84:0/201073029 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f692806afc0 0x7f692806d480 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:43.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.117+0000 7f694e9a9640 1 --2- 172.21.15.84:0/201073029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f694815ad70 0x7f69480a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:43.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.117+0000 7f694e9a9640 1 --2- 172.21.15.84:0/201073029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69480ca3b0 0x7f69480b2650 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:43.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.117+0000 7f694e9a9640 1 --2- 172.21.15.84:0/201073029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69480c9ca0 0x7f6948159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:43.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.117+0000 7f694e9a9640 1 -- 172.21.15.84:0/201073029 >> 172.21.15.84:0/201073029 conn(0x7f69480ce9b0 msgr2=0x7f69480ca840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:43.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.120+0000 7f694e9a9640 1 -- 172.21.15.84:0/201073029 shutdown_connections 2023-10-16T11:22:43.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.121+0000 7f694e9a9640 1 -- 172.21.15.84:0/201073029 wait complete. 2023-10-16T11:22:43.158 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:22:43.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.339+0000 7f0d42993640 1 Processor -- start 2023-10-16T11:22:43.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.339+0000 7f0d42993640 1 -- start start 2023-10-16T11:22:43.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.340+0000 7f0d42993640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d3c159950 0x7f0d3c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:43.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.341+0000 7f0d42993640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d3c06c810 0x7f0d3c0708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:43.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.341+0000 7f0d42993640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d3c070df0 con 0x7f0d3c070450 2023-10-16T11:22:43.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.341+0000 7f0d42993640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d3c15e490 con 0x7f0d3c06c810 2023-10-16T11:22:43.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.341+0000 7f0d42993640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c15aef0 con 0x7f0d3c159950 2023-10-16T11:22:43.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.341+0000 7f0d42192640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d3c06c810 0x7f0d3c0708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:43.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.341+0000 7f0d41991640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d3c159950 0x7f0d3c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:43.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.342+0000 7f0d41991640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d3c159950 0x7f0d3c15a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:55048/0 (socket says 172.21.15.84:55048) 2023-10-16T11:22:43.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.342+0000 7f0d42192640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d3c06c810 0x7f0d3c0708b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:51506/0 (socket says 172.21.15.84:51506) 2023-10-16T11:22:43.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.342+0000 7f0d41991640 1 -- 172.21.15.84:0/1380873335 learned_addr learned my addr 172.21.15.84:0/1380873335 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:43.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.342+0000 7f0d41190640 1 --1- 172.21.15.84:0/1380873335 >> v1:172.21.15.157:6789/0 conn(0x7f0d3c070450 0x7f0d3c06c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:37220/0 (socket says 172.21.15.84:37220) 2023-10-16T11:22:43.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.343+0000 7f0d42192640 1 -- 172.21.15.84:0/1380873335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d3c159950 msgr2=0x7f0d3c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:43.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.343+0000 7f0d42192640 1 --2- 172.21.15.84:0/1380873335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d3c159950 0x7f0d3c15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:43.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.343+0000 7f0d42192640 1 -- 172.21.15.84:0/1380873335 >> v1:172.21.15.157:6789/0 conn(0x7f0d3c070450 legacy=0x7f0d3c06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:43.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.343+0000 7f0d42192640 1 -- 172.21.15.84:0/1380873335 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3c06cc80 con 0x7f0d3c06c810 2023-10-16T11:22:43.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.343+0000 7f0d41991640 1 --2- 172.21.15.84:0/1380873335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d3c159950 0x7f0d3c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:43.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.343+0000 7f0d4098f640 1 -- 172.21.15.84:0/1380873335 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3732167493 0 0) 0x7f0d3c070df0 con 0x7f0d3c070450 2023-10-16T11:22:43.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.344+0000 7f0d42192640 1 --2- 172.21.15.84:0/1380873335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c06c810 0x7f0d3c0708b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400bfa0 tx=0x7f0d3402e010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f80b9dffa5397c92 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:43.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.344+0000 7f0d4098f640 1 -- 172.21.15.84:0/1380873335 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3402e9b0 con 0x7f0d3c06c810 2023-10-16T11:22:43.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.345+0000 7f0d4098f640 1 -- 172.21.15.84:0/1380873335 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3402eb10 con 0x7f0d3c06c810 2023-10-16T11:22:43.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.345+0000 7f0d4098f640 1 -- 172.21.15.84:0/1380873335 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3402eea0 con 0x7f0d3c06c810 2023-10-16T11:22:43.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.345+0000 7f0d42993640 1 -- 172.21.15.84:0/1380873335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c06c810 msgr2=0x7f0d3c0708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:43.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.345+0000 7f0d42993640 1 --2- 172.21.15.84:0/1380873335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c06c810 0x7f0d3c0708b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400bfa0 tx=0x7f0d3402e010 comp rx=0 tx=0).stop 2023-10-16T11:22:43.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.345+0000 7f0d42993640 1 -- 172.21.15.84:0/1380873335 shutdown_connections 2023-10-16T11:22:43.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.345+0000 7f0d42993640 1 --2- 172.21.15.84:0/1380873335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c06c810 0x7f0d3c0708b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:43.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.345+0000 7f0d42993640 1 --2- 172.21.15.84:0/1380873335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d3c159950 0x7f0d3c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:43.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.345+0000 7f0d42993640 1 -- 172.21.15.84:0/1380873335 >> 172.21.15.84:0/1380873335 conn(0x7f0d3c0ce6a0 msgr2=0x7f0d3c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:43.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.345+0000 7f0d42993640 1 -- 172.21.15.84:0/1380873335 shutdown_connections 2023-10-16T11:22:43.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.345+0000 7f0d42993640 1 -- 172.21.15.84:0/1380873335 wait complete. 2023-10-16T11:22:43.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.347+0000 7f0d42993640 1 Processor -- start 2023-10-16T11:22:43.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.347+0000 7f0d42993640 1 -- start start 2023-10-16T11:22:43.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.348+0000 7f0d42993640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c06c810 0x7f0d3c1f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:43.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.348+0000 7f0d42993640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d3c070450 0x7f0d3c0cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:43.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.348+0000 7f0d42993640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d3c159950 0x7f0d3c0aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:43.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.348+0000 7f0d42993640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c1f3a30 con 0x7f0d3c159950 2023-10-16T11:22:43.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.348+0000 7f0d42993640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c0aef20 con 0x7f0d3c06c810 2023-10-16T11:22:43.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.348+0000 7f0d42993640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c0afb70 con 0x7f0d3c070450 2023-10-16T11:22:43.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.348+0000 7f0d42192640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d3c159950 0x7f0d3c0aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:43.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.348+0000 7f0d42192640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d3c159950 0x7f0d3c0aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48408/0 (socket says 172.21.15.84:48408) 2023-10-16T11:22:43.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.348+0000 7f0d42192640 1 -- 172.21.15.84:0/4062065202 learned_addr learned my addr 172.21.15.84:0/4062065202 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:43.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.349+0000 7f0d41190640 1 --2- 172.21.15.84:0/4062065202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d3c070450 0x7f0d3c0cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:43.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.349+0000 7f0d41991640 1 --2- 172.21.15.84:0/4062065202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c06c810 0x7f0d3c1f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:43.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.349+0000 7f0d42192640 1 -- 172.21.15.84:0/4062065202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d3c070450 msgr2=0x7f0d3c0cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:43.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.349+0000 7f0d42192640 1 --2- 172.21.15.84:0/4062065202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d3c070450 0x7f0d3c0cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:43.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.349+0000 7f0d42192640 1 -- 172.21.15.84:0/4062065202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c06c810 msgr2=0x7f0d3c1f34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:43.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.349+0000 7f0d42192640 1 --2- 172.21.15.84:0/4062065202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c06c810 0x7f0d3c1f34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:43.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.349+0000 7f0d42192640 1 -- 172.21.15.84:0/4062065202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d380094d0 con 0x7f0d3c159950 2023-10-16T11:22:43.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.349+0000 7f0d41991640 1 --2- 172.21.15.84:0/4062065202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c06c810 0x7f0d3c1f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:43.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.349+0000 7f0d42192640 1 --2- 172.21.15.84:0/4062065202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d3c159950 0x7f0d3c0aff30 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400bfa0 tx=0x7f0d34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:43.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.349+0000 7f0d327fc640 1 -- 172.21.15.84:0/4062065202 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34040620 con 0x7f0d3c159950 2023-10-16T11:22:43.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.349+0000 7f0d327fc640 1 -- 172.21.15.84:0/4062065202 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d340407a0 con 0x7f0d3c159950 2023-10-16T11:22:43.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.349+0000 7f0d42993640 1 -- 172.21.15.84:0/4062065202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d3400bc80 con 0x7f0d3c159950 2023-10-16T11:22:43.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.349+0000 7f0d327fc640 1 -- 172.21.15.84:0/4062065202 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34040b50 con 0x7f0d3c159950 2023-10-16T11:22:43.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.350+0000 7f0d42993640 1 -- 172.21.15.84:0/4062065202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d3c0ae3a0 con 0x7f0d3c159950 2023-10-16T11:22:43.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.351+0000 7f0d42993640 1 -- 172.21.15.84:0/4062065202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3c04e9b0 con 0x7f0d3c159950 2023-10-16T11:22:43.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.352+0000 7f0d327fc640 1 -- 172.21.15.84:0/4062065202 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0d34040cb0 con 0x7f0d3c159950 2023-10-16T11:22:43.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.353+0000 7f0d327fc640 1 --2- 172.21.15.84:0/4062065202 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f0d2006af90 0x7f0d2006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:43.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.353+0000 7f0d327fc640 1 -- 172.21.15.84:0/4062065202 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0d340ac950 con 0x7f0d3c159950 2023-10-16T11:22:43.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.353+0000 7f0d41991640 1 --2- 172.21.15.84:0/4062065202 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f0d2006af90 0x7f0d2006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:43.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.353+0000 7f0d41991640 1 --2- 172.21.15.84:0/4062065202 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f0d2006af90 0x7f0d2006d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009c50 tx=0x7f0d38008040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:43.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.357+0000 7f0d327fc640 1 -- 172.21.15.84:0/4062065202 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0d34077830 con 0x7f0d3c159950 2023-10-16T11:22:43.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.551+0000 7f0d42993640 1 -- 172.21.15.84:0/4062065202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d3c0b4620 con 0x7f0d3c159950 2023-10-16T11:22:43.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.554+0000 7f0d327fc640 1 -- 172.21.15.84:0/4062065202 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+16612 (secure 0 0 0) 0x7f0d3407ae50 con 0x7f0d3c159950 2023-10-16T11:22:43.556 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:43.557 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":200,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:22:42.577971+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:22:40.385524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-16T11:22:41.592928+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:22:43.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.557+0000 7f0d42993640 1 -- 172.21.15.84:0/4062065202 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f0d2006af90 msgr2=0x7f0d2006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:43.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.557+0000 7f0d42993640 1 --2- 172.21.15.84:0/4062065202 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f0d2006af90 0x7f0d2006d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009c50 tx=0x7f0d38008040 comp rx=0 tx=0).stop 2023-10-16T11:22:43.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.557+0000 7f0d42993640 1 -- 172.21.15.84:0/4062065202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d3c159950 msgr2=0x7f0d3c0aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:43.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.557+0000 7f0d42993640 1 --2- 172.21.15.84:0/4062065202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d3c159950 0x7f0d3c0aff30 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400bfa0 tx=0x7f0d34004a40 comp rx=0 tx=0).stop 2023-10-16T11:22:43.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.557+0000 7f0d42993640 1 -- 172.21.15.84:0/4062065202 shutdown_connections 2023-10-16T11:22:43.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.557+0000 7f0d42993640 1 --2- 172.21.15.84:0/4062065202 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f0d2006af90 0x7f0d2006d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:43.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.557+0000 7f0d42993640 1 --2- 172.21.15.84:0/4062065202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d3c159950 0x7f0d3c0aff30 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:43.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.557+0000 7f0d42993640 1 --2- 172.21.15.84:0/4062065202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d3c070450 0x7f0d3c0cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:43.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.557+0000 7f0d42993640 1 --2- 172.21.15.84:0/4062065202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c06c810 0x7f0d3c1f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:43.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.557+0000 7f0d42993640 1 -- 172.21.15.84:0/4062065202 >> 172.21.15.84:0/4062065202 conn(0x7f0d3c0ce6a0 msgr2=0x7f0d3c1581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:43.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.557+0000 7f0d42993640 1 -- 172.21.15.84:0/4062065202 shutdown_connections 2023-10-16T11:22:43.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.557+0000 7f0d42993640 1 -- 172.21.15.84:0/4062065202 wait complete. 2023-10-16T11:22:43.577 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:22:43.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.786+0000 7fc65b70f640 1 Processor -- start 2023-10-16T11:22:43.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.786+0000 7fc65b70f640 1 -- start start 2023-10-16T11:22:43.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.787+0000 7fc65b70f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6540adb50 0x7fc65406f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:43.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.787+0000 7fc65b70f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc654070fd0 0x7fc6540b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:43.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.787+0000 7fc65b70f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc654071450 con 0x7fc6540adf30 2023-10-16T11:22:43.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.787+0000 7fc65b70f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6540cc600 con 0x7fc654070fd0 2023-10-16T11:22:43.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.787+0000 7fc65b70f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540cc8e0 con 0x7fc6540adb50 2023-10-16T11:22:43.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.788+0000 7fc658caa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc654070fd0 0x7fc6540b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:43.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.788+0000 7fc6594ab640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6540adb50 0x7fc65406f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:43.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.788+0000 7fc6594ab640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6540adb50 0x7fc65406f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:55070/0 (socket says 172.21.15.84:55070) 2023-10-16T11:22:43.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.788+0000 7fc6594ab640 1 -- 172.21.15.84:0/938495521 learned_addr learned my addr 172.21.15.84:0/938495521 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:43.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.788+0000 7fc658caa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc654070fd0 0x7fc6540b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:51516/0 (socket says 172.21.15.84:51516) 2023-10-16T11:22:43.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.789+0000 7fc6594ab640 1 -- 172.21.15.84:0/938495521 >> v2:172.21.15.157:3300/0 conn(0x7fc654070fd0 msgr2=0x7fc6540b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:43.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.789+0000 7fc6594ab640 1 --2- 172.21.15.84:0/938495521 >> v2:172.21.15.157:3300/0 conn(0x7fc654070fd0 0x7fc6540b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:43.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.789+0000 7fc6594ab640 1 -- 172.21.15.84:0/938495521 >> v1:172.21.15.157:6789/0 conn(0x7fc6540adf30 legacy=0x7fc654164f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:43.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.789+0000 7fc6594ab640 1 -- 172.21.15.84:0/938495521 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6540ccbc0 con 0x7fc6540adb50 2023-10-16T11:22:43.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.790+0000 7fc6594ab640 1 --2- 172.21.15.84:0/938495521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6540adb50 0x7fc65406f550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc650009870 tx=0x7fc65002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab9473ce102ea0bc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:43.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.791+0000 7fc64bfff640 1 -- 172.21.15.84:0/938495521 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc650030840 con 0x7fc6540adb50 2023-10-16T11:22:43.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.791+0000 7fc64bfff640 1 -- 172.21.15.84:0/938495521 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6500309a0 con 0x7fc6540adb50 2023-10-16T11:22:43.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.791+0000 7fc64bfff640 1 -- 172.21.15.84:0/938495521 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc650030d30 con 0x7fc6540adb50 2023-10-16T11:22:43.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.791+0000 7fc65b70f640 1 -- 172.21.15.84:0/938495521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6540adb50 msgr2=0x7fc65406f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:43.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.791+0000 7fc65b70f640 1 --2- 172.21.15.84:0/938495521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6540adb50 0x7fc65406f550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc650009870 tx=0x7fc65002b980 comp rx=0 tx=0).stop 2023-10-16T11:22:43.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.792+0000 7fc65b70f640 1 -- 172.21.15.84:0/938495521 shutdown_connections 2023-10-16T11:22:43.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.792+0000 7fc65b70f640 1 --2- 172.21.15.84:0/938495521 >> v2:172.21.15.157:3300/0 conn(0x7fc654070fd0 0x7fc6540b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:43.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.792+0000 7fc65b70f640 1 --2- 172.21.15.84:0/938495521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6540adb50 0x7fc65406f550 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:43.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.792+0000 7fc65b70f640 1 -- 172.21.15.84:0/938495521 >> 172.21.15.84:0/938495521 conn(0x7fc6540cb410 msgr2=0x7fc6540cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:43.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.792+0000 7fc65b70f640 1 -- 172.21.15.84:0/938495521 shutdown_connections 2023-10-16T11:22:43.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.792+0000 7fc65b70f640 1 -- 172.21.15.84:0/938495521 wait complete. 2023-10-16T11:22:43.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.793+0000 7fc65b70f640 1 Processor -- start 2023-10-16T11:22:43.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.794+0000 7fc65b70f640 1 -- start start 2023-10-16T11:22:43.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.794+0000 7fc65b70f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc654070fd0 0x7fc6540be3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:43.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.794+0000 7fc65b70f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6540adb50 0x7fc6540c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:43.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.794+0000 7fc65b70f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6540adf30 0x7fc6540bda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:43.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.794+0000 7fc65b70f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65414ec30 con 0x7fc6540adf30 2023-10-16T11:22:43.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.794+0000 7fc65b70f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65414ef10 con 0x7fc654070fd0 2023-10-16T11:22:43.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.794+0000 7fc65b70f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65414f1f0 con 0x7fc6540adb50 2023-10-16T11:22:43.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.795+0000 7fc659cac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6540adf30 0x7fc6540bda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:43.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.795+0000 7fc659cac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6540adf30 0x7fc6540bda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48414/0 (socket says 172.21.15.84:48414) 2023-10-16T11:22:43.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.795+0000 7fc659cac640 1 -- 172.21.15.84:0/2078221623 learned_addr learned my addr 172.21.15.84:0/2078221623 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:43.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.795+0000 7fc6594ab640 1 --2- 172.21.15.84:0/2078221623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc654070fd0 0x7fc6540be3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:43.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.795+0000 7fc658caa640 1 --2- 172.21.15.84:0/2078221623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6540adb50 0x7fc6540c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:43.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.795+0000 7fc659cac640 1 -- 172.21.15.84:0/2078221623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6540adb50 msgr2=0x7fc6540c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:43.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.795+0000 7fc659cac640 1 --2- 172.21.15.84:0/2078221623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6540adb50 0x7fc6540c4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:43.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.795+0000 7fc659cac640 1 -- 172.21.15.84:0/2078221623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc654070fd0 msgr2=0x7fc6540be3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:43.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.795+0000 7fc659cac640 1 --2- 172.21.15.84:0/2078221623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc654070fd0 0x7fc6540be3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:43.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.795+0000 7fc659cac640 1 -- 172.21.15.84:0/2078221623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6500094d0 con 0x7fc6540adf30 2023-10-16T11:22:43.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.796+0000 7fc659cac640 1 --2- 172.21.15.84:0/2078221623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6540adf30 0x7fc6540bda60 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7fc64c00c7a0 tx=0x7fc64c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:43.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.796+0000 7fc649ffb640 1 -- 172.21.15.84:0/2078221623 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c003b70 con 0x7fc6540adf30 2023-10-16T11:22:43.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.796+0000 7fc649ffb640 1 -- 172.21.15.84:0/2078221623 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64c003cf0 con 0x7fc6540adf30 2023-10-16T11:22:43.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.796+0000 7fc649ffb640 1 -- 172.21.15.84:0/2078221623 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c003ea0 con 0x7fc6540adf30 2023-10-16T11:22:43.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.796+0000 7fc65b70f640 1 -- 172.21.15.84:0/2078221623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65414f4d0 con 0x7fc6540adf30 2023-10-16T11:22:43.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.796+0000 7fc65b70f640 1 -- 172.21.15.84:0/2078221623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6540bdfa0 con 0x7fc6540adf30 2023-10-16T11:22:43.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.797+0000 7fc65b70f640 1 -- 172.21.15.84:0/2078221623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65414f9f0 con 0x7fc6540adf30 2023-10-16T11:22:43.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.798+0000 7fc649ffb640 1 -- 172.21.15.84:0/2078221623 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc64c00aa90 con 0x7fc6540adf30 2023-10-16T11:22:43.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.799+0000 7fc649ffb640 1 --2- 172.21.15.84:0/2078221623 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc63806af90 0x7fc63806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:43.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.799+0000 7fc649ffb640 1 -- 172.21.15.84:0/2078221623 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc64c0895e0 con 0x7fc6540adf30 2023-10-16T11:22:43.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.799+0000 7fc6594ab640 1 --2- 172.21.15.84:0/2078221623 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc63806af90 0x7fc63806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:43.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.800+0000 7fc6594ab640 1 --2- 172.21.15.84:0/2078221623 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc63806af90 0x7fc63806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc650036000 tx=0x7fc650008030 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:43.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.803+0000 7fc649ffb640 1 -- 172.21.15.84:0/2078221623 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc64c054560 con 0x7fc6540adf30 2023-10-16T11:22:43.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.995+0000 7fc65b70f640 1 -- 172.21.15.84:0/2078221623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc65404e160 con 0x7fc6540adf30 2023-10-16T11:22:43.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.996+0000 7fc649ffb640 1 -- 172.21.15.84:0/2078221623 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+16612 (secure 0 0 0) 0x7fc64c057b80 con 0x7fc6540adf30 2023-10-16T11:22:43.998 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:43.999 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":200,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:22:42.577971+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:22:40.385524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-16T11:22:41.592928+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:22:44.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.998+0000 7fc65b70f640 1 -- 172.21.15.84:0/2078221623 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc63806af90 msgr2=0x7fc63806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:44.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.998+0000 7fc65b70f640 1 --2- 172.21.15.84:0/2078221623 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc63806af90 0x7fc63806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc650036000 tx=0x7fc650008030 comp rx=0 tx=0).stop 2023-10-16T11:22:44.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.998+0000 7fc65b70f640 1 -- 172.21.15.84:0/2078221623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6540adf30 msgr2=0x7fc6540bda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:44.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.998+0000 7fc65b70f640 1 --2- 172.21.15.84:0/2078221623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6540adf30 0x7fc6540bda60 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7fc64c00c7a0 tx=0x7fc64c00cc60 comp rx=0 tx=0).stop 2023-10-16T11:22:44.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.999+0000 7fc65b70f640 1 -- 172.21.15.84:0/2078221623 shutdown_connections 2023-10-16T11:22:44.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.999+0000 7fc65b70f640 1 --2- 172.21.15.84:0/2078221623 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fc63806af90 0x7fc63806d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:44.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.999+0000 7fc65b70f640 1 --2- 172.21.15.84:0/2078221623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6540adf30 0x7fc6540bda60 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:44.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.999+0000 7fc65b70f640 1 --2- 172.21.15.84:0/2078221623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6540adb50 0x7fc6540c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:44.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.999+0000 7fc65b70f640 1 --2- 172.21.15.84:0/2078221623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc654070fd0 0x7fc6540be3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:44.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.999+0000 7fc65b70f640 1 -- 172.21.15.84:0/2078221623 >> 172.21.15.84:0/2078221623 conn(0x7fc6540cb410 msgr2=0x7fc6540cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:44.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.999+0000 7fc65b70f640 1 -- 172.21.15.84:0/2078221623 shutdown_connections 2023-10-16T11:22:44.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:43.999+0000 7fc65b70f640 1 -- 172.21.15.84:0/2078221623 wait complete. 2023-10-16T11:22:44.019 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T11:22:44.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.231+0000 7f6c25b1d640 1 Processor -- start 2023-10-16T11:22:44.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.231+0000 7f6c25b1d640 1 -- start start 2023-10-16T11:22:44.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.233+0000 7f6c25b1d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6c200addb0 0x7f6c200caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:44.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.233+0000 7f6c25b1d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2006a4b0 0x7f6c201650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:44.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.233+0000 7f6c25b1d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c200cb140 con 0x7f6c200c9000 2023-10-16T11:22:44.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.233+0000 7f6c25b1d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c20161a50 con 0x7f6c200addb0 2023-10-16T11:22:44.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.233+0000 7f6c25b1d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2006b030 con 0x7f6c2006a4b0 2023-10-16T11:22:44.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.233+0000 7f6c1f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6c200addb0 0x7f6c200caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:44.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.233+0000 7f6c1f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6c200addb0 0x7f6c200caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48420/0 (socket says 172.21.15.84:48420) 2023-10-16T11:22:44.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.233+0000 7f6c1f7fe640 1 -- 172.21.15.84:0/3372206573 learned_addr learned my addr 172.21.15.84:0/3372206573 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:44.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.234+0000 7f6c1ffff640 1 --2- 172.21.15.84:0/3372206573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2006a4b0 0x7f6c201650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:44.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.234+0000 7f6c1f7fe640 1 -- 172.21.15.84:0/3372206573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2006a4b0 msgr2=0x7f6c201650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:44.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.234+0000 7f6c1f7fe640 1 --2- 172.21.15.84:0/3372206573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2006a4b0 0x7f6c201650d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:44.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.234+0000 7f6c1f7fe640 1 -- 172.21.15.84:0/3372206573 >> v1:172.21.15.157:6789/0 conn(0x7f6c200c9000 legacy=0x7f6c20069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:44.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.234+0000 7f6c1f7fe640 1 -- 172.21.15.84:0/3372206573 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2006b310 con 0x7f6c200addb0 2023-10-16T11:22:44.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.235+0000 7f6c1f7fe640 1 --2- 172.21.15.84:0/3372206573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c200addb0 0x7f6c200caa80 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009870 tx=0x7f6c1002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf4397bd80684fa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:44.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.235+0000 7f6c1effd640 1 -- 172.21.15.84:0/3372206573 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c10031640 con 0x7f6c200addb0 2023-10-16T11:22:44.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.235+0000 7f6c1effd640 1 -- 172.21.15.84:0/3372206573 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c100317a0 con 0x7f6c200addb0 2023-10-16T11:22:44.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.235+0000 7f6c1effd640 1 -- 172.21.15.84:0/3372206573 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c10031b30 con 0x7f6c200addb0 2023-10-16T11:22:44.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.236+0000 7f6c25b1d640 1 -- 172.21.15.84:0/3372206573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c200addb0 msgr2=0x7f6c200caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:44.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.236+0000 7f6c25b1d640 1 --2- 172.21.15.84:0/3372206573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c200addb0 0x7f6c200caa80 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009870 tx=0x7f6c1002b8f0 comp rx=0 tx=0).stop 2023-10-16T11:22:44.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.236+0000 7f6c25b1d640 1 -- 172.21.15.84:0/3372206573 shutdown_connections 2023-10-16T11:22:44.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.236+0000 7f6c25b1d640 1 --2- 172.21.15.84:0/3372206573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2006a4b0 0x7f6c201650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:44.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.236+0000 7f6c25b1d640 1 --2- 172.21.15.84:0/3372206573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c200addb0 0x7f6c200caa80 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:44.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.236+0000 7f6c25b1d640 1 -- 172.21.15.84:0/3372206573 >> 172.21.15.84:0/3372206573 conn(0x7f6c200ce9b0 msgr2=0x7f6c200cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:44.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.236+0000 7f6c25b1d640 1 -- 172.21.15.84:0/3372206573 shutdown_connections 2023-10-16T11:22:44.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.236+0000 7f6c25b1d640 1 -- 172.21.15.84:0/3372206573 wait complete. 2023-10-16T11:22:44.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.239+0000 7f6c25b1d640 1 Processor -- start 2023-10-16T11:22:44.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.239+0000 7f6c25b1d640 1 -- start start 2023-10-16T11:22:44.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.240+0000 7f6c25b1d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2006a4b0 0x7f6c2014fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:44.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.240+0000 7f6c25b1d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c200addb0 0x7f6c200bfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:44.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.240+0000 7f6c25b1d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c200c9000 0x7f6c200c0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:44.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.240+0000 7f6c25b1d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c200bd8c0 con 0x7f6c200addb0 2023-10-16T11:22:44.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.240+0000 7f6c25b1d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c200c07d0 con 0x7f6c200c9000 2023-10-16T11:22:44.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.240+0000 7f6c25b1d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c200c0ab0 con 0x7f6c2006a4b0 2023-10-16T11:22:44.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.240+0000 7f6c17fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c200addb0 0x7f6c200bfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:44.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.240+0000 7f6c17fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c200addb0 0x7f6c200bfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48430/0 (socket says 172.21.15.84:48430) 2023-10-16T11:22:44.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.240+0000 7f6c17fff640 1 -- 172.21.15.84:0/1644530113 learned_addr learned my addr 172.21.15.84:0/1644530113 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:44.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.241+0000 7f6c1f7fe640 1 --2- 172.21.15.84:0/1644530113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2006a4b0 0x7f6c2014fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:44.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.241+0000 7f6c1ffff640 1 --2- 172.21.15.84:0/1644530113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c200c9000 0x7f6c200c0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:44.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.241+0000 7f6c17fff640 1 -- 172.21.15.84:0/1644530113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2006a4b0 msgr2=0x7f6c2014fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:44.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.241+0000 7f6c17fff640 1 --2- 172.21.15.84:0/1644530113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2006a4b0 0x7f6c2014fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:44.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.241+0000 7f6c17fff640 1 -- 172.21.15.84:0/1644530113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c200c9000 msgr2=0x7f6c200c0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:44.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.241+0000 7f6c17fff640 1 --2- 172.21.15.84:0/1644530113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c200c9000 0x7f6c200c0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:44.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.241+0000 7f6c17fff640 1 -- 172.21.15.84:0/1644530113 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c100094d0 con 0x7f6c200addb0 2023-10-16T11:22:44.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.241+0000 7f6c17fff640 1 --2- 172.21.15.84:0/1644530113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c200addb0 0x7f6c200bfc10 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800b3f0 tx=0x7f6c0800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:44.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.241+0000 7f6c1cff9640 1 -- 172.21.15.84:0/1644530113 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c08011830 con 0x7f6c200addb0 2023-10-16T11:22:44.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.241+0000 7f6c1cff9640 1 -- 172.21.15.84:0/1644530113 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c08011990 con 0x7f6c200addb0 2023-10-16T11:22:44.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.241+0000 7f6c1cff9640 1 -- 172.21.15.84:0/1644530113 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c08011d40 con 0x7f6c200addb0 2023-10-16T11:22:44.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.242+0000 7f6c25b1d640 1 -- 172.21.15.84:0/1644530113 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c200c0d90 con 0x7f6c200addb0 2023-10-16T11:22:44.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.242+0000 7f6c25b1d640 1 -- 172.21.15.84:0/1644530113 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c200c6110 con 0x7f6c200addb0 2023-10-16T11:22:44.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.243+0000 7f6c25b1d640 1 -- 172.21.15.84:0/1644530113 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2004e160 con 0x7f6c200addb0 2023-10-16T11:22:44.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.244+0000 7f6c1cff9640 1 -- 172.21.15.84:0/1644530113 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6c0801a400 con 0x7f6c200addb0 2023-10-16T11:22:44.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.245+0000 7f6c1cff9640 1 --2- 172.21.15.84:0/1644530113 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f6c0006afc0 0x7f6c0006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:44.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.245+0000 7f6c1cff9640 1 -- 172.21.15.84:0/1644530113 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6c080895b0 con 0x7f6c200addb0 2023-10-16T11:22:44.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.245+0000 7f6c1f7fe640 1 --2- 172.21.15.84:0/1644530113 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f6c0006afc0 0x7f6c0006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:44.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.246+0000 7f6c1f7fe640 1 --2- 172.21.15.84:0/1644530113 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f6c0006afc0 0x7f6c0006d480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6c10036000 tx=0x7f6c1003c040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:44.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.249+0000 7f6c1cff9640 1 -- 172.21.15.84:0/1644530113 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6c08054530 con 0x7f6c200addb0 2023-10-16T11:22:44.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:44.477+0000 7f6c25b1d640 1 -- 172.21.15.84:0/1644530113 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6c2006c5b0 con 0x7f6c200addb0 2023-10-16T11:22:45.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.146+0000 7f6c1cff9640 1 -- 172.21.15.84:0/1644530113 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v104) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6c08057b50 con 0x7f6c200addb0 2023-10-16T11:22:45.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.150+0000 7f6c25b1d640 1 -- 172.21.15.84:0/1644530113 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f6c0006afc0 msgr2=0x7f6c0006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:45.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.150+0000 7f6c25b1d640 1 --2- 172.21.15.84:0/1644530113 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f6c0006afc0 0x7f6c0006d480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6c10036000 tx=0x7f6c1003c040 comp rx=0 tx=0).stop 2023-10-16T11:22:45.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.150+0000 7f6c25b1d640 1 -- 172.21.15.84:0/1644530113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c200addb0 msgr2=0x7f6c200bfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:45.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.150+0000 7f6c25b1d640 1 --2- 172.21.15.84:0/1644530113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c200addb0 0x7f6c200bfc10 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800b3f0 tx=0x7f6c0800b8b0 comp rx=0 tx=0).stop 2023-10-16T11:22:45.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.150+0000 7f6c25b1d640 1 -- 172.21.15.84:0/1644530113 shutdown_connections 2023-10-16T11:22:45.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.150+0000 7f6c25b1d640 1 --2- 172.21.15.84:0/1644530113 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f6c0006afc0 0x7f6c0006d480 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:45.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.150+0000 7f6c25b1d640 1 --2- 172.21.15.84:0/1644530113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c200c9000 0x7f6c200c0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:45.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.150+0000 7f6c25b1d640 1 --2- 172.21.15.84:0/1644530113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c200addb0 0x7f6c200bfc10 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:45.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.150+0000 7f6c25b1d640 1 --2- 172.21.15.84:0/1644530113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2006a4b0 0x7f6c2014fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:45.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.150+0000 7f6c25b1d640 1 -- 172.21.15.84:0/1644530113 >> 172.21.15.84:0/1644530113 conn(0x7f6c200ce9b0 msgr2=0x7f6c200cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:45.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.151+0000 7f6c25b1d640 1 -- 172.21.15.84:0/1644530113 shutdown_connections 2023-10-16T11:22:45.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.151+0000 7f6c25b1d640 1 -- 172.21.15.84:0/1644530113 wait complete. 2023-10-16T11:22:45.173 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T11:22:45.174 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T11:22:45.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.387+0000 7fdc081b0640 1 Processor -- start 2023-10-16T11:22:45.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.387+0000 7fdc081b0640 1 -- start start 2023-10-16T11:22:45.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.389+0000 7fdc081b0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc0006a4b0 0x7fdc000ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:45.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.389+0000 7fdc081b0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdc000c9ca0 0x7fdc000ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:45.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.389+0000 7fdc081b0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc000cb030 con 0x7fdc0015ad70 2023-10-16T11:22:45.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.389+0000 7fdc081b0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc000cae30 con 0x7fdc000c9ca0 2023-10-16T11:22:45.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.389+0000 7fdc081b0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc000c91c0 con 0x7fdc0006a4b0 2023-10-16T11:22:45.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.389+0000 7fdc0674d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdc000c9ca0 0x7fdc000ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:45.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.389+0000 7fdc0674d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdc000c9ca0 0x7fdc000ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48446/0 (socket says 172.21.15.84:48446) 2023-10-16T11:22:45.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.389+0000 7fdc0674d640 1 -- 172.21.15.84:0/2754881142 learned_addr learned my addr 172.21.15.84:0/2754881142 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:45.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.390+0000 7fdc0674d640 1 -- 172.21.15.84:0/2754881142 >> v2:172.21.15.157:3300/0 conn(0x7fdc0006a4b0 msgr2=0x7fdc000ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:45.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.390+0000 7fdc0674d640 1 --2- 172.21.15.84:0/2754881142 >> v2:172.21.15.157:3300/0 conn(0x7fdc0006a4b0 0x7fdc000ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:45.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.390+0000 7fdc0674d640 1 -- 172.21.15.84:0/2754881142 >> v1:172.21.15.157:6789/0 conn(0x7fdc0015ad70 legacy=0x7fdc00069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:45.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.390+0000 7fdc0674d640 1 -- 172.21.15.84:0/2754881142 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc000c94a0 con 0x7fdc000c9ca0 2023-10-16T11:22:45.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.391+0000 7fdc0674d640 1 --2- 172.21.15.84:0/2754881142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc000c9ca0 0x7fdc000ca8f0 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fdc000ce870 tx=0x7fdbf802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94bebbd6996e889 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:45.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.391+0000 7fdc04f4a640 1 -- 172.21.15.84:0/2754881142 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf802ca80 con 0x7fdc000c9ca0 2023-10-16T11:22:45.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.391+0000 7fdc04f4a640 1 -- 172.21.15.84:0/2754881142 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf802cbe0 con 0x7fdc000c9ca0 2023-10-16T11:22:45.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.391+0000 7fdc04f4a640 1 -- 172.21.15.84:0/2754881142 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf80343f0 con 0x7fdc000c9ca0 2023-10-16T11:22:45.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.392+0000 7fdc081b0640 1 -- 172.21.15.84:0/2754881142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc000c9ca0 msgr2=0x7fdc000ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:45.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.392+0000 7fdc081b0640 1 --2- 172.21.15.84:0/2754881142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc000c9ca0 0x7fdc000ca8f0 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fdc000ce870 tx=0x7fdbf802bee0 comp rx=0 tx=0).stop 2023-10-16T11:22:45.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.392+0000 7fdc081b0640 1 -- 172.21.15.84:0/2754881142 shutdown_connections 2023-10-16T11:22:45.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.392+0000 7fdc081b0640 1 --2- 172.21.15.84:0/2754881142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc000c9ca0 0x7fdc000ca8f0 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:45.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.392+0000 7fdc081b0640 1 --2- 172.21.15.84:0/2754881142 >> v2:172.21.15.157:3300/0 conn(0x7fdc0006a4b0 0x7fdc000ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:45.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.392+0000 7fdc081b0640 1 -- 172.21.15.84:0/2754881142 >> 172.21.15.84:0/2754881142 conn(0x7fdc000ce9b0 msgr2=0x7fdc000cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:45.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.392+0000 7fdc081b0640 1 -- 172.21.15.84:0/2754881142 shutdown_connections 2023-10-16T11:22:45.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.392+0000 7fdc081b0640 1 -- 172.21.15.84:0/2754881142 wait complete. 2023-10-16T11:22:45.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.394+0000 7fdc081b0640 1 Processor -- start 2023-10-16T11:22:45.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.394+0000 7fdc081b0640 1 -- start start 2023-10-16T11:22:45.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.395+0000 7fdc081b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc0006a4b0 0x7fdc000af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:45.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.395+0000 7fdc081b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc000c9ca0 0x7fdc001f2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:45.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.395+0000 7fdc081b0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc0015ad70 0x7fdc001f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:45.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.395+0000 7fdc081b0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc001f2be0 con 0x7fdc0015ad70 2023-10-16T11:22:45.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.395+0000 7fdc081b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc001599d0 con 0x7fdc0006a4b0 2023-10-16T11:22:45.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.395+0000 7fdc081b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000af790 con 0x7fdc000c9ca0 2023-10-16T11:22:45.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.395+0000 7fdc0674d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc0015ad70 0x7fdc001f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:45.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.395+0000 7fdc0674d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc0015ad70 0x7fdc001f26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48460/0 (socket says 172.21.15.84:48460) 2023-10-16T11:22:45.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.395+0000 7fdc0674d640 1 -- 172.21.15.84:0/2907430579 learned_addr learned my addr 172.21.15.84:0/2907430579 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:45.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.395+0000 7fdc0674d640 1 -- 172.21.15.84:0/2907430579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc000c9ca0 msgr2=0x7fdc001f2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:45.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.395+0000 7fdc0574b640 1 --2- 172.21.15.84:0/2907430579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc000c9ca0 0x7fdc001f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:45.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.395+0000 7fdc05f4c640 1 --2- 172.21.15.84:0/2907430579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc0006a4b0 0x7fdc000af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:45.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.395+0000 7fdc0674d640 1 --2- 172.21.15.84:0/2907430579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc000c9ca0 0x7fdc001f2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:45.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.395+0000 7fdc0674d640 1 -- 172.21.15.84:0/2907430579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc0006a4b0 msgr2=0x7fdc000af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:45.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.395+0000 7fdc0674d640 1 --2- 172.21.15.84:0/2907430579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc0006a4b0 0x7fdc000af250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:45.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.395+0000 7fdc0674d640 1 -- 172.21.15.84:0/2907430579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf8009b30 con 0x7fdc0015ad70 2023-10-16T11:22:45.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.395+0000 7fdc05f4c640 1 --2- 172.21.15.84:0/2907430579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc0006a4b0 0x7fdc000af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:45.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.396+0000 7fdc0674d640 1 --2- 172.21.15.84:0/2907430579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc0015ad70 0x7fdc001f26a0 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8036040 tx=0x7fdbf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:45.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.396+0000 7fdbf27fc640 1 -- 172.21.15.84:0/2907430579 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf803e450 con 0x7fdc0015ad70 2023-10-16T11:22:45.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.396+0000 7fdbf27fc640 1 -- 172.21.15.84:0/2907430579 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf803e5d0 con 0x7fdc0015ad70 2023-10-16T11:22:45.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.396+0000 7fdbf27fc640 1 -- 172.21.15.84:0/2907430579 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf803e980 con 0x7fdc0015ad70 2023-10-16T11:22:45.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.396+0000 7fdc081b0640 1 -- 172.21.15.84:0/2907430579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc000c31d0 con 0x7fdc0015ad70 2023-10-16T11:22:45.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.396+0000 7fdc081b0640 1 -- 172.21.15.84:0/2907430579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc000c3790 con 0x7fdc0015ad70 2023-10-16T11:22:45.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.397+0000 7fdc081b0640 1 -- 172.21.15.84:0/2907430579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc000c5a80 con 0x7fdc0015ad70 2023-10-16T11:22:45.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.398+0000 7fdbf27fc640 1 -- 172.21.15.84:0/2907430579 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdbf802cd50 con 0x7fdc0015ad70 2023-10-16T11:22:45.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.399+0000 7fdbf27fc640 1 --2- 172.21.15.84:0/2907430579 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fdbe406af90 0x7fdbe406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:45.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.399+0000 7fdbf27fc640 1 -- 172.21.15.84:0/2907430579 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fdbf80ac480 con 0x7fdc0015ad70 2023-10-16T11:22:45.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.399+0000 7fdc05f4c640 1 --2- 172.21.15.84:0/2907430579 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fdbe406af90 0x7fdbe406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:45.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.400+0000 7fdc05f4c640 1 --2- 172.21.15.84:0/2907430579 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fdbe406af90 0x7fdbe406d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00a810 tx=0x7fdbfc008040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:45.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.404+0000 7fdbf27fc640 1 -- 172.21.15.84:0/2907430579 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdc000c5a80 con 0x7fdc0015ad70 2023-10-16T11:22:45.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:45.633+0000 7fdc081b0640 1 -- 172.21.15.84:0/2907430579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdc000c5a80 con 0x7fdc0015ad70 2023-10-16T11:22:46.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.157+0000 7fdbf27fc640 1 -- 172.21.15.84:0/2907430579 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v105) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdc000c5a80 con 0x7fdc0015ad70 2023-10-16T11:22:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.160+0000 7fdc081b0640 1 -- 172.21.15.84:0/2907430579 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fdbe406af90 msgr2=0x7fdbe406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.160+0000 7fdc081b0640 1 --2- 172.21.15.84:0/2907430579 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fdbe406af90 0x7fdbe406d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00a810 tx=0x7fdbfc008040 comp rx=0 tx=0).stop 2023-10-16T11:22:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.160+0000 7fdc081b0640 1 -- 172.21.15.84:0/2907430579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc0015ad70 msgr2=0x7fdc001f26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.160+0000 7fdc081b0640 1 --2- 172.21.15.84:0/2907430579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc0015ad70 0x7fdc001f26a0 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8036040 tx=0x7fdbf8004a40 comp rx=0 tx=0).stop 2023-10-16T11:22:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.161+0000 7fdc081b0640 1 -- 172.21.15.84:0/2907430579 shutdown_connections 2023-10-16T11:22:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.161+0000 7fdc081b0640 1 --2- 172.21.15.84:0/2907430579 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fdbe406af90 0x7fdbe406d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.161+0000 7fdc081b0640 1 --2- 172.21.15.84:0/2907430579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc0015ad70 0x7fdc001f26a0 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.161+0000 7fdc081b0640 1 --2- 172.21.15.84:0/2907430579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc000c9ca0 0x7fdc001f2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.161+0000 7fdc081b0640 1 --2- 172.21.15.84:0/2907430579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc0006a4b0 0x7fdc000af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.161+0000 7fdc081b0640 1 -- 172.21.15.84:0/2907430579 >> 172.21.15.84:0/2907430579 conn(0x7fdc000ce9b0 msgr2=0x7fdc000ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:46.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.161+0000 7fdc081b0640 1 -- 172.21.15.84:0/2907430579 shutdown_connections 2023-10-16T11:22:46.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.161+0000 7fdc081b0640 1 -- 172.21.15.84:0/2907430579 wait complete. 2023-10-16T11:22:46.182 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T11:22:46.183 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:22:46.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.393+0000 7f7208c0e640 1 Processor -- start 2023-10-16T11:22:46.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.394+0000 7f7208c0e640 1 -- start start 2023-10-16T11:22:46.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.395+0000 7f7208c0e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f72040cd0c0 0x7f72041615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:46.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.395+0000 7f7208c0e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f72040b0410 0x7f72040b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:46.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.395+0000 7f7208c0e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f720415e3e0 con 0x7f720415d3f0 2023-10-16T11:22:46.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.395+0000 7f7208c0e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72040b0de0 con 0x7f72040cd0c0 2023-10-16T11:22:46.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.395+0000 7f7208c0e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72040cd4a0 con 0x7f72040b0410 2023-10-16T11:22:46.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.395+0000 7f720259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f72040cd0c0 0x7f72041615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:46.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.396+0000 7f720259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f72040cd0c0 0x7f72041615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38390/0 (socket says 172.21.15.84:38390) 2023-10-16T11:22:46.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.396+0000 7f720259c640 1 -- 172.21.15.84:0/654662533 learned_addr learned my addr 172.21.15.84:0/654662533 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:46.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.396+0000 7f7201d9b640 1 --2- 172.21.15.84:0/654662533 >> v2:172.21.15.157:3300/0 conn(0x7f72040b0410 0x7f72040b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:46.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.396+0000 7f720259c640 1 -- 172.21.15.84:0/654662533 >> v2:172.21.15.157:3300/0 conn(0x7f72040b0410 msgr2=0x7f72040b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:46.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.396+0000 7f720259c640 1 --2- 172.21.15.84:0/654662533 >> v2:172.21.15.157:3300/0 conn(0x7f72040b0410 0x7f72040b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.396+0000 7f720259c640 1 -- 172.21.15.84:0/654662533 >> v1:172.21.15.157:6789/0 conn(0x7f720415d3f0 legacy=0x7f720415d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:46.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.396+0000 7f720259c640 1 -- 172.21.15.84:0/654662533 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720415e100 con 0x7f72040cd0c0 2023-10-16T11:22:46.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.398+0000 7f720259c640 1 --2- 172.21.15.84:0/654662533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72040cd0c0 0x7f72041615f0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f72040cdad0 tx=0x7f71ec02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5aeae26e7c37df server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:46.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.398+0000 7f720159a640 1 -- 172.21.15.84:0/654662533 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec0317e0 con 0x7f72040cd0c0 2023-10-16T11:22:46.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.398+0000 7f720159a640 1 -- 172.21.15.84:0/654662533 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ec031940 con 0x7f72040cd0c0 2023-10-16T11:22:46.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.398+0000 7f720159a640 1 -- 172.21.15.84:0/654662533 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec031cd0 con 0x7f72040cd0c0 2023-10-16T11:22:46.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.399+0000 7f7208c0e640 1 -- 172.21.15.84:0/654662533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72040cd0c0 msgr2=0x7f72041615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:46.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.399+0000 7f7208c0e640 1 --2- 172.21.15.84:0/654662533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72040cd0c0 0x7f72041615f0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f72040cdad0 tx=0x7f71ec02b880 comp rx=0 tx=0).stop 2023-10-16T11:22:46.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.399+0000 7f7208c0e640 1 -- 172.21.15.84:0/654662533 shutdown_connections 2023-10-16T11:22:46.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.399+0000 7f7208c0e640 1 --2- 172.21.15.84:0/654662533 >> v2:172.21.15.157:3300/0 conn(0x7f72040b0410 0x7f72040b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.399+0000 7f7208c0e640 1 --2- 172.21.15.84:0/654662533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72040cd0c0 0x7f72041615f0 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.399+0000 7f7208c0e640 1 -- 172.21.15.84:0/654662533 >> 172.21.15.84:0/654662533 conn(0x7f72040cb410 msgr2=0x7f72040cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:46.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.399+0000 7f7208c0e640 1 -- 172.21.15.84:0/654662533 shutdown_connections 2023-10-16T11:22:46.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.399+0000 7f7208c0e640 1 -- 172.21.15.84:0/654662533 wait complete. 2023-10-16T11:22:46.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.401+0000 7f7208c0e640 1 Processor -- start 2023-10-16T11:22:46.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.401+0000 7f7208c0e640 1 -- start start 2023-10-16T11:22:46.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.402+0000 7f7208c0e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72040b0410 0x7f7204155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:46.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.402+0000 7f7208c0e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72040cd0c0 0x7f72040c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:46.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.402+0000 7f7208c0e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720415d3f0 0x7f72040c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:46.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.402+0000 7f7208c0e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72040c5370 con 0x7f720415d3f0 2023-10-16T11:22:46.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.402+0000 7f7208c0e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72040c6da0 con 0x7f72040b0410 2023-10-16T11:22:46.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.402+0000 7f7208c0e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72040c47f0 con 0x7f72040cd0c0 2023-10-16T11:22:46.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.402+0000 7f7202d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720415d3f0 0x7f72040c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:46.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.402+0000 7f7202d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720415d3f0 0x7f72040c5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38406/0 (socket says 172.21.15.84:38406) 2023-10-16T11:22:46.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.402+0000 7f7202d9d640 1 -- 172.21.15.84:0/2955009821 learned_addr learned my addr 172.21.15.84:0/2955009821 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:46.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.402+0000 7f7201d9b640 1 --2- 172.21.15.84:0/2955009821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72040cd0c0 0x7f72040c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:46.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.402+0000 7f7202d9d640 1 -- 172.21.15.84:0/2955009821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72040cd0c0 msgr2=0x7f72040c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:46.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.402+0000 7f7202d9d640 1 --2- 172.21.15.84:0/2955009821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72040cd0c0 0x7f72040c5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.402+0000 7f7202d9d640 1 -- 172.21.15.84:0/2955009821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72040b0410 msgr2=0x7f7204155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:46.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.402+0000 7f720259c640 1 --2- 172.21.15.84:0/2955009821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72040b0410 0x7f7204155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:46.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.403+0000 7f7202d9d640 1 --2- 172.21.15.84:0/2955009821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72040b0410 0x7f7204155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.403+0000 7f7202d9d640 1 -- 172.21.15.84:0/2955009821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ec0094d0 con 0x7f720415d3f0 2023-10-16T11:22:46.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.403+0000 7f7202d9d640 1 --2- 172.21.15.84:0/2955009821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720415d3f0 0x7f72040c5fa0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00c410 tx=0x7f71fc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:46.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.403+0000 7f71faffd640 1 -- 172.21.15.84:0/2955009821 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc011790 con 0x7f720415d3f0 2023-10-16T11:22:46.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.403+0000 7f71faffd640 1 -- 172.21.15.84:0/2955009821 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71fc0118f0 con 0x7f720415d3f0 2023-10-16T11:22:46.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.403+0000 7f71faffd640 1 -- 172.21.15.84:0/2955009821 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc011ca0 con 0x7f720415d3f0 2023-10-16T11:22:46.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.403+0000 7f7208c0e640 1 -- 172.21.15.84:0/2955009821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72040c4ad0 con 0x7f720415d3f0 2023-10-16T11:22:46.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.403+0000 7f7208c0e640 1 -- 172.21.15.84:0/2955009821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72040c5090 con 0x7f720415d3f0 2023-10-16T11:22:46.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.404+0000 7f7208c0e640 1 -- 172.21.15.84:0/2955009821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720404e160 con 0x7f720415d3f0 2023-10-16T11:22:46.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.405+0000 7f71faffd640 1 -- 172.21.15.84:0/2955009821 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f71fc01c460 con 0x7f720415d3f0 2023-10-16T11:22:46.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.406+0000 7f71faffd640 1 --2- 172.21.15.84:0/2955009821 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f71e406af90 0x7f71e406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:46.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.406+0000 7f71faffd640 1 -- 172.21.15.84:0/2955009821 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f71fc089480 con 0x7f720415d3f0 2023-10-16T11:22:46.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.407+0000 7f720259c640 1 --2- 172.21.15.84:0/2955009821 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f71e406af90 0x7f71e406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:46.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.407+0000 7f720259c640 1 --2- 172.21.15.84:0/2955009821 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f71e406af90 0x7f71e406d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f71ec036000 tx=0x7f71ec03d040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:46.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.411+0000 7f71faffd640 1 -- 172.21.15.84:0/2955009821 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f71fc054400 con 0x7f720415d3f0 2023-10-16T11:22:46.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.639+0000 7f7208c0e640 1 -- 172.21.15.84:0/2955009821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72040b4910 con 0x7f720415d3f0 2023-10-16T11:22:46.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.640+0000 7f71faffd640 1 -- 172.21.15.84:0/2955009821 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4929 (secure 0 0 0) 0x7f71fc057a20 con 0x7f720415d3f0 2023-10-16T11:22:46.642 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:46.642 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6575,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/855929374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":855929374},{"type":"v1","addr":"172.21.15.157:6834","nonce":855929374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6578,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2581460972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2581460972},{"type":"v1","addr":"172.21.15.157:6835","nonce":2581460972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":10806,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1731880648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1731880648},{"type":"v1","addr":"172.21.15.84:6836","nonce":1731880648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:22:42.578443+0000","modified":"2023-10-16T11:22:45.635565+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10809":{"gid":10809,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2679249765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2679249765},{"type":"v1","addr":"172.21.15.84:6837","nonce":2679249765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-16T11:22:46.643 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 105 2023-10-16T11:22:46.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.643+0000 7f7208c0e640 1 -- 172.21.15.84:0/2955009821 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f71e406af90 msgr2=0x7f71e406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:46.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.643+0000 7f7208c0e640 1 --2- 172.21.15.84:0/2955009821 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f71e406af90 0x7f71e406d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f71ec036000 tx=0x7f71ec03d040 comp rx=0 tx=0).stop 2023-10-16T11:22:46.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.643+0000 7f7208c0e640 1 -- 172.21.15.84:0/2955009821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720415d3f0 msgr2=0x7f72040c5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:46.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.643+0000 7f7208c0e640 1 --2- 172.21.15.84:0/2955009821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720415d3f0 0x7f72040c5fa0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00c410 tx=0x7f71fc00c8d0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.643+0000 7f7208c0e640 1 -- 172.21.15.84:0/2955009821 shutdown_connections 2023-10-16T11:22:46.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.643+0000 7f7208c0e640 1 --2- 172.21.15.84:0/2955009821 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f71e406af90 0x7f71e406d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.643+0000 7f7208c0e640 1 --2- 172.21.15.84:0/2955009821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720415d3f0 0x7f72040c5fa0 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.643+0000 7f7208c0e640 1 --2- 172.21.15.84:0/2955009821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72040cd0c0 0x7f72040c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.643+0000 7f7208c0e640 1 --2- 172.21.15.84:0/2955009821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72040b0410 0x7f7204155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.643+0000 7f7208c0e640 1 -- 172.21.15.84:0/2955009821 >> 172.21.15.84:0/2955009821 conn(0x7f72040cb410 msgr2=0x7f7204156d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:46.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.643+0000 7f7208c0e640 1 -- 172.21.15.84:0/2955009821 shutdown_connections 2023-10-16T11:22:46.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.643+0000 7f7208c0e640 1 -- 172.21.15.84:0/2955009821 wait complete. 2023-10-16T11:22:46.670 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:22:46.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.872+0000 7f030b305640 1 Processor -- start 2023-10-16T11:22:46.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.873+0000 7f030b305640 1 -- start start 2023-10-16T11:22:46.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.874+0000 7f030b305640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f030415adc0 0x7f03040cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:46.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.874+0000 7f030b305640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0304071380 0x7f030415f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:46.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.874+0000 7f030b305640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03040b4380 con 0x7f03040ad990 2023-10-16T11:22:46.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.874+0000 7f030b305640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03040b4620 con 0x7f030415adc0 2023-10-16T11:22:46.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.874+0000 7f030b305640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f030415e9b0 con 0x7f0304071380 2023-10-16T11:22:46.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.874+0000 7f03090a1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f030415adc0 0x7f03040cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:46.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.875+0000 7f03090a1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f030415adc0 0x7f03040cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38422/0 (socket says 172.21.15.84:38422) 2023-10-16T11:22:46.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.875+0000 7f03090a1640 1 -- 172.21.15.84:0/552235185 learned_addr learned my addr 172.21.15.84:0/552235185 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:46.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.875+0000 7f03098a2640 1 --2- 172.21.15.84:0/552235185 >> v2:172.21.15.157:3300/0 conn(0x7f0304071380 0x7f030415f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:46.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.875+0000 7f03090a1640 1 -- 172.21.15.84:0/552235185 >> v2:172.21.15.157:3300/0 conn(0x7f0304071380 msgr2=0x7f030415f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:46.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.875+0000 7f03090a1640 1 --2- 172.21.15.84:0/552235185 >> v2:172.21.15.157:3300/0 conn(0x7f0304071380 0x7f030415f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.875+0000 7f03090a1640 1 -- 172.21.15.84:0/552235185 >> v1:172.21.15.157:6789/0 conn(0x7f03040ad990 legacy=0x7f0304070c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:46.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.875+0000 7f03090a1640 1 -- 172.21.15.84:0/552235185 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030415ec90 con 0x7f030415adc0 2023-10-16T11:22:46.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.876+0000 7f03090a1640 1 --2- 172.21.15.84:0/552235185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f030415adc0 0x7f03040cf380 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f03040ae270 tx=0x7f030002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f2a6e2918c8532 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:46.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.877+0000 7f02fbfff640 1 -- 172.21.15.84:0/552235185 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0300031870 con 0x7f030415adc0 2023-10-16T11:22:46.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.877+0000 7f02fbfff640 1 -- 172.21.15.84:0/552235185 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03000319d0 con 0x7f030415adc0 2023-10-16T11:22:46.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.877+0000 7f02fbfff640 1 -- 172.21.15.84:0/552235185 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0300031d60 con 0x7f030415adc0 2023-10-16T11:22:46.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.878+0000 7f030b305640 1 -- 172.21.15.84:0/552235185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f030415adc0 msgr2=0x7f03040cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:46.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.878+0000 7f030b305640 1 --2- 172.21.15.84:0/552235185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f030415adc0 0x7f03040cf380 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f03040ae270 tx=0x7f030002b910 comp rx=0 tx=0).stop 2023-10-16T11:22:46.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.878+0000 7f030b305640 1 -- 172.21.15.84:0/552235185 shutdown_connections 2023-10-16T11:22:46.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.878+0000 7f030b305640 1 --2- 172.21.15.84:0/552235185 >> v2:172.21.15.157:3300/0 conn(0x7f0304071380 0x7f030415f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.878+0000 7f030b305640 1 --2- 172.21.15.84:0/552235185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f030415adc0 0x7f03040cf380 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.878+0000 7f030b305640 1 -- 172.21.15.84:0/552235185 >> 172.21.15.84:0/552235185 conn(0x7f03040ce860 msgr2=0x7f03040cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:46.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.878+0000 7f030b305640 1 -- 172.21.15.84:0/552235185 shutdown_connections 2023-10-16T11:22:46.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.878+0000 7f030b305640 1 -- 172.21.15.84:0/552235185 wait complete. 2023-10-16T11:22:46.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.880+0000 7f030b305640 1 Processor -- start 2023-10-16T11:22:46.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.880+0000 7f030b305640 1 -- start start 2023-10-16T11:22:46.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.881+0000 7f030b305640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0304071380 0x7f030414ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:46.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.881+0000 7f030b305640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03040ad990 0x7f03040ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:46.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.881+0000 7f030b305640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f030415adc0 0x7f03040acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:46.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.881+0000 7f030b305640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03040b3ff0 con 0x7f03040ad990 2023-10-16T11:22:46.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.881+0000 7f030b305640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03040c2390 con 0x7f0304071380 2023-10-16T11:22:46.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.881+0000 7f030b305640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030415e2d0 con 0x7f030415adc0 2023-10-16T11:22:46.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.882+0000 7f03088a0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03040ad990 0x7f03040ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:46.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.882+0000 7f03088a0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03040ad990 0x7f03040ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38434/0 (socket says 172.21.15.84:38434) 2023-10-16T11:22:46.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.882+0000 7f03088a0640 1 -- 172.21.15.84:0/2966707758 learned_addr learned my addr 172.21.15.84:0/2966707758 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:46.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.882+0000 7f03090a1640 1 --2- 172.21.15.84:0/2966707758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0304071380 0x7f030414ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:46.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.882+0000 7f03098a2640 1 --2- 172.21.15.84:0/2966707758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f030415adc0 0x7f03040acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:46.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.882+0000 7f03088a0640 1 -- 172.21.15.84:0/2966707758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f030415adc0 msgr2=0x7f03040acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:46.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.882+0000 7f03088a0640 1 --2- 172.21.15.84:0/2966707758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f030415adc0 0x7f03040acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.882+0000 7f03088a0640 1 -- 172.21.15.84:0/2966707758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0304071380 msgr2=0x7f030414ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:46.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.882+0000 7f03088a0640 1 --2- 172.21.15.84:0/2966707758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0304071380 0x7f030414ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:46.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.882+0000 7f03088a0640 1 -- 172.21.15.84:0/2966707758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03000094d0 con 0x7f03040ad990 2023-10-16T11:22:46.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.882+0000 7f03090a1640 1 --2- 172.21.15.84:0/2966707758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0304071380 0x7f030414ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:46.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.882+0000 7f03088a0640 1 --2- 172.21.15.84:0/2966707758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03040ad990 0x7f03040ac890 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f02f400c430 tx=0x7f02f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:46.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.883+0000 7f02f9ffb640 1 -- 172.21.15.84:0/2966707758 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f4011790 con 0x7f03040ad990 2023-10-16T11:22:46.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.883+0000 7f02f9ffb640 1 -- 172.21.15.84:0/2966707758 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f40118f0 con 0x7f03040ad990 2023-10-16T11:22:46.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.883+0000 7f02f9ffb640 1 -- 172.21.15.84:0/2966707758 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f4011ca0 con 0x7f03040ad990 2023-10-16T11:22:46.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.883+0000 7f030b305640 1 -- 172.21.15.84:0/2966707758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f030415b320 con 0x7f03040ad990 2023-10-16T11:22:46.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.883+0000 7f030b305640 1 -- 172.21.15.84:0/2966707758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03040c92c0 con 0x7f03040ad990 2023-10-16T11:22:46.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.884+0000 7f030b305640 1 -- 172.21.15.84:0/2966707758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f030404e160 con 0x7f03040ad990 2023-10-16T11:22:46.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.885+0000 7f02f9ffb640 1 -- 172.21.15.84:0/2966707758 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f02f401c460 con 0x7f03040ad990 2023-10-16T11:22:46.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.886+0000 7f02f9ffb640 1 --2- 172.21.15.84:0/2966707758 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f02e806afc0 0x7f02e806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:46.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.886+0000 7f02f9ffb640 1 -- 172.21.15.84:0/2966707758 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f02f4089690 con 0x7f03040ad990 2023-10-16T11:22:46.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.886+0000 7f03090a1640 1 --2- 172.21.15.84:0/2966707758 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f02e806afc0 0x7f02e806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:46.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.887+0000 7f03090a1640 1 --2- 172.21.15.84:0/2966707758 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f02e806afc0 0x7f02e806d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0300036000 tx=0x7f030003c040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:46.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:46.891+0000 7f02f9ffb640 1 -- 172.21.15.84:0/2966707758 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f02f40546a0 con 0x7f03040ad990 2023-10-16T11:22:47.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.082+0000 7f030b305640 1 -- 172.21.15.84:0/2966707758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03040c5450 con 0x7f03040ad990 2023-10-16T11:22:47.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.083+0000 7f02f9ffb640 1 -- 172.21.15.84:0/2966707758 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v201) v1 ==== 74+0+16599 (secure 0 0 0) 0x7f02f4057cc0 con 0x7f03040ad990 2023-10-16T11:22:47.086 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:47.086 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":201,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:22:44.125184+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:22:40.385524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-16T11:22:41.592928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:22:47.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.086+0000 7f030b305640 1 -- 172.21.15.84:0/2966707758 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f02e806afc0 msgr2=0x7f02e806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:47.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.086+0000 7f030b305640 1 --2- 172.21.15.84:0/2966707758 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f02e806afc0 0x7f02e806d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0300036000 tx=0x7f030003c040 comp rx=0 tx=0).stop 2023-10-16T11:22:47.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.086+0000 7f030b305640 1 -- 172.21.15.84:0/2966707758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03040ad990 msgr2=0x7f03040ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:47.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.086+0000 7f030b305640 1 --2- 172.21.15.84:0/2966707758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03040ad990 0x7f03040ac890 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f02f400c430 tx=0x7f02f400c8f0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.086+0000 7f030b305640 1 -- 172.21.15.84:0/2966707758 shutdown_connections 2023-10-16T11:22:47.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.086+0000 7f030b305640 1 --2- 172.21.15.84:0/2966707758 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f02e806afc0 0x7f02e806d480 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.086+0000 7f030b305640 1 --2- 172.21.15.84:0/2966707758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f030415adc0 0x7f03040acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.086+0000 7f030b305640 1 --2- 172.21.15.84:0/2966707758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03040ad990 0x7f03040ac890 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.086+0000 7f030b305640 1 --2- 172.21.15.84:0/2966707758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0304071380 0x7f030414ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.086+0000 7f030b305640 1 -- 172.21.15.84:0/2966707758 >> 172.21.15.84:0/2966707758 conn(0x7f03040ce860 msgr2=0x7f030406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:47.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.087+0000 7f030b305640 1 -- 172.21.15.84:0/2966707758 shutdown_connections 2023-10-16T11:22:47.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.087+0000 7f030b305640 1 -- 172.21.15.84:0/2966707758 wait complete. 2023-10-16T11:22:47.106 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T11:22:47.107 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:22:47.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.302+0000 7f976c741640 1 Processor -- start 2023-10-16T11:22:47.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.302+0000 7f976c741640 1 -- start start 2023-10-16T11:22:47.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.304+0000 7f976c741640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f976415ae80 0x7f976415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:47.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.304+0000 7f976c741640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f97640ce390 0x7f9764070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:47.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.304+0000 7f976c741640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976415ed60 con 0x7f976415aaa0 2023-10-16T11:22:47.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.304+0000 7f976c741640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f976406b810 con 0x7f97640ce390 2023-10-16T11:22:47.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.304+0000 7f976c741640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97640707e0 con 0x7f976415ae80 2023-10-16T11:22:47.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.304+0000 7f976acde640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f97640ce390 0x7f9764070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:47.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.304+0000 7f976a4dd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f976415aaa0 0x7f97640cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54928/0 (socket says 172.21.15.84:54928) 2023-10-16T11:22:47.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.304+0000 7f976acde640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f97640ce390 0x7f9764070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38446/0 (socket says 172.21.15.84:38446) 2023-10-16T11:22:47.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.304+0000 7f976a4dd640 1 -- 172.21.15.84:0/193162472 learned_addr learned my addr 172.21.15.84:0/193162472 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:47.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.304+0000 7f9769cdc640 1 --2- 172.21.15.84:0/193162472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f976415ae80 0x7f976415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:47.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.305+0000 7f976acde640 1 -- 172.21.15.84:0/193162472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f976415ae80 msgr2=0x7f976415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:47.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.305+0000 7f976acde640 1 --2- 172.21.15.84:0/193162472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f976415ae80 0x7f976415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.305+0000 7f97694db640 1 -- 172.21.15.84:0/193162472 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415491085 0 0) 0x7f976415ed60 con 0x7f976415aaa0 2023-10-16T11:22:47.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.305+0000 7f976acde640 1 -- 172.21.15.84:0/193162472 >> v1:172.21.15.84:6789/0 conn(0x7f976415aaa0 legacy=0x7f97640cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:47.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.305+0000 7f976acde640 1 -- 172.21.15.84:0/193162472 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976415ef90 con 0x7f97640ce390 2023-10-16T11:22:47.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.306+0000 7f976acde640 1 --2- 172.21.15.84:0/193162472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97640ce390 0x7f9764070f50 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f975c009d90 tx=0x7f975c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9b6ed0e4a44f91 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:47.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.306+0000 7f97694db640 1 -- 172.21.15.84:0/193162472 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c02cc60 con 0x7f97640ce390 2023-10-16T11:22:47.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.306+0000 7f97694db640 1 -- 172.21.15.84:0/193162472 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975c02cdc0 con 0x7f97640ce390 2023-10-16T11:22:47.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.306+0000 7f97694db640 1 -- 172.21.15.84:0/193162472 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c034660 con 0x7f97640ce390 2023-10-16T11:22:47.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.307+0000 7f976c741640 1 -- 172.21.15.84:0/193162472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97640ce390 msgr2=0x7f9764070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:47.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.307+0000 7f976c741640 1 --2- 172.21.15.84:0/193162472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97640ce390 0x7f9764070f50 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f975c009d90 tx=0x7f975c02beb0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.307+0000 7f976c741640 1 -- 172.21.15.84:0/193162472 shutdown_connections 2023-10-16T11:22:47.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.307+0000 7f976c741640 1 --2- 172.21.15.84:0/193162472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97640ce390 0x7f9764070f50 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.307+0000 7f976c741640 1 --2- 172.21.15.84:0/193162472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f976415ae80 0x7f976415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.307+0000 7f976c741640 1 -- 172.21.15.84:0/193162472 >> 172.21.15.84:0/193162472 conn(0x7f97640cb410 msgr2=0x7f97640cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:47.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.307+0000 7f976c741640 1 -- 172.21.15.84:0/193162472 shutdown_connections 2023-10-16T11:22:47.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.307+0000 7f976c741640 1 -- 172.21.15.84:0/193162472 wait complete. 2023-10-16T11:22:47.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.309+0000 7f976c741640 1 Processor -- start 2023-10-16T11:22:47.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.309+0000 7f976c741640 1 -- start start 2023-10-16T11:22:47.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.310+0000 7f976c741640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97640ce390 0x7f97640caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:47.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.310+0000 7f976c741640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f976415aaa0 0x7f9764158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:47.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.310+0000 7f976c741640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f976415ae80 0x7f97640c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:47.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.310+0000 7f976c741640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976414f290 con 0x7f976415aaa0 2023-10-16T11:22:47.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.310+0000 7f976c741640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97640c2c10 con 0x7f976415ae80 2023-10-16T11:22:47.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.310+0000 7f976c741640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97640caf40 con 0x7f97640ce390 2023-10-16T11:22:47.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.311+0000 7f9769cdc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f976415aaa0 0x7f9764158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:47.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.311+0000 7f9769cdc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f976415aaa0 0x7f9764158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38448/0 (socket says 172.21.15.84:38448) 2023-10-16T11:22:47.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.311+0000 7f9769cdc640 1 -- 172.21.15.84:0/672322532 learned_addr learned my addr 172.21.15.84:0/672322532 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:47.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.311+0000 7f9769cdc640 1 -- 172.21.15.84:0/672322532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97640ce390 msgr2=0x7f97640caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:47.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.311+0000 7f9769cdc640 1 --2- 172.21.15.84:0/672322532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97640ce390 0x7f97640caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.311+0000 7f9769cdc640 1 -- 172.21.15.84:0/672322532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f976415ae80 msgr2=0x7f97640c2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:47.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.311+0000 7f9769cdc640 1 --2- 172.21.15.84:0/672322532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f976415ae80 0x7f97640c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.311+0000 7f9769cdc640 1 -- 172.21.15.84:0/672322532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975c009a70 con 0x7f976415aaa0 2023-10-16T11:22:47.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.311+0000 7f9769cdc640 1 --2- 172.21.15.84:0/672322532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f976415aaa0 0x7f9764158730 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f975400d800 tx=0x7f975400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:47.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.312+0000 7f975affd640 1 -- 172.21.15.84:0/672322532 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975400b6c0 con 0x7f976415aaa0 2023-10-16T11:22:47.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.312+0000 7f975affd640 1 -- 172.21.15.84:0/672322532 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975400b820 con 0x7f976415aaa0 2023-10-16T11:22:47.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.312+0000 7f976c741640 1 -- 172.21.15.84:0/672322532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97640c24a0 con 0x7f976415aaa0 2023-10-16T11:22:47.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.312+0000 7f975affd640 1 -- 172.21.15.84:0/672322532 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975400bbb0 con 0x7f976415aaa0 2023-10-16T11:22:47.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.312+0000 7f976c741640 1 -- 172.21.15.84:0/672322532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97640c51a0 con 0x7f976415aaa0 2023-10-16T11:22:47.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.313+0000 7f976c741640 1 -- 172.21.15.84:0/672322532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976404e160 con 0x7f976415aaa0 2023-10-16T11:22:47.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.314+0000 7f975affd640 1 -- 172.21.15.84:0/672322532 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f975400bd10 con 0x7f976415aaa0 2023-10-16T11:22:47.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.315+0000 7f975affd640 1 --2- 172.21.15.84:0/672322532 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f974806af90 0x7f974806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:47.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.315+0000 7f975affd640 1 -- 172.21.15.84:0/672322532 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9754089ce0 con 0x7f976415aaa0 2023-10-16T11:22:47.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.315+0000 7f976a4dd640 1 --2- 172.21.15.84:0/672322532 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f974806af90 0x7f974806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:47.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.316+0000 7f976a4dd640 1 --2- 172.21.15.84:0/672322532 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f974806af90 0x7f974806d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9760007fd0 tx=0x7f9760007920 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:47.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.319+0000 7f975affd640 1 -- 172.21.15.84:0/672322532 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9754054cf0 con 0x7f976415aaa0 2023-10-16T11:22:47.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.491+0000 7f976c741640 1 -- 172.21.15.84:0/672322532 --> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f976415f550 con 0x7f974806af90 2023-10-16T11:22:47.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.502+0000 7f975affd640 1 -- 172.21.15.84:0/672322532 <== mgr.10557 v2:172.21.15.84:6832/1985463812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286523 (secure 0 0 0) 0x7f97600070d0 con 0x7f974806af90 2023-10-16T11:22:47.508 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:47.509 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:22:47.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.506+0000 7f976c741640 1 -- 172.21.15.84:0/672322532 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f974806af90 msgr2=0x7f974806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:47.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.507+0000 7f976c741640 1 --2- 172.21.15.84:0/672322532 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f974806af90 0x7f974806d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9760007fd0 tx=0x7f9760007920 comp rx=0 tx=0).stop 2023-10-16T11:22:47.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.507+0000 7f976c741640 1 -- 172.21.15.84:0/672322532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f976415aaa0 msgr2=0x7f9764158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:47.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.507+0000 7f976c741640 1 --2- 172.21.15.84:0/672322532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f976415aaa0 0x7f9764158730 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f975400d800 tx=0x7f975400dcc0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.507+0000 7f976c741640 1 -- 172.21.15.84:0/672322532 shutdown_connections 2023-10-16T11:22:47.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.507+0000 7f976c741640 1 --2- 172.21.15.84:0/672322532 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f974806af90 0x7f974806d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.507+0000 7f976c741640 1 --2- 172.21.15.84:0/672322532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f976415ae80 0x7f97640c2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.507+0000 7f976c741640 1 --2- 172.21.15.84:0/672322532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f976415aaa0 0x7f9764158730 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.507+0000 7f976c741640 1 --2- 172.21.15.84:0/672322532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97640ce390 0x7f97640caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.507+0000 7f976c741640 1 -- 172.21.15.84:0/672322532 >> 172.21.15.84:0/672322532 conn(0x7f97640cb410 msgr2=0x7f97640b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:47.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.507+0000 7f976c741640 1 -- 172.21.15.84:0/672322532 shutdown_connections 2023-10-16T11:22:47.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.507+0000 7f976c741640 1 -- 172.21.15.84:0/672322532 wait complete. 2023-10-16T11:22:47.528 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-16T11:22:47.265038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1984,"num_read_kb":4056,"num_write":585,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2783,"ondisk_log_size":2783,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":370344,"kb_used_data":20904,"kb_used_omap":12,"kb_used_meta":349299,"kb_avail":754604376,"statfs":{"total":773094113280,"available":772714881024,"internally_reserved":0,"allocated":21405696,"data_stored":10939500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":357682745},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001514"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.782334+0000","last_change":"2023-10-16T11:22:43.149474+0000","last_active":"2023-10-16T11:22:44.782334+0000","last_peered":"2023-10-16T11:22:44.782334+0000","last_clean":"2023-10-16T11:22:44.782334+0000","last_became_active":"2023-10-16T11:22:43.144741+0000","last_became_peered":"2023-10-16T11:22:43.144741+0000","last_unstale":"2023-10-16T11:22:44.782334+0000","last_undegraded":"2023-10-16T11:22:44.782334+0000","last_fullsized":"2023-10-16T11:22:44.782334+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:39:40.256126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.778221+0000","last_change":"2023-10-16T11:22:43.153828+0000","last_active":"2023-10-16T11:22:44.778221+0000","last_peered":"2023-10-16T11:22:44.778221+0000","last_clean":"2023-10-16T11:22:44.778221+0000","last_became_active":"2023-10-16T11:22:43.153565+0000","last_became_peered":"2023-10-16T11:22:43.153565+0000","last_unstale":"2023-10-16T11:22:44.778221+0000","last_undegraded":"2023-10-16T11:22:44.778221+0000","last_fullsized":"2023-10-16T11:22:44.778221+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:15:00.818017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.365643+0000","last_change":"2023-10-16T11:22:43.151850+0000","last_active":"2023-10-16T11:22:44.365643+0000","last_peered":"2023-10-16T11:22:44.365643+0000","last_clean":"2023-10-16T11:22:44.365643+0000","last_became_active":"2023-10-16T11:22:43.151265+0000","last_became_peered":"2023-10-16T11:22:43.151265+0000","last_unstale":"2023-10-16T11:22:44.365643+0000","last_undegraded":"2023-10-16T11:22:44.365643+0000","last_fullsized":"2023-10-16T11:22:44.365643+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:49:19.037725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.779278+0000","last_change":"2023-10-16T11:22:43.158178+0000","last_active":"2023-10-16T11:22:44.779278+0000","last_peered":"2023-10-16T11:22:44.779278+0000","last_clean":"2023-10-16T11:22:44.779278+0000","last_became_active":"2023-10-16T11:22:43.157152+0000","last_became_peered":"2023-10-16T11:22:43.157152+0000","last_unstale":"2023-10-16T11:22:44.779278+0000","last_undegraded":"2023-10-16T11:22:44.779278+0000","last_fullsized":"2023-10-16T11:22:44.779278+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:14.278273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.139701+0000","last_change":"2023-10-16T11:22:43.153640+0000","last_active":"2023-10-16T11:22:44.139701+0000","last_peered":"2023-10-16T11:22:44.139701+0000","last_clean":"2023-10-16T11:22:44.139701+0000","last_became_active":"2023-10-16T11:22:43.153068+0000","last_became_peered":"2023-10-16T11:22:43.153068+0000","last_unstale":"2023-10-16T11:22:44.139701+0000","last_undegraded":"2023-10-16T11:22:44.139701+0000","last_fullsized":"2023-10-16T11:22:44.139701+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:30.932439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.365549+0000","last_change":"2023-10-16T11:22:43.147968+0000","last_active":"2023-10-16T11:22:44.365549+0000","last_peered":"2023-10-16T11:22:44.365549+0000","last_clean":"2023-10-16T11:22:44.365549+0000","last_became_active":"2023-10-16T11:22:43.147504+0000","last_became_peered":"2023-10-16T11:22:43.147504+0000","last_unstale":"2023-10-16T11:22:44.365549+0000","last_undegraded":"2023-10-16T11:22:44.365549+0000","last_fullsized":"2023-10-16T11:22:44.365549+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:02:28.333340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.143882+0000","last_change":"2023-10-16T11:22:43.143997+0000","last_active":"2023-10-16T11:22:44.143882+0000","last_peered":"2023-10-16T11:22:44.143882+0000","last_clean":"2023-10-16T11:22:44.143882+0000","last_became_active":"2023-10-16T11:22:43.143687+0000","last_became_peered":"2023-10-16T11:22:43.143687+0000","last_unstale":"2023-10-16T11:22:44.143882+0000","last_undegraded":"2023-10-16T11:22:44.143882+0000","last_fullsized":"2023-10-16T11:22:44.143882+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:18.031947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.143025+0000","last_change":"2023-10-16T11:22:43.141942+0000","last_active":"2023-10-16T11:22:44.143025+0000","last_peered":"2023-10-16T11:22:44.143025+0000","last_clean":"2023-10-16T11:22:44.143025+0000","last_became_active":"2023-10-16T11:22:43.141497+0000","last_became_peered":"2023-10-16T11:22:43.141497+0000","last_unstale":"2023-10-16T11:22:44.143025+0000","last_undegraded":"2023-10-16T11:22:44.143025+0000","last_fullsized":"2023-10-16T11:22:44.143025+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:41:45.318600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.781545+0000","last_change":"2023-10-16T11:22:43.161474+0000","last_active":"2023-10-16T11:22:44.781545+0000","last_peered":"2023-10-16T11:22:44.781545+0000","last_clean":"2023-10-16T11:22:44.781545+0000","last_became_active":"2023-10-16T11:22:43.160437+0000","last_became_peered":"2023-10-16T11:22:43.160437+0000","last_unstale":"2023-10-16T11:22:44.781545+0000","last_undegraded":"2023-10-16T11:22:44.781545+0000","last_fullsized":"2023-10-16T11:22:44.781545+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:46:18.164883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.777974+0000","last_change":"2023-10-16T11:22:43.130871+0000","last_active":"2023-10-16T11:22:44.777974+0000","last_peered":"2023-10-16T11:22:44.777974+0000","last_clean":"2023-10-16T11:22:44.777974+0000","last_became_active":"2023-10-16T11:22:43.128640+0000","last_became_peered":"2023-10-16T11:22:43.128640+0000","last_unstale":"2023-10-16T11:22:44.777974+0000","last_undegraded":"2023-10-16T11:22:44.777974+0000","last_fullsized":"2023-10-16T11:22:44.777974+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:17:16.487265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.145083+0000","last_change":"2023-10-16T11:22:43.148766+0000","last_active":"2023-10-16T11:22:44.145083+0000","last_peered":"2023-10-16T11:22:44.145083+0000","last_clean":"2023-10-16T11:22:44.145083+0000","last_became_active":"2023-10-16T11:22:43.148223+0000","last_became_peered":"2023-10-16T11:22:43.148223+0000","last_unstale":"2023-10-16T11:22:44.145083+0000","last_undegraded":"2023-10-16T11:22:44.145083+0000","last_fullsized":"2023-10-16T11:22:44.145083+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:27:59.982771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.779086+0000","last_change":"2023-10-16T11:22:43.157913+0000","last_active":"2023-10-16T11:22:44.779086+0000","last_peered":"2023-10-16T11:22:44.779086+0000","last_clean":"2023-10-16T11:22:44.779086+0000","last_became_active":"2023-10-16T11:22:43.157647+0000","last_became_peered":"2023-10-16T11:22:43.157647+0000","last_unstale":"2023-10-16T11:22:44.779086+0000","last_undegraded":"2023-10-16T11:22:44.779086+0000","last_fullsized":"2023-10-16T11:22:44.779086+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:41:02.900841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.778243+0000","last_change":"2023-10-16T11:22:43.157581+0000","last_active":"2023-10-16T11:22:44.778243+0000","last_peered":"2023-10-16T11:22:44.778243+0000","last_clean":"2023-10-16T11:22:44.778243+0000","last_became_active":"2023-10-16T11:22:43.157323+0000","last_became_peered":"2023-10-16T11:22:43.157323+0000","last_unstale":"2023-10-16T11:22:44.778243+0000","last_undegraded":"2023-10-16T11:22:44.778243+0000","last_fullsized":"2023-10-16T11:22:44.778243+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:18.076470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.779718+0000","last_change":"2023-10-16T11:22:43.149124+0000","last_active":"2023-10-16T11:22:44.779718+0000","last_peered":"2023-10-16T11:22:44.779718+0000","last_clean":"2023-10-16T11:22:44.779718+0000","last_became_active":"2023-10-16T11:22:43.148819+0000","last_became_peered":"2023-10-16T11:22:43.148819+0000","last_unstale":"2023-10-16T11:22:44.779718+0000","last_undegraded":"2023-10-16T11:22:44.779718+0000","last_fullsized":"2023-10-16T11:22:44.779718+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:38.845724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.778999+0000","last_change":"2023-10-16T11:22:43.157461+0000","last_active":"2023-10-16T11:22:44.778999+0000","last_peered":"2023-10-16T11:22:44.778999+0000","last_clean":"2023-10-16T11:22:44.778999+0000","last_became_active":"2023-10-16T11:22:43.156806+0000","last_became_peered":"2023-10-16T11:22:43.156806+0000","last_unstale":"2023-10-16T11:22:44.778999+0000","last_undegraded":"2023-10-16T11:22:44.778999+0000","last_fullsized":"2023-10-16T11:22:44.778999+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:41.900125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.143039+0000","last_change":"2023-10-16T11:22:43.154539+0000","last_active":"2023-10-16T11:22:44.143039+0000","last_peered":"2023-10-16T11:22:44.143039+0000","last_clean":"2023-10-16T11:22:44.143039+0000","last_became_active":"2023-10-16T11:22:43.154228+0000","last_became_peered":"2023-10-16T11:22:43.154228+0000","last_unstale":"2023-10-16T11:22:44.143039+0000","last_undegraded":"2023-10-16T11:22:44.143039+0000","last_fullsized":"2023-10-16T11:22:44.143039+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:25.932368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.139700+0000","last_change":"2023-10-16T11:22:43.160126+0000","last_active":"2023-10-16T11:22:44.139700+0000","last_peered":"2023-10-16T11:22:44.139700+0000","last_clean":"2023-10-16T11:22:44.139700+0000","last_became_active":"2023-10-16T11:22:43.159776+0000","last_became_peered":"2023-10-16T11:22:43.159776+0000","last_unstale":"2023-10-16T11:22:44.139700+0000","last_undegraded":"2023-10-16T11:22:44.139700+0000","last_fullsized":"2023-10-16T11:22:44.139700+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:00:27.289128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.782077+0000","last_change":"2023-10-16T11:22:43.162922+0000","last_active":"2023-10-16T11:22:44.782077+0000","last_peered":"2023-10-16T11:22:44.782077+0000","last_clean":"2023-10-16T11:22:44.782077+0000","last_became_active":"2023-10-16T11:22:43.162388+0000","last_became_peered":"2023-10-16T11:22:43.162388+0000","last_unstale":"2023-10-16T11:22:44.782077+0000","last_undegraded":"2023-10-16T11:22:44.782077+0000","last_fullsized":"2023-10-16T11:22:44.782077+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:57.609751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.777484+0000","last_change":"2023-10-16T11:22:43.152352+0000","last_active":"2023-10-16T11:22:44.777484+0000","last_peered":"2023-10-16T11:22:44.777484+0000","last_clean":"2023-10-16T11:22:44.777484+0000","last_became_active":"2023-10-16T11:22:43.151971+0000","last_became_peered":"2023-10-16T11:22:43.151971+0000","last_unstale":"2023-10-16T11:22:44.777484+0000","last_undegraded":"2023-10-16T11:22:44.777484+0000","last_fullsized":"2023-10-16T11:22:44.777484+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:27.191744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.139381+0000","last_change":"2023-10-16T11:22:43.157919+0000","last_active":"2023-10-16T11:22:44.139381+0000","last_peered":"2023-10-16T11:22:44.139381+0000","last_clean":"2023-10-16T11:22:44.139381+0000","last_became_active":"2023-10-16T11:22:43.157396+0000","last_became_peered":"2023-10-16T11:22:43.157396+0000","last_unstale":"2023-10-16T11:22:44.139381+0000","last_undegraded":"2023-10-16T11:22:44.139381+0000","last_fullsized":"2023-10-16T11:22:44.139381+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:27.191744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.139815+0000","last_change":"2023-10-16T11:22:43.152444+0000","last_active":"2023-10-16T11:22:44.139815+0000","last_peered":"2023-10-16T11:22:44.139815+0000","last_clean":"2023-10-16T11:22:44.139815+0000","last_became_active":"2023-10-16T11:22:43.151892+0000","last_became_peered":"2023-10-16T11:22:43.151892+0000","last_unstale":"2023-10-16T11:22:44.139815+0000","last_undegraded":"2023-10-16T11:22:44.139815+0000","last_fullsized":"2023-10-16T11:22:44.139815+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:30.024400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.777315+0000","last_change":"2023-10-16T11:22:43.141916+0000","last_active":"2023-10-16T11:22:44.777315+0000","last_peered":"2023-10-16T11:22:44.777315+0000","last_clean":"2023-10-16T11:22:44.777315+0000","last_became_active":"2023-10-16T11:22:43.141616+0000","last_became_peered":"2023-10-16T11:22:43.141616+0000","last_unstale":"2023-10-16T11:22:44.777315+0000","last_undegraded":"2023-10-16T11:22:44.777315+0000","last_fullsized":"2023-10-16T11:22:44.777315+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:10:36.929431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.365542+0000","last_change":"2023-10-16T11:22:43.136522+0000","last_active":"2023-10-16T11:22:44.365542+0000","last_peered":"2023-10-16T11:22:44.365542+0000","last_clean":"2023-10-16T11:22:44.365542+0000","last_became_active":"2023-10-16T11:22:43.135779+0000","last_became_peered":"2023-10-16T11:22:43.135779+0000","last_unstale":"2023-10-16T11:22:44.365542+0000","last_undegraded":"2023-10-16T11:22:44.365542+0000","last_fullsized":"2023-10-16T11:22:44.365542+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:19:40.726706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.143438+0000","last_change":"2023-10-16T11:22:43.141753+0000","last_active":"2023-10-16T11:22:44.143438+0000","last_peered":"2023-10-16T11:22:44.143438+0000","last_clean":"2023-10-16T11:22:44.143438+0000","last_became_active":"2023-10-16T11:22:43.140601+0000","last_became_peered":"2023-10-16T11:22:43.140601+0000","last_unstale":"2023-10-16T11:22:44.143438+0000","last_undegraded":"2023-10-16T11:22:44.143438+0000","last_fullsized":"2023-10-16T11:22:44.143438+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:57.628121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.779802+0000","last_change":"2023-10-16T11:22:43.155429+0000","last_active":"2023-10-16T11:22:44.779802+0000","last_peered":"2023-10-16T11:22:44.779802+0000","last_clean":"2023-10-16T11:22:44.779802+0000","last_became_active":"2023-10-16T11:22:43.151462+0000","last_became_peered":"2023-10-16T11:22:43.151462+0000","last_unstale":"2023-10-16T11:22:44.779802+0000","last_undegraded":"2023-10-16T11:22:44.779802+0000","last_fullsized":"2023-10-16T11:22:44.779802+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:21.406202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.144741+0000","last_change":"2023-10-16T11:22:43.144609+0000","last_active":"2023-10-16T11:22:44.144741+0000","last_peered":"2023-10-16T11:22:44.144741+0000","last_clean":"2023-10-16T11:22:44.144741+0000","last_became_active":"2023-10-16T11:22:43.144352+0000","last_became_peered":"2023-10-16T11:22:43.144352+0000","last_unstale":"2023-10-16T11:22:44.144741+0000","last_undegraded":"2023-10-16T11:22:44.144741+0000","last_fullsized":"2023-10-16T11:22:44.144741+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:29:23.693408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.365792+0000","last_change":"2023-10-16T11:22:43.144593+0000","last_active":"2023-10-16T11:22:44.365792+0000","last_peered":"2023-10-16T11:22:44.365792+0000","last_clean":"2023-10-16T11:22:44.365792+0000","last_became_active":"2023-10-16T11:22:43.144241+0000","last_became_peered":"2023-10-16T11:22:43.144241+0000","last_unstale":"2023-10-16T11:22:44.365792+0000","last_undegraded":"2023-10-16T11:22:44.365792+0000","last_fullsized":"2023-10-16T11:22:44.365792+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:14:56.763801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.139511+0000","last_change":"2023-10-16T11:22:43.127494+0000","last_active":"2023-10-16T11:22:44.139511+0000","last_peered":"2023-10-16T11:22:44.139511+0000","last_clean":"2023-10-16T11:22:44.139511+0000","last_became_active":"2023-10-16T11:22:43.127215+0000","last_became_peered":"2023-10-16T11:22:43.127215+0000","last_unstale":"2023-10-16T11:22:44.139511+0000","last_undegraded":"2023-10-16T11:22:44.139511+0000","last_fullsized":"2023-10-16T11:22:44.139511+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:00:57.360176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.778593+0000","last_change":"2023-10-16T11:22:43.161993+0000","last_active":"2023-10-16T11:22:44.778593+0000","last_peered":"2023-10-16T11:22:44.778593+0000","last_clean":"2023-10-16T11:22:44.778593+0000","last_became_active":"2023-10-16T11:22:43.161573+0000","last_became_peered":"2023-10-16T11:22:43.161573+0000","last_unstale":"2023-10-16T11:22:44.778593+0000","last_undegraded":"2023-10-16T11:22:44.778593+0000","last_fullsized":"2023-10-16T11:22:44.778593+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:19:43.925889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.780097+0000","last_change":"2023-10-16T11:22:43.151736+0000","last_active":"2023-10-16T11:22:44.780097+0000","last_peered":"2023-10-16T11:22:44.780097+0000","last_clean":"2023-10-16T11:22:44.780097+0000","last_became_active":"2023-10-16T11:22:43.151465+0000","last_became_peered":"2023-10-16T11:22:43.151465+0000","last_unstale":"2023-10-16T11:22:44.780097+0000","last_undegraded":"2023-10-16T11:22:44.780097+0000","last_fullsized":"2023-10-16T11:22:44.780097+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:29:23.693408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.365703+0000","last_change":"2023-10-16T11:22:43.159343+0000","last_active":"2023-10-16T11:22:44.365703+0000","last_peered":"2023-10-16T11:22:44.365703+0000","last_clean":"2023-10-16T11:22:44.365703+0000","last_became_active":"2023-10-16T11:22:43.158982+0000","last_became_peered":"2023-10-16T11:22:43.158982+0000","last_unstale":"2023-10-16T11:22:44.365703+0000","last_undegraded":"2023-10-16T11:22:44.365703+0000","last_fullsized":"2023-10-16T11:22:44.365703+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:58.410334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.139969+0000","last_change":"2023-10-16T11:22:43.127397+0000","last_active":"2023-10-16T11:22:44.139969+0000","last_peered":"2023-10-16T11:22:44.139969+0000","last_clean":"2023-10-16T11:22:44.139969+0000","last_became_active":"2023-10-16T11:22:43.127128+0000","last_became_peered":"2023-10-16T11:22:43.127128+0000","last_unstale":"2023-10-16T11:22:44.139969+0000","last_undegraded":"2023-10-16T11:22:44.139969+0000","last_fullsized":"2023-10-16T11:22:44.139969+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:39:55.469848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.143850+0000","last_change":"2023-10-16T11:22:43.150123+0000","last_active":"2023-10-16T11:22:44.143850+0000","last_peered":"2023-10-16T11:22:44.143850+0000","last_clean":"2023-10-16T11:22:44.143850+0000","last_became_active":"2023-10-16T11:22:43.149862+0000","last_became_peered":"2023-10-16T11:22:43.149862+0000","last_unstale":"2023-10-16T11:22:44.143850+0000","last_undegraded":"2023-10-16T11:22:44.143850+0000","last_fullsized":"2023-10-16T11:22:44.143850+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:57:38.033406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"200'4","reported_seq":16,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.153780+0000","last_change":"2023-10-16T11:22:42.142006+0000","last_active":"2023-10-16T11:22:43.153780+0000","last_peered":"2023-10-16T11:22:43.153780+0000","last_clean":"2023-10-16T11:22:43.153780+0000","last_became_active":"2023-10-16T11:22:42.141599+0000","last_became_peered":"2023-10-16T11:22:42.141599+0000","last_unstale":"2023-10-16T11:22:43.153780+0000","last_undegraded":"2023-10-16T11:22:43.153780+0000","last_fullsized":"2023-10-16T11:22:43.153780+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:31:53.479584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.778321+0000","last_change":"2023-10-16T11:22:43.136208+0000","last_active":"2023-10-16T11:22:44.778321+0000","last_peered":"2023-10-16T11:22:44.778321+0000","last_clean":"2023-10-16T11:22:44.778321+0000","last_became_active":"2023-10-16T11:22:43.135583+0000","last_became_peered":"2023-10-16T11:22:43.135583+0000","last_unstale":"2023-10-16T11:22:44.778321+0000","last_undegraded":"2023-10-16T11:22:44.778321+0000","last_fullsized":"2023-10-16T11:22:44.778321+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:30.932439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"200'1","reported_seq":13,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.163784+0000","last_change":"2023-10-16T11:22:42.143922+0000","last_active":"2023-10-16T11:22:43.163784+0000","last_peered":"2023-10-16T11:22:43.163784+0000","last_clean":"2023-10-16T11:22:43.163784+0000","last_became_active":"2023-10-16T11:22:42.143630+0000","last_became_peered":"2023-10-16T11:22:42.143630+0000","last_unstale":"2023-10-16T11:22:43.163784+0000","last_undegraded":"2023-10-16T11:22:43.163784+0000","last_fullsized":"2023-10-16T11:22:43.163784+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:04.995252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.144350+0000","last_change":"2023-10-16T11:22:43.135610+0000","last_active":"2023-10-16T11:22:44.144350+0000","last_peered":"2023-10-16T11:22:44.144350+0000","last_clean":"2023-10-16T11:22:44.144350+0000","last_became_active":"2023-10-16T11:22:43.134534+0000","last_became_peered":"2023-10-16T11:22:43.134534+0000","last_unstale":"2023-10-16T11:22:44.144350+0000","last_undegraded":"2023-10-16T11:22:44.144350+0000","last_fullsized":"2023-10-16T11:22:44.144350+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:38.845724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"200'2","reported_seq":14,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.134884+0000","last_change":"2023-10-16T11:22:42.135404+0000","last_active":"2023-10-16T11:22:43.134884+0000","last_peered":"2023-10-16T11:22:43.134884+0000","last_clean":"2023-10-16T11:22:43.134884+0000","last_became_active":"2023-10-16T11:22:42.134875+0000","last_became_peered":"2023-10-16T11:22:42.134875+0000","last_unstale":"2023-10-16T11:22:43.134884+0000","last_undegraded":"2023-10-16T11:22:43.134884+0000","last_fullsized":"2023-10-16T11:22:43.134884+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:37.827150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.781042+0000","last_change":"2023-10-16T11:22:43.146755+0000","last_active":"2023-10-16T11:22:44.781042+0000","last_peered":"2023-10-16T11:22:44.781042+0000","last_clean":"2023-10-16T11:22:44.781042+0000","last_became_active":"2023-10-16T11:22:43.146367+0000","last_became_peered":"2023-10-16T11:22:43.146367+0000","last_unstale":"2023-10-16T11:22:44.781042+0000","last_undegraded":"2023-10-16T11:22:44.781042+0000","last_fullsized":"2023-10-16T11:22:44.781042+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:27:59.982771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"200'2","reported_seq":14,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.163307+0000","last_change":"2023-10-16T11:22:42.146169+0000","last_active":"2023-10-16T11:22:43.163307+0000","last_peered":"2023-10-16T11:22:43.163307+0000","last_clean":"2023-10-16T11:22:43.163307+0000","last_became_active":"2023-10-16T11:22:42.145790+0000","last_became_peered":"2023-10-16T11:22:42.145790+0000","last_unstale":"2023-10-16T11:22:43.163307+0000","last_undegraded":"2023-10-16T11:22:43.163307+0000","last_fullsized":"2023-10-16T11:22:43.163307+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:47:31.440178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.777405+0000","last_change":"2023-10-16T11:22:43.141311+0000","last_active":"2023-10-16T11:22:44.777405+0000","last_peered":"2023-10-16T11:22:44.777405+0000","last_clean":"2023-10-16T11:22:44.777405+0000","last_became_active":"2023-10-16T11:22:43.139288+0000","last_became_peered":"2023-10-16T11:22:43.139288+0000","last_unstale":"2023-10-16T11:22:44.777405+0000","last_undegraded":"2023-10-16T11:22:44.777405+0000","last_fullsized":"2023-10-16T11:22:44.777405+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:14.840220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"200'1","reported_seq":13,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.141765+0000","last_change":"2023-10-16T11:22:42.133138+0000","last_active":"2023-10-16T11:22:43.141765+0000","last_peered":"2023-10-16T11:22:43.141765+0000","last_clean":"2023-10-16T11:22:43.141765+0000","last_became_active":"2023-10-16T11:22:42.131907+0000","last_became_peered":"2023-10-16T11:22:42.131907+0000","last_unstale":"2023-10-16T11:22:43.141765+0000","last_undegraded":"2023-10-16T11:22:43.141765+0000","last_fullsized":"2023-10-16T11:22:43.141765+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:10:35.910857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.143394+0000","last_change":"2023-10-16T11:22:43.157137+0000","last_active":"2023-10-16T11:22:44.143394+0000","last_peered":"2023-10-16T11:22:44.143394+0000","last_clean":"2023-10-16T11:22:44.143394+0000","last_became_active":"2023-10-16T11:22:43.156523+0000","last_became_peered":"2023-10-16T11:22:43.156523+0000","last_unstale":"2023-10-16T11:22:44.143394+0000","last_undegraded":"2023-10-16T11:22:44.143394+0000","last_fullsized":"2023-10-16T11:22:44.143394+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:06:20.008986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"200'1","reported_seq":13,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.138395+0000","last_change":"2023-10-16T11:22:42.124819+0000","last_active":"2023-10-16T11:22:43.138395+0000","last_peered":"2023-10-16T11:22:43.138395+0000","last_clean":"2023-10-16T11:22:43.138395+0000","last_became_active":"2023-10-16T11:22:42.124531+0000","last_became_peered":"2023-10-16T11:22:42.124531+0000","last_unstale":"2023-10-16T11:22:43.138395+0000","last_undegraded":"2023-10-16T11:22:43.138395+0000","last_fullsized":"2023-10-16T11:22:43.138395+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:41.901577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.143949+0000","last_change":"2023-10-16T11:22:43.154104+0000","last_active":"2023-10-16T11:22:44.143949+0000","last_peered":"2023-10-16T11:22:44.143949+0000","last_clean":"2023-10-16T11:22:44.143949+0000","last_became_active":"2023-10-16T11:22:43.152997+0000","last_became_peered":"2023-10-16T11:22:43.152997+0000","last_unstale":"2023-10-16T11:22:44.143949+0000","last_undegraded":"2023-10-16T11:22:44.143949+0000","last_fullsized":"2023-10-16T11:22:44.143949+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:51.502199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.134025+0000","last_change":"2023-10-16T11:22:42.139731+0000","last_active":"2023-10-16T11:22:43.134025+0000","last_peered":"2023-10-16T11:22:43.134025+0000","last_clean":"2023-10-16T11:22:43.134025+0000","last_became_active":"2023-10-16T11:22:42.139416+0000","last_became_peered":"2023-10-16T11:22:42.139416+0000","last_unstale":"2023-10-16T11:22:43.134025+0000","last_undegraded":"2023-10-16T11:22:43.134025+0000","last_fullsized":"2023-10-16T11:22:43.134025+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:24.913795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.366282+0000","last_change":"2023-10-16T11:22:43.137756+0000","last_active":"2023-10-16T11:22:44.366282+0000","last_peered":"2023-10-16T11:22:44.366282+0000","last_clean":"2023-10-16T11:22:44.366282+0000","last_became_active":"2023-10-16T11:22:43.137384+0000","last_became_peered":"2023-10-16T11:22:43.137384+0000","last_unstale":"2023-10-16T11:22:44.366282+0000","last_undegraded":"2023-10-16T11:22:44.366282+0000","last_fullsized":"2023-10-16T11:22:44.366282+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:51.998114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.127361+0000","last_change":"2023-10-16T11:22:42.144254+0000","last_active":"2023-10-16T11:22:43.127361+0000","last_peered":"2023-10-16T11:22:43.127361+0000","last_clean":"2023-10-16T11:22:43.127361+0000","last_became_active":"2023-10-16T11:22:42.143949+0000","last_became_peered":"2023-10-16T11:22:42.143949+0000","last_unstale":"2023-10-16T11:22:43.127361+0000","last_undegraded":"2023-10-16T11:22:43.127361+0000","last_fullsized":"2023-10-16T11:22:43.127361+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:39:54.451275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.143933+0000","last_change":"2023-10-16T11:22:43.145118+0000","last_active":"2023-10-16T11:22:44.143933+0000","last_peered":"2023-10-16T11:22:44.143933+0000","last_clean":"2023-10-16T11:22:44.143933+0000","last_became_active":"2023-10-16T11:22:43.144858+0000","last_became_peered":"2023-10-16T11:22:43.144858+0000","last_unstale":"2023-10-16T11:22:44.143933+0000","last_undegraded":"2023-10-16T11:22:44.143933+0000","last_fullsized":"2023-10-16T11:22:44.143933+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:19:52.601828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.116915+0000","last_change":"2023-10-16T11:22:42.137290+0000","last_active":"2023-10-16T11:22:43.116915+0000","last_peered":"2023-10-16T11:22:43.116915+0000","last_clean":"2023-10-16T11:22:43.116915+0000","last_became_active":"2023-10-16T11:22:42.136782+0000","last_became_peered":"2023-10-16T11:22:42.136782+0000","last_unstale":"2023-10-16T11:22:43.116915+0000","last_undegraded":"2023-10-16T11:22:43.116915+0000","last_fullsized":"2023-10-16T11:22:43.116915+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:16.132848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.144303+0000","last_change":"2023-10-16T11:22:43.150241+0000","last_active":"2023-10-16T11:22:44.144303+0000","last_peered":"2023-10-16T11:22:44.144303+0000","last_clean":"2023-10-16T11:22:44.144303+0000","last_became_active":"2023-10-16T11:22:43.149428+0000","last_became_peered":"2023-10-16T11:22:43.149428+0000","last_unstale":"2023-10-16T11:22:44.144303+0000","last_undegraded":"2023-10-16T11:22:44.144303+0000","last_fullsized":"2023-10-16T11:22:44.144303+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:05.826888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"200'3","reported_seq":15,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.145459+0000","last_change":"2023-10-16T11:22:42.126340+0000","last_active":"2023-10-16T11:22:43.145459+0000","last_peered":"2023-10-16T11:22:43.145459+0000","last_clean":"2023-10-16T11:22:43.145459+0000","last_became_active":"2023-10-16T11:22:42.126054+0000","last_became_peered":"2023-10-16T11:22:42.126054+0000","last_unstale":"2023-10-16T11:22:43.145459+0000","last_undegraded":"2023-10-16T11:22:43.145459+0000","last_fullsized":"2023-10-16T11:22:43.145459+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:35:26.282525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.777767+0000","last_change":"2023-10-16T11:22:43.141026+0000","last_active":"2023-10-16T11:22:44.777767+0000","last_peered":"2023-10-16T11:22:44.777767+0000","last_clean":"2023-10-16T11:22:44.777767+0000","last_became_active":"2023-10-16T11:22:43.139354+0000","last_became_peered":"2023-10-16T11:22:43.139354+0000","last_unstale":"2023-10-16T11:22:44.777767+0000","last_undegraded":"2023-10-16T11:22:44.777767+0000","last_fullsized":"2023-10-16T11:22:44.777767+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:00:57.360176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"200'6","reported_seq":18,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.148664+0000","last_change":"2023-10-16T11:22:42.142251+0000","last_active":"2023-10-16T11:22:43.148664+0000","last_peered":"2023-10-16T11:22:43.148664+0000","last_clean":"2023-10-16T11:22:43.148664+0000","last_became_active":"2023-10-16T11:22:42.141801+0000","last_became_peered":"2023-10-16T11:22:42.141801+0000","last_unstale":"2023-10-16T11:22:43.148664+0000","last_undegraded":"2023-10-16T11:22:43.148664+0000","last_fullsized":"2023-10-16T11:22:43.148664+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:04.808314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.366111+0000","last_change":"2023-10-16T11:22:43.136318+0000","last_active":"2023-10-16T11:22:44.366111+0000","last_peered":"2023-10-16T11:22:44.366111+0000","last_clean":"2023-10-16T11:22:44.366111+0000","last_became_active":"2023-10-16T11:22:43.136054+0000","last_became_peered":"2023-10-16T11:22:43.136054+0000","last_unstale":"2023-10-16T11:22:44.366111+0000","last_undegraded":"2023-10-16T11:22:44.366111+0000","last_fullsized":"2023-10-16T11:22:44.366111+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:31.448819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"200'2","reported_seq":14,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.153214+0000","last_change":"2023-10-16T11:22:42.141840+0000","last_active":"2023-10-16T11:22:43.153214+0000","last_peered":"2023-10-16T11:22:43.153214+0000","last_clean":"2023-10-16T11:22:43.153214+0000","last_became_active":"2023-10-16T11:22:42.141513+0000","last_became_peered":"2023-10-16T11:22:42.141513+0000","last_unstale":"2023-10-16T11:22:43.153214+0000","last_undegraded":"2023-10-16T11:22:43.153214+0000","last_fullsized":"2023-10-16T11:22:43.153214+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:57:37.014832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.144996+0000","last_change":"2023-10-16T11:22:43.134668+0000","last_active":"2023-10-16T11:22:44.144996+0000","last_peered":"2023-10-16T11:22:44.144996+0000","last_clean":"2023-10-16T11:22:44.144996+0000","last_became_active":"2023-10-16T11:22:43.134348+0000","last_became_peered":"2023-10-16T11:22:43.134348+0000","last_unstale":"2023-10-16T11:22:44.144996+0000","last_undegraded":"2023-10-16T11:22:44.144996+0000","last_fullsized":"2023-10-16T11:22:44.144996+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:46:18.164883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.118395+0000","last_change":"2023-10-16T11:22:42.130286+0000","last_active":"2023-10-16T11:22:43.118395+0000","last_peered":"2023-10-16T11:22:43.118395+0000","last_clean":"2023-10-16T11:22:43.118395+0000","last_became_active":"2023-10-16T11:22:42.129097+0000","last_became_peered":"2023-10-16T11:22:42.129097+0000","last_unstale":"2023-10-16T11:22:43.118395+0000","last_undegraded":"2023-10-16T11:22:43.118395+0000","last_fullsized":"2023-10-16T11:22:43.118395+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:03:11.366187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.778170+0000","last_change":"2023-10-16T11:22:43.150104+0000","last_active":"2023-10-16T11:22:44.778170+0000","last_peered":"2023-10-16T11:22:44.778170+0000","last_clean":"2023-10-16T11:22:44.778170+0000","last_became_active":"2023-10-16T11:22:43.149844+0000","last_became_peered":"2023-10-16T11:22:43.149844+0000","last_unstale":"2023-10-16T11:22:44.778170+0000","last_undegraded":"2023-10-16T11:22:44.778170+0000","last_fullsized":"2023-10-16T11:22:44.778170+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:05.826888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.124390+0000","last_change":"2023-10-16T11:22:42.132900+0000","last_active":"2023-10-16T11:22:43.124390+0000","last_peered":"2023-10-16T11:22:43.124390+0000","last_clean":"2023-10-16T11:22:43.124390+0000","last_became_active":"2023-10-16T11:22:42.132561+0000","last_became_peered":"2023-10-16T11:22:42.132561+0000","last_unstale":"2023-10-16T11:22:43.124390+0000","last_undegraded":"2023-10-16T11:22:43.124390+0000","last_fullsized":"2023-10-16T11:22:43.124390+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:49.297250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.777475+0000","last_change":"2023-10-16T11:22:43.130692+0000","last_active":"2023-10-16T11:22:44.777475+0000","last_peered":"2023-10-16T11:22:44.777475+0000","last_clean":"2023-10-16T11:22:44.777475+0000","last_became_active":"2023-10-16T11:22:43.128801+0000","last_became_peered":"2023-10-16T11:22:43.128801+0000","last_unstale":"2023-10-16T11:22:44.777475+0000","last_undegraded":"2023-10-16T11:22:44.777475+0000","last_fullsized":"2023-10-16T11:22:44.777475+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:35:27.301099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"200'8","reported_seq":20,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.157499+0000","last_change":"2023-10-16T11:22:42.139889+0000","last_active":"2023-10-16T11:22:43.157499+0000","last_peered":"2023-10-16T11:22:43.157499+0000","last_clean":"2023-10-16T11:22:43.157499+0000","last_became_active":"2023-10-16T11:22:42.138021+0000","last_became_peered":"2023-10-16T11:22:42.138021+0000","last_unstale":"2023-10-16T11:22:43.157499+0000","last_undegraded":"2023-10-16T11:22:43.157499+0000","last_fullsized":"2023-10-16T11:22:43.157499+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:06:18.990412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.778209+0000","last_change":"2023-10-16T11:22:43.133379+0000","last_active":"2023-10-16T11:22:44.778209+0000","last_peered":"2023-10-16T11:22:44.778209+0000","last_clean":"2023-10-16T11:22:44.778209+0000","last_became_active":"2023-10-16T11:22:43.132886+0000","last_became_peered":"2023-10-16T11:22:43.132886+0000","last_unstale":"2023-10-16T11:22:44.778209+0000","last_undegraded":"2023-10-16T11:22:44.778209+0000","last_fullsized":"2023-10-16T11:22:44.778209+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:57:39.650999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"200'1","reported_seq":13,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.141646+0000","last_change":"2023-10-16T11:22:42.134657+0000","last_active":"2023-10-16T11:22:43.141646+0000","last_peered":"2023-10-16T11:22:43.141646+0000","last_clean":"2023-10-16T11:22:43.141646+0000","last_became_active":"2023-10-16T11:22:42.134388+0000","last_became_peered":"2023-10-16T11:22:42.134388+0000","last_unstale":"2023-10-16T11:22:43.141646+0000","last_undegraded":"2023-10-16T11:22:43.141646+0000","last_fullsized":"2023-10-16T11:22:43.141646+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:34.656291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.781557+0000","last_change":"2023-10-16T11:22:43.147584+0000","last_active":"2023-10-16T11:22:44.781557+0000","last_peered":"2023-10-16T11:22:44.781557+0000","last_clean":"2023-10-16T11:22:44.781557+0000","last_became_active":"2023-10-16T11:22:43.147307+0000","last_became_peered":"2023-10-16T11:22:43.147307+0000","last_unstale":"2023-10-16T11:22:44.781557+0000","last_undegraded":"2023-10-16T11:22:44.781557+0000","last_fullsized":"2023-10-16T11:22:44.781557+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:19:40.726706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"200'2","reported_seq":14,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.147493+0000","last_change":"2023-10-16T11:22:42.144754+0000","last_active":"2023-10-16T11:22:43.147493+0000","last_peered":"2023-10-16T11:22:43.147493+0000","last_clean":"2023-10-16T11:22:43.147493+0000","last_became_active":"2023-10-16T11:22:42.144507+0000","last_became_peered":"2023-10-16T11:22:42.144507+0000","last_unstale":"2023-10-16T11:22:43.147493+0000","last_undegraded":"2023-10-16T11:22:43.147493+0000","last_fullsized":"2023-10-16T11:22:43.147493+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:00.711474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.780587+0000","last_change":"2023-10-16T11:22:43.155065+0000","last_active":"2023-10-16T11:22:44.780587+0000","last_peered":"2023-10-16T11:22:44.780587+0000","last_clean":"2023-10-16T11:22:44.780587+0000","last_became_active":"2023-10-16T11:22:43.153416+0000","last_became_peered":"2023-10-16T11:22:43.153416+0000","last_unstale":"2023-10-16T11:22:44.780587+0000","last_undegraded":"2023-10-16T11:22:44.780587+0000","last_fullsized":"2023-10-16T11:22:44.780587+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:39:10.429029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"200'1","reported_seq":13,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.147374+0000","last_change":"2023-10-16T11:22:42.139840+0000","last_active":"2023-10-16T11:22:43.147374+0000","last_peered":"2023-10-16T11:22:43.147374+0000","last_clean":"2023-10-16T11:22:43.147374+0000","last_became_active":"2023-10-16T11:22:42.139543+0000","last_became_peered":"2023-10-16T11:22:42.139543+0000","last_unstale":"2023-10-16T11:22:43.147374+0000","last_undegraded":"2023-10-16T11:22:43.147374+0000","last_fullsized":"2023-10-16T11:22:43.147374+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:24.457369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.778590+0000","last_change":"2023-10-16T11:22:43.154863+0000","last_active":"2023-10-16T11:22:44.778590+0000","last_peered":"2023-10-16T11:22:44.778590+0000","last_clean":"2023-10-16T11:22:44.778590+0000","last_became_active":"2023-10-16T11:22:43.154592+0000","last_became_peered":"2023-10-16T11:22:43.154592+0000","last_unstale":"2023-10-16T11:22:44.778590+0000","last_undegraded":"2023-10-16T11:22:44.778590+0000","last_fullsized":"2023-10-16T11:22:44.778590+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:32:04.649162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.127704+0000","last_change":"2023-10-16T11:22:42.141538+0000","last_active":"2023-10-16T11:22:43.127704+0000","last_peered":"2023-10-16T11:22:43.127704+0000","last_clean":"2023-10-16T11:22:43.127704+0000","last_became_active":"2023-10-16T11:22:42.140570+0000","last_became_peered":"2023-10-16T11:22:42.140570+0000","last_unstale":"2023-10-16T11:22:43.127704+0000","last_undegraded":"2023-10-16T11:22:43.127704+0000","last_fullsized":"2023-10-16T11:22:43.127704+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:48.644799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.777678+0000","last_change":"2023-10-16T11:22:43.124735+0000","last_active":"2023-10-16T11:22:44.777678+0000","last_peered":"2023-10-16T11:22:44.777678+0000","last_clean":"2023-10-16T11:22:44.777678+0000","last_became_active":"2023-10-16T11:22:43.124478+0000","last_became_peered":"2023-10-16T11:22:43.124478+0000","last_unstale":"2023-10-16T11:22:44.777678+0000","last_undegraded":"2023-10-16T11:22:44.777678+0000","last_fullsized":"2023-10-16T11:22:44.777678+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:00.954663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"200'2","reported_seq":14,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.132482+0000","last_change":"2023-10-16T11:22:42.126251+0000","last_active":"2023-10-16T11:22:43.132482+0000","last_peered":"2023-10-16T11:22:43.132482+0000","last_clean":"2023-10-16T11:22:43.132482+0000","last_became_active":"2023-10-16T11:22:42.125897+0000","last_became_peered":"2023-10-16T11:22:42.125897+0000","last_unstale":"2023-10-16T11:22:43.132482+0000","last_undegraded":"2023-10-16T11:22:43.132482+0000","last_fullsized":"2023-10-16T11:22:43.132482+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:48.644799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.144792+0000","last_change":"2023-10-16T11:22:43.141863+0000","last_active":"2023-10-16T11:22:44.144792+0000","last_peered":"2023-10-16T11:22:44.144792+0000","last_clean":"2023-10-16T11:22:44.144792+0000","last_became_active":"2023-10-16T11:22:43.141584+0000","last_became_peered":"2023-10-16T11:22:43.141584+0000","last_unstale":"2023-10-16T11:22:44.144792+0000","last_undegraded":"2023-10-16T11:22:44.144792+0000","last_fullsized":"2023-10-16T11:22:44.144792+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:39:10.429029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.114847+0000","last_change":"2023-10-16T11:22:42.138321+0000","last_active":"2023-10-16T11:22:43.114847+0000","last_peered":"2023-10-16T11:22:43.114847+0000","last_clean":"2023-10-16T11:22:43.114847+0000","last_became_active":"2023-10-16T11:22:42.137996+0000","last_became_peered":"2023-10-16T11:22:42.137996+0000","last_unstale":"2023-10-16T11:22:43.114847+0000","last_undegraded":"2023-10-16T11:22:43.114847+0000","last_fullsized":"2023-10-16T11:22:43.114847+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:01:45.670632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.779531+0000","last_change":"2023-10-16T11:22:43.162659+0000","last_active":"2023-10-16T11:22:44.779531+0000","last_peered":"2023-10-16T11:22:44.779531+0000","last_clean":"2023-10-16T11:22:44.779531+0000","last_became_active":"2023-10-16T11:22:43.162383+0000","last_became_peered":"2023-10-16T11:22:43.162383+0000","last_unstale":"2023-10-16T11:22:44.779531+0000","last_undegraded":"2023-10-16T11:22:44.779531+0000","last_fullsized":"2023-10-16T11:22:44.779531+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:01:46.689205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.124369+0000","last_change":"2023-10-16T11:22:42.126170+0000","last_active":"2023-10-16T11:22:43.124369+0000","last_peered":"2023-10-16T11:22:43.124369+0000","last_clean":"2023-10-16T11:22:43.124369+0000","last_became_active":"2023-10-16T11:22:42.124316+0000","last_became_peered":"2023-10-16T11:22:42.124316+0000","last_unstale":"2023-10-16T11:22:43.124369+0000","last_undegraded":"2023-10-16T11:22:43.124369+0000","last_fullsized":"2023-10-16T11:22:43.124369+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:39:09.410456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.139565+0000","last_change":"2023-10-16T11:22:43.128512+0000","last_active":"2023-10-16T11:22:44.139565+0000","last_peered":"2023-10-16T11:22:44.139565+0000","last_clean":"2023-10-16T11:22:44.139565+0000","last_became_active":"2023-10-16T11:22:43.127595+0000","last_became_peered":"2023-10-16T11:22:43.127595+0000","last_unstale":"2023-10-16T11:22:44.139565+0000","last_undegraded":"2023-10-16T11:22:44.139565+0000","last_fullsized":"2023-10-16T11:22:44.139565+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:17:16.487265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.139998+0000","last_change":"2023-10-16T11:22:42.140303+0000","last_active":"2023-10-16T11:22:43.139998+0000","last_peered":"2023-10-16T11:22:43.139998+0000","last_clean":"2023-10-16T11:22:43.139998+0000","last_became_active":"2023-10-16T11:22:42.139994+0000","last_became_peered":"2023-10-16T11:22:42.139994+0000","last_unstale":"2023-10-16T11:22:43.139998+0000","last_undegraded":"2023-10-16T11:22:43.139998+0000","last_fullsized":"2023-10-16T11:22:43.139998+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:48.644799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.139428+0000","last_change":"2023-10-16T11:22:43.129643+0000","last_active":"2023-10-16T11:22:44.139428+0000","last_peered":"2023-10-16T11:22:44.139428+0000","last_clean":"2023-10-16T11:22:44.139428+0000","last_became_active":"2023-10-16T11:22:43.129356+0000","last_became_peered":"2023-10-16T11:22:43.129356+0000","last_unstale":"2023-10-16T11:22:44.139428+0000","last_undegraded":"2023-10-16T11:22:44.139428+0000","last_fullsized":"2023-10-16T11:22:44.139428+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:00.954663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"200'2","reported_seq":14,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.144555+0000","last_change":"2023-10-16T11:22:42.137600+0000","last_active":"2023-10-16T11:22:43.144555+0000","last_peered":"2023-10-16T11:22:43.144555+0000","last_clean":"2023-10-16T11:22:43.144555+0000","last_became_active":"2023-10-16T11:22:42.137307+0000","last_became_peered":"2023-10-16T11:22:42.137307+0000","last_unstale":"2023-10-16T11:22:43.144555+0000","last_undegraded":"2023-10-16T11:22:43.144555+0000","last_fullsized":"2023-10-16T11:22:43.144555+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:41:44.300027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.778285+0000","last_change":"2023-10-16T11:22:43.147905+0000","last_active":"2023-10-16T11:22:44.778285+0000","last_peered":"2023-10-16T11:22:44.778285+0000","last_clean":"2023-10-16T11:22:44.778285+0000","last_became_active":"2023-10-16T11:22:43.147601+0000","last_became_peered":"2023-10-16T11:22:43.147601+0000","last_unstale":"2023-10-16T11:22:44.778285+0000","last_undegraded":"2023-10-16T11:22:44.778285+0000","last_fullsized":"2023-10-16T11:22:44.778285+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:00:27.289128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.119887+0000","last_change":"2023-10-16T11:22:42.132077+0000","last_active":"2023-10-16T11:22:43.119887+0000","last_peered":"2023-10-16T11:22:43.119887+0000","last_clean":"2023-10-16T11:22:43.119887+0000","last_became_active":"2023-10-16T11:22:42.131806+0000","last_became_peered":"2023-10-16T11:22:42.131806+0000","last_unstale":"2023-10-16T11:22:43.119887+0000","last_undegraded":"2023-10-16T11:22:43.119887+0000","last_fullsized":"2023-10-16T11:22:43.119887+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:35:09.498951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.143482+0000","last_change":"2023-10-16T11:22:43.140925+0000","last_active":"2023-10-16T11:22:44.143482+0000","last_peered":"2023-10-16T11:22:44.143482+0000","last_clean":"2023-10-16T11:22:44.143482+0000","last_became_active":"2023-10-16T11:22:43.140661+0000","last_became_peered":"2023-10-16T11:22:43.140661+0000","last_unstale":"2023-10-16T11:22:44.143482+0000","last_undegraded":"2023-10-16T11:22:44.143482+0000","last_fullsized":"2023-10-16T11:22:44.143482+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:53:10.906621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.120125+0000","last_change":"2023-10-16T11:22:42.131666+0000","last_active":"2023-10-16T11:22:43.120125+0000","last_peered":"2023-10-16T11:22:43.120125+0000","last_clean":"2023-10-16T11:22:43.120125+0000","last_became_active":"2023-10-16T11:22:42.131386+0000","last_became_peered":"2023-10-16T11:22:42.131386+0000","last_unstale":"2023-10-16T11:22:43.120125+0000","last_undegraded":"2023-10-16T11:22:43.120125+0000","last_fullsized":"2023-10-16T11:22:43.120125+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:31:53.479584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.144016+0000","last_change":"2023-10-16T11:22:43.154520+0000","last_active":"2023-10-16T11:22:44.144016+0000","last_peered":"2023-10-16T11:22:44.144016+0000","last_clean":"2023-10-16T11:22:44.144016+0000","last_became_active":"2023-10-16T11:22:43.154208+0000","last_became_peered":"2023-10-16T11:22:43.154208+0000","last_unstale":"2023-10-16T11:22:44.144016+0000","last_undegraded":"2023-10-16T11:22:44.144016+0000","last_fullsized":"2023-10-16T11:22:44.144016+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:45:22.939067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.133807+0000","last_change":"2023-10-16T11:22:42.143263+0000","last_active":"2023-10-16T11:22:43.133807+0000","last_peered":"2023-10-16T11:22:43.133807+0000","last_clean":"2023-10-16T11:22:43.133807+0000","last_became_active":"2023-10-16T11:22:42.142837+0000","last_became_peered":"2023-10-16T11:22:42.142837+0000","last_unstale":"2023-10-16T11:22:43.133807+0000","last_undegraded":"2023-10-16T11:22:43.133807+0000","last_fullsized":"2023-10-16T11:22:43.133807+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:14:31.933379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.781214+0000","last_change":"2023-10-16T11:22:43.148812+0000","last_active":"2023-10-16T11:22:44.781214+0000","last_peered":"2023-10-16T11:22:44.781214+0000","last_clean":"2023-10-16T11:22:44.781214+0000","last_became_active":"2023-10-16T11:22:43.148400+0000","last_became_peered":"2023-10-16T11:22:43.148400+0000","last_unstale":"2023-10-16T11:22:44.781214+0000","last_undegraded":"2023-10-16T11:22:44.781214+0000","last_fullsized":"2023-10-16T11:22:44.781214+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:12:15.344443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.152268+0000","last_change":"2023-10-16T11:22:42.135767+0000","last_active":"2023-10-16T11:22:43.152268+0000","last_peered":"2023-10-16T11:22:43.152268+0000","last_clean":"2023-10-16T11:22:43.152268+0000","last_became_active":"2023-10-16T11:22:42.135478+0000","last_became_peered":"2023-10-16T11:22:42.135478+0000","last_unstale":"2023-10-16T11:22:43.152268+0000","last_undegraded":"2023-10-16T11:22:43.152268+0000","last_fullsized":"2023-10-16T11:22:43.152268+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:50.483625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.781558+0000","last_change":"2023-10-16T11:22:43.157997+0000","last_active":"2023-10-16T11:22:44.781558+0000","last_peered":"2023-10-16T11:22:44.781558+0000","last_clean":"2023-10-16T11:22:44.781558+0000","last_became_active":"2023-10-16T11:22:43.152986+0000","last_became_peered":"2023-10-16T11:22:43.152986+0000","last_unstale":"2023-10-16T11:22:44.781558+0000","last_undegraded":"2023-10-16T11:22:44.781558+0000","last_fullsized":"2023-10-16T11:22:44.781558+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:20.897287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.151139+0000","last_change":"2023-10-16T11:22:42.144266+0000","last_active":"2023-10-16T11:22:43.151139+0000","last_peered":"2023-10-16T11:22:43.151139+0000","last_clean":"2023-10-16T11:22:43.151139+0000","last_became_active":"2023-10-16T11:22:42.142881+0000","last_became_peered":"2023-10-16T11:22:42.142881+0000","last_unstale":"2023-10-16T11:22:43.151139+0000","last_undegraded":"2023-10-16T11:22:43.151139+0000","last_fullsized":"2023-10-16T11:22:43.151139+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:17.013373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.777842+0000","last_change":"2023-10-16T11:22:43.151092+0000","last_active":"2023-10-16T11:22:44.777842+0000","last_peered":"2023-10-16T11:22:44.777842+0000","last_clean":"2023-10-16T11:22:44.777842+0000","last_became_active":"2023-10-16T11:22:43.150817+0000","last_became_peered":"2023-10-16T11:22:43.150817+0000","last_unstale":"2023-10-16T11:22:44.777842+0000","last_undegraded":"2023-10-16T11:22:44.777842+0000","last_fullsized":"2023-10-16T11:22:44.777842+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:10.267804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.145215+0000","last_change":"2023-10-16T11:22:42.141567+0000","last_active":"2023-10-16T11:22:43.145215+0000","last_peered":"2023-10-16T11:22:43.145215+0000","last_clean":"2023-10-16T11:22:43.145215+0000","last_became_active":"2023-10-16T11:22:42.141238+0000","last_became_peered":"2023-10-16T11:22:42.141238+0000","last_unstale":"2023-10-16T11:22:43.145215+0000","last_undegraded":"2023-10-16T11:22:43.145215+0000","last_fullsized":"2023-10-16T11:22:43.145215+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:55.513664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.139623+0000","last_change":"2023-10-16T11:22:43.157589+0000","last_active":"2023-10-16T11:22:44.139623+0000","last_peered":"2023-10-16T11:22:44.139623+0000","last_clean":"2023-10-16T11:22:44.139623+0000","last_became_active":"2023-10-16T11:22:43.157284+0000","last_became_peered":"2023-10-16T11:22:43.157284+0000","last_unstale":"2023-10-16T11:22:44.139623+0000","last_undegraded":"2023-10-16T11:22:44.139623+0000","last_fullsized":"2023-10-16T11:22:44.139623+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:57:39.650999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.121749+0000","last_change":"2023-10-16T11:22:42.119659+0000","last_active":"2023-10-16T11:22:43.121749+0000","last_peered":"2023-10-16T11:22:43.121749+0000","last_clean":"2023-10-16T11:22:43.121749+0000","last_became_active":"2023-10-16T11:22:42.119380+0000","last_became_peered":"2023-10-16T11:22:42.119380+0000","last_unstale":"2023-10-16T11:22:43.121749+0000","last_undegraded":"2023-10-16T11:22:43.121749+0000","last_fullsized":"2023-10-16T11:22:43.121749+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:29.005826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.777472+0000","last_change":"2023-10-16T11:22:43.158475+0000","last_active":"2023-10-16T11:22:44.777472+0000","last_peered":"2023-10-16T11:22:44.777472+0000","last_clean":"2023-10-16T11:22:44.777472+0000","last_became_active":"2023-10-16T11:22:43.157982+0000","last_became_peered":"2023-10-16T11:22:43.157982+0000","last_unstale":"2023-10-16T11:22:44.777472+0000","last_undegraded":"2023-10-16T11:22:44.777472+0000","last_fullsized":"2023-10-16T11:22:44.777472+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:17.151421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.138199+0000","last_change":"2023-10-16T11:22:42.141735+0000","last_active":"2023-10-16T11:22:43.138199+0000","last_peered":"2023-10-16T11:22:43.138199+0000","last_clean":"2023-10-16T11:22:43.138199+0000","last_became_active":"2023-10-16T11:22:42.137927+0000","last_became_peered":"2023-10-16T11:22:42.137927+0000","last_unstale":"2023-10-16T11:22:43.138199+0000","last_undegraded":"2023-10-16T11:22:43.138199+0000","last_fullsized":"2023-10-16T11:22:43.138199+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:45:21.920494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"189'2745","reported_seq":3558,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.134292+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:22:43.134292+0000","last_peered":"2023-10-16T11:22:43.134292+0000","last_clean":"2023-10-16T11:22:43.134292+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:22:43.134292+0000","last_undegraded":"2023-10-16T11:22:43.134292+0000","last_fullsized":"2023-10-16T11:22:43.134292+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2745,"log_dups_size":0,"ondisk_log_size":2745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1984,"num_read_kb":4056,"num_write":541,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1984,"num_read_kb":4056,"num_write":541,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2745,"ondisk_log_size":2745,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575966,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48116,"kb_used_data":3048,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323724,"statfs":{"total":96636764160,"available":96587493376,"internally_reserved":0,"allocated":3121152,"data_stored":1809715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2,"15min":1.234},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.73799999999999999},"max":{"1min":1.641,"5min":2.085,"15min":2.3399999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.206,"15min":1.23},"min":{"1min":0.80000000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5549999999999999,"5min":1.6850000000000001,"15min":1.8320000000000001},"last":1.151}]},{"osd":1,"last update":"Mon Oct 16 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.2250000000000001,"15min":1.2589999999999999},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.69699999999999995},"max":{"1min":1.7589999999999999,"5min":1.913,"15min":2.0129999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.2709999999999999,"15min":1.28},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":1.726,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.264}]},{"osd":2,"last update":"Mon Oct 16 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.2869999999999999,"15min":1.3129999999999999},"min":{"1min":0.96299999999999997,"5min":0.745,"15min":0.745},"max":{"1min":1.599,"5min":1.8680000000000001,"15min":2.4460000000000002},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.361,"15min":1.363},"min":{"1min":0.97099999999999997,"5min":0.93700000000000006,"15min":0.90400000000000003},"max":{"1min":1.6579999999999999,"5min":1.859,"15min":1.974},"last":1.3839999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.389,"15min":1.3919999999999999},"min":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":0.97799999999999998},"max":{"1min":1.5860000000000001,"5min":1.8280000000000001,"15min":1.972},"last":1.464},{"interface":"front","average":{"1min":1.423,"5min":1.4530000000000001,"15min":1.4690000000000001},"min":{"1min":1.1339999999999999,"5min":0.997,"15min":0.96299999999999997},"max":{"1min":1.796,"5min":2.0680000000000001,"15min":2.577},"last":1.0620000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.373,"15min":1.417},"min":{"1min":0.95599999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.8140000000000001,"5min":1.954,"15min":2.6779999999999999},"last":1.214},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.343,"15min":1.399},"min":{"1min":0.95899999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.573,"5min":2.109,"15min":2.109},"last":0.86799999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.5549999999999999,"15min":1.5960000000000001},"min":{"1min":1.236,"5min":1.089,"15min":1.012},"max":{"1min":1.8600000000000001,"5min":2.2309999999999999,"15min":3.3519999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.46,"15min":1.5029999999999999},"min":{"1min":1.069,"5min":1.028,"15min":0.74199999999999999},"max":{"1min":1.6970000000000001,"5min":2.2349999999999999,"15min":2.4969999999999999},"last":1.321}]},{"osd":6,"last update":"Mon Oct 16 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.466,"15min":1.5149999999999999},"min":{"1min":1.165,"5min":0.93500000000000005,"15min":0.90500000000000003},"max":{"1min":1.823,"5min":1.9319999999999999,"15min":2.7250000000000001},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.635,"5min":1.6699999999999999,"15min":1.714},"min":{"1min":1.284,"5min":1.105,"15min":1.105},"max":{"1min":1.9279999999999999,"5min":2.298,"15min":3.1440000000000001},"last":1.3009999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575966,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29480,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342360,"statfs":{"total":96636764160,"available":96606576640,"internally_reserved":0,"allocated":2519040,"data_stored":1219243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1990000000000001,"15min":1.22},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.64000000000000001},"max":{"1min":1.698,"5min":1.698,"15min":2.0419999999999998},"last":1.224},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2509999999999999,"15min":1.264},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.66400000000000003},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.6799999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:22:33 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.2330000000000001,"15min":1.2170000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.8420000000000001},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.367,"15min":1.3480000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.77,"5min":1.796,"15min":1.796},"last":1.331}]},{"osd":2,"last update":"Mon Oct 16 11:22:33 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2749999999999999,"15min":1.2470000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.68000000000000005},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.861},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3500000000000001,"15min":1.357},"min":{"1min":0.92700000000000005,"5min":0.85799999999999998,"15min":0.72699999999999998},"max":{"1min":1.625,"5min":1.7949999999999999,"15min":2.0310000000000001},"last":1.738}]},{"osd":3,"last update":"Mon Oct 16 11:22:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3069999999999999,"15min":1.298},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7370000000000001,"5min":1.8260000000000001,"15min":1.859},"last":1.841},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2669999999999999,"15min":1.2649999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.744,"5min":1.744,"15min":1.8839999999999999},"last":1.1020000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:22:33 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.246,"15min":1.2949999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.9099999999999999},"last":1.8999999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.512,"15min":1.4970000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.9019999999999999,"5min":1.929,"15min":1.9530000000000001},"last":2.161}]},{"osd":5,"last update":"Mon Oct 16 11:22:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.3360000000000001,"15min":1.3340000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.5680000000000001,"5min":1.925,"15min":2.1669999999999998},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.518,"15min":1.5429999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.8520000000000001,"5min":1.978,"15min":2.1280000000000001},"last":2.1059999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:22:33 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4830000000000001,"15min":1.4930000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.766,"5min":1.915,"15min":2.149},"last":1.994},{"interface":"front","average":{"1min":1.502,"5min":1.5369999999999999,"15min":1.546},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.8420000000000001,"5min":2.0230000000000001,"15min":2.3530000000000002},"last":1.778}]}]},{"osd":5,"up_from":13,"seq":55834575966,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29484,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342356,"statfs":{"total":96636764160,"available":96606572544,"internally_reserved":0,"allocated":2519040,"data_stored":1219243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.2609999999999999,"15min":1.2809999999999999},"min":{"1min":0.876,"5min":0.755,"15min":0.70799999999999996},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3460000000000001,"15min":1.3620000000000001},"min":{"1min":1.036,"5min":0.88500000000000001,"15min":0.83899999999999997},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.887},"last":1.704}]},{"osd":1,"last update":"Mon Oct 16 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.2070000000000001,"15min":1.2170000000000001},"min":{"1min":0.90100000000000002,"5min":0.80800000000000005,"15min":0.76100000000000001},"max":{"1min":1.704,"5min":1.7869999999999999,"15min":1.992},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.2569999999999999,"15min":1.2310000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.80800000000000005},"max":{"1min":2.9399999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.081}]},{"osd":2,"last update":"Mon Oct 16 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.25,"15min":1.242},"min":{"1min":0.876,"5min":0.876,"15min":0.83799999999999997},"max":{"1min":1.952,"5min":1.952,"15min":2.052},"last":1.8600000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.2689999999999999,"15min":1.323},"min":{"1min":0.98399999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.5760000000000001,"5min":1.766,"15min":2.1640000000000001},"last":1.2130000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.335,"15min":1.333},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":3.012,"5min":3.012,"15min":3.012},"last":1.397},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.3979999999999999,"15min":1.4299999999999999},"min":{"1min":1.1100000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.7190000000000001,"5min":1.7669999999999999,"15min":2.2250000000000001},"last":1.4690000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.304,"15min":1.353},"min":{"1min":0.84199999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.349},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.556,"15min":1.573},"min":{"1min":1.145,"5min":1.0660000000000001,"15min":0.94999999999999996},"max":{"1min":2.903,"5min":2.903,"15min":2.903},"last":1.7669999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.407,"15min":1.4199999999999999},"min":{"1min":1.0489999999999999,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8100000000000001,"5min":2.0649999999999999,"15min":2.492},"last":1.323},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.466,"15min":1.4890000000000001},"min":{"1min":1.2050000000000001,"5min":0.89200000000000002,"15min":0.78400000000000003},"max":{"1min":3.073,"5min":3.073,"15min":3.073},"last":1.5189999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.5289999999999999,"15min":1.5469999999999999},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90600000000000003},"max":{"1min":3.1160000000000001,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.613},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.6180000000000001,"15min":1.657},"min":{"1min":1.454,"5min":1.1859999999999999,"15min":0.88100000000000001},"max":{"1min":2.0750000000000002,"5min":2.1459999999999999,"15min":2.7290000000000001},"last":1.573}]}]},{"osd":4,"up_from":13,"seq":55834575967,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51712,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320128,"statfs":{"total":96636764160,"available":96583811072,"internally_reserved":0,"allocated":2535424,"data_stored":1221437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2,"15min":1.1859999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.125},{"interface":"front","average":{"1min":1.202,"5min":1.24,"15min":1.2490000000000001},"min":{"1min":0.873,"5min":0.75,"15min":0.75},"max":{"1min":1.754,"5min":1.7609999999999999,"15min":1.9299999999999999},"last":1.0449999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.272,"15min":1.3029999999999999},"min":{"1min":0.89300000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.708,"5min":1.7569999999999999,"15min":1.913},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.238,"5min":1.234,"15min":1.252},"min":{"1min":0.95599999999999996,"5min":0.85299999999999998,"15min":0.84399999999999997},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.889},"last":0.92900000000000005}]},{"osd":2,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3100000000000001,"15min":1.2949999999999999},"min":{"1min":0.90200000000000002,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.724,"5min":1.7370000000000001,"15min":1.835},"last":1.619},{"interface":"front","average":{"1min":1.407,"5min":1.409,"15min":1.3979999999999999},"min":{"1min":1.008,"5min":0.872,"15min":0.77600000000000002},"max":{"1min":1.861,"5min":1.8779999999999999,"15min":1.9139999999999999},"last":1.194}]},{"osd":3,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.97299999999999998,"5min":0.94499999999999995,"15min":0.86599999999999999},"max":{"1min":1.613,"5min":1.849,"15min":1.849},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.4650000000000001,"15min":1.4279999999999999},"min":{"1min":1.2470000000000001,"5min":1.042,"15min":0.90000000000000002},"max":{"1min":1.8360000000000001,"5min":1.863,"15min":2.0720000000000001},"last":1.008}]},{"osd":5,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.46,"15min":1.431},"min":{"1min":1.0049999999999999,"5min":0.98699999999999999,"15min":0.88200000000000001},"max":{"1min":1.9530000000000001,"5min":2.0790000000000002,"15min":2.0910000000000002},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.425,"15min":1.4219999999999999},"min":{"1min":1.2330000000000001,"5min":0.84499999999999997,"15min":0.78600000000000003},"max":{"1min":1.804,"5min":1.9139999999999999,"15min":2.0059999999999998},"last":1.5049999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":1.54,"15min":1.486},"min":{"1min":1.1830000000000001,"5min":0.97799999999999998,"15min":0.78000000000000003},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":2.2309999999999999},"last":1.722},{"interface":"front","average":{"1min":1.653,"5min":1.607,"15min":1.613},"min":{"1min":1.3520000000000001,"5min":1.0920000000000001,"15min":1.038},"max":{"1min":2.02,"5min":2.0390000000000001,"15min":2.2770000000000001},"last":1.597}]},{"osd":7,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6240000000000001,"5min":1.5669999999999999,"15min":1.5660000000000001},"min":{"1min":1.302,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.8340000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.403},{"interface":"front","average":{"1min":1.74,"5min":1.6759999999999999,"15min":1.6319999999999999},"min":{"1min":1.4970000000000001,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":2.1469999999999998,"5min":2.1509999999999998,"15min":2.3839999999999999},"last":1.845}]}]},{"osd":3,"up_from":13,"seq":55834575966,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47544,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324296,"statfs":{"total":96636764160,"available":96588079104,"internally_reserved":0,"allocated":2531328,"data_stored":1219437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.92800000000000005,"15min":0.91200000000000003},"min":{"1min":0.67800000000000005,"5min":0.64500000000000002,"15min":0.54400000000000004},"max":{"1min":1.3839999999999999,"5min":1.4930000000000001,"15min":1.901},"last":0.755},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.087,"15min":1.0840000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.56999999999999995},"max":{"1min":1.575,"5min":1.6180000000000001,"15min":1.6679999999999999},"last":0.68799999999999994}]},{"osd":1,"last update":"Mon Oct 16 11:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.96999999999999997,"15min":0.98999999999999999},"min":{"1min":0.76000000000000001,"5min":0.626,"15min":0.60299999999999998},"max":{"1min":1.361,"5min":1.4219999999999999,"15min":1.4510000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1499999999999999,"15min":1.1539999999999999},"min":{"1min":0.84699999999999998,"5min":0.68500000000000005,"15min":0.67600000000000005},"max":{"1min":1.5640000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.59299999999999997}]},{"osd":2,"last update":"Mon Oct 16 11:22:02 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.123,"15min":1.125},"min":{"1min":0.82199999999999995,"5min":0.79700000000000004,"15min":0.79200000000000004},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.238,"5min":1.2370000000000001,"15min":1.2430000000000001},"min":{"1min":0.878,"5min":0.87,"15min":0.67400000000000004},"max":{"1min":1.5209999999999999,"5min":1.677,"15min":1.843},"last":0.78500000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:22:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.371,"15min":1.3640000000000001},"min":{"1min":1.01,"5min":1.01,"15min":0.95999999999999996},"max":{"1min":1.5660000000000001,"5min":1.7330000000000001,"15min":1.873},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3640000000000001,"15min":1.369},"min":{"1min":1.04,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":1.6060000000000001,"5min":1.8240000000000001,"15min":1.8819999999999999},"last":1.1819999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:22:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4219999999999999,"15min":1.411},"min":{"1min":1.0960000000000001,"5min":1.0940000000000001,"15min":0.96199999999999997},"max":{"1min":1.9219999999999999,"5min":2.1509999999999998,"15min":2.2410000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.462,"15min":1.45},"min":{"1min":1.131,"5min":1.004,"15min":1.004},"max":{"1min":2.0739999999999998,"5min":2.1150000000000002,"15min":2.1579999999999999},"last":1.3380000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:22:02 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.538,"15min":1.546},"min":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"max":{"1min":1.8120000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5329999999999999,"15min":1.5429999999999999},"min":{"1min":1.2410000000000001,"5min":1.1060000000000001,"15min":1.083},"max":{"1min":2.0409999999999999,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.28}]},{"osd":7,"last update":"Mon Oct 16 11:22:02 2023","interfaces":[{"interface":"back","average":{"1min":1.605,"5min":1.548,"15min":1.5720000000000001},"min":{"1min":1.325,"5min":1.0609999999999999,"15min":1.0429999999999999},"max":{"1min":2.0790000000000002,"5min":2.1960000000000002,"15min":2.2679999999999998},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.7190000000000001,"5min":1.6879999999999999,"15min":1.6919999999999999},"min":{"1min":1.323,"5min":1.323,"15min":1.1479999999999999},"max":{"1min":2.1549999999999998,"5min":2.2549999999999999,"15min":2.5129999999999999},"last":1.569}]}]},{"osd":2,"up_from":13,"seq":55834575965,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51708,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320132,"statfs":{"total":96636764160,"available":96583815168,"internally_reserved":0,"allocated":2531328,"data_stored":1219937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:05 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.90300000000000002,"15min":0.91700000000000004},"min":{"1min":0.629,"5min":0.5,"15min":0.5},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.669},"last":1.079},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0920000000000001,"15min":1.091},"min":{"1min":0.72799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.51,"5min":1.8180000000000001,"15min":2.1429999999999998},"last":0.76100000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:22:05 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.085,"15min":1.1020000000000001},"min":{"1min":0.754,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.645,"5min":1.6779999999999999,"15min":1.978},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.163,"15min":1.181},"min":{"1min":0.87,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.609,"5min":1.6819999999999999,"15min":1.869},"last":0.96399999999999997}]},{"osd":3,"last update":"Mon Oct 16 11:22:05 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.0680000000000001,"15min":1.0960000000000001},"min":{"1min":0.75900000000000001,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.651,"5min":1.651,"15min":1.8560000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2709999999999999,"15min":1.2849999999999999},"min":{"1min":0.91300000000000003,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.617,"5min":1.6719999999999999,"15min":1.8740000000000001},"last":0.90900000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:22:05 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.3979999999999999,"15min":1.4359999999999999},"min":{"1min":1.095,"5min":0.96899999999999997,"15min":0.73499999999999999},"max":{"1min":1.77,"5min":2.2770000000000001,"15min":2.5059999999999998},"last":1.028},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3300000000000001,"15min":1.347},"min":{"1min":1.002,"5min":0.748,"15min":0.748},"max":{"1min":1.6259999999999999,"5min":1.7749999999999999,"15min":2.0880000000000001},"last":1.5369999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:22:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.454,"15min":1.462},"min":{"1min":1.141,"5min":1.0860000000000001,"15min":0.78500000000000003},"max":{"1min":1.992,"5min":2.0099999999999998,"15min":2.2269999999999999},"last":1.569},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.464,"15min":1.504},"min":{"1min":1.1439999999999999,"5min":0.874,"15min":0.874},"max":{"1min":1.8939999999999999,"5min":2.1349999999999998,"15min":2.6259999999999999},"last":1.165}]},{"osd":6,"last update":"Mon Oct 16 11:22:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.377,"15min":1.4099999999999999},"min":{"1min":1.1699999999999999,"5min":0.84199999999999997,"15min":0.80500000000000005},"max":{"1min":1.8620000000000001,"5min":1.8660000000000001,"15min":2.2109999999999999},"last":1.7010000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5209999999999999,"15min":1.5389999999999999},"min":{"1min":1.052,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.0600000000000001,"5min":2.089,"15min":2.2029999999999998},"last":1.3260000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:22:05 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.5580000000000001,"15min":1.5960000000000001},"min":{"1min":1.2490000000000001,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":1.9470000000000001,"5min":2.4670000000000001,"15min":2.8109999999999999},"last":1.464},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.4710000000000001,"15min":1.5229999999999999},"min":{"1min":1.1040000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2829999999999999},"last":1.6120000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575965,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60024,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311816,"statfs":{"total":96636764160,"available":96575299584,"internally_reserved":0,"allocated":2531328,"data_stored":1220855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.91800000000000004,"15min":0.91500000000000004},"min":{"1min":0.71099999999999997,"5min":0.66700000000000004,"15min":0.39100000000000001},"max":{"1min":1.494,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.056,"15min":1.0609999999999999},"min":{"1min":0.80700000000000005,"5min":0.621,"15min":0.54500000000000004},"max":{"1min":1.331,"5min":1.633,"15min":1.667},"last":0.746}]},{"osd":2,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.99299999999999999,"15min":1.024},"min":{"1min":0.75800000000000001,"5min":0.72999999999999998,"15min":0.42699999999999999},"max":{"1min":1.2290000000000001,"5min":1.4279999999999999,"15min":1.972},"last":1.421},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.278,"15min":1.2869999999999999},"min":{"1min":0.94299999999999995,"5min":0.78600000000000003,"15min":0.55700000000000005},"max":{"1min":1.556,"5min":1.6910000000000001,"15min":2.161},"last":1.2729999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.1759999999999999,"15min":1.21},"min":{"1min":0.91900000000000004,"5min":0.78300000000000003,"15min":0.49399999999999999},"max":{"1min":1.538,"5min":1.7729999999999999,"15min":1.79},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.1839999999999999,"15min":1.1779999999999999},"min":{"1min":0.90000000000000002,"5min":0.86899999999999999,"15min":0.51300000000000001},"max":{"1min":1.5009999999999999,"5min":1.609,"15min":1.802},"last":1.204}]},{"osd":4,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.292,"15min":1.28},"min":{"1min":1.0029999999999999,"5min":0.90700000000000003,"15min":0.80700000000000005},"max":{"1min":1.7290000000000001,"5min":1.887,"15min":1.9159999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.401,"15min":1.4139999999999999},"min":{"1min":1.139,"5min":1.0549999999999999,"15min":0.64700000000000002},"max":{"1min":1.825,"5min":2.0619999999999998,"15min":2.093},"last":1.3440000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.4219999999999999,"15min":1.425},"min":{"1min":1.103,"5min":0.999,"15min":0.68400000000000005},"max":{"1min":1.7370000000000001,"5min":1.948,"15min":2.0899999999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4059999999999999,"15min":1.4119999999999999},"min":{"1min":0.996,"5min":0.996,"15min":0.82599999999999996},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":2.0649999999999999},"last":1.3069999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.498,"15min":1.4970000000000001},"min":{"1min":1.222,"5min":1.1519999999999999,"15min":0.85999999999999999},"max":{"1min":2.1179999999999999,"5min":2.1970000000000001,"15min":2.2949999999999999},"last":1.51},{"interface":"front","average":{"1min":1.52,"5min":1.498,"15min":1.49},"min":{"1min":1.125,"5min":1.125,"15min":0.97299999999999998},"max":{"1min":1.8740000000000001,"5min":1.9079999999999999,"15min":2.1970000000000001},"last":1.8140000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.5680000000000001,"15min":1.5629999999999999},"min":{"1min":1.3440000000000001,"5min":1.1819999999999999,"15min":0.85699999999999998},"max":{"1min":1.8959999999999999,"5min":2.077,"15min":2.1720000000000002},"last":1.552},{"interface":"front","average":{"1min":1.681,"5min":1.6240000000000001,"15min":1.6100000000000001},"min":{"1min":1.226,"5min":1.103,"15min":0.90800000000000003},"max":{"1min":2.1699999999999999,"5min":2.1989999999999998,"15min":2.2810000000000001},"last":1.621}]}]},{"osd":0,"up_from":13,"seq":55834575966,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52276,"kb_used_data":3044,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319564,"statfs":{"total":96636764160,"available":96583233536,"internally_reserved":0,"allocated":3117056,"data_stored":1809633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89100000000000001,"15min":0.89500000000000002},"min":{"1min":0.68500000000000005,"5min":0.63,"15min":0.41599999999999998},"max":{"1min":1.52,"5min":1.546,"15min":1.619},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.056,"5min":1.103,"15min":1.1060000000000001},"min":{"1min":0.69099999999999995,"5min":0.67700000000000005,"15min":0.46300000000000002},"max":{"1min":1.4950000000000001,"5min":1.681,"15min":1.76},"last":0.96799999999999997}]},{"osd":2,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.097,"15min":1.079},"min":{"1min":0.80000000000000004,"5min":0.66300000000000003,"15min":0.51100000000000001},"max":{"1min":1.468,"5min":1.649,"15min":1.649},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1379999999999999,"15min":1.1559999999999999},"min":{"1min":0.81699999999999995,"5min":0.76500000000000001,"15min":0.53100000000000003},"max":{"1min":1.6220000000000001,"5min":1.623,"15min":1.9319999999999999},"last":0.92600000000000005}]},{"osd":3,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.113,"15min":1.087},"min":{"1min":0.81799999999999995,"5min":0.78800000000000003,"15min":0.42599999999999999},"max":{"1min":1.5129999999999999,"5min":1.8100000000000001,"15min":1.8839999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.24,"5min":1.2789999999999999,"15min":1.2609999999999999},"min":{"1min":0.91100000000000003,"5min":0.90600000000000003,"15min":0.64200000000000002},"max":{"1min":1.585,"5min":1.7609999999999999,"15min":1.9830000000000001},"last":0.86099999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.4219999999999999,"15min":1.3899999999999999},"min":{"1min":1.1160000000000001,"5min":1.0860000000000001,"15min":0.78700000000000003},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.95},"last":1.105},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3500000000000001,"15min":1.3580000000000001},"min":{"1min":1.0469999999999999,"5min":1,"15min":0.88},"max":{"1min":1.6619999999999999,"5min":1.9019999999999999,"15min":2.4820000000000002},"last":2.1539999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.369,"15min":1.3879999999999999},"min":{"1min":1.23,"5min":1.046,"15min":0.76600000000000001},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.6669999999999998},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.462,"15min":1.462},"min":{"1min":1.149,"5min":1.0860000000000001,"15min":0.752},"max":{"1min":1.748,"5min":1.9350000000000001,"15min":2.2040000000000002},"last":1.1799999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.516,"15min":1.514},"min":{"1min":1.202,"5min":1.1080000000000001,"15min":1.0329999999999999},"max":{"1min":1.8100000000000001,"5min":1.984,"15min":2.226},"last":1.278},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.508,"15min":1.5},"min":{"1min":1.099,"5min":1.099,"15min":0.79200000000000004},"max":{"1min":1.9750000000000001,"5min":2.262,"15min":2.3969999999999998},"last":1.1699999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5309999999999999,"5min":1.5629999999999999,"15min":1.5609999999999999},"min":{"1min":1.149,"5min":1.149,"15min":0.88100000000000001},"max":{"1min":1.929,"5min":2.347,"15min":2.347},"last":1.224},{"interface":"front","average":{"1min":1.552,"5min":1.528,"15min":1.5229999999999999},"min":{"1min":1.159,"5min":1.008,"15min":1.008},"max":{"1min":1.919,"5min":2.2919999999999998,"15min":2.8199999999999998},"last":1.135}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:22:47.535 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:22:47.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.779+0000 7f008c0cf640 1 Processor -- start 2023-10-16T11:22:47.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.780+0000 7f008c0cf640 1 -- start start 2023-10-16T11:22:47.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.781+0000 7f008c0cf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f00840cd0c0 0x7f00841615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:47.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.781+0000 7f008c0cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f008415db00 0x7f00840b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:47.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.781+0000 7f008c0cf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f008415e3e0 con 0x7f00840b0410 2023-10-16T11:22:47.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.781+0000 7f008c0cf640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00840b0de0 con 0x7f008415db00 2023-10-16T11:22:47.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.781+0000 7f008c0cf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00840cd4a0 con 0x7f00840cd0c0 2023-10-16T11:22:47.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.782+0000 7f008a66c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f008415db00 0x7f00840b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:47.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.782+0000 7f008a66c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f008415db00 0x7f00840b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38464/0 (socket says 172.21.15.84:38464) 2023-10-16T11:22:47.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.782+0000 7f008a66c640 1 -- 172.21.15.84:0/4292295696 learned_addr learned my addr 172.21.15.84:0/4292295696 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:47.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.782+0000 7f0089e6b640 1 --2- 172.21.15.84:0/4292295696 >> v2:172.21.15.157:3300/0 conn(0x7f00840cd0c0 0x7f00841615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:47.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.782+0000 7f008a66c640 1 -- 172.21.15.84:0/4292295696 >> v2:172.21.15.157:3300/0 conn(0x7f00840cd0c0 msgr2=0x7f00841615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:47.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.782+0000 7f008a66c640 1 --2- 172.21.15.84:0/4292295696 >> v2:172.21.15.157:3300/0 conn(0x7f00840cd0c0 0x7f00841615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.782+0000 7f008a66c640 1 -- 172.21.15.84:0/4292295696 >> v1:172.21.15.157:6789/0 conn(0x7f00840b0410 legacy=0x7f008415d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:47.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.782+0000 7f008a66c640 1 -- 172.21.15.84:0/4292295696 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008415e100 con 0x7f008415db00 2023-10-16T11:22:47.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.783+0000 7f008a66c640 1 --2- 172.21.15.84:0/4292295696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f008415db00 0x7f00840b08a0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f00840cc4e0 tx=0x7f007c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d381af4280347cd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:47.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.783+0000 7f0088e69640 1 -- 172.21.15.84:0/4292295696 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c0317e0 con 0x7f008415db00 2023-10-16T11:22:47.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.783+0000 7f0088e69640 1 -- 172.21.15.84:0/4292295696 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007c031940 con 0x7f008415db00 2023-10-16T11:22:47.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.783+0000 7f0088e69640 1 -- 172.21.15.84:0/4292295696 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c031cd0 con 0x7f008415db00 2023-10-16T11:22:47.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.784+0000 7f008c0cf640 1 -- 172.21.15.84:0/4292295696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f008415db00 msgr2=0x7f00840b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:47.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.784+0000 7f008c0cf640 1 --2- 172.21.15.84:0/4292295696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f008415db00 0x7f00840b08a0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f00840cc4e0 tx=0x7f007c02b880 comp rx=0 tx=0).stop 2023-10-16T11:22:47.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.784+0000 7f008c0cf640 1 -- 172.21.15.84:0/4292295696 shutdown_connections 2023-10-16T11:22:47.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.784+0000 7f008c0cf640 1 --2- 172.21.15.84:0/4292295696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f008415db00 0x7f00840b08a0 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.784+0000 7f008c0cf640 1 --2- 172.21.15.84:0/4292295696 >> v2:172.21.15.157:3300/0 conn(0x7f00840cd0c0 0x7f00841615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.784+0000 7f008c0cf640 1 -- 172.21.15.84:0/4292295696 >> 172.21.15.84:0/4292295696 conn(0x7f00840cb410 msgr2=0x7f00840cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:47.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.784+0000 7f008c0cf640 1 -- 172.21.15.84:0/4292295696 shutdown_connections 2023-10-16T11:22:47.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.784+0000 7f008c0cf640 1 -- 172.21.15.84:0/4292295696 wait complete. 2023-10-16T11:22:47.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.786+0000 7f008c0cf640 1 Processor -- start 2023-10-16T11:22:47.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.787+0000 7f008c0cf640 1 -- start start 2023-10-16T11:22:47.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.788+0000 7f008c0cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00840b0410 0x7f0084150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:47.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.788+0000 7f008c0cf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00840cd0c0 0x7f00841e70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:47.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.788+0000 7f008c0cf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f008415db00 0x7f00841562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:47.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.788+0000 7f008c0cf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0084151420 con 0x7f00840b0410 2023-10-16T11:22:47.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.788+0000 7f008c0cf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00840c3bc0 con 0x7f008415db00 2023-10-16T11:22:47.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.788+0000 7f008c0cf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00840c3ea0 con 0x7f00840cd0c0 2023-10-16T11:22:47.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.788+0000 7f0089e6b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00840b0410 0x7f0084150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:47.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.788+0000 7f0089e6b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00840b0410 0x7f0084150ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38480/0 (socket says 172.21.15.84:38480) 2023-10-16T11:22:47.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.788+0000 7f0089e6b640 1 -- 172.21.15.84:0/2028720839 learned_addr learned my addr 172.21.15.84:0/2028720839 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:47.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.788+0000 7f0089e6b640 1 -- 172.21.15.84:0/2028720839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00840cd0c0 msgr2=0x7f00841e70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:47.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.788+0000 7f0089e6b640 1 --2- 172.21.15.84:0/2028720839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00840cd0c0 0x7f00841e70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.788+0000 7f0089e6b640 1 -- 172.21.15.84:0/2028720839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f008415db00 msgr2=0x7f00841562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:47.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.788+0000 7f0089e6b640 1 --2- 172.21.15.84:0/2028720839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f008415db00 0x7f00841562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:47.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.788+0000 7f0089e6b640 1 -- 172.21.15.84:0/2028720839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007c0094d0 con 0x7f00840b0410 2023-10-16T11:22:47.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.789+0000 7f0089e6b640 1 --2- 172.21.15.84:0/2028720839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00840b0410 0x7f0084150ee0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f0078007ef0 tx=0x7f007800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:47.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.789+0000 7f00827fc640 1 -- 172.21.15.84:0/2028720839 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078014620 con 0x7f00840b0410 2023-10-16T11:22:47.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.789+0000 7f00827fc640 1 -- 172.21.15.84:0/2028720839 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0078014780 con 0x7f00840b0410 2023-10-16T11:22:47.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.789+0000 7f00827fc640 1 -- 172.21.15.84:0/2028720839 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078014b30 con 0x7f00840b0410 2023-10-16T11:22:47.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.789+0000 7f008c0cf640 1 -- 172.21.15.84:0/2028720839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00840c4180 con 0x7f00840b0410 2023-10-16T11:22:47.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.789+0000 7f008c0cf640 1 -- 172.21.15.84:0/2028720839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00840c4740 con 0x7f00840b0410 2023-10-16T11:22:47.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.790+0000 7f008c0cf640 1 -- 172.21.15.84:0/2028720839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00840c48d0 con 0x7f00840b0410 2023-10-16T11:22:47.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.791+0000 7f00827fc640 1 -- 172.21.15.84:0/2028720839 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0078014c90 con 0x7f00840b0410 2023-10-16T11:22:47.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.792+0000 7f00827fc640 1 --2- 172.21.15.84:0/2028720839 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f006806af90 0x7f006806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:47.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.792+0000 7f00827fc640 1 -- 172.21.15.84:0/2028720839 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f007808a110 con 0x7f00840b0410 2023-10-16T11:22:47.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.793+0000 7f008966a640 1 --2- 172.21.15.84:0/2028720839 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f006806af90 0x7f006806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:47.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.793+0000 7f008966a640 1 --2- 172.21.15.84:0/2028720839 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f006806af90 0x7f006806d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0070006fd0 tx=0x7f0070006bc0 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:47.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.797+0000 7f00827fc640 1 -- 172.21.15.84:0/2028720839 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0078055120 con 0x7f00840b0410 2023-10-16T11:22:47.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.985+0000 7f008c0cf640 1 -- 172.21.15.84:0/2028720839 --> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0084160df0 con 0x7f006806af90 2023-10-16T11:22:48.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:47.998+0000 7f00827fc640 1 -- 172.21.15.84:0/2028720839 <== mgr.10557 v2:172.21.15.84:6832/1985463812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286523 (secure 0 0 0) 0x7f0070013430 con 0x7f006806af90 2023-10-16T11:22:48.004 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:48.005 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:22:48.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.004+0000 7f008c0cf640 1 -- 172.21.15.84:0/2028720839 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f006806af90 msgr2=0x7f006806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:48.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.004+0000 7f008c0cf640 1 --2- 172.21.15.84:0/2028720839 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f006806af90 0x7f006806d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0070006fd0 tx=0x7f0070006bc0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.004+0000 7f008c0cf640 1 -- 172.21.15.84:0/2028720839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00840b0410 msgr2=0x7f0084150ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:48.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.004+0000 7f008c0cf640 1 --2- 172.21.15.84:0/2028720839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00840b0410 0x7f0084150ee0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f0078007ef0 tx=0x7f007800e4b0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.004+0000 7f008c0cf640 1 -- 172.21.15.84:0/2028720839 shutdown_connections 2023-10-16T11:22:48.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.004+0000 7f008c0cf640 1 --2- 172.21.15.84:0/2028720839 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f006806af90 0x7f006806d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.004+0000 7f008c0cf640 1 --2- 172.21.15.84:0/2028720839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f008415db00 0x7f00841562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.004+0000 7f008c0cf640 1 --2- 172.21.15.84:0/2028720839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00840cd0c0 0x7f00841e70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.004+0000 7f008c0cf640 1 --2- 172.21.15.84:0/2028720839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00840b0410 0x7f0084150ee0 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.004+0000 7f008c0cf640 1 -- 172.21.15.84:0/2028720839 >> 172.21.15.84:0/2028720839 conn(0x7f00840cb410 msgr2=0x7f00840ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:48.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.004+0000 7f008c0cf640 1 -- 172.21.15.84:0/2028720839 shutdown_connections 2023-10-16T11:22:48.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.004+0000 7f008c0cf640 1 -- 172.21.15.84:0/2028720839 wait complete. 2023-10-16T11:22:48.024 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-16T11:22:47.265038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1984,"num_read_kb":4056,"num_write":585,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2783,"ondisk_log_size":2783,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":370344,"kb_used_data":20904,"kb_used_omap":12,"kb_used_meta":349299,"kb_avail":754604376,"statfs":{"total":773094113280,"available":772714881024,"internally_reserved":0,"allocated":21405696,"data_stored":10939500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":357682745},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001514"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.782334+0000","last_change":"2023-10-16T11:22:43.149474+0000","last_active":"2023-10-16T11:22:44.782334+0000","last_peered":"2023-10-16T11:22:44.782334+0000","last_clean":"2023-10-16T11:22:44.782334+0000","last_became_active":"2023-10-16T11:22:43.144741+0000","last_became_peered":"2023-10-16T11:22:43.144741+0000","last_unstale":"2023-10-16T11:22:44.782334+0000","last_undegraded":"2023-10-16T11:22:44.782334+0000","last_fullsized":"2023-10-16T11:22:44.782334+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:39:40.256126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.778221+0000","last_change":"2023-10-16T11:22:43.153828+0000","last_active":"2023-10-16T11:22:44.778221+0000","last_peered":"2023-10-16T11:22:44.778221+0000","last_clean":"2023-10-16T11:22:44.778221+0000","last_became_active":"2023-10-16T11:22:43.153565+0000","last_became_peered":"2023-10-16T11:22:43.153565+0000","last_unstale":"2023-10-16T11:22:44.778221+0000","last_undegraded":"2023-10-16T11:22:44.778221+0000","last_fullsized":"2023-10-16T11:22:44.778221+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:15:00.818017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.365643+0000","last_change":"2023-10-16T11:22:43.151850+0000","last_active":"2023-10-16T11:22:44.365643+0000","last_peered":"2023-10-16T11:22:44.365643+0000","last_clean":"2023-10-16T11:22:44.365643+0000","last_became_active":"2023-10-16T11:22:43.151265+0000","last_became_peered":"2023-10-16T11:22:43.151265+0000","last_unstale":"2023-10-16T11:22:44.365643+0000","last_undegraded":"2023-10-16T11:22:44.365643+0000","last_fullsized":"2023-10-16T11:22:44.365643+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:49:19.037725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.779278+0000","last_change":"2023-10-16T11:22:43.158178+0000","last_active":"2023-10-16T11:22:44.779278+0000","last_peered":"2023-10-16T11:22:44.779278+0000","last_clean":"2023-10-16T11:22:44.779278+0000","last_became_active":"2023-10-16T11:22:43.157152+0000","last_became_peered":"2023-10-16T11:22:43.157152+0000","last_unstale":"2023-10-16T11:22:44.779278+0000","last_undegraded":"2023-10-16T11:22:44.779278+0000","last_fullsized":"2023-10-16T11:22:44.779278+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:14.278273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.139701+0000","last_change":"2023-10-16T11:22:43.153640+0000","last_active":"2023-10-16T11:22:44.139701+0000","last_peered":"2023-10-16T11:22:44.139701+0000","last_clean":"2023-10-16T11:22:44.139701+0000","last_became_active":"2023-10-16T11:22:43.153068+0000","last_became_peered":"2023-10-16T11:22:43.153068+0000","last_unstale":"2023-10-16T11:22:44.139701+0000","last_undegraded":"2023-10-16T11:22:44.139701+0000","last_fullsized":"2023-10-16T11:22:44.139701+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:30.932439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.365549+0000","last_change":"2023-10-16T11:22:43.147968+0000","last_active":"2023-10-16T11:22:44.365549+0000","last_peered":"2023-10-16T11:22:44.365549+0000","last_clean":"2023-10-16T11:22:44.365549+0000","last_became_active":"2023-10-16T11:22:43.147504+0000","last_became_peered":"2023-10-16T11:22:43.147504+0000","last_unstale":"2023-10-16T11:22:44.365549+0000","last_undegraded":"2023-10-16T11:22:44.365549+0000","last_fullsized":"2023-10-16T11:22:44.365549+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:02:28.333340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.143882+0000","last_change":"2023-10-16T11:22:43.143997+0000","last_active":"2023-10-16T11:22:44.143882+0000","last_peered":"2023-10-16T11:22:44.143882+0000","last_clean":"2023-10-16T11:22:44.143882+0000","last_became_active":"2023-10-16T11:22:43.143687+0000","last_became_peered":"2023-10-16T11:22:43.143687+0000","last_unstale":"2023-10-16T11:22:44.143882+0000","last_undegraded":"2023-10-16T11:22:44.143882+0000","last_fullsized":"2023-10-16T11:22:44.143882+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:18.031947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.143025+0000","last_change":"2023-10-16T11:22:43.141942+0000","last_active":"2023-10-16T11:22:44.143025+0000","last_peered":"2023-10-16T11:22:44.143025+0000","last_clean":"2023-10-16T11:22:44.143025+0000","last_became_active":"2023-10-16T11:22:43.141497+0000","last_became_peered":"2023-10-16T11:22:43.141497+0000","last_unstale":"2023-10-16T11:22:44.143025+0000","last_undegraded":"2023-10-16T11:22:44.143025+0000","last_fullsized":"2023-10-16T11:22:44.143025+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:41:45.318600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.781545+0000","last_change":"2023-10-16T11:22:43.161474+0000","last_active":"2023-10-16T11:22:44.781545+0000","last_peered":"2023-10-16T11:22:44.781545+0000","last_clean":"2023-10-16T11:22:44.781545+0000","last_became_active":"2023-10-16T11:22:43.160437+0000","last_became_peered":"2023-10-16T11:22:43.160437+0000","last_unstale":"2023-10-16T11:22:44.781545+0000","last_undegraded":"2023-10-16T11:22:44.781545+0000","last_fullsized":"2023-10-16T11:22:44.781545+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:46:18.164883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.777974+0000","last_change":"2023-10-16T11:22:43.130871+0000","last_active":"2023-10-16T11:22:44.777974+0000","last_peered":"2023-10-16T11:22:44.777974+0000","last_clean":"2023-10-16T11:22:44.777974+0000","last_became_active":"2023-10-16T11:22:43.128640+0000","last_became_peered":"2023-10-16T11:22:43.128640+0000","last_unstale":"2023-10-16T11:22:44.777974+0000","last_undegraded":"2023-10-16T11:22:44.777974+0000","last_fullsized":"2023-10-16T11:22:44.777974+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:17:16.487265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.145083+0000","last_change":"2023-10-16T11:22:43.148766+0000","last_active":"2023-10-16T11:22:44.145083+0000","last_peered":"2023-10-16T11:22:44.145083+0000","last_clean":"2023-10-16T11:22:44.145083+0000","last_became_active":"2023-10-16T11:22:43.148223+0000","last_became_peered":"2023-10-16T11:22:43.148223+0000","last_unstale":"2023-10-16T11:22:44.145083+0000","last_undegraded":"2023-10-16T11:22:44.145083+0000","last_fullsized":"2023-10-16T11:22:44.145083+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:27:59.982771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.779086+0000","last_change":"2023-10-16T11:22:43.157913+0000","last_active":"2023-10-16T11:22:44.779086+0000","last_peered":"2023-10-16T11:22:44.779086+0000","last_clean":"2023-10-16T11:22:44.779086+0000","last_became_active":"2023-10-16T11:22:43.157647+0000","last_became_peered":"2023-10-16T11:22:43.157647+0000","last_unstale":"2023-10-16T11:22:44.779086+0000","last_undegraded":"2023-10-16T11:22:44.779086+0000","last_fullsized":"2023-10-16T11:22:44.779086+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:41:02.900841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.778243+0000","last_change":"2023-10-16T11:22:43.157581+0000","last_active":"2023-10-16T11:22:44.778243+0000","last_peered":"2023-10-16T11:22:44.778243+0000","last_clean":"2023-10-16T11:22:44.778243+0000","last_became_active":"2023-10-16T11:22:43.157323+0000","last_became_peered":"2023-10-16T11:22:43.157323+0000","last_unstale":"2023-10-16T11:22:44.778243+0000","last_undegraded":"2023-10-16T11:22:44.778243+0000","last_fullsized":"2023-10-16T11:22:44.778243+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:18.076470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.779718+0000","last_change":"2023-10-16T11:22:43.149124+0000","last_active":"2023-10-16T11:22:44.779718+0000","last_peered":"2023-10-16T11:22:44.779718+0000","last_clean":"2023-10-16T11:22:44.779718+0000","last_became_active":"2023-10-16T11:22:43.148819+0000","last_became_peered":"2023-10-16T11:22:43.148819+0000","last_unstale":"2023-10-16T11:22:44.779718+0000","last_undegraded":"2023-10-16T11:22:44.779718+0000","last_fullsized":"2023-10-16T11:22:44.779718+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:38.845724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.778999+0000","last_change":"2023-10-16T11:22:43.157461+0000","last_active":"2023-10-16T11:22:44.778999+0000","last_peered":"2023-10-16T11:22:44.778999+0000","last_clean":"2023-10-16T11:22:44.778999+0000","last_became_active":"2023-10-16T11:22:43.156806+0000","last_became_peered":"2023-10-16T11:22:43.156806+0000","last_unstale":"2023-10-16T11:22:44.778999+0000","last_undegraded":"2023-10-16T11:22:44.778999+0000","last_fullsized":"2023-10-16T11:22:44.778999+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:41.900125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.143039+0000","last_change":"2023-10-16T11:22:43.154539+0000","last_active":"2023-10-16T11:22:44.143039+0000","last_peered":"2023-10-16T11:22:44.143039+0000","last_clean":"2023-10-16T11:22:44.143039+0000","last_became_active":"2023-10-16T11:22:43.154228+0000","last_became_peered":"2023-10-16T11:22:43.154228+0000","last_unstale":"2023-10-16T11:22:44.143039+0000","last_undegraded":"2023-10-16T11:22:44.143039+0000","last_fullsized":"2023-10-16T11:22:44.143039+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:25.932368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.139700+0000","last_change":"2023-10-16T11:22:43.160126+0000","last_active":"2023-10-16T11:22:44.139700+0000","last_peered":"2023-10-16T11:22:44.139700+0000","last_clean":"2023-10-16T11:22:44.139700+0000","last_became_active":"2023-10-16T11:22:43.159776+0000","last_became_peered":"2023-10-16T11:22:43.159776+0000","last_unstale":"2023-10-16T11:22:44.139700+0000","last_undegraded":"2023-10-16T11:22:44.139700+0000","last_fullsized":"2023-10-16T11:22:44.139700+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:00:27.289128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.782077+0000","last_change":"2023-10-16T11:22:43.162922+0000","last_active":"2023-10-16T11:22:44.782077+0000","last_peered":"2023-10-16T11:22:44.782077+0000","last_clean":"2023-10-16T11:22:44.782077+0000","last_became_active":"2023-10-16T11:22:43.162388+0000","last_became_peered":"2023-10-16T11:22:43.162388+0000","last_unstale":"2023-10-16T11:22:44.782077+0000","last_undegraded":"2023-10-16T11:22:44.782077+0000","last_fullsized":"2023-10-16T11:22:44.782077+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:57.609751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.777484+0000","last_change":"2023-10-16T11:22:43.152352+0000","last_active":"2023-10-16T11:22:44.777484+0000","last_peered":"2023-10-16T11:22:44.777484+0000","last_clean":"2023-10-16T11:22:44.777484+0000","last_became_active":"2023-10-16T11:22:43.151971+0000","last_became_peered":"2023-10-16T11:22:43.151971+0000","last_unstale":"2023-10-16T11:22:44.777484+0000","last_undegraded":"2023-10-16T11:22:44.777484+0000","last_fullsized":"2023-10-16T11:22:44.777484+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:27.191744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.139381+0000","last_change":"2023-10-16T11:22:43.157919+0000","last_active":"2023-10-16T11:22:44.139381+0000","last_peered":"2023-10-16T11:22:44.139381+0000","last_clean":"2023-10-16T11:22:44.139381+0000","last_became_active":"2023-10-16T11:22:43.157396+0000","last_became_peered":"2023-10-16T11:22:43.157396+0000","last_unstale":"2023-10-16T11:22:44.139381+0000","last_undegraded":"2023-10-16T11:22:44.139381+0000","last_fullsized":"2023-10-16T11:22:44.139381+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:27.191744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.139815+0000","last_change":"2023-10-16T11:22:43.152444+0000","last_active":"2023-10-16T11:22:44.139815+0000","last_peered":"2023-10-16T11:22:44.139815+0000","last_clean":"2023-10-16T11:22:44.139815+0000","last_became_active":"2023-10-16T11:22:43.151892+0000","last_became_peered":"2023-10-16T11:22:43.151892+0000","last_unstale":"2023-10-16T11:22:44.139815+0000","last_undegraded":"2023-10-16T11:22:44.139815+0000","last_fullsized":"2023-10-16T11:22:44.139815+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:30.024400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.777315+0000","last_change":"2023-10-16T11:22:43.141916+0000","last_active":"2023-10-16T11:22:44.777315+0000","last_peered":"2023-10-16T11:22:44.777315+0000","last_clean":"2023-10-16T11:22:44.777315+0000","last_became_active":"2023-10-16T11:22:43.141616+0000","last_became_peered":"2023-10-16T11:22:43.141616+0000","last_unstale":"2023-10-16T11:22:44.777315+0000","last_undegraded":"2023-10-16T11:22:44.777315+0000","last_fullsized":"2023-10-16T11:22:44.777315+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:10:36.929431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.365542+0000","last_change":"2023-10-16T11:22:43.136522+0000","last_active":"2023-10-16T11:22:44.365542+0000","last_peered":"2023-10-16T11:22:44.365542+0000","last_clean":"2023-10-16T11:22:44.365542+0000","last_became_active":"2023-10-16T11:22:43.135779+0000","last_became_peered":"2023-10-16T11:22:43.135779+0000","last_unstale":"2023-10-16T11:22:44.365542+0000","last_undegraded":"2023-10-16T11:22:44.365542+0000","last_fullsized":"2023-10-16T11:22:44.365542+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:19:40.726706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.143438+0000","last_change":"2023-10-16T11:22:43.141753+0000","last_active":"2023-10-16T11:22:44.143438+0000","last_peered":"2023-10-16T11:22:44.143438+0000","last_clean":"2023-10-16T11:22:44.143438+0000","last_became_active":"2023-10-16T11:22:43.140601+0000","last_became_peered":"2023-10-16T11:22:43.140601+0000","last_unstale":"2023-10-16T11:22:44.143438+0000","last_undegraded":"2023-10-16T11:22:44.143438+0000","last_fullsized":"2023-10-16T11:22:44.143438+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:57.628121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.779802+0000","last_change":"2023-10-16T11:22:43.155429+0000","last_active":"2023-10-16T11:22:44.779802+0000","last_peered":"2023-10-16T11:22:44.779802+0000","last_clean":"2023-10-16T11:22:44.779802+0000","last_became_active":"2023-10-16T11:22:43.151462+0000","last_became_peered":"2023-10-16T11:22:43.151462+0000","last_unstale":"2023-10-16T11:22:44.779802+0000","last_undegraded":"2023-10-16T11:22:44.779802+0000","last_fullsized":"2023-10-16T11:22:44.779802+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:21.406202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.144741+0000","last_change":"2023-10-16T11:22:43.144609+0000","last_active":"2023-10-16T11:22:44.144741+0000","last_peered":"2023-10-16T11:22:44.144741+0000","last_clean":"2023-10-16T11:22:44.144741+0000","last_became_active":"2023-10-16T11:22:43.144352+0000","last_became_peered":"2023-10-16T11:22:43.144352+0000","last_unstale":"2023-10-16T11:22:44.144741+0000","last_undegraded":"2023-10-16T11:22:44.144741+0000","last_fullsized":"2023-10-16T11:22:44.144741+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:29:23.693408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.365792+0000","last_change":"2023-10-16T11:22:43.144593+0000","last_active":"2023-10-16T11:22:44.365792+0000","last_peered":"2023-10-16T11:22:44.365792+0000","last_clean":"2023-10-16T11:22:44.365792+0000","last_became_active":"2023-10-16T11:22:43.144241+0000","last_became_peered":"2023-10-16T11:22:43.144241+0000","last_unstale":"2023-10-16T11:22:44.365792+0000","last_undegraded":"2023-10-16T11:22:44.365792+0000","last_fullsized":"2023-10-16T11:22:44.365792+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:14:56.763801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.139511+0000","last_change":"2023-10-16T11:22:43.127494+0000","last_active":"2023-10-16T11:22:44.139511+0000","last_peered":"2023-10-16T11:22:44.139511+0000","last_clean":"2023-10-16T11:22:44.139511+0000","last_became_active":"2023-10-16T11:22:43.127215+0000","last_became_peered":"2023-10-16T11:22:43.127215+0000","last_unstale":"2023-10-16T11:22:44.139511+0000","last_undegraded":"2023-10-16T11:22:44.139511+0000","last_fullsized":"2023-10-16T11:22:44.139511+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:00:57.360176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.778593+0000","last_change":"2023-10-16T11:22:43.161993+0000","last_active":"2023-10-16T11:22:44.778593+0000","last_peered":"2023-10-16T11:22:44.778593+0000","last_clean":"2023-10-16T11:22:44.778593+0000","last_became_active":"2023-10-16T11:22:43.161573+0000","last_became_peered":"2023-10-16T11:22:43.161573+0000","last_unstale":"2023-10-16T11:22:44.778593+0000","last_undegraded":"2023-10-16T11:22:44.778593+0000","last_fullsized":"2023-10-16T11:22:44.778593+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:19:43.925889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.780097+0000","last_change":"2023-10-16T11:22:43.151736+0000","last_active":"2023-10-16T11:22:44.780097+0000","last_peered":"2023-10-16T11:22:44.780097+0000","last_clean":"2023-10-16T11:22:44.780097+0000","last_became_active":"2023-10-16T11:22:43.151465+0000","last_became_peered":"2023-10-16T11:22:43.151465+0000","last_unstale":"2023-10-16T11:22:44.780097+0000","last_undegraded":"2023-10-16T11:22:44.780097+0000","last_fullsized":"2023-10-16T11:22:44.780097+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:29:23.693408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.365703+0000","last_change":"2023-10-16T11:22:43.159343+0000","last_active":"2023-10-16T11:22:44.365703+0000","last_peered":"2023-10-16T11:22:44.365703+0000","last_clean":"2023-10-16T11:22:44.365703+0000","last_became_active":"2023-10-16T11:22:43.158982+0000","last_became_peered":"2023-10-16T11:22:43.158982+0000","last_unstale":"2023-10-16T11:22:44.365703+0000","last_undegraded":"2023-10-16T11:22:44.365703+0000","last_fullsized":"2023-10-16T11:22:44.365703+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:58.410334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.139969+0000","last_change":"2023-10-16T11:22:43.127397+0000","last_active":"2023-10-16T11:22:44.139969+0000","last_peered":"2023-10-16T11:22:44.139969+0000","last_clean":"2023-10-16T11:22:44.139969+0000","last_became_active":"2023-10-16T11:22:43.127128+0000","last_became_peered":"2023-10-16T11:22:43.127128+0000","last_unstale":"2023-10-16T11:22:44.139969+0000","last_undegraded":"2023-10-16T11:22:44.139969+0000","last_fullsized":"2023-10-16T11:22:44.139969+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:39:55.469848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.143850+0000","last_change":"2023-10-16T11:22:43.150123+0000","last_active":"2023-10-16T11:22:44.143850+0000","last_peered":"2023-10-16T11:22:44.143850+0000","last_clean":"2023-10-16T11:22:44.143850+0000","last_became_active":"2023-10-16T11:22:43.149862+0000","last_became_peered":"2023-10-16T11:22:43.149862+0000","last_unstale":"2023-10-16T11:22:44.143850+0000","last_undegraded":"2023-10-16T11:22:44.143850+0000","last_fullsized":"2023-10-16T11:22:44.143850+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:57:38.033406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"200'4","reported_seq":16,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.153780+0000","last_change":"2023-10-16T11:22:42.142006+0000","last_active":"2023-10-16T11:22:43.153780+0000","last_peered":"2023-10-16T11:22:43.153780+0000","last_clean":"2023-10-16T11:22:43.153780+0000","last_became_active":"2023-10-16T11:22:42.141599+0000","last_became_peered":"2023-10-16T11:22:42.141599+0000","last_unstale":"2023-10-16T11:22:43.153780+0000","last_undegraded":"2023-10-16T11:22:43.153780+0000","last_fullsized":"2023-10-16T11:22:43.153780+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:31:53.479584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.778321+0000","last_change":"2023-10-16T11:22:43.136208+0000","last_active":"2023-10-16T11:22:44.778321+0000","last_peered":"2023-10-16T11:22:44.778321+0000","last_clean":"2023-10-16T11:22:44.778321+0000","last_became_active":"2023-10-16T11:22:43.135583+0000","last_became_peered":"2023-10-16T11:22:43.135583+0000","last_unstale":"2023-10-16T11:22:44.778321+0000","last_undegraded":"2023-10-16T11:22:44.778321+0000","last_fullsized":"2023-10-16T11:22:44.778321+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:30.932439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"200'1","reported_seq":13,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.163784+0000","last_change":"2023-10-16T11:22:42.143922+0000","last_active":"2023-10-16T11:22:43.163784+0000","last_peered":"2023-10-16T11:22:43.163784+0000","last_clean":"2023-10-16T11:22:43.163784+0000","last_became_active":"2023-10-16T11:22:42.143630+0000","last_became_peered":"2023-10-16T11:22:42.143630+0000","last_unstale":"2023-10-16T11:22:43.163784+0000","last_undegraded":"2023-10-16T11:22:43.163784+0000","last_fullsized":"2023-10-16T11:22:43.163784+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:04.995252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.144350+0000","last_change":"2023-10-16T11:22:43.135610+0000","last_active":"2023-10-16T11:22:44.144350+0000","last_peered":"2023-10-16T11:22:44.144350+0000","last_clean":"2023-10-16T11:22:44.144350+0000","last_became_active":"2023-10-16T11:22:43.134534+0000","last_became_peered":"2023-10-16T11:22:43.134534+0000","last_unstale":"2023-10-16T11:22:44.144350+0000","last_undegraded":"2023-10-16T11:22:44.144350+0000","last_fullsized":"2023-10-16T11:22:44.144350+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:38.845724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"200'2","reported_seq":14,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.134884+0000","last_change":"2023-10-16T11:22:42.135404+0000","last_active":"2023-10-16T11:22:43.134884+0000","last_peered":"2023-10-16T11:22:43.134884+0000","last_clean":"2023-10-16T11:22:43.134884+0000","last_became_active":"2023-10-16T11:22:42.134875+0000","last_became_peered":"2023-10-16T11:22:42.134875+0000","last_unstale":"2023-10-16T11:22:43.134884+0000","last_undegraded":"2023-10-16T11:22:43.134884+0000","last_fullsized":"2023-10-16T11:22:43.134884+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:37.827150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.781042+0000","last_change":"2023-10-16T11:22:43.146755+0000","last_active":"2023-10-16T11:22:44.781042+0000","last_peered":"2023-10-16T11:22:44.781042+0000","last_clean":"2023-10-16T11:22:44.781042+0000","last_became_active":"2023-10-16T11:22:43.146367+0000","last_became_peered":"2023-10-16T11:22:43.146367+0000","last_unstale":"2023-10-16T11:22:44.781042+0000","last_undegraded":"2023-10-16T11:22:44.781042+0000","last_fullsized":"2023-10-16T11:22:44.781042+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:27:59.982771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"200'2","reported_seq":14,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.163307+0000","last_change":"2023-10-16T11:22:42.146169+0000","last_active":"2023-10-16T11:22:43.163307+0000","last_peered":"2023-10-16T11:22:43.163307+0000","last_clean":"2023-10-16T11:22:43.163307+0000","last_became_active":"2023-10-16T11:22:42.145790+0000","last_became_peered":"2023-10-16T11:22:42.145790+0000","last_unstale":"2023-10-16T11:22:43.163307+0000","last_undegraded":"2023-10-16T11:22:43.163307+0000","last_fullsized":"2023-10-16T11:22:43.163307+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:47:31.440178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.777405+0000","last_change":"2023-10-16T11:22:43.141311+0000","last_active":"2023-10-16T11:22:44.777405+0000","last_peered":"2023-10-16T11:22:44.777405+0000","last_clean":"2023-10-16T11:22:44.777405+0000","last_became_active":"2023-10-16T11:22:43.139288+0000","last_became_peered":"2023-10-16T11:22:43.139288+0000","last_unstale":"2023-10-16T11:22:44.777405+0000","last_undegraded":"2023-10-16T11:22:44.777405+0000","last_fullsized":"2023-10-16T11:22:44.777405+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:14.840220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"200'1","reported_seq":13,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.141765+0000","last_change":"2023-10-16T11:22:42.133138+0000","last_active":"2023-10-16T11:22:43.141765+0000","last_peered":"2023-10-16T11:22:43.141765+0000","last_clean":"2023-10-16T11:22:43.141765+0000","last_became_active":"2023-10-16T11:22:42.131907+0000","last_became_peered":"2023-10-16T11:22:42.131907+0000","last_unstale":"2023-10-16T11:22:43.141765+0000","last_undegraded":"2023-10-16T11:22:43.141765+0000","last_fullsized":"2023-10-16T11:22:43.141765+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:10:35.910857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.143394+0000","last_change":"2023-10-16T11:22:43.157137+0000","last_active":"2023-10-16T11:22:44.143394+0000","last_peered":"2023-10-16T11:22:44.143394+0000","last_clean":"2023-10-16T11:22:44.143394+0000","last_became_active":"2023-10-16T11:22:43.156523+0000","last_became_peered":"2023-10-16T11:22:43.156523+0000","last_unstale":"2023-10-16T11:22:44.143394+0000","last_undegraded":"2023-10-16T11:22:44.143394+0000","last_fullsized":"2023-10-16T11:22:44.143394+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:06:20.008986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"200'1","reported_seq":13,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.138395+0000","last_change":"2023-10-16T11:22:42.124819+0000","last_active":"2023-10-16T11:22:43.138395+0000","last_peered":"2023-10-16T11:22:43.138395+0000","last_clean":"2023-10-16T11:22:43.138395+0000","last_became_active":"2023-10-16T11:22:42.124531+0000","last_became_peered":"2023-10-16T11:22:42.124531+0000","last_unstale":"2023-10-16T11:22:43.138395+0000","last_undegraded":"2023-10-16T11:22:43.138395+0000","last_fullsized":"2023-10-16T11:22:43.138395+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:41.901577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.143949+0000","last_change":"2023-10-16T11:22:43.154104+0000","last_active":"2023-10-16T11:22:44.143949+0000","last_peered":"2023-10-16T11:22:44.143949+0000","last_clean":"2023-10-16T11:22:44.143949+0000","last_became_active":"2023-10-16T11:22:43.152997+0000","last_became_peered":"2023-10-16T11:22:43.152997+0000","last_unstale":"2023-10-16T11:22:44.143949+0000","last_undegraded":"2023-10-16T11:22:44.143949+0000","last_fullsized":"2023-10-16T11:22:44.143949+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:51.502199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.134025+0000","last_change":"2023-10-16T11:22:42.139731+0000","last_active":"2023-10-16T11:22:43.134025+0000","last_peered":"2023-10-16T11:22:43.134025+0000","last_clean":"2023-10-16T11:22:43.134025+0000","last_became_active":"2023-10-16T11:22:42.139416+0000","last_became_peered":"2023-10-16T11:22:42.139416+0000","last_unstale":"2023-10-16T11:22:43.134025+0000","last_undegraded":"2023-10-16T11:22:43.134025+0000","last_fullsized":"2023-10-16T11:22:43.134025+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:24.913795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.366282+0000","last_change":"2023-10-16T11:22:43.137756+0000","last_active":"2023-10-16T11:22:44.366282+0000","last_peered":"2023-10-16T11:22:44.366282+0000","last_clean":"2023-10-16T11:22:44.366282+0000","last_became_active":"2023-10-16T11:22:43.137384+0000","last_became_peered":"2023-10-16T11:22:43.137384+0000","last_unstale":"2023-10-16T11:22:44.366282+0000","last_undegraded":"2023-10-16T11:22:44.366282+0000","last_fullsized":"2023-10-16T11:22:44.366282+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:51.998114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.127361+0000","last_change":"2023-10-16T11:22:42.144254+0000","last_active":"2023-10-16T11:22:43.127361+0000","last_peered":"2023-10-16T11:22:43.127361+0000","last_clean":"2023-10-16T11:22:43.127361+0000","last_became_active":"2023-10-16T11:22:42.143949+0000","last_became_peered":"2023-10-16T11:22:42.143949+0000","last_unstale":"2023-10-16T11:22:43.127361+0000","last_undegraded":"2023-10-16T11:22:43.127361+0000","last_fullsized":"2023-10-16T11:22:43.127361+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:39:54.451275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.143933+0000","last_change":"2023-10-16T11:22:43.145118+0000","last_active":"2023-10-16T11:22:44.143933+0000","last_peered":"2023-10-16T11:22:44.143933+0000","last_clean":"2023-10-16T11:22:44.143933+0000","last_became_active":"2023-10-16T11:22:43.144858+0000","last_became_peered":"2023-10-16T11:22:43.144858+0000","last_unstale":"2023-10-16T11:22:44.143933+0000","last_undegraded":"2023-10-16T11:22:44.143933+0000","last_fullsized":"2023-10-16T11:22:44.143933+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:19:52.601828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.116915+0000","last_change":"2023-10-16T11:22:42.137290+0000","last_active":"2023-10-16T11:22:43.116915+0000","last_peered":"2023-10-16T11:22:43.116915+0000","last_clean":"2023-10-16T11:22:43.116915+0000","last_became_active":"2023-10-16T11:22:42.136782+0000","last_became_peered":"2023-10-16T11:22:42.136782+0000","last_unstale":"2023-10-16T11:22:43.116915+0000","last_undegraded":"2023-10-16T11:22:43.116915+0000","last_fullsized":"2023-10-16T11:22:43.116915+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:16.132848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.144303+0000","last_change":"2023-10-16T11:22:43.150241+0000","last_active":"2023-10-16T11:22:44.144303+0000","last_peered":"2023-10-16T11:22:44.144303+0000","last_clean":"2023-10-16T11:22:44.144303+0000","last_became_active":"2023-10-16T11:22:43.149428+0000","last_became_peered":"2023-10-16T11:22:43.149428+0000","last_unstale":"2023-10-16T11:22:44.144303+0000","last_undegraded":"2023-10-16T11:22:44.144303+0000","last_fullsized":"2023-10-16T11:22:44.144303+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:05.826888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"200'3","reported_seq":15,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.145459+0000","last_change":"2023-10-16T11:22:42.126340+0000","last_active":"2023-10-16T11:22:43.145459+0000","last_peered":"2023-10-16T11:22:43.145459+0000","last_clean":"2023-10-16T11:22:43.145459+0000","last_became_active":"2023-10-16T11:22:42.126054+0000","last_became_peered":"2023-10-16T11:22:42.126054+0000","last_unstale":"2023-10-16T11:22:43.145459+0000","last_undegraded":"2023-10-16T11:22:43.145459+0000","last_fullsized":"2023-10-16T11:22:43.145459+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:35:26.282525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.777767+0000","last_change":"2023-10-16T11:22:43.141026+0000","last_active":"2023-10-16T11:22:44.777767+0000","last_peered":"2023-10-16T11:22:44.777767+0000","last_clean":"2023-10-16T11:22:44.777767+0000","last_became_active":"2023-10-16T11:22:43.139354+0000","last_became_peered":"2023-10-16T11:22:43.139354+0000","last_unstale":"2023-10-16T11:22:44.777767+0000","last_undegraded":"2023-10-16T11:22:44.777767+0000","last_fullsized":"2023-10-16T11:22:44.777767+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:00:57.360176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"200'6","reported_seq":18,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.148664+0000","last_change":"2023-10-16T11:22:42.142251+0000","last_active":"2023-10-16T11:22:43.148664+0000","last_peered":"2023-10-16T11:22:43.148664+0000","last_clean":"2023-10-16T11:22:43.148664+0000","last_became_active":"2023-10-16T11:22:42.141801+0000","last_became_peered":"2023-10-16T11:22:42.141801+0000","last_unstale":"2023-10-16T11:22:43.148664+0000","last_undegraded":"2023-10-16T11:22:43.148664+0000","last_fullsized":"2023-10-16T11:22:43.148664+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:04.808314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.366111+0000","last_change":"2023-10-16T11:22:43.136318+0000","last_active":"2023-10-16T11:22:44.366111+0000","last_peered":"2023-10-16T11:22:44.366111+0000","last_clean":"2023-10-16T11:22:44.366111+0000","last_became_active":"2023-10-16T11:22:43.136054+0000","last_became_peered":"2023-10-16T11:22:43.136054+0000","last_unstale":"2023-10-16T11:22:44.366111+0000","last_undegraded":"2023-10-16T11:22:44.366111+0000","last_fullsized":"2023-10-16T11:22:44.366111+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:31.448819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"200'2","reported_seq":14,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.153214+0000","last_change":"2023-10-16T11:22:42.141840+0000","last_active":"2023-10-16T11:22:43.153214+0000","last_peered":"2023-10-16T11:22:43.153214+0000","last_clean":"2023-10-16T11:22:43.153214+0000","last_became_active":"2023-10-16T11:22:42.141513+0000","last_became_peered":"2023-10-16T11:22:42.141513+0000","last_unstale":"2023-10-16T11:22:43.153214+0000","last_undegraded":"2023-10-16T11:22:43.153214+0000","last_fullsized":"2023-10-16T11:22:43.153214+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:57:37.014832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.144996+0000","last_change":"2023-10-16T11:22:43.134668+0000","last_active":"2023-10-16T11:22:44.144996+0000","last_peered":"2023-10-16T11:22:44.144996+0000","last_clean":"2023-10-16T11:22:44.144996+0000","last_became_active":"2023-10-16T11:22:43.134348+0000","last_became_peered":"2023-10-16T11:22:43.134348+0000","last_unstale":"2023-10-16T11:22:44.144996+0000","last_undegraded":"2023-10-16T11:22:44.144996+0000","last_fullsized":"2023-10-16T11:22:44.144996+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:46:18.164883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.118395+0000","last_change":"2023-10-16T11:22:42.130286+0000","last_active":"2023-10-16T11:22:43.118395+0000","last_peered":"2023-10-16T11:22:43.118395+0000","last_clean":"2023-10-16T11:22:43.118395+0000","last_became_active":"2023-10-16T11:22:42.129097+0000","last_became_peered":"2023-10-16T11:22:42.129097+0000","last_unstale":"2023-10-16T11:22:43.118395+0000","last_undegraded":"2023-10-16T11:22:43.118395+0000","last_fullsized":"2023-10-16T11:22:43.118395+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:03:11.366187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.778170+0000","last_change":"2023-10-16T11:22:43.150104+0000","last_active":"2023-10-16T11:22:44.778170+0000","last_peered":"2023-10-16T11:22:44.778170+0000","last_clean":"2023-10-16T11:22:44.778170+0000","last_became_active":"2023-10-16T11:22:43.149844+0000","last_became_peered":"2023-10-16T11:22:43.149844+0000","last_unstale":"2023-10-16T11:22:44.778170+0000","last_undegraded":"2023-10-16T11:22:44.778170+0000","last_fullsized":"2023-10-16T11:22:44.778170+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:05.826888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.124390+0000","last_change":"2023-10-16T11:22:42.132900+0000","last_active":"2023-10-16T11:22:43.124390+0000","last_peered":"2023-10-16T11:22:43.124390+0000","last_clean":"2023-10-16T11:22:43.124390+0000","last_became_active":"2023-10-16T11:22:42.132561+0000","last_became_peered":"2023-10-16T11:22:42.132561+0000","last_unstale":"2023-10-16T11:22:43.124390+0000","last_undegraded":"2023-10-16T11:22:43.124390+0000","last_fullsized":"2023-10-16T11:22:43.124390+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:27:49.297250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.777475+0000","last_change":"2023-10-16T11:22:43.130692+0000","last_active":"2023-10-16T11:22:44.777475+0000","last_peered":"2023-10-16T11:22:44.777475+0000","last_clean":"2023-10-16T11:22:44.777475+0000","last_became_active":"2023-10-16T11:22:43.128801+0000","last_became_peered":"2023-10-16T11:22:43.128801+0000","last_unstale":"2023-10-16T11:22:44.777475+0000","last_undegraded":"2023-10-16T11:22:44.777475+0000","last_fullsized":"2023-10-16T11:22:44.777475+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:35:27.301099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"200'8","reported_seq":20,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.157499+0000","last_change":"2023-10-16T11:22:42.139889+0000","last_active":"2023-10-16T11:22:43.157499+0000","last_peered":"2023-10-16T11:22:43.157499+0000","last_clean":"2023-10-16T11:22:43.157499+0000","last_became_active":"2023-10-16T11:22:42.138021+0000","last_became_peered":"2023-10-16T11:22:42.138021+0000","last_unstale":"2023-10-16T11:22:43.157499+0000","last_undegraded":"2023-10-16T11:22:43.157499+0000","last_fullsized":"2023-10-16T11:22:43.157499+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:06:18.990412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.778209+0000","last_change":"2023-10-16T11:22:43.133379+0000","last_active":"2023-10-16T11:22:44.778209+0000","last_peered":"2023-10-16T11:22:44.778209+0000","last_clean":"2023-10-16T11:22:44.778209+0000","last_became_active":"2023-10-16T11:22:43.132886+0000","last_became_peered":"2023-10-16T11:22:43.132886+0000","last_unstale":"2023-10-16T11:22:44.778209+0000","last_undegraded":"2023-10-16T11:22:44.778209+0000","last_fullsized":"2023-10-16T11:22:44.778209+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:57:39.650999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"200'1","reported_seq":13,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.141646+0000","last_change":"2023-10-16T11:22:42.134657+0000","last_active":"2023-10-16T11:22:43.141646+0000","last_peered":"2023-10-16T11:22:43.141646+0000","last_clean":"2023-10-16T11:22:43.141646+0000","last_became_active":"2023-10-16T11:22:42.134388+0000","last_became_peered":"2023-10-16T11:22:42.134388+0000","last_unstale":"2023-10-16T11:22:43.141646+0000","last_undegraded":"2023-10-16T11:22:43.141646+0000","last_fullsized":"2023-10-16T11:22:43.141646+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:34.656291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.781557+0000","last_change":"2023-10-16T11:22:43.147584+0000","last_active":"2023-10-16T11:22:44.781557+0000","last_peered":"2023-10-16T11:22:44.781557+0000","last_clean":"2023-10-16T11:22:44.781557+0000","last_became_active":"2023-10-16T11:22:43.147307+0000","last_became_peered":"2023-10-16T11:22:43.147307+0000","last_unstale":"2023-10-16T11:22:44.781557+0000","last_undegraded":"2023-10-16T11:22:44.781557+0000","last_fullsized":"2023-10-16T11:22:44.781557+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:19:40.726706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"200'2","reported_seq":14,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.147493+0000","last_change":"2023-10-16T11:22:42.144754+0000","last_active":"2023-10-16T11:22:43.147493+0000","last_peered":"2023-10-16T11:22:43.147493+0000","last_clean":"2023-10-16T11:22:43.147493+0000","last_became_active":"2023-10-16T11:22:42.144507+0000","last_became_peered":"2023-10-16T11:22:42.144507+0000","last_unstale":"2023-10-16T11:22:43.147493+0000","last_undegraded":"2023-10-16T11:22:43.147493+0000","last_fullsized":"2023-10-16T11:22:43.147493+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:00.711474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.780587+0000","last_change":"2023-10-16T11:22:43.155065+0000","last_active":"2023-10-16T11:22:44.780587+0000","last_peered":"2023-10-16T11:22:44.780587+0000","last_clean":"2023-10-16T11:22:44.780587+0000","last_became_active":"2023-10-16T11:22:43.153416+0000","last_became_peered":"2023-10-16T11:22:43.153416+0000","last_unstale":"2023-10-16T11:22:44.780587+0000","last_undegraded":"2023-10-16T11:22:44.780587+0000","last_fullsized":"2023-10-16T11:22:44.780587+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:39:10.429029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"200'1","reported_seq":13,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.147374+0000","last_change":"2023-10-16T11:22:42.139840+0000","last_active":"2023-10-16T11:22:43.147374+0000","last_peered":"2023-10-16T11:22:43.147374+0000","last_clean":"2023-10-16T11:22:43.147374+0000","last_became_active":"2023-10-16T11:22:42.139543+0000","last_became_peered":"2023-10-16T11:22:42.139543+0000","last_unstale":"2023-10-16T11:22:43.147374+0000","last_undegraded":"2023-10-16T11:22:43.147374+0000","last_fullsized":"2023-10-16T11:22:43.147374+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:24.457369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.778590+0000","last_change":"2023-10-16T11:22:43.154863+0000","last_active":"2023-10-16T11:22:44.778590+0000","last_peered":"2023-10-16T11:22:44.778590+0000","last_clean":"2023-10-16T11:22:44.778590+0000","last_became_active":"2023-10-16T11:22:43.154592+0000","last_became_peered":"2023-10-16T11:22:43.154592+0000","last_unstale":"2023-10-16T11:22:44.778590+0000","last_undegraded":"2023-10-16T11:22:44.778590+0000","last_fullsized":"2023-10-16T11:22:44.778590+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:32:04.649162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.127704+0000","last_change":"2023-10-16T11:22:42.141538+0000","last_active":"2023-10-16T11:22:43.127704+0000","last_peered":"2023-10-16T11:22:43.127704+0000","last_clean":"2023-10-16T11:22:43.127704+0000","last_became_active":"2023-10-16T11:22:42.140570+0000","last_became_peered":"2023-10-16T11:22:42.140570+0000","last_unstale":"2023-10-16T11:22:43.127704+0000","last_undegraded":"2023-10-16T11:22:43.127704+0000","last_fullsized":"2023-10-16T11:22:43.127704+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:48.644799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.777678+0000","last_change":"2023-10-16T11:22:43.124735+0000","last_active":"2023-10-16T11:22:44.777678+0000","last_peered":"2023-10-16T11:22:44.777678+0000","last_clean":"2023-10-16T11:22:44.777678+0000","last_became_active":"2023-10-16T11:22:43.124478+0000","last_became_peered":"2023-10-16T11:22:43.124478+0000","last_unstale":"2023-10-16T11:22:44.777678+0000","last_undegraded":"2023-10-16T11:22:44.777678+0000","last_fullsized":"2023-10-16T11:22:44.777678+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:00.954663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"200'2","reported_seq":14,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.132482+0000","last_change":"2023-10-16T11:22:42.126251+0000","last_active":"2023-10-16T11:22:43.132482+0000","last_peered":"2023-10-16T11:22:43.132482+0000","last_clean":"2023-10-16T11:22:43.132482+0000","last_became_active":"2023-10-16T11:22:42.125897+0000","last_became_peered":"2023-10-16T11:22:42.125897+0000","last_unstale":"2023-10-16T11:22:43.132482+0000","last_undegraded":"2023-10-16T11:22:43.132482+0000","last_fullsized":"2023-10-16T11:22:43.132482+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:48.644799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.144792+0000","last_change":"2023-10-16T11:22:43.141863+0000","last_active":"2023-10-16T11:22:44.144792+0000","last_peered":"2023-10-16T11:22:44.144792+0000","last_clean":"2023-10-16T11:22:44.144792+0000","last_became_active":"2023-10-16T11:22:43.141584+0000","last_became_peered":"2023-10-16T11:22:43.141584+0000","last_unstale":"2023-10-16T11:22:44.144792+0000","last_undegraded":"2023-10-16T11:22:44.144792+0000","last_fullsized":"2023-10-16T11:22:44.144792+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:39:10.429029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.114847+0000","last_change":"2023-10-16T11:22:42.138321+0000","last_active":"2023-10-16T11:22:43.114847+0000","last_peered":"2023-10-16T11:22:43.114847+0000","last_clean":"2023-10-16T11:22:43.114847+0000","last_became_active":"2023-10-16T11:22:42.137996+0000","last_became_peered":"2023-10-16T11:22:42.137996+0000","last_unstale":"2023-10-16T11:22:43.114847+0000","last_undegraded":"2023-10-16T11:22:43.114847+0000","last_fullsized":"2023-10-16T11:22:43.114847+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:01:45.670632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.779531+0000","last_change":"2023-10-16T11:22:43.162659+0000","last_active":"2023-10-16T11:22:44.779531+0000","last_peered":"2023-10-16T11:22:44.779531+0000","last_clean":"2023-10-16T11:22:44.779531+0000","last_became_active":"2023-10-16T11:22:43.162383+0000","last_became_peered":"2023-10-16T11:22:43.162383+0000","last_unstale":"2023-10-16T11:22:44.779531+0000","last_undegraded":"2023-10-16T11:22:44.779531+0000","last_fullsized":"2023-10-16T11:22:44.779531+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:01:46.689205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.124369+0000","last_change":"2023-10-16T11:22:42.126170+0000","last_active":"2023-10-16T11:22:43.124369+0000","last_peered":"2023-10-16T11:22:43.124369+0000","last_clean":"2023-10-16T11:22:43.124369+0000","last_became_active":"2023-10-16T11:22:42.124316+0000","last_became_peered":"2023-10-16T11:22:42.124316+0000","last_unstale":"2023-10-16T11:22:43.124369+0000","last_undegraded":"2023-10-16T11:22:43.124369+0000","last_fullsized":"2023-10-16T11:22:43.124369+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:39:09.410456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.139565+0000","last_change":"2023-10-16T11:22:43.128512+0000","last_active":"2023-10-16T11:22:44.139565+0000","last_peered":"2023-10-16T11:22:44.139565+0000","last_clean":"2023-10-16T11:22:44.139565+0000","last_became_active":"2023-10-16T11:22:43.127595+0000","last_became_peered":"2023-10-16T11:22:43.127595+0000","last_unstale":"2023-10-16T11:22:44.139565+0000","last_undegraded":"2023-10-16T11:22:44.139565+0000","last_fullsized":"2023-10-16T11:22:44.139565+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:17:16.487265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.139998+0000","last_change":"2023-10-16T11:22:42.140303+0000","last_active":"2023-10-16T11:22:43.139998+0000","last_peered":"2023-10-16T11:22:43.139998+0000","last_clean":"2023-10-16T11:22:43.139998+0000","last_became_active":"2023-10-16T11:22:42.139994+0000","last_became_peered":"2023-10-16T11:22:42.139994+0000","last_unstale":"2023-10-16T11:22:43.139998+0000","last_undegraded":"2023-10-16T11:22:43.139998+0000","last_fullsized":"2023-10-16T11:22:43.139998+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:48.644799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.139428+0000","last_change":"2023-10-16T11:22:43.129643+0000","last_active":"2023-10-16T11:22:44.139428+0000","last_peered":"2023-10-16T11:22:44.139428+0000","last_clean":"2023-10-16T11:22:44.139428+0000","last_became_active":"2023-10-16T11:22:43.129356+0000","last_became_peered":"2023-10-16T11:22:43.129356+0000","last_unstale":"2023-10-16T11:22:44.139428+0000","last_undegraded":"2023-10-16T11:22:44.139428+0000","last_fullsized":"2023-10-16T11:22:44.139428+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:00.954663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"200'2","reported_seq":14,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.144555+0000","last_change":"2023-10-16T11:22:42.137600+0000","last_active":"2023-10-16T11:22:43.144555+0000","last_peered":"2023-10-16T11:22:43.144555+0000","last_clean":"2023-10-16T11:22:43.144555+0000","last_became_active":"2023-10-16T11:22:42.137307+0000","last_became_peered":"2023-10-16T11:22:42.137307+0000","last_unstale":"2023-10-16T11:22:43.144555+0000","last_undegraded":"2023-10-16T11:22:43.144555+0000","last_fullsized":"2023-10-16T11:22:43.144555+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:41:44.300027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.778285+0000","last_change":"2023-10-16T11:22:43.147905+0000","last_active":"2023-10-16T11:22:44.778285+0000","last_peered":"2023-10-16T11:22:44.778285+0000","last_clean":"2023-10-16T11:22:44.778285+0000","last_became_active":"2023-10-16T11:22:43.147601+0000","last_became_peered":"2023-10-16T11:22:43.147601+0000","last_unstale":"2023-10-16T11:22:44.778285+0000","last_undegraded":"2023-10-16T11:22:44.778285+0000","last_fullsized":"2023-10-16T11:22:44.778285+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:00:27.289128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.119887+0000","last_change":"2023-10-16T11:22:42.132077+0000","last_active":"2023-10-16T11:22:43.119887+0000","last_peered":"2023-10-16T11:22:43.119887+0000","last_clean":"2023-10-16T11:22:43.119887+0000","last_became_active":"2023-10-16T11:22:42.131806+0000","last_became_peered":"2023-10-16T11:22:42.131806+0000","last_unstale":"2023-10-16T11:22:43.119887+0000","last_undegraded":"2023-10-16T11:22:43.119887+0000","last_fullsized":"2023-10-16T11:22:43.119887+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:35:09.498951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.143482+0000","last_change":"2023-10-16T11:22:43.140925+0000","last_active":"2023-10-16T11:22:44.143482+0000","last_peered":"2023-10-16T11:22:44.143482+0000","last_clean":"2023-10-16T11:22:44.143482+0000","last_became_active":"2023-10-16T11:22:43.140661+0000","last_became_peered":"2023-10-16T11:22:43.140661+0000","last_unstale":"2023-10-16T11:22:44.143482+0000","last_undegraded":"2023-10-16T11:22:44.143482+0000","last_fullsized":"2023-10-16T11:22:44.143482+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:53:10.906621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.120125+0000","last_change":"2023-10-16T11:22:42.131666+0000","last_active":"2023-10-16T11:22:43.120125+0000","last_peered":"2023-10-16T11:22:43.120125+0000","last_clean":"2023-10-16T11:22:43.120125+0000","last_became_active":"2023-10-16T11:22:42.131386+0000","last_became_peered":"2023-10-16T11:22:42.131386+0000","last_unstale":"2023-10-16T11:22:43.120125+0000","last_undegraded":"2023-10-16T11:22:43.120125+0000","last_fullsized":"2023-10-16T11:22:43.120125+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:31:53.479584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.144016+0000","last_change":"2023-10-16T11:22:43.154520+0000","last_active":"2023-10-16T11:22:44.144016+0000","last_peered":"2023-10-16T11:22:44.144016+0000","last_clean":"2023-10-16T11:22:44.144016+0000","last_became_active":"2023-10-16T11:22:43.154208+0000","last_became_peered":"2023-10-16T11:22:43.154208+0000","last_unstale":"2023-10-16T11:22:44.144016+0000","last_undegraded":"2023-10-16T11:22:44.144016+0000","last_fullsized":"2023-10-16T11:22:44.144016+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:45:22.939067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.133807+0000","last_change":"2023-10-16T11:22:42.143263+0000","last_active":"2023-10-16T11:22:43.133807+0000","last_peered":"2023-10-16T11:22:43.133807+0000","last_clean":"2023-10-16T11:22:43.133807+0000","last_became_active":"2023-10-16T11:22:42.142837+0000","last_became_peered":"2023-10-16T11:22:42.142837+0000","last_unstale":"2023-10-16T11:22:43.133807+0000","last_undegraded":"2023-10-16T11:22:43.133807+0000","last_fullsized":"2023-10-16T11:22:43.133807+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:14:31.933379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.781214+0000","last_change":"2023-10-16T11:22:43.148812+0000","last_active":"2023-10-16T11:22:44.781214+0000","last_peered":"2023-10-16T11:22:44.781214+0000","last_clean":"2023-10-16T11:22:44.781214+0000","last_became_active":"2023-10-16T11:22:43.148400+0000","last_became_peered":"2023-10-16T11:22:43.148400+0000","last_unstale":"2023-10-16T11:22:44.781214+0000","last_undegraded":"2023-10-16T11:22:44.781214+0000","last_fullsized":"2023-10-16T11:22:44.781214+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:12:15.344443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.152268+0000","last_change":"2023-10-16T11:22:42.135767+0000","last_active":"2023-10-16T11:22:43.152268+0000","last_peered":"2023-10-16T11:22:43.152268+0000","last_clean":"2023-10-16T11:22:43.152268+0000","last_became_active":"2023-10-16T11:22:42.135478+0000","last_became_peered":"2023-10-16T11:22:42.135478+0000","last_unstale":"2023-10-16T11:22:43.152268+0000","last_undegraded":"2023-10-16T11:22:43.152268+0000","last_fullsized":"2023-10-16T11:22:43.152268+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:50.483625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.781558+0000","last_change":"2023-10-16T11:22:43.157997+0000","last_active":"2023-10-16T11:22:44.781558+0000","last_peered":"2023-10-16T11:22:44.781558+0000","last_clean":"2023-10-16T11:22:44.781558+0000","last_became_active":"2023-10-16T11:22:43.152986+0000","last_became_peered":"2023-10-16T11:22:43.152986+0000","last_unstale":"2023-10-16T11:22:44.781558+0000","last_undegraded":"2023-10-16T11:22:44.781558+0000","last_fullsized":"2023-10-16T11:22:44.781558+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:20.897287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.151139+0000","last_change":"2023-10-16T11:22:42.144266+0000","last_active":"2023-10-16T11:22:43.151139+0000","last_peered":"2023-10-16T11:22:43.151139+0000","last_clean":"2023-10-16T11:22:43.151139+0000","last_became_active":"2023-10-16T11:22:42.142881+0000","last_became_peered":"2023-10-16T11:22:42.142881+0000","last_unstale":"2023-10-16T11:22:43.151139+0000","last_undegraded":"2023-10-16T11:22:43.151139+0000","last_fullsized":"2023-10-16T11:22:43.151139+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:17.013373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.777842+0000","last_change":"2023-10-16T11:22:43.151092+0000","last_active":"2023-10-16T11:22:44.777842+0000","last_peered":"2023-10-16T11:22:44.777842+0000","last_clean":"2023-10-16T11:22:44.777842+0000","last_became_active":"2023-10-16T11:22:43.150817+0000","last_became_peered":"2023-10-16T11:22:43.150817+0000","last_unstale":"2023-10-16T11:22:44.777842+0000","last_undegraded":"2023-10-16T11:22:44.777842+0000","last_fullsized":"2023-10-16T11:22:44.777842+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:10.267804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.145215+0000","last_change":"2023-10-16T11:22:42.141567+0000","last_active":"2023-10-16T11:22:43.145215+0000","last_peered":"2023-10-16T11:22:43.145215+0000","last_clean":"2023-10-16T11:22:43.145215+0000","last_became_active":"2023-10-16T11:22:42.141238+0000","last_became_peered":"2023-10-16T11:22:42.141238+0000","last_unstale":"2023-10-16T11:22:43.145215+0000","last_undegraded":"2023-10-16T11:22:43.145215+0000","last_fullsized":"2023-10-16T11:22:43.145215+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:55.513664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.139623+0000","last_change":"2023-10-16T11:22:43.157589+0000","last_active":"2023-10-16T11:22:44.139623+0000","last_peered":"2023-10-16T11:22:44.139623+0000","last_clean":"2023-10-16T11:22:44.139623+0000","last_became_active":"2023-10-16T11:22:43.157284+0000","last_became_peered":"2023-10-16T11:22:43.157284+0000","last_unstale":"2023-10-16T11:22:44.139623+0000","last_undegraded":"2023-10-16T11:22:44.139623+0000","last_fullsized":"2023-10-16T11:22:44.139623+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:57:39.650999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.121749+0000","last_change":"2023-10-16T11:22:42.119659+0000","last_active":"2023-10-16T11:22:43.121749+0000","last_peered":"2023-10-16T11:22:43.121749+0000","last_clean":"2023-10-16T11:22:43.121749+0000","last_became_active":"2023-10-16T11:22:42.119380+0000","last_became_peered":"2023-10-16T11:22:42.119380+0000","last_unstale":"2023-10-16T11:22:43.121749+0000","last_undegraded":"2023-10-16T11:22:43.121749+0000","last_fullsized":"2023-10-16T11:22:43.121749+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:29.005826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-16T11:22:44.777472+0000","last_change":"2023-10-16T11:22:43.158475+0000","last_active":"2023-10-16T11:22:44.777472+0000","last_peered":"2023-10-16T11:22:44.777472+0000","last_clean":"2023-10-16T11:22:44.777472+0000","last_became_active":"2023-10-16T11:22:43.157982+0000","last_became_peered":"2023-10-16T11:22:43.157982+0000","last_unstale":"2023-10-16T11:22:44.777472+0000","last_undegraded":"2023-10-16T11:22:44.777472+0000","last_fullsized":"2023-10-16T11:22:44.777472+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:42.087064+0000","last_clean_scrub_stamp":"2023-10-16T11:22:42.087064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:17.151421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.138199+0000","last_change":"2023-10-16T11:22:42.141735+0000","last_active":"2023-10-16T11:22:43.138199+0000","last_peered":"2023-10-16T11:22:43.138199+0000","last_clean":"2023-10-16T11:22:43.138199+0000","last_became_active":"2023-10-16T11:22:42.137927+0000","last_became_peered":"2023-10-16T11:22:42.137927+0000","last_unstale":"2023-10-16T11:22:43.138199+0000","last_undegraded":"2023-10-16T11:22:43.138199+0000","last_fullsized":"2023-10-16T11:22:43.138199+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:22:41.068490+0000","last_clean_scrub_stamp":"2023-10-16T11:22:41.068490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:45:21.920494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"189'2745","reported_seq":3558,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-16T11:22:43.134292+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:22:43.134292+0000","last_peered":"2023-10-16T11:22:43.134292+0000","last_clean":"2023-10-16T11:22:43.134292+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:22:43.134292+0000","last_undegraded":"2023-10-16T11:22:43.134292+0000","last_fullsized":"2023-10-16T11:22:43.134292+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2745,"log_dups_size":0,"ondisk_log_size":2745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1984,"num_read_kb":4056,"num_write":541,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1984,"num_read_kb":4056,"num_write":541,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2745,"ondisk_log_size":2745,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575966,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48116,"kb_used_data":3048,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323724,"statfs":{"total":96636764160,"available":96587493376,"internally_reserved":0,"allocated":3121152,"data_stored":1809715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2,"15min":1.234},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.73799999999999999},"max":{"1min":1.641,"5min":2.085,"15min":2.3399999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.206,"15min":1.23},"min":{"1min":0.80000000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5549999999999999,"5min":1.6850000000000001,"15min":1.8320000000000001},"last":1.151}]},{"osd":1,"last update":"Mon Oct 16 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.2250000000000001,"15min":1.2589999999999999},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.69699999999999995},"max":{"1min":1.7589999999999999,"5min":1.913,"15min":2.0129999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.2709999999999999,"15min":1.28},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":1.726,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.264}]},{"osd":2,"last update":"Mon Oct 16 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.2869999999999999,"15min":1.3129999999999999},"min":{"1min":0.96299999999999997,"5min":0.745,"15min":0.745},"max":{"1min":1.599,"5min":1.8680000000000001,"15min":2.4460000000000002},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.361,"15min":1.363},"min":{"1min":0.97099999999999997,"5min":0.93700000000000006,"15min":0.90400000000000003},"max":{"1min":1.6579999999999999,"5min":1.859,"15min":1.974},"last":1.3839999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.389,"15min":1.3919999999999999},"min":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":0.97799999999999998},"max":{"1min":1.5860000000000001,"5min":1.8280000000000001,"15min":1.972},"last":1.464},{"interface":"front","average":{"1min":1.423,"5min":1.4530000000000001,"15min":1.4690000000000001},"min":{"1min":1.1339999999999999,"5min":0.997,"15min":0.96299999999999997},"max":{"1min":1.796,"5min":2.0680000000000001,"15min":2.577},"last":1.0620000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.373,"15min":1.417},"min":{"1min":0.95599999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.8140000000000001,"5min":1.954,"15min":2.6779999999999999},"last":1.214},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.343,"15min":1.399},"min":{"1min":0.95899999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.573,"5min":2.109,"15min":2.109},"last":0.86799999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.5549999999999999,"15min":1.5960000000000001},"min":{"1min":1.236,"5min":1.089,"15min":1.012},"max":{"1min":1.8600000000000001,"5min":2.2309999999999999,"15min":3.3519999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.46,"15min":1.5029999999999999},"min":{"1min":1.069,"5min":1.028,"15min":0.74199999999999999},"max":{"1min":1.6970000000000001,"5min":2.2349999999999999,"15min":2.4969999999999999},"last":1.321}]},{"osd":6,"last update":"Mon Oct 16 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.466,"15min":1.5149999999999999},"min":{"1min":1.165,"5min":0.93500000000000005,"15min":0.90500000000000003},"max":{"1min":1.823,"5min":1.9319999999999999,"15min":2.7250000000000001},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.635,"5min":1.6699999999999999,"15min":1.714},"min":{"1min":1.284,"5min":1.105,"15min":1.105},"max":{"1min":1.9279999999999999,"5min":2.298,"15min":3.1440000000000001},"last":1.3009999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575966,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29480,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342360,"statfs":{"total":96636764160,"available":96606576640,"internally_reserved":0,"allocated":2519040,"data_stored":1219243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1990000000000001,"15min":1.22},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.64000000000000001},"max":{"1min":1.698,"5min":1.698,"15min":2.0419999999999998},"last":1.224},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2509999999999999,"15min":1.264},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.66400000000000003},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.6799999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:22:33 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.2330000000000001,"15min":1.2170000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.8420000000000001},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.367,"15min":1.3480000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.77,"5min":1.796,"15min":1.796},"last":1.331}]},{"osd":2,"last update":"Mon Oct 16 11:22:33 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2749999999999999,"15min":1.2470000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.68000000000000005},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.861},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3500000000000001,"15min":1.357},"min":{"1min":0.92700000000000005,"5min":0.85799999999999998,"15min":0.72699999999999998},"max":{"1min":1.625,"5min":1.7949999999999999,"15min":2.0310000000000001},"last":1.738}]},{"osd":3,"last update":"Mon Oct 16 11:22:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3069999999999999,"15min":1.298},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7370000000000001,"5min":1.8260000000000001,"15min":1.859},"last":1.841},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2669999999999999,"15min":1.2649999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.744,"5min":1.744,"15min":1.8839999999999999},"last":1.1020000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:22:33 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.246,"15min":1.2949999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.9099999999999999},"last":1.8999999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.512,"15min":1.4970000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.9019999999999999,"5min":1.929,"15min":1.9530000000000001},"last":2.161}]},{"osd":5,"last update":"Mon Oct 16 11:22:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.3360000000000001,"15min":1.3340000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.5680000000000001,"5min":1.925,"15min":2.1669999999999998},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.518,"15min":1.5429999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.8520000000000001,"5min":1.978,"15min":2.1280000000000001},"last":2.1059999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:22:33 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4830000000000001,"15min":1.4930000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.766,"5min":1.915,"15min":2.149},"last":1.994},{"interface":"front","average":{"1min":1.502,"5min":1.5369999999999999,"15min":1.546},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.8420000000000001,"5min":2.0230000000000001,"15min":2.3530000000000002},"last":1.778}]}]},{"osd":5,"up_from":13,"seq":55834575966,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29484,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342356,"statfs":{"total":96636764160,"available":96606572544,"internally_reserved":0,"allocated":2519040,"data_stored":1219243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.2609999999999999,"15min":1.2809999999999999},"min":{"1min":0.876,"5min":0.755,"15min":0.70799999999999996},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3460000000000001,"15min":1.3620000000000001},"min":{"1min":1.036,"5min":0.88500000000000001,"15min":0.83899999999999997},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.887},"last":1.704}]},{"osd":1,"last update":"Mon Oct 16 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.2070000000000001,"15min":1.2170000000000001},"min":{"1min":0.90100000000000002,"5min":0.80800000000000005,"15min":0.76100000000000001},"max":{"1min":1.704,"5min":1.7869999999999999,"15min":1.992},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.2569999999999999,"15min":1.2310000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.80800000000000005},"max":{"1min":2.9399999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.081}]},{"osd":2,"last update":"Mon Oct 16 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.25,"15min":1.242},"min":{"1min":0.876,"5min":0.876,"15min":0.83799999999999997},"max":{"1min":1.952,"5min":1.952,"15min":2.052},"last":1.8600000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.2689999999999999,"15min":1.323},"min":{"1min":0.98399999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.5760000000000001,"5min":1.766,"15min":2.1640000000000001},"last":1.2130000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.335,"15min":1.333},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":3.012,"5min":3.012,"15min":3.012},"last":1.397},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.3979999999999999,"15min":1.4299999999999999},"min":{"1min":1.1100000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.7190000000000001,"5min":1.7669999999999999,"15min":2.2250000000000001},"last":1.4690000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.304,"15min":1.353},"min":{"1min":0.84199999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.349},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.556,"15min":1.573},"min":{"1min":1.145,"5min":1.0660000000000001,"15min":0.94999999999999996},"max":{"1min":2.903,"5min":2.903,"15min":2.903},"last":1.7669999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.407,"15min":1.4199999999999999},"min":{"1min":1.0489999999999999,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8100000000000001,"5min":2.0649999999999999,"15min":2.492},"last":1.323},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.466,"15min":1.4890000000000001},"min":{"1min":1.2050000000000001,"5min":0.89200000000000002,"15min":0.78400000000000003},"max":{"1min":3.073,"5min":3.073,"15min":3.073},"last":1.5189999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.5289999999999999,"15min":1.5469999999999999},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90600000000000003},"max":{"1min":3.1160000000000001,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.613},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.6180000000000001,"15min":1.657},"min":{"1min":1.454,"5min":1.1859999999999999,"15min":0.88100000000000001},"max":{"1min":2.0750000000000002,"5min":2.1459999999999999,"15min":2.7290000000000001},"last":1.573}]}]},{"osd":4,"up_from":13,"seq":55834575967,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51712,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320128,"statfs":{"total":96636764160,"available":96583811072,"internally_reserved":0,"allocated":2535424,"data_stored":1221437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2,"15min":1.1859999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.125},{"interface":"front","average":{"1min":1.202,"5min":1.24,"15min":1.2490000000000001},"min":{"1min":0.873,"5min":0.75,"15min":0.75},"max":{"1min":1.754,"5min":1.7609999999999999,"15min":1.9299999999999999},"last":1.0449999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.272,"15min":1.3029999999999999},"min":{"1min":0.89300000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.708,"5min":1.7569999999999999,"15min":1.913},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.238,"5min":1.234,"15min":1.252},"min":{"1min":0.95599999999999996,"5min":0.85299999999999998,"15min":0.84399999999999997},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.889},"last":0.92900000000000005}]},{"osd":2,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3100000000000001,"15min":1.2949999999999999},"min":{"1min":0.90200000000000002,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.724,"5min":1.7370000000000001,"15min":1.835},"last":1.619},{"interface":"front","average":{"1min":1.407,"5min":1.409,"15min":1.3979999999999999},"min":{"1min":1.008,"5min":0.872,"15min":0.77600000000000002},"max":{"1min":1.861,"5min":1.8779999999999999,"15min":1.9139999999999999},"last":1.194}]},{"osd":3,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.97299999999999998,"5min":0.94499999999999995,"15min":0.86599999999999999},"max":{"1min":1.613,"5min":1.849,"15min":1.849},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.4650000000000001,"15min":1.4279999999999999},"min":{"1min":1.2470000000000001,"5min":1.042,"15min":0.90000000000000002},"max":{"1min":1.8360000000000001,"5min":1.863,"15min":2.0720000000000001},"last":1.008}]},{"osd":5,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.46,"15min":1.431},"min":{"1min":1.0049999999999999,"5min":0.98699999999999999,"15min":0.88200000000000001},"max":{"1min":1.9530000000000001,"5min":2.0790000000000002,"15min":2.0910000000000002},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.425,"15min":1.4219999999999999},"min":{"1min":1.2330000000000001,"5min":0.84499999999999997,"15min":0.78600000000000003},"max":{"1min":1.804,"5min":1.9139999999999999,"15min":2.0059999999999998},"last":1.5049999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":1.54,"15min":1.486},"min":{"1min":1.1830000000000001,"5min":0.97799999999999998,"15min":0.78000000000000003},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":2.2309999999999999},"last":1.722},{"interface":"front","average":{"1min":1.653,"5min":1.607,"15min":1.613},"min":{"1min":1.3520000000000001,"5min":1.0920000000000001,"15min":1.038},"max":{"1min":2.02,"5min":2.0390000000000001,"15min":2.2770000000000001},"last":1.597}]},{"osd":7,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6240000000000001,"5min":1.5669999999999999,"15min":1.5660000000000001},"min":{"1min":1.302,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.8340000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.403},{"interface":"front","average":{"1min":1.74,"5min":1.6759999999999999,"15min":1.6319999999999999},"min":{"1min":1.4970000000000001,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":2.1469999999999998,"5min":2.1509999999999998,"15min":2.3839999999999999},"last":1.845}]}]},{"osd":3,"up_from":13,"seq":55834575966,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47544,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324296,"statfs":{"total":96636764160,"available":96588079104,"internally_reserved":0,"allocated":2531328,"data_stored":1219437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.92800000000000005,"15min":0.91200000000000003},"min":{"1min":0.67800000000000005,"5min":0.64500000000000002,"15min":0.54400000000000004},"max":{"1min":1.3839999999999999,"5min":1.4930000000000001,"15min":1.901},"last":0.755},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.087,"15min":1.0840000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.56999999999999995},"max":{"1min":1.575,"5min":1.6180000000000001,"15min":1.6679999999999999},"last":0.68799999999999994}]},{"osd":1,"last update":"Mon Oct 16 11:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.96999999999999997,"15min":0.98999999999999999},"min":{"1min":0.76000000000000001,"5min":0.626,"15min":0.60299999999999998},"max":{"1min":1.361,"5min":1.4219999999999999,"15min":1.4510000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1499999999999999,"15min":1.1539999999999999},"min":{"1min":0.84699999999999998,"5min":0.68500000000000005,"15min":0.67600000000000005},"max":{"1min":1.5640000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.59299999999999997}]},{"osd":2,"last update":"Mon Oct 16 11:22:02 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.123,"15min":1.125},"min":{"1min":0.82199999999999995,"5min":0.79700000000000004,"15min":0.79200000000000004},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.238,"5min":1.2370000000000001,"15min":1.2430000000000001},"min":{"1min":0.878,"5min":0.87,"15min":0.67400000000000004},"max":{"1min":1.5209999999999999,"5min":1.677,"15min":1.843},"last":0.78500000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:22:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.371,"15min":1.3640000000000001},"min":{"1min":1.01,"5min":1.01,"15min":0.95999999999999996},"max":{"1min":1.5660000000000001,"5min":1.7330000000000001,"15min":1.873},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3640000000000001,"15min":1.369},"min":{"1min":1.04,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":1.6060000000000001,"5min":1.8240000000000001,"15min":1.8819999999999999},"last":1.1819999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:22:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4219999999999999,"15min":1.411},"min":{"1min":1.0960000000000001,"5min":1.0940000000000001,"15min":0.96199999999999997},"max":{"1min":1.9219999999999999,"5min":2.1509999999999998,"15min":2.2410000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.462,"15min":1.45},"min":{"1min":1.131,"5min":1.004,"15min":1.004},"max":{"1min":2.0739999999999998,"5min":2.1150000000000002,"15min":2.1579999999999999},"last":1.3380000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:22:02 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.538,"15min":1.546},"min":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"max":{"1min":1.8120000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5329999999999999,"15min":1.5429999999999999},"min":{"1min":1.2410000000000001,"5min":1.1060000000000001,"15min":1.083},"max":{"1min":2.0409999999999999,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.28}]},{"osd":7,"last update":"Mon Oct 16 11:22:02 2023","interfaces":[{"interface":"back","average":{"1min":1.605,"5min":1.548,"15min":1.5720000000000001},"min":{"1min":1.325,"5min":1.0609999999999999,"15min":1.0429999999999999},"max":{"1min":2.0790000000000002,"5min":2.1960000000000002,"15min":2.2679999999999998},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.7190000000000001,"5min":1.6879999999999999,"15min":1.6919999999999999},"min":{"1min":1.323,"5min":1.323,"15min":1.1479999999999999},"max":{"1min":2.1549999999999998,"5min":2.2549999999999999,"15min":2.5129999999999999},"last":1.569}]}]},{"osd":2,"up_from":13,"seq":55834575965,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51708,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320132,"statfs":{"total":96636764160,"available":96583815168,"internally_reserved":0,"allocated":2531328,"data_stored":1219937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:05 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.90300000000000002,"15min":0.91700000000000004},"min":{"1min":0.629,"5min":0.5,"15min":0.5},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.669},"last":1.079},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0920000000000001,"15min":1.091},"min":{"1min":0.72799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.51,"5min":1.8180000000000001,"15min":2.1429999999999998},"last":0.76100000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:22:05 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.085,"15min":1.1020000000000001},"min":{"1min":0.754,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.645,"5min":1.6779999999999999,"15min":1.978},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.163,"15min":1.181},"min":{"1min":0.87,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.609,"5min":1.6819999999999999,"15min":1.869},"last":0.96399999999999997}]},{"osd":3,"last update":"Mon Oct 16 11:22:05 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.0680000000000001,"15min":1.0960000000000001},"min":{"1min":0.75900000000000001,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.651,"5min":1.651,"15min":1.8560000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2709999999999999,"15min":1.2849999999999999},"min":{"1min":0.91300000000000003,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.617,"5min":1.6719999999999999,"15min":1.8740000000000001},"last":0.90900000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:22:05 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.3979999999999999,"15min":1.4359999999999999},"min":{"1min":1.095,"5min":0.96899999999999997,"15min":0.73499999999999999},"max":{"1min":1.77,"5min":2.2770000000000001,"15min":2.5059999999999998},"last":1.028},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3300000000000001,"15min":1.347},"min":{"1min":1.002,"5min":0.748,"15min":0.748},"max":{"1min":1.6259999999999999,"5min":1.7749999999999999,"15min":2.0880000000000001},"last":1.5369999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:22:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.454,"15min":1.462},"min":{"1min":1.141,"5min":1.0860000000000001,"15min":0.78500000000000003},"max":{"1min":1.992,"5min":2.0099999999999998,"15min":2.2269999999999999},"last":1.569},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.464,"15min":1.504},"min":{"1min":1.1439999999999999,"5min":0.874,"15min":0.874},"max":{"1min":1.8939999999999999,"5min":2.1349999999999998,"15min":2.6259999999999999},"last":1.165}]},{"osd":6,"last update":"Mon Oct 16 11:22:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.377,"15min":1.4099999999999999},"min":{"1min":1.1699999999999999,"5min":0.84199999999999997,"15min":0.80500000000000005},"max":{"1min":1.8620000000000001,"5min":1.8660000000000001,"15min":2.2109999999999999},"last":1.7010000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5209999999999999,"15min":1.5389999999999999},"min":{"1min":1.052,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.0600000000000001,"5min":2.089,"15min":2.2029999999999998},"last":1.3260000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:22:05 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.5580000000000001,"15min":1.5960000000000001},"min":{"1min":1.2490000000000001,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":1.9470000000000001,"5min":2.4670000000000001,"15min":2.8109999999999999},"last":1.464},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.4710000000000001,"15min":1.5229999999999999},"min":{"1min":1.1040000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2829999999999999},"last":1.6120000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575965,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60024,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311816,"statfs":{"total":96636764160,"available":96575299584,"internally_reserved":0,"allocated":2531328,"data_stored":1220855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.91800000000000004,"15min":0.91500000000000004},"min":{"1min":0.71099999999999997,"5min":0.66700000000000004,"15min":0.39100000000000001},"max":{"1min":1.494,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.056,"15min":1.0609999999999999},"min":{"1min":0.80700000000000005,"5min":0.621,"15min":0.54500000000000004},"max":{"1min":1.331,"5min":1.633,"15min":1.667},"last":0.746}]},{"osd":2,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.99299999999999999,"15min":1.024},"min":{"1min":0.75800000000000001,"5min":0.72999999999999998,"15min":0.42699999999999999},"max":{"1min":1.2290000000000001,"5min":1.4279999999999999,"15min":1.972},"last":1.421},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.278,"15min":1.2869999999999999},"min":{"1min":0.94299999999999995,"5min":0.78600000000000003,"15min":0.55700000000000005},"max":{"1min":1.556,"5min":1.6910000000000001,"15min":2.161},"last":1.2729999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.1759999999999999,"15min":1.21},"min":{"1min":0.91900000000000004,"5min":0.78300000000000003,"15min":0.49399999999999999},"max":{"1min":1.538,"5min":1.7729999999999999,"15min":1.79},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.1839999999999999,"15min":1.1779999999999999},"min":{"1min":0.90000000000000002,"5min":0.86899999999999999,"15min":0.51300000000000001},"max":{"1min":1.5009999999999999,"5min":1.609,"15min":1.802},"last":1.204}]},{"osd":4,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.292,"15min":1.28},"min":{"1min":1.0029999999999999,"5min":0.90700000000000003,"15min":0.80700000000000005},"max":{"1min":1.7290000000000001,"5min":1.887,"15min":1.9159999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.401,"15min":1.4139999999999999},"min":{"1min":1.139,"5min":1.0549999999999999,"15min":0.64700000000000002},"max":{"1min":1.825,"5min":2.0619999999999998,"15min":2.093},"last":1.3440000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.4219999999999999,"15min":1.425},"min":{"1min":1.103,"5min":0.999,"15min":0.68400000000000005},"max":{"1min":1.7370000000000001,"5min":1.948,"15min":2.0899999999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4059999999999999,"15min":1.4119999999999999},"min":{"1min":0.996,"5min":0.996,"15min":0.82599999999999996},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":2.0649999999999999},"last":1.3069999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.498,"15min":1.4970000000000001},"min":{"1min":1.222,"5min":1.1519999999999999,"15min":0.85999999999999999},"max":{"1min":2.1179999999999999,"5min":2.1970000000000001,"15min":2.2949999999999999},"last":1.51},{"interface":"front","average":{"1min":1.52,"5min":1.498,"15min":1.49},"min":{"1min":1.125,"5min":1.125,"15min":0.97299999999999998},"max":{"1min":1.8740000000000001,"5min":1.9079999999999999,"15min":2.1970000000000001},"last":1.8140000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.5680000000000001,"15min":1.5629999999999999},"min":{"1min":1.3440000000000001,"5min":1.1819999999999999,"15min":0.85699999999999998},"max":{"1min":1.8959999999999999,"5min":2.077,"15min":2.1720000000000002},"last":1.552},{"interface":"front","average":{"1min":1.681,"5min":1.6240000000000001,"15min":1.6100000000000001},"min":{"1min":1.226,"5min":1.103,"15min":0.90800000000000003},"max":{"1min":2.1699999999999999,"5min":2.1989999999999998,"15min":2.2810000000000001},"last":1.621}]}]},{"osd":0,"up_from":13,"seq":55834575966,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52276,"kb_used_data":3044,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319564,"statfs":{"total":96636764160,"available":96583233536,"internally_reserved":0,"allocated":3117056,"data_stored":1809633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89100000000000001,"15min":0.89500000000000002},"min":{"1min":0.68500000000000005,"5min":0.63,"15min":0.41599999999999998},"max":{"1min":1.52,"5min":1.546,"15min":1.619},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.056,"5min":1.103,"15min":1.1060000000000001},"min":{"1min":0.69099999999999995,"5min":0.67700000000000005,"15min":0.46300000000000002},"max":{"1min":1.4950000000000001,"5min":1.681,"15min":1.76},"last":0.96799999999999997}]},{"osd":2,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.097,"15min":1.079},"min":{"1min":0.80000000000000004,"5min":0.66300000000000003,"15min":0.51100000000000001},"max":{"1min":1.468,"5min":1.649,"15min":1.649},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1379999999999999,"15min":1.1559999999999999},"min":{"1min":0.81699999999999995,"5min":0.76500000000000001,"15min":0.53100000000000003},"max":{"1min":1.6220000000000001,"5min":1.623,"15min":1.9319999999999999},"last":0.92600000000000005}]},{"osd":3,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.113,"15min":1.087},"min":{"1min":0.81799999999999995,"5min":0.78800000000000003,"15min":0.42599999999999999},"max":{"1min":1.5129999999999999,"5min":1.8100000000000001,"15min":1.8839999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.24,"5min":1.2789999999999999,"15min":1.2609999999999999},"min":{"1min":0.91100000000000003,"5min":0.90600000000000003,"15min":0.64200000000000002},"max":{"1min":1.585,"5min":1.7609999999999999,"15min":1.9830000000000001},"last":0.86099999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.4219999999999999,"15min":1.3899999999999999},"min":{"1min":1.1160000000000001,"5min":1.0860000000000001,"15min":0.78700000000000003},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.95},"last":1.105},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3500000000000001,"15min":1.3580000000000001},"min":{"1min":1.0469999999999999,"5min":1,"15min":0.88},"max":{"1min":1.6619999999999999,"5min":1.9019999999999999,"15min":2.4820000000000002},"last":2.1539999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.369,"15min":1.3879999999999999},"min":{"1min":1.23,"5min":1.046,"15min":0.76600000000000001},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.6669999999999998},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.462,"15min":1.462},"min":{"1min":1.149,"5min":1.0860000000000001,"15min":0.752},"max":{"1min":1.748,"5min":1.9350000000000001,"15min":2.2040000000000002},"last":1.1799999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.516,"15min":1.514},"min":{"1min":1.202,"5min":1.1080000000000001,"15min":1.0329999999999999},"max":{"1min":1.8100000000000001,"5min":1.984,"15min":2.226},"last":1.278},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.508,"15min":1.5},"min":{"1min":1.099,"5min":1.099,"15min":0.79200000000000004},"max":{"1min":1.9750000000000001,"5min":2.262,"15min":2.3969999999999998},"last":1.1699999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5309999999999999,"5min":1.5629999999999999,"15min":1.5609999999999999},"min":{"1min":1.149,"5min":1.149,"15min":0.88100000000000001},"max":{"1min":1.929,"5min":2.347,"15min":2.347},"last":1.224},{"interface":"front","average":{"1min":1.552,"5min":1.528,"15min":1.5229999999999999},"min":{"1min":1.159,"5min":1.008,"15min":1.008},"max":{"1min":1.919,"5min":2.2919999999999998,"15min":2.8199999999999998},"last":1.135}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:22:48.029 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T11:22:48.030 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T11:22:48.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.291+0000 7f582689b640 1 Processor -- start 2023-10-16T11:22:48.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.291+0000 7f582689b640 1 -- start start 2023-10-16T11:22:48.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.293+0000 7f582689b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5820070fd0 0x7f58200cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:48.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.293+0000 7f582689b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f58200adfc0 0x7f58200b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:48.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.293+0000 7f582689b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58200cc600 con 0x7f58200adbe0 2023-10-16T11:22:48.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.293+0000 7f582689b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5820071470 con 0x7f58200adfc0 2023-10-16T11:22:48.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.293+0000 7f582689b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58200cc870 con 0x7f5820070fd0 2023-10-16T11:22:48.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.293+0000 7f581ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f58200adbe0 0x7f582006f550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54932/0 (socket says 172.21.15.84:54932) 2023-10-16T11:22:48.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.293+0000 7f581ffff640 1 -- 172.21.15.84:0/3444600523 learned_addr learned my addr 172.21.15.84:0/3444600523 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:48.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.293+0000 7f5824e38640 1 --2- 172.21.15.84:0/3444600523 >> v2:172.21.15.84:3300/0 conn(0x7f58200adfc0 0x7f58200b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:48.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.294+0000 7f5817fff640 1 --2- 172.21.15.84:0/3444600523 >> v2:172.21.15.157:3300/0 conn(0x7f5820070fd0 0x7f58200cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:48.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.294+0000 7f581f7fe640 1 -- 172.21.15.84:0/3444600523 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3419531860 0 0) 0x7f58200cc600 con 0x7f58200adbe0 2023-10-16T11:22:48.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.294+0000 7f581f7fe640 1 -- 172.21.15.84:0/3444600523 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f580c003520 con 0x7f58200adbe0 2023-10-16T11:22:48.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.294+0000 7f5824e38640 1 -- 172.21.15.84:0/3444600523 >> v2:172.21.15.157:3300/0 conn(0x7f5820070fd0 msgr2=0x7f58200cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:48.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.294+0000 7f5824e38640 1 --2- 172.21.15.84:0/3444600523 >> v2:172.21.15.157:3300/0 conn(0x7f5820070fd0 0x7f58200cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.294+0000 7f5824e38640 1 -- 172.21.15.84:0/3444600523 >> v1:172.21.15.84:6789/0 conn(0x7f58200adbe0 legacy=0x7f582006f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:48.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.294+0000 7f5824e38640 1 -- 172.21.15.84:0/3444600523 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58200ccb50 con 0x7f58200adfc0 2023-10-16T11:22:48.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.295+0000 7f5824e38640 1 --2- 172.21.15.84:0/3444600523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58200adfc0 0x7f58200b0410 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f58200cc2b0 tx=0x7f581802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac1a4af1341f457 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:48.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.295+0000 7f581f7fe640 1 -- 172.21.15.84:0/3444600523 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581802cc70 con 0x7f58200adfc0 2023-10-16T11:22:48.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.295+0000 7f581f7fe640 1 -- 172.21.15.84:0/3444600523 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581802cdd0 con 0x7f58200adfc0 2023-10-16T11:22:48.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.295+0000 7f581f7fe640 1 -- 172.21.15.84:0/3444600523 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818034660 con 0x7f58200adfc0 2023-10-16T11:22:48.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.296+0000 7f582689b640 1 -- 172.21.15.84:0/3444600523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58200adfc0 msgr2=0x7f58200b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:48.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.296+0000 7f582689b640 1 --2- 172.21.15.84:0/3444600523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58200adfc0 0x7f58200b0410 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f58200cc2b0 tx=0x7f581802bec0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.296+0000 7f582689b640 1 -- 172.21.15.84:0/3444600523 shutdown_connections 2023-10-16T11:22:48.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.296+0000 7f582689b640 1 --2- 172.21.15.84:0/3444600523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58200adfc0 0x7f58200b0410 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.296+0000 7f582689b640 1 --2- 172.21.15.84:0/3444600523 >> v2:172.21.15.157:3300/0 conn(0x7f5820070fd0 0x7f58200cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.296+0000 7f582689b640 1 -- 172.21.15.84:0/3444600523 >> 172.21.15.84:0/3444600523 conn(0x7f58200cb410 msgr2=0x7f58200cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:48.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.296+0000 7f582689b640 1 -- 172.21.15.84:0/3444600523 shutdown_connections 2023-10-16T11:22:48.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.296+0000 7f582689b640 1 -- 172.21.15.84:0/3444600523 wait complete. 2023-10-16T11:22:48.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.298+0000 7f582689b640 1 Processor -- start 2023-10-16T11:22:48.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.298+0000 7f582689b640 1 -- start start 2023-10-16T11:22:48.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.299+0000 7f582689b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5820070fd0 0x7f58200b3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:48.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.299+0000 7f582689b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200adbe0 0x7f58200b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:48.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.299+0000 7f582689b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58200adfc0 0x7f58201f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:48.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.299+0000 7f582689b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58200b2c50 con 0x7f58200adfc0 2023-10-16T11:22:48.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.299+0000 7f582689b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58200b1fd0 con 0x7f58200adbe0 2023-10-16T11:22:48.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.299+0000 7f582689b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58200b22b0 con 0x7f5820070fd0 2023-10-16T11:22:48.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.299+0000 7f5824e38640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58200adfc0 0x7f58201f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:48.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.299+0000 7f5824e38640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58200adfc0 0x7f58201f8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38506/0 (socket says 172.21.15.84:38506) 2023-10-16T11:22:48.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.299+0000 7f5824e38640 1 -- 172.21.15.84:0/1986228700 learned_addr learned my addr 172.21.15.84:0/1986228700 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:48.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.299+0000 7f581ffff640 1 --2- 172.21.15.84:0/1986228700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5820070fd0 0x7f58200b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:48.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.299+0000 7f5817fff640 1 --2- 172.21.15.84:0/1986228700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200adbe0 0x7f58200b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:48.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.299+0000 7f5824e38640 1 -- 172.21.15.84:0/1986228700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5820070fd0 msgr2=0x7f58200b3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:48.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.299+0000 7f5824e38640 1 --2- 172.21.15.84:0/1986228700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5820070fd0 0x7f58200b3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.299+0000 7f5824e38640 1 -- 172.21.15.84:0/1986228700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200adbe0 msgr2=0x7f58200b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:48.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.299+0000 7f5824e38640 1 --2- 172.21.15.84:0/1986228700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200adbe0 0x7f58200b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.299+0000 7f5824e38640 1 -- 172.21.15.84:0/1986228700 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5818009a70 con 0x7f58200adfc0 2023-10-16T11:22:48.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.299+0000 7f581ffff640 1 --2- 172.21.15.84:0/1986228700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5820070fd0 0x7f58200b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:48.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.299+0000 7f5817fff640 1 --2- 172.21.15.84:0/1986228700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200adbe0 0x7f58200b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:48.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.300+0000 7f5824e38640 1 --2- 172.21.15.84:0/1986228700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58200adfc0 0x7f58201f8000 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f5818009e80 tx=0x7f5818004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:48.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.300+0000 7f581d7fa640 1 -- 172.21.15.84:0/1986228700 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818040650 con 0x7f58200adfc0 2023-10-16T11:22:48.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.300+0000 7f581d7fa640 1 -- 172.21.15.84:0/1986228700 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58180407b0 con 0x7f58200adfc0 2023-10-16T11:22:48.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.300+0000 7f581d7fa640 1 -- 172.21.15.84:0/1986228700 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818040b60 con 0x7f58200adfc0 2023-10-16T11:22:48.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.300+0000 7f582689b640 1 -- 172.21.15.84:0/1986228700 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58200b2590 con 0x7f58200adfc0 2023-10-16T11:22:48.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.300+0000 7f582689b640 1 -- 172.21.15.84:0/1986228700 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58200b1ba0 con 0x7f58200adfc0 2023-10-16T11:22:48.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.301+0000 7f582689b640 1 -- 172.21.15.84:0/1986228700 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58200b2d90 con 0x7f58200adfc0 2023-10-16T11:22:48.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.302+0000 7f581d7fa640 1 -- 172.21.15.84:0/1986228700 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5818040cc0 con 0x7f58200adfc0 2023-10-16T11:22:48.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.303+0000 7f581d7fa640 1 --2- 172.21.15.84:0/1986228700 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f580006af90 0x7f580006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:48.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.303+0000 7f581d7fa640 1 -- 172.21.15.84:0/1986228700 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f58180ac4b0 con 0x7f58200adfc0 2023-10-16T11:22:48.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.303+0000 7f581ffff640 1 --2- 172.21.15.84:0/1986228700 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f580006af90 0x7f580006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:48.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.304+0000 7f581ffff640 1 --2- 172.21.15.84:0/1986228700 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f580006af90 0x7f580006d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f58100011e0 tx=0x7f5810009040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:48.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.308+0000 7f581d7fa640 1 -- 172.21.15.84:0/1986228700 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f58200b2d90 con 0x7f58200adfc0 2023-10-16T11:22:48.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.568+0000 7f582689b640 1 -- 172.21.15.84:0/1986228700 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f58200b2d90 con 0x7f58200adfc0 2023-10-16T11:22:48.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.573+0000 7f581d7fa640 1 -- 172.21.15.84:0/1986228700 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v54) v1 ==== 190+0+147 (secure 0 0 0) 0x7f58200b2d90 con 0x7f58200adfc0 2023-10-16T11:22:48.575 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2023-10-16T11:22:48.575 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ== 2023-10-16T11:22:48.576 INFO:teuthology.orchestra.run.smithi084.stdout: caps mds = "allow" 2023-10-16T11:22:48.576 INFO:teuthology.orchestra.run.smithi084.stdout: caps mon = "allow r" 2023-10-16T11:22:48.576 INFO:teuthology.orchestra.run.smithi084.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T11:22:48.576 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-16T11:22:48.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.575+0000 7f582689b640 1 -- 172.21.15.84:0/1986228700 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f580006af90 msgr2=0x7f580006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:48.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.575+0000 7f582689b640 1 --2- 172.21.15.84:0/1986228700 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f580006af90 0x7f580006d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f58100011e0 tx=0x7f5810009040 comp rx=0 tx=0).stop 2023-10-16T11:22:48.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.576+0000 7f582689b640 1 -- 172.21.15.84:0/1986228700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58200adfc0 msgr2=0x7f58201f8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:48.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.576+0000 7f582689b640 1 --2- 172.21.15.84:0/1986228700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58200adfc0 0x7f58201f8000 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f5818009e80 tx=0x7f5818004a40 comp rx=0 tx=0).stop 2023-10-16T11:22:48.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.576+0000 7f582689b640 1 -- 172.21.15.84:0/1986228700 shutdown_connections 2023-10-16T11:22:48.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.576+0000 7f582689b640 1 --2- 172.21.15.84:0/1986228700 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f580006af90 0x7f580006d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.576+0000 7f582689b640 1 --2- 172.21.15.84:0/1986228700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58200adfc0 0x7f58201f8000 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.576+0000 7f582689b640 1 --2- 172.21.15.84:0/1986228700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58200adbe0 0x7f58200b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.576+0000 7f582689b640 1 --2- 172.21.15.84:0/1986228700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5820070fd0 0x7f58200b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.576+0000 7f582689b640 1 -- 172.21.15.84:0/1986228700 >> 172.21.15.84:0/1986228700 conn(0x7f58200cb410 msgr2=0x7f582015f900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:48.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.576+0000 7f582689b640 1 -- 172.21.15.84:0/1986228700 shutdown_connections 2023-10-16T11:22:48.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.576+0000 7f582689b640 1 -- 172.21.15.84:0/1986228700 wait complete. 2023-10-16T11:22:48.597 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:22:48.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.806+0000 7fe9e51b0640 1 Processor -- start 2023-10-16T11:22:48.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.806+0000 7fe9e51b0640 1 -- start start 2023-10-16T11:22:48.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.808+0000 7fe9e51b0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe9e00cd930 0x7fe9e0164240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:48.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.808+0000 7fe9e51b0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e0159790 con 0x7fe9e01593b0 2023-10-16T11:22:48.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.808+0000 7fe9e51b0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e00c9b20 con 0x7fe9e00caa10 2023-10-16T11:22:48.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.808+0000 7fe9e51b0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e0070370 con 0x7fe9e00cd930 2023-10-16T11:22:48.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.808+0000 7fe9ded9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe9e01593b0 0x7fe9e006f550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54940/0 (socket says 172.21.15.84:54940) 2023-10-16T11:22:48.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.808+0000 7fe9ded9d640 1 -- 172.21.15.84:0/3841897463 learned_addr learned my addr 172.21.15.84:0/3841897463 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:48.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.808+0000 7fe9df59e640 1 --2- 172.21.15.84:0/3841897463 >> v2:172.21.15.84:3300/0 conn(0x7fe9e00cd930 0x7fe9e0164240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:48.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.809+0000 7fe9ddd9b640 1 -- 172.21.15.84:0/3841897463 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988791136 0 0) 0x7fe9e0159790 con 0x7fe9e01593b0 2023-10-16T11:22:48.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.809+0000 7fe9ddd9b640 1 -- 172.21.15.84:0/3841897463 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9cc003520 con 0x7fe9e01593b0 2023-10-16T11:22:48.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.809+0000 7fe9df59e640 1 -- 172.21.15.84:0/3841897463 >> v1:172.21.15.157:6789/0 conn(0x7fe9e00caa10 legacy=0x7fe9e006fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:48.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.809+0000 7fe9df59e640 1 -- 172.21.15.84:0/3841897463 >> v1:172.21.15.84:6789/0 conn(0x7fe9e01593b0 legacy=0x7fe9e006f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:48.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.809+0000 7fe9ddd9b640 1 -- 172.21.15.84:0/3841897463 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4155586184 0 0) 0x7fe9cc003520 con 0x7fe9e01593b0 2023-10-16T11:22:48.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.809+0000 7fe9df59e640 1 -- 172.21.15.84:0/3841897463 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e00c9e00 con 0x7fe9e00cd930 2023-10-16T11:22:48.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.810+0000 7fe9df59e640 1 --2- 172.21.15.84:0/3841897463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e00cd930 0x7fe9e0164240 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800a940 tx=0x7fe9d802cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefb110d0e4f556f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:48.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.811+0000 7fe9ddd9b640 1 -- 172.21.15.84:0/3841897463 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d00019f0 con 0x7fe9e00cd930 2023-10-16T11:22:48.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.811+0000 7fe9ddd9b640 1 -- 172.21.15.84:0/3841897463 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d802e720 con 0x7fe9e00cd930 2023-10-16T11:22:48.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.811+0000 7fe9ddd9b640 1 -- 172.21.15.84:0/3841897463 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d802eab0 con 0x7fe9e00cd930 2023-10-16T11:22:48.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.811+0000 7fe9e51b0640 1 -- 172.21.15.84:0/3841897463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e00cd930 msgr2=0x7fe9e0164240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:48.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.811+0000 7fe9e51b0640 1 --2- 172.21.15.84:0/3841897463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e00cd930 0x7fe9e0164240 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800a940 tx=0x7fe9d802cbe0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.811+0000 7fe9e51b0640 1 -- 172.21.15.84:0/3841897463 shutdown_connections 2023-10-16T11:22:48.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.811+0000 7fe9e51b0640 1 --2- 172.21.15.84:0/3841897463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e00cd930 0x7fe9e0164240 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.811+0000 7fe9e51b0640 1 -- 172.21.15.84:0/3841897463 >> 172.21.15.84:0/3841897463 conn(0x7fe9e00cb410 msgr2=0x7fe9e00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:48.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.811+0000 7fe9e51b0640 1 -- 172.21.15.84:0/3841897463 shutdown_connections 2023-10-16T11:22:48.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.811+0000 7fe9e51b0640 1 -- 172.21.15.84:0/3841897463 wait complete. 2023-10-16T11:22:48.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.813+0000 7fe9e51b0640 1 Processor -- start 2023-10-16T11:22:48.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.813+0000 7fe9e51b0640 1 -- start start 2023-10-16T11:22:48.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.814+0000 7fe9e51b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e00caa10 0x7fe9e01f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:48.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.814+0000 7fe9e51b0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e00cd930 0x7fe9e0154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:48.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.814+0000 7fe9e51b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9e01593b0 0x7fe9e01f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:48.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.814+0000 7fe9e51b0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01f6db0 con 0x7fe9e00cd930 2023-10-16T11:22:48.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.814+0000 7fe9e51b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01f3c50 con 0x7fe9e00caa10 2023-10-16T11:22:48.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.814+0000 7fe9e51b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01f3f30 con 0x7fe9e01593b0 2023-10-16T11:22:48.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.815+0000 7fe9de59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e00cd930 0x7fe9e0154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:48.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.815+0000 7fe9de59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e00cd930 0x7fe9e0154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38536/0 (socket says 172.21.15.84:38536) 2023-10-16T11:22:48.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.815+0000 7fe9de59c640 1 -- 172.21.15.84:0/2225471489 learned_addr learned my addr 172.21.15.84:0/2225471489 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:48.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.815+0000 7fe9ded9d640 1 --2- 172.21.15.84:0/2225471489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e00caa10 0x7fe9e01f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:48.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.815+0000 7fe9de59c640 1 -- 172.21.15.84:0/2225471489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9e01593b0 msgr2=0x7fe9e01f30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:48.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.815+0000 7fe9de59c640 1 --2- 172.21.15.84:0/2225471489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9e01593b0 0x7fe9e01f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.815+0000 7fe9de59c640 1 -- 172.21.15.84:0/2225471489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e00caa10 msgr2=0x7fe9e01f71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:48.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.815+0000 7fe9de59c640 1 --2- 172.21.15.84:0/2225471489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e00caa10 0x7fe9e01f71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:48.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.815+0000 7fe9de59c640 1 -- 172.21.15.84:0/2225471489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d800a5a0 con 0x7fe9e00cd930 2023-10-16T11:22:48.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.815+0000 7fe9ded9d640 1 --2- 172.21.15.84:0/2225471489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e00caa10 0x7fe9e01f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:48.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.815+0000 7fe9de59c640 1 --2- 172.21.15.84:0/2225471489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e00cd930 0x7fe9e0154180 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800c400 tx=0x7fe9c800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:48.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.816+0000 7fe9d77fe640 1 -- 172.21.15.84:0/2225471489 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80052d0 con 0x7fe9e00cd930 2023-10-16T11:22:48.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.816+0000 7fe9d77fe640 1 -- 172.21.15.84:0/2225471489 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c8005430 con 0x7fe9e00cd930 2023-10-16T11:22:48.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.816+0000 7fe9d77fe640 1 -- 172.21.15.84:0/2225471489 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8003830 con 0x7fe9e00cd930 2023-10-16T11:22:48.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.816+0000 7fe9e51b0640 1 -- 172.21.15.84:0/2225471489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e01f66f0 con 0x7fe9e00cd930 2023-10-16T11:22:48.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.816+0000 7fe9e51b0640 1 -- 172.21.15.84:0/2225471489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e01e48f0 con 0x7fe9e00cd930 2023-10-16T11:22:48.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.817+0000 7fe9e51b0640 1 -- 172.21.15.84:0/2225471489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e01f6ef0 con 0x7fe9e00cd930 2023-10-16T11:22:48.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.818+0000 7fe9d77fe640 1 -- 172.21.15.84:0/2225471489 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe9c8003990 con 0x7fe9e00cd930 2023-10-16T11:22:48.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.819+0000 7fe9d77fe640 1 --2- 172.21.15.84:0/2225471489 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fe9c006af90 0x7fe9c006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:48.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.819+0000 7fe9d77fe640 1 -- 172.21.15.84:0/2225471489 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe9c8088ca0 con 0x7fe9e00cd930 2023-10-16T11:22:48.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.819+0000 7fe9ded9d640 1 --2- 172.21.15.84:0/2225471489 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fe9c006af90 0x7fe9c006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:48.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.820+0000 7fe9ded9d640 1 --2- 172.21.15.84:0/2225471489 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fe9c006af90 0x7fe9c006d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe9d00012e0 tx=0x7fe9d0009040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:48.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:48.823+0000 7fe9d77fe640 1 -- 172.21.15.84:0/2225471489 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe9e01f6ef0 con 0x7fe9e00cd930 2023-10-16T11:22:49.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.048+0000 7fe9e51b0640 1 -- 172.21.15.84:0/2225471489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9e01f6ef0 con 0x7fe9e00cd930 2023-10-16T11:22:49.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.049+0000 7fe9d77fe640 1 -- 172.21.15.84:0/2225471489 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4929 (secure 0 0 0) 0x7fe9e01f6ef0 con 0x7fe9e00cd930 2023-10-16T11:22:49.052 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:49.052 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6575,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/855929374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":855929374},{"type":"v1","addr":"172.21.15.157:6834","nonce":855929374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6578,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2581460972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2581460972},{"type":"v1","addr":"172.21.15.157:6835","nonce":2581460972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":10806,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1731880648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1731880648},{"type":"v1","addr":"172.21.15.84:6836","nonce":1731880648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:22:42.578443+0000","modified":"2023-10-16T11:22:45.635565+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10809":{"gid":10809,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2679249765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2679249765},{"type":"v1","addr":"172.21.15.84:6837","nonce":2679249765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-16T11:22:49.053 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 105 2023-10-16T11:22:49.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.051+0000 7fe9e51b0640 1 -- 172.21.15.84:0/2225471489 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fe9c006af90 msgr2=0x7fe9c006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:49.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.051+0000 7fe9e51b0640 1 --2- 172.21.15.84:0/2225471489 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fe9c006af90 0x7fe9c006d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe9d00012e0 tx=0x7fe9d0009040 comp rx=0 tx=0).stop 2023-10-16T11:22:49.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.051+0000 7fe9e51b0640 1 -- 172.21.15.84:0/2225471489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e00cd930 msgr2=0x7fe9e0154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:49.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.051+0000 7fe9e51b0640 1 --2- 172.21.15.84:0/2225471489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e00cd930 0x7fe9e0154180 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800c400 tx=0x7fe9c800c8c0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.051+0000 7fe9e51b0640 1 -- 172.21.15.84:0/2225471489 shutdown_connections 2023-10-16T11:22:49.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.052+0000 7fe9e51b0640 1 --2- 172.21.15.84:0/2225471489 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fe9c006af90 0x7fe9c006d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.052+0000 7fe9e51b0640 1 --2- 172.21.15.84:0/2225471489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9e01593b0 0x7fe9e01f30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.052+0000 7fe9e51b0640 1 --2- 172.21.15.84:0/2225471489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e00cd930 0x7fe9e0154180 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.052+0000 7fe9e51b0640 1 --2- 172.21.15.84:0/2225471489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e00caa10 0x7fe9e01f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.052+0000 7fe9e51b0640 1 -- 172.21.15.84:0/2225471489 >> 172.21.15.84:0/2225471489 conn(0x7fe9e00cb410 msgr2=0x7fe9e00ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:49.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.052+0000 7fe9e51b0640 1 -- 172.21.15.84:0/2225471489 shutdown_connections 2023-10-16T11:22:49.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.052+0000 7fe9e51b0640 1 -- 172.21.15.84:0/2225471489 wait complete. 2023-10-16T11:22:49.066 INFO:tasks.ceph.mds.c.smithi084.stderr:2023-10-16T11:22:49.064+0000 7f54248b1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:22:49.474 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T11:22:42.578443+0000', 'modified': '2023-10-16T11:22:45.635565+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10809}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10809': {'gid': 10809, 'name': 'c', 'rank': 0, 'incarnation': 101, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.84:6837/2679249765', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6835', 'nonce': 2679249765}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 2679249765}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T11:22:49.475 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T11:22:49.475 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-16T11:22:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.698+0000 7f1627061640 1 Processor -- start 2023-10-16T11:22:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.698+0000 7f1627061640 1 -- start start 2023-10-16T11:22:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.700+0000 7f1627061640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f162015ad70 0x7f1620069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.700+0000 7f1627061640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16200c9ca0 0x7f162006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.700+0000 7f1627061640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16200cb030 con 0x7f16200ca3b0 2023-10-16T11:22:49.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.700+0000 7f1627061640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16200c91c0 con 0x7f162015ad70 2023-10-16T11:22:49.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.700+0000 7f1627061640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16200c94a0 con 0x7f16200c9ca0 2023-10-16T11:22:49.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.701+0000 7f161ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16200c9ca0 0x7f162006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.701+0000 7f1624dfd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f162015ad70 0x7f1620069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.701+0000 7f161ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16200c9ca0 0x7f162006a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:56534/0 (socket says 172.21.15.84:56534) 2023-10-16T11:22:49.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.701+0000 7f161ffff640 1 -- 172.21.15.84:0/760071710 learned_addr learned my addr 172.21.15.84:0/760071710 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:49.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.701+0000 7f1624dfd640 1 --2- 172.21.15.84:0/760071710 >> v2:172.21.15.157:3300/0 conn(0x7f162015ad70 0x7f1620069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:57620/0 (socket says 172.21.15.84:57620) 2023-10-16T11:22:49.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.702+0000 7f161ffff640 1 -- 172.21.15.84:0/760071710 >> v2:172.21.15.157:3300/0 conn(0x7f162015ad70 msgr2=0x7f1620069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:49.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.702+0000 7f161ffff640 1 --2- 172.21.15.84:0/760071710 >> v2:172.21.15.157:3300/0 conn(0x7f162015ad70 0x7f1620069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.702+0000 7f161ffff640 1 -- 172.21.15.84:0/760071710 >> v1:172.21.15.157:6789/0 conn(0x7f16200ca3b0 legacy=0x7f16200ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:49.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.702+0000 7f161ffff640 1 -- 172.21.15.84:0/760071710 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16200c9780 con 0x7f16200c9ca0 2023-10-16T11:22:49.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.702+0000 7f1624dfd640 1 --2- 172.21.15.84:0/760071710 >> v2:172.21.15.157:3300/0 conn(0x7f162015ad70 0x7f1620069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:49.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.703+0000 7f161ffff640 1 --2- 172.21.15.84:0/760071710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16200c9ca0 0x7f162006a2e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1610009870 tx=0x7f161002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94b955085418520e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:49.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.703+0000 7f161f7fe640 1 -- 172.21.15.84:0/760071710 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610030840 con 0x7f16200c9ca0 2023-10-16T11:22:49.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.704+0000 7f161f7fe640 1 -- 172.21.15.84:0/760071710 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16100309a0 con 0x7f16200c9ca0 2023-10-16T11:22:49.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.704+0000 7f161f7fe640 1 -- 172.21.15.84:0/760071710 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610030d30 con 0x7f16200c9ca0 2023-10-16T11:22:49.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.704+0000 7f1627061640 1 -- 172.21.15.84:0/760071710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16200c9ca0 msgr2=0x7f162006a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:49.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.704+0000 7f1627061640 1 --2- 172.21.15.84:0/760071710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16200c9ca0 0x7f162006a2e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1610009870 tx=0x7f161002b980 comp rx=0 tx=0).stop 2023-10-16T11:22:49.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.704+0000 7f1627061640 1 -- 172.21.15.84:0/760071710 shutdown_connections 2023-10-16T11:22:49.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.704+0000 7f1627061640 1 --2- 172.21.15.84:0/760071710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16200c9ca0 0x7f162006a2e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.704+0000 7f1627061640 1 --2- 172.21.15.84:0/760071710 >> v2:172.21.15.157:3300/0 conn(0x7f162015ad70 0x7f1620069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.704+0000 7f1627061640 1 -- 172.21.15.84:0/760071710 >> 172.21.15.84:0/760071710 conn(0x7f16200ce9b0 msgr2=0x7f16200cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:49.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.704+0000 7f1627061640 1 -- 172.21.15.84:0/760071710 shutdown_connections 2023-10-16T11:22:49.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.704+0000 7f1627061640 1 -- 172.21.15.84:0/760071710 wait complete. 2023-10-16T11:22:49.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.706+0000 7f1627061640 1 Processor -- start 2023-10-16T11:22:49.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.706+0000 7f1627061640 1 -- start start 2023-10-16T11:22:49.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.707+0000 7f1627061640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16200c9ca0 0x7f16201f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.707+0000 7f1627061640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16200ca3b0 0x7f1620159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.707+0000 7f1627061640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162015ad70 0x7f16200c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.707+0000 7f1627061640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16200b0510 con 0x7f16200c9ca0 2023-10-16T11:22:49.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.707+0000 7f1627061640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16201596c0 con 0x7f162015ad70 2023-10-16T11:22:49.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.707+0000 7f1627061640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16200c5fc0 con 0x7f16200ca3b0 2023-10-16T11:22:49.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.708+0000 7f1624dfd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16200c9ca0 0x7f16201f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.708+0000 7f1624dfd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16200c9ca0 0x7f16201f3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38544/0 (socket says 172.21.15.84:38544) 2023-10-16T11:22:49.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.708+0000 7f1624dfd640 1 -- 172.21.15.84:0/3896972465 learned_addr learned my addr 172.21.15.84:0/3896972465 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:49.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.708+0000 7f161ffff640 1 --2- 172.21.15.84:0/3896972465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16200ca3b0 0x7f1620159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.708+0000 7f16255fe640 1 --2- 172.21.15.84:0/3896972465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162015ad70 0x7f16200c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.708+0000 7f1624dfd640 1 -- 172.21.15.84:0/3896972465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16200ca3b0 msgr2=0x7f1620159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:49.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.708+0000 7f1624dfd640 1 --2- 172.21.15.84:0/3896972465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16200ca3b0 0x7f1620159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.708+0000 7f1624dfd640 1 -- 172.21.15.84:0/3896972465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162015ad70 msgr2=0x7f16200c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:49.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.708+0000 7f1624dfd640 1 --2- 172.21.15.84:0/3896972465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162015ad70 0x7f16200c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.708+0000 7f1624dfd640 1 -- 172.21.15.84:0/3896972465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1614009400 con 0x7f16200c9ca0 2023-10-16T11:22:49.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.708+0000 7f1624dfd640 1 --2- 172.21.15.84:0/3896972465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16200c9ca0 0x7f16201f3030 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f161400e960 tx=0x7f161400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:49.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.709+0000 7f161d7fa640 1 -- 172.21.15.84:0/3896972465 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161400ca00 con 0x7f16200c9ca0 2023-10-16T11:22:49.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.709+0000 7f161d7fa640 1 -- 172.21.15.84:0/3896972465 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161400cb60 con 0x7f16200c9ca0 2023-10-16T11:22:49.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.709+0000 7f161d7fa640 1 -- 172.21.15.84:0/3896972465 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614013470 con 0x7f16200c9ca0 2023-10-16T11:22:49.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.709+0000 7f1627061640 1 -- 172.21.15.84:0/3896972465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16100094d0 con 0x7f16200c9ca0 2023-10-16T11:22:49.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.709+0000 7f1627061640 1 -- 172.21.15.84:0/3896972465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16200c6570 con 0x7f16200c9ca0 2023-10-16T11:22:49.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.711+0000 7f161d7fa640 1 -- 172.21.15.84:0/3896972465 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f16140135d0 con 0x7f16200c9ca0 2023-10-16T11:22:49.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.712+0000 7f161d7fa640 1 --2- 172.21.15.84:0/3896972465 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f160c06af90 0x7f160c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.712+0000 7f161d7fa640 1 -- 172.21.15.84:0/3896972465 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1614089a30 con 0x7f16200c9ca0 2023-10-16T11:22:49.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.712+0000 7f161ffff640 1 --2- 172.21.15.84:0/3896972465 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f160c06af90 0x7f160c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.713+0000 7f1629bf2740 1 Processor -- start 2023-10-16T11:22:49.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.713+0000 7f161ffff640 1 --2- 172.21.15.84:0/3896972465 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f160c06af90 0x7f160c06d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1610036000 tx=0x7f161003b040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:49.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.713+0000 7f1629bf2740 1 -- start start 2023-10-16T11:22:49.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.714+0000 7f1629bf2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a8582f80 0x55f8a8555e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.714+0000 7f1629bf2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a8583360 0x55f8a8585d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.714+0000 7f1629bf2740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a85563c0 0x55f8a8586250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.714+0000 7f1629bf2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a84767a0 con 0x55f8a8582f80 2023-10-16T11:22:49.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.714+0000 7f1629bf2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a84c8740 con 0x55f8a8583360 2023-10-16T11:22:49.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.714+0000 7f1629bf2740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8a84c5e30 con 0x55f8a85563c0 2023-10-16T11:22:49.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.714+0000 7f1624dfd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a8582f80 0x55f8a8555e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.714+0000 7f1624dfd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a8582f80 0x55f8a8555e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38552/0 (socket says 172.21.15.84:38552) 2023-10-16T11:22:49.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.714+0000 7f1624dfd640 1 -- 172.21.15.84:0/1602368597 learned_addr learned my addr 172.21.15.84:0/1602368597 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:49.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.715+0000 7f1624dfd640 1 -- 172.21.15.84:0/1602368597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a85563c0 msgr2=0x55f8a8586250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:49.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.715+0000 7f16255fe640 1 --2- 172.21.15.84:0/1602368597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a85563c0 0x55f8a8586250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.715+0000 7f161ffff640 1 --2- 172.21.15.84:0/1602368597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a8583360 0x55f8a8585d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.715+0000 7f1624dfd640 1 --2- 172.21.15.84:0/1602368597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a85563c0 0x55f8a8586250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.715+0000 7f1624dfd640 1 -- 172.21.15.84:0/1602368597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a8583360 msgr2=0x55f8a8585d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:49.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.715+0000 7f1624dfd640 1 --2- 172.21.15.84:0/1602368597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a8583360 0x55f8a8585d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.715+0000 7f1624dfd640 1 -- 172.21.15.84:0/1602368597 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8a8485910 con 0x55f8a8582f80 2023-10-16T11:22:49.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.715+0000 7f1624dfd640 1 --2- 172.21.15.84:0/1602368597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a8582f80 0x55f8a8555e80 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f1614087d50 tx=0x7f16140580c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0ff7963c51b09f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:49.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.715+0000 7f16067fc640 1 -- 172.21.15.84:0/1602368597 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614060960 con 0x55f8a8582f80 2023-10-16T11:22:49.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.715+0000 7f16067fc640 1 -- 172.21.15.84:0/1602368597 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1614087e90 con 0x55f8a8582f80 2023-10-16T11:22:49.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.715+0000 7f16067fc640 1 -- 172.21.15.84:0/1602368597 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614088000 con 0x55f8a8582f80 2023-10-16T11:22:49.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.716+0000 7f1629bf2740 1 -- 172.21.15.84:0/1602368597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a8582f80 msgr2=0x55f8a8555e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:49.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.716+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1602368597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a8582f80 0x55f8a8555e80 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f1614087d50 tx=0x7f16140580c0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.716+0000 7f1629bf2740 1 -- 172.21.15.84:0/1602368597 shutdown_connections 2023-10-16T11:22:49.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.716+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1602368597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a85563c0 0x55f8a8586250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.716+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1602368597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a8583360 0x55f8a8585d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.716+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1602368597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a8582f80 0x55f8a8555e80 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.716+0000 7f1629bf2740 1 -- 172.21.15.84:0/1602368597 >> 172.21.15.84:0/1602368597 conn(0x55f8a8558190 msgr2=0x55f8a85a0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:49.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.716+0000 7f1629bf2740 1 -- 172.21.15.84:0/1602368597 shutdown_connections 2023-10-16T11:22:49.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.716+0000 7f1629bf2740 1 -- 172.21.15.84:0/1602368597 wait complete. 2023-10-16T11:22:49.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.717+0000 7f1629bf2740 1 -- start start 2023-10-16T11:22:49.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.717+0000 7f1629bf2740 1 Processor -- start 2023-10-16T11:22:49.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.718+0000 7f1629bf2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a8582f80 0x55f8a8574290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.718+0000 7f1629bf2740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a8583360 0x55f8a859b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.718+0000 7f1629bf2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a859b880 0x55f8a8588270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.718+0000 7f1629bf2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a84c1720 con 0x55f8a859b880 2023-10-16T11:22:49.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.718+0000 7f1629bf2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a845f970 con 0x55f8a8582f80 2023-10-16T11:22:49.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.718+0000 7f1629bf2740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8a845f610 con 0x55f8a8583360 2023-10-16T11:22:49.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.718+0000 7f1629bf2740 10 client.0 _ll_register_callbacks cb 0x55f8a84819d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:22:49.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.718+0000 7f16255fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a859b880 0x55f8a8588270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.718+0000 7f16255fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a859b880 0x55f8a8588270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38554/0 (socket says 172.21.15.84:38554) 2023-10-16T11:22:49.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.718+0000 7f16255fe640 1 -- 172.21.15.84:0/1025974112 learned_addr learned my addr 172.21.15.84:0/1025974112 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:49.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.719+0000 7f161ffff640 1 --2- 172.21.15.84:0/1025974112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a8583360 0x55f8a859b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.719+0000 7f1624dfd640 1 --2- 172.21.15.84:0/1025974112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a8582f80 0x55f8a8574290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.719+0000 7f16255fe640 1 -- 172.21.15.84:0/1025974112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a8583360 msgr2=0x55f8a859b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:49.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.719+0000 7f16255fe640 1 --2- 172.21.15.84:0/1025974112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a8583360 0x55f8a859b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.719+0000 7f16255fe640 1 -- 172.21.15.84:0/1025974112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a8582f80 msgr2=0x55f8a8574290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:49.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.719+0000 7f16255fe640 1 --2- 172.21.15.84:0/1025974112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a8582f80 0x55f8a8574290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.719+0000 7f16255fe640 1 -- 172.21.15.84:0/1025974112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16200c6570 con 0x55f8a859b880 2023-10-16T11:22:49.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.719+0000 7f16255fe640 1 --2- 172.21.15.84:0/1025974112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a859b880 0x55f8a8588270 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f161800e4d0 tx=0x7f161800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a2e8dfec43ac88 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:49.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.719+0000 7f15ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T11:22:49.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.719+0000 7f15ff7fe640 1 -- 172.21.15.84:0/1025974112 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1618003b90 con 0x55f8a859b880 2023-10-16T11:22:49.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.719+0000 7f15ff7fe640 1 -- 172.21.15.84:0/1025974112 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(201..201 src has 1..201) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1618003cf0 con 0x55f8a859b880 2023-10-16T11:22:49.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.719+0000 7f1629bf2740 1 -- 172.21.15.84:0/1025974112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8a84e8160 con 0x55f8a859b880 2023-10-16T11:22:49.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.719+0000 7f15ff7fe640 1 -- 172.21.15.84:0/1025974112 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16180050d0 con 0x55f8a859b880 2023-10-16T11:22:49.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.720+0000 7f1629bf2740 10 client.10944 fetch_fsmap learned FSMap version 105 2023-10-16T11:22:49.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.720+0000 7f1629bf2740 1 -- 172.21.15.84:0/1025974112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x55f8a84bfda0 con 0x55f8a859b880 2023-10-16T11:22:49.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.720+0000 7f15ff7fe640 1 -- 172.21.15.84:0/1025974112 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f8a84e8160 con 0x55f8a859b880 2023-10-16T11:22:49.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.720+0000 7f15ff7fe640 1 -- 172.21.15.84:0/1025974112 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1618005230 con 0x55f8a859b880 2023-10-16T11:22:49.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.720+0000 7f1629bf2740 10 client.10944 fetch_fsmap finished waiting for FSMap version 105 2023-10-16T11:22:49.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.720+0000 7f1629bf2740 10 client.10944 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T11:22:49.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.720+0000 7f1629bf2740 20 client.10944 populate_metadata read hostname 'smithi084' 2023-10-16T11:22:49.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.720+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1025974112 >> [v2:172.21.15.84:6835/2679249765,v1:172.21.15.84:6837/2679249765] conn(0x55f8a8598490 0x55f8a85998b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.720+0000 7f1629bf2740 4 client.10944 mds_command: new command op to 10809 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T11:22:49.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.720+0000 7f1629bf2740 1 -- 172.21.15.84:0/1025974112 --> [v2:172.21.15.84:6835/2679249765,v1:172.21.15.84:6837/2679249765] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8a8493cc0 con 0x55f8a8598490 2023-10-16T11:22:49.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.720+0000 7f1624dfd640 1 --2- 172.21.15.84:0/1025974112 >> [v2:172.21.15.84:6835/2679249765,v1:172.21.15.84:6837/2679249765] conn(0x55f8a8598490 0x55f8a85998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.721+0000 7f1624dfd640 1 --2- 172.21.15.84:0/1025974112 >> [v2:172.21.15.84:6835/2679249765,v1:172.21.15.84:6837/2679249765] conn(0x55f8a8598490 0x55f8a85998b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ff6577acbf99cfe server_cookie=8cc6a6c0d8c352bc in_seq=0 out_seq=0 2023-10-16T11:22:49.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.721+0000 7f15ff7fe640 10 client.10944 ms_handle_connect on v2:172.21.15.84:6835/2679249765 2023-10-16T11:22:49.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.721+0000 7f15ff7fe640 1 -- 172.21.15.84:0/1025974112 <== mds.0 v2:172.21.15.84:6835/2679249765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f8a8493cc0 con 0x55f8a8598490 2023-10-16T11:22:49.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.721+0000 7f15ff7fe640 10 client.10944 handle_command_reply: tid=0 2023-10-16T11:22:49.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.721+0000 7f1629bf2740 1 client.10944 shutdown 2023-10-16T11:22:49.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.722+0000 7f1629bf2740 1 -- 172.21.15.84:0/1025974112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a859b880 msgr2=0x55f8a8588270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:49.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.722+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1025974112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a859b880 0x55f8a8588270 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f161800e4d0 tx=0x7f161800e990 comp rx=0 tx=0).stop 2023-10-16T11:22:49.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.722+0000 7f1629bf2740 1 -- 172.21.15.84:0/1025974112 shutdown_connections 2023-10-16T11:22:49.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.722+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1025974112 >> [v2:172.21.15.84:6835/2679249765,v1:172.21.15.84:6837/2679249765] conn(0x55f8a8598490 0x55f8a85998b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.722+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1025974112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a859b880 0x55f8a8588270 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.722+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1025974112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a8583360 0x55f8a859b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.722+0000 7f15ff7fe640 0 client.10944 ms_handle_reset on v2:172.21.15.84:6835/2679249765 2023-10-16T11:22:49.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.722+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1025974112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a8582f80 0x55f8a8574290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.722+0000 7f1629bf2740 1 -- 172.21.15.84:0/1025974112 >> 172.21.15.84:0/1025974112 conn(0x55f8a85563c0 msgr2=0x55f8a8559e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:49.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.722+0000 7f1629bf2740 1 -- 172.21.15.84:0/1025974112 shutdown_connections 2023-10-16T11:22:49.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.722+0000 7f1629bf2740 1 -- 172.21.15.84:0/1025974112 wait complete. 2023-10-16T11:22:49.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.722+0000 7f1629bf2740 20 client.10944 trim_cache size 0 max 16384 2023-10-16T11:22:49.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.739+0000 7f1629bf2740 1 Processor -- start 2023-10-16T11:22:49.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.739+0000 7f1629bf2740 1 -- start start 2023-10-16T11:22:49.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.740+0000 7f1629bf2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a8582f80 0x55f8a8559a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.740+0000 7f1629bf2740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a8583360 0x55f8a8559fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.740+0000 7f1629bf2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a859b880 0x55f8a85731b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.740+0000 7f1629bf2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a8225f20 con 0x55f8a8582f80 2023-10-16T11:22:49.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.740+0000 7f1629bf2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a858a170 con 0x55f8a859b880 2023-10-16T11:22:49.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.740+0000 7f1629bf2740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8a85c37b0 con 0x55f8a8583360 2023-10-16T11:22:49.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.740+0000 7f1624dfd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a8582f80 0x55f8a8559a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.740+0000 7f1624dfd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a8582f80 0x55f8a8559a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38560/0 (socket says 172.21.15.84:38560) 2023-10-16T11:22:49.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.740+0000 7f1624dfd640 1 -- 172.21.15.84:0/1896332424 learned_addr learned my addr 172.21.15.84:0/1896332424 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:49.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.740+0000 7f1624dfd640 1 -- 172.21.15.84:0/1896332424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a8583360 msgr2=0x55f8a8559fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:49.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.740+0000 7f1624dfd640 1 --2- 172.21.15.84:0/1896332424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a8583360 0x55f8a8559fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.740+0000 7f1624dfd640 1 -- 172.21.15.84:0/1896332424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a859b880 msgr2=0x55f8a85731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:49.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.740+0000 7f1624dfd640 1 --2- 172.21.15.84:0/1896332424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a859b880 0x55f8a85731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.740+0000 7f1624dfd640 1 -- 172.21.15.84:0/1896332424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8a845eb80 con 0x55f8a8582f80 2023-10-16T11:22:49.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.741+0000 7f1624dfd640 1 --2- 172.21.15.84:0/1896332424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a8582f80 0x55f8a8559a80 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f16140569b0 tx=0x7f161401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d81cc61d350e0c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:49.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.741+0000 7f1606ffd640 1 -- 172.21.15.84:0/1896332424 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614087040 con 0x55f8a8582f80 2023-10-16T11:22:49.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.741+0000 7f1606ffd640 1 -- 172.21.15.84:0/1896332424 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1614058430 con 0x55f8a8582f80 2023-10-16T11:22:49.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.741+0000 7f1606ffd640 1 -- 172.21.15.84:0/1896332424 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614054d70 con 0x55f8a8582f80 2023-10-16T11:22:49.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.741+0000 7f1629bf2740 1 -- 172.21.15.84:0/1896332424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a8582f80 msgr2=0x55f8a8559a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:49.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.741+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1896332424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a8582f80 0x55f8a8559a80 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f16140569b0 tx=0x7f161401d000 comp rx=0 tx=0).stop 2023-10-16T11:22:49.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.742+0000 7f1629bf2740 1 -- 172.21.15.84:0/1896332424 shutdown_connections 2023-10-16T11:22:49.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.742+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1896332424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a859b880 0x55f8a85731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.742+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1896332424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a8583360 0x55f8a8559fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.742+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1896332424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a8582f80 0x55f8a8559a80 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.742+0000 7f1629bf2740 1 -- 172.21.15.84:0/1896332424 >> 172.21.15.84:0/1896332424 conn(0x55f8a85563c0 msgr2=0x55f8a85740a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:49.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.742+0000 7f1629bf2740 1 -- 172.21.15.84:0/1896332424 shutdown_connections 2023-10-16T11:22:49.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.742+0000 7f1629bf2740 1 -- 172.21.15.84:0/1896332424 wait complete. 2023-10-16T11:22:49.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.742+0000 7f1629bf2740 1 -- start start 2023-10-16T11:22:49.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.743+0000 7f1629bf2740 1 Processor -- start 2023-10-16T11:22:49.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.744+0000 7f1629bf2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a8582f80 0x55f8a85bacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.744+0000 7f1629bf2740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a8583360 0x55f8a855ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.744+0000 7f1629bf2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a859b880 0x55f8a855b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.744+0000 7f1629bf2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a8597ae0 con 0x55f8a859b880 2023-10-16T11:22:49.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.744+0000 7f1629bf2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a859a480 con 0x55f8a8582f80 2023-10-16T11:22:49.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.744+0000 7f1629bf2740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8a859c420 con 0x55f8a8583360 2023-10-16T11:22:49.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.744+0000 7f1629bf2740 10 client.0 _ll_register_callbacks cb 0x7f1618007ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:22:49.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.744+0000 7f16255fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a859b880 0x55f8a855b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.744+0000 7f16255fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a859b880 0x55f8a855b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38564/0 (socket says 172.21.15.84:38564) 2023-10-16T11:22:49.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.744+0000 7f16255fe640 1 -- 172.21.15.84:0/1485578571 learned_addr learned my addr 172.21.15.84:0/1485578571 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:49.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.744+0000 7f161ffff640 1 --2- 172.21.15.84:0/1485578571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a8583360 0x55f8a855ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.744+0000 7f16255fe640 1 -- 172.21.15.84:0/1485578571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a8583360 msgr2=0x55f8a855ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:49.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.744+0000 7f1624dfd640 1 --2- 172.21.15.84:0/1485578571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a8582f80 0x55f8a85bacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.744+0000 7f16255fe640 1 --2- 172.21.15.84:0/1485578571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a8583360 0x55f8a855ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.744+0000 7f16255fe640 1 -- 172.21.15.84:0/1485578571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a8582f80 msgr2=0x55f8a85bacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:49.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.744+0000 7f16255fe640 1 --2- 172.21.15.84:0/1485578571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a8582f80 0x55f8a85bacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.744+0000 7f16255fe640 1 -- 172.21.15.84:0/1485578571 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8a8485910 con 0x55f8a859b880 2023-10-16T11:22:49.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.745+0000 7f16255fe640 1 --2- 172.21.15.84:0/1485578571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a859b880 0x55f8a855b160 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f161801d040 tx=0x7f1618001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8f7d0958785dff server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:49.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.745+0000 7f15ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T11:22:49.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.745+0000 7f15ff7fe640 1 -- 172.21.15.84:0/1485578571 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1618003d80 con 0x55f8a859b880 2023-10-16T11:22:49.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.745+0000 7f15ff7fe640 1 -- 172.21.15.84:0/1485578571 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(201..201 src has 1..201) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1618007ba0 con 0x55f8a859b880 2023-10-16T11:22:49.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.745+0000 7f1629bf2740 1 -- 172.21.15.84:0/1485578571 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1614087e90 con 0x55f8a859b880 2023-10-16T11:22:49.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.745+0000 7f15ff7fe640 1 -- 172.21.15.84:0/1485578571 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1618013670 con 0x55f8a859b880 2023-10-16T11:22:49.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.745+0000 7f1629bf2740 10 client.10950 fetch_fsmap learned FSMap version 105 2023-10-16T11:22:49.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.745+0000 7f1629bf2740 1 -- 172.21.15.84:0/1485578571 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x55f8a8484170 con 0x55f8a859b880 2023-10-16T11:22:49.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.745+0000 7f15ff7fe640 1 -- 172.21.15.84:0/1485578571 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1614087e90 con 0x55f8a859b880 2023-10-16T11:22:49.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.745+0000 7f15ff7fe640 1 -- 172.21.15.84:0/1485578571 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f16180137d0 con 0x55f8a859b880 2023-10-16T11:22:49.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.745+0000 7f1629bf2740 10 client.10950 fetch_fsmap finished waiting for FSMap version 105 2023-10-16T11:22:49.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.745+0000 7f1629bf2740 10 client.10950 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T11:22:49.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.745+0000 7f1629bf2740 20 client.10950 populate_metadata read hostname 'smithi084' 2023-10-16T11:22:49.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.745+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1485578571 >> [v2:172.21.15.84:6835/2679249765,v1:172.21.15.84:6837/2679249765] conn(0x55f8a8596ed0 0x55f8a85a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.745+0000 7f1629bf2740 4 client.10950 mds_command: new command op to 10809 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T11:22:49.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.745+0000 7f1629bf2740 1 -- 172.21.15.84:0/1485578571 --> [v2:172.21.15.84:6835/2679249765,v1:172.21.15.84:6837/2679249765] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f8a84ba700 con 0x55f8a8596ed0 2023-10-16T11:22:49.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.746+0000 7f1624dfd640 1 --2- 172.21.15.84:0/1485578571 >> [v2:172.21.15.84:6835/2679249765,v1:172.21.15.84:6837/2679249765] conn(0x55f8a8596ed0 0x55f8a85a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.746+0000 7f1624dfd640 1 --2- 172.21.15.84:0/1485578571 >> [v2:172.21.15.84:6835/2679249765,v1:172.21.15.84:6837/2679249765] conn(0x55f8a8596ed0 0x55f8a85a1f60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32943741771f349d server_cookie=1d45e1bfb8b8e12e in_seq=0 out_seq=0 2023-10-16T11:22:49.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.746+0000 7f15ff7fe640 10 client.10950 ms_handle_connect on v2:172.21.15.84:6835/2679249765 2023-10-16T11:22:49.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.746+0000 7f15ff7fe640 1 -- 172.21.15.84:0/1485578571 <== mds.0 v2:172.21.15.84:6835/2679249765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55f8a84ba700 con 0x55f8a8596ed0 2023-10-16T11:22:49.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.746+0000 7f15ff7fe640 10 client.10950 handle_command_reply: tid=0 2023-10-16T11:22:49.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.746+0000 7f1629bf2740 1 client.10950 shutdown 2023-10-16T11:22:49.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.747+0000 7f1629bf2740 1 -- 172.21.15.84:0/1485578571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a859b880 msgr2=0x55f8a855b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:49.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.747+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1485578571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a859b880 0x55f8a855b160 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f161801d040 tx=0x7f1618001250 comp rx=0 tx=0).stop 2023-10-16T11:22:49.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.747+0000 7f1629bf2740 1 -- 172.21.15.84:0/1485578571 shutdown_connections 2023-10-16T11:22:49.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.747+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1485578571 >> [v2:172.21.15.84:6835/2679249765,v1:172.21.15.84:6837/2679249765] conn(0x55f8a8596ed0 0x55f8a85a1f60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.747+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1485578571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8a859b880 0x55f8a855b160 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.747+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1485578571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f8a8583360 0x55f8a855ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.747+0000 7f1629bf2740 1 --2- 172.21.15.84:0/1485578571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f8a8582f80 0x55f8a85bacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.747+0000 7f15ff7fe640 0 client.10950 ms_handle_reset on v2:172.21.15.84:6835/2679249765 2023-10-16T11:22:49.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.747+0000 7f1629bf2740 1 -- 172.21.15.84:0/1485578571 >> 172.21.15.84:0/1485578571 conn(0x55f8a85563c0 msgr2=0x55f8a8587e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:49.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.747+0000 7f1629bf2740 1 -- 172.21.15.84:0/1485578571 shutdown_connections 2023-10-16T11:22:49.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.747+0000 7f1629bf2740 1 -- 172.21.15.84:0/1485578571 wait complete. 2023-10-16T11:22:49.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.747+0000 7f1629bf2740 20 client.10950 trim_cache size 0 max 16384 2023-10-16T11:22:49.765 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:22:49.765 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "63fb4d68-f881-4694-b880-d10369105c0e", 2023-10-16T11:22:49.765 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-16T11:22:49.766 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 10809, 2023-10-16T11:22:49.766 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-16T11:22:49.766 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-16T11:22:49.766 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-16T11:22:49.767 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 6.6358481100000004, 2023-10-16T11:22:49.767 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 105, 2023-10-16T11:22:49.767 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 201, 2023-10-16T11:22:49.767 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 201, 2023-10-16T11:22:49.767 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 12.032667029000001 2023-10-16T11:22:49.767 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:22:49.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.747+0000 7f1627061640 1 -- 172.21.15.84:0/3896972465 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f160c06af90 msgr2=0x7f160c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:49.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.747+0000 7f1627061640 1 --2- 172.21.15.84:0/3896972465 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f160c06af90 0x7f160c06d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1610036000 tx=0x7f161003b040 comp rx=0 tx=0).stop 2023-10-16T11:22:49.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.747+0000 7f1627061640 1 -- 172.21.15.84:0/3896972465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16200c9ca0 msgr2=0x7f16201f3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:49.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.747+0000 7f1627061640 1 --2- 172.21.15.84:0/3896972465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16200c9ca0 0x7f16201f3030 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f161400e960 tx=0x7f161400ee20 comp rx=0 tx=0).stop 2023-10-16T11:22:49.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.748+0000 7f1627061640 1 -- 172.21.15.84:0/3896972465 shutdown_connections 2023-10-16T11:22:49.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.748+0000 7f1627061640 1 --2- 172.21.15.84:0/3896972465 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f160c06af90 0x7f160c06d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.748+0000 7f1627061640 1 --2- 172.21.15.84:0/3896972465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162015ad70 0x7f16200c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.748+0000 7f1627061640 1 --2- 172.21.15.84:0/3896972465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16200ca3b0 0x7f1620159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.748+0000 7f1627061640 1 --2- 172.21.15.84:0/3896972465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16200c9ca0 0x7f16201f3030 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.748+0000 7f1627061640 1 -- 172.21.15.84:0/3896972465 >> 172.21.15.84:0/3896972465 conn(0x7f16200ce9b0 msgr2=0x7f162015ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:49.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.748+0000 7f1627061640 1 -- 172.21.15.84:0/3896972465 shutdown_connections 2023-10-16T11:22:49.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.748+0000 7f1627061640 1 -- 172.21.15.84:0/3896972465 wait complete. 2023-10-16T11:22:49.771 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T11:22:49.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.978+0000 7f35b398a640 1 Processor -- start 2023-10-16T11:22:49.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.979+0000 7f35b398a640 1 -- start start 2023-10-16T11:22:49.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.980+0000 7f35b398a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35ac15ae80 0x7f35ac15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.980+0000 7f35b398a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35ac15ed60 con 0x7f35ac15aaa0 2023-10-16T11:22:49.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.980+0000 7f35b398a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35ac0707e0 con 0x7f35ac0ce390 2023-10-16T11:22:49.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.980+0000 7f35b398a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35ac06b810 con 0x7f35ac15ae80 2023-10-16T11:22:49.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.980+0000 7f35b1726640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f35ac15aaa0 0x7f35ac0cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54944/0 (socket says 172.21.15.84:54944) 2023-10-16T11:22:49.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.980+0000 7f35b1726640 1 -- 172.21.15.84:0/2070818677 learned_addr learned my addr 172.21.15.84:0/2070818677 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:49.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.981+0000 7f35a3fff640 1 -- 172.21.15.84:0/2070818677 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499871517 0 0) 0x7f35ac15ed60 con 0x7f35ac15aaa0 2023-10-16T11:22:49.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.981+0000 7f35a3fff640 1 -- 172.21.15.84:0/2070818677 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3598003520 con 0x7f35ac15aaa0 2023-10-16T11:22:49.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.981+0000 7f35b0f25640 1 --2- 172.21.15.84:0/2070818677 >> v2:172.21.15.157:3300/0 conn(0x7f35ac15ae80 0x7f35ac15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.981+0000 7f35a3fff640 1 -- 172.21.15.84:0/2070818677 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3986867834 0 0) 0x7f3598003520 con 0x7f35ac15aaa0 2023-10-16T11:22:49.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.981+0000 7f35a3fff640 1 -- 172.21.15.84:0/2070818677 >> v2:172.21.15.157:3300/0 conn(0x7f35ac15ae80 msgr2=0x7f35ac15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:49.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.981+0000 7f35a3fff640 1 --2- 172.21.15.84:0/2070818677 >> v2:172.21.15.157:3300/0 conn(0x7f35ac15ae80 0x7f35ac15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.981+0000 7f35a3fff640 1 -- 172.21.15.84:0/2070818677 >> v1:172.21.15.157:6789/0 conn(0x7f35ac0ce390 legacy=0x7f35ac070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:49.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.981+0000 7f35a3fff640 1 -- 172.21.15.84:0/2070818677 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac15ef90 con 0x7f35ac15aaa0 2023-10-16T11:22:49.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.981+0000 7f35a3fff640 1 -- 172.21.15.84:0/2070818677 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f35a8001670 con 0x7f35ac15aaa0 2023-10-16T11:22:49.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.981+0000 7f35a3fff640 1 -- 172.21.15.84:0/2070818677 >> v1:172.21.15.84:6789/0 conn(0x7f35ac15aaa0 legacy=0x7f35ac0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:49.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.981+0000 7f35a3fff640 1 --2- 172.21.15.84:0/2070818677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3598003970 0x7f3598003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.981+0000 7f35a3fff640 1 --2- 172.21.15.84:0/2070818677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35980047c0 0x7f3598004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.981+0000 7f35a3fff640 1 --2- 172.21.15.84:0/2070818677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598005180 0x7f359800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.981+0000 7f35a3fff640 1 -- 172.21.15.84:0/2070818677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac06b810 con 0x7f35980047c0 2023-10-16T11:22:49.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.981+0000 7f35a3fff640 1 -- 172.21.15.84:0/2070818677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359800c0b0 con 0x7f3598005180 2023-10-16T11:22:49.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.981+0000 7f35a3fff640 1 -- 172.21.15.84:0/2070818677 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359800c270 con 0x7f3598003970 2023-10-16T11:22:49.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.982+0000 7f35b0f25640 1 --2- 172.21.15.84:0/2070818677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35980047c0 0x7f3598004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.982+0000 7f35b1726640 1 --2- 172.21.15.84:0/2070818677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3598003970 0x7f3598003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.982+0000 7f35b1f27640 1 --2- 172.21.15.84:0/2070818677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598005180 0x7f359800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.982+0000 7f35b0f25640 1 -- 172.21.15.84:0/2070818677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3598003970 msgr2=0x7f3598003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:49.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.982+0000 7f35b0f25640 1 --2- 172.21.15.84:0/2070818677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3598003970 0x7f3598003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.982+0000 7f35b0f25640 1 -- 172.21.15.84:0/2070818677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598005180 msgr2=0x7f359800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:49.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.982+0000 7f35b0f25640 1 --2- 172.21.15.84:0/2070818677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598005180 0x7f359800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.982+0000 7f35b0f25640 1 -- 172.21.15.84:0/2070818677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3598003520 con 0x7f35980047c0 2023-10-16T11:22:49.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.982+0000 7f35b1f27640 1 -- 172.21.15.84:0/2070818677 reap_dead start 2023-10-16T11:22:49.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.982+0000 7f35b1726640 1 --2- 172.21.15.84:0/2070818677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3598003970 0x7f3598003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:49.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.983+0000 7f35b0f25640 1 --2- 172.21.15.84:0/2070818677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35980047c0 0x7f3598004c40 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f35ac15b2f0 tx=0x7f35980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb3d9f854889105 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.983+0000 7f35a3fff640 1 -- 172.21.15.84:0/2070818677 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c033680 con 0x7f35980047c0 2023-10-16T11:22:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.983+0000 7f35a3fff640 1 -- 172.21.15.84:0/2070818677 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359c0337e0 con 0x7f35980047c0 2023-10-16T11:22:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.983+0000 7f35a3fff640 1 -- 172.21.15.84:0/2070818677 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c033b70 con 0x7f35980047c0 2023-10-16T11:22:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.984+0000 7f35b398a640 1 -- 172.21.15.84:0/2070818677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35980047c0 msgr2=0x7f3598004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.984+0000 7f35b398a640 1 --2- 172.21.15.84:0/2070818677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35980047c0 0x7f3598004c40 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f35ac15b2f0 tx=0x7f35980057f0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.984+0000 7f35b398a640 1 -- 172.21.15.84:0/2070818677 shutdown_connections 2023-10-16T11:22:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.984+0000 7f35b398a640 1 --2- 172.21.15.84:0/2070818677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35980047c0 0x7f3598004c40 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.984+0000 7f35b398a640 1 -- 172.21.15.84:0/2070818677 >> 172.21.15.84:0/2070818677 conn(0x7f35ac0cb410 msgr2=0x7f35ac0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.984+0000 7f35b398a640 1 -- 172.21.15.84:0/2070818677 shutdown_connections 2023-10-16T11:22:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.984+0000 7f35b398a640 1 -- 172.21.15.84:0/2070818677 wait complete. 2023-10-16T11:22:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.986+0000 7f35b398a640 1 Processor -- start 2023-10-16T11:22:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.986+0000 7f35b398a640 1 -- start start 2023-10-16T11:22:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.987+0000 7f35b398a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35980047c0 0x7f35ac15b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.987+0000 7f35b398a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35ac159170 0x7f35ac15ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.987+0000 7f35b398a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac147c90 0x7f35ac06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.987+0000 7f35b398a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac0c53d0 con 0x7f35ac159170 2023-10-16T11:22:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.987+0000 7f35b398a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac15b7d0 con 0x7f35ac147c90 2023-10-16T11:22:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.987+0000 7f35b398a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac15b9e0 con 0x7f35980047c0 2023-10-16T11:22:49.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.987+0000 7f35b0f25640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35ac159170 0x7f35ac15ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:49.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.987+0000 7f35b0f25640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35ac159170 0x7f35ac15ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38578/0 (socket says 172.21.15.84:38578) 2023-10-16T11:22:49.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.987+0000 7f35b0f25640 1 -- 172.21.15.84:0/4078675919 learned_addr learned my addr 172.21.15.84:0/4078675919 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:49.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.988+0000 7f35b0f25640 1 -- 172.21.15.84:0/4078675919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35980047c0 msgr2=0x7f35ac15b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.988+0000 7f35b0f25640 1 --2- 172.21.15.84:0/4078675919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35980047c0 0x7f35ac15b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.988+0000 7f35b0f25640 1 -- 172.21.15.84:0/4078675919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac147c90 msgr2=0x7f35ac06fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.988+0000 7f35b0f25640 1 --2- 172.21.15.84:0/4078675919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac147c90 0x7f35ac06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.988+0000 7f35b0f25640 1 -- 172.21.15.84:0/4078675919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac15bcc0 con 0x7f35ac159170 2023-10-16T11:22:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.988+0000 7f35b0f25640 1 --2- 172.21.15.84:0/4078675919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35ac159170 0x7f35ac15ca80 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f359c02be60 tx=0x7f359c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:50.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.988+0000 7f35a1ffb640 1 -- 172.21.15.84:0/4078675919 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c004220 con 0x7f35ac159170 2023-10-16T11:22:50.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.988+0000 7f35a1ffb640 1 -- 172.21.15.84:0/4078675919 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359c040070 con 0x7f35ac159170 2023-10-16T11:22:50.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.988+0000 7f35b398a640 1 -- 172.21.15.84:0/4078675919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ac15bfa0 con 0x7f35ac159170 2023-10-16T11:22:50.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.988+0000 7f35a1ffb640 1 -- 172.21.15.84:0/4078675919 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c033610 con 0x7f35ac159170 2023-10-16T11:22:50.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.988+0000 7f35b398a640 1 -- 172.21.15.84:0/4078675919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ac15c560 con 0x7f35ac159170 2023-10-16T11:22:50.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.989+0000 7f35b398a640 1 -- 172.21.15.84:0/4078675919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35ac15c720 con 0x7f35ac159170 2023-10-16T11:22:50.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.990+0000 7f35a1ffb640 1 -- 172.21.15.84:0/4078675919 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f359c03c600 con 0x7f35ac159170 2023-10-16T11:22:50.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.992+0000 7f35a1ffb640 1 --2- 172.21.15.84:0/4078675919 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f359006af90 0x7f359006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:50.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.992+0000 7f35a1ffb640 1 -- 172.21.15.84:0/4078675919 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f359c0abad0 con 0x7f35ac159170 2023-10-16T11:22:50.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.992+0000 7f35b1726640 1 --2- 172.21.15.84:0/4078675919 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f359006af90 0x7f359006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:50.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.993+0000 7f35b1726640 1 --2- 172.21.15.84:0/4078675919 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f359006af90 0x7f359006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f35a8000fd0 tx=0x7f35a8009040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:50.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:49.996+0000 7f35a1ffb640 1 -- 172.21.15.84:0/4078675919 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f359c07a4e0 con 0x7f35ac159170 2023-10-16T11:22:50.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:50.218+0000 7f35b398a640 1 -- 172.21.15.84:0/4078675919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f35ac04e160 con 0x7f35ac159170 2023-10-16T11:22:50.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:50.219+0000 7f35a1ffb640 1 -- 172.21.15.84:0/4078675919 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) v1 ==== 53+0+73 (secure 0 0 0) 0x7f359c042070 con 0x7f35ac159170 2023-10-16T11:22:50.221 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T11:22:50.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:50.221+0000 7f35b398a640 1 -- 172.21.15.84:0/4078675919 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f359006af90 msgr2=0x7f359006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:50.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:50.221+0000 7f35b398a640 1 --2- 172.21.15.84:0/4078675919 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f359006af90 0x7f359006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f35a8000fd0 tx=0x7f35a8009040 comp rx=0 tx=0).stop 2023-10-16T11:22:50.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:50.221+0000 7f35b398a640 1 -- 172.21.15.84:0/4078675919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35ac159170 msgr2=0x7f35ac15ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:50.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:50.221+0000 7f35b398a640 1 --2- 172.21.15.84:0/4078675919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35ac159170 0x7f35ac15ca80 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f359c02be60 tx=0x7f359c004730 comp rx=0 tx=0).stop 2023-10-16T11:22:50.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:50.221+0000 7f35b398a640 1 -- 172.21.15.84:0/4078675919 shutdown_connections 2023-10-16T11:22:50.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:50.222+0000 7f35b398a640 1 --2- 172.21.15.84:0/4078675919 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f359006af90 0x7f359006d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:50.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:50.222+0000 7f35b398a640 1 --2- 172.21.15.84:0/4078675919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac147c90 0x7f35ac06fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:50.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:50.222+0000 7f35b398a640 1 --2- 172.21.15.84:0/4078675919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35ac159170 0x7f35ac15ca80 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:50.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:50.222+0000 7f35b398a640 1 --2- 172.21.15.84:0/4078675919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35980047c0 0x7f35ac15b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:50.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:50.222+0000 7f35b398a640 1 -- 172.21.15.84:0/4078675919 >> 172.21.15.84:0/4078675919 conn(0x7f35ac0cb410 msgr2=0x7f35ac0b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:50.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:50.222+0000 7f35b398a640 1 -- 172.21.15.84:0/4078675919 shutdown_connections 2023-10-16T11:22:50.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:50.222+0000 7f35b398a640 1 -- 172.21.15.84:0/4078675919 wait complete. 2023-10-16T11:22:50.243 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T11:22:50.243 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T11:22:50.244 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-16T11:22:50.244 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T11:22:50.244 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T11:22:50.245 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T11:22:50.245 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T11:22:50.246 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T11:22:50.246 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:22:50.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:50.247 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-16T11:22:50.301 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T11:22:50.301 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T11:22:50.302 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T11:22:50.302 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:22:50.302 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T11:22:50.302 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:22:50.302 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:22:50.303 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:22:50.303 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-16T11:22:50.303 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:22:50.304 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:22:50.304 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-16T11:22:50.305 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:22:50.305 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:22:50.305 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T11:22:50.305 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:22:50.306 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T11:22:50.306 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 36781sec preferred_lft 36781sec 2023-10-16T11:22:50.307 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-16T11:22:50.307 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:22:50.307 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T11:22:50.307 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:22:50.308 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T11:22:50.308 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:22:50.308 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::20a8:eaff:fe9d:5cea/64 scope link 2023-10-16T11:22:50.308 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:22:50.308 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:22:50.309 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T11:22:50.309 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1854:13ff:fe47:be92/64 scope link 2023-10-16T11:22:50.309 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:22:50.309 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:22:50.309 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether ee:d5:41:a3:a5:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T11:22:50.310 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ecd5:41ff:fea3:a570/64 scope link 2023-10-16T11:22:50.310 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:22:50.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:50.312 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T11:22:50.402 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-16T11:22:50.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:50.406 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-16T11:22:50.460 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-16T11:22:50.460 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T11:22:50.460 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T11:22:50.461 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T11:22:50.461 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T11:22:50.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:50.462 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T11:22:50.462 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T11:22:50.463 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T11:22:50.463 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T11:22:50.463 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T11:22:50.464 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T11:22:50.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:50.648 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T11:22:50.699 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T11:22:50.699 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T11:22:50.701 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:22:50.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:50.702 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-16T11:22:50.798 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:22:50.798 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T11:22:50.850 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:22:50.850 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:22:50.850 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:22:50.902 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-16T11:22:50.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:50.903 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:22:51.042 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:22:51.043 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T11:22:51.075 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:22:51.076 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T11:22:51.076 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T11:22:51.076 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with new device syntax: 0@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T11:22:51.076 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T11:22:51.078 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-16T11:22:51.078 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2023-10-16T11:22:51.078 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T11:22:51.079 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T11:22:51.079 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T11:22:51.082 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T11:22:51.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:51.083 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:22:51.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:51.168 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T11:22:51.232 INFO:teuthology.orchestra.run.smithi084.stdout:client6611 2023-10-16T11:22:51.232 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:22:51.233 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:51.285 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T11:22:51.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.492+0000 7fb4a3cc9640 1 Processor -- start 2023-10-16T11:22:51.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.492+0000 7fb4a3cc9640 1 -- start start 2023-10-16T11:22:51.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.494+0000 7fb4a3cc9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb49c15ad70 0x7fb49c069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:51.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.494+0000 7fb4a3cc9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb49c0c9ca0 0x7fb49c06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:51.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.494+0000 7fb4a3cc9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb49c0ca3b0 0x7fb49c0ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:51.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.494+0000 7fb4a3cc9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb49c0cad80 con 0x7fb49c0c9ca0 2023-10-16T11:22:51.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.494+0000 7fb4a3cc9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb49c0cb030 con 0x7fb49c0ca3b0 2023-10-16T11:22:51.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.494+0000 7fb4a3cc9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c0c91c0 con 0x7fb49c15ad70 2023-10-16T11:22:51.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.494+0000 7fb4a1264640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb49c0c9ca0 0x7fb49c06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:51.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.494+0000 7fb4a1264640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb49c0c9ca0 0x7fb49c06a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38584/0 (socket says 172.21.15.84:38584) 2023-10-16T11:22:51.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.494+0000 7fb4a1264640 1 -- 172.21.15.84:0/2460067106 learned_addr learned my addr 172.21.15.84:0/2460067106 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:51.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.494+0000 7fb4a1264640 1 -- 172.21.15.84:0/2460067106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb49c15ad70 msgr2=0x7fb49c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:51.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.494+0000 7fb4a2266640 1 --2- 172.21.15.84:0/2460067106 >> v2:172.21.15.157:3300/0 conn(0x7fb49c0ca3b0 0x7fb49c0ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:51.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.494+0000 7fb4a1a65640 1 --2- 172.21.15.84:0/2460067106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb49c15ad70 0x7fb49c069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:51.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.495+0000 7fb4a1264640 1 --2- 172.21.15.84:0/2460067106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb49c15ad70 0x7fb49c069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:51.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.495+0000 7fb4a1264640 1 -- 172.21.15.84:0/2460067106 >> v2:172.21.15.157:3300/0 conn(0x7fb49c0ca3b0 msgr2=0x7fb49c0ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:51.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.495+0000 7fb4a1264640 1 --2- 172.21.15.84:0/2460067106 >> v2:172.21.15.157:3300/0 conn(0x7fb49c0ca3b0 0x7fb49c0ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:51.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.495+0000 7fb4a1264640 1 -- 172.21.15.84:0/2460067106 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c0c94a0 con 0x7fb49c0c9ca0 2023-10-16T11:22:51.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.495+0000 7fb4a2266640 1 --2- 172.21.15.84:0/2460067106 >> v2:172.21.15.157:3300/0 conn(0x7fb49c0ca3b0 0x7fb49c0ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:51.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.496+0000 7fb4a1264640 1 --2- 172.21.15.84:0/2460067106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb49c0c9ca0 0x7fb49c06a2e0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7fb49c0cf820 tx=0x7fb48c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608aa7595bb895ff server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:51.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.496+0000 7fb4a0a63640 1 -- 172.21.15.84:0/2460067106 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c031860 con 0x7fb49c0c9ca0 2023-10-16T11:22:51.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.496+0000 7fb4a0a63640 1 -- 172.21.15.84:0/2460067106 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48c0319c0 con 0x7fb49c0c9ca0 2023-10-16T11:22:51.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.496+0000 7fb4a0a63640 1 -- 172.21.15.84:0/2460067106 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c031d50 con 0x7fb49c0c9ca0 2023-10-16T11:22:51.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.497+0000 7fb4a3cc9640 1 -- 172.21.15.84:0/2460067106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb49c0c9ca0 msgr2=0x7fb49c06a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:51.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.497+0000 7fb4a3cc9640 1 --2- 172.21.15.84:0/2460067106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb49c0c9ca0 0x7fb49c06a2e0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7fb49c0cf820 tx=0x7fb48c02b900 comp rx=0 tx=0).stop 2023-10-16T11:22:51.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.497+0000 7fb4a3cc9640 1 -- 172.21.15.84:0/2460067106 shutdown_connections 2023-10-16T11:22:51.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.497+0000 7fb4a3cc9640 1 --2- 172.21.15.84:0/2460067106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb49c0c9ca0 0x7fb49c06a2e0 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:51.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.497+0000 7fb4a3cc9640 1 --2- 172.21.15.84:0/2460067106 >> v2:172.21.15.157:3300/0 conn(0x7fb49c0ca3b0 0x7fb49c0ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:51.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.497+0000 7fb4a3cc9640 1 --2- 172.21.15.84:0/2460067106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb49c15ad70 0x7fb49c069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:51.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.497+0000 7fb4a3cc9640 1 -- 172.21.15.84:0/2460067106 >> 172.21.15.84:0/2460067106 conn(0x7fb49c0ce9b0 msgr2=0x7fb49c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:51.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.497+0000 7fb4a3cc9640 1 -- 172.21.15.84:0/2460067106 shutdown_connections 2023-10-16T11:22:51.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.497+0000 7fb4a3cc9640 1 -- 172.21.15.84:0/2460067106 wait complete. 2023-10-16T11:22:51.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.499+0000 7fb4a3cc9640 1 Processor -- start 2023-10-16T11:22:51.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.499+0000 7fb4a3cc9640 1 -- start start 2023-10-16T11:22:51.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.500+0000 7fb4a3cc9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb49c0c9ca0 0x7fb49c0c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:51.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.500+0000 7fb4a3cc9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb49c0ca3b0 0x7fb49c0b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:51.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.500+0000 7fb4a3cc9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb49c15ad70 0x7fb49c15da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:51.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.500+0000 7fb4a3cc9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c0b09c0 con 0x7fb49c0ca3b0 2023-10-16T11:22:51.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.500+0000 7fb4a3cc9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c0b0bf0 con 0x7fb49c15ad70 2023-10-16T11:22:51.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.500+0000 7fb4a3cc9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c0b0ed0 con 0x7fb49c0c9ca0 2023-10-16T11:22:51.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.501+0000 7fb4a1264640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb49c0ca3b0 0x7fb49c0b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:51.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.501+0000 7fb4a1264640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb49c0ca3b0 0x7fb49c0b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38586/0 (socket says 172.21.15.84:38586) 2023-10-16T11:22:51.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.501+0000 7fb4a1264640 1 -- 172.21.15.84:0/4259999812 learned_addr learned my addr 172.21.15.84:0/4259999812 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:51.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.501+0000 7fb4a1a65640 1 --2- 172.21.15.84:0/4259999812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb49c0c9ca0 0x7fb49c0c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:51.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.501+0000 7fb4a2266640 1 --2- 172.21.15.84:0/4259999812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb49c15ad70 0x7fb49c15da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:51.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.501+0000 7fb4a1264640 1 -- 172.21.15.84:0/4259999812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb49c0c9ca0 msgr2=0x7fb49c0c38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:51.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.501+0000 7fb4a1264640 1 --2- 172.21.15.84:0/4259999812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb49c0c9ca0 0x7fb49c0c38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:51.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.501+0000 7fb4a1264640 1 -- 172.21.15.84:0/4259999812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb49c15ad70 msgr2=0x7fb49c15da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:51.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.501+0000 7fb4a1264640 1 --2- 172.21.15.84:0/4259999812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb49c15ad70 0x7fb49c15da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:51.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.501+0000 7fb4a1264640 1 -- 172.21.15.84:0/4259999812 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c0094d0 con 0x7fb49c0ca3b0 2023-10-16T11:22:51.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.501+0000 7fb4a1264640 1 --2- 172.21.15.84:0/4259999812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb49c0ca3b0 0x7fb49c0b0480 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7fb48c000c00 tx=0x7fb48c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:51.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.502+0000 7fb4927fc640 1 -- 172.21.15.84:0/4259999812 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c031830 con 0x7fb49c0ca3b0 2023-10-16T11:22:51.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.502+0000 7fb4927fc640 1 -- 172.21.15.84:0/4259999812 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48c0443f0 con 0x7fb49c0ca3b0 2023-10-16T11:22:51.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.502+0000 7fb4927fc640 1 -- 172.21.15.84:0/4259999812 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c0447a0 con 0x7fb49c0ca3b0 2023-10-16T11:22:51.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.502+0000 7fb4a3cc9640 1 -- 172.21.15.84:0/4259999812 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb49c0b11b0 con 0x7fb49c0ca3b0 2023-10-16T11:22:51.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.502+0000 7fb4a3cc9640 1 -- 172.21.15.84:0/4259999812 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49c0b1490 con 0x7fb49c0ca3b0 2023-10-16T11:22:51.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.503+0000 7fb4a3cc9640 1 -- 172.21.15.84:0/4259999812 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49c0b1bb0 con 0x7fb49c0ca3b0 2023-10-16T11:22:51.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.504+0000 7fb4927fc640 1 -- 172.21.15.84:0/4259999812 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb48c04d450 con 0x7fb49c0ca3b0 2023-10-16T11:22:51.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.505+0000 7fb4927fc640 1 --2- 172.21.15.84:0/4259999812 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fb48006af90 0x7fb48006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:51.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.505+0000 7fb4927fc640 1 -- 172.21.15.84:0/4259999812 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb48c0ac110 con 0x7fb49c0ca3b0 2023-10-16T11:22:51.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.505+0000 7fb4a1a65640 1 --2- 172.21.15.84:0/4259999812 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fb48006af90 0x7fb48006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:51.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.506+0000 7fb4a1a65640 1 --2- 172.21.15.84:0/4259999812 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fb48006af90 0x7fb48006d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb498006fd0 tx=0x7fb498008040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:51.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.510+0000 7fb4927fc640 1 -- 172.21.15.84:0/4259999812 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb49c0b1bb0 con 0x7fb49c0ca3b0 2023-10-16T11:22:51.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.726+0000 7fb4a3cc9640 1 -- 172.21.15.84:0/4259999812 --> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb49c0c9780 con 0x7fb48006af90 2023-10-16T11:22:51.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.728+0000 7fb4927fc640 1 -- 172.21.15.84:0/4259999812 <== mgr.10557 v2:172.21.15.84:6832/1985463812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb49800c050 con 0x7fb48006af90 2023-10-16T11:22:51.730 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T11:22:51.730 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:22:51.730 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephfs" 2023-10-16T11:22:51.730 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:22:51.731 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T11:22:51.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.730+0000 7fb4a3cc9640 1 -- 172.21.15.84:0/4259999812 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fb48006af90 msgr2=0x7fb48006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:51.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.730+0000 7fb4a3cc9640 1 --2- 172.21.15.84:0/4259999812 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fb48006af90 0x7fb48006d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb498006fd0 tx=0x7fb498008040 comp rx=0 tx=0).stop 2023-10-16T11:22:51.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.730+0000 7fb4a3cc9640 1 -- 172.21.15.84:0/4259999812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb49c0ca3b0 msgr2=0x7fb49c0b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:51.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.730+0000 7fb4a3cc9640 1 --2- 172.21.15.84:0/4259999812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb49c0ca3b0 0x7fb49c0b0480 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7fb48c000c00 tx=0x7fb48c0046c0 comp rx=0 tx=0).stop 2023-10-16T11:22:51.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.730+0000 7fb4a3cc9640 1 -- 172.21.15.84:0/4259999812 shutdown_connections 2023-10-16T11:22:51.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.730+0000 7fb4a3cc9640 1 --2- 172.21.15.84:0/4259999812 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fb48006af90 0x7fb48006d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:51.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.730+0000 7fb4a3cc9640 1 --2- 172.21.15.84:0/4259999812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb49c15ad70 0x7fb49c15da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:51.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.730+0000 7fb4a3cc9640 1 --2- 172.21.15.84:0/4259999812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb49c0ca3b0 0x7fb49c0b0480 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:51.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.730+0000 7fb4a3cc9640 1 --2- 172.21.15.84:0/4259999812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb49c0c9ca0 0x7fb49c0c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:51.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.730+0000 7fb4a3cc9640 1 -- 172.21.15.84:0/4259999812 >> 172.21.15.84:0/4259999812 conn(0x7fb49c0ce9b0 msgr2=0x7fb49c069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:51.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.731+0000 7fb4a3cc9640 1 -- 172.21.15.84:0/4259999812 shutdown_connections 2023-10-16T11:22:51.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.731+0000 7fb4a3cc9640 1 -- 172.21.15.84:0/4259999812 wait complete. 2023-10-16T11:22:51.752 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T11:22:51.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.968+0000 7fb47dbc0640 1 Processor -- start 2023-10-16T11:22:51.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.968+0000 7fb47dbc0640 1 -- start start 2023-10-16T11:22:51.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.970+0000 7fb47dbc0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb47815ad70 0x7fb478069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:51.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.970+0000 7fb47dbc0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb4780c9ca0 0x7fb47806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:51.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.970+0000 7fb47dbc0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4780cb030 con 0x7fb4780ca3b0 2023-10-16T11:22:51.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.970+0000 7fb47dbc0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4780c91c0 con 0x7fb4780c9ca0 2023-10-16T11:22:51.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.970+0000 7fb47dbc0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780c94a0 con 0x7fb47815ad70 2023-10-16T11:22:51.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.970+0000 7fb46ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb4780c9ca0 0x7fb47806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:51.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.970+0000 7fb4777fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb47815ad70 0x7fb478069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:51.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.971+0000 7fb46ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb4780c9ca0 0x7fb47806a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:57706/0 (socket says 172.21.15.84:57706) 2023-10-16T11:22:51.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.971+0000 7fb46ffff640 1 -- 172.21.15.84:0/3480469117 learned_addr learned my addr 172.21.15.84:0/3480469117 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:51.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.971+0000 7fb46ffff640 1 -- 172.21.15.84:0/3480469117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb47815ad70 msgr2=0x7fb478069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:51.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.971+0000 7fb46ffff640 1 --2- 172.21.15.84:0/3480469117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb47815ad70 0x7fb478069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:51.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.971+0000 7fb46ffff640 1 -- 172.21.15.84:0/3480469117 >> v1:172.21.15.157:6789/0 conn(0x7fb4780ca3b0 legacy=0x7fb4780ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:51.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.971+0000 7fb46ffff640 1 -- 172.21.15.84:0/3480469117 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4780c9780 con 0x7fb4780c9ca0 2023-10-16T11:22:51.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.973+0000 7fb46ffff640 1 --2- 172.21.15.84:0/3480469117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4780c9ca0 0x7fb47806a2e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb460009870 tx=0x7fb46002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c59033a8e6218358 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:51.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.973+0000 7fb476ffd640 1 -- 172.21.15.84:0/3480469117 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460030840 con 0x7fb4780c9ca0 2023-10-16T11:22:51.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.973+0000 7fb476ffd640 1 -- 172.21.15.84:0/3480469117 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4600309a0 con 0x7fb4780c9ca0 2023-10-16T11:22:51.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.973+0000 7fb476ffd640 1 -- 172.21.15.84:0/3480469117 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460030d30 con 0x7fb4780c9ca0 2023-10-16T11:22:51.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.974+0000 7fb47dbc0640 1 -- 172.21.15.84:0/3480469117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4780c9ca0 msgr2=0x7fb47806a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:51.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.974+0000 7fb47dbc0640 1 --2- 172.21.15.84:0/3480469117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4780c9ca0 0x7fb47806a2e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb460009870 tx=0x7fb46002b980 comp rx=0 tx=0).stop 2023-10-16T11:22:51.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.974+0000 7fb47dbc0640 1 -- 172.21.15.84:0/3480469117 shutdown_connections 2023-10-16T11:22:51.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.974+0000 7fb47dbc0640 1 --2- 172.21.15.84:0/3480469117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4780c9ca0 0x7fb47806a2e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:51.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.974+0000 7fb47dbc0640 1 --2- 172.21.15.84:0/3480469117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb47815ad70 0x7fb478069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:51.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.974+0000 7fb47dbc0640 1 -- 172.21.15.84:0/3480469117 >> 172.21.15.84:0/3480469117 conn(0x7fb4780ce9b0 msgr2=0x7fb4780cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:51.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.974+0000 7fb47dbc0640 1 -- 172.21.15.84:0/3480469117 shutdown_connections 2023-10-16T11:22:51.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.974+0000 7fb47dbc0640 1 -- 172.21.15.84:0/3480469117 wait complete. 2023-10-16T11:22:51.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.977+0000 7fb47dbc0640 1 Processor -- start 2023-10-16T11:22:51.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.977+0000 7fb47dbc0640 1 -- start start 2023-10-16T11:22:51.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.978+0000 7fb47dbc0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4780c9ca0 0x7fb4780c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:51.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.978+0000 7fb47dbc0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4780ca3b0 0x7fb4780b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:51.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.978+0000 7fb47dbc0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb47815ad70 0x7fb4781f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:51.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.978+0000 7fb47dbc0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780b2ea0 con 0x7fb47815ad70 2023-10-16T11:22:51.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.978+0000 7fb47dbc0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780b2840 con 0x7fb4780c9ca0 2023-10-16T11:22:51.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.978+0000 7fb47dbc0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780c6c70 con 0x7fb4780ca3b0 2023-10-16T11:22:51.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.978+0000 7fb477fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb47815ad70 0x7fb4781f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:51.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.978+0000 7fb477fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb47815ad70 0x7fb4781f59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38592/0 (socket says 172.21.15.84:38592) 2023-10-16T11:22:51.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.978+0000 7fb477fff640 1 -- 172.21.15.84:0/2591084984 learned_addr learned my addr 172.21.15.84:0/2591084984 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:51.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.979+0000 7fb4777fe640 1 --2- 172.21.15.84:0/2591084984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4780c9ca0 0x7fb4780c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:51.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.979+0000 7fb477fff640 1 -- 172.21.15.84:0/2591084984 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4780ca3b0 msgr2=0x7fb4780b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:51.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.979+0000 7fb477fff640 1 --2- 172.21.15.84:0/2591084984 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4780ca3b0 0x7fb4780b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:51.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.979+0000 7fb477fff640 1 -- 172.21.15.84:0/2591084984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4780c9ca0 msgr2=0x7fb4780c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:51.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.979+0000 7fb477fff640 1 --2- 172.21.15.84:0/2591084984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4780c9ca0 0x7fb4780c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:51.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.979+0000 7fb477fff640 1 -- 172.21.15.84:0/2591084984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4600094d0 con 0x7fb47815ad70 2023-10-16T11:22:51.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.979+0000 7fb4777fe640 1 --2- 172.21.15.84:0/2591084984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4780c9ca0 0x7fb4780c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:51.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.979+0000 7fb477fff640 1 --2- 172.21.15.84:0/2591084984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb47815ad70 0x7fb4781f59b0 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7fb47000c7a0 tx=0x7fb47000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:51.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.979+0000 7fb474ff9640 1 -- 172.21.15.84:0/2591084984 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb470003b70 con 0x7fb47815ad70 2023-10-16T11:22:51.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.979+0000 7fb474ff9640 1 -- 172.21.15.84:0/2591084984 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb470003cf0 con 0x7fb47815ad70 2023-10-16T11:22:51.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.979+0000 7fb47dbc0640 1 -- 172.21.15.84:0/2591084984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4780c6060 con 0x7fb47815ad70 2023-10-16T11:22:51.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.979+0000 7fb474ff9640 1 -- 172.21.15.84:0/2591084984 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47000aa90 con 0x7fb47815ad70 2023-10-16T11:22:51.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.980+0000 7fb47dbc0640 1 -- 172.21.15.84:0/2591084984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4780c6620 con 0x7fb47815ad70 2023-10-16T11:22:51.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.981+0000 7fb47dbc0640 1 -- 172.21.15.84:0/2591084984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4780c6de0 con 0x7fb47815ad70 2023-10-16T11:22:51.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.982+0000 7fb474ff9640 1 -- 172.21.15.84:0/2591084984 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 122) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb47000abf0 con 0x7fb47815ad70 2023-10-16T11:22:51.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.983+0000 7fb474ff9640 1 --2- 172.21.15.84:0/2591084984 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fb45806af90 0x7fb45806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:51.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.983+0000 7fb474ff9640 1 -- 172.21.15.84:0/2591084984 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb4700896b0 con 0x7fb47815ad70 2023-10-16T11:22:51.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.983+0000 7fb4777fe640 1 --2- 172.21.15.84:0/2591084984 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fb45806af90 0x7fb45806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:51.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.984+0000 7fb4777fe640 1 --2- 172.21.15.84:0/2591084984 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fb45806af90 0x7fb45806d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb47806cef0 tx=0x7fb468008040 comp rx=0 tx=0).ready entity=mgr.10557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:51.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:51.987+0000 7fb474ff9640 1 -- 172.21.15.84:0/2591084984 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb470054590 con 0x7fb47815ad70 2023-10-16T11:22:52.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:52.231+0000 7fb47dbc0640 1 -- 172.21.15.84:0/2591084984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fb47804e160 con 0x7fb47815ad70 2023-10-16T11:22:52.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:52.797+0000 7fb474ff9640 1 -- 172.21.15.84:0/2591084984 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v123) v1 ==== 92+0+0 (secure 0 0 0) 0x7fb470003e60 con 0x7fb47815ad70 2023-10-16T11:22:52.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:52.801+0000 7fb47dbc0640 1 -- 172.21.15.84:0/2591084984 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fb45806af90 msgr2=0x7fb45806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:52.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:52.801+0000 7fb47dbc0640 1 --2- 172.21.15.84:0/2591084984 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fb45806af90 0x7fb45806d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb47806cef0 tx=0x7fb468008040 comp rx=0 tx=0).stop 2023-10-16T11:22:52.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:52.801+0000 7fb47dbc0640 1 -- 172.21.15.84:0/2591084984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb47815ad70 msgr2=0x7fb4781f59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:52.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:52.801+0000 7fb47dbc0640 1 --2- 172.21.15.84:0/2591084984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb47815ad70 0x7fb4781f59b0 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7fb47000c7a0 tx=0x7fb47000cc60 comp rx=0 tx=0).stop 2023-10-16T11:22:52.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:52.801+0000 7fb47dbc0640 1 -- 172.21.15.84:0/2591084984 shutdown_connections 2023-10-16T11:22:52.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:52.801+0000 7fb47dbc0640 1 --2- 172.21.15.84:0/2591084984 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fb45806af90 0x7fb45806d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:52.801+0000 7fb47dbc0640 1 --2- 172.21.15.84:0/2591084984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb47815ad70 0x7fb4781f59b0 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:52.801+0000 7fb47dbc0640 1 --2- 172.21.15.84:0/2591084984 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4780ca3b0 0x7fb4780b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:52.801+0000 7fb47dbc0640 1 --2- 172.21.15.84:0/2591084984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4780c9ca0 0x7fb4780c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:52.801+0000 7fb47dbc0640 1 -- 172.21.15.84:0/2591084984 >> 172.21.15.84:0/2591084984 conn(0x7fb4780ce9b0 msgr2=0x7fb4780ad910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:52.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:52.802+0000 7fb47dbc0640 1 -- 172.21.15.84:0/2591084984 shutdown_connections 2023-10-16T11:22:52.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:52.802+0000 7fb47dbc0640 1 -- 172.21.15.84:0/2591084984 wait complete. 2023-10-16T11:22:52.832 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T11:22:53.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.031+0000 7f32d3c09640 1 Processor -- start 2023-10-16T11:22:53.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.032+0000 7f32d3c09640 1 -- start start 2023-10-16T11:22:53.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.033+0000 7f32d3c09640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f32cc0cbf20 0x7f32cc0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.033+0000 7f32d3c09640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32cc15e9b0 0x7f32cc15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.033+0000 7f32d3c09640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32cc06c140 con 0x7f32cc15aee0 2023-10-16T11:22:53.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.033+0000 7f32d3c09640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32cc15b2c0 con 0x7f32cc0cbf20 2023-10-16T11:22:53.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.033+0000 7f32d3c09640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc06b270 con 0x7f32cc15e9b0 2023-10-16T11:22:53.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.033+0000 7f32d11a4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f32cc0cbf20 0x7f32cc0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.033+0000 7f32d11a4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f32cc0cbf20 0x7f32cc0aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38630/0 (socket says 172.21.15.84:38630) 2023-10-16T11:22:53.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.033+0000 7f32d11a4640 1 -- 172.21.15.84:0/1477411517 learned_addr learned my addr 172.21.15.84:0/1477411517 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:53.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.034+0000 7f32d11a4640 1 -- 172.21.15.84:0/1477411517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32cc15e9b0 msgr2=0x7f32cc15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:53.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.034+0000 7f32d11a4640 1 --2- 172.21.15.84:0/1477411517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32cc15e9b0 0x7f32cc15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.034+0000 7f32d11a4640 1 -- 172.21.15.84:0/1477411517 >> v1:172.21.15.157:6789/0 conn(0x7f32cc15aee0 legacy=0x7f32cc0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:53.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.034+0000 7f32d11a4640 1 -- 172.21.15.84:0/1477411517 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32cc06daf0 con 0x7f32cc0cbf20 2023-10-16T11:22:53.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.035+0000 7f32d11a4640 1 --2- 172.21.15.84:0/1477411517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32cc0cbf20 0x7f32cc0aeef0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009870 tx=0x7f32bc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6864860c1da896cb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:53.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.035+0000 7f32d09a3640 1 -- 172.21.15.84:0/1477411517 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc0317c0 con 0x7f32cc0cbf20 2023-10-16T11:22:53.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.035+0000 7f32d09a3640 1 -- 172.21.15.84:0/1477411517 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32bc031920 con 0x7f32cc0cbf20 2023-10-16T11:22:53.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.035+0000 7f32d09a3640 1 -- 172.21.15.84:0/1477411517 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc031cb0 con 0x7f32cc0cbf20 2023-10-16T11:22:53.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.036+0000 7f32d3c09640 1 -- 172.21.15.84:0/1477411517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32cc0cbf20 msgr2=0x7f32cc0aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.036+0000 7f32d3c09640 1 --2- 172.21.15.84:0/1477411517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32cc0cbf20 0x7f32cc0aeef0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009870 tx=0x7f32bc02b860 comp rx=0 tx=0).stop 2023-10-16T11:22:53.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.036+0000 7f32d3c09640 1 -- 172.21.15.84:0/1477411517 shutdown_connections 2023-10-16T11:22:53.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.036+0000 7f32d3c09640 1 --2- 172.21.15.84:0/1477411517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32cc15e9b0 0x7f32cc15ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.036+0000 7f32d3c09640 1 --2- 172.21.15.84:0/1477411517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32cc0cbf20 0x7f32cc0aeef0 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.036+0000 7f32d3c09640 1 -- 172.21.15.84:0/1477411517 >> 172.21.15.84:0/1477411517 conn(0x7f32cc0cfac0 msgr2=0x7f32cc0cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:53.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.036+0000 7f32d3c09640 1 -- 172.21.15.84:0/1477411517 shutdown_connections 2023-10-16T11:22:53.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.036+0000 7f32d3c09640 1 -- 172.21.15.84:0/1477411517 wait complete. 2023-10-16T11:22:53.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.038+0000 7f32d3c09640 1 Processor -- start 2023-10-16T11:22:53.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.038+0000 7f32d3c09640 1 -- start start 2023-10-16T11:22:53.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.039+0000 7f32d3c09640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32cc0cbf20 0x7f32cc15ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.039+0000 7f32d3c09640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32cc15aee0 0x7f32cc14e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.039+0000 7f32d3c09640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32cc15e9b0 0x7f32cc0c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.039+0000 7f32d3c09640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc0c82c0 con 0x7f32cc0cbf20 2023-10-16T11:22:53.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.039+0000 7f32d3c09640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc0c8e40 con 0x7f32cc15aee0 2023-10-16T11:22:53.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.039+0000 7f32d3c09640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc0c8510 con 0x7f32cc15e9b0 2023-10-16T11:22:53.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.039+0000 7f32d19a5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32cc0cbf20 0x7f32cc15ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.039+0000 7f32d19a5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32cc0cbf20 0x7f32cc15ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38640/0 (socket says 172.21.15.84:38640) 2023-10-16T11:22:53.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.039+0000 7f32d19a5640 1 -- 172.21.15.84:0/3734395259 learned_addr learned my addr 172.21.15.84:0/3734395259 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:53.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.039+0000 7f32d11a4640 1 --2- 172.21.15.84:0/3734395259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32cc15aee0 0x7f32cc14e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.039+0000 7f32d21a6640 1 --2- 172.21.15.84:0/3734395259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32cc15e9b0 0x7f32cc0c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.040+0000 7f32d19a5640 1 -- 172.21.15.84:0/3734395259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32cc15e9b0 msgr2=0x7f32cc0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.040+0000 7f32d19a5640 1 --2- 172.21.15.84:0/3734395259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32cc15e9b0 0x7f32cc0c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.040+0000 7f32d19a5640 1 -- 172.21.15.84:0/3734395259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32cc15aee0 msgr2=0x7f32cc14e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.040+0000 7f32d19a5640 1 --2- 172.21.15.84:0/3734395259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32cc15aee0 0x7f32cc14e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.040+0000 7f32d19a5640 1 -- 172.21.15.84:0/3734395259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc0094d0 con 0x7f32cc0cbf20 2023-10-16T11:22:53.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.040+0000 7f32d11a4640 1 --2- 172.21.15.84:0/3734395259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32cc15aee0 0x7f32cc14e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:53.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.040+0000 7f32d19a5640 1 --2- 172.21.15.84:0/3734395259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32cc0cbf20 0x7f32cc15ce60 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f32c800c440 tx=0x7f32c800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:53.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.040+0000 7f32c27fc640 1 -- 172.21.15.84:0/3734395259 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8005480 con 0x7f32cc0cbf20 2023-10-16T11:22:53.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.040+0000 7f32c27fc640 1 -- 172.21.15.84:0/3734395259 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c80035f0 con 0x7f32cc0cbf20 2023-10-16T11:22:53.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.040+0000 7f32d3c09640 1 -- 172.21.15.84:0/3734395259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32cc0c87f0 con 0x7f32cc0cbf20 2023-10-16T11:22:53.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.040+0000 7f32c27fc640 1 -- 172.21.15.84:0/3734395259 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c80039c0 con 0x7f32cc0cbf20 2023-10-16T11:22:53.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.040+0000 7f32d3c09640 1 -- 172.21.15.84:0/3734395259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32cc14ee10 con 0x7f32cc0cbf20 2023-10-16T11:22:53.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.042+0000 7f32d3c09640 1 -- 172.21.15.84:0/3734395259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32cc04e160 con 0x7f32cc0cbf20 2023-10-16T11:22:53.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.042+0000 7f32c27fc640 1 -- 172.21.15.84:0/3734395259 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 123) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f32c8003b20 con 0x7f32cc0cbf20 2023-10-16T11:22:53.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.043+0000 7f32c27fc640 1 --2- 172.21.15.84:0/3734395259 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f32b006afe0 0x7f32b006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.043+0000 7f32c27fc640 1 -- 172.21.15.84:0/3734395259 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f32c8088d30 con 0x7f32cc0cbf20 2023-10-16T11:22:53.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.043+0000 7f32d11a4640 1 -- 172.21.15.84:0/3734395259 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f32b006afe0 msgr2=0x7f32b006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1985463812 2023-10-16T11:22:53.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.044+0000 7f32d11a4640 1 --2- 172.21.15.84:0/3734395259 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f32b006afe0 0x7f32b006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:53.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.048+0000 7f32c27fc640 1 -- 172.21.15.84:0/3734395259 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f32c8053d20 con 0x7f32cc0cbf20 2023-10-16T11:22:53.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.238+0000 7f32d3c09640 1 -- 172.21.15.84:0/3734395259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f32cc06d7b0 con 0x7f32cc0cbf20 2023-10-16T11:22:53.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.245+0000 7f32d11a4640 1 -- 172.21.15.84:0/3734395259 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f32b006afe0 msgr2=0x7f32b006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1985463812 2023-10-16T11:22:53.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.245+0000 7f32d11a4640 1 --2- 172.21.15.84:0/3734395259 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f32b006afe0 0x7f32b006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:53.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.246+0000 7f32c27fc640 1 -- 172.21.15.84:0/3734395259 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v71)=0 v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7f32c8017070 con 0x7f32cc0cbf20 2023-10-16T11:22:53.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.249+0000 7f32d3c09640 1 -- 172.21.15.84:0/3734395259 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f32b006afe0 msgr2=0x7f32b006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:53.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.249+0000 7f32d3c09640 1 --2- 172.21.15.84:0/3734395259 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f32b006afe0 0x7f32b006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.249+0000 7f32d3c09640 1 -- 172.21.15.84:0/3734395259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32cc0cbf20 msgr2=0x7f32cc15ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.249+0000 7f32d3c09640 1 --2- 172.21.15.84:0/3734395259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32cc0cbf20 0x7f32cc15ce60 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f32c800c440 tx=0x7f32c800c900 comp rx=0 tx=0).stop 2023-10-16T11:22:53.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.249+0000 7f32d3c09640 1 -- 172.21.15.84:0/3734395259 shutdown_connections 2023-10-16T11:22:53.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.249+0000 7f32d3c09640 1 --2- 172.21.15.84:0/3734395259 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7f32b006afe0 0x7f32b006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.249+0000 7f32d3c09640 1 --2- 172.21.15.84:0/3734395259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32cc15e9b0 0x7f32cc0c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.249+0000 7f32d3c09640 1 --2- 172.21.15.84:0/3734395259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32cc15aee0 0x7f32cc14e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.249+0000 7f32d3c09640 1 --2- 172.21.15.84:0/3734395259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32cc0cbf20 0x7f32cc15ce60 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.249+0000 7f32d3c09640 1 -- 172.21.15.84:0/3734395259 >> 172.21.15.84:0/3734395259 conn(0x7f32cc0cfac0 msgr2=0x7f32cc06d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:53.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.249+0000 7f32d3c09640 1 -- 172.21.15.84:0/3734395259 shutdown_connections 2023-10-16T11:22:53.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.249+0000 7f32d3c09640 1 -- 172.21.15.84:0/3734395259 wait complete. 2023-10-16T11:22:53.269 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T11:22:53.333 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:53.332+0000 7f52f2d7b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:22:53.338 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:53.336+0000 7f723d02b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:22:53.388 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:53.386+0000 7f52f2d7b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:22:53.394 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:53.392+0000 7f723d02b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:22:53.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.466+0000 7fca5759e640 1 Processor -- start 2023-10-16T11:22:53.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.467+0000 7fca5759e640 1 -- start start 2023-10-16T11:22:53.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.468+0000 7fca5759e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca5806d220 0x7fca580c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.468+0000 7fca5759e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca580acd70 con 0x7fca580ce280 2023-10-16T11:22:53.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.468+0000 7fca5759e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca580ac4a0 con 0x7fca58065ab0 2023-10-16T11:22:53.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.468+0000 7fca5759e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca58065f40 con 0x7fca5806d220 2023-10-16T11:22:53.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.468+0000 7fca56d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fca580ce280 0x7fca580ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54946/0 (socket says 172.21.15.84:54946) 2023-10-16T11:22:53.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.468+0000 7fca56d9d640 1 -- 172.21.15.84:0/762811786 learned_addr learned my addr 172.21.15.84:0/762811786 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:53.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.469+0000 7fca5659c640 1 --2- 172.21.15.84:0/762811786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca5806d220 0x7fca580c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.469+0000 7fca55d9b640 1 -- 172.21.15.84:0/762811786 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659432837 0 0) 0x7fca580acd70 con 0x7fca580ce280 2023-10-16T11:22:53.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.469+0000 7fca55d9b640 1 -- 172.21.15.84:0/762811786 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca44003520 con 0x7fca580ce280 2023-10-16T11:22:53.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.469+0000 7fca55d9b640 1 -- 172.21.15.84:0/762811786 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 534437771 0 0) 0x7fca44003520 con 0x7fca580ce280 2023-10-16T11:22:53.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.469+0000 7fca55d9b640 1 -- 172.21.15.84:0/762811786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca5806d220 msgr2=0x7fca580c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:53.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.469+0000 7fca55d9b640 1 --2- 172.21.15.84:0/762811786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca5806d220 0x7fca580c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.469+0000 7fca55d9b640 1 -- 172.21.15.84:0/762811786 >> v1:172.21.15.157:6789/0 conn(0x7fca58065ab0 legacy=0x7fca580cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.469+0000 7fca55d9b640 1 -- 172.21.15.84:0/762811786 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca580ac720 con 0x7fca580ce280 2023-10-16T11:22:53.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.469+0000 7fca55d9b640 1 -- 172.21.15.84:0/762811786 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fca500019f0 con 0x7fca580ce280 2023-10-16T11:22:53.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.469+0000 7fca55d9b640 1 -- 172.21.15.84:0/762811786 >> v1:172.21.15.84:6789/0 conn(0x7fca580ce280 legacy=0x7fca580ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.470+0000 7fca55d9b640 1 --2- 172.21.15.84:0/762811786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca440039a0 0x7fca44003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.470+0000 7fca55d9b640 1 --2- 172.21.15.84:0/762811786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca5806d220 0x7fca440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.470+0000 7fca55d9b640 1 --2- 172.21.15.84:0/762811786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca44004d00 0x7fca44005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.470+0000 7fca55d9b640 1 -- 172.21.15.84:0/762811786 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca58065f40 con 0x7fca440039a0 2023-10-16T11:22:53.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.470+0000 7fca55d9b640 1 -- 172.21.15.84:0/762811786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4400ba30 con 0x7fca44004d00 2023-10-16T11:22:53.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.470+0000 7fca55d9b640 1 -- 172.21.15.84:0/762811786 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4400bbf0 con 0x7fca5806d220 2023-10-16T11:22:53.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.470+0000 7fca5659c640 1 --2- 172.21.15.84:0/762811786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca440039a0 0x7fca44003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.470+0000 7fca4ffff640 1 --2- 172.21.15.84:0/762811786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca5806d220 0x7fca440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.470+0000 7fca5659c640 1 -- 172.21.15.84:0/762811786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca5806d220 msgr2=0x7fca440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:53.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.470+0000 7fca5659c640 1 --2- 172.21.15.84:0/762811786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca5806d220 0x7fca440047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.470+0000 7fca56d9d640 1 --2- 172.21.15.84:0/762811786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca44004d00 0x7fca44005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.470+0000 7fca5659c640 1 -- 172.21.15.84:0/762811786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca44004d00 msgr2=0x7fca44005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:53.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.470+0000 7fca5659c640 1 --2- 172.21.15.84:0/762811786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca44004d00 0x7fca44005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.470+0000 7fca5659c640 1 -- 172.21.15.84:0/762811786 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca44003520 con 0x7fca440039a0 2023-10-16T11:22:53.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.470+0000 7fca56d9d640 1 --2- 172.21.15.84:0/762811786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca44004d00 0x7fca44005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:53.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.471+0000 7fca5659c640 1 --2- 172.21.15.84:0/762811786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca440039a0 0x7fca44003dc0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7fca48009a70 tx=0x7fca440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbe5fb270005f35 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:53.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.472+0000 7fca55d9b640 1 -- 172.21.15.84:0/762811786 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4802c680 con 0x7fca440039a0 2023-10-16T11:22:53.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.472+0000 7fca55d9b640 1 -- 172.21.15.84:0/762811786 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4802c7e0 con 0x7fca440039a0 2023-10-16T11:22:53.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.472+0000 7fca55d9b640 1 -- 172.21.15.84:0/762811786 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4802cb90 con 0x7fca440039a0 2023-10-16T11:22:53.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.472+0000 7fca5759e640 1 -- 172.21.15.84:0/762811786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca440039a0 msgr2=0x7fca44003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.472+0000 7fca5759e640 1 --2- 172.21.15.84:0/762811786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca440039a0 0x7fca44003dc0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7fca48009a70 tx=0x7fca440057f0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.473+0000 7fca56d9d640 1 -- 172.21.15.84:0/762811786 reap_dead start 2023-10-16T11:22:53.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.473+0000 7fca5759e640 1 -- 172.21.15.84:0/762811786 shutdown_connections 2023-10-16T11:22:53.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.473+0000 7fca5759e640 1 -- 172.21.15.84:0/762811786 >> 172.21.15.84:0/762811786 conn(0x7fca580b0a80 msgr2=0x7fca580b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:53.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.473+0000 7fca5759e640 1 -- 172.21.15.84:0/762811786 shutdown_connections 2023-10-16T11:22:53.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.473+0000 7fca5759e640 1 -- 172.21.15.84:0/762811786 wait complete. 2023-10-16T11:22:53.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.474+0000 7fca5759e640 1 Processor -- start 2023-10-16T11:22:53.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.475+0000 7fca5759e640 1 -- start start 2023-10-16T11:22:53.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.475+0000 7fca5759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca440039a0 0x7fca581567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.476+0000 7fca5759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca581f2e50 0x7fca580b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.476+0000 7fca5759e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca581599a0 0x7fca580afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.476+0000 7fca5759e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca58156cf0 con 0x7fca581f2e50 2023-10-16T11:22:53.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.476+0000 7fca5759e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca581f29c0 con 0x7fca440039a0 2023-10-16T11:22:53.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.476+0000 7fca5759e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca581f20b0 con 0x7fca581599a0 2023-10-16T11:22:53.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.476+0000 7fca4ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca581f2e50 0x7fca580b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.476+0000 7fca4ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca581f2e50 0x7fca580b25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38660/0 (socket says 172.21.15.84:38660) 2023-10-16T11:22:53.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.476+0000 7fca4ffff640 1 -- 172.21.15.84:0/3391077299 learned_addr learned my addr 172.21.15.84:0/3391077299 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:53.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.476+0000 7fca5659c640 1 --2- 172.21.15.84:0/3391077299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca440039a0 0x7fca581567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.476+0000 7fca56d9d640 1 --2- 172.21.15.84:0/3391077299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca581599a0 0x7fca580afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.476+0000 7fca4ffff640 1 -- 172.21.15.84:0/3391077299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca581599a0 msgr2=0x7fca580afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.476+0000 7fca4ffff640 1 --2- 172.21.15.84:0/3391077299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca581599a0 0x7fca580afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.476+0000 7fca4ffff640 1 -- 172.21.15.84:0/3391077299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca440039a0 msgr2=0x7fca581567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.476+0000 7fca4ffff640 1 --2- 172.21.15.84:0/3391077299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca440039a0 0x7fca581567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.476+0000 7fca4ffff640 1 -- 172.21.15.84:0/3391077299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca58159f40 con 0x7fca581f2e50 2023-10-16T11:22:53.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.477+0000 7fca4ffff640 1 --2- 172.21.15.84:0/3391077299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca581f2e50 0x7fca580b25c0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fca4000e550 tx=0x7fca4000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:53.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.477+0000 7fca4effd640 1 -- 172.21.15.84:0/3391077299 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca400052e0 con 0x7fca581f2e50 2023-10-16T11:22:53.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.477+0000 7fca4effd640 1 -- 172.21.15.84:0/3391077299 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca40005440 con 0x7fca581f2e50 2023-10-16T11:22:53.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.477+0000 7fca5759e640 1 -- 172.21.15.84:0/3391077299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca581f2370 con 0x7fca581f2e50 2023-10-16T11:22:53.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.477+0000 7fca4effd640 1 -- 172.21.15.84:0/3391077299 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40003810 con 0x7fca581f2e50 2023-10-16T11:22:53.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.477+0000 7fca5759e640 1 -- 172.21.15.84:0/3391077299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca580b2b00 con 0x7fca581f2e50 2023-10-16T11:22:53.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.478+0000 7fca5759e640 1 -- 172.21.15.84:0/3391077299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca581f2b70 con 0x7fca581f2e50 2023-10-16T11:22:53.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.480+0000 7fca4effd640 1 -- 172.21.15.84:0/3391077299 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 123) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fca40003970 con 0x7fca581f2e50 2023-10-16T11:22:53.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.481+0000 7fca4effd640 1 --2- 172.21.15.84:0/3391077299 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fca3806afe0 0x7fca3806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.481+0000 7fca4effd640 1 -- 172.21.15.84:0/3391077299 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fca400894d0 con 0x7fca581f2e50 2023-10-16T11:22:53.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.481+0000 7fca5659c640 1 -- 172.21.15.84:0/3391077299 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fca3806afe0 msgr2=0x7fca3806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1985463812 2023-10-16T11:22:53.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.481+0000 7fca5659c640 1 --2- 172.21.15.84:0/3391077299 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fca3806afe0 0x7fca3806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:53.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.486+0000 7fca4effd640 1 -- 172.21.15.84:0/3391077299 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fca581f2b70 con 0x7fca581f2e50 2023-10-16T11:22:53.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.667+0000 7fca5759e640 1 -- 172.21.15.84:0/3391077299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fca581f2b30 con 0x7fca581f2e50 2023-10-16T11:22:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.672+0000 7fca4effd640 1 -- 172.21.15.84:0/3391077299 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v72)=0 v72) v1 ==== 133+0+0 (secure 0 0 0) 0x7fca400544e0 con 0x7fca581f2e50 2023-10-16T11:22:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.674+0000 7fca5759e640 1 -- 172.21.15.84:0/3391077299 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fca3806afe0 msgr2=0x7fca3806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:53.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.674+0000 7fca5759e640 1 --2- 172.21.15.84:0/3391077299 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fca3806afe0 0x7fca3806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.674+0000 7fca5759e640 1 -- 172.21.15.84:0/3391077299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca581f2e50 msgr2=0x7fca580b25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.674+0000 7fca5759e640 1 --2- 172.21.15.84:0/3391077299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca581f2e50 0x7fca580b25c0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fca4000e550 tx=0x7fca4000ea10 comp rx=0 tx=0).stop 2023-10-16T11:22:53.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.674+0000 7fca5759e640 1 -- 172.21.15.84:0/3391077299 shutdown_connections 2023-10-16T11:22:53.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.674+0000 7fca5759e640 1 --2- 172.21.15.84:0/3391077299 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7fca3806afe0 0x7fca3806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.674+0000 7fca5759e640 1 --2- 172.21.15.84:0/3391077299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca581599a0 0x7fca580afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.674+0000 7fca5759e640 1 --2- 172.21.15.84:0/3391077299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca581f2e50 0x7fca580b25c0 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.674+0000 7fca5759e640 1 --2- 172.21.15.84:0/3391077299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca440039a0 0x7fca581567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.674+0000 7fca5759e640 1 -- 172.21.15.84:0/3391077299 >> 172.21.15.84:0/3391077299 conn(0x7fca580b0a80 msgr2=0x7fca580ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.675+0000 7fca5759e640 1 -- 172.21.15.84:0/3391077299 shutdown_connections 2023-10-16T11:22:53.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.675+0000 7fca5759e640 1 -- 172.21.15.84:0/3391077299 wait complete. 2023-10-16T11:22:53.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:53.694 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-16T11:22:53.746 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-16T11:22:53.839 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:53.838+0000 7f52f2d7b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:22:53.858 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:53.856+0000 7f723d02b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:22:53.891 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:53.889+0000 7f52f2d7b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:22:53.910 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:53.908+0000 7f723d02b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:22:53.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.932+0000 7ff607527640 1 Processor -- start 2023-10-16T11:22:53.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.932+0000 7ff607527640 1 -- start start 2023-10-16T11:22:53.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.934+0000 7ff607527640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff6000b8480 0x7ff6000cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.934+0000 7ff607527640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60015d150 con 0x7ff60015cd70 2023-10-16T11:22:53.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.934+0000 7ff607527640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6000d1180 con 0x7ff6000d0be0 2023-10-16T11:22:53.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.934+0000 7ff607527640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6000d1460 con 0x7ff6000b8480 2023-10-16T11:22:53.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.934+0000 7ff606525640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff60015cd70 0x7ff6000d2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54956/0 (socket says 172.21.15.84:54956) 2023-10-16T11:22:53.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.934+0000 7ff606525640 1 -- 172.21.15.84:0/3803687281 learned_addr learned my addr 172.21.15.84:0/3803687281 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:53.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.935+0000 7ff605d24640 1 --2- 172.21.15.84:0/3803687281 >> v2:172.21.15.84:3300/0 conn(0x7ff6000b8480 0x7ff6000cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.935+0000 7ff605523640 1 -- 172.21.15.84:0/3803687281 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145481918 0 0) 0x7ff60015d150 con 0x7ff60015cd70 2023-10-16T11:22:53.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.935+0000 7ff605523640 1 -- 172.21.15.84:0/3803687281 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5ec003520 con 0x7ff60015cd70 2023-10-16T11:22:53.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.935+0000 7ff605d24640 1 -- 172.21.15.84:0/3803687281 >> v1:172.21.15.157:6789/0 conn(0x7ff6000d0be0 legacy=0x7ff6000b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.935+0000 7ff605d24640 1 -- 172.21.15.84:0/3803687281 >> v1:172.21.15.84:6789/0 conn(0x7ff60015cd70 legacy=0x7ff6000d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.935+0000 7ff605d24640 1 -- 172.21.15.84:0/3803687281 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6000d1740 con 0x7ff6000b8480 2023-10-16T11:22:53.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.936+0000 7ff605d24640 1 --2- 172.21.15.84:0/3803687281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6000b8480 0x7ff6000cd7c0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40097a0 tx=0x7ff5f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdeb713c610380fd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:53.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.936+0000 7ff605523640 1 -- 172.21.15.84:0/3803687281 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4031830 con 0x7ff6000b8480 2023-10-16T11:22:53.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.936+0000 7ff605523640 1 -- 172.21.15.84:0/3803687281 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f4031990 con 0x7ff6000b8480 2023-10-16T11:22:53.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.936+0000 7ff605523640 1 -- 172.21.15.84:0/3803687281 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4031d20 con 0x7ff6000b8480 2023-10-16T11:22:53.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.937+0000 7ff607527640 1 -- 172.21.15.84:0/3803687281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6000b8480 msgr2=0x7ff6000cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.937+0000 7ff607527640 1 --2- 172.21.15.84:0/3803687281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6000b8480 0x7ff6000cd7c0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40097a0 tx=0x7ff5f402b8d0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.937+0000 7ff607527640 1 -- 172.21.15.84:0/3803687281 shutdown_connections 2023-10-16T11:22:53.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.937+0000 7ff607527640 1 --2- 172.21.15.84:0/3803687281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6000b8480 0x7ff6000cd7c0 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.937+0000 7ff607527640 1 -- 172.21.15.84:0/3803687281 >> 172.21.15.84:0/3803687281 conn(0x7ff6000cf570 msgr2=0x7ff6000cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:53.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.937+0000 7ff607527640 1 -- 172.21.15.84:0/3803687281 shutdown_connections 2023-10-16T11:22:53.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.937+0000 7ff607527640 1 -- 172.21.15.84:0/3803687281 wait complete. 2023-10-16T11:22:53.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.939+0000 7ff607527640 1 Processor -- start 2023-10-16T11:22:53.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.939+0000 7ff607527640 1 -- start start 2023-10-16T11:22:53.943 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:53.939+0000 7f52f2d7b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:22:53.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.940+0000 7ff607527640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6000b8480 0x7ff6001f3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.940+0000 7ff607527640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6000d0be0 0x7ff6001f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.940+0000 7ff607527640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff60015cd70 0x7ff6001f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.940+0000 7ff607527640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6001f2bf0 con 0x7ff60015cd70 2023-10-16T11:22:53.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.940+0000 7ff607527640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6001f3550 con 0x7ff6000b8480 2023-10-16T11:22:53.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.940+0000 7ff607527640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6001f65c0 con 0x7ff6000d0be0 2023-10-16T11:22:53.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.941+0000 7ff606d26640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff60015cd70 0x7ff6001f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.941+0000 7ff606d26640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff60015cd70 0x7ff6001f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38680/0 (socket says 172.21.15.84:38680) 2023-10-16T11:22:53.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.941+0000 7ff606d26640 1 -- 172.21.15.84:0/2576803500 learned_addr learned my addr 172.21.15.84:0/2576803500 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:53.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.941+0000 7ff606525640 1 --2- 172.21.15.84:0/2576803500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6000b8480 0x7ff6001f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.941+0000 7ff605d24640 1 --2- 172.21.15.84:0/2576803500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6000d0be0 0x7ff6001f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.941+0000 7ff606d26640 1 -- 172.21.15.84:0/2576803500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6000d0be0 msgr2=0x7ff6001f1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.941+0000 7ff606d26640 1 --2- 172.21.15.84:0/2576803500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6000d0be0 0x7ff6001f1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.941+0000 7ff606d26640 1 -- 172.21.15.84:0/2576803500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6000b8480 msgr2=0x7ff6001f3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.941+0000 7ff606d26640 1 --2- 172.21.15.84:0/2576803500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6000b8480 0x7ff6001f3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.941+0000 7ff606d26640 1 -- 172.21.15.84:0/2576803500 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f4009400 con 0x7ff60015cd70 2023-10-16T11:22:53.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.941+0000 7ff606525640 1 --2- 172.21.15.84:0/2576803500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6000b8480 0x7ff6001f3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:53.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.941+0000 7ff606d26640 1 --2- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff60015cd70 0x7ff6001f5b40 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800c730 tx=0x7ff5f800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:53.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.942+0000 7ff5f2ffd640 1 -- 172.21.15.84:0/2576803500 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8003b70 con 0x7ff60015cd70 2023-10-16T11:22:53.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.942+0000 7ff5f2ffd640 1 -- 172.21.15.84:0/2576803500 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f8003cf0 con 0x7ff60015cd70 2023-10-16T11:22:53.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.942+0000 7ff5f2ffd640 1 -- 172.21.15.84:0/2576803500 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8003ea0 con 0x7ff60015cd70 2023-10-16T11:22:53.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.942+0000 7ff607527640 1 -- 172.21.15.84:0/2576803500 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6001f68a0 con 0x7ff60015cd70 2023-10-16T11:22:53.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.942+0000 7ff607527640 1 -- 172.21.15.84:0/2576803500 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6000c9e70 con 0x7ff60015cd70 2023-10-16T11:22:53.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.943+0000 7ff607527640 1 -- 172.21.15.84:0/2576803500 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6001f2d30 con 0x7ff60015cd70 2023-10-16T11:22:53.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.944+0000 7ff5f2ffd640 1 -- 172.21.15.84:0/2576803500 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 123) v1 ==== 89348+0+0 (secure 0 0 0) 0x7ff5f8005520 con 0x7ff60015cd70 2023-10-16T11:22:53.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.945+0000 7ff5f2ffd640 1 --2- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7ff5e406afe0 0x7ff5e406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.945+0000 7ff5f2ffd640 1 -- 172.21.15.84:0/2576803500 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff5f80897d0 con 0x7ff60015cd70 2023-10-16T11:22:53.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.945+0000 7ff606525640 1 -- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7ff5e406afe0 msgr2=0x7ff5e406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1985463812 2023-10-16T11:22:53.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.945+0000 7ff606525640 1 --2- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7ff5e406afe0 0x7ff5e406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:53.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:53.949+0000 7ff5f2ffd640 1 -- 172.21.15.84:0/2576803500 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff6001f2d30 con 0x7ff60015cd70 2023-10-16T11:22:53.964 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:53.963+0000 7f723d02b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:22:54.003 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:54.001+0000 7f52f2d7b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:22:54.025 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:54.023+0000 7f723d02b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:22:54.051 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:54.050+0000 7f52f2d7b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:22:54.076 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:54.074+0000 7f723d02b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:22:54.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:54.146+0000 7ff606525640 1 -- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7ff5e406afe0 msgr2=0x7ff5e406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1985463812 2023-10-16T11:22:54.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:54.146+0000 7ff606525640 1 --2- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7ff5e406afe0 0x7ff5e406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:54.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:54.154+0000 7ff607527640 1 -- 172.21.15.84:0/2576803500 --> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6000d1d00 con 0x7ff5e406afe0 2023-10-16T11:22:54.162 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:54.160+0000 7f52f2d7b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:22:54.188 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:54.186+0000 7f723d02b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:22:54.224 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:54.222+0000 7f52f2d7b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:22:54.250 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:54.248+0000 7f723d02b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:22:54.273 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:54.271+0000 7f52f2d7b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:22:54.299 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:54.297+0000 7f723d02b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:22:54.331 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:54.329+0000 7f52f2d7b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:22:54.355 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:54.353+0000 7f723d02b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:22:54.381 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:54.379+0000 7f52f2d7b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:22:54.405 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:54.403+0000 7f723d02b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:22:54.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:54.547+0000 7ff606525640 1 -- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7ff5e406afe0 msgr2=0x7ff5e406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1985463812 2023-10-16T11:22:54.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:54.547+0000 7ff606525640 1 --2- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7ff5e406afe0 0x7ff5e406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T11:22:54.763 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:54.761+0000 7f52f2d7b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:22:54.778 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:54.776+0000 7f723d02b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:22:54.924 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:54.922+0000 7f52f2d7b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:22:54.939 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:54.937+0000 7f723d02b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:22:55.032 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:55.030+0000 7f52f2d7b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:22:55.046 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:55.044+0000 7f723d02b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:22:55.082 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:55.080+0000 7f52f2d7b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:22:55.098 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:55.096+0000 7f723d02b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:22:55.189 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:55.187+0000 7f52f2d7b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:22:55.205 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:55.204+0000 7f723d02b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:22:55.298 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:55.296+0000 7f52f2d7b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:22:55.310 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:55.308+0000 7f723d02b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:22:55.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:55.348+0000 7ff606525640 1 -- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7ff5e406afe0 msgr2=0x7ff5e406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1985463812 2023-10-16T11:22:55.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:55.348+0000 7ff606525640 1 --2- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7ff5e406afe0 0x7ff5e406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T11:22:55.456 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:55.454+0000 7f52f2d7b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:22:55.466 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:55.464+0000 7f723d02b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:22:55.504 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:55.503+0000 7f52f2d7b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:22:55.514 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:55.512+0000 7f723d02b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:22:55.624 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:55.621+0000 7f52f2d7b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:22:55.631 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:55.629+0000 7f723d02b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:22:56.440 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:56.438+0000 7f723d02b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:22:56.443 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:56.441+0000 7f52f2d7b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:22:56.730 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:56.728+0000 7f723d02b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:22:56.735 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:56.733+0000 7f52f2d7b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:22:56.787 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:56.785+0000 7f723d02b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:22:56.792 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:56.791+0000 7f52f2d7b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:22:56.898 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:56.896+0000 7f723d02b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:22:56.906 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:22:56.904+0000 7f52f2d7b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:22:56.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:56.915+0000 7ff5f2ffd640 1 -- 172.21.15.84:0/2576803500 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 124) v1 ==== 88957+0+0 (secure 0 0 0) 0x7ff5f8054b10 con 0x7ff60015cd70 2023-10-16T11:22:56.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:56.916+0000 7ff5f2ffd640 1 -- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7ff5e406afe0 msgr2=0x7ff5e406d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:56.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:56.916+0000 7ff5f2ffd640 1 --2- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:6832/1985463812,v1:172.21.15.84:6833/1985463812] conn(0x7ff5e406afe0 0x7ff5e406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:57.195 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:57.193+0000 7f71b93d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:57.195 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:57.193+0000 7f71b93d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:57.196 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:57.193+0000 7f71b93d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:57.196 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:57.193+0000 7f71b93d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:57.196 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:57.193+0000 7f71b93d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:57.196 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:57.194+0000 7f71bbc17640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:57.197 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:57.194+0000 7f71bbc17640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:57.197 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:57.194+0000 7f71bbc17640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:57.197 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:57.194+0000 7f71bbc17640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:57.197 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:22:57.194+0000 7f71bbc17640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:57.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:57.923+0000 7ff5f2ffd640 1 -- 172.21.15.84:0/2576803500 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 125) v1 ==== 89084+0+0 (secure 0 0 0) 0x7ff5f8087c30 con 0x7ff60015cd70 2023-10-16T11:22:57.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:57.924+0000 7ff5f2ffd640 1 --2- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7ff5e40355a0 0x7ff5e4035980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:57.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:57.924+0000 7ff5f2ffd640 1 -- 172.21.15.84:0/2576803500 --> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5e4035ec0 con 0x7ff5e40355a0 2023-10-16T11:22:57.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:57.926+0000 7ff606525640 1 --2- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7ff5e40355a0 0x7ff5e4035980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:57.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:57.935+0000 7ff606525640 1 --2- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7ff5e40355a0 0x7ff5e4035980 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc007fd0 tx=0x7ff5fc009040 comp rx=0 tx=0).ready entity=mgr.10974 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:58.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.116+0000 7ff5f2ffd640 1 -- 172.21.15.84:0/2576803500 <== mgr.10974 v2:172.21.15.84:6832/855077515 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff5e4035ec0 con 0x7ff5e40355a0 2023-10-16T11:22:58.119 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path snap_test_dir1 2023-10-16T11:22:58.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.120+0000 7ff607527640 1 -- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7ff5e40355a0 msgr2=0x7ff5e4035980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:58.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.120+0000 7ff607527640 1 --2- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7ff5e40355a0 0x7ff5e4035980 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc007fd0 tx=0x7ff5fc009040 comp rx=0 tx=0).stop 2023-10-16T11:22:58.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.120+0000 7ff607527640 1 -- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff60015cd70 msgr2=0x7ff6001f5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:58.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.120+0000 7ff607527640 1 --2- 172.21.15.84:0/2576803500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff60015cd70 0x7ff6001f5b40 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800c730 tx=0x7ff5f800cbf0 comp rx=0 tx=0).stop 2023-10-16T11:22:58.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.120+0000 7ff606d26640 1 -- 172.21.15.84:0/2576803500 reap_dead start 2023-10-16T11:22:58.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.120+0000 7ff607527640 1 -- 172.21.15.84:0/2576803500 shutdown_connections 2023-10-16T11:22:58.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.120+0000 7ff607527640 1 -- 172.21.15.84:0/2576803500 >> 172.21.15.84:0/2576803500 conn(0x7ff6000cf570 msgr2=0x7ff6000b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:58.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.120+0000 7ff607527640 1 -- 172.21.15.84:0/2576803500 shutdown_connections 2023-10-16T11:22:58.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.120+0000 7ff607527640 1 -- 172.21.15.84:0/2576803500 wait complete. 2023-10-16T11:22:58.147 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-16T11:22:58.147 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1m 2023-10-16T11:22:58.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.358+0000 7fb8cb501640 1 Processor -- start 2023-10-16T11:22:58.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.359+0000 7fb8cb501640 1 -- start start 2023-10-16T11:22:58.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.360+0000 7fb8cb501640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb8c40c1d30 0x7fb8c4165aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:58.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.360+0000 7fb8cb501640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c40b0360 0x7fb8c40b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:58.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.360+0000 7fb8cb501640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c4165610 con 0x7fb8c40c8fa0 2023-10-16T11:22:58.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.360+0000 7fb8cb501640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c40b0cb0 con 0x7fb8c40c1d30 2023-10-16T11:22:58.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.360+0000 7fb8cb501640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c40ce370 con 0x7fb8c40b0360 2023-10-16T11:22:58.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.361+0000 7fb8c8a9c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb8c40c8fa0 0x7fb8c40cda90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:59682/0 (socket says 172.21.15.84:59682) 2023-10-16T11:22:58.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.361+0000 7fb8c8a9c640 1 -- 172.21.15.84:0/4153599168 learned_addr learned my addr 172.21.15.84:0/4153599168 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:22:58.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.361+0000 7fb8c929d640 1 --2- 172.21.15.84:0/4153599168 >> v2:172.21.15.84:3300/0 conn(0x7fb8c40c1d30 0x7fb8c4165aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:58.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.361+0000 7fb8c9a9e640 1 --2- 172.21.15.84:0/4153599168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c40b0360 0x7fb8c40b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:58.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.361+0000 7fb8c3fff640 1 -- 172.21.15.84:0/4153599168 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323676417 0 0) 0x7fb8c4165610 con 0x7fb8c40c8fa0 2023-10-16T11:22:58.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.361+0000 7fb8c3fff640 1 -- 172.21.15.84:0/4153599168 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8b4003520 con 0x7fb8c40c8fa0 2023-10-16T11:22:58.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.361+0000 7fb8c929d640 1 -- 172.21.15.84:0/4153599168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c40b0360 msgr2=0x7fb8c40b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:58.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.361+0000 7fb8c929d640 1 --2- 172.21.15.84:0/4153599168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c40b0360 0x7fb8c40b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:58.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.361+0000 7fb8c929d640 1 -- 172.21.15.84:0/4153599168 >> v1:172.21.15.84:6789/0 conn(0x7fb8c40c8fa0 legacy=0x7fb8c40cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:58.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.361+0000 7fb8c929d640 1 -- 172.21.15.84:0/4153599168 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c40ce650 con 0x7fb8c40c1d30 2023-10-16T11:22:58.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.362+0000 7fb8c929d640 1 --2- 172.21.15.84:0/4153599168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c40c1d30 0x7fb8c4165aa0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009d90 tx=0x7fb8b802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1f79f2f0814bda server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:58.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.363+0000 7fb8c3fff640 1 -- 172.21.15.84:0/4153599168 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b802cc50 con 0x7fb8c40c1d30 2023-10-16T11:22:58.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.363+0000 7fb8c3fff640 1 -- 172.21.15.84:0/4153599168 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b802cdb0 con 0x7fb8c40c1d30 2023-10-16T11:22:58.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.363+0000 7fb8c3fff640 1 -- 172.21.15.84:0/4153599168 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8034660 con 0x7fb8c40c1d30 2023-10-16T11:22:58.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.363+0000 7fb8cb501640 1 -- 172.21.15.84:0/4153599168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c40c1d30 msgr2=0x7fb8c4165aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:58.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.363+0000 7fb8cb501640 1 --2- 172.21.15.84:0/4153599168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c40c1d30 0x7fb8c4165aa0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009d90 tx=0x7fb8b802bea0 comp rx=0 tx=0).stop 2023-10-16T11:22:58.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.363+0000 7fb8cb501640 1 -- 172.21.15.84:0/4153599168 shutdown_connections 2023-10-16T11:22:58.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.363+0000 7fb8cb501640 1 --2- 172.21.15.84:0/4153599168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c40b0360 0x7fb8c40b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:58.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.363+0000 7fb8cb501640 1 --2- 172.21.15.84:0/4153599168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c40c1d30 0x7fb8c4165aa0 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:58.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.363+0000 7fb8cb501640 1 -- 172.21.15.84:0/4153599168 >> 172.21.15.84:0/4153599168 conn(0x7fb8c40cb360 msgr2=0x7fb8c40cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:58.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.363+0000 7fb8cb501640 1 -- 172.21.15.84:0/4153599168 shutdown_connections 2023-10-16T11:22:58.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.363+0000 7fb8cb501640 1 -- 172.21.15.84:0/4153599168 wait complete. 2023-10-16T11:22:58.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.365+0000 7fb8cb501640 1 Processor -- start 2023-10-16T11:22:58.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.365+0000 7fb8cb501640 1 -- start start 2023-10-16T11:22:58.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.366+0000 7fb8cb501640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8c40b0360 0x7fb8c4163990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:58.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.366+0000 7fb8cb501640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c40c1d30 0x7fb8c415cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:58.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.366+0000 7fb8cb501640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c40c8fa0 0x7fb8c40af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:58.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.366+0000 7fb8cb501640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c40b1970 con 0x7fb8c40c1d30 2023-10-16T11:22:58.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.366+0000 7fb8cb501640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c40b1c50 con 0x7fb8c40b0360 2023-10-16T11:22:58.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.366+0000 7fb8cb501640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c40b1f30 con 0x7fb8c40c8fa0 2023-10-16T11:22:58.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.366+0000 7fb8c8a9c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c40c1d30 0x7fb8c415cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:58.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.366+0000 7fb8c8a9c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c40c1d30 0x7fb8c415cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33800/0 (socket says 172.21.15.84:33800) 2023-10-16T11:22:58.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.366+0000 7fb8c8a9c640 1 -- 172.21.15.84:0/2875149862 learned_addr learned my addr 172.21.15.84:0/2875149862 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:58.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.367+0000 7fb8c929d640 1 --2- 172.21.15.84:0/2875149862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8c40b0360 0x7fb8c4163990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:58.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.367+0000 7fb8c8a9c640 1 -- 172.21.15.84:0/2875149862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c40c8fa0 msgr2=0x7fb8c40af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:58.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.367+0000 7fb8c8a9c640 1 --2- 172.21.15.84:0/2875149862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c40c8fa0 0x7fb8c40af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:58.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.367+0000 7fb8c8a9c640 1 -- 172.21.15.84:0/2875149862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8c40b0360 msgr2=0x7fb8c4163990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:58.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.367+0000 7fb8c8a9c640 1 --2- 172.21.15.84:0/2875149862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8c40b0360 0x7fb8c4163990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:58.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.367+0000 7fb8c8a9c640 1 -- 172.21.15.84:0/2875149862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b8009a70 con 0x7fb8c40c1d30 2023-10-16T11:22:58.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.367+0000 7fb8c8a9c640 1 --2- 172.21.15.84:0/2875149862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c40c1d30 0x7fb8c415cdb0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000c500 tx=0x7fb8b000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:58.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.367+0000 7fb8c1ffb640 1 -- 172.21.15.84:0/2875149862 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0005ce0 con 0x7fb8c40c1d30 2023-10-16T11:22:58.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.367+0000 7fb8c1ffb640 1 -- 172.21.15.84:0/2875149862 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b0005e40 con 0x7fb8c40c1d30 2023-10-16T11:22:58.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.367+0000 7fb8cb501640 1 -- 172.21.15.84:0/2875149862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c40b2210 con 0x7fb8c40c1d30 2023-10-16T11:22:58.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.367+0000 7fb8c1ffb640 1 -- 172.21.15.84:0/2875149862 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0007330 con 0x7fb8c40c1d30 2023-10-16T11:22:58.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.368+0000 7fb8cb501640 1 -- 172.21.15.84:0/2875149862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c415d2f0 con 0x7fb8c40c1d30 2023-10-16T11:22:58.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.369+0000 7fb8cb501640 1 -- 172.21.15.84:0/2875149862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c404e160 con 0x7fb8c40c1d30 2023-10-16T11:22:58.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.370+0000 7fb8c1ffb640 1 -- 172.21.15.84:0/2875149862 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 125) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fb8b0007490 con 0x7fb8c40c1d30 2023-10-16T11:22:58.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.371+0000 7fb8c1ffb640 1 --2- 172.21.15.84:0/2875149862 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fb8a806ad40 0x7fb8a806d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:58.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.371+0000 7fb8c1ffb640 1 -- 172.21.15.84:0/2875149862 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6081+0+0 (secure 0 0 0) 0x7fb8b0088a80 con 0x7fb8c40c1d30 2023-10-16T11:22:58.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.372+0000 7fb8c929d640 1 --2- 172.21.15.84:0/2875149862 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fb8a806ad40 0x7fb8a806d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:58.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.372+0000 7fb8c929d640 1 --2- 172.21.15.84:0/2875149862 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fb8a806ad40 0x7fb8a806d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8036000 tx=0x7fb8b8030000 comp rx=0 tx=0).ready entity=mgr.10974 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:58.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.376+0000 7fb8c1ffb640 1 -- 172.21.15.84:0/2875149862 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb8b0053db0 con 0x7fb8c40c1d30 2023-10-16T11:22:58.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.588+0000 7fb8cb501640 1 -- 172.21.15.84:0/2875149862 --> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8c41652a0 con 0x7fb8a806ad40 2023-10-16T11:22:58.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.744+0000 7fb8c1ffb640 1 -- 172.21.15.84:0/2875149862 <== mgr.10974 v2:172.21.15.84:6832/855077515 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7fb8b80440d0 con 0x7fb8a806ad40 2023-10-16T11:22:58.746 INFO:teuthology.orchestra.run.smithi084.stdout:Retention added to path snap_test_dir1 2023-10-16T11:22:58.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.747+0000 7fb8cb501640 1 -- 172.21.15.84:0/2875149862 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fb8a806ad40 msgr2=0x7fb8a806d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:58.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.747+0000 7fb8cb501640 1 --2- 172.21.15.84:0/2875149862 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fb8a806ad40 0x7fb8a806d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8036000 tx=0x7fb8b8030000 comp rx=0 tx=0).stop 2023-10-16T11:22:58.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.747+0000 7fb8cb501640 1 -- 172.21.15.84:0/2875149862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c40c1d30 msgr2=0x7fb8c415cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:58.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.747+0000 7fb8cb501640 1 --2- 172.21.15.84:0/2875149862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c40c1d30 0x7fb8c415cdb0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000c500 tx=0x7fb8b000c9c0 comp rx=0 tx=0).stop 2023-10-16T11:22:58.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.747+0000 7fb8cb501640 1 -- 172.21.15.84:0/2875149862 shutdown_connections 2023-10-16T11:22:58.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.747+0000 7fb8cb501640 1 --2- 172.21.15.84:0/2875149862 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fb8a806ad40 0x7fb8a806d200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:58.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.747+0000 7fb8cb501640 1 --2- 172.21.15.84:0/2875149862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c40c8fa0 0x7fb8c40af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:58.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.747+0000 7fb8cb501640 1 --2- 172.21.15.84:0/2875149862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c40c1d30 0x7fb8c415cdb0 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:58.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.747+0000 7fb8cb501640 1 --2- 172.21.15.84:0/2875149862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8c40b0360 0x7fb8c4163990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:58.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.747+0000 7fb8cb501640 1 -- 172.21.15.84:0/2875149862 >> 172.21.15.84:0/2875149862 conn(0x7fb8c40cb360 msgr2=0x7fb8c415fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:58.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.747+0000 7fb8cb501640 1 -- 172.21.15.84:0/2875149862 shutdown_connections 2023-10-16T11:22:58.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.747+0000 7fb8cb501640 1 -- 172.21.15.84:0/2875149862 wait complete. 2023-10-16T11:22:58.770 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-10-16T11:22:58.771 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-16-11_23_00 in ~2s... 2023-10-16T11:22:58.771 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [{'m': 1}] 2023-10-16T11:22:58.771 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-16T11:22:58.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.992+0000 7f8d99333640 1 Processor -- start 2023-10-16T11:22:58.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.992+0000 7f8d99333640 1 -- start start 2023-10-16T11:22:58.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.994+0000 7f8d99333640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d94159d40 0x7f8d94071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:58.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.994+0000 7f8d99333640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8d940c9800 0x7f8d9415d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:58.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.994+0000 7f8d99333640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9415a120 con 0x7f8d9406af40 2023-10-16T11:22:58.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.994+0000 7f8d99333640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9415e3b0 con 0x7f8d940c9800 2023-10-16T11:22:58.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.994+0000 7f8d99333640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9415e690 con 0x7f8d94159d40 2023-10-16T11:22:58.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.994+0000 7f8d92ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d94159d40 0x7f8d94071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:58.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.994+0000 7f8d8a7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8d940c9800 0x7f8d9415d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:58.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.994+0000 7f8d92ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d94159d40 0x7f8d94071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:47262/0 (socket says 172.21.15.84:47262) 2023-10-16T11:22:58.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.994+0000 7f8d92ffd640 1 -- 172.21.15.84:0/2401430734 learned_addr learned my addr 172.21.15.84:0/2401430734 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:59.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.995+0000 7f8d92ffd640 1 -- 172.21.15.84:0/2401430734 >> v2:172.21.15.157:3300/0 conn(0x7f8d940c9800 msgr2=0x7f8d9415d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:59.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.995+0000 7f8d92ffd640 1 --2- 172.21.15.84:0/2401430734 >> v2:172.21.15.157:3300/0 conn(0x7f8d940c9800 0x7f8d9415d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:59.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.995+0000 7f8d92ffd640 1 -- 172.21.15.84:0/2401430734 >> v1:172.21.15.157:6789/0 conn(0x7f8d9406af40 legacy=0x7f8d940ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:59.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.995+0000 7f8d92ffd640 1 -- 172.21.15.84:0/2401430734 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d94070f60 con 0x7f8d94159d40 2023-10-16T11:22:59.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.995+0000 7f8d8a7fc640 1 --2- 172.21.15.84:0/2401430734 >> v2:172.21.15.157:3300/0 conn(0x7f8d940c9800 0x7f8d9415d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:59.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.997+0000 7f8d92ffd640 1 --2- 172.21.15.84:0/2401430734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d94159d40 0x7f8d94071a40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009870 tx=0x7f8d8402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6f70471aab11c5b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:59.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.997+0000 7f8d927fc640 1 -- 172.21.15.84:0/2401430734 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d84030840 con 0x7f8d94159d40 2023-10-16T11:22:59.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.997+0000 7f8d927fc640 1 -- 172.21.15.84:0/2401430734 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d840309a0 con 0x7f8d94159d40 2023-10-16T11:22:59.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.997+0000 7f8d927fc640 1 -- 172.21.15.84:0/2401430734 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d84030d30 con 0x7f8d94159d40 2023-10-16T11:22:59.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.998+0000 7f8d99333640 1 -- 172.21.15.84:0/2401430734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d94159d40 msgr2=0x7f8d94071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:59.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.998+0000 7f8d99333640 1 --2- 172.21.15.84:0/2401430734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d94159d40 0x7f8d94071a40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009870 tx=0x7f8d8402b980 comp rx=0 tx=0).stop 2023-10-16T11:22:59.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.998+0000 7f8d99333640 1 -- 172.21.15.84:0/2401430734 shutdown_connections 2023-10-16T11:22:59.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.998+0000 7f8d99333640 1 --2- 172.21.15.84:0/2401430734 >> v2:172.21.15.157:3300/0 conn(0x7f8d940c9800 0x7f8d9415d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:59.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.998+0000 7f8d99333640 1 --2- 172.21.15.84:0/2401430734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d94159d40 0x7f8d94071a40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:59.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.998+0000 7f8d99333640 1 -- 172.21.15.84:0/2401430734 >> 172.21.15.84:0/2401430734 conn(0x7f8d9406bc10 msgr2=0x7f8d9406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:59.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.998+0000 7f8d99333640 1 -- 172.21.15.84:0/2401430734 shutdown_connections 2023-10-16T11:22:59.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:58.998+0000 7f8d99333640 1 -- 172.21.15.84:0/2401430734 wait complete. 2023-10-16T11:22:59.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.000+0000 7f8d99333640 1 Processor -- start 2023-10-16T11:22:59.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.000+0000 7f8d99333640 1 -- start start 2023-10-16T11:22:59.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.001+0000 7f8d99333640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9406af40 0x7f8d940c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:59.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.001+0000 7f8d99333640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d940c9800 0x7f8d94145620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:59.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.001+0000 7f8d99333640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d94159d40 0x7f8d94143590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:59.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.001+0000 7f8d99333640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d940c3280 con 0x7f8d94159d40 2023-10-16T11:22:59.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.001+0000 7f8d99333640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d940ca060 con 0x7f8d9406af40 2023-10-16T11:22:59.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.001+0000 7f8d99333640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d940ca340 con 0x7f8d940c9800 2023-10-16T11:22:59.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.002+0000 7f8d937fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d94159d40 0x7f8d94143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:59.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.002+0000 7f8d937fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d94159d40 0x7f8d94143590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33814/0 (socket says 172.21.15.84:33814) 2023-10-16T11:22:59.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.002+0000 7f8d937fe640 1 -- 172.21.15.84:0/437845899 learned_addr learned my addr 172.21.15.84:0/437845899 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:22:59.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.002+0000 7f8d92ffd640 1 --2- 172.21.15.84:0/437845899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9406af40 0x7f8d940c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:59.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.002+0000 7f8d8a7fc640 1 --2- 172.21.15.84:0/437845899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d940c9800 0x7f8d94145620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:59.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.002+0000 7f8d937fe640 1 -- 172.21.15.84:0/437845899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d940c9800 msgr2=0x7f8d94145620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:59.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.002+0000 7f8d937fe640 1 --2- 172.21.15.84:0/437845899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d940c9800 0x7f8d94145620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:59.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.002+0000 7f8d937fe640 1 -- 172.21.15.84:0/437845899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9406af40 msgr2=0x7f8d940c1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:59.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.002+0000 7f8d937fe640 1 --2- 172.21.15.84:0/437845899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9406af40 0x7f8d940c1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:59.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.002+0000 7f8d937fe640 1 -- 172.21.15.84:0/437845899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7c009400 con 0x7f8d94159d40 2023-10-16T11:22:59.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.002+0000 7f8d8a7fc640 1 --2- 172.21.15.84:0/437845899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d940c9800 0x7f8d94145620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:59.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.003+0000 7f8d937fe640 1 --2- 172.21.15.84:0/437845899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d94159d40 0x7f8d94143590 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00c6b0 tx=0x7f8d8c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:59.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.003+0000 7f8d8bfff640 1 -- 172.21.15.84:0/437845899 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c003b70 con 0x7f8d94159d40 2023-10-16T11:22:59.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.003+0000 7f8d8bfff640 1 -- 172.21.15.84:0/437845899 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8c003cf0 con 0x7f8d94159d40 2023-10-16T11:22:59.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.003+0000 7f8d8bfff640 1 -- 172.21.15.84:0/437845899 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c003ea0 con 0x7f8d94159d40 2023-10-16T11:22:59.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.003+0000 7f8d99333640 1 -- 172.21.15.84:0/437845899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d840094d0 con 0x7f8d94159d40 2023-10-16T11:22:59.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.003+0000 7f8d99333640 1 -- 172.21.15.84:0/437845899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d940ca900 con 0x7f8d94159d40 2023-10-16T11:22:59.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.005+0000 7f8d99333640 1 -- 172.21.15.84:0/437845899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9404e160 con 0x7f8d94159d40 2023-10-16T11:22:59.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.006+0000 7f8d8bfff640 1 -- 172.21.15.84:0/437845899 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 126) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f8d8c005310 con 0x7f8d94159d40 2023-10-16T11:22:59.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.007+0000 7f8d8bfff640 1 --2- 172.21.15.84:0/437845899 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f8d7406afe0 0x7f8d7406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:59.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.007+0000 7f8d8bfff640 1 -- 172.21.15.84:0/437845899 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f8d8c089730 con 0x7f8d94159d40 2023-10-16T11:22:59.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.007+0000 7f8d92ffd640 1 --2- 172.21.15.84:0/437845899 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f8d7406afe0 0x7f8d7406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:59.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.008+0000 7f8d92ffd640 1 --2- 172.21.15.84:0/437845899 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f8d7406afe0 0x7f8d7406d4a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8d84036000 tx=0x7f8d84008030 comp rx=0 tx=0).ready entity=mgr.10974 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:59.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.011+0000 7f8d8bfff640 1 -- 172.21.15.84:0/437845899 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8d8c054540 con 0x7f8d94159d40 2023-10-16T11:22:59.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.234+0000 7f8d99333640 1 -- 172.21.15.84:0/437845899 --> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8d94071520 con 0x7f8d7406afe0 2023-10-16T11:22:59.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.247+0000 7f8d8bfff640 1 -- 172.21.15.84:0/437845899 <== mgr.10974 v2:172.21.15.84:6832/855077515 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f8d84030b90 con 0x7f8d7406afe0 2023-10-16T11:22:59.250 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:22:59.250 INFO:teuthology.orchestra.run.smithi084.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2023-10-16T11:22:59.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.250+0000 7f8d99333640 1 -- 172.21.15.84:0/437845899 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f8d7406afe0 msgr2=0x7f8d7406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:59.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.250+0000 7f8d99333640 1 --2- 172.21.15.84:0/437845899 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f8d7406afe0 0x7f8d7406d4a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8d84036000 tx=0x7f8d84008030 comp rx=0 tx=0).stop 2023-10-16T11:22:59.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.250+0000 7f8d99333640 1 -- 172.21.15.84:0/437845899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d94159d40 msgr2=0x7f8d94143590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:59.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.250+0000 7f8d99333640 1 --2- 172.21.15.84:0/437845899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d94159d40 0x7f8d94143590 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00c6b0 tx=0x7f8d8c00cb70 comp rx=0 tx=0).stop 2023-10-16T11:22:59.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.250+0000 7f8d99333640 1 -- 172.21.15.84:0/437845899 shutdown_connections 2023-10-16T11:22:59.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.250+0000 7f8d99333640 1 --2- 172.21.15.84:0/437845899 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f8d7406afe0 0x7f8d7406d4a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:59.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.250+0000 7f8d99333640 1 --2- 172.21.15.84:0/437845899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d94159d40 0x7f8d94143590 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:59.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.250+0000 7f8d99333640 1 --2- 172.21.15.84:0/437845899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d940c9800 0x7f8d94145620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:59.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.250+0000 7f8d99333640 1 --2- 172.21.15.84:0/437845899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9406af40 0x7f8d940c1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:59.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.250+0000 7f8d99333640 1 -- 172.21.15.84:0/437845899 >> 172.21.15.84:0/437845899 conn(0x7f8d9406bc10 msgr2=0x7f8d940aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:59.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.250+0000 7f8d99333640 1 -- 172.21.15.84:0/437845899 shutdown_connections 2023-10-16T11:22:59.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:22:59.250+0000 7f8d99333640 1 -- 172.21.15.84:0/437845899 wait complete. 2023-10-16T11:22:59.271 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2023-10-16T11:22:59.271 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{'m': 1}]} 2023-10-16T11:22:59.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:59.272 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:22:59.329 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T11:22:59.329 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:22:59.330 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:00.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:00.332 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:00.395 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T11:23:00.396 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:00.396 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:00.767 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:23:00.764+0000 7f71ab3b6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:23:00.767 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:23:00.764+0000 7f71ab3b6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:23:00.767 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:23:00.764+0000 7f71ab3b6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:23:00.767 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:23:00.764+0000 7f71ab3b6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:23:00.768 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:23:00.764+0000 7f71ab3b6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:23:01.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:01.398 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:01.460 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:01.461 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:01.461 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:01.462 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:01.462 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-16-11_23_00_UTC'] 2023-10-16T11:23:01.462 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697455378.770978, now = 1697455381.4628909, timo = 2 2023-10-16T11:23:02.464 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2023-10-16-11_23_00 in ~62s... 2023-10-16T11:23:02.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:02.465 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:02.525 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:02.527 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:02.527 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:02.527 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:03.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:03.529 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:03.591 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:03.592 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:03.592 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:03.593 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:03.707 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:23:03.716 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:23:04.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:04.595 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:04.658 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:04.658 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:04.659 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:04.659 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:05.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:05.660 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:05.723 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:05.725 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:05.725 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:05.726 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:06.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:06.727 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:06.789 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:06.791 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:06.791 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:06.791 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:07.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:07.793 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:07.856 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:07.857 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:07.857 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:07.858 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:08.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:08.859 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:08.920 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:08.921 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:08.922 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:08.922 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:09.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:09.924 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:09.988 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:09.989 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:09.989 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:09.990 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:10.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:10.992 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:11.055 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:11.056 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:11.057 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:11.057 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:12.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:12.059 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:12.117 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:12.119 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:12.119 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:12.119 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:13.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:13.121 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:13.181 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:13.182 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:13.182 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:13.183 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:14.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:14.185 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:14.246 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:14.247 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:14.248 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:14.248 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:15.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:15.250 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:15.312 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:15.313 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:15.314 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:15.314 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:16.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:16.315 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:16.377 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:16.378 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:16.378 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:16.379 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:17.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:17.381 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:17.440 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:17.441 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:17.441 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:17.442 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:18.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:18.443 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:18.504 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:18.505 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:18.506 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:18.506 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:19.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:19.507 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:19.570 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:19.571 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:19.571 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:19.572 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:20.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:20.573 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:20.635 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:20.636 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:20.636 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:20.637 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:21.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:21.638 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:21.705 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:21.707 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:21.707 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:21.707 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:22.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:22.709 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:22.772 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:22.773 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:22.774 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:22.774 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:23.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:23.776 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:23.837 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:23.839 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:23.839 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:23.839 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:24.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:24.841 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:24.901 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:24.903 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:24.903 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:24.903 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:25.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:25.905 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:25.964 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:25.965 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:25.966 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:25.966 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:26.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:26.968 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:27.030 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:27.031 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:27.031 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:27.032 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:28.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:28.033 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:28.095 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:28.096 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:28.097 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:28.097 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:29.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:29.099 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:29.159 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:29.160 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:29.161 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:29.161 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:30.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:30.163 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:30.224 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:30.225 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:30.226 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:30.226 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:31.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:31.228 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:31.290 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:31.291 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:31.291 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:31.292 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:32.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:32.293 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:32.356 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:32.357 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:32.358 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:32.358 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:33.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:33.360 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:33.423 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:33.424 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:33.424 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:33.425 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:33.863 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:23:33.872 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:23:34.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:34.426 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:34.488 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:34.489 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:34.489 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:34.490 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:35.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:35.492 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:35.554 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:35.556 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:35.556 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:35.556 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:36.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:36.558 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:36.620 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:36.622 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:36.622 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:36.622 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:37.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:37.625 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:37.688 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:37.689 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:37.689 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:37.689 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:38.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:38.691 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:38.753 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:38.754 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:38.754 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:38.755 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:39.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:39.756 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:39.818 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:39.819 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:39.820 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:39.820 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:40.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:40.822 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:40.883 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:40.884 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:40.884 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:40.885 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:41.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:41.886 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:41.949 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:41.950 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:41.950 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:41.950 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:42.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:42.952 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:43.014 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:43.015 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:43.016 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:43.016 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:44.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:44.018 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:44.079 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:44.081 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:44.081 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:44.081 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:45.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:45.082 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:45.143 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:45.144 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:45.145 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:45.145 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:46.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:46.146 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:46.206 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:46.207 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:46.208 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:46.208 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:47.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:47.210 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:47.270 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:47.271 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:47.271 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:47.273 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:48.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:48.274 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:48.338 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:48.339 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:48.339 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:48.340 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:49.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:49.342 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:49.404 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:49.405 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:49.406 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:49.406 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:50.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:50.408 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:50.469 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:50.470 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:50.470 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:50.471 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:51.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:51.472 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:51.534 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:51.535 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:51.536 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:51.536 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:52.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:52.538 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:52.600 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:52.601 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:52.601 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:52.602 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:53.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:53.604 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:53.664 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:53.665 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:53.666 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:53.666 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:54.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:54.668 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:54.730 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:54.731 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:54.732 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:54.732 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:55.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:55.734 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:55.797 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:55.798 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:55.799 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:55.799 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:56.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:56.801 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:56.861 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:56.863 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:56.863 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:56.863 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:57.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:57.865 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:57.921 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:57.923 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:57.923 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:57.923 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:58.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:58.925 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:23:58.986 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:23:58.987 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:23:58.987 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:23:58.987 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:23:59.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:23:59.989 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:24:00.050 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_23_00_UTC 2023-10-16T11:24:00.051 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:24:00.052 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:24:00.052 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:24:01.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:24:01.054 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:24:01.113 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_24_00_UTC 2023-10-16T11:24:01.115 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_24_00_UTC'} 2023-10-16T11:24:01.115 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-16-11_24_00_UTC'} 2023-10-16T11:24:01.115 INFO:tasks.cephfs.test_snap_schedules:removed: {'scheduled-2023-10-16-11_23_00_UTC'} 2023-10-16T11:24:01.116 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2023-10-16-11_23_00_UTC'] 2023-10-16T11:24:01.116 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697455378.770978, now = 1697455441.1164777, timo = 62 2023-10-16T11:24:02.117 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-16T11:24:02.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.342+0000 7fd54644b640 1 Processor -- start 2023-10-16T11:24:02.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.343+0000 7fd54644b640 1 -- start start 2023-10-16T11:24:02.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.344+0000 7fd54644b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd540159e90 0x7fd5400ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:02.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.344+0000 7fd54644b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd54006c140 0x7fd54006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:02.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.344+0000 7fd54644b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd540070360 con 0x7fd540159ab0 2023-10-16T11:24:02.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.344+0000 7fd54644b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54006cb10 con 0x7fd540159e90 2023-10-16T11:24:02.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.344+0000 7fd54644b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54015d890 con 0x7fd54006c140 2023-10-16T11:24:02.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.344+0000 7fd53ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd540159ab0 0x7fd54006fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33886/0 (socket says 172.21.15.84:33886) 2023-10-16T11:24:02.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.344+0000 7fd53ffff640 1 -- 172.21.15.84:0/1643161754 learned_addr learned my addr 172.21.15.84:0/1643161754 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:02.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.344+0000 7fd53f7fe640 1 -- 172.21.15.84:0/1643161754 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259647222 0 0) 0x7fd540070360 con 0x7fd540159ab0 2023-10-16T11:24:02.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.344+0000 7fd53f7fe640 1 -- 172.21.15.84:0/1643161754 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd52c003520 con 0x7fd540159ab0 2023-10-16T11:24:02.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.345+0000 7fd537fff640 1 --2- 172.21.15.84:0/1643161754 >> v2:172.21.15.157:3300/0 conn(0x7fd540159e90 0x7fd5400ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:02.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.345+0000 7fd5449e8640 1 --2- 172.21.15.84:0/1643161754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd54006c140 0x7fd54006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:02.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.345+0000 7fd53f7fe640 1 -- 172.21.15.84:0/1643161754 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2175461503 0 0) 0x7fd52c003520 con 0x7fd540159ab0 2023-10-16T11:24:02.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.345+0000 7fd53f7fe640 1 -- 172.21.15.84:0/1643161754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd54006c140 msgr2=0x7fd54006c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:02.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.345+0000 7fd53f7fe640 1 --2- 172.21.15.84:0/1643161754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd54006c140 0x7fd54006c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:02.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.345+0000 7fd53f7fe640 1 -- 172.21.15.84:0/1643161754 >> v2:172.21.15.157:3300/0 conn(0x7fd540159e90 msgr2=0x7fd5400ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:02.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.345+0000 7fd53f7fe640 1 --2- 172.21.15.84:0/1643161754 >> v2:172.21.15.157:3300/0 conn(0x7fd540159e90 0x7fd5400ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:02.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.345+0000 7fd53f7fe640 1 -- 172.21.15.84:0/1643161754 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5400cd1f0 con 0x7fd540159ab0 2023-10-16T11:24:02.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.345+0000 7fd53f7fe640 1 -- 172.21.15.84:0/1643161754 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd530001670 con 0x7fd540159ab0 2023-10-16T11:24:02.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.345+0000 7fd5449e8640 1 --2- 172.21.15.84:0/1643161754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd54006c140 0x7fd54006c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:02.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.345+0000 7fd53f7fe640 1 -- 172.21.15.84:0/1643161754 >> v1:172.21.15.84:6789/0 conn(0x7fd540159ab0 legacy=0x7fd54006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:02.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.345+0000 7fd53f7fe640 1 --2- 172.21.15.84:0/1643161754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd54006c140 0x7fd52c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:02.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.345+0000 7fd53f7fe640 1 --2- 172.21.15.84:0/1643161754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd52c003f60 0x7fd52c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:02.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.345+0000 7fd53f7fe640 1 --2- 172.21.15.84:0/1643161754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c004d00 0x7fd52c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:02.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.345+0000 7fd53f7fe640 1 -- 172.21.15.84:0/1643161754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54006cb10 con 0x7fd52c003f60 2023-10-16T11:24:02.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.345+0000 7fd53f7fe640 1 -- 172.21.15.84:0/1643161754 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54015d890 con 0x7fd52c004d00 2023-10-16T11:24:02.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.345+0000 7fd53f7fe640 1 -- 172.21.15.84:0/1643161754 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c00bc60 con 0x7fd54006c140 2023-10-16T11:24:02.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.345+0000 7fd537fff640 1 --2- 172.21.15.84:0/1643161754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd52c003f60 0x7fd52c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:02.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.346+0000 7fd53ffff640 1 --2- 172.21.15.84:0/1643161754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd54006c140 0x7fd52c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:02.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.346+0000 7fd537fff640 1 -- 172.21.15.84:0/1643161754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd54006c140 msgr2=0x7fd52c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:02.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.346+0000 7fd537fff640 1 --2- 172.21.15.84:0/1643161754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd54006c140 0x7fd52c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:02.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.346+0000 7fd537fff640 1 -- 172.21.15.84:0/1643161754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c004d00 msgr2=0x7fd52c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:02.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.346+0000 7fd5449e8640 1 --2- 172.21.15.84:0/1643161754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c004d00 0x7fd52c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:02.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.346+0000 7fd537fff640 1 --2- 172.21.15.84:0/1643161754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c004d00 0x7fd52c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:02.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.346+0000 7fd537fff640 1 -- 172.21.15.84:0/1643161754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c003520 con 0x7fd52c003f60 2023-10-16T11:24:02.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.347+0000 7fd537fff640 1 --2- 172.21.15.84:0/1643161754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd52c003f60 0x7fd52c0047c0 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fd52800ba30 tx=0x7fd52c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da275068ad86699d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:02.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.347+0000 7fd53f7fe640 1 -- 172.21.15.84:0/1643161754 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528032680 con 0x7fd52c003f60 2023-10-16T11:24:02.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.347+0000 7fd53f7fe640 1 -- 172.21.15.84:0/1643161754 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5280327e0 con 0x7fd52c003f60 2023-10-16T11:24:02.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.347+0000 7fd53f7fe640 1 -- 172.21.15.84:0/1643161754 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528032b70 con 0x7fd52c003f60 2023-10-16T11:24:02.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.348+0000 7fd54644b640 1 -- 172.21.15.84:0/1643161754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd52c003f60 msgr2=0x7fd52c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:02.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.348+0000 7fd54644b640 1 --2- 172.21.15.84:0/1643161754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd52c003f60 0x7fd52c0047c0 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fd52800ba30 tx=0x7fd52c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:02.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.348+0000 7fd5449e8640 1 -- 172.21.15.84:0/1643161754 reap_dead start 2023-10-16T11:24:02.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.348+0000 7fd54644b640 1 -- 172.21.15.84:0/1643161754 shutdown_connections 2023-10-16T11:24:02.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.348+0000 7fd54644b640 1 -- 172.21.15.84:0/1643161754 >> 172.21.15.84:0/1643161754 conn(0x7fd5400ce9b0 msgr2=0x7fd5400cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:02.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.348+0000 7fd54644b640 1 -- 172.21.15.84:0/1643161754 shutdown_connections 2023-10-16T11:24:02.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.348+0000 7fd54644b640 1 -- 172.21.15.84:0/1643161754 wait complete. 2023-10-16T11:24:02.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.350+0000 7fd54644b640 1 Processor -- start 2023-10-16T11:24:02.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.350+0000 7fd54644b640 1 -- start start 2023-10-16T11:24:02.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.351+0000 7fd54644b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c003f60 0x7fd5400c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:02.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.351+0000 7fd54644b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd540158f10 0x7fd5400c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:02.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.351+0000 7fd54644b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5400c3190 0x7fd54014f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:02.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.351+0000 7fd54644b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5400c24e0 con 0x7fd540158f10 2023-10-16T11:24:02.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.351+0000 7fd54644b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5400c27c0 con 0x7fd52c003f60 2023-10-16T11:24:02.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.351+0000 7fd54644b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5400c41f0 con 0x7fd5400c3190 2023-10-16T11:24:02.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.351+0000 7fd537fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd540158f10 0x7fd5400c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:02.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.351+0000 7fd537fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd540158f10 0x7fd5400c2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45324/0 (socket says 172.21.15.84:45324) 2023-10-16T11:24:02.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.351+0000 7fd537fff640 1 -- 172.21.15.84:0/1806405431 learned_addr learned my addr 172.21.15.84:0/1806405431 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:02.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.351+0000 7fd537fff640 1 -- 172.21.15.84:0/1806405431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5400c3190 msgr2=0x7fd54014f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:02.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.351+0000 7fd53ffff640 1 --2- 172.21.15.84:0/1806405431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c003f60 0x7fd5400c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:02.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.351+0000 7fd537fff640 1 --2- 172.21.15.84:0/1806405431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5400c3190 0x7fd54014f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:02.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.351+0000 7fd537fff640 1 -- 172.21.15.84:0/1806405431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c003f60 msgr2=0x7fd5400c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:02.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.351+0000 7fd537fff640 1 --2- 172.21.15.84:0/1806405431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c003f60 0x7fd5400c3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:02.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.351+0000 7fd537fff640 1 -- 172.21.15.84:0/1806405431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5400c44d0 con 0x7fd540158f10 2023-10-16T11:24:02.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.352+0000 7fd537fff640 1 --2- 172.21.15.84:0/1806405431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd540158f10 0x7fd5400c2c50 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7fd528035040 tx=0x7fd528004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:02.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.352+0000 7fd53d7fa640 1 -- 172.21.15.84:0/1806405431 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528004220 con 0x7fd540158f10 2023-10-16T11:24:02.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.352+0000 7fd53d7fa640 1 -- 172.21.15.84:0/1806405431 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5280043a0 con 0x7fd540158f10 2023-10-16T11:24:02.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.352+0000 7fd53d7fa640 1 -- 172.21.15.84:0/1806405431 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528030aa0 con 0x7fd540158f10 2023-10-16T11:24:02.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.352+0000 7fd54644b640 1 -- 172.21.15.84:0/1806405431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5400c47b0 con 0x7fd540158f10 2023-10-16T11:24:02.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.352+0000 7fd54644b640 1 -- 172.21.15.84:0/1806405431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54014fd80 con 0x7fd540158f10 2023-10-16T11:24:02.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.353+0000 7fd54644b640 1 -- 172.21.15.84:0/1806405431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5400c2950 con 0x7fd540158f10 2023-10-16T11:24:02.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.354+0000 7fd53d7fa640 1 -- 172.21.15.84:0/1806405431 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 127) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fd528030c00 con 0x7fd540158f10 2023-10-16T11:24:02.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.356+0000 7fd53d7fa640 1 --2- 172.21.15.84:0/1806405431 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fd52006b130 0x7fd52006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:02.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.356+0000 7fd53d7fa640 1 -- 172.21.15.84:0/1806405431 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6139+0+0 (secure 0 0 0) 0x7fd5280ac890 con 0x7fd540158f10 2023-10-16T11:24:02.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.356+0000 7fd53ffff640 1 --2- 172.21.15.84:0/1806405431 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fd52006b130 0x7fd52006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:02.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.356+0000 7fd53ffff640 1 --2- 172.21.15.84:0/1806405431 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fd52006b130 0x7fd52006d5f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd530001eb0 tx=0x7fd530001da0 comp rx=0 tx=0).ready entity=mgr.10974 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:02.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.360+0000 7fd53d7fa640 1 -- 172.21.15.84:0/1806405431 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd528077280 con 0x7fd540158f10 2023-10-16T11:24:02.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.571+0000 7fd54644b640 1 -- 172.21.15.84:0/1806405431 --> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5400cd7b0 con 0x7fd52006b130 2023-10-16T11:24:02.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.743+0000 7fd53d7fa640 1 -- 172.21.15.84:0/1806405431 <== mgr.10974 v2:172.21.15.84:6832/855077515 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fd530005e60 con 0x7fd52006b130 2023-10-16T11:24:02.746 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path snap_test_dir1 2023-10-16T11:24:02.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.747+0000 7fd54644b640 1 -- 172.21.15.84:0/1806405431 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fd52006b130 msgr2=0x7fd52006d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:02.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.747+0000 7fd54644b640 1 --2- 172.21.15.84:0/1806405431 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fd52006b130 0x7fd52006d5f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd530001eb0 tx=0x7fd530001da0 comp rx=0 tx=0).stop 2023-10-16T11:24:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.747+0000 7fd54644b640 1 -- 172.21.15.84:0/1806405431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd540158f10 msgr2=0x7fd5400c2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.747+0000 7fd54644b640 1 --2- 172.21.15.84:0/1806405431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd540158f10 0x7fd5400c2c50 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7fd528035040 tx=0x7fd528004730 comp rx=0 tx=0).stop 2023-10-16T11:24:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.747+0000 7fd54644b640 1 -- 172.21.15.84:0/1806405431 shutdown_connections 2023-10-16T11:24:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.747+0000 7fd54644b640 1 --2- 172.21.15.84:0/1806405431 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fd52006b130 0x7fd52006d5f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:02.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.747+0000 7fd54644b640 1 --2- 172.21.15.84:0/1806405431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5400c3190 0x7fd54014f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:02.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.747+0000 7fd54644b640 1 --2- 172.21.15.84:0/1806405431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd540158f10 0x7fd5400c2c50 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:02.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.747+0000 7fd54644b640 1 --2- 172.21.15.84:0/1806405431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd52c003f60 0x7fd5400c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:02.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.747+0000 7fd54644b640 1 -- 172.21.15.84:0/1806405431 >> 172.21.15.84:0/1806405431 conn(0x7fd5400ce9b0 msgr2=0x7fd5400c1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:02.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.747+0000 7fd54644b640 1 -- 172.21.15.84:0/1806405431 shutdown_connections 2023-10-16T11:24:02.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:02.747+0000 7fd54644b640 1 -- 172.21.15.84:0/1806405431 wait complete. 2023-10-16T11:24:02.768 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-16T11:24:02.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:24:02.769 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:24:02.819 INFO:teuthology.orchestra.run.smithi084.stdout:scheduled-2023-10-16-11_24_00_UTC 2023-10-16T11:24:02.820 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-11_24_00_UTC 2023-10-16T11:24:02.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:24:02.821 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-11_24_00_UTC) 2023-10-16T11:24:02.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:24:02.880 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-16T11:24:02.935 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T11:24:03.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.150+0000 7fabbeab4640 1 Processor -- start 2023-10-16T11:24:03.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.150+0000 7fabbeab4640 1 -- start start 2023-10-16T11:24:03.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.152+0000 7fabbeab4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fabb8065ab0 0x7fabb80c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:03.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.152+0000 7fabbeab4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb80acd70 con 0x7fabb806d220 2023-10-16T11:24:03.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.152+0000 7fabbeab4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb80ac4a0 con 0x7fabb80ce280 2023-10-16T11:24:03.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.152+0000 7fabbeab4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb8065f40 con 0x7fabb8065ab0 2023-10-16T11:24:03.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.152+0000 7fabbd2b1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fabb8065ab0 0x7fabb80c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:03.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.152+0000 7fabbdab2640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fabb806d220 0x7fabb80cdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33902/0 (socket says 172.21.15.84:33902) 2023-10-16T11:24:03.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.152+0000 7fabbdab2640 1 -- 172.21.15.84:0/3456762725 learned_addr learned my addr 172.21.15.84:0/3456762725 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:03.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.153+0000 7fabbd2b1640 1 -- 172.21.15.84:0/3456762725 >> v1:172.21.15.157:6789/0 conn(0x7fabb80ce280 legacy=0x7fabb80ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:03.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.153+0000 7fabbd2b1640 1 -- 172.21.15.84:0/3456762725 >> v1:172.21.15.84:6789/0 conn(0x7fabb806d220 legacy=0x7fabb80cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:03.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.153+0000 7fabbd2b1640 1 -- 172.21.15.84:0/3456762725 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb80ac720 con 0x7fabb8065ab0 2023-10-16T11:24:03.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.154+0000 7fabbd2b1640 1 --2- 172.21.15.84:0/3456762725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabb8065ab0 0x7fabb80c95e0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7faba80097a0 tx=0x7faba802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d13624b530b52a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:03.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.154+0000 7fabbcab0640 1 -- 172.21.15.84:0/3456762725 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba802e070 con 0x7fabb8065ab0 2023-10-16T11:24:03.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.154+0000 7fabbcab0640 1 -- 172.21.15.84:0/3456762725 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba80315a0 con 0x7fabb8065ab0 2023-10-16T11:24:03.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.154+0000 7fabbcab0640 1 -- 172.21.15.84:0/3456762725 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8031930 con 0x7fabb8065ab0 2023-10-16T11:24:03.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.155+0000 7fabbeab4640 1 -- 172.21.15.84:0/3456762725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabb8065ab0 msgr2=0x7fabb80c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:03.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.155+0000 7fabbeab4640 1 --2- 172.21.15.84:0/3456762725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabb8065ab0 0x7fabb80c95e0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7faba80097a0 tx=0x7faba802b840 comp rx=0 tx=0).stop 2023-10-16T11:24:03.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.155+0000 7fabbeab4640 1 -- 172.21.15.84:0/3456762725 shutdown_connections 2023-10-16T11:24:03.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.155+0000 7fabbeab4640 1 --2- 172.21.15.84:0/3456762725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabb8065ab0 0x7fabb80c95e0 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:03.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.155+0000 7fabbeab4640 1 -- 172.21.15.84:0/3456762725 >> 172.21.15.84:0/3456762725 conn(0x7fabb80b0a80 msgr2=0x7fabb80b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:03.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.155+0000 7fabbeab4640 1 -- 172.21.15.84:0/3456762725 shutdown_connections 2023-10-16T11:24:03.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.155+0000 7fabbeab4640 1 -- 172.21.15.84:0/3456762725 wait complete. 2023-10-16T11:24:03.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.157+0000 7fabbeab4640 1 Processor -- start 2023-10-16T11:24:03.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.157+0000 7fabbeab4640 1 -- start start 2023-10-16T11:24:03.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.158+0000 7fabbeab4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabb806d220 0x7fabb809a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:03.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.158+0000 7fabbeab4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabb80ce280 0x7fabb809b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:03.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.158+0000 7fabbeab4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabb81579e0 0x7fabb80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:03.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.158+0000 7fabbeab4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb809b5b0 con 0x7fabb80ce280 2023-10-16T11:24:03.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.158+0000 7fabbeab4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb815e450 con 0x7fabb806d220 2023-10-16T11:24:03.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.158+0000 7fabbeab4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb809b7c0 con 0x7fabb81579e0 2023-10-16T11:24:03.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.159+0000 7fabbd2b1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabb80ce280 0x7fabb809b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:03.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.159+0000 7fabbd2b1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabb80ce280 0x7fabb809b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45338/0 (socket says 172.21.15.84:45338) 2023-10-16T11:24:03.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.159+0000 7fabbd2b1640 1 -- 172.21.15.84:0/3335009373 learned_addr learned my addr 172.21.15.84:0/3335009373 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:03.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.159+0000 7fabbd2b1640 1 -- 172.21.15.84:0/3335009373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabb81579e0 msgr2=0x7fabb80b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:03.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.159+0000 7fabbe2b3640 1 --2- 172.21.15.84:0/3335009373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabb81579e0 0x7fabb80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:03.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.159+0000 7fabbd2b1640 1 --2- 172.21.15.84:0/3335009373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabb81579e0 0x7fabb80b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:03.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.159+0000 7fabbd2b1640 1 -- 172.21.15.84:0/3335009373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabb806d220 msgr2=0x7fabb809a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:03.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.159+0000 7fabbd2b1640 1 --2- 172.21.15.84:0/3335009373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabb806d220 0x7fabb809a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:03.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.159+0000 7fabbd2b1640 1 -- 172.21.15.84:0/3335009373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba8009400 con 0x7fabb80ce280 2023-10-16T11:24:03.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.160+0000 7fabbd2b1640 1 --2- 172.21.15.84:0/3335009373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabb80ce280 0x7fabb809b070 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7faba8000c00 tx=0x7faba80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:03.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.160+0000 7fabae7fc640 1 -- 172.21.15.84:0/3335009373 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8032900 con 0x7fabb80ce280 2023-10-16T11:24:03.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.160+0000 7fabae7fc640 1 -- 172.21.15.84:0/3335009373 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba8032a80 con 0x7fabb80ce280 2023-10-16T11:24:03.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.160+0000 7fabae7fc640 1 -- 172.21.15.84:0/3335009373 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8032e30 con 0x7fabb80ce280 2023-10-16T11:24:03.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.160+0000 7fabbeab4640 1 -- 172.21.15.84:0/3335009373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb809baa0 con 0x7fabb80ce280 2023-10-16T11:24:03.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.160+0000 7fabbeab4640 1 -- 172.21.15.84:0/3335009373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb815e9d0 con 0x7fabb80ce280 2023-10-16T11:24:03.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.161+0000 7fabbeab4640 1 -- 172.21.15.84:0/3335009373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb815ebc0 con 0x7fabb80ce280 2023-10-16T11:24:03.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.162+0000 7fabae7fc640 1 -- 172.21.15.84:0/3335009373 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 127) v1 ==== 89462+0+0 (secure 0 0 0) 0x7faba803b3f0 con 0x7fabb80ce280 2023-10-16T11:24:03.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.163+0000 7fabae7fc640 1 --2- 172.21.15.84:0/3335009373 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fab9c06b100 0x7fab9c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:03.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.163+0000 7fabae7fc640 1 -- 172.21.15.84:0/3335009373 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6139+0+0 (secure 0 0 0) 0x7faba80abf00 con 0x7fabb80ce280 2023-10-16T11:24:03.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.164+0000 7fabbdab2640 1 --2- 172.21.15.84:0/3335009373 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fab9c06b100 0x7fab9c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:03.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.164+0000 7fabbdab2640 1 --2- 172.21.15.84:0/3335009373 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fab9c06b100 0x7fab9c06d5c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fabb40015f0 tx=0x7fabb4007a40 comp rx=0 tx=0).ready entity=mgr.10974 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:03.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.167+0000 7fabae7fc640 1 -- 172.21.15.84:0/3335009373 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7faba80768f0 con 0x7fabb80ce280 2023-10-16T11:24:03.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.410+0000 7fabbeab4640 1 -- 172.21.15.84:0/3335009373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fabb809bc60 con 0x7fabb80ce280 2023-10-16T11:24:03.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.882+0000 7fabae7fc640 1 -- 172.21.15.84:0/3335009373 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v128) v1 ==== 93+0+0 (secure 0 0 0) 0x7faba8079f10 con 0x7fabb80ce280 2023-10-16T11:24:03.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.885+0000 7fabbeab4640 1 -- 172.21.15.84:0/3335009373 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fab9c06b100 msgr2=0x7fab9c06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:03.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.885+0000 7fabbeab4640 1 --2- 172.21.15.84:0/3335009373 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fab9c06b100 0x7fab9c06d5c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fabb40015f0 tx=0x7fabb4007a40 comp rx=0 tx=0).stop 2023-10-16T11:24:03.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.885+0000 7fabbeab4640 1 -- 172.21.15.84:0/3335009373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabb80ce280 msgr2=0x7fabb809b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:03.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.885+0000 7fabbeab4640 1 --2- 172.21.15.84:0/3335009373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabb80ce280 0x7fabb809b070 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7faba8000c00 tx=0x7faba80046c0 comp rx=0 tx=0).stop 2023-10-16T11:24:03.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.886+0000 7fabbeab4640 1 -- 172.21.15.84:0/3335009373 shutdown_connections 2023-10-16T11:24:03.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.886+0000 7fabbeab4640 1 --2- 172.21.15.84:0/3335009373 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fab9c06b100 0x7fab9c06d5c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:03.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.886+0000 7fabbeab4640 1 --2- 172.21.15.84:0/3335009373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabb81579e0 0x7fabb80b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:03.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.886+0000 7fabbeab4640 1 --2- 172.21.15.84:0/3335009373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabb80ce280 0x7fabb809b070 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.886+0000 7fabbeab4640 1 --2- 172.21.15.84:0/3335009373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabb806d220 0x7fabb809a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.886+0000 7fabbeab4640 1 -- 172.21.15.84:0/3335009373 >> 172.21.15.84:0/3335009373 conn(0x7fabb80b0a80 msgr2=0x7fabb8070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.886+0000 7fabbeab4640 1 -- 172.21.15.84:0/3335009373 shutdown_connections 2023-10-16T11:24:03.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:03.886+0000 7fabbeab4640 1 -- 172.21.15.84:0/3335009373 wait complete. 2023-10-16T11:24:03.912 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:24:03.913 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:24:04.023 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:24:04.038 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:24:04.043 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:24:04.114 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:24:04.115 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:24:04.115 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:24:04.162 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:24:04.163 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:24:04.252 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:24:04.413 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:04.411+0000 7fe738816040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:24:04.434 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:04.434+0000 7f32ee044040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:24:04.468 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:04.466+0000 7fe738816040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:24:04.488 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:04.488+0000 7f32ee044040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:24:04.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.499+0000 7f22162e0640 1 Processor -- start 2023-10-16T11:24:04.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.499+0000 7f22162e0640 1 -- start start 2023-10-16T11:24:04.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.501+0000 7f22162e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22100caa10 0x7f221006fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:04.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.501+0000 7f22162e0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f22100cc920 0x7f22100ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:04.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.501+0000 7f22162e0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2210159790 con 0x7f22101593b0 2023-10-16T11:24:04.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.501+0000 7f22162e0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22100c9b20 con 0x7f22100cc920 2023-10-16T11:24:04.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.501+0000 7f22162e0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22100c9e00 con 0x7f22100caa10 2023-10-16T11:24:04.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.501+0000 7f220ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f22101593b0 0x7f221006f550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33910/0 (socket says 172.21.15.84:33910) 2023-10-16T11:24:04.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.501+0000 7f220ffff640 1 -- 172.21.15.84:0/2827501100 learned_addr learned my addr 172.21.15.84:0/2827501100 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:04.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f221487d640 1 --2- 172.21.15.84:0/2827501100 >> v2:172.21.15.157:3300/0 conn(0x7f22100cc920 0x7f22100ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:04.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f2207fff640 1 --2- 172.21.15.84:0/2827501100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22100caa10 0x7f221006fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:04.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f220f7fe640 1 -- 172.21.15.84:0/2827501100 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523454852 0 0) 0x7f2210159790 con 0x7f22101593b0 2023-10-16T11:24:04.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f220f7fe640 1 -- 172.21.15.84:0/2827501100 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21fc005730 con 0x7f22101593b0 2023-10-16T11:24:04.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f220f7fe640 1 -- 172.21.15.84:0/2827501100 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2644784812 0 0) 0x7f21fc005730 con 0x7f22101593b0 2023-10-16T11:24:04.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f220f7fe640 1 -- 172.21.15.84:0/2827501100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22100caa10 msgr2=0x7f221006fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:04.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f220f7fe640 1 --2- 172.21.15.84:0/2827501100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22100caa10 0x7f221006fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:04.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f220f7fe640 1 -- 172.21.15.84:0/2827501100 >> v2:172.21.15.157:3300/0 conn(0x7f22100cc920 msgr2=0x7f22100ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:04.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f220f7fe640 1 --2- 172.21.15.84:0/2827501100 >> v2:172.21.15.157:3300/0 conn(0x7f22100cc920 0x7f22100ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:04.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f220f7fe640 1 -- 172.21.15.84:0/2827501100 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22100ca0e0 con 0x7f22101593b0 2023-10-16T11:24:04.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f220f7fe640 1 -- 172.21.15.84:0/2827501100 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f2200001670 con 0x7f22101593b0 2023-10-16T11:24:04.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f221487d640 1 --2- 172.21.15.84:0/2827501100 >> v2:172.21.15.157:3300/0 conn(0x7f22100cc920 0x7f22100ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:24:04.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f220f7fe640 1 -- 172.21.15.84:0/2827501100 >> v1:172.21.15.84:6789/0 conn(0x7f22101593b0 legacy=0x7f221006f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:04.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f220f7fe640 1 --2- 172.21.15.84:0/2827501100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21fc005b80 0x7f21fc005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:04.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f220f7fe640 1 --2- 172.21.15.84:0/2827501100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22100caa10 0x7f21fc0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:04.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f220f7fe640 1 --2- 172.21.15.84:0/2827501100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21fc006f10 0x7f21fc007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:04.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f220f7fe640 1 -- 172.21.15.84:0/2827501100 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22100c9b20 con 0x7f21fc006f10 2023-10-16T11:24:04.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f220f7fe640 1 -- 172.21.15.84:0/2827501100 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22100c9e00 con 0x7f21fc005b80 2023-10-16T11:24:04.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.502+0000 7f220f7fe640 1 -- 172.21.15.84:0/2827501100 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc00dcc0 con 0x7f22100caa10 2023-10-16T11:24:04.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.503+0000 7f221487d640 1 --2- 172.21.15.84:0/2827501100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21fc006f10 0x7f21fc007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:04.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.503+0000 7f220ffff640 1 --2- 172.21.15.84:0/2827501100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21fc005b80 0x7f21fc005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:04.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.503+0000 7f2207fff640 1 --2- 172.21.15.84:0/2827501100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22100caa10 0x7f21fc0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:04.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.503+0000 7f221487d640 1 -- 172.21.15.84:0/2827501100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22100caa10 msgr2=0x7f21fc0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:04.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.503+0000 7f221487d640 1 --2- 172.21.15.84:0/2827501100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22100caa10 0x7f21fc0069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:04.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.503+0000 7f221487d640 1 -- 172.21.15.84:0/2827501100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21fc005b80 msgr2=0x7f21fc005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:04.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.503+0000 7f221487d640 1 --2- 172.21.15.84:0/2827501100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21fc005b80 0x7f21fc005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:04.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.503+0000 7f221487d640 1 -- 172.21.15.84:0/2827501100 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21fc005730 con 0x7f21fc006f10 2023-10-16T11:24:04.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.504+0000 7f221487d640 1 --2- 172.21.15.84:0/2827501100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21fc006f10 0x7f21fc007390 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f220800e350 tx=0x7f21fc007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ae99a3b11ac322 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:04.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.504+0000 7f220f7fe640 1 -- 172.21.15.84:0/2827501100 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220802ce00 con 0x7f21fc006f10 2023-10-16T11:24:04.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.504+0000 7f220f7fe640 1 -- 172.21.15.84:0/2827501100 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220800f040 con 0x7f21fc006f10 2023-10-16T11:24:04.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.504+0000 7f220f7fe640 1 -- 172.21.15.84:0/2827501100 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2208034640 con 0x7f21fc006f10 2023-10-16T11:24:04.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.505+0000 7f22162e0640 1 -- 172.21.15.84:0/2827501100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21fc006f10 msgr2=0x7f21fc007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:04.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.505+0000 7f22162e0640 1 --2- 172.21.15.84:0/2827501100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21fc006f10 0x7f21fc007390 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f220800e350 tx=0x7f21fc007a00 comp rx=0 tx=0).stop 2023-10-16T11:24:04.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.505+0000 7f221487d640 1 -- 172.21.15.84:0/2827501100 reap_dead start 2023-10-16T11:24:04.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.505+0000 7f22162e0640 1 -- 172.21.15.84:0/2827501100 shutdown_connections 2023-10-16T11:24:04.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.505+0000 7f22162e0640 1 -- 172.21.15.84:0/2827501100 >> 172.21.15.84:0/2827501100 conn(0x7f22100cb410 msgr2=0x7f22100cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:04.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.505+0000 7f22162e0640 1 -- 172.21.15.84:0/2827501100 shutdown_connections 2023-10-16T11:24:04.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.505+0000 7f22162e0640 1 -- 172.21.15.84:0/2827501100 wait complete. 2023-10-16T11:24:04.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.507+0000 7f22162e0640 1 Processor -- start 2023-10-16T11:24:04.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.507+0000 7f22162e0640 1 -- start start 2023-10-16T11:24:04.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.508+0000 7f22162e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21fc006f10 0x7f22100acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:04.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.508+0000 7f22162e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f221014e260 0x7f22100c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:04.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.508+0000 7f22162e0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22100c80b0 0x7f22100c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:04.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.508+0000 7f22162e0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221006b6d0 con 0x7f221014e260 2023-10-16T11:24:04.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.508+0000 7f22162e0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22100c8550 con 0x7f22100c80b0 2023-10-16T11:24:04.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.508+0000 7f22162e0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221006b970 con 0x7f21fc006f10 2023-10-16T11:24:04.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.508+0000 7f2207fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f221014e260 0x7f22100c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:04.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.509+0000 7f2207fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f221014e260 0x7f22100c7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45384/0 (socket says 172.21.15.84:45384) 2023-10-16T11:24:04.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.509+0000 7f2207fff640 1 -- 172.21.15.84:0/1299328544 learned_addr learned my addr 172.21.15.84:0/1299328544 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:04.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.509+0000 7f221487d640 1 --2- 172.21.15.84:0/1299328544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22100c80b0 0x7f22100c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:04.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.509+0000 7f2207fff640 1 -- 172.21.15.84:0/1299328544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21fc006f10 msgr2=0x7f22100acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:04.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.509+0000 7f2207fff640 1 --2- 172.21.15.84:0/1299328544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21fc006f10 0x7f22100acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:04.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.509+0000 7f2207fff640 1 -- 172.21.15.84:0/1299328544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22100c80b0 msgr2=0x7f22100c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:04.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.509+0000 7f2207fff640 1 --2- 172.21.15.84:0/1299328544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22100c80b0 0x7f22100c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:04.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.509+0000 7f2207fff640 1 -- 172.21.15.84:0/1299328544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221006bc50 con 0x7f221014e260 2023-10-16T11:24:04.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.509+0000 7f2207fff640 1 --2- 172.21.15.84:0/1299328544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f221014e260 0x7f22100c7b70 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f21f800f580 tx=0x7f21f800fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:04.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.509+0000 7f220d7fa640 1 -- 172.21.15.84:0/1299328544 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8014870 con 0x7f221014e260 2023-10-16T11:24:04.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.509+0000 7f220d7fa640 1 -- 172.21.15.84:0/1299328544 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f80149d0 con 0x7f221014e260 2023-10-16T11:24:04.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.509+0000 7f220d7fa640 1 -- 172.21.15.84:0/1299328544 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8014d80 con 0x7f221014e260 2023-10-16T11:24:04.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.509+0000 7f22162e0640 1 -- 172.21.15.84:0/1299328544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22100c8830 con 0x7f221014e260 2023-10-16T11:24:04.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.510+0000 7f22162e0640 1 -- 172.21.15.84:0/1299328544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22100ad220 con 0x7f221014e260 2023-10-16T11:24:04.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.511+0000 7f22162e0640 1 -- 172.21.15.84:0/1299328544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f221004e160 con 0x7f221014e260 2023-10-16T11:24:04.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.512+0000 7f220d7fa640 1 -- 172.21.15.84:0/1299328544 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 128) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f21f8015410 con 0x7f221014e260 2023-10-16T11:24:04.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.512+0000 7f220d7fa640 1 --2- 172.21.15.84:0/1299328544 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f21f006b0b0 0x7f21f006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:04.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.513+0000 7f220d7fa640 1 -- 172.21.15.84:0/1299328544 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6139+0+0 (secure 0 0 0) 0x7f21f808a700 con 0x7f221014e260 2023-10-16T11:24:04.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.513+0000 7f220ffff640 1 -- 172.21.15.84:0/1299328544 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f21f006b0b0 msgr2=0x7f21f006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/855077515 2023-10-16T11:24:04.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.513+0000 7f220ffff640 1 --2- 172.21.15.84:0/1299328544 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f21f006b0b0 0x7f21f006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:24:04.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.517+0000 7f220d7fa640 1 -- 172.21.15.84:0/1299328544 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f21f8055180 con 0x7f221014e260 2023-10-16T11:24:04.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.712+0000 7f22162e0640 1 -- 172.21.15.84:0/1299328544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f221006bde0 con 0x7f221014e260 2023-10-16T11:24:04.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.714+0000 7f220ffff640 1 -- 172.21.15.84:0/1299328544 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f21f006b0b0 msgr2=0x7f21f006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/855077515 2023-10-16T11:24:04.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.714+0000 7f220ffff640 1 --2- 172.21.15.84:0/1299328544 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f21f006b0b0 0x7f21f006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:24:04.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.715+0000 7f220d7fa640 1 -- 172.21.15.84:0/1299328544 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v204) v1 ==== 74+0+17141 (secure 0 0 0) 0x7f21f80587a0 con 0x7f221014e260 2023-10-16T11:24:04.717 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:04.717 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":204,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:23:58.844672+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:22:40.385524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-16T11:22:41.592928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.18","mappings":[{"from":0,"to":2}]},{"pgid":"25.1f","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/3564300528":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/2349522146":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/2191803332":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/2199276620":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/4262224959":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/956990500":"2023-10-17T11:22:56.900947+0000","172.21.15.84:6833/1985463812":"2023-10-17T11:22:56.900947+0000","172.21.15.84:6832/1985463812":"2023-10-17T11:22:56.900947+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:24:04.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.717+0000 7f22162e0640 1 -- 172.21.15.84:0/1299328544 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f21f006b0b0 msgr2=0x7f21f006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:24:04.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.717+0000 7f22162e0640 1 --2- 172.21.15.84:0/1299328544 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f21f006b0b0 0x7f21f006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:04.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.717+0000 7f22162e0640 1 -- 172.21.15.84:0/1299328544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f221014e260 msgr2=0x7f22100c7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:04.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.717+0000 7f22162e0640 1 --2- 172.21.15.84:0/1299328544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f221014e260 0x7f22100c7b70 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f21f800f580 tx=0x7f21f800fa40 comp rx=0 tx=0).stop 2023-10-16T11:24:04.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.717+0000 7f22162e0640 1 -- 172.21.15.84:0/1299328544 shutdown_connections 2023-10-16T11:24:04.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.717+0000 7f22162e0640 1 --2- 172.21.15.84:0/1299328544 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f21f006b0b0 0x7f21f006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:04.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.717+0000 7f22162e0640 1 --2- 172.21.15.84:0/1299328544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22100c80b0 0x7f22100c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:04.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.717+0000 7f22162e0640 1 --2- 172.21.15.84:0/1299328544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f221014e260 0x7f22100c7b70 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:04.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.717+0000 7f22162e0640 1 --2- 172.21.15.84:0/1299328544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21fc006f10 0x7f22100acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:04.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.717+0000 7f22162e0640 1 -- 172.21.15.84:0/1299328544 >> 172.21.15.84:0/1299328544 conn(0x7f22100cb410 msgr2=0x7f22100ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:04.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.718+0000 7f22162e0640 1 -- 172.21.15.84:0/1299328544 shutdown_connections 2023-10-16T11:24:04.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.718+0000 7f22162e0640 1 -- 172.21.15.84:0/1299328544 wait complete. 2023-10-16T11:24:04.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:22:40.385524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '200', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:22:41.592928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '201', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:24:04.737 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:24:04.938 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:04.937+0000 7f32ee044040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:24:04.939 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:04.938+0000 7fe738816040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:24:04.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.940+0000 7f5421213640 1 Processor -- start 2023-10-16T11:24:04.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.940+0000 7f5421213640 1 -- start start 2023-10-16T11:24:04.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.941+0000 7f5421213640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f541c0b1ca0 0x7f541c0d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:04.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.941+0000 7f5421213640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541c0d0970 con 0x7f541c0cfc40 2023-10-16T11:24:04.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.941+0000 7f5421213640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541c0d0ea0 con 0x7f541c0b2080 2023-10-16T11:24:04.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.941+0000 7f5421213640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f541c0d0bc0 con 0x7f541c0b1ca0 2023-10-16T11:24:04.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.942+0000 7f5420a12640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f541c0cfc40 0x7f541c0b24c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33914/0 (socket says 172.21.15.84:33914) 2023-10-16T11:24:04.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.942+0000 7f5420a12640 1 -- 172.21.15.84:0/2889102918 learned_addr learned my addr 172.21.15.84:0/2889102918 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:04.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.942+0000 7f541bfff640 1 --2- 172.21.15.84:0/2889102918 >> v2:172.21.15.157:3300/0 conn(0x7f541c0b1ca0 0x7f541c0d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:04.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.942+0000 7f541b7fe640 1 -- 172.21.15.84:0/2889102918 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338878842 0 0) 0x7f541c0d0970 con 0x7f541c0cfc40 2023-10-16T11:24:04.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.942+0000 7f541b7fe640 1 -- 172.21.15.84:0/2889102918 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5408003520 con 0x7f541c0cfc40 2023-10-16T11:24:04.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.942+0000 7f541bfff640 1 -- 172.21.15.84:0/2889102918 >> v1:172.21.15.157:6789/0 conn(0x7f541c0b2080 legacy=0x7f541c0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:04.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.942+0000 7f541bfff640 1 -- 172.21.15.84:0/2889102918 >> v1:172.21.15.84:6789/0 conn(0x7f541c0cfc40 legacy=0x7f541c0b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:04.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.942+0000 7f541bfff640 1 -- 172.21.15.84:0/2889102918 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c0cd940 con 0x7f541c0b1ca0 2023-10-16T11:24:04.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.944+0000 7f541bfff640 1 --2- 172.21.15.84:0/2889102918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f541c0b1ca0 0x7f541c0d36b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f540c0097a0 tx=0x7f540c02b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f442f376ddf6e44 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:04.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.944+0000 7f541b7fe640 1 -- 172.21.15.84:0/2889102918 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c02f810 con 0x7f541c0b1ca0 2023-10-16T11:24:04.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.944+0000 7f541b7fe640 1 -- 172.21.15.84:0/2889102918 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540c02f970 con 0x7f541c0b1ca0 2023-10-16T11:24:04.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.944+0000 7f541b7fe640 1 -- 172.21.15.84:0/2889102918 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c02fce0 con 0x7f541c0b1ca0 2023-10-16T11:24:04.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.945+0000 7f5421213640 1 -- 172.21.15.84:0/2889102918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f541c0b1ca0 msgr2=0x7f541c0d36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:04.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.945+0000 7f5421213640 1 --2- 172.21.15.84:0/2889102918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f541c0b1ca0 0x7f541c0d36b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f540c0097a0 tx=0x7f540c02b9f0 comp rx=0 tx=0).stop 2023-10-16T11:24:04.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.945+0000 7f5421213640 1 -- 172.21.15.84:0/2889102918 shutdown_connections 2023-10-16T11:24:04.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.945+0000 7f5421213640 1 --2- 172.21.15.84:0/2889102918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f541c0b1ca0 0x7f541c0d36b0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:04.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.945+0000 7f5421213640 1 -- 172.21.15.84:0/2889102918 >> 172.21.15.84:0/2889102918 conn(0x7f541c0d2b90 msgr2=0x7f541c0d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:04.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.945+0000 7f5421213640 1 -- 172.21.15.84:0/2889102918 shutdown_connections 2023-10-16T11:24:04.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.945+0000 7f5421213640 1 -- 172.21.15.84:0/2889102918 wait complete. 2023-10-16T11:24:04.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.946+0000 7f5421213640 1 Processor -- start 2023-10-16T11:24:04.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.947+0000 7f5421213640 1 -- start start 2023-10-16T11:24:04.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.947+0000 7f5421213640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f541c0b1ca0 0x7f541c09f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:04.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.947+0000 7f5421213640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f541c0b2080 0x7f541c1f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:04.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.948+0000 7f5421213640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f541c0cfc40 0x7f541c0d1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:04.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.948+0000 7f5421213640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c0c8440 con 0x7f541c0b1ca0 2023-10-16T11:24:04.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.948+0000 7f5421213640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c0c8720 con 0x7f541c0cfc40 2023-10-16T11:24:04.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.948+0000 7f5421213640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541c09c970 con 0x7f541c0b2080 2023-10-16T11:24:04.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.948+0000 7f541bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f541c0b1ca0 0x7f541c09f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:04.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.948+0000 7f541bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f541c0b1ca0 0x7f541c09f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45396/0 (socket says 172.21.15.84:45396) 2023-10-16T11:24:04.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.948+0000 7f541bfff640 1 -- 172.21.15.84:0/3787206694 learned_addr learned my addr 172.21.15.84:0/3787206694 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:04.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.948+0000 7f5420a12640 1 --2- 172.21.15.84:0/3787206694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f541c0cfc40 0x7f541c0d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:04.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.948+0000 7f5413fff640 1 --2- 172.21.15.84:0/3787206694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f541c0b2080 0x7f541c1f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:04.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.948+0000 7f541bfff640 1 -- 172.21.15.84:0/3787206694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f541c0b2080 msgr2=0x7f541c1f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:04.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.948+0000 7f541bfff640 1 --2- 172.21.15.84:0/3787206694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f541c0b2080 0x7f541c1f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:04.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.948+0000 7f541bfff640 1 -- 172.21.15.84:0/3787206694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f541c0cfc40 msgr2=0x7f541c0d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:04.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.948+0000 7f541bfff640 1 --2- 172.21.15.84:0/3787206694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f541c0cfc40 0x7f541c0d1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:04.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.948+0000 7f541bfff640 1 -- 172.21.15.84:0/3787206694 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c009400 con 0x7f541c0b1ca0 2023-10-16T11:24:04.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.948+0000 7f5413fff640 1 --2- 172.21.15.84:0/3787206694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f541c0b2080 0x7f541c1f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:04.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.949+0000 7f541bfff640 1 --2- 172.21.15.84:0/3787206694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f541c0b1ca0 0x7f541c09f140 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f540c0097a0 tx=0x7f540c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:04.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.949+0000 7f54197fa640 1 -- 172.21.15.84:0/3787206694 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c004450 con 0x7f541c0b1ca0 2023-10-16T11:24:04.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.949+0000 7f54197fa640 1 -- 172.21.15.84:0/3787206694 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540c03c070 con 0x7f541c0b1ca0 2023-10-16T11:24:04.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.949+0000 7f54197fa640 1 -- 172.21.15.84:0/3787206694 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c003810 con 0x7f541c0b1ca0 2023-10-16T11:24:04.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.949+0000 7f5421213640 1 -- 172.21.15.84:0/3787206694 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541c09cc50 con 0x7f541c0b1ca0 2023-10-16T11:24:04.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.949+0000 7f5421213640 1 -- 172.21.15.84:0/3787206694 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541c09f680 con 0x7f541c0b1ca0 2023-10-16T11:24:04.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.950+0000 7f5421213640 1 -- 172.21.15.84:0/3787206694 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541c09f810 con 0x7f541c0b1ca0 2023-10-16T11:24:04.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.951+0000 7f54197fa640 1 -- 172.21.15.84:0/3787206694 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 128) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f540c003970 con 0x7f541c0b1ca0 2023-10-16T11:24:04.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.952+0000 7f54197fa640 1 --2- 172.21.15.84:0/3787206694 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f53fc06b0b0 0x7f53fc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:04.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.952+0000 7f5413fff640 1 -- 172.21.15.84:0/3787206694 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f53fc06b0b0 msgr2=0x7f53fc06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/855077515 2023-10-16T11:24:04.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.952+0000 7f54197fa640 1 -- 172.21.15.84:0/3787206694 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6139+0+0 (secure 0 0 0) 0x7f540c0ac370 con 0x7f541c0b1ca0 2023-10-16T11:24:04.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.952+0000 7f5413fff640 1 --2- 172.21.15.84:0/3787206694 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f53fc06b0b0 0x7f53fc06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:24:04.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:04.957+0000 7f54197fa640 1 -- 172.21.15.84:0/3787206694 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f540c076fb0 con 0x7f541c0b1ca0 2023-10-16T11:24:04.992 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:04.991+0000 7f32ee044040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:24:04.996 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:04.994+0000 7fe738816040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:24:05.042 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:05.041+0000 7f32ee044040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:24:05.047 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:05.045+0000 7fe738816040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:24:05.102 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:05.101+0000 7f32ee044040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:24:05.111 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:05.109+0000 7fe738816040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:24:05.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.148+0000 7f5421213640 1 -- 172.21.15.84:0/3787206694 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f541c04e9b0 con 0x7f541c0b1ca0 2023-10-16T11:24:05.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.149+0000 7f54197fa640 1 -- 172.21.15.84:0/3787206694 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v204) v1 ==== 93+0+10 (secure 0 0 0) 0x7f540c07a5d0 con 0x7f541c0b1ca0 2023-10-16T11:24:05.151 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:24:05.151 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:05.150+0000 7f32ee044040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:24:05.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.151+0000 7f5421213640 1 -- 172.21.15.84:0/3787206694 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f53fc06b0b0 msgr2=0x7f53fc06d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:24:05.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.151+0000 7f5421213640 1 --2- 172.21.15.84:0/3787206694 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f53fc06b0b0 0x7f53fc06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.151+0000 7f5421213640 1 -- 172.21.15.84:0/3787206694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f541c0b1ca0 msgr2=0x7f541c09f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:05.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.151+0000 7f5421213640 1 --2- 172.21.15.84:0/3787206694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f541c0b1ca0 0x7f541c09f140 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f540c0097a0 tx=0x7f540c0046c0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.151+0000 7f5421213640 1 -- 172.21.15.84:0/3787206694 shutdown_connections 2023-10-16T11:24:05.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.151+0000 7f5421213640 1 --2- 172.21.15.84:0/3787206694 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f53fc06b0b0 0x7f53fc06d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.151+0000 7f5421213640 1 --2- 172.21.15.84:0/3787206694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f541c0cfc40 0x7f541c0d1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.151+0000 7f5421213640 1 --2- 172.21.15.84:0/3787206694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f541c0b2080 0x7f541c1f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.151+0000 7f5421213640 1 --2- 172.21.15.84:0/3787206694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f541c0b1ca0 0x7f541c09f140 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.151+0000 7f5421213640 1 -- 172.21.15.84:0/3787206694 >> 172.21.15.84:0/3787206694 conn(0x7f541c0d2b90 msgr2=0x7f541c0b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:05.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.151+0000 7f5421213640 1 -- 172.21.15.84:0/3787206694 shutdown_connections 2023-10-16T11:24:05.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.152+0000 7f5421213640 1 -- 172.21.15.84:0/3787206694 wait complete. 2023-10-16T11:24:05.167 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:05.165+0000 7fe738816040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:24:05.178 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:24:05.263 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:05.262+0000 7f32ee044040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:24:05.287 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:05.285+0000 7fe738816040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:24:05.326 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:05.325+0000 7f32ee044040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:24:05.352 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:05.350+0000 7fe738816040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:24:05.376 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:05.375+0000 7f32ee044040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:24:05.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.388+0000 7f4646059640 1 Processor -- start 2023-10-16T11:24:05.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.388+0000 7f4646059640 1 -- start start 2023-10-16T11:24:05.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.390+0000 7f4646059640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f46400cded0 0x7f46400b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:05.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.390+0000 7f4646059640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f464015d150 con 0x7f464015cd70 2023-10-16T11:24:05.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.390+0000 7f4646059640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46400d1180 con 0x7f46400b8480 2023-10-16T11:24:05.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.390+0000 7f4646059640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46400d1460 con 0x7f46400cded0 2023-10-16T11:24:05.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.390+0000 7f4645057640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f464015cd70 0x7f46400d2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33918/0 (socket says 172.21.15.84:33918) 2023-10-16T11:24:05.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.390+0000 7f4645057640 1 -- 172.21.15.84:0/840768804 learned_addr learned my addr 172.21.15.84:0/840768804 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:05.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.390+0000 7f4645858640 1 --2- 172.21.15.84:0/840768804 >> v2:172.21.15.157:3300/0 conn(0x7f46400cded0 0x7f46400b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.390+0000 7f4637fff640 1 -- 172.21.15.84:0/840768804 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941765150 0 0) 0x7f464015d150 con 0x7f464015cd70 2023-10-16T11:24:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.390+0000 7f4637fff640 1 -- 172.21.15.84:0/840768804 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f462c003520 con 0x7f464015cd70 2023-10-16T11:24:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.391+0000 7f4637fff640 1 -- 172.21.15.84:0/840768804 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 785242099 0 0) 0x7f462c003520 con 0x7f464015cd70 2023-10-16T11:24:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.391+0000 7f4637fff640 1 -- 172.21.15.84:0/840768804 >> v2:172.21.15.157:3300/0 conn(0x7f46400cded0 msgr2=0x7f46400b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.391+0000 7f4637fff640 1 --2- 172.21.15.84:0/840768804 >> v2:172.21.15.157:3300/0 conn(0x7f46400cded0 0x7f46400b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.391+0000 7f4637fff640 1 -- 172.21.15.84:0/840768804 >> v1:172.21.15.157:6789/0 conn(0x7f46400b8480 legacy=0x7f46400cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.391+0000 7f4637fff640 1 -- 172.21.15.84:0/840768804 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46400d1740 con 0x7f464015cd70 2023-10-16T11:24:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.391+0000 7f4637fff640 1 -- 172.21.15.84:0/840768804 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f463c001670 con 0x7f464015cd70 2023-10-16T11:24:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.391+0000 7f4637fff640 1 -- 172.21.15.84:0/840768804 >> v1:172.21.15.84:6789/0 conn(0x7f464015cd70 legacy=0x7f46400d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.391+0000 7f4637fff640 1 --2- 172.21.15.84:0/840768804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f462c003970 0x7f462c003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:05.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.391+0000 7f4637fff640 1 --2- 172.21.15.84:0/840768804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f462c0047c0 0x7f462c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:05.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.391+0000 7f4637fff640 1 --2- 172.21.15.84:0/840768804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f462c005180 0x7f462c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:05.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.391+0000 7f4637fff640 1 -- 172.21.15.84:0/840768804 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46400d1460 con 0x7f462c0047c0 2023-10-16T11:24:05.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.391+0000 7f4637fff640 1 -- 172.21.15.84:0/840768804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c00bfd0 con 0x7f462c005180 2023-10-16T11:24:05.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.391+0000 7f4637fff640 1 -- 172.21.15.84:0/840768804 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462c00c190 con 0x7f462c003970 2023-10-16T11:24:05.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.391+0000 7f4644856640 1 --2- 172.21.15.84:0/840768804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f462c0047c0 0x7f462c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:05.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.392+0000 7f4645858640 1 --2- 172.21.15.84:0/840768804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f462c005180 0x7f462c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:05.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.392+0000 7f4644856640 1 -- 172.21.15.84:0/840768804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f462c003970 msgr2=0x7f462c003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:05.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.392+0000 7f4644856640 1 --2- 172.21.15.84:0/840768804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f462c003970 0x7f462c003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.392+0000 7f4644856640 1 -- 172.21.15.84:0/840768804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f462c005180 msgr2=0x7f462c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:05.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.392+0000 7f4644856640 1 --2- 172.21.15.84:0/840768804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f462c005180 0x7f462c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.392+0000 7f4644856640 1 -- 172.21.15.84:0/840768804 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462c003520 con 0x7f462c0047c0 2023-10-16T11:24:05.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.392+0000 7f4645858640 1 -- 172.21.15.84:0/840768804 reap_dead start 2023-10-16T11:24:05.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.393+0000 7f4644856640 1 --2- 172.21.15.84:0/840768804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f462c0047c0 0x7f462c004c40 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f4630001860 tx=0x7f462c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94742a615b468881 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:05.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.393+0000 7f4637fff640 1 -- 172.21.15.84:0/840768804 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630032600 con 0x7f462c0047c0 2023-10-16T11:24:05.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.393+0000 7f4637fff640 1 -- 172.21.15.84:0/840768804 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463002e050 con 0x7f462c0047c0 2023-10-16T11:24:05.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.393+0000 7f4637fff640 1 -- 172.21.15.84:0/840768804 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46300329c0 con 0x7f462c0047c0 2023-10-16T11:24:05.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.393+0000 7f4646059640 1 -- 172.21.15.84:0/840768804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f462c0047c0 msgr2=0x7f462c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:05.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.394+0000 7f4646059640 1 --2- 172.21.15.84:0/840768804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f462c0047c0 0x7f462c004c40 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f4630001860 tx=0x7f462c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.394+0000 7f4646059640 1 -- 172.21.15.84:0/840768804 shutdown_connections 2023-10-16T11:24:05.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.394+0000 7f4646059640 1 --2- 172.21.15.84:0/840768804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f462c0047c0 0x7f462c004c40 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.394+0000 7f4646059640 1 -- 172.21.15.84:0/840768804 >> 172.21.15.84:0/840768804 conn(0x7f46400cf570 msgr2=0x7f46400cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:05.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.394+0000 7f4646059640 1 -- 172.21.15.84:0/840768804 shutdown_connections 2023-10-16T11:24:05.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.394+0000 7f4646059640 1 -- 172.21.15.84:0/840768804 wait complete. 2023-10-16T11:24:05.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.395+0000 7f4646059640 1 Processor -- start 2023-10-16T11:24:05.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.396+0000 7f4646059640 1 -- start start 2023-10-16T11:24:05.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.396+0000 7f4646059640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f462c0047c0 0x7f46400cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:05.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.396+0000 7f4646059640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f464015db30 0x7f4640153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:05.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.396+0000 7f4646059640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46400b3f20 0x7f46400b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:05.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.396+0000 7f4646059640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46400b48e0 con 0x7f464015db30 2023-10-16T11:24:05.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.397+0000 7f4646059640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46401508a0 con 0x7f462c0047c0 2023-10-16T11:24:05.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.397+0000 7f4646059640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4640155340 con 0x7f46400b3f20 2023-10-16T11:24:05.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.397+0000 7f4644856640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f464015db30 0x7f4640153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:05.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.397+0000 7f4644856640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f464015db30 0x7f4640153ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45410/0 (socket says 172.21.15.84:45410) 2023-10-16T11:24:05.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.397+0000 7f4644856640 1 -- 172.21.15.84:0/1464028076 learned_addr learned my addr 172.21.15.84:0/1464028076 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:05.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.397+0000 7f4645057640 1 --2- 172.21.15.84:0/1464028076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f462c0047c0 0x7f46400cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:05.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.397+0000 7f4645858640 1 --2- 172.21.15.84:0/1464028076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46400b3f20 0x7f46400b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:05.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.397+0000 7f4644856640 1 -- 172.21.15.84:0/1464028076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46400b3f20 msgr2=0x7f46400b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:05.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.397+0000 7f4644856640 1 --2- 172.21.15.84:0/1464028076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46400b3f20 0x7f46400b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.397+0000 7f4644856640 1 -- 172.21.15.84:0/1464028076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f462c0047c0 msgr2=0x7f46400cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:05.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.397+0000 7f4644856640 1 --2- 172.21.15.84:0/1464028076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f462c0047c0 0x7f46400cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.397+0000 7f4644856640 1 -- 172.21.15.84:0/1464028076 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4640154730 con 0x7f464015db30 2023-10-16T11:24:05.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.397+0000 7f4645858640 1 --2- 172.21.15.84:0/1464028076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46400b3f20 0x7f46400b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:05.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.397+0000 7f4645057640 1 --2- 172.21.15.84:0/1464028076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f462c0047c0 0x7f46400cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:05.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.398+0000 7f4644856640 1 --2- 172.21.15.84:0/1464028076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f464015db30 0x7f4640153ee0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f4630001950 tx=0x7f4630001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:05.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.398+0000 7f4635ffb640 1 -- 172.21.15.84:0/1464028076 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463003c910 con 0x7f464015db30 2023-10-16T11:24:05.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.398+0000 7f4635ffb640 1 -- 172.21.15.84:0/1464028076 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463003ca70 con 0x7f464015db30 2023-10-16T11:24:05.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.398+0000 7f4635ffb640 1 -- 172.21.15.84:0/1464028076 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463003ce00 con 0x7f464015db30 2023-10-16T11:24:05.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.398+0000 7f4646059640 1 -- 172.21.15.84:0/1464028076 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4640154a10 con 0x7f464015db30 2023-10-16T11:24:05.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.398+0000 7f4646059640 1 -- 172.21.15.84:0/1464028076 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4640154fd0 con 0x7f464015db30 2023-10-16T11:24:05.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.399+0000 7f4646059640 1 -- 172.21.15.84:0/1464028076 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4640155160 con 0x7f464015db30 2023-10-16T11:24:05.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.400+0000 7f4635ffb640 1 -- 172.21.15.84:0/1464028076 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 128) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f46300493b0 con 0x7f464015db30 2023-10-16T11:24:05.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.401+0000 7f4635ffb640 1 --2- 172.21.15.84:0/1464028076 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f462406b0b0 0x7f462406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:05.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.401+0000 7f4635ffb640 1 -- 172.21.15.84:0/1464028076 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6139+0+0 (secure 0 0 0) 0x7f46300aafa0 con 0x7f464015db30 2023-10-16T11:24:05.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.401+0000 7f4645057640 1 -- 172.21.15.84:0/1464028076 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f462406b0b0 msgr2=0x7f462406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/855077515 2023-10-16T11:24:05.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.401+0000 7f4645057640 1 --2- 172.21.15.84:0/1464028076 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f462406b0b0 0x7f462406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:24:05.411 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:05.404+0000 7fe738816040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:24:05.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.406+0000 7f4635ffb640 1 -- 172.21.15.84:0/1464028076 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4630075af0 con 0x7f464015db30 2023-10-16T11:24:05.434 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:05.433+0000 7f32ee044040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:24:05.464 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:05.462+0000 7fe738816040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:24:05.485 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:05.484+0000 7f32ee044040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:24:05.517 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:05.514+0000 7fe738816040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:24:05.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.598+0000 7f4646059640 1 -- 172.21.15.84:0/1464028076 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f464004e9b0 con 0x7f464015db30 2023-10-16T11:24:05.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.599+0000 7f4635ffb640 1 -- 172.21.15.84:0/1464028076 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v204) v1 ==== 104+0+11 (secure 0 0 0) 0x7f46300450a0 con 0x7f464015db30 2023-10-16T11:24:05.601 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:24:05.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.601+0000 7f4646059640 1 -- 172.21.15.84:0/1464028076 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f462406b0b0 msgr2=0x7f462406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:24:05.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.601+0000 7f4646059640 1 --2- 172.21.15.84:0/1464028076 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f462406b0b0 0x7f462406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.601+0000 7f4646059640 1 -- 172.21.15.84:0/1464028076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f464015db30 msgr2=0x7f4640153ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:05.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.601+0000 7f4646059640 1 --2- 172.21.15.84:0/1464028076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f464015db30 0x7f4640153ee0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f4630001950 tx=0x7f4630001230 comp rx=0 tx=0).stop 2023-10-16T11:24:05.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.601+0000 7f4646059640 1 -- 172.21.15.84:0/1464028076 shutdown_connections 2023-10-16T11:24:05.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.601+0000 7f4646059640 1 --2- 172.21.15.84:0/1464028076 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f462406b0b0 0x7f462406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.601+0000 7f4646059640 1 --2- 172.21.15.84:0/1464028076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46400b3f20 0x7f46400b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.601+0000 7f4646059640 1 --2- 172.21.15.84:0/1464028076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f464015db30 0x7f4640153ee0 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.601+0000 7f4646059640 1 --2- 172.21.15.84:0/1464028076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f462c0047c0 0x7f46400cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.601+0000 7f4646059640 1 -- 172.21.15.84:0/1464028076 >> 172.21.15.84:0/1464028076 conn(0x7f46400cf570 msgr2=0x7f46400b7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:05.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.601+0000 7f4646059640 1 -- 172.21.15.84:0/1464028076 shutdown_connections 2023-10-16T11:24:05.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.601+0000 7f4646059640 1 -- 172.21.15.84:0/1464028076 wait complete. 2023-10-16T11:24:05.621 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:24:05.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.824+0000 7f8f0e597640 1 Processor -- start 2023-10-16T11:24:05.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.824+0000 7f8f0e597640 1 -- start start 2023-10-16T11:24:05.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.826+0000 7f8f0e597640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f08159ab0 0x7f8f080ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:05.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.826+0000 7f8f0e597640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8f0806c140 0x7f8f0806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:05.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.826+0000 7f8f0e597640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f08070360 con 0x7f8f08159e90 2023-10-16T11:24:05.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.826+0000 7f8f0e597640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0806cb10 con 0x7f8f0806c140 2023-10-16T11:24:05.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.826+0000 7f8f0e597640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0815d890 con 0x7f8f08159ab0 2023-10-16T11:24:05.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.826+0000 7f8f0cb34640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8f0806c140 0x7f8f0806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:05.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.826+0000 7f8efffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8f08159e90 0x7f8f0806fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48556/0 (socket says 172.21.15.84:48556) 2023-10-16T11:24:05.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.826+0000 7f8efffff640 1 -- 172.21.15.84:0/2670270625 learned_addr learned my addr 172.21.15.84:0/2670270625 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:05.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.827+0000 7f8f07fff640 1 --2- 172.21.15.84:0/2670270625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f08159ab0 0x7f8f080ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:05.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.827+0000 7f8f0cb34640 1 -- 172.21.15.84:0/2670270625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f08159ab0 msgr2=0x7f8f080ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:05.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.827+0000 7f8f0cb34640 1 --2- 172.21.15.84:0/2670270625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f08159ab0 0x7f8f080ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.827+0000 7f8f0cb34640 1 -- 172.21.15.84:0/2670270625 >> v1:172.21.15.84:6789/0 conn(0x7f8f08159e90 legacy=0x7f8f0806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:05.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.827+0000 7f8f077fe640 1 -- 172.21.15.84:0/2670270625 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749149011 0 0) 0x7f8f08070360 con 0x7f8f08159e90 2023-10-16T11:24:05.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.827+0000 7f8f0cb34640 1 -- 172.21.15.84:0/2670270625 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f080cd1f0 con 0x7f8f0806c140 2023-10-16T11:24:05.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.828+0000 7f8f0cb34640 1 --2- 172.21.15.84:0/2670270625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f0806c140 0x7f8f0806c5d0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009dc0 tx=0x7f8f0002bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eecee2a965ce0897 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:05.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.828+0000 7f8f077fe640 1 -- 172.21.15.84:0/2670270625 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0002cb20 con 0x7f8f0806c140 2023-10-16T11:24:05.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.828+0000 7f8f077fe640 1 -- 172.21.15.84:0/2670270625 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0002cc80 con 0x7f8f0806c140 2023-10-16T11:24:05.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.828+0000 7f8f077fe640 1 -- 172.21.15.84:0/2670270625 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00034440 con 0x7f8f0806c140 2023-10-16T11:24:05.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.829+0000 7f8f0e597640 1 -- 172.21.15.84:0/2670270625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f0806c140 msgr2=0x7f8f0806c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:05.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.829+0000 7f8f0e597640 1 --2- 172.21.15.84:0/2670270625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f0806c140 0x7f8f0806c5d0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009dc0 tx=0x7f8f0002bd70 comp rx=0 tx=0).stop 2023-10-16T11:24:05.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.829+0000 7f8f0e597640 1 -- 172.21.15.84:0/2670270625 shutdown_connections 2023-10-16T11:24:05.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.829+0000 7f8f0e597640 1 --2- 172.21.15.84:0/2670270625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f0806c140 0x7f8f0806c5d0 unknown :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.829+0000 7f8f0e597640 1 --2- 172.21.15.84:0/2670270625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f08159ab0 0x7f8f080ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.829+0000 7f8f0e597640 1 -- 172.21.15.84:0/2670270625 >> 172.21.15.84:0/2670270625 conn(0x7f8f080ce9b0 msgr2=0x7f8f080cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:05.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.829+0000 7f8f0e597640 1 -- 172.21.15.84:0/2670270625 shutdown_connections 2023-10-16T11:24:05.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.829+0000 7f8f0e597640 1 -- 172.21.15.84:0/2670270625 wait complete. 2023-10-16T11:24:05.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.831+0000 7f8f0e597640 1 Processor -- start 2023-10-16T11:24:05.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.831+0000 7f8f0e597640 1 -- start start 2023-10-16T11:24:05.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.832+0000 7f8f0e597640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f0806c140 0x7f8f080c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:05.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.832+0000 7f8f0e597640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f08159ab0 0x7f8f080c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:05.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.832+0000 7f8f0e597640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f08159e90 0x7f8f080c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:05.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.832+0000 7f8f0e597640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f080c84e0 con 0x7f8f0806c140 2023-10-16T11:24:05.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.832+0000 7f8f0e597640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f080c87c0 con 0x7f8f08159ab0 2023-10-16T11:24:05.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.832+0000 7f8f0e597640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f080c8aa0 con 0x7f8f08159e90 2023-10-16T11:24:05.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.832+0000 7f8f07fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f0806c140 0x7f8f080c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:05.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.832+0000 7f8f07fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f0806c140 0x7f8f080c5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44570/0 (socket says 172.21.15.84:44570) 2023-10-16T11:24:05.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.832+0000 7f8f07fff640 1 -- 172.21.15.84:0/1373099335 learned_addr learned my addr 172.21.15.84:0/1373099335 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:05.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.832+0000 7f8f07fff640 1 -- 172.21.15.84:0/1373099335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f08159e90 msgr2=0x7f8f080c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:05.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.832+0000 7f8f07fff640 1 --2- 172.21.15.84:0/1373099335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f08159e90 0x7f8f080c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.832+0000 7f8f07fff640 1 -- 172.21.15.84:0/1373099335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f08159ab0 msgr2=0x7f8f080c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:05.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.832+0000 7f8f07fff640 1 --2- 172.21.15.84:0/1373099335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f08159ab0 0x7f8f080c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:05.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.832+0000 7f8f07fff640 1 -- 172.21.15.84:0/1373099335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f00009a70 con 0x7f8f0806c140 2023-10-16T11:24:05.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.833+0000 7f8f07fff640 1 --2- 172.21.15.84:0/1373099335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f0806c140 0x7f8f080c5f20 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800d5f0 tx=0x7f8ef800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:05.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.833+0000 7f8f057fa640 1 -- 172.21.15.84:0/1373099335 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80125f0 con 0x7f8f0806c140 2023-10-16T11:24:05.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.833+0000 7f8f057fa640 1 -- 172.21.15.84:0/1373099335 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef8012750 con 0x7f8f0806c140 2023-10-16T11:24:05.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.833+0000 7f8f0e597640 1 -- 172.21.15.84:0/1373099335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f080c8d80 con 0x7f8f0806c140 2023-10-16T11:24:05.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.833+0000 7f8f057fa640 1 -- 172.21.15.84:0/1373099335 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8012ae0 con 0x7f8f0806c140 2023-10-16T11:24:05.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.833+0000 7f8f0e597640 1 -- 172.21.15.84:0/1373099335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f081560d0 con 0x7f8f0806c140 2023-10-16T11:24:05.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.834+0000 7f8f0e597640 1 -- 172.21.15.84:0/1373099335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0804e160 con 0x7f8f0806c140 2023-10-16T11:24:05.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.835+0000 7f8f057fa640 1 -- 172.21.15.84:0/1373099335 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 128) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f8ef8012c40 con 0x7f8f0806c140 2023-10-16T11:24:05.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.836+0000 7f8f057fa640 1 --2- 172.21.15.84:0/1373099335 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f8ee806b0b0 0x7f8ee806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:05.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.836+0000 7f8f057fa640 1 -- 172.21.15.84:0/1373099335 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6139+0+0 (secure 0 0 0) 0x7f8ef808a210 con 0x7f8f0806c140 2023-10-16T11:24:05.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.836+0000 7f8efffff640 1 -- 172.21.15.84:0/1373099335 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f8ee806b0b0 msgr2=0x7f8ee806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/855077515 2023-10-16T11:24:05.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.836+0000 7f8efffff640 1 --2- 172.21.15.84:0/1373099335 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f8ee806b0b0 0x7f8ee806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:24:05.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:05.840+0000 7f8f057fa640 1 -- 172.21.15.84:0/1373099335 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8ef8054c80 con 0x7f8f0806c140 2023-10-16T11:24:05.871 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:05.870+0000 7f32ee044040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:24:05.914 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:05.913+0000 7fe738816040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:24:06.032 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:06.031+0000 7f32ee044040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:24:06.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.032+0000 7f8f0e597640 1 -- 172.21.15.84:0/1373099335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8f08144e60 con 0x7f8f0806c140 2023-10-16T11:24:06.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.033+0000 7f8f057fa640 1 -- 172.21.15.84:0/1373099335 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v204) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8ef80582a0 con 0x7f8f0806c140 2023-10-16T11:24:06.036 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:24:06.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.036+0000 7f8f0e597640 1 -- 172.21.15.84:0/1373099335 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f8ee806b0b0 msgr2=0x7f8ee806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:24:06.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.036+0000 7f8f0e597640 1 --2- 172.21.15.84:0/1373099335 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f8ee806b0b0 0x7f8ee806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.036+0000 7f8f0e597640 1 -- 172.21.15.84:0/1373099335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f0806c140 msgr2=0x7f8f080c5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:06.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.036+0000 7f8f0e597640 1 --2- 172.21.15.84:0/1373099335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f0806c140 0x7f8f080c5f20 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800d5f0 tx=0x7f8ef800dab0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.036+0000 7f8f0e597640 1 -- 172.21.15.84:0/1373099335 shutdown_connections 2023-10-16T11:24:06.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.036+0000 7f8f0e597640 1 --2- 172.21.15.84:0/1373099335 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f8ee806b0b0 0x7f8ee806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.036+0000 7f8f0e597640 1 --2- 172.21.15.84:0/1373099335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f08159e90 0x7f8f080c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.036+0000 7f8f0e597640 1 --2- 172.21.15.84:0/1373099335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f08159ab0 0x7f8f080c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.036+0000 7f8f0e597640 1 --2- 172.21.15.84:0/1373099335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f0806c140 0x7f8f080c5f20 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.036+0000 7f8f0e597640 1 -- 172.21.15.84:0/1373099335 >> 172.21.15.84:0/1373099335 conn(0x7f8f080ce9b0 msgr2=0x7f8f080afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:06.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.036+0000 7f8f0e597640 1 -- 172.21.15.84:0/1373099335 shutdown_connections 2023-10-16T11:24:06.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.036+0000 7f8f0e597640 1 -- 172.21.15.84:0/1373099335 wait complete. 2023-10-16T11:24:06.057 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T11:24:06.077 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:06.075+0000 7fe738816040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:24:06.140 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:06.139+0000 7f32ee044040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:24:06.190 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:06.188+0000 7fe738816040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:24:06.190 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:06.189+0000 7f32ee044040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:24:06.241 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:06.239+0000 7fe738816040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:24:06.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.259+0000 7f9ec155e640 1 Processor -- start 2023-10-16T11:24:06.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.260+0000 7f9ec155e640 1 -- start start 2023-10-16T11:24:06.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.261+0000 7f9ec155e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9ebc15cd70 0x7f9ebc0d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:06.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.261+0000 7f9ec155e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ebc15d150 con 0x7f9ebc0cded0 2023-10-16T11:24:06.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.261+0000 7f9ec155e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ebc0d1180 con 0x7f9ebc0b8480 2023-10-16T11:24:06.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.261+0000 7f9ec155e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ebc0d1460 con 0x7f9ebc15cd70 2023-10-16T11:24:06.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.261+0000 7f9ec0d5d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9ebc0cded0 0x7f9ebc0b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48560/0 (socket says 172.21.15.84:48560) 2023-10-16T11:24:06.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.261+0000 7f9ec0d5d640 1 -- 172.21.15.84:0/389737126 learned_addr learned my addr 172.21.15.84:0/389737126 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:06.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.262+0000 7f9ebbfff640 1 --2- 172.21.15.84:0/389737126 >> v2:172.21.15.157:3300/0 conn(0x7f9ebc15cd70 0x7f9ebc0d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:06.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.262+0000 7f9ebb7fe640 1 -- 172.21.15.84:0/389737126 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3919854957 0 0) 0x7f9ebc15d150 con 0x7f9ebc0cded0 2023-10-16T11:24:06.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.262+0000 7f9ebb7fe640 1 -- 172.21.15.84:0/389737126 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ea8003520 con 0x7f9ebc0cded0 2023-10-16T11:24:06.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.262+0000 7f9ebb7fe640 1 -- 172.21.15.84:0/389737126 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3299793536 0 0) 0x7f9ea8003520 con 0x7f9ebc0cded0 2023-10-16T11:24:06.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.262+0000 7f9ebb7fe640 1 -- 172.21.15.84:0/389737126 >> v2:172.21.15.157:3300/0 conn(0x7f9ebc15cd70 msgr2=0x7f9ebc0d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:06.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.262+0000 7f9ebb7fe640 1 --2- 172.21.15.84:0/389737126 >> v2:172.21.15.157:3300/0 conn(0x7f9ebc15cd70 0x7f9ebc0d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.262+0000 7f9ebb7fe640 1 -- 172.21.15.84:0/389737126 >> v1:172.21.15.157:6789/0 conn(0x7f9ebc0b8480 legacy=0x7f9ebc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:06.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.262+0000 7f9ebb7fe640 1 -- 172.21.15.84:0/389737126 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc0d1740 con 0x7f9ebc0cded0 2023-10-16T11:24:06.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.262+0000 7f9ebb7fe640 1 -- 172.21.15.84:0/389737126 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f9eb40019f0 con 0x7f9ebc0cded0 2023-10-16T11:24:06.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.262+0000 7f9ebb7fe640 1 -- 172.21.15.84:0/389737126 >> v1:172.21.15.84:6789/0 conn(0x7f9ebc0cded0 legacy=0x7f9ebc0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:06.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.262+0000 7f9ebb7fe640 1 --2- 172.21.15.84:0/389737126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ea80039c0 0x7f9ea8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:06.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.262+0000 7f9ebb7fe640 1 --2- 172.21.15.84:0/389737126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea80047c0 0x7f9ea8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:06.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.262+0000 7f9ebb7fe640 1 --2- 172.21.15.84:0/389737126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea8005180 0x7f9ea800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:06.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.262+0000 7f9ebb7fe640 1 -- 172.21.15.84:0/389737126 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc0d1460 con 0x7f9ea8005180 2023-10-16T11:24:06.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.262+0000 7f9ebb7fe640 1 -- 172.21.15.84:0/389737126 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea800c080 con 0x7f9ea80047c0 2023-10-16T11:24:06.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.262+0000 7f9ebb7fe640 1 -- 172.21.15.84:0/389737126 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea800c240 con 0x7f9ea80039c0 2023-10-16T11:24:06.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.263+0000 7f9ec0d5d640 1 --2- 172.21.15.84:0/389737126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea8005180 0x7f9ea800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:06.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.263+0000 7f9eb3fff640 1 --2- 172.21.15.84:0/389737126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea80047c0 0x7f9ea8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:06.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.263+0000 7f9ebbfff640 1 --2- 172.21.15.84:0/389737126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ea80039c0 0x7f9ea8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:06.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.263+0000 7f9ec0d5d640 1 -- 172.21.15.84:0/389737126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ea80039c0 msgr2=0x7f9ea8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:06.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.263+0000 7f9ec0d5d640 1 --2- 172.21.15.84:0/389737126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ea80039c0 0x7f9ea8003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.263+0000 7f9ec0d5d640 1 -- 172.21.15.84:0/389737126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea80047c0 msgr2=0x7f9ea8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:06.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.263+0000 7f9ec0d5d640 1 --2- 172.21.15.84:0/389737126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea80047c0 0x7f9ea8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.263+0000 7f9ec0d5d640 1 -- 172.21.15.84:0/389737126 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea8003520 con 0x7f9ea8005180 2023-10-16T11:24:06.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.264+0000 7f9ec0d5d640 1 -- 172.21.15.84:0/389737126 reap_dead start 2023-10-16T11:24:06.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.264+0000 7f9ec0d5d640 1 --2- 172.21.15.84:0/389737126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea8005180 0x7f9ea800b980 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400a760 tx=0x7f9ea80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909036789591c829 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:06.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.264+0000 7f9ebb7fe640 1 -- 172.21.15.84:0/389737126 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4034630 con 0x7f9ea8005180 2023-10-16T11:24:06.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.264+0000 7f9ebb7fe640 1 -- 172.21.15.84:0/389737126 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb402f070 con 0x7f9ea8005180 2023-10-16T11:24:06.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.264+0000 7f9ebb7fe640 1 -- 172.21.15.84:0/389737126 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4034a10 con 0x7f9ea8005180 2023-10-16T11:24:06.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.265+0000 7f9ec155e640 1 -- 172.21.15.84:0/389737126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea8005180 msgr2=0x7f9ea800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:06.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.265+0000 7f9ec155e640 1 --2- 172.21.15.84:0/389737126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea8005180 0x7f9ea800b980 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400a760 tx=0x7f9ea80057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.265+0000 7f9ec155e640 1 -- 172.21.15.84:0/389737126 shutdown_connections 2023-10-16T11:24:06.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.265+0000 7f9ec155e640 1 --2- 172.21.15.84:0/389737126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea8005180 0x7f9ea800b980 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.265+0000 7f9ec155e640 1 -- 172.21.15.84:0/389737126 >> 172.21.15.84:0/389737126 conn(0x7f9ebc0cf570 msgr2=0x7f9ebc0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:06.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.265+0000 7f9ec155e640 1 -- 172.21.15.84:0/389737126 shutdown_connections 2023-10-16T11:24:06.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.265+0000 7f9ec155e640 1 -- 172.21.15.84:0/389737126 wait complete. 2023-10-16T11:24:06.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.267+0000 7f9ec155e640 1 Processor -- start 2023-10-16T11:24:06.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.267+0000 7f9ec155e640 1 -- start start 2023-10-16T11:24:06.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.268+0000 7f9ec155e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea8005180 0x7f9ebc0cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:06.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.268+0000 7f9ec155e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ebc0b0e60 0x7f9ebc0b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:06.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.268+0000 7f9ec155e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ebc0b12e0 0x7f9ebc0b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:06.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.268+0000 7f9ec155e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc0a9d00 con 0x7f9ebc0b12e0 2023-10-16T11:24:06.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.268+0000 7f9ec155e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc0b2990 con 0x7f9ea8005180 2023-10-16T11:24:06.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.268+0000 7f9ec155e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc0a94f0 con 0x7f9ebc0b0e60 2023-10-16T11:24:06.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.268+0000 7f9ec0d5d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ebc0b12e0 0x7f9ebc0b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:06.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.268+0000 7f9ec0d5d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ebc0b12e0 0x7f9ebc0b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44588/0 (socket says 172.21.15.84:44588) 2023-10-16T11:24:06.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.268+0000 7f9ec0d5d640 1 -- 172.21.15.84:0/3172342701 learned_addr learned my addr 172.21.15.84:0/3172342701 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:06.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.268+0000 7f9eb3fff640 1 --2- 172.21.15.84:0/3172342701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ebc0b0e60 0x7f9ebc0b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:06.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.268+0000 7f9ebbfff640 1 --2- 172.21.15.84:0/3172342701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea8005180 0x7f9ebc0cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:06.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.269+0000 7f9ec0d5d640 1 -- 172.21.15.84:0/3172342701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ebc0b0e60 msgr2=0x7f9ebc0b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:06.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.269+0000 7f9ec0d5d640 1 --2- 172.21.15.84:0/3172342701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ebc0b0e60 0x7f9ebc0b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.269+0000 7f9ec0d5d640 1 -- 172.21.15.84:0/3172342701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea8005180 msgr2=0x7f9ebc0cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:06.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.269+0000 7f9ec0d5d640 1 --2- 172.21.15.84:0/3172342701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea8005180 0x7f9ebc0cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.269+0000 7f9ec0d5d640 1 -- 172.21.15.84:0/3172342701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc0b2f10 con 0x7f9ebc0b12e0 2023-10-16T11:24:06.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.269+0000 7f9ebbfff640 1 --2- 172.21.15.84:0/3172342701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea8005180 0x7f9ebc0cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:06.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.270+0000 7f9ec0d5d640 1 --2- 172.21.15.84:0/3172342701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ebc0b12e0 0x7f9ebc0b4060 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4002230 tx=0x7f9eb4001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:06.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.270+0000 7f9eb97fa640 1 -- 172.21.15.84:0/3172342701 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4001930 con 0x7f9ebc0b12e0 2023-10-16T11:24:06.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.270+0000 7f9eb97fa640 1 -- 172.21.15.84:0/3172342701 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb4004b60 con 0x7f9ebc0b12e0 2023-10-16T11:24:06.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.270+0000 7f9eb97fa640 1 -- 172.21.15.84:0/3172342701 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb40347b0 con 0x7f9ebc0b12e0 2023-10-16T11:24:06.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.270+0000 7f9ec155e640 1 -- 172.21.15.84:0/3172342701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ebc0a9740 con 0x7f9ebc0b12e0 2023-10-16T11:24:06.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.270+0000 7f9ec155e640 1 -- 172.21.15.84:0/3172342701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ebc0b2ba0 con 0x7f9ebc0b12e0 2023-10-16T11:24:06.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.272+0000 7f9ec155e640 1 -- 172.21.15.84:0/3172342701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ebc0b30a0 con 0x7f9ebc0b12e0 2023-10-16T11:24:06.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.272+0000 7f9eb97fa640 1 -- 172.21.15.84:0/3172342701 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 128) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f9eb4031030 con 0x7f9ebc0b12e0 2023-10-16T11:24:06.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.273+0000 7f9eb97fa640 1 --2- 172.21.15.84:0/3172342701 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f9e9c06b0b0 0x7f9e9c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:06.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.273+0000 7f9ebbfff640 1 -- 172.21.15.84:0/3172342701 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f9e9c06b0b0 msgr2=0x7f9e9c06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/855077515 2023-10-16T11:24:06.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.273+0000 7f9eb97fa640 1 -- 172.21.15.84:0/3172342701 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6139+0+0 (secure 0 0 0) 0x7f9eb40ab010 con 0x7f9ebc0b12e0 2023-10-16T11:24:06.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.273+0000 7f9ebbfff640 1 --2- 172.21.15.84:0/3172342701 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f9e9c06b0b0 0x7f9e9c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:24:06.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.277+0000 7f9eb97fa640 1 -- 172.21.15.84:0/3172342701 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9eb4075b70 con 0x7f9ebc0b12e0 2023-10-16T11:24:06.299 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:06.299+0000 7f32ee044040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:24:06.383 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:06.381+0000 7fe738816040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:24:06.409 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:06.408+0000 7f32ee044040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:24:06.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.474+0000 7f9ebbfff640 1 -- 172.21.15.84:0/3172342701 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f9e9c06b0b0 msgr2=0x7f9e9c06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/855077515 2023-10-16T11:24:06.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.474+0000 7f9ebbfff640 1 --2- 172.21.15.84:0/3172342701 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f9e9c06b0b0 0x7f9e9c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:24:06.497 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:06.495+0000 7fe738816040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:24:06.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.502+0000 7f9ec155e640 1 -- 172.21.15.84:0/3172342701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9ebc04e9b0 con 0x7f9ebc0b12e0 2023-10-16T11:24:06.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.503+0000 7f9eb97fa640 1 -- 172.21.15.84:0/3172342701 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v204) v1 ==== 80+0+469 (secure 0 0 0) 0x7f9eb4079190 con 0x7f9ebc0b12e0 2023-10-16T11:24:06.505 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3564300528 2023-10-17T11:22:56.900947+0000 2023-10-16T11:24:06.505 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2349522146 2023-10-17T11:22:56.900947+0000 2023-10-16T11:24:06.505 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2191803332 2023-10-17T11:22:56.900947+0000 2023-10-16T11:24:06.505 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2199276620 2023-10-17T11:22:56.900947+0000 2023-10-16T11:24:06.506 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/4262224959 2023-10-17T11:22:56.900947+0000 2023-10-16T11:24:06.506 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/956990500 2023-10-17T11:22:56.900947+0000 2023-10-16T11:24:06.506 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6833/1985463812 2023-10-17T11:22:56.900947+0000 2023-10-16T11:24:06.506 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6832/1985463812 2023-10-17T11:22:56.900947+0000 2023-10-16T11:24:06.507 INFO:teuthology.orchestra.run.smithi084.stderr:listed 8 entries 2023-10-16T11:24:06.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.505+0000 7f9ec155e640 1 -- 172.21.15.84:0/3172342701 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f9e9c06b0b0 msgr2=0x7f9e9c06d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:24:06.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.505+0000 7f9ec155e640 1 --2- 172.21.15.84:0/3172342701 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f9e9c06b0b0 0x7f9e9c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.505+0000 7f9ec155e640 1 -- 172.21.15.84:0/3172342701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ebc0b12e0 msgr2=0x7f9ebc0b4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:06.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.505+0000 7f9ec155e640 1 --2- 172.21.15.84:0/3172342701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ebc0b12e0 0x7f9ebc0b4060 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4002230 tx=0x7f9eb4001cb0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.505+0000 7f9ec155e640 1 -- 172.21.15.84:0/3172342701 shutdown_connections 2023-10-16T11:24:06.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.505+0000 7f9ec155e640 1 --2- 172.21.15.84:0/3172342701 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f9e9c06b0b0 0x7f9e9c06d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.505+0000 7f9ec155e640 1 --2- 172.21.15.84:0/3172342701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ebc0b12e0 0x7f9ebc0b4060 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.505+0000 7f9ec155e640 1 --2- 172.21.15.84:0/3172342701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ebc0b0e60 0x7f9ebc0b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.505+0000 7f9ec155e640 1 --2- 172.21.15.84:0/3172342701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea8005180 0x7f9ebc0cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:06.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.505+0000 7f9ec155e640 1 -- 172.21.15.84:0/3172342701 >> 172.21.15.84:0/3172342701 conn(0x7f9ebc0cf570 msgr2=0x7f9ebc0b7380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:06.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.505+0000 7f9ec155e640 1 -- 172.21.15.84:0/3172342701 shutdown_connections 2023-10-16T11:24:06.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:06.506+0000 7f9ec155e640 1 -- 172.21.15.84:0/3172342701 wait complete. 2023-10-16T11:24:06.525 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T11:24:06.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:24:06.525 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T11:24:06.566 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:06.565+0000 7f32ee044040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:24:06.615 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:06.614+0000 7f32ee044040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:24:06.669 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T11:24:06.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:24:06.669 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T11:24:06.675 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:06.670+0000 7fe738816040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:24:06.715 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:24:06.716 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:24:06.724 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:06.722+0000 7fe738816040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:24:06.737 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:06.737+0000 7f32ee044040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:24:06.771 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:24:06.772 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:24:06.772 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:24:06.828 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:24:06.829 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:24:06.829 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:24:06.846 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:06.844+0000 7fe738816040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:24:06.887 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:24:06.887 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:24:07.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.092+0000 7ffaa19c7640 1 Processor -- start 2023-10-16T11:24:07.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.092+0000 7ffaa19c7640 1 -- start start 2023-10-16T11:24:07.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.093+0000 7ffaa19c7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa9c0afee0 0x7ffa9c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.093+0000 7ffaa19c7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9c0cee00 con 0x7ffa9c0cf730 2023-10-16T11:24:07.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.093+0000 7ffaa19c7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9c0aeba0 con 0x7ffa9c15b010 2023-10-16T11:24:07.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.093+0000 7ffaa19c7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9c0ae270 con 0x7ffa9c0afee0 2023-10-16T11:24:07.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.094+0000 7ffa9a7fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ffa9c0cf730 0x7ffa9c0af030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48576/0 (socket says 172.21.15.84:48576) 2023-10-16T11:24:07.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.094+0000 7ffa9a7fc640 1 -- 172.21.15.84:0/3857248351 learned_addr learned my addr 172.21.15.84:0/3857248351 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:07.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.094+0000 7ffa9b7fe640 1 --2- 172.21.15.84:0/3857248351 >> v2:172.21.15.157:3300/0 conn(0x7ffa9c0afee0 0x7ffa9c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:07.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.094+0000 7ffa99ffb640 1 -- 172.21.15.84:0/3857248351 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2894064627 0 0) 0x7ffa90001110 con 0x7ffa9c0cf730 2023-10-16T11:24:07.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.094+0000 7ffa99ffb640 1 -- 172.21.15.84:0/3857248351 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa84003520 con 0x7ffa9c0cf730 2023-10-16T11:24:07.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.095+0000 7ffa99ffb640 1 -- 172.21.15.84:0/3857248351 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 391719776 0 0) 0x7ffa84003520 con 0x7ffa9c0cf730 2023-10-16T11:24:07.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.095+0000 7ffa99ffb640 1 -- 172.21.15.84:0/3857248351 >> v2:172.21.15.157:3300/0 conn(0x7ffa9c0afee0 msgr2=0x7ffa9c0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:07.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.095+0000 7ffa99ffb640 1 --2- 172.21.15.84:0/3857248351 >> v2:172.21.15.157:3300/0 conn(0x7ffa9c0afee0 0x7ffa9c0ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.095+0000 7ffa99ffb640 1 -- 172.21.15.84:0/3857248351 >> v1:172.21.15.157:6789/0 conn(0x7ffa9c15b010 legacy=0x7ffa9c0cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:07.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.095+0000 7ffa99ffb640 1 -- 172.21.15.84:0/3857248351 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c0ae550 con 0x7ffa9c0cf730 2023-10-16T11:24:07.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.095+0000 7ffa99ffb640 1 -- 172.21.15.84:0/3857248351 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ffa90001800 con 0x7ffa9c0cf730 2023-10-16T11:24:07.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.095+0000 7ffa99ffb640 1 -- 172.21.15.84:0/3857248351 >> v1:172.21.15.84:6789/0 conn(0x7ffa9c0cf730 legacy=0x7ffa9c0af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:07.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.095+0000 7ffa99ffb640 1 --2- 172.21.15.84:0/3857248351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa84003970 0x7ffa84003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.095+0000 7ffa99ffb640 1 --2- 172.21.15.84:0/3857248351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa840047c0 0x7ffa84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.095+0000 7ffa99ffb640 1 --2- 172.21.15.84:0/3857248351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa84005180 0x7ffa8400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.095+0000 7ffa99ffb640 1 -- 172.21.15.84:0/3857248351 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c0ae270 con 0x7ffa84003970 2023-10-16T11:24:07.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.095+0000 7ffa99ffb640 1 -- 172.21.15.84:0/3857248351 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8400c030 con 0x7ffa84005180 2023-10-16T11:24:07.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.095+0000 7ffa99ffb640 1 -- 172.21.15.84:0/3857248351 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8400c1f0 con 0x7ffa840047c0 2023-10-16T11:24:07.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.095+0000 7ffa9affd640 1 --2- 172.21.15.84:0/3857248351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa84003970 0x7ffa84003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:07.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.096+0000 7ffa9a7fc640 1 --2- 172.21.15.84:0/3857248351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa840047c0 0x7ffa84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:07.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.096+0000 7ffa9affd640 1 -- 172.21.15.84:0/3857248351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa840047c0 msgr2=0x7ffa84004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:07.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.096+0000 7ffa9affd640 1 --2- 172.21.15.84:0/3857248351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa840047c0 0x7ffa84004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.096+0000 7ffa9b7fe640 1 --2- 172.21.15.84:0/3857248351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa84005180 0x7ffa8400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:07.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.096+0000 7ffa9affd640 1 -- 172.21.15.84:0/3857248351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa84005180 msgr2=0x7ffa8400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:07.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.096+0000 7ffa9affd640 1 --2- 172.21.15.84:0/3857248351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa84005180 0x7ffa8400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.096+0000 7ffa9affd640 1 -- 172.21.15.84:0/3857248351 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa84003520 con 0x7ffa84003970 2023-10-16T11:24:07.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.096+0000 7ffa9b7fe640 1 -- 172.21.15.84:0/3857248351 reap_dead start 2023-10-16T11:24:07.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.097+0000 7ffa9affd640 1 --2- 172.21.15.84:0/3857248351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa84003970 0x7ffa84003d90 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c001770 tx=0x7ffa840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366dd23c6e54761e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:07.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.097+0000 7ffa99ffb640 1 -- 172.21.15.84:0/3857248351 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c0052d0 con 0x7ffa84003970 2023-10-16T11:24:07.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.097+0000 7ffa99ffb640 1 -- 172.21.15.84:0/3857248351 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8c02e050 con 0x7ffa84003970 2023-10-16T11:24:07.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.097+0000 7ffa99ffb640 1 -- 172.21.15.84:0/3857248351 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c003640 con 0x7ffa84003970 2023-10-16T11:24:07.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.098+0000 7ffaa19c7640 1 -- 172.21.15.84:0/3857248351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa84003970 msgr2=0x7ffa84003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:07.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.098+0000 7ffaa19c7640 1 --2- 172.21.15.84:0/3857248351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa84003970 0x7ffa84003d90 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c001770 tx=0x7ffa840057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.098+0000 7ffaa19c7640 1 -- 172.21.15.84:0/3857248351 shutdown_connections 2023-10-16T11:24:07.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.098+0000 7ffaa19c7640 1 --2- 172.21.15.84:0/3857248351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa84003970 0x7ffa84003d90 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.098+0000 7ffaa19c7640 1 -- 172.21.15.84:0/3857248351 >> 172.21.15.84:0/3857248351 conn(0x7ffa9c0d0ad0 msgr2=0x7ffa9c0d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:07.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.098+0000 7ffaa19c7640 1 -- 172.21.15.84:0/3857248351 shutdown_connections 2023-10-16T11:24:07.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.098+0000 7ffaa19c7640 1 -- 172.21.15.84:0/3857248351 wait complete. 2023-10-16T11:24:07.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.101+0000 7ffaa19c7640 1 Processor -- start 2023-10-16T11:24:07.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.101+0000 7ffaa19c7640 1 -- start start 2023-10-16T11:24:07.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.102+0000 7ffaa19c7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa84003970 0x7ffa9c154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.102+0000 7ffaa19c7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9c0b03c0 0x7ffa9c06ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.102+0000 7ffaa19c7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa9c0b0840 0x7ffa9c06ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.102+0000 7ffaa19c7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c0b0ef0 con 0x7ffa9c0b0840 2023-10-16T11:24:07.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.102+0000 7ffaa19c7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c0b1f00 con 0x7ffa84003970 2023-10-16T11:24:07.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.102+0000 7ffaa19c7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c0b1140 con 0x7ffa9c0b03c0 2023-10-16T11:24:07.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.103+0000 7ffa9b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa9c0b0840 0x7ffa9c06ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:07.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.103+0000 7ffa9b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa9c0b0840 0x7ffa9c06ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44598/0 (socket says 172.21.15.84:44598) 2023-10-16T11:24:07.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.103+0000 7ffa9b7fe640 1 -- 172.21.15.84:0/1571865856 learned_addr learned my addr 172.21.15.84:0/1571865856 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:07.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.103+0000 7ffa9a7fc640 1 --2- 172.21.15.84:0/1571865856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9c0b03c0 0x7ffa9c06ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:07.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.103+0000 7ffa9affd640 1 --2- 172.21.15.84:0/1571865856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa84003970 0x7ffa9c154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:07.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.103+0000 7ffa9b7fe640 1 -- 172.21.15.84:0/1571865856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9c0b03c0 msgr2=0x7ffa9c06ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:07.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.103+0000 7ffa9b7fe640 1 --2- 172.21.15.84:0/1571865856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9c0b03c0 0x7ffa9c06ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.103+0000 7ffa9b7fe640 1 -- 172.21.15.84:0/1571865856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa84003970 msgr2=0x7ffa9c154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:07.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.103+0000 7ffa9b7fe640 1 --2- 172.21.15.84:0/1571865856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa84003970 0x7ffa9c154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.103+0000 7ffa9b7fe640 1 -- 172.21.15.84:0/1571865856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c0b1420 con 0x7ffa9c0b0840 2023-10-16T11:24:07.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.103+0000 7ffa9b7fe640 1 --2- 172.21.15.84:0/1571865856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa9c0b0840 0x7ffa9c06ef70 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0d0a80 tx=0x7ffa9400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:07.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.104+0000 7ffa83fff640 1 -- 172.21.15.84:0/1571865856 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa94016420 con 0x7ffa9c0b0840 2023-10-16T11:24:07.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.104+0000 7ffa83fff640 1 -- 172.21.15.84:0/1571865856 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa940165a0 con 0x7ffa9c0b0840 2023-10-16T11:24:07.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.104+0000 7ffaa19c7640 1 -- 172.21.15.84:0/1571865856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9c0b1700 con 0x7ffa9c0b0840 2023-10-16T11:24:07.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.104+0000 7ffa83fff640 1 -- 172.21.15.84:0/1571865856 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa94016950 con 0x7ffa9c0b0840 2023-10-16T11:24:07.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.104+0000 7ffaa19c7640 1 -- 172.21.15.84:0/1571865856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9c155460 con 0x7ffa9c0b0840 2023-10-16T11:24:07.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.105+0000 7ffaa19c7640 1 -- 172.21.15.84:0/1571865856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa9c0b1c20 con 0x7ffa9c0b0840 2023-10-16T11:24:07.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.107+0000 7ffa83fff640 1 -- 172.21.15.84:0/1571865856 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 128) v1 ==== 89445+0+0 (secure 0 0 0) 0x7ffa94016ab0 con 0x7ffa9c0b0840 2023-10-16T11:24:07.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.108+0000 7ffa83fff640 1 --2- 172.21.15.84:0/1571865856 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7ffa8806b0b0 0x7ffa8806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.108+0000 7ffa83fff640 1 -- 172.21.15.84:0/1571865856 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6139+0+0 (secure 0 0 0) 0x7ffa9408aac0 con 0x7ffa9c0b0840 2023-10-16T11:24:07.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.108+0000 7ffa9affd640 1 -- 172.21.15.84:0/1571865856 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7ffa8806b0b0 msgr2=0x7ffa8806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/855077515 2023-10-16T11:24:07.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.108+0000 7ffa9affd640 1 --2- 172.21.15.84:0/1571865856 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7ffa8806b0b0 0x7ffa8806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:24:07.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.113+0000 7ffa83fff640 1 -- 172.21.15.84:0/1571865856 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffa9c0b1c20 con 0x7ffa9c0b0840 2023-10-16T11:24:07.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.309+0000 7ffa9affd640 1 -- 172.21.15.84:0/1571865856 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7ffa8806b0b0 msgr2=0x7ffa8806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/855077515 2023-10-16T11:24:07.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.309+0000 7ffa9affd640 1 --2- 172.21.15.84:0/1571865856 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7ffa8806b0b0 0x7ffa8806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:24:07.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.341+0000 7ffaa19c7640 1 -- 172.21.15.84:0/1571865856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa9c0b1c20 con 0x7ffa9c0b0840 2023-10-16T11:24:07.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.341+0000 7ffa83fff640 1 -- 172.21.15.84:0/1571865856 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4929 (secure 0 0 0) 0x7ffa9c0b1c20 con 0x7ffa9c0b0840 2023-10-16T11:24:07.343 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:07.344 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6575,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/855929374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":855929374},{"type":"v1","addr":"172.21.15.157:6834","nonce":855929374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6578,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2581460972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2581460972},{"type":"v1","addr":"172.21.15.157:6835","nonce":2581460972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":10806,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1731880648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1731880648},{"type":"v1","addr":"172.21.15.84:6836","nonce":1731880648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:22:42.578443+0000","modified":"2023-10-16T11:22:45.635565+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10809":{"gid":10809,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2679249765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2679249765},{"type":"v1","addr":"172.21.15.84:6837","nonce":2679249765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-16T11:24:07.344 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 105 2023-10-16T11:24:07.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.344+0000 7ffaa19c7640 1 -- 172.21.15.84:0/1571865856 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7ffa8806b0b0 msgr2=0x7ffa8806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:24:07.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.344+0000 7ffaa19c7640 1 --2- 172.21.15.84:0/1571865856 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7ffa8806b0b0 0x7ffa8806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.344+0000 7ffaa19c7640 1 -- 172.21.15.84:0/1571865856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa9c0b0840 msgr2=0x7ffa9c06ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:07.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.344+0000 7ffaa19c7640 1 --2- 172.21.15.84:0/1571865856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa9c0b0840 0x7ffa9c06ef70 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0d0a80 tx=0x7ffa9400f520 comp rx=0 tx=0).stop 2023-10-16T11:24:07.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.344+0000 7ffaa19c7640 1 -- 172.21.15.84:0/1571865856 shutdown_connections 2023-10-16T11:24:07.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.344+0000 7ffaa19c7640 1 --2- 172.21.15.84:0/1571865856 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7ffa8806b0b0 0x7ffa8806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.344+0000 7ffaa19c7640 1 --2- 172.21.15.84:0/1571865856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa9c0b0840 0x7ffa9c06ef70 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.344+0000 7ffaa19c7640 1 --2- 172.21.15.84:0/1571865856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa9c0b03c0 0x7ffa9c06ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.344+0000 7ffaa19c7640 1 --2- 172.21.15.84:0/1571865856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa84003970 0x7ffa9c154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.344+0000 7ffaa19c7640 1 -- 172.21.15.84:0/1571865856 >> 172.21.15.84:0/1571865856 conn(0x7ffa9c0d0ad0 msgr2=0x7ffa9c15b480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:07.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.344+0000 7ffaa19c7640 1 -- 172.21.15.84:0/1571865856 shutdown_connections 2023-10-16T11:24:07.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.344+0000 7ffaa19c7640 1 -- 172.21.15.84:0/1571865856 wait complete. 2023-10-16T11:24:07.363 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:24:07.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.546+0000 7f10735fc640 1 Processor -- start 2023-10-16T11:24:07.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.546+0000 7f10735fc640 1 -- start start 2023-10-16T11:24:07.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.547+0000 7f10735fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f106c15aaa0 0x7f106c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.547+0000 7f10735fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106c0ce390 0x7f106c070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.547+0000 7f10735fc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106c15ed60 con 0x7f106c15ae80 2023-10-16T11:24:07.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.547+0000 7f10735fc640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106c06b810 con 0x7f106c15aaa0 2023-10-16T11:24:07.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.547+0000 7f10735fc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c0707e0 con 0x7f106c0ce390 2023-10-16T11:24:07.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.547+0000 7f1071398640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f106c15aaa0 0x7f106c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:07.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.547+0000 7f1070b97640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f106c15ae80 0x7f106c0cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48592/0 (socket says 172.21.15.84:48592) 2023-10-16T11:24:07.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.547+0000 7f1070b97640 1 -- 172.21.15.84:0/1842045839 learned_addr learned my addr 172.21.15.84:0/1842045839 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:07.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.548+0000 7f1071b99640 1 --2- 172.21.15.84:0/1842045839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106c0ce390 0x7f106c070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:07.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.548+0000 7f1071398640 1 -- 172.21.15.84:0/1842045839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106c0ce390 msgr2=0x7f106c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:07.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.548+0000 7f1071398640 1 --2- 172.21.15.84:0/1842045839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106c0ce390 0x7f106c070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.548+0000 7f1071398640 1 -- 172.21.15.84:0/1842045839 >> v1:172.21.15.84:6789/0 conn(0x7f106c15ae80 legacy=0x7f106c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:07.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.548+0000 7f1071398640 1 -- 172.21.15.84:0/1842045839 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c15ef90 con 0x7f106c15aaa0 2023-10-16T11:24:07.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.549+0000 7f1071398640 1 --2- 172.21.15.84:0/1842045839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f106c15aaa0 0x7f106c15e820 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f10680097a0 tx=0x7f106802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabad49134cb055a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:07.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.549+0000 7f1063fff640 1 -- 172.21.15.84:0/1842045839 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106802d040 con 0x7f106c15aaa0 2023-10-16T11:24:07.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.549+0000 7f1063fff640 1 -- 172.21.15.84:0/1842045839 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1068031810 con 0x7f106c15aaa0 2023-10-16T11:24:07.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.549+0000 7f1063fff640 1 -- 172.21.15.84:0/1842045839 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068031ba0 con 0x7f106c15aaa0 2023-10-16T11:24:07.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.550+0000 7f10735fc640 1 -- 172.21.15.84:0/1842045839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f106c15aaa0 msgr2=0x7f106c15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:07.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.550+0000 7f10735fc640 1 --2- 172.21.15.84:0/1842045839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f106c15aaa0 0x7f106c15e820 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f10680097a0 tx=0x7f106802b8b0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.550+0000 7f10735fc640 1 -- 172.21.15.84:0/1842045839 shutdown_connections 2023-10-16T11:24:07.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.550+0000 7f10735fc640 1 --2- 172.21.15.84:0/1842045839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106c0ce390 0x7f106c070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.550+0000 7f10735fc640 1 --2- 172.21.15.84:0/1842045839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f106c15aaa0 0x7f106c15e820 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.550+0000 7f10735fc640 1 -- 172.21.15.84:0/1842045839 >> 172.21.15.84:0/1842045839 conn(0x7f106c0cb410 msgr2=0x7f106c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:07.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.550+0000 7f10735fc640 1 -- 172.21.15.84:0/1842045839 shutdown_connections 2023-10-16T11:24:07.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.550+0000 7f10735fc640 1 -- 172.21.15.84:0/1842045839 wait complete. 2023-10-16T11:24:07.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.552+0000 7f10735fc640 1 Processor -- start 2023-10-16T11:24:07.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.552+0000 7f10735fc640 1 -- start start 2023-10-16T11:24:07.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.553+0000 7f10735fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f106c0ce390 0x7f106c15d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.553+0000 7f10735fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106c15aaa0 0x7f106c1f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.553+0000 7f10735fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106c15ae80 0x7f106c0c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.553+0000 7f10735fc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c1f6dc0 con 0x7f106c0ce390 2023-10-16T11:24:07.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.553+0000 7f10735fc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c15bd20 con 0x7f106c15aaa0 2023-10-16T11:24:07.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.553+0000 7f10735fc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c15c000 con 0x7f106c15ae80 2023-10-16T11:24:07.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.553+0000 7f1071398640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f106c0ce390 0x7f106c15d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:07.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.553+0000 7f1071398640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f106c0ce390 0x7f106c15d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44618/0 (socket says 172.21.15.84:44618) 2023-10-16T11:24:07.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.553+0000 7f1071398640 1 -- 172.21.15.84:0/2452390614 learned_addr learned my addr 172.21.15.84:0/2452390614 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:07.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.554+0000 7f1070b97640 1 --2- 172.21.15.84:0/2452390614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106c15aaa0 0x7f106c1f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:07.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.554+0000 7f1071b99640 1 --2- 172.21.15.84:0/2452390614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106c15ae80 0x7f106c0c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:07.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.554+0000 7f1071398640 1 -- 172.21.15.84:0/2452390614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106c15ae80 msgr2=0x7f106c0c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:07.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.554+0000 7f1071398640 1 --2- 172.21.15.84:0/2452390614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106c15ae80 0x7f106c0c8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.554+0000 7f1071398640 1 -- 172.21.15.84:0/2452390614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106c15aaa0 msgr2=0x7f106c1f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:07.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.554+0000 7f1071398640 1 --2- 172.21.15.84:0/2452390614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106c15aaa0 0x7f106c1f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.554+0000 7f1071398640 1 -- 172.21.15.84:0/2452390614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1068009400 con 0x7f106c0ce390 2023-10-16T11:24:07.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.554+0000 7f1070b97640 1 --2- 172.21.15.84:0/2452390614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106c15aaa0 0x7f106c1f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:07.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.554+0000 7f1071b99640 1 --2- 172.21.15.84:0/2452390614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106c15ae80 0x7f106c0c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:07.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.554+0000 7f1071398640 1 --2- 172.21.15.84:0/2452390614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f106c0ce390 0x7f106c15d930 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f10680098c0 tx=0x7f10680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:07.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.554+0000 7f1061ffb640 1 -- 172.21.15.84:0/2452390614 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106803ad60 con 0x7f106c0ce390 2023-10-16T11:24:07.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.554+0000 7f1061ffb640 1 -- 172.21.15.84:0/2452390614 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1068044400 con 0x7f106c0ce390 2023-10-16T11:24:07.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.554+0000 7f1061ffb640 1 -- 172.21.15.84:0/2452390614 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10680447b0 con 0x7f106c0ce390 2023-10-16T11:24:07.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.555+0000 7f10735fc640 1 -- 172.21.15.84:0/2452390614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106c15c2e0 con 0x7f106c0ce390 2023-10-16T11:24:07.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.555+0000 7f10735fc640 1 -- 172.21.15.84:0/2452390614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106c155390 con 0x7f106c0ce390 2023-10-16T11:24:07.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.556+0000 7f10735fc640 1 -- 172.21.15.84:0/2452390614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106c155520 con 0x7f106c0ce390 2023-10-16T11:24:07.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.557+0000 7f1061ffb640 1 -- 172.21.15.84:0/2452390614 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 128) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f106804d440 con 0x7f106c0ce390 2023-10-16T11:24:07.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.558+0000 7f1061ffb640 1 --2- 172.21.15.84:0/2452390614 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f105006b0b0 0x7f105006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.558+0000 7f1061ffb640 1 -- 172.21.15.84:0/2452390614 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6139+0+0 (secure 0 0 0) 0x7f10680ac0d0 con 0x7f106c0ce390 2023-10-16T11:24:07.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.558+0000 7f1070b97640 1 -- 172.21.15.84:0/2452390614 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f105006b0b0 msgr2=0x7f105006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/855077515 2023-10-16T11:24:07.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.558+0000 7f1070b97640 1 --2- 172.21.15.84:0/2452390614 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f105006b0b0 0x7f105006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:24:07.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.562+0000 7f1061ffb640 1 -- 172.21.15.84:0/2452390614 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1068076b60 con 0x7f106c0ce390 2023-10-16T11:24:07.574 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:07.572+0000 7f32ee044040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:24:07.699 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:07.697+0000 7fe738816040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:24:07.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.756+0000 7f10735fc640 1 -- 172.21.15.84:0/2452390614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f106c04e160 con 0x7f106c0ce390 2023-10-16T11:24:07.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.758+0000 7f1061ffb640 1 -- 172.21.15.84:0/2452390614 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v204) v1 ==== 74+0+17141 (secure 0 0 0) 0x7f106807a180 con 0x7f106c0ce390 2023-10-16T11:24:07.760 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:07.760 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":204,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:23:58.844672+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:22:40.385524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-16T11:22:41.592928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.18","mappings":[{"from":0,"to":2}]},{"pgid":"25.1f","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/3564300528":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/2349522146":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/2191803332":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/2199276620":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/4262224959":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/956990500":"2023-10-17T11:22:56.900947+0000","172.21.15.84:6833/1985463812":"2023-10-17T11:22:56.900947+0000","172.21.15.84:6832/1985463812":"2023-10-17T11:22:56.900947+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:24:07.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.759+0000 7f1070b97640 1 -- 172.21.15.84:0/2452390614 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f105006b0b0 msgr2=0x7f105006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/855077515 2023-10-16T11:24:07.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.759+0000 7f1070b97640 1 --2- 172.21.15.84:0/2452390614 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f105006b0b0 0x7f105006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:24:07.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.760+0000 7f10735fc640 1 -- 172.21.15.84:0/2452390614 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f105006b0b0 msgr2=0x7f105006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:24:07.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.760+0000 7f10735fc640 1 --2- 172.21.15.84:0/2452390614 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f105006b0b0 0x7f105006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.760+0000 7f10735fc640 1 -- 172.21.15.84:0/2452390614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f106c0ce390 msgr2=0x7f106c15d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:07.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.760+0000 7f10735fc640 1 --2- 172.21.15.84:0/2452390614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f106c0ce390 0x7f106c15d930 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f10680098c0 tx=0x7f10680046c0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.761+0000 7f10735fc640 1 -- 172.21.15.84:0/2452390614 shutdown_connections 2023-10-16T11:24:07.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.761+0000 7f10735fc640 1 --2- 172.21.15.84:0/2452390614 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7f105006b0b0 0x7f105006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.761+0000 7f10735fc640 1 --2- 172.21.15.84:0/2452390614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f106c15ae80 0x7f106c0c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.761+0000 7f10735fc640 1 --2- 172.21.15.84:0/2452390614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106c15aaa0 0x7f106c1f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.761+0000 7f10735fc640 1 --2- 172.21.15.84:0/2452390614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f106c0ce390 0x7f106c15d930 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.761+0000 7f10735fc640 1 -- 172.21.15.84:0/2452390614 >> 172.21.15.84:0/2452390614 conn(0x7f106c0cb410 msgr2=0x7f106c159380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:07.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.761+0000 7f10735fc640 1 -- 172.21.15.84:0/2452390614 shutdown_connections 2023-10-16T11:24:07.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.761+0000 7f10735fc640 1 -- 172.21.15.84:0/2452390614 wait complete. 2023-10-16T11:24:07.780 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:22:40.385524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '200', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:22:41.592928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '201', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:24:07.781 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:24:07.868 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:07.866+0000 7f32ee044040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:24:07.927 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:07.925+0000 7f32ee044040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:24:07.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.981+0000 7fc26759e640 1 Processor -- start 2023-10-16T11:24:07.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.981+0000 7fc26759e640 1 -- start start 2023-10-16T11:24:07.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.982+0000 7fc26759e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc2680b1ca0 0x7fc2680d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.982+0000 7fc26759e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc2680b2080 0x7fc2680ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.982+0000 7fc26759e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2680d0970 con 0x7fc2680ce960 2023-10-16T11:24:07.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.982+0000 7fc26759e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2680d0ea0 con 0x7fc2680b1ca0 2023-10-16T11:24:07.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.982+0000 7fc26759e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2680d0bc0 con 0x7fc2680b2080 2023-10-16T11:24:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.982+0000 7fc26659c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc2680b1ca0 0x7fc2680d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.982+0000 7fc26659c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc2680b1ca0 0x7fc2680d36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44620/0 (socket says 172.21.15.84:44620) 2023-10-16T11:24:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.983+0000 7fc26659c640 1 -- 172.21.15.84:0/2784448650 learned_addr learned my addr 172.21.15.84:0/2784448650 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.982+0000 7fc266d9d640 1 --1- 172.21.15.84:0/2784448650 >> v1:172.21.15.84:6789/0 conn(0x7fc2680ce960 0x7fc2680b24c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48604/0 (socket says 172.21.15.84:48604) 2023-10-16T11:24:07.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.983+0000 7fc25ffff640 1 --2- 172.21.15.84:0/2784448650 >> v2:172.21.15.157:3300/0 conn(0x7fc2680b2080 0x7fc2680ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.983+0000 7fc26659c640 1 -- 172.21.15.84:0/2784448650 >> v2:172.21.15.157:3300/0 conn(0x7fc2680b2080 msgr2=0x7fc2680ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.983+0000 7fc26659c640 1 --2- 172.21.15.84:0/2784448650 >> v2:172.21.15.157:3300/0 conn(0x7fc2680b2080 0x7fc2680ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.983+0000 7fc26659c640 1 -- 172.21.15.84:0/2784448650 >> v1:172.21.15.84:6789/0 conn(0x7fc2680ce960 legacy=0x7fc2680b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:07.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.983+0000 7fc26659c640 1 -- 172.21.15.84:0/2784448650 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2680cd940 con 0x7fc2680b1ca0 2023-10-16T11:24:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.984+0000 7fc26659c640 1 --2- 172.21.15.84:0/2784448650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2680b1ca0 0x7fc2680d36b0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7fc2680671e0 tx=0x7fc25802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea6a8ce8490d50f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.984+0000 7fc265d9b640 1 -- 172.21.15.84:0/2784448650 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25802f850 con 0x7fc2680b1ca0 2023-10-16T11:24:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.984+0000 7fc265d9b640 1 -- 172.21.15.84:0/2784448650 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25802f9b0 con 0x7fc2680b1ca0 2023-10-16T11:24:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.984+0000 7fc265d9b640 1 -- 172.21.15.84:0/2784448650 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25802fd40 con 0x7fc2680b1ca0 2023-10-16T11:24:07.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.985+0000 7fc26759e640 1 -- 172.21.15.84:0/2784448650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2680b1ca0 msgr2=0x7fc2680d36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:07.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.985+0000 7fc26759e640 1 --2- 172.21.15.84:0/2784448650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2680b1ca0 0x7fc2680d36b0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7fc2680671e0 tx=0x7fc25802ba30 comp rx=0 tx=0).stop 2023-10-16T11:24:07.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.985+0000 7fc26759e640 1 -- 172.21.15.84:0/2784448650 shutdown_connections 2023-10-16T11:24:07.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.985+0000 7fc26759e640 1 --2- 172.21.15.84:0/2784448650 >> v2:172.21.15.157:3300/0 conn(0x7fc2680b2080 0x7fc2680ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.985+0000 7fc26759e640 1 --2- 172.21.15.84:0/2784448650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2680b1ca0 0x7fc2680d36b0 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.985+0000 7fc26759e640 1 -- 172.21.15.84:0/2784448650 >> 172.21.15.84:0/2784448650 conn(0x7fc2680d2b90 msgr2=0x7fc2680d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:07.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.985+0000 7fc26759e640 1 -- 172.21.15.84:0/2784448650 shutdown_connections 2023-10-16T11:24:07.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.985+0000 7fc26759e640 1 -- 172.21.15.84:0/2784448650 wait complete. 2023-10-16T11:24:07.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.987+0000 7fc26759e640 1 Processor -- start 2023-10-16T11:24:07.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.987+0000 7fc26759e640 1 -- start start 2023-10-16T11:24:07.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.988+0000 7fc26759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2680b1ca0 0x7fc268155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.988+0000 7fc26759e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2680b2080 0x7fc2680cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.988+0000 7fc26759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2680ce960 0x7fc2681f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.988+0000 7fc26759e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2680cb0d0 con 0x7fc2680b1ca0 2023-10-16T11:24:07.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.988+0000 7fc26759e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2680cb560 con 0x7fc2680ce960 2023-10-16T11:24:07.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.988+0000 7fc26759e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2680ccd50 con 0x7fc2680b2080 2023-10-16T11:24:07.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.988+0000 7fc26659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2680b1ca0 0x7fc268155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:07.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.988+0000 7fc26659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2680b1ca0 0x7fc268155000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44622/0 (socket says 172.21.15.84:44622) 2023-10-16T11:24:07.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.988+0000 7fc26659c640 1 -- 172.21.15.84:0/111581846 learned_addr learned my addr 172.21.15.84:0/111581846 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:07.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.988+0000 7fc26659c640 1 -- 172.21.15.84:0/111581846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2680b2080 msgr2=0x7fc2680cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:07.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.988+0000 7fc26659c640 1 --2- 172.21.15.84:0/111581846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2680b2080 0x7fc2680cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.988+0000 7fc26659c640 1 -- 172.21.15.84:0/111581846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2680ce960 msgr2=0x7fc2681f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:07.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.988+0000 7fc26659c640 1 --2- 172.21.15.84:0/111581846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2680ce960 0x7fc2681f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:07.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.988+0000 7fc26659c640 1 -- 172.21.15.84:0/111581846 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2580094d0 con 0x7fc2680b1ca0 2023-10-16T11:24:07.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.989+0000 7fc26659c640 1 --2- 172.21.15.84:0/111581846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2680b1ca0 0x7fc268155000 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fc25802ba70 tx=0x7fc258004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:07.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.989+0000 7fc25effd640 1 -- 172.21.15.84:0/111581846 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258037d10 con 0x7fc2680b1ca0 2023-10-16T11:24:07.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.989+0000 7fc25effd640 1 -- 172.21.15.84:0/111581846 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc258037e90 con 0x7fc2680b1ca0 2023-10-16T11:24:07.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.989+0000 7fc25effd640 1 -- 172.21.15.84:0/111581846 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258041640 con 0x7fc2680b1ca0 2023-10-16T11:24:07.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.989+0000 7fc26759e640 1 -- 172.21.15.84:0/111581846 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2680ccab0 con 0x7fc2680b1ca0 2023-10-16T11:24:07.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.989+0000 7fc26759e640 1 -- 172.21.15.84:0/111581846 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2680ca7a0 con 0x7fc2680b1ca0 2023-10-16T11:24:07.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.990+0000 7fc26759e640 1 -- 172.21.15.84:0/111581846 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc26804e9b0 con 0x7fc2680b1ca0 2023-10-16T11:24:07.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.991+0000 7fc25effd640 1 -- 172.21.15.84:0/111581846 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 128) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fc2580417a0 con 0x7fc2680b1ca0 2023-10-16T11:24:07.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.992+0000 7fc25effd640 1 --2- 172.21.15.84:0/111581846 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fc24806b0b0 0x7fc24806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:07.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.992+0000 7fc25ffff640 1 -- 172.21.15.84:0/111581846 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fc24806b0b0 msgr2=0x7fc24806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/855077515 2023-10-16T11:24:07.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.992+0000 7fc25effd640 1 -- 172.21.15.84:0/111581846 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6195+0+0 (secure 0 0 0) 0x7fc25803c070 con 0x7fc2680b1ca0 2023-10-16T11:24:07.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:07.992+0000 7fc25ffff640 1 --2- 172.21.15.84:0/111581846 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fc24806b0b0 0x7fc24806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:24:08.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.000+0000 7fc25effd640 1 -- 172.21.15.84:0/111581846 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc258077500 con 0x7fc2680b1ca0 2023-10-16T11:24:08.034 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:08.032+0000 7fe738816040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:24:08.049 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:08.048+0000 7f32ee044040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:24:08.093 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:08.091+0000 7fe738816040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:24:08.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.193+0000 7fc25ffff640 1 -- 172.21.15.84:0/111581846 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fc24806b0b0 msgr2=0x7fc24806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/855077515 2023-10-16T11:24:08.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.193+0000 7fc25ffff640 1 --2- 172.21.15.84:0/111581846 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fc24806b0b0 0x7fc24806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:24:08.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.199+0000 7fc26759e640 1 -- 172.21.15.84:0/111581846 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc268073a40 con 0x7fc2680b1ca0 2023-10-16T11:24:08.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.200+0000 7fc25effd640 1 -- 172.21.15.84:0/111581846 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v205) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc25807ab20 con 0x7fc2680b1ca0 2023-10-16T11:24:08.202 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:24:08.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.202+0000 7fc26759e640 1 -- 172.21.15.84:0/111581846 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fc24806b0b0 msgr2=0x7fc24806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:24:08.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.202+0000 7fc26759e640 1 --2- 172.21.15.84:0/111581846 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fc24806b0b0 0x7fc24806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.202+0000 7fc26759e640 1 -- 172.21.15.84:0/111581846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2680b1ca0 msgr2=0x7fc268155000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:08.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.202+0000 7fc26759e640 1 --2- 172.21.15.84:0/111581846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2680b1ca0 0x7fc268155000 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fc25802ba70 tx=0x7fc258004730 comp rx=0 tx=0).stop 2023-10-16T11:24:08.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.202+0000 7fc26759e640 1 -- 172.21.15.84:0/111581846 shutdown_connections 2023-10-16T11:24:08.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.202+0000 7fc26759e640 1 --2- 172.21.15.84:0/111581846 >> [v2:172.21.15.84:6832/855077515,v1:172.21.15.84:6833/855077515] conn(0x7fc24806b0b0 0x7fc24806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.202+0000 7fc26759e640 1 --2- 172.21.15.84:0/111581846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2680ce960 0x7fc2681f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.202+0000 7fc26759e640 1 --2- 172.21.15.84:0/111581846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2680b2080 0x7fc2680cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.202+0000 7fc26759e640 1 --2- 172.21.15.84:0/111581846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2680b1ca0 0x7fc268155000 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.202+0000 7fc26759e640 1 -- 172.21.15.84:0/111581846 >> 172.21.15.84:0/111581846 conn(0x7fc2680d2b90 msgr2=0x7fc268160090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:08.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.202+0000 7fc26759e640 1 -- 172.21.15.84:0/111581846 shutdown_connections 2023-10-16T11:24:08.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.203+0000 7fc26759e640 1 -- 172.21.15.84:0/111581846 wait complete. 2023-10-16T11:24:08.216 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:08.214+0000 7fe738816040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:24:08.221 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:24:08.300 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:08.298+0000 7fe6bceff640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:08.301 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:08.298+0000 7fe6bceff640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:08.301 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:08.298+0000 7fe6bceff640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:08.301 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:08.298+0000 7fe6bceff640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:08.301 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:08.298+0000 7fe6bceff640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:08.302 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:08.299+0000 7fe6b9ef9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:08.302 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:08.299+0000 7fe6b9ef9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:08.302 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:08.299+0000 7fe6b9ef9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:08.303 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:08.299+0000 7fe6b9ef9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:08.303 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:08.299+0000 7fe6b9ef9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:08.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.408+0000 7f7898fa9640 1 Processor -- start 2023-10-16T11:24:08.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.408+0000 7f7898fa9640 1 -- start start 2023-10-16T11:24:08.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.409+0000 7f7898fa9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f789415ad70 0x7f7894069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:08.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.409+0000 7f7898fa9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78940c9ca0 0x7f789406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:08.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.410+0000 7f7898fa9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78940cb030 con 0x7f78940ca3b0 2023-10-16T11:24:08.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.410+0000 7f7898fa9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78940c91c0 con 0x7f789415ad70 2023-10-16T11:24:08.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.410+0000 7f7898fa9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78940c94a0 con 0x7f78940c9ca0 2023-10-16T11:24:08.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.410+0000 7f7892d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f78940ca3b0 0x7f78940ca840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48616/0 (socket says 172.21.15.84:48616) 2023-10-16T11:24:08.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.410+0000 7f7892d9d640 1 -- 172.21.15.84:0/4074105965 learned_addr learned my addr 172.21.15.84:0/4074105965 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:08.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.410+0000 7f789259c640 1 --2- 172.21.15.84:0/4074105965 >> v2:172.21.15.157:3300/0 conn(0x7f789415ad70 0x7f7894069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:08.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.410+0000 7f7891d9b640 1 -- 172.21.15.84:0/4074105965 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256307590 0 0) 0x7f78940cb030 con 0x7f78940ca3b0 2023-10-16T11:24:08.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.410+0000 7f7891d9b640 1 -- 172.21.15.84:0/4074105965 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7880003520 con 0x7f78940ca3b0 2023-10-16T11:24:08.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.410+0000 7f7889d9b640 1 --2- 172.21.15.84:0/4074105965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78940c9ca0 0x7f789406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:08.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7891d9b640 1 -- 172.21.15.84:0/4074105965 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2500076914 0 0) 0x7f7880003520 con 0x7f78940ca3b0 2023-10-16T11:24:08.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7891d9b640 1 -- 172.21.15.84:0/4074105965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78940c9ca0 msgr2=0x7f789406a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:08.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7891d9b640 1 --2- 172.21.15.84:0/4074105965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78940c9ca0 0x7f789406a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7891d9b640 1 -- 172.21.15.84:0/4074105965 >> v2:172.21.15.157:3300/0 conn(0x7f789415ad70 msgr2=0x7f7894069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:08.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7891d9b640 1 --2- 172.21.15.84:0/4074105965 >> v2:172.21.15.157:3300/0 conn(0x7f789415ad70 0x7f7894069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7891d9b640 1 -- 172.21.15.84:0/4074105965 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78940c9780 con 0x7f78940ca3b0 2023-10-16T11:24:08.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7891d9b640 1 -- 172.21.15.84:0/4074105965 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f788c0019f0 con 0x7f78940ca3b0 2023-10-16T11:24:08.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f789259c640 1 --2- 172.21.15.84:0/4074105965 >> v2:172.21.15.157:3300/0 conn(0x7f789415ad70 0x7f7894069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:24:08.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7891d9b640 1 -- 172.21.15.84:0/4074105965 >> v1:172.21.15.84:6789/0 conn(0x7f78940ca3b0 legacy=0x7f78940ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:08.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7891d9b640 1 --2- 172.21.15.84:0/4074105965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7880003970 0x7f7880003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:08.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7891d9b640 1 --2- 172.21.15.84:0/4074105965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78800047c0 0x7f7880004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:08.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7891d9b640 1 --2- 172.21.15.84:0/4074105965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78940c9ca0 0x7f7880005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:08.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7891d9b640 1 -- 172.21.15.84:0/4074105965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78940c91c0 con 0x7f78800047c0 2023-10-16T11:24:08.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7891d9b640 1 -- 172.21.15.84:0/4074105965 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78940c94a0 con 0x7f7880003970 2023-10-16T11:24:08.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7891d9b640 1 -- 172.21.15.84:0/4074105965 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788000ba80 con 0x7f78940c9ca0 2023-10-16T11:24:08.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7889d9b640 1 --2- 172.21.15.84:0/4074105965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78800047c0 0x7f7880004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:08.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f789259c640 1 --2- 172.21.15.84:0/4074105965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7880003970 0x7f7880003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:08.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7889d9b640 1 -- 172.21.15.84:0/4074105965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78940c9ca0 msgr2=0x7f7880005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:08.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7889d9b640 1 --2- 172.21.15.84:0/4074105965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78940c9ca0 0x7f7880005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7889d9b640 1 -- 172.21.15.84:0/4074105965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7880003970 msgr2=0x7f7880003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:08.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7889d9b640 1 --2- 172.21.15.84:0/4074105965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7880003970 0x7f7880003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.411+0000 7f7889d9b640 1 -- 172.21.15.84:0/4074105965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7880003520 con 0x7f78800047c0 2023-10-16T11:24:08.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.412+0000 7f7889d9b640 1 --2- 172.21.15.84:0/4074105965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78800047c0 0x7f7880004c40 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f787c00ba00 tx=0x7f78800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bba2f85ca23acd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:08.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.413+0000 7f7891d9b640 1 -- 172.21.15.84:0/4074105965 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c032620 con 0x7f78800047c0 2023-10-16T11:24:08.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.413+0000 7f7891d9b640 1 -- 172.21.15.84:0/4074105965 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787c032780 con 0x7f78800047c0 2023-10-16T11:24:08.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.413+0000 7f7891d9b640 1 -- 172.21.15.84:0/4074105965 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c032b10 con 0x7f78800047c0 2023-10-16T11:24:08.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.413+0000 7f7898fa9640 1 -- 172.21.15.84:0/4074105965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78800047c0 msgr2=0x7f7880004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:08.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.413+0000 7f7898fa9640 1 --2- 172.21.15.84:0/4074105965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78800047c0 0x7f7880004c40 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f787c00ba00 tx=0x7f78800057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.413+0000 7f7892d9d640 1 -- 172.21.15.84:0/4074105965 reap_dead start 2023-10-16T11:24:08.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.413+0000 7f7898fa9640 1 -- 172.21.15.84:0/4074105965 shutdown_connections 2023-10-16T11:24:08.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.413+0000 7f7898fa9640 1 -- 172.21.15.84:0/4074105965 >> 172.21.15.84:0/4074105965 conn(0x7f78940ce9b0 msgr2=0x7f78940cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:08.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.413+0000 7f7898fa9640 1 -- 172.21.15.84:0/4074105965 shutdown_connections 2023-10-16T11:24:08.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.414+0000 7f7898fa9640 1 -- 172.21.15.84:0/4074105965 wait complete. 2023-10-16T11:24:08.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.415+0000 7f7898fa9640 1 Processor -- start 2023-10-16T11:24:08.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.415+0000 7f7898fa9640 1 -- start start 2023-10-16T11:24:08.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.416+0000 7f7898fa9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78800047c0 0x7f78940cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:08.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.416+0000 7f7898fa9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78940b1140 0x7f789414eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:08.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.416+0000 7f7898fa9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78940aea60 0x7f78940aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:08.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.416+0000 7f7898fa9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78940b24a0 con 0x7f78940b1140 2023-10-16T11:24:08.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.416+0000 7f7898fa9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789414f420 con 0x7f78800047c0 2023-10-16T11:24:08.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.416+0000 7f7898fa9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7894152610 con 0x7f78940aea60 2023-10-16T11:24:08.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.416+0000 7f7889d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78940b1140 0x7f789414eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:08.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.416+0000 7f7889d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78940b1140 0x7f789414eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44722/0 (socket says 172.21.15.84:44722) 2023-10-16T11:24:08.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.416+0000 7f7889d9b640 1 -- 172.21.15.84:0/2734321993 learned_addr learned my addr 172.21.15.84:0/2734321993 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:08.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.417+0000 7f7889d9b640 1 -- 172.21.15.84:0/2734321993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78940aea60 msgr2=0x7f78940aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:08.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.417+0000 7f7889d9b640 1 --2- 172.21.15.84:0/2734321993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78940aea60 0x7f78940aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.417+0000 7f7889d9b640 1 -- 172.21.15.84:0/2734321993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78800047c0 msgr2=0x7f78940cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:08.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.417+0000 7f7889d9b640 1 --2- 172.21.15.84:0/2734321993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78800047c0 0x7f78940cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.417+0000 7f7889d9b640 1 -- 172.21.15.84:0/2734321993 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78940af420 con 0x7f78940b1140 2023-10-16T11:24:08.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.417+0000 7f7889d9b640 1 --2- 172.21.15.84:0/2734321993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78940b1140 0x7f789414eee0 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f787c035040 tx=0x7f787c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:08.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.417+0000 7f788b7fe640 1 -- 172.21.15.84:0/2734321993 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c004020 con 0x7f78940b1140 2023-10-16T11:24:08.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.417+0000 7f788b7fe640 1 -- 172.21.15.84:0/2734321993 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787c004180 con 0x7f78940b1140 2023-10-16T11:24:08.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.417+0000 7f788b7fe640 1 -- 172.21.15.84:0/2734321993 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c003610 con 0x7f78940b1140 2023-10-16T11:24:08.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.417+0000 7f7898fa9640 1 -- 172.21.15.84:0/2734321993 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78940b1580 con 0x7f78940b1140 2023-10-16T11:24:08.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.417+0000 7f7898fa9640 1 -- 172.21.15.84:0/2734321993 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78940b1b00 con 0x7f78940b1140 2023-10-16T11:24:08.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.418+0000 7f7898fa9640 1 -- 172.21.15.84:0/2734321993 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789404e160 con 0x7f78940b1140 2023-10-16T11:24:08.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.419+0000 7f788b7fe640 1 -- 172.21.15.84:0/2734321993 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 129) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f787c003770 con 0x7f78940b1140 2023-10-16T11:24:08.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.420+0000 7f788b7fe640 1 -- 172.21.15.84:0/2734321993 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f787c0ab4c0 con 0x7f78940b1140 2023-10-16T11:24:08.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.424+0000 7f788b7fe640 1 -- 172.21.15.84:0/2734321993 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f787c076670 con 0x7f78940b1140 2023-10-16T11:24:08.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.605+0000 7f7898fa9640 1 -- 172.21.15.84:0/2734321993 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78940af420 con 0x7f78940b1140 2023-10-16T11:24:08.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.606+0000 7f788b7fe640 1 -- 172.21.15.84:0/2734321993 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v206) v1 ==== 104+0+11 (secure 0 0 0) 0x7f78940af420 con 0x7f78940b1140 2023-10-16T11:24:08.608 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:24:08.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.608+0000 7f7898fa9640 1 -- 172.21.15.84:0/2734321993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78940b1140 msgr2=0x7f789414eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:08.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.608+0000 7f7898fa9640 1 --2- 172.21.15.84:0/2734321993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78940b1140 0x7f789414eee0 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f787c035040 tx=0x7f787c004730 comp rx=0 tx=0).stop 2023-10-16T11:24:08.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.609+0000 7f7898fa9640 1 -- 172.21.15.84:0/2734321993 shutdown_connections 2023-10-16T11:24:08.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.609+0000 7f7898fa9640 1 --2- 172.21.15.84:0/2734321993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78940aea60 0x7f78940aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.609+0000 7f7898fa9640 1 --2- 172.21.15.84:0/2734321993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78940b1140 0x7f789414eee0 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.609+0000 7f7898fa9640 1 --2- 172.21.15.84:0/2734321993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78800047c0 0x7f78940cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.609+0000 7f7898fa9640 1 -- 172.21.15.84:0/2734321993 >> 172.21.15.84:0/2734321993 conn(0x7f78940ce9b0 msgr2=0x7f78940c5150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:08.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.609+0000 7f7898fa9640 1 -- 172.21.15.84:0/2734321993 shutdown_connections 2023-10-16T11:24:08.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.609+0000 7f7898fa9640 1 -- 172.21.15.84:0/2734321993 wait complete. 2023-10-16T11:24:08.627 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:24:08.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.832+0000 7f4257661640 1 Processor -- start 2023-10-16T11:24:08.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.832+0000 7f4257661640 1 -- start start 2023-10-16T11:24:08.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.833+0000 7f4257661640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f425015ad70 0x7f4250069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:08.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.833+0000 7f4257661640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f42500c9ca0 0x7f425006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:08.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.833+0000 7f4257661640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42500cb030 con 0x7f42500ca3b0 2023-10-16T11:24:08.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.833+0000 7f4257661640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42500c91c0 con 0x7f42500c9ca0 2023-10-16T11:24:08.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.833+0000 7f4257661640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42500c94a0 con 0x7f425015ad70 2023-10-16T11:24:08.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.834+0000 7f4254bfc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f42500c9ca0 0x7f425006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:08.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.834+0000 7f4254bfc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f42500c9ca0 0x7f425006a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44728/0 (socket says 172.21.15.84:44728) 2023-10-16T11:24:08.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.834+0000 7f4254bfc640 1 -- 172.21.15.84:0/1706754679 learned_addr learned my addr 172.21.15.84:0/1706754679 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:08.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.834+0000 7f42553fd640 1 --2- 172.21.15.84:0/1706754679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f425015ad70 0x7f4250069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:08.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.834+0000 7f4254bfc640 1 -- 172.21.15.84:0/1706754679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f425015ad70 msgr2=0x7f4250069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:08.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.834+0000 7f4254bfc640 1 --2- 172.21.15.84:0/1706754679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f425015ad70 0x7f4250069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.834+0000 7f4254bfc640 1 -- 172.21.15.84:0/1706754679 >> v1:172.21.15.157:6789/0 conn(0x7f42500ca3b0 legacy=0x7f42500ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:08.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.834+0000 7f4254bfc640 1 -- 172.21.15.84:0/1706754679 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42500c9780 con 0x7f42500c9ca0 2023-10-16T11:24:08.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.834+0000 7f42553fd640 1 --2- 172.21.15.84:0/1706754679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f425015ad70 0x7f4250069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:08.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.835+0000 7f4254bfc640 1 --2- 172.21.15.84:0/1706754679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42500c9ca0 0x7f425006a2e0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f4244009870 tx=0x7f424402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58de65b77e9268d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:08.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.836+0000 7f424ffff640 1 -- 172.21.15.84:0/1706754679 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4244031640 con 0x7f42500c9ca0 2023-10-16T11:24:08.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.836+0000 7f424ffff640 1 -- 172.21.15.84:0/1706754679 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42440317a0 con 0x7f42500c9ca0 2023-10-16T11:24:08.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.836+0000 7f424ffff640 1 -- 172.21.15.84:0/1706754679 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4244031b30 con 0x7f42500c9ca0 2023-10-16T11:24:08.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.836+0000 7f4257661640 1 -- 172.21.15.84:0/1706754679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42500c9ca0 msgr2=0x7f425006a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:08.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.836+0000 7f4257661640 1 --2- 172.21.15.84:0/1706754679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42500c9ca0 0x7f425006a2e0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f4244009870 tx=0x7f424402b8f0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.836+0000 7f4257661640 1 -- 172.21.15.84:0/1706754679 shutdown_connections 2023-10-16T11:24:08.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.836+0000 7f4257661640 1 --2- 172.21.15.84:0/1706754679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42500c9ca0 0x7f425006a2e0 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.836+0000 7f4257661640 1 --2- 172.21.15.84:0/1706754679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f425015ad70 0x7f4250069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.836+0000 7f4257661640 1 -- 172.21.15.84:0/1706754679 >> 172.21.15.84:0/1706754679 conn(0x7f42500ce9b0 msgr2=0x7f42500cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:08.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.836+0000 7f4257661640 1 -- 172.21.15.84:0/1706754679 shutdown_connections 2023-10-16T11:24:08.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.836+0000 7f4257661640 1 -- 172.21.15.84:0/1706754679 wait complete. 2023-10-16T11:24:08.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.838+0000 7f4257661640 1 Processor -- start 2023-10-16T11:24:08.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.838+0000 7f4257661640 1 -- start start 2023-10-16T11:24:08.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.839+0000 7f4257661640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42500c9ca0 0x7f4250153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:08.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.839+0000 7f4257661640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42500ca3b0 0x7f42501586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:08.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.839+0000 7f4257661640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f425015ad70 0x7f4250158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:08.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.839+0000 7f4257661640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42500c6630 con 0x7f425015ad70 2023-10-16T11:24:08.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.839+0000 7f4257661640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42501f2210 con 0x7f42500ca3b0 2023-10-16T11:24:08.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.839+0000 7f4257661640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42500ce520 con 0x7f42500c9ca0 2023-10-16T11:24:08.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.839+0000 7f4255bfe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f425015ad70 0x7f4250158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:08.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.839+0000 7f4255bfe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f425015ad70 0x7f4250158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44740/0 (socket says 172.21.15.84:44740) 2023-10-16T11:24:08.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.839+0000 7f4255bfe640 1 -- 172.21.15.84:0/3960901003 learned_addr learned my addr 172.21.15.84:0/3960901003 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:08.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.840+0000 7f4254bfc640 1 --2- 172.21.15.84:0/3960901003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42500ca3b0 0x7f42501586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:08.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.840+0000 7f4255bfe640 1 -- 172.21.15.84:0/3960901003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42500c9ca0 msgr2=0x7f4250153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:08.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.840+0000 7f42553fd640 1 --2- 172.21.15.84:0/3960901003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42500c9ca0 0x7f4250153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:08.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.840+0000 7f4255bfe640 1 --2- 172.21.15.84:0/3960901003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42500c9ca0 0x7f4250153a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.840+0000 7f4255bfe640 1 -- 172.21.15.84:0/3960901003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42500ca3b0 msgr2=0x7f42501586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:08.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.840+0000 7f4255bfe640 1 --2- 172.21.15.84:0/3960901003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42500ca3b0 0x7f42501586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:08.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.840+0000 7f4255bfe640 1 -- 172.21.15.84:0/3960901003 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42440094d0 con 0x7f425015ad70 2023-10-16T11:24:08.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.840+0000 7f4254bfc640 1 --2- 172.21.15.84:0/3960901003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42500ca3b0 0x7f42501586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:08.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.840+0000 7f4255bfe640 1 --2- 172.21.15.84:0/3960901003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f425015ad70 0x7f4250158bf0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f424000c410 tx=0x7f424000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:08.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.840+0000 7f424dffb640 1 -- 172.21.15.84:0/3960901003 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240011790 con 0x7f425015ad70 2023-10-16T11:24:08.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.840+0000 7f424dffb640 1 -- 172.21.15.84:0/3960901003 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42400118f0 con 0x7f425015ad70 2023-10-16T11:24:08.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.840+0000 7f4257661640 1 -- 172.21.15.84:0/3960901003 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42500cdf60 con 0x7f425015ad70 2023-10-16T11:24:08.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.840+0000 7f424dffb640 1 -- 172.21.15.84:0/3960901003 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240011ce0 con 0x7f425015ad70 2023-10-16T11:24:08.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.841+0000 7f4257661640 1 -- 172.21.15.84:0/3960901003 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42500c5730 con 0x7f425015ad70 2023-10-16T11:24:08.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.842+0000 7f4257661640 1 -- 172.21.15.84:0/3960901003 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42500c6770 con 0x7f425015ad70 2023-10-16T11:24:08.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.843+0000 7f424dffb640 1 -- 172.21.15.84:0/3960901003 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 129) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f424001c460 con 0x7f425015ad70 2023-10-16T11:24:08.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.844+0000 7f424dffb640 1 -- 172.21.15.84:0/3960901003 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f4240089190 con 0x7f425015ad70 2023-10-16T11:24:08.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:08.848+0000 7f424dffb640 1 -- 172.21.15.84:0/3960901003 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f42500c6770 con 0x7f425015ad70 2023-10-16T11:24:09.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.038+0000 7f4257661640 1 -- 172.21.15.84:0/3960901003 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f42500c6770 con 0x7f425015ad70 2023-10-16T11:24:09.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.039+0000 7f424dffb640 1 -- 172.21.15.84:0/3960901003 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v206) v1 ==== 100+0+11 (secure 0 0 0) 0x7f42500c6770 con 0x7f425015ad70 2023-10-16T11:24:09.041 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:24:09.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.042+0000 7f4257661640 1 -- 172.21.15.84:0/3960901003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f425015ad70 msgr2=0x7f4250158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:09.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.042+0000 7f4257661640 1 --2- 172.21.15.84:0/3960901003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f425015ad70 0x7f4250158bf0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f424000c410 tx=0x7f424000c8d0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.042+0000 7f4257661640 1 -- 172.21.15.84:0/3960901003 shutdown_connections 2023-10-16T11:24:09.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.042+0000 7f4257661640 1 --2- 172.21.15.84:0/3960901003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f425015ad70 0x7f4250158bf0 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.042+0000 7f4257661640 1 --2- 172.21.15.84:0/3960901003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42500ca3b0 0x7f42501586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.042+0000 7f4257661640 1 --2- 172.21.15.84:0/3960901003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42500c9ca0 0x7f4250153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.042+0000 7f4257661640 1 -- 172.21.15.84:0/3960901003 >> 172.21.15.84:0/3960901003 conn(0x7f42500ce9b0 msgr2=0x7f42500ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:09.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.042+0000 7f4257661640 1 -- 172.21.15.84:0/3960901003 shutdown_connections 2023-10-16T11:24:09.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.042+0000 7f4257661640 1 -- 172.21.15.84:0/3960901003 wait complete. 2023-10-16T11:24:09.061 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:24:09.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.268+0000 7fc259415640 1 Processor -- start 2023-10-16T11:24:09.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.268+0000 7fc259415640 1 -- start start 2023-10-16T11:24:09.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.269+0000 7fc259415640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc2540cd0c0 0x7fc2541615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:09.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.269+0000 7fc259415640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc2540b0410 0x7fc2540b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:09.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.269+0000 7fc259415640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc25415d3f0 0x7fc25415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:09.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.269+0000 7fc259415640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2540b0de0 con 0x7fc2540b0410 2023-10-16T11:24:09.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.269+0000 7fc259415640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc25415e3e0 con 0x7fc2540cd0c0 2023-10-16T11:24:09.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.269+0000 7fc259415640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2540cd4a0 con 0x7fc25415d3f0 2023-10-16T11:24:09.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.270+0000 7fc2527fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc2540b0410 0x7fc2540b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:09.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.270+0000 7fc2527fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc2540b0410 0x7fc2540b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44754/0 (socket says 172.21.15.84:44754) 2023-10-16T11:24:09.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.270+0000 7fc2527fc640 1 -- 172.21.15.84:0/2963556536 learned_addr learned my addr 172.21.15.84:0/2963556536 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:09.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.270+0000 7fc2527fc640 1 -- 172.21.15.84:0/2963556536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc25415d3f0 msgr2=0x7fc25415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:09.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.270+0000 7fc2527fc640 1 --2- 172.21.15.84:0/2963556536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc25415d3f0 0x7fc25415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.270+0000 7fc2527fc640 1 -- 172.21.15.84:0/2963556536 >> v2:172.21.15.157:3300/0 conn(0x7fc2540cd0c0 msgr2=0x7fc2541615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:09.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.270+0000 7fc252ffd640 1 --2- 172.21.15.84:0/2963556536 >> v2:172.21.15.157:3300/0 conn(0x7fc2540cd0c0 0x7fc2541615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:09.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.270+0000 7fc2527fc640 1 --2- 172.21.15.84:0/2963556536 >> v2:172.21.15.157:3300/0 conn(0x7fc2540cd0c0 0x7fc2541615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.270+0000 7fc2527fc640 1 -- 172.21.15.84:0/2963556536 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25415e100 con 0x7fc2540b0410 2023-10-16T11:24:09.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.270+0000 7fc252ffd640 1 --2- 172.21.15.84:0/2963556536 >> v2:172.21.15.157:3300/0 conn(0x7fc2540cd0c0 0x7fc2541615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:09.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.271+0000 7fc2527fc640 1 --2- 172.21.15.84:0/2963556536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2540b0410 0x7fc2540b08a0 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7fc2540cc4e0 tx=0x7fc24802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80827b17eaeab7b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:09.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.272+0000 7fc251ffb640 1 -- 172.21.15.84:0/2963556536 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2480317e0 con 0x7fc2540b0410 2023-10-16T11:24:09.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.272+0000 7fc251ffb640 1 -- 172.21.15.84:0/2963556536 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc248031940 con 0x7fc2540b0410 2023-10-16T11:24:09.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.272+0000 7fc251ffb640 1 -- 172.21.15.84:0/2963556536 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248031cd0 con 0x7fc2540b0410 2023-10-16T11:24:09.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.272+0000 7fc259415640 1 -- 172.21.15.84:0/2963556536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2540b0410 msgr2=0x7fc2540b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:09.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.272+0000 7fc259415640 1 --2- 172.21.15.84:0/2963556536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2540b0410 0x7fc2540b08a0 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7fc2540cc4e0 tx=0x7fc24802b880 comp rx=0 tx=0).stop 2023-10-16T11:24:09.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.272+0000 7fc259415640 1 -- 172.21.15.84:0/2963556536 shutdown_connections 2023-10-16T11:24:09.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.272+0000 7fc259415640 1 --2- 172.21.15.84:0/2963556536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2540b0410 0x7fc2540b08a0 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.272+0000 7fc259415640 1 --2- 172.21.15.84:0/2963556536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc25415d3f0 0x7fc25415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.272+0000 7fc259415640 1 --2- 172.21.15.84:0/2963556536 >> v2:172.21.15.157:3300/0 conn(0x7fc2540cd0c0 0x7fc2541615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.272+0000 7fc259415640 1 -- 172.21.15.84:0/2963556536 >> 172.21.15.84:0/2963556536 conn(0x7fc2540cb410 msgr2=0x7fc2540cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:09.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.272+0000 7fc259415640 1 -- 172.21.15.84:0/2963556536 shutdown_connections 2023-10-16T11:24:09.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.272+0000 7fc259415640 1 -- 172.21.15.84:0/2963556536 wait complete. 2023-10-16T11:24:09.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.274+0000 7fc259415640 1 Processor -- start 2023-10-16T11:24:09.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.274+0000 7fc259415640 1 -- start start 2023-10-16T11:24:09.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.275+0000 7fc259415640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2540b0410 0x7fc254146580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:09.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.275+0000 7fc259415640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2540cd0c0 0x7fc2540c2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:09.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.275+0000 7fc259415640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc25415d3f0 0x7fc25415c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:09.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.275+0000 7fc259415640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2540c3fe0 con 0x7fc25415d3f0 2023-10-16T11:24:09.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.275+0000 7fc259415640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2541436b0 con 0x7fc2540b0410 2023-10-16T11:24:09.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.275+0000 7fc259415640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2540c52b0 con 0x7fc2540cd0c0 2023-10-16T11:24:09.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.275+0000 7fc2537fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc25415d3f0 0x7fc25415c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:09.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.275+0000 7fc2537fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc25415d3f0 0x7fc25415c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44756/0 (socket says 172.21.15.84:44756) 2023-10-16T11:24:09.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.275+0000 7fc2537fe640 1 -- 172.21.15.84:0/2587836554 learned_addr learned my addr 172.21.15.84:0/2587836554 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:09.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.276+0000 7fc252ffd640 1 --2- 172.21.15.84:0/2587836554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2540b0410 0x7fc254146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:09.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.276+0000 7fc2537fe640 1 -- 172.21.15.84:0/2587836554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2540cd0c0 msgr2=0x7fc2540c2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:09.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.276+0000 7fc2537fe640 1 --2- 172.21.15.84:0/2587836554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2540cd0c0 0x7fc2540c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.276+0000 7fc2537fe640 1 -- 172.21.15.84:0/2587836554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2540b0410 msgr2=0x7fc254146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:09.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.276+0000 7fc2537fe640 1 --2- 172.21.15.84:0/2587836554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2540b0410 0x7fc254146580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.276+0000 7fc2537fe640 1 -- 172.21.15.84:0/2587836554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2480094d0 con 0x7fc25415d3f0 2023-10-16T11:24:09.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.276+0000 7fc252ffd640 1 --2- 172.21.15.84:0/2587836554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2540b0410 0x7fc254146580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:09.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.276+0000 7fc2537fe640 1 --2- 172.21.15.84:0/2587836554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc25415d3f0 0x7fc25415c5e0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00b970 tx=0x7fc24c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:09.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.276+0000 7fc23b7fe640 1 -- 172.21.15.84:0/2587836554 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c00cb40 con 0x7fc25415d3f0 2023-10-16T11:24:09.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.276+0000 7fc23b7fe640 1 -- 172.21.15.84:0/2587836554 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24c00ccc0 con 0x7fc25415d3f0 2023-10-16T11:24:09.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.276+0000 7fc23b7fe640 1 -- 172.21.15.84:0/2587836554 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c014680 con 0x7fc25415d3f0 2023-10-16T11:24:09.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.276+0000 7fc259415640 1 -- 172.21.15.84:0/2587836554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2540c4210 con 0x7fc25415d3f0 2023-10-16T11:24:09.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.276+0000 7fc259415640 1 -- 172.21.15.84:0/2587836554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2540c47d0 con 0x7fc25415d3f0 2023-10-16T11:24:09.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.278+0000 7fc259415640 1 -- 172.21.15.84:0/2587836554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc25404e160 con 0x7fc25415d3f0 2023-10-16T11:24:09.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.279+0000 7fc23b7fe640 1 -- 172.21.15.84:0/2587836554 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 130) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fc24c01d3b0 con 0x7fc25415d3f0 2023-10-16T11:24:09.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.280+0000 7fc23b7fe640 1 --2- 172.21.15.84:0/2587836554 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fc24006acc0 0x7fc24006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:09.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.280+0000 7fc23b7fe640 1 -- 172.21.15.84:0/2587836554 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fc24c089560 con 0x7fc25415d3f0 2023-10-16T11:24:09.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.280+0000 7fc252ffd640 1 --2- 172.21.15.84:0/2587836554 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fc24006acc0 0x7fc24006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:09.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.281+0000 7fc252ffd640 1 --2- 172.21.15.84:0/2587836554 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fc24006acc0 0x7fc24006d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc244006fd0 tx=0x7fc244008040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:09.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.284+0000 7fc23b7fe640 1 -- 172.21.15.84:0/2587836554 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc24c0545d0 con 0x7fc25415d3f0 2023-10-16T11:24:09.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.514+0000 7fc259415640 1 -- 172.21.15.84:0/2587836554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2540b4910 con 0x7fc25415d3f0 2023-10-16T11:24:09.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.515+0000 7fc23b7fe640 1 -- 172.21.15.84:0/2587836554 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4929 (secure 0 0 0) 0x7fc24c019090 con 0x7fc25415d3f0 2023-10-16T11:24:09.517 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:09.517 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6575,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/855929374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":855929374},{"type":"v1","addr":"172.21.15.157:6834","nonce":855929374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6578,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2581460972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2581460972},{"type":"v1","addr":"172.21.15.157:6835","nonce":2581460972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":10806,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1731880648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1731880648},{"type":"v1","addr":"172.21.15.84:6836","nonce":1731880648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:22:42.578443+0000","modified":"2023-10-16T11:22:45.635565+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10809":{"gid":10809,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2679249765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2679249765},{"type":"v1","addr":"172.21.15.84:6837","nonce":2679249765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-16T11:24:09.518 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 105 2023-10-16T11:24:09.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.517+0000 7fc259415640 1 -- 172.21.15.84:0/2587836554 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fc24006acc0 msgr2=0x7fc24006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:09.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.517+0000 7fc259415640 1 --2- 172.21.15.84:0/2587836554 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fc24006acc0 0x7fc24006d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc244006fd0 tx=0x7fc244008040 comp rx=0 tx=0).stop 2023-10-16T11:24:09.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.517+0000 7fc259415640 1 -- 172.21.15.84:0/2587836554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc25415d3f0 msgr2=0x7fc25415c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:09.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.517+0000 7fc259415640 1 --2- 172.21.15.84:0/2587836554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc25415d3f0 0x7fc25415c5e0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00b970 tx=0x7fc24c00be30 comp rx=0 tx=0).stop 2023-10-16T11:24:09.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.517+0000 7fc259415640 1 -- 172.21.15.84:0/2587836554 shutdown_connections 2023-10-16T11:24:09.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.517+0000 7fc259415640 1 --2- 172.21.15.84:0/2587836554 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fc24006acc0 0x7fc24006d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.517+0000 7fc259415640 1 --2- 172.21.15.84:0/2587836554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc25415d3f0 0x7fc25415c5e0 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.517+0000 7fc259415640 1 --2- 172.21.15.84:0/2587836554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2540cd0c0 0x7fc2540c2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.517+0000 7fc259415640 1 --2- 172.21.15.84:0/2587836554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2540b0410 0x7fc254146580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.517+0000 7fc259415640 1 -- 172.21.15.84:0/2587836554 >> 172.21.15.84:0/2587836554 conn(0x7fc2540cb410 msgr2=0x7fc25415b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:09.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.518+0000 7fc259415640 1 -- 172.21.15.84:0/2587836554 shutdown_connections 2023-10-16T11:24:09.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.518+0000 7fc259415640 1 -- 172.21.15.84:0/2587836554 wait complete. 2023-10-16T11:24:09.537 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:24:09.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.753+0000 7f10c1c56640 1 Processor -- start 2023-10-16T11:24:09.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.754+0000 7f10c1c56640 1 -- start start 2023-10-16T11:24:09.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.755+0000 7f10c1c56640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10bc1615f0 0x7f10bc15db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:09.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.755+0000 7f10c1c56640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10bc15e3e0 con 0x7f10bc0cd0c0 2023-10-16T11:24:09.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.755+0000 7f10c1c56640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10bc0cd4a0 con 0x7f10bc0b0b20 2023-10-16T11:24:09.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.755+0000 7f10c1c56640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc15e100 con 0x7f10bc1615f0 2023-10-16T11:24:09.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.755+0000 7f10bb7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f10bc0cd0c0 0x7f10bc0b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48632/0 (socket says 172.21.15.84:48632) 2023-10-16T11:24:09.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.755+0000 7f10bb7fe640 1 -- 172.21.15.84:0/1023880600 learned_addr learned my addr 172.21.15.84:0/1023880600 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:09.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.756+0000 7f10bbfff640 1 --2- 172.21.15.84:0/1023880600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10bc1615f0 0x7f10bc15db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:09.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.756+0000 7f10baffd640 1 -- 172.21.15.84:0/1023880600 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507482650 0 0) 0x7f10bc15e3e0 con 0x7f10bc0cd0c0 2023-10-16T11:24:09.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.756+0000 7f10baffd640 1 -- 172.21.15.84:0/1023880600 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10a8003520 con 0x7f10bc0cd0c0 2023-10-16T11:24:09.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.756+0000 7f10baffd640 1 -- 172.21.15.84:0/1023880600 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2041904675 0 0) 0x7f10a8003520 con 0x7f10bc0cd0c0 2023-10-16T11:24:09.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.756+0000 7f10baffd640 1 -- 172.21.15.84:0/1023880600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10bc1615f0 msgr2=0x7f10bc15db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:09.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.756+0000 7f10baffd640 1 --2- 172.21.15.84:0/1023880600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10bc1615f0 0x7f10bc15db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.756+0000 7f10baffd640 1 -- 172.21.15.84:0/1023880600 >> v1:172.21.15.157:6789/0 conn(0x7f10bc0b0b20 legacy=0x7f10bc15d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:09.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.756+0000 7f10baffd640 1 -- 172.21.15.84:0/1023880600 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10bc160b10 con 0x7f10bc0cd0c0 2023-10-16T11:24:09.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.756+0000 7f10baffd640 1 -- 172.21.15.84:0/1023880600 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f10ac001670 con 0x7f10bc0cd0c0 2023-10-16T11:24:09.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.756+0000 7f10baffd640 1 -- 172.21.15.84:0/1023880600 >> v1:172.21.15.84:6789/0 conn(0x7f10bc0cd0c0 legacy=0x7f10bc0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:09.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.756+0000 7f10baffd640 1 --2- 172.21.15.84:0/1023880600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10a80039a0 0x7f10a8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:09.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.757+0000 7f10baffd640 1 --2- 172.21.15.84:0/1023880600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10a80047c0 0x7f10a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:09.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.757+0000 7f10baffd640 1 --2- 172.21.15.84:0/1023880600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10bc1615f0 0x7f10a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:09.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.757+0000 7f10baffd640 1 -- 172.21.15.84:0/1023880600 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc15e100 con 0x7f10a80047c0 2023-10-16T11:24:09.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.757+0000 7f10baffd640 1 -- 172.21.15.84:0/1023880600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a800bab0 con 0x7f10a80039a0 2023-10-16T11:24:09.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.757+0000 7f10baffd640 1 -- 172.21.15.84:0/1023880600 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a800bc70 con 0x7f10bc1615f0 2023-10-16T11:24:09.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.757+0000 7f10b3fff640 1 --2- 172.21.15.84:0/1023880600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10a80047c0 0x7f10a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:09.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.757+0000 7f10bb7fe640 1 --2- 172.21.15.84:0/1023880600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10a80039a0 0x7f10a8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:09.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.757+0000 7f10b3fff640 1 -- 172.21.15.84:0/1023880600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10bc1615f0 msgr2=0x7f10a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:09.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.757+0000 7f10b3fff640 1 --2- 172.21.15.84:0/1023880600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10bc1615f0 0x7f10a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.757+0000 7f10b3fff640 1 -- 172.21.15.84:0/1023880600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10a80039a0 msgr2=0x7f10a8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:09.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.757+0000 7f10b3fff640 1 --2- 172.21.15.84:0/1023880600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10a80039a0 0x7f10a8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.757+0000 7f10b3fff640 1 -- 172.21.15.84:0/1023880600 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a8003520 con 0x7f10a80047c0 2023-10-16T11:24:09.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.758+0000 7f10b3fff640 1 --2- 172.21.15.84:0/1023880600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10a80047c0 0x7f10a8004c40 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f10a4001750 tx=0x7f10a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efdc65331da2b35 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:09.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.759+0000 7f10baffd640 1 -- 172.21.15.84:0/1023880600 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a40052d0 con 0x7f10a80047c0 2023-10-16T11:24:09.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.759+0000 7f10baffd640 1 -- 172.21.15.84:0/1023880600 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a402e050 con 0x7f10a80047c0 2023-10-16T11:24:09.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.759+0000 7f10baffd640 1 -- 172.21.15.84:0/1023880600 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a40035f0 con 0x7f10a80047c0 2023-10-16T11:24:09.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.759+0000 7f10c1c56640 1 -- 172.21.15.84:0/1023880600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10a80047c0 msgr2=0x7f10a8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:09.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.759+0000 7f10c1c56640 1 --2- 172.21.15.84:0/1023880600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10a80047c0 0x7f10a8004c40 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f10a4001750 tx=0x7f10a80057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.759+0000 7f10bbfff640 1 -- 172.21.15.84:0/1023880600 reap_dead start 2023-10-16T11:24:09.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.759+0000 7f10c1c56640 1 -- 172.21.15.84:0/1023880600 shutdown_connections 2023-10-16T11:24:09.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.759+0000 7f10c1c56640 1 -- 172.21.15.84:0/1023880600 >> 172.21.15.84:0/1023880600 conn(0x7f10bc0cb410 msgr2=0x7f10bc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:09.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.759+0000 7f10c1c56640 1 -- 172.21.15.84:0/1023880600 shutdown_connections 2023-10-16T11:24:09.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.759+0000 7f10c1c56640 1 -- 172.21.15.84:0/1023880600 wait complete. 2023-10-16T11:24:09.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.761+0000 7f10c1c56640 1 Processor -- start 2023-10-16T11:24:09.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.761+0000 7f10c1c56640 1 -- start start 2023-10-16T11:24:09.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.762+0000 7f10c1c56640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10a80047c0 0x7f10bc1f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:09.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.762+0000 7f10c1c56640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10bc0c6440 0x7f10bc148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:09.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.762+0000 7f10c1c56640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10bc0ca720 0x7f10bc0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:09.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.762+0000 7f10c1c56640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc0adc70 con 0x7f10bc0ca720 2023-10-16T11:24:09.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.762+0000 7f10c1c56640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc0cb0e0 con 0x7f10a80047c0 2023-10-16T11:24:09.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.762+0000 7f10c1c56640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc1f6cc0 con 0x7f10bc0c6440 2023-10-16T11:24:09.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.763+0000 7f10bbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10bc0ca720 0x7f10bc0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:09.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.763+0000 7f10bbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10bc0ca720 0x7f10bc0caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44774/0 (socket says 172.21.15.84:44774) 2023-10-16T11:24:09.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.763+0000 7f10bbfff640 1 -- 172.21.15.84:0/827700271 learned_addr learned my addr 172.21.15.84:0/827700271 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:09.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.763+0000 7f10bb7fe640 1 --2- 172.21.15.84:0/827700271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10a80047c0 0x7f10bc1f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:09.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.763+0000 7f10bbfff640 1 -- 172.21.15.84:0/827700271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10bc0c6440 msgr2=0x7f10bc148d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:09.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.763+0000 7f10b3fff640 1 --2- 172.21.15.84:0/827700271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10bc0c6440 0x7f10bc148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:09.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.763+0000 7f10bbfff640 1 --2- 172.21.15.84:0/827700271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10bc0c6440 0x7f10bc148d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.763+0000 7f10bbfff640 1 -- 172.21.15.84:0/827700271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10a80047c0 msgr2=0x7f10bc1f5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:09.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.763+0000 7f10bbfff640 1 --2- 172.21.15.84:0/827700271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10a80047c0 0x7f10bc1f5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.763+0000 7f10bbfff640 1 -- 172.21.15.84:0/827700271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10bc0c68c0 con 0x7f10bc0ca720 2023-10-16T11:24:09.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.763+0000 7f10bb7fe640 1 --2- 172.21.15.84:0/827700271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10a80047c0 0x7f10bc1f5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:09.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.763+0000 7f10bbfff640 1 --2- 172.21.15.84:0/827700271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10bc0ca720 0x7f10bc0caba0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0c8f70 tx=0x7f10b400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:09.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.764+0000 7f10b8ff9640 1 -- 172.21.15.84:0/827700271 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b4010040 con 0x7f10bc0ca720 2023-10-16T11:24:09.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.764+0000 7f10b8ff9640 1 -- 172.21.15.84:0/827700271 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b400bce0 con 0x7f10bc0ca720 2023-10-16T11:24:09.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.764+0000 7f10b8ff9640 1 -- 172.21.15.84:0/827700271 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b4014640 con 0x7f10bc0ca720 2023-10-16T11:24:09.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.764+0000 7f10c1c56640 1 -- 172.21.15.84:0/827700271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10bc1f6420 con 0x7f10bc0ca720 2023-10-16T11:24:09.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.764+0000 7f10c1c56640 1 -- 172.21.15.84:0/827700271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10bc1f69e0 con 0x7f10bc0ca720 2023-10-16T11:24:09.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.765+0000 7f10c1c56640 1 -- 172.21.15.84:0/827700271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10bc04e160 con 0x7f10bc0ca720 2023-10-16T11:24:09.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.766+0000 7f10b8ff9640 1 -- 172.21.15.84:0/827700271 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 130) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f10b40147a0 con 0x7f10bc0ca720 2023-10-16T11:24:09.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.767+0000 7f10b8ff9640 1 --2- 172.21.15.84:0/827700271 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f109c06acc0 0x7f109c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:09.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.767+0000 7f10b8ff9640 1 -- 172.21.15.84:0/827700271 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f10b4089e80 con 0x7f10bc0ca720 2023-10-16T11:24:09.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.767+0000 7f10bb7fe640 1 --2- 172.21.15.84:0/827700271 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f109c06acc0 0x7f109c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:09.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.767+0000 7f10bb7fe640 1 --2- 172.21.15.84:0/827700271 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f109c06acc0 0x7f109c06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f10ac001eb0 tx=0x7f10ac000f50 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:09.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.771+0000 7f10b8ff9640 1 -- 172.21.15.84:0/827700271 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f10b4054e20 con 0x7f10bc0ca720 2023-10-16T11:24:09.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.970+0000 7f10c1c56640 1 -- 172.21.15.84:0/827700271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10bc0b4910 con 0x7f10bc0ca720 2023-10-16T11:24:09.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.972+0000 7f10b8ff9640 1 -- 172.21.15.84:0/827700271 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v206) v1 ==== 74+0+17829 (secure 0 0 0) 0x7f10b4018090 con 0x7f10bc0ca720 2023-10-16T11:24:09.975 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:09.975 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":206,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:24:08.217312+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:22:40.385524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-16T11:22:41.592928+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":205,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.18","mappings":[{"from":0,"to":2}]},{"pgid":"25.1f","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/65166914":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/26641613":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/1473149491":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/3116175047":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/4203014950":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/1585465362":"2023-10-17T11:24:08.217257+0000","172.21.15.84:6832/855077515":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/3564300528":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/2349522146":"2023-10-17T11:22:56.900947+0000","172.21.15.84:6833/855077515":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/2191803332":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/2199276620":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/3418988137":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/4262224959":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/1032287219":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/956990500":"2023-10-17T11:22:56.900947+0000","172.21.15.84:6833/1985463812":"2023-10-17T11:22:56.900947+0000","172.21.15.84:6832/1985463812":"2023-10-17T11:22:56.900947+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:24:09.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.975+0000 7f10c1c56640 1 -- 172.21.15.84:0/827700271 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f109c06acc0 msgr2=0x7f109c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:09.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.975+0000 7f10c1c56640 1 --2- 172.21.15.84:0/827700271 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f109c06acc0 0x7f109c06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f10ac001eb0 tx=0x7f10ac000f50 comp rx=0 tx=0).stop 2023-10-16T11:24:09.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.975+0000 7f10c1c56640 1 -- 172.21.15.84:0/827700271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10bc0ca720 msgr2=0x7f10bc0caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:09.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.975+0000 7f10c1c56640 1 --2- 172.21.15.84:0/827700271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10bc0ca720 0x7f10bc0caba0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0c8f70 tx=0x7f10b400b480 comp rx=0 tx=0).stop 2023-10-16T11:24:09.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.975+0000 7f10c1c56640 1 -- 172.21.15.84:0/827700271 shutdown_connections 2023-10-16T11:24:09.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.975+0000 7f10c1c56640 1 --2- 172.21.15.84:0/827700271 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f109c06acc0 0x7f109c06d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.975+0000 7f10c1c56640 1 --2- 172.21.15.84:0/827700271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10bc0ca720 0x7f10bc0caba0 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.975+0000 7f10c1c56640 1 --2- 172.21.15.84:0/827700271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10bc0c6440 0x7f10bc148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.975+0000 7f10c1c56640 1 --2- 172.21.15.84:0/827700271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10a80047c0 0x7f10bc1f5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:09.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.975+0000 7f10c1c56640 1 -- 172.21.15.84:0/827700271 >> 172.21.15.84:0/827700271 conn(0x7f10bc0cb410 msgr2=0x7f10bc06cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:09.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.975+0000 7f10c1c56640 1 -- 172.21.15.84:0/827700271 shutdown_connections 2023-10-16T11:24:09.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:09.975+0000 7f10c1c56640 1 -- 172.21.15.84:0/827700271 wait complete. 2023-10-16T11:24:09.995 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T11:24:09.995 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:24:10.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.207+0000 7f7dfd4c9640 1 Processor -- start 2023-10-16T11:24:10.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.207+0000 7f7dfd4c9640 1 -- start start 2023-10-16T11:24:10.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.208+0000 7f7dfd4c9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7df8159b20 0x7f7df806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:10.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.208+0000 7f7dfd4c9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7df8159f00 0x7f7df80b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:10.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.208+0000 7f7dfd4c9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df806f7f0 con 0x7f7df806c120 2023-10-16T11:24:10.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.209+0000 7f7dfd4c9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df806c5e0 con 0x7f7df8159f00 2023-10-16T11:24:10.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.209+0000 7f7dfd4c9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df806c8c0 con 0x7f7df8159b20 2023-10-16T11:24:10.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.209+0000 7f7df67fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7df8159f00 0x7f7df80b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:10.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.209+0000 7f7df67fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7df8159f00 0x7f7df80b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44778/0 (socket says 172.21.15.84:44778) 2023-10-16T11:24:10.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.209+0000 7f7df77fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7df806c120 0x7f7df80cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48642/0 (socket says 172.21.15.84:48642) 2023-10-16T11:24:10.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.209+0000 7f7df67fc640 1 -- 172.21.15.84:0/4264743514 learned_addr learned my addr 172.21.15.84:0/4264743514 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:10.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.209+0000 7f7df6ffd640 1 --2- 172.21.15.84:0/4264743514 >> v2:172.21.15.157:3300/0 conn(0x7f7df8159b20 0x7f7df806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:10.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.209+0000 7f7df67fc640 1 -- 172.21.15.84:0/4264743514 >> v2:172.21.15.157:3300/0 conn(0x7f7df8159b20 msgr2=0x7f7df806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:10.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.209+0000 7f7df67fc640 1 --2- 172.21.15.84:0/4264743514 >> v2:172.21.15.157:3300/0 conn(0x7f7df8159b20 0x7f7df806bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.209+0000 7f7df67fc640 1 -- 172.21.15.84:0/4264743514 >> v1:172.21.15.84:6789/0 conn(0x7f7df806c120 legacy=0x7f7df80cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:10.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.209+0000 7f7df67fc640 1 -- 172.21.15.84:0/4264743514 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df806cba0 con 0x7f7df8159f00 2023-10-16T11:24:10.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.209+0000 7f7df5ffb640 1 -- 172.21.15.84:0/4264743514 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567365825 0 0) 0x7f7df806f7f0 con 0x7f7df806c120 2023-10-16T11:24:10.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.210+0000 7f7df67fc640 1 --2- 172.21.15.84:0/4264743514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df8159f00 0x7f7df80b0410 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f7df80c9340 tx=0x7f7de002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20454469c6086765 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:10.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.211+0000 7f7df5ffb640 1 -- 172.21.15.84:0/4264743514 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de0030860 con 0x7f7df8159f00 2023-10-16T11:24:10.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.211+0000 7f7df5ffb640 1 -- 172.21.15.84:0/4264743514 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de00309c0 con 0x7f7df8159f00 2023-10-16T11:24:10.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.211+0000 7f7df5ffb640 1 -- 172.21.15.84:0/4264743514 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de0030d50 con 0x7f7df8159f00 2023-10-16T11:24:10.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.211+0000 7f7dfd4c9640 1 -- 172.21.15.84:0/4264743514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df8159f00 msgr2=0x7f7df80b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:10.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.211+0000 7f7dfd4c9640 1 --2- 172.21.15.84:0/4264743514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df8159f00 0x7f7df80b0410 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f7df80c9340 tx=0x7f7de002b9a0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.211+0000 7f7dfd4c9640 1 -- 172.21.15.84:0/4264743514 shutdown_connections 2023-10-16T11:24:10.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.211+0000 7f7dfd4c9640 1 --2- 172.21.15.84:0/4264743514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df8159f00 0x7f7df80b0410 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.211+0000 7f7dfd4c9640 1 --2- 172.21.15.84:0/4264743514 >> v2:172.21.15.157:3300/0 conn(0x7f7df8159b20 0x7f7df806bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.211+0000 7f7dfd4c9640 1 -- 172.21.15.84:0/4264743514 >> 172.21.15.84:0/4264743514 conn(0x7f7df80cb410 msgr2=0x7f7df80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:10.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.211+0000 7f7dfd4c9640 1 -- 172.21.15.84:0/4264743514 shutdown_connections 2023-10-16T11:24:10.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.211+0000 7f7dfd4c9640 1 -- 172.21.15.84:0/4264743514 wait complete. 2023-10-16T11:24:10.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.213+0000 7f7dfd4c9640 1 Processor -- start 2023-10-16T11:24:10.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.213+0000 7f7dfd4c9640 1 -- start start 2023-10-16T11:24:10.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.214+0000 7f7dfd4c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7df806c120 0x7f7df81549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:10.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.214+0000 7f7dfd4c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7df8159b20 0x7f7df8150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:10.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.214+0000 7f7dfd4c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df8159f00 0x7f7df815b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:10.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.214+0000 7f7dfd4c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df814fac0 con 0x7f7df8159f00 2023-10-16T11:24:10.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.214+0000 7f7dfd4c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df815c580 con 0x7f7df8159b20 2023-10-16T11:24:10.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.214+0000 7f7dfd4c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df815bfc0 con 0x7f7df806c120 2023-10-16T11:24:10.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.214+0000 7f7df77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df8159f00 0x7f7df815b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:10.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.214+0000 7f7df77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df8159f00 0x7f7df815b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44792/0 (socket says 172.21.15.84:44792) 2023-10-16T11:24:10.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.214+0000 7f7df77fe640 1 -- 172.21.15.84:0/1424833198 learned_addr learned my addr 172.21.15.84:0/1424833198 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:10.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.215+0000 7f7df6ffd640 1 --2- 172.21.15.84:0/1424833198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7df806c120 0x7f7df81549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:10.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.215+0000 7f7df77fe640 1 -- 172.21.15.84:0/1424833198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7df806c120 msgr2=0x7f7df81549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:10.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.215+0000 7f7df77fe640 1 --2- 172.21.15.84:0/1424833198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7df806c120 0x7f7df81549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.215+0000 7f7df77fe640 1 -- 172.21.15.84:0/1424833198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7df8159b20 msgr2=0x7f7df8150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:10.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.215+0000 7f7df77fe640 1 --2- 172.21.15.84:0/1424833198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7df8159b20 0x7f7df8150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.215+0000 7f7df77fe640 1 -- 172.21.15.84:0/1424833198 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de00094d0 con 0x7f7df8159f00 2023-10-16T11:24:10.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.215+0000 7f7df77fe640 1 --2- 172.21.15.84:0/1424833198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df8159f00 0x7f7df815b500 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f7df000c700 tx=0x7f7df000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:10.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.215+0000 7f7def7fe640 1 -- 172.21.15.84:0/1424833198 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df000ce10 con 0x7f7df8159f00 2023-10-16T11:24:10.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.216+0000 7f7def7fe640 1 -- 172.21.15.84:0/1424833198 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df000a640 con 0x7f7df8159f00 2023-10-16T11:24:10.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.216+0000 7f7def7fe640 1 -- 172.21.15.84:0/1424833198 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df000aa70 con 0x7f7df8159f00 2023-10-16T11:24:10.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.216+0000 7f7dfd4c9640 1 -- 172.21.15.84:0/1424833198 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df815c2a0 con 0x7f7df8159f00 2023-10-16T11:24:10.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.216+0000 7f7dfd4c9640 1 -- 172.21.15.84:0/1424833198 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df814e4d0 con 0x7f7df8159f00 2023-10-16T11:24:10.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.217+0000 7f7dfd4c9640 1 -- 172.21.15.84:0/1424833198 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df814f920 con 0x7f7df8159f00 2023-10-16T11:24:10.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.218+0000 7f7def7fe640 1 -- 172.21.15.84:0/1424833198 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 130) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f7df000abd0 con 0x7f7df8159f00 2023-10-16T11:24:10.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.219+0000 7f7def7fe640 1 --2- 172.21.15.84:0/1424833198 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f7dd806acf0 0x7f7dd806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:10.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.219+0000 7f7def7fe640 1 -- 172.21.15.84:0/1424833198 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f7df0088a10 con 0x7f7df8159f00 2023-10-16T11:24:10.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.219+0000 7f7df6ffd640 1 --2- 172.21.15.84:0/1424833198 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f7dd806acf0 0x7f7dd806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:10.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.220+0000 7f7df6ffd640 1 --2- 172.21.15.84:0/1424833198 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f7dd806acf0 0x7f7dd806d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7de8006fd0 tx=0x7f7de8008040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:10.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.224+0000 7f7def7fe640 1 -- 172.21.15.84:0/1424833198 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7df814f920 con 0x7f7df8159f00 2023-10-16T11:24:10.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.459+0000 7f7dfd4c9640 1 -- 172.21.15.84:0/1424833198 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7df814ffa0 con 0x7f7df8159f00 2023-10-16T11:24:10.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.460+0000 7f7def7fe640 1 -- 172.21.15.84:0/1424833198 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4929 (secure 0 0 0) 0x7f7df814ffa0 con 0x7f7df8159f00 2023-10-16T11:24:10.462 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:10.462 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6575,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/855929374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":855929374},{"type":"v1","addr":"172.21.15.157:6834","nonce":855929374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6578,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2581460972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2581460972},{"type":"v1","addr":"172.21.15.157:6835","nonce":2581460972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":10806,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1731880648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1731880648},{"type":"v1","addr":"172.21.15.84:6836","nonce":1731880648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:22:42.578443+0000","modified":"2023-10-16T11:22:45.635565+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10809":{"gid":10809,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2679249765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2679249765},{"type":"v1","addr":"172.21.15.84:6837","nonce":2679249765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-16T11:24:10.463 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 105 2023-10-16T11:24:10.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.463+0000 7f7dfd4c9640 1 -- 172.21.15.84:0/1424833198 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f7dd806acf0 msgr2=0x7f7dd806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:10.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.463+0000 7f7dfd4c9640 1 --2- 172.21.15.84:0/1424833198 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f7dd806acf0 0x7f7dd806d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7de8006fd0 tx=0x7f7de8008040 comp rx=0 tx=0).stop 2023-10-16T11:24:10.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.463+0000 7f7dfd4c9640 1 -- 172.21.15.84:0/1424833198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df8159f00 msgr2=0x7f7df815b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:10.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.463+0000 7f7dfd4c9640 1 --2- 172.21.15.84:0/1424833198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df8159f00 0x7f7df815b500 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f7df000c700 tx=0x7f7df000cbc0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.463+0000 7f7dfd4c9640 1 -- 172.21.15.84:0/1424833198 shutdown_connections 2023-10-16T11:24:10.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.463+0000 7f7dfd4c9640 1 --2- 172.21.15.84:0/1424833198 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f7dd806acf0 0x7f7dd806d1b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.463+0000 7f7dfd4c9640 1 --2- 172.21.15.84:0/1424833198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7df8159f00 0x7f7df815b500 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.463+0000 7f7dfd4c9640 1 --2- 172.21.15.84:0/1424833198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7df8159b20 0x7f7df8150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.463+0000 7f7dfd4c9640 1 --2- 172.21.15.84:0/1424833198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7df806c120 0x7f7df81549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.463+0000 7f7dfd4c9640 1 -- 172.21.15.84:0/1424833198 >> 172.21.15.84:0/1424833198 conn(0x7f7df80cb410 msgr2=0x7f7df80cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:10.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.463+0000 7f7dfd4c9640 1 -- 172.21.15.84:0/1424833198 shutdown_connections 2023-10-16T11:24:10.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.463+0000 7f7dfd4c9640 1 -- 172.21.15.84:0/1424833198 wait complete. 2023-10-16T11:24:10.483 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:24:10.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.693+0000 7f4c5b568640 1 Processor -- start 2023-10-16T11:24:10.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.693+0000 7f4c5b568640 1 -- start start 2023-10-16T11:24:10.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.695+0000 7f4c5b568640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4c5415ae80 0x7f4c5415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:10.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.695+0000 7f4c5b568640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c540ce390 0x7f4c54070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:10.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.695+0000 7f4c5b568640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c5415ed60 con 0x7f4c5415aaa0 2023-10-16T11:24:10.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.695+0000 7f4c5b568640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c5406b810 con 0x7f4c5415ae80 2023-10-16T11:24:10.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.695+0000 7f4c5b568640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c540707e0 con 0x7f4c540ce390 2023-10-16T11:24:10.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.695+0000 7f4c59304640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4c5415aaa0 0x7f4c540cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48644/0 (socket says 172.21.15.84:48644) 2023-10-16T11:24:10.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.695+0000 7f4c59304640 1 -- 172.21.15.84:0/4101282422 learned_addr learned my addr 172.21.15.84:0/4101282422 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:10.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.695+0000 7f4c58b03640 1 --2- 172.21.15.84:0/4101282422 >> v2:172.21.15.84:3300/0 conn(0x7f4c5415ae80 0x7f4c5415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:10.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.696+0000 7f4c53fff640 1 -- 172.21.15.84:0/4101282422 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3450356570 0 0) 0x7f4c5415ed60 con 0x7f4c5415aaa0 2023-10-16T11:24:10.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.696+0000 7f4c53fff640 1 -- 172.21.15.84:0/4101282422 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c44003520 con 0x7f4c5415aaa0 2023-10-16T11:24:10.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.696+0000 7f4c59b05640 1 --2- 172.21.15.84:0/4101282422 >> v2:172.21.15.157:3300/0 conn(0x7f4c540ce390 0x7f4c54070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:10.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.696+0000 7f4c58b03640 1 -- 172.21.15.84:0/4101282422 >> v2:172.21.15.157:3300/0 conn(0x7f4c540ce390 msgr2=0x7f4c54070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:10.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.696+0000 7f4c58b03640 1 --2- 172.21.15.84:0/4101282422 >> v2:172.21.15.157:3300/0 conn(0x7f4c540ce390 0x7f4c54070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.696+0000 7f4c58b03640 1 -- 172.21.15.84:0/4101282422 >> v1:172.21.15.84:6789/0 conn(0x7f4c5415aaa0 legacy=0x7f4c540cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:10.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.696+0000 7f4c58b03640 1 -- 172.21.15.84:0/4101282422 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5415ef90 con 0x7f4c5415ae80 2023-10-16T11:24:10.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.696+0000 7f4c59b05640 1 --2- 172.21.15.84:0/4101282422 >> v2:172.21.15.157:3300/0 conn(0x7f4c540ce390 0x7f4c54070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:10.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.697+0000 7f4c58b03640 1 --2- 172.21.15.84:0/4101282422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c5415ae80 0x7f4c5415e820 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f4c540c2490 tx=0x7f4c4002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1741d0cf685f80 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:10.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.697+0000 7f4c53fff640 1 -- 172.21.15.84:0/4101282422 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40031830 con 0x7f4c5415ae80 2023-10-16T11:24:10.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.697+0000 7f4c53fff640 1 -- 172.21.15.84:0/4101282422 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c40031990 con 0x7f4c5415ae80 2023-10-16T11:24:10.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.697+0000 7f4c53fff640 1 -- 172.21.15.84:0/4101282422 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40031d20 con 0x7f4c5415ae80 2023-10-16T11:24:10.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.698+0000 7f4c5b568640 1 -- 172.21.15.84:0/4101282422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c5415ae80 msgr2=0x7f4c5415e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:10.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.698+0000 7f4c5b568640 1 --2- 172.21.15.84:0/4101282422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c5415ae80 0x7f4c5415e820 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f4c540c2490 tx=0x7f4c4002b8d0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.698+0000 7f4c5b568640 1 -- 172.21.15.84:0/4101282422 shutdown_connections 2023-10-16T11:24:10.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.698+0000 7f4c5b568640 1 --2- 172.21.15.84:0/4101282422 >> v2:172.21.15.157:3300/0 conn(0x7f4c540ce390 0x7f4c54070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.698+0000 7f4c5b568640 1 --2- 172.21.15.84:0/4101282422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c5415ae80 0x7f4c5415e820 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.698+0000 7f4c5b568640 1 -- 172.21.15.84:0/4101282422 >> 172.21.15.84:0/4101282422 conn(0x7f4c540cb410 msgr2=0x7f4c540cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:10.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.698+0000 7f4c5b568640 1 -- 172.21.15.84:0/4101282422 shutdown_connections 2023-10-16T11:24:10.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.698+0000 7f4c5b568640 1 -- 172.21.15.84:0/4101282422 wait complete. 2023-10-16T11:24:10.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.700+0000 7f4c5b568640 1 Processor -- start 2023-10-16T11:24:10.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.700+0000 7f4c5b568640 1 -- start start 2023-10-16T11:24:10.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.701+0000 7f4c5b568640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c540ce390 0x7f4c540ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:10.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.701+0000 7f4c5b568640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c5415aaa0 0x7f4c5415ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:10.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.701+0000 7f4c5b568640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c5415ae80 0x7f4c5414cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:10.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.701+0000 7f4c5b568640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5414a8c0 con 0x7f4c5415aaa0 2023-10-16T11:24:10.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.701+0000 7f4c5b568640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5414ad10 con 0x7f4c5415ae80 2023-10-16T11:24:10.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.701+0000 7f4c5b568640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5415de80 con 0x7f4c540ce390 2023-10-16T11:24:10.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.701+0000 7f4c58b03640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c5415aaa0 0x7f4c5415ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:10.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.701+0000 7f4c58b03640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c5415aaa0 0x7f4c5415ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44824/0 (socket says 172.21.15.84:44824) 2023-10-16T11:24:10.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.701+0000 7f4c58b03640 1 -- 172.21.15.84:0/3055490309 learned_addr learned my addr 172.21.15.84:0/3055490309 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:10.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.701+0000 7f4c59304640 1 --2- 172.21.15.84:0/3055490309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c540ce390 0x7f4c540ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:10.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.701+0000 7f4c58b03640 1 -- 172.21.15.84:0/3055490309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c540ce390 msgr2=0x7f4c540ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:10.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.701+0000 7f4c58b03640 1 --2- 172.21.15.84:0/3055490309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c540ce390 0x7f4c540ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.701+0000 7f4c58b03640 1 -- 172.21.15.84:0/3055490309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c5415ae80 msgr2=0x7f4c5414cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:10.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.701+0000 7f4c59b05640 1 --2- 172.21.15.84:0/3055490309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c5415ae80 0x7f4c5414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:10.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.701+0000 7f4c58b03640 1 --2- 172.21.15.84:0/3055490309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c5415ae80 0x7f4c5414cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.701+0000 7f4c58b03640 1 -- 172.21.15.84:0/3055490309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c40009400 con 0x7f4c5415aaa0 2023-10-16T11:24:10.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.701+0000 7f4c59304640 1 --2- 172.21.15.84:0/3055490309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c540ce390 0x7f4c540ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:10.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.702+0000 7f4c58b03640 1 --2- 172.21.15.84:0/3055490309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c5415aaa0 0x7f4c5415ce10 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f4c4002bee0 tx=0x7f4c400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:10.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.702+0000 7f4c51ffb640 1 -- 172.21.15.84:0/3055490309 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4003a8f0 con 0x7f4c5415aaa0 2023-10-16T11:24:10.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.702+0000 7f4c51ffb640 1 -- 172.21.15.84:0/3055490309 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4003aa70 con 0x7f4c5415aaa0 2023-10-16T11:24:10.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.702+0000 7f4c51ffb640 1 -- 172.21.15.84:0/3055490309 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4003ae20 con 0x7f4c5415aaa0 2023-10-16T11:24:10.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.702+0000 7f4c5b568640 1 -- 172.21.15.84:0/3055490309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5415d350 con 0x7f4c5415aaa0 2023-10-16T11:24:10.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.702+0000 7f4c5b568640 1 -- 172.21.15.84:0/3055490309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5415d8c0 con 0x7f4c5415aaa0 2023-10-16T11:24:10.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.703+0000 7f4c5b568640 1 -- 172.21.15.84:0/3055490309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5404e160 con 0x7f4c5415aaa0 2023-10-16T11:24:10.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.705+0000 7f4c51ffb640 1 -- 172.21.15.84:0/3055490309 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 130) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f4c400443f0 con 0x7f4c5415aaa0 2023-10-16T11:24:10.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.705+0000 7f4c51ffb640 1 --2- 172.21.15.84:0/3055490309 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4c3806acc0 0x7f4c3806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:10.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.706+0000 7f4c51ffb640 1 -- 172.21.15.84:0/3055490309 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f4c400ab7f0 con 0x7f4c5415aaa0 2023-10-16T11:24:10.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.706+0000 7f4c59304640 1 --2- 172.21.15.84:0/3055490309 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4c3806acc0 0x7f4c3806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:10.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.706+0000 7f4c59304640 1 --2- 172.21.15.84:0/3055490309 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4c3806acc0 0x7f4c3806d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4c48001560 tx=0x7f4c48009040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:10.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.710+0000 7f4c51ffb640 1 -- 172.21.15.84:0/3055490309 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4c400768b0 con 0x7f4c5415aaa0 2023-10-16T11:24:10.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.940+0000 7f4c5b568640 1 -- 172.21.15.84:0/3055490309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c540cdc60 con 0x7f4c5415aaa0 2023-10-16T11:24:10.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.951+0000 7f4c51ffb640 1 -- 172.21.15.84:0/3055490309 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 131) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f4c40076c80 con 0x7f4c5415aaa0 2023-10-16T11:24:10.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.953+0000 7f4c51ffb640 1 -- 172.21.15.84:0/3055490309 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4929 (secure 0 0 0) 0x7f4c4007dc30 con 0x7f4c5415aaa0 2023-10-16T11:24:10.955 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:10.956 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6575,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/855929374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":855929374},{"type":"v1","addr":"172.21.15.157:6834","nonce":855929374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6578,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2581460972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2581460972},{"type":"v1","addr":"172.21.15.157:6835","nonce":2581460972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":10806,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1731880648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1731880648},{"type":"v1","addr":"172.21.15.84:6836","nonce":1731880648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:22:42.578443+0000","modified":"2023-10-16T11:22:45.635565+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10809":{"gid":10809,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2679249765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2679249765},{"type":"v1","addr":"172.21.15.84:6837","nonce":2679249765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-16T11:24:10.956 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 105 2023-10-16T11:24:10.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.956+0000 7f4c5b568640 1 -- 172.21.15.84:0/3055490309 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4c3806acc0 msgr2=0x7f4c3806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:10.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.956+0000 7f4c5b568640 1 --2- 172.21.15.84:0/3055490309 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4c3806acc0 0x7f4c3806d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4c48001560 tx=0x7f4c48009040 comp rx=0 tx=0).stop 2023-10-16T11:24:10.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.956+0000 7f4c5b568640 1 -- 172.21.15.84:0/3055490309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c5415aaa0 msgr2=0x7f4c5415ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:10.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.956+0000 7f4c5b568640 1 --2- 172.21.15.84:0/3055490309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c5415aaa0 0x7f4c5415ce10 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f4c4002bee0 tx=0x7f4c400046c0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.956+0000 7f4c5b568640 1 -- 172.21.15.84:0/3055490309 shutdown_connections 2023-10-16T11:24:10.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.956+0000 7f4c5b568640 1 --2- 172.21.15.84:0/3055490309 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4c3806acc0 0x7f4c3806d180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.956+0000 7f4c5b568640 1 --2- 172.21.15.84:0/3055490309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c5415ae80 0x7f4c5414cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.956+0000 7f4c5b568640 1 --2- 172.21.15.84:0/3055490309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c5415aaa0 0x7f4c5415ce10 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.956+0000 7f4c5b568640 1 --2- 172.21.15.84:0/3055490309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c540ce390 0x7f4c540ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:10.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.956+0000 7f4c5b568640 1 -- 172.21.15.84:0/3055490309 >> 172.21.15.84:0/3055490309 conn(0x7f4c540cb410 msgr2=0x7f4c540b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:10.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.956+0000 7f4c5b568640 1 -- 172.21.15.84:0/3055490309 shutdown_connections 2023-10-16T11:24:10.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:10.956+0000 7f4c5b568640 1 -- 172.21.15.84:0/3055490309 wait complete. 2023-10-16T11:24:10.976 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:24:11.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.184+0000 7f1d4b3c9640 1 Processor -- start 2023-10-16T11:24:11.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.184+0000 7f1d4b3c9640 1 -- start start 2023-10-16T11:24:11.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.185+0000 7f1d4b3c9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1d440ce390 0x7f1d4415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:11.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.185+0000 7f1d4b3c9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4415ed60 con 0x7f1d4415aaa0 2023-10-16T11:24:11.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.185+0000 7f1d4b3c9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d440707e0 con 0x7f1d4415ae80 2023-10-16T11:24:11.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.185+0000 7f1d4b3c9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d4406b810 con 0x7f1d440ce390 2023-10-16T11:24:11.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.186+0000 7f1d49165640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1d4415aaa0 0x7f1d440cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48658/0 (socket says 172.21.15.84:48658) 2023-10-16T11:24:11.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.186+0000 7f1d49165640 1 -- 172.21.15.84:0/2830423626 learned_addr learned my addr 172.21.15.84:0/2830423626 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:11.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.186+0000 7f1d49966640 1 --2- 172.21.15.84:0/2830423626 >> v2:172.21.15.84:3300/0 conn(0x7f1d440ce390 0x7f1d4415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:11.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.186+0000 7f1d3bfff640 1 -- 172.21.15.84:0/2830423626 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 902185540 0 0) 0x7f1d4415ed60 con 0x7f1d4415aaa0 2023-10-16T11:24:11.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.186+0000 7f1d3bfff640 1 -- 172.21.15.84:0/2830423626 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d30003520 con 0x7f1d4415aaa0 2023-10-16T11:24:11.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.186+0000 7f1d49966640 1 -- 172.21.15.84:0/2830423626 >> v1:172.21.15.157:6789/0 conn(0x7f1d4415ae80 legacy=0x7f1d44070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:11.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.186+0000 7f1d49966640 1 -- 172.21.15.84:0/2830423626 >> v1:172.21.15.84:6789/0 conn(0x7f1d4415aaa0 legacy=0x7f1d440cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:11.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.186+0000 7f1d49966640 1 -- 172.21.15.84:0/2830423626 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4415ef90 con 0x7f1d440ce390 2023-10-16T11:24:11.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.187+0000 7f1d49966640 1 --2- 172.21.15.84:0/2830423626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d440ce390 0x7f1d4415e820 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009d90 tx=0x7f1d3c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e09b024e3f1a339 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:11.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.188+0000 7f1d3bfff640 1 -- 172.21.15.84:0/2830423626 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c02cc60 con 0x7f1d440ce390 2023-10-16T11:24:11.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.188+0000 7f1d3bfff640 1 -- 172.21.15.84:0/2830423626 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3c02cdc0 con 0x7f1d440ce390 2023-10-16T11:24:11.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.188+0000 7f1d3bfff640 1 -- 172.21.15.84:0/2830423626 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c034660 con 0x7f1d440ce390 2023-10-16T11:24:11.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.188+0000 7f1d4b3c9640 1 -- 172.21.15.84:0/2830423626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d440ce390 msgr2=0x7f1d4415e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:11.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.188+0000 7f1d4b3c9640 1 --2- 172.21.15.84:0/2830423626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d440ce390 0x7f1d4415e820 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009d90 tx=0x7f1d3c02beb0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.188+0000 7f1d4b3c9640 1 -- 172.21.15.84:0/2830423626 shutdown_connections 2023-10-16T11:24:11.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.188+0000 7f1d4b3c9640 1 --2- 172.21.15.84:0/2830423626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d440ce390 0x7f1d4415e820 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.188+0000 7f1d4b3c9640 1 -- 172.21.15.84:0/2830423626 >> 172.21.15.84:0/2830423626 conn(0x7f1d440cb410 msgr2=0x7f1d440cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:11.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.188+0000 7f1d4b3c9640 1 -- 172.21.15.84:0/2830423626 shutdown_connections 2023-10-16T11:24:11.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.189+0000 7f1d4b3c9640 1 -- 172.21.15.84:0/2830423626 wait complete. 2023-10-16T11:24:11.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.190+0000 7f1d4b3c9640 1 Processor -- start 2023-10-16T11:24:11.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.190+0000 7f1d4b3c9640 1 -- start start 2023-10-16T11:24:11.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.191+0000 7f1d4b3c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d440ce390 0x7f1d44157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:11.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.191+0000 7f1d4b3c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d4415aaa0 0x7f1d440cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:11.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.191+0000 7f1d4b3c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d4415ae80 0x7f1d440cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:11.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.191+0000 7f1d4b3c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d44157660 con 0x7f1d4415aaa0 2023-10-16T11:24:11.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.191+0000 7f1d4b3c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4415d7a0 con 0x7f1d4415ae80 2023-10-16T11:24:11.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.191+0000 7f1d4b3c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d441e85e0 con 0x7f1d440ce390 2023-10-16T11:24:11.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.192+0000 7f1d48964640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d4415aaa0 0x7f1d440cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:11.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.192+0000 7f1d48964640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d4415aaa0 0x7f1d440cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44854/0 (socket says 172.21.15.84:44854) 2023-10-16T11:24:11.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.192+0000 7f1d48964640 1 -- 172.21.15.84:0/726992568 learned_addr learned my addr 172.21.15.84:0/726992568 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:11.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.192+0000 7f1d49165640 1 --2- 172.21.15.84:0/726992568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d440ce390 0x7f1d44157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:11.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.192+0000 7f1d49966640 1 --2- 172.21.15.84:0/726992568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d4415ae80 0x7f1d440cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:11.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.192+0000 7f1d48964640 1 -- 172.21.15.84:0/726992568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d440ce390 msgr2=0x7f1d44157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:11.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.192+0000 7f1d48964640 1 --2- 172.21.15.84:0/726992568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d440ce390 0x7f1d44157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.192+0000 7f1d48964640 1 -- 172.21.15.84:0/726992568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d4415ae80 msgr2=0x7f1d440cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:11.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.192+0000 7f1d48964640 1 --2- 172.21.15.84:0/726992568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d4415ae80 0x7f1d440cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.192+0000 7f1d48964640 1 -- 172.21.15.84:0/726992568 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3c009a70 con 0x7f1d4415aaa0 2023-10-16T11:24:11.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.192+0000 7f1d49165640 1 --2- 172.21.15.84:0/726992568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d440ce390 0x7f1d44157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:11.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.192+0000 7f1d49966640 1 --2- 172.21.15.84:0/726992568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d4415ae80 0x7f1d440cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:11.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.192+0000 7f1d48964640 1 --2- 172.21.15.84:0/726992568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d4415aaa0 0x7f1d440cc4b0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400b3c0 tx=0x7f1d3400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:11.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.193+0000 7f1d39ffb640 1 -- 172.21.15.84:0/726992568 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34011830 con 0x7f1d4415aaa0 2023-10-16T11:24:11.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.193+0000 7f1d39ffb640 1 -- 172.21.15.84:0/726992568 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d34011990 con 0x7f1d4415aaa0 2023-10-16T11:24:11.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.193+0000 7f1d39ffb640 1 -- 172.21.15.84:0/726992568 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34011d80 con 0x7f1d4415aaa0 2023-10-16T11:24:11.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.193+0000 7f1d4b3c9640 1 -- 172.21.15.84:0/726992568 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4415d4c0 con 0x7f1d4415aaa0 2023-10-16T11:24:11.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.193+0000 7f1d4b3c9640 1 -- 172.21.15.84:0/726992568 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d441e8b60 con 0x7f1d4415aaa0 2023-10-16T11:24:11.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.194+0000 7f1d4b3c9640 1 -- 172.21.15.84:0/726992568 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d4404e160 con 0x7f1d4415aaa0 2023-10-16T11:24:11.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.195+0000 7f1d39ffb640 1 -- 172.21.15.84:0/726992568 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 131) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f1d3401b460 con 0x7f1d4415aaa0 2023-10-16T11:24:11.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.196+0000 7f1d39ffb640 1 --2- 172.21.15.84:0/726992568 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f1d2806af30 0x7f1d2806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:11.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.196+0000 7f1d39ffb640 1 -- 172.21.15.84:0/726992568 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f1d340895d0 con 0x7f1d4415aaa0 2023-10-16T11:24:11.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.196+0000 7f1d49165640 1 --2- 172.21.15.84:0/726992568 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f1d2806af30 0x7f1d2806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:11.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.197+0000 7f1d49165640 1 --2- 172.21.15.84:0/726992568 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f1d2806af30 0x7f1d2806d3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1d40005880 tx=0x7f1d40009040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:11.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.201+0000 7f1d39ffb640 1 -- 172.21.15.84:0/726992568 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1d340542a0 con 0x7f1d4415aaa0 2023-10-16T11:24:11.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.393+0000 7f1d4b3c9640 1 -- 172.21.15.84:0/726992568 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d440cdc60 con 0x7f1d4415aaa0 2023-10-16T11:24:11.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.395+0000 7f1d39ffb640 1 -- 172.21.15.84:0/726992568 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v206) v1 ==== 74+0+17829 (secure 0 0 0) 0x7f1d340578c0 con 0x7f1d4415aaa0 2023-10-16T11:24:11.398 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:11.398 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":206,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:24:08.217312+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:22:40.385524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-16T11:22:41.592928+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":205,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.18","mappings":[{"from":0,"to":2}]},{"pgid":"25.1f","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/65166914":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/26641613":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/1473149491":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/3116175047":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/4203014950":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/1585465362":"2023-10-17T11:24:08.217257+0000","172.21.15.84:6832/855077515":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/3564300528":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/2349522146":"2023-10-17T11:22:56.900947+0000","172.21.15.84:6833/855077515":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/2191803332":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/2199276620":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/3418988137":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/4262224959":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/1032287219":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/956990500":"2023-10-17T11:22:56.900947+0000","172.21.15.84:6833/1985463812":"2023-10-17T11:22:56.900947+0000","172.21.15.84:6832/1985463812":"2023-10-17T11:22:56.900947+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:24:11.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.398+0000 7f1d4b3c9640 1 -- 172.21.15.84:0/726992568 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f1d2806af30 msgr2=0x7f1d2806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:11.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.398+0000 7f1d4b3c9640 1 --2- 172.21.15.84:0/726992568 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f1d2806af30 0x7f1d2806d3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1d40005880 tx=0x7f1d40009040 comp rx=0 tx=0).stop 2023-10-16T11:24:11.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.398+0000 7f1d4b3c9640 1 -- 172.21.15.84:0/726992568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d4415aaa0 msgr2=0x7f1d440cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:11.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.398+0000 7f1d4b3c9640 1 --2- 172.21.15.84:0/726992568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d4415aaa0 0x7f1d440cc4b0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400b3c0 tx=0x7f1d3400b880 comp rx=0 tx=0).stop 2023-10-16T11:24:11.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.398+0000 7f1d4b3c9640 1 -- 172.21.15.84:0/726992568 shutdown_connections 2023-10-16T11:24:11.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.398+0000 7f1d4b3c9640 1 --2- 172.21.15.84:0/726992568 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f1d2806af30 0x7f1d2806d3f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.398+0000 7f1d4b3c9640 1 --2- 172.21.15.84:0/726992568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d4415ae80 0x7f1d440cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.398+0000 7f1d4b3c9640 1 --2- 172.21.15.84:0/726992568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d4415aaa0 0x7f1d440cc4b0 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.398+0000 7f1d4b3c9640 1 --2- 172.21.15.84:0/726992568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d440ce390 0x7f1d44157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.398+0000 7f1d4b3c9640 1 -- 172.21.15.84:0/726992568 >> 172.21.15.84:0/726992568 conn(0x7f1d440cb410 msgr2=0x7f1d4415a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:11.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.399+0000 7f1d4b3c9640 1 -- 172.21.15.84:0/726992568 shutdown_connections 2023-10-16T11:24:11.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.399+0000 7f1d4b3c9640 1 -- 172.21.15.84:0/726992568 wait complete. 2023-10-16T11:24:11.419 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T11:24:11.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.619+0000 7fb50329f640 1 Processor -- start 2023-10-16T11:24:11.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.619+0000 7fb50329f640 1 -- start start 2023-10-16T11:24:11.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.621+0000 7fb50329f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4fc06c040 0x7fb4fc06c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:11.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.621+0000 7fb50329f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb4fc0cda80 0x7fb4fc15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:11.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.621+0000 7fb50329f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4fc06c960 con 0x7fb4fc0ad970 2023-10-16T11:24:11.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.621+0000 7fb50329f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4fc15b9d0 con 0x7fb4fc0cda80 2023-10-16T11:24:11.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.621+0000 7fb50329f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc15bcb0 con 0x7fb4fc06c040 2023-10-16T11:24:11.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.621+0000 7fb501a9c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb4fc0ad970 0x7fb4fc15e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48664/0 (socket says 172.21.15.84:48664) 2023-10-16T11:24:11.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.621+0000 7fb501a9c640 1 -- 172.21.15.84:0/4120989732 learned_addr learned my addr 172.21.15.84:0/4120989732 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:11.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.621+0000 7fb502a9e640 1 --2- 172.21.15.84:0/4120989732 >> v2:172.21.15.157:3300/0 conn(0x7fb4fc0cda80 0x7fb4fc15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:11.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.621+0000 7fb50229d640 1 --2- 172.21.15.84:0/4120989732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4fc06c040 0x7fb4fc06c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:11.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.622+0000 7fb50129b640 1 -- 172.21.15.84:0/4120989732 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4083257762 0 0) 0x7fb4fc06c960 con 0x7fb4fc0ad970 2023-10-16T11:24:11.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.622+0000 7fb50129b640 1 -- 172.21.15.84:0/4120989732 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4e8003520 con 0x7fb4fc0ad970 2023-10-16T11:24:11.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.622+0000 7fb50129b640 1 -- 172.21.15.84:0/4120989732 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1410044306 0 0) 0x7fb4e8003520 con 0x7fb4fc0ad970 2023-10-16T11:24:11.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.622+0000 7fb50129b640 1 -- 172.21.15.84:0/4120989732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4fc06c040 msgr2=0x7fb4fc06c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:11.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.622+0000 7fb50129b640 1 --2- 172.21.15.84:0/4120989732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4fc06c040 0x7fb4fc06c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.622+0000 7fb50129b640 1 -- 172.21.15.84:0/4120989732 >> v2:172.21.15.157:3300/0 conn(0x7fb4fc0cda80 msgr2=0x7fb4fc15ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:11.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.622+0000 7fb50129b640 1 --2- 172.21.15.84:0/4120989732 >> v2:172.21.15.157:3300/0 conn(0x7fb4fc0cda80 0x7fb4fc15ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.622+0000 7fb50129b640 1 -- 172.21.15.84:0/4120989732 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc15bf90 con 0x7fb4fc0ad970 2023-10-16T11:24:11.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.622+0000 7fb50129b640 1 -- 172.21.15.84:0/4120989732 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fb4ec001670 con 0x7fb4fc0ad970 2023-10-16T11:24:11.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.622+0000 7fb50129b640 1 -- 172.21.15.84:0/4120989732 >> v1:172.21.15.84:6789/0 conn(0x7fb4fc0ad970 legacy=0x7fb4fc15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:11.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.622+0000 7fb50129b640 1 --2- 172.21.15.84:0/4120989732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4e8003970 0x7fb4e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:11.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.622+0000 7fb50129b640 1 --2- 172.21.15.84:0/4120989732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4fc06c040 0x7fb4e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:11.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.622+0000 7fb50129b640 1 --2- 172.21.15.84:0/4120989732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4e8004d00 0x7fb4e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:11.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.622+0000 7fb50129b640 1 -- 172.21.15.84:0/4120989732 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc15b9d0 con 0x7fb4e8004d00 2023-10-16T11:24:11.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.622+0000 7fb50129b640 1 -- 172.21.15.84:0/4120989732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc15bcb0 con 0x7fb4e8003970 2023-10-16T11:24:11.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.622+0000 7fb50129b640 1 -- 172.21.15.84:0/4120989732 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e800b980 con 0x7fb4fc06c040 2023-10-16T11:24:11.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.623+0000 7fb502a9e640 1 --2- 172.21.15.84:0/4120989732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4e8004d00 0x7fb4e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:11.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.623+0000 7fb501a9c640 1 --2- 172.21.15.84:0/4120989732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4fc06c040 0x7fb4e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:11.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.623+0000 7fb50229d640 1 --2- 172.21.15.84:0/4120989732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4e8003970 0x7fb4e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:11.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.623+0000 7fb502a9e640 1 -- 172.21.15.84:0/4120989732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4fc06c040 msgr2=0x7fb4e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:11.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.623+0000 7fb502a9e640 1 --2- 172.21.15.84:0/4120989732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4fc06c040 0x7fb4e80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.623+0000 7fb502a9e640 1 -- 172.21.15.84:0/4120989732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4e8003970 msgr2=0x7fb4e8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:11.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.623+0000 7fb502a9e640 1 --2- 172.21.15.84:0/4120989732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4e8003970 0x7fb4e8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.623+0000 7fb502a9e640 1 -- 172.21.15.84:0/4120989732 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e8003520 con 0x7fb4e8004d00 2023-10-16T11:24:11.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.623+0000 7fb50229d640 1 --2- 172.21.15.84:0/4120989732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4e8003970 0x7fb4e8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:11.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.624+0000 7fb502a9e640 1 --2- 172.21.15.84:0/4120989732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4e8004d00 0x7fb4e8005180 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009fc0 tx=0x7fb4e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b85c492c876eaf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:11.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.624+0000 7fb50129b640 1 -- 172.21.15.84:0/4120989732 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f402cc60 con 0x7fb4e8004d00 2023-10-16T11:24:11.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.624+0000 7fb50129b640 1 -- 172.21.15.84:0/4120989732 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f400f040 con 0x7fb4e8004d00 2023-10-16T11:24:11.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.624+0000 7fb50129b640 1 -- 172.21.15.84:0/4120989732 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4034440 con 0x7fb4e8004d00 2023-10-16T11:24:11.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.625+0000 7fb50329f640 1 -- 172.21.15.84:0/4120989732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4e8004d00 msgr2=0x7fb4e8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:11.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.625+0000 7fb50329f640 1 --2- 172.21.15.84:0/4120989732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4e8004d00 0x7fb4e8005180 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009fc0 tx=0x7fb4e80057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.625+0000 7fb502a9e640 1 -- 172.21.15.84:0/4120989732 reap_dead start 2023-10-16T11:24:11.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.625+0000 7fb50329f640 1 -- 172.21.15.84:0/4120989732 shutdown_connections 2023-10-16T11:24:11.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.625+0000 7fb50329f640 1 -- 172.21.15.84:0/4120989732 >> 172.21.15.84:0/4120989732 conn(0x7fb4fc0ce6a0 msgr2=0x7fb4fc0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:11.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.625+0000 7fb50329f640 1 -- 172.21.15.84:0/4120989732 shutdown_connections 2023-10-16T11:24:11.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.625+0000 7fb50329f640 1 -- 172.21.15.84:0/4120989732 wait complete. 2023-10-16T11:24:11.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.627+0000 7fb50329f640 1 Processor -- start 2023-10-16T11:24:11.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.627+0000 7fb50329f640 1 -- start start 2023-10-16T11:24:11.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.628+0000 7fb50329f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4e8004d00 0x7fb4fc1f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:11.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.628+0000 7fb50329f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4fc144700 0x7fb4fc0b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:11.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.628+0000 7fb50329f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4fc0b0bf0 0x7fb4fc151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:11.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.628+0000 7fb50329f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0b1070 con 0x7fb4fc144700 2023-10-16T11:24:11.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.628+0000 7fb50329f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0b2b70 con 0x7fb4e8004d00 2023-10-16T11:24:11.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.628+0000 7fb50329f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc154c10 con 0x7fb4fc0b0bf0 2023-10-16T11:24:11.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.628+0000 7fb501a9c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4fc144700 0x7fb4fc0b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:11.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.628+0000 7fb501a9c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4fc144700 0x7fb4fc0b06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44866/0 (socket says 172.21.15.84:44866) 2023-10-16T11:24:11.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.629+0000 7fb501a9c640 1 -- 172.21.15.84:0/2841666120 learned_addr learned my addr 172.21.15.84:0/2841666120 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:11.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.629+0000 7fb50229d640 1 --2- 172.21.15.84:0/2841666120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4e8004d00 0x7fb4fc1f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:11.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.629+0000 7fb502a9e640 1 --2- 172.21.15.84:0/2841666120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4fc0b0bf0 0x7fb4fc151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:11.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.629+0000 7fb501a9c640 1 -- 172.21.15.84:0/2841666120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4fc0b0bf0 msgr2=0x7fb4fc151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:11.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.629+0000 7fb501a9c640 1 --2- 172.21.15.84:0/2841666120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4fc0b0bf0 0x7fb4fc151a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.629+0000 7fb501a9c640 1 -- 172.21.15.84:0/2841666120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4e8004d00 msgr2=0x7fb4fc1f42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:11.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.629+0000 7fb501a9c640 1 --2- 172.21.15.84:0/2841666120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4e8004d00 0x7fb4fc1f42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.629+0000 7fb501a9c640 1 -- 172.21.15.84:0/2841666120 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc0b2240 con 0x7fb4fc144700 2023-10-16T11:24:11.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.629+0000 7fb501a9c640 1 --2- 172.21.15.84:0/2841666120 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4fc144700 0x7fb4fc0b06b0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00e840 tx=0x7fb4ec00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:11.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.629+0000 7fb4f2ffd640 1 -- 172.21.15.84:0/2841666120 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec00c8d0 con 0x7fb4fc144700 2023-10-16T11:24:11.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.630+0000 7fb4f2ffd640 1 -- 172.21.15.84:0/2841666120 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec00ca50 con 0x7fb4fc144700 2023-10-16T11:24:11.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.630+0000 7fb4f2ffd640 1 -- 172.21.15.84:0/2841666120 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec00ce00 con 0x7fb4fc144700 2023-10-16T11:24:11.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.630+0000 7fb50329f640 1 -- 172.21.15.84:0/2841666120 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4fc0b2520 con 0x7fb4fc144700 2023-10-16T11:24:11.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.630+0000 7fb50329f640 1 -- 172.21.15.84:0/2841666120 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4fc0c4f90 con 0x7fb4fc144700 2023-10-16T11:24:11.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.631+0000 7fb50329f640 1 -- 172.21.15.84:0/2841666120 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4fc0b2d20 con 0x7fb4fc144700 2023-10-16T11:24:11.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.632+0000 7fb4f2ffd640 1 -- 172.21.15.84:0/2841666120 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 131) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fb4ec01d460 con 0x7fb4fc144700 2023-10-16T11:24:11.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.633+0000 7fb4f2ffd640 1 --2- 172.21.15.84:0/2841666120 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fb4e006af00 0x7fb4e006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:11.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.633+0000 7fb4f2ffd640 1 -- 172.21.15.84:0/2841666120 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fb4ec089a30 con 0x7fb4fc144700 2023-10-16T11:24:11.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.633+0000 7fb50229d640 1 --2- 172.21.15.84:0/2841666120 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fb4e006af00 0x7fb4e006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:11.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.633+0000 7fb50229d640 1 --2- 172.21.15.84:0/2841666120 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fb4e006af00 0x7fb4e006d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009aa0 tx=0x7fb4f8008040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:11.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.637+0000 7fb4f2ffd640 1 -- 172.21.15.84:0/2841666120 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb4fc0b2d20 con 0x7fb4fc144700 2023-10-16T11:24:11.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.850+0000 7fb50329f640 1 -- 172.21.15.84:0/2841666120 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb4fc0b2ce0 con 0x7fb4fc144700 2023-10-16T11:24:11.854 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-16T11:24:11.852+0000 7fca0c7ff640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:24:11.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.856+0000 7fb4f2ffd640 1 -- 172.21.15.84:0/2841666120 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb4ec054770 con 0x7fb4fc144700 2023-10-16T11:24:11.858 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:24:11.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.859+0000 7fb50329f640 1 -- 172.21.15.84:0/2841666120 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fb4e006af00 msgr2=0x7fb4e006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:11.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.859+0000 7fb50329f640 1 --2- 172.21.15.84:0/2841666120 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fb4e006af00 0x7fb4e006d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009aa0 tx=0x7fb4f8008040 comp rx=0 tx=0).stop 2023-10-16T11:24:11.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.859+0000 7fb50329f640 1 -- 172.21.15.84:0/2841666120 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4fc144700 msgr2=0x7fb4fc0b06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:11.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.859+0000 7fb50329f640 1 --2- 172.21.15.84:0/2841666120 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4fc144700 0x7fb4fc0b06b0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00e840 tx=0x7fb4ec00ed00 comp rx=0 tx=0).stop 2023-10-16T11:24:11.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.859+0000 7fb50329f640 1 -- 172.21.15.84:0/2841666120 shutdown_connections 2023-10-16T11:24:11.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.859+0000 7fb50329f640 1 --2- 172.21.15.84:0/2841666120 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fb4e006af00 0x7fb4e006d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.859+0000 7fb50329f640 1 --2- 172.21.15.84:0/2841666120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4fc0b0bf0 0x7fb4fc151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.859+0000 7fb50329f640 1 --2- 172.21.15.84:0/2841666120 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4fc144700 0x7fb4fc0b06b0 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.859+0000 7fb50329f640 1 --2- 172.21.15.84:0/2841666120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4e8004d00 0x7fb4fc1f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:11.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.859+0000 7fb50329f640 1 -- 172.21.15.84:0/2841666120 >> 172.21.15.84:0/2841666120 conn(0x7fb4fc0ce6a0 msgr2=0x7fb4fc0c5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:11.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.859+0000 7fb50329f640 1 -- 172.21.15.84:0/2841666120 shutdown_connections 2023-10-16T11:24:11.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:11.859+0000 7fb50329f640 1 -- 172.21.15.84:0/2841666120 wait complete. 2023-10-16T11:24:11.880 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T11:24:11.973 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T11:24:12.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.075+0000 7ffa671a5640 1 Processor -- start 2023-10-16T11:24:12.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.076+0000 7ffa671a5640 1 -- start start 2023-10-16T11:24:12.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.077+0000 7ffa671a5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa6015c9f0 0x7ffa600c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:12.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.077+0000 7ffa671a5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa6015dde0 0x7ffa600ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:12.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.077+0000 7ffa671a5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa60160a90 con 0x7ffa600c9230 2023-10-16T11:24:12.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.077+0000 7ffa671a5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa600cdec0 con 0x7ffa6015c9f0 2023-10-16T11:24:12.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.077+0000 7ffa671a5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa600cd900 con 0x7ffa6015dde0 2023-10-16T11:24:12.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.078+0000 7ffa661a3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa6015c9f0 0x7ffa600c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:12.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.078+0000 7ffa661a3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa6015c9f0 0x7ffa600c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:45046/0 (socket says 172.21.15.84:45046) 2023-10-16T11:24:12.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.078+0000 7ffa661a3640 1 -- 172.21.15.84:0/3145224384 learned_addr learned my addr 172.21.15.84:0/3145224384 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:12.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.078+0000 7ffa669a4640 1 --2- 172.21.15.84:0/3145224384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa6015dde0 0x7ffa600ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:12.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.078+0000 7ffa661a3640 1 -- 172.21.15.84:0/3145224384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa6015dde0 msgr2=0x7ffa600ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:12.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.078+0000 7ffa661a3640 1 --2- 172.21.15.84:0/3145224384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa6015dde0 0x7ffa600ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:12.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.078+0000 7ffa661a3640 1 -- 172.21.15.84:0/3145224384 >> v1:172.21.15.157:6789/0 conn(0x7ffa600c9230 legacy=0x7ffa600709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:12.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.078+0000 7ffa661a3640 1 -- 172.21.15.84:0/3145224384 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa600cdbe0 con 0x7ffa6015c9f0 2023-10-16T11:24:12.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.080+0000 7ffa661a3640 1 --2- 172.21.15.84:0/3145224384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa6015c9f0 0x7ffa600c8cf0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009870 tx=0x7ffa5c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55534f864bc82e80 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:12.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.080+0000 7ffa651a1640 1 -- 172.21.15.84:0/3145224384 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c030840 con 0x7ffa6015c9f0 2023-10-16T11:24:12.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.080+0000 7ffa651a1640 1 -- 172.21.15.84:0/3145224384 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5c0309a0 con 0x7ffa6015c9f0 2023-10-16T11:24:12.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.080+0000 7ffa651a1640 1 -- 172.21.15.84:0/3145224384 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c030d30 con 0x7ffa6015c9f0 2023-10-16T11:24:12.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.081+0000 7ffa671a5640 1 -- 172.21.15.84:0/3145224384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa6015c9f0 msgr2=0x7ffa600c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:12.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.081+0000 7ffa671a5640 1 --2- 172.21.15.84:0/3145224384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa6015c9f0 0x7ffa600c8cf0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009870 tx=0x7ffa5c02b980 comp rx=0 tx=0).stop 2023-10-16T11:24:12.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.081+0000 7ffa671a5640 1 -- 172.21.15.84:0/3145224384 shutdown_connections 2023-10-16T11:24:12.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.081+0000 7ffa671a5640 1 --2- 172.21.15.84:0/3145224384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa6015dde0 0x7ffa600ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:12.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.081+0000 7ffa671a5640 1 --2- 172.21.15.84:0/3145224384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa6015c9f0 0x7ffa600c8cf0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:12.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.081+0000 7ffa671a5640 1 -- 172.21.15.84:0/3145224384 >> 172.21.15.84:0/3145224384 conn(0x7ffa600ce6a0 msgr2=0x7ffa600ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:12.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.081+0000 7ffa671a5640 1 -- 172.21.15.84:0/3145224384 shutdown_connections 2023-10-16T11:24:12.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.081+0000 7ffa671a5640 1 -- 172.21.15.84:0/3145224384 wait complete. 2023-10-16T11:24:12.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.083+0000 7ffa671a5640 1 Processor -- start 2023-10-16T11:24:12.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.083+0000 7ffa671a5640 1 -- start start 2023-10-16T11:24:12.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.084+0000 7ffa671a5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa600c9230 0x7ffa601504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:12.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.084+0000 7ffa671a5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa6015c9f0 0x7ffa60069da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:12.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.084+0000 7ffa671a5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa6015dde0 0x7ffa601f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:12.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.084+0000 7ffa671a5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa60150a20 con 0x7ffa6015dde0 2023-10-16T11:24:12.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.084+0000 7ffa671a5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa600c2fe0 con 0x7ffa600c9230 2023-10-16T11:24:12.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.084+0000 7ffa671a5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa600c32c0 con 0x7ffa6015c9f0 2023-10-16T11:24:12.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.084+0000 7ffa669a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa6015dde0 0x7ffa601f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:12.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.084+0000 7ffa669a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa6015dde0 0x7ffa601f5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44908/0 (socket says 172.21.15.84:44908) 2023-10-16T11:24:12.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.084+0000 7ffa669a4640 1 -- 172.21.15.84:0/3728753959 learned_addr learned my addr 172.21.15.84:0/3728753959 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:12.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.084+0000 7ffa661a3640 1 --2- 172.21.15.84:0/3728753959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa600c9230 0x7ffa601504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:12.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.084+0000 7ffa659a2640 1 --2- 172.21.15.84:0/3728753959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa6015c9f0 0x7ffa60069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:12.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.084+0000 7ffa669a4640 1 -- 172.21.15.84:0/3728753959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa6015c9f0 msgr2=0x7ffa60069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:12.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.084+0000 7ffa669a4640 1 --2- 172.21.15.84:0/3728753959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa6015c9f0 0x7ffa60069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:12.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.084+0000 7ffa669a4640 1 -- 172.21.15.84:0/3728753959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa600c9230 msgr2=0x7ffa601504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:12.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.084+0000 7ffa669a4640 1 --2- 172.21.15.84:0/3728753959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa600c9230 0x7ffa601504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:12.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.084+0000 7ffa669a4640 1 -- 172.21.15.84:0/3728753959 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5c0094d0 con 0x7ffa6015dde0 2023-10-16T11:24:12.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.084+0000 7ffa661a3640 1 --2- 172.21.15.84:0/3728753959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa600c9230 0x7ffa601504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:12.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.085+0000 7ffa669a4640 1 --2- 172.21.15.84:0/3728753959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa6015dde0 0x7ffa601f5cf0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800efc0 tx=0x7ffa5800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:12.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.085+0000 7ffa56ffd640 1 -- 172.21.15.84:0/3728753959 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58009b20 con 0x7ffa6015dde0 2023-10-16T11:24:12.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.085+0000 7ffa56ffd640 1 -- 172.21.15.84:0/3728753959 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5800f040 con 0x7ffa6015dde0 2023-10-16T11:24:12.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.085+0000 7ffa56ffd640 1 -- 172.21.15.84:0/3728753959 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa580146c0 con 0x7ffa6015dde0 2023-10-16T11:24:12.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.085+0000 7ffa671a5640 1 -- 172.21.15.84:0/3728753959 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa600c35a0 con 0x7ffa6015dde0 2023-10-16T11:24:12.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.085+0000 7ffa671a5640 1 -- 172.21.15.84:0/3728753959 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa600c3b60 con 0x7ffa6015dde0 2023-10-16T11:24:12.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.086+0000 7ffa671a5640 1 -- 172.21.15.84:0/3728753959 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa60161670 con 0x7ffa6015dde0 2023-10-16T11:24:12.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.087+0000 7ffa56ffd640 1 -- 172.21.15.84:0/3728753959 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 131) v1 ==== 89275+0+0 (secure 0 0 0) 0x7ffa58014820 con 0x7ffa6015dde0 2023-10-16T11:24:12.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.088+0000 7ffa56ffd640 1 --2- 172.21.15.84:0/3728753959 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ffa4406af00 0x7ffa4406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:12.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.088+0000 7ffa56ffd640 1 -- 172.21.15.84:0/3728753959 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6683+0+0 (secure 0 0 0) 0x7ffa5808a8c0 con 0x7ffa6015dde0 2023-10-16T11:24:12.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.088+0000 7ffa661a3640 1 --2- 172.21.15.84:0/3728753959 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ffa4406af00 0x7ffa4406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:12.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.089+0000 7ffa661a3640 1 --2- 172.21.15.84:0/3728753959 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ffa4406af00 0x7ffa4406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c036000 tx=0x7ffa5c03b040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:12.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.092+0000 7ffa56ffd640 1 -- 172.21.15.84:0/3728753959 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffa580554a0 con 0x7ffa6015dde0 2023-10-16T11:24:12.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.306+0000 7ffa671a5640 1 -- 172.21.15.84:0/3728753959 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ffa600c3d80 con 0x7ffa6015dde0 2023-10-16T11:24:12.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.945+0000 7ffa56ffd640 1 -- 172.21.15.84:0/3728753959 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v107) v1 ==== 104+0+0 (secure 0 0 0) 0x7ffa58009e60 con 0x7ffa6015dde0 2023-10-16T11:24:12.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.948+0000 7ffa671a5640 1 -- 172.21.15.84:0/3728753959 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ffa4406af00 msgr2=0x7ffa4406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:12.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.948+0000 7ffa671a5640 1 --2- 172.21.15.84:0/3728753959 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ffa4406af00 0x7ffa4406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c036000 tx=0x7ffa5c03b040 comp rx=0 tx=0).stop 2023-10-16T11:24:12.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.948+0000 7ffa671a5640 1 -- 172.21.15.84:0/3728753959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa6015dde0 msgr2=0x7ffa601f5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:12.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.948+0000 7ffa671a5640 1 --2- 172.21.15.84:0/3728753959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa6015dde0 0x7ffa601f5cf0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800efc0 tx=0x7ffa5800c490 comp rx=0 tx=0).stop 2023-10-16T11:24:12.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.948+0000 7ffa671a5640 1 -- 172.21.15.84:0/3728753959 shutdown_connections 2023-10-16T11:24:12.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.948+0000 7ffa671a5640 1 --2- 172.21.15.84:0/3728753959 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ffa4406af00 0x7ffa4406d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:12.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.948+0000 7ffa671a5640 1 --2- 172.21.15.84:0/3728753959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa6015dde0 0x7ffa601f5cf0 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:12.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.948+0000 7ffa671a5640 1 --2- 172.21.15.84:0/3728753959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa6015c9f0 0x7ffa60069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:12.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.948+0000 7ffa671a5640 1 --2- 172.21.15.84:0/3728753959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa600c9230 0x7ffa601504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:12.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.948+0000 7ffa671a5640 1 -- 172.21.15.84:0/3728753959 >> 172.21.15.84:0/3728753959 conn(0x7ffa600ce6a0 msgr2=0x7ffa60160ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:12.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.949+0000 7ffa671a5640 1 -- 172.21.15.84:0/3728753959 shutdown_connections 2023-10-16T11:24:12.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:12.949+0000 7ffa671a5640 1 -- 172.21.15.84:0/3728753959 wait complete. 2023-10-16T11:24:12.974 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T11:24:13.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.180+0000 7fd8b30bb640 1 Processor -- start 2023-10-16T11:24:13.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.180+0000 7fd8b30bb640 1 -- start start 2023-10-16T11:24:13.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b30bb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd8ac159950 0x7fd8ac15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:13.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b30bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ac070450 0x7fd8ac0708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:13.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b30bb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ac070df0 con 0x7fd8ac06c100 2023-10-16T11:24:13.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b30bb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8ac15e490 con 0x7fd8ac159950 2023-10-16T11:24:13.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b30bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac15aef0 con 0x7fd8ac070450 2023-10-16T11:24:13.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b28ba640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd8ac06c100 0x7fd8ac06c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48666/0 (socket says 172.21.15.84:48666) 2023-10-16T11:24:13.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b28ba640 1 -- 172.21.15.84:0/1080184199 learned_addr learned my addr 172.21.15.84:0/1080184199 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:13.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b20b9640 1 --2- 172.21.15.84:0/1080184199 >> v2:172.21.15.157:3300/0 conn(0x7fd8ac159950 0x7fd8ac15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:13.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b10b7640 1 -- 172.21.15.84:0/1080184199 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 97697650 0 0) 0x7fd8ac070df0 con 0x7fd8ac06c100 2023-10-16T11:24:13.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b10b7640 1 -- 172.21.15.84:0/1080184199 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd898003520 con 0x7fd8ac06c100 2023-10-16T11:24:13.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b18b8640 1 --2- 172.21.15.84:0/1080184199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ac070450 0x7fd8ac0708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:13.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b10b7640 1 -- 172.21.15.84:0/1080184199 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1311719901 0 0) 0x7fd898003520 con 0x7fd8ac06c100 2023-10-16T11:24:13.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b10b7640 1 -- 172.21.15.84:0/1080184199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ac070450 msgr2=0x7fd8ac0708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:13.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b10b7640 1 --2- 172.21.15.84:0/1080184199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ac070450 0x7fd8ac0708b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:13.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b10b7640 1 -- 172.21.15.84:0/1080184199 >> v2:172.21.15.157:3300/0 conn(0x7fd8ac159950 msgr2=0x7fd8ac15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:13.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b10b7640 1 --2- 172.21.15.84:0/1080184199 >> v2:172.21.15.157:3300/0 conn(0x7fd8ac159950 0x7fd8ac15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:13.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b10b7640 1 -- 172.21.15.84:0/1080184199 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac06cc80 con 0x7fd8ac06c100 2023-10-16T11:24:13.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b10b7640 1 -- 172.21.15.84:0/1080184199 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd8a40019f0 con 0x7fd8ac06c100 2023-10-16T11:24:13.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b10b7640 1 -- 172.21.15.84:0/1080184199 >> v1:172.21.15.84:6789/0 conn(0x7fd8ac06c100 legacy=0x7fd8ac06c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:13.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b10b7640 1 --2- 172.21.15.84:0/1080184199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd898003970 0x7fd898003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:13.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b10b7640 1 --2- 172.21.15.84:0/1080184199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8980047c0 0x7fd898004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:13.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b10b7640 1 --2- 172.21.15.84:0/1080184199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ac070450 0x7fd898005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:13.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b10b7640 1 -- 172.21.15.84:0/1080184199 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac15e490 con 0x7fd898003970 2023-10-16T11:24:13.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b10b7640 1 -- 172.21.15.84:0/1080184199 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac15aef0 con 0x7fd8980047c0 2023-10-16T11:24:13.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b10b7640 1 -- 172.21.15.84:0/1080184199 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89800dd20 con 0x7fd8ac070450 2023-10-16T11:24:13.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b20b9640 1 --2- 172.21.15.84:0/1080184199 >> v2:172.21.15.157:3300/0 conn(0x7fd8ac159950 0x7fd8ac15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:24:13.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.183+0000 7fd8b20b9640 1 --2- 172.21.15.84:0/1080184199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd898003970 0x7fd898003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:13.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.184+0000 7fd8b18b8640 1 --2- 172.21.15.84:0/1080184199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8980047c0 0x7fd898004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:13.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.184+0000 7fd8b28ba640 1 --2- 172.21.15.84:0/1080184199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ac070450 0x7fd898005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:13.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.184+0000 7fd8b20b9640 1 -- 172.21.15.84:0/1080184199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ac070450 msgr2=0x7fd898005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:13.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.184+0000 7fd8b20b9640 1 --2- 172.21.15.84:0/1080184199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ac070450 0x7fd898005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:13.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.184+0000 7fd8b20b9640 1 -- 172.21.15.84:0/1080184199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8980047c0 msgr2=0x7fd898004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:13.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.184+0000 7fd8b20b9640 1 --2- 172.21.15.84:0/1080184199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8980047c0 0x7fd898004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:13.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.184+0000 7fd8b20b9640 1 -- 172.21.15.84:0/1080184199 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd898003520 con 0x7fd898003970 2023-10-16T11:24:13.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.184+0000 7fd8b18b8640 1 --2- 172.21.15.84:0/1080184199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8980047c0 0x7fd898004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:13.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.185+0000 7fd8b20b9640 1 --2- 172.21.15.84:0/1080184199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd898003970 0x7fd898003d90 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8006fd0 tx=0x7fd8980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9796963560673300 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:13.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.185+0000 7fd8b10b7640 1 -- 172.21.15.84:0/1080184199 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8030660 con 0x7fd898003970 2023-10-16T11:24:13.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.185+0000 7fd8b10b7640 1 -- 172.21.15.84:0/1080184199 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a80307c0 con 0x7fd898003970 2023-10-16T11:24:13.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.185+0000 7fd8b10b7640 1 -- 172.21.15.84:0/1080184199 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8030b30 con 0x7fd898003970 2023-10-16T11:24:13.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.186+0000 7fd8b30bb640 1 -- 172.21.15.84:0/1080184199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd898003970 msgr2=0x7fd898003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:13.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.186+0000 7fd8b30bb640 1 --2- 172.21.15.84:0/1080184199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd898003970 0x7fd898003d90 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8006fd0 tx=0x7fd8980057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:13.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.186+0000 7fd8b28ba640 1 -- 172.21.15.84:0/1080184199 reap_dead start 2023-10-16T11:24:13.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.186+0000 7fd8b30bb640 1 -- 172.21.15.84:0/1080184199 shutdown_connections 2023-10-16T11:24:13.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.186+0000 7fd8b30bb640 1 -- 172.21.15.84:0/1080184199 >> 172.21.15.84:0/1080184199 conn(0x7fd8ac0ce6a0 msgr2=0x7fd8ac0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:13.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.186+0000 7fd8b30bb640 1 -- 172.21.15.84:0/1080184199 shutdown_connections 2023-10-16T11:24:13.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.186+0000 7fd8b30bb640 1 -- 172.21.15.84:0/1080184199 wait complete. 2023-10-16T11:24:13.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.188+0000 7fd8b30bb640 1 Processor -- start 2023-10-16T11:24:13.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.188+0000 7fd8b30bb640 1 -- start start 2023-10-16T11:24:13.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.189+0000 7fd8b30bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd898003970 0x7fd8ac148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:13.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.189+0000 7fd8b30bb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8ac1568c0 0x7fd8ac14c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:13.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.189+0000 7fd8b30bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ac0c46c0 0x7fd8ac149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:13.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.189+0000 7fd8b30bb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac149500 con 0x7fd8ac1568c0 2023-10-16T11:24:13.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.189+0000 7fd8b30bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac14c720 con 0x7fd898003970 2023-10-16T11:24:13.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.189+0000 7fd8b30bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac0c3620 con 0x7fd8ac0c46c0 2023-10-16T11:24:13.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.189+0000 7fd8b18b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8ac1568c0 0x7fd8ac14c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:13.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.189+0000 7fd8b18b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8ac1568c0 0x7fd8ac14c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44916/0 (socket says 172.21.15.84:44916) 2023-10-16T11:24:13.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.189+0000 7fd8b18b8640 1 -- 172.21.15.84:0/390298619 learned_addr learned my addr 172.21.15.84:0/390298619 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:13.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.189+0000 7fd8b28ba640 1 --2- 172.21.15.84:0/390298619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ac0c46c0 0x7fd8ac149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:13.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.189+0000 7fd8b20b9640 1 --2- 172.21.15.84:0/390298619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd898003970 0x7fd8ac148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:13.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.189+0000 7fd8b18b8640 1 -- 172.21.15.84:0/390298619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ac0c46c0 msgr2=0x7fd8ac149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:13.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.189+0000 7fd8b18b8640 1 --2- 172.21.15.84:0/390298619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ac0c46c0 0x7fd8ac149770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:13.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.189+0000 7fd8b18b8640 1 -- 172.21.15.84:0/390298619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd898003970 msgr2=0x7fd8ac148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:13.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.189+0000 7fd8b18b8640 1 --2- 172.21.15.84:0/390298619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd898003970 0x7fd8ac148fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:13.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.189+0000 7fd8b18b8640 1 -- 172.21.15.84:0/390298619 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac0c3900 con 0x7fd8ac1568c0 2023-10-16T11:24:13.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.189+0000 7fd8b28ba640 1 --2- 172.21.15.84:0/390298619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ac0c46c0 0x7fd8ac149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:13.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.189+0000 7fd8b20b9640 1 --2- 172.21.15.84:0/390298619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd898003970 0x7fd8ac148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:13.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.190+0000 7fd8b18b8640 1 --2- 172.21.15.84:0/390298619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8ac1568c0 0x7fd8ac14c1e0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00e7e0 tx=0x7fd89c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:13.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.190+0000 7fd8a2ffd640 1 -- 172.21.15.84:0/390298619 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c00c6d0 con 0x7fd8ac1568c0 2023-10-16T11:24:13.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.190+0000 7fd8a2ffd640 1 -- 172.21.15.84:0/390298619 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89c00c830 con 0x7fd8ac1568c0 2023-10-16T11:24:13.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.190+0000 7fd8a2ffd640 1 -- 172.21.15.84:0/390298619 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c00cba0 con 0x7fd8ac1568c0 2023-10-16T11:24:13.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.190+0000 7fd8b30bb640 1 -- 172.21.15.84:0/390298619 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ac0c3be0 con 0x7fd8ac1568c0 2023-10-16T11:24:13.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.190+0000 7fd8b30bb640 1 -- 172.21.15.84:0/390298619 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ac0c41a0 con 0x7fd8ac1568c0 2023-10-16T11:24:13.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.191+0000 7fd8b30bb640 1 -- 172.21.15.84:0/390298619 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ac0c4360 con 0x7fd8ac1568c0 2023-10-16T11:24:13.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.192+0000 7fd8a2ffd640 1 -- 172.21.15.84:0/390298619 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd89c00cd00 con 0x7fd8ac1568c0 2023-10-16T11:24:13.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.193+0000 7fd8a2ffd640 1 --2- 172.21.15.84:0/390298619 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fd89006af90 0x7fd89006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:13.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.193+0000 7fd8a2ffd640 1 -- 172.21.15.84:0/390298619 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6683+0+0 (secure 0 0 0) 0x7fd89c089a20 con 0x7fd8ac1568c0 2023-10-16T11:24:13.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.193+0000 7fd8b20b9640 1 --2- 172.21.15.84:0/390298619 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fd89006af90 0x7fd89006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:13.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.194+0000 7fd8b20b9640 1 --2- 172.21.15.84:0/390298619 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fd89006af90 0x7fd89006d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8036000 tx=0x7fd8a8031000 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:13.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.197+0000 7fd8a2ffd640 1 -- 172.21.15.84:0/390298619 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd89c0544e0 con 0x7fd8ac1568c0 2023-10-16T11:24:13.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.389+0000 7fd8b30bb640 1 -- 172.21.15.84:0/390298619 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd8ac04e9b0 con 0x7fd8ac1568c0 2023-10-16T11:24:13.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.968+0000 7fd8a2ffd640 1 -- 172.21.15.84:0/390298619 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v208) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd89c057b00 con 0x7fd8ac1568c0 2023-10-16T11:24:13.971 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-16T11:24:13.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.972+0000 7fd8b30bb640 1 -- 172.21.15.84:0/390298619 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fd89006af90 msgr2=0x7fd89006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:13.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.972+0000 7fd8b30bb640 1 --2- 172.21.15.84:0/390298619 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fd89006af90 0x7fd89006d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8036000 tx=0x7fd8a8031000 comp rx=0 tx=0).stop 2023-10-16T11:24:13.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.972+0000 7fd8b30bb640 1 -- 172.21.15.84:0/390298619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8ac1568c0 msgr2=0x7fd8ac14c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:13.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.972+0000 7fd8b30bb640 1 --2- 172.21.15.84:0/390298619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8ac1568c0 0x7fd8ac14c1e0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00e7e0 tx=0x7fd89c00eca0 comp rx=0 tx=0).stop 2023-10-16T11:24:13.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.974+0000 7fd8b30bb640 1 -- 172.21.15.84:0/390298619 shutdown_connections 2023-10-16T11:24:13.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.974+0000 7fd8b30bb640 1 --2- 172.21.15.84:0/390298619 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fd89006af90 0x7fd89006d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:13.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.974+0000 7fd8b30bb640 1 --2- 172.21.15.84:0/390298619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ac0c46c0 0x7fd8ac149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:13.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.974+0000 7fd8b30bb640 1 --2- 172.21.15.84:0/390298619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8ac1568c0 0x7fd8ac14c1e0 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:13.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.974+0000 7fd8b30bb640 1 --2- 172.21.15.84:0/390298619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd898003970 0x7fd8ac148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:13.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.974+0000 7fd8b30bb640 1 -- 172.21.15.84:0/390298619 >> 172.21.15.84:0/390298619 conn(0x7fd8ac0ce6a0 msgr2=0x7fd8ac0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:13.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.975+0000 7fd8b30bb640 1 -- 172.21.15.84:0/390298619 shutdown_connections 2023-10-16T11:24:13.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:13.975+0000 7fd8b30bb640 1 -- 172.21.15.84:0/390298619 wait complete. 2023-10-16T11:24:14.000 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T11:24:14.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.205+0000 7fa39553a640 1 Processor -- start 2023-10-16T11:24:14.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.205+0000 7fa39553a640 1 -- start start 2023-10-16T11:24:14.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.206+0000 7fa39553a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa3901599c0 0x7fa3900c9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:14.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.206+0000 7fa39553a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa390159da0 0x7fa39015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:14.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.206+0000 7fa39553a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa3900cce20 0x7fa39006c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:14.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.206+0000 7fa39553a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39015dd10 con 0x7fa3901599c0 2023-10-16T11:24:14.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.206+0000 7fa39553a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39015dfb0 con 0x7fa3900cce20 2023-10-16T11:24:14.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.206+0000 7fa39553a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39015e290 con 0x7fa390159da0 2023-10-16T11:24:14.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.206+0000 7fa38effd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa3901599c0 0x7fa3900c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:14.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.207+0000 7fa38effd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa3901599c0 0x7fa3900c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44918/0 (socket says 172.21.15.84:44918) 2023-10-16T11:24:14.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.207+0000 7fa38effd640 1 -- 172.21.15.84:0/359055870 learned_addr learned my addr 172.21.15.84:0/359055870 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:14.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.207+0000 7fa38f7fe640 1 --2- 172.21.15.84:0/359055870 >> v2:172.21.15.157:3300/0 conn(0x7fa3900cce20 0x7fa39006c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:14.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.207+0000 7fa38e7fc640 1 --2- 172.21.15.84:0/359055870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa390159da0 0x7fa39015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:14.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.207+0000 7fa38effd640 1 -- 172.21.15.84:0/359055870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa390159da0 msgr2=0x7fa39015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:14.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.207+0000 7fa38effd640 1 --2- 172.21.15.84:0/359055870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa390159da0 0x7fa39015d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:14.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.207+0000 7fa38effd640 1 -- 172.21.15.84:0/359055870 >> v2:172.21.15.157:3300/0 conn(0x7fa3900cce20 msgr2=0x7fa39006c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:14.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.207+0000 7fa38effd640 1 --2- 172.21.15.84:0/359055870 >> v2:172.21.15.157:3300/0 conn(0x7fa3900cce20 0x7fa39006c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:14.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.207+0000 7fa38effd640 1 -- 172.21.15.84:0/359055870 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39015e570 con 0x7fa3901599c0 2023-10-16T11:24:14.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.207+0000 7fa38f7fe640 1 --2- 172.21.15.84:0/359055870 >> v2:172.21.15.157:3300/0 conn(0x7fa3900cce20 0x7fa39006c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:24:14.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.209+0000 7fa38effd640 1 --2- 172.21.15.84:0/359055870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3901599c0 0x7fa3900c9300 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7fa3900c9840 tx=0x7fa38002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b528bb6535ade2db server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:14.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.209+0000 7fa38dffb640 1 -- 172.21.15.84:0/359055870 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa380031870 con 0x7fa3901599c0 2023-10-16T11:24:14.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.209+0000 7fa38dffb640 1 -- 172.21.15.84:0/359055870 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3800319d0 con 0x7fa3901599c0 2023-10-16T11:24:14.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.209+0000 7fa38dffb640 1 -- 172.21.15.84:0/359055870 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa380031d60 con 0x7fa3901599c0 2023-10-16T11:24:14.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.210+0000 7fa39553a640 1 -- 172.21.15.84:0/359055870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3901599c0 msgr2=0x7fa3900c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:14.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.210+0000 7fa39553a640 1 --2- 172.21.15.84:0/359055870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3901599c0 0x7fa3900c9300 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7fa3900c9840 tx=0x7fa38002b910 comp rx=0 tx=0).stop 2023-10-16T11:24:14.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.210+0000 7fa39553a640 1 -- 172.21.15.84:0/359055870 shutdown_connections 2023-10-16T11:24:14.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.210+0000 7fa39553a640 1 --2- 172.21.15.84:0/359055870 >> v2:172.21.15.157:3300/0 conn(0x7fa3900cce20 0x7fa39006c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:14.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.210+0000 7fa39553a640 1 --2- 172.21.15.84:0/359055870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa390159da0 0x7fa39015d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:14.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.210+0000 7fa39553a640 1 --2- 172.21.15.84:0/359055870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3901599c0 0x7fa3900c9300 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:14.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.210+0000 7fa39553a640 1 -- 172.21.15.84:0/359055870 >> 172.21.15.84:0/359055870 conn(0x7fa3900cb360 msgr2=0x7fa3900cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:14.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.210+0000 7fa39553a640 1 -- 172.21.15.84:0/359055870 shutdown_connections 2023-10-16T11:24:14.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.210+0000 7fa39553a640 1 -- 172.21.15.84:0/359055870 wait complete. 2023-10-16T11:24:14.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.211+0000 7fa39553a640 1 Processor -- start 2023-10-16T11:24:14.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.212+0000 7fa39553a640 1 -- start start 2023-10-16T11:24:14.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.212+0000 7fa39553a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3900cce20 0x7fa390156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:14.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.212+0000 7fa39553a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3901599c0 0x7fa39014fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:14.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.212+0000 7fa39553a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa390159da0 0x7fa3900c3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:14.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa39553a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900ae0b0 con 0x7fa3900cce20 2023-10-16T11:24:14.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa39553a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa390151980 con 0x7fa3901599c0 2023-10-16T11:24:14.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa39553a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900ae850 con 0x7fa390159da0 2023-10-16T11:24:14.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa38effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3900cce20 0x7fa390156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:14.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa38effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3900cce20 0x7fa390156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44930/0 (socket says 172.21.15.84:44930) 2023-10-16T11:24:14.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa38effd640 1 -- 172.21.15.84:0/1629711293 learned_addr learned my addr 172.21.15.84:0/1629711293 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:14.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa38e7fc640 1 --2- 172.21.15.84:0/1629711293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3901599c0 0x7fa39014fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:14.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa38f7fe640 1 --2- 172.21.15.84:0/1629711293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa390159da0 0x7fa3900c3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:14.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa38effd640 1 -- 172.21.15.84:0/1629711293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa390159da0 msgr2=0x7fa3900c3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:14.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa38effd640 1 --2- 172.21.15.84:0/1629711293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa390159da0 0x7fa3900c3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:14.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa38effd640 1 -- 172.21.15.84:0/1629711293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3901599c0 msgr2=0x7fa39014fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:14.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa38effd640 1 --2- 172.21.15.84:0/1629711293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3901599c0 0x7fa39014fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:14.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa38effd640 1 -- 172.21.15.84:0/1629711293 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3800094d0 con 0x7fa3900cce20 2023-10-16T11:24:14.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa38effd640 1 --2- 172.21.15.84:0/1629711293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3900cce20 0x7fa390156e40 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fa380000c00 tx=0x7fa3800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:14.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa3777fe640 1 -- 172.21.15.84:0/1629711293 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38003ad30 con 0x7fa3900cce20 2023-10-16T11:24:14.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa3777fe640 1 -- 172.21.15.84:0/1629711293 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3800443f0 con 0x7fa3900cce20 2023-10-16T11:24:14.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa39553a640 1 -- 172.21.15.84:0/1629711293 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3900ae290 con 0x7fa3900cce20 2023-10-16T11:24:14.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa3777fe640 1 -- 172.21.15.84:0/1629711293 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3800447a0 con 0x7fa3900cce20 2023-10-16T11:24:14.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.214+0000 7fa39553a640 1 -- 172.21.15.84:0/1629711293 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3900c2f50 con 0x7fa3900cce20 2023-10-16T11:24:14.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.215+0000 7fa39553a640 1 -- 172.21.15.84:0/1629711293 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa39004e160 con 0x7fa3900cce20 2023-10-16T11:24:14.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.216+0000 7fa3777fe640 1 -- 172.21.15.84:0/1629711293 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa380044900 con 0x7fa3900cce20 2023-10-16T11:24:14.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.218+0000 7fa3777fe640 1 --2- 172.21.15.84:0/1629711293 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa37c06af90 0x7fa37c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:14.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.218+0000 7fa3777fe640 1 -- 172.21.15.84:0/1629711293 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fa38002d040 con 0x7fa3900cce20 2023-10-16T11:24:14.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.218+0000 7fa38e7fc640 1 --2- 172.21.15.84:0/1629711293 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa37c06af90 0x7fa37c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:14.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.219+0000 7fa38e7fc640 1 --2- 172.21.15.84:0/1629711293 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa37c06af90 0x7fa37c06d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa3900b4370 tx=0x7fa384008040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:14.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.222+0000 7fa3777fe640 1 -- 172.21.15.84:0/1629711293 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa380076d60 con 0x7fa3900cce20 2023-10-16T11:24:14.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.419+0000 7fa39553a640 1 -- 172.21.15.84:0/1629711293 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa39015d2c0 con 0x7fa3900cce20 2023-10-16T11:24:14.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.821+0000 7fa3777fe640 1 -- 172.21.15.84:0/1629711293 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v209) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa3800350d0 con 0x7fa3900cce20 2023-10-16T11:24:14.822 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-16T11:24:14.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.825+0000 7fa39553a640 1 -- 172.21.15.84:0/1629711293 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa37c06af90 msgr2=0x7fa37c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:14.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.825+0000 7fa39553a640 1 --2- 172.21.15.84:0/1629711293 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa37c06af90 0x7fa37c06d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa3900b4370 tx=0x7fa384008040 comp rx=0 tx=0).stop 2023-10-16T11:24:14.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.825+0000 7fa39553a640 1 -- 172.21.15.84:0/1629711293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3900cce20 msgr2=0x7fa390156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:14.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.825+0000 7fa39553a640 1 --2- 172.21.15.84:0/1629711293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3900cce20 0x7fa390156e40 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fa380000c00 tx=0x7fa3800046c0 comp rx=0 tx=0).stop 2023-10-16T11:24:14.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.825+0000 7fa39553a640 1 -- 172.21.15.84:0/1629711293 shutdown_connections 2023-10-16T11:24:14.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.825+0000 7fa39553a640 1 --2- 172.21.15.84:0/1629711293 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa37c06af90 0x7fa37c06d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:14.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.825+0000 7fa39553a640 1 --2- 172.21.15.84:0/1629711293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa390159da0 0x7fa3900c3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:14.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.825+0000 7fa39553a640 1 --2- 172.21.15.84:0/1629711293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3901599c0 0x7fa39014fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:14.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.825+0000 7fa39553a640 1 --2- 172.21.15.84:0/1629711293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3900cce20 0x7fa390156e40 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:14.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.825+0000 7fa39553a640 1 -- 172.21.15.84:0/1629711293 >> 172.21.15.84:0/1629711293 conn(0x7fa3900cb360 msgr2=0x7fa3901566b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:14.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.826+0000 7fa39553a640 1 -- 172.21.15.84:0/1629711293 shutdown_connections 2023-10-16T11:24:14.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:14.826+0000 7fa39553a640 1 -- 172.21.15.84:0/1629711293 wait complete. 2023-10-16T11:24:14.854 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T11:24:15.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.050+0000 7f81961fb640 1 Processor -- start 2023-10-16T11:24:15.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.050+0000 7f81961fb640 1 -- start start 2023-10-16T11:24:15.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.051+0000 7f81961fb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8190159ab0 0x7f81900ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.051+0000 7f81961fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f819006c140 0x7f819006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.051+0000 7f81961fb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8190070360 con 0x7f8190159e90 2023-10-16T11:24:15.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.051+0000 7f81961fb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f819006cb10 con 0x7f8190159ab0 2023-10-16T11:24:15.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.051+0000 7f81961fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819015d890 con 0x7f819006c140 2023-10-16T11:24:15.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.052+0000 7f8187fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8190159e90 0x7f819006fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48678/0 (socket says 172.21.15.84:48678) 2023-10-16T11:24:15.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.052+0000 7f8187fff640 1 -- 172.21.15.84:0/1411836085 learned_addr learned my addr 172.21.15.84:0/1411836085 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:15.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.052+0000 7f818f7fe640 1 --2- 172.21.15.84:0/1411836085 >> v2:172.21.15.157:3300/0 conn(0x7f8190159ab0 0x7f81900ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:15.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.052+0000 7f818ffff640 1 --2- 172.21.15.84:0/1411836085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f819006c140 0x7f819006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:15.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.052+0000 7f818effd640 1 -- 172.21.15.84:0/1411836085 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1160363871 0 0) 0x7f8190070360 con 0x7f8190159e90 2023-10-16T11:24:15.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.052+0000 7f818effd640 1 -- 172.21.15.84:0/1411836085 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f817c003520 con 0x7f8190159e90 2023-10-16T11:24:15.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.053+0000 7f818effd640 1 -- 172.21.15.84:0/1411836085 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4189164976 0 0) 0x7f817c003520 con 0x7f8190159e90 2023-10-16T11:24:15.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.053+0000 7f818effd640 1 -- 172.21.15.84:0/1411836085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f819006c140 msgr2=0x7f819006c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:15.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.053+0000 7f818effd640 1 --2- 172.21.15.84:0/1411836085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f819006c140 0x7f819006c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.053+0000 7f818effd640 1 -- 172.21.15.84:0/1411836085 >> v2:172.21.15.157:3300/0 conn(0x7f8190159ab0 msgr2=0x7f81900ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:15.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.053+0000 7f818effd640 1 --2- 172.21.15.84:0/1411836085 >> v2:172.21.15.157:3300/0 conn(0x7f8190159ab0 0x7f81900ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.053+0000 7f818effd640 1 -- 172.21.15.84:0/1411836085 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81900cd1f0 con 0x7f8190159e90 2023-10-16T11:24:15.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.053+0000 7f818effd640 1 -- 172.21.15.84:0/1411836085 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f8178001670 con 0x7f8190159e90 2023-10-16T11:24:15.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.053+0000 7f818effd640 1 -- 172.21.15.84:0/1411836085 >> v1:172.21.15.84:6789/0 conn(0x7f8190159e90 legacy=0x7f819006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:15.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.053+0000 7f818effd640 1 --2- 172.21.15.84:0/1411836085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f819006c140 0x7f817c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.053+0000 7f818effd640 1 --2- 172.21.15.84:0/1411836085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f817c003ef0 0x7f817c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.053+0000 7f818effd640 1 --2- 172.21.15.84:0/1411836085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f817c004d00 0x7f817c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.053+0000 7f818effd640 1 -- 172.21.15.84:0/1411836085 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819006cb10 con 0x7f817c004d00 2023-10-16T11:24:15.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.053+0000 7f818effd640 1 -- 172.21.15.84:0/1411836085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819015d890 con 0x7f817c003ef0 2023-10-16T11:24:15.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.053+0000 7f818effd640 1 -- 172.21.15.84:0/1411836085 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817c00ba60 con 0x7f819006c140 2023-10-16T11:24:15.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.053+0000 7f818ffff640 1 --2- 172.21.15.84:0/1411836085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f817c004d00 0x7f817c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:15.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.053+0000 7f8187fff640 1 --2- 172.21.15.84:0/1411836085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f817c003ef0 0x7f817c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:15.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.054+0000 7f818ffff640 1 -- 172.21.15.84:0/1411836085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f819006c140 msgr2=0x7f817c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:15.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.054+0000 7f818ffff640 1 --2- 172.21.15.84:0/1411836085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f819006c140 0x7f817c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.054+0000 7f818ffff640 1 -- 172.21.15.84:0/1411836085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f817c003ef0 msgr2=0x7f817c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:15.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.054+0000 7f818ffff640 1 --2- 172.21.15.84:0/1411836085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f817c003ef0 0x7f817c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.054+0000 7f818ffff640 1 -- 172.21.15.84:0/1411836085 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817c003520 con 0x7f817c004d00 2023-10-16T11:24:15.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.055+0000 7f818ffff640 1 --2- 172.21.15.84:0/1411836085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f817c004d00 0x7f817c005180 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f8188009fc0 tx=0x7f817c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404fb50e60b1c89c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:15.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.055+0000 7f818effd640 1 -- 172.21.15.84:0/1411836085 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818802cc60 con 0x7f817c004d00 2023-10-16T11:24:15.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.055+0000 7f818effd640 1 -- 172.21.15.84:0/1411836085 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818800f040 con 0x7f817c004d00 2023-10-16T11:24:15.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.055+0000 7f818effd640 1 -- 172.21.15.84:0/1411836085 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8188034440 con 0x7f817c004d00 2023-10-16T11:24:15.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.055+0000 7f81961fb640 1 -- 172.21.15.84:0/1411836085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f817c004d00 msgr2=0x7f817c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:15.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.055+0000 7f81961fb640 1 --2- 172.21.15.84:0/1411836085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f817c004d00 0x7f817c005180 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f8188009fc0 tx=0x7f817c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.055+0000 7f818ffff640 1 -- 172.21.15.84:0/1411836085 reap_dead start 2023-10-16T11:24:15.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.056+0000 7f81961fb640 1 -- 172.21.15.84:0/1411836085 shutdown_connections 2023-10-16T11:24:15.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.056+0000 7f81961fb640 1 -- 172.21.15.84:0/1411836085 >> 172.21.15.84:0/1411836085 conn(0x7f81900ce9b0 msgr2=0x7f81900cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:15.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.056+0000 7f81961fb640 1 -- 172.21.15.84:0/1411836085 shutdown_connections 2023-10-16T11:24:15.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.056+0000 7f81961fb640 1 -- 172.21.15.84:0/1411836085 wait complete. 2023-10-16T11:24:15.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.058+0000 7f81961fb640 1 Processor -- start 2023-10-16T11:24:15.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.058+0000 7f81961fb640 1 -- start start 2023-10-16T11:24:15.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.059+0000 7f81961fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f817c004d00 0x7f819014e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.059+0000 7f81961fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f819015b9a0 0x7f81900b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.059+0000 7f81961fb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f819015cf00 0x7f81900c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.059+0000 7f81961fb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819015cae0 con 0x7f819015cf00 2023-10-16T11:24:15.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.059+0000 7f81961fb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819015be60 con 0x7f817c004d00 2023-10-16T11:24:15.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.059+0000 7f81961fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819015c140 con 0x7f819015b9a0 2023-10-16T11:24:15.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.059+0000 7f818ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f819015cf00 0x7f81900c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:15.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.059+0000 7f818ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f819015cf00 0x7f81900c4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44942/0 (socket says 172.21.15.84:44942) 2023-10-16T11:24:15.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.059+0000 7f818ffff640 1 -- 172.21.15.84:0/2871069881 learned_addr learned my addr 172.21.15.84:0/2871069881 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:15.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.059+0000 7f818f7fe640 1 --2- 172.21.15.84:0/2871069881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f817c004d00 0x7f819014e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:15.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.059+0000 7f818ffff640 1 -- 172.21.15.84:0/2871069881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f819015b9a0 msgr2=0x7f81900b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:15.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.059+0000 7f818ffff640 1 --2- 172.21.15.84:0/2871069881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f819015b9a0 0x7f81900b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.059+0000 7f818ffff640 1 -- 172.21.15.84:0/2871069881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f817c004d00 msgr2=0x7f819014e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:15.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.059+0000 7f818ffff640 1 --2- 172.21.15.84:0/2871069881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f817c004d00 0x7f819014e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.059+0000 7f818ffff640 1 -- 172.21.15.84:0/2871069881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819015c420 con 0x7f819015cf00 2023-10-16T11:24:15.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.060+0000 7f818ffff640 1 --2- 172.21.15.84:0/2871069881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f819015cf00 0x7f81900c4b50 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f8188037040 tx=0x7f8188009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:15.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.060+0000 7f818cff9640 1 -- 172.21.15.84:0/2871069881 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8188043070 con 0x7f819015cf00 2023-10-16T11:24:15.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.060+0000 7f818cff9640 1 -- 172.21.15.84:0/2871069881 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8188034ad0 con 0x7f819015cf00 2023-10-16T11:24:15.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.060+0000 7f81961fb640 1 -- 172.21.15.84:0/2871069881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819015c700 con 0x7f819015cf00 2023-10-16T11:24:15.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.060+0000 7f818cff9640 1 -- 172.21.15.84:0/2871069881 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8188034e40 con 0x7f819015cf00 2023-10-16T11:24:15.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.060+0000 7f81961fb640 1 -- 172.21.15.84:0/2871069881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81900b1c00 con 0x7f819015cf00 2023-10-16T11:24:15.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.061+0000 7f81961fb640 1 -- 172.21.15.84:0/2871069881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f819015cc20 con 0x7f819015cf00 2023-10-16T11:24:15.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.062+0000 7f818cff9640 1 -- 172.21.15.84:0/2871069881 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f818803e440 con 0x7f819015cf00 2023-10-16T11:24:15.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.063+0000 7f818cff9640 1 --2- 172.21.15.84:0/2871069881 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f817006af90 0x7f817006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.063+0000 7f818cff9640 1 -- 172.21.15.84:0/2871069881 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5773+0+0 (secure 0 0 0) 0x7f81880ab2c0 con 0x7f819015cf00 2023-10-16T11:24:15.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.063+0000 7f818f7fe640 1 --2- 172.21.15.84:0/2871069881 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f817006af90 0x7f817006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:15.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.064+0000 7f818f7fe640 1 --2- 172.21.15.84:0/2871069881 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f817006af90 0x7f817006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8180004770 tx=0x7f8180008040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:15.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.068+0000 7f818cff9640 1 -- 172.21.15.84:0/2871069881 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f819015cc20 con 0x7f819015cf00 2023-10-16T11:24:15.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.260+0000 7f81961fb640 1 -- 172.21.15.84:0/2871069881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f819015cc20 con 0x7f819015cf00 2023-10-16T11:24:15.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.267+0000 7f818cff9640 1 -- 172.21.15.84:0/2871069881 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7f819015cc20 con 0x7f819015cf00 2023-10-16T11:24:15.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.271+0000 7f81961fb640 1 -- 172.21.15.84:0/2871069881 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f817006af90 msgr2=0x7f817006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:15.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.271+0000 7f81961fb640 1 --2- 172.21.15.84:0/2871069881 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f817006af90 0x7f817006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8180004770 tx=0x7f8180008040 comp rx=0 tx=0).stop 2023-10-16T11:24:15.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.271+0000 7f81961fb640 1 -- 172.21.15.84:0/2871069881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f819015cf00 msgr2=0x7f81900c4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:15.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.271+0000 7f81961fb640 1 --2- 172.21.15.84:0/2871069881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f819015cf00 0x7f81900c4b50 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f8188037040 tx=0x7f8188009750 comp rx=0 tx=0).stop 2023-10-16T11:24:15.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.271+0000 7f81961fb640 1 -- 172.21.15.84:0/2871069881 shutdown_connections 2023-10-16T11:24:15.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.271+0000 7f81961fb640 1 --2- 172.21.15.84:0/2871069881 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f817006af90 0x7f817006d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.271+0000 7f81961fb640 1 --2- 172.21.15.84:0/2871069881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f819015cf00 0x7f81900c4b50 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.271+0000 7f81961fb640 1 --2- 172.21.15.84:0/2871069881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f819015b9a0 0x7f81900b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.271+0000 7f81961fb640 1 --2- 172.21.15.84:0/2871069881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f817c004d00 0x7f819014e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.271+0000 7f81961fb640 1 -- 172.21.15.84:0/2871069881 >> 172.21.15.84:0/2871069881 conn(0x7f81900ce9b0 msgr2=0x7f81900703d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:15.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.271+0000 7f81961fb640 1 -- 172.21.15.84:0/2871069881 shutdown_connections 2023-10-16T11:24:15.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.271+0000 7f81961fb640 1 -- 172.21.15.84:0/2871069881 wait complete. 2023-10-16T11:24:15.292 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T11:24:15.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.488+0000 7fa6b48f1640 1 Processor -- start 2023-10-16T11:24:15.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.489+0000 7fa6b48f1640 1 -- start start 2023-10-16T11:24:15.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.490+0000 7fa6b48f1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6b006c040 0x7fa6b006c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.490+0000 7fa6b48f1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6b00ad970 0x7fa6b015e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.490+0000 7fa6b48f1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b006c960 con 0x7fa6b015ebc0 2023-10-16T11:24:15.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.490+0000 7fa6b48f1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b015b9d0 con 0x7fa6b00ad970 2023-10-16T11:24:15.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.490+0000 7fa6b48f1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b015bcb0 con 0x7fa6b006c040 2023-10-16T11:24:15.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.491+0000 7fa6a7fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6b00ad970 0x7fa6b015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:15.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.491+0000 7fa6aed9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6b006c040 0x7fa6b006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:15.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.491+0000 7fa6a7fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6b00ad970 0x7fa6b015e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:45134/0 (socket says 172.21.15.84:45134) 2023-10-16T11:24:15.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.491+0000 7fa6a7fff640 1 -- 172.21.15.84:0/2102898902 learned_addr learned my addr 172.21.15.84:0/2102898902 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:15.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.492+0000 7fa6a7fff640 1 -- 172.21.15.84:0/2102898902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6b006c040 msgr2=0x7fa6b006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:15.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.492+0000 7fa6a7fff640 1 --2- 172.21.15.84:0/2102898902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6b006c040 0x7fa6b006c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.492+0000 7fa6a7fff640 1 -- 172.21.15.84:0/2102898902 >> v1:172.21.15.157:6789/0 conn(0x7fa6b015ebc0 legacy=0x7fa6b00cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:15.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.492+0000 7fa6a7fff640 1 -- 172.21.15.84:0/2102898902 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b015bf90 con 0x7fa6b00ad970 2023-10-16T11:24:15.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.494+0000 7fa6a7fff640 1 --2- 172.21.15.84:0/2102898902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6b00ad970 0x7fa6b015e680 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa6980098a0 tx=0x7fa69802b970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28590978b9961304 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:15.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.494+0000 7fa6ae59c640 1 -- 172.21.15.84:0/2102898902 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698030830 con 0x7fa6b00ad970 2023-10-16T11:24:15.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.495+0000 7fa6ae59c640 1 -- 172.21.15.84:0/2102898902 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa698030990 con 0x7fa6b00ad970 2023-10-16T11:24:15.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.495+0000 7fa6ae59c640 1 -- 172.21.15.84:0/2102898902 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698030d20 con 0x7fa6b00ad970 2023-10-16T11:24:15.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.495+0000 7fa6b48f1640 1 -- 172.21.15.84:0/2102898902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6b00ad970 msgr2=0x7fa6b015e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:15.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.495+0000 7fa6b48f1640 1 --2- 172.21.15.84:0/2102898902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6b00ad970 0x7fa6b015e680 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa6980098a0 tx=0x7fa69802b970 comp rx=0 tx=0).stop 2023-10-16T11:24:15.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.495+0000 7fa6b48f1640 1 -- 172.21.15.84:0/2102898902 shutdown_connections 2023-10-16T11:24:15.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.495+0000 7fa6b48f1640 1 --2- 172.21.15.84:0/2102898902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6b00ad970 0x7fa6b015e680 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.495+0000 7fa6b48f1640 1 --2- 172.21.15.84:0/2102898902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6b006c040 0x7fa6b006c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.495+0000 7fa6b48f1640 1 -- 172.21.15.84:0/2102898902 >> 172.21.15.84:0/2102898902 conn(0x7fa6b00ce6a0 msgr2=0x7fa6b00ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:15.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.496+0000 7fa6b48f1640 1 -- 172.21.15.84:0/2102898902 shutdown_connections 2023-10-16T11:24:15.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.496+0000 7fa6b48f1640 1 -- 172.21.15.84:0/2102898902 wait complete. 2023-10-16T11:24:15.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.498+0000 7fa6b48f1640 1 Processor -- start 2023-10-16T11:24:15.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.498+0000 7fa6b48f1640 1 -- start start 2023-10-16T11:24:15.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.499+0000 7fa6b48f1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6b006c040 0x7fa6b01f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.499+0000 7fa6b48f1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6b00ad970 0x7fa6b00acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.499+0000 7fa6b48f1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6b015ebc0 0x7fa6b0154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.499+0000 7fa6b48f1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b00c7790 con 0x7fa6b006c040 2023-10-16T11:24:15.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.499+0000 7fa6b48f1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b00c6b80 con 0x7fa6b015ebc0 2023-10-16T11:24:15.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.499+0000 7fa6b48f1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b00c6e60 con 0x7fa6b00ad970 2023-10-16T11:24:15.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.499+0000 7fa6aed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6b006c040 0x7fa6b01f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:15.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.499+0000 7fa6aed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6b006c040 0x7fa6b01f1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44944/0 (socket says 172.21.15.84:44944) 2023-10-16T11:24:15.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.499+0000 7fa6aed9d640 1 -- 172.21.15.84:0/1885664502 learned_addr learned my addr 172.21.15.84:0/1885664502 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:15.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.500+0000 7fa6af59e640 1 --2- 172.21.15.84:0/1885664502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6b015ebc0 0x7fa6b0154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:15.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.500+0000 7fa6aed9d640 1 -- 172.21.15.84:0/1885664502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6b00ad970 msgr2=0x7fa6b00acc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:15.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.500+0000 7fa6aed9d640 1 --2- 172.21.15.84:0/1885664502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6b00ad970 0x7fa6b00acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.500+0000 7fa6aed9d640 1 -- 172.21.15.84:0/1885664502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6b015ebc0 msgr2=0x7fa6b0154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:15.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.500+0000 7fa6aed9d640 1 --2- 172.21.15.84:0/1885664502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6b015ebc0 0x7fa6b0154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.500+0000 7fa6aed9d640 1 -- 172.21.15.84:0/1885664502 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6980094d0 con 0x7fa6b006c040 2023-10-16T11:24:15.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.500+0000 7fa6af59e640 1 --2- 172.21.15.84:0/1885664502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6b015ebc0 0x7fa6b0154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:15.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.500+0000 7fa6aed9d640 1 --2- 172.21.15.84:0/1885664502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6b006c040 0x7fa6b01f1f20 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000e870 tx=0x7fa6a000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:15.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.500+0000 7fa6a77fe640 1 -- 172.21.15.84:0/1885664502 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a00094f0 con 0x7fa6b006c040 2023-10-16T11:24:15.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.500+0000 7fa6a77fe640 1 -- 172.21.15.84:0/1885664502 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a000c940 con 0x7fa6b006c040 2023-10-16T11:24:15.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.500+0000 7fa6b48f1640 1 -- 172.21.15.84:0/1885664502 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b00c7140 con 0x7fa6b006c040 2023-10-16T11:24:15.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.500+0000 7fa6a77fe640 1 -- 172.21.15.84:0/1885664502 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a000ccf0 con 0x7fa6b006c040 2023-10-16T11:24:15.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.501+0000 7fa6b48f1640 1 -- 172.21.15.84:0/1885664502 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b00c67c0 con 0x7fa6b006c040 2023-10-16T11:24:15.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.502+0000 7fa6b48f1640 1 -- 172.21.15.84:0/1885664502 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b006b330 con 0x7fa6b006c040 2023-10-16T11:24:15.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.503+0000 7fa6a77fe640 1 -- 172.21.15.84:0/1885664502 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa6a0013470 con 0x7fa6b006c040 2023-10-16T11:24:15.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.504+0000 7fa6a77fe640 1 --2- 172.21.15.84:0/1885664502 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa69006af90 0x7fa69006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.504+0000 7fa6a77fe640 1 -- 172.21.15.84:0/1885664502 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5773+0+0 (secure 0 0 0) 0x7fa6a00898f0 con 0x7fa6b006c040 2023-10-16T11:24:15.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.504+0000 7fa6a7fff640 1 --2- 172.21.15.84:0/1885664502 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa69006af90 0x7fa69006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:15.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.505+0000 7fa6a7fff640 1 --2- 172.21.15.84:0/1885664502 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa69006af90 0x7fa69006d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa698036000 tx=0x7fa69803b040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:15.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.508+0000 7fa6a77fe640 1 -- 172.21.15.84:0/1885664502 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa6a0054850 con 0x7fa6b006c040 2023-10-16T11:24:15.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.686+0000 7fa6b48f1640 1 -- 172.21.15.84:0/1885664502 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fa6b004e9b0 con 0x7fa6b006c040 2023-10-16T11:24:15.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.695+0000 7fa6a77fe640 1 -- 172.21.15.84:0/1885664502 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v74) v1 ==== 120+0+0 (secure 0 0 0) 0x7fa6a0057e70 con 0x7fa6b006c040 2023-10-16T11:24:15.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.697+0000 7fa6b48f1640 1 -- 172.21.15.84:0/1885664502 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa69006af90 msgr2=0x7fa69006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:15.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.697+0000 7fa6b48f1640 1 --2- 172.21.15.84:0/1885664502 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa69006af90 0x7fa69006d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa698036000 tx=0x7fa69803b040 comp rx=0 tx=0).stop 2023-10-16T11:24:15.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.697+0000 7fa6b48f1640 1 -- 172.21.15.84:0/1885664502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6b006c040 msgr2=0x7fa6b01f1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:15.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.698+0000 7fa6b48f1640 1 --2- 172.21.15.84:0/1885664502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6b006c040 0x7fa6b01f1f20 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000e870 tx=0x7fa6a000ed30 comp rx=0 tx=0).stop 2023-10-16T11:24:15.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.698+0000 7fa6b48f1640 1 -- 172.21.15.84:0/1885664502 shutdown_connections 2023-10-16T11:24:15.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.698+0000 7fa6b48f1640 1 --2- 172.21.15.84:0/1885664502 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa69006af90 0x7fa69006d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.698+0000 7fa6b48f1640 1 --2- 172.21.15.84:0/1885664502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6b015ebc0 0x7fa6b0154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.698+0000 7fa6b48f1640 1 --2- 172.21.15.84:0/1885664502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6b00ad970 0x7fa6b00acc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.698+0000 7fa6b48f1640 1 --2- 172.21.15.84:0/1885664502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6b006c040 0x7fa6b01f1f20 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.698+0000 7fa6b48f1640 1 -- 172.21.15.84:0/1885664502 >> 172.21.15.84:0/1885664502 conn(0x7fa6b00ce6a0 msgr2=0x7fa6b00cd200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:15.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.698+0000 7fa6b48f1640 1 -- 172.21.15.84:0/1885664502 shutdown_connections 2023-10-16T11:24:15.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.698+0000 7fa6b48f1640 1 -- 172.21.15.84:0/1885664502 wait complete. 2023-10-16T11:24:15.716 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T11:24:15.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.920+0000 7f0f2840e640 1 Processor -- start 2023-10-16T11:24:15.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.920+0000 7f0f2840e640 1 -- start start 2023-10-16T11:24:15.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.922+0000 7f0f2840e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0f2015aaa0 0x7f0f2015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.922+0000 7f0f2840e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f200ce390 0x7f0f20070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.922+0000 7f0f2840e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f200707e0 con 0x7f0f2015ae80 2023-10-16T11:24:15.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.922+0000 7f0f2840e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f2006b7a0 con 0x7f0f2015aaa0 2023-10-16T11:24:15.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.922+0000 7f0f2840e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2015ef20 con 0x7f0f200ce390 2023-10-16T11:24:15.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.923+0000 7f0f261aa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0f2015aaa0 0x7f0f2015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:15.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.923+0000 7f0f269ab640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f200ce390 0x7f0f20070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:15.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.923+0000 7f0f261aa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0f2015aaa0 0x7f0f2015e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:49878/0 (socket says 172.21.15.84:49878) 2023-10-16T11:24:15.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.923+0000 7f0f261aa640 1 -- 172.21.15.84:0/2493856676 learned_addr learned my addr 172.21.15.84:0/2493856676 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:15.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.923+0000 7f0f261aa640 1 -- 172.21.15.84:0/2493856676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f200ce390 msgr2=0x7f0f20070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:15.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.923+0000 7f0f261aa640 1 --2- 172.21.15.84:0/2493856676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f200ce390 0x7f0f20070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.923+0000 7f0f261aa640 1 -- 172.21.15.84:0/2493856676 >> v1:172.21.15.157:6789/0 conn(0x7f0f2015ae80 legacy=0x7f0f200cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:15.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.923+0000 7f0f261aa640 1 -- 172.21.15.84:0/2493856676 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2015f200 con 0x7f0f2015aaa0 2023-10-16T11:24:15.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.925+0000 7f0f261aa640 1 --2- 172.21.15.84:0/2493856676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f2015aaa0 0x7f0f2015e820 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009e50 tx=0x7f0f1c02bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6c89b503d7a8393 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:15.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.925+0000 7f0f251a8640 1 -- 172.21.15.84:0/2493856676 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c02ca70 con 0x7f0f2015aaa0 2023-10-16T11:24:15.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.926+0000 7f0f251a8640 1 -- 172.21.15.84:0/2493856676 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1c02cbd0 con 0x7f0f2015aaa0 2023-10-16T11:24:15.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.926+0000 7f0f251a8640 1 -- 172.21.15.84:0/2493856676 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c0343f0 con 0x7f0f2015aaa0 2023-10-16T11:24:15.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.926+0000 7f0f2840e640 1 -- 172.21.15.84:0/2493856676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f2015aaa0 msgr2=0x7f0f2015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:15.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.926+0000 7f0f2840e640 1 --2- 172.21.15.84:0/2493856676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f2015aaa0 0x7f0f2015e820 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009e50 tx=0x7f0f1c02bed0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.927+0000 7f0f2840e640 1 -- 172.21.15.84:0/2493856676 shutdown_connections 2023-10-16T11:24:15.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.927+0000 7f0f2840e640 1 --2- 172.21.15.84:0/2493856676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f200ce390 0x7f0f20070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.927+0000 7f0f2840e640 1 --2- 172.21.15.84:0/2493856676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f2015aaa0 0x7f0f2015e820 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.927+0000 7f0f2840e640 1 -- 172.21.15.84:0/2493856676 >> 172.21.15.84:0/2493856676 conn(0x7f0f200cb410 msgr2=0x7f0f200cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.927+0000 7f0f2840e640 1 -- 172.21.15.84:0/2493856676 shutdown_connections 2023-10-16T11:24:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.927+0000 7f0f2840e640 1 -- 172.21.15.84:0/2493856676 wait complete. 2023-10-16T11:24:15.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.929+0000 7f0f2840e640 1 Processor -- start 2023-10-16T11:24:15.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.929+0000 7f0f2840e640 1 -- start start 2023-10-16T11:24:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.930+0000 7f0f2840e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f200ce390 0x7f0f20158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.930+0000 7f0f2840e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2015aaa0 0x7f0f200c4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.930+0000 7f0f2840e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f2015ae80 0x7f0f200c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.930+0000 7f0f2840e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f201599b0 con 0x7f0f2015aaa0 2023-10-16T11:24:15.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.930+0000 7f0f2840e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f201596d0 con 0x7f0f2015ae80 2023-10-16T11:24:15.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.930+0000 7f0f2840e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f200c3770 con 0x7f0f200ce390 2023-10-16T11:24:15.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.931+0000 7f0f259a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2015aaa0 0x7f0f200c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:15.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.931+0000 7f0f259a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2015aaa0 0x7f0f200c4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40168/0 (socket says 172.21.15.84:40168) 2023-10-16T11:24:15.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.931+0000 7f0f259a9640 1 -- 172.21.15.84:0/1382083235 learned_addr learned my addr 172.21.15.84:0/1382083235 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:15.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.931+0000 7f0f261aa640 1 --2- 172.21.15.84:0/1382083235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f200ce390 0x7f0f20158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:15.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.931+0000 7f0f269ab640 1 --2- 172.21.15.84:0/1382083235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f2015ae80 0x7f0f200c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:15.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.931+0000 7f0f259a9640 1 -- 172.21.15.84:0/1382083235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f200ce390 msgr2=0x7f0f20158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:15.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.931+0000 7f0f259a9640 1 --2- 172.21.15.84:0/1382083235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f200ce390 0x7f0f20158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.931+0000 7f0f259a9640 1 -- 172.21.15.84:0/1382083235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f2015ae80 msgr2=0x7f0f200c4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:15.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.931+0000 7f0f259a9640 1 --2- 172.21.15.84:0/1382083235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f2015ae80 0x7f0f200c4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:15.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.931+0000 7f0f259a9640 1 -- 172.21.15.84:0/1382083235 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c009b30 con 0x7f0f2015aaa0 2023-10-16T11:24:15.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.931+0000 7f0f269ab640 1 --2- 172.21.15.84:0/1382083235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f2015ae80 0x7f0f200c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:15.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.932+0000 7f0f259a9640 1 --2- 172.21.15.84:0/1382083235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2015aaa0 0x7f0f200c4250 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000c440 tx=0x7f0f10001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.932+0000 7f0f16ffd640 1 -- 172.21.15.84:0/1382083235 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f100052d0 con 0x7f0f2015aaa0 2023-10-16T11:24:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.932+0000 7f0f16ffd640 1 -- 172.21.15.84:0/1382083235 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f10005430 con 0x7f0f2015aaa0 2023-10-16T11:24:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.932+0000 7f0f16ffd640 1 -- 172.21.15.84:0/1382083235 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10007260 con 0x7f0f2015aaa0 2023-10-16T11:24:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.932+0000 7f0f2840e640 1 -- 172.21.15.84:0/1382083235 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f200c3a50 con 0x7f0f2015aaa0 2023-10-16T11:24:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.932+0000 7f0f2840e640 1 -- 172.21.15.84:0/1382083235 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f20156770 con 0x7f0f2015aaa0 2023-10-16T11:24:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.933+0000 7f0f2840e640 1 -- 172.21.15.84:0/1382083235 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f200c3f70 con 0x7f0f2015aaa0 2023-10-16T11:24:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.934+0000 7f0f16ffd640 1 -- 172.21.15.84:0/1382083235 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0f100073c0 con 0x7f0f2015aaa0 2023-10-16T11:24:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.935+0000 7f0f16ffd640 1 --2- 172.21.15.84:0/1382083235 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0f0406af90 0x7f0f0406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.935+0000 7f0f16ffd640 1 -- 172.21.15.84:0/1382083235 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5773+0+0 (secure 0 0 0) 0x7f0f10088c00 con 0x7f0f2015aaa0 2023-10-16T11:24:15.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.935+0000 7f0f261aa640 1 --2- 172.21.15.84:0/1382083235 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0f0406af90 0x7f0f0406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:15.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.936+0000 7f0f261aa640 1 --2- 172.21.15.84:0/1382083235 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0f0406af90 0x7f0f0406d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c035000 tx=0x7f0f1c030000 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:15.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:15.942+0000 7f0f16ffd640 1 -- 172.21.15.84:0/1382083235 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0f200c3f70 con 0x7f0f2015aaa0 2023-10-16T11:24:16.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.149+0000 7f0f2840e640 1 -- 172.21.15.84:0/1382083235 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f0f200c3f70 con 0x7f0f2015aaa0 2023-10-16T11:24:16.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.156+0000 7f0f16ffd640 1 -- 172.21.15.84:0/1382083235 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v75) v1 ==== 115+0+0 (secure 0 0 0) 0x7f0f200c3f70 con 0x7f0f2015aaa0 2023-10-16T11:24:16.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.159+0000 7f0f2840e640 1 -- 172.21.15.84:0/1382083235 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0f0406af90 msgr2=0x7f0f0406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:16.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.159+0000 7f0f2840e640 1 --2- 172.21.15.84:0/1382083235 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0f0406af90 0x7f0f0406d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c035000 tx=0x7f0f1c030000 comp rx=0 tx=0).stop 2023-10-16T11:24:16.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.159+0000 7f0f2840e640 1 -- 172.21.15.84:0/1382083235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2015aaa0 msgr2=0x7f0f200c4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:16.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.159+0000 7f0f2840e640 1 --2- 172.21.15.84:0/1382083235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2015aaa0 0x7f0f200c4250 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000c440 tx=0x7f0f10001e70 comp rx=0 tx=0).stop 2023-10-16T11:24:16.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.159+0000 7f0f2840e640 1 -- 172.21.15.84:0/1382083235 shutdown_connections 2023-10-16T11:24:16.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.159+0000 7f0f2840e640 1 --2- 172.21.15.84:0/1382083235 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0f0406af90 0x7f0f0406d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:16.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.159+0000 7f0f2840e640 1 --2- 172.21.15.84:0/1382083235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f2015ae80 0x7f0f200c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:16.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.159+0000 7f0f2840e640 1 --2- 172.21.15.84:0/1382083235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2015aaa0 0x7f0f200c4250 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:16.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.159+0000 7f0f2840e640 1 --2- 172.21.15.84:0/1382083235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f200ce390 0x7f0f20158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:16.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.159+0000 7f0f2840e640 1 -- 172.21.15.84:0/1382083235 >> 172.21.15.84:0/1382083235 conn(0x7f0f200cb410 msgr2=0x7f0f200b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:16.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.159+0000 7f0f2840e640 1 -- 172.21.15.84:0/1382083235 shutdown_connections 2023-10-16T11:24:16.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.160+0000 7f0f2840e640 1 -- 172.21.15.84:0/1382083235 wait complete. 2023-10-16T11:24:16.180 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-10-16T11:24:16.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.389+0000 7f500aead640 1 Processor -- start 2023-10-16T11:24:16.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.389+0000 7f500aead640 1 -- start start 2023-10-16T11:24:16.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.391+0000 7f500aead640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f50040ad970 0x7f50040cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:16.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.391+0000 7f500aead640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50040ccfb0 0x7f50040cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:16.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.391+0000 7f500aead640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f500415b9d0 con 0x7f500406c040 2023-10-16T11:24:16.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.391+0000 7f500aead640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f500415bcb0 con 0x7f50040ad970 2023-10-16T11:24:16.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.391+0000 7f500aead640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500415bf90 con 0x7f50040ccfb0 2023-10-16T11:24:16.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.391+0000 7f500a6ac640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50040ccfb0 0x7f50040cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:16.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.392+0000 7f50096aa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f50040ad970 0x7f50040cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:16.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.392+0000 7f500a6ac640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50040ccfb0 0x7f50040cd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:45268/0 (socket says 172.21.15.84:45268) 2023-10-16T11:24:16.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.392+0000 7f50096aa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f50040ad970 0x7f50040cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:49898/0 (socket says 172.21.15.84:49898) 2023-10-16T11:24:16.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.392+0000 7f500a6ac640 1 -- 172.21.15.84:0/2505001471 learned_addr learned my addr 172.21.15.84:0/2505001471 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:16.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.392+0000 7f50096aa640 1 -- 172.21.15.84:0/2505001471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50040ccfb0 msgr2=0x7f50040cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:16.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.392+0000 7f50096aa640 1 --2- 172.21.15.84:0/2505001471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50040ccfb0 0x7f50040cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:16.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.392+0000 7f50096aa640 1 -- 172.21.15.84:0/2505001471 >> v1:172.21.15.157:6789/0 conn(0x7f500406c040 legacy=0x7f500406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:16.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.392+0000 7f50096aa640 1 -- 172.21.15.84:0/2505001471 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50040cc270 con 0x7f50040ad970 2023-10-16T11:24:16.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.394+0000 7f50096aa640 1 --2- 172.21.15.84:0/2505001471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50040ad970 0x7f50040cca70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40097a0 tx=0x7f4ff402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83a40081f9acfe27 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:16.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.394+0000 7f5008ea9640 1 -- 172.21.15.84:0/2505001471 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff402d040 con 0x7f50040ad970 2023-10-16T11:24:16.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.394+0000 7f5008ea9640 1 -- 172.21.15.84:0/2505001471 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff4031810 con 0x7f50040ad970 2023-10-16T11:24:16.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.394+0000 7f5008ea9640 1 -- 172.21.15.84:0/2505001471 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4031ba0 con 0x7f50040ad970 2023-10-16T11:24:16.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.395+0000 7f500aead640 1 -- 172.21.15.84:0/2505001471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50040ad970 msgr2=0x7f50040cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:16.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.395+0000 7f500aead640 1 --2- 172.21.15.84:0/2505001471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50040ad970 0x7f50040cca70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40097a0 tx=0x7f4ff402b8b0 comp rx=0 tx=0).stop 2023-10-16T11:24:16.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.395+0000 7f500aead640 1 -- 172.21.15.84:0/2505001471 shutdown_connections 2023-10-16T11:24:16.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.395+0000 7f500aead640 1 --2- 172.21.15.84:0/2505001471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50040ccfb0 0x7f50040cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:16.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.395+0000 7f500aead640 1 --2- 172.21.15.84:0/2505001471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50040ad970 0x7f50040cca70 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:16.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.395+0000 7f500aead640 1 -- 172.21.15.84:0/2505001471 >> 172.21.15.84:0/2505001471 conn(0x7f50040ce6a0 msgr2=0x7f50040ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:16.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.395+0000 7f500aead640 1 -- 172.21.15.84:0/2505001471 shutdown_connections 2023-10-16T11:24:16.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.395+0000 7f500aead640 1 -- 172.21.15.84:0/2505001471 wait complete. 2023-10-16T11:24:16.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.397+0000 7f500aead640 1 Processor -- start 2023-10-16T11:24:16.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.398+0000 7f500aead640 1 -- start start 2023-10-16T11:24:16.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.399+0000 7f500aead640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f500406c040 0x7f50040c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:16.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.399+0000 7f500aead640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50040ad970 0x7f50040ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:16.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.399+0000 7f500aead640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50040ccfb0 0x7f50040aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:16.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.399+0000 7f500aead640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50040c5770 con 0x7f500406c040 2023-10-16T11:24:16.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.399+0000 7f500aead640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50040c6010 con 0x7f50040ccfb0 2023-10-16T11:24:16.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.399+0000 7f500aead640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50040ca9f0 con 0x7f50040ad970 2023-10-16T11:24:16.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.399+0000 7f5009eab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f500406c040 0x7f50040c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:16.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.399+0000 7f5009eab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f500406c040 0x7f50040c64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40170/0 (socket says 172.21.15.84:40170) 2023-10-16T11:24:16.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.399+0000 7f5009eab640 1 -- 172.21.15.84:0/2180796872 learned_addr learned my addr 172.21.15.84:0/2180796872 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:16.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.399+0000 7f50096aa640 1 --2- 172.21.15.84:0/2180796872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50040ad970 0x7f50040ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:16.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.400+0000 7f500a6ac640 1 --2- 172.21.15.84:0/2180796872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50040ccfb0 0x7f50040aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:16.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.400+0000 7f5009eab640 1 -- 172.21.15.84:0/2180796872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50040ad970 msgr2=0x7f50040ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:16.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.400+0000 7f5009eab640 1 --2- 172.21.15.84:0/2180796872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50040ad970 0x7f50040ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:16.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.400+0000 7f5009eab640 1 -- 172.21.15.84:0/2180796872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50040ccfb0 msgr2=0x7f50040aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:16.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.400+0000 7f5009eab640 1 --2- 172.21.15.84:0/2180796872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50040ccfb0 0x7f50040aca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:16.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.400+0000 7f5009eab640 1 -- 172.21.15.84:0/2180796872 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff4009400 con 0x7f500406c040 2023-10-16T11:24:16.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.400+0000 7f5009eab640 1 --2- 172.21.15.84:0/2180796872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f500406c040 0x7f50040c64a0 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f500000c450 tx=0x7f500000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:16.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.400+0000 7f4ffa7fc640 1 -- 172.21.15.84:0/2180796872 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50000052d0 con 0x7f500406c040 2023-10-16T11:24:16.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.400+0000 7f4ffa7fc640 1 -- 172.21.15.84:0/2180796872 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5000005430 con 0x7f500406c040 2023-10-16T11:24:16.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.400+0000 7f4ffa7fc640 1 -- 172.21.15.84:0/2180796872 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5000012870 con 0x7f500406c040 2023-10-16T11:24:16.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.400+0000 7f500aead640 1 -- 172.21.15.84:0/2180796872 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50040c59c0 con 0x7f500406c040 2023-10-16T11:24:16.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.400+0000 7f500aead640 1 -- 172.21.15.84:0/2180796872 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50041540f0 con 0x7f500406c040 2023-10-16T11:24:16.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.402+0000 7f500aead640 1 -- 172.21.15.84:0/2180796872 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500406b330 con 0x7f500406c040 2023-10-16T11:24:16.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.403+0000 7f4ffa7fc640 1 -- 172.21.15.84:0/2180796872 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f50000129d0 con 0x7f500406c040 2023-10-16T11:24:16.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.404+0000 7f4ffa7fc640 1 --2- 172.21.15.84:0/2180796872 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4fe806af90 0x7f4fe806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:16.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.404+0000 7f4ffa7fc640 1 -- 172.21.15.84:0/2180796872 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5773+0+0 (secure 0 0 0) 0x7f5000089320 con 0x7f500406c040 2023-10-16T11:24:16.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.405+0000 7f50096aa640 1 --2- 172.21.15.84:0/2180796872 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4fe806af90 0x7f4fe806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:16.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.405+0000 7f50096aa640 1 --2- 172.21.15.84:0/2180796872 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4fe806af90 0x7f4fe806d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4036000 tx=0x7f4ff403c040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:16.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.409+0000 7f4ffa7fc640 1 -- 172.21.15.84:0/2180796872 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5000054270 con 0x7f500406c040 2023-10-16T11:24:16.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.680+0000 7f500aead640 1 -- 172.21.15.84:0/2180796872 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f5004154280 con 0x7f500406c040 2023-10-16T11:24:16.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.984+0000 7f4ffa7fc640 1 -- 172.21.15.84:0/2180796872 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3109) v1 ==== 163+0+0 (secure 0 0 0) 0x7f5000057890 con 0x7f500406c040 2023-10-16T11:24:16.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.988+0000 7f500aead640 1 -- 172.21.15.84:0/2180796872 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4fe806af90 msgr2=0x7f4fe806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:16.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.988+0000 7f500aead640 1 --2- 172.21.15.84:0/2180796872 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4fe806af90 0x7f4fe806d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4036000 tx=0x7f4ff403c040 comp rx=0 tx=0).stop 2023-10-16T11:24:16.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.988+0000 7f500aead640 1 -- 172.21.15.84:0/2180796872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f500406c040 msgr2=0x7f50040c64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:16.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.988+0000 7f500aead640 1 --2- 172.21.15.84:0/2180796872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f500406c040 0x7f50040c64a0 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f500000c450 tx=0x7f500000c910 comp rx=0 tx=0).stop 2023-10-16T11:24:16.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.988+0000 7f500aead640 1 -- 172.21.15.84:0/2180796872 shutdown_connections 2023-10-16T11:24:16.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.988+0000 7f500aead640 1 --2- 172.21.15.84:0/2180796872 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4fe806af90 0x7f4fe806d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:16.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.988+0000 7f500aead640 1 --2- 172.21.15.84:0/2180796872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50040ccfb0 0x7f50040aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:16.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.988+0000 7f500aead640 1 --2- 172.21.15.84:0/2180796872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50040ad970 0x7f50040ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:16.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.988+0000 7f500aead640 1 --2- 172.21.15.84:0/2180796872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f500406c040 0x7f50040c64a0 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:16.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.988+0000 7f500aead640 1 -- 172.21.15.84:0/2180796872 >> 172.21.15.84:0/2180796872 conn(0x7f50040ce6a0 msgr2=0x7f500415c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:16.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.988+0000 7f500aead640 1 -- 172.21.15.84:0/2180796872 shutdown_connections 2023-10-16T11:24:16.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:16.988+0000 7f500aead640 1 -- 172.21.15.84:0/2180796872 wait complete. 2023-10-16T11:24:17.011 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2023-10-16T11:24:17.011 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_non_default_fs (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) Test command succes with --fs argument for non-default fs 2023-10-16T11:24:17.012 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_non_default_fs (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) 2023-10-16T11:24:17.012 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:24:17.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.228+0000 7f3f523b0640 1 Processor -- start 2023-10-16T11:24:17.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.228+0000 7f3f523b0640 1 -- start start 2023-10-16T11:24:17.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.230+0000 7f3f523b0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3f4c159b20 0x7f3f4c06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:17.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.230+0000 7f3f523b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f4c159f00 0x7f3f4c0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:17.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.230+0000 7f3f523b0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f4c06f7f0 con 0x7f3f4c06c120 2023-10-16T11:24:17.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.230+0000 7f3f523b0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f4c06c5e0 con 0x7f3f4c159b20 2023-10-16T11:24:17.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.230+0000 7f3f523b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c06c8c0 con 0x7f3f4c159f00 2023-10-16T11:24:17.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.230+0000 7f3f5094d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3f4c06c120 0x7f3f4c0cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58732/0 (socket says 172.21.15.84:58732) 2023-10-16T11:24:17.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.230+0000 7f3f5094d640 1 -- 172.21.15.84:0/313525884 learned_addr learned my addr 172.21.15.84:0/313525884 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:17.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.231+0000 7f3f4b7fe640 1 --2- 172.21.15.84:0/313525884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f4c159f00 0x7f3f4c0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:17.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.231+0000 7f3f4bfff640 1 --2- 172.21.15.84:0/313525884 >> v2:172.21.15.157:3300/0 conn(0x7f3f4c159b20 0x7f3f4c06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:17.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.231+0000 7f3f4affd640 1 -- 172.21.15.84:0/313525884 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955064500 0 0) 0x7f3f4c06f7f0 con 0x7f3f4c06c120 2023-10-16T11:24:17.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.231+0000 7f3f4affd640 1 -- 172.21.15.84:0/313525884 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f34003520 con 0x7f3f4c06c120 2023-10-16T11:24:17.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.231+0000 7f3f4affd640 1 -- 172.21.15.84:0/313525884 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 640149974 0 0) 0x7f3f34003520 con 0x7f3f4c06c120 2023-10-16T11:24:17.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.231+0000 7f3f4affd640 1 -- 172.21.15.84:0/313525884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f4c159f00 msgr2=0x7f3f4c0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:17.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.231+0000 7f3f4affd640 1 --2- 172.21.15.84:0/313525884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f4c159f00 0x7f3f4c0b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.231+0000 7f3f4affd640 1 -- 172.21.15.84:0/313525884 >> v2:172.21.15.157:3300/0 conn(0x7f3f4c159b20 msgr2=0x7f3f4c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:17.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.231+0000 7f3f4affd640 1 --2- 172.21.15.84:0/313525884 >> v2:172.21.15.157:3300/0 conn(0x7f3f4c159b20 0x7f3f4c06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.231+0000 7f3f4affd640 1 -- 172.21.15.84:0/313525884 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c06cba0 con 0x7f3f4c06c120 2023-10-16T11:24:17.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.231+0000 7f3f4affd640 1 -- 172.21.15.84:0/313525884 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f3f44001670 con 0x7f3f4c06c120 2023-10-16T11:24:17.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.231+0000 7f3f4affd640 1 -- 172.21.15.84:0/313525884 >> v1:172.21.15.84:6789/0 conn(0x7f3f4c06c120 legacy=0x7f3f4c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:17.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.231+0000 7f3f4affd640 1 --2- 172.21.15.84:0/313525884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f34003970 0x7f3f34003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:17.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.231+0000 7f3f4affd640 1 --2- 172.21.15.84:0/313525884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f4c159f00 0x7f3f340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:17.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.231+0000 7f3f4affd640 1 --2- 172.21.15.84:0/313525884 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f34004d00 0x7f3f34005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:17.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.231+0000 7f3f4affd640 1 -- 172.21.15.84:0/313525884 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c06c5e0 con 0x7f3f34004d00 2023-10-16T11:24:17.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.231+0000 7f3f4affd640 1 -- 172.21.15.84:0/313525884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c06c8c0 con 0x7f3f34003970 2023-10-16T11:24:17.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.231+0000 7f3f4affd640 1 -- 172.21.15.84:0/313525884 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3400b980 con 0x7f3f4c159f00 2023-10-16T11:24:17.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.232+0000 7f3f5094d640 1 --2- 172.21.15.84:0/313525884 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f34004d00 0x7f3f34005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:17.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.232+0000 7f3f4bfff640 1 --2- 172.21.15.84:0/313525884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f34003970 0x7f3f34003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:17.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.232+0000 7f3f4b7fe640 1 --2- 172.21.15.84:0/313525884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f4c159f00 0x7f3f340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:17.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.232+0000 7f3f5094d640 1 -- 172.21.15.84:0/313525884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f4c159f00 msgr2=0x7f3f340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:17.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.232+0000 7f3f5094d640 1 --2- 172.21.15.84:0/313525884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f4c159f00 0x7f3f340047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.232+0000 7f3f5094d640 1 -- 172.21.15.84:0/313525884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f34003970 msgr2=0x7f3f34003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:17.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.232+0000 7f3f5094d640 1 --2- 172.21.15.84:0/313525884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f34003970 0x7f3f34003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.232+0000 7f3f5094d640 1 -- 172.21.15.84:0/313525884 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f34003520 con 0x7f3f34004d00 2023-10-16T11:24:17.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.233+0000 7f3f5094d640 1 --2- 172.21.15.84:0/313525884 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f34004d00 0x7f3f34005180 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0c5650 tx=0x7f3f340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31dab2f352cb0b53 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:17.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.234+0000 7f3f4affd640 1 -- 172.21.15.84:0/313525884 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44005ad0 con 0x7f3f34004d00 2023-10-16T11:24:17.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.234+0000 7f3f4affd640 1 -- 172.21.15.84:0/313525884 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f44005c50 con 0x7f3f34004d00 2023-10-16T11:24:17.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.234+0000 7f3f4affd640 1 -- 172.21.15.84:0/313525884 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4402c7c0 con 0x7f3f34004d00 2023-10-16T11:24:17.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.234+0000 7f3f523b0640 1 -- 172.21.15.84:0/313525884 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f34004d00 msgr2=0x7f3f34005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:17.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.234+0000 7f3f523b0640 1 --2- 172.21.15.84:0/313525884 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f34004d00 0x7f3f34005180 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0c5650 tx=0x7f3f340057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.234+0000 7f3f5094d640 1 -- 172.21.15.84:0/313525884 reap_dead start 2023-10-16T11:24:17.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.234+0000 7f3f523b0640 1 -- 172.21.15.84:0/313525884 shutdown_connections 2023-10-16T11:24:17.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.234+0000 7f3f523b0640 1 -- 172.21.15.84:0/313525884 >> 172.21.15.84:0/313525884 conn(0x7f3f4c0cb410 msgr2=0x7f3f4c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:17.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.234+0000 7f3f523b0640 1 -- 172.21.15.84:0/313525884 shutdown_connections 2023-10-16T11:24:17.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.235+0000 7f3f523b0640 1 -- 172.21.15.84:0/313525884 wait complete. 2023-10-16T11:24:17.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.236+0000 7f3f523b0640 1 Processor -- start 2023-10-16T11:24:17.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.236+0000 7f3f523b0640 1 -- start start 2023-10-16T11:24:17.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.237+0000 7f3f523b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f34004d00 0x7f3f4c0b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:17.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.237+0000 7f3f523b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f4c155ac0 0x7f3f4c1552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:17.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.237+0000 7f3f523b0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f4c15c910 0x7f3f4c15cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:17.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.237+0000 7f3f523b0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c15d2d0 con 0x7f3f4c15c910 2023-10-16T11:24:17.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.237+0000 7f3f523b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c0b4110 con 0x7f3f4c155ac0 2023-10-16T11:24:17.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.237+0000 7f3f523b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c0b3070 con 0x7f3f34004d00 2023-10-16T11:24:17.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.238+0000 7f3f5094d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f4c15c910 0x7f3f4c15cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:17.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.238+0000 7f3f5094d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f4c15c910 0x7f3f4c15cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40192/0 (socket says 172.21.15.84:40192) 2023-10-16T11:24:17.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.238+0000 7f3f5094d640 1 -- 172.21.15.84:0/3952867339 learned_addr learned my addr 172.21.15.84:0/3952867339 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:17.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.238+0000 7f3f4bfff640 1 --2- 172.21.15.84:0/3952867339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f34004d00 0x7f3f4c0b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:17.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.238+0000 7f3f4b7fe640 1 --2- 172.21.15.84:0/3952867339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f4c155ac0 0x7f3f4c1552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:17.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.238+0000 7f3f5094d640 1 -- 172.21.15.84:0/3952867339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f34004d00 msgr2=0x7f3f4c0b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:17.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.238+0000 7f3f5094d640 1 --2- 172.21.15.84:0/3952867339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f34004d00 0x7f3f4c0b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.238+0000 7f3f5094d640 1 -- 172.21.15.84:0/3952867339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f4c155ac0 msgr2=0x7f3f4c1552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:17.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.238+0000 7f3f5094d640 1 --2- 172.21.15.84:0/3952867339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f4c155ac0 0x7f3f4c1552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.238+0000 7f3f5094d640 1 -- 172.21.15.84:0/3952867339 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c0b3350 con 0x7f3f4c15c910 2023-10-16T11:24:17.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.239+0000 7f3f5094d640 1 --2- 172.21.15.84:0/3952867339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f4c15c910 0x7f3f4c15cd90 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400c1d0 tx=0x7f3f440019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:17.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.239+0000 7f3f48ff9640 1 -- 172.21.15.84:0/3952867339 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4403c040 con 0x7f3f4c15c910 2023-10-16T11:24:17.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.239+0000 7f3f48ff9640 1 -- 172.21.15.84:0/3952867339 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4402cbd0 con 0x7f3f4c15c910 2023-10-16T11:24:17.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.239+0000 7f3f48ff9640 1 -- 172.21.15.84:0/3952867339 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44005aa0 con 0x7f3f4c15c910 2023-10-16T11:24:17.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.239+0000 7f3f523b0640 1 -- 172.21.15.84:0/3952867339 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4c0b3630 con 0x7f3f4c15c910 2023-10-16T11:24:17.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.239+0000 7f3f523b0640 1 -- 172.21.15.84:0/3952867339 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f4c0b3bf0 con 0x7f3f4c15c910 2023-10-16T11:24:17.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.240+0000 7f3f523b0640 1 -- 172.21.15.84:0/3952867339 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4c04e160 con 0x7f3f4c15c910 2023-10-16T11:24:17.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.241+0000 7f3f48ff9640 1 -- 172.21.15.84:0/3952867339 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3f44032070 con 0x7f3f4c15c910 2023-10-16T11:24:17.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.242+0000 7f3f48ff9640 1 --2- 172.21.15.84:0/3952867339 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3f3806af90 0x7f3f3806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:17.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.242+0000 7f3f48ff9640 1 -- 172.21.15.84:0/3952867339 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5773+0+0 (secure 0 0 0) 0x7f3f440aa180 con 0x7f3f4c15c910 2023-10-16T11:24:17.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.242+0000 7f3f4bfff640 1 --2- 172.21.15.84:0/3952867339 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3f3806af90 0x7f3f3806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:17.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.243+0000 7f3f4bfff640 1 --2- 172.21.15.84:0/3952867339 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3f3806af90 0x7f3f3806d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009b10 tx=0x7f3f3c008040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:17.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.246+0000 7f3f48ff9640 1 -- 172.21.15.84:0/3952867339 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3f440750e0 con 0x7f3f4c15c910 2023-10-16T11:24:17.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.440+0000 7f3f523b0640 1 -- 172.21.15.84:0/3952867339 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f4c0b10b0 con 0x7f3f4c15c910 2023-10-16T11:24:17.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.442+0000 7f3f48ff9640 1 -- 172.21.15.84:0/3952867339 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v209) v1 ==== 74+0+14181 (secure 0 0 0) 0x7f3f44078700 con 0x7f3f4c15c910 2023-10-16T11:24:17.443 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:17.443 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":209,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:24:14.816950+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6835/2679249765":"2023-10-17T11:24:11.852675+0000","172.21.15.84:0/65166914":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/26641613":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/1473149491":"2023-10-17T11:24:08.217257+0000","172.21.15.84:6837/2679249765":"2023-10-17T11:24:11.852675+0000","172.21.15.84:0/3116175047":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/4203014950":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/1585465362":"2023-10-17T11:24:08.217257+0000","172.21.15.84:6832/855077515":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/3564300528":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/2349522146":"2023-10-17T11:22:56.900947+0000","172.21.15.84:6833/855077515":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/2191803332":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/2199276620":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/3418988137":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/4262224959":"2023-10-17T11:22:56.900947+0000","172.21.15.84:0/1032287219":"2023-10-17T11:24:08.217257+0000","172.21.15.84:0/956990500":"2023-10-17T11:22:56.900947+0000","172.21.15.84:6833/1985463812":"2023-10-17T11:22:56.900947+0000","172.21.15.84:6832/1985463812":"2023-10-17T11:22:56.900947+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":25,"snaps":[{"begin":2,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:24:17.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.445+0000 7f3f523b0640 1 -- 172.21.15.84:0/3952867339 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3f3806af90 msgr2=0x7f3f3806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:17.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.445+0000 7f3f523b0640 1 --2- 172.21.15.84:0/3952867339 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3f3806af90 0x7f3f3806d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009b10 tx=0x7f3f3c008040 comp rx=0 tx=0).stop 2023-10-16T11:24:17.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.445+0000 7f3f523b0640 1 -- 172.21.15.84:0/3952867339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f4c15c910 msgr2=0x7f3f4c15cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:17.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.445+0000 7f3f523b0640 1 --2- 172.21.15.84:0/3952867339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f4c15c910 0x7f3f4c15cd90 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400c1d0 tx=0x7f3f440019a0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.445+0000 7f3f523b0640 1 -- 172.21.15.84:0/3952867339 shutdown_connections 2023-10-16T11:24:17.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.445+0000 7f3f523b0640 1 --2- 172.21.15.84:0/3952867339 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3f3806af90 0x7f3f3806d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.445+0000 7f3f523b0640 1 --2- 172.21.15.84:0/3952867339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f4c15c910 0x7f3f4c15cd90 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.445+0000 7f3f523b0640 1 --2- 172.21.15.84:0/3952867339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f4c155ac0 0x7f3f4c1552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.445+0000 7f3f523b0640 1 --2- 172.21.15.84:0/3952867339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3f34004d00 0x7f3f4c0b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.445+0000 7f3f523b0640 1 -- 172.21.15.84:0/3952867339 >> 172.21.15.84:0/3952867339 conn(0x7f3f4c0cb410 msgr2=0x7f3f4c0cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:17.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.445+0000 7f3f523b0640 1 -- 172.21.15.84:0/3952867339 shutdown_connections 2023-10-16T11:24:17.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.445+0000 7f3f523b0640 1 -- 172.21.15.84:0/3952867339 wait complete. 2023-10-16T11:24:17.465 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:24:17.466 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:24:17.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.685+0000 7f5be7b1d640 1 Processor -- start 2023-10-16T11:24:17.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.685+0000 7f5be7b1d640 1 -- start start 2023-10-16T11:24:17.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.687+0000 7f5be7b1d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be0159ab0 0x7f5be00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:17.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.687+0000 7f5be7b1d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5be006c140 0x7f5be006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:17.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.687+0000 7f5be7b1d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be0070360 con 0x7f5be0159e90 2023-10-16T11:24:17.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.687+0000 7f5be7b1d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be006cb10 con 0x7f5be006c140 2023-10-16T11:24:17.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.687+0000 7f5be7b1d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be015d890 con 0x7f5be0159ab0 2023-10-16T11:24:17.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.687+0000 7f5be60ba640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5be006c140 0x7f5be006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:17.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.687+0000 7f5be58b9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be0159ab0 0x7f5be00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:17.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.687+0000 7f5be60ba640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5be006c140 0x7f5be006c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:49962/0 (socket says 172.21.15.84:49962) 2023-10-16T11:24:17.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.687+0000 7f5be60ba640 1 -- 172.21.15.84:0/4118016897 learned_addr learned my addr 172.21.15.84:0/4118016897 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:17.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.688+0000 7f5be58b9640 1 -- 172.21.15.84:0/4118016897 >> v2:172.21.15.157:3300/0 conn(0x7f5be006c140 msgr2=0x7f5be006c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:17.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.688+0000 7f5be58b9640 1 --2- 172.21.15.84:0/4118016897 >> v2:172.21.15.157:3300/0 conn(0x7f5be006c140 0x7f5be006c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.688+0000 7f5be58b9640 1 -- 172.21.15.84:0/4118016897 >> v1:172.21.15.157:6789/0 conn(0x7f5be0159e90 legacy=0x7f5be006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:17.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.688+0000 7f5be58b9640 1 -- 172.21.15.84:0/4118016897 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be00cd1f0 con 0x7f5be0159ab0 2023-10-16T11:24:17.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.690+0000 7f5be58b9640 1 --2- 172.21.15.84:0/4118016897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be0159ab0 0x7f5be00ce370 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0097d0 tx=0x7f5bdc02d9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96a47aa43650071e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:17.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.690+0000 7f5be48b7640 1 -- 172.21.15.84:0/4118016897 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc032860 con 0x7f5be0159ab0 2023-10-16T11:24:17.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.690+0000 7f5be48b7640 1 -- 172.21.15.84:0/4118016897 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bdc0329c0 con 0x7f5be0159ab0 2023-10-16T11:24:17.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.690+0000 7f5be48b7640 1 -- 172.21.15.84:0/4118016897 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc032d50 con 0x7f5be0159ab0 2023-10-16T11:24:17.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.691+0000 7f5be7b1d640 1 -- 172.21.15.84:0/4118016897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be0159ab0 msgr2=0x7f5be00ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:17.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.691+0000 7f5be7b1d640 1 --2- 172.21.15.84:0/4118016897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be0159ab0 0x7f5be00ce370 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0097d0 tx=0x7f5bdc02d9a0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.691+0000 7f5be7b1d640 1 -- 172.21.15.84:0/4118016897 shutdown_connections 2023-10-16T11:24:17.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.691+0000 7f5be7b1d640 1 --2- 172.21.15.84:0/4118016897 >> v2:172.21.15.157:3300/0 conn(0x7f5be006c140 0x7f5be006c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.691+0000 7f5be7b1d640 1 --2- 172.21.15.84:0/4118016897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be0159ab0 0x7f5be00ce370 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.691+0000 7f5be7b1d640 1 -- 172.21.15.84:0/4118016897 >> 172.21.15.84:0/4118016897 conn(0x7f5be00ce9b0 msgr2=0x7f5be00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:17.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.691+0000 7f5be7b1d640 1 -- 172.21.15.84:0/4118016897 shutdown_connections 2023-10-16T11:24:17.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.691+0000 7f5be7b1d640 1 -- 172.21.15.84:0/4118016897 wait complete. 2023-10-16T11:24:17.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.693+0000 7f5be7b1d640 1 Processor -- start 2023-10-16T11:24:17.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.693+0000 7f5be7b1d640 1 -- start start 2023-10-16T11:24:17.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.694+0000 7f5be7b1d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be006c140 0x7f5be0158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:17.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.694+0000 7f5be7b1d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be0159ab0 0x7f5be0069da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:17.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.694+0000 7f5be7b1d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be0159e90 0x7f5be0157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:17.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.694+0000 7f5be7b1d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be01581e0 con 0x7f5be0159ab0 2023-10-16T11:24:17.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.694+0000 7f5be7b1d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be015a3d0 con 0x7f5be0159e90 2023-10-16T11:24:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.694+0000 7f5be7b1d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be006a430 con 0x7f5be006c140 2023-10-16T11:24:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.694+0000 7f5be50b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be0159ab0 0x7f5be0069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.694+0000 7f5be50b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be0159ab0 0x7f5be0069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40198/0 (socket says 172.21.15.84:40198) 2023-10-16T11:24:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.694+0000 7f5be50b8640 1 -- 172.21.15.84:0/1993286004 learned_addr learned my addr 172.21.15.84:0/1993286004 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.694+0000 7f5be58b9640 1 --2- 172.21.15.84:0/1993286004 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be006c140 0x7f5be0158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.694+0000 7f5be60ba640 1 --2- 172.21.15.84:0/1993286004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be0159e90 0x7f5be0157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.694+0000 7f5be50b8640 1 -- 172.21.15.84:0/1993286004 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be006c140 msgr2=0x7f5be0158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.694+0000 7f5be50b8640 1 --2- 172.21.15.84:0/1993286004 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be006c140 0x7f5be0158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.694+0000 7f5be50b8640 1 -- 172.21.15.84:0/1993286004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be0159e90 msgr2=0x7f5be0157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.694+0000 7f5be50b8640 1 --2- 172.21.15.84:0/1993286004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be0159e90 0x7f5be0157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.694+0000 7f5be50b8640 1 -- 172.21.15.84:0/1993286004 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc009400 con 0x7f5be0159ab0 2023-10-16T11:24:17.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.694+0000 7f5be60ba640 1 --2- 172.21.15.84:0/1993286004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be0159e90 0x7f5be0157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:17.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.695+0000 7f5be50b8640 1 --2- 172.21.15.84:0/1993286004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be0159ab0 0x7f5be0069da0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000c450 tx=0x7f5bd000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:17.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.695+0000 7f5bd67fc640 1 -- 172.21.15.84:0/1993286004 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd00052d0 con 0x7f5be0159ab0 2023-10-16T11:24:17.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.695+0000 7f5bd67fc640 1 -- 172.21.15.84:0/1993286004 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd0005430 con 0x7f5be0159ab0 2023-10-16T11:24:17.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.695+0000 7f5be7b1d640 1 -- 172.21.15.84:0/1993286004 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be015a670 con 0x7f5be0159ab0 2023-10-16T11:24:17.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.695+0000 7f5bd67fc640 1 -- 172.21.15.84:0/1993286004 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0012870 con 0x7f5be0159ab0 2023-10-16T11:24:17.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.695+0000 7f5be7b1d640 1 -- 172.21.15.84:0/1993286004 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be006a710 con 0x7f5be0159ab0 2023-10-16T11:24:17.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.697+0000 7f5be7b1d640 1 -- 172.21.15.84:0/1993286004 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be015ab90 con 0x7f5be0159ab0 2023-10-16T11:24:17.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.698+0000 7f5bd67fc640 1 -- 172.21.15.84:0/1993286004 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5bd00129d0 con 0x7f5be0159ab0 2023-10-16T11:24:17.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.699+0000 7f5bd67fc640 1 --2- 172.21.15.84:0/1993286004 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f5bc406af90 0x7f5bc406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:17.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.699+0000 7f5bd67fc640 1 -- 172.21.15.84:0/1993286004 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5773+0+0 (secure 0 0 0) 0x7f5bd0089400 con 0x7f5be0159ab0 2023-10-16T11:24:17.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.699+0000 7f5be58b9640 1 --2- 172.21.15.84:0/1993286004 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f5bc406af90 0x7f5bc406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:17.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.700+0000 7f5be58b9640 1 --2- 172.21.15.84:0/1993286004 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f5bc406af90 0x7f5bc406d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc038000 tx=0x7f5bdc03d040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:17.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.703+0000 7f5bd67fc640 1 -- 172.21.15.84:0/1993286004 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5be015ab90 con 0x7f5be0159ab0 2023-10-16T11:24:17.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.894+0000 7f5be7b1d640 1 -- 172.21.15.84:0/1993286004 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5be015ab90 con 0x7f5be0159ab0 2023-10-16T11:24:17.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.895+0000 7f5bd67fc640 1 -- 172.21.15.84:0/1993286004 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v209) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5be015ab90 con 0x7f5be0159ab0 2023-10-16T11:24:17.896 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:24:17.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.898+0000 7f5be7b1d640 1 -- 172.21.15.84:0/1993286004 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f5bc406af90 msgr2=0x7f5bc406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:17.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.898+0000 7f5be7b1d640 1 --2- 172.21.15.84:0/1993286004 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f5bc406af90 0x7f5bc406d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc038000 tx=0x7f5bdc03d040 comp rx=0 tx=0).stop 2023-10-16T11:24:17.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.898+0000 7f5be7b1d640 1 -- 172.21.15.84:0/1993286004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be0159ab0 msgr2=0x7f5be0069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:17.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.898+0000 7f5be7b1d640 1 --2- 172.21.15.84:0/1993286004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be0159ab0 0x7f5be0069da0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000c450 tx=0x7f5bd000c910 comp rx=0 tx=0).stop 2023-10-16T11:24:17.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.898+0000 7f5be7b1d640 1 -- 172.21.15.84:0/1993286004 shutdown_connections 2023-10-16T11:24:17.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.898+0000 7f5be7b1d640 1 --2- 172.21.15.84:0/1993286004 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f5bc406af90 0x7f5bc406d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.898+0000 7f5be7b1d640 1 --2- 172.21.15.84:0/1993286004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be0159e90 0x7f5be0157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.898+0000 7f5be7b1d640 1 --2- 172.21.15.84:0/1993286004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be0159ab0 0x7f5be0069da0 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.898+0000 7f5be7b1d640 1 --2- 172.21.15.84:0/1993286004 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be006c140 0x7f5be0158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:17.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.898+0000 7f5be7b1d640 1 -- 172.21.15.84:0/1993286004 >> 172.21.15.84:0/1993286004 conn(0x7f5be00ce9b0 msgr2=0x7f5be00ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:17.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.898+0000 7f5be7b1d640 1 -- 172.21.15.84:0/1993286004 shutdown_connections 2023-10-16T11:24:17.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:17.899+0000 7f5be7b1d640 1 -- 172.21.15.84:0/1993286004 wait complete. 2023-10-16T11:24:17.917 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs' 2023-10-16T11:24:18.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.129+0000 7ff25fc61640 1 Processor -- start 2023-10-16T11:24:18.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.130+0000 7ff25fc61640 1 -- start start 2023-10-16T11:24:18.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.131+0000 7ff25fc61640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff258159f00 0x7ff2580b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:18.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.131+0000 7ff25fc61640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff25815d880 0x7ff2580b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:18.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.131+0000 7ff25fc61640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff25806f7f0 con 0x7ff258159b20 2023-10-16T11:24:18.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.131+0000 7ff25fc61640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25806c5e0 con 0x7ff258159f00 2023-10-16T11:24:18.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.131+0000 7ff25fc61640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25806c8c0 con 0x7ff25815d880 2023-10-16T11:24:18.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.131+0000 7ff25d9fd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff258159b20 0x7ff25806bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58746/0 (socket says 172.21.15.84:58746) 2023-10-16T11:24:18.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.131+0000 7ff25d9fd640 1 -- 172.21.15.84:0/1168754642 learned_addr learned my addr 172.21.15.84:0/1168754642 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:18.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.131+0000 7ff25d1fc640 1 --2- 172.21.15.84:0/1168754642 >> v2:172.21.15.84:3300/0 conn(0x7ff258159f00 0x7ff2580b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:18.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.132+0000 7ff25e1fe640 1 --2- 172.21.15.84:0/1168754642 >> v2:172.21.15.157:3300/0 conn(0x7ff25815d880 0x7ff2580b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:18.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.132+0000 7ff25c9fb640 1 -- 172.21.15.84:0/1168754642 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3835831843 0 0) 0x7ff25806f7f0 con 0x7ff258159b20 2023-10-16T11:24:18.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.132+0000 7ff25d1fc640 1 -- 172.21.15.84:0/1168754642 >> v2:172.21.15.157:3300/0 conn(0x7ff25815d880 msgr2=0x7ff2580b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:18.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.132+0000 7ff25d1fc640 1 --2- 172.21.15.84:0/1168754642 >> v2:172.21.15.157:3300/0 conn(0x7ff25815d880 0x7ff2580b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:18.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.132+0000 7ff25d1fc640 1 -- 172.21.15.84:0/1168754642 >> v1:172.21.15.84:6789/0 conn(0x7ff258159b20 legacy=0x7ff25806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:18.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.132+0000 7ff25d1fc640 1 -- 172.21.15.84:0/1168754642 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25806cba0 con 0x7ff258159f00 2023-10-16T11:24:18.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.132+0000 7ff25e1fe640 1 --2- 172.21.15.84:0/1168754642 >> v2:172.21.15.157:3300/0 conn(0x7ff25815d880 0x7ff2580b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:18.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.133+0000 7ff25d1fc640 1 --2- 172.21.15.84:0/1168754642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff258159f00 0x7ff2580b0410 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7ff2580c8bb0 tx=0x7ff24802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcecbe4d01eb75 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:18.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.133+0000 7ff25c9fb640 1 -- 172.21.15.84:0/1168754642 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff248031830 con 0x7ff258159f00 2023-10-16T11:24:18.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.133+0000 7ff25c9fb640 1 -- 172.21.15.84:0/1168754642 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff248031990 con 0x7ff258159f00 2023-10-16T11:24:18.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.133+0000 7ff25c9fb640 1 -- 172.21.15.84:0/1168754642 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff248031d20 con 0x7ff258159f00 2023-10-16T11:24:18.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.134+0000 7ff25fc61640 1 -- 172.21.15.84:0/1168754642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff258159f00 msgr2=0x7ff2580b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:18.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.134+0000 7ff25fc61640 1 --2- 172.21.15.84:0/1168754642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff258159f00 0x7ff2580b0410 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7ff2580c8bb0 tx=0x7ff24802b8d0 comp rx=0 tx=0).stop 2023-10-16T11:24:18.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.134+0000 7ff25fc61640 1 -- 172.21.15.84:0/1168754642 shutdown_connections 2023-10-16T11:24:18.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.134+0000 7ff25fc61640 1 --2- 172.21.15.84:0/1168754642 >> v2:172.21.15.157:3300/0 conn(0x7ff25815d880 0x7ff2580b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:18.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.134+0000 7ff25fc61640 1 --2- 172.21.15.84:0/1168754642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff258159f00 0x7ff2580b0410 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:18.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.134+0000 7ff25fc61640 1 -- 172.21.15.84:0/1168754642 >> 172.21.15.84:0/1168754642 conn(0x7ff2580cb410 msgr2=0x7ff2580cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:18.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.134+0000 7ff25fc61640 1 -- 172.21.15.84:0/1168754642 shutdown_connections 2023-10-16T11:24:18.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.134+0000 7ff25fc61640 1 -- 172.21.15.84:0/1168754642 wait complete. 2023-10-16T11:24:18.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.136+0000 7ff25fc61640 1 Processor -- start 2023-10-16T11:24:18.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.136+0000 7ff25fc61640 1 -- start start 2023-10-16T11:24:18.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.137+0000 7ff25fc61640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff258159b20 0x7ff258158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:18.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.137+0000 7ff25fc61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff258159f00 0x7ff25815a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:18.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.137+0000 7ff25fc61640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25815d880 0x7ff2580acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:18.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.137+0000 7ff25fc61640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25815bae0 con 0x7ff25815d880 2023-10-16T11:24:18.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.137+0000 7ff25fc61640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2581594a0 con 0x7ff258159b20 2023-10-16T11:24:18.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.137+0000 7ff25fc61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25815abc0 con 0x7ff258159f00 2023-10-16T11:24:18.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.137+0000 7ff25e1fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25815d880 0x7ff2580acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:18.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.137+0000 7ff25e1fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25815d880 0x7ff2580acdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40212/0 (socket says 172.21.15.84:40212) 2023-10-16T11:24:18.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.137+0000 7ff25e1fe640 1 -- 172.21.15.84:0/696085500 learned_addr learned my addr 172.21.15.84:0/696085500 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:18.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.137+0000 7ff25d9fd640 1 --2- 172.21.15.84:0/696085500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff258159b20 0x7ff258158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:18.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.138+0000 7ff25e1fe640 1 -- 172.21.15.84:0/696085500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff258159f00 msgr2=0x7ff25815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:18.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.138+0000 7ff25e1fe640 1 --2- 172.21.15.84:0/696085500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff258159f00 0x7ff25815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:18.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.138+0000 7ff25e1fe640 1 -- 172.21.15.84:0/696085500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff258159b20 msgr2=0x7ff258158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:18.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.138+0000 7ff25e1fe640 1 --2- 172.21.15.84:0/696085500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff258159b20 0x7ff258158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:18.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.138+0000 7ff25e1fe640 1 -- 172.21.15.84:0/696085500 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff248009400 con 0x7ff25815d880 2023-10-16T11:24:18.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.138+0000 7ff25e1fe640 1 --2- 172.21.15.84:0/696085500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25815d880 0x7ff2580acdc0 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7ff25000cf20 tx=0x7ff25000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:18.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.138+0000 7ff24e7fc640 1 -- 172.21.15.84:0/696085500 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25000d840 con 0x7ff25815d880 2023-10-16T11:24:18.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.138+0000 7ff24e7fc640 1 -- 172.21.15.84:0/696085500 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25000d9a0 con 0x7ff25815d880 2023-10-16T11:24:18.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.138+0000 7ff25fc61640 1 -- 172.21.15.84:0/696085500 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff25815ae60 con 0x7ff25815d880 2023-10-16T11:24:18.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.138+0000 7ff24e7fc640 1 -- 172.21.15.84:0/696085500 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25000dd50 con 0x7ff25815d880 2023-10-16T11:24:18.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.138+0000 7ff25fc61640 1 -- 172.21.15.84:0/696085500 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25815b420 con 0x7ff25815d880 2023-10-16T11:24:18.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.139+0000 7ff25fc61640 1 -- 172.21.15.84:0/696085500 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25815bc20 con 0x7ff25815d880 2023-10-16T11:24:18.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.140+0000 7ff24e7fc640 1 -- 172.21.15.84:0/696085500 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff25001e3b0 con 0x7ff25815d880 2023-10-16T11:24:18.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.142+0000 7ff24e7fc640 1 --2- 172.21.15.84:0/696085500 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff23c06af90 0x7ff23c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:18.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.142+0000 7ff24e7fc640 1 -- 172.21.15.84:0/696085500 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5773+0+0 (secure 0 0 0) 0x7ff25008a2b0 con 0x7ff25815d880 2023-10-16T11:24:18.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.142+0000 7ff25d9fd640 1 --2- 172.21.15.84:0/696085500 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff23c06af90 0x7ff23c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:18.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.143+0000 7ff25d9fd640 1 --2- 172.21.15.84:0/696085500 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff23c06af90 0x7ff23c06d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff254000f60 tx=0x7ff254009040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:18.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.146+0000 7ff24e7fc640 1 -- 172.21.15.84:0/696085500 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff25815bc20 con 0x7ff25815d880 2023-10-16T11:24:18.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.390+0000 7ff25fc61640 1 -- 172.21.15.84:0/696085500 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]} v 0) v1 -- 0x7ff25815bc20 con 0x7ff25815d880 2023-10-16T11:24:18.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.990+0000 7ff24e7fc640 1 -- 172.21.15.84:0/696085500 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]}]=0 v3111) v1 ==== 189+0+0 (secure 0 0 0) 0x7ff25815bc20 con 0x7ff25815d880 2023-10-16T11:24:18.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.993+0000 7ff25fc61640 1 -- 172.21.15.84:0/696085500 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff23c06af90 msgr2=0x7ff23c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:18.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.993+0000 7ff25fc61640 1 --2- 172.21.15.84:0/696085500 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff23c06af90 0x7ff23c06d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff254000f60 tx=0x7ff254009040 comp rx=0 tx=0).stop 2023-10-16T11:24:18.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.993+0000 7ff25fc61640 1 -- 172.21.15.84:0/696085500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25815d880 msgr2=0x7ff2580acdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:18.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.993+0000 7ff25fc61640 1 --2- 172.21.15.84:0/696085500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25815d880 0x7ff2580acdc0 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7ff25000cf20 tx=0x7ff25000a7d0 comp rx=0 tx=0).stop 2023-10-16T11:24:18.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.993+0000 7ff25fc61640 1 -- 172.21.15.84:0/696085500 shutdown_connections 2023-10-16T11:24:18.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.993+0000 7ff25fc61640 1 --2- 172.21.15.84:0/696085500 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff23c06af90 0x7ff23c06d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:18.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.993+0000 7ff25fc61640 1 --2- 172.21.15.84:0/696085500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25815d880 0x7ff2580acdc0 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:18.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.993+0000 7ff25fc61640 1 --2- 172.21.15.84:0/696085500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff258159f00 0x7ff25815a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:18.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.993+0000 7ff25fc61640 1 --2- 172.21.15.84:0/696085500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff258159b20 0x7ff258158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:18.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.994+0000 7ff25fc61640 1 -- 172.21.15.84:0/696085500 >> 172.21.15.84:0/696085500 conn(0x7ff2580cb410 msgr2=0x7ff2580ca070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:18.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.994+0000 7ff25fc61640 1 -- 172.21.15.84:0/696085500 shutdown_connections 2023-10-16T11:24:18.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:18.994+0000 7ff25fc61640 1 -- 172.21.15.84:0/696085500 wait complete. 2023-10-16T11:24:19.015 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T11:24:19.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.229+0000 7f1589aa2640 1 Processor -- start 2023-10-16T11:24:19.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.230+0000 7f1589aa2640 1 -- start start 2023-10-16T11:24:19.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.231+0000 7f1589aa2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15840b3420 0x7f15840b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:19.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.231+0000 7f1589aa2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15840d3980 con 0x7f15840cfaf0 2023-10-16T11:24:19.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.231+0000 7f1589aa2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15840ce630 con 0x7f15840cf710 2023-10-16T11:24:19.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.231+0000 7f1589aa2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15840d3c60 con 0x7f15840b3420 2023-10-16T11:24:19.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.232+0000 7f1583fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f15840cfaf0 0x7f15840b6200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58754/0 (socket says 172.21.15.84:58754) 2023-10-16T11:24:19.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.232+0000 7f1583fff640 1 -- 172.21.15.84:0/2856582256 learned_addr learned my addr 172.21.15.84:0/2856582256 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:19.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.232+0000 7f15827fc640 1 -- 172.21.15.84:0/2856582256 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318425794 0 0) 0x7f15840d3980 con 0x7f15840cfaf0 2023-10-16T11:24:19.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.232+0000 7f15827fc640 1 -- 172.21.15.84:0/2856582256 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1570003520 con 0x7f15840cfaf0 2023-10-16T11:24:19.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.232+0000 7f1582ffd640 1 --2- 172.21.15.84:0/2856582256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15840b3420 0x7f15840b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:19.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.232+0000 7f15827fc640 1 -- 172.21.15.84:0/2856582256 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3012318830 0 0) 0x7f1570003520 con 0x7f15840cfaf0 2023-10-16T11:24:19.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.232+0000 7f15827fc640 1 -- 172.21.15.84:0/2856582256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15840b3420 msgr2=0x7f15840b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:19.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.232+0000 7f15827fc640 1 --2- 172.21.15.84:0/2856582256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15840b3420 0x7f15840b5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:19.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.232+0000 7f15827fc640 1 -- 172.21.15.84:0/2856582256 >> v1:172.21.15.157:6789/0 conn(0x7f15840cf710 legacy=0x7f15840b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:19.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.232+0000 7f15827fc640 1 -- 172.21.15.84:0/2856582256 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15840ce910 con 0x7f15840cfaf0 2023-10-16T11:24:19.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.232+0000 7f15827fc640 1 -- 172.21.15.84:0/2856582256 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f157c001670 con 0x7f15840cfaf0 2023-10-16T11:24:19.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.232+0000 7f15827fc640 1 -- 172.21.15.84:0/2856582256 >> v1:172.21.15.84:6789/0 conn(0x7f15840cfaf0 legacy=0x7f15840b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:19.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.232+0000 7f15827fc640 1 --2- 172.21.15.84:0/2856582256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15700039a0 0x7f1570003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:19.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.232+0000 7f15827fc640 1 --2- 172.21.15.84:0/2856582256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15840b3420 0x7f15700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:19.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.232+0000 7f15827fc640 1 --2- 172.21.15.84:0/2856582256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1570004d00 0x7f1570005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:19.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.233+0000 7f15827fc640 1 -- 172.21.15.84:0/2856582256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15840d3c60 con 0x7f1570004d00 2023-10-16T11:24:19.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.233+0000 7f15827fc640 1 -- 172.21.15.84:0/2856582256 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157000bb60 con 0x7f15700039a0 2023-10-16T11:24:19.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.233+0000 7f15827fc640 1 -- 172.21.15.84:0/2856582256 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157000bd20 con 0x7f15840b3420 2023-10-16T11:24:19.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.233+0000 7f1583fff640 1 --2- 172.21.15.84:0/2856582256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1570004d00 0x7f1570005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:19.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.233+0000 7f15837fe640 1 --2- 172.21.15.84:0/2856582256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15700039a0 0x7f1570003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:19.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.233+0000 7f1582ffd640 1 --2- 172.21.15.84:0/2856582256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15840b3420 0x7f15700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:19.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.233+0000 7f1583fff640 1 -- 172.21.15.84:0/2856582256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15840b3420 msgr2=0x7f15700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:19.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.233+0000 7f1583fff640 1 --2- 172.21.15.84:0/2856582256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15840b3420 0x7f15700047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:19.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.233+0000 7f1583fff640 1 -- 172.21.15.84:0/2856582256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15700039a0 msgr2=0x7f1570003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:19.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.233+0000 7f1583fff640 1 --2- 172.21.15.84:0/2856582256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15700039a0 0x7f1570003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:19.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.233+0000 7f1583fff640 1 -- 172.21.15.84:0/2856582256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1570003520 con 0x7f1570004d00 2023-10-16T11:24:19.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.234+0000 7f1583fff640 1 --2- 172.21.15.84:0/2856582256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1570004d00 0x7f1570005180 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7f157c00c190 tx=0x7f15700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323ee03132e69800 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:19.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.235+0000 7f15827fc640 1 -- 172.21.15.84:0/2856582256 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c005ad0 con 0x7f1570004d00 2023-10-16T11:24:19.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.235+0000 7f15827fc640 1 -- 172.21.15.84:0/2856582256 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c005c30 con 0x7f1570004d00 2023-10-16T11:24:19.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.235+0000 7f15827fc640 1 -- 172.21.15.84:0/2856582256 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c02c8a0 con 0x7f1570004d00 2023-10-16T11:24:19.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.235+0000 7f1589aa2640 1 -- 172.21.15.84:0/2856582256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1570004d00 msgr2=0x7f1570005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:19.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.235+0000 7f1589aa2640 1 --2- 172.21.15.84:0/2856582256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1570004d00 0x7f1570005180 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7f157c00c190 tx=0x7f15700057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:19.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.235+0000 7f1583fff640 1 -- 172.21.15.84:0/2856582256 reap_dead start 2023-10-16T11:24:19.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.235+0000 7f1589aa2640 1 -- 172.21.15.84:0/2856582256 shutdown_connections 2023-10-16T11:24:19.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.235+0000 7f1589aa2640 1 -- 172.21.15.84:0/2856582256 >> 172.21.15.84:0/2856582256 conn(0x7f15840d2cc0 msgr2=0x7f15840d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:19.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.235+0000 7f1589aa2640 1 -- 172.21.15.84:0/2856582256 shutdown_connections 2023-10-16T11:24:19.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.235+0000 7f1589aa2640 1 -- 172.21.15.84:0/2856582256 wait complete. 2023-10-16T11:24:19.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.237+0000 7f1589aa2640 1 Processor -- start 2023-10-16T11:24:19.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.237+0000 7f1589aa2640 1 -- start start 2023-10-16T11:24:19.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.238+0000 7f1589aa2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1570004d00 0x7f158406b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:19.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.238+0000 7f1589aa2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15840b1260 0x7f15841f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:19.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.238+0000 7f1589aa2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15840b16e0 0x7f15841f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:19.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.238+0000 7f1589aa2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15840ca110 con 0x7f1570004d00 2023-10-16T11:24:19.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.238+0000 7f1589aa2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15841f6d90 con 0x7f15840b16e0 2023-10-16T11:24:19.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.238+0000 7f1589aa2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15841f6ab0 con 0x7f15840b1260 2023-10-16T11:24:19.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.238+0000 7f15837fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1570004d00 0x7f158406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:19.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.239+0000 7f15837fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1570004d00 0x7f158406b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40240/0 (socket says 172.21.15.84:40240) 2023-10-16T11:24:19.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.239+0000 7f15837fe640 1 -- 172.21.15.84:0/3553395940 learned_addr learned my addr 172.21.15.84:0/3553395940 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:19.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.239+0000 7f15837fe640 1 -- 172.21.15.84:0/3553395940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15840b1260 msgr2=0x7f15841f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:19.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.239+0000 7f15837fe640 1 --2- 172.21.15.84:0/3553395940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15840b1260 0x7f15841f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:19.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.239+0000 7f15837fe640 1 -- 172.21.15.84:0/3553395940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15840b16e0 msgr2=0x7f15841f63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:19.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.239+0000 7f15837fe640 1 --2- 172.21.15.84:0/3553395940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15840b16e0 0x7f15841f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:19.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.239+0000 7f15837fe640 1 -- 172.21.15.84:0/3553395940 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15840ca3f0 con 0x7f1570004d00 2023-10-16T11:24:19.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.239+0000 7f15837fe640 1 --2- 172.21.15.84:0/3553395940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1570004d00 0x7f158406b900 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f157400e8f0 tx=0x7f157400edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:19.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.239+0000 7f157bfff640 1 -- 172.21.15.84:0/3553395940 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1574003bd0 con 0x7f1570004d00 2023-10-16T11:24:19.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.240+0000 7f157bfff640 1 -- 172.21.15.84:0/3553395940 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1574003d30 con 0x7f1570004d00 2023-10-16T11:24:19.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.240+0000 7f1589aa2640 1 -- 172.21.15.84:0/3553395940 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15840ca6d0 con 0x7f1570004d00 2023-10-16T11:24:19.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.240+0000 7f157bfff640 1 -- 172.21.15.84:0/3553395940 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1574003ea0 con 0x7f1570004d00 2023-10-16T11:24:19.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.240+0000 7f1589aa2640 1 -- 172.21.15.84:0/3553395940 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158406be40 con 0x7f1570004d00 2023-10-16T11:24:19.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.241+0000 7f1589aa2640 1 -- 172.21.15.84:0/3553395940 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158406bfd0 con 0x7f1570004d00 2023-10-16T11:24:19.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.242+0000 7f157bfff640 1 -- 172.21.15.84:0/3553395940 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f15740052f0 con 0x7f1570004d00 2023-10-16T11:24:19.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.243+0000 7f157bfff640 1 --2- 172.21.15.84:0/3553395940 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f156406af90 0x7f156406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:19.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.243+0000 7f157bfff640 1 -- 172.21.15.84:0/3553395940 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5773+0+0 (secure 0 0 0) 0x7f15740898f0 con 0x7f1570004d00 2023-10-16T11:24:19.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.243+0000 7f1582ffd640 1 --2- 172.21.15.84:0/3553395940 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f156406af90 0x7f156406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:19.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.244+0000 7f1582ffd640 1 --2- 172.21.15.84:0/3553395940 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f156406af90 0x7f156406d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f156c006a90 tx=0x7f156c006a20 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:19.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.248+0000 7f157bfff640 1 -- 172.21.15.84:0/3553395940 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1574054840 con 0x7f1570004d00 2023-10-16T11:24:19.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.443+0000 7f1589aa2640 1 -- 172.21.15.84:0/3553395940 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f158404e160 con 0x7f1570004d00 2023-10-16T11:24:19.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.449+0000 7f157bfff640 1 -- 172.21.15.84:0/3553395940 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v76)=0 v76) v1 ==== 122+0+0 (secure 0 0 0) 0x7f157401a310 con 0x7f1570004d00 2023-10-16T11:24:19.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.452+0000 7f1589aa2640 1 -- 172.21.15.84:0/3553395940 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f156406af90 msgr2=0x7f156406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:19.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.452+0000 7f1589aa2640 1 --2- 172.21.15.84:0/3553395940 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f156406af90 0x7f156406d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f156c006a90 tx=0x7f156c006a20 comp rx=0 tx=0).stop 2023-10-16T11:24:19.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.452+0000 7f1589aa2640 1 -- 172.21.15.84:0/3553395940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1570004d00 msgr2=0x7f158406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:19.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.452+0000 7f1589aa2640 1 --2- 172.21.15.84:0/3553395940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1570004d00 0x7f158406b900 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f157400e8f0 tx=0x7f157400edb0 comp rx=0 tx=0).stop 2023-10-16T11:24:19.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.452+0000 7f1589aa2640 1 -- 172.21.15.84:0/3553395940 shutdown_connections 2023-10-16T11:24:19.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.452+0000 7f1589aa2640 1 --2- 172.21.15.84:0/3553395940 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f156406af90 0x7f156406d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:19.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.452+0000 7f1589aa2640 1 --2- 172.21.15.84:0/3553395940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15840b16e0 0x7f15841f63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:19.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.452+0000 7f1589aa2640 1 --2- 172.21.15.84:0/3553395940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15840b1260 0x7f15841f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:19.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.452+0000 7f1589aa2640 1 --2- 172.21.15.84:0/3553395940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1570004d00 0x7f158406b900 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:19.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.452+0000 7f1589aa2640 1 -- 172.21.15.84:0/3553395940 >> 172.21.15.84:0/3553395940 conn(0x7f15840d2cc0 msgr2=0x7f15840cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:19.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.453+0000 7f1589aa2640 1 -- 172.21.15.84:0/3553395940 shutdown_connections 2023-10-16T11:24:19.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:19.453+0000 7f1589aa2640 1 -- 172.21.15.84:0/3553395940 wait complete. 2023-10-16T11:24:19.473 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:24:19.473 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:24:19.582 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:24:19.710 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:24:19.711 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:24:19.711 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:24:19.772 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:24:19.772 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:24:19.773 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:24:19.834 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:24:19.835 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:24:19.835 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:24:19.895 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:24:19.896 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:24:19.896 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:24:19.957 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:24:19.958 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T11:24:19.958 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:24:20.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.172+0000 7fb9d0db0640 1 Processor -- start 2023-10-16T11:24:20.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.172+0000 7fb9d0db0640 1 -- start start 2023-10-16T11:24:20.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.174+0000 7fb9d0db0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9cc15ae80 0x7fb9cc15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:20.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.174+0000 7fb9d0db0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9cc0ce390 0x7fb9cc070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:20.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.174+0000 7fb9d0db0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9cc15ed60 con 0x7fb9cc15aaa0 2023-10-16T11:24:20.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.174+0000 7fb9d0db0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9cc06b810 con 0x7fb9cc15ae80 2023-10-16T11:24:20.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.174+0000 7fb9d0db0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc0707e0 con 0x7fb9cc0ce390 2023-10-16T11:24:20.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.175+0000 7fb9c1d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9cc15ae80 0x7fb9cc15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:20.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.175+0000 7fb9cad9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9cc0ce390 0x7fb9cc070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:20.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.175+0000 7fb9c1d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9cc15ae80 0x7fb9cc15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:50024/0 (socket says 172.21.15.84:50024) 2023-10-16T11:24:20.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.175+0000 7fb9c1d9b640 1 -- 172.21.15.84:0/2619278526 learned_addr learned my addr 172.21.15.84:0/2619278526 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:20.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.175+0000 7fb9cad9d640 1 -- 172.21.15.84:0/2619278526 >> v2:172.21.15.157:3300/0 conn(0x7fb9cc15ae80 msgr2=0x7fb9cc15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:20.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.175+0000 7fb9cad9d640 1 --2- 172.21.15.84:0/2619278526 >> v2:172.21.15.157:3300/0 conn(0x7fb9cc15ae80 0x7fb9cc15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:20.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.175+0000 7fb9cad9d640 1 -- 172.21.15.84:0/2619278526 >> v1:172.21.15.157:6789/0 conn(0x7fb9cc15aaa0 legacy=0x7fb9cc0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:20.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.175+0000 7fb9cad9d640 1 -- 172.21.15.84:0/2619278526 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc15ef90 con 0x7fb9cc0ce390 2023-10-16T11:24:20.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.177+0000 7fb9cad9d640 1 --2- 172.21.15.84:0/2619278526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9cc0ce390 0x7fb9cc070f50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc0c59c0 tx=0x7fb9c402bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bedc0efdc74b2063 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:20.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.177+0000 7fb9c9d9b640 1 -- 172.21.15.84:0/2619278526 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c402cb90 con 0x7fb9cc0ce390 2023-10-16T11:24:20.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.177+0000 7fb9c9d9b640 1 -- 172.21.15.84:0/2619278526 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c402ccf0 con 0x7fb9cc0ce390 2023-10-16T11:24:20.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.177+0000 7fb9c9d9b640 1 -- 172.21.15.84:0/2619278526 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4034660 con 0x7fb9cc0ce390 2023-10-16T11:24:20.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.178+0000 7fb9d0db0640 1 -- 172.21.15.84:0/2619278526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9cc0ce390 msgr2=0x7fb9cc070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:20.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.178+0000 7fb9d0db0640 1 --2- 172.21.15.84:0/2619278526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9cc0ce390 0x7fb9cc070f50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc0c59c0 tx=0x7fb9c402bde0 comp rx=0 tx=0).stop 2023-10-16T11:24:20.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.178+0000 7fb9d0db0640 1 -- 172.21.15.84:0/2619278526 shutdown_connections 2023-10-16T11:24:20.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.178+0000 7fb9d0db0640 1 --2- 172.21.15.84:0/2619278526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9cc0ce390 0x7fb9cc070f50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:20.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.178+0000 7fb9d0db0640 1 --2- 172.21.15.84:0/2619278526 >> v2:172.21.15.157:3300/0 conn(0x7fb9cc15ae80 0x7fb9cc15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:20.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.178+0000 7fb9d0db0640 1 -- 172.21.15.84:0/2619278526 >> 172.21.15.84:0/2619278526 conn(0x7fb9cc0cb410 msgr2=0x7fb9cc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:20.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.178+0000 7fb9d0db0640 1 -- 172.21.15.84:0/2619278526 shutdown_connections 2023-10-16T11:24:20.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.178+0000 7fb9d0db0640 1 -- 172.21.15.84:0/2619278526 wait complete. 2023-10-16T11:24:20.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.180+0000 7fb9d0db0640 1 Processor -- start 2023-10-16T11:24:20.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.181+0000 7fb9d0db0640 1 -- start start 2023-10-16T11:24:20.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.182+0000 7fb9d0db0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9cc0ce390 0x7fb9cc0c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:20.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.182+0000 7fb9d0db0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9cc15aaa0 0x7fb9cc0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:20.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.182+0000 7fb9d0db0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9cc15ae80 0x7fb9cc1f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:20.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.182+0000 7fb9d0db0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc149680 con 0x7fb9cc0ce390 2023-10-16T11:24:20.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.182+0000 7fb9d0db0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc1f3d00 con 0x7fb9cc15ae80 2023-10-16T11:24:20.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.182+0000 7fb9d0db0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc0c5360 con 0x7fb9cc15aaa0 2023-10-16T11:24:20.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.182+0000 7fb9ca59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9cc0ce390 0x7fb9cc0c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:20.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.182+0000 7fb9ca59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9cc0ce390 0x7fb9cc0c4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40242/0 (socket says 172.21.15.84:40242) 2023-10-16T11:24:20.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.182+0000 7fb9ca59c640 1 -- 172.21.15.84:0/1521485 learned_addr learned my addr 172.21.15.84:0/1521485 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:20.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.183+0000 7fb9cad9d640 1 --2- 172.21.15.84:0/1521485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9cc15ae80 0x7fb9cc1f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:20.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.183+0000 7fb9ca59c640 1 -- 172.21.15.84:0/1521485 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9cc15aaa0 msgr2=0x7fb9cc0c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:20.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.183+0000 7fb9c1d9b640 1 --2- 172.21.15.84:0/1521485 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9cc15aaa0 0x7fb9cc0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:20.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.183+0000 7fb9ca59c640 1 --2- 172.21.15.84:0/1521485 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9cc15aaa0 0x7fb9cc0c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:20.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.183+0000 7fb9ca59c640 1 -- 172.21.15.84:0/1521485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9cc15ae80 msgr2=0x7fb9cc1f2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:20.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.183+0000 7fb9ca59c640 1 --2- 172.21.15.84:0/1521485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9cc15ae80 0x7fb9cc1f2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:20.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.183+0000 7fb9ca59c640 1 -- 172.21.15.84:0/1521485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c4009a70 con 0x7fb9cc0ce390 2023-10-16T11:24:20.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.183+0000 7fb9cad9d640 1 --2- 172.21.15.84:0/1521485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9cc15ae80 0x7fb9cc1f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:20.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.183+0000 7fb9ca59c640 1 --2- 172.21.15.84:0/1521485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9cc0ce390 0x7fb9cc0c4e20 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00c430 tx=0x7fb9bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:20.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.183+0000 7fb9c37fe640 1 -- 172.21.15.84:0/1521485 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc011790 con 0x7fb9cc0ce390 2023-10-16T11:24:20.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.183+0000 7fb9c37fe640 1 -- 172.21.15.84:0/1521485 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9bc0118f0 con 0x7fb9cc0ce390 2023-10-16T11:24:20.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.183+0000 7fb9c37fe640 1 -- 172.21.15.84:0/1521485 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc011ce0 con 0x7fb9cc0ce390 2023-10-16T11:24:20.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.183+0000 7fb9d0db0640 1 -- 172.21.15.84:0/1521485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9cc15d240 con 0x7fb9cc0ce390 2023-10-16T11:24:20.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.183+0000 7fb9d0db0640 1 -- 172.21.15.84:0/1521485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9cc15d800 con 0x7fb9cc0ce390 2023-10-16T11:24:20.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.185+0000 7fb9d0db0640 1 -- 172.21.15.84:0/1521485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9cc1f3e90 con 0x7fb9cc0ce390 2023-10-16T11:24:20.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.186+0000 7fb9c37fe640 1 -- 172.21.15.84:0/1521485 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb9bc01c460 con 0x7fb9cc0ce390 2023-10-16T11:24:20.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.186+0000 7fb9c37fe640 1 --2- 172.21.15.84:0/1521485 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fb9ac06af90 0x7fb9ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:20.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.187+0000 7fb9c37fe640 1 -- 172.21.15.84:0/1521485 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5773+0+0 (secure 0 0 0) 0x7fb9bc089790 con 0x7fb9cc0ce390 2023-10-16T11:24:20.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.187+0000 7fb9c1d9b640 1 --2- 172.21.15.84:0/1521485 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fb9ac06af90 0x7fb9ac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:20.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.187+0000 7fb9c1d9b640 1 --2- 172.21.15.84:0/1521485 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fb9ac06af90 0x7fb9ac06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc1f6090 tx=0x7fb9b4008040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:20.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.191+0000 7fb9c37fe640 1 -- 172.21.15.84:0/1521485 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb9bc0546f0 con 0x7fb9cc0ce390 2023-10-16T11:24:20.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.422+0000 7fb9d0db0640 1 -- 172.21.15.84:0/1521485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9cc04e160 con 0x7fb9cc0ce390 2023-10-16T11:24:20.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.423+0000 7fb9c37fe640 1 -- 172.21.15.84:0/1521485 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+3708 (secure 0 0 0) 0x7fb9bc057d10 con 0x7fb9cc0ce390 2023-10-16T11:24:20.424 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:20.424 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":107,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6575,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/855929374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":855929374},{"type":"v1","addr":"172.21.15.157:6834","nonce":855929374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6578,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2581460972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2581460972},{"type":"v1","addr":"172.21.15.157:6835","nonce":2581460972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":10806,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1731880648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1731880648},{"type":"v1","addr":"172.21.15.84:6836","nonce":1731880648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99},{"gid":11238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4239427736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":4239427736},{"type":"v1","addr":"172.21.15.84:6837","nonce":4239427736}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107}],"filesystems":[]} 2023-10-16T11:24:20.425 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 107 2023-10-16T11:24:20.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.425+0000 7fb9d0db0640 1 -- 172.21.15.84:0/1521485 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fb9ac06af90 msgr2=0x7fb9ac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:20.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.425+0000 7fb9d0db0640 1 --2- 172.21.15.84:0/1521485 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fb9ac06af90 0x7fb9ac06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc1f6090 tx=0x7fb9b4008040 comp rx=0 tx=0).stop 2023-10-16T11:24:20.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.425+0000 7fb9d0db0640 1 -- 172.21.15.84:0/1521485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9cc0ce390 msgr2=0x7fb9cc0c4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:20.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.425+0000 7fb9d0db0640 1 --2- 172.21.15.84:0/1521485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9cc0ce390 0x7fb9cc0c4e20 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00c430 tx=0x7fb9bc00c8f0 comp rx=0 tx=0).stop 2023-10-16T11:24:20.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.426+0000 7fb9d0db0640 1 -- 172.21.15.84:0/1521485 shutdown_connections 2023-10-16T11:24:20.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.426+0000 7fb9d0db0640 1 --2- 172.21.15.84:0/1521485 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fb9ac06af90 0x7fb9ac06d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:20.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.426+0000 7fb9d0db0640 1 --2- 172.21.15.84:0/1521485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9cc15ae80 0x7fb9cc1f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:20.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.426+0000 7fb9d0db0640 1 --2- 172.21.15.84:0/1521485 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9cc15aaa0 0x7fb9cc0c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:20.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.426+0000 7fb9d0db0640 1 --2- 172.21.15.84:0/1521485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9cc0ce390 0x7fb9cc0c4e20 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:20.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.426+0000 7fb9d0db0640 1 -- 172.21.15.84:0/1521485 >> 172.21.15.84:0/1521485 conn(0x7fb9cc0cb410 msgr2=0x7fb9cc0b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:20.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.426+0000 7fb9d0db0640 1 -- 172.21.15.84:0/1521485 shutdown_connections 2023-10-16T11:24:20.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.426+0000 7fb9d0db0640 1 -- 172.21.15.84:0/1521485 wait complete. 2023-10-16T11:24:20.446 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T11:24:20.446 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T11:24:20.447 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T11:24:20.447 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:24:20.447 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T11:24:20.447 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T11:24:20.448 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T11:24:20.448 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:24:20.448 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T11:24:20.449 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T11:24:20.449 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T11:24:20.449 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:24:20.450 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T11:24:20.450 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T11:24:20.450 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T11:24:20.450 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:24:20.463 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T11:24:20.463 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T11:24:20.466 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T11:24:20.467 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T11:24:20.473 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T11:24:20.473 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T11:24:20.476 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T11:24:20.477 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T11:24:20.514 INFO:tasks.ceph.mds.c:Started 2023-10-16T11:24:20.515 DEBUG:teuthology.parallel:result is None 2023-10-16T11:24:20.522 INFO:tasks.ceph.mds.a:Started 2023-10-16T11:24:20.522 DEBUG:teuthology.parallel:result is None 2023-10-16T11:24:20.524 INFO:tasks.ceph.mds.d:Started 2023-10-16T11:24:20.525 DEBUG:teuthology.parallel:result is None 2023-10-16T11:24:20.532 INFO:tasks.ceph.mds.b:Started 2023-10-16T11:24:20.532 DEBUG:teuthology.parallel:result is None 2023-10-16T11:24:20.533 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T11:24:20.674 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T11:24:20.675 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T11:24:20.695 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T11:24:20.695 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-10-16T11:24:20.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.719+0000 7f6ff20a2640 1 Processor -- start 2023-10-16T11:24:20.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.719+0000 7f6ff20a2640 1 -- start start 2023-10-16T11:24:20.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.721+0000 7f6ff20a2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6fec15ad70 0x7f6fec069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:20.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.721+0000 7f6ff20a2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fec0cb030 con 0x7f6fec0c9ca0 2023-10-16T11:24:20.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.721+0000 7f6ff20a2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fec0c91c0 con 0x7f6fec0caac0 2023-10-16T11:24:20.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.721+0000 7f6ff20a2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fec0c94a0 con 0x7f6fec15ad70 2023-10-16T11:24:20.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.721+0000 7f6feb7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6fec15ad70 0x7f6fec069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:20.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.721+0000 7f6feb7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6fec15ad70 0x7f6fec069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40282/0 (socket says 172.21.15.84:40282) 2023-10-16T11:24:20.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.721+0000 7f6feb7fe640 1 -- 172.21.15.84:0/1657565516 learned_addr learned my addr 172.21.15.84:0/1657565516 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:20.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.721+0000 7f6fe3fff640 1 --1- 172.21.15.84:0/1657565516 >> v1:172.21.15.84:6789/0 conn(0x7f6fec0c9ca0 0x7f6fec0ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58782/0 (socket says 172.21.15.84:58782) 2023-10-16T11:24:20.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.721+0000 7f6feb7fe640 1 -- 172.21.15.84:0/1657565516 >> v1:172.21.15.157:6789/0 conn(0x7f6fec0caac0 legacy=0x7f6fec0ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:20.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.721+0000 7f6feb7fe640 1 -- 172.21.15.84:0/1657565516 >> v1:172.21.15.84:6789/0 conn(0x7f6fec0c9ca0 legacy=0x7f6fec0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:20.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.721+0000 7f6feb7fe640 1 -- 172.21.15.84:0/1657565516 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec0c9780 con 0x7f6fec15ad70 2023-10-16T11:24:20.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.722+0000 7f6feb7fe640 1 --2- 172.21.15.84:0/1657565516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fec15ad70 0x7f6fec069da0 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc009870 tx=0x7f6fdc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ad635b2bfa01e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:20.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.723+0000 7f6feaffd640 1 -- 172.21.15.84:0/1657565516 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc031740 con 0x7f6fec15ad70 2023-10-16T11:24:20.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.723+0000 7f6feaffd640 1 -- 172.21.15.84:0/1657565516 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fdc0318a0 con 0x7f6fec15ad70 2023-10-16T11:24:20.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.723+0000 7f6feaffd640 1 -- 172.21.15.84:0/1657565516 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc031c30 con 0x7f6fec15ad70 2023-10-16T11:24:20.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.723+0000 7f6ff20a2640 1 -- 172.21.15.84:0/1657565516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fec15ad70 msgr2=0x7f6fec069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:20.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.723+0000 7f6ff20a2640 1 --2- 172.21.15.84:0/1657565516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fec15ad70 0x7f6fec069da0 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc009870 tx=0x7f6fdc02b860 comp rx=0 tx=0).stop 2023-10-16T11:24:20.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.723+0000 7f6ff20a2640 1 -- 172.21.15.84:0/1657565516 shutdown_connections 2023-10-16T11:24:20.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.723+0000 7f6ff20a2640 1 --2- 172.21.15.84:0/1657565516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fec15ad70 0x7f6fec069da0 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:20.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.723+0000 7f6ff20a2640 1 -- 172.21.15.84:0/1657565516 >> 172.21.15.84:0/1657565516 conn(0x7f6fec0ce9b0 msgr2=0x7f6fec0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:20.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.724+0000 7f6ff20a2640 1 -- 172.21.15.84:0/1657565516 shutdown_connections 2023-10-16T11:24:20.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.724+0000 7f6ff20a2640 1 -- 172.21.15.84:0/1657565516 wait complete. 2023-10-16T11:24:20.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.725+0000 7f6ff20a2640 1 Processor -- start 2023-10-16T11:24:20.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.725+0000 7f6ff20a2640 1 -- start start 2023-10-16T11:24:20.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.726+0000 7f6ff20a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fec0c9ca0 0x7f6fec150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:20.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.726+0000 7f6ff20a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fec0caac0 0x7f6fec0ad330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:20.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.726+0000 7f6ff20a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fec15ad70 0x7f6fec0ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:20.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.726+0000 7f6ff20a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec0cd310 con 0x7f6fec0c9ca0 2023-10-16T11:24:20.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.726+0000 7f6ff20a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec0cd5f0 con 0x7f6fec15ad70 2023-10-16T11:24:20.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.726+0000 7f6ff20a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec0cd8d0 con 0x7f6fec0caac0 2023-10-16T11:24:20.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.726+0000 7f6feb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fec0c9ca0 0x7f6fec150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:20.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.727+0000 7f6feb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fec0c9ca0 0x7f6fec150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40296/0 (socket says 172.21.15.84:40296) 2023-10-16T11:24:20.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.727+0000 7f6feb7fe640 1 -- 172.21.15.84:0/4243660928 learned_addr learned my addr 172.21.15.84:0/4243660928 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:20.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.727+0000 7f6fe3fff640 1 --2- 172.21.15.84:0/4243660928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fec0caac0 0x7f6fec0ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:20.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.727+0000 7f6febfff640 1 --2- 172.21.15.84:0/4243660928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fec15ad70 0x7f6fec0ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:20.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.727+0000 7f6feb7fe640 1 -- 172.21.15.84:0/4243660928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fec0caac0 msgr2=0x7f6fec0ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:20.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.727+0000 7f6feb7fe640 1 --2- 172.21.15.84:0/4243660928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fec0caac0 0x7f6fec0ad330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:20.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.727+0000 7f6feb7fe640 1 -- 172.21.15.84:0/4243660928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fec15ad70 msgr2=0x7f6fec0ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:20.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.727+0000 7f6feb7fe640 1 --2- 172.21.15.84:0/4243660928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fec15ad70 0x7f6fec0ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:20.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.727+0000 7f6feb7fe640 1 -- 172.21.15.84:0/4243660928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc0094d0 con 0x7f6fec0c9ca0 2023-10-16T11:24:20.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.727+0000 7f6feb7fe640 1 --2- 172.21.15.84:0/4243660928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fec0c9ca0 0x7f6fec150ab0 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc009870 tx=0x7f6fdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:20.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.727+0000 7f6fe8ff9640 1 -- 172.21.15.84:0/4243660928 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc033c70 con 0x7f6fec0c9ca0 2023-10-16T11:24:20.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.728+0000 7f6fe8ff9640 1 -- 172.21.15.84:0/4243660928 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fdc033dd0 con 0x7f6fec0c9ca0 2023-10-16T11:24:20.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.728+0000 7f6fe8ff9640 1 -- 172.21.15.84:0/4243660928 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc03d6b0 con 0x7f6fec0c9ca0 2023-10-16T11:24:20.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.728+0000 7f6ff20a2640 1 -- 172.21.15.84:0/4243660928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fec0cdbb0 con 0x7f6fec0c9ca0 2023-10-16T11:24:20.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.728+0000 7f6ff20a2640 1 -- 172.21.15.84:0/4243660928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fec150ff0 con 0x7f6fec0c9ca0 2023-10-16T11:24:20.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.729+0000 7f6ff20a2640 1 -- 172.21.15.84:0/4243660928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fec04e160 con 0x7f6fec0c9ca0 2023-10-16T11:24:20.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.730+0000 7f6fe8ff9640 1 -- 172.21.15.84:0/4243660928 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6fdc03a040 con 0x7f6fec0c9ca0 2023-10-16T11:24:20.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.731+0000 7f6fe8ff9640 1 --2- 172.21.15.84:0/4243660928 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f6fcc06afc0 0x7f6fcc06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:20.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.731+0000 7f6fe8ff9640 1 -- 172.21.15.84:0/4243660928 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5773+0+0 (secure 0 0 0) 0x7f6fdc0ab780 con 0x7f6fec0c9ca0 2023-10-16T11:24:20.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.731+0000 7f6fe3fff640 1 --2- 172.21.15.84:0/4243660928 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f6fcc06afc0 0x7f6fcc06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:20.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.732+0000 7f6fe3fff640 1 --2- 172.21.15.84:0/4243660928 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f6fcc06afc0 0x7f6fcc06d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6fec159ef0 tx=0x7f6fd4007e00 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:20.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.735+0000 7f6fe8ff9640 1 -- 172.21.15.84:0/4243660928 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6fdc076670 con 0x7f6fec0c9ca0 2023-10-16T11:24:20.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:20.942+0000 7f6ff20a2640 1 -- 172.21.15.84:0/4243660928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6fec06a290 con 0x7f6fec0c9ca0 2023-10-16T11:24:21.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.017+0000 7f6fe8ff9640 1 -- 172.21.15.84:0/4243660928 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v211) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6fdc079c90 con 0x7f6fec0c9ca0 2023-10-16T11:24:21.019 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-10-16T11:24:21.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.020+0000 7f6ff20a2640 1 -- 172.21.15.84:0/4243660928 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f6fcc06afc0 msgr2=0x7f6fcc06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:21.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.020+0000 7f6ff20a2640 1 --2- 172.21.15.84:0/4243660928 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f6fcc06afc0 0x7f6fcc06d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6fec159ef0 tx=0x7f6fd4007e00 comp rx=0 tx=0).stop 2023-10-16T11:24:21.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.020+0000 7f6ff20a2640 1 -- 172.21.15.84:0/4243660928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fec0c9ca0 msgr2=0x7f6fec150ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:21.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.020+0000 7f6ff20a2640 1 --2- 172.21.15.84:0/4243660928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fec0c9ca0 0x7f6fec150ab0 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc009870 tx=0x7f6fdc0046c0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.021+0000 7f6ff20a2640 1 -- 172.21.15.84:0/4243660928 shutdown_connections 2023-10-16T11:24:21.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.021+0000 7f6ff20a2640 1 --2- 172.21.15.84:0/4243660928 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f6fcc06afc0 0x7f6fcc06d480 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.021+0000 7f6ff20a2640 1 --2- 172.21.15.84:0/4243660928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fec15ad70 0x7f6fec0ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.021+0000 7f6ff20a2640 1 --2- 172.21.15.84:0/4243660928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fec0caac0 0x7f6fec0ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.021+0000 7f6ff20a2640 1 --2- 172.21.15.84:0/4243660928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fec0c9ca0 0x7f6fec150ab0 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.021+0000 7f6ff20a2640 1 -- 172.21.15.84:0/4243660928 >> 172.21.15.84:0/4243660928 conn(0x7f6fec0ce9b0 msgr2=0x7f6fec0b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:21.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.021+0000 7f6ff20a2640 1 -- 172.21.15.84:0/4243660928 shutdown_connections 2023-10-16T11:24:21.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.021+0000 7f6ff20a2640 1 -- 172.21.15.84:0/4243660928 wait complete. 2023-10-16T11:24:21.041 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T11:24:21.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.247+0000 7f5653075640 1 Processor -- start 2023-10-16T11:24:21.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.247+0000 7f5653075640 1 -- start start 2023-10-16T11:24:21.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.248+0000 7f5653075640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f564c0b0b20 0x7f564c164b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:21.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.248+0000 7f5653075640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f564c06f7f0 con 0x7f564c159f00 2023-10-16T11:24:21.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.248+0000 7f5653075640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f564c06c5e0 con 0x7f564c159b20 2023-10-16T11:24:21.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.248+0000 7f5653075640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f564c06c8c0 con 0x7f564c0b0b20 2023-10-16T11:24:21.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.249+0000 7f5651612640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f564c0b0b20 0x7f564c164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:21.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.249+0000 7f564bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f564c159f00 0x7f564c0b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58786/0 (socket says 172.21.15.84:58786) 2023-10-16T11:24:21.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.249+0000 7f564bfff640 1 -- 172.21.15.84:0/2211274564 learned_addr learned my addr 172.21.15.84:0/2211274564 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:21.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.249+0000 7f5651612640 1 -- 172.21.15.84:0/2211274564 >> v1:172.21.15.157:6789/0 conn(0x7f564c159b20 legacy=0x7f564c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:21.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.249+0000 7f5651612640 1 -- 172.21.15.84:0/2211274564 >> v1:172.21.15.84:6789/0 conn(0x7f564c159f00 legacy=0x7f564c0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:21.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.249+0000 7f5651612640 1 -- 172.21.15.84:0/2211274564 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c06cba0 con 0x7f564c0b0b20 2023-10-16T11:24:21.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.250+0000 7f5651612640 1 --2- 172.21.15.84:0/2211274564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f564c0b0b20 0x7f564c164b40 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f5644009d90 tx=0x7f564402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32769bc9c8021d74 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:21.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.250+0000 7f564b7fe640 1 -- 172.21.15.84:0/2211274564 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564402cc60 con 0x7f564c0b0b20 2023-10-16T11:24:21.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.250+0000 7f564b7fe640 1 -- 172.21.15.84:0/2211274564 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564402cdc0 con 0x7f564c0b0b20 2023-10-16T11:24:21.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.250+0000 7f564b7fe640 1 -- 172.21.15.84:0/2211274564 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644034660 con 0x7f564c0b0b20 2023-10-16T11:24:21.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.251+0000 7f5653075640 1 -- 172.21.15.84:0/2211274564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f564c0b0b20 msgr2=0x7f564c164b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:21.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.251+0000 7f5653075640 1 --2- 172.21.15.84:0/2211274564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f564c0b0b20 0x7f564c164b40 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f5644009d90 tx=0x7f564402beb0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.251+0000 7f5653075640 1 -- 172.21.15.84:0/2211274564 shutdown_connections 2023-10-16T11:24:21.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.251+0000 7f5653075640 1 --2- 172.21.15.84:0/2211274564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f564c0b0b20 0x7f564c164b40 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.251+0000 7f5653075640 1 -- 172.21.15.84:0/2211274564 >> 172.21.15.84:0/2211274564 conn(0x7f564c0cb410 msgr2=0x7f564c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:21.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.251+0000 7f5653075640 1 -- 172.21.15.84:0/2211274564 shutdown_connections 2023-10-16T11:24:21.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.251+0000 7f5653075640 1 -- 172.21.15.84:0/2211274564 wait complete. 2023-10-16T11:24:21.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.253+0000 7f5653075640 1 Processor -- start 2023-10-16T11:24:21.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.253+0000 7f5653075640 1 -- start start 2023-10-16T11:24:21.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.254+0000 7f5653075640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f564c0b0b20 0x7f564c145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:21.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.254+0000 7f5653075640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564c159b20 0x7f564c143490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:21.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.254+0000 7f5653075640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f564c159f00 0x7f564c14e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:21.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.254+0000 7f5653075640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c145740 con 0x7f564c0b0b20 2023-10-16T11:24:21.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.254+0000 7f5653075640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c144b30 con 0x7f564c159f00 2023-10-16T11:24:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.254+0000 7f5653075640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564c144e10 con 0x7f564c159b20 2023-10-16T11:24:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.254+0000 7f5650e11640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f564c0b0b20 0x7f564c145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.254+0000 7f5650e11640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f564c0b0b20 0x7f564c145bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40318/0 (socket says 172.21.15.84:40318) 2023-10-16T11:24:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.254+0000 7f5650e11640 1 -- 172.21.15.84:0/1748968914 learned_addr learned my addr 172.21.15.84:0/1748968914 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.254+0000 7f564bfff640 1 --2- 172.21.15.84:0/1748968914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564c159b20 0x7f564c143490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.255+0000 7f5651612640 1 --2- 172.21.15.84:0/1748968914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f564c159f00 0x7f564c14e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.255+0000 7f5650e11640 1 -- 172.21.15.84:0/1748968914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564c159b20 msgr2=0x7f564c143490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.255+0000 7f5650e11640 1 --2- 172.21.15.84:0/1748968914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564c159b20 0x7f564c143490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.255+0000 7f5650e11640 1 -- 172.21.15.84:0/1748968914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f564c159f00 msgr2=0x7f564c14e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.255+0000 7f5650e11640 1 --2- 172.21.15.84:0/1748968914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f564c159f00 0x7f564c14e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.255+0000 7f5650e11640 1 -- 172.21.15.84:0/1748968914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5644009a70 con 0x7f564c0b0b20 2023-10-16T11:24:21.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.255+0000 7f5651612640 1 --2- 172.21.15.84:0/1748968914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f564c159f00 0x7f564c14e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:21.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.255+0000 7f5650e11640 1 --2- 172.21.15.84:0/1748968914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f564c0b0b20 0x7f564c145bd0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f564000c430 tx=0x7f564000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:21.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.255+0000 7f56497fa640 1 -- 172.21.15.84:0/1748968914 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640011790 con 0x7f564c0b0b20 2023-10-16T11:24:21.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.255+0000 7f56497fa640 1 -- 172.21.15.84:0/1748968914 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56400118f0 con 0x7f564c0b0b20 2023-10-16T11:24:21.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.255+0000 7f5653075640 1 -- 172.21.15.84:0/1748968914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564c1450f0 con 0x7f564c0b0b20 2023-10-16T11:24:21.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.255+0000 7f56497fa640 1 -- 172.21.15.84:0/1748968914 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640011ce0 con 0x7f564c0b0b20 2023-10-16T11:24:21.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.255+0000 7f5653075640 1 -- 172.21.15.84:0/1748968914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f564c1439d0 con 0x7f564c0b0b20 2023-10-16T11:24:21.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.256+0000 7f5653075640 1 -- 172.21.15.84:0/1748968914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564c04e160 con 0x7f564c0b0b20 2023-10-16T11:24:21.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.257+0000 7f56497fa640 1 -- 172.21.15.84:0/1748968914 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f564001c460 con 0x7f564c0b0b20 2023-10-16T11:24:21.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.258+0000 7f56497fa640 1 --2- 172.21.15.84:0/1748968914 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f563006af90 0x7f563006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:21.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.258+0000 7f56497fa640 1 -- 172.21.15.84:0/1748968914 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f5640089400 con 0x7f564c0b0b20 2023-10-16T11:24:21.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.258+0000 7f564bfff640 1 --2- 172.21.15.84:0/1748968914 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f563006af90 0x7f563006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:21.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.259+0000 7f564bfff640 1 --2- 172.21.15.84:0/1748968914 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f563006af90 0x7f563006d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f564c0c1fe0 tx=0x7f5638009040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:21.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.263+0000 7f56497fa640 1 -- 172.21.15.84:0/1748968914 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f56400546d0 con 0x7f564c0b0b20 2023-10-16T11:24:21.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.526+0000 7f5653075640 1 -- 172.21.15.84:0/1748968914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f564c06d580 con 0x7f564c0b0b20 2023-10-16T11:24:21.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.527+0000 7f56497fa640 1 -- 172.21.15.84:0/1748968914 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v54) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5640057cf0 con 0x7f564c0b0b20 2023-10-16T11:24:21.528 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:21.529 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:24:21.529 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-10-16T11:24:21.529 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.529 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-10-16T11:24:21.530 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l6dhjKRAA0pMvY6AK6qtFXDprNhwsqg==", 2023-10-16T11:24:21.530 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.530 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:24:21.530 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:24:21.530 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:24:21.531 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:24:21.531 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.531 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:21.531 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.531 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-10-16T11:24:21.532 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lFpMxEBAA0TLPRt5q3FwwBvMG019cjQ==", 2023-10-16T11:24:21.532 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.532 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:24:21.532 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:24:21.533 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:24:21.533 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:24:21.533 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.534 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:21.534 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.534 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-10-16T11:24:21.534 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l73RBOhAAOMEPxfUAMHhn5Blyvd5oMw==", 2023-10-16T11:24:21.534 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.535 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:24:21.535 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:24:21.535 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:24:21.535 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:24:21.535 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.536 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:21.536 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.536 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-10-16T11:24:21.536 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lcTIRIhAA0+FWm4b4yw77LD3+WKbk7w==", 2023-10-16T11:24:21.536 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.537 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:24:21.537 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:24:21.537 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:24:21.537 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:24:21.538 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.538 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:21.538 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.539 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-10-16T11:24:21.539 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDYBi1l1zBLERAA3YLW3xIYQoRVw7KSkwN3Og==", 2023-10-16T11:24:21.539 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.539 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:24:21.539 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:24:21.540 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:24:21.540 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.540 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:21.540 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.541 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-10-16T11:24:21.541 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDbBi1lFUUxKRAAQbus8SmYprJRmWTjoCznNA==", 2023-10-16T11:24:21.541 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.541 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:24:21.541 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:24:21.541 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:24:21.542 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.542 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:21.542 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.542 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-10-16T11:24:21.542 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDeBi1l+9lBMxAArl6sb/h15uBG/a9lVH40Hg==", 2023-10-16T11:24:21.543 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.543 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:24:21.543 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:24:21.543 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:24:21.543 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.544 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:21.544 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.544 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-10-16T11:24:21.544 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDiBi1l2rnEERAA3oRD71xa0fEOynM8zKuBEg==", 2023-10-16T11:24:21.544 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.545 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:24:21.545 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:24:21.545 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:24:21.545 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.546 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:21.546 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.546 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-10-16T11:24:21.546 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDnBi1lR7oLChAAZto8fDovDbpOd3Fcnc7C6w==", 2023-10-16T11:24:21.546 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.547 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:24:21.547 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:24:21.547 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:24:21.547 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.547 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:21.548 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.548 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-10-16T11:24:21.548 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDqBi1lmjg/JBAAlINKRGBc0/UWJJHMVFw57w==", 2023-10-16T11:24:21.548 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.549 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:24:21.549 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:24:21.549 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:24:21.549 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.549 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:21.550 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.550 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-10-16T11:24:21.550 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDuBi1lu0kMARAAtuSeM8F3yr8CVrc3p1biaQ==", 2023-10-16T11:24:21.550 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.550 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:24:21.551 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:24:21.551 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:24:21.551 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.551 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:21.551 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.552 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-10-16T11:24:21.552 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDxBi1lf6mNGBAAHGbqzJ0fmDxQIsasboHMfg==", 2023-10-16T11:24:21.552 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.552 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:24:21.552 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:24:21.553 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:24:21.553 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.553 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:21.553 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.553 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-10-16T11:24:21.554 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ==", 2023-10-16T11:24:21.554 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.554 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:24:21.554 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow r", 2023-10-16T11:24:21.554 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T11:24:21.555 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.555 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:21.555 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.555 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-10-16T11:24:21.555 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1lHU+4BxAAd8ET8hQ8Y+EUwdTJr9/+Ig==", 2023-10-16T11:24:21.556 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.556 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:24:21.556 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:24:21.556 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:24:21.556 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:24:21.557 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.557 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:21.557 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.557 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-10-16T11:24:21.557 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lA1BOORAAFNyfqgEF34MVR9YX3KBKkQ==", 2023-10-16T11:24:21.557 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.558 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:24:21.558 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:24:21.558 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:24:21.559 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:24:21.559 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.559 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:21.559 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.560 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-10-16T11:24:21.560 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1l848oERAANMfrZz9JrlgNVh7ZFbtnaA==", 2023-10-16T11:24:21.560 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.560 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:24:21.560 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:24:21.561 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:24:21.561 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:24:21.561 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.561 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:21.561 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.562 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-10-16T11:24:21.562 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDRBi1lTAxTGRAAMptxnxy3Qit/cTCXie6CrQ==", 2023-10-16T11:24:21.562 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.562 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:24:21.562 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:24:21.563 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:24:21.563 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:24:21.563 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.563 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:21.564 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.564 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-10-16T11:24:21.564 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lPuJxHRAABAnGPVCrkq0JR4Y6iFGZlw==", 2023-10-16T11:24:21.564 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.564 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:24:21.565 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T11:24:21.565 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:24:21.565 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.565 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:21.565 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:21.566 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-10-16T11:24:21.566 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lLxnKEBAA+a/pOHUp7Sg3m4weAust5w==", 2023-10-16T11:24:21.566 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:24:21.566 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:24:21.566 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T11:24:21.567 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:24:21.567 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.567 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:21.567 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-10-16T11:24:21.567 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:24:21.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.529+0000 7f5653075640 1 -- 172.21.15.84:0/1748968914 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f563006af90 msgr2=0x7f563006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:21.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.529+0000 7f5653075640 1 --2- 172.21.15.84:0/1748968914 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f563006af90 0x7f563006d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f564c0c1fe0 tx=0x7f5638009040 comp rx=0 tx=0).stop 2023-10-16T11:24:21.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.529+0000 7f5653075640 1 -- 172.21.15.84:0/1748968914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f564c0b0b20 msgr2=0x7f564c145bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:21.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.529+0000 7f5653075640 1 --2- 172.21.15.84:0/1748968914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f564c0b0b20 0x7f564c145bd0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f564000c430 tx=0x7f564000c8f0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.529+0000 7f5653075640 1 -- 172.21.15.84:0/1748968914 shutdown_connections 2023-10-16T11:24:21.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.529+0000 7f5653075640 1 --2- 172.21.15.84:0/1748968914 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f563006af90 0x7f563006d450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.529+0000 7f5653075640 1 --2- 172.21.15.84:0/1748968914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f564c159f00 0x7f564c14e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.529+0000 7f5653075640 1 --2- 172.21.15.84:0/1748968914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f564c159b20 0x7f564c143490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.529+0000 7f5653075640 1 --2- 172.21.15.84:0/1748968914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f564c0b0b20 0x7f564c145bd0 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.529+0000 7f5653075640 1 -- 172.21.15.84:0/1748968914 >> 172.21.15.84:0/1748968914 conn(0x7f564c0cb410 msgr2=0x7f564c0afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:21.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.530+0000 7f5653075640 1 -- 172.21.15.84:0/1748968914 shutdown_connections 2023-10-16T11:24:21.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.530+0000 7f5653075640 1 -- 172.21.15.84:0/1748968914 wait complete. 2023-10-16T11:24:21.572 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:24:21.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.791+0000 7f8a2487c640 1 Processor -- start 2023-10-16T11:24:21.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.791+0000 7f8a2487c640 1 -- start start 2023-10-16T11:24:21.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.792+0000 7f8a2487c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8a2015aaa0 0x7f8a2015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:21.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.792+0000 7f8a2487c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8a2015ae80 0x7f8a200cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:21.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.792+0000 7f8a2487c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a2015ed60 con 0x7f8a200ccaa0 2023-10-16T11:24:21.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.793+0000 7f8a2487c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a2006b810 con 0x7f8a2015aaa0 2023-10-16T11:24:21.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.793+0000 7f8a2487c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a200707e0 con 0x7f8a2015ae80 2023-10-16T11:24:21.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.793+0000 7f8a1e59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8a2015aaa0 0x7f8a2015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:21.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.793+0000 7f8a1ed9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8a200ccaa0 0x7f8a20070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58790/0 (socket says 172.21.15.84:58790) 2023-10-16T11:24:21.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.793+0000 7f8a1ed9d640 1 -- 172.21.15.84:0/2686910028 learned_addr learned my addr 172.21.15.84:0/2686910028 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:21.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.793+0000 7f8a1dd9b640 1 --2- 172.21.15.84:0/2686910028 >> v2:172.21.15.157:3300/0 conn(0x7f8a2015ae80 0x7f8a200cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:21.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.793+0000 7f8a1d59a640 1 -- 172.21.15.84:0/2686910028 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3419455338 0 0) 0x7f8a2015ed60 con 0x7f8a200ccaa0 2023-10-16T11:24:21.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.793+0000 7f8a1e59c640 1 -- 172.21.15.84:0/2686910028 >> v2:172.21.15.157:3300/0 conn(0x7f8a2015ae80 msgr2=0x7f8a200cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:21.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.793+0000 7f8a1e59c640 1 --2- 172.21.15.84:0/2686910028 >> v2:172.21.15.157:3300/0 conn(0x7f8a2015ae80 0x7f8a200cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.793+0000 7f8a1e59c640 1 -- 172.21.15.84:0/2686910028 >> v1:172.21.15.84:6789/0 conn(0x7f8a200ccaa0 legacy=0x7f8a20070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.793+0000 7f8a1e59c640 1 -- 172.21.15.84:0/2686910028 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2015ef90 con 0x7f8a2015aaa0 2023-10-16T11:24:21.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.795+0000 7f8a1e59c640 1 --2- 172.21.15.84:0/2686910028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a2015aaa0 0x7f8a2015e820 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f8a200c59c0 tx=0x7f8a1002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac071c06dd686cac server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:21.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.795+0000 7f8a1d59a640 1 -- 172.21.15.84:0/2686910028 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10031830 con 0x7f8a2015aaa0 2023-10-16T11:24:21.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.795+0000 7f8a1d59a640 1 -- 172.21.15.84:0/2686910028 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a10031990 con 0x7f8a2015aaa0 2023-10-16T11:24:21.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.795+0000 7f8a1d59a640 1 -- 172.21.15.84:0/2686910028 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10031d20 con 0x7f8a2015aaa0 2023-10-16T11:24:21.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.795+0000 7f8a2487c640 1 -- 172.21.15.84:0/2686910028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a2015aaa0 msgr2=0x7f8a2015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:21.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.795+0000 7f8a2487c640 1 --2- 172.21.15.84:0/2686910028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a2015aaa0 0x7f8a2015e820 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f8a200c59c0 tx=0x7f8a1002b8d0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.796+0000 7f8a2487c640 1 -- 172.21.15.84:0/2686910028 shutdown_connections 2023-10-16T11:24:21.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.796+0000 7f8a2487c640 1 --2- 172.21.15.84:0/2686910028 >> v2:172.21.15.157:3300/0 conn(0x7f8a2015ae80 0x7f8a200cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.796+0000 7f8a2487c640 1 --2- 172.21.15.84:0/2686910028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a2015aaa0 0x7f8a2015e820 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.796+0000 7f8a2487c640 1 -- 172.21.15.84:0/2686910028 >> 172.21.15.84:0/2686910028 conn(0x7f8a200cb410 msgr2=0x7f8a200cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:21.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.796+0000 7f8a2487c640 1 -- 172.21.15.84:0/2686910028 shutdown_connections 2023-10-16T11:24:21.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.796+0000 7f8a2487c640 1 -- 172.21.15.84:0/2686910028 wait complete. 2023-10-16T11:24:21.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.797+0000 7f8a2487c640 1 Processor -- start 2023-10-16T11:24:21.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.798+0000 7f8a2487c640 1 -- start start 2023-10-16T11:24:21.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.798+0000 7f8a2487c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a200ccaa0 0x7f8a20159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:21.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.798+0000 7f8a2487c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a2015aaa0 0x7f8a200c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:21.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.799+0000 7f8a2487c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a2015ae80 0x7f8a200c3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:21.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.799+0000 7f8a2487c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2015e090 con 0x7f8a200ccaa0 2023-10-16T11:24:21.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.799+0000 7f8a2487c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a200ca180 con 0x7f8a2015ae80 2023-10-16T11:24:21.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.799+0000 7f8a2487c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a200ca460 con 0x7f8a2015aaa0 2023-10-16T11:24:21.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.799+0000 7f8a1e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a200ccaa0 0x7f8a20159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:21.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.799+0000 7f8a1e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a200ccaa0 0x7f8a20159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40334/0 (socket says 172.21.15.84:40334) 2023-10-16T11:24:21.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.799+0000 7f8a1e59c640 1 -- 172.21.15.84:0/224083928 learned_addr learned my addr 172.21.15.84:0/224083928 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:21.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.799+0000 7f8a1dd9b640 1 --2- 172.21.15.84:0/224083928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a2015aaa0 0x7f8a200c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:21.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.799+0000 7f8a1e59c640 1 -- 172.21.15.84:0/224083928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a2015aaa0 msgr2=0x7f8a200c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:21.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.799+0000 7f8a1e59c640 1 --2- 172.21.15.84:0/224083928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a2015aaa0 0x7f8a200c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.799+0000 7f8a1e59c640 1 -- 172.21.15.84:0/224083928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a2015ae80 msgr2=0x7f8a200c3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:21.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.799+0000 7f8a1e59c640 1 --2- 172.21.15.84:0/224083928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a2015ae80 0x7f8a200c3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.799+0000 7f8a1e59c640 1 -- 172.21.15.84:0/224083928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a10009400 con 0x7f8a200ccaa0 2023-10-16T11:24:21.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.800+0000 7f8a1e59c640 1 --2- 172.21.15.84:0/224083928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a200ccaa0 0x7f8a20159a00 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009890 tx=0x7f8a100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:21.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.800+0000 7f8a0effd640 1 -- 172.21.15.84:0/224083928 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a100317d0 con 0x7f8a200ccaa0 2023-10-16T11:24:21.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.800+0000 7f8a0effd640 1 -- 172.21.15.84:0/224083928 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1003ba70 con 0x7f8a200ccaa0 2023-10-16T11:24:21.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.800+0000 7f8a0effd640 1 -- 172.21.15.84:0/224083928 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1003be20 con 0x7f8a200ccaa0 2023-10-16T11:24:21.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.800+0000 7f8a2487c640 1 -- 172.21.15.84:0/224083928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a200ca740 con 0x7f8a200ccaa0 2023-10-16T11:24:21.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.800+0000 7f8a2487c640 1 -- 172.21.15.84:0/224083928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a200cad00 con 0x7f8a200ccaa0 2023-10-16T11:24:21.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.801+0000 7f8a2487c640 1 -- 172.21.15.84:0/224083928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a200caf20 con 0x7f8a200ccaa0 2023-10-16T11:24:21.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.802+0000 7f8a0effd640 1 -- 172.21.15.84:0/224083928 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8a100453f0 con 0x7f8a200ccaa0 2023-10-16T11:24:21.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.803+0000 7f8a0effd640 1 --2- 172.21.15.84:0/224083928 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f8a0006af90 0x7f8a0006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:21.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.803+0000 7f8a0effd640 1 -- 172.21.15.84:0/224083928 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f8a100ab890 con 0x7f8a200ccaa0 2023-10-16T11:24:21.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.803+0000 7f8a1dd9b640 1 --2- 172.21.15.84:0/224083928 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f8a0006af90 0x7f8a0006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:21.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.804+0000 7f8a1dd9b640 1 --2- 172.21.15.84:0/224083928 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f8a0006af90 0x7f8a0006d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8a14006fd0 tx=0x7f8a14008040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:21.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.807+0000 7f8a0effd640 1 -- 172.21.15.84:0/224083928 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8a10076b60 con 0x7f8a200ccaa0 2023-10-16T11:24:21.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.989+0000 7f8a2487c640 1 -- 172.21.15.84:0/224083928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a2015f520 con 0x7f8a200ccaa0 2023-10-16T11:24:21.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.991+0000 7f8a0effd640 1 -- 172.21.15.84:0/224083928 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v213) v1 ==== 74+0+12887 (secure 0 0 0) 0x7f8a10035090 con 0x7f8a200ccaa0 2023-10-16T11:24:21.992 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:21.992 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":213,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:24:21.032556+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:24:21.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.993+0000 7f8a2487c640 1 -- 172.21.15.84:0/224083928 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f8a0006af90 msgr2=0x7f8a0006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:21.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.993+0000 7f8a2487c640 1 --2- 172.21.15.84:0/224083928 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f8a0006af90 0x7f8a0006d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8a14006fd0 tx=0x7f8a14008040 comp rx=0 tx=0).stop 2023-10-16T11:24:21.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.993+0000 7f8a2487c640 1 -- 172.21.15.84:0/224083928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a200ccaa0 msgr2=0x7f8a20159a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:21.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.993+0000 7f8a2487c640 1 --2- 172.21.15.84:0/224083928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a200ccaa0 0x7f8a20159a00 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009890 tx=0x7f8a100046c0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.994+0000 7f8a2487c640 1 -- 172.21.15.84:0/224083928 shutdown_connections 2023-10-16T11:24:21.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.994+0000 7f8a2487c640 1 --2- 172.21.15.84:0/224083928 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f8a0006af90 0x7f8a0006d450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.994+0000 7f8a2487c640 1 --2- 172.21.15.84:0/224083928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a2015ae80 0x7f8a200c3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.994+0000 7f8a2487c640 1 --2- 172.21.15.84:0/224083928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a2015aaa0 0x7f8a200c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.994+0000 7f8a2487c640 1 --2- 172.21.15.84:0/224083928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a200ccaa0 0x7f8a20159a00 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:21.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.994+0000 7f8a2487c640 1 -- 172.21.15.84:0/224083928 >> 172.21.15.84:0/224083928 conn(0x7f8a200cb410 msgr2=0x7f8a200ae230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:21.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.994+0000 7f8a2487c640 1 -- 172.21.15.84:0/224083928 shutdown_connections 2023-10-16T11:24:21.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:21.994+0000 7f8a2487c640 1 -- 172.21.15.84:0/224083928 wait complete. 2023-10-16T11:24:22.015 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:24:22.015 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:24:22.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.227+0000 7f262aaa2640 1 Processor -- start 2023-10-16T11:24:22.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.227+0000 7f262aaa2640 1 -- start start 2023-10-16T11:24:22.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.229+0000 7f262aaa2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f262415ad70 0x7f2624069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:22.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.229+0000 7f262aaa2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f26240caac0 0x7f262406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:22.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.229+0000 7f262aaa2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26240cb030 con 0x7f26240c9ca0 2023-10-16T11:24:22.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.229+0000 7f262aaa2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26240c91c0 con 0x7f26240caac0 2023-10-16T11:24:22.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.229+0000 7f262aaa2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26240c94a0 con 0x7f262415ad70 2023-10-16T11:24:22.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.230+0000 7f262883e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f262415ad70 0x7f2624069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:22.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.230+0000 7f262903f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f26240caac0 0x7f262406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:22.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.230+0000 7f262903f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f26240caac0 0x7f262406a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:50092/0 (socket says 172.21.15.84:50092) 2023-10-16T11:24:22.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.230+0000 7f262903f640 1 -- 172.21.15.84:0/1287053140 learned_addr learned my addr 172.21.15.84:0/1287053140 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:22.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.230+0000 7f262903f640 1 -- 172.21.15.84:0/1287053140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f262415ad70 msgr2=0x7f2624069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:22.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.230+0000 7f262903f640 1 --2- 172.21.15.84:0/1287053140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f262415ad70 0x7f2624069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:22.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.230+0000 7f262903f640 1 -- 172.21.15.84:0/1287053140 >> v1:172.21.15.157:6789/0 conn(0x7f26240c9ca0 legacy=0x7f26240ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:22.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.231+0000 7f262903f640 1 -- 172.21.15.84:0/1287053140 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26240c9780 con 0x7f26240caac0 2023-10-16T11:24:22.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.231+0000 7f262883e640 1 --2- 172.21.15.84:0/1287053140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f262415ad70 0x7f2624069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:24:22.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.232+0000 7f262903f640 1 --2- 172.21.15.84:0/1287053140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26240caac0 0x7f262406a2e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f261c009e50 tx=0x7f261c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6043f33c191dae55 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:22.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.232+0000 7f26237fe640 1 -- 172.21.15.84:0/1287053140 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c02cb00 con 0x7f26240caac0 2023-10-16T11:24:22.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.232+0000 7f26237fe640 1 -- 172.21.15.84:0/1287053140 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c02cc60 con 0x7f26240caac0 2023-10-16T11:24:22.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.232+0000 7f26237fe640 1 -- 172.21.15.84:0/1287053140 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c0343f0 con 0x7f26240caac0 2023-10-16T11:24:22.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.233+0000 7f262aaa2640 1 -- 172.21.15.84:0/1287053140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26240caac0 msgr2=0x7f262406a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:22.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.233+0000 7f262aaa2640 1 --2- 172.21.15.84:0/1287053140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26240caac0 0x7f262406a2e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f261c009e50 tx=0x7f261c02bf60 comp rx=0 tx=0).stop 2023-10-16T11:24:22.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.233+0000 7f262aaa2640 1 -- 172.21.15.84:0/1287053140 shutdown_connections 2023-10-16T11:24:22.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.233+0000 7f262aaa2640 1 --2- 172.21.15.84:0/1287053140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26240caac0 0x7f262406a2e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:22.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.233+0000 7f262aaa2640 1 --2- 172.21.15.84:0/1287053140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f262415ad70 0x7f2624069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:22.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.233+0000 7f262aaa2640 1 -- 172.21.15.84:0/1287053140 >> 172.21.15.84:0/1287053140 conn(0x7f26240ce9b0 msgr2=0x7f26240cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:22.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.233+0000 7f262aaa2640 1 -- 172.21.15.84:0/1287053140 shutdown_connections 2023-10-16T11:24:22.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.233+0000 7f262aaa2640 1 -- 172.21.15.84:0/1287053140 wait complete. 2023-10-16T11:24:22.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.235+0000 7f262aaa2640 1 Processor -- start 2023-10-16T11:24:22.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.236+0000 7f262aaa2640 1 -- start start 2023-10-16T11:24:22.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.237+0000 7f262aaa2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26240c9ca0 0x7f26241f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:22.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.237+0000 7f262aaa2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26240caac0 0x7f26240c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:22.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.237+0000 7f262aaa2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262415ad70 0x7f2624151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:22.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.237+0000 7f262aaa2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26241f2af0 con 0x7f26240c9ca0 2023-10-16T11:24:22.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.237+0000 7f262aaa2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26241f35d0 con 0x7f262415ad70 2023-10-16T11:24:22.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.237+0000 7f262aaa2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26240c5c40 con 0x7f26240caac0 2023-10-16T11:24:22.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.237+0000 7f262883e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26240c9ca0 0x7f26241f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:22.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.237+0000 7f262883e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26240c9ca0 0x7f26241f25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40350/0 (socket says 172.21.15.84:40350) 2023-10-16T11:24:22.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.237+0000 7f262883e640 1 -- 172.21.15.84:0/1570629405 learned_addr learned my addr 172.21.15.84:0/1570629405 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:22.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.238+0000 7f262903f640 1 --2- 172.21.15.84:0/1570629405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262415ad70 0x7f2624151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:22.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.238+0000 7f2623fff640 1 --2- 172.21.15.84:0/1570629405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26240caac0 0x7f26240c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:22.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.238+0000 7f262883e640 1 -- 172.21.15.84:0/1570629405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26240caac0 msgr2=0x7f26240c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:22.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.238+0000 7f262883e640 1 --2- 172.21.15.84:0/1570629405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26240caac0 0x7f26240c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:22.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.238+0000 7f262883e640 1 -- 172.21.15.84:0/1570629405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262415ad70 msgr2=0x7f2624151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:22.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.238+0000 7f262883e640 1 --2- 172.21.15.84:0/1570629405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262415ad70 0x7f2624151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:22.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.238+0000 7f262883e640 1 -- 172.21.15.84:0/1570629405 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2618009400 con 0x7f26240c9ca0 2023-10-16T11:24:22.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.238+0000 7f262883e640 1 --2- 172.21.15.84:0/1570629405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26240c9ca0 0x7f26241f25b0 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7f261800e960 tx=0x7f261800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:22.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.238+0000 7f26217fa640 1 -- 172.21.15.84:0/1570629405 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261800ca00 con 0x7f26240c9ca0 2023-10-16T11:24:22.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.238+0000 7f26217fa640 1 -- 172.21.15.84:0/1570629405 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261800cb60 con 0x7f26240c9ca0 2023-10-16T11:24:22.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.238+0000 7f26217fa640 1 -- 172.21.15.84:0/1570629405 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618013470 con 0x7f26240c9ca0 2023-10-16T11:24:22.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.238+0000 7f262aaa2640 1 -- 172.21.15.84:0/1570629405 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261c009b30 con 0x7f26240c9ca0 2023-10-16T11:24:22.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.239+0000 7f262aaa2640 1 -- 172.21.15.84:0/1570629405 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26240c77d0 con 0x7f26240c9ca0 2023-10-16T11:24:22.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.240+0000 7f262aaa2640 1 -- 172.21.15.84:0/1570629405 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26240ad480 con 0x7f26240c9ca0 2023-10-16T11:24:22.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.240+0000 7f26217fa640 1 -- 172.21.15.84:0/1570629405 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f26180135d0 con 0x7f26240c9ca0 2023-10-16T11:24:22.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.242+0000 7f26217fa640 1 --2- 172.21.15.84:0/1570629405 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f260806af90 0x7f260806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:22.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.242+0000 7f26217fa640 1 -- 172.21.15.84:0/1570629405 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f26180895e0 con 0x7f26240c9ca0 2023-10-16T11:24:22.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.242+0000 7f2623fff640 1 --2- 172.21.15.84:0/1570629405 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f260806af90 0x7f260806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:22.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.243+0000 7f2623fff640 1 --2- 172.21.15.84:0/1570629405 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f260806af90 0x7f260806d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f26240ccb10 tx=0x7f2610009040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:22.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.246+0000 7f26217fa640 1 -- 172.21.15.84:0/1570629405 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2618054940 con 0x7f26240c9ca0 2023-10-16T11:24:22.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.435+0000 7f262aaa2640 1 -- 172.21.15.84:0/1570629405 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f262404e160 con 0x7f26240c9ca0 2023-10-16T11:24:22.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.437+0000 7f26217fa640 1 -- 172.21.15.84:0/1570629405 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v213) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2618057f60 con 0x7f26240c9ca0 2023-10-16T11:24:22.438 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:24:22.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.439+0000 7f262aaa2640 1 -- 172.21.15.84:0/1570629405 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f260806af90 msgr2=0x7f260806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:22.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.439+0000 7f262aaa2640 1 --2- 172.21.15.84:0/1570629405 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f260806af90 0x7f260806d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f26240ccb10 tx=0x7f2610009040 comp rx=0 tx=0).stop 2023-10-16T11:24:22.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.439+0000 7f262aaa2640 1 -- 172.21.15.84:0/1570629405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26240c9ca0 msgr2=0x7f26241f25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:22.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.439+0000 7f262aaa2640 1 --2- 172.21.15.84:0/1570629405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26240c9ca0 0x7f26241f25b0 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7f261800e960 tx=0x7f261800ee20 comp rx=0 tx=0).stop 2023-10-16T11:24:22.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.440+0000 7f262aaa2640 1 -- 172.21.15.84:0/1570629405 shutdown_connections 2023-10-16T11:24:22.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.440+0000 7f262aaa2640 1 --2- 172.21.15.84:0/1570629405 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f260806af90 0x7f260806d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:22.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.440+0000 7f262aaa2640 1 --2- 172.21.15.84:0/1570629405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262415ad70 0x7f2624151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:22.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.440+0000 7f262aaa2640 1 --2- 172.21.15.84:0/1570629405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26240caac0 0x7f26240c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:22.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.440+0000 7f262aaa2640 1 --2- 172.21.15.84:0/1570629405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26240c9ca0 0x7f26241f25b0 unknown :-1 s=CLOSED pgs=2595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:22.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.440+0000 7f262aaa2640 1 -- 172.21.15.84:0/1570629405 >> 172.21.15.84:0/1570629405 conn(0x7f26240ce9b0 msgr2=0x7f26240ca3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:22.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.440+0000 7f262aaa2640 1 -- 172.21.15.84:0/1570629405 shutdown_connections 2023-10-16T11:24:22.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.440+0000 7f262aaa2640 1 -- 172.21.15.84:0/1570629405 wait complete. 2023-10-16T11:24:22.459 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T11:24:22.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.666+0000 7fe5b8533640 1 Processor -- start 2023-10-16T11:24:22.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.667+0000 7fe5b8533640 1 -- start start 2023-10-16T11:24:22.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.668+0000 7fe5b8533640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe5b015ae80 0x7fe5b015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:22.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.668+0000 7fe5b8533640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b00ce390 0x7fe5b0070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:22.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.668+0000 7fe5b8533640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b015ed60 con 0x7fe5b015aaa0 2023-10-16T11:24:22.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.668+0000 7fe5b8533640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b006b810 con 0x7fe5b015ae80 2023-10-16T11:24:22.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.668+0000 7fe5b8533640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b00707e0 con 0x7fe5b00ce390 2023-10-16T11:24:22.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.669+0000 7fe5b5ace640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe5b015ae80 0x7fe5b015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:22.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.669+0000 7fe5b5ace640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe5b015ae80 0x7fe5b015e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:50112/0 (socket says 172.21.15.84:50112) 2023-10-16T11:24:22.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.669+0000 7fe5b5ace640 1 -- 172.21.15.84:0/1127111147 learned_addr learned my addr 172.21.15.84:0/1127111147 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:22.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.669+0000 7fe5b6ad0640 1 --2- 172.21.15.84:0/1127111147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b00ce390 0x7fe5b0070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:22.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.670+0000 7fe5b5ace640 1 -- 172.21.15.84:0/1127111147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b00ce390 msgr2=0x7fe5b0070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:22.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.670+0000 7fe5b5ace640 1 --2- 172.21.15.84:0/1127111147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b00ce390 0x7fe5b0070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:22.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.670+0000 7fe5b5ace640 1 -- 172.21.15.84:0/1127111147 >> v1:172.21.15.157:6789/0 conn(0x7fe5b015aaa0 legacy=0x7fe5b00cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:22.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.670+0000 7fe5b5ace640 1 -- 172.21.15.84:0/1127111147 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b015ef90 con 0x7fe5b015ae80 2023-10-16T11:24:22.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.671+0000 7fe5b5ace640 1 --2- 172.21.15.84:0/1127111147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5b015ae80 0x7fe5b015e820 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009870 tx=0x7fe5a402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1924b8550ef26b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:22.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.671+0000 7fe5b52cd640 1 -- 172.21.15.84:0/1127111147 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4030840 con 0x7fe5b015ae80 2023-10-16T11:24:22.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.672+0000 7fe5b52cd640 1 -- 172.21.15.84:0/1127111147 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a40309a0 con 0x7fe5b015ae80 2023-10-16T11:24:22.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.672+0000 7fe5b52cd640 1 -- 172.21.15.84:0/1127111147 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4030d30 con 0x7fe5b015ae80 2023-10-16T11:24:22.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.672+0000 7fe5b8533640 1 -- 172.21.15.84:0/1127111147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5b015ae80 msgr2=0x7fe5b015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:22.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.672+0000 7fe5b8533640 1 --2- 172.21.15.84:0/1127111147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5b015ae80 0x7fe5b015e820 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009870 tx=0x7fe5a402b980 comp rx=0 tx=0).stop 2023-10-16T11:24:22.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.672+0000 7fe5b8533640 1 -- 172.21.15.84:0/1127111147 shutdown_connections 2023-10-16T11:24:22.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.672+0000 7fe5b8533640 1 --2- 172.21.15.84:0/1127111147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b00ce390 0x7fe5b0070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:22.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.672+0000 7fe5b8533640 1 --2- 172.21.15.84:0/1127111147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5b015ae80 0x7fe5b015e820 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:22.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.672+0000 7fe5b8533640 1 -- 172.21.15.84:0/1127111147 >> 172.21.15.84:0/1127111147 conn(0x7fe5b00cb410 msgr2=0x7fe5b00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:22.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.672+0000 7fe5b8533640 1 -- 172.21.15.84:0/1127111147 shutdown_connections 2023-10-16T11:24:22.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.672+0000 7fe5b8533640 1 -- 172.21.15.84:0/1127111147 wait complete. 2023-10-16T11:24:22.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.675+0000 7fe5b8533640 1 Processor -- start 2023-10-16T11:24:22.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.675+0000 7fe5b8533640 1 -- start start 2023-10-16T11:24:22.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.676+0000 7fe5b8533640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5b00ce390 0x7fe5b0155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:22.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.676+0000 7fe5b8533640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5b015aaa0 0x7fe5b006cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:22.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.676+0000 7fe5b8533640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b015ae80 0x7fe5b006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:22.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.676+0000 7fe5b8533640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b0156ff0 con 0x7fe5b015aaa0 2023-10-16T11:24:22.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.676+0000 7fe5b8533640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b0157a90 con 0x7fe5b00ce390 2023-10-16T11:24:22.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.676+0000 7fe5b8533640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b0156d10 con 0x7fe5b015ae80 2023-10-16T11:24:22.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.676+0000 7fe5b5ace640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5b015aaa0 0x7fe5b006cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:22.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.676+0000 7fe5b5ace640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5b015aaa0 0x7fe5b006cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40360/0 (socket says 172.21.15.84:40360) 2023-10-16T11:24:22.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.676+0000 7fe5b5ace640 1 -- 172.21.15.84:0/1718145724 learned_addr learned my addr 172.21.15.84:0/1718145724 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:22.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.677+0000 7fe5b62cf640 1 --2- 172.21.15.84:0/1718145724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5b00ce390 0x7fe5b0155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:22.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.677+0000 7fe5b6ad0640 1 --2- 172.21.15.84:0/1718145724 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b015ae80 0x7fe5b006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:22.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.677+0000 7fe5b5ace640 1 -- 172.21.15.84:0/1718145724 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b015ae80 msgr2=0x7fe5b006d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:22.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.677+0000 7fe5b5ace640 1 --2- 172.21.15.84:0/1718145724 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b015ae80 0x7fe5b006d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:22.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.677+0000 7fe5b5ace640 1 -- 172.21.15.84:0/1718145724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5b00ce390 msgr2=0x7fe5b0155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:22.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.677+0000 7fe5b5ace640 1 --2- 172.21.15.84:0/1718145724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5b00ce390 0x7fe5b0155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:22.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.677+0000 7fe5b5ace640 1 -- 172.21.15.84:0/1718145724 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a40094d0 con 0x7fe5b015aaa0 2023-10-16T11:24:22.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.677+0000 7fe5b6ad0640 1 --2- 172.21.15.84:0/1718145724 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b015ae80 0x7fe5b006d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:22.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.677+0000 7fe5b5ace640 1 --2- 172.21.15.84:0/1718145724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5b015aaa0 0x7fe5b006cc80 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4000c00 tx=0x7fe5a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:22.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.677+0000 7fe5aeffd640 1 -- 172.21.15.84:0/1718145724 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4039a90 con 0x7fe5b015aaa0 2023-10-16T11:24:22.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.677+0000 7fe5aeffd640 1 -- 172.21.15.84:0/1718145724 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a4039bf0 con 0x7fe5b015aaa0 2023-10-16T11:24:22.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.677+0000 7fe5aeffd640 1 -- 172.21.15.84:0/1718145724 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4043410 con 0x7fe5b015aaa0 2023-10-16T11:24:22.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.678+0000 7fe5b8533640 1 -- 172.21.15.84:0/1718145724 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b0157440 con 0x7fe5b015aaa0 2023-10-16T11:24:22.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.678+0000 7fe5b8533640 1 -- 172.21.15.84:0/1718145724 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b01569a0 con 0x7fe5b015aaa0 2023-10-16T11:24:22.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.679+0000 7fe5b8533640 1 -- 172.21.15.84:0/1718145724 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b004e160 con 0x7fe5b015aaa0 2023-10-16T11:24:22.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.680+0000 7fe5aeffd640 1 -- 172.21.15.84:0/1718145724 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe5a402f390 con 0x7fe5b015aaa0 2023-10-16T11:24:22.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.680+0000 7fe5aeffd640 1 --2- 172.21.15.84:0/1718145724 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fe59c06af60 0x7fe59c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:22.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.680+0000 7fe5aeffd640 1 -- 172.21.15.84:0/1718145724 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fe5a40ab720 con 0x7fe5b015aaa0 2023-10-16T11:24:22.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.681+0000 7fe5b62cf640 1 --2- 172.21.15.84:0/1718145724 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fe59c06af60 0x7fe59c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:22.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.682+0000 7fe5b62cf640 1 --2- 172.21.15.84:0/1718145724 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fe59c06af60 0x7fe59c06d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0007fd0 tx=0x7fe5a0009040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:22.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.685+0000 7fe5aeffd640 1 -- 172.21.15.84:0/1718145724 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe5a4076960 con 0x7fe5b015aaa0 2023-10-16T11:24:22.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.864+0000 7fe5b8533640 1 -- 172.21.15.84:0/1718145724 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe5b00717e0 con 0x7fe5b015aaa0 2023-10-16T11:24:22.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.864+0000 7fe5aeffd640 1 -- 172.21.15.84:0/1718145724 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v213) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe5a4034060 con 0x7fe5b015aaa0 2023-10-16T11:24:22.866 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:22.866 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T11:24:22.866 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:22.866 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-16T11:24:22.867 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-16T11:24:22.867 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:22.867 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T11:24:22.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.867+0000 7fe5b8533640 1 -- 172.21.15.84:0/1718145724 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fe59c06af60 msgr2=0x7fe59c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:22.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.867+0000 7fe5b8533640 1 --2- 172.21.15.84:0/1718145724 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fe59c06af60 0x7fe59c06d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0007fd0 tx=0x7fe5a0009040 comp rx=0 tx=0).stop 2023-10-16T11:24:22.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.867+0000 7fe5b8533640 1 -- 172.21.15.84:0/1718145724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5b015aaa0 msgr2=0x7fe5b006cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:22.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.867+0000 7fe5b8533640 1 --2- 172.21.15.84:0/1718145724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5b015aaa0 0x7fe5b006cc80 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4000c00 tx=0x7fe5a4004730 comp rx=0 tx=0).stop 2023-10-16T11:24:22.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.867+0000 7fe5b8533640 1 -- 172.21.15.84:0/1718145724 shutdown_connections 2023-10-16T11:24:22.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.867+0000 7fe5b8533640 1 --2- 172.21.15.84:0/1718145724 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fe59c06af60 0x7fe59c06d420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:22.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.867+0000 7fe5b8533640 1 --2- 172.21.15.84:0/1718145724 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b015ae80 0x7fe5b006d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:22.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.867+0000 7fe5b8533640 1 --2- 172.21.15.84:0/1718145724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5b015aaa0 0x7fe5b006cc80 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:22.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.867+0000 7fe5b8533640 1 --2- 172.21.15.84:0/1718145724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5b00ce390 0x7fe5b0155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:22.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.867+0000 7fe5b8533640 1 -- 172.21.15.84:0/1718145724 >> 172.21.15.84:0/1718145724 conn(0x7fe5b00cb410 msgr2=0x7fe5b00b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:22.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.867+0000 7fe5b8533640 1 -- 172.21.15.84:0/1718145724 shutdown_connections 2023-10-16T11:24:22.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:22.867+0000 7fe5b8533640 1 -- 172.21.15.84:0/1718145724 wait complete. 2023-10-16T11:24:22.886 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T11:24:22.886 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T11:24:23.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.099+0000 7f20f751d640 1 Processor -- start 2023-10-16T11:24:23.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.099+0000 7f20f751d640 1 -- start start 2023-10-16T11:24:23.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.101+0000 7f20f751d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20f00ce690 0x7f20f00c9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:23.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.101+0000 7f20f751d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f20f00ae220 0x7f20f006e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:23.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.101+0000 7f20f751d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f00c9d60 con 0x7f20f006adb0 2023-10-16T11:24:23.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.101+0000 7f20f751d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f006e6b0 con 0x7f20f00ae220 2023-10-16T11:24:23.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.101+0000 7f20f751d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00cae30 con 0x7f20f00ce690 2023-10-16T11:24:23.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.101+0000 7f20f4ab8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f20f00ae220 0x7f20f006e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:23.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.101+0000 7f20f4ab8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f20f00ae220 0x7f20f006e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40370/0 (socket says 172.21.15.84:40370) 2023-10-16T11:24:23.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.101+0000 7f20f4ab8640 1 -- 172.21.15.84:0/3421712898 learned_addr learned my addr 172.21.15.84:0/3421712898 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:23.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.102+0000 7f20f4ab8640 1 -- 172.21.15.84:0/3421712898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20f00ce690 msgr2=0x7f20f00c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:23.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.102+0000 7f20f52b9640 1 --2- 172.21.15.84:0/3421712898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20f00ce690 0x7f20f00c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:23.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.102+0000 7f20f4ab8640 1 --2- 172.21.15.84:0/3421712898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20f00ce690 0x7f20f00c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:23.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.102+0000 7f20f4ab8640 1 -- 172.21.15.84:0/3421712898 >> v1:172.21.15.157:6789/0 conn(0x7f20f006adb0 legacy=0x7f20f0160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:23.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.102+0000 7f20f4ab8640 1 -- 172.21.15.84:0/3421712898 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f00cb110 con 0x7f20f00ae220 2023-10-16T11:24:23.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.103+0000 7f20f4ab8640 1 --2- 172.21.15.84:0/3421712898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20f00ae220 0x7f20f006e170 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009e50 tx=0x7f20ec02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b7fc7e8b678953 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:23.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.103+0000 7f20e7fff640 1 -- 172.21.15.84:0/3421712898 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec02c9e0 con 0x7f20f00ae220 2023-10-16T11:24:23.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.103+0000 7f20e7fff640 1 -- 172.21.15.84:0/3421712898 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ec02cb40 con 0x7f20f00ae220 2023-10-16T11:24:23.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.103+0000 7f20e7fff640 1 -- 172.21.15.84:0/3421712898 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec0343f0 con 0x7f20f00ae220 2023-10-16T11:24:23.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.104+0000 7f20f751d640 1 -- 172.21.15.84:0/3421712898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20f00ae220 msgr2=0x7f20f006e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:23.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.104+0000 7f20f751d640 1 --2- 172.21.15.84:0/3421712898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20f00ae220 0x7f20f006e170 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009e50 tx=0x7f20ec02be40 comp rx=0 tx=0).stop 2023-10-16T11:24:23.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.104+0000 7f20f751d640 1 -- 172.21.15.84:0/3421712898 shutdown_connections 2023-10-16T11:24:23.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.104+0000 7f20f751d640 1 --2- 172.21.15.84:0/3421712898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20f00ae220 0x7f20f006e170 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:23.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.104+0000 7f20f751d640 1 --2- 172.21.15.84:0/3421712898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20f00ce690 0x7f20f00c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:23.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.104+0000 7f20f751d640 1 -- 172.21.15.84:0/3421712898 >> 172.21.15.84:0/3421712898 conn(0x7f20f00b0cc0 msgr2=0x7f20f00b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:23.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.104+0000 7f20f751d640 1 -- 172.21.15.84:0/3421712898 shutdown_connections 2023-10-16T11:24:23.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.104+0000 7f20f751d640 1 -- 172.21.15.84:0/3421712898 wait complete. 2023-10-16T11:24:23.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.106+0000 7f20f751d640 1 Processor -- start 2023-10-16T11:24:23.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.106+0000 7f20f751d640 1 -- start start 2023-10-16T11:24:23.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.107+0000 7f20f751d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20f006adb0 0x7f20f0154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:23.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.107+0000 7f20f751d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20f00ae220 0x7f20f006bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:23.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.107+0000 7f20f751d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20f00ce690 0x7f20f006c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:23.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.107+0000 7f20f751d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f0154680 con 0x7f20f00ae220 2023-10-16T11:24:23.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.107+0000 7f20f751d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f009e050 con 0x7f20f00ce690 2023-10-16T11:24:23.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.107+0000 7f20f751d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f009d440 con 0x7f20f006adb0 2023-10-16T11:24:23.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.107+0000 7f20f4ab8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20f00ae220 0x7f20f006bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:23.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.107+0000 7f20f4ab8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20f00ae220 0x7f20f006bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40386/0 (socket says 172.21.15.84:40386) 2023-10-16T11:24:23.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.107+0000 7f20f4ab8640 1 -- 172.21.15.84:0/2520536688 learned_addr learned my addr 172.21.15.84:0/2520536688 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:23.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.107+0000 7f20f52b9640 1 --2- 172.21.15.84:0/2520536688 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20f006adb0 0x7f20f0154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:23.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.107+0000 7f20f4ab8640 1 -- 172.21.15.84:0/2520536688 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20f006adb0 msgr2=0x7f20f0154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:23.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.107+0000 7f20f4ab8640 1 --2- 172.21.15.84:0/2520536688 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20f006adb0 0x7f20f0154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:23.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.107+0000 7f20f4ab8640 1 -- 172.21.15.84:0/2520536688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20f00ce690 msgr2=0x7f20f006c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:23.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.107+0000 7f20f4ab8640 1 --2- 172.21.15.84:0/2520536688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20f00ce690 0x7f20f006c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:23.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.107+0000 7f20f4ab8640 1 -- 172.21.15.84:0/2520536688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec009b30 con 0x7f20f00ae220 2023-10-16T11:24:23.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.108+0000 7f20f4ab8640 1 --2- 172.21.15.84:0/2520536688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20f00ae220 0x7f20f006bdb0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009e50 tx=0x7f20ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:23.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.108+0000 7f20e5ffb640 1 -- 172.21.15.84:0/2520536688 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec03e450 con 0x7f20f00ae220 2023-10-16T11:24:23.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.108+0000 7f20e5ffb640 1 -- 172.21.15.84:0/2520536688 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ec03e5b0 con 0x7f20f00ae220 2023-10-16T11:24:23.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.108+0000 7f20e5ffb640 1 -- 172.21.15.84:0/2520536688 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec03e960 con 0x7f20f00ae220 2023-10-16T11:24:23.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.108+0000 7f20f751d640 1 -- 172.21.15.84:0/2520536688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f009d720 con 0x7f20f00ae220 2023-10-16T11:24:23.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.108+0000 7f20f751d640 1 -- 172.21.15.84:0/2520536688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f009dce0 con 0x7f20f00ae220 2023-10-16T11:24:23.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.108+0000 7f20f751d640 1 -- 172.21.15.84:0/2520536688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f009e200 con 0x7f20f00ae220 2023-10-16T11:24:23.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.109+0000 7f20e5ffb640 1 -- 172.21.15.84:0/2520536688 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f20ec03eac0 con 0x7f20f00ae220 2023-10-16T11:24:23.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.110+0000 7f20e5ffb640 1 --2- 172.21.15.84:0/2520536688 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f20dc06af90 0x7f20dc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:23.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.110+0000 7f20e5ffb640 1 -- 172.21.15.84:0/2520536688 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f20ec0ac340 con 0x7f20f00ae220 2023-10-16T11:24:23.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.110+0000 7f20f52b9640 1 --2- 172.21.15.84:0/2520536688 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f20dc06af90 0x7f20dc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:23.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.111+0000 7f20f52b9640 1 --2- 172.21.15.84:0/2520536688 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f20dc06af90 0x7f20dc06d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f20f00afc70 tx=0x7f20e0008040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:23.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.114+0000 7f20e5ffb640 1 -- 172.21.15.84:0/2520536688 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f20f009e200 con 0x7f20f00ae220 2023-10-16T11:24:23.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:23.306+0000 7f20f751d640 1 -- 172.21.15.84:0/2520536688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f20f009dea0 con 0x7f20f00ae220 2023-10-16T11:24:24.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.043+0000 7f20e5ffb640 1 -- 172.21.15.84:0/2520536688 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v214) v1 ==== 138+0+0 (secure 0 0 0) 0x7f20ec077600 con 0x7f20f00ae220 2023-10-16T11:24:24.045 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-16T11:24:24.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.046+0000 7f20f751d640 1 -- 172.21.15.84:0/2520536688 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f20dc06af90 msgr2=0x7f20dc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:24.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.046+0000 7f20f751d640 1 --2- 172.21.15.84:0/2520536688 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f20dc06af90 0x7f20dc06d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f20f00afc70 tx=0x7f20e0008040 comp rx=0 tx=0).stop 2023-10-16T11:24:24.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.047+0000 7f20f751d640 1 -- 172.21.15.84:0/2520536688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20f00ae220 msgr2=0x7f20f006bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:24.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.047+0000 7f20f751d640 1 --2- 172.21.15.84:0/2520536688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20f00ae220 0x7f20f006bdb0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009e50 tx=0x7f20ec004a40 comp rx=0 tx=0).stop 2023-10-16T11:24:24.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.047+0000 7f20f751d640 1 -- 172.21.15.84:0/2520536688 shutdown_connections 2023-10-16T11:24:24.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.047+0000 7f20f751d640 1 --2- 172.21.15.84:0/2520536688 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f20dc06af90 0x7f20dc06d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:24.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.047+0000 7f20f751d640 1 --2- 172.21.15.84:0/2520536688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20f00ce690 0x7f20f006c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:24.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.047+0000 7f20f751d640 1 --2- 172.21.15.84:0/2520536688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20f00ae220 0x7f20f006bdb0 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:24.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.047+0000 7f20f751d640 1 --2- 172.21.15.84:0/2520536688 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20f006adb0 0x7f20f0154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:24.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.047+0000 7f20f751d640 1 -- 172.21.15.84:0/2520536688 >> 172.21.15.84:0/2520536688 conn(0x7f20f00b0cc0 msgr2=0x7f20f015f880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:24.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.047+0000 7f20f751d640 1 -- 172.21.15.84:0/2520536688 shutdown_connections 2023-10-16T11:24:24.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.047+0000 7f20f751d640 1 -- 172.21.15.84:0/2520536688 wait complete. 2023-10-16T11:24:24.075 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T11:24:24.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.279+0000 7f1db28bb640 1 Processor -- start 2023-10-16T11:24:24.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.279+0000 7f1db28bb640 1 -- start start 2023-10-16T11:24:24.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.280+0000 7f1db28bb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1dac072170 0x7f1dac167b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:24.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.281+0000 7f1db28bb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dac0d12f0 con 0x7f1dac0cdf30 2023-10-16T11:24:24.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.281+0000 7f1db28bb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dac0d06e0 con 0x7f1dac15cc80 2023-10-16T11:24:24.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.281+0000 7f1db28bb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dac0d0500 con 0x7f1dac072170 2023-10-16T11:24:24.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.281+0000 7f1db10b8640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1dac0cdf30 0x7f1dac071a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58792/0 (socket says 172.21.15.84:58792) 2023-10-16T11:24:24.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.281+0000 7f1db10b8640 1 -- 172.21.15.84:0/1183989702 learned_addr learned my addr 172.21.15.84:0/1183989702 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:24.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.281+0000 7f1db20ba640 1 --2- 172.21.15.84:0/1183989702 >> v2:172.21.15.157:3300/0 conn(0x7f1dac072170 0x7f1dac167b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:24.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.281+0000 7f1db08b7640 1 -- 172.21.15.84:0/1183989702 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1007443873 0 0) 0x7f1dac0d12f0 con 0x7f1dac0cdf30 2023-10-16T11:24:24.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.281+0000 7f1db08b7640 1 -- 172.21.15.84:0/1183989702 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d98003520 con 0x7f1dac0cdf30 2023-10-16T11:24:24.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.282+0000 7f1db08b7640 1 -- 172.21.15.84:0/1183989702 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3492132548 0 0) 0x7f1d98003520 con 0x7f1dac0cdf30 2023-10-16T11:24:24.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.282+0000 7f1db08b7640 1 -- 172.21.15.84:0/1183989702 >> v2:172.21.15.157:3300/0 conn(0x7f1dac072170 msgr2=0x7f1dac167b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:24.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.282+0000 7f1db08b7640 1 --2- 172.21.15.84:0/1183989702 >> v2:172.21.15.157:3300/0 conn(0x7f1dac072170 0x7f1dac167b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:24.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.282+0000 7f1db08b7640 1 -- 172.21.15.84:0/1183989702 >> v1:172.21.15.157:6789/0 conn(0x7f1dac15cc80 legacy=0x7f1dac0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:24.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.282+0000 7f1db08b7640 1 -- 172.21.15.84:0/1183989702 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac0d09c0 con 0x7f1dac0cdf30 2023-10-16T11:24:24.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.282+0000 7f1db08b7640 1 -- 172.21.15.84:0/1183989702 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f1d9c001670 con 0x7f1dac0cdf30 2023-10-16T11:24:24.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.282+0000 7f1db08b7640 1 -- 172.21.15.84:0/1183989702 >> v1:172.21.15.84:6789/0 conn(0x7f1dac0cdf30 legacy=0x7f1dac071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:24.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.282+0000 7f1db08b7640 1 --2- 172.21.15.84:0/1183989702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d980039c0 0x7f1d98003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:24.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.282+0000 7f1db08b7640 1 --2- 172.21.15.84:0/1183989702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d980047c0 0x7f1d98004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:24.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.282+0000 7f1db08b7640 1 --2- 172.21.15.84:0/1183989702 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d98005180 0x7f1d9800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:24.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.282+0000 7f1db08b7640 1 -- 172.21.15.84:0/1183989702 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac0d0500 con 0x7f1d980047c0 2023-10-16T11:24:24.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.282+0000 7f1db08b7640 1 -- 172.21.15.84:0/1183989702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9800c080 con 0x7f1d980039c0 2023-10-16T11:24:24.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.282+0000 7f1db08b7640 1 -- 172.21.15.84:0/1183989702 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9800c240 con 0x7f1d98005180 2023-10-16T11:24:24.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.282+0000 7f1db10b8640 1 --2- 172.21.15.84:0/1183989702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d980047c0 0x7f1d98004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:24.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.283+0000 7f1db18b9640 1 --2- 172.21.15.84:0/1183989702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d980039c0 0x7f1d98003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:24.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.283+0000 7f1db10b8640 1 -- 172.21.15.84:0/1183989702 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d98005180 msgr2=0x7f1d9800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:24.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.283+0000 7f1db20ba640 1 --2- 172.21.15.84:0/1183989702 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d98005180 0x7f1d9800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:24.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.283+0000 7f1db10b8640 1 --2- 172.21.15.84:0/1183989702 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d98005180 0x7f1d9800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:24.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.283+0000 7f1db10b8640 1 -- 172.21.15.84:0/1183989702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d980039c0 msgr2=0x7f1d98003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:24.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.283+0000 7f1db10b8640 1 --2- 172.21.15.84:0/1183989702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d980039c0 0x7f1d98003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:24.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.283+0000 7f1db10b8640 1 -- 172.21.15.84:0/1183989702 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d98003520 con 0x7f1d980047c0 2023-10-16T11:24:24.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.283+0000 7f1db20ba640 1 -- 172.21.15.84:0/1183989702 reap_dead start 2023-10-16T11:24:24.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.284+0000 7f1db10b8640 1 --2- 172.21.15.84:0/1183989702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d980047c0 0x7f1d98004c40 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00c0e0 tx=0x7f1d980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101b02a27a2acd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:24.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.284+0000 7f1db08b7640 1 -- 172.21.15.84:0/1183989702 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c005ad0 con 0x7f1d980047c0 2023-10-16T11:24:24.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.284+0000 7f1db08b7640 1 -- 172.21.15.84:0/1183989702 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c005c30 con 0x7f1d980047c0 2023-10-16T11:24:24.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.284+0000 7f1db08b7640 1 -- 172.21.15.84:0/1183989702 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c0072b0 con 0x7f1d980047c0 2023-10-16T11:24:24.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.285+0000 7f1db28bb640 1 -- 172.21.15.84:0/1183989702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d980047c0 msgr2=0x7f1d98004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:24.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.285+0000 7f1db28bb640 1 --2- 172.21.15.84:0/1183989702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d980047c0 0x7f1d98004c40 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00c0e0 tx=0x7f1d980057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:24.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.285+0000 7f1db28bb640 1 -- 172.21.15.84:0/1183989702 shutdown_connections 2023-10-16T11:24:24.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.285+0000 7f1db28bb640 1 --2- 172.21.15.84:0/1183989702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d980047c0 0x7f1d98004c40 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:24.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.285+0000 7f1db28bb640 1 -- 172.21.15.84:0/1183989702 >> 172.21.15.84:0/1183989702 conn(0x7f1dac065ed0 msgr2=0x7f1dac0662e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:24.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.285+0000 7f1db28bb640 1 -- 172.21.15.84:0/1183989702 shutdown_connections 2023-10-16T11:24:24.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.285+0000 7f1db28bb640 1 -- 172.21.15.84:0/1183989702 wait complete. 2023-10-16T11:24:24.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.287+0000 7f1db28bb640 1 Processor -- start 2023-10-16T11:24:24.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.287+0000 7f1db28bb640 1 -- start start 2023-10-16T11:24:24.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.288+0000 7f1db28bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d980047c0 0x7f1dac1f1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:24.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.288+0000 7f1db28bb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dac0b43f0 0x7f1dac15ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:24.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.288+0000 7f1db28bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1dac15b3a0 0x7f1dac0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:24.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.288+0000 7f1db28bb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac14f4b0 con 0x7f1dac0b43f0 2023-10-16T11:24:24.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.288+0000 7f1db28bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac15b820 con 0x7f1d980047c0 2023-10-16T11:24:24.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.288+0000 7f1db28bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac0af7f0 con 0x7f1dac15b3a0 2023-10-16T11:24:24.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.289+0000 7f1db10b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dac0b43f0 0x7f1dac15ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:24.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.289+0000 7f1db10b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dac0b43f0 0x7f1dac15ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40408/0 (socket says 172.21.15.84:40408) 2023-10-16T11:24:24.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.289+0000 7f1db10b8640 1 -- 172.21.15.84:0/3697278452 learned_addr learned my addr 172.21.15.84:0/3697278452 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:24.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.289+0000 7f1db18b9640 1 --2- 172.21.15.84:0/3697278452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d980047c0 0x7f1dac1f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:24.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.289+0000 7f1db20ba640 1 --2- 172.21.15.84:0/3697278452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1dac15b3a0 0x7f1dac0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:24.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.289+0000 7f1db10b8640 1 -- 172.21.15.84:0/3697278452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1dac15b3a0 msgr2=0x7f1dac0aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:24.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.289+0000 7f1db10b8640 1 --2- 172.21.15.84:0/3697278452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1dac15b3a0 0x7f1dac0aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:24.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.289+0000 7f1db10b8640 1 -- 172.21.15.84:0/3697278452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d980047c0 msgr2=0x7f1dac1f1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:24.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.289+0000 7f1db10b8640 1 --2- 172.21.15.84:0/3697278452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d980047c0 0x7f1dac1f1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:24.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.289+0000 7f1db10b8640 1 -- 172.21.15.84:0/3697278452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac0b3630 con 0x7f1dac0b43f0 2023-10-16T11:24:24.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.290+0000 7f1db10b8640 1 --2- 172.21.15.84:0/3697278452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dac0b43f0 0x7f1dac15ae60 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c000c00 tx=0x7f1d9c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:24.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.290+0000 7f1da27fc640 1 -- 172.21.15.84:0/3697278452 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c03c040 con 0x7f1dac0b43f0 2023-10-16T11:24:24.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.290+0000 7f1da27fc640 1 -- 172.21.15.84:0/3697278452 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c007560 con 0x7f1dac0b43f0 2023-10-16T11:24:24.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.290+0000 7f1da27fc640 1 -- 172.21.15.84:0/3697278452 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c02c700 con 0x7f1dac0b43f0 2023-10-16T11:24:24.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.290+0000 7f1db28bb640 1 -- 172.21.15.84:0/3697278452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dac0b3910 con 0x7f1dac0b43f0 2023-10-16T11:24:24.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.290+0000 7f1db28bb640 1 -- 172.21.15.84:0/3697278452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dac0b3ed0 con 0x7f1dac0b43f0 2023-10-16T11:24:24.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.291+0000 7f1db28bb640 1 -- 172.21.15.84:0/3697278452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dac0b4060 con 0x7f1dac0b43f0 2023-10-16T11:24:24.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.293+0000 7f1da27fc640 1 -- 172.21.15.84:0/3697278452 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1d9c032070 con 0x7f1dac0b43f0 2023-10-16T11:24:24.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.293+0000 7f1da27fc640 1 --2- 172.21.15.84:0/3697278452 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f1d9006afc0 0x7f1d9006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:24.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.293+0000 7f1da27fc640 1 -- 172.21.15.84:0/3697278452 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5255+0+0 (secure 0 0 0) 0x7f1d9c0ab1d0 con 0x7f1dac0b43f0 2023-10-16T11:24:24.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.294+0000 7f1db18b9640 1 --2- 172.21.15.84:0/3697278452 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f1d9006afc0 0x7f1d9006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:24.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.294+0000 7f1db18b9640 1 --2- 172.21.15.84:0/3697278452 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f1d9006afc0 0x7f1d9006d480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1da8000c00 tx=0x7f1da8009040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:24.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.297+0000 7f1da27fc640 1 -- 172.21.15.84:0/3697278452 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1d9c0763a0 con 0x7f1dac0b43f0 2023-10-16T11:24:24.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:24.495+0000 7f1db28bb640 1 -- 172.21.15.84:0/3697278452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1dac04e9b0 con 0x7f1dac0b43f0 2023-10-16T11:24:25.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.055+0000 7f1da27fc640 1 -- 172.21.15.84:0/3697278452 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v215) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1d9c04f0d0 con 0x7f1dac0b43f0 2023-10-16T11:24:25.058 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-16T11:24:25.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.059+0000 7f1db28bb640 1 -- 172.21.15.84:0/3697278452 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f1d9006afc0 msgr2=0x7f1d9006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:25.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.059+0000 7f1db28bb640 1 --2- 172.21.15.84:0/3697278452 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f1d9006afc0 0x7f1d9006d480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1da8000c00 tx=0x7f1da8009040 comp rx=0 tx=0).stop 2023-10-16T11:24:25.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.059+0000 7f1db28bb640 1 -- 172.21.15.84:0/3697278452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dac0b43f0 msgr2=0x7f1dac15ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:25.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.059+0000 7f1db28bb640 1 --2- 172.21.15.84:0/3697278452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dac0b43f0 0x7f1dac15ae60 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c000c00 tx=0x7f1d9c02e070 comp rx=0 tx=0).stop 2023-10-16T11:24:25.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.061+0000 7f1db28bb640 1 -- 172.21.15.84:0/3697278452 shutdown_connections 2023-10-16T11:24:25.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.061+0000 7f1db28bb640 1 --2- 172.21.15.84:0/3697278452 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f1d9006afc0 0x7f1d9006d480 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:25.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.061+0000 7f1db28bb640 1 --2- 172.21.15.84:0/3697278452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1dac15b3a0 0x7f1dac0aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:25.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.061+0000 7f1db28bb640 1 --2- 172.21.15.84:0/3697278452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dac0b43f0 0x7f1dac15ae60 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:25.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.061+0000 7f1db28bb640 1 --2- 172.21.15.84:0/3697278452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d980047c0 0x7f1dac1f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:25.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.061+0000 7f1db28bb640 1 -- 172.21.15.84:0/3697278452 >> 172.21.15.84:0/3697278452 conn(0x7f1dac065ed0 msgr2=0x7f1dac071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:25.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.063+0000 7f1db28bb640 1 -- 172.21.15.84:0/3697278452 shutdown_connections 2023-10-16T11:24:25.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.064+0000 7f1db28bb640 1 -- 172.21.15.84:0/3697278452 wait complete. 2023-10-16T11:24:25.100 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T11:24:25.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.287+0000 7ff603d05640 1 Processor -- start 2023-10-16T11:24:25.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.288+0000 7ff603d05640 1 -- start start 2023-10-16T11:24:25.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.289+0000 7ff603d05640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff5fc06a4b0 0x7ff5fc0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:25.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.289+0000 7ff603d05640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff5fc0c9ca0 0x7ff5fc0ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:25.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.289+0000 7ff603d05640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5fc0cb030 con 0x7ff5fc15ad70 2023-10-16T11:24:25.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.289+0000 7ff603d05640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5fc0cae30 con 0x7ff5fc0c9ca0 2023-10-16T11:24:25.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.289+0000 7ff603d05640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5fc0c91c0 con 0x7ff5fc06a4b0 2023-10-16T11:24:25.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.289+0000 7ff6022a2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff5fc0c9ca0 0x7ff5fc0ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:25.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.289+0000 7ff6022a2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff5fc0c9ca0 0x7ff5fc0ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40424/0 (socket says 172.21.15.84:40424) 2023-10-16T11:24:25.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.289+0000 7ff6022a2640 1 -- 172.21.15.84:0/1579795730 learned_addr learned my addr 172.21.15.84:0/1579795730 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:25.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.290+0000 7ff6012a0640 1 --2- 172.21.15.84:0/1579795730 >> v2:172.21.15.157:3300/0 conn(0x7ff5fc06a4b0 0x7ff5fc0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:25.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.290+0000 7ff6022a2640 1 -- 172.21.15.84:0/1579795730 >> v2:172.21.15.157:3300/0 conn(0x7ff5fc06a4b0 msgr2=0x7ff5fc0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:25.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.290+0000 7ff6022a2640 1 --2- 172.21.15.84:0/1579795730 >> v2:172.21.15.157:3300/0 conn(0x7ff5fc06a4b0 0x7ff5fc0ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:25.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.290+0000 7ff6022a2640 1 -- 172.21.15.84:0/1579795730 >> v1:172.21.15.157:6789/0 conn(0x7ff5fc15ad70 legacy=0x7ff5fc069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:25.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.290+0000 7ff6022a2640 1 -- 172.21.15.84:0/1579795730 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc0c94a0 con 0x7ff5fc0c9ca0 2023-10-16T11:24:25.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.291+0000 7ff6022a2640 1 --2- 172.21.15.84:0/1579795730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5fc0c9ca0 0x7ff5fc0ca8f0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0ce870 tx=0x7ff5f402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbdc19a36ec1e83e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:25.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.291+0000 7ff600a9f640 1 -- 172.21.15.84:0/1579795730 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f402ca80 con 0x7ff5fc0c9ca0 2023-10-16T11:24:25.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.291+0000 7ff600a9f640 1 -- 172.21.15.84:0/1579795730 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f402cbe0 con 0x7ff5fc0c9ca0 2023-10-16T11:24:25.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.291+0000 7ff600a9f640 1 -- 172.21.15.84:0/1579795730 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f40343f0 con 0x7ff5fc0c9ca0 2023-10-16T11:24:25.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.292+0000 7ff603d05640 1 -- 172.21.15.84:0/1579795730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5fc0c9ca0 msgr2=0x7ff5fc0ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:25.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.292+0000 7ff603d05640 1 --2- 172.21.15.84:0/1579795730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5fc0c9ca0 0x7ff5fc0ca8f0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0ce870 tx=0x7ff5f402bee0 comp rx=0 tx=0).stop 2023-10-16T11:24:25.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.292+0000 7ff603d05640 1 -- 172.21.15.84:0/1579795730 shutdown_connections 2023-10-16T11:24:25.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.292+0000 7ff603d05640 1 --2- 172.21.15.84:0/1579795730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5fc0c9ca0 0x7ff5fc0ca8f0 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:25.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.292+0000 7ff603d05640 1 --2- 172.21.15.84:0/1579795730 >> v2:172.21.15.157:3300/0 conn(0x7ff5fc06a4b0 0x7ff5fc0ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:25.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.292+0000 7ff603d05640 1 -- 172.21.15.84:0/1579795730 >> 172.21.15.84:0/1579795730 conn(0x7ff5fc0ce9b0 msgr2=0x7ff5fc0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:25.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.292+0000 7ff603d05640 1 -- 172.21.15.84:0/1579795730 shutdown_connections 2023-10-16T11:24:25.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.292+0000 7ff603d05640 1 -- 172.21.15.84:0/1579795730 wait complete. 2023-10-16T11:24:25.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.294+0000 7ff603d05640 1 Processor -- start 2023-10-16T11:24:25.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.294+0000 7ff603d05640 1 -- start start 2023-10-16T11:24:25.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.295+0000 7ff603d05640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5fc06a4b0 0x7ff5fc15a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:25.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.295+0000 7ff603d05640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5fc0c9ca0 0x7ff5fc0c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:25.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.295+0000 7ff603d05640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5fc15ad70 0x7ff5fc0b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:25.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.295+0000 7ff603d05640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0c9050 con 0x7ff5fc15ad70 2023-10-16T11:24:25.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.295+0000 7ff603d05640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0c4e50 con 0x7ff5fc0c9ca0 2023-10-16T11:24:25.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.295+0000 7ff603d05640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0c5130 con 0x7ff5fc06a4b0 2023-10-16T11:24:25.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.295+0000 7ff6022a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5fc15ad70 0x7ff5fc0b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:25.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.295+0000 7ff6022a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5fc15ad70 0x7ff5fc0b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40440/0 (socket says 172.21.15.84:40440) 2023-10-16T11:24:25.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.295+0000 7ff6022a2640 1 -- 172.21.15.84:0/27776624 learned_addr learned my addr 172.21.15.84:0/27776624 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:25.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.296+0000 7ff6022a2640 1 -- 172.21.15.84:0/27776624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5fc06a4b0 msgr2=0x7ff5fc15a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:25.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.296+0000 7ff6022a2640 1 --2- 172.21.15.84:0/27776624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5fc06a4b0 0x7ff5fc15a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:25.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.296+0000 7ff6022a2640 1 -- 172.21.15.84:0/27776624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5fc0c9ca0 msgr2=0x7ff5fc0c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:25.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.296+0000 7ff6022a2640 1 --2- 172.21.15.84:0/27776624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5fc0c9ca0 0x7ff5fc0c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:25.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.296+0000 7ff6022a2640 1 -- 172.21.15.84:0/27776624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f4009b30 con 0x7ff5fc15ad70 2023-10-16T11:24:25.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.296+0000 7ff6022a2640 1 --2- 172.21.15.84:0/27776624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5fc15ad70 0x7ff5fc0b2e80 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4036040 tx=0x7ff5f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:25.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.296+0000 7ff5f27fc640 1 -- 172.21.15.84:0/27776624 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4009510 con 0x7ff5fc15ad70 2023-10-16T11:24:25.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.296+0000 7ff5f27fc640 1 -- 172.21.15.84:0/27776624 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f403e450 con 0x7ff5fc15ad70 2023-10-16T11:24:25.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.296+0000 7ff5f27fc640 1 -- 172.21.15.84:0/27776624 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f403e7e0 con 0x7ff5fc15ad70 2023-10-16T11:24:25.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.296+0000 7ff603d05640 1 -- 172.21.15.84:0/27776624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5fc0c5410 con 0x7ff5fc15ad70 2023-10-16T11:24:25.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.296+0000 7ff603d05640 1 -- 172.21.15.84:0/27776624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5fc0c2820 con 0x7ff5fc15ad70 2023-10-16T11:24:25.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.297+0000 7ff603d05640 1 -- 172.21.15.84:0/27776624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5fc0c2a10 con 0x7ff5fc15ad70 2023-10-16T11:24:25.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.299+0000 7ff5f27fc640 1 -- 172.21.15.84:0/27776624 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff5f402cd50 con 0x7ff5fc15ad70 2023-10-16T11:24:25.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.300+0000 7ff5f27fc640 1 --2- 172.21.15.84:0/27776624 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff5e006af90 0x7ff5e006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:25.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.300+0000 7ff5f27fc640 1 -- 172.21.15.84:0/27776624 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 5637+0+0 (secure 0 0 0) 0x7ff5f40ac7a0 con 0x7ff5fc15ad70 2023-10-16T11:24:25.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.300+0000 7ff601aa1640 1 --2- 172.21.15.84:0/27776624 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff5e006af90 0x7ff5e006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:25.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.301+0000 7ff601aa1640 1 --2- 172.21.15.84:0/27776624 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff5e006af90 0x7ff5e006d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8006fd0 tx=0x7ff5f8006bc0 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:25.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.304+0000 7ff5f27fc640 1 -- 172.21.15.84:0/27776624 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff5f4077910 con 0x7ff5fc15ad70 2023-10-16T11:24:25.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:25.528+0000 7ff603d05640 1 -- 172.21.15.84:0/27776624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff5fc04e160 con 0x7ff5fc15ad70 2023-10-16T11:24:26.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.067+0000 7ff5f27fc640 1 -- 172.21.15.84:0/27776624 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v110) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff5f4033e80 con 0x7ff5fc15ad70 2023-10-16T11:24:26.076 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 26 and data pool 27 2023-10-16T11:24:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.077+0000 7ff603d05640 1 -- 172.21.15.84:0/27776624 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff5e006af90 msgr2=0x7ff5e006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.077+0000 7ff603d05640 1 --2- 172.21.15.84:0/27776624 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff5e006af90 0x7ff5e006d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8006fd0 tx=0x7ff5f8006bc0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.077+0000 7ff603d05640 1 -- 172.21.15.84:0/27776624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5fc15ad70 msgr2=0x7ff5fc0b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.077+0000 7ff603d05640 1 --2- 172.21.15.84:0/27776624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5fc15ad70 0x7ff5fc0b2e80 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4036040 tx=0x7ff5f4004a40 comp rx=0 tx=0).stop 2023-10-16T11:24:26.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.079+0000 7ff603d05640 1 -- 172.21.15.84:0/27776624 shutdown_connections 2023-10-16T11:24:26.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.079+0000 7ff603d05640 1 --2- 172.21.15.84:0/27776624 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff5e006af90 0x7ff5e006d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.079+0000 7ff603d05640 1 --2- 172.21.15.84:0/27776624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5fc15ad70 0x7ff5fc0b2e80 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.079+0000 7ff603d05640 1 --2- 172.21.15.84:0/27776624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5fc0c9ca0 0x7ff5fc0c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.079+0000 7ff603d05640 1 --2- 172.21.15.84:0/27776624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5fc06a4b0 0x7ff5fc15a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.079+0000 7ff603d05640 1 -- 172.21.15.84:0/27776624 >> 172.21.15.84:0/27776624 conn(0x7ff5fc0ce9b0 msgr2=0x7ff5fc15d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:26.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.080+0000 7ff603d05640 1 -- 172.21.15.84:0/27776624 shutdown_connections 2023-10-16T11:24:26.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.082+0000 7ff603d05640 1 -- 172.21.15.84:0/27776624 wait complete. 2023-10-16T11:24:26.116 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:24:26.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.293+0000 7ff499f90640 1 Processor -- start 2023-10-16T11:24:26.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.293+0000 7ff499f90640 1 -- start start 2023-10-16T11:24:26.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.294+0000 7ff499f90640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff4940cd0c0 0x7ff4941615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:26.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.294+0000 7ff499f90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4940b0410 0x7ff4940b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:26.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.294+0000 7ff499f90640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff49415d3f0 0x7ff49415d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:26.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.295+0000 7ff499f90640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4940b0db0 con 0x7ff49415d3f0 2023-10-16T11:24:26.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.295+0000 7ff499f90640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff49415e3e0 con 0x7ff4940cd0c0 2023-10-16T11:24:26.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.295+0000 7ff499f90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4940cd4a0 con 0x7ff4940b0410 2023-10-16T11:24:26.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.295+0000 7ff493fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff49415d3f0 0x7ff49415d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:26.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.295+0000 7ff493fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff49415d3f0 0x7ff49415d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56174/0 (socket says 172.21.15.84:56174) 2023-10-16T11:24:26.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.295+0000 7ff493fff640 1 -- 172.21.15.84:0/1530644696 learned_addr learned my addr 172.21.15.84:0/1530644696 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:26.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.295+0000 7ff493fff640 1 -- 172.21.15.84:0/1530644696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4940b0410 msgr2=0x7ff4940b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:26.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.295+0000 7ff4937fe640 1 --2- 172.21.15.84:0/1530644696 >> v2:172.21.15.157:3300/0 conn(0x7ff4940cd0c0 0x7ff4941615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:26.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.295+0000 7ff48bfff640 1 --2- 172.21.15.84:0/1530644696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4940b0410 0x7ff4940b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:26.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.295+0000 7ff493fff640 1 --2- 172.21.15.84:0/1530644696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4940b0410 0x7ff4940b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.295+0000 7ff493fff640 1 -- 172.21.15.84:0/1530644696 >> v2:172.21.15.157:3300/0 conn(0x7ff4940cd0c0 msgr2=0x7ff4941615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:26.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.295+0000 7ff493fff640 1 --2- 172.21.15.84:0/1530644696 >> v2:172.21.15.157:3300/0 conn(0x7ff4940cd0c0 0x7ff4941615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.295+0000 7ff493fff640 1 -- 172.21.15.84:0/1530644696 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49415e100 con 0x7ff49415d3f0 2023-10-16T11:24:26.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.295+0000 7ff4937fe640 1 --2- 172.21.15.84:0/1530644696 >> v2:172.21.15.157:3300/0 conn(0x7ff4940cd0c0 0x7ff4941615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:26.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.295+0000 7ff48bfff640 1 --2- 172.21.15.84:0/1530644696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4940b0410 0x7ff4940b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:26.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.297+0000 7ff493fff640 1 --2- 172.21.15.84:0/1530644696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49415d3f0 0x7ff49415d880 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7ff4940cc4e0 tx=0x7ff48c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2ea2a476b78e70 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:26.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.297+0000 7ff492ffd640 1 -- 172.21.15.84:0/1530644696 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c02c9f0 con 0x7ff49415d3f0 2023-10-16T11:24:26.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.297+0000 7ff492ffd640 1 -- 172.21.15.84:0/1530644696 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48c02cb50 con 0x7ff49415d3f0 2023-10-16T11:24:26.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.297+0000 7ff492ffd640 1 -- 172.21.15.84:0/1530644696 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0343f0 con 0x7ff49415d3f0 2023-10-16T11:24:26.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.297+0000 7ff499f90640 1 -- 172.21.15.84:0/1530644696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49415d3f0 msgr2=0x7ff49415d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:26.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.297+0000 7ff499f90640 1 --2- 172.21.15.84:0/1530644696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49415d3f0 0x7ff49415d880 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7ff4940cc4e0 tx=0x7ff48c02be50 comp rx=0 tx=0).stop 2023-10-16T11:24:26.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.298+0000 7ff499f90640 1 -- 172.21.15.84:0/1530644696 shutdown_connections 2023-10-16T11:24:26.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.298+0000 7ff499f90640 1 --2- 172.21.15.84:0/1530644696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49415d3f0 0x7ff49415d880 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.298+0000 7ff499f90640 1 --2- 172.21.15.84:0/1530644696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4940b0410 0x7ff4940b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.298+0000 7ff499f90640 1 --2- 172.21.15.84:0/1530644696 >> v2:172.21.15.157:3300/0 conn(0x7ff4940cd0c0 0x7ff4941615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.298+0000 7ff499f90640 1 -- 172.21.15.84:0/1530644696 >> 172.21.15.84:0/1530644696 conn(0x7ff4940cb410 msgr2=0x7ff4940cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:26.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.298+0000 7ff499f90640 1 -- 172.21.15.84:0/1530644696 shutdown_connections 2023-10-16T11:24:26.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.298+0000 7ff499f90640 1 -- 172.21.15.84:0/1530644696 wait complete. 2023-10-16T11:24:26.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.299+0000 7ff499f90640 1 Processor -- start 2023-10-16T11:24:26.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.299+0000 7ff499f90640 1 -- start start 2023-10-16T11:24:26.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.300+0000 7ff499f90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4940b0410 0x7ff49415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:26.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.300+0000 7ff499f90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4940cd0c0 0x7ff4940c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:26.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.300+0000 7ff499f90640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49415d3f0 0x7ff4940c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:26.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.300+0000 7ff499f90640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4940c88b0 con 0x7ff49415d3f0 2023-10-16T11:24:26.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.300+0000 7ff499f90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4940c8b90 con 0x7ff4940b0410 2023-10-16T11:24:26.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.300+0000 7ff499f90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4940b1fc0 con 0x7ff4940cd0c0 2023-10-16T11:24:26.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.301+0000 7ff493fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49415d3f0 0x7ff4940c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:26.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.301+0000 7ff493fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49415d3f0 0x7ff4940c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56176/0 (socket says 172.21.15.84:56176) 2023-10-16T11:24:26.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.301+0000 7ff493fff640 1 -- 172.21.15.84:0/902765032 learned_addr learned my addr 172.21.15.84:0/902765032 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:26.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.301+0000 7ff493fff640 1 -- 172.21.15.84:0/902765032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4940cd0c0 msgr2=0x7ff4940c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:26.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.301+0000 7ff48bfff640 1 --2- 172.21.15.84:0/902765032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4940cd0c0 0x7ff4940c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:26.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.301+0000 7ff493fff640 1 --2- 172.21.15.84:0/902765032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4940cd0c0 0x7ff4940c7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.301+0000 7ff493fff640 1 -- 172.21.15.84:0/902765032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4940b0410 msgr2=0x7ff49415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:26.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.301+0000 7ff4937fe640 1 --2- 172.21.15.84:0/902765032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4940b0410 0x7ff49415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:26.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.301+0000 7ff493fff640 1 --2- 172.21.15.84:0/902765032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4940b0410 0x7ff49415c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.301+0000 7ff493fff640 1 -- 172.21.15.84:0/902765032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c009b30 con 0x7ff49415d3f0 2023-10-16T11:24:26.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.301+0000 7ff493fff640 1 --2- 172.21.15.84:0/902765032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49415d3f0 0x7ff4940c8230 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7ff48c036040 tx=0x7ff48c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:26.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.302+0000 7ff490ff9640 1 -- 172.21.15.84:0/902765032 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c009510 con 0x7ff49415d3f0 2023-10-16T11:24:26.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.302+0000 7ff490ff9640 1 -- 172.21.15.84:0/902765032 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48c03e450 con 0x7ff49415d3f0 2023-10-16T11:24:26.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.302+0000 7ff490ff9640 1 -- 172.21.15.84:0/902765032 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c03e800 con 0x7ff49415d3f0 2023-10-16T11:24:26.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.302+0000 7ff499f90640 1 -- 172.21.15.84:0/902765032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4940b22a0 con 0x7ff49415d3f0 2023-10-16T11:24:26.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.302+0000 7ff499f90640 1 -- 172.21.15.84:0/902765032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49415c920 con 0x7ff49415d3f0 2023-10-16T11:24:26.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.303+0000 7ff499f90640 1 -- 172.21.15.84:0/902765032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4940c8db0 con 0x7ff49415d3f0 2023-10-16T11:24:26.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.304+0000 7ff490ff9640 1 -- 172.21.15.84:0/902765032 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff48c03e960 con 0x7ff49415d3f0 2023-10-16T11:24:26.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.305+0000 7ff490ff9640 1 --2- 172.21.15.84:0/902765032 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff47406af90 0x7ff47406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:26.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.305+0000 7ff490ff9640 1 -- 172.21.15.84:0/902765032 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff48c0ac790 con 0x7ff49415d3f0 2023-10-16T11:24:26.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.305+0000 7ff4937fe640 1 --2- 172.21.15.84:0/902765032 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff47406af90 0x7ff47406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:26.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.306+0000 7ff4937fe640 1 --2- 172.21.15.84:0/902765032 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff47406af90 0x7ff47406d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff48400b840 tx=0x7ff484008040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:26.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.309+0000 7ff490ff9640 1 -- 172.21.15.84:0/902765032 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff48c0777b0 con 0x7ff49415d3f0 2023-10-16T11:24:26.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.488+0000 7ff499f90640 1 -- 172.21.15.84:0/902765032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4940b24c0 con 0x7ff49415d3f0 2023-10-16T11:24:26.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.492+0000 7ff490ff9640 1 -- 172.21.15.84:0/902765032 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v216) v1 ==== 74+0+16616 (secure 0 0 0) 0x7ff48c07add0 con 0x7ff49415d3f0 2023-10-16T11:24:26.494 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:26.494 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":216,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:24:25.530901+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:24:23.363032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:24:24.551270+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:24:26.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.494+0000 7ff499f90640 1 -- 172.21.15.84:0/902765032 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff47406af90 msgr2=0x7ff47406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:26.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.494+0000 7ff499f90640 1 --2- 172.21.15.84:0/902765032 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff47406af90 0x7ff47406d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff48400b840 tx=0x7ff484008040 comp rx=0 tx=0).stop 2023-10-16T11:24:26.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.494+0000 7ff499f90640 1 -- 172.21.15.84:0/902765032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49415d3f0 msgr2=0x7ff4940c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:26.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.494+0000 7ff499f90640 1 --2- 172.21.15.84:0/902765032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49415d3f0 0x7ff4940c8230 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7ff48c036040 tx=0x7ff48c004a40 comp rx=0 tx=0).stop 2023-10-16T11:24:26.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.495+0000 7ff499f90640 1 -- 172.21.15.84:0/902765032 shutdown_connections 2023-10-16T11:24:26.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.495+0000 7ff499f90640 1 --2- 172.21.15.84:0/902765032 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff47406af90 0x7ff47406d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.495+0000 7ff499f90640 1 --2- 172.21.15.84:0/902765032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49415d3f0 0x7ff4940c8230 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.495+0000 7ff499f90640 1 --2- 172.21.15.84:0/902765032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4940cd0c0 0x7ff4940c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.495+0000 7ff499f90640 1 --2- 172.21.15.84:0/902765032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4940b0410 0x7ff49415c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.495+0000 7ff499f90640 1 -- 172.21.15.84:0/902765032 >> 172.21.15.84:0/902765032 conn(0x7ff4940cb410 msgr2=0x7ff4940af760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:26.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.495+0000 7ff499f90640 1 -- 172.21.15.84:0/902765032 shutdown_connections 2023-10-16T11:24:26.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.495+0000 7ff499f90640 1 -- 172.21.15.84:0/902765032 wait complete. 2023-10-16T11:24:26.515 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:24:26.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.717+0000 7fa9e3f89640 1 Processor -- start 2023-10-16T11:24:26.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.718+0000 7fa9e3f89640 1 -- start start 2023-10-16T11:24:26.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.719+0000 7fa9e3f89640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa9dc0ce390 0x7fa9dc15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:26.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.719+0000 7fa9e3f89640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9dc15ed60 con 0x7fa9dc15aaa0 2023-10-16T11:24:26.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.719+0000 7fa9e3f89640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9dc0707e0 con 0x7fa9dc15ae80 2023-10-16T11:24:26.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.719+0000 7fa9e3f89640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9dc06b810 con 0x7fa9dc0ce390 2023-10-16T11:24:26.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.719+0000 7fa9e1d25640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa9dc15aaa0 0x7fa9dc0cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:59468/0 (socket says 172.21.15.84:59468) 2023-10-16T11:24:26.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.719+0000 7fa9e1d25640 1 -- 172.21.15.84:0/274447741 learned_addr learned my addr 172.21.15.84:0/274447741 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:26.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.720+0000 7fa9e2526640 1 --2- 172.21.15.84:0/274447741 >> v2:172.21.15.84:3300/0 conn(0x7fa9dc0ce390 0x7fa9dc15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:26.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.720+0000 7fa9e0d23640 1 -- 172.21.15.84:0/274447741 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620656469 0 0) 0x7fa9dc15ed60 con 0x7fa9dc15aaa0 2023-10-16T11:24:26.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.720+0000 7fa9e0d23640 1 -- 172.21.15.84:0/274447741 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9c8003520 con 0x7fa9dc15aaa0 2023-10-16T11:24:26.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.720+0000 7fa9e2526640 1 -- 172.21.15.84:0/274447741 >> v1:172.21.15.157:6789/0 conn(0x7fa9dc15ae80 legacy=0x7fa9dc070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:26.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.720+0000 7fa9e2526640 1 -- 172.21.15.84:0/274447741 >> v1:172.21.15.84:6789/0 conn(0x7fa9dc15aaa0 legacy=0x7fa9dc0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:26.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.720+0000 7fa9e2526640 1 -- 172.21.15.84:0/274447741 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc15ef90 con 0x7fa9dc0ce390 2023-10-16T11:24:26.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.721+0000 7fa9e2526640 1 --2- 172.21.15.84:0/274447741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc0ce390 0x7fa9dc15e820 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009d90 tx=0x7fa9d402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ce5d8eaf8ab8a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:26.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.721+0000 7fa9e0d23640 1 -- 172.21.15.84:0/274447741 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d402cc60 con 0x7fa9dc0ce390 2023-10-16T11:24:26.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.721+0000 7fa9e0d23640 1 -- 172.21.15.84:0/274447741 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d402cdc0 con 0x7fa9dc0ce390 2023-10-16T11:24:26.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.721+0000 7fa9e0d23640 1 -- 172.21.15.84:0/274447741 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4034660 con 0x7fa9dc0ce390 2023-10-16T11:24:26.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.722+0000 7fa9e3f89640 1 -- 172.21.15.84:0/274447741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc0ce390 msgr2=0x7fa9dc15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:26.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.722+0000 7fa9e3f89640 1 --2- 172.21.15.84:0/274447741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc0ce390 0x7fa9dc15e820 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009d90 tx=0x7fa9d402beb0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.722+0000 7fa9e3f89640 1 -- 172.21.15.84:0/274447741 shutdown_connections 2023-10-16T11:24:26.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.722+0000 7fa9e3f89640 1 --2- 172.21.15.84:0/274447741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc0ce390 0x7fa9dc15e820 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.722+0000 7fa9e3f89640 1 -- 172.21.15.84:0/274447741 >> 172.21.15.84:0/274447741 conn(0x7fa9dc0cb410 msgr2=0x7fa9dc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:26.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.722+0000 7fa9e3f89640 1 -- 172.21.15.84:0/274447741 shutdown_connections 2023-10-16T11:24:26.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.722+0000 7fa9e3f89640 1 -- 172.21.15.84:0/274447741 wait complete. 2023-10-16T11:24:26.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.724+0000 7fa9e3f89640 1 Processor -- start 2023-10-16T11:24:26.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.724+0000 7fa9e3f89640 1 -- start start 2023-10-16T11:24:26.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.725+0000 7fa9e3f89640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9dc0ce390 0x7fa9dc0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:26.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.725+0000 7fa9e3f89640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc15aaa0 0x7fa9dc15d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:26.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.725+0000 7fa9e3f89640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9dc15ae80 0x7fa9dc154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:26.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.725+0000 7fa9e3f89640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc1590f0 con 0x7fa9dc15aaa0 2023-10-16T11:24:26.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.725+0000 7fa9e3f89640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc159340 con 0x7fa9dc0ce390 2023-10-16T11:24:26.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.725+0000 7fa9e3f89640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc159620 con 0x7fa9dc15ae80 2023-10-16T11:24:26.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.725+0000 7fa9e1524640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc15aaa0 0x7fa9dc15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:26.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.725+0000 7fa9e1524640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc15aaa0 0x7fa9dc15d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56188/0 (socket says 172.21.15.84:56188) 2023-10-16T11:24:26.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.725+0000 7fa9e1524640 1 -- 172.21.15.84:0/2069221037 learned_addr learned my addr 172.21.15.84:0/2069221037 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:26.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.725+0000 7fa9e1d25640 1 --2- 172.21.15.84:0/2069221037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9dc0ce390 0x7fa9dc0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:26.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.725+0000 7fa9e2526640 1 --2- 172.21.15.84:0/2069221037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9dc15ae80 0x7fa9dc154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:26.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.725+0000 7fa9e1524640 1 -- 172.21.15.84:0/2069221037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9dc15ae80 msgr2=0x7fa9dc154fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:26.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.725+0000 7fa9e1524640 1 --2- 172.21.15.84:0/2069221037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9dc15ae80 0x7fa9dc154fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.725+0000 7fa9e1524640 1 -- 172.21.15.84:0/2069221037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9dc0ce390 msgr2=0x7fa9dc0c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:26.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.726+0000 7fa9e1524640 1 --2- 172.21.15.84:0/2069221037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9dc0ce390 0x7fa9dc0c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.726+0000 7fa9e1524640 1 -- 172.21.15.84:0/2069221037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d4009a70 con 0x7fa9dc15aaa0 2023-10-16T11:24:26.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.726+0000 7fa9e1d25640 1 --2- 172.21.15.84:0/2069221037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9dc0ce390 0x7fa9dc0c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:26.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.726+0000 7fa9e1524640 1 --2- 172.21.15.84:0/2069221037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc15aaa0 0x7fa9dc15d090 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00b3c0 tx=0x7fa9cc00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:26.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.726+0000 7fa9d27fc640 1 -- 172.21.15.84:0/2069221037 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc011830 con 0x7fa9dc15aaa0 2023-10-16T11:24:26.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.726+0000 7fa9d27fc640 1 -- 172.21.15.84:0/2069221037 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc011990 con 0x7fa9dc15aaa0 2023-10-16T11:24:26.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.726+0000 7fa9d27fc640 1 -- 172.21.15.84:0/2069221037 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc011d80 con 0x7fa9dc15aaa0 2023-10-16T11:24:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.726+0000 7fa9e3f89640 1 -- 172.21.15.84:0/2069221037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9dc159900 con 0x7fa9dc15aaa0 2023-10-16T11:24:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.726+0000 7fa9e3f89640 1 -- 172.21.15.84:0/2069221037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9dc159ec0 con 0x7fa9dc15aaa0 2023-10-16T11:24:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.728+0000 7fa9e3f89640 1 -- 172.21.15.84:0/2069221037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9dc15a0e0 con 0x7fa9dc15aaa0 2023-10-16T11:24:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.728+0000 7fa9d27fc640 1 -- 172.21.15.84:0/2069221037 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa9cc01b460 con 0x7fa9dc15aaa0 2023-10-16T11:24:26.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.729+0000 7fa9d27fc640 1 --2- 172.21.15.84:0/2069221037 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa9c006af90 0x7fa9c006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:26.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.729+0000 7fa9d27fc640 1 -- 172.21.15.84:0/2069221037 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa9cc0892c0 con 0x7fa9dc15aaa0 2023-10-16T11:24:26.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.730+0000 7fa9e1d25640 1 --2- 172.21.15.84:0/2069221037 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa9c006af90 0x7fa9c006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:26.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.730+0000 7fa9e1d25640 1 --2- 172.21.15.84:0/2069221037 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa9c006af90 0x7fa9c006d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80011e0 tx=0x7fa9d8009040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:26.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.733+0000 7fa9d27fc640 1 -- 172.21.15.84:0/2069221037 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa9cc054240 con 0x7fa9dc15aaa0 2023-10-16T11:24:26.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.926+0000 7fa9e3f89640 1 -- 172.21.15.84:0/2069221037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9dc0b1c50 con 0x7fa9dc15aaa0 2023-10-16T11:24:26.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.928+0000 7fa9d27fc640 1 -- 172.21.15.84:0/2069221037 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v216) v1 ==== 74+0+16616 (secure 0 0 0) 0x7fa9cc057860 con 0x7fa9dc15aaa0 2023-10-16T11:24:26.930 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:26.931 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":216,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:24:25.530901+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:24:23.363032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:24:24.551270+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:24:26.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.930+0000 7fa9e3f89640 1 -- 172.21.15.84:0/2069221037 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa9c006af90 msgr2=0x7fa9c006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:26.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.930+0000 7fa9e3f89640 1 --2- 172.21.15.84:0/2069221037 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa9c006af90 0x7fa9c006d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80011e0 tx=0x7fa9d8009040 comp rx=0 tx=0).stop 2023-10-16T11:24:26.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.930+0000 7fa9e3f89640 1 -- 172.21.15.84:0/2069221037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc15aaa0 msgr2=0x7fa9dc15d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:26.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.930+0000 7fa9e3f89640 1 --2- 172.21.15.84:0/2069221037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc15aaa0 0x7fa9dc15d090 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00b3c0 tx=0x7fa9cc00b880 comp rx=0 tx=0).stop 2023-10-16T11:24:26.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.931+0000 7fa9e3f89640 1 -- 172.21.15.84:0/2069221037 shutdown_connections 2023-10-16T11:24:26.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.931+0000 7fa9e3f89640 1 --2- 172.21.15.84:0/2069221037 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa9c006af90 0x7fa9c006d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.931+0000 7fa9e3f89640 1 --2- 172.21.15.84:0/2069221037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9dc15ae80 0x7fa9dc154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.931+0000 7fa9e3f89640 1 --2- 172.21.15.84:0/2069221037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc15aaa0 0x7fa9dc15d090 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.931+0000 7fa9e3f89640 1 --2- 172.21.15.84:0/2069221037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9dc0ce390 0x7fa9dc0c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:26.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.931+0000 7fa9e3f89640 1 -- 172.21.15.84:0/2069221037 >> 172.21.15.84:0/2069221037 conn(0x7fa9dc0cb410 msgr2=0x7fa9dc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:26.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.931+0000 7fa9e3f89640 1 -- 172.21.15.84:0/2069221037 shutdown_connections 2023-10-16T11:24:26.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:26.931+0000 7fa9e3f89640 1 -- 172.21.15.84:0/2069221037 wait complete. 2023-10-16T11:24:26.950 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T11:24:27.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.159+0000 7f3d51c21640 1 Processor -- start 2023-10-16T11:24:27.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.159+0000 7f3d51c21640 1 -- start start 2023-10-16T11:24:27.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.161+0000 7f3d51c21640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3d4c0ad350 0x7f3d4c069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:27.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.161+0000 7f3d51c21640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d4c06a730 con 0x7f3d4c06a2e0 2023-10-16T11:24:27.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.161+0000 7f3d51c21640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d4c0ac890 con 0x7f3d4c15d150 2023-10-16T11:24:27.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.161+0000 7f3d51c21640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d4c15d530 con 0x7f3d4c0ad350 2023-10-16T11:24:27.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.161+0000 7f3d43fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3d4c0ad350 0x7f3d4c069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:27.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.161+0000 7f3d43fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3d4c0ad350 0x7f3d4c069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56200/0 (socket says 172.21.15.84:56200) 2023-10-16T11:24:27.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.161+0000 7f3d43fff640 1 -- 172.21.15.84:0/1969161782 learned_addr learned my addr 172.21.15.84:0/1969161782 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:27.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.162+0000 7f3d43fff640 1 -- 172.21.15.84:0/1969161782 >> v1:172.21.15.157:6789/0 conn(0x7f3d4c15d150 legacy=0x7f3d4c160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:27.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.162+0000 7f3d43fff640 1 -- 172.21.15.84:0/1969161782 >> v1:172.21.15.84:6789/0 conn(0x7f3d4c06a2e0 legacy=0x7f3d4c0c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:27.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.162+0000 7f3d4affd640 1 -- 172.21.15.84:0/1969161782 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020318780 0 0) 0x7f3d44001490 con 0x7f3d4c06a2e0 2023-10-16T11:24:27.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.162+0000 7f3d43fff640 1 -- 172.21.15.84:0/1969161782 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c0ad7a0 con 0x7f3d4c0ad350 2023-10-16T11:24:27.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.163+0000 7f3d43fff640 1 --2- 172.21.15.84:0/1969161782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d4c0ad350 0x7f3d4c069da0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f3d34009870 tx=0x7f3d3402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2262993ee4ebee22 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:27.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.163+0000 7f3d4affd640 1 -- 172.21.15.84:0/1969161782 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d34030850 con 0x7f3d4c0ad350 2023-10-16T11:24:27.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.163+0000 7f3d4affd640 1 -- 172.21.15.84:0/1969161782 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d340309b0 con 0x7f3d4c0ad350 2023-10-16T11:24:27.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.163+0000 7f3d4affd640 1 -- 172.21.15.84:0/1969161782 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d34030d40 con 0x7f3d4c0ad350 2023-10-16T11:24:27.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.164+0000 7f3d51c21640 1 -- 172.21.15.84:0/1969161782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d4c0ad350 msgr2=0x7f3d4c069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:27.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.164+0000 7f3d51c21640 1 --2- 172.21.15.84:0/1969161782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d4c0ad350 0x7f3d4c069da0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f3d34009870 tx=0x7f3d3402b990 comp rx=0 tx=0).stop 2023-10-16T11:24:27.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.164+0000 7f3d51c21640 1 -- 172.21.15.84:0/1969161782 shutdown_connections 2023-10-16T11:24:27.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.164+0000 7f3d51c21640 1 --2- 172.21.15.84:0/1969161782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d4c0ad350 0x7f3d4c069da0 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:27.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.164+0000 7f3d51c21640 1 -- 172.21.15.84:0/1969161782 >> 172.21.15.84:0/1969161782 conn(0x7f3d4c0ce9b0 msgr2=0x7f3d4c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:27.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.164+0000 7f3d51c21640 1 -- 172.21.15.84:0/1969161782 shutdown_connections 2023-10-16T11:24:27.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.164+0000 7f3d51c21640 1 -- 172.21.15.84:0/1969161782 wait complete. 2023-10-16T11:24:27.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.165+0000 7f3d51c21640 1 Processor -- start 2023-10-16T11:24:27.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.166+0000 7f3d51c21640 1 -- start start 2023-10-16T11:24:27.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.166+0000 7f3d51c21640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d4c06a2e0 0x7f3d4c155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:27.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.166+0000 7f3d51c21640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d4c0ad350 0x7f3d4c0b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:27.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.166+0000 7f3d51c21640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d4c15d150 0x7f3d4c158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:27.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.167+0000 7f3d51c21640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c155cd0 con 0x7f3d4c06a2e0 2023-10-16T11:24:27.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.167+0000 7f3d51c21640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c0aecc0 con 0x7f3d4c15d150 2023-10-16T11:24:27.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.167+0000 7f3d51c21640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c0c3ce0 con 0x7f3d4c0ad350 2023-10-16T11:24:27.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.167+0000 7f3d4b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d4c06a2e0 0x7f3d4c155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:27.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.167+0000 7f3d4b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d4c06a2e0 0x7f3d4c155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56202/0 (socket says 172.21.15.84:56202) 2023-10-16T11:24:27.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.167+0000 7f3d4b7fe640 1 -- 172.21.15.84:0/972849890 learned_addr learned my addr 172.21.15.84:0/972849890 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:27.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.167+0000 7f3d4b7fe640 1 -- 172.21.15.84:0/972849890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d4c0ad350 msgr2=0x7f3d4c0b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:27.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.167+0000 7f3d4b7fe640 1 --2- 172.21.15.84:0/972849890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d4c0ad350 0x7f3d4c0b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:27.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.167+0000 7f3d4b7fe640 1 -- 172.21.15.84:0/972849890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d4c15d150 msgr2=0x7f3d4c158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:27.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.167+0000 7f3d4b7fe640 1 --2- 172.21.15.84:0/972849890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d4c15d150 0x7f3d4c158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:27.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.167+0000 7f3d4b7fe640 1 -- 172.21.15.84:0/972849890 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d340094d0 con 0x7f3d4c06a2e0 2023-10-16T11:24:27.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.167+0000 7f3d4b7fe640 1 --2- 172.21.15.84:0/972849890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d4c06a2e0 0x7f3d4c155790 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00b3f0 tx=0x7f3d3c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:27.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.168+0000 7f3d48ff9640 1 -- 172.21.15.84:0/972849890 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c011830 con 0x7f3d4c06a2e0 2023-10-16T11:24:27.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.168+0000 7f3d48ff9640 1 -- 172.21.15.84:0/972849890 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3c011990 con 0x7f3d4c06a2e0 2023-10-16T11:24:27.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.168+0000 7f3d51c21640 1 -- 172.21.15.84:0/972849890 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4c0b0f60 con 0x7f3d4c06a2e0 2023-10-16T11:24:27.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.168+0000 7f3d48ff9640 1 -- 172.21.15.84:0/972849890 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c011d40 con 0x7f3d4c06a2e0 2023-10-16T11:24:27.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.168+0000 7f3d51c21640 1 -- 172.21.15.84:0/972849890 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4c0c4210 con 0x7f3d4c06a2e0 2023-10-16T11:24:27.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.169+0000 7f3d51c21640 1 -- 172.21.15.84:0/972849890 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d4c04e160 con 0x7f3d4c06a2e0 2023-10-16T11:24:27.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.170+0000 7f3d48ff9640 1 -- 172.21.15.84:0/972849890 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3d3c01a400 con 0x7f3d4c06a2e0 2023-10-16T11:24:27.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.170+0000 7f3d48ff9640 1 --2- 172.21.15.84:0/972849890 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3d2c06af90 0x7f3d2c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:27.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.171+0000 7f3d48ff9640 1 -- 172.21.15.84:0/972849890 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f3d3c089670 con 0x7f3d4c06a2e0 2023-10-16T11:24:27.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.171+0000 7f3d43fff640 1 --2- 172.21.15.84:0/972849890 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3d2c06af90 0x7f3d2c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:27.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.172+0000 7f3d43fff640 1 --2- 172.21.15.84:0/972849890 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3d2c06af90 0x7f3d2c06d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3d34036000 tx=0x7f3d3403b040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:27.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.175+0000 7f3d48ff9640 1 -- 172.21.15.84:0/972849890 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3d3c0545e0 con 0x7f3d4c06a2e0 2023-10-16T11:24:27.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:27.408+0000 7f3d51c21640 1 -- 172.21.15.84:0/972849890 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3d4c0c5510 con 0x7f3d4c06a2e0 2023-10-16T11:24:28.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.079+0000 7f3d48ff9640 1 -- 172.21.15.84:0/972849890 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v113) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3d3c057c00 con 0x7f3d4c06a2e0 2023-10-16T11:24:28.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.083+0000 7f3d51c21640 1 -- 172.21.15.84:0/972849890 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3d2c06af90 msgr2=0x7f3d2c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:28.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.083+0000 7f3d51c21640 1 --2- 172.21.15.84:0/972849890 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3d2c06af90 0x7f3d2c06d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3d34036000 tx=0x7f3d3403b040 comp rx=0 tx=0).stop 2023-10-16T11:24:28.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.083+0000 7f3d51c21640 1 -- 172.21.15.84:0/972849890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d4c06a2e0 msgr2=0x7f3d4c155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:28.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.083+0000 7f3d51c21640 1 --2- 172.21.15.84:0/972849890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d4c06a2e0 0x7f3d4c155790 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00b3f0 tx=0x7f3d3c00b8b0 comp rx=0 tx=0).stop 2023-10-16T11:24:28.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.083+0000 7f3d51c21640 1 -- 172.21.15.84:0/972849890 shutdown_connections 2023-10-16T11:24:28.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.083+0000 7f3d51c21640 1 --2- 172.21.15.84:0/972849890 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3d2c06af90 0x7f3d2c06d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:28.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.083+0000 7f3d51c21640 1 --2- 172.21.15.84:0/972849890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d4c15d150 0x7f3d4c158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:28.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.083+0000 7f3d51c21640 1 --2- 172.21.15.84:0/972849890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d4c0ad350 0x7f3d4c0b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:28.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.083+0000 7f3d51c21640 1 --2- 172.21.15.84:0/972849890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d4c06a2e0 0x7f3d4c155790 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:28.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.083+0000 7f3d51c21640 1 -- 172.21.15.84:0/972849890 >> 172.21.15.84:0/972849890 conn(0x7f3d4c0ce9b0 msgr2=0x7f3d4c06f770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:28.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.083+0000 7f3d51c21640 1 -- 172.21.15.84:0/972849890 shutdown_connections 2023-10-16T11:24:28.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.083+0000 7f3d51c21640 1 -- 172.21.15.84:0/972849890 wait complete. 2023-10-16T11:24:28.104 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T11:24:28.105 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T11:24:28.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.320+0000 7f78aba75640 1 Processor -- start 2023-10-16T11:24:28.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.321+0000 7f78aba75640 1 -- start start 2023-10-16T11:24:28.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.322+0000 7f78aba75640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78a415d030 0x7f78a40d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:28.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.322+0000 7f78aba75640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f78a415d410 0x7f78a4161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:28.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.322+0000 7f78aba75640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a40b5d90 con 0x7f78a4161990 2023-10-16T11:24:28.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.322+0000 7f78aba75640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a40d2b50 con 0x7f78a415d410 2023-10-16T11:24:28.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.322+0000 7f78aba75640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a4160fc0 con 0x7f78a415d030 2023-10-16T11:24:28.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.322+0000 7f78a9010640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f78a415d410 0x7f78a4161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:28.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.323+0000 7f78a9010640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f78a415d410 0x7f78a4161450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56206/0 (socket says 172.21.15.84:56206) 2023-10-16T11:24:28.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.323+0000 7f78a9010640 1 -- 172.21.15.84:0/1155652640 learned_addr learned my addr 172.21.15.84:0/1155652640 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:28.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.323+0000 7f78a9811640 1 --2- 172.21.15.84:0/1155652640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78a415d030 0x7f78a40d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:28.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.323+0000 7f78a9010640 1 -- 172.21.15.84:0/1155652640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78a415d030 msgr2=0x7f78a40d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:28.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.323+0000 7f78a9010640 1 --2- 172.21.15.84:0/1155652640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78a415d030 0x7f78a40d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:28.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.323+0000 7f78a9010640 1 -- 172.21.15.84:0/1155652640 >> v1:172.21.15.84:6789/0 conn(0x7f78a4161990 legacy=0x7f78a40cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:28.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.323+0000 7f78a9010640 1 -- 172.21.15.84:0/1155652640 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a40d25d0 con 0x7f78a415d410 2023-10-16T11:24:28.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.324+0000 7f78a9010640 1 --2- 172.21.15.84:0/1155652640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78a415d410 0x7f78a4161450 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f7894009870 tx=0x7f789402ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a4d90abeb7b2f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:28.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.324+0000 7f78a880f640 1 -- 172.21.15.84:0/1155652640 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789402f830 con 0x7f78a415d410 2023-10-16T11:24:28.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.324+0000 7f78a880f640 1 -- 172.21.15.84:0/1155652640 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789402f990 con 0x7f78a415d410 2023-10-16T11:24:28.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.324+0000 7f78a880f640 1 -- 172.21.15.84:0/1155652640 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789402fd20 con 0x7f78a415d410 2023-10-16T11:24:28.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.325+0000 7f78aba75640 1 -- 172.21.15.84:0/1155652640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78a415d410 msgr2=0x7f78a4161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:28.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.325+0000 7f78aba75640 1 --2- 172.21.15.84:0/1155652640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78a415d410 0x7f78a4161450 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f7894009870 tx=0x7f789402ba10 comp rx=0 tx=0).stop 2023-10-16T11:24:28.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.325+0000 7f78aba75640 1 -- 172.21.15.84:0/1155652640 shutdown_connections 2023-10-16T11:24:28.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.325+0000 7f78aba75640 1 --2- 172.21.15.84:0/1155652640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78a415d410 0x7f78a4161450 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:28.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.325+0000 7f78aba75640 1 --2- 172.21.15.84:0/1155652640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78a415d030 0x7f78a40d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:28.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.325+0000 7f78aba75640 1 -- 172.21.15.84:0/1155652640 >> 172.21.15.84:0/1155652640 conn(0x7f78a40cf720 msgr2=0x7f78a40cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:28.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.325+0000 7f78aba75640 1 -- 172.21.15.84:0/1155652640 shutdown_connections 2023-10-16T11:24:28.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.325+0000 7f78aba75640 1 -- 172.21.15.84:0/1155652640 wait complete. 2023-10-16T11:24:28.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.327+0000 7f78aba75640 1 Processor -- start 2023-10-16T11:24:28.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.327+0000 7f78aba75640 1 -- start start 2023-10-16T11:24:28.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.328+0000 7f78aba75640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78a415d030 0x7f78a4156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:28.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.328+0000 7f78aba75640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78a415d410 0x7f78a40b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:28.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.328+0000 7f78aba75640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78a4161990 0x7f78a40b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:28.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.328+0000 7f78aba75640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a4157510 con 0x7f78a4161990 2023-10-16T11:24:28.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.328+0000 7f78aba75640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a415f470 con 0x7f78a415d410 2023-10-16T11:24:28.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.328+0000 7f78aba75640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a415da30 con 0x7f78a415d030 2023-10-16T11:24:28.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.328+0000 7f78aa012640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78a4161990 0x7f78a40b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:28.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.328+0000 7f78aa012640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78a4161990 0x7f78a40b3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56222/0 (socket says 172.21.15.84:56222) 2023-10-16T11:24:28.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.328+0000 7f78aa012640 1 -- 172.21.15.84:0/174122712 learned_addr learned my addr 172.21.15.84:0/174122712 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:28.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.329+0000 7f78a9811640 1 --2- 172.21.15.84:0/174122712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78a415d030 0x7f78a4156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:28.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.329+0000 7f78aa012640 1 -- 172.21.15.84:0/174122712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78a415d030 msgr2=0x7f78a4156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:28.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.329+0000 7f78a9010640 1 --2- 172.21.15.84:0/174122712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78a415d410 0x7f78a40b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:28.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.329+0000 7f78aa012640 1 --2- 172.21.15.84:0/174122712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78a415d030 0x7f78a4156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:28.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.329+0000 7f78aa012640 1 -- 172.21.15.84:0/174122712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78a415d410 msgr2=0x7f78a40b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:28.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.329+0000 7f78aa012640 1 --2- 172.21.15.84:0/174122712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78a415d410 0x7f78a40b3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:28.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.329+0000 7f78aa012640 1 -- 172.21.15.84:0/174122712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78940094d0 con 0x7f78a4161990 2023-10-16T11:24:28.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.329+0000 7f78aa012640 1 --2- 172.21.15.84:0/174122712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78a4161990 0x7f78a40b3b90 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f789c00c6b0 tx=0x7f789c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:28.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.329+0000 7f789a7fc640 1 -- 172.21.15.84:0/174122712 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c003b70 con 0x7f78a4161990 2023-10-16T11:24:28.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.329+0000 7f789a7fc640 1 -- 172.21.15.84:0/174122712 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789c003cf0 con 0x7f78a4161990 2023-10-16T11:24:28.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.329+0000 7f789a7fc640 1 -- 172.21.15.84:0/174122712 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c003ea0 con 0x7f78a4161990 2023-10-16T11:24:28.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.329+0000 7f78aba75640 1 -- 172.21.15.84:0/174122712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a415dd10 con 0x7f78a4161990 2023-10-16T11:24:28.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.329+0000 7f78aba75640 1 -- 172.21.15.84:0/174122712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a40c85d0 con 0x7f78a4161990 2023-10-16T11:24:28.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.331+0000 7f78aba75640 1 -- 172.21.15.84:0/174122712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a415d890 con 0x7f78a4161990 2023-10-16T11:24:28.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.332+0000 7f789a7fc640 1 -- 172.21.15.84:0/174122712 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f789c005310 con 0x7f78a4161990 2023-10-16T11:24:28.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.332+0000 7f789a7fc640 1 --2- 172.21.15.84:0/174122712 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f788806af90 0x7f788806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:28.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.333+0000 7f789a7fc640 1 -- 172.21.15.84:0/174122712 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f789c089600 con 0x7f78a4161990 2023-10-16T11:24:28.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.333+0000 7f78a9811640 1 --2- 172.21.15.84:0/174122712 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f788806af90 0x7f788806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:28.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.334+0000 7f78a9811640 1 --2- 172.21.15.84:0/174122712 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f788806af90 0x7f788806d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f78a0006fd0 tx=0x7f78a0008040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:28.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.337+0000 7f789a7fc640 1 -- 172.21.15.84:0/174122712 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f78a415d890 con 0x7f78a4161990 2023-10-16T11:24:28.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:28.572+0000 7f78aba75640 1 -- 172.21.15.84:0/174122712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f78a415d890 con 0x7f78a4161990 2023-10-16T11:24:29.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.090+0000 7f789a7fc640 1 -- 172.21.15.84:0/174122712 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v114) v1 ==== 122+0+0 (secure 0 0 0) 0x7f78a415d890 con 0x7f78a4161990 2023-10-16T11:24:29.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.093+0000 7f78aba75640 1 -- 172.21.15.84:0/174122712 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f788806af90 msgr2=0x7f788806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:29.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.093+0000 7f78aba75640 1 --2- 172.21.15.84:0/174122712 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f788806af90 0x7f788806d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f78a0006fd0 tx=0x7f78a0008040 comp rx=0 tx=0).stop 2023-10-16T11:24:29.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.094+0000 7f78aba75640 1 -- 172.21.15.84:0/174122712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78a4161990 msgr2=0x7f78a40b3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:29.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.094+0000 7f78aba75640 1 --2- 172.21.15.84:0/174122712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78a4161990 0x7f78a40b3b90 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f789c00c6b0 tx=0x7f789c00cb70 comp rx=0 tx=0).stop 2023-10-16T11:24:29.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.094+0000 7f78aba75640 1 -- 172.21.15.84:0/174122712 shutdown_connections 2023-10-16T11:24:29.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.094+0000 7f78aba75640 1 --2- 172.21.15.84:0/174122712 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f788806af90 0x7f788806d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.094+0000 7f78aba75640 1 --2- 172.21.15.84:0/174122712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78a4161990 0x7f78a40b3b90 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.094+0000 7f78aba75640 1 --2- 172.21.15.84:0/174122712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78a415d410 0x7f78a40b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.094+0000 7f78aba75640 1 --2- 172.21.15.84:0/174122712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78a415d030 0x7f78a4156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.094+0000 7f78aba75640 1 -- 172.21.15.84:0/174122712 >> 172.21.15.84:0/174122712 conn(0x7f78a40cf720 msgr2=0x7f78a41584c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:29.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.094+0000 7f78aba75640 1 -- 172.21.15.84:0/174122712 shutdown_connections 2023-10-16T11:24:29.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.094+0000 7f78aba75640 1 -- 172.21.15.84:0/174122712 wait complete. 2023-10-16T11:24:29.118 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T11:24:29.118 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:24:29.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.334+0000 7ff6497e2640 1 Processor -- start 2023-10-16T11:24:29.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.334+0000 7ff6497e2640 1 -- start start 2023-10-16T11:24:29.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.336+0000 7ff6497e2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff64415aaa0 0x7ff64415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:29.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.336+0000 7ff6497e2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6440ce390 0x7ff644070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:29.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.336+0000 7ff6497e2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64415ed60 con 0x7ff64415ae80 2023-10-16T11:24:29.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.336+0000 7ff6497e2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff64406b810 con 0x7ff64415aaa0 2023-10-16T11:24:29.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.336+0000 7ff6497e2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6440707e0 con 0x7ff6440ce390 2023-10-16T11:24:29.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.337+0000 7ff63bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff64415ae80 0x7ff6440cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:59486/0 (socket says 172.21.15.84:59486) 2023-10-16T11:24:29.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.337+0000 7ff63bfff640 1 -- 172.21.15.84:0/257330474 learned_addr learned my addr 172.21.15.84:0/257330474 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:29.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.337+0000 7ff642ffd640 1 --2- 172.21.15.84:0/257330474 >> v2:172.21.15.157:3300/0 conn(0x7ff64415aaa0 0x7ff64415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:29.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.337+0000 7ff6437fe640 1 --2- 172.21.15.84:0/257330474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6440ce390 0x7ff644070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:29.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.337+0000 7ff6427fc640 1 -- 172.21.15.84:0/257330474 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986554972 0 0) 0x7ff64415ed60 con 0x7ff64415ae80 2023-10-16T11:24:29.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.337+0000 7ff6427fc640 1 -- 172.21.15.84:0/257330474 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff630003520 con 0x7ff64415ae80 2023-10-16T11:24:29.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.337+0000 7ff6427fc640 1 -- 172.21.15.84:0/257330474 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3869726887 0 0) 0x7ff630003520 con 0x7ff64415ae80 2023-10-16T11:24:29.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.337+0000 7ff6427fc640 1 -- 172.21.15.84:0/257330474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6440ce390 msgr2=0x7ff644070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:29.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff6427fc640 1 --2- 172.21.15.84:0/257330474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6440ce390 0x7ff644070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff6427fc640 1 -- 172.21.15.84:0/257330474 >> v2:172.21.15.157:3300/0 conn(0x7ff64415aaa0 msgr2=0x7ff64415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:29.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff6427fc640 1 --2- 172.21.15.84:0/257330474 >> v2:172.21.15.157:3300/0 conn(0x7ff64415aaa0 0x7ff64415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff6427fc640 1 -- 172.21.15.84:0/257330474 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64415ef90 con 0x7ff64415ae80 2023-10-16T11:24:29.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff6427fc640 1 -- 172.21.15.84:0/257330474 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ff62c001690 con 0x7ff64415ae80 2023-10-16T11:24:29.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff6427fc640 1 -- 172.21.15.84:0/257330474 >> v1:172.21.15.84:6789/0 conn(0x7ff64415ae80 legacy=0x7ff6440cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:29.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff6427fc640 1 --2- 172.21.15.84:0/257330474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff630003970 0x7ff630003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:29.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff6427fc640 1 --2- 172.21.15.84:0/257330474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6300047c0 0x7ff630004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:29.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff6427fc640 1 --2- 172.21.15.84:0/257330474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6440ce390 0x7ff630005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:29.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff6427fc640 1 -- 172.21.15.84:0/257330474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64406b810 con 0x7ff630003970 2023-10-16T11:24:29.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff6427fc640 1 -- 172.21.15.84:0/257330474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6440707e0 con 0x7ff6300047c0 2023-10-16T11:24:29.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff6427fc640 1 -- 172.21.15.84:0/257330474 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63000b980 con 0x7ff6440ce390 2023-10-16T11:24:29.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff642ffd640 1 --2- 172.21.15.84:0/257330474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff630003970 0x7ff630003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:29.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff63bfff640 1 --2- 172.21.15.84:0/257330474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6300047c0 0x7ff630004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:29.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff642ffd640 1 -- 172.21.15.84:0/257330474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6440ce390 msgr2=0x7ff630005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:29.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff642ffd640 1 --2- 172.21.15.84:0/257330474 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6440ce390 0x7ff630005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff642ffd640 1 -- 172.21.15.84:0/257330474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6300047c0 msgr2=0x7ff630004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:29.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff642ffd640 1 --2- 172.21.15.84:0/257330474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6300047c0 0x7ff630004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff642ffd640 1 -- 172.21.15.84:0/257330474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff630003520 con 0x7ff630003970 2023-10-16T11:24:29.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.338+0000 7ff63bfff640 1 --2- 172.21.15.84:0/257330474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6300047c0 0x7ff630004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:29.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.340+0000 7ff642ffd640 1 --2- 172.21.15.84:0/257330474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff630003970 0x7ff630003d90 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7ff634009ac0 tx=0x7ff6300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1a47ee7e6832de server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:29.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.340+0000 7ff6427fc640 1 -- 172.21.15.84:0/257330474 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63402c6d0 con 0x7ff630003970 2023-10-16T11:24:29.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.340+0000 7ff6427fc640 1 -- 172.21.15.84:0/257330474 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63402c830 con 0x7ff630003970 2023-10-16T11:24:29.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.340+0000 7ff6427fc640 1 -- 172.21.15.84:0/257330474 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63402cbc0 con 0x7ff630003970 2023-10-16T11:24:29.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.340+0000 7ff6497e2640 1 -- 172.21.15.84:0/257330474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff630003970 msgr2=0x7ff630003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:29.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.340+0000 7ff6497e2640 1 --2- 172.21.15.84:0/257330474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff630003970 0x7ff630003d90 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7ff634009ac0 tx=0x7ff6300057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.340+0000 7ff6437fe640 1 -- 172.21.15.84:0/257330474 reap_dead start 2023-10-16T11:24:29.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.341+0000 7ff6497e2640 1 -- 172.21.15.84:0/257330474 shutdown_connections 2023-10-16T11:24:29.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.341+0000 7ff6497e2640 1 -- 172.21.15.84:0/257330474 >> 172.21.15.84:0/257330474 conn(0x7ff6440cb410 msgr2=0x7ff6440cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:29.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.341+0000 7ff6497e2640 1 -- 172.21.15.84:0/257330474 shutdown_connections 2023-10-16T11:24:29.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.341+0000 7ff6497e2640 1 -- 172.21.15.84:0/257330474 wait complete. 2023-10-16T11:24:29.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.342+0000 7ff6497e2640 1 Processor -- start 2023-10-16T11:24:29.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.343+0000 7ff6497e2640 1 -- start start 2023-10-16T11:24:29.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.343+0000 7ff6497e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff630003970 0x7ff6440c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:29.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.343+0000 7ff6497e2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6440c4d40 0x7ff6440c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:29.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.343+0000 7ff6497e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64414ec70 0x7ff6440b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:29.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.343+0000 7ff6497e2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6440c5220 con 0x7ff630003970 2023-10-16T11:24:29.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.343+0000 7ff6497e2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64415bdd0 con 0x7ff64414ec70 2023-10-16T11:24:29.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.343+0000 7ff6497e2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64415c0b0 con 0x7ff6440c4d40 2023-10-16T11:24:29.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.344+0000 7ff642ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff630003970 0x7ff6440c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:29.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.344+0000 7ff642ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff630003970 0x7ff6440c5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56234/0 (socket says 172.21.15.84:56234) 2023-10-16T11:24:29.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.344+0000 7ff642ffd640 1 -- 172.21.15.84:0/434171389 learned_addr learned my addr 172.21.15.84:0/434171389 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:29.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.344+0000 7ff63bfff640 1 --2- 172.21.15.84:0/434171389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6440c4d40 0x7ff6440c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:29.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.344+0000 7ff6437fe640 1 --2- 172.21.15.84:0/434171389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64414ec70 0x7ff6440b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:29.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.344+0000 7ff642ffd640 1 -- 172.21.15.84:0/434171389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6440c4d40 msgr2=0x7ff6440c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:29.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.344+0000 7ff642ffd640 1 --2- 172.21.15.84:0/434171389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6440c4d40 0x7ff6440c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.344+0000 7ff642ffd640 1 -- 172.21.15.84:0/434171389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64414ec70 msgr2=0x7ff6440b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:29.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.344+0000 7ff642ffd640 1 --2- 172.21.15.84:0/434171389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64414ec70 0x7ff6440b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.344+0000 7ff642ffd640 1 -- 172.21.15.84:0/434171389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64415c390 con 0x7ff630003970 2023-10-16T11:24:29.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.345+0000 7ff642ffd640 1 --2- 172.21.15.84:0/434171389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff630003970 0x7ff6440c5a60 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7ff63402c2d0 tx=0x7ff634004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:29.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.345+0000 7ff63b7fe640 1 -- 172.21.15.84:0/434171389 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff634003810 con 0x7ff630003970 2023-10-16T11:24:29.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.345+0000 7ff63b7fe640 1 -- 172.21.15.84:0/434171389 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff634003970 con 0x7ff630003970 2023-10-16T11:24:29.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.345+0000 7ff6497e2640 1 -- 172.21.15.84:0/434171389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64415c670 con 0x7ff630003970 2023-10-16T11:24:29.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.345+0000 7ff63b7fe640 1 -- 172.21.15.84:0/434171389 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff634003d20 con 0x7ff630003970 2023-10-16T11:24:29.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.345+0000 7ff6497e2640 1 -- 172.21.15.84:0/434171389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6440c8ac0 con 0x7ff630003970 2023-10-16T11:24:29.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.346+0000 7ff6497e2640 1 -- 172.21.15.84:0/434171389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64404e160 con 0x7ff630003970 2023-10-16T11:24:29.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.347+0000 7ff63b7fe640 1 -- 172.21.15.84:0/434171389 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff634040450 con 0x7ff630003970 2023-10-16T11:24:29.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.348+0000 7ff63b7fe640 1 --2- 172.21.15.84:0/434171389 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff62406afc0 0x7ff62406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:29.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.348+0000 7ff63b7fe640 1 -- 172.21.15.84:0/434171389 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff6340ab660 con 0x7ff630003970 2023-10-16T11:24:29.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.348+0000 7ff63bfff640 1 --2- 172.21.15.84:0/434171389 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff62406afc0 0x7ff62406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:29.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.349+0000 7ff63bfff640 1 --2- 172.21.15.84:0/434171389 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff62406afc0 0x7ff62406d480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff62c0017c0 tx=0x7ff62c009040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:29.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.352+0000 7ff63b7fe640 1 -- 172.21.15.84:0/434171389 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff6340765e0 con 0x7ff630003970 2023-10-16T11:24:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.582+0000 7ff6497e2640 1 -- 172.21.15.84:0/434171389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6440c5360 con 0x7ff630003970 2023-10-16T11:24:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.583+0000 7ff63b7fe640 1 -- 172.21.15.84:0/434171389 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4929 (secure 0 0 0) 0x7ff6340044e0 con 0x7ff630003970 2023-10-16T11:24:29.586 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:29.586 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":114,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3106898227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3106898227},{"type":"v1","addr":"172.21.15.84:6837","nonce":3106898227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":7471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2296953749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2296953749},{"type":"v1","addr":"172.21.15.157:6834","nonce":2296953749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":7474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3992377882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3992377882},{"type":"v1","addr":"172.21.15.157:6835","nonce":3992377882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:24:25.531691+0000","modified":"2023-10-16T11:24:28.574116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11307},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11307":{"gid":11307,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/551895595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":551895595},{"type":"v1","addr":"172.21.15.84:6836","nonce":551895595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-10-16T11:24:29.586 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 114 2023-10-16T11:24:29.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.586+0000 7ff6497e2640 1 -- 172.21.15.84:0/434171389 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff62406afc0 msgr2=0x7ff62406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:29.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.586+0000 7ff6497e2640 1 --2- 172.21.15.84:0/434171389 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff62406afc0 0x7ff62406d480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff62c0017c0 tx=0x7ff62c009040 comp rx=0 tx=0).stop 2023-10-16T11:24:29.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.586+0000 7ff6497e2640 1 -- 172.21.15.84:0/434171389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff630003970 msgr2=0x7ff6440c5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:29.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.586+0000 7ff6497e2640 1 --2- 172.21.15.84:0/434171389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff630003970 0x7ff6440c5a60 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7ff63402c2d0 tx=0x7ff634004730 comp rx=0 tx=0).stop 2023-10-16T11:24:29.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.586+0000 7ff6497e2640 1 -- 172.21.15.84:0/434171389 shutdown_connections 2023-10-16T11:24:29.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.586+0000 7ff6497e2640 1 --2- 172.21.15.84:0/434171389 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff62406afc0 0x7ff62406d480 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.586+0000 7ff6497e2640 1 --2- 172.21.15.84:0/434171389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64414ec70 0x7ff6440b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.586+0000 7ff6497e2640 1 --2- 172.21.15.84:0/434171389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6440c4d40 0x7ff6440c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.586+0000 7ff6497e2640 1 --2- 172.21.15.84:0/434171389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff630003970 0x7ff6440c5a60 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.586+0000 7ff6497e2640 1 -- 172.21.15.84:0/434171389 >> 172.21.15.84:0/434171389 conn(0x7ff6440cb410 msgr2=0x7ff644070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:29.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.586+0000 7ff6497e2640 1 -- 172.21.15.84:0/434171389 shutdown_connections 2023-10-16T11:24:29.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.586+0000 7ff6497e2640 1 -- 172.21.15.84:0/434171389 wait complete. 2023-10-16T11:24:29.605 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:24:29.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.812+0000 7f8031d2e640 1 Processor -- start 2023-10-16T11:24:29.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.812+0000 7f8031d2e640 1 -- start start 2023-10-16T11:24:29.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.813+0000 7f8031d2e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f802c159950 0x7f802c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:29.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.813+0000 7f8031d2e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802c15aef0 con 0x7f802c070450 2023-10-16T11:24:29.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.813+0000 7f8031d2e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802c06cc80 con 0x7f802c06c810 2023-10-16T11:24:29.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.813+0000 7f8031d2e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c15e490 con 0x7f802c159950 2023-10-16T11:24:29.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.814+0000 7f802bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f802c070450 0x7f802c06c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:59494/0 (socket says 172.21.15.84:59494) 2023-10-16T11:24:29.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.814+0000 7f802bfff640 1 -- 172.21.15.84:0/2152873786 learned_addr learned my addr 172.21.15.84:0/2152873786 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:29.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.814+0000 7f8030d2c640 1 --2- 172.21.15.84:0/2152873786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f802c159950 0x7f802c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:29.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.814+0000 7f802b7fe640 1 -- 172.21.15.84:0/2152873786 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1485411901 0 0) 0x7f8020001490 con 0x7f802c070450 2023-10-16T11:24:29.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.814+0000 7f802b7fe640 1 -- 172.21.15.84:0/2152873786 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8014003520 con 0x7f802c070450 2023-10-16T11:24:29.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.815+0000 7f802b7fe640 1 -- 172.21.15.84:0/2152873786 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3596677971 0 0) 0x7f8014003520 con 0x7f802c070450 2023-10-16T11:24:29.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.815+0000 7f802b7fe640 1 -- 172.21.15.84:0/2152873786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f802c159950 msgr2=0x7f802c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:29.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.815+0000 7f802b7fe640 1 --2- 172.21.15.84:0/2152873786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f802c159950 0x7f802c15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.815+0000 7f802b7fe640 1 -- 172.21.15.84:0/2152873786 >> v1:172.21.15.157:6789/0 conn(0x7f802c06c810 legacy=0x7f802c0c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:29.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.815+0000 7f802b7fe640 1 -- 172.21.15.84:0/2152873786 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c15d910 con 0x7f802c070450 2023-10-16T11:24:29.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.815+0000 7f802b7fe640 1 -- 172.21.15.84:0/2152873786 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f8020001b80 con 0x7f802c070450 2023-10-16T11:24:29.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.815+0000 7f802b7fe640 1 -- 172.21.15.84:0/2152873786 >> v1:172.21.15.84:6789/0 conn(0x7f802c070450 legacy=0x7f802c06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:29.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.815+0000 7f802b7fe640 1 --2- 172.21.15.84:0/2152873786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80140039a0 0x7f8014003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:29.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.815+0000 7f802b7fe640 1 --2- 172.21.15.84:0/2152873786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f802c159950 0x7f80140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:29.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.815+0000 7f802b7fe640 1 --2- 172.21.15.84:0/2152873786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8014004d00 0x7f8014005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:29.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.815+0000 7f802b7fe640 1 -- 172.21.15.84:0/2152873786 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c15e490 con 0x7f8014004d00 2023-10-16T11:24:29.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.815+0000 7f802b7fe640 1 -- 172.21.15.84:0/2152873786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801400bab0 con 0x7f80140039a0 2023-10-16T11:24:29.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.815+0000 7f802b7fe640 1 -- 172.21.15.84:0/2152873786 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801400de80 con 0x7f802c159950 2023-10-16T11:24:29.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.815+0000 7f803152d640 1 --2- 172.21.15.84:0/2152873786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8014004d00 0x7f8014005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:29.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.815+0000 7f8030d2c640 1 --2- 172.21.15.84:0/2152873786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80140039a0 0x7f8014003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:29.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.816+0000 7f803152d640 1 -- 172.21.15.84:0/2152873786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f802c159950 msgr2=0x7f80140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:29.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.816+0000 7f803152d640 1 --2- 172.21.15.84:0/2152873786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f802c159950 0x7f80140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.816+0000 7f803152d640 1 -- 172.21.15.84:0/2152873786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80140039a0 msgr2=0x7f8014003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:29.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.816+0000 7f803152d640 1 --2- 172.21.15.84:0/2152873786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80140039a0 0x7f8014003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.816+0000 7f803152d640 1 -- 172.21.15.84:0/2152873786 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8014003520 con 0x7f8014004d00 2023-10-16T11:24:29.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.817+0000 7f803152d640 1 --2- 172.21.15.84:0/2152873786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8014004d00 0x7f8014005180 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f8024001750 tx=0x7f80140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dec972817bf86af server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:29.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.817+0000 7f802b7fe640 1 -- 172.21.15.84:0/2152873786 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80240052d0 con 0x7f8014004d00 2023-10-16T11:24:29.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.817+0000 7f802b7fe640 1 -- 172.21.15.84:0/2152873786 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802402e050 con 0x7f8014004d00 2023-10-16T11:24:29.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.817+0000 7f802b7fe640 1 -- 172.21.15.84:0/2152873786 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80240035f0 con 0x7f8014004d00 2023-10-16T11:24:29.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.817+0000 7f8031d2e640 1 -- 172.21.15.84:0/2152873786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8014004d00 msgr2=0x7f8014005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:29.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.817+0000 7f8031d2e640 1 --2- 172.21.15.84:0/2152873786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8014004d00 0x7f8014005180 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f8024001750 tx=0x7f80140057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.817+0000 7f803152d640 1 -- 172.21.15.84:0/2152873786 reap_dead start 2023-10-16T11:24:29.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.817+0000 7f8031d2e640 1 -- 172.21.15.84:0/2152873786 shutdown_connections 2023-10-16T11:24:29.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.817+0000 7f8031d2e640 1 -- 172.21.15.84:0/2152873786 >> 172.21.15.84:0/2152873786 conn(0x7f802c0ce6a0 msgr2=0x7f802c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:29.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.818+0000 7f8031d2e640 1 -- 172.21.15.84:0/2152873786 shutdown_connections 2023-10-16T11:24:29.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.818+0000 7f8031d2e640 1 -- 172.21.15.84:0/2152873786 wait complete. 2023-10-16T11:24:29.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.819+0000 7f8031d2e640 1 Processor -- start 2023-10-16T11:24:29.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.819+0000 7f8031d2e640 1 -- start start 2023-10-16T11:24:29.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.820+0000 7f8031d2e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8014004d00 0x7f802c154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:29.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.820+0000 7f8031d2e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f802c0b0880 0x7f802c0b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:29.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.820+0000 7f8031d2e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f802c0ac890 0x7f802c0acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:29.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.820+0000 7f8031d2e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0ad250 con 0x7f802c0ac890 2023-10-16T11:24:29.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.820+0000 7f8031d2e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0c2f30 con 0x7f8014004d00 2023-10-16T11:24:29.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.820+0000 7f8031d2e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0c1f20 con 0x7f802c0b0880 2023-10-16T11:24:29.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.821+0000 7f803152d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f802c0ac890 0x7f802c0acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:29.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.821+0000 7f803152d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f802c0ac890 0x7f802c0acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56254/0 (socket says 172.21.15.84:56254) 2023-10-16T11:24:29.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.821+0000 7f803152d640 1 -- 172.21.15.84:0/867824928 learned_addr learned my addr 172.21.15.84:0/867824928 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:29.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.821+0000 7f803152d640 1 -- 172.21.15.84:0/867824928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f802c0b0880 msgr2=0x7f802c0b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:29.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.821+0000 7f802bfff640 1 --2- 172.21.15.84:0/867824928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f802c0b0880 0x7f802c0b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:29.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.821+0000 7f8030d2c640 1 --2- 172.21.15.84:0/867824928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8014004d00 0x7f802c154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:29.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.821+0000 7f803152d640 1 --2- 172.21.15.84:0/867824928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f802c0b0880 0x7f802c0b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.821+0000 7f803152d640 1 -- 172.21.15.84:0/867824928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8014004d00 msgr2=0x7f802c154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:29.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.821+0000 7f803152d640 1 --2- 172.21.15.84:0/867824928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8014004d00 0x7f802c154e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:29.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.821+0000 7f803152d640 1 -- 172.21.15.84:0/867824928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c0c2170 con 0x7f802c0ac890 2023-10-16T11:24:29.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.821+0000 7f802bfff640 1 --2- 172.21.15.84:0/867824928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f802c0b0880 0x7f802c0b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:29.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.821+0000 7f803152d640 1 --2- 172.21.15.84:0/867824928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f802c0ac890 0x7f802c0acd10 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f8024001870 tx=0x7f8024001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:29.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.822+0000 7f80297fa640 1 -- 172.21.15.84:0/867824928 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024007a30 con 0x7f802c0ac890 2023-10-16T11:24:29.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.822+0000 7f80297fa640 1 -- 172.21.15.84:0/867824928 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80240067d0 con 0x7f802c0ac890 2023-10-16T11:24:29.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.822+0000 7f80297fa640 1 -- 172.21.15.84:0/867824928 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024006b80 con 0x7f802c0ac890 2023-10-16T11:24:29.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.822+0000 7f8031d2e640 1 -- 172.21.15.84:0/867824928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802c0c2450 con 0x7f802c0ac890 2023-10-16T11:24:29.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.822+0000 7f8031d2e640 1 -- 172.21.15.84:0/867824928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c0c2a10 con 0x7f802c0ac890 2023-10-16T11:24:29.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.823+0000 7f8031d2e640 1 -- 172.21.15.84:0/867824928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c04e9b0 con 0x7f802c0ac890 2023-10-16T11:24:29.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.824+0000 7f80297fa640 1 -- 172.21.15.84:0/867824928 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8024006ce0 con 0x7f802c0ac890 2023-10-16T11:24:29.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.825+0000 7f80297fa640 1 --2- 172.21.15.84:0/867824928 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f801806afc0 0x7f801806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:29.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.825+0000 7f80297fa640 1 -- 172.21.15.84:0/867824928 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f80240ab240 con 0x7f802c0ac890 2023-10-16T11:24:29.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.825+0000 7f8030d2c640 1 --2- 172.21.15.84:0/867824928 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f801806afc0 0x7f801806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:29.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.826+0000 7f8030d2c640 1 --2- 172.21.15.84:0/867824928 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f801806afc0 0x7f801806d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f801c009930 tx=0x7f801c008040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:29.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:29.829+0000 7f80297fa640 1 -- 172.21.15.84:0/867824928 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f80240761c0 con 0x7f802c0ac890 2023-10-16T11:24:30.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.006+0000 7f8031d2e640 1 -- 172.21.15.84:0/867824928 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f802c0ad250 con 0x7f802c0ac890 2023-10-16T11:24:30.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.009+0000 7f80297fa640 1 -- 172.21.15.84:0/867824928 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v217) v1 ==== 74+0+16603 (secure 0 0 0) 0x7f80240797e0 con 0x7f802c0ac890 2023-10-16T11:24:30.012 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:30.012 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":217,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:24:27.068476+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:24:23.363032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:24:24.551270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:24:30.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.012+0000 7f8031d2e640 1 -- 172.21.15.84:0/867824928 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f801806afc0 msgr2=0x7f801806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:30.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.012+0000 7f8031d2e640 1 --2- 172.21.15.84:0/867824928 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f801806afc0 0x7f801806d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f801c009930 tx=0x7f801c008040 comp rx=0 tx=0).stop 2023-10-16T11:24:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.012+0000 7f8031d2e640 1 -- 172.21.15.84:0/867824928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f802c0ac890 msgr2=0x7f802c0acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.012+0000 7f8031d2e640 1 --2- 172.21.15.84:0/867824928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f802c0ac890 0x7f802c0acd10 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f8024001870 tx=0x7f8024001120 comp rx=0 tx=0).stop 2023-10-16T11:24:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.012+0000 7f8031d2e640 1 -- 172.21.15.84:0/867824928 shutdown_connections 2023-10-16T11:24:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.012+0000 7f8031d2e640 1 --2- 172.21.15.84:0/867824928 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f801806afc0 0x7f801806d480 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.012+0000 7f8031d2e640 1 --2- 172.21.15.84:0/867824928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f802c0ac890 0x7f802c0acd10 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.012+0000 7f8031d2e640 1 --2- 172.21.15.84:0/867824928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f802c0b0880 0x7f802c0b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.012+0000 7f8031d2e640 1 --2- 172.21.15.84:0/867824928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8014004d00 0x7f802c154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.012+0000 7f8031d2e640 1 -- 172.21.15.84:0/867824928 >> 172.21.15.84:0/867824928 conn(0x7f802c0ce6a0 msgr2=0x7f802c0ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.012+0000 7f8031d2e640 1 -- 172.21.15.84:0/867824928 shutdown_connections 2023-10-16T11:24:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.012+0000 7f8031d2e640 1 -- 172.21.15.84:0/867824928 wait complete. 2023-10-16T11:24:30.033 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T11:24:30.034 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:24:30.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.245+0000 7f7bf6e44640 1 Processor -- start 2023-10-16T11:24:30.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.245+0000 7f7bf6e44640 1 -- start start 2023-10-16T11:24:30.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.247+0000 7f7bf6e44640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bf00adb50 0x7f7bf006f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:30.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.247+0000 7f7bf6e44640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7bf0070fd0 0x7f7bf00b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:30.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.247+0000 7f7bf6e44640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf0071450 con 0x7f7bf00adf30 2023-10-16T11:24:30.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.247+0000 7f7bf6e44640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf00cc600 con 0x7f7bf0070fd0 2023-10-16T11:24:30.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.247+0000 7f7bf6e44640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf00cc8e0 con 0x7f7bf00adb50 2023-10-16T11:24:30.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.247+0000 7f7beffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7bf0070fd0 0x7f7bf00b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:30.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.247+0000 7f7beffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7bf0070fd0 0x7f7bf00b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56262/0 (socket says 172.21.15.84:56262) 2023-10-16T11:24:30.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.247+0000 7f7beffff640 1 -- 172.21.15.84:0/1460274045 learned_addr learned my addr 172.21.15.84:0/1460274045 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:30.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.247+0000 7f7bf53e1640 1 --1- 172.21.15.84:0/1460274045 >> v1:172.21.15.84:6789/0 conn(0x7f7bf00adf30 0x7f7bf0164f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:59504/0 (socket says 172.21.15.84:59504) 2023-10-16T11:24:30.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.248+0000 7f7bf4be0640 1 --2- 172.21.15.84:0/1460274045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bf00adb50 0x7f7bf006f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:30.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.248+0000 7f7beffff640 1 -- 172.21.15.84:0/1460274045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bf00adb50 msgr2=0x7f7bf006f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:30.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.248+0000 7f7beffff640 1 --2- 172.21.15.84:0/1460274045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bf00adb50 0x7f7bf006f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.248+0000 7f7beffff640 1 -- 172.21.15.84:0/1460274045 >> v1:172.21.15.84:6789/0 conn(0x7f7bf00adf30 legacy=0x7f7bf0164f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:30.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.248+0000 7f7beffff640 1 -- 172.21.15.84:0/1460274045 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf00ccbc0 con 0x7f7bf0070fd0 2023-10-16T11:24:30.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.248+0000 7f7bef7fe640 1 -- 172.21.15.84:0/1460274045 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394407082 0 0) 0x7f7bf0071450 con 0x7f7bf00adf30 2023-10-16T11:24:30.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.248+0000 7f7bf4be0640 1 --2- 172.21.15.84:0/1460274045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bf00adb50 0x7f7bf006f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:30.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.249+0000 7f7beffff640 1 --2- 172.21.15.84:0/1460274045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf0070fd0 0x7f7bf00b0410 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009870 tx=0x7f7bd802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753b85275859746a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:30.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.249+0000 7f7bef7fe640 1 -- 172.21.15.84:0/1460274045 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8030840 con 0x7f7bf0070fd0 2023-10-16T11:24:30.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.249+0000 7f7bef7fe640 1 -- 172.21.15.84:0/1460274045 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd80309a0 con 0x7f7bf0070fd0 2023-10-16T11:24:30.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.249+0000 7f7bef7fe640 1 -- 172.21.15.84:0/1460274045 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8030d30 con 0x7f7bf0070fd0 2023-10-16T11:24:30.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.250+0000 7f7bf6e44640 1 -- 172.21.15.84:0/1460274045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf0070fd0 msgr2=0x7f7bf00b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:30.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.250+0000 7f7bf6e44640 1 --2- 172.21.15.84:0/1460274045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf0070fd0 0x7f7bf00b0410 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009870 tx=0x7f7bd802b980 comp rx=0 tx=0).stop 2023-10-16T11:24:30.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.250+0000 7f7bf6e44640 1 -- 172.21.15.84:0/1460274045 shutdown_connections 2023-10-16T11:24:30.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.250+0000 7f7bf6e44640 1 --2- 172.21.15.84:0/1460274045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf0070fd0 0x7f7bf00b0410 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.250+0000 7f7bf6e44640 1 --2- 172.21.15.84:0/1460274045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bf00adb50 0x7f7bf006f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.250+0000 7f7bf6e44640 1 -- 172.21.15.84:0/1460274045 >> 172.21.15.84:0/1460274045 conn(0x7f7bf00cb410 msgr2=0x7f7bf00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:30.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.250+0000 7f7bf6e44640 1 -- 172.21.15.84:0/1460274045 shutdown_connections 2023-10-16T11:24:30.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.250+0000 7f7bf6e44640 1 -- 172.21.15.84:0/1460274045 wait complete. 2023-10-16T11:24:30.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.252+0000 7f7bf6e44640 1 Processor -- start 2023-10-16T11:24:30.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.252+0000 7f7bf6e44640 1 -- start start 2023-10-16T11:24:30.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.253+0000 7f7bf6e44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf0070fd0 0x7f7bf00b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:30.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.253+0000 7f7bf6e44640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bf00adb50 0x7f7bf0154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:30.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.253+0000 7f7bf6e44640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bf00adf30 0x7f7bf00c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:30.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.253+0000 7f7bf6e44640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf00b25e0 con 0x7f7bf0070fd0 2023-10-16T11:24:30.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.253+0000 7f7bf6e44640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf00b28c0 con 0x7f7bf00adf30 2023-10-16T11:24:30.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.253+0000 7f7bf6e44640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf00b2ba0 con 0x7f7bf00adb50 2023-10-16T11:24:30.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.253+0000 7f7bf4be0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf0070fd0 0x7f7bf00b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:30.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.253+0000 7f7bf4be0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf0070fd0 0x7f7bf00b1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56272/0 (socket says 172.21.15.84:56272) 2023-10-16T11:24:30.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.253+0000 7f7bf4be0640 1 -- 172.21.15.84:0/2622167670 learned_addr learned my addr 172.21.15.84:0/2622167670 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:30.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.254+0000 7f7bf4be0640 1 -- 172.21.15.84:0/2622167670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bf00adb50 msgr2=0x7f7bf0154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:30.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.254+0000 7f7bf4be0640 1 --2- 172.21.15.84:0/2622167670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bf00adb50 0x7f7bf0154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.254+0000 7f7bf4be0640 1 -- 172.21.15.84:0/2622167670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bf00adf30 msgr2=0x7f7bf00c44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:30.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.254+0000 7f7bf4be0640 1 --2- 172.21.15.84:0/2622167670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bf00adf30 0x7f7bf00c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.254+0000 7f7bf4be0640 1 -- 172.21.15.84:0/2622167670 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd80094d0 con 0x7f7bf0070fd0 2023-10-16T11:24:30.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.254+0000 7f7bf4be0640 1 --2- 172.21.15.84:0/2622167670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf0070fd0 0x7f7bf00b1f70 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7f7be000c7f0 tx=0x7f7be000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:30.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.254+0000 7f7bed7fa640 1 -- 172.21.15.84:0/2622167670 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be0007a30 con 0x7f7bf0070fd0 2023-10-16T11:24:30.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.254+0000 7f7bed7fa640 1 -- 172.21.15.84:0/2622167670 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be0007b90 con 0x7f7bf0070fd0 2023-10-16T11:24:30.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.254+0000 7f7bed7fa640 1 -- 172.21.15.84:0/2622167670 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be000ce40 con 0x7f7bf0070fd0 2023-10-16T11:24:30.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.254+0000 7f7bf6e44640 1 -- 172.21.15.84:0/2622167670 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf00b2e80 con 0x7f7bf0070fd0 2023-10-16T11:24:30.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.254+0000 7f7bf6e44640 1 -- 172.21.15.84:0/2622167670 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf00c2480 con 0x7f7bf0070fd0 2023-10-16T11:24:30.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.256+0000 7f7bf6e44640 1 -- 172.21.15.84:0/2622167670 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf01f6f30 con 0x7f7bf0070fd0 2023-10-16T11:24:30.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.257+0000 7f7bed7fa640 1 -- 172.21.15.84:0/2622167670 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7be00113f0 con 0x7f7bf0070fd0 2023-10-16T11:24:30.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.258+0000 7f7bed7fa640 1 --2- 172.21.15.84:0/2622167670 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f7bd006afc0 0x7f7bd006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:30.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.258+0000 7f7bed7fa640 1 -- 172.21.15.84:0/2622167670 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f7be0089980 con 0x7f7bf0070fd0 2023-10-16T11:24:30.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.258+0000 7f7beffff640 1 --2- 172.21.15.84:0/2622167670 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f7bd006afc0 0x7f7bd006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:30.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.259+0000 7f7beffff640 1 --2- 172.21.15.84:0/2622167670 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f7bd006afc0 0x7f7bd006d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8036000 tx=0x7f7bd803b040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:30.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.262+0000 7f7bed7fa640 1 -- 172.21.15.84:0/2622167670 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7bf01f6f30 con 0x7f7bf0070fd0 2023-10-16T11:24:30.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.449+0000 7f7bf6e44640 1 -- 172.21.15.84:0/2622167670 --> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7bf006fa90 con 0x7f7bd006afc0 2023-10-16T11:24:30.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.476+0000 7f7bed7fa640 1 -- 172.21.15.84:0/2622167670 <== mgr.11076 v2:172.21.15.84:6832/1425409017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286793 (secure 0 0 0) 0x7f7bd8030c10 con 0x7f7bd006afc0 2023-10-16T11:24:30.482 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:30.484 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:24:30.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.483+0000 7f7bf6e44640 1 -- 172.21.15.84:0/2622167670 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f7bd006afc0 msgr2=0x7f7bd006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:30.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.483+0000 7f7bf6e44640 1 --2- 172.21.15.84:0/2622167670 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f7bd006afc0 0x7f7bd006d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8036000 tx=0x7f7bd803b040 comp rx=0 tx=0).stop 2023-10-16T11:24:30.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.483+0000 7f7bf6e44640 1 -- 172.21.15.84:0/2622167670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf0070fd0 msgr2=0x7f7bf00b1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:30.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.483+0000 7f7bf6e44640 1 --2- 172.21.15.84:0/2622167670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf0070fd0 0x7f7bf00b1f70 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7f7be000c7f0 tx=0x7f7be000cbf0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.483+0000 7f7bf6e44640 1 -- 172.21.15.84:0/2622167670 shutdown_connections 2023-10-16T11:24:30.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.483+0000 7f7bf6e44640 1 --2- 172.21.15.84:0/2622167670 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f7bd006afc0 0x7f7bd006d480 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.483+0000 7f7bf6e44640 1 --2- 172.21.15.84:0/2622167670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bf00adf30 0x7f7bf00c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.483+0000 7f7bf6e44640 1 --2- 172.21.15.84:0/2622167670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bf00adb50 0x7f7bf0154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.483+0000 7f7bf6e44640 1 --2- 172.21.15.84:0/2622167670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf0070fd0 0x7f7bf00b1f70 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.483+0000 7f7bf6e44640 1 -- 172.21.15.84:0/2622167670 >> 172.21.15.84:0/2622167670 conn(0x7f7bf00cb410 msgr2=0x7f7bf0160ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:30.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.483+0000 7f7bf6e44640 1 -- 172.21.15.84:0/2622167670 shutdown_connections 2023-10-16T11:24:30.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.483+0000 7f7bf6e44640 1 -- 172.21.15.84:0/2622167670 wait complete. 2023-10-16T11:24:30.505 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-16T11:24:30.252128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":625,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2805,"ondisk_log_size":2805,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":380264,"kb_used_data":22504,"kb_used_omap":12,"kb_used_meta":357619,"kb_avail":754594456,"statfs":{"total":773094113280,"available":772704722944,"internally_reserved":0,"allocated":23044096,"data_stored":11749132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":366202425},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001760"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070051+0000","last_change":"2023-10-16T11:24:26.086488+0000","last_active":"2023-10-16T11:24:28.070051+0000","last_peered":"2023-10-16T11:24:28.070051+0000","last_clean":"2023-10-16T11:24:28.070051+0000","last_became_active":"2023-10-16T11:24:26.085969+0000","last_became_peered":"2023-10-16T11:24:26.085969+0000","last_unstale":"2023-10-16T11:24:28.070051+0000","last_undegraded":"2023-10-16T11:24:28.070051+0000","last_fullsized":"2023-10-16T11:24:28.070051+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.200941+0000","last_change":"2023-10-16T11:24:26.089885+0000","last_active":"2023-10-16T11:24:27.200941+0000","last_peered":"2023-10-16T11:24:27.200941+0000","last_clean":"2023-10-16T11:24:27.200941+0000","last_became_active":"2023-10-16T11:24:26.089406+0000","last_became_peered":"2023-10-16T11:24:26.089406+0000","last_unstale":"2023-10-16T11:24:27.200941+0000","last_undegraded":"2023-10-16T11:24:27.200941+0000","last_fullsized":"2023-10-16T11:24:27.200941+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.080857+0000","last_change":"2023-10-16T11:24:26.085606+0000","last_active":"2023-10-16T11:24:27.080857+0000","last_peered":"2023-10-16T11:24:27.080857+0000","last_clean":"2023-10-16T11:24:27.080857+0000","last_became_active":"2023-10-16T11:24:26.085318+0000","last_became_peered":"2023-10-16T11:24:26.085318+0000","last_unstale":"2023-10-16T11:24:27.080857+0000","last_undegraded":"2023-10-16T11:24:27.080857+0000","last_fullsized":"2023-10-16T11:24:27.080857+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069514+0000","last_change":"2023-10-16T11:24:26.091440+0000","last_active":"2023-10-16T11:24:28.069514+0000","last_peered":"2023-10-16T11:24:28.069514+0000","last_clean":"2023-10-16T11:24:28.069514+0000","last_became_active":"2023-10-16T11:24:26.088757+0000","last_became_peered":"2023-10-16T11:24:26.088757+0000","last_unstale":"2023-10-16T11:24:28.069514+0000","last_undegraded":"2023-10-16T11:24:28.069514+0000","last_fullsized":"2023-10-16T11:24:28.069514+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:27.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070231+0000","last_change":"2023-10-16T11:24:26.096814+0000","last_active":"2023-10-16T11:24:28.070231+0000","last_peered":"2023-10-16T11:24:28.070231+0000","last_clean":"2023-10-16T11:24:28.070231+0000","last_became_active":"2023-10-16T11:24:26.095777+0000","last_became_peered":"2023-10-16T11:24:26.095777+0000","last_unstale":"2023-10-16T11:24:28.070231+0000","last_undegraded":"2023-10-16T11:24:28.070231+0000","last_fullsized":"2023-10-16T11:24:28.070231+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:08.889514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070908+0000","last_change":"2023-10-16T11:24:26.083687+0000","last_active":"2023-10-16T11:24:28.070908+0000","last_peered":"2023-10-16T11:24:28.070908+0000","last_clean":"2023-10-16T11:24:28.070908+0000","last_became_active":"2023-10-16T11:24:26.083213+0000","last_became_peered":"2023-10-16T11:24:26.083213+0000","last_unstale":"2023-10-16T11:24:28.070908+0000","last_undegraded":"2023-10-16T11:24:28.070908+0000","last_fullsized":"2023-10-16T11:24:28.070908+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:08:02.966132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.068359+0000","last_change":"2023-10-16T11:24:26.096348+0000","last_active":"2023-10-16T11:24:28.068359+0000","last_peered":"2023-10-16T11:24:28.068359+0000","last_clean":"2023-10-16T11:24:28.068359+0000","last_became_active":"2023-10-16T11:24:26.093712+0000","last_became_peered":"2023-10-16T11:24:26.093712+0000","last_unstale":"2023-10-16T11:24:28.068359+0000","last_undegraded":"2023-10-16T11:24:28.068359+0000","last_fullsized":"2023-10-16T11:24:28.068359+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:40.970357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.068908+0000","last_change":"2023-10-16T11:24:26.098845+0000","last_active":"2023-10-16T11:24:28.068908+0000","last_peered":"2023-10-16T11:24:28.068908+0000","last_clean":"2023-10-16T11:24:28.068908+0000","last_became_active":"2023-10-16T11:24:26.098568+0000","last_became_peered":"2023-10-16T11:24:26.098568+0000","last_unstale":"2023-10-16T11:24:28.068908+0000","last_undegraded":"2023-10-16T11:24:28.068908+0000","last_fullsized":"2023-10-16T11:24:28.068908+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:15.909098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.201569+0000","last_change":"2023-10-16T11:24:26.080292+0000","last_active":"2023-10-16T11:24:27.201569+0000","last_peered":"2023-10-16T11:24:27.201569+0000","last_clean":"2023-10-16T11:24:27.201569+0000","last_became_active":"2023-10-16T11:24:26.080004+0000","last_became_peered":"2023-10-16T11:24:26.080004+0000","last_unstale":"2023-10-16T11:24:27.201569+0000","last_undegraded":"2023-10-16T11:24:27.201569+0000","last_fullsized":"2023-10-16T11:24:27.201569+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.082247+0000","last_change":"2023-10-16T11:24:26.088202+0000","last_active":"2023-10-16T11:24:27.082247+0000","last_peered":"2023-10-16T11:24:27.082247+0000","last_clean":"2023-10-16T11:24:27.082247+0000","last_became_active":"2023-10-16T11:24:26.087929+0000","last_became_peered":"2023-10-16T11:24:26.087929+0000","last_unstale":"2023-10-16T11:24:27.082247+0000","last_undegraded":"2023-10-16T11:24:27.082247+0000","last_fullsized":"2023-10-16T11:24:27.082247+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:22.009042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.080938+0000","last_change":"2023-10-16T11:24:26.077430+0000","last_active":"2023-10-16T11:24:27.080938+0000","last_peered":"2023-10-16T11:24:27.080938+0000","last_clean":"2023-10-16T11:24:27.080938+0000","last_became_active":"2023-10-16T11:24:26.077145+0000","last_became_peered":"2023-10-16T11:24:26.077145+0000","last_unstale":"2023-10-16T11:24:27.080938+0000","last_undegraded":"2023-10-16T11:24:27.080938+0000","last_fullsized":"2023-10-16T11:24:27.080938+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.976285+0000","last_change":"2023-10-16T11:24:26.089768+0000","last_active":"2023-10-16T11:24:27.976285+0000","last_peered":"2023-10-16T11:24:27.976285+0000","last_clean":"2023-10-16T11:24:27.976285+0000","last_became_active":"2023-10-16T11:24:26.087028+0000","last_became_peered":"2023-10-16T11:24:26.087028+0000","last_unstale":"2023-10-16T11:24:27.976285+0000","last_undegraded":"2023-10-16T11:24:27.976285+0000","last_fullsized":"2023-10-16T11:24:27.976285+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:33.050118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.976905+0000","last_change":"2023-10-16T11:24:26.088938+0000","last_active":"2023-10-16T11:24:27.976905+0000","last_peered":"2023-10-16T11:24:27.976905+0000","last_clean":"2023-10-16T11:24:27.976905+0000","last_became_active":"2023-10-16T11:24:26.088689+0000","last_became_peered":"2023-10-16T11:24:26.088689+0000","last_unstale":"2023-10-16T11:24:27.976905+0000","last_undegraded":"2023-10-16T11:24:27.976905+0000","last_fullsized":"2023-10-16T11:24:27.976905+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:18:12.479766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069095+0000","last_change":"2023-10-16T11:24:26.088199+0000","last_active":"2023-10-16T11:24:28.069095+0000","last_peered":"2023-10-16T11:24:28.069095+0000","last_clean":"2023-10-16T11:24:28.069095+0000","last_became_active":"2023-10-16T11:24:26.087892+0000","last_became_peered":"2023-10-16T11:24:26.087892+0000","last_unstale":"2023-10-16T11:24:28.069095+0000","last_undegraded":"2023-10-16T11:24:28.069095+0000","last_fullsized":"2023-10-16T11:24:28.069095+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.068584+0000","last_change":"2023-10-16T11:24:26.091403+0000","last_active":"2023-10-16T11:24:28.068584+0000","last_peered":"2023-10-16T11:24:28.068584+0000","last_clean":"2023-10-16T11:24:28.068584+0000","last_became_active":"2023-10-16T11:24:26.091114+0000","last_became_peered":"2023-10-16T11:24:26.091114+0000","last_unstale":"2023-10-16T11:24:28.068584+0000","last_undegraded":"2023-10-16T11:24:28.068584+0000","last_fullsized":"2023-10-16T11:24:28.068584+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.081909+0000","last_change":"2023-10-16T11:24:26.081938+0000","last_active":"2023-10-16T11:24:27.081909+0000","last_peered":"2023-10-16T11:24:27.081909+0000","last_clean":"2023-10-16T11:24:27.081909+0000","last_became_active":"2023-10-16T11:24:26.081671+0000","last_became_peered":"2023-10-16T11:24:26.081671+0000","last_unstale":"2023-10-16T11:24:27.081909+0000","last_undegraded":"2023-10-16T11:24:27.081909+0000","last_fullsized":"2023-10-16T11:24:27.081909+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:17.072626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070195+0000","last_change":"2023-10-16T11:24:26.086029+0000","last_active":"2023-10-16T11:24:28.070195+0000","last_peered":"2023-10-16T11:24:28.070195+0000","last_clean":"2023-10-16T11:24:28.070195+0000","last_became_active":"2023-10-16T11:24:26.085644+0000","last_became_peered":"2023-10-16T11:24:26.085644+0000","last_unstale":"2023-10-16T11:24:28.070195+0000","last_undegraded":"2023-10-16T11:24:28.070195+0000","last_fullsized":"2023-10-16T11:24:28.070195+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:39.802117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.082380+0000","last_change":"2023-10-16T11:24:26.092139+0000","last_active":"2023-10-16T11:24:27.082380+0000","last_peered":"2023-10-16T11:24:27.082380+0000","last_clean":"2023-10-16T11:24:27.082380+0000","last_became_active":"2023-10-16T11:24:26.091860+0000","last_became_peered":"2023-10-16T11:24:26.091860+0000","last_unstale":"2023-10-16T11:24:27.082380+0000","last_undegraded":"2023-10-16T11:24:27.082380+0000","last_fullsized":"2023-10-16T11:24:27.082380+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:00:31.371733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.082805+0000","last_change":"2023-10-16T11:24:26.086807+0000","last_active":"2023-10-16T11:24:27.082805+0000","last_peered":"2023-10-16T11:24:27.082805+0000","last_clean":"2023-10-16T11:24:27.082805+0000","last_became_active":"2023-10-16T11:24:26.085382+0000","last_became_peered":"2023-10-16T11:24:26.085382+0000","last_unstale":"2023-10-16T11:24:27.082805+0000","last_undegraded":"2023-10-16T11:24:27.082805+0000","last_fullsized":"2023-10-16T11:24:27.082805+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:56.195463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070150+0000","last_change":"2023-10-16T11:24:26.093168+0000","last_active":"2023-10-16T11:24:28.070150+0000","last_peered":"2023-10-16T11:24:28.070150+0000","last_clean":"2023-10-16T11:24:28.070150+0000","last_became_active":"2023-10-16T11:24:26.091801+0000","last_became_peered":"2023-10-16T11:24:26.091801+0000","last_unstale":"2023-10-16T11:24:28.070150+0000","last_undegraded":"2023-10-16T11:24:28.070150+0000","last_fullsized":"2023-10-16T11:24:28.070150+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:43:28.275746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069327+0000","last_change":"2023-10-16T11:24:26.092718+0000","last_active":"2023-10-16T11:24:28.069327+0000","last_peered":"2023-10-16T11:24:28.069327+0000","last_clean":"2023-10-16T11:24:28.069327+0000","last_became_active":"2023-10-16T11:24:26.091377+0000","last_became_peered":"2023-10-16T11:24:26.091377+0000","last_unstale":"2023-10-16T11:24:28.069327+0000","last_undegraded":"2023-10-16T11:24:28.069327+0000","last_fullsized":"2023-10-16T11:24:28.069327+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:17.072626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.083541+0000","last_change":"2023-10-16T11:24:26.101039+0000","last_active":"2023-10-16T11:24:27.083541+0000","last_peered":"2023-10-16T11:24:27.083541+0000","last_clean":"2023-10-16T11:24:27.083541+0000","last_became_active":"2023-10-16T11:24:26.100757+0000","last_became_peered":"2023-10-16T11:24:26.100757+0000","last_unstale":"2023-10-16T11:24:27.083541+0000","last_undegraded":"2023-10-16T11:24:27.083541+0000","last_fullsized":"2023-10-16T11:24:27.083541+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.083471+0000","last_change":"2023-10-16T11:24:26.097221+0000","last_active":"2023-10-16T11:24:27.083471+0000","last_peered":"2023-10-16T11:24:27.083471+0000","last_clean":"2023-10-16T11:24:27.083471+0000","last_became_active":"2023-10-16T11:24:26.096572+0000","last_became_peered":"2023-10-16T11:24:26.096572+0000","last_unstale":"2023-10-16T11:24:27.083471+0000","last_undegraded":"2023-10-16T11:24:27.083471+0000","last_fullsized":"2023-10-16T11:24:27.083471+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069065+0000","last_change":"2023-10-16T11:24:26.091894+0000","last_active":"2023-10-16T11:24:28.069065+0000","last_peered":"2023-10-16T11:24:28.069065+0000","last_clean":"2023-10-16T11:24:28.069065+0000","last_became_active":"2023-10-16T11:24:26.091627+0000","last_became_peered":"2023-10-16T11:24:26.091627+0000","last_unstale":"2023-10-16T11:24:28.069065+0000","last_undegraded":"2023-10-16T11:24:28.069065+0000","last_fullsized":"2023-10-16T11:24:28.069065+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:46:59.832609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.200984+0000","last_change":"2023-10-16T11:24:26.094066+0000","last_active":"2023-10-16T11:24:27.200984+0000","last_peered":"2023-10-16T11:24:27.200984+0000","last_clean":"2023-10-16T11:24:27.200984+0000","last_became_active":"2023-10-16T11:24:26.093428+0000","last_became_peered":"2023-10-16T11:24:26.093428+0000","last_unstale":"2023-10-16T11:24:27.200984+0000","last_undegraded":"2023-10-16T11:24:27.200984+0000","last_fullsized":"2023-10-16T11:24:27.200984+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069693+0000","last_change":"2023-10-16T11:24:26.095913+0000","last_active":"2023-10-16T11:24:28.069693+0000","last_peered":"2023-10-16T11:24:28.069693+0000","last_clean":"2023-10-16T11:24:28.069693+0000","last_became_active":"2023-10-16T11:24:26.093113+0000","last_became_peered":"2023-10-16T11:24:26.093113+0000","last_unstale":"2023-10-16T11:24:28.069693+0000","last_undegraded":"2023-10-16T11:24:28.069693+0000","last_fullsized":"2023-10-16T11:24:28.069693+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.976416+0000","last_change":"2023-10-16T11:24:26.088015+0000","last_active":"2023-10-16T11:24:27.976416+0000","last_peered":"2023-10-16T11:24:27.976416+0000","last_clean":"2023-10-16T11:24:27.976416+0000","last_became_active":"2023-10-16T11:24:26.086932+0000","last_became_peered":"2023-10-16T11:24:26.086932+0000","last_unstale":"2023-10-16T11:24:27.976416+0000","last_undegraded":"2023-10-16T11:24:27.976416+0000","last_fullsized":"2023-10-16T11:24:27.976416+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:44.137821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.081098+0000","last_change":"2023-10-16T11:24:26.086228+0000","last_active":"2023-10-16T11:24:27.081098+0000","last_peered":"2023-10-16T11:24:27.081098+0000","last_clean":"2023-10-16T11:24:27.081098+0000","last_became_active":"2023-10-16T11:24:26.085912+0000","last_became_peered":"2023-10-16T11:24:26.085912+0000","last_unstale":"2023-10-16T11:24:27.081098+0000","last_undegraded":"2023-10-16T11:24:27.081098+0000","last_fullsized":"2023-10-16T11:24:27.081098+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.976152+0000","last_change":"2023-10-16T11:24:26.086172+0000","last_active":"2023-10-16T11:24:27.976152+0000","last_peered":"2023-10-16T11:24:27.976152+0000","last_clean":"2023-10-16T11:24:27.976152+0000","last_became_active":"2023-10-16T11:24:26.085882+0000","last_became_peered":"2023-10-16T11:24:26.085882+0000","last_unstale":"2023-10-16T11:24:27.976152+0000","last_undegraded":"2023-10-16T11:24:27.976152+0000","last_fullsized":"2023-10-16T11:24:27.976152+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:44.265192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.200688+0000","last_change":"2023-10-16T11:24:26.086008+0000","last_active":"2023-10-16T11:24:27.200688+0000","last_peered":"2023-10-16T11:24:27.200688+0000","last_clean":"2023-10-16T11:24:27.200688+0000","last_became_active":"2023-10-16T11:24:26.085598+0000","last_became_peered":"2023-10-16T11:24:26.085598+0000","last_unstale":"2023-10-16T11:24:27.200688+0000","last_undegraded":"2023-10-16T11:24:27.200688+0000","last_fullsized":"2023-10-16T11:24:27.200688+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.082611+0000","last_change":"2023-10-16T11:24:26.084964+0000","last_active":"2023-10-16T11:24:27.082611+0000","last_peered":"2023-10-16T11:24:27.082611+0000","last_clean":"2023-10-16T11:24:27.082611+0000","last_became_active":"2023-10-16T11:24:26.084006+0000","last_became_peered":"2023-10-16T11:24:26.084006+0000","last_unstale":"2023-10-16T11:24:27.082611+0000","last_undegraded":"2023-10-16T11:24:27.082611+0000","last_fullsized":"2023-10-16T11:24:27.082611+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:03.949526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.068378+0000","last_change":"2023-10-16T11:24:26.085375+0000","last_active":"2023-10-16T11:24:28.068378+0000","last_peered":"2023-10-16T11:24:28.068378+0000","last_clean":"2023-10-16T11:24:28.068378+0000","last_became_active":"2023-10-16T11:24:26.085120+0000","last_became_peered":"2023-10-16T11:24:26.085120+0000","last_unstale":"2023-10-16T11:24:28.068378+0000","last_undegraded":"2023-10-16T11:24:28.068378+0000","last_fullsized":"2023-10-16T11:24:28.068378+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.200979+0000","last_change":"2023-10-16T11:24:26.069227+0000","last_active":"2023-10-16T11:24:27.200979+0000","last_peered":"2023-10-16T11:24:27.200979+0000","last_clean":"2023-10-16T11:24:27.200979+0000","last_became_active":"2023-10-16T11:24:26.068456+0000","last_became_peered":"2023-10-16T11:24:26.068456+0000","last_unstale":"2023-10-16T11:24:27.200979+0000","last_undegraded":"2023-10-16T11:24:27.200979+0000","last_fullsized":"2023-10-16T11:24:27.200979+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"216'4","reported_seq":16,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.105044+0000","last_change":"2023-10-16T11:24:25.096644+0000","last_active":"2023-10-16T11:24:26.105044+0000","last_peered":"2023-10-16T11:24:26.105044+0000","last_clean":"2023-10-16T11:24:26.105044+0000","last_became_active":"2023-10-16T11:24:25.096306+0000","last_became_peered":"2023-10-16T11:24:25.096306+0000","last_unstale":"2023-10-16T11:24:26.105044+0000","last_undegraded":"2023-10-16T11:24:26.105044+0000","last_fullsized":"2023-10-16T11:24:26.105044+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:22.618393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.084146+0000","last_change":"2023-10-16T11:24:26.094536+0000","last_active":"2023-10-16T11:24:27.084146+0000","last_peered":"2023-10-16T11:24:27.084146+0000","last_clean":"2023-10-16T11:24:27.084146+0000","last_became_active":"2023-10-16T11:24:26.094272+0000","last_became_peered":"2023-10-16T11:24:26.094272+0000","last_unstale":"2023-10-16T11:24:27.084146+0000","last_undegraded":"2023-10-16T11:24:27.084146+0000","last_fullsized":"2023-10-16T11:24:27.084146+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"216'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.105407+0000","last_change":"2023-10-16T11:24:25.094483+0000","last_active":"2023-10-16T11:24:26.105407+0000","last_peered":"2023-10-16T11:24:26.105407+0000","last_clean":"2023-10-16T11:24:26.105407+0000","last_became_active":"2023-10-16T11:24:25.094134+0000","last_became_peered":"2023-10-16T11:24:25.094134+0000","last_unstale":"2023-10-16T11:24:26.105407+0000","last_undegraded":"2023-10-16T11:24:26.105407+0000","last_fullsized":"2023-10-16T11:24:26.105407+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069698+0000","last_change":"2023-10-16T11:24:26.089791+0000","last_active":"2023-10-16T11:24:28.069698+0000","last_peered":"2023-10-16T11:24:28.069698+0000","last_clean":"2023-10-16T11:24:28.069698+0000","last_became_active":"2023-10-16T11:24:26.089516+0000","last_became_peered":"2023-10-16T11:24:26.089516+0000","last_unstale":"2023-10-16T11:24:28.069698+0000","last_undegraded":"2023-10-16T11:24:28.069698+0000","last_fullsized":"2023-10-16T11:24:28.069698+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"216'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.105661+0000","last_change":"2023-10-16T11:24:25.081731+0000","last_active":"2023-10-16T11:24:26.105661+0000","last_peered":"2023-10-16T11:24:26.105661+0000","last_clean":"2023-10-16T11:24:26.105661+0000","last_became_active":"2023-10-16T11:24:25.081336+0000","last_became_peered":"2023-10-16T11:24:25.081336+0000","last_unstale":"2023-10-16T11:24:26.105661+0000","last_undegraded":"2023-10-16T11:24:26.105661+0000","last_fullsized":"2023-10-16T11:24:26.105661+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:09.330801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.082147+0000","last_change":"2023-10-16T11:24:26.096297+0000","last_active":"2023-10-16T11:24:27.082147+0000","last_peered":"2023-10-16T11:24:27.082147+0000","last_clean":"2023-10-16T11:24:27.082147+0000","last_became_active":"2023-10-16T11:24:26.095538+0000","last_became_peered":"2023-10-16T11:24:26.095538+0000","last_unstale":"2023-10-16T11:24:27.082147+0000","last_undegraded":"2023-10-16T11:24:27.082147+0000","last_fullsized":"2023-10-16T11:24:27.082147+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:47.400027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"216'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.103907+0000","last_change":"2023-10-16T11:24:25.095179+0000","last_active":"2023-10-16T11:24:26.103907+0000","last_peered":"2023-10-16T11:24:26.103907+0000","last_clean":"2023-10-16T11:24:26.103907+0000","last_became_active":"2023-10-16T11:24:25.094714+0000","last_became_peered":"2023-10-16T11:24:25.094714+0000","last_unstale":"2023-10-16T11:24:26.103907+0000","last_undegraded":"2023-10-16T11:24:26.103907+0000","last_fullsized":"2023-10-16T11:24:26.103907+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:26.128614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069793+0000","last_change":"2023-10-16T11:24:26.089982+0000","last_active":"2023-10-16T11:24:28.069793+0000","last_peered":"2023-10-16T11:24:28.069793+0000","last_clean":"2023-10-16T11:24:28.069793+0000","last_became_active":"2023-10-16T11:24:26.089677+0000","last_became_peered":"2023-10-16T11:24:26.089677+0000","last_unstale":"2023-10-16T11:24:28.069793+0000","last_undegraded":"2023-10-16T11:24:28.069793+0000","last_fullsized":"2023-10-16T11:24:28.069793+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:22.009042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"216'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.104624+0000","last_change":"2023-10-16T11:24:25.084983+0000","last_active":"2023-10-16T11:24:26.104624+0000","last_peered":"2023-10-16T11:24:26.104624+0000","last_clean":"2023-10-16T11:24:26.104624+0000","last_became_active":"2023-10-16T11:24:25.084622+0000","last_became_peered":"2023-10-16T11:24:25.084622+0000","last_unstale":"2023-10-16T11:24:26.104624+0000","last_undegraded":"2023-10-16T11:24:26.104624+0000","last_fullsized":"2023-10-16T11:24:26.104624+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:36.588809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069256+0000","last_change":"2023-10-16T11:24:26.088785+0000","last_active":"2023-10-16T11:24:28.069256+0000","last_peered":"2023-10-16T11:24:28.069256+0000","last_clean":"2023-10-16T11:24:28.069256+0000","last_became_active":"2023-10-16T11:24:26.088521+0000","last_became_peered":"2023-10-16T11:24:26.088521+0000","last_unstale":"2023-10-16T11:24:28.069256+0000","last_undegraded":"2023-10-16T11:24:28.069256+0000","last_fullsized":"2023-10-16T11:24:28.069256+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:29.145795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.074788+0000","last_change":"2023-10-16T11:24:25.074647+0000","last_active":"2023-10-16T11:24:26.074788+0000","last_peered":"2023-10-16T11:24:26.074788+0000","last_clean":"2023-10-16T11:24:26.074788+0000","last_became_active":"2023-10-16T11:24:25.074373+0000","last_became_peered":"2023-10-16T11:24:25.074373+0000","last_unstale":"2023-10-16T11:24:26.074788+0000","last_undegraded":"2023-10-16T11:24:26.074788+0000","last_fullsized":"2023-10-16T11:24:26.074788+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:38.544520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069199+0000","last_change":"2023-10-16T11:24:26.091641+0000","last_active":"2023-10-16T11:24:28.069199+0000","last_peered":"2023-10-16T11:24:28.069199+0000","last_clean":"2023-10-16T11:24:28.069199+0000","last_became_active":"2023-10-16T11:24:26.089186+0000","last_became_peered":"2023-10-16T11:24:26.089186+0000","last_unstale":"2023-10-16T11:24:28.069199+0000","last_undegraded":"2023-10-16T11:24:28.069199+0000","last_fullsized":"2023-10-16T11:24:28.069199+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:32.247878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.078739+0000","last_change":"2023-10-16T11:24:25.084521+0000","last_active":"2023-10-16T11:24:26.078739+0000","last_peered":"2023-10-16T11:24:26.078739+0000","last_clean":"2023-10-16T11:24:26.078739+0000","last_became_active":"2023-10-16T11:24:25.084003+0000","last_became_peered":"2023-10-16T11:24:25.084003+0000","last_unstale":"2023-10-16T11:24:26.078739+0000","last_undegraded":"2023-10-16T11:24:26.078739+0000","last_fullsized":"2023-10-16T11:24:26.078739+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:36.060066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070258+0000","last_change":"2023-10-16T11:24:26.082915+0000","last_active":"2023-10-16T11:24:28.070258+0000","last_peered":"2023-10-16T11:24:28.070258+0000","last_clean":"2023-10-16T11:24:28.070258+0000","last_became_active":"2023-10-16T11:24:26.082458+0000","last_became_peered":"2023-10-16T11:24:26.082458+0000","last_unstale":"2023-10-16T11:24:28.070258+0000","last_undegraded":"2023-10-16T11:24:28.070258+0000","last_fullsized":"2023-10-16T11:24:28.070258+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:10.387018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.076876+0000","last_change":"2023-10-16T11:24:25.094384+0000","last_active":"2023-10-16T11:24:26.076876+0000","last_peered":"2023-10-16T11:24:26.076876+0000","last_clean":"2023-10-16T11:24:26.076876+0000","last_became_active":"2023-10-16T11:24:25.094026+0000","last_became_peered":"2023-10-16T11:24:25.094026+0000","last_unstale":"2023-10-16T11:24:26.076876+0000","last_undegraded":"2023-10-16T11:24:26.076876+0000","last_fullsized":"2023-10-16T11:24:26.076876+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:51.349970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.081340+0000","last_change":"2023-10-16T11:24:26.081980+0000","last_active":"2023-10-16T11:24:27.081340+0000","last_peered":"2023-10-16T11:24:27.081340+0000","last_clean":"2023-10-16T11:24:27.081340+0000","last_became_active":"2023-10-16T11:24:26.081395+0000","last_became_peered":"2023-10-16T11:24:26.081395+0000","last_unstale":"2023-10-16T11:24:27.081340+0000","last_undegraded":"2023-10-16T11:24:27.081340+0000","last_fullsized":"2023-10-16T11:24:27.081340+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:39.802117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.086320+0000","last_change":"2023-10-16T11:24:25.092100+0000","last_active":"2023-10-16T11:24:26.086320+0000","last_peered":"2023-10-16T11:24:26.086320+0000","last_clean":"2023-10-16T11:24:26.086320+0000","last_became_active":"2023-10-16T11:24:25.091237+0000","last_became_peered":"2023-10-16T11:24:25.091237+0000","last_unstale":"2023-10-16T11:24:26.086320+0000","last_undegraded":"2023-10-16T11:24:26.086320+0000","last_fullsized":"2023-10-16T11:24:26.086320+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:21.985839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069623+0000","last_change":"2023-10-16T11:24:26.084487+0000","last_active":"2023-10-16T11:24:28.069623+0000","last_peered":"2023-10-16T11:24:28.069623+0000","last_clean":"2023-10-16T11:24:28.069623+0000","last_became_active":"2023-10-16T11:24:26.084188+0000","last_became_peered":"2023-10-16T11:24:26.084188+0000","last_unstale":"2023-10-16T11:24:28.069623+0000","last_undegraded":"2023-10-16T11:24:28.069623+0000","last_fullsized":"2023-10-16T11:24:28.069623+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:47.400027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"216'3","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.108001+0000","last_change":"2023-10-16T11:24:25.082968+0000","last_active":"2023-10-16T11:24:26.108001+0000","last_peered":"2023-10-16T11:24:26.108001+0000","last_clean":"2023-10-16T11:24:26.108001+0000","last_became_active":"2023-10-16T11:24:25.082316+0000","last_became_peered":"2023-10-16T11:24:25.082316+0000","last_unstale":"2023-10-16T11:24:26.108001+0000","last_undegraded":"2023-10-16T11:24:26.108001+0000","last_fullsized":"2023-10-16T11:24:26.108001+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:59.924424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069834+0000","last_change":"2023-10-16T11:24:26.092185+0000","last_active":"2023-10-16T11:24:28.069834+0000","last_peered":"2023-10-16T11:24:28.069834+0000","last_clean":"2023-10-16T11:24:28.069834+0000","last_became_active":"2023-10-16T11:24:26.091648+0000","last_became_peered":"2023-10-16T11:24:26.091648+0000","last_unstale":"2023-10-16T11:24:28.069834+0000","last_undegraded":"2023-10-16T11:24:28.069834+0000","last_fullsized":"2023-10-16T11:24:28.069834+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"216'6","reported_seq":18,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.107445+0000","last_change":"2023-10-16T11:24:25.095357+0000","last_active":"2023-10-16T11:24:26.107445+0000","last_peered":"2023-10-16T11:24:26.107445+0000","last_clean":"2023-10-16T11:24:26.107445+0000","last_became_active":"2023-10-16T11:24:25.093832+0000","last_became_peered":"2023-10-16T11:24:25.093832+0000","last_unstale":"2023-10-16T11:24:26.107445+0000","last_undegraded":"2023-10-16T11:24:26.107445+0000","last_fullsized":"2023-10-16T11:24:26.107445+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:14:19.482469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.071056+0000","last_change":"2023-10-16T11:24:26.099039+0000","last_active":"2023-10-16T11:24:28.071056+0000","last_peered":"2023-10-16T11:24:28.071056+0000","last_clean":"2023-10-16T11:24:28.071056+0000","last_became_active":"2023-10-16T11:24:26.097980+0000","last_became_peered":"2023-10-16T11:24:26.097980+0000","last_unstale":"2023-10-16T11:24:28.071056+0000","last_undegraded":"2023-10-16T11:24:28.071056+0000","last_fullsized":"2023-10-16T11:24:28.071056+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:40.585267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"216'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.102723+0000","last_change":"2023-10-16T11:24:25.081377+0000","last_active":"2023-10-16T11:24:26.102723+0000","last_peered":"2023-10-16T11:24:26.102723+0000","last_clean":"2023-10-16T11:24:26.102723+0000","last_became_active":"2023-10-16T11:24:25.080881+0000","last_became_peered":"2023-10-16T11:24:25.080881+0000","last_unstale":"2023-10-16T11:24:26.102723+0000","last_undegraded":"2023-10-16T11:24:26.102723+0000","last_fullsized":"2023-10-16T11:24:26.102723+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:26.054702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.082295+0000","last_change":"2023-10-16T11:24:26.089221+0000","last_active":"2023-10-16T11:24:27.082295+0000","last_peered":"2023-10-16T11:24:27.082295+0000","last_clean":"2023-10-16T11:24:27.082295+0000","last_became_active":"2023-10-16T11:24:26.088644+0000","last_became_peered":"2023-10-16T11:24:26.088644+0000","last_unstale":"2023-10-16T11:24:27.082295+0000","last_undegraded":"2023-10-16T11:24:27.082295+0000","last_fullsized":"2023-10-16T11:24:27.082295+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:10.387018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.082105+0000","last_change":"2023-10-16T11:24:25.084194+0000","last_active":"2023-10-16T11:24:26.082105+0000","last_peered":"2023-10-16T11:24:26.082105+0000","last_clean":"2023-10-16T11:24:26.082105+0000","last_became_active":"2023-10-16T11:24:25.083141+0000","last_became_peered":"2023-10-16T11:24:25.083141+0000","last_unstale":"2023-10-16T11:24:26.082105+0000","last_undegraded":"2023-10-16T11:24:26.082105+0000","last_fullsized":"2023-10-16T11:24:26.082105+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:46.770213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.080140+0000","last_change":"2023-10-16T11:24:26.079602+0000","last_active":"2023-10-16T11:24:27.080140+0000","last_peered":"2023-10-16T11:24:27.080140+0000","last_clean":"2023-10-16T11:24:27.080140+0000","last_became_active":"2023-10-16T11:24:26.079344+0000","last_became_peered":"2023-10-16T11:24:26.079344+0000","last_unstale":"2023-10-16T11:24:27.080140+0000","last_undegraded":"2023-10-16T11:24:27.080140+0000","last_fullsized":"2023-10-16T11:24:27.080140+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.077949+0000","last_change":"2023-10-16T11:24:25.089470+0000","last_active":"2023-10-16T11:24:26.077949+0000","last_peered":"2023-10-16T11:24:26.077949+0000","last_clean":"2023-10-16T11:24:26.077949+0000","last_became_active":"2023-10-16T11:24:25.089213+0000","last_became_peered":"2023-10-16T11:24:25.089213+0000","last_unstale":"2023-10-16T11:24:26.077949+0000","last_undegraded":"2023-10-16T11:24:26.077949+0000","last_fullsized":"2023-10-16T11:24:26.077949+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:34.540849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.068721+0000","last_change":"2023-10-16T11:24:26.084139+0000","last_active":"2023-10-16T11:24:28.068721+0000","last_peered":"2023-10-16T11:24:28.068721+0000","last_clean":"2023-10-16T11:24:28.068721+0000","last_became_active":"2023-10-16T11:24:26.083876+0000","last_became_peered":"2023-10-16T11:24:26.083876+0000","last_unstale":"2023-10-16T11:24:28.068721+0000","last_undegraded":"2023-10-16T11:24:28.068721+0000","last_fullsized":"2023-10-16T11:24:28.068721+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"216'8","reported_seq":20,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.107007+0000","last_change":"2023-10-16T11:24:25.076954+0000","last_active":"2023-10-16T11:24:26.107007+0000","last_peered":"2023-10-16T11:24:26.107007+0000","last_clean":"2023-10-16T11:24:26.107007+0000","last_became_active":"2023-10-16T11:24:25.076509+0000","last_became_peered":"2023-10-16T11:24:25.076509+0000","last_unstale":"2023-10-16T11:24:26.107007+0000","last_undegraded":"2023-10-16T11:24:26.107007+0000","last_fullsized":"2023-10-16T11:24:26.107007+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:46.770213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.201200+0000","last_change":"2023-10-16T11:24:26.081796+0000","last_active":"2023-10-16T11:24:27.201200+0000","last_peered":"2023-10-16T11:24:27.201200+0000","last_clean":"2023-10-16T11:24:27.201200+0000","last_became_active":"2023-10-16T11:24:26.080287+0000","last_became_peered":"2023-10-16T11:24:26.080287+0000","last_unstale":"2023-10-16T11:24:27.201200+0000","last_undegraded":"2023-10-16T11:24:27.201200+0000","last_fullsized":"2023-10-16T11:24:27.201200+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"216'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.105872+0000","last_change":"2023-10-16T11:24:25.079789+0000","last_active":"2023-10-16T11:24:26.105872+0000","last_peered":"2023-10-16T11:24:26.105872+0000","last_clean":"2023-10-16T11:24:26.105872+0000","last_became_active":"2023-10-16T11:24:25.079495+0000","last_became_peered":"2023-10-16T11:24:25.079495+0000","last_unstale":"2023-10-16T11:24:26.105872+0000","last_undegraded":"2023-10-16T11:24:26.105872+0000","last_fullsized":"2023-10-16T11:24:26.105872+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:44.531367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.201013+0000","last_change":"2023-10-16T11:24:26.086833+0000","last_active":"2023-10-16T11:24:27.201013+0000","last_peered":"2023-10-16T11:24:27.201013+0000","last_clean":"2023-10-16T11:24:27.201013+0000","last_became_active":"2023-10-16T11:24:26.085367+0000","last_became_peered":"2023-10-16T11:24:26.085367+0000","last_unstale":"2023-10-16T11:24:27.201013+0000","last_undegraded":"2023-10-16T11:24:27.201013+0000","last_fullsized":"2023-10-16T11:24:27.201013+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"216'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.106988+0000","last_change":"2023-10-16T11:24:25.087765+0000","last_active":"2023-10-16T11:24:26.106988+0000","last_peered":"2023-10-16T11:24:26.106988+0000","last_clean":"2023-10-16T11:24:26.106988+0000","last_became_active":"2023-10-16T11:24:25.086967+0000","last_became_peered":"2023-10-16T11:24:25.086967+0000","last_unstale":"2023-10-16T11:24:26.106988+0000","last_undegraded":"2023-10-16T11:24:26.106988+0000","last_fullsized":"2023-10-16T11:24:26.106988+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.080678+0000","last_change":"2023-10-16T11:24:26.069780+0000","last_active":"2023-10-16T11:24:27.080678+0000","last_peered":"2023-10-16T11:24:27.080678+0000","last_clean":"2023-10-16T11:24:27.080678+0000","last_became_active":"2023-10-16T11:24:26.068772+0000","last_became_peered":"2023-10-16T11:24:26.068772+0000","last_unstale":"2023-10-16T11:24:27.080678+0000","last_undegraded":"2023-10-16T11:24:27.080678+0000","last_fullsized":"2023-10-16T11:24:27.080678+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"216'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.107193+0000","last_change":"2023-10-16T11:24:25.083155+0000","last_active":"2023-10-16T11:24:26.107193+0000","last_peered":"2023-10-16T11:24:26.107193+0000","last_clean":"2023-10-16T11:24:26.107193+0000","last_became_active":"2023-10-16T11:24:25.082690+0000","last_became_peered":"2023-10-16T11:24:25.082690+0000","last_unstale":"2023-10-16T11:24:26.107193+0000","last_undegraded":"2023-10-16T11:24:26.107193+0000","last_fullsized":"2023-10-16T11:24:26.107193+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:46:58.816018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.082502+0000","last_change":"2023-10-16T11:24:26.092876+0000","last_active":"2023-10-16T11:24:27.082502+0000","last_peered":"2023-10-16T11:24:27.082502+0000","last_clean":"2023-10-16T11:24:27.082502+0000","last_became_active":"2023-10-16T11:24:26.091310+0000","last_became_peered":"2023-10-16T11:24:26.091310+0000","last_unstale":"2023-10-16T11:24:27.082502+0000","last_undegraded":"2023-10-16T11:24:27.082502+0000","last_fullsized":"2023-10-16T11:24:27.082502+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:22.891780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.057300+0000","last_change":"2023-10-16T11:24:25.079021+0000","last_active":"2023-10-16T11:24:26.057300+0000","last_peered":"2023-10-16T11:24:26.057300+0000","last_clean":"2023-10-16T11:24:26.057300+0000","last_became_active":"2023-10-16T11:24:25.076800+0000","last_became_peered":"2023-10-16T11:24:25.076800+0000","last_unstale":"2023-10-16T11:24:26.057300+0000","last_undegraded":"2023-10-16T11:24:26.057300+0000","last_fullsized":"2023-10-16T11:24:26.057300+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:56.161407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.082026+0000","last_change":"2023-10-16T11:24:26.091096+0000","last_active":"2023-10-16T11:24:27.082026+0000","last_peered":"2023-10-16T11:24:27.082026+0000","last_clean":"2023-10-16T11:24:27.082026+0000","last_became_active":"2023-10-16T11:24:26.090774+0000","last_became_peered":"2023-10-16T11:24:26.090774+0000","last_unstale":"2023-10-16T11:24:27.082026+0000","last_undegraded":"2023-10-16T11:24:27.082026+0000","last_fullsized":"2023-10-16T11:24:27.082026+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:27.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"216'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.107501+0000","last_change":"2023-10-16T11:24:25.072340+0000","last_active":"2023-10-16T11:24:26.107501+0000","last_peered":"2023-10-16T11:24:26.107501+0000","last_clean":"2023-10-16T11:24:26.107501+0000","last_became_active":"2023-10-16T11:24:25.071728+0000","last_became_peered":"2023-10-16T11:24:25.071728+0000","last_unstale":"2023-10-16T11:24:26.107501+0000","last_undegraded":"2023-10-16T11:24:26.107501+0000","last_fullsized":"2023-10-16T11:24:26.107501+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:49.041521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.081078+0000","last_change":"2023-10-16T11:24:26.085260+0000","last_active":"2023-10-16T11:24:27.081078+0000","last_peered":"2023-10-16T11:24:27.081078+0000","last_clean":"2023-10-16T11:24:27.081078+0000","last_became_active":"2023-10-16T11:24:26.084860+0000","last_became_peered":"2023-10-16T11:24:26.084860+0000","last_unstale":"2023-10-16T11:24:27.081078+0000","last_undegraded":"2023-10-16T11:24:27.081078+0000","last_fullsized":"2023-10-16T11:24:27.081078+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.081605+0000","last_change":"2023-10-16T11:24:25.091656+0000","last_active":"2023-10-16T11:24:26.081605+0000","last_peered":"2023-10-16T11:24:26.081605+0000","last_clean":"2023-10-16T11:24:26.081605+0000","last_became_active":"2023-10-16T11:24:25.091316+0000","last_became_peered":"2023-10-16T11:24:25.091316+0000","last_unstale":"2023-10-16T11:24:26.081605+0000","last_undegraded":"2023-10-16T11:24:26.081605+0000","last_fullsized":"2023-10-16T11:24:26.081605+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:26.054702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.200399+0000","last_change":"2023-10-16T11:24:26.090951+0000","last_active":"2023-10-16T11:24:27.200399+0000","last_peered":"2023-10-16T11:24:27.200399+0000","last_clean":"2023-10-16T11:24:27.200399+0000","last_became_active":"2023-10-16T11:24:26.089241+0000","last_became_peered":"2023-10-16T11:24:26.089241+0000","last_unstale":"2023-10-16T11:24:27.200399+0000","last_undegraded":"2023-10-16T11:24:27.200399+0000","last_fullsized":"2023-10-16T11:24:27.200399+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:40.970357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.074587+0000","last_change":"2023-10-16T11:24:25.086180+0000","last_active":"2023-10-16T11:24:26.074587+0000","last_peered":"2023-10-16T11:24:26.074587+0000","last_clean":"2023-10-16T11:24:26.074587+0000","last_became_active":"2023-10-16T11:24:25.085316+0000","last_became_peered":"2023-10-16T11:24:25.085316+0000","last_unstale":"2023-10-16T11:24:26.074587+0000","last_undegraded":"2023-10-16T11:24:26.074587+0000","last_fullsized":"2023-10-16T11:24:26.074587+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.083967+0000","last_change":"2023-10-16T11:24:26.088438+0000","last_active":"2023-10-16T11:24:27.083967+0000","last_peered":"2023-10-16T11:24:27.083967+0000","last_clean":"2023-10-16T11:24:27.083967+0000","last_became_active":"2023-10-16T11:24:26.087973+0000","last_became_peered":"2023-10-16T11:24:26.087973+0000","last_unstale":"2023-10-16T11:24:27.083967+0000","last_undegraded":"2023-10-16T11:24:27.083967+0000","last_fullsized":"2023-10-16T11:24:27.083967+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.076600+0000","last_change":"2023-10-16T11:24:25.080225+0000","last_active":"2023-10-16T11:24:26.076600+0000","last_peered":"2023-10-16T11:24:26.076600+0000","last_clean":"2023-10-16T11:24:26.076600+0000","last_became_active":"2023-10-16T11:24:25.079921+0000","last_became_peered":"2023-10-16T11:24:25.079921+0000","last_unstale":"2023-10-16T11:24:26.076600+0000","last_undegraded":"2023-10-16T11:24:26.076600+0000","last_fullsized":"2023-10-16T11:24:26.076600+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:46.550429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.976480+0000","last_change":"2023-10-16T11:24:26.085909+0000","last_active":"2023-10-16T11:24:27.976480+0000","last_peered":"2023-10-16T11:24:27.976480+0000","last_clean":"2023-10-16T11:24:27.976480+0000","last_became_active":"2023-10-16T11:24:26.085620+0000","last_became_peered":"2023-10-16T11:24:26.085620+0000","last_unstale":"2023-10-16T11:24:27.976480+0000","last_undegraded":"2023-10-16T11:24:27.976480+0000","last_fullsized":"2023-10-16T11:24:27.976480+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:41.023833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"216'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.106261+0000","last_change":"2023-10-16T11:24:25.094230+0000","last_active":"2023-10-16T11:24:26.106261+0000","last_peered":"2023-10-16T11:24:26.106261+0000","last_clean":"2023-10-16T11:24:26.106261+0000","last_became_active":"2023-10-16T11:24:25.093896+0000","last_became_peered":"2023-10-16T11:24:25.093896+0000","last_unstale":"2023-10-16T11:24:26.106261+0000","last_undegraded":"2023-10-16T11:24:26.106261+0000","last_fullsized":"2023-10-16T11:24:26.106261+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070457+0000","last_change":"2023-10-16T11:24:26.089940+0000","last_active":"2023-10-16T11:24:28.070457+0000","last_peered":"2023-10-16T11:24:28.070457+0000","last_clean":"2023-10-16T11:24:28.070457+0000","last_became_active":"2023-10-16T11:24:26.087906+0000","last_became_peered":"2023-10-16T11:24:26.087906+0000","last_unstale":"2023-10-16T11:24:28.070457+0000","last_undegraded":"2023-10-16T11:24:28.070457+0000","last_fullsized":"2023-10-16T11:24:28.070457+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:44.687193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.076347+0000","last_change":"2023-10-16T11:24:25.093214+0000","last_active":"2023-10-16T11:24:26.076347+0000","last_peered":"2023-10-16T11:24:26.076347+0000","last_clean":"2023-10-16T11:24:26.076347+0000","last_became_active":"2023-10-16T11:24:25.092877+0000","last_became_peered":"2023-10-16T11:24:25.092877+0000","last_unstale":"2023-10-16T11:24:26.076347+0000","last_undegraded":"2023-10-16T11:24:26.076347+0000","last_fullsized":"2023-10-16T11:24:26.076347+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:59.924424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.200857+0000","last_change":"2023-10-16T11:24:26.085231+0000","last_active":"2023-10-16T11:24:27.200857+0000","last_peered":"2023-10-16T11:24:27.200857+0000","last_clean":"2023-10-16T11:24:27.200857+0000","last_became_active":"2023-10-16T11:24:26.084889+0000","last_became_peered":"2023-10-16T11:24:26.084889+0000","last_unstale":"2023-10-16T11:24:27.200857+0000","last_undegraded":"2023-10-16T11:24:27.200857+0000","last_fullsized":"2023-10-16T11:24:27.200857+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.077130+0000","last_change":"2023-10-16T11:24:25.077036+0000","last_active":"2023-10-16T11:24:26.077130+0000","last_peered":"2023-10-16T11:24:26.077130+0000","last_clean":"2023-10-16T11:24:26.077130+0000","last_became_active":"2023-10-16T11:24:25.076342+0000","last_became_peered":"2023-10-16T11:24:25.076342+0000","last_unstale":"2023-10-16T11:24:26.077130+0000","last_undegraded":"2023-10-16T11:24:26.077130+0000","last_fullsized":"2023-10-16T11:24:26.077130+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:36.162662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.084158+0000","last_change":"2023-10-16T11:24:26.096341+0000","last_active":"2023-10-16T11:24:27.084158+0000","last_peered":"2023-10-16T11:24:27.084158+0000","last_clean":"2023-10-16T11:24:27.084158+0000","last_became_active":"2023-10-16T11:24:26.095836+0000","last_became_peered":"2023-10-16T11:24:26.095836+0000","last_unstale":"2023-10-16T11:24:27.084158+0000","last_undegraded":"2023-10-16T11:24:27.084158+0000","last_fullsized":"2023-10-16T11:24:27.084158+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.057631+0000","last_change":"2023-10-16T11:24:25.088585+0000","last_active":"2023-10-16T11:24:26.057631+0000","last_peered":"2023-10-16T11:24:26.057631+0000","last_clean":"2023-10-16T11:24:26.057631+0000","last_became_active":"2023-10-16T11:24:25.088011+0000","last_became_peered":"2023-10-16T11:24:25.088011+0000","last_unstale":"2023-10-16T11:24:26.057631+0000","last_undegraded":"2023-10-16T11:24:26.057631+0000","last_fullsized":"2023-10-16T11:24:26.057631+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069619+0000","last_change":"2023-10-16T11:24:26.093027+0000","last_active":"2023-10-16T11:24:28.069619+0000","last_peered":"2023-10-16T11:24:28.069619+0000","last_clean":"2023-10-16T11:24:28.069619+0000","last_became_active":"2023-10-16T11:24:26.092766+0000","last_became_peered":"2023-10-16T11:24:26.092766+0000","last_unstale":"2023-10-16T11:24:28.069619+0000","last_undegraded":"2023-10-16T11:24:28.069619+0000","last_fullsized":"2023-10-16T11:24:28.069619+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.073725+0000","last_change":"2023-10-16T11:24:25.090432+0000","last_active":"2023-10-16T11:24:26.073725+0000","last_peered":"2023-10-16T11:24:26.073725+0000","last_clean":"2023-10-16T11:24:26.073725+0000","last_became_active":"2023-10-16T11:24:25.090142+0000","last_became_peered":"2023-10-16T11:24:25.090142+0000","last_unstale":"2023-10-16T11:24:26.073725+0000","last_undegraded":"2023-10-16T11:24:26.073725+0000","last_fullsized":"2023-10-16T11:24:26.073725+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:22.858690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.976530+0000","last_change":"2023-10-16T11:24:26.083766+0000","last_active":"2023-10-16T11:24:27.976530+0000","last_peered":"2023-10-16T11:24:27.976530+0000","last_clean":"2023-10-16T11:24:27.976530+0000","last_became_active":"2023-10-16T11:24:26.083447+0000","last_became_peered":"2023-10-16T11:24:26.083447+0000","last_unstale":"2023-10-16T11:24:27.976530+0000","last_undegraded":"2023-10-16T11:24:27.976530+0000","last_fullsized":"2023-10-16T11:24:27.976530+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:44:14.656443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.078744+0000","last_change":"2023-10-16T11:24:25.080689+0000","last_active":"2023-10-16T11:24:26.078744+0000","last_peered":"2023-10-16T11:24:26.078744+0000","last_clean":"2023-10-16T11:24:26.078744+0000","last_became_active":"2023-10-16T11:24:25.080305+0000","last_became_peered":"2023-10-16T11:24:25.080305+0000","last_unstale":"2023-10-16T11:24:26.078744+0000","last_undegraded":"2023-10-16T11:24:26.078744+0000","last_fullsized":"2023-10-16T11:24:26.078744+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:43.670603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.080245+0000","last_change":"2023-10-16T11:24:26.083366+0000","last_active":"2023-10-16T11:24:27.080245+0000","last_peered":"2023-10-16T11:24:27.080245+0000","last_clean":"2023-10-16T11:24:27.080245+0000","last_became_active":"2023-10-16T11:24:26.082635+0000","last_became_peered":"2023-10-16T11:24:26.082635+0000","last_unstale":"2023-10-16T11:24:27.080245+0000","last_undegraded":"2023-10-16T11:24:27.080245+0000","last_fullsized":"2023-10-16T11:24:27.080245+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.087162+0000","last_change":"2023-10-16T11:24:25.088161+0000","last_active":"2023-10-16T11:24:26.087162+0000","last_peered":"2023-10-16T11:24:26.087162+0000","last_clean":"2023-10-16T11:24:26.087162+0000","last_became_active":"2023-10-16T11:24:25.086051+0000","last_became_peered":"2023-10-16T11:24:25.086051+0000","last_unstale":"2023-10-16T11:24:26.087162+0000","last_undegraded":"2023-10-16T11:24:26.087162+0000","last_fullsized":"2023-10-16T11:24:26.087162+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.200464+0000","last_change":"2023-10-16T11:24:26.081718+0000","last_active":"2023-10-16T11:24:27.200464+0000","last_peered":"2023-10-16T11:24:27.200464+0000","last_clean":"2023-10-16T11:24:27.200464+0000","last_became_active":"2023-10-16T11:24:26.081447+0000","last_became_peered":"2023-10-16T11:24:26.081447+0000","last_unstale":"2023-10-16T11:24:27.200464+0000","last_undegraded":"2023-10-16T11:24:27.200464+0000","last_fullsized":"2023-10-16T11:24:27.200464+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.067543+0000","last_change":"2023-10-16T11:24:25.087755+0000","last_active":"2023-10-16T11:24:26.067543+0000","last_peered":"2023-10-16T11:24:26.067543+0000","last_clean":"2023-10-16T11:24:26.067543+0000","last_became_active":"2023-10-16T11:24:25.087497+0000","last_became_peered":"2023-10-16T11:24:25.087497+0000","last_unstale":"2023-10-16T11:24:26.067543+0000","last_undegraded":"2023-10-16T11:24:26.067543+0000","last_fullsized":"2023-10-16T11:24:26.067543+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:59:26.196708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070405+0000","last_change":"2023-10-16T11:24:26.091340+0000","last_active":"2023-10-16T11:24:28.070405+0000","last_peered":"2023-10-16T11:24:28.070405+0000","last_clean":"2023-10-16T11:24:28.070405+0000","last_became_active":"2023-10-16T11:24:26.091059+0000","last_became_peered":"2023-10-16T11:24:26.091059+0000","last_unstale":"2023-10-16T11:24:28.070405+0000","last_undegraded":"2023-10-16T11:24:28.070405+0000","last_fullsized":"2023-10-16T11:24:28.070405+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:37.605400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"216'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.103314+0000","last_change":"2023-10-16T11:24:25.087491+0000","last_active":"2023-10-16T11:24:26.103314+0000","last_peered":"2023-10-16T11:24:26.103314+0000","last_clean":"2023-10-16T11:24:26.103314+0000","last_became_active":"2023-10-16T11:24:25.086826+0000","last_became_peered":"2023-10-16T11:24:25.086826+0000","last_unstale":"2023-10-16T11:24:26.103314+0000","last_undegraded":"2023-10-16T11:24:26.103314+0000","last_fullsized":"2023-10-16T11:24:26.103314+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:07.303719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"206'2767","reported_seq":3648,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.072119+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:24:26.072119+0000","last_peered":"2023-10-16T11:24:26.072119+0000","last_clean":"2023-10-16T11:24:26.072119+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:24:26.072119+0000","last_undegraded":"2023-10-16T11:24:26.072119+0000","last_fullsized":"2023-10-16T11:24:26.072119+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2767,"log_dups_size":0,"ondisk_log_size":2767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":581,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":581,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2767,"ondisk_log_size":2767,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575986,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52480,"kb_used_data":3252,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319360,"statfs":{"total":96636764160,"available":96583024640,"internally_reserved":0,"allocated":3330048,"data_stored":1912427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.179,"15min":1.2170000000000001},"min":{"1min":0.81499999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.7250000000000001,"5min":2.085,"15min":2.3399999999999999},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.1719999999999999,"15min":1.218},"min":{"1min":0.876,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.8320000000000001},"last":1.333}]},{"osd":1,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2689999999999999,"15min":1.2509999999999999},"min":{"1min":0.90700000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.6479999999999999,"5min":1.913,"15min":1.913},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2430000000000001,"15min":1.2749999999999999},"min":{"1min":0.76700000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6279999999999999,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.133}]},{"osd":2,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3140000000000001,"15min":1.3109999999999999},"min":{"1min":1.008,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6579999999999999,"5min":1.8680000000000001,"15min":2.4460000000000002},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.343,"5min":1.3540000000000001,"15min":1.3640000000000001},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.708,"5min":1.859,"15min":1.974},"last":1.004}]},{"osd":3,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.381,"15min":1.3899999999999999},"min":{"1min":1.149,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7290000000000001,"5min":1.8280000000000001,"15min":1.972},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.4430000000000001,"15min":1.4630000000000001},"min":{"1min":1.0860000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.8799999999999999,"5min":2.0680000000000001,"15min":2.577},"last":1.1850000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.423,"15min":1.417},"min":{"1min":0.98599999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0030000000000001,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.425},{"interface":"front","average":{"1min":1.498,"5min":1.391,"15min":1.3899999999999999},"min":{"1min":1.0469999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8280000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.5089999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.585,"15min":1.5960000000000001},"min":{"1min":1.1579999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.946,"5min":2.2309999999999999,"15min":3.3519999999999999},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.641,"5min":1.514,"15min":1.5089999999999999},"min":{"1min":1.351,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.1600000000000001,"5min":2.3460000000000001,"15min":2.4969999999999999},"last":1.6950000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.488,"15min":1.5209999999999999},"min":{"1min":0.97099999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.845,"5min":2.101,"15min":2.7250000000000001},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.7210000000000001,"5min":1.6890000000000001,"15min":1.71},"min":{"1min":1.425,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1040000000000001,"5min":2.298,"15min":3.1440000000000001},"last":1.5649999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575986,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33848,"kb_used_data":2664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337992,"statfs":{"total":96636764160,"available":96602103808,"internally_reserved":0,"allocated":2727936,"data_stored":1321029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.2,"15min":1.22},"min":{"1min":0.78000000000000003,"5min":0.77200000000000002,"15min":0.64000000000000001},"max":{"1min":1.5740000000000001,"5min":1.698,"15min":2.0419999999999998},"last":1.278},{"interface":"front","average":{"1min":1.355,"5min":1.2589999999999999,"15min":1.2669999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.66400000000000003},"max":{"1min":1.6799999999999999,"5min":1.839,"15min":1.839},"last":1.466}]},{"osd":1,"last update":"Mon Oct 16 11:23:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2230000000000001,"15min":1.2130000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7350000000000001,"5min":1.7629999999999999,"15min":1.8420000000000001},"last":1.409},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3640000000000001,"15min":1.351},"min":{"1min":0.94099999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6910000000000001,"5min":1.796,"15min":1.796},"last":1.2130000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:23:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.2410000000000001,"15min":1.244},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.68000000000000005},"max":{"1min":1.579,"5min":1.7589999999999999,"15min":1.861},"last":1.806},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3500000000000001,"15min":1.3540000000000001},"min":{"1min":0.92000000000000004,"5min":0.85799999999999998,"15min":0.72699999999999998},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.9790000000000001},"last":1.119}]},{"osd":3,"last update":"Mon Oct 16 11:23:35 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.341,"15min":1.3100000000000001},"min":{"1min":0.95699999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.6379999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.708},{"interface":"front","average":{"1min":1.236,"5min":1.256,"15min":1.264},"min":{"1min":0.91700000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.671,"5min":1.744,"15min":1.744},"last":1.528}]},{"osd":4,"last update":"Mon Oct 16 11:23:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.278,"15min":1.302},"min":{"1min":0.96099999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.514,"15min":1.5029999999999999},"min":{"1min":0.94999999999999996,"5min":0.622,"15min":0.622},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":1.778}]},{"osd":5,"last update":"Mon Oct 16 11:23:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.3380000000000001,"15min":1.333},"min":{"1min":0.747,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.7410000000000001,"5min":1.8069999999999999,"15min":2.1669999999999998},"last":1.881},{"interface":"front","average":{"1min":1.605,"5min":1.528,"15min":1.5429999999999999},"min":{"1min":0.90800000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":1.7490000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:23:35 2023","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.51,"15min":1.494},"min":{"1min":0.98999999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.8039999999999998,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":1.8340000000000001},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.5489999999999999,"15min":1.552},"min":{"1min":1.3,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.5449999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":2.137}]}]},{"osd":5,"up_from":13,"seq":55834575986,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29688,"kb_used_data":2664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342152,"statfs":{"total":96636764160,"available":96606363648,"internally_reserved":0,"allocated":2727936,"data_stored":1321029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2350000000000001,"15min":1.27},"min":{"1min":0.74199999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.583,"5min":2.661,"15min":2.661},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.306,"5min":1.3029999999999999,"15min":1.349},"min":{"1min":0.95799999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7909999999999999,"5min":1.7949999999999999,"15min":1.865},"last":0.88600000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1819999999999999,"15min":1.1950000000000001},"min":{"1min":0.81999999999999995,"5min":0.80800000000000005,"15min":0.79100000000000004},"max":{"1min":1.5009999999999999,"5min":1.7869999999999999,"15min":1.992},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.258,"15min":1.2290000000000001},"min":{"1min":0.84399999999999997,"5min":0.84199999999999997,"15min":0.80800000000000005},"max":{"1min":1.635,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.0920000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.264,"15min":1.244},"min":{"1min":0.97899999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.625,"5min":1.952,"15min":1.952},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.329,"5min":1.2949999999999999,"15min":1.3220000000000001},"min":{"1min":0.92400000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":2.1640000000000001},"last":1.341}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3320000000000001,"15min":1.3340000000000001},"min":{"1min":0.98999999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":3.012,"15min":3.012},"last":1.296},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.429,"15min":1.425},"min":{"1min":1.0900000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":2.2250000000000001},"last":1.149}]},{"osd":4,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.339,"15min":1.339},"min":{"1min":0.94199999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.6990000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.387},{"interface":"front","average":{"1min":1.6799999999999999,"5min":1.5720000000000001,"15min":1.5740000000000001},"min":{"1min":1.353,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.948,"5min":2.903,"15min":2.903},"last":1.5640000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4330000000000001,"15min":1.4139999999999999},"min":{"1min":0.873,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8660000000000001,"5min":2.0649999999999999,"15min":2.0979999999999999},"last":1.266},{"interface":"front","average":{"1min":1.51,"5min":1.494,"15min":1.4770000000000001},"min":{"1min":0.95399999999999996,"5min":0.90700000000000003,"15min":0.78400000000000003},"max":{"1min":1.841,"5min":3.073,"15min":3.073},"last":1.5980000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.675,"5min":1.5660000000000001,"15min":1.5469999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0779999999999998,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.645,"15min":1.645},"min":{"1min":1.1779999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.1099999999999999,"5min":2.1459999999999999,"15min":2.2930000000000001},"last":1.77}]}]},{"osd":4,"up_from":13,"seq":55834575987,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51908,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319932,"statfs":{"total":96636764160,"available":96583610368,"internally_reserved":0,"allocated":2744320,"data_stored":1322163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1519999999999999,"15min":1.1859999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.7290000000000001,"5min":1.792,"15min":1.792},"last":2.899},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.218,"15min":1.2350000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.9299999999999999},"last":3.016}]},{"osd":1,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2769999999999999,"15min":1.298},"min":{"1min":0.81799999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.841,"5min":1.841,"15min":1.913},"last":2.6560000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.236,"15min":1.25},"min":{"1min":0.78500000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.877,"5min":1.877,"15min":1.889},"last":3.0739999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.29,"15min":1.3089999999999999},"min":{"1min":1.0369999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.835},"last":3.2170000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.3999999999999999,"15min":1.403},"min":{"1min":0.879,"5min":0.879,"15min":0.872},"max":{"1min":1.8380000000000001,"5min":1.903,"15min":1.9139999999999999},"last":3.1600000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3169999999999999,"15min":1.337},"min":{"1min":0.91100000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.6830000000000001,"5min":1.8280000000000001,"15min":1.849},"last":3.6019999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4410000000000001,"15min":1.4330000000000001},"min":{"1min":1.145,"5min":1.008,"15min":0.96999999999999997},"max":{"1min":1.7769999999999999,"5min":1.865,"15min":2.0720000000000001},"last":3.4220000000000002}]},{"osd":5,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.4810000000000001,"15min":1.4399999999999999},"min":{"1min":1.0149999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9399999999999999,"5min":2.105,"15min":2.105},"last":3.2549999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.431,"15min":1.415},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.78600000000000003},"max":{"1min":1.8240000000000001,"5min":2.04,"15min":2.04},"last":3.7989999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.5409999999999999,"15min":1.5029999999999999},"min":{"1min":0.94299999999999995,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":3.157,"5min":3.157,"15min":3.157},"last":3.3290000000000002},{"interface":"front","average":{"1min":1.5,"5min":1.6100000000000001,"15min":1.611},"min":{"1min":1.0940000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0089999999999999,"5min":2.2330000000000001,"15min":2.2770000000000001},"last":3.5019999999999998}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.5640000000000001,"15min":1.5660000000000001},"min":{"1min":1.0549999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.855,"5min":2.113,"15min":2.3420000000000001},"last":4.2050000000000001},{"interface":"front","average":{"1min":1.639,"5min":1.6619999999999999,"15min":1.6399999999999999},"min":{"1min":1.0660000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9630000000000001,"5min":2.1469999999999998,"15min":2.3839999999999999},"last":3.7000000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575986,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47748,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324092,"statfs":{"total":96636764160,"available":96587870208,"internally_reserved":0,"allocated":2736128,"data_stored":1320629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.89400000000000002,"15min":0.90700000000000003},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.478,"5min":1.478,"15min":1.901},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0369999999999999,"15min":1.0720000000000001},"min":{"1min":0.59299999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.391,"5min":1.6180000000000001,"15min":1.6679999999999999},"last":1.085}]},{"osd":1,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.95599999999999996,"15min":0.98799999999999999},"min":{"1min":0.60799999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.105,"15min":1.133},"min":{"1min":0.48399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.601,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.72699999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0760000000000001,"15min":1.117},"min":{"1min":0.65500000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.524,"5min":1.694,"15min":1.694},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.198,"15min":1.236},"min":{"1min":0.625,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.53100000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3169999999999999,"15min":1.3500000000000001},"min":{"1min":0.83599999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.806,"5min":1.806,"15min":1.873},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.357,"15min":1.367},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.8819999999999999},"last":1.032}]},{"osd":5,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.385,"15min":1.401},"min":{"1min":0.80600000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.4059999999999999,"15min":1.444},"min":{"1min":1.151,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.3}]},{"osd":6,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.504,"15min":1.5329999999999999},"min":{"1min":0.876,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":3.9129999999999998,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5,"15min":1.5369999999999999},"min":{"1min":1.153,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":3.9319999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.4119999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.528,"15min":1.5600000000000001},"min":{"1min":0.95399999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.877,"5min":2.0790000000000002,"15min":2.2679999999999998},"last":1.196},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.639,"15min":1.673},"min":{"1min":1.1279999999999999,"5min":1,"15min":1},"max":{"1min":2.052,"5min":2.198,"15min":2.5129999999999999},"last":1.4470000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575985,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51896,"kb_used_data":2660,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319944,"statfs":{"total":96636764160,"available":96583622656,"internally_reserved":0,"allocated":2723840,"data_stored":1320447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.91000000000000003,"15min":0.92200000000000004},"min":{"1min":0.74199999999999999,"5min":0.51900000000000002,"15min":0.5},"max":{"1min":1.532,"5min":1.532,"15min":1.669},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.0569999999999999,"15min":1.0820000000000001},"min":{"1min":0.72499999999999998,"5min":0.47699999999999998,"15min":0.32600000000000001},"max":{"1min":1.496,"5min":1.681,"15min":2.1429999999999998},"last":1.3460000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.089,"15min":1.097},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.39200000000000002},"max":{"1min":1.415,"5min":1.6779999999999999,"15min":1.978},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1479999999999999,"15min":1.165},"min":{"1min":0.57599999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.448,"5min":1.609,"15min":1.869},"last":1.1659999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.0649999999999999,"15min":1.087},"min":{"1min":0.54600000000000004,"5min":0.437,"15min":0.437},"max":{"1min":1.464,"5min":1.651,"15min":1.8560000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.252,"15min":1.274},"min":{"1min":0.82199999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6120000000000001,"5min":1.7450000000000001,"15min":1.857},"last":1}]},{"osd":4,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3740000000000001,"15min":1.415},"min":{"1min":0.91500000000000004,"5min":0.86599999999999999,"15min":0.73499999999999999},"max":{"1min":1.6910000000000001,"5min":1.8029999999999999,"15min":2.5059999999999998},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3440000000000001,"15min":1.3520000000000001},"min":{"1min":0.97899999999999998,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":1.825,"5min":1.825,"15min":2.0880000000000001},"last":1.125}]},{"osd":5,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.444,"15min":1.464},"min":{"1min":1.0720000000000001,"5min":0.84999999999999998,"15min":0.78500000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2269999999999999},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4550000000000001,"15min":1.4890000000000001},"min":{"1min":1.0509999999999999,"5min":0.95799999999999996,"15min":0.874},"max":{"1min":1.798,"5min":1.931,"15min":2.6259999999999999},"last":1.528}]},{"osd":6,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4119999999999999,"15min":1.411},"min":{"1min":1.0569999999999999,"5min":0.84099999999999997,"15min":0.80500000000000005},"max":{"1min":1.71,"5min":1.9019999999999999,"15min":2.2109999999999999},"last":1.401},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5,"15min":1.5369999999999999},"min":{"1min":1.1899999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.2909999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.5660000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":1.5489999999999999,"15min":1.5860000000000001},"min":{"1min":1.2290000000000001,"5min":0.995,"15min":0.92500000000000004},"max":{"1min":2.1469999999999998,"5min":2.262,"15min":2.8109999999999999},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.494,"5min":1.4870000000000001,"15min":1.5149999999999999},"min":{"1min":1.242,"5min":0.81599999999999995,"15min":0.80100000000000005},"max":{"1min":1.8180000000000001,"5min":2.2269999999999999,"15min":2.2829999999999999},"last":1.284}]}]},{"osd":1,"up_from":13,"seq":55834575985,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60220,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311620,"statfs":{"total":96636764160,"available":96575098880,"internally_reserved":0,"allocated":2732032,"data_stored":1320503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.91900000000000004,"15min":0.91500000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.575,"5min":1.575,"15min":1.7929999999999999},"last":5.117},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0289999999999999,"15min":1.0580000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.494,"5min":1.5509999999999999,"15min":1.667},"last":1.5069999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.98399999999999999,"15min":1.0189999999999999},"min":{"1min":0.64300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.72},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.252,"15min":1.276},"min":{"1min":0.69799999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":3.5840000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.1879999999999999,"15min":1.216},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.774,"5min":1.774,"15min":1.79},"last":3.4670000000000001},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.169,"15min":1.1739999999999999},"min":{"1min":0.66700000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.633,"5min":1.633,"15min":1.802},"last":1.5600000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.2609999999999999,"15min":1.274},"min":{"1min":0.86299999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.653,"5min":1.887,"15min":1.9159999999999999},"last":5.29},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.409,"15min":1.415},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.887,"5min":2.0619999999999998,"15min":2.093},"last":5.0179999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.379,"15min":1.409},"min":{"1min":0.93700000000000006,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.948,"5min":2.0009999999999999,"15min":2.0640000000000001},"last":6.0979999999999999},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4239999999999999,"15min":1.421},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":1.825,"5min":1.825,"15min":2.0649999999999999},"last":1.976}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4850000000000001,"15min":1.496},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.879,"5min":2.1970000000000001,"15min":2.2949999999999999},"last":4.0499999999999998},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4710000000000001,"15min":1.486},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.891,"5min":1.891,"15min":2.1970000000000001},"last":2.024}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5649999999999999,"15min":1.5620000000000001},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.1720000000000002},"last":2.6930000000000001},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.625,"15min":1.607},"min":{"1min":1.1080000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.9359999999999999,"5min":2.1989999999999998,"15min":2.2810000000000001},"last":4.1040000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575986,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52476,"kb_used_data":3244,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319364,"statfs":{"total":96636764160,"available":96583028736,"internally_reserved":0,"allocated":3321856,"data_stored":1910905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.871,"15min":0.90000000000000002},"min":{"1min":0.73199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.502,"5min":1.52,"15min":1.619},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.1060000000000001},"min":{"1min":0.627,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.621,"5min":1.6399999999999999,"15min":1.76},"last":1.222}]},{"osd":2,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.097,"15min":1.0860000000000001},"min":{"1min":0.65300000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.556,"5min":1.649,"15min":1.649},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.095,"15min":1.1459999999999999},"min":{"1min":0.81100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4179999999999999,"5min":1.623,"15min":1.6870000000000001},"last":1.337}]},{"osd":3,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.056,"15min":1.085},"min":{"1min":0.69499999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.556},{"interface":"front","average":{"1min":1.274,"5min":1.244,"15min":1.256},"min":{"1min":0.91500000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.716,"5min":1.7609999999999999,"15min":1.9830000000000001},"last":1.1519999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.387,"15min":1.389},"min":{"1min":1.129,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.681,"5min":1.9390000000000001,"15min":1.95},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3180000000000001,"15min":1.3500000000000001},"min":{"1min":1.0329999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.645,"5min":2.1539999999999999,"15min":2.4820000000000002},"last":1.395}]},{"osd":5,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3660000000000001,"15min":1.3879999999999999},"min":{"1min":0.996,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":1.829,"5min":2.0750000000000002,"15min":2.6669999999999998},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.454},"min":{"1min":1.028,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.748,"5min":2.2029999999999998,"15min":2.2040000000000002},"last":1.373}]},{"osd":6,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.4750000000000001,"15min":1.5029999999999999},"min":{"1min":1.1080000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.819,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.321},{"interface":"front","average":{"1min":1.405,"5min":1.4630000000000001,"15min":1.492},"min":{"1min":0.93799999999999994,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.758,"5min":2.262,"15min":2.262},"last":1.8799999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.52,"15min":1.5549999999999999},"min":{"1min":1.2749999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7150000000000001,"5min":2.347,"15min":2.347},"last":1.9370000000000001},{"interface":"front","average":{"1min":1.454,"5min":1.474,"15min":1.5109999999999999},"min":{"1min":1.129,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.9530000000000001,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.4770000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:24:30.511 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:24:30.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.763+0000 7f19baeda640 1 Processor -- start 2023-10-16T11:24:30.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.763+0000 7f19baeda640 1 -- start start 2023-10-16T11:24:30.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.764+0000 7f19baeda640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f19b4159b20 0x7f19b406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:30.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.764+0000 7f19baeda640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19b4159f00 0x7f19b40b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:30.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.764+0000 7f19baeda640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b406f7f0 con 0x7f19b406c120 2023-10-16T11:24:30.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.765+0000 7f19baeda640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19b406c5e0 con 0x7f19b4159b20 2023-10-16T11:24:30.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.765+0000 7f19baeda640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b406c8c0 con 0x7f19b4159f00 2023-10-16T11:24:30.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.765+0000 7f19b9477640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f19b406c120 0x7f19b40cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:59514/0 (socket says 172.21.15.84:59514) 2023-10-16T11:24:30.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.765+0000 7f19b9477640 1 -- 172.21.15.84:0/4128587714 learned_addr learned my addr 172.21.15.84:0/4128587714 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:30.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.765+0000 7f19b8c76640 1 --2- 172.21.15.84:0/4128587714 >> v2:172.21.15.157:3300/0 conn(0x7f19b4159b20 0x7f19b406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:30.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.765+0000 7f19b3fff640 1 --2- 172.21.15.84:0/4128587714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19b4159f00 0x7f19b40b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:30.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.765+0000 7f19b37fe640 1 -- 172.21.15.84:0/4128587714 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653220699 0 0) 0x7f19b406f7f0 con 0x7f19b406c120 2023-10-16T11:24:30.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.765+0000 7f19b37fe640 1 -- 172.21.15.84:0/4128587714 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19a4003520 con 0x7f19b406c120 2023-10-16T11:24:30.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.766+0000 7f19b37fe640 1 -- 172.21.15.84:0/4128587714 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1019372255 0 0) 0x7f19a4003520 con 0x7f19b406c120 2023-10-16T11:24:30.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.766+0000 7f19b37fe640 1 -- 172.21.15.84:0/4128587714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19b4159f00 msgr2=0x7f19b40b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:30.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.766+0000 7f19b37fe640 1 --2- 172.21.15.84:0/4128587714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19b4159f00 0x7f19b40b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.766+0000 7f19b37fe640 1 -- 172.21.15.84:0/4128587714 >> v2:172.21.15.157:3300/0 conn(0x7f19b4159b20 msgr2=0x7f19b406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:30.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.766+0000 7f19b37fe640 1 --2- 172.21.15.84:0/4128587714 >> v2:172.21.15.157:3300/0 conn(0x7f19b4159b20 0x7f19b406bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.766+0000 7f19b37fe640 1 -- 172.21.15.84:0/4128587714 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b406cba0 con 0x7f19b406c120 2023-10-16T11:24:30.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.766+0000 7f19b37fe640 1 -- 172.21.15.84:0/4128587714 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f19ac0019f0 con 0x7f19b406c120 2023-10-16T11:24:30.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.766+0000 7f19b37fe640 1 -- 172.21.15.84:0/4128587714 >> v1:172.21.15.84:6789/0 conn(0x7f19b406c120 legacy=0x7f19b40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:30.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.766+0000 7f19b37fe640 1 --2- 172.21.15.84:0/4128587714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19b4159f00 0x7f19a4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:30.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.766+0000 7f19b37fe640 1 --2- 172.21.15.84:0/4128587714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19a4003ff0 0x7f19a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:30.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.766+0000 7f19b37fe640 1 --2- 172.21.15.84:0/4128587714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a4004d00 0x7f19a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:30.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.766+0000 7f19b37fe640 1 -- 172.21.15.84:0/4128587714 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b406c5e0 con 0x7f19a4003ff0 2023-10-16T11:24:30.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.766+0000 7f19b37fe640 1 -- 172.21.15.84:0/4128587714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b406c8c0 con 0x7f19a4004d00 2023-10-16T11:24:30.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.766+0000 7f19b37fe640 1 -- 172.21.15.84:0/4128587714 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a400bae0 con 0x7f19b4159f00 2023-10-16T11:24:30.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.766+0000 7f19b3fff640 1 --2- 172.21.15.84:0/4128587714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19a4003ff0 0x7f19a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:30.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.767+0000 7f19b8c76640 1 --2- 172.21.15.84:0/4128587714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19b4159f00 0x7f19a4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:30.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.767+0000 7f19b9477640 1 --2- 172.21.15.84:0/4128587714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a4004d00 0x7f19a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:30.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.767+0000 7f19b3fff640 1 -- 172.21.15.84:0/4128587714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19b4159f00 msgr2=0x7f19a4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:30.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.767+0000 7f19b3fff640 1 --2- 172.21.15.84:0/4128587714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19b4159f00 0x7f19a4003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.767+0000 7f19b3fff640 1 -- 172.21.15.84:0/4128587714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a4004d00 msgr2=0x7f19a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:30.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.767+0000 7f19b3fff640 1 --2- 172.21.15.84:0/4128587714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a4004d00 0x7f19a4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.767+0000 7f19b3fff640 1 -- 172.21.15.84:0/4128587714 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a4003520 con 0x7f19a4003ff0 2023-10-16T11:24:30.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.767+0000 7f19b9477640 1 --2- 172.21.15.84:0/4128587714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a4004d00 0x7f19a4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:30.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.768+0000 7f19b3fff640 1 --2- 172.21.15.84:0/4128587714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19a4003ff0 0x7f19a40047c0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f19a000ba00 tx=0x7f19a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270251cf76897fcb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:30.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.768+0000 7f19b37fe640 1 -- 172.21.15.84:0/4128587714 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0032620 con 0x7f19a4003ff0 2023-10-16T11:24:30.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.768+0000 7f19b37fe640 1 -- 172.21.15.84:0/4128587714 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a0032780 con 0x7f19a4003ff0 2023-10-16T11:24:30.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.768+0000 7f19b37fe640 1 -- 172.21.15.84:0/4128587714 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0032b10 con 0x7f19a4003ff0 2023-10-16T11:24:30.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.769+0000 7f19baeda640 1 -- 172.21.15.84:0/4128587714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19a4003ff0 msgr2=0x7f19a40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:30.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.769+0000 7f19baeda640 1 --2- 172.21.15.84:0/4128587714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19a4003ff0 0x7f19a40047c0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f19a000ba00 tx=0x7f19a40057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.769+0000 7f19b9477640 1 -- 172.21.15.84:0/4128587714 reap_dead start 2023-10-16T11:24:30.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.769+0000 7f19baeda640 1 -- 172.21.15.84:0/4128587714 shutdown_connections 2023-10-16T11:24:30.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.769+0000 7f19baeda640 1 -- 172.21.15.84:0/4128587714 >> 172.21.15.84:0/4128587714 conn(0x7f19b40cb410 msgr2=0x7f19b40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:30.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.769+0000 7f19baeda640 1 -- 172.21.15.84:0/4128587714 shutdown_connections 2023-10-16T11:24:30.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.769+0000 7f19baeda640 1 -- 172.21.15.84:0/4128587714 wait complete. 2023-10-16T11:24:30.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.771+0000 7f19baeda640 1 Processor -- start 2023-10-16T11:24:30.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.771+0000 7f19baeda640 1 -- start start 2023-10-16T11:24:30.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.772+0000 7f19baeda640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19a4003ff0 0x7f19b4153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:30.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.772+0000 7f19baeda640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19b4150c70 0x7f19b40c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:30.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.772+0000 7f19baeda640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19b40ac890 0x7f19b40acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:30.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.772+0000 7f19baeda640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b40ad250 con 0x7f19a4003ff0 2023-10-16T11:24:30.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.772+0000 7f19baeda640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b4152e80 con 0x7f19b40ac890 2023-10-16T11:24:30.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.772+0000 7f19baeda640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b4153d50 con 0x7f19b4150c70 2023-10-16T11:24:30.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.772+0000 7f19b8c76640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19a4003ff0 0x7f19b4153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:30.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.772+0000 7f19b8c76640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19a4003ff0 0x7f19b4153310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56292/0 (socket says 172.21.15.84:56292) 2023-10-16T11:24:30.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.772+0000 7f19b8c76640 1 -- 172.21.15.84:0/1739664567 learned_addr learned my addr 172.21.15.84:0/1739664567 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:30.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.772+0000 7f19b8c76640 1 -- 172.21.15.84:0/1739664567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19b4150c70 msgr2=0x7f19b40c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:30.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.772+0000 7f19b3fff640 1 --2- 172.21.15.84:0/1739664567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19b4150c70 0x7f19b40c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:30.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.772+0000 7f19b8c76640 1 --2- 172.21.15.84:0/1739664567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19b4150c70 0x7f19b40c4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.772+0000 7f19b8c76640 1 -- 172.21.15.84:0/1739664567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19b40ac890 msgr2=0x7f19b40acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.772+0000 7f19b8c76640 1 --2- 172.21.15.84:0/1739664567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19b40ac890 0x7f19b40acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.772+0000 7f19b8c76640 1 -- 172.21.15.84:0/1739664567 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b4153850 con 0x7f19a4003ff0 2023-10-16T11:24:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.773+0000 7f19b8c76640 1 --2- 172.21.15.84:0/1739664567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19a4003ff0 0x7f19b4153310 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f19a800e7e0 tx=0x7f19a800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.773+0000 7f19b17fa640 1 -- 172.21.15.84:0/1739664567 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a800c6d0 con 0x7f19a4003ff0 2023-10-16T11:24:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.773+0000 7f19b17fa640 1 -- 172.21.15.84:0/1739664567 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a800c830 con 0x7f19a4003ff0 2023-10-16T11:24:30.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.773+0000 7f19b17fa640 1 -- 172.21.15.84:0/1739664567 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a800cba0 con 0x7f19a4003ff0 2023-10-16T11:24:30.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.773+0000 7f19baeda640 1 -- 172.21.15.84:0/1739664567 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b4152270 con 0x7f19a4003ff0 2023-10-16T11:24:30.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.773+0000 7f19baeda640 1 -- 172.21.15.84:0/1739664567 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b4152830 con 0x7f19a4003ff0 2023-10-16T11:24:30.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.774+0000 7f19baeda640 1 -- 172.21.15.84:0/1739664567 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b404e160 con 0x7f19a4003ff0 2023-10-16T11:24:30.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.775+0000 7f19b17fa640 1 -- 172.21.15.84:0/1739664567 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f19a800cd00 con 0x7f19a4003ff0 2023-10-16T11:24:30.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.776+0000 7f19b17fa640 1 --2- 172.21.15.84:0/1739664567 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f199806af90 0x7f199806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:30.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.776+0000 7f19b17fa640 1 -- 172.21.15.84:0/1739664567 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f19a80896f0 con 0x7f19a4003ff0 2023-10-16T11:24:30.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.776+0000 7f19b3fff640 1 --2- 172.21.15.84:0/1739664567 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f199806af90 0x7f199806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:30.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.777+0000 7f19b3fff640 1 --2- 172.21.15.84:0/1739664567 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f199806af90 0x7f199806d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f19b415a2e0 tx=0x7f19a0030000 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:30.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.782+0000 7f19b17fa640 1 -- 172.21.15.84:0/1739664567 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f19a8054560 con 0x7f19a4003ff0 2023-10-16T11:24:30.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.970+0000 7f19baeda640 1 -- 172.21.15.84:0/1739664567 --> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f19b406ce80 con 0x7f199806af90 2023-10-16T11:24:30.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.992+0000 7f19b17fa640 1 -- 172.21.15.84:0/1739664567 <== mgr.11076 v2:172.21.15.84:6832/1425409017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286793 (secure 0 0 0) 0x7f19a00328f0 con 0x7f199806af90 2023-10-16T11:24:30.997 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:31.000 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:24:31.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.998+0000 7f19baeda640 1 -- 172.21.15.84:0/1739664567 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f199806af90 msgr2=0x7f199806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:31.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.998+0000 7f19baeda640 1 --2- 172.21.15.84:0/1739664567 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f199806af90 0x7f199806d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f19b415a2e0 tx=0x7f19a0030000 comp rx=0 tx=0).stop 2023-10-16T11:24:31.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.998+0000 7f19baeda640 1 -- 172.21.15.84:0/1739664567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19a4003ff0 msgr2=0x7f19b4153310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:31.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.998+0000 7f19baeda640 1 --2- 172.21.15.84:0/1739664567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19a4003ff0 0x7f19b4153310 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f19a800e7e0 tx=0x7f19a800eca0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.998+0000 7f19baeda640 1 -- 172.21.15.84:0/1739664567 shutdown_connections 2023-10-16T11:24:31.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.998+0000 7f19baeda640 1 --2- 172.21.15.84:0/1739664567 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f199806af90 0x7f199806d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.998+0000 7f19baeda640 1 --2- 172.21.15.84:0/1739664567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19b40ac890 0x7f19b40acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.998+0000 7f19baeda640 1 --2- 172.21.15.84:0/1739664567 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19b4150c70 0x7f19b40c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.998+0000 7f19baeda640 1 --2- 172.21.15.84:0/1739664567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19a4003ff0 0x7f19b4153310 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.998+0000 7f19baeda640 1 -- 172.21.15.84:0/1739664567 >> 172.21.15.84:0/1739664567 conn(0x7f19b40cb410 msgr2=0x7f19b40ce8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:31.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.999+0000 7f19baeda640 1 -- 172.21.15.84:0/1739664567 shutdown_connections 2023-10-16T11:24:31.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:30.999+0000 7f19baeda640 1 -- 172.21.15.84:0/1739664567 wait complete. 2023-10-16T11:24:31.013 INFO:tasks.ceph.mds.a.smithi084.stderr:2023-10-16T11:24:31.012+0000 7f63aab48640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:24:31.020 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-16T11:24:30.252128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":625,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2805,"ondisk_log_size":2805,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":380264,"kb_used_data":22504,"kb_used_omap":12,"kb_used_meta":357619,"kb_avail":754594456,"statfs":{"total":773094113280,"available":772704722944,"internally_reserved":0,"allocated":23044096,"data_stored":11749132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":366202425},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001760"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070051+0000","last_change":"2023-10-16T11:24:26.086488+0000","last_active":"2023-10-16T11:24:28.070051+0000","last_peered":"2023-10-16T11:24:28.070051+0000","last_clean":"2023-10-16T11:24:28.070051+0000","last_became_active":"2023-10-16T11:24:26.085969+0000","last_became_peered":"2023-10-16T11:24:26.085969+0000","last_unstale":"2023-10-16T11:24:28.070051+0000","last_undegraded":"2023-10-16T11:24:28.070051+0000","last_fullsized":"2023-10-16T11:24:28.070051+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.200941+0000","last_change":"2023-10-16T11:24:26.089885+0000","last_active":"2023-10-16T11:24:27.200941+0000","last_peered":"2023-10-16T11:24:27.200941+0000","last_clean":"2023-10-16T11:24:27.200941+0000","last_became_active":"2023-10-16T11:24:26.089406+0000","last_became_peered":"2023-10-16T11:24:26.089406+0000","last_unstale":"2023-10-16T11:24:27.200941+0000","last_undegraded":"2023-10-16T11:24:27.200941+0000","last_fullsized":"2023-10-16T11:24:27.200941+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.080857+0000","last_change":"2023-10-16T11:24:26.085606+0000","last_active":"2023-10-16T11:24:27.080857+0000","last_peered":"2023-10-16T11:24:27.080857+0000","last_clean":"2023-10-16T11:24:27.080857+0000","last_became_active":"2023-10-16T11:24:26.085318+0000","last_became_peered":"2023-10-16T11:24:26.085318+0000","last_unstale":"2023-10-16T11:24:27.080857+0000","last_undegraded":"2023-10-16T11:24:27.080857+0000","last_fullsized":"2023-10-16T11:24:27.080857+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069514+0000","last_change":"2023-10-16T11:24:26.091440+0000","last_active":"2023-10-16T11:24:28.069514+0000","last_peered":"2023-10-16T11:24:28.069514+0000","last_clean":"2023-10-16T11:24:28.069514+0000","last_became_active":"2023-10-16T11:24:26.088757+0000","last_became_peered":"2023-10-16T11:24:26.088757+0000","last_unstale":"2023-10-16T11:24:28.069514+0000","last_undegraded":"2023-10-16T11:24:28.069514+0000","last_fullsized":"2023-10-16T11:24:28.069514+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:27.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070231+0000","last_change":"2023-10-16T11:24:26.096814+0000","last_active":"2023-10-16T11:24:28.070231+0000","last_peered":"2023-10-16T11:24:28.070231+0000","last_clean":"2023-10-16T11:24:28.070231+0000","last_became_active":"2023-10-16T11:24:26.095777+0000","last_became_peered":"2023-10-16T11:24:26.095777+0000","last_unstale":"2023-10-16T11:24:28.070231+0000","last_undegraded":"2023-10-16T11:24:28.070231+0000","last_fullsized":"2023-10-16T11:24:28.070231+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:08.889514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070908+0000","last_change":"2023-10-16T11:24:26.083687+0000","last_active":"2023-10-16T11:24:28.070908+0000","last_peered":"2023-10-16T11:24:28.070908+0000","last_clean":"2023-10-16T11:24:28.070908+0000","last_became_active":"2023-10-16T11:24:26.083213+0000","last_became_peered":"2023-10-16T11:24:26.083213+0000","last_unstale":"2023-10-16T11:24:28.070908+0000","last_undegraded":"2023-10-16T11:24:28.070908+0000","last_fullsized":"2023-10-16T11:24:28.070908+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:08:02.966132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.068359+0000","last_change":"2023-10-16T11:24:26.096348+0000","last_active":"2023-10-16T11:24:28.068359+0000","last_peered":"2023-10-16T11:24:28.068359+0000","last_clean":"2023-10-16T11:24:28.068359+0000","last_became_active":"2023-10-16T11:24:26.093712+0000","last_became_peered":"2023-10-16T11:24:26.093712+0000","last_unstale":"2023-10-16T11:24:28.068359+0000","last_undegraded":"2023-10-16T11:24:28.068359+0000","last_fullsized":"2023-10-16T11:24:28.068359+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:40.970357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.068908+0000","last_change":"2023-10-16T11:24:26.098845+0000","last_active":"2023-10-16T11:24:28.068908+0000","last_peered":"2023-10-16T11:24:28.068908+0000","last_clean":"2023-10-16T11:24:28.068908+0000","last_became_active":"2023-10-16T11:24:26.098568+0000","last_became_peered":"2023-10-16T11:24:26.098568+0000","last_unstale":"2023-10-16T11:24:28.068908+0000","last_undegraded":"2023-10-16T11:24:28.068908+0000","last_fullsized":"2023-10-16T11:24:28.068908+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:15.909098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.201569+0000","last_change":"2023-10-16T11:24:26.080292+0000","last_active":"2023-10-16T11:24:27.201569+0000","last_peered":"2023-10-16T11:24:27.201569+0000","last_clean":"2023-10-16T11:24:27.201569+0000","last_became_active":"2023-10-16T11:24:26.080004+0000","last_became_peered":"2023-10-16T11:24:26.080004+0000","last_unstale":"2023-10-16T11:24:27.201569+0000","last_undegraded":"2023-10-16T11:24:27.201569+0000","last_fullsized":"2023-10-16T11:24:27.201569+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.082247+0000","last_change":"2023-10-16T11:24:26.088202+0000","last_active":"2023-10-16T11:24:27.082247+0000","last_peered":"2023-10-16T11:24:27.082247+0000","last_clean":"2023-10-16T11:24:27.082247+0000","last_became_active":"2023-10-16T11:24:26.087929+0000","last_became_peered":"2023-10-16T11:24:26.087929+0000","last_unstale":"2023-10-16T11:24:27.082247+0000","last_undegraded":"2023-10-16T11:24:27.082247+0000","last_fullsized":"2023-10-16T11:24:27.082247+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:22.009042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.080938+0000","last_change":"2023-10-16T11:24:26.077430+0000","last_active":"2023-10-16T11:24:27.080938+0000","last_peered":"2023-10-16T11:24:27.080938+0000","last_clean":"2023-10-16T11:24:27.080938+0000","last_became_active":"2023-10-16T11:24:26.077145+0000","last_became_peered":"2023-10-16T11:24:26.077145+0000","last_unstale":"2023-10-16T11:24:27.080938+0000","last_undegraded":"2023-10-16T11:24:27.080938+0000","last_fullsized":"2023-10-16T11:24:27.080938+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.976285+0000","last_change":"2023-10-16T11:24:26.089768+0000","last_active":"2023-10-16T11:24:27.976285+0000","last_peered":"2023-10-16T11:24:27.976285+0000","last_clean":"2023-10-16T11:24:27.976285+0000","last_became_active":"2023-10-16T11:24:26.087028+0000","last_became_peered":"2023-10-16T11:24:26.087028+0000","last_unstale":"2023-10-16T11:24:27.976285+0000","last_undegraded":"2023-10-16T11:24:27.976285+0000","last_fullsized":"2023-10-16T11:24:27.976285+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:33.050118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.976905+0000","last_change":"2023-10-16T11:24:26.088938+0000","last_active":"2023-10-16T11:24:27.976905+0000","last_peered":"2023-10-16T11:24:27.976905+0000","last_clean":"2023-10-16T11:24:27.976905+0000","last_became_active":"2023-10-16T11:24:26.088689+0000","last_became_peered":"2023-10-16T11:24:26.088689+0000","last_unstale":"2023-10-16T11:24:27.976905+0000","last_undegraded":"2023-10-16T11:24:27.976905+0000","last_fullsized":"2023-10-16T11:24:27.976905+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:18:12.479766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069095+0000","last_change":"2023-10-16T11:24:26.088199+0000","last_active":"2023-10-16T11:24:28.069095+0000","last_peered":"2023-10-16T11:24:28.069095+0000","last_clean":"2023-10-16T11:24:28.069095+0000","last_became_active":"2023-10-16T11:24:26.087892+0000","last_became_peered":"2023-10-16T11:24:26.087892+0000","last_unstale":"2023-10-16T11:24:28.069095+0000","last_undegraded":"2023-10-16T11:24:28.069095+0000","last_fullsized":"2023-10-16T11:24:28.069095+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.068584+0000","last_change":"2023-10-16T11:24:26.091403+0000","last_active":"2023-10-16T11:24:28.068584+0000","last_peered":"2023-10-16T11:24:28.068584+0000","last_clean":"2023-10-16T11:24:28.068584+0000","last_became_active":"2023-10-16T11:24:26.091114+0000","last_became_peered":"2023-10-16T11:24:26.091114+0000","last_unstale":"2023-10-16T11:24:28.068584+0000","last_undegraded":"2023-10-16T11:24:28.068584+0000","last_fullsized":"2023-10-16T11:24:28.068584+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.081909+0000","last_change":"2023-10-16T11:24:26.081938+0000","last_active":"2023-10-16T11:24:27.081909+0000","last_peered":"2023-10-16T11:24:27.081909+0000","last_clean":"2023-10-16T11:24:27.081909+0000","last_became_active":"2023-10-16T11:24:26.081671+0000","last_became_peered":"2023-10-16T11:24:26.081671+0000","last_unstale":"2023-10-16T11:24:27.081909+0000","last_undegraded":"2023-10-16T11:24:27.081909+0000","last_fullsized":"2023-10-16T11:24:27.081909+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:17.072626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070195+0000","last_change":"2023-10-16T11:24:26.086029+0000","last_active":"2023-10-16T11:24:28.070195+0000","last_peered":"2023-10-16T11:24:28.070195+0000","last_clean":"2023-10-16T11:24:28.070195+0000","last_became_active":"2023-10-16T11:24:26.085644+0000","last_became_peered":"2023-10-16T11:24:26.085644+0000","last_unstale":"2023-10-16T11:24:28.070195+0000","last_undegraded":"2023-10-16T11:24:28.070195+0000","last_fullsized":"2023-10-16T11:24:28.070195+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:39.802117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.082380+0000","last_change":"2023-10-16T11:24:26.092139+0000","last_active":"2023-10-16T11:24:27.082380+0000","last_peered":"2023-10-16T11:24:27.082380+0000","last_clean":"2023-10-16T11:24:27.082380+0000","last_became_active":"2023-10-16T11:24:26.091860+0000","last_became_peered":"2023-10-16T11:24:26.091860+0000","last_unstale":"2023-10-16T11:24:27.082380+0000","last_undegraded":"2023-10-16T11:24:27.082380+0000","last_fullsized":"2023-10-16T11:24:27.082380+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:00:31.371733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.082805+0000","last_change":"2023-10-16T11:24:26.086807+0000","last_active":"2023-10-16T11:24:27.082805+0000","last_peered":"2023-10-16T11:24:27.082805+0000","last_clean":"2023-10-16T11:24:27.082805+0000","last_became_active":"2023-10-16T11:24:26.085382+0000","last_became_peered":"2023-10-16T11:24:26.085382+0000","last_unstale":"2023-10-16T11:24:27.082805+0000","last_undegraded":"2023-10-16T11:24:27.082805+0000","last_fullsized":"2023-10-16T11:24:27.082805+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:56.195463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070150+0000","last_change":"2023-10-16T11:24:26.093168+0000","last_active":"2023-10-16T11:24:28.070150+0000","last_peered":"2023-10-16T11:24:28.070150+0000","last_clean":"2023-10-16T11:24:28.070150+0000","last_became_active":"2023-10-16T11:24:26.091801+0000","last_became_peered":"2023-10-16T11:24:26.091801+0000","last_unstale":"2023-10-16T11:24:28.070150+0000","last_undegraded":"2023-10-16T11:24:28.070150+0000","last_fullsized":"2023-10-16T11:24:28.070150+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:43:28.275746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069327+0000","last_change":"2023-10-16T11:24:26.092718+0000","last_active":"2023-10-16T11:24:28.069327+0000","last_peered":"2023-10-16T11:24:28.069327+0000","last_clean":"2023-10-16T11:24:28.069327+0000","last_became_active":"2023-10-16T11:24:26.091377+0000","last_became_peered":"2023-10-16T11:24:26.091377+0000","last_unstale":"2023-10-16T11:24:28.069327+0000","last_undegraded":"2023-10-16T11:24:28.069327+0000","last_fullsized":"2023-10-16T11:24:28.069327+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:17.072626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.083541+0000","last_change":"2023-10-16T11:24:26.101039+0000","last_active":"2023-10-16T11:24:27.083541+0000","last_peered":"2023-10-16T11:24:27.083541+0000","last_clean":"2023-10-16T11:24:27.083541+0000","last_became_active":"2023-10-16T11:24:26.100757+0000","last_became_peered":"2023-10-16T11:24:26.100757+0000","last_unstale":"2023-10-16T11:24:27.083541+0000","last_undegraded":"2023-10-16T11:24:27.083541+0000","last_fullsized":"2023-10-16T11:24:27.083541+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.083471+0000","last_change":"2023-10-16T11:24:26.097221+0000","last_active":"2023-10-16T11:24:27.083471+0000","last_peered":"2023-10-16T11:24:27.083471+0000","last_clean":"2023-10-16T11:24:27.083471+0000","last_became_active":"2023-10-16T11:24:26.096572+0000","last_became_peered":"2023-10-16T11:24:26.096572+0000","last_unstale":"2023-10-16T11:24:27.083471+0000","last_undegraded":"2023-10-16T11:24:27.083471+0000","last_fullsized":"2023-10-16T11:24:27.083471+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069065+0000","last_change":"2023-10-16T11:24:26.091894+0000","last_active":"2023-10-16T11:24:28.069065+0000","last_peered":"2023-10-16T11:24:28.069065+0000","last_clean":"2023-10-16T11:24:28.069065+0000","last_became_active":"2023-10-16T11:24:26.091627+0000","last_became_peered":"2023-10-16T11:24:26.091627+0000","last_unstale":"2023-10-16T11:24:28.069065+0000","last_undegraded":"2023-10-16T11:24:28.069065+0000","last_fullsized":"2023-10-16T11:24:28.069065+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:46:59.832609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.200984+0000","last_change":"2023-10-16T11:24:26.094066+0000","last_active":"2023-10-16T11:24:27.200984+0000","last_peered":"2023-10-16T11:24:27.200984+0000","last_clean":"2023-10-16T11:24:27.200984+0000","last_became_active":"2023-10-16T11:24:26.093428+0000","last_became_peered":"2023-10-16T11:24:26.093428+0000","last_unstale":"2023-10-16T11:24:27.200984+0000","last_undegraded":"2023-10-16T11:24:27.200984+0000","last_fullsized":"2023-10-16T11:24:27.200984+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069693+0000","last_change":"2023-10-16T11:24:26.095913+0000","last_active":"2023-10-16T11:24:28.069693+0000","last_peered":"2023-10-16T11:24:28.069693+0000","last_clean":"2023-10-16T11:24:28.069693+0000","last_became_active":"2023-10-16T11:24:26.093113+0000","last_became_peered":"2023-10-16T11:24:26.093113+0000","last_unstale":"2023-10-16T11:24:28.069693+0000","last_undegraded":"2023-10-16T11:24:28.069693+0000","last_fullsized":"2023-10-16T11:24:28.069693+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.976416+0000","last_change":"2023-10-16T11:24:26.088015+0000","last_active":"2023-10-16T11:24:27.976416+0000","last_peered":"2023-10-16T11:24:27.976416+0000","last_clean":"2023-10-16T11:24:27.976416+0000","last_became_active":"2023-10-16T11:24:26.086932+0000","last_became_peered":"2023-10-16T11:24:26.086932+0000","last_unstale":"2023-10-16T11:24:27.976416+0000","last_undegraded":"2023-10-16T11:24:27.976416+0000","last_fullsized":"2023-10-16T11:24:27.976416+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:44.137821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.081098+0000","last_change":"2023-10-16T11:24:26.086228+0000","last_active":"2023-10-16T11:24:27.081098+0000","last_peered":"2023-10-16T11:24:27.081098+0000","last_clean":"2023-10-16T11:24:27.081098+0000","last_became_active":"2023-10-16T11:24:26.085912+0000","last_became_peered":"2023-10-16T11:24:26.085912+0000","last_unstale":"2023-10-16T11:24:27.081098+0000","last_undegraded":"2023-10-16T11:24:27.081098+0000","last_fullsized":"2023-10-16T11:24:27.081098+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.976152+0000","last_change":"2023-10-16T11:24:26.086172+0000","last_active":"2023-10-16T11:24:27.976152+0000","last_peered":"2023-10-16T11:24:27.976152+0000","last_clean":"2023-10-16T11:24:27.976152+0000","last_became_active":"2023-10-16T11:24:26.085882+0000","last_became_peered":"2023-10-16T11:24:26.085882+0000","last_unstale":"2023-10-16T11:24:27.976152+0000","last_undegraded":"2023-10-16T11:24:27.976152+0000","last_fullsized":"2023-10-16T11:24:27.976152+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:44.265192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.200688+0000","last_change":"2023-10-16T11:24:26.086008+0000","last_active":"2023-10-16T11:24:27.200688+0000","last_peered":"2023-10-16T11:24:27.200688+0000","last_clean":"2023-10-16T11:24:27.200688+0000","last_became_active":"2023-10-16T11:24:26.085598+0000","last_became_peered":"2023-10-16T11:24:26.085598+0000","last_unstale":"2023-10-16T11:24:27.200688+0000","last_undegraded":"2023-10-16T11:24:27.200688+0000","last_fullsized":"2023-10-16T11:24:27.200688+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.082611+0000","last_change":"2023-10-16T11:24:26.084964+0000","last_active":"2023-10-16T11:24:27.082611+0000","last_peered":"2023-10-16T11:24:27.082611+0000","last_clean":"2023-10-16T11:24:27.082611+0000","last_became_active":"2023-10-16T11:24:26.084006+0000","last_became_peered":"2023-10-16T11:24:26.084006+0000","last_unstale":"2023-10-16T11:24:27.082611+0000","last_undegraded":"2023-10-16T11:24:27.082611+0000","last_fullsized":"2023-10-16T11:24:27.082611+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:03.949526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.068378+0000","last_change":"2023-10-16T11:24:26.085375+0000","last_active":"2023-10-16T11:24:28.068378+0000","last_peered":"2023-10-16T11:24:28.068378+0000","last_clean":"2023-10-16T11:24:28.068378+0000","last_became_active":"2023-10-16T11:24:26.085120+0000","last_became_peered":"2023-10-16T11:24:26.085120+0000","last_unstale":"2023-10-16T11:24:28.068378+0000","last_undegraded":"2023-10-16T11:24:28.068378+0000","last_fullsized":"2023-10-16T11:24:28.068378+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.200979+0000","last_change":"2023-10-16T11:24:26.069227+0000","last_active":"2023-10-16T11:24:27.200979+0000","last_peered":"2023-10-16T11:24:27.200979+0000","last_clean":"2023-10-16T11:24:27.200979+0000","last_became_active":"2023-10-16T11:24:26.068456+0000","last_became_peered":"2023-10-16T11:24:26.068456+0000","last_unstale":"2023-10-16T11:24:27.200979+0000","last_undegraded":"2023-10-16T11:24:27.200979+0000","last_fullsized":"2023-10-16T11:24:27.200979+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"216'4","reported_seq":16,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.105044+0000","last_change":"2023-10-16T11:24:25.096644+0000","last_active":"2023-10-16T11:24:26.105044+0000","last_peered":"2023-10-16T11:24:26.105044+0000","last_clean":"2023-10-16T11:24:26.105044+0000","last_became_active":"2023-10-16T11:24:25.096306+0000","last_became_peered":"2023-10-16T11:24:25.096306+0000","last_unstale":"2023-10-16T11:24:26.105044+0000","last_undegraded":"2023-10-16T11:24:26.105044+0000","last_fullsized":"2023-10-16T11:24:26.105044+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:22.618393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.084146+0000","last_change":"2023-10-16T11:24:26.094536+0000","last_active":"2023-10-16T11:24:27.084146+0000","last_peered":"2023-10-16T11:24:27.084146+0000","last_clean":"2023-10-16T11:24:27.084146+0000","last_became_active":"2023-10-16T11:24:26.094272+0000","last_became_peered":"2023-10-16T11:24:26.094272+0000","last_unstale":"2023-10-16T11:24:27.084146+0000","last_undegraded":"2023-10-16T11:24:27.084146+0000","last_fullsized":"2023-10-16T11:24:27.084146+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"216'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.105407+0000","last_change":"2023-10-16T11:24:25.094483+0000","last_active":"2023-10-16T11:24:26.105407+0000","last_peered":"2023-10-16T11:24:26.105407+0000","last_clean":"2023-10-16T11:24:26.105407+0000","last_became_active":"2023-10-16T11:24:25.094134+0000","last_became_peered":"2023-10-16T11:24:25.094134+0000","last_unstale":"2023-10-16T11:24:26.105407+0000","last_undegraded":"2023-10-16T11:24:26.105407+0000","last_fullsized":"2023-10-16T11:24:26.105407+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069698+0000","last_change":"2023-10-16T11:24:26.089791+0000","last_active":"2023-10-16T11:24:28.069698+0000","last_peered":"2023-10-16T11:24:28.069698+0000","last_clean":"2023-10-16T11:24:28.069698+0000","last_became_active":"2023-10-16T11:24:26.089516+0000","last_became_peered":"2023-10-16T11:24:26.089516+0000","last_unstale":"2023-10-16T11:24:28.069698+0000","last_undegraded":"2023-10-16T11:24:28.069698+0000","last_fullsized":"2023-10-16T11:24:28.069698+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"216'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.105661+0000","last_change":"2023-10-16T11:24:25.081731+0000","last_active":"2023-10-16T11:24:26.105661+0000","last_peered":"2023-10-16T11:24:26.105661+0000","last_clean":"2023-10-16T11:24:26.105661+0000","last_became_active":"2023-10-16T11:24:25.081336+0000","last_became_peered":"2023-10-16T11:24:25.081336+0000","last_unstale":"2023-10-16T11:24:26.105661+0000","last_undegraded":"2023-10-16T11:24:26.105661+0000","last_fullsized":"2023-10-16T11:24:26.105661+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:09.330801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.082147+0000","last_change":"2023-10-16T11:24:26.096297+0000","last_active":"2023-10-16T11:24:27.082147+0000","last_peered":"2023-10-16T11:24:27.082147+0000","last_clean":"2023-10-16T11:24:27.082147+0000","last_became_active":"2023-10-16T11:24:26.095538+0000","last_became_peered":"2023-10-16T11:24:26.095538+0000","last_unstale":"2023-10-16T11:24:27.082147+0000","last_undegraded":"2023-10-16T11:24:27.082147+0000","last_fullsized":"2023-10-16T11:24:27.082147+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:47.400027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"216'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.103907+0000","last_change":"2023-10-16T11:24:25.095179+0000","last_active":"2023-10-16T11:24:26.103907+0000","last_peered":"2023-10-16T11:24:26.103907+0000","last_clean":"2023-10-16T11:24:26.103907+0000","last_became_active":"2023-10-16T11:24:25.094714+0000","last_became_peered":"2023-10-16T11:24:25.094714+0000","last_unstale":"2023-10-16T11:24:26.103907+0000","last_undegraded":"2023-10-16T11:24:26.103907+0000","last_fullsized":"2023-10-16T11:24:26.103907+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:26.128614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069793+0000","last_change":"2023-10-16T11:24:26.089982+0000","last_active":"2023-10-16T11:24:28.069793+0000","last_peered":"2023-10-16T11:24:28.069793+0000","last_clean":"2023-10-16T11:24:28.069793+0000","last_became_active":"2023-10-16T11:24:26.089677+0000","last_became_peered":"2023-10-16T11:24:26.089677+0000","last_unstale":"2023-10-16T11:24:28.069793+0000","last_undegraded":"2023-10-16T11:24:28.069793+0000","last_fullsized":"2023-10-16T11:24:28.069793+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:22.009042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"216'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.104624+0000","last_change":"2023-10-16T11:24:25.084983+0000","last_active":"2023-10-16T11:24:26.104624+0000","last_peered":"2023-10-16T11:24:26.104624+0000","last_clean":"2023-10-16T11:24:26.104624+0000","last_became_active":"2023-10-16T11:24:25.084622+0000","last_became_peered":"2023-10-16T11:24:25.084622+0000","last_unstale":"2023-10-16T11:24:26.104624+0000","last_undegraded":"2023-10-16T11:24:26.104624+0000","last_fullsized":"2023-10-16T11:24:26.104624+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:36.588809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069256+0000","last_change":"2023-10-16T11:24:26.088785+0000","last_active":"2023-10-16T11:24:28.069256+0000","last_peered":"2023-10-16T11:24:28.069256+0000","last_clean":"2023-10-16T11:24:28.069256+0000","last_became_active":"2023-10-16T11:24:26.088521+0000","last_became_peered":"2023-10-16T11:24:26.088521+0000","last_unstale":"2023-10-16T11:24:28.069256+0000","last_undegraded":"2023-10-16T11:24:28.069256+0000","last_fullsized":"2023-10-16T11:24:28.069256+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:29.145795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.074788+0000","last_change":"2023-10-16T11:24:25.074647+0000","last_active":"2023-10-16T11:24:26.074788+0000","last_peered":"2023-10-16T11:24:26.074788+0000","last_clean":"2023-10-16T11:24:26.074788+0000","last_became_active":"2023-10-16T11:24:25.074373+0000","last_became_peered":"2023-10-16T11:24:25.074373+0000","last_unstale":"2023-10-16T11:24:26.074788+0000","last_undegraded":"2023-10-16T11:24:26.074788+0000","last_fullsized":"2023-10-16T11:24:26.074788+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:38.544520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069199+0000","last_change":"2023-10-16T11:24:26.091641+0000","last_active":"2023-10-16T11:24:28.069199+0000","last_peered":"2023-10-16T11:24:28.069199+0000","last_clean":"2023-10-16T11:24:28.069199+0000","last_became_active":"2023-10-16T11:24:26.089186+0000","last_became_peered":"2023-10-16T11:24:26.089186+0000","last_unstale":"2023-10-16T11:24:28.069199+0000","last_undegraded":"2023-10-16T11:24:28.069199+0000","last_fullsized":"2023-10-16T11:24:28.069199+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:32.247878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.078739+0000","last_change":"2023-10-16T11:24:25.084521+0000","last_active":"2023-10-16T11:24:26.078739+0000","last_peered":"2023-10-16T11:24:26.078739+0000","last_clean":"2023-10-16T11:24:26.078739+0000","last_became_active":"2023-10-16T11:24:25.084003+0000","last_became_peered":"2023-10-16T11:24:25.084003+0000","last_unstale":"2023-10-16T11:24:26.078739+0000","last_undegraded":"2023-10-16T11:24:26.078739+0000","last_fullsized":"2023-10-16T11:24:26.078739+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:36.060066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070258+0000","last_change":"2023-10-16T11:24:26.082915+0000","last_active":"2023-10-16T11:24:28.070258+0000","last_peered":"2023-10-16T11:24:28.070258+0000","last_clean":"2023-10-16T11:24:28.070258+0000","last_became_active":"2023-10-16T11:24:26.082458+0000","last_became_peered":"2023-10-16T11:24:26.082458+0000","last_unstale":"2023-10-16T11:24:28.070258+0000","last_undegraded":"2023-10-16T11:24:28.070258+0000","last_fullsized":"2023-10-16T11:24:28.070258+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:10.387018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.076876+0000","last_change":"2023-10-16T11:24:25.094384+0000","last_active":"2023-10-16T11:24:26.076876+0000","last_peered":"2023-10-16T11:24:26.076876+0000","last_clean":"2023-10-16T11:24:26.076876+0000","last_became_active":"2023-10-16T11:24:25.094026+0000","last_became_peered":"2023-10-16T11:24:25.094026+0000","last_unstale":"2023-10-16T11:24:26.076876+0000","last_undegraded":"2023-10-16T11:24:26.076876+0000","last_fullsized":"2023-10-16T11:24:26.076876+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:51.349970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.081340+0000","last_change":"2023-10-16T11:24:26.081980+0000","last_active":"2023-10-16T11:24:27.081340+0000","last_peered":"2023-10-16T11:24:27.081340+0000","last_clean":"2023-10-16T11:24:27.081340+0000","last_became_active":"2023-10-16T11:24:26.081395+0000","last_became_peered":"2023-10-16T11:24:26.081395+0000","last_unstale":"2023-10-16T11:24:27.081340+0000","last_undegraded":"2023-10-16T11:24:27.081340+0000","last_fullsized":"2023-10-16T11:24:27.081340+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:39.802117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.086320+0000","last_change":"2023-10-16T11:24:25.092100+0000","last_active":"2023-10-16T11:24:26.086320+0000","last_peered":"2023-10-16T11:24:26.086320+0000","last_clean":"2023-10-16T11:24:26.086320+0000","last_became_active":"2023-10-16T11:24:25.091237+0000","last_became_peered":"2023-10-16T11:24:25.091237+0000","last_unstale":"2023-10-16T11:24:26.086320+0000","last_undegraded":"2023-10-16T11:24:26.086320+0000","last_fullsized":"2023-10-16T11:24:26.086320+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:21.985839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069623+0000","last_change":"2023-10-16T11:24:26.084487+0000","last_active":"2023-10-16T11:24:28.069623+0000","last_peered":"2023-10-16T11:24:28.069623+0000","last_clean":"2023-10-16T11:24:28.069623+0000","last_became_active":"2023-10-16T11:24:26.084188+0000","last_became_peered":"2023-10-16T11:24:26.084188+0000","last_unstale":"2023-10-16T11:24:28.069623+0000","last_undegraded":"2023-10-16T11:24:28.069623+0000","last_fullsized":"2023-10-16T11:24:28.069623+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:47.400027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"216'3","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.108001+0000","last_change":"2023-10-16T11:24:25.082968+0000","last_active":"2023-10-16T11:24:26.108001+0000","last_peered":"2023-10-16T11:24:26.108001+0000","last_clean":"2023-10-16T11:24:26.108001+0000","last_became_active":"2023-10-16T11:24:25.082316+0000","last_became_peered":"2023-10-16T11:24:25.082316+0000","last_unstale":"2023-10-16T11:24:26.108001+0000","last_undegraded":"2023-10-16T11:24:26.108001+0000","last_fullsized":"2023-10-16T11:24:26.108001+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:59.924424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069834+0000","last_change":"2023-10-16T11:24:26.092185+0000","last_active":"2023-10-16T11:24:28.069834+0000","last_peered":"2023-10-16T11:24:28.069834+0000","last_clean":"2023-10-16T11:24:28.069834+0000","last_became_active":"2023-10-16T11:24:26.091648+0000","last_became_peered":"2023-10-16T11:24:26.091648+0000","last_unstale":"2023-10-16T11:24:28.069834+0000","last_undegraded":"2023-10-16T11:24:28.069834+0000","last_fullsized":"2023-10-16T11:24:28.069834+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"216'6","reported_seq":18,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.107445+0000","last_change":"2023-10-16T11:24:25.095357+0000","last_active":"2023-10-16T11:24:26.107445+0000","last_peered":"2023-10-16T11:24:26.107445+0000","last_clean":"2023-10-16T11:24:26.107445+0000","last_became_active":"2023-10-16T11:24:25.093832+0000","last_became_peered":"2023-10-16T11:24:25.093832+0000","last_unstale":"2023-10-16T11:24:26.107445+0000","last_undegraded":"2023-10-16T11:24:26.107445+0000","last_fullsized":"2023-10-16T11:24:26.107445+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:14:19.482469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.071056+0000","last_change":"2023-10-16T11:24:26.099039+0000","last_active":"2023-10-16T11:24:28.071056+0000","last_peered":"2023-10-16T11:24:28.071056+0000","last_clean":"2023-10-16T11:24:28.071056+0000","last_became_active":"2023-10-16T11:24:26.097980+0000","last_became_peered":"2023-10-16T11:24:26.097980+0000","last_unstale":"2023-10-16T11:24:28.071056+0000","last_undegraded":"2023-10-16T11:24:28.071056+0000","last_fullsized":"2023-10-16T11:24:28.071056+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:40.585267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"216'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.102723+0000","last_change":"2023-10-16T11:24:25.081377+0000","last_active":"2023-10-16T11:24:26.102723+0000","last_peered":"2023-10-16T11:24:26.102723+0000","last_clean":"2023-10-16T11:24:26.102723+0000","last_became_active":"2023-10-16T11:24:25.080881+0000","last_became_peered":"2023-10-16T11:24:25.080881+0000","last_unstale":"2023-10-16T11:24:26.102723+0000","last_undegraded":"2023-10-16T11:24:26.102723+0000","last_fullsized":"2023-10-16T11:24:26.102723+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:26.054702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.082295+0000","last_change":"2023-10-16T11:24:26.089221+0000","last_active":"2023-10-16T11:24:27.082295+0000","last_peered":"2023-10-16T11:24:27.082295+0000","last_clean":"2023-10-16T11:24:27.082295+0000","last_became_active":"2023-10-16T11:24:26.088644+0000","last_became_peered":"2023-10-16T11:24:26.088644+0000","last_unstale":"2023-10-16T11:24:27.082295+0000","last_undegraded":"2023-10-16T11:24:27.082295+0000","last_fullsized":"2023-10-16T11:24:27.082295+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:10.387018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.082105+0000","last_change":"2023-10-16T11:24:25.084194+0000","last_active":"2023-10-16T11:24:26.082105+0000","last_peered":"2023-10-16T11:24:26.082105+0000","last_clean":"2023-10-16T11:24:26.082105+0000","last_became_active":"2023-10-16T11:24:25.083141+0000","last_became_peered":"2023-10-16T11:24:25.083141+0000","last_unstale":"2023-10-16T11:24:26.082105+0000","last_undegraded":"2023-10-16T11:24:26.082105+0000","last_fullsized":"2023-10-16T11:24:26.082105+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:46.770213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.080140+0000","last_change":"2023-10-16T11:24:26.079602+0000","last_active":"2023-10-16T11:24:27.080140+0000","last_peered":"2023-10-16T11:24:27.080140+0000","last_clean":"2023-10-16T11:24:27.080140+0000","last_became_active":"2023-10-16T11:24:26.079344+0000","last_became_peered":"2023-10-16T11:24:26.079344+0000","last_unstale":"2023-10-16T11:24:27.080140+0000","last_undegraded":"2023-10-16T11:24:27.080140+0000","last_fullsized":"2023-10-16T11:24:27.080140+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.077949+0000","last_change":"2023-10-16T11:24:25.089470+0000","last_active":"2023-10-16T11:24:26.077949+0000","last_peered":"2023-10-16T11:24:26.077949+0000","last_clean":"2023-10-16T11:24:26.077949+0000","last_became_active":"2023-10-16T11:24:25.089213+0000","last_became_peered":"2023-10-16T11:24:25.089213+0000","last_unstale":"2023-10-16T11:24:26.077949+0000","last_undegraded":"2023-10-16T11:24:26.077949+0000","last_fullsized":"2023-10-16T11:24:26.077949+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:34.540849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.068721+0000","last_change":"2023-10-16T11:24:26.084139+0000","last_active":"2023-10-16T11:24:28.068721+0000","last_peered":"2023-10-16T11:24:28.068721+0000","last_clean":"2023-10-16T11:24:28.068721+0000","last_became_active":"2023-10-16T11:24:26.083876+0000","last_became_peered":"2023-10-16T11:24:26.083876+0000","last_unstale":"2023-10-16T11:24:28.068721+0000","last_undegraded":"2023-10-16T11:24:28.068721+0000","last_fullsized":"2023-10-16T11:24:28.068721+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"216'8","reported_seq":20,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.107007+0000","last_change":"2023-10-16T11:24:25.076954+0000","last_active":"2023-10-16T11:24:26.107007+0000","last_peered":"2023-10-16T11:24:26.107007+0000","last_clean":"2023-10-16T11:24:26.107007+0000","last_became_active":"2023-10-16T11:24:25.076509+0000","last_became_peered":"2023-10-16T11:24:25.076509+0000","last_unstale":"2023-10-16T11:24:26.107007+0000","last_undegraded":"2023-10-16T11:24:26.107007+0000","last_fullsized":"2023-10-16T11:24:26.107007+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:46.770213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.201200+0000","last_change":"2023-10-16T11:24:26.081796+0000","last_active":"2023-10-16T11:24:27.201200+0000","last_peered":"2023-10-16T11:24:27.201200+0000","last_clean":"2023-10-16T11:24:27.201200+0000","last_became_active":"2023-10-16T11:24:26.080287+0000","last_became_peered":"2023-10-16T11:24:26.080287+0000","last_unstale":"2023-10-16T11:24:27.201200+0000","last_undegraded":"2023-10-16T11:24:27.201200+0000","last_fullsized":"2023-10-16T11:24:27.201200+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"216'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.105872+0000","last_change":"2023-10-16T11:24:25.079789+0000","last_active":"2023-10-16T11:24:26.105872+0000","last_peered":"2023-10-16T11:24:26.105872+0000","last_clean":"2023-10-16T11:24:26.105872+0000","last_became_active":"2023-10-16T11:24:25.079495+0000","last_became_peered":"2023-10-16T11:24:25.079495+0000","last_unstale":"2023-10-16T11:24:26.105872+0000","last_undegraded":"2023-10-16T11:24:26.105872+0000","last_fullsized":"2023-10-16T11:24:26.105872+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:44.531367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.201013+0000","last_change":"2023-10-16T11:24:26.086833+0000","last_active":"2023-10-16T11:24:27.201013+0000","last_peered":"2023-10-16T11:24:27.201013+0000","last_clean":"2023-10-16T11:24:27.201013+0000","last_became_active":"2023-10-16T11:24:26.085367+0000","last_became_peered":"2023-10-16T11:24:26.085367+0000","last_unstale":"2023-10-16T11:24:27.201013+0000","last_undegraded":"2023-10-16T11:24:27.201013+0000","last_fullsized":"2023-10-16T11:24:27.201013+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"216'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.106988+0000","last_change":"2023-10-16T11:24:25.087765+0000","last_active":"2023-10-16T11:24:26.106988+0000","last_peered":"2023-10-16T11:24:26.106988+0000","last_clean":"2023-10-16T11:24:26.106988+0000","last_became_active":"2023-10-16T11:24:25.086967+0000","last_became_peered":"2023-10-16T11:24:25.086967+0000","last_unstale":"2023-10-16T11:24:26.106988+0000","last_undegraded":"2023-10-16T11:24:26.106988+0000","last_fullsized":"2023-10-16T11:24:26.106988+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.080678+0000","last_change":"2023-10-16T11:24:26.069780+0000","last_active":"2023-10-16T11:24:27.080678+0000","last_peered":"2023-10-16T11:24:27.080678+0000","last_clean":"2023-10-16T11:24:27.080678+0000","last_became_active":"2023-10-16T11:24:26.068772+0000","last_became_peered":"2023-10-16T11:24:26.068772+0000","last_unstale":"2023-10-16T11:24:27.080678+0000","last_undegraded":"2023-10-16T11:24:27.080678+0000","last_fullsized":"2023-10-16T11:24:27.080678+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"216'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.107193+0000","last_change":"2023-10-16T11:24:25.083155+0000","last_active":"2023-10-16T11:24:26.107193+0000","last_peered":"2023-10-16T11:24:26.107193+0000","last_clean":"2023-10-16T11:24:26.107193+0000","last_became_active":"2023-10-16T11:24:25.082690+0000","last_became_peered":"2023-10-16T11:24:25.082690+0000","last_unstale":"2023-10-16T11:24:26.107193+0000","last_undegraded":"2023-10-16T11:24:26.107193+0000","last_fullsized":"2023-10-16T11:24:26.107193+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:46:58.816018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.082502+0000","last_change":"2023-10-16T11:24:26.092876+0000","last_active":"2023-10-16T11:24:27.082502+0000","last_peered":"2023-10-16T11:24:27.082502+0000","last_clean":"2023-10-16T11:24:27.082502+0000","last_became_active":"2023-10-16T11:24:26.091310+0000","last_became_peered":"2023-10-16T11:24:26.091310+0000","last_unstale":"2023-10-16T11:24:27.082502+0000","last_undegraded":"2023-10-16T11:24:27.082502+0000","last_fullsized":"2023-10-16T11:24:27.082502+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:22.891780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.057300+0000","last_change":"2023-10-16T11:24:25.079021+0000","last_active":"2023-10-16T11:24:26.057300+0000","last_peered":"2023-10-16T11:24:26.057300+0000","last_clean":"2023-10-16T11:24:26.057300+0000","last_became_active":"2023-10-16T11:24:25.076800+0000","last_became_peered":"2023-10-16T11:24:25.076800+0000","last_unstale":"2023-10-16T11:24:26.057300+0000","last_undegraded":"2023-10-16T11:24:26.057300+0000","last_fullsized":"2023-10-16T11:24:26.057300+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:56.161407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.082026+0000","last_change":"2023-10-16T11:24:26.091096+0000","last_active":"2023-10-16T11:24:27.082026+0000","last_peered":"2023-10-16T11:24:27.082026+0000","last_clean":"2023-10-16T11:24:27.082026+0000","last_became_active":"2023-10-16T11:24:26.090774+0000","last_became_peered":"2023-10-16T11:24:26.090774+0000","last_unstale":"2023-10-16T11:24:27.082026+0000","last_undegraded":"2023-10-16T11:24:27.082026+0000","last_fullsized":"2023-10-16T11:24:27.082026+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:27.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"216'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.107501+0000","last_change":"2023-10-16T11:24:25.072340+0000","last_active":"2023-10-16T11:24:26.107501+0000","last_peered":"2023-10-16T11:24:26.107501+0000","last_clean":"2023-10-16T11:24:26.107501+0000","last_became_active":"2023-10-16T11:24:25.071728+0000","last_became_peered":"2023-10-16T11:24:25.071728+0000","last_unstale":"2023-10-16T11:24:26.107501+0000","last_undegraded":"2023-10-16T11:24:26.107501+0000","last_fullsized":"2023-10-16T11:24:26.107501+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:49.041521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.081078+0000","last_change":"2023-10-16T11:24:26.085260+0000","last_active":"2023-10-16T11:24:27.081078+0000","last_peered":"2023-10-16T11:24:27.081078+0000","last_clean":"2023-10-16T11:24:27.081078+0000","last_became_active":"2023-10-16T11:24:26.084860+0000","last_became_peered":"2023-10-16T11:24:26.084860+0000","last_unstale":"2023-10-16T11:24:27.081078+0000","last_undegraded":"2023-10-16T11:24:27.081078+0000","last_fullsized":"2023-10-16T11:24:27.081078+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.081605+0000","last_change":"2023-10-16T11:24:25.091656+0000","last_active":"2023-10-16T11:24:26.081605+0000","last_peered":"2023-10-16T11:24:26.081605+0000","last_clean":"2023-10-16T11:24:26.081605+0000","last_became_active":"2023-10-16T11:24:25.091316+0000","last_became_peered":"2023-10-16T11:24:25.091316+0000","last_unstale":"2023-10-16T11:24:26.081605+0000","last_undegraded":"2023-10-16T11:24:26.081605+0000","last_fullsized":"2023-10-16T11:24:26.081605+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:26.054702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.200399+0000","last_change":"2023-10-16T11:24:26.090951+0000","last_active":"2023-10-16T11:24:27.200399+0000","last_peered":"2023-10-16T11:24:27.200399+0000","last_clean":"2023-10-16T11:24:27.200399+0000","last_became_active":"2023-10-16T11:24:26.089241+0000","last_became_peered":"2023-10-16T11:24:26.089241+0000","last_unstale":"2023-10-16T11:24:27.200399+0000","last_undegraded":"2023-10-16T11:24:27.200399+0000","last_fullsized":"2023-10-16T11:24:27.200399+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:40.970357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.074587+0000","last_change":"2023-10-16T11:24:25.086180+0000","last_active":"2023-10-16T11:24:26.074587+0000","last_peered":"2023-10-16T11:24:26.074587+0000","last_clean":"2023-10-16T11:24:26.074587+0000","last_became_active":"2023-10-16T11:24:25.085316+0000","last_became_peered":"2023-10-16T11:24:25.085316+0000","last_unstale":"2023-10-16T11:24:26.074587+0000","last_undegraded":"2023-10-16T11:24:26.074587+0000","last_fullsized":"2023-10-16T11:24:26.074587+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.083967+0000","last_change":"2023-10-16T11:24:26.088438+0000","last_active":"2023-10-16T11:24:27.083967+0000","last_peered":"2023-10-16T11:24:27.083967+0000","last_clean":"2023-10-16T11:24:27.083967+0000","last_became_active":"2023-10-16T11:24:26.087973+0000","last_became_peered":"2023-10-16T11:24:26.087973+0000","last_unstale":"2023-10-16T11:24:27.083967+0000","last_undegraded":"2023-10-16T11:24:27.083967+0000","last_fullsized":"2023-10-16T11:24:27.083967+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.076600+0000","last_change":"2023-10-16T11:24:25.080225+0000","last_active":"2023-10-16T11:24:26.076600+0000","last_peered":"2023-10-16T11:24:26.076600+0000","last_clean":"2023-10-16T11:24:26.076600+0000","last_became_active":"2023-10-16T11:24:25.079921+0000","last_became_peered":"2023-10-16T11:24:25.079921+0000","last_unstale":"2023-10-16T11:24:26.076600+0000","last_undegraded":"2023-10-16T11:24:26.076600+0000","last_fullsized":"2023-10-16T11:24:26.076600+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:46.550429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.976480+0000","last_change":"2023-10-16T11:24:26.085909+0000","last_active":"2023-10-16T11:24:27.976480+0000","last_peered":"2023-10-16T11:24:27.976480+0000","last_clean":"2023-10-16T11:24:27.976480+0000","last_became_active":"2023-10-16T11:24:26.085620+0000","last_became_peered":"2023-10-16T11:24:26.085620+0000","last_unstale":"2023-10-16T11:24:27.976480+0000","last_undegraded":"2023-10-16T11:24:27.976480+0000","last_fullsized":"2023-10-16T11:24:27.976480+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:41.023833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"216'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.106261+0000","last_change":"2023-10-16T11:24:25.094230+0000","last_active":"2023-10-16T11:24:26.106261+0000","last_peered":"2023-10-16T11:24:26.106261+0000","last_clean":"2023-10-16T11:24:26.106261+0000","last_became_active":"2023-10-16T11:24:25.093896+0000","last_became_peered":"2023-10-16T11:24:25.093896+0000","last_unstale":"2023-10-16T11:24:26.106261+0000","last_undegraded":"2023-10-16T11:24:26.106261+0000","last_fullsized":"2023-10-16T11:24:26.106261+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070457+0000","last_change":"2023-10-16T11:24:26.089940+0000","last_active":"2023-10-16T11:24:28.070457+0000","last_peered":"2023-10-16T11:24:28.070457+0000","last_clean":"2023-10-16T11:24:28.070457+0000","last_became_active":"2023-10-16T11:24:26.087906+0000","last_became_peered":"2023-10-16T11:24:26.087906+0000","last_unstale":"2023-10-16T11:24:28.070457+0000","last_undegraded":"2023-10-16T11:24:28.070457+0000","last_fullsized":"2023-10-16T11:24:28.070457+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:44.687193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.076347+0000","last_change":"2023-10-16T11:24:25.093214+0000","last_active":"2023-10-16T11:24:26.076347+0000","last_peered":"2023-10-16T11:24:26.076347+0000","last_clean":"2023-10-16T11:24:26.076347+0000","last_became_active":"2023-10-16T11:24:25.092877+0000","last_became_peered":"2023-10-16T11:24:25.092877+0000","last_unstale":"2023-10-16T11:24:26.076347+0000","last_undegraded":"2023-10-16T11:24:26.076347+0000","last_fullsized":"2023-10-16T11:24:26.076347+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:59.924424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.200857+0000","last_change":"2023-10-16T11:24:26.085231+0000","last_active":"2023-10-16T11:24:27.200857+0000","last_peered":"2023-10-16T11:24:27.200857+0000","last_clean":"2023-10-16T11:24:27.200857+0000","last_became_active":"2023-10-16T11:24:26.084889+0000","last_became_peered":"2023-10-16T11:24:26.084889+0000","last_unstale":"2023-10-16T11:24:27.200857+0000","last_undegraded":"2023-10-16T11:24:27.200857+0000","last_fullsized":"2023-10-16T11:24:27.200857+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.077130+0000","last_change":"2023-10-16T11:24:25.077036+0000","last_active":"2023-10-16T11:24:26.077130+0000","last_peered":"2023-10-16T11:24:26.077130+0000","last_clean":"2023-10-16T11:24:26.077130+0000","last_became_active":"2023-10-16T11:24:25.076342+0000","last_became_peered":"2023-10-16T11:24:25.076342+0000","last_unstale":"2023-10-16T11:24:26.077130+0000","last_undegraded":"2023-10-16T11:24:26.077130+0000","last_fullsized":"2023-10-16T11:24:26.077130+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:36.162662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.084158+0000","last_change":"2023-10-16T11:24:26.096341+0000","last_active":"2023-10-16T11:24:27.084158+0000","last_peered":"2023-10-16T11:24:27.084158+0000","last_clean":"2023-10-16T11:24:27.084158+0000","last_became_active":"2023-10-16T11:24:26.095836+0000","last_became_peered":"2023-10-16T11:24:26.095836+0000","last_unstale":"2023-10-16T11:24:27.084158+0000","last_undegraded":"2023-10-16T11:24:27.084158+0000","last_fullsized":"2023-10-16T11:24:27.084158+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.057631+0000","last_change":"2023-10-16T11:24:25.088585+0000","last_active":"2023-10-16T11:24:26.057631+0000","last_peered":"2023-10-16T11:24:26.057631+0000","last_clean":"2023-10-16T11:24:26.057631+0000","last_became_active":"2023-10-16T11:24:25.088011+0000","last_became_peered":"2023-10-16T11:24:25.088011+0000","last_unstale":"2023-10-16T11:24:26.057631+0000","last_undegraded":"2023-10-16T11:24:26.057631+0000","last_fullsized":"2023-10-16T11:24:26.057631+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069619+0000","last_change":"2023-10-16T11:24:26.093027+0000","last_active":"2023-10-16T11:24:28.069619+0000","last_peered":"2023-10-16T11:24:28.069619+0000","last_clean":"2023-10-16T11:24:28.069619+0000","last_became_active":"2023-10-16T11:24:26.092766+0000","last_became_peered":"2023-10-16T11:24:26.092766+0000","last_unstale":"2023-10-16T11:24:28.069619+0000","last_undegraded":"2023-10-16T11:24:28.069619+0000","last_fullsized":"2023-10-16T11:24:28.069619+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.073725+0000","last_change":"2023-10-16T11:24:25.090432+0000","last_active":"2023-10-16T11:24:26.073725+0000","last_peered":"2023-10-16T11:24:26.073725+0000","last_clean":"2023-10-16T11:24:26.073725+0000","last_became_active":"2023-10-16T11:24:25.090142+0000","last_became_peered":"2023-10-16T11:24:25.090142+0000","last_unstale":"2023-10-16T11:24:26.073725+0000","last_undegraded":"2023-10-16T11:24:26.073725+0000","last_fullsized":"2023-10-16T11:24:26.073725+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:22.858690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.976530+0000","last_change":"2023-10-16T11:24:26.083766+0000","last_active":"2023-10-16T11:24:27.976530+0000","last_peered":"2023-10-16T11:24:27.976530+0000","last_clean":"2023-10-16T11:24:27.976530+0000","last_became_active":"2023-10-16T11:24:26.083447+0000","last_became_peered":"2023-10-16T11:24:26.083447+0000","last_unstale":"2023-10-16T11:24:27.976530+0000","last_undegraded":"2023-10-16T11:24:27.976530+0000","last_fullsized":"2023-10-16T11:24:27.976530+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:44:14.656443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.078744+0000","last_change":"2023-10-16T11:24:25.080689+0000","last_active":"2023-10-16T11:24:26.078744+0000","last_peered":"2023-10-16T11:24:26.078744+0000","last_clean":"2023-10-16T11:24:26.078744+0000","last_became_active":"2023-10-16T11:24:25.080305+0000","last_became_peered":"2023-10-16T11:24:25.080305+0000","last_unstale":"2023-10-16T11:24:26.078744+0000","last_undegraded":"2023-10-16T11:24:26.078744+0000","last_fullsized":"2023-10-16T11:24:26.078744+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:43.670603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.080245+0000","last_change":"2023-10-16T11:24:26.083366+0000","last_active":"2023-10-16T11:24:27.080245+0000","last_peered":"2023-10-16T11:24:27.080245+0000","last_clean":"2023-10-16T11:24:27.080245+0000","last_became_active":"2023-10-16T11:24:26.082635+0000","last_became_peered":"2023-10-16T11:24:26.082635+0000","last_unstale":"2023-10-16T11:24:27.080245+0000","last_undegraded":"2023-10-16T11:24:27.080245+0000","last_fullsized":"2023-10-16T11:24:27.080245+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.087162+0000","last_change":"2023-10-16T11:24:25.088161+0000","last_active":"2023-10-16T11:24:26.087162+0000","last_peered":"2023-10-16T11:24:26.087162+0000","last_clean":"2023-10-16T11:24:26.087162+0000","last_became_active":"2023-10-16T11:24:25.086051+0000","last_became_peered":"2023-10-16T11:24:25.086051+0000","last_unstale":"2023-10-16T11:24:26.087162+0000","last_undegraded":"2023-10-16T11:24:26.087162+0000","last_fullsized":"2023-10-16T11:24:26.087162+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.200464+0000","last_change":"2023-10-16T11:24:26.081718+0000","last_active":"2023-10-16T11:24:27.200464+0000","last_peered":"2023-10-16T11:24:27.200464+0000","last_clean":"2023-10-16T11:24:27.200464+0000","last_became_active":"2023-10-16T11:24:26.081447+0000","last_became_peered":"2023-10-16T11:24:26.081447+0000","last_unstale":"2023-10-16T11:24:27.200464+0000","last_undegraded":"2023-10-16T11:24:27.200464+0000","last_fullsized":"2023-10-16T11:24:27.200464+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.067543+0000","last_change":"2023-10-16T11:24:25.087755+0000","last_active":"2023-10-16T11:24:26.067543+0000","last_peered":"2023-10-16T11:24:26.067543+0000","last_clean":"2023-10-16T11:24:26.067543+0000","last_became_active":"2023-10-16T11:24:25.087497+0000","last_became_peered":"2023-10-16T11:24:25.087497+0000","last_unstale":"2023-10-16T11:24:26.067543+0000","last_undegraded":"2023-10-16T11:24:26.067543+0000","last_fullsized":"2023-10-16T11:24:26.067543+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:59:26.196708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070405+0000","last_change":"2023-10-16T11:24:26.091340+0000","last_active":"2023-10-16T11:24:28.070405+0000","last_peered":"2023-10-16T11:24:28.070405+0000","last_clean":"2023-10-16T11:24:28.070405+0000","last_became_active":"2023-10-16T11:24:26.091059+0000","last_became_peered":"2023-10-16T11:24:26.091059+0000","last_unstale":"2023-10-16T11:24:28.070405+0000","last_undegraded":"2023-10-16T11:24:28.070405+0000","last_fullsized":"2023-10-16T11:24:28.070405+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:37.605400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"216'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.103314+0000","last_change":"2023-10-16T11:24:25.087491+0000","last_active":"2023-10-16T11:24:26.103314+0000","last_peered":"2023-10-16T11:24:26.103314+0000","last_clean":"2023-10-16T11:24:26.103314+0000","last_became_active":"2023-10-16T11:24:25.086826+0000","last_became_peered":"2023-10-16T11:24:25.086826+0000","last_unstale":"2023-10-16T11:24:26.103314+0000","last_undegraded":"2023-10-16T11:24:26.103314+0000","last_fullsized":"2023-10-16T11:24:26.103314+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:07.303719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"206'2767","reported_seq":3648,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:24:26.072119+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:24:26.072119+0000","last_peered":"2023-10-16T11:24:26.072119+0000","last_clean":"2023-10-16T11:24:26.072119+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:24:26.072119+0000","last_undegraded":"2023-10-16T11:24:26.072119+0000","last_fullsized":"2023-10-16T11:24:26.072119+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2767,"log_dups_size":0,"ondisk_log_size":2767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":581,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":581,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2767,"ondisk_log_size":2767,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575986,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52480,"kb_used_data":3252,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319360,"statfs":{"total":96636764160,"available":96583024640,"internally_reserved":0,"allocated":3330048,"data_stored":1912427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.179,"15min":1.2170000000000001},"min":{"1min":0.81499999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.7250000000000001,"5min":2.085,"15min":2.3399999999999999},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.1719999999999999,"15min":1.218},"min":{"1min":0.876,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.8320000000000001},"last":1.333}]},{"osd":1,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2689999999999999,"15min":1.2509999999999999},"min":{"1min":0.90700000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.6479999999999999,"5min":1.913,"15min":1.913},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2430000000000001,"15min":1.2749999999999999},"min":{"1min":0.76700000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6279999999999999,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.133}]},{"osd":2,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3140000000000001,"15min":1.3109999999999999},"min":{"1min":1.008,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6579999999999999,"5min":1.8680000000000001,"15min":2.4460000000000002},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.343,"5min":1.3540000000000001,"15min":1.3640000000000001},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.708,"5min":1.859,"15min":1.974},"last":1.004}]},{"osd":3,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.381,"15min":1.3899999999999999},"min":{"1min":1.149,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7290000000000001,"5min":1.8280000000000001,"15min":1.972},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.4430000000000001,"15min":1.4630000000000001},"min":{"1min":1.0860000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.8799999999999999,"5min":2.0680000000000001,"15min":2.577},"last":1.1850000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.423,"15min":1.417},"min":{"1min":0.98599999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0030000000000001,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.425},{"interface":"front","average":{"1min":1.498,"5min":1.391,"15min":1.3899999999999999},"min":{"1min":1.0469999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8280000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.5089999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.585,"15min":1.5960000000000001},"min":{"1min":1.1579999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.946,"5min":2.2309999999999999,"15min":3.3519999999999999},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.641,"5min":1.514,"15min":1.5089999999999999},"min":{"1min":1.351,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.1600000000000001,"5min":2.3460000000000001,"15min":2.4969999999999999},"last":1.6950000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.488,"15min":1.5209999999999999},"min":{"1min":0.97099999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.845,"5min":2.101,"15min":2.7250000000000001},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.7210000000000001,"5min":1.6890000000000001,"15min":1.71},"min":{"1min":1.425,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1040000000000001,"5min":2.298,"15min":3.1440000000000001},"last":1.5649999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575986,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33848,"kb_used_data":2664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337992,"statfs":{"total":96636764160,"available":96602103808,"internally_reserved":0,"allocated":2727936,"data_stored":1321029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.2,"15min":1.22},"min":{"1min":0.78000000000000003,"5min":0.77200000000000002,"15min":0.64000000000000001},"max":{"1min":1.5740000000000001,"5min":1.698,"15min":2.0419999999999998},"last":1.278},{"interface":"front","average":{"1min":1.355,"5min":1.2589999999999999,"15min":1.2669999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.66400000000000003},"max":{"1min":1.6799999999999999,"5min":1.839,"15min":1.839},"last":1.466}]},{"osd":1,"last update":"Mon Oct 16 11:23:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2230000000000001,"15min":1.2130000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7350000000000001,"5min":1.7629999999999999,"15min":1.8420000000000001},"last":1.409},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3640000000000001,"15min":1.351},"min":{"1min":0.94099999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6910000000000001,"5min":1.796,"15min":1.796},"last":1.2130000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:23:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.2410000000000001,"15min":1.244},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.68000000000000005},"max":{"1min":1.579,"5min":1.7589999999999999,"15min":1.861},"last":1.806},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3500000000000001,"15min":1.3540000000000001},"min":{"1min":0.92000000000000004,"5min":0.85799999999999998,"15min":0.72699999999999998},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.9790000000000001},"last":1.119}]},{"osd":3,"last update":"Mon Oct 16 11:23:35 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.341,"15min":1.3100000000000001},"min":{"1min":0.95699999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.6379999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.708},{"interface":"front","average":{"1min":1.236,"5min":1.256,"15min":1.264},"min":{"1min":0.91700000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.671,"5min":1.744,"15min":1.744},"last":1.528}]},{"osd":4,"last update":"Mon Oct 16 11:23:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.278,"15min":1.302},"min":{"1min":0.96099999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.514,"15min":1.5029999999999999},"min":{"1min":0.94999999999999996,"5min":0.622,"15min":0.622},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":1.778}]},{"osd":5,"last update":"Mon Oct 16 11:23:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.3380000000000001,"15min":1.333},"min":{"1min":0.747,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.7410000000000001,"5min":1.8069999999999999,"15min":2.1669999999999998},"last":1.881},{"interface":"front","average":{"1min":1.605,"5min":1.528,"15min":1.5429999999999999},"min":{"1min":0.90800000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":1.7490000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:23:35 2023","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.51,"15min":1.494},"min":{"1min":0.98999999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.8039999999999998,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":1.8340000000000001},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.5489999999999999,"15min":1.552},"min":{"1min":1.3,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.5449999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":2.137}]}]},{"osd":5,"up_from":13,"seq":55834575986,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29688,"kb_used_data":2664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342152,"statfs":{"total":96636764160,"available":96606363648,"internally_reserved":0,"allocated":2727936,"data_stored":1321029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2350000000000001,"15min":1.27},"min":{"1min":0.74199999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.583,"5min":2.661,"15min":2.661},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.306,"5min":1.3029999999999999,"15min":1.349},"min":{"1min":0.95799999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7909999999999999,"5min":1.7949999999999999,"15min":1.865},"last":0.88600000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1819999999999999,"15min":1.1950000000000001},"min":{"1min":0.81999999999999995,"5min":0.80800000000000005,"15min":0.79100000000000004},"max":{"1min":1.5009999999999999,"5min":1.7869999999999999,"15min":1.992},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.258,"15min":1.2290000000000001},"min":{"1min":0.84399999999999997,"5min":0.84199999999999997,"15min":0.80800000000000005},"max":{"1min":1.635,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.0920000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.264,"15min":1.244},"min":{"1min":0.97899999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.625,"5min":1.952,"15min":1.952},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.329,"5min":1.2949999999999999,"15min":1.3220000000000001},"min":{"1min":0.92400000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":2.1640000000000001},"last":1.341}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3320000000000001,"15min":1.3340000000000001},"min":{"1min":0.98999999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":3.012,"15min":3.012},"last":1.296},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.429,"15min":1.425},"min":{"1min":1.0900000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":2.2250000000000001},"last":1.149}]},{"osd":4,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.339,"15min":1.339},"min":{"1min":0.94199999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.6990000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.387},{"interface":"front","average":{"1min":1.6799999999999999,"5min":1.5720000000000001,"15min":1.5740000000000001},"min":{"1min":1.353,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.948,"5min":2.903,"15min":2.903},"last":1.5640000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4330000000000001,"15min":1.4139999999999999},"min":{"1min":0.873,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8660000000000001,"5min":2.0649999999999999,"15min":2.0979999999999999},"last":1.266},{"interface":"front","average":{"1min":1.51,"5min":1.494,"15min":1.4770000000000001},"min":{"1min":0.95399999999999996,"5min":0.90700000000000003,"15min":0.78400000000000003},"max":{"1min":1.841,"5min":3.073,"15min":3.073},"last":1.5980000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.675,"5min":1.5660000000000001,"15min":1.5469999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0779999999999998,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.645,"15min":1.645},"min":{"1min":1.1779999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.1099999999999999,"5min":2.1459999999999999,"15min":2.2930000000000001},"last":1.77}]}]},{"osd":4,"up_from":13,"seq":55834575987,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51908,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319932,"statfs":{"total":96636764160,"available":96583610368,"internally_reserved":0,"allocated":2744320,"data_stored":1322163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1519999999999999,"15min":1.1859999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.7290000000000001,"5min":1.792,"15min":1.792},"last":2.899},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.218,"15min":1.2350000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.9299999999999999},"last":3.016}]},{"osd":1,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2769999999999999,"15min":1.298},"min":{"1min":0.81799999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.841,"5min":1.841,"15min":1.913},"last":2.6560000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.236,"15min":1.25},"min":{"1min":0.78500000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.877,"5min":1.877,"15min":1.889},"last":3.0739999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.29,"15min":1.3089999999999999},"min":{"1min":1.0369999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.835},"last":3.2170000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.3999999999999999,"15min":1.403},"min":{"1min":0.879,"5min":0.879,"15min":0.872},"max":{"1min":1.8380000000000001,"5min":1.903,"15min":1.9139999999999999},"last":3.1600000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3169999999999999,"15min":1.337},"min":{"1min":0.91100000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.6830000000000001,"5min":1.8280000000000001,"15min":1.849},"last":3.6019999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4410000000000001,"15min":1.4330000000000001},"min":{"1min":1.145,"5min":1.008,"15min":0.96999999999999997},"max":{"1min":1.7769999999999999,"5min":1.865,"15min":2.0720000000000001},"last":3.4220000000000002}]},{"osd":5,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.4810000000000001,"15min":1.4399999999999999},"min":{"1min":1.0149999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9399999999999999,"5min":2.105,"15min":2.105},"last":3.2549999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.431,"15min":1.415},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.78600000000000003},"max":{"1min":1.8240000000000001,"5min":2.04,"15min":2.04},"last":3.7989999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.5409999999999999,"15min":1.5029999999999999},"min":{"1min":0.94299999999999995,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":3.157,"5min":3.157,"15min":3.157},"last":3.3290000000000002},{"interface":"front","average":{"1min":1.5,"5min":1.6100000000000001,"15min":1.611},"min":{"1min":1.0940000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0089999999999999,"5min":2.2330000000000001,"15min":2.2770000000000001},"last":3.5019999999999998}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.5640000000000001,"15min":1.5660000000000001},"min":{"1min":1.0549999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.855,"5min":2.113,"15min":2.3420000000000001},"last":4.2050000000000001},{"interface":"front","average":{"1min":1.639,"5min":1.6619999999999999,"15min":1.6399999999999999},"min":{"1min":1.0660000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9630000000000001,"5min":2.1469999999999998,"15min":2.3839999999999999},"last":3.7000000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575986,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47748,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324092,"statfs":{"total":96636764160,"available":96587870208,"internally_reserved":0,"allocated":2736128,"data_stored":1320629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.89400000000000002,"15min":0.90700000000000003},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.478,"5min":1.478,"15min":1.901},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0369999999999999,"15min":1.0720000000000001},"min":{"1min":0.59299999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.391,"5min":1.6180000000000001,"15min":1.6679999999999999},"last":1.085}]},{"osd":1,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.95599999999999996,"15min":0.98799999999999999},"min":{"1min":0.60799999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.105,"15min":1.133},"min":{"1min":0.48399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.601,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.72699999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0760000000000001,"15min":1.117},"min":{"1min":0.65500000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.524,"5min":1.694,"15min":1.694},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.198,"15min":1.236},"min":{"1min":0.625,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.53100000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3169999999999999,"15min":1.3500000000000001},"min":{"1min":0.83599999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.806,"5min":1.806,"15min":1.873},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.357,"15min":1.367},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.8819999999999999},"last":1.032}]},{"osd":5,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.385,"15min":1.401},"min":{"1min":0.80600000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.4059999999999999,"15min":1.444},"min":{"1min":1.151,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.3}]},{"osd":6,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.504,"15min":1.5329999999999999},"min":{"1min":0.876,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":3.9129999999999998,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5,"15min":1.5369999999999999},"min":{"1min":1.153,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":3.9319999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.4119999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.528,"15min":1.5600000000000001},"min":{"1min":0.95399999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.877,"5min":2.0790000000000002,"15min":2.2679999999999998},"last":1.196},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.639,"15min":1.673},"min":{"1min":1.1279999999999999,"5min":1,"15min":1},"max":{"1min":2.052,"5min":2.198,"15min":2.5129999999999999},"last":1.4470000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575985,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51896,"kb_used_data":2660,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319944,"statfs":{"total":96636764160,"available":96583622656,"internally_reserved":0,"allocated":2723840,"data_stored":1320447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.91000000000000003,"15min":0.92200000000000004},"min":{"1min":0.74199999999999999,"5min":0.51900000000000002,"15min":0.5},"max":{"1min":1.532,"5min":1.532,"15min":1.669},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.0569999999999999,"15min":1.0820000000000001},"min":{"1min":0.72499999999999998,"5min":0.47699999999999998,"15min":0.32600000000000001},"max":{"1min":1.496,"5min":1.681,"15min":2.1429999999999998},"last":1.3460000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.089,"15min":1.097},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.39200000000000002},"max":{"1min":1.415,"5min":1.6779999999999999,"15min":1.978},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1479999999999999,"15min":1.165},"min":{"1min":0.57599999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.448,"5min":1.609,"15min":1.869},"last":1.1659999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.0649999999999999,"15min":1.087},"min":{"1min":0.54600000000000004,"5min":0.437,"15min":0.437},"max":{"1min":1.464,"5min":1.651,"15min":1.8560000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.252,"15min":1.274},"min":{"1min":0.82199999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6120000000000001,"5min":1.7450000000000001,"15min":1.857},"last":1}]},{"osd":4,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3740000000000001,"15min":1.415},"min":{"1min":0.91500000000000004,"5min":0.86599999999999999,"15min":0.73499999999999999},"max":{"1min":1.6910000000000001,"5min":1.8029999999999999,"15min":2.5059999999999998},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3440000000000001,"15min":1.3520000000000001},"min":{"1min":0.97899999999999998,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":1.825,"5min":1.825,"15min":2.0880000000000001},"last":1.125}]},{"osd":5,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.444,"15min":1.464},"min":{"1min":1.0720000000000001,"5min":0.84999999999999998,"15min":0.78500000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2269999999999999},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4550000000000001,"15min":1.4890000000000001},"min":{"1min":1.0509999999999999,"5min":0.95799999999999996,"15min":0.874},"max":{"1min":1.798,"5min":1.931,"15min":2.6259999999999999},"last":1.528}]},{"osd":6,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4119999999999999,"15min":1.411},"min":{"1min":1.0569999999999999,"5min":0.84099999999999997,"15min":0.80500000000000005},"max":{"1min":1.71,"5min":1.9019999999999999,"15min":2.2109999999999999},"last":1.401},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5,"15min":1.5369999999999999},"min":{"1min":1.1899999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.2909999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.5660000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":1.5489999999999999,"15min":1.5860000000000001},"min":{"1min":1.2290000000000001,"5min":0.995,"15min":0.92500000000000004},"max":{"1min":2.1469999999999998,"5min":2.262,"15min":2.8109999999999999},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.494,"5min":1.4870000000000001,"15min":1.5149999999999999},"min":{"1min":1.242,"5min":0.81599999999999995,"15min":0.80100000000000005},"max":{"1min":1.8180000000000001,"5min":2.2269999999999999,"15min":2.2829999999999999},"last":1.284}]}]},{"osd":1,"up_from":13,"seq":55834575985,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60220,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311620,"statfs":{"total":96636764160,"available":96575098880,"internally_reserved":0,"allocated":2732032,"data_stored":1320503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.91900000000000004,"15min":0.91500000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.575,"5min":1.575,"15min":1.7929999999999999},"last":5.117},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0289999999999999,"15min":1.0580000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.494,"5min":1.5509999999999999,"15min":1.667},"last":1.5069999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.98399999999999999,"15min":1.0189999999999999},"min":{"1min":0.64300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.72},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.252,"15min":1.276},"min":{"1min":0.69799999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":3.5840000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.1879999999999999,"15min":1.216},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.774,"5min":1.774,"15min":1.79},"last":3.4670000000000001},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.169,"15min":1.1739999999999999},"min":{"1min":0.66700000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.633,"5min":1.633,"15min":1.802},"last":1.5600000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.2609999999999999,"15min":1.274},"min":{"1min":0.86299999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.653,"5min":1.887,"15min":1.9159999999999999},"last":5.29},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.409,"15min":1.415},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.887,"5min":2.0619999999999998,"15min":2.093},"last":5.0179999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.379,"15min":1.409},"min":{"1min":0.93700000000000006,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.948,"5min":2.0009999999999999,"15min":2.0640000000000001},"last":6.0979999999999999},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4239999999999999,"15min":1.421},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":1.825,"5min":1.825,"15min":2.0649999999999999},"last":1.976}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4850000000000001,"15min":1.496},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.879,"5min":2.1970000000000001,"15min":2.2949999999999999},"last":4.0499999999999998},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4710000000000001,"15min":1.486},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.891,"5min":1.891,"15min":2.1970000000000001},"last":2.024}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5649999999999999,"15min":1.5620000000000001},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.1720000000000002},"last":2.6930000000000001},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.625,"15min":1.607},"min":{"1min":1.1080000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.9359999999999999,"5min":2.1989999999999998,"15min":2.2810000000000001},"last":4.1040000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575986,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52476,"kb_used_data":3244,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319364,"statfs":{"total":96636764160,"available":96583028736,"internally_reserved":0,"allocated":3321856,"data_stored":1910905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.871,"15min":0.90000000000000002},"min":{"1min":0.73199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.502,"5min":1.52,"15min":1.619},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.1060000000000001},"min":{"1min":0.627,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.621,"5min":1.6399999999999999,"15min":1.76},"last":1.222}]},{"osd":2,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.097,"15min":1.0860000000000001},"min":{"1min":0.65300000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.556,"5min":1.649,"15min":1.649},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.095,"15min":1.1459999999999999},"min":{"1min":0.81100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4179999999999999,"5min":1.623,"15min":1.6870000000000001},"last":1.337}]},{"osd":3,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.056,"15min":1.085},"min":{"1min":0.69499999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.556},{"interface":"front","average":{"1min":1.274,"5min":1.244,"15min":1.256},"min":{"1min":0.91500000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.716,"5min":1.7609999999999999,"15min":1.9830000000000001},"last":1.1519999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.387,"15min":1.389},"min":{"1min":1.129,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.681,"5min":1.9390000000000001,"15min":1.95},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3180000000000001,"15min":1.3500000000000001},"min":{"1min":1.0329999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.645,"5min":2.1539999999999999,"15min":2.4820000000000002},"last":1.395}]},{"osd":5,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3660000000000001,"15min":1.3879999999999999},"min":{"1min":0.996,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":1.829,"5min":2.0750000000000002,"15min":2.6669999999999998},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.454},"min":{"1min":1.028,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.748,"5min":2.2029999999999998,"15min":2.2040000000000002},"last":1.373}]},{"osd":6,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.4750000000000001,"15min":1.5029999999999999},"min":{"1min":1.1080000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.819,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.321},{"interface":"front","average":{"1min":1.405,"5min":1.4630000000000001,"15min":1.492},"min":{"1min":0.93799999999999994,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.758,"5min":2.262,"15min":2.262},"last":1.8799999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.52,"15min":1.5549999999999999},"min":{"1min":1.2749999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7150000000000001,"5min":2.347,"15min":2.347},"last":1.9370000000000001},{"interface":"front","average":{"1min":1.454,"5min":1.474,"15min":1.5109999999999999},"min":{"1min":1.129,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.9530000000000001,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.4770000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:24:31.026 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T11:24:31.026 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T11:24:31.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.281+0000 7fa5e2061640 1 Processor -- start 2023-10-16T11:24:31.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.282+0000 7fa5e2061640 1 -- start start 2023-10-16T11:24:31.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.283+0000 7fa5e2061640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa5dc159b20 0x7fa5dc06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:31.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.283+0000 7fa5e2061640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa5dc159f00 0x7fa5dc0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:31.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.283+0000 7fa5e2061640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5dc15d8c0 0x7fa5dc0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:31.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.283+0000 7fa5e2061640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5dc06f890 con 0x7fa5dc159f00 2023-10-16T11:24:31.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.283+0000 7fa5e2061640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5dc06c650 con 0x7fa5dc159b20 2023-10-16T11:24:31.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.283+0000 7fa5e2061640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc06c930 con 0x7fa5dc15d8c0 2023-10-16T11:24:31.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.283+0000 7fa5daffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa5dc159f00 0x7fa5dc0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:31.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.284+0000 7fa5daffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa5dc159f00 0x7fa5dc0b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56298/0 (socket says 172.21.15.84:56298) 2023-10-16T11:24:31.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.284+0000 7fa5daffd640 1 -- 172.21.15.84:0/1011418789 learned_addr learned my addr 172.21.15.84:0/1011418789 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:31.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.284+0000 7fa5db7fe640 1 --2- 172.21.15.84:0/1011418789 >> v2:172.21.15.157:3300/0 conn(0x7fa5dc159b20 0x7fa5dc06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:31.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.284+0000 7fa5dbfff640 1 --2- 172.21.15.84:0/1011418789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5dc15d8c0 0x7fa5dc0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:31.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.284+0000 7fa5daffd640 1 -- 172.21.15.84:0/1011418789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5dc15d8c0 msgr2=0x7fa5dc0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:31.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.284+0000 7fa5daffd640 1 --2- 172.21.15.84:0/1011418789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5dc15d8c0 0x7fa5dc0b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.284+0000 7fa5daffd640 1 -- 172.21.15.84:0/1011418789 >> v2:172.21.15.157:3300/0 conn(0x7fa5dc159b20 msgr2=0x7fa5dc06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:31.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.284+0000 7fa5daffd640 1 --2- 172.21.15.84:0/1011418789 >> v2:172.21.15.157:3300/0 conn(0x7fa5dc159b20 0x7fa5dc06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.284+0000 7fa5daffd640 1 -- 172.21.15.84:0/1011418789 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc06cc10 con 0x7fa5dc159f00 2023-10-16T11:24:31.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.284+0000 7fa5dbfff640 1 --2- 172.21.15.84:0/1011418789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5dc15d8c0 0x7fa5dc0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:31.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.284+0000 7fa5db7fe640 1 --2- 172.21.15.84:0/1011418789 >> v2:172.21.15.157:3300/0 conn(0x7fa5dc159b20 0x7fa5dc06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:31.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.285+0000 7fa5daffd640 1 --2- 172.21.15.84:0/1011418789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5dc159f00 0x7fa5dc0b0410 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0c9340 tx=0x7fa5cc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2e5e31e99b9282 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:31.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.285+0000 7fa5da7fc640 1 -- 172.21.15.84:0/1011418789 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc02ca80 con 0x7fa5dc159f00 2023-10-16T11:24:31.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.286+0000 7fa5da7fc640 1 -- 172.21.15.84:0/1011418789 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc02cbe0 con 0x7fa5dc159f00 2023-10-16T11:24:31.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.286+0000 7fa5da7fc640 1 -- 172.21.15.84:0/1011418789 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc0343f0 con 0x7fa5dc159f00 2023-10-16T11:24:31.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.286+0000 7fa5e2061640 1 -- 172.21.15.84:0/1011418789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5dc159f00 msgr2=0x7fa5dc0b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:31.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.286+0000 7fa5e2061640 1 --2- 172.21.15.84:0/1011418789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5dc159f00 0x7fa5dc0b0410 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0c9340 tx=0x7fa5cc02bee0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.286+0000 7fa5e2061640 1 -- 172.21.15.84:0/1011418789 shutdown_connections 2023-10-16T11:24:31.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.286+0000 7fa5e2061640 1 --2- 172.21.15.84:0/1011418789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5dc159f00 0x7fa5dc0b0410 unknown :-1 s=CLOSED pgs=2627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.286+0000 7fa5e2061640 1 --2- 172.21.15.84:0/1011418789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5dc15d8c0 0x7fa5dc0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.286+0000 7fa5e2061640 1 --2- 172.21.15.84:0/1011418789 >> v2:172.21.15.157:3300/0 conn(0x7fa5dc159b20 0x7fa5dc06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.286+0000 7fa5e2061640 1 -- 172.21.15.84:0/1011418789 >> 172.21.15.84:0/1011418789 conn(0x7fa5dc0cb410 msgr2=0x7fa5dc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:31.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.287+0000 7fa5e2061640 1 -- 172.21.15.84:0/1011418789 shutdown_connections 2023-10-16T11:24:31.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.287+0000 7fa5e2061640 1 -- 172.21.15.84:0/1011418789 wait complete. 2023-10-16T11:24:31.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.289+0000 7fa5e2061640 1 Processor -- start 2023-10-16T11:24:31.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.289+0000 7fa5e2061640 1 -- start start 2023-10-16T11:24:31.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.290+0000 7fa5e2061640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5dc159b20 0x7fa5dc0ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:31.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.290+0000 7fa5e2061640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5dc159f00 0x7fa5dc152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:31.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.290+0000 7fa5e2061640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5dc15d8c0 0x7fa5dc148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:31.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.290+0000 7fa5e2061640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc0af700 con 0x7fa5dc159b20 2023-10-16T11:24:31.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.290+0000 7fa5e2061640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc152550 con 0x7fa5dc15d8c0 2023-10-16T11:24:31.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.290+0000 7fa5e2061640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc0aea80 con 0x7fa5dc159f00 2023-10-16T11:24:31.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.290+0000 7fa5db7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5dc159b20 0x7fa5dc0ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:31.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.290+0000 7fa5db7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5dc159b20 0x7fa5dc0ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56312/0 (socket says 172.21.15.84:56312) 2023-10-16T11:24:31.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.290+0000 7fa5db7fe640 1 -- 172.21.15.84:0/2916241615 learned_addr learned my addr 172.21.15.84:0/2916241615 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:31.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.291+0000 7fa5db7fe640 1 -- 172.21.15.84:0/2916241615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5dc159f00 msgr2=0x7fa5dc152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:31.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.291+0000 7fa5db7fe640 1 --2- 172.21.15.84:0/2916241615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5dc159f00 0x7fa5dc152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.291+0000 7fa5dbfff640 1 --2- 172.21.15.84:0/2916241615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5dc15d8c0 0x7fa5dc148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:31.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.291+0000 7fa5db7fe640 1 -- 172.21.15.84:0/2916241615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5dc15d8c0 msgr2=0x7fa5dc148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:31.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.291+0000 7fa5db7fe640 1 --2- 172.21.15.84:0/2916241615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5dc15d8c0 0x7fa5dc148d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.291+0000 7fa5db7fe640 1 -- 172.21.15.84:0/2916241615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc009b30 con 0x7fa5dc159b20 2023-10-16T11:24:31.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.291+0000 7fa5db7fe640 1 --2- 172.21.15.84:0/2916241615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5dc159b20 0x7fa5dc0ae3f0 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400c7f0 tx=0x7fa5c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:31.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.291+0000 7fa5d3fff640 1 -- 172.21.15.84:0/2916241615 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4007a30 con 0x7fa5dc159b20 2023-10-16T11:24:31.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.292+0000 7fa5d3fff640 1 -- 172.21.15.84:0/2916241615 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c4007b90 con 0x7fa5dc159b20 2023-10-16T11:24:31.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.292+0000 7fa5d3fff640 1 -- 172.21.15.84:0/2916241615 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c400ce40 con 0x7fa5dc159b20 2023-10-16T11:24:31.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.292+0000 7fa5e2061640 1 -- 172.21.15.84:0/2916241615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5dc0aed60 con 0x7fa5dc159b20 2023-10-16T11:24:31.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.292+0000 7fa5e2061640 1 -- 172.21.15.84:0/2916241615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5dc0af320 con 0x7fa5dc159b20 2023-10-16T11:24:31.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.293+0000 7fa5e2061640 1 -- 172.21.15.84:0/2916241615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5dc0af840 con 0x7fa5dc159b20 2023-10-16T11:24:31.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.294+0000 7fa5d3fff640 1 -- 172.21.15.84:0/2916241615 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa5c40113f0 con 0x7fa5dc159b20 2023-10-16T11:24:31.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.295+0000 7fa5d3fff640 1 --2- 172.21.15.84:0/2916241615 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa5bc06afc0 0x7fa5bc06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:31.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.295+0000 7fa5d3fff640 1 -- 172.21.15.84:0/2916241615 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa5c4089a20 con 0x7fa5dc159b20 2023-10-16T11:24:31.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.295+0000 7fa5daffd640 1 --2- 172.21.15.84:0/2916241615 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa5bc06afc0 0x7fa5bc06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:31.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.296+0000 7fa5daffd640 1 --2- 172.21.15.84:0/2916241615 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa5bc06afc0 0x7fa5bc06d480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc035000 tx=0x7fa5cc02e000 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:31.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.299+0000 7fa5d3fff640 1 -- 172.21.15.84:0/2916241615 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa5c4054880 con 0x7fa5dc159b20 2023-10-16T11:24:31.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.554+0000 7fa5e2061640 1 -- 172.21.15.84:0/2916241615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa5dc04e160 con 0x7fa5dc159b20 2023-10-16T11:24:31.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.559+0000 7fa5d3fff640 1 -- 172.21.15.84:0/2916241615 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v55) v1 ==== 190+0+147 (secure 0 0 0) 0x7fa5c401f0e0 con 0x7fa5dc159b20 2023-10-16T11:24:31.561 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2023-10-16T11:24:31.562 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ== 2023-10-16T11:24:31.562 INFO:teuthology.orchestra.run.smithi084.stdout: caps mds = "allow" 2023-10-16T11:24:31.562 INFO:teuthology.orchestra.run.smithi084.stdout: caps mon = "allow r" 2023-10-16T11:24:31.562 INFO:teuthology.orchestra.run.smithi084.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T11:24:31.563 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-16T11:24:31.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.562+0000 7fa5e2061640 1 -- 172.21.15.84:0/2916241615 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa5bc06afc0 msgr2=0x7fa5bc06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:31.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.562+0000 7fa5e2061640 1 --2- 172.21.15.84:0/2916241615 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa5bc06afc0 0x7fa5bc06d480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc035000 tx=0x7fa5cc02e000 comp rx=0 tx=0).stop 2023-10-16T11:24:31.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.562+0000 7fa5e2061640 1 -- 172.21.15.84:0/2916241615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5dc159b20 msgr2=0x7fa5dc0ae3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:31.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.562+0000 7fa5e2061640 1 --2- 172.21.15.84:0/2916241615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5dc159b20 0x7fa5dc0ae3f0 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400c7f0 tx=0x7fa5c400cbf0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.563+0000 7fa5e2061640 1 -- 172.21.15.84:0/2916241615 shutdown_connections 2023-10-16T11:24:31.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.563+0000 7fa5e2061640 1 --2- 172.21.15.84:0/2916241615 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa5bc06afc0 0x7fa5bc06d480 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.563+0000 7fa5e2061640 1 --2- 172.21.15.84:0/2916241615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5dc15d8c0 0x7fa5dc148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.563+0000 7fa5e2061640 1 --2- 172.21.15.84:0/2916241615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5dc159f00 0x7fa5dc152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.563+0000 7fa5e2061640 1 --2- 172.21.15.84:0/2916241615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5dc159b20 0x7fa5dc0ae3f0 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.563+0000 7fa5e2061640 1 -- 172.21.15.84:0/2916241615 >> 172.21.15.84:0/2916241615 conn(0x7fa5dc0cb410 msgr2=0x7fa5dc156660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:31.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.563+0000 7fa5e2061640 1 -- 172.21.15.84:0/2916241615 shutdown_connections 2023-10-16T11:24:31.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.563+0000 7fa5e2061640 1 -- 172.21.15.84:0/2916241615 wait complete. 2023-10-16T11:24:31.583 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:24:31.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.788+0000 7f31f5e15640 1 Processor -- start 2023-10-16T11:24:31.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.788+0000 7f31f5e15640 1 -- start start 2023-10-16T11:24:31.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.789+0000 7f31f5e15640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31f015ae80 0x7f31f015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:31.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.789+0000 7f31f5e15640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f00ce390 0x7f31f0070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:31.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.789+0000 7f31f5e15640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f015ed60 con 0x7f31f015aaa0 2023-10-16T11:24:31.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.789+0000 7f31f5e15640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f006b810 con 0x7f31f015ae80 2023-10-16T11:24:31.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.789+0000 7f31f5e15640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00707e0 con 0x7f31f00ce390 2023-10-16T11:24:31.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.790+0000 7f31ef7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f31f015aaa0 0x7f31f00cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:59522/0 (socket says 172.21.15.84:59522) 2023-10-16T11:24:31.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.790+0000 7f31ef7fe640 1 -- 172.21.15.84:0/2123518454 learned_addr learned my addr 172.21.15.84:0/2123518454 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:31.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.790+0000 7f31ee7fc640 1 -- 172.21.15.84:0/2123518454 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3888277828 0 0) 0x7f31f015ed60 con 0x7f31f015aaa0 2023-10-16T11:24:31.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.790+0000 7f31ee7fc640 1 -- 172.21.15.84:0/2123518454 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31d8003520 con 0x7f31f015aaa0 2023-10-16T11:24:31.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.790+0000 7f31effff640 1 --2- 172.21.15.84:0/2123518454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f00ce390 0x7f31f0070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:31.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.790+0000 7f31eeffd640 1 --2- 172.21.15.84:0/2123518454 >> v2:172.21.15.157:3300/0 conn(0x7f31f015ae80 0x7f31f015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:31.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.790+0000 7f31ee7fc640 1 -- 172.21.15.84:0/2123518454 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1668547778 0 0) 0x7f31d8003520 con 0x7f31f015aaa0 2023-10-16T11:24:31.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.790+0000 7f31ee7fc640 1 -- 172.21.15.84:0/2123518454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f00ce390 msgr2=0x7f31f0070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.790+0000 7f31ee7fc640 1 --2- 172.21.15.84:0/2123518454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f00ce390 0x7f31f0070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.790+0000 7f31ee7fc640 1 -- 172.21.15.84:0/2123518454 >> v2:172.21.15.157:3300/0 conn(0x7f31f015ae80 msgr2=0x7f31f015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.790+0000 7f31ee7fc640 1 --2- 172.21.15.84:0/2123518454 >> v2:172.21.15.157:3300/0 conn(0x7f31f015ae80 0x7f31f015e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.790+0000 7f31ee7fc640 1 -- 172.21.15.84:0/2123518454 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f015ef90 con 0x7f31f015aaa0 2023-10-16T11:24:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.790+0000 7f31ee7fc640 1 -- 172.21.15.84:0/2123518454 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f31e0001690 con 0x7f31f015aaa0 2023-10-16T11:24:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.791+0000 7f31ee7fc640 1 -- 172.21.15.84:0/2123518454 >> v1:172.21.15.84:6789/0 conn(0x7f31f015aaa0 legacy=0x7f31f00cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.791+0000 7f31ee7fc640 1 --2- 172.21.15.84:0/2123518454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31d8003970 0x7f31d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.791+0000 7f31ee7fc640 1 --2- 172.21.15.84:0/2123518454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f00ce390 0x7f31d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.791+0000 7f31ee7fc640 1 --2- 172.21.15.84:0/2123518454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d8004d00 0x7f31d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.791+0000 7f31ee7fc640 1 -- 172.21.15.84:0/2123518454 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f006b810 con 0x7f31d8004d00 2023-10-16T11:24:31.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.791+0000 7f31ee7fc640 1 -- 172.21.15.84:0/2123518454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00707e0 con 0x7f31d8003970 2023-10-16T11:24:31.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.791+0000 7f31ee7fc640 1 -- 172.21.15.84:0/2123518454 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d800ba30 con 0x7f31f00ce390 2023-10-16T11:24:31.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.791+0000 7f31effff640 1 --2- 172.21.15.84:0/2123518454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d8004d00 0x7f31d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:31.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.791+0000 7f31ef7fe640 1 --2- 172.21.15.84:0/2123518454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31d8003970 0x7f31d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:31.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.791+0000 7f31eeffd640 1 --2- 172.21.15.84:0/2123518454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f00ce390 0x7f31d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:31.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.791+0000 7f31effff640 1 -- 172.21.15.84:0/2123518454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f00ce390 msgr2=0x7f31d80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:31.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.791+0000 7f31effff640 1 --2- 172.21.15.84:0/2123518454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f00ce390 0x7f31d80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.791+0000 7f31effff640 1 -- 172.21.15.84:0/2123518454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31d8003970 msgr2=0x7f31d8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:31.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.791+0000 7f31effff640 1 --2- 172.21.15.84:0/2123518454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31d8003970 0x7f31d8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.791+0000 7f31effff640 1 -- 172.21.15.84:0/2123518454 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d8003520 con 0x7f31d8004d00 2023-10-16T11:24:31.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.791+0000 7f31ef7fe640 1 --2- 172.21.15.84:0/2123518454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31d8003970 0x7f31d8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:31.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.792+0000 7f31effff640 1 --2- 172.21.15.84:0/2123518454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d8004d00 0x7f31d8005180 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f31e8007390 tx=0x7f31d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c73a90f68f43901 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:31.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.793+0000 7f31ee7fc640 1 -- 172.21.15.84:0/2123518454 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e802ccb0 con 0x7f31d8004d00 2023-10-16T11:24:31.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.793+0000 7f31ee7fc640 1 -- 172.21.15.84:0/2123518454 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e802ce10 con 0x7f31d8004d00 2023-10-16T11:24:31.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.793+0000 7f31ee7fc640 1 -- 172.21.15.84:0/2123518454 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e8034660 con 0x7f31d8004d00 2023-10-16T11:24:31.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.793+0000 7f31f5e15640 1 -- 172.21.15.84:0/2123518454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d8004d00 msgr2=0x7f31d8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:31.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.793+0000 7f31f5e15640 1 --2- 172.21.15.84:0/2123518454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d8004d00 0x7f31d8005180 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f31e8007390 tx=0x7f31d80057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.793+0000 7f31effff640 1 -- 172.21.15.84:0/2123518454 reap_dead start 2023-10-16T11:24:31.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.793+0000 7f31f5e15640 1 -- 172.21.15.84:0/2123518454 shutdown_connections 2023-10-16T11:24:31.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.793+0000 7f31f5e15640 1 -- 172.21.15.84:0/2123518454 >> 172.21.15.84:0/2123518454 conn(0x7f31f00cb410 msgr2=0x7f31f00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:31.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.793+0000 7f31f5e15640 1 -- 172.21.15.84:0/2123518454 shutdown_connections 2023-10-16T11:24:31.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.793+0000 7f31f5e15640 1 -- 172.21.15.84:0/2123518454 wait complete. 2023-10-16T11:24:31.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.795+0000 7f31f5e15640 1 Processor -- start 2023-10-16T11:24:31.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.795+0000 7f31f5e15640 1 -- start start 2023-10-16T11:24:31.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.796+0000 7f31f5e15640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d8004d00 0x7f31f00c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:31.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.796+0000 7f31f5e15640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f00aec10 0x7f31f0150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:31.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.796+0000 7f31f5e15640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f00b0290 0x7f31f00b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:31.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.796+0000 7f31f5e15640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00b0c50 con 0x7f31d8004d00 2023-10-16T11:24:31.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.796+0000 7f31f5e15640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f0150cd0 con 0x7f31f00aec10 2023-10-16T11:24:31.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.796+0000 7f31f5e15640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f015cb90 con 0x7f31f00b0290 2023-10-16T11:24:31.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.796+0000 7f31ef7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d8004d00 0x7f31f00c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:31.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.796+0000 7f31ef7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d8004d00 0x7f31f00c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56320/0 (socket says 172.21.15.84:56320) 2023-10-16T11:24:31.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.796+0000 7f31ef7fe640 1 -- 172.21.15.84:0/2718297842 learned_addr learned my addr 172.21.15.84:0/2718297842 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:31.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.797+0000 7f31ef7fe640 1 -- 172.21.15.84:0/2718297842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f00b0290 msgr2=0x7f31f00b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:31.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.797+0000 7f31ef7fe640 1 --2- 172.21.15.84:0/2718297842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f00b0290 0x7f31f00b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.797+0000 7f31ef7fe640 1 -- 172.21.15.84:0/2718297842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f00aec10 msgr2=0x7f31f0150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:31.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.797+0000 7f31ef7fe640 1 --2- 172.21.15.84:0/2718297842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f00aec10 0x7f31f0150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:31.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.797+0000 7f31ef7fe640 1 -- 172.21.15.84:0/2718297842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f00afc30 con 0x7f31d8004d00 2023-10-16T11:24:31.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.797+0000 7f31ef7fe640 1 --2- 172.21.15.84:0/2718297842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d8004d00 0x7f31f00c4460 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f31e000d8e0 tx=0x7f31e000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:31.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.797+0000 7f31d7fff640 1 -- 172.21.15.84:0/2718297842 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e00074e0 con 0x7f31d8004d00 2023-10-16T11:24:31.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.797+0000 7f31d7fff640 1 -- 172.21.15.84:0/2718297842 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e0007640 con 0x7f31d8004d00 2023-10-16T11:24:31.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.797+0000 7f31d7fff640 1 -- 172.21.15.84:0/2718297842 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0004a00 con 0x7f31d8004d00 2023-10-16T11:24:31.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.797+0000 7f31f5e15640 1 -- 172.21.15.84:0/2718297842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f00af390 con 0x7f31d8004d00 2023-10-16T11:24:31.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.798+0000 7f31f5e15640 1 -- 172.21.15.84:0/2718297842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f00af950 con 0x7f31d8004d00 2023-10-16T11:24:31.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.799+0000 7f31f5e15640 1 -- 172.21.15.84:0/2718297842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f00b3e00 con 0x7f31d8004d00 2023-10-16T11:24:31.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.800+0000 7f31d7fff640 1 -- 172.21.15.84:0/2718297842 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f31e0004b60 con 0x7f31d8004d00 2023-10-16T11:24:31.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.801+0000 7f31d7fff640 1 --2- 172.21.15.84:0/2718297842 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f31dc06af50 0x7f31dc06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:31.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.801+0000 7f31d7fff640 1 -- 172.21.15.84:0/2718297842 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f31e0089370 con 0x7f31d8004d00 2023-10-16T11:24:31.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.801+0000 7f31eeffd640 1 --2- 172.21.15.84:0/2718297842 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f31dc06af50 0x7f31dc06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:31.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.802+0000 7f31eeffd640 1 --2- 172.21.15.84:0/2718297842 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f31dc06af50 0x7f31dc06d410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f31e4006e80 tx=0x7f31e4006e10 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:31.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:31.805+0000 7f31d7fff640 1 -- 172.21.15.84:0/2718297842 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f31e0054140 con 0x7f31d8004d00 2023-10-16T11:24:32.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.038+0000 7f31f5e15640 1 -- 172.21.15.84:0/2718297842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31f004e160 con 0x7f31d8004d00 2023-10-16T11:24:32.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.040+0000 7f31d7fff640 1 -- 172.21.15.84:0/2718297842 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4929 (secure 0 0 0) 0x7f31e0057760 con 0x7f31d8004d00 2023-10-16T11:24:32.042 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:32.042 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":114,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3106898227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3106898227},{"type":"v1","addr":"172.21.15.84:6837","nonce":3106898227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":7471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2296953749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2296953749},{"type":"v1","addr":"172.21.15.157:6834","nonce":2296953749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":7474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3992377882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3992377882},{"type":"v1","addr":"172.21.15.157:6835","nonce":3992377882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:24:25.531691+0000","modified":"2023-10-16T11:24:28.574116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11307},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11307":{"gid":11307,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/551895595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":551895595},{"type":"v1","addr":"172.21.15.84:6836","nonce":551895595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-10-16T11:24:32.042 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 114 2023-10-16T11:24:32.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.042+0000 7f31f5e15640 1 -- 172.21.15.84:0/2718297842 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f31dc06af50 msgr2=0x7f31dc06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:32.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.043+0000 7f31f5e15640 1 --2- 172.21.15.84:0/2718297842 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f31dc06af50 0x7f31dc06d410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f31e4006e80 tx=0x7f31e4006e10 comp rx=0 tx=0).stop 2023-10-16T11:24:32.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.043+0000 7f31f5e15640 1 -- 172.21.15.84:0/2718297842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d8004d00 msgr2=0x7f31f00c4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:32.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.043+0000 7f31f5e15640 1 --2- 172.21.15.84:0/2718297842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d8004d00 0x7f31f00c4460 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f31e000d8e0 tx=0x7f31e000dda0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.043+0000 7f31f5e15640 1 -- 172.21.15.84:0/2718297842 shutdown_connections 2023-10-16T11:24:32.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.043+0000 7f31f5e15640 1 --2- 172.21.15.84:0/2718297842 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f31dc06af50 0x7f31dc06d410 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.043+0000 7f31f5e15640 1 --2- 172.21.15.84:0/2718297842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31f00b0290 0x7f31f00b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.043+0000 7f31f5e15640 1 --2- 172.21.15.84:0/2718297842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f00aec10 0x7f31f0150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.043+0000 7f31f5e15640 1 --2- 172.21.15.84:0/2718297842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d8004d00 0x7f31f00c4460 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.043+0000 7f31f5e15640 1 -- 172.21.15.84:0/2718297842 >> 172.21.15.84:0/2718297842 conn(0x7f31f00cb410 msgr2=0x7f31f006ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:32.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.043+0000 7f31f5e15640 1 -- 172.21.15.84:0/2718297842 shutdown_connections 2023-10-16T11:24:32.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.043+0000 7f31f5e15640 1 -- 172.21.15.84:0/2718297842 wait complete. 2023-10-16T11:24:32.465 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 114, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T11:24:25.531691+0000', 'modified': '2023-10-16T11:24:28.574116+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11307}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11307': {'gid': 11307, 'name': 'a', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.84:6836/551895595', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6834', 'nonce': 551895595}, {'type': 'v1', 'addr': '172.21.15.84:6836', 'nonce': 551895595}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T11:24:32.466 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T11:24:32.466 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-16T11:24:32.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.695+0000 7f352915a640 1 Processor -- start 2023-10-16T11:24:32.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.695+0000 7f352915a640 1 -- start start 2023-10-16T11:24:32.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.697+0000 7f352915a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f352406a4b0 0x7f35240ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.697+0000 7f352915a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35240cb030 con 0x7f35240c9ca0 2023-10-16T11:24:32.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.697+0000 7f352915a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35240c91c0 con 0x7f352415ad70 2023-10-16T11:24:32.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.697+0000 7f352915a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35240c94a0 con 0x7f352406a4b0 2023-10-16T11:24:32.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.697+0000 7f352359e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f35240c9ca0 0x7f35240ad4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:59526/0 (socket says 172.21.15.84:59526) 2023-10-16T11:24:32.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.697+0000 7f352359e640 1 -- 172.21.15.84:0/1344109617 learned_addr learned my addr 172.21.15.84:0/1344109617 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:32.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.697+0000 7f3521d9b640 1 -- 172.21.15.84:0/1344109617 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3534510989 0 0) 0x7f35240cb030 con 0x7f35240c9ca0 2023-10-16T11:24:32.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.697+0000 7f352259c640 1 --2- 172.21.15.84:0/1344109617 >> v2:172.21.15.157:3300/0 conn(0x7f352406a4b0 0x7f35240ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:32.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.697+0000 7f3521d9b640 1 -- 172.21.15.84:0/1344109617 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f350c003520 con 0x7f35240c9ca0 2023-10-16T11:24:32.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.698+0000 7f3521d9b640 1 -- 172.21.15.84:0/1344109617 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3990701038 0 0) 0x7f350c003520 con 0x7f35240c9ca0 2023-10-16T11:24:32.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.698+0000 7f3521d9b640 1 -- 172.21.15.84:0/1344109617 >> v2:172.21.15.157:3300/0 conn(0x7f352406a4b0 msgr2=0x7f35240ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:32.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.698+0000 7f3521d9b640 1 --2- 172.21.15.84:0/1344109617 >> v2:172.21.15.157:3300/0 conn(0x7f352406a4b0 0x7f35240ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.698+0000 7f3521d9b640 1 -- 172.21.15.84:0/1344109617 >> v1:172.21.15.157:6789/0 conn(0x7f352415ad70 legacy=0x7f3524069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:32.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.698+0000 7f3521d9b640 1 -- 172.21.15.84:0/1344109617 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35240c9780 con 0x7f35240c9ca0 2023-10-16T11:24:32.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.698+0000 7f3521d9b640 1 -- 172.21.15.84:0/1344109617 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f351c0019f0 con 0x7f35240c9ca0 2023-10-16T11:24:32.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.698+0000 7f3521d9b640 1 -- 172.21.15.84:0/1344109617 >> v1:172.21.15.84:6789/0 conn(0x7f35240c9ca0 legacy=0x7f35240ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:32.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.698+0000 7f3521d9b640 1 --2- 172.21.15.84:0/1344109617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f350c0039c0 0x7f350c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.698+0000 7f3521d9b640 1 --2- 172.21.15.84:0/1344109617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f350c0047c0 0x7f350c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.698+0000 7f3521d9b640 1 --2- 172.21.15.84:0/1344109617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f350c005180 0x7f350c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.698+0000 7f3521d9b640 1 -- 172.21.15.84:0/1344109617 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35240c94a0 con 0x7f350c005180 2023-10-16T11:24:32.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.698+0000 7f3521d9b640 1 -- 172.21.15.84:0/1344109617 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c00c080 con 0x7f350c0047c0 2023-10-16T11:24:32.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.698+0000 7f3521d9b640 1 -- 172.21.15.84:0/1344109617 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350c00c240 con 0x7f350c0039c0 2023-10-16T11:24:32.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.698+0000 7f352359e640 1 --2- 172.21.15.84:0/1344109617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f350c005180 0x7f350c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:32.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.698+0000 7f352259c640 1 --2- 172.21.15.84:0/1344109617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f350c0047c0 0x7f350c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:32.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.699+0000 7f3522d9d640 1 --2- 172.21.15.84:0/1344109617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f350c0039c0 0x7f350c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:32.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.699+0000 7f352359e640 1 -- 172.21.15.84:0/1344109617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f350c0039c0 msgr2=0x7f350c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:32.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.699+0000 7f352359e640 1 --2- 172.21.15.84:0/1344109617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f350c0039c0 0x7f350c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.699+0000 7f352359e640 1 -- 172.21.15.84:0/1344109617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f350c0047c0 msgr2=0x7f350c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:32.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.699+0000 7f352359e640 1 --2- 172.21.15.84:0/1344109617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f350c0047c0 0x7f350c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.699+0000 7f352359e640 1 -- 172.21.15.84:0/1344109617 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350c003520 con 0x7f350c005180 2023-10-16T11:24:32.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.700+0000 7f352359e640 1 -- 172.21.15.84:0/1344109617 reap_dead start 2023-10-16T11:24:32.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.700+0000 7f352359e640 1 --2- 172.21.15.84:0/1344109617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f350c005180 0x7f350c00b980 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f351c00a760 tx=0x7f350c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72433d6f700ca32 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:32.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.700+0000 7f3521d9b640 1 -- 172.21.15.84:0/1344109617 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c034630 con 0x7f350c005180 2023-10-16T11:24:32.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.700+0000 7f3521d9b640 1 -- 172.21.15.84:0/1344109617 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351c02f070 con 0x7f350c005180 2023-10-16T11:24:32.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.700+0000 7f3521d9b640 1 -- 172.21.15.84:0/1344109617 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c034a10 con 0x7f350c005180 2023-10-16T11:24:32.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.701+0000 7f352915a640 1 -- 172.21.15.84:0/1344109617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f350c005180 msgr2=0x7f350c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:32.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.701+0000 7f352915a640 1 --2- 172.21.15.84:0/1344109617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f350c005180 0x7f350c00b980 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f351c00a760 tx=0x7f350c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.701+0000 7f352915a640 1 -- 172.21.15.84:0/1344109617 shutdown_connections 2023-10-16T11:24:32.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.701+0000 7f352915a640 1 --2- 172.21.15.84:0/1344109617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f350c005180 0x7f350c00b980 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.701+0000 7f352915a640 1 -- 172.21.15.84:0/1344109617 >> 172.21.15.84:0/1344109617 conn(0x7f35240ce9b0 msgr2=0x7f35240cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:32.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.701+0000 7f352915a640 1 -- 172.21.15.84:0/1344109617 shutdown_connections 2023-10-16T11:24:32.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.701+0000 7f352915a640 1 -- 172.21.15.84:0/1344109617 wait complete. 2023-10-16T11:24:32.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.703+0000 7f352915a640 1 Processor -- start 2023-10-16T11:24:32.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.703+0000 7f352915a640 1 -- start start 2023-10-16T11:24:32.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.704+0000 7f352915a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f350c005180 0x7f3524158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.704+0000 7f352915a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35240ca6f0 0x7f35240aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.704+0000 7f352915a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3524159d10 0x7f35240af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.704+0000 7f352915a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35240b1a60 con 0x7f35240ca6f0 2023-10-16T11:24:32.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.704+0000 7f352915a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3524159530 con 0x7f3524159d10 2023-10-16T11:24:32.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.704+0000 7f352915a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35240ae8c0 con 0x7f350c005180 2023-10-16T11:24:32.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.704+0000 7f352259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35240ca6f0 0x7f35240aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:32.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.704+0000 7f352259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35240ca6f0 0x7f35240aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56344/0 (socket says 172.21.15.84:56344) 2023-10-16T11:24:32.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.704+0000 7f352259c640 1 -- 172.21.15.84:0/3666864990 learned_addr learned my addr 172.21.15.84:0/3666864990 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.704+0000 7f3522d9d640 1 --2- 172.21.15.84:0/3666864990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f350c005180 0x7f3524158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.704+0000 7f352359e640 1 --2- 172.21.15.84:0/3666864990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3524159d10 0x7f35240af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.704+0000 7f352259c640 1 -- 172.21.15.84:0/3666864990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f350c005180 msgr2=0x7f3524158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.704+0000 7f352259c640 1 --2- 172.21.15.84:0/3666864990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f350c005180 0x7f3524158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.704+0000 7f352259c640 1 -- 172.21.15.84:0/3666864990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3524159d10 msgr2=0x7f35240af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:32.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.704+0000 7f352259c640 1 --2- 172.21.15.84:0/3666864990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3524159d10 0x7f35240af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.705+0000 7f352259c640 1 -- 172.21.15.84:0/3666864990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35240ae020 con 0x7f35240ca6f0 2023-10-16T11:24:32.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.705+0000 7f352259c640 1 --2- 172.21.15.84:0/3666864990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35240ca6f0 0x7f35240aece0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f3518011540 tx=0x7f3518011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:32.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.705+0000 7f350b7fe640 1 -- 172.21.15.84:0/3666864990 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518015930 con 0x7f35240ca6f0 2023-10-16T11:24:32.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.705+0000 7f350b7fe640 1 -- 172.21.15.84:0/3666864990 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3518015a90 con 0x7f35240ca6f0 2023-10-16T11:24:32.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.705+0000 7f350b7fe640 1 -- 172.21.15.84:0/3666864990 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518015e60 con 0x7f35240ca6f0 2023-10-16T11:24:32.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.705+0000 7f352915a640 1 -- 172.21.15.84:0/3666864990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35240ae300 con 0x7f35240ca6f0 2023-10-16T11:24:32.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.705+0000 7f352915a640 1 -- 172.21.15.84:0/3666864990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3524159730 con 0x7f35240ca6f0 2023-10-16T11:24:32.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.708+0000 7f350b7fe640 1 -- 172.21.15.84:0/3666864990 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f351800f460 con 0x7f35240ca6f0 2023-10-16T11:24:32.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.709+0000 7f350b7fe640 1 --2- 172.21.15.84:0/3666864990 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f351006af90 0x7f351006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.709+0000 7f350b7fe640 1 -- 172.21.15.84:0/3666864990 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f351808a0a0 con 0x7f35240ca6f0 2023-10-16T11:24:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.709+0000 7f3522d9d640 1 --2- 172.21.15.84:0/3666864990 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f351006af90 0x7f351006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.710+0000 7f352bdc7740 1 Processor -- start 2023-10-16T11:24:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.710+0000 7f352bdc7740 1 -- start start 2023-10-16T11:24:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.710+0000 7f3522d9d640 1 --2- 172.21.15.84:0/3666864990 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f351006af90 0x7f351006d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3514001690 tx=0x7f3514009040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:32.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.711+0000 7f352bdc7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 0x563ab07d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.711+0000 7f352bdc7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab07ff360 0x563ab0801d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.711+0000 7f352bdc7740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab07d23c0 0x563ab0802250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.711+0000 7f352bdc7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab06f27a0 con 0x563ab07fef80 2023-10-16T11:24:32.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.711+0000 7f352bdc7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab0744740 con 0x563ab07ff360 2023-10-16T11:24:32.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.711+0000 7f352bdc7740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ab0741e30 con 0x563ab07d23c0 2023-10-16T11:24:32.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.711+0000 7f3522d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 0x563ab07d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:32.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.711+0000 7f3522d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 0x563ab07d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56346/0 (socket says 172.21.15.84:56346) 2023-10-16T11:24:32.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.711+0000 7f3522d9d640 1 -- 172.21.15.84:0/3671533696 learned_addr learned my addr 172.21.15.84:0/3671533696 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:32.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.712+0000 7f3522d9d640 1 -- 172.21.15.84:0/3671533696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab07d23c0 msgr2=0x563ab0802250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:32.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.712+0000 7f3522d9d640 1 --2- 172.21.15.84:0/3671533696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab07d23c0 0x563ab0802250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.712+0000 7f3522d9d640 1 -- 172.21.15.84:0/3671533696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab07ff360 msgr2=0x563ab0801d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:32.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.712+0000 7f3522d9d640 1 --2- 172.21.15.84:0/3671533696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab07ff360 0x563ab0801d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.712+0000 7f3522d9d640 1 -- 172.21.15.84:0/3671533696 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ab0701910 con 0x563ab07fef80 2023-10-16T11:24:32.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.712+0000 7f3522d9d640 1 --2- 172.21.15.84:0/3671533696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 0x563ab07d1e80 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f3514005100 tx=0x7f3514012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7e69d0aab3ad18 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:32.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.712+0000 7f35097fa640 1 -- 172.21.15.84:0/3671533696 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35140035f0 con 0x563ab07fef80 2023-10-16T11:24:32.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.712+0000 7f35097fa640 1 -- 172.21.15.84:0/3671533696 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351400d050 con 0x563ab07fef80 2023-10-16T11:24:32.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.712+0000 7f35097fa640 1 -- 172.21.15.84:0/3671533696 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3514003990 con 0x563ab07fef80 2023-10-16T11:24:32.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.713+0000 7f352bdc7740 1 -- 172.21.15.84:0/3671533696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 msgr2=0x563ab07d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:32.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.713+0000 7f352bdc7740 1 --2- 172.21.15.84:0/3671533696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 0x563ab07d1e80 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f3514005100 tx=0x7f3514012000 comp rx=0 tx=0).stop 2023-10-16T11:24:32.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.713+0000 7f352bdc7740 1 -- 172.21.15.84:0/3671533696 shutdown_connections 2023-10-16T11:24:32.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.713+0000 7f352bdc7740 1 --2- 172.21.15.84:0/3671533696 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab07d23c0 0x563ab0802250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.713+0000 7f352bdc7740 1 --2- 172.21.15.84:0/3671533696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab07ff360 0x563ab0801d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.713+0000 7f352bdc7740 1 --2- 172.21.15.84:0/3671533696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 0x563ab07d1e80 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.713+0000 7f352bdc7740 1 -- 172.21.15.84:0/3671533696 >> 172.21.15.84:0/3671533696 conn(0x563ab07d4190 msgr2=0x563ab081cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:32.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.713+0000 7f352bdc7740 1 -- 172.21.15.84:0/3671533696 shutdown_connections 2023-10-16T11:24:32.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.713+0000 7f352bdc7740 1 -- 172.21.15.84:0/3671533696 wait complete. 2023-10-16T11:24:32.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.714+0000 7f352bdc7740 1 -- start start 2023-10-16T11:24:32.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.714+0000 7f352bdc7740 1 Processor -- start 2023-10-16T11:24:32.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.715+0000 7f352bdc7740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab07fef80 0x563ab0817770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.715+0000 7f352bdc7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07ff360 0x563ab082d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.715+0000 7f352bdc7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab0816690 0x563ab0816ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.715+0000 7f352bdc7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab073d720 con 0x563ab07ff360 2023-10-16T11:24:32.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.715+0000 7f352bdc7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab06db970 con 0x563ab0816690 2023-10-16T11:24:32.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.715+0000 7f352bdc7740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ab06db610 con 0x563ab07fef80 2023-10-16T11:24:32.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.715+0000 7f352bdc7740 10 client.0 _ll_register_callbacks cb 0x563ab06fd9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:24:32.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.715+0000 7f352259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07ff360 0x563ab082d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:32.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.715+0000 7f352259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07ff360 0x563ab082d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56352/0 (socket says 172.21.15.84:56352) 2023-10-16T11:24:32.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.715+0000 7f352259c640 1 -- 172.21.15.84:0/1583826462 learned_addr learned my addr 172.21.15.84:0/1583826462 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:32.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.716+0000 7f352259c640 1 -- 172.21.15.84:0/1583826462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab07fef80 msgr2=0x563ab0817770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:32.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.716+0000 7f352259c640 1 --2- 172.21.15.84:0/1583826462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab07fef80 0x563ab0817770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.716+0000 7f352359e640 1 --2- 172.21.15.84:0/1583826462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab0816690 0x563ab0816ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:32.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.716+0000 7f352259c640 1 -- 172.21.15.84:0/1583826462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab0816690 msgr2=0x563ab0816ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:32.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.716+0000 7f352259c640 1 --2- 172.21.15.84:0/1583826462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab0816690 0x563ab0816ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.716+0000 7f352259c640 1 -- 172.21.15.84:0/1583826462 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35240c91c0 con 0x563ab07ff360 2023-10-16T11:24:32.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.716+0000 7f352259c640 1 --2- 172.21.15.84:0/1583826462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07ff360 0x563ab082d4b0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f3518058940 tx=0x7f3518059480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b8dbd595f136e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:32.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.716+0000 7f3501ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T11:24:32.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.716+0000 7f3501ffb640 1 -- 172.21.15.84:0/1583826462 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35180590a0 con 0x563ab07ff360 2023-10-16T11:24:32.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.716+0000 7f3501ffb640 1 -- 172.21.15.84:0/1583826462 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f3518058130 con 0x563ab07ff360 2023-10-16T11:24:32.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.716+0000 7f352bdc7740 1 -- 172.21.15.84:0/1583826462 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ab0764160 con 0x563ab07ff360 2023-10-16T11:24:32.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.716+0000 7f3501ffb640 1 -- 172.21.15.84:0/1583826462 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3518088080 con 0x563ab07ff360 2023-10-16T11:24:32.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.716+0000 7f352bdc7740 10 client.11460 fetch_fsmap learned FSMap version 114 2023-10-16T11:24:32.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.716+0000 7f352bdc7740 1 -- 172.21.15.84:0/1583826462 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) v3 -- 0x563ab073bda0 con 0x563ab07ff360 2023-10-16T11:24:32.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.717+0000 7f3501ffb640 1 -- 172.21.15.84:0/1583826462 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3518074070 con 0x563ab07ff360 2023-10-16T11:24:32.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.717+0000 7f3501ffb640 1 -- 172.21.15.84:0/1583826462 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 114) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3518025070 con 0x563ab07ff360 2023-10-16T11:24:32.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.717+0000 7f352bdc7740 10 client.11460 fetch_fsmap finished waiting for FSMap version 114 2023-10-16T11:24:32.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.717+0000 7f352bdc7740 10 client.11460 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T11:24:32.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.717+0000 7f352bdc7740 20 client.11460 populate_metadata read hostname 'smithi084' 2023-10-16T11:24:32.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.717+0000 7f352bdc7740 1 --2- 172.21.15.84:0/1583826462 >> [v2:172.21.15.84:6834/551895595,v1:172.21.15.84:6836/551895595] conn(0x563ab08311c0 0x563ab082e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.717+0000 7f352bdc7740 4 client.11460 mds_command: new command op to 11307 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T11:24:32.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.717+0000 7f352bdc7740 1 -- 172.21.15.84:0/1583826462 --> [v2:172.21.15.84:6834/551895595,v1:172.21.15.84:6836/551895595] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ab070fcc0 con 0x563ab08311c0 2023-10-16T11:24:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.717+0000 7f3522d9d640 1 --2- 172.21.15.84:0/1583826462 >> [v2:172.21.15.84:6834/551895595,v1:172.21.15.84:6836/551895595] conn(0x563ab08311c0 0x563ab082e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.718+0000 7f3522d9d640 1 --2- 172.21.15.84:0/1583826462 >> [v2:172.21.15.84:6834/551895595,v1:172.21.15.84:6836/551895595] conn(0x563ab08311c0 0x563ab082e710 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5aa36427cfa6dd8 server_cookie=f14860dac87fd9f0 in_seq=0 out_seq=0 2023-10-16T11:24:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.718+0000 7f3501ffb640 10 client.11460 ms_handle_connect on v2:172.21.15.84:6834/551895595 2023-10-16T11:24:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.718+0000 7f3501ffb640 1 -- 172.21.15.84:0/1583826462 <== mds.0 v2:172.21.15.84:6834/551895595 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563ab070fcc0 con 0x563ab08311c0 2023-10-16T11:24:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.718+0000 7f3501ffb640 10 client.11460 handle_command_reply: tid=0 2023-10-16T11:24:32.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.719+0000 7f352bdc7740 1 client.11460 shutdown 2023-10-16T11:24:32.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.719+0000 7f352bdc7740 1 -- 172.21.15.84:0/1583826462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07ff360 msgr2=0x563ab082d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:32.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.719+0000 7f352bdc7740 1 --2- 172.21.15.84:0/1583826462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07ff360 0x563ab082d4b0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f3518058940 tx=0x7f3518059480 comp rx=0 tx=0).stop 2023-10-16T11:24:32.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.719+0000 7f352bdc7740 1 -- 172.21.15.84:0/1583826462 shutdown_connections 2023-10-16T11:24:32.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.719+0000 7f352bdc7740 1 --2- 172.21.15.84:0/1583826462 >> [v2:172.21.15.84:6834/551895595,v1:172.21.15.84:6836/551895595] conn(0x563ab08311c0 0x563ab082e710 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.719+0000 7f352bdc7740 1 --2- 172.21.15.84:0/1583826462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab0816690 0x563ab0816ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.719+0000 7f352bdc7740 1 --2- 172.21.15.84:0/1583826462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07ff360 0x563ab082d4b0 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.719+0000 7f352bdc7740 1 --2- 172.21.15.84:0/1583826462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab07fef80 0x563ab0817770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.719+0000 7f3501ffb640 0 client.11460 ms_handle_reset on v2:172.21.15.84:6834/551895595 2023-10-16T11:24:32.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.719+0000 7f352bdc7740 1 -- 172.21.15.84:0/1583826462 >> 172.21.15.84:0/1583826462 conn(0x563ab07d23c0 msgr2=0x563ab07d5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:32.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.719+0000 7f352bdc7740 1 -- 172.21.15.84:0/1583826462 shutdown_connections 2023-10-16T11:24:32.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.719+0000 7f352bdc7740 1 -- 172.21.15.84:0/1583826462 wait complete. 2023-10-16T11:24:32.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.719+0000 7f352bdc7740 20 client.11460 trim_cache size 0 max 16384 2023-10-16T11:24:32.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.736+0000 7f352bdc7740 1 Processor -- start 2023-10-16T11:24:32.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.736+0000 7f352bdc7740 1 -- start start 2023-10-16T11:24:32.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.736+0000 7f352bdc7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 0x563ab0801cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.736+0000 7f352bdc7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab07ff360 0x563ab0802210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.736+0000 7f352bdc7740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab0816690 0x563ab081cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.736+0000 7f352bdc7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab04a1f20 con 0x563ab07fef80 2023-10-16T11:24:32.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.736+0000 7f352bdc7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab082fdc0 con 0x563ab07ff360 2023-10-16T11:24:32.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.736+0000 7f352bdc7740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ab070a860 con 0x563ab0816690 2023-10-16T11:24:32.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.737+0000 7f3522d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 0x563ab0801cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:32.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.737+0000 7f3522d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 0x563ab0801cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56354/0 (socket says 172.21.15.84:56354) 2023-10-16T11:24:32.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.737+0000 7f3522d9d640 1 -- 172.21.15.84:0/1081100376 learned_addr learned my addr 172.21.15.84:0/1081100376 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:32.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.737+0000 7f3522d9d640 1 -- 172.21.15.84:0/1081100376 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab0816690 msgr2=0x563ab081cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:32.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.737+0000 7f3522d9d640 1 --2- 172.21.15.84:0/1081100376 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab0816690 0x563ab081cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.737+0000 7f3522d9d640 1 -- 172.21.15.84:0/1081100376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab07ff360 msgr2=0x563ab0802210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:32.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.737+0000 7f3522d9d640 1 --2- 172.21.15.84:0/1081100376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab07ff360 0x563ab0802210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.737+0000 7f3522d9d640 1 -- 172.21.15.84:0/1081100376 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ab06dab80 con 0x563ab07fef80 2023-10-16T11:24:32.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.737+0000 7f3522d9d640 1 --2- 172.21.15.84:0/1081100376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 0x563ab0801cd0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f351400f5a0 tx=0x7f3514005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1f8b15634af8c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:32.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.738+0000 7f3509ffb640 1 -- 172.21.15.84:0/1081100376 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35140324a0 con 0x563ab07fef80 2023-10-16T11:24:32.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.738+0000 7f3509ffb640 1 -- 172.21.15.84:0/1081100376 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35140327a0 con 0x563ab07fef80 2023-10-16T11:24:32.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.738+0000 7f3509ffb640 1 -- 172.21.15.84:0/1081100376 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3514032b30 con 0x563ab07fef80 2023-10-16T11:24:32.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.738+0000 7f352bdc7740 1 -- 172.21.15.84:0/1081100376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 msgr2=0x563ab0801cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:32.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.738+0000 7f352bdc7740 1 --2- 172.21.15.84:0/1081100376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 0x563ab0801cd0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f351400f5a0 tx=0x7f3514005000 comp rx=0 tx=0).stop 2023-10-16T11:24:32.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.738+0000 7f352bdc7740 1 -- 172.21.15.84:0/1081100376 shutdown_connections 2023-10-16T11:24:32.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.738+0000 7f352bdc7740 1 --2- 172.21.15.84:0/1081100376 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab0816690 0x563ab081cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.738+0000 7f352bdc7740 1 --2- 172.21.15.84:0/1081100376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab07ff360 0x563ab0802210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.738+0000 7f352bdc7740 1 --2- 172.21.15.84:0/1081100376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 0x563ab0801cd0 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.738+0000 7f352bdc7740 1 -- 172.21.15.84:0/1081100376 >> 172.21.15.84:0/1081100376 conn(0x563ab07d23c0 msgr2=0x563ab082d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:32.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.738+0000 7f352bdc7740 1 -- 172.21.15.84:0/1081100376 shutdown_connections 2023-10-16T11:24:32.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.738+0000 7f352bdc7740 1 -- 172.21.15.84:0/1081100376 wait complete. 2023-10-16T11:24:32.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.739+0000 7f352bdc7740 1 -- start start 2023-10-16T11:24:32.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.739+0000 7f352bdc7740 1 Processor -- start 2023-10-16T11:24:32.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.740+0000 7f352bdc7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 0x563ab0801cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.740+0000 7f352bdc7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab07ff360 0x563ab0802210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.740+0000 7f352bdc7740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab0816690 0x563ab081e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.740+0000 7f352bdc7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab082f930 con 0x563ab07fef80 2023-10-16T11:24:32.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.740+0000 7f352bdc7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab081b480 con 0x563ab07ff360 2023-10-16T11:24:32.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.740+0000 7f352bdc7740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ab082d700 con 0x563ab0816690 2023-10-16T11:24:32.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.740+0000 7f352bdc7740 10 client.0 _ll_register_callbacks cb 0x7f35180046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:24:32.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.740+0000 7f3522d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 0x563ab0801cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:32.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.740+0000 7f3522d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 0x563ab0801cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56358/0 (socket says 172.21.15.84:56358) 2023-10-16T11:24:32.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.740+0000 7f3522d9d640 1 -- 172.21.15.84:0/3481056021 learned_addr learned my addr 172.21.15.84:0/3481056021 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:32.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.741+0000 7f3522d9d640 1 -- 172.21.15.84:0/3481056021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab0816690 msgr2=0x563ab081e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:32.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.741+0000 7f3522d9d640 1 --2- 172.21.15.84:0/3481056021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab0816690 0x563ab081e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.741+0000 7f3522d9d640 1 -- 172.21.15.84:0/3481056021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab07ff360 msgr2=0x563ab0802210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:32.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.741+0000 7f3522d9d640 1 --2- 172.21.15.84:0/3481056021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab07ff360 0x563ab0802210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.741+0000 7f3522d9d640 1 -- 172.21.15.84:0/3481056021 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ab0701910 con 0x563ab07fef80 2023-10-16T11:24:32.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.741+0000 7f3522d9d640 1 --2- 172.21.15.84:0/3481056021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 0x563ab0801cd0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f3514005100 tx=0x7f3514001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736f1a5c389cb2e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:32.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.741+0000 7f3501ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T11:24:32.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.741+0000 7f3501ffb640 1 -- 172.21.15.84:0/3481056021 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3514032910 con 0x563ab07fef80 2023-10-16T11:24:32.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.741+0000 7f3501ffb640 1 -- 172.21.15.84:0/3481056021 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f351401b050 con 0x563ab07fef80 2023-10-16T11:24:32.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.741+0000 7f352bdc7740 1 -- 172.21.15.84:0/3481056021 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f351400d050 con 0x563ab07fef80 2023-10-16T11:24:32.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.742+0000 7f3501ffb640 1 -- 172.21.15.84:0/3481056021 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3514032a70 con 0x563ab07fef80 2023-10-16T11:24:32.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.742+0000 7f352bdc7740 10 client.11466 fetch_fsmap learned FSMap version 114 2023-10-16T11:24:32.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.742+0000 7f352bdc7740 1 -- 172.21.15.84:0/3481056021 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) v3 -- 0x563ab0700170 con 0x563ab07fef80 2023-10-16T11:24:32.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.742+0000 7f3501ffb640 1 -- 172.21.15.84:0/3481056021 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351400d050 con 0x563ab07fef80 2023-10-16T11:24:32.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.742+0000 7f3501ffb640 1 -- 172.21.15.84:0/3481056021 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 114) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3514012070 con 0x563ab07fef80 2023-10-16T11:24:32.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.742+0000 7f352bdc7740 10 client.11466 fetch_fsmap finished waiting for FSMap version 114 2023-10-16T11:24:32.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.742+0000 7f352bdc7740 10 client.11466 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T11:24:32.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.742+0000 7f352bdc7740 20 client.11466 populate_metadata read hostname 'smithi084' 2023-10-16T11:24:32.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.742+0000 7f352bdc7740 1 --2- 172.21.15.84:0/3481056021 >> [v2:172.21.15.84:6834/551895595,v1:172.21.15.84:6836/551895595] conn(0x563ab0876e80 0x563ab081e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:32.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.742+0000 7f352bdc7740 4 client.11466 mds_command: new command op to 11307 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T11:24:32.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.742+0000 7f352bdc7740 1 -- 172.21.15.84:0/3481056021 --> [v2:172.21.15.84:6834/551895595,v1:172.21.15.84:6836/551895595] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563ab0736700 con 0x563ab0876e80 2023-10-16T11:24:32.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.742+0000 7f352259c640 1 --2- 172.21.15.84:0/3481056021 >> [v2:172.21.15.84:6834/551895595,v1:172.21.15.84:6836/551895595] conn(0x563ab0876e80 0x563ab081e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:32.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.743+0000 7f3501ffb640 10 client.11466 ms_handle_connect on v2:172.21.15.84:6834/551895595 2023-10-16T11:24:32.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.743+0000 7f352259c640 1 --2- 172.21.15.84:0/3481056021 >> [v2:172.21.15.84:6834/551895595,v1:172.21.15.84:6836/551895595] conn(0x563ab0876e80 0x563ab081e880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe568e9f5004afa1 server_cookie=ce3cf83c50aa1b07 in_seq=0 out_seq=0 2023-10-16T11:24:32.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.743+0000 7f3501ffb640 1 -- 172.21.15.84:0/3481056021 <== mds.0 v2:172.21.15.84:6834/551895595 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x563ab0736700 con 0x563ab0876e80 2023-10-16T11:24:32.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.743+0000 7f3501ffb640 10 client.11466 handle_command_reply: tid=0 2023-10-16T11:24:32.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.743+0000 7f352bdc7740 1 client.11466 shutdown 2023-10-16T11:24:32.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.743+0000 7f352bdc7740 1 -- 172.21.15.84:0/3481056021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 msgr2=0x563ab0801cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:32.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.743+0000 7f352bdc7740 1 --2- 172.21.15.84:0/3481056021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 0x563ab0801cd0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f3514005100 tx=0x7f3514001190 comp rx=0 tx=0).stop 2023-10-16T11:24:32.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.743+0000 7f352bdc7740 1 -- 172.21.15.84:0/3481056021 shutdown_connections 2023-10-16T11:24:32.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.743+0000 7f352bdc7740 1 --2- 172.21.15.84:0/3481056021 >> [v2:172.21.15.84:6834/551895595,v1:172.21.15.84:6836/551895595] conn(0x563ab0876e80 0x563ab081e880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.743+0000 7f352bdc7740 1 --2- 172.21.15.84:0/3481056021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x563ab0816690 0x563ab081e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.743+0000 7f352bdc7740 1 --2- 172.21.15.84:0/3481056021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ab07ff360 0x563ab0802210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.743+0000 7f352bdc7740 1 --2- 172.21.15.84:0/3481056021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563ab07fef80 0x563ab0801cd0 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.743+0000 7f3501ffb640 0 client.11466 ms_handle_reset on v2:172.21.15.84:6834/551895595 2023-10-16T11:24:32.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.743+0000 7f352bdc7740 1 -- 172.21.15.84:0/3481056021 >> 172.21.15.84:0/3481056021 conn(0x563ab07d23c0 msgr2=0x563ab081f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:32.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.744+0000 7f352bdc7740 1 -- 172.21.15.84:0/3481056021 shutdown_connections 2023-10-16T11:24:32.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.744+0000 7f352bdc7740 1 -- 172.21.15.84:0/3481056021 wait complete. 2023-10-16T11:24:32.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.744+0000 7f352bdc7740 20 client.11466 trim_cache size 0 max 16384 2023-10-16T11:24:32.760 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:24:32.760 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "63fb4d68-f881-4694-b880-d10369105c0e", 2023-10-16T11:24:32.760 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-16T11:24:32.760 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 11307, 2023-10-16T11:24:32.760 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-16T11:24:32.761 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-16T11:24:32.761 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-16T11:24:32.761 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 6.673162542, 2023-10-16T11:24:32.761 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 114, 2023-10-16T11:24:32.761 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 217, 2023-10-16T11:24:32.762 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 217, 2023-10-16T11:24:32.762 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 12.066739755 2023-10-16T11:24:32.762 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:24:32.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.744+0000 7f352915a640 1 -- 172.21.15.84:0/3666864990 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f351006af90 msgr2=0x7f351006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:32.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.744+0000 7f352915a640 1 --2- 172.21.15.84:0/3666864990 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f351006af90 0x7f351006d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3514001690 tx=0x7f3514009040 comp rx=0 tx=0).stop 2023-10-16T11:24:32.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.744+0000 7f352915a640 1 -- 172.21.15.84:0/3666864990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35240ca6f0 msgr2=0x7f35240aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:32.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.744+0000 7f352915a640 1 --2- 172.21.15.84:0/3666864990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35240ca6f0 0x7f35240aece0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f3518011540 tx=0x7f3518011a00 comp rx=0 tx=0).stop 2023-10-16T11:24:32.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.744+0000 7f352915a640 1 -- 172.21.15.84:0/3666864990 shutdown_connections 2023-10-16T11:24:32.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.744+0000 7f352915a640 1 --2- 172.21.15.84:0/3666864990 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f351006af90 0x7f351006d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.744+0000 7f352915a640 1 --2- 172.21.15.84:0/3666864990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3524159d10 0x7f35240af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.744+0000 7f352915a640 1 --2- 172.21.15.84:0/3666864990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35240ca6f0 0x7f35240aece0 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.744+0000 7f352915a640 1 --2- 172.21.15.84:0/3666864990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f350c005180 0x7f3524158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:32.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.744+0000 7f352915a640 1 -- 172.21.15.84:0/3666864990 >> 172.21.15.84:0/3666864990 conn(0x7f35240ce9b0 msgr2=0x7f352415dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:32.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.745+0000 7f352915a640 1 -- 172.21.15.84:0/3666864990 shutdown_connections 2023-10-16T11:24:32.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:32.745+0000 7f352915a640 1 -- 172.21.15.84:0/3666864990 wait complete. 2023-10-16T11:24:32.872 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T11:24:33.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.089+0000 7fcb7759e640 1 Processor -- start 2023-10-16T11:24:33.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.089+0000 7fcb7759e640 1 -- start start 2023-10-16T11:24:33.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.090+0000 7fcb7759e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcb7806c040 0x7fcb7806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:33.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.090+0000 7fcb7759e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcb780cda80 0x7fcb7815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:33.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.090+0000 7fcb7759e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb7806c960 con 0x7fcb780ad970 2023-10-16T11:24:33.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.090+0000 7fcb7759e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb7815b9d0 con 0x7fcb780cda80 2023-10-16T11:24:33.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.090+0000 7fcb7759e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb7815bcb0 con 0x7fcb7806c040 2023-10-16T11:24:33.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.091+0000 7fcb76d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcb780cda80 0x7fcb7815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:33.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.091+0000 7fcb76d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcb780cda80 0x7fcb7815ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56368/0 (socket says 172.21.15.84:56368) 2023-10-16T11:24:33.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.091+0000 7fcb76d9d640 1 -- 172.21.15.84:0/1126012970 learned_addr learned my addr 172.21.15.84:0/1126012970 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:33.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.091+0000 7fcb7659c640 1 --2- 172.21.15.84:0/1126012970 >> v2:172.21.15.157:3300/0 conn(0x7fcb7806c040 0x7fcb7806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:33.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.091+0000 7fcb76d9d640 1 -- 172.21.15.84:0/1126012970 >> v2:172.21.15.157:3300/0 conn(0x7fcb7806c040 msgr2=0x7fcb7806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:33.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.091+0000 7fcb76d9d640 1 --2- 172.21.15.84:0/1126012970 >> v2:172.21.15.157:3300/0 conn(0x7fcb7806c040 0x7fcb7806c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:33.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.091+0000 7fcb76d9d640 1 -- 172.21.15.84:0/1126012970 >> v1:172.21.15.157:6789/0 conn(0x7fcb780ad970 legacy=0x7fcb7815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:33.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.091+0000 7fcb76d9d640 1 -- 172.21.15.84:0/1126012970 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7815bf90 con 0x7fcb780cda80 2023-10-16T11:24:33.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.092+0000 7fcb76d9d640 1 --2- 172.21.15.84:0/1126012970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb780cda80 0x7fcb7815ed90 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7fcb780cf350 tx=0x7fcb7002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43c78f04b617136 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:33.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.093+0000 7fcb75d9b640 1 -- 172.21.15.84:0/1126012970 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7002ca80 con 0x7fcb780cda80 2023-10-16T11:24:33.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.093+0000 7fcb75d9b640 1 -- 172.21.15.84:0/1126012970 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7002cbe0 con 0x7fcb780cda80 2023-10-16T11:24:33.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.093+0000 7fcb75d9b640 1 -- 172.21.15.84:0/1126012970 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb700343f0 con 0x7fcb780cda80 2023-10-16T11:24:33.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.093+0000 7fcb7759e640 1 -- 172.21.15.84:0/1126012970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb780cda80 msgr2=0x7fcb7815ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:33.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.093+0000 7fcb7759e640 1 --2- 172.21.15.84:0/1126012970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb780cda80 0x7fcb7815ed90 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7fcb780cf350 tx=0x7fcb7002bee0 comp rx=0 tx=0).stop 2023-10-16T11:24:33.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.093+0000 7fcb7759e640 1 -- 172.21.15.84:0/1126012970 shutdown_connections 2023-10-16T11:24:33.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.093+0000 7fcb7759e640 1 --2- 172.21.15.84:0/1126012970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb780cda80 0x7fcb7815ed90 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:33.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.093+0000 7fcb7759e640 1 --2- 172.21.15.84:0/1126012970 >> v2:172.21.15.157:3300/0 conn(0x7fcb7806c040 0x7fcb7806c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:33.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.093+0000 7fcb7759e640 1 -- 172.21.15.84:0/1126012970 >> 172.21.15.84:0/1126012970 conn(0x7fcb780ce6a0 msgr2=0x7fcb780ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:33.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.093+0000 7fcb7759e640 1 -- 172.21.15.84:0/1126012970 shutdown_connections 2023-10-16T11:24:33.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.093+0000 7fcb7759e640 1 -- 172.21.15.84:0/1126012970 wait complete. 2023-10-16T11:24:33.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.095+0000 7fcb7759e640 1 Processor -- start 2023-10-16T11:24:33.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.095+0000 7fcb7759e640 1 -- start start 2023-10-16T11:24:33.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.096+0000 7fcb7759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb7806c040 0x7fcb780b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:33.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.096+0000 7fcb7759e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb780ad970 0x7fcb780acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:33.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.096+0000 7fcb7759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb780cda80 0x7fcb78150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:33.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.096+0000 7fcb7759e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7815d9a0 con 0x7fcb7806c040 2023-10-16T11:24:33.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.096+0000 7fcb7759e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb780b0700 con 0x7fcb780cda80 2023-10-16T11:24:33.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.096+0000 7fcb7759e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb780b09e0 con 0x7fcb780ad970 2023-10-16T11:24:33.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.096+0000 7fcb7659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb7806c040 0x7fcb780b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:33.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.096+0000 7fcb7659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb7806c040 0x7fcb780b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56382/0 (socket says 172.21.15.84:56382) 2023-10-16T11:24:33.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.096+0000 7fcb7659c640 1 -- 172.21.15.84:0/3561722866 learned_addr learned my addr 172.21.15.84:0/3561722866 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:33.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.097+0000 7fcb7659c640 1 -- 172.21.15.84:0/3561722866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb780ad970 msgr2=0x7fcb780acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:33.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.097+0000 7fcb7659c640 1 --2- 172.21.15.84:0/3561722866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb780ad970 0x7fcb780acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:33.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.097+0000 7fcb7659c640 1 -- 172.21.15.84:0/3561722866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb780cda80 msgr2=0x7fcb78150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:33.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.097+0000 7fcb7659c640 1 --2- 172.21.15.84:0/3561722866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb780cda80 0x7fcb78150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:33.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.097+0000 7fcb7659c640 1 -- 172.21.15.84:0/3561722866 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb70009b30 con 0x7fcb7806c040 2023-10-16T11:24:33.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.097+0000 7fcb7659c640 1 --2- 172.21.15.84:0/3561722866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb7806c040 0x7fcb780b2f60 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800d5d0 tx=0x7fcb6800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:33.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.097+0000 7fcb6f7fe640 1 -- 172.21.15.84:0/3561722866 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb680125f0 con 0x7fcb7806c040 2023-10-16T11:24:33.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.097+0000 7fcb6f7fe640 1 -- 172.21.15.84:0/3561722866 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb68012770 con 0x7fcb7806c040 2023-10-16T11:24:33.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.097+0000 7fcb6f7fe640 1 -- 172.21.15.84:0/3561722866 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68012b20 con 0x7fcb7806c040 2023-10-16T11:24:33.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.097+0000 7fcb7759e640 1 -- 172.21.15.84:0/3561722866 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb780b0cc0 con 0x7fcb7806c040 2023-10-16T11:24:33.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.098+0000 7fcb7759e640 1 -- 172.21.15.84:0/3561722866 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb780c8d90 con 0x7fcb7806c040 2023-10-16T11:24:33.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.099+0000 7fcb7759e640 1 -- 172.21.15.84:0/3561722866 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb7806b330 con 0x7fcb7806c040 2023-10-16T11:24:33.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.099+0000 7fcb6f7fe640 1 -- 172.21.15.84:0/3561722866 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcb68012c80 con 0x7fcb7806c040 2023-10-16T11:24:33.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.100+0000 7fcb6f7fe640 1 --2- 172.21.15.84:0/3561722866 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fcb5806af90 0x7fcb5806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:33.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.100+0000 7fcb6f7fe640 1 -- 172.21.15.84:0/3561722866 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fcb68089c70 con 0x7fcb7806c040 2023-10-16T11:24:33.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.101+0000 7fcb6dd9b640 1 --2- 172.21.15.84:0/3561722866 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fcb5806af90 0x7fcb5806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:33.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.101+0000 7fcb6dd9b640 1 --2- 172.21.15.84:0/3561722866 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fcb5806af90 0x7fcb5806d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb7815f050 tx=0x7fcb600079b0 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:33.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.104+0000 7fcb6f7fe640 1 -- 172.21.15.84:0/3561722866 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcb68054c80 con 0x7fcb7806c040 2023-10-16T11:24:33.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.336+0000 7fcb7759e640 1 -- 172.21.15.84:0/3561722866 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcb7804e9b0 con 0x7fcb7806c040 2023-10-16T11:24:33.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.337+0000 7fcb6f7fe640 1 -- 172.21.15.84:0/3561722866 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v114) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcb680582a0 con 0x7fcb7806c040 2023-10-16T11:24:33.339 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T11:24:33.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.340+0000 7fcb7759e640 1 -- 172.21.15.84:0/3561722866 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fcb5806af90 msgr2=0x7fcb5806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:33.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.340+0000 7fcb7759e640 1 --2- 172.21.15.84:0/3561722866 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fcb5806af90 0x7fcb5806d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb7815f050 tx=0x7fcb600079b0 comp rx=0 tx=0).stop 2023-10-16T11:24:33.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.340+0000 7fcb7759e640 1 -- 172.21.15.84:0/3561722866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb7806c040 msgr2=0x7fcb780b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:33.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.340+0000 7fcb7759e640 1 --2- 172.21.15.84:0/3561722866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb7806c040 0x7fcb780b2f60 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800d5d0 tx=0x7fcb6800da90 comp rx=0 tx=0).stop 2023-10-16T11:24:33.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.340+0000 7fcb7759e640 1 -- 172.21.15.84:0/3561722866 shutdown_connections 2023-10-16T11:24:33.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.340+0000 7fcb7759e640 1 --2- 172.21.15.84:0/3561722866 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fcb5806af90 0x7fcb5806d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:33.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.340+0000 7fcb7759e640 1 --2- 172.21.15.84:0/3561722866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb780cda80 0x7fcb78150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:33.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.340+0000 7fcb7759e640 1 --2- 172.21.15.84:0/3561722866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb780ad970 0x7fcb780acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:33.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.340+0000 7fcb7759e640 1 --2- 172.21.15.84:0/3561722866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb7806c040 0x7fcb780b2f60 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:33.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.340+0000 7fcb7759e640 1 -- 172.21.15.84:0/3561722866 >> 172.21.15.84:0/3561722866 conn(0x7fcb780ce6a0 msgr2=0x7fcb78159ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:33.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.340+0000 7fcb7759e640 1 -- 172.21.15.84:0/3561722866 shutdown_connections 2023-10-16T11:24:33.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:33.340+0000 7fcb7759e640 1 -- 172.21.15.84:0/3561722866 wait complete. 2023-10-16T11:24:33.360 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T11:24:33.360 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T11:24:33.360 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-16T11:24:33.361 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T11:24:33.361 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T11:24:33.361 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T11:24:33.361 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T11:24:33.362 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T11:24:33.362 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:24:33.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:24:33.363 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-16T11:24:33.414 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T11:24:33.414 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T11:24:33.414 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T11:24:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:24:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T11:24:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:24:33.415 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:24:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:24:33.416 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-16T11:24:33.416 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:24:33.416 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:24:33.416 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-16T11:24:33.416 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:24:33.417 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:24:33.417 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T11:24:33.417 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:24:33.417 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T11:24:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 36678sec preferred_lft 36678sec 2023-10-16T11:24:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-16T11:24:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:24:33.418 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T11:24:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:24:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T11:24:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:24:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::20a8:eaff:fe9d:5cea/64 scope link 2023-10-16T11:24:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:24:33.419 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:24:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T11:24:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1854:13ff:fe47:be92/64 scope link 2023-10-16T11:24:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:24:33.420 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:24:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether ee:d5:41:a3:a5:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T11:24:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ecd5:41ff:fea3:a570/64 scope link 2023-10-16T11:24:33.421 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:24:33.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:24:33.422 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T11:24:33.514 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-16T11:24:33.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:24:33.519 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-16T11:24:33.575 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-16T11:24:33.575 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T11:24:33.575 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T11:24:33.575 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T11:24:33.576 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T11:24:33.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:24:33.576 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T11:24:33.577 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T11:24:33.577 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T11:24:33.578 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T11:24:33.578 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T11:24:33.578 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T11:24:33.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:24:33.767 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T11:24:33.817 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T11:24:33.817 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T11:24:33.818 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:24:33.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:24:33.819 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-16T11:24:33.911 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:24:33.912 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T11:24:33.961 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:24:33.962 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:24:33.962 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:24:34.010 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-16T11:24:34.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:24:34.011 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:24:34.148 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:24:34.149 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T11:24:34.171 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:24:34.179 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:24:34.185 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:24:34.185 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T11:24:34.185 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T11:24:34.186 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with new device syntax: 0@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T11:24:34.186 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T11:24:34.189 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-16T11:24:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2023-10-16T11:24:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T11:24:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T11:24:34.190 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T11:24:34.192 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T11:24:34.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:24:34.193 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:24:34.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:24:34.311 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T11:24:34.420 INFO:teuthology.orchestra.run.smithi084.stdout:client6797 2023-10-16T11:24:34.421 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:24:34.421 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:24:34.470 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-16T11:24:34.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.727+0000 7f1c83a26640 1 Processor -- start 2023-10-16T11:24:34.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.727+0000 7f1c83a26640 1 -- start start 2023-10-16T11:24:34.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.728+0000 7f1c83a26640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1c7c15ac10 0x7f1c7c06c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:34.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.728+0000 7f1c83a26640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c7c0b0360 0x7f1c7c0b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:34.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.728+0000 7f1c83a26640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c7c0b0d00 con 0x7f1c7c0ca510 2023-10-16T11:24:34.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.728+0000 7f1c83a26640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7c15f800 con 0x7f1c7c15ac10 2023-10-16T11:24:34.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.728+0000 7f1c83a26640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c15b310 con 0x7f1c7c0b0360 2023-10-16T11:24:34.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.729+0000 7f1c817c2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1c7c15ac10 0x7f1c7c06c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:34.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.729+0000 7f1c817c2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1c7c15ac10 0x7f1c7c06c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56394/0 (socket says 172.21.15.84:56394) 2023-10-16T11:24:34.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.729+0000 7f1c817c2640 1 -- 172.21.15.84:0/3021951899 learned_addr learned my addr 172.21.15.84:0/3021951899 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:34.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.729+0000 7f1c817c2640 1 -- 172.21.15.84:0/3021951899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c7c0b0360 msgr2=0x7f1c7c0b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:34.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.729+0000 7f1c817c2640 1 --2- 172.21.15.84:0/3021951899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c7c0b0360 0x7f1c7c0b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:34.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.729+0000 7f1c817c2640 1 -- 172.21.15.84:0/3021951899 >> v1:172.21.15.84:6789/0 conn(0x7f1c7c0ca510 legacy=0x7f1c7c0ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:34.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.729+0000 7f1c817c2640 1 -- 172.21.15.84:0/3021951899 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c06f640 con 0x7f1c7c15ac10 2023-10-16T11:24:34.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.730+0000 7f1c817c2640 1 --2- 172.21.15.84:0/3021951899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c7c15ac10 0x7f1c7c06c730 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009e50 tx=0x7f1c7002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6aad8413a0ecfff server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:34.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.731+0000 7f1c7bfff640 1 -- 172.21.15.84:0/3021951899 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7002ca80 con 0x7f1c7c15ac10 2023-10-16T11:24:34.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.731+0000 7f1c7bfff640 1 -- 172.21.15.84:0/3021951899 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7002cbe0 con 0x7f1c7c15ac10 2023-10-16T11:24:34.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.731+0000 7f1c7bfff640 1 -- 172.21.15.84:0/3021951899 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c700343f0 con 0x7f1c7c15ac10 2023-10-16T11:24:34.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.731+0000 7f1c83a26640 1 -- 172.21.15.84:0/3021951899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c7c15ac10 msgr2=0x7f1c7c06c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:34.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.731+0000 7f1c83a26640 1 --2- 172.21.15.84:0/3021951899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c7c15ac10 0x7f1c7c06c730 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009e50 tx=0x7f1c7002bee0 comp rx=0 tx=0).stop 2023-10-16T11:24:34.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.732+0000 7f1c83a26640 1 -- 172.21.15.84:0/3021951899 shutdown_connections 2023-10-16T11:24:34.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.732+0000 7f1c83a26640 1 --2- 172.21.15.84:0/3021951899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c7c0b0360 0x7f1c7c0b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:34.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.732+0000 7f1c83a26640 1 --2- 172.21.15.84:0/3021951899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c7c15ac10 0x7f1c7c06c730 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:34.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.732+0000 7f1c83a26640 1 -- 172.21.15.84:0/3021951899 >> 172.21.15.84:0/3021951899 conn(0x7f1c7c0cb360 msgr2=0x7f1c7c0cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:34.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.732+0000 7f1c83a26640 1 -- 172.21.15.84:0/3021951899 shutdown_connections 2023-10-16T11:24:34.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.732+0000 7f1c83a26640 1 -- 172.21.15.84:0/3021951899 wait complete. 2023-10-16T11:24:34.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.733+0000 7f1c83a26640 1 Processor -- start 2023-10-16T11:24:34.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.734+0000 7f1c83a26640 1 -- start start 2023-10-16T11:24:34.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.734+0000 7f1c83a26640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c7c0b0360 0x7f1c7c0b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:34.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.734+0000 7f1c83a26640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c7c0ca510 0x7f1c7c152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:34.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.735+0000 7f1c83a26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c7c15ac10 0x7f1c7c0c2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:34.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.735+0000 7f1c83a26640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c0b3b10 con 0x7f1c7c0b0360 2023-10-16T11:24:34.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.735+0000 7f1c83a26640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c1533e0 con 0x7f1c7c15ac10 2023-10-16T11:24:34.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.735+0000 7f1c83a26640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c0b27d0 con 0x7f1c7c0ca510 2023-10-16T11:24:34.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.735+0000 7f1c817c2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c7c0b0360 0x7f1c7c0b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:34.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.735+0000 7f1c817c2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c7c0b0360 0x7f1c7c0b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56400/0 (socket says 172.21.15.84:56400) 2023-10-16T11:24:34.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.735+0000 7f1c817c2640 1 -- 172.21.15.84:0/2347882442 learned_addr learned my addr 172.21.15.84:0/2347882442 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:34.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.735+0000 7f1c817c2640 1 -- 172.21.15.84:0/2347882442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c7c0ca510 msgr2=0x7f1c7c152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:34.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.735+0000 7f1c817c2640 1 --2- 172.21.15.84:0/2347882442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c7c0ca510 0x7f1c7c152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:34.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.735+0000 7f1c817c2640 1 -- 172.21.15.84:0/2347882442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c7c15ac10 msgr2=0x7f1c7c0c2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:34.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.735+0000 7f1c817c2640 1 --2- 172.21.15.84:0/2347882442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c7c15ac10 0x7f1c7c0c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:34.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.735+0000 7f1c817c2640 1 -- 172.21.15.84:0/2347882442 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c70009b30 con 0x7f1c7c0b0360 2023-10-16T11:24:34.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.735+0000 7f1c817c2640 1 --2- 172.21.15.84:0/2347882442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c7c0b0360 0x7f1c7c0b35d0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009e50 tx=0x7f1c70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:34.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.736+0000 7f1c79ffb640 1 -- 172.21.15.84:0/2347882442 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7003e450 con 0x7f1c7c0b0360 2023-10-16T11:24:34.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.736+0000 7f1c79ffb640 1 -- 172.21.15.84:0/2347882442 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7003e5f0 con 0x7f1c7c0b0360 2023-10-16T11:24:34.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.736+0000 7f1c79ffb640 1 -- 172.21.15.84:0/2347882442 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7003e9a0 con 0x7f1c7c0b0360 2023-10-16T11:24:34.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.736+0000 7f1c83a26640 1 -- 172.21.15.84:0/2347882442 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c7c0b2ab0 con 0x7f1c7c0b0360 2023-10-16T11:24:34.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.736+0000 7f1c83a26640 1 -- 172.21.15.84:0/2347882442 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7c0c6460 con 0x7f1c7c0b0360 2023-10-16T11:24:34.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.737+0000 7f1c83a26640 1 -- 172.21.15.84:0/2347882442 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7c04e160 con 0x7f1c7c0b0360 2023-10-16T11:24:34.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.738+0000 7f1c79ffb640 1 -- 172.21.15.84:0/2347882442 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1c7002cd50 con 0x7f1c7c0b0360 2023-10-16T11:24:34.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.739+0000 7f1c79ffb640 1 --2- 172.21.15.84:0/2347882442 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f1c6006afc0 0x7f1c6006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:34.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.739+0000 7f1c79ffb640 1 -- 172.21.15.84:0/2347882442 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1c700ac260 con 0x7f1c7c0b0360 2023-10-16T11:24:34.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.739+0000 7f1c80fc1640 1 --2- 172.21.15.84:0/2347882442 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f1c6006afc0 0x7f1c6006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:34.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.740+0000 7f1c80fc1640 1 --2- 172.21.15.84:0/2347882442 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f1c6006afc0 0x7f1c6006d480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c0c56b0 tx=0x7f1c68006bf0 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:34.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.743+0000 7f1c79ffb640 1 -- 172.21.15.84:0/2347882442 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1c70077300 con 0x7f1c7c0b0360 2023-10-16T11:24:34.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:34.976+0000 7f1c83a26640 1 -- 172.21.15.84:0/2347882442 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1c7c15bb60 con 0x7f1c7c0b0360 2023-10-16T11:24:35.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:35.845+0000 7f1c79ffb640 1 -- 172.21.15.84:0/2347882442 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v115) v1 ==== 136+0+0 (secure 0 0 0) 0x7f1c70033e80 con 0x7f1c7c0b0360 2023-10-16T11:24:35.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:35.849+0000 7f1c83a26640 1 -- 172.21.15.84:0/2347882442 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f1c6006afc0 msgr2=0x7f1c6006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:35.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:35.850+0000 7f1c83a26640 1 --2- 172.21.15.84:0/2347882442 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f1c6006afc0 0x7f1c6006d480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c0c56b0 tx=0x7f1c68006bf0 comp rx=0 tx=0).stop 2023-10-16T11:24:35.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:35.850+0000 7f1c83a26640 1 -- 172.21.15.84:0/2347882442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c7c0b0360 msgr2=0x7f1c7c0b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:35.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:35.850+0000 7f1c83a26640 1 --2- 172.21.15.84:0/2347882442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c7c0b0360 0x7f1c7c0b35d0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009e50 tx=0x7f1c70004a40 comp rx=0 tx=0).stop 2023-10-16T11:24:35.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:35.850+0000 7f1c83a26640 1 -- 172.21.15.84:0/2347882442 shutdown_connections 2023-10-16T11:24:35.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:35.850+0000 7f1c83a26640 1 --2- 172.21.15.84:0/2347882442 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f1c6006afc0 0x7f1c6006d480 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:35.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:35.850+0000 7f1c83a26640 1 --2- 172.21.15.84:0/2347882442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c7c15ac10 0x7f1c7c0c2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:35.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:35.850+0000 7f1c83a26640 1 --2- 172.21.15.84:0/2347882442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c7c0ca510 0x7f1c7c152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:35.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:35.850+0000 7f1c83a26640 1 --2- 172.21.15.84:0/2347882442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c7c0b0360 0x7f1c7c0b35d0 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:35.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:35.850+0000 7f1c83a26640 1 -- 172.21.15.84:0/2347882442 >> 172.21.15.84:0/2347882442 conn(0x7f1c7c0cb360 msgr2=0x7f1c7c0ce850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:35.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:35.850+0000 7f1c83a26640 1 -- 172.21.15.84:0/2347882442 shutdown_connections 2023-10-16T11:24:35.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:35.850+0000 7f1c83a26640 1 -- 172.21.15.84:0/2347882442 wait complete. 2023-10-16T11:24:35.871 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:24:36.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.078+0000 7fc64e747640 1 Processor -- start 2023-10-16T11:24:36.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.078+0000 7fc64e747640 1 -- start start 2023-10-16T11:24:36.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.079+0000 7fc64e747640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc648144060 0x7fc6480b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:36.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.080+0000 7fc64e747640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc64815ecb0 0x7fc64806f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:36.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.080+0000 7fc64e747640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc64806f1d0 con 0x7fc648144440 2023-10-16T11:24:36.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.080+0000 7fc64e747640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6480cd430 con 0x7fc648144060 2023-10-16T11:24:36.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.080+0000 7fc64e747640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6480cd610 con 0x7fc64815ecb0 2023-10-16T11:24:36.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.080+0000 7fc64cf44640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc648144440 0x7fc64815e5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38442/0 (socket says 172.21.15.84:38442) 2023-10-16T11:24:36.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.080+0000 7fc64cf44640 1 -- 172.21.15.84:0/1058364966 learned_addr learned my addr 172.21.15.84:0/1058364966 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:36.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.080+0000 7fc64d745640 1 --2- 172.21.15.84:0/1058364966 >> v2:172.21.15.84:3300/0 conn(0x7fc648144060 0x7fc6480b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:36.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.080+0000 7fc64df46640 1 --2- 172.21.15.84:0/1058364966 >> v2:172.21.15.157:3300/0 conn(0x7fc64815ecb0 0x7fc64806f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:36.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.080+0000 7fc63ffff640 1 -- 172.21.15.84:0/1058364966 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988646115 0 0) 0x7fc64806f1d0 con 0x7fc648144440 2023-10-16T11:24:36.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.080+0000 7fc64d745640 1 -- 172.21.15.84:0/1058364966 >> v2:172.21.15.157:3300/0 conn(0x7fc64815ecb0 msgr2=0x7fc64806f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:36.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.080+0000 7fc64d745640 1 --2- 172.21.15.84:0/1058364966 >> v2:172.21.15.157:3300/0 conn(0x7fc64815ecb0 0x7fc64806f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.080+0000 7fc64d745640 1 -- 172.21.15.84:0/1058364966 >> v1:172.21.15.84:6789/0 conn(0x7fc648144440 legacy=0x7fc64815e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:36.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.080+0000 7fc64d745640 1 -- 172.21.15.84:0/1058364966 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6480cd8f0 con 0x7fc648144060 2023-10-16T11:24:36.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.080+0000 7fc64df46640 1 --2- 172.21.15.84:0/1058364966 >> v2:172.21.15.157:3300/0 conn(0x7fc64815ecb0 0x7fc64806f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:36.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.081+0000 7fc64d745640 1 --2- 172.21.15.84:0/1058364966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc648144060 0x7fc6480b0a40 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7fc64815bfc0 tx=0x7fc64402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ad706ad8ac8ce server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:36.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.082+0000 7fc63ffff640 1 -- 172.21.15.84:0/1058364966 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64402e070 con 0x7fc648144060 2023-10-16T11:24:36.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.082+0000 7fc63ffff640 1 -- 172.21.15.84:0/1058364966 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6440315a0 con 0x7fc648144060 2023-10-16T11:24:36.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.082+0000 7fc63ffff640 1 -- 172.21.15.84:0/1058364966 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644031930 con 0x7fc648144060 2023-10-16T11:24:36.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.082+0000 7fc64e747640 1 -- 172.21.15.84:0/1058364966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc648144060 msgr2=0x7fc6480b0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:36.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.082+0000 7fc64e747640 1 --2- 172.21.15.84:0/1058364966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc648144060 0x7fc6480b0a40 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7fc64815bfc0 tx=0x7fc64402b840 comp rx=0 tx=0).stop 2023-10-16T11:24:36.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.082+0000 7fc64e747640 1 -- 172.21.15.84:0/1058364966 shutdown_connections 2023-10-16T11:24:36.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.082+0000 7fc64e747640 1 --2- 172.21.15.84:0/1058364966 >> v2:172.21.15.157:3300/0 conn(0x7fc64815ecb0 0x7fc64806f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.082+0000 7fc64e747640 1 --2- 172.21.15.84:0/1058364966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc648144060 0x7fc6480b0a40 unknown :-1 s=CLOSED pgs=2645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.082+0000 7fc64e747640 1 -- 172.21.15.84:0/1058364966 >> 172.21.15.84:0/1058364966 conn(0x7fc648069da0 msgr2=0x7fc64806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:36.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.082+0000 7fc64e747640 1 -- 172.21.15.84:0/1058364966 shutdown_connections 2023-10-16T11:24:36.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.083+0000 7fc64e747640 1 -- 172.21.15.84:0/1058364966 wait complete. 2023-10-16T11:24:36.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.084+0000 7fc64e747640 1 Processor -- start 2023-10-16T11:24:36.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.084+0000 7fc64e747640 1 -- start start 2023-10-16T11:24:36.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.085+0000 7fc64e747640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc648144060 0x7fc6480a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:36.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.085+0000 7fc64e747640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc648144440 0x7fc648157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:36.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.085+0000 7fc64e747640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc64815ecb0 0x7fc64815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:36.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.085+0000 7fc64e747640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6481f3210 con 0x7fc648144440 2023-10-16T11:24:36.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.085+0000 7fc64e747640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6480aa4a0 con 0x7fc648144060 2023-10-16T11:24:36.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.085+0000 7fc64e747640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6481f3460 con 0x7fc64815ecb0 2023-10-16T11:24:36.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.086+0000 7fc64cf44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc648144440 0x7fc648157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:36.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.086+0000 7fc64cf44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc648144440 0x7fc648157be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49888/0 (socket says 172.21.15.84:49888) 2023-10-16T11:24:36.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.086+0000 7fc64cf44640 1 -- 172.21.15.84:0/1338214634 learned_addr learned my addr 172.21.15.84:0/1338214634 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:36.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.086+0000 7fc64d745640 1 --2- 172.21.15.84:0/1338214634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc648144060 0x7fc6480a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:36.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.086+0000 7fc64df46640 1 --2- 172.21.15.84:0/1338214634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc64815ecb0 0x7fc64815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:36.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.086+0000 7fc64cf44640 1 -- 172.21.15.84:0/1338214634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc64815ecb0 msgr2=0x7fc64815aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:36.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.086+0000 7fc64cf44640 1 --2- 172.21.15.84:0/1338214634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc64815ecb0 0x7fc64815aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.086+0000 7fc64cf44640 1 -- 172.21.15.84:0/1338214634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc648144060 msgr2=0x7fc6480a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:36.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.086+0000 7fc64cf44640 1 --2- 172.21.15.84:0/1338214634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc648144060 0x7fc6480a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.086+0000 7fc64cf44640 1 -- 172.21.15.84:0/1338214634 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc644009400 con 0x7fc648144440 2023-10-16T11:24:36.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.087+0000 7fc64cf44640 1 --2- 172.21.15.84:0/1338214634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc648144440 0x7fc648157be0 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7fc63800c470 tx=0x7fc63800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:36.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.087+0000 7fc63dffb640 1 -- 172.21.15.84:0/1338214634 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6380074e0 con 0x7fc648144440 2023-10-16T11:24:36.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.087+0000 7fc63dffb640 1 -- 172.21.15.84:0/1338214634 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc638007640 con 0x7fc648144440 2023-10-16T11:24:36.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.087+0000 7fc64e747640 1 -- 172.21.15.84:0/1338214634 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6481f3740 con 0x7fc648144440 2023-10-16T11:24:36.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.087+0000 7fc63dffb640 1 -- 172.21.15.84:0/1338214634 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc638004a20 con 0x7fc648144440 2023-10-16T11:24:36.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.087+0000 7fc64e747640 1 -- 172.21.15.84:0/1338214634 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6481f3f20 con 0x7fc648144440 2023-10-16T11:24:36.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.088+0000 7fc64e747640 1 -- 172.21.15.84:0/1338214634 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6480674f0 con 0x7fc648144440 2023-10-16T11:24:36.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.089+0000 7fc63dffb640 1 -- 172.21.15.84:0/1338214634 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc638004b80 con 0x7fc648144440 2023-10-16T11:24:36.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.090+0000 7fc63dffb640 1 --2- 172.21.15.84:0/1338214634 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fc62c06af90 0x7fc62c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:36.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.090+0000 7fc63dffb640 1 -- 172.21.15.84:0/1338214634 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc638088cf0 con 0x7fc648144440 2023-10-16T11:24:36.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.090+0000 7fc64d745640 1 --2- 172.21.15.84:0/1338214634 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fc62c06af90 0x7fc62c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:36.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.091+0000 7fc64d745640 1 --2- 172.21.15.84:0/1338214634 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fc62c06af90 0x7fc62c06d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc644036000 tx=0x7fc64403d040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:36.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.095+0000 7fc63dffb640 1 -- 172.21.15.84:0/1338214634 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc638053bd0 con 0x7fc648144440 2023-10-16T11:24:36.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.289+0000 7fc64e747640 1 -- 172.21.15.84:0/1338214634 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc648070720 con 0x7fc648144440 2023-10-16T11:24:36.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.292+0000 7fc63dffb640 1 -- 172.21.15.84:0/1338214634 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v217) v1 ==== 74+0+16603 (secure 0 0 0) 0x7fc638018020 con 0x7fc648144440 2023-10-16T11:24:36.294 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:36.294 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":217,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:24:27.068476+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:24:23.363032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:24:24.551270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:24:36.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.294+0000 7fc64e747640 1 -- 172.21.15.84:0/1338214634 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fc62c06af90 msgr2=0x7fc62c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:36.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.294+0000 7fc64e747640 1 --2- 172.21.15.84:0/1338214634 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fc62c06af90 0x7fc62c06d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc644036000 tx=0x7fc64403d040 comp rx=0 tx=0).stop 2023-10-16T11:24:36.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.294+0000 7fc64e747640 1 -- 172.21.15.84:0/1338214634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc648144440 msgr2=0x7fc648157be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:36.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.294+0000 7fc64e747640 1 --2- 172.21.15.84:0/1338214634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc648144440 0x7fc648157be0 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7fc63800c470 tx=0x7fc63800c930 comp rx=0 tx=0).stop 2023-10-16T11:24:36.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.294+0000 7fc64e747640 1 -- 172.21.15.84:0/1338214634 shutdown_connections 2023-10-16T11:24:36.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.294+0000 7fc64e747640 1 --2- 172.21.15.84:0/1338214634 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fc62c06af90 0x7fc62c06d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.294+0000 7fc64e747640 1 --2- 172.21.15.84:0/1338214634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc64815ecb0 0x7fc64815aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.294+0000 7fc64e747640 1 --2- 172.21.15.84:0/1338214634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc648144440 0x7fc648157be0 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.294+0000 7fc64e747640 1 --2- 172.21.15.84:0/1338214634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc648144060 0x7fc6480a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.294+0000 7fc64e747640 1 -- 172.21.15.84:0/1338214634 >> 172.21.15.84:0/1338214634 conn(0x7fc648069da0 msgr2=0x7fc64806f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:36.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.295+0000 7fc64e747640 1 -- 172.21.15.84:0/1338214634 shutdown_connections 2023-10-16T11:24:36.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.295+0000 7fc64e747640 1 -- 172.21.15.84:0/1338214634 wait complete. 2023-10-16T11:24:36.315 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:24:23.363032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '216', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:24:24.551270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '217', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:24:36.316 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:24:36.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.526+0000 7fe6979c0640 1 Processor -- start 2023-10-16T11:24:36.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.526+0000 7fe6979c0640 1 -- start start 2023-10-16T11:24:36.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.528+0000 7fe6979c0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe690160530 0x7fe6901609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:36.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.528+0000 7fe6979c0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe690069fa0 0x7fe69006a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:36.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.528+0000 7fe6979c0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6900ac890 con 0x7fe690065ab0 2023-10-16T11:24:36.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.528+0000 7fe6979c0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6900ae800 con 0x7fe690069fa0 2023-10-16T11:24:36.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.528+0000 7fe6979c0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6900ae1f0 con 0x7fe690160530 2023-10-16T11:24:36.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.528+0000 7fe695f5d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe690069fa0 0x7fe69006a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:36.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.528+0000 7fe695f5d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe690069fa0 0x7fe69006a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49902/0 (socket says 172.21.15.84:49902) 2023-10-16T11:24:36.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.529+0000 7fe695f5d640 1 -- 172.21.15.84:0/2615528321 learned_addr learned my addr 172.21.15.84:0/2615528321 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:36.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.529+0000 7fe69575c640 1 --2- 172.21.15.84:0/2615528321 >> v2:172.21.15.157:3300/0 conn(0x7fe690160530 0x7fe6901609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:36.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.529+0000 7fe695f5d640 1 -- 172.21.15.84:0/2615528321 >> v2:172.21.15.157:3300/0 conn(0x7fe690160530 msgr2=0x7fe6901609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:36.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.529+0000 7fe695f5d640 1 --2- 172.21.15.84:0/2615528321 >> v2:172.21.15.157:3300/0 conn(0x7fe690160530 0x7fe6901609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.529+0000 7fe695f5d640 1 -- 172.21.15.84:0/2615528321 >> v1:172.21.15.84:6789/0 conn(0x7fe690065ab0 legacy=0x7fe690160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:36.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.529+0000 7fe695f5d640 1 -- 172.21.15.84:0/2615528321 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6900acae0 con 0x7fe690069fa0 2023-10-16T11:24:36.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.529+0000 7fe69575c640 1 --2- 172.21.15.84:0/2615528321 >> v2:172.21.15.157:3300/0 conn(0x7fe690160530 0x7fe6901609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:36.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.530+0000 7fe695f5d640 1 --2- 172.21.15.84:0/2615528321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe690069fa0 0x7fe69006a430 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7fe6900cf030 tx=0x7fe68802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1704ccbc1dbeb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:36.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.530+0000 7fe687fff640 1 -- 172.21.15.84:0/2615528321 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68802ca80 con 0x7fe690069fa0 2023-10-16T11:24:36.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.530+0000 7fe687fff640 1 -- 172.21.15.84:0/2615528321 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68802cbe0 con 0x7fe690069fa0 2023-10-16T11:24:36.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.530+0000 7fe687fff640 1 -- 172.21.15.84:0/2615528321 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6880343f0 con 0x7fe690069fa0 2023-10-16T11:24:36.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.531+0000 7fe6979c0640 1 -- 172.21.15.84:0/2615528321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe690069fa0 msgr2=0x7fe69006a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:36.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.531+0000 7fe6979c0640 1 --2- 172.21.15.84:0/2615528321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe690069fa0 0x7fe69006a430 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7fe6900cf030 tx=0x7fe68802bee0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.531+0000 7fe6979c0640 1 -- 172.21.15.84:0/2615528321 shutdown_connections 2023-10-16T11:24:36.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.531+0000 7fe6979c0640 1 --2- 172.21.15.84:0/2615528321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe690069fa0 0x7fe69006a430 unknown :-1 s=CLOSED pgs=2648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.531+0000 7fe6979c0640 1 --2- 172.21.15.84:0/2615528321 >> v2:172.21.15.157:3300/0 conn(0x7fe690160530 0x7fe6901609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.531+0000 7fe6979c0640 1 -- 172.21.15.84:0/2615528321 >> 172.21.15.84:0/2615528321 conn(0x7fe69006bc10 msgr2=0x7fe69006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.531+0000 7fe6979c0640 1 -- 172.21.15.84:0/2615528321 shutdown_connections 2023-10-16T11:24:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.531+0000 7fe6979c0640 1 -- 172.21.15.84:0/2615528321 wait complete. 2023-10-16T11:24:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.533+0000 7fe6979c0640 1 Processor -- start 2023-10-16T11:24:36.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.533+0000 7fe6979c0640 1 -- start start 2023-10-16T11:24:36.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.534+0000 7fe6979c0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe690065ab0 0x7fe6900c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:36.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.534+0000 7fe6979c0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe690069fa0 0x7fe69014b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.534+0000 7fe6979c0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe690160530 0x7fe6901f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.534+0000 7fe6979c0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69014e1d0 con 0x7fe690160530 2023-10-16T11:24:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.534+0000 7fe6979c0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6900c6b40 con 0x7fe690069fa0 2023-10-16T11:24:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.534+0000 7fe6979c0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69014bec0 con 0x7fe690065ab0 2023-10-16T11:24:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.534+0000 7fe695f5d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe690160530 0x7fe6901f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:36.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.534+0000 7fe695f5d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe690160530 0x7fe6901f2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49916/0 (socket says 172.21.15.84:49916) 2023-10-16T11:24:36.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.534+0000 7fe695f5d640 1 -- 172.21.15.84:0/1309437047 learned_addr learned my addr 172.21.15.84:0/1309437047 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:36.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.535+0000 7fe695f5d640 1 -- 172.21.15.84:0/1309437047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe690065ab0 msgr2=0x7fe6900c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:36.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.535+0000 7fe695f5d640 1 --2- 172.21.15.84:0/1309437047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe690065ab0 0x7fe6900c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.535+0000 7fe695f5d640 1 -- 172.21.15.84:0/1309437047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe690069fa0 msgr2=0x7fe69014b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:36.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.535+0000 7fe695f5d640 1 --2- 172.21.15.84:0/1309437047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe690069fa0 0x7fe69014b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.535+0000 7fe695f5d640 1 -- 172.21.15.84:0/1309437047 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe688009b30 con 0x7fe690160530 2023-10-16T11:24:36.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.535+0000 7fe695f5d640 1 --2- 172.21.15.84:0/1309437047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe690160530 0x7fe6901f2db0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7fe688036040 tx=0x7fe688004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.535+0000 7fe685ffb640 1 -- 172.21.15.84:0/1309437047 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68803e450 con 0x7fe690160530 2023-10-16T11:24:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.535+0000 7fe685ffb640 1 -- 172.21.15.84:0/1309437047 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68803e5b0 con 0x7fe690160530 2023-10-16T11:24:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.535+0000 7fe685ffb640 1 -- 172.21.15.84:0/1309437047 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68803e960 con 0x7fe690160530 2023-10-16T11:24:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.535+0000 7fe6979c0640 1 -- 172.21.15.84:0/1309437047 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6901f32f0 con 0x7fe690160530 2023-10-16T11:24:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.535+0000 7fe6979c0640 1 -- 172.21.15.84:0/1309437047 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69014e3f0 con 0x7fe690160530 2023-10-16T11:24:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.537+0000 7fe6979c0640 1 -- 172.21.15.84:0/1309437047 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69004e160 con 0x7fe690160530 2023-10-16T11:24:36.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.537+0000 7fe685ffb640 1 -- 172.21.15.84:0/1309437047 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe68802cd50 con 0x7fe690160530 2023-10-16T11:24:36.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.539+0000 7fe685ffb640 1 --2- 172.21.15.84:0/1309437047 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fe67406afc0 0x7fe67406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:36.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.539+0000 7fe685ffb640 1 -- 172.21.15.84:0/1309437047 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe6880ac430 con 0x7fe690160530 2023-10-16T11:24:36.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.539+0000 7fe69575c640 1 --2- 172.21.15.84:0/1309437047 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fe67406afc0 0x7fe67406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:36.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.540+0000 7fe69575c640 1 --2- 172.21.15.84:0/1309437047 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fe67406afc0 0x7fe67406d480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe68c006fd0 tx=0x7fe68c008040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:36.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.543+0000 7fe685ffb640 1 -- 172.21.15.84:0/1309437047 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe6880773c0 con 0x7fe690160530 2023-10-16T11:24:36.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.735+0000 7fe6979c0640 1 -- 172.21.15.84:0/1309437047 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe69006c8e0 con 0x7fe690160530 2023-10-16T11:24:36.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.736+0000 7fe685ffb640 1 -- 172.21.15.84:0/1309437047 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v217) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe68807fe60 con 0x7fe690160530 2023-10-16T11:24:36.738 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:24:36.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.738+0000 7fe6979c0640 1 -- 172.21.15.84:0/1309437047 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fe67406afc0 msgr2=0x7fe67406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:36.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.738+0000 7fe6979c0640 1 --2- 172.21.15.84:0/1309437047 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fe67406afc0 0x7fe67406d480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe68c006fd0 tx=0x7fe68c008040 comp rx=0 tx=0).stop 2023-10-16T11:24:36.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.738+0000 7fe6979c0640 1 -- 172.21.15.84:0/1309437047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe690160530 msgr2=0x7fe6901f2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:36.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.738+0000 7fe6979c0640 1 --2- 172.21.15.84:0/1309437047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe690160530 0x7fe6901f2db0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7fe688036040 tx=0x7fe688004a40 comp rx=0 tx=0).stop 2023-10-16T11:24:36.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.739+0000 7fe6979c0640 1 -- 172.21.15.84:0/1309437047 shutdown_connections 2023-10-16T11:24:36.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.739+0000 7fe6979c0640 1 --2- 172.21.15.84:0/1309437047 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fe67406afc0 0x7fe67406d480 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.739+0000 7fe6979c0640 1 --2- 172.21.15.84:0/1309437047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe690160530 0x7fe6901f2db0 unknown :-1 s=CLOSED pgs=2649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.739+0000 7fe6979c0640 1 --2- 172.21.15.84:0/1309437047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe690069fa0 0x7fe69014b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.739+0000 7fe6979c0640 1 --2- 172.21.15.84:0/1309437047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe690065ab0 0x7fe6900c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.739+0000 7fe6979c0640 1 -- 172.21.15.84:0/1309437047 >> 172.21.15.84:0/1309437047 conn(0x7fe69006bc10 msgr2=0x7fe69015e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:36.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.739+0000 7fe6979c0640 1 -- 172.21.15.84:0/1309437047 shutdown_connections 2023-10-16T11:24:36.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.739+0000 7fe6979c0640 1 -- 172.21.15.84:0/1309437047 wait complete. 2023-10-16T11:24:36.760 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:24:36.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.976+0000 7f0b702cf640 1 Processor -- start 2023-10-16T11:24:36.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.977+0000 7f0b702cf640 1 -- start start 2023-10-16T11:24:36.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.978+0000 7f0b702cf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0b6806a4b0 0x7f0b680ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:36.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.978+0000 7f0b702cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0b680c9ca0 0x7f0b680ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:36.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.978+0000 7f0b702cf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b680cb030 con 0x7f0b6815ad70 2023-10-16T11:24:36.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.978+0000 7f0b702cf640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b680cae30 con 0x7f0b680c9ca0 2023-10-16T11:24:36.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.978+0000 7f0b702cf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b680c91c0 con 0x7f0b6806a4b0 2023-10-16T11:24:36.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.979+0000 7f0b6e06b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0b6815ad70 0x7f0b68069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38470/0 (socket says 172.21.15.84:38470) 2023-10-16T11:24:36.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.979+0000 7f0b6e06b640 1 -- 172.21.15.84:0/3819205471 learned_addr learned my addr 172.21.15.84:0/3819205471 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:36.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.979+0000 7f0b6e86c640 1 --2- 172.21.15.84:0/3819205471 >> v2:172.21.15.84:3300/0 conn(0x7f0b680c9ca0 0x7f0b680ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:36.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.979+0000 7f0b6d86a640 1 --2- 172.21.15.84:0/3819205471 >> v2:172.21.15.157:3300/0 conn(0x7f0b6806a4b0 0x7f0b680ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:36.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.979+0000 7f0b6d069640 1 -- 172.21.15.84:0/3819205471 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136415524 0 0) 0x7f0b680cb030 con 0x7f0b6815ad70 2023-10-16T11:24:36.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.979+0000 7f0b6d069640 1 -- 172.21.15.84:0/3819205471 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b54003520 con 0x7f0b6815ad70 2023-10-16T11:24:36.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.979+0000 7f0b6e86c640 1 -- 172.21.15.84:0/3819205471 >> v2:172.21.15.157:3300/0 conn(0x7f0b6806a4b0 msgr2=0x7f0b680ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:36.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.979+0000 7f0b6e86c640 1 --2- 172.21.15.84:0/3819205471 >> v2:172.21.15.157:3300/0 conn(0x7f0b6806a4b0 0x7f0b680ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.979+0000 7f0b6e86c640 1 -- 172.21.15.84:0/3819205471 >> v1:172.21.15.84:6789/0 conn(0x7f0b6815ad70 legacy=0x7f0b68069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:36.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.979+0000 7f0b6e86c640 1 -- 172.21.15.84:0/3819205471 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b680c94a0 con 0x7f0b680c9ca0 2023-10-16T11:24:36.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.980+0000 7f0b6e86c640 1 --2- 172.21.15.84:0/3819205471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b680c9ca0 0x7f0b680ca8f0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f0b680ce870 tx=0x7f0b6002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf852b99a1d18c39 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:36.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.981+0000 7f0b6d069640 1 -- 172.21.15.84:0/3819205471 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6002cc80 con 0x7f0b680c9ca0 2023-10-16T11:24:36.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.981+0000 7f0b6d069640 1 -- 172.21.15.84:0/3819205471 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6002cde0 con 0x7f0b680c9ca0 2023-10-16T11:24:36.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.981+0000 7f0b6d069640 1 -- 172.21.15.84:0/3819205471 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60034640 con 0x7f0b680c9ca0 2023-10-16T11:24:36.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.981+0000 7f0b702cf640 1 -- 172.21.15.84:0/3819205471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b680c9ca0 msgr2=0x7f0b680ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:36.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.981+0000 7f0b702cf640 1 --2- 172.21.15.84:0/3819205471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b680c9ca0 0x7f0b680ca8f0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f0b680ce870 tx=0x7f0b6002bed0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.981+0000 7f0b702cf640 1 -- 172.21.15.84:0/3819205471 shutdown_connections 2023-10-16T11:24:36.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.981+0000 7f0b702cf640 1 --2- 172.21.15.84:0/3819205471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b680c9ca0 0x7f0b680ca8f0 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.981+0000 7f0b702cf640 1 --2- 172.21.15.84:0/3819205471 >> v2:172.21.15.157:3300/0 conn(0x7f0b6806a4b0 0x7f0b680ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.981+0000 7f0b702cf640 1 -- 172.21.15.84:0/3819205471 >> 172.21.15.84:0/3819205471 conn(0x7f0b680ce9b0 msgr2=0x7f0b680cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:36.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.982+0000 7f0b702cf640 1 -- 172.21.15.84:0/3819205471 shutdown_connections 2023-10-16T11:24:36.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.982+0000 7f0b702cf640 1 -- 172.21.15.84:0/3819205471 wait complete. 2023-10-16T11:24:36.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.983+0000 7f0b702cf640 1 Processor -- start 2023-10-16T11:24:36.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.984+0000 7f0b702cf640 1 -- start start 2023-10-16T11:24:36.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.984+0000 7f0b702cf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b6806a4b0 0x7f0b681f3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:36.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.984+0000 7f0b702cf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b680c9ca0 0x7f0b680c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:36.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.984+0000 7f0b702cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b6815ad70 0x7f0b680c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:36.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.984+0000 7f0b702cf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b680c4f60 con 0x7f0b6815ad70 2023-10-16T11:24:36.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.984+0000 7f0b702cf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b681f3540 con 0x7f0b680c9ca0 2023-10-16T11:24:36.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.984+0000 7f0b702cf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b680c42e0 con 0x7f0b6806a4b0 2023-10-16T11:24:36.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.985+0000 7f0b6e86c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b6815ad70 0x7f0b680c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:36.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.985+0000 7f0b6e86c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b6815ad70 0x7f0b680c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49932/0 (socket says 172.21.15.84:49932) 2023-10-16T11:24:36.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.985+0000 7f0b6e86c640 1 -- 172.21.15.84:0/2431660354 learned_addr learned my addr 172.21.15.84:0/2431660354 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:36.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.985+0000 7f0b6e86c640 1 -- 172.21.15.84:0/2431660354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b6806a4b0 msgr2=0x7f0b681f3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:36.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.985+0000 7f0b6e06b640 1 --2- 172.21.15.84:0/2431660354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b6806a4b0 0x7f0b681f3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:36.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.985+0000 7f0b6d86a640 1 --2- 172.21.15.84:0/2431660354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b680c9ca0 0x7f0b680c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:36.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.985+0000 7f0b6e86c640 1 --2- 172.21.15.84:0/2431660354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b6806a4b0 0x7f0b681f3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.985+0000 7f0b6e86c640 1 -- 172.21.15.84:0/2431660354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b680c9ca0 msgr2=0x7f0b680c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:36.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.985+0000 7f0b6e86c640 1 --2- 172.21.15.84:0/2431660354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b680c9ca0 0x7f0b680c7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:36.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.985+0000 7f0b6e86c640 1 -- 172.21.15.84:0/2431660354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b60009a70 con 0x7f0b6815ad70 2023-10-16T11:24:36.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.985+0000 7f0b6e86c640 1 --2- 172.21.15.84:0/2431660354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b6815ad70 0x7f0b680c8310 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f0b6002c4e0 tx=0x7f0b60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:36.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.986+0000 7f0b5affd640 1 -- 172.21.15.84:0/2431660354 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6003e610 con 0x7f0b6815ad70 2023-10-16T11:24:36.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.986+0000 7f0b5affd640 1 -- 172.21.15.84:0/2431660354 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6003e770 con 0x7f0b6815ad70 2023-10-16T11:24:36.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.986+0000 7f0b5affd640 1 -- 172.21.15.84:0/2431660354 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6003eb60 con 0x7f0b6815ad70 2023-10-16T11:24:36.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.986+0000 7f0b702cf640 1 -- 172.21.15.84:0/2431660354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b680c45c0 con 0x7f0b6815ad70 2023-10-16T11:24:36.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.986+0000 7f0b702cf640 1 -- 172.21.15.84:0/2431660354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b680c4b80 con 0x7f0b6815ad70 2023-10-16T11:24:36.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.987+0000 7f0b702cf640 1 -- 172.21.15.84:0/2431660354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b680c50a0 con 0x7f0b6815ad70 2023-10-16T11:24:36.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.988+0000 7f0b5affd640 1 -- 172.21.15.84:0/2431660354 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0b6003b050 con 0x7f0b6815ad70 2023-10-16T11:24:36.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.989+0000 7f0b5affd640 1 --2- 172.21.15.84:0/2431660354 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0b4c06af90 0x7f0b4c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:36.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.989+0000 7f0b5affd640 1 -- 172.21.15.84:0/2431660354 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0b600ab8b0 con 0x7f0b6815ad70 2023-10-16T11:24:36.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.989+0000 7f0b6e06b640 1 --2- 172.21.15.84:0/2431660354 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0b4c06af90 0x7f0b4c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:36.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.990+0000 7f0b6e06b640 1 --2- 172.21.15.84:0/2431660354 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0b4c06af90 0x7f0b4c06d450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0b64000f60 tx=0x7f0b64009040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:36.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:36.993+0000 7f0b5affd640 1 -- 172.21.15.84:0/2431660354 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0b680c50a0 con 0x7f0b6815ad70 2023-10-16T11:24:37.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.177+0000 7f0b702cf640 1 -- 172.21.15.84:0/2431660354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0b680c50a0 con 0x7f0b6815ad70 2023-10-16T11:24:37.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.178+0000 7f0b5affd640 1 -- 172.21.15.84:0/2431660354 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v217) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0b60039020 con 0x7f0b6815ad70 2023-10-16T11:24:37.180 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:24:37.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.180+0000 7f0b702cf640 1 -- 172.21.15.84:0/2431660354 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0b4c06af90 msgr2=0x7f0b4c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:37.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.180+0000 7f0b702cf640 1 --2- 172.21.15.84:0/2431660354 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0b4c06af90 0x7f0b4c06d450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0b64000f60 tx=0x7f0b64009040 comp rx=0 tx=0).stop 2023-10-16T11:24:37.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.180+0000 7f0b702cf640 1 -- 172.21.15.84:0/2431660354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b6815ad70 msgr2=0x7f0b680c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:37.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.180+0000 7f0b702cf640 1 --2- 172.21.15.84:0/2431660354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b6815ad70 0x7f0b680c8310 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f0b6002c4e0 tx=0x7f0b60004a40 comp rx=0 tx=0).stop 2023-10-16T11:24:37.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.180+0000 7f0b702cf640 1 -- 172.21.15.84:0/2431660354 shutdown_connections 2023-10-16T11:24:37.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.180+0000 7f0b702cf640 1 --2- 172.21.15.84:0/2431660354 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0b4c06af90 0x7f0b4c06d450 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:37.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.180+0000 7f0b702cf640 1 --2- 172.21.15.84:0/2431660354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b6815ad70 0x7f0b680c8310 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:37.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.180+0000 7f0b702cf640 1 --2- 172.21.15.84:0/2431660354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b680c9ca0 0x7f0b680c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:37.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.180+0000 7f0b702cf640 1 --2- 172.21.15.84:0/2431660354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b6806a4b0 0x7f0b681f3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:37.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.180+0000 7f0b702cf640 1 -- 172.21.15.84:0/2431660354 >> 172.21.15.84:0/2431660354 conn(0x7f0b680ce9b0 msgr2=0x7f0b6815d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:37.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.181+0000 7f0b702cf640 1 -- 172.21.15.84:0/2431660354 shutdown_connections 2023-10-16T11:24:37.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.181+0000 7f0b702cf640 1 -- 172.21.15.84:0/2431660354 wait complete. 2023-10-16T11:24:37.201 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:24:37.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.412+0000 7f91e24f8640 1 Processor -- start 2023-10-16T11:24:37.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.412+0000 7f91e24f8640 1 -- start start 2023-10-16T11:24:37.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.414+0000 7f91e24f8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f91dc06c040 0x7f91dc06c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:37.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.414+0000 7f91e24f8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91dc0cda80 0x7f91dc15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:37.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.414+0000 7f91e24f8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91dc06c960 con 0x7f91dc0ad970 2023-10-16T11:24:37.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.414+0000 7f91e24f8640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91dc15b9d0 con 0x7f91dc06c040 2023-10-16T11:24:37.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.414+0000 7f91e24f8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc15bcb0 con 0x7f91dc0cda80 2023-10-16T11:24:37.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.414+0000 7f91e14f6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f91dc06c040 0x7f91dc06c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:37.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.414+0000 7f91e14f6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f91dc06c040 0x7f91dc06c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49936/0 (socket says 172.21.15.84:49936) 2023-10-16T11:24:37.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.414+0000 7f91e14f6640 1 -- 172.21.15.84:0/1289684332 learned_addr learned my addr 172.21.15.84:0/1289684332 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:37.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.414+0000 7f91e0cf5640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f91dc0ad970 0x7f91dc15e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38482/0 (socket says 172.21.15.84:38482) 2023-10-16T11:24:37.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.415+0000 7f91e1cf7640 1 --2- 172.21.15.84:0/1289684332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91dc0cda80 0x7f91dc15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:37.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.415+0000 7f91e14f6640 1 -- 172.21.15.84:0/1289684332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91dc0cda80 msgr2=0x7f91dc15ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:37.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.415+0000 7f91e14f6640 1 --2- 172.21.15.84:0/1289684332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91dc0cda80 0x7f91dc15ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:37.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.415+0000 7f91e14f6640 1 -- 172.21.15.84:0/1289684332 >> v1:172.21.15.84:6789/0 conn(0x7f91dc0ad970 legacy=0x7f91dc15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:37.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.415+0000 7f91e14f6640 1 -- 172.21.15.84:0/1289684332 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc15bf90 con 0x7f91dc06c040 2023-10-16T11:24:37.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.415+0000 7f91d3fff640 1 -- 172.21.15.84:0/1289684332 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140192775 0 0) 0x7f91dc06c960 con 0x7f91dc0ad970 2023-10-16T11:24:37.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.416+0000 7f91e14f6640 1 --2- 172.21.15.84:0/1289684332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91dc06c040 0x7f91dc06c420 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009870 tx=0x7f91d802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e91a5e3de000e1a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:37.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.416+0000 7f91d3fff640 1 -- 172.21.15.84:0/1289684332 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8030840 con 0x7f91dc06c040 2023-10-16T11:24:37.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.416+0000 7f91d3fff640 1 -- 172.21.15.84:0/1289684332 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d80309a0 con 0x7f91dc06c040 2023-10-16T11:24:37.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.416+0000 7f91d3fff640 1 -- 172.21.15.84:0/1289684332 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8030d30 con 0x7f91dc06c040 2023-10-16T11:24:37.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.417+0000 7f91e24f8640 1 -- 172.21.15.84:0/1289684332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91dc06c040 msgr2=0x7f91dc06c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:37.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.417+0000 7f91e24f8640 1 --2- 172.21.15.84:0/1289684332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91dc06c040 0x7f91dc06c420 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009870 tx=0x7f91d802b980 comp rx=0 tx=0).stop 2023-10-16T11:24:37.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.417+0000 7f91e24f8640 1 -- 172.21.15.84:0/1289684332 shutdown_connections 2023-10-16T11:24:37.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.417+0000 7f91e24f8640 1 --2- 172.21.15.84:0/1289684332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91dc0cda80 0x7f91dc15ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:37.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.417+0000 7f91e24f8640 1 --2- 172.21.15.84:0/1289684332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91dc06c040 0x7f91dc06c420 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:37.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.417+0000 7f91e24f8640 1 -- 172.21.15.84:0/1289684332 >> 172.21.15.84:0/1289684332 conn(0x7f91dc0ce6a0 msgr2=0x7f91dc0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:37.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.417+0000 7f91e24f8640 1 -- 172.21.15.84:0/1289684332 shutdown_connections 2023-10-16T11:24:37.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.417+0000 7f91e24f8640 1 -- 172.21.15.84:0/1289684332 wait complete. 2023-10-16T11:24:37.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.419+0000 7f91e24f8640 1 Processor -- start 2023-10-16T11:24:37.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.419+0000 7f91e24f8640 1 -- start start 2023-10-16T11:24:37.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.420+0000 7f91e24f8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91dc06c040 0x7f91dc0c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:37.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.420+0000 7f91e24f8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91dc0ad970 0x7f91dc150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:37.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.420+0000 7f91e24f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91dc0cda80 0x7f91dc154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:37.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.420+0000 7f91e24f8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc1f0260 con 0x7f91dc0ad970 2023-10-16T11:24:37.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.420+0000 7f91e24f8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc155290 con 0x7f91dc0cda80 2023-10-16T11:24:37.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.420+0000 7f91e24f8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc155530 con 0x7f91dc06c040 2023-10-16T11:24:37.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.420+0000 7f91e0cf5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91dc0ad970 0x7f91dc150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:37.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.420+0000 7f91e0cf5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91dc0ad970 0x7f91dc150770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49952/0 (socket says 172.21.15.84:49952) 2023-10-16T11:24:37.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.420+0000 7f91e0cf5640 1 -- 172.21.15.84:0/1929500610 learned_addr learned my addr 172.21.15.84:0/1929500610 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:37.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.421+0000 7f91e1cf7640 1 --2- 172.21.15.84:0/1929500610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91dc0cda80 0x7f91dc154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:37.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.421+0000 7f91e0cf5640 1 -- 172.21.15.84:0/1929500610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91dc06c040 msgr2=0x7f91dc0c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:37.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.421+0000 7f91e0cf5640 1 --2- 172.21.15.84:0/1929500610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91dc06c040 0x7f91dc0c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:37.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.421+0000 7f91e0cf5640 1 -- 172.21.15.84:0/1929500610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91dc0cda80 msgr2=0x7f91dc154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:37.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.421+0000 7f91e0cf5640 1 --2- 172.21.15.84:0/1929500610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91dc0cda80 0x7f91dc154d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:37.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.421+0000 7f91e0cf5640 1 -- 172.21.15.84:0/1929500610 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d80094d0 con 0x7f91dc0ad970 2023-10-16T11:24:37.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.421+0000 7f91e0cf5640 1 --2- 172.21.15.84:0/1929500610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91dc0ad970 0x7f91dc150770 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00c4b0 tx=0x7f91cc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:37.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.421+0000 7f91d1ffb640 1 -- 172.21.15.84:0/1929500610 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0074e0 con 0x7f91dc0ad970 2023-10-16T11:24:37.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.421+0000 7f91d1ffb640 1 -- 172.21.15.84:0/1929500610 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc007660 con 0x7f91dc0ad970 2023-10-16T11:24:37.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.421+0000 7f91e24f8640 1 -- 172.21.15.84:0/1929500610 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91dc155810 con 0x7f91dc0ad970 2023-10-16T11:24:37.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.422+0000 7f91d1ffb640 1 -- 172.21.15.84:0/1929500610 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc004a20 con 0x7f91dc0ad970 2023-10-16T11:24:37.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.422+0000 7f91e24f8640 1 -- 172.21.15.84:0/1929500610 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91dc155dd0 con 0x7f91dc0ad970 2023-10-16T11:24:37.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.423+0000 7f91e24f8640 1 -- 172.21.15.84:0/1929500610 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91dc06b330 con 0x7f91dc0ad970 2023-10-16T11:24:37.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.424+0000 7f91d1ffb640 1 -- 172.21.15.84:0/1929500610 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f91cc004b80 con 0x7f91dc0ad970 2023-10-16T11:24:37.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.425+0000 7f91d1ffb640 1 --2- 172.21.15.84:0/1929500610 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f91c006afc0 0x7f91c006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:37.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.425+0000 7f91d1ffb640 1 -- 172.21.15.84:0/1929500610 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f91cc088ce0 con 0x7f91dc0ad970 2023-10-16T11:24:37.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.426+0000 7f91e14f6640 1 --2- 172.21.15.84:0/1929500610 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f91c006afc0 0x7f91c006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:37.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.426+0000 7f91e14f6640 1 --2- 172.21.15.84:0/1929500610 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f91c006afc0 0x7f91c006d480 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f91d8036000 tx=0x7f91d803b040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:37.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.430+0000 7f91d1ffb640 1 -- 172.21.15.84:0/1929500610 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f91cc053c80 con 0x7f91dc0ad970 2023-10-16T11:24:37.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.606+0000 7f91e24f8640 1 -- 172.21.15.84:0/1929500610 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91dc04e9b0 con 0x7f91dc0ad970 2023-10-16T11:24:37.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.607+0000 7f91d1ffb640 1 -- 172.21.15.84:0/1929500610 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v217) v1 ==== 100+0+11 (secure 0 0 0) 0x7f91cc018020 con 0x7f91dc0ad970 2023-10-16T11:24:37.610 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:24:37.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.609+0000 7f91e24f8640 1 -- 172.21.15.84:0/1929500610 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f91c006afc0 msgr2=0x7f91c006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:37.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.609+0000 7f91e24f8640 1 --2- 172.21.15.84:0/1929500610 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f91c006afc0 0x7f91c006d480 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f91d8036000 tx=0x7f91d803b040 comp rx=0 tx=0).stop 2023-10-16T11:24:37.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.610+0000 7f91e24f8640 1 -- 172.21.15.84:0/1929500610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91dc0ad970 msgr2=0x7f91dc150770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:37.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.610+0000 7f91e24f8640 1 --2- 172.21.15.84:0/1929500610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91dc0ad970 0x7f91dc150770 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00c4b0 tx=0x7f91cc00c970 comp rx=0 tx=0).stop 2023-10-16T11:24:37.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.610+0000 7f91e24f8640 1 -- 172.21.15.84:0/1929500610 shutdown_connections 2023-10-16T11:24:37.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.610+0000 7f91e24f8640 1 --2- 172.21.15.84:0/1929500610 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f91c006afc0 0x7f91c006d480 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:37.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.610+0000 7f91e24f8640 1 --2- 172.21.15.84:0/1929500610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91dc0cda80 0x7f91dc154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:37.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.610+0000 7f91e24f8640 1 --2- 172.21.15.84:0/1929500610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91dc0ad970 0x7f91dc150770 unknown :-1 s=CLOSED pgs=2655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:37.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.610+0000 7f91e24f8640 1 --2- 172.21.15.84:0/1929500610 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91dc06c040 0x7f91dc0c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:37.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.610+0000 7f91e24f8640 1 -- 172.21.15.84:0/1929500610 >> 172.21.15.84:0/1929500610 conn(0x7f91dc0ce6a0 msgr2=0x7f91dc0b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:37.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.610+0000 7f91e24f8640 1 -- 172.21.15.84:0/1929500610 shutdown_connections 2023-10-16T11:24:37.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.610+0000 7f91e24f8640 1 -- 172.21.15.84:0/1929500610 wait complete. 2023-10-16T11:24:37.630 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T11:24:37.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.840+0000 7f41a94a3640 1 Processor -- start 2023-10-16T11:24:37.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.841+0000 7f41a94a3640 1 -- start start 2023-10-16T11:24:37.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.842+0000 7f41a94a3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41a406c040 0x7f41a406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:37.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.842+0000 7f41a94a3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f41a40cda80 0x7f41a415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:37.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.842+0000 7f41a94a3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a406c960 con 0x7f41a40ad970 2023-10-16T11:24:37.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.842+0000 7f41a94a3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a415b9d0 con 0x7f41a40cda80 2023-10-16T11:24:37.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.842+0000 7f41a94a3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a415bcb0 con 0x7f41a406c040 2023-10-16T11:24:37.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.843+0000 7f41a8ca2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f41a40cda80 0x7f41a415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:37.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.843+0000 7f41a8ca2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f41a40cda80 0x7f41a415ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:59856/0 (socket says 172.21.15.84:59856) 2023-10-16T11:24:37.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.843+0000 7f41a8ca2640 1 -- 172.21.15.84:0/3831030426 learned_addr learned my addr 172.21.15.84:0/3831030426 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:37.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.843+0000 7f41a3fff640 1 --2- 172.21.15.84:0/3831030426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41a406c040 0x7f41a406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:37.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.844+0000 7f41a8ca2640 1 -- 172.21.15.84:0/3831030426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41a406c040 msgr2=0x7f41a406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:37.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.844+0000 7f41a8ca2640 1 --2- 172.21.15.84:0/3831030426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41a406c040 0x7f41a406c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:37.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.844+0000 7f41a8ca2640 1 -- 172.21.15.84:0/3831030426 >> v1:172.21.15.157:6789/0 conn(0x7f41a40ad970 legacy=0x7f41a415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:37.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.844+0000 7f41a8ca2640 1 -- 172.21.15.84:0/3831030426 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a415bf90 con 0x7f41a40cda80 2023-10-16T11:24:37.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.845+0000 7f41a8ca2640 1 --2- 172.21.15.84:0/3831030426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41a40cda80 0x7f41a415ed90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f419c009e50 tx=0x7f419c02bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5129ca4bd8c2b90 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:37.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.846+0000 7f41a37fe640 1 -- 172.21.15.84:0/3831030426 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c02caf0 con 0x7f41a40cda80 2023-10-16T11:24:37.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.846+0000 7f41a37fe640 1 -- 172.21.15.84:0/3831030426 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419c02cc50 con 0x7f41a40cda80 2023-10-16T11:24:37.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.846+0000 7f41a37fe640 1 -- 172.21.15.84:0/3831030426 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c0343f0 con 0x7f41a40cda80 2023-10-16T11:24:37.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.847+0000 7f41a94a3640 1 -- 172.21.15.84:0/3831030426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41a40cda80 msgr2=0x7f41a415ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:37.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.847+0000 7f41a94a3640 1 --2- 172.21.15.84:0/3831030426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41a40cda80 0x7f41a415ed90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f419c009e50 tx=0x7f419c02bf50 comp rx=0 tx=0).stop 2023-10-16T11:24:37.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.847+0000 7f41a94a3640 1 -- 172.21.15.84:0/3831030426 shutdown_connections 2023-10-16T11:24:37.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.847+0000 7f41a94a3640 1 --2- 172.21.15.84:0/3831030426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41a40cda80 0x7f41a415ed90 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:37.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.847+0000 7f41a94a3640 1 --2- 172.21.15.84:0/3831030426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41a406c040 0x7f41a406c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:37.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.847+0000 7f41a94a3640 1 -- 172.21.15.84:0/3831030426 >> 172.21.15.84:0/3831030426 conn(0x7f41a40ce6a0 msgr2=0x7f41a40ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:37.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.847+0000 7f41a94a3640 1 -- 172.21.15.84:0/3831030426 shutdown_connections 2023-10-16T11:24:37.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.847+0000 7f41a94a3640 1 -- 172.21.15.84:0/3831030426 wait complete. 2023-10-16T11:24:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.848+0000 7f41a94a3640 1 Processor -- start 2023-10-16T11:24:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.849+0000 7f41a94a3640 1 -- start start 2023-10-16T11:24:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.850+0000 7f41a94a3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41a406c040 0x7f41a40c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.850+0000 7f41a94a3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41a40ad970 0x7f41a41447e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.850+0000 7f41a94a3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41a40cda80 0x7f41a4152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:37.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.850+0000 7f41a94a3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a4144d20 con 0x7f41a40ad970 2023-10-16T11:24:37.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.850+0000 7f41a94a3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a40cacd0 con 0x7f41a406c040 2023-10-16T11:24:37.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.850+0000 7f41a94a3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a40c9c30 con 0x7f41a40cda80 2023-10-16T11:24:37.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.850+0000 7f419bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41a40ad970 0x7f41a41447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:37.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.850+0000 7f419bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41a40ad970 0x7f41a41447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49966/0 (socket says 172.21.15.84:49966) 2023-10-16T11:24:37.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.850+0000 7f419bfff640 1 -- 172.21.15.84:0/2860153633 learned_addr learned my addr 172.21.15.84:0/2860153633 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:37.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.850+0000 7f41a3fff640 1 --2- 172.21.15.84:0/2860153633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41a406c040 0x7f41a40c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:37.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.850+0000 7f419bfff640 1 -- 172.21.15.84:0/2860153633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41a40cda80 msgr2=0x7f41a4152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:37.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.850+0000 7f41a8ca2640 1 --2- 172.21.15.84:0/2860153633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41a40cda80 0x7f41a4152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:37.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.850+0000 7f419bfff640 1 --2- 172.21.15.84:0/2860153633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41a40cda80 0x7f41a4152550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.850+0000 7f419bfff640 1 -- 172.21.15.84:0/2860153633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41a406c040 msgr2=0x7f41a40c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.850+0000 7f419bfff640 1 --2- 172.21.15.84:0/2860153633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41a406c040 0x7f41a40c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.850+0000 7f419bfff640 1 -- 172.21.15.84:0/2860153633 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419c009b30 con 0x7f41a40ad970 2023-10-16T11:24:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.850+0000 7f41a3fff640 1 --2- 172.21.15.84:0/2860153633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41a406c040 0x7f41a40c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.851+0000 7f419bfff640 1 --2- 172.21.15.84:0/2860153633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41a40ad970 0x7f41a41447e0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7f418c00c400 tx=0x7f418c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:37.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.851+0000 7f41a17fa640 1 -- 172.21.15.84:0/2860153633 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c0052d0 con 0x7f41a40ad970 2023-10-16T11:24:37.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.851+0000 7f41a17fa640 1 -- 172.21.15.84:0/2860153633 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418c005430 con 0x7f41a40ad970 2023-10-16T11:24:37.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.851+0000 7f41a17fa640 1 -- 172.21.15.84:0/2860153633 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c003830 con 0x7f41a40ad970 2023-10-16T11:24:37.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.851+0000 7f41a94a3640 1 -- 172.21.15.84:0/2860153633 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a40c9f10 con 0x7f41a40ad970 2023-10-16T11:24:37.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.851+0000 7f41a94a3640 1 -- 172.21.15.84:0/2860153633 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a40ca4d0 con 0x7f41a40ad970 2023-10-16T11:24:37.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.852+0000 7f41a94a3640 1 -- 172.21.15.84:0/2860153633 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a406b330 con 0x7f41a40ad970 2023-10-16T11:24:37.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.853+0000 7f41a17fa640 1 -- 172.21.15.84:0/2860153633 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f418c003990 con 0x7f41a40ad970 2023-10-16T11:24:37.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.854+0000 7f41a17fa640 1 --2- 172.21.15.84:0/2860153633 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f418406af90 0x7f418406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:37.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.854+0000 7f41a17fa640 1 -- 172.21.15.84:0/2860153633 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f418c088d70 con 0x7f41a40ad970 2023-10-16T11:24:37.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.854+0000 7f41a3fff640 1 --2- 172.21.15.84:0/2860153633 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f418406af90 0x7f418406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:37.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.855+0000 7f41a3fff640 1 --2- 172.21.15.84:0/2860153633 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f418406af90 0x7f418406d450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4194009a20 tx=0x7f4194008040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:37.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:37.859+0000 7f41a17fa640 1 -- 172.21.15.84:0/2860153633 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f418c053ce0 con 0x7f41a40ad970 2023-10-16T11:24:38.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.038+0000 7f41a94a3640 1 -- 172.21.15.84:0/2860153633 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f41a404e9b0 con 0x7f41a40ad970 2023-10-16T11:24:38.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.039+0000 7f41a17fa640 1 -- 172.21.15.84:0/2860153633 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v217) v1 ==== 84+0+209 (secure 0 0 0) 0x7f418c017090 con 0x7f41a40ad970 2023-10-16T11:24:38.041 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:38.041 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T11:24:38.042 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:38.042 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-16T11:24:38.042 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-16T11:24:38.042 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:38.042 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:38.043 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 26, 2023-10-16T11:24:38.043 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": "cephfs_metadata" 2023-10-16T11:24:38.043 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:38.043 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:38.043 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 27, 2023-10-16T11:24:38.043 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": "cephfs_data" 2023-10-16T11:24:38.044 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:38.044 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T11:24:38.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.041+0000 7f41a94a3640 1 -- 172.21.15.84:0/2860153633 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f418406af90 msgr2=0x7f418406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:38.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.041+0000 7f41a94a3640 1 --2- 172.21.15.84:0/2860153633 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f418406af90 0x7f418406d450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4194009a20 tx=0x7f4194008040 comp rx=0 tx=0).stop 2023-10-16T11:24:38.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.042+0000 7f41a94a3640 1 -- 172.21.15.84:0/2860153633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41a40ad970 msgr2=0x7f41a41447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:38.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.042+0000 7f41a94a3640 1 --2- 172.21.15.84:0/2860153633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41a40ad970 0x7f41a41447e0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7f418c00c400 tx=0x7f418c00c8c0 comp rx=0 tx=0).stop 2023-10-16T11:24:38.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.042+0000 7f41a94a3640 1 -- 172.21.15.84:0/2860153633 shutdown_connections 2023-10-16T11:24:38.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.042+0000 7f41a94a3640 1 --2- 172.21.15.84:0/2860153633 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f418406af90 0x7f418406d450 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:38.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.042+0000 7f41a94a3640 1 --2- 172.21.15.84:0/2860153633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41a40cda80 0x7f41a4152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:38.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.042+0000 7f41a94a3640 1 --2- 172.21.15.84:0/2860153633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41a40ad970 0x7f41a41447e0 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:38.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.042+0000 7f41a94a3640 1 --2- 172.21.15.84:0/2860153633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41a406c040 0x7f41a40c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:38.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.042+0000 7f41a94a3640 1 -- 172.21.15.84:0/2860153633 >> 172.21.15.84:0/2860153633 conn(0x7f41a40ce6a0 msgr2=0x7f41a4159d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:38.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.042+0000 7f41a94a3640 1 -- 172.21.15.84:0/2860153633 shutdown_connections 2023-10-16T11:24:38.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.042+0000 7f41a94a3640 1 -- 172.21.15.84:0/2860153633 wait complete. 2023-10-16T11:24:38.061 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2023-10-16T11:24:38.062 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2023-10-16T11:24:38.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.265+0000 7f1580533640 1 Processor -- start 2023-10-16T11:24:38.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.265+0000 7f1580533640 1 -- start start 2023-10-16T11:24:38.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.266+0000 7f1580533640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f15780c9ab0 0x7f15780c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:38.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.266+0000 7f1580533640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f15780b0360 0x7f15780b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:38.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.266+0000 7f1580533640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15780b0cf0 con 0x7f157815cf60 2023-10-16T11:24:38.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.266+0000 7f1580533640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f157815d340 con 0x7f15780b0360 2023-10-16T11:24:38.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.266+0000 7f1580533640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15780cc860 con 0x7f15780c9ab0 2023-10-16T11:24:38.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.267+0000 7f157e2cf640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f157815cf60 0x7f157806f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38490/0 (socket says 172.21.15.84:38490) 2023-10-16T11:24:38.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.267+0000 7f157e2cf640 1 -- 172.21.15.84:0/2982584938 learned_addr learned my addr 172.21.15.84:0/2982584938 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:38.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.267+0000 7f157ead0640 1 --2- 172.21.15.84:0/2982584938 >> v2:172.21.15.84:3300/0 conn(0x7f15780b0360 0x7f15780b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:38.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.267+0000 7f157dace640 1 --2- 172.21.15.84:0/2982584938 >> v2:172.21.15.157:3300/0 conn(0x7f15780c9ab0 0x7f15780c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:38.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.267+0000 7f157d2cd640 1 -- 172.21.15.84:0/2982584938 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713779490 0 0) 0x7f1568001490 con 0x7f157815cf60 2023-10-16T11:24:38.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.267+0000 7f157d2cd640 1 -- 172.21.15.84:0/2982584938 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1560003520 con 0x7f157815cf60 2023-10-16T11:24:38.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.267+0000 7f157ead0640 1 -- 172.21.15.84:0/2982584938 >> v2:172.21.15.157:3300/0 conn(0x7f15780c9ab0 msgr2=0x7f15780c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:38.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.267+0000 7f157ead0640 1 --2- 172.21.15.84:0/2982584938 >> v2:172.21.15.157:3300/0 conn(0x7f15780c9ab0 0x7f15780c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:38.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.267+0000 7f157ead0640 1 -- 172.21.15.84:0/2982584938 >> v1:172.21.15.84:6789/0 conn(0x7f157815cf60 legacy=0x7f157806f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:38.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.267+0000 7f157ead0640 1 -- 172.21.15.84:0/2982584938 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15780c9620 con 0x7f15780b0360 2023-10-16T11:24:38.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.268+0000 7f157dace640 1 --2- 172.21.15.84:0/2982584938 >> v2:172.21.15.157:3300/0 conn(0x7f15780c9ab0 0x7f15780c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:38.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.269+0000 7f157ead0640 1 --2- 172.21.15.84:0/2982584938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15780b0360 0x7f15780b07b0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7f15700097a0 tx=0x7f157002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83505bb5142a738f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:38.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.269+0000 7f157d2cd640 1 -- 172.21.15.84:0/2982584938 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1570031830 con 0x7f15780b0360 2023-10-16T11:24:38.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.269+0000 7f157d2cd640 1 -- 172.21.15.84:0/2982584938 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1570031990 con 0x7f15780b0360 2023-10-16T11:24:38.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.269+0000 7f157d2cd640 1 -- 172.21.15.84:0/2982584938 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1570031d20 con 0x7f15780b0360 2023-10-16T11:24:38.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.269+0000 7f1580533640 1 -- 172.21.15.84:0/2982584938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15780b0360 msgr2=0x7f15780b07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:38.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.269+0000 7f1580533640 1 --2- 172.21.15.84:0/2982584938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15780b0360 0x7f15780b07b0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7f15700097a0 tx=0x7f157002b8d0 comp rx=0 tx=0).stop 2023-10-16T11:24:38.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.270+0000 7f1580533640 1 -- 172.21.15.84:0/2982584938 shutdown_connections 2023-10-16T11:24:38.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.270+0000 7f1580533640 1 --2- 172.21.15.84:0/2982584938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15780b0360 0x7f15780b07b0 unknown :-1 s=CLOSED pgs=2658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:38.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.270+0000 7f1580533640 1 --2- 172.21.15.84:0/2982584938 >> v2:172.21.15.157:3300/0 conn(0x7f15780c9ab0 0x7f15780c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:38.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.270+0000 7f1580533640 1 -- 172.21.15.84:0/2982584938 >> 172.21.15.84:0/2982584938 conn(0x7f15780cb360 msgr2=0x7f15780cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:38.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.270+0000 7f1580533640 1 -- 172.21.15.84:0/2982584938 shutdown_connections 2023-10-16T11:24:38.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.270+0000 7f1580533640 1 -- 172.21.15.84:0/2982584938 wait complete. 2023-10-16T11:24:38.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.272+0000 7f1580533640 1 Processor -- start 2023-10-16T11:24:38.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.272+0000 7f1580533640 1 -- start start 2023-10-16T11:24:38.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.274+0000 7f1580533640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15780b0360 0x7f157814adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:38.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.274+0000 7f1580533640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15780c9ab0 0x7f15781f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:38.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.274+0000 7f1580533640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f157815cf60 0x7f15781f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:38.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.274+0000 7f1580533640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15780c5200 con 0x7f15780b0360 2023-10-16T11:24:38.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.274+0000 7f1580533640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157814b2f0 con 0x7f15780c9ab0 2023-10-16T11:24:38.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.274+0000 7f1580533640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15780c4580 con 0x7f157815cf60 2023-10-16T11:24:38.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.274+0000 7f157e2cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15780b0360 0x7f157814adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:38.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.274+0000 7f157e2cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15780b0360 0x7f157814adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49978/0 (socket says 172.21.15.84:49978) 2023-10-16T11:24:38.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.274+0000 7f157e2cf640 1 -- 172.21.15.84:0/3259689637 learned_addr learned my addr 172.21.15.84:0/3259689637 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:38.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.274+0000 7f157ead0640 1 --2- 172.21.15.84:0/3259689637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f157815cf60 0x7f15781f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:38.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.274+0000 7f157dace640 1 --2- 172.21.15.84:0/3259689637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15780c9ab0 0x7f15781f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:38.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.274+0000 7f157e2cf640 1 -- 172.21.15.84:0/3259689637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f157815cf60 msgr2=0x7f15781f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:38.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.274+0000 7f157e2cf640 1 --2- 172.21.15.84:0/3259689637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f157815cf60 0x7f15781f6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:38.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.274+0000 7f157e2cf640 1 -- 172.21.15.84:0/3259689637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15780c9ab0 msgr2=0x7f15781f5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:38.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.274+0000 7f157e2cf640 1 --2- 172.21.15.84:0/3259689637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15780c9ab0 0x7f15781f5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:38.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.274+0000 7f157e2cf640 1 -- 172.21.15.84:0/3259689637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1570009400 con 0x7f15780b0360 2023-10-16T11:24:38.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.275+0000 7f157e2cf640 1 --2- 172.21.15.84:0/3259689637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15780b0360 0x7f157814adb0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f156800cbc0 tx=0x7f156800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:38.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.275+0000 7f1576ffd640 1 -- 172.21.15.84:0/3259689637 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156800ad40 con 0x7f15780b0360 2023-10-16T11:24:38.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.275+0000 7f1576ffd640 1 -- 172.21.15.84:0/3259689637 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1568012400 con 0x7f15780b0360 2023-10-16T11:24:38.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.275+0000 7f1576ffd640 1 -- 172.21.15.84:0/3259689637 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156800aea0 con 0x7f15780b0360 2023-10-16T11:24:38.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.275+0000 7f1580533640 1 -- 172.21.15.84:0/3259689637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15780c4860 con 0x7f15780b0360 2023-10-16T11:24:38.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.275+0000 7f1580533640 1 -- 172.21.15.84:0/3259689637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15780c4e20 con 0x7f15780b0360 2023-10-16T11:24:38.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.276+0000 7f1580533640 1 -- 172.21.15.84:0/3259689637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15780b0ae0 con 0x7f15780b0360 2023-10-16T11:24:38.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.277+0000 7f1576ffd640 1 -- 172.21.15.84:0/3259689637 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1568007070 con 0x7f15780b0360 2023-10-16T11:24:38.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.278+0000 7f1576ffd640 1 --2- 172.21.15.84:0/3259689637 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f156406afc0 0x7f156406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:38.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.278+0000 7f1576ffd640 1 -- 172.21.15.84:0/3259689637 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1568089660 con 0x7f15780b0360 2023-10-16T11:24:38.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.278+0000 7f157dace640 1 --2- 172.21.15.84:0/3259689637 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f156406afc0 0x7f156406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:38.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.279+0000 7f157dace640 1 --2- 172.21.15.84:0/3259689637 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f156406afc0 0x7f156406d480 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f15780b3f60 tx=0x7f156c008040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:38.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.282+0000 7f1576ffd640 1 -- 172.21.15.84:0/3259689637 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f15680545f0 con 0x7f15780b0360 2023-10-16T11:24:38.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.472+0000 7f1580533640 1 -- 172.21.15.84:0/3259689637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f15780b0cf0 con 0x7f15780b0360 2023-10-16T11:24:38.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.877+0000 7f1576ffd640 1 -- 172.21.15.84:0/3259689637 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v218) v1 ==== 144+0+0 (secure 0 0 0) 0x7f15780b0cf0 con 0x7f15780b0360 2023-10-16T11:24:38.879 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'backup_fs_metadata' created 2023-10-16T11:24:38.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.881+0000 7f1580533640 1 -- 172.21.15.84:0/3259689637 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f156406afc0 msgr2=0x7f156406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:38.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.881+0000 7f1580533640 1 --2- 172.21.15.84:0/3259689637 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f156406afc0 0x7f156406d480 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f15780b3f60 tx=0x7f156c008040 comp rx=0 tx=0).stop 2023-10-16T11:24:38.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.881+0000 7f1580533640 1 -- 172.21.15.84:0/3259689637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15780b0360 msgr2=0x7f157814adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:38.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.881+0000 7f1580533640 1 --2- 172.21.15.84:0/3259689637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15780b0360 0x7f157814adb0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f156800cbc0 tx=0x7f156800a430 comp rx=0 tx=0).stop 2023-10-16T11:24:38.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.883+0000 7f1580533640 1 -- 172.21.15.84:0/3259689637 shutdown_connections 2023-10-16T11:24:38.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.883+0000 7f1580533640 1 --2- 172.21.15.84:0/3259689637 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f156406afc0 0x7f156406d480 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:38.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.883+0000 7f1580533640 1 --2- 172.21.15.84:0/3259689637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f157815cf60 0x7f15781f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:38.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.883+0000 7f1580533640 1 --2- 172.21.15.84:0/3259689637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15780c9ab0 0x7f15781f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:38.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.883+0000 7f1580533640 1 --2- 172.21.15.84:0/3259689637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15780b0360 0x7f157814adb0 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:38.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.883+0000 7f1580533640 1 -- 172.21.15.84:0/3259689637 >> 172.21.15.84:0/3259689637 conn(0x7f15780cb360 msgr2=0x7f157806b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:38.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.884+0000 7f1580533640 1 -- 172.21.15.84:0/3259689637 shutdown_connections 2023-10-16T11:24:38.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:38.884+0000 7f1580533640 1 -- 172.21.15.84:0/3259689637 wait complete. 2023-10-16T11:24:38.911 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T11:24:39.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.118+0000 7ff7d8b47640 1 Processor -- start 2023-10-16T11:24:39.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.119+0000 7ff7d8b47640 1 -- start start 2023-10-16T11:24:39.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.120+0000 7ff7d8b47640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff7d4159f00 0x7ff7d40b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:39.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.120+0000 7ff7d8b47640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7d415d8c0 0x7ff7d40b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:39.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.120+0000 7ff7d8b47640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d406f890 con 0x7ff7d4159b20 2023-10-16T11:24:39.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.120+0000 7ff7d8b47640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d4070d30 con 0x7ff7d4159f00 2023-10-16T11:24:39.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.120+0000 7ff7d8b47640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d406c650 con 0x7ff7d415d8c0 2023-10-16T11:24:39.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.120+0000 7ff7d259c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff7d4159b20 0x7ff7d406bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38498/0 (socket says 172.21.15.84:38498) 2023-10-16T11:24:39.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.120+0000 7ff7d259c640 1 -- 172.21.15.84:0/4038887786 learned_addr learned my addr 172.21.15.84:0/4038887786 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:39.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.120+0000 7ff7d1d9b640 1 --2- 172.21.15.84:0/4038887786 >> v2:172.21.15.84:3300/0 conn(0x7ff7d4159f00 0x7ff7d40b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:39.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.121+0000 7ff7d2d9d640 1 --2- 172.21.15.84:0/4038887786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7d415d8c0 0x7ff7d40b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:39.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.121+0000 7ff7d159a640 1 -- 172.21.15.84:0/4038887786 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669474183 0 0) 0x7ff7d406f890 con 0x7ff7d4159b20 2023-10-16T11:24:39.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.121+0000 7ff7d159a640 1 -- 172.21.15.84:0/4038887786 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7c0005730 con 0x7ff7d4159b20 2023-10-16T11:24:39.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.121+0000 7ff7d1d9b640 1 -- 172.21.15.84:0/4038887786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7d415d8c0 msgr2=0x7ff7d40b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:39.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.121+0000 7ff7d1d9b640 1 --2- 172.21.15.84:0/4038887786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7d415d8c0 0x7ff7d40b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:39.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.121+0000 7ff7d1d9b640 1 -- 172.21.15.84:0/4038887786 >> v1:172.21.15.84:6789/0 conn(0x7ff7d4159b20 legacy=0x7ff7d406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:39.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.121+0000 7ff7d1d9b640 1 -- 172.21.15.84:0/4038887786 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d406c930 con 0x7ff7d4159f00 2023-10-16T11:24:39.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.122+0000 7ff7d1d9b640 1 --2- 172.21.15.84:0/4038887786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7d4159f00 0x7ff7d40b0410 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0097a0 tx=0x7ff7bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af275875c3a6b4f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:39.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.123+0000 7ff7d159a640 1 -- 172.21.15.84:0/4038887786 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc031830 con 0x7ff7d4159f00 2023-10-16T11:24:39.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.123+0000 7ff7d159a640 1 -- 172.21.15.84:0/4038887786 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7bc031990 con 0x7ff7d4159f00 2023-10-16T11:24:39.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.123+0000 7ff7d159a640 1 -- 172.21.15.84:0/4038887786 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc031d00 con 0x7ff7d4159f00 2023-10-16T11:24:39.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.123+0000 7ff7d8b47640 1 -- 172.21.15.84:0/4038887786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7d4159f00 msgr2=0x7ff7d40b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:39.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.123+0000 7ff7d8b47640 1 --2- 172.21.15.84:0/4038887786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7d4159f00 0x7ff7d40b0410 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0097a0 tx=0x7ff7bc02b8d0 comp rx=0 tx=0).stop 2023-10-16T11:24:39.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.123+0000 7ff7d8b47640 1 -- 172.21.15.84:0/4038887786 shutdown_connections 2023-10-16T11:24:39.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.123+0000 7ff7d8b47640 1 --2- 172.21.15.84:0/4038887786 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7d415d8c0 0x7ff7d40b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:39.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.123+0000 7ff7d8b47640 1 --2- 172.21.15.84:0/4038887786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7d4159f00 0x7ff7d40b0410 unknown :-1 s=CLOSED pgs=2661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:39.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.123+0000 7ff7d8b47640 1 -- 172.21.15.84:0/4038887786 >> 172.21.15.84:0/4038887786 conn(0x7ff7d40cb410 msgr2=0x7ff7d40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:39.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.124+0000 7ff7d8b47640 1 -- 172.21.15.84:0/4038887786 shutdown_connections 2023-10-16T11:24:39.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.124+0000 7ff7d8b47640 1 -- 172.21.15.84:0/4038887786 wait complete. 2023-10-16T11:24:39.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.125+0000 7ff7d8b47640 1 Processor -- start 2023-10-16T11:24:39.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.125+0000 7ff7d8b47640 1 -- start start 2023-10-16T11:24:39.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.126+0000 7ff7d8b47640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d4159b20 0x7ff7d4150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:39.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.126+0000 7ff7d8b47640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7d4159f00 0x7ff7d40c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:39.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.126+0000 7ff7d8b47640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7d415d8c0 0x7ff7d41500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:39.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.126+0000 7ff7d8b47640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40c4fc0 con 0x7ff7d4159f00 2023-10-16T11:24:39.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.126+0000 7ff7d8b47640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40c5450 con 0x7ff7d4159b20 2023-10-16T11:24:39.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.126+0000 7ff7d8b47640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40c2a00 con 0x7ff7d415d8c0 2023-10-16T11:24:39.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.127+0000 7ff7d1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7d4159f00 0x7ff7d40c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:39.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.127+0000 7ff7d1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7d4159f00 0x7ff7d40c24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49988/0 (socket says 172.21.15.84:49988) 2023-10-16T11:24:39.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.127+0000 7ff7d1d9b640 1 -- 172.21.15.84:0/3924941098 learned_addr learned my addr 172.21.15.84:0/3924941098 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:39.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.127+0000 7ff7d259c640 1 --2- 172.21.15.84:0/3924941098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d4159b20 0x7ff7d4150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:39.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.127+0000 7ff7d2d9d640 1 --2- 172.21.15.84:0/3924941098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7d415d8c0 0x7ff7d41500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:39.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.127+0000 7ff7d1d9b640 1 -- 172.21.15.84:0/3924941098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7d415d8c0 msgr2=0x7ff7d41500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:39.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.127+0000 7ff7d1d9b640 1 --2- 172.21.15.84:0/3924941098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7d415d8c0 0x7ff7d41500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:39.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.127+0000 7ff7d1d9b640 1 -- 172.21.15.84:0/3924941098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d4159b20 msgr2=0x7ff7d4150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:39.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.127+0000 7ff7d1d9b640 1 --2- 172.21.15.84:0/3924941098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d4159b20 0x7ff7d4150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:39.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.127+0000 7ff7d1d9b640 1 -- 172.21.15.84:0/3924941098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc009400 con 0x7ff7d4159f00 2023-10-16T11:24:39.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.128+0000 7ff7d1d9b640 1 --2- 172.21.15.84:0/3924941098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7d4159f00 0x7ff7d40c24c0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc02bee0 tx=0x7ff7bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:39.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.128+0000 7ff7caffd640 1 -- 172.21.15.84:0/3924941098 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc03a6f0 con 0x7ff7d4159f00 2023-10-16T11:24:39.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.128+0000 7ff7caffd640 1 -- 172.21.15.84:0/3924941098 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7bc03a850 con 0x7ff7d4159f00 2023-10-16T11:24:39.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.128+0000 7ff7d8b47640 1 -- 172.21.15.84:0/3924941098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d4150630 con 0x7ff7d4159f00 2023-10-16T11:24:39.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.128+0000 7ff7caffd640 1 -- 172.21.15.84:0/3924941098 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc03ac00 con 0x7ff7d4159f00 2023-10-16T11:24:39.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.128+0000 7ff7d8b47640 1 -- 172.21.15.84:0/3924941098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d40c85e0 con 0x7ff7d4159f00 2023-10-16T11:24:39.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.129+0000 7ff7d8b47640 1 -- 172.21.15.84:0/3924941098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d404e160 con 0x7ff7d4159f00 2023-10-16T11:24:39.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.131+0000 7ff7caffd640 1 -- 172.21.15.84:0/3924941098 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff7bc04c3b0 con 0x7ff7d4159f00 2023-10-16T11:24:39.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.132+0000 7ff7caffd640 1 --2- 172.21.15.84:0/3924941098 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff7b406af90 0x7ff7b406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:39.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.132+0000 7ff7caffd640 1 -- 172.21.15.84:0/3924941098 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6110+0+0 (secure 0 0 0) 0x7ff7bc0abb50 con 0x7ff7d4159f00 2023-10-16T11:24:39.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.132+0000 7ff7d259c640 1 --2- 172.21.15.84:0/3924941098 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff7b406af90 0x7ff7b406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:39.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.133+0000 7ff7d259c640 1 --2- 172.21.15.84:0/3924941098 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff7b406af90 0x7ff7b406d450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4005c00 tx=0x7ff7c4009040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:39.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.136+0000 7ff7caffd640 1 -- 172.21.15.84:0/3924941098 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff7bc0768c0 con 0x7ff7d4159f00 2023-10-16T11:24:39.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.330+0000 7ff7d8b47640 1 -- 172.21.15.84:0/3924941098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff7d40b10b0 con 0x7ff7d4159f00 2023-10-16T11:24:39.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.883+0000 7ff7caffd640 1 -- 172.21.15.84:0/3924941098 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v219) v1 ==== 176+0+0 (secure 0 0 0) 0x7ff7bc0350d0 con 0x7ff7d4159f00 2023-10-16T11:24:39.885 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'backup_fs_data' created 2023-10-16T11:24:39.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.888+0000 7ff7d8b47640 1 -- 172.21.15.84:0/3924941098 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff7b406af90 msgr2=0x7ff7b406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:39.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.888+0000 7ff7d8b47640 1 --2- 172.21.15.84:0/3924941098 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff7b406af90 0x7ff7b406d450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4005c00 tx=0x7ff7c4009040 comp rx=0 tx=0).stop 2023-10-16T11:24:39.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.888+0000 7ff7d8b47640 1 -- 172.21.15.84:0/3924941098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7d4159f00 msgr2=0x7ff7d40c24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:39.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.888+0000 7ff7d8b47640 1 --2- 172.21.15.84:0/3924941098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7d4159f00 0x7ff7d40c24c0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc02bee0 tx=0x7ff7bc0046c0 comp rx=0 tx=0).stop 2023-10-16T11:24:39.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.892+0000 7ff7d8b47640 1 -- 172.21.15.84:0/3924941098 shutdown_connections 2023-10-16T11:24:39.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.892+0000 7ff7d8b47640 1 --2- 172.21.15.84:0/3924941098 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff7b406af90 0x7ff7b406d450 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:39.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.892+0000 7ff7d8b47640 1 --2- 172.21.15.84:0/3924941098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7d415d8c0 0x7ff7d41500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:39.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.892+0000 7ff7d8b47640 1 --2- 172.21.15.84:0/3924941098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7d4159f00 0x7ff7d40c24c0 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:39.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.892+0000 7ff7d8b47640 1 --2- 172.21.15.84:0/3924941098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7d4159b20 0x7ff7d4150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:39.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.892+0000 7ff7d8b47640 1 -- 172.21.15.84:0/3924941098 >> 172.21.15.84:0/3924941098 conn(0x7ff7d40cb410 msgr2=0x7ff7d40b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:39.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.895+0000 7ff7d8b47640 1 -- 172.21.15.84:0/3924941098 shutdown_connections 2023-10-16T11:24:39.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:39.897+0000 7ff7d8b47640 1 -- 172.21.15.84:0/3924941098 wait complete. 2023-10-16T11:24:39.929 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2023-10-16T11:24:40.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.114+0000 7f9dfd0a2640 1 Processor -- start 2023-10-16T11:24:40.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.114+0000 7f9dfd0a2640 1 -- start start 2023-10-16T11:24:40.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.116+0000 7f9dfd0a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9df80ad350 0x7f9df8069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:40.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.116+0000 7f9dfd0a2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9df806a2e0 0x7f9df80c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:40.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.116+0000 7f9dfd0a2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df806a6f0 con 0x7f9df815d150 2023-10-16T11:24:40.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.116+0000 7f9dfd0a2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df815d530 con 0x7f9df806a2e0 2023-10-16T11:24:40.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.116+0000 7f9dfd0a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df80ac890 con 0x7f9df80ad350 2023-10-16T11:24:40.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.116+0000 7f9df6d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9df815d150 0x7f9df8160b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38504/0 (socket says 172.21.15.84:38504) 2023-10-16T11:24:40.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.116+0000 7f9df6d9d640 1 -- 172.21.15.84:0/2506419817 learned_addr learned my addr 172.21.15.84:0/2506419817 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:40.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.116+0000 7f9df759e640 1 --2- 172.21.15.84:0/2506419817 >> v2:172.21.15.157:3300/0 conn(0x7f9df806a2e0 0x7f9df80c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:40.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.116+0000 7f9df659c640 1 --2- 172.21.15.84:0/2506419817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9df80ad350 0x7f9df8069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:40.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.116+0000 7f9df5d9b640 1 -- 172.21.15.84:0/2506419817 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638188213 0 0) 0x7f9df806a6f0 con 0x7f9df815d150 2023-10-16T11:24:40.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.117+0000 7f9df5d9b640 1 -- 172.21.15.84:0/2506419817 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9de0003520 con 0x7f9df815d150 2023-10-16T11:24:40.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.117+0000 7f9df5d9b640 1 -- 172.21.15.84:0/2506419817 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2253909383 0 0) 0x7f9de0003520 con 0x7f9df815d150 2023-10-16T11:24:40.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.117+0000 7f9df5d9b640 1 -- 172.21.15.84:0/2506419817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9df80ad350 msgr2=0x7f9df8069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:40.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.117+0000 7f9df5d9b640 1 --2- 172.21.15.84:0/2506419817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9df80ad350 0x7f9df8069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:40.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.117+0000 7f9df5d9b640 1 -- 172.21.15.84:0/2506419817 >> v2:172.21.15.157:3300/0 conn(0x7f9df806a2e0 msgr2=0x7f9df80c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:40.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.117+0000 7f9df5d9b640 1 --2- 172.21.15.84:0/2506419817 >> v2:172.21.15.157:3300/0 conn(0x7f9df806a2e0 0x7f9df80c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:40.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.117+0000 7f9df5d9b640 1 -- 172.21.15.84:0/2506419817 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df8161650 con 0x7f9df815d150 2023-10-16T11:24:40.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.117+0000 7f9df5d9b640 1 -- 172.21.15.84:0/2506419817 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f9de8001670 con 0x7f9df815d150 2023-10-16T11:24:40.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.117+0000 7f9df5d9b640 1 -- 172.21.15.84:0/2506419817 >> v1:172.21.15.84:6789/0 conn(0x7f9df815d150 legacy=0x7f9df8160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:40.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.117+0000 7f9df5d9b640 1 --2- 172.21.15.84:0/2506419817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9de0003970 0x7f9de0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:40.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.117+0000 7f9df5d9b640 1 --2- 172.21.15.84:0/2506419817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9de00047c0 0x7f9de0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.117+0000 7f9df5d9b640 1 --2- 172.21.15.84:0/2506419817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9df80ad350 0x7f9de0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.117+0000 7f9df5d9b640 1 -- 172.21.15.84:0/2506419817 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df815d530 con 0x7f9de00047c0 2023-10-16T11:24:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.117+0000 7f9df5d9b640 1 -- 172.21.15.84:0/2506419817 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df80ac890 con 0x7f9de0003970 2023-10-16T11:24:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.117+0000 7f9df5d9b640 1 -- 172.21.15.84:0/2506419817 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de000b980 con 0x7f9df80ad350 2023-10-16T11:24:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.117+0000 7f9df659c640 1 --2- 172.21.15.84:0/2506419817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9de00047c0 0x7f9de0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:40.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.118+0000 7f9df6d9d640 1 --2- 172.21.15.84:0/2506419817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9de0003970 0x7f9de0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:40.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.118+0000 7f9df759e640 1 --2- 172.21.15.84:0/2506419817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9df80ad350 0x7f9de0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:40.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.118+0000 7f9df659c640 1 -- 172.21.15.84:0/2506419817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9df80ad350 msgr2=0x7f9de0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:40.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.118+0000 7f9df659c640 1 --2- 172.21.15.84:0/2506419817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9df80ad350 0x7f9de0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:40.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.118+0000 7f9df659c640 1 -- 172.21.15.84:0/2506419817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9de0003970 msgr2=0x7f9de0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:40.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.118+0000 7f9df659c640 1 --2- 172.21.15.84:0/2506419817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9de0003970 0x7f9de0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:40.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.118+0000 7f9df659c640 1 -- 172.21.15.84:0/2506419817 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de0003520 con 0x7f9de00047c0 2023-10-16T11:24:40.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.118+0000 7f9df759e640 1 --2- 172.21.15.84:0/2506419817 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9df80ad350 0x7f9de0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:40.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.119+0000 7f9df659c640 1 --2- 172.21.15.84:0/2506419817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9de00047c0 0x7f9de0004c40 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009ac0 tx=0x7f9de00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29c835338757ba2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:40.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.119+0000 7f9df5d9b640 1 -- 172.21.15.84:0/2506419817 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec02c6d0 con 0x7f9de00047c0 2023-10-16T11:24:40.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.119+0000 7f9df5d9b640 1 -- 172.21.15.84:0/2506419817 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec02c830 con 0x7f9de00047c0 2023-10-16T11:24:40.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.119+0000 7f9df5d9b640 1 -- 172.21.15.84:0/2506419817 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec02cbc0 con 0x7f9de00047c0 2023-10-16T11:24:40.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.120+0000 7f9dfd0a2640 1 -- 172.21.15.84:0/2506419817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9de00047c0 msgr2=0x7f9de0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:40.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.120+0000 7f9dfd0a2640 1 --2- 172.21.15.84:0/2506419817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9de00047c0 0x7f9de0004c40 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009ac0 tx=0x7f9de00057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:40.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.120+0000 7f9df759e640 1 -- 172.21.15.84:0/2506419817 reap_dead start 2023-10-16T11:24:40.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.120+0000 7f9dfd0a2640 1 -- 172.21.15.84:0/2506419817 shutdown_connections 2023-10-16T11:24:40.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.120+0000 7f9dfd0a2640 1 -- 172.21.15.84:0/2506419817 >> 172.21.15.84:0/2506419817 conn(0x7f9df80ce9b0 msgr2=0x7f9df80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:40.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.120+0000 7f9dfd0a2640 1 -- 172.21.15.84:0/2506419817 shutdown_connections 2023-10-16T11:24:40.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.120+0000 7f9dfd0a2640 1 -- 172.21.15.84:0/2506419817 wait complete. 2023-10-16T11:24:40.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.122+0000 7f9dfd0a2640 1 Processor -- start 2023-10-16T11:24:40.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.122+0000 7f9dfd0a2640 1 -- start start 2023-10-16T11:24:40.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.123+0000 7f9dfd0a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9de00047c0 0x7f9df8154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:40.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.123+0000 7f9dfd0a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9df80c87c0 0x7f9df80c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:40.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.123+0000 7f9dfd0a2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9df80c6770 0x7f9df80c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:40.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.123+0000 7f9dfd0a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df8155210 con 0x7f9df80c87c0 2023-10-16T11:24:40.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.123+0000 7f9dfd0a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df80c7720 con 0x7f9de00047c0 2023-10-16T11:24:40.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.123+0000 7f9dfd0a2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df80c7a00 con 0x7f9df80c6770 2023-10-16T11:24:40.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.123+0000 7f9df659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9df80c87c0 0x7f9df80c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:40.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.123+0000 7f9df659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9df80c87c0 0x7f9df80c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50006/0 (socket says 172.21.15.84:50006) 2023-10-16T11:24:40.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.123+0000 7f9df659c640 1 -- 172.21.15.84:0/3506132681 learned_addr learned my addr 172.21.15.84:0/3506132681 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:40.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.124+0000 7f9df759e640 1 --2- 172.21.15.84:0/3506132681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9df80c6770 0x7f9df80c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:40.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.124+0000 7f9df659c640 1 -- 172.21.15.84:0/3506132681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9df80c6770 msgr2=0x7f9df80c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:40.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.124+0000 7f9df659c640 1 --2- 172.21.15.84:0/3506132681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9df80c6770 0x7f9df80c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:40.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.124+0000 7f9df6d9d640 1 --2- 172.21.15.84:0/3506132681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9de00047c0 0x7f9df8154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:40.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.124+0000 7f9df659c640 1 -- 172.21.15.84:0/3506132681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9de00047c0 msgr2=0x7f9df8154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:40.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.124+0000 7f9df659c640 1 --2- 172.21.15.84:0/3506132681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9de00047c0 0x7f9df8154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:40.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.124+0000 7f9df659c640 1 -- 172.21.15.84:0/3506132681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df80c7ce0 con 0x7f9df80c87c0 2023-10-16T11:24:40.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.124+0000 7f9df659c640 1 --2- 172.21.15.84:0/3506132681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9df80c87c0 0x7f9df80c6d80 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7f9dec02c270 tx=0x7f9dec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:40.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.124+0000 7f9ddf7fe640 1 -- 172.21.15.84:0/3506132681 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec003810 con 0x7f9df80c87c0 2023-10-16T11:24:40.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.124+0000 7f9ddf7fe640 1 -- 172.21.15.84:0/3506132681 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec003970 con 0x7f9df80c87c0 2023-10-16T11:24:40.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.124+0000 7f9ddf7fe640 1 -- 172.21.15.84:0/3506132681 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec003ce0 con 0x7f9df80c87c0 2023-10-16T11:24:40.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.124+0000 7f9dfd0a2640 1 -- 172.21.15.84:0/3506132681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df80c7fc0 con 0x7f9df80c87c0 2023-10-16T11:24:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.125+0000 7f9dfd0a2640 1 -- 172.21.15.84:0/3506132681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df80c72c0 con 0x7f9df80c87c0 2023-10-16T11:24:40.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.126+0000 7f9dfd0a2640 1 -- 172.21.15.84:0/3506132681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df80ace10 con 0x7f9df80c87c0 2023-10-16T11:24:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.127+0000 7f9ddf7fe640 1 -- 172.21.15.84:0/3506132681 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9dec051480 con 0x7f9df80c87c0 2023-10-16T11:24:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.128+0000 7f9ddf7fe640 1 --2- 172.21.15.84:0/3506132681 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f9de406af90 0x7f9de406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.128+0000 7f9ddf7fe640 1 -- 172.21.15.84:0/3506132681 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6495+0+0 (secure 0 0 0) 0x7f9dec0abd30 con 0x7f9df80c87c0 2023-10-16T11:24:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.128+0000 7f9df6d9d640 1 --2- 172.21.15.84:0/3506132681 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f9de406af90 0x7f9de406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:40.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.129+0000 7f9df6d9d640 1 --2- 172.21.15.84:0/3506132681 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f9de406af90 0x7f9de406d450 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9de8000f60 tx=0x7f9de8000ed0 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:40.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.132+0000 7f9ddf7fe640 1 -- 172.21.15.84:0/3506132681 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9dec076b50 con 0x7f9df80c87c0 2023-10-16T11:24:40.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.364+0000 7f9dfd0a2640 1 -- 172.21.15.84:0/3506132681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) v1 -- 0x7f9df80ace10 con 0x7f9df80c87c0 2023-10-16T11:24:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.901+0000 7f9ddf7fe640 1 -- 172.21.15.84:0/3506132681 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 28 and data pool 29 v116) v1 ==== 183+0+0 (secure 0 0 0) 0x7f9df80ace10 con 0x7f9df80c87c0 2023-10-16T11:24:40.904 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 28 and data pool 29 2023-10-16T11:24:40.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.906+0000 7f9dfd0a2640 1 -- 172.21.15.84:0/3506132681 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f9de406af90 msgr2=0x7f9de406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:40.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.906+0000 7f9dfd0a2640 1 --2- 172.21.15.84:0/3506132681 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f9de406af90 0x7f9de406d450 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9de8000f60 tx=0x7f9de8000ed0 comp rx=0 tx=0).stop 2023-10-16T11:24:40.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.906+0000 7f9dfd0a2640 1 -- 172.21.15.84:0/3506132681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9df80c87c0 msgr2=0x7f9df80c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:40.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.906+0000 7f9dfd0a2640 1 --2- 172.21.15.84:0/3506132681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9df80c87c0 0x7f9df80c6d80 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7f9dec02c270 tx=0x7f9dec004730 comp rx=0 tx=0).stop 2023-10-16T11:24:40.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.906+0000 7f9dfd0a2640 1 -- 172.21.15.84:0/3506132681 shutdown_connections 2023-10-16T11:24:40.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.906+0000 7f9dfd0a2640 1 --2- 172.21.15.84:0/3506132681 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f9de406af90 0x7f9de406d450 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:40.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.906+0000 7f9dfd0a2640 1 --2- 172.21.15.84:0/3506132681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9df80c6770 0x7f9df80c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:40.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.906+0000 7f9dfd0a2640 1 --2- 172.21.15.84:0/3506132681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9df80c87c0 0x7f9df80c6d80 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:40.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.906+0000 7f9dfd0a2640 1 --2- 172.21.15.84:0/3506132681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9de00047c0 0x7f9df8154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:40.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.906+0000 7f9dfd0a2640 1 -- 172.21.15.84:0/3506132681 >> 172.21.15.84:0/3506132681 conn(0x7f9df80ce9b0 msgr2=0x7f9df8158be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:40.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.907+0000 7f9dfd0a2640 1 -- 172.21.15.84:0/3506132681 shutdown_connections 2023-10-16T11:24:40.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:40.909+0000 7f9dfd0a2640 1 -- 172.21.15.84:0/3506132681 wait complete. 2023-10-16T11:24:40.947 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:24:41.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.131+0000 7fdcb840e640 1 Processor -- start 2023-10-16T11:24:41.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.131+0000 7fdcb840e640 1 -- start start 2023-10-16T11:24:41.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.133+0000 7fdcb840e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdcb00cd0c0 0x7fdcb01615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:41.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.133+0000 7fdcb840e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdcb015db00 0x7fdcb00b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:41.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.133+0000 7fdcb840e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb015e3e0 con 0x7fdcb00b0410 2023-10-16T11:24:41.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.133+0000 7fdcb840e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb00b0de0 con 0x7fdcb00cd0c0 2023-10-16T11:24:41.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.133+0000 7fdcb840e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb00cd4a0 con 0x7fdcb015db00 2023-10-16T11:24:41.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.133+0000 7fdcb61aa640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdcb00cd0c0 0x7fdcb01615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:41.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.133+0000 7fdcb61aa640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdcb00cd0c0 0x7fdcb01615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50016/0 (socket says 172.21.15.84:50016) 2023-10-16T11:24:41.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.133+0000 7fdcb61aa640 1 -- 172.21.15.84:0/2806130602 learned_addr learned my addr 172.21.15.84:0/2806130602 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:41.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.134+0000 7fdcb61aa640 1 -- 172.21.15.84:0/2806130602 >> v2:172.21.15.157:3300/0 conn(0x7fdcb015db00 msgr2=0x7fdcb00b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:41.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.134+0000 7fdcb61aa640 1 --2- 172.21.15.84:0/2806130602 >> v2:172.21.15.157:3300/0 conn(0x7fdcb015db00 0x7fdcb00b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.134+0000 7fdcb61aa640 1 -- 172.21.15.84:0/2806130602 >> v1:172.21.15.157:6789/0 conn(0x7fdcb00b0410 legacy=0x7fdcb015d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:41.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.134+0000 7fdcb61aa640 1 -- 172.21.15.84:0/2806130602 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb015e100 con 0x7fdcb00cd0c0 2023-10-16T11:24:41.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.135+0000 7fdcb61aa640 1 --2- 172.21.15.84:0/2806130602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcb00cd0c0 0x7fdcb01615f0 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7fdcb015afc0 tx=0x7fdcac02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d407eb8257d14534 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:41.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.135+0000 7fdcb51a8640 1 -- 172.21.15.84:0/2806130602 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac031800 con 0x7fdcb00cd0c0 2023-10-16T11:24:41.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.135+0000 7fdcb51a8640 1 -- 172.21.15.84:0/2806130602 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcac031960 con 0x7fdcb00cd0c0 2023-10-16T11:24:41.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.135+0000 7fdcb51a8640 1 -- 172.21.15.84:0/2806130602 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac031cf0 con 0x7fdcb00cd0c0 2023-10-16T11:24:41.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.136+0000 7fdcb840e640 1 -- 172.21.15.84:0/2806130602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcb00cd0c0 msgr2=0x7fdcb01615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:41.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.136+0000 7fdcb840e640 1 --2- 172.21.15.84:0/2806130602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcb00cd0c0 0x7fdcb01615f0 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7fdcb015afc0 tx=0x7fdcac02b8a0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.136+0000 7fdcb840e640 1 -- 172.21.15.84:0/2806130602 shutdown_connections 2023-10-16T11:24:41.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.136+0000 7fdcb840e640 1 --2- 172.21.15.84:0/2806130602 >> v2:172.21.15.157:3300/0 conn(0x7fdcb015db00 0x7fdcb00b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.136+0000 7fdcb840e640 1 --2- 172.21.15.84:0/2806130602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcb00cd0c0 0x7fdcb01615f0 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.136+0000 7fdcb840e640 1 -- 172.21.15.84:0/2806130602 >> 172.21.15.84:0/2806130602 conn(0x7fdcb00cb410 msgr2=0x7fdcb00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:41.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.136+0000 7fdcb840e640 1 -- 172.21.15.84:0/2806130602 shutdown_connections 2023-10-16T11:24:41.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.136+0000 7fdcb840e640 1 -- 172.21.15.84:0/2806130602 wait complete. 2023-10-16T11:24:41.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.138+0000 7fdcb840e640 1 Processor -- start 2023-10-16T11:24:41.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.138+0000 7fdcb840e640 1 -- start start 2023-10-16T11:24:41.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.139+0000 7fdcb840e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcb00b0410 0x7fdcb00c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:41.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.139+0000 7fdcb840e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcb00cd0c0 0x7fdcb01526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:41.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.139+0000 7fdcb840e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcb015db00 0x7fdcb00c3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:41.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.139+0000 7fdcb840e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb0152c10 con 0x7fdcb015db00 2023-10-16T11:24:41.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.139+0000 7fdcb840e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb01538c0 con 0x7fdcb00b0410 2023-10-16T11:24:41.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.139+0000 7fdcb840e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb00b1550 con 0x7fdcb00cd0c0 2023-10-16T11:24:41.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.139+0000 7fdcb69ab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcb015db00 0x7fdcb00c3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:41.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.139+0000 7fdcb69ab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcb015db00 0x7fdcb00c3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50026/0 (socket says 172.21.15.84:50026) 2023-10-16T11:24:41.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.139+0000 7fdcb69ab640 1 -- 172.21.15.84:0/2912664929 learned_addr learned my addr 172.21.15.84:0/2912664929 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:41.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.140+0000 7fdcb61aa640 1 --2- 172.21.15.84:0/2912664929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcb00b0410 0x7fdcb00c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:41.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.140+0000 7fdcb59a9640 1 --2- 172.21.15.84:0/2912664929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcb00cd0c0 0x7fdcb01526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:41.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.140+0000 7fdcb69ab640 1 -- 172.21.15.84:0/2912664929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcb00cd0c0 msgr2=0x7fdcb01526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:41.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.140+0000 7fdcb69ab640 1 --2- 172.21.15.84:0/2912664929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcb00cd0c0 0x7fdcb01526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.140+0000 7fdcb69ab640 1 -- 172.21.15.84:0/2912664929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcb00b0410 msgr2=0x7fdcb00c24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:41.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.140+0000 7fdcb69ab640 1 --2- 172.21.15.84:0/2912664929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcb00b0410 0x7fdcb00c24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.140+0000 7fdcb69ab640 1 -- 172.21.15.84:0/2912664929 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcac0094d0 con 0x7fdcb015db00 2023-10-16T11:24:41.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.140+0000 7fdcb69ab640 1 --2- 172.21.15.84:0/2912664929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcb015db00 0x7fdcb00c3370 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7fdca800b9a0 tx=0x7fdca800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:41.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.140+0000 7fdca6ffd640 1 -- 172.21.15.84:0/2912664929 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca800cba0 con 0x7fdcb015db00 2023-10-16T11:24:41.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.140+0000 7fdca6ffd640 1 -- 172.21.15.84:0/2912664929 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca800cd20 con 0x7fdcb015db00 2023-10-16T11:24:41.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.140+0000 7fdcb840e640 1 -- 172.21.15.84:0/2912664929 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb00b40a0 con 0x7fdcb015db00 2023-10-16T11:24:41.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.140+0000 7fdca6ffd640 1 -- 172.21.15.84:0/2912664929 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8014680 con 0x7fdcb015db00 2023-10-16T11:24:41.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.141+0000 7fdcb840e640 1 -- 172.21.15.84:0/2912664929 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb0153b30 con 0x7fdcb015db00 2023-10-16T11:24:41.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.142+0000 7fdcb840e640 1 -- 172.21.15.84:0/2912664929 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb0153d50 con 0x7fdcb015db00 2023-10-16T11:24:41.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.143+0000 7fdca6ffd640 1 -- 172.21.15.84:0/2912664929 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdca801d3b0 con 0x7fdcb015db00 2023-10-16T11:24:41.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.144+0000 7fdca6ffd640 1 --2- 172.21.15.84:0/2912664929 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fdc9406af90 0x7fdc9406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:41.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.144+0000 7fdca6ffd640 1 -- 172.21.15.84:0/2912664929 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 6601+0+0 (secure 0 0 0) 0x7fdca8089ab0 con 0x7fdcb015db00 2023-10-16T11:24:41.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.144+0000 7fdcb61aa640 1 --2- 172.21.15.84:0/2912664929 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fdc9406af90 0x7fdc9406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:41.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.145+0000 7fdcb61aa640 1 --2- 172.21.15.84:0/2912664929 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fdc9406af90 0x7fdc9406d450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdcac036000 tx=0x7fdcac03c040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:41.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.148+0000 7fdca6ffd640 1 -- 172.21.15.84:0/2912664929 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdca80546d0 con 0x7fdcb015db00 2023-10-16T11:24:41.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.340+0000 7fdcb840e640 1 -- 172.21.15.84:0/2912664929 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcb004e160 con 0x7fdcb015db00 2023-10-16T11:24:41.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.342+0000 7fdca6ffd640 1 -- 172.21.15.84:0/2912664929 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v220) v1 ==== 74+0+20340 (secure 0 0 0) 0x7fdca8019020 con 0x7fdcb015db00 2023-10-16T11:24:41.345 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:41.345 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":220,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:24:40.366366+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:24:23.363032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:24:24.551270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:24:38.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:24:39.387564+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:24:41.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.345+0000 7fdcb840e640 1 -- 172.21.15.84:0/2912664929 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fdc9406af90 msgr2=0x7fdc9406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:41.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.345+0000 7fdcb840e640 1 --2- 172.21.15.84:0/2912664929 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fdc9406af90 0x7fdc9406d450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdcac036000 tx=0x7fdcac03c040 comp rx=0 tx=0).stop 2023-10-16T11:24:41.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.345+0000 7fdcb840e640 1 -- 172.21.15.84:0/2912664929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcb015db00 msgr2=0x7fdcb00c3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:41.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.345+0000 7fdcb840e640 1 --2- 172.21.15.84:0/2912664929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcb015db00 0x7fdcb00c3370 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7fdca800b9a0 tx=0x7fdca800be60 comp rx=0 tx=0).stop 2023-10-16T11:24:41.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.346+0000 7fdcb840e640 1 -- 172.21.15.84:0/2912664929 shutdown_connections 2023-10-16T11:24:41.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.346+0000 7fdcb840e640 1 --2- 172.21.15.84:0/2912664929 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fdc9406af90 0x7fdc9406d450 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.346+0000 7fdcb840e640 1 --2- 172.21.15.84:0/2912664929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcb015db00 0x7fdcb00c3370 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.346+0000 7fdcb840e640 1 --2- 172.21.15.84:0/2912664929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcb00cd0c0 0x7fdcb01526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.346+0000 7fdcb840e640 1 --2- 172.21.15.84:0/2912664929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcb00b0410 0x7fdcb00c24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.346+0000 7fdcb840e640 1 -- 172.21.15.84:0/2912664929 >> 172.21.15.84:0/2912664929 conn(0x7fdcb00cb410 msgr2=0x7fdcb015b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:41.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.346+0000 7fdcb840e640 1 -- 172.21.15.84:0/2912664929 shutdown_connections 2023-10-16T11:24:41.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.346+0000 7fdcb840e640 1 -- 172.21.15.84:0/2912664929 wait complete. 2023-10-16T11:24:41.365 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:24:41.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.573+0000 7f0ca2490640 1 Processor -- start 2023-10-16T11:24:41.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.573+0000 7f0ca2490640 1 -- start start 2023-10-16T11:24:41.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.574+0000 7f0ca2490640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c070b60 0x7f0c9c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:41.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.574+0000 7f0ca2490640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c9c15aef0 con 0x7f0c9c06c100 2023-10-16T11:24:41.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.575+0000 7f0ca2490640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c9c06cc60 con 0x7f0c9c159950 2023-10-16T11:24:41.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.575+0000 7f0ca2490640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c15e490 con 0x7f0c9c070b60 2023-10-16T11:24:41.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.575+0000 7f0ca1c8f640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0c9c06c100 0x7f0c9c06c550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38518/0 (socket says 172.21.15.84:38518) 2023-10-16T11:24:41.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.575+0000 7f0ca1c8f640 1 -- 172.21.15.84:0/3300993734 learned_addr learned my addr 172.21.15.84:0/3300993734 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:41.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.575+0000 7f0c9bfff640 1 -- 172.21.15.84:0/3300993734 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288095083 0 0) 0x7f0c94001110 con 0x7f0c9c06c100 2023-10-16T11:24:41.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.575+0000 7f0ca0c8d640 1 --2- 172.21.15.84:0/3300993734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c070b60 0x7f0c9c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:41.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.575+0000 7f0c9bfff640 1 -- 172.21.15.84:0/3300993734 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c84003520 con 0x7f0c9c06c100 2023-10-16T11:24:41.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.576+0000 7f0c9bfff640 1 -- 172.21.15.84:0/3300993734 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 742311784 0 0) 0x7f0c84003520 con 0x7f0c9c06c100 2023-10-16T11:24:41.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.576+0000 7f0c9bfff640 1 -- 172.21.15.84:0/3300993734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c070b60 msgr2=0x7f0c9c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:41.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.576+0000 7f0c9bfff640 1 --2- 172.21.15.84:0/3300993734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c070b60 0x7f0c9c15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.576+0000 7f0c9bfff640 1 -- 172.21.15.84:0/3300993734 >> v1:172.21.15.157:6789/0 conn(0x7f0c9c159950 legacy=0x7f0c9c070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:41.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.576+0000 7f0c9bfff640 1 -- 172.21.15.84:0/3300993734 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9c15d910 con 0x7f0c9c06c100 2023-10-16T11:24:41.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.576+0000 7f0c9bfff640 1 -- 172.21.15.84:0/3300993734 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f0c94001800 con 0x7f0c9c06c100 2023-10-16T11:24:41.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.576+0000 7f0c9bfff640 1 -- 172.21.15.84:0/3300993734 >> v1:172.21.15.84:6789/0 conn(0x7f0c9c06c100 legacy=0x7f0c9c06c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:41.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.576+0000 7f0c9bfff640 1 --2- 172.21.15.84:0/3300993734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c070b60 0x7f0c840039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:41.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.576+0000 7f0c9bfff640 1 --2- 172.21.15.84:0/3300993734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c84003fd0 0x7f0c840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:41.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.576+0000 7f0c9bfff640 1 --2- 172.21.15.84:0/3300993734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c84004d00 0x7f0c84005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:41.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.576+0000 7f0c9bfff640 1 -- 172.21.15.84:0/3300993734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c15e490 con 0x7f0c84004d00 2023-10-16T11:24:41.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.576+0000 7f0c9bfff640 1 -- 172.21.15.84:0/3300993734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8400bbc0 con 0x7f0c84003fd0 2023-10-16T11:24:41.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.576+0000 7f0c9bfff640 1 -- 172.21.15.84:0/3300993734 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8400df90 con 0x7f0c9c070b60 2023-10-16T11:24:41.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.576+0000 7f0ca1c8f640 1 --2- 172.21.15.84:0/3300993734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c84004d00 0x7f0c84005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:41.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.576+0000 7f0ca148e640 1 --2- 172.21.15.84:0/3300993734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c070b60 0x7f0c840039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:41.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.576+0000 7f0ca0c8d640 1 --2- 172.21.15.84:0/3300993734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c84003fd0 0x7f0c840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:41.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.577+0000 7f0ca1c8f640 1 -- 172.21.15.84:0/3300993734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c070b60 msgr2=0x7f0c840039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:41.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.577+0000 7f0ca1c8f640 1 --2- 172.21.15.84:0/3300993734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c070b60 0x7f0c840039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.577+0000 7f0ca1c8f640 1 -- 172.21.15.84:0/3300993734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c84003fd0 msgr2=0x7f0c840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:41.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.577+0000 7f0ca1c8f640 1 --2- 172.21.15.84:0/3300993734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c84003fd0 0x7f0c840047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.577+0000 7f0ca1c8f640 1 -- 172.21.15.84:0/3300993734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c84003520 con 0x7f0c84004d00 2023-10-16T11:24:41.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.578+0000 7f0ca1c8f640 1 --2- 172.21.15.84:0/3300993734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c84004d00 0x7f0c84005180 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400c240 tx=0x7f0c840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff10b97349d59b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:41.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.578+0000 7f0c9bfff640 1 -- 172.21.15.84:0/3300993734 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c94005ad0 con 0x7f0c84004d00 2023-10-16T11:24:41.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.578+0000 7f0c9bfff640 1 -- 172.21.15.84:0/3300993734 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c94005c50 con 0x7f0c84004d00 2023-10-16T11:24:41.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.578+0000 7f0c9bfff640 1 -- 172.21.15.84:0/3300993734 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9402c8b0 con 0x7f0c84004d00 2023-10-16T11:24:41.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.578+0000 7f0ca2490640 1 -- 172.21.15.84:0/3300993734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c84004d00 msgr2=0x7f0c84005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:41.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.578+0000 7f0ca2490640 1 --2- 172.21.15.84:0/3300993734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c84004d00 0x7f0c84005180 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400c240 tx=0x7f0c840057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.578+0000 7f0ca1c8f640 1 -- 172.21.15.84:0/3300993734 reap_dead start 2023-10-16T11:24:41.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.579+0000 7f0ca2490640 1 -- 172.21.15.84:0/3300993734 shutdown_connections 2023-10-16T11:24:41.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.579+0000 7f0ca2490640 1 -- 172.21.15.84:0/3300993734 >> 172.21.15.84:0/3300993734 conn(0x7f0c9c0ce6a0 msgr2=0x7f0c9c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:41.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.579+0000 7f0ca2490640 1 -- 172.21.15.84:0/3300993734 shutdown_connections 2023-10-16T11:24:41.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.579+0000 7f0ca2490640 1 -- 172.21.15.84:0/3300993734 wait complete. 2023-10-16T11:24:41.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.580+0000 7f0ca2490640 1 Processor -- start 2023-10-16T11:24:41.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.581+0000 7f0ca2490640 1 -- start start 2023-10-16T11:24:41.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.581+0000 7f0ca2490640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c84004d00 0x7f0c9c1f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:41.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.581+0000 7f0ca2490640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c9c0c9ae0 0x7f0c9c1f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:41.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.581+0000 7f0ca2490640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c0c9f60 0x7f0c9c14a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:41.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.581+0000 7f0ca2490640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c1f6730 con 0x7f0c9c0c9ae0 2023-10-16T11:24:41.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.581+0000 7f0ca2490640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c1f6100 con 0x7f0c84004d00 2023-10-16T11:24:41.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.581+0000 7f0ca2490640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c1f3580 con 0x7f0c9c0c9f60 2023-10-16T11:24:41.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.582+0000 7f0ca0c8d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c9c0c9ae0 0x7f0c9c1f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:41.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.582+0000 7f0ca0c8d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c9c0c9ae0 0x7f0c9c1f2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50036/0 (socket says 172.21.15.84:50036) 2023-10-16T11:24:41.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.582+0000 7f0ca0c8d640 1 -- 172.21.15.84:0/1451784517 learned_addr learned my addr 172.21.15.84:0/1451784517 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:41.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.582+0000 7f0ca148e640 1 --2- 172.21.15.84:0/1451784517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c84004d00 0x7f0c9c1f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:41.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.582+0000 7f0ca1c8f640 1 --2- 172.21.15.84:0/1451784517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c0c9f60 0x7f0c9c14a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:41.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.582+0000 7f0ca0c8d640 1 -- 172.21.15.84:0/1451784517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c0c9f60 msgr2=0x7f0c9c14a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:41.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.582+0000 7f0ca0c8d640 1 --2- 172.21.15.84:0/1451784517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c0c9f60 0x7f0c9c14a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.582+0000 7f0ca0c8d640 1 -- 172.21.15.84:0/1451784517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c84004d00 msgr2=0x7f0c9c1f6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:41.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.582+0000 7f0ca0c8d640 1 --2- 172.21.15.84:0/1451784517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c84004d00 0x7f0c9c1f6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.582+0000 7f0ca0c8d640 1 -- 172.21.15.84:0/1451784517 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9c1f3860 con 0x7f0c9c0c9ae0 2023-10-16T11:24:41.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.582+0000 7f0ca148e640 1 --2- 172.21.15.84:0/1451784517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c84004d00 0x7f0c9c1f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:41.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.582+0000 7f0ca0c8d640 1 --2- 172.21.15.84:0/1451784517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c9c0c9ae0 0x7f0c9c1f2ab0 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000e7e0 tx=0x7f0c9000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:41.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.583+0000 7f0c99ffb640 1 -- 172.21.15.84:0/1451784517 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000c6d0 con 0x7f0c9c0c9ae0 2023-10-16T11:24:41.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.583+0000 7f0c99ffb640 1 -- 172.21.15.84:0/1451784517 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9000c830 con 0x7f0c9c0c9ae0 2023-10-16T11:24:41.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.583+0000 7f0c99ffb640 1 -- 172.21.15.84:0/1451784517 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000cba0 con 0x7f0c9c0c9ae0 2023-10-16T11:24:41.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.583+0000 7f0ca2490640 1 -- 172.21.15.84:0/1451784517 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9c1f6350 con 0x7f0c9c0c9ae0 2023-10-16T11:24:41.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.583+0000 7f0ca2490640 1 -- 172.21.15.84:0/1451784517 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9c157ef0 con 0x7f0c9c0c9ae0 2023-10-16T11:24:41.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.584+0000 7f0ca2490640 1 -- 172.21.15.84:0/1451784517 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9c15deb0 con 0x7f0c9c0c9ae0 2023-10-16T11:24:41.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.585+0000 7f0c99ffb640 1 -- 172.21.15.84:0/1451784517 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0c9000cd00 con 0x7f0c9c0c9ae0 2023-10-16T11:24:41.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.586+0000 7f0c99ffb640 1 --2- 172.21.15.84:0/1451784517 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0c8806afc0 0x7f0c8806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:41.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.586+0000 7f0c99ffb640 1 -- 172.21.15.84:0/1451784517 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f0c900899a0 con 0x7f0c9c0c9ae0 2023-10-16T11:24:41.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.586+0000 7f0ca148e640 1 --2- 172.21.15.84:0/1451784517 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0c8806afc0 0x7f0c8806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:41.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.587+0000 7f0ca148e640 1 --2- 172.21.15.84:0/1451784517 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0c8806afc0 0x7f0c8806d480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c007fd0 tx=0x7f0c8c009040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:41.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.590+0000 7f0c99ffb640 1 -- 172.21.15.84:0/1451784517 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0c9c15deb0 con 0x7f0c9c0c9ae0 2023-10-16T11:24:41.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.784+0000 7f0ca2490640 1 -- 172.21.15.84:0/1451784517 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c9c15deb0 con 0x7f0c9c0c9ae0 2023-10-16T11:24:41.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.788+0000 7f0c99ffb640 1 -- 172.21.15.84:0/1451784517 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v220) v1 ==== 74+0+20340 (secure 0 0 0) 0x7f0c9c15deb0 con 0x7f0c9c0c9ae0 2023-10-16T11:24:41.791 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:41.791 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":220,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:24:40.366366+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:24:23.363032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:24:24.551270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:24:38.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:24:39.387564+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:24:41.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.791+0000 7f0ca2490640 1 -- 172.21.15.84:0/1451784517 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0c8806afc0 msgr2=0x7f0c8806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:41.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.791+0000 7f0ca2490640 1 --2- 172.21.15.84:0/1451784517 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0c8806afc0 0x7f0c8806d480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c007fd0 tx=0x7f0c8c009040 comp rx=0 tx=0).stop 2023-10-16T11:24:41.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.791+0000 7f0ca2490640 1 -- 172.21.15.84:0/1451784517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c9c0c9ae0 msgr2=0x7f0c9c1f2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:41.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.791+0000 7f0ca2490640 1 --2- 172.21.15.84:0/1451784517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c9c0c9ae0 0x7f0c9c1f2ab0 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000e7e0 tx=0x7f0c9000eca0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.791+0000 7f0ca2490640 1 -- 172.21.15.84:0/1451784517 shutdown_connections 2023-10-16T11:24:41.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.791+0000 7f0ca2490640 1 --2- 172.21.15.84:0/1451784517 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0c8806afc0 0x7f0c8806d480 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.791+0000 7f0ca2490640 1 --2- 172.21.15.84:0/1451784517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c0c9f60 0x7f0c9c14a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.791+0000 7f0ca2490640 1 --2- 172.21.15.84:0/1451784517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c9c0c9ae0 0x7f0c9c1f2ab0 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.791+0000 7f0ca2490640 1 --2- 172.21.15.84:0/1451784517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c84004d00 0x7f0c9c1f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:41.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.791+0000 7f0ca2490640 1 -- 172.21.15.84:0/1451784517 >> 172.21.15.84:0/1451784517 conn(0x7f0c9c0ce6a0 msgr2=0x7f0c9c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:41.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.791+0000 7f0ca2490640 1 -- 172.21.15.84:0/1451784517 shutdown_connections 2023-10-16T11:24:41.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:41.791+0000 7f0ca2490640 1 -- 172.21.15.84:0/1451784517 wait complete. 2023-10-16T11:24:41.813 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2023-10-16T11:24:42.032 INFO:tasks.ceph.mds.d.smithi157.stderr:2023-10-16T11:24:42.030+0000 7f98f1c64640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:24:42.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.035+0000 7f0ae091d640 1 Processor -- start 2023-10-16T11:24:42.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.035+0000 7f0ae091d640 1 -- start start 2023-10-16T11:24:42.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.036+0000 7f0ae091d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0adc1428c0 0x7f0adc0c9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:42.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.037+0000 7f0ae091d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0adc0c9ae0 con 0x7f0adc06b660 2023-10-16T11:24:42.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.037+0000 7f0ae091d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0adc0cc6b0 con 0x7f0adc0c9690 2023-10-16T11:24:42.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.037+0000 7f0ae091d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0adc1599b0 con 0x7f0adc1428c0 2023-10-16T11:24:42.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.037+0000 7f0ad3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0adc1428c0 0x7f0adc0c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:42.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.037+0000 7f0ad3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0adc1428c0 0x7f0adc0c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50038/0 (socket says 172.21.15.84:50038) 2023-10-16T11:24:42.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.037+0000 7f0ad3fff640 1 -- 172.21.15.84:0/2510235588 learned_addr learned my addr 172.21.15.84:0/2510235588 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:42.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.037+0000 7f0ad3fff640 1 -- 172.21.15.84:0/2510235588 >> v1:172.21.15.157:6789/0 conn(0x7f0adc0c9690 legacy=0x7f0adc0ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:42.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.037+0000 7f0ad3fff640 1 -- 172.21.15.84:0/2510235588 >> v1:172.21.15.84:6789/0 conn(0x7f0adc06b660 legacy=0x7f0adc06ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:42.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.037+0000 7f0ad3fff640 1 -- 172.21.15.84:0/2510235588 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0adc06c290 con 0x7f0adc1428c0 2023-10-16T11:24:42.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.039+0000 7f0ad3fff640 1 --2- 172.21.15.84:0/2510235588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0adc1428c0 0x7f0adc0c9150 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400bb00 tx=0x7f0ac402df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724db5a8ce6dbb58 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:42.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.039+0000 7f0ad9d9b640 1 -- 172.21.15.84:0/2510235588 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac402ece0 con 0x7f0adc1428c0 2023-10-16T11:24:42.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.039+0000 7f0ad9d9b640 1 -- 172.21.15.84:0/2510235588 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac4032070 con 0x7f0adc1428c0 2023-10-16T11:24:42.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.039+0000 7f0ad9d9b640 1 -- 172.21.15.84:0/2510235588 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4038620 con 0x7f0adc1428c0 2023-10-16T11:24:42.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.040+0000 7f0ae091d640 1 -- 172.21.15.84:0/2510235588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0adc1428c0 msgr2=0x7f0adc0c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:42.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.040+0000 7f0ae091d640 1 --2- 172.21.15.84:0/2510235588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0adc1428c0 0x7f0adc0c9150 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400bb00 tx=0x7f0ac402df30 comp rx=0 tx=0).stop 2023-10-16T11:24:42.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.040+0000 7f0ae091d640 1 -- 172.21.15.84:0/2510235588 shutdown_connections 2023-10-16T11:24:42.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.040+0000 7f0ae091d640 1 --2- 172.21.15.84:0/2510235588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0adc1428c0 0x7f0adc0c9150 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:42.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.040+0000 7f0ae091d640 1 -- 172.21.15.84:0/2510235588 >> 172.21.15.84:0/2510235588 conn(0x7f0adc0ce9b0 msgr2=0x7f0adc0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:42.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.040+0000 7f0ae091d640 1 -- 172.21.15.84:0/2510235588 shutdown_connections 2023-10-16T11:24:42.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.040+0000 7f0ae091d640 1 -- 172.21.15.84:0/2510235588 wait complete. 2023-10-16T11:24:42.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.042+0000 7f0ae091d640 1 Processor -- start 2023-10-16T11:24:42.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.042+0000 7f0ae091d640 1 -- start start 2023-10-16T11:24:42.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.043+0000 7f0ae091d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0adc06b660 0x7f0adc0ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:42.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.043+0000 7f0ae091d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0adc0c9690 0x7f0adc0c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:42.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.043+0000 7f0ae091d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0adc1428c0 0x7f0adc0c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:42.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.043+0000 7f0ae091d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc158060 con 0x7f0adc06b660 2023-10-16T11:24:42.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.043+0000 7f0ae091d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc1f2a90 con 0x7f0adc1428c0 2023-10-16T11:24:42.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.043+0000 7f0ae091d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc070aa0 con 0x7f0adc0c9690 2023-10-16T11:24:42.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.044+0000 7f0ada59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0adc06b660 0x7f0adc0ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:42.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.044+0000 7f0ada59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0adc06b660 0x7f0adc0ace00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50046/0 (socket says 172.21.15.84:50046) 2023-10-16T11:24:42.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.044+0000 7f0ada59c640 1 -- 172.21.15.84:0/3699140989 learned_addr learned my addr 172.21.15.84:0/3699140989 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:42.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.044+0000 7f0adad9d640 1 --2- 172.21.15.84:0/3699140989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0adc1428c0 0x7f0adc0c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:42.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.044+0000 7f0ada59c640 1 -- 172.21.15.84:0/3699140989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0adc0c9690 msgr2=0x7f0adc0c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:42.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.044+0000 7f0ada59c640 1 --2- 172.21.15.84:0/3699140989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0adc0c9690 0x7f0adc0c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:42.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.044+0000 7f0ada59c640 1 -- 172.21.15.84:0/3699140989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0adc1428c0 msgr2=0x7f0adc0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:42.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.044+0000 7f0ada59c640 1 --2- 172.21.15.84:0/3699140989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0adc1428c0 0x7f0adc0c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:42.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.044+0000 7f0ada59c640 1 -- 172.21.15.84:0/3699140989 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac400b760 con 0x7f0adc06b660 2023-10-16T11:24:42.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.044+0000 7f0ada59c640 1 --2- 172.21.15.84:0/3699140989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0adc06b660 0x7f0adc0ace00 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00c400 tx=0x7f0acc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:42.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.045+0000 7f0ad2ffd640 1 -- 172.21.15.84:0/3699140989 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0acc0052d0 con 0x7f0adc06b660 2023-10-16T11:24:42.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.045+0000 7f0ad2ffd640 1 -- 172.21.15.84:0/3699140989 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0acc005430 con 0x7f0adc06b660 2023-10-16T11:24:42.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.045+0000 7f0ad2ffd640 1 -- 172.21.15.84:0/3699140989 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0acc003830 con 0x7f0adc06b660 2023-10-16T11:24:42.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.045+0000 7f0ae091d640 1 -- 172.21.15.84:0/3699140989 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0adc1f1f10 con 0x7f0adc06b660 2023-10-16T11:24:42.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.045+0000 7f0ae091d640 1 -- 172.21.15.84:0/3699140989 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0adc1f24d0 con 0x7f0adc06b660 2023-10-16T11:24:42.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.046+0000 7f0ae091d640 1 -- 172.21.15.84:0/3699140989 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0adc1ef7d0 con 0x7f0adc06b660 2023-10-16T11:24:42.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.047+0000 7f0ad2ffd640 1 -- 172.21.15.84:0/3699140989 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0acc003990 con 0x7f0adc06b660 2023-10-16T11:24:42.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.048+0000 7f0ad2ffd640 1 --2- 172.21.15.84:0/3699140989 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0abc06af90 0x7f0abc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:42.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.048+0000 7f0ad2ffd640 1 -- 172.21.15.84:0/3699140989 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f0acc088e60 con 0x7f0adc06b660 2023-10-16T11:24:42.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.048+0000 7f0ad3fff640 1 --2- 172.21.15.84:0/3699140989 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0abc06af90 0x7f0abc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:42.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.049+0000 7f0ad3fff640 1 --2- 172.21.15.84:0/3699140989 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0abc06af90 0x7f0abc06d450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4039000 tx=0x7f0ac4030000 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:42.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.052+0000 7f0ad2ffd640 1 -- 172.21.15.84:0/3699140989 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0adc1ef7d0 con 0x7f0adc06b660 2023-10-16T11:24:42.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.292+0000 7f0ae091d640 1 -- 172.21.15.84:0/3699140989 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0adc0c9ae0 con 0x7f0adc06b660 2023-10-16T11:24:42.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.930+0000 7f0ad2ffd640 1 -- 172.21.15.84:0/3699140989 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v119) v1 ==== 121+0+0 (secure 0 0 0) 0x7f0adc0c9ae0 con 0x7f0adc06b660 2023-10-16T11:24:42.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.934+0000 7f0ae091d640 1 -- 172.21.15.84:0/3699140989 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0abc06af90 msgr2=0x7f0abc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:42.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.934+0000 7f0ae091d640 1 --2- 172.21.15.84:0/3699140989 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0abc06af90 0x7f0abc06d450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4039000 tx=0x7f0ac4030000 comp rx=0 tx=0).stop 2023-10-16T11:24:42.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.934+0000 7f0ae091d640 1 -- 172.21.15.84:0/3699140989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0adc06b660 msgr2=0x7f0adc0ace00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:42.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.934+0000 7f0ae091d640 1 --2- 172.21.15.84:0/3699140989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0adc06b660 0x7f0adc0ace00 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00c400 tx=0x7f0acc00c8c0 comp rx=0 tx=0).stop 2023-10-16T11:24:42.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.934+0000 7f0ae091d640 1 -- 172.21.15.84:0/3699140989 shutdown_connections 2023-10-16T11:24:42.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.934+0000 7f0ae091d640 1 --2- 172.21.15.84:0/3699140989 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f0abc06af90 0x7f0abc06d450 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:42.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.934+0000 7f0ae091d640 1 --2- 172.21.15.84:0/3699140989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0adc1428c0 0x7f0adc0c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:42.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.934+0000 7f0ae091d640 1 --2- 172.21.15.84:0/3699140989 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0adc0c9690 0x7f0adc0c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:42.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.934+0000 7f0ae091d640 1 --2- 172.21.15.84:0/3699140989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0adc06b660 0x7f0adc0ace00 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:42.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.934+0000 7f0ae091d640 1 -- 172.21.15.84:0/3699140989 >> 172.21.15.84:0/3699140989 conn(0x7f0adc0ce9b0 msgr2=0x7f0adc15bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:42.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.935+0000 7f0ae091d640 1 -- 172.21.15.84:0/3699140989 shutdown_connections 2023-10-16T11:24:42.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:42.935+0000 7f0ae091d640 1 -- 172.21.15.84:0/3699140989 wait complete. 2023-10-16T11:24:42.958 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T11:24:42.959 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2023-10-16T11:24:43.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.176+0000 7ff4e3fff640 1 Processor -- start 2023-10-16T11:24:43.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.176+0000 7ff4e3fff640 1 -- start start 2023-10-16T11:24:43.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.177+0000 7ff4e3fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff4e4070b60 0x7ff4e415a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:43.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.177+0000 7ff4e3fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4e406c100 0x7ff4e406c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:43.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.177+0000 7ff4e3fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e415aef0 con 0x7ff4e4159950 2023-10-16T11:24:43.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.177+0000 7ff4e3fff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e415e490 con 0x7ff4e4070b60 2023-10-16T11:24:43.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.177+0000 7ff4e3fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e415d910 con 0x7ff4e406c100 2023-10-16T11:24:43.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.178+0000 7ff4e2ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff4e4159950 0x7ff4e4070450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38534/0 (socket says 172.21.15.84:38534) 2023-10-16T11:24:43.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.178+0000 7ff4e2ffd640 1 -- 172.21.15.84:0/382617706 learned_addr learned my addr 172.21.15.84:0/382617706 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:43.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.178+0000 7ff4e27fc640 1 --2- 172.21.15.84:0/382617706 >> v2:172.21.15.157:3300/0 conn(0x7ff4e4070b60 0x7ff4e415a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:43.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.178+0000 7ff4e37fe640 1 --2- 172.21.15.84:0/382617706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4e406c100 0x7ff4e406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:43.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.178+0000 7ff4e1ffb640 1 -- 172.21.15.84:0/382617706 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845251561 0 0) 0x7ff4d4001110 con 0x7ff4e4159950 2023-10-16T11:24:43.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.178+0000 7ff4e1ffb640 1 -- 172.21.15.84:0/382617706 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4cc003520 con 0x7ff4e4159950 2023-10-16T11:24:43.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.179+0000 7ff4e1ffb640 1 -- 172.21.15.84:0/382617706 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1728231403 0 0) 0x7ff4cc003520 con 0x7ff4e4159950 2023-10-16T11:24:43.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.179+0000 7ff4e1ffb640 1 -- 172.21.15.84:0/382617706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4e406c100 msgr2=0x7ff4e406c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:43.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.179+0000 7ff4e1ffb640 1 --2- 172.21.15.84:0/382617706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4e406c100 0x7ff4e406c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:43.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.179+0000 7ff4e1ffb640 1 -- 172.21.15.84:0/382617706 >> v2:172.21.15.157:3300/0 conn(0x7ff4e4070b60 msgr2=0x7ff4e415a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:43.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.179+0000 7ff4e1ffb640 1 --2- 172.21.15.84:0/382617706 >> v2:172.21.15.157:3300/0 conn(0x7ff4e4070b60 0x7ff4e415a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:43.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.179+0000 7ff4e1ffb640 1 -- 172.21.15.84:0/382617706 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e415dbf0 con 0x7ff4e4159950 2023-10-16T11:24:43.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.179+0000 7ff4e1ffb640 1 -- 172.21.15.84:0/382617706 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ff4d4001800 con 0x7ff4e4159950 2023-10-16T11:24:43.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.179+0000 7ff4e1ffb640 1 -- 172.21.15.84:0/382617706 >> v1:172.21.15.84:6789/0 conn(0x7ff4e4159950 legacy=0x7ff4e4070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:43.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.179+0000 7ff4e1ffb640 1 --2- 172.21.15.84:0/382617706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4cc003970 0x7ff4cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:43.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.179+0000 7ff4e1ffb640 1 --2- 172.21.15.84:0/382617706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4cc0047c0 0x7ff4cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:43.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.179+0000 7ff4e1ffb640 1 --2- 172.21.15.84:0/382617706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4cc005180 0x7ff4cc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:43.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.179+0000 7ff4e1ffb640 1 -- 172.21.15.84:0/382617706 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e415e490 con 0x7ff4cc003970 2023-10-16T11:24:43.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.179+0000 7ff4e1ffb640 1 -- 172.21.15.84:0/382617706 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e415d910 con 0x7ff4cc0047c0 2023-10-16T11:24:43.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.179+0000 7ff4e1ffb640 1 -- 172.21.15.84:0/382617706 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc00bfa0 con 0x7ff4cc005180 2023-10-16T11:24:43.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.179+0000 7ff4e37fe640 1 --2- 172.21.15.84:0/382617706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4e406c100 0x7ff4e406c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:24:43.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.179+0000 7ff4e2ffd640 1 --2- 172.21.15.84:0/382617706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4cc003970 0x7ff4cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:43.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.180+0000 7ff4e27fc640 1 --2- 172.21.15.84:0/382617706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4cc0047c0 0x7ff4cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:43.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.180+0000 7ff4e2ffd640 1 -- 172.21.15.84:0/382617706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4cc005180 msgr2=0x7ff4cc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:43.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.180+0000 7ff4e2ffd640 1 --2- 172.21.15.84:0/382617706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4cc005180 0x7ff4cc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:43.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.180+0000 7ff4e2ffd640 1 -- 172.21.15.84:0/382617706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4cc0047c0 msgr2=0x7ff4cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:43.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.180+0000 7ff4e2ffd640 1 --2- 172.21.15.84:0/382617706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4cc0047c0 0x7ff4cc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:43.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.180+0000 7ff4e2ffd640 1 -- 172.21.15.84:0/382617706 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc003520 con 0x7ff4cc003970 2023-10-16T11:24:43.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.180+0000 7ff4e27fc640 1 --2- 172.21.15.84:0/382617706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4cc0047c0 0x7ff4cc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:43.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.181+0000 7ff4e2ffd640 1 --2- 172.21.15.84:0/382617706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4cc003970 0x7ff4cc003d90 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4007f90 tx=0x7ff4cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba95747be9d1ec6c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:43.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.181+0000 7ff4e1ffb640 1 -- 172.21.15.84:0/382617706 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d402ece0 con 0x7ff4cc003970 2023-10-16T11:24:43.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.181+0000 7ff4e1ffb640 1 -- 172.21.15.84:0/382617706 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d402ee60 con 0x7ff4cc003970 2023-10-16T11:24:43.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.181+0000 7ff4e1ffb640 1 -- 172.21.15.84:0/382617706 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4005b10 con 0x7ff4cc003970 2023-10-16T11:24:43.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.182+0000 7ff4e3fff640 1 -- 172.21.15.84:0/382617706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4cc003970 msgr2=0x7ff4cc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:43.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.182+0000 7ff4e3fff640 1 --2- 172.21.15.84:0/382617706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4cc003970 0x7ff4cc003d90 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4007f90 tx=0x7ff4cc0057f0 comp rx=0 tx=0).stop 2023-10-16T11:24:43.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.182+0000 7ff4e37fe640 1 -- 172.21.15.84:0/382617706 reap_dead start 2023-10-16T11:24:43.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.182+0000 7ff4e3fff640 1 -- 172.21.15.84:0/382617706 shutdown_connections 2023-10-16T11:24:43.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.182+0000 7ff4e3fff640 1 -- 172.21.15.84:0/382617706 >> 172.21.15.84:0/382617706 conn(0x7ff4e40ce6a0 msgr2=0x7ff4e40ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:43.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.182+0000 7ff4e3fff640 1 -- 172.21.15.84:0/382617706 shutdown_connections 2023-10-16T11:24:43.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.182+0000 7ff4e3fff640 1 -- 172.21.15.84:0/382617706 wait complete. 2023-10-16T11:24:43.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.184+0000 7ff4e3fff640 1 Processor -- start 2023-10-16T11:24:43.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.184+0000 7ff4e3fff640 1 -- start start 2023-10-16T11:24:43.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.185+0000 7ff4e3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4cc003970 0x7ff4e4155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:43.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.185+0000 7ff4e3fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4e40c7c10 0x7ff4e4154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:43.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.185+0000 7ff4e3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4e41551a0 0x7ff4e40b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:43.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.185+0000 7ff4e3fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e40c80a0 con 0x7ff4e41551a0 2023-10-16T11:24:43.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.185+0000 7ff4e3fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e40c8380 con 0x7ff4cc003970 2023-10-16T11:24:43.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.185+0000 7ff4e3fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e40c8660 con 0x7ff4e40c7c10 2023-10-16T11:24:43.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.185+0000 7ff4e37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4e41551a0 0x7ff4e40b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:43.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.185+0000 7ff4e37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4e41551a0 0x7ff4e40b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50072/0 (socket says 172.21.15.84:50072) 2023-10-16T11:24:43.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.185+0000 7ff4e37fe640 1 -- 172.21.15.84:0/2618226236 learned_addr learned my addr 172.21.15.84:0/2618226236 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:43.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.185+0000 7ff4e37fe640 1 -- 172.21.15.84:0/2618226236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4e40c7c10 msgr2=0x7ff4e4154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:43.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.185+0000 7ff4e37fe640 1 --2- 172.21.15.84:0/2618226236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4e40c7c10 0x7ff4e4154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:43.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.185+0000 7ff4e37fe640 1 -- 172.21.15.84:0/2618226236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4cc003970 msgr2=0x7ff4e4155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:43.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.185+0000 7ff4e37fe640 1 --2- 172.21.15.84:0/2618226236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4cc003970 0x7ff4e4155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:43.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.185+0000 7ff4e37fe640 1 -- 172.21.15.84:0/2618226236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e40c6780 con 0x7ff4e41551a0 2023-10-16T11:24:43.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.186+0000 7ff4e37fe640 1 --2- 172.21.15.84:0/2618226236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4e41551a0 0x7ff4e40b2f60 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40c8ca0 tx=0x7ff4dc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:43.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.186+0000 7ff4cb7fe640 1 -- 172.21.15.84:0/2618226236 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc00dc90 con 0x7ff4e41551a0 2023-10-16T11:24:43.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.186+0000 7ff4cb7fe640 1 -- 172.21.15.84:0/2618226236 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4dc00f040 con 0x7ff4e41551a0 2023-10-16T11:24:43.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.186+0000 7ff4cb7fe640 1 -- 172.21.15.84:0/2618226236 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc003990 con 0x7ff4e41551a0 2023-10-16T11:24:43.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.186+0000 7ff4e3fff640 1 -- 172.21.15.84:0/2618226236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e4155bc0 con 0x7ff4e41551a0 2023-10-16T11:24:43.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.186+0000 7ff4e3fff640 1 -- 172.21.15.84:0/2618226236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e40c6430 con 0x7ff4e41551a0 2023-10-16T11:24:43.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.188+0000 7ff4e3fff640 1 -- 172.21.15.84:0/2618226236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e404e9b0 con 0x7ff4e41551a0 2023-10-16T11:24:43.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.188+0000 7ff4cb7fe640 1 -- 172.21.15.84:0/2618226236 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff4dc003af0 con 0x7ff4e41551a0 2023-10-16T11:24:43.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.189+0000 7ff4cb7fe640 1 --2- 172.21.15.84:0/2618226236 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff4d006af80 0x7ff4d006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:43.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.189+0000 7ff4cb7fe640 1 -- 172.21.15.84:0/2618226236 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7ff4dc089af0 con 0x7ff4e41551a0 2023-10-16T11:24:43.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.190+0000 7ff4e2ffd640 1 --2- 172.21.15.84:0/2618226236 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff4d006af80 0x7ff4d006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:43.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.190+0000 7ff4e2ffd640 1 --2- 172.21.15.84:0/2618226236 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff4d006af80 0x7ff4d006d440 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4036000 tx=0x7ff4d4030000 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:43.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.194+0000 7ff4cb7fe640 1 -- 172.21.15.84:0/2618226236 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff4dc054690 con 0x7ff4e41551a0 2023-10-16T11:24:43.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.429+0000 7ff4e3fff640 1 -- 172.21.15.84:0/2618226236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff4e4159150 con 0x7ff4e41551a0 2023-10-16T11:24:43.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.949+0000 7ff4cb7fe640 1 -- 172.21.15.84:0/2618226236 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v120) v1 ==== 125+0+0 (secure 0 0 0) 0x7ff4dc057cb0 con 0x7ff4e41551a0 2023-10-16T11:24:43.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.953+0000 7ff4e3fff640 1 -- 172.21.15.84:0/2618226236 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff4d006af80 msgr2=0x7ff4d006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:43.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.953+0000 7ff4e3fff640 1 --2- 172.21.15.84:0/2618226236 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff4d006af80 0x7ff4d006d440 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4036000 tx=0x7ff4d4030000 comp rx=0 tx=0).stop 2023-10-16T11:24:43.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.953+0000 7ff4e3fff640 1 -- 172.21.15.84:0/2618226236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4e41551a0 msgr2=0x7ff4e40b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:43.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.953+0000 7ff4e3fff640 1 --2- 172.21.15.84:0/2618226236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4e41551a0 0x7ff4e40b2f60 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40c8ca0 tx=0x7ff4dc00d4a0 comp rx=0 tx=0).stop 2023-10-16T11:24:43.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.953+0000 7ff4e3fff640 1 -- 172.21.15.84:0/2618226236 shutdown_connections 2023-10-16T11:24:43.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.953+0000 7ff4e3fff640 1 --2- 172.21.15.84:0/2618226236 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff4d006af80 0x7ff4d006d440 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:43.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.953+0000 7ff4e3fff640 1 --2- 172.21.15.84:0/2618226236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4e41551a0 0x7ff4e40b2f60 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:43.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.953+0000 7ff4e3fff640 1 --2- 172.21.15.84:0/2618226236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4e40c7c10 0x7ff4e4154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:43.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.953+0000 7ff4e3fff640 1 --2- 172.21.15.84:0/2618226236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4cc003970 0x7ff4e4155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:43.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.953+0000 7ff4e3fff640 1 -- 172.21.15.84:0/2618226236 >> 172.21.15.84:0/2618226236 conn(0x7ff4e40ce6a0 msgr2=0x7ff4e40ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:43.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.954+0000 7ff4e3fff640 1 -- 172.21.15.84:0/2618226236 shutdown_connections 2023-10-16T11:24:43.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:43.954+0000 7ff4e3fff640 1 -- 172.21.15.84:0/2618226236 wait complete. 2023-10-16T11:24:43.975 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2023-10-16T11:24:43.976 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:24:44.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.180+0000 7fe495f83640 1 Processor -- start 2023-10-16T11:24:44.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.180+0000 7fe495f83640 1 -- start start 2023-10-16T11:24:44.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.182+0000 7fe495f83640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49015ae80 0x7fe49015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:44.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.182+0000 7fe495f83640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe4900ce390 0x7fe490070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:44.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.182+0000 7fe495f83640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe49015ed60 con 0x7fe49015aaa0 2023-10-16T11:24:44.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.182+0000 7fe495f83640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49006b810 con 0x7fe4900ce390 2023-10-16T11:24:44.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.182+0000 7fe495f83640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900707e0 con 0x7fe49015ae80 2023-10-16T11:24:44.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.182+0000 7fe48ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe4900ce390 0x7fe490070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:44.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.182+0000 7fe48ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe4900ce390 0x7fe490070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:59998/0 (socket says 172.21.15.84:59998) 2023-10-16T11:24:44.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.182+0000 7fe48ffff640 1 -- 172.21.15.84:0/2593561759 learned_addr learned my addr 172.21.15.84:0/2593561759 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:44.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.182+0000 7fe48effd640 1 --2- 172.21.15.84:0/2593561759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49015ae80 0x7fe49015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:44.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.183+0000 7fe48ffff640 1 -- 172.21.15.84:0/2593561759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49015ae80 msgr2=0x7fe49015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:44.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.183+0000 7fe48ffff640 1 --2- 172.21.15.84:0/2593561759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49015ae80 0x7fe49015e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.183+0000 7fe48ffff640 1 -- 172.21.15.84:0/2593561759 >> v1:172.21.15.157:6789/0 conn(0x7fe49015aaa0 legacy=0x7fe4900cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:44.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.183+0000 7fe48ffff640 1 -- 172.21.15.84:0/2593561759 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49015ef90 con 0x7fe4900ce390 2023-10-16T11:24:44.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.184+0000 7fe48ffff640 1 --2- 172.21.15.84:0/2593561759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4900ce390 0x7fe490070f50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe488009e50 tx=0x7fe48802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=510d2f5dad0de343 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:44.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.185+0000 7fe48e7fc640 1 -- 172.21.15.84:0/2593561759 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48802cb00 con 0x7fe4900ce390 2023-10-16T11:24:44.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.185+0000 7fe48e7fc640 1 -- 172.21.15.84:0/2593561759 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48802cc60 con 0x7fe4900ce390 2023-10-16T11:24:44.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.185+0000 7fe48e7fc640 1 -- 172.21.15.84:0/2593561759 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4880343f0 con 0x7fe4900ce390 2023-10-16T11:24:44.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.186+0000 7fe495f83640 1 -- 172.21.15.84:0/2593561759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4900ce390 msgr2=0x7fe490070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:44.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.186+0000 7fe495f83640 1 --2- 172.21.15.84:0/2593561759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4900ce390 0x7fe490070f50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe488009e50 tx=0x7fe48802bf60 comp rx=0 tx=0).stop 2023-10-16T11:24:44.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.186+0000 7fe495f83640 1 -- 172.21.15.84:0/2593561759 shutdown_connections 2023-10-16T11:24:44.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.186+0000 7fe495f83640 1 --2- 172.21.15.84:0/2593561759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4900ce390 0x7fe490070f50 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.186+0000 7fe495f83640 1 --2- 172.21.15.84:0/2593561759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49015ae80 0x7fe49015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.186+0000 7fe495f83640 1 -- 172.21.15.84:0/2593561759 >> 172.21.15.84:0/2593561759 conn(0x7fe4900cb410 msgr2=0x7fe4900cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:44.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.186+0000 7fe495f83640 1 -- 172.21.15.84:0/2593561759 shutdown_connections 2023-10-16T11:24:44.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.186+0000 7fe495f83640 1 -- 172.21.15.84:0/2593561759 wait complete. 2023-10-16T11:24:44.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.188+0000 7fe495f83640 1 Processor -- start 2023-10-16T11:24:44.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.188+0000 7fe495f83640 1 -- start start 2023-10-16T11:24:44.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.189+0000 7fe495f83640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4900ce390 0x7fe4901f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:44.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.189+0000 7fe495f83640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe49015aaa0 0x7fe4901592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:44.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.190+0000 7fe495f83640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49015ae80 0x7fe4901479d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:44.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.190+0000 7fe495f83640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe490158e50 con 0x7fe4900ce390 2023-10-16T11:24:44.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.190+0000 7fe495f83640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4901585a0 con 0x7fe49015aaa0 2023-10-16T11:24:44.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.190+0000 7fe495f83640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe490158820 con 0x7fe49015ae80 2023-10-16T11:24:44.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.190+0000 7fe48f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4900ce390 0x7fe4901f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:44.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.190+0000 7fe48f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4900ce390 0x7fe4901f1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50076/0 (socket says 172.21.15.84:50076) 2023-10-16T11:24:44.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.190+0000 7fe48f7fe640 1 -- 172.21.15.84:0/2213414561 learned_addr learned my addr 172.21.15.84:0/2213414561 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:44.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.190+0000 7fe48effd640 1 --2- 172.21.15.84:0/2213414561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe49015aaa0 0x7fe4901592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:44.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.190+0000 7fe48ffff640 1 --2- 172.21.15.84:0/2213414561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49015ae80 0x7fe4901479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:44.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.190+0000 7fe48f7fe640 1 -- 172.21.15.84:0/2213414561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49015ae80 msgr2=0x7fe4901479d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:44.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.190+0000 7fe48f7fe640 1 --2- 172.21.15.84:0/2213414561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49015ae80 0x7fe4901479d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.190+0000 7fe48f7fe640 1 -- 172.21.15.84:0/2213414561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe49015aaa0 msgr2=0x7fe4901592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:44.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.190+0000 7fe48f7fe640 1 --2- 172.21.15.84:0/2213414561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe49015aaa0 0x7fe4901592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.190+0000 7fe48f7fe640 1 -- 172.21.15.84:0/2213414561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe488009b30 con 0x7fe4900ce390 2023-10-16T11:24:44.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.190+0000 7fe48ffff640 1 --2- 172.21.15.84:0/2213414561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49015ae80 0x7fe4901479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:44.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.191+0000 7fe48f7fe640 1 --2- 172.21.15.84:0/2213414561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4900ce390 0x7fe4901f1ed0 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7fe48000c400 tx=0x7fe48000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:44.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.191+0000 7fe477fff640 1 -- 172.21.15.84:0/2213414561 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4800052d0 con 0x7fe4900ce390 2023-10-16T11:24:44.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.191+0000 7fe477fff640 1 -- 172.21.15.84:0/2213414561 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe480005430 con 0x7fe4900ce390 2023-10-16T11:24:44.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.191+0000 7fe477fff640 1 -- 172.21.15.84:0/2213414561 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480003830 con 0x7fe4900ce390 2023-10-16T11:24:44.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.191+0000 7fe495f83640 1 -- 172.21.15.84:0/2213414561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4901f6900 con 0x7fe4900ce390 2023-10-16T11:24:44.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.191+0000 7fe495f83640 1 -- 172.21.15.84:0/2213414561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4901f2410 con 0x7fe4900ce390 2023-10-16T11:24:44.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.193+0000 7fe495f83640 1 -- 172.21.15.84:0/2213414561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49004e160 con 0x7fe4900ce390 2023-10-16T11:24:44.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.193+0000 7fe477fff640 1 -- 172.21.15.84:0/2213414561 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe480003990 con 0x7fe4900ce390 2023-10-16T11:24:44.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.194+0000 7fe477fff640 1 --2- 172.21.15.84:0/2213414561 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fe47c06af60 0x7fe47c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:44.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.194+0000 7fe477fff640 1 -- 172.21.15.84:0/2213414561 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7fe480089070 con 0x7fe4900ce390 2023-10-16T11:24:44.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.195+0000 7fe48effd640 1 --2- 172.21.15.84:0/2213414561 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fe47c06af60 0x7fe47c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:44.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.195+0000 7fe48effd640 1 --2- 172.21.15.84:0/2213414561 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fe47c06af60 0x7fe47c06d420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe4900b1230 tx=0x7fe484008040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:44.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.199+0000 7fe477fff640 1 -- 172.21.15.84:0/2213414561 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe480053c90 con 0x7fe4900ce390 2023-10-16T11:24:44.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.431+0000 7fe495f83640 1 -- 172.21.15.84:0/2213414561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4900cbcb0 con 0x7fe4900ce390 2023-10-16T11:24:44.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.432+0000 7fe477fff640 1 -- 172.21.15.84:0/2213414561 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+6152 (secure 0 0 0) 0x7fe4800572b0 con 0x7fe4900ce390 2023-10-16T11:24:44.435 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:44.435 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":120,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3106898227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3106898227},{"type":"v1","addr":"172.21.15.84:6837","nonce":3106898227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":7471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2296953749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2296953749},{"type":"v1","addr":"172.21.15.157:6834","nonce":2296953749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:24:25.531691+0000","modified":"2023-10-16T11:24:28.574116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11307},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11307":{"gid":11307,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/551895595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":551895595},{"type":"v1","addr":"172.21.15.84:6836","nonce":551895595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:24:40.366708+0000","modified":"2023-10-16T11:24:43.431532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7474":{"gid":7474,"name":"d","rank":0,"incarnation":116,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6835/3992377882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3992377882},{"type":"v1","addr":"172.21.15.157:6835","nonce":3992377882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-16T11:24:44.436 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 120 2023-10-16T11:24:44.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.435+0000 7fe495f83640 1 -- 172.21.15.84:0/2213414561 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fe47c06af60 msgr2=0x7fe47c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:44.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.435+0000 7fe495f83640 1 --2- 172.21.15.84:0/2213414561 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fe47c06af60 0x7fe47c06d420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe4900b1230 tx=0x7fe484008040 comp rx=0 tx=0).stop 2023-10-16T11:24:44.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.435+0000 7fe495f83640 1 -- 172.21.15.84:0/2213414561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4900ce390 msgr2=0x7fe4901f1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:44.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.435+0000 7fe495f83640 1 --2- 172.21.15.84:0/2213414561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4900ce390 0x7fe4901f1ed0 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7fe48000c400 tx=0x7fe48000c8c0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.436+0000 7fe495f83640 1 -- 172.21.15.84:0/2213414561 shutdown_connections 2023-10-16T11:24:44.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.436+0000 7fe495f83640 1 --2- 172.21.15.84:0/2213414561 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fe47c06af60 0x7fe47c06d420 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.436+0000 7fe495f83640 1 --2- 172.21.15.84:0/2213414561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49015ae80 0x7fe4901479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.436+0000 7fe495f83640 1 --2- 172.21.15.84:0/2213414561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe49015aaa0 0x7fe4901592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.436+0000 7fe495f83640 1 --2- 172.21.15.84:0/2213414561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4900ce390 0x7fe4901f1ed0 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.436+0000 7fe495f83640 1 -- 172.21.15.84:0/2213414561 >> 172.21.15.84:0/2213414561 conn(0x7fe4900cb410 msgr2=0x7fe4900b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:44.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.436+0000 7fe495f83640 1 -- 172.21.15.84:0/2213414561 shutdown_connections 2023-10-16T11:24:44.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.436+0000 7fe495f83640 1 -- 172.21.15.84:0/2213414561 wait complete. 2023-10-16T11:24:44.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:24:44.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.668+0000 7f0381c83640 1 Processor -- start 2023-10-16T11:24:44.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.668+0000 7f0381c83640 1 -- start start 2023-10-16T11:24:44.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.670+0000 7f0381c83640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f037c15ae80 0x7f037c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:44.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.670+0000 7f0381c83640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f037c0ce390 0x7f037c070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:44.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.670+0000 7f0381c83640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f037c15ed60 con 0x7f037c15aaa0 2023-10-16T11:24:44.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.670+0000 7f0381c83640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f037c06b810 con 0x7f037c15ae80 2023-10-16T11:24:44.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.670+0000 7f0381c83640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f037c0707e0 con 0x7f037c0ce390 2023-10-16T11:24:44.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.670+0000 7f0373fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f037c15ae80 0x7f037c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:44.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.670+0000 7f0373fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f037c15ae80 0x7f037c15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50092/0 (socket says 172.21.15.84:50092) 2023-10-16T11:24:44.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.670+0000 7f0373fff640 1 -- 172.21.15.84:0/9568583 learned_addr learned my addr 172.21.15.84:0/9568583 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:44.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.670+0000 7f037b7fe640 1 --1- 172.21.15.84:0/9568583 >> v1:172.21.15.84:6789/0 conn(0x7f037c15aaa0 0x7f037c0cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38544/0 (socket says 172.21.15.84:38544) 2023-10-16T11:24:44.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.671+0000 7f037bfff640 1 --2- 172.21.15.84:0/9568583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f037c0ce390 0x7f037c070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:44.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.671+0000 7f0373fff640 1 -- 172.21.15.84:0/9568583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f037c0ce390 msgr2=0x7f037c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:44.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.671+0000 7f0373fff640 1 --2- 172.21.15.84:0/9568583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f037c0ce390 0x7f037c070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.671+0000 7f0373fff640 1 -- 172.21.15.84:0/9568583 >> v1:172.21.15.84:6789/0 conn(0x7f037c15aaa0 legacy=0x7f037c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:44.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.671+0000 7f0373fff640 1 -- 172.21.15.84:0/9568583 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037c15ef90 con 0x7f037c15ae80 2023-10-16T11:24:44.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.672+0000 7f0373fff640 1 --2- 172.21.15.84:0/9568583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f037c15ae80 0x7f037c15e820 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f0364009870 tx=0x7f036402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0cc7d0fbf4720f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:44.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.672+0000 7f037affd640 1 -- 172.21.15.84:0/9568583 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364030840 con 0x7f037c15ae80 2023-10-16T11:24:44.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.672+0000 7f037affd640 1 -- 172.21.15.84:0/9568583 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03640309a0 con 0x7f037c15ae80 2023-10-16T11:24:44.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.672+0000 7f037affd640 1 -- 172.21.15.84:0/9568583 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364030d30 con 0x7f037c15ae80 2023-10-16T11:24:44.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.673+0000 7f0381c83640 1 -- 172.21.15.84:0/9568583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f037c15ae80 msgr2=0x7f037c15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:44.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.673+0000 7f0381c83640 1 --2- 172.21.15.84:0/9568583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f037c15ae80 0x7f037c15e820 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f0364009870 tx=0x7f036402b980 comp rx=0 tx=0).stop 2023-10-16T11:24:44.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.673+0000 7f0381c83640 1 -- 172.21.15.84:0/9568583 shutdown_connections 2023-10-16T11:24:44.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.673+0000 7f0381c83640 1 --2- 172.21.15.84:0/9568583 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f037c0ce390 0x7f037c070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.673+0000 7f0381c83640 1 --2- 172.21.15.84:0/9568583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f037c15ae80 0x7f037c15e820 unknown :-1 s=CLOSED pgs=2679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.673+0000 7f0381c83640 1 -- 172.21.15.84:0/9568583 >> 172.21.15.84:0/9568583 conn(0x7f037c0cb410 msgr2=0x7f037c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:44.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.673+0000 7f0381c83640 1 -- 172.21.15.84:0/9568583 shutdown_connections 2023-10-16T11:24:44.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.673+0000 7f0381c83640 1 -- 172.21.15.84:0/9568583 wait complete. 2023-10-16T11:24:44.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.675+0000 7f0381c83640 1 Processor -- start 2023-10-16T11:24:44.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.675+0000 7f0381c83640 1 -- start start 2023-10-16T11:24:44.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.676+0000 7f0381c83640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f037c0ce390 0x7f037c15de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:44.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.676+0000 7f0381c83640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f037c15aaa0 0x7f037c0c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:44.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.676+0000 7f0381c83640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f037c15ae80 0x7f037c0c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:44.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.676+0000 7f0381c83640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037c157770 con 0x7f037c15aaa0 2023-10-16T11:24:44.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.676+0000 7f0381c83640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037c15b2b0 con 0x7f037c15ae80 2023-10-16T11:24:44.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.676+0000 7f0381c83640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f037c15bb50 con 0x7f037c0ce390 2023-10-16T11:24:44.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.676+0000 7f0373fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f037c15aaa0 0x7f037c0c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:44.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.676+0000 7f0373fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f037c15aaa0 0x7f037c0c4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50104/0 (socket says 172.21.15.84:50104) 2023-10-16T11:24:44.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.676+0000 7f0373fff640 1 -- 172.21.15.84:0/3617676083 learned_addr learned my addr 172.21.15.84:0/3617676083 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:44.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.676+0000 7f0373fff640 1 -- 172.21.15.84:0/3617676083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f037c0ce390 msgr2=0x7f037c15de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:44.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.676+0000 7f0373fff640 1 --2- 172.21.15.84:0/3617676083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f037c0ce390 0x7f037c15de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.676+0000 7f0373fff640 1 -- 172.21.15.84:0/3617676083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f037c15ae80 msgr2=0x7f037c0c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:44.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.676+0000 7f037bfff640 1 --2- 172.21.15.84:0/3617676083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f037c15ae80 0x7f037c0c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:44.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.676+0000 7f0373fff640 1 --2- 172.21.15.84:0/3617676083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f037c15ae80 0x7f037c0c4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.676+0000 7f0373fff640 1 -- 172.21.15.84:0/3617676083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03640094d0 con 0x7f037c15aaa0 2023-10-16T11:24:44.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.677+0000 7f0373fff640 1 --2- 172.21.15.84:0/3617676083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f037c15aaa0 0x7f037c0c4690 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7f0364000c00 tx=0x7f0364004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:44.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.677+0000 7f0378ff9640 1 -- 172.21.15.84:0/3617676083 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364030810 con 0x7f037c15aaa0 2023-10-16T11:24:44.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.677+0000 7f0378ff9640 1 -- 172.21.15.84:0/3617676083 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0364039d90 con 0x7f037c15aaa0 2023-10-16T11:24:44.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.677+0000 7f0378ff9640 1 -- 172.21.15.84:0/3617676083 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364043670 con 0x7f037c15aaa0 2023-10-16T11:24:44.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.677+0000 7f0381c83640 1 -- 172.21.15.84:0/3617676083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f037c156270 con 0x7f037c15aaa0 2023-10-16T11:24:44.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.677+0000 7f0381c83640 1 -- 172.21.15.84:0/3617676083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f037c15b500 con 0x7f037c15aaa0 2023-10-16T11:24:44.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.678+0000 7f0381c83640 1 -- 172.21.15.84:0/3617676083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f037c04e160 con 0x7f037c15aaa0 2023-10-16T11:24:44.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.679+0000 7f0378ff9640 1 -- 172.21.15.84:0/3617676083 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f036402f390 con 0x7f037c15aaa0 2023-10-16T11:24:44.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.680+0000 7f0378ff9640 1 --2- 172.21.15.84:0/3617676083 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f035c06af60 0x7f035c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:44.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.680+0000 7f0378ff9640 1 -- 172.21.15.84:0/3617676083 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f03640ac0b0 con 0x7f037c15aaa0 2023-10-16T11:24:44.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.680+0000 7f037b7fe640 1 --2- 172.21.15.84:0/3617676083 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f035c06af60 0x7f035c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:44.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.681+0000 7f037b7fe640 1 --2- 172.21.15.84:0/3617676083 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f035c06af60 0x7f035c06d420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f036c007fd0 tx=0x7f036c007980 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:44.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.685+0000 7f0378ff9640 1 -- 172.21.15.84:0/3617676083 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0364076b20 con 0x7f037c15aaa0 2023-10-16T11:24:44.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.876+0000 7f0381c83640 1 -- 172.21.15.84:0/3617676083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f037c15bcc0 con 0x7f037c15aaa0 2023-10-16T11:24:44.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.879+0000 7f0378ff9640 1 -- 172.21.15.84:0/3617676083 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v221) v1 ==== 74+0+20327 (secure 0 0 0) 0x7f037c15bcc0 con 0x7f037c15aaa0 2023-10-16T11:24:44.882 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:44.882 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":221,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:24:41.917348+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:24:23.363032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:24:24.551270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:24:38.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:24:39.387564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:24:44.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.882+0000 7f0381c83640 1 -- 172.21.15.84:0/3617676083 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f035c06af60 msgr2=0x7f035c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:44.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.882+0000 7f0381c83640 1 --2- 172.21.15.84:0/3617676083 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f035c06af60 0x7f035c06d420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f036c007fd0 tx=0x7f036c007980 comp rx=0 tx=0).stop 2023-10-16T11:24:44.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.882+0000 7f0381c83640 1 -- 172.21.15.84:0/3617676083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f037c15aaa0 msgr2=0x7f037c0c4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:44.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.882+0000 7f0381c83640 1 --2- 172.21.15.84:0/3617676083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f037c15aaa0 0x7f037c0c4690 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7f0364000c00 tx=0x7f0364004730 comp rx=0 tx=0).stop 2023-10-16T11:24:44.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.882+0000 7f0381c83640 1 -- 172.21.15.84:0/3617676083 shutdown_connections 2023-10-16T11:24:44.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.882+0000 7f0381c83640 1 --2- 172.21.15.84:0/3617676083 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f035c06af60 0x7f035c06d420 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.882+0000 7f0381c83640 1 --2- 172.21.15.84:0/3617676083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f037c15ae80 0x7f037c0c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.882+0000 7f0381c83640 1 --2- 172.21.15.84:0/3617676083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f037c15aaa0 0x7f037c0c4690 unknown :-1 s=CLOSED pgs=2680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.882+0000 7f0381c83640 1 --2- 172.21.15.84:0/3617676083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f037c0ce390 0x7f037c15de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:44.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.882+0000 7f0381c83640 1 -- 172.21.15.84:0/3617676083 >> 172.21.15.84:0/3617676083 conn(0x7f037c0cb410 msgr2=0x7f037c0b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:44.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.883+0000 7f0381c83640 1 -- 172.21.15.84:0/3617676083 shutdown_connections 2023-10-16T11:24:44.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:44.883+0000 7f0381c83640 1 -- 172.21.15.84:0/3617676083 wait complete. 2023-10-16T11:24:44.902 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T11:24:44.902 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:24:45.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.114+0000 7fcd912d7640 1 Processor -- start 2023-10-16T11:24:45.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.114+0000 7fcd912d7640 1 -- start start 2023-10-16T11:24:45.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.116+0000 7fcd912d7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8c15aaa0 0x7fcd8c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:45.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.116+0000 7fcd912d7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcd8c15ae80 0x7fcd8c0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:45.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.116+0000 7fcd912d7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c15ed60 con 0x7fcd8c0ccaa0 2023-10-16T11:24:45.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.116+0000 7fcd912d7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd8c06b810 con 0x7fcd8c15ae80 2023-10-16T11:24:45.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.116+0000 7fcd912d7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0707e0 con 0x7fcd8c15aaa0 2023-10-16T11:24:45.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.116+0000 7fcd8a7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcd8c15ae80 0x7fcd8c0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:45.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.116+0000 7fcd8a7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcd8c15ae80 0x7fcd8c0cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50114/0 (socket says 172.21.15.84:50114) 2023-10-16T11:24:45.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.116+0000 7fcd8a7fc640 1 -- 172.21.15.84:0/975788473 learned_addr learned my addr 172.21.15.84:0/975788473 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:45.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.117+0000 7fcd8affd640 1 --2- 172.21.15.84:0/975788473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8c15aaa0 0x7fcd8c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:45.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.117+0000 7fcd8a7fc640 1 -- 172.21.15.84:0/975788473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8c15aaa0 msgr2=0x7fcd8c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:45.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.117+0000 7fcd8a7fc640 1 --2- 172.21.15.84:0/975788473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8c15aaa0 0x7fcd8c15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.117+0000 7fcd8a7fc640 1 -- 172.21.15.84:0/975788473 >> v1:172.21.15.157:6789/0 conn(0x7fcd8c0ccaa0 legacy=0x7fcd8c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:45.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.117+0000 7fcd8a7fc640 1 -- 172.21.15.84:0/975788473 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c15ef90 con 0x7fcd8c15ae80 2023-10-16T11:24:45.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.118+0000 7fcd8a7fc640 1 --2- 172.21.15.84:0/975788473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd8c15ae80 0x7fcd8c0cc560 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009e50 tx=0x7fcd7402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1eec86dffbdda8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:45.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.118+0000 7fcd89ffb640 1 -- 172.21.15.84:0/975788473 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7402ca80 con 0x7fcd8c15ae80 2023-10-16T11:24:45.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.118+0000 7fcd89ffb640 1 -- 172.21.15.84:0/975788473 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7402cbe0 con 0x7fcd8c15ae80 2023-10-16T11:24:45.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.118+0000 7fcd89ffb640 1 -- 172.21.15.84:0/975788473 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd740343f0 con 0x7fcd8c15ae80 2023-10-16T11:24:45.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.119+0000 7fcd912d7640 1 -- 172.21.15.84:0/975788473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd8c15ae80 msgr2=0x7fcd8c0cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:45.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.119+0000 7fcd912d7640 1 --2- 172.21.15.84:0/975788473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd8c15ae80 0x7fcd8c0cc560 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009e50 tx=0x7fcd7402bee0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.119+0000 7fcd912d7640 1 -- 172.21.15.84:0/975788473 shutdown_connections 2023-10-16T11:24:45.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.119+0000 7fcd912d7640 1 --2- 172.21.15.84:0/975788473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd8c15ae80 0x7fcd8c0cc560 unknown :-1 s=CLOSED pgs=2681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.119+0000 7fcd912d7640 1 --2- 172.21.15.84:0/975788473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8c15aaa0 0x7fcd8c15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.119+0000 7fcd912d7640 1 -- 172.21.15.84:0/975788473 >> 172.21.15.84:0/975788473 conn(0x7fcd8c0cb410 msgr2=0x7fcd8c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:45.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.119+0000 7fcd912d7640 1 -- 172.21.15.84:0/975788473 shutdown_connections 2023-10-16T11:24:45.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.119+0000 7fcd912d7640 1 -- 172.21.15.84:0/975788473 wait complete. 2023-10-16T11:24:45.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.121+0000 7fcd912d7640 1 Processor -- start 2023-10-16T11:24:45.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.121+0000 7fcd912d7640 1 -- start start 2023-10-16T11:24:45.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.122+0000 7fcd912d7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd8c0ccaa0 0x7fcd8c0c2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:45.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.122+0000 7fcd912d7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8c15aaa0 0x7fcd8c0ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:45.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.122+0000 7fcd912d7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd8c15ae80 0x7fcd8c15d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:45.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.122+0000 7fcd912d7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c14cb10 con 0x7fcd8c15ae80 2023-10-16T11:24:45.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.122+0000 7fcd912d7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c14bc40 con 0x7fcd8c0ccaa0 2023-10-16T11:24:45.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.122+0000 7fcd912d7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c14be90 con 0x7fcd8c15aaa0 2023-10-16T11:24:45.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.122+0000 7fcd8b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd8c15ae80 0x7fcd8c15d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:45.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.122+0000 7fcd8b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd8c15ae80 0x7fcd8c15d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50122/0 (socket says 172.21.15.84:50122) 2023-10-16T11:24:45.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.122+0000 7fcd8b7fe640 1 -- 172.21.15.84:0/528554197 learned_addr learned my addr 172.21.15.84:0/528554197 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:45.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.122+0000 7fcd8a7fc640 1 --2- 172.21.15.84:0/528554197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8c15aaa0 0x7fcd8c0ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:45.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.122+0000 7fcd8affd640 1 --2- 172.21.15.84:0/528554197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd8c0ccaa0 0x7fcd8c0c2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:45.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.123+0000 7fcd8b7fe640 1 -- 172.21.15.84:0/528554197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8c15aaa0 msgr2=0x7fcd8c0ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:45.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.123+0000 7fcd8b7fe640 1 --2- 172.21.15.84:0/528554197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8c15aaa0 0x7fcd8c0ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.123+0000 7fcd8b7fe640 1 -- 172.21.15.84:0/528554197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd8c0ccaa0 msgr2=0x7fcd8c0c2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:45.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.123+0000 7fcd8b7fe640 1 --2- 172.21.15.84:0/528554197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd8c0ccaa0 0x7fcd8c0c2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.123+0000 7fcd8b7fe640 1 -- 172.21.15.84:0/528554197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd74009b30 con 0x7fcd8c15ae80 2023-10-16T11:24:45.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.123+0000 7fcd8b7fe640 1 --2- 172.21.15.84:0/528554197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd8c15ae80 0x7fcd8c15d340 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400b490 tx=0x7fcd8400b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:45.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.123+0000 7fcd837fe640 1 -- 172.21.15.84:0/528554197 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84011830 con 0x7fcd8c15ae80 2023-10-16T11:24:45.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.123+0000 7fcd837fe640 1 -- 172.21.15.84:0/528554197 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd840119b0 con 0x7fcd8c15ae80 2023-10-16T11:24:45.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.123+0000 7fcd912d7640 1 -- 172.21.15.84:0/528554197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c14c170 con 0x7fcd8c15ae80 2023-10-16T11:24:45.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.123+0000 7fcd837fe640 1 -- 172.21.15.84:0/528554197 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84011d60 con 0x7fcd8c15ae80 2023-10-16T11:24:45.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.123+0000 7fcd912d7640 1 -- 172.21.15.84:0/528554197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c14c730 con 0x7fcd8c15ae80 2023-10-16T11:24:45.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.124+0000 7fcd912d7640 1 -- 172.21.15.84:0/528554197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8c0b1bc0 con 0x7fcd8c15ae80 2023-10-16T11:24:45.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.126+0000 7fcd837fe640 1 -- 172.21.15.84:0/528554197 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcd8401a400 con 0x7fcd8c15ae80 2023-10-16T11:24:45.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.127+0000 7fcd837fe640 1 --2- 172.21.15.84:0/528554197 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fcd6c06af90 0x7fcd6c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:45.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.127+0000 7fcd837fe640 1 -- 172.21.15.84:0/528554197 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7fcd84089c90 con 0x7fcd8c15ae80 2023-10-16T11:24:45.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.127+0000 7fcd8affd640 1 --2- 172.21.15.84:0/528554197 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fcd6c06af90 0x7fcd6c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:45.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.128+0000 7fcd8affd640 1 --2- 172.21.15.84:0/528554197 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fcd6c06af90 0x7fcd6c06d450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c006fd0 tx=0x7fcd7c008040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:45.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.132+0000 7fcd837fe640 1 -- 172.21.15.84:0/528554197 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcd84054810 con 0x7fcd8c15ae80 2023-10-16T11:24:45.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.323+0000 7fcd912d7640 1 -- 172.21.15.84:0/528554197 --> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd8c15f550 con 0x7fcd6c06af90 2023-10-16T11:24:45.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.353+0000 7fcd837fe640 1 -- 172.21.15.84:0/528554197 <== mgr.11076 v2:172.21.15.84:6832/1425409017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+514753 (secure 0 0 0) 0x7fcd7c00c050 con 0x7fcd6c06af90 2023-10-16T11:24:45.364 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:45.367 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:24:45.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.364+0000 7fcd912d7640 1 -- 172.21.15.84:0/528554197 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fcd6c06af90 msgr2=0x7fcd6c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:45.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.364+0000 7fcd912d7640 1 --2- 172.21.15.84:0/528554197 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fcd6c06af90 0x7fcd6c06d450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c006fd0 tx=0x7fcd7c008040 comp rx=0 tx=0).stop 2023-10-16T11:24:45.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.364+0000 7fcd912d7640 1 -- 172.21.15.84:0/528554197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd8c15ae80 msgr2=0x7fcd8c15d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:45.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.364+0000 7fcd912d7640 1 --2- 172.21.15.84:0/528554197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd8c15ae80 0x7fcd8c15d340 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400b490 tx=0x7fcd8400b950 comp rx=0 tx=0).stop 2023-10-16T11:24:45.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.364+0000 7fcd912d7640 1 -- 172.21.15.84:0/528554197 shutdown_connections 2023-10-16T11:24:45.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.364+0000 7fcd912d7640 1 --2- 172.21.15.84:0/528554197 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fcd6c06af90 0x7fcd6c06d450 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.364+0000 7fcd912d7640 1 --2- 172.21.15.84:0/528554197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd8c15ae80 0x7fcd8c15d340 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.364+0000 7fcd912d7640 1 --2- 172.21.15.84:0/528554197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8c15aaa0 0x7fcd8c0ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.364+0000 7fcd912d7640 1 --2- 172.21.15.84:0/528554197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd8c0ccaa0 0x7fcd8c0c2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.364+0000 7fcd912d7640 1 -- 172.21.15.84:0/528554197 >> 172.21.15.84:0/528554197 conn(0x7fcd8c0cb410 msgr2=0x7fcd8c0b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:45.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.364+0000 7fcd912d7640 1 -- 172.21.15.84:0/528554197 shutdown_connections 2023-10-16T11:24:45.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.364+0000 7fcd912d7640 1 -- 172.21.15.84:0/528554197 wait complete. 2023-10-16T11:24:45.387 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2023-10-16T11:24:44.256729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594605,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":635,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2814,"ondisk_log_size":2814,"up":266,"acting":266,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":215,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":380328,"kb_used_data":22584,"kb_used_omap":12,"kb_used_meta":357619,"kb_avail":754594392,"statfs":{"total":773094113280,"available":772704657408,"internally_reserved":0,"allocated":23126016,"data_stored":11794435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":366202425},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1569,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.377056"},"pg_stats":[{"pgid":"29.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"206'2767","reported_seq":3649,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069366+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:24:28.069366+0000","last_peered":"2023-10-16T11:24:28.069366+0000","last_clean":"2023-10-16T11:24:28.069366+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:24:28.069366+0000","last_undegraded":"2023-10-16T11:24:28.069366+0000","last_fullsized":"2023-10-16T11:24:28.069366+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2767,"log_dups_size":0,"ondisk_log_size":2767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":581,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"216'1","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.976811+0000","last_change":"2023-10-16T11:24:25.087491+0000","last_active":"2023-10-16T11:24:27.976811+0000","last_peered":"2023-10-16T11:24:27.976811+0000","last_clean":"2023-10-16T11:24:27.976811+0000","last_became_active":"2023-10-16T11:24:25.086826+0000","last_became_peered":"2023-10-16T11:24:25.086826+0000","last_unstale":"2023-10-16T11:24:27.976811+0000","last_undegraded":"2023-10-16T11:24:27.976811+0000","last_fullsized":"2023-10-16T11:24:27.976811+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:07.303719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918232+0000","last_change":"2023-10-16T11:24:26.091340+0000","last_active":"2023-10-16T11:24:40.918232+0000","last_peered":"2023-10-16T11:24:40.918232+0000","last_clean":"2023-10-16T11:24:40.918232+0000","last_became_active":"2023-10-16T11:24:26.091059+0000","last_became_peered":"2023-10-16T11:24:26.091059+0000","last_unstale":"2023-10-16T11:24:40.918232+0000","last_undegraded":"2023-10-16T11:24:40.918232+0000","last_fullsized":"2023-10-16T11:24:40.918232+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:37.605400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.892205+0000","last_change":"2023-10-16T11:24:38.892205+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.892205+0000","last_undegraded":"2023-10-16T11:24:38.892205+0000","last_fullsized":"2023-10-16T11:24:38.892205+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.975997+0000","last_change":"2023-10-16T11:24:25.087755+0000","last_active":"2023-10-16T11:24:27.975997+0000","last_peered":"2023-10-16T11:24:27.975997+0000","last_clean":"2023-10-16T11:24:27.975997+0000","last_became_active":"2023-10-16T11:24:25.087497+0000","last_became_peered":"2023-10-16T11:24:25.087497+0000","last_unstale":"2023-10-16T11:24:27.975997+0000","last_undegraded":"2023-10-16T11:24:27.975997+0000","last_fullsized":"2023-10-16T11:24:27.975997+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:59:26.196708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255404+0000","last_change":"2023-10-16T11:24:26.081718+0000","last_active":"2023-10-16T11:24:29.255404+0000","last_peered":"2023-10-16T11:24:29.255404+0000","last_clean":"2023-10-16T11:24:29.255404+0000","last_became_active":"2023-10-16T11:24:26.081447+0000","last_became_peered":"2023-10-16T11:24:26.081447+0000","last_unstale":"2023-10-16T11:24:29.255404+0000","last_undegraded":"2023-10-16T11:24:29.255404+0000","last_fullsized":"2023-10-16T11:24:29.255404+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.884705+0000","last_change":"2023-10-16T11:24:38.884705+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.884705+0000","last_undegraded":"2023-10-16T11:24:38.884705+0000","last_fullsized":"2023-10-16T11:24:38.884705+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.083691+0000","last_change":"2023-10-16T11:24:25.088161+0000","last_active":"2023-10-16T11:24:27.083691+0000","last_peered":"2023-10-16T11:24:27.083691+0000","last_clean":"2023-10-16T11:24:27.083691+0000","last_became_active":"2023-10-16T11:24:25.086051+0000","last_became_peered":"2023-10-16T11:24:25.086051+0000","last_unstale":"2023-10-16T11:24:27.083691+0000","last_undegraded":"2023-10-16T11:24:27.083691+0000","last_fullsized":"2023-10-16T11:24:27.083691+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.257788+0000","last_change":"2023-10-16T11:24:26.083366+0000","last_active":"2023-10-16T11:24:29.257788+0000","last_peered":"2023-10-16T11:24:29.257788+0000","last_clean":"2023-10-16T11:24:29.257788+0000","last_became_active":"2023-10-16T11:24:26.082635+0000","last_became_peered":"2023-10-16T11:24:26.082635+0000","last_unstale":"2023-10-16T11:24:29.257788+0000","last_undegraded":"2023-10-16T11:24:29.257788+0000","last_fullsized":"2023-10-16T11:24:29.257788+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.890077+0000","last_change":"2023-10-16T11:24:38.890077+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.890077+0000","last_undegraded":"2023-10-16T11:24:38.890077+0000","last_fullsized":"2023-10-16T11:24:38.890077+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.071234+0000","last_change":"2023-10-16T11:24:25.080689+0000","last_active":"2023-10-16T11:24:28.071234+0000","last_peered":"2023-10-16T11:24:28.071234+0000","last_clean":"2023-10-16T11:24:28.071234+0000","last_became_active":"2023-10-16T11:24:25.080305+0000","last_became_peered":"2023-10-16T11:24:25.080305+0000","last_unstale":"2023-10-16T11:24:28.071234+0000","last_undegraded":"2023-10-16T11:24:28.071234+0000","last_fullsized":"2023-10-16T11:24:28.071234+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:43.670603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.254685+0000","last_change":"2023-10-16T11:24:26.083766+0000","last_active":"2023-10-16T11:24:29.254685+0000","last_peered":"2023-10-16T11:24:29.254685+0000","last_clean":"2023-10-16T11:24:29.254685+0000","last_became_active":"2023-10-16T11:24:26.083447+0000","last_became_peered":"2023-10-16T11:24:26.083447+0000","last_unstale":"2023-10-16T11:24:29.254685+0000","last_undegraded":"2023-10-16T11:24:29.254685+0000","last_fullsized":"2023-10-16T11:24:29.254685+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:44:14.656443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.901288+0000","last_change":"2023-10-16T11:24:38.901288+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.901288+0000","last_undegraded":"2023-10-16T11:24:38.901288+0000","last_fullsized":"2023-10-16T11:24:38.901288+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069569+0000","last_change":"2023-10-16T11:24:25.090432+0000","last_active":"2023-10-16T11:24:28.069569+0000","last_peered":"2023-10-16T11:24:28.069569+0000","last_clean":"2023-10-16T11:24:28.069569+0000","last_became_active":"2023-10-16T11:24:25.090142+0000","last_became_peered":"2023-10-16T11:24:25.090142+0000","last_unstale":"2023-10-16T11:24:28.069569+0000","last_undegraded":"2023-10-16T11:24:28.069569+0000","last_fullsized":"2023-10-16T11:24:28.069569+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:22.858690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918327+0000","last_change":"2023-10-16T11:24:26.093027+0000","last_active":"2023-10-16T11:24:40.918327+0000","last_peered":"2023-10-16T11:24:40.918327+0000","last_clean":"2023-10-16T11:24:40.918327+0000","last_became_active":"2023-10-16T11:24:26.092766+0000","last_became_peered":"2023-10-16T11:24:26.092766+0000","last_unstale":"2023-10-16T11:24:40.918327+0000","last_undegraded":"2023-10-16T11:24:40.918327+0000","last_fullsized":"2023-10-16T11:24:40.918327+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.884720+0000","last_change":"2023-10-16T11:24:38.884720+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.884720+0000","last_undegraded":"2023-10-16T11:24:38.884720+0000","last_fullsized":"2023-10-16T11:24:38.884720+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.080926+0000","last_change":"2023-10-16T11:24:25.088585+0000","last_active":"2023-10-16T11:24:27.080926+0000","last_peered":"2023-10-16T11:24:27.080926+0000","last_clean":"2023-10-16T11:24:27.080926+0000","last_became_active":"2023-10-16T11:24:25.088011+0000","last_became_peered":"2023-10-16T11:24:25.088011+0000","last_unstale":"2023-10-16T11:24:27.080926+0000","last_undegraded":"2023-10-16T11:24:27.080926+0000","last_fullsized":"2023-10-16T11:24:27.080926+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.256412+0000","last_change":"2023-10-16T11:24:26.096341+0000","last_active":"2023-10-16T11:24:29.256412+0000","last_peered":"2023-10-16T11:24:29.256412+0000","last_clean":"2023-10-16T11:24:29.256412+0000","last_became_active":"2023-10-16T11:24:26.095836+0000","last_became_peered":"2023-10-16T11:24:26.095836+0000","last_unstale":"2023-10-16T11:24:29.256412+0000","last_undegraded":"2023-10-16T11:24:29.256412+0000","last_fullsized":"2023-10-16T11:24:29.256412+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917221+0000","last_change":"2023-10-16T11:24:39.909816+0000","last_active":"2023-10-16T11:24:40.917221+0000","last_peered":"2023-10-16T11:24:40.917221+0000","last_clean":"2023-10-16T11:24:40.917221+0000","last_became_active":"2023-10-16T11:24:39.909526+0000","last_became_peered":"2023-10-16T11:24:39.909526+0000","last_unstale":"2023-10-16T11:24:40.917221+0000","last_undegraded":"2023-10-16T11:24:40.917221+0000","last_fullsized":"2023-10-16T11:24:40.917221+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:12.120581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070811+0000","last_change":"2023-10-16T11:24:25.077036+0000","last_active":"2023-10-16T11:24:28.070811+0000","last_peered":"2023-10-16T11:24:28.070811+0000","last_clean":"2023-10-16T11:24:28.070811+0000","last_became_active":"2023-10-16T11:24:25.076342+0000","last_became_peered":"2023-10-16T11:24:25.076342+0000","last_unstale":"2023-10-16T11:24:28.070811+0000","last_undegraded":"2023-10-16T11:24:28.070811+0000","last_fullsized":"2023-10-16T11:24:28.070811+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:36.162662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255460+0000","last_change":"2023-10-16T11:24:26.085231+0000","last_active":"2023-10-16T11:24:29.255460+0000","last_peered":"2023-10-16T11:24:29.255460+0000","last_clean":"2023-10-16T11:24:29.255460+0000","last_became_active":"2023-10-16T11:24:26.084889+0000","last_became_peered":"2023-10-16T11:24:26.084889+0000","last_unstale":"2023-10-16T11:24:29.255460+0000","last_undegraded":"2023-10-16T11:24:29.255460+0000","last_fullsized":"2023-10-16T11:24:29.255460+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.891881+0000","last_change":"2023-10-16T11:24:38.891881+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.891881+0000","last_undegraded":"2023-10-16T11:24:38.891881+0000","last_fullsized":"2023-10-16T11:24:38.891881+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.6","version":"217'1","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:30.685218+0000","last_change":"2023-10-16T11:24:25.093214+0000","last_active":"2023-10-16T11:24:30.685218+0000","last_peered":"2023-10-16T11:24:30.685218+0000","last_clean":"2023-10-16T11:24:30.685218+0000","last_became_active":"2023-10-16T11:24:25.092877+0000","last_became_peered":"2023-10-16T11:24:25.092877+0000","last_unstale":"2023-10-16T11:24:30.685218+0000","last_undegraded":"2023-10-16T11:24:30.685218+0000","last_fullsized":"2023-10-16T11:24:30.685218+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:59.924424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.919655+0000","last_change":"2023-10-16T11:24:26.089940+0000","last_active":"2023-10-16T11:24:40.919655+0000","last_peered":"2023-10-16T11:24:40.919655+0000","last_clean":"2023-10-16T11:24:40.919655+0000","last_became_active":"2023-10-16T11:24:26.087906+0000","last_became_peered":"2023-10-16T11:24:26.087906+0000","last_unstale":"2023-10-16T11:24:40.919655+0000","last_undegraded":"2023-10-16T11:24:40.919655+0000","last_fullsized":"2023-10-16T11:24:40.919655+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:44.687193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.901030+0000","last_change":"2023-10-16T11:24:38.901030+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.901030+0000","last_undegraded":"2023-10-16T11:24:38.901030+0000","last_fullsized":"2023-10-16T11:24:38.901030+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.7","version":"216'2","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.081261+0000","last_change":"2023-10-16T11:24:25.094230+0000","last_active":"2023-10-16T11:24:27.081261+0000","last_peered":"2023-10-16T11:24:27.081261+0000","last_clean":"2023-10-16T11:24:27.081261+0000","last_became_active":"2023-10-16T11:24:25.093896+0000","last_became_peered":"2023-10-16T11:24:25.093896+0000","last_unstale":"2023-10-16T11:24:27.081261+0000","last_undegraded":"2023-10-16T11:24:27.081261+0000","last_fullsized":"2023-10-16T11:24:27.081261+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.254860+0000","last_change":"2023-10-16T11:24:26.085909+0000","last_active":"2023-10-16T11:24:29.254860+0000","last_peered":"2023-10-16T11:24:29.254860+0000","last_clean":"2023-10-16T11:24:29.254860+0000","last_became_active":"2023-10-16T11:24:26.085620+0000","last_became_peered":"2023-10-16T11:24:26.085620+0000","last_unstale":"2023-10-16T11:24:29.254860+0000","last_undegraded":"2023-10-16T11:24:29.254860+0000","last_fullsized":"2023-10-16T11:24:29.254860+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:41.023833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.889481+0000","last_change":"2023-10-16T11:24:38.889481+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.889481+0000","last_undegraded":"2023-10-16T11:24:38.889481+0000","last_fullsized":"2023-10-16T11:24:38.889481+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069148+0000","last_change":"2023-10-16T11:24:25.080225+0000","last_active":"2023-10-16T11:24:28.069148+0000","last_peered":"2023-10-16T11:24:28.069148+0000","last_clean":"2023-10-16T11:24:28.069148+0000","last_became_active":"2023-10-16T11:24:25.079921+0000","last_became_peered":"2023-10-16T11:24:25.079921+0000","last_unstale":"2023-10-16T11:24:28.069148+0000","last_undegraded":"2023-10-16T11:24:28.069148+0000","last_fullsized":"2023-10-16T11:24:28.069148+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:46.550429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.256329+0000","last_change":"2023-10-16T11:24:26.088438+0000","last_active":"2023-10-16T11:24:29.256329+0000","last_peered":"2023-10-16T11:24:29.256329+0000","last_clean":"2023-10-16T11:24:29.256329+0000","last_became_active":"2023-10-16T11:24:26.087973+0000","last_became_peered":"2023-10-16T11:24:26.087973+0000","last_unstale":"2023-10-16T11:24:29.256329+0000","last_undegraded":"2023-10-16T11:24:29.256329+0000","last_fullsized":"2023-10-16T11:24:29.256329+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.944794+0000","last_change":"2023-10-16T11:24:39.915986+0000","last_active":"2023-10-16T11:24:40.944794+0000","last_peered":"2023-10-16T11:24:40.944794+0000","last_clean":"2023-10-16T11:24:40.944794+0000","last_became_active":"2023-10-16T11:24:39.915611+0000","last_became_peered":"2023-10-16T11:24:39.915611+0000","last_unstale":"2023-10-16T11:24:40.944794+0000","last_undegraded":"2023-10-16T11:24:40.944794+0000","last_fullsized":"2023-10-16T11:24:40.944794+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:37.032264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917511+0000","last_change":"2023-10-16T11:24:25.086180+0000","last_active":"2023-10-16T11:24:40.917511+0000","last_peered":"2023-10-16T11:24:40.917511+0000","last_clean":"2023-10-16T11:24:40.917511+0000","last_became_active":"2023-10-16T11:24:25.085316+0000","last_became_peered":"2023-10-16T11:24:25.085316+0000","last_unstale":"2023-10-16T11:24:40.917511+0000","last_undegraded":"2023-10-16T11:24:40.917511+0000","last_fullsized":"2023-10-16T11:24:40.917511+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255372+0000","last_change":"2023-10-16T11:24:26.090951+0000","last_active":"2023-10-16T11:24:29.255372+0000","last_peered":"2023-10-16T11:24:29.255372+0000","last_clean":"2023-10-16T11:24:29.255372+0000","last_became_active":"2023-10-16T11:24:26.089241+0000","last_became_peered":"2023-10-16T11:24:26.089241+0000","last_unstale":"2023-10-16T11:24:29.255372+0000","last_undegraded":"2023-10-16T11:24:29.255372+0000","last_fullsized":"2023-10-16T11:24:29.255372+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:40.970357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.901898+0000","last_change":"2023-10-16T11:24:38.901898+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.901898+0000","last_undegraded":"2023-10-16T11:24:38.901898+0000","last_fullsized":"2023-10-16T11:24:38.901898+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918490+0000","last_change":"2023-10-16T11:24:25.091656+0000","last_active":"2023-10-16T11:24:40.918490+0000","last_peered":"2023-10-16T11:24:40.918490+0000","last_clean":"2023-10-16T11:24:40.918490+0000","last_became_active":"2023-10-16T11:24:25.091316+0000","last_became_peered":"2023-10-16T11:24:25.091316+0000","last_unstale":"2023-10-16T11:24:40.918490+0000","last_undegraded":"2023-10-16T11:24:40.918490+0000","last_fullsized":"2023-10-16T11:24:40.918490+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:26.054702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.257873+0000","last_change":"2023-10-16T11:24:26.085260+0000","last_active":"2023-10-16T11:24:29.257873+0000","last_peered":"2023-10-16T11:24:29.257873+0000","last_clean":"2023-10-16T11:24:29.257873+0000","last_became_active":"2023-10-16T11:24:26.084860+0000","last_became_peered":"2023-10-16T11:24:26.084860+0000","last_unstale":"2023-10-16T11:24:29.257873+0000","last_undegraded":"2023-10-16T11:24:29.257873+0000","last_fullsized":"2023-10-16T11:24:29.257873+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.894485+0000","last_change":"2023-10-16T11:24:38.894485+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.894485+0000","last_undegraded":"2023-10-16T11:24:38.894485+0000","last_fullsized":"2023-10-16T11:24:38.894485+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.b","version":"216'2","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.082378+0000","last_change":"2023-10-16T11:24:25.072340+0000","last_active":"2023-10-16T11:24:27.082378+0000","last_peered":"2023-10-16T11:24:27.082378+0000","last_clean":"2023-10-16T11:24:27.082378+0000","last_became_active":"2023-10-16T11:24:25.071728+0000","last_became_peered":"2023-10-16T11:24:25.071728+0000","last_unstale":"2023-10-16T11:24:27.082378+0000","last_undegraded":"2023-10-16T11:24:27.082378+0000","last_fullsized":"2023-10-16T11:24:27.082378+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:49.041521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.262758+0000","last_change":"2023-10-16T11:24:26.091096+0000","last_active":"2023-10-16T11:24:29.262758+0000","last_peered":"2023-10-16T11:24:29.262758+0000","last_clean":"2023-10-16T11:24:29.262758+0000","last_became_active":"2023-10-16T11:24:26.090774+0000","last_became_peered":"2023-10-16T11:24:26.090774+0000","last_unstale":"2023-10-16T11:24:29.262758+0000","last_undegraded":"2023-10-16T11:24:29.262758+0000","last_fullsized":"2023-10-16T11:24:29.262758+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:27.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.885799+0000","last_change":"2023-10-16T11:24:38.885799+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.885799+0000","last_undegraded":"2023-10-16T11:24:38.885799+0000","last_fullsized":"2023-10-16T11:24:38.885799+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.080804+0000","last_change":"2023-10-16T11:24:25.079021+0000","last_active":"2023-10-16T11:24:27.080804+0000","last_peered":"2023-10-16T11:24:27.080804+0000","last_clean":"2023-10-16T11:24:27.080804+0000","last_became_active":"2023-10-16T11:24:25.076800+0000","last_became_peered":"2023-10-16T11:24:25.076800+0000","last_unstale":"2023-10-16T11:24:27.080804+0000","last_undegraded":"2023-10-16T11:24:27.080804+0000","last_fullsized":"2023-10-16T11:24:27.080804+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:56.161407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.262906+0000","last_change":"2023-10-16T11:24:26.092876+0000","last_active":"2023-10-16T11:24:29.262906+0000","last_peered":"2023-10-16T11:24:29.262906+0000","last_clean":"2023-10-16T11:24:29.262906+0000","last_became_active":"2023-10-16T11:24:26.091310+0000","last_became_peered":"2023-10-16T11:24:26.091310+0000","last_unstale":"2023-10-16T11:24:29.262906+0000","last_undegraded":"2023-10-16T11:24:29.262906+0000","last_fullsized":"2023-10-16T11:24:29.262906+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:22.891780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.892205+0000","last_change":"2023-10-16T11:24:38.892205+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.892205+0000","last_undegraded":"2023-10-16T11:24:38.892205+0000","last_fullsized":"2023-10-16T11:24:38.892205+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.d","version":"216'1","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.083113+0000","last_change":"2023-10-16T11:24:25.083155+0000","last_active":"2023-10-16T11:24:27.083113+0000","last_peered":"2023-10-16T11:24:27.083113+0000","last_clean":"2023-10-16T11:24:27.083113+0000","last_became_active":"2023-10-16T11:24:25.082690+0000","last_became_peered":"2023-10-16T11:24:25.082690+0000","last_unstale":"2023-10-16T11:24:27.083113+0000","last_undegraded":"2023-10-16T11:24:27.083113+0000","last_fullsized":"2023-10-16T11:24:27.083113+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:46:58.816018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.257935+0000","last_change":"2023-10-16T11:24:26.069780+0000","last_active":"2023-10-16T11:24:29.257935+0000","last_peered":"2023-10-16T11:24:29.257935+0000","last_clean":"2023-10-16T11:24:29.257935+0000","last_became_active":"2023-10-16T11:24:26.068772+0000","last_became_peered":"2023-10-16T11:24:26.068772+0000","last_unstale":"2023-10-16T11:24:29.257935+0000","last_undegraded":"2023-10-16T11:24:29.257935+0000","last_fullsized":"2023-10-16T11:24:29.257935+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.901546+0000","last_change":"2023-10-16T11:24:38.901546+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.901546+0000","last_undegraded":"2023-10-16T11:24:38.901546+0000","last_fullsized":"2023-10-16T11:24:38.901546+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.e","version":"216'2","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.201299+0000","last_change":"2023-10-16T11:24:25.087765+0000","last_active":"2023-10-16T11:24:27.201299+0000","last_peered":"2023-10-16T11:24:27.201299+0000","last_clean":"2023-10-16T11:24:27.201299+0000","last_became_active":"2023-10-16T11:24:25.086967+0000","last_became_peered":"2023-10-16T11:24:25.086967+0000","last_unstale":"2023-10-16T11:24:27.201299+0000","last_undegraded":"2023-10-16T11:24:27.201299+0000","last_fullsized":"2023-10-16T11:24:27.201299+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255655+0000","last_change":"2023-10-16T11:24:26.086833+0000","last_active":"2023-10-16T11:24:29.255655+0000","last_peered":"2023-10-16T11:24:29.255655+0000","last_clean":"2023-10-16T11:24:29.255655+0000","last_became_active":"2023-10-16T11:24:26.085367+0000","last_became_peered":"2023-10-16T11:24:26.085367+0000","last_unstale":"2023-10-16T11:24:29.255655+0000","last_undegraded":"2023-10-16T11:24:29.255655+0000","last_fullsized":"2023-10-16T11:24:29.255655+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.902329+0000","last_change":"2023-10-16T11:24:38.902329+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.902329+0000","last_undegraded":"2023-10-16T11:24:38.902329+0000","last_fullsized":"2023-10-16T11:24:38.902329+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"216'1","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.071099+0000","last_change":"2023-10-16T11:24:25.079789+0000","last_active":"2023-10-16T11:24:28.071099+0000","last_peered":"2023-10-16T11:24:28.071099+0000","last_clean":"2023-10-16T11:24:28.071099+0000","last_became_active":"2023-10-16T11:24:25.079495+0000","last_became_peered":"2023-10-16T11:24:25.079495+0000","last_unstale":"2023-10-16T11:24:28.071099+0000","last_undegraded":"2023-10-16T11:24:28.071099+0000","last_fullsized":"2023-10-16T11:24:28.071099+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:44.531367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255585+0000","last_change":"2023-10-16T11:24:26.081796+0000","last_active":"2023-10-16T11:24:29.255585+0000","last_peered":"2023-10-16T11:24:29.255585+0000","last_clean":"2023-10-16T11:24:29.255585+0000","last_became_active":"2023-10-16T11:24:26.080287+0000","last_became_peered":"2023-10-16T11:24:26.080287+0000","last_unstale":"2023-10-16T11:24:29.255585+0000","last_undegraded":"2023-10-16T11:24:29.255585+0000","last_fullsized":"2023-10-16T11:24:29.255585+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.891885+0000","last_change":"2023-10-16T11:24:38.891885+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.891885+0000","last_undegraded":"2023-10-16T11:24:38.891885+0000","last_fullsized":"2023-10-16T11:24:38.891885+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.10","version":"216'8","reported_seq":21,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.080292+0000","last_change":"2023-10-16T11:24:25.076954+0000","last_active":"2023-10-16T11:24:27.080292+0000","last_peered":"2023-10-16T11:24:27.080292+0000","last_clean":"2023-10-16T11:24:27.080292+0000","last_became_active":"2023-10-16T11:24:25.076509+0000","last_became_peered":"2023-10-16T11:24:25.076509+0000","last_unstale":"2023-10-16T11:24:27.080292+0000","last_undegraded":"2023-10-16T11:24:27.080292+0000","last_fullsized":"2023-10-16T11:24:27.080292+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:46.770213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917418+0000","last_change":"2023-10-16T11:24:26.084139+0000","last_active":"2023-10-16T11:24:40.917418+0000","last_peered":"2023-10-16T11:24:40.917418+0000","last_clean":"2023-10-16T11:24:40.917418+0000","last_became_active":"2023-10-16T11:24:26.083876+0000","last_became_peered":"2023-10-16T11:24:26.083876+0000","last_unstale":"2023-10-16T11:24:40.917418+0000","last_undegraded":"2023-10-16T11:24:40.917418+0000","last_fullsized":"2023-10-16T11:24:40.917418+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918095+0000","last_change":"2023-10-16T11:24:39.904420+0000","last_active":"2023-10-16T11:24:40.918095+0000","last_peered":"2023-10-16T11:24:40.918095+0000","last_clean":"2023-10-16T11:24:40.918095+0000","last_became_active":"2023-10-16T11:24:39.904039+0000","last_became_peered":"2023-10-16T11:24:39.904039+0000","last_unstale":"2023-10-16T11:24:40.918095+0000","last_undegraded":"2023-10-16T11:24:40.918095+0000","last_fullsized":"2023-10-16T11:24:40.918095+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:17.673425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.081912+0000","last_change":"2023-10-16T11:24:25.089470+0000","last_active":"2023-10-16T11:24:27.081912+0000","last_peered":"2023-10-16T11:24:27.081912+0000","last_clean":"2023-10-16T11:24:27.081912+0000","last_became_active":"2023-10-16T11:24:25.089213+0000","last_became_peered":"2023-10-16T11:24:25.089213+0000","last_unstale":"2023-10-16T11:24:27.081912+0000","last_undegraded":"2023-10-16T11:24:27.081912+0000","last_fullsized":"2023-10-16T11:24:27.081912+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:34.540849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.257718+0000","last_change":"2023-10-16T11:24:26.079602+0000","last_active":"2023-10-16T11:24:29.257718+0000","last_peered":"2023-10-16T11:24:29.257718+0000","last_clean":"2023-10-16T11:24:29.257718+0000","last_became_active":"2023-10-16T11:24:26.079344+0000","last_became_peered":"2023-10-16T11:24:26.079344+0000","last_unstale":"2023-10-16T11:24:29.257718+0000","last_undegraded":"2023-10-16T11:24:29.257718+0000","last_fullsized":"2023-10-16T11:24:29.257718+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.884929+0000","last_change":"2023-10-16T11:24:38.884929+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.884929+0000","last_undegraded":"2023-10-16T11:24:38.884929+0000","last_fullsized":"2023-10-16T11:24:38.884929+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.083648+0000","last_change":"2023-10-16T11:24:25.084194+0000","last_active":"2023-10-16T11:24:27.083648+0000","last_peered":"2023-10-16T11:24:27.083648+0000","last_clean":"2023-10-16T11:24:27.083648+0000","last_became_active":"2023-10-16T11:24:25.083141+0000","last_became_peered":"2023-10-16T11:24:25.083141+0000","last_unstale":"2023-10-16T11:24:27.083648+0000","last_undegraded":"2023-10-16T11:24:27.083648+0000","last_fullsized":"2023-10-16T11:24:27.083648+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:46.770213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.262799+0000","last_change":"2023-10-16T11:24:26.089221+0000","last_active":"2023-10-16T11:24:29.262799+0000","last_peered":"2023-10-16T11:24:29.262799+0000","last_clean":"2023-10-16T11:24:29.262799+0000","last_became_active":"2023-10-16T11:24:26.088644+0000","last_became_peered":"2023-10-16T11:24:26.088644+0000","last_unstale":"2023-10-16T11:24:29.262799+0000","last_undegraded":"2023-10-16T11:24:29.262799+0000","last_fullsized":"2023-10-16T11:24:29.262799+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:10.387018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.895034+0000","last_change":"2023-10-16T11:24:38.895034+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.895034+0000","last_undegraded":"2023-10-16T11:24:38.895034+0000","last_fullsized":"2023-10-16T11:24:38.895034+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.13","version":"216'2","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.083651+0000","last_change":"2023-10-16T11:24:25.081377+0000","last_active":"2023-10-16T11:24:27.083651+0000","last_peered":"2023-10-16T11:24:27.083651+0000","last_clean":"2023-10-16T11:24:27.083651+0000","last_became_active":"2023-10-16T11:24:25.080881+0000","last_became_peered":"2023-10-16T11:24:25.080881+0000","last_unstale":"2023-10-16T11:24:27.083651+0000","last_undegraded":"2023-10-16T11:24:27.083651+0000","last_fullsized":"2023-10-16T11:24:27.083651+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:26.054702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.264563+0000","last_change":"2023-10-16T11:24:26.099039+0000","last_active":"2023-10-16T11:24:29.264563+0000","last_peered":"2023-10-16T11:24:29.264563+0000","last_clean":"2023-10-16T11:24:29.264563+0000","last_became_active":"2023-10-16T11:24:26.097980+0000","last_became_peered":"2023-10-16T11:24:26.097980+0000","last_unstale":"2023-10-16T11:24:29.264563+0000","last_undegraded":"2023-10-16T11:24:29.264563+0000","last_fullsized":"2023-10-16T11:24:29.264563+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:40.585267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.889517+0000","last_change":"2023-10-16T11:24:38.889517+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.889517+0000","last_undegraded":"2023-10-16T11:24:38.889517+0000","last_fullsized":"2023-10-16T11:24:38.889517+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.14","version":"217'8","reported_seq":21,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:35.685595+0000","last_change":"2023-10-16T11:24:25.095357+0000","last_active":"2023-10-16T11:24:35.685595+0000","last_peered":"2023-10-16T11:24:35.685595+0000","last_clean":"2023-10-16T11:24:35.685595+0000","last_became_active":"2023-10-16T11:24:25.093832+0000","last_became_peered":"2023-10-16T11:24:25.093832+0000","last_unstale":"2023-10-16T11:24:35.685595+0000","last_undegraded":"2023-10-16T11:24:35.685595+0000","last_fullsized":"2023-10-16T11:24:35.685595+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:14:19.482469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918256+0000","last_change":"2023-10-16T11:24:26.092185+0000","last_active":"2023-10-16T11:24:40.918256+0000","last_peered":"2023-10-16T11:24:40.918256+0000","last_clean":"2023-10-16T11:24:40.918256+0000","last_became_active":"2023-10-16T11:24:26.091648+0000","last_became_peered":"2023-10-16T11:24:26.091648+0000","last_unstale":"2023-10-16T11:24:40.918256+0000","last_undegraded":"2023-10-16T11:24:40.918256+0000","last_fullsized":"2023-10-16T11:24:40.918256+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918129+0000","last_change":"2023-10-16T11:24:39.919312+0000","last_active":"2023-10-16T11:24:40.918129+0000","last_peered":"2023-10-16T11:24:40.918129+0000","last_clean":"2023-10-16T11:24:40.918129+0000","last_became_active":"2023-10-16T11:24:39.918986+0000","last_became_peered":"2023-10-16T11:24:39.918986+0000","last_unstale":"2023-10-16T11:24:40.918129+0000","last_undegraded":"2023-10-16T11:24:40.918129+0000","last_fullsized":"2023-10-16T11:24:40.918129+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:54.385890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072402+0000","last_change":"2023-10-16T11:24:40.925767+0000","last_active":"2023-10-16T11:24:42.072402+0000","last_peered":"2023-10-16T11:24:42.072402+0000","last_clean":"2023-10-16T11:24:42.072402+0000","last_became_active":"2023-10-16T11:24:40.925501+0000","last_became_peered":"2023-10-16T11:24:40.925501+0000","last_unstale":"2023-10-16T11:24:42.072402+0000","last_undegraded":"2023-10-16T11:24:42.072402+0000","last_fullsized":"2023-10-16T11:24:42.072402+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:49:08.490072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.15","version":"216'3","reported_seq":16,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.083002+0000","last_change":"2023-10-16T11:24:25.082968+0000","last_active":"2023-10-16T11:24:27.083002+0000","last_peered":"2023-10-16T11:24:27.083002+0000","last_clean":"2023-10-16T11:24:27.083002+0000","last_became_active":"2023-10-16T11:24:25.082316+0000","last_became_peered":"2023-10-16T11:24:25.082316+0000","last_unstale":"2023-10-16T11:24:27.083002+0000","last_undegraded":"2023-10-16T11:24:27.083002+0000","last_fullsized":"2023-10-16T11:24:27.083002+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:59.924424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.259843+0000","last_change":"2023-10-16T11:24:26.084487+0000","last_active":"2023-10-16T11:24:29.259843+0000","last_peered":"2023-10-16T11:24:29.259843+0000","last_clean":"2023-10-16T11:24:29.259843+0000","last_became_active":"2023-10-16T11:24:26.084188+0000","last_became_peered":"2023-10-16T11:24:26.084188+0000","last_unstale":"2023-10-16T11:24:29.259843+0000","last_undegraded":"2023-10-16T11:24:29.259843+0000","last_fullsized":"2023-10-16T11:24:29.259843+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:47.400027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.889936+0000","last_change":"2023-10-16T11:24:38.889936+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.889936+0000","last_undegraded":"2023-10-16T11:24:38.889936+0000","last_fullsized":"2023-10-16T11:24:38.889936+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.084335+0000","last_change":"2023-10-16T11:24:25.092100+0000","last_active":"2023-10-16T11:24:27.084335+0000","last_peered":"2023-10-16T11:24:27.084335+0000","last_clean":"2023-10-16T11:24:27.084335+0000","last_became_active":"2023-10-16T11:24:25.091237+0000","last_became_peered":"2023-10-16T11:24:25.091237+0000","last_unstale":"2023-10-16T11:24:27.084335+0000","last_undegraded":"2023-10-16T11:24:27.084335+0000","last_fullsized":"2023-10-16T11:24:27.084335+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:21.985839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.258100+0000","last_change":"2023-10-16T11:24:26.081980+0000","last_active":"2023-10-16T11:24:29.258100+0000","last_peered":"2023-10-16T11:24:29.258100+0000","last_clean":"2023-10-16T11:24:29.258100+0000","last_became_active":"2023-10-16T11:24:26.081395+0000","last_became_peered":"2023-10-16T11:24:26.081395+0000","last_unstale":"2023-10-16T11:24:29.258100+0000","last_undegraded":"2023-10-16T11:24:29.258100+0000","last_fullsized":"2023-10-16T11:24:29.258100+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:39.802117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.893232+0000","last_change":"2023-10-16T11:24:38.893232+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.893232+0000","last_undegraded":"2023-10-16T11:24:38.893232+0000","last_fullsized":"2023-10-16T11:24:38.893232+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070986+0000","last_change":"2023-10-16T11:24:25.094384+0000","last_active":"2023-10-16T11:24:28.070986+0000","last_peered":"2023-10-16T11:24:28.070986+0000","last_clean":"2023-10-16T11:24:28.070986+0000","last_became_active":"2023-10-16T11:24:25.094026+0000","last_became_peered":"2023-10-16T11:24:25.094026+0000","last_unstale":"2023-10-16T11:24:28.070986+0000","last_undegraded":"2023-10-16T11:24:28.070986+0000","last_fullsized":"2023-10-16T11:24:28.070986+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:51.349970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.259941+0000","last_change":"2023-10-16T11:24:26.082915+0000","last_active":"2023-10-16T11:24:29.259941+0000","last_peered":"2023-10-16T11:24:29.259941+0000","last_clean":"2023-10-16T11:24:29.259941+0000","last_became_active":"2023-10-16T11:24:26.082458+0000","last_became_peered":"2023-10-16T11:24:26.082458+0000","last_unstale":"2023-10-16T11:24:29.259941+0000","last_undegraded":"2023-10-16T11:24:29.259941+0000","last_fullsized":"2023-10-16T11:24:29.259941+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:10.387018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.890075+0000","last_change":"2023-10-16T11:24:38.890075+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.890075+0000","last_undegraded":"2023-10-16T11:24:38.890075+0000","last_fullsized":"2023-10-16T11:24:38.890075+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.894457+0000","last_change":"2023-10-16T11:24:38.894457+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.894457+0000","last_undegraded":"2023-10-16T11:24:38.894457+0000","last_fullsized":"2023-10-16T11:24:38.894457+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.083169+0000","last_change":"2023-10-16T11:24:25.084521+0000","last_active":"2023-10-16T11:24:27.083169+0000","last_peered":"2023-10-16T11:24:27.083169+0000","last_clean":"2023-10-16T11:24:27.083169+0000","last_became_active":"2023-10-16T11:24:25.084003+0000","last_became_peered":"2023-10-16T11:24:25.084003+0000","last_unstale":"2023-10-16T11:24:27.083169+0000","last_undegraded":"2023-10-16T11:24:27.083169+0000","last_fullsized":"2023-10-16T11:24:27.083169+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:36.060066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.259652+0000","last_change":"2023-10-16T11:24:26.091641+0000","last_active":"2023-10-16T11:24:29.259652+0000","last_peered":"2023-10-16T11:24:29.259652+0000","last_clean":"2023-10-16T11:24:29.259652+0000","last_became_active":"2023-10-16T11:24:26.089186+0000","last_became_peered":"2023-10-16T11:24:26.089186+0000","last_unstale":"2023-10-16T11:24:29.259652+0000","last_undegraded":"2023-10-16T11:24:29.259652+0000","last_fullsized":"2023-10-16T11:24:29.259652+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:32.247878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.1f","version":"220'4","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.947367+0000","last_change":"2023-10-16T11:24:39.915794+0000","last_active":"2023-10-16T11:24:40.947367+0000","last_peered":"2023-10-16T11:24:40.947367+0000","last_clean":"2023-10-16T11:24:40.947367+0000","last_became_active":"2023-10-16T11:24:39.915438+0000","last_became_peered":"2023-10-16T11:24:39.915438+0000","last_unstale":"2023-10-16T11:24:40.947367+0000","last_undegraded":"2023-10-16T11:24:40.947367+0000","last_fullsized":"2023-10-16T11:24:40.947367+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:59.121334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.068765+0000","last_change":"2023-10-16T11:24:25.074647+0000","last_active":"2023-10-16T11:24:28.068765+0000","last_peered":"2023-10-16T11:24:28.068765+0000","last_clean":"2023-10-16T11:24:28.068765+0000","last_became_active":"2023-10-16T11:24:25.074373+0000","last_became_peered":"2023-10-16T11:24:25.074373+0000","last_unstale":"2023-10-16T11:24:28.068765+0000","last_undegraded":"2023-10-16T11:24:28.068765+0000","last_fullsized":"2023-10-16T11:24:28.068765+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:38.544520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.264495+0000","last_change":"2023-10-16T11:24:26.088785+0000","last_active":"2023-10-16T11:24:29.264495+0000","last_peered":"2023-10-16T11:24:29.264495+0000","last_clean":"2023-10-16T11:24:29.264495+0000","last_became_active":"2023-10-16T11:24:26.088521+0000","last_became_peered":"2023-10-16T11:24:26.088521+0000","last_unstale":"2023-10-16T11:24:29.264495+0000","last_undegraded":"2023-10-16T11:24:29.264495+0000","last_fullsized":"2023-10-16T11:24:29.264495+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:29.145795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1a","version":"216'1","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.071147+0000","last_change":"2023-10-16T11:24:25.084983+0000","last_active":"2023-10-16T11:24:28.071147+0000","last_peered":"2023-10-16T11:24:28.071147+0000","last_clean":"2023-10-16T11:24:28.071147+0000","last_became_active":"2023-10-16T11:24:25.084622+0000","last_became_peered":"2023-10-16T11:24:25.084622+0000","last_unstale":"2023-10-16T11:24:28.071147+0000","last_undegraded":"2023-10-16T11:24:28.071147+0000","last_fullsized":"2023-10-16T11:24:28.071147+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:36.588809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.259739+0000","last_change":"2023-10-16T11:24:26.089982+0000","last_active":"2023-10-16T11:24:29.259739+0000","last_peered":"2023-10-16T11:24:29.259739+0000","last_clean":"2023-10-16T11:24:29.259739+0000","last_became_active":"2023-10-16T11:24:26.089677+0000","last_became_peered":"2023-10-16T11:24:26.089677+0000","last_unstale":"2023-10-16T11:24:29.259739+0000","last_undegraded":"2023-10-16T11:24:29.259739+0000","last_fullsized":"2023-10-16T11:24:29.259739+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:22.009042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.893762+0000","last_change":"2023-10-16T11:24:38.893762+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.893762+0000","last_undegraded":"2023-10-16T11:24:38.893762+0000","last_fullsized":"2023-10-16T11:24:38.893762+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1c","version":"216'2","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069779+0000","last_change":"2023-10-16T11:24:25.095179+0000","last_active":"2023-10-16T11:24:28.069779+0000","last_peered":"2023-10-16T11:24:28.069779+0000","last_clean":"2023-10-16T11:24:28.069779+0000","last_became_active":"2023-10-16T11:24:25.094714+0000","last_became_peered":"2023-10-16T11:24:25.094714+0000","last_unstale":"2023-10-16T11:24:28.069779+0000","last_undegraded":"2023-10-16T11:24:28.069779+0000","last_fullsized":"2023-10-16T11:24:28.069779+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:26.128614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.262865+0000","last_change":"2023-10-16T11:24:26.096297+0000","last_active":"2023-10-16T11:24:29.262865+0000","last_peered":"2023-10-16T11:24:29.262865+0000","last_clean":"2023-10-16T11:24:29.262865+0000","last_became_active":"2023-10-16T11:24:26.095538+0000","last_became_peered":"2023-10-16T11:24:26.095538+0000","last_unstale":"2023-10-16T11:24:29.262865+0000","last_undegraded":"2023-10-16T11:24:29.262865+0000","last_fullsized":"2023-10-16T11:24:29.262865+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:47.400027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.901826+0000","last_change":"2023-10-16T11:24:38.901826+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.901826+0000","last_undegraded":"2023-10-16T11:24:38.901826+0000","last_fullsized":"2023-10-16T11:24:38.901826+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1d","version":"216'2","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.080963+0000","last_change":"2023-10-16T11:24:25.081731+0000","last_active":"2023-10-16T11:24:27.080963+0000","last_peered":"2023-10-16T11:24:27.080963+0000","last_clean":"2023-10-16T11:24:27.080963+0000","last_became_active":"2023-10-16T11:24:25.081336+0000","last_became_peered":"2023-10-16T11:24:25.081336+0000","last_unstale":"2023-10-16T11:24:27.080963+0000","last_undegraded":"2023-10-16T11:24:27.080963+0000","last_fullsized":"2023-10-16T11:24:27.080963+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:09.330801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918704+0000","last_change":"2023-10-16T11:24:26.089791+0000","last_active":"2023-10-16T11:24:40.918704+0000","last_peered":"2023-10-16T11:24:40.918704+0000","last_clean":"2023-10-16T11:24:40.918704+0000","last_became_active":"2023-10-16T11:24:26.089516+0000","last_became_peered":"2023-10-16T11:24:26.089516+0000","last_unstale":"2023-10-16T11:24:40.918704+0000","last_undegraded":"2023-10-16T11:24:40.918704+0000","last_fullsized":"2023-10-16T11:24:40.918704+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.890428+0000","last_change":"2023-10-16T11:24:38.890428+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.890428+0000","last_undegraded":"2023-10-16T11:24:38.890428+0000","last_fullsized":"2023-10-16T11:24:38.890428+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1e","version":"216'1","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.201161+0000","last_change":"2023-10-16T11:24:25.094483+0000","last_active":"2023-10-16T11:24:27.201161+0000","last_peered":"2023-10-16T11:24:27.201161+0000","last_clean":"2023-10-16T11:24:27.201161+0000","last_became_active":"2023-10-16T11:24:25.094134+0000","last_became_peered":"2023-10-16T11:24:25.094134+0000","last_unstale":"2023-10-16T11:24:27.201161+0000","last_undegraded":"2023-10-16T11:24:27.201161+0000","last_fullsized":"2023-10-16T11:24:27.201161+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.256450+0000","last_change":"2023-10-16T11:24:26.094536+0000","last_active":"2023-10-16T11:24:29.256450+0000","last_peered":"2023-10-16T11:24:29.256450+0000","last_clean":"2023-10-16T11:24:29.256450+0000","last_became_active":"2023-10-16T11:24:26.094272+0000","last_became_peered":"2023-10-16T11:24:26.094272+0000","last_unstale":"2023-10-16T11:24:29.256450+0000","last_undegraded":"2023-10-16T11:24:29.256450+0000","last_fullsized":"2023-10-16T11:24:29.256450+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.894744+0000","last_change":"2023-10-16T11:24:38.894744+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.894744+0000","last_undegraded":"2023-10-16T11:24:38.894744+0000","last_fullsized":"2023-10-16T11:24:38.894744+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1f","version":"216'4","reported_seq":17,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.976699+0000","last_change":"2023-10-16T11:24:25.096644+0000","last_active":"2023-10-16T11:24:27.976699+0000","last_peered":"2023-10-16T11:24:27.976699+0000","last_clean":"2023-10-16T11:24:27.976699+0000","last_became_active":"2023-10-16T11:24:25.096306+0000","last_became_peered":"2023-10-16T11:24:25.096306+0000","last_unstale":"2023-10-16T11:24:27.976699+0000","last_undegraded":"2023-10-16T11:24:27.976699+0000","last_fullsized":"2023-10-16T11:24:27.976699+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:22.618393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255563+0000","last_change":"2023-10-16T11:24:26.069227+0000","last_active":"2023-10-16T11:24:29.255563+0000","last_peered":"2023-10-16T11:24:29.255563+0000","last_clean":"2023-10-16T11:24:29.255563+0000","last_became_active":"2023-10-16T11:24:26.068456+0000","last_became_peered":"2023-10-16T11:24:26.068456+0000","last_unstale":"2023-10-16T11:24:29.255563+0000","last_undegraded":"2023-10-16T11:24:29.255563+0000","last_fullsized":"2023-10-16T11:24:29.255563+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.893285+0000","last_change":"2023-10-16T11:24:38.893285+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.893285+0000","last_undegraded":"2023-10-16T11:24:38.893285+0000","last_fullsized":"2023-10-16T11:24:38.893285+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.916906+0000","last_change":"2023-10-16T11:24:26.085375+0000","last_active":"2023-10-16T11:24:40.916906+0000","last_peered":"2023-10-16T11:24:40.916906+0000","last_clean":"2023-10-16T11:24:40.916906+0000","last_became_active":"2023-10-16T11:24:26.085120+0000","last_became_peered":"2023-10-16T11:24:26.085120+0000","last_unstale":"2023-10-16T11:24:40.916906+0000","last_undegraded":"2023-10-16T11:24:40.916906+0000","last_fullsized":"2023-10-16T11:24:40.916906+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.262693+0000","last_change":"2023-10-16T11:24:26.084964+0000","last_active":"2023-10-16T11:24:29.262693+0000","last_peered":"2023-10-16T11:24:29.262693+0000","last_clean":"2023-10-16T11:24:29.262693+0000","last_became_active":"2023-10-16T11:24:26.084006+0000","last_became_peered":"2023-10-16T11:24:26.084006+0000","last_unstale":"2023-10-16T11:24:29.262693+0000","last_undegraded":"2023-10-16T11:24:29.262693+0000","last_fullsized":"2023-10-16T11:24:29.262693+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:03.949526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255428+0000","last_change":"2023-10-16T11:24:26.086008+0000","last_active":"2023-10-16T11:24:29.255428+0000","last_peered":"2023-10-16T11:24:29.255428+0000","last_clean":"2023-10-16T11:24:29.255428+0000","last_became_active":"2023-10-16T11:24:26.085598+0000","last_became_peered":"2023-10-16T11:24:26.085598+0000","last_unstale":"2023-10-16T11:24:29.255428+0000","last_undegraded":"2023-10-16T11:24:29.255428+0000","last_fullsized":"2023-10-16T11:24:29.255428+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.254634+0000","last_change":"2023-10-16T11:24:26.086172+0000","last_active":"2023-10-16T11:24:29.254634+0000","last_peered":"2023-10-16T11:24:29.254634+0000","last_clean":"2023-10-16T11:24:29.254634+0000","last_became_active":"2023-10-16T11:24:26.085882+0000","last_became_peered":"2023-10-16T11:24:26.085882+0000","last_unstale":"2023-10-16T11:24:29.254634+0000","last_undegraded":"2023-10-16T11:24:29.254634+0000","last_fullsized":"2023-10-16T11:24:29.254634+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:44.265192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.257913+0000","last_change":"2023-10-16T11:24:26.086228+0000","last_active":"2023-10-16T11:24:29.257913+0000","last_peered":"2023-10-16T11:24:29.257913+0000","last_clean":"2023-10-16T11:24:29.257913+0000","last_became_active":"2023-10-16T11:24:26.085912+0000","last_became_peered":"2023-10-16T11:24:26.085912+0000","last_unstale":"2023-10-16T11:24:29.257913+0000","last_undegraded":"2023-10-16T11:24:29.257913+0000","last_fullsized":"2023-10-16T11:24:29.257913+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.254833+0000","last_change":"2023-10-16T11:24:26.088015+0000","last_active":"2023-10-16T11:24:29.254833+0000","last_peered":"2023-10-16T11:24:29.254833+0000","last_clean":"2023-10-16T11:24:29.254833+0000","last_became_active":"2023-10-16T11:24:26.086932+0000","last_became_peered":"2023-10-16T11:24:26.086932+0000","last_unstale":"2023-10-16T11:24:29.254833+0000","last_undegraded":"2023-10-16T11:24:29.254833+0000","last_fullsized":"2023-10-16T11:24:29.254833+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:44.137821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.071053+0000","last_change":"2023-10-16T11:24:40.935550+0000","last_active":"2023-10-16T11:24:42.071053+0000","last_peered":"2023-10-16T11:24:42.071053+0000","last_clean":"2023-10-16T11:24:42.071053+0000","last_became_active":"2023-10-16T11:24:40.935237+0000","last_became_peered":"2023-10-16T11:24:40.935237+0000","last_unstale":"2023-10-16T11:24:42.071053+0000","last_undegraded":"2023-10-16T11:24:42.071053+0000","last_fullsized":"2023-10-16T11:24:42.071053+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:13.011085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918841+0000","last_change":"2023-10-16T11:24:26.095913+0000","last_active":"2023-10-16T11:24:40.918841+0000","last_peered":"2023-10-16T11:24:40.918841+0000","last_clean":"2023-10-16T11:24:40.918841+0000","last_became_active":"2023-10-16T11:24:26.093113+0000","last_became_peered":"2023-10-16T11:24:26.093113+0000","last_unstale":"2023-10-16T11:24:40.918841+0000","last_undegraded":"2023-10-16T11:24:40.918841+0000","last_fullsized":"2023-10-16T11:24:40.918841+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255633+0000","last_change":"2023-10-16T11:24:26.094066+0000","last_active":"2023-10-16T11:24:29.255633+0000","last_peered":"2023-10-16T11:24:29.255633+0000","last_clean":"2023-10-16T11:24:29.255633+0000","last_became_active":"2023-10-16T11:24:26.093428+0000","last_became_peered":"2023-10-16T11:24:26.093428+0000","last_unstale":"2023-10-16T11:24:29.255633+0000","last_undegraded":"2023-10-16T11:24:29.255633+0000","last_fullsized":"2023-10-16T11:24:29.255633+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.259561+0000","last_change":"2023-10-16T11:24:26.091894+0000","last_active":"2023-10-16T11:24:29.259561+0000","last_peered":"2023-10-16T11:24:29.259561+0000","last_clean":"2023-10-16T11:24:29.259561+0000","last_became_active":"2023-10-16T11:24:26.091627+0000","last_became_peered":"2023-10-16T11:24:26.091627+0000","last_unstale":"2023-10-16T11:24:29.259561+0000","last_undegraded":"2023-10-16T11:24:29.259561+0000","last_fullsized":"2023-10-16T11:24:29.259561+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:46:59.832609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.256300+0000","last_change":"2023-10-16T11:24:26.097221+0000","last_active":"2023-10-16T11:24:29.256300+0000","last_peered":"2023-10-16T11:24:29.256300+0000","last_clean":"2023-10-16T11:24:29.256300+0000","last_became_active":"2023-10-16T11:24:26.096572+0000","last_became_peered":"2023-10-16T11:24:26.096572+0000","last_unstale":"2023-10-16T11:24:29.256300+0000","last_undegraded":"2023-10-16T11:24:29.256300+0000","last_fullsized":"2023-10-16T11:24:29.256300+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.256364+0000","last_change":"2023-10-16T11:24:26.101039+0000","last_active":"2023-10-16T11:24:29.256364+0000","last_peered":"2023-10-16T11:24:29.256364+0000","last_clean":"2023-10-16T11:24:29.256364+0000","last_became_active":"2023-10-16T11:24:26.100757+0000","last_became_peered":"2023-10-16T11:24:26.100757+0000","last_unstale":"2023-10-16T11:24:29.256364+0000","last_undegraded":"2023-10-16T11:24:29.256364+0000","last_fullsized":"2023-10-16T11:24:29.256364+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.259697+0000","last_change":"2023-10-16T11:24:26.092718+0000","last_active":"2023-10-16T11:24:29.259697+0000","last_peered":"2023-10-16T11:24:29.259697+0000","last_clean":"2023-10-16T11:24:29.259697+0000","last_became_active":"2023-10-16T11:24:26.091377+0000","last_became_peered":"2023-10-16T11:24:26.091377+0000","last_unstale":"2023-10-16T11:24:29.259697+0000","last_undegraded":"2023-10-16T11:24:29.259697+0000","last_fullsized":"2023-10-16T11:24:29.259697+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:17.072626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072172+0000","last_change":"2023-10-16T11:24:40.933022+0000","last_active":"2023-10-16T11:24:42.072172+0000","last_peered":"2023-10-16T11:24:42.072172+0000","last_clean":"2023-10-16T11:24:42.072172+0000","last_became_active":"2023-10-16T11:24:40.932746+0000","last_became_peered":"2023-10-16T11:24:40.932746+0000","last_unstale":"2023-10-16T11:24:42.072172+0000","last_undegraded":"2023-10-16T11:24:42.072172+0000","last_fullsized":"2023-10-16T11:24:42.072172+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:42.730933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.264637+0000","last_change":"2023-10-16T11:24:26.093168+0000","last_active":"2023-10-16T11:24:29.264637+0000","last_peered":"2023-10-16T11:24:29.264637+0000","last_clean":"2023-10-16T11:24:29.264637+0000","last_became_active":"2023-10-16T11:24:26.091801+0000","last_became_peered":"2023-10-16T11:24:26.091801+0000","last_unstale":"2023-10-16T11:24:29.264637+0000","last_undegraded":"2023-10-16T11:24:29.264637+0000","last_fullsized":"2023-10-16T11:24:29.264637+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:43:28.275746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.262948+0000","last_change":"2023-10-16T11:24:26.086807+0000","last_active":"2023-10-16T11:24:29.262948+0000","last_peered":"2023-10-16T11:24:29.262948+0000","last_clean":"2023-10-16T11:24:29.262948+0000","last_became_active":"2023-10-16T11:24:26.085382+0000","last_became_peered":"2023-10-16T11:24:26.085382+0000","last_unstale":"2023-10-16T11:24:29.262948+0000","last_undegraded":"2023-10-16T11:24:29.262948+0000","last_fullsized":"2023-10-16T11:24:29.262948+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:56.195463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.263076+0000","last_change":"2023-10-16T11:24:26.092139+0000","last_active":"2023-10-16T11:24:29.263076+0000","last_peered":"2023-10-16T11:24:29.263076+0000","last_clean":"2023-10-16T11:24:29.263076+0000","last_became_active":"2023-10-16T11:24:26.091860+0000","last_became_peered":"2023-10-16T11:24:26.091860+0000","last_unstale":"2023-10-16T11:24:29.263076+0000","last_undegraded":"2023-10-16T11:24:29.263076+0000","last_fullsized":"2023-10-16T11:24:29.263076+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:00:31.371733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.919569+0000","last_change":"2023-10-16T11:24:26.086029+0000","last_active":"2023-10-16T11:24:40.919569+0000","last_peered":"2023-10-16T11:24:40.919569+0000","last_clean":"2023-10-16T11:24:40.919569+0000","last_became_active":"2023-10-16T11:24:26.085644+0000","last_became_peered":"2023-10-16T11:24:26.085644+0000","last_unstale":"2023-10-16T11:24:40.919569+0000","last_undegraded":"2023-10-16T11:24:40.919569+0000","last_fullsized":"2023-10-16T11:24:40.919569+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:39.802117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.262637+0000","last_change":"2023-10-16T11:24:26.081938+0000","last_active":"2023-10-16T11:24:29.262637+0000","last_peered":"2023-10-16T11:24:29.262637+0000","last_clean":"2023-10-16T11:24:29.262637+0000","last_became_active":"2023-10-16T11:24:26.081671+0000","last_became_peered":"2023-10-16T11:24:26.081671+0000","last_unstale":"2023-10-16T11:24:29.262637+0000","last_undegraded":"2023-10-16T11:24:29.262637+0000","last_fullsized":"2023-10-16T11:24:29.262637+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:17.072626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917335+0000","last_change":"2023-10-16T11:24:26.091403+0000","last_active":"2023-10-16T11:24:40.917335+0000","last_peered":"2023-10-16T11:24:40.917335+0000","last_clean":"2023-10-16T11:24:40.917335+0000","last_became_active":"2023-10-16T11:24:26.091114+0000","last_became_peered":"2023-10-16T11:24:26.091114+0000","last_unstale":"2023-10-16T11:24:40.917335+0000","last_undegraded":"2023-10-16T11:24:40.917335+0000","last_fullsized":"2023-10-16T11:24:40.917335+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917983+0000","last_change":"2023-10-16T11:24:26.088199+0000","last_active":"2023-10-16T11:24:40.917983+0000","last_peered":"2023-10-16T11:24:40.917983+0000","last_clean":"2023-10-16T11:24:40.917983+0000","last_became_active":"2023-10-16T11:24:26.087892+0000","last_became_peered":"2023-10-16T11:24:26.087892+0000","last_unstale":"2023-10-16T11:24:40.917983+0000","last_undegraded":"2023-10-16T11:24:40.917983+0000","last_fullsized":"2023-10-16T11:24:40.917983+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.254728+0000","last_change":"2023-10-16T11:24:26.088938+0000","last_active":"2023-10-16T11:24:29.254728+0000","last_peered":"2023-10-16T11:24:29.254728+0000","last_clean":"2023-10-16T11:24:29.254728+0000","last_became_active":"2023-10-16T11:24:26.088689+0000","last_became_peered":"2023-10-16T11:24:26.088689+0000","last_unstale":"2023-10-16T11:24:29.254728+0000","last_undegraded":"2023-10-16T11:24:29.254728+0000","last_fullsized":"2023-10-16T11:24:29.254728+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:18:12.479766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.254769+0000","last_change":"2023-10-16T11:24:26.089768+0000","last_active":"2023-10-16T11:24:29.254769+0000","last_peered":"2023-10-16T11:24:29.254769+0000","last_clean":"2023-10-16T11:24:29.254769+0000","last_became_active":"2023-10-16T11:24:26.087028+0000","last_became_peered":"2023-10-16T11:24:26.087028+0000","last_unstale":"2023-10-16T11:24:29.254769+0000","last_undegraded":"2023-10-16T11:24:29.254769+0000","last_fullsized":"2023-10-16T11:24:29.254769+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:33.050118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.258028+0000","last_change":"2023-10-16T11:24:26.077430+0000","last_active":"2023-10-16T11:24:29.258028+0000","last_peered":"2023-10-16T11:24:29.258028+0000","last_clean":"2023-10-16T11:24:29.258028+0000","last_became_active":"2023-10-16T11:24:26.077145+0000","last_became_peered":"2023-10-16T11:24:26.077145+0000","last_unstale":"2023-10-16T11:24:29.258028+0000","last_undegraded":"2023-10-16T11:24:29.258028+0000","last_fullsized":"2023-10-16T11:24:29.258028+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.263034+0000","last_change":"2023-10-16T11:24:26.088202+0000","last_active":"2023-10-16T11:24:29.263034+0000","last_peered":"2023-10-16T11:24:29.263034+0000","last_clean":"2023-10-16T11:24:29.263034+0000","last_became_active":"2023-10-16T11:24:26.087929+0000","last_became_peered":"2023-10-16T11:24:26.087929+0000","last_unstale":"2023-10-16T11:24:29.263034+0000","last_undegraded":"2023-10-16T11:24:29.263034+0000","last_fullsized":"2023-10-16T11:24:29.263034+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:22.009042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.070991+0000","last_change":"2023-10-16T11:24:40.941405+0000","last_active":"2023-10-16T11:24:42.070991+0000","last_peered":"2023-10-16T11:24:42.070991+0000","last_clean":"2023-10-16T11:24:42.070991+0000","last_became_active":"2023-10-16T11:24:40.941060+0000","last_became_peered":"2023-10-16T11:24:40.941060+0000","last_unstale":"2023-10-16T11:24:42.070991+0000","last_undegraded":"2023-10-16T11:24:42.070991+0000","last_fullsized":"2023-10-16T11:24:42.070991+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:22:53.973191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255677+0000","last_change":"2023-10-16T11:24:26.080292+0000","last_active":"2023-10-16T11:24:29.255677+0000","last_peered":"2023-10-16T11:24:29.255677+0000","last_clean":"2023-10-16T11:24:29.255677+0000","last_became_active":"2023-10-16T11:24:26.080004+0000","last_became_peered":"2023-10-16T11:24:26.080004+0000","last_unstale":"2023-10-16T11:24:29.255677+0000","last_undegraded":"2023-10-16T11:24:29.255677+0000","last_fullsized":"2023-10-16T11:24:29.255677+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.264465+0000","last_change":"2023-10-16T11:24:26.098845+0000","last_active":"2023-10-16T11:24:29.264465+0000","last_peered":"2023-10-16T11:24:29.264465+0000","last_clean":"2023-10-16T11:24:29.264465+0000","last_became_active":"2023-10-16T11:24:26.098568+0000","last_became_peered":"2023-10-16T11:24:26.098568+0000","last_unstale":"2023-10-16T11:24:29.264465+0000","last_undegraded":"2023-10-16T11:24:29.264465+0000","last_fullsized":"2023-10-16T11:24:29.264465+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:15.909098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917233+0000","last_change":"2023-10-16T11:24:26.096348+0000","last_active":"2023-10-16T11:24:40.917233+0000","last_peered":"2023-10-16T11:24:40.917233+0000","last_clean":"2023-10-16T11:24:40.917233+0000","last_became_active":"2023-10-16T11:24:26.093712+0000","last_became_peered":"2023-10-16T11:24:26.093712+0000","last_unstale":"2023-10-16T11:24:40.917233+0000","last_undegraded":"2023-10-16T11:24:40.917233+0000","last_fullsized":"2023-10-16T11:24:40.917233+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:40.970357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.264542+0000","last_change":"2023-10-16T11:24:26.083687+0000","last_active":"2023-10-16T11:24:29.264542+0000","last_peered":"2023-10-16T11:24:29.264542+0000","last_clean":"2023-10-16T11:24:29.264542+0000","last_became_active":"2023-10-16T11:24:26.083213+0000","last_became_peered":"2023-10-16T11:24:26.083213+0000","last_unstale":"2023-10-16T11:24:29.264542+0000","last_undegraded":"2023-10-16T11:24:29.264542+0000","last_fullsized":"2023-10-16T11:24:29.264542+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:08:02.966132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.264608+0000","last_change":"2023-10-16T11:24:26.096814+0000","last_active":"2023-10-16T11:24:29.264608+0000","last_peered":"2023-10-16T11:24:29.264608+0000","last_clean":"2023-10-16T11:24:29.264608+0000","last_became_active":"2023-10-16T11:24:26.095777+0000","last_became_peered":"2023-10-16T11:24:26.095777+0000","last_unstale":"2023-10-16T11:24:29.264608+0000","last_undegraded":"2023-10-16T11:24:29.264608+0000","last_fullsized":"2023-10-16T11:24:29.264608+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:08.889514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.259801+0000","last_change":"2023-10-16T11:24:26.091440+0000","last_active":"2023-10-16T11:24:29.259801+0000","last_peered":"2023-10-16T11:24:29.259801+0000","last_clean":"2023-10-16T11:24:29.259801+0000","last_became_active":"2023-10-16T11:24:26.088757+0000","last_became_peered":"2023-10-16T11:24:26.088757+0000","last_unstale":"2023-10-16T11:24:29.259801+0000","last_undegraded":"2023-10-16T11:24:29.259801+0000","last_fullsized":"2023-10-16T11:24:29.259801+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:27.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.257982+0000","last_change":"2023-10-16T11:24:26.085606+0000","last_active":"2023-10-16T11:24:29.257982+0000","last_peered":"2023-10-16T11:24:29.257982+0000","last_clean":"2023-10-16T11:24:29.257982+0000","last_became_active":"2023-10-16T11:24:26.085318+0000","last_became_peered":"2023-10-16T11:24:26.085318+0000","last_unstale":"2023-10-16T11:24:29.257982+0000","last_undegraded":"2023-10-16T11:24:29.257982+0000","last_fullsized":"2023-10-16T11:24:29.257982+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255483+0000","last_change":"2023-10-16T11:24:26.089885+0000","last_active":"2023-10-16T11:24:29.255483+0000","last_peered":"2023-10-16T11:24:29.255483+0000","last_clean":"2023-10-16T11:24:29.255483+0000","last_became_active":"2023-10-16T11:24:26.089406+0000","last_became_peered":"2023-10-16T11:24:26.089406+0000","last_unstale":"2023-10-16T11:24:29.255483+0000","last_undegraded":"2023-10-16T11:24:29.255483+0000","last_fullsized":"2023-10-16T11:24:29.255483+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.919488+0000","last_change":"2023-10-16T11:24:26.086488+0000","last_active":"2023-10-16T11:24:40.919488+0000","last_peered":"2023-10-16T11:24:40.919488+0000","last_clean":"2023-10-16T11:24:40.919488+0000","last_became_active":"2023-10-16T11:24:26.085969+0000","last_became_peered":"2023-10-16T11:24:26.085969+0000","last_unstale":"2023-10-16T11:24:40.919488+0000","last_undegraded":"2023-10-16T11:24:40.919488+0000","last_fullsized":"2023-10-16T11:24:40.919488+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":1},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":581,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2767,"ondisk_log_size":2767,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575988,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52476,"kb_used_data":3252,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319364,"statfs":{"total":96636764160,"available":96583028736,"internally_reserved":0,"allocated":3330048,"data_stored":1913672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.179,"15min":1.2170000000000001},"min":{"1min":0.81499999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.7250000000000001,"5min":2.085,"15min":2.3399999999999999},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.1719999999999999,"15min":1.218},"min":{"1min":0.876,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.8320000000000001},"last":1.097}]},{"osd":1,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2689999999999999,"15min":1.2509999999999999},"min":{"1min":0.90700000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.6479999999999999,"5min":1.913,"15min":1.913},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2430000000000001,"15min":1.2749999999999999},"min":{"1min":0.76700000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6279999999999999,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.67100000000000004}]},{"osd":2,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3140000000000001,"15min":1.3109999999999999},"min":{"1min":1.008,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6579999999999999,"5min":1.8680000000000001,"15min":2.4460000000000002},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.3540000000000001,"15min":1.3640000000000001},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.708,"5min":1.859,"15min":1.974},"last":1.056}]},{"osd":3,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.381,"15min":1.3899999999999999},"min":{"1min":1.149,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7290000000000001,"5min":1.8280000000000001,"15min":1.972},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.45,"5min":1.4430000000000001,"15min":1.4630000000000001},"min":{"1min":1.0860000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.8799999999999999,"5min":2.0680000000000001,"15min":2.577},"last":0.80300000000000005}]},{"osd":4,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.423,"15min":1.417},"min":{"1min":0.98599999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0030000000000001,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.498,"5min":1.391,"15min":1.3899999999999999},"min":{"1min":1.0469999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8280000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.3320000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.585,"15min":1.5960000000000001},"min":{"1min":1.1579999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.946,"5min":2.2309999999999999,"15min":3.3519999999999999},"last":1.484},{"interface":"front","average":{"1min":1.641,"5min":1.514,"15min":1.5089999999999999},"min":{"1min":1.351,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.1600000000000001,"5min":2.3460000000000001,"15min":2.4969999999999999},"last":1.762}]},{"osd":6,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.488,"15min":1.5209999999999999},"min":{"1min":0.97099999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.845,"5min":2.101,"15min":2.7250000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.7210000000000001,"5min":1.6890000000000001,"15min":1.71},"min":{"1min":1.425,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1040000000000001,"5min":2.298,"15min":3.1440000000000001},"last":1.544}]}]},{"osd":6,"up_from":13,"seq":55834575988,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33848,"kb_used_data":2664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337992,"statfs":{"total":96636764160,"available":96602103808,"internally_reserved":0,"allocated":2727936,"data_stored":1321029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.2350000000000001,"15min":1.2250000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.64000000000000001},"max":{"1min":6.298,"5min":6.298,"15min":6.298},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.3380000000000001,"15min":1.2909999999999999},"min":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.66400000000000003},"max":{"1min":8.2650000000000006,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":0.97099999999999997}]},{"osd":1,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2450000000000001,"15min":1.222},"min":{"1min":0.80200000000000005,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":1.038},{"interface":"front","average":{"1min":1.8520000000000001,"5min":1.4510000000000001,"15min":1.3859999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":12.468,"5min":12.468,"15min":12.468},"last":1.333}]},{"osd":2,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.7889999999999999,"5min":1.337,"15min":1.2829999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":11.586,"5min":11.586,"15min":11.586},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.569,"5min":1.3799999999999999,"15min":1.369},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.77500000000000002},"max":{"1min":4.7030000000000003,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.131}]},{"osd":3,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.9239999999999999,"5min":1.452,"15min":1.353},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":12.801,"5min":12.801,"15min":12.801},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.302,"15min":1.284},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":9.1539999999999999,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":0.85599999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.3700000000000001,"15min":1.3220000000000001},"min":{"1min":0.65400000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.2470000000000001},{"interface":"front","average":{"1min":2.141,"5min":1.609,"15min":1.548},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":13.981999999999999,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.5429999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":2.246,"5min":1.5109999999999999,"15min":1.391},"min":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":18.247,"5min":18.247,"15min":18.247},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.8360000000000001,"5min":1.5900000000000001,"15min":1.5629999999999999},"min":{"1min":0.76200000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":6.4039999999999999,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.4870000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":2.0609999999999999,"5min":1.603,"15min":1.5349999999999999},"min":{"1min":0.749,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":12.589,"5min":12.589,"15min":12.589},"last":1.6659999999999999},{"interface":"front","average":{"1min":2.1749999999999998,"5min":1.6739999999999999,"15min":1.595},"min":{"1min":0.78200000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":12.725,"5min":12.725,"15min":12.725},"last":1.4530000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575988,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29696,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342144,"statfs":{"total":96636764160,"available":96606355456,"internally_reserved":0,"allocated":2740224,"data_stored":1327732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2350000000000001,"15min":1.27},"min":{"1min":0.74199999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.583,"5min":2.661,"15min":2.661},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.3029999999999999,"15min":1.349},"min":{"1min":0.95799999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7909999999999999,"5min":1.7949999999999999,"15min":1.865},"last":1.341}]},{"osd":1,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1819999999999999,"15min":1.1950000000000001},"min":{"1min":0.81999999999999995,"5min":0.80800000000000005,"15min":0.79100000000000004},"max":{"1min":1.5009999999999999,"5min":1.7869999999999999,"15min":1.992},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.258,"15min":1.2290000000000001},"min":{"1min":0.84399999999999997,"5min":0.84199999999999997,"15min":0.80800000000000005},"max":{"1min":1.635,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":0.89100000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.264,"15min":1.244},"min":{"1min":0.97899999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.625,"5min":1.952,"15min":1.952},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.329,"5min":1.2949999999999999,"15min":1.3220000000000001},"min":{"1min":0.92400000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":2.1640000000000001},"last":1.3080000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3320000000000001,"15min":1.3340000000000001},"min":{"1min":0.98999999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":3.012,"15min":3.012},"last":1.242},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.429,"15min":1.425},"min":{"1min":1.0900000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":2.2250000000000001},"last":1.1799999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.339,"15min":1.339},"min":{"1min":0.94199999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.6990000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.6799999999999999,"5min":1.5720000000000001,"15min":1.5740000000000001},"min":{"1min":1.353,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.948,"5min":2.903,"15min":2.903},"last":1.456}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4330000000000001,"15min":1.4139999999999999},"min":{"1min":0.873,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8660000000000001,"5min":2.0649999999999999,"15min":2.0979999999999999},"last":1.407},{"interface":"front","average":{"1min":1.51,"5min":1.494,"15min":1.4770000000000001},"min":{"1min":0.95399999999999996,"5min":0.90700000000000003,"15min":0.78400000000000003},"max":{"1min":1.841,"5min":3.073,"15min":3.073},"last":1.216}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.675,"5min":1.5660000000000001,"15min":1.5469999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0779999999999998,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.645,"15min":1.645},"min":{"1min":1.1779999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.1099999999999999,"5min":2.1459999999999999,"15min":2.2930000000000001},"last":1.5649999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575989,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51904,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319936,"statfs":{"total":96636764160,"available":96583614464,"internally_reserved":0,"allocated":2744320,"data_stored":1323408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1519999999999999,"15min":1.1859999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.7290000000000001,"5min":1.792,"15min":1.792},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.218,"15min":1.2350000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.9299999999999999},"last":1.419}]},{"osd":1,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2769999999999999,"15min":1.298},"min":{"1min":0.81799999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.841,"5min":1.841,"15min":1.913},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.236,"15min":1.25},"min":{"1min":0.78500000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.877,"5min":1.877,"15min":1.889},"last":1.018}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.29,"15min":1.3089999999999999},"min":{"1min":1.0369999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.835},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.351,"5min":1.3999999999999999,"15min":1.403},"min":{"1min":0.879,"5min":0.879,"15min":0.872},"max":{"1min":1.8380000000000001,"5min":1.903,"15min":1.9139999999999999},"last":1.2529999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3169999999999999,"15min":1.337},"min":{"1min":0.91100000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.6830000000000001,"5min":1.8280000000000001,"15min":1.849},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4410000000000001,"15min":1.4330000000000001},"min":{"1min":1.145,"5min":1.008,"15min":0.96999999999999997},"max":{"1min":1.7769999999999999,"5min":1.865,"15min":2.0720000000000001},"last":1.1830000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.4810000000000001,"15min":1.4399999999999999},"min":{"1min":1.0149999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9399999999999999,"5min":2.105,"15min":2.105},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.431,"15min":1.415},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.78600000000000003},"max":{"1min":1.8240000000000001,"5min":2.04,"15min":2.04},"last":1.2290000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.5409999999999999,"15min":1.5029999999999999},"min":{"1min":0.94299999999999995,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":3.157,"5min":3.157,"15min":3.157},"last":1.492},{"interface":"front","average":{"1min":1.5,"5min":1.6100000000000001,"15min":1.611},"min":{"1min":1.0940000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0089999999999999,"5min":2.2330000000000001,"15min":2.2770000000000001},"last":1.6040000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.5640000000000001,"15min":1.5660000000000001},"min":{"1min":1.0549999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.855,"5min":2.113,"15min":2.3420000000000001},"last":1.3},{"interface":"front","average":{"1min":1.639,"5min":1.6619999999999999,"15min":1.6399999999999999},"min":{"1min":1.0660000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9630000000000001,"5min":2.1469999999999998,"15min":2.3839999999999999},"last":1.3340000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575988,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47748,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324092,"statfs":{"total":96636764160,"available":96587870208,"internally_reserved":0,"allocated":2736128,"data_stored":1320629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.89400000000000002,"15min":0.90700000000000003},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.478,"5min":1.478,"15min":1.901},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0369999999999999,"15min":1.0720000000000001},"min":{"1min":0.59299999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.391,"5min":1.6180000000000001,"15min":1.6679999999999999},"last":1.0660000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.95599999999999996,"15min":0.98799999999999999},"min":{"1min":0.60799999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.105,"15min":1.133},"min":{"1min":0.48399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.601,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.88100000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0760000000000001,"15min":1.117},"min":{"1min":0.65500000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.524,"5min":1.694,"15min":1.694},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.198,"15min":1.236},"min":{"1min":0.625,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.91400000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3169999999999999,"15min":1.3500000000000001},"min":{"1min":0.83599999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.806,"5min":1.806,"15min":1.873},"last":1.466},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.357,"15min":1.367},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.8819999999999999},"last":1.2689999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.385,"15min":1.401},"min":{"1min":0.80600000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.516},{"interface":"front","average":{"1min":1.419,"5min":1.4059999999999999,"15min":1.444},"min":{"1min":1.151,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.552}]},{"osd":6,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.504,"15min":1.5329999999999999},"min":{"1min":0.876,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":3.9129999999999998,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.657},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5,"15min":1.5369999999999999},"min":{"1min":1.153,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":3.9319999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.4910000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.528,"15min":1.5600000000000001},"min":{"1min":0.95399999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.877,"5min":2.0790000000000002,"15min":2.2679999999999998},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.639,"15min":1.673},"min":{"1min":1.1279999999999999,"5min":1,"15min":1},"max":{"1min":2.052,"5min":2.198,"15min":2.5129999999999999},"last":1.6739999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575987,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51896,"kb_used_data":2660,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319944,"statfs":{"total":96636764160,"available":96583622656,"internally_reserved":0,"allocated":2723840,"data_stored":1320447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.91000000000000003,"15min":0.92200000000000004},"min":{"1min":0.74199999999999999,"5min":0.51900000000000002,"15min":0.5},"max":{"1min":1.532,"5min":1.532,"15min":1.669},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.0569999999999999,"15min":1.0820000000000001},"min":{"1min":0.72499999999999998,"5min":0.47699999999999998,"15min":0.32600000000000001},"max":{"1min":1.496,"5min":1.681,"15min":2.1429999999999998},"last":0.80400000000000005}]},{"osd":1,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.089,"15min":1.097},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.39200000000000002},"max":{"1min":1.415,"5min":1.6779999999999999,"15min":1.978},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1479999999999999,"15min":1.165},"min":{"1min":0.57599999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.448,"5min":1.609,"15min":1.869},"last":1.0920000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.0649999999999999,"15min":1.087},"min":{"1min":0.54600000000000004,"5min":0.437,"15min":0.437},"max":{"1min":1.464,"5min":1.651,"15min":1.8560000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.252,"15min":1.274},"min":{"1min":0.82199999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6120000000000001,"5min":1.7450000000000001,"15min":1.857},"last":1.2509999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3740000000000001,"15min":1.415},"min":{"1min":0.91500000000000004,"5min":0.86599999999999999,"15min":0.73499999999999999},"max":{"1min":1.6910000000000001,"5min":1.8029999999999999,"15min":2.5059999999999998},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3440000000000001,"15min":1.3520000000000001},"min":{"1min":0.97899999999999998,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":1.825,"5min":1.825,"15min":2.0880000000000001},"last":1.407}]},{"osd":5,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.444,"15min":1.464},"min":{"1min":1.0720000000000001,"5min":0.84999999999999998,"15min":0.78500000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2269999999999999},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4550000000000001,"15min":1.4890000000000001},"min":{"1min":1.0509999999999999,"5min":0.95799999999999996,"15min":0.874},"max":{"1min":1.798,"5min":1.931,"15min":2.6259999999999999},"last":1.518}]},{"osd":6,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4119999999999999,"15min":1.411},"min":{"1min":1.0569999999999999,"5min":0.84099999999999997,"15min":0.80500000000000005},"max":{"1min":1.71,"5min":1.9019999999999999,"15min":2.2109999999999999},"last":1.8260000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5,"15min":1.5369999999999999},"min":{"1min":1.1899999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.2909999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.6899999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":1.5489999999999999,"15min":1.5860000000000001},"min":{"1min":1.2290000000000001,"5min":0.995,"15min":0.92500000000000004},"max":{"1min":2.1469999999999998,"5min":2.262,"15min":2.8109999999999999},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.494,"5min":1.4870000000000001,"15min":1.5149999999999999},"min":{"1min":1.242,"5min":0.81599999999999995,"15min":0.80100000000000005},"max":{"1min":1.8180000000000001,"5min":2.2269999999999999,"15min":2.2829999999999999},"last":1.907}]}]},{"osd":1,"up_from":13,"seq":55834575987,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60228,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311612,"statfs":{"total":96636764160,"available":96575090688,"internally_reserved":0,"allocated":2744320,"data_stored":1327206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.91900000000000004,"15min":0.91500000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.575,"5min":1.575,"15min":1.7929999999999999},"last":1.196},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0289999999999999,"15min":1.0580000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.494,"5min":1.5509999999999999,"15min":1.667},"last":1.0229999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.98399999999999999,"15min":1.0189999999999999},"min":{"1min":0.64300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.72},"last":2.093},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.252,"15min":1.276},"min":{"1min":0.69799999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":1.6639999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.1879999999999999,"15min":1.216},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.774,"5min":1.774,"15min":1.79},"last":1.601},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.169,"15min":1.1739999999999999},"min":{"1min":0.66700000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.633,"5min":1.633,"15min":1.802},"last":1.0880000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.2609999999999999,"15min":1.274},"min":{"1min":0.86299999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.653,"5min":1.887,"15min":1.9159999999999999},"last":3.7759999999999998},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.409,"15min":1.415},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.887,"5min":2.0619999999999998,"15min":2.093},"last":2.3519999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.379,"15min":1.409},"min":{"1min":0.93700000000000006,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.948,"5min":2.0009999999999999,"15min":2.0640000000000001},"last":1.756},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4239999999999999,"15min":1.421},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":1.825,"5min":1.825,"15min":2.0649999999999999},"last":2.5299999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4850000000000001,"15min":1.496},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.879,"5min":2.1970000000000001,"15min":2.2949999999999999},"last":2.1960000000000002},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4710000000000001,"15min":1.486},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.891,"5min":1.891,"15min":2.1970000000000001},"last":3.8799999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5649999999999999,"15min":1.5620000000000001},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.1720000000000002},"last":4.46},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.625,"15min":1.607},"min":{"1min":1.1080000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.9359999999999999,"5min":2.1989999999999998,"15min":2.2810000000000001},"last":2.5960000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575989,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52532,"kb_used_data":3300,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319308,"statfs":{"total":96636764160,"available":96582971392,"internally_reserved":0,"allocated":3379200,"data_stored":1940312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.871,"15min":0.90000000000000002},"min":{"1min":0.73199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.502,"5min":1.52,"15min":1.619},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.1060000000000001},"min":{"1min":0.627,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.621,"5min":1.6399999999999999,"15min":1.76},"last":1.109}]},{"osd":2,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.097,"15min":1.0860000000000001},"min":{"1min":0.65300000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.556,"5min":1.649,"15min":1.649},"last":1.21},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.095,"15min":1.1459999999999999},"min":{"1min":0.81100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4179999999999999,"5min":1.623,"15min":1.6870000000000001},"last":1.508}]},{"osd":3,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.056,"15min":1.085},"min":{"1min":0.69499999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.244,"15min":1.256},"min":{"1min":0.91500000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.716,"5min":1.7609999999999999,"15min":1.9830000000000001},"last":1.456}]},{"osd":4,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.387,"15min":1.389},"min":{"1min":1.129,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.681,"5min":1.9390000000000001,"15min":1.95},"last":1.611},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3180000000000001,"15min":1.3500000000000001},"min":{"1min":1.0329999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.645,"5min":2.1539999999999999,"15min":2.4820000000000002},"last":1.252}]},{"osd":5,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3660000000000001,"15min":1.3879999999999999},"min":{"1min":0.996,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":1.829,"5min":2.0750000000000002,"15min":2.6669999999999998},"last":1.401},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.454},"min":{"1min":1.028,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.748,"5min":2.2029999999999998,"15min":2.2040000000000002},"last":1.639}]},{"osd":6,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.4750000000000001,"15min":1.5029999999999999},"min":{"1min":1.1080000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.819,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.405,"5min":1.4630000000000001,"15min":1.492},"min":{"1min":0.93799999999999994,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.758,"5min":2.262,"15min":2.262},"last":1.7170000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.52,"15min":1.5549999999999999},"min":{"1min":1.2749999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7150000000000001,"5min":2.347,"15min":2.347},"last":1.794},{"interface":"front","average":{"1min":1.454,"5min":1.474,"15min":1.5109999999999999},"min":{"1min":1.129,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.9530000000000001,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.49}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:24:45.397 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:24:45.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.647+0000 7fde8df3e640 1 Processor -- start 2023-10-16T11:24:45.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.647+0000 7fde8df3e640 1 -- start start 2023-10-16T11:24:45.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.648+0000 7fde8df3e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fde88159b20 0x7fde8806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:45.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.649+0000 7fde8df3e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fde880b0b20 0x7fde8806fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:45.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.649+0000 7fde8df3e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8806f7f0 con 0x7fde88159f00 2023-10-16T11:24:45.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.649+0000 7fde8df3e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8806c5e0 con 0x7fde880b0b20 2023-10-16T11:24:45.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.649+0000 7fde8df3e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8806c8c0 con 0x7fde88159b20 2023-10-16T11:24:45.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.649+0000 7fde86ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fde88159f00 0x7fde880b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47948/0 (socket says 172.21.15.84:47948) 2023-10-16T11:24:45.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.649+0000 7fde86ffd640 1 -- 172.21.15.84:0/1031804927 learned_addr learned my addr 172.21.15.84:0/1031804927 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:45.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.649+0000 7fde87fff640 1 --2- 172.21.15.84:0/1031804927 >> v2:172.21.15.84:3300/0 conn(0x7fde880b0b20 0x7fde8806fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:45.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.649+0000 7fde877fe640 1 --2- 172.21.15.84:0/1031804927 >> v2:172.21.15.157:3300/0 conn(0x7fde88159b20 0x7fde8806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:45.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.649+0000 7fde867fc640 1 -- 172.21.15.84:0/1031804927 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2197062147 0 0) 0x7fde8806f7f0 con 0x7fde88159f00 2023-10-16T11:24:45.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.649+0000 7fde867fc640 1 -- 172.21.15.84:0/1031804927 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde74003520 con 0x7fde88159f00 2023-10-16T11:24:45.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.650+0000 7fde87fff640 1 -- 172.21.15.84:0/1031804927 >> v2:172.21.15.157:3300/0 conn(0x7fde88159b20 msgr2=0x7fde8806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:45.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.650+0000 7fde87fff640 1 --2- 172.21.15.84:0/1031804927 >> v2:172.21.15.157:3300/0 conn(0x7fde88159b20 0x7fde8806bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.650+0000 7fde87fff640 1 -- 172.21.15.84:0/1031804927 >> v1:172.21.15.84:6789/0 conn(0x7fde88159f00 legacy=0x7fde880b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:45.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.650+0000 7fde87fff640 1 -- 172.21.15.84:0/1031804927 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8806cba0 con 0x7fde880b0b20 2023-10-16T11:24:45.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.651+0000 7fde87fff640 1 --2- 172.21.15.84:0/1031804927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde880b0b20 0x7fde8806fc80 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7fde880c9340 tx=0x7fde8002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b77507b3ee6662 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:45.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.651+0000 7fde867fc640 1 -- 172.21.15.84:0/1031804927 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde80031830 con 0x7fde880b0b20 2023-10-16T11:24:45.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.651+0000 7fde867fc640 1 -- 172.21.15.84:0/1031804927 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde80031990 con 0x7fde880b0b20 2023-10-16T11:24:45.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.651+0000 7fde867fc640 1 -- 172.21.15.84:0/1031804927 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde80031d20 con 0x7fde880b0b20 2023-10-16T11:24:45.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.652+0000 7fde8df3e640 1 -- 172.21.15.84:0/1031804927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde880b0b20 msgr2=0x7fde8806fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:45.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.652+0000 7fde8df3e640 1 --2- 172.21.15.84:0/1031804927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde880b0b20 0x7fde8806fc80 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7fde880c9340 tx=0x7fde8002b8d0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.652+0000 7fde8df3e640 1 -- 172.21.15.84:0/1031804927 shutdown_connections 2023-10-16T11:24:45.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.652+0000 7fde8df3e640 1 --2- 172.21.15.84:0/1031804927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde880b0b20 0x7fde8806fc80 unknown :-1 s=CLOSED pgs=2684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.652+0000 7fde8df3e640 1 --2- 172.21.15.84:0/1031804927 >> v2:172.21.15.157:3300/0 conn(0x7fde88159b20 0x7fde8806bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.652+0000 7fde8df3e640 1 -- 172.21.15.84:0/1031804927 >> 172.21.15.84:0/1031804927 conn(0x7fde880cb410 msgr2=0x7fde880cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:45.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.652+0000 7fde8df3e640 1 -- 172.21.15.84:0/1031804927 shutdown_connections 2023-10-16T11:24:45.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.652+0000 7fde8df3e640 1 -- 172.21.15.84:0/1031804927 wait complete. 2023-10-16T11:24:45.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.654+0000 7fde8df3e640 1 Processor -- start 2023-10-16T11:24:45.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.654+0000 7fde8df3e640 1 -- start start 2023-10-16T11:24:45.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.655+0000 7fde8df3e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde880b0b20 0x7fde8814e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:45.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.655+0000 7fde8df3e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde88159b20 0x7fde88152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:45.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.655+0000 7fde8df3e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88159f00 0x7fde880c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:45.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.655+0000 7fde8df3e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8814e500 con 0x7fde88159b20 2023-10-16T11:24:45.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.655+0000 7fde8df3e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8814d980 con 0x7fde88159f00 2023-10-16T11:24:45.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.655+0000 7fde8df3e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8814dbd0 con 0x7fde880b0b20 2023-10-16T11:24:45.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.655+0000 7fde86ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde88159b20 0x7fde88152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:45.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.655+0000 7fde86ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde88159b20 0x7fde88152f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51262/0 (socket says 172.21.15.84:51262) 2023-10-16T11:24:45.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.655+0000 7fde86ffd640 1 -- 172.21.15.84:0/1663496252 learned_addr learned my addr 172.21.15.84:0/1663496252 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:45.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.655+0000 7fde86ffd640 1 -- 172.21.15.84:0/1663496252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde880b0b20 msgr2=0x7fde8814e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:45.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.655+0000 7fde86ffd640 1 --2- 172.21.15.84:0/1663496252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde880b0b20 0x7fde8814e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.655+0000 7fde87fff640 1 --2- 172.21.15.84:0/1663496252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88159f00 0x7fde880c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:45.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.655+0000 7fde86ffd640 1 -- 172.21.15.84:0/1663496252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88159f00 msgr2=0x7fde880c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:45.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.655+0000 7fde86ffd640 1 --2- 172.21.15.84:0/1663496252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88159f00 0x7fde880c7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.655+0000 7fde86ffd640 1 -- 172.21.15.84:0/1663496252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde80009400 con 0x7fde88159b20 2023-10-16T11:24:45.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.656+0000 7fde86ffd640 1 --2- 172.21.15.84:0/1663496252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde88159b20 0x7fde88152f70 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7fde7000c4d0 tx=0x7fde7000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:45.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.656+0000 7fde7ffff640 1 -- 172.21.15.84:0/1663496252 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde70005ce0 con 0x7fde88159b20 2023-10-16T11:24:45.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.656+0000 7fde7ffff640 1 -- 172.21.15.84:0/1663496252 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde70005e40 con 0x7fde88159b20 2023-10-16T11:24:45.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.656+0000 7fde7ffff640 1 -- 172.21.15.84:0/1663496252 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde70007270 con 0x7fde88159b20 2023-10-16T11:24:45.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.656+0000 7fde8df3e640 1 -- 172.21.15.84:0/1663496252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8814deb0 con 0x7fde88159b20 2023-10-16T11:24:45.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.656+0000 7fde8df3e640 1 -- 172.21.15.84:0/1663496252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8814eed0 con 0x7fde88159b20 2023-10-16T11:24:45.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.658+0000 7fde8df3e640 1 -- 172.21.15.84:0/1663496252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8814f0c0 con 0x7fde88159b20 2023-10-16T11:24:45.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.658+0000 7fde7ffff640 1 -- 172.21.15.84:0/1663496252 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fde70007490 con 0x7fde88159b20 2023-10-16T11:24:45.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.659+0000 7fde7ffff640 1 --2- 172.21.15.84:0/1663496252 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fde6806af90 0x7fde6806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:45.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.659+0000 7fde7ffff640 1 -- 172.21.15.84:0/1663496252 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7fde70088f70 con 0x7fde88159b20 2023-10-16T11:24:45.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.659+0000 7fde877fe640 1 --2- 172.21.15.84:0/1663496252 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fde6806af90 0x7fde6806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:45.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.660+0000 7fde877fe640 1 --2- 172.21.15.84:0/1663496252 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fde6806af90 0x7fde6806d450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fde780070b0 tx=0x7fde78007040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:45.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.664+0000 7fde7ffff640 1 -- 172.21.15.84:0/1663496252 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fde700bbfe0 con 0x7fde88159b20 2023-10-16T11:24:45.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.852+0000 7fde8df3e640 1 -- 172.21.15.84:0/1663496252 --> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fde8806ce80 con 0x7fde6806af90 2023-10-16T11:24:45.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.874+0000 7fde7ffff640 1 -- 172.21.15.84:0/1663496252 <== mgr.11076 v2:172.21.15.84:6832/1425409017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+514753 (secure 0 0 0) 0x7fde78009bb0 con 0x7fde6806af90 2023-10-16T11:24:45.883 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:45.887 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:24:45.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.884+0000 7fde8df3e640 1 -- 172.21.15.84:0/1663496252 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fde6806af90 msgr2=0x7fde6806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:45.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.884+0000 7fde8df3e640 1 --2- 172.21.15.84:0/1663496252 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fde6806af90 0x7fde6806d450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fde780070b0 tx=0x7fde78007040 comp rx=0 tx=0).stop 2023-10-16T11:24:45.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.884+0000 7fde8df3e640 1 -- 172.21.15.84:0/1663496252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde88159b20 msgr2=0x7fde88152f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:45.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.884+0000 7fde8df3e640 1 --2- 172.21.15.84:0/1663496252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde88159b20 0x7fde88152f70 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7fde7000c4d0 tx=0x7fde7000c990 comp rx=0 tx=0).stop 2023-10-16T11:24:45.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.885+0000 7fde8df3e640 1 -- 172.21.15.84:0/1663496252 shutdown_connections 2023-10-16T11:24:45.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.885+0000 7fde8df3e640 1 --2- 172.21.15.84:0/1663496252 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fde6806af90 0x7fde6806d450 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.885+0000 7fde8df3e640 1 --2- 172.21.15.84:0/1663496252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88159f00 0x7fde880c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.885+0000 7fde8df3e640 1 --2- 172.21.15.84:0/1663496252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde88159b20 0x7fde88152f70 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.885+0000 7fde8df3e640 1 --2- 172.21.15.84:0/1663496252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde880b0b20 0x7fde8814e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:45.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.885+0000 7fde8df3e640 1 -- 172.21.15.84:0/1663496252 >> 172.21.15.84:0/1663496252 conn(0x7fde880cb410 msgr2=0x7fde8815d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:45.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.885+0000 7fde8df3e640 1 -- 172.21.15.84:0/1663496252 shutdown_connections 2023-10-16T11:24:45.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:45.885+0000 7fde8df3e640 1 -- 172.21.15.84:0/1663496252 wait complete. 2023-10-16T11:24:45.908 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2023-10-16T11:24:44.256729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594605,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":635,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2814,"ondisk_log_size":2814,"up":266,"acting":266,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":215,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":380328,"kb_used_data":22584,"kb_used_omap":12,"kb_used_meta":357619,"kb_avail":754594392,"statfs":{"total":773094113280,"available":772704657408,"internally_reserved":0,"allocated":23126016,"data_stored":11794435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":366202425},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1569,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.377056"},"pg_stats":[{"pgid":"29.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"206'2767","reported_seq":3649,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069366+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:24:28.069366+0000","last_peered":"2023-10-16T11:24:28.069366+0000","last_clean":"2023-10-16T11:24:28.069366+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:24:28.069366+0000","last_undegraded":"2023-10-16T11:24:28.069366+0000","last_fullsized":"2023-10-16T11:24:28.069366+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2767,"log_dups_size":0,"ondisk_log_size":2767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":581,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"216'1","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.976811+0000","last_change":"2023-10-16T11:24:25.087491+0000","last_active":"2023-10-16T11:24:27.976811+0000","last_peered":"2023-10-16T11:24:27.976811+0000","last_clean":"2023-10-16T11:24:27.976811+0000","last_became_active":"2023-10-16T11:24:25.086826+0000","last_became_peered":"2023-10-16T11:24:25.086826+0000","last_unstale":"2023-10-16T11:24:27.976811+0000","last_undegraded":"2023-10-16T11:24:27.976811+0000","last_fullsized":"2023-10-16T11:24:27.976811+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:07.303719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918232+0000","last_change":"2023-10-16T11:24:26.091340+0000","last_active":"2023-10-16T11:24:40.918232+0000","last_peered":"2023-10-16T11:24:40.918232+0000","last_clean":"2023-10-16T11:24:40.918232+0000","last_became_active":"2023-10-16T11:24:26.091059+0000","last_became_peered":"2023-10-16T11:24:26.091059+0000","last_unstale":"2023-10-16T11:24:40.918232+0000","last_undegraded":"2023-10-16T11:24:40.918232+0000","last_fullsized":"2023-10-16T11:24:40.918232+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:37.605400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.892205+0000","last_change":"2023-10-16T11:24:38.892205+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.892205+0000","last_undegraded":"2023-10-16T11:24:38.892205+0000","last_fullsized":"2023-10-16T11:24:38.892205+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.975997+0000","last_change":"2023-10-16T11:24:25.087755+0000","last_active":"2023-10-16T11:24:27.975997+0000","last_peered":"2023-10-16T11:24:27.975997+0000","last_clean":"2023-10-16T11:24:27.975997+0000","last_became_active":"2023-10-16T11:24:25.087497+0000","last_became_peered":"2023-10-16T11:24:25.087497+0000","last_unstale":"2023-10-16T11:24:27.975997+0000","last_undegraded":"2023-10-16T11:24:27.975997+0000","last_fullsized":"2023-10-16T11:24:27.975997+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:59:26.196708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255404+0000","last_change":"2023-10-16T11:24:26.081718+0000","last_active":"2023-10-16T11:24:29.255404+0000","last_peered":"2023-10-16T11:24:29.255404+0000","last_clean":"2023-10-16T11:24:29.255404+0000","last_became_active":"2023-10-16T11:24:26.081447+0000","last_became_peered":"2023-10-16T11:24:26.081447+0000","last_unstale":"2023-10-16T11:24:29.255404+0000","last_undegraded":"2023-10-16T11:24:29.255404+0000","last_fullsized":"2023-10-16T11:24:29.255404+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.884705+0000","last_change":"2023-10-16T11:24:38.884705+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.884705+0000","last_undegraded":"2023-10-16T11:24:38.884705+0000","last_fullsized":"2023-10-16T11:24:38.884705+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.083691+0000","last_change":"2023-10-16T11:24:25.088161+0000","last_active":"2023-10-16T11:24:27.083691+0000","last_peered":"2023-10-16T11:24:27.083691+0000","last_clean":"2023-10-16T11:24:27.083691+0000","last_became_active":"2023-10-16T11:24:25.086051+0000","last_became_peered":"2023-10-16T11:24:25.086051+0000","last_unstale":"2023-10-16T11:24:27.083691+0000","last_undegraded":"2023-10-16T11:24:27.083691+0000","last_fullsized":"2023-10-16T11:24:27.083691+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.257788+0000","last_change":"2023-10-16T11:24:26.083366+0000","last_active":"2023-10-16T11:24:29.257788+0000","last_peered":"2023-10-16T11:24:29.257788+0000","last_clean":"2023-10-16T11:24:29.257788+0000","last_became_active":"2023-10-16T11:24:26.082635+0000","last_became_peered":"2023-10-16T11:24:26.082635+0000","last_unstale":"2023-10-16T11:24:29.257788+0000","last_undegraded":"2023-10-16T11:24:29.257788+0000","last_fullsized":"2023-10-16T11:24:29.257788+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.890077+0000","last_change":"2023-10-16T11:24:38.890077+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.890077+0000","last_undegraded":"2023-10-16T11:24:38.890077+0000","last_fullsized":"2023-10-16T11:24:38.890077+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.071234+0000","last_change":"2023-10-16T11:24:25.080689+0000","last_active":"2023-10-16T11:24:28.071234+0000","last_peered":"2023-10-16T11:24:28.071234+0000","last_clean":"2023-10-16T11:24:28.071234+0000","last_became_active":"2023-10-16T11:24:25.080305+0000","last_became_peered":"2023-10-16T11:24:25.080305+0000","last_unstale":"2023-10-16T11:24:28.071234+0000","last_undegraded":"2023-10-16T11:24:28.071234+0000","last_fullsized":"2023-10-16T11:24:28.071234+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:43.670603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.254685+0000","last_change":"2023-10-16T11:24:26.083766+0000","last_active":"2023-10-16T11:24:29.254685+0000","last_peered":"2023-10-16T11:24:29.254685+0000","last_clean":"2023-10-16T11:24:29.254685+0000","last_became_active":"2023-10-16T11:24:26.083447+0000","last_became_peered":"2023-10-16T11:24:26.083447+0000","last_unstale":"2023-10-16T11:24:29.254685+0000","last_undegraded":"2023-10-16T11:24:29.254685+0000","last_fullsized":"2023-10-16T11:24:29.254685+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:44:14.656443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.901288+0000","last_change":"2023-10-16T11:24:38.901288+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.901288+0000","last_undegraded":"2023-10-16T11:24:38.901288+0000","last_fullsized":"2023-10-16T11:24:38.901288+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069569+0000","last_change":"2023-10-16T11:24:25.090432+0000","last_active":"2023-10-16T11:24:28.069569+0000","last_peered":"2023-10-16T11:24:28.069569+0000","last_clean":"2023-10-16T11:24:28.069569+0000","last_became_active":"2023-10-16T11:24:25.090142+0000","last_became_peered":"2023-10-16T11:24:25.090142+0000","last_unstale":"2023-10-16T11:24:28.069569+0000","last_undegraded":"2023-10-16T11:24:28.069569+0000","last_fullsized":"2023-10-16T11:24:28.069569+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:22.858690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918327+0000","last_change":"2023-10-16T11:24:26.093027+0000","last_active":"2023-10-16T11:24:40.918327+0000","last_peered":"2023-10-16T11:24:40.918327+0000","last_clean":"2023-10-16T11:24:40.918327+0000","last_became_active":"2023-10-16T11:24:26.092766+0000","last_became_peered":"2023-10-16T11:24:26.092766+0000","last_unstale":"2023-10-16T11:24:40.918327+0000","last_undegraded":"2023-10-16T11:24:40.918327+0000","last_fullsized":"2023-10-16T11:24:40.918327+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.884720+0000","last_change":"2023-10-16T11:24:38.884720+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.884720+0000","last_undegraded":"2023-10-16T11:24:38.884720+0000","last_fullsized":"2023-10-16T11:24:38.884720+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.080926+0000","last_change":"2023-10-16T11:24:25.088585+0000","last_active":"2023-10-16T11:24:27.080926+0000","last_peered":"2023-10-16T11:24:27.080926+0000","last_clean":"2023-10-16T11:24:27.080926+0000","last_became_active":"2023-10-16T11:24:25.088011+0000","last_became_peered":"2023-10-16T11:24:25.088011+0000","last_unstale":"2023-10-16T11:24:27.080926+0000","last_undegraded":"2023-10-16T11:24:27.080926+0000","last_fullsized":"2023-10-16T11:24:27.080926+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.256412+0000","last_change":"2023-10-16T11:24:26.096341+0000","last_active":"2023-10-16T11:24:29.256412+0000","last_peered":"2023-10-16T11:24:29.256412+0000","last_clean":"2023-10-16T11:24:29.256412+0000","last_became_active":"2023-10-16T11:24:26.095836+0000","last_became_peered":"2023-10-16T11:24:26.095836+0000","last_unstale":"2023-10-16T11:24:29.256412+0000","last_undegraded":"2023-10-16T11:24:29.256412+0000","last_fullsized":"2023-10-16T11:24:29.256412+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917221+0000","last_change":"2023-10-16T11:24:39.909816+0000","last_active":"2023-10-16T11:24:40.917221+0000","last_peered":"2023-10-16T11:24:40.917221+0000","last_clean":"2023-10-16T11:24:40.917221+0000","last_became_active":"2023-10-16T11:24:39.909526+0000","last_became_peered":"2023-10-16T11:24:39.909526+0000","last_unstale":"2023-10-16T11:24:40.917221+0000","last_undegraded":"2023-10-16T11:24:40.917221+0000","last_fullsized":"2023-10-16T11:24:40.917221+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:12.120581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070811+0000","last_change":"2023-10-16T11:24:25.077036+0000","last_active":"2023-10-16T11:24:28.070811+0000","last_peered":"2023-10-16T11:24:28.070811+0000","last_clean":"2023-10-16T11:24:28.070811+0000","last_became_active":"2023-10-16T11:24:25.076342+0000","last_became_peered":"2023-10-16T11:24:25.076342+0000","last_unstale":"2023-10-16T11:24:28.070811+0000","last_undegraded":"2023-10-16T11:24:28.070811+0000","last_fullsized":"2023-10-16T11:24:28.070811+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:36.162662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255460+0000","last_change":"2023-10-16T11:24:26.085231+0000","last_active":"2023-10-16T11:24:29.255460+0000","last_peered":"2023-10-16T11:24:29.255460+0000","last_clean":"2023-10-16T11:24:29.255460+0000","last_became_active":"2023-10-16T11:24:26.084889+0000","last_became_peered":"2023-10-16T11:24:26.084889+0000","last_unstale":"2023-10-16T11:24:29.255460+0000","last_undegraded":"2023-10-16T11:24:29.255460+0000","last_fullsized":"2023-10-16T11:24:29.255460+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.891881+0000","last_change":"2023-10-16T11:24:38.891881+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.891881+0000","last_undegraded":"2023-10-16T11:24:38.891881+0000","last_fullsized":"2023-10-16T11:24:38.891881+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.6","version":"217'1","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:30.685218+0000","last_change":"2023-10-16T11:24:25.093214+0000","last_active":"2023-10-16T11:24:30.685218+0000","last_peered":"2023-10-16T11:24:30.685218+0000","last_clean":"2023-10-16T11:24:30.685218+0000","last_became_active":"2023-10-16T11:24:25.092877+0000","last_became_peered":"2023-10-16T11:24:25.092877+0000","last_unstale":"2023-10-16T11:24:30.685218+0000","last_undegraded":"2023-10-16T11:24:30.685218+0000","last_fullsized":"2023-10-16T11:24:30.685218+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:59.924424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.919655+0000","last_change":"2023-10-16T11:24:26.089940+0000","last_active":"2023-10-16T11:24:40.919655+0000","last_peered":"2023-10-16T11:24:40.919655+0000","last_clean":"2023-10-16T11:24:40.919655+0000","last_became_active":"2023-10-16T11:24:26.087906+0000","last_became_peered":"2023-10-16T11:24:26.087906+0000","last_unstale":"2023-10-16T11:24:40.919655+0000","last_undegraded":"2023-10-16T11:24:40.919655+0000","last_fullsized":"2023-10-16T11:24:40.919655+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:44.687193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.901030+0000","last_change":"2023-10-16T11:24:38.901030+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.901030+0000","last_undegraded":"2023-10-16T11:24:38.901030+0000","last_fullsized":"2023-10-16T11:24:38.901030+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.7","version":"216'2","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.081261+0000","last_change":"2023-10-16T11:24:25.094230+0000","last_active":"2023-10-16T11:24:27.081261+0000","last_peered":"2023-10-16T11:24:27.081261+0000","last_clean":"2023-10-16T11:24:27.081261+0000","last_became_active":"2023-10-16T11:24:25.093896+0000","last_became_peered":"2023-10-16T11:24:25.093896+0000","last_unstale":"2023-10-16T11:24:27.081261+0000","last_undegraded":"2023-10-16T11:24:27.081261+0000","last_fullsized":"2023-10-16T11:24:27.081261+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.254860+0000","last_change":"2023-10-16T11:24:26.085909+0000","last_active":"2023-10-16T11:24:29.254860+0000","last_peered":"2023-10-16T11:24:29.254860+0000","last_clean":"2023-10-16T11:24:29.254860+0000","last_became_active":"2023-10-16T11:24:26.085620+0000","last_became_peered":"2023-10-16T11:24:26.085620+0000","last_unstale":"2023-10-16T11:24:29.254860+0000","last_undegraded":"2023-10-16T11:24:29.254860+0000","last_fullsized":"2023-10-16T11:24:29.254860+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:41.023833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.889481+0000","last_change":"2023-10-16T11:24:38.889481+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.889481+0000","last_undegraded":"2023-10-16T11:24:38.889481+0000","last_fullsized":"2023-10-16T11:24:38.889481+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069148+0000","last_change":"2023-10-16T11:24:25.080225+0000","last_active":"2023-10-16T11:24:28.069148+0000","last_peered":"2023-10-16T11:24:28.069148+0000","last_clean":"2023-10-16T11:24:28.069148+0000","last_became_active":"2023-10-16T11:24:25.079921+0000","last_became_peered":"2023-10-16T11:24:25.079921+0000","last_unstale":"2023-10-16T11:24:28.069148+0000","last_undegraded":"2023-10-16T11:24:28.069148+0000","last_fullsized":"2023-10-16T11:24:28.069148+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:46.550429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.256329+0000","last_change":"2023-10-16T11:24:26.088438+0000","last_active":"2023-10-16T11:24:29.256329+0000","last_peered":"2023-10-16T11:24:29.256329+0000","last_clean":"2023-10-16T11:24:29.256329+0000","last_became_active":"2023-10-16T11:24:26.087973+0000","last_became_peered":"2023-10-16T11:24:26.087973+0000","last_unstale":"2023-10-16T11:24:29.256329+0000","last_undegraded":"2023-10-16T11:24:29.256329+0000","last_fullsized":"2023-10-16T11:24:29.256329+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.944794+0000","last_change":"2023-10-16T11:24:39.915986+0000","last_active":"2023-10-16T11:24:40.944794+0000","last_peered":"2023-10-16T11:24:40.944794+0000","last_clean":"2023-10-16T11:24:40.944794+0000","last_became_active":"2023-10-16T11:24:39.915611+0000","last_became_peered":"2023-10-16T11:24:39.915611+0000","last_unstale":"2023-10-16T11:24:40.944794+0000","last_undegraded":"2023-10-16T11:24:40.944794+0000","last_fullsized":"2023-10-16T11:24:40.944794+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:37.032264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917511+0000","last_change":"2023-10-16T11:24:25.086180+0000","last_active":"2023-10-16T11:24:40.917511+0000","last_peered":"2023-10-16T11:24:40.917511+0000","last_clean":"2023-10-16T11:24:40.917511+0000","last_became_active":"2023-10-16T11:24:25.085316+0000","last_became_peered":"2023-10-16T11:24:25.085316+0000","last_unstale":"2023-10-16T11:24:40.917511+0000","last_undegraded":"2023-10-16T11:24:40.917511+0000","last_fullsized":"2023-10-16T11:24:40.917511+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255372+0000","last_change":"2023-10-16T11:24:26.090951+0000","last_active":"2023-10-16T11:24:29.255372+0000","last_peered":"2023-10-16T11:24:29.255372+0000","last_clean":"2023-10-16T11:24:29.255372+0000","last_became_active":"2023-10-16T11:24:26.089241+0000","last_became_peered":"2023-10-16T11:24:26.089241+0000","last_unstale":"2023-10-16T11:24:29.255372+0000","last_undegraded":"2023-10-16T11:24:29.255372+0000","last_fullsized":"2023-10-16T11:24:29.255372+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:40.970357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.901898+0000","last_change":"2023-10-16T11:24:38.901898+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.901898+0000","last_undegraded":"2023-10-16T11:24:38.901898+0000","last_fullsized":"2023-10-16T11:24:38.901898+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918490+0000","last_change":"2023-10-16T11:24:25.091656+0000","last_active":"2023-10-16T11:24:40.918490+0000","last_peered":"2023-10-16T11:24:40.918490+0000","last_clean":"2023-10-16T11:24:40.918490+0000","last_became_active":"2023-10-16T11:24:25.091316+0000","last_became_peered":"2023-10-16T11:24:25.091316+0000","last_unstale":"2023-10-16T11:24:40.918490+0000","last_undegraded":"2023-10-16T11:24:40.918490+0000","last_fullsized":"2023-10-16T11:24:40.918490+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:26.054702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.257873+0000","last_change":"2023-10-16T11:24:26.085260+0000","last_active":"2023-10-16T11:24:29.257873+0000","last_peered":"2023-10-16T11:24:29.257873+0000","last_clean":"2023-10-16T11:24:29.257873+0000","last_became_active":"2023-10-16T11:24:26.084860+0000","last_became_peered":"2023-10-16T11:24:26.084860+0000","last_unstale":"2023-10-16T11:24:29.257873+0000","last_undegraded":"2023-10-16T11:24:29.257873+0000","last_fullsized":"2023-10-16T11:24:29.257873+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.894485+0000","last_change":"2023-10-16T11:24:38.894485+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.894485+0000","last_undegraded":"2023-10-16T11:24:38.894485+0000","last_fullsized":"2023-10-16T11:24:38.894485+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.b","version":"216'2","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.082378+0000","last_change":"2023-10-16T11:24:25.072340+0000","last_active":"2023-10-16T11:24:27.082378+0000","last_peered":"2023-10-16T11:24:27.082378+0000","last_clean":"2023-10-16T11:24:27.082378+0000","last_became_active":"2023-10-16T11:24:25.071728+0000","last_became_peered":"2023-10-16T11:24:25.071728+0000","last_unstale":"2023-10-16T11:24:27.082378+0000","last_undegraded":"2023-10-16T11:24:27.082378+0000","last_fullsized":"2023-10-16T11:24:27.082378+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:49.041521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.262758+0000","last_change":"2023-10-16T11:24:26.091096+0000","last_active":"2023-10-16T11:24:29.262758+0000","last_peered":"2023-10-16T11:24:29.262758+0000","last_clean":"2023-10-16T11:24:29.262758+0000","last_became_active":"2023-10-16T11:24:26.090774+0000","last_became_peered":"2023-10-16T11:24:26.090774+0000","last_unstale":"2023-10-16T11:24:29.262758+0000","last_undegraded":"2023-10-16T11:24:29.262758+0000","last_fullsized":"2023-10-16T11:24:29.262758+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:27.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.885799+0000","last_change":"2023-10-16T11:24:38.885799+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.885799+0000","last_undegraded":"2023-10-16T11:24:38.885799+0000","last_fullsized":"2023-10-16T11:24:38.885799+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.080804+0000","last_change":"2023-10-16T11:24:25.079021+0000","last_active":"2023-10-16T11:24:27.080804+0000","last_peered":"2023-10-16T11:24:27.080804+0000","last_clean":"2023-10-16T11:24:27.080804+0000","last_became_active":"2023-10-16T11:24:25.076800+0000","last_became_peered":"2023-10-16T11:24:25.076800+0000","last_unstale":"2023-10-16T11:24:27.080804+0000","last_undegraded":"2023-10-16T11:24:27.080804+0000","last_fullsized":"2023-10-16T11:24:27.080804+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:56.161407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.262906+0000","last_change":"2023-10-16T11:24:26.092876+0000","last_active":"2023-10-16T11:24:29.262906+0000","last_peered":"2023-10-16T11:24:29.262906+0000","last_clean":"2023-10-16T11:24:29.262906+0000","last_became_active":"2023-10-16T11:24:26.091310+0000","last_became_peered":"2023-10-16T11:24:26.091310+0000","last_unstale":"2023-10-16T11:24:29.262906+0000","last_undegraded":"2023-10-16T11:24:29.262906+0000","last_fullsized":"2023-10-16T11:24:29.262906+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:22.891780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.892205+0000","last_change":"2023-10-16T11:24:38.892205+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.892205+0000","last_undegraded":"2023-10-16T11:24:38.892205+0000","last_fullsized":"2023-10-16T11:24:38.892205+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.d","version":"216'1","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.083113+0000","last_change":"2023-10-16T11:24:25.083155+0000","last_active":"2023-10-16T11:24:27.083113+0000","last_peered":"2023-10-16T11:24:27.083113+0000","last_clean":"2023-10-16T11:24:27.083113+0000","last_became_active":"2023-10-16T11:24:25.082690+0000","last_became_peered":"2023-10-16T11:24:25.082690+0000","last_unstale":"2023-10-16T11:24:27.083113+0000","last_undegraded":"2023-10-16T11:24:27.083113+0000","last_fullsized":"2023-10-16T11:24:27.083113+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:46:58.816018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.257935+0000","last_change":"2023-10-16T11:24:26.069780+0000","last_active":"2023-10-16T11:24:29.257935+0000","last_peered":"2023-10-16T11:24:29.257935+0000","last_clean":"2023-10-16T11:24:29.257935+0000","last_became_active":"2023-10-16T11:24:26.068772+0000","last_became_peered":"2023-10-16T11:24:26.068772+0000","last_unstale":"2023-10-16T11:24:29.257935+0000","last_undegraded":"2023-10-16T11:24:29.257935+0000","last_fullsized":"2023-10-16T11:24:29.257935+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.901546+0000","last_change":"2023-10-16T11:24:38.901546+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.901546+0000","last_undegraded":"2023-10-16T11:24:38.901546+0000","last_fullsized":"2023-10-16T11:24:38.901546+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.e","version":"216'2","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.201299+0000","last_change":"2023-10-16T11:24:25.087765+0000","last_active":"2023-10-16T11:24:27.201299+0000","last_peered":"2023-10-16T11:24:27.201299+0000","last_clean":"2023-10-16T11:24:27.201299+0000","last_became_active":"2023-10-16T11:24:25.086967+0000","last_became_peered":"2023-10-16T11:24:25.086967+0000","last_unstale":"2023-10-16T11:24:27.201299+0000","last_undegraded":"2023-10-16T11:24:27.201299+0000","last_fullsized":"2023-10-16T11:24:27.201299+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255655+0000","last_change":"2023-10-16T11:24:26.086833+0000","last_active":"2023-10-16T11:24:29.255655+0000","last_peered":"2023-10-16T11:24:29.255655+0000","last_clean":"2023-10-16T11:24:29.255655+0000","last_became_active":"2023-10-16T11:24:26.085367+0000","last_became_peered":"2023-10-16T11:24:26.085367+0000","last_unstale":"2023-10-16T11:24:29.255655+0000","last_undegraded":"2023-10-16T11:24:29.255655+0000","last_fullsized":"2023-10-16T11:24:29.255655+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.902329+0000","last_change":"2023-10-16T11:24:38.902329+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.902329+0000","last_undegraded":"2023-10-16T11:24:38.902329+0000","last_fullsized":"2023-10-16T11:24:38.902329+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"216'1","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.071099+0000","last_change":"2023-10-16T11:24:25.079789+0000","last_active":"2023-10-16T11:24:28.071099+0000","last_peered":"2023-10-16T11:24:28.071099+0000","last_clean":"2023-10-16T11:24:28.071099+0000","last_became_active":"2023-10-16T11:24:25.079495+0000","last_became_peered":"2023-10-16T11:24:25.079495+0000","last_unstale":"2023-10-16T11:24:28.071099+0000","last_undegraded":"2023-10-16T11:24:28.071099+0000","last_fullsized":"2023-10-16T11:24:28.071099+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:44.531367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255585+0000","last_change":"2023-10-16T11:24:26.081796+0000","last_active":"2023-10-16T11:24:29.255585+0000","last_peered":"2023-10-16T11:24:29.255585+0000","last_clean":"2023-10-16T11:24:29.255585+0000","last_became_active":"2023-10-16T11:24:26.080287+0000","last_became_peered":"2023-10-16T11:24:26.080287+0000","last_unstale":"2023-10-16T11:24:29.255585+0000","last_undegraded":"2023-10-16T11:24:29.255585+0000","last_fullsized":"2023-10-16T11:24:29.255585+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.891885+0000","last_change":"2023-10-16T11:24:38.891885+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.891885+0000","last_undegraded":"2023-10-16T11:24:38.891885+0000","last_fullsized":"2023-10-16T11:24:38.891885+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.10","version":"216'8","reported_seq":21,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.080292+0000","last_change":"2023-10-16T11:24:25.076954+0000","last_active":"2023-10-16T11:24:27.080292+0000","last_peered":"2023-10-16T11:24:27.080292+0000","last_clean":"2023-10-16T11:24:27.080292+0000","last_became_active":"2023-10-16T11:24:25.076509+0000","last_became_peered":"2023-10-16T11:24:25.076509+0000","last_unstale":"2023-10-16T11:24:27.080292+0000","last_undegraded":"2023-10-16T11:24:27.080292+0000","last_fullsized":"2023-10-16T11:24:27.080292+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:46.770213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917418+0000","last_change":"2023-10-16T11:24:26.084139+0000","last_active":"2023-10-16T11:24:40.917418+0000","last_peered":"2023-10-16T11:24:40.917418+0000","last_clean":"2023-10-16T11:24:40.917418+0000","last_became_active":"2023-10-16T11:24:26.083876+0000","last_became_peered":"2023-10-16T11:24:26.083876+0000","last_unstale":"2023-10-16T11:24:40.917418+0000","last_undegraded":"2023-10-16T11:24:40.917418+0000","last_fullsized":"2023-10-16T11:24:40.917418+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918095+0000","last_change":"2023-10-16T11:24:39.904420+0000","last_active":"2023-10-16T11:24:40.918095+0000","last_peered":"2023-10-16T11:24:40.918095+0000","last_clean":"2023-10-16T11:24:40.918095+0000","last_became_active":"2023-10-16T11:24:39.904039+0000","last_became_peered":"2023-10-16T11:24:39.904039+0000","last_unstale":"2023-10-16T11:24:40.918095+0000","last_undegraded":"2023-10-16T11:24:40.918095+0000","last_fullsized":"2023-10-16T11:24:40.918095+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:17.673425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.081912+0000","last_change":"2023-10-16T11:24:25.089470+0000","last_active":"2023-10-16T11:24:27.081912+0000","last_peered":"2023-10-16T11:24:27.081912+0000","last_clean":"2023-10-16T11:24:27.081912+0000","last_became_active":"2023-10-16T11:24:25.089213+0000","last_became_peered":"2023-10-16T11:24:25.089213+0000","last_unstale":"2023-10-16T11:24:27.081912+0000","last_undegraded":"2023-10-16T11:24:27.081912+0000","last_fullsized":"2023-10-16T11:24:27.081912+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:34.540849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.257718+0000","last_change":"2023-10-16T11:24:26.079602+0000","last_active":"2023-10-16T11:24:29.257718+0000","last_peered":"2023-10-16T11:24:29.257718+0000","last_clean":"2023-10-16T11:24:29.257718+0000","last_became_active":"2023-10-16T11:24:26.079344+0000","last_became_peered":"2023-10-16T11:24:26.079344+0000","last_unstale":"2023-10-16T11:24:29.257718+0000","last_undegraded":"2023-10-16T11:24:29.257718+0000","last_fullsized":"2023-10-16T11:24:29.257718+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.884929+0000","last_change":"2023-10-16T11:24:38.884929+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.884929+0000","last_undegraded":"2023-10-16T11:24:38.884929+0000","last_fullsized":"2023-10-16T11:24:38.884929+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.083648+0000","last_change":"2023-10-16T11:24:25.084194+0000","last_active":"2023-10-16T11:24:27.083648+0000","last_peered":"2023-10-16T11:24:27.083648+0000","last_clean":"2023-10-16T11:24:27.083648+0000","last_became_active":"2023-10-16T11:24:25.083141+0000","last_became_peered":"2023-10-16T11:24:25.083141+0000","last_unstale":"2023-10-16T11:24:27.083648+0000","last_undegraded":"2023-10-16T11:24:27.083648+0000","last_fullsized":"2023-10-16T11:24:27.083648+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:46.770213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.262799+0000","last_change":"2023-10-16T11:24:26.089221+0000","last_active":"2023-10-16T11:24:29.262799+0000","last_peered":"2023-10-16T11:24:29.262799+0000","last_clean":"2023-10-16T11:24:29.262799+0000","last_became_active":"2023-10-16T11:24:26.088644+0000","last_became_peered":"2023-10-16T11:24:26.088644+0000","last_unstale":"2023-10-16T11:24:29.262799+0000","last_undegraded":"2023-10-16T11:24:29.262799+0000","last_fullsized":"2023-10-16T11:24:29.262799+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:10.387018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.895034+0000","last_change":"2023-10-16T11:24:38.895034+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.895034+0000","last_undegraded":"2023-10-16T11:24:38.895034+0000","last_fullsized":"2023-10-16T11:24:38.895034+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.13","version":"216'2","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.083651+0000","last_change":"2023-10-16T11:24:25.081377+0000","last_active":"2023-10-16T11:24:27.083651+0000","last_peered":"2023-10-16T11:24:27.083651+0000","last_clean":"2023-10-16T11:24:27.083651+0000","last_became_active":"2023-10-16T11:24:25.080881+0000","last_became_peered":"2023-10-16T11:24:25.080881+0000","last_unstale":"2023-10-16T11:24:27.083651+0000","last_undegraded":"2023-10-16T11:24:27.083651+0000","last_fullsized":"2023-10-16T11:24:27.083651+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:26.054702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.264563+0000","last_change":"2023-10-16T11:24:26.099039+0000","last_active":"2023-10-16T11:24:29.264563+0000","last_peered":"2023-10-16T11:24:29.264563+0000","last_clean":"2023-10-16T11:24:29.264563+0000","last_became_active":"2023-10-16T11:24:26.097980+0000","last_became_peered":"2023-10-16T11:24:26.097980+0000","last_unstale":"2023-10-16T11:24:29.264563+0000","last_undegraded":"2023-10-16T11:24:29.264563+0000","last_fullsized":"2023-10-16T11:24:29.264563+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:40.585267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.889517+0000","last_change":"2023-10-16T11:24:38.889517+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.889517+0000","last_undegraded":"2023-10-16T11:24:38.889517+0000","last_fullsized":"2023-10-16T11:24:38.889517+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.14","version":"217'8","reported_seq":21,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:35.685595+0000","last_change":"2023-10-16T11:24:25.095357+0000","last_active":"2023-10-16T11:24:35.685595+0000","last_peered":"2023-10-16T11:24:35.685595+0000","last_clean":"2023-10-16T11:24:35.685595+0000","last_became_active":"2023-10-16T11:24:25.093832+0000","last_became_peered":"2023-10-16T11:24:25.093832+0000","last_unstale":"2023-10-16T11:24:35.685595+0000","last_undegraded":"2023-10-16T11:24:35.685595+0000","last_fullsized":"2023-10-16T11:24:35.685595+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:14:19.482469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918256+0000","last_change":"2023-10-16T11:24:26.092185+0000","last_active":"2023-10-16T11:24:40.918256+0000","last_peered":"2023-10-16T11:24:40.918256+0000","last_clean":"2023-10-16T11:24:40.918256+0000","last_became_active":"2023-10-16T11:24:26.091648+0000","last_became_peered":"2023-10-16T11:24:26.091648+0000","last_unstale":"2023-10-16T11:24:40.918256+0000","last_undegraded":"2023-10-16T11:24:40.918256+0000","last_fullsized":"2023-10-16T11:24:40.918256+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918129+0000","last_change":"2023-10-16T11:24:39.919312+0000","last_active":"2023-10-16T11:24:40.918129+0000","last_peered":"2023-10-16T11:24:40.918129+0000","last_clean":"2023-10-16T11:24:40.918129+0000","last_became_active":"2023-10-16T11:24:39.918986+0000","last_became_peered":"2023-10-16T11:24:39.918986+0000","last_unstale":"2023-10-16T11:24:40.918129+0000","last_undegraded":"2023-10-16T11:24:40.918129+0000","last_fullsized":"2023-10-16T11:24:40.918129+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:54.385890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072402+0000","last_change":"2023-10-16T11:24:40.925767+0000","last_active":"2023-10-16T11:24:42.072402+0000","last_peered":"2023-10-16T11:24:42.072402+0000","last_clean":"2023-10-16T11:24:42.072402+0000","last_became_active":"2023-10-16T11:24:40.925501+0000","last_became_peered":"2023-10-16T11:24:40.925501+0000","last_unstale":"2023-10-16T11:24:42.072402+0000","last_undegraded":"2023-10-16T11:24:42.072402+0000","last_fullsized":"2023-10-16T11:24:42.072402+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:49:08.490072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.15","version":"216'3","reported_seq":16,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.083002+0000","last_change":"2023-10-16T11:24:25.082968+0000","last_active":"2023-10-16T11:24:27.083002+0000","last_peered":"2023-10-16T11:24:27.083002+0000","last_clean":"2023-10-16T11:24:27.083002+0000","last_became_active":"2023-10-16T11:24:25.082316+0000","last_became_peered":"2023-10-16T11:24:25.082316+0000","last_unstale":"2023-10-16T11:24:27.083002+0000","last_undegraded":"2023-10-16T11:24:27.083002+0000","last_fullsized":"2023-10-16T11:24:27.083002+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:59.924424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.259843+0000","last_change":"2023-10-16T11:24:26.084487+0000","last_active":"2023-10-16T11:24:29.259843+0000","last_peered":"2023-10-16T11:24:29.259843+0000","last_clean":"2023-10-16T11:24:29.259843+0000","last_became_active":"2023-10-16T11:24:26.084188+0000","last_became_peered":"2023-10-16T11:24:26.084188+0000","last_unstale":"2023-10-16T11:24:29.259843+0000","last_undegraded":"2023-10-16T11:24:29.259843+0000","last_fullsized":"2023-10-16T11:24:29.259843+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:47.400027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.889936+0000","last_change":"2023-10-16T11:24:38.889936+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.889936+0000","last_undegraded":"2023-10-16T11:24:38.889936+0000","last_fullsized":"2023-10-16T11:24:38.889936+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.084335+0000","last_change":"2023-10-16T11:24:25.092100+0000","last_active":"2023-10-16T11:24:27.084335+0000","last_peered":"2023-10-16T11:24:27.084335+0000","last_clean":"2023-10-16T11:24:27.084335+0000","last_became_active":"2023-10-16T11:24:25.091237+0000","last_became_peered":"2023-10-16T11:24:25.091237+0000","last_unstale":"2023-10-16T11:24:27.084335+0000","last_undegraded":"2023-10-16T11:24:27.084335+0000","last_fullsized":"2023-10-16T11:24:27.084335+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:21.985839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.258100+0000","last_change":"2023-10-16T11:24:26.081980+0000","last_active":"2023-10-16T11:24:29.258100+0000","last_peered":"2023-10-16T11:24:29.258100+0000","last_clean":"2023-10-16T11:24:29.258100+0000","last_became_active":"2023-10-16T11:24:26.081395+0000","last_became_peered":"2023-10-16T11:24:26.081395+0000","last_unstale":"2023-10-16T11:24:29.258100+0000","last_undegraded":"2023-10-16T11:24:29.258100+0000","last_fullsized":"2023-10-16T11:24:29.258100+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:39.802117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.893232+0000","last_change":"2023-10-16T11:24:38.893232+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.893232+0000","last_undegraded":"2023-10-16T11:24:38.893232+0000","last_fullsized":"2023-10-16T11:24:38.893232+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.070986+0000","last_change":"2023-10-16T11:24:25.094384+0000","last_active":"2023-10-16T11:24:28.070986+0000","last_peered":"2023-10-16T11:24:28.070986+0000","last_clean":"2023-10-16T11:24:28.070986+0000","last_became_active":"2023-10-16T11:24:25.094026+0000","last_became_peered":"2023-10-16T11:24:25.094026+0000","last_unstale":"2023-10-16T11:24:28.070986+0000","last_undegraded":"2023-10-16T11:24:28.070986+0000","last_fullsized":"2023-10-16T11:24:28.070986+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:51.349970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.259941+0000","last_change":"2023-10-16T11:24:26.082915+0000","last_active":"2023-10-16T11:24:29.259941+0000","last_peered":"2023-10-16T11:24:29.259941+0000","last_clean":"2023-10-16T11:24:29.259941+0000","last_became_active":"2023-10-16T11:24:26.082458+0000","last_became_peered":"2023-10-16T11:24:26.082458+0000","last_unstale":"2023-10-16T11:24:29.259941+0000","last_undegraded":"2023-10-16T11:24:29.259941+0000","last_fullsized":"2023-10-16T11:24:29.259941+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:10.387018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.890075+0000","last_change":"2023-10-16T11:24:38.890075+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.890075+0000","last_undegraded":"2023-10-16T11:24:38.890075+0000","last_fullsized":"2023-10-16T11:24:38.890075+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.894457+0000","last_change":"2023-10-16T11:24:38.894457+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.894457+0000","last_undegraded":"2023-10-16T11:24:38.894457+0000","last_fullsized":"2023-10-16T11:24:38.894457+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.083169+0000","last_change":"2023-10-16T11:24:25.084521+0000","last_active":"2023-10-16T11:24:27.083169+0000","last_peered":"2023-10-16T11:24:27.083169+0000","last_clean":"2023-10-16T11:24:27.083169+0000","last_became_active":"2023-10-16T11:24:25.084003+0000","last_became_peered":"2023-10-16T11:24:25.084003+0000","last_unstale":"2023-10-16T11:24:27.083169+0000","last_undegraded":"2023-10-16T11:24:27.083169+0000","last_fullsized":"2023-10-16T11:24:27.083169+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:36.060066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.259652+0000","last_change":"2023-10-16T11:24:26.091641+0000","last_active":"2023-10-16T11:24:29.259652+0000","last_peered":"2023-10-16T11:24:29.259652+0000","last_clean":"2023-10-16T11:24:29.259652+0000","last_became_active":"2023-10-16T11:24:26.089186+0000","last_became_peered":"2023-10-16T11:24:26.089186+0000","last_unstale":"2023-10-16T11:24:29.259652+0000","last_undegraded":"2023-10-16T11:24:29.259652+0000","last_fullsized":"2023-10-16T11:24:29.259652+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:32.247878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.1f","version":"220'4","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.947367+0000","last_change":"2023-10-16T11:24:39.915794+0000","last_active":"2023-10-16T11:24:40.947367+0000","last_peered":"2023-10-16T11:24:40.947367+0000","last_clean":"2023-10-16T11:24:40.947367+0000","last_became_active":"2023-10-16T11:24:39.915438+0000","last_became_peered":"2023-10-16T11:24:39.915438+0000","last_unstale":"2023-10-16T11:24:40.947367+0000","last_undegraded":"2023-10-16T11:24:40.947367+0000","last_fullsized":"2023-10-16T11:24:40.947367+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:59.121334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.068765+0000","last_change":"2023-10-16T11:24:25.074647+0000","last_active":"2023-10-16T11:24:28.068765+0000","last_peered":"2023-10-16T11:24:28.068765+0000","last_clean":"2023-10-16T11:24:28.068765+0000","last_became_active":"2023-10-16T11:24:25.074373+0000","last_became_peered":"2023-10-16T11:24:25.074373+0000","last_unstale":"2023-10-16T11:24:28.068765+0000","last_undegraded":"2023-10-16T11:24:28.068765+0000","last_fullsized":"2023-10-16T11:24:28.068765+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:38.544520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.264495+0000","last_change":"2023-10-16T11:24:26.088785+0000","last_active":"2023-10-16T11:24:29.264495+0000","last_peered":"2023-10-16T11:24:29.264495+0000","last_clean":"2023-10-16T11:24:29.264495+0000","last_became_active":"2023-10-16T11:24:26.088521+0000","last_became_peered":"2023-10-16T11:24:26.088521+0000","last_unstale":"2023-10-16T11:24:29.264495+0000","last_undegraded":"2023-10-16T11:24:29.264495+0000","last_fullsized":"2023-10-16T11:24:29.264495+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:29.145795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1a","version":"216'1","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.071147+0000","last_change":"2023-10-16T11:24:25.084983+0000","last_active":"2023-10-16T11:24:28.071147+0000","last_peered":"2023-10-16T11:24:28.071147+0000","last_clean":"2023-10-16T11:24:28.071147+0000","last_became_active":"2023-10-16T11:24:25.084622+0000","last_became_peered":"2023-10-16T11:24:25.084622+0000","last_unstale":"2023-10-16T11:24:28.071147+0000","last_undegraded":"2023-10-16T11:24:28.071147+0000","last_fullsized":"2023-10-16T11:24:28.071147+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:36.588809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.259739+0000","last_change":"2023-10-16T11:24:26.089982+0000","last_active":"2023-10-16T11:24:29.259739+0000","last_peered":"2023-10-16T11:24:29.259739+0000","last_clean":"2023-10-16T11:24:29.259739+0000","last_became_active":"2023-10-16T11:24:26.089677+0000","last_became_peered":"2023-10-16T11:24:26.089677+0000","last_unstale":"2023-10-16T11:24:29.259739+0000","last_undegraded":"2023-10-16T11:24:29.259739+0000","last_fullsized":"2023-10-16T11:24:29.259739+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:22.009042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.893762+0000","last_change":"2023-10-16T11:24:38.893762+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.893762+0000","last_undegraded":"2023-10-16T11:24:38.893762+0000","last_fullsized":"2023-10-16T11:24:38.893762+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1c","version":"216'2","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:28.069779+0000","last_change":"2023-10-16T11:24:25.095179+0000","last_active":"2023-10-16T11:24:28.069779+0000","last_peered":"2023-10-16T11:24:28.069779+0000","last_clean":"2023-10-16T11:24:28.069779+0000","last_became_active":"2023-10-16T11:24:25.094714+0000","last_became_peered":"2023-10-16T11:24:25.094714+0000","last_unstale":"2023-10-16T11:24:28.069779+0000","last_undegraded":"2023-10-16T11:24:28.069779+0000","last_fullsized":"2023-10-16T11:24:28.069779+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:26.128614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.262865+0000","last_change":"2023-10-16T11:24:26.096297+0000","last_active":"2023-10-16T11:24:29.262865+0000","last_peered":"2023-10-16T11:24:29.262865+0000","last_clean":"2023-10-16T11:24:29.262865+0000","last_became_active":"2023-10-16T11:24:26.095538+0000","last_became_peered":"2023-10-16T11:24:26.095538+0000","last_unstale":"2023-10-16T11:24:29.262865+0000","last_undegraded":"2023-10-16T11:24:29.262865+0000","last_fullsized":"2023-10-16T11:24:29.262865+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:47.400027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.901826+0000","last_change":"2023-10-16T11:24:38.901826+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.901826+0000","last_undegraded":"2023-10-16T11:24:38.901826+0000","last_fullsized":"2023-10-16T11:24:38.901826+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1d","version":"216'2","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.080963+0000","last_change":"2023-10-16T11:24:25.081731+0000","last_active":"2023-10-16T11:24:27.080963+0000","last_peered":"2023-10-16T11:24:27.080963+0000","last_clean":"2023-10-16T11:24:27.080963+0000","last_became_active":"2023-10-16T11:24:25.081336+0000","last_became_peered":"2023-10-16T11:24:25.081336+0000","last_unstale":"2023-10-16T11:24:27.080963+0000","last_undegraded":"2023-10-16T11:24:27.080963+0000","last_fullsized":"2023-10-16T11:24:27.080963+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:09.330801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918704+0000","last_change":"2023-10-16T11:24:26.089791+0000","last_active":"2023-10-16T11:24:40.918704+0000","last_peered":"2023-10-16T11:24:40.918704+0000","last_clean":"2023-10-16T11:24:40.918704+0000","last_became_active":"2023-10-16T11:24:26.089516+0000","last_became_peered":"2023-10-16T11:24:26.089516+0000","last_unstale":"2023-10-16T11:24:40.918704+0000","last_undegraded":"2023-10-16T11:24:40.918704+0000","last_fullsized":"2023-10-16T11:24:40.918704+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.890428+0000","last_change":"2023-10-16T11:24:38.890428+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.890428+0000","last_undegraded":"2023-10-16T11:24:38.890428+0000","last_fullsized":"2023-10-16T11:24:38.890428+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1e","version":"216'1","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.201161+0000","last_change":"2023-10-16T11:24:25.094483+0000","last_active":"2023-10-16T11:24:27.201161+0000","last_peered":"2023-10-16T11:24:27.201161+0000","last_clean":"2023-10-16T11:24:27.201161+0000","last_became_active":"2023-10-16T11:24:25.094134+0000","last_became_peered":"2023-10-16T11:24:25.094134+0000","last_unstale":"2023-10-16T11:24:27.201161+0000","last_undegraded":"2023-10-16T11:24:27.201161+0000","last_fullsized":"2023-10-16T11:24:27.201161+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.256450+0000","last_change":"2023-10-16T11:24:26.094536+0000","last_active":"2023-10-16T11:24:29.256450+0000","last_peered":"2023-10-16T11:24:29.256450+0000","last_clean":"2023-10-16T11:24:29.256450+0000","last_became_active":"2023-10-16T11:24:26.094272+0000","last_became_peered":"2023-10-16T11:24:26.094272+0000","last_unstale":"2023-10-16T11:24:29.256450+0000","last_undegraded":"2023-10-16T11:24:29.256450+0000","last_fullsized":"2023-10-16T11:24:29.256450+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.894744+0000","last_change":"2023-10-16T11:24:38.894744+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.894744+0000","last_undegraded":"2023-10-16T11:24:38.894744+0000","last_fullsized":"2023-10-16T11:24:38.894744+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1f","version":"216'4","reported_seq":17,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:27.976699+0000","last_change":"2023-10-16T11:24:25.096644+0000","last_active":"2023-10-16T11:24:27.976699+0000","last_peered":"2023-10-16T11:24:27.976699+0000","last_clean":"2023-10-16T11:24:27.976699+0000","last_became_active":"2023-10-16T11:24:25.096306+0000","last_became_peered":"2023-10-16T11:24:25.096306+0000","last_unstale":"2023-10-16T11:24:27.976699+0000","last_undegraded":"2023-10-16T11:24:27.976699+0000","last_fullsized":"2023-10-16T11:24:27.976699+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:22.618393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255563+0000","last_change":"2023-10-16T11:24:26.069227+0000","last_active":"2023-10-16T11:24:29.255563+0000","last_peered":"2023-10-16T11:24:29.255563+0000","last_clean":"2023-10-16T11:24:29.255563+0000","last_became_active":"2023-10-16T11:24:26.068456+0000","last_became_peered":"2023-10-16T11:24:26.068456+0000","last_unstale":"2023-10-16T11:24:29.255563+0000","last_undegraded":"2023-10-16T11:24:29.255563+0000","last_fullsized":"2023-10-16T11:24:29.255563+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":1,"reported_epoch":218,"state":"creating+peering","last_fresh":"2023-10-16T11:24:38.893285+0000","last_change":"2023-10-16T11:24:38.893285+0000","last_active":"2023-10-16T11:24:38.863202+0000","last_peered":"2023-10-16T11:24:38.863202+0000","last_clean":"2023-10-16T11:24:38.863202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:38.893285+0000","last_undegraded":"2023-10-16T11:24:38.893285+0000","last_fullsized":"2023-10-16T11:24:38.893285+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.916906+0000","last_change":"2023-10-16T11:24:26.085375+0000","last_active":"2023-10-16T11:24:40.916906+0000","last_peered":"2023-10-16T11:24:40.916906+0000","last_clean":"2023-10-16T11:24:40.916906+0000","last_became_active":"2023-10-16T11:24:26.085120+0000","last_became_peered":"2023-10-16T11:24:26.085120+0000","last_unstale":"2023-10-16T11:24:40.916906+0000","last_undegraded":"2023-10-16T11:24:40.916906+0000","last_fullsized":"2023-10-16T11:24:40.916906+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.262693+0000","last_change":"2023-10-16T11:24:26.084964+0000","last_active":"2023-10-16T11:24:29.262693+0000","last_peered":"2023-10-16T11:24:29.262693+0000","last_clean":"2023-10-16T11:24:29.262693+0000","last_became_active":"2023-10-16T11:24:26.084006+0000","last_became_peered":"2023-10-16T11:24:26.084006+0000","last_unstale":"2023-10-16T11:24:29.262693+0000","last_undegraded":"2023-10-16T11:24:29.262693+0000","last_fullsized":"2023-10-16T11:24:29.262693+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:03.949526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255428+0000","last_change":"2023-10-16T11:24:26.086008+0000","last_active":"2023-10-16T11:24:29.255428+0000","last_peered":"2023-10-16T11:24:29.255428+0000","last_clean":"2023-10-16T11:24:29.255428+0000","last_became_active":"2023-10-16T11:24:26.085598+0000","last_became_peered":"2023-10-16T11:24:26.085598+0000","last_unstale":"2023-10-16T11:24:29.255428+0000","last_undegraded":"2023-10-16T11:24:29.255428+0000","last_fullsized":"2023-10-16T11:24:29.255428+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.254634+0000","last_change":"2023-10-16T11:24:26.086172+0000","last_active":"2023-10-16T11:24:29.254634+0000","last_peered":"2023-10-16T11:24:29.254634+0000","last_clean":"2023-10-16T11:24:29.254634+0000","last_became_active":"2023-10-16T11:24:26.085882+0000","last_became_peered":"2023-10-16T11:24:26.085882+0000","last_unstale":"2023-10-16T11:24:29.254634+0000","last_undegraded":"2023-10-16T11:24:29.254634+0000","last_fullsized":"2023-10-16T11:24:29.254634+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:44.265192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.257913+0000","last_change":"2023-10-16T11:24:26.086228+0000","last_active":"2023-10-16T11:24:29.257913+0000","last_peered":"2023-10-16T11:24:29.257913+0000","last_clean":"2023-10-16T11:24:29.257913+0000","last_became_active":"2023-10-16T11:24:26.085912+0000","last_became_peered":"2023-10-16T11:24:26.085912+0000","last_unstale":"2023-10-16T11:24:29.257913+0000","last_undegraded":"2023-10-16T11:24:29.257913+0000","last_fullsized":"2023-10-16T11:24:29.257913+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.254833+0000","last_change":"2023-10-16T11:24:26.088015+0000","last_active":"2023-10-16T11:24:29.254833+0000","last_peered":"2023-10-16T11:24:29.254833+0000","last_clean":"2023-10-16T11:24:29.254833+0000","last_became_active":"2023-10-16T11:24:26.086932+0000","last_became_peered":"2023-10-16T11:24:26.086932+0000","last_unstale":"2023-10-16T11:24:29.254833+0000","last_undegraded":"2023-10-16T11:24:29.254833+0000","last_fullsized":"2023-10-16T11:24:29.254833+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:44.137821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.071053+0000","last_change":"2023-10-16T11:24:40.935550+0000","last_active":"2023-10-16T11:24:42.071053+0000","last_peered":"2023-10-16T11:24:42.071053+0000","last_clean":"2023-10-16T11:24:42.071053+0000","last_became_active":"2023-10-16T11:24:40.935237+0000","last_became_peered":"2023-10-16T11:24:40.935237+0000","last_unstale":"2023-10-16T11:24:42.071053+0000","last_undegraded":"2023-10-16T11:24:42.071053+0000","last_fullsized":"2023-10-16T11:24:42.071053+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:13.011085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918841+0000","last_change":"2023-10-16T11:24:26.095913+0000","last_active":"2023-10-16T11:24:40.918841+0000","last_peered":"2023-10-16T11:24:40.918841+0000","last_clean":"2023-10-16T11:24:40.918841+0000","last_became_active":"2023-10-16T11:24:26.093113+0000","last_became_peered":"2023-10-16T11:24:26.093113+0000","last_unstale":"2023-10-16T11:24:40.918841+0000","last_undegraded":"2023-10-16T11:24:40.918841+0000","last_fullsized":"2023-10-16T11:24:40.918841+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255633+0000","last_change":"2023-10-16T11:24:26.094066+0000","last_active":"2023-10-16T11:24:29.255633+0000","last_peered":"2023-10-16T11:24:29.255633+0000","last_clean":"2023-10-16T11:24:29.255633+0000","last_became_active":"2023-10-16T11:24:26.093428+0000","last_became_peered":"2023-10-16T11:24:26.093428+0000","last_unstale":"2023-10-16T11:24:29.255633+0000","last_undegraded":"2023-10-16T11:24:29.255633+0000","last_fullsized":"2023-10-16T11:24:29.255633+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.259561+0000","last_change":"2023-10-16T11:24:26.091894+0000","last_active":"2023-10-16T11:24:29.259561+0000","last_peered":"2023-10-16T11:24:29.259561+0000","last_clean":"2023-10-16T11:24:29.259561+0000","last_became_active":"2023-10-16T11:24:26.091627+0000","last_became_peered":"2023-10-16T11:24:26.091627+0000","last_unstale":"2023-10-16T11:24:29.259561+0000","last_undegraded":"2023-10-16T11:24:29.259561+0000","last_fullsized":"2023-10-16T11:24:29.259561+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:46:59.832609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.256300+0000","last_change":"2023-10-16T11:24:26.097221+0000","last_active":"2023-10-16T11:24:29.256300+0000","last_peered":"2023-10-16T11:24:29.256300+0000","last_clean":"2023-10-16T11:24:29.256300+0000","last_became_active":"2023-10-16T11:24:26.096572+0000","last_became_peered":"2023-10-16T11:24:26.096572+0000","last_unstale":"2023-10-16T11:24:29.256300+0000","last_undegraded":"2023-10-16T11:24:29.256300+0000","last_fullsized":"2023-10-16T11:24:29.256300+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.256364+0000","last_change":"2023-10-16T11:24:26.101039+0000","last_active":"2023-10-16T11:24:29.256364+0000","last_peered":"2023-10-16T11:24:29.256364+0000","last_clean":"2023-10-16T11:24:29.256364+0000","last_became_active":"2023-10-16T11:24:26.100757+0000","last_became_peered":"2023-10-16T11:24:26.100757+0000","last_unstale":"2023-10-16T11:24:29.256364+0000","last_undegraded":"2023-10-16T11:24:29.256364+0000","last_fullsized":"2023-10-16T11:24:29.256364+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.259697+0000","last_change":"2023-10-16T11:24:26.092718+0000","last_active":"2023-10-16T11:24:29.259697+0000","last_peered":"2023-10-16T11:24:29.259697+0000","last_clean":"2023-10-16T11:24:29.259697+0000","last_became_active":"2023-10-16T11:24:26.091377+0000","last_became_peered":"2023-10-16T11:24:26.091377+0000","last_unstale":"2023-10-16T11:24:29.259697+0000","last_undegraded":"2023-10-16T11:24:29.259697+0000","last_fullsized":"2023-10-16T11:24:29.259697+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:17.072626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072172+0000","last_change":"2023-10-16T11:24:40.933022+0000","last_active":"2023-10-16T11:24:42.072172+0000","last_peered":"2023-10-16T11:24:42.072172+0000","last_clean":"2023-10-16T11:24:42.072172+0000","last_became_active":"2023-10-16T11:24:40.932746+0000","last_became_peered":"2023-10-16T11:24:40.932746+0000","last_unstale":"2023-10-16T11:24:42.072172+0000","last_undegraded":"2023-10-16T11:24:42.072172+0000","last_fullsized":"2023-10-16T11:24:42.072172+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:42.730933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.264637+0000","last_change":"2023-10-16T11:24:26.093168+0000","last_active":"2023-10-16T11:24:29.264637+0000","last_peered":"2023-10-16T11:24:29.264637+0000","last_clean":"2023-10-16T11:24:29.264637+0000","last_became_active":"2023-10-16T11:24:26.091801+0000","last_became_peered":"2023-10-16T11:24:26.091801+0000","last_unstale":"2023-10-16T11:24:29.264637+0000","last_undegraded":"2023-10-16T11:24:29.264637+0000","last_fullsized":"2023-10-16T11:24:29.264637+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:43:28.275746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.262948+0000","last_change":"2023-10-16T11:24:26.086807+0000","last_active":"2023-10-16T11:24:29.262948+0000","last_peered":"2023-10-16T11:24:29.262948+0000","last_clean":"2023-10-16T11:24:29.262948+0000","last_became_active":"2023-10-16T11:24:26.085382+0000","last_became_peered":"2023-10-16T11:24:26.085382+0000","last_unstale":"2023-10-16T11:24:29.262948+0000","last_undegraded":"2023-10-16T11:24:29.262948+0000","last_fullsized":"2023-10-16T11:24:29.262948+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:56.195463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.263076+0000","last_change":"2023-10-16T11:24:26.092139+0000","last_active":"2023-10-16T11:24:29.263076+0000","last_peered":"2023-10-16T11:24:29.263076+0000","last_clean":"2023-10-16T11:24:29.263076+0000","last_became_active":"2023-10-16T11:24:26.091860+0000","last_became_peered":"2023-10-16T11:24:26.091860+0000","last_unstale":"2023-10-16T11:24:29.263076+0000","last_undegraded":"2023-10-16T11:24:29.263076+0000","last_fullsized":"2023-10-16T11:24:29.263076+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:00:31.371733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.919569+0000","last_change":"2023-10-16T11:24:26.086029+0000","last_active":"2023-10-16T11:24:40.919569+0000","last_peered":"2023-10-16T11:24:40.919569+0000","last_clean":"2023-10-16T11:24:40.919569+0000","last_became_active":"2023-10-16T11:24:26.085644+0000","last_became_peered":"2023-10-16T11:24:26.085644+0000","last_unstale":"2023-10-16T11:24:40.919569+0000","last_undegraded":"2023-10-16T11:24:40.919569+0000","last_fullsized":"2023-10-16T11:24:40.919569+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:39.802117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.262637+0000","last_change":"2023-10-16T11:24:26.081938+0000","last_active":"2023-10-16T11:24:29.262637+0000","last_peered":"2023-10-16T11:24:29.262637+0000","last_clean":"2023-10-16T11:24:29.262637+0000","last_became_active":"2023-10-16T11:24:26.081671+0000","last_became_peered":"2023-10-16T11:24:26.081671+0000","last_unstale":"2023-10-16T11:24:29.262637+0000","last_undegraded":"2023-10-16T11:24:29.262637+0000","last_fullsized":"2023-10-16T11:24:29.262637+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:17.072626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917335+0000","last_change":"2023-10-16T11:24:26.091403+0000","last_active":"2023-10-16T11:24:40.917335+0000","last_peered":"2023-10-16T11:24:40.917335+0000","last_clean":"2023-10-16T11:24:40.917335+0000","last_became_active":"2023-10-16T11:24:26.091114+0000","last_became_peered":"2023-10-16T11:24:26.091114+0000","last_unstale":"2023-10-16T11:24:40.917335+0000","last_undegraded":"2023-10-16T11:24:40.917335+0000","last_fullsized":"2023-10-16T11:24:40.917335+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917983+0000","last_change":"2023-10-16T11:24:26.088199+0000","last_active":"2023-10-16T11:24:40.917983+0000","last_peered":"2023-10-16T11:24:40.917983+0000","last_clean":"2023-10-16T11:24:40.917983+0000","last_became_active":"2023-10-16T11:24:26.087892+0000","last_became_peered":"2023-10-16T11:24:26.087892+0000","last_unstale":"2023-10-16T11:24:40.917983+0000","last_undegraded":"2023-10-16T11:24:40.917983+0000","last_fullsized":"2023-10-16T11:24:40.917983+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.254728+0000","last_change":"2023-10-16T11:24:26.088938+0000","last_active":"2023-10-16T11:24:29.254728+0000","last_peered":"2023-10-16T11:24:29.254728+0000","last_clean":"2023-10-16T11:24:29.254728+0000","last_became_active":"2023-10-16T11:24:26.088689+0000","last_became_peered":"2023-10-16T11:24:26.088689+0000","last_unstale":"2023-10-16T11:24:29.254728+0000","last_undegraded":"2023-10-16T11:24:29.254728+0000","last_fullsized":"2023-10-16T11:24:29.254728+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:18:12.479766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.254769+0000","last_change":"2023-10-16T11:24:26.089768+0000","last_active":"2023-10-16T11:24:29.254769+0000","last_peered":"2023-10-16T11:24:29.254769+0000","last_clean":"2023-10-16T11:24:29.254769+0000","last_became_active":"2023-10-16T11:24:26.087028+0000","last_became_peered":"2023-10-16T11:24:26.087028+0000","last_unstale":"2023-10-16T11:24:29.254769+0000","last_undegraded":"2023-10-16T11:24:29.254769+0000","last_fullsized":"2023-10-16T11:24:29.254769+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:33.050118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.258028+0000","last_change":"2023-10-16T11:24:26.077430+0000","last_active":"2023-10-16T11:24:29.258028+0000","last_peered":"2023-10-16T11:24:29.258028+0000","last_clean":"2023-10-16T11:24:29.258028+0000","last_became_active":"2023-10-16T11:24:26.077145+0000","last_became_peered":"2023-10-16T11:24:26.077145+0000","last_unstale":"2023-10-16T11:24:29.258028+0000","last_undegraded":"2023-10-16T11:24:29.258028+0000","last_fullsized":"2023-10-16T11:24:29.258028+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.263034+0000","last_change":"2023-10-16T11:24:26.088202+0000","last_active":"2023-10-16T11:24:29.263034+0000","last_peered":"2023-10-16T11:24:29.263034+0000","last_clean":"2023-10-16T11:24:29.263034+0000","last_became_active":"2023-10-16T11:24:26.087929+0000","last_became_peered":"2023-10-16T11:24:26.087929+0000","last_unstale":"2023-10-16T11:24:29.263034+0000","last_undegraded":"2023-10-16T11:24:29.263034+0000","last_fullsized":"2023-10-16T11:24:29.263034+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:22.009042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.070991+0000","last_change":"2023-10-16T11:24:40.941405+0000","last_active":"2023-10-16T11:24:42.070991+0000","last_peered":"2023-10-16T11:24:42.070991+0000","last_clean":"2023-10-16T11:24:42.070991+0000","last_became_active":"2023-10-16T11:24:40.941060+0000","last_became_peered":"2023-10-16T11:24:40.941060+0000","last_unstale":"2023-10-16T11:24:42.070991+0000","last_undegraded":"2023-10-16T11:24:42.070991+0000","last_fullsized":"2023-10-16T11:24:42.070991+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:22:53.973191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255677+0000","last_change":"2023-10-16T11:24:26.080292+0000","last_active":"2023-10-16T11:24:29.255677+0000","last_peered":"2023-10-16T11:24:29.255677+0000","last_clean":"2023-10-16T11:24:29.255677+0000","last_became_active":"2023-10-16T11:24:26.080004+0000","last_became_peered":"2023-10-16T11:24:26.080004+0000","last_unstale":"2023-10-16T11:24:29.255677+0000","last_undegraded":"2023-10-16T11:24:29.255677+0000","last_fullsized":"2023-10-16T11:24:29.255677+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.264465+0000","last_change":"2023-10-16T11:24:26.098845+0000","last_active":"2023-10-16T11:24:29.264465+0000","last_peered":"2023-10-16T11:24:29.264465+0000","last_clean":"2023-10-16T11:24:29.264465+0000","last_became_active":"2023-10-16T11:24:26.098568+0000","last_became_peered":"2023-10-16T11:24:26.098568+0000","last_unstale":"2023-10-16T11:24:29.264465+0000","last_undegraded":"2023-10-16T11:24:29.264465+0000","last_fullsized":"2023-10-16T11:24:29.264465+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:15.909098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917233+0000","last_change":"2023-10-16T11:24:26.096348+0000","last_active":"2023-10-16T11:24:40.917233+0000","last_peered":"2023-10-16T11:24:40.917233+0000","last_clean":"2023-10-16T11:24:40.917233+0000","last_became_active":"2023-10-16T11:24:26.093712+0000","last_became_peered":"2023-10-16T11:24:26.093712+0000","last_unstale":"2023-10-16T11:24:40.917233+0000","last_undegraded":"2023-10-16T11:24:40.917233+0000","last_fullsized":"2023-10-16T11:24:40.917233+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:40.970357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.264542+0000","last_change":"2023-10-16T11:24:26.083687+0000","last_active":"2023-10-16T11:24:29.264542+0000","last_peered":"2023-10-16T11:24:29.264542+0000","last_clean":"2023-10-16T11:24:29.264542+0000","last_became_active":"2023-10-16T11:24:26.083213+0000","last_became_peered":"2023-10-16T11:24:26.083213+0000","last_unstale":"2023-10-16T11:24:29.264542+0000","last_undegraded":"2023-10-16T11:24:29.264542+0000","last_fullsized":"2023-10-16T11:24:29.264542+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:08:02.966132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.264608+0000","last_change":"2023-10-16T11:24:26.096814+0000","last_active":"2023-10-16T11:24:29.264608+0000","last_peered":"2023-10-16T11:24:29.264608+0000","last_clean":"2023-10-16T11:24:29.264608+0000","last_became_active":"2023-10-16T11:24:26.095777+0000","last_became_peered":"2023-10-16T11:24:26.095777+0000","last_unstale":"2023-10-16T11:24:29.264608+0000","last_undegraded":"2023-10-16T11:24:29.264608+0000","last_fullsized":"2023-10-16T11:24:29.264608+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:08.889514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.259801+0000","last_change":"2023-10-16T11:24:26.091440+0000","last_active":"2023-10-16T11:24:29.259801+0000","last_peered":"2023-10-16T11:24:29.259801+0000","last_clean":"2023-10-16T11:24:29.259801+0000","last_became_active":"2023-10-16T11:24:26.088757+0000","last_became_peered":"2023-10-16T11:24:26.088757+0000","last_unstale":"2023-10-16T11:24:29.259801+0000","last_undegraded":"2023-10-16T11:24:29.259801+0000","last_fullsized":"2023-10-16T11:24:29.259801+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:27.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.257982+0000","last_change":"2023-10-16T11:24:26.085606+0000","last_active":"2023-10-16T11:24:29.257982+0000","last_peered":"2023-10-16T11:24:29.257982+0000","last_clean":"2023-10-16T11:24:29.257982+0000","last_became_active":"2023-10-16T11:24:26.085318+0000","last_became_peered":"2023-10-16T11:24:26.085318+0000","last_unstale":"2023-10-16T11:24:29.257982+0000","last_undegraded":"2023-10-16T11:24:29.257982+0000","last_fullsized":"2023-10-16T11:24:29.257982+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-10-16T11:24:29.255483+0000","last_change":"2023-10-16T11:24:26.089885+0000","last_active":"2023-10-16T11:24:29.255483+0000","last_peered":"2023-10-16T11:24:29.255483+0000","last_clean":"2023-10-16T11:24:29.255483+0000","last_became_active":"2023-10-16T11:24:26.089406+0000","last_became_peered":"2023-10-16T11:24:26.089406+0000","last_unstale":"2023-10-16T11:24:29.255483+0000","last_undegraded":"2023-10-16T11:24:29.255483+0000","last_fullsized":"2023-10-16T11:24:29.255483+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:24:39.878001+0000","last_change":"2023-10-16T11:24:39.878001+0000","last_active":"2023-10-16T11:24:39.878001+0000","last_peered":"2023-10-16T11:24:39.878001+0000","last_clean":"2023-10-16T11:24:39.878001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:24:39.878001+0000","last_undegraded":"2023-10-16T11:24:39.878001+0000","last_fullsized":"2023-10-16T11:24:39.878001+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.919488+0000","last_change":"2023-10-16T11:24:26.086488+0000","last_active":"2023-10-16T11:24:40.919488+0000","last_peered":"2023-10-16T11:24:40.919488+0000","last_clean":"2023-10-16T11:24:40.919488+0000","last_became_active":"2023-10-16T11:24:26.085969+0000","last_became_peered":"2023-10-16T11:24:26.085969+0000","last_unstale":"2023-10-16T11:24:40.919488+0000","last_undegraded":"2023-10-16T11:24:40.919488+0000","last_fullsized":"2023-10-16T11:24:40.919488+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":1},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":581,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2767,"ondisk_log_size":2767,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575988,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52476,"kb_used_data":3252,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319364,"statfs":{"total":96636764160,"available":96583028736,"internally_reserved":0,"allocated":3330048,"data_stored":1913672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.179,"15min":1.2170000000000001},"min":{"1min":0.81499999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.7250000000000001,"5min":2.085,"15min":2.3399999999999999},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.1719999999999999,"15min":1.218},"min":{"1min":0.876,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.8320000000000001},"last":1.097}]},{"osd":1,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2689999999999999,"15min":1.2509999999999999},"min":{"1min":0.90700000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.6479999999999999,"5min":1.913,"15min":1.913},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2430000000000001,"15min":1.2749999999999999},"min":{"1min":0.76700000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6279999999999999,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.67100000000000004}]},{"osd":2,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3140000000000001,"15min":1.3109999999999999},"min":{"1min":1.008,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6579999999999999,"5min":1.8680000000000001,"15min":2.4460000000000002},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.3540000000000001,"15min":1.3640000000000001},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.708,"5min":1.859,"15min":1.974},"last":1.056}]},{"osd":3,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.381,"15min":1.3899999999999999},"min":{"1min":1.149,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7290000000000001,"5min":1.8280000000000001,"15min":1.972},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.45,"5min":1.4430000000000001,"15min":1.4630000000000001},"min":{"1min":1.0860000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.8799999999999999,"5min":2.0680000000000001,"15min":2.577},"last":0.80300000000000005}]},{"osd":4,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.423,"15min":1.417},"min":{"1min":0.98599999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0030000000000001,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.498,"5min":1.391,"15min":1.3899999999999999},"min":{"1min":1.0469999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8280000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.3320000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.585,"15min":1.5960000000000001},"min":{"1min":1.1579999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.946,"5min":2.2309999999999999,"15min":3.3519999999999999},"last":1.484},{"interface":"front","average":{"1min":1.641,"5min":1.514,"15min":1.5089999999999999},"min":{"1min":1.351,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.1600000000000001,"5min":2.3460000000000001,"15min":2.4969999999999999},"last":1.762}]},{"osd":6,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.488,"15min":1.5209999999999999},"min":{"1min":0.97099999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.845,"5min":2.101,"15min":2.7250000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.7210000000000001,"5min":1.6890000000000001,"15min":1.71},"min":{"1min":1.425,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1040000000000001,"5min":2.298,"15min":3.1440000000000001},"last":1.544}]}]},{"osd":6,"up_from":13,"seq":55834575988,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33848,"kb_used_data":2664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337992,"statfs":{"total":96636764160,"available":96602103808,"internally_reserved":0,"allocated":2727936,"data_stored":1321029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.2350000000000001,"15min":1.2250000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.64000000000000001},"max":{"1min":6.298,"5min":6.298,"15min":6.298},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.3380000000000001,"15min":1.2909999999999999},"min":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.66400000000000003},"max":{"1min":8.2650000000000006,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":0.97099999999999997}]},{"osd":1,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2450000000000001,"15min":1.222},"min":{"1min":0.80200000000000005,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":1.038},{"interface":"front","average":{"1min":1.8520000000000001,"5min":1.4510000000000001,"15min":1.3859999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":12.468,"5min":12.468,"15min":12.468},"last":1.333}]},{"osd":2,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.7889999999999999,"5min":1.337,"15min":1.2829999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":11.586,"5min":11.586,"15min":11.586},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.569,"5min":1.3799999999999999,"15min":1.369},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.77500000000000002},"max":{"1min":4.7030000000000003,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.131}]},{"osd":3,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.9239999999999999,"5min":1.452,"15min":1.353},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":12.801,"5min":12.801,"15min":12.801},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.302,"15min":1.284},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":9.1539999999999999,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":0.85599999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.3700000000000001,"15min":1.3220000000000001},"min":{"1min":0.65400000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.2470000000000001},{"interface":"front","average":{"1min":2.141,"5min":1.609,"15min":1.548},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":13.981999999999999,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.5429999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":2.246,"5min":1.5109999999999999,"15min":1.391},"min":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":18.247,"5min":18.247,"15min":18.247},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.8360000000000001,"5min":1.5900000000000001,"15min":1.5629999999999999},"min":{"1min":0.76200000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":6.4039999999999999,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.4870000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":2.0609999999999999,"5min":1.603,"15min":1.5349999999999999},"min":{"1min":0.749,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":12.589,"5min":12.589,"15min":12.589},"last":1.6659999999999999},{"interface":"front","average":{"1min":2.1749999999999998,"5min":1.6739999999999999,"15min":1.595},"min":{"1min":0.78200000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":12.725,"5min":12.725,"15min":12.725},"last":1.4530000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575988,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29696,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342144,"statfs":{"total":96636764160,"available":96606355456,"internally_reserved":0,"allocated":2740224,"data_stored":1327732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2350000000000001,"15min":1.27},"min":{"1min":0.74199999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.583,"5min":2.661,"15min":2.661},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.3029999999999999,"15min":1.349},"min":{"1min":0.95799999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7909999999999999,"5min":1.7949999999999999,"15min":1.865},"last":1.341}]},{"osd":1,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1819999999999999,"15min":1.1950000000000001},"min":{"1min":0.81999999999999995,"5min":0.80800000000000005,"15min":0.79100000000000004},"max":{"1min":1.5009999999999999,"5min":1.7869999999999999,"15min":1.992},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.258,"15min":1.2290000000000001},"min":{"1min":0.84399999999999997,"5min":0.84199999999999997,"15min":0.80800000000000005},"max":{"1min":1.635,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":0.89100000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.264,"15min":1.244},"min":{"1min":0.97899999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.625,"5min":1.952,"15min":1.952},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.329,"5min":1.2949999999999999,"15min":1.3220000000000001},"min":{"1min":0.92400000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":2.1640000000000001},"last":1.3080000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3320000000000001,"15min":1.3340000000000001},"min":{"1min":0.98999999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":3.012,"15min":3.012},"last":1.242},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.429,"15min":1.425},"min":{"1min":1.0900000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":2.2250000000000001},"last":1.1799999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.339,"15min":1.339},"min":{"1min":0.94199999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.6990000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.6799999999999999,"5min":1.5720000000000001,"15min":1.5740000000000001},"min":{"1min":1.353,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.948,"5min":2.903,"15min":2.903},"last":1.456}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4330000000000001,"15min":1.4139999999999999},"min":{"1min":0.873,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8660000000000001,"5min":2.0649999999999999,"15min":2.0979999999999999},"last":1.407},{"interface":"front","average":{"1min":1.51,"5min":1.494,"15min":1.4770000000000001},"min":{"1min":0.95399999999999996,"5min":0.90700000000000003,"15min":0.78400000000000003},"max":{"1min":1.841,"5min":3.073,"15min":3.073},"last":1.216}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.675,"5min":1.5660000000000001,"15min":1.5469999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0779999999999998,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.645,"15min":1.645},"min":{"1min":1.1779999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.1099999999999999,"5min":2.1459999999999999,"15min":2.2930000000000001},"last":1.5649999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575989,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51904,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319936,"statfs":{"total":96636764160,"available":96583614464,"internally_reserved":0,"allocated":2744320,"data_stored":1323408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1519999999999999,"15min":1.1859999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.7290000000000001,"5min":1.792,"15min":1.792},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.218,"15min":1.2350000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.9299999999999999},"last":1.419}]},{"osd":1,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2769999999999999,"15min":1.298},"min":{"1min":0.81799999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.841,"5min":1.841,"15min":1.913},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.236,"15min":1.25},"min":{"1min":0.78500000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.877,"5min":1.877,"15min":1.889},"last":1.018}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.29,"15min":1.3089999999999999},"min":{"1min":1.0369999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.835},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.351,"5min":1.3999999999999999,"15min":1.403},"min":{"1min":0.879,"5min":0.879,"15min":0.872},"max":{"1min":1.8380000000000001,"5min":1.903,"15min":1.9139999999999999},"last":1.2529999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3169999999999999,"15min":1.337},"min":{"1min":0.91100000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.6830000000000001,"5min":1.8280000000000001,"15min":1.849},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4410000000000001,"15min":1.4330000000000001},"min":{"1min":1.145,"5min":1.008,"15min":0.96999999999999997},"max":{"1min":1.7769999999999999,"5min":1.865,"15min":2.0720000000000001},"last":1.1830000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.4810000000000001,"15min":1.4399999999999999},"min":{"1min":1.0149999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9399999999999999,"5min":2.105,"15min":2.105},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.431,"15min":1.415},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.78600000000000003},"max":{"1min":1.8240000000000001,"5min":2.04,"15min":2.04},"last":1.2290000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.5409999999999999,"15min":1.5029999999999999},"min":{"1min":0.94299999999999995,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":3.157,"5min":3.157,"15min":3.157},"last":1.492},{"interface":"front","average":{"1min":1.5,"5min":1.6100000000000001,"15min":1.611},"min":{"1min":1.0940000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0089999999999999,"5min":2.2330000000000001,"15min":2.2770000000000001},"last":1.6040000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.5640000000000001,"15min":1.5660000000000001},"min":{"1min":1.0549999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.855,"5min":2.113,"15min":2.3420000000000001},"last":1.3},{"interface":"front","average":{"1min":1.639,"5min":1.6619999999999999,"15min":1.6399999999999999},"min":{"1min":1.0660000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9630000000000001,"5min":2.1469999999999998,"15min":2.3839999999999999},"last":1.3340000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575988,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47748,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324092,"statfs":{"total":96636764160,"available":96587870208,"internally_reserved":0,"allocated":2736128,"data_stored":1320629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.89400000000000002,"15min":0.90700000000000003},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.478,"5min":1.478,"15min":1.901},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0369999999999999,"15min":1.0720000000000001},"min":{"1min":0.59299999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.391,"5min":1.6180000000000001,"15min":1.6679999999999999},"last":1.0660000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.95599999999999996,"15min":0.98799999999999999},"min":{"1min":0.60799999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.105,"15min":1.133},"min":{"1min":0.48399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.601,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.88100000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0760000000000001,"15min":1.117},"min":{"1min":0.65500000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.524,"5min":1.694,"15min":1.694},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.198,"15min":1.236},"min":{"1min":0.625,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.91400000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3169999999999999,"15min":1.3500000000000001},"min":{"1min":0.83599999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.806,"5min":1.806,"15min":1.873},"last":1.466},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.357,"15min":1.367},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.8819999999999999},"last":1.2689999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.385,"15min":1.401},"min":{"1min":0.80600000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.516},{"interface":"front","average":{"1min":1.419,"5min":1.4059999999999999,"15min":1.444},"min":{"1min":1.151,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.552}]},{"osd":6,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.504,"15min":1.5329999999999999},"min":{"1min":0.876,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":3.9129999999999998,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.657},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5,"15min":1.5369999999999999},"min":{"1min":1.153,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":3.9319999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.4910000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.528,"15min":1.5600000000000001},"min":{"1min":0.95399999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.877,"5min":2.0790000000000002,"15min":2.2679999999999998},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.639,"15min":1.673},"min":{"1min":1.1279999999999999,"5min":1,"15min":1},"max":{"1min":2.052,"5min":2.198,"15min":2.5129999999999999},"last":1.6739999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575987,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51896,"kb_used_data":2660,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319944,"statfs":{"total":96636764160,"available":96583622656,"internally_reserved":0,"allocated":2723840,"data_stored":1320447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.91000000000000003,"15min":0.92200000000000004},"min":{"1min":0.74199999999999999,"5min":0.51900000000000002,"15min":0.5},"max":{"1min":1.532,"5min":1.532,"15min":1.669},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.0569999999999999,"15min":1.0820000000000001},"min":{"1min":0.72499999999999998,"5min":0.47699999999999998,"15min":0.32600000000000001},"max":{"1min":1.496,"5min":1.681,"15min":2.1429999999999998},"last":0.80400000000000005}]},{"osd":1,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.089,"15min":1.097},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.39200000000000002},"max":{"1min":1.415,"5min":1.6779999999999999,"15min":1.978},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1479999999999999,"15min":1.165},"min":{"1min":0.57599999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.448,"5min":1.609,"15min":1.869},"last":1.0920000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.0649999999999999,"15min":1.087},"min":{"1min":0.54600000000000004,"5min":0.437,"15min":0.437},"max":{"1min":1.464,"5min":1.651,"15min":1.8560000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.252,"15min":1.274},"min":{"1min":0.82199999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6120000000000001,"5min":1.7450000000000001,"15min":1.857},"last":1.2509999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3740000000000001,"15min":1.415},"min":{"1min":0.91500000000000004,"5min":0.86599999999999999,"15min":0.73499999999999999},"max":{"1min":1.6910000000000001,"5min":1.8029999999999999,"15min":2.5059999999999998},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3440000000000001,"15min":1.3520000000000001},"min":{"1min":0.97899999999999998,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":1.825,"5min":1.825,"15min":2.0880000000000001},"last":1.407}]},{"osd":5,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.444,"15min":1.464},"min":{"1min":1.0720000000000001,"5min":0.84999999999999998,"15min":0.78500000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2269999999999999},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4550000000000001,"15min":1.4890000000000001},"min":{"1min":1.0509999999999999,"5min":0.95799999999999996,"15min":0.874},"max":{"1min":1.798,"5min":1.931,"15min":2.6259999999999999},"last":1.518}]},{"osd":6,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4119999999999999,"15min":1.411},"min":{"1min":1.0569999999999999,"5min":0.84099999999999997,"15min":0.80500000000000005},"max":{"1min":1.71,"5min":1.9019999999999999,"15min":2.2109999999999999},"last":1.8260000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5,"15min":1.5369999999999999},"min":{"1min":1.1899999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.2909999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.6899999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":1.5489999999999999,"15min":1.5860000000000001},"min":{"1min":1.2290000000000001,"5min":0.995,"15min":0.92500000000000004},"max":{"1min":2.1469999999999998,"5min":2.262,"15min":2.8109999999999999},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.494,"5min":1.4870000000000001,"15min":1.5149999999999999},"min":{"1min":1.242,"5min":0.81599999999999995,"15min":0.80100000000000005},"max":{"1min":1.8180000000000001,"5min":2.2269999999999999,"15min":2.2829999999999999},"last":1.907}]}]},{"osd":1,"up_from":13,"seq":55834575987,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60228,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311612,"statfs":{"total":96636764160,"available":96575090688,"internally_reserved":0,"allocated":2744320,"data_stored":1327206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.91900000000000004,"15min":0.91500000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.575,"5min":1.575,"15min":1.7929999999999999},"last":1.196},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0289999999999999,"15min":1.0580000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.494,"5min":1.5509999999999999,"15min":1.667},"last":1.0229999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.98399999999999999,"15min":1.0189999999999999},"min":{"1min":0.64300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.72},"last":2.093},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.252,"15min":1.276},"min":{"1min":0.69799999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":1.6639999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.1879999999999999,"15min":1.216},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.774,"5min":1.774,"15min":1.79},"last":1.601},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.169,"15min":1.1739999999999999},"min":{"1min":0.66700000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.633,"5min":1.633,"15min":1.802},"last":1.0880000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.2609999999999999,"15min":1.274},"min":{"1min":0.86299999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.653,"5min":1.887,"15min":1.9159999999999999},"last":3.7759999999999998},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.409,"15min":1.415},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.887,"5min":2.0619999999999998,"15min":2.093},"last":2.3519999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.379,"15min":1.409},"min":{"1min":0.93700000000000006,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.948,"5min":2.0009999999999999,"15min":2.0640000000000001},"last":1.756},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4239999999999999,"15min":1.421},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":1.825,"5min":1.825,"15min":2.0649999999999999},"last":2.5299999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4850000000000001,"15min":1.496},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.879,"5min":2.1970000000000001,"15min":2.2949999999999999},"last":2.1960000000000002},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4710000000000001,"15min":1.486},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.891,"5min":1.891,"15min":2.1970000000000001},"last":3.8799999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5649999999999999,"15min":1.5620000000000001},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.1720000000000002},"last":4.46},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.625,"15min":1.607},"min":{"1min":1.1080000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.9359999999999999,"5min":2.1989999999999998,"15min":2.2810000000000001},"last":2.5960000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575989,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52532,"kb_used_data":3300,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319308,"statfs":{"total":96636764160,"available":96582971392,"internally_reserved":0,"allocated":3379200,"data_stored":1940312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.871,"15min":0.90000000000000002},"min":{"1min":0.73199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.502,"5min":1.52,"15min":1.619},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.1060000000000001},"min":{"1min":0.627,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.621,"5min":1.6399999999999999,"15min":1.76},"last":1.109}]},{"osd":2,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.097,"15min":1.0860000000000001},"min":{"1min":0.65300000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.556,"5min":1.649,"15min":1.649},"last":1.21},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.095,"15min":1.1459999999999999},"min":{"1min":0.81100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4179999999999999,"5min":1.623,"15min":1.6870000000000001},"last":1.508}]},{"osd":3,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.056,"15min":1.085},"min":{"1min":0.69499999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.244,"15min":1.256},"min":{"1min":0.91500000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.716,"5min":1.7609999999999999,"15min":1.9830000000000001},"last":1.456}]},{"osd":4,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.387,"15min":1.389},"min":{"1min":1.129,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.681,"5min":1.9390000000000001,"15min":1.95},"last":1.611},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3180000000000001,"15min":1.3500000000000001},"min":{"1min":1.0329999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.645,"5min":2.1539999999999999,"15min":2.4820000000000002},"last":1.252}]},{"osd":5,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3660000000000001,"15min":1.3879999999999999},"min":{"1min":0.996,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":1.829,"5min":2.0750000000000002,"15min":2.6669999999999998},"last":1.401},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.454},"min":{"1min":1.028,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.748,"5min":2.2029999999999998,"15min":2.2040000000000002},"last":1.639}]},{"osd":6,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.4750000000000001,"15min":1.5029999999999999},"min":{"1min":1.1080000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.819,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.405,"5min":1.4630000000000001,"15min":1.492},"min":{"1min":0.93799999999999994,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.758,"5min":2.262,"15min":2.262},"last":1.7170000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.52,"15min":1.5549999999999999},"min":{"1min":1.2749999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7150000000000001,"5min":2.347,"15min":2.347},"last":1.794},{"interface":"front","average":{"1min":1.454,"5min":1.474,"15min":1.5109999999999999},"min":{"1min":1.129,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.9530000000000001,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.49}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:24:45.918 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:24:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.162+0000 7f6100ff0640 1 Processor -- start 2023-10-16T11:24:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.162+0000 7f6100ff0640 1 -- start start 2023-10-16T11:24:46.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.163+0000 7f6100ff0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f60fc0adb50 0x7f60fc06f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:46.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.164+0000 7f6100ff0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f60fc070fd0 0x7f60fc0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:46.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.164+0000 7f6100ff0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60fc0adf30 0x7f60fc0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:46.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.164+0000 7f6100ff0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60fc071450 con 0x7f60fc0adb50 2023-10-16T11:24:46.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.164+0000 7f6100ff0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60fc0cc600 con 0x7f60fc070fd0 2023-10-16T11:24:46.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.164+0000 7f6100ff0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc0cc8e0 con 0x7f60fc0adf30 2023-10-16T11:24:46.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.164+0000 7f60fa59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f60fc0adb50 0x7f60fc06f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:46.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.164+0000 7f60fa59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f60fc0adb50 0x7f60fc06f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51276/0 (socket says 172.21.15.84:51276) 2023-10-16T11:24:46.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.164+0000 7f60fa59c640 1 -- 172.21.15.84:0/3368180349 learned_addr learned my addr 172.21.15.84:0/3368180349 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:46.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.164+0000 7f60f9d9b640 1 --2- 172.21.15.84:0/3368180349 >> v2:172.21.15.157:3300/0 conn(0x7f60fc070fd0 0x7f60fc0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:46.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.164+0000 7f60fad9d640 1 --2- 172.21.15.84:0/3368180349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60fc0adf30 0x7f60fc0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:46.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.165+0000 7f60fa59c640 1 -- 172.21.15.84:0/3368180349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60fc0adf30 msgr2=0x7f60fc0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:46.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.165+0000 7f60fa59c640 1 --2- 172.21.15.84:0/3368180349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60fc0adf30 0x7f60fc0b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.165+0000 7f60fa59c640 1 -- 172.21.15.84:0/3368180349 >> v2:172.21.15.157:3300/0 conn(0x7f60fc070fd0 msgr2=0x7f60fc0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:46.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.165+0000 7f60fa59c640 1 --2- 172.21.15.84:0/3368180349 >> v2:172.21.15.157:3300/0 conn(0x7f60fc070fd0 0x7f60fc0b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.165+0000 7f60fa59c640 1 -- 172.21.15.84:0/3368180349 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60fc0ccbc0 con 0x7f60fc0adb50 2023-10-16T11:24:46.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.166+0000 7f60fa59c640 1 --2- 172.21.15.84:0/3368180349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60fc0adb50 0x7f60fc06f550 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0cc2b0 tx=0x7f60ec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b209e7178820971b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:46.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.166+0000 7f60f959a640 1 -- 172.21.15.84:0/3368180349 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec02ca70 con 0x7f60fc0adb50 2023-10-16T11:24:46.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.167+0000 7f60f959a640 1 -- 172.21.15.84:0/3368180349 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec02cbd0 con 0x7f60fc0adb50 2023-10-16T11:24:46.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.167+0000 7f60f959a640 1 -- 172.21.15.84:0/3368180349 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec0343f0 con 0x7f60fc0adb50 2023-10-16T11:24:46.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.167+0000 7f6100ff0640 1 -- 172.21.15.84:0/3368180349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60fc0adb50 msgr2=0x7f60fc06f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:46.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.167+0000 7f6100ff0640 1 --2- 172.21.15.84:0/3368180349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60fc0adb50 0x7f60fc06f550 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0cc2b0 tx=0x7f60ec02bed0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.167+0000 7f6100ff0640 1 -- 172.21.15.84:0/3368180349 shutdown_connections 2023-10-16T11:24:46.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.167+0000 7f6100ff0640 1 --2- 172.21.15.84:0/3368180349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60fc0adf30 0x7f60fc0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.167+0000 7f6100ff0640 1 --2- 172.21.15.84:0/3368180349 >> v2:172.21.15.157:3300/0 conn(0x7f60fc070fd0 0x7f60fc0b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.167+0000 7f6100ff0640 1 --2- 172.21.15.84:0/3368180349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60fc0adb50 0x7f60fc06f550 unknown :-1 s=CLOSED pgs=2686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.167+0000 7f6100ff0640 1 -- 172.21.15.84:0/3368180349 >> 172.21.15.84:0/3368180349 conn(0x7f60fc0cb410 msgr2=0x7f60fc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:46.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.167+0000 7f6100ff0640 1 -- 172.21.15.84:0/3368180349 shutdown_connections 2023-10-16T11:24:46.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.167+0000 7f6100ff0640 1 -- 172.21.15.84:0/3368180349 wait complete. 2023-10-16T11:24:46.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.169+0000 7f6100ff0640 1 Processor -- start 2023-10-16T11:24:46.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.169+0000 7f6100ff0640 1 -- start start 2023-10-16T11:24:46.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.170+0000 7f6100ff0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60fc070fd0 0x7f60fc150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:46.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.170+0000 7f6100ff0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60fc0adb50 0x7f60fc0bd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:46.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.170+0000 7f6100ff0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60fc0adf30 0x7f60fc1631b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:46.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.170+0000 7f6100ff0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc0b4760 con 0x7f60fc0adb50 2023-10-16T11:24:46.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.170+0000 7f6100ff0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc0b4480 con 0x7f60fc0adf30 2023-10-16T11:24:46.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.170+0000 7f6100ff0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc0c08d0 con 0x7f60fc070fd0 2023-10-16T11:24:46.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.170+0000 7f60f9d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60fc0adb50 0x7f60fc0bd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:46.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.171+0000 7f60f9d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60fc0adb50 0x7f60fc0bd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51278/0 (socket says 172.21.15.84:51278) 2023-10-16T11:24:46.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.171+0000 7f60f9d9b640 1 -- 172.21.15.84:0/3185839261 learned_addr learned my addr 172.21.15.84:0/3185839261 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:46.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.171+0000 7f60fa59c640 1 --2- 172.21.15.84:0/3185839261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60fc070fd0 0x7f60fc150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:46.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.171+0000 7f60f9d9b640 1 -- 172.21.15.84:0/3185839261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60fc070fd0 msgr2=0x7f60fc150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:46.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.171+0000 7f60fad9d640 1 --2- 172.21.15.84:0/3185839261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60fc0adf30 0x7f60fc1631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:46.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.171+0000 7f60f9d9b640 1 --2- 172.21.15.84:0/3185839261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60fc070fd0 0x7f60fc150400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.171+0000 7f60f9d9b640 1 -- 172.21.15.84:0/3185839261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60fc0adf30 msgr2=0x7f60fc1631b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:46.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.171+0000 7f60f9d9b640 1 --2- 172.21.15.84:0/3185839261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60fc0adf30 0x7f60fc1631b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.171+0000 7f60f9d9b640 1 -- 172.21.15.84:0/3185839261 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec009b30 con 0x7f60fc0adb50 2023-10-16T11:24:46.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.171+0000 7f60f9d9b640 1 --2- 172.21.15.84:0/3185839261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60fc0adb50 0x7f60fc0bd550 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7f60e400d5f0 tx=0x7f60e400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:46.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.171+0000 7f60f2ffd640 1 -- 172.21.15.84:0/3185839261 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e40125f0 con 0x7f60fc0adb50 2023-10-16T11:24:46.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.172+0000 7f60f2ffd640 1 -- 172.21.15.84:0/3185839261 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e4012750 con 0x7f60fc0adb50 2023-10-16T11:24:46.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.172+0000 7f60f2ffd640 1 -- 172.21.15.84:0/3185839261 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e4012b00 con 0x7f60fc0adb50 2023-10-16T11:24:46.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.172+0000 7f6100ff0640 1 -- 172.21.15.84:0/3185839261 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60fc0c0bb0 con 0x7f60fc0adb50 2023-10-16T11:24:46.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.172+0000 7f6100ff0640 1 -- 172.21.15.84:0/3185839261 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60fc0bda90 con 0x7f60fc0adb50 2023-10-16T11:24:46.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.173+0000 7f6100ff0640 1 -- 172.21.15.84:0/3185839261 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60fc0bdd10 con 0x7f60fc0adb50 2023-10-16T11:24:46.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.174+0000 7f60f2ffd640 1 -- 172.21.15.84:0/3185839261 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f60e4012c60 con 0x7f60fc0adb50 2023-10-16T11:24:46.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.175+0000 7f60f2ffd640 1 --2- 172.21.15.84:0/3185839261 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f60dc06af90 0x7f60dc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:46.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.175+0000 7f60f2ffd640 1 -- 172.21.15.84:0/3185839261 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f60e4089ff0 con 0x7f60fc0adb50 2023-10-16T11:24:46.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.175+0000 7f60fa59c640 1 --2- 172.21.15.84:0/3185839261 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f60dc06af90 0x7f60dc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:46.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.176+0000 7f60fa59c640 1 --2- 172.21.15.84:0/3185839261 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f60dc06af90 0x7f60dc06d450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f60ec035000 tx=0x7f60ec030000 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:46.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.180+0000 7f60f2ffd640 1 -- 172.21.15.84:0/3185839261 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f60e4054c60 con 0x7f60fc0adb50 2023-10-16T11:24:46.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.370+0000 7f6100ff0640 1 -- 172.21.15.84:0/3185839261 --> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60fc06fa90 con 0x7f60dc06af90 2023-10-16T11:24:46.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.412+0000 7f60f2ffd640 1 -- 172.21.15.84:0/3185839261 <== mgr.11076 v2:172.21.15.84:6832/1425409017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+525743 (secure 0 0 0) 0x7f60ec02ca40 con 0x7f60dc06af90 2023-10-16T11:24:46.421 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:46.424 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:24:46.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.420+0000 7f6100ff0640 1 -- 172.21.15.84:0/3185839261 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f60dc06af90 msgr2=0x7f60dc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:46.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.420+0000 7f6100ff0640 1 --2- 172.21.15.84:0/3185839261 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f60dc06af90 0x7f60dc06d450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f60ec035000 tx=0x7f60ec030000 comp rx=0 tx=0).stop 2023-10-16T11:24:46.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.420+0000 7f6100ff0640 1 -- 172.21.15.84:0/3185839261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60fc0adb50 msgr2=0x7f60fc0bd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:46.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.420+0000 7f6100ff0640 1 --2- 172.21.15.84:0/3185839261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60fc0adb50 0x7f60fc0bd550 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7f60e400d5f0 tx=0x7f60e400dab0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.420+0000 7f6100ff0640 1 -- 172.21.15.84:0/3185839261 shutdown_connections 2023-10-16T11:24:46.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.420+0000 7f6100ff0640 1 --2- 172.21.15.84:0/3185839261 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f60dc06af90 0x7f60dc06d450 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.420+0000 7f6100ff0640 1 --2- 172.21.15.84:0/3185839261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60fc0adf30 0x7f60fc1631b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.420+0000 7f6100ff0640 1 --2- 172.21.15.84:0/3185839261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60fc0adb50 0x7f60fc0bd550 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.420+0000 7f6100ff0640 1 --2- 172.21.15.84:0/3185839261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60fc070fd0 0x7f60fc150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.420+0000 7f6100ff0640 1 -- 172.21.15.84:0/3185839261 >> 172.21.15.84:0/3185839261 conn(0x7f60fc0cb410 msgr2=0x7f60fc06cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:46.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.420+0000 7f6100ff0640 1 -- 172.21.15.84:0/3185839261 shutdown_connections 2023-10-16T11:24:46.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.421+0000 7f6100ff0640 1 -- 172.21.15.84:0/3185839261 wait complete. 2023-10-16T11:24:46.445 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-10-16T11:24:46.257450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":672,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2846,"ondisk_log_size":2846,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":367,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":380712,"kb_used_data":22952,"kb_used_omap":12,"kb_used_meta":357619,"kb_avail":754594008,"statfs":{"total":773094113280,"available":772704264192,"internally_reserved":0,"allocated":23502848,"data_stored":11986322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":366202425},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.371803"},"pg_stats":[{"pgid":"29.1c","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076487+0000","last_change":"2023-10-16T11:24:40.949704+0000","last_active":"2023-10-16T11:24:42.076487+0000","last_peered":"2023-10-16T11:24:42.076487+0000","last_clean":"2023-10-16T11:24:42.076487+0000","last_became_active":"2023-10-16T11:24:40.944766+0000","last_became_peered":"2023-10-16T11:24:40.944766+0000","last_unstale":"2023-10-16T11:24:42.076487+0000","last_undegraded":"2023-10-16T11:24:42.076487+0000","last_fullsized":"2023-10-16T11:24:42.076487+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:39:05.461904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"206'2767","reported_seq":3652,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.928589+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:24:40.928589+0000","last_peered":"2023-10-16T11:24:40.928589+0000","last_clean":"2023-10-16T11:24:40.928589+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:24:40.928589+0000","last_undegraded":"2023-10-16T11:24:40.928589+0000","last_fullsized":"2023-10-16T11:24:40.928589+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2767,"log_dups_size":0,"ondisk_log_size":2767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":581,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"216'1","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.916192+0000","last_change":"2023-10-16T11:24:25.087491+0000","last_active":"2023-10-16T11:24:40.916192+0000","last_peered":"2023-10-16T11:24:40.916192+0000","last_clean":"2023-10-16T11:24:40.916192+0000","last_became_active":"2023-10-16T11:24:25.086826+0000","last_became_peered":"2023-10-16T11:24:25.086826+0000","last_unstale":"2023-10-16T11:24:40.916192+0000","last_undegraded":"2023-10-16T11:24:40.916192+0000","last_fullsized":"2023-10-16T11:24:40.916192+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:07.303719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918232+0000","last_change":"2023-10-16T11:24:26.091340+0000","last_active":"2023-10-16T11:24:40.918232+0000","last_peered":"2023-10-16T11:24:40.918232+0000","last_clean":"2023-10-16T11:24:40.918232+0000","last_became_active":"2023-10-16T11:24:26.091059+0000","last_became_peered":"2023-10-16T11:24:26.091059+0000","last_unstale":"2023-10-16T11:24:40.918232+0000","last_undegraded":"2023-10-16T11:24:40.918232+0000","last_fullsized":"2023-10-16T11:24:40.918232+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:37.605400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.948878+0000","last_change":"2023-10-16T11:24:39.927746+0000","last_active":"2023-10-16T11:24:40.948878+0000","last_peered":"2023-10-16T11:24:40.948878+0000","last_clean":"2023-10-16T11:24:40.948878+0000","last_became_active":"2023-10-16T11:24:39.927422+0000","last_became_peered":"2023-10-16T11:24:39.927422+0000","last_unstale":"2023-10-16T11:24:40.948878+0000","last_undegraded":"2023-10-16T11:24:40.948878+0000","last_fullsized":"2023-10-16T11:24:40.948878+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:37.032264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.080660+0000","last_change":"2023-10-16T11:24:40.934994+0000","last_active":"2023-10-16T11:24:42.080660+0000","last_peered":"2023-10-16T11:24:42.080660+0000","last_clean":"2023-10-16T11:24:42.080660+0000","last_became_active":"2023-10-16T11:24:40.929932+0000","last_became_peered":"2023-10-16T11:24:40.929932+0000","last_unstale":"2023-10-16T11:24:42.080660+0000","last_undegraded":"2023-10-16T11:24:42.080660+0000","last_fullsized":"2023-10-16T11:24:42.080660+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:23.723306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913447+0000","last_change":"2023-10-16T11:24:25.087755+0000","last_active":"2023-10-16T11:24:40.913447+0000","last_peered":"2023-10-16T11:24:40.913447+0000","last_clean":"2023-10-16T11:24:40.913447+0000","last_became_active":"2023-10-16T11:24:25.087497+0000","last_became_peered":"2023-10-16T11:24:25.087497+0000","last_unstale":"2023-10-16T11:24:40.913447+0000","last_undegraded":"2023-10-16T11:24:40.913447+0000","last_fullsized":"2023-10-16T11:24:40.913447+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:59:26.196708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.929543+0000","last_change":"2023-10-16T11:24:26.081718+0000","last_active":"2023-10-16T11:24:40.929543+0000","last_peered":"2023-10-16T11:24:40.929543+0000","last_clean":"2023-10-16T11:24:40.929543+0000","last_became_active":"2023-10-16T11:24:26.081447+0000","last_became_peered":"2023-10-16T11:24:26.081447+0000","last_unstale":"2023-10-16T11:24:40.929543+0000","last_undegraded":"2023-10-16T11:24:40.929543+0000","last_fullsized":"2023-10-16T11:24:40.929543+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.928257+0000","last_change":"2023-10-16T11:24:39.923647+0000","last_active":"2023-10-16T11:24:40.928257+0000","last_peered":"2023-10-16T11:24:40.928257+0000","last_clean":"2023-10-16T11:24:40.928257+0000","last_became_active":"2023-10-16T11:24:39.923359+0000","last_became_peered":"2023-10-16T11:24:39.923359+0000","last_unstale":"2023-10-16T11:24:40.928257+0000","last_undegraded":"2023-10-16T11:24:40.928257+0000","last_fullsized":"2023-10-16T11:24:40.928257+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:18:00.023742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.073930+0000","last_change":"2023-10-16T11:24:40.934136+0000","last_active":"2023-10-16T11:24:42.073930+0000","last_peered":"2023-10-16T11:24:42.073930+0000","last_clean":"2023-10-16T11:24:42.073930+0000","last_became_active":"2023-10-16T11:24:40.931559+0000","last_became_peered":"2023-10-16T11:24:40.931559+0000","last_unstale":"2023-10-16T11:24:42.073930+0000","last_undegraded":"2023-10-16T11:24:42.073930+0000","last_fullsized":"2023-10-16T11:24:42.073930+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:54:04.701711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.926384+0000","last_change":"2023-10-16T11:24:25.088161+0000","last_active":"2023-10-16T11:24:40.926384+0000","last_peered":"2023-10-16T11:24:40.926384+0000","last_clean":"2023-10-16T11:24:40.926384+0000","last_became_active":"2023-10-16T11:24:25.086051+0000","last_became_peered":"2023-10-16T11:24:25.086051+0000","last_unstale":"2023-10-16T11:24:40.926384+0000","last_undegraded":"2023-10-16T11:24:40.926384+0000","last_fullsized":"2023-10-16T11:24:40.926384+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.907231+0000","last_change":"2023-10-16T11:24:26.083366+0000","last_active":"2023-10-16T11:24:40.907231+0000","last_peered":"2023-10-16T11:24:40.907231+0000","last_clean":"2023-10-16T11:24:40.907231+0000","last_became_active":"2023-10-16T11:24:26.082635+0000","last_became_peered":"2023-10-16T11:24:26.082635+0000","last_unstale":"2023-10-16T11:24:40.907231+0000","last_undegraded":"2023-10-16T11:24:40.907231+0000","last_fullsized":"2023-10-16T11:24:40.907231+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.942903+0000","last_change":"2023-10-16T11:24:39.926910+0000","last_active":"2023-10-16T11:24:40.942903+0000","last_peered":"2023-10-16T11:24:40.942903+0000","last_clean":"2023-10-16T11:24:40.942903+0000","last_became_active":"2023-10-16T11:24:39.926615+0000","last_became_peered":"2023-10-16T11:24:39.926615+0000","last_unstale":"2023-10-16T11:24:40.942903+0000","last_undegraded":"2023-10-16T11:24:40.942903+0000","last_fullsized":"2023-10-16T11:24:40.942903+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:06.744007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.073721+0000","last_change":"2023-10-16T11:24:40.923750+0000","last_active":"2023-10-16T11:24:42.073721+0000","last_peered":"2023-10-16T11:24:42.073721+0000","last_clean":"2023-10-16T11:24:42.073721+0000","last_became_active":"2023-10-16T11:24:40.922455+0000","last_became_peered":"2023-10-16T11:24:40.922455+0000","last_unstale":"2023-10-16T11:24:42.073721+0000","last_undegraded":"2023-10-16T11:24:42.073721+0000","last_fullsized":"2023-10-16T11:24:42.073721+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:40.760759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912963+0000","last_change":"2023-10-16T11:24:25.080689+0000","last_active":"2023-10-16T11:24:40.912963+0000","last_peered":"2023-10-16T11:24:40.912963+0000","last_clean":"2023-10-16T11:24:40.912963+0000","last_became_active":"2023-10-16T11:24:25.080305+0000","last_became_peered":"2023-10-16T11:24:25.080305+0000","last_unstale":"2023-10-16T11:24:40.912963+0000","last_undegraded":"2023-10-16T11:24:40.912963+0000","last_fullsized":"2023-10-16T11:24:40.912963+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:43.670603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.916175+0000","last_change":"2023-10-16T11:24:26.083766+0000","last_active":"2023-10-16T11:24:40.916175+0000","last_peered":"2023-10-16T11:24:40.916175+0000","last_clean":"2023-10-16T11:24:40.916175+0000","last_became_active":"2023-10-16T11:24:26.083447+0000","last_became_peered":"2023-10-16T11:24:26.083447+0000","last_unstale":"2023-10-16T11:24:40.916175+0000","last_undegraded":"2023-10-16T11:24:40.916175+0000","last_fullsized":"2023-10-16T11:24:40.916175+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:44:14.656443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.914990+0000","last_change":"2023-10-16T11:24:39.926629+0000","last_active":"2023-10-16T11:24:40.914990+0000","last_peered":"2023-10-16T11:24:40.914990+0000","last_clean":"2023-10-16T11:24:40.914990+0000","last_became_active":"2023-10-16T11:24:39.926334+0000","last_became_peered":"2023-10-16T11:24:39.926334+0000","last_unstale":"2023-10-16T11:24:40.914990+0000","last_undegraded":"2023-10-16T11:24:40.914990+0000","last_fullsized":"2023-10-16T11:24:40.914990+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:24.166384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076395+0000","last_change":"2023-10-16T11:24:40.940956+0000","last_active":"2023-10-16T11:24:42.076395+0000","last_peered":"2023-10-16T11:24:42.076395+0000","last_clean":"2023-10-16T11:24:42.076395+0000","last_became_active":"2023-10-16T11:24:40.940649+0000","last_became_peered":"2023-10-16T11:24:40.940649+0000","last_unstale":"2023-10-16T11:24:42.076395+0000","last_undegraded":"2023-10-16T11:24:42.076395+0000","last_fullsized":"2023-10-16T11:24:42.076395+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:59.520985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.921975+0000","last_change":"2023-10-16T11:24:25.090432+0000","last_active":"2023-10-16T11:24:40.921975+0000","last_peered":"2023-10-16T11:24:40.921975+0000","last_clean":"2023-10-16T11:24:40.921975+0000","last_became_active":"2023-10-16T11:24:25.090142+0000","last_became_peered":"2023-10-16T11:24:25.090142+0000","last_unstale":"2023-10-16T11:24:40.921975+0000","last_undegraded":"2023-10-16T11:24:40.921975+0000","last_fullsized":"2023-10-16T11:24:40.921975+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:22.858690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918327+0000","last_change":"2023-10-16T11:24:26.093027+0000","last_active":"2023-10-16T11:24:40.918327+0000","last_peered":"2023-10-16T11:24:40.918327+0000","last_clean":"2023-10-16T11:24:40.918327+0000","last_became_active":"2023-10-16T11:24:26.092766+0000","last_became_peered":"2023-10-16T11:24:26.092766+0000","last_unstale":"2023-10-16T11:24:40.918327+0000","last_undegraded":"2023-10-16T11:24:40.918327+0000","last_fullsized":"2023-10-16T11:24:40.918327+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.923353+0000","last_change":"2023-10-16T11:24:39.924175+0000","last_active":"2023-10-16T11:24:40.923353+0000","last_peered":"2023-10-16T11:24:40.923353+0000","last_clean":"2023-10-16T11:24:40.923353+0000","last_became_active":"2023-10-16T11:24:39.923702+0000","last_became_peered":"2023-10-16T11:24:39.923702+0000","last_unstale":"2023-10-16T11:24:40.923353+0000","last_undegraded":"2023-10-16T11:24:40.923353+0000","last_fullsized":"2023-10-16T11:24:40.923353+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:33:55.188653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.077311+0000","last_change":"2023-10-16T11:24:40.937310+0000","last_active":"2023-10-16T11:24:42.077311+0000","last_peered":"2023-10-16T11:24:42.077311+0000","last_clean":"2023-10-16T11:24:42.077311+0000","last_became_active":"2023-10-16T11:24:40.937033+0000","last_became_peered":"2023-10-16T11:24:40.937033+0000","last_unstale":"2023-10-16T11:24:42.077311+0000","last_undegraded":"2023-10-16T11:24:42.077311+0000","last_fullsized":"2023-10-16T11:24:42.077311+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:51:40.711088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.908646+0000","last_change":"2023-10-16T11:24:25.088585+0000","last_active":"2023-10-16T11:24:40.908646+0000","last_peered":"2023-10-16T11:24:40.908646+0000","last_clean":"2023-10-16T11:24:40.908646+0000","last_became_active":"2023-10-16T11:24:25.088011+0000","last_became_peered":"2023-10-16T11:24:25.088011+0000","last_unstale":"2023-10-16T11:24:40.908646+0000","last_undegraded":"2023-10-16T11:24:40.908646+0000","last_fullsized":"2023-10-16T11:24:40.908646+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920936+0000","last_change":"2023-10-16T11:24:26.096341+0000","last_active":"2023-10-16T11:24:40.920936+0000","last_peered":"2023-10-16T11:24:40.920936+0000","last_clean":"2023-10-16T11:24:40.920936+0000","last_became_active":"2023-10-16T11:24:26.095836+0000","last_became_peered":"2023-10-16T11:24:26.095836+0000","last_unstale":"2023-10-16T11:24:40.920936+0000","last_undegraded":"2023-10-16T11:24:40.920936+0000","last_fullsized":"2023-10-16T11:24:40.920936+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917221+0000","last_change":"2023-10-16T11:24:39.909816+0000","last_active":"2023-10-16T11:24:40.917221+0000","last_peered":"2023-10-16T11:24:40.917221+0000","last_clean":"2023-10-16T11:24:40.917221+0000","last_became_active":"2023-10-16T11:24:39.909526+0000","last_became_peered":"2023-10-16T11:24:39.909526+0000","last_unstale":"2023-10-16T11:24:40.917221+0000","last_undegraded":"2023-10-16T11:24:40.917221+0000","last_fullsized":"2023-10-16T11:24:40.917221+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:12.120581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935566+0000","last_change":"2023-10-16T11:24:40.938174+0000","last_active":"2023-10-16T11:24:41.935566+0000","last_peered":"2023-10-16T11:24:41.935566+0000","last_clean":"2023-10-16T11:24:41.935566+0000","last_became_active":"2023-10-16T11:24:40.937904+0000","last_became_peered":"2023-10-16T11:24:40.937904+0000","last_unstale":"2023-10-16T11:24:41.935566+0000","last_undegraded":"2023-10-16T11:24:41.935566+0000","last_fullsized":"2023-10-16T11:24:41.935566+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:13.135380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913653+0000","last_change":"2023-10-16T11:24:25.077036+0000","last_active":"2023-10-16T11:24:40.913653+0000","last_peered":"2023-10-16T11:24:40.913653+0000","last_clean":"2023-10-16T11:24:40.913653+0000","last_became_active":"2023-10-16T11:24:25.076342+0000","last_became_peered":"2023-10-16T11:24:25.076342+0000","last_unstale":"2023-10-16T11:24:40.913653+0000","last_undegraded":"2023-10-16T11:24:40.913653+0000","last_fullsized":"2023-10-16T11:24:40.913653+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:36.162662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.935192+0000","last_change":"2023-10-16T11:24:26.085231+0000","last_active":"2023-10-16T11:24:40.935192+0000","last_peered":"2023-10-16T11:24:40.935192+0000","last_clean":"2023-10-16T11:24:40.935192+0000","last_became_active":"2023-10-16T11:24:26.084889+0000","last_became_peered":"2023-10-16T11:24:26.084889+0000","last_unstale":"2023-10-16T11:24:40.935192+0000","last_undegraded":"2023-10-16T11:24:40.935192+0000","last_fullsized":"2023-10-16T11:24:40.935192+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.935331+0000","last_change":"2023-10-16T11:24:39.920299+0000","last_active":"2023-10-16T11:24:40.935331+0000","last_peered":"2023-10-16T11:24:40.935331+0000","last_clean":"2023-10-16T11:24:40.935331+0000","last_became_active":"2023-10-16T11:24:39.919985+0000","last_became_peered":"2023-10-16T11:24:39.919985+0000","last_unstale":"2023-10-16T11:24:40.935331+0000","last_undegraded":"2023-10-16T11:24:40.935331+0000","last_fullsized":"2023-10-16T11:24:40.935331+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:59.121334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.077429+0000","last_change":"2023-10-16T11:24:40.940322+0000","last_active":"2023-10-16T11:24:42.077429+0000","last_peered":"2023-10-16T11:24:42.077429+0000","last_clean":"2023-10-16T11:24:42.077429+0000","last_became_active":"2023-10-16T11:24:40.940072+0000","last_became_peered":"2023-10-16T11:24:40.940072+0000","last_unstale":"2023-10-16T11:24:42.077429+0000","last_undegraded":"2023-10-16T11:24:42.077429+0000","last_fullsized":"2023-10-16T11:24:42.077429+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:33.978064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.6","version":"217'1","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.928431+0000","last_change":"2023-10-16T11:24:25.093214+0000","last_active":"2023-10-16T11:24:40.928431+0000","last_peered":"2023-10-16T11:24:40.928431+0000","last_clean":"2023-10-16T11:24:40.928431+0000","last_became_active":"2023-10-16T11:24:25.092877+0000","last_became_peered":"2023-10-16T11:24:25.092877+0000","last_unstale":"2023-10-16T11:24:40.928431+0000","last_undegraded":"2023-10-16T11:24:40.928431+0000","last_fullsized":"2023-10-16T11:24:40.928431+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:59.924424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.919655+0000","last_change":"2023-10-16T11:24:26.089940+0000","last_active":"2023-10-16T11:24:40.919655+0000","last_peered":"2023-10-16T11:24:40.919655+0000","last_clean":"2023-10-16T11:24:40.919655+0000","last_became_active":"2023-10-16T11:24:26.087906+0000","last_became_peered":"2023-10-16T11:24:26.087906+0000","last_unstale":"2023-10-16T11:24:40.919655+0000","last_undegraded":"2023-10-16T11:24:40.919655+0000","last_fullsized":"2023-10-16T11:24:40.919655+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:44.687193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913130+0000","last_change":"2023-10-16T11:24:39.918078+0000","last_active":"2023-10-16T11:24:40.913130+0000","last_peered":"2023-10-16T11:24:40.913130+0000","last_clean":"2023-10-16T11:24:40.913130+0000","last_became_active":"2023-10-16T11:24:39.914432+0000","last_became_peered":"2023-10-16T11:24:39.914432+0000","last_unstale":"2023-10-16T11:24:40.913130+0000","last_undegraded":"2023-10-16T11:24:40.913130+0000","last_fullsized":"2023-10-16T11:24:40.913130+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:07:10.996990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.074496+0000","last_change":"2023-10-16T11:24:40.940070+0000","last_active":"2023-10-16T11:24:42.074496+0000","last_peered":"2023-10-16T11:24:42.074496+0000","last_clean":"2023-10-16T11:24:42.074496+0000","last_became_active":"2023-10-16T11:24:40.939742+0000","last_became_peered":"2023-10-16T11:24:40.939742+0000","last_unstale":"2023-10-16T11:24:42.074496+0000","last_undegraded":"2023-10-16T11:24:42.074496+0000","last_fullsized":"2023-10-16T11:24:42.074496+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:07:00.381749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"216'2","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.909469+0000","last_change":"2023-10-16T11:24:25.094230+0000","last_active":"2023-10-16T11:24:40.909469+0000","last_peered":"2023-10-16T11:24:40.909469+0000","last_clean":"2023-10-16T11:24:40.909469+0000","last_became_active":"2023-10-16T11:24:25.093896+0000","last_became_peered":"2023-10-16T11:24:25.093896+0000","last_unstale":"2023-10-16T11:24:40.909469+0000","last_undegraded":"2023-10-16T11:24:40.909469+0000","last_fullsized":"2023-10-16T11:24:40.909469+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.915544+0000","last_change":"2023-10-16T11:24:26.085909+0000","last_active":"2023-10-16T11:24:40.915544+0000","last_peered":"2023-10-16T11:24:40.915544+0000","last_clean":"2023-10-16T11:24:40.915544+0000","last_became_active":"2023-10-16T11:24:26.085620+0000","last_became_peered":"2023-10-16T11:24:26.085620+0000","last_unstale":"2023-10-16T11:24:40.915544+0000","last_undegraded":"2023-10-16T11:24:40.915544+0000","last_fullsized":"2023-10-16T11:24:40.915544+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:41.023833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913673+0000","last_change":"2023-10-16T11:24:39.910612+0000","last_active":"2023-10-16T11:24:40.913673+0000","last_peered":"2023-10-16T11:24:40.913673+0000","last_clean":"2023-10-16T11:24:40.913673+0000","last_became_active":"2023-10-16T11:24:39.910324+0000","last_became_peered":"2023-10-16T11:24:39.910324+0000","last_unstale":"2023-10-16T11:24:40.913673+0000","last_undegraded":"2023-10-16T11:24:40.913673+0000","last_fullsized":"2023-10-16T11:24:40.913673+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:44:02.305284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.934777+0000","last_change":"2023-10-16T11:24:40.945289+0000","last_active":"2023-10-16T11:24:41.934777+0000","last_peered":"2023-10-16T11:24:41.934777+0000","last_clean":"2023-10-16T11:24:41.934777+0000","last_became_active":"2023-10-16T11:24:40.944986+0000","last_became_peered":"2023-10-16T11:24:40.944986+0000","last_unstale":"2023-10-16T11:24:41.934777+0000","last_undegraded":"2023-10-16T11:24:41.934777+0000","last_fullsized":"2023-10-16T11:24:41.934777+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:46.340170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912752+0000","last_change":"2023-10-16T11:24:25.080225+0000","last_active":"2023-10-16T11:24:40.912752+0000","last_peered":"2023-10-16T11:24:40.912752+0000","last_clean":"2023-10-16T11:24:40.912752+0000","last_became_active":"2023-10-16T11:24:25.079921+0000","last_became_peered":"2023-10-16T11:24:25.079921+0000","last_unstale":"2023-10-16T11:24:40.912752+0000","last_undegraded":"2023-10-16T11:24:40.912752+0000","last_fullsized":"2023-10-16T11:24:40.912752+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:46.550429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.926296+0000","last_change":"2023-10-16T11:24:26.088438+0000","last_active":"2023-10-16T11:24:40.926296+0000","last_peered":"2023-10-16T11:24:40.926296+0000","last_clean":"2023-10-16T11:24:40.926296+0000","last_became_active":"2023-10-16T11:24:26.087973+0000","last_became_peered":"2023-10-16T11:24:26.087973+0000","last_unstale":"2023-10-16T11:24:40.926296+0000","last_undegraded":"2023-10-16T11:24:40.926296+0000","last_fullsized":"2023-10-16T11:24:40.926296+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.944794+0000","last_change":"2023-10-16T11:24:39.915986+0000","last_active":"2023-10-16T11:24:40.944794+0000","last_peered":"2023-10-16T11:24:40.944794+0000","last_clean":"2023-10-16T11:24:40.944794+0000","last_became_active":"2023-10-16T11:24:39.915611+0000","last_became_peered":"2023-10-16T11:24:39.915611+0000","last_unstale":"2023-10-16T11:24:40.944794+0000","last_undegraded":"2023-10-16T11:24:40.944794+0000","last_fullsized":"2023-10-16T11:24:40.944794+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:37.032264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935666+0000","last_change":"2023-10-16T11:24:40.932578+0000","last_active":"2023-10-16T11:24:41.935666+0000","last_peered":"2023-10-16T11:24:41.935666+0000","last_clean":"2023-10-16T11:24:41.935666+0000","last_became_active":"2023-10-16T11:24:40.931761+0000","last_became_peered":"2023-10-16T11:24:40.931761+0000","last_unstale":"2023-10-16T11:24:41.935666+0000","last_undegraded":"2023-10-16T11:24:41.935666+0000","last_fullsized":"2023-10-16T11:24:41.935666+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:55.400689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917511+0000","last_change":"2023-10-16T11:24:25.086180+0000","last_active":"2023-10-16T11:24:40.917511+0000","last_peered":"2023-10-16T11:24:40.917511+0000","last_clean":"2023-10-16T11:24:40.917511+0000","last_became_active":"2023-10-16T11:24:25.085316+0000","last_became_peered":"2023-10-16T11:24:25.085316+0000","last_unstale":"2023-10-16T11:24:40.917511+0000","last_undegraded":"2023-10-16T11:24:40.917511+0000","last_fullsized":"2023-10-16T11:24:40.917511+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927499+0000","last_change":"2023-10-16T11:24:26.090951+0000","last_active":"2023-10-16T11:24:40.927499+0000","last_peered":"2023-10-16T11:24:40.927499+0000","last_clean":"2023-10-16T11:24:40.927499+0000","last_became_active":"2023-10-16T11:24:26.089241+0000","last_became_peered":"2023-10-16T11:24:26.089241+0000","last_unstale":"2023-10-16T11:24:40.927499+0000","last_undegraded":"2023-10-16T11:24:40.927499+0000","last_fullsized":"2023-10-16T11:24:40.927499+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:40.970357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"220'1","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.946450+0000","last_change":"2023-10-16T11:24:39.926741+0000","last_active":"2023-10-16T11:24:40.946450+0000","last_peered":"2023-10-16T11:24:40.946450+0000","last_clean":"2023-10-16T11:24:40.946450+0000","last_became_active":"2023-10-16T11:24:39.926377+0000","last_became_peered":"2023-10-16T11:24:39.926377+0000","last_unstale":"2023-10-16T11:24:40.946450+0000","last_undegraded":"2023-10-16T11:24:40.946450+0000","last_fullsized":"2023-10-16T11:24:40.946450+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:07:06.185553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.080553+0000","last_change":"2023-10-16T11:24:40.936952+0000","last_active":"2023-10-16T11:24:42.080553+0000","last_peered":"2023-10-16T11:24:42.080553+0000","last_clean":"2023-10-16T11:24:42.080553+0000","last_became_active":"2023-10-16T11:24:40.936526+0000","last_became_peered":"2023-10-16T11:24:40.936526+0000","last_unstale":"2023-10-16T11:24:42.080553+0000","last_undegraded":"2023-10-16T11:24:42.080553+0000","last_fullsized":"2023-10-16T11:24:42.080553+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:43:43.109538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918490+0000","last_change":"2023-10-16T11:24:25.091656+0000","last_active":"2023-10-16T11:24:40.918490+0000","last_peered":"2023-10-16T11:24:40.918490+0000","last_clean":"2023-10-16T11:24:40.918490+0000","last_became_active":"2023-10-16T11:24:25.091316+0000","last_became_peered":"2023-10-16T11:24:25.091316+0000","last_unstale":"2023-10-16T11:24:40.918490+0000","last_undegraded":"2023-10-16T11:24:40.918490+0000","last_fullsized":"2023-10-16T11:24:40.918490+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:26.054702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.908836+0000","last_change":"2023-10-16T11:24:26.085260+0000","last_active":"2023-10-16T11:24:40.908836+0000","last_peered":"2023-10-16T11:24:40.908836+0000","last_clean":"2023-10-16T11:24:40.908836+0000","last_became_active":"2023-10-16T11:24:26.084860+0000","last_became_peered":"2023-10-16T11:24:26.084860+0000","last_unstale":"2023-10-16T11:24:40.908836+0000","last_undegraded":"2023-10-16T11:24:40.908836+0000","last_fullsized":"2023-10-16T11:24:40.908836+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920928+0000","last_change":"2023-10-16T11:24:39.908388+0000","last_active":"2023-10-16T11:24:40.920928+0000","last_peered":"2023-10-16T11:24:40.920928+0000","last_clean":"2023-10-16T11:24:40.920928+0000","last_became_active":"2023-10-16T11:24:39.908075+0000","last_became_peered":"2023-10-16T11:24:39.908075+0000","last_unstale":"2023-10-16T11:24:40.920928+0000","last_undegraded":"2023-10-16T11:24:40.920928+0000","last_fullsized":"2023-10-16T11:24:40.920928+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:12:56.357582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.074734+0000","last_change":"2023-10-16T11:24:40.934674+0000","last_active":"2023-10-16T11:24:42.074734+0000","last_peered":"2023-10-16T11:24:42.074734+0000","last_clean":"2023-10-16T11:24:42.074734+0000","last_became_active":"2023-10-16T11:24:40.934420+0000","last_became_peered":"2023-10-16T11:24:40.934420+0000","last_unstale":"2023-10-16T11:24:42.074734+0000","last_undegraded":"2023-10-16T11:24:42.074734+0000","last_fullsized":"2023-10-16T11:24:42.074734+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:52.013044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.b","version":"216'2","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.923129+0000","last_change":"2023-10-16T11:24:25.072340+0000","last_active":"2023-10-16T11:24:40.923129+0000","last_peered":"2023-10-16T11:24:40.923129+0000","last_clean":"2023-10-16T11:24:40.923129+0000","last_became_active":"2023-10-16T11:24:25.071728+0000","last_became_peered":"2023-10-16T11:24:25.071728+0000","last_unstale":"2023-10-16T11:24:40.923129+0000","last_undegraded":"2023-10-16T11:24:40.923129+0000","last_fullsized":"2023-10-16T11:24:40.923129+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:49.041521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927055+0000","last_change":"2023-10-16T11:24:26.091096+0000","last_active":"2023-10-16T11:24:40.927055+0000","last_peered":"2023-10-16T11:24:40.927055+0000","last_clean":"2023-10-16T11:24:40.927055+0000","last_became_active":"2023-10-16T11:24:26.090774+0000","last_became_peered":"2023-10-16T11:24:26.090774+0000","last_unstale":"2023-10-16T11:24:40.927055+0000","last_undegraded":"2023-10-16T11:24:40.927055+0000","last_fullsized":"2023-10-16T11:24:40.927055+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:27.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"220'1","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.944256+0000","last_change":"2023-10-16T11:24:39.921764+0000","last_active":"2023-10-16T11:24:40.944256+0000","last_peered":"2023-10-16T11:24:40.944256+0000","last_clean":"2023-10-16T11:24:40.944256+0000","last_became_active":"2023-10-16T11:24:39.921487+0000","last_became_peered":"2023-10-16T11:24:39.921487+0000","last_unstale":"2023-10-16T11:24:40.944256+0000","last_undegraded":"2023-10-16T11:24:40.944256+0000","last_fullsized":"2023-10-16T11:24:40.944256+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:25.891782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076327+0000","last_change":"2023-10-16T11:24:40.946254+0000","last_active":"2023-10-16T11:24:42.076327+0000","last_peered":"2023-10-16T11:24:42.076327+0000","last_clean":"2023-10-16T11:24:42.076327+0000","last_became_active":"2023-10-16T11:24:40.945902+0000","last_became_peered":"2023-10-16T11:24:40.945902+0000","last_unstale":"2023-10-16T11:24:42.076327+0000","last_undegraded":"2023-10-16T11:24:42.076327+0000","last_fullsized":"2023-10-16T11:24:42.076327+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:52.439191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.908290+0000","last_change":"2023-10-16T11:24:25.079021+0000","last_active":"2023-10-16T11:24:40.908290+0000","last_peered":"2023-10-16T11:24:40.908290+0000","last_clean":"2023-10-16T11:24:40.908290+0000","last_became_active":"2023-10-16T11:24:25.076800+0000","last_became_peered":"2023-10-16T11:24:25.076800+0000","last_unstale":"2023-10-16T11:24:40.908290+0000","last_undegraded":"2023-10-16T11:24:40.908290+0000","last_fullsized":"2023-10-16T11:24:40.908290+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:56.161407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920389+0000","last_change":"2023-10-16T11:24:26.092876+0000","last_active":"2023-10-16T11:24:40.920389+0000","last_peered":"2023-10-16T11:24:40.920389+0000","last_clean":"2023-10-16T11:24:40.920389+0000","last_became_active":"2023-10-16T11:24:26.091310+0000","last_became_peered":"2023-10-16T11:24:26.091310+0000","last_unstale":"2023-10-16T11:24:40.920389+0000","last_undegraded":"2023-10-16T11:24:40.920389+0000","last_fullsized":"2023-10-16T11:24:40.920389+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:22.891780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.931170+0000","last_change":"2023-10-16T11:24:39.918307+0000","last_active":"2023-10-16T11:24:40.931170+0000","last_peered":"2023-10-16T11:24:40.931170+0000","last_clean":"2023-10-16T11:24:40.931170+0000","last_became_active":"2023-10-16T11:24:39.916755+0000","last_became_peered":"2023-10-16T11:24:39.916755+0000","last_unstale":"2023-10-16T11:24:40.931170+0000","last_undegraded":"2023-10-16T11:24:40.931170+0000","last_fullsized":"2023-10-16T11:24:40.931170+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:54.385890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072744+0000","last_change":"2023-10-16T11:24:40.929984+0000","last_active":"2023-10-16T11:24:42.072744+0000","last_peered":"2023-10-16T11:24:42.072744+0000","last_clean":"2023-10-16T11:24:42.072744+0000","last_became_active":"2023-10-16T11:24:40.928183+0000","last_became_peered":"2023-10-16T11:24:40.928183+0000","last_unstale":"2023-10-16T11:24:42.072744+0000","last_undegraded":"2023-10-16T11:24:42.072744+0000","last_fullsized":"2023-10-16T11:24:42.072744+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:00:55.030170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.d","version":"216'1","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920754+0000","last_change":"2023-10-16T11:24:25.083155+0000","last_active":"2023-10-16T11:24:40.920754+0000","last_peered":"2023-10-16T11:24:40.920754+0000","last_clean":"2023-10-16T11:24:40.920754+0000","last_became_active":"2023-10-16T11:24:25.082690+0000","last_became_peered":"2023-10-16T11:24:25.082690+0000","last_unstale":"2023-10-16T11:24:40.920754+0000","last_undegraded":"2023-10-16T11:24:40.920754+0000","last_fullsized":"2023-10-16T11:24:40.920754+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:46:58.816018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.908131+0000","last_change":"2023-10-16T11:24:26.069780+0000","last_active":"2023-10-16T11:24:40.908131+0000","last_peered":"2023-10-16T11:24:40.908131+0000","last_clean":"2023-10-16T11:24:40.908131+0000","last_became_active":"2023-10-16T11:24:26.068772+0000","last_became_peered":"2023-10-16T11:24:26.068772+0000","last_unstale":"2023-10-16T11:24:40.908131+0000","last_undegraded":"2023-10-16T11:24:40.908131+0000","last_fullsized":"2023-10-16T11:24:40.908131+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.944704+0000","last_change":"2023-10-16T11:24:39.928140+0000","last_active":"2023-10-16T11:24:40.944704+0000","last_peered":"2023-10-16T11:24:40.944704+0000","last_clean":"2023-10-16T11:24:40.944704+0000","last_became_active":"2023-10-16T11:24:39.927182+0000","last_became_peered":"2023-10-16T11:24:39.927182+0000","last_unstale":"2023-10-16T11:24:40.944704+0000","last_undegraded":"2023-10-16T11:24:40.944704+0000","last_fullsized":"2023-10-16T11:24:40.944704+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:04:01.386012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072533+0000","last_change":"2023-10-16T11:24:40.937098+0000","last_active":"2023-10-16T11:24:42.072533+0000","last_peered":"2023-10-16T11:24:42.072533+0000","last_clean":"2023-10-16T11:24:42.072533+0000","last_became_active":"2023-10-16T11:24:40.936844+0000","last_became_peered":"2023-10-16T11:24:40.936844+0000","last_unstale":"2023-10-16T11:24:42.072533+0000","last_undegraded":"2023-10-16T11:24:42.072533+0000","last_fullsized":"2023-10-16T11:24:42.072533+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:06.892515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.e","version":"216'2","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.933425+0000","last_change":"2023-10-16T11:24:25.087765+0000","last_active":"2023-10-16T11:24:40.933425+0000","last_peered":"2023-10-16T11:24:40.933425+0000","last_clean":"2023-10-16T11:24:40.933425+0000","last_became_active":"2023-10-16T11:24:25.086967+0000","last_became_peered":"2023-10-16T11:24:25.086967+0000","last_unstale":"2023-10-16T11:24:40.933425+0000","last_undegraded":"2023-10-16T11:24:40.933425+0000","last_fullsized":"2023-10-16T11:24:40.933425+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.932831+0000","last_change":"2023-10-16T11:24:26.086833+0000","last_active":"2023-10-16T11:24:40.932831+0000","last_peered":"2023-10-16T11:24:40.932831+0000","last_clean":"2023-10-16T11:24:40.932831+0000","last_became_active":"2023-10-16T11:24:26.085367+0000","last_became_peered":"2023-10-16T11:24:26.085367+0000","last_unstale":"2023-10-16T11:24:40.932831+0000","last_undegraded":"2023-10-16T11:24:40.932831+0000","last_fullsized":"2023-10-16T11:24:40.932831+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913101+0000","last_change":"2023-10-16T11:24:39.913504+0000","last_active":"2023-10-16T11:24:40.913101+0000","last_peered":"2023-10-16T11:24:40.913101+0000","last_clean":"2023-10-16T11:24:40.913101+0000","last_became_active":"2023-10-16T11:24:39.913184+0000","last_became_peered":"2023-10-16T11:24:39.913184+0000","last_unstale":"2023-10-16T11:24:40.913101+0000","last_undegraded":"2023-10-16T11:24:40.913101+0000","last_fullsized":"2023-10-16T11:24:40.913101+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:36.821422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.931761+0000","last_change":"2023-10-16T11:24:40.936862+0000","last_active":"2023-10-16T11:24:41.931761+0000","last_peered":"2023-10-16T11:24:41.931761+0000","last_clean":"2023-10-16T11:24:41.931761+0000","last_became_active":"2023-10-16T11:24:40.933139+0000","last_became_peered":"2023-10-16T11:24:40.933139+0000","last_unstale":"2023-10-16T11:24:41.931761+0000","last_undegraded":"2023-10-16T11:24:41.931761+0000","last_fullsized":"2023-10-16T11:24:41.931761+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:23.266881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.f","version":"216'1","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913585+0000","last_change":"2023-10-16T11:24:25.079789+0000","last_active":"2023-10-16T11:24:40.913585+0000","last_peered":"2023-10-16T11:24:40.913585+0000","last_clean":"2023-10-16T11:24:40.913585+0000","last_became_active":"2023-10-16T11:24:25.079495+0000","last_became_peered":"2023-10-16T11:24:25.079495+0000","last_unstale":"2023-10-16T11:24:40.913585+0000","last_undegraded":"2023-10-16T11:24:40.913585+0000","last_fullsized":"2023-10-16T11:24:40.913585+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:44.531367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.933148+0000","last_change":"2023-10-16T11:24:26.081796+0000","last_active":"2023-10-16T11:24:40.933148+0000","last_peered":"2023-10-16T11:24:40.933148+0000","last_clean":"2023-10-16T11:24:40.933148+0000","last_became_active":"2023-10-16T11:24:26.080287+0000","last_became_peered":"2023-10-16T11:24:26.080287+0000","last_unstale":"2023-10-16T11:24:40.933148+0000","last_undegraded":"2023-10-16T11:24:40.933148+0000","last_fullsized":"2023-10-16T11:24:40.933148+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927703+0000","last_change":"2023-10-16T11:24:39.922329+0000","last_active":"2023-10-16T11:24:40.927703+0000","last_peered":"2023-10-16T11:24:40.927703+0000","last_clean":"2023-10-16T11:24:40.927703+0000","last_became_active":"2023-10-16T11:24:39.921982+0000","last_became_peered":"2023-10-16T11:24:39.921982+0000","last_unstale":"2023-10-16T11:24:40.927703+0000","last_undegraded":"2023-10-16T11:24:40.927703+0000","last_fullsized":"2023-10-16T11:24:40.927703+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:17.673425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.080239+0000","last_change":"2023-10-16T11:24:40.949583+0000","last_active":"2023-10-16T11:24:42.080239+0000","last_peered":"2023-10-16T11:24:42.080239+0000","last_clean":"2023-10-16T11:24:42.080239+0000","last_became_active":"2023-10-16T11:24:40.947541+0000","last_became_peered":"2023-10-16T11:24:40.947541+0000","last_unstale":"2023-10-16T11:24:42.080239+0000","last_undegraded":"2023-10-16T11:24:42.080239+0000","last_fullsized":"2023-10-16T11:24:42.080239+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:10:12.883381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.10","version":"216'8","reported_seq":24,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.907330+0000","last_change":"2023-10-16T11:24:25.076954+0000","last_active":"2023-10-16T11:24:40.907330+0000","last_peered":"2023-10-16T11:24:40.907330+0000","last_clean":"2023-10-16T11:24:40.907330+0000","last_became_active":"2023-10-16T11:24:25.076509+0000","last_became_peered":"2023-10-16T11:24:25.076509+0000","last_unstale":"2023-10-16T11:24:40.907330+0000","last_undegraded":"2023-10-16T11:24:40.907330+0000","last_fullsized":"2023-10-16T11:24:40.907330+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:46.770213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917418+0000","last_change":"2023-10-16T11:24:26.084139+0000","last_active":"2023-10-16T11:24:40.917418+0000","last_peered":"2023-10-16T11:24:40.917418+0000","last_clean":"2023-10-16T11:24:40.917418+0000","last_became_active":"2023-10-16T11:24:26.083876+0000","last_became_peered":"2023-10-16T11:24:26.083876+0000","last_unstale":"2023-10-16T11:24:40.917418+0000","last_undegraded":"2023-10-16T11:24:40.917418+0000","last_fullsized":"2023-10-16T11:24:40.917418+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918095+0000","last_change":"2023-10-16T11:24:39.904420+0000","last_active":"2023-10-16T11:24:40.918095+0000","last_peered":"2023-10-16T11:24:40.918095+0000","last_clean":"2023-10-16T11:24:40.918095+0000","last_became_active":"2023-10-16T11:24:39.904039+0000","last_became_peered":"2023-10-16T11:24:39.904039+0000","last_unstale":"2023-10-16T11:24:40.918095+0000","last_undegraded":"2023-10-16T11:24:40.918095+0000","last_fullsized":"2023-10-16T11:24:40.918095+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:17.673425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.936696+0000","last_change":"2023-10-16T11:24:40.947940+0000","last_active":"2023-10-16T11:24:41.936696+0000","last_peered":"2023-10-16T11:24:41.936696+0000","last_clean":"2023-10-16T11:24:41.936696+0000","last_became_active":"2023-10-16T11:24:40.947659+0000","last_became_peered":"2023-10-16T11:24:40.947659+0000","last_unstale":"2023-10-16T11:24:41.936696+0000","last_undegraded":"2023-10-16T11:24:41.936696+0000","last_fullsized":"2023-10-16T11:24:41.936696+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:15:00.136133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.923325+0000","last_change":"2023-10-16T11:24:25.089470+0000","last_active":"2023-10-16T11:24:40.923325+0000","last_peered":"2023-10-16T11:24:40.923325+0000","last_clean":"2023-10-16T11:24:40.923325+0000","last_became_active":"2023-10-16T11:24:25.089213+0000","last_became_peered":"2023-10-16T11:24:25.089213+0000","last_unstale":"2023-10-16T11:24:40.923325+0000","last_undegraded":"2023-10-16T11:24:40.923325+0000","last_fullsized":"2023-10-16T11:24:40.923325+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:34.540849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.907117+0000","last_change":"2023-10-16T11:24:26.079602+0000","last_active":"2023-10-16T11:24:40.907117+0000","last_peered":"2023-10-16T11:24:40.907117+0000","last_clean":"2023-10-16T11:24:40.907117+0000","last_became_active":"2023-10-16T11:24:26.079344+0000","last_became_peered":"2023-10-16T11:24:26.079344+0000","last_unstale":"2023-10-16T11:24:40.907117+0000","last_undegraded":"2023-10-16T11:24:40.907117+0000","last_fullsized":"2023-10-16T11:24:40.907117+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.930050+0000","last_change":"2023-10-16T11:24:39.924572+0000","last_active":"2023-10-16T11:24:40.930050+0000","last_peered":"2023-10-16T11:24:40.930050+0000","last_clean":"2023-10-16T11:24:40.930050+0000","last_became_active":"2023-10-16T11:24:39.924266+0000","last_became_peered":"2023-10-16T11:24:39.924266+0000","last_unstale":"2023-10-16T11:24:40.930050+0000","last_undegraded":"2023-10-16T11:24:40.930050+0000","last_fullsized":"2023-10-16T11:24:40.930050+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:57:57.289628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.077212+0000","last_change":"2023-10-16T11:24:40.938221+0000","last_active":"2023-10-16T11:24:42.077212+0000","last_peered":"2023-10-16T11:24:42.077212+0000","last_clean":"2023-10-16T11:24:42.077212+0000","last_became_active":"2023-10-16T11:24:40.937659+0000","last_became_peered":"2023-10-16T11:24:40.937659+0000","last_unstale":"2023-10-16T11:24:42.077212+0000","last_undegraded":"2023-10-16T11:24:42.077212+0000","last_fullsized":"2023-10-16T11:24:42.077212+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:27.817701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.924919+0000","last_change":"2023-10-16T11:24:25.084194+0000","last_active":"2023-10-16T11:24:40.924919+0000","last_peered":"2023-10-16T11:24:40.924919+0000","last_clean":"2023-10-16T11:24:40.924919+0000","last_became_active":"2023-10-16T11:24:25.083141+0000","last_became_peered":"2023-10-16T11:24:25.083141+0000","last_unstale":"2023-10-16T11:24:40.924919+0000","last_undegraded":"2023-10-16T11:24:40.924919+0000","last_fullsized":"2023-10-16T11:24:40.924919+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:46.770213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.923041+0000","last_change":"2023-10-16T11:24:26.089221+0000","last_active":"2023-10-16T11:24:40.923041+0000","last_peered":"2023-10-16T11:24:40.923041+0000","last_clean":"2023-10-16T11:24:40.923041+0000","last_became_active":"2023-10-16T11:24:26.088644+0000","last_became_peered":"2023-10-16T11:24:26.088644+0000","last_unstale":"2023-10-16T11:24:40.923041+0000","last_undegraded":"2023-10-16T11:24:40.923041+0000","last_fullsized":"2023-10-16T11:24:40.923041+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:10.387018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"220'6","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.948264+0000","last_change":"2023-10-16T11:24:39.904338+0000","last_active":"2023-10-16T11:24:40.948264+0000","last_peered":"2023-10-16T11:24:40.948264+0000","last_clean":"2023-10-16T11:24:40.948264+0000","last_became_active":"2023-10-16T11:24:39.904034+0000","last_became_peered":"2023-10-16T11:24:39.904034+0000","last_unstale":"2023-10-16T11:24:40.948264+0000","last_undegraded":"2023-10-16T11:24:40.948264+0000","last_fullsized":"2023-10-16T11:24:40.948264+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:58.317757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.078231+0000","last_change":"2023-10-16T11:24:40.949476+0000","last_active":"2023-10-16T11:24:42.078231+0000","last_peered":"2023-10-16T11:24:42.078231+0000","last_clean":"2023-10-16T11:24:42.078231+0000","last_became_active":"2023-10-16T11:24:40.949199+0000","last_became_peered":"2023-10-16T11:24:40.949199+0000","last_unstale":"2023-10-16T11:24:42.078231+0000","last_undegraded":"2023-10-16T11:24:42.078231+0000","last_fullsized":"2023-10-16T11:24:42.078231+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:25:21.239825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.13","version":"216'2","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920754+0000","last_change":"2023-10-16T11:24:25.081377+0000","last_active":"2023-10-16T11:24:40.920754+0000","last_peered":"2023-10-16T11:24:40.920754+0000","last_clean":"2023-10-16T11:24:40.920754+0000","last_became_active":"2023-10-16T11:24:25.080881+0000","last_became_peered":"2023-10-16T11:24:25.080881+0000","last_unstale":"2023-10-16T11:24:40.920754+0000","last_undegraded":"2023-10-16T11:24:40.920754+0000","last_fullsized":"2023-10-16T11:24:40.920754+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:26.054702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912668+0000","last_change":"2023-10-16T11:24:26.099039+0000","last_active":"2023-10-16T11:24:40.912668+0000","last_peered":"2023-10-16T11:24:40.912668+0000","last_clean":"2023-10-16T11:24:40.912668+0000","last_became_active":"2023-10-16T11:24:26.097980+0000","last_became_peered":"2023-10-16T11:24:26.097980+0000","last_unstale":"2023-10-16T11:24:40.912668+0000","last_undegraded":"2023-10-16T11:24:40.912668+0000","last_fullsized":"2023-10-16T11:24:40.912668+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:40.585267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"220'3","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.936626+0000","last_change":"2023-10-16T11:24:39.906086+0000","last_active":"2023-10-16T11:24:40.936626+0000","last_peered":"2023-10-16T11:24:40.936626+0000","last_clean":"2023-10-16T11:24:40.936626+0000","last_became_active":"2023-10-16T11:24:39.905781+0000","last_became_peered":"2023-10-16T11:24:39.905781+0000","last_unstale":"2023-10-16T11:24:40.936626+0000","last_undegraded":"2023-10-16T11:24:40.936626+0000","last_fullsized":"2023-10-16T11:24:40.936626+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:07:47.547377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.078344+0000","last_change":"2023-10-16T11:24:40.936652+0000","last_active":"2023-10-16T11:24:42.078344+0000","last_peered":"2023-10-16T11:24:42.078344+0000","last_clean":"2023-10-16T11:24:42.078344+0000","last_became_active":"2023-10-16T11:24:40.934276+0000","last_became_peered":"2023-10-16T11:24:40.934276+0000","last_unstale":"2023-10-16T11:24:42.078344+0000","last_undegraded":"2023-10-16T11:24:42.078344+0000","last_fullsized":"2023-10-16T11:24:42.078344+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:30.742890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.14","version":"217'8","reported_seq":24,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.922693+0000","last_change":"2023-10-16T11:24:25.095357+0000","last_active":"2023-10-16T11:24:40.922693+0000","last_peered":"2023-10-16T11:24:40.922693+0000","last_clean":"2023-10-16T11:24:40.922693+0000","last_became_active":"2023-10-16T11:24:25.093832+0000","last_became_peered":"2023-10-16T11:24:25.093832+0000","last_unstale":"2023-10-16T11:24:40.922693+0000","last_undegraded":"2023-10-16T11:24:40.922693+0000","last_fullsized":"2023-10-16T11:24:40.922693+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:14:19.482469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918256+0000","last_change":"2023-10-16T11:24:26.092185+0000","last_active":"2023-10-16T11:24:40.918256+0000","last_peered":"2023-10-16T11:24:40.918256+0000","last_clean":"2023-10-16T11:24:40.918256+0000","last_became_active":"2023-10-16T11:24:26.091648+0000","last_became_peered":"2023-10-16T11:24:26.091648+0000","last_unstale":"2023-10-16T11:24:40.918256+0000","last_undegraded":"2023-10-16T11:24:40.918256+0000","last_fullsized":"2023-10-16T11:24:40.918256+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918129+0000","last_change":"2023-10-16T11:24:39.919312+0000","last_active":"2023-10-16T11:24:40.918129+0000","last_peered":"2023-10-16T11:24:40.918129+0000","last_clean":"2023-10-16T11:24:40.918129+0000","last_became_active":"2023-10-16T11:24:39.918986+0000","last_became_peered":"2023-10-16T11:24:39.918986+0000","last_unstale":"2023-10-16T11:24:40.918129+0000","last_undegraded":"2023-10-16T11:24:40.918129+0000","last_fullsized":"2023-10-16T11:24:40.918129+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:54.385890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072402+0000","last_change":"2023-10-16T11:24:40.925767+0000","last_active":"2023-10-16T11:24:42.072402+0000","last_peered":"2023-10-16T11:24:42.072402+0000","last_clean":"2023-10-16T11:24:42.072402+0000","last_became_active":"2023-10-16T11:24:40.925501+0000","last_became_peered":"2023-10-16T11:24:40.925501+0000","last_unstale":"2023-10-16T11:24:42.072402+0000","last_undegraded":"2023-10-16T11:24:42.072402+0000","last_fullsized":"2023-10-16T11:24:42.072402+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:49:08.490072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.15","version":"216'3","reported_seq":19,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920628+0000","last_change":"2023-10-16T11:24:25.082968+0000","last_active":"2023-10-16T11:24:40.920628+0000","last_peered":"2023-10-16T11:24:40.920628+0000","last_clean":"2023-10-16T11:24:40.920628+0000","last_became_active":"2023-10-16T11:24:25.082316+0000","last_became_peered":"2023-10-16T11:24:25.082316+0000","last_unstale":"2023-10-16T11:24:40.920628+0000","last_undegraded":"2023-10-16T11:24:40.920628+0000","last_fullsized":"2023-10-16T11:24:40.920628+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:59.924424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.922137+0000","last_change":"2023-10-16T11:24:26.084487+0000","last_active":"2023-10-16T11:24:40.922137+0000","last_peered":"2023-10-16T11:24:40.922137+0000","last_clean":"2023-10-16T11:24:40.922137+0000","last_became_active":"2023-10-16T11:24:26.084188+0000","last_became_peered":"2023-10-16T11:24:26.084188+0000","last_unstale":"2023-10-16T11:24:40.922137+0000","last_undegraded":"2023-10-16T11:24:40.922137+0000","last_fullsized":"2023-10-16T11:24:40.922137+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:47.400027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.945716+0000","last_change":"2023-10-16T11:24:39.928051+0000","last_active":"2023-10-16T11:24:40.945716+0000","last_peered":"2023-10-16T11:24:40.945716+0000","last_clean":"2023-10-16T11:24:40.945716+0000","last_became_active":"2023-10-16T11:24:39.927740+0000","last_became_peered":"2023-10-16T11:24:39.927740+0000","last_unstale":"2023-10-16T11:24:40.945716+0000","last_undegraded":"2023-10-16T11:24:40.945716+0000","last_fullsized":"2023-10-16T11:24:40.945716+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:07.682759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.075589+0000","last_change":"2023-10-16T11:24:40.931397+0000","last_active":"2023-10-16T11:24:42.075589+0000","last_peered":"2023-10-16T11:24:42.075589+0000","last_clean":"2023-10-16T11:24:42.075589+0000","last_became_active":"2023-10-16T11:24:40.931130+0000","last_became_peered":"2023-10-16T11:24:40.931130+0000","last_unstale":"2023-10-16T11:24:42.075589+0000","last_undegraded":"2023-10-16T11:24:42.075589+0000","last_fullsized":"2023-10-16T11:24:42.075589+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:13.011085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.926531+0000","last_change":"2023-10-16T11:24:25.092100+0000","last_active":"2023-10-16T11:24:40.926531+0000","last_peered":"2023-10-16T11:24:40.926531+0000","last_clean":"2023-10-16T11:24:40.926531+0000","last_became_active":"2023-10-16T11:24:25.091237+0000","last_became_peered":"2023-10-16T11:24:25.091237+0000","last_unstale":"2023-10-16T11:24:40.926531+0000","last_undegraded":"2023-10-16T11:24:40.926531+0000","last_fullsized":"2023-10-16T11:24:40.926531+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:21.985839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.909348+0000","last_change":"2023-10-16T11:24:26.081980+0000","last_active":"2023-10-16T11:24:40.909348+0000","last_peered":"2023-10-16T11:24:40.909348+0000","last_clean":"2023-10-16T11:24:40.909348+0000","last_became_active":"2023-10-16T11:24:26.081395+0000","last_became_peered":"2023-10-16T11:24:26.081395+0000","last_unstale":"2023-10-16T11:24:40.909348+0000","last_undegraded":"2023-10-16T11:24:40.909348+0000","last_fullsized":"2023-10-16T11:24:40.909348+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:39.802117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"220'8","reported_seq":20,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.949404+0000","last_change":"2023-10-16T11:24:39.915652+0000","last_active":"2023-10-16T11:24:40.949404+0000","last_peered":"2023-10-16T11:24:40.949404+0000","last_clean":"2023-10-16T11:24:40.949404+0000","last_became_active":"2023-10-16T11:24:39.915277+0000","last_became_peered":"2023-10-16T11:24:39.915277+0000","last_unstale":"2023-10-16T11:24:40.949404+0000","last_undegraded":"2023-10-16T11:24:40.949404+0000","last_fullsized":"2023-10-16T11:24:40.949404+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:49.377967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.080669+0000","last_change":"2023-10-16T11:24:40.938686+0000","last_active":"2023-10-16T11:24:42.080669+0000","last_peered":"2023-10-16T11:24:42.080669+0000","last_clean":"2023-10-16T11:24:42.080669+0000","last_became_active":"2023-10-16T11:24:40.937440+0000","last_became_peered":"2023-10-16T11:24:40.937440+0000","last_unstale":"2023-10-16T11:24:42.080669+0000","last_undegraded":"2023-10-16T11:24:42.080669+0000","last_fullsized":"2023-10-16T11:24:42.080669+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:08:17.799924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913537+0000","last_change":"2023-10-16T11:24:25.094384+0000","last_active":"2023-10-16T11:24:40.913537+0000","last_peered":"2023-10-16T11:24:40.913537+0000","last_clean":"2023-10-16T11:24:40.913537+0000","last_became_active":"2023-10-16T11:24:25.094026+0000","last_became_peered":"2023-10-16T11:24:25.094026+0000","last_unstale":"2023-10-16T11:24:40.913537+0000","last_undegraded":"2023-10-16T11:24:40.913537+0000","last_fullsized":"2023-10-16T11:24:40.913537+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:51.349970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.928344+0000","last_change":"2023-10-16T11:24:26.082915+0000","last_active":"2023-10-16T11:24:40.928344+0000","last_peered":"2023-10-16T11:24:40.928344+0000","last_clean":"2023-10-16T11:24:40.928344+0000","last_became_active":"2023-10-16T11:24:26.082458+0000","last_became_peered":"2023-10-16T11:24:26.082458+0000","last_unstale":"2023-10-16T11:24:40.928344+0000","last_undegraded":"2023-10-16T11:24:40.928344+0000","last_fullsized":"2023-10-16T11:24:40.928344+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:10.387018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913132+0000","last_change":"2023-10-16T11:24:39.903226+0000","last_active":"2023-10-16T11:24:40.913132+0000","last_peered":"2023-10-16T11:24:40.913132+0000","last_clean":"2023-10-16T11:24:40.913132+0000","last_became_active":"2023-10-16T11:24:39.902943+0000","last_became_peered":"2023-10-16T11:24:39.902943+0000","last_unstale":"2023-10-16T11:24:40.913132+0000","last_undegraded":"2023-10-16T11:24:40.913132+0000","last_fullsized":"2023-10-16T11:24:40.913132+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:28:29.998083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.936311+0000","last_change":"2023-10-16T11:24:40.938912+0000","last_active":"2023-10-16T11:24:41.936311+0000","last_peered":"2023-10-16T11:24:41.936311+0000","last_clean":"2023-10-16T11:24:41.936311+0000","last_became_active":"2023-10-16T11:24:40.938615+0000","last_became_peered":"2023-10-16T11:24:40.938615+0000","last_unstale":"2023-10-16T11:24:41.936311+0000","last_undegraded":"2023-10-16T11:24:41.936311+0000","last_fullsized":"2023-10-16T11:24:41.936311+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:38.047063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"220'1","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.947170+0000","last_change":"2023-10-16T11:24:39.916128+0000","last_active":"2023-10-16T11:24:40.947170+0000","last_peered":"2023-10-16T11:24:40.947170+0000","last_clean":"2023-10-16T11:24:40.947170+0000","last_became_active":"2023-10-16T11:24:39.911215+0000","last_became_peered":"2023-10-16T11:24:39.911215+0000","last_unstale":"2023-10-16T11:24:40.947170+0000","last_undegraded":"2023-10-16T11:24:40.947170+0000","last_fullsized":"2023-10-16T11:24:40.947170+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:59:34.809544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927021+0000","last_change":"2023-10-16T11:24:25.084521+0000","last_active":"2023-10-16T11:24:40.927021+0000","last_peered":"2023-10-16T11:24:40.927021+0000","last_clean":"2023-10-16T11:24:40.927021+0000","last_became_active":"2023-10-16T11:24:25.084003+0000","last_became_peered":"2023-10-16T11:24:25.084003+0000","last_unstale":"2023-10-16T11:24:40.927021+0000","last_undegraded":"2023-10-16T11:24:40.927021+0000","last_fullsized":"2023-10-16T11:24:40.927021+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:36.060066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.919781+0000","last_change":"2023-10-16T11:24:26.091641+0000","last_active":"2023-10-16T11:24:40.919781+0000","last_peered":"2023-10-16T11:24:40.919781+0000","last_clean":"2023-10-16T11:24:40.919781+0000","last_became_active":"2023-10-16T11:24:26.089186+0000","last_became_peered":"2023-10-16T11:24:26.089186+0000","last_unstale":"2023-10-16T11:24:40.919781+0000","last_undegraded":"2023-10-16T11:24:40.919781+0000","last_fullsized":"2023-10-16T11:24:40.919781+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:32.247878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.082815+0000","last_change":"2023-10-16T11:24:40.946117+0000","last_active":"2023-10-16T11:24:42.082815+0000","last_peered":"2023-10-16T11:24:42.082815+0000","last_clean":"2023-10-16T11:24:42.082815+0000","last_became_active":"2023-10-16T11:24:40.943815+0000","last_became_peered":"2023-10-16T11:24:40.943815+0000","last_unstale":"2023-10-16T11:24:42.082815+0000","last_undegraded":"2023-10-16T11:24:42.082815+0000","last_fullsized":"2023-10-16T11:24:42.082815+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:08.697558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"220'4","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.947367+0000","last_change":"2023-10-16T11:24:39.915794+0000","last_active":"2023-10-16T11:24:40.947367+0000","last_peered":"2023-10-16T11:24:40.947367+0000","last_clean":"2023-10-16T11:24:40.947367+0000","last_became_active":"2023-10-16T11:24:39.915438+0000","last_became_peered":"2023-10-16T11:24:39.915438+0000","last_unstale":"2023-10-16T11:24:40.947367+0000","last_undegraded":"2023-10-16T11:24:40.947367+0000","last_fullsized":"2023-10-16T11:24:40.947367+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:59.121334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913267+0000","last_change":"2023-10-16T11:24:25.074647+0000","last_active":"2023-10-16T11:24:40.913267+0000","last_peered":"2023-10-16T11:24:40.913267+0000","last_clean":"2023-10-16T11:24:40.913267+0000","last_became_active":"2023-10-16T11:24:25.074373+0000","last_became_peered":"2023-10-16T11:24:25.074373+0000","last_unstale":"2023-10-16T11:24:40.913267+0000","last_undegraded":"2023-10-16T11:24:40.913267+0000","last_fullsized":"2023-10-16T11:24:40.913267+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:38.544520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912670+0000","last_change":"2023-10-16T11:24:26.088785+0000","last_active":"2023-10-16T11:24:40.912670+0000","last_peered":"2023-10-16T11:24:40.912670+0000","last_clean":"2023-10-16T11:24:40.912670+0000","last_became_active":"2023-10-16T11:24:26.088521+0000","last_became_peered":"2023-10-16T11:24:26.088521+0000","last_unstale":"2023-10-16T11:24:40.912670+0000","last_undegraded":"2023-10-16T11:24:40.912670+0000","last_fullsized":"2023-10-16T11:24:40.912670+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:29.145795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.078267+0000","last_change":"2023-10-16T11:24:40.950578+0000","last_active":"2023-10-16T11:24:42.078267+0000","last_peered":"2023-10-16T11:24:42.078267+0000","last_clean":"2023-10-16T11:24:42.078267+0000","last_became_active":"2023-10-16T11:24:40.947861+0000","last_became_peered":"2023-10-16T11:24:40.947861+0000","last_unstale":"2023-10-16T11:24:42.078267+0000","last_undegraded":"2023-10-16T11:24:42.078267+0000","last_fullsized":"2023-10-16T11:24:42.078267+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:49:08.490072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1a","version":"216'1","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912820+0000","last_change":"2023-10-16T11:24:25.084983+0000","last_active":"2023-10-16T11:24:40.912820+0000","last_peered":"2023-10-16T11:24:40.912820+0000","last_clean":"2023-10-16T11:24:40.912820+0000","last_became_active":"2023-10-16T11:24:25.084622+0000","last_became_peered":"2023-10-16T11:24:25.084622+0000","last_unstale":"2023-10-16T11:24:40.912820+0000","last_undegraded":"2023-10-16T11:24:40.912820+0000","last_fullsized":"2023-10-16T11:24:40.912820+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:36.588809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.921894+0000","last_change":"2023-10-16T11:24:26.089982+0000","last_active":"2023-10-16T11:24:40.921894+0000","last_peered":"2023-10-16T11:24:40.921894+0000","last_clean":"2023-10-16T11:24:40.921894+0000","last_became_active":"2023-10-16T11:24:26.089677+0000","last_became_peered":"2023-10-16T11:24:26.089677+0000","last_unstale":"2023-10-16T11:24:40.921894+0000","last_undegraded":"2023-10-16T11:24:40.921894+0000","last_fullsized":"2023-10-16T11:24:40.921894+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:22.009042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.940586+0000","last_change":"2023-10-16T11:24:39.914078+0000","last_active":"2023-10-16T11:24:40.940586+0000","last_peered":"2023-10-16T11:24:40.940586+0000","last_clean":"2023-10-16T11:24:40.940586+0000","last_became_active":"2023-10-16T11:24:39.913693+0000","last_became_peered":"2023-10-16T11:24:39.913693+0000","last_unstale":"2023-10-16T11:24:40.940586+0000","last_undegraded":"2023-10-16T11:24:40.940586+0000","last_fullsized":"2023-10-16T11:24:40.940586+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:08:16.785125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.073246+0000","last_change":"2023-10-16T11:24:40.943302+0000","last_active":"2023-10-16T11:24:42.073246+0000","last_peered":"2023-10-16T11:24:42.073246+0000","last_clean":"2023-10-16T11:24:42.073246+0000","last_became_active":"2023-10-16T11:24:40.939215+0000","last_became_peered":"2023-10-16T11:24:40.939215+0000","last_unstale":"2023-10-16T11:24:42.073246+0000","last_undegraded":"2023-10-16T11:24:42.073246+0000","last_fullsized":"2023-10-16T11:24:42.073246+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:22.925745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1c","version":"216'2","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.922233+0000","last_change":"2023-10-16T11:24:25.095179+0000","last_active":"2023-10-16T11:24:40.922233+0000","last_peered":"2023-10-16T11:24:40.922233+0000","last_clean":"2023-10-16T11:24:40.922233+0000","last_became_active":"2023-10-16T11:24:25.094714+0000","last_became_peered":"2023-10-16T11:24:25.094714+0000","last_unstale":"2023-10-16T11:24:40.922233+0000","last_undegraded":"2023-10-16T11:24:40.922233+0000","last_fullsized":"2023-10-16T11:24:40.922233+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:26.128614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920276+0000","last_change":"2023-10-16T11:24:26.096297+0000","last_active":"2023-10-16T11:24:40.920276+0000","last_peered":"2023-10-16T11:24:40.920276+0000","last_clean":"2023-10-16T11:24:40.920276+0000","last_became_active":"2023-10-16T11:24:26.095538+0000","last_became_peered":"2023-10-16T11:24:26.095538+0000","last_unstale":"2023-10-16T11:24:40.920276+0000","last_undegraded":"2023-10-16T11:24:40.920276+0000","last_fullsized":"2023-10-16T11:24:40.920276+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:47.400027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"220'1","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.946397+0000","last_change":"2023-10-16T11:24:39.923043+0000","last_active":"2023-10-16T11:24:40.946397+0000","last_peered":"2023-10-16T11:24:40.946397+0000","last_clean":"2023-10-16T11:24:40.946397+0000","last_became_active":"2023-10-16T11:24:39.922745+0000","last_became_peered":"2023-10-16T11:24:39.922745+0000","last_unstale":"2023-10-16T11:24:40.946397+0000","last_undegraded":"2023-10-16T11:24:40.946397+0000","last_fullsized":"2023-10-16T11:24:40.946397+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:03.774885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.075183+0000","last_change":"2023-10-16T11:24:40.948393+0000","last_active":"2023-10-16T11:24:42.075183+0000","last_peered":"2023-10-16T11:24:42.075183+0000","last_clean":"2023-10-16T11:24:42.075183+0000","last_became_active":"2023-10-16T11:24:40.946737+0000","last_became_peered":"2023-10-16T11:24:40.946737+0000","last_unstale":"2023-10-16T11:24:42.075183+0000","last_undegraded":"2023-10-16T11:24:42.075183+0000","last_fullsized":"2023-10-16T11:24:42.075183+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:49:23.318330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"216'2","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.909085+0000","last_change":"2023-10-16T11:24:25.081731+0000","last_active":"2023-10-16T11:24:40.909085+0000","last_peered":"2023-10-16T11:24:40.909085+0000","last_clean":"2023-10-16T11:24:40.909085+0000","last_became_active":"2023-10-16T11:24:25.081336+0000","last_became_peered":"2023-10-16T11:24:25.081336+0000","last_unstale":"2023-10-16T11:24:40.909085+0000","last_undegraded":"2023-10-16T11:24:40.909085+0000","last_fullsized":"2023-10-16T11:24:40.909085+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:09.330801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918704+0000","last_change":"2023-10-16T11:24:26.089791+0000","last_active":"2023-10-16T11:24:40.918704+0000","last_peered":"2023-10-16T11:24:40.918704+0000","last_clean":"2023-10-16T11:24:40.918704+0000","last_became_active":"2023-10-16T11:24:26.089516+0000","last_became_peered":"2023-10-16T11:24:26.089516+0000","last_unstale":"2023-10-16T11:24:40.918704+0000","last_undegraded":"2023-10-16T11:24:40.918704+0000","last_fullsized":"2023-10-16T11:24:40.918704+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"220'1","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.941517+0000","last_change":"2023-10-16T11:24:39.901227+0000","last_active":"2023-10-16T11:24:40.941517+0000","last_peered":"2023-10-16T11:24:40.941517+0000","last_clean":"2023-10-16T11:24:40.941517+0000","last_became_active":"2023-10-16T11:24:39.900852+0000","last_became_peered":"2023-10-16T11:24:39.900852+0000","last_unstale":"2023-10-16T11:24:40.941517+0000","last_undegraded":"2023-10-16T11:24:40.941517+0000","last_fullsized":"2023-10-16T11:24:40.941517+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:54.404259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935060+0000","last_change":"2023-10-16T11:24:40.944861+0000","last_active":"2023-10-16T11:24:41.935060+0000","last_peered":"2023-10-16T11:24:41.935060+0000","last_clean":"2023-10-16T11:24:41.935060+0000","last_became_active":"2023-10-16T11:24:40.944579+0000","last_became_peered":"2023-10-16T11:24:40.944579+0000","last_unstale":"2023-10-16T11:24:41.935060+0000","last_undegraded":"2023-10-16T11:24:41.935060+0000","last_fullsized":"2023-10-16T11:24:41.935060+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:47:20.730005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"216'1","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.933239+0000","last_change":"2023-10-16T11:24:25.094483+0000","last_active":"2023-10-16T11:24:40.933239+0000","last_peered":"2023-10-16T11:24:40.933239+0000","last_clean":"2023-10-16T11:24:40.933239+0000","last_became_active":"2023-10-16T11:24:25.094134+0000","last_became_peered":"2023-10-16T11:24:25.094134+0000","last_unstale":"2023-10-16T11:24:40.933239+0000","last_undegraded":"2023-10-16T11:24:40.933239+0000","last_fullsized":"2023-10-16T11:24:40.933239+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.923291+0000","last_change":"2023-10-16T11:24:26.094536+0000","last_active":"2023-10-16T11:24:40.923291+0000","last_peered":"2023-10-16T11:24:40.923291+0000","last_clean":"2023-10-16T11:24:40.923291+0000","last_became_active":"2023-10-16T11:24:26.094272+0000","last_became_peered":"2023-10-16T11:24:26.094272+0000","last_unstale":"2023-10-16T11:24:40.923291+0000","last_undegraded":"2023-10-16T11:24:40.923291+0000","last_fullsized":"2023-10-16T11:24:40.923291+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.924102+0000","last_change":"2023-10-16T11:24:39.916827+0000","last_active":"2023-10-16T11:24:40.924102+0000","last_peered":"2023-10-16T11:24:40.924102+0000","last_clean":"2023-10-16T11:24:40.924102+0000","last_became_active":"2023-10-16T11:24:39.916464+0000","last_became_peered":"2023-10-16T11:24:39.916464+0000","last_unstale":"2023-10-16T11:24:40.924102+0000","last_undegraded":"2023-10-16T11:24:40.924102+0000","last_fullsized":"2023-10-16T11:24:40.924102+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:25.891782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.932973+0000","last_change":"2023-10-16T11:24:40.928353+0000","last_active":"2023-10-16T11:24:41.932973+0000","last_peered":"2023-10-16T11:24:41.932973+0000","last_clean":"2023-10-16T11:24:41.932973+0000","last_became_active":"2023-10-16T11:24:40.928042+0000","last_became_peered":"2023-10-16T11:24:40.928042+0000","last_unstale":"2023-10-16T11:24:41.932973+0000","last_undegraded":"2023-10-16T11:24:41.932973+0000","last_fullsized":"2023-10-16T11:24:41.932973+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:57:58.304427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1f","version":"216'4","reported_seq":20,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918030+0000","last_change":"2023-10-16T11:24:25.096644+0000","last_active":"2023-10-16T11:24:40.918030+0000","last_peered":"2023-10-16T11:24:40.918030+0000","last_clean":"2023-10-16T11:24:40.918030+0000","last_became_active":"2023-10-16T11:24:25.096306+0000","last_became_peered":"2023-10-16T11:24:25.096306+0000","last_unstale":"2023-10-16T11:24:40.918030+0000","last_undegraded":"2023-10-16T11:24:40.918030+0000","last_fullsized":"2023-10-16T11:24:40.918030+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:22.618393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.933050+0000","last_change":"2023-10-16T11:24:26.069227+0000","last_active":"2023-10-16T11:24:40.933050+0000","last_peered":"2023-10-16T11:24:40.933050+0000","last_clean":"2023-10-16T11:24:40.933050+0000","last_became_active":"2023-10-16T11:24:26.068456+0000","last_became_peered":"2023-10-16T11:24:26.068456+0000","last_unstale":"2023-10-16T11:24:40.933050+0000","last_undegraded":"2023-10-16T11:24:40.933050+0000","last_fullsized":"2023-10-16T11:24:40.933050+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927501+0000","last_change":"2023-10-16T11:24:39.916911+0000","last_active":"2023-10-16T11:24:40.927501+0000","last_peered":"2023-10-16T11:24:40.927501+0000","last_clean":"2023-10-16T11:24:40.927501+0000","last_became_active":"2023-10-16T11:24:39.916584+0000","last_became_peered":"2023-10-16T11:24:39.916584+0000","last_unstale":"2023-10-16T11:24:40.927501+0000","last_undegraded":"2023-10-16T11:24:40.927501+0000","last_fullsized":"2023-10-16T11:24:40.927501+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:12.120581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935402+0000","last_change":"2023-10-16T11:24:40.929604+0000","last_active":"2023-10-16T11:24:41.935402+0000","last_peered":"2023-10-16T11:24:41.935402+0000","last_clean":"2023-10-16T11:24:41.935402+0000","last_became_active":"2023-10-16T11:24:40.929340+0000","last_became_peered":"2023-10-16T11:24:40.929340+0000","last_unstale":"2023-10-16T11:24:41.935402+0000","last_undegraded":"2023-10-16T11:24:41.935402+0000","last_fullsized":"2023-10-16T11:24:41.935402+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:18.688225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.916906+0000","last_change":"2023-10-16T11:24:26.085375+0000","last_active":"2023-10-16T11:24:40.916906+0000","last_peered":"2023-10-16T11:24:40.916906+0000","last_clean":"2023-10-16T11:24:40.916906+0000","last_became_active":"2023-10-16T11:24:26.085120+0000","last_became_peered":"2023-10-16T11:24:26.085120+0000","last_unstale":"2023-10-16T11:24:40.916906+0000","last_undegraded":"2023-10-16T11:24:40.916906+0000","last_fullsized":"2023-10-16T11:24:40.916906+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.079858+0000","last_change":"2023-10-16T11:24:40.943381+0000","last_active":"2023-10-16T11:24:42.079858+0000","last_peered":"2023-10-16T11:24:42.079858+0000","last_clean":"2023-10-16T11:24:42.079858+0000","last_became_active":"2023-10-16T11:24:40.941771+0000","last_became_peered":"2023-10-16T11:24:40.941771+0000","last_unstale":"2023-10-16T11:24:42.079858+0000","last_undegraded":"2023-10-16T11:24:42.079858+0000","last_fullsized":"2023-10-16T11:24:42.079858+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:30:10.159127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.923208+0000","last_change":"2023-10-16T11:24:26.084964+0000","last_active":"2023-10-16T11:24:40.923208+0000","last_peered":"2023-10-16T11:24:40.923208+0000","last_clean":"2023-10-16T11:24:40.923208+0000","last_became_active":"2023-10-16T11:24:26.084006+0000","last_became_peered":"2023-10-16T11:24:26.084006+0000","last_unstale":"2023-10-16T11:24:40.923208+0000","last_undegraded":"2023-10-16T11:24:40.923208+0000","last_fullsized":"2023-10-16T11:24:40.923208+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:03.949526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.934466+0000","last_change":"2023-10-16T11:24:40.937991+0000","last_active":"2023-10-16T11:24:41.934466+0000","last_peered":"2023-10-16T11:24:41.934466+0000","last_clean":"2023-10-16T11:24:41.934466+0000","last_became_active":"2023-10-16T11:24:40.934656+0000","last_became_peered":"2023-10-16T11:24:40.934656+0000","last_unstale":"2023-10-16T11:24:41.934466+0000","last_undegraded":"2023-10-16T11:24:41.934466+0000","last_fullsized":"2023-10-16T11:24:41.934466+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:49:23.318330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.931374+0000","last_change":"2023-10-16T11:24:26.086008+0000","last_active":"2023-10-16T11:24:40.931374+0000","last_peered":"2023-10-16T11:24:40.931374+0000","last_clean":"2023-10-16T11:24:40.931374+0000","last_became_active":"2023-10-16T11:24:26.085598+0000","last_became_peered":"2023-10-16T11:24:26.085598+0000","last_unstale":"2023-10-16T11:24:40.931374+0000","last_undegraded":"2023-10-16T11:24:40.931374+0000","last_fullsized":"2023-10-16T11:24:40.931374+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.077368+0000","last_change":"2023-10-16T11:24:40.941097+0000","last_active":"2023-10-16T11:24:42.077368+0000","last_peered":"2023-10-16T11:24:42.077368+0000","last_clean":"2023-10-16T11:24:42.077368+0000","last_became_active":"2023-10-16T11:24:40.938428+0000","last_became_peered":"2023-10-16T11:24:40.938428+0000","last_unstale":"2023-10-16T11:24:42.077368+0000","last_undegraded":"2023-10-16T11:24:42.077368+0000","last_fullsized":"2023-10-16T11:24:42.077368+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:12.311557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.915966+0000","last_change":"2023-10-16T11:24:26.086172+0000","last_active":"2023-10-16T11:24:40.915966+0000","last_peered":"2023-10-16T11:24:40.915966+0000","last_clean":"2023-10-16T11:24:40.915966+0000","last_became_active":"2023-10-16T11:24:26.085882+0000","last_became_peered":"2023-10-16T11:24:26.085882+0000","last_unstale":"2023-10-16T11:24:40.915966+0000","last_undegraded":"2023-10-16T11:24:40.915966+0000","last_fullsized":"2023-10-16T11:24:40.915966+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:44.265192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076084+0000","last_change":"2023-10-16T11:24:40.950225+0000","last_active":"2023-10-16T11:24:42.076084+0000","last_peered":"2023-10-16T11:24:42.076084+0000","last_clean":"2023-10-16T11:24:42.076084+0000","last_became_active":"2023-10-16T11:24:40.948659+0000","last_became_peered":"2023-10-16T11:24:40.948659+0000","last_unstale":"2023-10-16T11:24:42.076084+0000","last_undegraded":"2023-10-16T11:24:42.076084+0000","last_fullsized":"2023-10-16T11:24:42.076084+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:46.340170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.907529+0000","last_change":"2023-10-16T11:24:26.086228+0000","last_active":"2023-10-16T11:24:40.907529+0000","last_peered":"2023-10-16T11:24:40.907529+0000","last_clean":"2023-10-16T11:24:40.907529+0000","last_became_active":"2023-10-16T11:24:26.085912+0000","last_became_peered":"2023-10-16T11:24:26.085912+0000","last_unstale":"2023-10-16T11:24:40.907529+0000","last_undegraded":"2023-10-16T11:24:40.907529+0000","last_fullsized":"2023-10-16T11:24:40.907529+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.075408+0000","last_change":"2023-10-16T11:24:40.949598+0000","last_active":"2023-10-16T11:24:42.075408+0000","last_peered":"2023-10-16T11:24:42.075408+0000","last_clean":"2023-10-16T11:24:42.075408+0000","last_became_active":"2023-10-16T11:24:40.949322+0000","last_became_peered":"2023-10-16T11:24:40.949322+0000","last_unstale":"2023-10-16T11:24:42.075408+0000","last_undegraded":"2023-10-16T11:24:42.075408+0000","last_fullsized":"2023-10-16T11:24:42.075408+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:22:53.973191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.915484+0000","last_change":"2023-10-16T11:24:26.088015+0000","last_active":"2023-10-16T11:24:40.915484+0000","last_peered":"2023-10-16T11:24:40.915484+0000","last_clean":"2023-10-16T11:24:40.915484+0000","last_became_active":"2023-10-16T11:24:26.086932+0000","last_became_peered":"2023-10-16T11:24:26.086932+0000","last_unstale":"2023-10-16T11:24:40.915484+0000","last_undegraded":"2023-10-16T11:24:40.915484+0000","last_fullsized":"2023-10-16T11:24:40.915484+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:44.137821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.071053+0000","last_change":"2023-10-16T11:24:40.935550+0000","last_active":"2023-10-16T11:24:42.071053+0000","last_peered":"2023-10-16T11:24:42.071053+0000","last_clean":"2023-10-16T11:24:42.071053+0000","last_became_active":"2023-10-16T11:24:40.935237+0000","last_became_peered":"2023-10-16T11:24:40.935237+0000","last_unstale":"2023-10-16T11:24:42.071053+0000","last_undegraded":"2023-10-16T11:24:42.071053+0000","last_fullsized":"2023-10-16T11:24:42.071053+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:13.011085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918841+0000","last_change":"2023-10-16T11:24:26.095913+0000","last_active":"2023-10-16T11:24:40.918841+0000","last_peered":"2023-10-16T11:24:40.918841+0000","last_clean":"2023-10-16T11:24:40.918841+0000","last_became_active":"2023-10-16T11:24:26.093113+0000","last_became_peered":"2023-10-16T11:24:26.093113+0000","last_unstale":"2023-10-16T11:24:40.918841+0000","last_undegraded":"2023-10-16T11:24:40.918841+0000","last_fullsized":"2023-10-16T11:24:40.918841+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072486+0000","last_change":"2023-10-16T11:24:40.932619+0000","last_active":"2023-10-16T11:24:42.072486+0000","last_peered":"2023-10-16T11:24:42.072486+0000","last_clean":"2023-10-16T11:24:42.072486+0000","last_became_active":"2023-10-16T11:24:40.930562+0000","last_became_peered":"2023-10-16T11:24:40.930562+0000","last_unstale":"2023-10-16T11:24:42.072486+0000","last_undegraded":"2023-10-16T11:24:42.072486+0000","last_fullsized":"2023-10-16T11:24:42.072486+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:11.184845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.932735+0000","last_change":"2023-10-16T11:24:26.094066+0000","last_active":"2023-10-16T11:24:40.932735+0000","last_peered":"2023-10-16T11:24:40.932735+0000","last_clean":"2023-10-16T11:24:40.932735+0000","last_became_active":"2023-10-16T11:24:26.093428+0000","last_became_peered":"2023-10-16T11:24:26.093428+0000","last_unstale":"2023-10-16T11:24:40.932735+0000","last_undegraded":"2023-10-16T11:24:40.932735+0000","last_fullsized":"2023-10-16T11:24:40.932735+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935302+0000","last_change":"2023-10-16T11:24:40.944382+0000","last_active":"2023-10-16T11:24:41.935302+0000","last_peered":"2023-10-16T11:24:41.935302+0000","last_clean":"2023-10-16T11:24:41.935302+0000","last_became_active":"2023-10-16T11:24:40.943690+0000","last_became_peered":"2023-10-16T11:24:40.943690+0000","last_unstale":"2023-10-16T11:24:41.935302+0000","last_undegraded":"2023-10-16T11:24:41.935302+0000","last_fullsized":"2023-10-16T11:24:41.935302+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:15.346746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.921063+0000","last_change":"2023-10-16T11:24:26.091894+0000","last_active":"2023-10-16T11:24:40.921063+0000","last_peered":"2023-10-16T11:24:40.921063+0000","last_clean":"2023-10-16T11:24:40.921063+0000","last_became_active":"2023-10-16T11:24:26.091627+0000","last_became_peered":"2023-10-16T11:24:26.091627+0000","last_unstale":"2023-10-16T11:24:40.921063+0000","last_undegraded":"2023-10-16T11:24:40.921063+0000","last_fullsized":"2023-10-16T11:24:40.921063+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:46:59.832609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.081684+0000","last_change":"2023-10-16T11:24:40.941368+0000","last_active":"2023-10-16T11:24:42.081684+0000","last_peered":"2023-10-16T11:24:42.081684+0000","last_clean":"2023-10-16T11:24:42.081684+0000","last_became_active":"2023-10-16T11:24:40.940598+0000","last_became_peered":"2023-10-16T11:24:40.940598+0000","last_unstale":"2023-10-16T11:24:42.081684+0000","last_undegraded":"2023-10-16T11:24:42.081684+0000","last_fullsized":"2023-10-16T11:24:42.081684+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:50:47.255058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927515+0000","last_change":"2023-10-16T11:24:26.097221+0000","last_active":"2023-10-16T11:24:40.927515+0000","last_peered":"2023-10-16T11:24:40.927515+0000","last_clean":"2023-10-16T11:24:40.927515+0000","last_became_active":"2023-10-16T11:24:26.096572+0000","last_became_peered":"2023-10-16T11:24:26.096572+0000","last_unstale":"2023-10-16T11:24:40.927515+0000","last_undegraded":"2023-10-16T11:24:40.927515+0000","last_fullsized":"2023-10-16T11:24:40.927515+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.934384+0000","last_change":"2023-10-16T11:24:40.935022+0000","last_active":"2023-10-16T11:24:41.934384+0000","last_peered":"2023-10-16T11:24:41.934384+0000","last_clean":"2023-10-16T11:24:41.934384+0000","last_became_active":"2023-10-16T11:24:40.934325+0000","last_became_peered":"2023-10-16T11:24:40.934325+0000","last_unstale":"2023-10-16T11:24:41.934384+0000","last_undegraded":"2023-10-16T11:24:41.934384+0000","last_fullsized":"2023-10-16T11:24:41.934384+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:58.258235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.924840+0000","last_change":"2023-10-16T11:24:26.101039+0000","last_active":"2023-10-16T11:24:40.924840+0000","last_peered":"2023-10-16T11:24:40.924840+0000","last_clean":"2023-10-16T11:24:40.924840+0000","last_became_active":"2023-10-16T11:24:26.100757+0000","last_became_peered":"2023-10-16T11:24:26.100757+0000","last_unstale":"2023-10-16T11:24:40.924840+0000","last_undegraded":"2023-10-16T11:24:40.924840+0000","last_fullsized":"2023-10-16T11:24:40.924840+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.073731+0000","last_change":"2023-10-16T11:24:40.934747+0000","last_active":"2023-10-16T11:24:42.073731+0000","last_peered":"2023-10-16T11:24:42.073731+0000","last_clean":"2023-10-16T11:24:42.073731+0000","last_became_active":"2023-10-16T11:24:40.934319+0000","last_became_peered":"2023-10-16T11:24:40.934319+0000","last_unstale":"2023-10-16T11:24:42.073731+0000","last_undegraded":"2023-10-16T11:24:42.073731+0000","last_fullsized":"2023-10-16T11:24:42.073731+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:12:33.440509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.921807+0000","last_change":"2023-10-16T11:24:26.092718+0000","last_active":"2023-10-16T11:24:40.921807+0000","last_peered":"2023-10-16T11:24:40.921807+0000","last_clean":"2023-10-16T11:24:40.921807+0000","last_became_active":"2023-10-16T11:24:26.091377+0000","last_became_peered":"2023-10-16T11:24:26.091377+0000","last_unstale":"2023-10-16T11:24:40.921807+0000","last_undegraded":"2023-10-16T11:24:40.921807+0000","last_fullsized":"2023-10-16T11:24:40.921807+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:17.072626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072172+0000","last_change":"2023-10-16T11:24:40.933022+0000","last_active":"2023-10-16T11:24:42.072172+0000","last_peered":"2023-10-16T11:24:42.072172+0000","last_clean":"2023-10-16T11:24:42.072172+0000","last_became_active":"2023-10-16T11:24:40.932746+0000","last_became_peered":"2023-10-16T11:24:40.932746+0000","last_unstale":"2023-10-16T11:24:42.072172+0000","last_undegraded":"2023-10-16T11:24:42.072172+0000","last_fullsized":"2023-10-16T11:24:42.072172+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:42.730933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.915755+0000","last_change":"2023-10-16T11:24:26.093168+0000","last_active":"2023-10-16T11:24:40.915755+0000","last_peered":"2023-10-16T11:24:40.915755+0000","last_clean":"2023-10-16T11:24:40.915755+0000","last_became_active":"2023-10-16T11:24:26.091801+0000","last_became_peered":"2023-10-16T11:24:26.091801+0000","last_unstale":"2023-10-16T11:24:40.915755+0000","last_undegraded":"2023-10-16T11:24:40.915755+0000","last_fullsized":"2023-10-16T11:24:40.915755+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:43:28.275746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072745+0000","last_change":"2023-10-16T11:24:40.934530+0000","last_active":"2023-10-16T11:24:42.072745+0000","last_peered":"2023-10-16T11:24:42.072745+0000","last_clean":"2023-10-16T11:24:42.072745+0000","last_became_active":"2023-10-16T11:24:40.934262+0000","last_became_peered":"2023-10-16T11:24:40.934262+0000","last_unstale":"2023-10-16T11:24:42.072745+0000","last_undegraded":"2023-10-16T11:24:42.072745+0000","last_fullsized":"2023-10-16T11:24:42.072745+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:30:07.384687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920507+0000","last_change":"2023-10-16T11:24:26.086807+0000","last_active":"2023-10-16T11:24:40.920507+0000","last_peered":"2023-10-16T11:24:40.920507+0000","last_clean":"2023-10-16T11:24:40.920507+0000","last_became_active":"2023-10-16T11:24:26.085382+0000","last_became_peered":"2023-10-16T11:24:26.085382+0000","last_unstale":"2023-10-16T11:24:40.920507+0000","last_undegraded":"2023-10-16T11:24:40.920507+0000","last_fullsized":"2023-10-16T11:24:40.920507+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:56.195463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.071952+0000","last_change":"2023-10-16T11:24:40.931627+0000","last_active":"2023-10-16T11:24:42.071952+0000","last_peered":"2023-10-16T11:24:42.071952+0000","last_clean":"2023-10-16T11:24:42.071952+0000","last_became_active":"2023-10-16T11:24:40.931219+0000","last_became_peered":"2023-10-16T11:24:40.931219+0000","last_unstale":"2023-10-16T11:24:42.071952+0000","last_undegraded":"2023-10-16T11:24:42.071952+0000","last_fullsized":"2023-10-16T11:24:42.071952+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:04.153171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927380+0000","last_change":"2023-10-16T11:24:26.092139+0000","last_active":"2023-10-16T11:24:40.927380+0000","last_peered":"2023-10-16T11:24:40.927380+0000","last_clean":"2023-10-16T11:24:40.927380+0000","last_became_active":"2023-10-16T11:24:26.091860+0000","last_became_peered":"2023-10-16T11:24:26.091860+0000","last_unstale":"2023-10-16T11:24:40.927380+0000","last_undegraded":"2023-10-16T11:24:40.927380+0000","last_fullsized":"2023-10-16T11:24:40.927380+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:00:31.371733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076871+0000","last_change":"2023-10-16T11:24:40.939792+0000","last_active":"2023-10-16T11:24:42.076871+0000","last_peered":"2023-10-16T11:24:42.076871+0000","last_clean":"2023-10-16T11:24:42.076871+0000","last_became_active":"2023-10-16T11:24:40.939547+0000","last_became_peered":"2023-10-16T11:24:40.939547+0000","last_unstale":"2023-10-16T11:24:42.076871+0000","last_undegraded":"2023-10-16T11:24:42.076871+0000","last_fullsized":"2023-10-16T11:24:42.076871+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:44:48.635235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.919569+0000","last_change":"2023-10-16T11:24:26.086029+0000","last_active":"2023-10-16T11:24:40.919569+0000","last_peered":"2023-10-16T11:24:40.919569+0000","last_clean":"2023-10-16T11:24:40.919569+0000","last_became_active":"2023-10-16T11:24:26.085644+0000","last_became_peered":"2023-10-16T11:24:26.085644+0000","last_unstale":"2023-10-16T11:24:40.919569+0000","last_undegraded":"2023-10-16T11:24:40.919569+0000","last_fullsized":"2023-10-16T11:24:40.919569+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:39.802117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.934967+0000","last_change":"2023-10-16T11:24:40.940881+0000","last_active":"2023-10-16T11:24:41.934967+0000","last_peered":"2023-10-16T11:24:41.934967+0000","last_clean":"2023-10-16T11:24:41.934967+0000","last_became_active":"2023-10-16T11:24:40.940465+0000","last_became_peered":"2023-10-16T11:24:40.940465+0000","last_unstale":"2023-10-16T11:24:41.934967+0000","last_undegraded":"2023-10-16T11:24:41.934967+0000","last_fullsized":"2023-10-16T11:24:41.934967+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:39:05.461904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.924194+0000","last_change":"2023-10-16T11:24:26.081938+0000","last_active":"2023-10-16T11:24:40.924194+0000","last_peered":"2023-10-16T11:24:40.924194+0000","last_clean":"2023-10-16T11:24:40.924194+0000","last_became_active":"2023-10-16T11:24:26.081671+0000","last_became_peered":"2023-10-16T11:24:26.081671+0000","last_unstale":"2023-10-16T11:24:40.924194+0000","last_undegraded":"2023-10-16T11:24:40.924194+0000","last_fullsized":"2023-10-16T11:24:40.924194+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:17.072626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.080281+0000","last_change":"2023-10-16T11:24:40.944207+0000","last_active":"2023-10-16T11:24:42.080281+0000","last_peered":"2023-10-16T11:24:42.080281+0000","last_clean":"2023-10-16T11:24:42.080281+0000","last_became_active":"2023-10-16T11:24:40.943832+0000","last_became_peered":"2023-10-16T11:24:40.943832+0000","last_unstale":"2023-10-16T11:24:42.080281+0000","last_undegraded":"2023-10-16T11:24:42.080281+0000","last_fullsized":"2023-10-16T11:24:42.080281+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:43.979587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917335+0000","last_change":"2023-10-16T11:24:26.091403+0000","last_active":"2023-10-16T11:24:40.917335+0000","last_peered":"2023-10-16T11:24:40.917335+0000","last_clean":"2023-10-16T11:24:40.917335+0000","last_became_active":"2023-10-16T11:24:26.091114+0000","last_became_peered":"2023-10-16T11:24:26.091114+0000","last_unstale":"2023-10-16T11:24:40.917335+0000","last_undegraded":"2023-10-16T11:24:40.917335+0000","last_fullsized":"2023-10-16T11:24:40.917335+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935841+0000","last_change":"2023-10-16T11:24:40.936062+0000","last_active":"2023-10-16T11:24:41.935841+0000","last_peered":"2023-10-16T11:24:41.935841+0000","last_clean":"2023-10-16T11:24:41.935841+0000","last_became_active":"2023-10-16T11:24:40.935597+0000","last_became_peered":"2023-10-16T11:24:40.935597+0000","last_unstale":"2023-10-16T11:24:41.935841+0000","last_undegraded":"2023-10-16T11:24:41.935841+0000","last_fullsized":"2023-10-16T11:24:41.935841+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:55.400689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917983+0000","last_change":"2023-10-16T11:24:26.088199+0000","last_active":"2023-10-16T11:24:40.917983+0000","last_peered":"2023-10-16T11:24:40.917983+0000","last_clean":"2023-10-16T11:24:40.917983+0000","last_became_active":"2023-10-16T11:24:26.087892+0000","last_became_peered":"2023-10-16T11:24:26.087892+0000","last_unstale":"2023-10-16T11:24:40.917983+0000","last_undegraded":"2023-10-16T11:24:40.917983+0000","last_fullsized":"2023-10-16T11:24:40.917983+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076815+0000","last_change":"2023-10-16T11:24:40.935695+0000","last_active":"2023-10-16T11:24:42.076815+0000","last_peered":"2023-10-16T11:24:42.076815+0000","last_clean":"2023-10-16T11:24:42.076815+0000","last_became_active":"2023-10-16T11:24:40.935358+0000","last_became_peered":"2023-10-16T11:24:40.935358+0000","last_unstale":"2023-10-16T11:24:42.076815+0000","last_undegraded":"2023-10-16T11:24:42.076815+0000","last_fullsized":"2023-10-16T11:24:42.076815+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:15.346746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.916085+0000","last_change":"2023-10-16T11:24:26.088938+0000","last_active":"2023-10-16T11:24:40.916085+0000","last_peered":"2023-10-16T11:24:40.916085+0000","last_clean":"2023-10-16T11:24:40.916085+0000","last_became_active":"2023-10-16T11:24:26.088689+0000","last_became_peered":"2023-10-16T11:24:26.088689+0000","last_unstale":"2023-10-16T11:24:40.916085+0000","last_undegraded":"2023-10-16T11:24:40.916085+0000","last_fullsized":"2023-10-16T11:24:40.916085+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:18:12.479766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076896+0000","last_change":"2023-10-16T11:24:40.941713+0000","last_active":"2023-10-16T11:24:42.076896+0000","last_peered":"2023-10-16T11:24:42.076896+0000","last_clean":"2023-10-16T11:24:42.076896+0000","last_became_active":"2023-10-16T11:24:40.938875+0000","last_became_peered":"2023-10-16T11:24:40.938875+0000","last_unstale":"2023-10-16T11:24:42.076896+0000","last_undegraded":"2023-10-16T11:24:42.076896+0000","last_fullsized":"2023-10-16T11:24:42.076896+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:42.730933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.915139+0000","last_change":"2023-10-16T11:24:26.089768+0000","last_active":"2023-10-16T11:24:40.915139+0000","last_peered":"2023-10-16T11:24:40.915139+0000","last_clean":"2023-10-16T11:24:40.915139+0000","last_became_active":"2023-10-16T11:24:26.087028+0000","last_became_peered":"2023-10-16T11:24:26.087028+0000","last_unstale":"2023-10-16T11:24:40.915139+0000","last_undegraded":"2023-10-16T11:24:40.915139+0000","last_fullsized":"2023-10-16T11:24:40.915139+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:33.050118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935632+0000","last_change":"2023-10-16T11:24:40.931431+0000","last_active":"2023-10-16T11:24:41.935632+0000","last_peered":"2023-10-16T11:24:41.935632+0000","last_clean":"2023-10-16T11:24:41.935632+0000","last_became_active":"2023-10-16T11:24:40.930331+0000","last_became_peered":"2023-10-16T11:24:40.930331+0000","last_unstale":"2023-10-16T11:24:41.935632+0000","last_undegraded":"2023-10-16T11:24:41.935632+0000","last_fullsized":"2023-10-16T11:24:41.935632+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:18.688225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.908959+0000","last_change":"2023-10-16T11:24:26.077430+0000","last_active":"2023-10-16T11:24:40.908959+0000","last_peered":"2023-10-16T11:24:40.908959+0000","last_clean":"2023-10-16T11:24:40.908959+0000","last_became_active":"2023-10-16T11:24:26.077145+0000","last_became_peered":"2023-10-16T11:24:26.077145+0000","last_unstale":"2023-10-16T11:24:40.908959+0000","last_undegraded":"2023-10-16T11:24:40.908959+0000","last_fullsized":"2023-10-16T11:24:40.908959+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935229+0000","last_change":"2023-10-16T11:24:40.932225+0000","last_active":"2023-10-16T11:24:41.935229+0000","last_peered":"2023-10-16T11:24:41.935229+0000","last_clean":"2023-10-16T11:24:41.935229+0000","last_became_active":"2023-10-16T11:24:40.931846+0000","last_became_peered":"2023-10-16T11:24:40.931846+0000","last_unstale":"2023-10-16T11:24:41.935229+0000","last_undegraded":"2023-10-16T11:24:41.935229+0000","last_fullsized":"2023-10-16T11:24:41.935229+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:23.266881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927275+0000","last_change":"2023-10-16T11:24:26.088202+0000","last_active":"2023-10-16T11:24:40.927275+0000","last_peered":"2023-10-16T11:24:40.927275+0000","last_clean":"2023-10-16T11:24:40.927275+0000","last_became_active":"2023-10-16T11:24:26.087929+0000","last_became_peered":"2023-10-16T11:24:26.087929+0000","last_unstale":"2023-10-16T11:24:40.927275+0000","last_undegraded":"2023-10-16T11:24:40.927275+0000","last_fullsized":"2023-10-16T11:24:40.927275+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:22.009042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.070991+0000","last_change":"2023-10-16T11:24:40.941405+0000","last_active":"2023-10-16T11:24:42.070991+0000","last_peered":"2023-10-16T11:24:42.070991+0000","last_clean":"2023-10-16T11:24:42.070991+0000","last_became_active":"2023-10-16T11:24:40.941060+0000","last_became_peered":"2023-10-16T11:24:40.941060+0000","last_unstale":"2023-10-16T11:24:42.070991+0000","last_undegraded":"2023-10-16T11:24:42.070991+0000","last_fullsized":"2023-10-16T11:24:42.070991+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:22:53.973191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.933068+0000","last_change":"2023-10-16T11:24:26.080292+0000","last_active":"2023-10-16T11:24:40.933068+0000","last_peered":"2023-10-16T11:24:40.933068+0000","last_clean":"2023-10-16T11:24:40.933068+0000","last_became_active":"2023-10-16T11:24:26.080004+0000","last_became_peered":"2023-10-16T11:24:26.080004+0000","last_unstale":"2023-10-16T11:24:40.933068+0000","last_undegraded":"2023-10-16T11:24:40.933068+0000","last_fullsized":"2023-10-16T11:24:40.933068+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.936334+0000","last_change":"2023-10-16T11:24:40.946129+0000","last_active":"2023-10-16T11:24:41.936334+0000","last_peered":"2023-10-16T11:24:41.936334+0000","last_clean":"2023-10-16T11:24:41.936334+0000","last_became_active":"2023-10-16T11:24:40.944566+0000","last_became_peered":"2023-10-16T11:24:40.944566+0000","last_unstale":"2023-10-16T11:24:41.936334+0000","last_undegraded":"2023-10-16T11:24:41.936334+0000","last_fullsized":"2023-10-16T11:24:41.936334+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:38.047063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912578+0000","last_change":"2023-10-16T11:24:26.098845+0000","last_active":"2023-10-16T11:24:40.912578+0000","last_peered":"2023-10-16T11:24:40.912578+0000","last_clean":"2023-10-16T11:24:40.912578+0000","last_became_active":"2023-10-16T11:24:26.098568+0000","last_became_peered":"2023-10-16T11:24:26.098568+0000","last_unstale":"2023-10-16T11:24:40.912578+0000","last_undegraded":"2023-10-16T11:24:40.912578+0000","last_fullsized":"2023-10-16T11:24:40.912578+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:15.909098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3f","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.079804+0000","last_change":"2023-10-16T11:24:40.942060+0000","last_active":"2023-10-16T11:24:42.079804+0000","last_peered":"2023-10-16T11:24:42.079804+0000","last_clean":"2023-10-16T11:24:42.079804+0000","last_became_active":"2023-10-16T11:24:40.941339+0000","last_became_peered":"2023-10-16T11:24:40.941339+0000","last_unstale":"2023-10-16T11:24:42.079804+0000","last_undegraded":"2023-10-16T11:24:42.079804+0000","last_fullsized":"2023-10-16T11:24:42.079804+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:44:34.145225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917233+0000","last_change":"2023-10-16T11:24:26.096348+0000","last_active":"2023-10-16T11:24:40.917233+0000","last_peered":"2023-10-16T11:24:40.917233+0000","last_clean":"2023-10-16T11:24:40.917233+0000","last_became_active":"2023-10-16T11:24:26.093712+0000","last_became_peered":"2023-10-16T11:24:26.093712+0000","last_unstale":"2023-10-16T11:24:40.917233+0000","last_undegraded":"2023-10-16T11:24:40.917233+0000","last_fullsized":"2023-10-16T11:24:40.917233+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:40.970357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.073372+0000","last_change":"2023-10-16T11:24:40.942740+0000","last_active":"2023-10-16T11:24:42.073372+0000","last_peered":"2023-10-16T11:24:42.073372+0000","last_clean":"2023-10-16T11:24:42.073372+0000","last_became_active":"2023-10-16T11:24:40.939522+0000","last_became_peered":"2023-10-16T11:24:40.939522+0000","last_unstale":"2023-10-16T11:24:42.073372+0000","last_undegraded":"2023-10-16T11:24:42.073372+0000","last_fullsized":"2023-10-16T11:24:42.073372+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:31:23.849707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912900+0000","last_change":"2023-10-16T11:24:26.083687+0000","last_active":"2023-10-16T11:24:40.912900+0000","last_peered":"2023-10-16T11:24:40.912900+0000","last_clean":"2023-10-16T11:24:40.912900+0000","last_became_active":"2023-10-16T11:24:26.083213+0000","last_became_peered":"2023-10-16T11:24:26.083213+0000","last_unstale":"2023-10-16T11:24:40.912900+0000","last_undegraded":"2023-10-16T11:24:40.912900+0000","last_fullsized":"2023-10-16T11:24:40.912900+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:08:02.966132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.081257+0000","last_change":"2023-10-16T11:24:40.917853+0000","last_active":"2023-10-16T11:24:42.081257+0000","last_peered":"2023-10-16T11:24:42.081257+0000","last_clean":"2023-10-16T11:24:42.081257+0000","last_became_active":"2023-10-16T11:24:40.917561+0000","last_became_peered":"2023-10-16T11:24:40.917561+0000","last_unstale":"2023-10-16T11:24:42.081257+0000","last_undegraded":"2023-10-16T11:24:42.081257+0000","last_fullsized":"2023-10-16T11:24:42.081257+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:55.419058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.914944+0000","last_change":"2023-10-16T11:24:26.096814+0000","last_active":"2023-10-16T11:24:40.914944+0000","last_peered":"2023-10-16T11:24:40.914944+0000","last_clean":"2023-10-16T11:24:40.914944+0000","last_became_active":"2023-10-16T11:24:26.095777+0000","last_became_peered":"2023-10-16T11:24:26.095777+0000","last_unstale":"2023-10-16T11:24:40.914944+0000","last_undegraded":"2023-10-16T11:24:40.914944+0000","last_fullsized":"2023-10-16T11:24:40.914944+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:08.889514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.933544+0000","last_change":"2023-10-16T11:24:40.935780+0000","last_active":"2023-10-16T11:24:41.933544+0000","last_peered":"2023-10-16T11:24:41.933544+0000","last_clean":"2023-10-16T11:24:41.933544+0000","last_became_active":"2023-10-16T11:24:40.935476+0000","last_became_peered":"2023-10-16T11:24:40.935476+0000","last_unstale":"2023-10-16T11:24:41.933544+0000","last_undegraded":"2023-10-16T11:24:41.933544+0000","last_fullsized":"2023-10-16T11:24:41.933544+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:54:11.551791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.922006+0000","last_change":"2023-10-16T11:24:26.091440+0000","last_active":"2023-10-16T11:24:40.922006+0000","last_peered":"2023-10-16T11:24:40.922006+0000","last_clean":"2023-10-16T11:24:40.922006+0000","last_became_active":"2023-10-16T11:24:26.088757+0000","last_became_peered":"2023-10-16T11:24:26.088757+0000","last_unstale":"2023-10-16T11:24:40.922006+0000","last_undegraded":"2023-10-16T11:24:40.922006+0000","last_fullsized":"2023-10-16T11:24:40.922006+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:27.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072970+0000","last_change":"2023-10-16T11:24:40.933226+0000","last_active":"2023-10-16T11:24:42.072970+0000","last_peered":"2023-10-16T11:24:42.072970+0000","last_clean":"2023-10-16T11:24:42.072970+0000","last_became_active":"2023-10-16T11:24:40.932168+0000","last_became_peered":"2023-10-16T11:24:40.932168+0000","last_unstale":"2023-10-16T11:24:42.072970+0000","last_undegraded":"2023-10-16T11:24:42.072970+0000","last_fullsized":"2023-10-16T11:24:42.072970+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:51:44.814417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.908841+0000","last_change":"2023-10-16T11:24:26.085606+0000","last_active":"2023-10-16T11:24:40.908841+0000","last_peered":"2023-10-16T11:24:40.908841+0000","last_clean":"2023-10-16T11:24:40.908841+0000","last_became_active":"2023-10-16T11:24:26.085318+0000","last_became_peered":"2023-10-16T11:24:26.085318+0000","last_unstale":"2023-10-16T11:24:40.908841+0000","last_undegraded":"2023-10-16T11:24:40.908841+0000","last_fullsized":"2023-10-16T11:24:40.908841+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.931913+0000","last_change":"2023-10-16T11:24:40.937629+0000","last_active":"2023-10-16T11:24:41.931913+0000","last_peered":"2023-10-16T11:24:41.931913+0000","last_clean":"2023-10-16T11:24:41.931913+0000","last_became_active":"2023-10-16T11:24:40.937340+0000","last_became_peered":"2023-10-16T11:24:40.937340+0000","last_unstale":"2023-10-16T11:24:41.931913+0000","last_undegraded":"2023-10-16T11:24:41.931913+0000","last_fullsized":"2023-10-16T11:24:41.931913+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:13.135380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.932956+0000","last_change":"2023-10-16T11:24:26.089885+0000","last_active":"2023-10-16T11:24:40.932956+0000","last_peered":"2023-10-16T11:24:40.932956+0000","last_clean":"2023-10-16T11:24:40.932956+0000","last_became_active":"2023-10-16T11:24:26.089406+0000","last_became_peered":"2023-10-16T11:24:26.089406+0000","last_unstale":"2023-10-16T11:24:40.932956+0000","last_undegraded":"2023-10-16T11:24:40.932956+0000","last_fullsized":"2023-10-16T11:24:40.932956+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.075155+0000","last_change":"2023-10-16T11:24:40.926983+0000","last_active":"2023-10-16T11:24:42.075155+0000","last_peered":"2023-10-16T11:24:42.075155+0000","last_clean":"2023-10-16T11:24:42.075155+0000","last_became_active":"2023-10-16T11:24:40.923594+0000","last_became_peered":"2023-10-16T11:24:40.923594+0000","last_unstale":"2023-10-16T11:24:42.075155+0000","last_undegraded":"2023-10-16T11:24:42.075155+0000","last_fullsized":"2023-10-16T11:24:42.075155+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:54.635909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.919488+0000","last_change":"2023-10-16T11:24:26.086488+0000","last_active":"2023-10-16T11:24:40.919488+0000","last_peered":"2023-10-16T11:24:40.919488+0000","last_clean":"2023-10-16T11:24:40.919488+0000","last_became_active":"2023-10-16T11:24:26.085969+0000","last_became_peered":"2023-10-16T11:24:26.085969+0000","last_unstale":"2023-10-16T11:24:40.919488+0000","last_undegraded":"2023-10-16T11:24:40.919488+0000","last_fullsized":"2023-10-16T11:24:40.919488+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":581,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2767,"ondisk_log_size":2767,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575989,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52532,"kb_used_data":3304,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319308,"statfs":{"total":96636764160,"available":96582971392,"internally_reserved":0,"allocated":3383296,"data_stored":1942477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.179,"15min":1.2170000000000001},"min":{"1min":0.81499999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.7250000000000001,"5min":2.085,"15min":2.3399999999999999},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.1719999999999999,"15min":1.218},"min":{"1min":0.876,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.8320000000000001},"last":1.448}]},{"osd":1,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2689999999999999,"15min":1.2509999999999999},"min":{"1min":0.90700000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.6479999999999999,"5min":1.913,"15min":1.913},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2430000000000001,"15min":1.2749999999999999},"min":{"1min":0.76700000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6279999999999999,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.95699999999999996}]},{"osd":2,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3140000000000001,"15min":1.3109999999999999},"min":{"1min":1.008,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6579999999999999,"5min":1.8680000000000001,"15min":2.4460000000000002},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.3540000000000001,"15min":1.3640000000000001},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.708,"5min":1.859,"15min":1.974},"last":1.083}]},{"osd":3,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.381,"15min":1.3899999999999999},"min":{"1min":1.149,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7290000000000001,"5min":1.8280000000000001,"15min":1.972},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.4430000000000001,"15min":1.4630000000000001},"min":{"1min":1.0860000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.8799999999999999,"5min":2.0680000000000001,"15min":2.577},"last":1.248}]},{"osd":4,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.423,"15min":1.417},"min":{"1min":0.98599999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0030000000000001,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.498,"5min":1.391,"15min":1.3899999999999999},"min":{"1min":1.0469999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8280000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.149}]},{"osd":5,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.585,"15min":1.5960000000000001},"min":{"1min":1.1579999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.946,"5min":2.2309999999999999,"15min":3.3519999999999999},"last":1.7190000000000001},{"interface":"front","average":{"1min":1.641,"5min":1.514,"15min":1.5089999999999999},"min":{"1min":1.351,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.1600000000000001,"5min":2.3460000000000001,"15min":2.4969999999999999},"last":1.3109999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.488,"15min":1.5209999999999999},"min":{"1min":0.97099999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.845,"5min":2.101,"15min":2.7250000000000001},"last":1.351},{"interface":"front","average":{"1min":1.7210000000000001,"5min":1.6890000000000001,"15min":1.71},"min":{"1min":1.425,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1040000000000001,"5min":2.298,"15min":3.1440000000000001},"last":1.661}]}]},{"osd":6,"up_from":13,"seq":55834575989,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33904,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337936,"statfs":{"total":96636764160,"available":96602046464,"internally_reserved":0,"allocated":2785280,"data_stored":1349820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.2350000000000001,"15min":1.2250000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.64000000000000001},"max":{"1min":6.298,"5min":6.298,"15min":6.298},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.3380000000000001,"15min":1.2909999999999999},"min":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.66400000000000003},"max":{"1min":8.2650000000000006,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":1.5409999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2450000000000001,"15min":1.222},"min":{"1min":0.80200000000000005,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.88},{"interface":"front","average":{"1min":1.8520000000000001,"5min":1.4510000000000001,"15min":1.3859999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":12.468,"5min":12.468,"15min":12.468},"last":1.411}]},{"osd":2,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.7889999999999999,"5min":1.337,"15min":1.2829999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":11.586,"5min":11.586,"15min":11.586},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.569,"5min":1.3799999999999999,"15min":1.369},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.77500000000000002},"max":{"1min":4.7030000000000003,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.1879999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.9239999999999999,"5min":1.452,"15min":1.353},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":12.801,"5min":12.801,"15min":12.801},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.302,"15min":1.284},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":9.1539999999999999,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":1.216}]},{"osd":4,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.3700000000000001,"15min":1.3220000000000001},"min":{"1min":0.65400000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.369},{"interface":"front","average":{"1min":2.141,"5min":1.609,"15min":1.548},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":13.981999999999999,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.1180000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":2.246,"5min":1.5109999999999999,"15min":1.391},"min":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":18.247,"5min":18.247,"15min":18.247},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.8360000000000001,"5min":1.5900000000000001,"15min":1.5629999999999999},"min":{"1min":0.76200000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":6.4039999999999999,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.4930000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":2.0609999999999999,"5min":1.603,"15min":1.5349999999999999},"min":{"1min":0.749,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":12.589,"5min":12.589,"15min":12.589},"last":1.27},{"interface":"front","average":{"1min":2.1749999999999998,"5min":1.6739999999999999,"15min":1.595},"min":{"1min":0.78200000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":12.725,"5min":12.725,"15min":12.725},"last":1.456}]}]},{"osd":5,"up_from":13,"seq":55834575989,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29736,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342104,"statfs":{"total":96636764160,"available":96606314496,"internally_reserved":0,"allocated":2777088,"data_stored":1349764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2350000000000001,"15min":1.27},"min":{"1min":0.74199999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.583,"5min":2.661,"15min":2.661},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.3029999999999999,"15min":1.349},"min":{"1min":0.95799999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7909999999999999,"5min":1.7949999999999999,"15min":1.865},"last":1.25}]},{"osd":1,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1819999999999999,"15min":1.1950000000000001},"min":{"1min":0.81999999999999995,"5min":0.80800000000000005,"15min":0.79100000000000004},"max":{"1min":1.5009999999999999,"5min":1.7869999999999999,"15min":1.992},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.258,"15min":1.2290000000000001},"min":{"1min":0.84399999999999997,"5min":0.84199999999999997,"15min":0.80800000000000005},"max":{"1min":1.635,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.6739999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.264,"15min":1.244},"min":{"1min":0.97899999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.625,"5min":1.952,"15min":1.952},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.329,"5min":1.2949999999999999,"15min":1.3220000000000001},"min":{"1min":0.92400000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":2.1640000000000001},"last":1.0680000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3320000000000001,"15min":1.3340000000000001},"min":{"1min":0.98999999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":3.012,"15min":3.012},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.429,"15min":1.425},"min":{"1min":1.0900000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":2.2250000000000001},"last":1.2010000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.339,"15min":1.339},"min":{"1min":0.94199999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.6990000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.6799999999999999,"5min":1.5720000000000001,"15min":1.5740000000000001},"min":{"1min":1.353,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.948,"5min":2.903,"15min":2.903},"last":1.3879999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4330000000000001,"15min":1.4139999999999999},"min":{"1min":0.873,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8660000000000001,"5min":2.0649999999999999,"15min":2.0979999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.51,"5min":1.494,"15min":1.4770000000000001},"min":{"1min":0.95399999999999996,"5min":0.90700000000000003,"15min":0.78400000000000003},"max":{"1min":1.841,"5min":3.073,"15min":3.073},"last":1.607}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.675,"5min":1.5660000000000001,"15min":1.5469999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0779999999999998,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.579},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.645,"15min":1.645},"min":{"1min":1.1779999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.1099999999999999,"5min":2.1459999999999999,"15min":2.2930000000000001},"last":1.4370000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575990,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51960,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319880,"statfs":{"total":96636764160,"available":96583557120,"internally_reserved":0,"allocated":2797568,"data_stored":1352725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1519999999999999,"15min":1.1859999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.7290000000000001,"5min":1.792,"15min":1.792},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.218,"15min":1.2350000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.9299999999999999},"last":1.1599999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2769999999999999,"15min":1.298},"min":{"1min":0.81799999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.841,"5min":1.841,"15min":1.913},"last":1.409},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.236,"15min":1.25},"min":{"1min":0.78500000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.877,"5min":1.877,"15min":1.889},"last":1.1020000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.29,"15min":1.3089999999999999},"min":{"1min":1.0369999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.835},"last":1.302},{"interface":"front","average":{"1min":1.351,"5min":1.3999999999999999,"15min":1.403},"min":{"1min":0.879,"5min":0.879,"15min":0.872},"max":{"1min":1.8380000000000001,"5min":1.903,"15min":1.9139999999999999},"last":1.6339999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3169999999999999,"15min":1.337},"min":{"1min":0.91100000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.6830000000000001,"5min":1.8280000000000001,"15min":1.849},"last":1.595},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4410000000000001,"15min":1.4330000000000001},"min":{"1min":1.145,"5min":1.008,"15min":0.96999999999999997},"max":{"1min":1.7769999999999999,"5min":1.865,"15min":2.0720000000000001},"last":1.6739999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.4810000000000001,"15min":1.4399999999999999},"min":{"1min":1.0149999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9399999999999999,"5min":2.105,"15min":2.105},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.431,"15min":1.415},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.78600000000000003},"max":{"1min":1.8240000000000001,"5min":2.04,"15min":2.04},"last":1.4590000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.5409999999999999,"15min":1.5029999999999999},"min":{"1min":0.94299999999999995,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":3.157,"5min":3.157,"15min":3.157},"last":1.528},{"interface":"front","average":{"1min":1.5,"5min":1.6100000000000001,"15min":1.611},"min":{"1min":1.0940000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0089999999999999,"5min":2.2330000000000001,"15min":2.2770000000000001},"last":1.498}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.5640000000000001,"15min":1.5660000000000001},"min":{"1min":1.0549999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.855,"5min":2.113,"15min":2.3420000000000001},"last":1.375},{"interface":"front","average":{"1min":1.639,"5min":1.6619999999999999,"15min":1.6399999999999999},"min":{"1min":1.0660000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9630000000000001,"5min":2.1469999999999998,"15min":2.3839999999999999},"last":1.756}]}]},{"osd":3,"up_from":13,"seq":55834575989,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47812,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324028,"statfs":{"total":96636764160,"available":96587804672,"internally_reserved":0,"allocated":2801664,"data_stored":1351046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.89400000000000002,"15min":0.90700000000000003},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.478,"5min":1.478,"15min":1.901},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0369999999999999,"15min":1.0720000000000001},"min":{"1min":0.59299999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.391,"5min":1.6180000000000001,"15min":1.6679999999999999},"last":1.0660000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.95599999999999996,"15min":0.98799999999999999},"min":{"1min":0.60799999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.105,"15min":1.133},"min":{"1min":0.48399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.601,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.88100000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0760000000000001,"15min":1.117},"min":{"1min":0.65500000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.524,"5min":1.694,"15min":1.694},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.198,"15min":1.236},"min":{"1min":0.625,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.91400000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3169999999999999,"15min":1.3500000000000001},"min":{"1min":0.83599999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.806,"5min":1.806,"15min":1.873},"last":1.466},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.357,"15min":1.367},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.8819999999999999},"last":1.2689999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.385,"15min":1.401},"min":{"1min":0.80600000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.516},{"interface":"front","average":{"1min":1.419,"5min":1.4059999999999999,"15min":1.444},"min":{"1min":1.151,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.552}]},{"osd":6,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.504,"15min":1.5329999999999999},"min":{"1min":0.876,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":3.9129999999999998,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.657},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5,"15min":1.5369999999999999},"min":{"1min":1.153,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":3.9319999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.4910000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.528,"15min":1.5600000000000001},"min":{"1min":0.95399999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.877,"5min":2.0790000000000002,"15min":2.2679999999999998},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.639,"15min":1.673},"min":{"1min":1.1279999999999999,"5min":1,"15min":1},"max":{"1min":2.052,"5min":2.198,"15min":2.5129999999999999},"last":1.6739999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575988,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51964,"kb_used_data":2728,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319876,"statfs":{"total":96636764160,"available":96583553024,"internally_reserved":0,"allocated":2793472,"data_stored":1350850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.91000000000000003,"15min":0.92200000000000004},"min":{"1min":0.74199999999999999,"5min":0.51900000000000002,"15min":0.5},"max":{"1min":1.532,"5min":1.532,"15min":1.669},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.0569999999999999,"15min":1.0820000000000001},"min":{"1min":0.72499999999999998,"5min":0.47699999999999998,"15min":0.32600000000000001},"max":{"1min":1.496,"5min":1.681,"15min":2.1429999999999998},"last":0.69799999999999995}]},{"osd":1,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.089,"15min":1.097},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.39200000000000002},"max":{"1min":1.415,"5min":1.6779999999999999,"15min":1.978},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1479999999999999,"15min":1.165},"min":{"1min":0.57599999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.448,"5min":1.609,"15min":1.869},"last":0.99299999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.0649999999999999,"15min":1.087},"min":{"1min":0.54600000000000004,"5min":0.437,"15min":0.437},"max":{"1min":1.464,"5min":1.651,"15min":1.8560000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.252,"15min":1.274},"min":{"1min":0.82199999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6120000000000001,"5min":1.7450000000000001,"15min":1.857},"last":1.1930000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3740000000000001,"15min":1.415},"min":{"1min":0.91500000000000004,"5min":0.86599999999999999,"15min":0.73499999999999999},"max":{"1min":1.6910000000000001,"5min":1.8029999999999999,"15min":2.5059999999999998},"last":1.145},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3440000000000001,"15min":1.3520000000000001},"min":{"1min":0.97899999999999998,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":1.825,"5min":1.825,"15min":2.0880000000000001},"last":0.97599999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.444,"15min":1.464},"min":{"1min":1.0720000000000001,"5min":0.84999999999999998,"15min":0.78500000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2269999999999999},"last":1.345},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4550000000000001,"15min":1.4890000000000001},"min":{"1min":1.0509999999999999,"5min":0.95799999999999996,"15min":0.874},"max":{"1min":1.798,"5min":1.931,"15min":2.6259999999999999},"last":1.258}]},{"osd":6,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4119999999999999,"15min":1.411},"min":{"1min":1.0569999999999999,"5min":0.84099999999999997,"15min":0.80500000000000005},"max":{"1min":1.71,"5min":1.9019999999999999,"15min":2.2109999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5,"15min":1.5369999999999999},"min":{"1min":1.1899999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.2909999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.417}]},{"osd":7,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":1.5489999999999999,"15min":1.5860000000000001},"min":{"1min":1.2290000000000001,"5min":0.995,"15min":0.92500000000000004},"max":{"1min":2.1469999999999998,"5min":2.262,"15min":2.8109999999999999},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.494,"5min":1.4870000000000001,"15min":1.5149999999999999},"min":{"1min":1.242,"5min":0.81599999999999995,"15min":0.80100000000000005},"max":{"1min":1.8180000000000001,"5min":2.2269999999999999,"15min":2.2829999999999999},"last":1.323}]}]},{"osd":1,"up_from":13,"seq":55834575988,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60272,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311568,"statfs":{"total":96636764160,"available":96575045632,"internally_reserved":0,"allocated":2785280,"data_stored":1349328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.91900000000000004,"15min":0.91500000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.575,"5min":1.575,"15min":1.7929999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0289999999999999,"15min":1.0580000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.494,"5min":1.5509999999999999,"15min":1.667},"last":0.60899999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.98399999999999999,"15min":1.0189999999999999},"min":{"1min":0.64300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.72},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.252,"15min":1.276},"min":{"1min":0.69799999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":1.4379999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.1879999999999999,"15min":1.216},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.774,"5min":1.774,"15min":1.79},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.169,"15min":1.1739999999999999},"min":{"1min":0.66700000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.633,"5min":1.633,"15min":1.802},"last":0.67500000000000004}]},{"osd":4,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.2609999999999999,"15min":1.274},"min":{"1min":0.86299999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.653,"5min":1.887,"15min":1.9159999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.409,"15min":1.415},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.887,"5min":2.0619999999999998,"15min":2.093},"last":1.23}]},{"osd":5,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.379,"15min":1.409},"min":{"1min":0.93700000000000006,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.948,"5min":2.0009999999999999,"15min":2.0640000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4239999999999999,"15min":1.421},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":1.825,"5min":1.825,"15min":2.0649999999999999},"last":1.141}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4850000000000001,"15min":1.496},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.879,"5min":2.1970000000000001,"15min":2.2949999999999999},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4710000000000001,"15min":1.486},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.891,"5min":1.891,"15min":2.1970000000000001},"last":1.0349999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5649999999999999,"15min":1.5620000000000001},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.1720000000000002},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.625,"15min":1.607},"min":{"1min":1.1080000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.9359999999999999,"5min":2.1989999999999998,"15min":2.2810000000000001},"last":1.367}]}]},{"osd":0,"up_from":13,"seq":55834575989,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52532,"kb_used_data":3300,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319308,"statfs":{"total":96636764160,"available":96582971392,"internally_reserved":0,"allocated":3379200,"data_stored":1940312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.871,"15min":0.90000000000000002},"min":{"1min":0.73199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.502,"5min":1.52,"15min":1.619},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.1060000000000001},"min":{"1min":0.627,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.621,"5min":1.6399999999999999,"15min":1.76},"last":1.109}]},{"osd":2,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.097,"15min":1.0860000000000001},"min":{"1min":0.65300000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.556,"5min":1.649,"15min":1.649},"last":1.21},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.095,"15min":1.1459999999999999},"min":{"1min":0.81100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4179999999999999,"5min":1.623,"15min":1.6870000000000001},"last":1.508}]},{"osd":3,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.056,"15min":1.085},"min":{"1min":0.69499999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.244,"15min":1.256},"min":{"1min":0.91500000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.716,"5min":1.7609999999999999,"15min":1.9830000000000001},"last":1.456}]},{"osd":4,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.387,"15min":1.389},"min":{"1min":1.129,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.681,"5min":1.9390000000000001,"15min":1.95},"last":1.611},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3180000000000001,"15min":1.3500000000000001},"min":{"1min":1.0329999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.645,"5min":2.1539999999999999,"15min":2.4820000000000002},"last":1.252}]},{"osd":5,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3660000000000001,"15min":1.3879999999999999},"min":{"1min":0.996,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":1.829,"5min":2.0750000000000002,"15min":2.6669999999999998},"last":1.401},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.454},"min":{"1min":1.028,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.748,"5min":2.2029999999999998,"15min":2.2040000000000002},"last":1.639}]},{"osd":6,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.4750000000000001,"15min":1.5029999999999999},"min":{"1min":1.1080000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.819,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.405,"5min":1.4630000000000001,"15min":1.492},"min":{"1min":0.93799999999999994,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.758,"5min":2.262,"15min":2.262},"last":1.7170000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.52,"15min":1.5549999999999999},"min":{"1min":1.2749999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7150000000000001,"5min":2.347,"15min":2.347},"last":1.794},{"interface":"front","average":{"1min":1.454,"5min":1.474,"15min":1.5109999999999999},"min":{"1min":1.129,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.9530000000000001,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.49}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:24:46.456 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-16T11:24:46.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.705+0000 7f12a269b640 1 Processor -- start 2023-10-16T11:24:46.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.706+0000 7f12a269b640 1 -- start start 2023-10-16T11:24:46.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.707+0000 7f12a269b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f129c15ad70 0x7f129c069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:46.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.707+0000 7f12a269b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f129c0c9ca0 0x7f129c06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:46.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.707+0000 7f12a269b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129c0cb030 con 0x7f129c0ca3b0 2023-10-16T11:24:46.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.707+0000 7f12a269b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f129c0c91c0 con 0x7f129c15ad70 2023-10-16T11:24:46.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.707+0000 7f12a269b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129c0c94a0 con 0x7f129c0c9ca0 2023-10-16T11:24:46.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.707+0000 7f129bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f129c15ad70 0x7f129c069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:46.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.707+0000 7f129bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f129c15ad70 0x7f129c069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51290/0 (socket says 172.21.15.84:51290) 2023-10-16T11:24:46.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.707+0000 7f129bfff640 1 -- 172.21.15.84:0/1583401016 learned_addr learned my addr 172.21.15.84:0/1583401016 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:46.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.707+0000 7f12a0c38640 1 --1- 172.21.15.84:0/1583401016 >> v1:172.21.15.84:6789/0 conn(0x7f129c0ca3b0 0x7f129c0ca840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47958/0 (socket says 172.21.15.84:47958) 2023-10-16T11:24:46.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.708+0000 7f129bfff640 1 -- 172.21.15.84:0/1583401016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f129c0c9ca0 msgr2=0x7f129c06a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:46.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.708+0000 7f129b7fe640 1 --2- 172.21.15.84:0/1583401016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f129c0c9ca0 0x7f129c06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:46.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.708+0000 7f129affd640 1 -- 172.21.15.84:0/1583401016 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097629752 0 0) 0x7f129c0cb030 con 0x7f129c0ca3b0 2023-10-16T11:24:46.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.708+0000 7f129bfff640 1 --2- 172.21.15.84:0/1583401016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f129c0c9ca0 0x7f129c06a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.708+0000 7f129bfff640 1 -- 172.21.15.84:0/1583401016 >> v1:172.21.15.84:6789/0 conn(0x7f129c0ca3b0 legacy=0x7f129c0ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:46.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.708+0000 7f129bfff640 1 -- 172.21.15.84:0/1583401016 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c0c9780 con 0x7f129c15ad70 2023-10-16T11:24:46.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.708+0000 7f129b7fe640 1 --2- 172.21.15.84:0/1583401016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f129c0c9ca0 0x7f129c06a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:46.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.709+0000 7f129bfff640 1 --2- 172.21.15.84:0/1583401016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c15ad70 0x7f129c069da0 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f1284009870 tx=0x7f128402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff5fe765e5503d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:46.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.710+0000 7f129affd640 1 -- 172.21.15.84:0/1583401016 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1284030840 con 0x7f129c15ad70 2023-10-16T11:24:46.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.710+0000 7f129affd640 1 -- 172.21.15.84:0/1583401016 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12840309a0 con 0x7f129c15ad70 2023-10-16T11:24:46.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.710+0000 7f129affd640 1 -- 172.21.15.84:0/1583401016 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1284030d30 con 0x7f129c15ad70 2023-10-16T11:24:46.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.710+0000 7f12a269b640 1 -- 172.21.15.84:0/1583401016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c15ad70 msgr2=0x7f129c069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:46.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.710+0000 7f12a269b640 1 --2- 172.21.15.84:0/1583401016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c15ad70 0x7f129c069da0 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f1284009870 tx=0x7f128402b980 comp rx=0 tx=0).stop 2023-10-16T11:24:46.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.710+0000 7f12a269b640 1 -- 172.21.15.84:0/1583401016 shutdown_connections 2023-10-16T11:24:46.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.710+0000 7f12a269b640 1 --2- 172.21.15.84:0/1583401016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f129c0c9ca0 0x7f129c06a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.710+0000 7f12a269b640 1 --2- 172.21.15.84:0/1583401016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c15ad70 0x7f129c069da0 unknown :-1 s=CLOSED pgs=2689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.710+0000 7f12a269b640 1 -- 172.21.15.84:0/1583401016 >> 172.21.15.84:0/1583401016 conn(0x7f129c0ce9b0 msgr2=0x7f129c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:46.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.710+0000 7f12a269b640 1 -- 172.21.15.84:0/1583401016 shutdown_connections 2023-10-16T11:24:46.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.710+0000 7f12a269b640 1 -- 172.21.15.84:0/1583401016 wait complete. 2023-10-16T11:24:46.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.712+0000 7f12a269b640 1 Processor -- start 2023-10-16T11:24:46.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.712+0000 7f12a269b640 1 -- start start 2023-10-16T11:24:46.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.713+0000 7f12a269b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c0c9ca0 0x7f129c14d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:46.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.713+0000 7f12a269b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f129c0ca3b0 0x7f129c0c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:46.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.713+0000 7f12a269b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f129c15ad70 0x7f129c150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:46.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.713+0000 7f12a269b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c14ea20 con 0x7f129c0c9ca0 2023-10-16T11:24:46.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.713+0000 7f12a269b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c14de10 con 0x7f129c15ad70 2023-10-16T11:24:46.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.713+0000 7f12a269b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129c14e0f0 con 0x7f129c0ca3b0 2023-10-16T11:24:46.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.713+0000 7f129bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c0c9ca0 0x7f129c14d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:46.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.713+0000 7f129bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c0c9ca0 0x7f129c14d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51306/0 (socket says 172.21.15.84:51306) 2023-10-16T11:24:46.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.713+0000 7f129bfff640 1 -- 172.21.15.84:0/221228699 learned_addr learned my addr 172.21.15.84:0/221228699 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:46.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.714+0000 7f129bfff640 1 -- 172.21.15.84:0/221228699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f129c0ca3b0 msgr2=0x7f129c0c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:46.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.714+0000 7f129bfff640 1 --2- 172.21.15.84:0/221228699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f129c0ca3b0 0x7f129c0c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.714+0000 7f129bfff640 1 -- 172.21.15.84:0/221228699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f129c15ad70 msgr2=0x7f129c150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:46.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.714+0000 7f129bfff640 1 --2- 172.21.15.84:0/221228699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f129c15ad70 0x7f129c150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.714+0000 7f129bfff640 1 -- 172.21.15.84:0/221228699 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12840094d0 con 0x7f129c0c9ca0 2023-10-16T11:24:46.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.714+0000 7f129bfff640 1 --2- 172.21.15.84:0/221228699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c0c9ca0 0x7f129c14d550 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f1284009870 tx=0x7f12840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:46.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.714+0000 7f1298ff9640 1 -- 172.21.15.84:0/221228699 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128403e070 con 0x7f129c0c9ca0 2023-10-16T11:24:46.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.714+0000 7f1298ff9640 1 -- 172.21.15.84:0/221228699 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1284004030 con 0x7f129c0c9ca0 2023-10-16T11:24:46.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.714+0000 7f1298ff9640 1 -- 172.21.15.84:0/221228699 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12840043c0 con 0x7f129c0c9ca0 2023-10-16T11:24:46.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.714+0000 7f12a269b640 1 -- 172.21.15.84:0/221228699 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129c14e3d0 con 0x7f129c0c9ca0 2023-10-16T11:24:46.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.715+0000 7f12a269b640 1 -- 172.21.15.84:0/221228699 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129c14da90 con 0x7f129c0c9ca0 2023-10-16T11:24:46.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.716+0000 7f12a269b640 1 -- 172.21.15.84:0/221228699 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f129c04e160 con 0x7f129c0c9ca0 2023-10-16T11:24:46.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.717+0000 7f1298ff9640 1 -- 172.21.15.84:0/221228699 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1284042070 con 0x7f129c0c9ca0 2023-10-16T11:24:46.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.718+0000 7f1298ff9640 1 --2- 172.21.15.84:0/221228699 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f127c06af90 0x7f127c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:46.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.718+0000 7f1298ff9640 1 -- 172.21.15.84:0/221228699 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f12840abc40 con 0x7f129c0c9ca0 2023-10-16T11:24:46.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.718+0000 7f129b7fe640 1 --2- 172.21.15.84:0/221228699 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f127c06af90 0x7f127c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:46.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.719+0000 7f129b7fe640 1 --2- 172.21.15.84:0/221228699 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f127c06af90 0x7f127c06d450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f129c06a6c0 tx=0x7f128c0083d0 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:46.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.723+0000 7f1298ff9640 1 -- 172.21.15.84:0/221228699 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f12840767d0 con 0x7f129c0c9ca0 2023-10-16T11:24:46.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.983+0000 7f12a269b640 1 -- 172.21.15.84:0/221228699 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f129c0caa70 con 0x7f129c0c9ca0 2023-10-16T11:24:46.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.984+0000 7f1298ff9640 1 -- 172.21.15.84:0/221228699 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3319 (secure 0 0 0) 0x7f1284034d10 con 0x7f129c0c9ca0 2023-10-16T11:24:46.986 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:46.986 INFO:teuthology.orchestra.run.smithi084.stdout:{"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":5764,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":221,"num_osds":8,"num_up_osds":8,"osd_up_since":1697449734,"num_in_osds":8,"osd_in_since":1697449726,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":193}],"num_pgs":193,"num_pools":5,"num_objects":45,"data_bytes":596433,"bytes_used":389849088,"bytes_avail":772704264192,"bytes_total":773094113280,"write_bytes_sec":2249,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":120,"id":13,"up":1,"in":1,"max":1,"id":14,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":14,"rank":0,"name":"d","status":"up:active","gid":7474},{"filesystem_id":13,"rank":0,"name":"a","status":"up:active","gid":11307}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2023-10-16T11:24:12.247740+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-16T11:24:46.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.986+0000 7f12a269b640 1 -- 172.21.15.84:0/221228699 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f127c06af90 msgr2=0x7f127c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:46.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.987+0000 7f12a269b640 1 --2- 172.21.15.84:0/221228699 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f127c06af90 0x7f127c06d450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f129c06a6c0 tx=0x7f128c0083d0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.987+0000 7f12a269b640 1 -- 172.21.15.84:0/221228699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c0c9ca0 msgr2=0x7f129c14d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:46.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.987+0000 7f12a269b640 1 --2- 172.21.15.84:0/221228699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c0c9ca0 0x7f129c14d550 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f1284009870 tx=0x7f12840046c0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.987+0000 7f12a269b640 1 -- 172.21.15.84:0/221228699 shutdown_connections 2023-10-16T11:24:46.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.987+0000 7f12a269b640 1 --2- 172.21.15.84:0/221228699 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f127c06af90 0x7f127c06d450 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.987+0000 7f12a269b640 1 --2- 172.21.15.84:0/221228699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f129c15ad70 0x7f129c150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.987+0000 7f12a269b640 1 --2- 172.21.15.84:0/221228699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f129c0ca3b0 0x7f129c0c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.987+0000 7f12a269b640 1 --2- 172.21.15.84:0/221228699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c0c9ca0 0x7f129c14d550 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:46.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.987+0000 7f12a269b640 1 -- 172.21.15.84:0/221228699 >> 172.21.15.84:0/221228699 conn(0x7f129c0ce9b0 msgr2=0x7f129c15d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:46.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.987+0000 7f12a269b640 1 -- 172.21.15.84:0/221228699 shutdown_connections 2023-10-16T11:24:46.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:46.987+0000 7f12a269b640 1 -- 172.21.15.84:0/221228699 wait complete. 2023-10-16T11:24:47.008 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-16T11:24:47.009 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:24:47.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.226+0000 7fa0d5c87640 1 Processor -- start 2023-10-16T11:24:47.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.227+0000 7fa0d5c87640 1 -- start start 2023-10-16T11:24:47.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.228+0000 7fa0d5c87640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa0d015ae80 0x7fa0d015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:47.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.228+0000 7fa0d5c87640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d00ce390 0x7fa0d0070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:47.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.228+0000 7fa0d5c87640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d015ed60 con 0x7fa0d015aaa0 2023-10-16T11:24:47.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.228+0000 7fa0d5c87640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0d006b810 con 0x7fa0d015ae80 2023-10-16T11:24:47.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.228+0000 7fa0d5c87640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d00707e0 con 0x7fa0d00ce390 2023-10-16T11:24:47.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.228+0000 7fa0c6ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa0d015ae80 0x7fa0d015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:47.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.228+0000 7fa0c6ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa0d015ae80 0x7fa0d015e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51314/0 (socket says 172.21.15.84:51314) 2023-10-16T11:24:47.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.229+0000 7fa0c6ffd640 1 -- 172.21.15.84:0/1153620263 learned_addr learned my addr 172.21.15.84:0/1153620263 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:47.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.229+0000 7fa0cffff640 1 --2- 172.21.15.84:0/1153620263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d00ce390 0x7fa0d0070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:47.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.229+0000 7fa0c6ffd640 1 -- 172.21.15.84:0/1153620263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d00ce390 msgr2=0x7fa0d0070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:47.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.229+0000 7fa0c6ffd640 1 --2- 172.21.15.84:0/1153620263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d00ce390 0x7fa0d0070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:47.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.229+0000 7fa0c6ffd640 1 -- 172.21.15.84:0/1153620263 >> v1:172.21.15.157:6789/0 conn(0x7fa0d015aaa0 legacy=0x7fa0d00cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:47.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.229+0000 7fa0c6ffd640 1 -- 172.21.15.84:0/1153620263 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d015ef90 con 0x7fa0d015ae80 2023-10-16T11:24:47.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.229+0000 7fa0cffff640 1 --2- 172.21.15.84:0/1153620263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d00ce390 0x7fa0d0070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:47.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.230+0000 7fa0c6ffd640 1 --2- 172.21.15.84:0/1153620263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d015ae80 0x7fa0d015e820 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7fa0d00c59c0 tx=0x7fa0b802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e05f50f9a6825 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:47.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.230+0000 7fa0ceffd640 1 -- 172.21.15.84:0/1153620263 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8031640 con 0x7fa0d015ae80 2023-10-16T11:24:47.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.230+0000 7fa0ceffd640 1 -- 172.21.15.84:0/1153620263 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b80317a0 con 0x7fa0d015ae80 2023-10-16T11:24:47.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.230+0000 7fa0ceffd640 1 -- 172.21.15.84:0/1153620263 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8031b30 con 0x7fa0d015ae80 2023-10-16T11:24:47.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.231+0000 7fa0d5c87640 1 -- 172.21.15.84:0/1153620263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d015ae80 msgr2=0x7fa0d015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:47.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.231+0000 7fa0d5c87640 1 --2- 172.21.15.84:0/1153620263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d015ae80 0x7fa0d015e820 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7fa0d00c59c0 tx=0x7fa0b802b8f0 comp rx=0 tx=0).stop 2023-10-16T11:24:47.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.231+0000 7fa0d5c87640 1 -- 172.21.15.84:0/1153620263 shutdown_connections 2023-10-16T11:24:47.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.231+0000 7fa0d5c87640 1 --2- 172.21.15.84:0/1153620263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d00ce390 0x7fa0d0070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:47.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.231+0000 7fa0d5c87640 1 --2- 172.21.15.84:0/1153620263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d015ae80 0x7fa0d015e820 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:47.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.231+0000 7fa0d5c87640 1 -- 172.21.15.84:0/1153620263 >> 172.21.15.84:0/1153620263 conn(0x7fa0d00cb410 msgr2=0x7fa0d00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:47.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.231+0000 7fa0d5c87640 1 -- 172.21.15.84:0/1153620263 shutdown_connections 2023-10-16T11:24:47.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.231+0000 7fa0d5c87640 1 -- 172.21.15.84:0/1153620263 wait complete. 2023-10-16T11:24:47.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.233+0000 7fa0d5c87640 1 Processor -- start 2023-10-16T11:24:47.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.233+0000 7fa0d5c87640 1 -- start start 2023-10-16T11:24:47.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.234+0000 7fa0d5c87640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d00ce390 0x7fa0d00c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:47.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.234+0000 7fa0d5c87640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d015aaa0 0x7fa0d0156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:47.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.234+0000 7fa0d5c87640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d015ae80 0x7fa0d0142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:47.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.234+0000 7fa0d5c87640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d01f2ae0 con 0x7fa0d00ce390 2023-10-16T11:24:47.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.234+0000 7fa0d5c87640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d00c3e30 con 0x7fa0d015ae80 2023-10-16T11:24:47.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.234+0000 7fa0d5c87640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d00c4110 con 0x7fa0d015aaa0 2023-10-16T11:24:47.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.234+0000 7fa0cf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d00ce390 0x7fa0d00c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:47.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.234+0000 7fa0cf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d00ce390 0x7fa0d00c34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51316/0 (socket says 172.21.15.84:51316) 2023-10-16T11:24:47.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.234+0000 7fa0cf7fe640 1 -- 172.21.15.84:0/1884146131 learned_addr learned my addr 172.21.15.84:0/1884146131 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:47.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.235+0000 7fa0cffff640 1 --2- 172.21.15.84:0/1884146131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d015ae80 0x7fa0d0142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:47.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.235+0000 7fa0cf7fe640 1 -- 172.21.15.84:0/1884146131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d015aaa0 msgr2=0x7fa0d0156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:47.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.235+0000 7fa0cf7fe640 1 --2- 172.21.15.84:0/1884146131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d015aaa0 0x7fa0d0156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:47.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.235+0000 7fa0cf7fe640 1 -- 172.21.15.84:0/1884146131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d015ae80 msgr2=0x7fa0d0142de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:47.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.235+0000 7fa0cf7fe640 1 --2- 172.21.15.84:0/1884146131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d015ae80 0x7fa0d0142de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:47.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.235+0000 7fa0cf7fe640 1 -- 172.21.15.84:0/1884146131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b80094d0 con 0x7fa0d00ce390 2023-10-16T11:24:47.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.235+0000 7fa0cf7fe640 1 --2- 172.21.15.84:0/1884146131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d00ce390 0x7fa0d00c34e0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000c7a0 tx=0x7fa0c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:47.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.235+0000 7fa0ccff9640 1 -- 172.21.15.84:0/1884146131 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0003d80 con 0x7fa0d00ce390 2023-10-16T11:24:47.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.235+0000 7fa0ccff9640 1 -- 172.21.15.84:0/1884146131 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c0005070 con 0x7fa0d00ce390 2023-10-16T11:24:47.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.235+0000 7fa0ccff9640 1 -- 172.21.15.84:0/1884146131 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0005420 con 0x7fa0d00ce390 2023-10-16T11:24:47.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.235+0000 7fa0d5c87640 1 -- 172.21.15.84:0/1884146131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d00c43f0 con 0x7fa0d00ce390 2023-10-16T11:24:47.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.236+0000 7fa0d5c87640 1 -- 172.21.15.84:0/1884146131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d00c49b0 con 0x7fa0d00ce390 2023-10-16T11:24:47.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.237+0000 7fa0d5c87640 1 -- 172.21.15.84:0/1884146131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d004e160 con 0x7fa0d00ce390 2023-10-16T11:24:47.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.238+0000 7fa0ccff9640 1 -- 172.21.15.84:0/1884146131 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa0c0005580 con 0x7fa0d00ce390 2023-10-16T11:24:47.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.239+0000 7fa0ccff9640 1 --2- 172.21.15.84:0/1884146131 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa0b006af90 0x7fa0b006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:47.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.239+0000 7fa0ccff9640 1 -- 172.21.15.84:0/1884146131 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7fa0c0089100 con 0x7fa0d00ce390 2023-10-16T11:24:47.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.239+0000 7fa0c6ffd640 1 --2- 172.21.15.84:0/1884146131 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa0b006af90 0x7fa0b006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:47.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.240+0000 7fa0c6ffd640 1 --2- 172.21.15.84:0/1884146131 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa0b006af90 0x7fa0b006d450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8036000 tx=0x7fa0b8030000 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:47.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.243+0000 7fa0ccff9640 1 -- 172.21.15.84:0/1884146131 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa0c0053c80 con 0x7fa0d00ce390 2023-10-16T11:24:47.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.431+0000 7fa0d5c87640 1 -- 172.21.15.84:0/1884146131 --> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0d015f550 con 0x7fa0b006af90 2023-10-16T11:24:47.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.469+0000 7fa0ccff9640 1 -- 172.21.15.84:0/1884146131 <== mgr.11076 v2:172.21.15.84:6832/1425409017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+525743 (secure 0 0 0) 0x7fa0b8031a10 con 0x7fa0b006af90 2023-10-16T11:24:47.478 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:47.481 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:24:47.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.476+0000 7fa0d5c87640 1 -- 172.21.15.84:0/1884146131 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa0b006af90 msgr2=0x7fa0b006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:47.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.476+0000 7fa0d5c87640 1 --2- 172.21.15.84:0/1884146131 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa0b006af90 0x7fa0b006d450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8036000 tx=0x7fa0b8030000 comp rx=0 tx=0).stop 2023-10-16T11:24:47.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.476+0000 7fa0d5c87640 1 -- 172.21.15.84:0/1884146131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d00ce390 msgr2=0x7fa0d00c34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:47.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.476+0000 7fa0d5c87640 1 --2- 172.21.15.84:0/1884146131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d00ce390 0x7fa0d00c34e0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000c7a0 tx=0x7fa0c000cc60 comp rx=0 tx=0).stop 2023-10-16T11:24:47.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.476+0000 7fa0d5c87640 1 -- 172.21.15.84:0/1884146131 shutdown_connections 2023-10-16T11:24:47.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.476+0000 7fa0d5c87640 1 --2- 172.21.15.84:0/1884146131 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa0b006af90 0x7fa0b006d450 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:47.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.476+0000 7fa0d5c87640 1 --2- 172.21.15.84:0/1884146131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d015ae80 0x7fa0d0142de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:47.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.476+0000 7fa0d5c87640 1 --2- 172.21.15.84:0/1884146131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d015aaa0 0x7fa0d0156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:47.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.476+0000 7fa0d5c87640 1 --2- 172.21.15.84:0/1884146131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d00ce390 0x7fa0d00c34e0 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:47.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.476+0000 7fa0d5c87640 1 -- 172.21.15.84:0/1884146131 >> 172.21.15.84:0/1884146131 conn(0x7fa0d00cb410 msgr2=0x7fa0d006f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:47.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.477+0000 7fa0d5c87640 1 -- 172.21.15.84:0/1884146131 shutdown_connections 2023-10-16T11:24:47.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:47.477+0000 7fa0d5c87640 1 -- 172.21.15.84:0/1884146131 wait complete. 2023-10-16T11:24:47.499 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-10-16T11:24:46.257450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":672,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2846,"ondisk_log_size":2846,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":367,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":380712,"kb_used_data":22952,"kb_used_omap":12,"kb_used_meta":357619,"kb_avail":754594008,"statfs":{"total":773094113280,"available":772704264192,"internally_reserved":0,"allocated":23502848,"data_stored":11986322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":366202425},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.371803"},"pg_stats":[{"pgid":"29.1c","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076487+0000","last_change":"2023-10-16T11:24:40.949704+0000","last_active":"2023-10-16T11:24:42.076487+0000","last_peered":"2023-10-16T11:24:42.076487+0000","last_clean":"2023-10-16T11:24:42.076487+0000","last_became_active":"2023-10-16T11:24:40.944766+0000","last_became_peered":"2023-10-16T11:24:40.944766+0000","last_unstale":"2023-10-16T11:24:42.076487+0000","last_undegraded":"2023-10-16T11:24:42.076487+0000","last_fullsized":"2023-10-16T11:24:42.076487+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:39:05.461904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"206'2767","reported_seq":3652,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.928589+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:24:40.928589+0000","last_peered":"2023-10-16T11:24:40.928589+0000","last_clean":"2023-10-16T11:24:40.928589+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:24:40.928589+0000","last_undegraded":"2023-10-16T11:24:40.928589+0000","last_fullsized":"2023-10-16T11:24:40.928589+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2767,"log_dups_size":0,"ondisk_log_size":2767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":581,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"216'1","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.916192+0000","last_change":"2023-10-16T11:24:25.087491+0000","last_active":"2023-10-16T11:24:40.916192+0000","last_peered":"2023-10-16T11:24:40.916192+0000","last_clean":"2023-10-16T11:24:40.916192+0000","last_became_active":"2023-10-16T11:24:25.086826+0000","last_became_peered":"2023-10-16T11:24:25.086826+0000","last_unstale":"2023-10-16T11:24:40.916192+0000","last_undegraded":"2023-10-16T11:24:40.916192+0000","last_fullsized":"2023-10-16T11:24:40.916192+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:07.303719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918232+0000","last_change":"2023-10-16T11:24:26.091340+0000","last_active":"2023-10-16T11:24:40.918232+0000","last_peered":"2023-10-16T11:24:40.918232+0000","last_clean":"2023-10-16T11:24:40.918232+0000","last_became_active":"2023-10-16T11:24:26.091059+0000","last_became_peered":"2023-10-16T11:24:26.091059+0000","last_unstale":"2023-10-16T11:24:40.918232+0000","last_undegraded":"2023-10-16T11:24:40.918232+0000","last_fullsized":"2023-10-16T11:24:40.918232+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:37.605400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.948878+0000","last_change":"2023-10-16T11:24:39.927746+0000","last_active":"2023-10-16T11:24:40.948878+0000","last_peered":"2023-10-16T11:24:40.948878+0000","last_clean":"2023-10-16T11:24:40.948878+0000","last_became_active":"2023-10-16T11:24:39.927422+0000","last_became_peered":"2023-10-16T11:24:39.927422+0000","last_unstale":"2023-10-16T11:24:40.948878+0000","last_undegraded":"2023-10-16T11:24:40.948878+0000","last_fullsized":"2023-10-16T11:24:40.948878+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:37.032264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.080660+0000","last_change":"2023-10-16T11:24:40.934994+0000","last_active":"2023-10-16T11:24:42.080660+0000","last_peered":"2023-10-16T11:24:42.080660+0000","last_clean":"2023-10-16T11:24:42.080660+0000","last_became_active":"2023-10-16T11:24:40.929932+0000","last_became_peered":"2023-10-16T11:24:40.929932+0000","last_unstale":"2023-10-16T11:24:42.080660+0000","last_undegraded":"2023-10-16T11:24:42.080660+0000","last_fullsized":"2023-10-16T11:24:42.080660+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:23.723306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913447+0000","last_change":"2023-10-16T11:24:25.087755+0000","last_active":"2023-10-16T11:24:40.913447+0000","last_peered":"2023-10-16T11:24:40.913447+0000","last_clean":"2023-10-16T11:24:40.913447+0000","last_became_active":"2023-10-16T11:24:25.087497+0000","last_became_peered":"2023-10-16T11:24:25.087497+0000","last_unstale":"2023-10-16T11:24:40.913447+0000","last_undegraded":"2023-10-16T11:24:40.913447+0000","last_fullsized":"2023-10-16T11:24:40.913447+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:59:26.196708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.929543+0000","last_change":"2023-10-16T11:24:26.081718+0000","last_active":"2023-10-16T11:24:40.929543+0000","last_peered":"2023-10-16T11:24:40.929543+0000","last_clean":"2023-10-16T11:24:40.929543+0000","last_became_active":"2023-10-16T11:24:26.081447+0000","last_became_peered":"2023-10-16T11:24:26.081447+0000","last_unstale":"2023-10-16T11:24:40.929543+0000","last_undegraded":"2023-10-16T11:24:40.929543+0000","last_fullsized":"2023-10-16T11:24:40.929543+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.928257+0000","last_change":"2023-10-16T11:24:39.923647+0000","last_active":"2023-10-16T11:24:40.928257+0000","last_peered":"2023-10-16T11:24:40.928257+0000","last_clean":"2023-10-16T11:24:40.928257+0000","last_became_active":"2023-10-16T11:24:39.923359+0000","last_became_peered":"2023-10-16T11:24:39.923359+0000","last_unstale":"2023-10-16T11:24:40.928257+0000","last_undegraded":"2023-10-16T11:24:40.928257+0000","last_fullsized":"2023-10-16T11:24:40.928257+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:18:00.023742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.073930+0000","last_change":"2023-10-16T11:24:40.934136+0000","last_active":"2023-10-16T11:24:42.073930+0000","last_peered":"2023-10-16T11:24:42.073930+0000","last_clean":"2023-10-16T11:24:42.073930+0000","last_became_active":"2023-10-16T11:24:40.931559+0000","last_became_peered":"2023-10-16T11:24:40.931559+0000","last_unstale":"2023-10-16T11:24:42.073930+0000","last_undegraded":"2023-10-16T11:24:42.073930+0000","last_fullsized":"2023-10-16T11:24:42.073930+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:54:04.701711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.926384+0000","last_change":"2023-10-16T11:24:25.088161+0000","last_active":"2023-10-16T11:24:40.926384+0000","last_peered":"2023-10-16T11:24:40.926384+0000","last_clean":"2023-10-16T11:24:40.926384+0000","last_became_active":"2023-10-16T11:24:25.086051+0000","last_became_peered":"2023-10-16T11:24:25.086051+0000","last_unstale":"2023-10-16T11:24:40.926384+0000","last_undegraded":"2023-10-16T11:24:40.926384+0000","last_fullsized":"2023-10-16T11:24:40.926384+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.907231+0000","last_change":"2023-10-16T11:24:26.083366+0000","last_active":"2023-10-16T11:24:40.907231+0000","last_peered":"2023-10-16T11:24:40.907231+0000","last_clean":"2023-10-16T11:24:40.907231+0000","last_became_active":"2023-10-16T11:24:26.082635+0000","last_became_peered":"2023-10-16T11:24:26.082635+0000","last_unstale":"2023-10-16T11:24:40.907231+0000","last_undegraded":"2023-10-16T11:24:40.907231+0000","last_fullsized":"2023-10-16T11:24:40.907231+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.942903+0000","last_change":"2023-10-16T11:24:39.926910+0000","last_active":"2023-10-16T11:24:40.942903+0000","last_peered":"2023-10-16T11:24:40.942903+0000","last_clean":"2023-10-16T11:24:40.942903+0000","last_became_active":"2023-10-16T11:24:39.926615+0000","last_became_peered":"2023-10-16T11:24:39.926615+0000","last_unstale":"2023-10-16T11:24:40.942903+0000","last_undegraded":"2023-10-16T11:24:40.942903+0000","last_fullsized":"2023-10-16T11:24:40.942903+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:06.744007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.073721+0000","last_change":"2023-10-16T11:24:40.923750+0000","last_active":"2023-10-16T11:24:42.073721+0000","last_peered":"2023-10-16T11:24:42.073721+0000","last_clean":"2023-10-16T11:24:42.073721+0000","last_became_active":"2023-10-16T11:24:40.922455+0000","last_became_peered":"2023-10-16T11:24:40.922455+0000","last_unstale":"2023-10-16T11:24:42.073721+0000","last_undegraded":"2023-10-16T11:24:42.073721+0000","last_fullsized":"2023-10-16T11:24:42.073721+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:40.760759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912963+0000","last_change":"2023-10-16T11:24:25.080689+0000","last_active":"2023-10-16T11:24:40.912963+0000","last_peered":"2023-10-16T11:24:40.912963+0000","last_clean":"2023-10-16T11:24:40.912963+0000","last_became_active":"2023-10-16T11:24:25.080305+0000","last_became_peered":"2023-10-16T11:24:25.080305+0000","last_unstale":"2023-10-16T11:24:40.912963+0000","last_undegraded":"2023-10-16T11:24:40.912963+0000","last_fullsized":"2023-10-16T11:24:40.912963+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:43.670603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.916175+0000","last_change":"2023-10-16T11:24:26.083766+0000","last_active":"2023-10-16T11:24:40.916175+0000","last_peered":"2023-10-16T11:24:40.916175+0000","last_clean":"2023-10-16T11:24:40.916175+0000","last_became_active":"2023-10-16T11:24:26.083447+0000","last_became_peered":"2023-10-16T11:24:26.083447+0000","last_unstale":"2023-10-16T11:24:40.916175+0000","last_undegraded":"2023-10-16T11:24:40.916175+0000","last_fullsized":"2023-10-16T11:24:40.916175+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:44:14.656443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.914990+0000","last_change":"2023-10-16T11:24:39.926629+0000","last_active":"2023-10-16T11:24:40.914990+0000","last_peered":"2023-10-16T11:24:40.914990+0000","last_clean":"2023-10-16T11:24:40.914990+0000","last_became_active":"2023-10-16T11:24:39.926334+0000","last_became_peered":"2023-10-16T11:24:39.926334+0000","last_unstale":"2023-10-16T11:24:40.914990+0000","last_undegraded":"2023-10-16T11:24:40.914990+0000","last_fullsized":"2023-10-16T11:24:40.914990+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:24.166384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076395+0000","last_change":"2023-10-16T11:24:40.940956+0000","last_active":"2023-10-16T11:24:42.076395+0000","last_peered":"2023-10-16T11:24:42.076395+0000","last_clean":"2023-10-16T11:24:42.076395+0000","last_became_active":"2023-10-16T11:24:40.940649+0000","last_became_peered":"2023-10-16T11:24:40.940649+0000","last_unstale":"2023-10-16T11:24:42.076395+0000","last_undegraded":"2023-10-16T11:24:42.076395+0000","last_fullsized":"2023-10-16T11:24:42.076395+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:59.520985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.921975+0000","last_change":"2023-10-16T11:24:25.090432+0000","last_active":"2023-10-16T11:24:40.921975+0000","last_peered":"2023-10-16T11:24:40.921975+0000","last_clean":"2023-10-16T11:24:40.921975+0000","last_became_active":"2023-10-16T11:24:25.090142+0000","last_became_peered":"2023-10-16T11:24:25.090142+0000","last_unstale":"2023-10-16T11:24:40.921975+0000","last_undegraded":"2023-10-16T11:24:40.921975+0000","last_fullsized":"2023-10-16T11:24:40.921975+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:22.858690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918327+0000","last_change":"2023-10-16T11:24:26.093027+0000","last_active":"2023-10-16T11:24:40.918327+0000","last_peered":"2023-10-16T11:24:40.918327+0000","last_clean":"2023-10-16T11:24:40.918327+0000","last_became_active":"2023-10-16T11:24:26.092766+0000","last_became_peered":"2023-10-16T11:24:26.092766+0000","last_unstale":"2023-10-16T11:24:40.918327+0000","last_undegraded":"2023-10-16T11:24:40.918327+0000","last_fullsized":"2023-10-16T11:24:40.918327+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.923353+0000","last_change":"2023-10-16T11:24:39.924175+0000","last_active":"2023-10-16T11:24:40.923353+0000","last_peered":"2023-10-16T11:24:40.923353+0000","last_clean":"2023-10-16T11:24:40.923353+0000","last_became_active":"2023-10-16T11:24:39.923702+0000","last_became_peered":"2023-10-16T11:24:39.923702+0000","last_unstale":"2023-10-16T11:24:40.923353+0000","last_undegraded":"2023-10-16T11:24:40.923353+0000","last_fullsized":"2023-10-16T11:24:40.923353+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:33:55.188653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.077311+0000","last_change":"2023-10-16T11:24:40.937310+0000","last_active":"2023-10-16T11:24:42.077311+0000","last_peered":"2023-10-16T11:24:42.077311+0000","last_clean":"2023-10-16T11:24:42.077311+0000","last_became_active":"2023-10-16T11:24:40.937033+0000","last_became_peered":"2023-10-16T11:24:40.937033+0000","last_unstale":"2023-10-16T11:24:42.077311+0000","last_undegraded":"2023-10-16T11:24:42.077311+0000","last_fullsized":"2023-10-16T11:24:42.077311+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:51:40.711088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.908646+0000","last_change":"2023-10-16T11:24:25.088585+0000","last_active":"2023-10-16T11:24:40.908646+0000","last_peered":"2023-10-16T11:24:40.908646+0000","last_clean":"2023-10-16T11:24:40.908646+0000","last_became_active":"2023-10-16T11:24:25.088011+0000","last_became_peered":"2023-10-16T11:24:25.088011+0000","last_unstale":"2023-10-16T11:24:40.908646+0000","last_undegraded":"2023-10-16T11:24:40.908646+0000","last_fullsized":"2023-10-16T11:24:40.908646+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920936+0000","last_change":"2023-10-16T11:24:26.096341+0000","last_active":"2023-10-16T11:24:40.920936+0000","last_peered":"2023-10-16T11:24:40.920936+0000","last_clean":"2023-10-16T11:24:40.920936+0000","last_became_active":"2023-10-16T11:24:26.095836+0000","last_became_peered":"2023-10-16T11:24:26.095836+0000","last_unstale":"2023-10-16T11:24:40.920936+0000","last_undegraded":"2023-10-16T11:24:40.920936+0000","last_fullsized":"2023-10-16T11:24:40.920936+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917221+0000","last_change":"2023-10-16T11:24:39.909816+0000","last_active":"2023-10-16T11:24:40.917221+0000","last_peered":"2023-10-16T11:24:40.917221+0000","last_clean":"2023-10-16T11:24:40.917221+0000","last_became_active":"2023-10-16T11:24:39.909526+0000","last_became_peered":"2023-10-16T11:24:39.909526+0000","last_unstale":"2023-10-16T11:24:40.917221+0000","last_undegraded":"2023-10-16T11:24:40.917221+0000","last_fullsized":"2023-10-16T11:24:40.917221+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:12.120581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935566+0000","last_change":"2023-10-16T11:24:40.938174+0000","last_active":"2023-10-16T11:24:41.935566+0000","last_peered":"2023-10-16T11:24:41.935566+0000","last_clean":"2023-10-16T11:24:41.935566+0000","last_became_active":"2023-10-16T11:24:40.937904+0000","last_became_peered":"2023-10-16T11:24:40.937904+0000","last_unstale":"2023-10-16T11:24:41.935566+0000","last_undegraded":"2023-10-16T11:24:41.935566+0000","last_fullsized":"2023-10-16T11:24:41.935566+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:13.135380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913653+0000","last_change":"2023-10-16T11:24:25.077036+0000","last_active":"2023-10-16T11:24:40.913653+0000","last_peered":"2023-10-16T11:24:40.913653+0000","last_clean":"2023-10-16T11:24:40.913653+0000","last_became_active":"2023-10-16T11:24:25.076342+0000","last_became_peered":"2023-10-16T11:24:25.076342+0000","last_unstale":"2023-10-16T11:24:40.913653+0000","last_undegraded":"2023-10-16T11:24:40.913653+0000","last_fullsized":"2023-10-16T11:24:40.913653+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:36.162662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.935192+0000","last_change":"2023-10-16T11:24:26.085231+0000","last_active":"2023-10-16T11:24:40.935192+0000","last_peered":"2023-10-16T11:24:40.935192+0000","last_clean":"2023-10-16T11:24:40.935192+0000","last_became_active":"2023-10-16T11:24:26.084889+0000","last_became_peered":"2023-10-16T11:24:26.084889+0000","last_unstale":"2023-10-16T11:24:40.935192+0000","last_undegraded":"2023-10-16T11:24:40.935192+0000","last_fullsized":"2023-10-16T11:24:40.935192+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.935331+0000","last_change":"2023-10-16T11:24:39.920299+0000","last_active":"2023-10-16T11:24:40.935331+0000","last_peered":"2023-10-16T11:24:40.935331+0000","last_clean":"2023-10-16T11:24:40.935331+0000","last_became_active":"2023-10-16T11:24:39.919985+0000","last_became_peered":"2023-10-16T11:24:39.919985+0000","last_unstale":"2023-10-16T11:24:40.935331+0000","last_undegraded":"2023-10-16T11:24:40.935331+0000","last_fullsized":"2023-10-16T11:24:40.935331+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:59.121334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.077429+0000","last_change":"2023-10-16T11:24:40.940322+0000","last_active":"2023-10-16T11:24:42.077429+0000","last_peered":"2023-10-16T11:24:42.077429+0000","last_clean":"2023-10-16T11:24:42.077429+0000","last_became_active":"2023-10-16T11:24:40.940072+0000","last_became_peered":"2023-10-16T11:24:40.940072+0000","last_unstale":"2023-10-16T11:24:42.077429+0000","last_undegraded":"2023-10-16T11:24:42.077429+0000","last_fullsized":"2023-10-16T11:24:42.077429+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:33.978064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.6","version":"217'1","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.928431+0000","last_change":"2023-10-16T11:24:25.093214+0000","last_active":"2023-10-16T11:24:40.928431+0000","last_peered":"2023-10-16T11:24:40.928431+0000","last_clean":"2023-10-16T11:24:40.928431+0000","last_became_active":"2023-10-16T11:24:25.092877+0000","last_became_peered":"2023-10-16T11:24:25.092877+0000","last_unstale":"2023-10-16T11:24:40.928431+0000","last_undegraded":"2023-10-16T11:24:40.928431+0000","last_fullsized":"2023-10-16T11:24:40.928431+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:59.924424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.919655+0000","last_change":"2023-10-16T11:24:26.089940+0000","last_active":"2023-10-16T11:24:40.919655+0000","last_peered":"2023-10-16T11:24:40.919655+0000","last_clean":"2023-10-16T11:24:40.919655+0000","last_became_active":"2023-10-16T11:24:26.087906+0000","last_became_peered":"2023-10-16T11:24:26.087906+0000","last_unstale":"2023-10-16T11:24:40.919655+0000","last_undegraded":"2023-10-16T11:24:40.919655+0000","last_fullsized":"2023-10-16T11:24:40.919655+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:44.687193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913130+0000","last_change":"2023-10-16T11:24:39.918078+0000","last_active":"2023-10-16T11:24:40.913130+0000","last_peered":"2023-10-16T11:24:40.913130+0000","last_clean":"2023-10-16T11:24:40.913130+0000","last_became_active":"2023-10-16T11:24:39.914432+0000","last_became_peered":"2023-10-16T11:24:39.914432+0000","last_unstale":"2023-10-16T11:24:40.913130+0000","last_undegraded":"2023-10-16T11:24:40.913130+0000","last_fullsized":"2023-10-16T11:24:40.913130+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:07:10.996990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.074496+0000","last_change":"2023-10-16T11:24:40.940070+0000","last_active":"2023-10-16T11:24:42.074496+0000","last_peered":"2023-10-16T11:24:42.074496+0000","last_clean":"2023-10-16T11:24:42.074496+0000","last_became_active":"2023-10-16T11:24:40.939742+0000","last_became_peered":"2023-10-16T11:24:40.939742+0000","last_unstale":"2023-10-16T11:24:42.074496+0000","last_undegraded":"2023-10-16T11:24:42.074496+0000","last_fullsized":"2023-10-16T11:24:42.074496+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:07:00.381749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"216'2","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.909469+0000","last_change":"2023-10-16T11:24:25.094230+0000","last_active":"2023-10-16T11:24:40.909469+0000","last_peered":"2023-10-16T11:24:40.909469+0000","last_clean":"2023-10-16T11:24:40.909469+0000","last_became_active":"2023-10-16T11:24:25.093896+0000","last_became_peered":"2023-10-16T11:24:25.093896+0000","last_unstale":"2023-10-16T11:24:40.909469+0000","last_undegraded":"2023-10-16T11:24:40.909469+0000","last_fullsized":"2023-10-16T11:24:40.909469+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.915544+0000","last_change":"2023-10-16T11:24:26.085909+0000","last_active":"2023-10-16T11:24:40.915544+0000","last_peered":"2023-10-16T11:24:40.915544+0000","last_clean":"2023-10-16T11:24:40.915544+0000","last_became_active":"2023-10-16T11:24:26.085620+0000","last_became_peered":"2023-10-16T11:24:26.085620+0000","last_unstale":"2023-10-16T11:24:40.915544+0000","last_undegraded":"2023-10-16T11:24:40.915544+0000","last_fullsized":"2023-10-16T11:24:40.915544+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:41.023833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913673+0000","last_change":"2023-10-16T11:24:39.910612+0000","last_active":"2023-10-16T11:24:40.913673+0000","last_peered":"2023-10-16T11:24:40.913673+0000","last_clean":"2023-10-16T11:24:40.913673+0000","last_became_active":"2023-10-16T11:24:39.910324+0000","last_became_peered":"2023-10-16T11:24:39.910324+0000","last_unstale":"2023-10-16T11:24:40.913673+0000","last_undegraded":"2023-10-16T11:24:40.913673+0000","last_fullsized":"2023-10-16T11:24:40.913673+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:44:02.305284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.934777+0000","last_change":"2023-10-16T11:24:40.945289+0000","last_active":"2023-10-16T11:24:41.934777+0000","last_peered":"2023-10-16T11:24:41.934777+0000","last_clean":"2023-10-16T11:24:41.934777+0000","last_became_active":"2023-10-16T11:24:40.944986+0000","last_became_peered":"2023-10-16T11:24:40.944986+0000","last_unstale":"2023-10-16T11:24:41.934777+0000","last_undegraded":"2023-10-16T11:24:41.934777+0000","last_fullsized":"2023-10-16T11:24:41.934777+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:46.340170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912752+0000","last_change":"2023-10-16T11:24:25.080225+0000","last_active":"2023-10-16T11:24:40.912752+0000","last_peered":"2023-10-16T11:24:40.912752+0000","last_clean":"2023-10-16T11:24:40.912752+0000","last_became_active":"2023-10-16T11:24:25.079921+0000","last_became_peered":"2023-10-16T11:24:25.079921+0000","last_unstale":"2023-10-16T11:24:40.912752+0000","last_undegraded":"2023-10-16T11:24:40.912752+0000","last_fullsized":"2023-10-16T11:24:40.912752+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:46.550429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.926296+0000","last_change":"2023-10-16T11:24:26.088438+0000","last_active":"2023-10-16T11:24:40.926296+0000","last_peered":"2023-10-16T11:24:40.926296+0000","last_clean":"2023-10-16T11:24:40.926296+0000","last_became_active":"2023-10-16T11:24:26.087973+0000","last_became_peered":"2023-10-16T11:24:26.087973+0000","last_unstale":"2023-10-16T11:24:40.926296+0000","last_undegraded":"2023-10-16T11:24:40.926296+0000","last_fullsized":"2023-10-16T11:24:40.926296+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.944794+0000","last_change":"2023-10-16T11:24:39.915986+0000","last_active":"2023-10-16T11:24:40.944794+0000","last_peered":"2023-10-16T11:24:40.944794+0000","last_clean":"2023-10-16T11:24:40.944794+0000","last_became_active":"2023-10-16T11:24:39.915611+0000","last_became_peered":"2023-10-16T11:24:39.915611+0000","last_unstale":"2023-10-16T11:24:40.944794+0000","last_undegraded":"2023-10-16T11:24:40.944794+0000","last_fullsized":"2023-10-16T11:24:40.944794+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:37.032264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935666+0000","last_change":"2023-10-16T11:24:40.932578+0000","last_active":"2023-10-16T11:24:41.935666+0000","last_peered":"2023-10-16T11:24:41.935666+0000","last_clean":"2023-10-16T11:24:41.935666+0000","last_became_active":"2023-10-16T11:24:40.931761+0000","last_became_peered":"2023-10-16T11:24:40.931761+0000","last_unstale":"2023-10-16T11:24:41.935666+0000","last_undegraded":"2023-10-16T11:24:41.935666+0000","last_fullsized":"2023-10-16T11:24:41.935666+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:55.400689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917511+0000","last_change":"2023-10-16T11:24:25.086180+0000","last_active":"2023-10-16T11:24:40.917511+0000","last_peered":"2023-10-16T11:24:40.917511+0000","last_clean":"2023-10-16T11:24:40.917511+0000","last_became_active":"2023-10-16T11:24:25.085316+0000","last_became_peered":"2023-10-16T11:24:25.085316+0000","last_unstale":"2023-10-16T11:24:40.917511+0000","last_undegraded":"2023-10-16T11:24:40.917511+0000","last_fullsized":"2023-10-16T11:24:40.917511+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927499+0000","last_change":"2023-10-16T11:24:26.090951+0000","last_active":"2023-10-16T11:24:40.927499+0000","last_peered":"2023-10-16T11:24:40.927499+0000","last_clean":"2023-10-16T11:24:40.927499+0000","last_became_active":"2023-10-16T11:24:26.089241+0000","last_became_peered":"2023-10-16T11:24:26.089241+0000","last_unstale":"2023-10-16T11:24:40.927499+0000","last_undegraded":"2023-10-16T11:24:40.927499+0000","last_fullsized":"2023-10-16T11:24:40.927499+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:40.970357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"220'1","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.946450+0000","last_change":"2023-10-16T11:24:39.926741+0000","last_active":"2023-10-16T11:24:40.946450+0000","last_peered":"2023-10-16T11:24:40.946450+0000","last_clean":"2023-10-16T11:24:40.946450+0000","last_became_active":"2023-10-16T11:24:39.926377+0000","last_became_peered":"2023-10-16T11:24:39.926377+0000","last_unstale":"2023-10-16T11:24:40.946450+0000","last_undegraded":"2023-10-16T11:24:40.946450+0000","last_fullsized":"2023-10-16T11:24:40.946450+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:07:06.185553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.080553+0000","last_change":"2023-10-16T11:24:40.936952+0000","last_active":"2023-10-16T11:24:42.080553+0000","last_peered":"2023-10-16T11:24:42.080553+0000","last_clean":"2023-10-16T11:24:42.080553+0000","last_became_active":"2023-10-16T11:24:40.936526+0000","last_became_peered":"2023-10-16T11:24:40.936526+0000","last_unstale":"2023-10-16T11:24:42.080553+0000","last_undegraded":"2023-10-16T11:24:42.080553+0000","last_fullsized":"2023-10-16T11:24:42.080553+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:43:43.109538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918490+0000","last_change":"2023-10-16T11:24:25.091656+0000","last_active":"2023-10-16T11:24:40.918490+0000","last_peered":"2023-10-16T11:24:40.918490+0000","last_clean":"2023-10-16T11:24:40.918490+0000","last_became_active":"2023-10-16T11:24:25.091316+0000","last_became_peered":"2023-10-16T11:24:25.091316+0000","last_unstale":"2023-10-16T11:24:40.918490+0000","last_undegraded":"2023-10-16T11:24:40.918490+0000","last_fullsized":"2023-10-16T11:24:40.918490+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:26.054702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.908836+0000","last_change":"2023-10-16T11:24:26.085260+0000","last_active":"2023-10-16T11:24:40.908836+0000","last_peered":"2023-10-16T11:24:40.908836+0000","last_clean":"2023-10-16T11:24:40.908836+0000","last_became_active":"2023-10-16T11:24:26.084860+0000","last_became_peered":"2023-10-16T11:24:26.084860+0000","last_unstale":"2023-10-16T11:24:40.908836+0000","last_undegraded":"2023-10-16T11:24:40.908836+0000","last_fullsized":"2023-10-16T11:24:40.908836+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920928+0000","last_change":"2023-10-16T11:24:39.908388+0000","last_active":"2023-10-16T11:24:40.920928+0000","last_peered":"2023-10-16T11:24:40.920928+0000","last_clean":"2023-10-16T11:24:40.920928+0000","last_became_active":"2023-10-16T11:24:39.908075+0000","last_became_peered":"2023-10-16T11:24:39.908075+0000","last_unstale":"2023-10-16T11:24:40.920928+0000","last_undegraded":"2023-10-16T11:24:40.920928+0000","last_fullsized":"2023-10-16T11:24:40.920928+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:12:56.357582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.074734+0000","last_change":"2023-10-16T11:24:40.934674+0000","last_active":"2023-10-16T11:24:42.074734+0000","last_peered":"2023-10-16T11:24:42.074734+0000","last_clean":"2023-10-16T11:24:42.074734+0000","last_became_active":"2023-10-16T11:24:40.934420+0000","last_became_peered":"2023-10-16T11:24:40.934420+0000","last_unstale":"2023-10-16T11:24:42.074734+0000","last_undegraded":"2023-10-16T11:24:42.074734+0000","last_fullsized":"2023-10-16T11:24:42.074734+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:52.013044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.b","version":"216'2","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.923129+0000","last_change":"2023-10-16T11:24:25.072340+0000","last_active":"2023-10-16T11:24:40.923129+0000","last_peered":"2023-10-16T11:24:40.923129+0000","last_clean":"2023-10-16T11:24:40.923129+0000","last_became_active":"2023-10-16T11:24:25.071728+0000","last_became_peered":"2023-10-16T11:24:25.071728+0000","last_unstale":"2023-10-16T11:24:40.923129+0000","last_undegraded":"2023-10-16T11:24:40.923129+0000","last_fullsized":"2023-10-16T11:24:40.923129+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:49.041521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927055+0000","last_change":"2023-10-16T11:24:26.091096+0000","last_active":"2023-10-16T11:24:40.927055+0000","last_peered":"2023-10-16T11:24:40.927055+0000","last_clean":"2023-10-16T11:24:40.927055+0000","last_became_active":"2023-10-16T11:24:26.090774+0000","last_became_peered":"2023-10-16T11:24:26.090774+0000","last_unstale":"2023-10-16T11:24:40.927055+0000","last_undegraded":"2023-10-16T11:24:40.927055+0000","last_fullsized":"2023-10-16T11:24:40.927055+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:27.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"220'1","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.944256+0000","last_change":"2023-10-16T11:24:39.921764+0000","last_active":"2023-10-16T11:24:40.944256+0000","last_peered":"2023-10-16T11:24:40.944256+0000","last_clean":"2023-10-16T11:24:40.944256+0000","last_became_active":"2023-10-16T11:24:39.921487+0000","last_became_peered":"2023-10-16T11:24:39.921487+0000","last_unstale":"2023-10-16T11:24:40.944256+0000","last_undegraded":"2023-10-16T11:24:40.944256+0000","last_fullsized":"2023-10-16T11:24:40.944256+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:25.891782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076327+0000","last_change":"2023-10-16T11:24:40.946254+0000","last_active":"2023-10-16T11:24:42.076327+0000","last_peered":"2023-10-16T11:24:42.076327+0000","last_clean":"2023-10-16T11:24:42.076327+0000","last_became_active":"2023-10-16T11:24:40.945902+0000","last_became_peered":"2023-10-16T11:24:40.945902+0000","last_unstale":"2023-10-16T11:24:42.076327+0000","last_undegraded":"2023-10-16T11:24:42.076327+0000","last_fullsized":"2023-10-16T11:24:42.076327+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:52.439191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.908290+0000","last_change":"2023-10-16T11:24:25.079021+0000","last_active":"2023-10-16T11:24:40.908290+0000","last_peered":"2023-10-16T11:24:40.908290+0000","last_clean":"2023-10-16T11:24:40.908290+0000","last_became_active":"2023-10-16T11:24:25.076800+0000","last_became_peered":"2023-10-16T11:24:25.076800+0000","last_unstale":"2023-10-16T11:24:40.908290+0000","last_undegraded":"2023-10-16T11:24:40.908290+0000","last_fullsized":"2023-10-16T11:24:40.908290+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:56.161407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920389+0000","last_change":"2023-10-16T11:24:26.092876+0000","last_active":"2023-10-16T11:24:40.920389+0000","last_peered":"2023-10-16T11:24:40.920389+0000","last_clean":"2023-10-16T11:24:40.920389+0000","last_became_active":"2023-10-16T11:24:26.091310+0000","last_became_peered":"2023-10-16T11:24:26.091310+0000","last_unstale":"2023-10-16T11:24:40.920389+0000","last_undegraded":"2023-10-16T11:24:40.920389+0000","last_fullsized":"2023-10-16T11:24:40.920389+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:22.891780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.931170+0000","last_change":"2023-10-16T11:24:39.918307+0000","last_active":"2023-10-16T11:24:40.931170+0000","last_peered":"2023-10-16T11:24:40.931170+0000","last_clean":"2023-10-16T11:24:40.931170+0000","last_became_active":"2023-10-16T11:24:39.916755+0000","last_became_peered":"2023-10-16T11:24:39.916755+0000","last_unstale":"2023-10-16T11:24:40.931170+0000","last_undegraded":"2023-10-16T11:24:40.931170+0000","last_fullsized":"2023-10-16T11:24:40.931170+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:54.385890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072744+0000","last_change":"2023-10-16T11:24:40.929984+0000","last_active":"2023-10-16T11:24:42.072744+0000","last_peered":"2023-10-16T11:24:42.072744+0000","last_clean":"2023-10-16T11:24:42.072744+0000","last_became_active":"2023-10-16T11:24:40.928183+0000","last_became_peered":"2023-10-16T11:24:40.928183+0000","last_unstale":"2023-10-16T11:24:42.072744+0000","last_undegraded":"2023-10-16T11:24:42.072744+0000","last_fullsized":"2023-10-16T11:24:42.072744+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:00:55.030170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.d","version":"216'1","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920754+0000","last_change":"2023-10-16T11:24:25.083155+0000","last_active":"2023-10-16T11:24:40.920754+0000","last_peered":"2023-10-16T11:24:40.920754+0000","last_clean":"2023-10-16T11:24:40.920754+0000","last_became_active":"2023-10-16T11:24:25.082690+0000","last_became_peered":"2023-10-16T11:24:25.082690+0000","last_unstale":"2023-10-16T11:24:40.920754+0000","last_undegraded":"2023-10-16T11:24:40.920754+0000","last_fullsized":"2023-10-16T11:24:40.920754+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:46:58.816018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.908131+0000","last_change":"2023-10-16T11:24:26.069780+0000","last_active":"2023-10-16T11:24:40.908131+0000","last_peered":"2023-10-16T11:24:40.908131+0000","last_clean":"2023-10-16T11:24:40.908131+0000","last_became_active":"2023-10-16T11:24:26.068772+0000","last_became_peered":"2023-10-16T11:24:26.068772+0000","last_unstale":"2023-10-16T11:24:40.908131+0000","last_undegraded":"2023-10-16T11:24:40.908131+0000","last_fullsized":"2023-10-16T11:24:40.908131+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.944704+0000","last_change":"2023-10-16T11:24:39.928140+0000","last_active":"2023-10-16T11:24:40.944704+0000","last_peered":"2023-10-16T11:24:40.944704+0000","last_clean":"2023-10-16T11:24:40.944704+0000","last_became_active":"2023-10-16T11:24:39.927182+0000","last_became_peered":"2023-10-16T11:24:39.927182+0000","last_unstale":"2023-10-16T11:24:40.944704+0000","last_undegraded":"2023-10-16T11:24:40.944704+0000","last_fullsized":"2023-10-16T11:24:40.944704+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:04:01.386012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072533+0000","last_change":"2023-10-16T11:24:40.937098+0000","last_active":"2023-10-16T11:24:42.072533+0000","last_peered":"2023-10-16T11:24:42.072533+0000","last_clean":"2023-10-16T11:24:42.072533+0000","last_became_active":"2023-10-16T11:24:40.936844+0000","last_became_peered":"2023-10-16T11:24:40.936844+0000","last_unstale":"2023-10-16T11:24:42.072533+0000","last_undegraded":"2023-10-16T11:24:42.072533+0000","last_fullsized":"2023-10-16T11:24:42.072533+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:06.892515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.e","version":"216'2","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.933425+0000","last_change":"2023-10-16T11:24:25.087765+0000","last_active":"2023-10-16T11:24:40.933425+0000","last_peered":"2023-10-16T11:24:40.933425+0000","last_clean":"2023-10-16T11:24:40.933425+0000","last_became_active":"2023-10-16T11:24:25.086967+0000","last_became_peered":"2023-10-16T11:24:25.086967+0000","last_unstale":"2023-10-16T11:24:40.933425+0000","last_undegraded":"2023-10-16T11:24:40.933425+0000","last_fullsized":"2023-10-16T11:24:40.933425+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.932831+0000","last_change":"2023-10-16T11:24:26.086833+0000","last_active":"2023-10-16T11:24:40.932831+0000","last_peered":"2023-10-16T11:24:40.932831+0000","last_clean":"2023-10-16T11:24:40.932831+0000","last_became_active":"2023-10-16T11:24:26.085367+0000","last_became_peered":"2023-10-16T11:24:26.085367+0000","last_unstale":"2023-10-16T11:24:40.932831+0000","last_undegraded":"2023-10-16T11:24:40.932831+0000","last_fullsized":"2023-10-16T11:24:40.932831+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913101+0000","last_change":"2023-10-16T11:24:39.913504+0000","last_active":"2023-10-16T11:24:40.913101+0000","last_peered":"2023-10-16T11:24:40.913101+0000","last_clean":"2023-10-16T11:24:40.913101+0000","last_became_active":"2023-10-16T11:24:39.913184+0000","last_became_peered":"2023-10-16T11:24:39.913184+0000","last_unstale":"2023-10-16T11:24:40.913101+0000","last_undegraded":"2023-10-16T11:24:40.913101+0000","last_fullsized":"2023-10-16T11:24:40.913101+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:36.821422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.931761+0000","last_change":"2023-10-16T11:24:40.936862+0000","last_active":"2023-10-16T11:24:41.931761+0000","last_peered":"2023-10-16T11:24:41.931761+0000","last_clean":"2023-10-16T11:24:41.931761+0000","last_became_active":"2023-10-16T11:24:40.933139+0000","last_became_peered":"2023-10-16T11:24:40.933139+0000","last_unstale":"2023-10-16T11:24:41.931761+0000","last_undegraded":"2023-10-16T11:24:41.931761+0000","last_fullsized":"2023-10-16T11:24:41.931761+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:23.266881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.f","version":"216'1","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913585+0000","last_change":"2023-10-16T11:24:25.079789+0000","last_active":"2023-10-16T11:24:40.913585+0000","last_peered":"2023-10-16T11:24:40.913585+0000","last_clean":"2023-10-16T11:24:40.913585+0000","last_became_active":"2023-10-16T11:24:25.079495+0000","last_became_peered":"2023-10-16T11:24:25.079495+0000","last_unstale":"2023-10-16T11:24:40.913585+0000","last_undegraded":"2023-10-16T11:24:40.913585+0000","last_fullsized":"2023-10-16T11:24:40.913585+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:44.531367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.933148+0000","last_change":"2023-10-16T11:24:26.081796+0000","last_active":"2023-10-16T11:24:40.933148+0000","last_peered":"2023-10-16T11:24:40.933148+0000","last_clean":"2023-10-16T11:24:40.933148+0000","last_became_active":"2023-10-16T11:24:26.080287+0000","last_became_peered":"2023-10-16T11:24:26.080287+0000","last_unstale":"2023-10-16T11:24:40.933148+0000","last_undegraded":"2023-10-16T11:24:40.933148+0000","last_fullsized":"2023-10-16T11:24:40.933148+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927703+0000","last_change":"2023-10-16T11:24:39.922329+0000","last_active":"2023-10-16T11:24:40.927703+0000","last_peered":"2023-10-16T11:24:40.927703+0000","last_clean":"2023-10-16T11:24:40.927703+0000","last_became_active":"2023-10-16T11:24:39.921982+0000","last_became_peered":"2023-10-16T11:24:39.921982+0000","last_unstale":"2023-10-16T11:24:40.927703+0000","last_undegraded":"2023-10-16T11:24:40.927703+0000","last_fullsized":"2023-10-16T11:24:40.927703+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:17.673425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.080239+0000","last_change":"2023-10-16T11:24:40.949583+0000","last_active":"2023-10-16T11:24:42.080239+0000","last_peered":"2023-10-16T11:24:42.080239+0000","last_clean":"2023-10-16T11:24:42.080239+0000","last_became_active":"2023-10-16T11:24:40.947541+0000","last_became_peered":"2023-10-16T11:24:40.947541+0000","last_unstale":"2023-10-16T11:24:42.080239+0000","last_undegraded":"2023-10-16T11:24:42.080239+0000","last_fullsized":"2023-10-16T11:24:42.080239+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:10:12.883381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.10","version":"216'8","reported_seq":24,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.907330+0000","last_change":"2023-10-16T11:24:25.076954+0000","last_active":"2023-10-16T11:24:40.907330+0000","last_peered":"2023-10-16T11:24:40.907330+0000","last_clean":"2023-10-16T11:24:40.907330+0000","last_became_active":"2023-10-16T11:24:25.076509+0000","last_became_peered":"2023-10-16T11:24:25.076509+0000","last_unstale":"2023-10-16T11:24:40.907330+0000","last_undegraded":"2023-10-16T11:24:40.907330+0000","last_fullsized":"2023-10-16T11:24:40.907330+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:46.770213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917418+0000","last_change":"2023-10-16T11:24:26.084139+0000","last_active":"2023-10-16T11:24:40.917418+0000","last_peered":"2023-10-16T11:24:40.917418+0000","last_clean":"2023-10-16T11:24:40.917418+0000","last_became_active":"2023-10-16T11:24:26.083876+0000","last_became_peered":"2023-10-16T11:24:26.083876+0000","last_unstale":"2023-10-16T11:24:40.917418+0000","last_undegraded":"2023-10-16T11:24:40.917418+0000","last_fullsized":"2023-10-16T11:24:40.917418+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918095+0000","last_change":"2023-10-16T11:24:39.904420+0000","last_active":"2023-10-16T11:24:40.918095+0000","last_peered":"2023-10-16T11:24:40.918095+0000","last_clean":"2023-10-16T11:24:40.918095+0000","last_became_active":"2023-10-16T11:24:39.904039+0000","last_became_peered":"2023-10-16T11:24:39.904039+0000","last_unstale":"2023-10-16T11:24:40.918095+0000","last_undegraded":"2023-10-16T11:24:40.918095+0000","last_fullsized":"2023-10-16T11:24:40.918095+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:17.673425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.936696+0000","last_change":"2023-10-16T11:24:40.947940+0000","last_active":"2023-10-16T11:24:41.936696+0000","last_peered":"2023-10-16T11:24:41.936696+0000","last_clean":"2023-10-16T11:24:41.936696+0000","last_became_active":"2023-10-16T11:24:40.947659+0000","last_became_peered":"2023-10-16T11:24:40.947659+0000","last_unstale":"2023-10-16T11:24:41.936696+0000","last_undegraded":"2023-10-16T11:24:41.936696+0000","last_fullsized":"2023-10-16T11:24:41.936696+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:15:00.136133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.923325+0000","last_change":"2023-10-16T11:24:25.089470+0000","last_active":"2023-10-16T11:24:40.923325+0000","last_peered":"2023-10-16T11:24:40.923325+0000","last_clean":"2023-10-16T11:24:40.923325+0000","last_became_active":"2023-10-16T11:24:25.089213+0000","last_became_peered":"2023-10-16T11:24:25.089213+0000","last_unstale":"2023-10-16T11:24:40.923325+0000","last_undegraded":"2023-10-16T11:24:40.923325+0000","last_fullsized":"2023-10-16T11:24:40.923325+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:34.540849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.907117+0000","last_change":"2023-10-16T11:24:26.079602+0000","last_active":"2023-10-16T11:24:40.907117+0000","last_peered":"2023-10-16T11:24:40.907117+0000","last_clean":"2023-10-16T11:24:40.907117+0000","last_became_active":"2023-10-16T11:24:26.079344+0000","last_became_peered":"2023-10-16T11:24:26.079344+0000","last_unstale":"2023-10-16T11:24:40.907117+0000","last_undegraded":"2023-10-16T11:24:40.907117+0000","last_fullsized":"2023-10-16T11:24:40.907117+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.930050+0000","last_change":"2023-10-16T11:24:39.924572+0000","last_active":"2023-10-16T11:24:40.930050+0000","last_peered":"2023-10-16T11:24:40.930050+0000","last_clean":"2023-10-16T11:24:40.930050+0000","last_became_active":"2023-10-16T11:24:39.924266+0000","last_became_peered":"2023-10-16T11:24:39.924266+0000","last_unstale":"2023-10-16T11:24:40.930050+0000","last_undegraded":"2023-10-16T11:24:40.930050+0000","last_fullsized":"2023-10-16T11:24:40.930050+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:57:57.289628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.077212+0000","last_change":"2023-10-16T11:24:40.938221+0000","last_active":"2023-10-16T11:24:42.077212+0000","last_peered":"2023-10-16T11:24:42.077212+0000","last_clean":"2023-10-16T11:24:42.077212+0000","last_became_active":"2023-10-16T11:24:40.937659+0000","last_became_peered":"2023-10-16T11:24:40.937659+0000","last_unstale":"2023-10-16T11:24:42.077212+0000","last_undegraded":"2023-10-16T11:24:42.077212+0000","last_fullsized":"2023-10-16T11:24:42.077212+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:27.817701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.924919+0000","last_change":"2023-10-16T11:24:25.084194+0000","last_active":"2023-10-16T11:24:40.924919+0000","last_peered":"2023-10-16T11:24:40.924919+0000","last_clean":"2023-10-16T11:24:40.924919+0000","last_became_active":"2023-10-16T11:24:25.083141+0000","last_became_peered":"2023-10-16T11:24:25.083141+0000","last_unstale":"2023-10-16T11:24:40.924919+0000","last_undegraded":"2023-10-16T11:24:40.924919+0000","last_fullsized":"2023-10-16T11:24:40.924919+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:46.770213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.923041+0000","last_change":"2023-10-16T11:24:26.089221+0000","last_active":"2023-10-16T11:24:40.923041+0000","last_peered":"2023-10-16T11:24:40.923041+0000","last_clean":"2023-10-16T11:24:40.923041+0000","last_became_active":"2023-10-16T11:24:26.088644+0000","last_became_peered":"2023-10-16T11:24:26.088644+0000","last_unstale":"2023-10-16T11:24:40.923041+0000","last_undegraded":"2023-10-16T11:24:40.923041+0000","last_fullsized":"2023-10-16T11:24:40.923041+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:10.387018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"220'6","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.948264+0000","last_change":"2023-10-16T11:24:39.904338+0000","last_active":"2023-10-16T11:24:40.948264+0000","last_peered":"2023-10-16T11:24:40.948264+0000","last_clean":"2023-10-16T11:24:40.948264+0000","last_became_active":"2023-10-16T11:24:39.904034+0000","last_became_peered":"2023-10-16T11:24:39.904034+0000","last_unstale":"2023-10-16T11:24:40.948264+0000","last_undegraded":"2023-10-16T11:24:40.948264+0000","last_fullsized":"2023-10-16T11:24:40.948264+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:58.317757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.078231+0000","last_change":"2023-10-16T11:24:40.949476+0000","last_active":"2023-10-16T11:24:42.078231+0000","last_peered":"2023-10-16T11:24:42.078231+0000","last_clean":"2023-10-16T11:24:42.078231+0000","last_became_active":"2023-10-16T11:24:40.949199+0000","last_became_peered":"2023-10-16T11:24:40.949199+0000","last_unstale":"2023-10-16T11:24:42.078231+0000","last_undegraded":"2023-10-16T11:24:42.078231+0000","last_fullsized":"2023-10-16T11:24:42.078231+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:25:21.239825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.13","version":"216'2","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920754+0000","last_change":"2023-10-16T11:24:25.081377+0000","last_active":"2023-10-16T11:24:40.920754+0000","last_peered":"2023-10-16T11:24:40.920754+0000","last_clean":"2023-10-16T11:24:40.920754+0000","last_became_active":"2023-10-16T11:24:25.080881+0000","last_became_peered":"2023-10-16T11:24:25.080881+0000","last_unstale":"2023-10-16T11:24:40.920754+0000","last_undegraded":"2023-10-16T11:24:40.920754+0000","last_fullsized":"2023-10-16T11:24:40.920754+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:26.054702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912668+0000","last_change":"2023-10-16T11:24:26.099039+0000","last_active":"2023-10-16T11:24:40.912668+0000","last_peered":"2023-10-16T11:24:40.912668+0000","last_clean":"2023-10-16T11:24:40.912668+0000","last_became_active":"2023-10-16T11:24:26.097980+0000","last_became_peered":"2023-10-16T11:24:26.097980+0000","last_unstale":"2023-10-16T11:24:40.912668+0000","last_undegraded":"2023-10-16T11:24:40.912668+0000","last_fullsized":"2023-10-16T11:24:40.912668+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:40.585267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"220'3","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.936626+0000","last_change":"2023-10-16T11:24:39.906086+0000","last_active":"2023-10-16T11:24:40.936626+0000","last_peered":"2023-10-16T11:24:40.936626+0000","last_clean":"2023-10-16T11:24:40.936626+0000","last_became_active":"2023-10-16T11:24:39.905781+0000","last_became_peered":"2023-10-16T11:24:39.905781+0000","last_unstale":"2023-10-16T11:24:40.936626+0000","last_undegraded":"2023-10-16T11:24:40.936626+0000","last_fullsized":"2023-10-16T11:24:40.936626+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:07:47.547377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.078344+0000","last_change":"2023-10-16T11:24:40.936652+0000","last_active":"2023-10-16T11:24:42.078344+0000","last_peered":"2023-10-16T11:24:42.078344+0000","last_clean":"2023-10-16T11:24:42.078344+0000","last_became_active":"2023-10-16T11:24:40.934276+0000","last_became_peered":"2023-10-16T11:24:40.934276+0000","last_unstale":"2023-10-16T11:24:42.078344+0000","last_undegraded":"2023-10-16T11:24:42.078344+0000","last_fullsized":"2023-10-16T11:24:42.078344+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:30.742890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.14","version":"217'8","reported_seq":24,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.922693+0000","last_change":"2023-10-16T11:24:25.095357+0000","last_active":"2023-10-16T11:24:40.922693+0000","last_peered":"2023-10-16T11:24:40.922693+0000","last_clean":"2023-10-16T11:24:40.922693+0000","last_became_active":"2023-10-16T11:24:25.093832+0000","last_became_peered":"2023-10-16T11:24:25.093832+0000","last_unstale":"2023-10-16T11:24:40.922693+0000","last_undegraded":"2023-10-16T11:24:40.922693+0000","last_fullsized":"2023-10-16T11:24:40.922693+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:14:19.482469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918256+0000","last_change":"2023-10-16T11:24:26.092185+0000","last_active":"2023-10-16T11:24:40.918256+0000","last_peered":"2023-10-16T11:24:40.918256+0000","last_clean":"2023-10-16T11:24:40.918256+0000","last_became_active":"2023-10-16T11:24:26.091648+0000","last_became_peered":"2023-10-16T11:24:26.091648+0000","last_unstale":"2023-10-16T11:24:40.918256+0000","last_undegraded":"2023-10-16T11:24:40.918256+0000","last_fullsized":"2023-10-16T11:24:40.918256+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918129+0000","last_change":"2023-10-16T11:24:39.919312+0000","last_active":"2023-10-16T11:24:40.918129+0000","last_peered":"2023-10-16T11:24:40.918129+0000","last_clean":"2023-10-16T11:24:40.918129+0000","last_became_active":"2023-10-16T11:24:39.918986+0000","last_became_peered":"2023-10-16T11:24:39.918986+0000","last_unstale":"2023-10-16T11:24:40.918129+0000","last_undegraded":"2023-10-16T11:24:40.918129+0000","last_fullsized":"2023-10-16T11:24:40.918129+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:54.385890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072402+0000","last_change":"2023-10-16T11:24:40.925767+0000","last_active":"2023-10-16T11:24:42.072402+0000","last_peered":"2023-10-16T11:24:42.072402+0000","last_clean":"2023-10-16T11:24:42.072402+0000","last_became_active":"2023-10-16T11:24:40.925501+0000","last_became_peered":"2023-10-16T11:24:40.925501+0000","last_unstale":"2023-10-16T11:24:42.072402+0000","last_undegraded":"2023-10-16T11:24:42.072402+0000","last_fullsized":"2023-10-16T11:24:42.072402+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:49:08.490072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.15","version":"216'3","reported_seq":19,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920628+0000","last_change":"2023-10-16T11:24:25.082968+0000","last_active":"2023-10-16T11:24:40.920628+0000","last_peered":"2023-10-16T11:24:40.920628+0000","last_clean":"2023-10-16T11:24:40.920628+0000","last_became_active":"2023-10-16T11:24:25.082316+0000","last_became_peered":"2023-10-16T11:24:25.082316+0000","last_unstale":"2023-10-16T11:24:40.920628+0000","last_undegraded":"2023-10-16T11:24:40.920628+0000","last_fullsized":"2023-10-16T11:24:40.920628+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:59.924424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.922137+0000","last_change":"2023-10-16T11:24:26.084487+0000","last_active":"2023-10-16T11:24:40.922137+0000","last_peered":"2023-10-16T11:24:40.922137+0000","last_clean":"2023-10-16T11:24:40.922137+0000","last_became_active":"2023-10-16T11:24:26.084188+0000","last_became_peered":"2023-10-16T11:24:26.084188+0000","last_unstale":"2023-10-16T11:24:40.922137+0000","last_undegraded":"2023-10-16T11:24:40.922137+0000","last_fullsized":"2023-10-16T11:24:40.922137+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:47.400027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.945716+0000","last_change":"2023-10-16T11:24:39.928051+0000","last_active":"2023-10-16T11:24:40.945716+0000","last_peered":"2023-10-16T11:24:40.945716+0000","last_clean":"2023-10-16T11:24:40.945716+0000","last_became_active":"2023-10-16T11:24:39.927740+0000","last_became_peered":"2023-10-16T11:24:39.927740+0000","last_unstale":"2023-10-16T11:24:40.945716+0000","last_undegraded":"2023-10-16T11:24:40.945716+0000","last_fullsized":"2023-10-16T11:24:40.945716+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:07.682759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.075589+0000","last_change":"2023-10-16T11:24:40.931397+0000","last_active":"2023-10-16T11:24:42.075589+0000","last_peered":"2023-10-16T11:24:42.075589+0000","last_clean":"2023-10-16T11:24:42.075589+0000","last_became_active":"2023-10-16T11:24:40.931130+0000","last_became_peered":"2023-10-16T11:24:40.931130+0000","last_unstale":"2023-10-16T11:24:42.075589+0000","last_undegraded":"2023-10-16T11:24:42.075589+0000","last_fullsized":"2023-10-16T11:24:42.075589+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:13.011085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.926531+0000","last_change":"2023-10-16T11:24:25.092100+0000","last_active":"2023-10-16T11:24:40.926531+0000","last_peered":"2023-10-16T11:24:40.926531+0000","last_clean":"2023-10-16T11:24:40.926531+0000","last_became_active":"2023-10-16T11:24:25.091237+0000","last_became_peered":"2023-10-16T11:24:25.091237+0000","last_unstale":"2023-10-16T11:24:40.926531+0000","last_undegraded":"2023-10-16T11:24:40.926531+0000","last_fullsized":"2023-10-16T11:24:40.926531+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:21.985839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.909348+0000","last_change":"2023-10-16T11:24:26.081980+0000","last_active":"2023-10-16T11:24:40.909348+0000","last_peered":"2023-10-16T11:24:40.909348+0000","last_clean":"2023-10-16T11:24:40.909348+0000","last_became_active":"2023-10-16T11:24:26.081395+0000","last_became_peered":"2023-10-16T11:24:26.081395+0000","last_unstale":"2023-10-16T11:24:40.909348+0000","last_undegraded":"2023-10-16T11:24:40.909348+0000","last_fullsized":"2023-10-16T11:24:40.909348+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:39.802117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"220'8","reported_seq":20,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.949404+0000","last_change":"2023-10-16T11:24:39.915652+0000","last_active":"2023-10-16T11:24:40.949404+0000","last_peered":"2023-10-16T11:24:40.949404+0000","last_clean":"2023-10-16T11:24:40.949404+0000","last_became_active":"2023-10-16T11:24:39.915277+0000","last_became_peered":"2023-10-16T11:24:39.915277+0000","last_unstale":"2023-10-16T11:24:40.949404+0000","last_undegraded":"2023-10-16T11:24:40.949404+0000","last_fullsized":"2023-10-16T11:24:40.949404+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:49.377967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.080669+0000","last_change":"2023-10-16T11:24:40.938686+0000","last_active":"2023-10-16T11:24:42.080669+0000","last_peered":"2023-10-16T11:24:42.080669+0000","last_clean":"2023-10-16T11:24:42.080669+0000","last_became_active":"2023-10-16T11:24:40.937440+0000","last_became_peered":"2023-10-16T11:24:40.937440+0000","last_unstale":"2023-10-16T11:24:42.080669+0000","last_undegraded":"2023-10-16T11:24:42.080669+0000","last_fullsized":"2023-10-16T11:24:42.080669+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:08:17.799924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913537+0000","last_change":"2023-10-16T11:24:25.094384+0000","last_active":"2023-10-16T11:24:40.913537+0000","last_peered":"2023-10-16T11:24:40.913537+0000","last_clean":"2023-10-16T11:24:40.913537+0000","last_became_active":"2023-10-16T11:24:25.094026+0000","last_became_peered":"2023-10-16T11:24:25.094026+0000","last_unstale":"2023-10-16T11:24:40.913537+0000","last_undegraded":"2023-10-16T11:24:40.913537+0000","last_fullsized":"2023-10-16T11:24:40.913537+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:51.349970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.928344+0000","last_change":"2023-10-16T11:24:26.082915+0000","last_active":"2023-10-16T11:24:40.928344+0000","last_peered":"2023-10-16T11:24:40.928344+0000","last_clean":"2023-10-16T11:24:40.928344+0000","last_became_active":"2023-10-16T11:24:26.082458+0000","last_became_peered":"2023-10-16T11:24:26.082458+0000","last_unstale":"2023-10-16T11:24:40.928344+0000","last_undegraded":"2023-10-16T11:24:40.928344+0000","last_fullsized":"2023-10-16T11:24:40.928344+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:10.387018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913132+0000","last_change":"2023-10-16T11:24:39.903226+0000","last_active":"2023-10-16T11:24:40.913132+0000","last_peered":"2023-10-16T11:24:40.913132+0000","last_clean":"2023-10-16T11:24:40.913132+0000","last_became_active":"2023-10-16T11:24:39.902943+0000","last_became_peered":"2023-10-16T11:24:39.902943+0000","last_unstale":"2023-10-16T11:24:40.913132+0000","last_undegraded":"2023-10-16T11:24:40.913132+0000","last_fullsized":"2023-10-16T11:24:40.913132+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:28:29.998083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.936311+0000","last_change":"2023-10-16T11:24:40.938912+0000","last_active":"2023-10-16T11:24:41.936311+0000","last_peered":"2023-10-16T11:24:41.936311+0000","last_clean":"2023-10-16T11:24:41.936311+0000","last_became_active":"2023-10-16T11:24:40.938615+0000","last_became_peered":"2023-10-16T11:24:40.938615+0000","last_unstale":"2023-10-16T11:24:41.936311+0000","last_undegraded":"2023-10-16T11:24:41.936311+0000","last_fullsized":"2023-10-16T11:24:41.936311+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:38.047063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"220'1","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.947170+0000","last_change":"2023-10-16T11:24:39.916128+0000","last_active":"2023-10-16T11:24:40.947170+0000","last_peered":"2023-10-16T11:24:40.947170+0000","last_clean":"2023-10-16T11:24:40.947170+0000","last_became_active":"2023-10-16T11:24:39.911215+0000","last_became_peered":"2023-10-16T11:24:39.911215+0000","last_unstale":"2023-10-16T11:24:40.947170+0000","last_undegraded":"2023-10-16T11:24:40.947170+0000","last_fullsized":"2023-10-16T11:24:40.947170+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:59:34.809544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927021+0000","last_change":"2023-10-16T11:24:25.084521+0000","last_active":"2023-10-16T11:24:40.927021+0000","last_peered":"2023-10-16T11:24:40.927021+0000","last_clean":"2023-10-16T11:24:40.927021+0000","last_became_active":"2023-10-16T11:24:25.084003+0000","last_became_peered":"2023-10-16T11:24:25.084003+0000","last_unstale":"2023-10-16T11:24:40.927021+0000","last_undegraded":"2023-10-16T11:24:40.927021+0000","last_fullsized":"2023-10-16T11:24:40.927021+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:36.060066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.919781+0000","last_change":"2023-10-16T11:24:26.091641+0000","last_active":"2023-10-16T11:24:40.919781+0000","last_peered":"2023-10-16T11:24:40.919781+0000","last_clean":"2023-10-16T11:24:40.919781+0000","last_became_active":"2023-10-16T11:24:26.089186+0000","last_became_peered":"2023-10-16T11:24:26.089186+0000","last_unstale":"2023-10-16T11:24:40.919781+0000","last_undegraded":"2023-10-16T11:24:40.919781+0000","last_fullsized":"2023-10-16T11:24:40.919781+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:32.247878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.082815+0000","last_change":"2023-10-16T11:24:40.946117+0000","last_active":"2023-10-16T11:24:42.082815+0000","last_peered":"2023-10-16T11:24:42.082815+0000","last_clean":"2023-10-16T11:24:42.082815+0000","last_became_active":"2023-10-16T11:24:40.943815+0000","last_became_peered":"2023-10-16T11:24:40.943815+0000","last_unstale":"2023-10-16T11:24:42.082815+0000","last_undegraded":"2023-10-16T11:24:42.082815+0000","last_fullsized":"2023-10-16T11:24:42.082815+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:08.697558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"220'4","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.947367+0000","last_change":"2023-10-16T11:24:39.915794+0000","last_active":"2023-10-16T11:24:40.947367+0000","last_peered":"2023-10-16T11:24:40.947367+0000","last_clean":"2023-10-16T11:24:40.947367+0000","last_became_active":"2023-10-16T11:24:39.915438+0000","last_became_peered":"2023-10-16T11:24:39.915438+0000","last_unstale":"2023-10-16T11:24:40.947367+0000","last_undegraded":"2023-10-16T11:24:40.947367+0000","last_fullsized":"2023-10-16T11:24:40.947367+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:59.121334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913267+0000","last_change":"2023-10-16T11:24:25.074647+0000","last_active":"2023-10-16T11:24:40.913267+0000","last_peered":"2023-10-16T11:24:40.913267+0000","last_clean":"2023-10-16T11:24:40.913267+0000","last_became_active":"2023-10-16T11:24:25.074373+0000","last_became_peered":"2023-10-16T11:24:25.074373+0000","last_unstale":"2023-10-16T11:24:40.913267+0000","last_undegraded":"2023-10-16T11:24:40.913267+0000","last_fullsized":"2023-10-16T11:24:40.913267+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:38.544520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912670+0000","last_change":"2023-10-16T11:24:26.088785+0000","last_active":"2023-10-16T11:24:40.912670+0000","last_peered":"2023-10-16T11:24:40.912670+0000","last_clean":"2023-10-16T11:24:40.912670+0000","last_became_active":"2023-10-16T11:24:26.088521+0000","last_became_peered":"2023-10-16T11:24:26.088521+0000","last_unstale":"2023-10-16T11:24:40.912670+0000","last_undegraded":"2023-10-16T11:24:40.912670+0000","last_fullsized":"2023-10-16T11:24:40.912670+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:29.145795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.078267+0000","last_change":"2023-10-16T11:24:40.950578+0000","last_active":"2023-10-16T11:24:42.078267+0000","last_peered":"2023-10-16T11:24:42.078267+0000","last_clean":"2023-10-16T11:24:42.078267+0000","last_became_active":"2023-10-16T11:24:40.947861+0000","last_became_peered":"2023-10-16T11:24:40.947861+0000","last_unstale":"2023-10-16T11:24:42.078267+0000","last_undegraded":"2023-10-16T11:24:42.078267+0000","last_fullsized":"2023-10-16T11:24:42.078267+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:49:08.490072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1a","version":"216'1","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912820+0000","last_change":"2023-10-16T11:24:25.084983+0000","last_active":"2023-10-16T11:24:40.912820+0000","last_peered":"2023-10-16T11:24:40.912820+0000","last_clean":"2023-10-16T11:24:40.912820+0000","last_became_active":"2023-10-16T11:24:25.084622+0000","last_became_peered":"2023-10-16T11:24:25.084622+0000","last_unstale":"2023-10-16T11:24:40.912820+0000","last_undegraded":"2023-10-16T11:24:40.912820+0000","last_fullsized":"2023-10-16T11:24:40.912820+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:36.588809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.921894+0000","last_change":"2023-10-16T11:24:26.089982+0000","last_active":"2023-10-16T11:24:40.921894+0000","last_peered":"2023-10-16T11:24:40.921894+0000","last_clean":"2023-10-16T11:24:40.921894+0000","last_became_active":"2023-10-16T11:24:26.089677+0000","last_became_peered":"2023-10-16T11:24:26.089677+0000","last_unstale":"2023-10-16T11:24:40.921894+0000","last_undegraded":"2023-10-16T11:24:40.921894+0000","last_fullsized":"2023-10-16T11:24:40.921894+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:22.009042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.940586+0000","last_change":"2023-10-16T11:24:39.914078+0000","last_active":"2023-10-16T11:24:40.940586+0000","last_peered":"2023-10-16T11:24:40.940586+0000","last_clean":"2023-10-16T11:24:40.940586+0000","last_became_active":"2023-10-16T11:24:39.913693+0000","last_became_peered":"2023-10-16T11:24:39.913693+0000","last_unstale":"2023-10-16T11:24:40.940586+0000","last_undegraded":"2023-10-16T11:24:40.940586+0000","last_fullsized":"2023-10-16T11:24:40.940586+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:08:16.785125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.073246+0000","last_change":"2023-10-16T11:24:40.943302+0000","last_active":"2023-10-16T11:24:42.073246+0000","last_peered":"2023-10-16T11:24:42.073246+0000","last_clean":"2023-10-16T11:24:42.073246+0000","last_became_active":"2023-10-16T11:24:40.939215+0000","last_became_peered":"2023-10-16T11:24:40.939215+0000","last_unstale":"2023-10-16T11:24:42.073246+0000","last_undegraded":"2023-10-16T11:24:42.073246+0000","last_fullsized":"2023-10-16T11:24:42.073246+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:22.925745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1c","version":"216'2","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.922233+0000","last_change":"2023-10-16T11:24:25.095179+0000","last_active":"2023-10-16T11:24:40.922233+0000","last_peered":"2023-10-16T11:24:40.922233+0000","last_clean":"2023-10-16T11:24:40.922233+0000","last_became_active":"2023-10-16T11:24:25.094714+0000","last_became_peered":"2023-10-16T11:24:25.094714+0000","last_unstale":"2023-10-16T11:24:40.922233+0000","last_undegraded":"2023-10-16T11:24:40.922233+0000","last_fullsized":"2023-10-16T11:24:40.922233+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:26.128614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920276+0000","last_change":"2023-10-16T11:24:26.096297+0000","last_active":"2023-10-16T11:24:40.920276+0000","last_peered":"2023-10-16T11:24:40.920276+0000","last_clean":"2023-10-16T11:24:40.920276+0000","last_became_active":"2023-10-16T11:24:26.095538+0000","last_became_peered":"2023-10-16T11:24:26.095538+0000","last_unstale":"2023-10-16T11:24:40.920276+0000","last_undegraded":"2023-10-16T11:24:40.920276+0000","last_fullsized":"2023-10-16T11:24:40.920276+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:47.400027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"220'1","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.946397+0000","last_change":"2023-10-16T11:24:39.923043+0000","last_active":"2023-10-16T11:24:40.946397+0000","last_peered":"2023-10-16T11:24:40.946397+0000","last_clean":"2023-10-16T11:24:40.946397+0000","last_became_active":"2023-10-16T11:24:39.922745+0000","last_became_peered":"2023-10-16T11:24:39.922745+0000","last_unstale":"2023-10-16T11:24:40.946397+0000","last_undegraded":"2023-10-16T11:24:40.946397+0000","last_fullsized":"2023-10-16T11:24:40.946397+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:03.774885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.075183+0000","last_change":"2023-10-16T11:24:40.948393+0000","last_active":"2023-10-16T11:24:42.075183+0000","last_peered":"2023-10-16T11:24:42.075183+0000","last_clean":"2023-10-16T11:24:42.075183+0000","last_became_active":"2023-10-16T11:24:40.946737+0000","last_became_peered":"2023-10-16T11:24:40.946737+0000","last_unstale":"2023-10-16T11:24:42.075183+0000","last_undegraded":"2023-10-16T11:24:42.075183+0000","last_fullsized":"2023-10-16T11:24:42.075183+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:49:23.318330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"216'2","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.909085+0000","last_change":"2023-10-16T11:24:25.081731+0000","last_active":"2023-10-16T11:24:40.909085+0000","last_peered":"2023-10-16T11:24:40.909085+0000","last_clean":"2023-10-16T11:24:40.909085+0000","last_became_active":"2023-10-16T11:24:25.081336+0000","last_became_peered":"2023-10-16T11:24:25.081336+0000","last_unstale":"2023-10-16T11:24:40.909085+0000","last_undegraded":"2023-10-16T11:24:40.909085+0000","last_fullsized":"2023-10-16T11:24:40.909085+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:09.330801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918704+0000","last_change":"2023-10-16T11:24:26.089791+0000","last_active":"2023-10-16T11:24:40.918704+0000","last_peered":"2023-10-16T11:24:40.918704+0000","last_clean":"2023-10-16T11:24:40.918704+0000","last_became_active":"2023-10-16T11:24:26.089516+0000","last_became_peered":"2023-10-16T11:24:26.089516+0000","last_unstale":"2023-10-16T11:24:40.918704+0000","last_undegraded":"2023-10-16T11:24:40.918704+0000","last_fullsized":"2023-10-16T11:24:40.918704+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"220'1","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.941517+0000","last_change":"2023-10-16T11:24:39.901227+0000","last_active":"2023-10-16T11:24:40.941517+0000","last_peered":"2023-10-16T11:24:40.941517+0000","last_clean":"2023-10-16T11:24:40.941517+0000","last_became_active":"2023-10-16T11:24:39.900852+0000","last_became_peered":"2023-10-16T11:24:39.900852+0000","last_unstale":"2023-10-16T11:24:40.941517+0000","last_undegraded":"2023-10-16T11:24:40.941517+0000","last_fullsized":"2023-10-16T11:24:40.941517+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:54.404259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935060+0000","last_change":"2023-10-16T11:24:40.944861+0000","last_active":"2023-10-16T11:24:41.935060+0000","last_peered":"2023-10-16T11:24:41.935060+0000","last_clean":"2023-10-16T11:24:41.935060+0000","last_became_active":"2023-10-16T11:24:40.944579+0000","last_became_peered":"2023-10-16T11:24:40.944579+0000","last_unstale":"2023-10-16T11:24:41.935060+0000","last_undegraded":"2023-10-16T11:24:41.935060+0000","last_fullsized":"2023-10-16T11:24:41.935060+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:47:20.730005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"216'1","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.933239+0000","last_change":"2023-10-16T11:24:25.094483+0000","last_active":"2023-10-16T11:24:40.933239+0000","last_peered":"2023-10-16T11:24:40.933239+0000","last_clean":"2023-10-16T11:24:40.933239+0000","last_became_active":"2023-10-16T11:24:25.094134+0000","last_became_peered":"2023-10-16T11:24:25.094134+0000","last_unstale":"2023-10-16T11:24:40.933239+0000","last_undegraded":"2023-10-16T11:24:40.933239+0000","last_fullsized":"2023-10-16T11:24:40.933239+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.923291+0000","last_change":"2023-10-16T11:24:26.094536+0000","last_active":"2023-10-16T11:24:40.923291+0000","last_peered":"2023-10-16T11:24:40.923291+0000","last_clean":"2023-10-16T11:24:40.923291+0000","last_became_active":"2023-10-16T11:24:26.094272+0000","last_became_peered":"2023-10-16T11:24:26.094272+0000","last_unstale":"2023-10-16T11:24:40.923291+0000","last_undegraded":"2023-10-16T11:24:40.923291+0000","last_fullsized":"2023-10-16T11:24:40.923291+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.924102+0000","last_change":"2023-10-16T11:24:39.916827+0000","last_active":"2023-10-16T11:24:40.924102+0000","last_peered":"2023-10-16T11:24:40.924102+0000","last_clean":"2023-10-16T11:24:40.924102+0000","last_became_active":"2023-10-16T11:24:39.916464+0000","last_became_peered":"2023-10-16T11:24:39.916464+0000","last_unstale":"2023-10-16T11:24:40.924102+0000","last_undegraded":"2023-10-16T11:24:40.924102+0000","last_fullsized":"2023-10-16T11:24:40.924102+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:25.891782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.932973+0000","last_change":"2023-10-16T11:24:40.928353+0000","last_active":"2023-10-16T11:24:41.932973+0000","last_peered":"2023-10-16T11:24:41.932973+0000","last_clean":"2023-10-16T11:24:41.932973+0000","last_became_active":"2023-10-16T11:24:40.928042+0000","last_became_peered":"2023-10-16T11:24:40.928042+0000","last_unstale":"2023-10-16T11:24:41.932973+0000","last_undegraded":"2023-10-16T11:24:41.932973+0000","last_fullsized":"2023-10-16T11:24:41.932973+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:57:58.304427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1f","version":"216'4","reported_seq":20,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918030+0000","last_change":"2023-10-16T11:24:25.096644+0000","last_active":"2023-10-16T11:24:40.918030+0000","last_peered":"2023-10-16T11:24:40.918030+0000","last_clean":"2023-10-16T11:24:40.918030+0000","last_became_active":"2023-10-16T11:24:25.096306+0000","last_became_peered":"2023-10-16T11:24:25.096306+0000","last_unstale":"2023-10-16T11:24:40.918030+0000","last_undegraded":"2023-10-16T11:24:40.918030+0000","last_fullsized":"2023-10-16T11:24:40.918030+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:22.618393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.933050+0000","last_change":"2023-10-16T11:24:26.069227+0000","last_active":"2023-10-16T11:24:40.933050+0000","last_peered":"2023-10-16T11:24:40.933050+0000","last_clean":"2023-10-16T11:24:40.933050+0000","last_became_active":"2023-10-16T11:24:26.068456+0000","last_became_peered":"2023-10-16T11:24:26.068456+0000","last_unstale":"2023-10-16T11:24:40.933050+0000","last_undegraded":"2023-10-16T11:24:40.933050+0000","last_fullsized":"2023-10-16T11:24:40.933050+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927501+0000","last_change":"2023-10-16T11:24:39.916911+0000","last_active":"2023-10-16T11:24:40.927501+0000","last_peered":"2023-10-16T11:24:40.927501+0000","last_clean":"2023-10-16T11:24:40.927501+0000","last_became_active":"2023-10-16T11:24:39.916584+0000","last_became_peered":"2023-10-16T11:24:39.916584+0000","last_unstale":"2023-10-16T11:24:40.927501+0000","last_undegraded":"2023-10-16T11:24:40.927501+0000","last_fullsized":"2023-10-16T11:24:40.927501+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:12.120581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935402+0000","last_change":"2023-10-16T11:24:40.929604+0000","last_active":"2023-10-16T11:24:41.935402+0000","last_peered":"2023-10-16T11:24:41.935402+0000","last_clean":"2023-10-16T11:24:41.935402+0000","last_became_active":"2023-10-16T11:24:40.929340+0000","last_became_peered":"2023-10-16T11:24:40.929340+0000","last_unstale":"2023-10-16T11:24:41.935402+0000","last_undegraded":"2023-10-16T11:24:41.935402+0000","last_fullsized":"2023-10-16T11:24:41.935402+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:18.688225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.916906+0000","last_change":"2023-10-16T11:24:26.085375+0000","last_active":"2023-10-16T11:24:40.916906+0000","last_peered":"2023-10-16T11:24:40.916906+0000","last_clean":"2023-10-16T11:24:40.916906+0000","last_became_active":"2023-10-16T11:24:26.085120+0000","last_became_peered":"2023-10-16T11:24:26.085120+0000","last_unstale":"2023-10-16T11:24:40.916906+0000","last_undegraded":"2023-10-16T11:24:40.916906+0000","last_fullsized":"2023-10-16T11:24:40.916906+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.079858+0000","last_change":"2023-10-16T11:24:40.943381+0000","last_active":"2023-10-16T11:24:42.079858+0000","last_peered":"2023-10-16T11:24:42.079858+0000","last_clean":"2023-10-16T11:24:42.079858+0000","last_became_active":"2023-10-16T11:24:40.941771+0000","last_became_peered":"2023-10-16T11:24:40.941771+0000","last_unstale":"2023-10-16T11:24:42.079858+0000","last_undegraded":"2023-10-16T11:24:42.079858+0000","last_fullsized":"2023-10-16T11:24:42.079858+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:30:10.159127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.923208+0000","last_change":"2023-10-16T11:24:26.084964+0000","last_active":"2023-10-16T11:24:40.923208+0000","last_peered":"2023-10-16T11:24:40.923208+0000","last_clean":"2023-10-16T11:24:40.923208+0000","last_became_active":"2023-10-16T11:24:26.084006+0000","last_became_peered":"2023-10-16T11:24:26.084006+0000","last_unstale":"2023-10-16T11:24:40.923208+0000","last_undegraded":"2023-10-16T11:24:40.923208+0000","last_fullsized":"2023-10-16T11:24:40.923208+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:03.949526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.934466+0000","last_change":"2023-10-16T11:24:40.937991+0000","last_active":"2023-10-16T11:24:41.934466+0000","last_peered":"2023-10-16T11:24:41.934466+0000","last_clean":"2023-10-16T11:24:41.934466+0000","last_became_active":"2023-10-16T11:24:40.934656+0000","last_became_peered":"2023-10-16T11:24:40.934656+0000","last_unstale":"2023-10-16T11:24:41.934466+0000","last_undegraded":"2023-10-16T11:24:41.934466+0000","last_fullsized":"2023-10-16T11:24:41.934466+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:49:23.318330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.931374+0000","last_change":"2023-10-16T11:24:26.086008+0000","last_active":"2023-10-16T11:24:40.931374+0000","last_peered":"2023-10-16T11:24:40.931374+0000","last_clean":"2023-10-16T11:24:40.931374+0000","last_became_active":"2023-10-16T11:24:26.085598+0000","last_became_peered":"2023-10-16T11:24:26.085598+0000","last_unstale":"2023-10-16T11:24:40.931374+0000","last_undegraded":"2023-10-16T11:24:40.931374+0000","last_fullsized":"2023-10-16T11:24:40.931374+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.077368+0000","last_change":"2023-10-16T11:24:40.941097+0000","last_active":"2023-10-16T11:24:42.077368+0000","last_peered":"2023-10-16T11:24:42.077368+0000","last_clean":"2023-10-16T11:24:42.077368+0000","last_became_active":"2023-10-16T11:24:40.938428+0000","last_became_peered":"2023-10-16T11:24:40.938428+0000","last_unstale":"2023-10-16T11:24:42.077368+0000","last_undegraded":"2023-10-16T11:24:42.077368+0000","last_fullsized":"2023-10-16T11:24:42.077368+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:12.311557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.915966+0000","last_change":"2023-10-16T11:24:26.086172+0000","last_active":"2023-10-16T11:24:40.915966+0000","last_peered":"2023-10-16T11:24:40.915966+0000","last_clean":"2023-10-16T11:24:40.915966+0000","last_became_active":"2023-10-16T11:24:26.085882+0000","last_became_peered":"2023-10-16T11:24:26.085882+0000","last_unstale":"2023-10-16T11:24:40.915966+0000","last_undegraded":"2023-10-16T11:24:40.915966+0000","last_fullsized":"2023-10-16T11:24:40.915966+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:44.265192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076084+0000","last_change":"2023-10-16T11:24:40.950225+0000","last_active":"2023-10-16T11:24:42.076084+0000","last_peered":"2023-10-16T11:24:42.076084+0000","last_clean":"2023-10-16T11:24:42.076084+0000","last_became_active":"2023-10-16T11:24:40.948659+0000","last_became_peered":"2023-10-16T11:24:40.948659+0000","last_unstale":"2023-10-16T11:24:42.076084+0000","last_undegraded":"2023-10-16T11:24:42.076084+0000","last_fullsized":"2023-10-16T11:24:42.076084+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:46.340170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.907529+0000","last_change":"2023-10-16T11:24:26.086228+0000","last_active":"2023-10-16T11:24:40.907529+0000","last_peered":"2023-10-16T11:24:40.907529+0000","last_clean":"2023-10-16T11:24:40.907529+0000","last_became_active":"2023-10-16T11:24:26.085912+0000","last_became_peered":"2023-10-16T11:24:26.085912+0000","last_unstale":"2023-10-16T11:24:40.907529+0000","last_undegraded":"2023-10-16T11:24:40.907529+0000","last_fullsized":"2023-10-16T11:24:40.907529+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.075408+0000","last_change":"2023-10-16T11:24:40.949598+0000","last_active":"2023-10-16T11:24:42.075408+0000","last_peered":"2023-10-16T11:24:42.075408+0000","last_clean":"2023-10-16T11:24:42.075408+0000","last_became_active":"2023-10-16T11:24:40.949322+0000","last_became_peered":"2023-10-16T11:24:40.949322+0000","last_unstale":"2023-10-16T11:24:42.075408+0000","last_undegraded":"2023-10-16T11:24:42.075408+0000","last_fullsized":"2023-10-16T11:24:42.075408+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:22:53.973191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.915484+0000","last_change":"2023-10-16T11:24:26.088015+0000","last_active":"2023-10-16T11:24:40.915484+0000","last_peered":"2023-10-16T11:24:40.915484+0000","last_clean":"2023-10-16T11:24:40.915484+0000","last_became_active":"2023-10-16T11:24:26.086932+0000","last_became_peered":"2023-10-16T11:24:26.086932+0000","last_unstale":"2023-10-16T11:24:40.915484+0000","last_undegraded":"2023-10-16T11:24:40.915484+0000","last_fullsized":"2023-10-16T11:24:40.915484+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:44.137821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.071053+0000","last_change":"2023-10-16T11:24:40.935550+0000","last_active":"2023-10-16T11:24:42.071053+0000","last_peered":"2023-10-16T11:24:42.071053+0000","last_clean":"2023-10-16T11:24:42.071053+0000","last_became_active":"2023-10-16T11:24:40.935237+0000","last_became_peered":"2023-10-16T11:24:40.935237+0000","last_unstale":"2023-10-16T11:24:42.071053+0000","last_undegraded":"2023-10-16T11:24:42.071053+0000","last_fullsized":"2023-10-16T11:24:42.071053+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:13.011085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918841+0000","last_change":"2023-10-16T11:24:26.095913+0000","last_active":"2023-10-16T11:24:40.918841+0000","last_peered":"2023-10-16T11:24:40.918841+0000","last_clean":"2023-10-16T11:24:40.918841+0000","last_became_active":"2023-10-16T11:24:26.093113+0000","last_became_peered":"2023-10-16T11:24:26.093113+0000","last_unstale":"2023-10-16T11:24:40.918841+0000","last_undegraded":"2023-10-16T11:24:40.918841+0000","last_fullsized":"2023-10-16T11:24:40.918841+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072486+0000","last_change":"2023-10-16T11:24:40.932619+0000","last_active":"2023-10-16T11:24:42.072486+0000","last_peered":"2023-10-16T11:24:42.072486+0000","last_clean":"2023-10-16T11:24:42.072486+0000","last_became_active":"2023-10-16T11:24:40.930562+0000","last_became_peered":"2023-10-16T11:24:40.930562+0000","last_unstale":"2023-10-16T11:24:42.072486+0000","last_undegraded":"2023-10-16T11:24:42.072486+0000","last_fullsized":"2023-10-16T11:24:42.072486+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:11.184845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.932735+0000","last_change":"2023-10-16T11:24:26.094066+0000","last_active":"2023-10-16T11:24:40.932735+0000","last_peered":"2023-10-16T11:24:40.932735+0000","last_clean":"2023-10-16T11:24:40.932735+0000","last_became_active":"2023-10-16T11:24:26.093428+0000","last_became_peered":"2023-10-16T11:24:26.093428+0000","last_unstale":"2023-10-16T11:24:40.932735+0000","last_undegraded":"2023-10-16T11:24:40.932735+0000","last_fullsized":"2023-10-16T11:24:40.932735+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935302+0000","last_change":"2023-10-16T11:24:40.944382+0000","last_active":"2023-10-16T11:24:41.935302+0000","last_peered":"2023-10-16T11:24:41.935302+0000","last_clean":"2023-10-16T11:24:41.935302+0000","last_became_active":"2023-10-16T11:24:40.943690+0000","last_became_peered":"2023-10-16T11:24:40.943690+0000","last_unstale":"2023-10-16T11:24:41.935302+0000","last_undegraded":"2023-10-16T11:24:41.935302+0000","last_fullsized":"2023-10-16T11:24:41.935302+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:15.346746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.921063+0000","last_change":"2023-10-16T11:24:26.091894+0000","last_active":"2023-10-16T11:24:40.921063+0000","last_peered":"2023-10-16T11:24:40.921063+0000","last_clean":"2023-10-16T11:24:40.921063+0000","last_became_active":"2023-10-16T11:24:26.091627+0000","last_became_peered":"2023-10-16T11:24:26.091627+0000","last_unstale":"2023-10-16T11:24:40.921063+0000","last_undegraded":"2023-10-16T11:24:40.921063+0000","last_fullsized":"2023-10-16T11:24:40.921063+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:46:59.832609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.081684+0000","last_change":"2023-10-16T11:24:40.941368+0000","last_active":"2023-10-16T11:24:42.081684+0000","last_peered":"2023-10-16T11:24:42.081684+0000","last_clean":"2023-10-16T11:24:42.081684+0000","last_became_active":"2023-10-16T11:24:40.940598+0000","last_became_peered":"2023-10-16T11:24:40.940598+0000","last_unstale":"2023-10-16T11:24:42.081684+0000","last_undegraded":"2023-10-16T11:24:42.081684+0000","last_fullsized":"2023-10-16T11:24:42.081684+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:50:47.255058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927515+0000","last_change":"2023-10-16T11:24:26.097221+0000","last_active":"2023-10-16T11:24:40.927515+0000","last_peered":"2023-10-16T11:24:40.927515+0000","last_clean":"2023-10-16T11:24:40.927515+0000","last_became_active":"2023-10-16T11:24:26.096572+0000","last_became_peered":"2023-10-16T11:24:26.096572+0000","last_unstale":"2023-10-16T11:24:40.927515+0000","last_undegraded":"2023-10-16T11:24:40.927515+0000","last_fullsized":"2023-10-16T11:24:40.927515+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.934384+0000","last_change":"2023-10-16T11:24:40.935022+0000","last_active":"2023-10-16T11:24:41.934384+0000","last_peered":"2023-10-16T11:24:41.934384+0000","last_clean":"2023-10-16T11:24:41.934384+0000","last_became_active":"2023-10-16T11:24:40.934325+0000","last_became_peered":"2023-10-16T11:24:40.934325+0000","last_unstale":"2023-10-16T11:24:41.934384+0000","last_undegraded":"2023-10-16T11:24:41.934384+0000","last_fullsized":"2023-10-16T11:24:41.934384+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:58.258235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.924840+0000","last_change":"2023-10-16T11:24:26.101039+0000","last_active":"2023-10-16T11:24:40.924840+0000","last_peered":"2023-10-16T11:24:40.924840+0000","last_clean":"2023-10-16T11:24:40.924840+0000","last_became_active":"2023-10-16T11:24:26.100757+0000","last_became_peered":"2023-10-16T11:24:26.100757+0000","last_unstale":"2023-10-16T11:24:40.924840+0000","last_undegraded":"2023-10-16T11:24:40.924840+0000","last_fullsized":"2023-10-16T11:24:40.924840+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.073731+0000","last_change":"2023-10-16T11:24:40.934747+0000","last_active":"2023-10-16T11:24:42.073731+0000","last_peered":"2023-10-16T11:24:42.073731+0000","last_clean":"2023-10-16T11:24:42.073731+0000","last_became_active":"2023-10-16T11:24:40.934319+0000","last_became_peered":"2023-10-16T11:24:40.934319+0000","last_unstale":"2023-10-16T11:24:42.073731+0000","last_undegraded":"2023-10-16T11:24:42.073731+0000","last_fullsized":"2023-10-16T11:24:42.073731+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:12:33.440509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.921807+0000","last_change":"2023-10-16T11:24:26.092718+0000","last_active":"2023-10-16T11:24:40.921807+0000","last_peered":"2023-10-16T11:24:40.921807+0000","last_clean":"2023-10-16T11:24:40.921807+0000","last_became_active":"2023-10-16T11:24:26.091377+0000","last_became_peered":"2023-10-16T11:24:26.091377+0000","last_unstale":"2023-10-16T11:24:40.921807+0000","last_undegraded":"2023-10-16T11:24:40.921807+0000","last_fullsized":"2023-10-16T11:24:40.921807+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:17.072626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072172+0000","last_change":"2023-10-16T11:24:40.933022+0000","last_active":"2023-10-16T11:24:42.072172+0000","last_peered":"2023-10-16T11:24:42.072172+0000","last_clean":"2023-10-16T11:24:42.072172+0000","last_became_active":"2023-10-16T11:24:40.932746+0000","last_became_peered":"2023-10-16T11:24:40.932746+0000","last_unstale":"2023-10-16T11:24:42.072172+0000","last_undegraded":"2023-10-16T11:24:42.072172+0000","last_fullsized":"2023-10-16T11:24:42.072172+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:42.730933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.915755+0000","last_change":"2023-10-16T11:24:26.093168+0000","last_active":"2023-10-16T11:24:40.915755+0000","last_peered":"2023-10-16T11:24:40.915755+0000","last_clean":"2023-10-16T11:24:40.915755+0000","last_became_active":"2023-10-16T11:24:26.091801+0000","last_became_peered":"2023-10-16T11:24:26.091801+0000","last_unstale":"2023-10-16T11:24:40.915755+0000","last_undegraded":"2023-10-16T11:24:40.915755+0000","last_fullsized":"2023-10-16T11:24:40.915755+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:43:28.275746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072745+0000","last_change":"2023-10-16T11:24:40.934530+0000","last_active":"2023-10-16T11:24:42.072745+0000","last_peered":"2023-10-16T11:24:42.072745+0000","last_clean":"2023-10-16T11:24:42.072745+0000","last_became_active":"2023-10-16T11:24:40.934262+0000","last_became_peered":"2023-10-16T11:24:40.934262+0000","last_unstale":"2023-10-16T11:24:42.072745+0000","last_undegraded":"2023-10-16T11:24:42.072745+0000","last_fullsized":"2023-10-16T11:24:42.072745+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:30:07.384687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920507+0000","last_change":"2023-10-16T11:24:26.086807+0000","last_active":"2023-10-16T11:24:40.920507+0000","last_peered":"2023-10-16T11:24:40.920507+0000","last_clean":"2023-10-16T11:24:40.920507+0000","last_became_active":"2023-10-16T11:24:26.085382+0000","last_became_peered":"2023-10-16T11:24:26.085382+0000","last_unstale":"2023-10-16T11:24:40.920507+0000","last_undegraded":"2023-10-16T11:24:40.920507+0000","last_fullsized":"2023-10-16T11:24:40.920507+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:56.195463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.071952+0000","last_change":"2023-10-16T11:24:40.931627+0000","last_active":"2023-10-16T11:24:42.071952+0000","last_peered":"2023-10-16T11:24:42.071952+0000","last_clean":"2023-10-16T11:24:42.071952+0000","last_became_active":"2023-10-16T11:24:40.931219+0000","last_became_peered":"2023-10-16T11:24:40.931219+0000","last_unstale":"2023-10-16T11:24:42.071952+0000","last_undegraded":"2023-10-16T11:24:42.071952+0000","last_fullsized":"2023-10-16T11:24:42.071952+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:04.153171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927380+0000","last_change":"2023-10-16T11:24:26.092139+0000","last_active":"2023-10-16T11:24:40.927380+0000","last_peered":"2023-10-16T11:24:40.927380+0000","last_clean":"2023-10-16T11:24:40.927380+0000","last_became_active":"2023-10-16T11:24:26.091860+0000","last_became_peered":"2023-10-16T11:24:26.091860+0000","last_unstale":"2023-10-16T11:24:40.927380+0000","last_undegraded":"2023-10-16T11:24:40.927380+0000","last_fullsized":"2023-10-16T11:24:40.927380+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:00:31.371733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076871+0000","last_change":"2023-10-16T11:24:40.939792+0000","last_active":"2023-10-16T11:24:42.076871+0000","last_peered":"2023-10-16T11:24:42.076871+0000","last_clean":"2023-10-16T11:24:42.076871+0000","last_became_active":"2023-10-16T11:24:40.939547+0000","last_became_peered":"2023-10-16T11:24:40.939547+0000","last_unstale":"2023-10-16T11:24:42.076871+0000","last_undegraded":"2023-10-16T11:24:42.076871+0000","last_fullsized":"2023-10-16T11:24:42.076871+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:44:48.635235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.919569+0000","last_change":"2023-10-16T11:24:26.086029+0000","last_active":"2023-10-16T11:24:40.919569+0000","last_peered":"2023-10-16T11:24:40.919569+0000","last_clean":"2023-10-16T11:24:40.919569+0000","last_became_active":"2023-10-16T11:24:26.085644+0000","last_became_peered":"2023-10-16T11:24:26.085644+0000","last_unstale":"2023-10-16T11:24:40.919569+0000","last_undegraded":"2023-10-16T11:24:40.919569+0000","last_fullsized":"2023-10-16T11:24:40.919569+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:39.802117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.934967+0000","last_change":"2023-10-16T11:24:40.940881+0000","last_active":"2023-10-16T11:24:41.934967+0000","last_peered":"2023-10-16T11:24:41.934967+0000","last_clean":"2023-10-16T11:24:41.934967+0000","last_became_active":"2023-10-16T11:24:40.940465+0000","last_became_peered":"2023-10-16T11:24:40.940465+0000","last_unstale":"2023-10-16T11:24:41.934967+0000","last_undegraded":"2023-10-16T11:24:41.934967+0000","last_fullsized":"2023-10-16T11:24:41.934967+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:39:05.461904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.924194+0000","last_change":"2023-10-16T11:24:26.081938+0000","last_active":"2023-10-16T11:24:40.924194+0000","last_peered":"2023-10-16T11:24:40.924194+0000","last_clean":"2023-10-16T11:24:40.924194+0000","last_became_active":"2023-10-16T11:24:26.081671+0000","last_became_peered":"2023-10-16T11:24:26.081671+0000","last_unstale":"2023-10-16T11:24:40.924194+0000","last_undegraded":"2023-10-16T11:24:40.924194+0000","last_fullsized":"2023-10-16T11:24:40.924194+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:17.072626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.080281+0000","last_change":"2023-10-16T11:24:40.944207+0000","last_active":"2023-10-16T11:24:42.080281+0000","last_peered":"2023-10-16T11:24:42.080281+0000","last_clean":"2023-10-16T11:24:42.080281+0000","last_became_active":"2023-10-16T11:24:40.943832+0000","last_became_peered":"2023-10-16T11:24:40.943832+0000","last_unstale":"2023-10-16T11:24:42.080281+0000","last_undegraded":"2023-10-16T11:24:42.080281+0000","last_fullsized":"2023-10-16T11:24:42.080281+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:43.979587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917335+0000","last_change":"2023-10-16T11:24:26.091403+0000","last_active":"2023-10-16T11:24:40.917335+0000","last_peered":"2023-10-16T11:24:40.917335+0000","last_clean":"2023-10-16T11:24:40.917335+0000","last_became_active":"2023-10-16T11:24:26.091114+0000","last_became_peered":"2023-10-16T11:24:26.091114+0000","last_unstale":"2023-10-16T11:24:40.917335+0000","last_undegraded":"2023-10-16T11:24:40.917335+0000","last_fullsized":"2023-10-16T11:24:40.917335+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935841+0000","last_change":"2023-10-16T11:24:40.936062+0000","last_active":"2023-10-16T11:24:41.935841+0000","last_peered":"2023-10-16T11:24:41.935841+0000","last_clean":"2023-10-16T11:24:41.935841+0000","last_became_active":"2023-10-16T11:24:40.935597+0000","last_became_peered":"2023-10-16T11:24:40.935597+0000","last_unstale":"2023-10-16T11:24:41.935841+0000","last_undegraded":"2023-10-16T11:24:41.935841+0000","last_fullsized":"2023-10-16T11:24:41.935841+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:55.400689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917983+0000","last_change":"2023-10-16T11:24:26.088199+0000","last_active":"2023-10-16T11:24:40.917983+0000","last_peered":"2023-10-16T11:24:40.917983+0000","last_clean":"2023-10-16T11:24:40.917983+0000","last_became_active":"2023-10-16T11:24:26.087892+0000","last_became_peered":"2023-10-16T11:24:26.087892+0000","last_unstale":"2023-10-16T11:24:40.917983+0000","last_undegraded":"2023-10-16T11:24:40.917983+0000","last_fullsized":"2023-10-16T11:24:40.917983+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076815+0000","last_change":"2023-10-16T11:24:40.935695+0000","last_active":"2023-10-16T11:24:42.076815+0000","last_peered":"2023-10-16T11:24:42.076815+0000","last_clean":"2023-10-16T11:24:42.076815+0000","last_became_active":"2023-10-16T11:24:40.935358+0000","last_became_peered":"2023-10-16T11:24:40.935358+0000","last_unstale":"2023-10-16T11:24:42.076815+0000","last_undegraded":"2023-10-16T11:24:42.076815+0000","last_fullsized":"2023-10-16T11:24:42.076815+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:15.346746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.916085+0000","last_change":"2023-10-16T11:24:26.088938+0000","last_active":"2023-10-16T11:24:40.916085+0000","last_peered":"2023-10-16T11:24:40.916085+0000","last_clean":"2023-10-16T11:24:40.916085+0000","last_became_active":"2023-10-16T11:24:26.088689+0000","last_became_peered":"2023-10-16T11:24:26.088689+0000","last_unstale":"2023-10-16T11:24:40.916085+0000","last_undegraded":"2023-10-16T11:24:40.916085+0000","last_fullsized":"2023-10-16T11:24:40.916085+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:18:12.479766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076896+0000","last_change":"2023-10-16T11:24:40.941713+0000","last_active":"2023-10-16T11:24:42.076896+0000","last_peered":"2023-10-16T11:24:42.076896+0000","last_clean":"2023-10-16T11:24:42.076896+0000","last_became_active":"2023-10-16T11:24:40.938875+0000","last_became_peered":"2023-10-16T11:24:40.938875+0000","last_unstale":"2023-10-16T11:24:42.076896+0000","last_undegraded":"2023-10-16T11:24:42.076896+0000","last_fullsized":"2023-10-16T11:24:42.076896+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:42.730933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.915139+0000","last_change":"2023-10-16T11:24:26.089768+0000","last_active":"2023-10-16T11:24:40.915139+0000","last_peered":"2023-10-16T11:24:40.915139+0000","last_clean":"2023-10-16T11:24:40.915139+0000","last_became_active":"2023-10-16T11:24:26.087028+0000","last_became_peered":"2023-10-16T11:24:26.087028+0000","last_unstale":"2023-10-16T11:24:40.915139+0000","last_undegraded":"2023-10-16T11:24:40.915139+0000","last_fullsized":"2023-10-16T11:24:40.915139+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:33.050118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935632+0000","last_change":"2023-10-16T11:24:40.931431+0000","last_active":"2023-10-16T11:24:41.935632+0000","last_peered":"2023-10-16T11:24:41.935632+0000","last_clean":"2023-10-16T11:24:41.935632+0000","last_became_active":"2023-10-16T11:24:40.930331+0000","last_became_peered":"2023-10-16T11:24:40.930331+0000","last_unstale":"2023-10-16T11:24:41.935632+0000","last_undegraded":"2023-10-16T11:24:41.935632+0000","last_fullsized":"2023-10-16T11:24:41.935632+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:18.688225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.908959+0000","last_change":"2023-10-16T11:24:26.077430+0000","last_active":"2023-10-16T11:24:40.908959+0000","last_peered":"2023-10-16T11:24:40.908959+0000","last_clean":"2023-10-16T11:24:40.908959+0000","last_became_active":"2023-10-16T11:24:26.077145+0000","last_became_peered":"2023-10-16T11:24:26.077145+0000","last_unstale":"2023-10-16T11:24:40.908959+0000","last_undegraded":"2023-10-16T11:24:40.908959+0000","last_fullsized":"2023-10-16T11:24:40.908959+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935229+0000","last_change":"2023-10-16T11:24:40.932225+0000","last_active":"2023-10-16T11:24:41.935229+0000","last_peered":"2023-10-16T11:24:41.935229+0000","last_clean":"2023-10-16T11:24:41.935229+0000","last_became_active":"2023-10-16T11:24:40.931846+0000","last_became_peered":"2023-10-16T11:24:40.931846+0000","last_unstale":"2023-10-16T11:24:41.935229+0000","last_undegraded":"2023-10-16T11:24:41.935229+0000","last_fullsized":"2023-10-16T11:24:41.935229+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:23.266881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927275+0000","last_change":"2023-10-16T11:24:26.088202+0000","last_active":"2023-10-16T11:24:40.927275+0000","last_peered":"2023-10-16T11:24:40.927275+0000","last_clean":"2023-10-16T11:24:40.927275+0000","last_became_active":"2023-10-16T11:24:26.087929+0000","last_became_peered":"2023-10-16T11:24:26.087929+0000","last_unstale":"2023-10-16T11:24:40.927275+0000","last_undegraded":"2023-10-16T11:24:40.927275+0000","last_fullsized":"2023-10-16T11:24:40.927275+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:22.009042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.070991+0000","last_change":"2023-10-16T11:24:40.941405+0000","last_active":"2023-10-16T11:24:42.070991+0000","last_peered":"2023-10-16T11:24:42.070991+0000","last_clean":"2023-10-16T11:24:42.070991+0000","last_became_active":"2023-10-16T11:24:40.941060+0000","last_became_peered":"2023-10-16T11:24:40.941060+0000","last_unstale":"2023-10-16T11:24:42.070991+0000","last_undegraded":"2023-10-16T11:24:42.070991+0000","last_fullsized":"2023-10-16T11:24:42.070991+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:22:53.973191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.933068+0000","last_change":"2023-10-16T11:24:26.080292+0000","last_active":"2023-10-16T11:24:40.933068+0000","last_peered":"2023-10-16T11:24:40.933068+0000","last_clean":"2023-10-16T11:24:40.933068+0000","last_became_active":"2023-10-16T11:24:26.080004+0000","last_became_peered":"2023-10-16T11:24:26.080004+0000","last_unstale":"2023-10-16T11:24:40.933068+0000","last_undegraded":"2023-10-16T11:24:40.933068+0000","last_fullsized":"2023-10-16T11:24:40.933068+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.936334+0000","last_change":"2023-10-16T11:24:40.946129+0000","last_active":"2023-10-16T11:24:41.936334+0000","last_peered":"2023-10-16T11:24:41.936334+0000","last_clean":"2023-10-16T11:24:41.936334+0000","last_became_active":"2023-10-16T11:24:40.944566+0000","last_became_peered":"2023-10-16T11:24:40.944566+0000","last_unstale":"2023-10-16T11:24:41.936334+0000","last_undegraded":"2023-10-16T11:24:41.936334+0000","last_fullsized":"2023-10-16T11:24:41.936334+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:38.047063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912578+0000","last_change":"2023-10-16T11:24:26.098845+0000","last_active":"2023-10-16T11:24:40.912578+0000","last_peered":"2023-10-16T11:24:40.912578+0000","last_clean":"2023-10-16T11:24:40.912578+0000","last_became_active":"2023-10-16T11:24:26.098568+0000","last_became_peered":"2023-10-16T11:24:26.098568+0000","last_unstale":"2023-10-16T11:24:40.912578+0000","last_undegraded":"2023-10-16T11:24:40.912578+0000","last_fullsized":"2023-10-16T11:24:40.912578+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:15.909098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3f","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.079804+0000","last_change":"2023-10-16T11:24:40.942060+0000","last_active":"2023-10-16T11:24:42.079804+0000","last_peered":"2023-10-16T11:24:42.079804+0000","last_clean":"2023-10-16T11:24:42.079804+0000","last_became_active":"2023-10-16T11:24:40.941339+0000","last_became_peered":"2023-10-16T11:24:40.941339+0000","last_unstale":"2023-10-16T11:24:42.079804+0000","last_undegraded":"2023-10-16T11:24:42.079804+0000","last_fullsized":"2023-10-16T11:24:42.079804+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:44:34.145225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917233+0000","last_change":"2023-10-16T11:24:26.096348+0000","last_active":"2023-10-16T11:24:40.917233+0000","last_peered":"2023-10-16T11:24:40.917233+0000","last_clean":"2023-10-16T11:24:40.917233+0000","last_became_active":"2023-10-16T11:24:26.093712+0000","last_became_peered":"2023-10-16T11:24:26.093712+0000","last_unstale":"2023-10-16T11:24:40.917233+0000","last_undegraded":"2023-10-16T11:24:40.917233+0000","last_fullsized":"2023-10-16T11:24:40.917233+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:40.970357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.073372+0000","last_change":"2023-10-16T11:24:40.942740+0000","last_active":"2023-10-16T11:24:42.073372+0000","last_peered":"2023-10-16T11:24:42.073372+0000","last_clean":"2023-10-16T11:24:42.073372+0000","last_became_active":"2023-10-16T11:24:40.939522+0000","last_became_peered":"2023-10-16T11:24:40.939522+0000","last_unstale":"2023-10-16T11:24:42.073372+0000","last_undegraded":"2023-10-16T11:24:42.073372+0000","last_fullsized":"2023-10-16T11:24:42.073372+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:31:23.849707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912900+0000","last_change":"2023-10-16T11:24:26.083687+0000","last_active":"2023-10-16T11:24:40.912900+0000","last_peered":"2023-10-16T11:24:40.912900+0000","last_clean":"2023-10-16T11:24:40.912900+0000","last_became_active":"2023-10-16T11:24:26.083213+0000","last_became_peered":"2023-10-16T11:24:26.083213+0000","last_unstale":"2023-10-16T11:24:40.912900+0000","last_undegraded":"2023-10-16T11:24:40.912900+0000","last_fullsized":"2023-10-16T11:24:40.912900+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:08:02.966132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.081257+0000","last_change":"2023-10-16T11:24:40.917853+0000","last_active":"2023-10-16T11:24:42.081257+0000","last_peered":"2023-10-16T11:24:42.081257+0000","last_clean":"2023-10-16T11:24:42.081257+0000","last_became_active":"2023-10-16T11:24:40.917561+0000","last_became_peered":"2023-10-16T11:24:40.917561+0000","last_unstale":"2023-10-16T11:24:42.081257+0000","last_undegraded":"2023-10-16T11:24:42.081257+0000","last_fullsized":"2023-10-16T11:24:42.081257+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:55.419058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.914944+0000","last_change":"2023-10-16T11:24:26.096814+0000","last_active":"2023-10-16T11:24:40.914944+0000","last_peered":"2023-10-16T11:24:40.914944+0000","last_clean":"2023-10-16T11:24:40.914944+0000","last_became_active":"2023-10-16T11:24:26.095777+0000","last_became_peered":"2023-10-16T11:24:26.095777+0000","last_unstale":"2023-10-16T11:24:40.914944+0000","last_undegraded":"2023-10-16T11:24:40.914944+0000","last_fullsized":"2023-10-16T11:24:40.914944+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:08.889514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.933544+0000","last_change":"2023-10-16T11:24:40.935780+0000","last_active":"2023-10-16T11:24:41.933544+0000","last_peered":"2023-10-16T11:24:41.933544+0000","last_clean":"2023-10-16T11:24:41.933544+0000","last_became_active":"2023-10-16T11:24:40.935476+0000","last_became_peered":"2023-10-16T11:24:40.935476+0000","last_unstale":"2023-10-16T11:24:41.933544+0000","last_undegraded":"2023-10-16T11:24:41.933544+0000","last_fullsized":"2023-10-16T11:24:41.933544+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:54:11.551791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.922006+0000","last_change":"2023-10-16T11:24:26.091440+0000","last_active":"2023-10-16T11:24:40.922006+0000","last_peered":"2023-10-16T11:24:40.922006+0000","last_clean":"2023-10-16T11:24:40.922006+0000","last_became_active":"2023-10-16T11:24:26.088757+0000","last_became_peered":"2023-10-16T11:24:26.088757+0000","last_unstale":"2023-10-16T11:24:40.922006+0000","last_undegraded":"2023-10-16T11:24:40.922006+0000","last_fullsized":"2023-10-16T11:24:40.922006+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:27.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072970+0000","last_change":"2023-10-16T11:24:40.933226+0000","last_active":"2023-10-16T11:24:42.072970+0000","last_peered":"2023-10-16T11:24:42.072970+0000","last_clean":"2023-10-16T11:24:42.072970+0000","last_became_active":"2023-10-16T11:24:40.932168+0000","last_became_peered":"2023-10-16T11:24:40.932168+0000","last_unstale":"2023-10-16T11:24:42.072970+0000","last_undegraded":"2023-10-16T11:24:42.072970+0000","last_fullsized":"2023-10-16T11:24:42.072970+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:51:44.814417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.908841+0000","last_change":"2023-10-16T11:24:26.085606+0000","last_active":"2023-10-16T11:24:40.908841+0000","last_peered":"2023-10-16T11:24:40.908841+0000","last_clean":"2023-10-16T11:24:40.908841+0000","last_became_active":"2023-10-16T11:24:26.085318+0000","last_became_peered":"2023-10-16T11:24:26.085318+0000","last_unstale":"2023-10-16T11:24:40.908841+0000","last_undegraded":"2023-10-16T11:24:40.908841+0000","last_fullsized":"2023-10-16T11:24:40.908841+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.931913+0000","last_change":"2023-10-16T11:24:40.937629+0000","last_active":"2023-10-16T11:24:41.931913+0000","last_peered":"2023-10-16T11:24:41.931913+0000","last_clean":"2023-10-16T11:24:41.931913+0000","last_became_active":"2023-10-16T11:24:40.937340+0000","last_became_peered":"2023-10-16T11:24:40.937340+0000","last_unstale":"2023-10-16T11:24:41.931913+0000","last_undegraded":"2023-10-16T11:24:41.931913+0000","last_fullsized":"2023-10-16T11:24:41.931913+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:13.135380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.932956+0000","last_change":"2023-10-16T11:24:26.089885+0000","last_active":"2023-10-16T11:24:40.932956+0000","last_peered":"2023-10-16T11:24:40.932956+0000","last_clean":"2023-10-16T11:24:40.932956+0000","last_became_active":"2023-10-16T11:24:26.089406+0000","last_became_peered":"2023-10-16T11:24:26.089406+0000","last_unstale":"2023-10-16T11:24:40.932956+0000","last_undegraded":"2023-10-16T11:24:40.932956+0000","last_fullsized":"2023-10-16T11:24:40.932956+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.075155+0000","last_change":"2023-10-16T11:24:40.926983+0000","last_active":"2023-10-16T11:24:42.075155+0000","last_peered":"2023-10-16T11:24:42.075155+0000","last_clean":"2023-10-16T11:24:42.075155+0000","last_became_active":"2023-10-16T11:24:40.923594+0000","last_became_peered":"2023-10-16T11:24:40.923594+0000","last_unstale":"2023-10-16T11:24:42.075155+0000","last_undegraded":"2023-10-16T11:24:42.075155+0000","last_fullsized":"2023-10-16T11:24:42.075155+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:54.635909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.919488+0000","last_change":"2023-10-16T11:24:26.086488+0000","last_active":"2023-10-16T11:24:40.919488+0000","last_peered":"2023-10-16T11:24:40.919488+0000","last_clean":"2023-10-16T11:24:40.919488+0000","last_became_active":"2023-10-16T11:24:26.085969+0000","last_became_peered":"2023-10-16T11:24:26.085969+0000","last_unstale":"2023-10-16T11:24:40.919488+0000","last_undegraded":"2023-10-16T11:24:40.919488+0000","last_fullsized":"2023-10-16T11:24:40.919488+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":581,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2767,"ondisk_log_size":2767,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575989,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52532,"kb_used_data":3304,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319308,"statfs":{"total":96636764160,"available":96582971392,"internally_reserved":0,"allocated":3383296,"data_stored":1942477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.179,"15min":1.2170000000000001},"min":{"1min":0.81499999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.7250000000000001,"5min":2.085,"15min":2.3399999999999999},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.1719999999999999,"15min":1.218},"min":{"1min":0.876,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.8320000000000001},"last":1.448}]},{"osd":1,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2689999999999999,"15min":1.2509999999999999},"min":{"1min":0.90700000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.6479999999999999,"5min":1.913,"15min":1.913},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2430000000000001,"15min":1.2749999999999999},"min":{"1min":0.76700000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6279999999999999,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.95699999999999996}]},{"osd":2,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3140000000000001,"15min":1.3109999999999999},"min":{"1min":1.008,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6579999999999999,"5min":1.8680000000000001,"15min":2.4460000000000002},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.3540000000000001,"15min":1.3640000000000001},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.708,"5min":1.859,"15min":1.974},"last":1.083}]},{"osd":3,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.381,"15min":1.3899999999999999},"min":{"1min":1.149,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7290000000000001,"5min":1.8280000000000001,"15min":1.972},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.4430000000000001,"15min":1.4630000000000001},"min":{"1min":1.0860000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.8799999999999999,"5min":2.0680000000000001,"15min":2.577},"last":1.248}]},{"osd":4,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.423,"15min":1.417},"min":{"1min":0.98599999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0030000000000001,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.498,"5min":1.391,"15min":1.3899999999999999},"min":{"1min":1.0469999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8280000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.149}]},{"osd":5,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.585,"15min":1.5960000000000001},"min":{"1min":1.1579999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.946,"5min":2.2309999999999999,"15min":3.3519999999999999},"last":1.7190000000000001},{"interface":"front","average":{"1min":1.641,"5min":1.514,"15min":1.5089999999999999},"min":{"1min":1.351,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.1600000000000001,"5min":2.3460000000000001,"15min":2.4969999999999999},"last":1.3109999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.488,"15min":1.5209999999999999},"min":{"1min":0.97099999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.845,"5min":2.101,"15min":2.7250000000000001},"last":1.351},{"interface":"front","average":{"1min":1.7210000000000001,"5min":1.6890000000000001,"15min":1.71},"min":{"1min":1.425,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1040000000000001,"5min":2.298,"15min":3.1440000000000001},"last":1.661}]}]},{"osd":6,"up_from":13,"seq":55834575989,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33904,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337936,"statfs":{"total":96636764160,"available":96602046464,"internally_reserved":0,"allocated":2785280,"data_stored":1349820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.2350000000000001,"15min":1.2250000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.64000000000000001},"max":{"1min":6.298,"5min":6.298,"15min":6.298},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.3380000000000001,"15min":1.2909999999999999},"min":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.66400000000000003},"max":{"1min":8.2650000000000006,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":1.5409999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2450000000000001,"15min":1.222},"min":{"1min":0.80200000000000005,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.88},{"interface":"front","average":{"1min":1.8520000000000001,"5min":1.4510000000000001,"15min":1.3859999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":12.468,"5min":12.468,"15min":12.468},"last":1.411}]},{"osd":2,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.7889999999999999,"5min":1.337,"15min":1.2829999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":11.586,"5min":11.586,"15min":11.586},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.569,"5min":1.3799999999999999,"15min":1.369},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.77500000000000002},"max":{"1min":4.7030000000000003,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.1879999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.9239999999999999,"5min":1.452,"15min":1.353},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":12.801,"5min":12.801,"15min":12.801},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.302,"15min":1.284},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":9.1539999999999999,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":1.216}]},{"osd":4,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.3700000000000001,"15min":1.3220000000000001},"min":{"1min":0.65400000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.369},{"interface":"front","average":{"1min":2.141,"5min":1.609,"15min":1.548},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":13.981999999999999,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.1180000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":2.246,"5min":1.5109999999999999,"15min":1.391},"min":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":18.247,"5min":18.247,"15min":18.247},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.8360000000000001,"5min":1.5900000000000001,"15min":1.5629999999999999},"min":{"1min":0.76200000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":6.4039999999999999,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.4930000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":2.0609999999999999,"5min":1.603,"15min":1.5349999999999999},"min":{"1min":0.749,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":12.589,"5min":12.589,"15min":12.589},"last":1.27},{"interface":"front","average":{"1min":2.1749999999999998,"5min":1.6739999999999999,"15min":1.595},"min":{"1min":0.78200000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":12.725,"5min":12.725,"15min":12.725},"last":1.456}]}]},{"osd":5,"up_from":13,"seq":55834575989,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29736,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342104,"statfs":{"total":96636764160,"available":96606314496,"internally_reserved":0,"allocated":2777088,"data_stored":1349764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2350000000000001,"15min":1.27},"min":{"1min":0.74199999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.583,"5min":2.661,"15min":2.661},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.3029999999999999,"15min":1.349},"min":{"1min":0.95799999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7909999999999999,"5min":1.7949999999999999,"15min":1.865},"last":1.25}]},{"osd":1,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1819999999999999,"15min":1.1950000000000001},"min":{"1min":0.81999999999999995,"5min":0.80800000000000005,"15min":0.79100000000000004},"max":{"1min":1.5009999999999999,"5min":1.7869999999999999,"15min":1.992},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.258,"15min":1.2290000000000001},"min":{"1min":0.84399999999999997,"5min":0.84199999999999997,"15min":0.80800000000000005},"max":{"1min":1.635,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.6739999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.264,"15min":1.244},"min":{"1min":0.97899999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.625,"5min":1.952,"15min":1.952},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.329,"5min":1.2949999999999999,"15min":1.3220000000000001},"min":{"1min":0.92400000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":2.1640000000000001},"last":1.0680000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3320000000000001,"15min":1.3340000000000001},"min":{"1min":0.98999999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":3.012,"15min":3.012},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.429,"15min":1.425},"min":{"1min":1.0900000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":2.2250000000000001},"last":1.2010000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.339,"15min":1.339},"min":{"1min":0.94199999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.6990000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.6799999999999999,"5min":1.5720000000000001,"15min":1.5740000000000001},"min":{"1min":1.353,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.948,"5min":2.903,"15min":2.903},"last":1.3879999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4330000000000001,"15min":1.4139999999999999},"min":{"1min":0.873,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8660000000000001,"5min":2.0649999999999999,"15min":2.0979999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.51,"5min":1.494,"15min":1.4770000000000001},"min":{"1min":0.95399999999999996,"5min":0.90700000000000003,"15min":0.78400000000000003},"max":{"1min":1.841,"5min":3.073,"15min":3.073},"last":1.607}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.675,"5min":1.5660000000000001,"15min":1.5469999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0779999999999998,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.579},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.645,"15min":1.645},"min":{"1min":1.1779999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.1099999999999999,"5min":2.1459999999999999,"15min":2.2930000000000001},"last":1.4370000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575990,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51960,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319880,"statfs":{"total":96636764160,"available":96583557120,"internally_reserved":0,"allocated":2797568,"data_stored":1352725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1519999999999999,"15min":1.1859999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.7290000000000001,"5min":1.792,"15min":1.792},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.218,"15min":1.2350000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.9299999999999999},"last":1.1599999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2769999999999999,"15min":1.298},"min":{"1min":0.81799999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.841,"5min":1.841,"15min":1.913},"last":1.409},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.236,"15min":1.25},"min":{"1min":0.78500000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.877,"5min":1.877,"15min":1.889},"last":1.1020000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.29,"15min":1.3089999999999999},"min":{"1min":1.0369999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.835},"last":1.302},{"interface":"front","average":{"1min":1.351,"5min":1.3999999999999999,"15min":1.403},"min":{"1min":0.879,"5min":0.879,"15min":0.872},"max":{"1min":1.8380000000000001,"5min":1.903,"15min":1.9139999999999999},"last":1.6339999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3169999999999999,"15min":1.337},"min":{"1min":0.91100000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.6830000000000001,"5min":1.8280000000000001,"15min":1.849},"last":1.595},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4410000000000001,"15min":1.4330000000000001},"min":{"1min":1.145,"5min":1.008,"15min":0.96999999999999997},"max":{"1min":1.7769999999999999,"5min":1.865,"15min":2.0720000000000001},"last":1.6739999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.4810000000000001,"15min":1.4399999999999999},"min":{"1min":1.0149999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9399999999999999,"5min":2.105,"15min":2.105},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.431,"15min":1.415},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.78600000000000003},"max":{"1min":1.8240000000000001,"5min":2.04,"15min":2.04},"last":1.4590000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.5409999999999999,"15min":1.5029999999999999},"min":{"1min":0.94299999999999995,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":3.157,"5min":3.157,"15min":3.157},"last":1.528},{"interface":"front","average":{"1min":1.5,"5min":1.6100000000000001,"15min":1.611},"min":{"1min":1.0940000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0089999999999999,"5min":2.2330000000000001,"15min":2.2770000000000001},"last":1.498}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.5640000000000001,"15min":1.5660000000000001},"min":{"1min":1.0549999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.855,"5min":2.113,"15min":2.3420000000000001},"last":1.375},{"interface":"front","average":{"1min":1.639,"5min":1.6619999999999999,"15min":1.6399999999999999},"min":{"1min":1.0660000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9630000000000001,"5min":2.1469999999999998,"15min":2.3839999999999999},"last":1.756}]}]},{"osd":3,"up_from":13,"seq":55834575989,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47812,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324028,"statfs":{"total":96636764160,"available":96587804672,"internally_reserved":0,"allocated":2801664,"data_stored":1351046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.89400000000000002,"15min":0.90700000000000003},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.478,"5min":1.478,"15min":1.901},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0369999999999999,"15min":1.0720000000000001},"min":{"1min":0.59299999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.391,"5min":1.6180000000000001,"15min":1.6679999999999999},"last":1.0660000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.95599999999999996,"15min":0.98799999999999999},"min":{"1min":0.60799999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.105,"15min":1.133},"min":{"1min":0.48399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.601,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.88100000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0760000000000001,"15min":1.117},"min":{"1min":0.65500000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.524,"5min":1.694,"15min":1.694},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.198,"15min":1.236},"min":{"1min":0.625,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.91400000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3169999999999999,"15min":1.3500000000000001},"min":{"1min":0.83599999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.806,"5min":1.806,"15min":1.873},"last":1.466},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.357,"15min":1.367},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.8819999999999999},"last":1.2689999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.385,"15min":1.401},"min":{"1min":0.80600000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.516},{"interface":"front","average":{"1min":1.419,"5min":1.4059999999999999,"15min":1.444},"min":{"1min":1.151,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.552}]},{"osd":6,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.504,"15min":1.5329999999999999},"min":{"1min":0.876,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":3.9129999999999998,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.657},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5,"15min":1.5369999999999999},"min":{"1min":1.153,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":3.9319999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.4910000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.528,"15min":1.5600000000000001},"min":{"1min":0.95399999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.877,"5min":2.0790000000000002,"15min":2.2679999999999998},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.639,"15min":1.673},"min":{"1min":1.1279999999999999,"5min":1,"15min":1},"max":{"1min":2.052,"5min":2.198,"15min":2.5129999999999999},"last":1.6739999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575988,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51964,"kb_used_data":2728,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319876,"statfs":{"total":96636764160,"available":96583553024,"internally_reserved":0,"allocated":2793472,"data_stored":1350850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.91000000000000003,"15min":0.92200000000000004},"min":{"1min":0.74199999999999999,"5min":0.51900000000000002,"15min":0.5},"max":{"1min":1.532,"5min":1.532,"15min":1.669},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.0569999999999999,"15min":1.0820000000000001},"min":{"1min":0.72499999999999998,"5min":0.47699999999999998,"15min":0.32600000000000001},"max":{"1min":1.496,"5min":1.681,"15min":2.1429999999999998},"last":0.69799999999999995}]},{"osd":1,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.089,"15min":1.097},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.39200000000000002},"max":{"1min":1.415,"5min":1.6779999999999999,"15min":1.978},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1479999999999999,"15min":1.165},"min":{"1min":0.57599999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.448,"5min":1.609,"15min":1.869},"last":0.99299999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.0649999999999999,"15min":1.087},"min":{"1min":0.54600000000000004,"5min":0.437,"15min":0.437},"max":{"1min":1.464,"5min":1.651,"15min":1.8560000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.252,"15min":1.274},"min":{"1min":0.82199999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6120000000000001,"5min":1.7450000000000001,"15min":1.857},"last":1.1930000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3740000000000001,"15min":1.415},"min":{"1min":0.91500000000000004,"5min":0.86599999999999999,"15min":0.73499999999999999},"max":{"1min":1.6910000000000001,"5min":1.8029999999999999,"15min":2.5059999999999998},"last":1.145},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3440000000000001,"15min":1.3520000000000001},"min":{"1min":0.97899999999999998,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":1.825,"5min":1.825,"15min":2.0880000000000001},"last":0.97599999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.444,"15min":1.464},"min":{"1min":1.0720000000000001,"5min":0.84999999999999998,"15min":0.78500000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2269999999999999},"last":1.345},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4550000000000001,"15min":1.4890000000000001},"min":{"1min":1.0509999999999999,"5min":0.95799999999999996,"15min":0.874},"max":{"1min":1.798,"5min":1.931,"15min":2.6259999999999999},"last":1.258}]},{"osd":6,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4119999999999999,"15min":1.411},"min":{"1min":1.0569999999999999,"5min":0.84099999999999997,"15min":0.80500000000000005},"max":{"1min":1.71,"5min":1.9019999999999999,"15min":2.2109999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5,"15min":1.5369999999999999},"min":{"1min":1.1899999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.2909999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.417}]},{"osd":7,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":1.5489999999999999,"15min":1.5860000000000001},"min":{"1min":1.2290000000000001,"5min":0.995,"15min":0.92500000000000004},"max":{"1min":2.1469999999999998,"5min":2.262,"15min":2.8109999999999999},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.494,"5min":1.4870000000000001,"15min":1.5149999999999999},"min":{"1min":1.242,"5min":0.81599999999999995,"15min":0.80100000000000005},"max":{"1min":1.8180000000000001,"5min":2.2269999999999999,"15min":2.2829999999999999},"last":1.323}]}]},{"osd":1,"up_from":13,"seq":55834575988,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60272,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311568,"statfs":{"total":96636764160,"available":96575045632,"internally_reserved":0,"allocated":2785280,"data_stored":1349328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.91900000000000004,"15min":0.91500000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.575,"5min":1.575,"15min":1.7929999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0289999999999999,"15min":1.0580000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.494,"5min":1.5509999999999999,"15min":1.667},"last":0.60899999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.98399999999999999,"15min":1.0189999999999999},"min":{"1min":0.64300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.72},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.252,"15min":1.276},"min":{"1min":0.69799999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":1.4379999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.1879999999999999,"15min":1.216},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.774,"5min":1.774,"15min":1.79},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.169,"15min":1.1739999999999999},"min":{"1min":0.66700000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.633,"5min":1.633,"15min":1.802},"last":0.67500000000000004}]},{"osd":4,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.2609999999999999,"15min":1.274},"min":{"1min":0.86299999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.653,"5min":1.887,"15min":1.9159999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.409,"15min":1.415},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.887,"5min":2.0619999999999998,"15min":2.093},"last":1.23}]},{"osd":5,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.379,"15min":1.409},"min":{"1min":0.93700000000000006,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.948,"5min":2.0009999999999999,"15min":2.0640000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4239999999999999,"15min":1.421},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":1.825,"5min":1.825,"15min":2.0649999999999999},"last":1.141}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4850000000000001,"15min":1.496},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.879,"5min":2.1970000000000001,"15min":2.2949999999999999},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4710000000000001,"15min":1.486},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.891,"5min":1.891,"15min":2.1970000000000001},"last":1.0349999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5649999999999999,"15min":1.5620000000000001},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.1720000000000002},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.625,"15min":1.607},"min":{"1min":1.1080000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.9359999999999999,"5min":2.1989999999999998,"15min":2.2810000000000001},"last":1.367}]}]},{"osd":0,"up_from":13,"seq":55834575989,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52532,"kb_used_data":3300,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319308,"statfs":{"total":96636764160,"available":96582971392,"internally_reserved":0,"allocated":3379200,"data_stored":1940312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.871,"15min":0.90000000000000002},"min":{"1min":0.73199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.502,"5min":1.52,"15min":1.619},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.1060000000000001},"min":{"1min":0.627,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.621,"5min":1.6399999999999999,"15min":1.76},"last":1.109}]},{"osd":2,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.097,"15min":1.0860000000000001},"min":{"1min":0.65300000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.556,"5min":1.649,"15min":1.649},"last":1.21},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.095,"15min":1.1459999999999999},"min":{"1min":0.81100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4179999999999999,"5min":1.623,"15min":1.6870000000000001},"last":1.508}]},{"osd":3,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.056,"15min":1.085},"min":{"1min":0.69499999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.244,"15min":1.256},"min":{"1min":0.91500000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.716,"5min":1.7609999999999999,"15min":1.9830000000000001},"last":1.456}]},{"osd":4,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.387,"15min":1.389},"min":{"1min":1.129,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.681,"5min":1.9390000000000001,"15min":1.95},"last":1.611},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3180000000000001,"15min":1.3500000000000001},"min":{"1min":1.0329999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.645,"5min":2.1539999999999999,"15min":2.4820000000000002},"last":1.252}]},{"osd":5,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3660000000000001,"15min":1.3879999999999999},"min":{"1min":0.996,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":1.829,"5min":2.0750000000000002,"15min":2.6669999999999998},"last":1.401},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.454},"min":{"1min":1.028,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.748,"5min":2.2029999999999998,"15min":2.2040000000000002},"last":1.639}]},{"osd":6,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.4750000000000001,"15min":1.5029999999999999},"min":{"1min":1.1080000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.819,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.405,"5min":1.4630000000000001,"15min":1.492},"min":{"1min":0.93799999999999994,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.758,"5min":2.262,"15min":2.262},"last":1.7170000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.52,"15min":1.5549999999999999},"min":{"1min":1.2749999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7150000000000001,"5min":2.347,"15min":2.347},"last":1.794},{"interface":"front","average":{"1min":1.454,"5min":1.474,"15min":1.5109999999999999},"min":{"1min":1.129,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.9530000000000001,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.49}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:24:50.544 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:24:50.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.765+0000 7fc9ff533640 1 Processor -- start 2023-10-16T11:24:50.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.765+0000 7fc9ff533640 1 -- start start 2023-10-16T11:24:50.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.767+0000 7fc9ff533640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc9f80cd0c0 0x7fc9f81615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:50.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.767+0000 7fc9ff533640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc9f815db00 0x7fc9f80b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:50.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.767+0000 7fc9ff533640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f815e3e0 con 0x7fc9f80b0410 2023-10-16T11:24:50.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.767+0000 7fc9ff533640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f80b0de0 con 0x7fc9f80cd0c0 2023-10-16T11:24:50.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.767+0000 7fc9ff533640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f80cd4a0 con 0x7fc9f815db00 2023-10-16T11:24:50.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.767+0000 7fc9fd2cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc9f80cd0c0 0x7fc9f81615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:50.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.767+0000 7fc9fd2cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc9f80cd0c0 0x7fc9f81615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51320/0 (socket says 172.21.15.84:51320) 2023-10-16T11:24:50.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.767+0000 7fc9fd2cf640 1 -- 172.21.15.84:0/1941439147 learned_addr learned my addr 172.21.15.84:0/1941439147 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:50.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.767+0000 7fc9fcace640 1 --1- 172.21.15.84:0/1941439147 >> v1:172.21.15.84:6789/0 conn(0x7fc9f80b0410 0x7fc9f815d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47960/0 (socket says 172.21.15.84:47960) 2023-10-16T11:24:50.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.767+0000 7fc9fd2cf640 1 -- 172.21.15.84:0/1941439147 >> v2:172.21.15.157:3300/0 conn(0x7fc9f815db00 msgr2=0x7fc9f80b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:50.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.767+0000 7fc9fd2cf640 1 --2- 172.21.15.84:0/1941439147 >> v2:172.21.15.157:3300/0 conn(0x7fc9f815db00 0x7fc9f80b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:50.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.768+0000 7fc9fd2cf640 1 -- 172.21.15.84:0/1941439147 >> v1:172.21.15.84:6789/0 conn(0x7fc9f80b0410 legacy=0x7fc9f815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:50.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.768+0000 7fc9fd2cf640 1 -- 172.21.15.84:0/1941439147 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f815e100 con 0x7fc9f80cd0c0 2023-10-16T11:24:50.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.769+0000 7fc9fd2cf640 1 --2- 172.21.15.84:0/1941439147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f80cd0c0 0x7fc9f81615f0 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7fc9f815afc0 tx=0x7fc9f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a576546de16b2710 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:50.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.769+0000 7fc9effff640 1 -- 172.21.15.84:0/1941439147 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4031870 con 0x7fc9f80cd0c0 2023-10-16T11:24:50.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.769+0000 7fc9effff640 1 -- 172.21.15.84:0/1941439147 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f40319d0 con 0x7fc9f80cd0c0 2023-10-16T11:24:50.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.769+0000 7fc9effff640 1 -- 172.21.15.84:0/1941439147 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4031d60 con 0x7fc9f80cd0c0 2023-10-16T11:24:50.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.769+0000 7fc9ff533640 1 -- 172.21.15.84:0/1941439147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f80cd0c0 msgr2=0x7fc9f81615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:50.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.769+0000 7fc9ff533640 1 --2- 172.21.15.84:0/1941439147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f80cd0c0 0x7fc9f81615f0 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7fc9f815afc0 tx=0x7fc9f402b910 comp rx=0 tx=0).stop 2023-10-16T11:24:50.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.770+0000 7fc9ff533640 1 -- 172.21.15.84:0/1941439147 shutdown_connections 2023-10-16T11:24:50.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.770+0000 7fc9ff533640 1 --2- 172.21.15.84:0/1941439147 >> v2:172.21.15.157:3300/0 conn(0x7fc9f815db00 0x7fc9f80b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:50.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.770+0000 7fc9ff533640 1 --2- 172.21.15.84:0/1941439147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f80cd0c0 0x7fc9f81615f0 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:50.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.770+0000 7fc9ff533640 1 -- 172.21.15.84:0/1941439147 >> 172.21.15.84:0/1941439147 conn(0x7fc9f80cb410 msgr2=0x7fc9f80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:50.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.770+0000 7fc9ff533640 1 -- 172.21.15.84:0/1941439147 shutdown_connections 2023-10-16T11:24:50.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.770+0000 7fc9ff533640 1 -- 172.21.15.84:0/1941439147 wait complete. 2023-10-16T11:24:50.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.771+0000 7fc9ff533640 1 Processor -- start 2023-10-16T11:24:50.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.771+0000 7fc9ff533640 1 -- start start 2023-10-16T11:24:50.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.772+0000 7fc9ff533640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9f80b0410 0x7fc9f80b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:50.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.772+0000 7fc9ff533640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9f80cd0c0 0x7fc9f815f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:50.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.772+0000 7fc9ff533640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f815db00 0x7fc9f80c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:50.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.772+0000 7fc9ff533640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80b27f0 con 0x7fc9f815db00 2023-10-16T11:24:50.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.772+0000 7fc9ff533640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f815f5e0 con 0x7fc9f80b0410 2023-10-16T11:24:50.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.772+0000 7fc9ff533640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80b1be0 con 0x7fc9f80cd0c0 2023-10-16T11:24:50.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.773+0000 7fc9fdad0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f815db00 0x7fc9f80c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:50.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.773+0000 7fc9fdad0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f815db00 0x7fc9f80c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51332/0 (socket says 172.21.15.84:51332) 2023-10-16T11:24:50.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.773+0000 7fc9fdad0640 1 -- 172.21.15.84:0/2904958227 learned_addr learned my addr 172.21.15.84:0/2904958227 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:50.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.773+0000 7fc9fd2cf640 1 --2- 172.21.15.84:0/2904958227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9f80b0410 0x7fc9f80b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:50.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.773+0000 7fc9fcace640 1 --2- 172.21.15.84:0/2904958227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9f80cd0c0 0x7fc9f815f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:50.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.773+0000 7fc9fdad0640 1 -- 172.21.15.84:0/2904958227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9f80cd0c0 msgr2=0x7fc9f815f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:50.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.773+0000 7fc9fdad0640 1 --2- 172.21.15.84:0/2904958227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9f80cd0c0 0x7fc9f815f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:50.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.773+0000 7fc9fdad0640 1 -- 172.21.15.84:0/2904958227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9f80b0410 msgr2=0x7fc9f80b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:50.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.773+0000 7fc9fdad0640 1 --2- 172.21.15.84:0/2904958227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9f80b0410 0x7fc9f80b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:50.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.773+0000 7fc9fdad0640 1 -- 172.21.15.84:0/2904958227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f40094d0 con 0x7fc9f815db00 2023-10-16T11:24:50.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.773+0000 7fc9fdad0640 1 --2- 172.21.15.84:0/2904958227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f815db00 0x7fc9f80c4ab0 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000b970 tx=0x7fc9f000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:50.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.774+0000 7fc9edffb640 1 -- 172.21.15.84:0/2904958227 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f000cb40 con 0x7fc9f815db00 2023-10-16T11:24:50.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.774+0000 7fc9edffb640 1 -- 172.21.15.84:0/2904958227 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f000ccc0 con 0x7fc9f815db00 2023-10-16T11:24:50.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.774+0000 7fc9edffb640 1 -- 172.21.15.84:0/2904958227 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0014680 con 0x7fc9f815db00 2023-10-16T11:24:50.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.774+0000 7fc9ff533640 1 -- 172.21.15.84:0/2904958227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f80b1ec0 con 0x7fc9f815db00 2023-10-16T11:24:50.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.774+0000 7fc9ff533640 1 -- 172.21.15.84:0/2904958227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f80b2480 con 0x7fc9f815db00 2023-10-16T11:24:50.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.775+0000 7fc9ff533640 1 -- 172.21.15.84:0/2904958227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f80b29a0 con 0x7fc9f815db00 2023-10-16T11:24:50.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.776+0000 7fc9edffb640 1 -- 172.21.15.84:0/2904958227 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc9f001d3b0 con 0x7fc9f815db00 2023-10-16T11:24:50.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.777+0000 7fc9edffb640 1 --2- 172.21.15.84:0/2904958227 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fc9dc06afc0 0x7fc9dc06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:50.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.777+0000 7fc9edffb640 1 -- 172.21.15.84:0/2904958227 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7fc9f0089a30 con 0x7fc9f815db00 2023-10-16T11:24:50.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.777+0000 7fc9fd2cf640 1 --2- 172.21.15.84:0/2904958227 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fc9dc06afc0 0x7fc9dc06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:50.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.778+0000 7fc9fd2cf640 1 --2- 172.21.15.84:0/2904958227 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fc9dc06afc0 0x7fc9dc06d480 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4036000 tx=0x7fc9f403c040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:50.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.781+0000 7fc9edffb640 1 -- 172.21.15.84:0/2904958227 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc9f80b29a0 con 0x7fc9f815db00 2023-10-16T11:24:50.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.970+0000 7fc9ff533640 1 -- 172.21.15.84:0/2904958227 --> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc9f8160df0 con 0x7fc9dc06afc0 2023-10-16T11:24:50.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:50.992+0000 7fc9edffb640 1 -- 172.21.15.84:0/2904958227 <== mgr.11076 v2:172.21.15.84:6832/1425409017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+525734 (secure 0 0 0) 0x7fc9f4031810 con 0x7fc9dc06afc0 2023-10-16T11:24:51.006 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:51.011 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:24:51.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.000+0000 7fc9ff533640 1 -- 172.21.15.84:0/2904958227 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fc9dc06afc0 msgr2=0x7fc9dc06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:51.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.000+0000 7fc9ff533640 1 --2- 172.21.15.84:0/2904958227 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fc9dc06afc0 0x7fc9dc06d480 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4036000 tx=0x7fc9f403c040 comp rx=0 tx=0).stop 2023-10-16T11:24:51.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.000+0000 7fc9ff533640 1 -- 172.21.15.84:0/2904958227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f815db00 msgr2=0x7fc9f80c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:51.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.000+0000 7fc9ff533640 1 --2- 172.21.15.84:0/2904958227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f815db00 0x7fc9f80c4ab0 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000b970 tx=0x7fc9f000be30 comp rx=0 tx=0).stop 2023-10-16T11:24:51.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.000+0000 7fc9ff533640 1 -- 172.21.15.84:0/2904958227 shutdown_connections 2023-10-16T11:24:51.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.000+0000 7fc9ff533640 1 --2- 172.21.15.84:0/2904958227 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fc9dc06afc0 0x7fc9dc06d480 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:51.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.000+0000 7fc9ff533640 1 --2- 172.21.15.84:0/2904958227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f815db00 0x7fc9f80c4ab0 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:51.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.000+0000 7fc9ff533640 1 --2- 172.21.15.84:0/2904958227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9f80cd0c0 0x7fc9f815f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:51.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.000+0000 7fc9ff533640 1 --2- 172.21.15.84:0/2904958227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9f80b0410 0x7fc9f80b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:51.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.000+0000 7fc9ff533640 1 -- 172.21.15.84:0/2904958227 >> 172.21.15.84:0/2904958227 conn(0x7fc9f80cb410 msgr2=0x7fc9f815ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:51.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.000+0000 7fc9ff533640 1 -- 172.21.15.84:0/2904958227 shutdown_connections 2023-10-16T11:24:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.000+0000 7fc9ff533640 1 -- 172.21.15.84:0/2904958227 wait complete. 2023-10-16T11:24:51.023 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2023-10-16T11:24:50.259228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":673,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2847,"ondisk_log_size":2847,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":380712,"kb_used_data":22952,"kb_used_omap":12,"kb_used_meta":357619,"kb_avail":754594008,"statfs":{"total":773094113280,"available":772704264192,"internally_reserved":0,"allocated":23502848,"data_stored":11986322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":366202425},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.354473"},"pg_stats":[{"pgid":"29.1c","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076487+0000","last_change":"2023-10-16T11:24:40.949704+0000","last_active":"2023-10-16T11:24:42.076487+0000","last_peered":"2023-10-16T11:24:42.076487+0000","last_clean":"2023-10-16T11:24:42.076487+0000","last_became_active":"2023-10-16T11:24:40.944766+0000","last_became_peered":"2023-10-16T11:24:40.944766+0000","last_unstale":"2023-10-16T11:24:42.076487+0000","last_undegraded":"2023-10-16T11:24:42.076487+0000","last_fullsized":"2023-10-16T11:24:42.076487+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:39:05.461904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"206'2767","reported_seq":3652,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.928589+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:24:40.928589+0000","last_peered":"2023-10-16T11:24:40.928589+0000","last_clean":"2023-10-16T11:24:40.928589+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:24:40.928589+0000","last_undegraded":"2023-10-16T11:24:40.928589+0000","last_fullsized":"2023-10-16T11:24:40.928589+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2767,"log_dups_size":0,"ondisk_log_size":2767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":581,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"216'1","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.916192+0000","last_change":"2023-10-16T11:24:25.087491+0000","last_active":"2023-10-16T11:24:40.916192+0000","last_peered":"2023-10-16T11:24:40.916192+0000","last_clean":"2023-10-16T11:24:40.916192+0000","last_became_active":"2023-10-16T11:24:25.086826+0000","last_became_peered":"2023-10-16T11:24:25.086826+0000","last_unstale":"2023-10-16T11:24:40.916192+0000","last_undegraded":"2023-10-16T11:24:40.916192+0000","last_fullsized":"2023-10-16T11:24:40.916192+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:07.303719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918232+0000","last_change":"2023-10-16T11:24:26.091340+0000","last_active":"2023-10-16T11:24:40.918232+0000","last_peered":"2023-10-16T11:24:40.918232+0000","last_clean":"2023-10-16T11:24:40.918232+0000","last_became_active":"2023-10-16T11:24:26.091059+0000","last_became_peered":"2023-10-16T11:24:26.091059+0000","last_unstale":"2023-10-16T11:24:40.918232+0000","last_undegraded":"2023-10-16T11:24:40.918232+0000","last_fullsized":"2023-10-16T11:24:40.918232+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:37.605400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.948878+0000","last_change":"2023-10-16T11:24:39.927746+0000","last_active":"2023-10-16T11:24:40.948878+0000","last_peered":"2023-10-16T11:24:40.948878+0000","last_clean":"2023-10-16T11:24:40.948878+0000","last_became_active":"2023-10-16T11:24:39.927422+0000","last_became_peered":"2023-10-16T11:24:39.927422+0000","last_unstale":"2023-10-16T11:24:40.948878+0000","last_undegraded":"2023-10-16T11:24:40.948878+0000","last_fullsized":"2023-10-16T11:24:40.948878+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:37.032264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.080660+0000","last_change":"2023-10-16T11:24:40.934994+0000","last_active":"2023-10-16T11:24:42.080660+0000","last_peered":"2023-10-16T11:24:42.080660+0000","last_clean":"2023-10-16T11:24:42.080660+0000","last_became_active":"2023-10-16T11:24:40.929932+0000","last_became_peered":"2023-10-16T11:24:40.929932+0000","last_unstale":"2023-10-16T11:24:42.080660+0000","last_undegraded":"2023-10-16T11:24:42.080660+0000","last_fullsized":"2023-10-16T11:24:42.080660+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:23.723306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913447+0000","last_change":"2023-10-16T11:24:25.087755+0000","last_active":"2023-10-16T11:24:40.913447+0000","last_peered":"2023-10-16T11:24:40.913447+0000","last_clean":"2023-10-16T11:24:40.913447+0000","last_became_active":"2023-10-16T11:24:25.087497+0000","last_became_peered":"2023-10-16T11:24:25.087497+0000","last_unstale":"2023-10-16T11:24:40.913447+0000","last_undegraded":"2023-10-16T11:24:40.913447+0000","last_fullsized":"2023-10-16T11:24:40.913447+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:59:26.196708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.929543+0000","last_change":"2023-10-16T11:24:26.081718+0000","last_active":"2023-10-16T11:24:40.929543+0000","last_peered":"2023-10-16T11:24:40.929543+0000","last_clean":"2023-10-16T11:24:40.929543+0000","last_became_active":"2023-10-16T11:24:26.081447+0000","last_became_peered":"2023-10-16T11:24:26.081447+0000","last_unstale":"2023-10-16T11:24:40.929543+0000","last_undegraded":"2023-10-16T11:24:40.929543+0000","last_fullsized":"2023-10-16T11:24:40.929543+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"221'1","reported_seq":13,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.079572+0000","last_change":"2023-10-16T11:24:39.923647+0000","last_active":"2023-10-16T11:24:42.079572+0000","last_peered":"2023-10-16T11:24:42.079572+0000","last_clean":"2023-10-16T11:24:42.079572+0000","last_became_active":"2023-10-16T11:24:39.923359+0000","last_became_peered":"2023-10-16T11:24:39.923359+0000","last_unstale":"2023-10-16T11:24:42.079572+0000","last_undegraded":"2023-10-16T11:24:42.079572+0000","last_fullsized":"2023-10-16T11:24:42.079572+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:18:00.023742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.073930+0000","last_change":"2023-10-16T11:24:40.934136+0000","last_active":"2023-10-16T11:24:42.073930+0000","last_peered":"2023-10-16T11:24:42.073930+0000","last_clean":"2023-10-16T11:24:42.073930+0000","last_became_active":"2023-10-16T11:24:40.931559+0000","last_became_peered":"2023-10-16T11:24:40.931559+0000","last_unstale":"2023-10-16T11:24:42.073930+0000","last_undegraded":"2023-10-16T11:24:42.073930+0000","last_fullsized":"2023-10-16T11:24:42.073930+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:54:04.701711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.926384+0000","last_change":"2023-10-16T11:24:25.088161+0000","last_active":"2023-10-16T11:24:40.926384+0000","last_peered":"2023-10-16T11:24:40.926384+0000","last_clean":"2023-10-16T11:24:40.926384+0000","last_became_active":"2023-10-16T11:24:25.086051+0000","last_became_peered":"2023-10-16T11:24:25.086051+0000","last_unstale":"2023-10-16T11:24:40.926384+0000","last_undegraded":"2023-10-16T11:24:40.926384+0000","last_fullsized":"2023-10-16T11:24:40.926384+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.907231+0000","last_change":"2023-10-16T11:24:26.083366+0000","last_active":"2023-10-16T11:24:40.907231+0000","last_peered":"2023-10-16T11:24:40.907231+0000","last_clean":"2023-10-16T11:24:40.907231+0000","last_became_active":"2023-10-16T11:24:26.082635+0000","last_became_peered":"2023-10-16T11:24:26.082635+0000","last_unstale":"2023-10-16T11:24:40.907231+0000","last_undegraded":"2023-10-16T11:24:40.907231+0000","last_fullsized":"2023-10-16T11:24:40.907231+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.942903+0000","last_change":"2023-10-16T11:24:39.926910+0000","last_active":"2023-10-16T11:24:40.942903+0000","last_peered":"2023-10-16T11:24:40.942903+0000","last_clean":"2023-10-16T11:24:40.942903+0000","last_became_active":"2023-10-16T11:24:39.926615+0000","last_became_peered":"2023-10-16T11:24:39.926615+0000","last_unstale":"2023-10-16T11:24:40.942903+0000","last_undegraded":"2023-10-16T11:24:40.942903+0000","last_fullsized":"2023-10-16T11:24:40.942903+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:06.744007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.073721+0000","last_change":"2023-10-16T11:24:40.923750+0000","last_active":"2023-10-16T11:24:42.073721+0000","last_peered":"2023-10-16T11:24:42.073721+0000","last_clean":"2023-10-16T11:24:42.073721+0000","last_became_active":"2023-10-16T11:24:40.922455+0000","last_became_peered":"2023-10-16T11:24:40.922455+0000","last_unstale":"2023-10-16T11:24:42.073721+0000","last_undegraded":"2023-10-16T11:24:42.073721+0000","last_fullsized":"2023-10-16T11:24:42.073721+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:40.760759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912963+0000","last_change":"2023-10-16T11:24:25.080689+0000","last_active":"2023-10-16T11:24:40.912963+0000","last_peered":"2023-10-16T11:24:40.912963+0000","last_clean":"2023-10-16T11:24:40.912963+0000","last_became_active":"2023-10-16T11:24:25.080305+0000","last_became_peered":"2023-10-16T11:24:25.080305+0000","last_unstale":"2023-10-16T11:24:40.912963+0000","last_undegraded":"2023-10-16T11:24:40.912963+0000","last_fullsized":"2023-10-16T11:24:40.912963+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:43.670603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.916175+0000","last_change":"2023-10-16T11:24:26.083766+0000","last_active":"2023-10-16T11:24:40.916175+0000","last_peered":"2023-10-16T11:24:40.916175+0000","last_clean":"2023-10-16T11:24:40.916175+0000","last_became_active":"2023-10-16T11:24:26.083447+0000","last_became_peered":"2023-10-16T11:24:26.083447+0000","last_unstale":"2023-10-16T11:24:40.916175+0000","last_undegraded":"2023-10-16T11:24:40.916175+0000","last_fullsized":"2023-10-16T11:24:40.916175+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:44:14.656443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.914990+0000","last_change":"2023-10-16T11:24:39.926629+0000","last_active":"2023-10-16T11:24:40.914990+0000","last_peered":"2023-10-16T11:24:40.914990+0000","last_clean":"2023-10-16T11:24:40.914990+0000","last_became_active":"2023-10-16T11:24:39.926334+0000","last_became_peered":"2023-10-16T11:24:39.926334+0000","last_unstale":"2023-10-16T11:24:40.914990+0000","last_undegraded":"2023-10-16T11:24:40.914990+0000","last_fullsized":"2023-10-16T11:24:40.914990+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:24.166384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076395+0000","last_change":"2023-10-16T11:24:40.940956+0000","last_active":"2023-10-16T11:24:42.076395+0000","last_peered":"2023-10-16T11:24:42.076395+0000","last_clean":"2023-10-16T11:24:42.076395+0000","last_became_active":"2023-10-16T11:24:40.940649+0000","last_became_peered":"2023-10-16T11:24:40.940649+0000","last_unstale":"2023-10-16T11:24:42.076395+0000","last_undegraded":"2023-10-16T11:24:42.076395+0000","last_fullsized":"2023-10-16T11:24:42.076395+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:59.520985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.921975+0000","last_change":"2023-10-16T11:24:25.090432+0000","last_active":"2023-10-16T11:24:40.921975+0000","last_peered":"2023-10-16T11:24:40.921975+0000","last_clean":"2023-10-16T11:24:40.921975+0000","last_became_active":"2023-10-16T11:24:25.090142+0000","last_became_peered":"2023-10-16T11:24:25.090142+0000","last_unstale":"2023-10-16T11:24:40.921975+0000","last_undegraded":"2023-10-16T11:24:40.921975+0000","last_fullsized":"2023-10-16T11:24:40.921975+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:22.858690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918327+0000","last_change":"2023-10-16T11:24:26.093027+0000","last_active":"2023-10-16T11:24:40.918327+0000","last_peered":"2023-10-16T11:24:40.918327+0000","last_clean":"2023-10-16T11:24:40.918327+0000","last_became_active":"2023-10-16T11:24:26.092766+0000","last_became_peered":"2023-10-16T11:24:26.092766+0000","last_unstale":"2023-10-16T11:24:40.918327+0000","last_undegraded":"2023-10-16T11:24:40.918327+0000","last_fullsized":"2023-10-16T11:24:40.918327+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.923353+0000","last_change":"2023-10-16T11:24:39.924175+0000","last_active":"2023-10-16T11:24:40.923353+0000","last_peered":"2023-10-16T11:24:40.923353+0000","last_clean":"2023-10-16T11:24:40.923353+0000","last_became_active":"2023-10-16T11:24:39.923702+0000","last_became_peered":"2023-10-16T11:24:39.923702+0000","last_unstale":"2023-10-16T11:24:40.923353+0000","last_undegraded":"2023-10-16T11:24:40.923353+0000","last_fullsized":"2023-10-16T11:24:40.923353+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:33:55.188653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.077311+0000","last_change":"2023-10-16T11:24:40.937310+0000","last_active":"2023-10-16T11:24:42.077311+0000","last_peered":"2023-10-16T11:24:42.077311+0000","last_clean":"2023-10-16T11:24:42.077311+0000","last_became_active":"2023-10-16T11:24:40.937033+0000","last_became_peered":"2023-10-16T11:24:40.937033+0000","last_unstale":"2023-10-16T11:24:42.077311+0000","last_undegraded":"2023-10-16T11:24:42.077311+0000","last_fullsized":"2023-10-16T11:24:42.077311+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:51:40.711088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.908646+0000","last_change":"2023-10-16T11:24:25.088585+0000","last_active":"2023-10-16T11:24:40.908646+0000","last_peered":"2023-10-16T11:24:40.908646+0000","last_clean":"2023-10-16T11:24:40.908646+0000","last_became_active":"2023-10-16T11:24:25.088011+0000","last_became_peered":"2023-10-16T11:24:25.088011+0000","last_unstale":"2023-10-16T11:24:40.908646+0000","last_undegraded":"2023-10-16T11:24:40.908646+0000","last_fullsized":"2023-10-16T11:24:40.908646+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920936+0000","last_change":"2023-10-16T11:24:26.096341+0000","last_active":"2023-10-16T11:24:40.920936+0000","last_peered":"2023-10-16T11:24:40.920936+0000","last_clean":"2023-10-16T11:24:40.920936+0000","last_became_active":"2023-10-16T11:24:26.095836+0000","last_became_peered":"2023-10-16T11:24:26.095836+0000","last_unstale":"2023-10-16T11:24:40.920936+0000","last_undegraded":"2023-10-16T11:24:40.920936+0000","last_fullsized":"2023-10-16T11:24:40.920936+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917221+0000","last_change":"2023-10-16T11:24:39.909816+0000","last_active":"2023-10-16T11:24:40.917221+0000","last_peered":"2023-10-16T11:24:40.917221+0000","last_clean":"2023-10-16T11:24:40.917221+0000","last_became_active":"2023-10-16T11:24:39.909526+0000","last_became_peered":"2023-10-16T11:24:39.909526+0000","last_unstale":"2023-10-16T11:24:40.917221+0000","last_undegraded":"2023-10-16T11:24:40.917221+0000","last_fullsized":"2023-10-16T11:24:40.917221+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:12.120581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935566+0000","last_change":"2023-10-16T11:24:40.938174+0000","last_active":"2023-10-16T11:24:41.935566+0000","last_peered":"2023-10-16T11:24:41.935566+0000","last_clean":"2023-10-16T11:24:41.935566+0000","last_became_active":"2023-10-16T11:24:40.937904+0000","last_became_peered":"2023-10-16T11:24:40.937904+0000","last_unstale":"2023-10-16T11:24:41.935566+0000","last_undegraded":"2023-10-16T11:24:41.935566+0000","last_fullsized":"2023-10-16T11:24:41.935566+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:13.135380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913653+0000","last_change":"2023-10-16T11:24:25.077036+0000","last_active":"2023-10-16T11:24:40.913653+0000","last_peered":"2023-10-16T11:24:40.913653+0000","last_clean":"2023-10-16T11:24:40.913653+0000","last_became_active":"2023-10-16T11:24:25.076342+0000","last_became_peered":"2023-10-16T11:24:25.076342+0000","last_unstale":"2023-10-16T11:24:40.913653+0000","last_undegraded":"2023-10-16T11:24:40.913653+0000","last_fullsized":"2023-10-16T11:24:40.913653+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:36.162662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.935192+0000","last_change":"2023-10-16T11:24:26.085231+0000","last_active":"2023-10-16T11:24:40.935192+0000","last_peered":"2023-10-16T11:24:40.935192+0000","last_clean":"2023-10-16T11:24:40.935192+0000","last_became_active":"2023-10-16T11:24:26.084889+0000","last_became_peered":"2023-10-16T11:24:26.084889+0000","last_unstale":"2023-10-16T11:24:40.935192+0000","last_undegraded":"2023-10-16T11:24:40.935192+0000","last_fullsized":"2023-10-16T11:24:40.935192+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.935331+0000","last_change":"2023-10-16T11:24:39.920299+0000","last_active":"2023-10-16T11:24:40.935331+0000","last_peered":"2023-10-16T11:24:40.935331+0000","last_clean":"2023-10-16T11:24:40.935331+0000","last_became_active":"2023-10-16T11:24:39.919985+0000","last_became_peered":"2023-10-16T11:24:39.919985+0000","last_unstale":"2023-10-16T11:24:40.935331+0000","last_undegraded":"2023-10-16T11:24:40.935331+0000","last_fullsized":"2023-10-16T11:24:40.935331+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:59.121334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.077429+0000","last_change":"2023-10-16T11:24:40.940322+0000","last_active":"2023-10-16T11:24:42.077429+0000","last_peered":"2023-10-16T11:24:42.077429+0000","last_clean":"2023-10-16T11:24:42.077429+0000","last_became_active":"2023-10-16T11:24:40.940072+0000","last_became_peered":"2023-10-16T11:24:40.940072+0000","last_unstale":"2023-10-16T11:24:42.077429+0000","last_undegraded":"2023-10-16T11:24:42.077429+0000","last_fullsized":"2023-10-16T11:24:42.077429+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:33.978064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.6","version":"217'1","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.928431+0000","last_change":"2023-10-16T11:24:25.093214+0000","last_active":"2023-10-16T11:24:40.928431+0000","last_peered":"2023-10-16T11:24:40.928431+0000","last_clean":"2023-10-16T11:24:40.928431+0000","last_became_active":"2023-10-16T11:24:25.092877+0000","last_became_peered":"2023-10-16T11:24:25.092877+0000","last_unstale":"2023-10-16T11:24:40.928431+0000","last_undegraded":"2023-10-16T11:24:40.928431+0000","last_fullsized":"2023-10-16T11:24:40.928431+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:59.924424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.919655+0000","last_change":"2023-10-16T11:24:26.089940+0000","last_active":"2023-10-16T11:24:40.919655+0000","last_peered":"2023-10-16T11:24:40.919655+0000","last_clean":"2023-10-16T11:24:40.919655+0000","last_became_active":"2023-10-16T11:24:26.087906+0000","last_became_peered":"2023-10-16T11:24:26.087906+0000","last_unstale":"2023-10-16T11:24:40.919655+0000","last_undegraded":"2023-10-16T11:24:40.919655+0000","last_fullsized":"2023-10-16T11:24:40.919655+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:44.687193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913130+0000","last_change":"2023-10-16T11:24:39.918078+0000","last_active":"2023-10-16T11:24:40.913130+0000","last_peered":"2023-10-16T11:24:40.913130+0000","last_clean":"2023-10-16T11:24:40.913130+0000","last_became_active":"2023-10-16T11:24:39.914432+0000","last_became_peered":"2023-10-16T11:24:39.914432+0000","last_unstale":"2023-10-16T11:24:40.913130+0000","last_undegraded":"2023-10-16T11:24:40.913130+0000","last_fullsized":"2023-10-16T11:24:40.913130+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:07:10.996990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.074496+0000","last_change":"2023-10-16T11:24:40.940070+0000","last_active":"2023-10-16T11:24:42.074496+0000","last_peered":"2023-10-16T11:24:42.074496+0000","last_clean":"2023-10-16T11:24:42.074496+0000","last_became_active":"2023-10-16T11:24:40.939742+0000","last_became_peered":"2023-10-16T11:24:40.939742+0000","last_unstale":"2023-10-16T11:24:42.074496+0000","last_undegraded":"2023-10-16T11:24:42.074496+0000","last_fullsized":"2023-10-16T11:24:42.074496+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:07:00.381749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"216'2","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.909469+0000","last_change":"2023-10-16T11:24:25.094230+0000","last_active":"2023-10-16T11:24:40.909469+0000","last_peered":"2023-10-16T11:24:40.909469+0000","last_clean":"2023-10-16T11:24:40.909469+0000","last_became_active":"2023-10-16T11:24:25.093896+0000","last_became_peered":"2023-10-16T11:24:25.093896+0000","last_unstale":"2023-10-16T11:24:40.909469+0000","last_undegraded":"2023-10-16T11:24:40.909469+0000","last_fullsized":"2023-10-16T11:24:40.909469+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.915544+0000","last_change":"2023-10-16T11:24:26.085909+0000","last_active":"2023-10-16T11:24:40.915544+0000","last_peered":"2023-10-16T11:24:40.915544+0000","last_clean":"2023-10-16T11:24:40.915544+0000","last_became_active":"2023-10-16T11:24:26.085620+0000","last_became_peered":"2023-10-16T11:24:26.085620+0000","last_unstale":"2023-10-16T11:24:40.915544+0000","last_undegraded":"2023-10-16T11:24:40.915544+0000","last_fullsized":"2023-10-16T11:24:40.915544+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:41.023833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913673+0000","last_change":"2023-10-16T11:24:39.910612+0000","last_active":"2023-10-16T11:24:40.913673+0000","last_peered":"2023-10-16T11:24:40.913673+0000","last_clean":"2023-10-16T11:24:40.913673+0000","last_became_active":"2023-10-16T11:24:39.910324+0000","last_became_peered":"2023-10-16T11:24:39.910324+0000","last_unstale":"2023-10-16T11:24:40.913673+0000","last_undegraded":"2023-10-16T11:24:40.913673+0000","last_fullsized":"2023-10-16T11:24:40.913673+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:44:02.305284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.934777+0000","last_change":"2023-10-16T11:24:40.945289+0000","last_active":"2023-10-16T11:24:41.934777+0000","last_peered":"2023-10-16T11:24:41.934777+0000","last_clean":"2023-10-16T11:24:41.934777+0000","last_became_active":"2023-10-16T11:24:40.944986+0000","last_became_peered":"2023-10-16T11:24:40.944986+0000","last_unstale":"2023-10-16T11:24:41.934777+0000","last_undegraded":"2023-10-16T11:24:41.934777+0000","last_fullsized":"2023-10-16T11:24:41.934777+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:46.340170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912752+0000","last_change":"2023-10-16T11:24:25.080225+0000","last_active":"2023-10-16T11:24:40.912752+0000","last_peered":"2023-10-16T11:24:40.912752+0000","last_clean":"2023-10-16T11:24:40.912752+0000","last_became_active":"2023-10-16T11:24:25.079921+0000","last_became_peered":"2023-10-16T11:24:25.079921+0000","last_unstale":"2023-10-16T11:24:40.912752+0000","last_undegraded":"2023-10-16T11:24:40.912752+0000","last_fullsized":"2023-10-16T11:24:40.912752+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:46.550429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.926296+0000","last_change":"2023-10-16T11:24:26.088438+0000","last_active":"2023-10-16T11:24:40.926296+0000","last_peered":"2023-10-16T11:24:40.926296+0000","last_clean":"2023-10-16T11:24:40.926296+0000","last_became_active":"2023-10-16T11:24:26.087973+0000","last_became_peered":"2023-10-16T11:24:26.087973+0000","last_unstale":"2023-10-16T11:24:40.926296+0000","last_undegraded":"2023-10-16T11:24:40.926296+0000","last_fullsized":"2023-10-16T11:24:40.926296+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.944794+0000","last_change":"2023-10-16T11:24:39.915986+0000","last_active":"2023-10-16T11:24:40.944794+0000","last_peered":"2023-10-16T11:24:40.944794+0000","last_clean":"2023-10-16T11:24:40.944794+0000","last_became_active":"2023-10-16T11:24:39.915611+0000","last_became_peered":"2023-10-16T11:24:39.915611+0000","last_unstale":"2023-10-16T11:24:40.944794+0000","last_undegraded":"2023-10-16T11:24:40.944794+0000","last_fullsized":"2023-10-16T11:24:40.944794+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:37.032264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935666+0000","last_change":"2023-10-16T11:24:40.932578+0000","last_active":"2023-10-16T11:24:41.935666+0000","last_peered":"2023-10-16T11:24:41.935666+0000","last_clean":"2023-10-16T11:24:41.935666+0000","last_became_active":"2023-10-16T11:24:40.931761+0000","last_became_peered":"2023-10-16T11:24:40.931761+0000","last_unstale":"2023-10-16T11:24:41.935666+0000","last_undegraded":"2023-10-16T11:24:41.935666+0000","last_fullsized":"2023-10-16T11:24:41.935666+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:55.400689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917511+0000","last_change":"2023-10-16T11:24:25.086180+0000","last_active":"2023-10-16T11:24:40.917511+0000","last_peered":"2023-10-16T11:24:40.917511+0000","last_clean":"2023-10-16T11:24:40.917511+0000","last_became_active":"2023-10-16T11:24:25.085316+0000","last_became_peered":"2023-10-16T11:24:25.085316+0000","last_unstale":"2023-10-16T11:24:40.917511+0000","last_undegraded":"2023-10-16T11:24:40.917511+0000","last_fullsized":"2023-10-16T11:24:40.917511+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927499+0000","last_change":"2023-10-16T11:24:26.090951+0000","last_active":"2023-10-16T11:24:40.927499+0000","last_peered":"2023-10-16T11:24:40.927499+0000","last_clean":"2023-10-16T11:24:40.927499+0000","last_became_active":"2023-10-16T11:24:26.089241+0000","last_became_peered":"2023-10-16T11:24:26.089241+0000","last_unstale":"2023-10-16T11:24:40.927499+0000","last_undegraded":"2023-10-16T11:24:40.927499+0000","last_fullsized":"2023-10-16T11:24:40.927499+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:40.970357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"220'1","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.946450+0000","last_change":"2023-10-16T11:24:39.926741+0000","last_active":"2023-10-16T11:24:40.946450+0000","last_peered":"2023-10-16T11:24:40.946450+0000","last_clean":"2023-10-16T11:24:40.946450+0000","last_became_active":"2023-10-16T11:24:39.926377+0000","last_became_peered":"2023-10-16T11:24:39.926377+0000","last_unstale":"2023-10-16T11:24:40.946450+0000","last_undegraded":"2023-10-16T11:24:40.946450+0000","last_fullsized":"2023-10-16T11:24:40.946450+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:07:06.185553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.080553+0000","last_change":"2023-10-16T11:24:40.936952+0000","last_active":"2023-10-16T11:24:42.080553+0000","last_peered":"2023-10-16T11:24:42.080553+0000","last_clean":"2023-10-16T11:24:42.080553+0000","last_became_active":"2023-10-16T11:24:40.936526+0000","last_became_peered":"2023-10-16T11:24:40.936526+0000","last_unstale":"2023-10-16T11:24:42.080553+0000","last_undegraded":"2023-10-16T11:24:42.080553+0000","last_fullsized":"2023-10-16T11:24:42.080553+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:43:43.109538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918490+0000","last_change":"2023-10-16T11:24:25.091656+0000","last_active":"2023-10-16T11:24:40.918490+0000","last_peered":"2023-10-16T11:24:40.918490+0000","last_clean":"2023-10-16T11:24:40.918490+0000","last_became_active":"2023-10-16T11:24:25.091316+0000","last_became_peered":"2023-10-16T11:24:25.091316+0000","last_unstale":"2023-10-16T11:24:40.918490+0000","last_undegraded":"2023-10-16T11:24:40.918490+0000","last_fullsized":"2023-10-16T11:24:40.918490+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:26.054702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.908836+0000","last_change":"2023-10-16T11:24:26.085260+0000","last_active":"2023-10-16T11:24:40.908836+0000","last_peered":"2023-10-16T11:24:40.908836+0000","last_clean":"2023-10-16T11:24:40.908836+0000","last_became_active":"2023-10-16T11:24:26.084860+0000","last_became_peered":"2023-10-16T11:24:26.084860+0000","last_unstale":"2023-10-16T11:24:40.908836+0000","last_undegraded":"2023-10-16T11:24:40.908836+0000","last_fullsized":"2023-10-16T11:24:40.908836+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920928+0000","last_change":"2023-10-16T11:24:39.908388+0000","last_active":"2023-10-16T11:24:40.920928+0000","last_peered":"2023-10-16T11:24:40.920928+0000","last_clean":"2023-10-16T11:24:40.920928+0000","last_became_active":"2023-10-16T11:24:39.908075+0000","last_became_peered":"2023-10-16T11:24:39.908075+0000","last_unstale":"2023-10-16T11:24:40.920928+0000","last_undegraded":"2023-10-16T11:24:40.920928+0000","last_fullsized":"2023-10-16T11:24:40.920928+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:12:56.357582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.074734+0000","last_change":"2023-10-16T11:24:40.934674+0000","last_active":"2023-10-16T11:24:42.074734+0000","last_peered":"2023-10-16T11:24:42.074734+0000","last_clean":"2023-10-16T11:24:42.074734+0000","last_became_active":"2023-10-16T11:24:40.934420+0000","last_became_peered":"2023-10-16T11:24:40.934420+0000","last_unstale":"2023-10-16T11:24:42.074734+0000","last_undegraded":"2023-10-16T11:24:42.074734+0000","last_fullsized":"2023-10-16T11:24:42.074734+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:52.013044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.b","version":"216'2","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.923129+0000","last_change":"2023-10-16T11:24:25.072340+0000","last_active":"2023-10-16T11:24:40.923129+0000","last_peered":"2023-10-16T11:24:40.923129+0000","last_clean":"2023-10-16T11:24:40.923129+0000","last_became_active":"2023-10-16T11:24:25.071728+0000","last_became_peered":"2023-10-16T11:24:25.071728+0000","last_unstale":"2023-10-16T11:24:40.923129+0000","last_undegraded":"2023-10-16T11:24:40.923129+0000","last_fullsized":"2023-10-16T11:24:40.923129+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:49.041521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927055+0000","last_change":"2023-10-16T11:24:26.091096+0000","last_active":"2023-10-16T11:24:40.927055+0000","last_peered":"2023-10-16T11:24:40.927055+0000","last_clean":"2023-10-16T11:24:40.927055+0000","last_became_active":"2023-10-16T11:24:26.090774+0000","last_became_peered":"2023-10-16T11:24:26.090774+0000","last_unstale":"2023-10-16T11:24:40.927055+0000","last_undegraded":"2023-10-16T11:24:40.927055+0000","last_fullsized":"2023-10-16T11:24:40.927055+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:27.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"220'1","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.944256+0000","last_change":"2023-10-16T11:24:39.921764+0000","last_active":"2023-10-16T11:24:40.944256+0000","last_peered":"2023-10-16T11:24:40.944256+0000","last_clean":"2023-10-16T11:24:40.944256+0000","last_became_active":"2023-10-16T11:24:39.921487+0000","last_became_peered":"2023-10-16T11:24:39.921487+0000","last_unstale":"2023-10-16T11:24:40.944256+0000","last_undegraded":"2023-10-16T11:24:40.944256+0000","last_fullsized":"2023-10-16T11:24:40.944256+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:25.891782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076327+0000","last_change":"2023-10-16T11:24:40.946254+0000","last_active":"2023-10-16T11:24:42.076327+0000","last_peered":"2023-10-16T11:24:42.076327+0000","last_clean":"2023-10-16T11:24:42.076327+0000","last_became_active":"2023-10-16T11:24:40.945902+0000","last_became_peered":"2023-10-16T11:24:40.945902+0000","last_unstale":"2023-10-16T11:24:42.076327+0000","last_undegraded":"2023-10-16T11:24:42.076327+0000","last_fullsized":"2023-10-16T11:24:42.076327+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:52.439191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.908290+0000","last_change":"2023-10-16T11:24:25.079021+0000","last_active":"2023-10-16T11:24:40.908290+0000","last_peered":"2023-10-16T11:24:40.908290+0000","last_clean":"2023-10-16T11:24:40.908290+0000","last_became_active":"2023-10-16T11:24:25.076800+0000","last_became_peered":"2023-10-16T11:24:25.076800+0000","last_unstale":"2023-10-16T11:24:40.908290+0000","last_undegraded":"2023-10-16T11:24:40.908290+0000","last_fullsized":"2023-10-16T11:24:40.908290+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:56.161407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920389+0000","last_change":"2023-10-16T11:24:26.092876+0000","last_active":"2023-10-16T11:24:40.920389+0000","last_peered":"2023-10-16T11:24:40.920389+0000","last_clean":"2023-10-16T11:24:40.920389+0000","last_became_active":"2023-10-16T11:24:26.091310+0000","last_became_peered":"2023-10-16T11:24:26.091310+0000","last_unstale":"2023-10-16T11:24:40.920389+0000","last_undegraded":"2023-10-16T11:24:40.920389+0000","last_fullsized":"2023-10-16T11:24:40.920389+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:22.891780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.931170+0000","last_change":"2023-10-16T11:24:39.918307+0000","last_active":"2023-10-16T11:24:40.931170+0000","last_peered":"2023-10-16T11:24:40.931170+0000","last_clean":"2023-10-16T11:24:40.931170+0000","last_became_active":"2023-10-16T11:24:39.916755+0000","last_became_peered":"2023-10-16T11:24:39.916755+0000","last_unstale":"2023-10-16T11:24:40.931170+0000","last_undegraded":"2023-10-16T11:24:40.931170+0000","last_fullsized":"2023-10-16T11:24:40.931170+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:54.385890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072744+0000","last_change":"2023-10-16T11:24:40.929984+0000","last_active":"2023-10-16T11:24:42.072744+0000","last_peered":"2023-10-16T11:24:42.072744+0000","last_clean":"2023-10-16T11:24:42.072744+0000","last_became_active":"2023-10-16T11:24:40.928183+0000","last_became_peered":"2023-10-16T11:24:40.928183+0000","last_unstale":"2023-10-16T11:24:42.072744+0000","last_undegraded":"2023-10-16T11:24:42.072744+0000","last_fullsized":"2023-10-16T11:24:42.072744+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:00:55.030170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.d","version":"216'1","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920754+0000","last_change":"2023-10-16T11:24:25.083155+0000","last_active":"2023-10-16T11:24:40.920754+0000","last_peered":"2023-10-16T11:24:40.920754+0000","last_clean":"2023-10-16T11:24:40.920754+0000","last_became_active":"2023-10-16T11:24:25.082690+0000","last_became_peered":"2023-10-16T11:24:25.082690+0000","last_unstale":"2023-10-16T11:24:40.920754+0000","last_undegraded":"2023-10-16T11:24:40.920754+0000","last_fullsized":"2023-10-16T11:24:40.920754+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:46:58.816018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.908131+0000","last_change":"2023-10-16T11:24:26.069780+0000","last_active":"2023-10-16T11:24:40.908131+0000","last_peered":"2023-10-16T11:24:40.908131+0000","last_clean":"2023-10-16T11:24:40.908131+0000","last_became_active":"2023-10-16T11:24:26.068772+0000","last_became_peered":"2023-10-16T11:24:26.068772+0000","last_unstale":"2023-10-16T11:24:40.908131+0000","last_undegraded":"2023-10-16T11:24:40.908131+0000","last_fullsized":"2023-10-16T11:24:40.908131+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.944704+0000","last_change":"2023-10-16T11:24:39.928140+0000","last_active":"2023-10-16T11:24:40.944704+0000","last_peered":"2023-10-16T11:24:40.944704+0000","last_clean":"2023-10-16T11:24:40.944704+0000","last_became_active":"2023-10-16T11:24:39.927182+0000","last_became_peered":"2023-10-16T11:24:39.927182+0000","last_unstale":"2023-10-16T11:24:40.944704+0000","last_undegraded":"2023-10-16T11:24:40.944704+0000","last_fullsized":"2023-10-16T11:24:40.944704+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:04:01.386012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072533+0000","last_change":"2023-10-16T11:24:40.937098+0000","last_active":"2023-10-16T11:24:42.072533+0000","last_peered":"2023-10-16T11:24:42.072533+0000","last_clean":"2023-10-16T11:24:42.072533+0000","last_became_active":"2023-10-16T11:24:40.936844+0000","last_became_peered":"2023-10-16T11:24:40.936844+0000","last_unstale":"2023-10-16T11:24:42.072533+0000","last_undegraded":"2023-10-16T11:24:42.072533+0000","last_fullsized":"2023-10-16T11:24:42.072533+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:06.892515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.e","version":"216'2","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.933425+0000","last_change":"2023-10-16T11:24:25.087765+0000","last_active":"2023-10-16T11:24:40.933425+0000","last_peered":"2023-10-16T11:24:40.933425+0000","last_clean":"2023-10-16T11:24:40.933425+0000","last_became_active":"2023-10-16T11:24:25.086967+0000","last_became_peered":"2023-10-16T11:24:25.086967+0000","last_unstale":"2023-10-16T11:24:40.933425+0000","last_undegraded":"2023-10-16T11:24:40.933425+0000","last_fullsized":"2023-10-16T11:24:40.933425+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:05.475547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.932831+0000","last_change":"2023-10-16T11:24:26.086833+0000","last_active":"2023-10-16T11:24:40.932831+0000","last_peered":"2023-10-16T11:24:40.932831+0000","last_clean":"2023-10-16T11:24:40.932831+0000","last_became_active":"2023-10-16T11:24:26.085367+0000","last_became_peered":"2023-10-16T11:24:26.085367+0000","last_unstale":"2023-10-16T11:24:40.932831+0000","last_undegraded":"2023-10-16T11:24:40.932831+0000","last_fullsized":"2023-10-16T11:24:40.932831+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913101+0000","last_change":"2023-10-16T11:24:39.913504+0000","last_active":"2023-10-16T11:24:40.913101+0000","last_peered":"2023-10-16T11:24:40.913101+0000","last_clean":"2023-10-16T11:24:40.913101+0000","last_became_active":"2023-10-16T11:24:39.913184+0000","last_became_peered":"2023-10-16T11:24:39.913184+0000","last_unstale":"2023-10-16T11:24:40.913101+0000","last_undegraded":"2023-10-16T11:24:40.913101+0000","last_fullsized":"2023-10-16T11:24:40.913101+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:36.821422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.931761+0000","last_change":"2023-10-16T11:24:40.936862+0000","last_active":"2023-10-16T11:24:41.931761+0000","last_peered":"2023-10-16T11:24:41.931761+0000","last_clean":"2023-10-16T11:24:41.931761+0000","last_became_active":"2023-10-16T11:24:40.933139+0000","last_became_peered":"2023-10-16T11:24:40.933139+0000","last_unstale":"2023-10-16T11:24:41.931761+0000","last_undegraded":"2023-10-16T11:24:41.931761+0000","last_fullsized":"2023-10-16T11:24:41.931761+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:23.266881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.f","version":"216'1","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913585+0000","last_change":"2023-10-16T11:24:25.079789+0000","last_active":"2023-10-16T11:24:40.913585+0000","last_peered":"2023-10-16T11:24:40.913585+0000","last_clean":"2023-10-16T11:24:40.913585+0000","last_became_active":"2023-10-16T11:24:25.079495+0000","last_became_peered":"2023-10-16T11:24:25.079495+0000","last_unstale":"2023-10-16T11:24:40.913585+0000","last_undegraded":"2023-10-16T11:24:40.913585+0000","last_fullsized":"2023-10-16T11:24:40.913585+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:44.531367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.933148+0000","last_change":"2023-10-16T11:24:26.081796+0000","last_active":"2023-10-16T11:24:40.933148+0000","last_peered":"2023-10-16T11:24:40.933148+0000","last_clean":"2023-10-16T11:24:40.933148+0000","last_became_active":"2023-10-16T11:24:26.080287+0000","last_became_peered":"2023-10-16T11:24:26.080287+0000","last_unstale":"2023-10-16T11:24:40.933148+0000","last_undegraded":"2023-10-16T11:24:40.933148+0000","last_fullsized":"2023-10-16T11:24:40.933148+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927703+0000","last_change":"2023-10-16T11:24:39.922329+0000","last_active":"2023-10-16T11:24:40.927703+0000","last_peered":"2023-10-16T11:24:40.927703+0000","last_clean":"2023-10-16T11:24:40.927703+0000","last_became_active":"2023-10-16T11:24:39.921982+0000","last_became_peered":"2023-10-16T11:24:39.921982+0000","last_unstale":"2023-10-16T11:24:40.927703+0000","last_undegraded":"2023-10-16T11:24:40.927703+0000","last_fullsized":"2023-10-16T11:24:40.927703+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:17.673425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.080239+0000","last_change":"2023-10-16T11:24:40.949583+0000","last_active":"2023-10-16T11:24:42.080239+0000","last_peered":"2023-10-16T11:24:42.080239+0000","last_clean":"2023-10-16T11:24:42.080239+0000","last_became_active":"2023-10-16T11:24:40.947541+0000","last_became_peered":"2023-10-16T11:24:40.947541+0000","last_unstale":"2023-10-16T11:24:42.080239+0000","last_undegraded":"2023-10-16T11:24:42.080239+0000","last_fullsized":"2023-10-16T11:24:42.080239+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:10:12.883381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.10","version":"216'8","reported_seq":24,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.907330+0000","last_change":"2023-10-16T11:24:25.076954+0000","last_active":"2023-10-16T11:24:40.907330+0000","last_peered":"2023-10-16T11:24:40.907330+0000","last_clean":"2023-10-16T11:24:40.907330+0000","last_became_active":"2023-10-16T11:24:25.076509+0000","last_became_peered":"2023-10-16T11:24:25.076509+0000","last_unstale":"2023-10-16T11:24:40.907330+0000","last_undegraded":"2023-10-16T11:24:40.907330+0000","last_fullsized":"2023-10-16T11:24:40.907330+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:46.770213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917418+0000","last_change":"2023-10-16T11:24:26.084139+0000","last_active":"2023-10-16T11:24:40.917418+0000","last_peered":"2023-10-16T11:24:40.917418+0000","last_clean":"2023-10-16T11:24:40.917418+0000","last_became_active":"2023-10-16T11:24:26.083876+0000","last_became_peered":"2023-10-16T11:24:26.083876+0000","last_unstale":"2023-10-16T11:24:40.917418+0000","last_undegraded":"2023-10-16T11:24:40.917418+0000","last_fullsized":"2023-10-16T11:24:40.917418+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918095+0000","last_change":"2023-10-16T11:24:39.904420+0000","last_active":"2023-10-16T11:24:40.918095+0000","last_peered":"2023-10-16T11:24:40.918095+0000","last_clean":"2023-10-16T11:24:40.918095+0000","last_became_active":"2023-10-16T11:24:39.904039+0000","last_became_peered":"2023-10-16T11:24:39.904039+0000","last_unstale":"2023-10-16T11:24:40.918095+0000","last_undegraded":"2023-10-16T11:24:40.918095+0000","last_fullsized":"2023-10-16T11:24:40.918095+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:17.673425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.936696+0000","last_change":"2023-10-16T11:24:40.947940+0000","last_active":"2023-10-16T11:24:41.936696+0000","last_peered":"2023-10-16T11:24:41.936696+0000","last_clean":"2023-10-16T11:24:41.936696+0000","last_became_active":"2023-10-16T11:24:40.947659+0000","last_became_peered":"2023-10-16T11:24:40.947659+0000","last_unstale":"2023-10-16T11:24:41.936696+0000","last_undegraded":"2023-10-16T11:24:41.936696+0000","last_fullsized":"2023-10-16T11:24:41.936696+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:15:00.136133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.923325+0000","last_change":"2023-10-16T11:24:25.089470+0000","last_active":"2023-10-16T11:24:40.923325+0000","last_peered":"2023-10-16T11:24:40.923325+0000","last_clean":"2023-10-16T11:24:40.923325+0000","last_became_active":"2023-10-16T11:24:25.089213+0000","last_became_peered":"2023-10-16T11:24:25.089213+0000","last_unstale":"2023-10-16T11:24:40.923325+0000","last_undegraded":"2023-10-16T11:24:40.923325+0000","last_fullsized":"2023-10-16T11:24:40.923325+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:34.540849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.907117+0000","last_change":"2023-10-16T11:24:26.079602+0000","last_active":"2023-10-16T11:24:40.907117+0000","last_peered":"2023-10-16T11:24:40.907117+0000","last_clean":"2023-10-16T11:24:40.907117+0000","last_became_active":"2023-10-16T11:24:26.079344+0000","last_became_peered":"2023-10-16T11:24:26.079344+0000","last_unstale":"2023-10-16T11:24:40.907117+0000","last_undegraded":"2023-10-16T11:24:40.907117+0000","last_fullsized":"2023-10-16T11:24:40.907117+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:39.561111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.930050+0000","last_change":"2023-10-16T11:24:39.924572+0000","last_active":"2023-10-16T11:24:40.930050+0000","last_peered":"2023-10-16T11:24:40.930050+0000","last_clean":"2023-10-16T11:24:40.930050+0000","last_became_active":"2023-10-16T11:24:39.924266+0000","last_became_peered":"2023-10-16T11:24:39.924266+0000","last_unstale":"2023-10-16T11:24:40.930050+0000","last_undegraded":"2023-10-16T11:24:40.930050+0000","last_fullsized":"2023-10-16T11:24:40.930050+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:57:57.289628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.077212+0000","last_change":"2023-10-16T11:24:40.938221+0000","last_active":"2023-10-16T11:24:42.077212+0000","last_peered":"2023-10-16T11:24:42.077212+0000","last_clean":"2023-10-16T11:24:42.077212+0000","last_became_active":"2023-10-16T11:24:40.937659+0000","last_became_peered":"2023-10-16T11:24:40.937659+0000","last_unstale":"2023-10-16T11:24:42.077212+0000","last_undegraded":"2023-10-16T11:24:42.077212+0000","last_fullsized":"2023-10-16T11:24:42.077212+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:27.817701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.924919+0000","last_change":"2023-10-16T11:24:25.084194+0000","last_active":"2023-10-16T11:24:40.924919+0000","last_peered":"2023-10-16T11:24:40.924919+0000","last_clean":"2023-10-16T11:24:40.924919+0000","last_became_active":"2023-10-16T11:24:25.083141+0000","last_became_peered":"2023-10-16T11:24:25.083141+0000","last_unstale":"2023-10-16T11:24:40.924919+0000","last_undegraded":"2023-10-16T11:24:40.924919+0000","last_fullsized":"2023-10-16T11:24:40.924919+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:46.770213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.923041+0000","last_change":"2023-10-16T11:24:26.089221+0000","last_active":"2023-10-16T11:24:40.923041+0000","last_peered":"2023-10-16T11:24:40.923041+0000","last_clean":"2023-10-16T11:24:40.923041+0000","last_became_active":"2023-10-16T11:24:26.088644+0000","last_became_peered":"2023-10-16T11:24:26.088644+0000","last_unstale":"2023-10-16T11:24:40.923041+0000","last_undegraded":"2023-10-16T11:24:40.923041+0000","last_fullsized":"2023-10-16T11:24:40.923041+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:10.387018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"220'6","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.948264+0000","last_change":"2023-10-16T11:24:39.904338+0000","last_active":"2023-10-16T11:24:40.948264+0000","last_peered":"2023-10-16T11:24:40.948264+0000","last_clean":"2023-10-16T11:24:40.948264+0000","last_became_active":"2023-10-16T11:24:39.904034+0000","last_became_peered":"2023-10-16T11:24:39.904034+0000","last_unstale":"2023-10-16T11:24:40.948264+0000","last_undegraded":"2023-10-16T11:24:40.948264+0000","last_fullsized":"2023-10-16T11:24:40.948264+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:58.317757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.078231+0000","last_change":"2023-10-16T11:24:40.949476+0000","last_active":"2023-10-16T11:24:42.078231+0000","last_peered":"2023-10-16T11:24:42.078231+0000","last_clean":"2023-10-16T11:24:42.078231+0000","last_became_active":"2023-10-16T11:24:40.949199+0000","last_became_peered":"2023-10-16T11:24:40.949199+0000","last_unstale":"2023-10-16T11:24:42.078231+0000","last_undegraded":"2023-10-16T11:24:42.078231+0000","last_fullsized":"2023-10-16T11:24:42.078231+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:25:21.239825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.13","version":"216'2","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920754+0000","last_change":"2023-10-16T11:24:25.081377+0000","last_active":"2023-10-16T11:24:40.920754+0000","last_peered":"2023-10-16T11:24:40.920754+0000","last_clean":"2023-10-16T11:24:40.920754+0000","last_became_active":"2023-10-16T11:24:25.080881+0000","last_became_peered":"2023-10-16T11:24:25.080881+0000","last_unstale":"2023-10-16T11:24:40.920754+0000","last_undegraded":"2023-10-16T11:24:40.920754+0000","last_fullsized":"2023-10-16T11:24:40.920754+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:26.054702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912668+0000","last_change":"2023-10-16T11:24:26.099039+0000","last_active":"2023-10-16T11:24:40.912668+0000","last_peered":"2023-10-16T11:24:40.912668+0000","last_clean":"2023-10-16T11:24:40.912668+0000","last_became_active":"2023-10-16T11:24:26.097980+0000","last_became_peered":"2023-10-16T11:24:26.097980+0000","last_unstale":"2023-10-16T11:24:40.912668+0000","last_undegraded":"2023-10-16T11:24:40.912668+0000","last_fullsized":"2023-10-16T11:24:40.912668+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:40.585267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"220'3","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.936626+0000","last_change":"2023-10-16T11:24:39.906086+0000","last_active":"2023-10-16T11:24:40.936626+0000","last_peered":"2023-10-16T11:24:40.936626+0000","last_clean":"2023-10-16T11:24:40.936626+0000","last_became_active":"2023-10-16T11:24:39.905781+0000","last_became_peered":"2023-10-16T11:24:39.905781+0000","last_unstale":"2023-10-16T11:24:40.936626+0000","last_undegraded":"2023-10-16T11:24:40.936626+0000","last_fullsized":"2023-10-16T11:24:40.936626+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:07:47.547377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.078344+0000","last_change":"2023-10-16T11:24:40.936652+0000","last_active":"2023-10-16T11:24:42.078344+0000","last_peered":"2023-10-16T11:24:42.078344+0000","last_clean":"2023-10-16T11:24:42.078344+0000","last_became_active":"2023-10-16T11:24:40.934276+0000","last_became_peered":"2023-10-16T11:24:40.934276+0000","last_unstale":"2023-10-16T11:24:42.078344+0000","last_undegraded":"2023-10-16T11:24:42.078344+0000","last_fullsized":"2023-10-16T11:24:42.078344+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:30.742890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.14","version":"217'8","reported_seq":24,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.922693+0000","last_change":"2023-10-16T11:24:25.095357+0000","last_active":"2023-10-16T11:24:40.922693+0000","last_peered":"2023-10-16T11:24:40.922693+0000","last_clean":"2023-10-16T11:24:40.922693+0000","last_became_active":"2023-10-16T11:24:25.093832+0000","last_became_peered":"2023-10-16T11:24:25.093832+0000","last_unstale":"2023-10-16T11:24:40.922693+0000","last_undegraded":"2023-10-16T11:24:40.922693+0000","last_fullsized":"2023-10-16T11:24:40.922693+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:14:19.482469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918256+0000","last_change":"2023-10-16T11:24:26.092185+0000","last_active":"2023-10-16T11:24:40.918256+0000","last_peered":"2023-10-16T11:24:40.918256+0000","last_clean":"2023-10-16T11:24:40.918256+0000","last_became_active":"2023-10-16T11:24:26.091648+0000","last_became_peered":"2023-10-16T11:24:26.091648+0000","last_unstale":"2023-10-16T11:24:40.918256+0000","last_undegraded":"2023-10-16T11:24:40.918256+0000","last_fullsized":"2023-10-16T11:24:40.918256+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:45.547958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918129+0000","last_change":"2023-10-16T11:24:39.919312+0000","last_active":"2023-10-16T11:24:40.918129+0000","last_peered":"2023-10-16T11:24:40.918129+0000","last_clean":"2023-10-16T11:24:40.918129+0000","last_became_active":"2023-10-16T11:24:39.918986+0000","last_became_peered":"2023-10-16T11:24:39.918986+0000","last_unstale":"2023-10-16T11:24:40.918129+0000","last_undegraded":"2023-10-16T11:24:40.918129+0000","last_fullsized":"2023-10-16T11:24:40.918129+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:54.385890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072402+0000","last_change":"2023-10-16T11:24:40.925767+0000","last_active":"2023-10-16T11:24:42.072402+0000","last_peered":"2023-10-16T11:24:42.072402+0000","last_clean":"2023-10-16T11:24:42.072402+0000","last_became_active":"2023-10-16T11:24:40.925501+0000","last_became_peered":"2023-10-16T11:24:40.925501+0000","last_unstale":"2023-10-16T11:24:42.072402+0000","last_undegraded":"2023-10-16T11:24:42.072402+0000","last_fullsized":"2023-10-16T11:24:42.072402+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:49:08.490072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.15","version":"216'3","reported_seq":19,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920628+0000","last_change":"2023-10-16T11:24:25.082968+0000","last_active":"2023-10-16T11:24:40.920628+0000","last_peered":"2023-10-16T11:24:40.920628+0000","last_clean":"2023-10-16T11:24:40.920628+0000","last_became_active":"2023-10-16T11:24:25.082316+0000","last_became_peered":"2023-10-16T11:24:25.082316+0000","last_unstale":"2023-10-16T11:24:40.920628+0000","last_undegraded":"2023-10-16T11:24:40.920628+0000","last_fullsized":"2023-10-16T11:24:40.920628+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:59.924424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.922137+0000","last_change":"2023-10-16T11:24:26.084487+0000","last_active":"2023-10-16T11:24:40.922137+0000","last_peered":"2023-10-16T11:24:40.922137+0000","last_clean":"2023-10-16T11:24:40.922137+0000","last_became_active":"2023-10-16T11:24:26.084188+0000","last_became_peered":"2023-10-16T11:24:26.084188+0000","last_unstale":"2023-10-16T11:24:40.922137+0000","last_undegraded":"2023-10-16T11:24:40.922137+0000","last_fullsized":"2023-10-16T11:24:40.922137+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:47.400027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.945716+0000","last_change":"2023-10-16T11:24:39.928051+0000","last_active":"2023-10-16T11:24:40.945716+0000","last_peered":"2023-10-16T11:24:40.945716+0000","last_clean":"2023-10-16T11:24:40.945716+0000","last_became_active":"2023-10-16T11:24:39.927740+0000","last_became_peered":"2023-10-16T11:24:39.927740+0000","last_unstale":"2023-10-16T11:24:40.945716+0000","last_undegraded":"2023-10-16T11:24:40.945716+0000","last_fullsized":"2023-10-16T11:24:40.945716+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:07.682759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.075589+0000","last_change":"2023-10-16T11:24:40.931397+0000","last_active":"2023-10-16T11:24:42.075589+0000","last_peered":"2023-10-16T11:24:42.075589+0000","last_clean":"2023-10-16T11:24:42.075589+0000","last_became_active":"2023-10-16T11:24:40.931130+0000","last_became_peered":"2023-10-16T11:24:40.931130+0000","last_unstale":"2023-10-16T11:24:42.075589+0000","last_undegraded":"2023-10-16T11:24:42.075589+0000","last_fullsized":"2023-10-16T11:24:42.075589+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:13.011085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.926531+0000","last_change":"2023-10-16T11:24:25.092100+0000","last_active":"2023-10-16T11:24:40.926531+0000","last_peered":"2023-10-16T11:24:40.926531+0000","last_clean":"2023-10-16T11:24:40.926531+0000","last_became_active":"2023-10-16T11:24:25.091237+0000","last_became_peered":"2023-10-16T11:24:25.091237+0000","last_unstale":"2023-10-16T11:24:40.926531+0000","last_undegraded":"2023-10-16T11:24:40.926531+0000","last_fullsized":"2023-10-16T11:24:40.926531+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:21.985839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.909348+0000","last_change":"2023-10-16T11:24:26.081980+0000","last_active":"2023-10-16T11:24:40.909348+0000","last_peered":"2023-10-16T11:24:40.909348+0000","last_clean":"2023-10-16T11:24:40.909348+0000","last_became_active":"2023-10-16T11:24:26.081395+0000","last_became_peered":"2023-10-16T11:24:26.081395+0000","last_unstale":"2023-10-16T11:24:40.909348+0000","last_undegraded":"2023-10-16T11:24:40.909348+0000","last_fullsized":"2023-10-16T11:24:40.909348+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:39.802117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"220'8","reported_seq":20,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.949404+0000","last_change":"2023-10-16T11:24:39.915652+0000","last_active":"2023-10-16T11:24:40.949404+0000","last_peered":"2023-10-16T11:24:40.949404+0000","last_clean":"2023-10-16T11:24:40.949404+0000","last_became_active":"2023-10-16T11:24:39.915277+0000","last_became_peered":"2023-10-16T11:24:39.915277+0000","last_unstale":"2023-10-16T11:24:40.949404+0000","last_undegraded":"2023-10-16T11:24:40.949404+0000","last_fullsized":"2023-10-16T11:24:40.949404+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:49.377967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.080669+0000","last_change":"2023-10-16T11:24:40.938686+0000","last_active":"2023-10-16T11:24:42.080669+0000","last_peered":"2023-10-16T11:24:42.080669+0000","last_clean":"2023-10-16T11:24:42.080669+0000","last_became_active":"2023-10-16T11:24:40.937440+0000","last_became_peered":"2023-10-16T11:24:40.937440+0000","last_unstale":"2023-10-16T11:24:42.080669+0000","last_undegraded":"2023-10-16T11:24:42.080669+0000","last_fullsized":"2023-10-16T11:24:42.080669+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:08:17.799924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913537+0000","last_change":"2023-10-16T11:24:25.094384+0000","last_active":"2023-10-16T11:24:40.913537+0000","last_peered":"2023-10-16T11:24:40.913537+0000","last_clean":"2023-10-16T11:24:40.913537+0000","last_became_active":"2023-10-16T11:24:25.094026+0000","last_became_peered":"2023-10-16T11:24:25.094026+0000","last_unstale":"2023-10-16T11:24:40.913537+0000","last_undegraded":"2023-10-16T11:24:40.913537+0000","last_fullsized":"2023-10-16T11:24:40.913537+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:51.349970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.928344+0000","last_change":"2023-10-16T11:24:26.082915+0000","last_active":"2023-10-16T11:24:40.928344+0000","last_peered":"2023-10-16T11:24:40.928344+0000","last_clean":"2023-10-16T11:24:40.928344+0000","last_became_active":"2023-10-16T11:24:26.082458+0000","last_became_peered":"2023-10-16T11:24:26.082458+0000","last_unstale":"2023-10-16T11:24:40.928344+0000","last_undegraded":"2023-10-16T11:24:40.928344+0000","last_fullsized":"2023-10-16T11:24:40.928344+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:10.387018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913132+0000","last_change":"2023-10-16T11:24:39.903226+0000","last_active":"2023-10-16T11:24:40.913132+0000","last_peered":"2023-10-16T11:24:40.913132+0000","last_clean":"2023-10-16T11:24:40.913132+0000","last_became_active":"2023-10-16T11:24:39.902943+0000","last_became_peered":"2023-10-16T11:24:39.902943+0000","last_unstale":"2023-10-16T11:24:40.913132+0000","last_undegraded":"2023-10-16T11:24:40.913132+0000","last_fullsized":"2023-10-16T11:24:40.913132+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:28:29.998083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.936311+0000","last_change":"2023-10-16T11:24:40.938912+0000","last_active":"2023-10-16T11:24:41.936311+0000","last_peered":"2023-10-16T11:24:41.936311+0000","last_clean":"2023-10-16T11:24:41.936311+0000","last_became_active":"2023-10-16T11:24:40.938615+0000","last_became_peered":"2023-10-16T11:24:40.938615+0000","last_unstale":"2023-10-16T11:24:41.936311+0000","last_undegraded":"2023-10-16T11:24:41.936311+0000","last_fullsized":"2023-10-16T11:24:41.936311+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:38.047063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"220'1","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.947170+0000","last_change":"2023-10-16T11:24:39.916128+0000","last_active":"2023-10-16T11:24:40.947170+0000","last_peered":"2023-10-16T11:24:40.947170+0000","last_clean":"2023-10-16T11:24:40.947170+0000","last_became_active":"2023-10-16T11:24:39.911215+0000","last_became_peered":"2023-10-16T11:24:39.911215+0000","last_unstale":"2023-10-16T11:24:40.947170+0000","last_undegraded":"2023-10-16T11:24:40.947170+0000","last_fullsized":"2023-10-16T11:24:40.947170+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:59:34.809544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927021+0000","last_change":"2023-10-16T11:24:25.084521+0000","last_active":"2023-10-16T11:24:40.927021+0000","last_peered":"2023-10-16T11:24:40.927021+0000","last_clean":"2023-10-16T11:24:40.927021+0000","last_became_active":"2023-10-16T11:24:25.084003+0000","last_became_peered":"2023-10-16T11:24:25.084003+0000","last_unstale":"2023-10-16T11:24:40.927021+0000","last_undegraded":"2023-10-16T11:24:40.927021+0000","last_fullsized":"2023-10-16T11:24:40.927021+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:36.060066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.919781+0000","last_change":"2023-10-16T11:24:26.091641+0000","last_active":"2023-10-16T11:24:40.919781+0000","last_peered":"2023-10-16T11:24:40.919781+0000","last_clean":"2023-10-16T11:24:40.919781+0000","last_became_active":"2023-10-16T11:24:26.089186+0000","last_became_peered":"2023-10-16T11:24:26.089186+0000","last_unstale":"2023-10-16T11:24:40.919781+0000","last_undegraded":"2023-10-16T11:24:40.919781+0000","last_fullsized":"2023-10-16T11:24:40.919781+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:32.247878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.082815+0000","last_change":"2023-10-16T11:24:40.946117+0000","last_active":"2023-10-16T11:24:42.082815+0000","last_peered":"2023-10-16T11:24:42.082815+0000","last_clean":"2023-10-16T11:24:42.082815+0000","last_became_active":"2023-10-16T11:24:40.943815+0000","last_became_peered":"2023-10-16T11:24:40.943815+0000","last_unstale":"2023-10-16T11:24:42.082815+0000","last_undegraded":"2023-10-16T11:24:42.082815+0000","last_fullsized":"2023-10-16T11:24:42.082815+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:08.697558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"220'4","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.947367+0000","last_change":"2023-10-16T11:24:39.915794+0000","last_active":"2023-10-16T11:24:40.947367+0000","last_peered":"2023-10-16T11:24:40.947367+0000","last_clean":"2023-10-16T11:24:40.947367+0000","last_became_active":"2023-10-16T11:24:39.915438+0000","last_became_peered":"2023-10-16T11:24:39.915438+0000","last_unstale":"2023-10-16T11:24:40.947367+0000","last_undegraded":"2023-10-16T11:24:40.947367+0000","last_fullsized":"2023-10-16T11:24:40.947367+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:59.121334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":16,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.913267+0000","last_change":"2023-10-16T11:24:25.074647+0000","last_active":"2023-10-16T11:24:40.913267+0000","last_peered":"2023-10-16T11:24:40.913267+0000","last_clean":"2023-10-16T11:24:40.913267+0000","last_became_active":"2023-10-16T11:24:25.074373+0000","last_became_peered":"2023-10-16T11:24:25.074373+0000","last_unstale":"2023-10-16T11:24:40.913267+0000","last_undegraded":"2023-10-16T11:24:40.913267+0000","last_fullsized":"2023-10-16T11:24:40.913267+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:38.544520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912670+0000","last_change":"2023-10-16T11:24:26.088785+0000","last_active":"2023-10-16T11:24:40.912670+0000","last_peered":"2023-10-16T11:24:40.912670+0000","last_clean":"2023-10-16T11:24:40.912670+0000","last_became_active":"2023-10-16T11:24:26.088521+0000","last_became_peered":"2023-10-16T11:24:26.088521+0000","last_unstale":"2023-10-16T11:24:40.912670+0000","last_undegraded":"2023-10-16T11:24:40.912670+0000","last_fullsized":"2023-10-16T11:24:40.912670+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:29.145795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.078267+0000","last_change":"2023-10-16T11:24:40.950578+0000","last_active":"2023-10-16T11:24:42.078267+0000","last_peered":"2023-10-16T11:24:42.078267+0000","last_clean":"2023-10-16T11:24:42.078267+0000","last_became_active":"2023-10-16T11:24:40.947861+0000","last_became_peered":"2023-10-16T11:24:40.947861+0000","last_unstale":"2023-10-16T11:24:42.078267+0000","last_undegraded":"2023-10-16T11:24:42.078267+0000","last_fullsized":"2023-10-16T11:24:42.078267+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:49:08.490072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1a","version":"216'1","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912820+0000","last_change":"2023-10-16T11:24:25.084983+0000","last_active":"2023-10-16T11:24:40.912820+0000","last_peered":"2023-10-16T11:24:40.912820+0000","last_clean":"2023-10-16T11:24:40.912820+0000","last_became_active":"2023-10-16T11:24:25.084622+0000","last_became_peered":"2023-10-16T11:24:25.084622+0000","last_unstale":"2023-10-16T11:24:40.912820+0000","last_undegraded":"2023-10-16T11:24:40.912820+0000","last_fullsized":"2023-10-16T11:24:40.912820+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:36.588809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.921894+0000","last_change":"2023-10-16T11:24:26.089982+0000","last_active":"2023-10-16T11:24:40.921894+0000","last_peered":"2023-10-16T11:24:40.921894+0000","last_clean":"2023-10-16T11:24:40.921894+0000","last_became_active":"2023-10-16T11:24:26.089677+0000","last_became_peered":"2023-10-16T11:24:26.089677+0000","last_unstale":"2023-10-16T11:24:40.921894+0000","last_undegraded":"2023-10-16T11:24:40.921894+0000","last_fullsized":"2023-10-16T11:24:40.921894+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:22.009042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"220'2","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.940586+0000","last_change":"2023-10-16T11:24:39.914078+0000","last_active":"2023-10-16T11:24:40.940586+0000","last_peered":"2023-10-16T11:24:40.940586+0000","last_clean":"2023-10-16T11:24:40.940586+0000","last_became_active":"2023-10-16T11:24:39.913693+0000","last_became_peered":"2023-10-16T11:24:39.913693+0000","last_unstale":"2023-10-16T11:24:40.940586+0000","last_undegraded":"2023-10-16T11:24:40.940586+0000","last_fullsized":"2023-10-16T11:24:40.940586+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:08:16.785125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.073246+0000","last_change":"2023-10-16T11:24:40.943302+0000","last_active":"2023-10-16T11:24:42.073246+0000","last_peered":"2023-10-16T11:24:42.073246+0000","last_clean":"2023-10-16T11:24:42.073246+0000","last_became_active":"2023-10-16T11:24:40.939215+0000","last_became_peered":"2023-10-16T11:24:40.939215+0000","last_unstale":"2023-10-16T11:24:42.073246+0000","last_undegraded":"2023-10-16T11:24:42.073246+0000","last_fullsized":"2023-10-16T11:24:42.073246+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:22.925745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1c","version":"216'2","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.922233+0000","last_change":"2023-10-16T11:24:25.095179+0000","last_active":"2023-10-16T11:24:40.922233+0000","last_peered":"2023-10-16T11:24:40.922233+0000","last_clean":"2023-10-16T11:24:40.922233+0000","last_became_active":"2023-10-16T11:24:25.094714+0000","last_became_peered":"2023-10-16T11:24:25.094714+0000","last_unstale":"2023-10-16T11:24:40.922233+0000","last_undegraded":"2023-10-16T11:24:40.922233+0000","last_fullsized":"2023-10-16T11:24:40.922233+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:26.128614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920276+0000","last_change":"2023-10-16T11:24:26.096297+0000","last_active":"2023-10-16T11:24:40.920276+0000","last_peered":"2023-10-16T11:24:40.920276+0000","last_clean":"2023-10-16T11:24:40.920276+0000","last_became_active":"2023-10-16T11:24:26.095538+0000","last_became_peered":"2023-10-16T11:24:26.095538+0000","last_unstale":"2023-10-16T11:24:40.920276+0000","last_undegraded":"2023-10-16T11:24:40.920276+0000","last_fullsized":"2023-10-16T11:24:40.920276+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:47.400027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"220'1","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.946397+0000","last_change":"2023-10-16T11:24:39.923043+0000","last_active":"2023-10-16T11:24:40.946397+0000","last_peered":"2023-10-16T11:24:40.946397+0000","last_clean":"2023-10-16T11:24:40.946397+0000","last_became_active":"2023-10-16T11:24:39.922745+0000","last_became_peered":"2023-10-16T11:24:39.922745+0000","last_unstale":"2023-10-16T11:24:40.946397+0000","last_undegraded":"2023-10-16T11:24:40.946397+0000","last_fullsized":"2023-10-16T11:24:40.946397+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:03.774885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.075183+0000","last_change":"2023-10-16T11:24:40.948393+0000","last_active":"2023-10-16T11:24:42.075183+0000","last_peered":"2023-10-16T11:24:42.075183+0000","last_clean":"2023-10-16T11:24:42.075183+0000","last_became_active":"2023-10-16T11:24:40.946737+0000","last_became_peered":"2023-10-16T11:24:40.946737+0000","last_unstale":"2023-10-16T11:24:42.075183+0000","last_undegraded":"2023-10-16T11:24:42.075183+0000","last_fullsized":"2023-10-16T11:24:42.075183+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:49:23.318330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"216'2","reported_seq":18,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.909085+0000","last_change":"2023-10-16T11:24:25.081731+0000","last_active":"2023-10-16T11:24:40.909085+0000","last_peered":"2023-10-16T11:24:40.909085+0000","last_clean":"2023-10-16T11:24:40.909085+0000","last_became_active":"2023-10-16T11:24:25.081336+0000","last_became_peered":"2023-10-16T11:24:25.081336+0000","last_unstale":"2023-10-16T11:24:40.909085+0000","last_undegraded":"2023-10-16T11:24:40.909085+0000","last_fullsized":"2023-10-16T11:24:40.909085+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:09.330801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918704+0000","last_change":"2023-10-16T11:24:26.089791+0000","last_active":"2023-10-16T11:24:40.918704+0000","last_peered":"2023-10-16T11:24:40.918704+0000","last_clean":"2023-10-16T11:24:40.918704+0000","last_became_active":"2023-10-16T11:24:26.089516+0000","last_became_peered":"2023-10-16T11:24:26.089516+0000","last_unstale":"2023-10-16T11:24:40.918704+0000","last_undegraded":"2023-10-16T11:24:40.918704+0000","last_fullsized":"2023-10-16T11:24:40.918704+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"220'1","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.941517+0000","last_change":"2023-10-16T11:24:39.901227+0000","last_active":"2023-10-16T11:24:40.941517+0000","last_peered":"2023-10-16T11:24:40.941517+0000","last_clean":"2023-10-16T11:24:40.941517+0000","last_became_active":"2023-10-16T11:24:39.900852+0000","last_became_peered":"2023-10-16T11:24:39.900852+0000","last_unstale":"2023-10-16T11:24:40.941517+0000","last_undegraded":"2023-10-16T11:24:40.941517+0000","last_fullsized":"2023-10-16T11:24:40.941517+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:54.404259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935060+0000","last_change":"2023-10-16T11:24:40.944861+0000","last_active":"2023-10-16T11:24:41.935060+0000","last_peered":"2023-10-16T11:24:41.935060+0000","last_clean":"2023-10-16T11:24:41.935060+0000","last_became_active":"2023-10-16T11:24:40.944579+0000","last_became_peered":"2023-10-16T11:24:40.944579+0000","last_unstale":"2023-10-16T11:24:41.935060+0000","last_undegraded":"2023-10-16T11:24:41.935060+0000","last_fullsized":"2023-10-16T11:24:41.935060+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:47:20.730005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"216'1","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.933239+0000","last_change":"2023-10-16T11:24:25.094483+0000","last_active":"2023-10-16T11:24:40.933239+0000","last_peered":"2023-10-16T11:24:40.933239+0000","last_clean":"2023-10-16T11:24:40.933239+0000","last_became_active":"2023-10-16T11:24:25.094134+0000","last_became_peered":"2023-10-16T11:24:25.094134+0000","last_unstale":"2023-10-16T11:24:40.933239+0000","last_undegraded":"2023-10-16T11:24:40.933239+0000","last_fullsized":"2023-10-16T11:24:40.933239+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:48.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.923291+0000","last_change":"2023-10-16T11:24:26.094536+0000","last_active":"2023-10-16T11:24:40.923291+0000","last_peered":"2023-10-16T11:24:40.923291+0000","last_clean":"2023-10-16T11:24:40.923291+0000","last_became_active":"2023-10-16T11:24:26.094272+0000","last_became_peered":"2023-10-16T11:24:26.094272+0000","last_unstale":"2023-10-16T11:24:40.923291+0000","last_undegraded":"2023-10-16T11:24:40.923291+0000","last_fullsized":"2023-10-16T11:24:40.923291+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.924102+0000","last_change":"2023-10-16T11:24:39.916827+0000","last_active":"2023-10-16T11:24:40.924102+0000","last_peered":"2023-10-16T11:24:40.924102+0000","last_clean":"2023-10-16T11:24:40.924102+0000","last_became_active":"2023-10-16T11:24:39.916464+0000","last_became_peered":"2023-10-16T11:24:39.916464+0000","last_unstale":"2023-10-16T11:24:40.924102+0000","last_undegraded":"2023-10-16T11:24:40.924102+0000","last_fullsized":"2023-10-16T11:24:40.924102+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:25.891782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.932973+0000","last_change":"2023-10-16T11:24:40.928353+0000","last_active":"2023-10-16T11:24:41.932973+0000","last_peered":"2023-10-16T11:24:41.932973+0000","last_clean":"2023-10-16T11:24:41.932973+0000","last_became_active":"2023-10-16T11:24:40.928042+0000","last_became_peered":"2023-10-16T11:24:40.928042+0000","last_unstale":"2023-10-16T11:24:41.932973+0000","last_undegraded":"2023-10-16T11:24:41.932973+0000","last_fullsized":"2023-10-16T11:24:41.932973+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:57:58.304427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1f","version":"216'4","reported_seq":20,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918030+0000","last_change":"2023-10-16T11:24:25.096644+0000","last_active":"2023-10-16T11:24:40.918030+0000","last_peered":"2023-10-16T11:24:40.918030+0000","last_clean":"2023-10-16T11:24:40.918030+0000","last_became_active":"2023-10-16T11:24:25.096306+0000","last_became_peered":"2023-10-16T11:24:25.096306+0000","last_unstale":"2023-10-16T11:24:40.918030+0000","last_undegraded":"2023-10-16T11:24:40.918030+0000","last_fullsized":"2023-10-16T11:24:40.918030+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:24.027619+0000","last_clean_scrub_stamp":"2023-10-16T11:24:24.027619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:22.618393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.933050+0000","last_change":"2023-10-16T11:24:26.069227+0000","last_active":"2023-10-16T11:24:40.933050+0000","last_peered":"2023-10-16T11:24:40.933050+0000","last_clean":"2023-10-16T11:24:40.933050+0000","last_became_active":"2023-10-16T11:24:26.068456+0000","last_became_peered":"2023-10-16T11:24:26.068456+0000","last_unstale":"2023-10-16T11:24:40.933050+0000","last_undegraded":"2023-10-16T11:24:40.933050+0000","last_fullsized":"2023-10-16T11:24:40.933050+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927501+0000","last_change":"2023-10-16T11:24:39.916911+0000","last_active":"2023-10-16T11:24:40.927501+0000","last_peered":"2023-10-16T11:24:40.927501+0000","last_clean":"2023-10-16T11:24:40.927501+0000","last_became_active":"2023-10-16T11:24:39.916584+0000","last_became_peered":"2023-10-16T11:24:39.916584+0000","last_unstale":"2023-10-16T11:24:40.927501+0000","last_undegraded":"2023-10-16T11:24:40.927501+0000","last_fullsized":"2023-10-16T11:24:40.927501+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:38.863202+0000","last_clean_scrub_stamp":"2023-10-16T11:24:38.863202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:12.120581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935402+0000","last_change":"2023-10-16T11:24:40.929604+0000","last_active":"2023-10-16T11:24:41.935402+0000","last_peered":"2023-10-16T11:24:41.935402+0000","last_clean":"2023-10-16T11:24:41.935402+0000","last_became_active":"2023-10-16T11:24:40.929340+0000","last_became_peered":"2023-10-16T11:24:40.929340+0000","last_unstale":"2023-10-16T11:24:41.935402+0000","last_undegraded":"2023-10-16T11:24:41.935402+0000","last_fullsized":"2023-10-16T11:24:41.935402+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:18.688225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.916906+0000","last_change":"2023-10-16T11:24:26.085375+0000","last_active":"2023-10-16T11:24:40.916906+0000","last_peered":"2023-10-16T11:24:40.916906+0000","last_clean":"2023-10-16T11:24:40.916906+0000","last_became_active":"2023-10-16T11:24:26.085120+0000","last_became_peered":"2023-10-16T11:24:26.085120+0000","last_unstale":"2023-10-16T11:24:40.916906+0000","last_undegraded":"2023-10-16T11:24:40.916906+0000","last_fullsized":"2023-10-16T11:24:40.916906+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.079858+0000","last_change":"2023-10-16T11:24:40.943381+0000","last_active":"2023-10-16T11:24:42.079858+0000","last_peered":"2023-10-16T11:24:42.079858+0000","last_clean":"2023-10-16T11:24:42.079858+0000","last_became_active":"2023-10-16T11:24:40.941771+0000","last_became_peered":"2023-10-16T11:24:40.941771+0000","last_unstale":"2023-10-16T11:24:42.079858+0000","last_undegraded":"2023-10-16T11:24:42.079858+0000","last_fullsized":"2023-10-16T11:24:42.079858+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:30:10.159127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.923208+0000","last_change":"2023-10-16T11:24:26.084964+0000","last_active":"2023-10-16T11:24:40.923208+0000","last_peered":"2023-10-16T11:24:40.923208+0000","last_clean":"2023-10-16T11:24:40.923208+0000","last_became_active":"2023-10-16T11:24:26.084006+0000","last_became_peered":"2023-10-16T11:24:26.084006+0000","last_unstale":"2023-10-16T11:24:40.923208+0000","last_undegraded":"2023-10-16T11:24:40.923208+0000","last_fullsized":"2023-10-16T11:24:40.923208+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:03.949526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.934466+0000","last_change":"2023-10-16T11:24:40.937991+0000","last_active":"2023-10-16T11:24:41.934466+0000","last_peered":"2023-10-16T11:24:41.934466+0000","last_clean":"2023-10-16T11:24:41.934466+0000","last_became_active":"2023-10-16T11:24:40.934656+0000","last_became_peered":"2023-10-16T11:24:40.934656+0000","last_unstale":"2023-10-16T11:24:41.934466+0000","last_undegraded":"2023-10-16T11:24:41.934466+0000","last_fullsized":"2023-10-16T11:24:41.934466+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:49:23.318330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.931374+0000","last_change":"2023-10-16T11:24:26.086008+0000","last_active":"2023-10-16T11:24:40.931374+0000","last_peered":"2023-10-16T11:24:40.931374+0000","last_clean":"2023-10-16T11:24:40.931374+0000","last_became_active":"2023-10-16T11:24:26.085598+0000","last_became_peered":"2023-10-16T11:24:26.085598+0000","last_unstale":"2023-10-16T11:24:40.931374+0000","last_undegraded":"2023-10-16T11:24:40.931374+0000","last_fullsized":"2023-10-16T11:24:40.931374+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.077368+0000","last_change":"2023-10-16T11:24:40.941097+0000","last_active":"2023-10-16T11:24:42.077368+0000","last_peered":"2023-10-16T11:24:42.077368+0000","last_clean":"2023-10-16T11:24:42.077368+0000","last_became_active":"2023-10-16T11:24:40.938428+0000","last_became_peered":"2023-10-16T11:24:40.938428+0000","last_unstale":"2023-10-16T11:24:42.077368+0000","last_undegraded":"2023-10-16T11:24:42.077368+0000","last_fullsized":"2023-10-16T11:24:42.077368+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:12.311557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.915966+0000","last_change":"2023-10-16T11:24:26.086172+0000","last_active":"2023-10-16T11:24:40.915966+0000","last_peered":"2023-10-16T11:24:40.915966+0000","last_clean":"2023-10-16T11:24:40.915966+0000","last_became_active":"2023-10-16T11:24:26.085882+0000","last_became_peered":"2023-10-16T11:24:26.085882+0000","last_unstale":"2023-10-16T11:24:40.915966+0000","last_undegraded":"2023-10-16T11:24:40.915966+0000","last_fullsized":"2023-10-16T11:24:40.915966+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:44.265192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076084+0000","last_change":"2023-10-16T11:24:40.950225+0000","last_active":"2023-10-16T11:24:42.076084+0000","last_peered":"2023-10-16T11:24:42.076084+0000","last_clean":"2023-10-16T11:24:42.076084+0000","last_became_active":"2023-10-16T11:24:40.948659+0000","last_became_peered":"2023-10-16T11:24:40.948659+0000","last_unstale":"2023-10-16T11:24:42.076084+0000","last_undegraded":"2023-10-16T11:24:42.076084+0000","last_fullsized":"2023-10-16T11:24:42.076084+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:46.340170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.907529+0000","last_change":"2023-10-16T11:24:26.086228+0000","last_active":"2023-10-16T11:24:40.907529+0000","last_peered":"2023-10-16T11:24:40.907529+0000","last_clean":"2023-10-16T11:24:40.907529+0000","last_became_active":"2023-10-16T11:24:26.085912+0000","last_became_peered":"2023-10-16T11:24:26.085912+0000","last_unstale":"2023-10-16T11:24:40.907529+0000","last_undegraded":"2023-10-16T11:24:40.907529+0000","last_fullsized":"2023-10-16T11:24:40.907529+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.075408+0000","last_change":"2023-10-16T11:24:40.949598+0000","last_active":"2023-10-16T11:24:42.075408+0000","last_peered":"2023-10-16T11:24:42.075408+0000","last_clean":"2023-10-16T11:24:42.075408+0000","last_became_active":"2023-10-16T11:24:40.949322+0000","last_became_peered":"2023-10-16T11:24:40.949322+0000","last_unstale":"2023-10-16T11:24:42.075408+0000","last_undegraded":"2023-10-16T11:24:42.075408+0000","last_fullsized":"2023-10-16T11:24:42.075408+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:22:53.973191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.915484+0000","last_change":"2023-10-16T11:24:26.088015+0000","last_active":"2023-10-16T11:24:40.915484+0000","last_peered":"2023-10-16T11:24:40.915484+0000","last_clean":"2023-10-16T11:24:40.915484+0000","last_became_active":"2023-10-16T11:24:26.086932+0000","last_became_peered":"2023-10-16T11:24:26.086932+0000","last_unstale":"2023-10-16T11:24:40.915484+0000","last_undegraded":"2023-10-16T11:24:40.915484+0000","last_fullsized":"2023-10-16T11:24:40.915484+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:44.137821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.071053+0000","last_change":"2023-10-16T11:24:40.935550+0000","last_active":"2023-10-16T11:24:42.071053+0000","last_peered":"2023-10-16T11:24:42.071053+0000","last_clean":"2023-10-16T11:24:42.071053+0000","last_became_active":"2023-10-16T11:24:40.935237+0000","last_became_peered":"2023-10-16T11:24:40.935237+0000","last_unstale":"2023-10-16T11:24:42.071053+0000","last_undegraded":"2023-10-16T11:24:42.071053+0000","last_fullsized":"2023-10-16T11:24:42.071053+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:13.011085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.918841+0000","last_change":"2023-10-16T11:24:26.095913+0000","last_active":"2023-10-16T11:24:40.918841+0000","last_peered":"2023-10-16T11:24:40.918841+0000","last_clean":"2023-10-16T11:24:40.918841+0000","last_became_active":"2023-10-16T11:24:26.093113+0000","last_became_peered":"2023-10-16T11:24:26.093113+0000","last_unstale":"2023-10-16T11:24:40.918841+0000","last_undegraded":"2023-10-16T11:24:40.918841+0000","last_fullsized":"2023-10-16T11:24:40.918841+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072486+0000","last_change":"2023-10-16T11:24:40.932619+0000","last_active":"2023-10-16T11:24:42.072486+0000","last_peered":"2023-10-16T11:24:42.072486+0000","last_clean":"2023-10-16T11:24:42.072486+0000","last_became_active":"2023-10-16T11:24:40.930562+0000","last_became_peered":"2023-10-16T11:24:40.930562+0000","last_unstale":"2023-10-16T11:24:42.072486+0000","last_undegraded":"2023-10-16T11:24:42.072486+0000","last_fullsized":"2023-10-16T11:24:42.072486+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:11.184845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.932735+0000","last_change":"2023-10-16T11:24:26.094066+0000","last_active":"2023-10-16T11:24:40.932735+0000","last_peered":"2023-10-16T11:24:40.932735+0000","last_clean":"2023-10-16T11:24:40.932735+0000","last_became_active":"2023-10-16T11:24:26.093428+0000","last_became_peered":"2023-10-16T11:24:26.093428+0000","last_unstale":"2023-10-16T11:24:40.932735+0000","last_undegraded":"2023-10-16T11:24:40.932735+0000","last_fullsized":"2023-10-16T11:24:40.932735+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:29.568355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935302+0000","last_change":"2023-10-16T11:24:40.944382+0000","last_active":"2023-10-16T11:24:41.935302+0000","last_peered":"2023-10-16T11:24:41.935302+0000","last_clean":"2023-10-16T11:24:41.935302+0000","last_became_active":"2023-10-16T11:24:40.943690+0000","last_became_peered":"2023-10-16T11:24:40.943690+0000","last_unstale":"2023-10-16T11:24:41.935302+0000","last_undegraded":"2023-10-16T11:24:41.935302+0000","last_fullsized":"2023-10-16T11:24:41.935302+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:15.346746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.921063+0000","last_change":"2023-10-16T11:24:26.091894+0000","last_active":"2023-10-16T11:24:40.921063+0000","last_peered":"2023-10-16T11:24:40.921063+0000","last_clean":"2023-10-16T11:24:40.921063+0000","last_became_active":"2023-10-16T11:24:26.091627+0000","last_became_peered":"2023-10-16T11:24:26.091627+0000","last_unstale":"2023-10-16T11:24:40.921063+0000","last_undegraded":"2023-10-16T11:24:40.921063+0000","last_fullsized":"2023-10-16T11:24:40.921063+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:46:59.832609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.081684+0000","last_change":"2023-10-16T11:24:40.941368+0000","last_active":"2023-10-16T11:24:42.081684+0000","last_peered":"2023-10-16T11:24:42.081684+0000","last_clean":"2023-10-16T11:24:42.081684+0000","last_became_active":"2023-10-16T11:24:40.940598+0000","last_became_peered":"2023-10-16T11:24:40.940598+0000","last_unstale":"2023-10-16T11:24:42.081684+0000","last_undegraded":"2023-10-16T11:24:42.081684+0000","last_fullsized":"2023-10-16T11:24:42.081684+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:50:47.255058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927515+0000","last_change":"2023-10-16T11:24:26.097221+0000","last_active":"2023-10-16T11:24:40.927515+0000","last_peered":"2023-10-16T11:24:40.927515+0000","last_clean":"2023-10-16T11:24:40.927515+0000","last_became_active":"2023-10-16T11:24:26.096572+0000","last_became_peered":"2023-10-16T11:24:26.096572+0000","last_unstale":"2023-10-16T11:24:40.927515+0000","last_undegraded":"2023-10-16T11:24:40.927515+0000","last_fullsized":"2023-10-16T11:24:40.927515+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.919162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.934384+0000","last_change":"2023-10-16T11:24:40.935022+0000","last_active":"2023-10-16T11:24:41.934384+0000","last_peered":"2023-10-16T11:24:41.934384+0000","last_clean":"2023-10-16T11:24:41.934384+0000","last_became_active":"2023-10-16T11:24:40.934325+0000","last_became_peered":"2023-10-16T11:24:40.934325+0000","last_unstale":"2023-10-16T11:24:41.934384+0000","last_undegraded":"2023-10-16T11:24:41.934384+0000","last_fullsized":"2023-10-16T11:24:41.934384+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:58.258235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.924840+0000","last_change":"2023-10-16T11:24:26.101039+0000","last_active":"2023-10-16T11:24:40.924840+0000","last_peered":"2023-10-16T11:24:40.924840+0000","last_clean":"2023-10-16T11:24:40.924840+0000","last_became_active":"2023-10-16T11:24:26.100757+0000","last_became_peered":"2023-10-16T11:24:26.100757+0000","last_unstale":"2023-10-16T11:24:40.924840+0000","last_undegraded":"2023-10-16T11:24:40.924840+0000","last_fullsized":"2023-10-16T11:24:40.924840+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.073731+0000","last_change":"2023-10-16T11:24:40.934747+0000","last_active":"2023-10-16T11:24:42.073731+0000","last_peered":"2023-10-16T11:24:42.073731+0000","last_clean":"2023-10-16T11:24:42.073731+0000","last_became_active":"2023-10-16T11:24:40.934319+0000","last_became_peered":"2023-10-16T11:24:40.934319+0000","last_unstale":"2023-10-16T11:24:42.073731+0000","last_undegraded":"2023-10-16T11:24:42.073731+0000","last_fullsized":"2023-10-16T11:24:42.073731+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:12:33.440509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.921807+0000","last_change":"2023-10-16T11:24:26.092718+0000","last_active":"2023-10-16T11:24:40.921807+0000","last_peered":"2023-10-16T11:24:40.921807+0000","last_clean":"2023-10-16T11:24:40.921807+0000","last_became_active":"2023-10-16T11:24:26.091377+0000","last_became_peered":"2023-10-16T11:24:26.091377+0000","last_unstale":"2023-10-16T11:24:40.921807+0000","last_undegraded":"2023-10-16T11:24:40.921807+0000","last_fullsized":"2023-10-16T11:24:40.921807+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:17.072626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072172+0000","last_change":"2023-10-16T11:24:40.933022+0000","last_active":"2023-10-16T11:24:42.072172+0000","last_peered":"2023-10-16T11:24:42.072172+0000","last_clean":"2023-10-16T11:24:42.072172+0000","last_became_active":"2023-10-16T11:24:40.932746+0000","last_became_peered":"2023-10-16T11:24:40.932746+0000","last_unstale":"2023-10-16T11:24:42.072172+0000","last_undegraded":"2023-10-16T11:24:42.072172+0000","last_fullsized":"2023-10-16T11:24:42.072172+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:42.730933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.915755+0000","last_change":"2023-10-16T11:24:26.093168+0000","last_active":"2023-10-16T11:24:40.915755+0000","last_peered":"2023-10-16T11:24:40.915755+0000","last_clean":"2023-10-16T11:24:40.915755+0000","last_became_active":"2023-10-16T11:24:26.091801+0000","last_became_peered":"2023-10-16T11:24:26.091801+0000","last_unstale":"2023-10-16T11:24:40.915755+0000","last_undegraded":"2023-10-16T11:24:40.915755+0000","last_fullsized":"2023-10-16T11:24:40.915755+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:43:28.275746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072745+0000","last_change":"2023-10-16T11:24:40.934530+0000","last_active":"2023-10-16T11:24:42.072745+0000","last_peered":"2023-10-16T11:24:42.072745+0000","last_clean":"2023-10-16T11:24:42.072745+0000","last_became_active":"2023-10-16T11:24:40.934262+0000","last_became_peered":"2023-10-16T11:24:40.934262+0000","last_unstale":"2023-10-16T11:24:42.072745+0000","last_undegraded":"2023-10-16T11:24:42.072745+0000","last_fullsized":"2023-10-16T11:24:42.072745+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:30:07.384687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.920507+0000","last_change":"2023-10-16T11:24:26.086807+0000","last_active":"2023-10-16T11:24:40.920507+0000","last_peered":"2023-10-16T11:24:40.920507+0000","last_clean":"2023-10-16T11:24:40.920507+0000","last_became_active":"2023-10-16T11:24:26.085382+0000","last_became_peered":"2023-10-16T11:24:26.085382+0000","last_unstale":"2023-10-16T11:24:40.920507+0000","last_undegraded":"2023-10-16T11:24:40.920507+0000","last_fullsized":"2023-10-16T11:24:40.920507+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:56.195463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.071952+0000","last_change":"2023-10-16T11:24:40.931627+0000","last_active":"2023-10-16T11:24:42.071952+0000","last_peered":"2023-10-16T11:24:42.071952+0000","last_clean":"2023-10-16T11:24:42.071952+0000","last_became_active":"2023-10-16T11:24:40.931219+0000","last_became_peered":"2023-10-16T11:24:40.931219+0000","last_unstale":"2023-10-16T11:24:42.071952+0000","last_undegraded":"2023-10-16T11:24:42.071952+0000","last_fullsized":"2023-10-16T11:24:42.071952+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:04.153171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927380+0000","last_change":"2023-10-16T11:24:26.092139+0000","last_active":"2023-10-16T11:24:40.927380+0000","last_peered":"2023-10-16T11:24:40.927380+0000","last_clean":"2023-10-16T11:24:40.927380+0000","last_became_active":"2023-10-16T11:24:26.091860+0000","last_became_peered":"2023-10-16T11:24:26.091860+0000","last_unstale":"2023-10-16T11:24:40.927380+0000","last_undegraded":"2023-10-16T11:24:40.927380+0000","last_fullsized":"2023-10-16T11:24:40.927380+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:00:31.371733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076871+0000","last_change":"2023-10-16T11:24:40.939792+0000","last_active":"2023-10-16T11:24:42.076871+0000","last_peered":"2023-10-16T11:24:42.076871+0000","last_clean":"2023-10-16T11:24:42.076871+0000","last_became_active":"2023-10-16T11:24:40.939547+0000","last_became_peered":"2023-10-16T11:24:40.939547+0000","last_unstale":"2023-10-16T11:24:42.076871+0000","last_undegraded":"2023-10-16T11:24:42.076871+0000","last_fullsized":"2023-10-16T11:24:42.076871+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:44:48.635235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.919569+0000","last_change":"2023-10-16T11:24:26.086029+0000","last_active":"2023-10-16T11:24:40.919569+0000","last_peered":"2023-10-16T11:24:40.919569+0000","last_clean":"2023-10-16T11:24:40.919569+0000","last_became_active":"2023-10-16T11:24:26.085644+0000","last_became_peered":"2023-10-16T11:24:26.085644+0000","last_unstale":"2023-10-16T11:24:40.919569+0000","last_undegraded":"2023-10-16T11:24:40.919569+0000","last_fullsized":"2023-10-16T11:24:40.919569+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:39.802117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.934967+0000","last_change":"2023-10-16T11:24:40.940881+0000","last_active":"2023-10-16T11:24:41.934967+0000","last_peered":"2023-10-16T11:24:41.934967+0000","last_clean":"2023-10-16T11:24:41.934967+0000","last_became_active":"2023-10-16T11:24:40.940465+0000","last_became_peered":"2023-10-16T11:24:40.940465+0000","last_unstale":"2023-10-16T11:24:41.934967+0000","last_undegraded":"2023-10-16T11:24:41.934967+0000","last_fullsized":"2023-10-16T11:24:41.934967+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:39:05.461904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.924194+0000","last_change":"2023-10-16T11:24:26.081938+0000","last_active":"2023-10-16T11:24:40.924194+0000","last_peered":"2023-10-16T11:24:40.924194+0000","last_clean":"2023-10-16T11:24:40.924194+0000","last_became_active":"2023-10-16T11:24:26.081671+0000","last_became_peered":"2023-10-16T11:24:26.081671+0000","last_unstale":"2023-10-16T11:24:40.924194+0000","last_undegraded":"2023-10-16T11:24:40.924194+0000","last_fullsized":"2023-10-16T11:24:40.924194+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:17.072626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.080281+0000","last_change":"2023-10-16T11:24:40.944207+0000","last_active":"2023-10-16T11:24:42.080281+0000","last_peered":"2023-10-16T11:24:42.080281+0000","last_clean":"2023-10-16T11:24:42.080281+0000","last_became_active":"2023-10-16T11:24:40.943832+0000","last_became_peered":"2023-10-16T11:24:40.943832+0000","last_unstale":"2023-10-16T11:24:42.080281+0000","last_undegraded":"2023-10-16T11:24:42.080281+0000","last_fullsized":"2023-10-16T11:24:42.080281+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:43.979587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917335+0000","last_change":"2023-10-16T11:24:26.091403+0000","last_active":"2023-10-16T11:24:40.917335+0000","last_peered":"2023-10-16T11:24:40.917335+0000","last_clean":"2023-10-16T11:24:40.917335+0000","last_became_active":"2023-10-16T11:24:26.091114+0000","last_became_peered":"2023-10-16T11:24:26.091114+0000","last_unstale":"2023-10-16T11:24:40.917335+0000","last_undegraded":"2023-10-16T11:24:40.917335+0000","last_fullsized":"2023-10-16T11:24:40.917335+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.347392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935841+0000","last_change":"2023-10-16T11:24:40.936062+0000","last_active":"2023-10-16T11:24:41.935841+0000","last_peered":"2023-10-16T11:24:41.935841+0000","last_clean":"2023-10-16T11:24:41.935841+0000","last_became_active":"2023-10-16T11:24:40.935597+0000","last_became_peered":"2023-10-16T11:24:40.935597+0000","last_unstale":"2023-10-16T11:24:41.935841+0000","last_undegraded":"2023-10-16T11:24:41.935841+0000","last_fullsized":"2023-10-16T11:24:41.935841+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:55.400689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917983+0000","last_change":"2023-10-16T11:24:26.088199+0000","last_active":"2023-10-16T11:24:40.917983+0000","last_peered":"2023-10-16T11:24:40.917983+0000","last_clean":"2023-10-16T11:24:40.917983+0000","last_became_active":"2023-10-16T11:24:26.087892+0000","last_became_peered":"2023-10-16T11:24:26.087892+0000","last_unstale":"2023-10-16T11:24:40.917983+0000","last_undegraded":"2023-10-16T11:24:40.917983+0000","last_fullsized":"2023-10-16T11:24:40.917983+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076815+0000","last_change":"2023-10-16T11:24:40.935695+0000","last_active":"2023-10-16T11:24:42.076815+0000","last_peered":"2023-10-16T11:24:42.076815+0000","last_clean":"2023-10-16T11:24:42.076815+0000","last_became_active":"2023-10-16T11:24:40.935358+0000","last_became_peered":"2023-10-16T11:24:40.935358+0000","last_unstale":"2023-10-16T11:24:42.076815+0000","last_undegraded":"2023-10-16T11:24:42.076815+0000","last_fullsized":"2023-10-16T11:24:42.076815+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:15.346746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.916085+0000","last_change":"2023-10-16T11:24:26.088938+0000","last_active":"2023-10-16T11:24:40.916085+0000","last_peered":"2023-10-16T11:24:40.916085+0000","last_clean":"2023-10-16T11:24:40.916085+0000","last_became_active":"2023-10-16T11:24:26.088689+0000","last_became_peered":"2023-10-16T11:24:26.088689+0000","last_unstale":"2023-10-16T11:24:40.916085+0000","last_undegraded":"2023-10-16T11:24:40.916085+0000","last_fullsized":"2023-10-16T11:24:40.916085+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:18:12.479766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.076896+0000","last_change":"2023-10-16T11:24:40.941713+0000","last_active":"2023-10-16T11:24:42.076896+0000","last_peered":"2023-10-16T11:24:42.076896+0000","last_clean":"2023-10-16T11:24:42.076896+0000","last_became_active":"2023-10-16T11:24:40.938875+0000","last_became_peered":"2023-10-16T11:24:40.938875+0000","last_unstale":"2023-10-16T11:24:42.076896+0000","last_undegraded":"2023-10-16T11:24:42.076896+0000","last_fullsized":"2023-10-16T11:24:42.076896+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:42.730933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.915139+0000","last_change":"2023-10-16T11:24:26.089768+0000","last_active":"2023-10-16T11:24:40.915139+0000","last_peered":"2023-10-16T11:24:40.915139+0000","last_clean":"2023-10-16T11:24:40.915139+0000","last_became_active":"2023-10-16T11:24:26.087028+0000","last_became_peered":"2023-10-16T11:24:26.087028+0000","last_unstale":"2023-10-16T11:24:40.915139+0000","last_undegraded":"2023-10-16T11:24:40.915139+0000","last_fullsized":"2023-10-16T11:24:40.915139+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:33.050118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935632+0000","last_change":"2023-10-16T11:24:40.931431+0000","last_active":"2023-10-16T11:24:41.935632+0000","last_peered":"2023-10-16T11:24:41.935632+0000","last_clean":"2023-10-16T11:24:41.935632+0000","last_became_active":"2023-10-16T11:24:40.930331+0000","last_became_peered":"2023-10-16T11:24:40.930331+0000","last_unstale":"2023-10-16T11:24:41.935632+0000","last_undegraded":"2023-10-16T11:24:41.935632+0000","last_fullsized":"2023-10-16T11:24:41.935632+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:18.688225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.908959+0000","last_change":"2023-10-16T11:24:26.077430+0000","last_active":"2023-10-16T11:24:40.908959+0000","last_peered":"2023-10-16T11:24:40.908959+0000","last_clean":"2023-10-16T11:24:40.908959+0000","last_became_active":"2023-10-16T11:24:26.077145+0000","last_became_peered":"2023-10-16T11:24:26.077145+0000","last_unstale":"2023-10-16T11:24:40.908959+0000","last_undegraded":"2023-10-16T11:24:40.908959+0000","last_fullsized":"2023-10-16T11:24:40.908959+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:06:52.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.935229+0000","last_change":"2023-10-16T11:24:40.932225+0000","last_active":"2023-10-16T11:24:41.935229+0000","last_peered":"2023-10-16T11:24:41.935229+0000","last_clean":"2023-10-16T11:24:41.935229+0000","last_became_active":"2023-10-16T11:24:40.931846+0000","last_became_peered":"2023-10-16T11:24:40.931846+0000","last_unstale":"2023-10-16T11:24:41.935229+0000","last_undegraded":"2023-10-16T11:24:41.935229+0000","last_fullsized":"2023-10-16T11:24:41.935229+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:23.266881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.927275+0000","last_change":"2023-10-16T11:24:26.088202+0000","last_active":"2023-10-16T11:24:40.927275+0000","last_peered":"2023-10-16T11:24:40.927275+0000","last_clean":"2023-10-16T11:24:40.927275+0000","last_became_active":"2023-10-16T11:24:26.087929+0000","last_became_peered":"2023-10-16T11:24:26.087929+0000","last_unstale":"2023-10-16T11:24:40.927275+0000","last_undegraded":"2023-10-16T11:24:40.927275+0000","last_fullsized":"2023-10-16T11:24:40.927275+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:22.009042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.070991+0000","last_change":"2023-10-16T11:24:40.941405+0000","last_active":"2023-10-16T11:24:42.070991+0000","last_peered":"2023-10-16T11:24:42.070991+0000","last_clean":"2023-10-16T11:24:42.070991+0000","last_became_active":"2023-10-16T11:24:40.941060+0000","last_became_peered":"2023-10-16T11:24:40.941060+0000","last_unstale":"2023-10-16T11:24:42.070991+0000","last_undegraded":"2023-10-16T11:24:42.070991+0000","last_fullsized":"2023-10-16T11:24:42.070991+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:22:53.973191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.933068+0000","last_change":"2023-10-16T11:24:26.080292+0000","last_active":"2023-10-16T11:24:40.933068+0000","last_peered":"2023-10-16T11:24:40.933068+0000","last_clean":"2023-10-16T11:24:40.933068+0000","last_became_active":"2023-10-16T11:24:26.080004+0000","last_became_peered":"2023-10-16T11:24:26.080004+0000","last_unstale":"2023-10-16T11:24:40.933068+0000","last_undegraded":"2023-10-16T11:24:40.933068+0000","last_fullsized":"2023-10-16T11:24:40.933068+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.936334+0000","last_change":"2023-10-16T11:24:40.946129+0000","last_active":"2023-10-16T11:24:41.936334+0000","last_peered":"2023-10-16T11:24:41.936334+0000","last_clean":"2023-10-16T11:24:41.936334+0000","last_became_active":"2023-10-16T11:24:40.944566+0000","last_became_peered":"2023-10-16T11:24:40.944566+0000","last_unstale":"2023-10-16T11:24:41.936334+0000","last_undegraded":"2023-10-16T11:24:41.936334+0000","last_fullsized":"2023-10-16T11:24:41.936334+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:41:38.047063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912578+0000","last_change":"2023-10-16T11:24:26.098845+0000","last_active":"2023-10-16T11:24:40.912578+0000","last_peered":"2023-10-16T11:24:40.912578+0000","last_clean":"2023-10-16T11:24:40.912578+0000","last_became_active":"2023-10-16T11:24:26.098568+0000","last_became_peered":"2023-10-16T11:24:26.098568+0000","last_unstale":"2023-10-16T11:24:40.912578+0000","last_undegraded":"2023-10-16T11:24:40.912578+0000","last_fullsized":"2023-10-16T11:24:40.912578+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:15.909098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3f","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.079804+0000","last_change":"2023-10-16T11:24:40.942060+0000","last_active":"2023-10-16T11:24:42.079804+0000","last_peered":"2023-10-16T11:24:42.079804+0000","last_clean":"2023-10-16T11:24:42.079804+0000","last_became_active":"2023-10-16T11:24:40.941339+0000","last_became_peered":"2023-10-16T11:24:40.941339+0000","last_unstale":"2023-10-16T11:24:42.079804+0000","last_undegraded":"2023-10-16T11:24:42.079804+0000","last_fullsized":"2023-10-16T11:24:42.079804+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:44:34.145225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.917233+0000","last_change":"2023-10-16T11:24:26.096348+0000","last_active":"2023-10-16T11:24:40.917233+0000","last_peered":"2023-10-16T11:24:40.917233+0000","last_clean":"2023-10-16T11:24:40.917233+0000","last_became_active":"2023-10-16T11:24:26.093712+0000","last_became_peered":"2023-10-16T11:24:26.093712+0000","last_unstale":"2023-10-16T11:24:40.917233+0000","last_undegraded":"2023-10-16T11:24:40.917233+0000","last_fullsized":"2023-10-16T11:24:40.917233+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:40.970357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.073372+0000","last_change":"2023-10-16T11:24:40.942740+0000","last_active":"2023-10-16T11:24:42.073372+0000","last_peered":"2023-10-16T11:24:42.073372+0000","last_clean":"2023-10-16T11:24:42.073372+0000","last_became_active":"2023-10-16T11:24:40.939522+0000","last_became_peered":"2023-10-16T11:24:40.939522+0000","last_unstale":"2023-10-16T11:24:42.073372+0000","last_undegraded":"2023-10-16T11:24:42.073372+0000","last_fullsized":"2023-10-16T11:24:42.073372+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:31:23.849707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.912900+0000","last_change":"2023-10-16T11:24:26.083687+0000","last_active":"2023-10-16T11:24:40.912900+0000","last_peered":"2023-10-16T11:24:40.912900+0000","last_clean":"2023-10-16T11:24:40.912900+0000","last_became_active":"2023-10-16T11:24:26.083213+0000","last_became_peered":"2023-10-16T11:24:26.083213+0000","last_unstale":"2023-10-16T11:24:40.912900+0000","last_undegraded":"2023-10-16T11:24:40.912900+0000","last_fullsized":"2023-10-16T11:24:40.912900+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:08:02.966132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.081257+0000","last_change":"2023-10-16T11:24:40.917853+0000","last_active":"2023-10-16T11:24:42.081257+0000","last_peered":"2023-10-16T11:24:42.081257+0000","last_clean":"2023-10-16T11:24:42.081257+0000","last_became_active":"2023-10-16T11:24:40.917561+0000","last_became_peered":"2023-10-16T11:24:40.917561+0000","last_unstale":"2023-10-16T11:24:42.081257+0000","last_undegraded":"2023-10-16T11:24:42.081257+0000","last_fullsized":"2023-10-16T11:24:42.081257+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:55.419058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.914944+0000","last_change":"2023-10-16T11:24:26.096814+0000","last_active":"2023-10-16T11:24:40.914944+0000","last_peered":"2023-10-16T11:24:40.914944+0000","last_clean":"2023-10-16T11:24:40.914944+0000","last_became_active":"2023-10-16T11:24:26.095777+0000","last_became_peered":"2023-10-16T11:24:26.095777+0000","last_unstale":"2023-10-16T11:24:40.914944+0000","last_undegraded":"2023-10-16T11:24:40.914944+0000","last_fullsized":"2023-10-16T11:24:40.914944+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:08.889514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.933544+0000","last_change":"2023-10-16T11:24:40.935780+0000","last_active":"2023-10-16T11:24:41.933544+0000","last_peered":"2023-10-16T11:24:41.933544+0000","last_clean":"2023-10-16T11:24:41.933544+0000","last_became_active":"2023-10-16T11:24:40.935476+0000","last_became_peered":"2023-10-16T11:24:40.935476+0000","last_unstale":"2023-10-16T11:24:41.933544+0000","last_undegraded":"2023-10-16T11:24:41.933544+0000","last_fullsized":"2023-10-16T11:24:41.933544+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:54:11.551791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.922006+0000","last_change":"2023-10-16T11:24:26.091440+0000","last_active":"2023-10-16T11:24:40.922006+0000","last_peered":"2023-10-16T11:24:40.922006+0000","last_clean":"2023-10-16T11:24:40.922006+0000","last_became_active":"2023-10-16T11:24:26.088757+0000","last_became_peered":"2023-10-16T11:24:26.088757+0000","last_unstale":"2023-10-16T11:24:40.922006+0000","last_undegraded":"2023-10-16T11:24:40.922006+0000","last_fullsized":"2023-10-16T11:24:40.922006+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:27.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.072970+0000","last_change":"2023-10-16T11:24:40.933226+0000","last_active":"2023-10-16T11:24:42.072970+0000","last_peered":"2023-10-16T11:24:42.072970+0000","last_clean":"2023-10-16T11:24:42.072970+0000","last_became_active":"2023-10-16T11:24:40.932168+0000","last_became_peered":"2023-10-16T11:24:40.932168+0000","last_unstale":"2023-10-16T11:24:42.072970+0000","last_undegraded":"2023-10-16T11:24:42.072970+0000","last_fullsized":"2023-10-16T11:24:42.072970+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:51:44.814417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.908841+0000","last_change":"2023-10-16T11:24:26.085606+0000","last_active":"2023-10-16T11:24:40.908841+0000","last_peered":"2023-10-16T11:24:40.908841+0000","last_clean":"2023-10-16T11:24:40.908841+0000","last_became_active":"2023-10-16T11:24:26.085318+0000","last_became_peered":"2023-10-16T11:24:26.085318+0000","last_unstale":"2023-10-16T11:24:40.908841+0000","last_undegraded":"2023-10-16T11:24:40.908841+0000","last_fullsized":"2023-10-16T11:24:40.908841+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:47.567020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:41.931913+0000","last_change":"2023-10-16T11:24:40.937629+0000","last_active":"2023-10-16T11:24:41.931913+0000","last_peered":"2023-10-16T11:24:41.931913+0000","last_clean":"2023-10-16T11:24:41.931913+0000","last_became_active":"2023-10-16T11:24:40.937340+0000","last_became_peered":"2023-10-16T11:24:40.937340+0000","last_unstale":"2023-10-16T11:24:41.931913+0000","last_undegraded":"2023-10-16T11:24:41.931913+0000","last_fullsized":"2023-10-16T11:24:41.931913+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:13.135380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.932956+0000","last_change":"2023-10-16T11:24:26.089885+0000","last_active":"2023-10-16T11:24:40.932956+0000","last_peered":"2023-10-16T11:24:40.932956+0000","last_clean":"2023-10-16T11:24:40.932956+0000","last_became_active":"2023-10-16T11:24:26.089406+0000","last_became_peered":"2023-10-16T11:24:26.089406+0000","last_unstale":"2023-10-16T11:24:40.932956+0000","last_undegraded":"2023-10-16T11:24:40.932956+0000","last_fullsized":"2023-10-16T11:24:40.932956+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:33.272969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-16T11:24:42.075155+0000","last_change":"2023-10-16T11:24:40.926983+0000","last_active":"2023-10-16T11:24:42.075155+0000","last_peered":"2023-10-16T11:24:42.075155+0000","last_clean":"2023-10-16T11:24:42.075155+0000","last_became_active":"2023-10-16T11:24:40.923594+0000","last_became_peered":"2023-10-16T11:24:40.923594+0000","last_unstale":"2023-10-16T11:24:42.075155+0000","last_undegraded":"2023-10-16T11:24:42.075155+0000","last_fullsized":"2023-10-16T11:24:42.075155+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:39.878001+0000","last_clean_scrub_stamp":"2023-10-16T11:24:39.878001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:54.635909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":15,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:24:40.919488+0000","last_change":"2023-10-16T11:24:26.086488+0000","last_active":"2023-10-16T11:24:40.919488+0000","last_peered":"2023-10-16T11:24:40.919488+0000","last_clean":"2023-10-16T11:24:40.919488+0000","last_became_active":"2023-10-16T11:24:26.085969+0000","last_became_peered":"2023-10-16T11:24:26.085969+0000","last_unstale":"2023-10-16T11:24:40.919488+0000","last_undegraded":"2023-10-16T11:24:40.919488+0000","last_fullsized":"2023-10-16T11:24:40.919488+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:24:25.044210+0000","last_clean_scrub_stamp":"2023-10-16T11:24:25.044210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:37.179253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2176,"num_read_kb":4468,"num_write":581,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2767,"ondisk_log_size":2767,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575990,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52532,"kb_used_data":3304,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319308,"statfs":{"total":96636764160,"available":96582971392,"internally_reserved":0,"allocated":3383296,"data_stored":1942477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.179,"15min":1.2170000000000001},"min":{"1min":0.81499999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.7250000000000001,"5min":2.085,"15min":2.3399999999999999},"last":1.365},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.1719999999999999,"15min":1.218},"min":{"1min":0.876,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.8320000000000001},"last":0.89700000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2689999999999999,"15min":1.2509999999999999},"min":{"1min":0.90700000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.6479999999999999,"5min":1.913,"15min":1.913},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2430000000000001,"15min":1.2749999999999999},"min":{"1min":0.76700000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6279999999999999,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.4550000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3140000000000001,"15min":1.3109999999999999},"min":{"1min":1.008,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6579999999999999,"5min":1.8680000000000001,"15min":2.4460000000000002},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.343,"5min":1.3540000000000001,"15min":1.3640000000000001},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.708,"5min":1.859,"15min":1.974},"last":1.325}]},{"osd":3,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.381,"15min":1.3899999999999999},"min":{"1min":1.149,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7290000000000001,"5min":1.8280000000000001,"15min":1.972},"last":1.53},{"interface":"front","average":{"1min":1.45,"5min":1.4430000000000001,"15min":1.4630000000000001},"min":{"1min":1.0860000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.8799999999999999,"5min":2.0680000000000001,"15min":2.577},"last":1.3}]},{"osd":4,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.423,"15min":1.417},"min":{"1min":0.98599999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0030000000000001,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.044},{"interface":"front","average":{"1min":1.498,"5min":1.391,"15min":1.3899999999999999},"min":{"1min":1.0469999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8280000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.399}]},{"osd":5,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.585,"15min":1.5960000000000001},"min":{"1min":1.1579999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.946,"5min":2.2309999999999999,"15min":3.3519999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.641,"5min":1.514,"15min":1.5089999999999999},"min":{"1min":1.351,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.1600000000000001,"5min":2.3460000000000001,"15min":2.4969999999999999},"last":1.5580000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:23:52 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.488,"15min":1.5209999999999999},"min":{"1min":0.97099999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.845,"5min":2.101,"15min":2.7250000000000001},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.7210000000000001,"5min":1.6890000000000001,"15min":1.71},"min":{"1min":1.425,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1040000000000001,"5min":2.298,"15min":3.1440000000000001},"last":1.4970000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575990,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33904,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337936,"statfs":{"total":96636764160,"available":96602046464,"internally_reserved":0,"allocated":2785280,"data_stored":1349820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.2350000000000001,"15min":1.2250000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.64000000000000001},"max":{"1min":6.298,"5min":6.298,"15min":6.298},"last":1.208},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.3380000000000001,"15min":1.2909999999999999},"min":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.66400000000000003},"max":{"1min":8.2650000000000006,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":1.607}]},{"osd":1,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2450000000000001,"15min":1.222},"min":{"1min":0.80200000000000005,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.8520000000000001,"5min":1.4510000000000001,"15min":1.3859999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":12.468,"5min":12.468,"15min":12.468},"last":1.375}]},{"osd":2,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.7889999999999999,"5min":1.337,"15min":1.2829999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":11.586,"5min":11.586,"15min":11.586},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.569,"5min":1.3799999999999999,"15min":1.369},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.77500000000000002},"max":{"1min":4.7030000000000003,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.335}]},{"osd":3,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.9239999999999999,"5min":1.452,"15min":1.353},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":12.801,"5min":12.801,"15min":12.801},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.302,"15min":1.284},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":9.1539999999999999,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":1.022}]},{"osd":4,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.3700000000000001,"15min":1.3220000000000001},"min":{"1min":0.65400000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.5649999999999999},{"interface":"front","average":{"1min":2.141,"5min":1.609,"15min":1.548},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":13.981999999999999,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.6779999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":2.246,"5min":1.5109999999999999,"15min":1.391},"min":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":18.247,"5min":18.247,"15min":18.247},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.8360000000000001,"5min":1.5900000000000001,"15min":1.5629999999999999},"min":{"1min":0.76200000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":6.4039999999999999,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.4450000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:24:35 2023","interfaces":[{"interface":"back","average":{"1min":2.0609999999999999,"5min":1.603,"15min":1.5349999999999999},"min":{"1min":0.749,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":12.589,"5min":12.589,"15min":12.589},"last":1.498},{"interface":"front","average":{"1min":2.1749999999999998,"5min":1.6739999999999999,"15min":1.595},"min":{"1min":0.78200000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":12.725,"5min":12.725,"15min":12.725},"last":1.532}]}]},{"osd":5,"up_from":13,"seq":55834575990,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29736,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342104,"statfs":{"total":96636764160,"available":96606314496,"internally_reserved":0,"allocated":2777088,"data_stored":1349764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2350000000000001,"15min":1.27},"min":{"1min":0.74199999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.583,"5min":2.661,"15min":2.661},"last":1.51},{"interface":"front","average":{"1min":1.306,"5min":1.3029999999999999,"15min":1.349},"min":{"1min":0.95799999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7909999999999999,"5min":1.7949999999999999,"15min":1.865},"last":1.431}]},{"osd":1,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1819999999999999,"15min":1.1950000000000001},"min":{"1min":0.81999999999999995,"5min":0.80800000000000005,"15min":0.79100000000000004},"max":{"1min":1.5009999999999999,"5min":1.7869999999999999,"15min":1.992},"last":1.139},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.258,"15min":1.2290000000000001},"min":{"1min":0.84399999999999997,"5min":0.84199999999999997,"15min":0.80800000000000005},"max":{"1min":1.635,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.272}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.264,"15min":1.244},"min":{"1min":0.97899999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.625,"5min":1.952,"15min":1.952},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.329,"5min":1.2949999999999999,"15min":1.3220000000000001},"min":{"1min":0.92400000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":2.1640000000000001},"last":1.5549999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3320000000000001,"15min":1.3340000000000001},"min":{"1min":0.98999999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":3.012,"15min":3.012},"last":1.073},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.429,"15min":1.425},"min":{"1min":1.0900000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":2.2250000000000001},"last":1.462}]},{"osd":4,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.339,"15min":1.339},"min":{"1min":0.94199999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.6990000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.6799999999999999,"5min":1.5720000000000001,"15min":1.5740000000000001},"min":{"1min":1.353,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.948,"5min":2.903,"15min":2.903},"last":1.323}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4330000000000001,"15min":1.4139999999999999},"min":{"1min":0.873,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8660000000000001,"5min":2.0649999999999999,"15min":2.0979999999999999},"last":1.373},{"interface":"front","average":{"1min":1.51,"5min":1.494,"15min":1.4770000000000001},"min":{"1min":0.95399999999999996,"5min":0.90700000000000003,"15min":0.78400000000000003},"max":{"1min":1.841,"5min":3.073,"15min":3.073},"last":1.401}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.675,"5min":1.5660000000000001,"15min":1.5469999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0779999999999998,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.645,"15min":1.645},"min":{"1min":1.1779999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.1099999999999999,"5min":2.1459999999999999,"15min":2.2930000000000001},"last":1.6679999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575991,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51960,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319880,"statfs":{"total":96636764160,"available":96583557120,"internally_reserved":0,"allocated":2797568,"data_stored":1352725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1519999999999999,"15min":1.1859999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.7290000000000001,"5min":1.792,"15min":1.792},"last":1.175},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.218,"15min":1.2350000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.9299999999999999},"last":1.556}]},{"osd":1,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2769999999999999,"15min":1.298},"min":{"1min":0.81799999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.841,"5min":1.841,"15min":1.913},"last":1.298},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.236,"15min":1.25},"min":{"1min":0.78500000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.877,"5min":1.877,"15min":1.889},"last":1.3700000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.29,"15min":1.3089999999999999},"min":{"1min":1.0369999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.835},"last":1.605},{"interface":"front","average":{"1min":1.351,"5min":1.3999999999999999,"15min":1.403},"min":{"1min":0.879,"5min":0.879,"15min":0.872},"max":{"1min":1.8380000000000001,"5min":1.903,"15min":1.9139999999999999},"last":1.865}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3169999999999999,"15min":1.337},"min":{"1min":0.91100000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.6830000000000001,"5min":1.8280000000000001,"15min":1.849},"last":1.52},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4410000000000001,"15min":1.4330000000000001},"min":{"1min":1.145,"5min":1.008,"15min":0.96999999999999997},"max":{"1min":1.7769999999999999,"5min":1.865,"15min":2.0720000000000001},"last":1.7210000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.4810000000000001,"15min":1.4399999999999999},"min":{"1min":1.0149999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9399999999999999,"5min":2.105,"15min":2.105},"last":1.9890000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.431,"15min":1.415},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.78600000000000003},"max":{"1min":1.8240000000000001,"5min":2.04,"15min":2.04},"last":1.659}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.5409999999999999,"15min":1.5029999999999999},"min":{"1min":0.94299999999999995,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":3.157,"5min":3.157,"15min":3.157},"last":1.8480000000000001},{"interface":"front","average":{"1min":1.5,"5min":1.6100000000000001,"15min":1.611},"min":{"1min":1.0940000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0089999999999999,"5min":2.2330000000000001,"15min":2.2770000000000001},"last":2.0950000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.5640000000000001,"15min":1.5660000000000001},"min":{"1min":1.0549999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.855,"5min":2.113,"15min":2.3420000000000001},"last":1.7789999999999999},{"interface":"front","average":{"1min":1.639,"5min":1.6619999999999999,"15min":1.6399999999999999},"min":{"1min":1.0660000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9630000000000001,"5min":2.1469999999999998,"15min":2.3839999999999999},"last":1.905}]}]},{"osd":3,"up_from":13,"seq":55834575990,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47812,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324028,"statfs":{"total":96636764160,"available":96587804672,"internally_reserved":0,"allocated":2801664,"data_stored":1351046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.89400000000000002,"15min":0.90700000000000003},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.478,"5min":1.478,"15min":1.901},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0369999999999999,"15min":1.0720000000000001},"min":{"1min":0.59299999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.391,"5min":1.6180000000000001,"15min":1.6679999999999999},"last":1.1850000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.95599999999999996,"15min":0.98799999999999999},"min":{"1min":0.60799999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.105,"15min":1.133},"min":{"1min":0.48399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.601,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.999}]},{"osd":2,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0760000000000001,"15min":1.117},"min":{"1min":0.65500000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.524,"5min":1.694,"15min":1.694},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.198,"15min":1.236},"min":{"1min":0.625,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.92800000000000005}]},{"osd":4,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3169999999999999,"15min":1.3500000000000001},"min":{"1min":0.83599999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.806,"5min":1.806,"15min":1.873},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.357,"15min":1.367},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.8819999999999999},"last":1.343}]},{"osd":5,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.385,"15min":1.401},"min":{"1min":0.80600000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.419,"5min":1.4059999999999999,"15min":1.444},"min":{"1min":1.151,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.3200000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.504,"15min":1.5329999999999999},"min":{"1min":0.876,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":3.9129999999999998,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5,"15min":1.5369999999999999},"min":{"1min":1.153,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":3.9319999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.409}]},{"osd":7,"last update":"Mon Oct 16 11:24:09 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.528,"15min":1.5600000000000001},"min":{"1min":0.95399999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.877,"5min":2.0790000000000002,"15min":2.2679999999999998},"last":1.712},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.639,"15min":1.673},"min":{"1min":1.1279999999999999,"5min":1,"15min":1},"max":{"1min":2.052,"5min":2.198,"15min":2.5129999999999999},"last":1.7709999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575989,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51964,"kb_used_data":2728,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319876,"statfs":{"total":96636764160,"available":96583553024,"internally_reserved":0,"allocated":2793472,"data_stored":1350850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.91000000000000003,"15min":0.92200000000000004},"min":{"1min":0.74199999999999999,"5min":0.51900000000000002,"15min":0.5},"max":{"1min":1.532,"5min":1.532,"15min":1.669},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.0569999999999999,"15min":1.0820000000000001},"min":{"1min":0.72499999999999998,"5min":0.47699999999999998,"15min":0.32600000000000001},"max":{"1min":1.496,"5min":1.681,"15min":2.1429999999999998},"last":0.91000000000000003}]},{"osd":1,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.089,"15min":1.097},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.39200000000000002},"max":{"1min":1.415,"5min":1.6779999999999999,"15min":1.978},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1479999999999999,"15min":1.165},"min":{"1min":0.57599999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.448,"5min":1.609,"15min":1.869},"last":1.127}]},{"osd":3,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.0649999999999999,"15min":1.087},"min":{"1min":0.54600000000000004,"5min":0.437,"15min":0.437},"max":{"1min":1.464,"5min":1.651,"15min":1.8560000000000001},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.252,"15min":1.274},"min":{"1min":0.82199999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6120000000000001,"5min":1.7450000000000001,"15min":1.857},"last":1.1659999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3740000000000001,"15min":1.415},"min":{"1min":0.91500000000000004,"5min":0.86599999999999999,"15min":0.73499999999999999},"max":{"1min":1.6910000000000001,"5min":1.8029999999999999,"15min":2.5059999999999998},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3440000000000001,"15min":1.3520000000000001},"min":{"1min":0.97899999999999998,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":1.825,"5min":1.825,"15min":2.0880000000000001},"last":1.538}]},{"osd":5,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.444,"15min":1.464},"min":{"1min":1.0720000000000001,"5min":0.84999999999999998,"15min":0.78500000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2269999999999999},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4550000000000001,"15min":1.4890000000000001},"min":{"1min":1.0509999999999999,"5min":0.95799999999999996,"15min":0.874},"max":{"1min":1.798,"5min":1.931,"15min":2.6259999999999999},"last":1.401}]},{"osd":6,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4119999999999999,"15min":1.411},"min":{"1min":1.0569999999999999,"5min":0.84099999999999997,"15min":0.80500000000000005},"max":{"1min":1.71,"5min":1.9019999999999999,"15min":2.2109999999999999},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5,"15min":1.5369999999999999},"min":{"1min":1.1899999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.2909999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.4379999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:24:07 2023","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":1.5489999999999999,"15min":1.5860000000000001},"min":{"1min":1.2290000000000001,"5min":0.995,"15min":0.92500000000000004},"max":{"1min":2.1469999999999998,"5min":2.262,"15min":2.8109999999999999},"last":1.325},{"interface":"front","average":{"1min":1.494,"5min":1.4870000000000001,"15min":1.5149999999999999},"min":{"1min":1.242,"5min":0.81599999999999995,"15min":0.80100000000000005},"max":{"1min":1.8180000000000001,"5min":2.2269999999999999,"15min":2.2829999999999999},"last":1.5960000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575989,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60272,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311568,"statfs":{"total":96636764160,"available":96575045632,"internally_reserved":0,"allocated":2785280,"data_stored":1349328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.91900000000000004,"15min":0.91500000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.575,"5min":1.575,"15min":1.7929999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0289999999999999,"15min":1.0580000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.494,"5min":1.5509999999999999,"15min":1.667},"last":1.149}]},{"osd":2,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.98399999999999999,"15min":1.0189999999999999},"min":{"1min":0.64300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.72},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.252,"15min":1.276},"min":{"1min":0.69799999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":1.28}]},{"osd":3,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.1879999999999999,"15min":1.216},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.774,"5min":1.774,"15min":1.79},"last":1.093},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.169,"15min":1.1739999999999999},"min":{"1min":0.66700000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.633,"5min":1.633,"15min":1.802},"last":0.89200000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.2609999999999999,"15min":1.274},"min":{"1min":0.86299999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.653,"5min":1.887,"15min":1.9159999999999999},"last":1.45},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.409,"15min":1.415},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.887,"5min":2.0619999999999998,"15min":2.093},"last":1.4870000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.379,"15min":1.409},"min":{"1min":0.93700000000000006,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.948,"5min":2.0009999999999999,"15min":2.0640000000000001},"last":1.321},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4239999999999999,"15min":1.421},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":1.825,"5min":1.825,"15min":2.0649999999999999},"last":1.3520000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4850000000000001,"15min":1.496},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.879,"5min":2.1970000000000001,"15min":2.2949999999999999},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4710000000000001,"15min":1.486},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.891,"5min":1.891,"15min":2.1970000000000001},"last":1.2270000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:23:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5649999999999999,"15min":1.5620000000000001},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.1720000000000002},"last":1.429},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.625,"15min":1.607},"min":{"1min":1.1080000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.9359999999999999,"5min":2.1989999999999998,"15min":2.2810000000000001},"last":1.4650000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575990,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52532,"kb_used_data":3300,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319308,"statfs":{"total":96636764160,"available":96582971392,"internally_reserved":0,"allocated":3379200,"data_stored":1940312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.86599999999999999,"15min":0.89500000000000002},"min":{"1min":0.42699999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.3420000000000001,"5min":1.52,"15min":1.619},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0649999999999999,"15min":1.1020000000000001},"min":{"1min":0.54100000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3680000000000001,"5min":1.6399999999999999,"15min":1.76},"last":1.3680000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0680000000000001,"15min":1.085},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.258},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0840000000000001,"15min":1.143},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.5169999999999999,"5min":1.6220000000000001,"15min":1.6870000000000001},"last":1.115}]},{"osd":3,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.071,"15min":1.0860000000000001},"min":{"1min":0.51400000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7609999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.206,"15min":1.2490000000000001},"min":{"1min":0.63800000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.9830000000000001},"last":1.4510000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3640000000000001,"15min":1.389},"min":{"1min":0.879,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.843,"5min":1.9390000000000001,"15min":1.95},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.339,"5min":1.321,"15min":1.343},"min":{"1min":1.0349999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.665,"5min":2.1539999999999999,"15min":2.4820000000000002},"last":1.498}]},{"osd":5,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.383,"15min":1.3839999999999999},"min":{"1min":1.032,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":1.7689999999999999,"5min":2.0750000000000002,"15min":2.6669999999999998},"last":1.633},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.401,"15min":1.446},"min":{"1min":0.83399999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.9450000000000001,"5min":2.2029999999999998,"15min":2.2040000000000002},"last":1.7130000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.454,"15min":1.498},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":2.004,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.9339999999999999},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.476,"15min":1.4950000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.073,"5min":2.073,"15min":2.262},"last":1.8520000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":1.605,"5min":1.5169999999999999,"15min":1.556},"min":{"1min":0.92500000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.347},"last":1.988},{"interface":"front","average":{"1min":1.536,"5min":1.4850000000000001,"15min":1.51},"min":{"1min":0.84399999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.9550000000000001,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.7490000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:24:51.034 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T11:24:51.035 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:24:51.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.285+0000 7f6919373640 1 Processor -- start 2023-10-16T11:24:51.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.286+0000 7f6919373640 1 -- start start 2023-10-16T11:24:51.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.287+0000 7f6919373640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f69140cd930 0x7f6914164240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:51.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.287+0000 7f6919373640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6914159790 con 0x7f69140caa10 2023-10-16T11:24:51.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.287+0000 7f6919373640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69140c9b20 con 0x7f69141593b0 2023-10-16T11:24:51.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.287+0000 7f6919373640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6914070370 con 0x7f69140cd930 2023-10-16T11:24:51.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.287+0000 7f690a7fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f69140caa10 0x7f691406fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47968/0 (socket says 172.21.15.84:47968) 2023-10-16T11:24:51.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.287+0000 7f690a7fc640 1 -- 172.21.15.84:0/1458309282 learned_addr learned my addr 172.21.15.84:0/1458309282 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:51.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.287+0000 7f69137fe640 1 --2- 172.21.15.84:0/1458309282 >> v2:172.21.15.84:3300/0 conn(0x7f69140cd930 0x7f6914164240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:51.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.288+0000 7f69127fc640 1 -- 172.21.15.84:0/1458309282 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385963742 0 0) 0x7f6914159790 con 0x7f69140caa10 2023-10-16T11:24:51.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.288+0000 7f69127fc640 1 -- 172.21.15.84:0/1458309282 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6900005730 con 0x7f69140caa10 2023-10-16T11:24:51.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.288+0000 7f69137fe640 1 -- 172.21.15.84:0/1458309282 >> v1:172.21.15.157:6789/0 conn(0x7f69141593b0 legacy=0x7f691406f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:51.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.288+0000 7f69137fe640 1 -- 172.21.15.84:0/1458309282 >> v1:172.21.15.84:6789/0 conn(0x7f69140caa10 legacy=0x7f691406fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:51.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.288+0000 7f69137fe640 1 -- 172.21.15.84:0/1458309282 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69140c9e00 con 0x7f69140cd930 2023-10-16T11:24:51.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.289+0000 7f69137fe640 1 --2- 172.21.15.84:0/1458309282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69140cd930 0x7f6914164240 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f690c009d90 tx=0x7f690c02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d273df2222fd9b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:51.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.289+0000 7f69127fc640 1 -- 172.21.15.84:0/1458309282 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c02cb40 con 0x7f69140cd930 2023-10-16T11:24:51.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.289+0000 7f69127fc640 1 -- 172.21.15.84:0/1458309282 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c02cca0 con 0x7f69140cd930 2023-10-16T11:24:51.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.289+0000 7f69127fc640 1 -- 172.21.15.84:0/1458309282 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c034660 con 0x7f69140cd930 2023-10-16T11:24:51.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.290+0000 7f6919373640 1 -- 172.21.15.84:0/1458309282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69140cd930 msgr2=0x7f6914164240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:51.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.290+0000 7f6919373640 1 --2- 172.21.15.84:0/1458309282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69140cd930 0x7f6914164240 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f690c009d90 tx=0x7f690c02bd90 comp rx=0 tx=0).stop 2023-10-16T11:24:51.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.290+0000 7f6919373640 1 -- 172.21.15.84:0/1458309282 shutdown_connections 2023-10-16T11:24:51.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.290+0000 7f6919373640 1 --2- 172.21.15.84:0/1458309282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69140cd930 0x7f6914164240 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:51.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.290+0000 7f6919373640 1 -- 172.21.15.84:0/1458309282 >> 172.21.15.84:0/1458309282 conn(0x7f69140cb410 msgr2=0x7f69140cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:51.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.290+0000 7f6919373640 1 -- 172.21.15.84:0/1458309282 shutdown_connections 2023-10-16T11:24:51.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.290+0000 7f6919373640 1 -- 172.21.15.84:0/1458309282 wait complete. 2023-10-16T11:24:51.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.292+0000 7f6919373640 1 Processor -- start 2023-10-16T11:24:51.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.292+0000 7f6919373640 1 -- start start 2023-10-16T11:24:51.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.293+0000 7f6919373640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69140caa10 0x7f691414c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:51.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.293+0000 7f6919373640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69140cd930 0x7f69140cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:51.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.293+0000 7f6919373640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69141593b0 0x7f69140cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:51.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.293+0000 7f6919373640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69141566d0 con 0x7f69140caa10 2023-10-16T11:24:51.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.293+0000 7f6919373640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6914157cd0 con 0x7f69140cd930 2023-10-16T11:24:51.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.293+0000 7f6919373640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6914157fb0 con 0x7f69141593b0 2023-10-16T11:24:51.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.293+0000 7f6912ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69140caa10 0x7f691414c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:51.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.293+0000 7f6912ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69140caa10 0x7f691414c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51362/0 (socket says 172.21.15.84:51362) 2023-10-16T11:24:51.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.293+0000 7f6912ffd640 1 -- 172.21.15.84:0/659584706 learned_addr learned my addr 172.21.15.84:0/659584706 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:51.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.294+0000 7f6912ffd640 1 -- 172.21.15.84:0/659584706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69141593b0 msgr2=0x7f69140cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:51.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.294+0000 7f6912ffd640 1 --2- 172.21.15.84:0/659584706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69141593b0 0x7f69140cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:51.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.294+0000 7f6912ffd640 1 -- 172.21.15.84:0/659584706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69140cd930 msgr2=0x7f69140cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:51.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.294+0000 7f6912ffd640 1 --2- 172.21.15.84:0/659584706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69140cd930 0x7f69140cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:51.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.294+0000 7f6912ffd640 1 -- 172.21.15.84:0/659584706 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c009a70 con 0x7f69140caa10 2023-10-16T11:24:51.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.294+0000 7f6912ffd640 1 --2- 172.21.15.84:0/659584706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69140caa10 0x7f691414c9a0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f690400b3f0 tx=0x7f690400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:51.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.294+0000 7f690bfff640 1 -- 172.21.15.84:0/659584706 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6904011830 con 0x7f69140caa10 2023-10-16T11:24:51.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.294+0000 7f690bfff640 1 -- 172.21.15.84:0/659584706 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6904011990 con 0x7f69140caa10 2023-10-16T11:24:51.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.294+0000 7f6919373640 1 -- 172.21.15.84:0/659584706 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6914158290 con 0x7f69140caa10 2023-10-16T11:24:51.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.294+0000 7f690bfff640 1 -- 172.21.15.84:0/659584706 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6904011d40 con 0x7f69140caa10 2023-10-16T11:24:51.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.294+0000 7f6919373640 1 -- 172.21.15.84:0/659584706 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f691414cee0 con 0x7f69140caa10 2023-10-16T11:24:51.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.296+0000 7f6919373640 1 -- 172.21.15.84:0/659584706 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69141587b0 con 0x7f69140caa10 2023-10-16T11:24:51.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.297+0000 7f690bfff640 1 -- 172.21.15.84:0/659584706 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f690401a400 con 0x7f69140caa10 2023-10-16T11:24:51.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.297+0000 7f690bfff640 1 --2- 172.21.15.84:0/659584706 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f68f406af90 0x7f68f406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:51.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.298+0000 7f690bfff640 1 -- 172.21.15.84:0/659584706 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f6904089960 con 0x7f69140caa10 2023-10-16T11:24:51.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.298+0000 7f690a7fc640 1 --2- 172.21.15.84:0/659584706 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f68f406af90 0x7f68f406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:51.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.298+0000 7f690a7fc640 1 --2- 172.21.15.84:0/659584706 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f68f406af90 0x7f68f406d450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f69140b0d20 tx=0x7f68fc007930 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:51.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.302+0000 7f690bfff640 1 -- 172.21.15.84:0/659584706 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f69141587b0 con 0x7f69140caa10 2023-10-16T11:24:51.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.531+0000 7f6919373640 1 -- 172.21.15.84:0/659584706 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69141587b0 con 0x7f69140caa10 2023-10-16T11:24:51.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.533+0000 7f690bfff640 1 -- 172.21.15.84:0/659584706 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+6152 (secure 0 0 0) 0x7f6904054570 con 0x7f69140caa10 2023-10-16T11:24:51.535 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:24:51.535 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":120,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3106898227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3106898227},{"type":"v1","addr":"172.21.15.84:6837","nonce":3106898227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":7471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2296953749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2296953749},{"type":"v1","addr":"172.21.15.157:6834","nonce":2296953749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:24:25.531691+0000","modified":"2023-10-16T11:24:28.574116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11307},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11307":{"gid":11307,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/551895595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":551895595},{"type":"v1","addr":"172.21.15.84:6836","nonce":551895595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:24:40.366708+0000","modified":"2023-10-16T11:24:43.431532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7474":{"gid":7474,"name":"d","rank":0,"incarnation":116,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6835/3992377882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3992377882},{"type":"v1","addr":"172.21.15.157:6835","nonce":3992377882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-16T11:24:51.536 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 120 2023-10-16T11:24:51.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.535+0000 7f6919373640 1 -- 172.21.15.84:0/659584706 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f68f406af90 msgr2=0x7f68f406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:51.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.535+0000 7f6919373640 1 --2- 172.21.15.84:0/659584706 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f68f406af90 0x7f68f406d450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f69140b0d20 tx=0x7f68fc007930 comp rx=0 tx=0).stop 2023-10-16T11:24:51.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.535+0000 7f6919373640 1 -- 172.21.15.84:0/659584706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69140caa10 msgr2=0x7f691414c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:51.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.535+0000 7f6919373640 1 --2- 172.21.15.84:0/659584706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69140caa10 0x7f691414c9a0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f690400b3f0 tx=0x7f690400b8b0 comp rx=0 tx=0).stop 2023-10-16T11:24:51.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.535+0000 7f6919373640 1 -- 172.21.15.84:0/659584706 shutdown_connections 2023-10-16T11:24:51.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.535+0000 7f6919373640 1 --2- 172.21.15.84:0/659584706 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f68f406af90 0x7f68f406d450 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:51.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.535+0000 7f6919373640 1 --2- 172.21.15.84:0/659584706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69141593b0 0x7f69140cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:51.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.535+0000 7f6919373640 1 --2- 172.21.15.84:0/659584706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69140cd930 0x7f69140cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:51.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.535+0000 7f6919373640 1 --2- 172.21.15.84:0/659584706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69140caa10 0x7f691414c9a0 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:51.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.535+0000 7f6919373640 1 -- 172.21.15.84:0/659584706 >> 172.21.15.84:0/659584706 conn(0x7f69140cb410 msgr2=0x7f69140cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:51.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.536+0000 7f6919373640 1 -- 172.21.15.84:0/659584706 shutdown_connections 2023-10-16T11:24:51.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:51.536+0000 7f6919373640 1 -- 172.21.15.84:0/659584706 wait complete. 2023-10-16T11:24:51.957 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 120, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T11:24:40.366708+0000', 'modified': '2023-10-16T11:24:43.431532+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7474}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7474': {'gid': 7474, 'name': 'd', 'rank': 0, 'incarnation': 116, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.157:6835/3992377882', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6833', 'nonce': 3992377882}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 3992377882}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T11:24:51.958 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T11:24:51.958 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-16T11:24:52.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.167+0000 7fa9a16d1640 1 Processor -- start 2023-10-16T11:24:52.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.168+0000 7fa9a16d1640 1 -- start start 2023-10-16T11:24:52.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.169+0000 7fa9a16d1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa99c15cd70 0x7fa99c0d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.169+0000 7fa9a16d1640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99c15d150 con 0x7fa99c0b8480 2023-10-16T11:24:52.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.169+0000 7fa9a16d1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99c0d1180 con 0x7fa99c0cded0 2023-10-16T11:24:52.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.169+0000 7fa9a16d1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa99c0d1460 con 0x7fa99c15cd70 2023-10-16T11:24:52.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.169+0000 7fa99b7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa99c0b8480 0x7fa99c0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47976/0 (socket says 172.21.15.84:47976) 2023-10-16T11:24:52.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.169+0000 7fa99b7fe640 1 -- 172.21.15.84:0/1172628420 learned_addr learned my addr 172.21.15.84:0/1172628420 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:52.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.169+0000 7fa99bfff640 1 --2- 172.21.15.84:0/1172628420 >> v2:172.21.15.84:3300/0 conn(0x7fa99c15cd70 0x7fa99c0d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.170+0000 7fa99bfff640 1 -- 172.21.15.84:0/1172628420 >> v1:172.21.15.157:6789/0 conn(0x7fa99c0cded0 legacy=0x7fa99c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:52.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.170+0000 7fa99bfff640 1 -- 172.21.15.84:0/1172628420 >> v1:172.21.15.84:6789/0 conn(0x7fa99c0b8480 legacy=0x7fa99c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.170+0000 7fa99bfff640 1 -- 172.21.15.84:0/1172628420 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c0d1740 con 0x7fa99c15cd70 2023-10-16T11:24:52.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.170+0000 7fa99affd640 1 -- 172.21.15.84:0/1172628420 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151634804 0 0) 0x7fa99c15d150 con 0x7fa99c0b8480 2023-10-16T11:24:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.171+0000 7fa99bfff640 1 --2- 172.21.15.84:0/1172628420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa99c15cd70 0x7fa99c0d2220 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009d90 tx=0x7fa98c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591371283287d12d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:52.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.171+0000 7fa99affd640 1 -- 172.21.15.84:0/1172628420 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c02cc60 con 0x7fa99c15cd70 2023-10-16T11:24:52.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.171+0000 7fa99affd640 1 -- 172.21.15.84:0/1172628420 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c02cdc0 con 0x7fa99c15cd70 2023-10-16T11:24:52.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.171+0000 7fa99affd640 1 -- 172.21.15.84:0/1172628420 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c034660 con 0x7fa99c15cd70 2023-10-16T11:24:52.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.172+0000 7fa9a16d1640 1 -- 172.21.15.84:0/1172628420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa99c15cd70 msgr2=0x7fa99c0d2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.172+0000 7fa9a16d1640 1 --2- 172.21.15.84:0/1172628420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa99c15cd70 0x7fa99c0d2220 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009d90 tx=0x7fa98c02beb0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.172+0000 7fa9a16d1640 1 -- 172.21.15.84:0/1172628420 shutdown_connections 2023-10-16T11:24:52.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.172+0000 7fa9a16d1640 1 --2- 172.21.15.84:0/1172628420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa99c15cd70 0x7fa99c0d2220 unknown :-1 s=CLOSED pgs=2700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.172+0000 7fa9a16d1640 1 -- 172.21.15.84:0/1172628420 >> 172.21.15.84:0/1172628420 conn(0x7fa99c0cf570 msgr2=0x7fa99c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:52.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.172+0000 7fa9a16d1640 1 -- 172.21.15.84:0/1172628420 shutdown_connections 2023-10-16T11:24:52.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.172+0000 7fa9a16d1640 1 -- 172.21.15.84:0/1172628420 wait complete. 2023-10-16T11:24:52.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.174+0000 7fa9a16d1640 1 Processor -- start 2023-10-16T11:24:52.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.174+0000 7fa9a16d1640 1 -- start start 2023-10-16T11:24:52.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.175+0000 7fa9a16d1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa99c0b8480 0x7fa99c1f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.175+0000 7fa9a16d1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa99c0cded0 0x7fa99c14ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.175+0000 7fa9a16d1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa99c15cd70 0x7fa99c0aa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.175+0000 7fa9a16d1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c1504a0 con 0x7fa99c0cded0 2023-10-16T11:24:52.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.175+0000 7fa9a16d1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c15e640 con 0x7fa99c0b8480 2023-10-16T11:24:52.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.175+0000 7fa9a16d1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c15e8e0 con 0x7fa99c15cd70 2023-10-16T11:24:52.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.175+0000 7fa99b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa99c0cded0 0x7fa99c14ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.175+0000 7fa99b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa99c0cded0 0x7fa99c14ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51380/0 (socket says 172.21.15.84:51380) 2023-10-16T11:24:52.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.175+0000 7fa99b7fe640 1 -- 172.21.15.84:0/3431554541 learned_addr learned my addr 172.21.15.84:0/3431554541 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:52.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.175+0000 7fa99b7fe640 1 -- 172.21.15.84:0/3431554541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa99c15cd70 msgr2=0x7fa99c0aa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.175+0000 7fa99b7fe640 1 --2- 172.21.15.84:0/3431554541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa99c15cd70 0x7fa99c0aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.175+0000 7fa99b7fe640 1 -- 172.21.15.84:0/3431554541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa99c0b8480 msgr2=0x7fa99c1f38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.175+0000 7fa99b7fe640 1 --2- 172.21.15.84:0/3431554541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa99c0b8480 0x7fa99c1f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.175+0000 7fa99b7fe640 1 -- 172.21.15.84:0/3431554541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c009a70 con 0x7fa99c0cded0 2023-10-16T11:24:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.176+0000 7fa99b7fe640 1 --2- 172.21.15.84:0/3431554541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa99c0cded0 0x7fa99c14ff60 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c4d0 tx=0x7fa98400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.176+0000 7fa998ff9640 1 -- 172.21.15.84:0/3431554541 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9840074e0 con 0x7fa99c0cded0 2023-10-16T11:24:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.176+0000 7fa998ff9640 1 -- 172.21.15.84:0/3431554541 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa984007660 con 0x7fa99c0cded0 2023-10-16T11:24:52.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.176+0000 7fa998ff9640 1 -- 172.21.15.84:0/3431554541 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984004a20 con 0x7fa99c0cded0 2023-10-16T11:24:52.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.176+0000 7fa9a16d1640 1 -- 172.21.15.84:0/3431554541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99c15ebc0 con 0x7fa99c0cded0 2023-10-16T11:24:52.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.176+0000 7fa9a16d1640 1 -- 172.21.15.84:0/3431554541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99c15f180 con 0x7fa99c0cded0 2023-10-16T11:24:52.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.179+0000 7fa998ff9640 1 -- 172.21.15.84:0/3431554541 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa984004b80 con 0x7fa99c0cded0 2023-10-16T11:24:52.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.179+0000 7fa998ff9640 1 --2- 172.21.15.84:0/3431554541 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa97c06af90 0x7fa97c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.180+0000 7fa998ff9640 1 -- 172.21.15.84:0/3431554541 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7fa984088d10 con 0x7fa99c0cded0 2023-10-16T11:24:52.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.180+0000 7fa99bfff640 1 --2- 172.21.15.84:0/3431554541 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa97c06af90 0x7fa97c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.180+0000 7fa9a559d740 1 Processor -- start 2023-10-16T11:24:52.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.180+0000 7fa9a559d740 1 -- start start 2023-10-16T11:24:52.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.180+0000 7fa99bfff640 1 --2- 172.21.15.84:0/3431554541 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa97c06af90 0x7fa97c06d450 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa98c036000 tx=0x7fa98c02e000 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:52.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.181+0000 7fa9a559d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57170 0x55d4b6c2a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.181+0000 7fa9a559d740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d4b6c57550 0x55d4b6c59f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.181+0000 7fa9a559d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c2a5b0 0x55d4b6c5a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.181+0000 7fa9a559d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b6b4a7a0 con 0x55d4b6c57170 2023-10-16T11:24:52.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.181+0000 7fa9a559d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b6b9c740 con 0x55d4b6c2a5b0 2023-10-16T11:24:52.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.181+0000 7fa9a559d740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4b6b99e30 con 0x55d4b6c57550 2023-10-16T11:24:52.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.181+0000 7fa99bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57170 0x55d4b6c2a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.181+0000 7fa99bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57170 0x55d4b6c2a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51386/0 (socket says 172.21.15.84:51386) 2023-10-16T11:24:52.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.181+0000 7fa99bfff640 1 -- 172.21.15.84:0/2774083722 learned_addr learned my addr 172.21.15.84:0/2774083722 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:52.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.181+0000 7fa99bfff640 1 -- 172.21.15.84:0/2774083722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d4b6c57550 msgr2=0x55d4b6c59f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:52.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.181+0000 7fa99bfff640 1 --2- 172.21.15.84:0/2774083722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d4b6c57550 0x55d4b6c59f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.182+0000 7fa99bfff640 1 -- 172.21.15.84:0/2774083722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c2a5b0 msgr2=0x55d4b6c5a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:52.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.182+0000 7fa9a0ed0640 1 --2- 172.21.15.84:0/2774083722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c2a5b0 0x55d4b6c5a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.182+0000 7fa99bfff640 1 --2- 172.21.15.84:0/2774083722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c2a5b0 0x55d4b6c5a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.182+0000 7fa99bfff640 1 -- 172.21.15.84:0/2774083722 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4b6b59910 con 0x55d4b6c57170 2023-10-16T11:24:52.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.182+0000 7fa9a0ed0640 1 --2- 172.21.15.84:0/2774083722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c2a5b0 0x55d4b6c5a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:52.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.182+0000 7fa99bfff640 1 --2- 172.21.15.84:0/2774083722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57170 0x55d4b6c2a070 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7fa98c02bef0 tx=0x7fa98c031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ea01ebc2a2c2e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:52.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.182+0000 7fa991ffb640 1 -- 172.21.15.84:0/2774083722 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c034e70 con 0x55d4b6c57170 2023-10-16T11:24:52.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.182+0000 7fa991ffb640 1 -- 172.21.15.84:0/2774083722 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c04a410 con 0x55d4b6c57170 2023-10-16T11:24:52.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.182+0000 7fa991ffb640 1 -- 172.21.15.84:0/2774083722 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c04a780 con 0x55d4b6c57170 2023-10-16T11:24:52.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.183+0000 7fa9a559d740 1 -- 172.21.15.84:0/2774083722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57170 msgr2=0x55d4b6c2a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.183+0000 7fa9a559d740 1 --2- 172.21.15.84:0/2774083722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57170 0x55d4b6c2a070 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7fa98c02bef0 tx=0x7fa98c031000 comp rx=0 tx=0).stop 2023-10-16T11:24:52.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.183+0000 7fa9a559d740 1 -- 172.21.15.84:0/2774083722 shutdown_connections 2023-10-16T11:24:52.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.183+0000 7fa9a559d740 1 --2- 172.21.15.84:0/2774083722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c2a5b0 0x55d4b6c5a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.183+0000 7fa9a559d740 1 --2- 172.21.15.84:0/2774083722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d4b6c57550 0x55d4b6c59f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.183+0000 7fa9a559d740 1 --2- 172.21.15.84:0/2774083722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57170 0x55d4b6c2a070 unknown :-1 s=CLOSED pgs=2702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.183+0000 7fa9a559d740 1 -- 172.21.15.84:0/2774083722 >> 172.21.15.84:0/2774083722 conn(0x55d4b6c2c380 msgr2=0x55d4b6c74f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:52.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.183+0000 7fa9a559d740 1 -- 172.21.15.84:0/2774083722 shutdown_connections 2023-10-16T11:24:52.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.183+0000 7fa9a559d740 1 -- 172.21.15.84:0/2774083722 wait complete. 2023-10-16T11:24:52.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.184+0000 7fa9a559d740 1 -- start start 2023-10-16T11:24:52.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.184+0000 7fa9a559d740 1 Processor -- start 2023-10-16T11:24:52.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.185+0000 7fa9a559d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57170 0x55d4b6c4fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.185+0000 7fa9a559d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c57550 0x55d4b6c50220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.185+0000 7fa9a559d740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d4b6c528b0 0x55d4b6c58e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.185+0000 7fa9a559d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b6b95720 con 0x55d4b6c57170 2023-10-16T11:24:52.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.185+0000 7fa9a559d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b6b33970 con 0x55d4b6c57550 2023-10-16T11:24:52.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.185+0000 7fa9a559d740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4b6b33610 con 0x55d4b6c528b0 2023-10-16T11:24:52.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.185+0000 7fa9a559d740 10 client.0 _ll_register_callbacks cb 0x55d4b6b559d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:24:52.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.185+0000 7fa99bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57170 0x55d4b6c4fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.185+0000 7fa99bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57170 0x55d4b6c4fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51390/0 (socket says 172.21.15.84:51390) 2023-10-16T11:24:52.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.185+0000 7fa99bfff640 1 -- 172.21.15.84:0/127922235 learned_addr learned my addr 172.21.15.84:0/127922235 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:52.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.185+0000 7fa99bfff640 1 -- 172.21.15.84:0/127922235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d4b6c528b0 msgr2=0x55d4b6c58e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:52.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.185+0000 7fa99bfff640 1 --2- 172.21.15.84:0/127922235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d4b6c528b0 0x55d4b6c58e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.185+0000 7fa99bfff640 1 -- 172.21.15.84:0/127922235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c57550 msgr2=0x55d4b6c50220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:52.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.185+0000 7fa99bfff640 1 --2- 172.21.15.84:0/127922235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c57550 0x55d4b6c50220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.185+0000 7fa99bfff640 1 -- 172.21.15.84:0/127922235 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99c0d1740 con 0x55d4b6c57170 2023-10-16T11:24:52.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.186+0000 7fa99bfff640 1 --2- 172.21.15.84:0/127922235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57170 0x55d4b6c4fce0 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7fa98c04c040 tx=0x7fa98c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d50441d0c46ce8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:52.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.186+0000 7fa97affd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T11:24:52.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.186+0000 7fa97affd640 1 -- 172.21.15.84:0/127922235 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c004510 con 0x55d4b6c57170 2023-10-16T11:24:52.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.186+0000 7fa9a559d740 1 -- 172.21.15.84:0/127922235 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4b6bbc160 con 0x55d4b6c57170 2023-10-16T11:24:52.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.186+0000 7fa97affd640 1 -- 172.21.15.84:0/127922235 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7fa98c046040 con 0x55d4b6c57170 2023-10-16T11:24:52.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.186+0000 7fa97affd640 1 -- 172.21.15.84:0/127922235 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa98c004670 con 0x55d4b6c57170 2023-10-16T11:24:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.186+0000 7fa9a559d740 10 client.11652 fetch_fsmap learned FSMap version 120 2023-10-16T11:24:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.186+0000 7fa9a559d740 1 -- 172.21.15.84:0/127922235 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=2+}) v3 -- 0x55d4b6b93da0 con 0x55d4b6c57170 2023-10-16T11:24:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.187+0000 7fa97affd640 1 -- 172.21.15.84:0/127922235 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d4b6bbc160 con 0x55d4b6c57170 2023-10-16T11:24:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.187+0000 7fa97affd640 1 -- 172.21.15.84:0/127922235 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 120) v1 ==== 3891+0+0 (secure 0 0 0) 0x7fa98c04a580 con 0x55d4b6c57170 2023-10-16T11:24:52.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.187+0000 7fa9a559d740 10 client.11652 fetch_fsmap finished waiting for FSMap version 120 2023-10-16T11:24:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.187+0000 7fa9a559d740 10 client.11652 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-16T11:24:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.187+0000 7fa9a559d740 20 client.11652 populate_metadata read hostname 'smithi084' 2023-10-16T11:24:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.187+0000 7fa9a559d740 1 --2- 172.21.15.84:0/127922235 >> [v2:172.21.15.157:6833/3992377882,v1:172.21.15.157:6835/3992377882] conn(0x55d4b6c59380 0x55d4b6c884f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.187+0000 7fa9a559d740 4 client.11652 mds_command: new command op to 7474 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T11:24:52.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.187+0000 7fa9a559d740 1 -- 172.21.15.84:0/127922235 --> [v2:172.21.15.157:6833/3992377882,v1:172.21.15.157:6835/3992377882] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4b6b67cc0 con 0x55d4b6c59380 2023-10-16T11:24:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.188+0000 7fa99b7fe640 1 --2- 172.21.15.84:0/127922235 >> [v2:172.21.15.157:6833/3992377882,v1:172.21.15.157:6835/3992377882] conn(0x55d4b6c59380 0x55d4b6c884f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.189+0000 7fa99b7fe640 1 --2- 172.21.15.84:0/127922235 >> [v2:172.21.15.157:6833/3992377882,v1:172.21.15.157:6835/3992377882] conn(0x55d4b6c59380 0x55d4b6c884f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c377efa4e06ab92 server_cookie=58350e27a595f0a4 in_seq=0 out_seq=0 2023-10-16T11:24:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.189+0000 7fa97affd640 10 client.11652 ms_handle_connect on v2:172.21.15.157:6833/3992377882 2023-10-16T11:24:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.190+0000 7fa97affd640 1 -- 172.21.15.84:0/127922235 <== mds.0 v2:172.21.15.157:6833/3992377882 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d4b6b67cc0 con 0x55d4b6c59380 2023-10-16T11:24:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.190+0000 7fa97affd640 10 client.11652 handle_command_reply: tid=0 2023-10-16T11:24:52.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.191+0000 7fa9a559d740 1 client.11652 shutdown 2023-10-16T11:24:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.191+0000 7fa9a559d740 1 -- 172.21.15.84:0/127922235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57170 msgr2=0x55d4b6c4fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.191+0000 7fa9a559d740 1 --2- 172.21.15.84:0/127922235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57170 0x55d4b6c4fce0 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7fa98c04c040 tx=0x7fa98c004ab0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.191+0000 7fa9a559d740 1 -- 172.21.15.84:0/127922235 shutdown_connections 2023-10-16T11:24:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.191+0000 7fa9a559d740 1 --2- 172.21.15.84:0/127922235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d4b6c528b0 0x55d4b6c58e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.191+0000 7fa9a559d740 1 --2- 172.21.15.84:0/127922235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c57550 0x55d4b6c50220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.191+0000 7fa9a559d740 1 --2- 172.21.15.84:0/127922235 >> [v2:172.21.15.157:6833/3992377882,v1:172.21.15.157:6835/3992377882] conn(0x55d4b6c59380 0x55d4b6c884f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.191+0000 7fa9a559d740 1 --2- 172.21.15.84:0/127922235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57170 0x55d4b6c4fce0 unknown :-1 s=CLOSED pgs=2703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.191+0000 7fa97affd640 0 client.11652 ms_handle_reset on v2:172.21.15.157:6833/3992377882 2023-10-16T11:24:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.191+0000 7fa9a559d740 1 -- 172.21.15.84:0/127922235 >> 172.21.15.84:0/127922235 conn(0x55d4b6c2a5b0 msgr2=0x55d4b6c2e020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:52.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.191+0000 7fa9a559d740 1 -- 172.21.15.84:0/127922235 shutdown_connections 2023-10-16T11:24:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.191+0000 7fa9a559d740 1 -- 172.21.15.84:0/127922235 wait complete. 2023-10-16T11:24:52.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.191+0000 7fa9a559d740 20 client.11652 trim_cache size 0 max 16384 2023-10-16T11:24:52.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.207+0000 7fa9a559d740 1 Processor -- start 2023-10-16T11:24:52.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.207+0000 7fa9a559d740 1 -- start start 2023-10-16T11:24:52.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.208+0000 7fa9a559d740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d4b6c528b0 0x55d4b6c43940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.208+0000 7fa9a559d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c57170 0x55d4b6c2db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.208+0000 7fa9a559d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57550 0x55d4b6c2e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.208+0000 7fa9a559d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b68f9f20 con 0x55d4b6c57550 2023-10-16T11:24:52.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.208+0000 7fa9a559d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b6c75f70 con 0x55d4b6c57170 2023-10-16T11:24:52.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.208+0000 7fa9a559d740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4b6c88fd0 con 0x55d4b6c528b0 2023-10-16T11:24:52.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.208+0000 7fa9a0ed0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57550 0x55d4b6c2e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.208+0000 7fa9a0ed0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57550 0x55d4b6c2e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51394/0 (socket says 172.21.15.84:51394) 2023-10-16T11:24:52.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.208+0000 7fa9a0ed0640 1 -- 172.21.15.84:0/79522480 learned_addr learned my addr 172.21.15.84:0/79522480 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:52.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.209+0000 7fa9a0ed0640 1 -- 172.21.15.84:0/79522480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d4b6c528b0 msgr2=0x55d4b6c43940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:52.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.209+0000 7fa9a0ed0640 1 --2- 172.21.15.84:0/79522480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d4b6c528b0 0x55d4b6c43940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.209+0000 7fa9a0ed0640 1 -- 172.21.15.84:0/79522480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c57170 msgr2=0x55d4b6c2db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:52.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.209+0000 7fa99b7fe640 1 --2- 172.21.15.84:0/79522480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c57170 0x55d4b6c2db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.209+0000 7fa9a0ed0640 1 --2- 172.21.15.84:0/79522480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c57170 0x55d4b6c2db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.209+0000 7fa9a0ed0640 1 -- 172.21.15.84:0/79522480 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4b6b32b80 con 0x55d4b6c57550 2023-10-16T11:24:52.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.209+0000 7fa9a0ed0640 1 --2- 172.21.15.84:0/79522480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57550 0x55d4b6c2e0b0 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7fa994003c90 tx=0x7fa994004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede1256ff940e884 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:52.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.209+0000 7fa9927fc640 1 -- 172.21.15.84:0/79522480 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99400d930 con 0x55d4b6c57550 2023-10-16T11:24:52.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.209+0000 7fa9927fc640 1 -- 172.21.15.84:0/79522480 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99400da90 con 0x55d4b6c57550 2023-10-16T11:24:52.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.209+0000 7fa9927fc640 1 -- 172.21.15.84:0/79522480 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99400de40 con 0x55d4b6c57550 2023-10-16T11:24:52.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.210+0000 7fa9a559d740 1 -- 172.21.15.84:0/79522480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57550 msgr2=0x55d4b6c2e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.210+0000 7fa9a559d740 1 --2- 172.21.15.84:0/79522480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57550 0x55d4b6c2e0b0 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7fa994003c90 tx=0x7fa994004150 comp rx=0 tx=0).stop 2023-10-16T11:24:52.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.210+0000 7fa9a559d740 1 -- 172.21.15.84:0/79522480 shutdown_connections 2023-10-16T11:24:52.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.210+0000 7fa9a559d740 1 --2- 172.21.15.84:0/79522480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c57550 0x55d4b6c2e0b0 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.210+0000 7fa9a559d740 1 --2- 172.21.15.84:0/79522480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c57170 0x55d4b6c2db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.210+0000 7fa9a559d740 1 --2- 172.21.15.84:0/79522480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d4b6c528b0 0x55d4b6c43940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.210+0000 7fa9a559d740 1 -- 172.21.15.84:0/79522480 >> 172.21.15.84:0/79522480 conn(0x55d4b6c2a5b0 msgr2=0x55d4b6c58c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:52.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.210+0000 7fa9a559d740 1 -- 172.21.15.84:0/79522480 shutdown_connections 2023-10-16T11:24:52.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.210+0000 7fa9a559d740 1 -- 172.21.15.84:0/79522480 wait complete. 2023-10-16T11:24:52.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.211+0000 7fa9a559d740 1 -- start start 2023-10-16T11:24:52.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.211+0000 7fa9a559d740 1 Processor -- start 2023-10-16T11:24:52.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.212+0000 7fa9a559d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c528b0 0x55d4b6cdd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.212+0000 7fa9a559d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c57170 0x55d4b6c4fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.212+0000 7fa9a559d740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d4b6c57550 0x55d4b6c503c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.212+0000 7fa9a559d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b6c555f0 con 0x55d4b6c528b0 2023-10-16T11:24:52.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.212+0000 7fa9a559d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b6c526d0 con 0x55d4b6c57170 2023-10-16T11:24:52.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.212+0000 7fa9a559d740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4b6cdef70 con 0x55d4b6c57550 2023-10-16T11:24:52.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.212+0000 7fa9a559d740 10 client.0 _ll_register_callbacks cb 0x7fa98c009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:24:52.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.212+0000 7fa99bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c528b0 0x55d4b6cdd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.212+0000 7fa99bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c528b0 0x55d4b6cdd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51404/0 (socket says 172.21.15.84:51404) 2023-10-16T11:24:52.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.212+0000 7fa99bfff640 1 -- 172.21.15.84:0/642696553 learned_addr learned my addr 172.21.15.84:0/642696553 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:52.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.212+0000 7fa99b7fe640 1 --2- 172.21.15.84:0/642696553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c57170 0x55d4b6c4fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.212+0000 7fa99bfff640 1 -- 172.21.15.84:0/642696553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d4b6c57550 msgr2=0x55d4b6c503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:52.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.212+0000 7fa99bfff640 1 --2- 172.21.15.84:0/642696553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d4b6c57550 0x55d4b6c503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.212+0000 7fa99bfff640 1 -- 172.21.15.84:0/642696553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c57170 msgr2=0x55d4b6c4fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:52.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.212+0000 7fa99bfff640 1 --2- 172.21.15.84:0/642696553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c57170 0x55d4b6c4fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.212+0000 7fa99bfff640 1 -- 172.21.15.84:0/642696553 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa994003970 con 0x55d4b6c528b0 2023-10-16T11:24:52.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.213+0000 7fa99bfff640 1 --2- 172.21.15.84:0/642696553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c528b0 0x55d4b6cdd320 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7fa98c02bef0 tx=0x7fa98c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52d7c1885a91f34 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:52.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.213+0000 7fa97affd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T11:24:52.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.213+0000 7fa97affd640 1 -- 172.21.15.84:0/642696553 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c004510 con 0x55d4b6c528b0 2023-10-16T11:24:52.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.213+0000 7fa97affd640 1 -- 172.21.15.84:0/642696553 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7fa98c004670 con 0x55d4b6c528b0 2023-10-16T11:24:52.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.213+0000 7fa9a559d740 1 -- 172.21.15.84:0/642696553 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa98c04a410 con 0x55d4b6c528b0 2023-10-16T11:24:52.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.213+0000 7fa97affd640 1 -- 172.21.15.84:0/642696553 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa98c034a30 con 0x55d4b6c528b0 2023-10-16T11:24:52.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.213+0000 7fa9a559d740 10 client.11658 fetch_fsmap learned FSMap version 120 2023-10-16T11:24:52.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.213+0000 7fa9a559d740 1 -- 172.21.15.84:0/642696553 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=2+}) v3 -- 0x55d4b6b58170 con 0x55d4b6c528b0 2023-10-16T11:24:52.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.214+0000 7fa97affd640 1 -- 172.21.15.84:0/642696553 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c04a410 con 0x55d4b6c528b0 2023-10-16T11:24:52.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.214+0000 7fa97affd640 1 -- 172.21.15.84:0/642696553 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 120) v1 ==== 3891+0+0 (secure 0 0 0) 0x7fa98c0733f0 con 0x55d4b6c528b0 2023-10-16T11:24:52.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.214+0000 7fa9a559d740 10 client.11658 fetch_fsmap finished waiting for FSMap version 120 2023-10-16T11:24:52.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.214+0000 7fa9a559d740 10 client.11658 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-16T11:24:52.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.214+0000 7fa9a559d740 20 client.11658 populate_metadata read hostname 'smithi084' 2023-10-16T11:24:52.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.214+0000 7fa9a559d740 1 --2- 172.21.15.84:0/642696553 >> [v2:172.21.15.157:6833/3992377882,v1:172.21.15.157:6835/3992377882] conn(0x55d4b6c55a10 0x55d4b6c55df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.214+0000 7fa9a559d740 4 client.11658 mds_command: new command op to 7474 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T11:24:52.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.214+0000 7fa9a559d740 1 -- 172.21.15.84:0/642696553 --> [v2:172.21.15.157:6833/3992377882,v1:172.21.15.157:6835/3992377882] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d4b6b8e700 con 0x55d4b6c55a10 2023-10-16T11:24:52.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.214+0000 7fa99b7fe640 1 --2- 172.21.15.84:0/642696553 >> [v2:172.21.15.157:6833/3992377882,v1:172.21.15.157:6835/3992377882] conn(0x55d4b6c55a10 0x55d4b6c55df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.215+0000 7fa99b7fe640 1 --2- 172.21.15.84:0/642696553 >> [v2:172.21.15.157:6833/3992377882,v1:172.21.15.157:6835/3992377882] conn(0x55d4b6c55a10 0x55d4b6c55df0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19137ec73983178b server_cookie=4d29972e9663d4b0 in_seq=0 out_seq=0 2023-10-16T11:24:52.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.215+0000 7fa97affd640 10 client.11658 ms_handle_connect on v2:172.21.15.157:6833/3992377882 2023-10-16T11:24:52.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.216+0000 7fa97affd640 1 -- 172.21.15.84:0/642696553 <== mds.0 v2:172.21.15.157:6833/3992377882 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55d4b6b8e700 con 0x55d4b6c55a10 2023-10-16T11:24:52.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.216+0000 7fa97affd640 10 client.11658 handle_command_reply: tid=0 2023-10-16T11:24:52.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.216+0000 7fa9a559d740 1 client.11658 shutdown 2023-10-16T11:24:52.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.216+0000 7fa9a559d740 1 -- 172.21.15.84:0/642696553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c528b0 msgr2=0x55d4b6cdd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.216+0000 7fa9a559d740 1 --2- 172.21.15.84:0/642696553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c528b0 0x55d4b6cdd320 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7fa98c02bef0 tx=0x7fa98c004a40 comp rx=0 tx=0).stop 2023-10-16T11:24:52.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.216+0000 7fa9a559d740 1 -- 172.21.15.84:0/642696553 shutdown_connections 2023-10-16T11:24:52.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.216+0000 7fa9a559d740 1 --2- 172.21.15.84:0/642696553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d4b6c57550 0x55d4b6c503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.216+0000 7fa9a559d740 1 --2- 172.21.15.84:0/642696553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4b6c57170 0x55d4b6c4fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.216+0000 7fa9a559d740 1 --2- 172.21.15.84:0/642696553 >> [v2:172.21.15.157:6833/3992377882,v1:172.21.15.157:6835/3992377882] conn(0x55d4b6c55a10 0x55d4b6c55df0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.216+0000 7fa9a559d740 1 --2- 172.21.15.84:0/642696553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4b6c528b0 0x55d4b6cdd320 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.216+0000 7fa97affd640 0 client.11658 ms_handle_reset on v2:172.21.15.157:6833/3992377882 2023-10-16T11:24:52.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.216+0000 7fa9a559d740 1 -- 172.21.15.84:0/642696553 >> 172.21.15.84:0/642696553 conn(0x55d4b6c2a5b0 msgr2=0x55d4b6c5dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:52.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.216+0000 7fa9a559d740 1 -- 172.21.15.84:0/642696553 shutdown_connections 2023-10-16T11:24:52.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.216+0000 7fa9a559d740 1 -- 172.21.15.84:0/642696553 wait complete. 2023-10-16T11:24:52.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.216+0000 7fa9a559d740 20 client.11658 trim_cache size 0 max 16384 2023-10-16T11:24:52.231 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:24:52.231 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "63fb4d68-f881-4694-b880-d10369105c0e", 2023-10-16T11:24:52.231 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-16T11:24:52.231 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 7474, 2023-10-16T11:24:52.231 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-16T11:24:52.232 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-16T11:24:52.232 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "backup_fs", 2023-10-16T11:24:52.232 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 11.310922929, 2023-10-16T11:24:52.232 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 120, 2023-10-16T11:24:52.232 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 221, 2023-10-16T11:24:52.233 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 221, 2023-10-16T11:24:52.233 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 31.518434442 2023-10-16T11:24:52.233 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:24:52.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.217+0000 7fa9a16d1640 1 -- 172.21.15.84:0/3431554541 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa97c06af90 msgr2=0x7fa97c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.217+0000 7fa9a16d1640 1 --2- 172.21.15.84:0/3431554541 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa97c06af90 0x7fa97c06d450 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa98c036000 tx=0x7fa98c02e000 comp rx=0 tx=0).stop 2023-10-16T11:24:52.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.217+0000 7fa9a16d1640 1 -- 172.21.15.84:0/3431554541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa99c0cded0 msgr2=0x7fa99c14ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.217+0000 7fa9a16d1640 1 --2- 172.21.15.84:0/3431554541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa99c0cded0 0x7fa99c14ff60 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c4d0 tx=0x7fa98400c990 comp rx=0 tx=0).stop 2023-10-16T11:24:52.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.217+0000 7fa9a16d1640 1 -- 172.21.15.84:0/3431554541 shutdown_connections 2023-10-16T11:24:52.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.217+0000 7fa9a16d1640 1 --2- 172.21.15.84:0/3431554541 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fa97c06af90 0x7fa97c06d450 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.217+0000 7fa9a16d1640 1 --2- 172.21.15.84:0/3431554541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa99c15cd70 0x7fa99c0aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.217+0000 7fa9a16d1640 1 --2- 172.21.15.84:0/3431554541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa99c0cded0 0x7fa99c14ff60 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.217+0000 7fa9a16d1640 1 --2- 172.21.15.84:0/3431554541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa99c0b8480 0x7fa99c1f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.217+0000 7fa9a16d1640 1 -- 172.21.15.84:0/3431554541 >> 172.21.15.84:0/3431554541 conn(0x7fa99c0cf570 msgr2=0x7fa99c0b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:52.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.217+0000 7fa9a16d1640 1 -- 172.21.15.84:0/3431554541 shutdown_connections 2023-10-16T11:24:52.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.217+0000 7fa9a16d1640 1 -- 172.21.15.84:0/3431554541 wait complete. 2023-10-16T11:24:52.237 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T11:24:52.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.439+0000 7f81a99e3640 1 Processor -- start 2023-10-16T11:24:52.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.439+0000 7f81a99e3640 1 -- start start 2023-10-16T11:24:52.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.441+0000 7f81a99e3640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f81a40addb0 0x7f81a40caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.441+0000 7f81a99e3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f81a406a4b0 0x7f81a41650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.441+0000 7f81a99e3640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a40cb140 con 0x7f81a40c9000 2023-10-16T11:24:52.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.441+0000 7f81a99e3640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a4161a50 con 0x7f81a40addb0 2023-10-16T11:24:52.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.441+0000 7f81a99e3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a406b030 con 0x7f81a406a4b0 2023-10-16T11:24:52.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.441+0000 7f81a2ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f81a40addb0 0x7f81a40caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.441+0000 7f81a2ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f81a40addb0 0x7f81a40caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51406/0 (socket says 172.21.15.84:51406) 2023-10-16T11:24:52.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.441+0000 7f81a2ffd640 1 -- 172.21.15.84:0/3313137060 learned_addr learned my addr 172.21.15.84:0/3313137060 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:52.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.441+0000 7f81a27fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f81a40c9000 0x7f81a4069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47990/0 (socket says 172.21.15.84:47990) 2023-10-16T11:24:52.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.441+0000 7f81a2ffd640 1 -- 172.21.15.84:0/3313137060 >> v2:172.21.15.157:3300/0 conn(0x7f81a406a4b0 msgr2=0x7f81a41650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:52.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.441+0000 7f81a2ffd640 1 --2- 172.21.15.84:0/3313137060 >> v2:172.21.15.157:3300/0 conn(0x7f81a406a4b0 0x7f81a41650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.441+0000 7f81a2ffd640 1 -- 172.21.15.84:0/3313137060 >> v1:172.21.15.84:6789/0 conn(0x7f81a40c9000 legacy=0x7f81a4069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.442+0000 7f81a2ffd640 1 -- 172.21.15.84:0/3313137060 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a406b310 con 0x7f81a40addb0 2023-10-16T11:24:52.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.441+0000 7f81a1ffb640 1 -- 172.21.15.84:0/3313137060 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2442449177 0 0) 0x7f81a40cb140 con 0x7f81a40c9000 2023-10-16T11:24:52.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.443+0000 7f81a2ffd640 1 --2- 172.21.15.84:0/3313137060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81a40addb0 0x7f81a40caa80 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f81a40cdd20 tx=0x7f819402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b802264e4aada3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:52.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.443+0000 7f81a1ffb640 1 -- 172.21.15.84:0/3313137060 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194030860 con 0x7f81a40addb0 2023-10-16T11:24:52.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.443+0000 7f81a1ffb640 1 -- 172.21.15.84:0/3313137060 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81940309c0 con 0x7f81a40addb0 2023-10-16T11:24:52.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.443+0000 7f81a1ffb640 1 -- 172.21.15.84:0/3313137060 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194030d50 con 0x7f81a40addb0 2023-10-16T11:24:52.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.443+0000 7f81a99e3640 1 -- 172.21.15.84:0/3313137060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81a40addb0 msgr2=0x7f81a40caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.443+0000 7f81a99e3640 1 --2- 172.21.15.84:0/3313137060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81a40addb0 0x7f81a40caa80 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f81a40cdd20 tx=0x7f819402b9a0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.444+0000 7f81a99e3640 1 -- 172.21.15.84:0/3313137060 shutdown_connections 2023-10-16T11:24:52.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.444+0000 7f81a99e3640 1 --2- 172.21.15.84:0/3313137060 >> v2:172.21.15.157:3300/0 conn(0x7f81a406a4b0 0x7f81a41650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.444+0000 7f81a99e3640 1 --2- 172.21.15.84:0/3313137060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81a40addb0 0x7f81a40caa80 unknown :-1 s=CLOSED pgs=2707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.444+0000 7f81a99e3640 1 -- 172.21.15.84:0/3313137060 >> 172.21.15.84:0/3313137060 conn(0x7f81a40ce9b0 msgr2=0x7f81a40cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:52.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.444+0000 7f81a99e3640 1 -- 172.21.15.84:0/3313137060 shutdown_connections 2023-10-16T11:24:52.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.444+0000 7f81a99e3640 1 -- 172.21.15.84:0/3313137060 wait complete. 2023-10-16T11:24:52.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.445+0000 7f81a99e3640 1 Processor -- start 2023-10-16T11:24:52.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.446+0000 7f81a99e3640 1 -- start start 2023-10-16T11:24:52.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.446+0000 7f81a99e3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81a406a4b0 0x7f81a41f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.446+0000 7f81a99e3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81a40addb0 0x7f81a40c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.446+0000 7f81a99e3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81a40c9000 0x7f81a415ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.446+0000 7f81a99e3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a40c3780 con 0x7f81a406a4b0 2023-10-16T11:24:52.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.446+0000 7f81a99e3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a41f7320 con 0x7f81a40c9000 2023-10-16T11:24:52.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.446+0000 7f81a99e3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a41f7600 con 0x7f81a40addb0 2023-10-16T11:24:52.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.447+0000 7f81a2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81a406a4b0 0x7f81a41f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.447+0000 7f81a2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81a406a4b0 0x7f81a41f6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51414/0 (socket says 172.21.15.84:51414) 2023-10-16T11:24:52.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.447+0000 7f81a2ffd640 1 -- 172.21.15.84:0/2976297643 learned_addr learned my addr 172.21.15.84:0/2976297643 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:52.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.447+0000 7f81a2ffd640 1 -- 172.21.15.84:0/2976297643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81a40addb0 msgr2=0x7f81a40c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.447+0000 7f81a2ffd640 1 --2- 172.21.15.84:0/2976297643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81a40addb0 0x7f81a40c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.447+0000 7f81a2ffd640 1 -- 172.21.15.84:0/2976297643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81a40c9000 msgr2=0x7f81a415ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.447+0000 7f81a2ffd640 1 --2- 172.21.15.84:0/2976297643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81a40c9000 0x7f81a415ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.447+0000 7f81a2ffd640 1 -- 172.21.15.84:0/2976297643 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81940094d0 con 0x7f81a406a4b0 2023-10-16T11:24:52.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.447+0000 7f81a2ffd640 1 --2- 172.21.15.84:0/2976297643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81a406a4b0 0x7f81a41f6a60 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f8194000c00 tx=0x7f8194004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:52.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.448+0000 7f818bfff640 1 -- 172.21.15.84:0/2976297643 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194039d40 con 0x7f81a406a4b0 2023-10-16T11:24:52.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.448+0000 7f818bfff640 1 -- 172.21.15.84:0/2976297643 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8194043400 con 0x7f81a406a4b0 2023-10-16T11:24:52.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.448+0000 7f818bfff640 1 -- 172.21.15.84:0/2976297643 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194039ea0 con 0x7f81a406a4b0 2023-10-16T11:24:52.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.448+0000 7f81a99e3640 1 -- 172.21.15.84:0/2976297643 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a41f78e0 con 0x7f81a406a4b0 2023-10-16T11:24:52.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.448+0000 7f81a99e3640 1 -- 172.21.15.84:0/2976297643 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a41f6fa0 con 0x7f81a406a4b0 2023-10-16T11:24:52.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.449+0000 7f81a99e3640 1 -- 172.21.15.84:0/2976297643 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a40c3c60 con 0x7f81a406a4b0 2023-10-16T11:24:52.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.450+0000 7f818bfff640 1 -- 172.21.15.84:0/2976297643 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f81940437b0 con 0x7f81a406a4b0 2023-10-16T11:24:52.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.451+0000 7f818bfff640 1 --2- 172.21.15.84:0/2976297643 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f819006af90 0x7f819006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.451+0000 7f818bfff640 1 -- 172.21.15.84:0/2976297643 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f819403e070 con 0x7f81a406a4b0 2023-10-16T11:24:52.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.451+0000 7f81a27fc640 1 --2- 172.21.15.84:0/2976297643 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f819006af90 0x7f819006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.452+0000 7f81a27fc640 1 --2- 172.21.15.84:0/2976297643 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f819006af90 0x7f819006d450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8198007fd0 tx=0x7f8198007960 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:52.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.455+0000 7f818bfff640 1 -- 172.21.15.84:0/2976297643 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f81a40c3c60 con 0x7f81a406a4b0 2023-10-16T11:24:52.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.656+0000 7f81a99e3640 1 -- 172.21.15.84:0/2976297643 --> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f81a406b8d0 con 0x7f819006af90 2023-10-16T11:24:52.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.660+0000 7f818bfff640 1 -- 172.21.15.84:0/2976297643 <== mgr.11076 v2:172.21.15.84:6832/1425409017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f81980070d0 con 0x7f819006af90 2023-10-16T11:24:52.662 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T11:24:52.662 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:52.663 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephfs" 2023-10-16T11:24:52.663 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:24:52.663 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:24:52.663 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "backup_fs" 2023-10-16T11:24:52.663 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:24:52.664 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T11:24:52.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.662+0000 7f81a99e3640 1 -- 172.21.15.84:0/2976297643 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f819006af90 msgr2=0x7f819006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.662+0000 7f81a99e3640 1 --2- 172.21.15.84:0/2976297643 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f819006af90 0x7f819006d450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8198007fd0 tx=0x7f8198007960 comp rx=0 tx=0).stop 2023-10-16T11:24:52.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.662+0000 7f81a99e3640 1 -- 172.21.15.84:0/2976297643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81a406a4b0 msgr2=0x7f81a41f6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.662+0000 7f81a99e3640 1 --2- 172.21.15.84:0/2976297643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81a406a4b0 0x7f81a41f6a60 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f8194000c00 tx=0x7f8194004730 comp rx=0 tx=0).stop 2023-10-16T11:24:52.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.662+0000 7f81a99e3640 1 -- 172.21.15.84:0/2976297643 shutdown_connections 2023-10-16T11:24:52.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.662+0000 7f81a99e3640 1 --2- 172.21.15.84:0/2976297643 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f819006af90 0x7f819006d450 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.662+0000 7f81a99e3640 1 --2- 172.21.15.84:0/2976297643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81a40c9000 0x7f81a415ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.662+0000 7f81a99e3640 1 --2- 172.21.15.84:0/2976297643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81a40addb0 0x7f81a40c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.662+0000 7f81a99e3640 1 --2- 172.21.15.84:0/2976297643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81a406a4b0 0x7f81a41f6a60 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.662+0000 7f81a99e3640 1 -- 172.21.15.84:0/2976297643 >> 172.21.15.84:0/2976297643 conn(0x7f81a40ce9b0 msgr2=0x7f81a40ae3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:52.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.662+0000 7f81a99e3640 1 -- 172.21.15.84:0/2976297643 shutdown_connections 2023-10-16T11:24:52.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.662+0000 7f81a99e3640 1 -- 172.21.15.84:0/2976297643 wait complete. 2023-10-16T11:24:52.682 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T11:24:52.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.892+0000 7fdc7579e640 1 Processor -- start 2023-10-16T11:24:52.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.892+0000 7fdc7579e640 1 -- start start 2023-10-16T11:24:52.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.894+0000 7fdc7579e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc7015ad70 0x7fdc70069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.894+0000 7fdc7579e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdc700caac0 0x7fdc7006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.894+0000 7fdc7579e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc700cb030 con 0x7fdc700c9ca0 2023-10-16T11:24:52.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.894+0000 7fdc7579e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc700c91c0 con 0x7fdc700caac0 2023-10-16T11:24:52.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.894+0000 7fdc7579e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc700c94a0 con 0x7fdc7015ad70 2023-10-16T11:24:52.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.894+0000 7fdc6f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdc700caac0 0x7fdc7006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.894+0000 7fdc6f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdc700caac0 0x7fdc7006a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51428/0 (socket says 172.21.15.84:51428) 2023-10-16T11:24:52.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.894+0000 7fdc6f7fe640 1 -- 172.21.15.84:0/2791735218 learned_addr learned my addr 172.21.15.84:0/2791735218 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:52.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.894+0000 7fdc6f7fe640 1 -- 172.21.15.84:0/2791735218 >> v2:172.21.15.157:3300/0 conn(0x7fdc7015ad70 msgr2=0x7fdc70069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:52.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.894+0000 7fdc6effd640 1 --2- 172.21.15.84:0/2791735218 >> v2:172.21.15.157:3300/0 conn(0x7fdc7015ad70 0x7fdc70069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.894+0000 7fdc6f7fe640 1 --2- 172.21.15.84:0/2791735218 >> v2:172.21.15.157:3300/0 conn(0x7fdc7015ad70 0x7fdc70069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.894+0000 7fdc6f7fe640 1 -- 172.21.15.84:0/2791735218 >> v1:172.21.15.157:6789/0 conn(0x7fdc700c9ca0 legacy=0x7fdc700ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:52.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.895+0000 7fdc6f7fe640 1 -- 172.21.15.84:0/2791735218 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc700c9780 con 0x7fdc700caac0 2023-10-16T11:24:52.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.895+0000 7fdc6effd640 1 --2- 172.21.15.84:0/2791735218 >> v2:172.21.15.157:3300/0 conn(0x7fdc7015ad70 0x7fdc70069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:52.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.896+0000 7fdc6f7fe640 1 --2- 172.21.15.84:0/2791735218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc700caac0 0x7fdc7006a2e0 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7fdc700cf780 tx=0x7fdc6802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678b220620a29a30 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:52.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.896+0000 7fdc6e7fc640 1 -- 172.21.15.84:0/2791735218 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6802ca80 con 0x7fdc700caac0 2023-10-16T11:24:52.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.896+0000 7fdc6e7fc640 1 -- 172.21.15.84:0/2791735218 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6802cbe0 con 0x7fdc700caac0 2023-10-16T11:24:52.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.896+0000 7fdc6e7fc640 1 -- 172.21.15.84:0/2791735218 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc680343f0 con 0x7fdc700caac0 2023-10-16T11:24:52.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.896+0000 7fdc7579e640 1 -- 172.21.15.84:0/2791735218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc700caac0 msgr2=0x7fdc7006a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.896+0000 7fdc7579e640 1 --2- 172.21.15.84:0/2791735218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc700caac0 0x7fdc7006a2e0 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7fdc700cf780 tx=0x7fdc6802bee0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.897+0000 7fdc7579e640 1 -- 172.21.15.84:0/2791735218 shutdown_connections 2023-10-16T11:24:52.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.897+0000 7fdc7579e640 1 --2- 172.21.15.84:0/2791735218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc700caac0 0x7fdc7006a2e0 unknown :-1 s=CLOSED pgs=2709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.897+0000 7fdc7579e640 1 --2- 172.21.15.84:0/2791735218 >> v2:172.21.15.157:3300/0 conn(0x7fdc7015ad70 0x7fdc70069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.897+0000 7fdc7579e640 1 -- 172.21.15.84:0/2791735218 >> 172.21.15.84:0/2791735218 conn(0x7fdc700ce9b0 msgr2=0x7fdc700cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:52.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.897+0000 7fdc7579e640 1 -- 172.21.15.84:0/2791735218 shutdown_connections 2023-10-16T11:24:52.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.897+0000 7fdc7579e640 1 -- 172.21.15.84:0/2791735218 wait complete. 2023-10-16T11:24:52.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.899+0000 7fdc7579e640 1 Processor -- start 2023-10-16T11:24:52.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.899+0000 7fdc7579e640 1 -- start start 2023-10-16T11:24:52.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.900+0000 7fdc7579e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc700c9ca0 0x7fdc70159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.900+0000 7fdc7579e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc700caac0 0x7fdc7015b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.900+0000 7fdc7579e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc7015ad70 0x7fdc700b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.900+0000 7fdc7579e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc701595c0 con 0x7fdc700c9ca0 2023-10-16T11:24:52.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.900+0000 7fdc7579e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7015bc80 con 0x7fdc7015ad70 2023-10-16T11:24:52.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.900+0000 7fdc7579e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc700c5b20 con 0x7fdc700caac0 2023-10-16T11:24:52.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.900+0000 7fdc6effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc700c9ca0 0x7fdc70159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.900+0000 7fdc6effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc700c9ca0 0x7fdc70159080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51442/0 (socket says 172.21.15.84:51442) 2023-10-16T11:24:52.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.900+0000 7fdc6effd640 1 -- 172.21.15.84:0/2891542482 learned_addr learned my addr 172.21.15.84:0/2891542482 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:52.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.901+0000 7fdc667fc640 1 --2- 172.21.15.84:0/2891542482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc700caac0 0x7fdc7015b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.901+0000 7fdc6effd640 1 -- 172.21.15.84:0/2891542482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc700caac0 msgr2=0x7fdc7015b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.901+0000 7fdc6effd640 1 --2- 172.21.15.84:0/2891542482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc700caac0 0x7fdc7015b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.901+0000 7fdc6effd640 1 -- 172.21.15.84:0/2891542482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc7015ad70 msgr2=0x7fdc700b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:52.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.901+0000 7fdc6effd640 1 --2- 172.21.15.84:0/2891542482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc7015ad70 0x7fdc700b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:52.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.901+0000 7fdc6effd640 1 -- 172.21.15.84:0/2891542482 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc68009b30 con 0x7fdc700c9ca0 2023-10-16T11:24:52.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.901+0000 7fdc6effd640 1 --2- 172.21.15.84:0/2891542482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc700c9ca0 0x7fdc70159080 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000c7f0 tx=0x7fdc6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:52.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.901+0000 7fdc67fff640 1 -- 172.21.15.84:0/2891542482 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc60007820 con 0x7fdc700c9ca0 2023-10-16T11:24:52.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.901+0000 7fdc67fff640 1 -- 172.21.15.84:0/2891542482 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc60007980 con 0x7fdc700c9ca0 2023-10-16T11:24:52.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.901+0000 7fdc67fff640 1 -- 172.21.15.84:0/2891542482 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc60007cf0 con 0x7fdc700c9ca0 2023-10-16T11:24:52.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.901+0000 7fdc7579e640 1 -- 172.21.15.84:0/2891542482 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc700c5e00 con 0x7fdc700c9ca0 2023-10-16T11:24:52.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.902+0000 7fdc7579e640 1 -- 172.21.15.84:0/2891542482 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc700c63c0 con 0x7fdc700c9ca0 2023-10-16T11:24:52.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.903+0000 7fdc7579e640 1 -- 172.21.15.84:0/2891542482 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7004e160 con 0x7fdc700c9ca0 2023-10-16T11:24:52.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.904+0000 7fdc67fff640 1 -- 172.21.15.84:0/2891542482 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 132) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdc600113f0 con 0x7fdc700c9ca0 2023-10-16T11:24:52.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.905+0000 7fdc67fff640 1 --2- 172.21.15.84:0/2891542482 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fdc5006af90 0x7fdc5006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:52.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.905+0000 7fdc67fff640 1 -- 172.21.15.84:0/2891542482 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7fdc60089fd0 con 0x7fdc700c9ca0 2023-10-16T11:24:52.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.905+0000 7fdc667fc640 1 --2- 172.21.15.84:0/2891542482 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fdc5006af90 0x7fdc5006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:52.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.905+0000 7fdc667fc640 1 --2- 172.21.15.84:0/2891542482 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fdc5006af90 0x7fdc5006d450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdc700ca5d0 tx=0x7fdc58009040 comp rx=0 tx=0).ready entity=mgr.11076 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:52.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:52.910+0000 7fdc67fff640 1 -- 172.21.15.84:0/2891542482 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdc60054a40 con 0x7fdc700c9ca0 2023-10-16T11:24:53.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:53.134+0000 7fdc7579e640 1 -- 172.21.15.84:0/2891542482 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fdc700ca600 con 0x7fdc700c9ca0 2023-10-16T11:24:54.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.007+0000 7fdc67fff640 1 -- 172.21.15.84:0/2891542482 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v133) v1 ==== 92+0+0 (secure 0 0 0) 0x7fdc6001f0a0 con 0x7fdc700c9ca0 2023-10-16T11:24:54.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.011+0000 7fdc7579e640 1 -- 172.21.15.84:0/2891542482 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fdc5006af90 msgr2=0x7fdc5006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:54.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.011+0000 7fdc7579e640 1 --2- 172.21.15.84:0/2891542482 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fdc5006af90 0x7fdc5006d450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdc700ca5d0 tx=0x7fdc58009040 comp rx=0 tx=0).stop 2023-10-16T11:24:54.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.011+0000 7fdc7579e640 1 -- 172.21.15.84:0/2891542482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc700c9ca0 msgr2=0x7fdc70159080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:54.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.011+0000 7fdc7579e640 1 --2- 172.21.15.84:0/2891542482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc700c9ca0 0x7fdc70159080 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000c7f0 tx=0x7fdc6000ccb0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.011+0000 7fdc7579e640 1 -- 172.21.15.84:0/2891542482 shutdown_connections 2023-10-16T11:24:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.011+0000 7fdc7579e640 1 --2- 172.21.15.84:0/2891542482 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7fdc5006af90 0x7fdc5006d450 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.011+0000 7fdc7579e640 1 --2- 172.21.15.84:0/2891542482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc7015ad70 0x7fdc700b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.011+0000 7fdc7579e640 1 --2- 172.21.15.84:0/2891542482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc700caac0 0x7fdc7015b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.011+0000 7fdc7579e640 1 --2- 172.21.15.84:0/2891542482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc700c9ca0 0x7fdc70159080 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.011+0000 7fdc7579e640 1 -- 172.21.15.84:0/2891542482 >> 172.21.15.84:0/2891542482 conn(0x7fdc700ce9b0 msgr2=0x7fdc700acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:54.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.011+0000 7fdc7579e640 1 -- 172.21.15.84:0/2891542482 shutdown_connections 2023-10-16T11:24:54.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.011+0000 7fdc7579e640 1 -- 172.21.15.84:0/2891542482 wait complete. 2023-10-16T11:24:54.037 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T11:24:54.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.228+0000 7f3e75cd7640 1 Processor -- start 2023-10-16T11:24:54.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.228+0000 7f3e75cd7640 1 -- start start 2023-10-16T11:24:54.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.229+0000 7f3e75cd7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e700cda70 0x7f3e7006aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:54.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.229+0000 7f3e75cd7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3e7006b3f0 0x7f3e7006ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:54.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.229+0000 7f3e75cd7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e700c3610 con 0x7f3e70159e40 2023-10-16T11:24:54.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.229+0000 7f3e75cd7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7006b880 con 0x7f3e7006b3f0 2023-10-16T11:24:54.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.229+0000 7f3e75cd7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e700cdeb0 con 0x7f3e700cda70 2023-10-16T11:24:54.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.229+0000 7f3e67fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e700cda70 0x7f3e7006aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:54.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.229+0000 7f3e67fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e700cda70 0x7f3e7006aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:52868/0 (socket says 172.21.15.84:52868) 2023-10-16T11:24:54.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.229+0000 7f3e67fff640 1 -- 172.21.15.84:0/3923066335 learned_addr learned my addr 172.21.15.84:0/3923066335 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:54.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.230+0000 7f3e6ffff640 1 --2- 172.21.15.84:0/3923066335 >> v2:172.21.15.157:3300/0 conn(0x7f3e7006b3f0 0x7f3e7006ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:54.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.230+0000 7f3e67fff640 1 -- 172.21.15.84:0/3923066335 >> v2:172.21.15.157:3300/0 conn(0x7f3e7006b3f0 msgr2=0x7f3e7006ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:54.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.230+0000 7f3e67fff640 1 --2- 172.21.15.84:0/3923066335 >> v2:172.21.15.157:3300/0 conn(0x7f3e7006b3f0 0x7f3e7006ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.230+0000 7f3e67fff640 1 -- 172.21.15.84:0/3923066335 >> v1:172.21.15.157:6789/0 conn(0x7f3e70159e40 legacy=0x7f3e700ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:54.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.230+0000 7f3e67fff640 1 -- 172.21.15.84:0/3923066335 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7015a220 con 0x7f3e700cda70 2023-10-16T11:24:54.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.231+0000 7f3e67fff640 1 --2- 172.21.15.84:0/3923066335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e700cda70 0x7f3e7006aeb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009870 tx=0x7f3e5802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed0bb7d404786491 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:54.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.232+0000 7f3e6effd640 1 -- 172.21.15.84:0/3923066335 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e58030840 con 0x7f3e700cda70 2023-10-16T11:24:54.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.232+0000 7f3e6effd640 1 -- 172.21.15.84:0/3923066335 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e580309a0 con 0x7f3e700cda70 2023-10-16T11:24:54.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.232+0000 7f3e6effd640 1 -- 172.21.15.84:0/3923066335 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e58030d30 con 0x7f3e700cda70 2023-10-16T11:24:54.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.232+0000 7f3e75cd7640 1 -- 172.21.15.84:0/3923066335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e700cda70 msgr2=0x7f3e7006aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:54.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.232+0000 7f3e75cd7640 1 --2- 172.21.15.84:0/3923066335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e700cda70 0x7f3e7006aeb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009870 tx=0x7f3e5802b980 comp rx=0 tx=0).stop 2023-10-16T11:24:54.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.232+0000 7f3e75cd7640 1 -- 172.21.15.84:0/3923066335 shutdown_connections 2023-10-16T11:24:54.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.232+0000 7f3e75cd7640 1 --2- 172.21.15.84:0/3923066335 >> v2:172.21.15.157:3300/0 conn(0x7f3e7006b3f0 0x7f3e7006ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.232+0000 7f3e75cd7640 1 --2- 172.21.15.84:0/3923066335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e700cda70 0x7f3e7006aeb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.232+0000 7f3e75cd7640 1 -- 172.21.15.84:0/3923066335 >> 172.21.15.84:0/3923066335 conn(0x7f3e700cfac0 msgr2=0x7f3e700cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:54.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.232+0000 7f3e75cd7640 1 -- 172.21.15.84:0/3923066335 shutdown_connections 2023-10-16T11:24:54.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.232+0000 7f3e75cd7640 1 -- 172.21.15.84:0/3923066335 wait complete. 2023-10-16T11:24:54.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.234+0000 7f3e75cd7640 1 Processor -- start 2023-10-16T11:24:54.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.234+0000 7f3e75cd7640 1 -- start start 2023-10-16T11:24:54.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.235+0000 7f3e75cd7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e7006b3f0 0x7f3e700c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:54.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.235+0000 7f3e75cd7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e700cda70 0x7f3e700b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:54.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.235+0000 7f3e75cd7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e70159e40 0x7f3e700c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:54.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.235+0000 7f3e75cd7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e700b6ce0 con 0x7f3e7006b3f0 2023-10-16T11:24:54.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.235+0000 7f3e75cd7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e700b6720 con 0x7f3e70159e40 2023-10-16T11:24:54.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.235+0000 7f3e75cd7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e700b6a00 con 0x7f3e700cda70 2023-10-16T11:24:54.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.235+0000 7f3e6f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e7006b3f0 0x7f3e700c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:54.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.235+0000 7f3e6f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e7006b3f0 0x7f3e700c25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51466/0 (socket says 172.21.15.84:51466) 2023-10-16T11:24:54.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.235+0000 7f3e6f7fe640 1 -- 172.21.15.84:0/2207058867 learned_addr learned my addr 172.21.15.84:0/2207058867 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:54.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.236+0000 7f3e67fff640 1 --2- 172.21.15.84:0/2207058867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e700cda70 0x7f3e700b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:54.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.236+0000 7f3e6ffff640 1 --2- 172.21.15.84:0/2207058867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e70159e40 0x7f3e700c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:54.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.236+0000 7f3e6f7fe640 1 -- 172.21.15.84:0/2207058867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e700cda70 msgr2=0x7f3e700b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:54.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.236+0000 7f3e6f7fe640 1 --2- 172.21.15.84:0/2207058867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e700cda70 0x7f3e700b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.236+0000 7f3e6f7fe640 1 -- 172.21.15.84:0/2207058867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e70159e40 msgr2=0x7f3e700c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:54.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.236+0000 7f3e6f7fe640 1 --2- 172.21.15.84:0/2207058867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e70159e40 0x7f3e700c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.236+0000 7f3e6f7fe640 1 -- 172.21.15.84:0/2207058867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e580094d0 con 0x7f3e7006b3f0 2023-10-16T11:24:54.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.236+0000 7f3e67fff640 1 --2- 172.21.15.84:0/2207058867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e700cda70 0x7f3e700b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:24:54.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.236+0000 7f3e6f7fe640 1 --2- 172.21.15.84:0/2207058867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e7006b3f0 0x7f3e700c25d0 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000c3c0 tx=0x7f3e6000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:54.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.236+0000 7f3e6cff9640 1 -- 172.21.15.84:0/2207058867 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e600052d0 con 0x7f3e7006b3f0 2023-10-16T11:24:54.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.236+0000 7f3e6cff9640 1 -- 172.21.15.84:0/2207058867 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e60005450 con 0x7f3e7006b3f0 2023-10-16T11:24:54.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.236+0000 7f3e6cff9640 1 -- 172.21.15.84:0/2207058867 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e60007260 con 0x7f3e7006b3f0 2023-10-16T11:24:54.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.237+0000 7f3e75cd7640 1 -- 172.21.15.84:0/2207058867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e70148ed0 con 0x7f3e7006b3f0 2023-10-16T11:24:54.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.237+0000 7f3e75cd7640 1 -- 172.21.15.84:0/2207058867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e700c2b10 con 0x7f3e7006b3f0 2023-10-16T11:24:54.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.238+0000 7f3e75cd7640 1 -- 172.21.15.84:0/2207058867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7004e160 con 0x7f3e7006b3f0 2023-10-16T11:24:54.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.239+0000 7f3e6cff9640 1 -- 172.21.15.84:0/2207058867 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 133) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f3e600073c0 con 0x7f3e7006b3f0 2023-10-16T11:24:54.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.240+0000 7f3e6cff9640 1 --2- 172.21.15.84:0/2207058867 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3e5006b010 0x7f3e5006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:54.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.240+0000 7f3e67fff640 1 -- 172.21.15.84:0/2207058867 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3e5006b010 msgr2=0x7f3e5006d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1425409017 2023-10-16T11:24:54.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.240+0000 7f3e6cff9640 1 -- 172.21.15.84:0/2207058867 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f3e60088f30 con 0x7f3e7006b3f0 2023-10-16T11:24:54.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.240+0000 7f3e67fff640 1 --2- 172.21.15.84:0/2207058867 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3e5006b010 0x7f3e5006d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:24:54.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.245+0000 7f3e6cff9640 1 -- 172.21.15.84:0/2207058867 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3e60053b40 con 0x7f3e7006b3f0 2023-10-16T11:24:54.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.435+0000 7f3e75cd7640 1 -- 172.21.15.84:0/2207058867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f3e700cab90 con 0x7f3e7006b3f0 2023-10-16T11:24:54.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.441+0000 7f3e67fff640 1 -- 172.21.15.84:0/2207058867 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3e5006b010 msgr2=0x7f3e5006d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1425409017 2023-10-16T11:24:54.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.441+0000 7f3e67fff640 1 --2- 172.21.15.84:0/2207058867 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3e5006b010 0x7f3e5006d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:24:54.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.441+0000 7f3e6cff9640 1 -- 172.21.15.84:0/2207058867 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v77)=0 v77) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3e60057160 con 0x7f3e7006b3f0 2023-10-16T11:24:54.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.444+0000 7f3e75cd7640 1 -- 172.21.15.84:0/2207058867 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3e5006b010 msgr2=0x7f3e5006d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:24:54.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.444+0000 7f3e75cd7640 1 --2- 172.21.15.84:0/2207058867 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3e5006b010 0x7f3e5006d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.444+0000 7f3e75cd7640 1 -- 172.21.15.84:0/2207058867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e7006b3f0 msgr2=0x7f3e700c25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:54.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.444+0000 7f3e75cd7640 1 --2- 172.21.15.84:0/2207058867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e7006b3f0 0x7f3e700c25d0 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000c3c0 tx=0x7f3e6000c880 comp rx=0 tx=0).stop 2023-10-16T11:24:54.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.444+0000 7f3e75cd7640 1 -- 172.21.15.84:0/2207058867 shutdown_connections 2023-10-16T11:24:54.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.444+0000 7f3e75cd7640 1 --2- 172.21.15.84:0/2207058867 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f3e5006b010 0x7f3e5006d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.444+0000 7f3e75cd7640 1 --2- 172.21.15.84:0/2207058867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e70159e40 0x7f3e700c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.444+0000 7f3e75cd7640 1 --2- 172.21.15.84:0/2207058867 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e700cda70 0x7f3e700b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.444+0000 7f3e75cd7640 1 --2- 172.21.15.84:0/2207058867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e7006b3f0 0x7f3e700c25d0 unknown :-1 s=CLOSED pgs=2714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.444+0000 7f3e75cd7640 1 -- 172.21.15.84:0/2207058867 >> 172.21.15.84:0/2207058867 conn(0x7f3e700cfac0 msgr2=0x7f3e7006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:54.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.444+0000 7f3e75cd7640 1 -- 172.21.15.84:0/2207058867 shutdown_connections 2023-10-16T11:24:54.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.444+0000 7f3e75cd7640 1 -- 172.21.15.84:0/2207058867 wait complete. 2023-10-16T11:24:54.464 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T11:24:54.526 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:54.525+0000 7ff51a445040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:24:54.553 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:54.551+0000 7f8f0fa56040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:24:54.580 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:54.579+0000 7ff51a445040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:24:54.610 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:54.608+0000 7f8f0fa56040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:24:54.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.667+0000 7f4a5b62d640 1 Processor -- start 2023-10-16T11:24:54.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.667+0000 7f4a5b62d640 1 -- start start 2023-10-16T11:24:54.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.668+0000 7f4a5b62d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4a5415af80 0x7f4a540cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:54.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.668+0000 7f4a5b62d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a5406faf0 0x7f4a5415f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:54.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.668+0000 7f4a5b62d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5415eff0 con 0x7f4a54065ab0 2023-10-16T11:24:54.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.668+0000 7f4a5b62d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a540d0930 con 0x7f4a5415af80 2023-10-16T11:24:54.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.668+0000 7f4a5b62d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5415ecc0 con 0x7f4a5406faf0 2023-10-16T11:24:54.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.669+0000 7f4a593c9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4a5415af80 0x7f4a540cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:54.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.669+0000 7f4a593c9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4a5415af80 0x7f4a540cb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51476/0 (socket says 172.21.15.84:51476) 2023-10-16T11:24:54.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.669+0000 7f4a593c9640 1 -- 172.21.15.84:0/848120998 learned_addr learned my addr 172.21.15.84:0/848120998 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:54.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.669+0000 7f4a59bca640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4a54065ab0 0x7f4a540ce9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48008/0 (socket says 172.21.15.84:48008) 2023-10-16T11:24:54.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.669+0000 7f4a58bc8640 1 --2- 172.21.15.84:0/848120998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a5406faf0 0x7f4a5415f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:54.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.669+0000 7f4a593c9640 1 -- 172.21.15.84:0/848120998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a5406faf0 msgr2=0x7f4a5415f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:54.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.669+0000 7f4a593c9640 1 --2- 172.21.15.84:0/848120998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a5406faf0 0x7f4a5415f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.669+0000 7f4a593c9640 1 -- 172.21.15.84:0/848120998 >> v1:172.21.15.84:6789/0 conn(0x7f4a54065ab0 legacy=0x7f4a540ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:54.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.669+0000 7f4a593c9640 1 -- 172.21.15.84:0/848120998 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5415b360 con 0x7f4a5415af80 2023-10-16T11:24:54.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.669+0000 7f4a4bfff640 1 -- 172.21.15.84:0/848120998 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370310388 0 0) 0x7f4a5415eff0 con 0x7f4a54065ab0 2023-10-16T11:24:54.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.669+0000 7f4a58bc8640 1 --2- 172.21.15.84:0/848120998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a5406faf0 0x7f4a5415f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:54.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.670+0000 7f4a593c9640 1 --2- 172.21.15.84:0/848120998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a5415af80 0x7f4a540cb1c0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009870 tx=0x7f4a5002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e25417c47480410 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:54.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.671+0000 7f4a4bfff640 1 -- 172.21.15.84:0/848120998 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a50030840 con 0x7f4a5415af80 2023-10-16T11:24:54.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.671+0000 7f4a4bfff640 1 -- 172.21.15.84:0/848120998 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a500309a0 con 0x7f4a5415af80 2023-10-16T11:24:54.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.671+0000 7f4a4bfff640 1 -- 172.21.15.84:0/848120998 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a50030d30 con 0x7f4a5415af80 2023-10-16T11:24:54.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.671+0000 7f4a5b62d640 1 -- 172.21.15.84:0/848120998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a5415af80 msgr2=0x7f4a540cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:54.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.671+0000 7f4a5b62d640 1 --2- 172.21.15.84:0/848120998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a5415af80 0x7f4a540cb1c0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009870 tx=0x7f4a5002b980 comp rx=0 tx=0).stop 2023-10-16T11:24:54.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.671+0000 7f4a5b62d640 1 -- 172.21.15.84:0/848120998 shutdown_connections 2023-10-16T11:24:54.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.671+0000 7f4a5b62d640 1 --2- 172.21.15.84:0/848120998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a5406faf0 0x7f4a5415f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.671+0000 7f4a5b62d640 1 --2- 172.21.15.84:0/848120998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a5415af80 0x7f4a540cb1c0 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.671+0000 7f4a5b62d640 1 -- 172.21.15.84:0/848120998 >> 172.21.15.84:0/848120998 conn(0x7f4a540d0ab0 msgr2=0x7f4a540d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:54.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.671+0000 7f4a5b62d640 1 -- 172.21.15.84:0/848120998 shutdown_connections 2023-10-16T11:24:54.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.671+0000 7f4a5b62d640 1 -- 172.21.15.84:0/848120998 wait complete. 2023-10-16T11:24:54.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.673+0000 7f4a5b62d640 1 Processor -- start 2023-10-16T11:24:54.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.673+0000 7f4a5b62d640 1 -- start start 2023-10-16T11:24:54.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.674+0000 7f4a5b62d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a54065ab0 0x7f4a540ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:54.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.674+0000 7f4a5b62d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a5406faf0 0x7f4a5406c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:54.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.674+0000 7f4a5b62d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a5415af80 0x7f4a540ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:54.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.674+0000 7f4a5b62d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a540ac890 con 0x7f4a54065ab0 2023-10-16T11:24:54.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.674+0000 7f4a5b62d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5406c920 con 0x7f4a5415af80 2023-10-16T11:24:54.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.674+0000 7f4a5b62d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5406cc00 con 0x7f4a5406faf0 2023-10-16T11:24:54.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.674+0000 7f4a593c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a54065ab0 0x7f4a540ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:54.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.675+0000 7f4a593c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a54065ab0 0x7f4a540ad2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51488/0 (socket says 172.21.15.84:51488) 2023-10-16T11:24:54.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.675+0000 7f4a593c9640 1 -- 172.21.15.84:0/806785171 learned_addr learned my addr 172.21.15.84:0/806785171 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:54.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.675+0000 7f4a58bc8640 1 --2- 172.21.15.84:0/806785171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a5406faf0 0x7f4a5406c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:54.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.675+0000 7f4a59bca640 1 --2- 172.21.15.84:0/806785171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a5415af80 0x7f4a540ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:54.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.675+0000 7f4a593c9640 1 -- 172.21.15.84:0/806785171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a5406faf0 msgr2=0x7f4a5406c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:54.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.675+0000 7f4a593c9640 1 --2- 172.21.15.84:0/806785171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a5406faf0 0x7f4a5406c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.675+0000 7f4a593c9640 1 -- 172.21.15.84:0/806785171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a5415af80 msgr2=0x7f4a540ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:54.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.675+0000 7f4a593c9640 1 --2- 172.21.15.84:0/806785171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a5415af80 0x7f4a540ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.675+0000 7f4a593c9640 1 -- 172.21.15.84:0/806785171 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a500094d0 con 0x7f4a54065ab0 2023-10-16T11:24:54.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.675+0000 7f4a593c9640 1 --2- 172.21.15.84:0/806785171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a54065ab0 0x7f4a540ad2e0 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f4a50000c00 tx=0x7f4a50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:54.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.676+0000 7f4a49ffb640 1 -- 172.21.15.84:0/806785171 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a50047070 con 0x7f4a54065ab0 2023-10-16T11:24:54.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.676+0000 7f4a49ffb640 1 -- 172.21.15.84:0/806785171 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a500307d0 con 0x7f4a54065ab0 2023-10-16T11:24:54.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.676+0000 7f4a49ffb640 1 -- 172.21.15.84:0/806785171 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a50043400 con 0x7f4a54065ab0 2023-10-16T11:24:54.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.676+0000 7f4a5b62d640 1 -- 172.21.15.84:0/806785171 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a540acae0 con 0x7f4a54065ab0 2023-10-16T11:24:54.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.676+0000 7f4a5b62d640 1 -- 172.21.15.84:0/806785171 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a540ad820 con 0x7f4a54065ab0 2023-10-16T11:24:54.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.677+0000 7f4a5b62d640 1 -- 172.21.15.84:0/806785171 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a5404e160 con 0x7f4a54065ab0 2023-10-16T11:24:54.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.679+0000 7f4a49ffb640 1 -- 172.21.15.84:0/806785171 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 133) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f4a5004c410 con 0x7f4a54065ab0 2023-10-16T11:24:54.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.680+0000 7f4a49ffb640 1 --2- 172.21.15.84:0/806785171 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4a3806afe0 0x7f4a3806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:54.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.680+0000 7f4a49ffb640 1 -- 172.21.15.84:0/806785171 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f4a5003e070 con 0x7f4a54065ab0 2023-10-16T11:24:54.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.680+0000 7f4a58bc8640 1 -- 172.21.15.84:0/806785171 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4a3806afe0 msgr2=0x7f4a3806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1425409017 2023-10-16T11:24:54.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.680+0000 7f4a58bc8640 1 --2- 172.21.15.84:0/806785171 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4a3806afe0 0x7f4a3806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:24:54.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.685+0000 7f4a49ffb640 1 -- 172.21.15.84:0/806785171 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4a50076f60 con 0x7f4a54065ab0 2023-10-16T11:24:54.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.874+0000 7f4a5b62d640 1 -- 172.21.15.84:0/806785171 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f4a5415e890 con 0x7f4a54065ab0 2023-10-16T11:24:54.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.879+0000 7f4a49ffb640 1 -- 172.21.15.84:0/806785171 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v78)=0 v78) v1 ==== 133+0+0 (secure 0 0 0) 0x7f4a5007a580 con 0x7f4a54065ab0 2023-10-16T11:24:54.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.881+0000 7f4a58bc8640 1 -- 172.21.15.84:0/806785171 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4a3806afe0 msgr2=0x7f4a3806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1425409017 2023-10-16T11:24:54.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.881+0000 7f4a58bc8640 1 --2- 172.21.15.84:0/806785171 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4a3806afe0 0x7f4a3806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:24:54.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.882+0000 7f4a5b62d640 1 -- 172.21.15.84:0/806785171 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4a3806afe0 msgr2=0x7f4a3806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:24:54.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.882+0000 7f4a5b62d640 1 --2- 172.21.15.84:0/806785171 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4a3806afe0 0x7f4a3806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.882+0000 7f4a5b62d640 1 -- 172.21.15.84:0/806785171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a54065ab0 msgr2=0x7f4a540ad2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:54.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.882+0000 7f4a5b62d640 1 --2- 172.21.15.84:0/806785171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a54065ab0 0x7f4a540ad2e0 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f4a50000c00 tx=0x7f4a50004730 comp rx=0 tx=0).stop 2023-10-16T11:24:54.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.882+0000 7f4a5b62d640 1 -- 172.21.15.84:0/806785171 shutdown_connections 2023-10-16T11:24:54.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.882+0000 7f4a5b62d640 1 --2- 172.21.15.84:0/806785171 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7f4a3806afe0 0x7f4a3806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.882+0000 7f4a5b62d640 1 --2- 172.21.15.84:0/806785171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a5415af80 0x7f4a540ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.882+0000 7f4a5b62d640 1 --2- 172.21.15.84:0/806785171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a5406faf0 0x7f4a5406c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.882+0000 7f4a5b62d640 1 --2- 172.21.15.84:0/806785171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a54065ab0 0x7f4a540ad2e0 unknown :-1 s=CLOSED pgs=2717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:54.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.882+0000 7f4a5b62d640 1 -- 172.21.15.84:0/806785171 >> 172.21.15.84:0/806785171 conn(0x7f4a540d0ab0 msgr2=0x7f4a54158d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:54.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.882+0000 7f4a5b62d640 1 -- 172.21.15.84:0/806785171 shutdown_connections 2023-10-16T11:24:54.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:54.882+0000 7f4a5b62d640 1 -- 172.21.15.84:0/806785171 wait complete. 2023-10-16T11:24:54.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:24:54.903 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p mandatory_fs_argument_test_dir) 2023-10-16T11:24:54.951 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-16T11:24:55.033 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:55.032+0000 7ff51a445040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:24:55.085 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:55.084+0000 7ff51a445040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:24:55.087 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:55.085+0000 7f8f0fa56040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:24:55.135 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:55.134+0000 7ff51a445040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:24:55.142 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:55.140+0000 7f8f0fa56040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:24:55.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.152+0000 7ff69cbf5640 1 Processor -- start 2023-10-16T11:24:55.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.153+0000 7ff69cbf5640 1 -- start start 2023-10-16T11:24:55.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.154+0000 7ff69cbf5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff6980b0360 0x7ff6980b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:55.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.154+0000 7ff69cbf5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff6980ca510 0x7ff6980ca920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:55.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.154+0000 7ff69cbf5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff69815b310 con 0x7ff69815ac10 2023-10-16T11:24:55.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.154+0000 7ff69cbf5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff69815f800 con 0x7ff6980b0360 2023-10-16T11:24:55.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.154+0000 7ff69cbf5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6980b0d30 con 0x7ff6980ca510 2023-10-16T11:24:55.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.154+0000 7ff68ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff6980b0360 0x7ff6980b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:55.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.154+0000 7ff68ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff6980b0360 0x7ff6980b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51492/0 (socket says 172.21.15.84:51492) 2023-10-16T11:24:55.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.154+0000 7ff68ffff640 1 -- 172.21.15.84:0/3845941080 learned_addr learned my addr 172.21.15.84:0/3845941080 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:55.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.154+0000 7ff68ffff640 1 -- 172.21.15.84:0/3845941080 >> v2:172.21.15.157:3300/0 conn(0x7ff6980ca510 msgr2=0x7ff6980ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:55.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.154+0000 7ff68ffff640 1 --2- 172.21.15.84:0/3845941080 >> v2:172.21.15.157:3300/0 conn(0x7ff6980ca510 0x7ff6980ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:55.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.154+0000 7ff68ffff640 1 -- 172.21.15.84:0/3845941080 >> v1:172.21.15.157:6789/0 conn(0x7ff69815ac10 legacy=0x7ff69806c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:55.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.155+0000 7ff68ffff640 1 -- 172.21.15.84:0/3845941080 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69806f640 con 0x7ff6980b0360 2023-10-16T11:24:55.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.156+0000 7ff68ffff640 1 --2- 172.21.15.84:0/3845941080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6980b0360 0x7ff6980b07f0 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7ff69806d730 tx=0x7ff68002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951c957e3ac55690 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:55.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.156+0000 7ff695d9b640 1 -- 172.21.15.84:0/3845941080 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6800317e0 con 0x7ff6980b0360 2023-10-16T11:24:55.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.156+0000 7ff695d9b640 1 -- 172.21.15.84:0/3845941080 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff680031940 con 0x7ff6980b0360 2023-10-16T11:24:55.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.156+0000 7ff695d9b640 1 -- 172.21.15.84:0/3845941080 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680031cd0 con 0x7ff6980b0360 2023-10-16T11:24:55.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.156+0000 7ff69cbf5640 1 -- 172.21.15.84:0/3845941080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6980b0360 msgr2=0x7ff6980b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:55.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.156+0000 7ff69cbf5640 1 --2- 172.21.15.84:0/3845941080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6980b0360 0x7ff6980b07f0 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7ff69806d730 tx=0x7ff68002b880 comp rx=0 tx=0).stop 2023-10-16T11:24:55.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.157+0000 7ff69cbf5640 1 -- 172.21.15.84:0/3845941080 shutdown_connections 2023-10-16T11:24:55.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.157+0000 7ff69cbf5640 1 --2- 172.21.15.84:0/3845941080 >> v2:172.21.15.157:3300/0 conn(0x7ff6980ca510 0x7ff6980ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:55.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.157+0000 7ff69cbf5640 1 --2- 172.21.15.84:0/3845941080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6980b0360 0x7ff6980b07f0 unknown :-1 s=CLOSED pgs=2718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:55.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.157+0000 7ff69cbf5640 1 -- 172.21.15.84:0/3845941080 >> 172.21.15.84:0/3845941080 conn(0x7ff6980cb360 msgr2=0x7ff6980cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:55.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.157+0000 7ff69cbf5640 1 -- 172.21.15.84:0/3845941080 shutdown_connections 2023-10-16T11:24:55.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.157+0000 7ff69cbf5640 1 -- 172.21.15.84:0/3845941080 wait complete. 2023-10-16T11:24:55.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.158+0000 7ff69cbf5640 1 Processor -- start 2023-10-16T11:24:55.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.159+0000 7ff69cbf5640 1 -- start start 2023-10-16T11:24:55.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.159+0000 7ff69cbf5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6980b0360 0x7ff6981579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:55.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.160+0000 7ff69cbf5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6980ca510 0x7ff6980c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:55.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.160+0000 7ff69cbf5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff69815ac10 0x7ff6980af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:55.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.160+0000 7ff69cbf5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6980b1e10 con 0x7ff6980ca510 2023-10-16T11:24:55.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.160+0000 7ff69cbf5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6980b1500 con 0x7ff6980b0360 2023-10-16T11:24:55.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.160+0000 7ff69cbf5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff698157f00 con 0x7ff69815ac10 2023-10-16T11:24:55.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.160+0000 7ff68ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6980ca510 0x7ff6980c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:55.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.160+0000 7ff68ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6980ca510 0x7ff6980c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51496/0 (socket says 172.21.15.84:51496) 2023-10-16T11:24:55.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.160+0000 7ff68ffff640 1 -- 172.21.15.84:0/1828950584 learned_addr learned my addr 172.21.15.84:0/1828950584 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:55.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.160+0000 7ff696d9d640 1 --2- 172.21.15.84:0/1828950584 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff69815ac10 0x7ff6980af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:55.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.160+0000 7ff69659c640 1 --2- 172.21.15.84:0/1828950584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6980b0360 0x7ff6981579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:55.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.160+0000 7ff68ffff640 1 -- 172.21.15.84:0/1828950584 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff69815ac10 msgr2=0x7ff6980af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:55.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.160+0000 7ff68ffff640 1 --2- 172.21.15.84:0/1828950584 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff69815ac10 0x7ff6980af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:55.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.160+0000 7ff68ffff640 1 -- 172.21.15.84:0/1828950584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6980b0360 msgr2=0x7ff6981579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:55.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.160+0000 7ff68ffff640 1 --2- 172.21.15.84:0/1828950584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6980b0360 0x7ff6981579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:55.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.160+0000 7ff68ffff640 1 -- 172.21.15.84:0/1828950584 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6800094d0 con 0x7ff6980ca510 2023-10-16T11:24:55.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.161+0000 7ff68ffff640 1 --2- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6980ca510 0x7ff6980c76e0 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7ff680000c00 tx=0x7ff6800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:55.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.161+0000 7ff68effd640 1 -- 172.21.15.84:0/1828950584 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680032940 con 0x7ff6980ca510 2023-10-16T11:24:55.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.161+0000 7ff68effd640 1 -- 172.21.15.84:0/1828950584 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff680032aa0 con 0x7ff6980ca510 2023-10-16T11:24:55.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.161+0000 7ff68effd640 1 -- 172.21.15.84:0/1828950584 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680032e50 con 0x7ff6980ca510 2023-10-16T11:24:55.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.161+0000 7ff69cbf5640 1 -- 172.21.15.84:0/1828950584 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6980c8050 con 0x7ff6980ca510 2023-10-16T11:24:55.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.161+0000 7ff69cbf5640 1 -- 172.21.15.84:0/1828950584 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6980b1a30 con 0x7ff6980ca510 2023-10-16T11:24:55.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.162+0000 7ff69cbf5640 1 -- 172.21.15.84:0/1828950584 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6980b1f50 con 0x7ff6980ca510 2023-10-16T11:24:55.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.164+0000 7ff68effd640 1 -- 172.21.15.84:0/1828950584 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 133) v1 ==== 89348+0+0 (secure 0 0 0) 0x7ff6800453f0 con 0x7ff6980ca510 2023-10-16T11:24:55.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.165+0000 7ff68effd640 1 --2- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff67806afe0 0x7ff67806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:55.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.165+0000 7ff68effd640 1 -- 172.21.15.84:0/1828950584 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6601+0+0 (secure 0 0 0) 0x7ff6800abe90 con 0x7ff6980ca510 2023-10-16T11:24:55.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.165+0000 7ff69659c640 1 -- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff67806afe0 msgr2=0x7ff67806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1425409017 2023-10-16T11:24:55.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.165+0000 7ff69659c640 1 --2- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff67806afe0 0x7ff67806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:24:55.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.170+0000 7ff68effd640 1 -- 172.21.15.84:0/1828950584 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff6980b1f50 con 0x7ff6980ca510 2023-10-16T11:24:55.197 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:55.195+0000 7f8f0fa56040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:24:55.197 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:55.196+0000 7ff51a445040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:24:55.247 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:55.246+0000 7ff51a445040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:24:55.260 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:55.259+0000 7f8f0fa56040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:24:55.312 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:55.310+0000 7f8f0fa56040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:24:55.359 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:55.358+0000 7ff51a445040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:24:55.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.366+0000 7ff69659c640 1 -- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff67806afe0 msgr2=0x7ff67806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1425409017 2023-10-16T11:24:55.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.366+0000 7ff69659c640 1 --2- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff67806afe0 0x7ff67806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:24:55.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.377+0000 7ff69cbf5640 1 -- 172.21.15.84:0/1828950584 --> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff69815b890 con 0x7ff67806afe0 2023-10-16T11:24:55.422 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:55.421+0000 7ff51a445040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:24:55.426 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:55.425+0000 7f8f0fa56040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:24:55.470 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:55.470+0000 7ff51a445040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:24:55.488 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:55.486+0000 7f8f0fa56040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:24:55.528 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:55.528+0000 7ff51a445040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:24:55.538 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:55.536+0000 7f8f0fa56040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:24:55.579 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:55.579+0000 7ff51a445040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:24:55.594 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:55.593+0000 7f8f0fa56040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:24:55.644 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:55.642+0000 7f8f0fa56040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:24:55.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.767+0000 7ff69659c640 1 -- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff67806afe0 msgr2=0x7ff67806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1425409017 2023-10-16T11:24:55.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:55.767+0000 7ff69659c640 1 --2- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff67806afe0 0x7ff67806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T11:24:55.967 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:55.966+0000 7ff51a445040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:24:56.028 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:56.026+0000 7f8f0fa56040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:24:56.124 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:56.124+0000 7ff51a445040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:24:56.186 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:56.184+0000 7f8f0fa56040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:24:56.234 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:56.233+0000 7ff51a445040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:24:56.284 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:56.283+0000 7ff51a445040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:24:56.294 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:56.292+0000 7f8f0fa56040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:24:56.344 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:56.342+0000 7f8f0fa56040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:24:56.391 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:56.390+0000 7ff51a445040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:24:56.451 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:56.449+0000 7f8f0fa56040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:24:56.499 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:56.498+0000 7ff51a445040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:24:56.557 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:56.555+0000 7f8f0fa56040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:24:56.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:56.568+0000 7ff69659c640 1 -- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff67806afe0 msgr2=0x7ff67806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1425409017 2023-10-16T11:24:56.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:56.568+0000 7ff69659c640 1 --2- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff67806afe0 0x7ff67806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T11:24:56.657 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:56.656+0000 7ff51a445040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:24:56.708 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:56.707+0000 7ff51a445040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:24:56.714 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:56.711+0000 7f8f0fa56040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:24:56.763 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:56.761+0000 7f8f0fa56040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:24:56.827 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:56.826+0000 7ff51a445040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:24:56.881 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:56.879+0000 7f8f0fa56040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:24:57.656 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:57.655+0000 7ff51a445040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:24:57.717 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:57.715+0000 7f8f0fa56040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:24:57.951 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:57.950+0000 7ff51a445040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:24:58.008 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:58.007+0000 7ff51a445040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:24:58.011 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.009+0000 7f8f0fa56040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:24:58.068 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.066+0000 7f8f0fa56040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:24:58.126 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:24:58.125+0000 7ff51a445040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:24:58.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:58.170+0000 7ff69659c640 1 -- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff67806afe0 msgr2=0x7ff67806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/1425409017 2023-10-16T11:24:58.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:58.170+0000 7ff69659c640 1 --2- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff67806afe0 0x7ff67806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T11:24:58.190 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.188+0000 7f8f0fa56040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:24:58.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:58.193+0000 7ff68effd640 1 -- 172.21.15.84:0/1828950584 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 134) v1 ==== 89348+0+0 (secure 0 0 0) 0x7ff680076e10 con 0x7ff6980ca510 2023-10-16T11:24:58.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:58.219+0000 7ff68effd640 1 -- 172.21.15.84:0/1828950584 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 135) v1 ==== 88957+0+0 (secure 0 0 0) 0x7ff6800a98a0 con 0x7ff6980ca510 2023-10-16T11:24:58.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:58.220+0000 7ff68effd640 1 -- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff67806afe0 msgr2=0x7ff67806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:24:58.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:58.220+0000 7ff68effd640 1 --2- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:6832/1425409017,v1:172.21.15.84:6833/1425409017] conn(0x7ff67806afe0 0x7ff67806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:58.820 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.817+0000 7f8e8f4a0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.820 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.817+0000 7f8e8f4a0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.820 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.817+0000 7f8e8f4a0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.820 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.817+0000 7f8e8f4a0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.821 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.817+0000 7f8e8f4a0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.821 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.818+0000 7f8e8c49a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.821 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.818+0000 7f8e8c49a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.821 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.818+0000 7f8e8c49a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.821 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.818+0000 7f8e8c49a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.822 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.818+0000 7f8e8c49a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.838 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.836+0000 7f8e8c49a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.838 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.836+0000 7f8e8c49a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.838 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.836+0000 7f8e8c49a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.838 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.836+0000 7f8e8c49a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.839 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.836+0000 7f8e8c49a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.839 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.836+0000 7f8e8f4a0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.839 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.836+0000 7f8e8f4a0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.839 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.836+0000 7f8e8f4a0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.839 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.836+0000 7f8e8f4a0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:58.839 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:24:58.836+0000 7f8e8f4a0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:24:59.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.227+0000 7ff68effd640 1 -- 172.21.15.84:0/1828950584 <== mon.0 v2:172.21.15.84:3300/0 9 ==== mgrmap(e 136) v1 ==== 89084+0+0 (secure 0 0 0) 0x7ff6800776e0 con 0x7ff6980ca510 2023-10-16T11:24:59.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.228+0000 7ff68effd640 1 --2- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7ff67801cd00 0x7ff67801d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:59.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.228+0000 7ff68effd640 1 -- 172.21.15.84:0/1828950584 --> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff678077530 con 0x7ff67801cd00 2023-10-16T11:24:59.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.230+0000 7ff69659c640 1 --2- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7ff67801cd00 0x7ff67801d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:59.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.234+0000 7ff69659c640 1 --2- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7ff67801cd00 0x7ff67801d0e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff6980acda0 tx=0x7ff688009060 comp rx=0 tx=0).ready entity=mgr.11682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:59.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.427+0000 7ff68effd640 1 -- 172.21.15.84:0/1828950584 <== mgr.11682 v2:172.21.15.84:6832/2880641466 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+53 (secure 0 0 0) 0x7ff678077530 con 0x7ff67801cd00 2023-10-16T11:24:59.429 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path mandatory_fs_argument_test_dir 2023-10-16T11:24:59.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.430+0000 7ff69cbf5640 1 -- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7ff67801cd00 msgr2=0x7ff67801d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:59.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.430+0000 7ff69cbf5640 1 --2- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7ff67801cd00 0x7ff67801d0e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff6980acda0 tx=0x7ff688009060 comp rx=0 tx=0).stop 2023-10-16T11:24:59.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.431+0000 7ff69cbf5640 1 -- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6980ca510 msgr2=0x7ff6980c76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:59.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.431+0000 7ff69cbf5640 1 --2- 172.21.15.84:0/1828950584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6980ca510 0x7ff6980c76e0 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7ff680000c00 tx=0x7ff6800046c0 comp rx=0 tx=0).stop 2023-10-16T11:24:59.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.431+0000 7ff696d9d640 1 -- 172.21.15.84:0/1828950584 reap_dead start 2023-10-16T11:24:59.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.431+0000 7ff69cbf5640 1 -- 172.21.15.84:0/1828950584 shutdown_connections 2023-10-16T11:24:59.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.431+0000 7ff69cbf5640 1 -- 172.21.15.84:0/1828950584 >> 172.21.15.84:0/1828950584 conn(0x7ff6980cb360 msgr2=0x7ff69806c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:59.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.431+0000 7ff69cbf5640 1 -- 172.21.15.84:0/1828950584 shutdown_connections 2023-10-16T11:24:59.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.431+0000 7ff69cbf5640 1 -- 172.21.15.84:0/1828950584 wait complete. 2023-10-16T11:24:59.453 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path mandatory_fs_argument_test_dir 2023-10-16T11:24:59.453 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-16T11:24:59.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.671+0000 7f2a43434640 1 Processor -- start 2023-10-16T11:24:59.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.672+0000 7f2a43434640 1 -- start start 2023-10-16T11:24:59.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.673+0000 7f2a43434640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2a3c1599c0 0x7f2a3c0c9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:59.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.673+0000 7f2a43434640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a3c159da0 0x7f2a3c15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:59.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.673+0000 7f2a43434640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a3c15dd10 con 0x7f2a3c0cce20 2023-10-16T11:24:59.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.673+0000 7f2a43434640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c15dfb0 con 0x7f2a3c1599c0 2023-10-16T11:24:59.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.673+0000 7f2a43434640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c15e290 con 0x7f2a3c159da0 2023-10-16T11:24:59.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.673+0000 7f2a419d1640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2a3c0cce20 0x7f2a3c06c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53852/0 (socket says 172.21.15.84:53852) 2023-10-16T11:24:59.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.673+0000 7f2a419d1640 1 -- 172.21.15.84:0/2662606319 learned_addr learned my addr 172.21.15.84:0/2662606319 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:24:59.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.674+0000 7f2a409cf640 1 --2- 172.21.15.84:0/2662606319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a3c159da0 0x7f2a3c15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:59.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.674+0000 7f2a411d0640 1 --2- 172.21.15.84:0/2662606319 >> v2:172.21.15.157:3300/0 conn(0x7f2a3c1599c0 0x7f2a3c0c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:59.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.674+0000 7f2a33fff640 1 -- 172.21.15.84:0/2662606319 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2492064904 0 0) 0x7f2a3c15dd10 con 0x7f2a3c0cce20 2023-10-16T11:24:59.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.674+0000 7f2a33fff640 1 -- 172.21.15.84:0/2662606319 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a28005730 con 0x7f2a3c0cce20 2023-10-16T11:24:59.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.674+0000 7f2a33fff640 1 -- 172.21.15.84:0/2662606319 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3129987279 0 0) 0x7f2a28005730 con 0x7f2a3c0cce20 2023-10-16T11:24:59.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.674+0000 7f2a33fff640 1 -- 172.21.15.84:0/2662606319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a3c159da0 msgr2=0x7f2a3c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:59.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.674+0000 7f2a33fff640 1 --2- 172.21.15.84:0/2662606319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a3c159da0 0x7f2a3c15d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:59.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.674+0000 7f2a33fff640 1 -- 172.21.15.84:0/2662606319 >> v2:172.21.15.157:3300/0 conn(0x7f2a3c1599c0 msgr2=0x7f2a3c0c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:59.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.674+0000 7f2a33fff640 1 --2- 172.21.15.84:0/2662606319 >> v2:172.21.15.157:3300/0 conn(0x7f2a3c1599c0 0x7f2a3c0c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:59.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.674+0000 7f2a33fff640 1 -- 172.21.15.84:0/2662606319 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c15e570 con 0x7f2a3c0cce20 2023-10-16T11:24:59.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.675+0000 7f2a33fff640 1 -- 172.21.15.84:0/2662606319 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f2a340019f0 con 0x7f2a3c0cce20 2023-10-16T11:24:59.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.675+0000 7f2a33fff640 1 -- 172.21.15.84:0/2662606319 >> v1:172.21.15.84:6789/0 conn(0x7f2a3c0cce20 legacy=0x7f2a3c06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:59.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.675+0000 7f2a33fff640 1 --2- 172.21.15.84:0/2662606319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a28005bf0 0x7f2a28006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:59.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.675+0000 7f2a33fff640 1 --2- 172.21.15.84:0/2662606319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a280069d0 0x7f2a28006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:59.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.675+0000 7f2a33fff640 1 --2- 172.21.15.84:0/2662606319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a3c159da0 0x7f2a28007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:59.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.675+0000 7f2a33fff640 1 -- 172.21.15.84:0/2662606319 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c15dfb0 con 0x7f2a28005bf0 2023-10-16T11:24:59.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.675+0000 7f2a33fff640 1 -- 172.21.15.84:0/2662606319 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c15e290 con 0x7f2a280069d0 2023-10-16T11:24:59.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.675+0000 7f2a33fff640 1 -- 172.21.15.84:0/2662606319 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2800db90 con 0x7f2a3c159da0 2023-10-16T11:24:59.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.675+0000 7f2a411d0640 1 --2- 172.21.15.84:0/2662606319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a28005bf0 0x7f2a28006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:59.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.675+0000 7f2a411d0640 1 -- 172.21.15.84:0/2662606319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a3c159da0 msgr2=0x7f2a28007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:59.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.675+0000 7f2a411d0640 1 --2- 172.21.15.84:0/2662606319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a3c159da0 0x7f2a28007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:59.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.675+0000 7f2a411d0640 1 -- 172.21.15.84:0/2662606319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a280069d0 msgr2=0x7f2a28006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:24:59.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.675+0000 7f2a411d0640 1 --2- 172.21.15.84:0/2662606319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a280069d0 0x7f2a28006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:59.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.675+0000 7f2a411d0640 1 -- 172.21.15.84:0/2662606319 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a28005730 con 0x7f2a28005bf0 2023-10-16T11:24:59.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.676+0000 7f2a411d0640 1 --2- 172.21.15.84:0/2662606319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a28005bf0 0x7f2a28006010 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009af0 tx=0x7f2a28007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780f273b75768d18 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:59.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.677+0000 7f2a33fff640 1 -- 172.21.15.84:0/2662606319 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3802c710 con 0x7f2a28005bf0 2023-10-16T11:24:59.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.677+0000 7f2a33fff640 1 -- 172.21.15.84:0/2662606319 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3802c870 con 0x7f2a28005bf0 2023-10-16T11:24:59.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.677+0000 7f2a33fff640 1 -- 172.21.15.84:0/2662606319 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3802cc00 con 0x7f2a28005bf0 2023-10-16T11:24:59.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.677+0000 7f2a43434640 1 -- 172.21.15.84:0/2662606319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a28005bf0 msgr2=0x7f2a28006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:59.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.677+0000 7f2a43434640 1 --2- 172.21.15.84:0/2662606319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a28005bf0 0x7f2a28006010 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009af0 tx=0x7f2a28007a00 comp rx=0 tx=0).stop 2023-10-16T11:24:59.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.677+0000 7f2a419d1640 1 -- 172.21.15.84:0/2662606319 reap_dead start 2023-10-16T11:24:59.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.677+0000 7f2a43434640 1 -- 172.21.15.84:0/2662606319 shutdown_connections 2023-10-16T11:24:59.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.677+0000 7f2a43434640 1 -- 172.21.15.84:0/2662606319 >> 172.21.15.84:0/2662606319 conn(0x7f2a3c0cb360 msgr2=0x7f2a3c0cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:24:59.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.677+0000 7f2a43434640 1 -- 172.21.15.84:0/2662606319 shutdown_connections 2023-10-16T11:24:59.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.677+0000 7f2a43434640 1 -- 172.21.15.84:0/2662606319 wait complete. 2023-10-16T11:24:59.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.679+0000 7f2a43434640 1 Processor -- start 2023-10-16T11:24:59.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.679+0000 7f2a43434640 1 -- start start 2023-10-16T11:24:59.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.680+0000 7f2a43434640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a28005bf0 0x7f2a3c153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:59.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.680+0000 7f2a43434640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a3c0c3a20 0x7f2a3c0c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:59.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.680+0000 7f2a43434640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a3c0b08b0 0x7f2a3c0b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:59.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.680+0000 7f2a43434640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c0c43e0 con 0x7f2a3c0c3a20 2023-10-16T11:24:59.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.680+0000 7f2a43434640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c154380 con 0x7f2a28005bf0 2023-10-16T11:24:59.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.680+0000 7f2a43434640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c0b1270 con 0x7f2a3c0b08b0 2023-10-16T11:24:59.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.680+0000 7f2a409cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a3c0c3a20 0x7f2a3c0c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:59.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.681+0000 7f2a409cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a3c0c3a20 0x7f2a3c0c3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56830/0 (socket says 172.21.15.84:56830) 2023-10-16T11:24:59.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.681+0000 7f2a409cf640 1 -- 172.21.15.84:0/504374927 learned_addr learned my addr 172.21.15.84:0/504374927 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:24:59.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.681+0000 7f2a419d1640 1 --2- 172.21.15.84:0/504374927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a3c0b08b0 0x7f2a3c0b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:59.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.681+0000 7f2a411d0640 1 --2- 172.21.15.84:0/504374927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a28005bf0 0x7f2a3c153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:59.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.681+0000 7f2a409cf640 1 -- 172.21.15.84:0/504374927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a3c0b08b0 msgr2=0x7f2a3c0b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:59.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.681+0000 7f2a409cf640 1 --2- 172.21.15.84:0/504374927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a3c0b08b0 0x7f2a3c0b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:59.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.681+0000 7f2a409cf640 1 -- 172.21.15.84:0/504374927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a28005bf0 msgr2=0x7f2a3c153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:24:59.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.681+0000 7f2a409cf640 1 --2- 172.21.15.84:0/504374927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a28005bf0 0x7f2a3c153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:24:59.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.681+0000 7f2a409cf640 1 -- 172.21.15.84:0/504374927 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c0c59b0 con 0x7f2a3c0c3a20 2023-10-16T11:24:59.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.681+0000 7f2a419d1640 1 --2- 172.21.15.84:0/504374927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a3c0b08b0 0x7f2a3c0b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:24:59.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.681+0000 7f2a409cf640 1 --2- 172.21.15.84:0/504374927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a3c0c3a20 0x7f2a3c0c3ea0 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00e7d0 tx=0x7f2a2c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:59.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.682+0000 7f2a31ffb640 1 -- 172.21.15.84:0/504374927 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c00eee0 con 0x7f2a3c0c3a20 2023-10-16T11:24:59.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.682+0000 7f2a31ffb640 1 -- 172.21.15.84:0/504374927 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2c00c940 con 0x7f2a3c0c3a20 2023-10-16T11:24:59.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.682+0000 7f2a43434640 1 -- 172.21.15.84:0/504374927 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3c0c6ca0 con 0x7f2a3c0c3a20 2023-10-16T11:24:59.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.682+0000 7f2a31ffb640 1 -- 172.21.15.84:0/504374927 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c00ccd0 con 0x7f2a3c0c3a20 2023-10-16T11:24:59.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.682+0000 7f2a43434640 1 -- 172.21.15.84:0/504374927 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c0c5ee0 con 0x7f2a3c0c3a20 2023-10-16T11:24:59.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.683+0000 7f2a43434640 1 -- 172.21.15.84:0/504374927 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3c04e160 con 0x7f2a3c0c3a20 2023-10-16T11:24:59.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.684+0000 7f2a31ffb640 1 -- 172.21.15.84:0/504374927 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 136) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f2a2c013470 con 0x7f2a3c0c3a20 2023-10-16T11:24:59.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.685+0000 7f2a31ffb640 1 --2- 172.21.15.84:0/504374927 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f2a2006ad10 0x7f2a2006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:24:59.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.686+0000 7f2a31ffb640 1 -- 172.21.15.84:0/504374927 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6945+0+0 (secure 0 0 0) 0x7f2a2c089890 con 0x7f2a3c0c3a20 2023-10-16T11:24:59.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.686+0000 7f2a411d0640 1 --2- 172.21.15.84:0/504374927 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f2a2006ad10 0x7f2a2006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:24:59.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.686+0000 7f2a411d0640 1 --2- 172.21.15.84:0/504374927 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f2a2006ad10 0x7f2a2006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2a38036000 tx=0x7f2a38030000 comp rx=0 tx=0).ready entity=mgr.11682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:24:59.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.690+0000 7f2a31ffb640 1 -- 172.21.15.84:0/504374927 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2a2c054760 con 0x7f2a3c0c3a20 2023-10-16T11:24:59.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:24:59.927+0000 7f2a43434640 1 -- 172.21.15.84:0/504374927 --> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a3c06fda0 con 0x7f2a2006ad10 2023-10-16T11:25:00.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.075+0000 7f2a31ffb640 1 -- 172.21.15.84:0/504374927 <== mgr.11682 v2:172.21.15.84:6832/2880641466 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f2a3802ca60 con 0x7f2a2006ad10 2023-10-16T11:25:00.078 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule activated for path mandatory_fs_argument_test_dir 2023-10-16T11:25:00.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.078+0000 7f2a43434640 1 -- 172.21.15.84:0/504374927 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f2a2006ad10 msgr2=0x7f2a2006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:00.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.078+0000 7f2a43434640 1 --2- 172.21.15.84:0/504374927 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f2a2006ad10 0x7f2a2006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2a38036000 tx=0x7f2a38030000 comp rx=0 tx=0).stop 2023-10-16T11:25:00.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.078+0000 7f2a43434640 1 -- 172.21.15.84:0/504374927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a3c0c3a20 msgr2=0x7f2a3c0c3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:00.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.078+0000 7f2a43434640 1 --2- 172.21.15.84:0/504374927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a3c0c3a20 0x7f2a3c0c3ea0 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00e7d0 tx=0x7f2a2c00ec90 comp rx=0 tx=0).stop 2023-10-16T11:25:00.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.079+0000 7f2a43434640 1 -- 172.21.15.84:0/504374927 shutdown_connections 2023-10-16T11:25:00.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.079+0000 7f2a43434640 1 --2- 172.21.15.84:0/504374927 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f2a2006ad10 0x7f2a2006d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:00.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.079+0000 7f2a43434640 1 --2- 172.21.15.84:0/504374927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a3c0b08b0 0x7f2a3c0b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:00.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.079+0000 7f2a43434640 1 --2- 172.21.15.84:0/504374927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a3c0c3a20 0x7f2a3c0c3ea0 unknown :-1 s=CLOSED pgs=2738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:00.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.079+0000 7f2a43434640 1 --2- 172.21.15.84:0/504374927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a28005bf0 0x7f2a3c153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:00.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.079+0000 7f2a43434640 1 -- 172.21.15.84:0/504374927 >> 172.21.15.84:0/504374927 conn(0x7f2a3c0cb360 msgr2=0x7f2a3c0cc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:00.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.079+0000 7f2a43434640 1 -- 172.21.15.84:0/504374927 shutdown_connections 2023-10-16T11:25:00.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.079+0000 7f2a43434640 1 -- 172.21.15.84:0/504374927 wait complete. 2023-10-16T11:25:00.101 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path mandatory_fs_argument_test_dir 2023-10-16T11:25:00.102 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-16T11:25:00.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.322+0000 7f184a901640 1 Processor -- start 2023-10-16T11:25:00.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.322+0000 7f184a901640 1 -- start start 2023-10-16T11:25:00.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.324+0000 7f184a901640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f184406a4b0 0x7f18440ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:00.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.324+0000 7f184a901640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18440c9ca0 0x7f18440ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:00.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.324+0000 7f184a901640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18440cb030 con 0x7f184415ad70 2023-10-16T11:25:00.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.324+0000 7f184a901640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18440cae30 con 0x7f18440c9ca0 2023-10-16T11:25:00.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.324+0000 7f184a901640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18440c91c0 con 0x7f184406a4b0 2023-10-16T11:25:00.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.324+0000 7f1848e9e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18440c9ca0 0x7f18440ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:00.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.324+0000 7f1848e9e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18440c9ca0 0x7f18440ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56838/0 (socket says 172.21.15.84:56838) 2023-10-16T11:25:00.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.324+0000 7f1848e9e640 1 -- 172.21.15.84:0/2848586310 learned_addr learned my addr 172.21.15.84:0/2848586310 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:00.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.325+0000 7f18437fe640 1 --2- 172.21.15.84:0/2848586310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f184406a4b0 0x7f18440ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:00.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.325+0000 7f1848e9e640 1 -- 172.21.15.84:0/2848586310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f184406a4b0 msgr2=0x7f18440ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:00.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.325+0000 7f1848e9e640 1 --2- 172.21.15.84:0/2848586310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f184406a4b0 0x7f18440ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:00.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.325+0000 7f1848e9e640 1 -- 172.21.15.84:0/2848586310 >> v1:172.21.15.157:6789/0 conn(0x7f184415ad70 legacy=0x7f1844069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:00.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.325+0000 7f1848e9e640 1 -- 172.21.15.84:0/2848586310 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18440c94a0 con 0x7f18440c9ca0 2023-10-16T11:25:00.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.325+0000 7f18437fe640 1 --2- 172.21.15.84:0/2848586310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f184406a4b0 0x7f18440ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:00.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.326+0000 7f1848e9e640 1 --2- 172.21.15.84:0/2848586310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18440c9ca0 0x7f18440ca8f0 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f183c009e50 tx=0x7f183c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e4eba8c6c65c7e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:00.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.326+0000 7f1842ffd640 1 -- 172.21.15.84:0/2848586310 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c02ca70 con 0x7f18440c9ca0 2023-10-16T11:25:00.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.326+0000 7f1842ffd640 1 -- 172.21.15.84:0/2848586310 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c02cbd0 con 0x7f18440c9ca0 2023-10-16T11:25:00.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.326+0000 7f1842ffd640 1 -- 172.21.15.84:0/2848586310 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c0343f0 con 0x7f18440c9ca0 2023-10-16T11:25:00.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.327+0000 7f184a901640 1 -- 172.21.15.84:0/2848586310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18440c9ca0 msgr2=0x7f18440ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:00.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.327+0000 7f184a901640 1 --2- 172.21.15.84:0/2848586310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18440c9ca0 0x7f18440ca8f0 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f183c009e50 tx=0x7f183c02bed0 comp rx=0 tx=0).stop 2023-10-16T11:25:00.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.327+0000 7f184a901640 1 -- 172.21.15.84:0/2848586310 shutdown_connections 2023-10-16T11:25:00.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.327+0000 7f184a901640 1 --2- 172.21.15.84:0/2848586310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18440c9ca0 0x7f18440ca8f0 unknown :-1 s=CLOSED pgs=2739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:00.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.327+0000 7f184a901640 1 --2- 172.21.15.84:0/2848586310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f184406a4b0 0x7f18440ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:00.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.327+0000 7f184a901640 1 -- 172.21.15.84:0/2848586310 >> 172.21.15.84:0/2848586310 conn(0x7f18440ce9b0 msgr2=0x7f18440cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:00.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.327+0000 7f184a901640 1 -- 172.21.15.84:0/2848586310 shutdown_connections 2023-10-16T11:25:00.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.327+0000 7f184a901640 1 -- 172.21.15.84:0/2848586310 wait complete. 2023-10-16T11:25:00.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.329+0000 7f184a901640 1 Processor -- start 2023-10-16T11:25:00.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.329+0000 7f184a901640 1 -- start start 2023-10-16T11:25:00.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.330+0000 7f184a901640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f184406a4b0 0x7f18440c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:00.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.330+0000 7f184a901640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18440c9ca0 0x7f18440b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:00.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.330+0000 7f184a901640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f184415ad70 0x7f18440b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:00.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.330+0000 7f184a901640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1844159670 con 0x7f184406a4b0 2023-10-16T11:25:00.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.330+0000 7f184a901640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18440c4bb0 con 0x7f18440c9ca0 2023-10-16T11:25:00.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.330+0000 7f184a901640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18440acee0 con 0x7f184415ad70 2023-10-16T11:25:00.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.330+0000 7f1843fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f184406a4b0 0x7f18440c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:00.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.331+0000 7f1843fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f184406a4b0 0x7f18440c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56840/0 (socket says 172.21.15.84:56840) 2023-10-16T11:25:00.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.331+0000 7f1843fff640 1 -- 172.21.15.84:0/3676967990 learned_addr learned my addr 172.21.15.84:0/3676967990 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:00.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.331+0000 7f18437fe640 1 --2- 172.21.15.84:0/3676967990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18440c9ca0 0x7f18440b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:00.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.331+0000 7f1843fff640 1 -- 172.21.15.84:0/3676967990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f184415ad70 msgr2=0x7f18440b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:00.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.331+0000 7f1843fff640 1 --2- 172.21.15.84:0/3676967990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f184415ad70 0x7f18440b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:00.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.331+0000 7f1843fff640 1 -- 172.21.15.84:0/3676967990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18440c9ca0 msgr2=0x7f18440b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:00.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.331+0000 7f1843fff640 1 --2- 172.21.15.84:0/3676967990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18440c9ca0 0x7f18440b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:00.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.331+0000 7f1843fff640 1 -- 172.21.15.84:0/3676967990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183c009b30 con 0x7f184406a4b0 2023-10-16T11:25:00.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.331+0000 7f1843fff640 1 --2- 172.21.15.84:0/3676967990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f184406a4b0 0x7f18440c4670 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f183400b3f0 tx=0x7f183400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:00.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.332+0000 7f1840ff9640 1 -- 172.21.15.84:0/3676967990 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1834011830 con 0x7f184406a4b0 2023-10-16T11:25:00.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.332+0000 7f1840ff9640 1 -- 172.21.15.84:0/3676967990 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1834011990 con 0x7f184406a4b0 2023-10-16T11:25:00.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.332+0000 7f184a901640 1 -- 172.21.15.84:0/3676967990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18440c6700 con 0x7f184406a4b0 2023-10-16T11:25:00.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.332+0000 7f1840ff9640 1 -- 172.21.15.84:0/3676967990 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1834011d40 con 0x7f184406a4b0 2023-10-16T11:25:00.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.332+0000 7f184a901640 1 -- 172.21.15.84:0/3676967990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18440ac890 con 0x7f184406a4b0 2023-10-16T11:25:00.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.333+0000 7f184a901640 1 -- 172.21.15.84:0/3676967990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f184404e160 con 0x7f184406a4b0 2023-10-16T11:25:00.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.334+0000 7f1840ff9640 1 -- 172.21.15.84:0/3676967990 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 136) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f183401a400 con 0x7f184406a4b0 2023-10-16T11:25:00.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.335+0000 7f1840ff9640 1 --2- 172.21.15.84:0/3676967990 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f183006ad40 0x7f183006d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:00.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.336+0000 7f1840ff9640 1 -- 172.21.15.84:0/3676967990 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6945+0+0 (secure 0 0 0) 0x7f1834089750 con 0x7f184406a4b0 2023-10-16T11:25:00.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.336+0000 7f18437fe640 1 --2- 172.21.15.84:0/3676967990 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f183006ad40 0x7f183006d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:00.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.336+0000 7f18437fe640 1 --2- 172.21.15.84:0/3676967990 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f183006ad40 0x7f183006d200 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f18440c8b10 tx=0x7f1838008040 comp rx=0 tx=0).ready entity=mgr.11682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:00.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.340+0000 7f1840ff9640 1 -- 172.21.15.84:0/3676967990 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1834054500 con 0x7f184406a4b0 2023-10-16T11:25:00.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.566+0000 7f184a901640 1 -- 172.21.15.84:0/3676967990 --> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f18440c9780 con 0x7f183006ad40 2023-10-16T11:25:00.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.752+0000 7f1840ff9640 1 -- 172.21.15.84:0/3676967990 <== mgr.11682 v2:172.21.15.84:6832/2880641466 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7f183800c040 con 0x7f183006ad40 2023-10-16T11:25:00.755 INFO:teuthology.orchestra.run.smithi084.stdout:Retention added to path mandatory_fs_argument_test_dir 2023-10-16T11:25:00.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.756+0000 7f184a901640 1 -- 172.21.15.84:0/3676967990 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f183006ad40 msgr2=0x7f183006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:00.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.756+0000 7f184a901640 1 --2- 172.21.15.84:0/3676967990 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f183006ad40 0x7f183006d200 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f18440c8b10 tx=0x7f1838008040 comp rx=0 tx=0).stop 2023-10-16T11:25:00.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.756+0000 7f184a901640 1 -- 172.21.15.84:0/3676967990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f184406a4b0 msgr2=0x7f18440c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:00.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.756+0000 7f184a901640 1 --2- 172.21.15.84:0/3676967990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f184406a4b0 0x7f18440c4670 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f183400b3f0 tx=0x7f183400b8b0 comp rx=0 tx=0).stop 2023-10-16T11:25:00.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.756+0000 7f184a901640 1 -- 172.21.15.84:0/3676967990 shutdown_connections 2023-10-16T11:25:00.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.756+0000 7f184a901640 1 --2- 172.21.15.84:0/3676967990 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f183006ad40 0x7f183006d200 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:00.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.756+0000 7f184a901640 1 --2- 172.21.15.84:0/3676967990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f184415ad70 0x7f18440b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:00.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.756+0000 7f184a901640 1 --2- 172.21.15.84:0/3676967990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18440c9ca0 0x7f18440b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:00.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.756+0000 7f184a901640 1 --2- 172.21.15.84:0/3676967990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f184406a4b0 0x7f18440c4670 unknown :-1 s=CLOSED pgs=2740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:00.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.756+0000 7f184a901640 1 -- 172.21.15.84:0/3676967990 >> 172.21.15.84:0/3676967990 conn(0x7f18440ce9b0 msgr2=0x7f18440afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:00.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.756+0000 7f184a901640 1 -- 172.21.15.84:0/3676967990 shutdown_connections 2023-10-16T11:25:00.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.756+0000 7f184a901640 1 -- 172.21.15.84:0/3676967990 wait complete. 2023-10-16T11:25:00.779 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path mandatory_fs_argument_test_dir 2023-10-16T11:25:00.779 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list mandatory_fs_argument_test_dir --fs backup_fs --format json 2023-10-16T11:25:00.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.991+0000 7fead0d78640 1 Processor -- start 2023-10-16T11:25:00.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.992+0000 7fead0d78640 1 -- start start 2023-10-16T11:25:00.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.993+0000 7fead0d78640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feacc1599c0 0x7feacc0c9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:00.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.993+0000 7fead0d78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feacc159da0 0x7feacc15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:00.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.993+0000 7fead0d78640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feacc15dd10 con 0x7feacc0cce20 2023-10-16T11:25:00.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.993+0000 7fead0d78640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feacc15dfb0 con 0x7feacc1599c0 2023-10-16T11:25:00.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.993+0000 7fead0d78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feacc15e290 con 0x7feacc159da0 2023-10-16T11:25:00.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.993+0000 7feaca59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feacc1599c0 0x7feacc0c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:00.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.993+0000 7feaca59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feacc1599c0 0x7feacc0c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56866/0 (socket says 172.21.15.84:56866) 2023-10-16T11:25:00.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.993+0000 7feaca59c640 1 -- 172.21.15.84:0/555832351 learned_addr learned my addr 172.21.15.84:0/555832351 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:00.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.994+0000 7feac3fff640 1 --2- 172.21.15.84:0/555832351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feacc159da0 0x7feacc15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:00.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.994+0000 7feaca59c640 1 -- 172.21.15.84:0/555832351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feacc159da0 msgr2=0x7feacc15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:00.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.994+0000 7feaca59c640 1 --2- 172.21.15.84:0/555832351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feacc159da0 0x7feacc15d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:00.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.994+0000 7feaca59c640 1 -- 172.21.15.84:0/555832351 >> v1:172.21.15.84:6789/0 conn(0x7feacc0cce20 legacy=0x7feacc06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:00.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.994+0000 7feaca59c640 1 -- 172.21.15.84:0/555832351 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feacc15e570 con 0x7feacc1599c0 2023-10-16T11:25:01.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.994+0000 7feac3fff640 1 --2- 172.21.15.84:0/555832351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feacc159da0 0x7feacc15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:01.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.995+0000 7feaca59c640 1 --2- 172.21.15.84:0/555832351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc1599c0 0x7feacc0c9300 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7feabc009870 tx=0x7feabc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f205fd090999f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:01.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.996+0000 7feac9d9b640 1 -- 172.21.15.84:0/555832351 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc030840 con 0x7feacc1599c0 2023-10-16T11:25:01.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.996+0000 7feac9d9b640 1 -- 172.21.15.84:0/555832351 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc0309a0 con 0x7feacc1599c0 2023-10-16T11:25:01.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.996+0000 7feac9d9b640 1 -- 172.21.15.84:0/555832351 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc030d30 con 0x7feacc1599c0 2023-10-16T11:25:01.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.996+0000 7fead0d78640 1 -- 172.21.15.84:0/555832351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc1599c0 msgr2=0x7feacc0c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:01.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.996+0000 7fead0d78640 1 --2- 172.21.15.84:0/555832351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc1599c0 0x7feacc0c9300 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7feabc009870 tx=0x7feabc02b980 comp rx=0 tx=0).stop 2023-10-16T11:25:01.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.996+0000 7fead0d78640 1 -- 172.21.15.84:0/555832351 shutdown_connections 2023-10-16T11:25:01.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.996+0000 7fead0d78640 1 --2- 172.21.15.84:0/555832351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feacc159da0 0x7feacc15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.996+0000 7fead0d78640 1 --2- 172.21.15.84:0/555832351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc1599c0 0x7feacc0c9300 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.996+0000 7fead0d78640 1 -- 172.21.15.84:0/555832351 >> 172.21.15.84:0/555832351 conn(0x7feacc0cb360 msgr2=0x7feacc0cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:01.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.997+0000 7fead0d78640 1 -- 172.21.15.84:0/555832351 shutdown_connections 2023-10-16T11:25:01.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.997+0000 7fead0d78640 1 -- 172.21.15.84:0/555832351 wait complete. 2023-10-16T11:25:01.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.998+0000 7fead0d78640 1 Processor -- start 2023-10-16T11:25:01.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.998+0000 7fead0d78640 1 -- start start 2023-10-16T11:25:01.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.999+0000 7fead0d78640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc0cce20 0x7feacc0b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.999+0000 7fead0d78640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feacc1599c0 0x7feacc0b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.999+0000 7fead0d78640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feacc159da0 0x7feacc14acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.999+0000 7fead0d78640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc0b2de0 con 0x7feacc0cce20 2023-10-16T11:25:01.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.999+0000 7fead0d78640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc0b3080 con 0x7feacc159da0 2023-10-16T11:25:01.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:00.999+0000 7fead0d78640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feacc0b3360 con 0x7feacc1599c0 2023-10-16T11:25:01.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.000+0000 7feaca59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc0cce20 0x7feacc0b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.000+0000 7feaca59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc0cce20 0x7feacc0b0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56878/0 (socket says 172.21.15.84:56878) 2023-10-16T11:25:01.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.000+0000 7feaca59c640 1 -- 172.21.15.84:0/183187424 learned_addr learned my addr 172.21.15.84:0/183187424 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:01.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.000+0000 7feac3fff640 1 --2- 172.21.15.84:0/183187424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feacc1599c0 0x7feacc0b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.000+0000 7feaca59c640 1 -- 172.21.15.84:0/183187424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feacc1599c0 msgr2=0x7feacc0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:01.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.000+0000 7feaca59c640 1 --2- 172.21.15.84:0/183187424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feacc1599c0 0x7feacc0b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.000+0000 7feaca59c640 1 -- 172.21.15.84:0/183187424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feacc159da0 msgr2=0x7feacc14acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:01.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.000+0000 7feacad9d640 1 --2- 172.21.15.84:0/183187424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feacc159da0 0x7feacc14acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.000+0000 7feaca59c640 1 --2- 172.21.15.84:0/183187424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feacc159da0 0x7feacc14acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.000+0000 7feaca59c640 1 -- 172.21.15.84:0/183187424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc0094d0 con 0x7feacc0cce20 2023-10-16T11:25:01.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.000+0000 7feac3fff640 1 --2- 172.21.15.84:0/183187424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feacc1599c0 0x7feacc0b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:01.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.001+0000 7feaca59c640 1 --2- 172.21.15.84:0/183187424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc0cce20 0x7feacc0b0cd0 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7feabc000c00 tx=0x7feabc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:01.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.001+0000 7feac2ffd640 1 -- 172.21.15.84:0/183187424 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc030810 con 0x7feacc0cce20 2023-10-16T11:25:01.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.001+0000 7feac2ffd640 1 -- 172.21.15.84:0/183187424 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc043400 con 0x7feacc0cce20 2023-10-16T11:25:01.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.001+0000 7feac2ffd640 1 -- 172.21.15.84:0/183187424 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc0437d0 con 0x7feacc0cce20 2023-10-16T11:25:01.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.001+0000 7fead0d78640 1 -- 172.21.15.84:0/183187424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feacc0b3640 con 0x7feacc0cce20 2023-10-16T11:25:01.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.001+0000 7fead0d78640 1 -- 172.21.15.84:0/183187424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feacc0b3c00 con 0x7feacc0cce20 2023-10-16T11:25:01.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.002+0000 7fead0d78640 1 -- 172.21.15.84:0/183187424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feacc0b3df0 con 0x7feacc0cce20 2023-10-16T11:25:01.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.003+0000 7feac2ffd640 1 -- 172.21.15.84:0/183187424 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 137) v1 ==== 89451+0+0 (secure 0 0 0) 0x7feabc043930 con 0x7feacc0cce20 2023-10-16T11:25:01.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.004+0000 7feac2ffd640 1 --2- 172.21.15.84:0/183187424 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7feaac06b100 0x7feaac06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.004+0000 7feac2ffd640 1 -- 172.21.15.84:0/183187424 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6945+0+0 (secure 0 0 0) 0x7feabc03e070 con 0x7feacc0cce20 2023-10-16T11:25:01.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.004+0000 7feac3fff640 1 --2- 172.21.15.84:0/183187424 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7feaac06b100 0x7feaac06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.005+0000 7feac3fff640 1 --2- 172.21.15.84:0/183187424 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7feaac06b100 0x7feaac06d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feab4006fd0 tx=0x7feab4008040 comp rx=0 tx=0).ready entity=mgr.11682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:01.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.009+0000 7feac2ffd640 1 -- 172.21.15.84:0/183187424 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7feabc077b60 con 0x7feacc0cce20 2023-10-16T11:25:01.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.222+0000 7fead0d78640 1 -- 172.21.15.84:0/183187424 --> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feacc06fa50 con 0x7feaac06b100 2023-10-16T11:25:01.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.238+0000 7feac2ffd640 1 -- 172.21.15.84:0/183187424 <== mgr.11682 v2:172.21.15.84:6832/2880641466 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+88 (secure 0 0 0) 0x7feab400c040 con 0x7feaac06b100 2023-10-16T11:25:01.240 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:01.240 INFO:teuthology.orchestra.run.smithi084.stdout:{"path": "mandatory_fs_argument_test_dir", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-16T11:25:01.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.240+0000 7fead0d78640 1 -- 172.21.15.84:0/183187424 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7feaac06b100 msgr2=0x7feaac06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:01.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.240+0000 7fead0d78640 1 --2- 172.21.15.84:0/183187424 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7feaac06b100 0x7feaac06d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feab4006fd0 tx=0x7feab4008040 comp rx=0 tx=0).stop 2023-10-16T11:25:01.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.240+0000 7fead0d78640 1 -- 172.21.15.84:0/183187424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc0cce20 msgr2=0x7feacc0b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:01.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.240+0000 7fead0d78640 1 --2- 172.21.15.84:0/183187424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc0cce20 0x7feacc0b0cd0 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7feabc000c00 tx=0x7feabc004730 comp rx=0 tx=0).stop 2023-10-16T11:25:01.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.241+0000 7fead0d78640 1 -- 172.21.15.84:0/183187424 shutdown_connections 2023-10-16T11:25:01.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.241+0000 7fead0d78640 1 --2- 172.21.15.84:0/183187424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feacc159da0 0x7feacc14acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.241+0000 7fead0d78640 1 --2- 172.21.15.84:0/183187424 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7feaac06b100 0x7feaac06d5c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.241+0000 7fead0d78640 1 --2- 172.21.15.84:0/183187424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feacc1599c0 0x7feacc0b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.241+0000 7fead0d78640 1 --2- 172.21.15.84:0/183187424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc0cce20 0x7feacc0b0cd0 unknown :-1 s=CLOSED pgs=2745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.241+0000 7fead0d78640 1 -- 172.21.15.84:0/183187424 >> 172.21.15.84:0/183187424 conn(0x7feacc0cb360 msgr2=0x7feacc06b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:01.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.241+0000 7fead0d78640 1 -- 172.21.15.84:0/183187424 shutdown_connections 2023-10-16T11:25:01.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.241+0000 7fead0d78640 1 -- 172.21.15.84:0/183187424 wait complete. 2023-10-16T11:25:01.261 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "mandatory_fs_argument_test_dir", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-16T11:25:01.262 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status mandatory_fs_argument_test_dir --fs backup_fs --format json 2023-10-16T11:25:01.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.477+0000 7fd1c4565640 1 Processor -- start 2023-10-16T11:25:01.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.477+0000 7fd1c4565640 1 -- start start 2023-10-16T11:25:01.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.479+0000 7fd1c4565640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1bc0cac20 0x7fd1bc0b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.479+0000 7fd1c4565640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1bc15b310 con 0x7fd1bc15ac10 2023-10-16T11:25:01.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.479+0000 7fd1c4565640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1bc06f640 con 0x7fd1bc0b0360 2023-10-16T11:25:01.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.479+0000 7fd1c4565640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc15f800 con 0x7fd1bc0cac20 2023-10-16T11:25:01.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.479+0000 7fd1c2301640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd1bc15ac10 0x7fd1bc06c730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53878/0 (socket says 172.21.15.84:53878) 2023-10-16T11:25:01.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.479+0000 7fd1c2301640 1 -- 172.21.15.84:0/1099372159 learned_addr learned my addr 172.21.15.84:0/1099372159 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:01.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.480+0000 7fd1c12ff640 1 -- 172.21.15.84:0/1099372159 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744872889 0 0) 0x7fd1bc15b310 con 0x7fd1bc15ac10 2023-10-16T11:25:01.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.480+0000 7fd1c2b02640 1 --2- 172.21.15.84:0/1099372159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1bc0cac20 0x7fd1bc0b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.480+0000 7fd1c12ff640 1 -- 172.21.15.84:0/1099372159 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1a8003520 con 0x7fd1bc15ac10 2023-10-16T11:25:01.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.480+0000 7fd1c12ff640 1 -- 172.21.15.84:0/1099372159 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3209925832 0 0) 0x7fd1a8003520 con 0x7fd1bc15ac10 2023-10-16T11:25:01.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.480+0000 7fd1c12ff640 1 -- 172.21.15.84:0/1099372159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1bc0cac20 msgr2=0x7fd1bc0b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:01.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.480+0000 7fd1c12ff640 1 --2- 172.21.15.84:0/1099372159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1bc0cac20 0x7fd1bc0b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.480+0000 7fd1c12ff640 1 -- 172.21.15.84:0/1099372159 >> v1:172.21.15.157:6789/0 conn(0x7fd1bc0b0360 legacy=0x7fd1bc0ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:01.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.480+0000 7fd1c12ff640 1 -- 172.21.15.84:0/1099372159 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1bc15b5b0 con 0x7fd1bc15ac10 2023-10-16T11:25:01.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.480+0000 7fd1c12ff640 1 -- 172.21.15.84:0/1099372159 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd1b8001670 con 0x7fd1bc15ac10 2023-10-16T11:25:01.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.480+0000 7fd1c12ff640 1 -- 172.21.15.84:0/1099372159 >> v1:172.21.15.84:6789/0 conn(0x7fd1bc15ac10 legacy=0x7fd1bc06c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:01.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.480+0000 7fd1c12ff640 1 --2- 172.21.15.84:0/1099372159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1bc0cac20 0x7fd1a8003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.480+0000 7fd1c12ff640 1 --2- 172.21.15.84:0/1099372159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1a8004040 0x7fd1a80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.480+0000 7fd1c12ff640 1 --2- 172.21.15.84:0/1099372159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1a8004d50 0x7fd1a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.480+0000 7fd1c12ff640 1 -- 172.21.15.84:0/1099372159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc15f800 con 0x7fd1a8004040 2023-10-16T11:25:01.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.480+0000 7fd1c12ff640 1 -- 172.21.15.84:0/1099372159 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a800bc30 con 0x7fd1a8004d50 2023-10-16T11:25:01.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.480+0000 7fd1c12ff640 1 -- 172.21.15.84:0/1099372159 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a800bdf0 con 0x7fd1bc0cac20 2023-10-16T11:25:01.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.481+0000 7fd1c1b00640 1 --2- 172.21.15.84:0/1099372159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1a8004040 0x7fd1a80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.481+0000 7fd1c2301640 1 --2- 172.21.15.84:0/1099372159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1bc0cac20 0x7fd1a8003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.481+0000 7fd1c2b02640 1 --2- 172.21.15.84:0/1099372159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1a8004d50 0x7fd1a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.481+0000 7fd1c1b00640 1 -- 172.21.15.84:0/1099372159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1bc0cac20 msgr2=0x7fd1a8003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:01.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.481+0000 7fd1c1b00640 1 --2- 172.21.15.84:0/1099372159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1bc0cac20 0x7fd1a8003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.481+0000 7fd1c1b00640 1 -- 172.21.15.84:0/1099372159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1a8004d50 msgr2=0x7fd1a80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:01.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.481+0000 7fd1c1b00640 1 --2- 172.21.15.84:0/1099372159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1a8004d50 0x7fd1a80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.481+0000 7fd1c1b00640 1 -- 172.21.15.84:0/1099372159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a8003520 con 0x7fd1a8004040 2023-10-16T11:25:01.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.482+0000 7fd1c1b00640 1 --2- 172.21.15.84:0/1099372159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1a8004040 0x7fd1a80047c0 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac001750 tx=0x7fd1a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5048cb80f52f2f3d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:01.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.482+0000 7fd1c12ff640 1 -- 172.21.15.84:0/1099372159 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac0052d0 con 0x7fd1a8004040 2023-10-16T11:25:01.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.482+0000 7fd1c12ff640 1 -- 172.21.15.84:0/1099372159 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ac02e050 con 0x7fd1a8004040 2023-10-16T11:25:01.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.482+0000 7fd1c12ff640 1 -- 172.21.15.84:0/1099372159 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac003830 con 0x7fd1a8004040 2023-10-16T11:25:01.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.483+0000 7fd1c4565640 1 -- 172.21.15.84:0/1099372159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1a8004040 msgr2=0x7fd1a80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:01.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.483+0000 7fd1c4565640 1 --2- 172.21.15.84:0/1099372159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1a8004040 0x7fd1a80047c0 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac001750 tx=0x7fd1a80057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.483+0000 7fd1c2b02640 1 -- 172.21.15.84:0/1099372159 reap_dead start 2023-10-16T11:25:01.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.483+0000 7fd1c4565640 1 -- 172.21.15.84:0/1099372159 shutdown_connections 2023-10-16T11:25:01.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.483+0000 7fd1c4565640 1 -- 172.21.15.84:0/1099372159 >> 172.21.15.84:0/1099372159 conn(0x7fd1bc0cb360 msgr2=0x7fd1bc0cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:01.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.483+0000 7fd1c4565640 1 -- 172.21.15.84:0/1099372159 shutdown_connections 2023-10-16T11:25:01.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.483+0000 7fd1c4565640 1 -- 172.21.15.84:0/1099372159 wait complete. 2023-10-16T11:25:01.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.485+0000 7fd1c4565640 1 Processor -- start 2023-10-16T11:25:01.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.485+0000 7fd1c4565640 1 -- start start 2023-10-16T11:25:01.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.486+0000 7fd1c4565640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1a8004040 0x7fd1bc0b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.486+0000 7fd1c4565640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1bc0c8370 0x7fd1bc0ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.486+0000 7fd1c4565640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1bc0ac890 0x7fd1bc0acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.486+0000 7fd1c4565640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc0ad250 con 0x7fd1a8004040 2023-10-16T11:25:01.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.486+0000 7fd1c4565640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc0cd400 con 0x7fd1bc0c8370 2023-10-16T11:25:01.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.486+0000 7fd1c4565640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc0c87f0 con 0x7fd1bc0ac890 2023-10-16T11:25:01.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.486+0000 7fd1c2301640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1a8004040 0x7fd1bc0b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.486+0000 7fd1c2301640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1a8004040 0x7fd1bc0b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56898/0 (socket says 172.21.15.84:56898) 2023-10-16T11:25:01.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.486+0000 7fd1c2301640 1 -- 172.21.15.84:0/3940567627 learned_addr learned my addr 172.21.15.84:0/3940567627 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:01.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.487+0000 7fd1c2b02640 1 --2- 172.21.15.84:0/3940567627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1bc0ac890 0x7fd1bc0acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.487+0000 7fd1c2301640 1 -- 172.21.15.84:0/3940567627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1bc0ac890 msgr2=0x7fd1bc0acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:01.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.487+0000 7fd1c1b00640 1 --2- 172.21.15.84:0/3940567627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1bc0c8370 0x7fd1bc0ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.487+0000 7fd1c2301640 1 --2- 172.21.15.84:0/3940567627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1bc0ac890 0x7fd1bc0acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.487+0000 7fd1c2301640 1 -- 172.21.15.84:0/3940567627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1bc0c8370 msgr2=0x7fd1bc0ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:01.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.487+0000 7fd1c2301640 1 --2- 172.21.15.84:0/3940567627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1bc0c8370 0x7fd1bc0ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.487+0000 7fd1c2301640 1 -- 172.21.15.84:0/3940567627 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1bc0cd630 con 0x7fd1a8004040 2023-10-16T11:25:01.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.487+0000 7fd1c2301640 1 --2- 172.21.15.84:0/3940567627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1a8004040 0x7fd1bc0b23a0 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800e740 tx=0x7fd1b800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:01.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.487+0000 7fd1b2ffd640 1 -- 172.21.15.84:0/3940567627 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b800ee50 con 0x7fd1a8004040 2023-10-16T11:25:01.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.487+0000 7fd1b2ffd640 1 -- 172.21.15.84:0/3940567627 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b800c8d0 con 0x7fd1a8004040 2023-10-16T11:25:01.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.487+0000 7fd1b2ffd640 1 -- 172.21.15.84:0/3940567627 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b800cc80 con 0x7fd1a8004040 2023-10-16T11:25:01.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.487+0000 7fd1c4565640 1 -- 172.21.15.84:0/3940567627 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1bc0cd910 con 0x7fd1a8004040 2023-10-16T11:25:01.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.487+0000 7fd1c4565640 1 -- 172.21.15.84:0/3940567627 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1bc0cded0 con 0x7fd1a8004040 2023-10-16T11:25:01.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.488+0000 7fd1c4565640 1 -- 172.21.15.84:0/3940567627 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1bc0c5f30 con 0x7fd1a8004040 2023-10-16T11:25:01.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.489+0000 7fd1b2ffd640 1 -- 172.21.15.84:0/3940567627 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 137) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fd1b801d460 con 0x7fd1a8004040 2023-10-16T11:25:01.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.490+0000 7fd1b2ffd640 1 --2- 172.21.15.84:0/3940567627 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fd1a006b100 0x7fd1a006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.490+0000 7fd1b2ffd640 1 -- 172.21.15.84:0/3940567627 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6945+0+0 (secure 0 0 0) 0x7fd1b8089dd0 con 0x7fd1a8004040 2023-10-16T11:25:01.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.491+0000 7fd1c1b00640 1 --2- 172.21.15.84:0/3940567627 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fd1a006b100 0x7fd1a006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.491+0000 7fd1c1b00640 1 --2- 172.21.15.84:0/3940567627 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fd1a006b100 0x7fd1a006d5c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac033000 tx=0x7fd1ac009030 comp rx=0 tx=0).ready entity=mgr.11682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:01.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.495+0000 7fd1b2ffd640 1 -- 172.21.15.84:0/3940567627 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd1bc0c5f30 con 0x7fd1a8004040 2023-10-16T11:25:01.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.706+0000 7fd1c4565640 1 -- 172.21.15.84:0/3940567627 --> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd1bc15bb70 con 0x7fd1a006b100 2023-10-16T11:25:01.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.723+0000 7fd1b2ffd640 1 -- 172.21.15.84:0/3940567627 <== mgr.11682 v2:172.21.15.84:6832/2880641466 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+353 (secure 0 0 0) 0x7fd1ac005370 con 0x7fd1a006b100 2023-10-16T11:25:01.726 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:01.726 INFO:teuthology.orchestra.run.smithi084.stdout:[{"fs": "backup_fs", "subvol": null, "group": null, "path": "mandatory_fs_argument_test_dir", "rel_path": "mandatory_fs_argument_test_dir", "schedule": "1M", "retention": {"M": 1}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T11:24:59", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-16T11:25:01.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.726+0000 7fd1c4565640 1 -- 172.21.15.84:0/3940567627 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fd1a006b100 msgr2=0x7fd1a006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:01.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.726+0000 7fd1c4565640 1 --2- 172.21.15.84:0/3940567627 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fd1a006b100 0x7fd1a006d5c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac033000 tx=0x7fd1ac009030 comp rx=0 tx=0).stop 2023-10-16T11:25:01.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.726+0000 7fd1c4565640 1 -- 172.21.15.84:0/3940567627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1a8004040 msgr2=0x7fd1bc0b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:01.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.726+0000 7fd1c4565640 1 --2- 172.21.15.84:0/3940567627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1a8004040 0x7fd1bc0b23a0 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800e740 tx=0x7fd1b800ec00 comp rx=0 tx=0).stop 2023-10-16T11:25:01.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.726+0000 7fd1c4565640 1 -- 172.21.15.84:0/3940567627 shutdown_connections 2023-10-16T11:25:01.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.726+0000 7fd1c4565640 1 --2- 172.21.15.84:0/3940567627 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fd1a006b100 0x7fd1a006d5c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.726+0000 7fd1c4565640 1 --2- 172.21.15.84:0/3940567627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1bc0ac890 0x7fd1bc0acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.726+0000 7fd1c4565640 1 --2- 172.21.15.84:0/3940567627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1bc0c8370 0x7fd1bc0ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.726+0000 7fd1c4565640 1 --2- 172.21.15.84:0/3940567627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1a8004040 0x7fd1bc0b23a0 unknown :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.726+0000 7fd1c4565640 1 -- 172.21.15.84:0/3940567627 >> 172.21.15.84:0/3940567627 conn(0x7fd1bc0cb360 msgr2=0x7fd1bc0ca310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:01.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.726+0000 7fd1c4565640 1 -- 172.21.15.84:0/3940567627 shutdown_connections 2023-10-16T11:25:01.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.726+0000 7fd1c4565640 1 -- 172.21.15.84:0/3940567627 wait complete. 2023-10-16T11:25:01.748 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "backup_fs", "subvol": null, "group": null, "path": "mandatory_fs_argument_test_dir", "rel_path": "mandatory_fs_argument_test_dir", "schedule": "1M", "retention": {"M": 1}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T11:24:59", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-16T11:25:01.748 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention remove mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-16T11:25:01.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.970+0000 7f2accd21640 1 Processor -- start 2023-10-16T11:25:01.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.970+0000 7f2accd21640 1 -- start start 2023-10-16T11:25:01.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.972+0000 7f2accd21640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ac80cd260 0x7f2ac80ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.972+0000 7f2accd21640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ac80ce480 con 0x7f2ac815cc00 2023-10-16T11:25:01.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.972+0000 7f2accd21640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ac80cd7b0 con 0x7f2ac80ad670 2023-10-16T11:25:01.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.972+0000 7f2accd21640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ac80cda90 con 0x7f2ac80cd260 2023-10-16T11:25:01.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.972+0000 7f2ac77fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2ac815cc00 0x7f2ac80ca410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53880/0 (socket says 172.21.15.84:53880) 2023-10-16T11:25:01.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.972+0000 7f2ac77fe640 1 -- 172.21.15.84:0/3237934023 learned_addr learned my addr 172.21.15.84:0/3237934023 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:01.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.973+0000 7f2abffff640 1 --2- 172.21.15.84:0/3237934023 >> v2:172.21.15.157:3300/0 conn(0x7f2ac80cd260 0x7f2ac80ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.973+0000 7f2ac6ffd640 1 -- 172.21.15.84:0/3237934023 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016181705 0 0) 0x7f2ac80ce480 con 0x7f2ac815cc00 2023-10-16T11:25:01.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.973+0000 7f2ac6ffd640 1 -- 172.21.15.84:0/3237934023 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ab4003520 con 0x7f2ac815cc00 2023-10-16T11:25:01.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.973+0000 7f2ac6ffd640 1 -- 172.21.15.84:0/3237934023 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 573314420 0 0) 0x7f2ab4003520 con 0x7f2ac815cc00 2023-10-16T11:25:01.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.973+0000 7f2ac6ffd640 1 -- 172.21.15.84:0/3237934023 >> v2:172.21.15.157:3300/0 conn(0x7f2ac80cd260 msgr2=0x7f2ac80ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:01.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.973+0000 7f2ac6ffd640 1 --2- 172.21.15.84:0/3237934023 >> v2:172.21.15.157:3300/0 conn(0x7f2ac80cd260 0x7f2ac80ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.973+0000 7f2ac6ffd640 1 -- 172.21.15.84:0/3237934023 >> v1:172.21.15.157:6789/0 conn(0x7f2ac80ad670 legacy=0x7f2ac80c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:01.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.973+0000 7f2ac6ffd640 1 -- 172.21.15.84:0/3237934023 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac80cdd70 con 0x7f2ac815cc00 2023-10-16T11:25:01.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.973+0000 7f2ac6ffd640 1 -- 172.21.15.84:0/3237934023 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f2ab8001670 con 0x7f2ac815cc00 2023-10-16T11:25:01.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.973+0000 7f2ac6ffd640 1 -- 172.21.15.84:0/3237934023 >> v1:172.21.15.84:6789/0 conn(0x7f2ac815cc00 legacy=0x7f2ac80ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:01.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.973+0000 7f2ac6ffd640 1 --2- 172.21.15.84:0/3237934023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ab40039a0 0x7f2ab4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.973+0000 7f2ac6ffd640 1 --2- 172.21.15.84:0/3237934023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ab40047c0 0x7f2ab4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.973+0000 7f2ac6ffd640 1 --2- 172.21.15.84:0/3237934023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ab4005180 0x7f2ab400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.973+0000 7f2ac6ffd640 1 -- 172.21.15.84:0/3237934023 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac80cda90 con 0x7f2ab40039a0 2023-10-16T11:25:01.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.973+0000 7f2ac6ffd640 1 -- 172.21.15.84:0/3237934023 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab400c020 con 0x7f2ab4005180 2023-10-16T11:25:01.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.973+0000 7f2ac6ffd640 1 -- 172.21.15.84:0/3237934023 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab400c1e0 con 0x7f2ab40047c0 2023-10-16T11:25:01.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.974+0000 7f2ac77fe640 1 --2- 172.21.15.84:0/3237934023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ab40039a0 0x7f2ab4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.974+0000 7f2ac7fff640 1 --2- 172.21.15.84:0/3237934023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ab4005180 0x7f2ab400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.974+0000 7f2ac77fe640 1 -- 172.21.15.84:0/3237934023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ab40047c0 msgr2=0x7f2ab4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:01.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.974+0000 7f2ac77fe640 1 --2- 172.21.15.84:0/3237934023 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ab40047c0 0x7f2ab4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.974+0000 7f2ac77fe640 1 -- 172.21.15.84:0/3237934023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ab4005180 msgr2=0x7f2ab400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:01.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.974+0000 7f2ac77fe640 1 --2- 172.21.15.84:0/3237934023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ab4005180 0x7f2ab400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.974+0000 7f2ac77fe640 1 -- 172.21.15.84:0/3237934023 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab4003520 con 0x7f2ab40039a0 2023-10-16T11:25:01.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.974+0000 7f2ac7fff640 1 --2- 172.21.15.84:0/3237934023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ab4005180 0x7f2ab400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:01.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.974+0000 7f2ac7fff640 1 -- 172.21.15.84:0/3237934023 reap_dead start 2023-10-16T11:25:01.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.975+0000 7f2ac77fe640 1 --2- 172.21.15.84:0/3237934023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ab40039a0 0x7f2ab4003da0 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8007fc0 tx=0x7f2ab40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa160af1920d7dd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:01.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.976+0000 7f2ac6ffd640 1 -- 172.21.15.84:0/3237934023 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8005ad0 con 0x7f2ab40039a0 2023-10-16T11:25:01.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.976+0000 7f2ac6ffd640 1 -- 172.21.15.84:0/3237934023 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab8005c50 con 0x7f2ab40039a0 2023-10-16T11:25:01.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.976+0000 7f2ac6ffd640 1 -- 172.21.15.84:0/3237934023 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab802c7c0 con 0x7f2ab40039a0 2023-10-16T11:25:01.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.976+0000 7f2accd21640 1 -- 172.21.15.84:0/3237934023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ab40039a0 msgr2=0x7f2ab4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:01.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.976+0000 7f2accd21640 1 --2- 172.21.15.84:0/3237934023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ab40039a0 0x7f2ab4003da0 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8007fc0 tx=0x7f2ab40057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.976+0000 7f2accd21640 1 -- 172.21.15.84:0/3237934023 shutdown_connections 2023-10-16T11:25:01.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.976+0000 7f2accd21640 1 --2- 172.21.15.84:0/3237934023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ab40039a0 0x7f2ab4003da0 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.976+0000 7f2accd21640 1 -- 172.21.15.84:0/3237934023 >> 172.21.15.84:0/3237934023 conn(0x7f2ac80b0170 msgr2=0x7f2ac80b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:01.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.976+0000 7f2accd21640 1 -- 172.21.15.84:0/3237934023 shutdown_connections 2023-10-16T11:25:01.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.976+0000 7f2accd21640 1 -- 172.21.15.84:0/3237934023 wait complete. 2023-10-16T11:25:01.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.978+0000 7f2accd21640 1 Processor -- start 2023-10-16T11:25:01.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.978+0000 7f2accd21640 1 -- start start 2023-10-16T11:25:01.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.979+0000 7f2accd21640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ab40039a0 0x7f2ac80c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.979+0000 7f2accd21640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac80c74e0 0x7f2ac80c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.979+0000 7f2accd21640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ac8147ad0 0x7f2ac8147f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.979+0000 7f2accd21640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac8148470 con 0x7f2ac80c74e0 2023-10-16T11:25:01.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.979+0000 7f2accd21640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac80b3e10 con 0x7f2ac8147ad0 2023-10-16T11:25:01.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.979+0000 7f2accd21640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac80c7fc0 con 0x7f2ab40039a0 2023-10-16T11:25:01.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.980+0000 7f2abffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac80c74e0 0x7f2ac80c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.980+0000 7f2abffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac80c74e0 0x7f2ac80c7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56922/0 (socket says 172.21.15.84:56922) 2023-10-16T11:25:01.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.980+0000 7f2abffff640 1 -- 172.21.15.84:0/823595598 learned_addr learned my addr 172.21.15.84:0/823595598 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:01.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.980+0000 7f2ac7fff640 1 --2- 172.21.15.84:0/823595598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ac8147ad0 0x7f2ac8147f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.980+0000 7f2ac77fe640 1 --2- 172.21.15.84:0/823595598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ab40039a0 0x7f2ac80c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.980+0000 7f2abffff640 1 -- 172.21.15.84:0/823595598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ab40039a0 msgr2=0x7f2ac80c4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:01.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.980+0000 7f2abffff640 1 --2- 172.21.15.84:0/823595598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ab40039a0 0x7f2ac80c4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.980+0000 7f2abffff640 1 -- 172.21.15.84:0/823595598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ac8147ad0 msgr2=0x7f2ac8147f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:01.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.980+0000 7f2abffff640 1 --2- 172.21.15.84:0/823595598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ac8147ad0 0x7f2ac8147f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:01.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.980+0000 7f2abffff640 1 -- 172.21.15.84:0/823595598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac80c82a0 con 0x7f2ac80c74e0 2023-10-16T11:25:01.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.980+0000 7f2ac7fff640 1 --2- 172.21.15.84:0/823595598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ac8147ad0 0x7f2ac8147f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:01.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.980+0000 7f2abffff640 1 --2- 172.21.15.84:0/823595598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac80c74e0 0x7f2ac80c7960 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000e770 tx=0x7f2ab000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:01.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.981+0000 7f2ac4ff9640 1 -- 172.21.15.84:0/823595598 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab000c730 con 0x7f2ac80c74e0 2023-10-16T11:25:01.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.981+0000 7f2ac4ff9640 1 -- 172.21.15.84:0/823595598 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab000c890 con 0x7f2ac80c74e0 2023-10-16T11:25:01.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.981+0000 7f2ac4ff9640 1 -- 172.21.15.84:0/823595598 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab000cc40 con 0x7f2ac80c74e0 2023-10-16T11:25:01.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.981+0000 7f2accd21640 1 -- 172.21.15.84:0/823595598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac80c8580 con 0x7f2ac80c74e0 2023-10-16T11:25:01.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.981+0000 7f2accd21640 1 -- 172.21.15.84:0/823595598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac806ae90 con 0x7f2ac80c74e0 2023-10-16T11:25:01.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.982+0000 7f2accd21640 1 -- 172.21.15.84:0/823595598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac80674f0 con 0x7f2ac80c74e0 2023-10-16T11:25:01.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.983+0000 7f2ac4ff9640 1 -- 172.21.15.84:0/823595598 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 137) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f2ab0013470 con 0x7f2ac80c74e0 2023-10-16T11:25:01.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.984+0000 7f2ac4ff9640 1 --2- 172.21.15.84:0/823595598 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f2aa806b100 0x7f2aa806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:01.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.984+0000 7f2ac4ff9640 1 -- 172.21.15.84:0/823595598 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6945+0+0 (secure 0 0 0) 0x7f2ab008a110 con 0x7f2ac80c74e0 2023-10-16T11:25:01.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.984+0000 7f2ac77fe640 1 --2- 172.21.15.84:0/823595598 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f2aa806b100 0x7f2aa806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:01.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.985+0000 7f2ac77fe640 1 --2- 172.21.15.84:0/823595598 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f2aa806b100 0x7f2aa806d5c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8034000 tx=0x7f2ab8007000 comp rx=0 tx=0).ready entity=mgr.11682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:01.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:01.989+0000 7f2ac4ff9640 1 -- 172.21.15.84:0/823595598 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2ab0054980 con 0x7f2ac80c74e0 2023-10-16T11:25:02.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.205+0000 7f2accd21640 1 -- 172.21.15.84:0/823595598 --> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention remove", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ac80acc10 con 0x7f2aa806b100 2023-10-16T11:25:02.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.378+0000 7f2ac4ff9640 1 -- 172.21.15.84:0/823595598 <== mgr.11682 v2:172.21.15.84:6832/2880641466 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f2ab80014b0 con 0x7f2aa806b100 2023-10-16T11:25:02.380 INFO:teuthology.orchestra.run.smithi084.stdout:Retention removed from path mandatory_fs_argument_test_dir 2023-10-16T11:25:02.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.381+0000 7f2accd21640 1 -- 172.21.15.84:0/823595598 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f2aa806b100 msgr2=0x7f2aa806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:02.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.381+0000 7f2accd21640 1 --2- 172.21.15.84:0/823595598 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f2aa806b100 0x7f2aa806d5c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8034000 tx=0x7f2ab8007000 comp rx=0 tx=0).stop 2023-10-16T11:25:02.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.381+0000 7f2accd21640 1 -- 172.21.15.84:0/823595598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac80c74e0 msgr2=0x7f2ac80c7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:02.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.381+0000 7f2accd21640 1 --2- 172.21.15.84:0/823595598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac80c74e0 0x7f2ac80c7960 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000e770 tx=0x7f2ab000ec30 comp rx=0 tx=0).stop 2023-10-16T11:25:02.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.381+0000 7f2accd21640 1 -- 172.21.15.84:0/823595598 shutdown_connections 2023-10-16T11:25:02.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.381+0000 7f2accd21640 1 --2- 172.21.15.84:0/823595598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ac8147ad0 0x7f2ac8147f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:02.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.381+0000 7f2accd21640 1 --2- 172.21.15.84:0/823595598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac80c74e0 0x7f2ac80c7960 unknown :-1 s=CLOSED pgs=2751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:02.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.381+0000 7f2accd21640 1 --2- 172.21.15.84:0/823595598 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f2aa806b100 0x7f2aa806d5c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:02.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.381+0000 7f2accd21640 1 --2- 172.21.15.84:0/823595598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ab40039a0 0x7f2ac80c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:02.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.381+0000 7f2accd21640 1 -- 172.21.15.84:0/823595598 >> 172.21.15.84:0/823595598 conn(0x7f2ac80b0170 msgr2=0x7f2ac80aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:02.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.382+0000 7f2accd21640 1 -- 172.21.15.84:0/823595598 shutdown_connections 2023-10-16T11:25:02.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.382+0000 7f2accd21640 1 -- 172.21.15.84:0/823595598 wait complete. 2023-10-16T11:25:02.404 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention removed from path mandatory_fs_argument_test_dir 2023-10-16T11:25:02.405 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-16T11:25:02.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.603+0000 7fe26f933640 1 Processor -- start 2023-10-16T11:25:02.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.604+0000 7fe26f933640 1 -- start start 2023-10-16T11:25:02.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.605+0000 7fe26f933640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe26815ac10 0x7fe26806c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:02.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.605+0000 7fe26f933640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe2680b0360 0x7fe2680b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:02.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.605+0000 7fe26f933640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe2680ca510 0x7fe2680ca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:02.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.605+0000 7fe26f933640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2680b0d00 con 0x7fe2680ca510 2023-10-16T11:25:02.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.605+0000 7fe26f933640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe26815b310 con 0x7fe2680b0360 2023-10-16T11:25:02.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.605+0000 7fe26f933640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26806f640 con 0x7fe26815ac10 2023-10-16T11:25:02.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.606+0000 7fe26ded0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe2680ca510 0x7fe2680ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:02.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.606+0000 7fe26ded0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe2680ca510 0x7fe2680ca940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56926/0 (socket says 172.21.15.84:56926) 2023-10-16T11:25:02.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.606+0000 7fe26ded0640 1 -- 172.21.15.84:0/1491889368 learned_addr learned my addr 172.21.15.84:0/1491889368 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:02.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.606+0000 7fe26cece640 1 --2- 172.21.15.84:0/1491889368 >> v2:172.21.15.157:3300/0 conn(0x7fe2680b0360 0x7fe2680b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:02.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.606+0000 7fe26d6cf640 1 --2- 172.21.15.84:0/1491889368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe26815ac10 0x7fe26806c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:02.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.606+0000 7fe26ded0640 1 -- 172.21.15.84:0/1491889368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe26815ac10 msgr2=0x7fe26806c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:02.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.606+0000 7fe26ded0640 1 --2- 172.21.15.84:0/1491889368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe26815ac10 0x7fe26806c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:02.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.606+0000 7fe26ded0640 1 -- 172.21.15.84:0/1491889368 >> v2:172.21.15.157:3300/0 conn(0x7fe2680b0360 msgr2=0x7fe2680b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:02.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.606+0000 7fe26ded0640 1 --2- 172.21.15.84:0/1491889368 >> v2:172.21.15.157:3300/0 conn(0x7fe2680b0360 0x7fe2680b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:02.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.606+0000 7fe26ded0640 1 -- 172.21.15.84:0/1491889368 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26815b5b0 con 0x7fe2680ca510 2023-10-16T11:25:02.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.607+0000 7fe26ded0640 1 --2- 172.21.15.84:0/1491889368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2680ca510 0x7fe2680ca940 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7fe26806d5c0 tx=0x7fe26002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef0f818c0a07ac server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:02.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.607+0000 7fe25ffff640 1 -- 172.21.15.84:0/1491889368 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26002c9e0 con 0x7fe2680ca510 2023-10-16T11:25:02.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.608+0000 7fe25ffff640 1 -- 172.21.15.84:0/1491889368 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26002cb40 con 0x7fe2680ca510 2023-10-16T11:25:02.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.608+0000 7fe25ffff640 1 -- 172.21.15.84:0/1491889368 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2600343f0 con 0x7fe2680ca510 2023-10-16T11:25:02.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.608+0000 7fe26f933640 1 -- 172.21.15.84:0/1491889368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2680ca510 msgr2=0x7fe2680ca940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:02.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.608+0000 7fe26f933640 1 --2- 172.21.15.84:0/1491889368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2680ca510 0x7fe2680ca940 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7fe26806d5c0 tx=0x7fe26002be40 comp rx=0 tx=0).stop 2023-10-16T11:25:02.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.608+0000 7fe26f933640 1 -- 172.21.15.84:0/1491889368 shutdown_connections 2023-10-16T11:25:02.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.608+0000 7fe26f933640 1 --2- 172.21.15.84:0/1491889368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2680ca510 0x7fe2680ca940 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:02.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.608+0000 7fe26f933640 1 --2- 172.21.15.84:0/1491889368 >> v2:172.21.15.157:3300/0 conn(0x7fe2680b0360 0x7fe2680b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:02.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.608+0000 7fe26f933640 1 --2- 172.21.15.84:0/1491889368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe26815ac10 0x7fe26806c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:02.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.608+0000 7fe26f933640 1 -- 172.21.15.84:0/1491889368 >> 172.21.15.84:0/1491889368 conn(0x7fe2680cb360 msgr2=0x7fe2680cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:02.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.608+0000 7fe26f933640 1 -- 172.21.15.84:0/1491889368 shutdown_connections 2023-10-16T11:25:02.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.608+0000 7fe26f933640 1 -- 172.21.15.84:0/1491889368 wait complete. 2023-10-16T11:25:02.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.610+0000 7fe26f933640 1 Processor -- start 2023-10-16T11:25:02.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.610+0000 7fe26f933640 1 -- start start 2023-10-16T11:25:02.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.611+0000 7fe26f933640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2680b0360 0x7fe2680c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:02.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.611+0000 7fe26f933640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2680ca510 0x7fe2680c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:02.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.611+0000 7fe26f933640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe26815ac10 0x7fe2680c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:02.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.611+0000 7fe26f933640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2680c5cc0 con 0x7fe2680ca510 2023-10-16T11:25:02.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.611+0000 7fe26f933640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2680c5fa0 con 0x7fe26815ac10 2023-10-16T11:25:02.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.611+0000 7fe26f933640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2680c7970 con 0x7fe2680b0360 2023-10-16T11:25:02.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.611+0000 7fe26cece640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2680ca510 0x7fe2680c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:02.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.611+0000 7fe26cece640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2680ca510 0x7fe2680c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56932/0 (socket says 172.21.15.84:56932) 2023-10-16T11:25:02.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.611+0000 7fe26cece640 1 -- 172.21.15.84:0/162723113 learned_addr learned my addr 172.21.15.84:0/162723113 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:02.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.612+0000 7fe26cece640 1 -- 172.21.15.84:0/162723113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2680b0360 msgr2=0x7fe2680c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:02.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.612+0000 7fe26cece640 1 --2- 172.21.15.84:0/162723113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2680b0360 0x7fe2680c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:02.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.612+0000 7fe26cece640 1 -- 172.21.15.84:0/162723113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe26815ac10 msgr2=0x7fe2680c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:02.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.612+0000 7fe26cece640 1 --2- 172.21.15.84:0/162723113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe26815ac10 0x7fe2680c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:02.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.612+0000 7fe26cece640 1 -- 172.21.15.84:0/162723113 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe260009b30 con 0x7fe2680ca510 2023-10-16T11:25:02.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.612+0000 7fe26cece640 1 --2- 172.21.15.84:0/162723113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2680ca510 0x7fe2680c9030 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7fe25800d800 tx=0x7fe25800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:02.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.612+0000 7fe25dffb640 1 -- 172.21.15.84:0/162723113 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2580096a0 con 0x7fe2680ca510 2023-10-16T11:25:02.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.612+0000 7fe25dffb640 1 -- 172.21.15.84:0/162723113 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe258009800 con 0x7fe2680ca510 2023-10-16T11:25:02.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.612+0000 7fe25dffb640 1 -- 172.21.15.84:0/162723113 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258009990 con 0x7fe2680ca510 2023-10-16T11:25:02.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.612+0000 7fe26f933640 1 -- 172.21.15.84:0/162723113 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2680c7c50 con 0x7fe2680ca510 2023-10-16T11:25:02.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.612+0000 7fe26f933640 1 -- 172.21.15.84:0/162723113 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2680c86b0 con 0x7fe2680ca510 2023-10-16T11:25:02.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.613+0000 7fe26f933640 1 -- 172.21.15.84:0/162723113 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2680c61c0 con 0x7fe2680ca510 2023-10-16T11:25:02.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.614+0000 7fe25dffb640 1 -- 172.21.15.84:0/162723113 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 137) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fe258009af0 con 0x7fe2680ca510 2023-10-16T11:25:02.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.615+0000 7fe25dffb640 1 --2- 172.21.15.84:0/162723113 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fe24c06b100 0x7fe24c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:02.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.615+0000 7fe25dffb640 1 -- 172.21.15.84:0/162723113 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6945+0+0 (secure 0 0 0) 0x7fe25808a3c0 con 0x7fe2680ca510 2023-10-16T11:25:02.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.616+0000 7fe26d6cf640 1 --2- 172.21.15.84:0/162723113 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fe24c06b100 0x7fe24c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:02.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.616+0000 7fe26d6cf640 1 --2- 172.21.15.84:0/162723113 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fe24c06b100 0x7fe24c06d5c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe264006fd0 tx=0x7fe264008040 comp rx=0 tx=0).ready entity=mgr.11682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:02.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.620+0000 7fe25dffb640 1 -- 172.21.15.84:0/162723113 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe258054c60 con 0x7fe2680ca510 2023-10-16T11:25:02.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.831+0000 7fe26f933640 1 -- 172.21.15.84:0/162723113 --> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe26815bb70 con 0x7fe24c06b100 2023-10-16T11:25:02.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:02.881+0000 7fe25dffb640 1 -- 172.21.15.84:0/162723113 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 138) v1 ==== 89507+0+0 (secure 0 0 0) 0x7fe258055030 con 0x7fe2680ca510 2023-10-16T11:25:03.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.020+0000 7fe25dffb640 1 -- 172.21.15.84:0/162723113 <== mgr.11682 v2:172.21.15.84:6832/2880641466 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+61 (secure 0 0 0) 0x7fe26400be20 con 0x7fe24c06b100 2023-10-16T11:25:03.023 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule deactivated for path mandatory_fs_argument_test_dir 2023-10-16T11:25:03.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.023+0000 7fe26f933640 1 -- 172.21.15.84:0/162723113 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fe24c06b100 msgr2=0x7fe24c06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:03.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.023+0000 7fe26f933640 1 --2- 172.21.15.84:0/162723113 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fe24c06b100 0x7fe24c06d5c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe264006fd0 tx=0x7fe264008040 comp rx=0 tx=0).stop 2023-10-16T11:25:03.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.023+0000 7fe26f933640 1 -- 172.21.15.84:0/162723113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2680ca510 msgr2=0x7fe2680c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:03.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.023+0000 7fe26f933640 1 --2- 172.21.15.84:0/162723113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2680ca510 0x7fe2680c9030 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7fe25800d800 tx=0x7fe25800dcc0 comp rx=0 tx=0).stop 2023-10-16T11:25:03.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.024+0000 7fe26f933640 1 -- 172.21.15.84:0/162723113 shutdown_connections 2023-10-16T11:25:03.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.024+0000 7fe26f933640 1 --2- 172.21.15.84:0/162723113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe26815ac10 0x7fe2680c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:03.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.024+0000 7fe26f933640 1 --2- 172.21.15.84:0/162723113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2680ca510 0x7fe2680c9030 unknown :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:03.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.024+0000 7fe26f933640 1 --2- 172.21.15.84:0/162723113 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fe24c06b100 0x7fe24c06d5c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:03.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.024+0000 7fe26f933640 1 --2- 172.21.15.84:0/162723113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2680b0360 0x7fe2680c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:03.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.024+0000 7fe26f933640 1 -- 172.21.15.84:0/162723113 >> 172.21.15.84:0/162723113 conn(0x7fe2680cb360 msgr2=0x7fe26806b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:03.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.024+0000 7fe26f933640 1 -- 172.21.15.84:0/162723113 shutdown_connections 2023-10-16T11:25:03.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.024+0000 7fe26f933640 1 -- 172.21.15.84:0/162723113 wait complete. 2023-10-16T11:25:03.046 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path mandatory_fs_argument_test_dir 2023-10-16T11:25:03.047 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-16T11:25:03.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.266+0000 7f43d0fc5640 1 Processor -- start 2023-10-16T11:25:03.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.267+0000 7f43d0fc5640 1 -- start start 2023-10-16T11:25:03.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.268+0000 7f43d0fc5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f43cc1599c0 0x7f43cc0c9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:03.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.268+0000 7f43d0fc5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f43cc159da0 0x7f43cc15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:03.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.268+0000 7f43d0fc5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43cc15dd10 con 0x7f43cc0cce20 2023-10-16T11:25:03.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.268+0000 7f43d0fc5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43cc15dfb0 con 0x7f43cc1599c0 2023-10-16T11:25:03.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.268+0000 7f43d0fc5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43cc15e290 con 0x7f43cc159da0 2023-10-16T11:25:03.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.268+0000 7f43ca59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f43cc1599c0 0x7f43cc0c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:03.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.268+0000 7f43ca59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f43cc1599c0 0x7f43cc0c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56938/0 (socket says 172.21.15.84:56938) 2023-10-16T11:25:03.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.268+0000 7f43ca59c640 1 -- 172.21.15.84:0/4051036676 learned_addr learned my addr 172.21.15.84:0/4051036676 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:03.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.269+0000 7f43c9d9b640 1 --2- 172.21.15.84:0/4051036676 >> v2:172.21.15.157:3300/0 conn(0x7f43cc159da0 0x7f43cc15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:03.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.269+0000 7f43ca59c640 1 -- 172.21.15.84:0/4051036676 >> v2:172.21.15.157:3300/0 conn(0x7f43cc159da0 msgr2=0x7f43cc15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:03.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.269+0000 7f43ca59c640 1 --2- 172.21.15.84:0/4051036676 >> v2:172.21.15.157:3300/0 conn(0x7f43cc159da0 0x7f43cc15d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:03.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.269+0000 7f43ca59c640 1 -- 172.21.15.84:0/4051036676 >> v1:172.21.15.84:6789/0 conn(0x7f43cc0cce20 legacy=0x7f43cc06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:03.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.269+0000 7f43ca59c640 1 -- 172.21.15.84:0/4051036676 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc15e570 con 0x7f43cc1599c0 2023-10-16T11:25:03.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.270+0000 7f43ca59c640 1 --2- 172.21.15.84:0/4051036676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43cc1599c0 0x7f43cc0c9300 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f43cc06cf30 tx=0x7f43bc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26615b2f69ce54e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:03.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.270+0000 7f43c959a640 1 -- 172.21.15.84:0/4051036676 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc030860 con 0x7f43cc1599c0 2023-10-16T11:25:03.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.271+0000 7f43c959a640 1 -- 172.21.15.84:0/4051036676 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43bc0309c0 con 0x7f43cc1599c0 2023-10-16T11:25:03.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.271+0000 7f43c959a640 1 -- 172.21.15.84:0/4051036676 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc030d50 con 0x7f43cc1599c0 2023-10-16T11:25:03.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.271+0000 7f43d0fc5640 1 -- 172.21.15.84:0/4051036676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43cc1599c0 msgr2=0x7f43cc0c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:03.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.271+0000 7f43d0fc5640 1 --2- 172.21.15.84:0/4051036676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43cc1599c0 0x7f43cc0c9300 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f43cc06cf30 tx=0x7f43bc02b9a0 comp rx=0 tx=0).stop 2023-10-16T11:25:03.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.271+0000 7f43d0fc5640 1 -- 172.21.15.84:0/4051036676 shutdown_connections 2023-10-16T11:25:03.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.271+0000 7f43d0fc5640 1 --2- 172.21.15.84:0/4051036676 >> v2:172.21.15.157:3300/0 conn(0x7f43cc159da0 0x7f43cc15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:03.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.271+0000 7f43d0fc5640 1 --2- 172.21.15.84:0/4051036676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43cc1599c0 0x7f43cc0c9300 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:03.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.271+0000 7f43d0fc5640 1 -- 172.21.15.84:0/4051036676 >> 172.21.15.84:0/4051036676 conn(0x7f43cc0cb360 msgr2=0x7f43cc0cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:03.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.271+0000 7f43d0fc5640 1 -- 172.21.15.84:0/4051036676 shutdown_connections 2023-10-16T11:25:03.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.271+0000 7f43d0fc5640 1 -- 172.21.15.84:0/4051036676 wait complete. 2023-10-16T11:25:03.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.273+0000 7f43d0fc5640 1 Processor -- start 2023-10-16T11:25:03.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.273+0000 7f43d0fc5640 1 -- start start 2023-10-16T11:25:03.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.274+0000 7f43d0fc5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43cc0cce20 0x7f43cc0b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:03.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.274+0000 7f43d0fc5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43cc1599c0 0x7f43cc0ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:03.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.274+0000 7f43d0fc5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43cc159da0 0x7f43cc0aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:03.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.274+0000 7f43d0fc5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0c4db0 con 0x7f43cc159da0 2023-10-16T11:25:03.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.274+0000 7f43d0fc5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc14ebf0 con 0x7f43cc1599c0 2023-10-16T11:25:03.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.274+0000 7f43d0fc5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0c3160 con 0x7f43cc0cce20 2023-10-16T11:25:03.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.274+0000 7f43cad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43cc159da0 0x7f43cc0aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:03.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.274+0000 7f43cad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43cc159da0 0x7f43cc0aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56940/0 (socket says 172.21.15.84:56940) 2023-10-16T11:25:03.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.274+0000 7f43cad9d640 1 -- 172.21.15.84:0/3277189893 learned_addr learned my addr 172.21.15.84:0/3277189893 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:03.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.275+0000 7f43c9d9b640 1 --2- 172.21.15.84:0/3277189893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43cc1599c0 0x7f43cc0ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:03.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.275+0000 7f43cad9d640 1 -- 172.21.15.84:0/3277189893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43cc0cce20 msgr2=0x7f43cc0b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:03.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.275+0000 7f43cad9d640 1 --2- 172.21.15.84:0/3277189893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43cc0cce20 0x7f43cc0b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:03.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.275+0000 7f43cad9d640 1 -- 172.21.15.84:0/3277189893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43cc1599c0 msgr2=0x7f43cc0ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:03.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.275+0000 7f43cad9d640 1 --2- 172.21.15.84:0/3277189893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43cc1599c0 0x7f43cc0ae550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:03.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.275+0000 7f43cad9d640 1 -- 172.21.15.84:0/3277189893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43bc0094d0 con 0x7f43cc159da0 2023-10-16T11:25:03.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.275+0000 7f43c9d9b640 1 --2- 172.21.15.84:0/3277189893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43cc1599c0 0x7f43cc0ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:03.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.275+0000 7f43cad9d640 1 --2- 172.21.15.84:0/3277189893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43cc159da0 0x7f43cc0aea90 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f43c400c6b0 tx=0x7f43c400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:03.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.275+0000 7f43baffd640 1 -- 172.21.15.84:0/3277189893 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4003b70 con 0x7f43cc159da0 2023-10-16T11:25:03.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.275+0000 7f43baffd640 1 -- 172.21.15.84:0/3277189893 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c4003cf0 con 0x7f43cc159da0 2023-10-16T11:25:03.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.276+0000 7f43d0fc5640 1 -- 172.21.15.84:0/3277189893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43cc0c3440 con 0x7f43cc159da0 2023-10-16T11:25:03.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.276+0000 7f43baffd640 1 -- 172.21.15.84:0/3277189893 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4003ea0 con 0x7f43cc159da0 2023-10-16T11:25:03.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.276+0000 7f43d0fc5640 1 -- 172.21.15.84:0/3277189893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43cc0b1160 con 0x7f43cc159da0 2023-10-16T11:25:03.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.277+0000 7f43d0fc5640 1 -- 172.21.15.84:0/3277189893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43cc0af5a0 con 0x7f43cc159da0 2023-10-16T11:25:03.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.278+0000 7f43baffd640 1 -- 172.21.15.84:0/3277189893 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 138) v1 ==== 89507+0+0 (secure 0 0 0) 0x7f43c4005310 con 0x7f43cc159da0 2023-10-16T11:25:03.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.279+0000 7f43baffd640 1 --2- 172.21.15.84:0/3277189893 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f43ac06b1c0 0x7f43ac06d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:03.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.279+0000 7f43baffd640 1 -- 172.21.15.84:0/3277189893 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6945+0+0 (secure 0 0 0) 0x7f43c4089d50 con 0x7f43cc159da0 2023-10-16T11:25:03.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.279+0000 7f43ca59c640 1 --2- 172.21.15.84:0/3277189893 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f43ac06b1c0 0x7f43ac06d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:03.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.280+0000 7f43ca59c640 1 --2- 172.21.15.84:0/3277189893 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f43ac06b1c0 0x7f43ac06d680 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f43bc036000 tx=0x7f43bc03b040 comp rx=0 tx=0).ready entity=mgr.11682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:03.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.284+0000 7f43baffd640 1 -- 172.21.15.84:0/3277189893 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f43cc0af5a0 con 0x7f43cc159da0 2023-10-16T11:25:03.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.499+0000 7f43d0fc5640 1 -- 172.21.15.84:0/3277189893 --> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f43cc06fa50 con 0x7f43ac06b1c0 2023-10-16T11:25:03.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.704+0000 7f43baffd640 1 -- 172.21.15.84:0/3277189893 <== mgr.11682 v2:172.21.15.84:6832/2880641466 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f43bc030cb0 con 0x7f43ac06b1c0 2023-10-16T11:25:03.706 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path mandatory_fs_argument_test_dir 2023-10-16T11:25:03.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.707+0000 7f43d0fc5640 1 -- 172.21.15.84:0/3277189893 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f43ac06b1c0 msgr2=0x7f43ac06d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:03.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.707+0000 7f43d0fc5640 1 --2- 172.21.15.84:0/3277189893 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f43ac06b1c0 0x7f43ac06d680 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f43bc036000 tx=0x7f43bc03b040 comp rx=0 tx=0).stop 2023-10-16T11:25:03.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.707+0000 7f43d0fc5640 1 -- 172.21.15.84:0/3277189893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43cc159da0 msgr2=0x7f43cc0aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:03.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.707+0000 7f43d0fc5640 1 --2- 172.21.15.84:0/3277189893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43cc159da0 0x7f43cc0aea90 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f43c400c6b0 tx=0x7f43c400cb70 comp rx=0 tx=0).stop 2023-10-16T11:25:03.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.707+0000 7f43d0fc5640 1 -- 172.21.15.84:0/3277189893 shutdown_connections 2023-10-16T11:25:03.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.707+0000 7f43d0fc5640 1 --2- 172.21.15.84:0/3277189893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43cc159da0 0x7f43cc0aea90 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:03.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.707+0000 7f43d0fc5640 1 --2- 172.21.15.84:0/3277189893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43cc1599c0 0x7f43cc0ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:03.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.707+0000 7f43d0fc5640 1 --2- 172.21.15.84:0/3277189893 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f43ac06b1c0 0x7f43ac06d680 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:03.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.707+0000 7f43d0fc5640 1 --2- 172.21.15.84:0/3277189893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43cc0cce20 0x7f43cc0b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:03.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.707+0000 7f43d0fc5640 1 -- 172.21.15.84:0/3277189893 >> 172.21.15.84:0/3277189893 conn(0x7f43cc0cb360 msgr2=0x7f43cc0adae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:03.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.708+0000 7f43d0fc5640 1 -- 172.21.15.84:0/3277189893 shutdown_connections 2023-10-16T11:25:03.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.708+0000 7f43d0fc5640 1 -- 172.21.15.84:0/3277189893 wait complete. 2023-10-16T11:25:03.732 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path mandatory_fs_argument_test_dir 2023-10-16T11:25:03.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:25:03.733 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir mandatory_fs_argument_test_dir) 2023-10-16T11:25:03.787 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T11:25:03.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.997+0000 7f424947c640 1 Processor -- start 2023-10-16T11:25:04.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.997+0000 7f424947c640 1 -- start start 2023-10-16T11:25:04.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.999+0000 7f424947c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f424415ad70 0x7f4244069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:04.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.999+0000 7f424947c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42440caac0 0x7f424406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:04.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.999+0000 7f424947c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42440cb030 con 0x7f42440c9ca0 2023-10-16T11:25:04.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.999+0000 7f424947c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42440c91c0 con 0x7f424415ad70 2023-10-16T11:25:04.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:03.999+0000 7f424947c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42440c94a0 con 0x7f42440caac0 2023-10-16T11:25:04.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.000+0000 7f4242ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f424415ad70 0x7f4244069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:04.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.000+0000 7f4242ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f424415ad70 0x7f4244069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56944/0 (socket says 172.21.15.84:56944) 2023-10-16T11:25:04.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.000+0000 7f4242ffd640 1 -- 172.21.15.84:0/3168343682 learned_addr learned my addr 172.21.15.84:0/3168343682 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:04.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.000+0000 7f42437fe640 1 --2- 172.21.15.84:0/3168343682 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42440caac0 0x7f424406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:04.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.000+0000 7f4242ffd640 1 -- 172.21.15.84:0/3168343682 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42440caac0 msgr2=0x7f424406a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:04.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.000+0000 7f4242ffd640 1 --2- 172.21.15.84:0/3168343682 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42440caac0 0x7f424406a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:04.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.000+0000 7f4242ffd640 1 -- 172.21.15.84:0/3168343682 >> v1:172.21.15.84:6789/0 conn(0x7f42440c9ca0 legacy=0x7f42440ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:04.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.000+0000 7f4242ffd640 1 -- 172.21.15.84:0/3168343682 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42440c9780 con 0x7f424415ad70 2023-10-16T11:25:04.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.001+0000 7f4242ffd640 1 --2- 172.21.15.84:0/3168343682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f424415ad70 0x7f4244069da0 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f423c009e50 tx=0x7f423c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8e9bda98fa8054 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:04.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.002+0000 7f4241ffb640 1 -- 172.21.15.84:0/3168343682 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c02cb00 con 0x7f424415ad70 2023-10-16T11:25:04.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.002+0000 7f4241ffb640 1 -- 172.21.15.84:0/3168343682 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423c02cc60 con 0x7f424415ad70 2023-10-16T11:25:04.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.002+0000 7f4241ffb640 1 -- 172.21.15.84:0/3168343682 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c0343f0 con 0x7f424415ad70 2023-10-16T11:25:04.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.002+0000 7f424947c640 1 -- 172.21.15.84:0/3168343682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f424415ad70 msgr2=0x7f4244069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:04.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.002+0000 7f424947c640 1 --2- 172.21.15.84:0/3168343682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f424415ad70 0x7f4244069da0 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f423c009e50 tx=0x7f423c02bf60 comp rx=0 tx=0).stop 2023-10-16T11:25:04.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.002+0000 7f424947c640 1 -- 172.21.15.84:0/3168343682 shutdown_connections 2023-10-16T11:25:04.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.002+0000 7f424947c640 1 --2- 172.21.15.84:0/3168343682 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42440caac0 0x7f424406a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:04.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.002+0000 7f424947c640 1 --2- 172.21.15.84:0/3168343682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f424415ad70 0x7f4244069da0 unknown :-1 s=CLOSED pgs=2758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:04.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.002+0000 7f424947c640 1 -- 172.21.15.84:0/3168343682 >> 172.21.15.84:0/3168343682 conn(0x7f42440ce9b0 msgr2=0x7f42440cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:04.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.002+0000 7f424947c640 1 -- 172.21.15.84:0/3168343682 shutdown_connections 2023-10-16T11:25:04.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.002+0000 7f424947c640 1 -- 172.21.15.84:0/3168343682 wait complete. 2023-10-16T11:25:04.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.004+0000 7f424947c640 1 Processor -- start 2023-10-16T11:25:04.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.004+0000 7f424947c640 1 -- start start 2023-10-16T11:25:04.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.005+0000 7f424947c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42440c9ca0 0x7f4244158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:04.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.005+0000 7f424947c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42440caac0 0x7f42440ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:04.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.005+0000 7f424947c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f424415ad70 0x7f42440acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:04.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.005+0000 7f424947c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424415da30 con 0x7f42440c9ca0 2023-10-16T11:25:04.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.005+0000 7f424947c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42441f2220 con 0x7f424415ad70 2023-10-16T11:25:04.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.005+0000 7f424947c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4244158db0 con 0x7f42440caac0 2023-10-16T11:25:04.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.006+0000 7f4242ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42440c9ca0 0x7f4244158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:04.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.006+0000 7f4242ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42440c9ca0 0x7f4244158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56952/0 (socket says 172.21.15.84:56952) 2023-10-16T11:25:04.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.006+0000 7f4242ffd640 1 -- 172.21.15.84:0/1325781939 learned_addr learned my addr 172.21.15.84:0/1325781939 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:04.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.006+0000 7f4242ffd640 1 -- 172.21.15.84:0/1325781939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42440caac0 msgr2=0x7f42440ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:04.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.006+0000 7f4242ffd640 1 --2- 172.21.15.84:0/1325781939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42440caac0 0x7f42440ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:04.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.006+0000 7f4242ffd640 1 -- 172.21.15.84:0/1325781939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f424415ad70 msgr2=0x7f42440acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:04.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.006+0000 7f4242ffd640 1 --2- 172.21.15.84:0/1325781939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f424415ad70 0x7f42440acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:04.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.006+0000 7f4242ffd640 1 -- 172.21.15.84:0/1325781939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c009b30 con 0x7f42440c9ca0 2023-10-16T11:25:04.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.006+0000 7f4242ffd640 1 --2- 172.21.15.84:0/1325781939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42440c9ca0 0x7f4244158870 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f423c036040 tx=0x7f423c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.007+0000 7f423b7fe640 1 -- 172.21.15.84:0/1325781939 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c009510 con 0x7f42440c9ca0 2023-10-16T11:25:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.007+0000 7f423b7fe640 1 -- 172.21.15.84:0/1325781939 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423c03e450 con 0x7f42440c9ca0 2023-10-16T11:25:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.007+0000 7f423b7fe640 1 -- 172.21.15.84:0/1325781939 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c03e800 con 0x7f42440c9ca0 2023-10-16T11:25:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.007+0000 7f424947c640 1 -- 172.21.15.84:0/1325781939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42441f2490 con 0x7f42440c9ca0 2023-10-16T11:25:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.007+0000 7f424947c640 1 -- 172.21.15.84:0/1325781939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42441f2a50 con 0x7f42440c9ca0 2023-10-16T11:25:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.008+0000 7f424947c640 1 -- 172.21.15.84:0/1325781939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f424404e160 con 0x7f42440c9ca0 2023-10-16T11:25:04.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.009+0000 7f423b7fe640 1 -- 172.21.15.84:0/1325781939 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 138) v1 ==== 89507+0+0 (secure 0 0 0) 0x7f423c03e960 con 0x7f42440c9ca0 2023-10-16T11:25:04.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.010+0000 7f423b7fe640 1 --2- 172.21.15.84:0/1325781939 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f422806b190 0x7f422806d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:04.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.010+0000 7f423b7fe640 1 -- 172.21.15.84:0/1325781939 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6945+0+0 (secure 0 0 0) 0x7f423c0ace60 con 0x7f42440c9ca0 2023-10-16T11:25:04.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.010+0000 7f42427fc640 1 --2- 172.21.15.84:0/1325781939 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f422806b190 0x7f422806d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:04.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.011+0000 7f42427fc640 1 --2- 172.21.15.84:0/1325781939 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f422806b190 0x7f422806d650 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f422c000f60 tx=0x7f422c0079b0 comp rx=0 tx=0).ready entity=mgr.11682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:04.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.015+0000 7f423b7fe640 1 -- 172.21.15.84:0/1325781939 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f423c0775c0 con 0x7f42440c9ca0 2023-10-16T11:25:04.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.258+0000 7f424947c640 1 -- 172.21.15.84:0/1325781939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f42440ca110 con 0x7f42440c9ca0 2023-10-16T11:25:04.319 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:25:04.328 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:25:04.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.879+0000 7f423b7fe640 1 -- 172.21.15.84:0/1325781939 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v139) v1 ==== 93+0+0 (secure 0 0 0) 0x7f423c07abe0 con 0x7f42440c9ca0 2023-10-16T11:25:04.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.883+0000 7f424947c640 1 -- 172.21.15.84:0/1325781939 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f422806b190 msgr2=0x7f422806d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:04.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.883+0000 7f424947c640 1 --2- 172.21.15.84:0/1325781939 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f422806b190 0x7f422806d650 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f422c000f60 tx=0x7f422c0079b0 comp rx=0 tx=0).stop 2023-10-16T11:25:04.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.883+0000 7f424947c640 1 -- 172.21.15.84:0/1325781939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42440c9ca0 msgr2=0x7f4244158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:04.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.883+0000 7f424947c640 1 --2- 172.21.15.84:0/1325781939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42440c9ca0 0x7f4244158870 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f423c036040 tx=0x7f423c004a40 comp rx=0 tx=0).stop 2023-10-16T11:25:04.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.884+0000 7f424947c640 1 -- 172.21.15.84:0/1325781939 shutdown_connections 2023-10-16T11:25:04.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.884+0000 7f424947c640 1 --2- 172.21.15.84:0/1325781939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f424415ad70 0x7f42440acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:04.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.884+0000 7f424947c640 1 --2- 172.21.15.84:0/1325781939 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f422806b190 0x7f422806d650 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:04.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.884+0000 7f424947c640 1 --2- 172.21.15.84:0/1325781939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42440caac0 0x7f42440ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:04.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.884+0000 7f424947c640 1 --2- 172.21.15.84:0/1325781939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42440c9ca0 0x7f4244158870 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:04.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.884+0000 7f424947c640 1 -- 172.21.15.84:0/1325781939 >> 172.21.15.84:0/1325781939 conn(0x7f42440ce9b0 msgr2=0x7f42440afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:04.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.885+0000 7f424947c640 1 -- 172.21.15.84:0/1325781939 shutdown_connections 2023-10-16T11:25:04.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:04.886+0000 7f424947c640 1 -- 172.21.15.84:0/1325781939 wait complete. 2023-10-16T11:25:04.910 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:25:04.910 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:25:05.012 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:25:05.115 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:25:05.115 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:25:05.116 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:25:05.165 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:25:05.166 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:25:05.258 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:25:05.434 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:05.432+0000 7f8c25a58040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:25:05.475 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:05.473+0000 7f014f02c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:25:05.490 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:05.488+0000 7f8c25a58040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:25:05.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.498+0000 7fab7359e640 1 Processor -- start 2023-10-16T11:25:05.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.499+0000 7fab7359e640 1 -- start start 2023-10-16T11:25:05.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.500+0000 7fab7359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab7415aa60 0x7fab740af820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:05.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.500+0000 7fab7359e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fab740afd60 0x7fab740cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:05.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.500+0000 7fab7359e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab740b3730 con 0x7fab740cc820 2023-10-16T11:25:05.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.500+0000 7fab7359e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab740b3980 con 0x7fab740afd60 2023-10-16T11:25:05.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.500+0000 7fab7359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab740b3c60 con 0x7fab7415aa60 2023-10-16T11:25:05.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.501+0000 7fab6bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fab740afd60 0x7fab740cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:05.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.501+0000 7fab72d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fab740cc820 0x7fab740b2360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:34292/0 (socket says 172.21.15.84:34292) 2023-10-16T11:25:05.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.501+0000 7fab72d9d640 1 -- 172.21.15.84:0/1621810722 learned_addr learned my addr 172.21.15.84:0/1621810722 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:05.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.501+0000 7fab7259c640 1 --2- 172.21.15.84:0/1621810722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab7415aa60 0x7fab740af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:05.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.501+0000 7fab71d9b640 1 -- 172.21.15.84:0/1621810722 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550536530 0 0) 0x7fab740b3730 con 0x7fab740cc820 2023-10-16T11:25:05.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.501+0000 7fab71d9b640 1 -- 172.21.15.84:0/1621810722 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab60003520 con 0x7fab740cc820 2023-10-16T11:25:05.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.501+0000 7fab6bfff640 1 -- 172.21.15.84:0/1621810722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab7415aa60 msgr2=0x7fab740af820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:05.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.501+0000 7fab6bfff640 1 --2- 172.21.15.84:0/1621810722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab7415aa60 0x7fab740af820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:05.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.501+0000 7fab6bfff640 1 -- 172.21.15.84:0/1621810722 >> v1:172.21.15.157:6789/0 conn(0x7fab740cc820 legacy=0x7fab740b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:05.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.501+0000 7fab6bfff640 1 -- 172.21.15.84:0/1621810722 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab740b3f40 con 0x7fab740afd60 2023-10-16T11:25:05.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.503+0000 7fab6bfff640 1 --2- 172.21.15.84:0/1621810722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab740afd60 0x7fab740cc2e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0097a0 tx=0x7fab5c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fb81498fdc4075 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:05.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.503+0000 7fab71d9b640 1 -- 172.21.15.84:0/1621810722 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c031830 con 0x7fab740afd60 2023-10-16T11:25:05.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.503+0000 7fab71d9b640 1 -- 172.21.15.84:0/1621810722 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c031990 con 0x7fab740afd60 2023-10-16T11:25:05.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.503+0000 7fab71d9b640 1 -- 172.21.15.84:0/1621810722 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c031d00 con 0x7fab740afd60 2023-10-16T11:25:05.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.504+0000 7fab7359e640 1 -- 172.21.15.84:0/1621810722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab740afd60 msgr2=0x7fab740cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:05.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.504+0000 7fab7359e640 1 --2- 172.21.15.84:0/1621810722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab740afd60 0x7fab740cc2e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0097a0 tx=0x7fab5c02b8d0 comp rx=0 tx=0).stop 2023-10-16T11:25:05.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.504+0000 7fab7359e640 1 -- 172.21.15.84:0/1621810722 shutdown_connections 2023-10-16T11:25:05.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.504+0000 7fab7359e640 1 --2- 172.21.15.84:0/1621810722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab740afd60 0x7fab740cc2e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:05.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.504+0000 7fab7359e640 1 --2- 172.21.15.84:0/1621810722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab7415aa60 0x7fab740af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:05.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.504+0000 7fab7359e640 1 -- 172.21.15.84:0/1621810722 >> 172.21.15.84:0/1621810722 conn(0x7fab74065ed0 msgr2=0x7fab740662e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:05.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.504+0000 7fab7359e640 1 -- 172.21.15.84:0/1621810722 shutdown_connections 2023-10-16T11:25:05.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.504+0000 7fab7359e640 1 -- 172.21.15.84:0/1621810722 wait complete. 2023-10-16T11:25:05.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.506+0000 7fab7359e640 1 Processor -- start 2023-10-16T11:25:05.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.506+0000 7fab7359e640 1 -- start start 2023-10-16T11:25:05.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.507+0000 7fab7359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab740afd60 0x7fab74158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:05.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.507+0000 7fab7359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab740cc820 0x7fab740b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:05.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.507+0000 7fab7359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab7415aa60 0x7fab740b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:05.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.507+0000 7fab7359e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7415b530 con 0x7fab740afd60 2023-10-16T11:25:05.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.507+0000 7fab7359e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab74159110 con 0x7fab740cc820 2023-10-16T11:25:05.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.507+0000 7fab7359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab740b1210 con 0x7fab7415aa60 2023-10-16T11:25:05.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.507+0000 7fab7259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab740afd60 0x7fab74158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:05.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.507+0000 7fab7259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab740afd60 0x7fab74158bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56976/0 (socket says 172.21.15.84:56976) 2023-10-16T11:25:05.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.507+0000 7fab7259c640 1 -- 172.21.15.84:0/3918107373 learned_addr learned my addr 172.21.15.84:0/3918107373 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:05.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.508+0000 7fab6bfff640 1 --2- 172.21.15.84:0/3918107373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab740cc820 0x7fab740b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:05.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.508+0000 7fab72d9d640 1 --2- 172.21.15.84:0/3918107373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab7415aa60 0x7fab740b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:05.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.508+0000 7fab7259c640 1 -- 172.21.15.84:0/3918107373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab7415aa60 msgr2=0x7fab740b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:05.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.508+0000 7fab7259c640 1 --2- 172.21.15.84:0/3918107373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab7415aa60 0x7fab740b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:05.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.508+0000 7fab7259c640 1 -- 172.21.15.84:0/3918107373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab740cc820 msgr2=0x7fab740b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:05.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.508+0000 7fab7259c640 1 --2- 172.21.15.84:0/3918107373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab740cc820 0x7fab740b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:05.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.508+0000 7fab7259c640 1 -- 172.21.15.84:0/3918107373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c009400 con 0x7fab740afd60 2023-10-16T11:25:05.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.508+0000 7fab72d9d640 1 --2- 172.21.15.84:0/3918107373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab7415aa60 0x7fab740b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:05.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.508+0000 7fab7259c640 1 --2- 172.21.15.84:0/3918107373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab740afd60 0x7fab74158bd0 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7fab6400e9e0 tx=0x7fab6400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:05.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.508+0000 7fab6affd640 1 -- 172.21.15.84:0/3918107373 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab64009670 con 0x7fab740afd60 2023-10-16T11:25:05.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.508+0000 7fab6affd640 1 -- 172.21.15.84:0/3918107373 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6400c9d0 con 0x7fab740afd60 2023-10-16T11:25:05.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.508+0000 7fab7359e640 1 -- 172.21.15.84:0/3918107373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7415d430 con 0x7fab740afd60 2023-10-16T11:25:05.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.508+0000 7fab6affd640 1 -- 172.21.15.84:0/3918107373 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6400cd80 con 0x7fab740afd60 2023-10-16T11:25:05.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.509+0000 7fab7359e640 1 -- 172.21.15.84:0/3918107373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7415aee0 con 0x7fab740afd60 2023-10-16T11:25:05.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.510+0000 7fab7359e640 1 -- 172.21.15.84:0/3918107373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab7404e9b0 con 0x7fab740afd60 2023-10-16T11:25:05.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.511+0000 7fab6affd640 1 -- 172.21.15.84:0/3918107373 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 139) v1 ==== 89490+0+0 (secure 0 0 0) 0x7fab64013470 con 0x7fab740afd60 2023-10-16T11:25:05.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.511+0000 7fab6affd640 1 --2- 172.21.15.84:0/3918107373 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fab5406b140 0x7fab5406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:05.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.512+0000 7fab6affd640 1 -- 172.21.15.84:0/3918107373 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6945+0+0 (secure 0 0 0) 0x7fab6408a0f0 con 0x7fab740afd60 2023-10-16T11:25:05.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.512+0000 7fab6bfff640 1 -- 172.21.15.84:0/3918107373 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fab5406b140 msgr2=0x7fab5406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2880641466 2023-10-16T11:25:05.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.512+0000 7fab6bfff640 1 --2- 172.21.15.84:0/3918107373 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fab5406b140 0x7fab5406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:25:05.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.516+0000 7fab6affd640 1 -- 172.21.15.84:0/3918107373 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fab64054a00 con 0x7fab740afd60 2023-10-16T11:25:05.532 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:05.530+0000 7f014f02c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:25:05.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.710+0000 7fab7359e640 1 -- 172.21.15.84:0/3918107373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab740d0ab0 con 0x7fab740afd60 2023-10-16T11:25:05.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.711+0000 7fab6affd640 1 -- 172.21.15.84:0/3918107373 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v222) v1 ==== 74+0+20827 (secure 0 0 0) 0x7fab64058020 con 0x7fab740afd60 2023-10-16T11:25:05.714 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:05.714 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":222,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:24:58.196649+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:24:23.363032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:24:24.551270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:24:38.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:24:39.387564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2877946366":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6833/1425409017":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6832/1425409017":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/1771162859":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/3655599055":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/1076506559":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/2748224910":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/857172551":"2023-10-17T11:24:58.196616+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:25:05.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.713+0000 7fab6bfff640 1 -- 172.21.15.84:0/3918107373 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fab5406b140 msgr2=0x7fab5406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2880641466 2023-10-16T11:25:05.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.713+0000 7fab6bfff640 1 --2- 172.21.15.84:0/3918107373 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fab5406b140 0x7fab5406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:25:05.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.714+0000 7fab7359e640 1 -- 172.21.15.84:0/3918107373 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fab5406b140 msgr2=0x7fab5406d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:25:05.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.714+0000 7fab7359e640 1 --2- 172.21.15.84:0/3918107373 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fab5406b140 0x7fab5406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:05.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.714+0000 7fab7359e640 1 -- 172.21.15.84:0/3918107373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab740afd60 msgr2=0x7fab74158bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:05.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.714+0000 7fab7359e640 1 --2- 172.21.15.84:0/3918107373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab740afd60 0x7fab74158bd0 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7fab6400e9e0 tx=0x7fab6400eea0 comp rx=0 tx=0).stop 2023-10-16T11:25:05.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.714+0000 7fab7359e640 1 -- 172.21.15.84:0/3918107373 shutdown_connections 2023-10-16T11:25:05.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.714+0000 7fab7359e640 1 --2- 172.21.15.84:0/3918107373 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fab5406b140 0x7fab5406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:05.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.714+0000 7fab7359e640 1 --2- 172.21.15.84:0/3918107373 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab7415aa60 0x7fab740b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:05.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.714+0000 7fab7359e640 1 --2- 172.21.15.84:0/3918107373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab740cc820 0x7fab740b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:05.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.714+0000 7fab7359e640 1 --2- 172.21.15.84:0/3918107373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab740afd60 0x7fab74158bd0 unknown :-1 s=CLOSED pgs=2763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:05.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.714+0000 7fab7359e640 1 -- 172.21.15.84:0/3918107373 >> 172.21.15.84:0/3918107373 conn(0x7fab74065ed0 msgr2=0x7fab740c8070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:05.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.714+0000 7fab7359e640 1 -- 172.21.15.84:0/3918107373 shutdown_connections 2023-10-16T11:25:05.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.714+0000 7fab7359e640 1 -- 172.21.15.84:0/3918107373 wait complete. 2023-10-16T11:25:05.734 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:24:23.363032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '216', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:24:24.551270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '217', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-16T11:24:38.531956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '220', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-16T11:24:39.387564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '221', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:25:05.735 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:25:05.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.938+0000 7fec4e40e640 1 Processor -- start 2023-10-16T11:25:05.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.938+0000 7fec4e40e640 1 -- start start 2023-10-16T11:25:05.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.939+0000 7fec4e40e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec48159e40 0x7fec480cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:05.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.939+0000 7fec4e40e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec480ca780 0x7fec4806aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:05.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.940+0000 7fec4e40e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec480c3610 con 0x7fec4806b3f0 2023-10-16T11:25:05.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.940+0000 7fec4e40e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec4806b880 con 0x7fec480ca780 2023-10-16T11:25:05.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.940+0000 7fec4e40e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4815a220 con 0x7fec48159e40 2023-10-16T11:25:05.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.940+0000 7fec3ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec480ca780 0x7fec4806aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:05.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.940+0000 7fec3ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec480ca780 0x7fec4806aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39946/0 (socket says 172.21.15.84:39946) 2023-10-16T11:25:05.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.940+0000 7fec3ffff640 1 -- 172.21.15.84:0/1669866711 learned_addr learned my addr 172.21.15.84:0/1669866711 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:05.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.940+0000 7fec47fff640 1 --2- 172.21.15.84:0/1669866711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec48159e40 0x7fec480cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:05.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.940+0000 7fec3ffff640 1 -- 172.21.15.84:0/1669866711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec48159e40 msgr2=0x7fec480cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:05.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.940+0000 7fec3ffff640 1 --2- 172.21.15.84:0/1669866711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec48159e40 0x7fec480cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:05.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.940+0000 7fec3ffff640 1 -- 172.21.15.84:0/1669866711 >> v1:172.21.15.157:6789/0 conn(0x7fec4806b3f0 legacy=0x7fec480705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:05.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.940+0000 7fec3ffff640 1 -- 172.21.15.84:0/1669866711 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec480cafd0 con 0x7fec480ca780 2023-10-16T11:25:05.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.940+0000 7fec47fff640 1 --2- 172.21.15.84:0/1669866711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec48159e40 0x7fec480cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:05.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.941+0000 7fec3ffff640 1 --2- 172.21.15.84:0/1669866711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec480ca780 0x7fec4806aeb0 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7fec3000bb10 tx=0x7fec3002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3706c71e5fdae60f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:05.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.942+0000 7fec477fe640 1 -- 172.21.15.84:0/1669866711 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3002ee70 con 0x7fec480ca780 2023-10-16T11:25:05.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.942+0000 7fec477fe640 1 -- 172.21.15.84:0/1669866711 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec30036440 con 0x7fec480ca780 2023-10-16T11:25:05.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.942+0000 7fec477fe640 1 -- 172.21.15.84:0/1669866711 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300367d0 con 0x7fec480ca780 2023-10-16T11:25:05.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.942+0000 7fec4e40e640 1 -- 172.21.15.84:0/1669866711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec480ca780 msgr2=0x7fec4806aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:05.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.942+0000 7fec4e40e640 1 --2- 172.21.15.84:0/1669866711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec480ca780 0x7fec4806aeb0 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7fec3000bb10 tx=0x7fec3002e0c0 comp rx=0 tx=0).stop 2023-10-16T11:25:05.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.943+0000 7fec4e40e640 1 -- 172.21.15.84:0/1669866711 shutdown_connections 2023-10-16T11:25:05.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.943+0000 7fec4e40e640 1 --2- 172.21.15.84:0/1669866711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec480ca780 0x7fec4806aeb0 unknown :-1 s=CLOSED pgs=2764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:05.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.943+0000 7fec4e40e640 1 --2- 172.21.15.84:0/1669866711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec48159e40 0x7fec480cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:05.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.943+0000 7fec4e40e640 1 -- 172.21.15.84:0/1669866711 >> 172.21.15.84:0/1669866711 conn(0x7fec480cfac0 msgr2=0x7fec480cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:05.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.943+0000 7fec4e40e640 1 -- 172.21.15.84:0/1669866711 shutdown_connections 2023-10-16T11:25:05.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.943+0000 7fec4e40e640 1 -- 172.21.15.84:0/1669866711 wait complete. 2023-10-16T11:25:05.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.944+0000 7fec4e40e640 1 Processor -- start 2023-10-16T11:25:05.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.945+0000 7fec4e40e640 1 -- start start 2023-10-16T11:25:05.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.945+0000 7fec4e40e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec4806b3f0 0x7fec4814af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:05.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.945+0000 7fec4e40e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec480ca780 0x7fec480c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:05.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.945+0000 7fec4e40e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec48159e40 0x7fec480c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:05.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.945+0000 7fec4e40e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4815c010 con 0x7fec4806b3f0 2023-10-16T11:25:05.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.945+0000 7fec4e40e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4814b4c0 con 0x7fec480ca780 2023-10-16T11:25:05.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.945+0000 7fec4e40e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4815bb80 con 0x7fec48159e40 2023-10-16T11:25:05.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.946+0000 7fec47fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec4806b3f0 0x7fec4814af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:05.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.946+0000 7fec47fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec4806b3f0 0x7fec4814af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39960/0 (socket says 172.21.15.84:39960) 2023-10-16T11:25:05.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.946+0000 7fec47fff640 1 -- 172.21.15.84:0/1453212488 learned_addr learned my addr 172.21.15.84:0/1453212488 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:05.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.946+0000 7fec3ffff640 1 --2- 172.21.15.84:0/1453212488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec480ca780 0x7fec480c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:05.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.946+0000 7fec4c9ab640 1 --2- 172.21.15.84:0/1453212488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec48159e40 0x7fec480c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:05.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.946+0000 7fec47fff640 1 -- 172.21.15.84:0/1453212488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec48159e40 msgr2=0x7fec480c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:05.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.946+0000 7fec47fff640 1 --2- 172.21.15.84:0/1453212488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec48159e40 0x7fec480c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:05.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.946+0000 7fec47fff640 1 -- 172.21.15.84:0/1453212488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec480ca780 msgr2=0x7fec480c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:05.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.946+0000 7fec47fff640 1 --2- 172.21.15.84:0/1453212488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec480ca780 0x7fec480c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:05.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.946+0000 7fec47fff640 1 -- 172.21.15.84:0/1453212488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3000b770 con 0x7fec4806b3f0 2023-10-16T11:25:05.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.946+0000 7fec4c9ab640 1 --2- 172.21.15.84:0/1453212488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec48159e40 0x7fec480c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:05.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.946+0000 7fec3ffff640 1 --2- 172.21.15.84:0/1453212488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec480ca780 0x7fec480c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:25:05.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.947+0000 7fec47fff640 1 --2- 172.21.15.84:0/1453212488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec4806b3f0 0x7fec4814af80 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7fec3800c820 tx=0x7fec3800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:05.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.947+0000 7fec457fa640 1 -- 172.21.15.84:0/1453212488 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38007a30 con 0x7fec4806b3f0 2023-10-16T11:25:05.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.947+0000 7fec457fa640 1 -- 172.21.15.84:0/1453212488 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec38007b90 con 0x7fec4806b3f0 2023-10-16T11:25:05.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.947+0000 7fec4e40e640 1 -- 172.21.15.84:0/1453212488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4815af70 con 0x7fec4806b3f0 2023-10-16T11:25:05.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.947+0000 7fec457fa640 1 -- 172.21.15.84:0/1453212488 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3800ce70 con 0x7fec4806b3f0 2023-10-16T11:25:05.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.947+0000 7fec4e40e640 1 -- 172.21.15.84:0/1453212488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4815b530 con 0x7fec4806b3f0 2023-10-16T11:25:05.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.948+0000 7fec4e40e640 1 -- 172.21.15.84:0/1453212488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec4804e160 con 0x7fec4806b3f0 2023-10-16T11:25:05.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.949+0000 7fec457fa640 1 -- 172.21.15.84:0/1453212488 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 139) v1 ==== 89490+0+0 (secure 0 0 0) 0x7fec380113f0 con 0x7fec4806b3f0 2023-10-16T11:25:05.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.950+0000 7fec457fa640 1 --2- 172.21.15.84:0/1453212488 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fec2806b140 0x7fec2806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:05.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.950+0000 7fec457fa640 1 -- 172.21.15.84:0/1453212488 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6945+0+0 (secure 0 0 0) 0x7fec3808a170 con 0x7fec4806b3f0 2023-10-16T11:25:05.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.950+0000 7fec3ffff640 1 -- 172.21.15.84:0/1453212488 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fec2806b140 msgr2=0x7fec2806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2880641466 2023-10-16T11:25:05.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.950+0000 7fec3ffff640 1 --2- 172.21.15.84:0/1453212488 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fec2806b140 0x7fec2806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:25:05.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:05.954+0000 7fec457fa640 1 -- 172.21.15.84:0/1453212488 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fec38054970 con 0x7fec4806b3f0 2023-10-16T11:25:05.963 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:05.962+0000 7f8c25a58040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:25:05.990 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:05.989+0000 7f014f02c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:25:06.017 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:06.015+0000 7f8c25a58040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:25:06.043 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:06.041+0000 7f014f02c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:25:06.069 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:06.067+0000 7f8c25a58040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:25:06.099 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:06.097+0000 7f014f02c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:25:06.138 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:06.136+0000 7f8c25a58040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:25:06.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.151+0000 7fec3ffff640 1 -- 172.21.15.84:0/1453212488 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fec2806b140 msgr2=0x7fec2806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2880641466 2023-10-16T11:25:06.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.151+0000 7fec3ffff640 1 --2- 172.21.15.84:0/1453212488 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fec2806b140 0x7fec2806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:25:06.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.151+0000 7fec4e40e640 1 -- 172.21.15.84:0/1453212488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec480d0220 con 0x7fec4806b3f0 2023-10-16T11:25:06.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.152+0000 7fec457fa640 1 -- 172.21.15.84:0/1453212488 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v222) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec38057f90 con 0x7fec4806b3f0 2023-10-16T11:25:06.154 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:25:06.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.154+0000 7fec4e40e640 1 -- 172.21.15.84:0/1453212488 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fec2806b140 msgr2=0x7fec2806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:25:06.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.154+0000 7fec4e40e640 1 --2- 172.21.15.84:0/1453212488 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fec2806b140 0x7fec2806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.154+0000 7fec4e40e640 1 -- 172.21.15.84:0/1453212488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec4806b3f0 msgr2=0x7fec4814af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:06.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.154+0000 7fec4e40e640 1 --2- 172.21.15.84:0/1453212488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec4806b3f0 0x7fec4814af80 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7fec3800c820 tx=0x7fec3800cc20 comp rx=0 tx=0).stop 2023-10-16T11:25:06.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.154+0000 7fec4e40e640 1 -- 172.21.15.84:0/1453212488 shutdown_connections 2023-10-16T11:25:06.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.154+0000 7fec4e40e640 1 --2- 172.21.15.84:0/1453212488 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fec2806b140 0x7fec2806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.154+0000 7fec4e40e640 1 --2- 172.21.15.84:0/1453212488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fec48159e40 0x7fec480c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.154+0000 7fec4e40e640 1 --2- 172.21.15.84:0/1453212488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec480ca780 0x7fec480c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.154+0000 7fec4e40e640 1 --2- 172.21.15.84:0/1453212488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec4806b3f0 0x7fec4814af80 unknown :-1 s=CLOSED pgs=2765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.154+0000 7fec4e40e640 1 -- 172.21.15.84:0/1453212488 >> 172.21.15.84:0/1453212488 conn(0x7fec480cfac0 msgr2=0x7fec480c40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:06.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.154+0000 7fec4e40e640 1 -- 172.21.15.84:0/1453212488 shutdown_connections 2023-10-16T11:25:06.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.154+0000 7fec4e40e640 1 -- 172.21.15.84:0/1453212488 wait complete. 2023-10-16T11:25:06.161 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:06.159+0000 7f014f02c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:25:06.174 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:25:06.190 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:06.188+0000 7f8c25a58040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:25:06.211 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:06.209+0000 7f014f02c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:25:06.307 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:06.305+0000 7f8c25a58040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:25:06.324 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:06.322+0000 7f014f02c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:25:06.373 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:06.371+0000 7f8c25a58040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:25:06.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.374+0000 7f5cf997a640 1 Processor -- start 2023-10-16T11:25:06.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.374+0000 7f5cf997a640 1 -- start start 2023-10-16T11:25:06.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.375+0000 7f5cf997a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cf40b8480 0x7f5cf40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:06.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.375+0000 7f5cf997a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf415d150 con 0x7f5cf40d0be0 2023-10-16T11:25:06.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.375+0000 7f5cf997a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf40d1180 con 0x7f5cf415cd70 2023-10-16T11:25:06.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.375+0000 7f5cf997a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40d1460 con 0x7f5cf40b8480 2023-10-16T11:25:06.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.376+0000 7f5cf9179640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f5cf40d0be0 0x7f5cf40b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35942/0 (socket says 172.21.15.84:35942) 2023-10-16T11:25:06.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.376+0000 7f5cf9179640 1 -- 172.21.15.84:0/3020085723 learned_addr learned my addr 172.21.15.84:0/3020085723 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:06.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.376+0000 7f5cf3fff640 1 --2- 172.21.15.84:0/3020085723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cf40b8480 0x7f5cf40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:06.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.376+0000 7f5cf37fe640 1 -- 172.21.15.84:0/3020085723 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850754084 0 0) 0x7f5cf415d150 con 0x7f5cf40d0be0 2023-10-16T11:25:06.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.376+0000 7f5cf37fe640 1 -- 172.21.15.84:0/3020085723 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ce4003520 con 0x7f5cf40d0be0 2023-10-16T11:25:06.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf37fe640 1 -- 172.21.15.84:0/3020085723 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2457772305 0 0) 0x7f5ce4003520 con 0x7f5cf40d0be0 2023-10-16T11:25:06.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf37fe640 1 -- 172.21.15.84:0/3020085723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cf40b8480 msgr2=0x7f5cf40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:06.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf37fe640 1 --2- 172.21.15.84:0/3020085723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cf40b8480 0x7f5cf40cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf37fe640 1 -- 172.21.15.84:0/3020085723 >> v1:172.21.15.157:6789/0 conn(0x7f5cf415cd70 legacy=0x7f5cf40d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:06.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf37fe640 1 -- 172.21.15.84:0/3020085723 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf40d1740 con 0x7f5cf40d0be0 2023-10-16T11:25:06.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf37fe640 1 -- 172.21.15.84:0/3020085723 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f5cec0019f0 con 0x7f5cf40d0be0 2023-10-16T11:25:06.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf37fe640 1 -- 172.21.15.84:0/3020085723 >> v1:172.21.15.84:6789/0 conn(0x7f5cf40d0be0 legacy=0x7f5cf40b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:06.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf37fe640 1 --2- 172.21.15.84:0/3020085723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cf40b8480 0x7f5ce40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:06.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf37fe640 1 --2- 172.21.15.84:0/3020085723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ce4003f20 0x7f5ce40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:06.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf37fe640 1 --2- 172.21.15.84:0/3020085723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce4004d00 0x7f5ce4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:06.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf37fe640 1 -- 172.21.15.84:0/3020085723 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40d1460 con 0x7f5ce4003f20 2023-10-16T11:25:06.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf37fe640 1 -- 172.21.15.84:0/3020085723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce400ba60 con 0x7f5ce4004d00 2023-10-16T11:25:06.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf37fe640 1 -- 172.21.15.84:0/3020085723 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce400bc20 con 0x7f5cf40b8480 2023-10-16T11:25:06.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf3fff640 1 --2- 172.21.15.84:0/3020085723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ce4003f20 0x7f5ce40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:06.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf3fff640 1 -- 172.21.15.84:0/3020085723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cf40b8480 msgr2=0x7f5ce40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:06.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf3fff640 1 --2- 172.21.15.84:0/3020085723 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cf40b8480 0x7f5ce40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf3fff640 1 -- 172.21.15.84:0/3020085723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce4004d00 msgr2=0x7f5ce4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:06.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf3fff640 1 --2- 172.21.15.84:0/3020085723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce4004d00 0x7f5ce4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.377+0000 7f5cf3fff640 1 -- 172.21.15.84:0/3020085723 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce4003520 con 0x7f5ce4003f20 2023-10-16T11:25:06.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.379+0000 7f5cf3fff640 1 --2- 172.21.15.84:0/3020085723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ce4003f20 0x7f5ce40047c0 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009ac0 tx=0x7f5ce40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402b2d4f24abce41 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:06.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.379+0000 7f5cf37fe640 1 -- 172.21.15.84:0/3020085723 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce002c710 con 0x7f5ce4003f20 2023-10-16T11:25:06.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.379+0000 7f5cf37fe640 1 -- 172.21.15.84:0/3020085723 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce002c870 con 0x7f5ce4003f20 2023-10-16T11:25:06.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.379+0000 7f5cf37fe640 1 -- 172.21.15.84:0/3020085723 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce002cc00 con 0x7f5ce4003f20 2023-10-16T11:25:06.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.380+0000 7f5cf997a640 1 -- 172.21.15.84:0/3020085723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ce4003f20 msgr2=0x7f5ce40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:06.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.380+0000 7f5cf997a640 1 --2- 172.21.15.84:0/3020085723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ce4003f20 0x7f5ce40047c0 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009ac0 tx=0x7f5ce40057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.380+0000 7f5cf9179640 1 -- 172.21.15.84:0/3020085723 reap_dead start 2023-10-16T11:25:06.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.380+0000 7f5cf997a640 1 -- 172.21.15.84:0/3020085723 shutdown_connections 2023-10-16T11:25:06.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.380+0000 7f5cf997a640 1 -- 172.21.15.84:0/3020085723 >> 172.21.15.84:0/3020085723 conn(0x7f5cf40cf570 msgr2=0x7f5cf40cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:06.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.380+0000 7f5cf997a640 1 -- 172.21.15.84:0/3020085723 shutdown_connections 2023-10-16T11:25:06.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.380+0000 7f5cf997a640 1 -- 172.21.15.84:0/3020085723 wait complete. 2023-10-16T11:25:06.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.382+0000 7f5cf997a640 1 Processor -- start 2023-10-16T11:25:06.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.382+0000 7f5cf997a640 1 -- start start 2023-10-16T11:25:06.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.383+0000 7f5cf997a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ce4003f20 0x7f5cf40c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:06.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.383+0000 7f5cf997a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cf40b41f0 0x7f5cf40b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:06.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.383+0000 7f5cf997a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf40c9bc0 0x7f5cf40ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:06.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.383+0000 7f5cf997a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40b4bb0 con 0x7f5ce4003f20 2023-10-16T11:25:06.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.383+0000 7f5cf997a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40ca8f0 con 0x7f5cf40c9bc0 2023-10-16T11:25:06.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.383+0000 7f5cf997a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40cb090 con 0x7f5cf40b41f0 2023-10-16T11:25:06.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.383+0000 7f5cf8978640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ce4003f20 0x7f5cf40c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:06.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.383+0000 7f5cf8978640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ce4003f20 0x7f5cf40c8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39972/0 (socket says 172.21.15.84:39972) 2023-10-16T11:25:06.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.383+0000 7f5cf8978640 1 -- 172.21.15.84:0/441437519 learned_addr learned my addr 172.21.15.84:0/441437519 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:06.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.383+0000 7f5cf3fff640 1 --2- 172.21.15.84:0/441437519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cf40b41f0 0x7f5cf40b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:06.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.383+0000 7f5cf9179640 1 --2- 172.21.15.84:0/441437519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf40c9bc0 0x7f5cf40ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:06.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.383+0000 7f5cf8978640 1 -- 172.21.15.84:0/441437519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cf40b41f0 msgr2=0x7f5cf40b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:06.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.383+0000 7f5cf8978640 1 --2- 172.21.15.84:0/441437519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cf40b41f0 0x7f5cf40b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.383+0000 7f5cf8978640 1 -- 172.21.15.84:0/441437519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf40c9bc0 msgr2=0x7f5cf40ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:06.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.383+0000 7f5cf8978640 1 --2- 172.21.15.84:0/441437519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf40c9bc0 0x7f5cf40ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.383+0000 7f5cf8978640 1 -- 172.21.15.84:0/441437519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf40cb370 con 0x7f5ce4003f20 2023-10-16T11:25:06.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.384+0000 7f5cf8978640 1 --2- 172.21.15.84:0/441437519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ce4003f20 0x7f5cf40c8440 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800c500 tx=0x7f5ce800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:06.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.384+0000 7f5cf17fa640 1 -- 172.21.15.84:0/441437519 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80052d0 con 0x7f5ce4003f20 2023-10-16T11:25:06.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.384+0000 7f5cf17fa640 1 -- 172.21.15.84:0/441437519 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce8005430 con 0x7f5ce4003f20 2023-10-16T11:25:06.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.384+0000 7f5cf17fa640 1 -- 172.21.15.84:0/441437519 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8012870 con 0x7f5ce4003f20 2023-10-16T11:25:06.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.384+0000 7f5cf997a640 1 -- 172.21.15.84:0/441437519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf40cb650 con 0x7f5ce4003f20 2023-10-16T11:25:06.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.384+0000 7f5cf997a640 1 -- 172.21.15.84:0/441437519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf40ca580 con 0x7f5ce4003f20 2023-10-16T11:25:06.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.385+0000 7f5cf997a640 1 -- 172.21.15.84:0/441437519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf404e9b0 con 0x7f5ce4003f20 2023-10-16T11:25:06.395 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:06.388+0000 7f014f02c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:25:06.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.386+0000 7f5cf17fa640 1 -- 172.21.15.84:0/441437519 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 139) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f5ce80129d0 con 0x7f5ce4003f20 2023-10-16T11:25:06.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.387+0000 7f5cf17fa640 1 --2- 172.21.15.84:0/441437519 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f5cd806b170 0x7f5cd806d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:06.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.387+0000 7f5cf17fa640 1 -- 172.21.15.84:0/441437519 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6945+0+0 (secure 0 0 0) 0x7f5ce8089a70 con 0x7f5ce4003f20 2023-10-16T11:25:06.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.387+0000 7f5cf3fff640 1 -- 172.21.15.84:0/441437519 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f5cd806b170 msgr2=0x7f5cd806d630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2880641466 2023-10-16T11:25:06.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.387+0000 7f5cf3fff640 1 --2- 172.21.15.84:0/441437519 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f5cd806b170 0x7f5cd806d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:25:06.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.391+0000 7f5cf17fa640 1 -- 172.21.15.84:0/441437519 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5ce8054380 con 0x7f5ce4003f20 2023-10-16T11:25:06.426 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:06.424+0000 7f8c25a58040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:25:06.441 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:06.439+0000 7f014f02c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:25:06.485 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:06.483+0000 7f8c25a58040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:25:06.498 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:06.496+0000 7f014f02c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:25:06.537 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:06.535+0000 7f8c25a58040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:25:06.549 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:06.547+0000 7f014f02c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:25:06.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.576+0000 7f5cf997a640 1 -- 172.21.15.84:0/441437519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5cf40c8ec0 con 0x7f5ce4003f20 2023-10-16T11:25:06.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.577+0000 7f5cf17fa640 1 -- 172.21.15.84:0/441437519 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v222) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5ce80579a0 con 0x7f5ce4003f20 2023-10-16T11:25:06.579 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:25:06.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.579+0000 7f5cf997a640 1 -- 172.21.15.84:0/441437519 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f5cd806b170 msgr2=0x7f5cd806d630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:25:06.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.579+0000 7f5cf997a640 1 --2- 172.21.15.84:0/441437519 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f5cd806b170 0x7f5cd806d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.579+0000 7f5cf997a640 1 -- 172.21.15.84:0/441437519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ce4003f20 msgr2=0x7f5cf40c8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:06.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.579+0000 7f5cf997a640 1 --2- 172.21.15.84:0/441437519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ce4003f20 0x7f5cf40c8440 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800c500 tx=0x7f5ce800c9c0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.579+0000 7f5cf997a640 1 -- 172.21.15.84:0/441437519 shutdown_connections 2023-10-16T11:25:06.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.579+0000 7f5cf997a640 1 --2- 172.21.15.84:0/441437519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf40c9bc0 0x7f5cf40ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.579+0000 7f5cf997a640 1 --2- 172.21.15.84:0/441437519 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f5cd806b170 0x7f5cd806d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.579+0000 7f5cf997a640 1 --2- 172.21.15.84:0/441437519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cf40b41f0 0x7f5cf40b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.579+0000 7f5cf997a640 1 --2- 172.21.15.84:0/441437519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ce4003f20 0x7f5cf40c8440 unknown :-1 s=CLOSED pgs=2768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.579+0000 7f5cf997a640 1 -- 172.21.15.84:0/441437519 >> 172.21.15.84:0/441437519 conn(0x7f5cf40cf570 msgr2=0x7f5cf40d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:06.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.580+0000 7f5cf997a640 1 -- 172.21.15.84:0/441437519 shutdown_connections 2023-10-16T11:25:06.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.580+0000 7f5cf997a640 1 -- 172.21.15.84:0/441437519 wait complete. 2023-10-16T11:25:06.599 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:25:06.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.848+0000 7f1e45a89640 1 Processor -- start 2023-10-16T11:25:06.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.849+0000 7f1e45a89640 1 -- start start 2023-10-16T11:25:06.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.850+0000 7f1e45a89640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1e400caa80 0x7f1e40069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:06.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.850+0000 7f1e45a89640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1e4006a2e0 0x7f1e401650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:06.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.850+0000 7f1e45a89640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e400caef0 con 0x7f1e400addb0 2023-10-16T11:25:06.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.850+0000 7f1e45a89640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e40161a50 con 0x7f1e4006a2e0 2023-10-16T11:25:06.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.850+0000 7f1e45a89640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4006a6f0 con 0x7f1e400caa80 2023-10-16T11:25:06.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.850+0000 7f1e3ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1e4006a2e0 0x7f1e401650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:06.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.850+0000 7f1e3f7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1e400addb0 0x7f1e400c9000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35944/0 (socket says 172.21.15.84:35944) 2023-10-16T11:25:06.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.850+0000 7f1e3f7fe640 1 -- 172.21.15.84:0/2783302300 learned_addr learned my addr 172.21.15.84:0/2783302300 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:06.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.851+0000 7f1e3effd640 1 --2- 172.21.15.84:0/2783302300 >> v2:172.21.15.157:3300/0 conn(0x7f1e400caa80 0x7f1e40069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:06.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.851+0000 7f1e3ffff640 1 -- 172.21.15.84:0/2783302300 >> v2:172.21.15.157:3300/0 conn(0x7f1e400caa80 msgr2=0x7f1e40069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:06.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.851+0000 7f1e3ffff640 1 --2- 172.21.15.84:0/2783302300 >> v2:172.21.15.157:3300/0 conn(0x7f1e400caa80 0x7f1e40069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.851+0000 7f1e3ffff640 1 -- 172.21.15.84:0/2783302300 >> v1:172.21.15.84:6789/0 conn(0x7f1e400addb0 legacy=0x7f1e400c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:06.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.851+0000 7f1e3ffff640 1 -- 172.21.15.84:0/2783302300 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e400cb140 con 0x7f1e4006a2e0 2023-10-16T11:25:06.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.851+0000 7f1e3e7fc640 1 -- 172.21.15.84:0/2783302300 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504034493 0 0) 0x7f1e400caef0 con 0x7f1e400addb0 2023-10-16T11:25:06.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.853+0000 7f1e3ffff640 1 --2- 172.21.15.84:0/2783302300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4006a2e0 0x7f1e401650d0 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f1e400cdbe0 tx=0x7f1e3802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32549ad957b2c64c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:06.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.853+0000 7f1e3e7fc640 1 -- 172.21.15.84:0/2783302300 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e38031830 con 0x7f1e4006a2e0 2023-10-16T11:25:06.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.853+0000 7f1e3e7fc640 1 -- 172.21.15.84:0/2783302300 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e38031990 con 0x7f1e4006a2e0 2023-10-16T11:25:06.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.853+0000 7f1e3e7fc640 1 -- 172.21.15.84:0/2783302300 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e38031d20 con 0x7f1e4006a2e0 2023-10-16T11:25:06.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.854+0000 7f1e45a89640 1 -- 172.21.15.84:0/2783302300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4006a2e0 msgr2=0x7f1e401650d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:06.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.854+0000 7f1e45a89640 1 --2- 172.21.15.84:0/2783302300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4006a2e0 0x7f1e401650d0 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f1e400cdbe0 tx=0x7f1e3802b8d0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.854+0000 7f1e45a89640 1 -- 172.21.15.84:0/2783302300 shutdown_connections 2023-10-16T11:25:06.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.854+0000 7f1e45a89640 1 --2- 172.21.15.84:0/2783302300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4006a2e0 0x7f1e401650d0 unknown :-1 s=CLOSED pgs=2770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.854+0000 7f1e45a89640 1 --2- 172.21.15.84:0/2783302300 >> v2:172.21.15.157:3300/0 conn(0x7f1e400caa80 0x7f1e40069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.854+0000 7f1e45a89640 1 -- 172.21.15.84:0/2783302300 >> 172.21.15.84:0/2783302300 conn(0x7f1e400ce9b0 msgr2=0x7f1e400cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:06.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.854+0000 7f1e45a89640 1 -- 172.21.15.84:0/2783302300 shutdown_connections 2023-10-16T11:25:06.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.854+0000 7f1e45a89640 1 -- 172.21.15.84:0/2783302300 wait complete. 2023-10-16T11:25:06.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.857+0000 7f1e45a89640 1 Processor -- start 2023-10-16T11:25:06.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.857+0000 7f1e45a89640 1 -- start start 2023-10-16T11:25:06.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.858+0000 7f1e45a89640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4006a2e0 0x7f1e401ea730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:06.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.858+0000 7f1e45a89640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e400addb0 0x7f1e401e8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:06.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.858+0000 7f1e45a89640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e400caa80 0x7f1e4015f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:06.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.858+0000 7f1e45a89640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e400c6270 con 0x7f1e4006a2e0 2023-10-16T11:25:06.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.858+0000 7f1e45a89640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4015f570 con 0x7f1e400caa80 2023-10-16T11:25:06.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.858+0000 7f1e45a89640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e401faed0 con 0x7f1e400addb0 2023-10-16T11:25:06.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.859+0000 7f1e3f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4006a2e0 0x7f1e401ea730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:06.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.859+0000 7f1e3f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4006a2e0 0x7f1e401ea730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39982/0 (socket says 172.21.15.84:39982) 2023-10-16T11:25:06.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.859+0000 7f1e3f7fe640 1 -- 172.21.15.84:0/1768661522 learned_addr learned my addr 172.21.15.84:0/1768661522 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:06.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.859+0000 7f1e3f7fe640 1 -- 172.21.15.84:0/1768661522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e400addb0 msgr2=0x7f1e401e8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:06.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.859+0000 7f1e3ffff640 1 --2- 172.21.15.84:0/1768661522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e400caa80 0x7f1e4015f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:06.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.859+0000 7f1e3f7fe640 1 --2- 172.21.15.84:0/1768661522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e400addb0 0x7f1e401e8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.859+0000 7f1e3f7fe640 1 -- 172.21.15.84:0/1768661522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e400caa80 msgr2=0x7f1e4015f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:06.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.859+0000 7f1e3f7fe640 1 --2- 172.21.15.84:0/1768661522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e400caa80 0x7f1e4015f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:06.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.859+0000 7f1e3f7fe640 1 -- 172.21.15.84:0/1768661522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e38009400 con 0x7f1e4006a2e0 2023-10-16T11:25:06.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.859+0000 7f1e3f7fe640 1 --2- 172.21.15.84:0/1768661522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4006a2e0 0x7f1e401ea730 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000c890 tx=0x7f1e3000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:06.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.860+0000 7f1e27fff640 1 -- 172.21.15.84:0/1768661522 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3000a640 con 0x7f1e4006a2e0 2023-10-16T11:25:06.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.860+0000 7f1e27fff640 1 -- 172.21.15.84:0/1768661522 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3000a7c0 con 0x7f1e4006a2e0 2023-10-16T11:25:06.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.860+0000 7f1e27fff640 1 -- 172.21.15.84:0/1768661522 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3000ab70 con 0x7f1e4006a2e0 2023-10-16T11:25:06.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.860+0000 7f1e45a89640 1 -- 172.21.15.84:0/1768661522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e401fb1b0 con 0x7f1e4006a2e0 2023-10-16T11:25:06.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.860+0000 7f1e45a89640 1 -- 172.21.15.84:0/1768661522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e400c64c0 con 0x7f1e4006a2e0 2023-10-16T11:25:06.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.861+0000 7f1e45a89640 1 -- 172.21.15.84:0/1768661522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4004e160 con 0x7f1e4006a2e0 2023-10-16T11:25:06.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.862+0000 7f1e27fff640 1 -- 172.21.15.84:0/1768661522 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 139) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f1e3000acd0 con 0x7f1e4006a2e0 2023-10-16T11:25:06.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.863+0000 7f1e27fff640 1 --2- 172.21.15.84:0/1768661522 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f1e2c06b140 0x7f1e2c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:06.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.863+0000 7f1e27fff640 1 -- 172.21.15.84:0/1768661522 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6945+0+0 (secure 0 0 0) 0x7f1e300894c0 con 0x7f1e4006a2e0 2023-10-16T11:25:06.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.863+0000 7f1e3effd640 1 -- 172.21.15.84:0/1768661522 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f1e2c06b140 msgr2=0x7f1e2c06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2880641466 2023-10-16T11:25:06.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.863+0000 7f1e3effd640 1 --2- 172.21.15.84:0/1768661522 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f1e2c06b140 0x7f1e2c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:25:06.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:06.867+0000 7f1e27fff640 1 -- 172.21.15.84:0/1768661522 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1e30053cf0 con 0x7f1e4006a2e0 2023-10-16T11:25:06.939 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:06.937+0000 7f8c25a58040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:25:06.942 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:06.940+0000 7f014f02c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:25:07.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.061+0000 7f1e45a89640 1 -- 172.21.15.84:0/1768661522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1e400ce230 con 0x7f1e4006a2e0 2023-10-16T11:25:07.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.062+0000 7f1e27fff640 1 -- 172.21.15.84:0/1768661522 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v222) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1e30057310 con 0x7f1e4006a2e0 2023-10-16T11:25:07.064 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:25:07.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.064+0000 7f1e3effd640 1 -- 172.21.15.84:0/1768661522 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f1e2c06b140 msgr2=0x7f1e2c06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2880641466 2023-10-16T11:25:07.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.064+0000 7f1e3effd640 1 --2- 172.21.15.84:0/1768661522 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f1e2c06b140 0x7f1e2c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:25:07.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.064+0000 7f1e45a89640 1 -- 172.21.15.84:0/1768661522 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f1e2c06b140 msgr2=0x7f1e2c06d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:25:07.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.064+0000 7f1e45a89640 1 --2- 172.21.15.84:0/1768661522 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f1e2c06b140 0x7f1e2c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.064+0000 7f1e45a89640 1 -- 172.21.15.84:0/1768661522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4006a2e0 msgr2=0x7f1e401ea730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:07.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.064+0000 7f1e45a89640 1 --2- 172.21.15.84:0/1768661522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4006a2e0 0x7f1e401ea730 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000c890 tx=0x7f1e3000cd50 comp rx=0 tx=0).stop 2023-10-16T11:25:07.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.065+0000 7f1e45a89640 1 -- 172.21.15.84:0/1768661522 shutdown_connections 2023-10-16T11:25:07.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.065+0000 7f1e45a89640 1 --2- 172.21.15.84:0/1768661522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e400caa80 0x7f1e4015f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.065+0000 7f1e45a89640 1 --2- 172.21.15.84:0/1768661522 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f1e2c06b140 0x7f1e2c06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.065+0000 7f1e45a89640 1 --2- 172.21.15.84:0/1768661522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e400addb0 0x7f1e401e8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.065+0000 7f1e45a89640 1 --2- 172.21.15.84:0/1768661522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4006a2e0 0x7f1e401ea730 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.065+0000 7f1e45a89640 1 -- 172.21.15.84:0/1768661522 >> 172.21.15.84:0/1768661522 conn(0x7f1e400ce9b0 msgr2=0x7f1e40160df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:07.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.065+0000 7f1e45a89640 1 -- 172.21.15.84:0/1768661522 shutdown_connections 2023-10-16T11:25:07.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.065+0000 7f1e45a89640 1 -- 172.21.15.84:0/1768661522 wait complete. 2023-10-16T11:25:07.085 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-16T11:25:07.100 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:07.099+0000 7f8c25a58040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:25:07.103 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:07.101+0000 7f014f02c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:25:07.216 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:07.214+0000 7f8c25a58040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:25:07.217 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:07.214+0000 7f014f02c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:25:07.267 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:07.266+0000 7f014f02c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:25:07.268 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:07.267+0000 7f8c25a58040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:25:07.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.284+0000 7f0c77347640 1 Processor -- start 2023-10-16T11:25:07.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.284+0000 7f0c77347640 1 -- start start 2023-10-16T11:25:07.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.286+0000 7f0c77347640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c700ad350 0x7f0c70069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:07.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.286+0000 7f0c77347640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0c7006a2e0 0x7f0c700c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:07.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.286+0000 7f0c77347640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c7006a6f0 con 0x7f0c7015d150 2023-10-16T11:25:07.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.286+0000 7f0c77347640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c7015d530 con 0x7f0c7006a2e0 2023-10-16T11:25:07.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.286+0000 7f0c77347640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c700ac890 con 0x7f0c700ad350 2023-10-16T11:25:07.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.286+0000 7f0c758e4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0c7006a2e0 0x7f0c700c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:07.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.286+0000 7f0c758e4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0c7006a2e0 0x7f0c700c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39998/0 (socket says 172.21.15.84:39998) 2023-10-16T11:25:07.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.286+0000 7f0c758e4640 1 -- 172.21.15.84:0/49968662 learned_addr learned my addr 172.21.15.84:0/49968662 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:07.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.286+0000 7f0c750e3640 1 --1- 172.21.15.84:0/49968662 >> v1:172.21.15.84:6789/0 conn(0x7f0c7015d150 0x7f0c70160b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35952/0 (socket says 172.21.15.84:35952) 2023-10-16T11:25:07.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.287+0000 7f0c748e2640 1 --2- 172.21.15.84:0/49968662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c700ad350 0x7f0c70069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:07.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.287+0000 7f0c758e4640 1 -- 172.21.15.84:0/49968662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c700ad350 msgr2=0x7f0c70069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:07.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.287+0000 7f0c758e4640 1 --2- 172.21.15.84:0/49968662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c700ad350 0x7f0c70069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.287+0000 7f0c758e4640 1 -- 172.21.15.84:0/49968662 >> v1:172.21.15.84:6789/0 conn(0x7f0c7015d150 legacy=0x7f0c70160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:07.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.287+0000 7f0c758e4640 1 -- 172.21.15.84:0/49968662 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c70161650 con 0x7f0c7006a2e0 2023-10-16T11:25:07.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.288+0000 7f0c758e4640 1 --2- 172.21.15.84:0/49968662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c7006a2e0 0x7f0c700c9000 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009870 tx=0x7f0c6802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87793d87338a3487 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:07.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.288+0000 7f0c67fff640 1 -- 172.21.15.84:0/49968662 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68030840 con 0x7f0c7006a2e0 2023-10-16T11:25:07.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.288+0000 7f0c67fff640 1 -- 172.21.15.84:0/49968662 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c680309a0 con 0x7f0c7006a2e0 2023-10-16T11:25:07.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.288+0000 7f0c67fff640 1 -- 172.21.15.84:0/49968662 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68030d30 con 0x7f0c7006a2e0 2023-10-16T11:25:07.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.289+0000 7f0c77347640 1 -- 172.21.15.84:0/49968662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c7006a2e0 msgr2=0x7f0c700c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:07.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.289+0000 7f0c77347640 1 --2- 172.21.15.84:0/49968662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c7006a2e0 0x7f0c700c9000 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009870 tx=0x7f0c6802b980 comp rx=0 tx=0).stop 2023-10-16T11:25:07.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.289+0000 7f0c77347640 1 -- 172.21.15.84:0/49968662 shutdown_connections 2023-10-16T11:25:07.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.289+0000 7f0c77347640 1 --2- 172.21.15.84:0/49968662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c7006a2e0 0x7f0c700c9000 unknown :-1 s=CLOSED pgs=2773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.289+0000 7f0c77347640 1 --2- 172.21.15.84:0/49968662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c700ad350 0x7f0c70069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.289+0000 7f0c77347640 1 -- 172.21.15.84:0/49968662 >> 172.21.15.84:0/49968662 conn(0x7f0c700ce9b0 msgr2=0x7f0c700cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:07.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.289+0000 7f0c77347640 1 -- 172.21.15.84:0/49968662 shutdown_connections 2023-10-16T11:25:07.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.289+0000 7f0c77347640 1 -- 172.21.15.84:0/49968662 wait complete. 2023-10-16T11:25:07.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.291+0000 7f0c77347640 1 Processor -- start 2023-10-16T11:25:07.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.291+0000 7f0c77347640 1 -- start start 2023-10-16T11:25:07.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.292+0000 7f0c77347640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c7006a2e0 0x7f0c700b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:07.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.292+0000 7f0c77347640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c700ad350 0x7f0c700b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:07.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.292+0000 7f0c77347640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c7015d150 0x7f0c700b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:07.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.292+0000 7f0c77347640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7015d7c0 con 0x7f0c7006a2e0 2023-10-16T11:25:07.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.292+0000 7f0c77347640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7015da60 con 0x7f0c7015d150 2023-10-16T11:25:07.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.292+0000 7f0c77347640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7015dd40 con 0x7f0c700ad350 2023-10-16T11:25:07.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.292+0000 7f0c750e3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c7006a2e0 0x7f0c700b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:07.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.292+0000 7f0c750e3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c7006a2e0 0x7f0c700b0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40008/0 (socket says 172.21.15.84:40008) 2023-10-16T11:25:07.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.292+0000 7f0c750e3640 1 -- 172.21.15.84:0/321514423 learned_addr learned my addr 172.21.15.84:0/321514423 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:07.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.293+0000 7f0c748e2640 1 --2- 172.21.15.84:0/321514423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c700ad350 0x7f0c700b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:07.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.293+0000 7f0c758e4640 1 --2- 172.21.15.84:0/321514423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c7015d150 0x7f0c700b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:07.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.293+0000 7f0c750e3640 1 -- 172.21.15.84:0/321514423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c700ad350 msgr2=0x7f0c700b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:07.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.293+0000 7f0c750e3640 1 --2- 172.21.15.84:0/321514423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c700ad350 0x7f0c700b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.293+0000 7f0c750e3640 1 -- 172.21.15.84:0/321514423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c7015d150 msgr2=0x7f0c700b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:07.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.293+0000 7f0c750e3640 1 --2- 172.21.15.84:0/321514423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c7015d150 0x7f0c700b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.293+0000 7f0c750e3640 1 -- 172.21.15.84:0/321514423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c680094d0 con 0x7f0c7006a2e0 2023-10-16T11:25:07.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.293+0000 7f0c748e2640 1 --2- 172.21.15.84:0/321514423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c700ad350 0x7f0c700b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:07.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.293+0000 7f0c750e3640 1 --2- 172.21.15.84:0/321514423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c7006a2e0 0x7f0c700b0a30 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00c7a0 tx=0x7f0c6c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:07.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.293+0000 7f0c65ffb640 1 -- 172.21.15.84:0/321514423 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c00a640 con 0x7f0c7006a2e0 2023-10-16T11:25:07.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.293+0000 7f0c65ffb640 1 -- 172.21.15.84:0/321514423 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6c00a7a0 con 0x7f0c7006a2e0 2023-10-16T11:25:07.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.293+0000 7f0c77347640 1 -- 172.21.15.84:0/321514423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c701e69b0 con 0x7f0c7006a2e0 2023-10-16T11:25:07.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.293+0000 7f0c65ffb640 1 -- 172.21.15.84:0/321514423 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c00ab10 con 0x7f0c7006a2e0 2023-10-16T11:25:07.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.293+0000 7f0c77347640 1 -- 172.21.15.84:0/321514423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c700b0f70 con 0x7f0c7006a2e0 2023-10-16T11:25:07.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.295+0000 7f0c77347640 1 -- 172.21.15.84:0/321514423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c701e6ed0 con 0x7f0c7006a2e0 2023-10-16T11:25:07.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.295+0000 7f0c65ffb640 1 -- 172.21.15.84:0/321514423 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 139) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f0c6c00ac70 con 0x7f0c7006a2e0 2023-10-16T11:25:07.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.296+0000 7f0c65ffb640 1 --2- 172.21.15.84:0/321514423 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f0c5406b140 0x7f0c5406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:07.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.296+0000 7f0c65ffb640 1 -- 172.21.15.84:0/321514423 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6945+0+0 (secure 0 0 0) 0x7f0c6c0896e0 con 0x7f0c7006a2e0 2023-10-16T11:25:07.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.296+0000 7f0c748e2640 1 -- 172.21.15.84:0/321514423 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f0c5406b140 msgr2=0x7f0c5406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2880641466 2023-10-16T11:25:07.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.297+0000 7f0c748e2640 1 --2- 172.21.15.84:0/321514423 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f0c5406b140 0x7f0c5406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:25:07.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.300+0000 7f0c65ffb640 1 -- 172.21.15.84:0/321514423 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0c701e6ed0 con 0x7f0c7006a2e0 2023-10-16T11:25:07.378 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:07.376+0000 7f014f02c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:25:07.383 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:07.381+0000 7f8c25a58040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:25:07.487 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:07.485+0000 7f014f02c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:25:07.494 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:07.492+0000 7f8c25a58040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:25:07.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.495+0000 7f0c77347640 1 -- 172.21.15.84:0/321514423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0c701e6ed0 con 0x7f0c7006a2e0 2023-10-16T11:25:07.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.496+0000 7f0c65ffb640 1 -- 172.21.15.84:0/321514423 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v222) v1 ==== 107+0+11 (secure 0 0 0) 0x7f0c701e6ed0 con 0x7f0c7006a2e0 2023-10-16T11:25:07.498 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:25:07.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.498+0000 7f0c748e2640 1 -- 172.21.15.84:0/321514423 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f0c5406b140 msgr2=0x7f0c5406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2880641466 2023-10-16T11:25:07.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.498+0000 7f0c748e2640 1 --2- 172.21.15.84:0/321514423 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f0c5406b140 0x7f0c5406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:25:07.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.498+0000 7f0c77347640 1 -- 172.21.15.84:0/321514423 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f0c5406b140 msgr2=0x7f0c5406d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:25:07.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.498+0000 7f0c77347640 1 --2- 172.21.15.84:0/321514423 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f0c5406b140 0x7f0c5406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.498+0000 7f0c77347640 1 -- 172.21.15.84:0/321514423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c7006a2e0 msgr2=0x7f0c700b0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:07.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.498+0000 7f0c77347640 1 --2- 172.21.15.84:0/321514423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c7006a2e0 0x7f0c700b0a30 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00c7a0 tx=0x7f0c6c00cc60 comp rx=0 tx=0).stop 2023-10-16T11:25:07.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.498+0000 7f0c77347640 1 -- 172.21.15.84:0/321514423 shutdown_connections 2023-10-16T11:25:07.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.498+0000 7f0c77347640 1 --2- 172.21.15.84:0/321514423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c7015d150 0x7f0c700b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.498+0000 7f0c77347640 1 --2- 172.21.15.84:0/321514423 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f0c5406b140 0x7f0c5406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.498+0000 7f0c77347640 1 --2- 172.21.15.84:0/321514423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c700ad350 0x7f0c700b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.498+0000 7f0c77347640 1 --2- 172.21.15.84:0/321514423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c7006a2e0 0x7f0c700b0a30 unknown :-1 s=CLOSED pgs=2774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.498+0000 7f0c77347640 1 -- 172.21.15.84:0/321514423 >> 172.21.15.84:0/321514423 conn(0x7f0c700ce9b0 msgr2=0x7f0c7015a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:07.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.499+0000 7f0c77347640 1 -- 172.21.15.84:0/321514423 shutdown_connections 2023-10-16T11:25:07.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.499+0000 7f0c77347640 1 -- 172.21.15.84:0/321514423 wait complete. 2023-10-16T11:25:07.519 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-16T11:25:07.647 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:07.644+0000 7f014f02c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:25:07.659 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:07.657+0000 7f8c25a58040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:25:07.696 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:07.694+0000 7f014f02c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:25:07.710 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:07.708+0000 7f8c25a58040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:25:07.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.767+0000 7f12d1a8a640 1 Processor -- start 2023-10-16T11:25:07.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.768+0000 7f12d1a8a640 1 -- start start 2023-10-16T11:25:07.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.769+0000 7f12d1a8a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f12cc0cded0 0x7f12cc0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:07.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.769+0000 7f12d1a8a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc15d150 con 0x7f12cc15cd70 2023-10-16T11:25:07.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.769+0000 7f12d1a8a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc0d1180 con 0x7f12cc0b8480 2023-10-16T11:25:07.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.769+0000 7f12d1a8a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12cc0d1460 con 0x7f12cc0cded0 2023-10-16T11:25:07.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.769+0000 7f12d0a88640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f12cc15cd70 0x7f12cc0d2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35960/0 (socket says 172.21.15.84:35960) 2023-10-16T11:25:07.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.769+0000 7f12d0a88640 1 -- 172.21.15.84:0/4199059385 learned_addr learned my addr 172.21.15.84:0/4199059385 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:07.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.769+0000 7f12d1289640 1 --2- 172.21.15.84:0/4199059385 >> v2:172.21.15.84:3300/0 conn(0x7f12cc0cded0 0x7f12cc0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:07.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.770+0000 7f12cb7fe640 1 -- 172.21.15.84:0/4199059385 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472056221 0 0) 0x7f12cc15d150 con 0x7f12cc15cd70 2023-10-16T11:25:07.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.770+0000 7f12cb7fe640 1 -- 172.21.15.84:0/4199059385 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12bc003520 con 0x7f12cc15cd70 2023-10-16T11:25:07.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.770+0000 7f12d1289640 1 -- 172.21.15.84:0/4199059385 >> v1:172.21.15.157:6789/0 conn(0x7f12cc0b8480 legacy=0x7f12cc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:07.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.770+0000 7f12d1289640 1 -- 172.21.15.84:0/4199059385 >> v1:172.21.15.84:6789/0 conn(0x7f12cc15cd70 legacy=0x7f12cc0d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:07.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.770+0000 7f12d1289640 1 -- 172.21.15.84:0/4199059385 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc0d1740 con 0x7f12cc0cded0 2023-10-16T11:25:07.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.771+0000 7f12d1289640 1 --2- 172.21.15.84:0/4199059385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12cc0cded0 0x7f12cc0b5c90 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009d90 tx=0x7f12c402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d8236e769c7182 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:07.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.771+0000 7f12cb7fe640 1 -- 172.21.15.84:0/4199059385 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c402cd00 con 0x7f12cc0cded0 2023-10-16T11:25:07.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.771+0000 7f12cb7fe640 1 -- 172.21.15.84:0/4199059385 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c402ce60 con 0x7f12cc0cded0 2023-10-16T11:25:07.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.771+0000 7f12cb7fe640 1 -- 172.21.15.84:0/4199059385 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4034660 con 0x7f12cc0cded0 2023-10-16T11:25:07.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.772+0000 7f12d1a8a640 1 -- 172.21.15.84:0/4199059385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12cc0cded0 msgr2=0x7f12cc0b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:07.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.772+0000 7f12d1a8a640 1 --2- 172.21.15.84:0/4199059385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12cc0cded0 0x7f12cc0b5c90 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009d90 tx=0x7f12c402bf50 comp rx=0 tx=0).stop 2023-10-16T11:25:07.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.772+0000 7f12d1a8a640 1 -- 172.21.15.84:0/4199059385 shutdown_connections 2023-10-16T11:25:07.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.772+0000 7f12d1a8a640 1 --2- 172.21.15.84:0/4199059385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12cc0cded0 0x7f12cc0b5c90 unknown :-1 s=CLOSED pgs=2776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.772+0000 7f12d1a8a640 1 -- 172.21.15.84:0/4199059385 >> 172.21.15.84:0/4199059385 conn(0x7f12cc0cf570 msgr2=0x7f12cc0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:07.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.772+0000 7f12d1a8a640 1 -- 172.21.15.84:0/4199059385 shutdown_connections 2023-10-16T11:25:07.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.772+0000 7f12d1a8a640 1 -- 172.21.15.84:0/4199059385 wait complete. 2023-10-16T11:25:07.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.774+0000 7f12d1a8a640 1 Processor -- start 2023-10-16T11:25:07.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.774+0000 7f12d1a8a640 1 -- start start 2023-10-16T11:25:07.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.775+0000 7f12d1a8a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12cc0b8480 0x7f12cc0b7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:07.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.775+0000 7f12d1a8a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12cc0cded0 0x7f12cc15f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:07.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.775+0000 7f12d1a8a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12cc15cd70 0x7f12cc15fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:07.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.775+0000 7f12d1a8a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc0b5e80 con 0x7f12cc0cded0 2023-10-16T11:25:07.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.775+0000 7f12d1a8a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc1601e0 con 0x7f12cc0b8480 2023-10-16T11:25:07.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.775+0000 7f12d1a8a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc0b8210 con 0x7f12cc15cd70 2023-10-16T11:25:07.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.775+0000 7f12cbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12cc0cded0 0x7f12cc15f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:07.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.775+0000 7f12cbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12cc0cded0 0x7f12cc15f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40030/0 (socket says 172.21.15.84:40030) 2023-10-16T11:25:07.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.775+0000 7f12cbfff640 1 -- 172.21.15.84:0/2544603284 learned_addr learned my addr 172.21.15.84:0/2544603284 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:07.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.775+0000 7f12d0a88640 1 --2- 172.21.15.84:0/2544603284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12cc0b8480 0x7f12cc0b7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:07.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.775+0000 7f12cbfff640 1 -- 172.21.15.84:0/2544603284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12cc15cd70 msgr2=0x7f12cc15fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:07.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.775+0000 7f12cbfff640 1 --2- 172.21.15.84:0/2544603284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12cc15cd70 0x7f12cc15fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.775+0000 7f12cbfff640 1 -- 172.21.15.84:0/2544603284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12cc0b8480 msgr2=0x7f12cc0b7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:07.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.775+0000 7f12cbfff640 1 --2- 172.21.15.84:0/2544603284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12cc0b8480 0x7f12cc0b7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.775+0000 7f12cbfff640 1 -- 172.21.15.84:0/2544603284 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c4009a70 con 0x7f12cc0cded0 2023-10-16T11:25:07.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.775+0000 7f12d0a88640 1 --2- 172.21.15.84:0/2544603284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12cc0b8480 0x7f12cc0b7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:07.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.776+0000 7f12cbfff640 1 --2- 172.21.15.84:0/2544603284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12cc0cded0 0x7f12cc15f760 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f12b800c480 tx=0x7f12b800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:07.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.776+0000 7f12c97fa640 1 -- 172.21.15.84:0/2544603284 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b80052d0 con 0x7f12cc0cded0 2023-10-16T11:25:07.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.776+0000 7f12c97fa640 1 -- 172.21.15.84:0/2544603284 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b8005430 con 0x7f12cc0cded0 2023-10-16T11:25:07.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.776+0000 7f12c97fa640 1 -- 172.21.15.84:0/2544603284 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b8012870 con 0x7f12cc0cded0 2023-10-16T11:25:07.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.776+0000 7f12d1a8a640 1 -- 172.21.15.84:0/2544603284 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12cc158420 con 0x7f12cc0cded0 2023-10-16T11:25:07.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.776+0000 7f12d1a8a640 1 -- 172.21.15.84:0/2544603284 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc1589e0 con 0x7f12cc0cded0 2023-10-16T11:25:07.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.778+0000 7f12d1a8a640 1 -- 172.21.15.84:0/2544603284 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12cc04e9b0 con 0x7f12cc0cded0 2023-10-16T11:25:07.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.778+0000 7f12c97fa640 1 -- 172.21.15.84:0/2544603284 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 139) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f12b80129d0 con 0x7f12cc0cded0 2023-10-16T11:25:07.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.779+0000 7f12c97fa640 1 --2- 172.21.15.84:0/2544603284 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f12b006b140 0x7f12b006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:07.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.779+0000 7f12c97fa640 1 -- 172.21.15.84:0/2544603284 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6945+0+0 (secure 0 0 0) 0x7f12b8089ac0 con 0x7f12cc0cded0 2023-10-16T11:25:07.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.779+0000 7f12d0a88640 1 -- 172.21.15.84:0/2544603284 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f12b006b140 msgr2=0x7f12b006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2880641466 2023-10-16T11:25:07.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.779+0000 7f12d0a88640 1 --2- 172.21.15.84:0/2544603284 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f12b006b140 0x7f12b006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:25:07.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.783+0000 7f12c97fa640 1 -- 172.21.15.84:0/2544603284 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f12b8054390 con 0x7f12cc0cded0 2023-10-16T11:25:07.818 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:07.816+0000 7f014f02c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:25:07.851 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:07.849+0000 7f8c25a58040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:25:07.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.980+0000 7f12d0a88640 1 -- 172.21.15.84:0/2544603284 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f12b006b140 msgr2=0x7f12b006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2880641466 2023-10-16T11:25:07.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.980+0000 7f12d0a88640 1 --2- 172.21.15.84:0/2544603284 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f12b006b140 0x7f12b006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:25:07.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.982+0000 7f12d1a8a640 1 -- 172.21.15.84:0/2544603284 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f12cc0d3730 con 0x7f12cc0cded0 2023-10-16T11:25:07.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.982+0000 7f12c97fa640 1 -- 172.21.15.84:0/2544603284 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v222) v1 ==== 103+0+11 (secure 0 0 0) 0x7f12b80579b0 con 0x7f12cc0cded0 2023-10-16T11:25:07.984 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:25:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.986+0000 7f12d1a8a640 1 -- 172.21.15.84:0/2544603284 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f12b006b140 msgr2=0x7f12b006d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:25:07.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.986+0000 7f12d1a8a640 1 --2- 172.21.15.84:0/2544603284 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f12b006b140 0x7f12b006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.986+0000 7f12d1a8a640 1 -- 172.21.15.84:0/2544603284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12cc0cded0 msgr2=0x7f12cc15f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.986+0000 7f12d1a8a640 1 --2- 172.21.15.84:0/2544603284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12cc0cded0 0x7f12cc15f760 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f12b800c480 tx=0x7f12b800c940 comp rx=0 tx=0).stop 2023-10-16T11:25:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.986+0000 7f12d1a8a640 1 -- 172.21.15.84:0/2544603284 shutdown_connections 2023-10-16T11:25:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.986+0000 7f12d1a8a640 1 --2- 172.21.15.84:0/2544603284 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f12b006b140 0x7f12b006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.986+0000 7f12d1a8a640 1 --2- 172.21.15.84:0/2544603284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12cc15cd70 0x7f12cc15fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.986+0000 7f12d1a8a640 1 --2- 172.21.15.84:0/2544603284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12cc0cded0 0x7f12cc15f760 unknown :-1 s=CLOSED pgs=2777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.986+0000 7f12d1a8a640 1 --2- 172.21.15.84:0/2544603284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12cc0b8480 0x7f12cc0b7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:07.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.986+0000 7f12d1a8a640 1 -- 172.21.15.84:0/2544603284 >> 172.21.15.84:0/2544603284 conn(0x7f12cc0cf570 msgr2=0x7f12cc15be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:07.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.986+0000 7f12d1a8a640 1 -- 172.21.15.84:0/2544603284 shutdown_connections 2023-10-16T11:25:07.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:07.986+0000 7f12d1a8a640 1 -- 172.21.15.84:0/2544603284 wait complete. 2023-10-16T11:25:08.013 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T11:25:08.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.219+0000 7fbe88d73640 1 Processor -- start 2023-10-16T11:25:08.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.219+0000 7fbe88d73640 1 -- start start 2023-10-16T11:25:08.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.220+0000 7fbe88d73640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbe840cf5f0 0x7fbe8406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:08.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.220+0000 7fbe88d73640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbe84070c10 0x7fbe840cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:08.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.220+0000 7fbe88d73640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe84161740 con 0x7fbe84070830 2023-10-16T11:25:08.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.220+0000 7fbe88d73640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe840cdbd0 con 0x7fbe840cf5f0 2023-10-16T11:25:08.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.220+0000 7fbe88d73640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe8406b0f0 con 0x7fbe84070c10 2023-10-16T11:25:08.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.221+0000 7fbe7bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbe840cf5f0 0x7fbe8406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:08.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.221+0000 7fbe7bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbe840cf5f0 0x7fbe8406d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40042/0 (socket says 172.21.15.84:40042) 2023-10-16T11:25:08.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.221+0000 7fbe7bfff640 1 -- 172.21.15.84:0/635132972 learned_addr learned my addr 172.21.15.84:0/635132972 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:08.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.221+0000 7fbe82d9d640 1 --2- 172.21.15.84:0/635132972 >> v2:172.21.15.157:3300/0 conn(0x7fbe84070c10 0x7fbe840cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:08.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.221+0000 7fbe7bfff640 1 -- 172.21.15.84:0/635132972 >> v2:172.21.15.157:3300/0 conn(0x7fbe84070c10 msgr2=0x7fbe840cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:08.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.221+0000 7fbe7bfff640 1 --2- 172.21.15.84:0/635132972 >> v2:172.21.15.157:3300/0 conn(0x7fbe84070c10 0x7fbe840cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:08.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.221+0000 7fbe7bfff640 1 -- 172.21.15.84:0/635132972 >> v1:172.21.15.157:6789/0 conn(0x7fbe84070830 legacy=0x7fbe8406c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:08.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.221+0000 7fbe7bfff640 1 -- 172.21.15.84:0/635132972 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe841619b0 con 0x7fbe840cf5f0 2023-10-16T11:25:08.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.223+0000 7fbe7bfff640 1 --2- 172.21.15.84:0/635132972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe840cf5f0 0x7fbe8406d3c0 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7fbe840d0b90 tx=0x7fbe6c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965af4f601a716e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:08.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.223+0000 7fbe81d9b640 1 -- 172.21.15.84:0/635132972 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c0317e0 con 0x7fbe840cf5f0 2023-10-16T11:25:08.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.224+0000 7fbe81d9b640 1 -- 172.21.15.84:0/635132972 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6c031940 con 0x7fbe840cf5f0 2023-10-16T11:25:08.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.224+0000 7fbe81d9b640 1 -- 172.21.15.84:0/635132972 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c031cd0 con 0x7fbe840cf5f0 2023-10-16T11:25:08.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.224+0000 7fbe88d73640 1 -- 172.21.15.84:0/635132972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe840cf5f0 msgr2=0x7fbe8406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:08.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.224+0000 7fbe88d73640 1 --2- 172.21.15.84:0/635132972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe840cf5f0 0x7fbe8406d3c0 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7fbe840d0b90 tx=0x7fbe6c02b880 comp rx=0 tx=0).stop 2023-10-16T11:25:08.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.224+0000 7fbe88d73640 1 -- 172.21.15.84:0/635132972 shutdown_connections 2023-10-16T11:25:08.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.224+0000 7fbe88d73640 1 --2- 172.21.15.84:0/635132972 >> v2:172.21.15.157:3300/0 conn(0x7fbe84070c10 0x7fbe840cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:08.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.224+0000 7fbe88d73640 1 --2- 172.21.15.84:0/635132972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe840cf5f0 0x7fbe8406d3c0 unknown :-1 s=CLOSED pgs=2778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:08.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.224+0000 7fbe88d73640 1 -- 172.21.15.84:0/635132972 >> 172.21.15.84:0/635132972 conn(0x7fbe840cfac0 msgr2=0x7fbe840cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:08.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.224+0000 7fbe88d73640 1 -- 172.21.15.84:0/635132972 shutdown_connections 2023-10-16T11:25:08.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.224+0000 7fbe88d73640 1 -- 172.21.15.84:0/635132972 wait complete. 2023-10-16T11:25:08.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.226+0000 7fbe88d73640 1 Processor -- start 2023-10-16T11:25:08.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.226+0000 7fbe88d73640 1 -- start start 2023-10-16T11:25:08.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.227+0000 7fbe88d73640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe84070830 0x7fbe84157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:08.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.227+0000 7fbe88d73640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe84070c10 0x7fbe8415eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:08.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.228+0000 7fbe88d73640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe840cf5f0 0x7fbe8415f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:08.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.228+0000 7fbe88d73640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8415be80 con 0x7fbe840cf5f0 2023-10-16T11:25:08.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.228+0000 7fbe88d73640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe841f3470 con 0x7fbe84070c10 2023-10-16T11:25:08.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.228+0000 7fbe88d73640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8415b8c0 con 0x7fbe84070830 2023-10-16T11:25:08.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.228+0000 7fbe82d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe840cf5f0 0x7fbe8415f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:08.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.228+0000 7fbe82d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe840cf5f0 0x7fbe8415f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40054/0 (socket says 172.21.15.84:40054) 2023-10-16T11:25:08.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.228+0000 7fbe82d9d640 1 -- 172.21.15.84:0/3618976386 learned_addr learned my addr 172.21.15.84:0/3618976386 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:08.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.228+0000 7fbe8259c640 1 --2- 172.21.15.84:0/3618976386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe84070830 0x7fbe84157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:08.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.228+0000 7fbe7bfff640 1 --2- 172.21.15.84:0/3618976386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe84070c10 0x7fbe8415eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:08.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.228+0000 7fbe82d9d640 1 -- 172.21.15.84:0/3618976386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe84070830 msgr2=0x7fbe84157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:08.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.228+0000 7fbe82d9d640 1 --2- 172.21.15.84:0/3618976386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe84070830 0x7fbe84157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:08.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.228+0000 7fbe82d9d640 1 -- 172.21.15.84:0/3618976386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe84070c10 msgr2=0x7fbe8415eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:08.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.228+0000 7fbe82d9d640 1 --2- 172.21.15.84:0/3618976386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe84070c10 0x7fbe8415eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:08.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.228+0000 7fbe82d9d640 1 -- 172.21.15.84:0/3618976386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c0094d0 con 0x7fbe840cf5f0 2023-10-16T11:25:08.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.228+0000 7fbe7bfff640 1 --2- 172.21.15.84:0/3618976386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe84070c10 0x7fbe8415eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:25:08.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.228+0000 7fbe8259c640 1 --2- 172.21.15.84:0/3618976386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe84070830 0x7fbe84157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:25:08.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.229+0000 7fbe82d9d640 1 --2- 172.21.15.84:0/3618976386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe840cf5f0 0x7fbe8415f0d0 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c007e40 tx=0x7fbe7c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:08.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.229+0000 7fbe7affd640 1 -- 172.21.15.84:0/3618976386 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c014620 con 0x7fbe840cf5f0 2023-10-16T11:25:08.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.229+0000 7fbe7affd640 1 -- 172.21.15.84:0/3618976386 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7c00ee90 con 0x7fbe840cf5f0 2023-10-16T11:25:08.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.229+0000 7fbe7affd640 1 -- 172.21.15.84:0/3618976386 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c0149e0 con 0x7fbe840cf5f0 2023-10-16T11:25:08.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.229+0000 7fbe88d73640 1 -- 172.21.15.84:0/3618976386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8415bba0 con 0x7fbe840cf5f0 2023-10-16T11:25:08.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.229+0000 7fbe88d73640 1 -- 172.21.15.84:0/3618976386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe84158520 con 0x7fbe840cf5f0 2023-10-16T11:25:08.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.230+0000 7fbe88d73640 1 -- 172.21.15.84:0/3618976386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe840c9350 con 0x7fbe840cf5f0 2023-10-16T11:25:08.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.231+0000 7fbe7affd640 1 -- 172.21.15.84:0/3618976386 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 139) v1 ==== 89490+0+0 (secure 0 0 0) 0x7fbe7c014b40 con 0x7fbe840cf5f0 2023-10-16T11:25:08.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.233+0000 7fbe7affd640 1 --2- 172.21.15.84:0/3618976386 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fbe6406b140 0x7fbe6406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:08.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.233+0000 7fbe8259c640 1 -- 172.21.15.84:0/3618976386 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fbe6406b140 msgr2=0x7fbe6406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2880641466 2023-10-16T11:25:08.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.233+0000 7fbe7affd640 1 -- 172.21.15.84:0/3618976386 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6945+0+0 (secure 0 0 0) 0x7fbe7c08a8c0 con 0x7fbe840cf5f0 2023-10-16T11:25:08.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.233+0000 7fbe8259c640 1 --2- 172.21.15.84:0/3618976386 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fbe6406b140 0x7fbe6406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:25:08.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.237+0000 7fbe7affd640 1 -- 172.21.15.84:0/3618976386 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbe840c9350 con 0x7fbe840cf5f0 2023-10-16T11:25:08.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.434+0000 7fbe8259c640 1 -- 172.21.15.84:0/3618976386 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fbe6406b140 msgr2=0x7fbe6406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2880641466 2023-10-16T11:25:08.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.434+0000 7fbe8259c640 1 --2- 172.21.15.84:0/3618976386 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fbe6406b140 0x7fbe6406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:25:08.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.442+0000 7fbe88d73640 1 -- 172.21.15.84:0/3618976386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbe840c9350 con 0x7fbe840cf5f0 2023-10-16T11:25:08.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.442+0000 7fbe7affd640 1 -- 172.21.15.84:0/3618976386 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v222) v1 ==== 80+0+469 (secure 0 0 0) 0x7fbe840c9350 con 0x7fbe840cf5f0 2023-10-16T11:25:08.444 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2877946366 2023-10-17T11:24:58.196616+0000 2023-10-16T11:25:08.445 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6833/1425409017 2023-10-17T11:24:58.196616+0000 2023-10-16T11:25:08.445 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6832/1425409017 2023-10-17T11:24:58.196616+0000 2023-10-16T11:25:08.445 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1771162859 2023-10-17T11:24:58.196616+0000 2023-10-16T11:25:08.445 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3655599055 2023-10-17T11:24:58.196616+0000 2023-10-16T11:25:08.445 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1076506559 2023-10-17T11:24:58.196616+0000 2023-10-16T11:25:08.446 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2748224910 2023-10-17T11:24:58.196616+0000 2023-10-16T11:25:08.446 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/857172551 2023-10-17T11:24:58.196616+0000 2023-10-16T11:25:08.447 INFO:teuthology.orchestra.run.smithi084.stderr:listed 8 entries 2023-10-16T11:25:08.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.444+0000 7fbe88d73640 1 -- 172.21.15.84:0/3618976386 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fbe6406b140 msgr2=0x7fbe6406d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:25:08.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.444+0000 7fbe88d73640 1 --2- 172.21.15.84:0/3618976386 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fbe6406b140 0x7fbe6406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:08.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.445+0000 7fbe88d73640 1 -- 172.21.15.84:0/3618976386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe840cf5f0 msgr2=0x7fbe8415f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:08.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.445+0000 7fbe88d73640 1 --2- 172.21.15.84:0/3618976386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe840cf5f0 0x7fbe8415f0d0 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c007e40 tx=0x7fbe7c00e4b0 comp rx=0 tx=0).stop 2023-10-16T11:25:08.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.445+0000 7fbe88d73640 1 -- 172.21.15.84:0/3618976386 shutdown_connections 2023-10-16T11:25:08.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.445+0000 7fbe88d73640 1 --2- 172.21.15.84:0/3618976386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe840cf5f0 0x7fbe8415f0d0 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:08.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.445+0000 7fbe88d73640 1 --2- 172.21.15.84:0/3618976386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe84070c10 0x7fbe8415eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:08.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.445+0000 7fbe88d73640 1 --2- 172.21.15.84:0/3618976386 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7fbe6406b140 0x7fbe6406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:08.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.445+0000 7fbe88d73640 1 --2- 172.21.15.84:0/3618976386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe84070830 0x7fbe84157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:08.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.445+0000 7fbe88d73640 1 -- 172.21.15.84:0/3618976386 >> 172.21.15.84:0/3618976386 conn(0x7fbe840cfac0 msgr2=0x7fbe840cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:08.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.445+0000 7fbe88d73640 1 -- 172.21.15.84:0/3618976386 shutdown_connections 2023-10-16T11:25:08.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.445+0000 7fbe88d73640 1 -- 172.21.15.84:0/3618976386 wait complete. 2023-10-16T11:25:08.463 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T11:25:08.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:25:08.464 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T11:25:08.555 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T11:25:08.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:25:08.556 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T11:25:08.597 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:25:08.598 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:25:08.652 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:25:08.652 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:25:08.653 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:25:08.663 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:08.662+0000 7f014f02c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:25:08.703 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:25:08.704 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:25:08.704 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:25:08.719 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:08.717+0000 7f8c25a58040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:25:08.759 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:25:08.759 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:25:08.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.962+0000 7f65f8421640 1 Processor -- start 2023-10-16T11:25:08.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.963+0000 7f65f8421640 1 -- start start 2023-10-16T11:25:08.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.964+0000 7f65f8421640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f65f015aaa0 0x7f65f015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:08.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.964+0000 7f65f8421640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f015ae80 0x7f65f00cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:08.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.964+0000 7f65f8421640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65f00ccaa0 0x7f65f0070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:08.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.964+0000 7f65f8421640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f006b810 con 0x7f65f015aaa0 2023-10-16T11:25:08.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.964+0000 7f65f8421640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f015ed60 con 0x7f65f00ccaa0 2023-10-16T11:25:08.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.964+0000 7f65f8421640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f00707e0 con 0x7f65f015ae80 2023-10-16T11:25:08.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.964+0000 7f65f61bd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f65f015aaa0 0x7f65f015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:08.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.964+0000 7f65f61bd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f65f015aaa0 0x7f65f015e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40064/0 (socket says 172.21.15.84:40064) 2023-10-16T11:25:08.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.964+0000 7f65f61bd640 1 -- 172.21.15.84:0/184388079 learned_addr learned my addr 172.21.15.84:0/184388079 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:08.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.965+0000 7f65f59bc640 1 --2- 172.21.15.84:0/184388079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f015ae80 0x7f65f00cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:08.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.965+0000 7f65f69be640 1 --2- 172.21.15.84:0/184388079 >> v2:172.21.15.157:3300/0 conn(0x7f65f00ccaa0 0x7f65f0070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:08.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.965+0000 7f65f61bd640 1 -- 172.21.15.84:0/184388079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f015ae80 msgr2=0x7f65f00cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:08.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.965+0000 7f65f61bd640 1 --2- 172.21.15.84:0/184388079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f015ae80 0x7f65f00cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:08.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.965+0000 7f65f61bd640 1 -- 172.21.15.84:0/184388079 >> v2:172.21.15.157:3300/0 conn(0x7f65f00ccaa0 msgr2=0x7f65f0070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:08.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.965+0000 7f65f61bd640 1 --2- 172.21.15.84:0/184388079 >> v2:172.21.15.157:3300/0 conn(0x7f65f00ccaa0 0x7f65f0070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:08.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.965+0000 7f65f61bd640 1 -- 172.21.15.84:0/184388079 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f015ef90 con 0x7f65f015aaa0 2023-10-16T11:25:08.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.966+0000 7f65f61bd640 1 --2- 172.21.15.84:0/184388079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f015aaa0 0x7f65f015e820 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f65f00ae3c0 tx=0x7f65ec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956dfc6e79f9e843 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:08.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.966+0000 7f65f51bb640 1 -- 172.21.15.84:0/184388079 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec031870 con 0x7f65f015aaa0 2023-10-16T11:25:08.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.966+0000 7f65f51bb640 1 -- 172.21.15.84:0/184388079 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ec0319d0 con 0x7f65f015aaa0 2023-10-16T11:25:08.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.966+0000 7f65f51bb640 1 -- 172.21.15.84:0/184388079 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec031d60 con 0x7f65f015aaa0 2023-10-16T11:25:08.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.967+0000 7f65f8421640 1 -- 172.21.15.84:0/184388079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f015aaa0 msgr2=0x7f65f015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:08.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.967+0000 7f65f8421640 1 --2- 172.21.15.84:0/184388079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f015aaa0 0x7f65f015e820 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f65f00ae3c0 tx=0x7f65ec02b910 comp rx=0 tx=0).stop 2023-10-16T11:25:08.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.967+0000 7f65f8421640 1 -- 172.21.15.84:0/184388079 shutdown_connections 2023-10-16T11:25:08.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.967+0000 7f65f8421640 1 --2- 172.21.15.84:0/184388079 >> v2:172.21.15.157:3300/0 conn(0x7f65f00ccaa0 0x7f65f0070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:08.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.967+0000 7f65f8421640 1 --2- 172.21.15.84:0/184388079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f015ae80 0x7f65f00cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:08.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.967+0000 7f65f8421640 1 --2- 172.21.15.84:0/184388079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f015aaa0 0x7f65f015e820 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:08.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.967+0000 7f65f8421640 1 -- 172.21.15.84:0/184388079 >> 172.21.15.84:0/184388079 conn(0x7f65f00cb410 msgr2=0x7f65f00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:08.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.967+0000 7f65f8421640 1 -- 172.21.15.84:0/184388079 shutdown_connections 2023-10-16T11:25:08.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.967+0000 7f65f8421640 1 -- 172.21.15.84:0/184388079 wait complete. 2023-10-16T11:25:08.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.969+0000 7f65f8421640 1 Processor -- start 2023-10-16T11:25:08.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.969+0000 7f65f8421640 1 -- start start 2023-10-16T11:25:08.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.970+0000 7f65f8421640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f00ccaa0 0x7f65f00c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:08.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.970+0000 7f65f8421640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f015aaa0 0x7f65f0157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:08.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.970+0000 7f65f8421640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f015ae80 0x7f65f01f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:08.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.970+0000 7f65f8421640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f0143610 con 0x7f65f015ae80 2023-10-16T11:25:08.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.970+0000 7f65f8421640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f00c4460 con 0x7f65f00ccaa0 2023-10-16T11:25:08.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.970+0000 7f65f8421640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f0158010 con 0x7f65f015aaa0 2023-10-16T11:25:08.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.970+0000 7f65f69be640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f015ae80 0x7f65f01f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:08.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.970+0000 7f65f69be640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f015ae80 0x7f65f01f1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40076/0 (socket says 172.21.15.84:40076) 2023-10-16T11:25:08.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.970+0000 7f65f69be640 1 -- 172.21.15.84:0/2809093961 learned_addr learned my addr 172.21.15.84:0/2809093961 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:08.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.970+0000 7f65f61bd640 1 --2- 172.21.15.84:0/2809093961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f00ccaa0 0x7f65f00c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:08.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.970+0000 7f65f59bc640 1 --2- 172.21.15.84:0/2809093961 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f015aaa0 0x7f65f0157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:08.978 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:08.970+0000 7f014f02c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:25:08.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.971+0000 7f65f69be640 1 -- 172.21.15.84:0/2809093961 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f015aaa0 msgr2=0x7f65f0157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:08.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.971+0000 7f65f69be640 1 --2- 172.21.15.84:0/2809093961 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f015aaa0 0x7f65f0157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:08.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.971+0000 7f65f69be640 1 -- 172.21.15.84:0/2809093961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f00ccaa0 msgr2=0x7f65f00c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:08.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.971+0000 7f65f69be640 1 --2- 172.21.15.84:0/2809093961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f00ccaa0 0x7f65f00c3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:08.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.971+0000 7f65f69be640 1 -- 172.21.15.84:0/2809093961 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec0094d0 con 0x7f65f015ae80 2023-10-16T11:25:08.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.971+0000 7f65f61bd640 1 --2- 172.21.15.84:0/2809093961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f00ccaa0 0x7f65f00c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:25:08.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.971+0000 7f65f69be640 1 --2- 172.21.15.84:0/2809093961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f015ae80 0x7f65f01f1fe0 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f65e8007d50 tx=0x7f65e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:08.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.971+0000 7f65e6ffd640 1 -- 172.21.15.84:0/2809093961 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8014420 con 0x7f65f015ae80 2023-10-16T11:25:08.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.971+0000 7f65e6ffd640 1 -- 172.21.15.84:0/2809093961 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e80145a0 con 0x7f65f015ae80 2023-10-16T11:25:08.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.971+0000 7f65e6ffd640 1 -- 172.21.15.84:0/2809093961 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8014950 con 0x7f65f015ae80 2023-10-16T11:25:08.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.971+0000 7f65f8421640 1 -- 172.21.15.84:0/2809093961 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f0142a00 con 0x7f65f015ae80 2023-10-16T11:25:08.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.972+0000 7f65f8421640 1 -- 172.21.15.84:0/2809093961 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f0142fc0 con 0x7f65f015ae80 2023-10-16T11:25:08.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.973+0000 7f65f8421640 1 -- 172.21.15.84:0/2809093961 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f015f530 con 0x7f65f015ae80 2023-10-16T11:25:08.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.974+0000 7f65e6ffd640 1 -- 172.21.15.84:0/2809093961 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 139) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f65e8014ab0 con 0x7f65f015ae80 2023-10-16T11:25:08.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.975+0000 7f65e6ffd640 1 --2- 172.21.15.84:0/2809093961 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f65d406b170 0x7f65d406d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:08.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.975+0000 7f65e6ffd640 1 -- 172.21.15.84:0/2809093961 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6945+0+0 (secure 0 0 0) 0x7f65e808a780 con 0x7f65f015ae80 2023-10-16T11:25:08.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.975+0000 7f65f61bd640 1 -- 172.21.15.84:0/2809093961 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f65d406b170 msgr2=0x7f65d406d630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2880641466 2023-10-16T11:25:08.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.975+0000 7f65f61bd640 1 --2- 172.21.15.84:0/2809093961 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f65d406b170 0x7f65d406d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:25:08.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:08.980+0000 7f65e6ffd640 1 -- 172.21.15.84:0/2809093961 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f65f015f530 con 0x7f65f015ae80 2023-10-16T11:25:09.030 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:09.029+0000 7f014f02c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:25:09.032 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.030+0000 7f8c25a58040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:25:09.091 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.089+0000 7f8c25a58040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:25:09.145 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:25:09.143+0000 7f014f02c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:25:09.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.176+0000 7f65f61bd640 1 -- 172.21.15.84:0/2809093961 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f65d406b170 msgr2=0x7f65d406d630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2880641466 2023-10-16T11:25:09.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.176+0000 7f65f61bd640 1 --2- 172.21.15.84:0/2809093961 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f65d406b170 0x7f65d406d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:25:09.208 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.206+0000 7f8c25a58040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:25:09.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.214+0000 7f65f8421640 1 -- 172.21.15.84:0/2809093961 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65f015f530 con 0x7f65f015ae80 2023-10-16T11:25:09.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.214+0000 7f65e6ffd640 1 -- 172.21.15.84:0/2809093961 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+6152 (secure 0 0 0) 0x7f65f015f530 con 0x7f65f015ae80 2023-10-16T11:25:09.216 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:09.217 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":120,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3106898227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3106898227},{"type":"v1","addr":"172.21.15.84:6837","nonce":3106898227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":7471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2296953749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2296953749},{"type":"v1","addr":"172.21.15.157:6834","nonce":2296953749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:24:25.531691+0000","modified":"2023-10-16T11:24:28.574116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11307},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11307":{"gid":11307,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/551895595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":551895595},{"type":"v1","addr":"172.21.15.84:6836","nonce":551895595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:24:40.366708+0000","modified":"2023-10-16T11:24:43.431532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7474":{"gid":7474,"name":"d","rank":0,"incarnation":116,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6835/3992377882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3992377882},{"type":"v1","addr":"172.21.15.157:6835","nonce":3992377882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-16T11:25:09.217 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 120 2023-10-16T11:25:09.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.217+0000 7f65f8421640 1 -- 172.21.15.84:0/2809093961 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f65d406b170 msgr2=0x7f65d406d630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:25:09.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.217+0000 7f65f8421640 1 --2- 172.21.15.84:0/2809093961 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f65d406b170 0x7f65d406d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.217+0000 7f65f8421640 1 -- 172.21.15.84:0/2809093961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f015ae80 msgr2=0x7f65f01f1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:09.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.217+0000 7f65f8421640 1 --2- 172.21.15.84:0/2809093961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f015ae80 0x7f65f01f1fe0 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f65e8007d50 tx=0x7f65e800e4b0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.217+0000 7f65f8421640 1 -- 172.21.15.84:0/2809093961 shutdown_connections 2023-10-16T11:25:09.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.217+0000 7f65f8421640 1 --2- 172.21.15.84:0/2809093961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f015ae80 0x7f65f01f1fe0 unknown :-1 s=CLOSED pgs=2781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.217+0000 7f65f8421640 1 --2- 172.21.15.84:0/2809093961 >> [v2:172.21.15.84:6832/2880641466,v1:172.21.15.84:6833/2880641466] conn(0x7f65d406b170 0x7f65d406d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.217+0000 7f65f8421640 1 --2- 172.21.15.84:0/2809093961 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f015aaa0 0x7f65f0157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.217+0000 7f65f8421640 1 --2- 172.21.15.84:0/2809093961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f00ccaa0 0x7f65f00c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.217+0000 7f65f8421640 1 -- 172.21.15.84:0/2809093961 >> 172.21.15.84:0/2809093961 conn(0x7f65f00cb410 msgr2=0x7f65f006ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:09.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.218+0000 7f65f8421640 1 -- 172.21.15.84:0/2809093961 shutdown_connections 2023-10-16T11:25:09.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.218+0000 7f65f8421640 1 -- 172.21.15.84:0/2809093961 wait complete. 2023-10-16T11:25:09.244 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:25:09.296 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.294+0000 7f8ba84a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.296 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.294+0000 7f8ba84a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.296 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.294+0000 7f8ba84a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.297 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.294+0000 7f8ba84a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.297 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.294+0000 7f8ba84a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.297 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.294+0000 7f8bab4ac640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.297 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.294+0000 7f8bab4ac640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.297 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.294+0000 7f8bab4ac640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.298 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.294+0000 7f8bab4ac640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.298 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.294+0000 7f8bab4ac640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.310 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.308+0000 7f8ba84a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.310 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.308+0000 7f8ba84a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.310 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.308+0000 7f8ba84a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.311 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.308+0000 7f8ba84a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.311 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.308+0000 7f8ba84a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.312 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.309+0000 7f8bab4ac640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.312 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.309+0000 7f8bab4ac640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.312 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.309+0000 7f8bab4ac640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.312 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.309+0000 7f8bab4ac640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.312 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:25:09.309+0000 7f8bab4ac640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:25:09.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.426+0000 7f0d2ffff640 1 Processor -- start 2023-10-16T11:25:09.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.426+0000 7f0d2ffff640 1 -- start start 2023-10-16T11:25:09.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.426+0000 7f0d2ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d3006c040 0x7f0d3006c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:09.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.426+0000 7f0d2ffff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d3015f4a0 con 0x7f0d300cda80 2023-10-16T11:25:09.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.426+0000 7f0d2ffff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d3006c960 con 0x7f0d300ad970 2023-10-16T11:25:09.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.426+0000 7f0d2ffff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d3015b9d0 con 0x7f0d3006c040 2023-10-16T11:25:09.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.427+0000 7f0d2f7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0d300cda80 0x7f0d3015ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35962/0 (socket says 172.21.15.84:35962) 2023-10-16T11:25:09.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.427+0000 7f0d2f7fe640 1 -- 172.21.15.84:0/3382492463 learned_addr learned my addr 172.21.15.84:0/3382492463 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:09.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.427+0000 7f0d2effd640 1 --2- 172.21.15.84:0/3382492463 >> v2:172.21.15.157:3300/0 conn(0x7f0d3006c040 0x7f0d3006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:09.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.427+0000 7f0d2dffb640 1 -- 172.21.15.84:0/3382492463 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 415243312 0 0) 0x7f0d3015f4a0 con 0x7f0d300cda80 2023-10-16T11:25:09.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.427+0000 7f0d2dffb640 1 -- 172.21.15.84:0/3382492463 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d18003520 con 0x7f0d300cda80 2023-10-16T11:25:09.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.427+0000 7f0d2dffb640 1 -- 172.21.15.84:0/3382492463 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3199558319 0 0) 0x7f0d18003520 con 0x7f0d300cda80 2023-10-16T11:25:09.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.427+0000 7f0d2dffb640 1 -- 172.21.15.84:0/3382492463 >> v2:172.21.15.157:3300/0 conn(0x7f0d3006c040 msgr2=0x7f0d3006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:09.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.427+0000 7f0d2dffb640 1 --2- 172.21.15.84:0/3382492463 >> v2:172.21.15.157:3300/0 conn(0x7f0d3006c040 0x7f0d3006c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.428+0000 7f0d2dffb640 1 -- 172.21.15.84:0/3382492463 >> v1:172.21.15.157:6789/0 conn(0x7f0d300ad970 legacy=0x7f0d3015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:09.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.428+0000 7f0d2dffb640 1 -- 172.21.15.84:0/3382492463 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3015bcb0 con 0x7f0d300cda80 2023-10-16T11:25:09.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.428+0000 7f0d2dffb640 1 -- 172.21.15.84:0/3382492463 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f0d280019f0 con 0x7f0d300cda80 2023-10-16T11:25:09.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.428+0000 7f0d2dffb640 1 -- 172.21.15.84:0/3382492463 >> v1:172.21.15.84:6789/0 conn(0x7f0d300cda80 legacy=0x7f0d3015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:09.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.428+0000 7f0d2dffb640 1 --2- 172.21.15.84:0/3382492463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d180039c0 0x7f0d18003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:09.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.428+0000 7f0d2dffb640 1 --2- 172.21.15.84:0/3382492463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d180047c0 0x7f0d18004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:09.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.428+0000 7f0d2dffb640 1 --2- 172.21.15.84:0/3382492463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d18005180 0x7f0d1800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:09.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.428+0000 7f0d2dffb640 1 -- 172.21.15.84:0/3382492463 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3015b9d0 con 0x7f0d18005180 2023-10-16T11:25:09.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.428+0000 7f0d2dffb640 1 -- 172.21.15.84:0/3382492463 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1800c080 con 0x7f0d180047c0 2023-10-16T11:25:09.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.428+0000 7f0d2dffb640 1 -- 172.21.15.84:0/3382492463 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1800c240 con 0x7f0d180039c0 2023-10-16T11:25:09.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.428+0000 7f0d2f7fe640 1 --2- 172.21.15.84:0/3382492463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d18005180 0x7f0d1800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:09.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.428+0000 7f0d2f7fe640 1 -- 172.21.15.84:0/3382492463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d180039c0 msgr2=0x7f0d18003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:09.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.428+0000 7f0d2f7fe640 1 --2- 172.21.15.84:0/3382492463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d180039c0 0x7f0d18003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.428+0000 7f0d2f7fe640 1 -- 172.21.15.84:0/3382492463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d180047c0 msgr2=0x7f0d18004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:09.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.428+0000 7f0d2f7fe640 1 --2- 172.21.15.84:0/3382492463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d180047c0 0x7f0d18004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.428+0000 7f0d2f7fe640 1 -- 172.21.15.84:0/3382492463 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d18003520 con 0x7f0d18005180 2023-10-16T11:25:09.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.429+0000 7f0d2f7fe640 1 -- 172.21.15.84:0/3382492463 reap_dead start 2023-10-16T11:25:09.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.429+0000 7f0d2f7fe640 1 --2- 172.21.15.84:0/3382492463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d18005180 0x7f0d1800b980 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800a760 tx=0x7f0d180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873e823794b660d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:09.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.429+0000 7f0d2dffb640 1 -- 172.21.15.84:0/3382492463 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28034630 con 0x7f0d18005180 2023-10-16T11:25:09.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.430+0000 7f0d2dffb640 1 -- 172.21.15.84:0/3382492463 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2802f070 con 0x7f0d18005180 2023-10-16T11:25:09.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.430+0000 7f0d2dffb640 1 -- 172.21.15.84:0/3382492463 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28034a10 con 0x7f0d18005180 2023-10-16T11:25:09.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.430+0000 7f0d2ffff640 1 -- 172.21.15.84:0/3382492463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d18005180 msgr2=0x7f0d1800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:09.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.430+0000 7f0d2ffff640 1 --2- 172.21.15.84:0/3382492463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d18005180 0x7f0d1800b980 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800a760 tx=0x7f0d180057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.430+0000 7f0d2ffff640 1 -- 172.21.15.84:0/3382492463 shutdown_connections 2023-10-16T11:25:09.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.430+0000 7f0d2ffff640 1 --2- 172.21.15.84:0/3382492463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d18005180 0x7f0d1800b980 unknown :-1 s=CLOSED pgs=2798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.430+0000 7f0d2ffff640 1 -- 172.21.15.84:0/3382492463 >> 172.21.15.84:0/3382492463 conn(0x7f0d300ce6a0 msgr2=0x7f0d300ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:09.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.430+0000 7f0d2ffff640 1 -- 172.21.15.84:0/3382492463 shutdown_connections 2023-10-16T11:25:09.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.430+0000 7f0d2ffff640 1 -- 172.21.15.84:0/3382492463 wait complete. 2023-10-16T11:25:09.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.432+0000 7f0d2ffff640 1 Processor -- start 2023-10-16T11:25:09.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.432+0000 7f0d2ffff640 1 -- start start 2023-10-16T11:25:09.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.433+0000 7f0d2ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d18005180 0x7f0d300b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:09.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.433+0000 7f0d2ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d300b3460 0x7f0d3015d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:09.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.433+0000 7f0d2ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3015d710 0x7f0d300b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:09.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.433+0000 7f0d2ffff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3015db90 con 0x7f0d300b3460 2023-10-16T11:25:09.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.433+0000 7f0d2ffff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d300b13f0 con 0x7f0d3015d710 2023-10-16T11:25:09.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.433+0000 7f0d2ffff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d300b38e0 con 0x7f0d18005180 2023-10-16T11:25:09.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.433+0000 7f0d2e7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d300b3460 0x7f0d3015d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:09.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.433+0000 7f0d2e7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d300b3460 0x7f0d3015d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40252/0 (socket says 172.21.15.84:40252) 2023-10-16T11:25:09.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.433+0000 7f0d2e7fc640 1 -- 172.21.15.84:0/3755759920 learned_addr learned my addr 172.21.15.84:0/3755759920 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:09.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.433+0000 7f0d2effd640 1 --2- 172.21.15.84:0/3755759920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d18005180 0x7f0d300b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:09.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.433+0000 7f0d2e7fc640 1 -- 172.21.15.84:0/3755759920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d18005180 msgr2=0x7f0d300b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:09.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.433+0000 7f0d2e7fc640 1 --2- 172.21.15.84:0/3755759920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d18005180 0x7f0d300b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.433+0000 7f0d2e7fc640 1 -- 172.21.15.84:0/3755759920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3015d710 msgr2=0x7f0d300b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:09.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.433+0000 7f0d2e7fc640 1 --2- 172.21.15.84:0/3755759920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3015d710 0x7f0d300b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.434+0000 7f0d2e7fc640 1 -- 172.21.15.84:0/3755759920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d300b1690 con 0x7f0d300b3460 2023-10-16T11:25:09.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.434+0000 7f0d2e7fc640 1 --2- 172.21.15.84:0/3755759920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d300b3460 0x7f0d3015d1d0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400c450 tx=0x7f0d2400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:09.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.434+0000 7f0d177fe640 1 -- 172.21.15.84:0/3755759920 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d240052d0 con 0x7f0d300b3460 2023-10-16T11:25:09.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.434+0000 7f0d177fe640 1 -- 172.21.15.84:0/3755759920 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d24005450 con 0x7f0d300b3460 2023-10-16T11:25:09.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.434+0000 7f0d177fe640 1 -- 172.21.15.84:0/3755759920 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24007260 con 0x7f0d300b3460 2023-10-16T11:25:09.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.434+0000 7f0d2ffff640 1 -- 172.21.15.84:0/3755759920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d300b1970 con 0x7f0d300b3460 2023-10-16T11:25:09.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.434+0000 7f0d2ffff640 1 -- 172.21.15.84:0/3755759920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d300b1f30 con 0x7f0d300b3460 2023-10-16T11:25:09.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.435+0000 7f0d2ffff640 1 -- 172.21.15.84:0/3755759920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3006b330 con 0x7f0d300b3460 2023-10-16T11:25:09.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.436+0000 7f0d177fe640 1 -- 172.21.15.84:0/3755759920 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 140) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f0d240073c0 con 0x7f0d300b3460 2023-10-16T11:25:09.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.437+0000 7f0d177fe640 1 -- 172.21.15.84:0/3755759920 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 7418+0+0 (secure 0 0 0) 0x7f0d24089080 con 0x7f0d300b3460 2023-10-16T11:25:09.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.442+0000 7f0d177fe640 1 -- 172.21.15.84:0/3755759920 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0d240bbfe0 con 0x7f0d300b3460 2023-10-16T11:25:09.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.636+0000 7f0d2ffff640 1 -- 172.21.15.84:0/3755759920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d3004e9b0 con 0x7f0d300b3460 2023-10-16T11:25:09.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.639+0000 7f0d177fe640 1 -- 172.21.15.84:0/3755759920 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v223) v1 ==== 74+0+21513 (secure 0 0 0) 0x7f0d24053e60 con 0x7f0d300b3460 2023-10-16T11:25:09.641 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:09.641 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":223,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:25:09.210192+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:24:23.363032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:24:24.551270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:24:38.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:24:39.387564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1287173368":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/595669474":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/4107174082":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2906390983":"2023-10-17T11:25:09.210125+0000","172.21.15.84:6833/2880641466":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/3419402221":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2161898622":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/455217819":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/4247809558":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/1814319372":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2877946366":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6833/1425409017":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6832/1425409017":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/1771162859":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6832/2880641466":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/3655599055":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/1076506559":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/2748224910":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/857172551":"2023-10-17T11:24:58.196616+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:25:09.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.641+0000 7f0d2ffff640 1 -- 172.21.15.84:0/3755759920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d300b3460 msgr2=0x7f0d3015d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:09.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.641+0000 7f0d2ffff640 1 --2- 172.21.15.84:0/3755759920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d300b3460 0x7f0d3015d1d0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400c450 tx=0x7f0d2400c910 comp rx=0 tx=0).stop 2023-10-16T11:25:09.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.641+0000 7f0d2ffff640 1 -- 172.21.15.84:0/3755759920 shutdown_connections 2023-10-16T11:25:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.641+0000 7f0d2ffff640 1 --2- 172.21.15.84:0/3755759920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3015d710 0x7f0d300b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.641+0000 7f0d2ffff640 1 --2- 172.21.15.84:0/3755759920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d300b3460 0x7f0d3015d1d0 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.641+0000 7f0d2ffff640 1 --2- 172.21.15.84:0/3755759920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d18005180 0x7f0d300b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.641+0000 7f0d2ffff640 1 -- 172.21.15.84:0/3755759920 >> 172.21.15.84:0/3755759920 conn(0x7f0d300ce6a0 msgr2=0x7f0d3015dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:09.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.641+0000 7f0d2ffff640 1 -- 172.21.15.84:0/3755759920 shutdown_connections 2023-10-16T11:25:09.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.641+0000 7f0d2ffff640 1 -- 172.21.15.84:0/3755759920 wait complete. 2023-10-16T11:25:09.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:24:23.363032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '216', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:24:24.551270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '217', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-16T11:24:38.531956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '220', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-16T11:24:39.387564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '221', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:25:09.662 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:25:09.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.858+0000 7fcfa8490640 1 Processor -- start 2023-10-16T11:25:09.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.858+0000 7fcfa8490640 1 -- start start 2023-10-16T11:25:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.861+0000 7fcfa8490640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcfa015ad70 0x7fcfa0069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.861+0000 7fcfa8490640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa00cb030 con 0x7fcfa00caac0 2023-10-16T11:25:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.861+0000 7fcfa8490640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa00c91c0 con 0x7fcfa00c9ca0 2023-10-16T11:25:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.861+0000 7fcfa8490640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa00c94a0 con 0x7fcfa015ad70 2023-10-16T11:25:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.861+0000 7fcfa622c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcfa015ad70 0x7fcfa0069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.861+0000 7fcfa622c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcfa015ad70 0x7fcfa0069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40256/0 (socket says 172.21.15.84:40256) 2023-10-16T11:25:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.861+0000 7fcfa622c640 1 -- 172.21.15.84:0/1697587164 learned_addr learned my addr 172.21.15.84:0/1697587164 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.861+0000 7fcfa6a2d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fcfa00caac0 0x7fcfa00ad4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35974/0 (socket says 172.21.15.84:35974) 2023-10-16T11:25:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.862+0000 7fcfa622c640 1 -- 172.21.15.84:0/1697587164 >> v1:172.21.15.157:6789/0 conn(0x7fcfa00c9ca0 legacy=0x7fcfa00ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.862+0000 7fcfa622c640 1 -- 172.21.15.84:0/1697587164 >> v1:172.21.15.84:6789/0 conn(0x7fcfa00caac0 legacy=0x7fcfa00ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.862+0000 7fcfa622c640 1 -- 172.21.15.84:0/1697587164 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa00c9780 con 0x7fcfa015ad70 2023-10-16T11:25:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.862+0000 7fcfa522a640 1 -- 172.21.15.84:0/1697587164 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1526842916 0 0) 0x7fcfa00cb030 con 0x7fcfa00caac0 2023-10-16T11:25:09.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.862+0000 7fcfa622c640 1 --2- 172.21.15.84:0/1697587164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa015ad70 0x7fcfa0069da0 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c009870 tx=0x7fcf9c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6d6a8e2b2878f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:09.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.862+0000 7fcfa522a640 1 -- 172.21.15.84:0/1697587164 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c030860 con 0x7fcfa015ad70 2023-10-16T11:25:09.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.862+0000 7fcfa522a640 1 -- 172.21.15.84:0/1697587164 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9c0309c0 con 0x7fcfa015ad70 2023-10-16T11:25:09.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.862+0000 7fcfa522a640 1 -- 172.21.15.84:0/1697587164 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c030d50 con 0x7fcfa015ad70 2023-10-16T11:25:09.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.863+0000 7fcfa8490640 1 -- 172.21.15.84:0/1697587164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa015ad70 msgr2=0x7fcfa0069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.863+0000 7fcfa8490640 1 --2- 172.21.15.84:0/1697587164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa015ad70 0x7fcfa0069da0 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c009870 tx=0x7fcf9c02b9a0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.863+0000 7fcfa8490640 1 -- 172.21.15.84:0/1697587164 shutdown_connections 2023-10-16T11:25:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.863+0000 7fcfa8490640 1 --2- 172.21.15.84:0/1697587164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa015ad70 0x7fcfa0069da0 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.863+0000 7fcfa8490640 1 -- 172.21.15.84:0/1697587164 >> 172.21.15.84:0/1697587164 conn(0x7fcfa00ce9b0 msgr2=0x7fcfa00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.863+0000 7fcfa8490640 1 -- 172.21.15.84:0/1697587164 shutdown_connections 2023-10-16T11:25:09.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.863+0000 7fcfa8490640 1 -- 172.21.15.84:0/1697587164 wait complete. 2023-10-16T11:25:09.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.865+0000 7fcfa8490640 1 Processor -- start 2023-10-16T11:25:09.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.865+0000 7fcfa8490640 1 -- start start 2023-10-16T11:25:09.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.866+0000 7fcfa8490640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfa00c9ca0 0x7fcfa01f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:09.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.866+0000 7fcfa8490640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfa00caac0 0x7fcfa01f0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:09.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.866+0000 7fcfa8490640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa015ad70 0x7fcfa00b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:09.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.866+0000 7fcfa8490640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa00a9a80 con 0x7fcfa015ad70 2023-10-16T11:25:09.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.866+0000 7fcfa8490640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa01f1150 con 0x7fcfa00c9ca0 2023-10-16T11:25:09.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.866+0000 7fcfa8490640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa00c7040 con 0x7fcfa00caac0 2023-10-16T11:25:09.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.866+0000 7fcfa6a2d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa015ad70 0x7fcfa00b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:09.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.866+0000 7fcfa6a2d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa015ad70 0x7fcfa00b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40270/0 (socket says 172.21.15.84:40270) 2023-10-16T11:25:09.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.866+0000 7fcfa6a2d640 1 -- 172.21.15.84:0/494664497 learned_addr learned my addr 172.21.15.84:0/494664497 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:09.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.866+0000 7fcfa6a2d640 1 -- 172.21.15.84:0/494664497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfa00caac0 msgr2=0x7fcfa01f0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:09.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.866+0000 7fcfa6a2d640 1 --2- 172.21.15.84:0/494664497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfa00caac0 0x7fcfa01f0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.866+0000 7fcfa6a2d640 1 -- 172.21.15.84:0/494664497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfa00c9ca0 msgr2=0x7fcfa01f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:09.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.866+0000 7fcfa6a2d640 1 --2- 172.21.15.84:0/494664497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfa00c9ca0 0x7fcfa01f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:09.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.866+0000 7fcfa6a2d640 1 -- 172.21.15.84:0/494664497 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9c0094d0 con 0x7fcfa015ad70 2023-10-16T11:25:09.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.867+0000 7fcfa6a2d640 1 --2- 172.21.15.84:0/494664497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa015ad70 0x7fcfa00b1c40 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800c7d0 tx=0x7fcf9800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:09.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.867+0000 7fcf96ffd640 1 -- 172.21.15.84:0/494664497 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9800a640 con 0x7fcfa015ad70 2023-10-16T11:25:09.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.867+0000 7fcf96ffd640 1 -- 172.21.15.84:0/494664497 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9800a7a0 con 0x7fcfa015ad70 2023-10-16T11:25:09.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.867+0000 7fcf96ffd640 1 -- 172.21.15.84:0/494664497 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9800ab90 con 0x7fcfa015ad70 2023-10-16T11:25:09.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.867+0000 7fcfa8490640 1 -- 172.21.15.84:0/494664497 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa00c7320 con 0x7fcfa015ad70 2023-10-16T11:25:09.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.867+0000 7fcfa8490640 1 -- 172.21.15.84:0/494664497 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa00c78e0 con 0x7fcfa015ad70 2023-10-16T11:25:09.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.868+0000 7fcfa8490640 1 -- 172.21.15.84:0/494664497 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa00a9c30 con 0x7fcfa015ad70 2023-10-16T11:25:09.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.869+0000 7fcf96ffd640 1 -- 172.21.15.84:0/494664497 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 140) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fcf9800acf0 con 0x7fcfa015ad70 2023-10-16T11:25:09.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.870+0000 7fcf96ffd640 1 -- 172.21.15.84:0/494664497 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 7418+0+0 (secure 0 0 0) 0x7fcf98088bb0 con 0x7fcfa015ad70 2023-10-16T11:25:09.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:09.876+0000 7fcf96ffd640 1 -- 172.21.15.84:0/494664497 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcfa00a9c30 con 0x7fcfa015ad70 2023-10-16T11:25:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.066+0000 7fcfa8490640 1 -- 172.21.15.84:0/494664497 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcfa00a9c30 con 0x7fcfa015ad70 2023-10-16T11:25:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.067+0000 7fcf96ffd640 1 -- 172.21.15.84:0/494664497 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v223) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcfa00a9c30 con 0x7fcfa015ad70 2023-10-16T11:25:10.069 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:25:10.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.071+0000 7fcfa8490640 1 -- 172.21.15.84:0/494664497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa015ad70 msgr2=0x7fcfa00b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:10.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.071+0000 7fcfa8490640 1 --2- 172.21.15.84:0/494664497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa015ad70 0x7fcfa00b1c40 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800c7d0 tx=0x7fcf9800cc90 comp rx=0 tx=0).stop 2023-10-16T11:25:10.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.071+0000 7fcfa8490640 1 -- 172.21.15.84:0/494664497 shutdown_connections 2023-10-16T11:25:10.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.071+0000 7fcfa8490640 1 --2- 172.21.15.84:0/494664497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa015ad70 0x7fcfa00b1c40 unknown :-1 s=CLOSED pgs=2802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.071+0000 7fcfa8490640 1 --2- 172.21.15.84:0/494664497 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfa00caac0 0x7fcfa01f0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.071+0000 7fcfa8490640 1 --2- 172.21.15.84:0/494664497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfa00c9ca0 0x7fcfa01f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.071+0000 7fcfa8490640 1 -- 172.21.15.84:0/494664497 >> 172.21.15.84:0/494664497 conn(0x7fcfa00ce9b0 msgr2=0x7fcfa015d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:10.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.071+0000 7fcfa8490640 1 -- 172.21.15.84:0/494664497 shutdown_connections 2023-10-16T11:25:10.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.072+0000 7fcfa8490640 1 -- 172.21.15.84:0/494664497 wait complete. 2023-10-16T11:25:10.090 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:25:10.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.312+0000 7f5dc1a02640 1 Processor -- start 2023-10-16T11:25:10.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.312+0000 7f5dc1a02640 1 -- start start 2023-10-16T11:25:10.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.314+0000 7f5dc1a02640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5dbc15d150 0x7f5dbc0ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:10.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.314+0000 7f5dc1a02640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5dbc06a4b0 0x7f5dbc160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:10.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.314+0000 7f5dc1a02640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dbc0ac890 con 0x7f5dbc160b90 2023-10-16T11:25:10.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.314+0000 7f5dc1a02640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dbc15d530 con 0x7f5dbc06a4b0 2023-10-16T11:25:10.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.314+0000 7f5dc1a02640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc161650 con 0x7f5dbc15d150 2023-10-16T11:25:10.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.315+0000 7f5dbb7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5dbc06a4b0 0x7f5dbc160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:10.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.315+0000 7f5dbb7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5dbc06a4b0 0x7f5dbc160fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40286/0 (socket says 172.21.15.84:40286) 2023-10-16T11:25:10.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.315+0000 7f5dbb7fe640 1 -- 172.21.15.84:0/139462411 learned_addr learned my addr 172.21.15.84:0/139462411 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:10.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.315+0000 7f5dbaffd640 1 --2- 172.21.15.84:0/139462411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5dbc15d150 0x7f5dbc0ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:10.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.315+0000 7f5dbb7fe640 1 -- 172.21.15.84:0/139462411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5dbc15d150 msgr2=0x7f5dbc0ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:10.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.315+0000 7f5dbb7fe640 1 --2- 172.21.15.84:0/139462411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5dbc15d150 0x7f5dbc0ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.315+0000 7f5dbb7fe640 1 -- 172.21.15.84:0/139462411 >> v1:172.21.15.84:6789/0 conn(0x7f5dbc160b90 legacy=0x7f5dbc069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:10.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.315+0000 7f5dbb7fe640 1 -- 172.21.15.84:0/139462411 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc161930 con 0x7f5dbc06a4b0 2023-10-16T11:25:10.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.316+0000 7f5dbb7fe640 1 --2- 172.21.15.84:0/139462411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dbc06a4b0 0x7f5dbc160fb0 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009e50 tx=0x7f5db402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37df088747da631 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:10.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.317+0000 7f5dba7fc640 1 -- 172.21.15.84:0/139462411 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db402caf0 con 0x7f5dbc06a4b0 2023-10-16T11:25:10.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.317+0000 7f5dba7fc640 1 -- 172.21.15.84:0/139462411 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db402cc50 con 0x7f5dbc06a4b0 2023-10-16T11:25:10.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.317+0000 7f5dba7fc640 1 -- 172.21.15.84:0/139462411 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db40343f0 con 0x7f5dbc06a4b0 2023-10-16T11:25:10.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.317+0000 7f5dc1a02640 1 -- 172.21.15.84:0/139462411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dbc06a4b0 msgr2=0x7f5dbc160fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:10.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.317+0000 7f5dc1a02640 1 --2- 172.21.15.84:0/139462411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dbc06a4b0 0x7f5dbc160fb0 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009e50 tx=0x7f5db402bf50 comp rx=0 tx=0).stop 2023-10-16T11:25:10.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.317+0000 7f5dc1a02640 1 -- 172.21.15.84:0/139462411 shutdown_connections 2023-10-16T11:25:10.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.317+0000 7f5dc1a02640 1 --2- 172.21.15.84:0/139462411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dbc06a4b0 0x7f5dbc160fb0 unknown :-1 s=CLOSED pgs=2804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.317+0000 7f5dc1a02640 1 --2- 172.21.15.84:0/139462411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5dbc15d150 0x7f5dbc0ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.317+0000 7f5dc1a02640 1 -- 172.21.15.84:0/139462411 >> 172.21.15.84:0/139462411 conn(0x7f5dbc0ce9b0 msgr2=0x7f5dbc0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:10.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.318+0000 7f5dc1a02640 1 -- 172.21.15.84:0/139462411 shutdown_connections 2023-10-16T11:25:10.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.318+0000 7f5dc1a02640 1 -- 172.21.15.84:0/139462411 wait complete. 2023-10-16T11:25:10.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.319+0000 7f5dc1a02640 1 Processor -- start 2023-10-16T11:25:10.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.319+0000 7f5dc1a02640 1 -- start start 2023-10-16T11:25:10.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.320+0000 7f5dc1a02640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5dbc06a4b0 0x7f5dbc1f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.320+0000 7f5dc1a02640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dbc15d150 0x7f5dbc0b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.320+0000 7f5dc1a02640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dbc160b90 0x7f5dbc0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.320+0000 7f5dc1a02640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0b3d30 con 0x7f5dbc160b90 2023-10-16T11:25:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.320+0000 7f5dc1a02640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0b47b0 con 0x7f5dbc15d150 2023-10-16T11:25:10.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.320+0000 7f5dc1a02640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc149c60 con 0x7f5dbc06a4b0 2023-10-16T11:25:10.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.321+0000 7f5dbb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dbc160b90 0x7f5dbc0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:10.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.321+0000 7f5dbb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dbc160b90 0x7f5dbc0b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40292/0 (socket says 172.21.15.84:40292) 2023-10-16T11:25:10.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.321+0000 7f5dbb7fe640 1 -- 172.21.15.84:0/3192321420 learned_addr learned my addr 172.21.15.84:0/3192321420 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:10.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.321+0000 7f5dbaffd640 1 --2- 172.21.15.84:0/3192321420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5dbc06a4b0 0x7f5dbc1f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:10.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.321+0000 7f5dbb7fe640 1 -- 172.21.15.84:0/3192321420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5dbc06a4b0 msgr2=0x7f5dbc1f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:10.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.321+0000 7f5dbb7fe640 1 --2- 172.21.15.84:0/3192321420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5dbc06a4b0 0x7f5dbc1f61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.321+0000 7f5dbb7fe640 1 -- 172.21.15.84:0/3192321420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dbc15d150 msgr2=0x7f5dbc0b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:10.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.321+0000 7f5dbb7fe640 1 --2- 172.21.15.84:0/3192321420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dbc15d150 0x7f5dbc0b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.321+0000 7f5dbb7fe640 1 -- 172.21.15.84:0/3192321420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db4009b30 con 0x7f5dbc160b90 2023-10-16T11:25:10.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.321+0000 7f5dbb7fe640 1 --2- 172.21.15.84:0/3192321420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dbc160b90 0x7f5dbc0b06c0 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7f5db4036040 tx=0x7f5db4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:10.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.322+0000 7f5db8ff9640 1 -- 172.21.15.84:0/3192321420 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4009510 con 0x7f5dbc160b90 2023-10-16T11:25:10.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.322+0000 7f5db8ff9640 1 -- 172.21.15.84:0/3192321420 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db403e450 con 0x7f5dbc160b90 2023-10-16T11:25:10.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.322+0000 7f5db8ff9640 1 -- 172.21.15.84:0/3192321420 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db403e800 con 0x7f5dbc160b90 2023-10-16T11:25:10.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.322+0000 7f5dc1a02640 1 -- 172.21.15.84:0/3192321420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dbc0b3f10 con 0x7f5dbc160b90 2023-10-16T11:25:10.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.322+0000 7f5dc1a02640 1 -- 172.21.15.84:0/3192321420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dbc0b44d0 con 0x7f5dbc160b90 2023-10-16T11:25:10.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.323+0000 7f5dc1a02640 1 -- 172.21.15.84:0/3192321420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dbc04e160 con 0x7f5dbc160b90 2023-10-16T11:25:10.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.324+0000 7f5db8ff9640 1 -- 172.21.15.84:0/3192321420 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 141) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f5db403e960 con 0x7f5dbc160b90 2023-10-16T11:25:10.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.325+0000 7f5db8ff9640 1 --2- 172.21.15.84:0/3192321420 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f5d9c06acc0 0x7f5d9c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:10.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.325+0000 7f5db8ff9640 1 -- 172.21.15.84:0/3192321420 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 7418+0+0 (secure 0 0 0) 0x7f5db40ac5f0 con 0x7f5dbc160b90 2023-10-16T11:25:10.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.325+0000 7f5dbaffd640 1 --2- 172.21.15.84:0/3192321420 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f5d9c06acc0 0x7f5d9c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:10.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.326+0000 7f5dbaffd640 1 --2- 172.21.15.84:0/3192321420 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f5d9c06acc0 0x7f5d9c06d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5dac006fd0 tx=0x7f5dac008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:10.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.330+0000 7f5db8ff9640 1 -- 172.21.15.84:0/3192321420 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5db4077330 con 0x7f5dbc160b90 2023-10-16T11:25:10.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.522+0000 7f5dc1a02640 1 -- 172.21.15.84:0/3192321420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5dbc0ade10 con 0x7f5dbc160b90 2023-10-16T11:25:10.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.522+0000 7f5db8ff9640 1 -- 172.21.15.84:0/3192321420 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v223) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5db407a950 con 0x7f5dbc160b90 2023-10-16T11:25:10.524 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:25:10.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.525+0000 7f5dc1a02640 1 -- 172.21.15.84:0/3192321420 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f5d9c06acc0 msgr2=0x7f5d9c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:10.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.525+0000 7f5dc1a02640 1 --2- 172.21.15.84:0/3192321420 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f5d9c06acc0 0x7f5d9c06d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5dac006fd0 tx=0x7f5dac008040 comp rx=0 tx=0).stop 2023-10-16T11:25:10.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.525+0000 7f5dc1a02640 1 -- 172.21.15.84:0/3192321420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dbc160b90 msgr2=0x7f5dbc0b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:10.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.525+0000 7f5dc1a02640 1 --2- 172.21.15.84:0/3192321420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dbc160b90 0x7f5dbc0b06c0 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7f5db4036040 tx=0x7f5db4004a40 comp rx=0 tx=0).stop 2023-10-16T11:25:10.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.526+0000 7f5dc1a02640 1 -- 172.21.15.84:0/3192321420 shutdown_connections 2023-10-16T11:25:10.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.526+0000 7f5dc1a02640 1 --2- 172.21.15.84:0/3192321420 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f5d9c06acc0 0x7f5d9c06d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.526+0000 7f5dc1a02640 1 --2- 172.21.15.84:0/3192321420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dbc160b90 0x7f5dbc0b06c0 unknown :-1 s=CLOSED pgs=2805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.526+0000 7f5dc1a02640 1 --2- 172.21.15.84:0/3192321420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dbc15d150 0x7f5dbc0b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.526+0000 7f5dc1a02640 1 --2- 172.21.15.84:0/3192321420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5dbc06a4b0 0x7f5dbc1f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.526+0000 7f5dc1a02640 1 -- 172.21.15.84:0/3192321420 >> 172.21.15.84:0/3192321420 conn(0x7f5dbc0ce9b0 msgr2=0x7f5dbc06f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:10.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.526+0000 7f5dc1a02640 1 -- 172.21.15.84:0/3192321420 shutdown_connections 2023-10-16T11:25:10.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.526+0000 7f5dc1a02640 1 -- 172.21.15.84:0/3192321420 wait complete. 2023-10-16T11:25:10.546 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:25:10.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.765+0000 7ff41359e640 1 Processor -- start 2023-10-16T11:25:10.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.765+0000 7ff41359e640 1 -- start start 2023-10-16T11:25:10.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.767+0000 7ff41359e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff40c15d150 0x7ff40c0ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:10.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.767+0000 7ff41359e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff40c06a4b0 0x7ff40c160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:10.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.767+0000 7ff41359e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40c0ac890 con 0x7ff40c160b90 2023-10-16T11:25:10.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.767+0000 7ff41359e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff40c15d530 con 0x7ff40c15d150 2023-10-16T11:25:10.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.767+0000 7ff41359e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff40c161650 con 0x7ff40c06a4b0 2023-10-16T11:25:10.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.767+0000 7ff41133a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff40c15d150 0x7ff40c0ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:10.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.767+0000 7ff41133a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff40c15d150 0x7ff40c0ad350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40298/0 (socket says 172.21.15.84:40298) 2023-10-16T11:25:10.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.767+0000 7ff41133a640 1 -- 172.21.15.84:0/3860345541 learned_addr learned my addr 172.21.15.84:0/3860345541 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:10.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.767+0000 7ff41133a640 1 -- 172.21.15.84:0/3860345541 >> v2:172.21.15.157:3300/0 conn(0x7ff40c06a4b0 msgr2=0x7ff40c160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:10.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.767+0000 7ff41133a640 1 --2- 172.21.15.84:0/3860345541 >> v2:172.21.15.157:3300/0 conn(0x7ff40c06a4b0 0x7ff40c160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.767+0000 7ff41133a640 1 -- 172.21.15.84:0/3860345541 >> v1:172.21.15.157:6789/0 conn(0x7ff40c160b90 legacy=0x7ff40c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:10.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.767+0000 7ff41133a640 1 -- 172.21.15.84:0/3860345541 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c161930 con 0x7ff40c15d150 2023-10-16T11:25:10.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.769+0000 7ff41133a640 1 --2- 172.21.15.84:0/3860345541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff40c15d150 0x7ff40c0ad350 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0adde0 tx=0x7ff40802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45525a8dae25e31b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:10.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.769+0000 7ff403fff640 1 -- 172.21.15.84:0/3860345541 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4080317e0 con 0x7ff40c15d150 2023-10-16T11:25:10.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.769+0000 7ff403fff640 1 -- 172.21.15.84:0/3860345541 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff408031940 con 0x7ff40c15d150 2023-10-16T11:25:10.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.769+0000 7ff403fff640 1 -- 172.21.15.84:0/3860345541 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408031cd0 con 0x7ff40c15d150 2023-10-16T11:25:10.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.770+0000 7ff41359e640 1 -- 172.21.15.84:0/3860345541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff40c15d150 msgr2=0x7ff40c0ad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:10.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.770+0000 7ff41359e640 1 --2- 172.21.15.84:0/3860345541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff40c15d150 0x7ff40c0ad350 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0adde0 tx=0x7ff40802b880 comp rx=0 tx=0).stop 2023-10-16T11:25:10.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.770+0000 7ff41359e640 1 -- 172.21.15.84:0/3860345541 shutdown_connections 2023-10-16T11:25:10.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.770+0000 7ff41359e640 1 --2- 172.21.15.84:0/3860345541 >> v2:172.21.15.157:3300/0 conn(0x7ff40c06a4b0 0x7ff40c160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.770+0000 7ff41359e640 1 --2- 172.21.15.84:0/3860345541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff40c15d150 0x7ff40c0ad350 unknown :-1 s=CLOSED pgs=2806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.770+0000 7ff41359e640 1 -- 172.21.15.84:0/3860345541 >> 172.21.15.84:0/3860345541 conn(0x7ff40c0ce9b0 msgr2=0x7ff40c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:10.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.770+0000 7ff41359e640 1 -- 172.21.15.84:0/3860345541 shutdown_connections 2023-10-16T11:25:10.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.770+0000 7ff41359e640 1 -- 172.21.15.84:0/3860345541 wait complete. 2023-10-16T11:25:10.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.772+0000 7ff41359e640 1 Processor -- start 2023-10-16T11:25:10.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.772+0000 7ff41359e640 1 -- start start 2023-10-16T11:25:10.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.773+0000 7ff41359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c06a4b0 0x7ff40c1f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:10.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.773+0000 7ff41359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff40c15d150 0x7ff40c155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:10.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.773+0000 7ff41359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff40c160b90 0x7ff40c1e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:10.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.773+0000 7ff41359e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c0b0fd0 con 0x7ff40c160b90 2023-10-16T11:25:10.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.773+0000 7ff41359e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c1f6b60 con 0x7ff40c06a4b0 2023-10-16T11:25:10.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.773+0000 7ff41359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c1e6ac0 con 0x7ff40c15d150 2023-10-16T11:25:10.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.774+0000 7ff411b3b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff40c160b90 0x7ff40c1e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:10.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.774+0000 7ff411b3b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff40c160b90 0x7ff40c1e7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40304/0 (socket says 172.21.15.84:40304) 2023-10-16T11:25:10.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.774+0000 7ff411b3b640 1 -- 172.21.15.84:0/4251601675 learned_addr learned my addr 172.21.15.84:0/4251601675 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:10.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.774+0000 7ff41133a640 1 --2- 172.21.15.84:0/4251601675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c06a4b0 0x7ff40c1f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:10.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.774+0000 7ff410b39640 1 --2- 172.21.15.84:0/4251601675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff40c15d150 0x7ff40c155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:10.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.774+0000 7ff411b3b640 1 -- 172.21.15.84:0/4251601675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff40c15d150 msgr2=0x7ff40c155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:10.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.774+0000 7ff411b3b640 1 --2- 172.21.15.84:0/4251601675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff40c15d150 0x7ff40c155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.774+0000 7ff411b3b640 1 -- 172.21.15.84:0/4251601675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c06a4b0 msgr2=0x7ff40c1f6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:10.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.774+0000 7ff411b3b640 1 --2- 172.21.15.84:0/4251601675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c06a4b0 0x7ff40c1f6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.774+0000 7ff411b3b640 1 -- 172.21.15.84:0/4251601675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4080094d0 con 0x7ff40c160b90 2023-10-16T11:25:10.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.774+0000 7ff41133a640 1 --2- 172.21.15.84:0/4251601675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c06a4b0 0x7ff40c1f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:10.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.774+0000 7ff411b3b640 1 --2- 172.21.15.84:0/4251601675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff40c160b90 0x7ff40c1e7b60 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7ff40400b970 tx=0x7ff40400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:10.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.775+0000 7ff401ffb640 1 -- 172.21.15.84:0/4251601675 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40400cb40 con 0x7ff40c160b90 2023-10-16T11:25:10.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.775+0000 7ff401ffb640 1 -- 172.21.15.84:0/4251601675 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40400ccc0 con 0x7ff40c160b90 2023-10-16T11:25:10.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.775+0000 7ff401ffb640 1 -- 172.21.15.84:0/4251601675 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff404014680 con 0x7ff40c160b90 2023-10-16T11:25:10.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.775+0000 7ff41359e640 1 -- 172.21.15.84:0/4251601675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40c1e6da0 con 0x7ff40c160b90 2023-10-16T11:25:10.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.775+0000 7ff41359e640 1 -- 172.21.15.84:0/4251601675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40c1e7360 con 0x7ff40c160b90 2023-10-16T11:25:10.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.776+0000 7ff41359e640 1 -- 172.21.15.84:0/4251601675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40c04e160 con 0x7ff40c160b90 2023-10-16T11:25:10.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.777+0000 7ff401ffb640 1 -- 172.21.15.84:0/4251601675 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 141) v1 ==== 89067+0+0 (secure 0 0 0) 0x7ff40401d3b0 con 0x7ff40c160b90 2023-10-16T11:25:10.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.778+0000 7ff401ffb640 1 --2- 172.21.15.84:0/4251601675 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7ff3f006acc0 0x7ff3f006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:10.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.778+0000 7ff401ffb640 1 -- 172.21.15.84:0/4251601675 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 7418+0+0 (secure 0 0 0) 0x7ff404089890 con 0x7ff40c160b90 2023-10-16T11:25:10.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.778+0000 7ff41133a640 1 --2- 172.21.15.84:0/4251601675 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7ff3f006acc0 0x7ff3f006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:10.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.779+0000 7ff41133a640 1 --2- 172.21.15.84:0/4251601675 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7ff3f006acc0 0x7ff3f006d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff408036000 tx=0x7ff40803d040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:10.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.783+0000 7ff401ffb640 1 -- 172.21.15.84:0/4251601675 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff4040545d0 con 0x7ff40c160b90 2023-10-16T11:25:10.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.977+0000 7ff41359e640 1 -- 172.21.15.84:0/4251601675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff40c0ade10 con 0x7ff40c160b90 2023-10-16T11:25:10.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.977+0000 7ff401ffb640 1 -- 172.21.15.84:0/4251601675 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v223) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff4040190a0 con 0x7ff40c160b90 2023-10-16T11:25:10.979 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:25:10.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.980+0000 7ff41359e640 1 -- 172.21.15.84:0/4251601675 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7ff3f006acc0 msgr2=0x7ff3f006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:10.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.980+0000 7ff41359e640 1 --2- 172.21.15.84:0/4251601675 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7ff3f006acc0 0x7ff3f006d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff408036000 tx=0x7ff40803d040 comp rx=0 tx=0).stop 2023-10-16T11:25:10.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.980+0000 7ff41359e640 1 -- 172.21.15.84:0/4251601675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff40c160b90 msgr2=0x7ff40c1e7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:10.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.980+0000 7ff41359e640 1 --2- 172.21.15.84:0/4251601675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff40c160b90 0x7ff40c1e7b60 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7ff40400b970 tx=0x7ff40400be30 comp rx=0 tx=0).stop 2023-10-16T11:25:10.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.980+0000 7ff41359e640 1 -- 172.21.15.84:0/4251601675 shutdown_connections 2023-10-16T11:25:10.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.980+0000 7ff41359e640 1 --2- 172.21.15.84:0/4251601675 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7ff3f006acc0 0x7ff3f006d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.980+0000 7ff41359e640 1 --2- 172.21.15.84:0/4251601675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff40c160b90 0x7ff40c1e7b60 unknown :-1 s=CLOSED pgs=2807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.980+0000 7ff41359e640 1 --2- 172.21.15.84:0/4251601675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff40c15d150 0x7ff40c155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.980+0000 7ff41359e640 1 --2- 172.21.15.84:0/4251601675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c06a4b0 0x7ff40c1f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:10.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.980+0000 7ff41359e640 1 -- 172.21.15.84:0/4251601675 >> 172.21.15.84:0/4251601675 conn(0x7ff40c0ce9b0 msgr2=0x7ff40c0ca840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:10.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.981+0000 7ff41359e640 1 -- 172.21.15.84:0/4251601675 shutdown_connections 2023-10-16T11:25:10.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:10.981+0000 7ff41359e640 1 -- 172.21.15.84:0/4251601675 wait complete. 2023-10-16T11:25:11.000 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-16T11:25:11.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.201+0000 7f5299501640 1 Processor -- start 2023-10-16T11:25:11.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.201+0000 7f5299501640 1 -- start start 2023-10-16T11:25:11.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.202+0000 7f5299501640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f529415ad70 0x7f5294069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:11.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.202+0000 7f5299501640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52940c9ca0 0x7f529406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:11.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.202+0000 7f5299501640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52940cb030 con 0x7f52940ca3b0 2023-10-16T11:25:11.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.202+0000 7f5299501640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52940c91c0 con 0x7f529415ad70 2023-10-16T11:25:11.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.202+0000 7f5299501640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52940c94a0 con 0x7f52940c9ca0 2023-10-16T11:25:11.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.202+0000 7f52937fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f52940ca3b0 0x7f52940ca840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36000/0 (socket says 172.21.15.84:36000) 2023-10-16T11:25:11.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.202+0000 7f52937fe640 1 -- 172.21.15.84:0/1037046978 learned_addr learned my addr 172.21.15.84:0/1037046978 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:11.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.203+0000 7f5292ffd640 1 --2- 172.21.15.84:0/1037046978 >> v2:172.21.15.157:3300/0 conn(0x7f529415ad70 0x7f5294069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:11.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.203+0000 7f52927fc640 1 --2- 172.21.15.84:0/1037046978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52940c9ca0 0x7f529406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:11.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.203+0000 7f5291ffb640 1 -- 172.21.15.84:0/1037046978 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2544764603 0 0) 0x7f52940cb030 con 0x7f52940ca3b0 2023-10-16T11:25:11.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.203+0000 7f5291ffb640 1 -- 172.21.15.84:0/1037046978 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5280003520 con 0x7f52940ca3b0 2023-10-16T11:25:11.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.203+0000 7f5291ffb640 1 -- 172.21.15.84:0/1037046978 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1262293098 0 0) 0x7f5280003520 con 0x7f52940ca3b0 2023-10-16T11:25:11.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.203+0000 7f5291ffb640 1 -- 172.21.15.84:0/1037046978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52940c9ca0 msgr2=0x7f529406a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:11.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.203+0000 7f5291ffb640 1 --2- 172.21.15.84:0/1037046978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52940c9ca0 0x7f529406a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.203+0000 7f5291ffb640 1 -- 172.21.15.84:0/1037046978 >> v2:172.21.15.157:3300/0 conn(0x7f529415ad70 msgr2=0x7f5294069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:11.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.203+0000 7f5291ffb640 1 --2- 172.21.15.84:0/1037046978 >> v2:172.21.15.157:3300/0 conn(0x7f529415ad70 0x7f5294069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.203+0000 7f5291ffb640 1 -- 172.21.15.84:0/1037046978 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52940c9780 con 0x7f52940ca3b0 2023-10-16T11:25:11.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.203+0000 7f5291ffb640 1 -- 172.21.15.84:0/1037046978 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f528c001670 con 0x7f52940ca3b0 2023-10-16T11:25:11.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.203+0000 7f5291ffb640 1 -- 172.21.15.84:0/1037046978 >> v1:172.21.15.84:6789/0 conn(0x7f52940ca3b0 legacy=0x7f52940ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:11.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.203+0000 7f5291ffb640 1 --2- 172.21.15.84:0/1037046978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52940c9ca0 0x7f5280003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:11.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.203+0000 7f5291ffb640 1 --2- 172.21.15.84:0/1037046978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5280003ef0 0x7f52800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:11.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.203+0000 7f5291ffb640 1 --2- 172.21.15.84:0/1037046978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5280004d00 0x7f5280005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:11.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.203+0000 7f5291ffb640 1 -- 172.21.15.84:0/1037046978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52940c91c0 con 0x7f5280004d00 2023-10-16T11:25:11.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.203+0000 7f5291ffb640 1 -- 172.21.15.84:0/1037046978 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52940c94a0 con 0x7f5280003ef0 2023-10-16T11:25:11.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.203+0000 7f5291ffb640 1 -- 172.21.15.84:0/1037046978 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528000b980 con 0x7f52940c9ca0 2023-10-16T11:25:11.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.204+0000 7f52937fe640 1 --2- 172.21.15.84:0/1037046978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5280004d00 0x7f5280005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:11.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.204+0000 7f5292ffd640 1 --2- 172.21.15.84:0/1037046978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52940c9ca0 0x7f5280003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:11.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.204+0000 7f52927fc640 1 --2- 172.21.15.84:0/1037046978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5280003ef0 0x7f52800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:11.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.204+0000 7f52937fe640 1 -- 172.21.15.84:0/1037046978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52940c9ca0 msgr2=0x7f5280003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:11.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.204+0000 7f52937fe640 1 --2- 172.21.15.84:0/1037046978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52940c9ca0 0x7f5280003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.204+0000 7f52937fe640 1 -- 172.21.15.84:0/1037046978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5280003ef0 msgr2=0x7f52800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:11.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.204+0000 7f52937fe640 1 --2- 172.21.15.84:0/1037046978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5280003ef0 0x7f52800047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.204+0000 7f52937fe640 1 -- 172.21.15.84:0/1037046978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5280003520 con 0x7f5280004d00 2023-10-16T11:25:11.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.205+0000 7f52937fe640 1 --2- 172.21.15.84:0/1037046978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5280004d00 0x7f5280005180 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7f528c00c240 tx=0x7f52800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555af368b6da7581 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:11.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.206+0000 7f5291ffb640 1 -- 172.21.15.84:0/1037046978 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c02cae0 con 0x7f5280004d00 2023-10-16T11:25:11.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.206+0000 7f5291ffb640 1 -- 172.21.15.84:0/1037046978 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528c02cc40 con 0x7f5280004d00 2023-10-16T11:25:11.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.206+0000 7f5291ffb640 1 -- 172.21.15.84:0/1037046978 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c005af0 con 0x7f5280004d00 2023-10-16T11:25:11.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.206+0000 7f5299501640 1 -- 172.21.15.84:0/1037046978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5280004d00 msgr2=0x7f5280005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:11.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.206+0000 7f5299501640 1 --2- 172.21.15.84:0/1037046978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5280004d00 0x7f5280005180 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7f528c00c240 tx=0x7f52800057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.206+0000 7f52937fe640 1 -- 172.21.15.84:0/1037046978 reap_dead start 2023-10-16T11:25:11.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.206+0000 7f5299501640 1 -- 172.21.15.84:0/1037046978 shutdown_connections 2023-10-16T11:25:11.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.206+0000 7f5299501640 1 -- 172.21.15.84:0/1037046978 >> 172.21.15.84:0/1037046978 conn(0x7f52940ce9b0 msgr2=0x7f52940cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:11.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.206+0000 7f5299501640 1 -- 172.21.15.84:0/1037046978 shutdown_connections 2023-10-16T11:25:11.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.206+0000 7f5299501640 1 -- 172.21.15.84:0/1037046978 wait complete. 2023-10-16T11:25:11.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.208+0000 7f5299501640 1 Processor -- start 2023-10-16T11:25:11.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.208+0000 7f5299501640 1 -- start start 2023-10-16T11:25:11.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.209+0000 7f5299501640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5280004d00 0x7f52940c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:11.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.209+0000 7f5299501640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f529415dfc0 0x7f52940cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:11.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.209+0000 7f5299501640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5294069ec0 0x7f52940adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:11.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.209+0000 7f5299501640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52941f2f30 con 0x7f529415dfc0 2023-10-16T11:25:11.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.209+0000 7f5299501640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52941f2a00 con 0x7f5294069ec0 2023-10-16T11:25:11.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.209+0000 7f5299501640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52941f2c50 con 0x7f5280004d00 2023-10-16T11:25:11.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.210+0000 7f52927fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f529415dfc0 0x7f52940cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:11.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.210+0000 7f52927fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f529415dfc0 0x7f52940cc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40320/0 (socket says 172.21.15.84:40320) 2023-10-16T11:25:11.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.210+0000 7f52927fc640 1 -- 172.21.15.84:0/3074906453 learned_addr learned my addr 172.21.15.84:0/3074906453 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:11.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.210+0000 7f5292ffd640 1 --2- 172.21.15.84:0/3074906453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5280004d00 0x7f52940c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:11.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.210+0000 7f52937fe640 1 --2- 172.21.15.84:0/3074906453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5294069ec0 0x7f52940adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:11.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.210+0000 7f52927fc640 1 -- 172.21.15.84:0/3074906453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5280004d00 msgr2=0x7f52940c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:11.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.210+0000 7f52927fc640 1 --2- 172.21.15.84:0/3074906453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5280004d00 0x7f52940c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.210+0000 7f52927fc640 1 -- 172.21.15.84:0/3074906453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5294069ec0 msgr2=0x7f52940adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:11.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.210+0000 7f52927fc640 1 --2- 172.21.15.84:0/3074906453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5294069ec0 0x7f52940adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.210+0000 7f52927fc640 1 -- 172.21.15.84:0/3074906453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52940aca30 con 0x7f529415dfc0 2023-10-16T11:25:11.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.210+0000 7f5292ffd640 1 --2- 172.21.15.84:0/3074906453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5280004d00 0x7f52940c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:11.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.210+0000 7f52927fc640 1 --2- 172.21.15.84:0/3074906453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f529415dfc0 0x7f52940cc580 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7f527c011600 tx=0x7f527c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:11.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.211+0000 7f528b7fe640 1 -- 172.21.15.84:0/3074906453 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c00f920 con 0x7f529415dfc0 2023-10-16T11:25:11.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.211+0000 7f528b7fe640 1 -- 172.21.15.84:0/3074906453 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527c00fa80 con 0x7f529415dfc0 2023-10-16T11:25:11.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.211+0000 7f528b7fe640 1 -- 172.21.15.84:0/3074906453 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c00fdf0 con 0x7f529415dfc0 2023-10-16T11:25:11.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.211+0000 7f5299501640 1 -- 172.21.15.84:0/3074906453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52940acd10 con 0x7f529415dfc0 2023-10-16T11:25:11.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.211+0000 7f5299501640 1 -- 172.21.15.84:0/3074906453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52940ccca0 con 0x7f529415dfc0 2023-10-16T11:25:11.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.212+0000 7f5299501640 1 -- 172.21.15.84:0/3074906453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52940ac890 con 0x7f529415dfc0 2023-10-16T11:25:11.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.213+0000 7f528b7fe640 1 -- 172.21.15.84:0/3074906453 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 141) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f527c015460 con 0x7f529415dfc0 2023-10-16T11:25:11.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.214+0000 7f528b7fe640 1 --2- 172.21.15.84:0/3074906453 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f527406acf0 0x7f527406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:11.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.214+0000 7f528b7fe640 1 -- 172.21.15.84:0/3074906453 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 7418+0+0 (secure 0 0 0) 0x7f527c089e60 con 0x7f529415dfc0 2023-10-16T11:25:11.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.214+0000 7f5292ffd640 1 --2- 172.21.15.84:0/3074906453 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f527406acf0 0x7f527406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:11.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.215+0000 7f5292ffd640 1 --2- 172.21.15.84:0/3074906453 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f527406acf0 0x7f527406d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5284007440 tx=0x7f528400b040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:11.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.218+0000 7f528b7fe640 1 -- 172.21.15.84:0/3074906453 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f52940ac890 con 0x7f529415dfc0 2023-10-16T11:25:11.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.418+0000 7f5299501640 1 -- 172.21.15.84:0/3074906453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52940ac890 con 0x7f529415dfc0 2023-10-16T11:25:11.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.419+0000 7f528b7fe640 1 -- 172.21.15.84:0/3074906453 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v223) v1 ==== 107+0+11 (secure 0 0 0) 0x7f52940ac890 con 0x7f529415dfc0 2023-10-16T11:25:11.421 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:25:11.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.422+0000 7f5299501640 1 -- 172.21.15.84:0/3074906453 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f527406acf0 msgr2=0x7f527406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:11.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.422+0000 7f5299501640 1 --2- 172.21.15.84:0/3074906453 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f527406acf0 0x7f527406d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5284007440 tx=0x7f528400b040 comp rx=0 tx=0).stop 2023-10-16T11:25:11.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.422+0000 7f5299501640 1 -- 172.21.15.84:0/3074906453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f529415dfc0 msgr2=0x7f52940cc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:11.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.422+0000 7f5299501640 1 --2- 172.21.15.84:0/3074906453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f529415dfc0 0x7f52940cc580 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7f527c011600 tx=0x7f527c011ac0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.422+0000 7f5299501640 1 -- 172.21.15.84:0/3074906453 shutdown_connections 2023-10-16T11:25:11.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.422+0000 7f5299501640 1 --2- 172.21.15.84:0/3074906453 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f527406acf0 0x7f527406d1b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.422+0000 7f5299501640 1 --2- 172.21.15.84:0/3074906453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5294069ec0 0x7f52940adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.422+0000 7f5299501640 1 --2- 172.21.15.84:0/3074906453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f529415dfc0 0x7f52940cc580 unknown :-1 s=CLOSED pgs=2810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.422+0000 7f5299501640 1 --2- 172.21.15.84:0/3074906453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5280004d00 0x7f52940c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.422+0000 7f5299501640 1 -- 172.21.15.84:0/3074906453 >> 172.21.15.84:0/3074906453 conn(0x7f52940ce9b0 msgr2=0x7f529406b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:11.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.422+0000 7f5299501640 1 -- 172.21.15.84:0/3074906453 shutdown_connections 2023-10-16T11:25:11.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.422+0000 7f5299501640 1 -- 172.21.15.84:0/3074906453 wait complete. 2023-10-16T11:25:11.441 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-16T11:25:11.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.652+0000 7f2a12aa2640 1 Processor -- start 2023-10-16T11:25:11.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.652+0000 7f2a12aa2640 1 -- start start 2023-10-16T11:25:11.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.653+0000 7f2a12aa2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2a0c15d150 0x7f2a0c0ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:11.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.653+0000 7f2a12aa2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a0c160b90 0x7f2a0c160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:11.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.654+0000 7f2a12aa2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2a0c069da0 0x7f2a0c06a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:11.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.654+0000 7f2a12aa2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a0c15d530 con 0x7f2a0c069da0 2023-10-16T11:25:11.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.654+0000 7f2a12aa2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a0c06a6f0 con 0x7f2a0c15d150 2023-10-16T11:25:11.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.654+0000 7f2a12aa2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c0ac890 con 0x7f2a0c160b90 2023-10-16T11:25:11.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.654+0000 7f2a1103f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2a0c069da0 0x7f2a0c06a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:11.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.654+0000 7f2a1103f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2a0c069da0 0x7f2a0c06a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40348/0 (socket says 172.21.15.84:40348) 2023-10-16T11:25:11.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.654+0000 7f2a1103f640 1 -- 172.21.15.84:0/2627639893 learned_addr learned my addr 172.21.15.84:0/2627639893 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:11.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.654+0000 7f2a1083e640 1 --2- 172.21.15.84:0/2627639893 >> v2:172.21.15.157:3300/0 conn(0x7f2a0c15d150 0x7f2a0c0ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:11.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.654+0000 7f2a0bfff640 1 --2- 172.21.15.84:0/2627639893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a0c160b90 0x7f2a0c160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:11.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.654+0000 7f2a1103f640 1 -- 172.21.15.84:0/2627639893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a0c160b90 msgr2=0x7f2a0c160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:11.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.654+0000 7f2a1103f640 1 --2- 172.21.15.84:0/2627639893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a0c160b90 0x7f2a0c160fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.654+0000 7f2a1103f640 1 -- 172.21.15.84:0/2627639893 >> v2:172.21.15.157:3300/0 conn(0x7f2a0c15d150 msgr2=0x7f2a0c0ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:11.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.654+0000 7f2a1103f640 1 --2- 172.21.15.84:0/2627639893 >> v2:172.21.15.157:3300/0 conn(0x7f2a0c15d150 0x7f2a0c0ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.655+0000 7f2a1103f640 1 -- 172.21.15.84:0/2627639893 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c161650 con 0x7f2a0c069da0 2023-10-16T11:25:11.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.656+0000 7f2a1103f640 1 --2- 172.21.15.84:0/2627639893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a0c069da0 0x7f2a0c06a1b0 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0b15a0 tx=0x7f2a0402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b2bb88dda4b937 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:11.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.656+0000 7f2a0b7fe640 1 -- 172.21.15.84:0/2627639893 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0402ca80 con 0x7f2a0c069da0 2023-10-16T11:25:11.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.656+0000 7f2a0b7fe640 1 -- 172.21.15.84:0/2627639893 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0402cbe0 con 0x7f2a0c069da0 2023-10-16T11:25:11.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.656+0000 7f2a0b7fe640 1 -- 172.21.15.84:0/2627639893 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a040343f0 con 0x7f2a0c069da0 2023-10-16T11:25:11.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.656+0000 7f2a12aa2640 1 -- 172.21.15.84:0/2627639893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a0c069da0 msgr2=0x7f2a0c06a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:11.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.656+0000 7f2a12aa2640 1 --2- 172.21.15.84:0/2627639893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a0c069da0 0x7f2a0c06a1b0 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0b15a0 tx=0x7f2a0402bee0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.657+0000 7f2a12aa2640 1 -- 172.21.15.84:0/2627639893 shutdown_connections 2023-10-16T11:25:11.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.657+0000 7f2a12aa2640 1 --2- 172.21.15.84:0/2627639893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a0c069da0 0x7f2a0c06a1b0 unknown :-1 s=CLOSED pgs=2813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.657+0000 7f2a12aa2640 1 --2- 172.21.15.84:0/2627639893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a0c160b90 0x7f2a0c160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.657+0000 7f2a12aa2640 1 --2- 172.21.15.84:0/2627639893 >> v2:172.21.15.157:3300/0 conn(0x7f2a0c15d150 0x7f2a0c0ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.657+0000 7f2a12aa2640 1 -- 172.21.15.84:0/2627639893 >> 172.21.15.84:0/2627639893 conn(0x7f2a0c0ce9b0 msgr2=0x7f2a0c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:11.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.657+0000 7f2a12aa2640 1 -- 172.21.15.84:0/2627639893 shutdown_connections 2023-10-16T11:25:11.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.657+0000 7f2a12aa2640 1 -- 172.21.15.84:0/2627639893 wait complete. 2023-10-16T11:25:11.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.658+0000 7f2a12aa2640 1 Processor -- start 2023-10-16T11:25:11.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.659+0000 7f2a12aa2640 1 -- start start 2023-10-16T11:25:11.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.660+0000 7f2a12aa2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a0c069da0 0x7f2a0c1f3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:11.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.660+0000 7f2a12aa2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a0c15d150 0x7f2a0c159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:11.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.660+0000 7f2a12aa2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a0c160b90 0x7f2a0c15efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:11.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.660+0000 7f2a12aa2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c1f3b90 con 0x7f2a0c15d150 2023-10-16T11:25:11.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.660+0000 7f2a12aa2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c160090 con 0x7f2a0c069da0 2023-10-16T11:25:11.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.660+0000 7f2a12aa2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c160330 con 0x7f2a0c160b90 2023-10-16T11:25:11.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.660+0000 7f2a0bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a0c15d150 0x7f2a0c159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:11.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.660+0000 7f2a0bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a0c15d150 0x7f2a0c159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40352/0 (socket says 172.21.15.84:40352) 2023-10-16T11:25:11.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.660+0000 7f2a0bfff640 1 -- 172.21.15.84:0/3966785238 learned_addr learned my addr 172.21.15.84:0/3966785238 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:11.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.660+0000 7f2a1083e640 1 --2- 172.21.15.84:0/3966785238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a0c069da0 0x7f2a0c1f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:11.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.660+0000 7f2a0bfff640 1 -- 172.21.15.84:0/3966785238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a0c160b90 msgr2=0x7f2a0c15efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:11.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.660+0000 7f2a0bfff640 1 --2- 172.21.15.84:0/3966785238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a0c160b90 0x7f2a0c15efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.660+0000 7f2a0bfff640 1 -- 172.21.15.84:0/3966785238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a0c069da0 msgr2=0x7f2a0c1f3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:11.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.660+0000 7f2a0bfff640 1 --2- 172.21.15.84:0/3966785238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a0c069da0 0x7f2a0c1f3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.660+0000 7f2a0bfff640 1 -- 172.21.15.84:0/3966785238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a04009b30 con 0x7f2a0c15d150 2023-10-16T11:25:11.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.661+0000 7f2a0bfff640 1 --2- 172.21.15.84:0/3966785238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a0c15d150 0x7f2a0c159880 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7f29f800d800 tx=0x7f29f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:11.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.661+0000 7f2a097fa640 1 -- 172.21.15.84:0/3966785238 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f800b6c0 con 0x7f2a0c15d150 2023-10-16T11:25:11.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.661+0000 7f2a097fa640 1 -- 172.21.15.84:0/3966785238 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f800b820 con 0x7f2a0c15d150 2023-10-16T11:25:11.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.661+0000 7f2a097fa640 1 -- 172.21.15.84:0/3966785238 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f800bbb0 con 0x7f2a0c15d150 2023-10-16T11:25:11.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.661+0000 7f2a12aa2640 1 -- 172.21.15.84:0/3966785238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0c160610 con 0x7f2a0c15d150 2023-10-16T11:25:11.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.661+0000 7f2a12aa2640 1 -- 172.21.15.84:0/3966785238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0c15eab0 con 0x7f2a0c15d150 2023-10-16T11:25:11.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.662+0000 7f2a12aa2640 1 -- 172.21.15.84:0/3966785238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a0c15ec40 con 0x7f2a0c15d150 2023-10-16T11:25:11.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.663+0000 7f2a097fa640 1 -- 172.21.15.84:0/3966785238 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 141) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f29f800bd10 con 0x7f2a0c15d150 2023-10-16T11:25:11.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.664+0000 7f2a097fa640 1 --2- 172.21.15.84:0/3966785238 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f29f006acc0 0x7f29f006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:11.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.664+0000 7f2a097fa640 1 -- 172.21.15.84:0/3966785238 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 7418+0+0 (secure 0 0 0) 0x7f29f8089f50 con 0x7f2a0c15d150 2023-10-16T11:25:11.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.664+0000 7f2a1083e640 1 --2- 172.21.15.84:0/3966785238 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f29f006acc0 0x7f29f006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:11.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.665+0000 7f2a1083e640 1 --2- 172.21.15.84:0/3966785238 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f29f006acc0 0x7f29f006d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2a00006fd0 tx=0x7f2a00008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:11.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.669+0000 7f2a097fa640 1 -- 172.21.15.84:0/3966785238 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f29f8054bc0 con 0x7f2a0c15d150 2023-10-16T11:25:11.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.859+0000 7f2a12aa2640 1 -- 172.21.15.84:0/3966785238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f2a0c04e160 con 0x7f2a0c15d150 2023-10-16T11:25:11.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.860+0000 7f2a097fa640 1 -- 172.21.15.84:0/3966785238 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v223) v1 ==== 103+0+11 (secure 0 0 0) 0x7f29f80581e0 con 0x7f2a0c15d150 2023-10-16T11:25:11.861 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:25:11.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.863+0000 7f2a12aa2640 1 -- 172.21.15.84:0/3966785238 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f29f006acc0 msgr2=0x7f29f006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:11.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.863+0000 7f2a12aa2640 1 --2- 172.21.15.84:0/3966785238 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f29f006acc0 0x7f29f006d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2a00006fd0 tx=0x7f2a00008040 comp rx=0 tx=0).stop 2023-10-16T11:25:11.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.863+0000 7f2a12aa2640 1 -- 172.21.15.84:0/3966785238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a0c15d150 msgr2=0x7f2a0c159880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:11.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.863+0000 7f2a12aa2640 1 --2- 172.21.15.84:0/3966785238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a0c15d150 0x7f2a0c159880 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7f29f800d800 tx=0x7f29f800dcc0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.863+0000 7f2a12aa2640 1 -- 172.21.15.84:0/3966785238 shutdown_connections 2023-10-16T11:25:11.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.863+0000 7f2a12aa2640 1 --2- 172.21.15.84:0/3966785238 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f29f006acc0 0x7f29f006d180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.863+0000 7f2a12aa2640 1 --2- 172.21.15.84:0/3966785238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a0c160b90 0x7f2a0c15efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.863+0000 7f2a12aa2640 1 --2- 172.21.15.84:0/3966785238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a0c15d150 0x7f2a0c159880 unknown :-1 s=CLOSED pgs=2814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.863+0000 7f2a12aa2640 1 --2- 172.21.15.84:0/3966785238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a0c069da0 0x7f2a0c1f3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:11.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.863+0000 7f2a12aa2640 1 -- 172.21.15.84:0/3966785238 >> 172.21.15.84:0/3966785238 conn(0x7f2a0c0ce9b0 msgr2=0x7f2a0c06f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:11.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.863+0000 7f2a12aa2640 1 -- 172.21.15.84:0/3966785238 shutdown_connections 2023-10-16T11:25:11.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:11.863+0000 7f2a12aa2640 1 -- 172.21.15.84:0/3966785238 wait complete. 2023-10-16T11:25:11.881 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:25:12.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.095+0000 7f2d6f5e2640 1 Processor -- start 2023-10-16T11:25:12.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.095+0000 7f2d6f5e2640 1 -- start start 2023-10-16T11:25:12.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.097+0000 7f2d6f5e2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2d680ce390 0x7f2d6815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:12.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.097+0000 7f2d6f5e2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6815ed60 con 0x7f2d6815aaa0 2023-10-16T11:25:12.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.097+0000 7f2d6f5e2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d680707e0 con 0x7f2d6815ae80 2023-10-16T11:25:12.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.097+0000 7f2d6f5e2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6806b810 con 0x7f2d680ce390 2023-10-16T11:25:12.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.097+0000 7f2d6d37e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2d6815aaa0 0x7f2d680cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36002/0 (socket says 172.21.15.84:36002) 2023-10-16T11:25:12.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.097+0000 7f2d6d37e640 1 -- 172.21.15.84:0/62310795 learned_addr learned my addr 172.21.15.84:0/62310795 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:12.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.097+0000 7f2d5ffff640 1 -- 172.21.15.84:0/62310795 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247781795 0 0) 0x7f2d6815ed60 con 0x7f2d6815aaa0 2023-10-16T11:25:12.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.097+0000 7f2d6db7f640 1 --2- 172.21.15.84:0/62310795 >> v2:172.21.15.157:3300/0 conn(0x7f2d680ce390 0x7f2d6815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:12.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.097+0000 7f2d5ffff640 1 -- 172.21.15.84:0/62310795 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d54003520 con 0x7f2d6815aaa0 2023-10-16T11:25:12.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.098+0000 7f2d5ffff640 1 -- 172.21.15.84:0/62310795 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1328234627 0 0) 0x7f2d54003520 con 0x7f2d6815aaa0 2023-10-16T11:25:12.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.098+0000 7f2d5ffff640 1 -- 172.21.15.84:0/62310795 >> v2:172.21.15.157:3300/0 conn(0x7f2d680ce390 msgr2=0x7f2d6815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:12.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.098+0000 7f2d5ffff640 1 --2- 172.21.15.84:0/62310795 >> v2:172.21.15.157:3300/0 conn(0x7f2d680ce390 0x7f2d6815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.098+0000 7f2d5ffff640 1 -- 172.21.15.84:0/62310795 >> v1:172.21.15.157:6789/0 conn(0x7f2d6815ae80 legacy=0x7f2d68070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:12.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.098+0000 7f2d5ffff640 1 -- 172.21.15.84:0/62310795 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6815ef90 con 0x7f2d6815aaa0 2023-10-16T11:25:12.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.098+0000 7f2d5ffff640 1 -- 172.21.15.84:0/62310795 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f2d64001670 con 0x7f2d6815aaa0 2023-10-16T11:25:12.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.098+0000 7f2d5ffff640 1 -- 172.21.15.84:0/62310795 >> v1:172.21.15.84:6789/0 conn(0x7f2d6815aaa0 legacy=0x7f2d680cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:12.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.098+0000 7f2d5ffff640 1 --2- 172.21.15.84:0/62310795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d54003970 0x7f2d54003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:12.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.098+0000 7f2d5ffff640 1 --2- 172.21.15.84:0/62310795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d540047c0 0x7f2d54004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:12.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.098+0000 7f2d5ffff640 1 --2- 172.21.15.84:0/62310795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d54005180 0x7f2d5400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:12.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.098+0000 7f2d5ffff640 1 -- 172.21.15.84:0/62310795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6806b810 con 0x7f2d54003970 2023-10-16T11:25:12.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.098+0000 7f2d5ffff640 1 -- 172.21.15.84:0/62310795 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5400c0b0 con 0x7f2d54005180 2023-10-16T11:25:12.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.098+0000 7f2d5ffff640 1 -- 172.21.15.84:0/62310795 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5400c270 con 0x7f2d540047c0 2023-10-16T11:25:12.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.098+0000 7f2d6d37e640 1 --2- 172.21.15.84:0/62310795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d54003970 0x7f2d54003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:12.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.099+0000 7f2d6d37e640 1 -- 172.21.15.84:0/62310795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d540047c0 msgr2=0x7f2d54004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:12.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.099+0000 7f2d6d37e640 1 --2- 172.21.15.84:0/62310795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d540047c0 0x7f2d54004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.099+0000 7f2d6d37e640 1 -- 172.21.15.84:0/62310795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d54005180 msgr2=0x7f2d5400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:12.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.099+0000 7f2d6d37e640 1 --2- 172.21.15.84:0/62310795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d54005180 0x7f2d5400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.099+0000 7f2d6d37e640 1 -- 172.21.15.84:0/62310795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d54003520 con 0x7f2d54003970 2023-10-16T11:25:12.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.099+0000 7f2d6db7f640 1 -- 172.21.15.84:0/62310795 reap_dead start 2023-10-16T11:25:12.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.100+0000 7f2d6d37e640 1 --2- 172.21.15.84:0/62310795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d54003970 0x7f2d54003d70 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400c210 tx=0x7f2d540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d014313a7b10f99 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:12.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.100+0000 7f2d5ffff640 1 -- 172.21.15.84:0/62310795 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64005ad0 con 0x7f2d54003970 2023-10-16T11:25:12.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.100+0000 7f2d5ffff640 1 -- 172.21.15.84:0/62310795 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d64005c30 con 0x7f2d54003970 2023-10-16T11:25:12.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.100+0000 7f2d5ffff640 1 -- 172.21.15.84:0/62310795 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6402c890 con 0x7f2d54003970 2023-10-16T11:25:12.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.100+0000 7f2d6f5e2640 1 -- 172.21.15.84:0/62310795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d54003970 msgr2=0x7f2d54003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:12.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.100+0000 7f2d6f5e2640 1 --2- 172.21.15.84:0/62310795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d54003970 0x7f2d54003d70 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400c210 tx=0x7f2d540057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.101+0000 7f2d6f5e2640 1 -- 172.21.15.84:0/62310795 shutdown_connections 2023-10-16T11:25:12.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.101+0000 7f2d6f5e2640 1 --2- 172.21.15.84:0/62310795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d54003970 0x7f2d54003d70 unknown :-1 s=CLOSED pgs=2816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.101+0000 7f2d6f5e2640 1 -- 172.21.15.84:0/62310795 >> 172.21.15.84:0/62310795 conn(0x7f2d680cb410 msgr2=0x7f2d680cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:12.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.101+0000 7f2d6f5e2640 1 -- 172.21.15.84:0/62310795 shutdown_connections 2023-10-16T11:25:12.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.101+0000 7f2d6f5e2640 1 -- 172.21.15.84:0/62310795 wait complete. 2023-10-16T11:25:12.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.102+0000 7f2d6f5e2640 1 Processor -- start 2023-10-16T11:25:12.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.102+0000 7f2d6f5e2640 1 -- start start 2023-10-16T11:25:12.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.103+0000 7f2d6f5e2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d54003970 0x7f2d6815dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:12.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.103+0000 7f2d6f5e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d6814dbf0 0x7f2d680c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:12.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.103+0000 7f2d6f5e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d680c7900 0x7f2d680c3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:12.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.103+0000 7f2d6f5e2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6815b3d0 con 0x7f2d6814dbf0 2023-10-16T11:25:12.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.103+0000 7f2d6f5e2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d680c82d0 con 0x7f2d680c7900 2023-10-16T11:25:12.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.103+0000 7f2d6f5e2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d680c85b0 con 0x7f2d54003970 2023-10-16T11:25:12.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.104+0000 7f2d6cb7d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d6814dbf0 0x7f2d680c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:12.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.104+0000 7f2d6cb7d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d6814dbf0 0x7f2d680c73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40368/0 (socket says 172.21.15.84:40368) 2023-10-16T11:25:12.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.104+0000 7f2d6cb7d640 1 -- 172.21.15.84:0/3241271118 learned_addr learned my addr 172.21.15.84:0/3241271118 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:12.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.104+0000 7f2d6d37e640 1 --2- 172.21.15.84:0/3241271118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d54003970 0x7f2d6815dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:12.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.104+0000 7f2d6db7f640 1 --2- 172.21.15.84:0/3241271118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d680c7900 0x7f2d680c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:12.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.104+0000 7f2d6cb7d640 1 -- 172.21.15.84:0/3241271118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d54003970 msgr2=0x7f2d6815dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:12.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.104+0000 7f2d6cb7d640 1 --2- 172.21.15.84:0/3241271118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d54003970 0x7f2d6815dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.104+0000 7f2d6cb7d640 1 -- 172.21.15.84:0/3241271118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d680c7900 msgr2=0x7f2d680c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:12.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.104+0000 7f2d6cb7d640 1 --2- 172.21.15.84:0/3241271118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d680c7900 0x7f2d680c3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.104+0000 7f2d6cb7d640 1 -- 172.21.15.84:0/3241271118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d680c8890 con 0x7f2d6814dbf0 2023-10-16T11:25:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.104+0000 7f2d6cb7d640 1 --2- 172.21.15.84:0/3241271118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d6814dbf0 0x7f2d680c73c0 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800c3c0 tx=0x7f2d5800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.105+0000 7f2d5dffb640 1 -- 172.21.15.84:0/3241271118 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d58011790 con 0x7f2d6814dbf0 2023-10-16T11:25:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.105+0000 7f2d5dffb640 1 -- 172.21.15.84:0/3241271118 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d580118f0 con 0x7f2d6814dbf0 2023-10-16T11:25:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.105+0000 7f2d6f5e2640 1 -- 172.21.15.84:0/3241271118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d680c8b70 con 0x7f2d6814dbf0 2023-10-16T11:25:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.105+0000 7f2d5dffb640 1 -- 172.21.15.84:0/3241271118 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d58011ce0 con 0x7f2d6814dbf0 2023-10-16T11:25:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.105+0000 7f2d6f5e2640 1 -- 172.21.15.84:0/3241271118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d681583c0 con 0x7f2d6814dbf0 2023-10-16T11:25:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.106+0000 7f2d6f5e2640 1 -- 172.21.15.84:0/3241271118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6804e160 con 0x7f2d6814dbf0 2023-10-16T11:25:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.107+0000 7f2d5dffb640 1 -- 172.21.15.84:0/3241271118 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 142) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f2d5801c460 con 0x7f2d6814dbf0 2023-10-16T11:25:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.108+0000 7f2d5dffb640 1 --2- 172.21.15.84:0/3241271118 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f2d4c06b020 0x7f2d4c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.108+0000 7f2d5dffb640 1 -- 172.21.15.84:0/3241271118 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 7418+0+0 (secure 0 0 0) 0x7f2d58089b80 con 0x7f2d6814dbf0 2023-10-16T11:25:12.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.108+0000 7f2d6d37e640 1 --2- 172.21.15.84:0/3241271118 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f2d4c06b020 0x7f2d4c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:12.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.109+0000 7f2d6d37e640 1 --2- 172.21.15.84:0/3241271118 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f2d4c06b020 0x7f2d4c06d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2d64034000 tx=0x7f2d64009030 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:12.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.113+0000 7f2d5dffb640 1 -- 172.21.15.84:0/3241271118 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2d58054300 con 0x7f2d6814dbf0 2023-10-16T11:25:12.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.341+0000 7f2d6f5e2640 1 -- 172.21.15.84:0/3241271118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d680b16b0 con 0x7f2d6814dbf0 2023-10-16T11:25:12.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.342+0000 7f2d5dffb640 1 -- 172.21.15.84:0/3241271118 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+6152 (secure 0 0 0) 0x7f2d58057920 con 0x7f2d6814dbf0 2023-10-16T11:25:12.343 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:12.344 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":120,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3106898227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3106898227},{"type":"v1","addr":"172.21.15.84:6837","nonce":3106898227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":7471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2296953749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2296953749},{"type":"v1","addr":"172.21.15.157:6834","nonce":2296953749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:24:25.531691+0000","modified":"2023-10-16T11:24:28.574116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11307},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11307":{"gid":11307,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/551895595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":551895595},{"type":"v1","addr":"172.21.15.84:6836","nonce":551895595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:24:40.366708+0000","modified":"2023-10-16T11:24:43.431532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7474":{"gid":7474,"name":"d","rank":0,"incarnation":116,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6835/3992377882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3992377882},{"type":"v1","addr":"172.21.15.157:6835","nonce":3992377882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-16T11:25:12.344 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 120 2023-10-16T11:25:12.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.345+0000 7f2d6f5e2640 1 -- 172.21.15.84:0/3241271118 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f2d4c06b020 msgr2=0x7f2d4c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:12.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.345+0000 7f2d6f5e2640 1 --2- 172.21.15.84:0/3241271118 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f2d4c06b020 0x7f2d4c06d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2d64034000 tx=0x7f2d64009030 comp rx=0 tx=0).stop 2023-10-16T11:25:12.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.345+0000 7f2d6f5e2640 1 -- 172.21.15.84:0/3241271118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d6814dbf0 msgr2=0x7f2d680c73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:12.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.345+0000 7f2d6f5e2640 1 --2- 172.21.15.84:0/3241271118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d6814dbf0 0x7f2d680c73c0 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800c3c0 tx=0x7f2d5800c880 comp rx=0 tx=0).stop 2023-10-16T11:25:12.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.345+0000 7f2d6f5e2640 1 -- 172.21.15.84:0/3241271118 shutdown_connections 2023-10-16T11:25:12.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.345+0000 7f2d6f5e2640 1 --2- 172.21.15.84:0/3241271118 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f2d4c06b020 0x7f2d4c06d4e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.345+0000 7f2d6f5e2640 1 --2- 172.21.15.84:0/3241271118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d680c7900 0x7f2d680c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.345+0000 7f2d6f5e2640 1 --2- 172.21.15.84:0/3241271118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d6814dbf0 0x7f2d680c73c0 unknown :-1 s=CLOSED pgs=2817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.345+0000 7f2d6f5e2640 1 --2- 172.21.15.84:0/3241271118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2d54003970 0x7f2d6815dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.345+0000 7f2d6f5e2640 1 -- 172.21.15.84:0/3241271118 >> 172.21.15.84:0/3241271118 conn(0x7f2d680cb410 msgr2=0x7f2d680b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:12.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.345+0000 7f2d6f5e2640 1 -- 172.21.15.84:0/3241271118 shutdown_connections 2023-10-16T11:25:12.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.345+0000 7f2d6f5e2640 1 -- 172.21.15.84:0/3241271118 wait complete. 2023-10-16T11:25:12.365 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:25:12.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.579+0000 7f63784d7640 1 Processor -- start 2023-10-16T11:25:12.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.580+0000 7f63784d7640 1 -- start start 2023-10-16T11:25:12.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.581+0000 7f63784d7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f637015aaa0 0x7f637015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:12.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.581+0000 7f63784d7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f63700ce390 0x7f6370070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:12.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.581+0000 7f63784d7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637015ed60 con 0x7f637015ae80 2023-10-16T11:25:12.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.581+0000 7f63784d7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f637006b810 con 0x7f63700ce390 2023-10-16T11:25:12.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.581+0000 7f63784d7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63700707e0 con 0x7f637015aaa0 2023-10-16T11:25:12.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.581+0000 7f6376a74640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f63700ce390 0x7f6370070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:12.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.582+0000 7f6376a74640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f63700ce390 0x7f6370070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40384/0 (socket says 172.21.15.84:40384) 2023-10-16T11:25:12.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.582+0000 7f6376a74640 1 -- 172.21.15.84:0/1454006995 learned_addr learned my addr 172.21.15.84:0/1454006995 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:12.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.582+0000 7f6376273640 1 --2- 172.21.15.84:0/1454006995 >> v2:172.21.15.157:3300/0 conn(0x7f637015aaa0 0x7f637015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:12.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.582+0000 7f6376a74640 1 -- 172.21.15.84:0/1454006995 >> v2:172.21.15.157:3300/0 conn(0x7f637015aaa0 msgr2=0x7f637015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:12.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.582+0000 7f6376a74640 1 --2- 172.21.15.84:0/1454006995 >> v2:172.21.15.157:3300/0 conn(0x7f637015aaa0 0x7f637015e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.582+0000 7f6376a74640 1 -- 172.21.15.84:0/1454006995 >> v1:172.21.15.157:6789/0 conn(0x7f637015ae80 legacy=0x7f63700cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:12.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.582+0000 7f6376a74640 1 -- 172.21.15.84:0/1454006995 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637015ef90 con 0x7f63700ce390 2023-10-16T11:25:12.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.583+0000 7f6376a74640 1 --2- 172.21.15.84:0/1454006995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63700ce390 0x7f6370070f50 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f63700cd990 tx=0x7f636802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0378c71dd2235b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:12.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.583+0000 7f6375271640 1 -- 172.21.15.84:0/1454006995 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636802ca80 con 0x7f63700ce390 2023-10-16T11:25:12.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.584+0000 7f6375271640 1 -- 172.21.15.84:0/1454006995 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636802cbe0 con 0x7f63700ce390 2023-10-16T11:25:12.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.584+0000 7f6375271640 1 -- 172.21.15.84:0/1454006995 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63680343f0 con 0x7f63700ce390 2023-10-16T11:25:12.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.584+0000 7f63784d7640 1 -- 172.21.15.84:0/1454006995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63700ce390 msgr2=0x7f6370070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:12.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.584+0000 7f63784d7640 1 --2- 172.21.15.84:0/1454006995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63700ce390 0x7f6370070f50 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f63700cd990 tx=0x7f636802bee0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.584+0000 7f63784d7640 1 -- 172.21.15.84:0/1454006995 shutdown_connections 2023-10-16T11:25:12.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.584+0000 7f63784d7640 1 --2- 172.21.15.84:0/1454006995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63700ce390 0x7f6370070f50 unknown :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.584+0000 7f63784d7640 1 --2- 172.21.15.84:0/1454006995 >> v2:172.21.15.157:3300/0 conn(0x7f637015aaa0 0x7f637015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.584+0000 7f63784d7640 1 -- 172.21.15.84:0/1454006995 >> 172.21.15.84:0/1454006995 conn(0x7f63700cb410 msgr2=0x7f63700cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:12.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.584+0000 7f63784d7640 1 -- 172.21.15.84:0/1454006995 shutdown_connections 2023-10-16T11:25:12.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.584+0000 7f63784d7640 1 -- 172.21.15.84:0/1454006995 wait complete. 2023-10-16T11:25:12.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.586+0000 7f63784d7640 1 Processor -- start 2023-10-16T11:25:12.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.587+0000 7f63784d7640 1 -- start start 2023-10-16T11:25:12.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.588+0000 7f63784d7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63700ce390 0x7f637014f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:12.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.588+0000 7f63784d7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f637015aaa0 0x7f63700c2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:12.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.588+0000 7f63784d7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f637015ae80 0x7f63700c1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:12.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.588+0000 7f63784d7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63700c3880 con 0x7f63700ce390 2023-10-16T11:25:12.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.588+0000 7f63784d7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63700c1f90 con 0x7f637015aaa0 2023-10-16T11:25:12.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.588+0000 7f63784d7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63700c9d90 con 0x7f637015ae80 2023-10-16T11:25:12.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.588+0000 7f6376273640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63700ce390 0x7f637014f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:12.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.588+0000 7f6376273640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63700ce390 0x7f637014f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40400/0 (socket says 172.21.15.84:40400) 2023-10-16T11:25:12.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.588+0000 7f6376273640 1 -- 172.21.15.84:0/1281461978 learned_addr learned my addr 172.21.15.84:0/1281461978 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:12.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.588+0000 7f6375a72640 1 --2- 172.21.15.84:0/1281461978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f637015aaa0 0x7f63700c2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:12.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.589+0000 7f6376273640 1 -- 172.21.15.84:0/1281461978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f637015ae80 msgr2=0x7f63700c1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:12.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.589+0000 7f6376273640 1 --2- 172.21.15.84:0/1281461978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f637015ae80 0x7f63700c1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.589+0000 7f6376273640 1 -- 172.21.15.84:0/1281461978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f637015aaa0 msgr2=0x7f63700c2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:12.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.589+0000 7f6376273640 1 --2- 172.21.15.84:0/1281461978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f637015aaa0 0x7f63700c2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.589+0000 7f6376273640 1 -- 172.21.15.84:0/1281461978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6368009b30 con 0x7f63700ce390 2023-10-16T11:25:12.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.589+0000 7f6376273640 1 --2- 172.21.15.84:0/1281461978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63700ce390 0x7f637014f250 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f636c00d830 tx=0x7f636c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:12.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.589+0000 7f6366ffd640 1 -- 172.21.15.84:0/1281461978 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c00b6c0 con 0x7f63700ce390 2023-10-16T11:25:12.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.589+0000 7f6366ffd640 1 -- 172.21.15.84:0/1281461978 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c00b820 con 0x7f63700ce390 2023-10-16T11:25:12.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.589+0000 7f63784d7640 1 -- 172.21.15.84:0/1281461978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63700c35a0 con 0x7f63700ce390 2023-10-16T11:25:12.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.589+0000 7f6366ffd640 1 -- 172.21.15.84:0/1281461978 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c00bbb0 con 0x7f63700ce390 2023-10-16T11:25:12.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.589+0000 7f63784d7640 1 -- 172.21.15.84:0/1281461978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63700c9a20 con 0x7f63700ce390 2023-10-16T11:25:12.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.590+0000 7f63784d7640 1 -- 172.21.15.84:0/1281461978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63700c95a0 con 0x7f63700ce390 2023-10-16T11:25:12.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.592+0000 7f6366ffd640 1 -- 172.21.15.84:0/1281461978 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 142) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f636c01e460 con 0x7f63700ce390 2023-10-16T11:25:12.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.592+0000 7f6366ffd640 1 --2- 172.21.15.84:0/1281461978 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f635406b050 0x7f635406d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:12.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.593+0000 7f6366ffd640 1 -- 172.21.15.84:0/1281461978 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 7418+0+0 (secure 0 0 0) 0x7f636c08a650 con 0x7f63700ce390 2023-10-16T11:25:12.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.593+0000 7f6375a72640 1 --2- 172.21.15.84:0/1281461978 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f635406b050 0x7f635406d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:12.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.594+0000 7f6375a72640 1 --2- 172.21.15.84:0/1281461978 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f635406b050 0x7f635406d510 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f63701f5ca0 tx=0x7f6360008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:12.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.597+0000 7f6366ffd640 1 -- 172.21.15.84:0/1281461978 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f63700c95a0 con 0x7f63700ce390 2023-10-16T11:25:12.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.789+0000 7f63784d7640 1 -- 172.21.15.84:0/1281461978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63700c95a0 con 0x7f63700ce390 2023-10-16T11:25:12.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.793+0000 7f6366ffd640 1 -- 172.21.15.84:0/1281461978 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v223) v1 ==== 74+0+21513 (secure 0 0 0) 0x7f63700c95a0 con 0x7f63700ce390 2023-10-16T11:25:12.795 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:12.795 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":223,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:25:09.210192+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:24:23.363032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:24:24.551270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:24:38.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:24:39.387564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1287173368":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/595669474":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/4107174082":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2906390983":"2023-10-17T11:25:09.210125+0000","172.21.15.84:6833/2880641466":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/3419402221":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2161898622":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/455217819":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/4247809558":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/1814319372":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2877946366":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6833/1425409017":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6832/1425409017":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/1771162859":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6832/2880641466":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/3655599055":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/1076506559":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/2748224910":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/857172551":"2023-10-17T11:24:58.196616+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:25:12.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.796+0000 7f63784d7640 1 -- 172.21.15.84:0/1281461978 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f635406b050 msgr2=0x7f635406d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:12.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.796+0000 7f63784d7640 1 --2- 172.21.15.84:0/1281461978 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f635406b050 0x7f635406d510 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f63701f5ca0 tx=0x7f6360008040 comp rx=0 tx=0).stop 2023-10-16T11:25:12.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.796+0000 7f63784d7640 1 -- 172.21.15.84:0/1281461978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63700ce390 msgr2=0x7f637014f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:12.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.796+0000 7f63784d7640 1 --2- 172.21.15.84:0/1281461978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63700ce390 0x7f637014f250 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f636c00d830 tx=0x7f636c00dcf0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.796+0000 7f63784d7640 1 -- 172.21.15.84:0/1281461978 shutdown_connections 2023-10-16T11:25:12.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.796+0000 7f63784d7640 1 --2- 172.21.15.84:0/1281461978 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f635406b050 0x7f635406d510 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.796+0000 7f63784d7640 1 --2- 172.21.15.84:0/1281461978 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f637015ae80 0x7f63700c1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.796+0000 7f63784d7640 1 --2- 172.21.15.84:0/1281461978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f637015aaa0 0x7f63700c2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.796+0000 7f63784d7640 1 --2- 172.21.15.84:0/1281461978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63700ce390 0x7f637014f250 unknown :-1 s=CLOSED pgs=2819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:12.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.796+0000 7f63784d7640 1 -- 172.21.15.84:0/1281461978 >> 172.21.15.84:0/1281461978 conn(0x7f63700cb410 msgr2=0x7f6370158c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:12.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.796+0000 7f63784d7640 1 -- 172.21.15.84:0/1281461978 shutdown_connections 2023-10-16T11:25:12.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:12.797+0000 7f63784d7640 1 -- 172.21.15.84:0/1281461978 wait complete. 2023-10-16T11:25:12.819 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T11:25:12.819 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:25:13.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.021+0000 7f4152f53640 1 Processor -- start 2023-10-16T11:25:13.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.021+0000 7f4152f53640 1 -- start start 2023-10-16T11:25:13.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.023+0000 7f4152f53640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f414c06b4c0 0x7f414c0ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:13.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.023+0000 7f4152f53640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f414c160800 0x7f414c0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:13.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.023+0000 7f4152f53640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f414c06e170 con 0x7f414c160420 2023-10-16T11:25:13.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.023+0000 7f4152f53640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f414c0cde30 con 0x7f414c160800 2023-10-16T11:25:13.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.023+0000 7f4152f53640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414c0cd460 con 0x7f414c06b4c0 2023-10-16T11:25:13.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.023+0000 7f4150cef640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f414c160420 0x7f414c06adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36012/0 (socket says 172.21.15.84:36012) 2023-10-16T11:25:13.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.023+0000 7f4150cef640 1 -- 172.21.15.84:0/969523012 learned_addr learned my addr 172.21.15.84:0/969523012 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:13.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.023+0000 7f41514f0640 1 --2- 172.21.15.84:0/969523012 >> v2:172.21.15.84:3300/0 conn(0x7f414c160800 0x7f414c0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:13.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.024+0000 7f414b7fe640 1 -- 172.21.15.84:0/969523012 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756531064 0 0) 0x7f414c06e170 con 0x7f414c160420 2023-10-16T11:25:13.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.024+0000 7f414bfff640 1 --2- 172.21.15.84:0/969523012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f414c06b4c0 0x7f414c0ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:13.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.024+0000 7f414b7fe640 1 -- 172.21.15.84:0/969523012 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f413c003520 con 0x7f414c160420 2023-10-16T11:25:13.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.024+0000 7f41514f0640 1 -- 172.21.15.84:0/969523012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f414c06b4c0 msgr2=0x7f414c0ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:13.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.024+0000 7f41514f0640 1 --2- 172.21.15.84:0/969523012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f414c06b4c0 0x7f414c0ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.024+0000 7f41514f0640 1 -- 172.21.15.84:0/969523012 >> v1:172.21.15.84:6789/0 conn(0x7f414c160420 legacy=0x7f414c06adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:13.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.024+0000 7f41514f0640 1 -- 172.21.15.84:0/969523012 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414c06e3c0 con 0x7f414c160800 2023-10-16T11:25:13.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.024+0000 7f414bfff640 1 --2- 172.21.15.84:0/969523012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f414c06b4c0 0x7f414c0ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:13.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.025+0000 7f41514f0640 1 --2- 172.21.15.84:0/969523012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f414c160800 0x7f414c0cd8f0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f4144009d90 tx=0x7f414402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2226e441c60c25c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:13.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.025+0000 7f414b7fe640 1 -- 172.21.15.84:0/969523012 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414402cc60 con 0x7f414c160800 2023-10-16T11:25:13.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.025+0000 7f414b7fe640 1 -- 172.21.15.84:0/969523012 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414402cdc0 con 0x7f414c160800 2023-10-16T11:25:13.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.025+0000 7f414b7fe640 1 -- 172.21.15.84:0/969523012 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144034660 con 0x7f414c160800 2023-10-16T11:25:13.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.026+0000 7f4152f53640 1 -- 172.21.15.84:0/969523012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f414c160800 msgr2=0x7f414c0cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:13.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.026+0000 7f4152f53640 1 --2- 172.21.15.84:0/969523012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f414c160800 0x7f414c0cd8f0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f4144009d90 tx=0x7f414402beb0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.026+0000 7f4152f53640 1 -- 172.21.15.84:0/969523012 shutdown_connections 2023-10-16T11:25:13.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.026+0000 7f4152f53640 1 --2- 172.21.15.84:0/969523012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f414c160800 0x7f414c0cd8f0 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.026+0000 7f4152f53640 1 --2- 172.21.15.84:0/969523012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f414c06b4c0 0x7f414c0ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.026+0000 7f4152f53640 1 -- 172.21.15.84:0/969523012 >> 172.21.15.84:0/969523012 conn(0x7f414c0b0d30 msgr2=0x7f414c0b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:13.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.026+0000 7f4152f53640 1 -- 172.21.15.84:0/969523012 shutdown_connections 2023-10-16T11:25:13.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.026+0000 7f4152f53640 1 -- 172.21.15.84:0/969523012 wait complete. 2023-10-16T11:25:13.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.027+0000 7f4152f53640 1 Processor -- start 2023-10-16T11:25:13.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.028+0000 7f4152f53640 1 -- start start 2023-10-16T11:25:13.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.028+0000 7f4152f53640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f414c06b4c0 0x7f414c0c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:13.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.029+0000 7f4152f53640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f414c160420 0x7f414c1f2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:13.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.029+0000 7f4152f53640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f414c160800 0x7f414c0ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:13.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.029+0000 7f4152f53640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c1f37d0 con 0x7f414c160420 2023-10-16T11:25:13.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.029+0000 7f4152f53640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c0c8630 con 0x7f414c160800 2023-10-16T11:25:13.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.029+0000 7f4152f53640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414c1f2b10 con 0x7f414c06b4c0 2023-10-16T11:25:13.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.029+0000 7f414bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f414c160420 0x7f414c1f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:13.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.029+0000 7f414bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f414c160420 0x7f414c1f2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40416/0 (socket says 172.21.15.84:40416) 2023-10-16T11:25:13.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.029+0000 7f414bfff640 1 -- 172.21.15.84:0/2843541370 learned_addr learned my addr 172.21.15.84:0/2843541370 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:13.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.029+0000 7f4150cef640 1 --2- 172.21.15.84:0/2843541370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f414c06b4c0 0x7f414c0c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:13.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.029+0000 7f41514f0640 1 --2- 172.21.15.84:0/2843541370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f414c160800 0x7f414c0ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:13.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.029+0000 7f414bfff640 1 -- 172.21.15.84:0/2843541370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f414c06b4c0 msgr2=0x7f414c0c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:13.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.029+0000 7f414bfff640 1 --2- 172.21.15.84:0/2843541370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f414c06b4c0 0x7f414c0c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.029+0000 7f414bfff640 1 -- 172.21.15.84:0/2843541370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f414c160800 msgr2=0x7f414c0ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:13.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.029+0000 7f414bfff640 1 --2- 172.21.15.84:0/2843541370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f414c160800 0x7f414c0ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.029+0000 7f414bfff640 1 -- 172.21.15.84:0/2843541370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4144009a70 con 0x7f414c160420 2023-10-16T11:25:13.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.030+0000 7f414bfff640 1 --2- 172.21.15.84:0/2843541370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f414c160420 0x7f414c1f2270 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f413800c7f0 tx=0x7f413800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:13.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.030+0000 7f41497fa640 1 -- 172.21.15.84:0/2843541370 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138007a30 con 0x7f414c160420 2023-10-16T11:25:13.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.030+0000 7f41497fa640 1 -- 172.21.15.84:0/2843541370 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4138007b90 con 0x7f414c160420 2023-10-16T11:25:13.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.030+0000 7f41497fa640 1 -- 172.21.15.84:0/2843541370 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413800ce40 con 0x7f414c160420 2023-10-16T11:25:13.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.030+0000 7f4152f53640 1 -- 172.21.15.84:0/2843541370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f414c1f2df0 con 0x7f414c160420 2023-10-16T11:25:13.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.030+0000 7f4152f53640 1 -- 172.21.15.84:0/2843541370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f414c1f33b0 con 0x7f414c160420 2023-10-16T11:25:13.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.031+0000 7f4152f53640 1 -- 172.21.15.84:0/2843541370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f414c0cd0d0 con 0x7f414c160420 2023-10-16T11:25:13.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.032+0000 7f41497fa640 1 -- 172.21.15.84:0/2843541370 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 142) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f41380113f0 con 0x7f414c160420 2023-10-16T11:25:13.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.033+0000 7f41497fa640 1 --2- 172.21.15.84:0/2843541370 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f413006b020 0x7f413006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:13.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.033+0000 7f41497fa640 1 -- 172.21.15.84:0/2843541370 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 7418+0+0 (secure 0 0 0) 0x7f413808a200 con 0x7f414c160420 2023-10-16T11:25:13.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.033+0000 7f4150cef640 1 --2- 172.21.15.84:0/2843541370 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f413006b020 0x7f413006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:13.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.034+0000 7f4150cef640 1 --2- 172.21.15.84:0/2843541370 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f413006b020 0x7f413006d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f41400011e0 tx=0x7f4140009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:13.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.037+0000 7f41497fa640 1 -- 172.21.15.84:0/2843541370 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f414c0cd0d0 con 0x7f414c160420 2023-10-16T11:25:13.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.274+0000 7f4152f53640 1 -- 172.21.15.84:0/2843541370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f414c0cd0d0 con 0x7f414c160420 2023-10-16T11:25:13.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.276+0000 7f41497fa640 1 -- 172.21.15.84:0/2843541370 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+6152 (secure 0 0 0) 0x7f414c0cd0d0 con 0x7f414c160420 2023-10-16T11:25:13.277 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:13.277 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":120,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3106898227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3106898227},{"type":"v1","addr":"172.21.15.84:6837","nonce":3106898227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":7471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2296953749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2296953749},{"type":"v1","addr":"172.21.15.157:6834","nonce":2296953749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:24:25.531691+0000","modified":"2023-10-16T11:24:28.574116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11307},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11307":{"gid":11307,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/551895595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":551895595},{"type":"v1","addr":"172.21.15.84:6836","nonce":551895595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:24:40.366708+0000","modified":"2023-10-16T11:24:43.431532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7474":{"gid":7474,"name":"d","rank":0,"incarnation":116,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6835/3992377882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3992377882},{"type":"v1","addr":"172.21.15.157:6835","nonce":3992377882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-16T11:25:13.278 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 120 2023-10-16T11:25:13.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.278+0000 7f4152f53640 1 -- 172.21.15.84:0/2843541370 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f413006b020 msgr2=0x7f413006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:13.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.278+0000 7f4152f53640 1 --2- 172.21.15.84:0/2843541370 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f413006b020 0x7f413006d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f41400011e0 tx=0x7f4140009040 comp rx=0 tx=0).stop 2023-10-16T11:25:13.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.278+0000 7f4152f53640 1 -- 172.21.15.84:0/2843541370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f414c160420 msgr2=0x7f414c1f2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:13.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.278+0000 7f4152f53640 1 --2- 172.21.15.84:0/2843541370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f414c160420 0x7f414c1f2270 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f413800c7f0 tx=0x7f413800cbf0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.278+0000 7f4152f53640 1 -- 172.21.15.84:0/2843541370 shutdown_connections 2023-10-16T11:25:13.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.278+0000 7f4152f53640 1 --2- 172.21.15.84:0/2843541370 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f413006b020 0x7f413006d4e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.278+0000 7f4152f53640 1 --2- 172.21.15.84:0/2843541370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f414c160800 0x7f414c0ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.278+0000 7f4152f53640 1 --2- 172.21.15.84:0/2843541370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f414c160420 0x7f414c1f2270 unknown :-1 s=CLOSED pgs=2822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.278+0000 7f4152f53640 1 --2- 172.21.15.84:0/2843541370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f414c06b4c0 0x7f414c0c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.278+0000 7f4152f53640 1 -- 172.21.15.84:0/2843541370 >> 172.21.15.84:0/2843541370 conn(0x7f414c0b0d30 msgr2=0x7f414c0cdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:13.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.278+0000 7f4152f53640 1 -- 172.21.15.84:0/2843541370 shutdown_connections 2023-10-16T11:25:13.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.279+0000 7f4152f53640 1 -- 172.21.15.84:0/2843541370 wait complete. 2023-10-16T11:25:13.298 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:25:13.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.509+0000 7f9326c47640 1 Processor -- start 2023-10-16T11:25:13.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.509+0000 7f9326c47640 1 -- start start 2023-10-16T11:25:13.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.510+0000 7f9326c47640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93200ad970 0x7f93200cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:13.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.510+0000 7f9326c47640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93200ccfb0 0x7f93200cd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:13.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.510+0000 7f9326c47640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932015b9d0 con 0x7f932006c040 2023-10-16T11:25:13.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.510+0000 7f9326c47640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932015bcb0 con 0x7f93200ccfb0 2023-10-16T11:25:13.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.510+0000 7f9326c47640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932015bf90 con 0x7f93200ad970 2023-10-16T11:25:13.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.511+0000 7f9326446640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93200ccfb0 0x7f93200cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:13.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.511+0000 7f9326446640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93200ccfb0 0x7f93200cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:59444/0 (socket says 172.21.15.84:59444) 2023-10-16T11:25:13.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.511+0000 7f9326446640 1 -- 172.21.15.84:0/373546036 learned_addr learned my addr 172.21.15.84:0/373546036 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:13.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.511+0000 7f9325444640 1 --2- 172.21.15.84:0/373546036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93200ad970 0x7f93200cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:13.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.512+0000 7f9325444640 1 -- 172.21.15.84:0/373546036 >> v2:172.21.15.157:3300/0 conn(0x7f93200ccfb0 msgr2=0x7f93200cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:13.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.512+0000 7f9325444640 1 --2- 172.21.15.84:0/373546036 >> v2:172.21.15.157:3300/0 conn(0x7f93200ccfb0 0x7f93200cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.512+0000 7f9325444640 1 -- 172.21.15.84:0/373546036 >> v1:172.21.15.157:6789/0 conn(0x7f932006c040 legacy=0x7f932006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:13.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.512+0000 7f9325444640 1 -- 172.21.15.84:0/373546036 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93200cc270 con 0x7f93200ad970 2023-10-16T11:25:13.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.512+0000 7f9326446640 1 --2- 172.21.15.84:0/373546036 >> v2:172.21.15.157:3300/0 conn(0x7f93200ccfb0 0x7f93200cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:25:13.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.512+0000 7f9324c43640 1 -- 172.21.15.84:0/373546036 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1360679534 0 0) 0x7f932015b9d0 con 0x7f932006c040 2023-10-16T11:25:13.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.513+0000 7f9325444640 1 --2- 172.21.15.84:0/373546036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93200ad970 0x7f93200cca70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f930c0097d0 tx=0x7f930c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e80655914fa1af0c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:13.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.514+0000 7f9324c43640 1 -- 172.21.15.84:0/373546036 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c02d040 con 0x7f93200ad970 2023-10-16T11:25:13.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.514+0000 7f9324c43640 1 -- 172.21.15.84:0/373546036 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c031810 con 0x7f93200ad970 2023-10-16T11:25:13.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.514+0000 7f9324c43640 1 -- 172.21.15.84:0/373546036 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c031ba0 con 0x7f93200ad970 2023-10-16T11:25:13.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.515+0000 7f9326c47640 1 -- 172.21.15.84:0/373546036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93200ad970 msgr2=0x7f93200cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:13.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.515+0000 7f9326c47640 1 --2- 172.21.15.84:0/373546036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93200ad970 0x7f93200cca70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f930c0097d0 tx=0x7f930c02b8b0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.515+0000 7f9326c47640 1 -- 172.21.15.84:0/373546036 shutdown_connections 2023-10-16T11:25:13.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.515+0000 7f9326c47640 1 --2- 172.21.15.84:0/373546036 >> v2:172.21.15.157:3300/0 conn(0x7f93200ccfb0 0x7f93200cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.515+0000 7f9326c47640 1 --2- 172.21.15.84:0/373546036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93200ad970 0x7f93200cca70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.515+0000 7f9326c47640 1 -- 172.21.15.84:0/373546036 >> 172.21.15.84:0/373546036 conn(0x7f93200ce6a0 msgr2=0x7f93200ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:13.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.515+0000 7f9326c47640 1 -- 172.21.15.84:0/373546036 shutdown_connections 2023-10-16T11:25:13.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.515+0000 7f9326c47640 1 -- 172.21.15.84:0/373546036 wait complete. 2023-10-16T11:25:13.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.517+0000 7f9326c47640 1 Processor -- start 2023-10-16T11:25:13.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.517+0000 7f9326c47640 1 -- start start 2023-10-16T11:25:13.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.519+0000 7f9326c47640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f932006c040 0x7f9320153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:13.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.519+0000 7f9326c47640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93200ad970 0x7f932014fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:13.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.519+0000 7f9326c47640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93200ccfb0 0x7f93200c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:13.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.519+0000 7f9326c47640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93200aeff0 con 0x7f93200ad970 2023-10-16T11:25:13.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.519+0000 7f9326c47640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93201543c0 con 0x7f93200ccfb0 2023-10-16T11:25:13.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.519+0000 7f9326c47640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93200ae100 con 0x7f932006c040 2023-10-16T11:25:13.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.519+0000 7f9325444640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93200ad970 0x7f932014fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:13.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.519+0000 7f9325444640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93200ad970 0x7f932014fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40426/0 (socket says 172.21.15.84:40426) 2023-10-16T11:25:13.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.519+0000 7f9325444640 1 -- 172.21.15.84:0/3381854974 learned_addr learned my addr 172.21.15.84:0/3381854974 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:13.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.519+0000 7f9325c45640 1 --2- 172.21.15.84:0/3381854974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f932006c040 0x7f9320153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:13.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.519+0000 7f9326446640 1 --2- 172.21.15.84:0/3381854974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93200ccfb0 0x7f93200c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:13.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.519+0000 7f9325444640 1 -- 172.21.15.84:0/3381854974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f932006c040 msgr2=0x7f9320153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:13.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.519+0000 7f9325444640 1 --2- 172.21.15.84:0/3381854974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f932006c040 0x7f9320153e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.519+0000 7f9325444640 1 -- 172.21.15.84:0/3381854974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93200ccfb0 msgr2=0x7f93200c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:13.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.519+0000 7f9325444640 1 --2- 172.21.15.84:0/3381854974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93200ccfb0 0x7f93200c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.519+0000 7f9325444640 1 -- 172.21.15.84:0/3381854974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93180094d0 con 0x7f93200ad970 2023-10-16T11:25:13.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.520+0000 7f9325444640 1 --2- 172.21.15.84:0/3381854974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93200ad970 0x7f932014fd30 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f930c02bec0 tx=0x7f930c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:13.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.520+0000 7f931e7fc640 1 -- 172.21.15.84:0/3381854974 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c03ace0 con 0x7f93200ad970 2023-10-16T11:25:13.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.520+0000 7f931e7fc640 1 -- 172.21.15.84:0/3381854974 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c03ae60 con 0x7f93200ad970 2023-10-16T11:25:13.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.520+0000 7f931e7fc640 1 -- 172.21.15.84:0/3381854974 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c044640 con 0x7f93200ad970 2023-10-16T11:25:13.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.520+0000 7f9326c47640 1 -- 172.21.15.84:0/3381854974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f930c009400 con 0x7f93200ad970 2023-10-16T11:25:13.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.520+0000 7f9326c47640 1 -- 172.21.15.84:0/3381854974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93200ae6c0 con 0x7f93200ad970 2023-10-16T11:25:13.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.521+0000 7f9326c47640 1 -- 172.21.15.84:0/3381854974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932006b330 con 0x7f93200ad970 2023-10-16T11:25:13.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.522+0000 7f931e7fc640 1 -- 172.21.15.84:0/3381854974 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 142) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f930c04d440 con 0x7f93200ad970 2023-10-16T11:25:13.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.523+0000 7f931e7fc640 1 --2- 172.21.15.84:0/3381854974 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f930406b020 0x7f930406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:13.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.523+0000 7f931e7fc640 1 -- 172.21.15.84:0/3381854974 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 7418+0+0 (secure 0 0 0) 0x7f930c0ac380 con 0x7f93200ad970 2023-10-16T11:25:13.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.524+0000 7f9325c45640 1 --2- 172.21.15.84:0/3381854974 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f930406b020 0x7f930406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:13.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.524+0000 7f9325c45640 1 --2- 172.21.15.84:0/3381854974 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f930406b020 0x7f930406d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f93140012e0 tx=0x7f9314009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:13.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.528+0000 7f931e7fc640 1 -- 172.21.15.84:0/3381854974 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f930c076b00 con 0x7f93200ad970 2023-10-16T11:25:13.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.740+0000 7f9326c47640 1 -- 172.21.15.84:0/3381854974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f932004e9b0 con 0x7f93200ad970 2023-10-16T11:25:13.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.741+0000 7f931e7fc640 1 -- 172.21.15.84:0/3381854974 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+6152 (secure 0 0 0) 0x7f930c07a120 con 0x7f93200ad970 2023-10-16T11:25:13.743 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:13.743 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":120,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3106898227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3106898227},{"type":"v1","addr":"172.21.15.84:6837","nonce":3106898227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":7471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2296953749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2296953749},{"type":"v1","addr":"172.21.15.157:6834","nonce":2296953749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:24:25.531691+0000","modified":"2023-10-16T11:24:28.574116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11307},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11307":{"gid":11307,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6836/551895595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":551895595},{"type":"v1","addr":"172.21.15.84:6836","nonce":551895595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:24:40.366708+0000","modified":"2023-10-16T11:24:43.431532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7474":{"gid":7474,"name":"d","rank":0,"incarnation":116,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6835/3992377882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3992377882},{"type":"v1","addr":"172.21.15.157:6835","nonce":3992377882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-16T11:25:13.743 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 120 2023-10-16T11:25:13.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.744+0000 7f9326c47640 1 -- 172.21.15.84:0/3381854974 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f930406b020 msgr2=0x7f930406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:13.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.744+0000 7f9326c47640 1 --2- 172.21.15.84:0/3381854974 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f930406b020 0x7f930406d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f93140012e0 tx=0x7f9314009040 comp rx=0 tx=0).stop 2023-10-16T11:25:13.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.744+0000 7f9326c47640 1 -- 172.21.15.84:0/3381854974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93200ad970 msgr2=0x7f932014fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:13.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.744+0000 7f9326c47640 1 --2- 172.21.15.84:0/3381854974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93200ad970 0x7f932014fd30 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f930c02bec0 tx=0x7f930c0046c0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.744+0000 7f9326c47640 1 -- 172.21.15.84:0/3381854974 shutdown_connections 2023-10-16T11:25:13.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.744+0000 7f9326c47640 1 --2- 172.21.15.84:0/3381854974 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f930406b020 0x7f930406d4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.744+0000 7f9326c47640 1 --2- 172.21.15.84:0/3381854974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93200ccfb0 0x7f93200c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.744+0000 7f9326c47640 1 --2- 172.21.15.84:0/3381854974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93200ad970 0x7f932014fd30 unknown :-1 s=CLOSED pgs=2823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.744+0000 7f9326c47640 1 --2- 172.21.15.84:0/3381854974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f932006c040 0x7f9320153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.744+0000 7f9326c47640 1 -- 172.21.15.84:0/3381854974 >> 172.21.15.84:0/3381854974 conn(0x7f93200ce6a0 msgr2=0x7f932015db50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:13.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.744+0000 7f9326c47640 1 -- 172.21.15.84:0/3381854974 shutdown_connections 2023-10-16T11:25:13.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.744+0000 7f9326c47640 1 -- 172.21.15.84:0/3381854974 wait complete. 2023-10-16T11:25:13.763 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:25:13.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.971+0000 7f223ffa9640 1 Processor -- start 2023-10-16T11:25:13.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.971+0000 7f223ffa9640 1 -- start start 2023-10-16T11:25:13.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.972+0000 7f223ffa9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f223815aaa0 0x7f223815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:13.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.972+0000 7f223ffa9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815ae80 0x7f22380cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:13.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.972+0000 7f223ffa9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f223815ed60 con 0x7f22380ccaa0 2023-10-16T11:25:13.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.972+0000 7f223ffa9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f223806b810 con 0x7f223815aaa0 2023-10-16T11:25:13.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.972+0000 7f223ffa9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22380707e0 con 0x7f223815ae80 2023-10-16T11:25:13.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.972+0000 7f223dd45640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f223815aaa0 0x7f223815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:13.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.972+0000 7f223e546640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f22380ccaa0 0x7f2238070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36016/0 (socket says 172.21.15.84:36016) 2023-10-16T11:25:13.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.972+0000 7f223e546640 1 -- 172.21.15.84:0/3994407341 learned_addr learned my addr 172.21.15.84:0/3994407341 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:13.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.973+0000 7f223d544640 1 --2- 172.21.15.84:0/3994407341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815ae80 0x7f22380cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:13.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.973+0000 7f223dd45640 1 -- 172.21.15.84:0/3994407341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815ae80 msgr2=0x7f22380cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:13.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.973+0000 7f223dd45640 1 --2- 172.21.15.84:0/3994407341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815ae80 0x7f22380cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.973+0000 7f223dd45640 1 -- 172.21.15.84:0/3994407341 >> v1:172.21.15.84:6789/0 conn(0x7f22380ccaa0 legacy=0x7f2238070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:13.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.973+0000 7f223dd45640 1 -- 172.21.15.84:0/3994407341 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223815ef90 con 0x7f223815aaa0 2023-10-16T11:25:13.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.974+0000 7f223dd45640 1 --2- 172.21.15.84:0/3994407341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f223815aaa0 0x7f223815e820 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7f22340097a0 tx=0x7f223402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0ced2159166743 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:13.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.974+0000 7f223cd43640 1 -- 172.21.15.84:0/3994407341 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223402d040 con 0x7f223815aaa0 2023-10-16T11:25:13.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.974+0000 7f223cd43640 1 -- 172.21.15.84:0/3994407341 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2234031810 con 0x7f223815aaa0 2023-10-16T11:25:13.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.974+0000 7f223cd43640 1 -- 172.21.15.84:0/3994407341 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2234031ba0 con 0x7f223815aaa0 2023-10-16T11:25:13.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.975+0000 7f223ffa9640 1 -- 172.21.15.84:0/3994407341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f223815aaa0 msgr2=0x7f223815e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:13.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.975+0000 7f223ffa9640 1 --2- 172.21.15.84:0/3994407341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f223815aaa0 0x7f223815e820 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7f22340097a0 tx=0x7f223402b8b0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.975+0000 7f223ffa9640 1 -- 172.21.15.84:0/3994407341 shutdown_connections 2023-10-16T11:25:13.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.975+0000 7f223ffa9640 1 --2- 172.21.15.84:0/3994407341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815ae80 0x7f22380cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.975+0000 7f223ffa9640 1 --2- 172.21.15.84:0/3994407341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f223815aaa0 0x7f223815e820 unknown :-1 s=CLOSED pgs=2825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.975+0000 7f223ffa9640 1 -- 172.21.15.84:0/3994407341 >> 172.21.15.84:0/3994407341 conn(0x7f22380cb410 msgr2=0x7f22380cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:13.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.975+0000 7f223ffa9640 1 -- 172.21.15.84:0/3994407341 shutdown_connections 2023-10-16T11:25:13.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.975+0000 7f223ffa9640 1 -- 172.21.15.84:0/3994407341 wait complete. 2023-10-16T11:25:13.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.977+0000 7f223ffa9640 1 Processor -- start 2023-10-16T11:25:13.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.977+0000 7f223ffa9640 1 -- start start 2023-10-16T11:25:13.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.978+0000 7f223ffa9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22380ccaa0 0x7f2238150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:13.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.978+0000 7f223ffa9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f223815aaa0 0x7f22380c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:13.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.978+0000 7f223ffa9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815ae80 0x7f22380c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:13.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.978+0000 7f223ffa9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22380c3620 con 0x7f223815aaa0 2023-10-16T11:25:13.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.978+0000 7f223ffa9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22380c4490 con 0x7f22380ccaa0 2023-10-16T11:25:13.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.978+0000 7f223ffa9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2238150c80 con 0x7f223815ae80 2023-10-16T11:25:13.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.978+0000 7f223d544640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f223815aaa0 0x7f22380c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:13.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.978+0000 7f223d544640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f223815aaa0 0x7f22380c6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40448/0 (socket says 172.21.15.84:40448) 2023-10-16T11:25:13.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.978+0000 7f223d544640 1 -- 172.21.15.84:0/2520056070 learned_addr learned my addr 172.21.15.84:0/2520056070 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:13.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.979+0000 7f223dd45640 1 --2- 172.21.15.84:0/2520056070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22380ccaa0 0x7f2238150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:13.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.979+0000 7f223e546640 1 --2- 172.21.15.84:0/2520056070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815ae80 0x7f22380c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:13.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.979+0000 7f223d544640 1 -- 172.21.15.84:0/2520056070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815ae80 msgr2=0x7f22380c24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:13.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.979+0000 7f223d544640 1 --2- 172.21.15.84:0/2520056070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815ae80 0x7f22380c24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.979+0000 7f223d544640 1 -- 172.21.15.84:0/2520056070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22380ccaa0 msgr2=0x7f2238150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:13.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.979+0000 7f223d544640 1 --2- 172.21.15.84:0/2520056070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22380ccaa0 0x7f2238150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:13.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.979+0000 7f223d544640 1 -- 172.21.15.84:0/2520056070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2234009400 con 0x7f223815aaa0 2023-10-16T11:25:13.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.979+0000 7f223dd45640 1 --2- 172.21.15.84:0/2520056070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22380ccaa0 0x7f2238150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:13.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.979+0000 7f223d544640 1 --2- 172.21.15.84:0/2520056070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f223815aaa0 0x7f22380c6020 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7f222800d800 tx=0x7f222800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:13.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.979+0000 7f222e7fc640 1 -- 172.21.15.84:0/2520056070 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222800b6c0 con 0x7f223815aaa0 2023-10-16T11:25:13.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.979+0000 7f222e7fc640 1 -- 172.21.15.84:0/2520056070 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222800b820 con 0x7f223815aaa0 2023-10-16T11:25:13.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.979+0000 7f222e7fc640 1 -- 172.21.15.84:0/2520056070 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222800bbb0 con 0x7f223815aaa0 2023-10-16T11:25:13.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.979+0000 7f223ffa9640 1 -- 172.21.15.84:0/2520056070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22380c3060 con 0x7f223815aaa0 2023-10-16T11:25:13.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.979+0000 7f223ffa9640 1 -- 172.21.15.84:0/2520056070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223815b480 con 0x7f223815aaa0 2023-10-16T11:25:13.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.981+0000 7f223ffa9640 1 -- 172.21.15.84:0/2520056070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f223815b2e0 con 0x7f223815aaa0 2023-10-16T11:25:13.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.981+0000 7f222e7fc640 1 -- 172.21.15.84:0/2520056070 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f222800bd10 con 0x7f223815aaa0 2023-10-16T11:25:13.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.982+0000 7f222e7fc640 1 --2- 172.21.15.84:0/2520056070 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f221c06b0b0 0x7f221c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:13.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.983+0000 7f222e7fc640 1 -- 172.21.15.84:0/2520056070 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 7418+0+0 (secure 0 0 0) 0x7f222808a690 con 0x7f223815aaa0 2023-10-16T11:25:13.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.983+0000 7f223dd45640 1 --2- 172.21.15.84:0/2520056070 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f221c06b0b0 0x7f221c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:13.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.983+0000 7f223dd45640 1 --2- 172.21.15.84:0/2520056070 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f221c06b0b0 0x7f221c06d570 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2234036000 tx=0x7f223403c040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:13.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:13.987+0000 7f222e7fc640 1 -- 172.21.15.84:0/2520056070 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f223815b2e0 con 0x7f223815aaa0 2023-10-16T11:25:14.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.189+0000 7f223ffa9640 1 -- 172.21.15.84:0/2520056070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f223815b2e0 con 0x7f223815aaa0 2023-10-16T11:25:14.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.192+0000 7f222e7fc640 1 -- 172.21.15.84:0/2520056070 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v223) v1 ==== 74+0+21513 (secure 0 0 0) 0x7f223815b2e0 con 0x7f223815aaa0 2023-10-16T11:25:14.194 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:14.194 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":223,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:25:09.210192+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:24:23.363032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:24:24.551270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:24:38.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:24:39.387564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1287173368":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/595669474":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/4107174082":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2906390983":"2023-10-17T11:25:09.210125+0000","172.21.15.84:6833/2880641466":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/3419402221":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2161898622":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/455217819":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/4247809558":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/1814319372":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2877946366":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6833/1425409017":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6832/1425409017":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/1771162859":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6832/2880641466":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/3655599055":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/1076506559":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/2748224910":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/857172551":"2023-10-17T11:24:58.196616+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:25:14.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.194+0000 7f223ffa9640 1 -- 172.21.15.84:0/2520056070 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f221c06b0b0 msgr2=0x7f221c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:14.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.194+0000 7f223ffa9640 1 --2- 172.21.15.84:0/2520056070 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f221c06b0b0 0x7f221c06d570 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2234036000 tx=0x7f223403c040 comp rx=0 tx=0).stop 2023-10-16T11:25:14.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.194+0000 7f223ffa9640 1 -- 172.21.15.84:0/2520056070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f223815aaa0 msgr2=0x7f22380c6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:14.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.194+0000 7f223ffa9640 1 --2- 172.21.15.84:0/2520056070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f223815aaa0 0x7f22380c6020 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7f222800d800 tx=0x7f222800dcc0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.194+0000 7f223ffa9640 1 -- 172.21.15.84:0/2520056070 shutdown_connections 2023-10-16T11:25:14.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.194+0000 7f223ffa9640 1 --2- 172.21.15.84:0/2520056070 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f221c06b0b0 0x7f221c06d570 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.194+0000 7f223ffa9640 1 --2- 172.21.15.84:0/2520056070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f223815ae80 0x7f22380c24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.194+0000 7f223ffa9640 1 --2- 172.21.15.84:0/2520056070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f223815aaa0 0x7f22380c6020 unknown :-1 s=CLOSED pgs=2826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.194+0000 7f223ffa9640 1 --2- 172.21.15.84:0/2520056070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22380ccaa0 0x7f2238150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.194+0000 7f223ffa9640 1 -- 172.21.15.84:0/2520056070 >> 172.21.15.84:0/2520056070 conn(0x7f22380cb410 msgr2=0x7f22380ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:14.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.195+0000 7f223ffa9640 1 -- 172.21.15.84:0/2520056070 shutdown_connections 2023-10-16T11:25:14.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.195+0000 7f223ffa9640 1 -- 172.21.15.84:0/2520056070 wait complete. 2023-10-16T11:25:14.213 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T11:25:14.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.420+0000 7fd7a9941640 1 Processor -- start 2023-10-16T11:25:14.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.420+0000 7fd7a9941640 1 -- start start 2023-10-16T11:25:14.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.422+0000 7fd7a9941640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd7a406a4b0 0x7fd7a40ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:14.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.422+0000 7fd7a9941640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a40cb030 con 0x7fd7a415ad70 2023-10-16T11:25:14.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.422+0000 7fd7a9941640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a40c91c0 con 0x7fd7a40c9ca0 2023-10-16T11:25:14.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.422+0000 7fd7a9941640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a40c94a0 con 0x7fd7a406a4b0 2023-10-16T11:25:14.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.422+0000 7fd7a2ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd7a415ad70 0x7fd7a4069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36030/0 (socket says 172.21.15.84:36030) 2023-10-16T11:25:14.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.422+0000 7fd7a2ffd640 1 -- 172.21.15.84:0/3716071602 learned_addr learned my addr 172.21.15.84:0/3716071602 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:14.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.422+0000 7fd7a27fc640 1 --2- 172.21.15.84:0/3716071602 >> v2:172.21.15.84:3300/0 conn(0x7fd7a406a4b0 0x7fd7a40ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:14.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.423+0000 7fd7a1ffb640 1 -- 172.21.15.84:0/3716071602 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3840625176 0 0) 0x7fd7a40cb030 con 0x7fd7a415ad70 2023-10-16T11:25:14.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.423+0000 7fd7a1ffb640 1 -- 172.21.15.84:0/3716071602 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd790003520 con 0x7fd7a415ad70 2023-10-16T11:25:14.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.423+0000 7fd7a27fc640 1 -- 172.21.15.84:0/3716071602 >> v1:172.21.15.157:6789/0 conn(0x7fd7a40c9ca0 legacy=0x7fd7a40ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:14.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.423+0000 7fd7a27fc640 1 -- 172.21.15.84:0/3716071602 >> v1:172.21.15.84:6789/0 conn(0x7fd7a415ad70 legacy=0x7fd7a4069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:14.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.423+0000 7fd7a27fc640 1 -- 172.21.15.84:0/3716071602 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a40c9780 con 0x7fd7a406a4b0 2023-10-16T11:25:14.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.424+0000 7fd7a27fc640 1 --2- 172.21.15.84:0/3716071602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7a406a4b0 0x7fd7a40ca3b0 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0097a0 tx=0x7fd78c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc872502a7040a71 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:14.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.424+0000 7fd7a1ffb640 1 -- 172.21.15.84:0/3716071602 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c031830 con 0x7fd7a406a4b0 2023-10-16T11:25:14.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.424+0000 7fd7a1ffb640 1 -- 172.21.15.84:0/3716071602 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78c031990 con 0x7fd7a406a4b0 2023-10-16T11:25:14.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.424+0000 7fd7a1ffb640 1 -- 172.21.15.84:0/3716071602 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c031d20 con 0x7fd7a406a4b0 2023-10-16T11:25:14.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.425+0000 7fd7a9941640 1 -- 172.21.15.84:0/3716071602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7a406a4b0 msgr2=0x7fd7a40ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:14.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.425+0000 7fd7a9941640 1 --2- 172.21.15.84:0/3716071602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7a406a4b0 0x7fd7a40ca3b0 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0097a0 tx=0x7fd78c02b8d0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.425+0000 7fd7a9941640 1 -- 172.21.15.84:0/3716071602 shutdown_connections 2023-10-16T11:25:14.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.425+0000 7fd7a9941640 1 --2- 172.21.15.84:0/3716071602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7a406a4b0 0x7fd7a40ca3b0 unknown :-1 s=CLOSED pgs=2828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.425+0000 7fd7a9941640 1 -- 172.21.15.84:0/3716071602 >> 172.21.15.84:0/3716071602 conn(0x7fd7a40ce9b0 msgr2=0x7fd7a40cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:14.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.425+0000 7fd7a9941640 1 -- 172.21.15.84:0/3716071602 shutdown_connections 2023-10-16T11:25:14.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.425+0000 7fd7a9941640 1 -- 172.21.15.84:0/3716071602 wait complete. 2023-10-16T11:25:14.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.427+0000 7fd7a9941640 1 Processor -- start 2023-10-16T11:25:14.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.427+0000 7fd7a9941640 1 -- start start 2023-10-16T11:25:14.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.428+0000 7fd7a9941640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7a406a4b0 0x7fd7a4159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:14.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.428+0000 7fd7a9941640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7a40c9ca0 0x7fd7a41eff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:14.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.428+0000 7fd7a9941640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7a415ad70 0x7fd7a40a8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:14.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.428+0000 7fd7a9941640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a40c6c20 con 0x7fd7a415ad70 2023-10-16T11:25:14.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.428+0000 7fd7a9941640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a4159da0 con 0x7fd7a40c9ca0 2023-10-16T11:25:14.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.428+0000 7fd7a9941640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a40c6e70 con 0x7fd7a406a4b0 2023-10-16T11:25:14.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.428+0000 7fd7a37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7a415ad70 0x7fd7a40a8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:14.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.428+0000 7fd7a37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7a415ad70 0x7fd7a40a8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40464/0 (socket says 172.21.15.84:40464) 2023-10-16T11:25:14.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.428+0000 7fd7a37fe640 1 -- 172.21.15.84:0/84477996 learned_addr learned my addr 172.21.15.84:0/84477996 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:14.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.429+0000 7fd7a27fc640 1 --2- 172.21.15.84:0/84477996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7a40c9ca0 0x7fd7a41eff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:14.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.429+0000 7fd7a37fe640 1 -- 172.21.15.84:0/84477996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7a406a4b0 msgr2=0x7fd7a4159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:14.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.429+0000 7fd7a2ffd640 1 --2- 172.21.15.84:0/84477996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7a406a4b0 0x7fd7a4159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:14.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.429+0000 7fd7a37fe640 1 --2- 172.21.15.84:0/84477996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7a406a4b0 0x7fd7a4159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.429+0000 7fd7a37fe640 1 -- 172.21.15.84:0/84477996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7a40c9ca0 msgr2=0x7fd7a41eff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:14.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.429+0000 7fd7a37fe640 1 --2- 172.21.15.84:0/84477996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7a40c9ca0 0x7fd7a41eff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.429+0000 7fd7a37fe640 1 -- 172.21.15.84:0/84477996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c009400 con 0x7fd7a415ad70 2023-10-16T11:25:14.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.429+0000 7fd7a27fc640 1 --2- 172.21.15.84:0/84477996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7a40c9ca0 0x7fd7a41eff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:14.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.429+0000 7fd7a37fe640 1 --2- 172.21.15.84:0/84477996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7a415ad70 0x7fd7a40a8c70 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00c410 tx=0x7fd79c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:14.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.429+0000 7fd79bfff640 1 -- 172.21.15.84:0/84477996 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c011790 con 0x7fd7a415ad70 2023-10-16T11:25:14.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.429+0000 7fd79bfff640 1 -- 172.21.15.84:0/84477996 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79c0118f0 con 0x7fd7a415ad70 2023-10-16T11:25:14.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.429+0000 7fd79bfff640 1 -- 172.21.15.84:0/84477996 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c011ce0 con 0x7fd7a415ad70 2023-10-16T11:25:14.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.429+0000 7fd7a9941640 1 -- 172.21.15.84:0/84477996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a40c7150 con 0x7fd7a415ad70 2023-10-16T11:25:14.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.430+0000 7fd7a9941640 1 -- 172.21.15.84:0/84477996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a40c7710 con 0x7fd7a415ad70 2023-10-16T11:25:14.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.431+0000 7fd7a9941640 1 -- 172.21.15.84:0/84477996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a404e160 con 0x7fd7a415ad70 2023-10-16T11:25:14.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.431+0000 7fd79bfff640 1 -- 172.21.15.84:0/84477996 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd79c01c460 con 0x7fd7a415ad70 2023-10-16T11:25:14.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.432+0000 7fd79bfff640 1 --2- 172.21.15.84:0/84477996 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd78406b0b0 0x7fd78406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:14.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.433+0000 7fd79bfff640 1 -- 172.21.15.84:0/84477996 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 7418+0+0 (secure 0 0 0) 0x7fd79c089cc0 con 0x7fd7a415ad70 2023-10-16T11:25:14.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.433+0000 7fd7a2ffd640 1 --2- 172.21.15.84:0/84477996 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd78406b0b0 0x7fd78406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:14.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.433+0000 7fd7a2ffd640 1 --2- 172.21.15.84:0/84477996 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd78406b0b0 0x7fd78406d570 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd794001560 tx=0x7fd794009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:14.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.437+0000 7fd79bfff640 1 -- 172.21.15.84:0/84477996 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd79c054430 con 0x7fd7a415ad70 2023-10-16T11:25:14.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.672+0000 7fd7a9941640 1 -- 172.21.15.84:0/84477996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd7a40c8f00 con 0x7fd7a415ad70 2023-10-16T11:25:14.674 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-16T11:25:14.673+0000 7fca0c7ff640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:25:14.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.677+0000 7fd79bfff640 1 -- 172.21.15.84:0/84477996 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v121) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd79c057a50 con 0x7fd7a415ad70 2023-10-16T11:25:14.679 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:25:14.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.680+0000 7fd7a9941640 1 -- 172.21.15.84:0/84477996 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd78406b0b0 msgr2=0x7fd78406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:14.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.680+0000 7fd7a9941640 1 --2- 172.21.15.84:0/84477996 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd78406b0b0 0x7fd78406d570 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd794001560 tx=0x7fd794009040 comp rx=0 tx=0).stop 2023-10-16T11:25:14.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.680+0000 7fd7a9941640 1 -- 172.21.15.84:0/84477996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7a415ad70 msgr2=0x7fd7a40a8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:14.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.680+0000 7fd7a9941640 1 --2- 172.21.15.84:0/84477996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7a415ad70 0x7fd7a40a8c70 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00c410 tx=0x7fd79c00c8d0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.681+0000 7fd7a9941640 1 -- 172.21.15.84:0/84477996 shutdown_connections 2023-10-16T11:25:14.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.681+0000 7fd7a9941640 1 --2- 172.21.15.84:0/84477996 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd78406b0b0 0x7fd78406d570 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.681+0000 7fd7a9941640 1 --2- 172.21.15.84:0/84477996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7a415ad70 0x7fd7a40a8c70 unknown :-1 s=CLOSED pgs=2829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.681+0000 7fd7a9941640 1 --2- 172.21.15.84:0/84477996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7a40c9ca0 0x7fd7a41eff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.681+0000 7fd7a9941640 1 --2- 172.21.15.84:0/84477996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7a406a4b0 0x7fd7a4159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.681+0000 7fd7a9941640 1 -- 172.21.15.84:0/84477996 >> 172.21.15.84:0/84477996 conn(0x7fd7a40ce9b0 msgr2=0x7fd7a40afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:14.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.681+0000 7fd7a9941640 1 -- 172.21.15.84:0/84477996 shutdown_connections 2023-10-16T11:25:14.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.682+0000 7fd7a9941640 1 -- 172.21.15.84:0/84477996 wait complete. 2023-10-16T11:25:14.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T11:25:14.711 INFO:tasks.ceph.mds.a.smithi084.stderr: -4814> 2023-10-16T11:24:31.012+0000 7f63aab48640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:25:14.794 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T11:25:14.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.903+0000 7f36a121f640 1 Processor -- start 2023-10-16T11:25:14.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.903+0000 7f36a121f640 1 -- start start 2023-10-16T11:25:14.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.904+0000 7f36a121f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369c15ad70 0x7f369c069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:14.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.905+0000 7f36a121f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f369c0caac0 0x7f369c06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:14.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.905+0000 7f36a121f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f369c0cb030 con 0x7f369c0c9ca0 2023-10-16T11:25:14.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.905+0000 7f36a121f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f369c0c91c0 con 0x7f369c0caac0 2023-10-16T11:25:14.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.905+0000 7f36a121f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369c0c94a0 con 0x7f369c15ad70 2023-10-16T11:25:14.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.905+0000 7f369b59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f369c0caac0 0x7f369c06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:14.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.905+0000 7f369b59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f369c0caac0 0x7f369c06a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40486/0 (socket says 172.21.15.84:40486) 2023-10-16T11:25:14.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.905+0000 7f369b59e640 1 -- 172.21.15.84:0/3633347484 learned_addr learned my addr 172.21.15.84:0/3633347484 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:14.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.905+0000 7f369ad9d640 1 --2- 172.21.15.84:0/3633347484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369c15ad70 0x7f369c069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:14.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.906+0000 7f369b59e640 1 -- 172.21.15.84:0/3633347484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369c15ad70 msgr2=0x7f369c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:14.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.906+0000 7f369b59e640 1 --2- 172.21.15.84:0/3633347484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369c15ad70 0x7f369c069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.906+0000 7f369b59e640 1 -- 172.21.15.84:0/3633347484 >> v1:172.21.15.157:6789/0 conn(0x7f369c0c9ca0 legacy=0x7f369c0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:14.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.906+0000 7f369b59e640 1 -- 172.21.15.84:0/3633347484 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c0c9780 con 0x7f369c0caac0 2023-10-16T11:25:14.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.907+0000 7f369b59e640 1 --2- 172.21.15.84:0/3633347484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f369c0caac0 0x7f369c06a2e0 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f3694009e50 tx=0x7f369402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa75a1fa97f83e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:14.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.907+0000 7f369a59c640 1 -- 172.21.15.84:0/3633347484 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369402ca70 con 0x7f369c0caac0 2023-10-16T11:25:14.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.907+0000 7f369a59c640 1 -- 172.21.15.84:0/3633347484 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369402cbd0 con 0x7f369c0caac0 2023-10-16T11:25:14.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.907+0000 7f369a59c640 1 -- 172.21.15.84:0/3633347484 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36940343f0 con 0x7f369c0caac0 2023-10-16T11:25:14.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.907+0000 7f36a121f640 1 -- 172.21.15.84:0/3633347484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f369c0caac0 msgr2=0x7f369c06a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:14.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.907+0000 7f36a121f640 1 --2- 172.21.15.84:0/3633347484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f369c0caac0 0x7f369c06a2e0 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f3694009e50 tx=0x7f369402bed0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.908+0000 7f36a121f640 1 -- 172.21.15.84:0/3633347484 shutdown_connections 2023-10-16T11:25:14.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.908+0000 7f36a121f640 1 --2- 172.21.15.84:0/3633347484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f369c0caac0 0x7f369c06a2e0 unknown :-1 s=CLOSED pgs=2833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.908+0000 7f36a121f640 1 --2- 172.21.15.84:0/3633347484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369c15ad70 0x7f369c069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.908+0000 7f36a121f640 1 -- 172.21.15.84:0/3633347484 >> 172.21.15.84:0/3633347484 conn(0x7f369c0ce9b0 msgr2=0x7f369c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:14.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.908+0000 7f36a121f640 1 -- 172.21.15.84:0/3633347484 shutdown_connections 2023-10-16T11:25:14.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.908+0000 7f36a121f640 1 -- 172.21.15.84:0/3633347484 wait complete. 2023-10-16T11:25:14.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.909+0000 7f36a121f640 1 Processor -- start 2023-10-16T11:25:14.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.910+0000 7f36a121f640 1 -- start start 2023-10-16T11:25:14.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.910+0000 7f36a121f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369c0c9ca0 0x7f369c1f4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:14.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.910+0000 7f36a121f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f369c0caac0 0x7f369c15be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:14.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.910+0000 7f36a121f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f369c15ad70 0x7f369c1efad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:14.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.910+0000 7f36a121f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c154160 con 0x7f369c15ad70 2023-10-16T11:25:14.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.910+0000 7f36a121f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c0c5a80 con 0x7f369c0caac0 2023-10-16T11:25:14.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.910+0000 7f36a121f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369c15c3a0 con 0x7f369c0c9ca0 2023-10-16T11:25:14.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.911+0000 7f369b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f369c15ad70 0x7f369c1efad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:14.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.911+0000 7f369b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f369c15ad70 0x7f369c1efad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40494/0 (socket says 172.21.15.84:40494) 2023-10-16T11:25:14.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.911+0000 7f369b59e640 1 -- 172.21.15.84:0/2110278047 learned_addr learned my addr 172.21.15.84:0/2110278047 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:14.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.911+0000 7f369ad9d640 1 --2- 172.21.15.84:0/2110278047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369c0c9ca0 0x7f369c1f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:14.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.911+0000 7f3693fff640 1 --2- 172.21.15.84:0/2110278047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f369c0caac0 0x7f369c15be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:14.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.911+0000 7f369b59e640 1 -- 172.21.15.84:0/2110278047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369c0c9ca0 msgr2=0x7f369c1f4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:14.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.911+0000 7f369b59e640 1 --2- 172.21.15.84:0/2110278047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369c0c9ca0 0x7f369c1f4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.911+0000 7f369b59e640 1 -- 172.21.15.84:0/2110278047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f369c0caac0 msgr2=0x7f369c15be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:14.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.911+0000 7f369b59e640 1 --2- 172.21.15.84:0/2110278047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f369c0caac0 0x7f369c15be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:14.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.911+0000 7f369b59e640 1 -- 172.21.15.84:0/2110278047 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3694009b30 con 0x7f369c15ad70 2023-10-16T11:25:14.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.911+0000 7f3693fff640 1 --2- 172.21.15.84:0/2110278047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f369c0caac0 0x7f369c15be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:14.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.912+0000 7f369b59e640 1 --2- 172.21.15.84:0/2110278047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f369c15ad70 0x7f369c1efad0 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7f3694009e50 tx=0x7f3694004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:14.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.912+0000 7f36937fe640 1 -- 172.21.15.84:0/2110278047 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3694009510 con 0x7f369c15ad70 2023-10-16T11:25:14.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.912+0000 7f36937fe640 1 -- 172.21.15.84:0/2110278047 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369403e450 con 0x7f369c15ad70 2023-10-16T11:25:14.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.912+0000 7f36937fe640 1 -- 172.21.15.84:0/2110278047 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369403e800 con 0x7f369c15ad70 2023-10-16T11:25:14.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.912+0000 7f36a121f640 1 -- 172.21.15.84:0/2110278047 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f369c15c670 con 0x7f369c15ad70 2023-10-16T11:25:14.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.912+0000 7f36a121f640 1 -- 172.21.15.84:0/2110278047 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369c15cc30 con 0x7f369c15ad70 2023-10-16T11:25:14.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.913+0000 7f36a121f640 1 -- 172.21.15.84:0/2110278047 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f369c0ad4b0 con 0x7f369c15ad70 2023-10-16T11:25:14.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.914+0000 7f36937fe640 1 -- 172.21.15.84:0/2110278047 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f369403e960 con 0x7f369c15ad70 2023-10-16T11:25:14.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.915+0000 7f36937fe640 1 --2- 172.21.15.84:0/2110278047 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f368006b0b0 0x7f368006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:14.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.915+0000 7f36937fe640 1 -- 172.21.15.84:0/2110278047 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 7504+0+0 (secure 0 0 0) 0x7f36940aced0 con 0x7f369c15ad70 2023-10-16T11:25:14.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.916+0000 7f369ad9d640 1 --2- 172.21.15.84:0/2110278047 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f368006b0b0 0x7f368006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:14.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.916+0000 7f369ad9d640 1 --2- 172.21.15.84:0/2110278047 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f368006b0b0 0x7f368006d570 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f368c006fd0 tx=0x7f368c008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:14.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:14.919+0000 7f36937fe640 1 -- 172.21.15.84:0/2110278047 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f369c0ad4b0 con 0x7f369c15ad70 2023-10-16T11:25:15.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.157+0000 7f36a121f640 1 -- 172.21.15.84:0/2110278047 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f369c0ad4b0 con 0x7f369c15ad70 2023-10-16T11:25:15.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.680+0000 7f36937fe640 1 -- 172.21.15.84:0/2110278047 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v122) v1 ==== 104+0+0 (secure 0 0 0) 0x7f369c0ad4b0 con 0x7f369c15ad70 2023-10-16T11:25:15.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.684+0000 7f36a121f640 1 -- 172.21.15.84:0/2110278047 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f368006b0b0 msgr2=0x7f368006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:15.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.684+0000 7f36a121f640 1 --2- 172.21.15.84:0/2110278047 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f368006b0b0 0x7f368006d570 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f368c006fd0 tx=0x7f368c008040 comp rx=0 tx=0).stop 2023-10-16T11:25:15.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.684+0000 7f36a121f640 1 -- 172.21.15.84:0/2110278047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f369c15ad70 msgr2=0x7f369c1efad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:15.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.684+0000 7f36a121f640 1 --2- 172.21.15.84:0/2110278047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f369c15ad70 0x7f369c1efad0 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7f3694009e50 tx=0x7f3694004a40 comp rx=0 tx=0).stop 2023-10-16T11:25:15.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.685+0000 7f36a121f640 1 -- 172.21.15.84:0/2110278047 shutdown_connections 2023-10-16T11:25:15.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.685+0000 7f36a121f640 1 --2- 172.21.15.84:0/2110278047 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f368006b0b0 0x7f368006d570 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:15.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.685+0000 7f36a121f640 1 --2- 172.21.15.84:0/2110278047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f369c15ad70 0x7f369c1efad0 unknown :-1 s=CLOSED pgs=2834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:15.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.685+0000 7f36a121f640 1 --2- 172.21.15.84:0/2110278047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f369c0caac0 0x7f369c15be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:15.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.685+0000 7f36a121f640 1 --2- 172.21.15.84:0/2110278047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f369c0c9ca0 0x7f369c1f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:15.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.685+0000 7f36a121f640 1 -- 172.21.15.84:0/2110278047 >> 172.21.15.84:0/2110278047 conn(0x7f369c0ce9b0 msgr2=0x7f369c0ca3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:15.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.685+0000 7f36a121f640 1 -- 172.21.15.84:0/2110278047 shutdown_connections 2023-10-16T11:25:15.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.685+0000 7f36a121f640 1 -- 172.21.15.84:0/2110278047 wait complete. 2023-10-16T11:25:15.707 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T11:25:15.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.924+0000 7f04efd47640 1 Processor -- start 2023-10-16T11:25:15.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.924+0000 7f04efd47640 1 -- start start 2023-10-16T11:25:15.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.925+0000 7f04efd47640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f04e815ac10 0x7f04e806c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:15.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.925+0000 7f04efd47640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04e80b0360 0x7f04e80b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:15.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.925+0000 7f04efd47640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04e80b0d00 con 0x7f04e80ca510 2023-10-16T11:25:15.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.925+0000 7f04efd47640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04e815f800 con 0x7f04e815ac10 2023-10-16T11:25:15.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.925+0000 7f04efd47640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e815b310 con 0x7f04e80b0360 2023-10-16T11:25:15.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.926+0000 7f04edae3640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f04e815ac10 0x7f04e806c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:15.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.926+0000 7f04edae3640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f04e815ac10 0x7f04e806c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42464/0 (socket says 172.21.15.84:42464) 2023-10-16T11:25:15.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.926+0000 7f04edae3640 1 -- 172.21.15.84:0/745852191 learned_addr learned my addr 172.21.15.84:0/745852191 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.926+0000 7f04edae3640 1 -- 172.21.15.84:0/745852191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04e80b0360 msgr2=0x7f04e80b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.926+0000 7f04edae3640 1 --2- 172.21.15.84:0/745852191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04e80b0360 0x7f04e80b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.926+0000 7f04edae3640 1 -- 172.21.15.84:0/745852191 >> v1:172.21.15.157:6789/0 conn(0x7f04e80ca510 legacy=0x7f04e80ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.926+0000 7f04edae3640 1 -- 172.21.15.84:0/745852191 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e806f640 con 0x7f04e815ac10 2023-10-16T11:25:15.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.927+0000 7f04edae3640 1 --2- 172.21.15.84:0/745852191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e815ac10 0x7f04e806c730 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009870 tx=0x7f04d802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68e928a7a36a81b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:15.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.928+0000 7f04ecae1640 1 -- 172.21.15.84:0/745852191 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d80317c0 con 0x7f04e815ac10 2023-10-16T11:25:15.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.928+0000 7f04ecae1640 1 -- 172.21.15.84:0/745852191 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d8031920 con 0x7f04e815ac10 2023-10-16T11:25:15.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.928+0000 7f04ecae1640 1 -- 172.21.15.84:0/745852191 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d8031cb0 con 0x7f04e815ac10 2023-10-16T11:25:15.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.929+0000 7f04efd47640 1 -- 172.21.15.84:0/745852191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e815ac10 msgr2=0x7f04e806c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:15.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.929+0000 7f04efd47640 1 --2- 172.21.15.84:0/745852191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e815ac10 0x7f04e806c730 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009870 tx=0x7f04d802b860 comp rx=0 tx=0).stop 2023-10-16T11:25:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.929+0000 7f04efd47640 1 -- 172.21.15.84:0/745852191 shutdown_connections 2023-10-16T11:25:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.929+0000 7f04efd47640 1 --2- 172.21.15.84:0/745852191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04e80b0360 0x7f04e80b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.929+0000 7f04efd47640 1 --2- 172.21.15.84:0/745852191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e815ac10 0x7f04e806c730 unknown :-1 s=CLOSED pgs=2835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.929+0000 7f04efd47640 1 -- 172.21.15.84:0/745852191 >> 172.21.15.84:0/745852191 conn(0x7f04e80cb360 msgr2=0x7f04e80cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.929+0000 7f04efd47640 1 -- 172.21.15.84:0/745852191 shutdown_connections 2023-10-16T11:25:15.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.929+0000 7f04efd47640 1 -- 172.21.15.84:0/745852191 wait complete. 2023-10-16T11:25:15.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.931+0000 7f04efd47640 1 Processor -- start 2023-10-16T11:25:15.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.931+0000 7f04efd47640 1 -- start start 2023-10-16T11:25:15.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.933+0000 7f04efd47640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e80b0360 0x7f04e80c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:15.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.933+0000 7f04efd47640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04e80ca510 0x7f04e814c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:15.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.933+0000 7f04efd47640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04e815ac10 0x7f04e814cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:15.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.933+0000 7f04efd47640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e80c7890 con 0x7f04e80b0360 2023-10-16T11:25:15.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.933+0000 7f04efd47640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e80c6c10 con 0x7f04e815ac10 2023-10-16T11:25:15.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.933+0000 7f04efd47640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e80c6ef0 con 0x7f04e80ca510 2023-10-16T11:25:15.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.933+0000 7f04edae3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e80b0360 0x7f04e80c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:15.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.933+0000 7f04edae3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e80b0360 0x7f04e80c63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42474/0 (socket says 172.21.15.84:42474) 2023-10-16T11:25:15.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.933+0000 7f04edae3640 1 -- 172.21.15.84:0/1768690084 learned_addr learned my addr 172.21.15.84:0/1768690084 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:15.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.933+0000 7f04edae3640 1 -- 172.21.15.84:0/1768690084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04e80ca510 msgr2=0x7f04e814c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.933+0000 7f04edae3640 1 --2- 172.21.15.84:0/1768690084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04e80ca510 0x7f04e814c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.933+0000 7f04edae3640 1 -- 172.21.15.84:0/1768690084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04e815ac10 msgr2=0x7f04e814cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.933+0000 7f04edae3640 1 --2- 172.21.15.84:0/1768690084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04e815ac10 0x7f04e814cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.933+0000 7f04edae3640 1 -- 172.21.15.84:0/1768690084 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d80094d0 con 0x7f04e80b0360 2023-10-16T11:25:15.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.934+0000 7f04edae3640 1 --2- 172.21.15.84:0/1768690084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e80b0360 0x7f04e80c63c0 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f04d8000c00 tx=0x7f04d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:15.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.934+0000 7f04e67fc640 1 -- 172.21.15.84:0/1768690084 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d8032860 con 0x7f04e80b0360 2023-10-16T11:25:15.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.934+0000 7f04e67fc640 1 -- 172.21.15.84:0/1768690084 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d80329e0 con 0x7f04e80b0360 2023-10-16T11:25:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.934+0000 7f04efd47640 1 -- 172.21.15.84:0/1768690084 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e80c71d0 con 0x7f04e80b0360 2023-10-16T11:25:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.934+0000 7f04e67fc640 1 -- 172.21.15.84:0/1768690084 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d8032d90 con 0x7f04e80b0360 2023-10-16T11:25:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.934+0000 7f04efd47640 1 -- 172.21.15.84:0/1768690084 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e80c6900 con 0x7f04e80b0360 2023-10-16T11:25:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.935+0000 7f04efd47640 1 -- 172.21.15.84:0/1768690084 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e80c79d0 con 0x7f04e80b0360 2023-10-16T11:25:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.936+0000 7f04e67fc640 1 -- 172.21.15.84:0/1768690084 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f04d804d3f0 con 0x7f04e80b0360 2023-10-16T11:25:15.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.937+0000 7f04e67fc640 1 --2- 172.21.15.84:0/1768690084 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f04d406b0b0 0x7f04d406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:15.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.937+0000 7f04e67fc640 1 -- 172.21.15.84:0/1768690084 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 7504+0+0 (secure 0 0 0) 0x7f04d80ac080 con 0x7f04e80b0360 2023-10-16T11:25:15.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.938+0000 7f04ed2e2640 1 --2- 172.21.15.84:0/1768690084 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f04d406b0b0 0x7f04d406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:15.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.938+0000 7f04ed2e2640 1 --2- 172.21.15.84:0/1768690084 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f04d406b0b0 0x7f04d406d570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f04e80cd290 tx=0x7f04dc006bf0 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:15.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:15.941+0000 7f04e67fc640 1 -- 172.21.15.84:0/1768690084 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f04d80defe0 con 0x7f04e80b0360 2023-10-16T11:25:16.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:16.137+0000 7f04efd47640 1 -- 172.21.15.84:0/1768690084 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f04e804e160 con 0x7f04e80b0360 2023-10-16T11:25:16.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:16.965+0000 7f04e67fc640 1 -- 172.21.15.84:0/1768690084 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v225) v1 ==== 181+0+0 (secure 0 0 0) 0x7f04e80c79d0 con 0x7f04e80b0360 2023-10-16T11:25:16.967 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-16T11:25:16.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:16.968+0000 7f04efd47640 1 -- 172.21.15.84:0/1768690084 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f04d406b0b0 msgr2=0x7f04d406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:16.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:16.968+0000 7f04efd47640 1 --2- 172.21.15.84:0/1768690084 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f04d406b0b0 0x7f04d406d570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f04e80cd290 tx=0x7f04dc006bf0 comp rx=0 tx=0).stop 2023-10-16T11:25:16.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:16.968+0000 7f04efd47640 1 -- 172.21.15.84:0/1768690084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e80b0360 msgr2=0x7f04e80c63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:16.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:16.968+0000 7f04efd47640 1 --2- 172.21.15.84:0/1768690084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e80b0360 0x7f04e80c63c0 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f04d8000c00 tx=0x7f04d80046c0 comp rx=0 tx=0).stop 2023-10-16T11:25:16.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:16.969+0000 7f04efd47640 1 -- 172.21.15.84:0/1768690084 shutdown_connections 2023-10-16T11:25:16.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:16.969+0000 7f04efd47640 1 --2- 172.21.15.84:0/1768690084 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f04d406b0b0 0x7f04d406d570 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:16.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:16.969+0000 7f04efd47640 1 --2- 172.21.15.84:0/1768690084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04e815ac10 0x7f04e814cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:16.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:16.969+0000 7f04efd47640 1 --2- 172.21.15.84:0/1768690084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04e80ca510 0x7f04e814c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:16.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:16.969+0000 7f04efd47640 1 --2- 172.21.15.84:0/1768690084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e80b0360 0x7f04e80c63c0 unknown :-1 s=CLOSED pgs=2836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:16.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:16.969+0000 7f04efd47640 1 -- 172.21.15.84:0/1768690084 >> 172.21.15.84:0/1768690084 conn(0x7f04e80cb360 msgr2=0x7f04e80c9970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:16.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:16.969+0000 7f04efd47640 1 -- 172.21.15.84:0/1768690084 shutdown_connections 2023-10-16T11:25:16.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:16.969+0000 7f04efd47640 1 -- 172.21.15.84:0/1768690084 wait complete. 2023-10-16T11:25:16.995 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T11:25:17.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.197+0000 7f8a76b1d640 1 Processor -- start 2023-10-16T11:25:17.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.198+0000 7f8a76b1d640 1 -- start start 2023-10-16T11:25:17.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.199+0000 7f8a76b1d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a700ce690 0x7f8a700c9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:17.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.199+0000 7f8a76b1d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8a700ae220 0x7f8a7006e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:17.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.199+0000 7f8a76b1d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8a7006adb0 0x7f8a7006b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:17.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.199+0000 7f8a76b1d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7006b780 con 0x7f8a7006adb0 2023-10-16T11:25:17.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.199+0000 7f8a76b1d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a700c9d60 con 0x7f8a700ae220 2023-10-16T11:25:17.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.199+0000 7f8a76b1d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7006e6b0 con 0x7f8a700ce690 2023-10-16T11:25:17.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.199+0000 7f8a750ba640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8a7006adb0 0x7f8a7006b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:17.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.199+0000 7f8a750ba640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8a7006adb0 0x7f8a7006b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42486/0 (socket says 172.21.15.84:42486) 2023-10-16T11:25:17.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.199+0000 7f8a750ba640 1 -- 172.21.15.84:0/1264605920 learned_addr learned my addr 172.21.15.84:0/1264605920 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:17.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.200+0000 7f8a748b9640 1 --2- 172.21.15.84:0/1264605920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a700ce690 0x7f8a700c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:17.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.200+0000 7f8a750ba640 1 -- 172.21.15.84:0/1264605920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a700ce690 msgr2=0x7f8a700c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:17.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.200+0000 7f8a6ffff640 1 --2- 172.21.15.84:0/1264605920 >> v2:172.21.15.157:3300/0 conn(0x7f8a700ae220 0x7f8a7006e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:17.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.200+0000 7f8a750ba640 1 --2- 172.21.15.84:0/1264605920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a700ce690 0x7f8a700c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:17.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.200+0000 7f8a750ba640 1 -- 172.21.15.84:0/1264605920 >> v2:172.21.15.157:3300/0 conn(0x7f8a700ae220 msgr2=0x7f8a7006e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:17.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.200+0000 7f8a750ba640 1 --2- 172.21.15.84:0/1264605920 >> v2:172.21.15.157:3300/0 conn(0x7f8a700ae220 0x7f8a7006e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:17.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.200+0000 7f8a750ba640 1 -- 172.21.15.84:0/1264605920 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a700cae30 con 0x7f8a7006adb0 2023-10-16T11:25:17.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.201+0000 7f8a750ba640 1 --2- 172.21.15.84:0/1264605920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7006adb0 0x7f8a7006b240 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f8a70067dc0 tx=0x7f8a6802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db3a0f381498101 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:17.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.201+0000 7f8a6f7fe640 1 -- 172.21.15.84:0/1264605920 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a680317e0 con 0x7f8a7006adb0 2023-10-16T11:25:17.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.201+0000 7f8a6f7fe640 1 -- 172.21.15.84:0/1264605920 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a68031940 con 0x7f8a7006adb0 2023-10-16T11:25:17.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.201+0000 7f8a6f7fe640 1 -- 172.21.15.84:0/1264605920 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a68031cd0 con 0x7f8a7006adb0 2023-10-16T11:25:17.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.202+0000 7f8a76b1d640 1 -- 172.21.15.84:0/1264605920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7006adb0 msgr2=0x7f8a7006b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:17.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.202+0000 7f8a76b1d640 1 --2- 172.21.15.84:0/1264605920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7006adb0 0x7f8a7006b240 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f8a70067dc0 tx=0x7f8a6802b880 comp rx=0 tx=0).stop 2023-10-16T11:25:17.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.202+0000 7f8a76b1d640 1 -- 172.21.15.84:0/1264605920 shutdown_connections 2023-10-16T11:25:17.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.202+0000 7f8a76b1d640 1 --2- 172.21.15.84:0/1264605920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7006adb0 0x7f8a7006b240 unknown :-1 s=CLOSED pgs=2837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:17.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.202+0000 7f8a76b1d640 1 --2- 172.21.15.84:0/1264605920 >> v2:172.21.15.157:3300/0 conn(0x7f8a700ae220 0x7f8a7006e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:17.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.202+0000 7f8a76b1d640 1 --2- 172.21.15.84:0/1264605920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a700ce690 0x7f8a700c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:17.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.202+0000 7f8a76b1d640 1 -- 172.21.15.84:0/1264605920 >> 172.21.15.84:0/1264605920 conn(0x7f8a700b0cc0 msgr2=0x7f8a700b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:17.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.202+0000 7f8a76b1d640 1 -- 172.21.15.84:0/1264605920 shutdown_connections 2023-10-16T11:25:17.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.202+0000 7f8a76b1d640 1 -- 172.21.15.84:0/1264605920 wait complete. 2023-10-16T11:25:17.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.205+0000 7f8a76b1d640 1 Processor -- start 2023-10-16T11:25:17.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.205+0000 7f8a76b1d640 1 -- start start 2023-10-16T11:25:17.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.208+0000 7f8a76b1d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7006adb0 0x7f8a7009d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:17.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.208+0000 7f8a76b1d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a700ae220 0x7f8a700b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:17.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.208+0000 7f8a76b1d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a700ce690 0x7f8a7006dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:17.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.208+0000 7f8a76b1d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700b39a0 con 0x7f8a7006adb0 2023-10-16T11:25:17.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.208+0000 7f8a76b1d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700b4530 con 0x7f8a700ce690 2023-10-16T11:25:17.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.208+0000 7f8a76b1d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700b4810 con 0x7f8a700ae220 2023-10-16T11:25:17.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.208+0000 7f8a748b9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7006adb0 0x7f8a7009d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:17.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.208+0000 7f8a748b9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7006adb0 0x7f8a7009d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42502/0 (socket says 172.21.15.84:42502) 2023-10-16T11:25:17.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.208+0000 7f8a748b9640 1 -- 172.21.15.84:0/209840250 learned_addr learned my addr 172.21.15.84:0/209840250 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:17.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.209+0000 7f8a6ffff640 1 --2- 172.21.15.84:0/209840250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a700ae220 0x7f8a700b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:17.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.209+0000 7f8a750ba640 1 --2- 172.21.15.84:0/209840250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a700ce690 0x7f8a7006dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:17.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.209+0000 7f8a748b9640 1 -- 172.21.15.84:0/209840250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a700ae220 msgr2=0x7f8a700b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:17.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.209+0000 7f8a748b9640 1 --2- 172.21.15.84:0/209840250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a700ae220 0x7f8a700b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:17.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.209+0000 7f8a748b9640 1 -- 172.21.15.84:0/209840250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a700ce690 msgr2=0x7f8a7006dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:17.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.209+0000 7f8a748b9640 1 --2- 172.21.15.84:0/209840250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a700ce690 0x7f8a7006dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:17.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.209+0000 7f8a748b9640 1 -- 172.21.15.84:0/209840250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a680094d0 con 0x7f8a7006adb0 2023-10-16T11:25:17.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.209+0000 7f8a6ffff640 1 --2- 172.21.15.84:0/209840250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a700ae220 0x7f8a700b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:17.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.209+0000 7f8a750ba640 1 --2- 172.21.15.84:0/209840250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a700ce690 0x7f8a7006dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:17.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.209+0000 7f8a748b9640 1 --2- 172.21.15.84:0/209840250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7006adb0 0x7f8a7009d3b0 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000ce10 tx=0x7f8a6000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:17.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.210+0000 7f8a6d7fa640 1 -- 172.21.15.84:0/209840250 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60003b90 con 0x7f8a7006adb0 2023-10-16T11:25:17.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.210+0000 7f8a6d7fa640 1 -- 172.21.15.84:0/209840250 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a60003cf0 con 0x7f8a7006adb0 2023-10-16T11:25:17.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.210+0000 7f8a6d7fa640 1 -- 172.21.15.84:0/209840250 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60003ea0 con 0x7f8a7006adb0 2023-10-16T11:25:17.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.210+0000 7f8a76b1d640 1 -- 172.21.15.84:0/209840250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a700b4af0 con 0x7f8a7006adb0 2023-10-16T11:25:17.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.211+0000 7f8a76b1d640 1 -- 172.21.15.84:0/209840250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7006e1d0 con 0x7f8a7006adb0 2023-10-16T11:25:17.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.212+0000 7f8a76b1d640 1 -- 172.21.15.84:0/209840250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a700c4200 con 0x7f8a7006adb0 2023-10-16T11:25:17.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.213+0000 7f8a6d7fa640 1 -- 172.21.15.84:0/209840250 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f8a60004240 con 0x7f8a7006adb0 2023-10-16T11:25:17.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.214+0000 7f8a6d7fa640 1 --2- 172.21.15.84:0/209840250 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8a5006b100 0x7f8a5006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:17.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.214+0000 7f8a6d7fa640 1 -- 172.21.15.84:0/209840250 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f8a60012070 con 0x7f8a7006adb0 2023-10-16T11:25:17.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.215+0000 7f8a6ffff640 1 --2- 172.21.15.84:0/209840250 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8a5006b100 0x7f8a5006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:17.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.215+0000 7f8a6ffff640 1 --2- 172.21.15.84:0/209840250 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8a5006b100 0x7f8a5006d5c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8a700c5970 tx=0x7f8a58008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:17.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.218+0000 7f8a6d7fa640 1 -- 172.21.15.84:0/209840250 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8a60055190 con 0x7f8a7006adb0 2023-10-16T11:25:17.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.418+0000 7f8a76b1d640 1 -- 172.21.15.84:0/209840250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8a7004e160 con 0x7f8a7006adb0 2023-10-16T11:25:17.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.990+0000 7f8a6d7fa640 1 -- 172.21.15.84:0/209840250 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v226) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8a600587b0 con 0x7f8a7006adb0 2023-10-16T11:25:17.992 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-16T11:25:17.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.994+0000 7f8a76b1d640 1 -- 172.21.15.84:0/209840250 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8a5006b100 msgr2=0x7f8a5006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:17.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.994+0000 7f8a76b1d640 1 --2- 172.21.15.84:0/209840250 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8a5006b100 0x7f8a5006d5c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8a700c5970 tx=0x7f8a58008040 comp rx=0 tx=0).stop 2023-10-16T11:25:17.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.994+0000 7f8a76b1d640 1 -- 172.21.15.84:0/209840250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7006adb0 msgr2=0x7f8a7009d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:17.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.994+0000 7f8a76b1d640 1 --2- 172.21.15.84:0/209840250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7006adb0 0x7f8a7009d3b0 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000ce10 tx=0x7f8a6000a660 comp rx=0 tx=0).stop 2023-10-16T11:25:17.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.995+0000 7f8a76b1d640 1 -- 172.21.15.84:0/209840250 shutdown_connections 2023-10-16T11:25:17.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.995+0000 7f8a76b1d640 1 --2- 172.21.15.84:0/209840250 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8a5006b100 0x7f8a5006d5c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:17.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.995+0000 7f8a76b1d640 1 --2- 172.21.15.84:0/209840250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a700ce690 0x7f8a7006dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:17.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.995+0000 7f8a76b1d640 1 --2- 172.21.15.84:0/209840250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a700ae220 0x7f8a700b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:17.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.995+0000 7f8a76b1d640 1 --2- 172.21.15.84:0/209840250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7006adb0 0x7f8a7009d3b0 unknown :-1 s=CLOSED pgs=2838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:17.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.995+0000 7f8a76b1d640 1 -- 172.21.15.84:0/209840250 >> 172.21.15.84:0/209840250 conn(0x7f8a700b0cc0 msgr2=0x7f8a700b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:17.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.997+0000 7f8a76b1d640 1 -- 172.21.15.84:0/209840250 shutdown_connections 2023-10-16T11:25:17.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:17.997+0000 7f8a76b1d640 1 -- 172.21.15.84:0/209840250 wait complete. 2023-10-16T11:25:18.022 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:25:18.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.225+0000 7fd96737c640 1 Processor -- start 2023-10-16T11:25:18.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.225+0000 7fd96737c640 1 -- start start 2023-10-16T11:25:18.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.226+0000 7fd96737c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd960159f00 0x7fd9600b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:18.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.226+0000 7fd96737c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd96015d880 0x7fd9600b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:18.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.226+0000 7fd96737c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd96006f7f0 con 0x7fd960159b20 2023-10-16T11:25:18.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.226+0000 7fd96737c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd96006c5e0 con 0x7fd96015d880 2023-10-16T11:25:18.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.226+0000 7fd96737c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96006c8c0 con 0x7fd960159f00 2023-10-16T11:25:18.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.227+0000 7fd965118640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd960159b20 0x7fd96006bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49092/0 (socket says 172.21.15.84:49092) 2023-10-16T11:25:18.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.227+0000 7fd965118640 1 -- 172.21.15.84:0/608665785 learned_addr learned my addr 172.21.15.84:0/608665785 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:18.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.227+0000 7fd957fff640 1 -- 172.21.15.84:0/608665785 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448464166 0 0) 0x7fd96006f7f0 con 0x7fd960159b20 2023-10-16T11:25:18.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.227+0000 7fd957fff640 1 -- 172.21.15.84:0/608665785 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd94c003520 con 0x7fd960159b20 2023-10-16T11:25:18.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.227+0000 7fd965919640 1 --2- 172.21.15.84:0/608665785 >> v2:172.21.15.157:3300/0 conn(0x7fd96015d880 0x7fd9600b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:18.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.227+0000 7fd964917640 1 --2- 172.21.15.84:0/608665785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd960159f00 0x7fd9600b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:18.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.227+0000 7fd957fff640 1 -- 172.21.15.84:0/608665785 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1672410106 0 0) 0x7fd94c003520 con 0x7fd960159b20 2023-10-16T11:25:18.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.227+0000 7fd957fff640 1 -- 172.21.15.84:0/608665785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd960159f00 msgr2=0x7fd9600b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:18.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.227+0000 7fd957fff640 1 --2- 172.21.15.84:0/608665785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd960159f00 0x7fd9600b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd957fff640 1 -- 172.21.15.84:0/608665785 >> v2:172.21.15.157:3300/0 conn(0x7fd96015d880 msgr2=0x7fd9600b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:18.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd957fff640 1 --2- 172.21.15.84:0/608665785 >> v2:172.21.15.157:3300/0 conn(0x7fd96015d880 0x7fd9600b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd957fff640 1 -- 172.21.15.84:0/608665785 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96006cba0 con 0x7fd960159b20 2023-10-16T11:25:18.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd957fff640 1 -- 172.21.15.84:0/608665785 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd95c001670 con 0x7fd960159b20 2023-10-16T11:25:18.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd957fff640 1 -- 172.21.15.84:0/608665785 >> v1:172.21.15.84:6789/0 conn(0x7fd960159b20 legacy=0x7fd96006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:18.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd957fff640 1 --2- 172.21.15.84:0/608665785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd94c003970 0x7fd94c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:18.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd957fff640 1 --2- 172.21.15.84:0/608665785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd960159f00 0x7fd94c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:18.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd957fff640 1 --2- 172.21.15.84:0/608665785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c004d00 0x7fd94c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:18.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd957fff640 1 -- 172.21.15.84:0/608665785 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96006c5e0 con 0x7fd94c003970 2023-10-16T11:25:18.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd957fff640 1 -- 172.21.15.84:0/608665785 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96006c8c0 con 0x7fd94c004d00 2023-10-16T11:25:18.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd957fff640 1 -- 172.21.15.84:0/608665785 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c00bab0 con 0x7fd960159f00 2023-10-16T11:25:18.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd965118640 1 --2- 172.21.15.84:0/608665785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd94c003970 0x7fd94c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:18.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd965919640 1 --2- 172.21.15.84:0/608665785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c004d00 0x7fd94c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:18.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd964917640 1 --2- 172.21.15.84:0/608665785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd960159f00 0x7fd94c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:18.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd965118640 1 -- 172.21.15.84:0/608665785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd960159f00 msgr2=0x7fd94c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:18.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd965118640 1 --2- 172.21.15.84:0/608665785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd960159f00 0x7fd94c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd965118640 1 -- 172.21.15.84:0/608665785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c004d00 msgr2=0x7fd94c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:18.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd965118640 1 --2- 172.21.15.84:0/608665785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c004d00 0x7fd94c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.228+0000 7fd965118640 1 -- 172.21.15.84:0/608665785 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94c003520 con 0x7fd94c003970 2023-10-16T11:25:18.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.230+0000 7fd965118640 1 --2- 172.21.15.84:0/608665785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd94c003970 0x7fd94c003d90 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00c160 tx=0x7fd94c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4d8999b1d60fc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:18.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.230+0000 7fd957fff640 1 -- 172.21.15.84:0/608665785 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c005ad0 con 0x7fd94c003970 2023-10-16T11:25:18.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.230+0000 7fd957fff640 1 -- 172.21.15.84:0/608665785 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95c005c30 con 0x7fd94c003970 2023-10-16T11:25:18.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.230+0000 7fd957fff640 1 -- 172.21.15.84:0/608665785 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c02c810 con 0x7fd94c003970 2023-10-16T11:25:18.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.230+0000 7fd96737c640 1 -- 172.21.15.84:0/608665785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd94c003970 msgr2=0x7fd94c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:18.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.230+0000 7fd96737c640 1 --2- 172.21.15.84:0/608665785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd94c003970 0x7fd94c003d90 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00c160 tx=0x7fd94c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.230+0000 7fd965919640 1 -- 172.21.15.84:0/608665785 reap_dead start 2023-10-16T11:25:18.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.230+0000 7fd96737c640 1 -- 172.21.15.84:0/608665785 shutdown_connections 2023-10-16T11:25:18.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.230+0000 7fd96737c640 1 -- 172.21.15.84:0/608665785 >> 172.21.15.84:0/608665785 conn(0x7fd9600cb410 msgr2=0x7fd9600cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:18.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.231+0000 7fd96737c640 1 -- 172.21.15.84:0/608665785 shutdown_connections 2023-10-16T11:25:18.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.231+0000 7fd96737c640 1 -- 172.21.15.84:0/608665785 wait complete. 2023-10-16T11:25:18.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.232+0000 7fd96737c640 1 Processor -- start 2023-10-16T11:25:18.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.232+0000 7fd96737c640 1 -- start start 2023-10-16T11:25:18.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.233+0000 7fd96737c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c003970 0x7fd960155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:18.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.233+0000 7fd96737c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9600b1cb0 0x7fd9600c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:18.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.233+0000 7fd96737c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9600b3770 0x7fd9600b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:18.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.233+0000 7fd96737c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9600b4130 con 0x7fd9600b1cb0 2023-10-16T11:25:18.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.233+0000 7fd96737c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9600b2260 con 0x7fd94c003970 2023-10-16T11:25:18.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.233+0000 7fd96737c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9600b2540 con 0x7fd9600b3770 2023-10-16T11:25:18.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.234+0000 7fd964917640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9600b1cb0 0x7fd9600c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:18.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.234+0000 7fd964917640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9600b1cb0 0x7fd9600c71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42522/0 (socket says 172.21.15.84:42522) 2023-10-16T11:25:18.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.234+0000 7fd964917640 1 -- 172.21.15.84:0/11904180 learned_addr learned my addr 172.21.15.84:0/11904180 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:18.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.234+0000 7fd964917640 1 -- 172.21.15.84:0/11904180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9600b3770 msgr2=0x7fd9600b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:18.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.234+0000 7fd964917640 1 --2- 172.21.15.84:0/11904180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9600b3770 0x7fd9600b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.234+0000 7fd964917640 1 -- 172.21.15.84:0/11904180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c003970 msgr2=0x7fd960155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:18.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.234+0000 7fd964917640 1 --2- 172.21.15.84:0/11904180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c003970 0x7fd960155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.234+0000 7fd964917640 1 -- 172.21.15.84:0/11904180 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9600b2820 con 0x7fd9600b1cb0 2023-10-16T11:25:18.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.234+0000 7fd964917640 1 --2- 172.21.15.84:0/11904180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9600b1cb0 0x7fd9600c71d0 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7fd95000e5d0 tx=0x7fd95000ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:18.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.235+0000 7fd955ffb640 1 -- 172.21.15.84:0/11904180 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950013610 con 0x7fd9600b1cb0 2023-10-16T11:25:18.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.235+0000 7fd955ffb640 1 -- 172.21.15.84:0/11904180 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd950013770 con 0x7fd9600b1cb0 2023-10-16T11:25:18.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.235+0000 7fd955ffb640 1 -- 172.21.15.84:0/11904180 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950013b00 con 0x7fd9600b1cb0 2023-10-16T11:25:18.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.235+0000 7fd96737c640 1 -- 172.21.15.84:0/11904180 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9600b2b00 con 0x7fd9600b1cb0 2023-10-16T11:25:18.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.235+0000 7fd96737c640 1 -- 172.21.15.84:0/11904180 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9600b3300 con 0x7fd9600b1cb0 2023-10-16T11:25:18.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.236+0000 7fd96737c640 1 -- 172.21.15.84:0/11904180 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd96006c270 con 0x7fd9600b1cb0 2023-10-16T11:25:18.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.237+0000 7fd955ffb640 1 -- 172.21.15.84:0/11904180 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd950013c60 con 0x7fd9600b1cb0 2023-10-16T11:25:18.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.238+0000 7fd955ffb640 1 --2- 172.21.15.84:0/11904180 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd94406b0b0 0x7fd94406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:18.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.238+0000 7fd955ffb640 1 -- 172.21.15.84:0/11904180 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6652+0+0 (secure 0 0 0) 0x7fd950089cd0 con 0x7fd9600b1cb0 2023-10-16T11:25:18.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.238+0000 7fd965118640 1 --2- 172.21.15.84:0/11904180 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd94406b0b0 0x7fd94406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:18.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.239+0000 7fd965118640 1 --2- 172.21.15.84:0/11904180 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd94406b0b0 0x7fd94406d570 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd95c001050 tx=0x7fd95c02e000 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:18.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.242+0000 7fd955ffb640 1 -- 172.21.15.84:0/11904180 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd950054550 con 0x7fd9600b1cb0 2023-10-16T11:25:18.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.435+0000 7fd96737c640 1 -- 172.21.15.84:0/11904180 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd96004e160 con 0x7fd9600b1cb0 2023-10-16T11:25:18.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.438+0000 7fd955ffb640 1 -- 172.21.15.84:0/11904180 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v226) v1 ==== 74+0+17925 (secure 0 0 0) 0x7fd950057b70 con 0x7fd9600b1cb0 2023-10-16T11:25:18.439 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:18.439 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":226,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:25:17.979565+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:24:38.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:24:39.387564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6834/551895595":"2023-10-17T11:25:14.673046+0000","172.21.15.84:0/1287173368":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/595669474":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/4107174082":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2906390983":"2023-10-17T11:25:09.210125+0000","172.21.15.84:6833/2880641466":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/3419402221":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2161898622":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/455217819":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/4247809558":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/1814319372":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2877946366":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6833/1425409017":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6836/551895595":"2023-10-17T11:25:14.673046+0000","172.21.15.84:6832/1425409017":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/1771162859":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6832/2880641466":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/3655599055":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/1076506559":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/2748224910":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/857172551":"2023-10-17T11:24:58.196616+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:25:18.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.441+0000 7fd96737c640 1 -- 172.21.15.84:0/11904180 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd94406b0b0 msgr2=0x7fd94406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:18.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.441+0000 7fd96737c640 1 --2- 172.21.15.84:0/11904180 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd94406b0b0 0x7fd94406d570 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd95c001050 tx=0x7fd95c02e000 comp rx=0 tx=0).stop 2023-10-16T11:25:18.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.441+0000 7fd96737c640 1 -- 172.21.15.84:0/11904180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9600b1cb0 msgr2=0x7fd9600c71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:18.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.441+0000 7fd96737c640 1 --2- 172.21.15.84:0/11904180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9600b1cb0 0x7fd9600c71d0 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7fd95000e5d0 tx=0x7fd95000ea90 comp rx=0 tx=0).stop 2023-10-16T11:25:18.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.441+0000 7fd96737c640 1 -- 172.21.15.84:0/11904180 shutdown_connections 2023-10-16T11:25:18.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.441+0000 7fd96737c640 1 --2- 172.21.15.84:0/11904180 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd94406b0b0 0x7fd94406d570 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.441+0000 7fd96737c640 1 --2- 172.21.15.84:0/11904180 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9600b3770 0x7fd9600b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.441+0000 7fd96737c640 1 --2- 172.21.15.84:0/11904180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9600b1cb0 0x7fd9600c71d0 unknown :-1 s=CLOSED pgs=2841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.441+0000 7fd96737c640 1 --2- 172.21.15.84:0/11904180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd94c003970 0x7fd960155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.441+0000 7fd96737c640 1 -- 172.21.15.84:0/11904180 >> 172.21.15.84:0/11904180 conn(0x7fd9600cb410 msgr2=0x7fd9600ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:18.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.441+0000 7fd96737c640 1 -- 172.21.15.84:0/11904180 shutdown_connections 2023-10-16T11:25:18.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.441+0000 7fd96737c640 1 -- 172.21.15.84:0/11904180 wait complete. 2023-10-16T11:25:18.460 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-16T11:24:38.531956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '220', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-16T11:24:39.387564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '221', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:25:18.461 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:25:18.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.680+0000 7f8085f48640 1 Processor -- start 2023-10-16T11:25:18.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.681+0000 7f8085f48640 1 -- start start 2023-10-16T11:25:18.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.682+0000 7f8085f48640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8080159e90 0x7f80800ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:18.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.682+0000 7f8085f48640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8080070360 con 0x7f8080159ab0 2023-10-16T11:25:18.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.682+0000 7f8085f48640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80800cd1f0 con 0x7f808006c140 2023-10-16T11:25:18.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.682+0000 7f8085f48640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808015d890 con 0x7f8080159e90 2023-10-16T11:25:18.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.682+0000 7f807f7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8080159ab0 0x7f808006fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49096/0 (socket says 172.21.15.84:49096) 2023-10-16T11:25:18.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.682+0000 7f807f7fe640 1 -- 172.21.15.84:0/4284405471 learned_addr learned my addr 172.21.15.84:0/4284405471 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:18.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.683+0000 7f807effd640 1 --2- 172.21.15.84:0/4284405471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8080159e90 0x7f80800ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:18.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.683+0000 7f807e7fc640 1 -- 172.21.15.84:0/4284405471 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230734262 0 0) 0x7f8080070360 con 0x7f8080159ab0 2023-10-16T11:25:18.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.683+0000 7f807e7fc640 1 -- 172.21.15.84:0/4284405471 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f806c003520 con 0x7f8080159ab0 2023-10-16T11:25:18.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.683+0000 7f807e7fc640 1 -- 172.21.15.84:0/4284405471 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2462964944 0 0) 0x7f806c003520 con 0x7f8080159ab0 2023-10-16T11:25:18.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.683+0000 7f807e7fc640 1 -- 172.21.15.84:0/4284405471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8080159e90 msgr2=0x7f80800ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:18.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.683+0000 7f807e7fc640 1 --2- 172.21.15.84:0/4284405471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8080159e90 0x7f80800ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.683+0000 7f807e7fc640 1 -- 172.21.15.84:0/4284405471 >> v1:172.21.15.157:6789/0 conn(0x7f808006c140 legacy=0x7f808015dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:18.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.683+0000 7f807e7fc640 1 -- 172.21.15.84:0/4284405471 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80800cd4d0 con 0x7f8080159ab0 2023-10-16T11:25:18.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.683+0000 7f807e7fc640 1 -- 172.21.15.84:0/4284405471 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f80700019f0 con 0x7f8080159ab0 2023-10-16T11:25:18.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.683+0000 7f807e7fc640 1 -- 172.21.15.84:0/4284405471 >> v1:172.21.15.84:6789/0 conn(0x7f8080159ab0 legacy=0x7f808006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:18.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.683+0000 7f807e7fc640 1 --2- 172.21.15.84:0/4284405471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806c0039a0 0x7f806c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:18.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.683+0000 7f807e7fc640 1 --2- 172.21.15.84:0/4284405471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8080159e90 0x7f806c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:18.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.683+0000 7f807e7fc640 1 --2- 172.21.15.84:0/4284405471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806c004d00 0x7f806c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:18.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.684+0000 7f807e7fc640 1 -- 172.21.15.84:0/4284405471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808015d890 con 0x7f806c004d00 2023-10-16T11:25:18.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.684+0000 7f807e7fc640 1 -- 172.21.15.84:0/4284405471 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c00bb40 con 0x7f806c0039a0 2023-10-16T11:25:18.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.684+0000 7f807e7fc640 1 -- 172.21.15.84:0/4284405471 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806c00bd00 con 0x7f8080159e90 2023-10-16T11:25:18.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.684+0000 7f807ffff640 1 --2- 172.21.15.84:0/4284405471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806c004d00 0x7f806c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:18.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.684+0000 7f807f7fe640 1 --2- 172.21.15.84:0/4284405471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806c0039a0 0x7f806c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:18.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.684+0000 7f807effd640 1 --2- 172.21.15.84:0/4284405471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8080159e90 0x7f806c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:18.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.684+0000 7f807ffff640 1 -- 172.21.15.84:0/4284405471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8080159e90 msgr2=0x7f806c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:18.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.684+0000 7f807ffff640 1 --2- 172.21.15.84:0/4284405471 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8080159e90 0x7f806c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.684+0000 7f807ffff640 1 -- 172.21.15.84:0/4284405471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806c0039a0 msgr2=0x7f806c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:18.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.684+0000 7f807ffff640 1 --2- 172.21.15.84:0/4284405471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806c0039a0 0x7f806c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.684+0000 7f807ffff640 1 -- 172.21.15.84:0/4284405471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806c003520 con 0x7f806c004d00 2023-10-16T11:25:18.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.686+0000 7f807ffff640 1 --2- 172.21.15.84:0/4284405471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806c004d00 0x7f806c005180 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7f8078001750 tx=0x7f806c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c4ef9d9f8c3579 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:18.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.686+0000 7f807e7fc640 1 -- 172.21.15.84:0/4284405471 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80780052d0 con 0x7f806c004d00 2023-10-16T11:25:18.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.686+0000 7f807e7fc640 1 -- 172.21.15.84:0/4284405471 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807802e050 con 0x7f806c004d00 2023-10-16T11:25:18.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.686+0000 7f807e7fc640 1 -- 172.21.15.84:0/4284405471 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80780035f0 con 0x7f806c004d00 2023-10-16T11:25:18.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.687+0000 7f8085f48640 1 -- 172.21.15.84:0/4284405471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806c004d00 msgr2=0x7f806c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:18.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.687+0000 7f8085f48640 1 --2- 172.21.15.84:0/4284405471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806c004d00 0x7f806c005180 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7f8078001750 tx=0x7f806c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.687+0000 7f807ffff640 1 -- 172.21.15.84:0/4284405471 reap_dead start 2023-10-16T11:25:18.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.687+0000 7f8085f48640 1 -- 172.21.15.84:0/4284405471 shutdown_connections 2023-10-16T11:25:18.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.687+0000 7f8085f48640 1 -- 172.21.15.84:0/4284405471 >> 172.21.15.84:0/4284405471 conn(0x7f80800ce9b0 msgr2=0x7f80800cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:18.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.687+0000 7f8085f48640 1 -- 172.21.15.84:0/4284405471 shutdown_connections 2023-10-16T11:25:18.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.687+0000 7f8085f48640 1 -- 172.21.15.84:0/4284405471 wait complete. 2023-10-16T11:25:18.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.691+0000 7f8085f48640 1 Processor -- start 2023-10-16T11:25:18.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.691+0000 7f8085f48640 1 -- start start 2023-10-16T11:25:18.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.692+0000 7f8085f48640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806c004d00 0x7f808015ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:18.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.692+0000 7f8085f48640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8080158920 0x7f8080157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:18.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.692+0000 7f8085f48640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f808015ad20 0x7f808015b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:18.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.692+0000 7f8085f48640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808015bba0 con 0x7f808015ad20 2023-10-16T11:25:18.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.692+0000 7f8085f48640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808015be80 con 0x7f806c004d00 2023-10-16T11:25:18.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.692+0000 7f8085f48640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808015c160 con 0x7f8080158920 2023-10-16T11:25:18.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.693+0000 7f807ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f808015ad20 0x7f808015b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:18.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.693+0000 7f807ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f808015ad20 0x7f808015b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42538/0 (socket says 172.21.15.84:42538) 2023-10-16T11:25:18.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.693+0000 7f807ffff640 1 -- 172.21.15.84:0/514178316 learned_addr learned my addr 172.21.15.84:0/514178316 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:18.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.694+0000 7f807ffff640 1 -- 172.21.15.84:0/514178316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8080158920 msgr2=0x7f8080157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:18.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.695+0000 7f807ffff640 1 --2- 172.21.15.84:0/514178316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8080158920 0x7f8080157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.695+0000 7f807ffff640 1 -- 172.21.15.84:0/514178316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806c004d00 msgr2=0x7f808015ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:18.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.695+0000 7f807ffff640 1 --2- 172.21.15.84:0/514178316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806c004d00 0x7f808015ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.695+0000 7f807ffff640 1 -- 172.21.15.84:0/514178316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808015c440 con 0x7f808015ad20 2023-10-16T11:25:18.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.696+0000 7f807ffff640 1 --2- 172.21.15.84:0/514178316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f808015ad20 0x7f808015b1b0 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f8078005040 tx=0x7f8078001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:18.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.697+0000 7f8077fff640 1 -- 172.21.15.84:0/514178316 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078007a30 con 0x7f808015ad20 2023-10-16T11:25:18.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.698+0000 7f8077fff640 1 -- 172.21.15.84:0/514178316 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80780067d0 con 0x7f808015ad20 2023-10-16T11:25:18.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.698+0000 7f8077fff640 1 -- 172.21.15.84:0/514178316 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078006b80 con 0x7f808015ad20 2023-10-16T11:25:18.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.698+0000 7f8085f48640 1 -- 172.21.15.84:0/514178316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80801f6aa0 con 0x7f808015ad20 2023-10-16T11:25:18.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.698+0000 7f8085f48640 1 -- 172.21.15.84:0/514178316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808015b6f0 con 0x7f808015ad20 2023-10-16T11:25:18.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.703+0000 7f8077fff640 1 -- 172.21.15.84:0/514178316 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f8078006ce0 con 0x7f808015ad20 2023-10-16T11:25:18.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.703+0000 7f8085f48640 1 -- 172.21.15.84:0/514178316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808004e160 con 0x7f808015ad20 2023-10-16T11:25:18.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.704+0000 7f8077fff640 1 --2- 172.21.15.84:0/514178316 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f806006b1b0 0x7f806006d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:18.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.705+0000 7f8077fff640 1 -- 172.21.15.84:0/514178316 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f80780ab740 con 0x7f808015ad20 2023-10-16T11:25:18.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.706+0000 7f807f7fe640 1 --2- 172.21.15.84:0/514178316 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f806006b1b0 0x7f806006d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:18.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.707+0000 7f807f7fe640 1 --2- 172.21.15.84:0/514178316 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f806006b1b0 0x7f806006d670 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8070001580 tx=0x7f80700020b0 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:18.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.710+0000 7f8077fff640 1 -- 172.21.15.84:0/514178316 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8078076230 con 0x7f808015ad20 2023-10-16T11:25:18.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.902+0000 7f8085f48640 1 -- 172.21.15.84:0/514178316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f80800ccb60 con 0x7f808015ad20 2023-10-16T11:25:18.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.903+0000 7f8077fff640 1 -- 172.21.15.84:0/514178316 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v226) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8078079850 con 0x7f808015ad20 2023-10-16T11:25:18.904 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:25:18.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.906+0000 7f8085f48640 1 -- 172.21.15.84:0/514178316 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f806006b1b0 msgr2=0x7f806006d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:18.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.906+0000 7f8085f48640 1 --2- 172.21.15.84:0/514178316 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f806006b1b0 0x7f806006d670 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8070001580 tx=0x7f80700020b0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.906+0000 7f8085f48640 1 -- 172.21.15.84:0/514178316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f808015ad20 msgr2=0x7f808015b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:18.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.906+0000 7f8085f48640 1 --2- 172.21.15.84:0/514178316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f808015ad20 0x7f808015b1b0 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f8078005040 tx=0x7f8078001120 comp rx=0 tx=0).stop 2023-10-16T11:25:18.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.906+0000 7f8085f48640 1 -- 172.21.15.84:0/514178316 shutdown_connections 2023-10-16T11:25:18.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.906+0000 7f8085f48640 1 --2- 172.21.15.84:0/514178316 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f806006b1b0 0x7f806006d670 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.906+0000 7f8085f48640 1 --2- 172.21.15.84:0/514178316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f808015ad20 0x7f808015b1b0 unknown :-1 s=CLOSED pgs=2844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.906+0000 7f8085f48640 1 --2- 172.21.15.84:0/514178316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8080158920 0x7f8080157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.906+0000 7f8085f48640 1 --2- 172.21.15.84:0/514178316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806c004d00 0x7f808015ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:18.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.906+0000 7f8085f48640 1 -- 172.21.15.84:0/514178316 >> 172.21.15.84:0/514178316 conn(0x7f80800ce9b0 msgr2=0x7f80800ae410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:18.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.906+0000 7f8085f48640 1 -- 172.21.15.84:0/514178316 shutdown_connections 2023-10-16T11:25:18.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:18.906+0000 7f8085f48640 1 -- 172.21.15.84:0/514178316 wait complete. 2023-10-16T11:25:18.926 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-16T11:25:19.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.137+0000 7fce8fbfb640 1 Processor -- start 2023-10-16T11:25:19.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.138+0000 7fce8fbfb640 1 -- start start 2023-10-16T11:25:19.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.139+0000 7fce8fbfb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce88160ad0 0x7fce880ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:19.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.139+0000 7fce8fbfb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce880ae260 0x7fce88160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:19.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.139+0000 7fce8fbfb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fce880ccd80 0x7fce880cd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:19.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.139+0000 7fce8fbfb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce88161900 con 0x7fce880ccd80 2023-10-16T11:25:19.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.139+0000 7fce8fbfb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce880acdd0 con 0x7fce880ae260 2023-10-16T11:25:19.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.139+0000 7fce8fbfb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce881613f0 con 0x7fce88160ad0 2023-10-16T11:25:19.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.139+0000 7fce8e198640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fce880ccd80 0x7fce880cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:19.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.139+0000 7fce8e198640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fce880ccd80 0x7fce880cd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42540/0 (socket says 172.21.15.84:42540) 2023-10-16T11:25:19.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.139+0000 7fce8e198640 1 -- 172.21.15.84:0/272769112 learned_addr learned my addr 172.21.15.84:0/272769112 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:19.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.140+0000 7fce8d997640 1 --2- 172.21.15.84:0/272769112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce88160ad0 0x7fce880ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:19.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.140+0000 7fce8e198640 1 -- 172.21.15.84:0/272769112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce88160ad0 msgr2=0x7fce880ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:19.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.140+0000 7fce8e198640 1 --2- 172.21.15.84:0/272769112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce88160ad0 0x7fce880ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.140+0000 7fce8e198640 1 -- 172.21.15.84:0/272769112 >> v2:172.21.15.157:3300/0 conn(0x7fce880ae260 msgr2=0x7fce88160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:19.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.140+0000 7fce8d196640 1 --2- 172.21.15.84:0/272769112 >> v2:172.21.15.157:3300/0 conn(0x7fce880ae260 0x7fce88160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:19.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.140+0000 7fce8e198640 1 --2- 172.21.15.84:0/272769112 >> v2:172.21.15.157:3300/0 conn(0x7fce880ae260 0x7fce88160eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.140+0000 7fce8e198640 1 -- 172.21.15.84:0/272769112 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce880ce8a0 con 0x7fce880ccd80 2023-10-16T11:25:19.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.141+0000 7fce8e198640 1 --2- 172.21.15.84:0/272769112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce880ccd80 0x7fce880cd210 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7fce8806f6a0 tx=0x7fce8002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92382210fde41d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:19.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.141+0000 7fce8c995640 1 -- 172.21.15.84:0/272769112 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80031860 con 0x7fce880ccd80 2023-10-16T11:25:19.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.141+0000 7fce8c995640 1 -- 172.21.15.84:0/272769112 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce800319c0 con 0x7fce880ccd80 2023-10-16T11:25:19.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.141+0000 7fce8c995640 1 -- 172.21.15.84:0/272769112 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80031d50 con 0x7fce880ccd80 2023-10-16T11:25:19.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.142+0000 7fce8fbfb640 1 -- 172.21.15.84:0/272769112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce880ccd80 msgr2=0x7fce880cd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:19.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.142+0000 7fce8fbfb640 1 --2- 172.21.15.84:0/272769112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce880ccd80 0x7fce880cd210 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7fce8806f6a0 tx=0x7fce8002b900 comp rx=0 tx=0).stop 2023-10-16T11:25:19.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.142+0000 7fce8fbfb640 1 -- 172.21.15.84:0/272769112 shutdown_connections 2023-10-16T11:25:19.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.142+0000 7fce8fbfb640 1 --2- 172.21.15.84:0/272769112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce880ccd80 0x7fce880cd210 unknown :-1 s=CLOSED pgs=2845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.142+0000 7fce8fbfb640 1 --2- 172.21.15.84:0/272769112 >> v2:172.21.15.157:3300/0 conn(0x7fce880ae260 0x7fce88160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.142+0000 7fce8fbfb640 1 --2- 172.21.15.84:0/272769112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce88160ad0 0x7fce880ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.142+0000 7fce8fbfb640 1 -- 172.21.15.84:0/272769112 >> 172.21.15.84:0/272769112 conn(0x7fce8806bc10 msgr2=0x7fce8806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:19.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.142+0000 7fce8fbfb640 1 -- 172.21.15.84:0/272769112 shutdown_connections 2023-10-16T11:25:19.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.142+0000 7fce8fbfb640 1 -- 172.21.15.84:0/272769112 wait complete. 2023-10-16T11:25:19.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.144+0000 7fce8fbfb640 1 Processor -- start 2023-10-16T11:25:19.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.144+0000 7fce8fbfb640 1 -- start start 2023-10-16T11:25:19.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.145+0000 7fce8fbfb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce880ae260 0x7fce880c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:19.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.145+0000 7fce8fbfb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce880ccd80 0x7fce8815be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:19.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.145+0000 7fce8fbfb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce88160ad0 0x7fce880c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:19.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.145+0000 7fce8fbfb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce880c4eb0 con 0x7fce88160ad0 2023-10-16T11:25:19.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.145+0000 7fce8fbfb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8815d2e0 con 0x7fce880ae260 2023-10-16T11:25:19.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.145+0000 7fce8fbfb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce880c3870 con 0x7fce880ccd80 2023-10-16T11:25:19.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.145+0000 7fce8e198640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce88160ad0 0x7fce880c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:19.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.145+0000 7fce8e198640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce88160ad0 0x7fce880c5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42548/0 (socket says 172.21.15.84:42548) 2023-10-16T11:25:19.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.145+0000 7fce8e198640 1 -- 172.21.15.84:0/2103645977 learned_addr learned my addr 172.21.15.84:0/2103645977 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:19.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.145+0000 7fce8d196640 1 --2- 172.21.15.84:0/2103645977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce880ccd80 0x7fce8815be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:19.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.146+0000 7fce8e198640 1 -- 172.21.15.84:0/2103645977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce880ccd80 msgr2=0x7fce8815be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:19.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.146+0000 7fce8e198640 1 --2- 172.21.15.84:0/2103645977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce880ccd80 0x7fce8815be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.146+0000 7fce8d997640 1 --2- 172.21.15.84:0/2103645977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce880ae260 0x7fce880c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:19.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.146+0000 7fce8e198640 1 -- 172.21.15.84:0/2103645977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce880ae260 msgr2=0x7fce880c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:19.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.146+0000 7fce8e198640 1 --2- 172.21.15.84:0/2103645977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce880ae260 0x7fce880c4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.146+0000 7fce8e198640 1 -- 172.21.15.84:0/2103645977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce800094d0 con 0x7fce88160ad0 2023-10-16T11:25:19.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.146+0000 7fce8e198640 1 --2- 172.21.15.84:0/2103645977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce88160ad0 0x7fce880c5060 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7fce800090b0 tx=0x7fce800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:19.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.146+0000 7fce867fc640 1 -- 172.21.15.84:0/2103645977 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce800443f0 con 0x7fce88160ad0 2023-10-16T11:25:19.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.146+0000 7fce867fc640 1 -- 172.21.15.84:0/2103645977 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce80044570 con 0x7fce88160ad0 2023-10-16T11:25:19.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.146+0000 7fce867fc640 1 -- 172.21.15.84:0/2103645977 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80044920 con 0x7fce88160ad0 2023-10-16T11:25:19.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.146+0000 7fce8fbfb640 1 -- 172.21.15.84:0/2103645977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce880c3b50 con 0x7fce88160ad0 2023-10-16T11:25:19.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.146+0000 7fce8fbfb640 1 -- 172.21.15.84:0/2103645977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce880c4bc0 con 0x7fce88160ad0 2023-10-16T11:25:19.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.147+0000 7fce8fbfb640 1 -- 172.21.15.84:0/2103645977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8815d450 con 0x7fce88160ad0 2023-10-16T11:25:19.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.148+0000 7fce867fc640 1 -- 172.21.15.84:0/2103645977 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fce8004d450 con 0x7fce88160ad0 2023-10-16T11:25:19.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.149+0000 7fce867fc640 1 --2- 172.21.15.84:0/2103645977 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fce7406b0b0 0x7fce7406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:19.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.149+0000 7fce867fc640 1 -- 172.21.15.84:0/2103645977 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6652+0+0 (secure 0 0 0) 0x7fce800ac6a0 con 0x7fce88160ad0 2023-10-16T11:25:19.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.149+0000 7fce8d997640 1 --2- 172.21.15.84:0/2103645977 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fce7406b0b0 0x7fce7406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:19.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.150+0000 7fce8d997640 1 --2- 172.21.15.84:0/2103645977 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fce7406b0b0 0x7fce7406d570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fce78008ef0 tx=0x7fce78008440 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:19.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.154+0000 7fce867fc640 1 -- 172.21.15.84:0/2103645977 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fce8815d450 con 0x7fce88160ad0 2023-10-16T11:25:19.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.344+0000 7fce8fbfb640 1 -- 172.21.15.84:0/2103645977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fce8815d450 con 0x7fce88160ad0 2023-10-16T11:25:19.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.345+0000 7fce867fc640 1 -- 172.21.15.84:0/2103645977 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v226) v1 ==== 107+0+11 (secure 0 0 0) 0x7fce8815d450 con 0x7fce88160ad0 2023-10-16T11:25:19.348 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:25:19.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.347+0000 7fce8fbfb640 1 -- 172.21.15.84:0/2103645977 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fce7406b0b0 msgr2=0x7fce7406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:19.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.347+0000 7fce8fbfb640 1 --2- 172.21.15.84:0/2103645977 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fce7406b0b0 0x7fce7406d570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fce78008ef0 tx=0x7fce78008440 comp rx=0 tx=0).stop 2023-10-16T11:25:19.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.347+0000 7fce8fbfb640 1 -- 172.21.15.84:0/2103645977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce88160ad0 msgr2=0x7fce880c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:19.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.347+0000 7fce8fbfb640 1 --2- 172.21.15.84:0/2103645977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce88160ad0 0x7fce880c5060 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7fce800090b0 tx=0x7fce800046c0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.347+0000 7fce8fbfb640 1 -- 172.21.15.84:0/2103645977 shutdown_connections 2023-10-16T11:25:19.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.348+0000 7fce8fbfb640 1 --2- 172.21.15.84:0/2103645977 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fce7406b0b0 0x7fce7406d570 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.348+0000 7fce8fbfb640 1 --2- 172.21.15.84:0/2103645977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce88160ad0 0x7fce880c5060 unknown :-1 s=CLOSED pgs=2846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.348+0000 7fce8fbfb640 1 --2- 172.21.15.84:0/2103645977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce880ccd80 0x7fce8815be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.348+0000 7fce8fbfb640 1 --2- 172.21.15.84:0/2103645977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce880ae260 0x7fce880c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.348+0000 7fce8fbfb640 1 -- 172.21.15.84:0/2103645977 >> 172.21.15.84:0/2103645977 conn(0x7fce8806bc10 msgr2=0x7fce880b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:19.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.348+0000 7fce8fbfb640 1 -- 172.21.15.84:0/2103645977 shutdown_connections 2023-10-16T11:25:19.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.348+0000 7fce8fbfb640 1 -- 172.21.15.84:0/2103645977 wait complete. 2023-10-16T11:25:19.369 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-16T11:25:19.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.585+0000 7fae099d2640 1 Processor -- start 2023-10-16T11:25:19.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.585+0000 7fae099d2640 1 -- start start 2023-10-16T11:25:19.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.587+0000 7fae099d2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fae04159950 0x7fae0415a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:19.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.587+0000 7fae099d2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0415aef0 con 0x7fae0406c810 2023-10-16T11:25:19.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.587+0000 7fae099d2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0415d910 con 0x7fae04070450 2023-10-16T11:25:19.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.587+0000 7fae099d2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0415e490 con 0x7fae04159950 2023-10-16T11:25:19.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.587+0000 7fae091d1640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fae0406c810 0x7fae040c9090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49112/0 (socket says 172.21.15.84:49112) 2023-10-16T11:25:19.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.587+0000 7fae091d1640 1 -- 172.21.15.84:0/647109403 learned_addr learned my addr 172.21.15.84:0/647109403 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:19.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.587+0000 7fae089d0640 1 --2- 172.21.15.84:0/647109403 >> v2:172.21.15.157:3300/0 conn(0x7fae04159950 0x7fae0415a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:19.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.588+0000 7fae037fe640 1 -- 172.21.15.84:0/647109403 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019436859 0 0) 0x7fadfc001490 con 0x7fae0406c810 2023-10-16T11:25:19.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.588+0000 7fae037fe640 1 -- 172.21.15.84:0/647109403 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadf4003520 con 0x7fae0406c810 2023-10-16T11:25:19.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.588+0000 7fae037fe640 1 -- 172.21.15.84:0/647109403 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2393694922 0 0) 0x7fadf4003520 con 0x7fae0406c810 2023-10-16T11:25:19.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.588+0000 7fae037fe640 1 -- 172.21.15.84:0/647109403 >> v2:172.21.15.157:3300/0 conn(0x7fae04159950 msgr2=0x7fae0415a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:19.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.588+0000 7fae037fe640 1 --2- 172.21.15.84:0/647109403 >> v2:172.21.15.157:3300/0 conn(0x7fae04159950 0x7fae0415a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.588+0000 7fae037fe640 1 -- 172.21.15.84:0/647109403 >> v1:172.21.15.157:6789/0 conn(0x7fae04070450 legacy=0x7fae0406c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:19.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.588+0000 7fae037fe640 1 -- 172.21.15.84:0/647109403 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0415dbf0 con 0x7fae0406c810 2023-10-16T11:25:19.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.588+0000 7fae037fe640 1 -- 172.21.15.84:0/647109403 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fadfc001b80 con 0x7fae0406c810 2023-10-16T11:25:19.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.588+0000 7fae037fe640 1 -- 172.21.15.84:0/647109403 >> v1:172.21.15.84:6789/0 conn(0x7fae0406c810 legacy=0x7fae040c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:19.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.588+0000 7fae037fe640 1 --2- 172.21.15.84:0/647109403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf40039c0 0x7fadf4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:19.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.588+0000 7fae037fe640 1 --2- 172.21.15.84:0/647109403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadf40047c0 0x7fadf4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:19.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.588+0000 7fae037fe640 1 --2- 172.21.15.84:0/647109403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadf40051d0 0x7fadf400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:19.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.588+0000 7fae037fe640 1 -- 172.21.15.84:0/647109403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0415e490 con 0x7fadf40039c0 2023-10-16T11:25:19.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.588+0000 7fae037fe640 1 -- 172.21.15.84:0/647109403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf400c120 con 0x7fadf40047c0 2023-10-16T11:25:19.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.588+0000 7fae037fe640 1 -- 172.21.15.84:0/647109403 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf400c2e0 con 0x7fadf40051d0 2023-10-16T11:25:19.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.589+0000 7fae089d0640 1 --2- 172.21.15.84:0/647109403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf40039c0 0x7fadf4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:19.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.589+0000 7fae089d0640 1 -- 172.21.15.84:0/647109403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadf40051d0 msgr2=0x7fadf400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:19.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.589+0000 7fae089d0640 1 --2- 172.21.15.84:0/647109403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadf40051d0 0x7fadf400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.589+0000 7fae089d0640 1 -- 172.21.15.84:0/647109403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadf40047c0 msgr2=0x7fadf4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:19.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.589+0000 7fae03fff640 1 --2- 172.21.15.84:0/647109403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadf40047c0 0x7fadf4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:19.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.589+0000 7fae089d0640 1 --2- 172.21.15.84:0/647109403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadf40047c0 0x7fadf4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.589+0000 7fae091d1640 1 -- 172.21.15.84:0/647109403 reap_dead start 2023-10-16T11:25:19.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.589+0000 7fae089d0640 1 -- 172.21.15.84:0/647109403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf4003520 con 0x7fadf40039c0 2023-10-16T11:25:19.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.590+0000 7fae089d0640 1 --2- 172.21.15.84:0/647109403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf40039c0 0x7fadf4003de0 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009a90 tx=0x7fadf40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2570f9a522684266 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:19.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.590+0000 7fae037fe640 1 -- 172.21.15.84:0/647109403 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf802c6b0 con 0x7fadf40039c0 2023-10-16T11:25:19.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.590+0000 7fae037fe640 1 -- 172.21.15.84:0/647109403 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf802c810 con 0x7fadf40039c0 2023-10-16T11:25:19.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.590+0000 7fae037fe640 1 -- 172.21.15.84:0/647109403 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf802cba0 con 0x7fadf40039c0 2023-10-16T11:25:19.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.591+0000 7fae099d2640 1 -- 172.21.15.84:0/647109403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf40039c0 msgr2=0x7fadf4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:19.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.591+0000 7fae099d2640 1 --2- 172.21.15.84:0/647109403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf40039c0 0x7fadf4003de0 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009a90 tx=0x7fadf40057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.591+0000 7fae099d2640 1 -- 172.21.15.84:0/647109403 shutdown_connections 2023-10-16T11:25:19.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.591+0000 7fae099d2640 1 --2- 172.21.15.84:0/647109403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf40039c0 0x7fadf4003de0 unknown :-1 s=CLOSED pgs=2848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.591+0000 7fae099d2640 1 -- 172.21.15.84:0/647109403 >> 172.21.15.84:0/647109403 conn(0x7fae040ce6a0 msgr2=0x7fae040ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:19.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.591+0000 7fae099d2640 1 -- 172.21.15.84:0/647109403 shutdown_connections 2023-10-16T11:25:19.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.591+0000 7fae099d2640 1 -- 172.21.15.84:0/647109403 wait complete. 2023-10-16T11:25:19.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.593+0000 7fae099d2640 1 Processor -- start 2023-10-16T11:25:19.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.593+0000 7fae099d2640 1 -- start start 2023-10-16T11:25:19.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.594+0000 7fae099d2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf40039c0 0x7fae041579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:19.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.594+0000 7fae099d2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae04154d80 0x7fae041551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:19.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.594+0000 7fae099d2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae040b2be0 0x7fae040b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:19.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.594+0000 7fae099d2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae040b12c0 con 0x7fadf40039c0 2023-10-16T11:25:19.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.594+0000 7fae099d2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae040b20e0 con 0x7fae04154d80 2023-10-16T11:25:19.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.594+0000 7fae099d2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae040b14d0 con 0x7fae040b2be0 2023-10-16T11:25:19.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.594+0000 7fae089d0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf40039c0 0x7fae041579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:19.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.594+0000 7fae089d0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf40039c0 0x7fae041579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42572/0 (socket says 172.21.15.84:42572) 2023-10-16T11:25:19.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.594+0000 7fae089d0640 1 -- 172.21.15.84:0/4183812542 learned_addr learned my addr 172.21.15.84:0/4183812542 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:19.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.595+0000 7fae03fff640 1 --2- 172.21.15.84:0/4183812542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae04154d80 0x7fae041551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:19.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.595+0000 7fae091d1640 1 --2- 172.21.15.84:0/4183812542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae040b2be0 0x7fae040b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:19.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.595+0000 7fae089d0640 1 -- 172.21.15.84:0/4183812542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae040b2be0 msgr2=0x7fae040b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:19.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.595+0000 7fae089d0640 1 --2- 172.21.15.84:0/4183812542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae040b2be0 0x7fae040b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.595+0000 7fae089d0640 1 -- 172.21.15.84:0/4183812542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae04154d80 msgr2=0x7fae041551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:19.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.595+0000 7fae089d0640 1 --2- 172.21.15.84:0/4183812542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae04154d80 0x7fae041551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.595+0000 7fae089d0640 1 -- 172.21.15.84:0/4183812542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae040b17b0 con 0x7fadf40039c0 2023-10-16T11:25:19.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.595+0000 7fae03fff640 1 --2- 172.21.15.84:0/4183812542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae04154d80 0x7fae041551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:19.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.595+0000 7fae089d0640 1 --2- 172.21.15.84:0/4183812542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf40039c0 0x7fae041579a0 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7fadf802c250 tx=0x7fadf8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:19.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.595+0000 7fae017fa640 1 -- 172.21.15.84:0/4183812542 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8003810 con 0x7fadf40039c0 2023-10-16T11:25:19.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.595+0000 7fae017fa640 1 -- 172.21.15.84:0/4183812542 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf8003970 con 0x7fadf40039c0 2023-10-16T11:25:19.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.595+0000 7fae017fa640 1 -- 172.21.15.84:0/4183812542 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8003ce0 con 0x7fadf40039c0 2023-10-16T11:25:19.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.595+0000 7fae099d2640 1 -- 172.21.15.84:0/4183812542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae040b1a90 con 0x7fadf40039c0 2023-10-16T11:25:19.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.596+0000 7fae099d2640 1 -- 172.21.15.84:0/4183812542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae040b35a0 con 0x7fadf40039c0 2023-10-16T11:25:19.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.597+0000 7fae099d2640 1 -- 172.21.15.84:0/4183812542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0406da40 con 0x7fadf40039c0 2023-10-16T11:25:19.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.598+0000 7fae017fa640 1 -- 172.21.15.84:0/4183812542 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fadf8040450 con 0x7fadf40039c0 2023-10-16T11:25:19.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.598+0000 7fae017fa640 1 --2- 172.21.15.84:0/4183812542 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fade806b0e0 0x7fade806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:19.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.599+0000 7fae017fa640 1 -- 172.21.15.84:0/4183812542 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6652+0+0 (secure 0 0 0) 0x7fadf80abcb0 con 0x7fadf40039c0 2023-10-16T11:25:19.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.599+0000 7fae03fff640 1 --2- 172.21.15.84:0/4183812542 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fade806b0e0 0x7fade806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:19.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.599+0000 7fae03fff640 1 --2- 172.21.15.84:0/4183812542 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fade806b0e0 0x7fade806d5a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fae040c2450 tx=0x7fadf0009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:19.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.603+0000 7fae017fa640 1 -- 172.21.15.84:0/4183812542 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fadf80765c0 con 0x7fadf40039c0 2023-10-16T11:25:19.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.797+0000 7fae099d2640 1 -- 172.21.15.84:0/4183812542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7fae0404e9b0 con 0x7fadf40039c0 2023-10-16T11:25:19.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.797+0000 7fae017fa640 1 -- 172.21.15.84:0/4183812542 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v226) v1 ==== 103+0+11 (secure 0 0 0) 0x7fadf80044e0 con 0x7fadf40039c0 2023-10-16T11:25:19.800 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:25:19.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.801+0000 7fae099d2640 1 -- 172.21.15.84:0/4183812542 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fade806b0e0 msgr2=0x7fade806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:19.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.801+0000 7fae099d2640 1 --2- 172.21.15.84:0/4183812542 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fade806b0e0 0x7fade806d5a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fae040c2450 tx=0x7fadf0009040 comp rx=0 tx=0).stop 2023-10-16T11:25:19.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.801+0000 7fae099d2640 1 -- 172.21.15.84:0/4183812542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf40039c0 msgr2=0x7fae041579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:19.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.801+0000 7fae099d2640 1 --2- 172.21.15.84:0/4183812542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf40039c0 0x7fae041579a0 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7fadf802c250 tx=0x7fadf8004730 comp rx=0 tx=0).stop 2023-10-16T11:25:19.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.801+0000 7fae099d2640 1 -- 172.21.15.84:0/4183812542 shutdown_connections 2023-10-16T11:25:19.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.801+0000 7fae099d2640 1 --2- 172.21.15.84:0/4183812542 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fade806b0e0 0x7fade806d5a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.801+0000 7fae099d2640 1 --2- 172.21.15.84:0/4183812542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fae040b2be0 0x7fae040b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.801+0000 7fae099d2640 1 --2- 172.21.15.84:0/4183812542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae04154d80 0x7fae041551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.801+0000 7fae099d2640 1 --2- 172.21.15.84:0/4183812542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadf40039c0 0x7fae041579a0 unknown :-1 s=CLOSED pgs=2849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:19.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.801+0000 7fae099d2640 1 -- 172.21.15.84:0/4183812542 >> 172.21.15.84:0/4183812542 conn(0x7fae040ce6a0 msgr2=0x7fae04156bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:19.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.801+0000 7fae099d2640 1 -- 172.21.15.84:0/4183812542 shutdown_connections 2023-10-16T11:25:19.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:19.801+0000 7fae099d2640 1 -- 172.21.15.84:0/4183812542 wait complete. 2023-10-16T11:25:19.821 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:25:20.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.026+0000 7f3b9fede640 1 Processor -- start 2023-10-16T11:25:20.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.026+0000 7f3b9fede640 1 -- start start 2023-10-16T11:25:20.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.027+0000 7f3b9fede640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b9815aaa0 0x7f3b9815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:20.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.028+0000 7f3b9fede640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b9815ae80 0x7f3b980cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:20.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.028+0000 7f3b9fede640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b9815ed60 con 0x7f3b980ccaa0 2023-10-16T11:25:20.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.028+0000 7f3b9fede640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b9806b810 con 0x7f3b9815aaa0 2023-10-16T11:25:20.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.028+0000 7f3b9fede640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b980707e0 con 0x7f3b9815ae80 2023-10-16T11:25:20.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.028+0000 7f3b9dc7a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b9815aaa0 0x7f3b9815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:20.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.028+0000 7f3b9d479640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b9815ae80 0x7f3b980cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:20.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.028+0000 7f3b9dc7a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b9815aaa0 0x7f3b9815e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:56888/0 (socket says 172.21.15.84:56888) 2023-10-16T11:25:20.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.028+0000 7f3b9dc7a640 1 -- 172.21.15.84:0/2727455768 learned_addr learned my addr 172.21.15.84:0/2727455768 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:20.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.029+0000 7f3b9dc7a640 1 -- 172.21.15.84:0/2727455768 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b9815ae80 msgr2=0x7f3b980cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:20.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.029+0000 7f3b9dc7a640 1 --2- 172.21.15.84:0/2727455768 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b9815ae80 0x7f3b980cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.029+0000 7f3b9dc7a640 1 -- 172.21.15.84:0/2727455768 >> v1:172.21.15.157:6789/0 conn(0x7f3b980ccaa0 legacy=0x7f3b98070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:20.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.029+0000 7f3b9dc7a640 1 -- 172.21.15.84:0/2727455768 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9815ef90 con 0x7f3b9815aaa0 2023-10-16T11:25:20.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.030+0000 7f3b9dc7a640 1 --2- 172.21.15.84:0/2727455768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9815aaa0 0x7f3b9815e820 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3b94009870 tx=0x7f3b9402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49e237799dc88801 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:20.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.031+0000 7f3b9cc78640 1 -- 172.21.15.84:0/2727455768 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94030840 con 0x7f3b9815aaa0 2023-10-16T11:25:20.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.031+0000 7f3b9cc78640 1 -- 172.21.15.84:0/2727455768 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b940309a0 con 0x7f3b9815aaa0 2023-10-16T11:25:20.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.031+0000 7f3b9cc78640 1 -- 172.21.15.84:0/2727455768 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94030d30 con 0x7f3b9815aaa0 2023-10-16T11:25:20.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.032+0000 7f3b9fede640 1 -- 172.21.15.84:0/2727455768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9815aaa0 msgr2=0x7f3b9815e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:20.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.032+0000 7f3b9fede640 1 --2- 172.21.15.84:0/2727455768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9815aaa0 0x7f3b9815e820 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3b94009870 tx=0x7f3b9402b980 comp rx=0 tx=0).stop 2023-10-16T11:25:20.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.032+0000 7f3b9fede640 1 -- 172.21.15.84:0/2727455768 shutdown_connections 2023-10-16T11:25:20.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.032+0000 7f3b9fede640 1 --2- 172.21.15.84:0/2727455768 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b9815ae80 0x7f3b980cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.032+0000 7f3b9fede640 1 --2- 172.21.15.84:0/2727455768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9815aaa0 0x7f3b9815e820 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.032+0000 7f3b9fede640 1 -- 172.21.15.84:0/2727455768 >> 172.21.15.84:0/2727455768 conn(0x7f3b980cb410 msgr2=0x7f3b980cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:20.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.032+0000 7f3b9fede640 1 -- 172.21.15.84:0/2727455768 shutdown_connections 2023-10-16T11:25:20.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.032+0000 7f3b9fede640 1 -- 172.21.15.84:0/2727455768 wait complete. 2023-10-16T11:25:20.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.034+0000 7f3b9fede640 1 Processor -- start 2023-10-16T11:25:20.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.034+0000 7f3b9fede640 1 -- start start 2023-10-16T11:25:20.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.035+0000 7f3b9fede640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b980ccaa0 0x7f3b981567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:20.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.036+0000 7f3b9fede640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9815aaa0 0x7f3b98156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:20.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.036+0000 7f3b9fede640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b9815ae80 0x7f3b981484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:20.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.036+0000 7f3b9fede640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b980c4ea0 con 0x7f3b980ccaa0 2023-10-16T11:25:20.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.036+0000 7f3b9fede640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9815c0d0 con 0x7f3b9815aaa0 2023-10-16T11:25:20.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.036+0000 7f3b9fede640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b980c48e0 con 0x7f3b9815ae80 2023-10-16T11:25:20.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.036+0000 7f3b9dc7a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b980ccaa0 0x7f3b981567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:20.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.036+0000 7f3b9dc7a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b980ccaa0 0x7f3b981567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42574/0 (socket says 172.21.15.84:42574) 2023-10-16T11:25:20.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.036+0000 7f3b9dc7a640 1 -- 172.21.15.84:0/1654823114 learned_addr learned my addr 172.21.15.84:0/1654823114 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:20.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.036+0000 7f3b9d479640 1 --2- 172.21.15.84:0/1654823114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9815aaa0 0x7f3b98156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:20.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.036+0000 7f3b9e47b640 1 --2- 172.21.15.84:0/1654823114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b9815ae80 0x7f3b981484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:20.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.036+0000 7f3b9dc7a640 1 -- 172.21.15.84:0/1654823114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b9815ae80 msgr2=0x7f3b981484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:20.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.036+0000 7f3b9dc7a640 1 --2- 172.21.15.84:0/1654823114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b9815ae80 0x7f3b981484e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.036+0000 7f3b9dc7a640 1 -- 172.21.15.84:0/1654823114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9815aaa0 msgr2=0x7f3b98156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:20.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.036+0000 7f3b9dc7a640 1 --2- 172.21.15.84:0/1654823114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9815aaa0 0x7f3b98156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.036+0000 7f3b9dc7a640 1 -- 172.21.15.84:0/1654823114 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b940094d0 con 0x7f3b980ccaa0 2023-10-16T11:25:20.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.036+0000 7f3b9d479640 1 --2- 172.21.15.84:0/1654823114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9815aaa0 0x7f3b98156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:20.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.037+0000 7f3b9dc7a640 1 --2- 172.21.15.84:0/1654823114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b980ccaa0 0x7f3b981567e0 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f3b94000c00 tx=0x7f3b94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:20.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.037+0000 7f3b8e7fc640 1 -- 172.21.15.84:0/1654823114 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94039c90 con 0x7f3b980ccaa0 2023-10-16T11:25:20.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.037+0000 7f3b8e7fc640 1 -- 172.21.15.84:0/1654823114 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b94039df0 con 0x7f3b980ccaa0 2023-10-16T11:25:20.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.037+0000 7f3b8e7fc640 1 -- 172.21.15.84:0/1654823114 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94043640 con 0x7f3b980ccaa0 2023-10-16T11:25:20.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.037+0000 7f3b9fede640 1 -- 172.21.15.84:0/1654823114 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b980c4bc0 con 0x7f3b980ccaa0 2023-10-16T11:25:20.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.037+0000 7f3b9fede640 1 -- 172.21.15.84:0/1654823114 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b98156100 con 0x7f3b980ccaa0 2023-10-16T11:25:20.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.038+0000 7f3b9fede640 1 -- 172.21.15.84:0/1654823114 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b98156350 con 0x7f3b980ccaa0 2023-10-16T11:25:20.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.039+0000 7f3b8e7fc640 1 -- 172.21.15.84:0/1654823114 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f3b940437a0 con 0x7f3b980ccaa0 2023-10-16T11:25:20.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.040+0000 7f3b8e7fc640 1 --2- 172.21.15.84:0/1654823114 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f3b7c06b0e0 0x7f3b7c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:20.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.040+0000 7f3b8e7fc640 1 -- 172.21.15.84:0/1654823114 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f3b9403e070 con 0x7f3b980ccaa0 2023-10-16T11:25:20.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.040+0000 7f3b9d479640 1 --2- 172.21.15.84:0/1654823114 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f3b7c06b0e0 0x7f3b7c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:20.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.041+0000 7f3b9d479640 1 --2- 172.21.15.84:0/1654823114 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f3b7c06b0e0 0x7f3b7c06d5a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3b98156790 tx=0x7f3b88008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:20.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.047+0000 7f3b8e7fc640 1 -- 172.21.15.84:0/1654823114 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3b940777b0 con 0x7f3b980ccaa0 2023-10-16T11:25:20.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.278+0000 7f3b9fede640 1 -- 172.21.15.84:0/1654823114 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b9815bbc0 con 0x7f3b980ccaa0 2023-10-16T11:25:20.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.279+0000 7f3b8e7fc640 1 -- 172.21.15.84:0/1654823114 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4933 (secure 0 0 0) 0x7f3b9815bbc0 con 0x7f3b980ccaa0 2023-10-16T11:25:20.281 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:20.282 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":122,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3106898227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3106898227},{"type":"v1","addr":"172.21.15.84:6837","nonce":3106898227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":7471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2296953749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2296953749},{"type":"v1","addr":"172.21.15.157:6834","nonce":2296953749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":12015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/3350751452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3350751452},{"type":"v1","addr":"172.21.15.84:6836","nonce":3350751452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:24:40.366708+0000","modified":"2023-10-16T11:24:43.431532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7474":{"gid":7474,"name":"d","rank":0,"incarnation":116,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6835/3992377882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3992377882},{"type":"v1","addr":"172.21.15.157:6835","nonce":3992377882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-16T11:25:20.282 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 122 2023-10-16T11:25:20.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.282+0000 7f3b9fede640 1 -- 172.21.15.84:0/1654823114 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f3b7c06b0e0 msgr2=0x7f3b7c06d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:20.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.282+0000 7f3b9fede640 1 --2- 172.21.15.84:0/1654823114 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f3b7c06b0e0 0x7f3b7c06d5a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3b98156790 tx=0x7f3b88008040 comp rx=0 tx=0).stop 2023-10-16T11:25:20.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.282+0000 7f3b9fede640 1 -- 172.21.15.84:0/1654823114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b980ccaa0 msgr2=0x7f3b981567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:20.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.282+0000 7f3b9fede640 1 --2- 172.21.15.84:0/1654823114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b980ccaa0 0x7f3b981567e0 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f3b94000c00 tx=0x7f3b94004730 comp rx=0 tx=0).stop 2023-10-16T11:25:20.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.282+0000 7f3b9fede640 1 -- 172.21.15.84:0/1654823114 shutdown_connections 2023-10-16T11:25:20.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.282+0000 7f3b9fede640 1 --2- 172.21.15.84:0/1654823114 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f3b7c06b0e0 0x7f3b7c06d5a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.282+0000 7f3b9fede640 1 --2- 172.21.15.84:0/1654823114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b9815ae80 0x7f3b981484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.282+0000 7f3b9fede640 1 --2- 172.21.15.84:0/1654823114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9815aaa0 0x7f3b98156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.282+0000 7f3b9fede640 1 --2- 172.21.15.84:0/1654823114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b980ccaa0 0x7f3b981567e0 unknown :-1 s=CLOSED pgs=2850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.282+0000 7f3b9fede640 1 -- 172.21.15.84:0/1654823114 >> 172.21.15.84:0/1654823114 conn(0x7f3b980cb410 msgr2=0x7f3b980b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:20.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.283+0000 7f3b9fede640 1 -- 172.21.15.84:0/1654823114 shutdown_connections 2023-10-16T11:25:20.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.283+0000 7f3b9fede640 1 -- 172.21.15.84:0/1654823114 wait complete. 2023-10-16T11:25:20.304 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:25:20.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.501+0000 7f44563ff640 1 Processor -- start 2023-10-16T11:25:20.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.502+0000 7f44563ff640 1 -- start start 2023-10-16T11:25:20.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.503+0000 7f44563ff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f445015cc80 0x7f44500cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:20.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.503+0000 7f44563ff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f44500cdd60 0x7f4450071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:20.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.503+0000 7f44563ff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44500ce1c0 con 0x7f4450071fa0 2023-10-16T11:25:20.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.503+0000 7f44563ff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44500d0500 con 0x7f44500cdd60 2023-10-16T11:25:20.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.503+0000 7f44563ff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44500d12f0 con 0x7f445015cc80 2023-10-16T11:25:20.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.503+0000 7f4454bfc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f44500cdd60 0x7f4450071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:20.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.503+0000 7f4454bfc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f44500cdd60 0x7f4450071a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42586/0 (socket says 172.21.15.84:42586) 2023-10-16T11:25:20.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.503+0000 7f4454bfc640 1 -- 172.21.15.84:0/2041279862 learned_addr learned my addr 172.21.15.84:0/2041279862 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:20.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.504+0000 7f44553fd640 1 --2- 172.21.15.84:0/2041279862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f445015cc80 0x7f44500cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:20.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.504+0000 7f4454bfc640 1 -- 172.21.15.84:0/2041279862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f445015cc80 msgr2=0x7f44500cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:20.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.504+0000 7f4454bfc640 1 --2- 172.21.15.84:0/2041279862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f445015cc80 0x7f44500cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.504+0000 7f4454bfc640 1 -- 172.21.15.84:0/2041279862 >> v1:172.21.15.157:6789/0 conn(0x7f4450071fa0 legacy=0x7f44501607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:20.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.504+0000 7f4454bfc640 1 -- 172.21.15.84:0/2041279862 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4450072410 con 0x7f44500cdd60 2023-10-16T11:25:20.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.505+0000 7f4454bfc640 1 --2- 172.21.15.84:0/2041279862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44500cdd60 0x7f4450071a60 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f444c009e50 tx=0x7f444c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b45ce79fad877ab server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:20.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.506+0000 7f4443fff640 1 -- 172.21.15.84:0/2041279862 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c02ca70 con 0x7f44500cdd60 2023-10-16T11:25:20.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.506+0000 7f4443fff640 1 -- 172.21.15.84:0/2041279862 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c02cbd0 con 0x7f44500cdd60 2023-10-16T11:25:20.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.506+0000 7f4443fff640 1 -- 172.21.15.84:0/2041279862 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c0343f0 con 0x7f44500cdd60 2023-10-16T11:25:20.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.506+0000 7f44563ff640 1 -- 172.21.15.84:0/2041279862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44500cdd60 msgr2=0x7f4450071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:20.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.506+0000 7f44563ff640 1 --2- 172.21.15.84:0/2041279862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44500cdd60 0x7f4450071a60 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f444c009e50 tx=0x7f444c02bed0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.506+0000 7f44563ff640 1 -- 172.21.15.84:0/2041279862 shutdown_connections 2023-10-16T11:25:20.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.506+0000 7f44563ff640 1 --2- 172.21.15.84:0/2041279862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44500cdd60 0x7f4450071a60 unknown :-1 s=CLOSED pgs=2851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.506+0000 7f44563ff640 1 --2- 172.21.15.84:0/2041279862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f445015cc80 0x7f44500cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.506+0000 7f44563ff640 1 -- 172.21.15.84:0/2041279862 >> 172.21.15.84:0/2041279862 conn(0x7f4450065ed0 msgr2=0x7f44500662e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:20.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.506+0000 7f44563ff640 1 -- 172.21.15.84:0/2041279862 shutdown_connections 2023-10-16T11:25:20.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.506+0000 7f44563ff640 1 -- 172.21.15.84:0/2041279862 wait complete. 2023-10-16T11:25:20.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.508+0000 7f44563ff640 1 Processor -- start 2023-10-16T11:25:20.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.508+0000 7f44563ff640 1 -- start start 2023-10-16T11:25:20.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.509+0000 7f44563ff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4450071fa0 0x7f4450151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:20.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.509+0000 7f44563ff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44500cdd60 0x7f44500b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:20.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.509+0000 7f44563ff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f445015cc80 0x7f44500b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:20.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.509+0000 7f44563ff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4450152870 con 0x7f4450071fa0 2023-10-16T11:25:20.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.509+0000 7f44563ff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4450152ac0 con 0x7f445015cc80 2023-10-16T11:25:20.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.509+0000 7f44563ff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4450152da0 con 0x7f44500cdd60 2023-10-16T11:25:20.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.509+0000 7f44553fd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4450071fa0 0x7f4450151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:20.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.510+0000 7f44553fd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4450071fa0 0x7f4450151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42590/0 (socket says 172.21.15.84:42590) 2023-10-16T11:25:20.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.510+0000 7f44553fd640 1 -- 172.21.15.84:0/1084660536 learned_addr learned my addr 172.21.15.84:0/1084660536 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:20.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.510+0000 7f4454bfc640 1 --2- 172.21.15.84:0/1084660536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44500cdd60 0x7f44500b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:20.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.510+0000 7f44553fd640 1 -- 172.21.15.84:0/1084660536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44500cdd60 msgr2=0x7f44500b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:20.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.510+0000 7f44553fd640 1 --2- 172.21.15.84:0/1084660536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44500cdd60 0x7f44500b5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.510+0000 7f44553fd640 1 -- 172.21.15.84:0/1084660536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f445015cc80 msgr2=0x7f44500b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:20.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.510+0000 7f44553fd640 1 --2- 172.21.15.84:0/1084660536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f445015cc80 0x7f44500b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.510+0000 7f44553fd640 1 -- 172.21.15.84:0/1084660536 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c009b30 con 0x7f4450071fa0 2023-10-16T11:25:20.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.510+0000 7f44553fd640 1 --2- 172.21.15.84:0/1084660536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4450071fa0 0x7f4450151ad0 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f444400d5f0 tx=0x7f444400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:20.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.510+0000 7f4441ffb640 1 -- 172.21.15.84:0/1084660536 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44440125f0 con 0x7f4450071fa0 2023-10-16T11:25:20.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.511+0000 7f4441ffb640 1 -- 172.21.15.84:0/1084660536 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4444012750 con 0x7f4450071fa0 2023-10-16T11:25:20.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.511+0000 7f4441ffb640 1 -- 172.21.15.84:0/1084660536 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444012b00 con 0x7f4450071fa0 2023-10-16T11:25:20.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.511+0000 7f44563ff640 1 -- 172.21.15.84:0/1084660536 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4450153080 con 0x7f4450071fa0 2023-10-16T11:25:20.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.511+0000 7f44563ff640 1 -- 172.21.15.84:0/1084660536 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4450152010 con 0x7f4450071fa0 2023-10-16T11:25:20.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.512+0000 7f44563ff640 1 -- 172.21.15.84:0/1084660536 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445006cc10 con 0x7f4450071fa0 2023-10-16T11:25:20.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.513+0000 7f4441ffb640 1 -- 172.21.15.84:0/1084660536 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f4444012c60 con 0x7f4450071fa0 2023-10-16T11:25:20.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.514+0000 7f4441ffb640 1 --2- 172.21.15.84:0/1084660536 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f443406b0b0 0x7f443406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:20.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.514+0000 7f4441ffb640 1 -- 172.21.15.84:0/1084660536 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f444408a360 con 0x7f4450071fa0 2023-10-16T11:25:20.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.514+0000 7f4454bfc640 1 --2- 172.21.15.84:0/1084660536 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f443406b0b0 0x7f443406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:20.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.514+0000 7f4454bfc640 1 --2- 172.21.15.84:0/1084660536 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f443406b0b0 0x7f443406d570 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f444c035000 tx=0x7f444c030000 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:20.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.518+0000 7f4441ffb640 1 -- 172.21.15.84:0/1084660536 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4444054c60 con 0x7f4450071fa0 2023-10-16T11:25:20.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.712+0000 7f44563ff640 1 -- 172.21.15.84:0/1084660536 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f445004e9b0 con 0x7f4450071fa0 2023-10-16T11:25:20.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.714+0000 7f4441ffb640 1 -- 172.21.15.84:0/1084660536 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v226) v1 ==== 74+0+17925 (secure 0 0 0) 0x7f4444058280 con 0x7f4450071fa0 2023-10-16T11:25:20.717 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:20.718 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":226,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:25:17.979565+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:24:38.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:24:39.387564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6834/551895595":"2023-10-17T11:25:14.673046+0000","172.21.15.84:0/1287173368":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/595669474":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/4107174082":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2906390983":"2023-10-17T11:25:09.210125+0000","172.21.15.84:6833/2880641466":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/3419402221":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2161898622":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/455217819":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/4247809558":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/1814319372":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2877946366":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6833/1425409017":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6836/551895595":"2023-10-17T11:25:14.673046+0000","172.21.15.84:6832/1425409017":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/1771162859":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6832/2880641466":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/3655599055":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/1076506559":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/2748224910":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/857172551":"2023-10-17T11:24:58.196616+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:25:20.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.717+0000 7f44563ff640 1 -- 172.21.15.84:0/1084660536 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f443406b0b0 msgr2=0x7f443406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:20.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.717+0000 7f44563ff640 1 --2- 172.21.15.84:0/1084660536 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f443406b0b0 0x7f443406d570 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f444c035000 tx=0x7f444c030000 comp rx=0 tx=0).stop 2023-10-16T11:25:20.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.717+0000 7f44563ff640 1 -- 172.21.15.84:0/1084660536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4450071fa0 msgr2=0x7f4450151ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:20.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.717+0000 7f44563ff640 1 --2- 172.21.15.84:0/1084660536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4450071fa0 0x7f4450151ad0 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f444400d5f0 tx=0x7f444400dab0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.717+0000 7f44563ff640 1 -- 172.21.15.84:0/1084660536 shutdown_connections 2023-10-16T11:25:20.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.717+0000 7f44563ff640 1 --2- 172.21.15.84:0/1084660536 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f443406b0b0 0x7f443406d570 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.717+0000 7f44563ff640 1 --2- 172.21.15.84:0/1084660536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f445015cc80 0x7f44500b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.717+0000 7f44563ff640 1 --2- 172.21.15.84:0/1084660536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44500cdd60 0x7f44500b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.717+0000 7f44563ff640 1 --2- 172.21.15.84:0/1084660536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4450071fa0 0x7f4450151ad0 unknown :-1 s=CLOSED pgs=2852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.717+0000 7f44563ff640 1 -- 172.21.15.84:0/1084660536 >> 172.21.15.84:0/1084660536 conn(0x7f4450065ed0 msgr2=0x7f445006c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:20.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.718+0000 7f44563ff640 1 -- 172.21.15.84:0/1084660536 shutdown_connections 2023-10-16T11:25:20.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.718+0000 7f44563ff640 1 -- 172.21.15.84:0/1084660536 wait complete. 2023-10-16T11:25:20.738 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2023-10-16T11:25:20.739 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:25:20.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.954+0000 7f4544ab4640 1 Processor -- start 2023-10-16T11:25:20.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.955+0000 7f4544ab4640 1 -- start start 2023-10-16T11:25:20.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.956+0000 7f4544ab4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4540159f00 0x7f45400b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:20.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.956+0000 7f4544ab4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f454006f7f0 con 0x7f454015d880 2023-10-16T11:25:20.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.956+0000 7f4544ab4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f454006c5e0 con 0x7f4540159b20 2023-10-16T11:25:20.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.956+0000 7f4544ab4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454006c8c0 con 0x7f4540159f00 2023-10-16T11:25:20.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.956+0000 7f453dd9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4540159f00 0x7f45400b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:20.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.957+0000 7f453dd9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4540159f00 0x7f45400b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42600/0 (socket says 172.21.15.84:42600) 2023-10-16T11:25:20.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.957+0000 7f453dd9b640 1 -- 172.21.15.84:0/3771481286 learned_addr learned my addr 172.21.15.84:0/3771481286 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:20.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.957+0000 7f453ed9d640 1 --1- 172.21.15.84:0/3771481286 >> v1:172.21.15.84:6789/0 conn(0x7f454015d880 0x7f454006fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49124/0 (socket says 172.21.15.84:49124) 2023-10-16T11:25:20.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.957+0000 7f453dd9b640 1 -- 172.21.15.84:0/3771481286 >> v1:172.21.15.157:6789/0 conn(0x7f4540159b20 legacy=0x7f454006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:20.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.957+0000 7f453dd9b640 1 -- 172.21.15.84:0/3771481286 >> v1:172.21.15.84:6789/0 conn(0x7f454015d880 legacy=0x7f454006fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:20.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.957+0000 7f453dd9b640 1 -- 172.21.15.84:0/3771481286 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454006cba0 con 0x7f4540159f00 2023-10-16T11:25:20.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.958+0000 7f453dd9b640 1 --2- 172.21.15.84:0/3771481286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4540159f00 0x7f45400b0410 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7f4530009e50 tx=0x7f453002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05a4e9e905886fd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:20.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.958+0000 7f453d59a640 1 -- 172.21.15.84:0/3771481286 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453002cb10 con 0x7f4540159f00 2023-10-16T11:25:20.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.958+0000 7f453d59a640 1 -- 172.21.15.84:0/3771481286 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453002cc70 con 0x7f4540159f00 2023-10-16T11:25:20.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.958+0000 7f453d59a640 1 -- 172.21.15.84:0/3771481286 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45300343f0 con 0x7f4540159f00 2023-10-16T11:25:20.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.959+0000 7f4544ab4640 1 -- 172.21.15.84:0/3771481286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4540159f00 msgr2=0x7f45400b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:20.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.959+0000 7f4544ab4640 1 --2- 172.21.15.84:0/3771481286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4540159f00 0x7f45400b0410 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7f4530009e50 tx=0x7f453002bf70 comp rx=0 tx=0).stop 2023-10-16T11:25:20.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.959+0000 7f4544ab4640 1 -- 172.21.15.84:0/3771481286 shutdown_connections 2023-10-16T11:25:20.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.959+0000 7f4544ab4640 1 --2- 172.21.15.84:0/3771481286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4540159f00 0x7f45400b0410 unknown :-1 s=CLOSED pgs=2854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.959+0000 7f4544ab4640 1 -- 172.21.15.84:0/3771481286 >> 172.21.15.84:0/3771481286 conn(0x7f45400cb410 msgr2=0x7f45400cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:20.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.959+0000 7f4544ab4640 1 -- 172.21.15.84:0/3771481286 shutdown_connections 2023-10-16T11:25:20.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.959+0000 7f4544ab4640 1 -- 172.21.15.84:0/3771481286 wait complete. 2023-10-16T11:25:20.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.961+0000 7f4544ab4640 1 Processor -- start 2023-10-16T11:25:20.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.961+0000 7f4544ab4640 1 -- start start 2023-10-16T11:25:20.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.962+0000 7f4544ab4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4540159b20 0x7f454014e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:20.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.962+0000 7f4544ab4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4540159f00 0x7f454015c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:20.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.962+0000 7f4544ab4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f454015d880 0x7f45400b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:20.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.962+0000 7f4544ab4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45400c8520 con 0x7f454015d880 2023-10-16T11:25:20.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.962+0000 7f4544ab4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454015c760 con 0x7f4540159f00 2023-10-16T11:25:20.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.962+0000 7f4544ab4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454015aea0 con 0x7f4540159b20 2023-10-16T11:25:20.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.962+0000 7f453ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f454015d880 0x7f45400b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:20.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.962+0000 7f453ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f454015d880 0x7f45400b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42610/0 (socket says 172.21.15.84:42610) 2023-10-16T11:25:20.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.962+0000 7f453ed9d640 1 -- 172.21.15.84:0/1512934534 learned_addr learned my addr 172.21.15.84:0/1512934534 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:20.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.963+0000 7f453ed9d640 1 -- 172.21.15.84:0/1512934534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4540159b20 msgr2=0x7f454014e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:20.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.963+0000 7f453ed9d640 1 --2- 172.21.15.84:0/1512934534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4540159b20 0x7f454014e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.963+0000 7f453ed9d640 1 -- 172.21.15.84:0/1512934534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4540159f00 msgr2=0x7f454015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:20.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.963+0000 7f453ed9d640 1 --2- 172.21.15.84:0/1512934534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4540159f00 0x7f454015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:20.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.963+0000 7f453ed9d640 1 -- 172.21.15.84:0/1512934534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4530009b30 con 0x7f454015d880 2023-10-16T11:25:20.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.963+0000 7f453ed9d640 1 --2- 172.21.15.84:0/1512934534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f454015d880 0x7f45400b12d0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f4538001eb0 tx=0x7f453800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:20.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.963+0000 7f4536ffd640 1 -- 172.21.15.84:0/1512934534 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45380074e0 con 0x7f454015d880 2023-10-16T11:25:20.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.963+0000 7f4536ffd640 1 -- 172.21.15.84:0/1512934534 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4538007640 con 0x7f454015d880 2023-10-16T11:25:20.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.963+0000 7f4536ffd640 1 -- 172.21.15.84:0/1512934534 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45380049e0 con 0x7f454015d880 2023-10-16T11:25:20.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.963+0000 7f4544ab4640 1 -- 172.21.15.84:0/1512934534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454015b180 con 0x7f454015d880 2023-10-16T11:25:20.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.963+0000 7f4544ab4640 1 -- 172.21.15.84:0/1512934534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454015b740 con 0x7f454015d880 2023-10-16T11:25:20.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.964+0000 7f4544ab4640 1 -- 172.21.15.84:0/1512934534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454004e160 con 0x7f454015d880 2023-10-16T11:25:20.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.965+0000 7f4536ffd640 1 -- 172.21.15.84:0/1512934534 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f4538004b40 con 0x7f454015d880 2023-10-16T11:25:20.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.966+0000 7f4536ffd640 1 --2- 172.21.15.84:0/1512934534 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f452006b0b0 0x7f452006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:20.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.966+0000 7f4536ffd640 1 -- 172.21.15.84:0/1512934534 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f4538089820 con 0x7f454015d880 2023-10-16T11:25:20.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.967+0000 7f453e59c640 1 --2- 172.21.15.84:0/1512934534 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f452006b0b0 0x7f452006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:20.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.967+0000 7f453e59c640 1 --2- 172.21.15.84:0/1512934534 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f452006b0b0 0x7f452006d570 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4528003cc0 tx=0x7f45280079b0 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:20.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:20.971+0000 7f4536ffd640 1 -- 172.21.15.84:0/1512934534 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4538054130 con 0x7f454015d880 2023-10-16T11:25:21.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.202+0000 7f4544ab4640 1 -- 172.21.15.84:0/1512934534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f454015d660 con 0x7f454015d880 2023-10-16T11:25:21.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.203+0000 7f4536ffd640 1 -- 172.21.15.84:0/1512934534 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4933 (secure 0 0 0) 0x7f4538057750 con 0x7f454015d880 2023-10-16T11:25:21.205 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:21.206 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":122,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3106898227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3106898227},{"type":"v1","addr":"172.21.15.84:6837","nonce":3106898227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":7471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2296953749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2296953749},{"type":"v1","addr":"172.21.15.157:6834","nonce":2296953749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":12015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/3350751452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3350751452},{"type":"v1","addr":"172.21.15.84:6836","nonce":3350751452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:24:40.366708+0000","modified":"2023-10-16T11:24:43.431532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7474":{"gid":7474,"name":"d","rank":0,"incarnation":116,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6835/3992377882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3992377882},{"type":"v1","addr":"172.21.15.157:6835","nonce":3992377882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-16T11:25:21.206 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 122 2023-10-16T11:25:21.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.206+0000 7f4544ab4640 1 -- 172.21.15.84:0/1512934534 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f452006b0b0 msgr2=0x7f452006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:21.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.206+0000 7f4544ab4640 1 --2- 172.21.15.84:0/1512934534 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f452006b0b0 0x7f452006d570 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4528003cc0 tx=0x7f45280079b0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.206+0000 7f4544ab4640 1 -- 172.21.15.84:0/1512934534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f454015d880 msgr2=0x7f45400b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:21.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.206+0000 7f4544ab4640 1 --2- 172.21.15.84:0/1512934534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f454015d880 0x7f45400b12d0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f4538001eb0 tx=0x7f453800c820 comp rx=0 tx=0).stop 2023-10-16T11:25:21.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.206+0000 7f4544ab4640 1 -- 172.21.15.84:0/1512934534 shutdown_connections 2023-10-16T11:25:21.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.206+0000 7f4544ab4640 1 --2- 172.21.15.84:0/1512934534 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f452006b0b0 0x7f452006d570 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.206+0000 7f4544ab4640 1 --2- 172.21.15.84:0/1512934534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f454015d880 0x7f45400b12d0 unknown :-1 s=CLOSED pgs=2855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.206+0000 7f4544ab4640 1 --2- 172.21.15.84:0/1512934534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4540159f00 0x7f454015c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.206+0000 7f4544ab4640 1 --2- 172.21.15.84:0/1512934534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4540159b20 0x7f454014e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.206+0000 7f4544ab4640 1 -- 172.21.15.84:0/1512934534 >> 172.21.15.84:0/1512934534 conn(0x7f45400cb410 msgr2=0x7f45400cc510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:21.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.206+0000 7f4544ab4640 1 -- 172.21.15.84:0/1512934534 shutdown_connections 2023-10-16T11:25:21.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.206+0000 7f4544ab4640 1 -- 172.21.15.84:0/1512934534 wait complete. 2023-10-16T11:25:21.226 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:25:21.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.431+0000 7f2353583640 1 Processor -- start 2023-10-16T11:25:21.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.432+0000 7f2353583640 1 -- start start 2023-10-16T11:25:21.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.433+0000 7f2353583640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f234c159b20 0x7f234c06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:21.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.433+0000 7f2353583640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f234c0b0b20 0x7f234c06fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:21.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.433+0000 7f2353583640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f234c06f7f0 con 0x7f234c159f00 2023-10-16T11:25:21.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.433+0000 7f2353583640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f234c06c5e0 con 0x7f234c0b0b20 2023-10-16T11:25:21.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.433+0000 7f2353583640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f234c06c8c0 con 0x7f234c159b20 2023-10-16T11:25:21.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.433+0000 7f2350b1e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f234c159f00 0x7f234c0b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49132/0 (socket says 172.21.15.84:49132) 2023-10-16T11:25:21.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.433+0000 7f2350b1e640 1 -- 172.21.15.84:0/346816984 learned_addr learned my addr 172.21.15.84:0/346816984 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:21.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.433+0000 7f2351b20640 1 --2- 172.21.15.84:0/346816984 >> v2:172.21.15.84:3300/0 conn(0x7f234c0b0b20 0x7f234c06fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:21.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.433+0000 7f235131f640 1 --2- 172.21.15.84:0/346816984 >> v2:172.21.15.157:3300/0 conn(0x7f234c159b20 0x7f234c06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:21.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.434+0000 7f2343fff640 1 -- 172.21.15.84:0/346816984 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922432746 0 0) 0x7f234c06f7f0 con 0x7f234c159f00 2023-10-16T11:25:21.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.434+0000 7f2343fff640 1 -- 172.21.15.84:0/346816984 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2338005730 con 0x7f234c159f00 2023-10-16T11:25:21.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.434+0000 7f2351b20640 1 -- 172.21.15.84:0/346816984 >> v2:172.21.15.157:3300/0 conn(0x7f234c159b20 msgr2=0x7f234c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:21.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.434+0000 7f2351b20640 1 --2- 172.21.15.84:0/346816984 >> v2:172.21.15.157:3300/0 conn(0x7f234c159b20 0x7f234c06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.434+0000 7f2351b20640 1 -- 172.21.15.84:0/346816984 >> v1:172.21.15.84:6789/0 conn(0x7f234c159f00 legacy=0x7f234c0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:21.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.434+0000 7f2351b20640 1 -- 172.21.15.84:0/346816984 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234c06cba0 con 0x7f234c0b0b20 2023-10-16T11:25:21.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.435+0000 7f2351b20640 1 --2- 172.21.15.84:0/346816984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f234c0b0b20 0x7f234c06fc80 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7f234c0c9340 tx=0x7f234402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609b2adad3c9ddff server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:21.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.435+0000 7f2343fff640 1 -- 172.21.15.84:0/346816984 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234402cc80 con 0x7f234c0b0b20 2023-10-16T11:25:21.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.435+0000 7f2343fff640 1 -- 172.21.15.84:0/346816984 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234402cde0 con 0x7f234c0b0b20 2023-10-16T11:25:21.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.435+0000 7f2343fff640 1 -- 172.21.15.84:0/346816984 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2344034660 con 0x7f234c0b0b20 2023-10-16T11:25:21.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.436+0000 7f2353583640 1 -- 172.21.15.84:0/346816984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f234c0b0b20 msgr2=0x7f234c06fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:21.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.436+0000 7f2353583640 1 --2- 172.21.15.84:0/346816984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f234c0b0b20 0x7f234c06fc80 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7f234c0c9340 tx=0x7f234402bed0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.436+0000 7f2353583640 1 -- 172.21.15.84:0/346816984 shutdown_connections 2023-10-16T11:25:21.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.436+0000 7f2353583640 1 --2- 172.21.15.84:0/346816984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f234c0b0b20 0x7f234c06fc80 unknown :-1 s=CLOSED pgs=2857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.436+0000 7f2353583640 1 --2- 172.21.15.84:0/346816984 >> v2:172.21.15.157:3300/0 conn(0x7f234c159b20 0x7f234c06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.436+0000 7f2353583640 1 -- 172.21.15.84:0/346816984 >> 172.21.15.84:0/346816984 conn(0x7f234c0cb410 msgr2=0x7f234c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:21.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.436+0000 7f2353583640 1 -- 172.21.15.84:0/346816984 shutdown_connections 2023-10-16T11:25:21.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.436+0000 7f2353583640 1 -- 172.21.15.84:0/346816984 wait complete. 2023-10-16T11:25:21.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.438+0000 7f2353583640 1 Processor -- start 2023-10-16T11:25:21.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.438+0000 7f2353583640 1 -- start start 2023-10-16T11:25:21.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.439+0000 7f2353583640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f234c0b0b20 0x7f234c0c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:21.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.439+0000 7f2353583640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f234c159b20 0x7f234c151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:21.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.439+0000 7f2353583640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c159f00 0x7f234c156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:21.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.439+0000 7f2353583640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c15bc30 con 0x7f234c0b0b20 2023-10-16T11:25:21.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.439+0000 7f2353583640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c15b390 con 0x7f234c159f00 2023-10-16T11:25:21.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.439+0000 7f2353583640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f234c15b670 con 0x7f234c159b20 2023-10-16T11:25:21.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.439+0000 7f235131f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f234c0b0b20 0x7f234c0c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:21.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.439+0000 7f235131f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f234c0b0b20 0x7f234c0c8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42622/0 (socket says 172.21.15.84:42622) 2023-10-16T11:25:21.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.439+0000 7f235131f640 1 -- 172.21.15.84:0/2698736420 learned_addr learned my addr 172.21.15.84:0/2698736420 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:21.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.439+0000 7f235131f640 1 -- 172.21.15.84:0/2698736420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f234c159b20 msgr2=0x7f234c151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:21.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.439+0000 7f235131f640 1 --2- 172.21.15.84:0/2698736420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f234c159b20 0x7f234c151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.440+0000 7f235131f640 1 -- 172.21.15.84:0/2698736420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c159f00 msgr2=0x7f234c156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:21.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.440+0000 7f235131f640 1 --2- 172.21.15.84:0/2698736420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c159f00 0x7f234c156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.440+0000 7f235131f640 1 -- 172.21.15.84:0/2698736420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2344009a70 con 0x7f234c0b0b20 2023-10-16T11:25:21.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.440+0000 7f235131f640 1 --2- 172.21.15.84:0/2698736420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f234c0b0b20 0x7f234c0c8600 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f234800d800 tx=0x7f234800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:21.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.440+0000 7f2341ffb640 1 -- 172.21.15.84:0/2698736420 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234800b6c0 con 0x7f234c0b0b20 2023-10-16T11:25:21.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.440+0000 7f2341ffb640 1 -- 172.21.15.84:0/2698736420 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234800b820 con 0x7f234c0b0b20 2023-10-16T11:25:21.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.440+0000 7f2353583640 1 -- 172.21.15.84:0/2698736420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234c15b950 con 0x7f234c0b0b20 2023-10-16T11:25:21.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.440+0000 7f2341ffb640 1 -- 172.21.15.84:0/2698736420 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234800bbb0 con 0x7f234c0b0b20 2023-10-16T11:25:21.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.440+0000 7f2353583640 1 -- 172.21.15.84:0/2698736420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234c152430 con 0x7f234c0b0b20 2023-10-16T11:25:21.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.441+0000 7f2353583640 1 -- 172.21.15.84:0/2698736420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f234c04e160 con 0x7f234c0b0b20 2023-10-16T11:25:21.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.442+0000 7f2341ffb640 1 -- 172.21.15.84:0/2698736420 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f234800bd10 con 0x7f234c0b0b20 2023-10-16T11:25:21.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.443+0000 7f2341ffb640 1 --2- 172.21.15.84:0/2698736420 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f233006b0b0 0x7f233006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:21.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.443+0000 7f2341ffb640 1 -- 172.21.15.84:0/2698736420 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f234808a350 con 0x7f234c0b0b20 2023-10-16T11:25:21.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.443+0000 7f2350b1e640 1 --2- 172.21.15.84:0/2698736420 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f233006b0b0 0x7f233006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:21.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.444+0000 7f2350b1e640 1 --2- 172.21.15.84:0/2698736420 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f233006b0b0 0x7f233006d570 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f234c0cd6b0 tx=0x7f233c007930 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:21.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.448+0000 7f2341ffb640 1 -- 172.21.15.84:0/2698736420 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2348054cf0 con 0x7f234c0b0b20 2023-10-16T11:25:21.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.677+0000 7f2353583640 1 -- 172.21.15.84:0/2698736420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f234c0ccdd0 con 0x7f234c0b0b20 2023-10-16T11:25:21.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.678+0000 7f2341ffb640 1 -- 172.21.15.84:0/2698736420 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4933 (secure 0 0 0) 0x7f2348058310 con 0x7f234c0b0b20 2023-10-16T11:25:21.680 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:21.680 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":122,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3106898227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3106898227},{"type":"v1","addr":"172.21.15.84:6837","nonce":3106898227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":7471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2296953749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2296953749},{"type":"v1","addr":"172.21.15.157:6834","nonce":2296953749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":12015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/3350751452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3350751452},{"type":"v1","addr":"172.21.15.84:6836","nonce":3350751452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:24:40.366708+0000","modified":"2023-10-16T11:24:43.431532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7474":{"gid":7474,"name":"d","rank":0,"incarnation":116,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6835/3992377882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3992377882},{"type":"v1","addr":"172.21.15.157:6835","nonce":3992377882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-16T11:25:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 122 2023-10-16T11:25:21.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.680+0000 7f2353583640 1 -- 172.21.15.84:0/2698736420 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f233006b0b0 msgr2=0x7f233006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.680+0000 7f2353583640 1 --2- 172.21.15.84:0/2698736420 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f233006b0b0 0x7f233006d570 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f234c0cd6b0 tx=0x7f233c007930 comp rx=0 tx=0).stop 2023-10-16T11:25:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.680+0000 7f2353583640 1 -- 172.21.15.84:0/2698736420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f234c0b0b20 msgr2=0x7f234c0c8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.680+0000 7f2353583640 1 --2- 172.21.15.84:0/2698736420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f234c0b0b20 0x7f234c0c8600 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f234800d800 tx=0x7f234800dcc0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.680+0000 7f2353583640 1 -- 172.21.15.84:0/2698736420 shutdown_connections 2023-10-16T11:25:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.680+0000 7f2353583640 1 --2- 172.21.15.84:0/2698736420 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f233006b0b0 0x7f233006d570 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.680+0000 7f2353583640 1 --2- 172.21.15.84:0/2698736420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c159f00 0x7f234c156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.680+0000 7f2353583640 1 --2- 172.21.15.84:0/2698736420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f234c159b20 0x7f234c151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.680+0000 7f2353583640 1 --2- 172.21.15.84:0/2698736420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f234c0b0b20 0x7f234c0c8600 unknown :-1 s=CLOSED pgs=2858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.680+0000 7f2353583640 1 -- 172.21.15.84:0/2698736420 >> 172.21.15.84:0/2698736420 conn(0x7f234c0cb410 msgr2=0x7f234c15d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.680+0000 7f2353583640 1 -- 172.21.15.84:0/2698736420 shutdown_connections 2023-10-16T11:25:21.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.681+0000 7f2353583640 1 -- 172.21.15.84:0/2698736420 wait complete. 2023-10-16T11:25:21.701 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:25:21.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.902+0000 7fa656a34640 1 Processor -- start 2023-10-16T11:25:21.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.902+0000 7fa656a34640 1 -- start start 2023-10-16T11:25:21.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.904+0000 7fa656a34640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6500caa10 0x7fa65006fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:21.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.904+0000 7fa656a34640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa6500cc920 0x7fa6500ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:21.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.904+0000 7fa656a34640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa650159790 con 0x7fa6501593b0 2023-10-16T11:25:21.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.904+0000 7fa656a34640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6500c9b20 con 0x7fa6500cc920 2023-10-16T11:25:21.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.904+0000 7fa656a34640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6500c9e00 con 0x7fa6500caa10 2023-10-16T11:25:21.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.904+0000 7fa654fd1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa6500cc920 0x7fa6500ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:21.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.905+0000 7fa654fd1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa6500cc920 0x7fa6500ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42626/0 (socket says 172.21.15.84:42626) 2023-10-16T11:25:21.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.905+0000 7fa654fd1640 1 -- 172.21.15.84:0/2901347671 learned_addr learned my addr 172.21.15.84:0/2901347671 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:21.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.905+0000 7fa647fff640 1 --2- 172.21.15.84:0/2901347671 >> v2:172.21.15.157:3300/0 conn(0x7fa6500caa10 0x7fa65006fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:21.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.905+0000 7fa654fd1640 1 -- 172.21.15.84:0/2901347671 >> v2:172.21.15.157:3300/0 conn(0x7fa6500caa10 msgr2=0x7fa65006fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:21.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.905+0000 7fa654fd1640 1 --2- 172.21.15.84:0/2901347671 >> v2:172.21.15.157:3300/0 conn(0x7fa6500caa10 0x7fa65006fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.905+0000 7fa654fd1640 1 -- 172.21.15.84:0/2901347671 >> v1:172.21.15.157:6789/0 conn(0x7fa6501593b0 legacy=0x7fa65006f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:21.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.905+0000 7fa654fd1640 1 -- 172.21.15.84:0/2901347671 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6500ca0e0 con 0x7fa6500cc920 2023-10-16T11:25:21.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.905+0000 7fa647fff640 1 --2- 172.21.15.84:0/2901347671 >> v2:172.21.15.157:3300/0 conn(0x7fa6500caa10 0x7fa65006fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:21.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.906+0000 7fa654fd1640 1 --2- 172.21.15.84:0/2901347671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6500cc920 0x7fa6500ccdd0 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7fa6500712d0 tx=0x7fa64802bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac22c05b8a7d2ab5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:21.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.906+0000 7fa64f7fe640 1 -- 172.21.15.84:0/2901347671 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64802c960 con 0x7fa6500cc920 2023-10-16T11:25:21.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.906+0000 7fa64f7fe640 1 -- 172.21.15.84:0/2901347671 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64802cac0 con 0x7fa6500cc920 2023-10-16T11:25:21.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.906+0000 7fa64f7fe640 1 -- 172.21.15.84:0/2901347671 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64802ce50 con 0x7fa6500cc920 2023-10-16T11:25:21.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.907+0000 7fa656a34640 1 -- 172.21.15.84:0/2901347671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6500cc920 msgr2=0x7fa6500ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:21.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.907+0000 7fa656a34640 1 --2- 172.21.15.84:0/2901347671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6500cc920 0x7fa6500ccdd0 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7fa6500712d0 tx=0x7fa64802bdc0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.907+0000 7fa656a34640 1 -- 172.21.15.84:0/2901347671 shutdown_connections 2023-10-16T11:25:21.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.907+0000 7fa656a34640 1 --2- 172.21.15.84:0/2901347671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6500cc920 0x7fa6500ccdd0 unknown :-1 s=CLOSED pgs=2859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.907+0000 7fa656a34640 1 --2- 172.21.15.84:0/2901347671 >> v2:172.21.15.157:3300/0 conn(0x7fa6500caa10 0x7fa65006fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.907+0000 7fa656a34640 1 -- 172.21.15.84:0/2901347671 >> 172.21.15.84:0/2901347671 conn(0x7fa6500cb410 msgr2=0x7fa6500cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:21.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.907+0000 7fa656a34640 1 -- 172.21.15.84:0/2901347671 shutdown_connections 2023-10-16T11:25:21.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.907+0000 7fa656a34640 1 -- 172.21.15.84:0/2901347671 wait complete. 2023-10-16T11:25:21.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.909+0000 7fa656a34640 1 Processor -- start 2023-10-16T11:25:21.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.909+0000 7fa656a34640 1 -- start start 2023-10-16T11:25:21.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.910+0000 7fa656a34640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6500caa10 0x7fa6500aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:21.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.910+0000 7fa656a34640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6500cc920 0x7fa6500b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:21.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.910+0000 7fa656a34640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6501593b0 0x7fa6500b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:21.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.910+0000 7fa656a34640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa650150390 con 0x7fa6500caa10 2023-10-16T11:25:21.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.910+0000 7fa656a34640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa650150630 con 0x7fa6500cc920 2023-10-16T11:25:21.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.910+0000 7fa656a34640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa650150910 con 0x7fa6501593b0 2023-10-16T11:25:21.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.911+0000 7fa64ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6500caa10 0x7fa6500aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:21.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.911+0000 7fa64ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6500caa10 0x7fa6500aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42642/0 (socket says 172.21.15.84:42642) 2023-10-16T11:25:21.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.911+0000 7fa64ffff640 1 -- 172.21.15.84:0/1830943544 learned_addr learned my addr 172.21.15.84:0/1830943544 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:21.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.911+0000 7fa647fff640 1 --2- 172.21.15.84:0/1830943544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6500cc920 0x7fa6500b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:21.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.911+0000 7fa654fd1640 1 --2- 172.21.15.84:0/1830943544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6501593b0 0x7fa6500b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:21.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.911+0000 7fa64ffff640 1 -- 172.21.15.84:0/1830943544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6501593b0 msgr2=0x7fa6500b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:21.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.911+0000 7fa64ffff640 1 --2- 172.21.15.84:0/1830943544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6501593b0 0x7fa6500b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.911+0000 7fa64ffff640 1 -- 172.21.15.84:0/1830943544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6500cc920 msgr2=0x7fa6500b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:21.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.911+0000 7fa64ffff640 1 --2- 172.21.15.84:0/1830943544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6500cc920 0x7fa6500b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:21.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.911+0000 7fa64ffff640 1 -- 172.21.15.84:0/1830943544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa648009b30 con 0x7fa6500caa10 2023-10-16T11:25:21.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.911+0000 7fa64ffff640 1 --2- 172.21.15.84:0/1830943544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6500caa10 0x7fa6500aedd0 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7fa64000c410 tx=0x7fa64000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:21.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.912+0000 7fa64d7fa640 1 -- 172.21.15.84:0/1830943544 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6400035f0 con 0x7fa6500caa10 2023-10-16T11:25:21.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.912+0000 7fa64d7fa640 1 -- 172.21.15.84:0/1830943544 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa640003750 con 0x7fa6500caa10 2023-10-16T11:25:21.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.912+0000 7fa64d7fa640 1 -- 172.21.15.84:0/1830943544 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640003b40 con 0x7fa6500caa10 2023-10-16T11:25:21.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.912+0000 7fa656a34640 1 -- 172.21.15.84:0/1830943544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa650150bf0 con 0x7fa6500caa10 2023-10-16T11:25:21.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.912+0000 7fa656a34640 1 -- 172.21.15.84:0/1830943544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6500b4520 con 0x7fa6500caa10 2023-10-16T11:25:21.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.913+0000 7fa656a34640 1 -- 172.21.15.84:0/1830943544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa650151110 con 0x7fa6500caa10 2023-10-16T11:25:21.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.914+0000 7fa64d7fa640 1 -- 172.21.15.84:0/1830943544 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fa640003ca0 con 0x7fa6500caa10 2023-10-16T11:25:21.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.915+0000 7fa64d7fa640 1 --2- 172.21.15.84:0/1830943544 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fa63006b0b0 0x7fa63006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:21.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.915+0000 7fa64d7fa640 1 -- 172.21.15.84:0/1830943544 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6652+0+0 (secure 0 0 0) 0x7fa640088f70 con 0x7fa6500caa10 2023-10-16T11:25:21.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.915+0000 7fa647fff640 1 --2- 172.21.15.84:0/1830943544 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fa63006b0b0 0x7fa63006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:21.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.916+0000 7fa647fff640 1 --2- 172.21.15.84:0/1830943544 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fa63006b0b0 0x7fa63006d570 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa6500cba50 tx=0x7fa638008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:21.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:21.919+0000 7fa64d7fa640 1 -- 172.21.15.84:0/1830943544 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa6400bbfe0 con 0x7fa6500caa10 2023-10-16T11:25:22.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.100+0000 7fa656a34640 1 -- 172.21.15.84:0/1830943544 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6500b46b0 con 0x7fa6500caa10 2023-10-16T11:25:22.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.102+0000 7fa64d7fa640 1 -- 172.21.15.84:0/1830943544 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v226) v1 ==== 74+0+17925 (secure 0 0 0) 0x7fa650151110 con 0x7fa6500caa10 2023-10-16T11:25:22.105 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:22.105 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":226,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:25:17.979565+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:24:38.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:24:39.387564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6834/551895595":"2023-10-17T11:25:14.673046+0000","172.21.15.84:0/1287173368":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/595669474":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/4107174082":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2906390983":"2023-10-17T11:25:09.210125+0000","172.21.15.84:6833/2880641466":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/3419402221":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2161898622":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/455217819":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/4247809558":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/1814319372":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2877946366":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6833/1425409017":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6836/551895595":"2023-10-17T11:25:14.673046+0000","172.21.15.84:6832/1425409017":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/1771162859":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6832/2880641466":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/3655599055":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/1076506559":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/2748224910":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/857172551":"2023-10-17T11:24:58.196616+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:25:22.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.105+0000 7fa656a34640 1 -- 172.21.15.84:0/1830943544 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fa63006b0b0 msgr2=0x7fa63006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:22.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.105+0000 7fa656a34640 1 --2- 172.21.15.84:0/1830943544 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fa63006b0b0 0x7fa63006d570 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa6500cba50 tx=0x7fa638008040 comp rx=0 tx=0).stop 2023-10-16T11:25:22.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.105+0000 7fa656a34640 1 -- 172.21.15.84:0/1830943544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6500caa10 msgr2=0x7fa6500aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:22.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.105+0000 7fa656a34640 1 --2- 172.21.15.84:0/1830943544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6500caa10 0x7fa6500aedd0 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7fa64000c410 tx=0x7fa64000c8d0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.105+0000 7fa656a34640 1 -- 172.21.15.84:0/1830943544 shutdown_connections 2023-10-16T11:25:22.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.105+0000 7fa656a34640 1 --2- 172.21.15.84:0/1830943544 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fa63006b0b0 0x7fa63006d570 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.105+0000 7fa656a34640 1 --2- 172.21.15.84:0/1830943544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6501593b0 0x7fa6500b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.105+0000 7fa656a34640 1 --2- 172.21.15.84:0/1830943544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6500cc920 0x7fa6500b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.105+0000 7fa656a34640 1 --2- 172.21.15.84:0/1830943544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6500caa10 0x7fa6500aedd0 unknown :-1 s=CLOSED pgs=2860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.105+0000 7fa656a34640 1 -- 172.21.15.84:0/1830943544 >> 172.21.15.84:0/1830943544 conn(0x7fa6500cb410 msgr2=0x7fa6500b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:22.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.105+0000 7fa656a34640 1 -- 172.21.15.84:0/1830943544 shutdown_connections 2023-10-16T11:25:22.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.105+0000 7fa656a34640 1 -- 172.21.15.84:0/1830943544 wait complete. 2023-10-16T11:25:22.123 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2023-10-16T11:25:22.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.324+0000 7fe330075640 1 Processor -- start 2023-10-16T11:25:22.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.324+0000 7fe330075640 1 -- start start 2023-10-16T11:25:22.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.325+0000 7fe330075640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe3280cf710 0x7fe3280b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:22.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.325+0000 7fe330075640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3280d18c0 con 0x7fe3280cfaf0 2023-10-16T11:25:22.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.325+0000 7fe330075640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3280d3980 con 0x7fe3280b6cd0 2023-10-16T11:25:22.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.325+0000 7fe330075640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3280d3c60 con 0x7fe3280cf710 2023-10-16T11:25:22.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.326+0000 7fe32de11640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe3280cf710 0x7fe3280b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:22.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.326+0000 7fe32de11640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe3280cf710 0x7fe3280b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42658/0 (socket says 172.21.15.84:42658) 2023-10-16T11:25:22.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.326+0000 7fe32de11640 1 -- 172.21.15.84:0/357919486 learned_addr learned my addr 172.21.15.84:0/357919486 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:22.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.326+0000 7fe32d610640 1 --1- 172.21.15.84:0/357919486 >> v1:172.21.15.84:6789/0 conn(0x7fe3280cfaf0 0x7fe3280d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49142/0 (socket says 172.21.15.84:49142) 2023-10-16T11:25:22.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.326+0000 7fe32de11640 1 -- 172.21.15.84:0/357919486 >> v1:172.21.15.157:6789/0 conn(0x7fe3280b6cd0 legacy=0x7fe3280d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:22.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.326+0000 7fe32de11640 1 -- 172.21.15.84:0/357919486 >> v1:172.21.15.84:6789/0 conn(0x7fe3280cfaf0 legacy=0x7fe3280d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:22.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.326+0000 7fe32de11640 1 -- 172.21.15.84:0/357919486 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3280ce630 con 0x7fe3280cf710 2023-10-16T11:25:22.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.326+0000 7fe32ce0f640 1 -- 172.21.15.84:0/357919486 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 535312352 0 0) 0x7fe3280d18c0 con 0x7fe3280cfaf0 2023-10-16T11:25:22.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.327+0000 7fe32de11640 1 --2- 172.21.15.84:0/357919486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3280cf710 0x7fe3280b2d10 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7fe318009ea0 tx=0x7fe31802bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e965ab2c80f1f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:22.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.328+0000 7fe32ce0f640 1 -- 172.21.15.84:0/357919486 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31802cac0 con 0x7fe3280cf710 2023-10-16T11:25:22.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.328+0000 7fe32ce0f640 1 -- 172.21.15.84:0/357919486 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31802cc20 con 0x7fe3280cf710 2023-10-16T11:25:22.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.328+0000 7fe32ce0f640 1 -- 172.21.15.84:0/357919486 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3180343f0 con 0x7fe3280cf710 2023-10-16T11:25:22.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.328+0000 7fe330075640 1 -- 172.21.15.84:0/357919486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3280cf710 msgr2=0x7fe3280b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:22.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.328+0000 7fe330075640 1 --2- 172.21.15.84:0/357919486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3280cf710 0x7fe3280b2d10 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7fe318009ea0 tx=0x7fe31802bf20 comp rx=0 tx=0).stop 2023-10-16T11:25:22.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.328+0000 7fe330075640 1 -- 172.21.15.84:0/357919486 shutdown_connections 2023-10-16T11:25:22.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.328+0000 7fe330075640 1 --2- 172.21.15.84:0/357919486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3280cf710 0x7fe3280b2d10 unknown :-1 s=CLOSED pgs=2862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.328+0000 7fe330075640 1 -- 172.21.15.84:0/357919486 >> 172.21.15.84:0/357919486 conn(0x7fe3280d2cc0 msgr2=0x7fe3280d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:22.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.329+0000 7fe330075640 1 -- 172.21.15.84:0/357919486 shutdown_connections 2023-10-16T11:25:22.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.329+0000 7fe330075640 1 -- 172.21.15.84:0/357919486 wait complete. 2023-10-16T11:25:22.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.330+0000 7fe330075640 1 Processor -- start 2023-10-16T11:25:22.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.330+0000 7fe330075640 1 -- start start 2023-10-16T11:25:22.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.331+0000 7fe330075640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3280b6cd0 0x7fe3281545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:22.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.331+0000 7fe330075640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3280cf710 0x7fe3281504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:22.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.331+0000 7fe330075640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3280cfaf0 0x7fe3280d0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:22.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.331+0000 7fe330075640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3280ca1a0 con 0x7fe3280cfaf0 2023-10-16T11:25:22.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.331+0000 7fe330075640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3281e59f0 con 0x7fe3280cf710 2023-10-16T11:25:22.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.331+0000 7fe330075640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3281e5cd0 con 0x7fe3280b6cd0 2023-10-16T11:25:22.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.331+0000 7fe32e612640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3280cfaf0 0x7fe3280d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:22.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.331+0000 7fe32e612640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3280cfaf0 0x7fe3280d0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42660/0 (socket says 172.21.15.84:42660) 2023-10-16T11:25:22.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.331+0000 7fe32e612640 1 -- 172.21.15.84:0/2158819690 learned_addr learned my addr 172.21.15.84:0/2158819690 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:22.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.332+0000 7fe32e612640 1 -- 172.21.15.84:0/2158819690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3280b6cd0 msgr2=0x7fe3281545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:22.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.332+0000 7fe32e612640 1 --2- 172.21.15.84:0/2158819690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3280b6cd0 0x7fe3281545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.332+0000 7fe32e612640 1 -- 172.21.15.84:0/2158819690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3280cf710 msgr2=0x7fe3281504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:22.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.332+0000 7fe32e612640 1 --2- 172.21.15.84:0/2158819690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3280cf710 0x7fe3281504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.332+0000 7fe32e612640 1 -- 172.21.15.84:0/2158819690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe318009b30 con 0x7fe3280cfaf0 2023-10-16T11:25:22.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.332+0000 7fe32e612640 1 --2- 172.21.15.84:0/2158819690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3280cfaf0 0x7fe3280d0870 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7fe32000c410 tx=0x7fe32000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:22.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.332+0000 7fe3267fc640 1 -- 172.21.15.84:0/2158819690 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe320011790 con 0x7fe3280cfaf0 2023-10-16T11:25:22.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.332+0000 7fe3267fc640 1 -- 172.21.15.84:0/2158819690 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3200118f0 con 0x7fe3280cfaf0 2023-10-16T11:25:22.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.332+0000 7fe3267fc640 1 -- 172.21.15.84:0/2158819690 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe320011ce0 con 0x7fe3280cfaf0 2023-10-16T11:25:22.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.332+0000 7fe330075640 1 -- 172.21.15.84:0/2158819690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3281e5fb0 con 0x7fe3280cfaf0 2023-10-16T11:25:22.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.333+0000 7fe330075640 1 -- 172.21.15.84:0/2158819690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe328154bb0 con 0x7fe3280cfaf0 2023-10-16T11:25:22.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.334+0000 7fe330075640 1 -- 172.21.15.84:0/2158819690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3281e64d0 con 0x7fe3280cfaf0 2023-10-16T11:25:22.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.334+0000 7fe3267fc640 1 -- 172.21.15.84:0/2158819690 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe32001c460 con 0x7fe3280cfaf0 2023-10-16T11:25:22.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.335+0000 7fe3267fc640 1 --2- 172.21.15.84:0/2158819690 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe31406b0b0 0x7fe31406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:22.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.335+0000 7fe3267fc640 1 -- 172.21.15.84:0/2158819690 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6652+0+0 (secure 0 0 0) 0x7fe320089a90 con 0x7fe3280cfaf0 2023-10-16T11:25:22.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.336+0000 7fe32de11640 1 --2- 172.21.15.84:0/2158819690 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe31406b0b0 0x7fe31406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:22.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.336+0000 7fe32de11640 1 --2- 172.21.15.84:0/2158819690 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe31406b0b0 0x7fe31406d570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe318035000 tx=0x7fe31802e000 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:22.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.340+0000 7fe3267fc640 1 -- 172.21.15.84:0/2158819690 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe320054460 con 0x7fe3280cfaf0 2023-10-16T11:25:22.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.568+0000 7fe330075640 1 -- 172.21.15.84:0/2158819690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) v1 -- 0x7fe32804e160 con 0x7fe3280cfaf0 2023-10-16T11:25:22.571 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-16T11:25:22.569+0000 7fca0c7ff640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:25:22.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.573+0000 7fe3267fc640 1 -- 172.21.15.84:0/2158819690 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v123) v1 ==== 167+0+0 (secure 0 0 0) 0x7fe320057a80 con 0x7fe3280cfaf0 2023-10-16T11:25:22.575 INFO:teuthology.orchestra.run.smithi084.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:25:22.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.576+0000 7fe330075640 1 -- 172.21.15.84:0/2158819690 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe31406b0b0 msgr2=0x7fe31406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:22.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.576+0000 7fe330075640 1 --2- 172.21.15.84:0/2158819690 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe31406b0b0 0x7fe31406d570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe318035000 tx=0x7fe31802e000 comp rx=0 tx=0).stop 2023-10-16T11:25:22.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.576+0000 7fe330075640 1 -- 172.21.15.84:0/2158819690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3280cfaf0 msgr2=0x7fe3280d0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:22.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.576+0000 7fe330075640 1 --2- 172.21.15.84:0/2158819690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3280cfaf0 0x7fe3280d0870 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7fe32000c410 tx=0x7fe32000c8d0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.576+0000 7fe330075640 1 -- 172.21.15.84:0/2158819690 shutdown_connections 2023-10-16T11:25:22.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.576+0000 7fe330075640 1 --2- 172.21.15.84:0/2158819690 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe31406b0b0 0x7fe31406d570 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.576+0000 7fe330075640 1 --2- 172.21.15.84:0/2158819690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3280cfaf0 0x7fe3280d0870 unknown :-1 s=CLOSED pgs=2863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.576+0000 7fe330075640 1 --2- 172.21.15.84:0/2158819690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3280cf710 0x7fe3281504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.576+0000 7fe330075640 1 --2- 172.21.15.84:0/2158819690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3280b6cd0 0x7fe3281545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.576+0000 7fe330075640 1 -- 172.21.15.84:0/2158819690 >> 172.21.15.84:0/2158819690 conn(0x7fe3280d2cc0 msgr2=0x7fe3280b8340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:22.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.577+0000 7fe330075640 1 -- 172.21.15.84:0/2158819690 shutdown_connections 2023-10-16T11:25:22.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.577+0000 7fe330075640 1 -- 172.21.15.84:0/2158819690 wait complete. 2023-10-16T11:25:22.596 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2023-10-16T11:25:22.599 INFO:tasks.ceph.mds.d.smithi157.stderr: -2206> 2023-10-16T11:24:42.030+0000 7f98f1c64640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:25:22.669 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-10-16T11:25:22.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.795+0000 7fbc1f59e640 1 Processor -- start 2023-10-16T11:25:22.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.796+0000 7fbc1f59e640 1 -- start start 2023-10-16T11:25:22.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.797+0000 7fbc1f59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbc200ad970 0x7fbc200cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:22.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.797+0000 7fbc1f59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc200ccfb0 0x7fbc200cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:22.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.797+0000 7fbc1f59e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2015b9d0 con 0x7fbc2006c040 2023-10-16T11:25:22.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.797+0000 7fbc1f59e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2015bcb0 con 0x7fbc200ad970 2023-10-16T11:25:22.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.797+0000 7fbc1f59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2015bf90 con 0x7fbc200ccfb0 2023-10-16T11:25:22.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.797+0000 7fbc17fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbc200ad970 0x7fbc200cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:22.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.797+0000 7fbc1e59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbc2006c040 0x7fbc2006c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49158/0 (socket says 172.21.15.84:49158) 2023-10-16T11:25:22.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.797+0000 7fbc1e59c640 1 -- 172.21.15.84:0/633716379 learned_addr learned my addr 172.21.15.84:0/633716379 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:22.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.797+0000 7fbc1ed9d640 1 --2- 172.21.15.84:0/633716379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc200ccfb0 0x7fbc200cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:22.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.797+0000 7fbc17fff640 1 -- 172.21.15.84:0/633716379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc200ccfb0 msgr2=0x7fbc200cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:22.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.797+0000 7fbc17fff640 1 --2- 172.21.15.84:0/633716379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc200ccfb0 0x7fbc200cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.798+0000 7fbc17fff640 1 -- 172.21.15.84:0/633716379 >> v1:172.21.15.84:6789/0 conn(0x7fbc2006c040 legacy=0x7fbc2006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.798+0000 7fbc17fff640 1 -- 172.21.15.84:0/633716379 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc200cc270 con 0x7fbc200ad970 2023-10-16T11:25:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.797+0000 7fbc1dd9b640 1 -- 172.21.15.84:0/633716379 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346823428 0 0) 0x7fbc2015b9d0 con 0x7fbc2006c040 2023-10-16T11:25:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.799+0000 7fbc17fff640 1 --2- 172.21.15.84:0/633716379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc200ad970 0x7fbc200cca70 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7fbc080097a0 tx=0x7fbc0802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf69266823499b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.799+0000 7fbc1dd9b640 1 -- 172.21.15.84:0/633716379 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0802d040 con 0x7fbc200ad970 2023-10-16T11:25:22.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.799+0000 7fbc1dd9b640 1 -- 172.21.15.84:0/633716379 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc08031810 con 0x7fbc200ad970 2023-10-16T11:25:22.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.799+0000 7fbc1dd9b640 1 -- 172.21.15.84:0/633716379 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08031ba0 con 0x7fbc200ad970 2023-10-16T11:25:22.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.800+0000 7fbc1f59e640 1 -- 172.21.15.84:0/633716379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc200ad970 msgr2=0x7fbc200cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:22.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.800+0000 7fbc1f59e640 1 --2- 172.21.15.84:0/633716379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc200ad970 0x7fbc200cca70 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7fbc080097a0 tx=0x7fbc0802b8b0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.800+0000 7fbc1f59e640 1 -- 172.21.15.84:0/633716379 shutdown_connections 2023-10-16T11:25:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.800+0000 7fbc1f59e640 1 --2- 172.21.15.84:0/633716379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc200ccfb0 0x7fbc200cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.800+0000 7fbc1f59e640 1 --2- 172.21.15.84:0/633716379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc200ad970 0x7fbc200cca70 unknown :-1 s=CLOSED pgs=2865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.800+0000 7fbc1f59e640 1 -- 172.21.15.84:0/633716379 >> 172.21.15.84:0/633716379 conn(0x7fbc200ce6a0 msgr2=0x7fbc200ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.800+0000 7fbc1f59e640 1 -- 172.21.15.84:0/633716379 shutdown_connections 2023-10-16T11:25:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.800+0000 7fbc1f59e640 1 -- 172.21.15.84:0/633716379 wait complete. 2023-10-16T11:25:22.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.801+0000 7fbc1f59e640 1 Processor -- start 2023-10-16T11:25:22.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.802+0000 7fbc1f59e640 1 -- start start 2023-10-16T11:25:22.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.802+0000 7fbc1f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc2006c040 0x7fbc20156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:22.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.802+0000 7fbc1f59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc200ad970 0x7fbc200c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:22.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.802+0000 7fbc1f59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc200ccfb0 0x7fbc20156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:22.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.802+0000 7fbc1f59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc201573d0 con 0x7fbc2006c040 2023-10-16T11:25:22.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.802+0000 7fbc1f59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc200c8620 con 0x7fbc200ccfb0 2023-10-16T11:25:22.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.802+0000 7fbc1f59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc200c21b0 con 0x7fbc200ad970 2023-10-16T11:25:22.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.803+0000 7fbc1e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc2006c040 0x7fbc20156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:22.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.803+0000 7fbc1e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc2006c040 0x7fbc20156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42680/0 (socket says 172.21.15.84:42680) 2023-10-16T11:25:22.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.803+0000 7fbc1e59c640 1 -- 172.21.15.84:0/431500022 learned_addr learned my addr 172.21.15.84:0/431500022 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:22.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.803+0000 7fbc1e59c640 1 -- 172.21.15.84:0/431500022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc200ad970 msgr2=0x7fbc200c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:22.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.803+0000 7fbc17fff640 1 --2- 172.21.15.84:0/431500022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc200ad970 0x7fbc200c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:22.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.803+0000 7fbc1e59c640 1 --2- 172.21.15.84:0/431500022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc200ad970 0x7fbc200c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.803+0000 7fbc1e59c640 1 -- 172.21.15.84:0/431500022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc200ccfb0 msgr2=0x7fbc20156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:22.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.803+0000 7fbc1e59c640 1 --2- 172.21.15.84:0/431500022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc200ccfb0 0x7fbc20156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:22.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.803+0000 7fbc1e59c640 1 -- 172.21.15.84:0/431500022 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc08009400 con 0x7fbc2006c040 2023-10-16T11:25:22.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.803+0000 7fbc1e59c640 1 --2- 172.21.15.84:0/431500022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc2006c040 0x7fbc20156ad0 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000c500 tx=0x7fbc1000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:22.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.804+0000 7fbc16ffd640 1 -- 172.21.15.84:0/431500022 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc10007560 con 0x7fbc2006c040 2023-10-16T11:25:22.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.804+0000 7fbc16ffd640 1 -- 172.21.15.84:0/431500022 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc100047e0 con 0x7fbc2006c040 2023-10-16T11:25:22.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.804+0000 7fbc1f59e640 1 -- 172.21.15.84:0/431500022 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc200afdf0 con 0x7fbc2006c040 2023-10-16T11:25:22.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.804+0000 7fbc16ffd640 1 -- 172.21.15.84:0/431500022 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc10004b90 con 0x7fbc2006c040 2023-10-16T11:25:22.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.804+0000 7fbc1f59e640 1 -- 172.21.15.84:0/431500022 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc200b0370 con 0x7fbc2006c040 2023-10-16T11:25:22.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.805+0000 7fbc1f59e640 1 -- 172.21.15.84:0/431500022 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2006b330 con 0x7fbc2006c040 2023-10-16T11:25:22.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.806+0000 7fbc16ffd640 1 -- 172.21.15.84:0/431500022 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fbc10004cf0 con 0x7fbc2006c040 2023-10-16T11:25:22.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.806+0000 7fbc16ffd640 1 --2- 172.21.15.84:0/431500022 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fbc0006b0e0 0x7fbc0006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:22.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.807+0000 7fbc16ffd640 1 -- 172.21.15.84:0/431500022 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 6738+0+0 (secure 0 0 0) 0x7fbc10089390 con 0x7fbc2006c040 2023-10-16T11:25:22.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.807+0000 7fbc17fff640 1 --2- 172.21.15.84:0/431500022 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fbc0006b0e0 0x7fbc0006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:22.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.807+0000 7fbc17fff640 1 --2- 172.21.15.84:0/431500022 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fbc0006b0e0 0x7fbc0006d5a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbc08036000 tx=0x7fbc0803c040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:22.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:22.811+0000 7fbc16ffd640 1 -- 172.21.15.84:0/431500022 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbc10053cc0 con 0x7fbc2006c040 2023-10-16T11:25:23.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.027+0000 7fbc1f59e640 1 -- 172.21.15.84:0/431500022 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbc2004e9b0 con 0x7fbc2006c040 2023-10-16T11:25:23.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.576+0000 7fbc16ffd640 1 -- 172.21.15.84:0/431500022 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v124) v1 ==== 107+0+0 (secure 0 0 0) 0x7fbc100572e0 con 0x7fbc2006c040 2023-10-16T11:25:23.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.580+0000 7fbc1f59e640 1 -- 172.21.15.84:0/431500022 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fbc0006b0e0 msgr2=0x7fbc0006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:23.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.580+0000 7fbc1f59e640 1 --2- 172.21.15.84:0/431500022 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fbc0006b0e0 0x7fbc0006d5a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbc08036000 tx=0x7fbc0803c040 comp rx=0 tx=0).stop 2023-10-16T11:25:23.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.580+0000 7fbc1f59e640 1 -- 172.21.15.84:0/431500022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc2006c040 msgr2=0x7fbc20156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:23.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.580+0000 7fbc1f59e640 1 --2- 172.21.15.84:0/431500022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc2006c040 0x7fbc20156ad0 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000c500 tx=0x7fbc1000c9c0 comp rx=0 tx=0).stop 2023-10-16T11:25:23.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.580+0000 7fbc1f59e640 1 -- 172.21.15.84:0/431500022 shutdown_connections 2023-10-16T11:25:23.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.580+0000 7fbc1f59e640 1 --2- 172.21.15.84:0/431500022 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fbc0006b0e0 0x7fbc0006d5a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:23.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.580+0000 7fbc1f59e640 1 --2- 172.21.15.84:0/431500022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc200ccfb0 0x7fbc20156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:23.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.580+0000 7fbc1f59e640 1 --2- 172.21.15.84:0/431500022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbc200ad970 0x7fbc200c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:23.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.580+0000 7fbc1f59e640 1 --2- 172.21.15.84:0/431500022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc2006c040 0x7fbc20156ad0 unknown :-1 s=CLOSED pgs=2866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:23.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.580+0000 7fbc1f59e640 1 -- 172.21.15.84:0/431500022 >> 172.21.15.84:0/431500022 conn(0x7fbc200ce6a0 msgr2=0x7fbc200ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:23.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.581+0000 7fbc1f59e640 1 -- 172.21.15.84:0/431500022 shutdown_connections 2023-10-16T11:25:23.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.581+0000 7fbc1f59e640 1 -- 172.21.15.84:0/431500022 wait complete. 2023-10-16T11:25:23.603 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2023-10-16T11:25:23.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.818+0000 7fc04ca61640 1 Processor -- start 2023-10-16T11:25:23.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.818+0000 7fc04ca61640 1 -- start start 2023-10-16T11:25:23.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.820+0000 7fc04ca61640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc04815cf60 0x7fc04806f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:23.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.820+0000 7fc04ca61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc04806f9d0 0x7fc0480c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:23.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.820+0000 7fc04ca61640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0480c9620 con 0x7fc0480c9ff0 2023-10-16T11:25:23.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.820+0000 7fc04ca61640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc04815d340 con 0x7fc04815cf60 2023-10-16T11:25:23.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.820+0000 7fc04ca61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0480cc860 con 0x7fc04806f9d0 2023-10-16T11:25:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.821+0000 7fc035d9b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc04806f9d0 0x7fc0480c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.821+0000 7fc035d9b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc04806f9d0 0x7fc0480c9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:57208/0 (socket says 172.21.15.84:57208) 2023-10-16T11:25:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.821+0000 7fc035d9b640 1 -- 172.21.15.84:0/3735798949 learned_addr learned my addr 172.21.15.84:0/3735798949 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.821+0000 7fc04659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc04815cf60 0x7fc04806f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.821+0000 7fc035d9b640 1 -- 172.21.15.84:0/3735798949 >> v2:172.21.15.157:3300/0 conn(0x7fc04815cf60 msgr2=0x7fc04806f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:23.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.821+0000 7fc035d9b640 1 --2- 172.21.15.84:0/3735798949 >> v2:172.21.15.157:3300/0 conn(0x7fc04815cf60 0x7fc04806f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:23.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.821+0000 7fc035d9b640 1 -- 172.21.15.84:0/3735798949 >> v1:172.21.15.157:6789/0 conn(0x7fc0480c9ff0 legacy=0x7fc0480b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:23.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.821+0000 7fc035d9b640 1 -- 172.21.15.84:0/3735798949 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0480ad4a0 con 0x7fc04806f9d0 2023-10-16T11:25:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.823+0000 7fc035d9b640 1 --2- 172.21.15.84:0/3735798949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc04806f9d0 0x7fc0480c9ab0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc030009870 tx=0x7fc03002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=631df33b7a216f9d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.823+0000 7fc045d9b640 1 -- 172.21.15.84:0/3735798949 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc030030840 con 0x7fc04806f9d0 2023-10-16T11:25:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.824+0000 7fc045d9b640 1 -- 172.21.15.84:0/3735798949 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0300309a0 con 0x7fc04806f9d0 2023-10-16T11:25:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.824+0000 7fc045d9b640 1 -- 172.21.15.84:0/3735798949 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc030030d30 con 0x7fc04806f9d0 2023-10-16T11:25:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.824+0000 7fc04ca61640 1 -- 172.21.15.84:0/3735798949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc04806f9d0 msgr2=0x7fc0480c9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.824+0000 7fc04ca61640 1 --2- 172.21.15.84:0/3735798949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc04806f9d0 0x7fc0480c9ab0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc030009870 tx=0x7fc03002b980 comp rx=0 tx=0).stop 2023-10-16T11:25:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.824+0000 7fc04ca61640 1 -- 172.21.15.84:0/3735798949 shutdown_connections 2023-10-16T11:25:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.824+0000 7fc04ca61640 1 --2- 172.21.15.84:0/3735798949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc04806f9d0 0x7fc0480c9ab0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.824+0000 7fc04ca61640 1 --2- 172.21.15.84:0/3735798949 >> v2:172.21.15.157:3300/0 conn(0x7fc04815cf60 0x7fc04806f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.824+0000 7fc04ca61640 1 -- 172.21.15.84:0/3735798949 >> 172.21.15.84:0/3735798949 conn(0x7fc0480cb360 msgr2=0x7fc0480cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.824+0000 7fc04ca61640 1 -- 172.21.15.84:0/3735798949 shutdown_connections 2023-10-16T11:25:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.824+0000 7fc04ca61640 1 -- 172.21.15.84:0/3735798949 wait complete. 2023-10-16T11:25:23.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.826+0000 7fc04ca61640 1 Processor -- start 2023-10-16T11:25:23.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.826+0000 7fc04ca61640 1 -- start start 2023-10-16T11:25:23.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.827+0000 7fc04ca61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc04806f9d0 0x7fc0480c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:23.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.827+0000 7fc04ca61640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0480c9ff0 0x7fc0481e8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:23.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.827+0000 7fc04ca61640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc04815cf60 0x7fc04806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:23.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.827+0000 7fc04ca61640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0480c52a0 con 0x7fc04815cf60 2023-10-16T11:25:23.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.827+0000 7fc04ca61640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0481f2e80 con 0x7fc0480c9ff0 2023-10-16T11:25:23.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.827+0000 7fc04ca61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0481f3160 con 0x7fc04806f9d0 2023-10-16T11:25:23.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.828+0000 7fc046d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc04815cf60 0x7fc04806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:23.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.828+0000 7fc046d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc04815cf60 0x7fc04806afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42682/0 (socket says 172.21.15.84:42682) 2023-10-16T11:25:23.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.828+0000 7fc046d9d640 1 -- 172.21.15.84:0/2155645939 learned_addr learned my addr 172.21.15.84:0/2155645939 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:23.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.828+0000 7fc035d9b640 1 --2- 172.21.15.84:0/2155645939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0480c9ff0 0x7fc0481e8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:23.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.828+0000 7fc04659c640 1 --2- 172.21.15.84:0/2155645939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc04806f9d0 0x7fc0480c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:23.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.828+0000 7fc046d9d640 1 -- 172.21.15.84:0/2155645939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc04806f9d0 msgr2=0x7fc0480c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:23.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.828+0000 7fc046d9d640 1 --2- 172.21.15.84:0/2155645939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc04806f9d0 0x7fc0480c4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:23.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.828+0000 7fc046d9d640 1 -- 172.21.15.84:0/2155645939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0480c9ff0 msgr2=0x7fc0481e8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:23.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.828+0000 7fc046d9d640 1 --2- 172.21.15.84:0/2155645939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0480c9ff0 0x7fc0481e8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:23.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.828+0000 7fc046d9d640 1 -- 172.21.15.84:0/2155645939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0300094d0 con 0x7fc04815cf60 2023-10-16T11:25:23.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.828+0000 7fc046d9d640 1 --2- 172.21.15.84:0/2155645939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc04815cf60 0x7fc04806afc0 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7fc04000c390 tx=0x7fc04000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:23.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.829+0000 7fc03f7fe640 1 -- 172.21.15.84:0/2155645939 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0400052d0 con 0x7fc04815cf60 2023-10-16T11:25:23.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.829+0000 7fc03f7fe640 1 -- 172.21.15.84:0/2155645939 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc040005430 con 0x7fc04815cf60 2023-10-16T11:25:23.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.829+0000 7fc03f7fe640 1 -- 172.21.15.84:0/2155645939 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc040003810 con 0x7fc04815cf60 2023-10-16T11:25:23.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.829+0000 7fc04ca61640 1 -- 172.21.15.84:0/2155645939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0481f3440 con 0x7fc04815cf60 2023-10-16T11:25:23.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.829+0000 7fc04ca61640 1 -- 172.21.15.84:0/2155645939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc048155120 con 0x7fc04815cf60 2023-10-16T11:25:23.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.830+0000 7fc04ca61640 1 -- 172.21.15.84:0/2155645939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0481f3960 con 0x7fc04815cf60 2023-10-16T11:25:23.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.831+0000 7fc03f7fe640 1 -- 172.21.15.84:0/2155645939 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fc040003970 con 0x7fc04815cf60 2023-10-16T11:25:23.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.832+0000 7fc03f7fe640 1 --2- 172.21.15.84:0/2155645939 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc02c06b0b0 0x7fc02c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:23.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.832+0000 7fc03f7fe640 1 -- 172.21.15.84:0/2155645939 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 6738+0+0 (secure 0 0 0) 0x7fc040089210 con 0x7fc04815cf60 2023-10-16T11:25:23.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.832+0000 7fc04659c640 1 --2- 172.21.15.84:0/2155645939 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc02c06b0b0 0x7fc02c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:23.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.833+0000 7fc04659c640 1 --2- 172.21.15.84:0/2155645939 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc02c06b0b0 0x7fc02c06d570 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc0380073f0 tx=0x7fc03800b040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:23.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:23.837+0000 7fc03f7fe640 1 -- 172.21.15.84:0/2155645939 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc0481f3960 con 0x7fc04815cf60 2023-10-16T11:25:24.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:24.028+0000 7fc04ca61640 1 -- 172.21.15.84:0/2155645939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc0481f3960 con 0x7fc04815cf60 2023-10-16T11:25:25.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.024+0000 7fc03f7fe640 1 -- 172.21.15.84:0/2155645939 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v228) v1 ==== 190+0+0 (secure 0 0 0) 0x7fc0481f3960 con 0x7fc04815cf60 2023-10-16T11:25:25.027 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'backup_fs_metadata' removed 2023-10-16T11:25:25.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.028+0000 7fc04ca61640 1 -- 172.21.15.84:0/2155645939 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc02c06b0b0 msgr2=0x7fc02c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:25.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.028+0000 7fc04ca61640 1 --2- 172.21.15.84:0/2155645939 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc02c06b0b0 0x7fc02c06d570 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc0380073f0 tx=0x7fc03800b040 comp rx=0 tx=0).stop 2023-10-16T11:25:25.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.028+0000 7fc04ca61640 1 -- 172.21.15.84:0/2155645939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc04815cf60 msgr2=0x7fc04806afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:25.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.028+0000 7fc04ca61640 1 --2- 172.21.15.84:0/2155645939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc04815cf60 0x7fc04806afc0 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7fc04000c390 tx=0x7fc04000c850 comp rx=0 tx=0).stop 2023-10-16T11:25:25.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.031+0000 7fc04ca61640 1 -- 172.21.15.84:0/2155645939 shutdown_connections 2023-10-16T11:25:25.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.031+0000 7fc04ca61640 1 --2- 172.21.15.84:0/2155645939 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc02c06b0b0 0x7fc02c06d570 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:25.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.031+0000 7fc04ca61640 1 --2- 172.21.15.84:0/2155645939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc04815cf60 0x7fc04806afc0 unknown :-1 s=CLOSED pgs=2867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:25.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.031+0000 7fc04ca61640 1 --2- 172.21.15.84:0/2155645939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0480c9ff0 0x7fc0481e8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:25.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.031+0000 7fc04ca61640 1 --2- 172.21.15.84:0/2155645939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc04806f9d0 0x7fc0480c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:25.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.031+0000 7fc04ca61640 1 -- 172.21.15.84:0/2155645939 >> 172.21.15.84:0/2155645939 conn(0x7fc0480cb360 msgr2=0x7fc04806c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:25.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.031+0000 7fc04ca61640 1 -- 172.21.15.84:0/2155645939 shutdown_connections 2023-10-16T11:25:25.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.032+0000 7fc04ca61640 1 -- 172.21.15.84:0/2155645939 wait complete. 2023-10-16T11:25:25.061 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2023-10-16T11:25:25.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.265+0000 7f08165a9640 1 Processor -- start 2023-10-16T11:25:25.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.265+0000 7f08165a9640 1 -- start start 2023-10-16T11:25:25.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.266+0000 7f08165a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f081015cf60 0x7f081006f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:25.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.266+0000 7f08165a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f081006f9d0 0x7f08100c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:25.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.266+0000 7f08165a9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08100c9ff0 0x7f08100b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:25.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.266+0000 7f08165a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f081015d340 con 0x7f081006f9d0 2023-10-16T11:25:25.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.266+0000 7f08165a9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08100cc860 con 0x7f08100c9ff0 2023-10-16T11:25:25.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.266+0000 7f08165a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08100c9620 con 0x7f081015cf60 2023-10-16T11:25:25.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.267+0000 7f080f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f081006f9d0 0x7f08100c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:25.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.267+0000 7f080f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f081006f9d0 0x7f08100c9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42696/0 (socket says 172.21.15.84:42696) 2023-10-16T11:25:25.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.267+0000 7f080f7fe640 1 -- 172.21.15.84:0/784109211 learned_addr learned my addr 172.21.15.84:0/784109211 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:25.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.267+0000 7f080ffff640 1 --2- 172.21.15.84:0/784109211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f081015cf60 0x7f081006f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:25.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.267+0000 7f0814b46640 1 --2- 172.21.15.84:0/784109211 >> v2:172.21.15.157:3300/0 conn(0x7f08100c9ff0 0x7f08100b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:25.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.267+0000 7f080f7fe640 1 -- 172.21.15.84:0/784109211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f081015cf60 msgr2=0x7f081006f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:25.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.267+0000 7f080f7fe640 1 --2- 172.21.15.84:0/784109211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f081015cf60 0x7f081006f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:25.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.267+0000 7f080f7fe640 1 -- 172.21.15.84:0/784109211 >> v2:172.21.15.157:3300/0 conn(0x7f08100c9ff0 msgr2=0x7f08100b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:25.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.267+0000 7f080f7fe640 1 --2- 172.21.15.84:0/784109211 >> v2:172.21.15.157:3300/0 conn(0x7f08100c9ff0 0x7f08100b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:25.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.267+0000 7f080f7fe640 1 -- 172.21.15.84:0/784109211 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08100ad4a0 con 0x7f081006f9d0 2023-10-16T11:25:25.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.269+0000 7f080f7fe640 1 --2- 172.21.15.84:0/784109211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f081006f9d0 0x7f08100c9ab0 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7f08100ce890 tx=0x7f07f802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d03da7b7beb0bac server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:25.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.269+0000 7f080effd640 1 -- 172.21.15.84:0/784109211 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8031870 con 0x7f081006f9d0 2023-10-16T11:25:25.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.269+0000 7f080effd640 1 -- 172.21.15.84:0/784109211 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f80319d0 con 0x7f081006f9d0 2023-10-16T11:25:25.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.269+0000 7f080effd640 1 -- 172.21.15.84:0/784109211 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8031d60 con 0x7f081006f9d0 2023-10-16T11:25:25.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.270+0000 7f08165a9640 1 -- 172.21.15.84:0/784109211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f081006f9d0 msgr2=0x7f08100c9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:25.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.270+0000 7f08165a9640 1 --2- 172.21.15.84:0/784109211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f081006f9d0 0x7f08100c9ab0 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7f08100ce890 tx=0x7f07f802b910 comp rx=0 tx=0).stop 2023-10-16T11:25:25.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.270+0000 7f08165a9640 1 -- 172.21.15.84:0/784109211 shutdown_connections 2023-10-16T11:25:25.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.270+0000 7f08165a9640 1 --2- 172.21.15.84:0/784109211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f081006f9d0 0x7f08100c9ab0 unknown :-1 s=CLOSED pgs=2868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:25.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.270+0000 7f08165a9640 1 --2- 172.21.15.84:0/784109211 >> v2:172.21.15.157:3300/0 conn(0x7f08100c9ff0 0x7f08100b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:25.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.270+0000 7f08165a9640 1 --2- 172.21.15.84:0/784109211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f081015cf60 0x7f081006f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:25.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.270+0000 7f08165a9640 1 -- 172.21.15.84:0/784109211 >> 172.21.15.84:0/784109211 conn(0x7f08100cb360 msgr2=0x7f08100cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:25.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.270+0000 7f08165a9640 1 -- 172.21.15.84:0/784109211 shutdown_connections 2023-10-16T11:25:25.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.270+0000 7f08165a9640 1 -- 172.21.15.84:0/784109211 wait complete. 2023-10-16T11:25:25.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.272+0000 7f08165a9640 1 Processor -- start 2023-10-16T11:25:25.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.272+0000 7f08165a9640 1 -- start start 2023-10-16T11:25:25.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.273+0000 7f08165a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081006f9d0 0x7f08100c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:25.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.273+0000 7f08165a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08100c9ff0 0x7f0810157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:25.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.273+0000 7f08165a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f081015cf60 0x7f081015ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:25.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.273+0000 7f08165a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081015fe80 con 0x7f08100c9ff0 2023-10-16T11:25:25.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.273+0000 7f08165a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08100c8350 con 0x7f081006f9d0 2023-10-16T11:25:25.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.273+0000 7f08165a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081015ef60 con 0x7f081015cf60 2023-10-16T11:25:25.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.273+0000 7f080f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08100c9ff0 0x7f0810157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:25.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.273+0000 7f080f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08100c9ff0 0x7f0810157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42704/0 (socket says 172.21.15.84:42704) 2023-10-16T11:25:25.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.273+0000 7f080f7fe640 1 -- 172.21.15.84:0/2557393406 learned_addr learned my addr 172.21.15.84:0/2557393406 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:25.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.273+0000 7f080ffff640 1 --2- 172.21.15.84:0/2557393406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081006f9d0 0x7f08100c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:25.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.273+0000 7f0814b46640 1 --2- 172.21.15.84:0/2557393406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f081015cf60 0x7f081015ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:25.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.273+0000 7f080f7fe640 1 -- 172.21.15.84:0/2557393406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f081015cf60 msgr2=0x7f081015ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:25.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.273+0000 7f080f7fe640 1 --2- 172.21.15.84:0/2557393406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f081015cf60 0x7f081015ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:25.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.273+0000 7f080f7fe640 1 -- 172.21.15.84:0/2557393406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081006f9d0 msgr2=0x7f08100c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:25.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.273+0000 7f080f7fe640 1 --2- 172.21.15.84:0/2557393406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081006f9d0 0x7f08100c6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:25.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.273+0000 7f080f7fe640 1 -- 172.21.15.84:0/2557393406 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f80094d0 con 0x7f08100c9ff0 2023-10-16T11:25:25.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.274+0000 7f080f7fe640 1 --2- 172.21.15.84:0/2557393406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08100c9ff0 0x7f0810157f10 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7f07f8000c00 tx=0x7f07f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:25.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.274+0000 7f080cff9640 1 -- 172.21.15.84:0/2557393406 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f803abe0 con 0x7f08100c9ff0 2023-10-16T11:25:25.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.274+0000 7f080cff9640 1 -- 172.21.15.84:0/2557393406 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f803ad40 con 0x7f08100c9ff0 2023-10-16T11:25:25.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.274+0000 7f080cff9640 1 -- 172.21.15.84:0/2557393406 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8044610 con 0x7f08100c9ff0 2023-10-16T11:25:25.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.274+0000 7f08165a9640 1 -- 172.21.15.84:0/2557393406 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081015f200 con 0x7f08100c9ff0 2023-10-16T11:25:25.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.274+0000 7f08165a9640 1 -- 172.21.15.84:0/2557393406 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081015f7c0 con 0x7f08100c9ff0 2023-10-16T11:25:25.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.275+0000 7f08165a9640 1 -- 172.21.15.84:0/2557393406 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081015ffc0 con 0x7f08100c9ff0 2023-10-16T11:25:25.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.276+0000 7f080cff9640 1 -- 172.21.15.84:0/2557393406 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f07f804d450 con 0x7f08100c9ff0 2023-10-16T11:25:25.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.277+0000 7f080cff9640 1 --2- 172.21.15.84:0/2557393406 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f07f006b0b0 0x7f07f006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:25.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.277+0000 7f080cff9640 1 -- 172.21.15.84:0/2557393406 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f07f802d040 con 0x7f08100c9ff0 2023-10-16T11:25:25.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.277+0000 7f080ffff640 1 --2- 172.21.15.84:0/2557393406 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f07f006b0b0 0x7f07f006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:25.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.278+0000 7f080ffff640 1 --2- 172.21.15.84:0/2557393406 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f07f006b0b0 0x7f07f006d570 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0800009fd0 tx=0x7f080000b040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:25.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.282+0000 7f080cff9640 1 -- 172.21.15.84:0/2557393406 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f081015ffc0 con 0x7f08100c9ff0 2023-10-16T11:25:25.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:25.474+0000 7f08165a9640 1 -- 172.21.15.84:0/2557393406 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f081015ffc0 con 0x7f08100c9ff0 2023-10-16T11:25:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.047+0000 7f080cff9640 1 -- 172.21.15.84:0/2557393406 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v229) v1 ==== 178+0+0 (secure 0 0 0) 0x7f081015ffc0 con 0x7f08100c9ff0 2023-10-16T11:25:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'backup_fs_data' removed 2023-10-16T11:25:26.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.051+0000 7f08165a9640 1 -- 172.21.15.84:0/2557393406 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f07f006b0b0 msgr2=0x7f07f006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:26.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.052+0000 7f08165a9640 1 --2- 172.21.15.84:0/2557393406 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f07f006b0b0 0x7f07f006d570 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0800009fd0 tx=0x7f080000b040 comp rx=0 tx=0).stop 2023-10-16T11:25:26.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.052+0000 7f08165a9640 1 -- 172.21.15.84:0/2557393406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08100c9ff0 msgr2=0x7f0810157f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:26.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.052+0000 7f08165a9640 1 --2- 172.21.15.84:0/2557393406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08100c9ff0 0x7f0810157f10 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7f07f8000c00 tx=0x7f07f80046c0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.052+0000 7f08165a9640 1 -- 172.21.15.84:0/2557393406 shutdown_connections 2023-10-16T11:25:26.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.052+0000 7f08165a9640 1 --2- 172.21.15.84:0/2557393406 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f07f006b0b0 0x7f07f006d570 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.052+0000 7f08165a9640 1 --2- 172.21.15.84:0/2557393406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f081015cf60 0x7f081015ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.052+0000 7f08165a9640 1 --2- 172.21.15.84:0/2557393406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08100c9ff0 0x7f0810157f10 unknown :-1 s=CLOSED pgs=2869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.052+0000 7f08165a9640 1 --2- 172.21.15.84:0/2557393406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081006f9d0 0x7f08100c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.052+0000 7f08165a9640 1 -- 172.21.15.84:0/2557393406 >> 172.21.15.84:0/2557393406 conn(0x7f08100cb360 msgr2=0x7f081006b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:26.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.053+0000 7f08165a9640 1 -- 172.21.15.84:0/2557393406 shutdown_connections 2023-10-16T11:25:26.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.054+0000 7f08165a9640 1 -- 172.21.15.84:0/2557393406 wait complete. 2023-10-16T11:25:26.080 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T11:25:26.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.281+0000 7f8843bc0640 1 Processor -- start 2023-10-16T11:25:26.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.281+0000 7f8843bc0640 1 -- start start 2023-10-16T11:25:26.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.282+0000 7f8843bc0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f883c06b660 0x7f883c06ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:26.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.282+0000 7f8843bc0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c0c9860 0x7f883c0ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:26.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.282+0000 7f8843bc0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f883c0cc6b0 con 0x7f883c1428c0 2023-10-16T11:25:26.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.283+0000 7f8843bc0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f883c1599b0 con 0x7f883c06b660 2023-10-16T11:25:26.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.283+0000 7f8843bc0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883c06c290 con 0x7f883c0c9860 2023-10-16T11:25:26.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.283+0000 7f884215d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c0c9860 0x7f883c0ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:26.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.283+0000 7f884195c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f883c06b660 0x7f883c06ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:26.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.283+0000 7f884215d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c0c9860 0x7f883c0ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:34182/0 (socket says 172.21.15.84:34182) 2023-10-16T11:25:26.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.283+0000 7f884215d640 1 -- 172.21.15.84:0/3894585923 learned_addr learned my addr 172.21.15.84:0/3894585923 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:26.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.284+0000 7f884215d640 1 -- 172.21.15.84:0/3894585923 >> v2:172.21.15.157:3300/0 conn(0x7f883c06b660 msgr2=0x7f883c06ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:26.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.284+0000 7f884215d640 1 --2- 172.21.15.84:0/3894585923 >> v2:172.21.15.157:3300/0 conn(0x7f883c06b660 0x7f883c06ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.284+0000 7f884215d640 1 -- 172.21.15.84:0/3894585923 >> v1:172.21.15.157:6789/0 conn(0x7f883c1428c0 legacy=0x7f883c0c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:26.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.284+0000 7f884215d640 1 -- 172.21.15.84:0/3894585923 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c06c570 con 0x7f883c0c9860 2023-10-16T11:25:26.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.284+0000 7f884195c640 1 --2- 172.21.15.84:0/3894585923 >> v2:172.21.15.157:3300/0 conn(0x7f883c06b660 0x7f883c06ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:25:26.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.285+0000 7f884215d640 1 --2- 172.21.15.84:0/3894585923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c0c9860 0x7f883c0ccb40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8834009e50 tx=0x7f883402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeeb56cffd774f55 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:26.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.286+0000 7f884095a640 1 -- 172.21.15.84:0/3894585923 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883402cb00 con 0x7f883c0c9860 2023-10-16T11:25:26.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.286+0000 7f884095a640 1 -- 172.21.15.84:0/3894585923 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883402cc60 con 0x7f883c0c9860 2023-10-16T11:25:26.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.286+0000 7f884095a640 1 -- 172.21.15.84:0/3894585923 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88340343f0 con 0x7f883c0c9860 2023-10-16T11:25:26.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.287+0000 7f8843bc0640 1 -- 172.21.15.84:0/3894585923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c0c9860 msgr2=0x7f883c0ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:26.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.287+0000 7f8843bc0640 1 --2- 172.21.15.84:0/3894585923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c0c9860 0x7f883c0ccb40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8834009e50 tx=0x7f883402bf60 comp rx=0 tx=0).stop 2023-10-16T11:25:26.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.287+0000 7f8843bc0640 1 -- 172.21.15.84:0/3894585923 shutdown_connections 2023-10-16T11:25:26.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.287+0000 7f8843bc0640 1 --2- 172.21.15.84:0/3894585923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c0c9860 0x7f883c0ccb40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.287+0000 7f8843bc0640 1 --2- 172.21.15.84:0/3894585923 >> v2:172.21.15.157:3300/0 conn(0x7f883c06b660 0x7f883c06ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.287+0000 7f8843bc0640 1 -- 172.21.15.84:0/3894585923 >> 172.21.15.84:0/3894585923 conn(0x7f883c0ce9b0 msgr2=0x7f883c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:26.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.287+0000 7f8843bc0640 1 -- 172.21.15.84:0/3894585923 shutdown_connections 2023-10-16T11:25:26.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.287+0000 7f8843bc0640 1 -- 172.21.15.84:0/3894585923 wait complete. 2023-10-16T11:25:26.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.289+0000 7f8843bc0640 1 Processor -- start 2023-10-16T11:25:26.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.289+0000 7f8843bc0640 1 -- start start 2023-10-16T11:25:26.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.290+0000 7f8843bc0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f883c06b660 0x7f883c150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:26.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.290+0000 7f8843bc0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c0c9860 0x7f883c1580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:26.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.290+0000 7f8843bc0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883c1428c0 0x7f883c1585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:26.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.290+0000 7f8843bc0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c151d90 con 0x7f883c06b660 2023-10-16T11:25:26.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.290+0000 7f8843bc0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c1511a0 con 0x7f883c1428c0 2023-10-16T11:25:26.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.290+0000 7f8843bc0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883c1513f0 con 0x7f883c0c9860 2023-10-16T11:25:26.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.290+0000 7f884195c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f883c06b660 0x7f883c150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:26.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.290+0000 7f884195c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f883c06b660 0x7f883c150a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55560/0 (socket says 172.21.15.84:55560) 2023-10-16T11:25:26.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.290+0000 7f884195c640 1 -- 172.21.15.84:0/305446440 learned_addr learned my addr 172.21.15.84:0/305446440 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:26.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.290+0000 7f884115b640 1 --2- 172.21.15.84:0/305446440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c0c9860 0x7f883c1580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:26.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.291+0000 7f884215d640 1 --2- 172.21.15.84:0/305446440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883c1428c0 0x7f883c1585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:26.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.291+0000 7f884195c640 1 -- 172.21.15.84:0/305446440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c0c9860 msgr2=0x7f883c1580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:26.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.291+0000 7f884195c640 1 --2- 172.21.15.84:0/305446440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c0c9860 0x7f883c1580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.291+0000 7f884195c640 1 -- 172.21.15.84:0/305446440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883c1428c0 msgr2=0x7f883c1585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:26.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.291+0000 7f884195c640 1 --2- 172.21.15.84:0/305446440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883c1428c0 0x7f883c1585e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.291+0000 7f884195c640 1 -- 172.21.15.84:0/305446440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8838009400 con 0x7f883c06b660 2023-10-16T11:25:26.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.291+0000 7f884115b640 1 --2- 172.21.15.84:0/305446440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c0c9860 0x7f883c1580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:26.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.291+0000 7f884195c640 1 --2- 172.21.15.84:0/305446440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f883c06b660 0x7f883c150a00 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f883800e960 tx=0x7f883800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:26.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.291+0000 7f88327fc640 1 -- 172.21.15.84:0/305446440 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883800ca00 con 0x7f883c06b660 2023-10-16T11:25:26.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.291+0000 7f88327fc640 1 -- 172.21.15.84:0/305446440 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883800cb60 con 0x7f883c06b660 2023-10-16T11:25:26.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.291+0000 7f88327fc640 1 -- 172.21.15.84:0/305446440 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8838013470 con 0x7f883c06b660 2023-10-16T11:25:26.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.291+0000 7f8843bc0640 1 -- 172.21.15.84:0/305446440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8834009b30 con 0x7f883c06b660 2023-10-16T11:25:26.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.291+0000 7f8843bc0640 1 -- 172.21.15.84:0/305446440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883c1519b0 con 0x7f883c06b660 2023-10-16T11:25:26.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.293+0000 7f8843bc0640 1 -- 172.21.15.84:0/305446440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f883c152000 con 0x7f883c06b660 2023-10-16T11:25:26.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.294+0000 7f88327fc640 1 -- 172.21.15.84:0/305446440 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f88380135d0 con 0x7f883c06b660 2023-10-16T11:25:26.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.295+0000 7f88327fc640 1 --2- 172.21.15.84:0/305446440 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f882006b0e0 0x7f882006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:26.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.295+0000 7f88327fc640 1 -- 172.21.15.84:0/305446440 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f8838089d90 con 0x7f883c06b660 2023-10-16T11:25:26.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.295+0000 7f884115b640 1 --2- 172.21.15.84:0/305446440 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f882006b0e0 0x7f882006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:26.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.295+0000 7f884115b640 1 --2- 172.21.15.84:0/305446440 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f882006b0e0 0x7f882006d5a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f883c158d80 tx=0x7f882c009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:26.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.299+0000 7f88327fc640 1 -- 172.21.15.84:0/305446440 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8838054990 con 0x7f883c06b660 2023-10-16T11:25:26.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.491+0000 7f8843bc0640 1 -- 172.21.15.84:0/305446440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f883c04e160 con 0x7f883c06b660 2023-10-16T11:25:26.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.497+0000 7f88327fc640 1 -- 172.21.15.84:0/305446440 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v79) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8838057fb0 con 0x7f883c06b660 2023-10-16T11:25:26.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.500+0000 7f8843bc0640 1 -- 172.21.15.84:0/305446440 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f882006b0e0 msgr2=0x7f882006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:26.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.500+0000 7f8843bc0640 1 --2- 172.21.15.84:0/305446440 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f882006b0e0 0x7f882006d5a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f883c158d80 tx=0x7f882c009040 comp rx=0 tx=0).stop 2023-10-16T11:25:26.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.500+0000 7f8843bc0640 1 -- 172.21.15.84:0/305446440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f883c06b660 msgr2=0x7f883c150a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:26.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.500+0000 7f8843bc0640 1 --2- 172.21.15.84:0/305446440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f883c06b660 0x7f883c150a00 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f883800e960 tx=0x7f883800ee20 comp rx=0 tx=0).stop 2023-10-16T11:25:26.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.501+0000 7f8843bc0640 1 -- 172.21.15.84:0/305446440 shutdown_connections 2023-10-16T11:25:26.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.501+0000 7f8843bc0640 1 --2- 172.21.15.84:0/305446440 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f882006b0e0 0x7f882006d5a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.501+0000 7f8843bc0640 1 --2- 172.21.15.84:0/305446440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883c1428c0 0x7f883c1585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.501+0000 7f8843bc0640 1 --2- 172.21.15.84:0/305446440 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f883c0c9860 0x7f883c1580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.501+0000 7f8843bc0640 1 --2- 172.21.15.84:0/305446440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f883c06b660 0x7f883c150a00 unknown :-1 s=CLOSED pgs=2870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.501+0000 7f8843bc0640 1 -- 172.21.15.84:0/305446440 >> 172.21.15.84:0/305446440 conn(0x7f883c0ce9b0 msgr2=0x7f883c0c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:26.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.501+0000 7f8843bc0640 1 -- 172.21.15.84:0/305446440 shutdown_connections 2023-10-16T11:25:26.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.501+0000 7f8843bc0640 1 -- 172.21.15.84:0/305446440 wait complete. 2023-10-16T11:25:26.520 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T11:25:26.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.730+0000 7f2fb7a15640 1 Processor -- start 2023-10-16T11:25:26.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.730+0000 7f2fb7a15640 1 -- start start 2023-10-16T11:25:26.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.731+0000 7f2fb7a15640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2fb0159e90 0x7f2fb00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:26.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.731+0000 7f2fb7a15640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fb006c140 0x7f2fb006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:26.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.731+0000 7f2fb7a15640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb0070360 con 0x7f2fb0159ab0 2023-10-16T11:25:26.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.731+0000 7f2fb7a15640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb006cb10 con 0x7f2fb0159e90 2023-10-16T11:25:26.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.731+0000 7f2fb7a15640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb015d890 con 0x7f2fb006c140 2023-10-16T11:25:26.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.732+0000 7f2fb4fb0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2fb0159e90 0x7f2fb00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:26.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.732+0000 7f2fb4fb0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2fb0159e90 0x7f2fb00ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55562/0 (socket says 172.21.15.84:55562) 2023-10-16T11:25:26.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.732+0000 7f2fb4fb0640 1 -- 172.21.15.84:0/356781324 learned_addr learned my addr 172.21.15.84:0/356781324 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:26.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.732+0000 7f2fb4fb0640 1 -- 172.21.15.84:0/356781324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fb006c140 msgr2=0x7f2fb006c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:26.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.732+0000 7f2fb5fb2640 1 --2- 172.21.15.84:0/356781324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fb006c140 0x7f2fb006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.732+0000 7f2fb4fb0640 1 --2- 172.21.15.84:0/356781324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fb006c140 0x7f2fb006c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.732+0000 7f2fb4fb0640 1 -- 172.21.15.84:0/356781324 >> v1:172.21.15.157:6789/0 conn(0x7f2fb0159ab0 legacy=0x7f2fb006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.732+0000 7f2fb4fb0640 1 -- 172.21.15.84:0/356781324 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb00cd1f0 con 0x7f2fb0159e90 2023-10-16T11:25:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.732+0000 7f2fb5fb2640 1 --2- 172.21.15.84:0/356781324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fb006c140 0x7f2fb006c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:26.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.733+0000 7f2fb4fb0640 1 --2- 172.21.15.84:0/356781324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fb0159e90 0x7f2fb00ce370 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009870 tx=0x7f2fa402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff4369114875eb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:26.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.734+0000 7f2faffff640 1 -- 172.21.15.84:0/356781324 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4031740 con 0x7f2fb0159e90 2023-10-16T11:25:26.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.734+0000 7f2faffff640 1 -- 172.21.15.84:0/356781324 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa40318a0 con 0x7f2fb0159e90 2023-10-16T11:25:26.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.734+0000 7f2faffff640 1 -- 172.21.15.84:0/356781324 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4031c30 con 0x7f2fb0159e90 2023-10-16T11:25:26.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.734+0000 7f2fb7a15640 1 -- 172.21.15.84:0/356781324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fb0159e90 msgr2=0x7f2fb00ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:26.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.734+0000 7f2fb7a15640 1 --2- 172.21.15.84:0/356781324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fb0159e90 0x7f2fb00ce370 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009870 tx=0x7f2fa402b7e0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.734+0000 7f2fb7a15640 1 -- 172.21.15.84:0/356781324 shutdown_connections 2023-10-16T11:25:26.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.734+0000 7f2fb7a15640 1 --2- 172.21.15.84:0/356781324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fb006c140 0x7f2fb006c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.734+0000 7f2fb7a15640 1 --2- 172.21.15.84:0/356781324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fb0159e90 0x7f2fb00ce370 unknown :-1 s=CLOSED pgs=2871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.735+0000 7f2fb7a15640 1 -- 172.21.15.84:0/356781324 >> 172.21.15.84:0/356781324 conn(0x7f2fb00ce9b0 msgr2=0x7f2fb00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:26.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.735+0000 7f2fb7a15640 1 -- 172.21.15.84:0/356781324 shutdown_connections 2023-10-16T11:25:26.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.735+0000 7f2fb7a15640 1 -- 172.21.15.84:0/356781324 wait complete. 2023-10-16T11:25:26.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.737+0000 7f2fb7a15640 1 Processor -- start 2023-10-16T11:25:26.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.737+0000 7f2fb7a15640 1 -- start start 2023-10-16T11:25:26.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.738+0000 7f2fb7a15640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fb006c140 0x7f2fb015d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:26.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.738+0000 7f2fb7a15640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fb0159ab0 0x7f2fb00acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:26.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.738+0000 7f2fb7a15640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fb0159e90 0x7f2fb006b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:26.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.738+0000 7f2fb7a15640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00c2df0 con 0x7f2fb0159e90 2023-10-16T11:25:26.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.738+0000 7f2fb7a15640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00c3370 con 0x7f2fb0159ab0 2023-10-16T11:25:26.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.738+0000 7f2fb7a15640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00c8990 con 0x7f2fb006c140 2023-10-16T11:25:26.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.738+0000 7f2fb5fb2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fb0159e90 0x7f2fb006b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:26.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.738+0000 7f2fb5fb2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fb0159e90 0x7f2fb006b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55570/0 (socket says 172.21.15.84:55570) 2023-10-16T11:25:26.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.738+0000 7f2fb5fb2640 1 -- 172.21.15.84:0/1017354581 learned_addr learned my addr 172.21.15.84:0/1017354581 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:26.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.739+0000 7f2fb4fb0640 1 --2- 172.21.15.84:0/1017354581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fb0159ab0 0x7f2fb00acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:26.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.739+0000 7f2fb57b1640 1 --2- 172.21.15.84:0/1017354581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fb006c140 0x7f2fb015d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:26.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.739+0000 7f2fb5fb2640 1 -- 172.21.15.84:0/1017354581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fb006c140 msgr2=0x7f2fb015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:26.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.739+0000 7f2fb5fb2640 1 --2- 172.21.15.84:0/1017354581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fb006c140 0x7f2fb015d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.739+0000 7f2fb5fb2640 1 -- 172.21.15.84:0/1017354581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fb0159ab0 msgr2=0x7f2fb00acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:26.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.739+0000 7f2fb5fb2640 1 --2- 172.21.15.84:0/1017354581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fb0159ab0 0x7f2fb00acb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.739+0000 7f2fb5fb2640 1 -- 172.21.15.84:0/1017354581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa40094d0 con 0x7f2fb0159e90 2023-10-16T11:25:26.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.739+0000 7f2fb5fb2640 1 --2- 172.21.15.84:0/1017354581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fb0159e90 0x7f2fb006b030 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800c7f0 tx=0x7f2fa800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:26.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.739+0000 7f2fadffb640 1 -- 172.21.15.84:0/1017354581 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8004230 con 0x7f2fb0159e90 2023-10-16T11:25:26.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.739+0000 7f2fadffb640 1 -- 172.21.15.84:0/1017354581 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa8004390 con 0x7f2fb0159e90 2023-10-16T11:25:26.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.739+0000 7f2fadffb640 1 -- 172.21.15.84:0/1017354581 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8007860 con 0x7f2fb0159e90 2023-10-16T11:25:26.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.739+0000 7f2fb7a15640 1 -- 172.21.15.84:0/1017354581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb00c25f0 con 0x7f2fb0159e90 2023-10-16T11:25:26.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.740+0000 7f2fb7a15640 1 -- 172.21.15.84:0/1017354581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb00c3000 con 0x7f2fb0159e90 2023-10-16T11:25:26.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.741+0000 7f2fb7a15640 1 -- 172.21.15.84:0/1017354581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb0165c30 con 0x7f2fb0159e90 2023-10-16T11:25:26.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.742+0000 7f2fadffb640 1 -- 172.21.15.84:0/1017354581 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f2fa8007bd0 con 0x7f2fb0159e90 2023-10-16T11:25:26.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.743+0000 7f2fadffb640 1 --2- 172.21.15.84:0/1017354581 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f2f9c06b0b0 0x7f2f9c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:26.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.743+0000 7f2fadffb640 1 -- 172.21.15.84:0/1017354581 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f2fa808a0f0 con 0x7f2fb0159e90 2023-10-16T11:25:26.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.743+0000 7f2fb57b1640 1 --2- 172.21.15.84:0/1017354581 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f2f9c06b0b0 0x7f2f9c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:26.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.744+0000 7f2fb57b1640 1 --2- 172.21.15.84:0/1017354581 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f2f9c06b0b0 0x7f2f9c06d570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009f20 tx=0x7f2fa00094d0 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:26.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.747+0000 7f2fadffb640 1 -- 172.21.15.84:0/1017354581 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2fa8054ca0 con 0x7f2fb0159e90 2023-10-16T11:25:26.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.941+0000 7f2fb7a15640 1 -- 172.21.15.84:0/1017354581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f2fb004e160 con 0x7f2fb0159e90 2023-10-16T11:25:26.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.949+0000 7f2fadffb640 1 -- 172.21.15.84:0/1017354581 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v80) v1 ==== 120+0+0 (secure 0 0 0) 0x7f2fa80582c0 con 0x7f2fb0159e90 2023-10-16T11:25:26.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.952+0000 7f2fb7a15640 1 -- 172.21.15.84:0/1017354581 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f2f9c06b0b0 msgr2=0x7f2f9c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:26.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.952+0000 7f2fb7a15640 1 --2- 172.21.15.84:0/1017354581 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f2f9c06b0b0 0x7f2f9c06d570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009f20 tx=0x7f2fa00094d0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.952+0000 7f2fb7a15640 1 -- 172.21.15.84:0/1017354581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fb0159e90 msgr2=0x7f2fb006b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:26.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.952+0000 7f2fb7a15640 1 --2- 172.21.15.84:0/1017354581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fb0159e90 0x7f2fb006b030 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800c7f0 tx=0x7f2fa800ccb0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.952+0000 7f2fb7a15640 1 -- 172.21.15.84:0/1017354581 shutdown_connections 2023-10-16T11:25:26.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.952+0000 7f2fb7a15640 1 --2- 172.21.15.84:0/1017354581 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f2f9c06b0b0 0x7f2f9c06d570 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.952+0000 7f2fb7a15640 1 --2- 172.21.15.84:0/1017354581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fb0159e90 0x7f2fb006b030 unknown :-1 s=CLOSED pgs=2872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.952+0000 7f2fb7a15640 1 --2- 172.21.15.84:0/1017354581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fb0159ab0 0x7f2fb00acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.952+0000 7f2fb7a15640 1 --2- 172.21.15.84:0/1017354581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fb006c140 0x7f2fb015d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:26.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.952+0000 7f2fb7a15640 1 -- 172.21.15.84:0/1017354581 >> 172.21.15.84:0/1017354581 conn(0x7f2fb00ce9b0 msgr2=0x7f2fb00c94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:26.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.953+0000 7f2fb7a15640 1 -- 172.21.15.84:0/1017354581 shutdown_connections 2023-10-16T11:25:26.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:26.953+0000 7f2fb7a15640 1 -- 172.21.15.84:0/1017354581 wait complete. 2023-10-16T11:25:26.974 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T11:25:27.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.181+0000 7f9b3736c640 1 Processor -- start 2023-10-16T11:25:27.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.182+0000 7f9b3736c640 1 -- start start 2023-10-16T11:25:27.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.183+0000 7f9b3736c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9b3015aaa0 0x7f9b3015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:27.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.183+0000 7f9b3736c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9b300ce390 0x7f9b30070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:27.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.183+0000 7f9b3736c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b300707e0 con 0x7f9b3015ae80 2023-10-16T11:25:27.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.183+0000 7f9b3736c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b3006b7a0 con 0x7f9b300ce390 2023-10-16T11:25:27.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.183+0000 7f9b3736c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b3015ef20 con 0x7f9b3015aaa0 2023-10-16T11:25:27.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.183+0000 7f9b35909640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9b300ce390 0x7f9b30070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:27.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.183+0000 7f9b35909640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9b300ce390 0x7f9b30070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55572/0 (socket says 172.21.15.84:55572) 2023-10-16T11:25:27.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.184+0000 7f9b35909640 1 -- 172.21.15.84:0/1324969638 learned_addr learned my addr 172.21.15.84:0/1324969638 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:27.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.184+0000 7f9b35108640 1 --2- 172.21.15.84:0/1324969638 >> v2:172.21.15.157:3300/0 conn(0x7f9b3015aaa0 0x7f9b3015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:27.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.184+0000 7f9b35909640 1 -- 172.21.15.84:0/1324969638 >> v2:172.21.15.157:3300/0 conn(0x7f9b3015aaa0 msgr2=0x7f9b3015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:27.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.184+0000 7f9b35909640 1 --2- 172.21.15.84:0/1324969638 >> v2:172.21.15.157:3300/0 conn(0x7f9b3015aaa0 0x7f9b3015e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:27.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.184+0000 7f9b35909640 1 -- 172.21.15.84:0/1324969638 >> v1:172.21.15.157:6789/0 conn(0x7f9b3015ae80 legacy=0x7f9b300cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:27.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.184+0000 7f9b35909640 1 -- 172.21.15.84:0/1324969638 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3015f200 con 0x7f9b300ce390 2023-10-16T11:25:27.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.184+0000 7f9b35108640 1 --2- 172.21.15.84:0/1324969638 >> v2:172.21.15.157:3300/0 conn(0x7f9b3015aaa0 0x7f9b3015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:27.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.185+0000 7f9b35909640 1 --2- 172.21.15.84:0/1324969638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b300ce390 0x7f9b30070f50 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7f9b300cd990 tx=0x7f9b2802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd52da1f5f9d98d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:27.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.185+0000 7f9b27fff640 1 -- 172.21.15.84:0/1324969638 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2802c9f0 con 0x7f9b300ce390 2023-10-16T11:25:27.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.185+0000 7f9b27fff640 1 -- 172.21.15.84:0/1324969638 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2802cb50 con 0x7f9b300ce390 2023-10-16T11:25:27.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.185+0000 7f9b27fff640 1 -- 172.21.15.84:0/1324969638 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b280343f0 con 0x7f9b300ce390 2023-10-16T11:25:27.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.186+0000 7f9b3736c640 1 -- 172.21.15.84:0/1324969638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b300ce390 msgr2=0x7f9b30070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:27.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.186+0000 7f9b3736c640 1 --2- 172.21.15.84:0/1324969638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b300ce390 0x7f9b30070f50 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7f9b300cd990 tx=0x7f9b2802be50 comp rx=0 tx=0).stop 2023-10-16T11:25:27.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.186+0000 7f9b3736c640 1 -- 172.21.15.84:0/1324969638 shutdown_connections 2023-10-16T11:25:27.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.186+0000 7f9b3736c640 1 --2- 172.21.15.84:0/1324969638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b300ce390 0x7f9b30070f50 unknown :-1 s=CLOSED pgs=2873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:27.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.186+0000 7f9b3736c640 1 --2- 172.21.15.84:0/1324969638 >> v2:172.21.15.157:3300/0 conn(0x7f9b3015aaa0 0x7f9b3015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:27.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.186+0000 7f9b3736c640 1 -- 172.21.15.84:0/1324969638 >> 172.21.15.84:0/1324969638 conn(0x7f9b300cb410 msgr2=0x7f9b300cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:27.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.186+0000 7f9b3736c640 1 -- 172.21.15.84:0/1324969638 shutdown_connections 2023-10-16T11:25:27.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.186+0000 7f9b3736c640 1 -- 172.21.15.84:0/1324969638 wait complete. 2023-10-16T11:25:27.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.188+0000 7f9b3736c640 1 Processor -- start 2023-10-16T11:25:27.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.188+0000 7f9b3736c640 1 -- start start 2023-10-16T11:25:27.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.189+0000 7f9b3736c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b300ce390 0x7f9b30153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:27.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.189+0000 7f9b3736c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b3015aaa0 0x7f9b300c25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:27.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.189+0000 7f9b3736c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b3015ae80 0x7f9b300c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:27.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.189+0000 7f9b3736c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b30154430 con 0x7f9b3015aaa0 2023-10-16T11:25:27.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.189+0000 7f9b3736c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3015c210 con 0x7f9b300ce390 2023-10-16T11:25:27.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.189+0000 7f9b3736c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3015b320 con 0x7f9b3015ae80 2023-10-16T11:25:27.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.189+0000 7f9b34907640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b3015aaa0 0x7f9b300c25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:27.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.189+0000 7f9b34907640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b3015aaa0 0x7f9b300c25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55578/0 (socket says 172.21.15.84:55578) 2023-10-16T11:25:27.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.189+0000 7f9b34907640 1 -- 172.21.15.84:0/852029936 learned_addr learned my addr 172.21.15.84:0/852029936 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:27.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.190+0000 7f9b35108640 1 --2- 172.21.15.84:0/852029936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b300ce390 0x7f9b30153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:27.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.190+0000 7f9b35909640 1 --2- 172.21.15.84:0/852029936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b3015ae80 0x7f9b300c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:27.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.190+0000 7f9b34907640 1 -- 172.21.15.84:0/852029936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b3015ae80 msgr2=0x7f9b300c2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:27.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.190+0000 7f9b34907640 1 --2- 172.21.15.84:0/852029936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b3015ae80 0x7f9b300c2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:27.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.190+0000 7f9b34907640 1 -- 172.21.15.84:0/852029936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b300ce390 msgr2=0x7f9b30153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:27.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.190+0000 7f9b34907640 1 --2- 172.21.15.84:0/852029936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b300ce390 0x7f9b30153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:27.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.190+0000 7f9b34907640 1 -- 172.21.15.84:0/852029936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b28009b30 con 0x7f9b3015aaa0 2023-10-16T11:25:27.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.190+0000 7f9b34907640 1 --2- 172.21.15.84:0/852029936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b3015aaa0 0x7f9b300c25e0 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000c400 tx=0x7f9b2000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:27.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.190+0000 7f9b25ffb640 1 -- 172.21.15.84:0/852029936 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20011790 con 0x7f9b3015aaa0 2023-10-16T11:25:27.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.190+0000 7f9b25ffb640 1 -- 172.21.15.84:0/852029936 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b20011930 con 0x7f9b3015aaa0 2023-10-16T11:25:27.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.190+0000 7f9b3736c640 1 -- 172.21.15.84:0/852029936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3015b600 con 0x7f9b3015aaa0 2023-10-16T11:25:27.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.190+0000 7f9b25ffb640 1 -- 172.21.15.84:0/852029936 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20011ce0 con 0x7f9b3015aaa0 2023-10-16T11:25:27.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.190+0000 7f9b3736c640 1 -- 172.21.15.84:0/852029936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3015bbc0 con 0x7f9b3015aaa0 2023-10-16T11:25:27.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.191+0000 7f9b3736c640 1 -- 172.21.15.84:0/852029936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b3004e160 con 0x7f9b3015aaa0 2023-10-16T11:25:27.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.192+0000 7f9b25ffb640 1 -- 172.21.15.84:0/852029936 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f9b2001c460 con 0x7f9b3015aaa0 2023-10-16T11:25:27.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.193+0000 7f9b25ffb640 1 --2- 172.21.15.84:0/852029936 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f9b1406b0e0 0x7f9b1406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:27.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.193+0000 7f9b25ffb640 1 -- 172.21.15.84:0/852029936 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f9b20089980 con 0x7f9b3015aaa0 2023-10-16T11:25:27.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.194+0000 7f9b35108640 1 --2- 172.21.15.84:0/852029936 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f9b1406b0e0 0x7f9b1406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:27.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.194+0000 7f9b35108640 1 --2- 172.21.15.84:0/852029936 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f9b1406b0e0 0x7f9b1406d5a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c008850 tx=0x7f9b2c008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:27.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.198+0000 7f9b25ffb640 1 -- 172.21.15.84:0/852029936 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9b20054650 con 0x7f9b3015aaa0 2023-10-16T11:25:27.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.398+0000 7f9b3736c640 1 -- 172.21.15.84:0/852029936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f9b300cdc60 con 0x7f9b3015aaa0 2023-10-16T11:25:27.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.405+0000 7f9b25ffb640 1 -- 172.21.15.84:0/852029936 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v81) v1 ==== 115+0+0 (secure 0 0 0) 0x7f9b20057c70 con 0x7f9b3015aaa0 2023-10-16T11:25:27.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.408+0000 7f9b3736c640 1 -- 172.21.15.84:0/852029936 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f9b1406b0e0 msgr2=0x7f9b1406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:27.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.408+0000 7f9b3736c640 1 --2- 172.21.15.84:0/852029936 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f9b1406b0e0 0x7f9b1406d5a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c008850 tx=0x7f9b2c008040 comp rx=0 tx=0).stop 2023-10-16T11:25:27.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.408+0000 7f9b3736c640 1 -- 172.21.15.84:0/852029936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b3015aaa0 msgr2=0x7f9b300c25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:27.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.408+0000 7f9b3736c640 1 --2- 172.21.15.84:0/852029936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b3015aaa0 0x7f9b300c25e0 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000c400 tx=0x7f9b2000c8c0 comp rx=0 tx=0).stop 2023-10-16T11:25:27.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.408+0000 7f9b3736c640 1 -- 172.21.15.84:0/852029936 shutdown_connections 2023-10-16T11:25:27.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.408+0000 7f9b3736c640 1 --2- 172.21.15.84:0/852029936 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f9b1406b0e0 0x7f9b1406d5a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:27.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.408+0000 7f9b3736c640 1 --2- 172.21.15.84:0/852029936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b3015ae80 0x7f9b300c2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:27.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.408+0000 7f9b3736c640 1 --2- 172.21.15.84:0/852029936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b3015aaa0 0x7f9b300c25e0 unknown :-1 s=CLOSED pgs=2874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:27.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.408+0000 7f9b3736c640 1 --2- 172.21.15.84:0/852029936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b300ce390 0x7f9b30153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:27.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.408+0000 7f9b3736c640 1 -- 172.21.15.84:0/852029936 >> 172.21.15.84:0/852029936 conn(0x7f9b300cb410 msgr2=0x7f9b3006f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:27.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.408+0000 7f9b3736c640 1 -- 172.21.15.84:0/852029936 shutdown_connections 2023-10-16T11:25:27.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.408+0000 7f9b3736c640 1 -- 172.21.15.84:0/852029936 wait complete. 2023-10-16T11:25:27.429 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs' 2023-10-16T11:25:27.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.640+0000 7fabaacbb640 1 Processor -- start 2023-10-16T11:25:27.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.640+0000 7fabaacbb640 1 -- start start 2023-10-16T11:25:27.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.641+0000 7fabaacbb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faba40cfdd0 0x7faba40b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:27.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.641+0000 7fabaacbb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faba40b09c0 0x7faba415d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:27.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.641+0000 7fabaacbb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba40cf940 con 0x7faba406b8c0 2023-10-16T11:25:27.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.641+0000 7fabaacbb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba40ce990 con 0x7faba40cfdd0 2023-10-16T11:25:27.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.641+0000 7fabaacbb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba415e290 con 0x7faba40b09c0 2023-10-16T11:25:27.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.641+0000 7faba3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faba40cfdd0 0x7faba40b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:27.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.642+0000 7faba3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faba40cfdd0 0x7faba40b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55594/0 (socket says 172.21.15.84:55594) 2023-10-16T11:25:27.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.642+0000 7faba3fff640 1 -- 172.21.15.84:0/1009996422 learned_addr learned my addr 172.21.15.84:0/1009996422 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:27.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.642+0000 7faba9258640 1 --2- 172.21.15.84:0/1009996422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faba40b09c0 0x7faba415d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:27.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.642+0000 7faba3fff640 1 -- 172.21.15.84:0/1009996422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faba40b09c0 msgr2=0x7faba415d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:27.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.642+0000 7faba3fff640 1 --2- 172.21.15.84:0/1009996422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faba40b09c0 0x7faba415d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:27.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.642+0000 7faba3fff640 1 -- 172.21.15.84:0/1009996422 >> v1:172.21.15.157:6789/0 conn(0x7faba406b8c0 legacy=0x7faba406bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:27.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.642+0000 7faba3fff640 1 -- 172.21.15.84:0/1009996422 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba40cf010 con 0x7faba40cfdd0 2023-10-16T11:25:27.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.643+0000 7faba3fff640 1 --2- 172.21.15.84:0/1009996422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faba40cfdd0 0x7faba40b0480 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7fab94009e50 tx=0x7fab9402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98bea4b9b59592 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:27.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.643+0000 7faba37fe640 1 -- 172.21.15.84:0/1009996422 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9402ca70 con 0x7faba40cfdd0 2023-10-16T11:25:27.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.643+0000 7faba37fe640 1 -- 172.21.15.84:0/1009996422 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9402cbd0 con 0x7faba40cfdd0 2023-10-16T11:25:27.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.643+0000 7faba37fe640 1 -- 172.21.15.84:0/1009996422 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab940343f0 con 0x7faba40cfdd0 2023-10-16T11:25:27.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.644+0000 7fabaacbb640 1 -- 172.21.15.84:0/1009996422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faba40cfdd0 msgr2=0x7faba40b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:27.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.644+0000 7fabaacbb640 1 --2- 172.21.15.84:0/1009996422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faba40cfdd0 0x7faba40b0480 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7fab94009e50 tx=0x7fab9402bed0 comp rx=0 tx=0).stop 2023-10-16T11:25:27.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.644+0000 7fabaacbb640 1 -- 172.21.15.84:0/1009996422 shutdown_connections 2023-10-16T11:25:27.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.644+0000 7fabaacbb640 1 --2- 172.21.15.84:0/1009996422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faba40b09c0 0x7faba415d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:27.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.644+0000 7fabaacbb640 1 --2- 172.21.15.84:0/1009996422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faba40cfdd0 0x7faba40b0480 unknown :-1 s=CLOSED pgs=2875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:27.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.644+0000 7fabaacbb640 1 -- 172.21.15.84:0/1009996422 >> 172.21.15.84:0/1009996422 conn(0x7faba4099ea0 msgr2=0x7faba40d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:27.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.644+0000 7fabaacbb640 1 -- 172.21.15.84:0/1009996422 shutdown_connections 2023-10-16T11:25:27.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.644+0000 7fabaacbb640 1 -- 172.21.15.84:0/1009996422 wait complete. 2023-10-16T11:25:27.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.646+0000 7fabaacbb640 1 Processor -- start 2023-10-16T11:25:27.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.646+0000 7fabaacbb640 1 -- start start 2023-10-16T11:25:27.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.647+0000 7fabaacbb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba406b8c0 0x7faba40cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:27.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.647+0000 7fabaacbb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faba40b09c0 0x7faba40cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:27.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.647+0000 7fabaacbb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faba40cfdd0 0x7faba406ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:27.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.647+0000 7fabaacbb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba415f020 con 0x7faba40b09c0 2023-10-16T11:25:27.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.647+0000 7fabaacbb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba40cb670 con 0x7faba406b8c0 2023-10-16T11:25:27.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.647+0000 7fabaacbb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba415e6f0 con 0x7faba40cfdd0 2023-10-16T11:25:27.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.647+0000 7faba3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faba40b09c0 0x7faba40cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:27.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.647+0000 7faba3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faba40b09c0 0x7faba40cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55602/0 (socket says 172.21.15.84:55602) 2023-10-16T11:25:27.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.647+0000 7faba3fff640 1 -- 172.21.15.84:0/26517599 learned_addr learned my addr 172.21.15.84:0/26517599 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:27.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.648+0000 7faba8a57640 1 --2- 172.21.15.84:0/26517599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba406b8c0 0x7faba40cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:27.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.648+0000 7faba9258640 1 --2- 172.21.15.84:0/26517599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faba40cfdd0 0x7faba406ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:27.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.648+0000 7faba3fff640 1 -- 172.21.15.84:0/26517599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faba40cfdd0 msgr2=0x7faba406ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:27.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.648+0000 7faba3fff640 1 --2- 172.21.15.84:0/26517599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faba40cfdd0 0x7faba406ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:27.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.648+0000 7faba3fff640 1 -- 172.21.15.84:0/26517599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba406b8c0 msgr2=0x7faba40cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:27.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.648+0000 7faba3fff640 1 --2- 172.21.15.84:0/26517599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba406b8c0 0x7faba40cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:27.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.648+0000 7faba3fff640 1 -- 172.21.15.84:0/26517599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab94009b30 con 0x7faba40b09c0 2023-10-16T11:25:27.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.648+0000 7faba8a57640 1 --2- 172.21.15.84:0/26517599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba406b8c0 0x7faba40cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:27.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.648+0000 7faba3fff640 1 --2- 172.21.15.84:0/26517599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faba40b09c0 0x7faba40cb130 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7fab94036040 tx=0x7fab94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:27.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.648+0000 7faba17fa640 1 -- 172.21.15.84:0/26517599 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab94040670 con 0x7faba40b09c0 2023-10-16T11:25:27.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.648+0000 7faba17fa640 1 -- 172.21.15.84:0/26517599 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab94040810 con 0x7faba40b09c0 2023-10-16T11:25:27.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.648+0000 7faba17fa640 1 -- 172.21.15.84:0/26517599 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab94040bc0 con 0x7faba40b09c0 2023-10-16T11:25:27.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.649+0000 7fabaacbb640 1 -- 172.21.15.84:0/26517599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba415e9d0 con 0x7faba40b09c0 2023-10-16T11:25:27.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.649+0000 7fabaacbb640 1 -- 172.21.15.84:0/26517599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba41f2aa0 con 0x7faba40b09c0 2023-10-16T11:25:27.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.650+0000 7fabaacbb640 1 -- 172.21.15.84:0/26517599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba415f1d0 con 0x7faba40b09c0 2023-10-16T11:25:27.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.651+0000 7faba17fa640 1 -- 172.21.15.84:0/26517599 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fab94040d20 con 0x7faba40b09c0 2023-10-16T11:25:27.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.651+0000 7faba17fa640 1 --2- 172.21.15.84:0/26517599 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fab8406b0b0 0x7fab8406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:27.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.652+0000 7faba17fa640 1 -- 172.21.15.84:0/26517599 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5874+0+0 (secure 0 0 0) 0x7fab940acda0 con 0x7faba40b09c0 2023-10-16T11:25:27.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.652+0000 7faba8a57640 1 --2- 172.21.15.84:0/26517599 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fab8406b0b0 0x7fab8406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:27.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.653+0000 7faba8a57640 1 --2- 172.21.15.84:0/26517599 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fab8406b0b0 0x7fab8406d570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00a830 tx=0x7fab8c008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:27.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.656+0000 7faba17fa640 1 -- 172.21.15.84:0/26517599 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7faba415f1d0 con 0x7faba40b09c0 2023-10-16T11:25:27.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:27.924+0000 7fabaacbb640 1 -- 172.21.15.84:0/26517599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]} v 0) v1 -- 0x7faba404e160 con 0x7faba40b09c0 2023-10-16T11:25:28.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.061+0000 7faba17fa640 1 -- 172.21.15.84:0/26517599 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]}]=0 v3174) v1 ==== 186+0+0 (secure 0 0 0) 0x7fab94077ac0 con 0x7faba40b09c0 2023-10-16T11:25:28.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.064+0000 7fabaacbb640 1 -- 172.21.15.84:0/26517599 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fab8406b0b0 msgr2=0x7fab8406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:28.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.064+0000 7fabaacbb640 1 --2- 172.21.15.84:0/26517599 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fab8406b0b0 0x7fab8406d570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00a830 tx=0x7fab8c008040 comp rx=0 tx=0).stop 2023-10-16T11:25:28.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.064+0000 7fabaacbb640 1 -- 172.21.15.84:0/26517599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faba40b09c0 msgr2=0x7faba40cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:28.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.064+0000 7fabaacbb640 1 --2- 172.21.15.84:0/26517599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faba40b09c0 0x7faba40cb130 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7fab94036040 tx=0x7fab94004a40 comp rx=0 tx=0).stop 2023-10-16T11:25:28.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.064+0000 7fabaacbb640 1 -- 172.21.15.84:0/26517599 shutdown_connections 2023-10-16T11:25:28.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.064+0000 7fabaacbb640 1 --2- 172.21.15.84:0/26517599 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fab8406b0b0 0x7fab8406d570 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.064+0000 7fabaacbb640 1 --2- 172.21.15.84:0/26517599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faba40cfdd0 0x7faba406ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.064+0000 7fabaacbb640 1 --2- 172.21.15.84:0/26517599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faba40b09c0 0x7faba40cb130 unknown :-1 s=CLOSED pgs=2876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.064+0000 7fabaacbb640 1 --2- 172.21.15.84:0/26517599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba406b8c0 0x7faba40cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.064+0000 7fabaacbb640 1 -- 172.21.15.84:0/26517599 >> 172.21.15.84:0/26517599 conn(0x7faba4099ea0 msgr2=0x7faba415fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:28.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.064+0000 7fabaacbb640 1 -- 172.21.15.84:0/26517599 shutdown_connections 2023-10-16T11:25:28.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.064+0000 7fabaacbb640 1 -- 172.21.15.84:0/26517599 wait complete. 2023-10-16T11:25:28.087 INFO:tasks.cephfs_test_runner:Test command succes with --fs argument for non-default fs ... ok 2023-10-16T11:25:28.087 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_without_fs_argument (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) Test command fails without --fs argument in presence of multiple fs 2023-10-16T11:25:28.088 INFO:tasks.cephfs_test_runner:test_snap_schedule_without_fs_argument (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) 2023-10-16T11:25:28.089 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:25:28.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.303+0000 7fa829fb7640 1 Processor -- start 2023-10-16T11:25:28.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.303+0000 7fa829fb7640 1 -- start start 2023-10-16T11:25:28.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.305+0000 7fa829fb7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa82415aaa0 0x7fa82415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:28.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.305+0000 7fa829fb7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa82415ae80 0x7fa8240cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:28.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.305+0000 7fa829fb7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa82415ed60 con 0x7fa8240ccaa0 2023-10-16T11:25:28.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.305+0000 7fa829fb7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82406b810 con 0x7fa82415aaa0 2023-10-16T11:25:28.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.305+0000 7fa829fb7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8240707e0 con 0x7fa82415ae80 2023-10-16T11:25:28.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.306+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa82415ae80 0x7fa8240cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:28.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.306+0000 7fa8237fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa82415aaa0 0x7fa82415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:28.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.306+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa82415ae80 0x7fa8240cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:34246/0 (socket says 172.21.15.84:34246) 2023-10-16T11:25:28.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.306+0000 7fa81bfff640 1 -- 172.21.15.84:0/2981642385 learned_addr learned my addr 172.21.15.84:0/2981642385 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:28.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.306+0000 7fa823fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa8240ccaa0 0x7fa824070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:54492/0 (socket says 172.21.15.84:54492) 2023-10-16T11:25:28.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.307+0000 7fa822ffd640 1 -- 172.21.15.84:0/2981642385 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995652356 0 0) 0x7fa82415ed60 con 0x7fa8240ccaa0 2023-10-16T11:25:28.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.307+0000 7fa822ffd640 1 -- 172.21.15.84:0/2981642385 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa810003520 con 0x7fa8240ccaa0 2023-10-16T11:25:28.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.307+0000 7fa81bfff640 1 -- 172.21.15.84:0/2981642385 >> v2:172.21.15.157:3300/0 conn(0x7fa82415aaa0 msgr2=0x7fa82415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:28.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.307+0000 7fa81bfff640 1 --2- 172.21.15.84:0/2981642385 >> v2:172.21.15.157:3300/0 conn(0x7fa82415aaa0 0x7fa82415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.307+0000 7fa81bfff640 1 -- 172.21.15.84:0/2981642385 >> v1:172.21.15.157:6789/0 conn(0x7fa8240ccaa0 legacy=0x7fa824070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:28.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.307+0000 7fa81bfff640 1 -- 172.21.15.84:0/2981642385 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82415ef90 con 0x7fa82415ae80 2023-10-16T11:25:28.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.307+0000 7fa8237fe640 1 --2- 172.21.15.84:0/2981642385 >> v2:172.21.15.157:3300/0 conn(0x7fa82415aaa0 0x7fa82415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:25:28.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.308+0000 7fa81bfff640 1 --2- 172.21.15.84:0/2981642385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa82415ae80 0x7fa8240cc560 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009870 tx=0x7fa80c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bea2b5105db29901 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:28.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.309+0000 7fa822ffd640 1 -- 172.21.15.84:0/2981642385 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c030840 con 0x7fa82415ae80 2023-10-16T11:25:28.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.309+0000 7fa822ffd640 1 -- 172.21.15.84:0/2981642385 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80c0309a0 con 0x7fa82415ae80 2023-10-16T11:25:28.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.309+0000 7fa822ffd640 1 -- 172.21.15.84:0/2981642385 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c030d10 con 0x7fa82415ae80 2023-10-16T11:25:28.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.309+0000 7fa829fb7640 1 -- 172.21.15.84:0/2981642385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa82415ae80 msgr2=0x7fa8240cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:28.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.309+0000 7fa829fb7640 1 --2- 172.21.15.84:0/2981642385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa82415ae80 0x7fa8240cc560 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009870 tx=0x7fa80c02b980 comp rx=0 tx=0).stop 2023-10-16T11:25:28.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.309+0000 7fa829fb7640 1 -- 172.21.15.84:0/2981642385 shutdown_connections 2023-10-16T11:25:28.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.309+0000 7fa829fb7640 1 --2- 172.21.15.84:0/2981642385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa82415ae80 0x7fa8240cc560 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.309+0000 7fa829fb7640 1 --2- 172.21.15.84:0/2981642385 >> v2:172.21.15.157:3300/0 conn(0x7fa82415aaa0 0x7fa82415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.309+0000 7fa829fb7640 1 -- 172.21.15.84:0/2981642385 >> 172.21.15.84:0/2981642385 conn(0x7fa8240cb410 msgr2=0x7fa8240cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:28.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.310+0000 7fa829fb7640 1 -- 172.21.15.84:0/2981642385 shutdown_connections 2023-10-16T11:25:28.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.310+0000 7fa829fb7640 1 -- 172.21.15.84:0/2981642385 wait complete. 2023-10-16T11:25:28.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.312+0000 7fa829fb7640 1 Processor -- start 2023-10-16T11:25:28.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.312+0000 7fa829fb7640 1 -- start start 2023-10-16T11:25:28.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.313+0000 7fa829fb7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8240ccaa0 0x7fa8240c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:28.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.313+0000 7fa829fb7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa82415aaa0 0x7fa824157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:28.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.313+0000 7fa829fb7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa82415ae80 0x7fa8240b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:28.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.313+0000 7fa829fb7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8241f02a0 con 0x7fa82415aaa0 2023-10-16T11:25:28.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.313+0000 7fa829fb7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8241583c0 con 0x7fa8240ccaa0 2023-10-16T11:25:28.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.313+0000 7fa829fb7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8240c3750 con 0x7fa82415ae80 2023-10-16T11:25:28.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.313+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa82415aaa0 0x7fa824157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:28.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.313+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa82415aaa0 0x7fa824157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55610/0 (socket says 172.21.15.84:55610) 2023-10-16T11:25:28.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.313+0000 7fa81bfff640 1 -- 172.21.15.84:0/3696842304 learned_addr learned my addr 172.21.15.84:0/3696842304 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:28.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.314+0000 7fa8237fe640 1 --2- 172.21.15.84:0/3696842304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8240ccaa0 0x7fa8240c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:28.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.314+0000 7fa823fff640 1 --2- 172.21.15.84:0/3696842304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa82415ae80 0x7fa8240b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:28.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.314+0000 7fa81bfff640 1 -- 172.21.15.84:0/3696842304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa82415ae80 msgr2=0x7fa8240b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:28.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.314+0000 7fa81bfff640 1 --2- 172.21.15.84:0/3696842304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa82415ae80 0x7fa8240b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.314+0000 7fa81bfff640 1 -- 172.21.15.84:0/3696842304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8240ccaa0 msgr2=0x7fa8240c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:28.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.314+0000 7fa81bfff640 1 --2- 172.21.15.84:0/3696842304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8240ccaa0 0x7fa8240c44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.314+0000 7fa81bfff640 1 -- 172.21.15.84:0/3696842304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa814009400 con 0x7fa82415aaa0 2023-10-16T11:25:28.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.314+0000 7fa81bfff640 1 --2- 172.21.15.84:0/3696842304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa82415aaa0 0x7fa824157840 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7fa80c02b9c0 tx=0x7fa80c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:28.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.315+0000 7fa820ff9640 1 -- 172.21.15.84:0/3696842304 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c03f070 con 0x7fa82415aaa0 2023-10-16T11:25:28.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.315+0000 7fa820ff9640 1 -- 172.21.15.84:0/3696842304 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80c003610 con 0x7fa82415aaa0 2023-10-16T11:25:28.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.315+0000 7fa829fb7640 1 -- 172.21.15.84:0/3696842304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80c0094d0 con 0x7fa82415aaa0 2023-10-16T11:25:28.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.315+0000 7fa820ff9640 1 -- 172.21.15.84:0/3696842304 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c003980 con 0x7fa82415aaa0 2023-10-16T11:25:28.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.315+0000 7fa829fb7640 1 -- 172.21.15.84:0/3696842304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8240c5140 con 0x7fa82415aaa0 2023-10-16T11:25:28.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.316+0000 7fa829fb7640 1 -- 172.21.15.84:0/3696842304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8240c38e0 con 0x7fa82415aaa0 2023-10-16T11:25:28.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.317+0000 7fa820ff9640 1 -- 172.21.15.84:0/3696842304 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fa80c003ae0 con 0x7fa82415aaa0 2023-10-16T11:25:28.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.318+0000 7fa820ff9640 1 --2- 172.21.15.84:0/3696842304 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fa80406b0b0 0x7fa80406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:28.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.318+0000 7fa820ff9640 1 -- 172.21.15.84:0/3696842304 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5874+0+0 (secure 0 0 0) 0x7fa80c0abb90 con 0x7fa82415aaa0 2023-10-16T11:25:28.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.318+0000 7fa8237fe640 1 --2- 172.21.15.84:0/3696842304 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fa80406b0b0 0x7fa80406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:28.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.319+0000 7fa8237fe640 1 --2- 172.21.15.84:0/3696842304 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fa80406b0b0 0x7fa80406d570 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa814009c60 tx=0x7fa814008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:28.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.322+0000 7fa820ff9640 1 -- 172.21.15.84:0/3696842304 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa80c0767a0 con 0x7fa82415aaa0 2023-10-16T11:25:28.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.520+0000 7fa829fb7640 1 -- 172.21.15.84:0/3696842304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa82404e160 con 0x7fa82415aaa0 2023-10-16T11:25:28.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.522+0000 7fa820ff9640 1 -- 172.21.15.84:0/3696842304 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v229) v1 ==== 74+0+14333 (secure 0 0 0) 0x7fa80c034090 con 0x7fa82415aaa0 2023-10-16T11:25:28.525 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:28.525 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":229,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:25:26.037889+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/3992377882":"2023-10-17T11:25:22.570274+0000","172.21.15.157:6833/3992377882":"2023-10-17T11:25:22.570274+0000","172.21.15.84:6834/551895595":"2023-10-17T11:25:14.673046+0000","172.21.15.84:0/1287173368":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/595669474":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/4107174082":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2906390983":"2023-10-17T11:25:09.210125+0000","172.21.15.84:6833/2880641466":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/3419402221":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2161898622":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/455217819":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/4247809558":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/1814319372":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/2877946366":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6833/1425409017":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6836/551895595":"2023-10-17T11:25:14.673046+0000","172.21.15.84:6832/1425409017":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/1771162859":"2023-10-17T11:24:58.196616+0000","172.21.15.84:6832/2880641466":"2023-10-17T11:25:09.210125+0000","172.21.15.84:0/3655599055":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/1076506559":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/2748224910":"2023-10-17T11:24:58.196616+0000","172.21.15.84:0/857172551":"2023-10-17T11:24:58.196616+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:25:28.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.524+0000 7fa829fb7640 1 -- 172.21.15.84:0/3696842304 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fa80406b0b0 msgr2=0x7fa80406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:28.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.524+0000 7fa829fb7640 1 --2- 172.21.15.84:0/3696842304 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fa80406b0b0 0x7fa80406d570 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa814009c60 tx=0x7fa814008040 comp rx=0 tx=0).stop 2023-10-16T11:25:28.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.524+0000 7fa829fb7640 1 -- 172.21.15.84:0/3696842304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa82415aaa0 msgr2=0x7fa824157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:28.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.524+0000 7fa829fb7640 1 --2- 172.21.15.84:0/3696842304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa82415aaa0 0x7fa824157840 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7fa80c02b9c0 tx=0x7fa80c0046c0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.525+0000 7fa829fb7640 1 -- 172.21.15.84:0/3696842304 shutdown_connections 2023-10-16T11:25:28.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.525+0000 7fa829fb7640 1 --2- 172.21.15.84:0/3696842304 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fa80406b0b0 0x7fa80406d570 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.525+0000 7fa829fb7640 1 --2- 172.21.15.84:0/3696842304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa82415ae80 0x7fa8240b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.525+0000 7fa829fb7640 1 --2- 172.21.15.84:0/3696842304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa82415aaa0 0x7fa824157840 unknown :-1 s=CLOSED pgs=2877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.525+0000 7fa829fb7640 1 --2- 172.21.15.84:0/3696842304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8240ccaa0 0x7fa8240c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.525+0000 7fa829fb7640 1 -- 172.21.15.84:0/3696842304 >> 172.21.15.84:0/3696842304 conn(0x7fa8240cb410 msgr2=0x7fa8240b37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:28.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.525+0000 7fa829fb7640 1 -- 172.21.15.84:0/3696842304 shutdown_connections 2023-10-16T11:25:28.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.525+0000 7fa829fb7640 1 -- 172.21.15.84:0/3696842304 wait complete. 2023-10-16T11:25:28.546 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:25:28.547 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:25:28.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.763+0000 7f18c1747640 1 Processor -- start 2023-10-16T11:25:28.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.764+0000 7f18c1747640 1 -- start start 2023-10-16T11:25:28.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.765+0000 7f18c1747640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18bc15ad70 0x7f18bc069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:28.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.766+0000 7f18c1747640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18bc0caac0 0x7f18bc06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:28.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.766+0000 7f18c1747640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18bc0cb030 con 0x7f18bc0c9ca0 2023-10-16T11:25:28.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.766+0000 7f18c1747640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18bc0c91c0 con 0x7f18bc15ad70 2023-10-16T11:25:28.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.766+0000 7f18c1747640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc0c94a0 con 0x7f18bc0caac0 2023-10-16T11:25:28.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.766+0000 7f18baffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18bc15ad70 0x7f18bc069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:28.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.766+0000 7f18baffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18bc15ad70 0x7f18bc069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55624/0 (socket says 172.21.15.84:55624) 2023-10-16T11:25:28.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.766+0000 7f18baffd640 1 -- 172.21.15.84:0/3464542138 learned_addr learned my addr 172.21.15.84:0/3464542138 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:28.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.766+0000 7f18bb7fe640 1 --2- 172.21.15.84:0/3464542138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18bc0caac0 0x7f18bc06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:28.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.766+0000 7f18baffd640 1 -- 172.21.15.84:0/3464542138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18bc0caac0 msgr2=0x7f18bc06a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:28.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.766+0000 7f18baffd640 1 --2- 172.21.15.84:0/3464542138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18bc0caac0 0x7f18bc06a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.766+0000 7f18baffd640 1 -- 172.21.15.84:0/3464542138 >> v1:172.21.15.157:6789/0 conn(0x7f18bc0c9ca0 legacy=0x7f18bc0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:28.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.766+0000 7f18baffd640 1 -- 172.21.15.84:0/3464542138 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18bc0c9780 con 0x7f18bc15ad70 2023-10-16T11:25:28.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.767+0000 7f18baffd640 1 --2- 172.21.15.84:0/3464542138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc15ad70 0x7f18bc069da0 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009870 tx=0x7f18ac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c88e7daee40fc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:28.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.768+0000 7f18b9ffb640 1 -- 172.21.15.84:0/3464542138 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac031640 con 0x7f18bc15ad70 2023-10-16T11:25:28.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.768+0000 7f18b9ffb640 1 -- 172.21.15.84:0/3464542138 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ac0317a0 con 0x7f18bc15ad70 2023-10-16T11:25:28.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.768+0000 7f18b9ffb640 1 -- 172.21.15.84:0/3464542138 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac031b30 con 0x7f18bc15ad70 2023-10-16T11:25:28.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.768+0000 7f18c1747640 1 -- 172.21.15.84:0/3464542138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc15ad70 msgr2=0x7f18bc069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:28.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.768+0000 7f18c1747640 1 --2- 172.21.15.84:0/3464542138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc15ad70 0x7f18bc069da0 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009870 tx=0x7f18ac02b8f0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.769+0000 7f18c1747640 1 -- 172.21.15.84:0/3464542138 shutdown_connections 2023-10-16T11:25:28.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.769+0000 7f18c1747640 1 --2- 172.21.15.84:0/3464542138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18bc0caac0 0x7f18bc06a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.769+0000 7f18c1747640 1 --2- 172.21.15.84:0/3464542138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc15ad70 0x7f18bc069da0 unknown :-1 s=CLOSED pgs=2878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.769+0000 7f18c1747640 1 -- 172.21.15.84:0/3464542138 >> 172.21.15.84:0/3464542138 conn(0x7f18bc0ce9b0 msgr2=0x7f18bc0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:28.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.769+0000 7f18c1747640 1 -- 172.21.15.84:0/3464542138 shutdown_connections 2023-10-16T11:25:28.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.769+0000 7f18c1747640 1 -- 172.21.15.84:0/3464542138 wait complete. 2023-10-16T11:25:28.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.770+0000 7f18c1747640 1 Processor -- start 2023-10-16T11:25:28.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.771+0000 7f18c1747640 1 -- start start 2023-10-16T11:25:28.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.771+0000 7f18c1747640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18bc0c9ca0 0x7f18bc143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:28.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.771+0000 7f18c1747640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc0caac0 0x7f18bc1f0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:28.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.771+0000 7f18c1747640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18bc15ad70 0x7f18bc0b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:28.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.772+0000 7f18c1747640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc1f3ed0 con 0x7f18bc0caac0 2023-10-16T11:25:28.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.772+0000 7f18c1747640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc1f41b0 con 0x7f18bc0c9ca0 2023-10-16T11:25:28.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.772+0000 7f18c1747640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc1f4490 con 0x7f18bc15ad70 2023-10-16T11:25:28.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.772+0000 7f18ba7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc0caac0 0x7f18bc1f0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:28.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.772+0000 7f18ba7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc0caac0 0x7f18bc1f0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55630/0 (socket says 172.21.15.84:55630) 2023-10-16T11:25:28.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.772+0000 7f18ba7fc640 1 -- 172.21.15.84:0/2903557057 learned_addr learned my addr 172.21.15.84:0/2903557057 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:28.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.772+0000 7f18baffd640 1 --2- 172.21.15.84:0/2903557057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18bc0c9ca0 0x7f18bc143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:28.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.772+0000 7f18bb7fe640 1 --2- 172.21.15.84:0/2903557057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18bc15ad70 0x7f18bc0b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:28.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.772+0000 7f18ba7fc640 1 -- 172.21.15.84:0/2903557057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18bc15ad70 msgr2=0x7f18bc0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:28.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.772+0000 7f18ba7fc640 1 --2- 172.21.15.84:0/2903557057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18bc15ad70 0x7f18bc0b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.772+0000 7f18ba7fc640 1 -- 172.21.15.84:0/2903557057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18bc0c9ca0 msgr2=0x7f18bc143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:28.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.772+0000 7f18ba7fc640 1 --2- 172.21.15.84:0/2903557057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18bc0c9ca0 0x7f18bc143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.772+0000 7f18ba7fc640 1 -- 172.21.15.84:0/2903557057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ac0094d0 con 0x7f18bc0caac0 2023-10-16T11:25:28.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.772+0000 7f18baffd640 1 --2- 172.21.15.84:0/2903557057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18bc0c9ca0 0x7f18bc143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:28.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.773+0000 7f18ba7fc640 1 --2- 172.21.15.84:0/2903557057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc0caac0 0x7f18bc1f0ed0 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7f18a400c7c0 tx=0x7f18a400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:28.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.773+0000 7f18b37fe640 1 -- 172.21.15.84:0/2903557057 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a400aa50 con 0x7f18bc0caac0 2023-10-16T11:25:28.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.773+0000 7f18b37fe640 1 -- 172.21.15.84:0/2903557057 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a400abb0 con 0x7f18bc0caac0 2023-10-16T11:25:28.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.773+0000 7f18b37fe640 1 -- 172.21.15.84:0/2903557057 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4012420 con 0x7f18bc0caac0 2023-10-16T11:25:28.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.773+0000 7f18c1747640 1 -- 172.21.15.84:0/2903557057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18bc1f4770 con 0x7f18bc0caac0 2023-10-16T11:25:28.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.773+0000 7f18c1747640 1 -- 172.21.15.84:0/2903557057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18bc144270 con 0x7f18bc0caac0 2023-10-16T11:25:28.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.774+0000 7f18c1747640 1 -- 172.21.15.84:0/2903557057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18bc1f4960 con 0x7f18bc0caac0 2023-10-16T11:25:28.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.775+0000 7f18b37fe640 1 -- 172.21.15.84:0/2903557057 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f18a4012580 con 0x7f18bc0caac0 2023-10-16T11:25:28.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.776+0000 7f18b37fe640 1 --2- 172.21.15.84:0/2903557057 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f189c06b0b0 0x7f189c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:28.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.776+0000 7f18b37fe640 1 -- 172.21.15.84:0/2903557057 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f18a4089a90 con 0x7f18bc0caac0 2023-10-16T11:25:28.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.776+0000 7f18baffd640 1 --2- 172.21.15.84:0/2903557057 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f189c06b0b0 0x7f189c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:28.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.777+0000 7f18baffd640 1 --2- 172.21.15.84:0/2903557057 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f189c06b0b0 0x7f189c06d570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f18ac036000 tx=0x7f18ac03c040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:28.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.781+0000 7f18b37fe640 1 -- 172.21.15.84:0/2903557057 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f18bc1f4960 con 0x7f18bc0caac0 2023-10-16T11:25:28.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.975+0000 7f18c1747640 1 -- 172.21.15.84:0/2903557057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f18bc156070 con 0x7f18bc0caac0 2023-10-16T11:25:28.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.976+0000 7f18b37fe640 1 -- 172.21.15.84:0/2903557057 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v229) v1 ==== 93+0+10 (secure 0 0 0) 0x7f18bc156070 con 0x7f18bc0caac0 2023-10-16T11:25:28.978 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:25:28.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.978+0000 7f18c1747640 1 -- 172.21.15.84:0/2903557057 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f189c06b0b0 msgr2=0x7f189c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:28.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.978+0000 7f18c1747640 1 --2- 172.21.15.84:0/2903557057 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f189c06b0b0 0x7f189c06d570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f18ac036000 tx=0x7f18ac03c040 comp rx=0 tx=0).stop 2023-10-16T11:25:28.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.979+0000 7f18c1747640 1 -- 172.21.15.84:0/2903557057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc0caac0 msgr2=0x7f18bc1f0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:28.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.979+0000 7f18c1747640 1 --2- 172.21.15.84:0/2903557057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc0caac0 0x7f18bc1f0ed0 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7f18a400c7c0 tx=0x7f18a400cc80 comp rx=0 tx=0).stop 2023-10-16T11:25:28.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.979+0000 7f18c1747640 1 -- 172.21.15.84:0/2903557057 shutdown_connections 2023-10-16T11:25:28.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.979+0000 7f18c1747640 1 --2- 172.21.15.84:0/2903557057 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f189c06b0b0 0x7f189c06d570 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.979+0000 7f18c1747640 1 --2- 172.21.15.84:0/2903557057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18bc15ad70 0x7f18bc0b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.979+0000 7f18c1747640 1 --2- 172.21.15.84:0/2903557057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18bc0caac0 0x7f18bc1f0ed0 unknown :-1 s=CLOSED pgs=2879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.979+0000 7f18c1747640 1 --2- 172.21.15.84:0/2903557057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18bc0c9ca0 0x7f18bc143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:28.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.979+0000 7f18c1747640 1 -- 172.21.15.84:0/2903557057 >> 172.21.15.84:0/2903557057 conn(0x7f18bc0ce9b0 msgr2=0x7f18bc0afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:28.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.979+0000 7f18c1747640 1 -- 172.21.15.84:0/2903557057 shutdown_connections 2023-10-16T11:25:28.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:28.979+0000 7f18c1747640 1 -- 172.21.15.84:0/2903557057 wait complete. 2023-10-16T11:25:29.000 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument' 2023-10-16T11:25:29.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.217+0000 7fddc759e640 1 Processor -- start 2023-10-16T11:25:29.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.217+0000 7fddc759e640 1 -- start start 2023-10-16T11:25:29.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.218+0000 7fddc759e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc806c040 0x7fddc806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:29.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.218+0000 7fddc759e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fddc80ad970 0x7fddc815e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:29.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.218+0000 7fddc759e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fddc815ebc0 0x7fddc815f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:29.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.218+0000 7fddc759e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc806c960 con 0x7fddc815ebc0 2023-10-16T11:25:29.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.218+0000 7fddc759e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc815b9d0 con 0x7fddc80ad970 2023-10-16T11:25:29.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.218+0000 7fddc759e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc815bcb0 con 0x7fddc806c040 2023-10-16T11:25:29.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.218+0000 7fddc6d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fddc815ebc0 0x7fddc815f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:29.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.219+0000 7fddc6d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fddc815ebc0 0x7fddc815f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55644/0 (socket says 172.21.15.84:55644) 2023-10-16T11:25:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.219+0000 7fddc6d9d640 1 -- 172.21.15.84:0/1024487641 learned_addr learned my addr 172.21.15.84:0/1024487641 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.219+0000 7fddc6d9d640 1 -- 172.21.15.84:0/1024487641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc806c040 msgr2=0x7fddc806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.219+0000 7fddc6d9d640 1 --2- 172.21.15.84:0/1024487641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc806c040 0x7fddc806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.219+0000 7fddc6d9d640 1 -- 172.21.15.84:0/1024487641 >> v2:172.21.15.157:3300/0 conn(0x7fddc80ad970 msgr2=0x7fddc815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.219+0000 7fddc6d9d640 1 --2- 172.21.15.84:0/1024487641 >> v2:172.21.15.157:3300/0 conn(0x7fddc80ad970 0x7fddc815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:29.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.219+0000 7fddc6d9d640 1 -- 172.21.15.84:0/1024487641 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc815bf90 con 0x7fddc815ebc0 2023-10-16T11:25:29.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.220+0000 7fddc6d9d640 1 --2- 172.21.15.84:0/1024487641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc815ebc0 0x7fddc815f050 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009870 tx=0x7fddc002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f265a4b514ae0618 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:29.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.220+0000 7fddc559a640 1 -- 172.21.15.84:0/1024487641 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc0031640 con 0x7fddc815ebc0 2023-10-16T11:25:29.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.220+0000 7fddc559a640 1 -- 172.21.15.84:0/1024487641 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc00317a0 con 0x7fddc815ebc0 2023-10-16T11:25:29.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.220+0000 7fddc559a640 1 -- 172.21.15.84:0/1024487641 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc0031b30 con 0x7fddc815ebc0 2023-10-16T11:25:29.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.221+0000 7fddc759e640 1 -- 172.21.15.84:0/1024487641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc815ebc0 msgr2=0x7fddc815f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:29.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.221+0000 7fddc759e640 1 --2- 172.21.15.84:0/1024487641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc815ebc0 0x7fddc815f050 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009870 tx=0x7fddc002b8f0 comp rx=0 tx=0).stop 2023-10-16T11:25:29.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.221+0000 7fddc759e640 1 -- 172.21.15.84:0/1024487641 shutdown_connections 2023-10-16T11:25:29.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.221+0000 7fddc759e640 1 --2- 172.21.15.84:0/1024487641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc815ebc0 0x7fddc815f050 unknown :-1 s=CLOSED pgs=2880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:29.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.221+0000 7fddc759e640 1 --2- 172.21.15.84:0/1024487641 >> v2:172.21.15.157:3300/0 conn(0x7fddc80ad970 0x7fddc815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:29.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.221+0000 7fddc759e640 1 --2- 172.21.15.84:0/1024487641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc806c040 0x7fddc806c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:29.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.221+0000 7fddc759e640 1 -- 172.21.15.84:0/1024487641 >> 172.21.15.84:0/1024487641 conn(0x7fddc80ce6a0 msgr2=0x7fddc80ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:29.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.221+0000 7fddc759e640 1 -- 172.21.15.84:0/1024487641 shutdown_connections 2023-10-16T11:25:29.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.221+0000 7fddc759e640 1 -- 172.21.15.84:0/1024487641 wait complete. 2023-10-16T11:25:29.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.223+0000 7fddc759e640 1 Processor -- start 2023-10-16T11:25:29.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.223+0000 7fddc759e640 1 -- start start 2023-10-16T11:25:29.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.224+0000 7fddc759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc806c040 0x7fddc8158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:29.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.224+0000 7fddc759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddc80ad970 0x7fddc81f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:29.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.224+0000 7fddc759e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc815ebc0 0x7fddc80aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:29.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.224+0000 7fddc759e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc81f5bd0 con 0x7fddc806c040 2023-10-16T11:25:29.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.224+0000 7fddc759e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc81f2d30 con 0x7fddc80ad970 2023-10-16T11:25:29.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.224+0000 7fddc759e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc80af850 con 0x7fddc815ebc0 2023-10-16T11:25:29.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.224+0000 7fddc659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc806c040 0x7fddc8158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:29.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.224+0000 7fddc659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc806c040 0x7fddc8158790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55646/0 (socket says 172.21.15.84:55646) 2023-10-16T11:25:29.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.224+0000 7fddc659c640 1 -- 172.21.15.84:0/132879355 learned_addr learned my addr 172.21.15.84:0/132879355 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:29.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.225+0000 7fddc659c640 1 -- 172.21.15.84:0/132879355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc815ebc0 msgr2=0x7fddc80aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:29.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.225+0000 7fddc659c640 1 --2- 172.21.15.84:0/132879355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc815ebc0 0x7fddc80aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:29.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.225+0000 7fddc659c640 1 -- 172.21.15.84:0/132879355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddc80ad970 msgr2=0x7fddc81f27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:29.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.225+0000 7fddc659c640 1 --2- 172.21.15.84:0/132879355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddc80ad970 0x7fddc81f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:29.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.225+0000 7fddc659c640 1 -- 172.21.15.84:0/132879355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc00094d0 con 0x7fddc806c040 2023-10-16T11:25:29.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.225+0000 7fddc659c640 1 --2- 172.21.15.84:0/132879355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc806c040 0x7fddc8158790 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7fddb800ba50 tx=0x7fddb800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:29.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.225+0000 7fddbeffd640 1 -- 172.21.15.84:0/132879355 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb800cbe0 con 0x7fddc806c040 2023-10-16T11:25:29.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.225+0000 7fddbeffd640 1 -- 172.21.15.84:0/132879355 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb800cd60 con 0x7fddc806c040 2023-10-16T11:25:29.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.225+0000 7fddbeffd640 1 -- 172.21.15.84:0/132879355 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8014680 con 0x7fddc806c040 2023-10-16T11:25:29.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.225+0000 7fddc759e640 1 -- 172.21.15.84:0/132879355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc80c6da0 con 0x7fddc806c040 2023-10-16T11:25:29.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.225+0000 7fddc759e640 1 -- 172.21.15.84:0/132879355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc80c7360 con 0x7fddc806c040 2023-10-16T11:25:29.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.226+0000 7fddc759e640 1 -- 172.21.15.84:0/132879355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc806b330 con 0x7fddc806c040 2023-10-16T11:25:29.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.228+0000 7fddbeffd640 1 -- 172.21.15.84:0/132879355 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fddb801d3b0 con 0x7fddc806c040 2023-10-16T11:25:29.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.228+0000 7fddbeffd640 1 --2- 172.21.15.84:0/132879355 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fdda806b0b0 0x7fdda806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:29.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.229+0000 7fddbeffd640 1 -- 172.21.15.84:0/132879355 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5874+0+0 (secure 0 0 0) 0x7fddb8089a80 con 0x7fddc806c040 2023-10-16T11:25:29.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.229+0000 7fddc5d9b640 1 --2- 172.21.15.84:0/132879355 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fdda806b0b0 0x7fdda806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:29.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.229+0000 7fddc5d9b640 1 --2- 172.21.15.84:0/132879355 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fdda806b0b0 0x7fdda806d570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fddc815b6f0 tx=0x7fddb0006bf0 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:29.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.233+0000 7fddbeffd640 1 -- 172.21.15.84:0/132879355 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fddb8054720 con 0x7fddc806c040 2023-10-16T11:25:29.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:29.492+0000 7fddc759e640 1 -- 172.21.15.84:0/132879355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]} v 0) v1 -- 0x7fddc80c6c00 con 0x7fddc806c040 2023-10-16T11:25:30.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.073+0000 7fddbeffd640 1 -- 172.21.15.84:0/132879355 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]}]=0 v3176) v1 ==== 190+0+0 (secure 0 0 0) 0x7fddc80c6c00 con 0x7fddc806c040 2023-10-16T11:25:30.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.077+0000 7fddc759e640 1 -- 172.21.15.84:0/132879355 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fdda806b0b0 msgr2=0x7fdda806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:30.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.077+0000 7fddc759e640 1 --2- 172.21.15.84:0/132879355 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fdda806b0b0 0x7fdda806d570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fddc815b6f0 tx=0x7fddb0006bf0 comp rx=0 tx=0).stop 2023-10-16T11:25:30.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.077+0000 7fddc759e640 1 -- 172.21.15.84:0/132879355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc806c040 msgr2=0x7fddc8158790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:30.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.077+0000 7fddc759e640 1 --2- 172.21.15.84:0/132879355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc806c040 0x7fddc8158790 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7fddb800ba50 tx=0x7fddb800bf10 comp rx=0 tx=0).stop 2023-10-16T11:25:30.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.077+0000 7fddc759e640 1 -- 172.21.15.84:0/132879355 shutdown_connections 2023-10-16T11:25:30.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.077+0000 7fddc759e640 1 --2- 172.21.15.84:0/132879355 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fdda806b0b0 0x7fdda806d570 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:30.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.077+0000 7fddc759e640 1 --2- 172.21.15.84:0/132879355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddc815ebc0 0x7fddc80aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:30.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.077+0000 7fddc759e640 1 --2- 172.21.15.84:0/132879355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddc80ad970 0x7fddc81f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:30.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.077+0000 7fddc759e640 1 --2- 172.21.15.84:0/132879355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc806c040 0x7fddc8158790 unknown :-1 s=CLOSED pgs=2881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:30.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.077+0000 7fddc759e640 1 -- 172.21.15.84:0/132879355 >> 172.21.15.84:0/132879355 conn(0x7fddc80ce6a0 msgr2=0x7fddc80ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:30.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.077+0000 7fddc759e640 1 -- 172.21.15.84:0/132879355 shutdown_connections 2023-10-16T11:25:30.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.077+0000 7fddc759e640 1 -- 172.21.15.84:0/132879355 wait complete. 2023-10-16T11:25:30.100 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T11:25:30.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.311+0000 7f697a116640 1 Processor -- start 2023-10-16T11:25:30.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.312+0000 7f697a116640 1 -- start start 2023-10-16T11:25:30.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.313+0000 7f697a116640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69740caac0 0x7f69740ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:30.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.313+0000 7f697a116640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69740cb030 con 0x7f697406a4b0 2023-10-16T11:25:30.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.313+0000 7f697a116640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69740c91c0 con 0x7f697415ad70 2023-10-16T11:25:30.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.313+0000 7f697a116640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69740c94a0 con 0x7f69740caac0 2023-10-16T11:25:30.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.313+0000 7f6972ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f697406a4b0 0x7f69740ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56012/0 (socket says 172.21.15.84:56012) 2023-10-16T11:25:30.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.313+0000 7f6972ffd640 1 -- 172.21.15.84:0/2003213747 learned_addr learned my addr 172.21.15.84:0/2003213747 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:30.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.314+0000 7f6973fff640 1 --2- 172.21.15.84:0/2003213747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69740caac0 0x7f69740ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:30.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.314+0000 7f69727fc640 1 -- 172.21.15.84:0/2003213747 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2070286150 0 0) 0x7f69740cb030 con 0x7f697406a4b0 2023-10-16T11:25:30.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.314+0000 7f69727fc640 1 -- 172.21.15.84:0/2003213747 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6960003520 con 0x7f697406a4b0 2023-10-16T11:25:30.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.314+0000 7f69727fc640 1 -- 172.21.15.84:0/2003213747 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1631397570 0 0) 0x7f6960003520 con 0x7f697406a4b0 2023-10-16T11:25:30.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.314+0000 7f69727fc640 1 -- 172.21.15.84:0/2003213747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69740caac0 msgr2=0x7f69740ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:30.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.314+0000 7f69727fc640 1 --2- 172.21.15.84:0/2003213747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69740caac0 0x7f69740ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:30.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.314+0000 7f69727fc640 1 -- 172.21.15.84:0/2003213747 >> v1:172.21.15.157:6789/0 conn(0x7f697415ad70 legacy=0x7f6974069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:30.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.314+0000 7f69727fc640 1 -- 172.21.15.84:0/2003213747 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69740c9780 con 0x7f697406a4b0 2023-10-16T11:25:30.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.314+0000 7f69727fc640 1 -- 172.21.15.84:0/2003213747 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f695c001670 con 0x7f697406a4b0 2023-10-16T11:25:30.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.314+0000 7f69727fc640 1 -- 172.21.15.84:0/2003213747 >> v1:172.21.15.84:6789/0 conn(0x7f697406a4b0 legacy=0x7f69740ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:30.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.314+0000 7f69727fc640 1 --2- 172.21.15.84:0/2003213747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69740caac0 0x7f69600039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:30.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.314+0000 7f69727fc640 1 --2- 172.21.15.84:0/2003213747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6960003f70 0x7f69600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:30.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.314+0000 7f69727fc640 1 --2- 172.21.15.84:0/2003213747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6960004d00 0x7f6960005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:30.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.314+0000 7f69727fc640 1 -- 172.21.15.84:0/2003213747 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69740c94a0 con 0x7f6960004d00 2023-10-16T11:25:30.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.314+0000 7f69727fc640 1 -- 172.21.15.84:0/2003213747 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696000bac0 con 0x7f6960003f70 2023-10-16T11:25:30.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.314+0000 7f69727fc640 1 -- 172.21.15.84:0/2003213747 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696000bc80 con 0x7f69740caac0 2023-10-16T11:25:30.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.315+0000 7f6973fff640 1 --2- 172.21.15.84:0/2003213747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6960004d00 0x7f6960005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:30.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.315+0000 7f6972ffd640 1 --2- 172.21.15.84:0/2003213747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6960003f70 0x7f69600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:30.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.315+0000 7f6973fff640 1 -- 172.21.15.84:0/2003213747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69740caac0 msgr2=0x7f69600039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:30.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.315+0000 7f6973fff640 1 --2- 172.21.15.84:0/2003213747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69740caac0 0x7f69600039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:30.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.315+0000 7f6973fff640 1 -- 172.21.15.84:0/2003213747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6960003f70 msgr2=0x7f69600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:30.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.315+0000 7f6973fff640 1 --2- 172.21.15.84:0/2003213747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6960003f70 0x7f69600047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:30.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.315+0000 7f6973fff640 1 -- 172.21.15.84:0/2003213747 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6960003520 con 0x7f6960004d00 2023-10-16T11:25:30.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.316+0000 7f6973fff640 1 --2- 172.21.15.84:0/2003213747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6960004d00 0x7f6960005180 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7f696c009ac0 tx=0x7f69600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8d90cd100879b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:30.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.316+0000 7f69727fc640 1 -- 172.21.15.84:0/2003213747 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c02c710 con 0x7f6960004d00 2023-10-16T11:25:30.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.316+0000 7f69727fc640 1 -- 172.21.15.84:0/2003213747 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696c02c870 con 0x7f6960004d00 2023-10-16T11:25:30.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.316+0000 7f69727fc640 1 -- 172.21.15.84:0/2003213747 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c02cc00 con 0x7f6960004d00 2023-10-16T11:25:30.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.317+0000 7f697a116640 1 -- 172.21.15.84:0/2003213747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6960004d00 msgr2=0x7f6960005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:30.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.317+0000 7f697a116640 1 --2- 172.21.15.84:0/2003213747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6960004d00 0x7f6960005180 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7f696c009ac0 tx=0x7f69600057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:30.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.317+0000 7f6973fff640 1 -- 172.21.15.84:0/2003213747 reap_dead start 2023-10-16T11:25:30.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.317+0000 7f697a116640 1 -- 172.21.15.84:0/2003213747 shutdown_connections 2023-10-16T11:25:30.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.317+0000 7f697a116640 1 -- 172.21.15.84:0/2003213747 >> 172.21.15.84:0/2003213747 conn(0x7f69740ce9b0 msgr2=0x7f69740cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:30.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.317+0000 7f697a116640 1 -- 172.21.15.84:0/2003213747 shutdown_connections 2023-10-16T11:25:30.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.317+0000 7f697a116640 1 -- 172.21.15.84:0/2003213747 wait complete. 2023-10-16T11:25:30.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.319+0000 7f697a116640 1 Processor -- start 2023-10-16T11:25:30.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.319+0000 7f697a116640 1 -- start start 2023-10-16T11:25:30.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.320+0000 7f697a116640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6960004d00 0x7f697415c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:30.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.320+0000 7f697a116640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69740c24a0 0x7f6974145da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:30.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.320+0000 7f697a116640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697415cd70 0x7f697415d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:30.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.320+0000 7f697a116640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697415cac0 con 0x7f69740c24a0 2023-10-16T11:25:30.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.320+0000 7f697a116640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697415d710 con 0x7f697415cd70 2023-10-16T11:25:30.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.320+0000 7f697a116640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69740c3800 con 0x7f6960004d00 2023-10-16T11:25:30.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.320+0000 7f6972ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69740c24a0 0x7f6974145da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:30.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.320+0000 7f6972ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69740c24a0 0x7f6974145da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55662/0 (socket says 172.21.15.84:55662) 2023-10-16T11:25:30.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.320+0000 7f6972ffd640 1 -- 172.21.15.84:0/2992249907 learned_addr learned my addr 172.21.15.84:0/2992249907 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:30.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.320+0000 7f6972ffd640 1 -- 172.21.15.84:0/2992249907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6960004d00 msgr2=0x7f697415c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:30.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.320+0000 7f6972ffd640 1 --2- 172.21.15.84:0/2992249907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6960004d00 0x7f697415c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:30.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.320+0000 7f6972ffd640 1 -- 172.21.15.84:0/2992249907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697415cd70 msgr2=0x7f697415d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:30.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.320+0000 7f6972ffd640 1 --2- 172.21.15.84:0/2992249907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697415cd70 0x7f697415d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:30.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.320+0000 7f6972ffd640 1 -- 172.21.15.84:0/2992249907 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69740c2920 con 0x7f69740c24a0 2023-10-16T11:25:30.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.321+0000 7f6972ffd640 1 --2- 172.21.15.84:0/2992249907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69740c24a0 0x7f6974145da0 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7f695c00d830 tx=0x7f695c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:30.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.321+0000 7f696bfff640 1 -- 172.21.15.84:0/2992249907 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c0074e0 con 0x7f69740c24a0 2023-10-16T11:25:30.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.321+0000 7f696bfff640 1 -- 172.21.15.84:0/2992249907 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695c007640 con 0x7f69740c24a0 2023-10-16T11:25:30.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.321+0000 7f696bfff640 1 -- 172.21.15.84:0/2992249907 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c0049e0 con 0x7f69740c24a0 2023-10-16T11:25:30.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.321+0000 7f697a116640 1 -- 172.21.15.84:0/2992249907 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69740c2bf0 con 0x7f69740c24a0 2023-10-16T11:25:30.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.321+0000 7f697a116640 1 -- 172.21.15.84:0/2992249907 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69740c31b0 con 0x7f69740c24a0 2023-10-16T11:25:30.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.322+0000 7f697a116640 1 -- 172.21.15.84:0/2992249907 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697404e160 con 0x7f69740c24a0 2023-10-16T11:25:30.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.323+0000 7f696bfff640 1 -- 172.21.15.84:0/2992249907 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f695c004b40 con 0x7f69740c24a0 2023-10-16T11:25:30.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.324+0000 7f696bfff640 1 --2- 172.21.15.84:0/2992249907 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f695406b0b0 0x7f695406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:30.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.324+0000 7f696bfff640 1 -- 172.21.15.84:0/2992249907 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f695c0895a0 con 0x7f69740c24a0 2023-10-16T11:25:30.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.324+0000 7f69737fe640 1 --2- 172.21.15.84:0/2992249907 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f695406b0b0 0x7f695406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:30.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.325+0000 7f69737fe640 1 --2- 172.21.15.84:0/2992249907 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f695406b0b0 0x7f695406d570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6964001c90 tx=0x7f6964007d10 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:30.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.329+0000 7f696bfff640 1 -- 172.21.15.84:0/2992249907 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f695c0541e0 con 0x7f69740c24a0 2023-10-16T11:25:30.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.520+0000 7f697a116640 1 -- 172.21.15.84:0/2992249907 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f697406acf0 con 0x7f69740c24a0 2023-10-16T11:25:30.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.527+0000 7f696bfff640 1 -- 172.21.15.84:0/2992249907 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v82)=0 v82) v1 ==== 122+0+0 (secure 0 0 0) 0x7f695c057800 con 0x7f69740c24a0 2023-10-16T11:25:30.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.530+0000 7f697a116640 1 -- 172.21.15.84:0/2992249907 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f695406b0b0 msgr2=0x7f695406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:30.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.530+0000 7f697a116640 1 --2- 172.21.15.84:0/2992249907 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f695406b0b0 0x7f695406d570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6964001c90 tx=0x7f6964007d10 comp rx=0 tx=0).stop 2023-10-16T11:25:30.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.530+0000 7f697a116640 1 -- 172.21.15.84:0/2992249907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69740c24a0 msgr2=0x7f6974145da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:30.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.530+0000 7f697a116640 1 --2- 172.21.15.84:0/2992249907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69740c24a0 0x7f6974145da0 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7f695c00d830 tx=0x7f695c00dcf0 comp rx=0 tx=0).stop 2023-10-16T11:25:30.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.531+0000 7f697a116640 1 -- 172.21.15.84:0/2992249907 shutdown_connections 2023-10-16T11:25:30.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.531+0000 7f697a116640 1 --2- 172.21.15.84:0/2992249907 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f695406b0b0 0x7f695406d570 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:30.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.531+0000 7f697a116640 1 --2- 172.21.15.84:0/2992249907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697415cd70 0x7f697415d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:30.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.531+0000 7f697a116640 1 --2- 172.21.15.84:0/2992249907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69740c24a0 0x7f6974145da0 unknown :-1 s=CLOSED pgs=2884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:30.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.531+0000 7f697a116640 1 --2- 172.21.15.84:0/2992249907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6960004d00 0x7f697415c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:30.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.531+0000 7f697a116640 1 -- 172.21.15.84:0/2992249907 >> 172.21.15.84:0/2992249907 conn(0x7f69740ce9b0 msgr2=0x7f6974069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:30.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.531+0000 7f697a116640 1 -- 172.21.15.84:0/2992249907 shutdown_connections 2023-10-16T11:25:30.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:30.531+0000 7f697a116640 1 -- 172.21.15.84:0/2992249907 wait complete. 2023-10-16T11:25:30.551 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:25:30.552 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:25:30.673 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:25:30.798 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:25:30.799 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:25:30.799 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:25:30.860 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:25:30.860 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:25:30.861 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:25:30.922 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:25:30.922 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:25:30.923 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:25:30.984 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:25:30.985 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:25:30.985 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:25:31.047 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:25:31.047 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T11:25:31.048 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:25:31.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.269+0000 7fe766547640 1 Processor -- start 2023-10-16T11:25:31.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.269+0000 7fe766547640 1 -- start start 2023-10-16T11:25:31.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.270+0000 7fe766547640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe76015c9f0 0x7fe7600c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:31.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.270+0000 7fe766547640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe760160a90 con 0x7fe7600c9230 2023-10-16T11:25:31.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.270+0000 7fe766547640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7600cd900 con 0x7fe76015dde0 2023-10-16T11:25:31.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.270+0000 7fe766547640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7600cdec0 con 0x7fe76015c9f0 2023-10-16T11:25:31.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.271+0000 7fe764d44640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe7600c9230 0x7fe7600709e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56014/0 (socket says 172.21.15.84:56014) 2023-10-16T11:25:31.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.271+0000 7fe764d44640 1 -- 172.21.15.84:0/3500508913 learned_addr learned my addr 172.21.15.84:0/3500508913 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:31.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.271+0000 7fe765545640 1 --2- 172.21.15.84:0/3500508913 >> v2:172.21.15.157:3300/0 conn(0x7fe76015c9f0 0x7fe7600c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:31.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.271+0000 7fe757fff640 1 -- 172.21.15.84:0/3500508913 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1751907832 0 0) 0x7fe760160a90 con 0x7fe7600c9230 2023-10-16T11:25:31.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.271+0000 7fe757fff640 1 -- 172.21.15.84:0/3500508913 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe74c003520 con 0x7fe7600c9230 2023-10-16T11:25:31.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.272+0000 7fe757fff640 1 -- 172.21.15.84:0/3500508913 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 218906187 0 0) 0x7fe74c003520 con 0x7fe7600c9230 2023-10-16T11:25:31.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.272+0000 7fe757fff640 1 -- 172.21.15.84:0/3500508913 >> v2:172.21.15.157:3300/0 conn(0x7fe76015c9f0 msgr2=0x7fe7600c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:31.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.272+0000 7fe757fff640 1 --2- 172.21.15.84:0/3500508913 >> v2:172.21.15.157:3300/0 conn(0x7fe76015c9f0 0x7fe7600c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:31.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.272+0000 7fe757fff640 1 -- 172.21.15.84:0/3500508913 >> v1:172.21.15.157:6789/0 conn(0x7fe76015dde0 legacy=0x7fe7600ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:31.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.272+0000 7fe757fff640 1 -- 172.21.15.84:0/3500508913 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7600cdbe0 con 0x7fe7600c9230 2023-10-16T11:25:31.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.272+0000 7fe757fff640 1 -- 172.21.15.84:0/3500508913 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fe750001670 con 0x7fe7600c9230 2023-10-16T11:25:31.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.272+0000 7fe757fff640 1 -- 172.21.15.84:0/3500508913 >> v1:172.21.15.84:6789/0 conn(0x7fe7600c9230 legacy=0x7fe7600709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:31.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.272+0000 7fe757fff640 1 --2- 172.21.15.84:0/3500508913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe74c0039c0 0x7fe74c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:31.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.272+0000 7fe757fff640 1 --2- 172.21.15.84:0/3500508913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe74c0047c0 0x7fe74c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:31.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.272+0000 7fe757fff640 1 --2- 172.21.15.84:0/3500508913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe74c005180 0x7fe74c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:31.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.272+0000 7fe757fff640 1 -- 172.21.15.84:0/3500508913 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c00c050 con 0x7fe74c0039c0 2023-10-16T11:25:31.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.272+0000 7fe757fff640 1 -- 172.21.15.84:0/3500508913 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c00c1c0 con 0x7fe74c005180 2023-10-16T11:25:31.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.272+0000 7fe757fff640 1 -- 172.21.15.84:0/3500508913 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c00c380 con 0x7fe74c0047c0 2023-10-16T11:25:31.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.272+0000 7fe765545640 1 --2- 172.21.15.84:0/3500508913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe74c0039c0 0x7fe74c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:31.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.273+0000 7fe765545640 1 -- 172.21.15.84:0/3500508913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe74c0047c0 msgr2=0x7fe74c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:31.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.273+0000 7fe764d44640 1 --2- 172.21.15.84:0/3500508913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe74c0047c0 0x7fe74c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:31.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.273+0000 7fe765d46640 1 --2- 172.21.15.84:0/3500508913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe74c005180 0x7fe74c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:31.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.273+0000 7fe765545640 1 --2- 172.21.15.84:0/3500508913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe74c0047c0 0x7fe74c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:31.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.273+0000 7fe765545640 1 -- 172.21.15.84:0/3500508913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe74c005180 msgr2=0x7fe74c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:31.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.273+0000 7fe765545640 1 --2- 172.21.15.84:0/3500508913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe74c005180 0x7fe74c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:31.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.273+0000 7fe765d46640 1 -- 172.21.15.84:0/3500508913 reap_dead start 2023-10-16T11:25:31.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.273+0000 7fe765545640 1 -- 172.21.15.84:0/3500508913 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74c003520 con 0x7fe74c0039c0 2023-10-16T11:25:31.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.273+0000 7fe764d44640 1 --2- 172.21.15.84:0/3500508913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe74c0047c0 0x7fe74c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:31.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.274+0000 7fe765545640 1 --2- 172.21.15.84:0/3500508913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe74c0039c0 0x7fe74c003de0 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009ac0 tx=0x7fe74c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10b4616722288cc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:31.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.274+0000 7fe757fff640 1 -- 172.21.15.84:0/3500508913 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c02c6b0 con 0x7fe74c0039c0 2023-10-16T11:25:31.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.274+0000 7fe757fff640 1 -- 172.21.15.84:0/3500508913 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c02c810 con 0x7fe74c0039c0 2023-10-16T11:25:31.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.274+0000 7fe757fff640 1 -- 172.21.15.84:0/3500508913 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c02cba0 con 0x7fe74c0039c0 2023-10-16T11:25:31.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.275+0000 7fe766547640 1 -- 172.21.15.84:0/3500508913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe74c0039c0 msgr2=0x7fe74c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:31.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.275+0000 7fe766547640 1 --2- 172.21.15.84:0/3500508913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe74c0039c0 0x7fe74c003de0 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009ac0 tx=0x7fe74c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:31.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.275+0000 7fe766547640 1 -- 172.21.15.84:0/3500508913 shutdown_connections 2023-10-16T11:25:31.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.275+0000 7fe766547640 1 --2- 172.21.15.84:0/3500508913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe74c0039c0 0x7fe74c003de0 unknown :-1 s=CLOSED pgs=2886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:31.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.275+0000 7fe766547640 1 -- 172.21.15.84:0/3500508913 >> 172.21.15.84:0/3500508913 conn(0x7fe7600ce6a0 msgr2=0x7fe7600ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:31.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.275+0000 7fe766547640 1 -- 172.21.15.84:0/3500508913 shutdown_connections 2023-10-16T11:25:31.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.275+0000 7fe766547640 1 -- 172.21.15.84:0/3500508913 wait complete. 2023-10-16T11:25:31.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.277+0000 7fe766547640 1 Processor -- start 2023-10-16T11:25:31.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.277+0000 7fe766547640 1 -- start start 2023-10-16T11:25:31.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.278+0000 7fe766547640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe74c0039c0 0x7fe760069da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:31.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.278+0000 7fe766547640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7600c4730 0x7fe7600c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:31.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.278+0000 7fe766547640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe76006ba80 0x7fe76006bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:31.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.278+0000 7fe766547640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76015f660 con 0x7fe74c0039c0 2023-10-16T11:25:31.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.278+0000 7fe766547640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76015ed30 con 0x7fe7600c4730 2023-10-16T11:25:31.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.278+0000 7fe766547640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76015f010 con 0x7fe76006ba80 2023-10-16T11:25:31.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.278+0000 7fe765545640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe74c0039c0 0x7fe760069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:31.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.278+0000 7fe765545640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe74c0039c0 0x7fe760069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55678/0 (socket says 172.21.15.84:55678) 2023-10-16T11:25:31.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.278+0000 7fe765545640 1 -- 172.21.15.84:0/2070744148 learned_addr learned my addr 172.21.15.84:0/2070744148 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:31.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.278+0000 7fe764d44640 1 --2- 172.21.15.84:0/2070744148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7600c4730 0x7fe7600c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:31.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.279+0000 7fe765545640 1 -- 172.21.15.84:0/2070744148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe76006ba80 msgr2=0x7fe76006bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:31.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.279+0000 7fe765d46640 1 --2- 172.21.15.84:0/2070744148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe76006ba80 0x7fe76006bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:31.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.279+0000 7fe765545640 1 --2- 172.21.15.84:0/2070744148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe76006ba80 0x7fe76006bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:31.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.279+0000 7fe765545640 1 -- 172.21.15.84:0/2070744148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7600c4730 msgr2=0x7fe7600c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:31.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.279+0000 7fe765545640 1 --2- 172.21.15.84:0/2070744148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7600c4730 0x7fe7600c4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:31.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.279+0000 7fe765545640 1 -- 172.21.15.84:0/2070744148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76015f2f0 con 0x7fe74c0039c0 2023-10-16T11:25:31.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.279+0000 7fe764d44640 1 --2- 172.21.15.84:0/2070744148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7600c4730 0x7fe7600c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:31.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.279+0000 7fe765545640 1 --2- 172.21.15.84:0/2070744148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe74c0039c0 0x7fe760069da0 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009ac0 tx=0x7fe75c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:31.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.279+0000 7fe755ffb640 1 -- 172.21.15.84:0/2070744148 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c003810 con 0x7fe74c0039c0 2023-10-16T11:25:31.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.279+0000 7fe755ffb640 1 -- 172.21.15.84:0/2070744148 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c003970 con 0x7fe74c0039c0 2023-10-16T11:25:31.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.279+0000 7fe755ffb640 1 -- 172.21.15.84:0/2070744148 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c003d00 con 0x7fe74c0039c0 2023-10-16T11:25:31.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.279+0000 7fe766547640 1 -- 172.21.15.84:0/2070744148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe76015faf0 con 0x7fe74c0039c0 2023-10-16T11:25:31.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.280+0000 7fe766547640 1 -- 172.21.15.84:0/2070744148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76006c440 con 0x7fe74c0039c0 2023-10-16T11:25:31.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.281+0000 7fe766547640 1 -- 172.21.15.84:0/2070744148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe76006feb0 con 0x7fe74c0039c0 2023-10-16T11:25:31.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.282+0000 7fe755ffb640 1 -- 172.21.15.84:0/2070744148 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe75c040450 con 0x7fe74c0039c0 2023-10-16T11:25:31.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.283+0000 7fe755ffb640 1 --2- 172.21.15.84:0/2070744148 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe74406b0b0 0x7fe74406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:31.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.283+0000 7fe755ffb640 1 -- 172.21.15.84:0/2070744148 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5874+0+0 (secure 0 0 0) 0x7fe75c0aba30 con 0x7fe74c0039c0 2023-10-16T11:25:31.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.283+0000 7fe764d44640 1 --2- 172.21.15.84:0/2070744148 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe74406b0b0 0x7fe74406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:31.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.283+0000 7fe764d44640 1 --2- 172.21.15.84:0/2070744148 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe74406b0b0 0x7fe74406d570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe750000c00 tx=0x7fe750009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:31.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.287+0000 7fe755ffb640 1 -- 172.21.15.84:0/2070744148 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe75c0766d0 con 0x7fe74c0039c0 2023-10-16T11:25:31.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.515+0000 7fe766547640 1 -- 172.21.15.84:0/2070744148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe76004e9b0 con 0x7fe74c0039c0 2023-10-16T11:25:31.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.516+0000 7fe755ffb640 1 -- 172.21.15.84:0/2070744148 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+3711 (secure 0 0 0) 0x7fe75c044e90 con 0x7fe74c0039c0 2023-10-16T11:25:31.518 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:31.518 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":124,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3106898227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3106898227},{"type":"v1","addr":"172.21.15.84:6837","nonce":3106898227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":7471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2296953749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2296953749},{"type":"v1","addr":"172.21.15.157:6834","nonce":2296953749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":7750,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2526057380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2526057380},{"type":"v1","addr":"172.21.15.157:6835","nonce":2526057380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":124},{"gid":12015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/3350751452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3350751452},{"type":"v1","addr":"172.21.15.84:6836","nonce":3350751452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[]} 2023-10-16T11:25:31.519 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 124 2023-10-16T11:25:31.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.518+0000 7fe766547640 1 -- 172.21.15.84:0/2070744148 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe74406b0b0 msgr2=0x7fe74406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:31.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.518+0000 7fe766547640 1 --2- 172.21.15.84:0/2070744148 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe74406b0b0 0x7fe74406d570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe750000c00 tx=0x7fe750009040 comp rx=0 tx=0).stop 2023-10-16T11:25:31.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.518+0000 7fe766547640 1 -- 172.21.15.84:0/2070744148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe74c0039c0 msgr2=0x7fe760069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:31.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.518+0000 7fe766547640 1 --2- 172.21.15.84:0/2070744148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe74c0039c0 0x7fe760069da0 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009ac0 tx=0x7fe75c004730 comp rx=0 tx=0).stop 2023-10-16T11:25:31.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.518+0000 7fe766547640 1 -- 172.21.15.84:0/2070744148 shutdown_connections 2023-10-16T11:25:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.518+0000 7fe766547640 1 --2- 172.21.15.84:0/2070744148 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe74406b0b0 0x7fe74406d570 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.518+0000 7fe766547640 1 --2- 172.21.15.84:0/2070744148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe76006ba80 0x7fe76006bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.518+0000 7fe766547640 1 --2- 172.21.15.84:0/2070744148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7600c4730 0x7fe7600c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.518+0000 7fe766547640 1 --2- 172.21.15.84:0/2070744148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe74c0039c0 0x7fe760069da0 unknown :-1 s=CLOSED pgs=2887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.518+0000 7fe766547640 1 -- 172.21.15.84:0/2070744148 >> 172.21.15.84:0/2070744148 conn(0x7fe7600ce6a0 msgr2=0x7fe7600c9710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.519+0000 7fe766547640 1 -- 172.21.15.84:0/2070744148 shutdown_connections 2023-10-16T11:25:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.519+0000 7fe766547640 1 -- 172.21.15.84:0/2070744148 wait complete. 2023-10-16T11:25:31.539 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T11:25:31.540 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T11:25:31.540 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T11:25:31.540 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:25:31.541 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T11:25:31.541 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T11:25:31.541 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T11:25:31.542 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:25:31.542 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T11:25:31.542 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T11:25:31.543 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T11:25:31.543 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:25:31.543 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T11:25:31.544 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T11:25:31.544 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T11:25:31.544 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:25:31.556 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T11:25:31.556 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T11:25:31.559 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T11:25:31.560 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T11:25:31.564 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T11:25:31.565 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T11:25:31.569 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T11:25:31.569 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T11:25:31.607 INFO:tasks.ceph.mds.c:Started 2023-10-16T11:25:31.608 DEBUG:teuthology.parallel:result is None 2023-10-16T11:25:31.616 INFO:tasks.ceph.mds.a:Started 2023-10-16T11:25:31.617 DEBUG:teuthology.parallel:result is None 2023-10-16T11:25:31.617 INFO:tasks.ceph.mds.b:Started 2023-10-16T11:25:31.618 DEBUG:teuthology.parallel:result is None 2023-10-16T11:25:31.625 INFO:tasks.ceph.mds.d:Started 2023-10-16T11:25:31.626 DEBUG:teuthology.parallel:result is None 2023-10-16T11:25:31.626 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T11:25:31.760 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T11:25:31.764 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T11:25:31.777 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T11:25:31.787 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-10-16T11:25:31.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.801+0000 7f98a73c4640 1 Processor -- start 2023-10-16T11:25:31.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.801+0000 7f98a73c4640 1 -- start start 2023-10-16T11:25:31.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.802+0000 7f98a73c4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98a015dde0 0x7f98a00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:31.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.802+0000 7f98a73c4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f98a0070f20 0x7f98a00ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:31.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.802+0000 7f98a73c4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a0160a90 con 0x7f98a015c9f0 2023-10-16T11:25:31.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.802+0000 7f98a73c4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a00cdec0 con 0x7f98a0070f20 2023-10-16T11:25:31.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.802+0000 7f98a73c4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00cd900 con 0x7f98a015dde0 2023-10-16T11:25:31.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.802+0000 7f98a63c2640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f98a015c9f0 0x7f98a00c8cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56038/0 (socket says 172.21.15.84:56038) 2023-10-16T11:25:31.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.802+0000 7f98a63c2640 1 -- 172.21.15.84:0/1358980882 learned_addr learned my addr 172.21.15.84:0/1358980882 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:31.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a5bc1640 1 --2- 172.21.15.84:0/1358980882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98a015dde0 0x7f98a00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:31.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a6bc3640 1 --2- 172.21.15.84:0/1358980882 >> v2:172.21.15.157:3300/0 conn(0x7f98a0070f20 0x7f98a00ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:31.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a53c0640 1 -- 172.21.15.84:0/1358980882 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489071761 0 0) 0x7f98a0160a90 con 0x7f98a015c9f0 2023-10-16T11:25:31.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a53c0640 1 -- 172.21.15.84:0/1358980882 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f988c003520 con 0x7f98a015c9f0 2023-10-16T11:25:31.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a53c0640 1 -- 172.21.15.84:0/1358980882 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 402362867 0 0) 0x7f988c003520 con 0x7f98a015c9f0 2023-10-16T11:25:31.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a53c0640 1 -- 172.21.15.84:0/1358980882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98a015dde0 msgr2=0x7f98a00709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:31.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a53c0640 1 --2- 172.21.15.84:0/1358980882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98a015dde0 0x7f98a00709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:31.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a53c0640 1 -- 172.21.15.84:0/1358980882 >> v2:172.21.15.157:3300/0 conn(0x7f98a0070f20 msgr2=0x7f98a00ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:31.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a53c0640 1 --2- 172.21.15.84:0/1358980882 >> v2:172.21.15.157:3300/0 conn(0x7f98a0070f20 0x7f98a00ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:31.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a53c0640 1 -- 172.21.15.84:0/1358980882 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a00cdbe0 con 0x7f98a015c9f0 2023-10-16T11:25:31.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a53c0640 1 -- 172.21.15.84:0/1358980882 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f989c001670 con 0x7f98a015c9f0 2023-10-16T11:25:31.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a6bc3640 1 --2- 172.21.15.84:0/1358980882 >> v2:172.21.15.157:3300/0 conn(0x7f98a0070f20 0x7f98a00ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:25:31.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a53c0640 1 -- 172.21.15.84:0/1358980882 >> v1:172.21.15.84:6789/0 conn(0x7f98a015c9f0 legacy=0x7f98a00c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:31.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a53c0640 1 --2- 172.21.15.84:0/1358980882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f988c003970 0x7f988c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:31.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a53c0640 1 --2- 172.21.15.84:0/1358980882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988c0047c0 0x7f988c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:31.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a53c0640 1 --2- 172.21.15.84:0/1358980882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98a015dde0 0x7f988c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:31.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a53c0640 1 -- 172.21.15.84:0/1358980882 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00cd900 con 0x7f988c0047c0 2023-10-16T11:25:31.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a53c0640 1 -- 172.21.15.84:0/1358980882 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c00b980 con 0x7f988c003970 2023-10-16T11:25:31.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.803+0000 7f98a53c0640 1 -- 172.21.15.84:0/1358980882 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988c00bac0 con 0x7f98a015dde0 2023-10-16T11:25:31.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.804+0000 7f98a5bc1640 1 --2- 172.21.15.84:0/1358980882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988c0047c0 0x7f988c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:31.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.804+0000 7f98a63c2640 1 --2- 172.21.15.84:0/1358980882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f988c003970 0x7f988c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:31.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.804+0000 7f98a6bc3640 1 --2- 172.21.15.84:0/1358980882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98a015dde0 0x7f988c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:31.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.804+0000 7f98a5bc1640 1 -- 172.21.15.84:0/1358980882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98a015dde0 msgr2=0x7f988c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:31.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.804+0000 7f98a5bc1640 1 --2- 172.21.15.84:0/1358980882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98a015dde0 0x7f988c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:31.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.804+0000 7f98a5bc1640 1 -- 172.21.15.84:0/1358980882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f988c003970 msgr2=0x7f988c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:31.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.804+0000 7f98a5bc1640 1 --2- 172.21.15.84:0/1358980882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f988c003970 0x7f988c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:31.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.804+0000 7f98a5bc1640 1 -- 172.21.15.84:0/1358980882 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c003520 con 0x7f988c0047c0 2023-10-16T11:25:31.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.805+0000 7f98a5bc1640 1 --2- 172.21.15.84:0/1358980882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988c0047c0 0x7f988c004c40 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7f9890009a90 tx=0x7f988c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f9ddc9aab9892c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:31.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.805+0000 7f98a53c0640 1 -- 172.21.15.84:0/1358980882 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989002c670 con 0x7f988c0047c0 2023-10-16T11:25:31.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.805+0000 7f98a53c0640 1 -- 172.21.15.84:0/1358980882 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989002c7d0 con 0x7f988c0047c0 2023-10-16T11:25:31.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.805+0000 7f98a53c0640 1 -- 172.21.15.84:0/1358980882 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989002cb60 con 0x7f988c0047c0 2023-10-16T11:25:31.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.806+0000 7f98a73c4640 1 -- 172.21.15.84:0/1358980882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988c0047c0 msgr2=0x7f988c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:31.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.806+0000 7f98a73c4640 1 --2- 172.21.15.84:0/1358980882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988c0047c0 0x7f988c004c40 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7f9890009a90 tx=0x7f988c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:31.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.806+0000 7f98a6bc3640 1 -- 172.21.15.84:0/1358980882 reap_dead start 2023-10-16T11:25:31.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.806+0000 7f98a73c4640 1 -- 172.21.15.84:0/1358980882 shutdown_connections 2023-10-16T11:25:31.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.806+0000 7f98a73c4640 1 -- 172.21.15.84:0/1358980882 >> 172.21.15.84:0/1358980882 conn(0x7f98a00ce6a0 msgr2=0x7f98a00ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:31.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.806+0000 7f98a73c4640 1 -- 172.21.15.84:0/1358980882 shutdown_connections 2023-10-16T11:25:31.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.806+0000 7f98a73c4640 1 -- 172.21.15.84:0/1358980882 wait complete. 2023-10-16T11:25:31.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.808+0000 7f98a73c4640 1 Processor -- start 2023-10-16T11:25:31.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.808+0000 7f98a73c4640 1 -- start start 2023-10-16T11:25:31.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.809+0000 7f98a73c4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988c0047c0 0x7f98a014ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:31.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.809+0000 7f98a73c4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98a00c2c50 0x7f98a015b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:31.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.809+0000 7f98a73c4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98a015bd20 0x7f98a0156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:31.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.809+0000 7f98a73c4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a015c1a0 con 0x7f988c0047c0 2023-10-16T11:25:31.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.809+0000 7f98a73c4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a01f5ba0 con 0x7f98a015bd20 2023-10-16T11:25:31.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.809+0000 7f98a73c4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a01f5e10 con 0x7f98a00c2c50 2023-10-16T11:25:31.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.809+0000 7f98a63c2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988c0047c0 0x7f98a014ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:31.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.809+0000 7f98a63c2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988c0047c0 0x7f98a014ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55726/0 (socket says 172.21.15.84:55726) 2023-10-16T11:25:31.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.809+0000 7f98a63c2640 1 -- 172.21.15.84:0/2874736541 learned_addr learned my addr 172.21.15.84:0/2874736541 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:31.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.809+0000 7f98a5bc1640 1 --2- 172.21.15.84:0/2874736541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98a00c2c50 0x7f98a015b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:31.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.809+0000 7f98a6bc3640 1 --2- 172.21.15.84:0/2874736541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98a015bd20 0x7f98a0156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:31.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.809+0000 7f98a63c2640 1 -- 172.21.15.84:0/2874736541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98a00c2c50 msgr2=0x7f98a015b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:31.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.809+0000 7f98a63c2640 1 --2- 172.21.15.84:0/2874736541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98a00c2c50 0x7f98a015b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:31.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.809+0000 7f98a63c2640 1 -- 172.21.15.84:0/2874736541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98a015bd20 msgr2=0x7f98a0156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:31.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.809+0000 7f98a63c2640 1 --2- 172.21.15.84:0/2874736541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98a015bd20 0x7f98a0156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:31.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.809+0000 7f98a63c2640 1 -- 172.21.15.84:0/2874736541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a01f60f0 con 0x7f988c0047c0 2023-10-16T11:25:31.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.810+0000 7f98a63c2640 1 --2- 172.21.15.84:0/2874736541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988c0047c0 0x7f98a014ccb0 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7f989c001050 tx=0x7f989c0076e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:31.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.810+0000 7f9896ffd640 1 -- 172.21.15.84:0/2874736541 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c0016d0 con 0x7f988c0047c0 2023-10-16T11:25:31.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.810+0000 7f9896ffd640 1 -- 172.21.15.84:0/2874736541 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c00f9c0 con 0x7f988c0047c0 2023-10-16T11:25:31.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.810+0000 7f9896ffd640 1 -- 172.21.15.84:0/2874736541 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c00fd70 con 0x7f988c0047c0 2023-10-16T11:25:31.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.810+0000 7f98a73c4640 1 -- 172.21.15.84:0/2874736541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a01f63d0 con 0x7f988c0047c0 2023-10-16T11:25:31.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.810+0000 7f98a73c4640 1 -- 172.21.15.84:0/2874736541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a014d1f0 con 0x7f988c0047c0 2023-10-16T11:25:31.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.811+0000 7f98a73c4640 1 -- 172.21.15.84:0/2874736541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a01f68f0 con 0x7f988c0047c0 2023-10-16T11:25:31.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.812+0000 7f9896ffd640 1 -- 172.21.15.84:0/2874736541 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f989c01d460 con 0x7f988c0047c0 2023-10-16T11:25:31.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.813+0000 7f9896ffd640 1 --2- 172.21.15.84:0/2874736541 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f988406b0e0 0x7f988406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:31.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.814+0000 7f9896ffd640 1 -- 172.21.15.84:0/2874736541 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f989c089b10 con 0x7f988c0047c0 2023-10-16T11:25:31.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.814+0000 7f98a5bc1640 1 --2- 172.21.15.84:0/2874736541 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f988406b0e0 0x7f988406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:31.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.814+0000 7f98a5bc1640 1 --2- 172.21.15.84:0/2874736541 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f988406b0e0 0x7f988406d5a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9890036000 tx=0x7f9890030000 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:31.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:31.818+0000 7f9896ffd640 1 -- 172.21.15.84:0/2874736541 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f98a01f68f0 con 0x7f988c0047c0 2023-10-16T11:25:32.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.006+0000 7f98a73c4640 1 -- 172.21.15.84:0/2874736541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f98a01f68f0 con 0x7f988c0047c0 2023-10-16T11:25:32.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.096+0000 7f9896ffd640 1 -- 172.21.15.84:0/2874736541 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v231) v1 ==== 97+0+0 (secure 0 0 0) 0x7f98a01f68f0 con 0x7f988c0047c0 2023-10-16T11:25:32.099 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-10-16T11:25:32.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.100+0000 7f98a73c4640 1 -- 172.21.15.84:0/2874736541 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f988406b0e0 msgr2=0x7f988406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:32.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.100+0000 7f98a73c4640 1 --2- 172.21.15.84:0/2874736541 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f988406b0e0 0x7f988406d5a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9890036000 tx=0x7f9890030000 comp rx=0 tx=0).stop 2023-10-16T11:25:32.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.100+0000 7f98a73c4640 1 -- 172.21.15.84:0/2874736541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988c0047c0 msgr2=0x7f98a014ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:32.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.100+0000 7f98a73c4640 1 --2- 172.21.15.84:0/2874736541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988c0047c0 0x7f98a014ccb0 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7f989c001050 tx=0x7f989c0076e0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.100+0000 7f98a73c4640 1 -- 172.21.15.84:0/2874736541 shutdown_connections 2023-10-16T11:25:32.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.100+0000 7f98a73c4640 1 --2- 172.21.15.84:0/2874736541 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f988406b0e0 0x7f988406d5a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.100+0000 7f98a73c4640 1 --2- 172.21.15.84:0/2874736541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98a015bd20 0x7f98a0156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.100+0000 7f98a73c4640 1 --2- 172.21.15.84:0/2874736541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98a00c2c50 0x7f98a015b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.100+0000 7f98a73c4640 1 --2- 172.21.15.84:0/2874736541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988c0047c0 0x7f98a014ccb0 unknown :-1 s=CLOSED pgs=2896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.100+0000 7f98a73c4640 1 -- 172.21.15.84:0/2874736541 >> 172.21.15.84:0/2874736541 conn(0x7f98a00ce6a0 msgr2=0x7f98a015ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:32.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.101+0000 7f98a73c4640 1 -- 172.21.15.84:0/2874736541 shutdown_connections 2023-10-16T11:25:32.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.101+0000 7f98a73c4640 1 -- 172.21.15.84:0/2874736541 wait complete. 2023-10-16T11:25:32.120 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T11:25:32.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.312+0000 7fbfe4a15640 1 Processor -- start 2023-10-16T11:25:32.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.312+0000 7fbfe4a15640 1 -- start start 2023-10-16T11:25:32.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.313+0000 7fbfe4a15640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbfe015aaa0 0x7fbfe015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:32.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.314+0000 7fbfe4a15640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbfe015ae80 0x7fbfe00cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:32.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.314+0000 7fbfe4a15640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe015ed60 con 0x7fbfe00ccaa0 2023-10-16T11:25:32.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.314+0000 7fbfe4a15640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe006b810 con 0x7fbfe015ae80 2023-10-16T11:25:32.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.314+0000 7fbfe4a15640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe00707e0 con 0x7fbfe015aaa0 2023-10-16T11:25:32.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.314+0000 7fbfd7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbfe015ae80 0x7fbfe00cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:32.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.314+0000 7fbfd7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbfe015ae80 0x7fbfe00cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55728/0 (socket says 172.21.15.84:55728) 2023-10-16T11:25:32.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.314+0000 7fbfd7fff640 1 -- 172.21.15.84:0/1067023448 learned_addr learned my addr 172.21.15.84:0/1067023448 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:32.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.314+0000 7fbfd7fff640 1 -- 172.21.15.84:0/1067023448 >> v2:172.21.15.157:3300/0 conn(0x7fbfe015aaa0 msgr2=0x7fbfe015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:32.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.314+0000 7fbfd7fff640 1 --2- 172.21.15.84:0/1067023448 >> v2:172.21.15.157:3300/0 conn(0x7fbfe015aaa0 0x7fbfe015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.314+0000 7fbfd7fff640 1 -- 172.21.15.84:0/1067023448 >> v1:172.21.15.157:6789/0 conn(0x7fbfe00ccaa0 legacy=0x7fbfe0070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:32.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.314+0000 7fbfd7fff640 1 -- 172.21.15.84:0/1067023448 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe015ef90 con 0x7fbfe015ae80 2023-10-16T11:25:32.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.315+0000 7fbfd7fff640 1 --2- 172.21.15.84:0/1067023448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe015ae80 0x7fbfe00cc560 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00cd990 tx=0x7fbfc802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aefa282896f7728 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:32.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.316+0000 7fbfddd9b640 1 -- 172.21.15.84:0/1067023448 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8031870 con 0x7fbfe015ae80 2023-10-16T11:25:32.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.316+0000 7fbfddd9b640 1 -- 172.21.15.84:0/1067023448 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc80319d0 con 0x7fbfe015ae80 2023-10-16T11:25:32.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.316+0000 7fbfddd9b640 1 -- 172.21.15.84:0/1067023448 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8031d60 con 0x7fbfe015ae80 2023-10-16T11:25:32.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.316+0000 7fbfe4a15640 1 -- 172.21.15.84:0/1067023448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe015ae80 msgr2=0x7fbfe00cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:32.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.316+0000 7fbfe4a15640 1 --2- 172.21.15.84:0/1067023448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe015ae80 0x7fbfe00cc560 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00cd990 tx=0x7fbfc802b910 comp rx=0 tx=0).stop 2023-10-16T11:25:32.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.317+0000 7fbfe4a15640 1 -- 172.21.15.84:0/1067023448 shutdown_connections 2023-10-16T11:25:32.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.317+0000 7fbfe4a15640 1 --2- 172.21.15.84:0/1067023448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe015ae80 0x7fbfe00cc560 unknown :-1 s=CLOSED pgs=2897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.317+0000 7fbfe4a15640 1 --2- 172.21.15.84:0/1067023448 >> v2:172.21.15.157:3300/0 conn(0x7fbfe015aaa0 0x7fbfe015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.317+0000 7fbfe4a15640 1 -- 172.21.15.84:0/1067023448 >> 172.21.15.84:0/1067023448 conn(0x7fbfe00cb410 msgr2=0x7fbfe00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:32.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.317+0000 7fbfe4a15640 1 -- 172.21.15.84:0/1067023448 shutdown_connections 2023-10-16T11:25:32.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.317+0000 7fbfe4a15640 1 -- 172.21.15.84:0/1067023448 wait complete. 2023-10-16T11:25:32.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.318+0000 7fbfe4a15640 1 Processor -- start 2023-10-16T11:25:32.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.319+0000 7fbfe4a15640 1 -- start start 2023-10-16T11:25:32.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.320+0000 7fbfe4a15640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfe00ccaa0 0x7fbfe00b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:32.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.320+0000 7fbfe4a15640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfe015aaa0 0x7fbfe00ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:32.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.320+0000 7fbfe4a15640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe015ae80 0x7fbfe00ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:32.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.320+0000 7fbfe4a15640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00c9080 con 0x7fbfe015ae80 2023-10-16T11:25:32.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.320+0000 7fbfe4a15640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00c9320 con 0x7fbfe00ccaa0 2023-10-16T11:25:32.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.320+0000 7fbfe4a15640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00c9600 con 0x7fbfe015aaa0 2023-10-16T11:25:32.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.320+0000 7fbfded9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe015ae80 0x7fbfe00ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:32.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.320+0000 7fbfded9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe015ae80 0x7fbfe00ca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55740/0 (socket says 172.21.15.84:55740) 2023-10-16T11:25:32.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.320+0000 7fbfded9d640 1 -- 172.21.15.84:0/891736973 learned_addr learned my addr 172.21.15.84:0/891736973 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:32.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.320+0000 7fbfde59c640 1 --2- 172.21.15.84:0/891736973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfe00ccaa0 0x7fbfe00b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:32.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.320+0000 7fbfd7fff640 1 --2- 172.21.15.84:0/891736973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfe015aaa0 0x7fbfe00ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:32.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.320+0000 7fbfded9d640 1 -- 172.21.15.84:0/891736973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfe015aaa0 msgr2=0x7fbfe00ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:32.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.320+0000 7fbfded9d640 1 --2- 172.21.15.84:0/891736973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfe015aaa0 0x7fbfe00ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.320+0000 7fbfded9d640 1 -- 172.21.15.84:0/891736973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfe00ccaa0 msgr2=0x7fbfe00b26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:32.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.320+0000 7fbfded9d640 1 --2- 172.21.15.84:0/891736973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfe00ccaa0 0x7fbfe00b26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.320+0000 7fbfded9d640 1 -- 172.21.15.84:0/891736973 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc80094d0 con 0x7fbfe015ae80 2023-10-16T11:25:32.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.320+0000 7fbfde59c640 1 --2- 172.21.15.84:0/891736973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfe00ccaa0 0x7fbfe00b26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:32.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.321+0000 7fbfded9d640 1 --2- 172.21.15.84:0/891736973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe015ae80 0x7fbfe00ca880 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800c780 tx=0x7fbfd800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:32.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.321+0000 7fbfd6ffd640 1 -- 172.21.15.84:0/891736973 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8003d80 con 0x7fbfe015ae80 2023-10-16T11:25:32.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.321+0000 7fbfd6ffd640 1 -- 172.21.15.84:0/891736973 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd800ce90 con 0x7fbfe015ae80 2023-10-16T11:25:32.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.321+0000 7fbfd6ffd640 1 -- 172.21.15.84:0/891736973 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd80052b0 con 0x7fbfe015ae80 2023-10-16T11:25:32.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.321+0000 7fbfe4a15640 1 -- 172.21.15.84:0/891736973 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe00c98e0 con 0x7fbfe015ae80 2023-10-16T11:25:32.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.321+0000 7fbfe4a15640 1 -- 172.21.15.84:0/891736973 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe00c9e00 con 0x7fbfe015ae80 2023-10-16T11:25:32.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.322+0000 7fbfe4a15640 1 -- 172.21.15.84:0/891736973 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe004e160 con 0x7fbfe015ae80 2023-10-16T11:25:32.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.323+0000 7fbfd6ffd640 1 -- 172.21.15.84:0/891736973 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fbfd8005410 con 0x7fbfe015ae80 2023-10-16T11:25:32.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.324+0000 7fbfd6ffd640 1 --2- 172.21.15.84:0/891736973 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fbfc006b0b0 0x7fbfc006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:32.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.324+0000 7fbfd6ffd640 1 -- 172.21.15.84:0/891736973 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fbfd8088c80 con 0x7fbfe015ae80 2023-10-16T11:25:32.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.324+0000 7fbfde59c640 1 --2- 172.21.15.84:0/891736973 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fbfc006b0b0 0x7fbfc006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:32.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.325+0000 7fbfde59c640 1 --2- 172.21.15.84:0/891736973 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fbfc006b0b0 0x7fbfc006d570 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0006fd0 tx=0x7fbfd0008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:32.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.328+0000 7fbfd6ffd640 1 -- 172.21.15.84:0/891736973 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbfd8053c60 con 0x7fbfe015ae80 2023-10-16T11:25:32.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.593+0000 7fbfe4a15640 1 -- 172.21.15.84:0/891736973 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbfe00cdc60 con 0x7fbfe015ae80 2023-10-16T11:25:32.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.594+0000 7fbfd6ffd640 1 -- 172.21.15.84:0/891736973 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v56) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fbfd8017020 con 0x7fbfe015ae80 2023-10-16T11:25:32.596 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:32.597 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:25:32.597 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-10-16T11:25:32.597 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.597 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-10-16T11:25:32.597 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l6dhjKRAA0pMvY6AK6qtFXDprNhwsqg==", 2023-10-16T11:25:32.598 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.598 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:25:32.598 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:25:32.598 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:25:32.598 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:25:32.599 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.599 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:32.599 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.599 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-10-16T11:25:32.599 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lFpMxEBAA0TLPRt5q3FwwBvMG019cjQ==", 2023-10-16T11:25:32.599 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.600 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:25:32.600 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:25:32.600 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:25:32.600 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:25:32.600 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.601 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:32.601 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.601 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-10-16T11:25:32.601 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l73RBOhAAOMEPxfUAMHhn5Blyvd5oMw==", 2023-10-16T11:25:32.601 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.602 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:25:32.602 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:25:32.602 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:25:32.602 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:25:32.602 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.603 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:32.603 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.603 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-10-16T11:25:32.603 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lcTIRIhAA0+FWm4b4yw77LD3+WKbk7w==", 2023-10-16T11:25:32.603 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.604 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:25:32.604 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:25:32.604 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:25:32.604 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:25:32.604 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.605 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:32.605 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.605 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-10-16T11:25:32.605 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDYBi1l1zBLERAA3YLW3xIYQoRVw7KSkwN3Og==", 2023-10-16T11:25:32.605 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.606 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:25:32.606 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:25:32.606 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:25:32.606 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.606 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:32.607 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.607 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-10-16T11:25:32.607 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDbBi1lFUUxKRAAQbus8SmYprJRmWTjoCznNA==", 2023-10-16T11:25:32.607 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.607 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:25:32.608 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:25:32.608 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:25:32.608 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.608 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:32.608 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.609 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-10-16T11:25:32.609 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDeBi1l+9lBMxAArl6sb/h15uBG/a9lVH40Hg==", 2023-10-16T11:25:32.609 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.609 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:25:32.609 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:25:32.610 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:25:32.610 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.610 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:32.610 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.610 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-10-16T11:25:32.611 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDiBi1l2rnEERAA3oRD71xa0fEOynM8zKuBEg==", 2023-10-16T11:25:32.611 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.611 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:25:32.611 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:25:32.611 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:25:32.612 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.612 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:32.612 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.612 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-10-16T11:25:32.612 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDnBi1lR7oLChAAZto8fDovDbpOd3Fcnc7C6w==", 2023-10-16T11:25:32.612 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.613 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:25:32.613 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:25:32.613 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:25:32.613 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.613 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:32.614 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.614 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-10-16T11:25:32.614 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDqBi1lmjg/JBAAlINKRGBc0/UWJJHMVFw57w==", 2023-10-16T11:25:32.614 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.614 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:25:32.615 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:25:32.615 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:25:32.615 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.615 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:32.615 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.616 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-10-16T11:25:32.616 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDuBi1lu0kMARAAtuSeM8F3yr8CVrc3p1biaQ==", 2023-10-16T11:25:32.616 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.616 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:25:32.616 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:25:32.617 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:25:32.617 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.617 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:32.617 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.617 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-10-16T11:25:32.618 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDxBi1lf6mNGBAAHGbqzJ0fmDxQIsasboHMfg==", 2023-10-16T11:25:32.618 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.618 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:25:32.618 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:25:32.618 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:25:32.619 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.619 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:32.619 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.619 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-10-16T11:25:32.619 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ==", 2023-10-16T11:25:32.620 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.620 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:25:32.620 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow r", 2023-10-16T11:25:32.620 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T11:25:32.620 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.620 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:32.621 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.621 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-10-16T11:25:32.621 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1lHU+4BxAAd8ET8hQ8Y+EUwdTJr9/+Ig==", 2023-10-16T11:25:32.621 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.621 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:25:32.622 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:25:32.622 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:25:32.622 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:25:32.622 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.622 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:32.622 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.623 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-10-16T11:25:32.623 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lA1BOORAAFNyfqgEF34MVR9YX3KBKkQ==", 2023-10-16T11:25:32.623 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.623 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:25:32.623 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:25:32.624 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:25:32.624 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:25:32.624 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.624 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:32.624 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.624 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-10-16T11:25:32.625 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1l848oERAANMfrZz9JrlgNVh7ZFbtnaA==", 2023-10-16T11:25:32.625 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.625 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:25:32.625 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:25:32.625 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:25:32.626 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:25:32.626 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.626 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:32.626 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.626 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-10-16T11:25:32.626 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDRBi1lTAxTGRAAMptxnxy3Qit/cTCXie6CrQ==", 2023-10-16T11:25:32.627 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.627 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:25:32.627 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:25:32.627 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:25:32.627 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:25:32.627 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.628 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:32.628 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.628 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-10-16T11:25:32.628 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lPuJxHRAABAnGPVCrkq0JR4Y6iFGZlw==", 2023-10-16T11:25:32.628 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.629 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:25:32.629 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T11:25:32.629 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:25:32.629 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.629 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:32.629 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:32.630 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-10-16T11:25:32.630 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lLxnKEBAA+a/pOHUp7Sg3m4weAust5w==", 2023-10-16T11:25:32.630 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:25:32.630 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:25:32.630 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T11:25:32.631 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:25:32.631 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.631 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:32.631 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-10-16T11:25:32.631 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:25:32.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.597+0000 7fbfe4a15640 1 -- 172.21.15.84:0/891736973 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fbfc006b0b0 msgr2=0x7fbfc006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:32.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.597+0000 7fbfe4a15640 1 --2- 172.21.15.84:0/891736973 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fbfc006b0b0 0x7fbfc006d570 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0006fd0 tx=0x7fbfd0008040 comp rx=0 tx=0).stop 2023-10-16T11:25:32.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.597+0000 7fbfe4a15640 1 -- 172.21.15.84:0/891736973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe015ae80 msgr2=0x7fbfe00ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:32.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.597+0000 7fbfe4a15640 1 --2- 172.21.15.84:0/891736973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe015ae80 0x7fbfe00ca880 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800c780 tx=0x7fbfd800cc40 comp rx=0 tx=0).stop 2023-10-16T11:25:32.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.597+0000 7fbfe4a15640 1 -- 172.21.15.84:0/891736973 shutdown_connections 2023-10-16T11:25:32.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.597+0000 7fbfe4a15640 1 --2- 172.21.15.84:0/891736973 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fbfc006b0b0 0x7fbfc006d570 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.597+0000 7fbfe4a15640 1 --2- 172.21.15.84:0/891736973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe015ae80 0x7fbfe00ca880 unknown :-1 s=CLOSED pgs=2898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.597+0000 7fbfe4a15640 1 --2- 172.21.15.84:0/891736973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfe015aaa0 0x7fbfe00ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.597+0000 7fbfe4a15640 1 --2- 172.21.15.84:0/891736973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfe00ccaa0 0x7fbfe00b26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.597+0000 7fbfe4a15640 1 -- 172.21.15.84:0/891736973 >> 172.21.15.84:0/891736973 conn(0x7fbfe00cb410 msgr2=0x7fbfe006cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:32.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.597+0000 7fbfe4a15640 1 -- 172.21.15.84:0/891736973 shutdown_connections 2023-10-16T11:25:32.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.597+0000 7fbfe4a15640 1 -- 172.21.15.84:0/891736973 wait complete. 2023-10-16T11:25:32.636 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:25:32.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.847+0000 7f83ed5cc640 1 Processor -- start 2023-10-16T11:25:32.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.847+0000 7f83ed5cc640 1 -- start start 2023-10-16T11:25:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.849+0000 7f83ed5cc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f83e815aaa0 0x7f83e815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.849+0000 7f83ed5cc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83e80ce390 0x7f83e8070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.849+0000 7f83ed5cc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e815ed60 con 0x7f83e815ae80 2023-10-16T11:25:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.849+0000 7f83ed5cc640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e806b810 con 0x7f83e80ce390 2023-10-16T11:25:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.849+0000 7f83ed5cc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e80707e0 con 0x7f83e815aaa0 2023-10-16T11:25:32.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.849+0000 7f83e67fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f83e815ae80 0x7f83e80cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56042/0 (socket says 172.21.15.84:56042) 2023-10-16T11:25:32.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.849+0000 7f83e77fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83e80ce390 0x7f83e8070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:32.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.849+0000 7f83e67fc640 1 -- 172.21.15.84:0/1556436630 learned_addr learned my addr 172.21.15.84:0/1556436630 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:32.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.849+0000 7f83e6ffd640 1 --2- 172.21.15.84:0/1556436630 >> v2:172.21.15.157:3300/0 conn(0x7f83e815aaa0 0x7f83e815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:32.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.849+0000 7f83e77fe640 1 -- 172.21.15.84:0/1556436630 >> v2:172.21.15.157:3300/0 conn(0x7f83e815aaa0 msgr2=0x7f83e815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:32.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.849+0000 7f83e77fe640 1 --2- 172.21.15.84:0/1556436630 >> v2:172.21.15.157:3300/0 conn(0x7f83e815aaa0 0x7f83e815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.849+0000 7f83e77fe640 1 -- 172.21.15.84:0/1556436630 >> v1:172.21.15.84:6789/0 conn(0x7f83e815ae80 legacy=0x7f83e80cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.849+0000 7f83e77fe640 1 -- 172.21.15.84:0/1556436630 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e815ef90 con 0x7f83e80ce390 2023-10-16T11:25:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.851+0000 7f83e77fe640 1 --2- 172.21.15.84:0/1556436630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e80ce390 0x7f83e8070f50 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7f83e80cd990 tx=0x7f83e002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1aca1e052bfc1e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.851+0000 7f83e5ffb640 1 -- 172.21.15.84:0/1556436630 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e002cc70 con 0x7f83e80ce390 2023-10-16T11:25:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.851+0000 7f83e5ffb640 1 -- 172.21.15.84:0/1556436630 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e002cdd0 con 0x7f83e80ce390 2023-10-16T11:25:32.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.851+0000 7f83e5ffb640 1 -- 172.21.15.84:0/1556436630 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0034660 con 0x7f83e80ce390 2023-10-16T11:25:32.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.851+0000 7f83ed5cc640 1 -- 172.21.15.84:0/1556436630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e80ce390 msgr2=0x7f83e8070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:32.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.851+0000 7f83ed5cc640 1 --2- 172.21.15.84:0/1556436630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e80ce390 0x7f83e8070f50 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7f83e80cd990 tx=0x7f83e002bec0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.852+0000 7f83ed5cc640 1 -- 172.21.15.84:0/1556436630 shutdown_connections 2023-10-16T11:25:32.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.852+0000 7f83ed5cc640 1 --2- 172.21.15.84:0/1556436630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e80ce390 0x7f83e8070f50 unknown :-1 s=CLOSED pgs=2900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.852+0000 7f83ed5cc640 1 --2- 172.21.15.84:0/1556436630 >> v2:172.21.15.157:3300/0 conn(0x7f83e815aaa0 0x7f83e815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.852+0000 7f83ed5cc640 1 -- 172.21.15.84:0/1556436630 >> 172.21.15.84:0/1556436630 conn(0x7f83e80cb410 msgr2=0x7f83e80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:32.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.852+0000 7f83ed5cc640 1 -- 172.21.15.84:0/1556436630 shutdown_connections 2023-10-16T11:25:32.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.852+0000 7f83ed5cc640 1 -- 172.21.15.84:0/1556436630 wait complete. 2023-10-16T11:25:32.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.854+0000 7f83ed5cc640 1 Processor -- start 2023-10-16T11:25:32.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.854+0000 7f83ed5cc640 1 -- start start 2023-10-16T11:25:32.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.855+0000 7f83ed5cc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83e80ce390 0x7f83e814c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:32.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.855+0000 7f83ed5cc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e815aaa0 0x7f83e815c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:32.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.855+0000 7f83ed5cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83e815ae80 0x7f83e80c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:32.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.855+0000 7f83ed5cc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e8158e30 con 0x7f83e815aaa0 2023-10-16T11:25:32.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.855+0000 7f83ed5cc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e8158500 con 0x7f83e815ae80 2023-10-16T11:25:32.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.855+0000 7f83ed5cc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e814c800 con 0x7f83e80ce390 2023-10-16T11:25:32.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.855+0000 7f83e67fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e815aaa0 0x7f83e815c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:32.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.855+0000 7f83e67fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e815aaa0 0x7f83e815c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55766/0 (socket says 172.21.15.84:55766) 2023-10-16T11:25:32.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.855+0000 7f83e67fc640 1 -- 172.21.15.84:0/1570344204 learned_addr learned my addr 172.21.15.84:0/1570344204 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:32.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.856+0000 7f83e6ffd640 1 --2- 172.21.15.84:0/1570344204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83e80ce390 0x7f83e814c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:32.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.856+0000 7f83e77fe640 1 --2- 172.21.15.84:0/1570344204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83e815ae80 0x7f83e80c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:32.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.856+0000 7f83e67fc640 1 -- 172.21.15.84:0/1570344204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83e80ce390 msgr2=0x7f83e814c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:32.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.856+0000 7f83e67fc640 1 --2- 172.21.15.84:0/1570344204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83e80ce390 0x7f83e814c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.856+0000 7f83e67fc640 1 -- 172.21.15.84:0/1570344204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83e815ae80 msgr2=0x7f83e80c34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:32.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.856+0000 7f83e67fc640 1 --2- 172.21.15.84:0/1570344204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83e815ae80 0x7f83e80c34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:32.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.856+0000 7f83e67fc640 1 -- 172.21.15.84:0/1570344204 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e0009a70 con 0x7f83e815aaa0 2023-10-16T11:25:32.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.856+0000 7f83e77fe640 1 --2- 172.21.15.84:0/1570344204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83e815ae80 0x7f83e80c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:32.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.856+0000 7f83e67fc640 1 --2- 172.21.15.84:0/1570344204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e815aaa0 0x7f83e815c240 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00c3f0 tx=0x7f83dc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:32.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.856+0000 7f83cf7fe640 1 -- 172.21.15.84:0/1570344204 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc005ce0 con 0x7f83e815aaa0 2023-10-16T11:25:32.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.856+0000 7f83cf7fe640 1 -- 172.21.15.84:0/1570344204 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83dc005e40 con 0x7f83e815aaa0 2023-10-16T11:25:32.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.856+0000 7f83ed5cc640 1 -- 172.21.15.84:0/1570344204 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e815c780 con 0x7f83e815aaa0 2023-10-16T11:25:32.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.856+0000 7f83cf7fe640 1 -- 172.21.15.84:0/1570344204 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc0072d0 con 0x7f83e815aaa0 2023-10-16T11:25:32.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.857+0000 7f83ed5cc640 1 -- 172.21.15.84:0/1570344204 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e8158a50 con 0x7f83e815aaa0 2023-10-16T11:25:32.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.858+0000 7f83ed5cc640 1 -- 172.21.15.84:0/1570344204 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e804e160 con 0x7f83e815aaa0 2023-10-16T11:25:32.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.859+0000 7f83cf7fe640 1 -- 172.21.15.84:0/1570344204 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f83dc007430 con 0x7f83e815aaa0 2023-10-16T11:25:32.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.859+0000 7f83cf7fe640 1 --2- 172.21.15.84:0/1570344204 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f83d406b0b0 0x7f83d406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:32.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.860+0000 7f83cf7fe640 1 -- 172.21.15.84:0/1570344204 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f83dc088b50 con 0x7f83e815aaa0 2023-10-16T11:25:32.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.860+0000 7f83e6ffd640 1 --2- 172.21.15.84:0/1570344204 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f83d406b0b0 0x7f83d406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:32.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.861+0000 7f83e6ffd640 1 --2- 172.21.15.84:0/1570344204 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f83d406b0b0 0x7f83d406d570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f83d8006fd0 tx=0x7f83d8008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:32.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:32.865+0000 7f83cf7fe640 1 -- 172.21.15.84:0/1570344204 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f83dc053c70 con 0x7f83e815aaa0 2023-10-16T11:25:33.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.058+0000 7f83ed5cc640 1 -- 172.21.15.84:0/1570344204 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83e80cbca0 con 0x7f83e815aaa0 2023-10-16T11:25:33.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.060+0000 7f83cf7fe640 1 -- 172.21.15.84:0/1570344204 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v233) v1 ==== 74+0+12887 (secure 0 0 0) 0x7f83dc017090 con 0x7f83e815aaa0 2023-10-16T11:25:33.062 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:33.062 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":233,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:25:32.107737+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:25:33.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.062+0000 7f83ed5cc640 1 -- 172.21.15.84:0/1570344204 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f83d406b0b0 msgr2=0x7f83d406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:33.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.062+0000 7f83ed5cc640 1 --2- 172.21.15.84:0/1570344204 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f83d406b0b0 0x7f83d406d570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f83d8006fd0 tx=0x7f83d8008040 comp rx=0 tx=0).stop 2023-10-16T11:25:33.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.062+0000 7f83ed5cc640 1 -- 172.21.15.84:0/1570344204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e815aaa0 msgr2=0x7f83e815c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:33.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.062+0000 7f83ed5cc640 1 --2- 172.21.15.84:0/1570344204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e815aaa0 0x7f83e815c240 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00c3f0 tx=0x7f83dc00c8b0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.062+0000 7f83ed5cc640 1 -- 172.21.15.84:0/1570344204 shutdown_connections 2023-10-16T11:25:33.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.062+0000 7f83ed5cc640 1 --2- 172.21.15.84:0/1570344204 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f83d406b0b0 0x7f83d406d570 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.062+0000 7f83ed5cc640 1 --2- 172.21.15.84:0/1570344204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83e815ae80 0x7f83e80c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.062+0000 7f83ed5cc640 1 --2- 172.21.15.84:0/1570344204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83e815aaa0 0x7f83e815c240 unknown :-1 s=CLOSED pgs=2901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.062+0000 7f83ed5cc640 1 --2- 172.21.15.84:0/1570344204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83e80ce390 0x7f83e814c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.062+0000 7f83ed5cc640 1 -- 172.21.15.84:0/1570344204 >> 172.21.15.84:0/1570344204 conn(0x7f83e80cb410 msgr2=0x7f83e80acad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:33.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.062+0000 7f83ed5cc640 1 -- 172.21.15.84:0/1570344204 shutdown_connections 2023-10-16T11:25:33.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.062+0000 7f83ed5cc640 1 -- 172.21.15.84:0/1570344204 wait complete. 2023-10-16T11:25:33.082 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:25:33.082 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:25:33.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.273+0000 7f96b5edc640 1 Processor -- start 2023-10-16T11:25:33.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.273+0000 7f96b5edc640 1 -- start start 2023-10-16T11:25:33.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.275+0000 7f96b5edc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96b00b3ec0 0x7f96b00cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:33.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.275+0000 7f96b5edc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f96b00b43c0 0x7f96b00cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:33.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.275+0000 7f96b5edc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b006be30 con 0x7f96b00b0ea0 2023-10-16T11:25:33.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.275+0000 7f96b5edc640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b006ca00 con 0x7f96b00b43c0 2023-10-16T11:25:33.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.275+0000 7f96b5edc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00ada10 con 0x7f96b00b3ec0 2023-10-16T11:25:33.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.275+0000 7f96b56db640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f96b00b43c0 0x7f96b00cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:33.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.275+0000 7f96b56db640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f96b00b43c0 0x7f96b00cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55782/0 (socket says 172.21.15.84:55782) 2023-10-16T11:25:33.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.275+0000 7f96b56db640 1 -- 172.21.15.84:0/3133244628 learned_addr learned my addr 172.21.15.84:0/3133244628 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:33.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.275+0000 7f96b56db640 1 -- 172.21.15.84:0/3133244628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96b00b3ec0 msgr2=0x7f96b00cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:33.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.275+0000 7f96b4eda640 1 --2- 172.21.15.84:0/3133244628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96b00b3ec0 0x7f96b00cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:33.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.275+0000 7f96b56db640 1 --2- 172.21.15.84:0/3133244628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96b00b3ec0 0x7f96b00cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.275+0000 7f96b56db640 1 -- 172.21.15.84:0/3133244628 >> v1:172.21.15.84:6789/0 conn(0x7f96b00b0ea0 legacy=0x7f96b015a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:33.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.275+0000 7f96b56db640 1 -- 172.21.15.84:0/3133244628 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b006c060 con 0x7f96b00b43c0 2023-10-16T11:25:33.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.276+0000 7f96b56db640 1 --2- 172.21.15.84:0/3133244628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96b00b43c0 0x7f96b00cd7a0 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009870 tx=0x7f96a802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96dd6e7aeb8d46d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:33.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.277+0000 7f96a77fe640 1 -- 172.21.15.84:0/3133244628 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8031640 con 0x7f96b00b43c0 2023-10-16T11:25:33.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.277+0000 7f96a77fe640 1 -- 172.21.15.84:0/3133244628 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a80317a0 con 0x7f96b00b43c0 2023-10-16T11:25:33.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.277+0000 7f96a77fe640 1 -- 172.21.15.84:0/3133244628 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8031b30 con 0x7f96b00b43c0 2023-10-16T11:25:33.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.277+0000 7f96b5edc640 1 -- 172.21.15.84:0/3133244628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96b00b43c0 msgr2=0x7f96b00cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:33.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.277+0000 7f96b5edc640 1 --2- 172.21.15.84:0/3133244628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96b00b43c0 0x7f96b00cd7a0 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009870 tx=0x7f96a802b8f0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.277+0000 7f96b5edc640 1 -- 172.21.15.84:0/3133244628 shutdown_connections 2023-10-16T11:25:33.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.277+0000 7f96b5edc640 1 --2- 172.21.15.84:0/3133244628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96b00b43c0 0x7f96b00cd7a0 unknown :-1 s=CLOSED pgs=2903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.277+0000 7f96b5edc640 1 --2- 172.21.15.84:0/3133244628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96b00b3ec0 0x7f96b00cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.277+0000 7f96b5edc640 1 -- 172.21.15.84:0/3133244628 >> 172.21.15.84:0/3133244628 conn(0x7f96b00b0170 msgr2=0x7f96b00b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:33.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.277+0000 7f96b5edc640 1 -- 172.21.15.84:0/3133244628 shutdown_connections 2023-10-16T11:25:33.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.277+0000 7f96b5edc640 1 -- 172.21.15.84:0/3133244628 wait complete. 2023-10-16T11:25:33.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.279+0000 7f96b5edc640 1 Processor -- start 2023-10-16T11:25:33.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.279+0000 7f96b5edc640 1 -- start start 2023-10-16T11:25:33.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.280+0000 7f96b5edc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96b00b0ea0 0x7f96b015a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:33.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.280+0000 7f96b5edc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96b00b3ec0 0x7f96b01f0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:33.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.280+0000 7f96b5edc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96b00b43c0 0x7f96b00a8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:33.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.280+0000 7f96b5edc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00c5e90 con 0x7f96b00b0ea0 2023-10-16T11:25:33.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.280+0000 7f96b5edc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00acc70 con 0x7f96b00b43c0 2023-10-16T11:25:33.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.280+0000 7f96b5edc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00a8c90 con 0x7f96b00b3ec0 2023-10-16T11:25:33.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.280+0000 7f96b4eda640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96b00b0ea0 0x7f96b015a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:33.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.280+0000 7f96b4eda640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96b00b0ea0 0x7f96b015a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55792/0 (socket says 172.21.15.84:55792) 2023-10-16T11:25:33.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.280+0000 7f96b4eda640 1 -- 172.21.15.84:0/2361681849 learned_addr learned my addr 172.21.15.84:0/2361681849 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:33.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.281+0000 7f96b56db640 1 --2- 172.21.15.84:0/2361681849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96b00b43c0 0x7f96b00a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:33.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.281+0000 7f96b4eda640 1 -- 172.21.15.84:0/2361681849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96b00b3ec0 msgr2=0x7f96b01f0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:33.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.281+0000 7f96b4eda640 1 --2- 172.21.15.84:0/2361681849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96b00b3ec0 0x7f96b01f0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.281+0000 7f96b4eda640 1 -- 172.21.15.84:0/2361681849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96b00b43c0 msgr2=0x7f96b00a8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:33.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.281+0000 7f96b4eda640 1 --2- 172.21.15.84:0/2361681849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96b00b43c0 0x7f96b00a8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.281+0000 7f96b4eda640 1 -- 172.21.15.84:0/2361681849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a80094d0 con 0x7f96b00b0ea0 2023-10-16T11:25:33.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.281+0000 7f96b4eda640 1 --2- 172.21.15.84:0/2361681849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96b00b0ea0 0x7f96b015a790 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00ccd0 tx=0x7f96ac00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:33.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.281+0000 7f96a57fa640 1 -- 172.21.15.84:0/2361681849 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac00ce80 con 0x7f96b00b0ea0 2023-10-16T11:25:33.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.281+0000 7f96a57fa640 1 -- 172.21.15.84:0/2361681849 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ac00ae80 con 0x7f96b00b0ea0 2023-10-16T11:25:33.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.281+0000 7f96a57fa640 1 -- 172.21.15.84:0/2361681849 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac003b90 con 0x7f96b00b0ea0 2023-10-16T11:25:33.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.281+0000 7f96b5edc640 1 -- 172.21.15.84:0/2361681849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b00c69a0 con 0x7f96b00b0ea0 2023-10-16T11:25:33.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.282+0000 7f96b5edc640 1 -- 172.21.15.84:0/2361681849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b00c6070 con 0x7f96b00b0ea0 2023-10-16T11:25:33.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.283+0000 7f96b5edc640 1 -- 172.21.15.84:0/2361681849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b004e9b0 con 0x7f96b00b0ea0 2023-10-16T11:25:33.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.284+0000 7f96a57fa640 1 -- 172.21.15.84:0/2361681849 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f96ac003cf0 con 0x7f96b00b0ea0 2023-10-16T11:25:33.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.284+0000 7f96a57fa640 1 --2- 172.21.15.84:0/2361681849 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f969c06b0b0 0x7f969c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:33.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.285+0000 7f96a57fa640 1 -- 172.21.15.84:0/2361681849 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f96ac089bb0 con 0x7f96b00b0ea0 2023-10-16T11:25:33.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.285+0000 7f96a7fff640 1 --2- 172.21.15.84:0/2361681849 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f969c06b0b0 0x7f969c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:33.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.285+0000 7f96a7fff640 1 --2- 172.21.15.84:0/2361681849 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f969c06b0b0 0x7f969c06d570 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f96a0000f60 tx=0x7f96a00079b0 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:33.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.289+0000 7f96a57fa640 1 -- 172.21.15.84:0/2361681849 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f96ac054b10 con 0x7f96b00b0ea0 2023-10-16T11:25:33.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.483+0000 7f96b5edc640 1 -- 172.21.15.84:0/2361681849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f96b006abd0 con 0x7f96b00b0ea0 2023-10-16T11:25:33.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.484+0000 7f96a57fa640 1 -- 172.21.15.84:0/2361681849 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v233) v1 ==== 93+0+10 (secure 0 0 0) 0x7f96ac01d070 con 0x7f96b00b0ea0 2023-10-16T11:25:33.487 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:25:33.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.487+0000 7f96b5edc640 1 -- 172.21.15.84:0/2361681849 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f969c06b0b0 msgr2=0x7f969c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:33.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.487+0000 7f96b5edc640 1 --2- 172.21.15.84:0/2361681849 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f969c06b0b0 0x7f969c06d570 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f96a0000f60 tx=0x7f96a00079b0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.487+0000 7f96b5edc640 1 -- 172.21.15.84:0/2361681849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96b00b0ea0 msgr2=0x7f96b015a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:33.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.487+0000 7f96b5edc640 1 --2- 172.21.15.84:0/2361681849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96b00b0ea0 0x7f96b015a790 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00ccd0 tx=0x7f96ac00a5c0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.487+0000 7f96b5edc640 1 -- 172.21.15.84:0/2361681849 shutdown_connections 2023-10-16T11:25:33.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.487+0000 7f96b5edc640 1 --2- 172.21.15.84:0/2361681849 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f969c06b0b0 0x7f969c06d570 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.487+0000 7f96b5edc640 1 --2- 172.21.15.84:0/2361681849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96b00b43c0 0x7f96b00a8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.487+0000 7f96b5edc640 1 --2- 172.21.15.84:0/2361681849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96b00b3ec0 0x7f96b01f0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.487+0000 7f96b5edc640 1 --2- 172.21.15.84:0/2361681849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96b00b0ea0 0x7f96b015a790 unknown :-1 s=CLOSED pgs=2904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.487+0000 7f96b5edc640 1 -- 172.21.15.84:0/2361681849 >> 172.21.15.84:0/2361681849 conn(0x7f96b00b0170 msgr2=0x7f96b00c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:33.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.487+0000 7f96b5edc640 1 -- 172.21.15.84:0/2361681849 shutdown_connections 2023-10-16T11:25:33.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.487+0000 7f96b5edc640 1 -- 172.21.15.84:0/2361681849 wait complete. 2023-10-16T11:25:33.508 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T11:25:33.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.721+0000 7f3ef1590640 1 Processor -- start 2023-10-16T11:25:33.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.721+0000 7f3ef1590640 1 -- start start 2023-10-16T11:25:33.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.723+0000 7f3ef1590640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3eec159b20 0x7f3eec06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:33.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.723+0000 7f3ef1590640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3eec159f00 0x7f3eec0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:33.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.723+0000 7f3ef1590640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eec06f7f0 con 0x7f3eec06c120 2023-10-16T11:25:33.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.723+0000 7f3ef1590640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eec06c5e0 con 0x7f3eec159f00 2023-10-16T11:25:33.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.723+0000 7f3ef1590640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec06c8c0 con 0x7f3eec159b20 2023-10-16T11:25:33.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.723+0000 7f3ee3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3eec159f00 0x7f3eec0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:33.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.723+0000 7f3ee3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3eec159f00 0x7f3eec0b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55804/0 (socket says 172.21.15.84:55804) 2023-10-16T11:25:33.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.723+0000 7f3ee3fff640 1 -- 172.21.15.84:0/1148000223 learned_addr learned my addr 172.21.15.84:0/1148000223 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:33.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.724+0000 7f3eeaffd640 1 --2- 172.21.15.84:0/1148000223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3eec159b20 0x7f3eec06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:33.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.724+0000 7f3ee3fff640 1 -- 172.21.15.84:0/1148000223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3eec159b20 msgr2=0x7f3eec06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:33.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.724+0000 7f3ee3fff640 1 --2- 172.21.15.84:0/1148000223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3eec159b20 0x7f3eec06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.724+0000 7f3ee3fff640 1 -- 172.21.15.84:0/1148000223 >> v1:172.21.15.157:6789/0 conn(0x7f3eec06c120 legacy=0x7f3eec0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:33.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.724+0000 7f3ee3fff640 1 -- 172.21.15.84:0/1148000223 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec06cba0 con 0x7f3eec159f00 2023-10-16T11:25:33.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.724+0000 7f3eeaffd640 1 --2- 172.21.15.84:0/1148000223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3eec159b20 0x7f3eec06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:33.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.725+0000 7f3ee3fff640 1 --2- 172.21.15.84:0/1148000223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eec159f00 0x7f3eec0b0410 secure :-1 s=READY pgs=2905 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009870 tx=0x7f3ed402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c582e41792915061 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:33.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.725+0000 7f3eea7fc640 1 -- 172.21.15.84:0/1148000223 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4031640 con 0x7f3eec159f00 2023-10-16T11:25:33.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.725+0000 7f3eea7fc640 1 -- 172.21.15.84:0/1148000223 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed40317a0 con 0x7f3eec159f00 2023-10-16T11:25:33.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.725+0000 7f3eea7fc640 1 -- 172.21.15.84:0/1148000223 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4031b30 con 0x7f3eec159f00 2023-10-16T11:25:33.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.726+0000 7f3ef1590640 1 -- 172.21.15.84:0/1148000223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eec159f00 msgr2=0x7f3eec0b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:33.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.726+0000 7f3ef1590640 1 --2- 172.21.15.84:0/1148000223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eec159f00 0x7f3eec0b0410 secure :-1 s=READY pgs=2905 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009870 tx=0x7f3ed402b8f0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.726+0000 7f3ef1590640 1 -- 172.21.15.84:0/1148000223 shutdown_connections 2023-10-16T11:25:33.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.726+0000 7f3ef1590640 1 --2- 172.21.15.84:0/1148000223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eec159f00 0x7f3eec0b0410 unknown :-1 s=CLOSED pgs=2905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.726+0000 7f3ef1590640 1 --2- 172.21.15.84:0/1148000223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3eec159b20 0x7f3eec06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.726+0000 7f3ef1590640 1 -- 172.21.15.84:0/1148000223 >> 172.21.15.84:0/1148000223 conn(0x7f3eec0cb410 msgr2=0x7f3eec0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:33.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.726+0000 7f3ef1590640 1 -- 172.21.15.84:0/1148000223 shutdown_connections 2023-10-16T11:25:33.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.726+0000 7f3ef1590640 1 -- 172.21.15.84:0/1148000223 wait complete. 2023-10-16T11:25:33.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.728+0000 7f3ef1590640 1 Processor -- start 2023-10-16T11:25:33.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.728+0000 7f3ef1590640 1 -- start start 2023-10-16T11:25:33.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.729+0000 7f3ef1590640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec06c120 0x7f3eec157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:33.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.729+0000 7f3ef1590640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3eec159b20 0x7f3eec0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:33.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.729+0000 7f3ef1590640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eec159f00 0x7f3eec15bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:33.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.729+0000 7f3ef1590640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec0b36d0 con 0x7f3eec159f00 2023-10-16T11:25:33.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.729+0000 7f3ef1590640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec0b2a50 con 0x7f3eec06c120 2023-10-16T11:25:33.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.729+0000 7f3ef1590640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec0b2d30 con 0x7f3eec159b20 2023-10-16T11:25:33.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.729+0000 7f3eeb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eec159f00 0x7f3eec15bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:33.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.729+0000 7f3eeb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eec159f00 0x7f3eec15bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55810/0 (socket says 172.21.15.84:55810) 2023-10-16T11:25:33.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.729+0000 7f3eeb7fe640 1 -- 172.21.15.84:0/1346243644 learned_addr learned my addr 172.21.15.84:0/1346243644 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:33.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.730+0000 7f3eeaffd640 1 --2- 172.21.15.84:0/1346243644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec06c120 0x7f3eec157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:33.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.730+0000 7f3ee3fff640 1 --2- 172.21.15.84:0/1346243644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3eec159b20 0x7f3eec0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:33.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.730+0000 7f3eeb7fe640 1 -- 172.21.15.84:0/1346243644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3eec159b20 msgr2=0x7f3eec0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:33.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.730+0000 7f3eeb7fe640 1 --2- 172.21.15.84:0/1346243644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3eec159b20 0x7f3eec0b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.730+0000 7f3eeb7fe640 1 -- 172.21.15.84:0/1346243644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec06c120 msgr2=0x7f3eec157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:33.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.730+0000 7f3eeb7fe640 1 --2- 172.21.15.84:0/1346243644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec06c120 0x7f3eec157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.730+0000 7f3eeb7fe640 1 -- 172.21.15.84:0/1346243644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed40094d0 con 0x7f3eec159f00 2023-10-16T11:25:33.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.730+0000 7f3eeb7fe640 1 --2- 172.21.15.84:0/1346243644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eec159f00 0x7f3eec15bde0 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400c6d0 tx=0x7f3ee400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:33.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.730+0000 7f3ee37fe640 1 -- 172.21.15.84:0/1346243644 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400a840 con 0x7f3eec159f00 2023-10-16T11:25:33.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.730+0000 7f3ee37fe640 1 -- 172.21.15.84:0/1346243644 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee400a9a0 con 0x7f3eec159f00 2023-10-16T11:25:33.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.730+0000 7f3ef1590640 1 -- 172.21.15.84:0/1346243644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eec0b3010 con 0x7f3eec159f00 2023-10-16T11:25:33.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.730+0000 7f3ee37fe640 1 -- 172.21.15.84:0/1346243644 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400ad90 con 0x7f3eec159f00 2023-10-16T11:25:33.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.731+0000 7f3ef1590640 1 -- 172.21.15.84:0/1346243644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eec157f90 con 0x7f3eec159f00 2023-10-16T11:25:33.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.732+0000 7f3ef1590640 1 -- 172.21.15.84:0/1346243644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eec0b3810 con 0x7f3eec159f00 2023-10-16T11:25:33.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.732+0000 7f3ee37fe640 1 -- 172.21.15.84:0/1346243644 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f3ee4012630 con 0x7f3eec159f00 2023-10-16T11:25:33.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.733+0000 7f3ee37fe640 1 --2- 172.21.15.84:0/1346243644 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f3ed006b0b0 0x7f3ed006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:33.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.733+0000 7f3ee37fe640 1 -- 172.21.15.84:0/1346243644 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f3ee4089e80 con 0x7f3eec159f00 2023-10-16T11:25:33.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.734+0000 7f3eeaffd640 1 --2- 172.21.15.84:0/1346243644 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f3ed006b0b0 0x7f3ed006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:33.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.734+0000 7f3eeaffd640 1 --2- 172.21.15.84:0/1346243644 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f3ed006b0b0 0x7f3ed006d570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00a990 tx=0x7f3edc008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:33.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.738+0000 7f3ee37fe640 1 -- 172.21.15.84:0/1346243644 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3eec0b3810 con 0x7f3eec159f00 2023-10-16T11:25:33.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.931+0000 7f3ef1590640 1 -- 172.21.15.84:0/1346243644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3eec0b3810 con 0x7f3eec159f00 2023-10-16T11:25:33.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.932+0000 7f3ee37fe640 1 -- 172.21.15.84:0/1346243644 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v233) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3eec0b3810 con 0x7f3eec159f00 2023-10-16T11:25:33.934 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:33.934 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T11:25:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-16T11:25:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-16T11:25:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:33.935 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T11:25:33.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.934+0000 7f3ef1590640 1 -- 172.21.15.84:0/1346243644 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f3ed006b0b0 msgr2=0x7f3ed006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:33.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.934+0000 7f3ef1590640 1 --2- 172.21.15.84:0/1346243644 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f3ed006b0b0 0x7f3ed006d570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00a990 tx=0x7f3edc008040 comp rx=0 tx=0).stop 2023-10-16T11:25:33.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.935+0000 7f3ef1590640 1 -- 172.21.15.84:0/1346243644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eec159f00 msgr2=0x7f3eec15bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:33.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.935+0000 7f3ef1590640 1 --2- 172.21.15.84:0/1346243644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eec159f00 0x7f3eec15bde0 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400c6d0 tx=0x7f3ee400cb90 comp rx=0 tx=0).stop 2023-10-16T11:25:33.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.935+0000 7f3ef1590640 1 -- 172.21.15.84:0/1346243644 shutdown_connections 2023-10-16T11:25:33.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.935+0000 7f3ef1590640 1 --2- 172.21.15.84:0/1346243644 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f3ed006b0b0 0x7f3ed006d570 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.935+0000 7f3ef1590640 1 --2- 172.21.15.84:0/1346243644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eec159f00 0x7f3eec15bde0 unknown :-1 s=CLOSED pgs=2906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.935+0000 7f3ef1590640 1 --2- 172.21.15.84:0/1346243644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3eec159b20 0x7f3eec0b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.935+0000 7f3ef1590640 1 --2- 172.21.15.84:0/1346243644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec06c120 0x7f3eec157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:33.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.935+0000 7f3ef1590640 1 -- 172.21.15.84:0/1346243644 >> 172.21.15.84:0/1346243644 conn(0x7f3eec0cb410 msgr2=0x7f3eec0b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:33.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.935+0000 7f3ef1590640 1 -- 172.21.15.84:0/1346243644 shutdown_connections 2023-10-16T11:25:33.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:33.935+0000 7f3ef1590640 1 -- 172.21.15.84:0/1346243644 wait complete. 2023-10-16T11:25:33.956 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T11:25:33.957 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T11:25:34.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.179+0000 7f6d3f983640 1 Processor -- start 2023-10-16T11:25:34.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.179+0000 7f6d3f983640 1 -- start start 2023-10-16T11:25:34.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.180+0000 7f6d3f983640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6d3815ac10 0x7f6d3806c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:34.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.181+0000 7f6d3f983640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d3815b310 con 0x7f6d380cac20 2023-10-16T11:25:34.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.181+0000 7f6d3f983640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d3806f640 con 0x7f6d380b0360 2023-10-16T11:25:34.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.181+0000 7f6d3f983640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d3815b5b0 con 0x7f6d3815ac10 2023-10-16T11:25:34.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.181+0000 7f6d3df20640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6d380cac20 0x7f6d3815e7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56068/0 (socket says 172.21.15.84:56068) 2023-10-16T11:25:34.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.181+0000 7f6d3df20640 1 -- 172.21.15.84:0/332478317 learned_addr learned my addr 172.21.15.84:0/332478317 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:34.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.181+0000 7f6d3d71f640 1 --2- 172.21.15.84:0/332478317 >> v2:172.21.15.84:3300/0 conn(0x7f6d3815ac10 0x7f6d3806c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:34.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.181+0000 7f6d2ffff640 1 -- 172.21.15.84:0/332478317 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044725715 0 0) 0x7f6d3815b310 con 0x7f6d380cac20 2023-10-16T11:25:34.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.181+0000 7f6d2ffff640 1 -- 172.21.15.84:0/332478317 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d24003520 con 0x7f6d380cac20 2023-10-16T11:25:34.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.182+0000 7f6d3d71f640 1 -- 172.21.15.84:0/332478317 >> v1:172.21.15.157:6789/0 conn(0x7f6d380b0360 legacy=0x7f6d380ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:34.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.182+0000 7f6d3d71f640 1 -- 172.21.15.84:0/332478317 >> v1:172.21.15.84:6789/0 conn(0x7f6d380cac20 legacy=0x7f6d3815e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:34.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.182+0000 7f6d2ffff640 1 -- 172.21.15.84:0/332478317 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2687850351 0 0) 0x7f6d24003520 con 0x7f6d380cac20 2023-10-16T11:25:34.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.182+0000 7f6d3d71f640 1 -- 172.21.15.84:0/332478317 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3815b890 con 0x7f6d3815ac10 2023-10-16T11:25:34.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.183+0000 7f6d3d71f640 1 --2- 172.21.15.84:0/332478317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d3815ac10 0x7f6d3806c730 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7f6d340097a0 tx=0x7f6d3402b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eed3050904b34db server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:34.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.183+0000 7f6d2ffff640 1 -- 172.21.15.84:0/332478317 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d300019f0 con 0x7f6d3815ac10 2023-10-16T11:25:34.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.183+0000 7f6d2ffff640 1 -- 172.21.15.84:0/332478317 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3402e050 con 0x7f6d3815ac10 2023-10-16T11:25:34.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.183+0000 7f6d2ffff640 1 -- 172.21.15.84:0/332478317 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34030710 con 0x7f6d3815ac10 2023-10-16T11:25:34.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.184+0000 7f6d3f983640 1 -- 172.21.15.84:0/332478317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d3815ac10 msgr2=0x7f6d3806c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:34.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.184+0000 7f6d3f983640 1 --2- 172.21.15.84:0/332478317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d3815ac10 0x7f6d3806c730 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7f6d340097a0 tx=0x7f6d3402b790 comp rx=0 tx=0).stop 2023-10-16T11:25:34.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.184+0000 7f6d3f983640 1 -- 172.21.15.84:0/332478317 shutdown_connections 2023-10-16T11:25:34.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.184+0000 7f6d3f983640 1 --2- 172.21.15.84:0/332478317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d3815ac10 0x7f6d3806c730 unknown :-1 s=CLOSED pgs=2908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:34.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.184+0000 7f6d3f983640 1 -- 172.21.15.84:0/332478317 >> 172.21.15.84:0/332478317 conn(0x7f6d380cb360 msgr2=0x7f6d380cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:34.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.184+0000 7f6d3f983640 1 -- 172.21.15.84:0/332478317 shutdown_connections 2023-10-16T11:25:34.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.184+0000 7f6d3f983640 1 -- 172.21.15.84:0/332478317 wait complete. 2023-10-16T11:25:34.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.186+0000 7f6d3f983640 1 Processor -- start 2023-10-16T11:25:34.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.186+0000 7f6d3f983640 1 -- start start 2023-10-16T11:25:34.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.187+0000 7f6d3f983640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d380b0360 0x7f6d38159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:34.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.187+0000 7f6d3f983640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d380cac20 0x7f6d381f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:34.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.187+0000 7f6d3f983640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d3815ac10 0x7f6d380a8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:34.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.187+0000 7f6d3f983640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d38159680 con 0x7f6d380b0360 2023-10-16T11:25:34.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.187+0000 7f6d3f983640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d381f0e00 con 0x7f6d380cac20 2023-10-16T11:25:34.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.187+0000 7f6d3f983640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d381f10e0 con 0x7f6d3815ac10 2023-10-16T11:25:34.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.187+0000 7f6d3d71f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d380b0360 0x7f6d38159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:34.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.187+0000 7f6d3d71f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d380b0360 0x7f6d38159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55824/0 (socket says 172.21.15.84:55824) 2023-10-16T11:25:34.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.187+0000 7f6d3d71f640 1 -- 172.21.15.84:0/2279813272 learned_addr learned my addr 172.21.15.84:0/2279813272 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:34.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.188+0000 7f6d3d71f640 1 -- 172.21.15.84:0/2279813272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d3815ac10 msgr2=0x7f6d380a8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:34.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.188+0000 7f6d3d71f640 1 --2- 172.21.15.84:0/2279813272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d3815ac10 0x7f6d380a8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:34.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.188+0000 7f6d3d71f640 1 -- 172.21.15.84:0/2279813272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d380cac20 msgr2=0x7f6d381f0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:34.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.188+0000 7f6d3d71f640 1 --2- 172.21.15.84:0/2279813272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d380cac20 0x7f6d381f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:34.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.188+0000 7f6d3d71f640 1 -- 172.21.15.84:0/2279813272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d34009400 con 0x7f6d380b0360 2023-10-16T11:25:34.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.188+0000 7f6d3d71f640 1 --2- 172.21.15.84:0/2279813272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d380b0360 0x7f6d38159140 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7f6d34000c00 tx=0x7f6d340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:34.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.188+0000 7f6d2dffb640 1 -- 172.21.15.84:0/2279813272 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3403d890 con 0x7f6d380b0360 2023-10-16T11:25:34.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.188+0000 7f6d2dffb640 1 -- 172.21.15.84:0/2279813272 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3403da10 con 0x7f6d380b0360 2023-10-16T11:25:34.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.188+0000 7f6d2dffb640 1 -- 172.21.15.84:0/2279813272 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3403ddc0 con 0x7f6d380b0360 2023-10-16T11:25:34.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.188+0000 7f6d3f983640 1 -- 172.21.15.84:0/2279813272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d381f13c0 con 0x7f6d380b0360 2023-10-16T11:25:34.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.188+0000 7f6d3f983640 1 -- 172.21.15.84:0/2279813272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d380a8760 con 0x7f6d380b0360 2023-10-16T11:25:34.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.189+0000 7f6d3f983640 1 -- 172.21.15.84:0/2279813272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d3804e160 con 0x7f6d380b0360 2023-10-16T11:25:34.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.191+0000 7f6d2dffb640 1 -- 172.21.15.84:0/2279813272 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f6d34039040 con 0x7f6d380b0360 2023-10-16T11:25:34.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.192+0000 7f6d2dffb640 1 --2- 172.21.15.84:0/2279813272 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f6d1c06b0e0 0x7f6d1c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:34.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.192+0000 7f6d2dffb640 1 -- 172.21.15.84:0/2279813272 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f6d340ab480 con 0x7f6d380b0360 2023-10-16T11:25:34.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.192+0000 7f6d3cf1e640 1 --2- 172.21.15.84:0/2279813272 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f6d1c06b0e0 0x7f6d1c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:34.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.193+0000 7f6d3cf1e640 1 --2- 172.21.15.84:0/2279813272 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f6d1c06b0e0 0x7f6d1c06d5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6d28000f60 tx=0x7f6d280079d0 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:34.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.196+0000 7f6d2dffb640 1 -- 172.21.15.84:0/2279813272 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6d34076420 con 0x7f6d380b0360 2023-10-16T11:25:34.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:34.387+0000 7f6d3f983640 1 -- 172.21.15.84:0/2279813272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6d3806c2e0 con 0x7f6d380b0360 2023-10-16T11:25:34.466 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:25:34.474 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:25:35.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.115+0000 7f6d2dffb640 1 -- 172.21.15.84:0/2279813272 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v234) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6d34079a40 con 0x7f6d380b0360 2023-10-16T11:25:35.118 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-16T11:25:35.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.119+0000 7f6d3f983640 1 -- 172.21.15.84:0/2279813272 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f6d1c06b0e0 msgr2=0x7f6d1c06d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:35.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.119+0000 7f6d3f983640 1 --2- 172.21.15.84:0/2279813272 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f6d1c06b0e0 0x7f6d1c06d5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6d28000f60 tx=0x7f6d280079d0 comp rx=0 tx=0).stop 2023-10-16T11:25:35.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.119+0000 7f6d3f983640 1 -- 172.21.15.84:0/2279813272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d380b0360 msgr2=0x7f6d38159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:35.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.119+0000 7f6d3f983640 1 --2- 172.21.15.84:0/2279813272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d380b0360 0x7f6d38159140 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7f6d34000c00 tx=0x7f6d340046c0 comp rx=0 tx=0).stop 2023-10-16T11:25:35.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.121+0000 7f6d3f983640 1 -- 172.21.15.84:0/2279813272 shutdown_connections 2023-10-16T11:25:35.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.121+0000 7f6d3f983640 1 --2- 172.21.15.84:0/2279813272 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f6d1c06b0e0 0x7f6d1c06d5a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:35.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.121+0000 7f6d3f983640 1 --2- 172.21.15.84:0/2279813272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d3815ac10 0x7f6d380a8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:35.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.121+0000 7f6d3f983640 1 --2- 172.21.15.84:0/2279813272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d380cac20 0x7f6d381f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:35.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.121+0000 7f6d3f983640 1 --2- 172.21.15.84:0/2279813272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d380b0360 0x7f6d38159140 unknown :-1 s=CLOSED pgs=2909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:35.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.121+0000 7f6d3f983640 1 -- 172.21.15.84:0/2279813272 >> 172.21.15.84:0/2279813272 conn(0x7f6d380cb360 msgr2=0x7f6d380b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:35.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.122+0000 7f6d3f983640 1 -- 172.21.15.84:0/2279813272 shutdown_connections 2023-10-16T11:25:35.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.122+0000 7f6d3f983640 1 -- 172.21.15.84:0/2279813272 wait complete. 2023-10-16T11:25:35.150 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T11:25:35.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.364+0000 7fde8fce7640 1 Processor -- start 2023-10-16T11:25:35.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.364+0000 7fde8fce7640 1 -- start start 2023-10-16T11:25:35.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.366+0000 7fde8fce7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fde8815aaa0 0x7fde8815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:35.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.366+0000 7fde8fce7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8815ae80 0x7fde880cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:35.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.366+0000 7fde8fce7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde880707e0 con 0x7fde880ccaa0 2023-10-16T11:25:35.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.366+0000 7fde8fce7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8806b7a0 con 0x7fde8815aaa0 2023-10-16T11:25:35.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.366+0000 7fde8fce7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8815ef20 con 0x7fde8815ae80 2023-10-16T11:25:35.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.366+0000 7fde8da83640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fde8815aaa0 0x7fde8815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:35.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.366+0000 7fde8e284640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fde880ccaa0 0x7fde88070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56072/0 (socket says 172.21.15.84:56072) 2023-10-16T11:25:35.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.366+0000 7fde8e284640 1 -- 172.21.15.84:0/3405409473 learned_addr learned my addr 172.21.15.84:0/3405409473 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:35.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.367+0000 7fde8d282640 1 --2- 172.21.15.84:0/3405409473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8815ae80 0x7fde880cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:35.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.367+0000 7fde8da83640 1 -- 172.21.15.84:0/3405409473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8815ae80 msgr2=0x7fde880cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:35.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.367+0000 7fde8da83640 1 --2- 172.21.15.84:0/3405409473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8815ae80 0x7fde880cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:35.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.367+0000 7fde8da83640 1 -- 172.21.15.84:0/3405409473 >> v1:172.21.15.84:6789/0 conn(0x7fde880ccaa0 legacy=0x7fde88070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:35.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.367+0000 7fde8da83640 1 -- 172.21.15.84:0/3405409473 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8815f200 con 0x7fde8815aaa0 2023-10-16T11:25:35.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.367+0000 7fde8ca81640 1 -- 172.21.15.84:0/3405409473 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 589083749 0 0) 0x7fde880707e0 con 0x7fde880ccaa0 2023-10-16T11:25:35.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.368+0000 7fde8da83640 1 --2- 172.21.15.84:0/3405409473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde8815aaa0 0x7fde8815e820 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7fde840097a0 tx=0x7fde8402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad90773f18efe6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:35.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.368+0000 7fde8ca81640 1 -- 172.21.15.84:0/3405409473 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84031780 con 0x7fde8815aaa0 2023-10-16T11:25:35.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.368+0000 7fde8ca81640 1 -- 172.21.15.84:0/3405409473 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde840318e0 con 0x7fde8815aaa0 2023-10-16T11:25:35.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.368+0000 7fde8ca81640 1 -- 172.21.15.84:0/3405409473 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84031c70 con 0x7fde8815aaa0 2023-10-16T11:25:35.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.369+0000 7fde8fce7640 1 -- 172.21.15.84:0/3405409473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde8815aaa0 msgr2=0x7fde8815e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:35.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.369+0000 7fde8fce7640 1 --2- 172.21.15.84:0/3405409473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde8815aaa0 0x7fde8815e820 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7fde840097a0 tx=0x7fde8402b820 comp rx=0 tx=0).stop 2023-10-16T11:25:35.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.369+0000 7fde8fce7640 1 -- 172.21.15.84:0/3405409473 shutdown_connections 2023-10-16T11:25:35.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.369+0000 7fde8fce7640 1 --2- 172.21.15.84:0/3405409473 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8815ae80 0x7fde880cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:35.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.369+0000 7fde8fce7640 1 --2- 172.21.15.84:0/3405409473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde8815aaa0 0x7fde8815e820 unknown :-1 s=CLOSED pgs=2911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:35.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.369+0000 7fde8fce7640 1 -- 172.21.15.84:0/3405409473 >> 172.21.15.84:0/3405409473 conn(0x7fde880cb410 msgr2=0x7fde880cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:35.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.369+0000 7fde8fce7640 1 -- 172.21.15.84:0/3405409473 shutdown_connections 2023-10-16T11:25:35.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.369+0000 7fde8fce7640 1 -- 172.21.15.84:0/3405409473 wait complete. 2023-10-16T11:25:35.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.371+0000 7fde8fce7640 1 Processor -- start 2023-10-16T11:25:35.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.371+0000 7fde8fce7640 1 -- start start 2023-10-16T11:25:35.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.372+0000 7fde8fce7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde880ccaa0 0x7fde88159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:35.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.372+0000 7fde8fce7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde8815aaa0 0x7fde880c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:35.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.372+0000 7fde8fce7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8815ae80 0x7fde88158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:35.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.372+0000 7fde8fce7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde880c3f10 con 0x7fde880ccaa0 2023-10-16T11:25:35.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.372+0000 7fde8fce7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde880c41f0 con 0x7fde8815aaa0 2023-10-16T11:25:35.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.372+0000 7fde8fce7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde880c44d0 con 0x7fde8815ae80 2023-10-16T11:25:35.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.372+0000 7fde8da83640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde880ccaa0 0x7fde88159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:35.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.372+0000 7fde8da83640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde880ccaa0 0x7fde88159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55828/0 (socket says 172.21.15.84:55828) 2023-10-16T11:25:35.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.372+0000 7fde8da83640 1 -- 172.21.15.84:0/1590264830 learned_addr learned my addr 172.21.15.84:0/1590264830 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:35.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.373+0000 7fde8d282640 1 --2- 172.21.15.84:0/1590264830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde8815aaa0 0x7fde880c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:35.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.373+0000 7fde8e284640 1 --2- 172.21.15.84:0/1590264830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8815ae80 0x7fde88158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:35.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.373+0000 7fde8da83640 1 -- 172.21.15.84:0/1590264830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8815ae80 msgr2=0x7fde88158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:35.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.373+0000 7fde8da83640 1 --2- 172.21.15.84:0/1590264830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8815ae80 0x7fde88158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:35.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.373+0000 7fde8da83640 1 -- 172.21.15.84:0/1590264830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde8815aaa0 msgr2=0x7fde880c4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:35.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.373+0000 7fde8da83640 1 --2- 172.21.15.84:0/1590264830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde8815aaa0 0x7fde880c4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:35.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.373+0000 7fde8da83640 1 -- 172.21.15.84:0/1590264830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde84009400 con 0x7fde880ccaa0 2023-10-16T11:25:35.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.373+0000 7fde8da83640 1 --2- 172.21.15.84:0/1590264830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde880ccaa0 0x7fde88159660 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7fde840097a0 tx=0x7fde840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:35.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.373+0000 7fde7e7fc640 1 -- 172.21.15.84:0/1590264830 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84004230 con 0x7fde880ccaa0 2023-10-16T11:25:35.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.373+0000 7fde7e7fc640 1 -- 172.21.15.84:0/1590264830 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde840043b0 con 0x7fde880ccaa0 2023-10-16T11:25:35.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.373+0000 7fde7e7fc640 1 -- 172.21.15.84:0/1590264830 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84004520 con 0x7fde880ccaa0 2023-10-16T11:25:35.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.373+0000 7fde8fce7640 1 -- 172.21.15.84:0/1590264830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8815ce20 con 0x7fde880ccaa0 2023-10-16T11:25:35.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.374+0000 7fde8fce7640 1 -- 172.21.15.84:0/1590264830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde88159310 con 0x7fde880ccaa0 2023-10-16T11:25:35.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.375+0000 7fde8fce7640 1 -- 172.21.15.84:0/1590264830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde881594a0 con 0x7fde880ccaa0 2023-10-16T11:25:35.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.376+0000 7fde7e7fc640 1 -- 172.21.15.84:0/1590264830 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fde84032860 con 0x7fde880ccaa0 2023-10-16T11:25:35.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.377+0000 7fde7e7fc640 1 --2- 172.21.15.84:0/1590264830 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fde6c06b0b0 0x7fde6c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:35.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.377+0000 7fde7e7fc640 1 -- 172.21.15.84:0/1590264830 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 5255+0+0 (secure 0 0 0) 0x7fde840aba00 con 0x7fde880ccaa0 2023-10-16T11:25:35.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.377+0000 7fde8d282640 1 --2- 172.21.15.84:0/1590264830 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fde6c06b0b0 0x7fde6c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:35.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.378+0000 7fde8d282640 1 --2- 172.21.15.84:0/1590264830 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fde6c06b0b0 0x7fde6c06d570 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fde881f5f00 tx=0x7fde78008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:35.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.382+0000 7fde7e7fc640 1 -- 172.21.15.84:0/1590264830 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fde881594a0 con 0x7fde880ccaa0 2023-10-16T11:25:35.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:35.574+0000 7fde8fce7640 1 -- 172.21.15.84:0/1590264830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fde881594a0 con 0x7fde880ccaa0 2023-10-16T11:25:36.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.135+0000 7fde7e7fc640 1 -- 172.21.15.84:0/1590264830 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v235) v1 ==== 170+0+0 (secure 0 0 0) 0x7fde881594a0 con 0x7fde880ccaa0 2023-10-16T11:25:36.137 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-16T11:25:36.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.140+0000 7fde8fce7640 1 -- 172.21.15.84:0/1590264830 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fde6c06b0b0 msgr2=0x7fde6c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:36.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.140+0000 7fde8fce7640 1 --2- 172.21.15.84:0/1590264830 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fde6c06b0b0 0x7fde6c06d570 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fde881f5f00 tx=0x7fde78008040 comp rx=0 tx=0).stop 2023-10-16T11:25:36.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.140+0000 7fde8fce7640 1 -- 172.21.15.84:0/1590264830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde880ccaa0 msgr2=0x7fde88159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:36.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.140+0000 7fde8fce7640 1 --2- 172.21.15.84:0/1590264830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde880ccaa0 0x7fde88159660 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7fde840097a0 tx=0x7fde840046c0 comp rx=0 tx=0).stop 2023-10-16T11:25:36.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.141+0000 7fde8fce7640 1 -- 172.21.15.84:0/1590264830 shutdown_connections 2023-10-16T11:25:36.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.141+0000 7fde8fce7640 1 --2- 172.21.15.84:0/1590264830 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fde6c06b0b0 0x7fde6c06d570 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:36.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.141+0000 7fde8fce7640 1 --2- 172.21.15.84:0/1590264830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde8815ae80 0x7fde88158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:36.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.141+0000 7fde8fce7640 1 --2- 172.21.15.84:0/1590264830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde8815aaa0 0x7fde880c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:36.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.141+0000 7fde8fce7640 1 --2- 172.21.15.84:0/1590264830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde880ccaa0 0x7fde88159660 unknown :-1 s=CLOSED pgs=2912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:36.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.141+0000 7fde8fce7640 1 -- 172.21.15.84:0/1590264830 >> 172.21.15.84:0/1590264830 conn(0x7fde880cb410 msgr2=0x7fde8806bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:36.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.141+0000 7fde8fce7640 1 -- 172.21.15.84:0/1590264830 shutdown_connections 2023-10-16T11:25:36.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.141+0000 7fde8fce7640 1 -- 172.21.15.84:0/1590264830 wait complete. 2023-10-16T11:25:36.179 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T11:25:36.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.366+0000 7fd70dc2e640 1 Processor -- start 2023-10-16T11:25:36.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.366+0000 7fd70dc2e640 1 -- start start 2023-10-16T11:25:36.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.367+0000 7fd70dc2e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd708159950 0x7fd70815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:36.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.367+0000 7fd70dc2e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd708070450 0x7fd7080708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:36.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.367+0000 7fd70dc2e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd708070df0 con 0x7fd70806c100 2023-10-16T11:25:36.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.367+0000 7fd70dc2e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd70815e490 con 0x7fd708070450 2023-10-16T11:25:36.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.367+0000 7fd70dc2e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70815aef0 con 0x7fd708159950 2023-10-16T11:25:36.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.368+0000 7fd70d42d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd70806c100 0x7fd70806c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42164/0 (socket says 172.21.15.84:42164) 2023-10-16T11:25:36.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.368+0000 7fd70d42d640 1 -- 172.21.15.84:0/2287180654 learned_addr learned my addr 172.21.15.84:0/2287180654 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:36.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.368+0000 7fd707fff640 1 --2- 172.21.15.84:0/2287180654 >> v2:172.21.15.157:3300/0 conn(0x7fd708070450 0x7fd7080708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:36.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.368+0000 7fd70cc2c640 1 --2- 172.21.15.84:0/2287180654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd708159950 0x7fd70815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:36.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.368+0000 7fd7077fe640 1 -- 172.21.15.84:0/2287180654 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3727296502 0 0) 0x7fd708070df0 con 0x7fd70806c100 2023-10-16T11:25:36.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.368+0000 7fd7077fe640 1 -- 172.21.15.84:0/2287180654 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6f4003520 con 0x7fd70806c100 2023-10-16T11:25:36.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd7077fe640 1 -- 172.21.15.84:0/2287180654 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1347065263 0 0) 0x7fd6f4003520 con 0x7fd70806c100 2023-10-16T11:25:36.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd7077fe640 1 -- 172.21.15.84:0/2287180654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd708159950 msgr2=0x7fd70815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:36.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd7077fe640 1 --2- 172.21.15.84:0/2287180654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd708159950 0x7fd70815a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:36.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd7077fe640 1 -- 172.21.15.84:0/2287180654 >> v2:172.21.15.157:3300/0 conn(0x7fd708070450 msgr2=0x7fd7080708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:36.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd7077fe640 1 --2- 172.21.15.84:0/2287180654 >> v2:172.21.15.157:3300/0 conn(0x7fd708070450 0x7fd7080708b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:36.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd7077fe640 1 -- 172.21.15.84:0/2287180654 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70806cc80 con 0x7fd70806c100 2023-10-16T11:25:36.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd7077fe640 1 -- 172.21.15.84:0/2287180654 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd700001670 con 0x7fd70806c100 2023-10-16T11:25:36.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd7077fe640 1 -- 172.21.15.84:0/2287180654 >> v1:172.21.15.84:6789/0 conn(0x7fd70806c100 legacy=0x7fd70806c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:36.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd7077fe640 1 --2- 172.21.15.84:0/2287180654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd708159950 0x7fd6f4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:36.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd7077fe640 1 --2- 172.21.15.84:0/2287180654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6f4003ef0 0x7fd6f40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:36.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd7077fe640 1 --2- 172.21.15.84:0/2287180654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6f4004d00 0x7fd6f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:36.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd7077fe640 1 -- 172.21.15.84:0/2287180654 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70815e490 con 0x7fd6f4004d00 2023-10-16T11:25:36.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd7077fe640 1 -- 172.21.15.84:0/2287180654 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70815aef0 con 0x7fd6f4003ef0 2023-10-16T11:25:36.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd7077fe640 1 -- 172.21.15.84:0/2287180654 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f400dbe0 con 0x7fd708159950 2023-10-16T11:25:36.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd70d42d640 1 --2- 172.21.15.84:0/2287180654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6f4004d00 0x7fd6f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:36.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd70cc2c640 1 --2- 172.21.15.84:0/2287180654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd708159950 0x7fd6f4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:36.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd707fff640 1 --2- 172.21.15.84:0/2287180654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6f4003ef0 0x7fd6f40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:36.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd70d42d640 1 -- 172.21.15.84:0/2287180654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd708159950 msgr2=0x7fd6f4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:36.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd70d42d640 1 --2- 172.21.15.84:0/2287180654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd708159950 0x7fd6f4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:36.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd70d42d640 1 -- 172.21.15.84:0/2287180654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6f4003ef0 msgr2=0x7fd6f40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:36.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.369+0000 7fd70d42d640 1 --2- 172.21.15.84:0/2287180654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6f4003ef0 0x7fd6f40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:36.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.370+0000 7fd70d42d640 1 -- 172.21.15.84:0/2287180654 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f4003520 con 0x7fd6f4004d00 2023-10-16T11:25:36.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.371+0000 7fd70d42d640 1 --2- 172.21.15.84:0/2287180654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6f4004d00 0x7fd6f4005180 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7fd70000c270 tx=0x7fd6f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1aad280453e3e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:36.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.371+0000 7fd7077fe640 1 -- 172.21.15.84:0/2287180654 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70002cae0 con 0x7fd6f4004d00 2023-10-16T11:25:36.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.371+0000 7fd7077fe640 1 -- 172.21.15.84:0/2287180654 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70002cc40 con 0x7fd6f4004d00 2023-10-16T11:25:36.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.371+0000 7fd7077fe640 1 -- 172.21.15.84:0/2287180654 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700005af0 con 0x7fd6f4004d00 2023-10-16T11:25:36.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.371+0000 7fd70dc2e640 1 -- 172.21.15.84:0/2287180654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6f4004d00 msgr2=0x7fd6f4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:36.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.371+0000 7fd70dc2e640 1 --2- 172.21.15.84:0/2287180654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6f4004d00 0x7fd6f4005180 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7fd70000c270 tx=0x7fd6f40057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:36.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.371+0000 7fd70d42d640 1 -- 172.21.15.84:0/2287180654 reap_dead start 2023-10-16T11:25:36.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.372+0000 7fd70dc2e640 1 -- 172.21.15.84:0/2287180654 shutdown_connections 2023-10-16T11:25:36.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.372+0000 7fd70dc2e640 1 -- 172.21.15.84:0/2287180654 >> 172.21.15.84:0/2287180654 conn(0x7fd7080ce6a0 msgr2=0x7fd7080ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:36.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.372+0000 7fd70dc2e640 1 -- 172.21.15.84:0/2287180654 shutdown_connections 2023-10-16T11:25:36.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.372+0000 7fd70dc2e640 1 -- 172.21.15.84:0/2287180654 wait complete. 2023-10-16T11:25:36.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.374+0000 7fd70dc2e640 1 Processor -- start 2023-10-16T11:25:36.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.374+0000 7fd70dc2e640 1 -- start start 2023-10-16T11:25:36.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.375+0000 7fd70dc2e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6f4004d00 0x7fd7081f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:36.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.375+0000 7fd70dc2e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7080b0170 0x7fd7080b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:36.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.375+0000 7fd70dc2e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7080c4080 0x7fd7080c4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:36.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.375+0000 7fd70dc2e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7080c4df0 con 0x7fd7080b0170 2023-10-16T11:25:36.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.375+0000 7fd70dc2e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7080b0b10 con 0x7fd7080c4080 2023-10-16T11:25:36.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.375+0000 7fd70dc2e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7081f1e30 con 0x7fd6f4004d00 2023-10-16T11:25:36.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.375+0000 7fd707fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7080b0170 0x7fd7080b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:36.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.375+0000 7fd707fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7080b0170 0x7fd7080b05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47188/0 (socket says 172.21.15.84:47188) 2023-10-16T11:25:36.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.375+0000 7fd707fff640 1 -- 172.21.15.84:0/3114463077 learned_addr learned my addr 172.21.15.84:0/3114463077 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:36.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.375+0000 7fd70cc2c640 1 --2- 172.21.15.84:0/3114463077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6f4004d00 0x7fd7081f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:36.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.375+0000 7fd70d42d640 1 --2- 172.21.15.84:0/3114463077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7080c4080 0x7fd7080c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:36.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.375+0000 7fd707fff640 1 -- 172.21.15.84:0/3114463077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6f4004d00 msgr2=0x7fd7081f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:36.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.375+0000 7fd707fff640 1 --2- 172.21.15.84:0/3114463077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6f4004d00 0x7fd7081f18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:36.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.375+0000 7fd707fff640 1 -- 172.21.15.84:0/3114463077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7080c4080 msgr2=0x7fd7080c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:36.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.375+0000 7fd707fff640 1 --2- 172.21.15.84:0/3114463077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7080c4080 0x7fd7080c4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:36.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.375+0000 7fd707fff640 1 -- 172.21.15.84:0/3114463077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd708156350 con 0x7fd7080b0170 2023-10-16T11:25:36.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.376+0000 7fd707fff640 1 --2- 172.21.15.84:0/3114463077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7080b0170 0x7fd7080b05d0 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800f400 tx=0x7fd6f800f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:36.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.376+0000 7fd7057fa640 1 -- 172.21.15.84:0/3114463077 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8016620 con 0x7fd7080b0170 2023-10-16T11:25:36.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.376+0000 7fd7057fa640 1 -- 172.21.15.84:0/3114463077 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f80167a0 con 0x7fd7080b0170 2023-10-16T11:25:36.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.376+0000 7fd7057fa640 1 -- 172.21.15.84:0/3114463077 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8016b50 con 0x7fd7080b0170 2023-10-16T11:25:36.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.376+0000 7fd70dc2e640 1 -- 172.21.15.84:0/3114463077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd708155a20 con 0x7fd7080b0170 2023-10-16T11:25:36.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.376+0000 7fd70dc2e640 1 -- 172.21.15.84:0/3114463077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd708155fe0 con 0x7fd7080b0170 2023-10-16T11:25:36.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.377+0000 7fd70dc2e640 1 -- 172.21.15.84:0/3114463077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7081564e0 con 0x7fd7080b0170 2023-10-16T11:25:36.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.379+0000 7fd7057fa640 1 -- 172.21.15.84:0/3114463077 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd6f8016cb0 con 0x7fd7080b0170 2023-10-16T11:25:36.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.380+0000 7fd7057fa640 1 --2- 172.21.15.84:0/3114463077 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd6e806b0e0 0x7fd6e806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:36.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.380+0000 7fd7057fa640 1 -- 172.21.15.84:0/3114463077 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5637+0+0 (secure 0 0 0) 0x7fd6f808b470 con 0x7fd7080b0170 2023-10-16T11:25:36.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.381+0000 7fd70cc2c640 1 --2- 172.21.15.84:0/3114463077 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd6e806b0e0 0x7fd6e806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:36.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.381+0000 7fd70cc2c640 1 --2- 172.21.15.84:0/3114463077 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd6e806b0e0 0x7fd6e806d5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009a10 tx=0x7fd6f0008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:36.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.384+0000 7fd7057fa640 1 -- 172.21.15.84:0/3114463077 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd6f80561d0 con 0x7fd7080b0170 2023-10-16T11:25:36.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:36.602+0000 7fd70dc2e640 1 -- 172.21.15.84:0/3114463077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd70804e9b0 con 0x7fd7080b0170 2023-10-16T11:25:37.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.139+0000 7fd7057fa640 1 -- 172.21.15.84:0/3114463077 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 30 and data pool 31 v127) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd6f80597f0 con 0x7fd7080b0170 2023-10-16T11:25:37.141 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 30 and data pool 31 2023-10-16T11:25:37.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.142+0000 7fd70dc2e640 1 -- 172.21.15.84:0/3114463077 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd6e806b0e0 msgr2=0x7fd6e806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:37.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.142+0000 7fd70dc2e640 1 --2- 172.21.15.84:0/3114463077 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd6e806b0e0 0x7fd6e806d5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009a10 tx=0x7fd6f0008040 comp rx=0 tx=0).stop 2023-10-16T11:25:37.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.142+0000 7fd70dc2e640 1 -- 172.21.15.84:0/3114463077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7080b0170 msgr2=0x7fd7080b05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:37.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.142+0000 7fd70dc2e640 1 --2- 172.21.15.84:0/3114463077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7080b0170 0x7fd7080b05d0 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800f400 tx=0x7fd6f800f8c0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.143+0000 7fd70dc2e640 1 -- 172.21.15.84:0/3114463077 shutdown_connections 2023-10-16T11:25:37.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.143+0000 7fd70dc2e640 1 --2- 172.21.15.84:0/3114463077 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd6e806b0e0 0x7fd6e806d5a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.143+0000 7fd70dc2e640 1 --2- 172.21.15.84:0/3114463077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7080c4080 0x7fd7080c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.143+0000 7fd70dc2e640 1 --2- 172.21.15.84:0/3114463077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7080b0170 0x7fd7080b05d0 unknown :-1 s=CLOSED pgs=2915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.143+0000 7fd70dc2e640 1 --2- 172.21.15.84:0/3114463077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6f4004d00 0x7fd7081f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.143+0000 7fd70dc2e640 1 -- 172.21.15.84:0/3114463077 >> 172.21.15.84:0/3114463077 conn(0x7fd7080ce6a0 msgr2=0x7fd708070830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:37.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.143+0000 7fd70dc2e640 1 -- 172.21.15.84:0/3114463077 shutdown_connections 2023-10-16T11:25:37.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.143+0000 7fd70dc2e640 1 -- 172.21.15.84:0/3114463077 wait complete. 2023-10-16T11:25:37.183 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:25:37.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.364+0000 7f0b6eabb640 1 Processor -- start 2023-10-16T11:25:37.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.364+0000 7f0b6eabb640 1 -- start start 2023-10-16T11:25:37.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.365+0000 7f0b6eabb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0b680cd0c0 0x7f0b681615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:37.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.366+0000 7f0b6eabb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0b680b0410 0x7f0b680b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:37.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.366+0000 7f0b6eabb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6815e3e0 con 0x7f0b6815d3f0 2023-10-16T11:25:37.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.366+0000 7f0b6eabb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b680b0de0 con 0x7f0b680cd0c0 2023-10-16T11:25:37.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.366+0000 7f0b6eabb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b680cd4a0 con 0x7f0b680b0410 2023-10-16T11:25:37.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.366+0000 7f0b6c857640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0b680cd0c0 0x7f0b681615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:37.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.366+0000 7f0b6c857640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0b680cd0c0 0x7f0b681615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47200/0 (socket says 172.21.15.84:47200) 2023-10-16T11:25:37.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.366+0000 7f0b6c857640 1 -- 172.21.15.84:0/1436599192 learned_addr learned my addr 172.21.15.84:0/1436599192 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:37.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.366+0000 7f0b67fff640 1 --2- 172.21.15.84:0/1436599192 >> v2:172.21.15.157:3300/0 conn(0x7f0b680b0410 0x7f0b680b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:37.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.366+0000 7f0b6c857640 1 -- 172.21.15.84:0/1436599192 >> v2:172.21.15.157:3300/0 conn(0x7f0b680b0410 msgr2=0x7f0b680b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:37.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.366+0000 7f0b6c857640 1 --2- 172.21.15.84:0/1436599192 >> v2:172.21.15.157:3300/0 conn(0x7f0b680b0410 0x7f0b680b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.366+0000 7f0b6c857640 1 -- 172.21.15.84:0/1436599192 >> v1:172.21.15.84:6789/0 conn(0x7f0b6815d3f0 legacy=0x7f0b6815d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:37.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.366+0000 7f0b6c857640 1 -- 172.21.15.84:0/1436599192 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6815e100 con 0x7f0b680cd0c0 2023-10-16T11:25:37.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.367+0000 7f0b6c857640 1 --2- 172.21.15.84:0/1436599192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b680cd0c0 0x7f0b681615f0 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7f0b680cdad0 tx=0x7f0b5c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f635fa18cab03af server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:37.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.368+0000 7f0b677fe640 1 -- 172.21.15.84:0/1436599192 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c031870 con 0x7f0b680cd0c0 2023-10-16T11:25:37.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.368+0000 7f0b677fe640 1 -- 172.21.15.84:0/1436599192 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5c0319d0 con 0x7f0b680cd0c0 2023-10-16T11:25:37.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.368+0000 7f0b677fe640 1 -- 172.21.15.84:0/1436599192 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c031d60 con 0x7f0b680cd0c0 2023-10-16T11:25:37.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.368+0000 7f0b6eabb640 1 -- 172.21.15.84:0/1436599192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b680cd0c0 msgr2=0x7f0b681615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:37.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.368+0000 7f0b6eabb640 1 --2- 172.21.15.84:0/1436599192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b680cd0c0 0x7f0b681615f0 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7f0b680cdad0 tx=0x7f0b5c02b910 comp rx=0 tx=0).stop 2023-10-16T11:25:37.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.368+0000 7f0b6eabb640 1 -- 172.21.15.84:0/1436599192 shutdown_connections 2023-10-16T11:25:37.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.368+0000 7f0b6eabb640 1 --2- 172.21.15.84:0/1436599192 >> v2:172.21.15.157:3300/0 conn(0x7f0b680b0410 0x7f0b680b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.368+0000 7f0b6eabb640 1 --2- 172.21.15.84:0/1436599192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b680cd0c0 0x7f0b681615f0 unknown :-1 s=CLOSED pgs=2917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.368+0000 7f0b6eabb640 1 -- 172.21.15.84:0/1436599192 >> 172.21.15.84:0/1436599192 conn(0x7f0b680cb410 msgr2=0x7f0b680cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:37.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.369+0000 7f0b6eabb640 1 -- 172.21.15.84:0/1436599192 shutdown_connections 2023-10-16T11:25:37.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.369+0000 7f0b6eabb640 1 -- 172.21.15.84:0/1436599192 wait complete. 2023-10-16T11:25:37.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.370+0000 7f0b6eabb640 1 Processor -- start 2023-10-16T11:25:37.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.370+0000 7f0b6eabb640 1 -- start start 2023-10-16T11:25:37.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.371+0000 7f0b6eabb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b680b0410 0x7f0b68156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:37.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.371+0000 7f0b6eabb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b680cd0c0 0x7f0b680c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:37.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.371+0000 7f0b6eabb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b6815d3f0 0x7f0b680c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:37.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.371+0000 7f0b6eabb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b681e7140 con 0x7f0b680cd0c0 2023-10-16T11:25:37.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.371+0000 7f0b6eabb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b681e6270 con 0x7f0b6815d3f0 2023-10-16T11:25:37.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.371+0000 7f0b6eabb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b68156fa0 con 0x7f0b680b0410 2023-10-16T11:25:37.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.372+0000 7f0b67fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b680cd0c0 0x7f0b680c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:37.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.372+0000 7f0b67fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b680cd0c0 0x7f0b680c5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47214/0 (socket says 172.21.15.84:47214) 2023-10-16T11:25:37.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.372+0000 7f0b67fff640 1 -- 172.21.15.84:0/2296665987 learned_addr learned my addr 172.21.15.84:0/2296665987 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:37.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.372+0000 7f0b6d058640 1 --2- 172.21.15.84:0/2296665987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b6815d3f0 0x7f0b680c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:37.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.372+0000 7f0b6c857640 1 --2- 172.21.15.84:0/2296665987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b680b0410 0x7f0b68156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:37.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.372+0000 7f0b67fff640 1 -- 172.21.15.84:0/2296665987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b680b0410 msgr2=0x7f0b68156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:37.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.372+0000 7f0b67fff640 1 --2- 172.21.15.84:0/2296665987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b680b0410 0x7f0b68156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.372+0000 7f0b67fff640 1 -- 172.21.15.84:0/2296665987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b6815d3f0 msgr2=0x7f0b680c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:37.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.372+0000 7f0b67fff640 1 --2- 172.21.15.84:0/2296665987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b6815d3f0 0x7f0b680c5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.372+0000 7f0b67fff640 1 -- 172.21.15.84:0/2296665987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c0094d0 con 0x7f0b680cd0c0 2023-10-16T11:25:37.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.372+0000 7f0b6c857640 1 --2- 172.21.15.84:0/2296665987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b680b0410 0x7f0b68156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:37.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.372+0000 7f0b67fff640 1 --2- 172.21.15.84:0/2296665987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b680cd0c0 0x7f0b680c5a60 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400d5f0 tx=0x7f0b5400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:37.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.373+0000 7f0b657fa640 1 -- 172.21.15.84:0/2296665987 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b540125f0 con 0x7f0b680cd0c0 2023-10-16T11:25:37.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.373+0000 7f0b657fa640 1 -- 172.21.15.84:0/2296665987 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b54012750 con 0x7f0b680cd0c0 2023-10-16T11:25:37.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.373+0000 7f0b657fa640 1 -- 172.21.15.84:0/2296665987 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b54012b00 con 0x7f0b680cd0c0 2023-10-16T11:25:37.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.373+0000 7f0b6eabb640 1 -- 172.21.15.84:0/2296665987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b681e64c0 con 0x7f0b680cd0c0 2023-10-16T11:25:37.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.373+0000 7f0b6eabb640 1 -- 172.21.15.84:0/2296665987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b681e6a80 con 0x7f0b680cd0c0 2023-10-16T11:25:37.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.374+0000 7f0b6eabb640 1 -- 172.21.15.84:0/2296665987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b681e7280 con 0x7f0b680cd0c0 2023-10-16T11:25:37.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.375+0000 7f0b657fa640 1 -- 172.21.15.84:0/2296665987 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f0b54012c60 con 0x7f0b680cd0c0 2023-10-16T11:25:37.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.376+0000 7f0b657fa640 1 --2- 172.21.15.84:0/2296665987 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f0b4c06b0b0 0x7f0b4c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:37.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.376+0000 7f0b657fa640 1 -- 172.21.15.84:0/2296665987 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0b54089ff0 con 0x7f0b680cd0c0 2023-10-16T11:25:37.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.376+0000 7f0b6c857640 1 --2- 172.21.15.84:0/2296665987 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f0b4c06b0b0 0x7f0b4c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:37.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.377+0000 7f0b6c857640 1 --2- 172.21.15.84:0/2296665987 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f0b4c06b0b0 0x7f0b4c06d570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c036000 tx=0x7f0b5c03c040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:37.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.380+0000 7f0b657fa640 1 -- 172.21.15.84:0/2296665987 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0b540bcfe0 con 0x7f0b680cd0c0 2023-10-16T11:25:37.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.559+0000 7f0b6eabb640 1 -- 172.21.15.84:0/2296665987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b6804e160 con 0x7f0b680cd0c0 2023-10-16T11:25:37.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.561+0000 7f0b657fa640 1 -- 172.21.15.84:0/2296665987 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v236) v1 ==== 74+0+16552 (secure 0 0 0) 0x7f0b681e7280 con 0x7f0b680cd0c0 2023-10-16T11:25:37.563 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:37.564 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":236,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:25:36.604319+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:25:34.443897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:25:35.635615+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:25:37.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.563+0000 7f0b6eabb640 1 -- 172.21.15.84:0/2296665987 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f0b4c06b0b0 msgr2=0x7f0b4c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:37.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.563+0000 7f0b6eabb640 1 --2- 172.21.15.84:0/2296665987 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f0b4c06b0b0 0x7f0b4c06d570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c036000 tx=0x7f0b5c03c040 comp rx=0 tx=0).stop 2023-10-16T11:25:37.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.563+0000 7f0b6eabb640 1 -- 172.21.15.84:0/2296665987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b680cd0c0 msgr2=0x7f0b680c5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:37.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.563+0000 7f0b6eabb640 1 --2- 172.21.15.84:0/2296665987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b680cd0c0 0x7f0b680c5a60 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400d5f0 tx=0x7f0b5400dab0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.563+0000 7f0b6eabb640 1 -- 172.21.15.84:0/2296665987 shutdown_connections 2023-10-16T11:25:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.563+0000 7f0b6eabb640 1 --2- 172.21.15.84:0/2296665987 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f0b4c06b0b0 0x7f0b4c06d570 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.563+0000 7f0b6eabb640 1 --2- 172.21.15.84:0/2296665987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b6815d3f0 0x7f0b680c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.563+0000 7f0b6eabb640 1 --2- 172.21.15.84:0/2296665987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b680cd0c0 0x7f0b680c5a60 unknown :-1 s=CLOSED pgs=2918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.563+0000 7f0b6eabb640 1 --2- 172.21.15.84:0/2296665987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b680b0410 0x7f0b68156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.563+0000 7f0b6eabb640 1 -- 172.21.15.84:0/2296665987 >> 172.21.15.84:0/2296665987 conn(0x7f0b680cb410 msgr2=0x7f0b680af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:37.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.564+0000 7f0b6eabb640 1 -- 172.21.15.84:0/2296665987 shutdown_connections 2023-10-16T11:25:37.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.564+0000 7f0b6eabb640 1 -- 172.21.15.84:0/2296665987 wait complete. 2023-10-16T11:25:37.583 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:25:37.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.793+0000 7f8735a6f640 1 Processor -- start 2023-10-16T11:25:37.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.793+0000 7f8735a6f640 1 -- start start 2023-10-16T11:25:37.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.795+0000 7f8735a6f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8730159b20 0x7f873006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:37.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.795+0000 7f8735a6f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f87300b0b20 0x7f873006fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:37.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.795+0000 7f8735a6f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f873006f7f0 con 0x7f8730159f00 2023-10-16T11:25:37.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.795+0000 7f8735a6f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f873006c5e0 con 0x7f87300b0b20 2023-10-16T11:25:37.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.795+0000 7f8735a6f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f873006c8c0 con 0x7f8730159b20 2023-10-16T11:25:37.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.795+0000 7f872ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f87300b0b20 0x7f873006fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:37.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.795+0000 7f872ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f87300b0b20 0x7f873006fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47230/0 (socket says 172.21.15.84:47230) 2023-10-16T11:25:37.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.795+0000 7f872ffff640 1 -- 172.21.15.84:0/2933250588 learned_addr learned my addr 172.21.15.84:0/2933250588 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:37.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.796+0000 7f872ffff640 1 -- 172.21.15.84:0/2933250588 >> v2:172.21.15.157:3300/0 conn(0x7f8730159b20 msgr2=0x7f873006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:37.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.796+0000 7f872ffff640 1 --2- 172.21.15.84:0/2933250588 >> v2:172.21.15.157:3300/0 conn(0x7f8730159b20 0x7f873006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.796+0000 7f872ffff640 1 -- 172.21.15.84:0/2933250588 >> v1:172.21.15.157:6789/0 conn(0x7f8730159f00 legacy=0x7f87300b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:37.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.796+0000 7f872ffff640 1 -- 172.21.15.84:0/2933250588 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873006cba0 con 0x7f87300b0b20 2023-10-16T11:25:37.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.797+0000 7f872ffff640 1 --2- 172.21.15.84:0/2933250588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87300b0b20 0x7f873006fc80 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7f87300c9340 tx=0x7f872802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc175db453f3853 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:37.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.797+0000 7f872effd640 1 -- 172.21.15.84:0/2933250588 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872802ca80 con 0x7f87300b0b20 2023-10-16T11:25:37.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.797+0000 7f872effd640 1 -- 172.21.15.84:0/2933250588 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872802cbe0 con 0x7f87300b0b20 2023-10-16T11:25:37.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.797+0000 7f872effd640 1 -- 172.21.15.84:0/2933250588 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87280343f0 con 0x7f87300b0b20 2023-10-16T11:25:37.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.798+0000 7f8735a6f640 1 -- 172.21.15.84:0/2933250588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87300b0b20 msgr2=0x7f873006fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:37.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.798+0000 7f8735a6f640 1 --2- 172.21.15.84:0/2933250588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87300b0b20 0x7f873006fc80 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7f87300c9340 tx=0x7f872802bee0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.798+0000 7f8735a6f640 1 -- 172.21.15.84:0/2933250588 shutdown_connections 2023-10-16T11:25:37.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.798+0000 7f8735a6f640 1 --2- 172.21.15.84:0/2933250588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87300b0b20 0x7f873006fc80 unknown :-1 s=CLOSED pgs=2919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.798+0000 7f8735a6f640 1 --2- 172.21.15.84:0/2933250588 >> v2:172.21.15.157:3300/0 conn(0x7f8730159b20 0x7f873006bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.798+0000 7f8735a6f640 1 -- 172.21.15.84:0/2933250588 >> 172.21.15.84:0/2933250588 conn(0x7f87300cb410 msgr2=0x7f87300cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:37.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.798+0000 7f8735a6f640 1 -- 172.21.15.84:0/2933250588 shutdown_connections 2023-10-16T11:25:37.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.798+0000 7f8735a6f640 1 -- 172.21.15.84:0/2933250588 wait complete. 2023-10-16T11:25:37.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.800+0000 7f8735a6f640 1 Processor -- start 2023-10-16T11:25:37.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.800+0000 7f8735a6f640 1 -- start start 2023-10-16T11:25:37.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.801+0000 7f8735a6f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87300b0b20 0x7f87301517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:37.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.801+0000 7f8735a6f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8730159b20 0x7f87300b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:37.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.801+0000 7f8735a6f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8730159f00 0x7f87300ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:37.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.801+0000 7f8735a6f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8730151d20 con 0x7f8730159f00 2023-10-16T11:25:37.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.801+0000 7f8735a6f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87300c8940 con 0x7f87300b0b20 2023-10-16T11:25:37.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.801+0000 7f8735a6f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87300c2770 con 0x7f8730159b20 2023-10-16T11:25:37.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.801+0000 7f872ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8730159f00 0x7f87300ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:37.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.801+0000 7f872ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8730159f00 0x7f87300ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47238/0 (socket says 172.21.15.84:47238) 2023-10-16T11:25:37.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.801+0000 7f872ffff640 1 -- 172.21.15.84:0/768143515 learned_addr learned my addr 172.21.15.84:0/768143515 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:37.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.801+0000 7f872f7fe640 1 --2- 172.21.15.84:0/768143515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87300b0b20 0x7f87301517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:37.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.801+0000 7f8727fff640 1 --2- 172.21.15.84:0/768143515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8730159b20 0x7f87300b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:37.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.801+0000 7f872ffff640 1 -- 172.21.15.84:0/768143515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8730159b20 msgr2=0x7f87300b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:37.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.801+0000 7f872ffff640 1 --2- 172.21.15.84:0/768143515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8730159b20 0x7f87300b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.801+0000 7f872ffff640 1 -- 172.21.15.84:0/768143515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87300b0b20 msgr2=0x7f87301517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:37.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.801+0000 7f872ffff640 1 --2- 172.21.15.84:0/768143515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87300b0b20 0x7f87301517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:37.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.801+0000 7f872ffff640 1 -- 172.21.15.84:0/768143515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8728009b30 con 0x7f8730159f00 2023-10-16T11:25:37.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.801+0000 7f872f7fe640 1 --2- 172.21.15.84:0/768143515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87300b0b20 0x7f87301517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:37.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.802+0000 7f872ffff640 1 --2- 172.21.15.84:0/768143515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8730159f00 0x7f87300ccd20 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7f8728036040 tx=0x7f8728004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:37.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.802+0000 7f872cff9640 1 -- 172.21.15.84:0/768143515 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872803e450 con 0x7f8730159f00 2023-10-16T11:25:37.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.802+0000 7f872cff9640 1 -- 172.21.15.84:0/768143515 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872803e5d0 con 0x7f8730159f00 2023-10-16T11:25:37.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.802+0000 7f872cff9640 1 -- 172.21.15.84:0/768143515 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872803e980 con 0x7f8730159f00 2023-10-16T11:25:37.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.802+0000 7f8735a6f640 1 -- 172.21.15.84:0/768143515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87300c2a50 con 0x7f8730159f00 2023-10-16T11:25:37.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.802+0000 7f8735a6f640 1 -- 172.21.15.84:0/768143515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87300c3010 con 0x7f8730159f00 2023-10-16T11:25:37.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.803+0000 7f8735a6f640 1 -- 172.21.15.84:0/768143515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f873004e160 con 0x7f8730159f00 2023-10-16T11:25:37.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.804+0000 7f872cff9640 1 -- 172.21.15.84:0/768143515 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f872802cd50 con 0x7f8730159f00 2023-10-16T11:25:37.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.805+0000 7f872cff9640 1 --2- 172.21.15.84:0/768143515 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f871006b0b0 0x7f871006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:37.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.805+0000 7f872cff9640 1 -- 172.21.15.84:0/768143515 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f87280ac6e0 con 0x7f8730159f00 2023-10-16T11:25:37.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.805+0000 7f872f7fe640 1 --2- 172.21.15.84:0/768143515 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f871006b0b0 0x7f871006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:37.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.806+0000 7f872f7fe640 1 --2- 172.21.15.84:0/768143515 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f871006b0b0 0x7f871006d570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8720006fd0 tx=0x7f8720008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:37.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:37.809+0000 7f872cff9640 1 -- 172.21.15.84:0/768143515 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8728077400 con 0x7f8730159f00 2023-10-16T11:25:38.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.002+0000 7f8735a6f640 1 -- 172.21.15.84:0/768143515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f873006d830 con 0x7f8730159f00 2023-10-16T11:25:38.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.005+0000 7f872cff9640 1 -- 172.21.15.84:0/768143515 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v236) v1 ==== 74+0+16552 (secure 0 0 0) 0x7f872807aa20 con 0x7f8730159f00 2023-10-16T11:25:38.008 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:38.008 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":236,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:25:36.604319+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:25:34.443897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:25:35.635615+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:25:38.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.008+0000 7f8735a6f640 1 -- 172.21.15.84:0/768143515 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f871006b0b0 msgr2=0x7f871006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:38.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.008+0000 7f8735a6f640 1 --2- 172.21.15.84:0/768143515 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f871006b0b0 0x7f871006d570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8720006fd0 tx=0x7f8720008040 comp rx=0 tx=0).stop 2023-10-16T11:25:38.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.008+0000 7f8735a6f640 1 -- 172.21.15.84:0/768143515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8730159f00 msgr2=0x7f87300ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:38.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.008+0000 7f8735a6f640 1 --2- 172.21.15.84:0/768143515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8730159f00 0x7f87300ccd20 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7f8728036040 tx=0x7f8728004a40 comp rx=0 tx=0).stop 2023-10-16T11:25:38.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.008+0000 7f8735a6f640 1 -- 172.21.15.84:0/768143515 shutdown_connections 2023-10-16T11:25:38.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.008+0000 7f8735a6f640 1 --2- 172.21.15.84:0/768143515 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f871006b0b0 0x7f871006d570 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:38.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.008+0000 7f8735a6f640 1 --2- 172.21.15.84:0/768143515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8730159f00 0x7f87300ccd20 unknown :-1 s=CLOSED pgs=2920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:38.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.008+0000 7f8735a6f640 1 --2- 172.21.15.84:0/768143515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8730159b20 0x7f87300b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:38.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.008+0000 7f8735a6f640 1 --2- 172.21.15.84:0/768143515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87300b0b20 0x7f87301517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:38.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.008+0000 7f8735a6f640 1 -- 172.21.15.84:0/768143515 >> 172.21.15.84:0/768143515 conn(0x7f87300cb410 msgr2=0x7f873006fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:38.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.008+0000 7f8735a6f640 1 -- 172.21.15.84:0/768143515 shutdown_connections 2023-10-16T11:25:38.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.008+0000 7f8735a6f640 1 -- 172.21.15.84:0/768143515 wait complete. 2023-10-16T11:25:38.030 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T11:25:38.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.229+0000 7fc2f5856640 1 Processor -- start 2023-10-16T11:25:38.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.229+0000 7fc2f5856640 1 -- start start 2023-10-16T11:25:38.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.230+0000 7fc2f5856640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2f00cf710 0x7fc2f00b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:38.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.230+0000 7fc2f5856640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc2f00cfaf0 0x7fc2f00b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:38.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.230+0000 7fc2f5856640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f00d3980 con 0x7fc2f00b38e0 2023-10-16T11:25:38.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.230+0000 7fc2f5856640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2f00d3c60 con 0x7fc2f00cfaf0 2023-10-16T11:25:38.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.230+0000 7fc2f5856640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f00ce630 con 0x7fc2f00cf710 2023-10-16T11:25:38.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.231+0000 7fc2ef7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc2f00b38e0 0x7fc2f00d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42186/0 (socket says 172.21.15.84:42186) 2023-10-16T11:25:38.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.231+0000 7fc2ef7fe640 1 -- 172.21.15.84:0/1964214453 learned_addr learned my addr 172.21.15.84:0/1964214453 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:38.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.231+0000 7fc2eeffd640 1 --2- 172.21.15.84:0/1964214453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2f00cf710 0x7fc2f00b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:38.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.231+0000 7fc2ee7fc640 1 --2- 172.21.15.84:0/1964214453 >> v2:172.21.15.157:3300/0 conn(0x7fc2f00cfaf0 0x7fc2f00b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:38.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.231+0000 7fc2edffb640 1 -- 172.21.15.84:0/1964214453 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3400146519 0 0) 0x7fc2f00d3980 con 0x7fc2f00b38e0 2023-10-16T11:25:38.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.231+0000 7fc2edffb640 1 -- 172.21.15.84:0/1964214453 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2dc003520 con 0x7fc2f00b38e0 2023-10-16T11:25:38.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.231+0000 7fc2edffb640 1 -- 172.21.15.84:0/1964214453 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2641331616 0 0) 0x7fc2dc003520 con 0x7fc2f00b38e0 2023-10-16T11:25:38.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.231+0000 7fc2edffb640 1 -- 172.21.15.84:0/1964214453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2f00cf710 msgr2=0x7fc2f00b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:38.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.231+0000 7fc2edffb640 1 --2- 172.21.15.84:0/1964214453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2f00cf710 0x7fc2f00b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:38.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.231+0000 7fc2edffb640 1 -- 172.21.15.84:0/1964214453 >> v2:172.21.15.157:3300/0 conn(0x7fc2f00cfaf0 msgr2=0x7fc2f00b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:38.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.231+0000 7fc2edffb640 1 --2- 172.21.15.84:0/1964214453 >> v2:172.21.15.157:3300/0 conn(0x7fc2f00cfaf0 0x7fc2f00b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:38.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.231+0000 7fc2edffb640 1 -- 172.21.15.84:0/1964214453 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f00ce910 con 0x7fc2f00b38e0 2023-10-16T11:25:38.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.231+0000 7fc2edffb640 1 -- 172.21.15.84:0/1964214453 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc2e8001670 con 0x7fc2f00b38e0 2023-10-16T11:25:38.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.231+0000 7fc2edffb640 1 -- 172.21.15.84:0/1964214453 >> v1:172.21.15.84:6789/0 conn(0x7fc2f00b38e0 legacy=0x7fc2f00d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:38.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.231+0000 7fc2edffb640 1 --2- 172.21.15.84:0/1964214453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2dc003970 0x7fc2dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.232+0000 7fc2edffb640 1 --2- 172.21.15.84:0/1964214453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2f00cf710 0x7fc2dc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.232+0000 7fc2edffb640 1 --2- 172.21.15.84:0/1964214453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2dc004d00 0x7fc2dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.232+0000 7fc2edffb640 1 -- 172.21.15.84:0/1964214453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f00d3c60 con 0x7fc2dc004d00 2023-10-16T11:25:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.232+0000 7fc2edffb640 1 -- 172.21.15.84:0/1964214453 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f00ce630 con 0x7fc2dc003970 2023-10-16T11:25:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.232+0000 7fc2edffb640 1 -- 172.21.15.84:0/1964214453 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc00ba30 con 0x7fc2f00cf710 2023-10-16T11:25:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.232+0000 7fc2ef7fe640 1 --2- 172.21.15.84:0/1964214453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2dc004d00 0x7fc2dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.232+0000 7fc2eeffd640 1 --2- 172.21.15.84:0/1964214453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2dc003970 0x7fc2dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.232+0000 7fc2ee7fc640 1 --2- 172.21.15.84:0/1964214453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2f00cf710 0x7fc2dc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.232+0000 7fc2ef7fe640 1 -- 172.21.15.84:0/1964214453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2f00cf710 msgr2=0x7fc2dc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.232+0000 7fc2ef7fe640 1 --2- 172.21.15.84:0/1964214453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2f00cf710 0x7fc2dc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:38.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.232+0000 7fc2ef7fe640 1 -- 172.21.15.84:0/1964214453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2dc003970 msgr2=0x7fc2dc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:38.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.232+0000 7fc2ef7fe640 1 --2- 172.21.15.84:0/1964214453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2dc003970 0x7fc2dc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:38.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.232+0000 7fc2ef7fe640 1 -- 172.21.15.84:0/1964214453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc003520 con 0x7fc2dc004d00 2023-10-16T11:25:38.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.233+0000 7fc2ef7fe640 1 --2- 172.21.15.84:0/1964214453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2dc004d00 0x7fc2dc005180 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800c210 tx=0x7fc2dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7177edbcab53ea9f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:38.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.234+0000 7fc2edffb640 1 -- 172.21.15.84:0/1964214453 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e802cae0 con 0x7fc2dc004d00 2023-10-16T11:25:38.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.234+0000 7fc2edffb640 1 -- 172.21.15.84:0/1964214453 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e802cc40 con 0x7fc2dc004d00 2023-10-16T11:25:38.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.234+0000 7fc2edffb640 1 -- 172.21.15.84:0/1964214453 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e80058d0 con 0x7fc2dc004d00 2023-10-16T11:25:38.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.234+0000 7fc2f5856640 1 -- 172.21.15.84:0/1964214453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2dc004d00 msgr2=0x7fc2dc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:38.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.234+0000 7fc2f5856640 1 --2- 172.21.15.84:0/1964214453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2dc004d00 0x7fc2dc005180 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800c210 tx=0x7fc2dc0057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:38.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.234+0000 7fc2ef7fe640 1 -- 172.21.15.84:0/1964214453 reap_dead start 2023-10-16T11:25:38.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.234+0000 7fc2f5856640 1 -- 172.21.15.84:0/1964214453 shutdown_connections 2023-10-16T11:25:38.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.234+0000 7fc2f5856640 1 -- 172.21.15.84:0/1964214453 >> 172.21.15.84:0/1964214453 conn(0x7fc2f00d2cc0 msgr2=0x7fc2f00d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:38.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.234+0000 7fc2f5856640 1 -- 172.21.15.84:0/1964214453 shutdown_connections 2023-10-16T11:25:38.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.234+0000 7fc2f5856640 1 -- 172.21.15.84:0/1964214453 wait complete. 2023-10-16T11:25:38.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.236+0000 7fc2f5856640 1 Processor -- start 2023-10-16T11:25:38.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.236+0000 7fc2f5856640 1 -- start start 2023-10-16T11:25:38.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.237+0000 7fc2f5856640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2dc004d00 0x7fc2f00ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:38.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.237+0000 7fc2f5856640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2f0160330 0x7fc2f0156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:38.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.237+0000 7fc2f5856640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2f00af890 0x7fc2f00afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:38.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.237+0000 7fc2f5856640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f00ca5a0 con 0x7fc2f0160330 2023-10-16T11:25:38.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.237+0000 7fc2f5856640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f015e600 con 0x7fc2dc004d00 2023-10-16T11:25:38.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.237+0000 7fc2f5856640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f0156740 con 0x7fc2f00af890 2023-10-16T11:25:38.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.237+0000 7fc2ee7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2f0160330 0x7fc2f0156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:38.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.237+0000 7fc2ee7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2f0160330 0x7fc2f0156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47256/0 (socket says 172.21.15.84:47256) 2023-10-16T11:25:38.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.238+0000 7fc2ee7fc640 1 -- 172.21.15.84:0/2807936674 learned_addr learned my addr 172.21.15.84:0/2807936674 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:38.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.238+0000 7fc2ee7fc640 1 -- 172.21.15.84:0/2807936674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2f00af890 msgr2=0x7fc2f00afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:38.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.238+0000 7fc2ee7fc640 1 --2- 172.21.15.84:0/2807936674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2f00af890 0x7fc2f00afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:38.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.238+0000 7fc2ee7fc640 1 -- 172.21.15.84:0/2807936674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2dc004d00 msgr2=0x7fc2f00ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:38.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.238+0000 7fc2ee7fc640 1 --2- 172.21.15.84:0/2807936674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2dc004d00 0x7fc2f00ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:38.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.238+0000 7fc2ee7fc640 1 -- 172.21.15.84:0/2807936674 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f00ca7b0 con 0x7fc2f0160330 2023-10-16T11:25:38.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.238+0000 7fc2ee7fc640 1 --2- 172.21.15.84:0/2807936674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2f0160330 0x7fc2f0156200 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800f630 tx=0x7fc2d800faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:38.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.238+0000 7fc2e7fff640 1 -- 172.21.15.84:0/2807936674 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8014870 con 0x7fc2f0160330 2023-10-16T11:25:38.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.238+0000 7fc2e7fff640 1 -- 172.21.15.84:0/2807936674 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d80149d0 con 0x7fc2f0160330 2023-10-16T11:25:38.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.238+0000 7fc2e7fff640 1 -- 172.21.15.84:0/2807936674 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8014d40 con 0x7fc2f0160330 2023-10-16T11:25:38.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.238+0000 7fc2f5856640 1 -- 172.21.15.84:0/2807936674 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f00caa90 con 0x7fc2f0160330 2023-10-16T11:25:38.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.239+0000 7fc2f5856640 1 -- 172.21.15.84:0/2807936674 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f015ea90 con 0x7fc2f0160330 2023-10-16T11:25:38.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.240+0000 7fc2f5856640 1 -- 172.21.15.84:0/2807936674 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f00d2220 con 0x7fc2f0160330 2023-10-16T11:25:38.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.241+0000 7fc2e7fff640 1 -- 172.21.15.84:0/2807936674 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fc2d8015410 con 0x7fc2f0160330 2023-10-16T11:25:38.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.242+0000 7fc2e7fff640 1 --2- 172.21.15.84:0/2807936674 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc2d006b0e0 0x7fc2d006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:38.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.242+0000 7fc2e7fff640 1 -- 172.21.15.84:0/2807936674 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc2d808a420 con 0x7fc2f0160330 2023-10-16T11:25:38.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.242+0000 7fc2eeffd640 1 --2- 172.21.15.84:0/2807936674 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc2d006b0e0 0x7fc2d006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:38.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.243+0000 7fc2eeffd640 1 --2- 172.21.15.84:0/2807936674 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc2d006b0e0 0x7fc2d006d5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0007330 tx=0x7fc2e0009750 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:38.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.247+0000 7fc2e7fff640 1 -- 172.21.15.84:0/2807936674 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc2d8055150 con 0x7fc2f0160330 2023-10-16T11:25:38.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:38.481+0000 7fc2f5856640 1 -- 172.21.15.84:0/2807936674 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc2f004e160 con 0x7fc2f0160330 2023-10-16T11:25:39.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.181+0000 7fc2e7fff640 1 -- 172.21.15.84:0/2807936674 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v130) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc2d8058770 con 0x7fc2f0160330 2023-10-16T11:25:39.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.185+0000 7fc2f5856640 1 -- 172.21.15.84:0/2807936674 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc2d006b0e0 msgr2=0x7fc2d006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:39.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.185+0000 7fc2f5856640 1 --2- 172.21.15.84:0/2807936674 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc2d006b0e0 0x7fc2d006d5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0007330 tx=0x7fc2e0009750 comp rx=0 tx=0).stop 2023-10-16T11:25:39.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.185+0000 7fc2f5856640 1 -- 172.21.15.84:0/2807936674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2f0160330 msgr2=0x7fc2f0156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:39.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.185+0000 7fc2f5856640 1 --2- 172.21.15.84:0/2807936674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2f0160330 0x7fc2f0156200 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800f630 tx=0x7fc2d800faf0 comp rx=0 tx=0).stop 2023-10-16T11:25:39.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.185+0000 7fc2f5856640 1 -- 172.21.15.84:0/2807936674 shutdown_connections 2023-10-16T11:25:39.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.185+0000 7fc2f5856640 1 --2- 172.21.15.84:0/2807936674 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc2d006b0e0 0x7fc2d006d5a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:39.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.185+0000 7fc2f5856640 1 --2- 172.21.15.84:0/2807936674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2f00af890 0x7fc2f00afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:39.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.185+0000 7fc2f5856640 1 --2- 172.21.15.84:0/2807936674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2f0160330 0x7fc2f0156200 unknown :-1 s=CLOSED pgs=2923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:39.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.185+0000 7fc2f5856640 1 --2- 172.21.15.84:0/2807936674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2dc004d00 0x7fc2f00ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:39.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.185+0000 7fc2f5856640 1 -- 172.21.15.84:0/2807936674 >> 172.21.15.84:0/2807936674 conn(0x7fc2f00d2cc0 msgr2=0x7fc2f00b8000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:39.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.185+0000 7fc2f5856640 1 -- 172.21.15.84:0/2807936674 shutdown_connections 2023-10-16T11:25:39.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.185+0000 7fc2f5856640 1 -- 172.21.15.84:0/2807936674 wait complete. 2023-10-16T11:25:39.207 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T11:25:39.208 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T11:25:39.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.420+0000 7f006c878640 1 Processor -- start 2023-10-16T11:25:39.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.420+0000 7f006c878640 1 -- start start 2023-10-16T11:25:39.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.422+0000 7f006c878640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00680c9060 0x7f006806f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:39.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.422+0000 7f006c878640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00680ca560 con 0x7f00680c9440 2023-10-16T11:25:39.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.422+0000 7f006c878640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0068165b70 con 0x7f006806c9e0 2023-10-16T11:25:39.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.422+0000 7f006c878640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00680c9950 con 0x7f00680c9060 2023-10-16T11:25:39.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.423+0000 7f0067fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f00680c9440 0x7f0068165460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42190/0 (socket says 172.21.15.84:42190) 2023-10-16T11:25:39.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.423+0000 7f0067fff640 1 -- 172.21.15.84:0/3426830830 learned_addr learned my addr 172.21.15.84:0/3426830830 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:39.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.423+0000 7f00677fe640 1 --2- 172.21.15.84:0/3426830830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00680c9060 0x7f006806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:39.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.423+0000 7f0066ffd640 1 -- 172.21.15.84:0/3426830830 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203308984 0 0) 0x7f00680ca560 con 0x7f00680c9440 2023-10-16T11:25:39.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.423+0000 7f0066ffd640 1 -- 172.21.15.84:0/3426830830 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0054003520 con 0x7f00680c9440 2023-10-16T11:25:39.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.423+0000 7f0066ffd640 1 -- 172.21.15.84:0/3426830830 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1063012412 0 0) 0x7f0054003520 con 0x7f00680c9440 2023-10-16T11:25:39.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.423+0000 7f0066ffd640 1 -- 172.21.15.84:0/3426830830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00680c9060 msgr2=0x7f006806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:39.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.423+0000 7f0066ffd640 1 --2- 172.21.15.84:0/3426830830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00680c9060 0x7f006806f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:39.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.423+0000 7f0066ffd640 1 -- 172.21.15.84:0/3426830830 >> v1:172.21.15.157:6789/0 conn(0x7f006806c9e0 legacy=0x7f0068164d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:39.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.423+0000 7f0066ffd640 1 -- 172.21.15.84:0/3426830830 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00680c9c30 con 0x7f00680c9440 2023-10-16T11:25:39.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.423+0000 7f0066ffd640 1 -- 172.21.15.84:0/3426830830 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f00600019f0 con 0x7f00680c9440 2023-10-16T11:25:39.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.423+0000 7f0066ffd640 1 -- 172.21.15.84:0/3426830830 >> v1:172.21.15.84:6789/0 conn(0x7f00680c9440 legacy=0x7f0068165460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:39.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.424+0000 7f0066ffd640 1 --2- 172.21.15.84:0/3426830830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00540039a0 0x7f0054003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:39.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.424+0000 7f0066ffd640 1 --2- 172.21.15.84:0/3426830830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540047c0 0x7f0054004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:39.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.424+0000 7f0066ffd640 1 --2- 172.21.15.84:0/3426830830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00680c9060 0x7f0054005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:39.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.424+0000 7f0066ffd640 1 -- 172.21.15.84:0/3426830830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00680c9950 con 0x7f00540039a0 2023-10-16T11:25:39.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.424+0000 7f0066ffd640 1 -- 172.21.15.84:0/3426830830 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005400ba60 con 0x7f00540047c0 2023-10-16T11:25:39.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.424+0000 7f0066ffd640 1 -- 172.21.15.84:0/3426830830 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005400bc20 con 0x7f00680c9060 2023-10-16T11:25:39.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.424+0000 7f00677fe640 1 --2- 172.21.15.84:0/3426830830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00540039a0 0x7f0054003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:39.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.424+0000 7f005ffff640 1 --2- 172.21.15.84:0/3426830830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540047c0 0x7f0054004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:39.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.424+0000 7f00677fe640 1 -- 172.21.15.84:0/3426830830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00680c9060 msgr2=0x7f0054005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:39.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.424+0000 7f00677fe640 1 --2- 172.21.15.84:0/3426830830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00680c9060 0x7f0054005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:39.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.424+0000 7f00677fe640 1 -- 172.21.15.84:0/3426830830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540047c0 msgr2=0x7f0054004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:39.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.424+0000 7f00677fe640 1 --2- 172.21.15.84:0/3426830830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540047c0 0x7f0054004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:39.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.424+0000 7f00677fe640 1 -- 172.21.15.84:0/3426830830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0054003520 con 0x7f00540039a0 2023-10-16T11:25:39.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.424+0000 7f005ffff640 1 --2- 172.21.15.84:0/3426830830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540047c0 0x7f0054004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:39.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.425+0000 7f00677fe640 1 --2- 172.21.15.84:0/3426830830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00540039a0 0x7f0054003dc0 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7f0058009a90 tx=0x7f00540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a387996fa8ccdca server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:39.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.426+0000 7f0066ffd640 1 -- 172.21.15.84:0/3426830830 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005802c6b0 con 0x7f00540039a0 2023-10-16T11:25:39.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.426+0000 7f0066ffd640 1 -- 172.21.15.84:0/3426830830 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005802c810 con 0x7f00540039a0 2023-10-16T11:25:39.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.426+0000 7f0066ffd640 1 -- 172.21.15.84:0/3426830830 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005802cba0 con 0x7f00540039a0 2023-10-16T11:25:39.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.426+0000 7f006c878640 1 -- 172.21.15.84:0/3426830830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00540039a0 msgr2=0x7f0054003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:39.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.426+0000 7f006c878640 1 --2- 172.21.15.84:0/3426830830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00540039a0 0x7f0054003dc0 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7f0058009a90 tx=0x7f00540057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:39.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.426+0000 7f0067fff640 1 -- 172.21.15.84:0/3426830830 reap_dead start 2023-10-16T11:25:39.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.426+0000 7f006c878640 1 -- 172.21.15.84:0/3426830830 shutdown_connections 2023-10-16T11:25:39.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.426+0000 7f006c878640 1 -- 172.21.15.84:0/3426830830 >> 172.21.15.84:0/3426830830 conn(0x7f00680ce6a0 msgr2=0x7f00680ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:39.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.426+0000 7f006c878640 1 -- 172.21.15.84:0/3426830830 shutdown_connections 2023-10-16T11:25:39.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.426+0000 7f006c878640 1 -- 172.21.15.84:0/3426830830 wait complete. 2023-10-16T11:25:39.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.428+0000 7f006c878640 1 Processor -- start 2023-10-16T11:25:39.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.428+0000 7f006c878640 1 -- start start 2023-10-16T11:25:39.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.429+0000 7f006c878640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540039a0 0x7f00680b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:39.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.429+0000 7f006c878640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f006815b850 0x7f0068163d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:39.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.429+0000 7f006c878640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f006815bcd0 0x7f006806b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:39.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.429+0000 7f006c878640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00680c3dd0 con 0x7f006815bcd0 2023-10-16T11:25:39.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.429+0000 7f006c878640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00680c31c0 con 0x7f00540039a0 2023-10-16T11:25:39.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.429+0000 7f006c878640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00680c34a0 con 0x7f006815b850 2023-10-16T11:25:39.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.430+0000 7f0067fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f006815bcd0 0x7f006806b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.430+0000 7f0067fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f006815bcd0 0x7f006806b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47282/0 (socket says 172.21.15.84:47282) 2023-10-16T11:25:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.430+0000 7f0067fff640 1 -- 172.21.15.84:0/736069935 learned_addr learned my addr 172.21.15.84:0/736069935 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.430+0000 7f00677fe640 1 --2- 172.21.15.84:0/736069935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540039a0 0x7f00680b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.430+0000 7f005ffff640 1 --2- 172.21.15.84:0/736069935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f006815b850 0x7f0068163d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:39.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.430+0000 7f0067fff640 1 -- 172.21.15.84:0/736069935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f006815b850 msgr2=0x7f0068163d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:39.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.430+0000 7f0067fff640 1 --2- 172.21.15.84:0/736069935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f006815b850 0x7f0068163d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:39.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.430+0000 7f0067fff640 1 -- 172.21.15.84:0/736069935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540039a0 msgr2=0x7f00680b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:39.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.430+0000 7f0067fff640 1 --2- 172.21.15.84:0/736069935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540039a0 0x7f00680b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:39.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.430+0000 7f0067fff640 1 -- 172.21.15.84:0/736069935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00680c3780 con 0x7f006815bcd0 2023-10-16T11:25:39.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.430+0000 7f00677fe640 1 --2- 172.21.15.84:0/736069935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540039a0 0x7f00680b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:25:39.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.431+0000 7f0067fff640 1 --2- 172.21.15.84:0/736069935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f006815bcd0 0x7f006806b460 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7f0060001370 tx=0x7f006000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:39.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.431+0000 7f0064ff9640 1 -- 172.21.15.84:0/736069935 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006000a5b0 con 0x7f006815bcd0 2023-10-16T11:25:39.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.431+0000 7f0064ff9640 1 -- 172.21.15.84:0/736069935 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006000a710 con 0x7f006815bcd0 2023-10-16T11:25:39.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.431+0000 7f0064ff9640 1 -- 172.21.15.84:0/736069935 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006000aac0 con 0x7f006815bcd0 2023-10-16T11:25:39.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.431+0000 7f006c878640 1 -- 172.21.15.84:0/736069935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00680c3a60 con 0x7f006815bcd0 2023-10-16T11:25:39.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.431+0000 7f006c878640 1 -- 172.21.15.84:0/736069935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0068164280 con 0x7f006815bcd0 2023-10-16T11:25:39.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.432+0000 7f006c878640 1 -- 172.21.15.84:0/736069935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00680c3bf0 con 0x7f006815bcd0 2023-10-16T11:25:39.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.433+0000 7f0064ff9640 1 -- 172.21.15.84:0/736069935 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f006000ac20 con 0x7f006815bcd0 2023-10-16T11:25:39.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.434+0000 7f0064ff9640 1 --2- 172.21.15.84:0/736069935 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f004806b0b0 0x7f004806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:39.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.434+0000 7f0064ff9640 1 -- 172.21.15.84:0/736069935 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0060089280 con 0x7f006815bcd0 2023-10-16T11:25:39.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.434+0000 7f00677fe640 1 --2- 172.21.15.84:0/736069935 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f004806b0b0 0x7f004806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:39.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.435+0000 7f00677fe640 1 --2- 172.21.15.84:0/736069935 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f004806b0b0 0x7f004806d570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0058036000 tx=0x7f0058030000 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:39.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.439+0000 7f0064ff9640 1 -- 172.21.15.84:0/736069935 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f00680c3bf0 con 0x7f006815bcd0 2023-10-16T11:25:39.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:39.673+0000 7f006c878640 1 -- 172.21.15.84:0/736069935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f00680c3bf0 con 0x7f006815bcd0 2023-10-16T11:25:40.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.191+0000 7f0064ff9640 1 -- 172.21.15.84:0/736069935 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v131) v1 ==== 122+0+0 (secure 0 0 0) 0x7f00680c3bf0 con 0x7f006815bcd0 2023-10-16T11:25:40.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.194+0000 7f006c878640 1 -- 172.21.15.84:0/736069935 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f004806b0b0 msgr2=0x7f004806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:40.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.194+0000 7f006c878640 1 --2- 172.21.15.84:0/736069935 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f004806b0b0 0x7f004806d570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0058036000 tx=0x7f0058030000 comp rx=0 tx=0).stop 2023-10-16T11:25:40.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.194+0000 7f006c878640 1 -- 172.21.15.84:0/736069935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f006815bcd0 msgr2=0x7f006806b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:40.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.194+0000 7f006c878640 1 --2- 172.21.15.84:0/736069935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f006815bcd0 0x7f006806b460 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7f0060001370 tx=0x7f006000ccd0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.195+0000 7f006c878640 1 -- 172.21.15.84:0/736069935 shutdown_connections 2023-10-16T11:25:40.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.195+0000 7f006c878640 1 --2- 172.21.15.84:0/736069935 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f004806b0b0 0x7f004806d570 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.195+0000 7f006c878640 1 --2- 172.21.15.84:0/736069935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f006815bcd0 0x7f006806b460 unknown :-1 s=CLOSED pgs=2926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.195+0000 7f006c878640 1 --2- 172.21.15.84:0/736069935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f006815b850 0x7f0068163d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.195+0000 7f006c878640 1 --2- 172.21.15.84:0/736069935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540039a0 0x7f00680b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.195+0000 7f006c878640 1 -- 172.21.15.84:0/736069935 >> 172.21.15.84:0/736069935 conn(0x7f00680ce6a0 msgr2=0x7f006815f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:40.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.195+0000 7f006c878640 1 -- 172.21.15.84:0/736069935 shutdown_connections 2023-10-16T11:25:40.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.195+0000 7f006c878640 1 -- 172.21.15.84:0/736069935 wait complete. 2023-10-16T11:25:40.217 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T11:25:40.218 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:25:40.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.427+0000 7fe465f06640 1 Processor -- start 2023-10-16T11:25:40.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.428+0000 7fe465f06640 1 -- start start 2023-10-16T11:25:40.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.429+0000 7fe465f06640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe46015cfc0 0x7fe4600b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:40.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.429+0000 7fe465f06640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4600d2640 0x7fe4600b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:40.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.429+0000 7fe465f06640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4600d15a0 con 0x7fe46015cbe0 2023-10-16T11:25:40.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.429+0000 7fe465f06640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4600d1880 con 0x7fe46015cfc0 2023-10-16T11:25:40.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.429+0000 7fe465f06640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4600d1b60 con 0x7fe4600d2640 2023-10-16T11:25:40.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.430+0000 7fe45ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe46015cfc0 0x7fe4600b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:40.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.430+0000 7fe465705640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4600d2640 0x7fe4600b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:40.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.430+0000 7fe464f04640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe46015cbe0 0x7fe4600d0b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:52414/0 (socket says 172.21.15.84:52414) 2023-10-16T11:25:40.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.430+0000 7fe464f04640 1 -- 172.21.15.84:0/2160665641 learned_addr learned my addr 172.21.15.84:0/2160665641 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:40.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.431+0000 7fe45f7fe640 1 -- 172.21.15.84:0/2160665641 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831617752 0 0) 0x7fe4600d15a0 con 0x7fe46015cbe0 2023-10-16T11:25:40.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.431+0000 7fe45f7fe640 1 -- 172.21.15.84:0/2160665641 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe44c003520 con 0x7fe46015cbe0 2023-10-16T11:25:40.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.431+0000 7fe465705640 1 -- 172.21.15.84:0/2160665641 >> v2:172.21.15.157:3300/0 conn(0x7fe46015cfc0 msgr2=0x7fe4600b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:40.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.431+0000 7fe465705640 1 --2- 172.21.15.84:0/2160665641 >> v2:172.21.15.157:3300/0 conn(0x7fe46015cfc0 0x7fe4600b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.431+0000 7fe465705640 1 -- 172.21.15.84:0/2160665641 >> v1:172.21.15.157:6789/0 conn(0x7fe46015cbe0 legacy=0x7fe4600d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:40.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.431+0000 7fe465705640 1 -- 172.21.15.84:0/2160665641 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4600d1e40 con 0x7fe4600d2640 2023-10-16T11:25:40.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.432+0000 7fe465705640 1 --2- 172.21.15.84:0/2160665641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4600d2640 0x7fe4600b4ab0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe4580097a0 tx=0x7fe45802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63a43022d2193fec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:40.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.433+0000 7fe45f7fe640 1 -- 172.21.15.84:0/2160665641 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45802d040 con 0x7fe4600d2640 2023-10-16T11:25:40.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.433+0000 7fe45f7fe640 1 -- 172.21.15.84:0/2160665641 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe458031810 con 0x7fe4600d2640 2023-10-16T11:25:40.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.433+0000 7fe45f7fe640 1 -- 172.21.15.84:0/2160665641 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458031ba0 con 0x7fe4600d2640 2023-10-16T11:25:40.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.434+0000 7fe465f06640 1 -- 172.21.15.84:0/2160665641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4600d2640 msgr2=0x7fe4600b4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:40.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.434+0000 7fe465f06640 1 --2- 172.21.15.84:0/2160665641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4600d2640 0x7fe4600b4ab0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe4580097a0 tx=0x7fe45802b8b0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.434+0000 7fe465f06640 1 -- 172.21.15.84:0/2160665641 shutdown_connections 2023-10-16T11:25:40.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.434+0000 7fe465f06640 1 --2- 172.21.15.84:0/2160665641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4600d2640 0x7fe4600b4ab0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.434+0000 7fe465f06640 1 --2- 172.21.15.84:0/2160665641 >> v2:172.21.15.157:3300/0 conn(0x7fe46015cfc0 0x7fe4600b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.434+0000 7fe465f06640 1 -- 172.21.15.84:0/2160665641 >> 172.21.15.84:0/2160665641 conn(0x7fe4600cf570 msgr2=0x7fe4600cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:40.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.434+0000 7fe465f06640 1 -- 172.21.15.84:0/2160665641 shutdown_connections 2023-10-16T11:25:40.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.434+0000 7fe465f06640 1 -- 172.21.15.84:0/2160665641 wait complete. 2023-10-16T11:25:40.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.436+0000 7fe465f06640 1 Processor -- start 2023-10-16T11:25:40.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.436+0000 7fe465f06640 1 -- start start 2023-10-16T11:25:40.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.437+0000 7fe465f06640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4600d2640 0x7fe4601508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:40.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.437+0000 7fe465f06640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe46015cbe0 0x7fe4600cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:40.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.437+0000 7fe465f06640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe46015cfc0 0x7fe4600cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:40.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.437+0000 7fe465f06640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe460150e30 con 0x7fe46015cbe0 2023-10-16T11:25:40.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.437+0000 7fe465f06640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4601f21b0 con 0x7fe46015cfc0 2023-10-16T11:25:40.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.437+0000 7fe465f06640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4600cb2c0 con 0x7fe4600d2640 2023-10-16T11:25:40.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.437+0000 7fe45ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe46015cbe0 0x7fe4600cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:40.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.437+0000 7fe45ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe46015cbe0 0x7fe4600cb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47290/0 (socket says 172.21.15.84:47290) 2023-10-16T11:25:40.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.437+0000 7fe45ffff640 1 -- 172.21.15.84:0/574429914 learned_addr learned my addr 172.21.15.84:0/574429914 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:40.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.438+0000 7fe464f04640 1 --2- 172.21.15.84:0/574429914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4600d2640 0x7fe4601508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:40.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.438+0000 7fe465705640 1 --2- 172.21.15.84:0/574429914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe46015cfc0 0x7fe4600cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:40.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.438+0000 7fe45ffff640 1 -- 172.21.15.84:0/574429914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4600d2640 msgr2=0x7fe4601508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:40.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.438+0000 7fe45ffff640 1 --2- 172.21.15.84:0/574429914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4600d2640 0x7fe4601508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.438+0000 7fe45ffff640 1 -- 172.21.15.84:0/574429914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe46015cfc0 msgr2=0x7fe4600cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:40.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.438+0000 7fe45ffff640 1 --2- 172.21.15.84:0/574429914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe46015cfc0 0x7fe4600cbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.438+0000 7fe45ffff640 1 -- 172.21.15.84:0/574429914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe458009400 con 0x7fe46015cbe0 2023-10-16T11:25:40.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.438+0000 7fe465705640 1 --2- 172.21.15.84:0/574429914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe46015cfc0 0x7fe4600cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:40.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.438+0000 7fe464f04640 1 --2- 172.21.15.84:0/574429914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4600d2640 0x7fe4601508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:40.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.438+0000 7fe45ffff640 1 --2- 172.21.15.84:0/574429914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe46015cbe0 0x7fe4600cb740 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7fe44800e840 tx=0x7fe44800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:40.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.438+0000 7fe45d7fa640 1 -- 172.21.15.84:0/574429914 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4480094f0 con 0x7fe46015cbe0 2023-10-16T11:25:40.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.438+0000 7fe45d7fa640 1 -- 172.21.15.84:0/574429914 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44800c940 con 0x7fe46015cbe0 2023-10-16T11:25:40.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.438+0000 7fe465f06640 1 -- 172.21.15.84:0/574429914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4600a9670 con 0x7fe46015cbe0 2023-10-16T11:25:40.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.438+0000 7fe45d7fa640 1 -- 172.21.15.84:0/574429914 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44800ccd0 con 0x7fe46015cbe0 2023-10-16T11:25:40.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.439+0000 7fe465f06640 1 -- 172.21.15.84:0/574429914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4600a9c30 con 0x7fe46015cbe0 2023-10-16T11:25:40.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.440+0000 7fe465f06640 1 -- 172.21.15.84:0/574429914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4600a9df0 con 0x7fe46015cbe0 2023-10-16T11:25:40.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.441+0000 7fe45d7fa640 1 -- 172.21.15.84:0/574429914 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe448013470 con 0x7fe46015cbe0 2023-10-16T11:25:40.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.442+0000 7fe45d7fa640 1 --2- 172.21.15.84:0/574429914 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe44006b0b0 0x7fe44006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:40.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.442+0000 7fe45d7fa640 1 -- 172.21.15.84:0/574429914 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe448089a30 con 0x7fe46015cbe0 2023-10-16T11:25:40.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.442+0000 7fe464f04640 1 --2- 172.21.15.84:0/574429914 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe44006b0b0 0x7fe44006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:40.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.443+0000 7fe464f04640 1 --2- 172.21.15.84:0/574429914 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe44006b0b0 0x7fe44006d570 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe450001560 tx=0x7fe450009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:40.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.446+0000 7fe45d7fa640 1 -- 172.21.15.84:0/574429914 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe4480547f0 con 0x7fe46015cbe0 2023-10-16T11:25:40.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.676+0000 7fe465f06640 1 -- 172.21.15.84:0/574429914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe46004e9b0 con 0x7fe46015cbe0 2023-10-16T11:25:40.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.678+0000 7fe45d7fa640 1 -- 172.21.15.84:0/574429914 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4930 (secure 0 0 0) 0x7fe448017090 con 0x7fe46015cbe0 2023-10-16T11:25:40.680 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:40.680 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":131,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2516077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2516077743},{"type":"v1","addr":"172.21.15.157:6833","nonce":2516077743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1527104462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1527104462},{"type":"v1","addr":"172.21.15.157:6835","nonce":1527104462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":12183,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/369067539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":369067539},{"type":"v1","addr":"172.21.15.84:6835","nonce":369067539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:25:36.605068+0000","modified":"2023-10-16T11:25:39.675208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12186":{"gid":12186,"name":"a","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/3547503033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3547503033},{"type":"v1","addr":"172.21.15.84:6837","nonce":3547503033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-10-16T11:25:40.681 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 131 2023-10-16T11:25:40.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.680+0000 7fe465f06640 1 -- 172.21.15.84:0/574429914 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe44006b0b0 msgr2=0x7fe44006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:40.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.680+0000 7fe465f06640 1 --2- 172.21.15.84:0/574429914 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe44006b0b0 0x7fe44006d570 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe450001560 tx=0x7fe450009040 comp rx=0 tx=0).stop 2023-10-16T11:25:40.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.680+0000 7fe465f06640 1 -- 172.21.15.84:0/574429914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe46015cbe0 msgr2=0x7fe4600cb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:40.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.680+0000 7fe465f06640 1 --2- 172.21.15.84:0/574429914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe46015cbe0 0x7fe4600cb740 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7fe44800e840 tx=0x7fe44800ed00 comp rx=0 tx=0).stop 2023-10-16T11:25:40.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.680+0000 7fe465f06640 1 -- 172.21.15.84:0/574429914 shutdown_connections 2023-10-16T11:25:40.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.680+0000 7fe465f06640 1 --2- 172.21.15.84:0/574429914 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe44006b0b0 0x7fe44006d570 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.680+0000 7fe465f06640 1 --2- 172.21.15.84:0/574429914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe46015cfc0 0x7fe4600cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.680+0000 7fe465f06640 1 --2- 172.21.15.84:0/574429914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe46015cbe0 0x7fe4600cb740 unknown :-1 s=CLOSED pgs=2927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.680+0000 7fe465f06640 1 --2- 172.21.15.84:0/574429914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4600d2640 0x7fe4601508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.680+0000 7fe465f06640 1 -- 172.21.15.84:0/574429914 >> 172.21.15.84:0/574429914 conn(0x7fe4600cf570 msgr2=0x7fe4600b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:40.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.680+0000 7fe465f06640 1 -- 172.21.15.84:0/574429914 shutdown_connections 2023-10-16T11:25:40.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.680+0000 7fe465f06640 1 -- 172.21.15.84:0/574429914 wait complete. 2023-10-16T11:25:40.701 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:25:40.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.912+0000 7f8a8d57c640 1 Processor -- start 2023-10-16T11:25:40.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.912+0000 7f8a8d57c640 1 -- start start 2023-10-16T11:25:40.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.913+0000 7f8a8d57c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a880cd0c0 0x7f8a881615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:40.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.913+0000 7f8a8d57c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8a880b0410 0x7f8a880b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:40.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.913+0000 7f8a8d57c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a880b0db0 con 0x7f8a8815d3f0 2023-10-16T11:25:40.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.913+0000 7f8a8d57c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a8815e3e0 con 0x7f8a880b0410 2023-10-16T11:25:40.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.913+0000 7f8a8d57c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880cd4a0 con 0x7f8a880cd0c0 2023-10-16T11:25:40.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.913+0000 7f8a867fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8a880b0410 0x7f8a880b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:40.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.914+0000 7f8a867fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8a880b0410 0x7f8a880b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47292/0 (socket says 172.21.15.84:47292) 2023-10-16T11:25:40.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.914+0000 7f8a867fc640 1 -- 172.21.15.84:0/3000887885 learned_addr learned my addr 172.21.15.84:0/3000887885 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:40.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.914+0000 7f8a867fc640 1 -- 172.21.15.84:0/3000887885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a880cd0c0 msgr2=0x7f8a881615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:40.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.914+0000 7f8a867fc640 1 --2- 172.21.15.84:0/3000887885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a880cd0c0 0x7f8a881615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.914+0000 7f8a867fc640 1 -- 172.21.15.84:0/3000887885 >> v1:172.21.15.157:6789/0 conn(0x7f8a8815d3f0 legacy=0x7f8a8815d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:40.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.914+0000 7f8a867fc640 1 -- 172.21.15.84:0/3000887885 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8815e100 con 0x7f8a880b0410 2023-10-16T11:25:40.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.915+0000 7f8a867fc640 1 --2- 172.21.15.84:0/3000887885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a880b0410 0x7f8a880b0870 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009e50 tx=0x7f8a7802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1ae5ed5971cc44 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:40.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.915+0000 7f8a85ffb640 1 -- 172.21.15.84:0/3000887885 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7802c9e0 con 0x7f8a880b0410 2023-10-16T11:25:40.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.915+0000 7f8a85ffb640 1 -- 172.21.15.84:0/3000887885 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7802cb40 con 0x7f8a880b0410 2023-10-16T11:25:40.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.915+0000 7f8a85ffb640 1 -- 172.21.15.84:0/3000887885 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a780343f0 con 0x7f8a880b0410 2023-10-16T11:25:40.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.916+0000 7f8a8d57c640 1 -- 172.21.15.84:0/3000887885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a880b0410 msgr2=0x7f8a880b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:40.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.916+0000 7f8a8d57c640 1 --2- 172.21.15.84:0/3000887885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a880b0410 0x7f8a880b0870 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009e50 tx=0x7f8a7802be40 comp rx=0 tx=0).stop 2023-10-16T11:25:40.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.916+0000 7f8a8d57c640 1 -- 172.21.15.84:0/3000887885 shutdown_connections 2023-10-16T11:25:40.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.916+0000 7f8a8d57c640 1 --2- 172.21.15.84:0/3000887885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a880b0410 0x7f8a880b0870 unknown :-1 s=CLOSED pgs=2928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.916+0000 7f8a8d57c640 1 --2- 172.21.15.84:0/3000887885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a880cd0c0 0x7f8a881615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.916+0000 7f8a8d57c640 1 -- 172.21.15.84:0/3000887885 >> 172.21.15.84:0/3000887885 conn(0x7f8a880cb410 msgr2=0x7f8a880cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:40.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.916+0000 7f8a8d57c640 1 -- 172.21.15.84:0/3000887885 shutdown_connections 2023-10-16T11:25:40.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.916+0000 7f8a8d57c640 1 -- 172.21.15.84:0/3000887885 wait complete. 2023-10-16T11:25:40.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.917+0000 7f8a8d57c640 1 Processor -- start 2023-10-16T11:25:40.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.918+0000 7f8a8d57c640 1 -- start start 2023-10-16T11:25:40.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.918+0000 7f8a8d57c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a880b0410 0x7f8a881562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:40.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.918+0000 7f8a8d57c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a880cd0c0 0x7f8a88156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:40.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.918+0000 7f8a8d57c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a8815d3f0 0x7f8a88156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:40.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.918+0000 7f8a8d57c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880c4ca0 con 0x7f8a8815d3f0 2023-10-16T11:25:40.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.918+0000 7f8a8d57c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880b3e90 con 0x7f8a880b0410 2023-10-16T11:25:40.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.918+0000 7f8a8d57c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880b3880 con 0x7f8a880cd0c0 2023-10-16T11:25:40.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.919+0000 7f8a877fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a8815d3f0 0x7f8a88156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:40.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.919+0000 7f8a877fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a8815d3f0 0x7f8a88156e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47296/0 (socket says 172.21.15.84:47296) 2023-10-16T11:25:40.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.919+0000 7f8a877fe640 1 -- 172.21.15.84:0/890470760 learned_addr learned my addr 172.21.15.84:0/890470760 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:40.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.919+0000 7f8a867fc640 1 --2- 172.21.15.84:0/890470760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a880cd0c0 0x7f8a88156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:40.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.919+0000 7f8a86ffd640 1 --2- 172.21.15.84:0/890470760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a880b0410 0x7f8a881562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:40.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.919+0000 7f8a877fe640 1 -- 172.21.15.84:0/890470760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a880cd0c0 msgr2=0x7f8a88156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:40.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.919+0000 7f8a877fe640 1 --2- 172.21.15.84:0/890470760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a880cd0c0 0x7f8a88156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.919+0000 7f8a877fe640 1 -- 172.21.15.84:0/890470760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a880b0410 msgr2=0x7f8a881562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:40.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.919+0000 7f8a877fe640 1 --2- 172.21.15.84:0/890470760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a880b0410 0x7f8a881562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:40.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.919+0000 7f8a877fe640 1 -- 172.21.15.84:0/890470760 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a78009b30 con 0x7f8a8815d3f0 2023-10-16T11:25:40.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.919+0000 7f8a867fc640 1 --2- 172.21.15.84:0/890470760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a880cd0c0 0x7f8a88156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:40.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.919+0000 7f8a86ffd640 1 --2- 172.21.15.84:0/890470760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a880b0410 0x7f8a881562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:40.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.919+0000 7f8a877fe640 1 --2- 172.21.15.84:0/890470760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a8815d3f0 0x7f8a88156e90 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000c3c0 tx=0x7f8a8000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:40.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.920+0000 7f8a7f7fe640 1 -- 172.21.15.84:0/890470760 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a800035f0 con 0x7f8a8815d3f0 2023-10-16T11:25:40.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.920+0000 7f8a7f7fe640 1 -- 172.21.15.84:0/890470760 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a80003750 con 0x7f8a8815d3f0 2023-10-16T11:25:40.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.920+0000 7f8a7f7fe640 1 -- 172.21.15.84:0/890470760 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80003b00 con 0x7f8a8815d3f0 2023-10-16T11:25:40.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.920+0000 7f8a8d57c640 1 -- 172.21.15.84:0/890470760 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a880b4320 con 0x7f8a8815d3f0 2023-10-16T11:25:40.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.920+0000 7f8a8d57c640 1 -- 172.21.15.84:0/890470760 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a880c4ef0 con 0x7f8a8815d3f0 2023-10-16T11:25:40.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.921+0000 7f8a8d57c640 1 -- 172.21.15.84:0/890470760 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8804e160 con 0x7f8a8815d3f0 2023-10-16T11:25:40.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.922+0000 7f8a7f7fe640 1 -- 172.21.15.84:0/890470760 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f8a80003c60 con 0x7f8a8815d3f0 2023-10-16T11:25:40.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.923+0000 7f8a7f7fe640 1 --2- 172.21.15.84:0/890470760 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8a6806b0b0 0x7f8a6806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:40.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.923+0000 7f8a7f7fe640 1 -- 172.21.15.84:0/890470760 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f8a80088b80 con 0x7f8a8815d3f0 2023-10-16T11:25:40.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.923+0000 7f8a86ffd640 1 --2- 172.21.15.84:0/890470760 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8a6806b0b0 0x7f8a6806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:40.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.923+0000 7f8a86ffd640 1 --2- 172.21.15.84:0/890470760 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8a6806b0b0 0x7f8a6806d570 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8a70006fd0 tx=0x7f8a70008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:40.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:40.927+0000 7f8a7f7fe640 1 -- 172.21.15.84:0/890470760 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8a800538b0 con 0x7f8a8815d3f0 2023-10-16T11:25:41.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.116+0000 7f8a8d57c640 1 -- 172.21.15.84:0/890470760 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a880b3fd0 con 0x7f8a8815d3f0 2023-10-16T11:25:41.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.119+0000 7f8a7f7fe640 1 -- 172.21.15.84:0/890470760 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v237) v1 ==== 74+0+16539 (secure 0 0 0) 0x7f8a80017090 con 0x7f8a8815d3f0 2023-10-16T11:25:41.121 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:41.121 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":237,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:25:38.157428+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:25:34.443897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:25:35.635615+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:25:41.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.121+0000 7f8a8d57c640 1 -- 172.21.15.84:0/890470760 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8a6806b0b0 msgr2=0x7f8a6806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:41.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.121+0000 7f8a8d57c640 1 --2- 172.21.15.84:0/890470760 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8a6806b0b0 0x7f8a6806d570 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8a70006fd0 tx=0x7f8a70008040 comp rx=0 tx=0).stop 2023-10-16T11:25:41.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.121+0000 7f8a8d57c640 1 -- 172.21.15.84:0/890470760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a8815d3f0 msgr2=0x7f8a88156e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:41.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.121+0000 7f8a8d57c640 1 --2- 172.21.15.84:0/890470760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a8815d3f0 0x7f8a88156e90 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000c3c0 tx=0x7f8a8000c880 comp rx=0 tx=0).stop 2023-10-16T11:25:41.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.122+0000 7f8a8d57c640 1 -- 172.21.15.84:0/890470760 shutdown_connections 2023-10-16T11:25:41.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.122+0000 7f8a8d57c640 1 --2- 172.21.15.84:0/890470760 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8a6806b0b0 0x7f8a6806d570 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.122+0000 7f8a8d57c640 1 --2- 172.21.15.84:0/890470760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a8815d3f0 0x7f8a88156e90 unknown :-1 s=CLOSED pgs=2929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.122+0000 7f8a8d57c640 1 --2- 172.21.15.84:0/890470760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a880cd0c0 0x7f8a88156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.122+0000 7f8a8d57c640 1 --2- 172.21.15.84:0/890470760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a880b0410 0x7f8a881562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.122+0000 7f8a8d57c640 1 -- 172.21.15.84:0/890470760 >> 172.21.15.84:0/890470760 conn(0x7f8a880cb410 msgr2=0x7f8a880cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:41.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.122+0000 7f8a8d57c640 1 -- 172.21.15.84:0/890470760 shutdown_connections 2023-10-16T11:25:41.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.122+0000 7f8a8d57c640 1 -- 172.21.15.84:0/890470760 wait complete. 2023-10-16T11:25:41.141 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T11:25:41.142 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:25:41.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.352+0000 7fb4ba07c640 1 Processor -- start 2023-10-16T11:25:41.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.352+0000 7fb4ba07c640 1 -- start start 2023-10-16T11:25:41.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.353+0000 7fb4ba07c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb4b40ad970 0x7fb4b40cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:41.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.354+0000 7fb4ba07c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b415b9d0 con 0x7fb4b406c040 2023-10-16T11:25:41.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.354+0000 7fb4ba07c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b415bcb0 con 0x7fb4b40ccfb0 2023-10-16T11:25:41.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.354+0000 7fb4ba07c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b415bf90 con 0x7fb4b40ad970 2023-10-16T11:25:41.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.354+0000 7fb4b907a640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb4b406c040 0x7fb4b406c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42200/0 (socket says 172.21.15.84:42200) 2023-10-16T11:25:41.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.354+0000 7fb4b907a640 1 -- 172.21.15.84:0/195646313 learned_addr learned my addr 172.21.15.84:0/195646313 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:41.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.354+0000 7fb4a7fff640 1 -- 172.21.15.84:0/195646313 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3984726308 0 0) 0x7fb4b415b9d0 con 0x7fb4b406c040 2023-10-16T11:25:41.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.354+0000 7fb4a7fff640 1 -- 172.21.15.84:0/195646313 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4a0003520 con 0x7fb4b406c040 2023-10-16T11:25:41.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4a7fff640 1 -- 172.21.15.84:0/195646313 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2103442390 0 0) 0x7fb4a0003520 con 0x7fb4b406c040 2023-10-16T11:25:41.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4a7fff640 1 -- 172.21.15.84:0/195646313 >> v2:172.21.15.157:3300/0 conn(0x7fb4b40ad970 msgr2=0x7fb4b40cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:41.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4a7fff640 1 --2- 172.21.15.84:0/195646313 >> v2:172.21.15.157:3300/0 conn(0x7fb4b40ad970 0x7fb4b40cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4a7fff640 1 -- 172.21.15.84:0/195646313 >> v1:172.21.15.157:6789/0 conn(0x7fb4b40ccfb0 legacy=0x7fb4b40cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:41.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4a7fff640 1 -- 172.21.15.84:0/195646313 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b40cc270 con 0x7fb4b406c040 2023-10-16T11:25:41.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4a7fff640 1 -- 172.21.15.84:0/195646313 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fb4a8001670 con 0x7fb4b406c040 2023-10-16T11:25:41.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4a7fff640 1 -- 172.21.15.84:0/195646313 >> v1:172.21.15.84:6789/0 conn(0x7fb4b406c040 legacy=0x7fb4b406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:41.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4a7fff640 1 --2- 172.21.15.84:0/195646313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4a0003970 0x7fb4a0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:41.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4a7fff640 1 --2- 172.21.15.84:0/195646313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a00047c0 0x7fb4a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:41.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4a7fff640 1 --2- 172.21.15.84:0/195646313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4a00052a0 0x7fb4a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:41.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4a7fff640 1 -- 172.21.15.84:0/195646313 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b415bf90 con 0x7fb4a00047c0 2023-10-16T11:25:41.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4a7fff640 1 -- 172.21.15.84:0/195646313 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a000c1a0 con 0x7fb4a0003970 2023-10-16T11:25:41.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4a7fff640 1 -- 172.21.15.84:0/195646313 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a000c360 con 0x7fb4a00052a0 2023-10-16T11:25:41.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4b8879640 1 --2- 172.21.15.84:0/195646313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a00047c0 0x7fb4a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:41.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4b907a640 1 --2- 172.21.15.84:0/195646313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4a0003970 0x7fb4a0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:41.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4b8879640 1 -- 172.21.15.84:0/195646313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4a00052a0 msgr2=0x7fb4a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:41.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4b8879640 1 --2- 172.21.15.84:0/195646313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4a00052a0 0x7fb4a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4b8879640 1 -- 172.21.15.84:0/195646313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4a0003970 msgr2=0x7fb4a0003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:41.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4b8879640 1 --2- 172.21.15.84:0/195646313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4a0003970 0x7fb4a0003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4b8879640 1 -- 172.21.15.84:0/195646313 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a0003520 con 0x7fb4a00047c0 2023-10-16T11:25:41.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.355+0000 7fb4b987b640 1 -- 172.21.15.84:0/195646313 reap_dead start 2023-10-16T11:25:41.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.356+0000 7fb4b8879640 1 --2- 172.21.15.84:0/195646313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a00047c0 0x7fb4a0004c40 secure :-1 s=READY pgs=2931 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40ade20 tx=0x7fb4a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7e12b394075b37 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:41.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.357+0000 7fb4a7fff640 1 -- 172.21.15.84:0/195646313 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b002cce0 con 0x7fb4a00047c0 2023-10-16T11:25:41.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.357+0000 7fb4a7fff640 1 -- 172.21.15.84:0/195646313 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b002ce60 con 0x7fb4a00047c0 2023-10-16T11:25:41.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.357+0000 7fb4a7fff640 1 -- 172.21.15.84:0/195646313 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0034680 con 0x7fb4a00047c0 2023-10-16T11:25:41.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.357+0000 7fb4ba07c640 1 -- 172.21.15.84:0/195646313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a00047c0 msgr2=0x7fb4a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:41.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.357+0000 7fb4ba07c640 1 --2- 172.21.15.84:0/195646313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a00047c0 0x7fb4a0004c40 secure :-1 s=READY pgs=2931 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40ade20 tx=0x7fb4a00057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.357+0000 7fb4ba07c640 1 -- 172.21.15.84:0/195646313 shutdown_connections 2023-10-16T11:25:41.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.357+0000 7fb4ba07c640 1 --2- 172.21.15.84:0/195646313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a00047c0 0x7fb4a0004c40 unknown :-1 s=CLOSED pgs=2931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.357+0000 7fb4ba07c640 1 -- 172.21.15.84:0/195646313 >> 172.21.15.84:0/195646313 conn(0x7fb4b40ce6a0 msgr2=0x7fb4b40ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:41.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.357+0000 7fb4ba07c640 1 -- 172.21.15.84:0/195646313 shutdown_connections 2023-10-16T11:25:41.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.357+0000 7fb4ba07c640 1 -- 172.21.15.84:0/195646313 wait complete. 2023-10-16T11:25:41.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.359+0000 7fb4ba07c640 1 Processor -- start 2023-10-16T11:25:41.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.359+0000 7fb4ba07c640 1 -- start start 2023-10-16T11:25:41.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.360+0000 7fb4ba07c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4a00047c0 0x7fb4b40b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:41.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.360+0000 7fb4ba07c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4b414ac90 0x7fb4b415c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:41.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.360+0000 7fb4ba07c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4b415e7d0 0x7fb4b415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:41.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.360+0000 7fb4ba07c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b415f190 con 0x7fb4b415e7d0 2023-10-16T11:25:41.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.360+0000 7fb4ba07c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b415c710 con 0x7fb4a00047c0 2023-10-16T11:25:41.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.360+0000 7fb4ba07c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b40b2cd0 con 0x7fb4b414ac90 2023-10-16T11:25:41.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.360+0000 7fb4b987b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4b415e7d0 0x7fb4b415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:41.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.360+0000 7fb4b987b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4b415e7d0 0x7fb4b415ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47310/0 (socket says 172.21.15.84:47310) 2023-10-16T11:25:41.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.360+0000 7fb4b987b640 1 -- 172.21.15.84:0/3944934734 learned_addr learned my addr 172.21.15.84:0/3944934734 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:41.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.361+0000 7fb4b907a640 1 --2- 172.21.15.84:0/3944934734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4a00047c0 0x7fb4b40b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:41.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.361+0000 7fb4b987b640 1 -- 172.21.15.84:0/3944934734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4b414ac90 msgr2=0x7fb4b415c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:41.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.361+0000 7fb4b8879640 1 --2- 172.21.15.84:0/3944934734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4b414ac90 0x7fb4b415c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:41.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.361+0000 7fb4b987b640 1 --2- 172.21.15.84:0/3944934734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4b414ac90 0x7fb4b415c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.361+0000 7fb4b987b640 1 -- 172.21.15.84:0/3944934734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4a00047c0 msgr2=0x7fb4b40b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:41.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.361+0000 7fb4b987b640 1 --2- 172.21.15.84:0/3944934734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4a00047c0 0x7fb4b40b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.361+0000 7fb4b987b640 1 -- 172.21.15.84:0/3944934734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b415de60 con 0x7fb4b415e7d0 2023-10-16T11:25:41.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.361+0000 7fb4b907a640 1 --2- 172.21.15.84:0/3944934734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4a00047c0 0x7fb4b40b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:41.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.361+0000 7fb4b987b640 1 --2- 172.21.15.84:0/3944934734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4b415e7d0 0x7fb4b415ec50 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40cd390 tx=0x7fb4ac00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:41.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.361+0000 7fb4a5ffb640 1 -- 172.21.15.84:0/3944934734 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac0072e0 con 0x7fb4b415e7d0 2023-10-16T11:25:41.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.361+0000 7fb4a5ffb640 1 -- 172.21.15.84:0/3944934734 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac007440 con 0x7fb4b415e7d0 2023-10-16T11:25:41.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.361+0000 7fb4a5ffb640 1 -- 172.21.15.84:0/3944934734 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac0057c0 con 0x7fb4b415e7d0 2023-10-16T11:25:41.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.361+0000 7fb4ba07c640 1 -- 172.21.15.84:0/3944934734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b40b2f30 con 0x7fb4b415e7d0 2023-10-16T11:25:41.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.362+0000 7fb4ba07c640 1 -- 172.21.15.84:0/3944934734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b40b34f0 con 0x7fb4b415e7d0 2023-10-16T11:25:41.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.363+0000 7fb4ba07c640 1 -- 172.21.15.84:0/3944934734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b406b330 con 0x7fb4b415e7d0 2023-10-16T11:25:41.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.363+0000 7fb4a5ffb640 1 -- 172.21.15.84:0/3944934734 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fb4ac005920 con 0x7fb4b415e7d0 2023-10-16T11:25:41.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.364+0000 7fb4a5ffb640 1 --2- 172.21.15.84:0/3944934734 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fb49806b0b0 0x7fb49806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:41.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.364+0000 7fb4a5ffb640 1 -- 172.21.15.84:0/3944934734 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb4ac088c80 con 0x7fb4b415e7d0 2023-10-16T11:25:41.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.365+0000 7fb4b907a640 1 --2- 172.21.15.84:0/3944934734 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fb49806b0b0 0x7fb49806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:41.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.365+0000 7fb4b907a640 1 --2- 172.21.15.84:0/3944934734 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fb49806b0b0 0x7fb49806d570 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8000f60 tx=0x7fb4a8000ed0 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:41.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.369+0000 7fb4a5ffb640 1 -- 172.21.15.84:0/3944934734 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb4ac057430 con 0x7fb4b415e7d0 2023-10-16T11:25:41.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.543+0000 7fb4ba07c640 1 -- 172.21.15.84:0/3944934734 --> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4b40cc550 con 0x7fb49806b0b0 2023-10-16T11:25:41.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.559+0000 7fb4a5ffb640 1 -- 172.21.15.84:0/3944934734 <== mgr.11814 v2:172.21.15.84:6832/2963529201 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287369 (secure 0 0 0) 0x7fb4a8005050 con 0x7fb49806b0b0 2023-10-16T11:25:41.565 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:41.567 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:25:41.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.565+0000 7fb4ba07c640 1 -- 172.21.15.84:0/3944934734 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fb49806b0b0 msgr2=0x7fb49806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:41.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.565+0000 7fb4ba07c640 1 --2- 172.21.15.84:0/3944934734 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fb49806b0b0 0x7fb49806d570 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8000f60 tx=0x7fb4a8000ed0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.565+0000 7fb4ba07c640 1 -- 172.21.15.84:0/3944934734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4b415e7d0 msgr2=0x7fb4b415ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:41.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.566+0000 7fb4ba07c640 1 --2- 172.21.15.84:0/3944934734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4b415e7d0 0x7fb4b415ec50 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40cd390 tx=0x7fb4ac00c820 comp rx=0 tx=0).stop 2023-10-16T11:25:41.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.566+0000 7fb4ba07c640 1 -- 172.21.15.84:0/3944934734 shutdown_connections 2023-10-16T11:25:41.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.566+0000 7fb4ba07c640 1 --2- 172.21.15.84:0/3944934734 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fb49806b0b0 0x7fb49806d570 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.566+0000 7fb4ba07c640 1 --2- 172.21.15.84:0/3944934734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4b415e7d0 0x7fb4b415ec50 unknown :-1 s=CLOSED pgs=2932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.566+0000 7fb4ba07c640 1 --2- 172.21.15.84:0/3944934734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4b414ac90 0x7fb4b415c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.566+0000 7fb4ba07c640 1 --2- 172.21.15.84:0/3944934734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4a00047c0 0x7fb4b40b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.566+0000 7fb4ba07c640 1 -- 172.21.15.84:0/3944934734 >> 172.21.15.84:0/3944934734 conn(0x7fb4b40ce6a0 msgr2=0x7fb4b40c5770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:41.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.566+0000 7fb4ba07c640 1 -- 172.21.15.84:0/3944934734 shutdown_connections 2023-10-16T11:25:41.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.566+0000 7fb4ba07c640 1 -- 172.21.15.84:0/3944934734 wait complete. 2023-10-16T11:25:41.586 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-10-16T11:25:41.243466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":665,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2827,"ondisk_log_size":2827,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":412816,"kb_used_data":24552,"kb_used_omap":12,"kb_used_meta":388147,"kb_avail":754561904,"statfs":{"total":773094113280,"available":772671389696,"internally_reserved":0,"allocated":25141248,"data_stored":12828940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":397463097},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001469"},"pg_stats":[{"pgid":"31.3f","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.194090+0000","last_change":"2023-10-16T11:25:37.173492+0000","last_active":"2023-10-16T11:25:39.194090+0000","last_peered":"2023-10-16T11:25:39.194090+0000","last_clean":"2023-10-16T11:25:39.194090+0000","last_became_active":"2023-10-16T11:25:37.167017+0000","last_became_peered":"2023-10-16T11:25:37.167017+0000","last_unstale":"2023-10-16T11:25:39.194090+0000","last_undegraded":"2023-10-16T11:25:39.194090+0000","last_fullsized":"2023-10-16T11:25:39.194090+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:07.791499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.3e","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193814+0000","last_change":"2023-10-16T11:25:37.179117+0000","last_active":"2023-10-16T11:25:39.193814+0000","last_peered":"2023-10-16T11:25:39.193814+0000","last_clean":"2023-10-16T11:25:39.193814+0000","last_became_active":"2023-10-16T11:25:37.178849+0000","last_became_peered":"2023-10-16T11:25:37.178849+0000","last_unstale":"2023-10-16T11:25:39.193814+0000","last_undegraded":"2023-10-16T11:25:39.193814+0000","last_fullsized":"2023-10-16T11:25:39.193814+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:18:57.278249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.3d","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616308+0000","last_change":"2023-10-16T11:25:37.179065+0000","last_active":"2023-10-16T11:25:38.616308+0000","last_peered":"2023-10-16T11:25:38.616308+0000","last_clean":"2023-10-16T11:25:38.616308+0000","last_became_active":"2023-10-16T11:25:37.178779+0000","last_became_peered":"2023-10-16T11:25:37.178779+0000","last_unstale":"2023-10-16T11:25:38.616308+0000","last_undegraded":"2023-10-16T11:25:38.616308+0000","last_fullsized":"2023-10-16T11:25:38.616308+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:09:47.367715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3c","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.171558+0000","last_change":"2023-10-16T11:25:37.172480+0000","last_active":"2023-10-16T11:25:38.171558+0000","last_peered":"2023-10-16T11:25:38.171558+0000","last_clean":"2023-10-16T11:25:38.171558+0000","last_became_active":"2023-10-16T11:25:37.172192+0000","last_became_peered":"2023-10-16T11:25:37.172192+0000","last_unstale":"2023-10-16T11:25:38.171558+0000","last_undegraded":"2023-10-16T11:25:38.171558+0000","last_fullsized":"2023-10-16T11:25:38.171558+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.3b","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.197487+0000","last_change":"2023-10-16T11:25:37.177505+0000","last_active":"2023-10-16T11:25:39.197487+0000","last_peered":"2023-10-16T11:25:39.197487+0000","last_clean":"2023-10-16T11:25:39.197487+0000","last_became_active":"2023-10-16T11:25:37.174334+0000","last_became_peered":"2023-10-16T11:25:37.174334+0000","last_unstale":"2023-10-16T11:25:39.197487+0000","last_undegraded":"2023-10-16T11:25:39.197487+0000","last_fullsized":"2023-10-16T11:25:39.197487+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.3a","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.171518+0000","last_change":"2023-10-16T11:25:37.177781+0000","last_active":"2023-10-16T11:25:38.171518+0000","last_peered":"2023-10-16T11:25:38.171518+0000","last_clean":"2023-10-16T11:25:38.171518+0000","last_became_active":"2023-10-16T11:25:37.177501+0000","last_became_peered":"2023-10-16T11:25:37.177501+0000","last_unstale":"2023-10-16T11:25:38.171518+0000","last_undegraded":"2023-10-16T11:25:38.171518+0000","last_fullsized":"2023-10-16T11:25:38.171518+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.39","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.173719+0000","last_change":"2023-10-16T11:25:37.182745+0000","last_active":"2023-10-16T11:25:38.173719+0000","last_peered":"2023-10-16T11:25:38.173719+0000","last_clean":"2023-10-16T11:25:38.173719+0000","last_became_active":"2023-10-16T11:25:37.182437+0000","last_became_peered":"2023-10-16T11:25:37.182437+0000","last_unstale":"2023-10-16T11:25:38.173719+0000","last_undegraded":"2023-10-16T11:25:38.173719+0000","last_fullsized":"2023-10-16T11:25:38.173719+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.38","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.197683+0000","last_change":"2023-10-16T11:25:37.174929+0000","last_active":"2023-10-16T11:25:39.197683+0000","last_peered":"2023-10-16T11:25:39.197683+0000","last_clean":"2023-10-16T11:25:39.197683+0000","last_became_active":"2023-10-16T11:25:37.172810+0000","last_became_peered":"2023-10-16T11:25:37.172810+0000","last_unstale":"2023-10-16T11:25:39.197683+0000","last_undegraded":"2023-10-16T11:25:39.197683+0000","last_fullsized":"2023-10-16T11:25:39.197683+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.37","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.175224+0000","last_change":"2023-10-16T11:25:37.177647+0000","last_active":"2023-10-16T11:25:38.175224+0000","last_peered":"2023-10-16T11:25:38.175224+0000","last_clean":"2023-10-16T11:25:38.175224+0000","last_became_active":"2023-10-16T11:25:37.176875+0000","last_became_peered":"2023-10-16T11:25:37.176875+0000","last_unstale":"2023-10-16T11:25:38.175224+0000","last_undegraded":"2023-10-16T11:25:38.175224+0000","last_fullsized":"2023-10-16T11:25:38.175224+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:21.630921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.36","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174687+0000","last_change":"2023-10-16T11:25:37.172841+0000","last_active":"2023-10-16T11:25:38.174687+0000","last_peered":"2023-10-16T11:25:38.174687+0000","last_clean":"2023-10-16T11:25:38.174687+0000","last_became_active":"2023-10-16T11:25:37.172550+0000","last_became_peered":"2023-10-16T11:25:37.172550+0000","last_unstale":"2023-10-16T11:25:38.174687+0000","last_undegraded":"2023-10-16T11:25:38.174687+0000","last_fullsized":"2023-10-16T11:25:38.174687+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.35","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.183071+0000","last_change":"2023-10-16T11:25:37.181386+0000","last_active":"2023-10-16T11:25:38.183071+0000","last_peered":"2023-10-16T11:25:38.183071+0000","last_clean":"2023-10-16T11:25:38.183071+0000","last_became_active":"2023-10-16T11:25:37.181007+0000","last_became_peered":"2023-10-16T11:25:37.181007+0000","last_unstale":"2023-10-16T11:25:38.183071+0000","last_undegraded":"2023-10-16T11:25:38.183071+0000","last_fullsized":"2023-10-16T11:25:38.183071+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:46.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.34","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.198283+0000","last_change":"2023-10-16T11:25:37.179103+0000","last_active":"2023-10-16T11:25:39.198283+0000","last_peered":"2023-10-16T11:25:39.198283+0000","last_clean":"2023-10-16T11:25:39.198283+0000","last_became_active":"2023-10-16T11:25:37.178445+0000","last_became_peered":"2023-10-16T11:25:37.178445+0000","last_unstale":"2023-10-16T11:25:39.198283+0000","last_undegraded":"2023-10-16T11:25:39.198283+0000","last_fullsized":"2023-10-16T11:25:39.198283+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:38.218704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.33","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.175976+0000","last_change":"2023-10-16T11:25:37.182921+0000","last_active":"2023-10-16T11:25:38.175976+0000","last_peered":"2023-10-16T11:25:38.175976+0000","last_clean":"2023-10-16T11:25:38.175976+0000","last_became_active":"2023-10-16T11:25:37.182603+0000","last_became_peered":"2023-10-16T11:25:37.182603+0000","last_unstale":"2023-10-16T11:25:38.175976+0000","last_undegraded":"2023-10-16T11:25:38.175976+0000","last_fullsized":"2023-10-16T11:25:38.175976+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.32","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193727+0000","last_change":"2023-10-16T11:25:37.171715+0000","last_active":"2023-10-16T11:25:39.193727+0000","last_peered":"2023-10-16T11:25:39.193727+0000","last_clean":"2023-10-16T11:25:39.193727+0000","last_became_active":"2023-10-16T11:25:37.171421+0000","last_became_peered":"2023-10-16T11:25:37.171421+0000","last_unstale":"2023-10-16T11:25:39.193727+0000","last_undegraded":"2023-10-16T11:25:39.193727+0000","last_fullsized":"2023-10-16T11:25:39.193727+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:10.207081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.31","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174027+0000","last_change":"2023-10-16T11:25:37.177928+0000","last_active":"2023-10-16T11:25:38.174027+0000","last_peered":"2023-10-16T11:25:38.174027+0000","last_clean":"2023-10-16T11:25:38.174027+0000","last_became_active":"2023-10-16T11:25:37.177612+0000","last_became_peered":"2023-10-16T11:25:37.177612+0000","last_unstale":"2023-10-16T11:25:38.174027+0000","last_undegraded":"2023-10-16T11:25:38.174027+0000","last_fullsized":"2023-10-16T11:25:38.174027+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:44.826687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.30","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.615796+0000","last_change":"2023-10-16T11:25:37.170292+0000","last_active":"2023-10-16T11:25:38.615796+0000","last_peered":"2023-10-16T11:25:38.615796+0000","last_clean":"2023-10-16T11:25:38.615796+0000","last_became_active":"2023-10-16T11:25:37.169997+0000","last_became_peered":"2023-10-16T11:25:37.169997+0000","last_unstale":"2023-10-16T11:25:38.615796+0000","last_undegraded":"2023-10-16T11:25:38.615796+0000","last_fullsized":"2023-10-16T11:25:38.615796+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:09.329721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2f","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.198740+0000","last_change":"2023-10-16T11:25:37.162923+0000","last_active":"2023-10-16T11:25:39.198740+0000","last_peered":"2023-10-16T11:25:39.198740+0000","last_clean":"2023-10-16T11:25:39.198740+0000","last_became_active":"2023-10-16T11:25:37.162649+0000","last_became_peered":"2023-10-16T11:25:37.162649+0000","last_unstale":"2023-10-16T11:25:39.198740+0000","last_undegraded":"2023-10-16T11:25:39.198740+0000","last_fullsized":"2023-10-16T11:25:39.198740+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:46.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2e","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193747+0000","last_change":"2023-10-16T11:25:37.174044+0000","last_active":"2023-10-16T11:25:39.193747+0000","last_peered":"2023-10-16T11:25:39.193747+0000","last_clean":"2023-10-16T11:25:39.193747+0000","last_became_active":"2023-10-16T11:25:37.166694+0000","last_became_peered":"2023-10-16T11:25:37.166694+0000","last_unstale":"2023-10-16T11:25:39.193747+0000","last_undegraded":"2023-10-16T11:25:39.193747+0000","last_fullsized":"2023-10-16T11:25:39.193747+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:23.146288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2d","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.197203+0000","last_change":"2023-10-16T11:25:37.183893+0000","last_active":"2023-10-16T11:25:39.197203+0000","last_peered":"2023-10-16T11:25:39.197203+0000","last_clean":"2023-10-16T11:25:39.197203+0000","last_became_active":"2023-10-16T11:25:37.183586+0000","last_became_peered":"2023-10-16T11:25:37.183586+0000","last_unstale":"2023-10-16T11:25:39.197203+0000","last_undegraded":"2023-10-16T11:25:39.197203+0000","last_fullsized":"2023-10-16T11:25:39.197203+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:43.813878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2c","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616722+0000","last_change":"2023-10-16T11:25:37.172984+0000","last_active":"2023-10-16T11:25:38.616722+0000","last_peered":"2023-10-16T11:25:38.616722+0000","last_clean":"2023-10-16T11:25:38.616722+0000","last_became_active":"2023-10-16T11:25:37.172403+0000","last_became_peered":"2023-10-16T11:25:37.172403+0000","last_unstale":"2023-10-16T11:25:38.616722+0000","last_undegraded":"2023-10-16T11:25:38.616722+0000","last_fullsized":"2023-10-16T11:25:38.616722+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:03.912995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2b","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.196844+0000","last_change":"2023-10-16T11:25:37.173026+0000","last_active":"2023-10-16T11:25:39.196844+0000","last_peered":"2023-10-16T11:25:39.196844+0000","last_clean":"2023-10-16T11:25:39.196844+0000","last_became_active":"2023-10-16T11:25:37.172680+0000","last_became_peered":"2023-10-16T11:25:37.172680+0000","last_unstale":"2023-10-16T11:25:39.196844+0000","last_undegraded":"2023-10-16T11:25:39.196844+0000","last_fullsized":"2023-10-16T11:25:39.196844+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:56.314696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2a","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.175816+0000","last_change":"2023-10-16T11:25:37.176870+0000","last_active":"2023-10-16T11:25:38.175816+0000","last_peered":"2023-10-16T11:25:38.175816+0000","last_clean":"2023-10-16T11:25:38.175816+0000","last_became_active":"2023-10-16T11:25:37.175230+0000","last_became_peered":"2023-10-16T11:25:37.175230+0000","last_unstale":"2023-10-16T11:25:38.175816+0000","last_undegraded":"2023-10-16T11:25:38.175816+0000","last_fullsized":"2023-10-16T11:25:38.175816+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.29","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.197804+0000","last_change":"2023-10-16T11:25:37.169637+0000","last_active":"2023-10-16T11:25:39.197804+0000","last_peered":"2023-10-16T11:25:39.197804+0000","last_clean":"2023-10-16T11:25:39.197804+0000","last_became_active":"2023-10-16T11:25:37.169375+0000","last_became_peered":"2023-10-16T11:25:37.169375+0000","last_unstale":"2023-10-16T11:25:39.197804+0000","last_undegraded":"2023-10-16T11:25:39.197804+0000","last_fullsized":"2023-10-16T11:25:39.197804+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:42:23.188876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.28","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193571+0000","last_change":"2023-10-16T11:25:37.187270+0000","last_active":"2023-10-16T11:25:39.193571+0000","last_peered":"2023-10-16T11:25:39.193571+0000","last_clean":"2023-10-16T11:25:39.193571+0000","last_became_active":"2023-10-16T11:25:37.186962+0000","last_became_peered":"2023-10-16T11:25:37.186962+0000","last_unstale":"2023-10-16T11:25:39.193571+0000","last_undegraded":"2023-10-16T11:25:39.193571+0000","last_fullsized":"2023-10-16T11:25:39.193571+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:56:45.915920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.27","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.172224+0000","last_change":"2023-10-16T11:25:37.177057+0000","last_active":"2023-10-16T11:25:38.172224+0000","last_peered":"2023-10-16T11:25:38.172224+0000","last_clean":"2023-10-16T11:25:38.172224+0000","last_became_active":"2023-10-16T11:25:37.171369+0000","last_became_peered":"2023-10-16T11:25:37.171369+0000","last_unstale":"2023-10-16T11:25:38.172224+0000","last_undegraded":"2023-10-16T11:25:38.172224+0000","last_fullsized":"2023-10-16T11:25:38.172224+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.26","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.176417+0000","last_change":"2023-10-16T11:25:37.182875+0000","last_active":"2023-10-16T11:25:38.176417+0000","last_peered":"2023-10-16T11:25:38.176417+0000","last_clean":"2023-10-16T11:25:38.176417+0000","last_became_active":"2023-10-16T11:25:37.182570+0000","last_became_peered":"2023-10-16T11:25:37.182570+0000","last_unstale":"2023-10-16T11:25:38.176417+0000","last_undegraded":"2023-10-16T11:25:38.176417+0000","last_fullsized":"2023-10-16T11:25:38.176417+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:14.504702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.25","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.183204+0000","last_change":"2023-10-16T11:25:37.178245+0000","last_active":"2023-10-16T11:25:38.183204+0000","last_peered":"2023-10-16T11:25:38.183204+0000","last_clean":"2023-10-16T11:25:38.183204+0000","last_became_active":"2023-10-16T11:25:37.177914+0000","last_became_peered":"2023-10-16T11:25:37.177914+0000","last_unstale":"2023-10-16T11:25:38.183204+0000","last_undegraded":"2023-10-16T11:25:38.183204+0000","last_fullsized":"2023-10-16T11:25:38.183204+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:12.014513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.24","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193750+0000","last_change":"2023-10-16T11:25:37.170623+0000","last_active":"2023-10-16T11:25:39.193750+0000","last_peered":"2023-10-16T11:25:39.193750+0000","last_clean":"2023-10-16T11:25:39.193750+0000","last_became_active":"2023-10-16T11:25:37.170365+0000","last_became_peered":"2023-10-16T11:25:37.170365+0000","last_unstale":"2023-10-16T11:25:39.193750+0000","last_undegraded":"2023-10-16T11:25:39.193750+0000","last_fullsized":"2023-10-16T11:25:39.193750+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:52.443160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.23","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182831+0000","last_change":"2023-10-16T11:25:37.178693+0000","last_active":"2023-10-16T11:25:38.182831+0000","last_peered":"2023-10-16T11:25:38.182831+0000","last_clean":"2023-10-16T11:25:38.182831+0000","last_became_active":"2023-10-16T11:25:37.175403+0000","last_became_peered":"2023-10-16T11:25:37.175403+0000","last_unstale":"2023-10-16T11:25:38.182831+0000","last_undegraded":"2023-10-16T11:25:38.182831+0000","last_fullsized":"2023-10-16T11:25:38.182831+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:15:31.572558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.22","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.196822+0000","last_change":"2023-10-16T11:25:37.181812+0000","last_active":"2023-10-16T11:25:39.196822+0000","last_peered":"2023-10-16T11:25:39.196822+0000","last_clean":"2023-10-16T11:25:39.196822+0000","last_became_active":"2023-10-16T11:25:37.181529+0000","last_became_peered":"2023-10-16T11:25:37.181529+0000","last_unstale":"2023-10-16T11:25:39.196822+0000","last_undegraded":"2023-10-16T11:25:39.196822+0000","last_fullsized":"2023-10-16T11:25:39.196822+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:35.912014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.21","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.171784+0000","last_change":"2023-10-16T11:25:37.166981+0000","last_active":"2023-10-16T11:25:38.171784+0000","last_peered":"2023-10-16T11:25:38.171784+0000","last_clean":"2023-10-16T11:25:38.171784+0000","last_became_active":"2023-10-16T11:25:37.164147+0000","last_became_peered":"2023-10-16T11:25:37.164147+0000","last_unstale":"2023-10-16T11:25:38.171784+0000","last_undegraded":"2023-10-16T11:25:38.171784+0000","last_fullsized":"2023-10-16T11:25:38.171784+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.20","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193639+0000","last_change":"2023-10-16T11:25:37.178631+0000","last_active":"2023-10-16T11:25:39.193639+0000","last_peered":"2023-10-16T11:25:39.193639+0000","last_clean":"2023-10-16T11:25:39.193639+0000","last_became_active":"2023-10-16T11:25:37.167402+0000","last_became_peered":"2023-10-16T11:25:37.167402+0000","last_unstale":"2023-10-16T11:25:39.193639+0000","last_undegraded":"2023-10-16T11:25:39.193639+0000","last_fullsized":"2023-10-16T11:25:39.193639+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:55.572264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616997+0000","last_change":"2023-10-16T11:25:37.177706+0000","last_active":"2023-10-16T11:25:38.616997+0000","last_peered":"2023-10-16T11:25:38.616997+0000","last_clean":"2023-10-16T11:25:38.616997+0000","last_became_active":"2023-10-16T11:25:37.176806+0000","last_became_peered":"2023-10-16T11:25:37.176806+0000","last_unstale":"2023-10-16T11:25:38.616997+0000","last_undegraded":"2023-10-16T11:25:38.616997+0000","last_fullsized":"2023-10-16T11:25:38.616997+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:42:32.585642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1e","version":"236'1","reported_seq":13,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.184494+0000","last_change":"2023-10-16T11:25:36.169327+0000","last_active":"2023-10-16T11:25:37.184494+0000","last_peered":"2023-10-16T11:25:37.184494+0000","last_clean":"2023-10-16T11:25:37.184494+0000","last_became_active":"2023-10-16T11:25:36.169046+0000","last_became_peered":"2023-10-16T11:25:36.169046+0000","last_unstale":"2023-10-16T11:25:37.184494+0000","last_undegraded":"2023-10-16T11:25:37.184494+0000","last_fullsized":"2023-10-16T11:25:37.184494+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:34.899695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182727+0000","last_change":"2023-10-16T11:25:37.173736+0000","last_active":"2023-10-16T11:25:38.182727+0000","last_peered":"2023-10-16T11:25:38.182727+0000","last_clean":"2023-10-16T11:25:38.182727+0000","last_became_active":"2023-10-16T11:25:37.173482+0000","last_became_peered":"2023-10-16T11:25:37.173482+0000","last_unstale":"2023-10-16T11:25:38.182727+0000","last_undegraded":"2023-10-16T11:25:38.182727+0000","last_fullsized":"2023-10-16T11:25:38.182727+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:42:23.188876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1d","version":"236'2","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.190697+0000","last_change":"2023-10-16T11:25:36.172861+0000","last_active":"2023-10-16T11:25:37.190697+0000","last_peered":"2023-10-16T11:25:37.190697+0000","last_clean":"2023-10-16T11:25:37.190697+0000","last_became_active":"2023-10-16T11:25:36.172574+0000","last_became_peered":"2023-10-16T11:25:36.172574+0000","last_unstale":"2023-10-16T11:25:37.190697+0000","last_undegraded":"2023-10-16T11:25:37.190697+0000","last_fullsized":"2023-10-16T11:25:37.190697+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:22:17.667795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.197306+0000","last_change":"2023-10-16T11:25:37.167451+0000","last_active":"2023-10-16T11:25:39.197306+0000","last_peered":"2023-10-16T11:25:39.197306+0000","last_clean":"2023-10-16T11:25:39.197306+0000","last_became_active":"2023-10-16T11:25:37.167157+0000","last_became_peered":"2023-10-16T11:25:37.167157+0000","last_unstale":"2023-10-16T11:25:39.197306+0000","last_undegraded":"2023-10-16T11:25:39.197306+0000","last_fullsized":"2023-10-16T11:25:39.197306+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:14.504702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1c","version":"236'2","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.179729+0000","last_change":"2023-10-16T11:25:36.153273+0000","last_active":"2023-10-16T11:25:37.179729+0000","last_peered":"2023-10-16T11:25:37.179729+0000","last_clean":"2023-10-16T11:25:37.179729+0000","last_became_active":"2023-10-16T11:25:36.152960+0000","last_became_peered":"2023-10-16T11:25:36.152960+0000","last_unstale":"2023-10-16T11:25:37.179729+0000","last_undegraded":"2023-10-16T11:25:37.179729+0000","last_fullsized":"2023-10-16T11:25:37.179729+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:08.218268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182469+0000","last_change":"2023-10-16T11:25:37.174071+0000","last_active":"2023-10-16T11:25:38.182469+0000","last_peered":"2023-10-16T11:25:38.182469+0000","last_clean":"2023-10-16T11:25:38.182469+0000","last_became_active":"2023-10-16T11:25:37.173436+0000","last_became_peered":"2023-10-16T11:25:37.173436+0000","last_unstale":"2023-10-16T11:25:38.182469+0000","last_undegraded":"2023-10-16T11:25:38.182469+0000","last_fullsized":"2023-10-16T11:25:38.182469+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1b","version":"236'1","reported_seq":13,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.178750+0000","last_change":"2023-10-16T11:25:36.156408+0000","last_active":"2023-10-16T11:25:37.178750+0000","last_peered":"2023-10-16T11:25:37.178750+0000","last_clean":"2023-10-16T11:25:37.178750+0000","last_became_active":"2023-10-16T11:25:36.155786+0000","last_became_peered":"2023-10-16T11:25:36.155786+0000","last_unstale":"2023-10-16T11:25:37.178750+0000","last_undegraded":"2023-10-16T11:25:37.178750+0000","last_fullsized":"2023-10-16T11:25:37.178750+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:01.925277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174081+0000","last_change":"2023-10-16T11:25:37.179656+0000","last_active":"2023-10-16T11:25:38.174081+0000","last_peered":"2023-10-16T11:25:38.174081+0000","last_clean":"2023-10-16T11:25:38.174081+0000","last_became_active":"2023-10-16T11:25:37.179361+0000","last_became_peered":"2023-10-16T11:25:37.179361+0000","last_unstale":"2023-10-16T11:25:38.174081+0000","last_undegraded":"2023-10-16T11:25:38.174081+0000","last_fullsized":"2023-10-16T11:25:38.174081+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:09.230587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"236'1","reported_seq":13,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.183638+0000","last_change":"2023-10-16T11:25:36.169348+0000","last_active":"2023-10-16T11:25:37.183638+0000","last_peered":"2023-10-16T11:25:37.183638+0000","last_clean":"2023-10-16T11:25:37.183638+0000","last_became_active":"2023-10-16T11:25:36.169053+0000","last_became_peered":"2023-10-16T11:25:36.169053+0000","last_unstale":"2023-10-16T11:25:37.183638+0000","last_undegraded":"2023-10-16T11:25:37.183638+0000","last_fullsized":"2023-10-16T11:25:37.183638+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:05.819502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.615841+0000","last_change":"2023-10-16T11:25:37.176966+0000","last_active":"2023-10-16T11:25:38.615841+0000","last_peered":"2023-10-16T11:25:38.615841+0000","last_clean":"2023-10-16T11:25:38.615841+0000","last_became_active":"2023-10-16T11:25:37.174621+0000","last_became_peered":"2023-10-16T11:25:37.174621+0000","last_unstale":"2023-10-16T11:25:38.615841+0000","last_undegraded":"2023-10-16T11:25:38.615841+0000","last_fullsized":"2023-10-16T11:25:38.615841+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:56:09.516122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.150607+0000","last_change":"2023-10-16T11:25:36.169635+0000","last_active":"2023-10-16T11:25:37.150607+0000","last_peered":"2023-10-16T11:25:37.150607+0000","last_clean":"2023-10-16T11:25:37.150607+0000","last_became_active":"2023-10-16T11:25:36.169016+0000","last_became_peered":"2023-10-16T11:25:36.169016+0000","last_unstale":"2023-10-16T11:25:37.150607+0000","last_undegraded":"2023-10-16T11:25:37.150607+0000","last_fullsized":"2023-10-16T11:25:37.150607+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:53:06.412234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193708+0000","last_change":"2023-10-16T11:25:37.184475+0000","last_active":"2023-10-16T11:25:39.193708+0000","last_peered":"2023-10-16T11:25:39.193708+0000","last_clean":"2023-10-16T11:25:39.193708+0000","last_became_active":"2023-10-16T11:25:37.179914+0000","last_became_peered":"2023-10-16T11:25:37.179914+0000","last_unstale":"2023-10-16T11:25:39.193708+0000","last_undegraded":"2023-10-16T11:25:39.193708+0000","last_fullsized":"2023-10-16T11:25:39.193708+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:53.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.164726+0000","last_change":"2023-10-16T11:25:36.155589+0000","last_active":"2023-10-16T11:25:37.164726+0000","last_peered":"2023-10-16T11:25:37.164726+0000","last_clean":"2023-10-16T11:25:37.164726+0000","last_became_active":"2023-10-16T11:25:36.155087+0000","last_became_peered":"2023-10-16T11:25:36.155087+0000","last_unstale":"2023-10-16T11:25:37.164726+0000","last_undegraded":"2023-10-16T11:25:37.164726+0000","last_fullsized":"2023-10-16T11:25:37.164726+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:52:38.937212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.194016+0000","last_change":"2023-10-16T11:25:37.175447+0000","last_active":"2023-10-16T11:25:39.194016+0000","last_peered":"2023-10-16T11:25:39.194016+0000","last_clean":"2023-10-16T11:25:39.194016+0000","last_became_active":"2023-10-16T11:25:37.173563+0000","last_became_peered":"2023-10-16T11:25:37.173563+0000","last_unstale":"2023-10-16T11:25:39.194016+0000","last_undegraded":"2023-10-16T11:25:39.194016+0000","last_fullsized":"2023-10-16T11:25:39.194016+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:54.544135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.149697+0000","last_change":"2023-10-16T11:25:36.169351+0000","last_active":"2023-10-16T11:25:37.149697+0000","last_peered":"2023-10-16T11:25:37.149697+0000","last_clean":"2023-10-16T11:25:37.149697+0000","last_became_active":"2023-10-16T11:25:36.168347+0000","last_became_peered":"2023-10-16T11:25:36.168347+0000","last_unstale":"2023-10-16T11:25:37.149697+0000","last_undegraded":"2023-10-16T11:25:37.149697+0000","last_fullsized":"2023-10-16T11:25:37.149697+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:19.077096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.183753+0000","last_change":"2023-10-16T11:25:37.179048+0000","last_active":"2023-10-16T11:25:38.183753+0000","last_peered":"2023-10-16T11:25:38.183753+0000","last_clean":"2023-10-16T11:25:38.183753+0000","last_became_active":"2023-10-16T11:25:37.178751+0000","last_became_peered":"2023-10-16T11:25:37.178751+0000","last_unstale":"2023-10-16T11:25:38.183753+0000","last_undegraded":"2023-10-16T11:25:38.183753+0000","last_fullsized":"2023-10-16T11:25:38.183753+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:48:10.906108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.165388+0000","last_change":"2023-10-16T11:25:36.154256+0000","last_active":"2023-10-16T11:25:37.165388+0000","last_peered":"2023-10-16T11:25:37.165388+0000","last_clean":"2023-10-16T11:25:37.165388+0000","last_became_active":"2023-10-16T11:25:36.153421+0000","last_became_peered":"2023-10-16T11:25:36.153421+0000","last_unstale":"2023-10-16T11:25:37.165388+0000","last_undegraded":"2023-10-16T11:25:37.165388+0000","last_fullsized":"2023-10-16T11:25:37.165388+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:09.436508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616767+0000","last_change":"2023-10-16T11:25:37.188270+0000","last_active":"2023-10-16T11:25:38.616767+0000","last_peered":"2023-10-16T11:25:38.616767+0000","last_clean":"2023-10-16T11:25:38.616767+0000","last_became_active":"2023-10-16T11:25:37.187995+0000","last_became_peered":"2023-10-16T11:25:37.187995+0000","last_unstale":"2023-10-16T11:25:38.616767+0000","last_undegraded":"2023-10-16T11:25:38.616767+0000","last_fullsized":"2023-10-16T11:25:38.616767+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:42:22.542072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.15","version":"236'3","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.191322+0000","last_change":"2023-10-16T11:25:36.171645+0000","last_active":"2023-10-16T11:25:37.191322+0000","last_peered":"2023-10-16T11:25:37.191322+0000","last_clean":"2023-10-16T11:25:37.191322+0000","last_became_active":"2023-10-16T11:25:36.170729+0000","last_became_peered":"2023-10-16T11:25:36.170729+0000","last_unstale":"2023-10-16T11:25:37.191322+0000","last_undegraded":"2023-10-16T11:25:37.191322+0000","last_fullsized":"2023-10-16T11:25:37.191322+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:34.362958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.177036+0000","last_change":"2023-10-16T11:25:37.163991+0000","last_active":"2023-10-16T11:25:38.177036+0000","last_peered":"2023-10-16T11:25:38.177036+0000","last_clean":"2023-10-16T11:25:38.177036+0000","last_became_active":"2023-10-16T11:25:37.163080+0000","last_became_peered":"2023-10-16T11:25:37.163080+0000","last_unstale":"2023-10-16T11:25:38.177036+0000","last_undegraded":"2023-10-16T11:25:38.177036+0000","last_fullsized":"2023-10-16T11:25:38.177036+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.14","version":"236'6","reported_seq":18,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.191864+0000","last_change":"2023-10-16T11:25:36.173259+0000","last_active":"2023-10-16T11:25:37.191864+0000","last_peered":"2023-10-16T11:25:37.191864+0000","last_clean":"2023-10-16T11:25:37.191864+0000","last_became_active":"2023-10-16T11:25:36.172980+0000","last_became_peered":"2023-10-16T11:25:36.172980+0000","last_unstale":"2023-10-16T11:25:37.191864+0000","last_undegraded":"2023-10-16T11:25:37.191864+0000","last_fullsized":"2023-10-16T11:25:37.191864+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:33.782641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.176159+0000","last_change":"2023-10-16T11:25:37.166081+0000","last_active":"2023-10-16T11:25:38.176159+0000","last_peered":"2023-10-16T11:25:38.176159+0000","last_clean":"2023-10-16T11:25:38.176159+0000","last_became_active":"2023-10-16T11:25:37.165150+0000","last_became_peered":"2023-10-16T11:25:37.165150+0000","last_unstale":"2023-10-16T11:25:38.176159+0000","last_undegraded":"2023-10-16T11:25:38.176159+0000","last_fullsized":"2023-10-16T11:25:38.176159+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:56.314696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"236'2","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.190914+0000","last_change":"2023-10-16T11:25:36.171468+0000","last_active":"2023-10-16T11:25:37.190914+0000","last_peered":"2023-10-16T11:25:37.190914+0000","last_clean":"2023-10-16T11:25:37.190914+0000","last_became_active":"2023-10-16T11:25:36.171236+0000","last_became_peered":"2023-10-16T11:25:36.171236+0000","last_unstale":"2023-10-16T11:25:37.190914+0000","last_undegraded":"2023-10-16T11:25:37.190914+0000","last_fullsized":"2023-10-16T11:25:37.190914+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:17.018903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174436+0000","last_change":"2023-10-16T11:25:37.178517+0000","last_active":"2023-10-16T11:25:38.174436+0000","last_peered":"2023-10-16T11:25:38.174436+0000","last_clean":"2023-10-16T11:25:38.174436+0000","last_became_active":"2023-10-16T11:25:37.177794+0000","last_became_peered":"2023-10-16T11:25:37.177794+0000","last_unstale":"2023-10-16T11:25:38.174436+0000","last_undegraded":"2023-10-16T11:25:38.174436+0000","last_fullsized":"2023-10-16T11:25:38.174436+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:52:39.949531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.164090+0000","last_change":"2023-10-16T11:25:36.162595+0000","last_active":"2023-10-16T11:25:37.164090+0000","last_peered":"2023-10-16T11:25:37.164090+0000","last_clean":"2023-10-16T11:25:37.164090+0000","last_became_active":"2023-10-16T11:25:36.162249+0000","last_became_peered":"2023-10-16T11:25:36.162249+0000","last_unstale":"2023-10-16T11:25:37.164090+0000","last_undegraded":"2023-10-16T11:25:37.164090+0000","last_fullsized":"2023-10-16T11:25:37.164090+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:49.547165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.171375+0000","last_change":"2023-10-16T11:25:37.164604+0000","last_active":"2023-10-16T11:25:38.171375+0000","last_peered":"2023-10-16T11:25:38.171375+0000","last_clean":"2023-10-16T11:25:38.171375+0000","last_became_active":"2023-10-16T11:25:37.162365+0000","last_became_peered":"2023-10-16T11:25:37.162365+0000","last_unstale":"2023-10-16T11:25:38.171375+0000","last_undegraded":"2023-10-16T11:25:38.171375+0000","last_fullsized":"2023-10-16T11:25:38.171375+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:35.912014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.150894+0000","last_change":"2023-10-16T11:25:36.170014+0000","last_active":"2023-10-16T11:25:37.150894+0000","last_peered":"2023-10-16T11:25:37.150894+0000","last_clean":"2023-10-16T11:25:37.150894+0000","last_became_active":"2023-10-16T11:25:36.169755+0000","last_became_peered":"2023-10-16T11:25:36.169755+0000","last_unstale":"2023-10-16T11:25:37.150894+0000","last_undegraded":"2023-10-16T11:25:37.150894+0000","last_fullsized":"2023-10-16T11:25:37.150894+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:02.119904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.173748+0000","last_change":"2023-10-16T11:25:37.178789+0000","last_active":"2023-10-16T11:25:38.173748+0000","last_peered":"2023-10-16T11:25:38.173748+0000","last_clean":"2023-10-16T11:25:38.173748+0000","last_became_active":"2023-10-16T11:25:37.178510+0000","last_became_peered":"2023-10-16T11:25:37.178510+0000","last_unstale":"2023-10-16T11:25:38.173748+0000","last_undegraded":"2023-10-16T11:25:38.173748+0000","last_fullsized":"2023-10-16T11:25:38.173748+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.10","version":"236'8","reported_seq":20,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.191147+0000","last_change":"2023-10-16T11:25:36.168889+0000","last_active":"2023-10-16T11:25:37.191147+0000","last_peered":"2023-10-16T11:25:37.191147+0000","last_clean":"2023-10-16T11:25:37.191147+0000","last_became_active":"2023-10-16T11:25:36.168590+0000","last_became_peered":"2023-10-16T11:25:36.168590+0000","last_unstale":"2023-10-16T11:25:37.191147+0000","last_undegraded":"2023-10-16T11:25:37.191147+0000","last_fullsized":"2023-10-16T11:25:37.191147+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:42.801560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.172236+0000","last_change":"2023-10-16T11:25:37.175992+0000","last_active":"2023-10-16T11:25:38.172236+0000","last_peered":"2023-10-16T11:25:38.172236+0000","last_clean":"2023-10-16T11:25:38.172236+0000","last_became_active":"2023-10-16T11:25:37.173732+0000","last_became_peered":"2023-10-16T11:25:37.173732+0000","last_unstale":"2023-10-16T11:25:38.172236+0000","last_undegraded":"2023-10-16T11:25:38.172236+0000","last_fullsized":"2023-10-16T11:25:38.172236+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.f","version":"236'1","reported_seq":13,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.182939+0000","last_change":"2023-10-16T11:25:36.173935+0000","last_active":"2023-10-16T11:25:37.182939+0000","last_peered":"2023-10-16T11:25:37.182939+0000","last_clean":"2023-10-16T11:25:37.182939+0000","last_became_active":"2023-10-16T11:25:36.173510+0000","last_became_peered":"2023-10-16T11:25:36.173510+0000","last_unstale":"2023-10-16T11:25:37.182939+0000","last_undegraded":"2023-10-16T11:25:37.182939+0000","last_fullsized":"2023-10-16T11:25:37.182939+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:19.077096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616968+0000","last_change":"2023-10-16T11:25:37.177543+0000","last_active":"2023-10-16T11:25:38.616968+0000","last_peered":"2023-10-16T11:25:38.616968+0000","last_clean":"2023-10-16T11:25:38.616968+0000","last_became_active":"2023-10-16T11:25:37.176420+0000","last_became_peered":"2023-10-16T11:25:37.176420+0000","last_unstale":"2023-10-16T11:25:38.616968+0000","last_undegraded":"2023-10-16T11:25:38.616968+0000","last_fullsized":"2023-10-16T11:25:38.616968+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:24.104981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.e","version":"236'2","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.189031+0000","last_change":"2023-10-16T11:25:36.170878+0000","last_active":"2023-10-16T11:25:37.189031+0000","last_peered":"2023-10-16T11:25:37.189031+0000","last_clean":"2023-10-16T11:25:37.189031+0000","last_became_active":"2023-10-16T11:25:36.170548+0000","last_became_peered":"2023-10-16T11:25:36.170548+0000","last_unstale":"2023-10-16T11:25:37.189031+0000","last_undegraded":"2023-10-16T11:25:37.189031+0000","last_fullsized":"2023-10-16T11:25:37.189031+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:34:14.932409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174918+0000","last_change":"2023-10-16T11:25:37.176420+0000","last_active":"2023-10-16T11:25:38.174918+0000","last_peered":"2023-10-16T11:25:38.174918+0000","last_clean":"2023-10-16T11:25:38.174918+0000","last_became_active":"2023-10-16T11:25:37.176101+0000","last_became_peered":"2023-10-16T11:25:37.176101+0000","last_unstale":"2023-10-16T11:25:38.174918+0000","last_undegraded":"2023-10-16T11:25:38.174918+0000","last_fullsized":"2023-10-16T11:25:38.174918+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:10.245184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"236'1","reported_seq":13,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.182034+0000","last_change":"2023-10-16T11:25:36.171476+0000","last_active":"2023-10-16T11:25:37.182034+0000","last_peered":"2023-10-16T11:25:37.182034+0000","last_clean":"2023-10-16T11:25:37.182034+0000","last_became_active":"2023-10-16T11:25:36.171201+0000","last_became_peered":"2023-10-16T11:25:36.171201+0000","last_unstale":"2023-10-16T11:25:37.182034+0000","last_undegraded":"2023-10-16T11:25:37.182034+0000","last_fullsized":"2023-10-16T11:25:37.182034+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:30.034841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.197561+0000","last_change":"2023-10-16T11:25:37.162366+0000","last_active":"2023-10-16T11:25:39.197561+0000","last_peered":"2023-10-16T11:25:39.197561+0000","last_clean":"2023-10-16T11:25:39.197561+0000","last_became_active":"2023-10-16T11:25:37.162032+0000","last_became_peered":"2023-10-16T11:25:37.162032+0000","last_unstale":"2023-10-16T11:25:39.197561+0000","last_undegraded":"2023-10-16T11:25:39.197561+0000","last_fullsized":"2023-10-16T11:25:39.197561+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.169247+0000","last_change":"2023-10-16T11:25:36.174246+0000","last_active":"2023-10-16T11:25:37.169247+0000","last_peered":"2023-10-16T11:25:37.169247+0000","last_clean":"2023-10-16T11:25:37.169247+0000","last_became_active":"2023-10-16T11:25:36.173975+0000","last_became_peered":"2023-10-16T11:25:36.173975+0000","last_unstale":"2023-10-16T11:25:37.169247+0000","last_undegraded":"2023-10-16T11:25:37.169247+0000","last_fullsized":"2023-10-16T11:25:37.169247+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:32.130559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.196788+0000","last_change":"2023-10-16T11:25:37.168715+0000","last_active":"2023-10-16T11:25:39.196788+0000","last_peered":"2023-10-16T11:25:39.196788+0000","last_clean":"2023-10-16T11:25:39.196788+0000","last_became_active":"2023-10-16T11:25:37.168387+0000","last_became_peered":"2023-10-16T11:25:37.168387+0000","last_unstale":"2023-10-16T11:25:39.196788+0000","last_undegraded":"2023-10-16T11:25:39.196788+0000","last_fullsized":"2023-10-16T11:25:39.196788+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.b","version":"236'2","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.183693+0000","last_change":"2023-10-16T11:25:36.174153+0000","last_active":"2023-10-16T11:25:37.183693+0000","last_peered":"2023-10-16T11:25:37.183693+0000","last_clean":"2023-10-16T11:25:37.183693+0000","last_became_active":"2023-10-16T11:25:36.173812+0000","last_became_peered":"2023-10-16T11:25:36.173812+0000","last_unstale":"2023-10-16T11:25:37.183693+0000","last_undegraded":"2023-10-16T11:25:37.183693+0000","last_fullsized":"2023-10-16T11:25:37.183693+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:07:05.931833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193818+0000","last_change":"2023-10-16T11:25:37.175767+0000","last_active":"2023-10-16T11:25:39.193818+0000","last_peered":"2023-10-16T11:25:39.193818+0000","last_clean":"2023-10-16T11:25:39.193818+0000","last_became_active":"2023-10-16T11:25:37.167389+0000","last_became_peered":"2023-10-16T11:25:37.167389+0000","last_unstale":"2023-10-16T11:25:39.193818+0000","last_undegraded":"2023-10-16T11:25:39.193818+0000","last_fullsized":"2023-10-16T11:25:39.193818+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:18.601118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.169039+0000","last_change":"2023-10-16T11:25:36.175478+0000","last_active":"2023-10-16T11:25:37.169039+0000","last_peered":"2023-10-16T11:25:37.169039+0000","last_clean":"2023-10-16T11:25:37.169039+0000","last_became_active":"2023-10-16T11:25:36.174935+0000","last_became_peered":"2023-10-16T11:25:36.174935+0000","last_unstale":"2023-10-16T11:25:37.169039+0000","last_undegraded":"2023-10-16T11:25:37.169039+0000","last_fullsized":"2023-10-16T11:25:37.169039+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:14.189489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182532+0000","last_change":"2023-10-16T11:25:37.173034+0000","last_active":"2023-10-16T11:25:38.182532+0000","last_peered":"2023-10-16T11:25:38.182532+0000","last_clean":"2023-10-16T11:25:38.182532+0000","last_became_active":"2023-10-16T11:25:37.172644+0000","last_became_peered":"2023-10-16T11:25:37.172644+0000","last_unstale":"2023-10-16T11:25:38.182532+0000","last_undegraded":"2023-10-16T11:25:38.182532+0000","last_fullsized":"2023-10-16T11:25:38.182532+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.162208+0000","last_change":"2023-10-16T11:25:36.163926+0000","last_active":"2023-10-16T11:25:37.162208+0000","last_peered":"2023-10-16T11:25:37.162208+0000","last_clean":"2023-10-16T11:25:37.162208+0000","last_became_active":"2023-10-16T11:25:36.162991+0000","last_became_peered":"2023-10-16T11:25:36.162991+0000","last_unstale":"2023-10-16T11:25:37.162208+0000","last_undegraded":"2023-10-16T11:25:37.162208+0000","last_fullsized":"2023-10-16T11:25:37.162208+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:49.547165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.196630+0000","last_change":"2023-10-16T11:25:37.158678+0000","last_active":"2023-10-16T11:25:39.196630+0000","last_peered":"2023-10-16T11:25:39.196630+0000","last_clean":"2023-10-16T11:25:39.196630+0000","last_became_active":"2023-10-16T11:25:37.157389+0000","last_became_peered":"2023-10-16T11:25:37.157389+0000","last_unstale":"2023-10-16T11:25:39.196630+0000","last_undegraded":"2023-10-16T11:25:39.196630+0000","last_fullsized":"2023-10-16T11:25:39.196630+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.153051+0000","last_change":"2023-10-16T11:25:36.171876+0000","last_active":"2023-10-16T11:25:37.153051+0000","last_peered":"2023-10-16T11:25:37.153051+0000","last_clean":"2023-10-16T11:25:37.153051+0000","last_became_active":"2023-10-16T11:25:36.169128+0000","last_became_peered":"2023-10-16T11:25:36.169128+0000","last_unstale":"2023-10-16T11:25:37.153051+0000","last_undegraded":"2023-10-16T11:25:37.153051+0000","last_fullsized":"2023-10-16T11:25:37.153051+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:16:18.153133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.176783+0000","last_change":"2023-10-16T11:25:37.179119+0000","last_active":"2023-10-16T11:25:38.176783+0000","last_peered":"2023-10-16T11:25:38.176783+0000","last_clean":"2023-10-16T11:25:38.176783+0000","last_became_active":"2023-10-16T11:25:37.175799+0000","last_became_peered":"2023-10-16T11:25:37.175799+0000","last_unstale":"2023-10-16T11:25:38.176783+0000","last_undegraded":"2023-10-16T11:25:38.176783+0000","last_fullsized":"2023-10-16T11:25:38.176783+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"236'2","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.185451+0000","last_change":"2023-10-16T11:25:36.162608+0000","last_active":"2023-10-16T11:25:37.185451+0000","last_peered":"2023-10-16T11:25:37.185451+0000","last_clean":"2023-10-16T11:25:37.185451+0000","last_became_active":"2023-10-16T11:25:36.162336+0000","last_became_peered":"2023-10-16T11:25:36.162336+0000","last_unstale":"2023-10-16T11:25:37.185451+0000","last_undegraded":"2023-10-16T11:25:37.185451+0000","last_fullsized":"2023-10-16T11:25:37.185451+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:19.377028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.194112+0000","last_change":"2023-10-16T11:25:37.177560+0000","last_active":"2023-10-16T11:25:39.194112+0000","last_peered":"2023-10-16T11:25:39.194112+0000","last_clean":"2023-10-16T11:25:39.194112+0000","last_became_active":"2023-10-16T11:25:37.176982+0000","last_became_peered":"2023-10-16T11:25:37.176982+0000","last_unstale":"2023-10-16T11:25:39.194112+0000","last_undegraded":"2023-10-16T11:25:39.194112+0000","last_fullsized":"2023-10-16T11:25:39.194112+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:54.497943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.167365+0000","last_change":"2023-10-16T11:25:36.174419+0000","last_active":"2023-10-16T11:25:37.167365+0000","last_peered":"2023-10-16T11:25:37.167365+0000","last_clean":"2023-10-16T11:25:37.167365+0000","last_became_active":"2023-10-16T11:25:36.174148+0000","last_became_peered":"2023-10-16T11:25:36.174148+0000","last_unstale":"2023-10-16T11:25:37.167365+0000","last_undegraded":"2023-10-16T11:25:37.167365+0000","last_fullsized":"2023-10-16T11:25:37.167365+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:38.739323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182976+0000","last_change":"2023-10-16T11:25:37.180683+0000","last_active":"2023-10-16T11:25:38.182976+0000","last_peered":"2023-10-16T11:25:38.182976+0000","last_clean":"2023-10-16T11:25:38.182976+0000","last_became_active":"2023-10-16T11:25:37.180393+0000","last_became_peered":"2023-10-16T11:25:37.180393+0000","last_unstale":"2023-10-16T11:25:38.182976+0000","last_undegraded":"2023-10-16T11:25:38.182976+0000","last_fullsized":"2023-10-16T11:25:38.182976+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:38.218704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.158199+0000","last_change":"2023-10-16T11:25:36.164519+0000","last_active":"2023-10-16T11:25:37.158199+0000","last_peered":"2023-10-16T11:25:37.158199+0000","last_clean":"2023-10-16T11:25:37.158199+0000","last_became_active":"2023-10-16T11:25:36.164214+0000","last_became_peered":"2023-10-16T11:25:36.164214+0000","last_unstale":"2023-10-16T11:25:37.158199+0000","last_undegraded":"2023-10-16T11:25:37.158199+0000","last_fullsized":"2023-10-16T11:25:37.158199+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:03.074535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.183331+0000","last_change":"2023-10-16T11:25:37.181808+0000","last_active":"2023-10-16T11:25:38.183331+0000","last_peered":"2023-10-16T11:25:38.183331+0000","last_clean":"2023-10-16T11:25:38.183331+0000","last_became_active":"2023-10-16T11:25:37.181575+0000","last_became_peered":"2023-10-16T11:25:37.181575+0000","last_unstale":"2023-10-16T11:25:38.183331+0000","last_undegraded":"2023-10-16T11:25:38.183331+0000","last_fullsized":"2023-10-16T11:25:38.183331+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:17:01.131611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.169371+0000","last_change":"2023-10-16T11:25:36.162153+0000","last_active":"2023-10-16T11:25:37.169371+0000","last_peered":"2023-10-16T11:25:37.169371+0000","last_clean":"2023-10-16T11:25:37.169371+0000","last_became_active":"2023-10-16T11:25:36.161842+0000","last_became_peered":"2023-10-16T11:25:36.161842+0000","last_unstale":"2023-10-16T11:25:37.169371+0000","last_undegraded":"2023-10-16T11:25:37.169371+0000","last_fullsized":"2023-10-16T11:25:37.169371+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:55.302377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.176225+0000","last_change":"2023-10-16T11:25:37.165420+0000","last_active":"2023-10-16T11:25:38.176225+0000","last_peered":"2023-10-16T11:25:38.176225+0000","last_clean":"2023-10-16T11:25:38.176225+0000","last_became_active":"2023-10-16T11:25:37.164820+0000","last_became_peered":"2023-10-16T11:25:37.164820+0000","last_unstale":"2023-10-16T11:25:38.176225+0000","last_undegraded":"2023-10-16T11:25:38.176225+0000","last_fullsized":"2023-10-16T11:25:38.176225+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:43.813878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.154862+0000","last_change":"2023-10-16T11:25:36.173209+0000","last_active":"2023-10-16T11:25:37.154862+0000","last_peered":"2023-10-16T11:25:37.154862+0000","last_clean":"2023-10-16T11:25:37.154862+0000","last_became_active":"2023-10-16T11:25:36.171799+0000","last_became_peered":"2023-10-16T11:25:36.171799+0000","last_unstale":"2023-10-16T11:25:37.154862+0000","last_undegraded":"2023-10-16T11:25:37.154862+0000","last_fullsized":"2023-10-16T11:25:37.154862+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:19.377028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.198049+0000","last_change":"2023-10-16T11:25:37.180243+0000","last_active":"2023-10-16T11:25:39.198049+0000","last_peered":"2023-10-16T11:25:39.198049+0000","last_clean":"2023-10-16T11:25:39.198049+0000","last_became_active":"2023-10-16T11:25:37.179869+0000","last_became_peered":"2023-10-16T11:25:37.179869+0000","last_unstale":"2023-10-16T11:25:39.198049+0000","last_undegraded":"2023-10-16T11:25:39.198049+0000","last_fullsized":"2023-10-16T11:25:39.198049+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:15:31.572558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.146030+0000","last_change":"2023-10-16T11:25:36.169146+0000","last_active":"2023-10-16T11:25:37.146030+0000","last_peered":"2023-10-16T11:25:37.146030+0000","last_clean":"2023-10-16T11:25:37.146030+0000","last_became_active":"2023-10-16T11:25:36.168877+0000","last_became_peered":"2023-10-16T11:25:36.168877+0000","last_unstale":"2023-10-16T11:25:37.146030+0000","last_undegraded":"2023-10-16T11:25:37.146030+0000","last_fullsized":"2023-10-16T11:25:37.146030+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:35.988148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.196353+0000","last_change":"2023-10-16T11:25:37.173516+0000","last_active":"2023-10-16T11:25:39.196353+0000","last_peered":"2023-10-16T11:25:39.196353+0000","last_clean":"2023-10-16T11:25:39.196353+0000","last_became_active":"2023-10-16T11:25:37.172964+0000","last_became_peered":"2023-10-16T11:25:37.172964+0000","last_unstale":"2023-10-16T11:25:39.196353+0000","last_undegraded":"2023-10-16T11:25:39.196353+0000","last_fullsized":"2023-10-16T11:25:39.196353+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.164633+0000","last_change":"2023-10-16T11:25:36.166442+0000","last_active":"2023-10-16T11:25:37.164633+0000","last_peered":"2023-10-16T11:25:37.164633+0000","last_clean":"2023-10-16T11:25:37.164633+0000","last_became_active":"2023-10-16T11:25:36.166131+0000","last_became_peered":"2023-10-16T11:25:36.166131+0000","last_unstale":"2023-10-16T11:25:37.164633+0000","last_undegraded":"2023-10-16T11:25:37.164633+0000","last_fullsized":"2023-10-16T11:25:37.164633+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:43.814368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193591+0000","last_change":"2023-10-16T11:25:37.183491+0000","last_active":"2023-10-16T11:25:39.193591+0000","last_peered":"2023-10-16T11:25:39.193591+0000","last_clean":"2023-10-16T11:25:39.193591+0000","last_became_active":"2023-10-16T11:25:37.179560+0000","last_became_peered":"2023-10-16T11:25:37.179560+0000","last_unstale":"2023-10-16T11:25:39.193591+0000","last_undegraded":"2023-10-16T11:25:39.193591+0000","last_fullsized":"2023-10-16T11:25:39.193591+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:31.820109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.164860+0000","last_change":"2023-10-16T11:25:36.175845+0000","last_active":"2023-10-16T11:25:37.164860+0000","last_peered":"2023-10-16T11:25:37.164860+0000","last_clean":"2023-10-16T11:25:37.164860+0000","last_became_active":"2023-10-16T11:25:36.175390+0000","last_became_peered":"2023-10-16T11:25:36.175390+0000","last_unstale":"2023-10-16T11:25:37.164860+0000","last_undegraded":"2023-10-16T11:25:37.164860+0000","last_fullsized":"2023-10-16T11:25:37.164860+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:09.232865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.198662+0000","last_change":"2023-10-16T11:25:37.178486+0000","last_active":"2023-10-16T11:25:39.198662+0000","last_peered":"2023-10-16T11:25:39.198662+0000","last_clean":"2023-10-16T11:25:39.198662+0000","last_became_active":"2023-10-16T11:25:37.177077+0000","last_became_peered":"2023-10-16T11:25:37.177077+0000","last_unstale":"2023-10-16T11:25:39.198662+0000","last_undegraded":"2023-10-16T11:25:39.198662+0000","last_fullsized":"2023-10-16T11:25:39.198662+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"236'4","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.182413+0000","last_change":"2023-10-16T11:25:36.174205+0000","last_active":"2023-10-16T11:25:37.182413+0000","last_peered":"2023-10-16T11:25:37.182413+0000","last_clean":"2023-10-16T11:25:37.182413+0000","last_became_active":"2023-10-16T11:25:36.173850+0000","last_became_peered":"2023-10-16T11:25:36.173850+0000","last_unstale":"2023-10-16T11:25:37.182413+0000","last_undegraded":"2023-10-16T11:25:37.182413+0000","last_fullsized":"2023-10-16T11:25:37.182413+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:20.618602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"223'2789","reported_seq":3745,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.158495+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:25:37.158495+0000","last_peered":"2023-10-16T11:25:37.158495+0000","last_clean":"2023-10-16T11:25:37.158495+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:25:37.158495+0000","last_undegraded":"2023-10-16T11:25:37.158495+0000","last_fullsized":"2023-10-16T11:25:37.158495+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2789,"log_dups_size":0,"ondisk_log_size":2789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":31,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":30,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2789,"ondisk_log_size":2789,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575999,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52732,"kb_used_data":3500,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319108,"statfs":{"total":96636764160,"available":96582766592,"internally_reserved":0,"allocated":3584000,"data_stored":2045861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.153,"15min":1.2090000000000001},"min":{"1min":0.71499999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4950000000000001,"5min":1.7549999999999999,"15min":2.3399999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.161,"15min":1.2150000000000001},"min":{"1min":0.623,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.448,"5min":1.6699999999999999,"15min":1.8320000000000001},"last":0.67700000000000005}]},{"osd":1,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.256,"15min":1.2430000000000001},"min":{"1min":0.84599999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.5029999999999999,"5min":1.913,"15min":1.913},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.2210000000000001,"15min":1.264},"min":{"1min":0.67100000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.522,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.90300000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.3,"15min":1.296},"min":{"1min":0.94199999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6879999999999999,"5min":1.8680000000000001,"15min":2.4460000000000002},"last":1.234},{"interface":"front","average":{"1min":1.21,"5min":1.3129999999999999,"15min":1.3500000000000001},"min":{"1min":0.94499999999999995,"5min":0.88,"15min":0.88},"max":{"1min":1.5589999999999999,"5min":1.859,"15min":1.974},"last":1.165}]},{"osd":3,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3520000000000001,"15min":1.3779999999999999},"min":{"1min":0.93700000000000006,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.653,"5min":1.8080000000000001,"15min":1.9099999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.4370000000000001,"15min":1.454},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.728,"5min":2.0070000000000001,"15min":2.577},"last":1.0860000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4319999999999999,"15min":1.413},"min":{"1min":0.89800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.865,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":1.149,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8660000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.488}]},{"osd":5,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.587,"15min":1.591},"min":{"1min":1.1799999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.95,"5min":2.101,"15min":3.3519999999999999},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.53,"15min":1.508},"min":{"1min":1.2330000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.9870000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.6279999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.492,"15min":1.5089999999999999},"min":{"1min":1.1459999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.8140000000000001,"5min":2.101,"15min":2.7250000000000001},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.6040000000000001,"5min":1.6819999999999999,"15min":1.698},"min":{"1min":1.3080000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.0129999999999999,"5min":2.2109999999999999,"15min":3.1440000000000001},"last":1.6890000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575999,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52152,"kb_used_data":2924,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319688,"statfs":{"total":96636764160,"available":96583360512,"internally_reserved":0,"allocated":2994176,"data_stored":1456027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.2130000000000001,"15min":1.21},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.343,"5min":6.298,"15min":6.298},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.331,"15min":1.292},"min":{"1min":0.72199999999999998,"5min":0.68200000000000005,"15min":0.66400000000000003},"max":{"1min":1.607,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":1.105}]},{"osd":1,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.224,"15min":1.212},"min":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.5660000000000001,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":1.236},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.421,"15min":1.3779999999999999},"min":{"1min":0.76200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.7310000000000001,"5min":12.468,"15min":12.468},"last":1.323}]},{"osd":2,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.3140000000000001,"15min":1.2769999999999999},"min":{"1min":0.77800000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.373,"5min":11.586,"15min":11.586},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.3740000000000001,"15min":1.3680000000000001},"min":{"1min":0.84399999999999997,"5min":0.82599999999999996,"15min":0.77500000000000002},"max":{"1min":1.7629999999999999,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.7629999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.425,"15min":1.3460000000000001},"min":{"1min":0.84799999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.859,"5min":12.801,"15min":12.801},"last":1.54},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2929999999999999,"15min":1.2769999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.657,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":1.371}]},{"osd":4,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3899999999999999,"15min":1.3280000000000001},"min":{"1min":0.68700000000000006,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.9510000000000001,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.9510000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.6220000000000001,"15min":1.5509999999999999},"min":{"1min":1.1180000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.069,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.831}]},{"osd":5,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.5189999999999999,"15min":1.397},"min":{"1min":0.92500000000000004,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.978,"5min":18.247,"15min":18.247},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.595,"15min":1.5620000000000001},"min":{"1min":0.93799999999999994,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.1240000000000001,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":2.1240000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.627,"15min":1.5469999999999999},"min":{"1min":1.2070000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.383,"5min":12.589,"15min":12.589},"last":2.2410000000000001},{"interface":"front","average":{"1min":1.661,"5min":1.702,"15min":1.603},"min":{"1min":1.0049999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.2970000000000002,"5min":12.725,"15min":12.725},"last":2.2970000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575999,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29936,"kb_used_data":2916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341904,"statfs":{"total":96636764160,"available":96606109696,"internally_reserved":0,"allocated":2985984,"data_stored":1455423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2150000000000001,"15min":1.2629999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.51,"5min":2.661,"15min":2.661},"last":1.296},{"interface":"front","average":{"1min":1.204,"5min":1.2769999999999999,"15min":1.339},"min":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7210000000000001,"5min":1.7949999999999999,"15min":1.865},"last":1.335}]},{"osd":1,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.145,"15min":1.1890000000000001},"min":{"1min":0.872,"5min":0.81299999999999994,"15min":0.79100000000000004},"max":{"1min":1.583,"5min":1.704,"15min":1.992},"last":1.649},{"interface":"front","average":{"1min":1.079,"5min":1.22,"15min":1.222},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6739999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.367}]},{"osd":2,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.26,"15min":1.236},"min":{"1min":0.73499999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.7450000000000001,"5min":1.952,"15min":1.952},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.284,"15min":1.3129999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5840000000000001,"5min":1.8109999999999999,"15min":2.1640000000000001},"last":1.052}]},{"osd":3,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.319,"15min":1.329},"min":{"1min":0.872,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5920000000000001,"5min":3.012,"15min":3.012},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.401,"15min":1.4139999999999999},"min":{"1min":0.95399999999999996,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.6639999999999999,"5min":1.9119999999999999,"15min":2.2250000000000001},"last":1.232}]},{"osd":4,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3600000000000001,"15min":1.341},"min":{"1min":0.91300000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.887,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.413,"5min":1.5529999999999999,"15min":1.5660000000000001},"min":{"1min":0.996,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.6499999999999999,"5min":2.903,"15min":2.903},"last":1.6910000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.4119999999999999,"15min":1.411},"min":{"1min":0.93799999999999994,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.837,"5min":1.8660000000000001,"15min":2.0979999999999999},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.486,"15min":1.4810000000000001},"min":{"1min":0.93300000000000005,"5min":0.90700000000000003,"15min":0.89200000000000002},"max":{"1min":2.0329999999999999,"5min":3.073,"15min":3.073},"last":1.125}]},{"osd":7,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.5649999999999999,"15min":1.5429999999999999},"min":{"1min":1.159,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.764,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.625,"15min":1.6399999999999999},"min":{"1min":1.349,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.9970000000000001,"5min":2.1099999999999999,"15min":2.2930000000000001},"last":1.54}]}]},{"osd":4,"up_from":13,"seq":55834576000,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52164,"kb_used_data":2936,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319676,"statfs":{"total":96636764160,"available":96583348224,"internally_reserved":0,"allocated":3006464,"data_stored":1457127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.167,"15min":1.1799999999999999},"min":{"1min":0.69099999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":1.8049999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.2370000000000001,"15min":1.2430000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":1.0760000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3069999999999999,"15min":1.3009999999999999},"min":{"1min":0.88800000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.24},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.252,"15min":1.252},"min":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":3.0739999999999998,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.593}]},{"osd":2,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3,"15min":1.3109999999999999},"min":{"1min":0.80500000000000005,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":3.2170000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.4359999999999999,"15min":1.4119999999999999},"min":{"1min":0.90300000000000002,"5min":0.879,"15min":0.872},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.6819999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.339,"15min":1.341},"min":{"1min":0.84599999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.431},{"interface":"front","average":{"1min":1.496,"5min":1.4550000000000001,"15min":1.4430000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":3.4220000000000002,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.6359999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.5249999999999999,"15min":1.4570000000000001},"min":{"1min":1.1339999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.8460000000000001},{"interface":"front","average":{"1min":1.613,"5min":1.46,"15min":1.4319999999999999},"min":{"1min":1.1519999999999999,"5min":0.85199999999999998,"15min":0.78600000000000003},"max":{"1min":3.7989999999999999,"5min":3.7989999999999999,"15min":3.7989999999999999},"last":1.2949999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6679999999999999,"5min":1.5649999999999999,"15min":1.5189999999999999},"min":{"1min":0.98599999999999999,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.776,"5min":1.643,"15min":1.6240000000000001},"min":{"1min":1.0269999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":3.5019999999999998,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.486}]},{"osd":7,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.8089999999999999,"5min":1.613,"15min":1.5860000000000001},"min":{"1min":1.3,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":4.2050000000000001,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":1.7450000000000001},{"interface":"front","average":{"1min":1.8460000000000001,"5min":1.702,"15min":1.6599999999999999},"min":{"1min":1.2370000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":3.7000000000000002,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.893}]}]},{"osd":3,"up_from":13,"seq":55834575999,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52148,"kb_used_data":2920,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319692,"statfs":{"total":96636764160,"available":96583364608,"internally_reserved":0,"allocated":2990080,"data_stored":1456005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.85899999999999999,"15min":0.89700000000000002},"min":{"1min":0.40000000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1339999999999999,"5min":1.478,"15min":1.901},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.997,"15min":1.0569999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1850000000000001,"5min":1.6180000000000001,"15min":1.6679999999999999},"last":1.581}]},{"osd":1,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.95299999999999996,"15min":0.98399999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.3400000000000001,"5min":1.508,"15min":1.508},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0609999999999999,"15min":1.121},"min":{"1min":0.64900000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5429999999999999,"5min":1.601,"15min":1.7190000000000001},"last":0.97899999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0580000000000001,"15min":1.1120000000000001},"min":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4199999999999999,"5min":1.694,"15min":1.694},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.161,"15min":1.22},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.7350000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.329}]},{"osd":4,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2989999999999999,"15min":1.3480000000000001},"min":{"1min":0.89400000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.724,"5min":1.806,"15min":1.873},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3300000000000001,"15min":1.359},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.5549999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.389,"15min":1.405},"min":{"1min":1.0549999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8260000000000001,"5min":2.798,"15min":2.798},"last":1.252},{"interface":"front","average":{"1min":1.347,"5min":1.395,"15min":1.4410000000000001},"min":{"1min":0.88800000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.883,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.282}]},{"osd":6,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.488,"15min":1.5329999999999999},"min":{"1min":1.01,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.911,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.7929999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4790000000000001,"15min":1.534},"min":{"1min":0.86699999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.0329999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.6830000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.518,"15min":1.5580000000000001},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.629},{"interface":"front","average":{"1min":1.575,"5min":1.613,"15min":1.669},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.3799999999999999,"5min":2.3799999999999999,"15min":2.5129999999999999},"last":1.4830000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575998,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56312,"kb_used_data":2920,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315528,"statfs":{"total":96636764160,"available":96579100672,"internally_reserved":0,"allocated":2990080,"data_stored":1455513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.89200000000000002,"15min":0.91600000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.349,"5min":1.532,"15min":1.669},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.99299999999999999,"15min":1.069},"min":{"1min":0.56899999999999995,"5min":0.47699999999999998,"15min":0.32600000000000001},"max":{"1min":1.3460000000000001,"5min":1.6359999999999999,"15min":2.1429999999999998},"last":1.5449999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0760000000000001,"15min":1.089},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.39200000000000002},"max":{"1min":1.591,"5min":1.645,"15min":1.978},"last":1.022},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.085,"15min":1.1379999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.393,"5min":1.609,"15min":1.7949999999999999},"last":1.2809999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0429999999999999,"15min":1.0740000000000001},"min":{"1min":0.499,"5min":0.437,"15min":0.437},"max":{"1min":1.522,"5min":1.651,"15min":1.8560000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.137,"5min":1.2170000000000001,"15min":1.2629999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.6930000000000001,"5min":1.7450000000000001,"15min":1.857},"last":1.2250000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3340000000000001,"15min":1.3999999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.73499999999999999},"max":{"1min":1.643,"5min":1.8029999999999999,"15min":2.5059999999999998},"last":1.083},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.3220000000000001,"15min":1.339},"min":{"1min":0.88200000000000001,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":1.6499999999999999,"5min":1.825,"15min":1.843},"last":1.159}]},{"osd":5,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.4139999999999999,"15min":1.4530000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.429,"15min":1.4730000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.885,"5min":1.931,"15min":2.6259999999999999},"last":1.6020000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3899999999999999,"15min":1.3999999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.8260000000000001,"5min":1.9019999999999999,"15min":2.1339999999999999},"last":1.379},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4870000000000001,"15min":1.526},"min":{"1min":0.80500000000000005,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.3380000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.5109999999999999,"15min":1.569},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.92500000000000004},"max":{"1min":2.0310000000000001,"5min":2.1469999999999998,"15min":2.8109999999999999},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4750000000000001,"15min":1.502},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80100000000000005},"max":{"1min":1.952,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.575}]}]},{"osd":1,"up_from":13,"seq":55834575998,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64644,"kb_used_data":2936,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307196,"statfs":{"total":96636764160,"available":96570568704,"internally_reserved":0,"allocated":3006464,"data_stored":1457159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.93799999999999994,"15min":0.91700000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":5.117,"5min":5.117,"15min":5.117},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.97299999999999998,"15min":1.0449999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.5069999999999999,"5min":1.5509999999999999,"15min":1.667},"last":0.66000000000000003}]},{"osd":2,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.97299999999999998,"15min":1.014},"min":{"1min":0.52600000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2350000000000001,"15min":1.27},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":3.5840000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":1.234}]},{"osd":3,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.1779999999999999,"15min":1.202},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.4670000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":0.747},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.1439999999999999,"15min":1.1639999999999999},"min":{"1min":0.48099999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5600000000000001,"5min":1.633,"15min":1.802},"last":1.169}]},{"osd":4,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.3260000000000001,"15min":1.3009999999999999},"min":{"1min":0.80500000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":5.29,"5min":5.29,"15min":5.29},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.4450000000000001,"15min":1.4259999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":5.0179999999999998,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":1.379}]},{"osd":5,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.397,"15min":1.419},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":6.0979999999999999,"5min":6.0979999999999999,"15min":6.0979999999999999},"last":1.47},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.415,"15min":1.4159999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.5299999999999998,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.1020000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.49,"15min":1.5029999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":4.0499999999999998,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.474,"15min":1.488},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":3.8799999999999999,"5min":3.8799999999999999,"15min":3.8799999999999999},"last":1.216}]},{"osd":7,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.577,"15min":1.569},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":4.46,"5min":4.46,"15min":4.46},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.625,"15min":1.613},"min":{"1min":1.0049999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":4.1040000000000001,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":1.6240000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575999,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52728,"kb_used_data":3500,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319112,"statfs":{"total":96636764160,"available":96582770688,"internally_reserved":0,"allocated":3584000,"data_stored":2045825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.86599999999999999,"15min":0.89500000000000002},"min":{"1min":0.42699999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.3420000000000001,"5min":1.52,"15min":1.619},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0649999999999999,"15min":1.1020000000000001},"min":{"1min":0.54100000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3680000000000001,"5min":1.6399999999999999,"15min":1.76},"last":0.99199999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0680000000000001,"15min":1.085},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0840000000000001,"15min":1.143},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.5169999999999999,"5min":1.6220000000000001,"15min":1.6870000000000001},"last":0.873}]},{"osd":3,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.071,"15min":1.0860000000000001},"min":{"1min":0.51400000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7609999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.206,"15min":1.2490000000000001},"min":{"1min":0.63800000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.9830000000000001},"last":1.0609999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3640000000000001,"15min":1.389},"min":{"1min":0.879,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.843,"5min":1.9390000000000001,"15min":1.95},"last":1.371},{"interface":"front","average":{"1min":1.339,"5min":1.321,"15min":1.343},"min":{"1min":1.0349999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.665,"5min":2.1539999999999999,"15min":2.4820000000000002},"last":1.2789999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.383,"15min":1.3839999999999999},"min":{"1min":1.032,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":1.7689999999999999,"5min":2.0750000000000002,"15min":2.6669999999999998},"last":2.0209999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.401,"15min":1.446},"min":{"1min":0.83399999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.9450000000000001,"5min":2.2029999999999998,"15min":2.2040000000000002},"last":1.885}]},{"osd":6,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.454,"15min":1.498},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":2.004,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.476,"15min":1.4950000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.073,"5min":2.073,"15min":2.262},"last":1.804}]},{"osd":7,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":1.605,"5min":1.5169999999999999,"15min":1.556},"min":{"1min":0.92500000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.347},"last":1.758},{"interface":"front","average":{"1min":1.536,"5min":1.4850000000000001,"15min":1.51},"min":{"1min":0.84399999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.9550000000000001,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.4670000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:25:41.593 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:25:41.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.848+0000 7f87149c4640 1 Processor -- start 2023-10-16T11:25:41.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.848+0000 7f87149c4640 1 -- start start 2023-10-16T11:25:41.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.850+0000 7f87149c4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f871006c040 0x7f871006c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:41.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.850+0000 7f87149c4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f87100ad970 0x7f871015e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:41.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.850+0000 7f87149c4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871006c960 con 0x7f871015ebc0 2023-10-16T11:25:41.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.850+0000 7f87149c4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871015b9d0 con 0x7f871006c040 2023-10-16T11:25:41.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.850+0000 7f87149c4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871015bcb0 con 0x7f87100ad970 2023-10-16T11:25:41.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.850+0000 7f870f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f871006c040 0x7f871006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:41.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.850+0000 7f870f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f871006c040 0x7f871006c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47312/0 (socket says 172.21.15.84:47312) 2023-10-16T11:25:41.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.850+0000 7f870f7fe640 1 -- 172.21.15.84:0/950566941 learned_addr learned my addr 172.21.15.84:0/950566941 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:41.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.850+0000 7f870f7fe640 1 -- 172.21.15.84:0/950566941 >> v2:172.21.15.157:3300/0 conn(0x7f87100ad970 msgr2=0x7f871015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:41.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.850+0000 7f870f7fe640 1 --2- 172.21.15.84:0/950566941 >> v2:172.21.15.157:3300/0 conn(0x7f87100ad970 0x7f871015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.850+0000 7f870f7fe640 1 -- 172.21.15.84:0/950566941 >> v1:172.21.15.157:6789/0 conn(0x7f871015ebc0 legacy=0x7f87100cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:41.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.850+0000 7f870f7fe640 1 -- 172.21.15.84:0/950566941 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871015bf90 con 0x7f871006c040 2023-10-16T11:25:41.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.852+0000 7f870f7fe640 1 --2- 172.21.15.84:0/950566941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f871006c040 0x7f871006c420 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7f87100cb060 tx=0x7f870002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7846dfec091441d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:41.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.852+0000 7f870e7fc640 1 -- 172.21.15.84:0/950566941 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8700031870 con 0x7f871006c040 2023-10-16T11:25:41.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.852+0000 7f870e7fc640 1 -- 172.21.15.84:0/950566941 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87000319d0 con 0x7f871006c040 2023-10-16T11:25:41.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.852+0000 7f870e7fc640 1 -- 172.21.15.84:0/950566941 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8700031d60 con 0x7f871006c040 2023-10-16T11:25:41.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.852+0000 7f87149c4640 1 -- 172.21.15.84:0/950566941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f871006c040 msgr2=0x7f871006c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:41.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.852+0000 7f87149c4640 1 --2- 172.21.15.84:0/950566941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f871006c040 0x7f871006c420 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7f87100cb060 tx=0x7f870002b910 comp rx=0 tx=0).stop 2023-10-16T11:25:41.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.853+0000 7f87149c4640 1 -- 172.21.15.84:0/950566941 shutdown_connections 2023-10-16T11:25:41.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.853+0000 7f87149c4640 1 --2- 172.21.15.84:0/950566941 >> v2:172.21.15.157:3300/0 conn(0x7f87100ad970 0x7f871015e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.853+0000 7f87149c4640 1 --2- 172.21.15.84:0/950566941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f871006c040 0x7f871006c420 unknown :-1 s=CLOSED pgs=2933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.853+0000 7f87149c4640 1 -- 172.21.15.84:0/950566941 >> 172.21.15.84:0/950566941 conn(0x7f87100ce6a0 msgr2=0x7f87100ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:41.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.853+0000 7f87149c4640 1 -- 172.21.15.84:0/950566941 shutdown_connections 2023-10-16T11:25:41.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.853+0000 7f87149c4640 1 -- 172.21.15.84:0/950566941 wait complete. 2023-10-16T11:25:41.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.854+0000 7f87149c4640 1 Processor -- start 2023-10-16T11:25:41.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.855+0000 7f87149c4640 1 -- start start 2023-10-16T11:25:41.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.855+0000 7f87149c4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f871006c040 0x7f87100af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:41.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.855+0000 7f87149c4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87100ad970 0x7f871015a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.856+0000 7f87149c4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f871015ebc0 0x7f871015ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.856+0000 7f87149c4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87101e4730 con 0x7f871015ebc0 2023-10-16T11:25:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.856+0000 7f87149c4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87101e56d0 con 0x7f871006c040 2023-10-16T11:25:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.856+0000 7f87149c4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87100af280 con 0x7f87100ad970 2023-10-16T11:25:41.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.856+0000 7f870ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f871015ebc0 0x7f871015ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:41.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.856+0000 7f870ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f871015ebc0 0x7f871015ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47324/0 (socket says 172.21.15.84:47324) 2023-10-16T11:25:41.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.856+0000 7f870ffff640 1 -- 172.21.15.84:0/308760114 learned_addr learned my addr 172.21.15.84:0/308760114 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:41.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.856+0000 7f870f7fe640 1 --2- 172.21.15.84:0/308760114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f871006c040 0x7f87100af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:41.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.856+0000 7f870effd640 1 --2- 172.21.15.84:0/308760114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87100ad970 0x7f871015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:41.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.856+0000 7f870ffff640 1 -- 172.21.15.84:0/308760114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87100ad970 msgr2=0x7f871015a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:41.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.856+0000 7f870ffff640 1 --2- 172.21.15.84:0/308760114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87100ad970 0x7f871015a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.856+0000 7f870ffff640 1 -- 172.21.15.84:0/308760114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f871006c040 msgr2=0x7f87100af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:41.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.856+0000 7f870ffff640 1 --2- 172.21.15.84:0/308760114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f871006c040 0x7f87100af710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:41.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.856+0000 7f870ffff640 1 -- 172.21.15.84:0/308760114 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87000094d0 con 0x7f871015ebc0 2023-10-16T11:25:41.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.856+0000 7f870effd640 1 --2- 172.21.15.84:0/308760114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87100ad970 0x7f871015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:41.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.857+0000 7f870ffff640 1 --2- 172.21.15.84:0/308760114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f871015ebc0 0x7f871015ac50 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7f870800c7b0 tx=0x7f870800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:41.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.857+0000 7f86f7fff640 1 -- 172.21.15.84:0/308760114 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8708003d80 con 0x7f871015ebc0 2023-10-16T11:25:41.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.857+0000 7f86f7fff640 1 -- 172.21.15.84:0/308760114 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8708005070 con 0x7f871015ebc0 2023-10-16T11:25:41.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.857+0000 7f86f7fff640 1 -- 172.21.15.84:0/308760114 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8708005420 con 0x7f871015ebc0 2023-10-16T11:25:41.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.857+0000 7f87149c4640 1 -- 172.21.15.84:0/308760114 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87100ae1e0 con 0x7f871015ebc0 2023-10-16T11:25:41.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.857+0000 7f87149c4640 1 -- 172.21.15.84:0/308760114 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87100ae450 con 0x7f871015ebc0 2023-10-16T11:25:41.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.858+0000 7f87149c4640 1 -- 172.21.15.84:0/308760114 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871006b330 con 0x7f871015ebc0 2023-10-16T11:25:41.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.859+0000 7f86f7fff640 1 -- 172.21.15.84:0/308760114 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f8708005580 con 0x7f871015ebc0 2023-10-16T11:25:41.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.861+0000 7f86f7fff640 1 --2- 172.21.15.84:0/308760114 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f86fc06b0e0 0x7f86fc06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:41.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.861+0000 7f86f7fff640 1 -- 172.21.15.84:0/308760114 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f8708088d90 con 0x7f871015ebc0 2023-10-16T11:25:41.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.861+0000 7f870f7fe640 1 --2- 172.21.15.84:0/308760114 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f86fc06b0e0 0x7f86fc06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:41.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.861+0000 7f870f7fe640 1 --2- 172.21.15.84:0/308760114 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f86fc06b0e0 0x7f86fc06d5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8700036000 tx=0x7f870003c040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:41.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:41.865+0000 7f86f7fff640 1 -- 172.21.15.84:0/308760114 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8708053ac0 con 0x7f871015ebc0 2023-10-16T11:25:42.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.054+0000 7f87149c4640 1 -- 172.21.15.84:0/308760114 --> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87100cc550 con 0x7f86fc06b0e0 2023-10-16T11:25:42.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.077+0000 7f86f7fff640 1 -- 172.21.15.84:0/308760114 <== mgr.11814 v2:172.21.15.84:6832/2963529201 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287369 (secure 0 0 0) 0x7f8700031810 con 0x7f86fc06b0e0 2023-10-16T11:25:42.084 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:42.085 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:25:42.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.084+0000 7f87149c4640 1 -- 172.21.15.84:0/308760114 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f86fc06b0e0 msgr2=0x7f86fc06d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:42.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.084+0000 7f87149c4640 1 --2- 172.21.15.84:0/308760114 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f86fc06b0e0 0x7f86fc06d5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8700036000 tx=0x7f870003c040 comp rx=0 tx=0).stop 2023-10-16T11:25:42.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.084+0000 7f87149c4640 1 -- 172.21.15.84:0/308760114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f871015ebc0 msgr2=0x7f871015ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:42.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.084+0000 7f87149c4640 1 --2- 172.21.15.84:0/308760114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f871015ebc0 0x7f871015ac50 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7f870800c7b0 tx=0x7f870800cc70 comp rx=0 tx=0).stop 2023-10-16T11:25:42.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.084+0000 7f87149c4640 1 -- 172.21.15.84:0/308760114 shutdown_connections 2023-10-16T11:25:42.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.084+0000 7f87149c4640 1 --2- 172.21.15.84:0/308760114 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f86fc06b0e0 0x7f86fc06d5a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.084+0000 7f87149c4640 1 --2- 172.21.15.84:0/308760114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f871015ebc0 0x7f871015ac50 unknown :-1 s=CLOSED pgs=2934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.084+0000 7f87149c4640 1 --2- 172.21.15.84:0/308760114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87100ad970 0x7f871015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.084+0000 7f87149c4640 1 --2- 172.21.15.84:0/308760114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f871006c040 0x7f87100af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.084+0000 7f87149c4640 1 -- 172.21.15.84:0/308760114 >> 172.21.15.84:0/308760114 conn(0x7f87100ce6a0 msgr2=0x7f871015db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:42.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.084+0000 7f87149c4640 1 -- 172.21.15.84:0/308760114 shutdown_connections 2023-10-16T11:25:42.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.084+0000 7f87149c4640 1 -- 172.21.15.84:0/308760114 wait complete. 2023-10-16T11:25:42.106 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-10-16T11:25:41.243466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":665,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2827,"ondisk_log_size":2827,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":412816,"kb_used_data":24552,"kb_used_omap":12,"kb_used_meta":388147,"kb_avail":754561904,"statfs":{"total":773094113280,"available":772671389696,"internally_reserved":0,"allocated":25141248,"data_stored":12828940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":397463097},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001469"},"pg_stats":[{"pgid":"31.3f","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.194090+0000","last_change":"2023-10-16T11:25:37.173492+0000","last_active":"2023-10-16T11:25:39.194090+0000","last_peered":"2023-10-16T11:25:39.194090+0000","last_clean":"2023-10-16T11:25:39.194090+0000","last_became_active":"2023-10-16T11:25:37.167017+0000","last_became_peered":"2023-10-16T11:25:37.167017+0000","last_unstale":"2023-10-16T11:25:39.194090+0000","last_undegraded":"2023-10-16T11:25:39.194090+0000","last_fullsized":"2023-10-16T11:25:39.194090+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:07.791499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.3e","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193814+0000","last_change":"2023-10-16T11:25:37.179117+0000","last_active":"2023-10-16T11:25:39.193814+0000","last_peered":"2023-10-16T11:25:39.193814+0000","last_clean":"2023-10-16T11:25:39.193814+0000","last_became_active":"2023-10-16T11:25:37.178849+0000","last_became_peered":"2023-10-16T11:25:37.178849+0000","last_unstale":"2023-10-16T11:25:39.193814+0000","last_undegraded":"2023-10-16T11:25:39.193814+0000","last_fullsized":"2023-10-16T11:25:39.193814+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:18:57.278249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.3d","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616308+0000","last_change":"2023-10-16T11:25:37.179065+0000","last_active":"2023-10-16T11:25:38.616308+0000","last_peered":"2023-10-16T11:25:38.616308+0000","last_clean":"2023-10-16T11:25:38.616308+0000","last_became_active":"2023-10-16T11:25:37.178779+0000","last_became_peered":"2023-10-16T11:25:37.178779+0000","last_unstale":"2023-10-16T11:25:38.616308+0000","last_undegraded":"2023-10-16T11:25:38.616308+0000","last_fullsized":"2023-10-16T11:25:38.616308+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:09:47.367715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3c","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.171558+0000","last_change":"2023-10-16T11:25:37.172480+0000","last_active":"2023-10-16T11:25:38.171558+0000","last_peered":"2023-10-16T11:25:38.171558+0000","last_clean":"2023-10-16T11:25:38.171558+0000","last_became_active":"2023-10-16T11:25:37.172192+0000","last_became_peered":"2023-10-16T11:25:37.172192+0000","last_unstale":"2023-10-16T11:25:38.171558+0000","last_undegraded":"2023-10-16T11:25:38.171558+0000","last_fullsized":"2023-10-16T11:25:38.171558+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.3b","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.197487+0000","last_change":"2023-10-16T11:25:37.177505+0000","last_active":"2023-10-16T11:25:39.197487+0000","last_peered":"2023-10-16T11:25:39.197487+0000","last_clean":"2023-10-16T11:25:39.197487+0000","last_became_active":"2023-10-16T11:25:37.174334+0000","last_became_peered":"2023-10-16T11:25:37.174334+0000","last_unstale":"2023-10-16T11:25:39.197487+0000","last_undegraded":"2023-10-16T11:25:39.197487+0000","last_fullsized":"2023-10-16T11:25:39.197487+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.3a","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.171518+0000","last_change":"2023-10-16T11:25:37.177781+0000","last_active":"2023-10-16T11:25:38.171518+0000","last_peered":"2023-10-16T11:25:38.171518+0000","last_clean":"2023-10-16T11:25:38.171518+0000","last_became_active":"2023-10-16T11:25:37.177501+0000","last_became_peered":"2023-10-16T11:25:37.177501+0000","last_unstale":"2023-10-16T11:25:38.171518+0000","last_undegraded":"2023-10-16T11:25:38.171518+0000","last_fullsized":"2023-10-16T11:25:38.171518+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.39","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.173719+0000","last_change":"2023-10-16T11:25:37.182745+0000","last_active":"2023-10-16T11:25:38.173719+0000","last_peered":"2023-10-16T11:25:38.173719+0000","last_clean":"2023-10-16T11:25:38.173719+0000","last_became_active":"2023-10-16T11:25:37.182437+0000","last_became_peered":"2023-10-16T11:25:37.182437+0000","last_unstale":"2023-10-16T11:25:38.173719+0000","last_undegraded":"2023-10-16T11:25:38.173719+0000","last_fullsized":"2023-10-16T11:25:38.173719+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.38","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.197683+0000","last_change":"2023-10-16T11:25:37.174929+0000","last_active":"2023-10-16T11:25:39.197683+0000","last_peered":"2023-10-16T11:25:39.197683+0000","last_clean":"2023-10-16T11:25:39.197683+0000","last_became_active":"2023-10-16T11:25:37.172810+0000","last_became_peered":"2023-10-16T11:25:37.172810+0000","last_unstale":"2023-10-16T11:25:39.197683+0000","last_undegraded":"2023-10-16T11:25:39.197683+0000","last_fullsized":"2023-10-16T11:25:39.197683+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.37","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.175224+0000","last_change":"2023-10-16T11:25:37.177647+0000","last_active":"2023-10-16T11:25:38.175224+0000","last_peered":"2023-10-16T11:25:38.175224+0000","last_clean":"2023-10-16T11:25:38.175224+0000","last_became_active":"2023-10-16T11:25:37.176875+0000","last_became_peered":"2023-10-16T11:25:37.176875+0000","last_unstale":"2023-10-16T11:25:38.175224+0000","last_undegraded":"2023-10-16T11:25:38.175224+0000","last_fullsized":"2023-10-16T11:25:38.175224+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:21.630921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.36","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174687+0000","last_change":"2023-10-16T11:25:37.172841+0000","last_active":"2023-10-16T11:25:38.174687+0000","last_peered":"2023-10-16T11:25:38.174687+0000","last_clean":"2023-10-16T11:25:38.174687+0000","last_became_active":"2023-10-16T11:25:37.172550+0000","last_became_peered":"2023-10-16T11:25:37.172550+0000","last_unstale":"2023-10-16T11:25:38.174687+0000","last_undegraded":"2023-10-16T11:25:38.174687+0000","last_fullsized":"2023-10-16T11:25:38.174687+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.35","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.183071+0000","last_change":"2023-10-16T11:25:37.181386+0000","last_active":"2023-10-16T11:25:38.183071+0000","last_peered":"2023-10-16T11:25:38.183071+0000","last_clean":"2023-10-16T11:25:38.183071+0000","last_became_active":"2023-10-16T11:25:37.181007+0000","last_became_peered":"2023-10-16T11:25:37.181007+0000","last_unstale":"2023-10-16T11:25:38.183071+0000","last_undegraded":"2023-10-16T11:25:38.183071+0000","last_fullsized":"2023-10-16T11:25:38.183071+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:46.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.34","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.198283+0000","last_change":"2023-10-16T11:25:37.179103+0000","last_active":"2023-10-16T11:25:39.198283+0000","last_peered":"2023-10-16T11:25:39.198283+0000","last_clean":"2023-10-16T11:25:39.198283+0000","last_became_active":"2023-10-16T11:25:37.178445+0000","last_became_peered":"2023-10-16T11:25:37.178445+0000","last_unstale":"2023-10-16T11:25:39.198283+0000","last_undegraded":"2023-10-16T11:25:39.198283+0000","last_fullsized":"2023-10-16T11:25:39.198283+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:38.218704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.33","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.175976+0000","last_change":"2023-10-16T11:25:37.182921+0000","last_active":"2023-10-16T11:25:38.175976+0000","last_peered":"2023-10-16T11:25:38.175976+0000","last_clean":"2023-10-16T11:25:38.175976+0000","last_became_active":"2023-10-16T11:25:37.182603+0000","last_became_peered":"2023-10-16T11:25:37.182603+0000","last_unstale":"2023-10-16T11:25:38.175976+0000","last_undegraded":"2023-10-16T11:25:38.175976+0000","last_fullsized":"2023-10-16T11:25:38.175976+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.32","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193727+0000","last_change":"2023-10-16T11:25:37.171715+0000","last_active":"2023-10-16T11:25:39.193727+0000","last_peered":"2023-10-16T11:25:39.193727+0000","last_clean":"2023-10-16T11:25:39.193727+0000","last_became_active":"2023-10-16T11:25:37.171421+0000","last_became_peered":"2023-10-16T11:25:37.171421+0000","last_unstale":"2023-10-16T11:25:39.193727+0000","last_undegraded":"2023-10-16T11:25:39.193727+0000","last_fullsized":"2023-10-16T11:25:39.193727+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:10.207081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.31","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174027+0000","last_change":"2023-10-16T11:25:37.177928+0000","last_active":"2023-10-16T11:25:38.174027+0000","last_peered":"2023-10-16T11:25:38.174027+0000","last_clean":"2023-10-16T11:25:38.174027+0000","last_became_active":"2023-10-16T11:25:37.177612+0000","last_became_peered":"2023-10-16T11:25:37.177612+0000","last_unstale":"2023-10-16T11:25:38.174027+0000","last_undegraded":"2023-10-16T11:25:38.174027+0000","last_fullsized":"2023-10-16T11:25:38.174027+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:44.826687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.30","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.615796+0000","last_change":"2023-10-16T11:25:37.170292+0000","last_active":"2023-10-16T11:25:38.615796+0000","last_peered":"2023-10-16T11:25:38.615796+0000","last_clean":"2023-10-16T11:25:38.615796+0000","last_became_active":"2023-10-16T11:25:37.169997+0000","last_became_peered":"2023-10-16T11:25:37.169997+0000","last_unstale":"2023-10-16T11:25:38.615796+0000","last_undegraded":"2023-10-16T11:25:38.615796+0000","last_fullsized":"2023-10-16T11:25:38.615796+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:09.329721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2f","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.198740+0000","last_change":"2023-10-16T11:25:37.162923+0000","last_active":"2023-10-16T11:25:39.198740+0000","last_peered":"2023-10-16T11:25:39.198740+0000","last_clean":"2023-10-16T11:25:39.198740+0000","last_became_active":"2023-10-16T11:25:37.162649+0000","last_became_peered":"2023-10-16T11:25:37.162649+0000","last_unstale":"2023-10-16T11:25:39.198740+0000","last_undegraded":"2023-10-16T11:25:39.198740+0000","last_fullsized":"2023-10-16T11:25:39.198740+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:46.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2e","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193747+0000","last_change":"2023-10-16T11:25:37.174044+0000","last_active":"2023-10-16T11:25:39.193747+0000","last_peered":"2023-10-16T11:25:39.193747+0000","last_clean":"2023-10-16T11:25:39.193747+0000","last_became_active":"2023-10-16T11:25:37.166694+0000","last_became_peered":"2023-10-16T11:25:37.166694+0000","last_unstale":"2023-10-16T11:25:39.193747+0000","last_undegraded":"2023-10-16T11:25:39.193747+0000","last_fullsized":"2023-10-16T11:25:39.193747+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:23.146288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2d","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.197203+0000","last_change":"2023-10-16T11:25:37.183893+0000","last_active":"2023-10-16T11:25:39.197203+0000","last_peered":"2023-10-16T11:25:39.197203+0000","last_clean":"2023-10-16T11:25:39.197203+0000","last_became_active":"2023-10-16T11:25:37.183586+0000","last_became_peered":"2023-10-16T11:25:37.183586+0000","last_unstale":"2023-10-16T11:25:39.197203+0000","last_undegraded":"2023-10-16T11:25:39.197203+0000","last_fullsized":"2023-10-16T11:25:39.197203+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:43.813878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2c","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616722+0000","last_change":"2023-10-16T11:25:37.172984+0000","last_active":"2023-10-16T11:25:38.616722+0000","last_peered":"2023-10-16T11:25:38.616722+0000","last_clean":"2023-10-16T11:25:38.616722+0000","last_became_active":"2023-10-16T11:25:37.172403+0000","last_became_peered":"2023-10-16T11:25:37.172403+0000","last_unstale":"2023-10-16T11:25:38.616722+0000","last_undegraded":"2023-10-16T11:25:38.616722+0000","last_fullsized":"2023-10-16T11:25:38.616722+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:03.912995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2b","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.196844+0000","last_change":"2023-10-16T11:25:37.173026+0000","last_active":"2023-10-16T11:25:39.196844+0000","last_peered":"2023-10-16T11:25:39.196844+0000","last_clean":"2023-10-16T11:25:39.196844+0000","last_became_active":"2023-10-16T11:25:37.172680+0000","last_became_peered":"2023-10-16T11:25:37.172680+0000","last_unstale":"2023-10-16T11:25:39.196844+0000","last_undegraded":"2023-10-16T11:25:39.196844+0000","last_fullsized":"2023-10-16T11:25:39.196844+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:56.314696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2a","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.175816+0000","last_change":"2023-10-16T11:25:37.176870+0000","last_active":"2023-10-16T11:25:38.175816+0000","last_peered":"2023-10-16T11:25:38.175816+0000","last_clean":"2023-10-16T11:25:38.175816+0000","last_became_active":"2023-10-16T11:25:37.175230+0000","last_became_peered":"2023-10-16T11:25:37.175230+0000","last_unstale":"2023-10-16T11:25:38.175816+0000","last_undegraded":"2023-10-16T11:25:38.175816+0000","last_fullsized":"2023-10-16T11:25:38.175816+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.29","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.197804+0000","last_change":"2023-10-16T11:25:37.169637+0000","last_active":"2023-10-16T11:25:39.197804+0000","last_peered":"2023-10-16T11:25:39.197804+0000","last_clean":"2023-10-16T11:25:39.197804+0000","last_became_active":"2023-10-16T11:25:37.169375+0000","last_became_peered":"2023-10-16T11:25:37.169375+0000","last_unstale":"2023-10-16T11:25:39.197804+0000","last_undegraded":"2023-10-16T11:25:39.197804+0000","last_fullsized":"2023-10-16T11:25:39.197804+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:42:23.188876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.28","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193571+0000","last_change":"2023-10-16T11:25:37.187270+0000","last_active":"2023-10-16T11:25:39.193571+0000","last_peered":"2023-10-16T11:25:39.193571+0000","last_clean":"2023-10-16T11:25:39.193571+0000","last_became_active":"2023-10-16T11:25:37.186962+0000","last_became_peered":"2023-10-16T11:25:37.186962+0000","last_unstale":"2023-10-16T11:25:39.193571+0000","last_undegraded":"2023-10-16T11:25:39.193571+0000","last_fullsized":"2023-10-16T11:25:39.193571+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:56:45.915920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.27","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.172224+0000","last_change":"2023-10-16T11:25:37.177057+0000","last_active":"2023-10-16T11:25:38.172224+0000","last_peered":"2023-10-16T11:25:38.172224+0000","last_clean":"2023-10-16T11:25:38.172224+0000","last_became_active":"2023-10-16T11:25:37.171369+0000","last_became_peered":"2023-10-16T11:25:37.171369+0000","last_unstale":"2023-10-16T11:25:38.172224+0000","last_undegraded":"2023-10-16T11:25:38.172224+0000","last_fullsized":"2023-10-16T11:25:38.172224+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.26","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.176417+0000","last_change":"2023-10-16T11:25:37.182875+0000","last_active":"2023-10-16T11:25:38.176417+0000","last_peered":"2023-10-16T11:25:38.176417+0000","last_clean":"2023-10-16T11:25:38.176417+0000","last_became_active":"2023-10-16T11:25:37.182570+0000","last_became_peered":"2023-10-16T11:25:37.182570+0000","last_unstale":"2023-10-16T11:25:38.176417+0000","last_undegraded":"2023-10-16T11:25:38.176417+0000","last_fullsized":"2023-10-16T11:25:38.176417+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:14.504702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.25","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.183204+0000","last_change":"2023-10-16T11:25:37.178245+0000","last_active":"2023-10-16T11:25:38.183204+0000","last_peered":"2023-10-16T11:25:38.183204+0000","last_clean":"2023-10-16T11:25:38.183204+0000","last_became_active":"2023-10-16T11:25:37.177914+0000","last_became_peered":"2023-10-16T11:25:37.177914+0000","last_unstale":"2023-10-16T11:25:38.183204+0000","last_undegraded":"2023-10-16T11:25:38.183204+0000","last_fullsized":"2023-10-16T11:25:38.183204+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:12.014513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.24","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193750+0000","last_change":"2023-10-16T11:25:37.170623+0000","last_active":"2023-10-16T11:25:39.193750+0000","last_peered":"2023-10-16T11:25:39.193750+0000","last_clean":"2023-10-16T11:25:39.193750+0000","last_became_active":"2023-10-16T11:25:37.170365+0000","last_became_peered":"2023-10-16T11:25:37.170365+0000","last_unstale":"2023-10-16T11:25:39.193750+0000","last_undegraded":"2023-10-16T11:25:39.193750+0000","last_fullsized":"2023-10-16T11:25:39.193750+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:52.443160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.23","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182831+0000","last_change":"2023-10-16T11:25:37.178693+0000","last_active":"2023-10-16T11:25:38.182831+0000","last_peered":"2023-10-16T11:25:38.182831+0000","last_clean":"2023-10-16T11:25:38.182831+0000","last_became_active":"2023-10-16T11:25:37.175403+0000","last_became_peered":"2023-10-16T11:25:37.175403+0000","last_unstale":"2023-10-16T11:25:38.182831+0000","last_undegraded":"2023-10-16T11:25:38.182831+0000","last_fullsized":"2023-10-16T11:25:38.182831+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:15:31.572558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.22","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.196822+0000","last_change":"2023-10-16T11:25:37.181812+0000","last_active":"2023-10-16T11:25:39.196822+0000","last_peered":"2023-10-16T11:25:39.196822+0000","last_clean":"2023-10-16T11:25:39.196822+0000","last_became_active":"2023-10-16T11:25:37.181529+0000","last_became_peered":"2023-10-16T11:25:37.181529+0000","last_unstale":"2023-10-16T11:25:39.196822+0000","last_undegraded":"2023-10-16T11:25:39.196822+0000","last_fullsized":"2023-10-16T11:25:39.196822+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:35.912014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.21","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.171784+0000","last_change":"2023-10-16T11:25:37.166981+0000","last_active":"2023-10-16T11:25:38.171784+0000","last_peered":"2023-10-16T11:25:38.171784+0000","last_clean":"2023-10-16T11:25:38.171784+0000","last_became_active":"2023-10-16T11:25:37.164147+0000","last_became_peered":"2023-10-16T11:25:37.164147+0000","last_unstale":"2023-10-16T11:25:38.171784+0000","last_undegraded":"2023-10-16T11:25:38.171784+0000","last_fullsized":"2023-10-16T11:25:38.171784+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.20","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193639+0000","last_change":"2023-10-16T11:25:37.178631+0000","last_active":"2023-10-16T11:25:39.193639+0000","last_peered":"2023-10-16T11:25:39.193639+0000","last_clean":"2023-10-16T11:25:39.193639+0000","last_became_active":"2023-10-16T11:25:37.167402+0000","last_became_peered":"2023-10-16T11:25:37.167402+0000","last_unstale":"2023-10-16T11:25:39.193639+0000","last_undegraded":"2023-10-16T11:25:39.193639+0000","last_fullsized":"2023-10-16T11:25:39.193639+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:55.572264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616997+0000","last_change":"2023-10-16T11:25:37.177706+0000","last_active":"2023-10-16T11:25:38.616997+0000","last_peered":"2023-10-16T11:25:38.616997+0000","last_clean":"2023-10-16T11:25:38.616997+0000","last_became_active":"2023-10-16T11:25:37.176806+0000","last_became_peered":"2023-10-16T11:25:37.176806+0000","last_unstale":"2023-10-16T11:25:38.616997+0000","last_undegraded":"2023-10-16T11:25:38.616997+0000","last_fullsized":"2023-10-16T11:25:38.616997+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:42:32.585642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1e","version":"236'1","reported_seq":13,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.184494+0000","last_change":"2023-10-16T11:25:36.169327+0000","last_active":"2023-10-16T11:25:37.184494+0000","last_peered":"2023-10-16T11:25:37.184494+0000","last_clean":"2023-10-16T11:25:37.184494+0000","last_became_active":"2023-10-16T11:25:36.169046+0000","last_became_peered":"2023-10-16T11:25:36.169046+0000","last_unstale":"2023-10-16T11:25:37.184494+0000","last_undegraded":"2023-10-16T11:25:37.184494+0000","last_fullsized":"2023-10-16T11:25:37.184494+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:34.899695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182727+0000","last_change":"2023-10-16T11:25:37.173736+0000","last_active":"2023-10-16T11:25:38.182727+0000","last_peered":"2023-10-16T11:25:38.182727+0000","last_clean":"2023-10-16T11:25:38.182727+0000","last_became_active":"2023-10-16T11:25:37.173482+0000","last_became_peered":"2023-10-16T11:25:37.173482+0000","last_unstale":"2023-10-16T11:25:38.182727+0000","last_undegraded":"2023-10-16T11:25:38.182727+0000","last_fullsized":"2023-10-16T11:25:38.182727+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:42:23.188876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1d","version":"236'2","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.190697+0000","last_change":"2023-10-16T11:25:36.172861+0000","last_active":"2023-10-16T11:25:37.190697+0000","last_peered":"2023-10-16T11:25:37.190697+0000","last_clean":"2023-10-16T11:25:37.190697+0000","last_became_active":"2023-10-16T11:25:36.172574+0000","last_became_peered":"2023-10-16T11:25:36.172574+0000","last_unstale":"2023-10-16T11:25:37.190697+0000","last_undegraded":"2023-10-16T11:25:37.190697+0000","last_fullsized":"2023-10-16T11:25:37.190697+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:22:17.667795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.197306+0000","last_change":"2023-10-16T11:25:37.167451+0000","last_active":"2023-10-16T11:25:39.197306+0000","last_peered":"2023-10-16T11:25:39.197306+0000","last_clean":"2023-10-16T11:25:39.197306+0000","last_became_active":"2023-10-16T11:25:37.167157+0000","last_became_peered":"2023-10-16T11:25:37.167157+0000","last_unstale":"2023-10-16T11:25:39.197306+0000","last_undegraded":"2023-10-16T11:25:39.197306+0000","last_fullsized":"2023-10-16T11:25:39.197306+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:14.504702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1c","version":"236'2","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.179729+0000","last_change":"2023-10-16T11:25:36.153273+0000","last_active":"2023-10-16T11:25:37.179729+0000","last_peered":"2023-10-16T11:25:37.179729+0000","last_clean":"2023-10-16T11:25:37.179729+0000","last_became_active":"2023-10-16T11:25:36.152960+0000","last_became_peered":"2023-10-16T11:25:36.152960+0000","last_unstale":"2023-10-16T11:25:37.179729+0000","last_undegraded":"2023-10-16T11:25:37.179729+0000","last_fullsized":"2023-10-16T11:25:37.179729+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:08.218268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182469+0000","last_change":"2023-10-16T11:25:37.174071+0000","last_active":"2023-10-16T11:25:38.182469+0000","last_peered":"2023-10-16T11:25:38.182469+0000","last_clean":"2023-10-16T11:25:38.182469+0000","last_became_active":"2023-10-16T11:25:37.173436+0000","last_became_peered":"2023-10-16T11:25:37.173436+0000","last_unstale":"2023-10-16T11:25:38.182469+0000","last_undegraded":"2023-10-16T11:25:38.182469+0000","last_fullsized":"2023-10-16T11:25:38.182469+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1b","version":"236'1","reported_seq":13,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.178750+0000","last_change":"2023-10-16T11:25:36.156408+0000","last_active":"2023-10-16T11:25:37.178750+0000","last_peered":"2023-10-16T11:25:37.178750+0000","last_clean":"2023-10-16T11:25:37.178750+0000","last_became_active":"2023-10-16T11:25:36.155786+0000","last_became_peered":"2023-10-16T11:25:36.155786+0000","last_unstale":"2023-10-16T11:25:37.178750+0000","last_undegraded":"2023-10-16T11:25:37.178750+0000","last_fullsized":"2023-10-16T11:25:37.178750+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:01.925277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174081+0000","last_change":"2023-10-16T11:25:37.179656+0000","last_active":"2023-10-16T11:25:38.174081+0000","last_peered":"2023-10-16T11:25:38.174081+0000","last_clean":"2023-10-16T11:25:38.174081+0000","last_became_active":"2023-10-16T11:25:37.179361+0000","last_became_peered":"2023-10-16T11:25:37.179361+0000","last_unstale":"2023-10-16T11:25:38.174081+0000","last_undegraded":"2023-10-16T11:25:38.174081+0000","last_fullsized":"2023-10-16T11:25:38.174081+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:09.230587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"236'1","reported_seq":13,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.183638+0000","last_change":"2023-10-16T11:25:36.169348+0000","last_active":"2023-10-16T11:25:37.183638+0000","last_peered":"2023-10-16T11:25:37.183638+0000","last_clean":"2023-10-16T11:25:37.183638+0000","last_became_active":"2023-10-16T11:25:36.169053+0000","last_became_peered":"2023-10-16T11:25:36.169053+0000","last_unstale":"2023-10-16T11:25:37.183638+0000","last_undegraded":"2023-10-16T11:25:37.183638+0000","last_fullsized":"2023-10-16T11:25:37.183638+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:05.819502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.615841+0000","last_change":"2023-10-16T11:25:37.176966+0000","last_active":"2023-10-16T11:25:38.615841+0000","last_peered":"2023-10-16T11:25:38.615841+0000","last_clean":"2023-10-16T11:25:38.615841+0000","last_became_active":"2023-10-16T11:25:37.174621+0000","last_became_peered":"2023-10-16T11:25:37.174621+0000","last_unstale":"2023-10-16T11:25:38.615841+0000","last_undegraded":"2023-10-16T11:25:38.615841+0000","last_fullsized":"2023-10-16T11:25:38.615841+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:56:09.516122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.150607+0000","last_change":"2023-10-16T11:25:36.169635+0000","last_active":"2023-10-16T11:25:37.150607+0000","last_peered":"2023-10-16T11:25:37.150607+0000","last_clean":"2023-10-16T11:25:37.150607+0000","last_became_active":"2023-10-16T11:25:36.169016+0000","last_became_peered":"2023-10-16T11:25:36.169016+0000","last_unstale":"2023-10-16T11:25:37.150607+0000","last_undegraded":"2023-10-16T11:25:37.150607+0000","last_fullsized":"2023-10-16T11:25:37.150607+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:53:06.412234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193708+0000","last_change":"2023-10-16T11:25:37.184475+0000","last_active":"2023-10-16T11:25:39.193708+0000","last_peered":"2023-10-16T11:25:39.193708+0000","last_clean":"2023-10-16T11:25:39.193708+0000","last_became_active":"2023-10-16T11:25:37.179914+0000","last_became_peered":"2023-10-16T11:25:37.179914+0000","last_unstale":"2023-10-16T11:25:39.193708+0000","last_undegraded":"2023-10-16T11:25:39.193708+0000","last_fullsized":"2023-10-16T11:25:39.193708+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:53.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.164726+0000","last_change":"2023-10-16T11:25:36.155589+0000","last_active":"2023-10-16T11:25:37.164726+0000","last_peered":"2023-10-16T11:25:37.164726+0000","last_clean":"2023-10-16T11:25:37.164726+0000","last_became_active":"2023-10-16T11:25:36.155087+0000","last_became_peered":"2023-10-16T11:25:36.155087+0000","last_unstale":"2023-10-16T11:25:37.164726+0000","last_undegraded":"2023-10-16T11:25:37.164726+0000","last_fullsized":"2023-10-16T11:25:37.164726+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:52:38.937212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.194016+0000","last_change":"2023-10-16T11:25:37.175447+0000","last_active":"2023-10-16T11:25:39.194016+0000","last_peered":"2023-10-16T11:25:39.194016+0000","last_clean":"2023-10-16T11:25:39.194016+0000","last_became_active":"2023-10-16T11:25:37.173563+0000","last_became_peered":"2023-10-16T11:25:37.173563+0000","last_unstale":"2023-10-16T11:25:39.194016+0000","last_undegraded":"2023-10-16T11:25:39.194016+0000","last_fullsized":"2023-10-16T11:25:39.194016+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:54.544135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.149697+0000","last_change":"2023-10-16T11:25:36.169351+0000","last_active":"2023-10-16T11:25:37.149697+0000","last_peered":"2023-10-16T11:25:37.149697+0000","last_clean":"2023-10-16T11:25:37.149697+0000","last_became_active":"2023-10-16T11:25:36.168347+0000","last_became_peered":"2023-10-16T11:25:36.168347+0000","last_unstale":"2023-10-16T11:25:37.149697+0000","last_undegraded":"2023-10-16T11:25:37.149697+0000","last_fullsized":"2023-10-16T11:25:37.149697+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:19.077096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.183753+0000","last_change":"2023-10-16T11:25:37.179048+0000","last_active":"2023-10-16T11:25:38.183753+0000","last_peered":"2023-10-16T11:25:38.183753+0000","last_clean":"2023-10-16T11:25:38.183753+0000","last_became_active":"2023-10-16T11:25:37.178751+0000","last_became_peered":"2023-10-16T11:25:37.178751+0000","last_unstale":"2023-10-16T11:25:38.183753+0000","last_undegraded":"2023-10-16T11:25:38.183753+0000","last_fullsized":"2023-10-16T11:25:38.183753+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:48:10.906108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.165388+0000","last_change":"2023-10-16T11:25:36.154256+0000","last_active":"2023-10-16T11:25:37.165388+0000","last_peered":"2023-10-16T11:25:37.165388+0000","last_clean":"2023-10-16T11:25:37.165388+0000","last_became_active":"2023-10-16T11:25:36.153421+0000","last_became_peered":"2023-10-16T11:25:36.153421+0000","last_unstale":"2023-10-16T11:25:37.165388+0000","last_undegraded":"2023-10-16T11:25:37.165388+0000","last_fullsized":"2023-10-16T11:25:37.165388+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:09.436508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616767+0000","last_change":"2023-10-16T11:25:37.188270+0000","last_active":"2023-10-16T11:25:38.616767+0000","last_peered":"2023-10-16T11:25:38.616767+0000","last_clean":"2023-10-16T11:25:38.616767+0000","last_became_active":"2023-10-16T11:25:37.187995+0000","last_became_peered":"2023-10-16T11:25:37.187995+0000","last_unstale":"2023-10-16T11:25:38.616767+0000","last_undegraded":"2023-10-16T11:25:38.616767+0000","last_fullsized":"2023-10-16T11:25:38.616767+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:42:22.542072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.15","version":"236'3","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.191322+0000","last_change":"2023-10-16T11:25:36.171645+0000","last_active":"2023-10-16T11:25:37.191322+0000","last_peered":"2023-10-16T11:25:37.191322+0000","last_clean":"2023-10-16T11:25:37.191322+0000","last_became_active":"2023-10-16T11:25:36.170729+0000","last_became_peered":"2023-10-16T11:25:36.170729+0000","last_unstale":"2023-10-16T11:25:37.191322+0000","last_undegraded":"2023-10-16T11:25:37.191322+0000","last_fullsized":"2023-10-16T11:25:37.191322+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:34.362958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.177036+0000","last_change":"2023-10-16T11:25:37.163991+0000","last_active":"2023-10-16T11:25:38.177036+0000","last_peered":"2023-10-16T11:25:38.177036+0000","last_clean":"2023-10-16T11:25:38.177036+0000","last_became_active":"2023-10-16T11:25:37.163080+0000","last_became_peered":"2023-10-16T11:25:37.163080+0000","last_unstale":"2023-10-16T11:25:38.177036+0000","last_undegraded":"2023-10-16T11:25:38.177036+0000","last_fullsized":"2023-10-16T11:25:38.177036+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.14","version":"236'6","reported_seq":18,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.191864+0000","last_change":"2023-10-16T11:25:36.173259+0000","last_active":"2023-10-16T11:25:37.191864+0000","last_peered":"2023-10-16T11:25:37.191864+0000","last_clean":"2023-10-16T11:25:37.191864+0000","last_became_active":"2023-10-16T11:25:36.172980+0000","last_became_peered":"2023-10-16T11:25:36.172980+0000","last_unstale":"2023-10-16T11:25:37.191864+0000","last_undegraded":"2023-10-16T11:25:37.191864+0000","last_fullsized":"2023-10-16T11:25:37.191864+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:33.782641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.176159+0000","last_change":"2023-10-16T11:25:37.166081+0000","last_active":"2023-10-16T11:25:38.176159+0000","last_peered":"2023-10-16T11:25:38.176159+0000","last_clean":"2023-10-16T11:25:38.176159+0000","last_became_active":"2023-10-16T11:25:37.165150+0000","last_became_peered":"2023-10-16T11:25:37.165150+0000","last_unstale":"2023-10-16T11:25:38.176159+0000","last_undegraded":"2023-10-16T11:25:38.176159+0000","last_fullsized":"2023-10-16T11:25:38.176159+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:56.314696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"236'2","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.190914+0000","last_change":"2023-10-16T11:25:36.171468+0000","last_active":"2023-10-16T11:25:37.190914+0000","last_peered":"2023-10-16T11:25:37.190914+0000","last_clean":"2023-10-16T11:25:37.190914+0000","last_became_active":"2023-10-16T11:25:36.171236+0000","last_became_peered":"2023-10-16T11:25:36.171236+0000","last_unstale":"2023-10-16T11:25:37.190914+0000","last_undegraded":"2023-10-16T11:25:37.190914+0000","last_fullsized":"2023-10-16T11:25:37.190914+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:17.018903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174436+0000","last_change":"2023-10-16T11:25:37.178517+0000","last_active":"2023-10-16T11:25:38.174436+0000","last_peered":"2023-10-16T11:25:38.174436+0000","last_clean":"2023-10-16T11:25:38.174436+0000","last_became_active":"2023-10-16T11:25:37.177794+0000","last_became_peered":"2023-10-16T11:25:37.177794+0000","last_unstale":"2023-10-16T11:25:38.174436+0000","last_undegraded":"2023-10-16T11:25:38.174436+0000","last_fullsized":"2023-10-16T11:25:38.174436+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:52:39.949531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.164090+0000","last_change":"2023-10-16T11:25:36.162595+0000","last_active":"2023-10-16T11:25:37.164090+0000","last_peered":"2023-10-16T11:25:37.164090+0000","last_clean":"2023-10-16T11:25:37.164090+0000","last_became_active":"2023-10-16T11:25:36.162249+0000","last_became_peered":"2023-10-16T11:25:36.162249+0000","last_unstale":"2023-10-16T11:25:37.164090+0000","last_undegraded":"2023-10-16T11:25:37.164090+0000","last_fullsized":"2023-10-16T11:25:37.164090+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:49.547165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.171375+0000","last_change":"2023-10-16T11:25:37.164604+0000","last_active":"2023-10-16T11:25:38.171375+0000","last_peered":"2023-10-16T11:25:38.171375+0000","last_clean":"2023-10-16T11:25:38.171375+0000","last_became_active":"2023-10-16T11:25:37.162365+0000","last_became_peered":"2023-10-16T11:25:37.162365+0000","last_unstale":"2023-10-16T11:25:38.171375+0000","last_undegraded":"2023-10-16T11:25:38.171375+0000","last_fullsized":"2023-10-16T11:25:38.171375+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:35.912014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.150894+0000","last_change":"2023-10-16T11:25:36.170014+0000","last_active":"2023-10-16T11:25:37.150894+0000","last_peered":"2023-10-16T11:25:37.150894+0000","last_clean":"2023-10-16T11:25:37.150894+0000","last_became_active":"2023-10-16T11:25:36.169755+0000","last_became_peered":"2023-10-16T11:25:36.169755+0000","last_unstale":"2023-10-16T11:25:37.150894+0000","last_undegraded":"2023-10-16T11:25:37.150894+0000","last_fullsized":"2023-10-16T11:25:37.150894+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:02.119904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.173748+0000","last_change":"2023-10-16T11:25:37.178789+0000","last_active":"2023-10-16T11:25:38.173748+0000","last_peered":"2023-10-16T11:25:38.173748+0000","last_clean":"2023-10-16T11:25:38.173748+0000","last_became_active":"2023-10-16T11:25:37.178510+0000","last_became_peered":"2023-10-16T11:25:37.178510+0000","last_unstale":"2023-10-16T11:25:38.173748+0000","last_undegraded":"2023-10-16T11:25:38.173748+0000","last_fullsized":"2023-10-16T11:25:38.173748+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.10","version":"236'8","reported_seq":20,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.191147+0000","last_change":"2023-10-16T11:25:36.168889+0000","last_active":"2023-10-16T11:25:37.191147+0000","last_peered":"2023-10-16T11:25:37.191147+0000","last_clean":"2023-10-16T11:25:37.191147+0000","last_became_active":"2023-10-16T11:25:36.168590+0000","last_became_peered":"2023-10-16T11:25:36.168590+0000","last_unstale":"2023-10-16T11:25:37.191147+0000","last_undegraded":"2023-10-16T11:25:37.191147+0000","last_fullsized":"2023-10-16T11:25:37.191147+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:42.801560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.172236+0000","last_change":"2023-10-16T11:25:37.175992+0000","last_active":"2023-10-16T11:25:38.172236+0000","last_peered":"2023-10-16T11:25:38.172236+0000","last_clean":"2023-10-16T11:25:38.172236+0000","last_became_active":"2023-10-16T11:25:37.173732+0000","last_became_peered":"2023-10-16T11:25:37.173732+0000","last_unstale":"2023-10-16T11:25:38.172236+0000","last_undegraded":"2023-10-16T11:25:38.172236+0000","last_fullsized":"2023-10-16T11:25:38.172236+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.f","version":"236'1","reported_seq":13,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.182939+0000","last_change":"2023-10-16T11:25:36.173935+0000","last_active":"2023-10-16T11:25:37.182939+0000","last_peered":"2023-10-16T11:25:37.182939+0000","last_clean":"2023-10-16T11:25:37.182939+0000","last_became_active":"2023-10-16T11:25:36.173510+0000","last_became_peered":"2023-10-16T11:25:36.173510+0000","last_unstale":"2023-10-16T11:25:37.182939+0000","last_undegraded":"2023-10-16T11:25:37.182939+0000","last_fullsized":"2023-10-16T11:25:37.182939+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:19.077096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616968+0000","last_change":"2023-10-16T11:25:37.177543+0000","last_active":"2023-10-16T11:25:38.616968+0000","last_peered":"2023-10-16T11:25:38.616968+0000","last_clean":"2023-10-16T11:25:38.616968+0000","last_became_active":"2023-10-16T11:25:37.176420+0000","last_became_peered":"2023-10-16T11:25:37.176420+0000","last_unstale":"2023-10-16T11:25:38.616968+0000","last_undegraded":"2023-10-16T11:25:38.616968+0000","last_fullsized":"2023-10-16T11:25:38.616968+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:24.104981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.e","version":"236'2","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.189031+0000","last_change":"2023-10-16T11:25:36.170878+0000","last_active":"2023-10-16T11:25:37.189031+0000","last_peered":"2023-10-16T11:25:37.189031+0000","last_clean":"2023-10-16T11:25:37.189031+0000","last_became_active":"2023-10-16T11:25:36.170548+0000","last_became_peered":"2023-10-16T11:25:36.170548+0000","last_unstale":"2023-10-16T11:25:37.189031+0000","last_undegraded":"2023-10-16T11:25:37.189031+0000","last_fullsized":"2023-10-16T11:25:37.189031+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:34:14.932409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174918+0000","last_change":"2023-10-16T11:25:37.176420+0000","last_active":"2023-10-16T11:25:38.174918+0000","last_peered":"2023-10-16T11:25:38.174918+0000","last_clean":"2023-10-16T11:25:38.174918+0000","last_became_active":"2023-10-16T11:25:37.176101+0000","last_became_peered":"2023-10-16T11:25:37.176101+0000","last_unstale":"2023-10-16T11:25:38.174918+0000","last_undegraded":"2023-10-16T11:25:38.174918+0000","last_fullsized":"2023-10-16T11:25:38.174918+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:10.245184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"236'1","reported_seq":13,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.182034+0000","last_change":"2023-10-16T11:25:36.171476+0000","last_active":"2023-10-16T11:25:37.182034+0000","last_peered":"2023-10-16T11:25:37.182034+0000","last_clean":"2023-10-16T11:25:37.182034+0000","last_became_active":"2023-10-16T11:25:36.171201+0000","last_became_peered":"2023-10-16T11:25:36.171201+0000","last_unstale":"2023-10-16T11:25:37.182034+0000","last_undegraded":"2023-10-16T11:25:37.182034+0000","last_fullsized":"2023-10-16T11:25:37.182034+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:30.034841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.197561+0000","last_change":"2023-10-16T11:25:37.162366+0000","last_active":"2023-10-16T11:25:39.197561+0000","last_peered":"2023-10-16T11:25:39.197561+0000","last_clean":"2023-10-16T11:25:39.197561+0000","last_became_active":"2023-10-16T11:25:37.162032+0000","last_became_peered":"2023-10-16T11:25:37.162032+0000","last_unstale":"2023-10-16T11:25:39.197561+0000","last_undegraded":"2023-10-16T11:25:39.197561+0000","last_fullsized":"2023-10-16T11:25:39.197561+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.169247+0000","last_change":"2023-10-16T11:25:36.174246+0000","last_active":"2023-10-16T11:25:37.169247+0000","last_peered":"2023-10-16T11:25:37.169247+0000","last_clean":"2023-10-16T11:25:37.169247+0000","last_became_active":"2023-10-16T11:25:36.173975+0000","last_became_peered":"2023-10-16T11:25:36.173975+0000","last_unstale":"2023-10-16T11:25:37.169247+0000","last_undegraded":"2023-10-16T11:25:37.169247+0000","last_fullsized":"2023-10-16T11:25:37.169247+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:32.130559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.196788+0000","last_change":"2023-10-16T11:25:37.168715+0000","last_active":"2023-10-16T11:25:39.196788+0000","last_peered":"2023-10-16T11:25:39.196788+0000","last_clean":"2023-10-16T11:25:39.196788+0000","last_became_active":"2023-10-16T11:25:37.168387+0000","last_became_peered":"2023-10-16T11:25:37.168387+0000","last_unstale":"2023-10-16T11:25:39.196788+0000","last_undegraded":"2023-10-16T11:25:39.196788+0000","last_fullsized":"2023-10-16T11:25:39.196788+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.b","version":"236'2","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.183693+0000","last_change":"2023-10-16T11:25:36.174153+0000","last_active":"2023-10-16T11:25:37.183693+0000","last_peered":"2023-10-16T11:25:37.183693+0000","last_clean":"2023-10-16T11:25:37.183693+0000","last_became_active":"2023-10-16T11:25:36.173812+0000","last_became_peered":"2023-10-16T11:25:36.173812+0000","last_unstale":"2023-10-16T11:25:37.183693+0000","last_undegraded":"2023-10-16T11:25:37.183693+0000","last_fullsized":"2023-10-16T11:25:37.183693+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:07:05.931833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193818+0000","last_change":"2023-10-16T11:25:37.175767+0000","last_active":"2023-10-16T11:25:39.193818+0000","last_peered":"2023-10-16T11:25:39.193818+0000","last_clean":"2023-10-16T11:25:39.193818+0000","last_became_active":"2023-10-16T11:25:37.167389+0000","last_became_peered":"2023-10-16T11:25:37.167389+0000","last_unstale":"2023-10-16T11:25:39.193818+0000","last_undegraded":"2023-10-16T11:25:39.193818+0000","last_fullsized":"2023-10-16T11:25:39.193818+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:18.601118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.169039+0000","last_change":"2023-10-16T11:25:36.175478+0000","last_active":"2023-10-16T11:25:37.169039+0000","last_peered":"2023-10-16T11:25:37.169039+0000","last_clean":"2023-10-16T11:25:37.169039+0000","last_became_active":"2023-10-16T11:25:36.174935+0000","last_became_peered":"2023-10-16T11:25:36.174935+0000","last_unstale":"2023-10-16T11:25:37.169039+0000","last_undegraded":"2023-10-16T11:25:37.169039+0000","last_fullsized":"2023-10-16T11:25:37.169039+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:14.189489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182532+0000","last_change":"2023-10-16T11:25:37.173034+0000","last_active":"2023-10-16T11:25:38.182532+0000","last_peered":"2023-10-16T11:25:38.182532+0000","last_clean":"2023-10-16T11:25:38.182532+0000","last_became_active":"2023-10-16T11:25:37.172644+0000","last_became_peered":"2023-10-16T11:25:37.172644+0000","last_unstale":"2023-10-16T11:25:38.182532+0000","last_undegraded":"2023-10-16T11:25:38.182532+0000","last_fullsized":"2023-10-16T11:25:38.182532+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.162208+0000","last_change":"2023-10-16T11:25:36.163926+0000","last_active":"2023-10-16T11:25:37.162208+0000","last_peered":"2023-10-16T11:25:37.162208+0000","last_clean":"2023-10-16T11:25:37.162208+0000","last_became_active":"2023-10-16T11:25:36.162991+0000","last_became_peered":"2023-10-16T11:25:36.162991+0000","last_unstale":"2023-10-16T11:25:37.162208+0000","last_undegraded":"2023-10-16T11:25:37.162208+0000","last_fullsized":"2023-10-16T11:25:37.162208+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:49.547165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.196630+0000","last_change":"2023-10-16T11:25:37.158678+0000","last_active":"2023-10-16T11:25:39.196630+0000","last_peered":"2023-10-16T11:25:39.196630+0000","last_clean":"2023-10-16T11:25:39.196630+0000","last_became_active":"2023-10-16T11:25:37.157389+0000","last_became_peered":"2023-10-16T11:25:37.157389+0000","last_unstale":"2023-10-16T11:25:39.196630+0000","last_undegraded":"2023-10-16T11:25:39.196630+0000","last_fullsized":"2023-10-16T11:25:39.196630+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.153051+0000","last_change":"2023-10-16T11:25:36.171876+0000","last_active":"2023-10-16T11:25:37.153051+0000","last_peered":"2023-10-16T11:25:37.153051+0000","last_clean":"2023-10-16T11:25:37.153051+0000","last_became_active":"2023-10-16T11:25:36.169128+0000","last_became_peered":"2023-10-16T11:25:36.169128+0000","last_unstale":"2023-10-16T11:25:37.153051+0000","last_undegraded":"2023-10-16T11:25:37.153051+0000","last_fullsized":"2023-10-16T11:25:37.153051+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:16:18.153133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.176783+0000","last_change":"2023-10-16T11:25:37.179119+0000","last_active":"2023-10-16T11:25:38.176783+0000","last_peered":"2023-10-16T11:25:38.176783+0000","last_clean":"2023-10-16T11:25:38.176783+0000","last_became_active":"2023-10-16T11:25:37.175799+0000","last_became_peered":"2023-10-16T11:25:37.175799+0000","last_unstale":"2023-10-16T11:25:38.176783+0000","last_undegraded":"2023-10-16T11:25:38.176783+0000","last_fullsized":"2023-10-16T11:25:38.176783+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"236'2","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.185451+0000","last_change":"2023-10-16T11:25:36.162608+0000","last_active":"2023-10-16T11:25:37.185451+0000","last_peered":"2023-10-16T11:25:37.185451+0000","last_clean":"2023-10-16T11:25:37.185451+0000","last_became_active":"2023-10-16T11:25:36.162336+0000","last_became_peered":"2023-10-16T11:25:36.162336+0000","last_unstale":"2023-10-16T11:25:37.185451+0000","last_undegraded":"2023-10-16T11:25:37.185451+0000","last_fullsized":"2023-10-16T11:25:37.185451+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:19.377028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.194112+0000","last_change":"2023-10-16T11:25:37.177560+0000","last_active":"2023-10-16T11:25:39.194112+0000","last_peered":"2023-10-16T11:25:39.194112+0000","last_clean":"2023-10-16T11:25:39.194112+0000","last_became_active":"2023-10-16T11:25:37.176982+0000","last_became_peered":"2023-10-16T11:25:37.176982+0000","last_unstale":"2023-10-16T11:25:39.194112+0000","last_undegraded":"2023-10-16T11:25:39.194112+0000","last_fullsized":"2023-10-16T11:25:39.194112+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:54.497943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.167365+0000","last_change":"2023-10-16T11:25:36.174419+0000","last_active":"2023-10-16T11:25:37.167365+0000","last_peered":"2023-10-16T11:25:37.167365+0000","last_clean":"2023-10-16T11:25:37.167365+0000","last_became_active":"2023-10-16T11:25:36.174148+0000","last_became_peered":"2023-10-16T11:25:36.174148+0000","last_unstale":"2023-10-16T11:25:37.167365+0000","last_undegraded":"2023-10-16T11:25:37.167365+0000","last_fullsized":"2023-10-16T11:25:37.167365+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:38.739323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182976+0000","last_change":"2023-10-16T11:25:37.180683+0000","last_active":"2023-10-16T11:25:38.182976+0000","last_peered":"2023-10-16T11:25:38.182976+0000","last_clean":"2023-10-16T11:25:38.182976+0000","last_became_active":"2023-10-16T11:25:37.180393+0000","last_became_peered":"2023-10-16T11:25:37.180393+0000","last_unstale":"2023-10-16T11:25:38.182976+0000","last_undegraded":"2023-10-16T11:25:38.182976+0000","last_fullsized":"2023-10-16T11:25:38.182976+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:38.218704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.158199+0000","last_change":"2023-10-16T11:25:36.164519+0000","last_active":"2023-10-16T11:25:37.158199+0000","last_peered":"2023-10-16T11:25:37.158199+0000","last_clean":"2023-10-16T11:25:37.158199+0000","last_became_active":"2023-10-16T11:25:36.164214+0000","last_became_peered":"2023-10-16T11:25:36.164214+0000","last_unstale":"2023-10-16T11:25:37.158199+0000","last_undegraded":"2023-10-16T11:25:37.158199+0000","last_fullsized":"2023-10-16T11:25:37.158199+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:03.074535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.183331+0000","last_change":"2023-10-16T11:25:37.181808+0000","last_active":"2023-10-16T11:25:38.183331+0000","last_peered":"2023-10-16T11:25:38.183331+0000","last_clean":"2023-10-16T11:25:38.183331+0000","last_became_active":"2023-10-16T11:25:37.181575+0000","last_became_peered":"2023-10-16T11:25:37.181575+0000","last_unstale":"2023-10-16T11:25:38.183331+0000","last_undegraded":"2023-10-16T11:25:38.183331+0000","last_fullsized":"2023-10-16T11:25:38.183331+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:17:01.131611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.169371+0000","last_change":"2023-10-16T11:25:36.162153+0000","last_active":"2023-10-16T11:25:37.169371+0000","last_peered":"2023-10-16T11:25:37.169371+0000","last_clean":"2023-10-16T11:25:37.169371+0000","last_became_active":"2023-10-16T11:25:36.161842+0000","last_became_peered":"2023-10-16T11:25:36.161842+0000","last_unstale":"2023-10-16T11:25:37.169371+0000","last_undegraded":"2023-10-16T11:25:37.169371+0000","last_fullsized":"2023-10-16T11:25:37.169371+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:55.302377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.176225+0000","last_change":"2023-10-16T11:25:37.165420+0000","last_active":"2023-10-16T11:25:38.176225+0000","last_peered":"2023-10-16T11:25:38.176225+0000","last_clean":"2023-10-16T11:25:38.176225+0000","last_became_active":"2023-10-16T11:25:37.164820+0000","last_became_peered":"2023-10-16T11:25:37.164820+0000","last_unstale":"2023-10-16T11:25:38.176225+0000","last_undegraded":"2023-10-16T11:25:38.176225+0000","last_fullsized":"2023-10-16T11:25:38.176225+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:43.813878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.154862+0000","last_change":"2023-10-16T11:25:36.173209+0000","last_active":"2023-10-16T11:25:37.154862+0000","last_peered":"2023-10-16T11:25:37.154862+0000","last_clean":"2023-10-16T11:25:37.154862+0000","last_became_active":"2023-10-16T11:25:36.171799+0000","last_became_peered":"2023-10-16T11:25:36.171799+0000","last_unstale":"2023-10-16T11:25:37.154862+0000","last_undegraded":"2023-10-16T11:25:37.154862+0000","last_fullsized":"2023-10-16T11:25:37.154862+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:19.377028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.198049+0000","last_change":"2023-10-16T11:25:37.180243+0000","last_active":"2023-10-16T11:25:39.198049+0000","last_peered":"2023-10-16T11:25:39.198049+0000","last_clean":"2023-10-16T11:25:39.198049+0000","last_became_active":"2023-10-16T11:25:37.179869+0000","last_became_peered":"2023-10-16T11:25:37.179869+0000","last_unstale":"2023-10-16T11:25:39.198049+0000","last_undegraded":"2023-10-16T11:25:39.198049+0000","last_fullsized":"2023-10-16T11:25:39.198049+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:15:31.572558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.146030+0000","last_change":"2023-10-16T11:25:36.169146+0000","last_active":"2023-10-16T11:25:37.146030+0000","last_peered":"2023-10-16T11:25:37.146030+0000","last_clean":"2023-10-16T11:25:37.146030+0000","last_became_active":"2023-10-16T11:25:36.168877+0000","last_became_peered":"2023-10-16T11:25:36.168877+0000","last_unstale":"2023-10-16T11:25:37.146030+0000","last_undegraded":"2023-10-16T11:25:37.146030+0000","last_fullsized":"2023-10-16T11:25:37.146030+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:35.988148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.196353+0000","last_change":"2023-10-16T11:25:37.173516+0000","last_active":"2023-10-16T11:25:39.196353+0000","last_peered":"2023-10-16T11:25:39.196353+0000","last_clean":"2023-10-16T11:25:39.196353+0000","last_became_active":"2023-10-16T11:25:37.172964+0000","last_became_peered":"2023-10-16T11:25:37.172964+0000","last_unstale":"2023-10-16T11:25:39.196353+0000","last_undegraded":"2023-10-16T11:25:39.196353+0000","last_fullsized":"2023-10-16T11:25:39.196353+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.164633+0000","last_change":"2023-10-16T11:25:36.166442+0000","last_active":"2023-10-16T11:25:37.164633+0000","last_peered":"2023-10-16T11:25:37.164633+0000","last_clean":"2023-10-16T11:25:37.164633+0000","last_became_active":"2023-10-16T11:25:36.166131+0000","last_became_peered":"2023-10-16T11:25:36.166131+0000","last_unstale":"2023-10-16T11:25:37.164633+0000","last_undegraded":"2023-10-16T11:25:37.164633+0000","last_fullsized":"2023-10-16T11:25:37.164633+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:43.814368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193591+0000","last_change":"2023-10-16T11:25:37.183491+0000","last_active":"2023-10-16T11:25:39.193591+0000","last_peered":"2023-10-16T11:25:39.193591+0000","last_clean":"2023-10-16T11:25:39.193591+0000","last_became_active":"2023-10-16T11:25:37.179560+0000","last_became_peered":"2023-10-16T11:25:37.179560+0000","last_unstale":"2023-10-16T11:25:39.193591+0000","last_undegraded":"2023-10-16T11:25:39.193591+0000","last_fullsized":"2023-10-16T11:25:39.193591+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:31.820109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.164860+0000","last_change":"2023-10-16T11:25:36.175845+0000","last_active":"2023-10-16T11:25:37.164860+0000","last_peered":"2023-10-16T11:25:37.164860+0000","last_clean":"2023-10-16T11:25:37.164860+0000","last_became_active":"2023-10-16T11:25:36.175390+0000","last_became_peered":"2023-10-16T11:25:36.175390+0000","last_unstale":"2023-10-16T11:25:37.164860+0000","last_undegraded":"2023-10-16T11:25:37.164860+0000","last_fullsized":"2023-10-16T11:25:37.164860+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:09.232865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.198662+0000","last_change":"2023-10-16T11:25:37.178486+0000","last_active":"2023-10-16T11:25:39.198662+0000","last_peered":"2023-10-16T11:25:39.198662+0000","last_clean":"2023-10-16T11:25:39.198662+0000","last_became_active":"2023-10-16T11:25:37.177077+0000","last_became_peered":"2023-10-16T11:25:37.177077+0000","last_unstale":"2023-10-16T11:25:39.198662+0000","last_undegraded":"2023-10-16T11:25:39.198662+0000","last_fullsized":"2023-10-16T11:25:39.198662+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"236'4","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.182413+0000","last_change":"2023-10-16T11:25:36.174205+0000","last_active":"2023-10-16T11:25:37.182413+0000","last_peered":"2023-10-16T11:25:37.182413+0000","last_clean":"2023-10-16T11:25:37.182413+0000","last_became_active":"2023-10-16T11:25:36.173850+0000","last_became_peered":"2023-10-16T11:25:36.173850+0000","last_unstale":"2023-10-16T11:25:37.182413+0000","last_undegraded":"2023-10-16T11:25:37.182413+0000","last_fullsized":"2023-10-16T11:25:37.182413+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:20.618602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"223'2789","reported_seq":3745,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:25:37.158495+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:25:37.158495+0000","last_peered":"2023-10-16T11:25:37.158495+0000","last_clean":"2023-10-16T11:25:37.158495+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:25:37.158495+0000","last_undegraded":"2023-10-16T11:25:37.158495+0000","last_fullsized":"2023-10-16T11:25:37.158495+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2789,"log_dups_size":0,"ondisk_log_size":2789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":31,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":30,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2789,"ondisk_log_size":2789,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575999,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52732,"kb_used_data":3500,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319108,"statfs":{"total":96636764160,"available":96582766592,"internally_reserved":0,"allocated":3584000,"data_stored":2045861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.153,"15min":1.2090000000000001},"min":{"1min":0.71499999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4950000000000001,"5min":1.7549999999999999,"15min":2.3399999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.161,"15min":1.2150000000000001},"min":{"1min":0.623,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.448,"5min":1.6699999999999999,"15min":1.8320000000000001},"last":0.67700000000000005}]},{"osd":1,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.256,"15min":1.2430000000000001},"min":{"1min":0.84599999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.5029999999999999,"5min":1.913,"15min":1.913},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.2210000000000001,"15min":1.264},"min":{"1min":0.67100000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.522,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.90300000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.3,"15min":1.296},"min":{"1min":0.94199999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6879999999999999,"5min":1.8680000000000001,"15min":2.4460000000000002},"last":1.234},{"interface":"front","average":{"1min":1.21,"5min":1.3129999999999999,"15min":1.3500000000000001},"min":{"1min":0.94499999999999995,"5min":0.88,"15min":0.88},"max":{"1min":1.5589999999999999,"5min":1.859,"15min":1.974},"last":1.165}]},{"osd":3,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3520000000000001,"15min":1.3779999999999999},"min":{"1min":0.93700000000000006,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.653,"5min":1.8080000000000001,"15min":1.9099999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.4370000000000001,"15min":1.454},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.728,"5min":2.0070000000000001,"15min":2.577},"last":1.0860000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4319999999999999,"15min":1.413},"min":{"1min":0.89800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.865,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":1.149,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8660000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.488}]},{"osd":5,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.587,"15min":1.591},"min":{"1min":1.1799999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.95,"5min":2.101,"15min":3.3519999999999999},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.53,"15min":1.508},"min":{"1min":1.2330000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.9870000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.6279999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.492,"15min":1.5089999999999999},"min":{"1min":1.1459999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.8140000000000001,"5min":2.101,"15min":2.7250000000000001},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.6040000000000001,"5min":1.6819999999999999,"15min":1.698},"min":{"1min":1.3080000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.0129999999999999,"5min":2.2109999999999999,"15min":3.1440000000000001},"last":1.6890000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575999,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52152,"kb_used_data":2924,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319688,"statfs":{"total":96636764160,"available":96583360512,"internally_reserved":0,"allocated":2994176,"data_stored":1456027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.2130000000000001,"15min":1.21},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.343,"5min":6.298,"15min":6.298},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.331,"15min":1.292},"min":{"1min":0.72199999999999998,"5min":0.68200000000000005,"15min":0.66400000000000003},"max":{"1min":1.607,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":1.105}]},{"osd":1,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.224,"15min":1.212},"min":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.5660000000000001,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":1.236},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.421,"15min":1.3779999999999999},"min":{"1min":0.76200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.7310000000000001,"5min":12.468,"15min":12.468},"last":1.323}]},{"osd":2,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.3140000000000001,"15min":1.2769999999999999},"min":{"1min":0.77800000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.373,"5min":11.586,"15min":11.586},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.3740000000000001,"15min":1.3680000000000001},"min":{"1min":0.84399999999999997,"5min":0.82599999999999996,"15min":0.77500000000000002},"max":{"1min":1.7629999999999999,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.7629999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.425,"15min":1.3460000000000001},"min":{"1min":0.84799999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.859,"5min":12.801,"15min":12.801},"last":1.54},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2929999999999999,"15min":1.2769999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.657,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":1.371}]},{"osd":4,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3899999999999999,"15min":1.3280000000000001},"min":{"1min":0.68700000000000006,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.9510000000000001,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.9510000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.6220000000000001,"15min":1.5509999999999999},"min":{"1min":1.1180000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.069,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.831}]},{"osd":5,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.5189999999999999,"15min":1.397},"min":{"1min":0.92500000000000004,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.978,"5min":18.247,"15min":18.247},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.595,"15min":1.5620000000000001},"min":{"1min":0.93799999999999994,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.1240000000000001,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":2.1240000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.627,"15min":1.5469999999999999},"min":{"1min":1.2070000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.383,"5min":12.589,"15min":12.589},"last":2.2410000000000001},{"interface":"front","average":{"1min":1.661,"5min":1.702,"15min":1.603},"min":{"1min":1.0049999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.2970000000000002,"5min":12.725,"15min":12.725},"last":2.2970000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575999,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29936,"kb_used_data":2916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341904,"statfs":{"total":96636764160,"available":96606109696,"internally_reserved":0,"allocated":2985984,"data_stored":1455423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2150000000000001,"15min":1.2629999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.51,"5min":2.661,"15min":2.661},"last":1.296},{"interface":"front","average":{"1min":1.204,"5min":1.2769999999999999,"15min":1.339},"min":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7210000000000001,"5min":1.7949999999999999,"15min":1.865},"last":1.335}]},{"osd":1,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.145,"15min":1.1890000000000001},"min":{"1min":0.872,"5min":0.81299999999999994,"15min":0.79100000000000004},"max":{"1min":1.583,"5min":1.704,"15min":1.992},"last":1.649},{"interface":"front","average":{"1min":1.079,"5min":1.22,"15min":1.222},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6739999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.367}]},{"osd":2,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.26,"15min":1.236},"min":{"1min":0.73499999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.7450000000000001,"5min":1.952,"15min":1.952},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.284,"15min":1.3129999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5840000000000001,"5min":1.8109999999999999,"15min":2.1640000000000001},"last":1.052}]},{"osd":3,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.319,"15min":1.329},"min":{"1min":0.872,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5920000000000001,"5min":3.012,"15min":3.012},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.401,"15min":1.4139999999999999},"min":{"1min":0.95399999999999996,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.6639999999999999,"5min":1.9119999999999999,"15min":2.2250000000000001},"last":1.232}]},{"osd":4,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3600000000000001,"15min":1.341},"min":{"1min":0.91300000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.887,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.413,"5min":1.5529999999999999,"15min":1.5660000000000001},"min":{"1min":0.996,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.6499999999999999,"5min":2.903,"15min":2.903},"last":1.6910000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.4119999999999999,"15min":1.411},"min":{"1min":0.93799999999999994,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.837,"5min":1.8660000000000001,"15min":2.0979999999999999},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.486,"15min":1.4810000000000001},"min":{"1min":0.93300000000000005,"5min":0.90700000000000003,"15min":0.89200000000000002},"max":{"1min":2.0329999999999999,"5min":3.073,"15min":3.073},"last":1.125}]},{"osd":7,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.5649999999999999,"15min":1.5429999999999999},"min":{"1min":1.159,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.764,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.625,"15min":1.6399999999999999},"min":{"1min":1.349,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.9970000000000001,"5min":2.1099999999999999,"15min":2.2930000000000001},"last":1.54}]}]},{"osd":4,"up_from":13,"seq":55834576000,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52164,"kb_used_data":2936,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319676,"statfs":{"total":96636764160,"available":96583348224,"internally_reserved":0,"allocated":3006464,"data_stored":1457127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.167,"15min":1.1799999999999999},"min":{"1min":0.69099999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":1.8049999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.2370000000000001,"15min":1.2430000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":1.0760000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3069999999999999,"15min":1.3009999999999999},"min":{"1min":0.88800000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.24},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.252,"15min":1.252},"min":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":3.0739999999999998,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.593}]},{"osd":2,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3,"15min":1.3109999999999999},"min":{"1min":0.80500000000000005,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":3.2170000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.4359999999999999,"15min":1.4119999999999999},"min":{"1min":0.90300000000000002,"5min":0.879,"15min":0.872},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.6819999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.339,"15min":1.341},"min":{"1min":0.84599999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.431},{"interface":"front","average":{"1min":1.496,"5min":1.4550000000000001,"15min":1.4430000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":3.4220000000000002,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.6359999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.5249999999999999,"15min":1.4570000000000001},"min":{"1min":1.1339999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.8460000000000001},{"interface":"front","average":{"1min":1.613,"5min":1.46,"15min":1.4319999999999999},"min":{"1min":1.1519999999999999,"5min":0.85199999999999998,"15min":0.78600000000000003},"max":{"1min":3.7989999999999999,"5min":3.7989999999999999,"15min":3.7989999999999999},"last":1.2949999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6679999999999999,"5min":1.5649999999999999,"15min":1.5189999999999999},"min":{"1min":0.98599999999999999,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.776,"5min":1.643,"15min":1.6240000000000001},"min":{"1min":1.0269999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":3.5019999999999998,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.486}]},{"osd":7,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.8089999999999999,"5min":1.613,"15min":1.5860000000000001},"min":{"1min":1.3,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":4.2050000000000001,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":1.7450000000000001},{"interface":"front","average":{"1min":1.8460000000000001,"5min":1.702,"15min":1.6599999999999999},"min":{"1min":1.2370000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":3.7000000000000002,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.893}]}]},{"osd":3,"up_from":13,"seq":55834575999,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52148,"kb_used_data":2920,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319692,"statfs":{"total":96636764160,"available":96583364608,"internally_reserved":0,"allocated":2990080,"data_stored":1456005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.85899999999999999,"15min":0.89700000000000002},"min":{"1min":0.40000000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1339999999999999,"5min":1.478,"15min":1.901},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.997,"15min":1.0569999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1850000000000001,"5min":1.6180000000000001,"15min":1.6679999999999999},"last":1.581}]},{"osd":1,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.95299999999999996,"15min":0.98399999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.3400000000000001,"5min":1.508,"15min":1.508},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0609999999999999,"15min":1.121},"min":{"1min":0.64900000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5429999999999999,"5min":1.601,"15min":1.7190000000000001},"last":0.97899999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0580000000000001,"15min":1.1120000000000001},"min":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4199999999999999,"5min":1.694,"15min":1.694},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.161,"15min":1.22},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.7350000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.329}]},{"osd":4,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2989999999999999,"15min":1.3480000000000001},"min":{"1min":0.89400000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.724,"5min":1.806,"15min":1.873},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3300000000000001,"15min":1.359},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.5549999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.389,"15min":1.405},"min":{"1min":1.0549999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8260000000000001,"5min":2.798,"15min":2.798},"last":1.252},{"interface":"front","average":{"1min":1.347,"5min":1.395,"15min":1.4410000000000001},"min":{"1min":0.88800000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.883,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.282}]},{"osd":6,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.488,"15min":1.5329999999999999},"min":{"1min":1.01,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.911,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.7929999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4790000000000001,"15min":1.534},"min":{"1min":0.86699999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.0329999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.6830000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.518,"15min":1.5580000000000001},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.629},{"interface":"front","average":{"1min":1.575,"5min":1.613,"15min":1.669},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.3799999999999999,"5min":2.3799999999999999,"15min":2.5129999999999999},"last":1.4830000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575998,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56312,"kb_used_data":2920,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315528,"statfs":{"total":96636764160,"available":96579100672,"internally_reserved":0,"allocated":2990080,"data_stored":1455513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.89200000000000002,"15min":0.91600000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.349,"5min":1.532,"15min":1.669},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.99299999999999999,"15min":1.069},"min":{"1min":0.56899999999999995,"5min":0.47699999999999998,"15min":0.32600000000000001},"max":{"1min":1.3460000000000001,"5min":1.6359999999999999,"15min":2.1429999999999998},"last":1.5449999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0760000000000001,"15min":1.089},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.39200000000000002},"max":{"1min":1.591,"5min":1.645,"15min":1.978},"last":1.022},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.085,"15min":1.1379999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.393,"5min":1.609,"15min":1.7949999999999999},"last":1.2809999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0429999999999999,"15min":1.0740000000000001},"min":{"1min":0.499,"5min":0.437,"15min":0.437},"max":{"1min":1.522,"5min":1.651,"15min":1.8560000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.137,"5min":1.2170000000000001,"15min":1.2629999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.6930000000000001,"5min":1.7450000000000001,"15min":1.857},"last":1.2250000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3340000000000001,"15min":1.3999999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.73499999999999999},"max":{"1min":1.643,"5min":1.8029999999999999,"15min":2.5059999999999998},"last":1.083},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.3220000000000001,"15min":1.339},"min":{"1min":0.88200000000000001,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":1.6499999999999999,"5min":1.825,"15min":1.843},"last":1.159}]},{"osd":5,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.4139999999999999,"15min":1.4530000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.429,"15min":1.4730000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.885,"5min":1.931,"15min":2.6259999999999999},"last":1.6020000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3899999999999999,"15min":1.3999999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.8260000000000001,"5min":1.9019999999999999,"15min":2.1339999999999999},"last":1.379},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4870000000000001,"15min":1.526},"min":{"1min":0.80500000000000005,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.3380000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.5109999999999999,"15min":1.569},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.92500000000000004},"max":{"1min":2.0310000000000001,"5min":2.1469999999999998,"15min":2.8109999999999999},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4750000000000001,"15min":1.502},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80100000000000005},"max":{"1min":1.952,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.575}]}]},{"osd":1,"up_from":13,"seq":55834575998,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64644,"kb_used_data":2936,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307196,"statfs":{"total":96636764160,"available":96570568704,"internally_reserved":0,"allocated":3006464,"data_stored":1457159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.93799999999999994,"15min":0.91700000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":5.117,"5min":5.117,"15min":5.117},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.97299999999999998,"15min":1.0449999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.5069999999999999,"5min":1.5509999999999999,"15min":1.667},"last":0.66000000000000003}]},{"osd":2,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.97299999999999998,"15min":1.014},"min":{"1min":0.52600000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2350000000000001,"15min":1.27},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":3.5840000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":1.234}]},{"osd":3,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.1779999999999999,"15min":1.202},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.4670000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":0.747},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.1439999999999999,"15min":1.1639999999999999},"min":{"1min":0.48099999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5600000000000001,"5min":1.633,"15min":1.802},"last":1.169}]},{"osd":4,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.3260000000000001,"15min":1.3009999999999999},"min":{"1min":0.80500000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":5.29,"5min":5.29,"15min":5.29},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.4450000000000001,"15min":1.4259999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":5.0179999999999998,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":1.379}]},{"osd":5,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.397,"15min":1.419},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":6.0979999999999999,"5min":6.0979999999999999,"15min":6.0979999999999999},"last":1.47},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.415,"15min":1.4159999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.5299999999999998,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.1020000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.49,"15min":1.5029999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":4.0499999999999998,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.474,"15min":1.488},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":3.8799999999999999,"5min":3.8799999999999999,"15min":3.8799999999999999},"last":1.216}]},{"osd":7,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.577,"15min":1.569},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":4.46,"5min":4.46,"15min":4.46},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.625,"15min":1.613},"min":{"1min":1.0049999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":4.1040000000000001,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":1.6240000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575999,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52728,"kb_used_data":3500,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319112,"statfs":{"total":96636764160,"available":96582770688,"internally_reserved":0,"allocated":3584000,"data_stored":2045825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.86599999999999999,"15min":0.89500000000000002},"min":{"1min":0.42699999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.3420000000000001,"5min":1.52,"15min":1.619},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0649999999999999,"15min":1.1020000000000001},"min":{"1min":0.54100000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3680000000000001,"5min":1.6399999999999999,"15min":1.76},"last":0.99199999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0680000000000001,"15min":1.085},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0840000000000001,"15min":1.143},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.5169999999999999,"5min":1.6220000000000001,"15min":1.6870000000000001},"last":0.873}]},{"osd":3,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.071,"15min":1.0860000000000001},"min":{"1min":0.51400000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7609999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.206,"15min":1.2490000000000001},"min":{"1min":0.63800000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.9830000000000001},"last":1.0609999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3640000000000001,"15min":1.389},"min":{"1min":0.879,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.843,"5min":1.9390000000000001,"15min":1.95},"last":1.371},{"interface":"front","average":{"1min":1.339,"5min":1.321,"15min":1.343},"min":{"1min":1.0349999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.665,"5min":2.1539999999999999,"15min":2.4820000000000002},"last":1.2789999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.383,"15min":1.3839999999999999},"min":{"1min":1.032,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":1.7689999999999999,"5min":2.0750000000000002,"15min":2.6669999999999998},"last":2.0209999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.401,"15min":1.446},"min":{"1min":0.83399999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.9450000000000001,"5min":2.2029999999999998,"15min":2.2040000000000002},"last":1.885}]},{"osd":6,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.454,"15min":1.498},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":2.004,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.476,"15min":1.4950000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.073,"5min":2.073,"15min":2.262},"last":1.804}]},{"osd":7,"last update":"Mon Oct 16 11:24:49 2023","interfaces":[{"interface":"back","average":{"1min":1.605,"5min":1.5169999999999999,"15min":1.556},"min":{"1min":0.92500000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.347},"last":1.758},{"interface":"front","average":{"1min":1.536,"5min":1.4850000000000001,"15min":1.51},"min":{"1min":0.84399999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.9550000000000001,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.4670000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:25:42.113 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T11:25:42.113 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T11:25:42.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.377+0000 7f96dfecf640 1 Processor -- start 2023-10-16T11:25:42.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.377+0000 7f96dfecf640 1 -- start start 2023-10-16T11:25:42.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.379+0000 7f96dfecf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f96d8070fd0 0x7f96d80cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:42.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.379+0000 7f96dfecf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d80cc600 con 0x7f96d80adbe0 2023-10-16T11:25:42.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.379+0000 7f96dfecf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d8071470 con 0x7f96d80adfc0 2023-10-16T11:25:42.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.379+0000 7f96dfecf640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d80cc870 con 0x7f96d8070fd0 2023-10-16T11:25:42.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.379+0000 7f96ddc6b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f96d80adbe0 0x7f96d806f550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42210/0 (socket says 172.21.15.84:42210) 2023-10-16T11:25:42.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.379+0000 7f96ddc6b640 1 -- 172.21.15.84:0/2905937931 learned_addr learned my addr 172.21.15.84:0/2905937931 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:42.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.379+0000 7f96dd46a640 1 --2- 172.21.15.84:0/2905937931 >> v2:172.21.15.84:3300/0 conn(0x7f96d8070fd0 0x7f96d80cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:42.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.379+0000 7f96dcc69640 1 -- 172.21.15.84:0/2905937931 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2502827911 0 0) 0x7f96d80cc600 con 0x7f96d80adbe0 2023-10-16T11:25:42.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.379+0000 7f96dcc69640 1 -- 172.21.15.84:0/2905937931 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96c4003520 con 0x7f96d80adbe0 2023-10-16T11:25:42.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.380+0000 7f96dd46a640 1 -- 172.21.15.84:0/2905937931 >> v1:172.21.15.157:6789/0 conn(0x7f96d80adfc0 legacy=0x7f96d80b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:42.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.380+0000 7f96dd46a640 1 -- 172.21.15.84:0/2905937931 >> v1:172.21.15.84:6789/0 conn(0x7f96d80adbe0 legacy=0x7f96d806f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:42.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.380+0000 7f96dd46a640 1 -- 172.21.15.84:0/2905937931 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d80ccb50 con 0x7f96d8070fd0 2023-10-16T11:25:42.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.381+0000 7f96dd46a640 1 --2- 172.21.15.84:0/2905937931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96d8070fd0 0x7f96d80cd070 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7f96c80097a0 tx=0x7f96c802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b68f922a3fe945 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:42.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.381+0000 7f96dcc69640 1 -- 172.21.15.84:0/2905937931 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8031830 con 0x7f96d8070fd0 2023-10-16T11:25:42.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.381+0000 7f96dcc69640 1 -- 172.21.15.84:0/2905937931 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c8031990 con 0x7f96d8070fd0 2023-10-16T11:25:42.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.381+0000 7f96dcc69640 1 -- 172.21.15.84:0/2905937931 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8031d20 con 0x7f96d8070fd0 2023-10-16T11:25:42.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.381+0000 7f96dfecf640 1 -- 172.21.15.84:0/2905937931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96d8070fd0 msgr2=0x7f96d80cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:42.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.381+0000 7f96dfecf640 1 --2- 172.21.15.84:0/2905937931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96d8070fd0 0x7f96d80cd070 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7f96c80097a0 tx=0x7f96c802b8d0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.382+0000 7f96dfecf640 1 -- 172.21.15.84:0/2905937931 shutdown_connections 2023-10-16T11:25:42.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.382+0000 7f96dfecf640 1 --2- 172.21.15.84:0/2905937931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96d8070fd0 0x7f96d80cd070 unknown :-1 s=CLOSED pgs=2936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.382+0000 7f96dfecf640 1 -- 172.21.15.84:0/2905937931 >> 172.21.15.84:0/2905937931 conn(0x7f96d80cb410 msgr2=0x7f96d80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:42.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.382+0000 7f96dfecf640 1 -- 172.21.15.84:0/2905937931 shutdown_connections 2023-10-16T11:25:42.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.382+0000 7f96dfecf640 1 -- 172.21.15.84:0/2905937931 wait complete. 2023-10-16T11:25:42.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.383+0000 7f96dfecf640 1 Processor -- start 2023-10-16T11:25:42.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.384+0000 7f96dfecf640 1 -- start start 2023-10-16T11:25:42.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.384+0000 7f96dfecf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96d8070fd0 0x7f96d8153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:42.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.384+0000 7f96dfecf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96d80adbe0 0x7f96d80bd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:42.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.384+0000 7f96dfecf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96d80adfc0 0x7f96d80b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:42.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.384+0000 7f96dfecf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d80c1010 con 0x7f96d8070fd0 2023-10-16T11:25:42.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.384+0000 7f96dfecf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d80c0400 con 0x7f96d80adfc0 2023-10-16T11:25:42.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.384+0000 7f96dfecf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d80c06e0 con 0x7f96d80adbe0 2023-10-16T11:25:42.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.385+0000 7f96ddc6b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96d8070fd0 0x7f96d8153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:42.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.385+0000 7f96ddc6b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96d8070fd0 0x7f96d8153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47344/0 (socket says 172.21.15.84:47344) 2023-10-16T11:25:42.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.385+0000 7f96ddc6b640 1 -- 172.21.15.84:0/2262478666 learned_addr learned my addr 172.21.15.84:0/2262478666 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:42.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.385+0000 7f96dd46a640 1 --2- 172.21.15.84:0/2262478666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96d80adbe0 0x7f96d80bd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:42.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.385+0000 7f96de46c640 1 --2- 172.21.15.84:0/2262478666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96d80adfc0 0x7f96d80b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:42.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.385+0000 7f96ddc6b640 1 -- 172.21.15.84:0/2262478666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96d80adbe0 msgr2=0x7f96d80bd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:42.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.385+0000 7f96ddc6b640 1 --2- 172.21.15.84:0/2262478666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96d80adbe0 0x7f96d80bd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.385+0000 7f96ddc6b640 1 -- 172.21.15.84:0/2262478666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96d80adfc0 msgr2=0x7f96d80b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:42.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.385+0000 7f96ddc6b640 1 --2- 172.21.15.84:0/2262478666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96d80adfc0 0x7f96d80b3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.385+0000 7f96ddc6b640 1 -- 172.21.15.84:0/2262478666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c8009400 con 0x7f96d8070fd0 2023-10-16T11:25:42.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.385+0000 7f96ddc6b640 1 --2- 172.21.15.84:0/2262478666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96d8070fd0 0x7f96d8153e10 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7f96d400c500 tx=0x7f96d400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:42.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.386+0000 7f96ce7fc640 1 -- 172.21.15.84:0/2262478666 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4005ce0 con 0x7f96d8070fd0 2023-10-16T11:25:42.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.386+0000 7f96ce7fc640 1 -- 172.21.15.84:0/2262478666 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d4005e40 con 0x7f96d8070fd0 2023-10-16T11:25:42.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.386+0000 7f96ce7fc640 1 -- 172.21.15.84:0/2262478666 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4007330 con 0x7f96d8070fd0 2023-10-16T11:25:42.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.386+0000 7f96dfecf640 1 -- 172.21.15.84:0/2262478666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d80c09c0 con 0x7f96d8070fd0 2023-10-16T11:25:42.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.386+0000 7f96dfecf640 1 -- 172.21.15.84:0/2262478666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d8154350 con 0x7f96d8070fd0 2023-10-16T11:25:42.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.387+0000 7f96dfecf640 1 -- 172.21.15.84:0/2262478666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d804e160 con 0x7f96d8070fd0 2023-10-16T11:25:42.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.388+0000 7f96ce7fc640 1 -- 172.21.15.84:0/2262478666 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f96d4007550 con 0x7f96d8070fd0 2023-10-16T11:25:42.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.389+0000 7f96ce7fc640 1 --2- 172.21.15.84:0/2262478666 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f96bc06b0b0 0x7f96bc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:42.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.389+0000 7f96ce7fc640 1 -- 172.21.15.84:0/2262478666 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f96d4089070 con 0x7f96d8070fd0 2023-10-16T11:25:42.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.389+0000 7f96dd46a640 1 --2- 172.21.15.84:0/2262478666 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f96bc06b0b0 0x7f96bc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:42.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.390+0000 7f96dd46a640 1 --2- 172.21.15.84:0/2262478666 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f96bc06b0b0 0x7f96bc06d570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f96c8036000 tx=0x7f96c803c040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:42.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.393+0000 7f96ce7fc640 1 -- 172.21.15.84:0/2262478666 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f96d4053ef0 con 0x7f96d8070fd0 2023-10-16T11:25:42.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.654+0000 7f96dfecf640 1 -- 172.21.15.84:0/2262478666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f96d80c1a30 con 0x7f96d8070fd0 2023-10-16T11:25:42.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.659+0000 7f96ce7fc640 1 -- 172.21.15.84:0/2262478666 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v57) v1 ==== 190+0+147 (secure 0 0 0) 0x7f96d40170e0 con 0x7f96d8070fd0 2023-10-16T11:25:42.661 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2023-10-16T11:25:42.662 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ== 2023-10-16T11:25:42.662 INFO:teuthology.orchestra.run.smithi084.stdout: caps mds = "allow" 2023-10-16T11:25:42.662 INFO:teuthology.orchestra.run.smithi084.stdout: caps mon = "allow r" 2023-10-16T11:25:42.662 INFO:teuthology.orchestra.run.smithi084.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T11:25:42.663 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-16T11:25:42.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.662+0000 7f96dfecf640 1 -- 172.21.15.84:0/2262478666 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f96bc06b0b0 msgr2=0x7f96bc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:42.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.662+0000 7f96dfecf640 1 --2- 172.21.15.84:0/2262478666 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f96bc06b0b0 0x7f96bc06d570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f96c8036000 tx=0x7f96c803c040 comp rx=0 tx=0).stop 2023-10-16T11:25:42.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.662+0000 7f96dfecf640 1 -- 172.21.15.84:0/2262478666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96d8070fd0 msgr2=0x7f96d8153e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:42.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.662+0000 7f96dfecf640 1 --2- 172.21.15.84:0/2262478666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96d8070fd0 0x7f96d8153e10 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7f96d400c500 tx=0x7f96d400c9c0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.662+0000 7f96dfecf640 1 -- 172.21.15.84:0/2262478666 shutdown_connections 2023-10-16T11:25:42.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.662+0000 7f96dfecf640 1 --2- 172.21.15.84:0/2262478666 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f96bc06b0b0 0x7f96bc06d570 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.662+0000 7f96dfecf640 1 --2- 172.21.15.84:0/2262478666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96d80adfc0 0x7f96d80b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.662+0000 7f96dfecf640 1 --2- 172.21.15.84:0/2262478666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96d80adbe0 0x7f96d80bd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.662+0000 7f96dfecf640 1 --2- 172.21.15.84:0/2262478666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96d8070fd0 0x7f96d8153e10 unknown :-1 s=CLOSED pgs=2937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.662+0000 7f96dfecf640 1 -- 172.21.15.84:0/2262478666 >> 172.21.15.84:0/2262478666 conn(0x7f96d80cb410 msgr2=0x7f96d80aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:42.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.662+0000 7f96dfecf640 1 -- 172.21.15.84:0/2262478666 shutdown_connections 2023-10-16T11:25:42.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.662+0000 7f96dfecf640 1 -- 172.21.15.84:0/2262478666 wait complete. 2023-10-16T11:25:42.682 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:25:42.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.893+0000 7f742741c640 1 Processor -- start 2023-10-16T11:25:42.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.893+0000 7f742741c640 1 -- start start 2023-10-16T11:25:42.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.894+0000 7f742741c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7420159b20 0x7f742006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:42.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.894+0000 7f742741c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f742006f7f0 con 0x7f74200b0b20 2023-10-16T11:25:42.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.894+0000 7f742741c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f742006c5e0 con 0x7f7420159f00 2023-10-16T11:25:42.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.894+0000 7f742741c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f742006c8c0 con 0x7f7420159b20 2023-10-16T11:25:42.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.895+0000 7f74259b9640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f74200b0b20 0x7f742006fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42214/0 (socket says 172.21.15.84:42214) 2023-10-16T11:25:42.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.895+0000 7f74259b9640 1 -- 172.21.15.84:0/1181685942 learned_addr learned my addr 172.21.15.84:0/1181685942 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:42.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.895+0000 7f7417fff640 1 -- 172.21.15.84:0/1181685942 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643822600 0 0) 0x7f742006f7f0 con 0x7f74200b0b20 2023-10-16T11:25:42.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.895+0000 7f74251b8640 1 --2- 172.21.15.84:0/1181685942 >> v2:172.21.15.157:3300/0 conn(0x7f7420159b20 0x7f742006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:42.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.895+0000 7f7417fff640 1 -- 172.21.15.84:0/1181685942 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f740c005730 con 0x7f74200b0b20 2023-10-16T11:25:42.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.895+0000 7f7417fff640 1 -- 172.21.15.84:0/1181685942 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4114763970 0 0) 0x7f740c005730 con 0x7f74200b0b20 2023-10-16T11:25:42.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.896+0000 7f7417fff640 1 -- 172.21.15.84:0/1181685942 >> v2:172.21.15.157:3300/0 conn(0x7f7420159b20 msgr2=0x7f742006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:42.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.896+0000 7f7417fff640 1 --2- 172.21.15.84:0/1181685942 >> v2:172.21.15.157:3300/0 conn(0x7f7420159b20 0x7f742006bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.896+0000 7f7417fff640 1 -- 172.21.15.84:0/1181685942 >> v1:172.21.15.157:6789/0 conn(0x7f7420159f00 legacy=0x7f74200b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:42.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.896+0000 7f7417fff640 1 -- 172.21.15.84:0/1181685942 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742006cba0 con 0x7f74200b0b20 2023-10-16T11:25:42.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.896+0000 7f7417fff640 1 -- 172.21.15.84:0/1181685942 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f74180019f0 con 0x7f74200b0b20 2023-10-16T11:25:42.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.896+0000 7f7417fff640 1 -- 172.21.15.84:0/1181685942 >> v1:172.21.15.84:6789/0 conn(0x7f74200b0b20 legacy=0x7f742006fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:42.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.896+0000 7f7417fff640 1 --2- 172.21.15.84:0/1181685942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f740c005b80 0x7f740c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:42.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.896+0000 7f7417fff640 1 --2- 172.21.15.84:0/1181685942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740c0069d0 0x7f740c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:42.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.896+0000 7f7417fff640 1 --2- 172.21.15.84:0/1181685942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f740c007390 0x7f740c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:42.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.896+0000 7f7417fff640 1 -- 172.21.15.84:0/1181685942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742006c8c0 con 0x7f740c007390 2023-10-16T11:25:42.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.896+0000 7f7417fff640 1 -- 172.21.15.84:0/1181685942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c00e2e0 con 0x7f740c0069d0 2023-10-16T11:25:42.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.896+0000 7f7417fff640 1 -- 172.21.15.84:0/1181685942 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c00e4a0 con 0x7f740c005b80 2023-10-16T11:25:42.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.896+0000 7f74259b9640 1 --2- 172.21.15.84:0/1181685942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f740c007390 0x7f740c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:42.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.896+0000 7f74251b8640 1 --2- 172.21.15.84:0/1181685942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f740c005b80 0x7f740c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:42.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.896+0000 7f74249b7640 1 --2- 172.21.15.84:0/1181685942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740c0069d0 0x7f740c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:42.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.896+0000 7f74259b9640 1 -- 172.21.15.84:0/1181685942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f740c005b80 msgr2=0x7f740c005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:42.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.897+0000 7f74259b9640 1 --2- 172.21.15.84:0/1181685942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f740c005b80 0x7f740c005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.897+0000 7f74259b9640 1 -- 172.21.15.84:0/1181685942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740c0069d0 msgr2=0x7f740c006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:42.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.897+0000 7f74259b9640 1 --2- 172.21.15.84:0/1181685942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740c0069d0 0x7f740c006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.897+0000 7f74259b9640 1 -- 172.21.15.84:0/1181685942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c005730 con 0x7f740c007390 2023-10-16T11:25:42.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.897+0000 7f74259b9640 1 -- 172.21.15.84:0/1181685942 reap_dead start 2023-10-16T11:25:42.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.898+0000 7f74259b9640 1 --2- 172.21.15.84:0/1181685942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f740c007390 0x7f740c00db90 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7f741800a820 tx=0x7f740c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae13dd147c0dff1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:42.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.898+0000 7f7417fff640 1 -- 172.21.15.84:0/1181685942 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74180072d0 con 0x7f740c007390 2023-10-16T11:25:42.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.898+0000 7f7417fff640 1 -- 172.21.15.84:0/1181685942 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741802f070 con 0x7f740c007390 2023-10-16T11:25:42.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.898+0000 7f7417fff640 1 -- 172.21.15.84:0/1181685942 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74180076b0 con 0x7f740c007390 2023-10-16T11:25:42.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.898+0000 7f742741c640 1 -- 172.21.15.84:0/1181685942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f740c007390 msgr2=0x7f740c00db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:42.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.898+0000 7f742741c640 1 --2- 172.21.15.84:0/1181685942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f740c007390 0x7f740c00db90 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7f741800a820 tx=0x7f740c007a00 comp rx=0 tx=0).stop 2023-10-16T11:25:42.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.899+0000 7f742741c640 1 -- 172.21.15.84:0/1181685942 shutdown_connections 2023-10-16T11:25:42.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.899+0000 7f742741c640 1 --2- 172.21.15.84:0/1181685942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f740c007390 0x7f740c00db90 unknown :-1 s=CLOSED pgs=2939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.899+0000 7f742741c640 1 -- 172.21.15.84:0/1181685942 >> 172.21.15.84:0/1181685942 conn(0x7f74200cb410 msgr2=0x7f74200cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:42.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.899+0000 7f742741c640 1 -- 172.21.15.84:0/1181685942 shutdown_connections 2023-10-16T11:25:42.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.899+0000 7f742741c640 1 -- 172.21.15.84:0/1181685942 wait complete. 2023-10-16T11:25:42.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.900+0000 7f742741c640 1 Processor -- start 2023-10-16T11:25:42.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.901+0000 7f742741c640 1 -- start start 2023-10-16T11:25:42.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.901+0000 7f742741c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740c007390 0x7f742014ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:42.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.901+0000 7f742741c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74200c7c30 0x7f74200c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:42.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.902+0000 7f742741c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f742015be60 0x7f742015c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:42.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.902+0000 7f742741c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74200c85f0 con 0x7f74200c7c30 2023-10-16T11:25:42.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.902+0000 7f742741c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74200b12d0 con 0x7f740c007390 2023-10-16T11:25:42.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.902+0000 7f742741c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742015cb00 con 0x7f742015be60 2023-10-16T11:25:42.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.902+0000 7f74249b7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74200c7c30 0x7f74200c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:42.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.902+0000 7f74249b7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74200c7c30 0x7f74200c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47350/0 (socket says 172.21.15.84:47350) 2023-10-16T11:25:42.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.902+0000 7f74249b7640 1 -- 172.21.15.84:0/3064546579 learned_addr learned my addr 172.21.15.84:0/3064546579 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:42.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.902+0000 7f74259b9640 1 --2- 172.21.15.84:0/3064546579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f742015be60 0x7f742015c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:42.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.902+0000 7f74251b8640 1 --2- 172.21.15.84:0/3064546579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740c007390 0x7f742014ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:42.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.902+0000 7f74249b7640 1 -- 172.21.15.84:0/3064546579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f742015be60 msgr2=0x7f742015c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:42.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.902+0000 7f74249b7640 1 --2- 172.21.15.84:0/3064546579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f742015be60 0x7f742015c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.902+0000 7f74249b7640 1 -- 172.21.15.84:0/3064546579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740c007390 msgr2=0x7f742014ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:42.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.902+0000 7f74249b7640 1 --2- 172.21.15.84:0/3064546579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740c007390 0x7f742014ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:42.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.902+0000 7f74249b7640 1 -- 172.21.15.84:0/3064546579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742015cde0 con 0x7f74200c7c30 2023-10-16T11:25:42.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.903+0000 7f74249b7640 1 --2- 172.21.15.84:0/3064546579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74200c7c30 0x7f74200c80b0 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7f741000e440 tx=0x7f741000e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:42.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.903+0000 7f7415ffb640 1 -- 172.21.15.84:0/3064546579 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74100052e0 con 0x7f74200c7c30 2023-10-16T11:25:42.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.903+0000 7f7415ffb640 1 -- 172.21.15.84:0/3064546579 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7410005440 con 0x7f74200c7c30 2023-10-16T11:25:42.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.903+0000 7f7415ffb640 1 -- 172.21.15.84:0/3064546579 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7410003830 con 0x7f74200c7c30 2023-10-16T11:25:42.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.903+0000 7f742741c640 1 -- 172.21.15.84:0/3064546579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f742015d0c0 con 0x7f74200c7c30 2023-10-16T11:25:42.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.903+0000 7f742741c640 1 -- 172.21.15.84:0/3064546579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742015c800 con 0x7f74200c7c30 2023-10-16T11:25:42.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.904+0000 7f742741c640 1 -- 172.21.15.84:0/3064546579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742004e160 con 0x7f74200c7c30 2023-10-16T11:25:42.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.905+0000 7f7415ffb640 1 -- 172.21.15.84:0/3064546579 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f7410003990 con 0x7f74200c7c30 2023-10-16T11:25:42.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.907+0000 7f7415ffb640 1 --2- 172.21.15.84:0/3064546579 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f740406b0b0 0x7f740406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:42.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.907+0000 7f7415ffb640 1 -- 172.21.15.84:0/3064546579 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f7410089630 con 0x7f74200c7c30 2023-10-16T11:25:42.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.907+0000 7f74251b8640 1 --2- 172.21.15.84:0/3064546579 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f740406b0b0 0x7f740406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:42.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.908+0000 7f74251b8640 1 --2- 172.21.15.84:0/3064546579 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f740406b0b0 0x7f740406d570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f741c006fd0 tx=0x7f741c008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:42.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:42.911+0000 7f7415ffb640 1 -- 172.21.15.84:0/3064546579 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f74100543f0 con 0x7f74200c7c30 2023-10-16T11:25:43.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.142+0000 7f742741c640 1 -- 172.21.15.84:0/3064546579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74200b10b0 con 0x7f74200c7c30 2023-10-16T11:25:43.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.143+0000 7f7415ffb640 1 -- 172.21.15.84:0/3064546579 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4930 (secure 0 0 0) 0x7f7410057a10 con 0x7f74200c7c30 2023-10-16T11:25:43.145 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:43.146 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":131,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2516077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2516077743},{"type":"v1","addr":"172.21.15.157:6833","nonce":2516077743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1527104462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1527104462},{"type":"v1","addr":"172.21.15.157:6835","nonce":1527104462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":12183,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/369067539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":369067539},{"type":"v1","addr":"172.21.15.84:6835","nonce":369067539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:25:36.605068+0000","modified":"2023-10-16T11:25:39.675208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12186":{"gid":12186,"name":"a","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/3547503033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3547503033},{"type":"v1","addr":"172.21.15.84:6837","nonce":3547503033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-10-16T11:25:43.146 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 131 2023-10-16T11:25:43.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.146+0000 7f742741c640 1 -- 172.21.15.84:0/3064546579 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f740406b0b0 msgr2=0x7f740406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:43.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.146+0000 7f742741c640 1 --2- 172.21.15.84:0/3064546579 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f740406b0b0 0x7f740406d570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f741c006fd0 tx=0x7f741c008040 comp rx=0 tx=0).stop 2023-10-16T11:25:43.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.146+0000 7f742741c640 1 -- 172.21.15.84:0/3064546579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74200c7c30 msgr2=0x7f74200c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:43.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.146+0000 7f742741c640 1 --2- 172.21.15.84:0/3064546579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74200c7c30 0x7f74200c80b0 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7f741000e440 tx=0x7f741000e900 comp rx=0 tx=0).stop 2023-10-16T11:25:43.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.146+0000 7f742741c640 1 -- 172.21.15.84:0/3064546579 shutdown_connections 2023-10-16T11:25:43.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.146+0000 7f742741c640 1 --2- 172.21.15.84:0/3064546579 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f740406b0b0 0x7f740406d570 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.146+0000 7f742741c640 1 --2- 172.21.15.84:0/3064546579 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f742015be60 0x7f742015c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.146+0000 7f742741c640 1 --2- 172.21.15.84:0/3064546579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74200c7c30 0x7f74200c80b0 unknown :-1 s=CLOSED pgs=2940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.146+0000 7f742741c640 1 --2- 172.21.15.84:0/3064546579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740c007390 0x7f742014ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.146+0000 7f742741c640 1 -- 172.21.15.84:0/3064546579 >> 172.21.15.84:0/3064546579 conn(0x7f74200cb410 msgr2=0x7f74200cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:43.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.146+0000 7f742741c640 1 -- 172.21.15.84:0/3064546579 shutdown_connections 2023-10-16T11:25:43.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.146+0000 7f742741c640 1 -- 172.21.15.84:0/3064546579 wait complete. 2023-10-16T11:25:43.568 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 131, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T11:25:36.605068+0000', 'modified': '2023-10-16T11:25:39.675208+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12186}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12186': {'gid': 12186, 'name': 'a', 'rank': 0, 'incarnation': 127, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.84:6837/3547503033', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6836', 'nonce': 3547503033}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 3547503033}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [31], 'metadata_pool': 30, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T11:25:43.569 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T11:25:43.569 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-16T11:25:43.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.793+0000 7f038759e640 1 Processor -- start 2023-10-16T11:25:43.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.793+0000 7f038759e640 1 -- start start 2023-10-16T11:25:43.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.795+0000 7f038759e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038806c9e0 0x7f03880ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.795+0000 7f038759e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03880ca560 con 0x7f03880c9060 2023-10-16T11:25:43.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.795+0000 7f038759e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03880c9950 con 0x7f038806fb50 2023-10-16T11:25:43.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.795+0000 7f038759e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03880c9c30 con 0x7f038806c9e0 2023-10-16T11:25:43.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.795+0000 7f038659c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f03880c9060 0x7f038806f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42230/0 (socket says 172.21.15.84:42230) 2023-10-16T11:25:43.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.795+0000 7f038659c640 1 -- 172.21.15.84:0/4153572936 learned_addr learned my addr 172.21.15.84:0/4153572936 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:43.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.796+0000 7f0385d9b640 1 -- 172.21.15.84:0/4153572936 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582518170 0 0) 0x7f03880ca560 con 0x7f03880c9060 2023-10-16T11:25:43.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.796+0000 7f0386d9d640 1 --2- 172.21.15.84:0/4153572936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038806c9e0 0x7f03880ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:43.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.796+0000 7f0385d9b640 1 -- 172.21.15.84:0/4153572936 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0374003520 con 0x7f03880c9060 2023-10-16T11:25:43.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.796+0000 7f0385d9b640 1 -- 172.21.15.84:0/4153572936 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 602879920 0 0) 0x7f0374003520 con 0x7f03880c9060 2023-10-16T11:25:43.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.796+0000 7f0385d9b640 1 -- 172.21.15.84:0/4153572936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038806c9e0 msgr2=0x7f03880ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:43.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.796+0000 7f0385d9b640 1 --2- 172.21.15.84:0/4153572936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038806c9e0 0x7f03880ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.796+0000 7f0385d9b640 1 -- 172.21.15.84:0/4153572936 >> v1:172.21.15.157:6789/0 conn(0x7f038806fb50 legacy=0x7f03880cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:43.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.796+0000 7f0385d9b640 1 -- 172.21.15.84:0/4153572936 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03880c9f10 con 0x7f03880c9060 2023-10-16T11:25:43.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.796+0000 7f0385d9b640 1 -- 172.21.15.84:0/4153572936 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f0378001670 con 0x7f03880c9060 2023-10-16T11:25:43.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.796+0000 7f0385d9b640 1 -- 172.21.15.84:0/4153572936 >> v1:172.21.15.84:6789/0 conn(0x7f03880c9060 legacy=0x7f038806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:43.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.796+0000 7f0385d9b640 1 --2- 172.21.15.84:0/4153572936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038806c9e0 0x7f03740039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.796+0000 7f0385d9b640 1 --2- 172.21.15.84:0/4153572936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0374003fd0 0x7f03740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.796+0000 7f0385d9b640 1 --2- 172.21.15.84:0/4153572936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0374004d00 0x7f0374005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.796+0000 7f0385d9b640 1 -- 172.21.15.84:0/4153572936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03880c9c30 con 0x7f0374003fd0 2023-10-16T11:25:43.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.796+0000 7f0385d9b640 1 -- 172.21.15.84:0/4153572936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037400bb40 con 0x7f0374004d00 2023-10-16T11:25:43.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.796+0000 7f0385d9b640 1 -- 172.21.15.84:0/4153572936 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f037400bd00 con 0x7f038806c9e0 2023-10-16T11:25:43.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.797+0000 7f037dd9b640 1 --2- 172.21.15.84:0/4153572936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0374003fd0 0x7f03740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:43.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.797+0000 7f038659c640 1 --2- 172.21.15.84:0/4153572936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038806c9e0 0x7f03740039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:43.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.797+0000 7f0386d9d640 1 --2- 172.21.15.84:0/4153572936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0374004d00 0x7f0374005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:43.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.797+0000 7f037dd9b640 1 -- 172.21.15.84:0/4153572936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038806c9e0 msgr2=0x7f03740039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:43.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.797+0000 7f037dd9b640 1 --2- 172.21.15.84:0/4153572936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f038806c9e0 0x7f03740039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.797+0000 7f037dd9b640 1 -- 172.21.15.84:0/4153572936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0374004d00 msgr2=0x7f0374005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:43.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.797+0000 7f037dd9b640 1 --2- 172.21.15.84:0/4153572936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0374004d00 0x7f0374005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.797+0000 7f037dd9b640 1 -- 172.21.15.84:0/4153572936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0374003520 con 0x7f0374003fd0 2023-10-16T11:25:43.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.797+0000 7f0386d9d640 1 --2- 172.21.15.84:0/4153572936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0374004d00 0x7f0374005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:43.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.798+0000 7f037dd9b640 1 --2- 172.21.15.84:0/4153572936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0374003fd0 0x7f03740047c0 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 crypto rx=0x7f0370001750 tx=0x7f03740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a253891c2b51a9ca server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:43.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.798+0000 7f0385d9b640 1 -- 172.21.15.84:0/4153572936 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03700052d0 con 0x7f0374003fd0 2023-10-16T11:25:43.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.798+0000 7f0385d9b640 1 -- 172.21.15.84:0/4153572936 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037002e050 con 0x7f0374003fd0 2023-10-16T11:25:43.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.798+0000 7f0385d9b640 1 -- 172.21.15.84:0/4153572936 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0370003830 con 0x7f0374003fd0 2023-10-16T11:25:43.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.799+0000 7f038759e640 1 -- 172.21.15.84:0/4153572936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0374003fd0 msgr2=0x7f03740047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:43.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.799+0000 7f038759e640 1 --2- 172.21.15.84:0/4153572936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0374003fd0 0x7f03740047c0 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 crypto rx=0x7f0370001750 tx=0x7f03740057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.799+0000 7f0386d9d640 1 -- 172.21.15.84:0/4153572936 reap_dead start 2023-10-16T11:25:43.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.799+0000 7f038759e640 1 -- 172.21.15.84:0/4153572936 shutdown_connections 2023-10-16T11:25:43.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.799+0000 7f038759e640 1 -- 172.21.15.84:0/4153572936 >> 172.21.15.84:0/4153572936 conn(0x7f03880ce6a0 msgr2=0x7f03880ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:43.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.799+0000 7f038759e640 1 -- 172.21.15.84:0/4153572936 shutdown_connections 2023-10-16T11:25:43.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.799+0000 7f038759e640 1 -- 172.21.15.84:0/4153572936 wait complete. 2023-10-16T11:25:43.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.801+0000 7f038759e640 1 Processor -- start 2023-10-16T11:25:43.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.801+0000 7f038759e640 1 -- start start 2023-10-16T11:25:43.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.802+0000 7f038759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0374003fd0 0x7f03880c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.802+0000 7f038759e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03880c5770 0x7f03881f9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.802+0000 7f038759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03881fa3e0 0x7f03881fa860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.802+0000 7f038759e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03880c8a50 con 0x7f03881fa3e0 2023-10-16T11:25:43.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.802+0000 7f038759e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03880c7dd0 con 0x7f0374003fd0 2023-10-16T11:25:43.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.802+0000 7f038759e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03880c80b0 con 0x7f03880c5770 2023-10-16T11:25:43.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.802+0000 7f0386d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03881fa3e0 0x7f03881fa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:43.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.802+0000 7f0386d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03881fa3e0 0x7f03881fa860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47376/0 (socket says 172.21.15.84:47376) 2023-10-16T11:25:43.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.802+0000 7f0386d9d640 1 -- 172.21.15.84:0/499870411 learned_addr learned my addr 172.21.15.84:0/499870411 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:43.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.803+0000 7f038659c640 1 --2- 172.21.15.84:0/499870411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0374003fd0 0x7f03880c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:43.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.803+0000 7f037dd9b640 1 --2- 172.21.15.84:0/499870411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03880c5770 0x7f03881f9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:43.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.803+0000 7f0386d9d640 1 -- 172.21.15.84:0/499870411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03880c5770 msgr2=0x7f03881f9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:43.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.803+0000 7f0386d9d640 1 --2- 172.21.15.84:0/499870411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03880c5770 0x7f03881f9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.803+0000 7f0386d9d640 1 -- 172.21.15.84:0/499870411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0374003fd0 msgr2=0x7f03880c4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:43.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.803+0000 7f0386d9d640 1 --2- 172.21.15.84:0/499870411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0374003fd0 0x7f03880c4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.803+0000 7f0386d9d640 1 -- 172.21.15.84:0/499870411 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03880c8390 con 0x7f03881fa3e0 2023-10-16T11:25:43.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.803+0000 7f038659c640 1 --2- 172.21.15.84:0/499870411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0374003fd0 0x7f03880c4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:43.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.803+0000 7f0386d9d640 1 --2- 172.21.15.84:0/499870411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03881fa3e0 0x7f03881fa860 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f03880c8c80 tx=0x7f0380009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:43.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.803+0000 7f037f7fe640 1 -- 172.21.15.84:0/499870411 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0380007cf0 con 0x7f03881fa3e0 2023-10-16T11:25:43.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.803+0000 7f037f7fe640 1 -- 172.21.15.84:0/499870411 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0380007e70 con 0x7f03881fa3e0 2023-10-16T11:25:43.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.803+0000 7f037f7fe640 1 -- 172.21.15.84:0/499870411 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0380014620 con 0x7f03881fa3e0 2023-10-16T11:25:43.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.803+0000 7f038759e640 1 -- 172.21.15.84:0/499870411 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03880c8670 con 0x7f03881fa3e0 2023-10-16T11:25:43.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.804+0000 7f038759e640 1 -- 172.21.15.84:0/499870411 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03881f6120 con 0x7f03881fa3e0 2023-10-16T11:25:43.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.807+0000 7f037f7fe640 1 -- 172.21.15.84:0/499870411 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f038001d3b0 con 0x7f03881fa3e0 2023-10-16T11:25:43.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.808+0000 7f037f7fe640 1 --2- 172.21.15.84:0/499870411 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f036806b0b0 0x7f036806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.808+0000 7f037f7fe640 1 -- 172.21.15.84:0/499870411 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0380089790 con 0x7f03881fa3e0 2023-10-16T11:25:43.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.808+0000 7f038659c640 1 --2- 172.21.15.84:0/499870411 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f036806b0b0 0x7f036806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:43.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.808+0000 7f038fa5e740 1 Processor -- start 2023-10-16T11:25:43.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.809+0000 7f038fa5e740 1 -- start start 2023-10-16T11:25:43.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.809+0000 7f038659c640 1 --2- 172.21.15.84:0/499870411 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f036806b0b0 0x7f036806d570 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0378001200 tx=0x7f0378009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:43.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.809+0000 7f038fa5e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c87afd6170 0x55c87afa9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.809+0000 7f038fa5e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87afd6550 0x55c87afd8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.809+0000 7f038fa5e740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87afa95b0 0x55c87afd9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.810+0000 7f038fa5e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87aec97a0 con 0x55c87afd6550 2023-10-16T11:25:43.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.810+0000 7f038fa5e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87af1b740 con 0x55c87afd6170 2023-10-16T11:25:43.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.810+0000 7f038fa5e740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c87af18e30 con 0x55c87afa95b0 2023-10-16T11:25:43.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.810+0000 7f037dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87afd6550 0x55c87afd8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:43.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.810+0000 7f037dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87afd6550 0x55c87afd8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47378/0 (socket says 172.21.15.84:47378) 2023-10-16T11:25:43.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.810+0000 7f037dd9b640 1 -- 172.21.15.84:0/1779982377 learned_addr learned my addr 172.21.15.84:0/1779982377 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:43.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.810+0000 7f038659c640 1 --2- 172.21.15.84:0/1779982377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c87afd6170 0x55c87afa9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:43.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.810+0000 7f0386d9d640 1 --2- 172.21.15.84:0/1779982377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87afa95b0 0x55c87afd9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:43.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.810+0000 7f037dd9b640 1 -- 172.21.15.84:0/1779982377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87afa95b0 msgr2=0x55c87afd9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:43.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.810+0000 7f037dd9b640 1 --2- 172.21.15.84:0/1779982377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87afa95b0 0x55c87afd9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.810+0000 7f037dd9b640 1 -- 172.21.15.84:0/1779982377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c87afd6170 msgr2=0x55c87afa9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:43.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.810+0000 7f037dd9b640 1 --2- 172.21.15.84:0/1779982377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c87afd6170 0x55c87afa9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.810+0000 7f037dd9b640 1 -- 172.21.15.84:0/1779982377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c87aed8910 con 0x55c87afd6550 2023-10-16T11:25:43.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.810+0000 7f0386d9d640 1 --2- 172.21.15.84:0/1779982377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87afa95b0 0x55c87afd9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:43.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.811+0000 7f037dd9b640 1 --2- 172.21.15.84:0/1779982377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87afd6550 0x55c87afd8f00 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7f0370001870 tx=0x7f037002f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1686c400f941840c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:43.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.811+0000 7f0367fff640 1 -- 172.21.15.84:0/1779982377 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0370005350 con 0x55c87afd6550 2023-10-16T11:25:43.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.811+0000 7f0367fff640 1 -- 172.21.15.84:0/1779982377 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03700054b0 con 0x55c87afd6550 2023-10-16T11:25:43.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.811+0000 7f0367fff640 1 -- 172.21.15.84:0/1779982377 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0370007280 con 0x55c87afd6550 2023-10-16T11:25:43.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.812+0000 7f038fa5e740 1 -- 172.21.15.84:0/1779982377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87afd6550 msgr2=0x55c87afd8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:43.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.812+0000 7f038fa5e740 1 --2- 172.21.15.84:0/1779982377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87afd6550 0x55c87afd8f00 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7f0370001870 tx=0x7f037002f000 comp rx=0 tx=0).stop 2023-10-16T11:25:43.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.812+0000 7f038fa5e740 1 -- 172.21.15.84:0/1779982377 shutdown_connections 2023-10-16T11:25:43.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.812+0000 7f038fa5e740 1 --2- 172.21.15.84:0/1779982377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87afa95b0 0x55c87afd9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.812+0000 7f038fa5e740 1 --2- 172.21.15.84:0/1779982377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87afd6550 0x55c87afd8f00 unknown :-1 s=CLOSED pgs=2944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.812+0000 7f038fa5e740 1 --2- 172.21.15.84:0/1779982377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c87afd6170 0x55c87afa9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.812+0000 7f038fa5e740 1 -- 172.21.15.84:0/1779982377 >> 172.21.15.84:0/1779982377 conn(0x55c87afab380 msgr2=0x55c87aff3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:43.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.812+0000 7f038fa5e740 1 -- 172.21.15.84:0/1779982377 shutdown_connections 2023-10-16T11:25:43.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.812+0000 7f038fa5e740 1 -- 172.21.15.84:0/1779982377 wait complete. 2023-10-16T11:25:43.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.812+0000 7f038fa5e740 1 -- start start 2023-10-16T11:25:43.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.813+0000 7f038fa5e740 1 Processor -- start 2023-10-16T11:25:43.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.814+0000 7f038fa5e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c87afd6170 0x55c87afce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.814+0000 7f038fa5e740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87afd6550 0x55c87afeafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.814+0000 7f038fa5e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87b007400 0x55c87b007880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.814+0000 7f038fa5e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87af14720 con 0x55c87b007400 2023-10-16T11:25:43.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.814+0000 7f038fa5e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87aeb2970 con 0x55c87afd6170 2023-10-16T11:25:43.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.814+0000 7f038fa5e740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c87aeb2610 con 0x55c87afd6550 2023-10-16T11:25:43.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.814+0000 7f038fa5e740 10 client.0 _ll_register_callbacks cb 0x55c87aed49d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:25:43.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.814+0000 7f0386d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87b007400 0x55c87b007880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:43.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.814+0000 7f0386d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87b007400 0x55c87b007880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47384/0 (socket says 172.21.15.84:47384) 2023-10-16T11:25:43.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.814+0000 7f0386d9d640 1 -- 172.21.15.84:0/4119975039 learned_addr learned my addr 172.21.15.84:0/4119975039 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:43.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.814+0000 7f0386d9d640 1 -- 172.21.15.84:0/4119975039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87afd6550 msgr2=0x55c87afeafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:43.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.814+0000 7f0386d9d640 1 --2- 172.21.15.84:0/4119975039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87afd6550 0x55c87afeafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.814+0000 7f0386d9d640 1 -- 172.21.15.84:0/4119975039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c87afd6170 msgr2=0x55c87afce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:43.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.814+0000 7f0386d9d640 1 --2- 172.21.15.84:0/4119975039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c87afd6170 0x55c87afce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.814+0000 7f0386d9d640 1 -- 172.21.15.84:0/4119975039 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0374003520 con 0x55c87b007400 2023-10-16T11:25:43.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.815+0000 7f0386d9d640 1 --2- 172.21.15.84:0/4119975039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87b007400 0x55c87b007880 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 crypto rx=0x7f0380060020 tx=0x7f038005c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6e815eba28d681 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:43.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.815+0000 7f03657fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T11:25:43.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.815+0000 7f03657fa640 1 -- 172.21.15.84:0/4119975039 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0380057740 con 0x55c87b007400 2023-10-16T11:25:43.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.815+0000 7f03657fa640 1 -- 172.21.15.84:0/4119975039 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f03800578a0 con 0x55c87b007400 2023-10-16T11:25:43.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.815+0000 7f038fa5e740 1 -- 172.21.15.84:0/4119975039 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c87af3b160 con 0x55c87b007400 2023-10-16T11:25:43.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.815+0000 7f03657fa640 1 -- 172.21.15.84:0/4119975039 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03800600c0 con 0x55c87b007400 2023-10-16T11:25:43.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.815+0000 7f038fa5e740 10 client.12333 fetch_fsmap learned FSMap version 131 2023-10-16T11:25:43.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.815+0000 7f038fa5e740 1 -- 172.21.15.84:0/4119975039 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=131,monmap=2+}) v3 -- 0x55c87af12da0 con 0x55c87b007400 2023-10-16T11:25:43.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.815+0000 7f03657fa640 1 -- 172.21.15.84:0/4119975039 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c87af3b160 con 0x55c87b007400 2023-10-16T11:25:43.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.815+0000 7f03657fa640 1 -- 172.21.15.84:0/4119975039 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 131) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0380087b70 con 0x55c87b007400 2023-10-16T11:25:43.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.815+0000 7f038fa5e740 10 client.12333 fetch_fsmap finished waiting for FSMap version 131 2023-10-16T11:25:43.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.816+0000 7f038fa5e740 10 client.12333 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T11:25:43.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.816+0000 7f038fa5e740 20 client.12333 populate_metadata read hostname 'smithi084' 2023-10-16T11:25:43.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.816+0000 7f038fa5e740 1 --2- 172.21.15.84:0/4119975039 >> [v2:172.21.15.84:6836/3547503033,v1:172.21.15.84:6837/3547503033] conn(0x55c87afda120 0x55c87afedf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.816+0000 7f038fa5e740 4 client.12333 mds_command: new command op to 12186 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T11:25:43.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.816+0000 7f038fa5e740 1 -- 172.21.15.84:0/4119975039 --> [v2:172.21.15.84:6836/3547503033,v1:172.21.15.84:6837/3547503033] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c87aee6cc0 con 0x55c87afda120 2023-10-16T11:25:43.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.816+0000 7f038659c640 1 --2- 172.21.15.84:0/4119975039 >> [v2:172.21.15.84:6836/3547503033,v1:172.21.15.84:6837/3547503033] conn(0x55c87afda120 0x55c87afedf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:43.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.816+0000 7f038659c640 1 --2- 172.21.15.84:0/4119975039 >> [v2:172.21.15.84:6836/3547503033,v1:172.21.15.84:6837/3547503033] conn(0x55c87afda120 0x55c87afedf00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48a94a5221f8c2d5 server_cookie=b5b1c824fed19142 in_seq=0 out_seq=0 2023-10-16T11:25:43.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.816+0000 7f03657fa640 10 client.12333 ms_handle_connect on v2:172.21.15.84:6836/3547503033 2023-10-16T11:25:43.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.817+0000 7f03657fa640 1 -- 172.21.15.84:0/4119975039 <== mds.0 v2:172.21.15.84:6836/3547503033 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c87aee6cc0 con 0x55c87afda120 2023-10-16T11:25:43.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.817+0000 7f03657fa640 10 client.12333 handle_command_reply: tid=0 2023-10-16T11:25:43.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.817+0000 7f038fa5e740 1 client.12333 shutdown 2023-10-16T11:25:43.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.817+0000 7f038fa5e740 1 -- 172.21.15.84:0/4119975039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87b007400 msgr2=0x55c87b007880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:43.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.817+0000 7f038fa5e740 1 --2- 172.21.15.84:0/4119975039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87b007400 0x55c87b007880 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 crypto rx=0x7f0380060020 tx=0x7f038005c3a0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.817+0000 7f038fa5e740 1 -- 172.21.15.84:0/4119975039 shutdown_connections 2023-10-16T11:25:43.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.817+0000 7f038fa5e740 1 --2- 172.21.15.84:0/4119975039 >> [v2:172.21.15.84:6836/3547503033,v1:172.21.15.84:6837/3547503033] conn(0x55c87afda120 0x55c87afedf00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.817+0000 7f038fa5e740 1 --2- 172.21.15.84:0/4119975039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87b007400 0x55c87b007880 unknown :-1 s=CLOSED pgs=2945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.817+0000 7f038fa5e740 1 --2- 172.21.15.84:0/4119975039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87afd6550 0x55c87afeafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.817+0000 7f038fa5e740 1 --2- 172.21.15.84:0/4119975039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c87afd6170 0x55c87afce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.817+0000 7f03657fa640 0 client.12333 ms_handle_reset on v2:172.21.15.84:6836/3547503033 2023-10-16T11:25:43.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.817+0000 7f038fa5e740 1 -- 172.21.15.84:0/4119975039 >> 172.21.15.84:0/4119975039 conn(0x55c87afa95b0 msgr2=0x55c87afacff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:43.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.818+0000 7f038fa5e740 1 -- 172.21.15.84:0/4119975039 shutdown_connections 2023-10-16T11:25:43.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.818+0000 7f038fa5e740 1 -- 172.21.15.84:0/4119975039 wait complete. 2023-10-16T11:25:43.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.818+0000 7f038fa5e740 20 client.12333 trim_cache size 0 max 16384 2023-10-16T11:25:43.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.834+0000 7f038fa5e740 1 Processor -- start 2023-10-16T11:25:43.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.834+0000 7f038fa5e740 1 -- start start 2023-10-16T11:25:43.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.835+0000 7f038fa5e740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87afd6170 0x55c87afaf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.835+0000 7f038fa5e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c87afd6550 0x55c87afeab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.835+0000 7f038fa5e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87b007400 0x55c87afeb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.835+0000 7f038fa5e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87ac78f20 con 0x55c87b007400 2023-10-16T11:25:43.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.835+0000 7f038fa5e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87aff7eb0 con 0x55c87afd6550 2023-10-16T11:25:43.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.835+0000 7f038fa5e740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c87afd93e0 con 0x55c87afd6170 2023-10-16T11:25:43.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.835+0000 7f0386d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87b007400 0x55c87afeb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:43.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.836+0000 7f0386d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87b007400 0x55c87afeb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47394/0 (socket says 172.21.15.84:47394) 2023-10-16T11:25:43.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.836+0000 7f0386d9d640 1 -- 172.21.15.84:0/1297198661 learned_addr learned my addr 172.21.15.84:0/1297198661 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:43.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.836+0000 7f038659c640 1 --2- 172.21.15.84:0/1297198661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87afd6170 0x55c87afaf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:43.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.836+0000 7f037dd9b640 1 --2- 172.21.15.84:0/1297198661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c87afd6550 0x55c87afeab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:43.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.836+0000 7f0386d9d640 1 -- 172.21.15.84:0/1297198661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87afd6170 msgr2=0x55c87afaf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:43.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.836+0000 7f0386d9d640 1 --2- 172.21.15.84:0/1297198661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87afd6170 0x55c87afaf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.836+0000 7f0386d9d640 1 -- 172.21.15.84:0/1297198661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c87afd6550 msgr2=0x55c87afeab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:43.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.836+0000 7f0386d9d640 1 --2- 172.21.15.84:0/1297198661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c87afd6550 0x55c87afeab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.836+0000 7f0386d9d640 1 -- 172.21.15.84:0/1297198661 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c87aeb1b80 con 0x55c87b007400 2023-10-16T11:25:43.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.836+0000 7f0386d9d640 1 --2- 172.21.15.84:0/1297198661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87b007400 0x55c87afeb070 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 crypto rx=0x7f03800581e0 tx=0x7f0380020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ed4ebb25e5ce91 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:43.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.836+0000 7f037cd99640 1 -- 172.21.15.84:0/1297198661 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0380058020 con 0x55c87b007400 2023-10-16T11:25:43.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.837+0000 7f037cd99640 1 -- 172.21.15.84:0/1297198661 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0380087b70 con 0x55c87b007400 2023-10-16T11:25:43.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.837+0000 7f037cd99640 1 -- 172.21.15.84:0/1297198661 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0380087ce0 con 0x55c87b007400 2023-10-16T11:25:43.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.837+0000 7f038fa5e740 1 -- 172.21.15.84:0/1297198661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87b007400 msgr2=0x55c87afeb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:43.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.837+0000 7f038fa5e740 1 --2- 172.21.15.84:0/1297198661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87b007400 0x55c87afeb070 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 crypto rx=0x7f03800581e0 tx=0x7f0380020000 comp rx=0 tx=0).stop 2023-10-16T11:25:43.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.837+0000 7f038fa5e740 1 -- 172.21.15.84:0/1297198661 shutdown_connections 2023-10-16T11:25:43.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.837+0000 7f038fa5e740 1 --2- 172.21.15.84:0/1297198661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87b007400 0x55c87afeb070 unknown :-1 s=CLOSED pgs=2946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.837+0000 7f038fa5e740 1 --2- 172.21.15.84:0/1297198661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c87afd6550 0x55c87afeab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.837+0000 7f038fa5e740 1 --2- 172.21.15.84:0/1297198661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87afd6170 0x55c87afaf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.837+0000 7f038fa5e740 1 -- 172.21.15.84:0/1297198661 >> 172.21.15.84:0/1297198661 conn(0x55c87afa95b0 msgr2=0x55c87afcdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:43.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.837+0000 7f038fa5e740 1 -- 172.21.15.84:0/1297198661 shutdown_connections 2023-10-16T11:25:43.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.837+0000 7f038fa5e740 1 -- 172.21.15.84:0/1297198661 wait complete. 2023-10-16T11:25:43.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.838+0000 7f038fa5e740 1 -- start start 2023-10-16T11:25:43.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.838+0000 7f038fa5e740 1 Processor -- start 2023-10-16T11:25:43.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.839+0000 7f038fa5e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87afd6170 0x55c87b05b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.839+0000 7f038fa5e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c87afd6550 0x55c87afd4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.839+0000 7f038fa5e740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87b007400 0x55c87afd50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.839+0000 7f038fa5e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87b05d770 con 0x55c87afd6170 2023-10-16T11:25:43.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.839+0000 7f038fa5e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87afa80f0 con 0x55c87afd6550 2023-10-16T11:25:43.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.839+0000 7f038fa5e740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c87afeb530 con 0x55c87b007400 2023-10-16T11:25:43.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.839+0000 7f038fa5e740 10 client.0 _ll_register_callbacks cb 0x7f0380057e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:25:43.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.839+0000 7f038659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87afd6170 0x55c87b05b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:43.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.839+0000 7f038659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87afd6170 0x55c87b05b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47410/0 (socket says 172.21.15.84:47410) 2023-10-16T11:25:43.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.839+0000 7f038659c640 1 -- 172.21.15.84:0/3521481197 learned_addr learned my addr 172.21.15.84:0/3521481197 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:43.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.840+0000 7f038659c640 1 -- 172.21.15.84:0/3521481197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87b007400 msgr2=0x55c87afd50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:43.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.840+0000 7f038659c640 1 --2- 172.21.15.84:0/3521481197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87b007400 0x55c87afd50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.840+0000 7f038659c640 1 -- 172.21.15.84:0/3521481197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c87afd6550 msgr2=0x55c87afd4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:43.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.840+0000 7f038659c640 1 --2- 172.21.15.84:0/3521481197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c87afd6550 0x55c87afd4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.840+0000 7f038659c640 1 -- 172.21.15.84:0/3521481197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c87af12da0 con 0x55c87afd6170 2023-10-16T11:25:43.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.840+0000 7f038659c640 1 --2- 172.21.15.84:0/3521481197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87afd6170 0x55c87b05b990 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7f037800eb30 tx=0x7f0378001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6e85fe9c86b41a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:43.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.840+0000 7f03657fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T11:25:43.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.840+0000 7f03657fa640 1 -- 172.21.15.84:0/3521481197 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037800c070 con 0x55c87afd6170 2023-10-16T11:25:43.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.840+0000 7f03657fa640 1 -- 172.21.15.84:0/3521481197 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0378007510 con 0x55c87afd6170 2023-10-16T11:25:43.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.840+0000 7f038fa5e740 1 -- 172.21.15.84:0/3521481197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03700054b0 con 0x55c87afd6170 2023-10-16T11:25:43.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.841+0000 7f03657fa640 1 -- 172.21.15.84:0/3521481197 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7f037800eed0 con 0x55c87afd6170 2023-10-16T11:25:43.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.841+0000 7f038fa5e740 10 client.12339 fetch_fsmap learned FSMap version 131 2023-10-16T11:25:43.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.841+0000 7f038fa5e740 1 -- 172.21.15.84:0/3521481197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=131,monmap=2+}) v3 -- 0x55c87aed7170 con 0x55c87afd6170 2023-10-16T11:25:43.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.841+0000 7f03657fa640 1 -- 172.21.15.84:0/3521481197 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03700054b0 con 0x55c87afd6170 2023-10-16T11:25:43.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.841+0000 7f03657fa640 1 -- 172.21.15.84:0/3521481197 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 131) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0378020050 con 0x55c87afd6170 2023-10-16T11:25:43.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.841+0000 7f038fa5e740 10 client.12339 fetch_fsmap finished waiting for FSMap version 131 2023-10-16T11:25:43.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.841+0000 7f038fa5e740 10 client.12339 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T11:25:43.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.841+0000 7f038fa5e740 20 client.12339 populate_metadata read hostname 'smithi084' 2023-10-16T11:25:43.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.841+0000 7f038fa5e740 1 --2- 172.21.15.84:0/3521481197 >> [v2:172.21.15.84:6836/3547503033,v1:172.21.15.84:6837/3547503033] conn(0x55c87b006300 0x55c87b0066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:43.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.841+0000 7f038fa5e740 4 client.12339 mds_command: new command op to 12186 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T11:25:43.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.841+0000 7f038fa5e740 1 -- 172.21.15.84:0/3521481197 --> [v2:172.21.15.84:6836/3547503033,v1:172.21.15.84:6837/3547503033] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c87af0d700 con 0x55c87b006300 2023-10-16T11:25:43.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.841+0000 7f037dd9b640 1 --2- 172.21.15.84:0/3521481197 >> [v2:172.21.15.84:6836/3547503033,v1:172.21.15.84:6837/3547503033] conn(0x55c87b006300 0x55c87b0066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:43.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.842+0000 7f037dd9b640 1 --2- 172.21.15.84:0/3521481197 >> [v2:172.21.15.84:6836/3547503033,v1:172.21.15.84:6837/3547503033] conn(0x55c87b006300 0x55c87b0066e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2050d576b4d1a1f server_cookie=74c746643aa18695 in_seq=0 out_seq=0 2023-10-16T11:25:43.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.842+0000 7f03657fa640 10 client.12339 ms_handle_connect on v2:172.21.15.84:6836/3547503033 2023-10-16T11:25:43.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.842+0000 7f03657fa640 1 -- 172.21.15.84:0/3521481197 <== mds.0 v2:172.21.15.84:6836/3547503033 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55c87af0d700 con 0x55c87b006300 2023-10-16T11:25:43.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.842+0000 7f03657fa640 10 client.12339 handle_command_reply: tid=0 2023-10-16T11:25:43.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.842+0000 7f038fa5e740 1 client.12339 shutdown 2023-10-16T11:25:43.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.842+0000 7f038fa5e740 1 -- 172.21.15.84:0/3521481197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87afd6170 msgr2=0x55c87b05b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:43.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.842+0000 7f038fa5e740 1 --2- 172.21.15.84:0/3521481197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87afd6170 0x55c87b05b990 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7f037800eb30 tx=0x7f0378001a00 comp rx=0 tx=0).stop 2023-10-16T11:25:43.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.842+0000 7f038fa5e740 1 -- 172.21.15.84:0/3521481197 shutdown_connections 2023-10-16T11:25:43.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.842+0000 7f038fa5e740 1 --2- 172.21.15.84:0/3521481197 >> [v2:172.21.15.84:6836/3547503033,v1:172.21.15.84:6837/3547503033] conn(0x55c87b006300 0x55c87b0066e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.842+0000 7f038fa5e740 1 --2- 172.21.15.84:0/3521481197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55c87b007400 0x55c87afd50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.842+0000 7f038fa5e740 1 --2- 172.21.15.84:0/3521481197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c87afd6550 0x55c87afd4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.842+0000 7f03657fa640 0 client.12339 ms_handle_reset on v2:172.21.15.84:6836/3547503033 2023-10-16T11:25:43.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.842+0000 7f038fa5e740 1 --2- 172.21.15.84:0/3521481197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c87afd6170 0x55c87b05b990 unknown :-1 s=CLOSED pgs=2947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.842+0000 7f038fa5e740 1 -- 172.21.15.84:0/3521481197 >> 172.21.15.84:0/3521481197 conn(0x55c87afa95b0 msgr2=0x55c87afcdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:43.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.842+0000 7f038fa5e740 1 -- 172.21.15.84:0/3521481197 shutdown_connections 2023-10-16T11:25:43.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.843+0000 7f038fa5e740 1 -- 172.21.15.84:0/3521481197 wait complete. 2023-10-16T11:25:43.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.843+0000 7f038fa5e740 20 client.12339 trim_cache size 0 max 16384 2023-10-16T11:25:43.861 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:25:43.862 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "63fb4d68-f881-4694-b880-d10369105c0e", 2023-10-16T11:25:43.862 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-16T11:25:43.862 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 12186, 2023-10-16T11:25:43.862 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-16T11:25:43.862 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-16T11:25:43.863 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-16T11:25:43.863 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 6.7009586670000001, 2023-10-16T11:25:43.863 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 131, 2023-10-16T11:25:43.863 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 237, 2023-10-16T11:25:43.863 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 237, 2023-10-16T11:25:43.863 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 12.075592201999999 2023-10-16T11:25:43.864 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:25:43.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.843+0000 7f038759e640 1 -- 172.21.15.84:0/499870411 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f036806b0b0 msgr2=0x7f036806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:43.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.843+0000 7f038759e640 1 --2- 172.21.15.84:0/499870411 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f036806b0b0 0x7f036806d570 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0378001200 tx=0x7f0378009040 comp rx=0 tx=0).stop 2023-10-16T11:25:43.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.843+0000 7f038759e640 1 -- 172.21.15.84:0/499870411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03881fa3e0 msgr2=0x7f03881fa860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:43.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.843+0000 7f038759e640 1 --2- 172.21.15.84:0/499870411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03881fa3e0 0x7f03881fa860 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f03880c8c80 tx=0x7f0380009d30 comp rx=0 tx=0).stop 2023-10-16T11:25:43.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.843+0000 7f038759e640 1 -- 172.21.15.84:0/499870411 shutdown_connections 2023-10-16T11:25:43.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.843+0000 7f038759e640 1 --2- 172.21.15.84:0/499870411 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f036806b0b0 0x7f036806d570 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.843+0000 7f038759e640 1 --2- 172.21.15.84:0/499870411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03881fa3e0 0x7f03881fa860 unknown :-1 s=CLOSED pgs=2943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.843+0000 7f038759e640 1 --2- 172.21.15.84:0/499870411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03880c5770 0x7f03881f9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.843+0000 7f038759e640 1 --2- 172.21.15.84:0/499870411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0374003fd0 0x7f03880c4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:43.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.843+0000 7f038759e640 1 -- 172.21.15.84:0/499870411 >> 172.21.15.84:0/499870411 conn(0x7f03880ce6a0 msgr2=0x7f038806b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:43.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.843+0000 7f038759e640 1 -- 172.21.15.84:0/499870411 shutdown_connections 2023-10-16T11:25:43.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:43.843+0000 7f038759e640 1 -- 172.21.15.84:0/499870411 wait complete. 2023-10-16T11:25:43.868 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T11:25:44.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.083+0000 7f685700e640 1 Processor -- start 2023-10-16T11:25:44.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.083+0000 7f685700e640 1 -- start start 2023-10-16T11:25:44.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.085+0000 7f685700e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6850159b20 0x7f685006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:44.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.085+0000 7f685700e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6850159f00 0x7f68500b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:44.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.085+0000 7f685700e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685006f7f0 con 0x7f685006c120 2023-10-16T11:25:44.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.085+0000 7f685700e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685006c5e0 con 0x7f6850159f00 2023-10-16T11:25:44.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.085+0000 7f685700e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685006c8c0 con 0x7f6850159b20 2023-10-16T11:25:44.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.085+0000 7f684ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6850159f00 0x7f68500b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:44.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.085+0000 7f68555ab640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f685006c120 0x7f68500cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42236/0 (socket says 172.21.15.84:42236) 2023-10-16T11:25:44.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.085+0000 7f68555ab640 1 -- 172.21.15.84:0/3258361944 learned_addr learned my addr 172.21.15.84:0/3258361944 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:44.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.086+0000 7f684ffff640 1 -- 172.21.15.84:0/3258361944 >> v2:172.21.15.157:3300/0 conn(0x7f6850159b20 msgr2=0x7f685006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:44.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.086+0000 7f6854daa640 1 --2- 172.21.15.84:0/3258361944 >> v2:172.21.15.157:3300/0 conn(0x7f6850159b20 0x7f685006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:44.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.086+0000 7f684ffff640 1 --2- 172.21.15.84:0/3258361944 >> v2:172.21.15.157:3300/0 conn(0x7f6850159b20 0x7f685006bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:44.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.086+0000 7f684ffff640 1 -- 172.21.15.84:0/3258361944 >> v1:172.21.15.84:6789/0 conn(0x7f685006c120 legacy=0x7f68500cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:44.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.086+0000 7f684f7fe640 1 -- 172.21.15.84:0/3258361944 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085630901 0 0) 0x7f685006f7f0 con 0x7f685006c120 2023-10-16T11:25:44.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.086+0000 7f684ffff640 1 -- 172.21.15.84:0/3258361944 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685006cba0 con 0x7f6850159f00 2023-10-16T11:25:44.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.086+0000 7f6854daa640 1 --2- 172.21.15.84:0/3258361944 >> v2:172.21.15.157:3300/0 conn(0x7f6850159b20 0x7f685006bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:44.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.087+0000 7f684ffff640 1 --2- 172.21.15.84:0/3258361944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6850159f00 0x7f68500b0410 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7f68500c9340 tx=0x7f683c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac209931efd39c34 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:44.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.087+0000 7f684f7fe640 1 -- 172.21.15.84:0/3258361944 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c031830 con 0x7f6850159f00 2023-10-16T11:25:44.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.087+0000 7f684f7fe640 1 -- 172.21.15.84:0/3258361944 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683c031990 con 0x7f6850159f00 2023-10-16T11:25:44.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.087+0000 7f684f7fe640 1 -- 172.21.15.84:0/3258361944 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c031d20 con 0x7f6850159f00 2023-10-16T11:25:44.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.088+0000 7f685700e640 1 -- 172.21.15.84:0/3258361944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6850159f00 msgr2=0x7f68500b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:44.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.088+0000 7f685700e640 1 --2- 172.21.15.84:0/3258361944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6850159f00 0x7f68500b0410 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7f68500c9340 tx=0x7f683c02b8d0 comp rx=0 tx=0).stop 2023-10-16T11:25:44.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.088+0000 7f685700e640 1 -- 172.21.15.84:0/3258361944 shutdown_connections 2023-10-16T11:25:44.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.088+0000 7f685700e640 1 --2- 172.21.15.84:0/3258361944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6850159f00 0x7f68500b0410 unknown :-1 s=CLOSED pgs=2949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:44.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.088+0000 7f685700e640 1 --2- 172.21.15.84:0/3258361944 >> v2:172.21.15.157:3300/0 conn(0x7f6850159b20 0x7f685006bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:44.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.088+0000 7f685700e640 1 -- 172.21.15.84:0/3258361944 >> 172.21.15.84:0/3258361944 conn(0x7f68500cb410 msgr2=0x7f68500cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:44.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.088+0000 7f685700e640 1 -- 172.21.15.84:0/3258361944 shutdown_connections 2023-10-16T11:25:44.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.088+0000 7f685700e640 1 -- 172.21.15.84:0/3258361944 wait complete. 2023-10-16T11:25:44.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.090+0000 7f685700e640 1 Processor -- start 2023-10-16T11:25:44.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.090+0000 7f685700e640 1 -- start start 2023-10-16T11:25:44.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.091+0000 7f685700e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f685006c120 0x7f6850150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:44.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.091+0000 7f685700e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6850159b20 0x7f68500c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:44.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.091+0000 7f685700e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6850159f00 0x7f68500ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:44.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.091+0000 7f685700e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500c5e70 con 0x7f685006c120 2023-10-16T11:25:44.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.091+0000 7f685700e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500c5b00 con 0x7f6850159f00 2023-10-16T11:25:44.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.091+0000 7f685700e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68500c4c50 con 0x7f6850159b20 2023-10-16T11:25:44.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.091+0000 7f6854daa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f685006c120 0x7f6850150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:44.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.091+0000 7f6854daa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f685006c120 0x7f6850150560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47432/0 (socket says 172.21.15.84:47432) 2023-10-16T11:25:44.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.091+0000 7f6854daa640 1 -- 172.21.15.84:0/408182918 learned_addr learned my addr 172.21.15.84:0/408182918 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:44.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.092+0000 7f684ffff640 1 --2- 172.21.15.84:0/408182918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6850159b20 0x7f68500c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:44.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.092+0000 7f6854daa640 1 -- 172.21.15.84:0/408182918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6850159b20 msgr2=0x7f68500c24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:44.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.092+0000 7f6854daa640 1 --2- 172.21.15.84:0/408182918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6850159b20 0x7f68500c24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:44.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.092+0000 7f6854daa640 1 -- 172.21.15.84:0/408182918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6850159f00 msgr2=0x7f68500ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:44.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.092+0000 7f6854daa640 1 --2- 172.21.15.84:0/408182918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6850159f00 0x7f68500ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:44.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.092+0000 7f6854daa640 1 -- 172.21.15.84:0/408182918 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c009400 con 0x7f685006c120 2023-10-16T11:25:44.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.092+0000 7f6854daa640 1 --2- 172.21.15.84:0/408182918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f685006c120 0x7f6850150560 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7f684400c7f0 tx=0x7f684400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:44.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.092+0000 7f684d7fa640 1 -- 172.21.15.84:0/408182918 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844007a30 con 0x7f685006c120 2023-10-16T11:25:44.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.092+0000 7f684d7fa640 1 -- 172.21.15.84:0/408182918 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6844007b90 con 0x7f685006c120 2023-10-16T11:25:44.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.092+0000 7f685700e640 1 -- 172.21.15.84:0/408182918 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68500c4f30 con 0x7f685006c120 2023-10-16T11:25:44.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.093+0000 7f684d7fa640 1 -- 172.21.15.84:0/408182918 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684400ce40 con 0x7f685006c120 2023-10-16T11:25:44.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.093+0000 7f685700e640 1 -- 172.21.15.84:0/408182918 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6850150aa0 con 0x7f685006c120 2023-10-16T11:25:44.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.094+0000 7f685700e640 1 -- 172.21.15.84:0/408182918 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685004e160 con 0x7f685006c120 2023-10-16T11:25:44.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.095+0000 7f684d7fa640 1 -- 172.21.15.84:0/408182918 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f68440113f0 con 0x7f685006c120 2023-10-16T11:25:44.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.095+0000 7f684d7fa640 1 --2- 172.21.15.84:0/408182918 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f683406b0b0 0x7f683406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:44.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.096+0000 7f684d7fa640 1 -- 172.21.15.84:0/408182918 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6844089c80 con 0x7f685006c120 2023-10-16T11:25:44.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.096+0000 7f684ffff640 1 --2- 172.21.15.84:0/408182918 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f683406b0b0 0x7f683406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:44.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.096+0000 7f684ffff640 1 --2- 172.21.15.84:0/408182918 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f683406b0b0 0x7f683406d570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f683c036000 tx=0x7f683c008030 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:44.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.100+0000 7f684d7fa640 1 -- 172.21.15.84:0/408182918 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6844054930 con 0x7f685006c120 2023-10-16T11:25:44.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.335+0000 7f685700e640 1 -- 172.21.15.84:0/408182918 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f68500b10b0 con 0x7f685006c120 2023-10-16T11:25:44.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.336+0000 7f684d7fa640 1 -- 172.21.15.84:0/408182918 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v131) v1 ==== 53+0+73 (secure 0 0 0) 0x7f684401f070 con 0x7f685006c120 2023-10-16T11:25:44.338 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T11:25:44.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.339+0000 7f685700e640 1 -- 172.21.15.84:0/408182918 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f683406b0b0 msgr2=0x7f683406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:44.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.339+0000 7f685700e640 1 --2- 172.21.15.84:0/408182918 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f683406b0b0 0x7f683406d570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f683c036000 tx=0x7f683c008030 comp rx=0 tx=0).stop 2023-10-16T11:25:44.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.339+0000 7f685700e640 1 -- 172.21.15.84:0/408182918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f685006c120 msgr2=0x7f6850150560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:44.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.339+0000 7f685700e640 1 --2- 172.21.15.84:0/408182918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f685006c120 0x7f6850150560 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7f684400c7f0 tx=0x7f684400cbf0 comp rx=0 tx=0).stop 2023-10-16T11:25:44.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.339+0000 7f685700e640 1 -- 172.21.15.84:0/408182918 shutdown_connections 2023-10-16T11:25:44.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.339+0000 7f685700e640 1 --2- 172.21.15.84:0/408182918 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f683406b0b0 0x7f683406d570 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:44.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.339+0000 7f685700e640 1 --2- 172.21.15.84:0/408182918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6850159f00 0x7f68500ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:44.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.339+0000 7f685700e640 1 --2- 172.21.15.84:0/408182918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6850159b20 0x7f68500c24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:44.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.339+0000 7f685700e640 1 --2- 172.21.15.84:0/408182918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f685006c120 0x7f6850150560 unknown :-1 s=CLOSED pgs=2950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:44.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.339+0000 7f685700e640 1 -- 172.21.15.84:0/408182918 >> 172.21.15.84:0/408182918 conn(0x7f68500cb410 msgr2=0x7f68500cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:44.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.339+0000 7f685700e640 1 -- 172.21.15.84:0/408182918 shutdown_connections 2023-10-16T11:25:44.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:44.339+0000 7f685700e640 1 -- 172.21.15.84:0/408182918 wait complete. 2023-10-16T11:25:44.359 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T11:25:44.359 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T11:25:44.359 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-16T11:25:44.360 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T11:25:44.360 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T11:25:44.360 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T11:25:44.361 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T11:25:44.361 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T11:25:44.361 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:25:44.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:25:44.362 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-16T11:25:44.419 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T11:25:44.419 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T11:25:44.419 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T11:25:44.419 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:25:44.420 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T11:25:44.420 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:25:44.420 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:25:44.420 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:25:44.420 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-16T11:25:44.421 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:25:44.421 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:25:44.421 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-16T11:25:44.421 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:25:44.421 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:25:44.422 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T11:25:44.422 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:25:44.422 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T11:25:44.422 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 36607sec preferred_lft 36607sec 2023-10-16T11:25:44.422 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-16T11:25:44.423 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:25:44.423 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T11:25:44.423 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:25:44.423 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T11:25:44.423 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:25:44.424 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::20a8:eaff:fe9d:5cea/64 scope link 2023-10-16T11:25:44.424 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:25:44.424 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:25:44.424 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T11:25:44.424 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1854:13ff:fe47:be92/64 scope link 2023-10-16T11:25:44.425 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:25:44.425 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:25:44.425 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether ee:d5:41:a3:a5:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T11:25:44.425 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ecd5:41ff:fea3:a570/64 scope link 2023-10-16T11:25:44.426 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:25:44.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:25:44.427 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T11:25:44.521 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-16T11:25:44.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:25:44.526 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-16T11:25:44.580 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-16T11:25:44.580 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T11:25:44.580 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T11:25:44.581 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T11:25:44.581 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T11:25:44.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:25:44.582 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T11:25:44.582 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T11:25:44.582 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T11:25:44.583 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T11:25:44.583 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T11:25:44.584 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T11:25:44.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:25:44.763 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T11:25:44.812 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T11:25:44.813 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T11:25:44.814 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:25:44.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:25:44.815 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-16T11:25:44.909 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:25:44.910 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T11:25:44.963 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:25:44.963 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:25:44.964 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:25:45.015 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-16T11:25:45.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:25:45.016 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:25:45.149 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:25:45.149 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T11:25:45.182 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:25:45.183 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T11:25:45.183 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T11:25:45.183 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with new device syntax: 0@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T11:25:45.183 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T11:25:45.185 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-16T11:25:45.185 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2023-10-16T11:25:45.185 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T11:25:45.185 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T11:25:45.186 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T11:25:45.188 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T11:25:45.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:25:45.189 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:25:45.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:25:45.274 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T11:25:45.341 INFO:teuthology.orchestra.run.smithi084.stdout:client12351 2023-10-16T11:25:45.342 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:25:45.342 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:25:45.398 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-16T11:25:45.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.620+0000 7ff61caa2640 1 Processor -- start 2023-10-16T11:25:45.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.620+0000 7ff61caa2640 1 -- start start 2023-10-16T11:25:45.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.621+0000 7ff61caa2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff61815e400 0x7ff6180cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:45.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.621+0000 7ff61caa2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff61806df90 0x7ff61806adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:45.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.621+0000 7ff61caa2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61815e7e0 con 0x7ff61806b2f0 2023-10-16T11:25:45.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.621+0000 7ff61caa2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61806b760 con 0x7ff61806df90 2023-10-16T11:25:45.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.621+0000 7ff61caa2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61815f3b0 con 0x7ff61815e400 2023-10-16T11:25:45.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.622+0000 7ff616d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff61806b2f0 0x7ff618070950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57984/0 (socket says 172.21.15.84:57984) 2023-10-16T11:25:45.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.622+0000 7ff616d9d640 1 -- 172.21.15.84:0/1486149461 learned_addr learned my addr 172.21.15.84:0/1486149461 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:45.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.622+0000 7ff615d9b640 1 --2- 172.21.15.84:0/1486149461 >> v2:172.21.15.157:3300/0 conn(0x7ff61806df90 0x7ff61806adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:45.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.622+0000 7ff61659c640 1 --2- 172.21.15.84:0/1486149461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff61815e400 0x7ff6180cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:45.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.622+0000 7ff61559a640 1 -- 172.21.15.84:0/1486149461 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345440399 0 0) 0x7ff61815e7e0 con 0x7ff61806b2f0 2023-10-16T11:25:45.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.622+0000 7ff61559a640 1 -- 172.21.15.84:0/1486149461 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff604003520 con 0x7ff61806b2f0 2023-10-16T11:25:45.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.622+0000 7ff61559a640 1 -- 172.21.15.84:0/1486149461 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3717094967 0 0) 0x7ff604003520 con 0x7ff61806b2f0 2023-10-16T11:25:45.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.622+0000 7ff61559a640 1 -- 172.21.15.84:0/1486149461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff61815e400 msgr2=0x7ff6180cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:45.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.622+0000 7ff61559a640 1 --2- 172.21.15.84:0/1486149461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff61815e400 0x7ff6180cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:45.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.622+0000 7ff61559a640 1 -- 172.21.15.84:0/1486149461 >> v2:172.21.15.157:3300/0 conn(0x7ff61806df90 msgr2=0x7ff61806adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:45.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.622+0000 7ff61559a640 1 --2- 172.21.15.84:0/1486149461 >> v2:172.21.15.157:3300/0 conn(0x7ff61806df90 0x7ff61806adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:45.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.622+0000 7ff61559a640 1 -- 172.21.15.84:0/1486149461 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff618070230 con 0x7ff61806b2f0 2023-10-16T11:25:45.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.622+0000 7ff61559a640 1 -- 172.21.15.84:0/1486149461 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ff610001670 con 0x7ff61806b2f0 2023-10-16T11:25:45.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.622+0000 7ff61559a640 1 -- 172.21.15.84:0/1486149461 >> v1:172.21.15.84:6789/0 conn(0x7ff61806b2f0 legacy=0x7ff618070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:45.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.623+0000 7ff61559a640 1 --2- 172.21.15.84:0/1486149461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff61815e400 0x7ff6040039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:45.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.623+0000 7ff61559a640 1 --2- 172.21.15.84:0/1486149461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff604003fb0 0x7ff6040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:45.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.623+0000 7ff61559a640 1 --2- 172.21.15.84:0/1486149461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff604004d00 0x7ff604005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:45.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.623+0000 7ff61559a640 1 -- 172.21.15.84:0/1486149461 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61806b760 con 0x7ff604004d00 2023-10-16T11:25:45.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.623+0000 7ff61559a640 1 -- 172.21.15.84:0/1486149461 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61815f3b0 con 0x7ff604003fb0 2023-10-16T11:25:45.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.623+0000 7ff61559a640 1 -- 172.21.15.84:0/1486149461 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60400ba00 con 0x7ff61815e400 2023-10-16T11:25:45.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.623+0000 7ff616d9d640 1 --2- 172.21.15.84:0/1486149461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff604004d00 0x7ff604005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:45.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.623+0000 7ff61659c640 1 --2- 172.21.15.84:0/1486149461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff61815e400 0x7ff6040039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:45.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.623+0000 7ff615d9b640 1 --2- 172.21.15.84:0/1486149461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff604003fb0 0x7ff6040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:45.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.623+0000 7ff616d9d640 1 -- 172.21.15.84:0/1486149461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff61815e400 msgr2=0x7ff6040039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:45.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.623+0000 7ff616d9d640 1 --2- 172.21.15.84:0/1486149461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff61815e400 0x7ff6040039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:45.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.623+0000 7ff616d9d640 1 -- 172.21.15.84:0/1486149461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff604003fb0 msgr2=0x7ff6040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:45.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.623+0000 7ff616d9d640 1 --2- 172.21.15.84:0/1486149461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff604003fb0 0x7ff6040047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:45.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.624+0000 7ff616d9d640 1 -- 172.21.15.84:0/1486149461 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff604003520 con 0x7ff604004d00 2023-10-16T11:25:45.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.625+0000 7ff616d9d640 1 --2- 172.21.15.84:0/1486149461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff604004d00 0x7ff604005180 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7ff61000c190 tx=0x7ff6040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb908b3c6f413f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:45.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.625+0000 7ff61559a640 1 -- 172.21.15.84:0/1486149461 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610005ad0 con 0x7ff604004d00 2023-10-16T11:25:45.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.625+0000 7ff61559a640 1 -- 172.21.15.84:0/1486149461 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff610005c50 con 0x7ff604004d00 2023-10-16T11:25:45.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.625+0000 7ff61559a640 1 -- 172.21.15.84:0/1486149461 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61002c810 con 0x7ff604004d00 2023-10-16T11:25:45.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.625+0000 7ff61caa2640 1 -- 172.21.15.84:0/1486149461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff604004d00 msgr2=0x7ff604005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:45.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.625+0000 7ff61caa2640 1 --2- 172.21.15.84:0/1486149461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff604004d00 0x7ff604005180 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7ff61000c190 tx=0x7ff6040057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:45.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.625+0000 7ff616d9d640 1 -- 172.21.15.84:0/1486149461 reap_dead start 2023-10-16T11:25:45.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.625+0000 7ff61caa2640 1 -- 172.21.15.84:0/1486149461 shutdown_connections 2023-10-16T11:25:45.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.625+0000 7ff61caa2640 1 -- 172.21.15.84:0/1486149461 >> 172.21.15.84:0/1486149461 conn(0x7ff6180b0cc0 msgr2=0x7ff6180b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:45.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.625+0000 7ff61caa2640 1 -- 172.21.15.84:0/1486149461 shutdown_connections 2023-10-16T11:25:45.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.626+0000 7ff61caa2640 1 -- 172.21.15.84:0/1486149461 wait complete. 2023-10-16T11:25:45.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.627+0000 7ff61caa2640 1 Processor -- start 2023-10-16T11:25:45.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.627+0000 7ff61caa2640 1 -- start start 2023-10-16T11:25:45.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.628+0000 7ff61caa2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff604004d00 0x7ff6180a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:45.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.628+0000 7ff61caa2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff618070e30 0x7ff6180afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:45.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.628+0000 7ff61caa2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6180a9570 0x7ff61809b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:45.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.628+0000 7ff61caa2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff618152ae0 con 0x7ff604004d00 2023-10-16T11:25:45.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.628+0000 7ff61caa2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff618152520 con 0x7ff6180a9570 2023-10-16T11:25:45.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.628+0000 7ff61caa2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff618152800 con 0x7ff618070e30 2023-10-16T11:25:45.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.629+0000 7ff61659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff604004d00 0x7ff6180a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:45.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.629+0000 7ff61659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff604004d00 0x7ff6180a6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34730/0 (socket says 172.21.15.84:34730) 2023-10-16T11:25:45.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.629+0000 7ff61659c640 1 -- 172.21.15.84:0/3716017423 learned_addr learned my addr 172.21.15.84:0/3716017423 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:45.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.629+0000 7ff616d9d640 1 --2- 172.21.15.84:0/3716017423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6180a9570 0x7ff61809b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:45.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.629+0000 7ff61659c640 1 -- 172.21.15.84:0/3716017423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff618070e30 msgr2=0x7ff6180afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:45.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.629+0000 7ff61659c640 1 --2- 172.21.15.84:0/3716017423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff618070e30 0x7ff6180afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:45.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.629+0000 7ff61659c640 1 -- 172.21.15.84:0/3716017423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6180a9570 msgr2=0x7ff61809b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:45.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.629+0000 7ff61659c640 1 --2- 172.21.15.84:0/3716017423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6180a9570 0x7ff61809b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:45.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.629+0000 7ff61659c640 1 -- 172.21.15.84:0/3716017423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61809b080 con 0x7ff604004d00 2023-10-16T11:25:45.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.629+0000 7ff61659c640 1 --2- 172.21.15.84:0/3716017423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff604004d00 0x7ff6180a6910 secure :-1 s=READY pgs=2954 cs=0 l=1 rev1=1 crypto rx=0x7ff608009e10 tx=0x7ff60800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:45.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.630+0000 7ff60effd640 1 -- 172.21.15.84:0/3716017423 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60800e040 con 0x7ff604004d00 2023-10-16T11:25:45.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.630+0000 7ff60effd640 1 -- 172.21.15.84:0/3716017423 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60800f040 con 0x7ff604004d00 2023-10-16T11:25:45.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.630+0000 7ff60effd640 1 -- 172.21.15.84:0/3716017423 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff608015870 con 0x7ff604004d00 2023-10-16T11:25:45.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.630+0000 7ff61caa2640 1 -- 172.21.15.84:0/3716017423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61809b360 con 0x7ff604004d00 2023-10-16T11:25:45.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.630+0000 7ff61caa2640 1 -- 172.21.15.84:0/3716017423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6180b0340 con 0x7ff604004d00 2023-10-16T11:25:45.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.631+0000 7ff61caa2640 1 -- 172.21.15.84:0/3716017423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61809aee0 con 0x7ff604004d00 2023-10-16T11:25:45.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.632+0000 7ff60effd640 1 -- 172.21.15.84:0/3716017423 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff6080159d0 con 0x7ff604004d00 2023-10-16T11:25:45.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.633+0000 7ff60effd640 1 --2- 172.21.15.84:0/3716017423 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7ff5f806b0e0 0x7ff5f806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:45.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.633+0000 7ff60effd640 1 -- 172.21.15.84:0/3716017423 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff60808aa50 con 0x7ff604004d00 2023-10-16T11:25:45.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.633+0000 7ff615d9b640 1 --2- 172.21.15.84:0/3716017423 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7ff5f806b0e0 0x7ff5f806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:45.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.634+0000 7ff615d9b640 1 --2- 172.21.15.84:0/3716017423 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7ff5f806b0e0 0x7ff5f806d5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff61806b2c0 tx=0x7ff600008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:45.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.637+0000 7ff60effd640 1 -- 172.21.15.84:0/3716017423 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff61809aee0 con 0x7ff604004d00 2023-10-16T11:25:45.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:45.868+0000 7ff61caa2640 1 -- 172.21.15.84:0/3716017423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff61809aee0 con 0x7ff604004d00 2023-10-16T11:25:46.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.229+0000 7ff60effd640 1 -- 172.21.15.84:0/3716017423 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v132) v1 ==== 136+0+0 (secure 0 0 0) 0x7ff61809aee0 con 0x7ff604004d00 2023-10-16T11:25:46.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.232+0000 7ff61caa2640 1 -- 172.21.15.84:0/3716017423 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7ff5f806b0e0 msgr2=0x7ff5f806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:46.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.232+0000 7ff61caa2640 1 --2- 172.21.15.84:0/3716017423 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7ff5f806b0e0 0x7ff5f806d5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff61806b2c0 tx=0x7ff600008040 comp rx=0 tx=0).stop 2023-10-16T11:25:46.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.232+0000 7ff61caa2640 1 -- 172.21.15.84:0/3716017423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff604004d00 msgr2=0x7ff6180a6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:46.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.232+0000 7ff61caa2640 1 --2- 172.21.15.84:0/3716017423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff604004d00 0x7ff6180a6910 secure :-1 s=READY pgs=2954 cs=0 l=1 rev1=1 crypto rx=0x7ff608009e10 tx=0x7ff60800f580 comp rx=0 tx=0).stop 2023-10-16T11:25:46.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.232+0000 7ff61caa2640 1 -- 172.21.15.84:0/3716017423 shutdown_connections 2023-10-16T11:25:46.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.232+0000 7ff61caa2640 1 --2- 172.21.15.84:0/3716017423 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7ff5f806b0e0 0x7ff5f806d5a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:46.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.232+0000 7ff61caa2640 1 --2- 172.21.15.84:0/3716017423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6180a9570 0x7ff61809b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:46.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.232+0000 7ff61caa2640 1 --2- 172.21.15.84:0/3716017423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff618070e30 0x7ff6180afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:46.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.232+0000 7ff61caa2640 1 --2- 172.21.15.84:0/3716017423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff604004d00 0x7ff6180a6910 unknown :-1 s=CLOSED pgs=2954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:46.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.232+0000 7ff61caa2640 1 -- 172.21.15.84:0/3716017423 >> 172.21.15.84:0/3716017423 conn(0x7ff6180b0cc0 msgr2=0x7ff6180cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:46.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.232+0000 7ff61caa2640 1 -- 172.21.15.84:0/3716017423 shutdown_connections 2023-10-16T11:25:46.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.232+0000 7ff61caa2640 1 -- 172.21.15.84:0/3716017423 wait complete. 2023-10-16T11:25:46.256 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:25:46.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.469+0000 7fe4ce4bb640 1 Processor -- start 2023-10-16T11:25:46.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.469+0000 7fe4ce4bb640 1 -- start start 2023-10-16T11:25:46.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.471+0000 7fe4ce4bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4c806c040 0x7fe4c806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:46.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.471+0000 7fe4ce4bb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe4c80ad970 0x7fe4c815e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:46.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.471+0000 7fe4ce4bb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c806c960 con 0x7fe4c815ebc0 2023-10-16T11:25:46.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.471+0000 7fe4ce4bb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c815b9d0 con 0x7fe4c80ad970 2023-10-16T11:25:46.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.471+0000 7fe4ce4bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c815bcb0 con 0x7fe4c806c040 2023-10-16T11:25:46.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.471+0000 7fe4cdcba640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe4c815ebc0 0x7fe4c80cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57988/0 (socket says 172.21.15.84:57988) 2023-10-16T11:25:46.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.471+0000 7fe4cdcba640 1 -- 172.21.15.84:0/3005074535 learned_addr learned my addr 172.21.15.84:0/3005074535 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:46.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.471+0000 7fe4cccb8640 1 --2- 172.21.15.84:0/3005074535 >> v2:172.21.15.157:3300/0 conn(0x7fe4c80ad970 0x7fe4c815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:46.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.471+0000 7fe4cd4b9640 1 --2- 172.21.15.84:0/3005074535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4c806c040 0x7fe4c806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:46.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.471+0000 7fe4bffff640 1 -- 172.21.15.84:0/3005074535 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2050074080 0 0) 0x7fe4c806c960 con 0x7fe4c815ebc0 2023-10-16T11:25:46.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.472+0000 7fe4bffff640 1 -- 172.21.15.84:0/3005074535 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4b4003520 con 0x7fe4c815ebc0 2023-10-16T11:25:46.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.472+0000 7fe4bffff640 1 -- 172.21.15.84:0/3005074535 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3527988482 0 0) 0x7fe4b4003520 con 0x7fe4c815ebc0 2023-10-16T11:25:46.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.472+0000 7fe4bffff640 1 -- 172.21.15.84:0/3005074535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4c806c040 msgr2=0x7fe4c806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:46.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.472+0000 7fe4bffff640 1 --2- 172.21.15.84:0/3005074535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4c806c040 0x7fe4c806c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:46.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.472+0000 7fe4bffff640 1 -- 172.21.15.84:0/3005074535 >> v2:172.21.15.157:3300/0 conn(0x7fe4c80ad970 msgr2=0x7fe4c815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:46.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.472+0000 7fe4bffff640 1 --2- 172.21.15.84:0/3005074535 >> v2:172.21.15.157:3300/0 conn(0x7fe4c80ad970 0x7fe4c815e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:46.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.472+0000 7fe4bffff640 1 -- 172.21.15.84:0/3005074535 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c815bf90 con 0x7fe4c815ebc0 2023-10-16T11:25:46.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.472+0000 7fe4bffff640 1 -- 172.21.15.84:0/3005074535 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fe4c00019f0 con 0x7fe4c815ebc0 2023-10-16T11:25:46.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.472+0000 7fe4bffff640 1 -- 172.21.15.84:0/3005074535 >> v1:172.21.15.84:6789/0 conn(0x7fe4c815ebc0 legacy=0x7fe4c80cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:46.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.472+0000 7fe4bffff640 1 --2- 172.21.15.84:0/3005074535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4b4003970 0x7fe4b4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:46.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.472+0000 7fe4bffff640 1 --2- 172.21.15.84:0/3005074535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4c806c040 0x7fe4b40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:46.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.472+0000 7fe4bffff640 1 --2- 172.21.15.84:0/3005074535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4b4004d00 0x7fe4b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:46.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.472+0000 7fe4bffff640 1 -- 172.21.15.84:0/3005074535 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c815b9d0 con 0x7fe4b4003970 2023-10-16T11:25:46.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.472+0000 7fe4bffff640 1 -- 172.21.15.84:0/3005074535 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c815bcb0 con 0x7fe4b4004d00 2023-10-16T11:25:46.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.472+0000 7fe4bffff640 1 -- 172.21.15.84:0/3005074535 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b400b980 con 0x7fe4c806c040 2023-10-16T11:25:46.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.472+0000 7fe4cd4b9640 1 --2- 172.21.15.84:0/3005074535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4b4003970 0x7fe4b4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:46.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.473+0000 7fe4cccb8640 1 --2- 172.21.15.84:0/3005074535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4c806c040 0x7fe4b40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:46.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.473+0000 7fe4cd4b9640 1 -- 172.21.15.84:0/3005074535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4c806c040 msgr2=0x7fe4b40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:46.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.473+0000 7fe4cd4b9640 1 --2- 172.21.15.84:0/3005074535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4c806c040 0x7fe4b40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:46.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.473+0000 7fe4cd4b9640 1 -- 172.21.15.84:0/3005074535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4b4004d00 msgr2=0x7fe4b4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:46.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.473+0000 7fe4cd4b9640 1 --2- 172.21.15.84:0/3005074535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4b4004d00 0x7fe4b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:46.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.473+0000 7fe4cd4b9640 1 -- 172.21.15.84:0/3005074535 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b4003520 con 0x7fe4b4003970 2023-10-16T11:25:46.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.474+0000 7fe4cd4b9640 1 --2- 172.21.15.84:0/3005074535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4b4003970 0x7fe4b4003d90 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009ac0 tx=0x7fe4b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37729ab6ac4d1a13 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:46.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.474+0000 7fe4bffff640 1 -- 172.21.15.84:0/3005074535 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c402c6b0 con 0x7fe4b4003970 2023-10-16T11:25:46.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.474+0000 7fe4bffff640 1 -- 172.21.15.84:0/3005074535 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c402c810 con 0x7fe4b4003970 2023-10-16T11:25:46.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.474+0000 7fe4bffff640 1 -- 172.21.15.84:0/3005074535 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c402cba0 con 0x7fe4b4003970 2023-10-16T11:25:46.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.475+0000 7fe4ce4bb640 1 -- 172.21.15.84:0/3005074535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4b4003970 msgr2=0x7fe4b4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:46.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.475+0000 7fe4ce4bb640 1 --2- 172.21.15.84:0/3005074535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4b4003970 0x7fe4b4003d90 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009ac0 tx=0x7fe4b40057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:46.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.475+0000 7fe4cdcba640 1 -- 172.21.15.84:0/3005074535 reap_dead start 2023-10-16T11:25:46.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.475+0000 7fe4ce4bb640 1 -- 172.21.15.84:0/3005074535 shutdown_connections 2023-10-16T11:25:46.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.475+0000 7fe4ce4bb640 1 -- 172.21.15.84:0/3005074535 >> 172.21.15.84:0/3005074535 conn(0x7fe4c80ce6a0 msgr2=0x7fe4c80ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:46.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.475+0000 7fe4ce4bb640 1 -- 172.21.15.84:0/3005074535 shutdown_connections 2023-10-16T11:25:46.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.475+0000 7fe4ce4bb640 1 -- 172.21.15.84:0/3005074535 wait complete. 2023-10-16T11:25:46.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.477+0000 7fe4ce4bb640 1 Processor -- start 2023-10-16T11:25:46.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.477+0000 7fe4ce4bb640 1 -- start start 2023-10-16T11:25:46.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.478+0000 7fe4ce4bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b4003970 0x7fe4c815ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:46.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.478+0000 7fe4ce4bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4c815a710 0x7fe4c80af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:46.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.478+0000 7fe4ce4bb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4c8159e80 0x7fe4c80ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:46.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.478+0000 7fe4ce4bb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c80af000 con 0x7fe4c8159e80 2023-10-16T11:25:46.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.478+0000 7fe4ce4bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c81e4690 con 0x7fe4c815a710 2023-10-16T11:25:46.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.478+0000 7fe4ce4bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c80af960 con 0x7fe4b4003970 2023-10-16T11:25:46.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.478+0000 7fe4cdcba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4c8159e80 0x7fe4c80ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:46.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.478+0000 7fe4cdcba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4c8159e80 0x7fe4c80ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34756/0 (socket says 172.21.15.84:34756) 2023-10-16T11:25:46.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.478+0000 7fe4cdcba640 1 -- 172.21.15.84:0/1942795870 learned_addr learned my addr 172.21.15.84:0/1942795870 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:46.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.478+0000 7fe4cccb8640 1 --2- 172.21.15.84:0/1942795870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4c815a710 0x7fe4c80af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:46.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.478+0000 7fe4cd4b9640 1 --2- 172.21.15.84:0/1942795870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b4003970 0x7fe4c815ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:46.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.479+0000 7fe4cdcba640 1 -- 172.21.15.84:0/1942795870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b4003970 msgr2=0x7fe4c815ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:46.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.479+0000 7fe4cdcba640 1 --2- 172.21.15.84:0/1942795870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b4003970 0x7fe4c815ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:46.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.479+0000 7fe4cdcba640 1 -- 172.21.15.84:0/1942795870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4c815a710 msgr2=0x7fe4c80af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:46.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.479+0000 7fe4cdcba640 1 --2- 172.21.15.84:0/1942795870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4c815a710 0x7fe4c80af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:46.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.479+0000 7fe4cdcba640 1 -- 172.21.15.84:0/1942795870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c80ae0a0 con 0x7fe4c8159e80 2023-10-16T11:25:46.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.479+0000 7fe4cd4b9640 1 --2- 172.21.15.84:0/1942795870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b4003970 0x7fe4c815ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:46.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.479+0000 7fe4cdcba640 1 --2- 172.21.15.84:0/1942795870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4c8159e80 0x7fe4c80ac890 secure :-1 s=READY pgs=2957 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00012e0 tx=0x7fe4c000ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:46.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.479+0000 7fe4bdffb640 1 -- 172.21.15.84:0/1942795870 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c000a610 con 0x7fe4c8159e80 2023-10-16T11:25:46.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.479+0000 7fe4bdffb640 1 -- 172.21.15.84:0/1942795870 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c000a770 con 0x7fe4c8159e80 2023-10-16T11:25:46.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.479+0000 7fe4bdffb640 1 -- 172.21.15.84:0/1942795870 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c000ab20 con 0x7fe4c8159e80 2023-10-16T11:25:46.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.479+0000 7fe4ce4bb640 1 -- 172.21.15.84:0/1942795870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c80ae380 con 0x7fe4c8159e80 2023-10-16T11:25:46.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.479+0000 7fe4ce4bb640 1 -- 172.21.15.84:0/1942795870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c80ae940 con 0x7fe4c8159e80 2023-10-16T11:25:46.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.480+0000 7fe4ce4bb640 1 -- 172.21.15.84:0/1942795870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c80af140 con 0x7fe4c8159e80 2023-10-16T11:25:46.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.481+0000 7fe4bdffb640 1 -- 172.21.15.84:0/1942795870 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe4c000ac80 con 0x7fe4c8159e80 2023-10-16T11:25:46.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.483+0000 7fe4bdffb640 1 --2- 172.21.15.84:0/1942795870 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe4ac06b0e0 0x7fe4ac06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:46.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.483+0000 7fe4bdffb640 1 -- 172.21.15.84:0/1942795870 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe4c0089380 con 0x7fe4c8159e80 2023-10-16T11:25:46.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.483+0000 7fe4cd4b9640 1 --2- 172.21.15.84:0/1942795870 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe4ac06b0e0 0x7fe4ac06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:46.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.484+0000 7fe4cd4b9640 1 --2- 172.21.15.84:0/1942795870 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe4ac06b0e0 0x7fe4ac06d5a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4036000 tx=0x7fe4c4030000 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:46.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.488+0000 7fe4bdffb640 1 -- 172.21.15.84:0/1942795870 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe4c80af140 con 0x7fe4c8159e80 2023-10-16T11:25:46.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.680+0000 7fe4ce4bb640 1 -- 172.21.15.84:0/1942795870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4c80af140 con 0x7fe4c8159e80 2023-10-16T11:25:46.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.683+0000 7fe4bdffb640 1 -- 172.21.15.84:0/1942795870 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v237) v1 ==== 74+0+16539 (secure 0 0 0) 0x7fe4c80af140 con 0x7fe4c8159e80 2023-10-16T11:25:46.685 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:46.686 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":237,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:25:38.157428+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:25:34.443897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:25:35.635615+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:25:46.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.686+0000 7fe4ce4bb640 1 -- 172.21.15.84:0/1942795870 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe4ac06b0e0 msgr2=0x7fe4ac06d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:46.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.686+0000 7fe4ce4bb640 1 --2- 172.21.15.84:0/1942795870 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe4ac06b0e0 0x7fe4ac06d5a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4036000 tx=0x7fe4c4030000 comp rx=0 tx=0).stop 2023-10-16T11:25:46.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.686+0000 7fe4ce4bb640 1 -- 172.21.15.84:0/1942795870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4c8159e80 msgr2=0x7fe4c80ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:46.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.686+0000 7fe4ce4bb640 1 --2- 172.21.15.84:0/1942795870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4c8159e80 0x7fe4c80ac890 secure :-1 s=READY pgs=2957 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00012e0 tx=0x7fe4c000ce80 comp rx=0 tx=0).stop 2023-10-16T11:25:46.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.686+0000 7fe4ce4bb640 1 -- 172.21.15.84:0/1942795870 shutdown_connections 2023-10-16T11:25:46.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.686+0000 7fe4ce4bb640 1 --2- 172.21.15.84:0/1942795870 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fe4ac06b0e0 0x7fe4ac06d5a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:46.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.686+0000 7fe4ce4bb640 1 --2- 172.21.15.84:0/1942795870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4c8159e80 0x7fe4c80ac890 unknown :-1 s=CLOSED pgs=2957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:46.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.686+0000 7fe4ce4bb640 1 --2- 172.21.15.84:0/1942795870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4c815a710 0x7fe4c80af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:46.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.686+0000 7fe4ce4bb640 1 --2- 172.21.15.84:0/1942795870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b4003970 0x7fe4c815ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:46.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.686+0000 7fe4ce4bb640 1 -- 172.21.15.84:0/1942795870 >> 172.21.15.84:0/1942795870 conn(0x7fe4c80ce6a0 msgr2=0x7fe4c80c9880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:46.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.686+0000 7fe4ce4bb640 1 -- 172.21.15.84:0/1942795870 shutdown_connections 2023-10-16T11:25:46.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.686+0000 7fe4ce4bb640 1 -- 172.21.15.84:0/1942795870 wait complete. 2023-10-16T11:25:46.708 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:25:34.443897+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '236', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:25:35.635615+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '237', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:25:46.709 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:25:46.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.923+0000 7eff70002640 1 Processor -- start 2023-10-16T11:25:46.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.923+0000 7eff70002640 1 -- start start 2023-10-16T11:25:46.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.925+0000 7eff70002640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7eff6806a4b0 0x7eff680ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:46.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.925+0000 7eff70002640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff680cb030 con 0x7eff680c9ca0 2023-10-16T11:25:46.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.925+0000 7eff70002640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff680c91c0 con 0x7eff6815ad70 2023-10-16T11:25:46.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.925+0000 7eff70002640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff680c94a0 con 0x7eff6806a4b0 2023-10-16T11:25:46.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.925+0000 7eff6e59f640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7eff680c9ca0 0x7eff680ad4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57996/0 (socket says 172.21.15.84:57996) 2023-10-16T11:25:46.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.925+0000 7eff6e59f640 1 -- 172.21.15.84:0/1583757663 learned_addr learned my addr 172.21.15.84:0/1583757663 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:46.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.925+0000 7eff6d59d640 1 --2- 172.21.15.84:0/1583757663 >> v2:172.21.15.157:3300/0 conn(0x7eff6806a4b0 0x7eff680ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:46.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.926+0000 7eff6cd9c640 1 -- 172.21.15.84:0/1583757663 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802553750 0 0) 0x7eff680cb030 con 0x7eff680c9ca0 2023-10-16T11:25:46.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.926+0000 7eff6cd9c640 1 -- 172.21.15.84:0/1583757663 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff54003520 con 0x7eff680c9ca0 2023-10-16T11:25:46.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.926+0000 7eff6d59d640 1 -- 172.21.15.84:0/1583757663 >> v1:172.21.15.157:6789/0 conn(0x7eff6815ad70 legacy=0x7eff68069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:46.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.926+0000 7eff6d59d640 1 -- 172.21.15.84:0/1583757663 >> v1:172.21.15.84:6789/0 conn(0x7eff680c9ca0 legacy=0x7eff680ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:46.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.926+0000 7eff6cd9c640 1 -- 172.21.15.84:0/1583757663 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2043496249 0 0) 0x7eff54003520 con 0x7eff680c9ca0 2023-10-16T11:25:46.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.926+0000 7eff6d59d640 1 -- 172.21.15.84:0/1583757663 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff680c9780 con 0x7eff6806a4b0 2023-10-16T11:25:46.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.927+0000 7eff6d59d640 1 --2- 172.21.15.84:0/1583757663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff6806a4b0 0x7eff680ca3b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7eff580097a0 tx=0x7eff5802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17f729fc67eeb2b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:46.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.928+0000 7eff6cd9c640 1 -- 172.21.15.84:0/1583757663 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5802f050 con 0x7eff6806a4b0 2023-10-16T11:25:46.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.928+0000 7eff6cd9c640 1 -- 172.21.15.84:0/1583757663 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff58031500 con 0x7eff6806a4b0 2023-10-16T11:25:46.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.928+0000 7eff6cd9c640 1 -- 172.21.15.84:0/1583757663 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60001670 con 0x7eff6806a4b0 2023-10-16T11:25:46.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.928+0000 7eff70002640 1 -- 172.21.15.84:0/1583757663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff6806a4b0 msgr2=0x7eff680ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:46.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.928+0000 7eff70002640 1 --2- 172.21.15.84:0/1583757663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff6806a4b0 0x7eff680ca3b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7eff580097a0 tx=0x7eff5802b820 comp rx=0 tx=0).stop 2023-10-16T11:25:46.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.928+0000 7eff70002640 1 -- 172.21.15.84:0/1583757663 shutdown_connections 2023-10-16T11:25:46.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.928+0000 7eff70002640 1 --2- 172.21.15.84:0/1583757663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff6806a4b0 0x7eff680ca3b0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:46.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.928+0000 7eff70002640 1 -- 172.21.15.84:0/1583757663 >> 172.21.15.84:0/1583757663 conn(0x7eff680ce9b0 msgr2=0x7eff680cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:46.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.929+0000 7eff70002640 1 -- 172.21.15.84:0/1583757663 shutdown_connections 2023-10-16T11:25:46.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.929+0000 7eff70002640 1 -- 172.21.15.84:0/1583757663 wait complete. 2023-10-16T11:25:46.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.931+0000 7eff70002640 1 Processor -- start 2023-10-16T11:25:46.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.931+0000 7eff70002640 1 -- start start 2023-10-16T11:25:46.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.932+0000 7eff70002640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff6806a4b0 0x7eff680ce120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:46.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.932+0000 7eff70002640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff680c9ca0 0x7eff681579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:46.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.932+0000 7eff70002640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff6815ad70 0x7eff680aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:46.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.932+0000 7eff70002640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff680ad900 con 0x7eff6806a4b0 2023-10-16T11:25:46.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.932+0000 7eff70002640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff68157f20 con 0x7eff680c9ca0 2023-10-16T11:25:46.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.932+0000 7eff70002640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff680ad340 con 0x7eff6815ad70 2023-10-16T11:25:46.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.932+0000 7eff6dd9e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff6806a4b0 0x7eff680ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:46.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.932+0000 7eff6dd9e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff6806a4b0 0x7eff680ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34758/0 (socket says 172.21.15.84:34758) 2023-10-16T11:25:46.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.932+0000 7eff6dd9e640 1 -- 172.21.15.84:0/1495370009 learned_addr learned my addr 172.21.15.84:0/1495370009 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:46.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.933+0000 7eff6dd9e640 1 -- 172.21.15.84:0/1495370009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff6815ad70 msgr2=0x7eff680aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:46.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.933+0000 7eff6dd9e640 1 --2- 172.21.15.84:0/1495370009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff6815ad70 0x7eff680aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:46.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.933+0000 7eff6dd9e640 1 -- 172.21.15.84:0/1495370009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff680c9ca0 msgr2=0x7eff681579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:46.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.933+0000 7eff6d59d640 1 --2- 172.21.15.84:0/1495370009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff680c9ca0 0x7eff681579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:46.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.933+0000 7eff6dd9e640 1 --2- 172.21.15.84:0/1495370009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff680c9ca0 0x7eff681579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:46.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.933+0000 7eff6dd9e640 1 -- 172.21.15.84:0/1495370009 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff58009400 con 0x7eff6806a4b0 2023-10-16T11:25:46.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.933+0000 7eff6dd9e640 1 --2- 172.21.15.84:0/1495370009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff6806a4b0 0x7eff680ce120 secure :-1 s=READY pgs=2959 cs=0 l=1 rev1=1 crypto rx=0x7eff6400c730 tx=0x7eff6400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:46.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.933+0000 7eff5e7fc640 1 -- 172.21.15.84:0/1495370009 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff64003b70 con 0x7eff6806a4b0 2023-10-16T11:25:46.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.933+0000 7eff5e7fc640 1 -- 172.21.15.84:0/1495370009 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff64003cf0 con 0x7eff6806a4b0 2023-10-16T11:25:46.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.933+0000 7eff5e7fc640 1 -- 172.21.15.84:0/1495370009 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff64003ea0 con 0x7eff6806a4b0 2023-10-16T11:25:46.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.933+0000 7eff70002640 1 -- 172.21.15.84:0/1495370009 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff680ad620 con 0x7eff6806a4b0 2023-10-16T11:25:46.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.934+0000 7eff70002640 1 -- 172.21.15.84:0/1495370009 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff680cdc00 con 0x7eff6806a4b0 2023-10-16T11:25:46.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.935+0000 7eff70002640 1 -- 172.21.15.84:0/1495370009 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6804e160 con 0x7eff6806a4b0 2023-10-16T11:25:46.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.936+0000 7eff5e7fc640 1 -- 172.21.15.84:0/1495370009 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7eff6400aa90 con 0x7eff6806a4b0 2023-10-16T11:25:46.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.937+0000 7eff5e7fc640 1 --2- 172.21.15.84:0/1495370009 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7eff4c06b0b0 0x7eff4c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:46.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.937+0000 7eff5e7fc640 1 -- 172.21.15.84:0/1495370009 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7eff640898a0 con 0x7eff6806a4b0 2023-10-16T11:25:46.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.937+0000 7eff6d59d640 1 --2- 172.21.15.84:0/1495370009 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7eff4c06b0b0 0x7eff4c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:46.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.938+0000 7eff6d59d640 1 --2- 172.21.15.84:0/1495370009 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7eff4c06b0b0 0x7eff4c06d570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7eff58035000 tx=0x7eff58008030 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:46.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:46.941+0000 7eff5e7fc640 1 -- 172.21.15.84:0/1495370009 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7eff640545c0 con 0x7eff6806a4b0 2023-10-16T11:25:47.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.131+0000 7eff70002640 1 -- 172.21.15.84:0/1495370009 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7eff6806c040 con 0x7eff6806a4b0 2023-10-16T11:25:47.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.132+0000 7eff5e7fc640 1 -- 172.21.15.84:0/1495370009 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v237) v1 ==== 93+0+10 (secure 0 0 0) 0x7eff64057be0 con 0x7eff6806a4b0 2023-10-16T11:25:47.134 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:25:47.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.134+0000 7eff70002640 1 -- 172.21.15.84:0/1495370009 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7eff4c06b0b0 msgr2=0x7eff4c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:47.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.134+0000 7eff70002640 1 --2- 172.21.15.84:0/1495370009 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7eff4c06b0b0 0x7eff4c06d570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7eff58035000 tx=0x7eff58008030 comp rx=0 tx=0).stop 2023-10-16T11:25:47.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.134+0000 7eff70002640 1 -- 172.21.15.84:0/1495370009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff6806a4b0 msgr2=0x7eff680ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:47.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.134+0000 7eff70002640 1 --2- 172.21.15.84:0/1495370009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff6806a4b0 0x7eff680ce120 secure :-1 s=READY pgs=2959 cs=0 l=1 rev1=1 crypto rx=0x7eff6400c730 tx=0x7eff6400cbf0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.134+0000 7eff70002640 1 -- 172.21.15.84:0/1495370009 shutdown_connections 2023-10-16T11:25:47.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.134+0000 7eff70002640 1 --2- 172.21.15.84:0/1495370009 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7eff4c06b0b0 0x7eff4c06d570 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.134+0000 7eff70002640 1 --2- 172.21.15.84:0/1495370009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff6815ad70 0x7eff680aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.134+0000 7eff70002640 1 --2- 172.21.15.84:0/1495370009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff680c9ca0 0x7eff681579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.134+0000 7eff70002640 1 --2- 172.21.15.84:0/1495370009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff6806a4b0 0x7eff680ce120 unknown :-1 s=CLOSED pgs=2959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.134+0000 7eff70002640 1 -- 172.21.15.84:0/1495370009 >> 172.21.15.84:0/1495370009 conn(0x7eff680ce9b0 msgr2=0x7eff68069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:47.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.135+0000 7eff70002640 1 -- 172.21.15.84:0/1495370009 shutdown_connections 2023-10-16T11:25:47.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.135+0000 7eff70002640 1 -- 172.21.15.84:0/1495370009 wait complete. 2023-10-16T11:25:47.155 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:25:47.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.367+0000 7f11ce147640 1 Processor -- start 2023-10-16T11:25:47.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.367+0000 7f11ce147640 1 -- start start 2023-10-16T11:25:47.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.369+0000 7f11ce147640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11c806b660 0x7f11c806ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:47.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.369+0000 7f11ce147640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f11c81428c0 0x7f11c80c9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:47.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.369+0000 7f11ce147640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c80cc6b0 con 0x7f11c80c9690 2023-10-16T11:25:47.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.369+0000 7f11ce147640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c80c9b20 con 0x7f11c81428c0 2023-10-16T11:25:47.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.369+0000 7f11ce147640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c81599b0 con 0x7f11c806b660 2023-10-16T11:25:47.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.369+0000 7f11bffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f11c81428c0 0x7f11c80c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:47.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.369+0000 7f11bffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f11c81428c0 0x7f11c80c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34764/0 (socket says 172.21.15.84:34764) 2023-10-16T11:25:47.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.369+0000 7f11bffff640 1 -- 172.21.15.84:0/233317761 learned_addr learned my addr 172.21.15.84:0/233317761 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:47.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.369+0000 7f11c7fff640 1 --1- 172.21.15.84:0/233317761 >> v1:172.21.15.84:6789/0 conn(0x7f11c80c9690 0x7f11c815d080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58002/0 (socket says 172.21.15.84:58002) 2023-10-16T11:25:47.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.369+0000 7f11c77fe640 1 --2- 172.21.15.84:0/233317761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11c806b660 0x7f11c806ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:47.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.370+0000 7f11bffff640 1 -- 172.21.15.84:0/233317761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11c806b660 msgr2=0x7f11c806ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:47.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.370+0000 7f11bffff640 1 --2- 172.21.15.84:0/233317761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11c806b660 0x7f11c806ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.370+0000 7f11bffff640 1 -- 172.21.15.84:0/233317761 >> v1:172.21.15.84:6789/0 conn(0x7f11c80c9690 legacy=0x7f11c815d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:47.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.370+0000 7f11bffff640 1 -- 172.21.15.84:0/233317761 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c806c290 con 0x7f11c81428c0 2023-10-16T11:25:47.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.370+0000 7f11c6ffd640 1 -- 172.21.15.84:0/233317761 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3222127546 0 0) 0x7f11c80cc6b0 con 0x7f11c80c9690 2023-10-16T11:25:47.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.371+0000 7f11bffff640 1 --2- 172.21.15.84:0/233317761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c81428c0 0x7f11c80c9150 secure :-1 s=READY pgs=2961 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009870 tx=0x7f11b002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693f72c0e411a2fe server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:47.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.372+0000 7f11c6ffd640 1 -- 172.21.15.84:0/233317761 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b0030840 con 0x7f11c81428c0 2023-10-16T11:25:47.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.372+0000 7f11c6ffd640 1 -- 172.21.15.84:0/233317761 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b00309a0 con 0x7f11c81428c0 2023-10-16T11:25:47.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.372+0000 7f11c6ffd640 1 -- 172.21.15.84:0/233317761 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b0030d30 con 0x7f11c81428c0 2023-10-16T11:25:47.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.372+0000 7f11ce147640 1 -- 172.21.15.84:0/233317761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c81428c0 msgr2=0x7f11c80c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:47.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.372+0000 7f11ce147640 1 --2- 172.21.15.84:0/233317761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c81428c0 0x7f11c80c9150 secure :-1 s=READY pgs=2961 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009870 tx=0x7f11b002b980 comp rx=0 tx=0).stop 2023-10-16T11:25:47.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.372+0000 7f11ce147640 1 -- 172.21.15.84:0/233317761 shutdown_connections 2023-10-16T11:25:47.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.372+0000 7f11ce147640 1 --2- 172.21.15.84:0/233317761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c81428c0 0x7f11c80c9150 unknown :-1 s=CLOSED pgs=2961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.372+0000 7f11ce147640 1 --2- 172.21.15.84:0/233317761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11c806b660 0x7f11c806ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.372+0000 7f11ce147640 1 -- 172.21.15.84:0/233317761 >> 172.21.15.84:0/233317761 conn(0x7f11c80ce9b0 msgr2=0x7f11c80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:47.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.373+0000 7f11ce147640 1 -- 172.21.15.84:0/233317761 shutdown_connections 2023-10-16T11:25:47.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.373+0000 7f11ce147640 1 -- 172.21.15.84:0/233317761 wait complete. 2023-10-16T11:25:47.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.374+0000 7f11ce147640 1 Processor -- start 2023-10-16T11:25:47.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.374+0000 7f11ce147640 1 -- start start 2023-10-16T11:25:47.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.375+0000 7f11ce147640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c806b660 0x7f11c80c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:47.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.375+0000 7f11ce147640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11c80c9690 0x7f11c80c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:47.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.375+0000 7f11ce147640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11c81428c0 0x7f11c80c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:47.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.375+0000 7f11ce147640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80c7880 con 0x7f11c806b660 2023-10-16T11:25:47.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.375+0000 7f11ce147640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80c8d00 con 0x7f11c80c9690 2023-10-16T11:25:47.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.375+0000 7f11ce147640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80c6c00 con 0x7f11c81428c0 2023-10-16T11:25:47.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.375+0000 7f11c77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c806b660 0x7f11c80c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:47.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.376+0000 7f11c77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c806b660 0x7f11c80c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34772/0 (socket says 172.21.15.84:34772) 2023-10-16T11:25:47.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.376+0000 7f11c77fe640 1 -- 172.21.15.84:0/3093658138 learned_addr learned my addr 172.21.15.84:0/3093658138 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:47.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.376+0000 7f11bffff640 1 --2- 172.21.15.84:0/3093658138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11c80c9690 0x7f11c80c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:47.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.376+0000 7f11c7fff640 1 --2- 172.21.15.84:0/3093658138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11c81428c0 0x7f11c80c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:47.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.376+0000 7f11c77fe640 1 -- 172.21.15.84:0/3093658138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11c81428c0 msgr2=0x7f11c80c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:47.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.376+0000 7f11c77fe640 1 --2- 172.21.15.84:0/3093658138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11c81428c0 0x7f11c80c43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.376+0000 7f11c77fe640 1 -- 172.21.15.84:0/3093658138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11c80c9690 msgr2=0x7f11c80c3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:47.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.376+0000 7f11c77fe640 1 --2- 172.21.15.84:0/3093658138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11c80c9690 0x7f11c80c3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.376+0000 7f11c77fe640 1 -- 172.21.15.84:0/3093658138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b00094d0 con 0x7f11c806b660 2023-10-16T11:25:47.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.376+0000 7f11bffff640 1 --2- 172.21.15.84:0/3093658138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11c80c9690 0x7f11c80c3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:47.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.376+0000 7f11c77fe640 1 --2- 172.21.15.84:0/3093658138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c806b660 0x7f11c80c4fe0 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 crypto rx=0x7f11b800d5f0 tx=0x7f11b800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:47.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.377+0000 7f11c4ff9640 1 -- 172.21.15.84:0/3093658138 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b80125f0 con 0x7f11c806b660 2023-10-16T11:25:47.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.377+0000 7f11c4ff9640 1 -- 172.21.15.84:0/3093658138 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b8012750 con 0x7f11c806b660 2023-10-16T11:25:47.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.377+0000 7f11c4ff9640 1 -- 172.21.15.84:0/3093658138 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b8012b00 con 0x7f11c806b660 2023-10-16T11:25:47.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.377+0000 7f11ce147640 1 -- 172.21.15.84:0/3093658138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c80c6ee0 con 0x7f11c806b660 2023-10-16T11:25:47.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.377+0000 7f11ce147640 1 -- 172.21.15.84:0/3093658138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c80c74a0 con 0x7f11c806b660 2023-10-16T11:25:47.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.378+0000 7f11ce147640 1 -- 172.21.15.84:0/3093658138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c804e160 con 0x7f11c806b660 2023-10-16T11:25:47.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.379+0000 7f11c4ff9640 1 -- 172.21.15.84:0/3093658138 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f11b8012c60 con 0x7f11c806b660 2023-10-16T11:25:47.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.380+0000 7f11c4ff9640 1 --2- 172.21.15.84:0/3093658138 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f11a806b0b0 0x7f11a806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:47.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.381+0000 7f11c4ff9640 1 -- 172.21.15.84:0/3093658138 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f11b8089f40 con 0x7f11c806b660 2023-10-16T11:25:47.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.381+0000 7f11bffff640 1 --2- 172.21.15.84:0/3093658138 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f11a806b0b0 0x7f11a806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:47.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.381+0000 7f11bffff640 1 --2- 172.21.15.84:0/3093658138 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f11a806b0b0 0x7f11a806d570 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f11b0036000 tx=0x7f11b003b040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:47.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.386+0000 7f11c4ff9640 1 -- 172.21.15.84:0/3093658138 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f11b8054c60 con 0x7f11c806b660 2023-10-16T11:25:47.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.578+0000 7f11ce147640 1 -- 172.21.15.84:0/3093658138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f11c80c8e70 con 0x7f11c806b660 2023-10-16T11:25:47.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.579+0000 7f11c4ff9640 1 -- 172.21.15.84:0/3093658138 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v237) v1 ==== 104+0+11 (secure 0 0 0) 0x7f11b8058280 con 0x7f11c806b660 2023-10-16T11:25:47.581 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:25:47.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.582+0000 7f11ce147640 1 -- 172.21.15.84:0/3093658138 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f11a806b0b0 msgr2=0x7f11a806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:47.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.582+0000 7f11ce147640 1 --2- 172.21.15.84:0/3093658138 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f11a806b0b0 0x7f11a806d570 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f11b0036000 tx=0x7f11b003b040 comp rx=0 tx=0).stop 2023-10-16T11:25:47.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.582+0000 7f11ce147640 1 -- 172.21.15.84:0/3093658138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c806b660 msgr2=0x7f11c80c4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:47.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.582+0000 7f11ce147640 1 --2- 172.21.15.84:0/3093658138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c806b660 0x7f11c80c4fe0 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 crypto rx=0x7f11b800d5f0 tx=0x7f11b800dab0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.582+0000 7f11ce147640 1 -- 172.21.15.84:0/3093658138 shutdown_connections 2023-10-16T11:25:47.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.582+0000 7f11ce147640 1 --2- 172.21.15.84:0/3093658138 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f11a806b0b0 0x7f11a806d570 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.582+0000 7f11ce147640 1 --2- 172.21.15.84:0/3093658138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11c81428c0 0x7f11c80c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.582+0000 7f11ce147640 1 --2- 172.21.15.84:0/3093658138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11c80c9690 0x7f11c80c3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.582+0000 7f11ce147640 1 --2- 172.21.15.84:0/3093658138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c806b660 0x7f11c80c4fe0 unknown :-1 s=CLOSED pgs=2962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.582+0000 7f11ce147640 1 -- 172.21.15.84:0/3093658138 >> 172.21.15.84:0/3093658138 conn(0x7f11c80ce9b0 msgr2=0x7f11c80ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:47.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.582+0000 7f11ce147640 1 -- 172.21.15.84:0/3093658138 shutdown_connections 2023-10-16T11:25:47.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.582+0000 7f11ce147640 1 -- 172.21.15.84:0/3093658138 wait complete. 2023-10-16T11:25:47.603 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:25:47.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.814+0000 7f1b099ce640 1 Processor -- start 2023-10-16T11:25:47.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.814+0000 7f1b099ce640 1 -- start start 2023-10-16T11:25:47.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.816+0000 7f1b099ce640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1b04159ab0 0x7f1b040ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:47.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.816+0000 7f1b099ce640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1b04159e90 0x7f1b0406fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:47.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.816+0000 7f1b099ce640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b0406c140 0x7f1b0406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:47.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.816+0000 7f1b099ce640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0406cb10 con 0x7f1b04159ab0 2023-10-16T11:25:47.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.816+0000 7f1b099ce640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0415d890 con 0x7f1b04159e90 2023-10-16T11:25:47.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.816+0000 7f1b099ce640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b040cd1f0 con 0x7f1b0406c140 2023-10-16T11:25:47.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.816+0000 7f1b02ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1b04159ab0 0x7f1b040ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:47.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.816+0000 7f1b02ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1b04159ab0 0x7f1b040ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34778/0 (socket says 172.21.15.84:34778) 2023-10-16T11:25:47.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.816+0000 7f1b02ffd640 1 -- 172.21.15.84:0/2386217362 learned_addr learned my addr 172.21.15.84:0/2386217362 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:47.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.817+0000 7f1afbfff640 1 --2- 172.21.15.84:0/2386217362 >> v2:172.21.15.157:3300/0 conn(0x7f1b04159e90 0x7f1b0406fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:47.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.817+0000 7f1b037fe640 1 --2- 172.21.15.84:0/2386217362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b0406c140 0x7f1b0406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:47.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.817+0000 7f1b02ffd640 1 -- 172.21.15.84:0/2386217362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b0406c140 msgr2=0x7f1b0406c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:47.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.817+0000 7f1b02ffd640 1 --2- 172.21.15.84:0/2386217362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b0406c140 0x7f1b0406c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.817+0000 7f1b02ffd640 1 -- 172.21.15.84:0/2386217362 >> v2:172.21.15.157:3300/0 conn(0x7f1b04159e90 msgr2=0x7f1b0406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:47.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.817+0000 7f1b02ffd640 1 --2- 172.21.15.84:0/2386217362 >> v2:172.21.15.157:3300/0 conn(0x7f1b04159e90 0x7f1b0406fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.817+0000 7f1b02ffd640 1 -- 172.21.15.84:0/2386217362 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b040cd4d0 con 0x7f1b04159ab0 2023-10-16T11:25:47.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.817+0000 7f1afbfff640 1 --2- 172.21.15.84:0/2386217362 >> v2:172.21.15.157:3300/0 conn(0x7f1b04159e90 0x7f1b0406fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:47.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.817+0000 7f1b037fe640 1 --2- 172.21.15.84:0/2386217362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b0406c140 0x7f1b0406c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:47.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.818+0000 7f1b02ffd640 1 --2- 172.21.15.84:0/2386217362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b04159ab0 0x7f1b040ce370 secure :-1 s=READY pgs=2963 cs=0 l=1 rev1=1 crypto rx=0x7f1b0406d900 tx=0x7f1af402dfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83dc0ff3b9f0079 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:47.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.818+0000 7f1b027fc640 1 -- 172.21.15.84:0/2386217362 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af402ed60 con 0x7f1b04159ab0 2023-10-16T11:25:47.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.818+0000 7f1b027fc640 1 -- 172.21.15.84:0/2386217362 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af4036440 con 0x7f1b04159ab0 2023-10-16T11:25:47.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.818+0000 7f1b027fc640 1 -- 172.21.15.84:0/2386217362 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af40367d0 con 0x7f1b04159ab0 2023-10-16T11:25:47.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.819+0000 7f1b099ce640 1 -- 172.21.15.84:0/2386217362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b04159ab0 msgr2=0x7f1b040ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:47.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.819+0000 7f1b099ce640 1 --2- 172.21.15.84:0/2386217362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b04159ab0 0x7f1b040ce370 secure :-1 s=READY pgs=2963 cs=0 l=1 rev1=1 crypto rx=0x7f1b0406d900 tx=0x7f1af402dfb0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.819+0000 7f1b099ce640 1 -- 172.21.15.84:0/2386217362 shutdown_connections 2023-10-16T11:25:47.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.819+0000 7f1b099ce640 1 --2- 172.21.15.84:0/2386217362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b0406c140 0x7f1b0406c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.819+0000 7f1b099ce640 1 --2- 172.21.15.84:0/2386217362 >> v2:172.21.15.157:3300/0 conn(0x7f1b04159e90 0x7f1b0406fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.819+0000 7f1b099ce640 1 --2- 172.21.15.84:0/2386217362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b04159ab0 0x7f1b040ce370 unknown :-1 s=CLOSED pgs=2963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.819+0000 7f1b099ce640 1 -- 172.21.15.84:0/2386217362 >> 172.21.15.84:0/2386217362 conn(0x7f1b040ce9b0 msgr2=0x7f1b040cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:47.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.819+0000 7f1b099ce640 1 -- 172.21.15.84:0/2386217362 shutdown_connections 2023-10-16T11:25:47.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.819+0000 7f1b099ce640 1 -- 172.21.15.84:0/2386217362 wait complete. 2023-10-16T11:25:47.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.821+0000 7f1b099ce640 1 Processor -- start 2023-10-16T11:25:47.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.821+0000 7f1b099ce640 1 -- start start 2023-10-16T11:25:47.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.822+0000 7f1b099ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b0406c140 0x7f1b040ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:47.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.822+0000 7f1b099ce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b04159ab0 0x7f1b04158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:47.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.822+0000 7f1b099ce640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b04159e90 0x7f1b040c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:47.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.822+0000 7f1b099ce640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b040c5330 con 0x7f1b0406c140 2023-10-16T11:25:47.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.822+0000 7f1b099ce640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b041585b0 con 0x7f1b04159ab0 2023-10-16T11:25:47.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.822+0000 7f1b099ce640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b040c4480 con 0x7f1b04159e90 2023-10-16T11:25:47.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.822+0000 7f1b02ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b0406c140 0x7f1b040ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:47.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.822+0000 7f1b02ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b0406c140 0x7f1b040ad1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34790/0 (socket says 172.21.15.84:34790) 2023-10-16T11:25:47.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.822+0000 7f1b02ffd640 1 -- 172.21.15.84:0/2951880073 learned_addr learned my addr 172.21.15.84:0/2951880073 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:47.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.823+0000 7f1b02ffd640 1 -- 172.21.15.84:0/2951880073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b04159e90 msgr2=0x7f1b040c3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:47.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.823+0000 7f1b02ffd640 1 --2- 172.21.15.84:0/2951880073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b04159e90 0x7f1b040c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.823+0000 7f1b02ffd640 1 -- 172.21.15.84:0/2951880073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b04159ab0 msgr2=0x7f1b04158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:47.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.823+0000 7f1b02ffd640 1 --2- 172.21.15.84:0/2951880073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b04159ab0 0x7f1b04158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:47.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.823+0000 7f1b02ffd640 1 -- 172.21.15.84:0/2951880073 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af400b770 con 0x7f1b0406c140 2023-10-16T11:25:47.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.823+0000 7f1b02ffd640 1 --2- 172.21.15.84:0/2951880073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b0406c140 0x7f1b040ad1d0 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 crypto rx=0x7f1af402dff0 tx=0x7f1af40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:47.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.823+0000 7f1b00ff9640 1 -- 172.21.15.84:0/2951880073 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af403f750 con 0x7f1b0406c140 2023-10-16T11:25:47.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.823+0000 7f1b00ff9640 1 -- 172.21.15.84:0/2951880073 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af403f8d0 con 0x7f1b0406c140 2023-10-16T11:25:47.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.823+0000 7f1b00ff9640 1 -- 172.21.15.84:0/2951880073 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af403fc80 con 0x7f1b0406c140 2023-10-16T11:25:47.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.823+0000 7f1b099ce640 1 -- 172.21.15.84:0/2951880073 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b040c4720 con 0x7f1b0406c140 2023-10-16T11:25:47.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.823+0000 7f1b099ce640 1 -- 172.21.15.84:0/2951880073 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b040c4ce0 con 0x7f1b0406c140 2023-10-16T11:25:47.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.825+0000 7f1b099ce640 1 -- 172.21.15.84:0/2951880073 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b0404e160 con 0x7f1b0406c140 2023-10-16T11:25:47.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.826+0000 7f1b00ff9640 1 -- 172.21.15.84:0/2951880073 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1af40403e0 con 0x7f1b0406c140 2023-10-16T11:25:47.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.827+0000 7f1b00ff9640 1 --2- 172.21.15.84:0/2951880073 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f1ae406b0b0 0x7f1ae406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:47.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.827+0000 7f1b00ff9640 1 -- 172.21.15.84:0/2951880073 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1af40abf30 con 0x7f1b0406c140 2023-10-16T11:25:47.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.827+0000 7f1afbfff640 1 --2- 172.21.15.84:0/2951880073 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f1ae406b0b0 0x7f1ae406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:47.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.828+0000 7f1afbfff640 1 --2- 172.21.15.84:0/2951880073 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f1ae406b0b0 0x7f1ae406d570 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1aec006fd0 tx=0x7f1aec008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:47.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:47.832+0000 7f1b00ff9640 1 -- 172.21.15.84:0/2951880073 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1af4076bc0 con 0x7f1b0406c140 2023-10-16T11:25:48.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.027+0000 7f1b099ce640 1 -- 172.21.15.84:0/2951880073 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b040cee90 con 0x7f1b0406c140 2023-10-16T11:25:48.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.027+0000 7f1b00ff9640 1 -- 172.21.15.84:0/2951880073 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v237) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1af40460a0 con 0x7f1b0406c140 2023-10-16T11:25:48.029 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:25:48.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.030+0000 7f1b099ce640 1 -- 172.21.15.84:0/2951880073 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f1ae406b0b0 msgr2=0x7f1ae406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:48.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.030+0000 7f1b099ce640 1 --2- 172.21.15.84:0/2951880073 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f1ae406b0b0 0x7f1ae406d570 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1aec006fd0 tx=0x7f1aec008040 comp rx=0 tx=0).stop 2023-10-16T11:25:48.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.030+0000 7f1b099ce640 1 -- 172.21.15.84:0/2951880073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b0406c140 msgr2=0x7f1b040ad1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:48.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.030+0000 7f1b099ce640 1 --2- 172.21.15.84:0/2951880073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b0406c140 0x7f1b040ad1d0 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 crypto rx=0x7f1af402dff0 tx=0x7f1af40046c0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.030+0000 7f1b099ce640 1 -- 172.21.15.84:0/2951880073 shutdown_connections 2023-10-16T11:25:48.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.030+0000 7f1b099ce640 1 --2- 172.21.15.84:0/2951880073 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f1ae406b0b0 0x7f1ae406d570 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.030+0000 7f1b099ce640 1 --2- 172.21.15.84:0/2951880073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b04159e90 0x7f1b040c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.030+0000 7f1b099ce640 1 --2- 172.21.15.84:0/2951880073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b04159ab0 0x7f1b04158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.030+0000 7f1b099ce640 1 --2- 172.21.15.84:0/2951880073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b0406c140 0x7f1b040ad1d0 unknown :-1 s=CLOSED pgs=2964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.030+0000 7f1b099ce640 1 -- 172.21.15.84:0/2951880073 >> 172.21.15.84:0/2951880073 conn(0x7f1b040ce9b0 msgr2=0x7f1b040ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:48.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.031+0000 7f1b099ce640 1 -- 172.21.15.84:0/2951880073 shutdown_connections 2023-10-16T11:25:48.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.031+0000 7f1b099ce640 1 -- 172.21.15.84:0/2951880073 wait complete. 2023-10-16T11:25:48.052 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T11:25:48.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.275+0000 7fda07547640 1 Processor -- start 2023-10-16T11:25:48.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.276+0000 7fda07547640 1 -- start start 2023-10-16T11:25:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.277+0000 7fda07547640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda0015aaa0 0x7fda0015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.277+0000 7fda07547640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fda000ce390 0x7fda00070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.277+0000 7fda07547640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda0015ed60 con 0x7fda0015ae80 2023-10-16T11:25:48.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.277+0000 7fda07547640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda0006b810 con 0x7fda000ce390 2023-10-16T11:25:48.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.277+0000 7fda07547640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda000707e0 con 0x7fda0015aaa0 2023-10-16T11:25:48.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.278+0000 7fda04ae2640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fda0015ae80 0x7fda000cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58008/0 (socket says 172.21.15.84:58008) 2023-10-16T11:25:48.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.278+0000 7fda04ae2640 1 -- 172.21.15.84:0/526800146 learned_addr learned my addr 172.21.15.84:0/526800146 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:48.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.278+0000 7fda05ae4640 1 --2- 172.21.15.84:0/526800146 >> v2:172.21.15.84:3300/0 conn(0x7fda000ce390 0x7fda00070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:48.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.278+0000 7fda052e3640 1 --2- 172.21.15.84:0/526800146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda0015aaa0 0x7fda0015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:48.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.278+0000 7fd9fffff640 1 -- 172.21.15.84:0/526800146 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1644101122 0 0) 0x7fda0015ed60 con 0x7fda0015ae80 2023-10-16T11:25:48.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.278+0000 7fd9fffff640 1 -- 172.21.15.84:0/526800146 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9e8003520 con 0x7fda0015ae80 2023-10-16T11:25:48.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.278+0000 7fda05ae4640 1 -- 172.21.15.84:0/526800146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda0015aaa0 msgr2=0x7fda0015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:48.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.278+0000 7fda05ae4640 1 --2- 172.21.15.84:0/526800146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda0015aaa0 0x7fda0015e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.278+0000 7fda05ae4640 1 -- 172.21.15.84:0/526800146 >> v1:172.21.15.84:6789/0 conn(0x7fda0015ae80 legacy=0x7fda000cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:48.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.278+0000 7fda05ae4640 1 -- 172.21.15.84:0/526800146 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0015ef90 con 0x7fda000ce390 2023-10-16T11:25:48.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.278+0000 7fda052e3640 1 --2- 172.21.15.84:0/526800146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda0015aaa0 0x7fda0015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:25:48.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.279+0000 7fda05ae4640 1 --2- 172.21.15.84:0/526800146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda000ce390 0x7fda00070f50 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80097a0 tx=0x7fd9f802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1125961db0aed044 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:48.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.280+0000 7fd9fffff640 1 -- 172.21.15.84:0/526800146 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f802d040 con 0x7fda000ce390 2023-10-16T11:25:48.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.280+0000 7fd9fffff640 1 -- 172.21.15.84:0/526800146 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f8031810 con 0x7fda000ce390 2023-10-16T11:25:48.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.280+0000 7fd9fffff640 1 -- 172.21.15.84:0/526800146 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8031ba0 con 0x7fda000ce390 2023-10-16T11:25:48.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.280+0000 7fda07547640 1 -- 172.21.15.84:0/526800146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda000ce390 msgr2=0x7fda00070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:48.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.280+0000 7fda07547640 1 --2- 172.21.15.84:0/526800146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda000ce390 0x7fda00070f50 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80097a0 tx=0x7fd9f802b8b0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.280+0000 7fda07547640 1 -- 172.21.15.84:0/526800146 shutdown_connections 2023-10-16T11:25:48.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.280+0000 7fda07547640 1 --2- 172.21.15.84:0/526800146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda000ce390 0x7fda00070f50 unknown :-1 s=CLOSED pgs=2966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.280+0000 7fda07547640 1 --2- 172.21.15.84:0/526800146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda0015aaa0 0x7fda0015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.280+0000 7fda07547640 1 -- 172.21.15.84:0/526800146 >> 172.21.15.84:0/526800146 conn(0x7fda000cb410 msgr2=0x7fda000cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:48.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.280+0000 7fda07547640 1 -- 172.21.15.84:0/526800146 shutdown_connections 2023-10-16T11:25:48.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.280+0000 7fda07547640 1 -- 172.21.15.84:0/526800146 wait complete. 2023-10-16T11:25:48.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.282+0000 7fda07547640 1 Processor -- start 2023-10-16T11:25:48.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.282+0000 7fda07547640 1 -- start start 2023-10-16T11:25:48.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.283+0000 7fda07547640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda000ce390 0x7fda000c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:48.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.283+0000 7fda07547640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda0015aaa0 0x7fda000c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:48.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.283+0000 7fda07547640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda0015ae80 0x7fda000c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:48.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.283+0000 7fda07547640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda000c6d70 con 0x7fda0015ae80 2023-10-16T11:25:48.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.283+0000 7fda07547640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0014f410 con 0x7fda000ce390 2023-10-16T11:25:48.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.283+0000 7fda07547640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda000c4b60 con 0x7fda0015aaa0 2023-10-16T11:25:48.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.283+0000 7fda05ae4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda0015ae80 0x7fda000c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:48.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.283+0000 7fda05ae4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda0015ae80 0x7fda000c5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34800/0 (socket says 172.21.15.84:34800) 2023-10-16T11:25:48.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.283+0000 7fda05ae4640 1 -- 172.21.15.84:0/2636790711 learned_addr learned my addr 172.21.15.84:0/2636790711 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:48.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.284+0000 7fda04ae2640 1 --2- 172.21.15.84:0/2636790711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda0015aaa0 0x7fda000c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:48.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.284+0000 7fda052e3640 1 --2- 172.21.15.84:0/2636790711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda000ce390 0x7fda000c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:48.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.284+0000 7fda05ae4640 1 -- 172.21.15.84:0/2636790711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda0015aaa0 msgr2=0x7fda000c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:48.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.284+0000 7fda05ae4640 1 --2- 172.21.15.84:0/2636790711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda0015aaa0 0x7fda000c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.284+0000 7fda05ae4640 1 -- 172.21.15.84:0/2636790711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda000ce390 msgr2=0x7fda000c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:48.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.284+0000 7fda05ae4640 1 --2- 172.21.15.84:0/2636790711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda000ce390 0x7fda000c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.284+0000 7fda05ae4640 1 -- 172.21.15.84:0/2636790711 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f8009400 con 0x7fda0015ae80 2023-10-16T11:25:48.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.284+0000 7fda04ae2640 1 --2- 172.21.15.84:0/2636790711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda0015aaa0 0x7fda000c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:48.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.284+0000 7fda05ae4640 1 --2- 172.21.15.84:0/2636790711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda0015ae80 0x7fda000c5fa0 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80097a0 tx=0x7fd9f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:48.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.285+0000 7fd9fdffb640 1 -- 172.21.15.84:0/2636790711 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8006e10 con 0x7fda0015ae80 2023-10-16T11:25:48.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.285+0000 7fd9fdffb640 1 -- 172.21.15.84:0/2636790711 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f803ace0 con 0x7fda0015ae80 2023-10-16T11:25:48.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.285+0000 7fda07547640 1 -- 172.21.15.84:0/2636790711 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda000c42c0 con 0x7fda0015ae80 2023-10-16T11:25:48.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.285+0000 7fd9fdffb640 1 -- 172.21.15.84:0/2636790711 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8044640 con 0x7fda0015ae80 2023-10-16T11:25:48.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.285+0000 7fda07547640 1 -- 172.21.15.84:0/2636790711 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda000c4880 con 0x7fda0015ae80 2023-10-16T11:25:48.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.286+0000 7fda07547640 1 -- 172.21.15.84:0/2636790711 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda0004e160 con 0x7fda0015ae80 2023-10-16T11:25:48.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.287+0000 7fd9fdffb640 1 -- 172.21.15.84:0/2636790711 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd9f804d440 con 0x7fda0015ae80 2023-10-16T11:25:48.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.288+0000 7fd9fdffb640 1 --2- 172.21.15.84:0/2636790711 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd9ec06b0b0 0x7fd9ec06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:48.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.288+0000 7fd9fdffb640 1 -- 172.21.15.84:0/2636790711 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd9f80abd30 con 0x7fda0015ae80 2023-10-16T11:25:48.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.288+0000 7fda052e3640 1 --2- 172.21.15.84:0/2636790711 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd9ec06b0b0 0x7fd9ec06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:48.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.289+0000 7fda052e3640 1 --2- 172.21.15.84:0/2636790711 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd9ec06b0b0 0x7fd9ec06d570 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0003fd0 tx=0x7fd9f00039f0 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:48.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.292+0000 7fd9fdffb640 1 -- 172.21.15.84:0/2636790711 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd9f80769e0 con 0x7fda0015ae80 2023-10-16T11:25:48.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.491+0000 7fda07547640 1 -- 172.21.15.84:0/2636790711 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fda000cdc60 con 0x7fda0015ae80 2023-10-16T11:25:48.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.492+0000 7fd9fdffb640 1 -- 172.21.15.84:0/2636790711 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v237) v1 ==== 84+0+209 (secure 0 0 0) 0x7fd9f80350f0 con 0x7fda0015ae80 2023-10-16T11:25:48.494 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:48.495 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T11:25:48.495 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:48.495 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-16T11:25:48.495 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-16T11:25:48.495 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:48.496 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:48.496 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 30, 2023-10-16T11:25:48.496 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": "cephfs_metadata" 2023-10-16T11:25:48.496 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:25:48.497 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:25:48.497 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 31, 2023-10-16T11:25:48.497 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": "cephfs_data" 2023-10-16T11:25:48.497 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:25:48.497 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T11:25:48.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.495+0000 7fda07547640 1 -- 172.21.15.84:0/2636790711 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd9ec06b0b0 msgr2=0x7fd9ec06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:48.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.495+0000 7fda07547640 1 --2- 172.21.15.84:0/2636790711 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd9ec06b0b0 0x7fd9ec06d570 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0003fd0 tx=0x7fd9f00039f0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.495+0000 7fda07547640 1 -- 172.21.15.84:0/2636790711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda0015ae80 msgr2=0x7fda000c5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:48.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.495+0000 7fda07547640 1 --2- 172.21.15.84:0/2636790711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda0015ae80 0x7fda000c5fa0 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80097a0 tx=0x7fd9f80046c0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.495+0000 7fda07547640 1 -- 172.21.15.84:0/2636790711 shutdown_connections 2023-10-16T11:25:48.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.495+0000 7fda07547640 1 --2- 172.21.15.84:0/2636790711 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd9ec06b0b0 0x7fd9ec06d570 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.495+0000 7fda07547640 1 --2- 172.21.15.84:0/2636790711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda0015ae80 0x7fda000c5fa0 unknown :-1 s=CLOSED pgs=2967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.495+0000 7fda07547640 1 --2- 172.21.15.84:0/2636790711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda0015aaa0 0x7fda000c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.495+0000 7fda07547640 1 --2- 172.21.15.84:0/2636790711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda000ce390 0x7fda000c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.495+0000 7fda07547640 1 -- 172.21.15.84:0/2636790711 >> 172.21.15.84:0/2636790711 conn(0x7fda000cb410 msgr2=0x7fda000b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:48.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.495+0000 7fda07547640 1 -- 172.21.15.84:0/2636790711 shutdown_connections 2023-10-16T11:25:48.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.495+0000 7fda07547640 1 -- 172.21.15.84:0/2636790711 wait complete. 2023-10-16T11:25:48.516 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2023-10-16T11:25:48.516 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2023-10-16T11:25:48.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.733+0000 7f585f6cf640 1 Processor -- start 2023-10-16T11:25:48.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.733+0000 7f585f6cf640 1 -- start start 2023-10-16T11:25:48.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.734+0000 7f585f6cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f58581599c0 0x7f58580c9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:48.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.734+0000 7f585f6cf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5858159da0 0x7f585815d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:48.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.734+0000 7f585f6cf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585815dd10 con 0x7f58580cce20 2023-10-16T11:25:48.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.734+0000 7f585f6cf640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f585815dfb0 con 0x7f58581599c0 2023-10-16T11:25:48.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.734+0000 7f585f6cf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585815e290 con 0x7f5858159da0 2023-10-16T11:25:48.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.734+0000 7f585d46b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f58581599c0 0x7f58580c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:48.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.734+0000 7f585d46b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f58581599c0 0x7f58580c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34806/0 (socket says 172.21.15.84:34806) 2023-10-16T11:25:48.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.734+0000 7f585d46b640 1 -- 172.21.15.84:0/1644567284 learned_addr learned my addr 172.21.15.84:0/1644567284 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:48.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.735+0000 7f585cc6a640 1 --2- 172.21.15.84:0/1644567284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5858159da0 0x7f585815d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:48.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.735+0000 7f585d46b640 1 -- 172.21.15.84:0/1644567284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5858159da0 msgr2=0x7f585815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:48.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.735+0000 7f585d46b640 1 --2- 172.21.15.84:0/1644567284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5858159da0 0x7f585815d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.735+0000 7f585d46b640 1 -- 172.21.15.84:0/1644567284 >> v1:172.21.15.157:6789/0 conn(0x7f58580cce20 legacy=0x7f585806c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:48.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.735+0000 7f585d46b640 1 -- 172.21.15.84:0/1644567284 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585815e570 con 0x7f58581599c0 2023-10-16T11:25:48.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.736+0000 7f585d46b640 1 --2- 172.21.15.84:0/1644567284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58581599c0 0x7f58580c9300 secure :-1 s=READY pgs=2968 cs=0 l=1 rev1=1 crypto rx=0x7f5854009870 tx=0x7f585402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e9fa782ecece03 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:48.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.736+0000 7f584ffff640 1 -- 172.21.15.84:0/1644567284 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5854031640 con 0x7f58581599c0 2023-10-16T11:25:48.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.736+0000 7f584ffff640 1 -- 172.21.15.84:0/1644567284 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58540317a0 con 0x7f58581599c0 2023-10-16T11:25:48.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.736+0000 7f584ffff640 1 -- 172.21.15.84:0/1644567284 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5854031b30 con 0x7f58581599c0 2023-10-16T11:25:48.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.737+0000 7f585f6cf640 1 -- 172.21.15.84:0/1644567284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58581599c0 msgr2=0x7f58580c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:48.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.737+0000 7f585f6cf640 1 --2- 172.21.15.84:0/1644567284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58581599c0 0x7f58580c9300 secure :-1 s=READY pgs=2968 cs=0 l=1 rev1=1 crypto rx=0x7f5854009870 tx=0x7f585402b8f0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.737+0000 7f585f6cf640 1 -- 172.21.15.84:0/1644567284 shutdown_connections 2023-10-16T11:25:48.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.737+0000 7f585f6cf640 1 --2- 172.21.15.84:0/1644567284 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5858159da0 0x7f585815d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.737+0000 7f585f6cf640 1 --2- 172.21.15.84:0/1644567284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58581599c0 0x7f58580c9300 unknown :-1 s=CLOSED pgs=2968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.737+0000 7f585f6cf640 1 -- 172.21.15.84:0/1644567284 >> 172.21.15.84:0/1644567284 conn(0x7f58580cb360 msgr2=0x7f58580cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:48.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.737+0000 7f585f6cf640 1 -- 172.21.15.84:0/1644567284 shutdown_connections 2023-10-16T11:25:48.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.737+0000 7f585f6cf640 1 -- 172.21.15.84:0/1644567284 wait complete. 2023-10-16T11:25:48.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.739+0000 7f585f6cf640 1 Processor -- start 2023-10-16T11:25:48.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.739+0000 7f585f6cf640 1 -- start start 2023-10-16T11:25:48.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.740+0000 7f585f6cf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58580cce20 0x7f58580b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:48.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.740+0000 7f585f6cf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58581599c0 0x7f585814df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:48.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.740+0000 7f585f6cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5858159da0 0x7f5858150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:48.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.740+0000 7f585f6cf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58580c2a10 con 0x7f5858159da0 2023-10-16T11:25:48.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.740+0000 7f585f6cf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58580c2670 con 0x7f58581599c0 2023-10-16T11:25:48.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.740+0000 7f585f6cf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58580ae4e0 con 0x7f58580cce20 2023-10-16T11:25:48.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.740+0000 7f585dc6c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5858159da0 0x7f5858150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:48.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.740+0000 7f585dc6c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5858159da0 0x7f5858150ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34812/0 (socket says 172.21.15.84:34812) 2023-10-16T11:25:48.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.740+0000 7f585dc6c640 1 -- 172.21.15.84:0/660226573 learned_addr learned my addr 172.21.15.84:0/660226573 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:48.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.741+0000 7f585dc6c640 1 -- 172.21.15.84:0/660226573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58580cce20 msgr2=0x7f58580b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:48.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.741+0000 7f585d46b640 1 --2- 172.21.15.84:0/660226573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58580cce20 0x7f58580b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:48.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.741+0000 7f585cc6a640 1 --2- 172.21.15.84:0/660226573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58581599c0 0x7f585814df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:48.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.741+0000 7f585dc6c640 1 --2- 172.21.15.84:0/660226573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58580cce20 0x7f58580b0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.741+0000 7f585dc6c640 1 -- 172.21.15.84:0/660226573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58581599c0 msgr2=0x7f585814df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:48.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.741+0000 7f585dc6c640 1 --2- 172.21.15.84:0/660226573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58581599c0 0x7f585814df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:48.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.741+0000 7f585dc6c640 1 -- 172.21.15.84:0/660226573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58540094d0 con 0x7f5858159da0 2023-10-16T11:25:48.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.741+0000 7f585cc6a640 1 --2- 172.21.15.84:0/660226573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58581599c0 0x7f585814df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:48.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.741+0000 7f585dc6c640 1 --2- 172.21.15.84:0/660226573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5858159da0 0x7f5858150ae0 secure :-1 s=READY pgs=2969 cs=0 l=1 rev1=1 crypto rx=0x7f585000c780 tx=0x7f585000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:48.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.741+0000 7f584dffb640 1 -- 172.21.15.84:0/660226573 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850004b60 con 0x7f5858159da0 2023-10-16T11:25:48.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.741+0000 7f584dffb640 1 -- 172.21.15.84:0/660226573 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585000ce90 con 0x7f5858159da0 2023-10-16T11:25:48.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.741+0000 7f584dffb640 1 -- 172.21.15.84:0/660226573 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585000a850 con 0x7f5858159da0 2023-10-16T11:25:48.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.741+0000 7f585f6cf640 1 -- 172.21.15.84:0/660226573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58580ae7c0 con 0x7f5858159da0 2023-10-16T11:25:48.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.741+0000 7f585f6cf640 1 -- 172.21.15.84:0/660226573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f585814e4d0 con 0x7f5858159da0 2023-10-16T11:25:48.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.743+0000 7f585f6cf640 1 -- 172.21.15.84:0/660226573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58580c2b50 con 0x7f5858159da0 2023-10-16T11:25:48.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.744+0000 7f584dffb640 1 -- 172.21.15.84:0/660226573 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f585000a9b0 con 0x7f5858159da0 2023-10-16T11:25:48.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.745+0000 7f584dffb640 1 --2- 172.21.15.84:0/660226573 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f583c06b0b0 0x7f583c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:48.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.745+0000 7f584dffb640 1 -- 172.21.15.84:0/660226573 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5850089c10 con 0x7f5858159da0 2023-10-16T11:25:48.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.745+0000 7f585d46b640 1 --2- 172.21.15.84:0/660226573 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f583c06b0b0 0x7f583c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:48.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.745+0000 7f585d46b640 1 --2- 172.21.15.84:0/660226573 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f583c06b0b0 0x7f583c06d570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5854036000 tx=0x7f585403c040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:48.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.749+0000 7f584dffb640 1 -- 172.21.15.84:0/660226573 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f58580c2b50 con 0x7f5858159da0 2023-10-16T11:25:48.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:48.938+0000 7f585f6cf640 1 -- 172.21.15.84:0/660226573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f58580c2b50 con 0x7f5858159da0 2023-10-16T11:25:49.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.259+0000 7f584dffb640 1 -- 172.21.15.84:0/660226573 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v238) v1 ==== 144+0+0 (secure 0 0 0) 0x7f58580c2b50 con 0x7f5858159da0 2023-10-16T11:25:49.262 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'backup_fs_metadata' created 2023-10-16T11:25:49.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.264+0000 7f585f6cf640 1 -- 172.21.15.84:0/660226573 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f583c06b0b0 msgr2=0x7f583c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:49.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.264+0000 7f585f6cf640 1 --2- 172.21.15.84:0/660226573 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f583c06b0b0 0x7f583c06d570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5854036000 tx=0x7f585403c040 comp rx=0 tx=0).stop 2023-10-16T11:25:49.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.264+0000 7f585f6cf640 1 -- 172.21.15.84:0/660226573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5858159da0 msgr2=0x7f5858150ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:49.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.264+0000 7f585f6cf640 1 --2- 172.21.15.84:0/660226573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5858159da0 0x7f5858150ae0 secure :-1 s=READY pgs=2969 cs=0 l=1 rev1=1 crypto rx=0x7f585000c780 tx=0x7f585000cc40 comp rx=0 tx=0).stop 2023-10-16T11:25:49.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.264+0000 7f585f6cf640 1 -- 172.21.15.84:0/660226573 shutdown_connections 2023-10-16T11:25:49.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.264+0000 7f585f6cf640 1 --2- 172.21.15.84:0/660226573 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f583c06b0b0 0x7f583c06d570 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:49.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.264+0000 7f585f6cf640 1 --2- 172.21.15.84:0/660226573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5858159da0 0x7f5858150ae0 unknown :-1 s=CLOSED pgs=2969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:49.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.264+0000 7f585f6cf640 1 --2- 172.21.15.84:0/660226573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58581599c0 0x7f585814df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:49.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.264+0000 7f585f6cf640 1 --2- 172.21.15.84:0/660226573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58580cce20 0x7f58580b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:49.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.264+0000 7f585f6cf640 1 -- 172.21.15.84:0/660226573 >> 172.21.15.84:0/660226573 conn(0x7f58580cb360 msgr2=0x7f58580cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:49.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.266+0000 7f585f6cf640 1 -- 172.21.15.84:0/660226573 shutdown_connections 2023-10-16T11:25:49.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.267+0000 7f585f6cf640 1 -- 172.21.15.84:0/660226573 wait complete. 2023-10-16T11:25:49.294 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T11:25:49.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.492+0000 7f9b29407640 1 Processor -- start 2023-10-16T11:25:49.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.492+0000 7f9b29407640 1 -- start start 2023-10-16T11:25:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.494+0000 7f9b29407640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b24159f00 0x7f9b240b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.494+0000 7f9b29407640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9b2415d8c0 0x7f9b240b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.494+0000 7f9b29407640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b2406f890 con 0x7f9b24159b20 2023-10-16T11:25:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.494+0000 7f9b29407640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b24070d30 con 0x7f9b2415d8c0 2023-10-16T11:25:49.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.494+0000 7f9b29407640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2406c650 con 0x7f9b24159f00 2023-10-16T11:25:49.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.494+0000 7f9b22ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9b24159b20 0x7f9b2406bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58016/0 (socket says 172.21.15.84:58016) 2023-10-16T11:25:49.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.494+0000 7f9b22ffd640 1 -- 172.21.15.84:0/2374842961 learned_addr learned my addr 172.21.15.84:0/2374842961 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:49.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.494+0000 7f9b237fe640 1 --2- 172.21.15.84:0/2374842961 >> v2:172.21.15.84:3300/0 conn(0x7f9b2415d8c0 0x7f9b240b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:49.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.495+0000 7f9b1bfff640 1 --2- 172.21.15.84:0/2374842961 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b24159f00 0x7f9b240b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:49.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.495+0000 7f9b237fe640 1 -- 172.21.15.84:0/2374842961 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b24159f00 msgr2=0x7f9b240b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:49.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.495+0000 7f9b237fe640 1 --2- 172.21.15.84:0/2374842961 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b24159f00 0x7f9b240b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:49.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.495+0000 7f9b237fe640 1 -- 172.21.15.84:0/2374842961 >> v1:172.21.15.84:6789/0 conn(0x7f9b24159b20 legacy=0x7f9b2406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:49.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.495+0000 7f9b227fc640 1 -- 172.21.15.84:0/2374842961 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058589051 0 0) 0x7f9b2406f890 con 0x7f9b24159b20 2023-10-16T11:25:49.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.495+0000 7f9b237fe640 1 -- 172.21.15.84:0/2374842961 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2406c930 con 0x7f9b2415d8c0 2023-10-16T11:25:49.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.496+0000 7f9b237fe640 1 --2- 172.21.15.84:0/2374842961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b2415d8c0 0x7f9b240b0950 secure :-1 s=READY pgs=2971 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009d90 tx=0x7f9b1c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3086f28e9521bdc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:49.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.496+0000 7f9b227fc640 1 -- 172.21.15.84:0/2374842961 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c02cc50 con 0x7f9b2415d8c0 2023-10-16T11:25:49.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.496+0000 7f9b227fc640 1 -- 172.21.15.84:0/2374842961 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1c02cdb0 con 0x7f9b2415d8c0 2023-10-16T11:25:49.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.496+0000 7f9b227fc640 1 -- 172.21.15.84:0/2374842961 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c034660 con 0x7f9b2415d8c0 2023-10-16T11:25:49.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.497+0000 7f9b29407640 1 -- 172.21.15.84:0/2374842961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b2415d8c0 msgr2=0x7f9b240b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:49.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.497+0000 7f9b29407640 1 --2- 172.21.15.84:0/2374842961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b2415d8c0 0x7f9b240b0950 secure :-1 s=READY pgs=2971 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009d90 tx=0x7f9b1c02bea0 comp rx=0 tx=0).stop 2023-10-16T11:25:49.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.497+0000 7f9b29407640 1 -- 172.21.15.84:0/2374842961 shutdown_connections 2023-10-16T11:25:49.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.497+0000 7f9b29407640 1 --2- 172.21.15.84:0/2374842961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b2415d8c0 0x7f9b240b0950 unknown :-1 s=CLOSED pgs=2971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:49.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.497+0000 7f9b29407640 1 --2- 172.21.15.84:0/2374842961 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b24159f00 0x7f9b240b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:49.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.497+0000 7f9b29407640 1 -- 172.21.15.84:0/2374842961 >> 172.21.15.84:0/2374842961 conn(0x7f9b240cb410 msgr2=0x7f9b240cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:49.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.497+0000 7f9b29407640 1 -- 172.21.15.84:0/2374842961 shutdown_connections 2023-10-16T11:25:49.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.497+0000 7f9b29407640 1 -- 172.21.15.84:0/2374842961 wait complete. 2023-10-16T11:25:49.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.499+0000 7f9b29407640 1 Processor -- start 2023-10-16T11:25:49.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.499+0000 7f9b29407640 1 -- start start 2023-10-16T11:25:49.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.500+0000 7f9b29407640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b24159b20 0x7f9b240b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:49.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.500+0000 7f9b29407640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b24159f00 0x7f9b240c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:49.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.500+0000 7f9b29407640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b2415d8c0 0x7f9b240b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:49.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.500+0000 7f9b29407640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b240c6650 con 0x7f9b24159b20 2023-10-16T11:25:49.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.500+0000 7f9b29407640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b240c5a60 con 0x7f9b2415d8c0 2023-10-16T11:25:49.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.500+0000 7f9b29407640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b240c5cb0 con 0x7f9b24159f00 2023-10-16T11:25:49.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.500+0000 7f9b22ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b24159b20 0x7f9b240b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:49.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.500+0000 7f9b22ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b24159b20 0x7f9b240b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34832/0 (socket says 172.21.15.84:34832) 2023-10-16T11:25:49.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.500+0000 7f9b22ffd640 1 -- 172.21.15.84:0/2952643777 learned_addr learned my addr 172.21.15.84:0/2952643777 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:49.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.500+0000 7f9b237fe640 1 --2- 172.21.15.84:0/2952643777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b2415d8c0 0x7f9b240b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:49.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.500+0000 7f9b1bfff640 1 --2- 172.21.15.84:0/2952643777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b24159f00 0x7f9b240c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:49.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.500+0000 7f9b22ffd640 1 -- 172.21.15.84:0/2952643777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b24159f00 msgr2=0x7f9b240c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:49.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.500+0000 7f9b22ffd640 1 --2- 172.21.15.84:0/2952643777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b24159f00 0x7f9b240c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:49.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.500+0000 7f9b22ffd640 1 -- 172.21.15.84:0/2952643777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b2415d8c0 msgr2=0x7f9b240b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:49.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.500+0000 7f9b22ffd640 1 --2- 172.21.15.84:0/2952643777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b2415d8c0 0x7f9b240b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:49.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.500+0000 7f9b22ffd640 1 -- 172.21.15.84:0/2952643777 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c009a70 con 0x7f9b24159b20 2023-10-16T11:25:49.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.501+0000 7f9b22ffd640 1 --2- 172.21.15.84:0/2952643777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b24159b20 0x7f9b240b1d70 secure :-1 s=READY pgs=2972 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400c500 tx=0x7f9b1400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:49.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.501+0000 7f9b1b7fe640 1 -- 172.21.15.84:0/2952643777 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b14007560 con 0x7f9b24159b20 2023-10-16T11:25:49.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.501+0000 7f9b1b7fe640 1 -- 172.21.15.84:0/2952643777 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b140047e0 con 0x7f9b24159b20 2023-10-16T11:25:49.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.501+0000 7f9b29407640 1 -- 172.21.15.84:0/2952643777 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b240c5f90 con 0x7f9b24159b20 2023-10-16T11:25:49.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.501+0000 7f9b1b7fe640 1 -- 172.21.15.84:0/2952643777 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b14004b90 con 0x7f9b24159b20 2023-10-16T11:25:49.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.501+0000 7f9b29407640 1 -- 172.21.15.84:0/2952643777 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b240b22b0 con 0x7f9b24159b20 2023-10-16T11:25:49.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.502+0000 7f9b29407640 1 -- 172.21.15.84:0/2952643777 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b2404e160 con 0x7f9b24159b20 2023-10-16T11:25:49.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.504+0000 7f9b1b7fe640 1 -- 172.21.15.84:0/2952643777 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f9b14004cf0 con 0x7f9b24159b20 2023-10-16T11:25:49.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.504+0000 7f9b1b7fe640 1 --2- 172.21.15.84:0/2952643777 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f9b0406b0b0 0x7f9b0406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:49.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.505+0000 7f9b1b7fe640 1 -- 172.21.15.84:0/2952643777 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 6110+0+0 (secure 0 0 0) 0x7f9b14088da0 con 0x7f9b24159b20 2023-10-16T11:25:49.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.505+0000 7f9b1bfff640 1 --2- 172.21.15.84:0/2952643777 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f9b0406b0b0 0x7f9b0406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:49.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.505+0000 7f9b1bfff640 1 --2- 172.21.15.84:0/2952643777 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f9b0406b0b0 0x7f9b0406d570 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9b240b05e0 tx=0x7f9b0c008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:49.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.509+0000 7f9b1b7fe640 1 -- 172.21.15.84:0/2952643777 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9b14053950 con 0x7f9b24159b20 2023-10-16T11:25:49.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:49.703+0000 7f9b29407640 1 -- 172.21.15.84:0/2952643777 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9b2406cbe0 con 0x7f9b24159b20 2023-10-16T11:25:50.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.265+0000 7f9b1b7fe640 1 -- 172.21.15.84:0/2952643777 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v239) v1 ==== 176+0+0 (secure 0 0 0) 0x7f9b14018020 con 0x7f9b24159b20 2023-10-16T11:25:50.268 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'backup_fs_data' created 2023-10-16T11:25:50.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.271+0000 7f9b29407640 1 -- 172.21.15.84:0/2952643777 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f9b0406b0b0 msgr2=0x7f9b0406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:50.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.271+0000 7f9b29407640 1 --2- 172.21.15.84:0/2952643777 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f9b0406b0b0 0x7f9b0406d570 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9b240b05e0 tx=0x7f9b0c008040 comp rx=0 tx=0).stop 2023-10-16T11:25:50.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.271+0000 7f9b29407640 1 -- 172.21.15.84:0/2952643777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b24159b20 msgr2=0x7f9b240b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:50.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.271+0000 7f9b29407640 1 --2- 172.21.15.84:0/2952643777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b24159b20 0x7f9b240b1d70 secure :-1 s=READY pgs=2972 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400c500 tx=0x7f9b1400c9c0 comp rx=0 tx=0).stop 2023-10-16T11:25:50.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.275+0000 7f9b29407640 1 -- 172.21.15.84:0/2952643777 shutdown_connections 2023-10-16T11:25:50.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.275+0000 7f9b29407640 1 --2- 172.21.15.84:0/2952643777 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f9b0406b0b0 0x7f9b0406d570 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:50.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.275+0000 7f9b29407640 1 --2- 172.21.15.84:0/2952643777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b2415d8c0 0x7f9b240b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:50.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.275+0000 7f9b29407640 1 --2- 172.21.15.84:0/2952643777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b24159f00 0x7f9b240c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:50.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.275+0000 7f9b29407640 1 --2- 172.21.15.84:0/2952643777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b24159b20 0x7f9b240b1d70 unknown :-1 s=CLOSED pgs=2972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:50.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.275+0000 7f9b29407640 1 -- 172.21.15.84:0/2952643777 >> 172.21.15.84:0/2952643777 conn(0x7f9b240cb410 msgr2=0x7f9b240ca860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:50.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.275+0000 7f9b29407640 1 -- 172.21.15.84:0/2952643777 shutdown_connections 2023-10-16T11:25:50.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.275+0000 7f9b29407640 1 -- 172.21.15.84:0/2952643777 wait complete. 2023-10-16T11:25:50.311 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2023-10-16T11:25:50.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.497+0000 7fbe8dbfb640 1 Processor -- start 2023-10-16T11:25:50.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.498+0000 7fbe8dbfb640 1 -- start start 2023-10-16T11:25:50.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.499+0000 7fbe8dbfb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbe8815ad70 0x7fbe88069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:50.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.499+0000 7fbe8dbfb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe880c9ca0 0x7fbe8806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:50.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.499+0000 7fbe8dbfb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe880cb030 con 0x7fbe880ca3b0 2023-10-16T11:25:50.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.499+0000 7fbe8dbfb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe880c91c0 con 0x7fbe8815ad70 2023-10-16T11:25:50.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.499+0000 7fbe8dbfb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe880c94a0 con 0x7fbe880c9ca0 2023-10-16T11:25:50.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.500+0000 7fbe877fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbe8815ad70 0x7fbe88069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:50.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.500+0000 7fbe877fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbe8815ad70 0x7fbe88069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34846/0 (socket says 172.21.15.84:34846) 2023-10-16T11:25:50.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.500+0000 7fbe877fe640 1 -- 172.21.15.84:0/501136441 learned_addr learned my addr 172.21.15.84:0/501136441 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.500+0000 7fbe7effd640 1 --2- 172.21.15.84:0/501136441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe880c9ca0 0x7fbe8806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.500+0000 7fbe877fe640 1 -- 172.21.15.84:0/501136441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe880c9ca0 msgr2=0x7fbe8806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.500+0000 7fbe877fe640 1 --2- 172.21.15.84:0/501136441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe880c9ca0 0x7fbe8806a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.500+0000 7fbe877fe640 1 -- 172.21.15.84:0/501136441 >> v1:172.21.15.157:6789/0 conn(0x7fbe880ca3b0 legacy=0x7fbe880ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.500+0000 7fbe877fe640 1 -- 172.21.15.84:0/501136441 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe880c9780 con 0x7fbe8815ad70 2023-10-16T11:25:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.500+0000 7fbe7effd640 1 --2- 172.21.15.84:0/501136441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe880c9ca0 0x7fbe8806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.501+0000 7fbe877fe640 1 --2- 172.21.15.84:0/501136441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe8815ad70 0x7fbe88069da0 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009e50 tx=0x7fbe7802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53aa63826061f65 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.502+0000 7fbe86ffd640 1 -- 172.21.15.84:0/501136441 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7802ca80 con 0x7fbe8815ad70 2023-10-16T11:25:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.502+0000 7fbe86ffd640 1 -- 172.21.15.84:0/501136441 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7802cbe0 con 0x7fbe8815ad70 2023-10-16T11:25:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.502+0000 7fbe86ffd640 1 -- 172.21.15.84:0/501136441 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe780343f0 con 0x7fbe8815ad70 2023-10-16T11:25:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.502+0000 7fbe8dbfb640 1 -- 172.21.15.84:0/501136441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe8815ad70 msgr2=0x7fbe88069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.502+0000 7fbe8dbfb640 1 --2- 172.21.15.84:0/501136441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe8815ad70 0x7fbe88069da0 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009e50 tx=0x7fbe7802bee0 comp rx=0 tx=0).stop 2023-10-16T11:25:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.502+0000 7fbe8dbfb640 1 -- 172.21.15.84:0/501136441 shutdown_connections 2023-10-16T11:25:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.502+0000 7fbe8dbfb640 1 --2- 172.21.15.84:0/501136441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe880c9ca0 0x7fbe8806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.502+0000 7fbe8dbfb640 1 --2- 172.21.15.84:0/501136441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe8815ad70 0x7fbe88069da0 unknown :-1 s=CLOSED pgs=2973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.502+0000 7fbe8dbfb640 1 -- 172.21.15.84:0/501136441 >> 172.21.15.84:0/501136441 conn(0x7fbe880ce9b0 msgr2=0x7fbe880cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.503+0000 7fbe8dbfb640 1 -- 172.21.15.84:0/501136441 shutdown_connections 2023-10-16T11:25:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.503+0000 7fbe8dbfb640 1 -- 172.21.15.84:0/501136441 wait complete. 2023-10-16T11:25:50.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.504+0000 7fbe8dbfb640 1 Processor -- start 2023-10-16T11:25:50.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.504+0000 7fbe8dbfb640 1 -- start start 2023-10-16T11:25:50.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.505+0000 7fbe8dbfb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe880c9ca0 0x7fbe8814bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:50.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.505+0000 7fbe8dbfb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe880ca3b0 0x7fbe88069da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:50.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.505+0000 7fbe8dbfb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe8815ad70 0x7fbe8806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:50.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.505+0000 7fbe8dbfb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe88157570 con 0x7fbe880ca3b0 2023-10-16T11:25:50.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.505+0000 7fbe8dbfb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe88157850 con 0x7fbe880c9ca0 2023-10-16T11:25:50.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.505+0000 7fbe8dbfb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe88157b30 con 0x7fbe8815ad70 2023-10-16T11:25:50.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.506+0000 7fbe7effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe880ca3b0 0x7fbe88069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:50.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.506+0000 7fbe7effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe880ca3b0 0x7fbe88069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34858/0 (socket says 172.21.15.84:34858) 2023-10-16T11:25:50.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.506+0000 7fbe7effd640 1 -- 172.21.15.84:0/927380790 learned_addr learned my addr 172.21.15.84:0/927380790 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:50.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.506+0000 7fbe877fe640 1 --2- 172.21.15.84:0/927380790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe880c9ca0 0x7fbe8814bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:50.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.506+0000 7fbe87fff640 1 --2- 172.21.15.84:0/927380790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe8815ad70 0x7fbe8806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:50.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.506+0000 7fbe7effd640 1 -- 172.21.15.84:0/927380790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe8815ad70 msgr2=0x7fbe8806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:50.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.506+0000 7fbe7effd640 1 --2- 172.21.15.84:0/927380790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe8815ad70 0x7fbe8806a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:50.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.506+0000 7fbe7effd640 1 -- 172.21.15.84:0/927380790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe880c9ca0 msgr2=0x7fbe8814bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:50.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.506+0000 7fbe7effd640 1 --2- 172.21.15.84:0/927380790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe880c9ca0 0x7fbe8814bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:50.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.506+0000 7fbe7effd640 1 -- 172.21.15.84:0/927380790 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe78009b30 con 0x7fbe880ca3b0 2023-10-16T11:25:50.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.506+0000 7fbe87fff640 1 --2- 172.21.15.84:0/927380790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe8815ad70 0x7fbe8806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:50.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.506+0000 7fbe7effd640 1 --2- 172.21.15.84:0/927380790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe880ca3b0 0x7fbe88069da0 secure :-1 s=READY pgs=2974 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000c7f0 tx=0x7fbe7000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:50.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.507+0000 7fbe84ff9640 1 -- 172.21.15.84:0/927380790 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70007a30 con 0x7fbe880ca3b0 2023-10-16T11:25:50.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.507+0000 7fbe84ff9640 1 -- 172.21.15.84:0/927380790 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe70007b90 con 0x7fbe880ca3b0 2023-10-16T11:25:50.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.507+0000 7fbe84ff9640 1 -- 172.21.15.84:0/927380790 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7000ce40 con 0x7fbe880ca3b0 2023-10-16T11:25:50.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.507+0000 7fbe8dbfb640 1 -- 172.21.15.84:0/927380790 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe88157e10 con 0x7fbe880ca3b0 2023-10-16T11:25:50.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.507+0000 7fbe8dbfb640 1 -- 172.21.15.84:0/927380790 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe88158330 con 0x7fbe880ca3b0 2023-10-16T11:25:50.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.508+0000 7fbe8dbfb640 1 -- 172.21.15.84:0/927380790 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8804e160 con 0x7fbe880ca3b0 2023-10-16T11:25:50.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.509+0000 7fbe84ff9640 1 -- 172.21.15.84:0/927380790 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fbe700113f0 con 0x7fbe880ca3b0 2023-10-16T11:25:50.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.510+0000 7fbe84ff9640 1 --2- 172.21.15.84:0/927380790 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fbe6806b0b0 0x7fbe6806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:50.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.510+0000 7fbe84ff9640 1 -- 172.21.15.84:0/927380790 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6495+0+0 (secure 0 0 0) 0x7fbe70089fb0 con 0x7fbe880ca3b0 2023-10-16T11:25:50.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.510+0000 7fbe877fe640 1 --2- 172.21.15.84:0/927380790 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fbe6806b0b0 0x7fbe6806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:50.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.511+0000 7fbe877fe640 1 --2- 172.21.15.84:0/927380790 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fbe6806b0b0 0x7fbe6806d570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe78035000 tx=0x7fbe78030000 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:50.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.516+0000 7fbe84ff9640 1 -- 172.21.15.84:0/927380790 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbe700bcfe0 con 0x7fbe880ca3b0 2023-10-16T11:25:50.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:50.736+0000 7fbe8dbfb640 1 -- 172.21.15.84:0/927380790 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) v1 -- 0x7fbe881576b0 con 0x7fbe880ca3b0 2023-10-16T11:25:51.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.310+0000 7fbe84ff9640 1 -- 172.21.15.84:0/927380790 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 32 and data pool 33 v133) v1 ==== 183+0+0 (secure 0 0 0) 0x7fbe881576b0 con 0x7fbe880ca3b0 2023-10-16T11:25:51.313 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 32 and data pool 33 2023-10-16T11:25:51.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.316+0000 7fbe8dbfb640 1 -- 172.21.15.84:0/927380790 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fbe6806b0b0 msgr2=0x7fbe6806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:51.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.316+0000 7fbe8dbfb640 1 --2- 172.21.15.84:0/927380790 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fbe6806b0b0 0x7fbe6806d570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe78035000 tx=0x7fbe78030000 comp rx=0 tx=0).stop 2023-10-16T11:25:51.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.316+0000 7fbe8dbfb640 1 -- 172.21.15.84:0/927380790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe880ca3b0 msgr2=0x7fbe88069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:51.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.316+0000 7fbe8dbfb640 1 --2- 172.21.15.84:0/927380790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe880ca3b0 0x7fbe88069da0 secure :-1 s=READY pgs=2974 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000c7f0 tx=0x7fbe7000cbf0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.317+0000 7fbe8dbfb640 1 -- 172.21.15.84:0/927380790 shutdown_connections 2023-10-16T11:25:51.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.317+0000 7fbe8dbfb640 1 --2- 172.21.15.84:0/927380790 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fbe6806b0b0 0x7fbe6806d570 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.317+0000 7fbe8dbfb640 1 --2- 172.21.15.84:0/927380790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe8815ad70 0x7fbe8806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.317+0000 7fbe8dbfb640 1 --2- 172.21.15.84:0/927380790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe880ca3b0 0x7fbe88069da0 unknown :-1 s=CLOSED pgs=2974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.317+0000 7fbe8dbfb640 1 --2- 172.21.15.84:0/927380790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe880c9ca0 0x7fbe8814bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.317+0000 7fbe8dbfb640 1 -- 172.21.15.84:0/927380790 >> 172.21.15.84:0/927380790 conn(0x7fbe880ce9b0 msgr2=0x7fbe880ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:51.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.317+0000 7fbe8dbfb640 1 -- 172.21.15.84:0/927380790 shutdown_connections 2023-10-16T11:25:51.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.317+0000 7fbe8dbfb640 1 -- 172.21.15.84:0/927380790 wait complete. 2023-10-16T11:25:51.353 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:25:51.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.526+0000 7f2b6f95a640 1 Processor -- start 2023-10-16T11:25:51.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.526+0000 7f2b6f95a640 1 -- start start 2023-10-16T11:25:51.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.527+0000 7f2b6f95a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2b6815ae80 0x7f2b6815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:51.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.527+0000 7f2b6f95a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b680ce390 0x7f2b68070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:51.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.527+0000 7f2b6f95a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b6815ed60 con 0x7f2b6815aaa0 2023-10-16T11:25:51.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.527+0000 7f2b6f95a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b6806b810 con 0x7f2b6815ae80 2023-10-16T11:25:51.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.527+0000 7f2b6f95a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b680707e0 con 0x7f2b680ce390 2023-10-16T11:25:51.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.528+0000 7f2b6d6f6640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2b6815aaa0 0x7f2b680cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58020/0 (socket says 172.21.15.84:58020) 2023-10-16T11:25:51.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.528+0000 7f2b6d6f6640 1 -- 172.21.15.84:0/3994505565 learned_addr learned my addr 172.21.15.84:0/3994505565 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:51.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.528+0000 7f2b6cef5640 1 --2- 172.21.15.84:0/3994505565 >> v2:172.21.15.157:3300/0 conn(0x7f2b6815ae80 0x7f2b6815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:51.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.528+0000 7f2b6def7640 1 --2- 172.21.15.84:0/3994505565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b680ce390 0x7f2b68070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:51.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.528+0000 7f2b5ffff640 1 -- 172.21.15.84:0/3994505565 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153342137 0 0) 0x7f2b6815ed60 con 0x7f2b6815aaa0 2023-10-16T11:25:51.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.528+0000 7f2b5ffff640 1 -- 172.21.15.84:0/3994505565 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b54003520 con 0x7f2b6815aaa0 2023-10-16T11:25:51.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.528+0000 7f2b5ffff640 1 -- 172.21.15.84:0/3994505565 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1716876212 0 0) 0x7f2b54003520 con 0x7f2b6815aaa0 2023-10-16T11:25:51.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.528+0000 7f2b5ffff640 1 -- 172.21.15.84:0/3994505565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b680ce390 msgr2=0x7f2b68070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:51.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.528+0000 7f2b5ffff640 1 --2- 172.21.15.84:0/3994505565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b680ce390 0x7f2b68070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.528+0000 7f2b5ffff640 1 -- 172.21.15.84:0/3994505565 >> v2:172.21.15.157:3300/0 conn(0x7f2b6815ae80 msgr2=0x7f2b6815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:51.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.528+0000 7f2b5ffff640 1 --2- 172.21.15.84:0/3994505565 >> v2:172.21.15.157:3300/0 conn(0x7f2b6815ae80 0x7f2b6815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.528+0000 7f2b5ffff640 1 -- 172.21.15.84:0/3994505565 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6815ef90 con 0x7f2b6815aaa0 2023-10-16T11:25:51.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.528+0000 7f2b5ffff640 1 -- 172.21.15.84:0/3994505565 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f2b64001690 con 0x7f2b6815aaa0 2023-10-16T11:25:51.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.529+0000 7f2b5ffff640 1 -- 172.21.15.84:0/3994505565 >> v1:172.21.15.84:6789/0 conn(0x7f2b6815aaa0 legacy=0x7f2b680cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:51.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.529+0000 7f2b5ffff640 1 --2- 172.21.15.84:0/3994505565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b680ce390 0x7f2b54003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:51.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.529+0000 7f2b5ffff640 1 --2- 172.21.15.84:0/3994505565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b54003ef0 0x7f2b540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:51.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.529+0000 7f2b5ffff640 1 --2- 172.21.15.84:0/3994505565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b54004d00 0x7f2b54005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:51.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.529+0000 7f2b5ffff640 1 -- 172.21.15.84:0/3994505565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6806b810 con 0x7f2b54003ef0 2023-10-16T11:25:51.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.529+0000 7f2b5ffff640 1 -- 172.21.15.84:0/3994505565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b680707e0 con 0x7f2b54004d00 2023-10-16T11:25:51.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.529+0000 7f2b5ffff640 1 -- 172.21.15.84:0/3994505565 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5400b9d0 con 0x7f2b680ce390 2023-10-16T11:25:51.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.529+0000 7f2b6cef5640 1 --2- 172.21.15.84:0/3994505565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b54003ef0 0x7f2b540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:51.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.529+0000 7f2b6d6f6640 1 --2- 172.21.15.84:0/3994505565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b680ce390 0x7f2b54003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:51.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.529+0000 7f2b6def7640 1 --2- 172.21.15.84:0/3994505565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b54004d00 0x7f2b54005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:51.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.529+0000 7f2b6cef5640 1 -- 172.21.15.84:0/3994505565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b680ce390 msgr2=0x7f2b54003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:51.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.529+0000 7f2b6cef5640 1 --2- 172.21.15.84:0/3994505565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b680ce390 0x7f2b54003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.529+0000 7f2b6cef5640 1 -- 172.21.15.84:0/3994505565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b54004d00 msgr2=0x7f2b54005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:51.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.529+0000 7f2b6cef5640 1 --2- 172.21.15.84:0/3994505565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b54004d00 0x7f2b54005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.529+0000 7f2b6cef5640 1 -- 172.21.15.84:0/3994505565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b54003520 con 0x7f2b54003ef0 2023-10-16T11:25:51.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.530+0000 7f2b6cef5640 1 --2- 172.21.15.84:0/3994505565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b54003ef0 0x7f2b540047c0 secure :-1 s=READY pgs=2976 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009a90 tx=0x7f2b540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e322fa1e8e418a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:51.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.531+0000 7f2b5ffff640 1 -- 172.21.15.84:0/3994505565 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5802c6b0 con 0x7f2b54003ef0 2023-10-16T11:25:51.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.531+0000 7f2b5ffff640 1 -- 172.21.15.84:0/3994505565 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5802c810 con 0x7f2b54003ef0 2023-10-16T11:25:51.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.531+0000 7f2b5ffff640 1 -- 172.21.15.84:0/3994505565 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5802cba0 con 0x7f2b54003ef0 2023-10-16T11:25:51.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.531+0000 7f2b6f95a640 1 -- 172.21.15.84:0/3994505565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b54003ef0 msgr2=0x7f2b540047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:51.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.531+0000 7f2b6f95a640 1 --2- 172.21.15.84:0/3994505565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b54003ef0 0x7f2b540047c0 secure :-1 s=READY pgs=2976 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009a90 tx=0x7f2b540057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.531+0000 7f2b6def7640 1 -- 172.21.15.84:0/3994505565 reap_dead start 2023-10-16T11:25:51.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.531+0000 7f2b6f95a640 1 -- 172.21.15.84:0/3994505565 shutdown_connections 2023-10-16T11:25:51.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.531+0000 7f2b6f95a640 1 -- 172.21.15.84:0/3994505565 >> 172.21.15.84:0/3994505565 conn(0x7f2b680cb410 msgr2=0x7f2b680cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:51.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.531+0000 7f2b6f95a640 1 -- 172.21.15.84:0/3994505565 shutdown_connections 2023-10-16T11:25:51.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.531+0000 7f2b6f95a640 1 -- 172.21.15.84:0/3994505565 wait complete. 2023-10-16T11:25:51.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.533+0000 7f2b6f95a640 1 Processor -- start 2023-10-16T11:25:51.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.533+0000 7f2b6f95a640 1 -- start start 2023-10-16T11:25:51.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.534+0000 7f2b6f95a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b54003ef0 0x7f2b68158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:51.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.534+0000 7f2b6f95a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b681f2f30 0x7f2b681f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:51.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.534+0000 7f2b6f95a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b680c5ad0 0x7f2b680c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:51.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.534+0000 7f2b6f95a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b680c7200 con 0x7f2b680c5ad0 2023-10-16T11:25:51.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.534+0000 7f2b6f95a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b681f27b0 con 0x7f2b681f2f30 2023-10-16T11:25:51.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.534+0000 7f2b6f95a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b680c6680 con 0x7f2b54003ef0 2023-10-16T11:25:51.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.534+0000 7f2b6def7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b680c5ad0 0x7f2b680c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:51.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.534+0000 7f2b6def7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b680c5ad0 0x7f2b680c5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34884/0 (socket says 172.21.15.84:34884) 2023-10-16T11:25:51.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.534+0000 7f2b6def7640 1 -- 172.21.15.84:0/348839371 learned_addr learned my addr 172.21.15.84:0/348839371 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:51.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.535+0000 7f2b6def7640 1 -- 172.21.15.84:0/348839371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b54003ef0 msgr2=0x7f2b68158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:51.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.535+0000 7f2b6def7640 1 --2- 172.21.15.84:0/348839371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b54003ef0 0x7f2b68158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.535+0000 7f2b6def7640 1 -- 172.21.15.84:0/348839371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b681f2f30 msgr2=0x7f2b681f33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:51.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.535+0000 7f2b6def7640 1 --2- 172.21.15.84:0/348839371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b681f2f30 0x7f2b681f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.535+0000 7f2b6def7640 1 -- 172.21.15.84:0/348839371 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b680c6960 con 0x7f2b680c5ad0 2023-10-16T11:25:51.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.535+0000 7f2b6def7640 1 --2- 172.21.15.84:0/348839371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b680c5ad0 0x7f2b680c5f50 secure :-1 s=READY pgs=2977 cs=0 l=1 rev1=1 crypto rx=0x7f2b680c5990 tx=0x7f2b60009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:51.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.535+0000 7f2b5dffb640 1 -- 172.21.15.84:0/348839371 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b60007cf0 con 0x7f2b680c5ad0 2023-10-16T11:25:51.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.535+0000 7f2b5dffb640 1 -- 172.21.15.84:0/348839371 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b60007e70 con 0x7f2b680c5ad0 2023-10-16T11:25:51.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.535+0000 7f2b6f95a640 1 -- 172.21.15.84:0/348839371 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b680c6c40 con 0x7f2b680c5ad0 2023-10-16T11:25:51.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.535+0000 7f2b5dffb640 1 -- 172.21.15.84:0/348839371 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b60014620 con 0x7f2b680c5ad0 2023-10-16T11:25:51.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.535+0000 7f2b6f95a640 1 -- 172.21.15.84:0/348839371 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b681f2a10 con 0x7f2b680c5ad0 2023-10-16T11:25:51.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.536+0000 7f2b6f95a640 1 -- 172.21.15.84:0/348839371 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b681f2c30 con 0x7f2b680c5ad0 2023-10-16T11:25:51.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.537+0000 7f2b5dffb640 1 -- 172.21.15.84:0/348839371 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f2b6001d3b0 con 0x7f2b680c5ad0 2023-10-16T11:25:51.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.538+0000 7f2b5dffb640 1 --2- 172.21.15.84:0/348839371 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f2b4c06b0e0 0x7f2b4c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:51.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.538+0000 7f2b5dffb640 1 -- 172.21.15.84:0/348839371 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f2b600898d0 con 0x7f2b680c5ad0 2023-10-16T11:25:51.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.538+0000 7f2b6d6f6640 1 --2- 172.21.15.84:0/348839371 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f2b4c06b0e0 0x7f2b4c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:51.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.539+0000 7f2b6d6f6640 1 --2- 172.21.15.84:0/348839371 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f2b4c06b0e0 0x7f2b4c06d5a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2b64000f60 tx=0x7f2b64009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:51.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.543+0000 7f2b5dffb640 1 -- 172.21.15.84:0/348839371 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2b60054370 con 0x7f2b680c5ad0 2023-10-16T11:25:51.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.719+0000 7f2b6f95a640 1 -- 172.21.15.84:0/348839371 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b6804e160 con 0x7f2b680c5ad0 2023-10-16T11:25:51.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.722+0000 7f2b5dffb640 1 -- 172.21.15.84:0/348839371 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v240) v1 ==== 74+0+20220 (secure 0 0 0) 0x7f2b60057990 con 0x7f2b680c5ad0 2023-10-16T11:25:51.724 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:51.725 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":240,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:25:50.738355+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:25:34.443897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:25:35.635615+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:25:49.001107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:25:49.760637+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:25:51.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.725+0000 7f2b6f95a640 1 -- 172.21.15.84:0/348839371 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f2b4c06b0e0 msgr2=0x7f2b4c06d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:51.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.725+0000 7f2b6f95a640 1 --2- 172.21.15.84:0/348839371 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f2b4c06b0e0 0x7f2b4c06d5a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2b64000f60 tx=0x7f2b64009040 comp rx=0 tx=0).stop 2023-10-16T11:25:51.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.725+0000 7f2b6f95a640 1 -- 172.21.15.84:0/348839371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b680c5ad0 msgr2=0x7f2b680c5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:51.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.725+0000 7f2b6f95a640 1 --2- 172.21.15.84:0/348839371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b680c5ad0 0x7f2b680c5f50 secure :-1 s=READY pgs=2977 cs=0 l=1 rev1=1 crypto rx=0x7f2b680c5990 tx=0x7f2b60009cb0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.725+0000 7f2b6f95a640 1 -- 172.21.15.84:0/348839371 shutdown_connections 2023-10-16T11:25:51.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.725+0000 7f2b6f95a640 1 --2- 172.21.15.84:0/348839371 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f2b4c06b0e0 0x7f2b4c06d5a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.725+0000 7f2b6f95a640 1 --2- 172.21.15.84:0/348839371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b680c5ad0 0x7f2b680c5f50 unknown :-1 s=CLOSED pgs=2977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.725+0000 7f2b6f95a640 1 --2- 172.21.15.84:0/348839371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b681f2f30 0x7f2b681f33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.725+0000 7f2b6f95a640 1 --2- 172.21.15.84:0/348839371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b54003ef0 0x7f2b68158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.725+0000 7f2b6f95a640 1 -- 172.21.15.84:0/348839371 >> 172.21.15.84:0/348839371 conn(0x7f2b680cb410 msgr2=0x7f2b6806f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:51.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.725+0000 7f2b6f95a640 1 -- 172.21.15.84:0/348839371 shutdown_connections 2023-10-16T11:25:51.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.725+0000 7f2b6f95a640 1 -- 172.21.15.84:0/348839371 wait complete. 2023-10-16T11:25:51.745 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:25:51.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.956+0000 7fc0e575a640 1 Processor -- start 2023-10-16T11:25:51.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.956+0000 7fc0e575a640 1 -- start start 2023-10-16T11:25:51.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.958+0000 7fc0e575a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc0e0159b20 0x7fc0e006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:51.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.958+0000 7fc0e575a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0e006f7f0 con 0x7fc0e00b0b20 2023-10-16T11:25:51.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.958+0000 7fc0e575a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0e006c5e0 con 0x7fc0e0159f00 2023-10-16T11:25:51.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.958+0000 7fc0e575a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0e006c8c0 con 0x7fc0e0159b20 2023-10-16T11:25:51.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.958+0000 7fc0deffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc0e0159b20 0x7fc0e006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:51.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.958+0000 7fc0df7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc0e00b0b20 0x7fc0e006fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58034/0 (socket says 172.21.15.84:58034) 2023-10-16T11:25:51.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.958+0000 7fc0df7fe640 1 -- 172.21.15.84:0/2633399119 learned_addr learned my addr 172.21.15.84:0/2633399119 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:51.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.958+0000 7fc0deffd640 1 -- 172.21.15.84:0/2633399119 >> v1:172.21.15.157:6789/0 conn(0x7fc0e0159f00 legacy=0x7fc0e00b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:51.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.958+0000 7fc0deffd640 1 -- 172.21.15.84:0/2633399119 >> v1:172.21.15.84:6789/0 conn(0x7fc0e00b0b20 legacy=0x7fc0e006fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:51.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.958+0000 7fc0deffd640 1 -- 172.21.15.84:0/2633399119 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e006cba0 con 0x7fc0e0159b20 2023-10-16T11:25:51.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.959+0000 7fc0deffd640 1 --2- 172.21.15.84:0/2633399119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0e0159b20 0x7fc0e006bbe0 secure :-1 s=READY pgs=2979 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00097a0 tx=0x7fc0d002dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512c566427077ec4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:51.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.960+0000 7fc0ddffb640 1 -- 172.21.15.84:0/2633399119 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d002f8a0 con 0x7fc0e0159b20 2023-10-16T11:25:51.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.960+0000 7fc0ddffb640 1 -- 172.21.15.84:0/2633399119 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d002fa00 con 0x7fc0e0159b20 2023-10-16T11:25:51.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.960+0000 7fc0ddffb640 1 -- 172.21.15.84:0/2633399119 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d002fd90 con 0x7fc0e0159b20 2023-10-16T11:25:51.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.960+0000 7fc0e575a640 1 -- 172.21.15.84:0/2633399119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0e0159b20 msgr2=0x7fc0e006bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:51.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.960+0000 7fc0e575a640 1 --2- 172.21.15.84:0/2633399119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0e0159b20 0x7fc0e006bbe0 secure :-1 s=READY pgs=2979 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00097a0 tx=0x7fc0d002dae0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.961+0000 7fc0e575a640 1 -- 172.21.15.84:0/2633399119 shutdown_connections 2023-10-16T11:25:51.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.961+0000 7fc0e575a640 1 --2- 172.21.15.84:0/2633399119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0e0159b20 0x7fc0e006bbe0 unknown :-1 s=CLOSED pgs=2979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.961+0000 7fc0e575a640 1 -- 172.21.15.84:0/2633399119 >> 172.21.15.84:0/2633399119 conn(0x7fc0e00cb410 msgr2=0x7fc0e00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:51.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.961+0000 7fc0e575a640 1 -- 172.21.15.84:0/2633399119 shutdown_connections 2023-10-16T11:25:51.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.961+0000 7fc0e575a640 1 -- 172.21.15.84:0/2633399119 wait complete. 2023-10-16T11:25:51.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.962+0000 7fc0e575a640 1 Processor -- start 2023-10-16T11:25:51.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.962+0000 7fc0e575a640 1 -- start start 2023-10-16T11:25:51.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.963+0000 7fc0e575a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0e00b0b20 0x7fc0e00c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:51.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.963+0000 7fc0e575a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0e0159b20 0x7fc0e015adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:51.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.963+0000 7fc0e575a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0e0159f00 0x7fc0e00c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:51.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.963+0000 7fc0e575a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e015c220 con 0x7fc0e0159f00 2023-10-16T11:25:51.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.963+0000 7fc0e575a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e015b300 con 0x7fc0e00b0b20 2023-10-16T11:25:51.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.963+0000 7fc0e575a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e015b5a0 con 0x7fc0e0159b20 2023-10-16T11:25:51.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.964+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0e0159f00 0x7fc0e00c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:51.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.964+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0e0159f00 0x7fc0e00c24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34894/0 (socket says 172.21.15.84:34894) 2023-10-16T11:25:51.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.964+0000 7fc0df7fe640 1 -- 172.21.15.84:0/3105329689 learned_addr learned my addr 172.21.15.84:0/3105329689 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:51.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.964+0000 7fc0deffd640 1 --2- 172.21.15.84:0/3105329689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0e00b0b20 0x7fc0e00c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:51.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.964+0000 7fc0de7fc640 1 --2- 172.21.15.84:0/3105329689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0e0159b20 0x7fc0e015adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:51.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.964+0000 7fc0df7fe640 1 -- 172.21.15.84:0/3105329689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0e0159b20 msgr2=0x7fc0e015adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:51.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.964+0000 7fc0df7fe640 1 --2- 172.21.15.84:0/3105329689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0e0159b20 0x7fc0e015adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.964+0000 7fc0df7fe640 1 -- 172.21.15.84:0/3105329689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0e00b0b20 msgr2=0x7fc0e00c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:51.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.964+0000 7fc0df7fe640 1 --2- 172.21.15.84:0/3105329689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0e00b0b20 0x7fc0e00c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:51.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.964+0000 7fc0df7fe640 1 -- 172.21.15.84:0/3105329689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d0009400 con 0x7fc0e0159f00 2023-10-16T11:25:51.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.964+0000 7fc0df7fe640 1 --2- 172.21.15.84:0/3105329689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0e0159f00 0x7fc0e00c24c0 secure :-1 s=READY pgs=2980 cs=0 l=1 rev1=1 crypto rx=0x7fc0d800ca80 tx=0x7fc0d800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:51.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.965+0000 7fc0c77fe640 1 -- 172.21.15.84:0/3105329689 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d800ac30 con 0x7fc0e0159f00 2023-10-16T11:25:51.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.965+0000 7fc0c77fe640 1 -- 172.21.15.84:0/3105329689 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d800ad90 con 0x7fc0e0159f00 2023-10-16T11:25:51.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.965+0000 7fc0c77fe640 1 -- 172.21.15.84:0/3105329689 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8012600 con 0x7fc0e0159f00 2023-10-16T11:25:51.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.965+0000 7fc0e575a640 1 -- 172.21.15.84:0/3105329689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e015b880 con 0x7fc0e0159f00 2023-10-16T11:25:51.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.965+0000 7fc0e575a640 1 -- 172.21.15.84:0/3105329689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e015be40 con 0x7fc0e0159f00 2023-10-16T11:25:51.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.966+0000 7fc0e575a640 1 -- 172.21.15.84:0/3105329689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e015c360 con 0x7fc0e0159f00 2023-10-16T11:25:51.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.967+0000 7fc0c77fe640 1 -- 172.21.15.84:0/3105329689 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fc0d8007070 con 0x7fc0e0159f00 2023-10-16T11:25:51.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.968+0000 7fc0c77fe640 1 --2- 172.21.15.84:0/3105329689 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc0cc06b100 0x7fc0cc06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:51.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.968+0000 7fc0c77fe640 1 -- 172.21.15.84:0/3105329689 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 6601+0+0 (secure 0 0 0) 0x7fc0d8089b70 con 0x7fc0e0159f00 2023-10-16T11:25:51.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.968+0000 7fc0deffd640 1 --2- 172.21.15.84:0/3105329689 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc0cc06b100 0x7fc0cc06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:51.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.969+0000 7fc0deffd640 1 --2- 172.21.15.84:0/3105329689 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc0cc06b100 0x7fc0cc06d5c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0038000 tx=0x7fc0d003b040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:51.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:51.972+0000 7fc0c77fe640 1 -- 172.21.15.84:0/3105329689 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc0e015c360 con 0x7fc0e0159f00 2023-10-16T11:25:52.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.150+0000 7fc0e575a640 1 -- 172.21.15.84:0/3105329689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0e015c360 con 0x7fc0e0159f00 2023-10-16T11:25:52.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.153+0000 7fc0c77fe640 1 -- 172.21.15.84:0/3105329689 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v240) v1 ==== 74+0+20220 (secure 0 0 0) 0x7fc0e015c360 con 0x7fc0e0159f00 2023-10-16T11:25:52.155 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:52.155 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":240,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:25:50.738355+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:25:34.443897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:25:35.635615+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:25:49.001107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:25:49.760637+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:25:52.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.155+0000 7fc0e575a640 1 -- 172.21.15.84:0/3105329689 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc0cc06b100 msgr2=0x7fc0cc06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:52.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.155+0000 7fc0e575a640 1 --2- 172.21.15.84:0/3105329689 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc0cc06b100 0x7fc0cc06d5c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0038000 tx=0x7fc0d003b040 comp rx=0 tx=0).stop 2023-10-16T11:25:52.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.155+0000 7fc0e575a640 1 -- 172.21.15.84:0/3105329689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0e0159f00 msgr2=0x7fc0e00c24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:52.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.155+0000 7fc0e575a640 1 --2- 172.21.15.84:0/3105329689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0e0159f00 0x7fc0e00c24c0 secure :-1 s=READY pgs=2980 cs=0 l=1 rev1=1 crypto rx=0x7fc0d800ca80 tx=0x7fc0d800cf40 comp rx=0 tx=0).stop 2023-10-16T11:25:52.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.156+0000 7fc0e575a640 1 -- 172.21.15.84:0/3105329689 shutdown_connections 2023-10-16T11:25:52.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.156+0000 7fc0e575a640 1 --2- 172.21.15.84:0/3105329689 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc0cc06b100 0x7fc0cc06d5c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:52.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.156+0000 7fc0e575a640 1 --2- 172.21.15.84:0/3105329689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0e0159f00 0x7fc0e00c24c0 unknown :-1 s=CLOSED pgs=2980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:52.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.156+0000 7fc0e575a640 1 --2- 172.21.15.84:0/3105329689 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0e0159b20 0x7fc0e015adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:52.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.156+0000 7fc0e575a640 1 --2- 172.21.15.84:0/3105329689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0e00b0b20 0x7fc0e00c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:52.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.156+0000 7fc0e575a640 1 -- 172.21.15.84:0/3105329689 >> 172.21.15.84:0/3105329689 conn(0x7fc0e00cb410 msgr2=0x7fc0e00ad250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:52.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.156+0000 7fc0e575a640 1 -- 172.21.15.84:0/3105329689 shutdown_connections 2023-10-16T11:25:52.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.156+0000 7fc0e575a640 1 -- 172.21.15.84:0/3105329689 wait complete. 2023-10-16T11:25:52.177 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2023-10-16T11:25:52.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.382+0000 7fa92cddc640 1 Processor -- start 2023-10-16T11:25:52.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.382+0000 7fa92cddc640 1 -- start start 2023-10-16T11:25:52.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.384+0000 7fa92cddc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa92815ad70 0x7fa928069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:52.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.384+0000 7fa92cddc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa9280caac0 0x7fa92806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:52.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.384+0000 7fa92cddc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9280cb030 con 0x7fa9280c9ca0 2023-10-16T11:25:52.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.384+0000 7fa92cddc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9280c91c0 con 0x7fa9280caac0 2023-10-16T11:25:52.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.384+0000 7fa92cddc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280c94a0 con 0x7fa92815ad70 2023-10-16T11:25:52.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.384+0000 7fa925d9b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa9280c9ca0 0x7fa9280ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58040/0 (socket says 172.21.15.84:58040) 2023-10-16T11:25:52.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.384+0000 7fa925d9b640 1 -- 172.21.15.84:0/1946253745 learned_addr learned my addr 172.21.15.84:0/1946253745 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:52.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa92559a640 1 -- 172.21.15.84:0/1946253745 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744041360 0 0) 0x7fa9280cb030 con 0x7fa9280c9ca0 2023-10-16T11:25:52.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa92659c640 1 --2- 172.21.15.84:0/1946253745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa92815ad70 0x7fa928069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:52.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa92559a640 1 -- 172.21.15.84:0/1946253745 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa914003520 con 0x7fa9280c9ca0 2023-10-16T11:25:52.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa926d9d640 1 --2- 172.21.15.84:0/1946253745 >> v2:172.21.15.157:3300/0 conn(0x7fa9280caac0 0x7fa92806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:52.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa92559a640 1 -- 172.21.15.84:0/1946253745 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2026604520 0 0) 0x7fa914003520 con 0x7fa9280c9ca0 2023-10-16T11:25:52.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa92559a640 1 -- 172.21.15.84:0/1946253745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa92815ad70 msgr2=0x7fa928069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:52.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa92559a640 1 --2- 172.21.15.84:0/1946253745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa92815ad70 0x7fa928069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:52.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa92559a640 1 -- 172.21.15.84:0/1946253745 >> v2:172.21.15.157:3300/0 conn(0x7fa9280caac0 msgr2=0x7fa92806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:52.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa92559a640 1 --2- 172.21.15.84:0/1946253745 >> v2:172.21.15.157:3300/0 conn(0x7fa9280caac0 0x7fa92806a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:52.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa92559a640 1 -- 172.21.15.84:0/1946253745 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9280c9780 con 0x7fa9280c9ca0 2023-10-16T11:25:52.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa92559a640 1 -- 172.21.15.84:0/1946253745 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fa910001670 con 0x7fa9280c9ca0 2023-10-16T11:25:52.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa92559a640 1 -- 172.21.15.84:0/1946253745 >> v1:172.21.15.84:6789/0 conn(0x7fa9280c9ca0 legacy=0x7fa9280ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:52.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa92559a640 1 --2- 172.21.15.84:0/1946253745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa92815ad70 0x7fa914003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:52.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa92559a640 1 --2- 172.21.15.84:0/1946253745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa914003ff0 0x7fa9140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:52.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa92559a640 1 --2- 172.21.15.84:0/1946253745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa914004d00 0x7fa914005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:52.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa92559a640 1 -- 172.21.15.84:0/1946253745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280c91c0 con 0x7fa914003ff0 2023-10-16T11:25:52.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa92559a640 1 -- 172.21.15.84:0/1946253745 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280c94a0 con 0x7fa914004d00 2023-10-16T11:25:52.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa92559a640 1 -- 172.21.15.84:0/1946253745 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91400bae0 con 0x7fa92815ad70 2023-10-16T11:25:52.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.385+0000 7fa925d9b640 1 --2- 172.21.15.84:0/1946253745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa914003ff0 0x7fa9140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:52.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.386+0000 7fa92659c640 1 --2- 172.21.15.84:0/1946253745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa92815ad70 0x7fa914003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:52.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.386+0000 7fa925d9b640 1 -- 172.21.15.84:0/1946253745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa92815ad70 msgr2=0x7fa914003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:52.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.386+0000 7fa925d9b640 1 --2- 172.21.15.84:0/1946253745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa92815ad70 0x7fa914003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:52.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.386+0000 7fa926d9d640 1 --2- 172.21.15.84:0/1946253745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa914004d00 0x7fa914005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:52.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.386+0000 7fa925d9b640 1 -- 172.21.15.84:0/1946253745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa914004d00 msgr2=0x7fa914005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:52.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.386+0000 7fa925d9b640 1 --2- 172.21.15.84:0/1946253745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa914004d00 0x7fa914005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:52.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.386+0000 7fa925d9b640 1 -- 172.21.15.84:0/1946253745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa914003520 con 0x7fa914003ff0 2023-10-16T11:25:52.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.387+0000 7fa925d9b640 1 --2- 172.21.15.84:0/1946253745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa914003ff0 0x7fa9140047c0 secure :-1 s=READY pgs=2982 cs=0 l=1 rev1=1 crypto rx=0x7fa91000c110 tx=0x7fa9140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aadab2054c5d862 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:52.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.387+0000 7fa92559a640 1 -- 172.21.15.84:0/1946253745 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910005ad0 con 0x7fa914003ff0 2023-10-16T11:25:52.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.387+0000 7fa92559a640 1 -- 172.21.15.84:0/1946253745 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa910005c50 con 0x7fa914003ff0 2023-10-16T11:25:52.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.387+0000 7fa92559a640 1 -- 172.21.15.84:0/1946253745 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91002c830 con 0x7fa914003ff0 2023-10-16T11:25:52.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.387+0000 7fa92cddc640 1 -- 172.21.15.84:0/1946253745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa914003ff0 msgr2=0x7fa9140047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:52.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.388+0000 7fa92cddc640 1 --2- 172.21.15.84:0/1946253745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa914003ff0 0x7fa9140047c0 secure :-1 s=READY pgs=2982 cs=0 l=1 rev1=1 crypto rx=0x7fa91000c110 tx=0x7fa9140057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:52.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.388+0000 7fa926d9d640 1 -- 172.21.15.84:0/1946253745 reap_dead start 2023-10-16T11:25:52.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.388+0000 7fa92cddc640 1 -- 172.21.15.84:0/1946253745 shutdown_connections 2023-10-16T11:25:52.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.388+0000 7fa92cddc640 1 -- 172.21.15.84:0/1946253745 >> 172.21.15.84:0/1946253745 conn(0x7fa9280ce9b0 msgr2=0x7fa9280cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:52.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.388+0000 7fa92cddc640 1 -- 172.21.15.84:0/1946253745 shutdown_connections 2023-10-16T11:25:52.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.388+0000 7fa92cddc640 1 -- 172.21.15.84:0/1946253745 wait complete. 2023-10-16T11:25:52.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.389+0000 7fa92cddc640 1 Processor -- start 2023-10-16T11:25:52.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.390+0000 7fa92cddc640 1 -- start start 2023-10-16T11:25:52.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.390+0000 7fa92cddc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa914003ff0 0x7fa92814a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:52.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.390+0000 7fa92cddc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa92814b330 0x7fa92814b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:52.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.390+0000 7fa92cddc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9280c7c50 0x7fa9280c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:52.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.390+0000 7fa92cddc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280c8620 con 0x7fa92814b330 2023-10-16T11:25:52.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.390+0000 7fa92cddc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9281f2080 con 0x7fa9280c7c50 2023-10-16T11:25:52.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.390+0000 7fa92cddc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92814bea0 con 0x7fa914003ff0 2023-10-16T11:25:52.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.391+0000 7fa925d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa92814b330 0x7fa92814b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:52.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.391+0000 7fa925d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa92814b330 0x7fa92814b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34910/0 (socket says 172.21.15.84:34910) 2023-10-16T11:25:52.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.391+0000 7fa925d9b640 1 -- 172.21.15.84:0/1138989057 learned_addr learned my addr 172.21.15.84:0/1138989057 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:52.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.391+0000 7fa92659c640 1 --2- 172.21.15.84:0/1138989057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa914003ff0 0x7fa92814a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:52.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.391+0000 7fa926d9d640 1 --2- 172.21.15.84:0/1138989057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9280c7c50 0x7fa9280c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:52.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.391+0000 7fa925d9b640 1 -- 172.21.15.84:0/1138989057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa914003ff0 msgr2=0x7fa92814a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:52.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.391+0000 7fa925d9b640 1 --2- 172.21.15.84:0/1138989057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa914003ff0 0x7fa92814a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:52.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.391+0000 7fa925d9b640 1 -- 172.21.15.84:0/1138989057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9280c7c50 msgr2=0x7fa9280c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:52.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.391+0000 7fa925d9b640 1 --2- 172.21.15.84:0/1138989057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9280c7c50 0x7fa9280c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:52.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.391+0000 7fa925d9b640 1 -- 172.21.15.84:0/1138989057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92814c180 con 0x7fa92814b330 2023-10-16T11:25:52.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.391+0000 7fa92659c640 1 --2- 172.21.15.84:0/1138989057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa914003ff0 0x7fa92814a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:52.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.391+0000 7fa925d9b640 1 --2- 172.21.15.84:0/1138989057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa92814b330 0x7fa92814b7b0 secure :-1 s=READY pgs=2983 cs=0 l=1 rev1=1 crypto rx=0x7fa910000fd0 tx=0x7fa910001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:52.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.392+0000 7fa91effd640 1 -- 172.21.15.84:0/1138989057 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91002cc40 con 0x7fa92814b330 2023-10-16T11:25:52.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.392+0000 7fa91effd640 1 -- 172.21.15.84:0/1138989057 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91002cda0 con 0x7fa92814b330 2023-10-16T11:25:52.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.392+0000 7fa91effd640 1 -- 172.21.15.84:0/1138989057 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910007270 con 0x7fa92814b330 2023-10-16T11:25:52.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.392+0000 7fa92cddc640 1 -- 172.21.15.84:0/1138989057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92814c460 con 0x7fa92814b330 2023-10-16T11:25:52.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.392+0000 7fa92cddc640 1 -- 172.21.15.84:0/1138989057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92814ca20 con 0x7fa92814b330 2023-10-16T11:25:52.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.393+0000 7fa92cddc640 1 -- 172.21.15.84:0/1138989057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa92814cbe0 con 0x7fa92814b330 2023-10-16T11:25:52.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.394+0000 7fa91effd640 1 -- 172.21.15.84:0/1138989057 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fa910032070 con 0x7fa92814b330 2023-10-16T11:25:52.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.395+0000 7fa91effd640 1 --2- 172.21.15.84:0/1138989057 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fa90806b0b0 0x7fa90806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:52.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.395+0000 7fa91effd640 1 -- 172.21.15.84:0/1138989057 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7fa91003c040 con 0x7fa92814b330 2023-10-16T11:25:52.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.395+0000 7fa92659c640 1 --2- 172.21.15.84:0/1138989057 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fa90806b0b0 0x7fa90806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:52.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.396+0000 7fa92659c640 1 --2- 172.21.15.84:0/1138989057 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fa90806b0b0 0x7fa90806d570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa92815a360 tx=0x7fa91800b040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:52.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.399+0000 7fa91effd640 1 -- 172.21.15.84:0/1138989057 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa910075910 con 0x7fa92814b330 2023-10-16T11:25:52.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:52.613+0000 7fa92cddc640 1 -- 172.21.15.84:0/1138989057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa92804e160 con 0x7fa92814b330 2023-10-16T11:25:53.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.329+0000 7fa91effd640 1 -- 172.21.15.84:0/1138989057 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v136) v1 ==== 121+0+0 (secure 0 0 0) 0x7fa910045020 con 0x7fa92814b330 2023-10-16T11:25:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.333+0000 7fa92cddc640 1 -- 172.21.15.84:0/1138989057 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fa90806b0b0 msgr2=0x7fa90806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.333+0000 7fa92cddc640 1 --2- 172.21.15.84:0/1138989057 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fa90806b0b0 0x7fa90806d570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa92815a360 tx=0x7fa91800b040 comp rx=0 tx=0).stop 2023-10-16T11:25:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.333+0000 7fa92cddc640 1 -- 172.21.15.84:0/1138989057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa92814b330 msgr2=0x7fa92814b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.333+0000 7fa92cddc640 1 --2- 172.21.15.84:0/1138989057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa92814b330 0x7fa92814b7b0 secure :-1 s=READY pgs=2983 cs=0 l=1 rev1=1 crypto rx=0x7fa910000fd0 tx=0x7fa910001930 comp rx=0 tx=0).stop 2023-10-16T11:25:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.334+0000 7fa92cddc640 1 -- 172.21.15.84:0/1138989057 shutdown_connections 2023-10-16T11:25:53.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.334+0000 7fa92cddc640 1 --2- 172.21.15.84:0/1138989057 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fa90806b0b0 0x7fa90806d570 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:53.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.334+0000 7fa92cddc640 1 --2- 172.21.15.84:0/1138989057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9280c7c50 0x7fa9280c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:53.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.334+0000 7fa92cddc640 1 --2- 172.21.15.84:0/1138989057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa92814b330 0x7fa92814b7b0 unknown :-1 s=CLOSED pgs=2983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:53.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.334+0000 7fa92cddc640 1 --2- 172.21.15.84:0/1138989057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa914003ff0 0x7fa92814a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:53.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.334+0000 7fa92cddc640 1 -- 172.21.15.84:0/1138989057 >> 172.21.15.84:0/1138989057 conn(0x7fa9280ce9b0 msgr2=0x7fa9280cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:53.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.334+0000 7fa92cddc640 1 -- 172.21.15.84:0/1138989057 shutdown_connections 2023-10-16T11:25:53.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.334+0000 7fa92cddc640 1 -- 172.21.15.84:0/1138989057 wait complete. 2023-10-16T11:25:53.357 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T11:25:53.357 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2023-10-16T11:25:53.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.560+0000 7f26a05a9640 1 Processor -- start 2023-10-16T11:25:53.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.560+0000 7f26a05a9640 1 -- start start 2023-10-16T11:25:53.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.562+0000 7f26a05a9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f269815d150 0x7f26980ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:53.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.562+0000 7f26a05a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f269806a4b0 0x7f2698160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:53.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.562+0000 7f26a05a9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26980ac890 con 0x7f2698160b90 2023-10-16T11:25:53.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.562+0000 7f26a05a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f269815d530 con 0x7f269806a4b0 2023-10-16T11:25:53.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.562+0000 7f26a05a9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2698161650 con 0x7f269815d150 2023-10-16T11:25:53.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.562+0000 7f269eb46640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f269806a4b0 0x7f2698160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:53.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.562+0000 7f269db44640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2698160b90 0x7f2698069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58048/0 (socket says 172.21.15.84:58048) 2023-10-16T11:25:53.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.562+0000 7f269db44640 1 -- 172.21.15.84:0/4260941202 learned_addr learned my addr 172.21.15.84:0/4260941202 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:53.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.562+0000 7f269e345640 1 --2- 172.21.15.84:0/4260941202 >> v2:172.21.15.157:3300/0 conn(0x7f269815d150 0x7f26980ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:53.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.562+0000 7f269d343640 1 -- 172.21.15.84:0/4260941202 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251985361 0 0) 0x7f26980ac890 con 0x7f2698160b90 2023-10-16T11:25:53.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.562+0000 7f269eb46640 1 -- 172.21.15.84:0/4260941202 >> v2:172.21.15.157:3300/0 conn(0x7f269815d150 msgr2=0x7f26980ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:53.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.562+0000 7f269eb46640 1 --2- 172.21.15.84:0/4260941202 >> v2:172.21.15.157:3300/0 conn(0x7f269815d150 0x7f26980ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:53.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.563+0000 7f269eb46640 1 -- 172.21.15.84:0/4260941202 >> v1:172.21.15.84:6789/0 conn(0x7f2698160b90 legacy=0x7f2698069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:53.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.563+0000 7f269eb46640 1 -- 172.21.15.84:0/4260941202 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2698161930 con 0x7f269806a4b0 2023-10-16T11:25:53.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.564+0000 7f269eb46640 1 --2- 172.21.15.84:0/4260941202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f269806a4b0 0x7f2698160fe0 secure :-1 s=READY pgs=2985 cs=0 l=1 rev1=1 crypto rx=0x7f26980b15a0 tx=0x7f269002be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519fac18cd39e48d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:53.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.564+0000 7f269d343640 1 -- 172.21.15.84:0/4260941202 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269002cbe0 con 0x7f269806a4b0 2023-10-16T11:25:53.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.564+0000 7f269d343640 1 -- 172.21.15.84:0/4260941202 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269002cd40 con 0x7f269806a4b0 2023-10-16T11:25:53.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.564+0000 7f269d343640 1 -- 172.21.15.84:0/4260941202 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2690034660 con 0x7f269806a4b0 2023-10-16T11:25:53.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.564+0000 7f26a05a9640 1 -- 172.21.15.84:0/4260941202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f269806a4b0 msgr2=0x7f2698160fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:53.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.564+0000 7f26a05a9640 1 --2- 172.21.15.84:0/4260941202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f269806a4b0 0x7f2698160fe0 secure :-1 s=READY pgs=2985 cs=0 l=1 rev1=1 crypto rx=0x7f26980b15a0 tx=0x7f269002be30 comp rx=0 tx=0).stop 2023-10-16T11:25:53.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.565+0000 7f26a05a9640 1 -- 172.21.15.84:0/4260941202 shutdown_connections 2023-10-16T11:25:53.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.565+0000 7f26a05a9640 1 --2- 172.21.15.84:0/4260941202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f269806a4b0 0x7f2698160fe0 unknown :-1 s=CLOSED pgs=2985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:53.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.565+0000 7f26a05a9640 1 --2- 172.21.15.84:0/4260941202 >> v2:172.21.15.157:3300/0 conn(0x7f269815d150 0x7f26980ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:53.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.565+0000 7f26a05a9640 1 -- 172.21.15.84:0/4260941202 >> 172.21.15.84:0/4260941202 conn(0x7f26980ce9b0 msgr2=0x7f26980cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:53.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.565+0000 7f26a05a9640 1 -- 172.21.15.84:0/4260941202 shutdown_connections 2023-10-16T11:25:53.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.565+0000 7f26a05a9640 1 -- 172.21.15.84:0/4260941202 wait complete. 2023-10-16T11:25:53.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.566+0000 7f26a05a9640 1 Processor -- start 2023-10-16T11:25:53.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.566+0000 7f26a05a9640 1 -- start start 2023-10-16T11:25:53.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.567+0000 7f26a05a9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f269806a4b0 0x7f26981f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:53.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.567+0000 7f26a05a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f269815d150 0x7f26980c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:53.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.567+0000 7f26a05a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2698160b90 0x7f26980c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:53.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.567+0000 7f26a05a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26980c6a70 con 0x7f269815d150 2023-10-16T11:25:53.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.567+0000 7f26a05a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26980c5df0 con 0x7f2698160b90 2023-10-16T11:25:53.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.567+0000 7f26a05a9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26980c60d0 con 0x7f269806a4b0 2023-10-16T11:25:53.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.568+0000 7f269db44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f269815d150 0x7f26980c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:53.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.568+0000 7f269db44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f269815d150 0x7f26980c6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34936/0 (socket says 172.21.15.84:34936) 2023-10-16T11:25:53.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.568+0000 7f269db44640 1 -- 172.21.15.84:0/2700826255 learned_addr learned my addr 172.21.15.84:0/2700826255 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:53.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.568+0000 7f269e345640 1 --2- 172.21.15.84:0/2700826255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f269806a4b0 0x7f26981f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:53.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.568+0000 7f269db44640 1 -- 172.21.15.84:0/2700826255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f269806a4b0 msgr2=0x7f26981f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:53.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.568+0000 7f269db44640 1 --2- 172.21.15.84:0/2700826255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f269806a4b0 0x7f26981f2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:53.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.568+0000 7f269db44640 1 -- 172.21.15.84:0/2700826255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2698160b90 msgr2=0x7f26980c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:53.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.568+0000 7f269db44640 1 --2- 172.21.15.84:0/2700826255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2698160b90 0x7f26980c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:53.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.568+0000 7f269db44640 1 -- 172.21.15.84:0/2700826255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2690009a70 con 0x7f269815d150 2023-10-16T11:25:53.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.568+0000 7f269db44640 1 --2- 172.21.15.84:0/2700826255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f269815d150 0x7f26980c6e90 secure :-1 s=READY pgs=2986 cs=0 l=1 rev1=1 crypto rx=0x7f268800c430 tx=0x7f268800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:53.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.569+0000 7f268effd640 1 -- 172.21.15.84:0/2700826255 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26880074e0 con 0x7f269815d150 2023-10-16T11:25:53.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.569+0000 7f268effd640 1 -- 172.21.15.84:0/2700826255 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2688007660 con 0x7f269815d150 2023-10-16T11:25:53.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.569+0000 7f268effd640 1 -- 172.21.15.84:0/2700826255 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688004a20 con 0x7f269815d150 2023-10-16T11:25:53.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.569+0000 7f26a05a9640 1 -- 172.21.15.84:0/2700826255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26980c63b0 con 0x7f269815d150 2023-10-16T11:25:53.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.569+0000 7f26a05a9640 1 -- 172.21.15.84:0/2700826255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269814f4b0 con 0x7f269815d150 2023-10-16T11:25:53.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.570+0000 7f26a05a9640 1 -- 172.21.15.84:0/2700826255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26980c6bb0 con 0x7f269815d150 2023-10-16T11:25:53.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.571+0000 7f268effd640 1 -- 172.21.15.84:0/2700826255 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f2688004b80 con 0x7f269815d150 2023-10-16T11:25:53.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.571+0000 7f268effd640 1 --2- 172.21.15.84:0/2700826255 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f267c06b0b0 0x7f267c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:53.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.572+0000 7f268effd640 1 -- 172.21.15.84:0/2700826255 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f2688089c90 con 0x7f269815d150 2023-10-16T11:25:53.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.572+0000 7f269e345640 1 --2- 172.21.15.84:0/2700826255 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f267c06b0b0 0x7f267c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:53.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.573+0000 7f269e345640 1 --2- 172.21.15.84:0/2700826255 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f267c06b0b0 0x7f267c06d570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2694006fd0 tx=0x7f2694008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:53.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.576+0000 7f268effd640 1 -- 172.21.15.84:0/2700826255 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f26980c6bb0 con 0x7f269815d150 2023-10-16T11:25:53.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:53.790+0000 7f26a05a9640 1 -- 172.21.15.84:0/2700826255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f26980c6bb0 con 0x7f269815d150 2023-10-16T11:25:54.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.339+0000 7f268effd640 1 -- 172.21.15.84:0/2700826255 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v137) v1 ==== 125+0+0 (secure 0 0 0) 0x7f26980c6bb0 con 0x7f269815d150 2023-10-16T11:25:54.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.343+0000 7f26a05a9640 1 -- 172.21.15.84:0/2700826255 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f267c06b0b0 msgr2=0x7f267c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:54.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.343+0000 7f26a05a9640 1 --2- 172.21.15.84:0/2700826255 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f267c06b0b0 0x7f267c06d570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2694006fd0 tx=0x7f2694008040 comp rx=0 tx=0).stop 2023-10-16T11:25:54.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.343+0000 7f26a05a9640 1 -- 172.21.15.84:0/2700826255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f269815d150 msgr2=0x7f26980c6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:54.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.343+0000 7f26a05a9640 1 --2- 172.21.15.84:0/2700826255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f269815d150 0x7f26980c6e90 secure :-1 s=READY pgs=2986 cs=0 l=1 rev1=1 crypto rx=0x7f268800c430 tx=0x7f268800c8f0 comp rx=0 tx=0).stop 2023-10-16T11:25:54.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.343+0000 7f26a05a9640 1 -- 172.21.15.84:0/2700826255 shutdown_connections 2023-10-16T11:25:54.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.343+0000 7f26a05a9640 1 --2- 172.21.15.84:0/2700826255 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f267c06b0b0 0x7f267c06d570 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:54.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.343+0000 7f26a05a9640 1 --2- 172.21.15.84:0/2700826255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2698160b90 0x7f26980c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:54.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.343+0000 7f26a05a9640 1 --2- 172.21.15.84:0/2700826255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f269815d150 0x7f26980c6e90 unknown :-1 s=CLOSED pgs=2986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:54.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.343+0000 7f26a05a9640 1 --2- 172.21.15.84:0/2700826255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f269806a4b0 0x7f26981f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:54.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.343+0000 7f26a05a9640 1 -- 172.21.15.84:0/2700826255 >> 172.21.15.84:0/2700826255 conn(0x7f26980ce9b0 msgr2=0x7f26980c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:54.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.344+0000 7f26a05a9640 1 -- 172.21.15.84:0/2700826255 shutdown_connections 2023-10-16T11:25:54.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.344+0000 7f26a05a9640 1 -- 172.21.15.84:0/2700826255 wait complete. 2023-10-16T11:25:54.366 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2023-10-16T11:25:54.367 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:25:54.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.573+0000 7f7b4a62e640 1 Processor -- start 2023-10-16T11:25:54.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.573+0000 7f7b4a62e640 1 -- start start 2023-10-16T11:25:54.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.575+0000 7f7b4a62e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b4406c9e0 0x7f7b440ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:54.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.575+0000 7f7b4a62e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b440ca560 con 0x7f7b440c9060 2023-10-16T11:25:54.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.575+0000 7f7b4a62e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b440c9950 con 0x7f7b4406fb50 2023-10-16T11:25:54.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.575+0000 7f7b4a62e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b440c9c30 con 0x7f7b4406c9e0 2023-10-16T11:25:54.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.575+0000 7f7b4962c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7b440c9060 0x7f7b4406f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58056/0 (socket says 172.21.15.84:58056) 2023-10-16T11:25:54.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.575+0000 7f7b4962c640 1 -- 172.21.15.84:0/3566958271 learned_addr learned my addr 172.21.15.84:0/3566958271 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:54.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.576+0000 7f7b3bfff640 1 -- 172.21.15.84:0/3566958271 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935050186 0 0) 0x7f7b440ca560 con 0x7f7b440c9060 2023-10-16T11:25:54.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.576+0000 7f7b3bfff640 1 -- 172.21.15.84:0/3566958271 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b30003520 con 0x7f7b440c9060 2023-10-16T11:25:54.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.576+0000 7f7b49e2d640 1 --2- 172.21.15.84:0/3566958271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b4406c9e0 0x7f7b440ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:54.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.576+0000 7f7b3bfff640 1 -- 172.21.15.84:0/3566958271 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1352117061 0 0) 0x7f7b30003520 con 0x7f7b440c9060 2023-10-16T11:25:54.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.576+0000 7f7b3bfff640 1 -- 172.21.15.84:0/3566958271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b4406c9e0 msgr2=0x7f7b440ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:54.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.576+0000 7f7b3bfff640 1 --2- 172.21.15.84:0/3566958271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b4406c9e0 0x7f7b440ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:54.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.576+0000 7f7b3bfff640 1 -- 172.21.15.84:0/3566958271 >> v1:172.21.15.157:6789/0 conn(0x7f7b4406fb50 legacy=0x7f7b440cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:54.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.576+0000 7f7b3bfff640 1 -- 172.21.15.84:0/3566958271 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b440c9f10 con 0x7f7b440c9060 2023-10-16T11:25:54.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.576+0000 7f7b3bfff640 1 -- 172.21.15.84:0/3566958271 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f7b40001670 con 0x7f7b440c9060 2023-10-16T11:25:54.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.576+0000 7f7b3bfff640 1 -- 172.21.15.84:0/3566958271 >> v1:172.21.15.84:6789/0 conn(0x7f7b440c9060 legacy=0x7f7b4406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:54.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.576+0000 7f7b3bfff640 1 --2- 172.21.15.84:0/3566958271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b300039a0 0x7f7b30003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:54.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.576+0000 7f7b3bfff640 1 --2- 172.21.15.84:0/3566958271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b4406c9e0 0x7f7b300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:54.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.576+0000 7f7b3bfff640 1 --2- 172.21.15.84:0/3566958271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b30004d00 0x7f7b30005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:54.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.576+0000 7f7b3bfff640 1 -- 172.21.15.84:0/3566958271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b440c9c30 con 0x7f7b300039a0 2023-10-16T11:25:54.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.576+0000 7f7b3bfff640 1 -- 172.21.15.84:0/3566958271 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3000bb60 con 0x7f7b30004d00 2023-10-16T11:25:54.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.576+0000 7f7b3bfff640 1 -- 172.21.15.84:0/3566958271 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3000bd20 con 0x7f7b4406c9e0 2023-10-16T11:25:54.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.576+0000 7f7b4962c640 1 --2- 172.21.15.84:0/3566958271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b300039a0 0x7f7b30003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:54.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.577+0000 7f7b4962c640 1 -- 172.21.15.84:0/3566958271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b4406c9e0 msgr2=0x7f7b300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:54.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.577+0000 7f7b4962c640 1 --2- 172.21.15.84:0/3566958271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b4406c9e0 0x7f7b300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:54.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.577+0000 7f7b4962c640 1 -- 172.21.15.84:0/3566958271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b30004d00 msgr2=0x7f7b30005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:54.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.577+0000 7f7b4962c640 1 --2- 172.21.15.84:0/3566958271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b30004d00 0x7f7b30005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:54.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.577+0000 7f7b4962c640 1 -- 172.21.15.84:0/3566958271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b30003520 con 0x7f7b300039a0 2023-10-16T11:25:54.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.578+0000 7f7b4962c640 1 --2- 172.21.15.84:0/3566958271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b300039a0 0x7f7b30003dc0 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000c0e0 tx=0x7f7b300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4322e7f244f2353 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:54.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.578+0000 7f7b3bfff640 1 -- 172.21.15.84:0/3566958271 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b40005ad0 con 0x7f7b300039a0 2023-10-16T11:25:54.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.578+0000 7f7b3bfff640 1 -- 172.21.15.84:0/3566958271 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b40005c30 con 0x7f7b300039a0 2023-10-16T11:25:54.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.578+0000 7f7b3bfff640 1 -- 172.21.15.84:0/3566958271 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4002c810 con 0x7f7b300039a0 2023-10-16T11:25:54.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.578+0000 7f7b4a62e640 1 -- 172.21.15.84:0/3566958271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b300039a0 msgr2=0x7f7b30003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:54.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.578+0000 7f7b4a62e640 1 --2- 172.21.15.84:0/3566958271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b300039a0 0x7f7b30003dc0 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000c0e0 tx=0x7f7b300057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:54.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.579+0000 7f7b49e2d640 1 -- 172.21.15.84:0/3566958271 reap_dead start 2023-10-16T11:25:54.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.579+0000 7f7b4a62e640 1 -- 172.21.15.84:0/3566958271 shutdown_connections 2023-10-16T11:25:54.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.579+0000 7f7b4a62e640 1 -- 172.21.15.84:0/3566958271 >> 172.21.15.84:0/3566958271 conn(0x7f7b440ce6a0 msgr2=0x7f7b440ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:54.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.579+0000 7f7b4a62e640 1 -- 172.21.15.84:0/3566958271 shutdown_connections 2023-10-16T11:25:54.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.579+0000 7f7b4a62e640 1 -- 172.21.15.84:0/3566958271 wait complete. 2023-10-16T11:25:54.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.580+0000 7f7b4a62e640 1 Processor -- start 2023-10-16T11:25:54.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.581+0000 7f7b4a62e640 1 -- start start 2023-10-16T11:25:54.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.582+0000 7f7b4a62e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b300039a0 0x7f7b440b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:54.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.582+0000 7f7b4a62e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b441640b0 0x7f7b44161510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:54.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.582+0000 7f7b4a62e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b44164530 0x7f7b44161a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:54.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.582+0000 7f7b4a62e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b440b4410 con 0x7f7b300039a0 2023-10-16T11:25:54.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.582+0000 7f7b4a62e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b44163650 con 0x7f7b44164530 2023-10-16T11:25:54.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.582+0000 7f7b4a62e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b441638f0 con 0x7f7b441640b0 2023-10-16T11:25:54.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.582+0000 7f7b4962c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b300039a0 0x7f7b440b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:54.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.582+0000 7f7b4962c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b300039a0 0x7f7b440b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34962/0 (socket says 172.21.15.84:34962) 2023-10-16T11:25:54.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.582+0000 7f7b4962c640 1 -- 172.21.15.84:0/699986018 learned_addr learned my addr 172.21.15.84:0/699986018 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:54.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.582+0000 7f7b48e2b640 1 --2- 172.21.15.84:0/699986018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b441640b0 0x7f7b44161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:54.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.582+0000 7f7b4962c640 1 -- 172.21.15.84:0/699986018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b441640b0 msgr2=0x7f7b44161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:54.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.582+0000 7f7b4962c640 1 --2- 172.21.15.84:0/699986018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b441640b0 0x7f7b44161510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:54.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.582+0000 7f7b4962c640 1 -- 172.21.15.84:0/699986018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b44164530 msgr2=0x7f7b44161a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:54.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.582+0000 7f7b4962c640 1 --2- 172.21.15.84:0/699986018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b44164530 0x7f7b44161a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:54.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.582+0000 7f7b4962c640 1 -- 172.21.15.84:0/699986018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b440b3e50 con 0x7f7b300039a0 2023-10-16T11:25:54.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.583+0000 7f7b4962c640 1 --2- 172.21.15.84:0/699986018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b300039a0 0x7f7b440b1c70 secure :-1 s=READY pgs=2989 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000c0e0 tx=0x7f7b40001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:54.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.583+0000 7f7b39ffb640 1 -- 172.21.15.84:0/699986018 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4003c040 con 0x7f7b300039a0 2023-10-16T11:25:54.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.583+0000 7f7b39ffb640 1 -- 172.21.15.84:0/699986018 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4002dcb0 con 0x7f7b300039a0 2023-10-16T11:25:54.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.583+0000 7f7b39ffb640 1 -- 172.21.15.84:0/699986018 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b40007270 con 0x7f7b300039a0 2023-10-16T11:25:54.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.583+0000 7f7b4a62e640 1 -- 172.21.15.84:0/699986018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b440b4130 con 0x7f7b300039a0 2023-10-16T11:25:54.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.583+0000 7f7b4a62e640 1 -- 172.21.15.84:0/699986018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b440b21b0 con 0x7f7b300039a0 2023-10-16T11:25:54.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.584+0000 7f7b4a62e640 1 -- 172.21.15.84:0/699986018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b4404e9b0 con 0x7f7b300039a0 2023-10-16T11:25:54.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.585+0000 7f7b39ffb640 1 -- 172.21.15.84:0/699986018 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f7b40032070 con 0x7f7b300039a0 2023-10-16T11:25:54.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.586+0000 7f7b39ffb640 1 --2- 172.21.15.84:0/699986018 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f7b2806b0b0 0x7f7b2806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:54.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.587+0000 7f7b39ffb640 1 -- 172.21.15.84:0/699986018 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f7b400ab7f0 con 0x7f7b300039a0 2023-10-16T11:25:54.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.587+0000 7f7b48e2b640 1 --2- 172.21.15.84:0/699986018 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f7b2806b0b0 0x7f7b2806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:54.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.587+0000 7f7b48e2b640 1 --2- 172.21.15.84:0/699986018 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f7b2806b0b0 0x7f7b2806d570 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7b441f7840 tx=0x7f7b34009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:54.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.591+0000 7f7b39ffb640 1 -- 172.21.15.84:0/699986018 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7b40076250 con 0x7f7b300039a0 2023-10-16T11:25:54.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.821+0000 7f7b4a62e640 1 -- 172.21.15.84:0/699986018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b440b4620 con 0x7f7b300039a0 2023-10-16T11:25:54.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.822+0000 7f7b39ffb640 1 -- 172.21.15.84:0/699986018 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+6155 (secure 0 0 0) 0x7f7b40079870 con 0x7f7b300039a0 2023-10-16T11:25:54.825 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:54.825 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":137,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2516077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2516077743},{"type":"v1","addr":"172.21.15.157:6833","nonce":2516077743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1527104462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1527104462},{"type":"v1","addr":"172.21.15.157:6835","nonce":1527104462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:25:36.605068+0000","modified":"2023-10-16T11:25:39.675208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12186":{"gid":12186,"name":"a","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/3547503033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3547503033},{"type":"v1","addr":"172.21.15.84:6837","nonce":3547503033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:25:50.739234+0000","modified":"2023-10-16T11:25:53.792071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12183":{"gid":12183,"name":"c","rank":0,"incarnation":133,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/369067539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":369067539},{"type":"v1","addr":"172.21.15.84:6835","nonce":369067539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-16T11:25:54.825 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 137 2023-10-16T11:25:54.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.825+0000 7f7b4a62e640 1 -- 172.21.15.84:0/699986018 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f7b2806b0b0 msgr2=0x7f7b2806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:54.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.825+0000 7f7b4a62e640 1 --2- 172.21.15.84:0/699986018 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f7b2806b0b0 0x7f7b2806d570 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7b441f7840 tx=0x7f7b34009040 comp rx=0 tx=0).stop 2023-10-16T11:25:54.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.825+0000 7f7b4a62e640 1 -- 172.21.15.84:0/699986018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b300039a0 msgr2=0x7f7b440b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:54.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.825+0000 7f7b4a62e640 1 --2- 172.21.15.84:0/699986018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b300039a0 0x7f7b440b1c70 secure :-1 s=READY pgs=2989 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000c0e0 tx=0x7f7b40001930 comp rx=0 tx=0).stop 2023-10-16T11:25:54.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.825+0000 7f7b4a62e640 1 -- 172.21.15.84:0/699986018 shutdown_connections 2023-10-16T11:25:54.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.825+0000 7f7b4a62e640 1 --2- 172.21.15.84:0/699986018 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f7b2806b0b0 0x7f7b2806d570 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:54.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.825+0000 7f7b4a62e640 1 --2- 172.21.15.84:0/699986018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b44164530 0x7f7b44161a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:54.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.825+0000 7f7b4a62e640 1 --2- 172.21.15.84:0/699986018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7b441640b0 0x7f7b44161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:54.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.825+0000 7f7b4a62e640 1 --2- 172.21.15.84:0/699986018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b300039a0 0x7f7b440b1c70 unknown :-1 s=CLOSED pgs=2989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:54.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.825+0000 7f7b4a62e640 1 -- 172.21.15.84:0/699986018 >> 172.21.15.84:0/699986018 conn(0x7f7b440ce6a0 msgr2=0x7f7b4406a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:54.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.825+0000 7f7b4a62e640 1 -- 172.21.15.84:0/699986018 shutdown_connections 2023-10-16T11:25:54.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:54.825+0000 7f7b4a62e640 1 -- 172.21.15.84:0/699986018 wait complete. 2023-10-16T11:25:54.844 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:25:55.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.045+0000 7f48227b0640 1 Processor -- start 2023-10-16T11:25:55.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.045+0000 7f48227b0640 1 -- start start 2023-10-16T11:25:55.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.047+0000 7f48227b0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f481c159b20 0x7f481c06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:55.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.047+0000 7f48227b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f481c159f00 0x7f481c0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:55.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.047+0000 7f48227b0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f481c06f7f0 con 0x7f481c06c120 2023-10-16T11:25:55.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.047+0000 7f48227b0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f481c06c5e0 con 0x7f481c159b20 2023-10-16T11:25:55.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.047+0000 7f48227b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481c06c8c0 con 0x7f481c159f00 2023-10-16T11:25:55.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.047+0000 7f4813fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f481c159f00 0x7f481c0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:55.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.048+0000 7f4813fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f481c159f00 0x7f481c0b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:38916/0 (socket says 172.21.15.84:38916) 2023-10-16T11:25:55.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.048+0000 7f4820d4d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f481c06c120 0x7f481c0cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:44462/0 (socket says 172.21.15.84:44462) 2023-10-16T11:25:55.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.048+0000 7f4813fff640 1 -- 172.21.15.84:0/2354030619 learned_addr learned my addr 172.21.15.84:0/2354030619 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:55.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.048+0000 7f481bfff640 1 --2- 172.21.15.84:0/2354030619 >> v2:172.21.15.157:3300/0 conn(0x7f481c159b20 0x7f481c06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:55.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.048+0000 7f481b7fe640 1 -- 172.21.15.84:0/2354030619 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372857916 0 0) 0x7f481c06f7f0 con 0x7f481c06c120 2023-10-16T11:25:55.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.048+0000 7f481b7fe640 1 -- 172.21.15.84:0/2354030619 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4808005730 con 0x7f481c06c120 2023-10-16T11:25:55.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.048+0000 7f4813fff640 1 -- 172.21.15.84:0/2354030619 >> v2:172.21.15.157:3300/0 conn(0x7f481c159b20 msgr2=0x7f481c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:55.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.048+0000 7f4813fff640 1 --2- 172.21.15.84:0/2354030619 >> v2:172.21.15.157:3300/0 conn(0x7f481c159b20 0x7f481c06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:55.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.048+0000 7f4813fff640 1 -- 172.21.15.84:0/2354030619 >> v1:172.21.15.157:6789/0 conn(0x7f481c06c120 legacy=0x7f481c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:55.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.048+0000 7f4813fff640 1 -- 172.21.15.84:0/2354030619 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c06cba0 con 0x7f481c159f00 2023-10-16T11:25:55.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.050+0000 7f4813fff640 1 --2- 172.21.15.84:0/2354030619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f481c159f00 0x7f481c0b0410 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4804009870 tx=0x7f480402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bc78a74e388a570 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:55.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.050+0000 7f481b7fe640 1 -- 172.21.15.84:0/2354030619 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4804030840 con 0x7f481c159f00 2023-10-16T11:25:55.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.050+0000 7f481b7fe640 1 -- 172.21.15.84:0/2354030619 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48040309a0 con 0x7f481c159f00 2023-10-16T11:25:55.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.051+0000 7f481b7fe640 1 -- 172.21.15.84:0/2354030619 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4804030d10 con 0x7f481c159f00 2023-10-16T11:25:55.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.051+0000 7f48227b0640 1 -- 172.21.15.84:0/2354030619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f481c159f00 msgr2=0x7f481c0b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:55.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.051+0000 7f48227b0640 1 --2- 172.21.15.84:0/2354030619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f481c159f00 0x7f481c0b0410 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4804009870 tx=0x7f480402b980 comp rx=0 tx=0).stop 2023-10-16T11:25:55.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.051+0000 7f48227b0640 1 -- 172.21.15.84:0/2354030619 shutdown_connections 2023-10-16T11:25:55.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.051+0000 7f48227b0640 1 --2- 172.21.15.84:0/2354030619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f481c159f00 0x7f481c0b0410 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:55.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.051+0000 7f48227b0640 1 --2- 172.21.15.84:0/2354030619 >> v2:172.21.15.157:3300/0 conn(0x7f481c159b20 0x7f481c06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:55.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.051+0000 7f48227b0640 1 -- 172.21.15.84:0/2354030619 >> 172.21.15.84:0/2354030619 conn(0x7f481c0cb410 msgr2=0x7f481c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:55.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.051+0000 7f48227b0640 1 -- 172.21.15.84:0/2354030619 shutdown_connections 2023-10-16T11:25:55.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.051+0000 7f48227b0640 1 -- 172.21.15.84:0/2354030619 wait complete. 2023-10-16T11:25:55.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.053+0000 7f48227b0640 1 Processor -- start 2023-10-16T11:25:55.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.053+0000 7f48227b0640 1 -- start start 2023-10-16T11:25:55.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.054+0000 7f48227b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f481c06c120 0x7f481c151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:55.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.054+0000 7f48227b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f481c159b20 0x7f481c0c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:55.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.054+0000 7f48227b0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f481c159f00 0x7f481c151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:55.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.054+0000 7f48227b0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c0aeed0 con 0x7f481c159f00 2023-10-16T11:25:55.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.054+0000 7f48227b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c1516b0 con 0x7f481c159b20 2023-10-16T11:25:55.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.054+0000 7f48227b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481c0af190 con 0x7f481c06c120 2023-10-16T11:25:55.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.054+0000 7f4820d4d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f481c159f00 0x7f481c151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:55.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.054+0000 7f4820d4d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f481c159f00 0x7f481c151170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34978/0 (socket says 172.21.15.84:34978) 2023-10-16T11:25:55.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.054+0000 7f4820d4d640 1 -- 172.21.15.84:0/128102859 learned_addr learned my addr 172.21.15.84:0/128102859 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.055+0000 7f4813fff640 1 --2- 172.21.15.84:0/128102859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f481c159b20 0x7f481c0c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.055+0000 7f4820d4d640 1 -- 172.21.15.84:0/128102859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f481c06c120 msgr2=0x7f481c151900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.055+0000 7f481bfff640 1 --2- 172.21.15.84:0/128102859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f481c06c120 0x7f481c151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.055+0000 7f4820d4d640 1 --2- 172.21.15.84:0/128102859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f481c06c120 0x7f481c151900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:55.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.055+0000 7f4820d4d640 1 -- 172.21.15.84:0/128102859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f481c159b20 msgr2=0x7f481c0c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:55.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.055+0000 7f4820d4d640 1 --2- 172.21.15.84:0/128102859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f481c159b20 0x7f481c0c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:55.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.055+0000 7f4820d4d640 1 -- 172.21.15.84:0/128102859 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48040094d0 con 0x7f481c159f00 2023-10-16T11:25:55.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.055+0000 7f4820d4d640 1 --2- 172.21.15.84:0/128102859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f481c159f00 0x7f481c151170 secure :-1 s=READY pgs=2990 cs=0 l=1 rev1=1 crypto rx=0x7f481400c7a0 tx=0x7f481400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:55.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.055+0000 7f48197fa640 1 -- 172.21.15.84:0/128102859 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481400a640 con 0x7f481c159f00 2023-10-16T11:25:55.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.055+0000 7f48197fa640 1 -- 172.21.15.84:0/128102859 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481400a7a0 con 0x7f481c159f00 2023-10-16T11:25:55.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.055+0000 7f48227b0640 1 -- 172.21.15.84:0/128102859 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481c0af470 con 0x7f481c159f00 2023-10-16T11:25:55.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.055+0000 7f48197fa640 1 -- 172.21.15.84:0/128102859 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481400ab70 con 0x7f481c159f00 2023-10-16T11:25:55.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.055+0000 7f48227b0640 1 -- 172.21.15.84:0/128102859 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481c0afa30 con 0x7f481c159f00 2023-10-16T11:25:55.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.057+0000 7f48227b0640 1 -- 172.21.15.84:0/128102859 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f481c0afc20 con 0x7f481c159f00 2023-10-16T11:25:55.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.058+0000 7f48197fa640 1 -- 172.21.15.84:0/128102859 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f481400acd0 con 0x7f481c159f00 2023-10-16T11:25:55.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.059+0000 7f48197fa640 1 --2- 172.21.15.84:0/128102859 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f47fc06b0b0 0x7f47fc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:55.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.059+0000 7f48197fa640 1 -- 172.21.15.84:0/128102859 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f4814089640 con 0x7f481c159f00 2023-10-16T11:25:55.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.059+0000 7f481bfff640 1 --2- 172.21.15.84:0/128102859 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f47fc06b0b0 0x7f47fc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:55.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.060+0000 7f481bfff640 1 --2- 172.21.15.84:0/128102859 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f47fc06b0b0 0x7f47fc06d570 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f480c009a10 tx=0x7f480c008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:55.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.064+0000 7f48197fa640 1 -- 172.21.15.84:0/128102859 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4814054010 con 0x7f481c159f00 2023-10-16T11:25:55.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.258+0000 7f48227b0640 1 -- 172.21.15.84:0/128102859 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f481c04e160 con 0x7f481c159f00 2023-10-16T11:25:55.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.262+0000 7f48197fa640 1 -- 172.21.15.84:0/128102859 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v241) v1 ==== 74+0+20207 (secure 0 0 0) 0x7f4814057630 con 0x7f481c159f00 2023-10-16T11:25:55.265 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:55.265 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":241,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:25:52.306030+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:25:34.443897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:25:35.635615+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:25:49.001107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:25:49.760637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:25:55.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.264+0000 7f48227b0640 1 -- 172.21.15.84:0/128102859 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f47fc06b0b0 msgr2=0x7f47fc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:55.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.265+0000 7f48227b0640 1 --2- 172.21.15.84:0/128102859 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f47fc06b0b0 0x7f47fc06d570 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f480c009a10 tx=0x7f480c008040 comp rx=0 tx=0).stop 2023-10-16T11:25:55.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.265+0000 7f48227b0640 1 -- 172.21.15.84:0/128102859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f481c159f00 msgr2=0x7f481c151170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:55.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.265+0000 7f48227b0640 1 --2- 172.21.15.84:0/128102859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f481c159f00 0x7f481c151170 secure :-1 s=READY pgs=2990 cs=0 l=1 rev1=1 crypto rx=0x7f481400c7a0 tx=0x7f481400cc60 comp rx=0 tx=0).stop 2023-10-16T11:25:55.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.265+0000 7f48227b0640 1 -- 172.21.15.84:0/128102859 shutdown_connections 2023-10-16T11:25:55.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.265+0000 7f48227b0640 1 --2- 172.21.15.84:0/128102859 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f47fc06b0b0 0x7f47fc06d570 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:55.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.265+0000 7f48227b0640 1 --2- 172.21.15.84:0/128102859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f481c159f00 0x7f481c151170 unknown :-1 s=CLOSED pgs=2990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:55.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.265+0000 7f48227b0640 1 --2- 172.21.15.84:0/128102859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f481c159b20 0x7f481c0c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:55.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.265+0000 7f48227b0640 1 --2- 172.21.15.84:0/128102859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f481c06c120 0x7f481c151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:55.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.265+0000 7f48227b0640 1 -- 172.21.15.84:0/128102859 >> 172.21.15.84:0/128102859 conn(0x7f481c0cb410 msgr2=0x7f481c0cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:55.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.265+0000 7f48227b0640 1 -- 172.21.15.84:0/128102859 shutdown_connections 2023-10-16T11:25:55.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.265+0000 7f48227b0640 1 -- 172.21.15.84:0/128102859 wait complete. 2023-10-16T11:25:55.286 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T11:25:55.287 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:25:55.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.499+0000 7ffbd4bdc640 1 Processor -- start 2023-10-16T11:25:55.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.499+0000 7ffbd4bdc640 1 -- start start 2023-10-16T11:25:55.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.501+0000 7ffbd4bdc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ffbd015aaa0 0x7ffbd015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:55.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.501+0000 7ffbd4bdc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffbd015ae80 0x7ffbd00cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:55.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.501+0000 7ffbd4bdc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd015ed60 con 0x7ffbd00ccaa0 2023-10-16T11:25:55.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.501+0000 7ffbd4bdc640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd006b810 con 0x7ffbd015aaa0 2023-10-16T11:25:55.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.501+0000 7ffbd4bdc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd00707e0 con 0x7ffbd015ae80 2023-10-16T11:25:55.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.501+0000 7ffbce59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ffbd015aaa0 0x7ffbd015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:55.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.501+0000 7ffbce59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ffbd015aaa0 0x7ffbd015e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34982/0 (socket says 172.21.15.84:34982) 2023-10-16T11:25:55.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.501+0000 7ffbce59c640 1 -- 172.21.15.84:0/3400013875 learned_addr learned my addr 172.21.15.84:0/3400013875 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:55.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.502+0000 7ffbce59c640 1 -- 172.21.15.84:0/3400013875 >> v2:172.21.15.157:3300/0 conn(0x7ffbd015ae80 msgr2=0x7ffbd00cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:55.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.502+0000 7ffbc5d9b640 1 --2- 172.21.15.84:0/3400013875 >> v2:172.21.15.157:3300/0 conn(0x7ffbd015ae80 0x7ffbd00cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:55.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.502+0000 7ffbce59c640 1 --2- 172.21.15.84:0/3400013875 >> v2:172.21.15.157:3300/0 conn(0x7ffbd015ae80 0x7ffbd00cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:55.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.502+0000 7ffbce59c640 1 -- 172.21.15.84:0/3400013875 >> v1:172.21.15.157:6789/0 conn(0x7ffbd00ccaa0 legacy=0x7ffbd0070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:55.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.502+0000 7ffbce59c640 1 -- 172.21.15.84:0/3400013875 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd015ef90 con 0x7ffbd015aaa0 2023-10-16T11:25:55.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.504+0000 7ffbce59c640 1 --2- 172.21.15.84:0/3400013875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd015aaa0 0x7ffbd015e820 secure :-1 s=READY pgs=2991 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00c59c0 tx=0x7ffbc002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf8d7d78c625b32 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:55.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.504+0000 7ffbcdd9b640 1 -- 172.21.15.84:0/3400013875 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0031870 con 0x7ffbd015aaa0 2023-10-16T11:25:55.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.504+0000 7ffbcdd9b640 1 -- 172.21.15.84:0/3400013875 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc00319d0 con 0x7ffbd015aaa0 2023-10-16T11:25:55.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.504+0000 7ffbcdd9b640 1 -- 172.21.15.84:0/3400013875 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0031d60 con 0x7ffbd015aaa0 2023-10-16T11:25:55.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.505+0000 7ffbd4bdc640 1 -- 172.21.15.84:0/3400013875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd015aaa0 msgr2=0x7ffbd015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:55.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.505+0000 7ffbd4bdc640 1 --2- 172.21.15.84:0/3400013875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd015aaa0 0x7ffbd015e820 secure :-1 s=READY pgs=2991 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00c59c0 tx=0x7ffbc002b910 comp rx=0 tx=0).stop 2023-10-16T11:25:55.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.505+0000 7ffbd4bdc640 1 -- 172.21.15.84:0/3400013875 shutdown_connections 2023-10-16T11:25:55.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.505+0000 7ffbd4bdc640 1 --2- 172.21.15.84:0/3400013875 >> v2:172.21.15.157:3300/0 conn(0x7ffbd015ae80 0x7ffbd00cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:55.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.505+0000 7ffbd4bdc640 1 --2- 172.21.15.84:0/3400013875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd015aaa0 0x7ffbd015e820 unknown :-1 s=CLOSED pgs=2991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:55.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.505+0000 7ffbd4bdc640 1 -- 172.21.15.84:0/3400013875 >> 172.21.15.84:0/3400013875 conn(0x7ffbd00cb410 msgr2=0x7ffbd00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:55.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.505+0000 7ffbd4bdc640 1 -- 172.21.15.84:0/3400013875 shutdown_connections 2023-10-16T11:25:55.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.505+0000 7ffbd4bdc640 1 -- 172.21.15.84:0/3400013875 wait complete. 2023-10-16T11:25:55.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.507+0000 7ffbd4bdc640 1 Processor -- start 2023-10-16T11:25:55.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.507+0000 7ffbd4bdc640 1 -- start start 2023-10-16T11:25:55.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.508+0000 7ffbd4bdc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffbd00ccaa0 0x7ffbd015cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:55.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.508+0000 7ffbd4bdc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbd015aaa0 0x7ffbd0157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:55.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.508+0000 7ffbd4bdc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd015ae80 0x7ffbd0155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:55.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.508+0000 7ffbd4bdc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd00c3050 con 0x7ffbd015ae80 2023-10-16T11:25:55.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.508+0000 7ffbd4bdc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd015d360 con 0x7ffbd015aaa0 2023-10-16T11:25:55.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.508+0000 7ffbd4bdc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd015d640 con 0x7ffbd00ccaa0 2023-10-16T11:25:55.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.508+0000 7ffbced9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd015ae80 0x7ffbd0155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:55.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.508+0000 7ffbced9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd015ae80 0x7ffbd0155be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34994/0 (socket says 172.21.15.84:34994) 2023-10-16T11:25:55.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.508+0000 7ffbced9d640 1 -- 172.21.15.84:0/2620235995 learned_addr learned my addr 172.21.15.84:0/2620235995 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:55.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.509+0000 7ffbced9d640 1 -- 172.21.15.84:0/2620235995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffbd00ccaa0 msgr2=0x7ffbd015cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:55.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.509+0000 7ffbc5d9b640 1 --2- 172.21.15.84:0/2620235995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbd015aaa0 0x7ffbd0157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:55.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.509+0000 7ffbce59c640 1 --2- 172.21.15.84:0/2620235995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffbd00ccaa0 0x7ffbd015cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:55.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.509+0000 7ffbced9d640 1 --2- 172.21.15.84:0/2620235995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffbd00ccaa0 0x7ffbd015cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:55.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.509+0000 7ffbced9d640 1 -- 172.21.15.84:0/2620235995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbd015aaa0 msgr2=0x7ffbd0157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:55.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.509+0000 7ffbced9d640 1 --2- 172.21.15.84:0/2620235995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbd015aaa0 0x7ffbd0157110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:55.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.509+0000 7ffbced9d640 1 -- 172.21.15.84:0/2620235995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc00094d0 con 0x7ffbd015ae80 2023-10-16T11:25:55.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.509+0000 7ffbced9d640 1 --2- 172.21.15.84:0/2620235995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd015ae80 0x7ffbd0155be0 secure :-1 s=READY pgs=2992 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800c780 tx=0x7ffbc800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:55.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.509+0000 7ffbc77fe640 1 -- 172.21.15.84:0/2620235995 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8004b60 con 0x7ffbd015ae80 2023-10-16T11:25:55.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.509+0000 7ffbc77fe640 1 -- 172.21.15.84:0/2620235995 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc800ce90 con 0x7ffbd015ae80 2023-10-16T11:25:55.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.509+0000 7ffbd4bdc640 1 -- 172.21.15.84:0/2620235995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd015d920 con 0x7ffbd015ae80 2023-10-16T11:25:55.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.509+0000 7ffbc77fe640 1 -- 172.21.15.84:0/2620235995 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc800a850 con 0x7ffbd015ae80 2023-10-16T11:25:55.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.510+0000 7ffbd4bdc640 1 -- 172.21.15.84:0/2620235995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd015dc00 con 0x7ffbd015ae80 2023-10-16T11:25:55.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.511+0000 7ffbd4bdc640 1 -- 172.21.15.84:0/2620235995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd00c3200 con 0x7ffbd015ae80 2023-10-16T11:25:55.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.512+0000 7ffbc77fe640 1 -- 172.21.15.84:0/2620235995 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ffbc800a9b0 con 0x7ffbd015ae80 2023-10-16T11:25:55.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.513+0000 7ffbc77fe640 1 --2- 172.21.15.84:0/2620235995 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7ffbb006b0b0 0x7ffbb006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:55.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.513+0000 7ffbc77fe640 1 -- 172.21.15.84:0/2620235995 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7ffbc8089fb0 con 0x7ffbd015ae80 2023-10-16T11:25:55.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.513+0000 7ffbce59c640 1 --2- 172.21.15.84:0/2620235995 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7ffbb006b0b0 0x7ffbb006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:55.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.514+0000 7ffbce59c640 1 --2- 172.21.15.84:0/2620235995 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7ffbb006b0b0 0x7ffbb006d570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0036000 tx=0x7ffbc003c040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:55.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.518+0000 7ffbc77fe640 1 -- 172.21.15.84:0/2620235995 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffbd00c3200 con 0x7ffbd015ae80 2023-10-16T11:25:55.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.705+0000 7ffbd4bdc640 1 -- 172.21.15.84:0/2620235995 --> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffbd015f550 con 0x7ffbb006b0b0 2023-10-16T11:25:55.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.745+0000 7ffbc77fe640 1 -- 172.21.15.84:0/2620235995 <== mgr.11814 v2:172.21.15.84:6832/2963529201 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+514284 (secure 0 0 0) 0x7ffbc0031cc0 con 0x7ffbb006b0b0 2023-10-16T11:25:55.753 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:55.758 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:25:55.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.755+0000 7ffbd4bdc640 1 -- 172.21.15.84:0/2620235995 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7ffbb006b0b0 msgr2=0x7ffbb006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:55.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.756+0000 7ffbd4bdc640 1 --2- 172.21.15.84:0/2620235995 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7ffbb006b0b0 0x7ffbb006d570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0036000 tx=0x7ffbc003c040 comp rx=0 tx=0).stop 2023-10-16T11:25:55.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.756+0000 7ffbd4bdc640 1 -- 172.21.15.84:0/2620235995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd015ae80 msgr2=0x7ffbd0155be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:55.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.756+0000 7ffbd4bdc640 1 --2- 172.21.15.84:0/2620235995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd015ae80 0x7ffbd0155be0 secure :-1 s=READY pgs=2992 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800c780 tx=0x7ffbc800cc40 comp rx=0 tx=0).stop 2023-10-16T11:25:55.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.756+0000 7ffbd4bdc640 1 -- 172.21.15.84:0/2620235995 shutdown_connections 2023-10-16T11:25:55.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.756+0000 7ffbd4bdc640 1 --2- 172.21.15.84:0/2620235995 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7ffbb006b0b0 0x7ffbb006d570 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:55.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.756+0000 7ffbd4bdc640 1 --2- 172.21.15.84:0/2620235995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd015ae80 0x7ffbd0155be0 unknown :-1 s=CLOSED pgs=2992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:55.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.756+0000 7ffbd4bdc640 1 --2- 172.21.15.84:0/2620235995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbd015aaa0 0x7ffbd0157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:55.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.756+0000 7ffbd4bdc640 1 --2- 172.21.15.84:0/2620235995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffbd00ccaa0 0x7ffbd015cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:55.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.756+0000 7ffbd4bdc640 1 -- 172.21.15.84:0/2620235995 >> 172.21.15.84:0/2620235995 conn(0x7ffbd00cb410 msgr2=0x7ffbd00cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:55.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.756+0000 7ffbd4bdc640 1 -- 172.21.15.84:0/2620235995 shutdown_connections 2023-10-16T11:25:55.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:55.756+0000 7ffbd4bdc640 1 -- 172.21.15.84:0/2620235995 wait complete. 2023-10-16T11:25:55.783 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2023-10-16T11:25:55.248077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594023,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":668,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2830,"ondisk_log_size":2830,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":204,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":430888,"kb_used_data":24648,"kb_used_omap":12,"kb_used_meta":406131,"kb_avail":754543832,"statfs":{"total":773094113280,"available":772652883968,"internally_reserved":0,"allocated":25239552,"data_stored":12885054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":415878713},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":987,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.984522"},"pg_stats":[{"pgid":"33.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.273967+0000","last_change":"2023-10-16T11:25:49.273967+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.273967+0000","last_undegraded":"2023-10-16T11:25:49.273967+0000","last_fullsized":"2023-10-16T11:25:49.273967+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3f","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248777+0000","last_change":"2023-10-16T11:25:37.173492+0000","last_active":"2023-10-16T11:25:40.248777+0000","last_peered":"2023-10-16T11:25:40.248777+0000","last_clean":"2023-10-16T11:25:40.248777+0000","last_became_active":"2023-10-16T11:25:37.167017+0000","last_became_peered":"2023-10-16T11:25:37.167017+0000","last_unstale":"2023-10-16T11:25:40.248777+0000","last_undegraded":"2023-10-16T11:25:40.248777+0000","last_fullsized":"2023-10-16T11:25:40.248777+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:07.791499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.270260+0000","last_change":"2023-10-16T11:25:49.270260+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.270260+0000","last_undegraded":"2023-10-16T11:25:49.270260+0000","last_fullsized":"2023-10-16T11:25:49.270260+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.3e","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248717+0000","last_change":"2023-10-16T11:25:37.179117+0000","last_active":"2023-10-16T11:25:40.248717+0000","last_peered":"2023-10-16T11:25:40.248717+0000","last_clean":"2023-10-16T11:25:40.248717+0000","last_became_active":"2023-10-16T11:25:37.178849+0000","last_became_peered":"2023-10-16T11:25:37.178849+0000","last_unstale":"2023-10-16T11:25:40.248717+0000","last_undegraded":"2023-10-16T11:25:40.248717+0000","last_fullsized":"2023-10-16T11:25:40.248717+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:18:57.278249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.273067+0000","last_change":"2023-10-16T11:25:49.273067+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.273067+0000","last_undegraded":"2023-10-16T11:25:49.273067+0000","last_fullsized":"2023-10-16T11:25:49.273067+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3d","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.250094+0000","last_change":"2023-10-16T11:25:37.179065+0000","last_active":"2023-10-16T11:25:40.250094+0000","last_peered":"2023-10-16T11:25:40.250094+0000","last_clean":"2023-10-16T11:25:40.250094+0000","last_became_active":"2023-10-16T11:25:37.178779+0000","last_became_peered":"2023-10-16T11:25:37.178779+0000","last_unstale":"2023-10-16T11:25:40.250094+0000","last_undegraded":"2023-10-16T11:25:40.250094+0000","last_fullsized":"2023-10-16T11:25:40.250094+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:09:47.367715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.271638+0000","last_change":"2023-10-16T11:25:49.271638+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.271638+0000","last_undegraded":"2023-10-16T11:25:49.271638+0000","last_fullsized":"2023-10-16T11:25:49.271638+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3c","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255257+0000","last_change":"2023-10-16T11:25:37.172480+0000","last_active":"2023-10-16T11:25:40.255257+0000","last_peered":"2023-10-16T11:25:40.255257+0000","last_clean":"2023-10-16T11:25:40.255257+0000","last_became_active":"2023-10-16T11:25:37.172192+0000","last_became_peered":"2023-10-16T11:25:37.172192+0000","last_unstale":"2023-10-16T11:25:40.255257+0000","last_undegraded":"2023-10-16T11:25:40.255257+0000","last_fullsized":"2023-10-16T11:25:40.255257+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.272120+0000","last_change":"2023-10-16T11:25:49.272120+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.272120+0000","last_undegraded":"2023-10-16T11:25:49.272120+0000","last_fullsized":"2023-10-16T11:25:49.272120+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3b","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.257086+0000","last_change":"2023-10-16T11:25:37.177505+0000","last_active":"2023-10-16T11:25:40.257086+0000","last_peered":"2023-10-16T11:25:40.257086+0000","last_clean":"2023-10-16T11:25:40.257086+0000","last_became_active":"2023-10-16T11:25:37.174334+0000","last_became_peered":"2023-10-16T11:25:37.174334+0000","last_unstale":"2023-10-16T11:25:40.257086+0000","last_undegraded":"2023-10-16T11:25:40.257086+0000","last_fullsized":"2023-10-16T11:25:40.257086+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.267494+0000","last_change":"2023-10-16T11:25:49.267494+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.267494+0000","last_undegraded":"2023-10-16T11:25:49.267494+0000","last_fullsized":"2023-10-16T11:25:49.267494+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3a","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255230+0000","last_change":"2023-10-16T11:25:37.177781+0000","last_active":"2023-10-16T11:25:40.255230+0000","last_peered":"2023-10-16T11:25:40.255230+0000","last_clean":"2023-10-16T11:25:40.255230+0000","last_became_active":"2023-10-16T11:25:37.177501+0000","last_became_peered":"2023-10-16T11:25:37.177501+0000","last_unstale":"2023-10-16T11:25:40.255230+0000","last_undegraded":"2023-10-16T11:25:40.255230+0000","last_fullsized":"2023-10-16T11:25:40.255230+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.279696+0000","last_change":"2023-10-16T11:25:49.279696+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.279696+0000","last_undegraded":"2023-10-16T11:25:49.279696+0000","last_fullsized":"2023-10-16T11:25:49.279696+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.39","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252215+0000","last_change":"2023-10-16T11:25:37.182745+0000","last_active":"2023-10-16T11:25:40.252215+0000","last_peered":"2023-10-16T11:25:40.252215+0000","last_clean":"2023-10-16T11:25:40.252215+0000","last_became_active":"2023-10-16T11:25:37.182437+0000","last_became_peered":"2023-10-16T11:25:37.182437+0000","last_unstale":"2023-10-16T11:25:40.252215+0000","last_undegraded":"2023-10-16T11:25:40.252215+0000","last_fullsized":"2023-10-16T11:25:40.252215+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.269438+0000","last_change":"2023-10-16T11:25:49.269438+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.269438+0000","last_undegraded":"2023-10-16T11:25:49.269438+0000","last_fullsized":"2023-10-16T11:25:49.269438+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.38","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254686+0000","last_change":"2023-10-16T11:25:37.174929+0000","last_active":"2023-10-16T11:25:40.254686+0000","last_peered":"2023-10-16T11:25:40.254686+0000","last_clean":"2023-10-16T11:25:40.254686+0000","last_became_active":"2023-10-16T11:25:37.172810+0000","last_became_peered":"2023-10-16T11:25:37.172810+0000","last_unstale":"2023-10-16T11:25:40.254686+0000","last_undegraded":"2023-10-16T11:25:40.254686+0000","last_fullsized":"2023-10-16T11:25:40.254686+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.273300+0000","last_change":"2023-10-16T11:25:49.273300+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.273300+0000","last_undegraded":"2023-10-16T11:25:49.273300+0000","last_fullsized":"2023-10-16T11:25:49.273300+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.37","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.259355+0000","last_change":"2023-10-16T11:25:37.177647+0000","last_active":"2023-10-16T11:25:40.259355+0000","last_peered":"2023-10-16T11:25:40.259355+0000","last_clean":"2023-10-16T11:25:40.259355+0000","last_became_active":"2023-10-16T11:25:37.176875+0000","last_became_peered":"2023-10-16T11:25:37.176875+0000","last_unstale":"2023-10-16T11:25:40.259355+0000","last_undegraded":"2023-10-16T11:25:40.259355+0000","last_fullsized":"2023-10-16T11:25:40.259355+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:21.630921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.283150+0000","last_change":"2023-10-16T11:25:49.283150+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.283150+0000","last_undegraded":"2023-10-16T11:25:49.283150+0000","last_fullsized":"2023-10-16T11:25:49.283150+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.36","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252664+0000","last_change":"2023-10-16T11:25:37.172841+0000","last_active":"2023-10-16T11:25:40.252664+0000","last_peered":"2023-10-16T11:25:40.252664+0000","last_clean":"2023-10-16T11:25:40.252664+0000","last_became_active":"2023-10-16T11:25:37.172550+0000","last_became_peered":"2023-10-16T11:25:37.172550+0000","last_unstale":"2023-10-16T11:25:40.252664+0000","last_undegraded":"2023-10-16T11:25:40.252664+0000","last_fullsized":"2023-10-16T11:25:40.252664+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.274776+0000","last_change":"2023-10-16T11:25:49.274776+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.274776+0000","last_undegraded":"2023-10-16T11:25:49.274776+0000","last_fullsized":"2023-10-16T11:25:49.274776+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.35","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249412+0000","last_change":"2023-10-16T11:25:37.181386+0000","last_active":"2023-10-16T11:25:40.249412+0000","last_peered":"2023-10-16T11:25:40.249412+0000","last_clean":"2023-10-16T11:25:40.249412+0000","last_became_active":"2023-10-16T11:25:37.181007+0000","last_became_peered":"2023-10-16T11:25:37.181007+0000","last_unstale":"2023-10-16T11:25:40.249412+0000","last_undegraded":"2023-10-16T11:25:40.249412+0000","last_fullsized":"2023-10-16T11:25:40.249412+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:46.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.277979+0000","last_change":"2023-10-16T11:25:49.277979+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.277979+0000","last_undegraded":"2023-10-16T11:25:49.277979+0000","last_fullsized":"2023-10-16T11:25:49.277979+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.34","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254920+0000","last_change":"2023-10-16T11:25:37.179103+0000","last_active":"2023-10-16T11:25:40.254920+0000","last_peered":"2023-10-16T11:25:40.254920+0000","last_clean":"2023-10-16T11:25:40.254920+0000","last_became_active":"2023-10-16T11:25:37.178445+0000","last_became_peered":"2023-10-16T11:25:37.178445+0000","last_unstale":"2023-10-16T11:25:40.254920+0000","last_undegraded":"2023-10-16T11:25:40.254920+0000","last_fullsized":"2023-10-16T11:25:40.254920+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:38.218704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.269375+0000","last_change":"2023-10-16T11:25:49.269375+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.269375+0000","last_undegraded":"2023-10-16T11:25:49.269375+0000","last_fullsized":"2023-10-16T11:25:49.269375+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.33","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252560+0000","last_change":"2023-10-16T11:25:37.182921+0000","last_active":"2023-10-16T11:25:40.252560+0000","last_peered":"2023-10-16T11:25:40.252560+0000","last_clean":"2023-10-16T11:25:40.252560+0000","last_became_active":"2023-10-16T11:25:37.182603+0000","last_became_peered":"2023-10-16T11:25:37.182603+0000","last_unstale":"2023-10-16T11:25:40.252560+0000","last_undegraded":"2023-10-16T11:25:40.252560+0000","last_fullsized":"2023-10-16T11:25:40.252560+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.283709+0000","last_change":"2023-10-16T11:25:49.283709+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.283709+0000","last_undegraded":"2023-10-16T11:25:49.283709+0000","last_fullsized":"2023-10-16T11:25:49.283709+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.32","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248649+0000","last_change":"2023-10-16T11:25:37.171715+0000","last_active":"2023-10-16T11:25:40.248649+0000","last_peered":"2023-10-16T11:25:40.248649+0000","last_clean":"2023-10-16T11:25:40.248649+0000","last_became_active":"2023-10-16T11:25:37.171421+0000","last_became_peered":"2023-10-16T11:25:37.171421+0000","last_unstale":"2023-10-16T11:25:40.248649+0000","last_undegraded":"2023-10-16T11:25:40.248649+0000","last_fullsized":"2023-10-16T11:25:40.248649+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:10.207081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.266505+0000","last_change":"2023-10-16T11:25:49.266505+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.266505+0000","last_undegraded":"2023-10-16T11:25:49.266505+0000","last_fullsized":"2023-10-16T11:25:49.266505+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.31","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.259204+0000","last_change":"2023-10-16T11:25:37.177928+0000","last_active":"2023-10-16T11:25:40.259204+0000","last_peered":"2023-10-16T11:25:40.259204+0000","last_clean":"2023-10-16T11:25:40.259204+0000","last_became_active":"2023-10-16T11:25:37.177612+0000","last_became_peered":"2023-10-16T11:25:37.177612+0000","last_unstale":"2023-10-16T11:25:40.259204+0000","last_undegraded":"2023-10-16T11:25:40.259204+0000","last_fullsized":"2023-10-16T11:25:40.259204+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:44.826687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.274689+0000","last_change":"2023-10-16T11:25:49.274689+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.274689+0000","last_undegraded":"2023-10-16T11:25:49.274689+0000","last_fullsized":"2023-10-16T11:25:49.274689+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.30","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249967+0000","last_change":"2023-10-16T11:25:37.170292+0000","last_active":"2023-10-16T11:25:40.249967+0000","last_peered":"2023-10-16T11:25:40.249967+0000","last_clean":"2023-10-16T11:25:40.249967+0000","last_became_active":"2023-10-16T11:25:37.169997+0000","last_became_peered":"2023-10-16T11:25:37.169997+0000","last_unstale":"2023-10-16T11:25:40.249967+0000","last_undegraded":"2023-10-16T11:25:40.249967+0000","last_fullsized":"2023-10-16T11:25:40.249967+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:09.329721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.267011+0000","last_change":"2023-10-16T11:25:49.267011+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.267011+0000","last_undegraded":"2023-10-16T11:25:49.267011+0000","last_fullsized":"2023-10-16T11:25:49.267011+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2f","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255135+0000","last_change":"2023-10-16T11:25:37.162923+0000","last_active":"2023-10-16T11:25:40.255135+0000","last_peered":"2023-10-16T11:25:40.255135+0000","last_clean":"2023-10-16T11:25:40.255135+0000","last_became_active":"2023-10-16T11:25:37.162649+0000","last_became_peered":"2023-10-16T11:25:37.162649+0000","last_unstale":"2023-10-16T11:25:40.255135+0000","last_undegraded":"2023-10-16T11:25:40.255135+0000","last_fullsized":"2023-10-16T11:25:40.255135+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:46.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.279612+0000","last_change":"2023-10-16T11:25:49.279612+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.279612+0000","last_undegraded":"2023-10-16T11:25:49.279612+0000","last_fullsized":"2023-10-16T11:25:49.279612+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.2e","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248737+0000","last_change":"2023-10-16T11:25:37.174044+0000","last_active":"2023-10-16T11:25:40.248737+0000","last_peered":"2023-10-16T11:25:40.248737+0000","last_clean":"2023-10-16T11:25:40.248737+0000","last_became_active":"2023-10-16T11:25:37.166694+0000","last_became_peered":"2023-10-16T11:25:37.166694+0000","last_unstale":"2023-10-16T11:25:40.248737+0000","last_undegraded":"2023-10-16T11:25:40.248737+0000","last_fullsized":"2023-10-16T11:25:40.248737+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:23.146288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.271754+0000","last_change":"2023-10-16T11:25:49.271754+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.271754+0000","last_undegraded":"2023-10-16T11:25:49.271754+0000","last_fullsized":"2023-10-16T11:25:49.271754+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2d","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254960+0000","last_change":"2023-10-16T11:25:37.183893+0000","last_active":"2023-10-16T11:25:40.254960+0000","last_peered":"2023-10-16T11:25:40.254960+0000","last_clean":"2023-10-16T11:25:40.254960+0000","last_became_active":"2023-10-16T11:25:37.183586+0000","last_became_peered":"2023-10-16T11:25:37.183586+0000","last_unstale":"2023-10-16T11:25:40.254960+0000","last_undegraded":"2023-10-16T11:25:40.254960+0000","last_fullsized":"2023-10-16T11:25:40.254960+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:43.813878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.274305+0000","last_change":"2023-10-16T11:25:49.274305+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.274305+0000","last_undegraded":"2023-10-16T11:25:49.274305+0000","last_fullsized":"2023-10-16T11:25:49.274305+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2c","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.250037+0000","last_change":"2023-10-16T11:25:37.172984+0000","last_active":"2023-10-16T11:25:40.250037+0000","last_peered":"2023-10-16T11:25:40.250037+0000","last_clean":"2023-10-16T11:25:40.250037+0000","last_became_active":"2023-10-16T11:25:37.172403+0000","last_became_peered":"2023-10-16T11:25:37.172403+0000","last_unstale":"2023-10-16T11:25:40.250037+0000","last_undegraded":"2023-10-16T11:25:40.250037+0000","last_fullsized":"2023-10-16T11:25:40.250037+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:03.912995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.272751+0000","last_change":"2023-10-16T11:25:49.272751+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.272751+0000","last_undegraded":"2023-10-16T11:25:49.272751+0000","last_fullsized":"2023-10-16T11:25:49.272751+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2b","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254876+0000","last_change":"2023-10-16T11:25:37.173026+0000","last_active":"2023-10-16T11:25:40.254876+0000","last_peered":"2023-10-16T11:25:40.254876+0000","last_clean":"2023-10-16T11:25:40.254876+0000","last_became_active":"2023-10-16T11:25:37.172680+0000","last_became_peered":"2023-10-16T11:25:37.172680+0000","last_unstale":"2023-10-16T11:25:40.254876+0000","last_undegraded":"2023-10-16T11:25:40.254876+0000","last_fullsized":"2023-10-16T11:25:40.254876+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:56.314696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.266977+0000","last_change":"2023-10-16T11:25:49.266977+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.266977+0000","last_undegraded":"2023-10-16T11:25:49.266977+0000","last_fullsized":"2023-10-16T11:25:49.266977+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2a","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252432+0000","last_change":"2023-10-16T11:25:37.176870+0000","last_active":"2023-10-16T11:25:40.252432+0000","last_peered":"2023-10-16T11:25:40.252432+0000","last_clean":"2023-10-16T11:25:40.252432+0000","last_became_active":"2023-10-16T11:25:37.175230+0000","last_became_peered":"2023-10-16T11:25:37.175230+0000","last_unstale":"2023-10-16T11:25:40.252432+0000","last_undegraded":"2023-10-16T11:25:40.252432+0000","last_fullsized":"2023-10-16T11:25:40.252432+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.267347+0000","last_change":"2023-10-16T11:25:49.267347+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.267347+0000","last_undegraded":"2023-10-16T11:25:49.267347+0000","last_fullsized":"2023-10-16T11:25:49.267347+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.29","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254808+0000","last_change":"2023-10-16T11:25:37.169637+0000","last_active":"2023-10-16T11:25:40.254808+0000","last_peered":"2023-10-16T11:25:40.254808+0000","last_clean":"2023-10-16T11:25:40.254808+0000","last_became_active":"2023-10-16T11:25:37.169375+0000","last_became_peered":"2023-10-16T11:25:37.169375+0000","last_unstale":"2023-10-16T11:25:40.254808+0000","last_undegraded":"2023-10-16T11:25:40.254808+0000","last_fullsized":"2023-10-16T11:25:40.254808+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:42:23.188876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.272728+0000","last_change":"2023-10-16T11:25:49.272728+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.272728+0000","last_undegraded":"2023-10-16T11:25:49.272728+0000","last_fullsized":"2023-10-16T11:25:49.272728+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.28","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248558+0000","last_change":"2023-10-16T11:25:37.187270+0000","last_active":"2023-10-16T11:25:40.248558+0000","last_peered":"2023-10-16T11:25:40.248558+0000","last_clean":"2023-10-16T11:25:40.248558+0000","last_became_active":"2023-10-16T11:25:37.186962+0000","last_became_peered":"2023-10-16T11:25:37.186962+0000","last_unstale":"2023-10-16T11:25:40.248558+0000","last_undegraded":"2023-10-16T11:25:40.248558+0000","last_fullsized":"2023-10-16T11:25:40.248558+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:56:45.915920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.267168+0000","last_change":"2023-10-16T11:25:49.267168+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.267168+0000","last_undegraded":"2023-10-16T11:25:49.267168+0000","last_fullsized":"2023-10-16T11:25:49.267168+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.27","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255281+0000","last_change":"2023-10-16T11:25:37.177057+0000","last_active":"2023-10-16T11:25:40.255281+0000","last_peered":"2023-10-16T11:25:40.255281+0000","last_clean":"2023-10-16T11:25:40.255281+0000","last_became_active":"2023-10-16T11:25:37.171369+0000","last_became_peered":"2023-10-16T11:25:37.171369+0000","last_unstale":"2023-10-16T11:25:40.255281+0000","last_undegraded":"2023-10-16T11:25:40.255281+0000","last_fullsized":"2023-10-16T11:25:40.255281+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.279039+0000","last_change":"2023-10-16T11:25:49.279039+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.279039+0000","last_undegraded":"2023-10-16T11:25:49.279039+0000","last_fullsized":"2023-10-16T11:25:49.279039+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.26","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252706+0000","last_change":"2023-10-16T11:25:37.182875+0000","last_active":"2023-10-16T11:25:40.252706+0000","last_peered":"2023-10-16T11:25:40.252706+0000","last_clean":"2023-10-16T11:25:40.252706+0000","last_became_active":"2023-10-16T11:25:37.182570+0000","last_became_peered":"2023-10-16T11:25:37.182570+0000","last_unstale":"2023-10-16T11:25:40.252706+0000","last_undegraded":"2023-10-16T11:25:40.252706+0000","last_fullsized":"2023-10-16T11:25:40.252706+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:14.504702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.272764+0000","last_change":"2023-10-16T11:25:49.272764+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.272764+0000","last_undegraded":"2023-10-16T11:25:49.272764+0000","last_fullsized":"2023-10-16T11:25:49.272764+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.25","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249433+0000","last_change":"2023-10-16T11:25:37.178245+0000","last_active":"2023-10-16T11:25:40.249433+0000","last_peered":"2023-10-16T11:25:40.249433+0000","last_clean":"2023-10-16T11:25:40.249433+0000","last_became_active":"2023-10-16T11:25:37.177914+0000","last_became_peered":"2023-10-16T11:25:37.177914+0000","last_unstale":"2023-10-16T11:25:40.249433+0000","last_undegraded":"2023-10-16T11:25:40.249433+0000","last_fullsized":"2023-10-16T11:25:40.249433+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:12.014513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.267216+0000","last_change":"2023-10-16T11:25:49.267216+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.267216+0000","last_undegraded":"2023-10-16T11:25:49.267216+0000","last_fullsized":"2023-10-16T11:25:49.267216+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.24","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248686+0000","last_change":"2023-10-16T11:25:37.170623+0000","last_active":"2023-10-16T11:25:40.248686+0000","last_peered":"2023-10-16T11:25:40.248686+0000","last_clean":"2023-10-16T11:25:40.248686+0000","last_became_active":"2023-10-16T11:25:37.170365+0000","last_became_peered":"2023-10-16T11:25:37.170365+0000","last_unstale":"2023-10-16T11:25:40.248686+0000","last_undegraded":"2023-10-16T11:25:40.248686+0000","last_fullsized":"2023-10-16T11:25:40.248686+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:52.443160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.266326+0000","last_change":"2023-10-16T11:25:49.266326+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.266326+0000","last_undegraded":"2023-10-16T11:25:49.266326+0000","last_fullsized":"2023-10-16T11:25:49.266326+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.23","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249332+0000","last_change":"2023-10-16T11:25:37.178693+0000","last_active":"2023-10-16T11:25:40.249332+0000","last_peered":"2023-10-16T11:25:40.249332+0000","last_clean":"2023-10-16T11:25:40.249332+0000","last_became_active":"2023-10-16T11:25:37.175403+0000","last_became_peered":"2023-10-16T11:25:37.175403+0000","last_unstale":"2023-10-16T11:25:40.249332+0000","last_undegraded":"2023-10-16T11:25:40.249332+0000","last_fullsized":"2023-10-16T11:25:40.249332+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:15:31.572558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.267625+0000","last_change":"2023-10-16T11:25:49.267625+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.267625+0000","last_undegraded":"2023-10-16T11:25:49.267625+0000","last_fullsized":"2023-10-16T11:25:49.267625+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.22","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.257055+0000","last_change":"2023-10-16T11:25:37.181812+0000","last_active":"2023-10-16T11:25:40.257055+0000","last_peered":"2023-10-16T11:25:40.257055+0000","last_clean":"2023-10-16T11:25:40.257055+0000","last_became_active":"2023-10-16T11:25:37.181529+0000","last_became_peered":"2023-10-16T11:25:37.181529+0000","last_unstale":"2023-10-16T11:25:40.257055+0000","last_undegraded":"2023-10-16T11:25:40.257055+0000","last_fullsized":"2023-10-16T11:25:40.257055+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:35.912014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.269415+0000","last_change":"2023-10-16T11:25:49.269415+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.269415+0000","last_undegraded":"2023-10-16T11:25:49.269415+0000","last_fullsized":"2023-10-16T11:25:49.269415+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.21","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255217+0000","last_change":"2023-10-16T11:25:37.166981+0000","last_active":"2023-10-16T11:25:40.255217+0000","last_peered":"2023-10-16T11:25:40.255217+0000","last_clean":"2023-10-16T11:25:40.255217+0000","last_became_active":"2023-10-16T11:25:37.164147+0000","last_became_peered":"2023-10-16T11:25:37.164147+0000","last_unstale":"2023-10-16T11:25:40.255217+0000","last_undegraded":"2023-10-16T11:25:40.255217+0000","last_fullsized":"2023-10-16T11:25:40.255217+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.273227+0000","last_change":"2023-10-16T11:25:49.273227+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.273227+0000","last_undegraded":"2023-10-16T11:25:49.273227+0000","last_fullsized":"2023-10-16T11:25:49.273227+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.20","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248589+0000","last_change":"2023-10-16T11:25:37.178631+0000","last_active":"2023-10-16T11:25:40.248589+0000","last_peered":"2023-10-16T11:25:40.248589+0000","last_clean":"2023-10-16T11:25:40.248589+0000","last_became_active":"2023-10-16T11:25:37.167402+0000","last_became_peered":"2023-10-16T11:25:37.167402+0000","last_unstale":"2023-10-16T11:25:40.248589+0000","last_undegraded":"2023-10-16T11:25:40.248589+0000","last_fullsized":"2023-10-16T11:25:40.248589+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:55.572264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.250139+0000","last_change":"2023-10-16T11:25:37.177706+0000","last_active":"2023-10-16T11:25:40.250139+0000","last_peered":"2023-10-16T11:25:40.250139+0000","last_clean":"2023-10-16T11:25:40.250139+0000","last_became_active":"2023-10-16T11:25:37.176806+0000","last_became_peered":"2023-10-16T11:25:37.176806+0000","last_unstale":"2023-10-16T11:25:40.250139+0000","last_undegraded":"2023-10-16T11:25:40.250139+0000","last_fullsized":"2023-10-16T11:25:40.250139+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:42:32.585642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1e","version":"236'1","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.177232+0000","last_change":"2023-10-16T11:25:36.169327+0000","last_active":"2023-10-16T11:25:38.177232+0000","last_peered":"2023-10-16T11:25:38.177232+0000","last_clean":"2023-10-16T11:25:38.177232+0000","last_became_active":"2023-10-16T11:25:36.169046+0000","last_became_peered":"2023-10-16T11:25:36.169046+0000","last_unstale":"2023-10-16T11:25:38.177232+0000","last_undegraded":"2023-10-16T11:25:38.177232+0000","last_fullsized":"2023-10-16T11:25:38.177232+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:34.899695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249353+0000","last_change":"2023-10-16T11:25:37.173736+0000","last_active":"2023-10-16T11:25:40.249353+0000","last_peered":"2023-10-16T11:25:40.249353+0000","last_clean":"2023-10-16T11:25:40.249353+0000","last_became_active":"2023-10-16T11:25:37.173482+0000","last_became_peered":"2023-10-16T11:25:37.173482+0000","last_unstale":"2023-10-16T11:25:40.249353+0000","last_undegraded":"2023-10-16T11:25:40.249353+0000","last_fullsized":"2023-10-16T11:25:40.249353+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:42:23.188876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1d","version":"236'2","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616465+0000","last_change":"2023-10-16T11:25:36.172861+0000","last_active":"2023-10-16T11:25:38.616465+0000","last_peered":"2023-10-16T11:25:38.616465+0000","last_clean":"2023-10-16T11:25:38.616465+0000","last_became_active":"2023-10-16T11:25:36.172574+0000","last_became_peered":"2023-10-16T11:25:36.172574+0000","last_unstale":"2023-10-16T11:25:38.616465+0000","last_undegraded":"2023-10-16T11:25:38.616465+0000","last_fullsized":"2023-10-16T11:25:38.616465+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:22:17.667795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255008+0000","last_change":"2023-10-16T11:25:37.167451+0000","last_active":"2023-10-16T11:25:40.255008+0000","last_peered":"2023-10-16T11:25:40.255008+0000","last_clean":"2023-10-16T11:25:40.255008+0000","last_became_active":"2023-10-16T11:25:37.167157+0000","last_became_peered":"2023-10-16T11:25:37.167157+0000","last_unstale":"2023-10-16T11:25:40.255008+0000","last_undegraded":"2023-10-16T11:25:40.255008+0000","last_fullsized":"2023-10-16T11:25:40.255008+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:14.504702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1c","version":"236'2","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616883+0000","last_change":"2023-10-16T11:25:36.153273+0000","last_active":"2023-10-16T11:25:38.616883+0000","last_peered":"2023-10-16T11:25:38.616883+0000","last_clean":"2023-10-16T11:25:38.616883+0000","last_became_active":"2023-10-16T11:25:36.152960+0000","last_became_peered":"2023-10-16T11:25:36.152960+0000","last_unstale":"2023-10-16T11:25:38.616883+0000","last_undegraded":"2023-10-16T11:25:38.616883+0000","last_fullsized":"2023-10-16T11:25:38.616883+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:08.218268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249306+0000","last_change":"2023-10-16T11:25:37.174071+0000","last_active":"2023-10-16T11:25:40.249306+0000","last_peered":"2023-10-16T11:25:40.249306+0000","last_clean":"2023-10-16T11:25:40.249306+0000","last_became_active":"2023-10-16T11:25:37.173436+0000","last_became_peered":"2023-10-16T11:25:37.173436+0000","last_unstale":"2023-10-16T11:25:40.249306+0000","last_undegraded":"2023-10-16T11:25:40.249306+0000","last_fullsized":"2023-10-16T11:25:40.249306+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1b","version":"236'1","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.171367+0000","last_change":"2023-10-16T11:25:36.156408+0000","last_active":"2023-10-16T11:25:38.171367+0000","last_peered":"2023-10-16T11:25:38.171367+0000","last_clean":"2023-10-16T11:25:38.171367+0000","last_became_active":"2023-10-16T11:25:36.155786+0000","last_became_peered":"2023-10-16T11:25:36.155786+0000","last_unstale":"2023-10-16T11:25:38.171367+0000","last_undegraded":"2023-10-16T11:25:38.171367+0000","last_fullsized":"2023-10-16T11:25:38.171367+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:01.925277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.259259+0000","last_change":"2023-10-16T11:25:37.179656+0000","last_active":"2023-10-16T11:25:40.259259+0000","last_peered":"2023-10-16T11:25:40.259259+0000","last_clean":"2023-10-16T11:25:40.259259+0000","last_became_active":"2023-10-16T11:25:37.179361+0000","last_became_peered":"2023-10-16T11:25:37.179361+0000","last_unstale":"2023-10-16T11:25:40.259259+0000","last_undegraded":"2023-10-16T11:25:40.259259+0000","last_fullsized":"2023-10-16T11:25:40.259259+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:09.230587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"236'1","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193985+0000","last_change":"2023-10-16T11:25:36.169348+0000","last_active":"2023-10-16T11:25:39.193985+0000","last_peered":"2023-10-16T11:25:39.193985+0000","last_clean":"2023-10-16T11:25:39.193985+0000","last_became_active":"2023-10-16T11:25:36.169053+0000","last_became_peered":"2023-10-16T11:25:36.169053+0000","last_unstale":"2023-10-16T11:25:39.193985+0000","last_undegraded":"2023-10-16T11:25:39.193985+0000","last_fullsized":"2023-10-16T11:25:39.193985+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:05.819502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249986+0000","last_change":"2023-10-16T11:25:37.176966+0000","last_active":"2023-10-16T11:25:40.249986+0000","last_peered":"2023-10-16T11:25:40.249986+0000","last_clean":"2023-10-16T11:25:40.249986+0000","last_became_active":"2023-10-16T11:25:37.174621+0000","last_became_peered":"2023-10-16T11:25:37.174621+0000","last_unstale":"2023-10-16T11:25:40.249986+0000","last_undegraded":"2023-10-16T11:25:40.249986+0000","last_fullsized":"2023-10-16T11:25:40.249986+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:56:09.516122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174368+0000","last_change":"2023-10-16T11:25:36.169635+0000","last_active":"2023-10-16T11:25:38.174368+0000","last_peered":"2023-10-16T11:25:38.174368+0000","last_clean":"2023-10-16T11:25:38.174368+0000","last_became_active":"2023-10-16T11:25:36.169016+0000","last_became_peered":"2023-10-16T11:25:36.169016+0000","last_unstale":"2023-10-16T11:25:38.174368+0000","last_undegraded":"2023-10-16T11:25:38.174368+0000","last_fullsized":"2023-10-16T11:25:38.174368+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:53:06.412234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248609+0000","last_change":"2023-10-16T11:25:37.184475+0000","last_active":"2023-10-16T11:25:40.248609+0000","last_peered":"2023-10-16T11:25:40.248609+0000","last_clean":"2023-10-16T11:25:40.248609+0000","last_became_active":"2023-10-16T11:25:37.179914+0000","last_became_peered":"2023-10-16T11:25:37.179914+0000","last_unstale":"2023-10-16T11:25:40.248609+0000","last_undegraded":"2023-10-16T11:25:40.248609+0000","last_fullsized":"2023-10-16T11:25:40.248609+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:53.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616094+0000","last_change":"2023-10-16T11:25:36.155589+0000","last_active":"2023-10-16T11:25:38.616094+0000","last_peered":"2023-10-16T11:25:38.616094+0000","last_clean":"2023-10-16T11:25:38.616094+0000","last_became_active":"2023-10-16T11:25:36.155087+0000","last_became_peered":"2023-10-16T11:25:36.155087+0000","last_unstale":"2023-10-16T11:25:38.616094+0000","last_undegraded":"2023-10-16T11:25:38.616094+0000","last_fullsized":"2023-10-16T11:25:38.616094+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:52:38.937212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248756+0000","last_change":"2023-10-16T11:25:37.175447+0000","last_active":"2023-10-16T11:25:40.248756+0000","last_peered":"2023-10-16T11:25:40.248756+0000","last_clean":"2023-10-16T11:25:40.248756+0000","last_became_active":"2023-10-16T11:25:37.173563+0000","last_became_peered":"2023-10-16T11:25:37.173563+0000","last_unstale":"2023-10-16T11:25:40.248756+0000","last_undegraded":"2023-10-16T11:25:40.248756+0000","last_fullsized":"2023-10-16T11:25:40.248756+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:54.544135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.198655+0000","last_change":"2023-10-16T11:25:36.169351+0000","last_active":"2023-10-16T11:25:39.198655+0000","last_peered":"2023-10-16T11:25:39.198655+0000","last_clean":"2023-10-16T11:25:39.198655+0000","last_became_active":"2023-10-16T11:25:36.168347+0000","last_became_peered":"2023-10-16T11:25:36.168347+0000","last_unstale":"2023-10-16T11:25:39.198655+0000","last_undegraded":"2023-10-16T11:25:39.198655+0000","last_fullsized":"2023-10-16T11:25:39.198655+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:19.077096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249471+0000","last_change":"2023-10-16T11:25:37.179048+0000","last_active":"2023-10-16T11:25:40.249471+0000","last_peered":"2023-10-16T11:25:40.249471+0000","last_clean":"2023-10-16T11:25:40.249471+0000","last_became_active":"2023-10-16T11:25:37.178751+0000","last_became_peered":"2023-10-16T11:25:37.178751+0000","last_unstale":"2023-10-16T11:25:40.249471+0000","last_undegraded":"2023-10-16T11:25:40.249471+0000","last_fullsized":"2023-10-16T11:25:40.249471+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:48:10.906108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616368+0000","last_change":"2023-10-16T11:25:36.154256+0000","last_active":"2023-10-16T11:25:38.616368+0000","last_peered":"2023-10-16T11:25:38.616368+0000","last_clean":"2023-10-16T11:25:38.616368+0000","last_became_active":"2023-10-16T11:25:36.153421+0000","last_became_peered":"2023-10-16T11:25:36.153421+0000","last_unstale":"2023-10-16T11:25:38.616368+0000","last_undegraded":"2023-10-16T11:25:38.616368+0000","last_fullsized":"2023-10-16T11:25:38.616368+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:09.436508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.250064+0000","last_change":"2023-10-16T11:25:37.188270+0000","last_active":"2023-10-16T11:25:40.250064+0000","last_peered":"2023-10-16T11:25:40.250064+0000","last_clean":"2023-10-16T11:25:40.250064+0000","last_became_active":"2023-10-16T11:25:37.187995+0000","last_became_peered":"2023-10-16T11:25:37.187995+0000","last_unstale":"2023-10-16T11:25:40.250064+0000","last_undegraded":"2023-10-16T11:25:40.250064+0000","last_fullsized":"2023-10-16T11:25:40.250064+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:42:22.542072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.15","version":"236'3","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193813+0000","last_change":"2023-10-16T11:25:36.171645+0000","last_active":"2023-10-16T11:25:39.193813+0000","last_peered":"2023-10-16T11:25:39.193813+0000","last_clean":"2023-10-16T11:25:39.193813+0000","last_became_active":"2023-10-16T11:25:36.170729+0000","last_became_peered":"2023-10-16T11:25:36.170729+0000","last_unstale":"2023-10-16T11:25:39.193813+0000","last_undegraded":"2023-10-16T11:25:39.193813+0000","last_fullsized":"2023-10-16T11:25:39.193813+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:34.362958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252622+0000","last_change":"2023-10-16T11:25:37.163991+0000","last_active":"2023-10-16T11:25:40.252622+0000","last_peered":"2023-10-16T11:25:40.252622+0000","last_clean":"2023-10-16T11:25:40.252622+0000","last_became_active":"2023-10-16T11:25:37.163080+0000","last_became_peered":"2023-10-16T11:25:37.163080+0000","last_unstale":"2023-10-16T11:25:40.252622+0000","last_undegraded":"2023-10-16T11:25:40.252622+0000","last_fullsized":"2023-10-16T11:25:40.252622+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.14","version":"237'8","reported_seq":21,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:46.774610+0000","last_change":"2023-10-16T11:25:36.173259+0000","last_active":"2023-10-16T11:25:46.774610+0000","last_peered":"2023-10-16T11:25:46.774610+0000","last_clean":"2023-10-16T11:25:46.774610+0000","last_became_active":"2023-10-16T11:25:36.172980+0000","last_became_peered":"2023-10-16T11:25:36.172980+0000","last_unstale":"2023-10-16T11:25:46.774610+0000","last_undegraded":"2023-10-16T11:25:46.774610+0000","last_fullsized":"2023-10-16T11:25:46.774610+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:33.782641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252475+0000","last_change":"2023-10-16T11:25:37.166081+0000","last_active":"2023-10-16T11:25:40.252475+0000","last_peered":"2023-10-16T11:25:40.252475+0000","last_clean":"2023-10-16T11:25:40.252475+0000","last_became_active":"2023-10-16T11:25:37.165150+0000","last_became_peered":"2023-10-16T11:25:37.165150+0000","last_unstale":"2023-10-16T11:25:40.252475+0000","last_undegraded":"2023-10-16T11:25:40.252475+0000","last_fullsized":"2023-10-16T11:25:40.252475+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:56.314696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"236'2","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193905+0000","last_change":"2023-10-16T11:25:36.171468+0000","last_active":"2023-10-16T11:25:39.193905+0000","last_peered":"2023-10-16T11:25:39.193905+0000","last_clean":"2023-10-16T11:25:39.193905+0000","last_became_active":"2023-10-16T11:25:36.171236+0000","last_became_peered":"2023-10-16T11:25:36.171236+0000","last_unstale":"2023-10-16T11:25:39.193905+0000","last_undegraded":"2023-10-16T11:25:39.193905+0000","last_fullsized":"2023-10-16T11:25:39.193905+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:17.018903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.259283+0000","last_change":"2023-10-16T11:25:37.178517+0000","last_active":"2023-10-16T11:25:40.259283+0000","last_peered":"2023-10-16T11:25:40.259283+0000","last_clean":"2023-10-16T11:25:40.259283+0000","last_became_active":"2023-10-16T11:25:37.177794+0000","last_became_peered":"2023-10-16T11:25:37.177794+0000","last_unstale":"2023-10-16T11:25:40.259283+0000","last_undegraded":"2023-10-16T11:25:40.259283+0000","last_fullsized":"2023-10-16T11:25:40.259283+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:52:39.949531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616389+0000","last_change":"2023-10-16T11:25:36.162595+0000","last_active":"2023-10-16T11:25:38.616389+0000","last_peered":"2023-10-16T11:25:38.616389+0000","last_clean":"2023-10-16T11:25:38.616389+0000","last_became_active":"2023-10-16T11:25:36.162249+0000","last_became_peered":"2023-10-16T11:25:36.162249+0000","last_unstale":"2023-10-16T11:25:38.616389+0000","last_undegraded":"2023-10-16T11:25:38.616389+0000","last_fullsized":"2023-10-16T11:25:38.616389+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:49.547165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255198+0000","last_change":"2023-10-16T11:25:37.164604+0000","last_active":"2023-10-16T11:25:40.255198+0000","last_peered":"2023-10-16T11:25:40.255198+0000","last_clean":"2023-10-16T11:25:40.255198+0000","last_became_active":"2023-10-16T11:25:37.162365+0000","last_became_peered":"2023-10-16T11:25:37.162365+0000","last_unstale":"2023-10-16T11:25:40.255198+0000","last_undegraded":"2023-10-16T11:25:40.255198+0000","last_fullsized":"2023-10-16T11:25:40.255198+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:35.912014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174625+0000","last_change":"2023-10-16T11:25:36.170014+0000","last_active":"2023-10-16T11:25:38.174625+0000","last_peered":"2023-10-16T11:25:38.174625+0000","last_clean":"2023-10-16T11:25:38.174625+0000","last_became_active":"2023-10-16T11:25:36.169755+0000","last_became_peered":"2023-10-16T11:25:36.169755+0000","last_unstale":"2023-10-16T11:25:38.174625+0000","last_undegraded":"2023-10-16T11:25:38.174625+0000","last_fullsized":"2023-10-16T11:25:38.174625+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:02.119904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252284+0000","last_change":"2023-10-16T11:25:37.178789+0000","last_active":"2023-10-16T11:25:40.252284+0000","last_peered":"2023-10-16T11:25:40.252284+0000","last_clean":"2023-10-16T11:25:40.252284+0000","last_became_active":"2023-10-16T11:25:37.178510+0000","last_became_peered":"2023-10-16T11:25:37.178510+0000","last_unstale":"2023-10-16T11:25:40.252284+0000","last_undegraded":"2023-10-16T11:25:40.252284+0000","last_fullsized":"2023-10-16T11:25:40.252284+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.10","version":"236'8","reported_seq":21,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182764+0000","last_change":"2023-10-16T11:25:36.168889+0000","last_active":"2023-10-16T11:25:38.182764+0000","last_peered":"2023-10-16T11:25:38.182764+0000","last_clean":"2023-10-16T11:25:38.182764+0000","last_became_active":"2023-10-16T11:25:36.168590+0000","last_became_peered":"2023-10-16T11:25:36.168590+0000","last_unstale":"2023-10-16T11:25:38.182764+0000","last_undegraded":"2023-10-16T11:25:38.182764+0000","last_fullsized":"2023-10-16T11:25:38.182764+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:42.801560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255269+0000","last_change":"2023-10-16T11:25:37.175992+0000","last_active":"2023-10-16T11:25:40.255269+0000","last_peered":"2023-10-16T11:25:40.255269+0000","last_clean":"2023-10-16T11:25:40.255269+0000","last_became_active":"2023-10-16T11:25:37.173732+0000","last_became_peered":"2023-10-16T11:25:37.173732+0000","last_unstale":"2023-10-16T11:25:40.255269+0000","last_undegraded":"2023-10-16T11:25:40.255269+0000","last_fullsized":"2023-10-16T11:25:40.255269+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.f","version":"236'1","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.177302+0000","last_change":"2023-10-16T11:25:36.173935+0000","last_active":"2023-10-16T11:25:38.177302+0000","last_peered":"2023-10-16T11:25:38.177302+0000","last_clean":"2023-10-16T11:25:38.177302+0000","last_became_active":"2023-10-16T11:25:36.173510+0000","last_became_peered":"2023-10-16T11:25:36.173510+0000","last_unstale":"2023-10-16T11:25:38.177302+0000","last_undegraded":"2023-10-16T11:25:38.177302+0000","last_fullsized":"2023-10-16T11:25:38.177302+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:19.077096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.250150+0000","last_change":"2023-10-16T11:25:37.177543+0000","last_active":"2023-10-16T11:25:40.250150+0000","last_peered":"2023-10-16T11:25:40.250150+0000","last_clean":"2023-10-16T11:25:40.250150+0000","last_became_active":"2023-10-16T11:25:37.176420+0000","last_became_peered":"2023-10-16T11:25:37.176420+0000","last_unstale":"2023-10-16T11:25:40.250150+0000","last_undegraded":"2023-10-16T11:25:40.250150+0000","last_fullsized":"2023-10-16T11:25:40.250150+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:24.104981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.e","version":"236'2","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616631+0000","last_change":"2023-10-16T11:25:36.170878+0000","last_active":"2023-10-16T11:25:38.616631+0000","last_peered":"2023-10-16T11:25:38.616631+0000","last_clean":"2023-10-16T11:25:38.616631+0000","last_became_active":"2023-10-16T11:25:36.170548+0000","last_became_peered":"2023-10-16T11:25:36.170548+0000","last_unstale":"2023-10-16T11:25:38.616631+0000","last_undegraded":"2023-10-16T11:25:38.616631+0000","last_fullsized":"2023-10-16T11:25:38.616631+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:34:14.932409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.259306+0000","last_change":"2023-10-16T11:25:37.176420+0000","last_active":"2023-10-16T11:25:40.259306+0000","last_peered":"2023-10-16T11:25:40.259306+0000","last_clean":"2023-10-16T11:25:40.259306+0000","last_became_active":"2023-10-16T11:25:37.176101+0000","last_became_peered":"2023-10-16T11:25:37.176101+0000","last_unstale":"2023-10-16T11:25:40.259306+0000","last_undegraded":"2023-10-16T11:25:40.259306+0000","last_fullsized":"2023-10-16T11:25:40.259306+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:10.245184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"236'1","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.175412+0000","last_change":"2023-10-16T11:25:36.171476+0000","last_active":"2023-10-16T11:25:38.175412+0000","last_peered":"2023-10-16T11:25:38.175412+0000","last_clean":"2023-10-16T11:25:38.175412+0000","last_became_active":"2023-10-16T11:25:36.171201+0000","last_became_peered":"2023-10-16T11:25:36.171201+0000","last_unstale":"2023-10-16T11:25:38.175412+0000","last_undegraded":"2023-10-16T11:25:38.175412+0000","last_fullsized":"2023-10-16T11:25:38.175412+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:30.034841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255057+0000","last_change":"2023-10-16T11:25:37.162366+0000","last_active":"2023-10-16T11:25:40.255057+0000","last_peered":"2023-10-16T11:25:40.255057+0000","last_clean":"2023-10-16T11:25:40.255057+0000","last_became_active":"2023-10-16T11:25:37.162032+0000","last_became_peered":"2023-10-16T11:25:37.162032+0000","last_unstale":"2023-10-16T11:25:40.255057+0000","last_undegraded":"2023-10-16T11:25:40.255057+0000","last_fullsized":"2023-10-16T11:25:40.255057+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182815+0000","last_change":"2023-10-16T11:25:36.174246+0000","last_active":"2023-10-16T11:25:38.182815+0000","last_peered":"2023-10-16T11:25:38.182815+0000","last_clean":"2023-10-16T11:25:38.182815+0000","last_became_active":"2023-10-16T11:25:36.173975+0000","last_became_peered":"2023-10-16T11:25:36.173975+0000","last_unstale":"2023-10-16T11:25:38.182815+0000","last_undegraded":"2023-10-16T11:25:38.182815+0000","last_fullsized":"2023-10-16T11:25:38.182815+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:32.130559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254853+0000","last_change":"2023-10-16T11:25:37.168715+0000","last_active":"2023-10-16T11:25:40.254853+0000","last_peered":"2023-10-16T11:25:40.254853+0000","last_clean":"2023-10-16T11:25:40.254853+0000","last_became_active":"2023-10-16T11:25:37.168387+0000","last_became_peered":"2023-10-16T11:25:37.168387+0000","last_unstale":"2023-10-16T11:25:40.254853+0000","last_undegraded":"2023-10-16T11:25:40.254853+0000","last_fullsized":"2023-10-16T11:25:40.254853+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.b","version":"236'2","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616716+0000","last_change":"2023-10-16T11:25:36.174153+0000","last_active":"2023-10-16T11:25:38.616716+0000","last_peered":"2023-10-16T11:25:38.616716+0000","last_clean":"2023-10-16T11:25:38.616716+0000","last_became_active":"2023-10-16T11:25:36.173812+0000","last_became_peered":"2023-10-16T11:25:36.173812+0000","last_unstale":"2023-10-16T11:25:38.616716+0000","last_undegraded":"2023-10-16T11:25:38.616716+0000","last_fullsized":"2023-10-16T11:25:38.616716+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:07:05.931833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248669+0000","last_change":"2023-10-16T11:25:37.175767+0000","last_active":"2023-10-16T11:25:40.248669+0000","last_peered":"2023-10-16T11:25:40.248669+0000","last_clean":"2023-10-16T11:25:40.248669+0000","last_became_active":"2023-10-16T11:25:37.167389+0000","last_became_peered":"2023-10-16T11:25:37.167389+0000","last_unstale":"2023-10-16T11:25:40.248669+0000","last_undegraded":"2023-10-16T11:25:40.248669+0000","last_fullsized":"2023-10-16T11:25:40.248669+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:18.601118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182571+0000","last_change":"2023-10-16T11:25:36.175478+0000","last_active":"2023-10-16T11:25:38.182571+0000","last_peered":"2023-10-16T11:25:38.182571+0000","last_clean":"2023-10-16T11:25:38.182571+0000","last_became_active":"2023-10-16T11:25:36.174935+0000","last_became_peered":"2023-10-16T11:25:36.174935+0000","last_unstale":"2023-10-16T11:25:38.182571+0000","last_undegraded":"2023-10-16T11:25:38.182571+0000","last_fullsized":"2023-10-16T11:25:38.182571+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:14.189489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249282+0000","last_change":"2023-10-16T11:25:37.173034+0000","last_active":"2023-10-16T11:25:40.249282+0000","last_peered":"2023-10-16T11:25:40.249282+0000","last_clean":"2023-10-16T11:25:40.249282+0000","last_became_active":"2023-10-16T11:25:37.172644+0000","last_became_peered":"2023-10-16T11:25:37.172644+0000","last_unstale":"2023-10-16T11:25:40.249282+0000","last_undegraded":"2023-10-16T11:25:40.249282+0000","last_fullsized":"2023-10-16T11:25:40.249282+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174975+0000","last_change":"2023-10-16T11:25:36.163926+0000","last_active":"2023-10-16T11:25:38.174975+0000","last_peered":"2023-10-16T11:25:38.174975+0000","last_clean":"2023-10-16T11:25:38.174975+0000","last_became_active":"2023-10-16T11:25:36.162991+0000","last_became_peered":"2023-10-16T11:25:36.162991+0000","last_unstale":"2023-10-16T11:25:38.174975+0000","last_undegraded":"2023-10-16T11:25:38.174975+0000","last_fullsized":"2023-10-16T11:25:38.174975+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:49.547165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254831+0000","last_change":"2023-10-16T11:25:37.158678+0000","last_active":"2023-10-16T11:25:40.254831+0000","last_peered":"2023-10-16T11:25:40.254831+0000","last_clean":"2023-10-16T11:25:40.254831+0000","last_became_active":"2023-10-16T11:25:37.157389+0000","last_became_peered":"2023-10-16T11:25:37.157389+0000","last_unstale":"2023-10-16T11:25:40.254831+0000","last_undegraded":"2023-10-16T11:25:40.254831+0000","last_fullsized":"2023-10-16T11:25:40.254831+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174650+0000","last_change":"2023-10-16T11:25:36.171876+0000","last_active":"2023-10-16T11:25:38.174650+0000","last_peered":"2023-10-16T11:25:38.174650+0000","last_clean":"2023-10-16T11:25:38.174650+0000","last_became_active":"2023-10-16T11:25:36.169128+0000","last_became_peered":"2023-10-16T11:25:36.169128+0000","last_unstale":"2023-10-16T11:25:38.174650+0000","last_undegraded":"2023-10-16T11:25:38.174650+0000","last_fullsized":"2023-10-16T11:25:38.174650+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:16:18.153133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252751+0000","last_change":"2023-10-16T11:25:37.179119+0000","last_active":"2023-10-16T11:25:40.252751+0000","last_peered":"2023-10-16T11:25:40.252751+0000","last_clean":"2023-10-16T11:25:40.252751+0000","last_became_active":"2023-10-16T11:25:37.175799+0000","last_became_peered":"2023-10-16T11:25:37.175799+0000","last_unstale":"2023-10-16T11:25:40.252751+0000","last_undegraded":"2023-10-16T11:25:40.252751+0000","last_fullsized":"2023-10-16T11:25:40.252751+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"236'2","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.172151+0000","last_change":"2023-10-16T11:25:36.162608+0000","last_active":"2023-10-16T11:25:38.172151+0000","last_peered":"2023-10-16T11:25:38.172151+0000","last_clean":"2023-10-16T11:25:38.172151+0000","last_became_active":"2023-10-16T11:25:36.162336+0000","last_became_peered":"2023-10-16T11:25:36.162336+0000","last_unstale":"2023-10-16T11:25:38.172151+0000","last_undegraded":"2023-10-16T11:25:38.172151+0000","last_fullsized":"2023-10-16T11:25:38.172151+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:19.377028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248788+0000","last_change":"2023-10-16T11:25:37.177560+0000","last_active":"2023-10-16T11:25:40.248788+0000","last_peered":"2023-10-16T11:25:40.248788+0000","last_clean":"2023-10-16T11:25:40.248788+0000","last_became_active":"2023-10-16T11:25:37.176982+0000","last_became_peered":"2023-10-16T11:25:37.176982+0000","last_unstale":"2023-10-16T11:25:40.248788+0000","last_undegraded":"2023-10-16T11:25:40.248788+0000","last_fullsized":"2023-10-16T11:25:40.248788+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:54.497943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"237'1","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:41.776279+0000","last_change":"2023-10-16T11:25:36.174419+0000","last_active":"2023-10-16T11:25:41.776279+0000","last_peered":"2023-10-16T11:25:41.776279+0000","last_clean":"2023-10-16T11:25:41.776279+0000","last_became_active":"2023-10-16T11:25:36.174148+0000","last_became_peered":"2023-10-16T11:25:36.174148+0000","last_unstale":"2023-10-16T11:25:41.776279+0000","last_undegraded":"2023-10-16T11:25:41.776279+0000","last_fullsized":"2023-10-16T11:25:41.776279+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:38.739323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249374+0000","last_change":"2023-10-16T11:25:37.180683+0000","last_active":"2023-10-16T11:25:40.249374+0000","last_peered":"2023-10-16T11:25:40.249374+0000","last_clean":"2023-10-16T11:25:40.249374+0000","last_became_active":"2023-10-16T11:25:37.180393+0000","last_became_peered":"2023-10-16T11:25:37.180393+0000","last_unstale":"2023-10-16T11:25:40.249374+0000","last_undegraded":"2023-10-16T11:25:40.249374+0000","last_fullsized":"2023-10-16T11:25:40.249374+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:38.218704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.175635+0000","last_change":"2023-10-16T11:25:36.164519+0000","last_active":"2023-10-16T11:25:38.175635+0000","last_peered":"2023-10-16T11:25:38.175635+0000","last_clean":"2023-10-16T11:25:38.175635+0000","last_became_active":"2023-10-16T11:25:36.164214+0000","last_became_peered":"2023-10-16T11:25:36.164214+0000","last_unstale":"2023-10-16T11:25:38.175635+0000","last_undegraded":"2023-10-16T11:25:38.175635+0000","last_fullsized":"2023-10-16T11:25:38.175635+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:03.074535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249451+0000","last_change":"2023-10-16T11:25:37.181808+0000","last_active":"2023-10-16T11:25:40.249451+0000","last_peered":"2023-10-16T11:25:40.249451+0000","last_clean":"2023-10-16T11:25:40.249451+0000","last_became_active":"2023-10-16T11:25:37.181575+0000","last_became_peered":"2023-10-16T11:25:37.181575+0000","last_unstale":"2023-10-16T11:25:40.249451+0000","last_undegraded":"2023-10-16T11:25:40.249451+0000","last_fullsized":"2023-10-16T11:25:40.249451+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:17:01.131611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182900+0000","last_change":"2023-10-16T11:25:36.162153+0000","last_active":"2023-10-16T11:25:38.182900+0000","last_peered":"2023-10-16T11:25:38.182900+0000","last_clean":"2023-10-16T11:25:38.182900+0000","last_became_active":"2023-10-16T11:25:36.161842+0000","last_became_peered":"2023-10-16T11:25:36.161842+0000","last_unstale":"2023-10-16T11:25:38.182900+0000","last_undegraded":"2023-10-16T11:25:38.182900+0000","last_fullsized":"2023-10-16T11:25:38.182900+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:55.302377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252517+0000","last_change":"2023-10-16T11:25:37.165420+0000","last_active":"2023-10-16T11:25:40.252517+0000","last_peered":"2023-10-16T11:25:40.252517+0000","last_clean":"2023-10-16T11:25:40.252517+0000","last_became_active":"2023-10-16T11:25:37.164820+0000","last_became_peered":"2023-10-16T11:25:37.164820+0000","last_unstale":"2023-10-16T11:25:40.252517+0000","last_undegraded":"2023-10-16T11:25:40.252517+0000","last_fullsized":"2023-10-16T11:25:40.252517+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:43.813878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.176218+0000","last_change":"2023-10-16T11:25:36.173209+0000","last_active":"2023-10-16T11:25:38.176218+0000","last_peered":"2023-10-16T11:25:38.176218+0000","last_clean":"2023-10-16T11:25:38.176218+0000","last_became_active":"2023-10-16T11:25:36.171799+0000","last_became_peered":"2023-10-16T11:25:36.171799+0000","last_unstale":"2023-10-16T11:25:38.176218+0000","last_undegraded":"2023-10-16T11:25:38.176218+0000","last_fullsized":"2023-10-16T11:25:38.176218+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:19.377028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254897+0000","last_change":"2023-10-16T11:25:37.180243+0000","last_active":"2023-10-16T11:25:40.254897+0000","last_peered":"2023-10-16T11:25:40.254897+0000","last_clean":"2023-10-16T11:25:40.254897+0000","last_became_active":"2023-10-16T11:25:37.179869+0000","last_became_peered":"2023-10-16T11:25:37.179869+0000","last_unstale":"2023-10-16T11:25:40.254897+0000","last_undegraded":"2023-10-16T11:25:40.254897+0000","last_fullsized":"2023-10-16T11:25:40.254897+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:15:31.572558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.171937+0000","last_change":"2023-10-16T11:25:36.169146+0000","last_active":"2023-10-16T11:25:38.171937+0000","last_peered":"2023-10-16T11:25:38.171937+0000","last_clean":"2023-10-16T11:25:38.171937+0000","last_became_active":"2023-10-16T11:25:36.168877+0000","last_became_peered":"2023-10-16T11:25:36.168877+0000","last_unstale":"2023-10-16T11:25:38.171937+0000","last_undegraded":"2023-10-16T11:25:38.171937+0000","last_fullsized":"2023-10-16T11:25:38.171937+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:35.988148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254732+0000","last_change":"2023-10-16T11:25:37.173516+0000","last_active":"2023-10-16T11:25:40.254732+0000","last_peered":"2023-10-16T11:25:40.254732+0000","last_clean":"2023-10-16T11:25:40.254732+0000","last_became_active":"2023-10-16T11:25:37.172964+0000","last_became_peered":"2023-10-16T11:25:37.172964+0000","last_unstale":"2023-10-16T11:25:40.254732+0000","last_undegraded":"2023-10-16T11:25:40.254732+0000","last_fullsized":"2023-10-16T11:25:40.254732+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616128+0000","last_change":"2023-10-16T11:25:36.166442+0000","last_active":"2023-10-16T11:25:38.616128+0000","last_peered":"2023-10-16T11:25:38.616128+0000","last_clean":"2023-10-16T11:25:38.616128+0000","last_became_active":"2023-10-16T11:25:36.166131+0000","last_became_peered":"2023-10-16T11:25:36.166131+0000","last_unstale":"2023-10-16T11:25:38.616128+0000","last_undegraded":"2023-10-16T11:25:38.616128+0000","last_fullsized":"2023-10-16T11:25:38.616128+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:43.814368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248629+0000","last_change":"2023-10-16T11:25:37.183491+0000","last_active":"2023-10-16T11:25:40.248629+0000","last_peered":"2023-10-16T11:25:40.248629+0000","last_clean":"2023-10-16T11:25:40.248629+0000","last_became_active":"2023-10-16T11:25:37.179560+0000","last_became_peered":"2023-10-16T11:25:37.179560+0000","last_unstale":"2023-10-16T11:25:40.248629+0000","last_undegraded":"2023-10-16T11:25:40.248629+0000","last_fullsized":"2023-10-16T11:25:40.248629+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:31.820109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616369+0000","last_change":"2023-10-16T11:25:36.175845+0000","last_active":"2023-10-16T11:25:38.616369+0000","last_peered":"2023-10-16T11:25:38.616369+0000","last_clean":"2023-10-16T11:25:38.616369+0000","last_became_active":"2023-10-16T11:25:36.175390+0000","last_became_peered":"2023-10-16T11:25:36.175390+0000","last_unstale":"2023-10-16T11:25:38.616369+0000","last_undegraded":"2023-10-16T11:25:38.616369+0000","last_fullsized":"2023-10-16T11:25:38.616369+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:09.232865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.257136+0000","last_change":"2023-10-16T11:25:37.178486+0000","last_active":"2023-10-16T11:25:40.257136+0000","last_peered":"2023-10-16T11:25:40.257136+0000","last_clean":"2023-10-16T11:25:40.257136+0000","last_became_active":"2023-10-16T11:25:37.177077+0000","last_became_peered":"2023-10-16T11:25:37.177077+0000","last_unstale":"2023-10-16T11:25:40.257136+0000","last_undegraded":"2023-10-16T11:25:40.257136+0000","last_fullsized":"2023-10-16T11:25:40.257136+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"236'4","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.617092+0000","last_change":"2023-10-16T11:25:36.174205+0000","last_active":"2023-10-16T11:25:38.617092+0000","last_peered":"2023-10-16T11:25:38.617092+0000","last_clean":"2023-10-16T11:25:38.617092+0000","last_became_active":"2023-10-16T11:25:36.173850+0000","last_became_peered":"2023-10-16T11:25:36.173850+0000","last_unstale":"2023-10-16T11:25:38.617092+0000","last_undegraded":"2023-10-16T11:25:38.617092+0000","last_fullsized":"2023-10-16T11:25:38.617092+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:20.618602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"223'2789","reported_seq":3746,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.173478+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:25:38.173478+0000","last_peered":"2023-10-16T11:25:38.173478+0000","last_clean":"2023-10-16T11:25:38.173478+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:25:38.173478+0000","last_undegraded":"2023-10-16T11:25:38.173478+0000","last_fullsized":"2023-10-16T11:25:38.173478+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2789,"log_dups_size":0,"ondisk_log_size":2789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":31,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":30,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2789,"ondisk_log_size":2789,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576001,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52740,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319100,"statfs":{"total":96636764160,"available":96582758400,"internally_reserved":0,"allocated":3596288,"data_stored":2052564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.153,"15min":1.2090000000000001},"min":{"1min":0.71499999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4950000000000001,"5min":1.7549999999999999,"15min":2.3399999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.161,"15min":1.2150000000000001},"min":{"1min":0.623,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.448,"5min":1.6699999999999999,"15min":1.8320000000000001},"last":1.113}]},{"osd":1,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.256,"15min":1.2430000000000001},"min":{"1min":0.84599999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.5029999999999999,"5min":1.913,"15min":1.913},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.171,"5min":1.2210000000000001,"15min":1.264},"min":{"1min":0.67100000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.522,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.0580000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.3,"15min":1.296},"min":{"1min":0.94199999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6879999999999999,"5min":1.8680000000000001,"15min":2.4460000000000002},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.21,"5min":1.3129999999999999,"15min":1.3500000000000001},"min":{"1min":0.94499999999999995,"5min":0.88,"15min":0.88},"max":{"1min":1.5589999999999999,"5min":1.859,"15min":1.974},"last":0.86799999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3520000000000001,"15min":1.3779999999999999},"min":{"1min":0.93700000000000006,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.653,"5min":1.8080000000000001,"15min":1.9099999999999999},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.4370000000000001,"15min":1.454},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.728,"5min":2.0070000000000001,"15min":2.577},"last":1.3460000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4319999999999999,"15min":1.413},"min":{"1min":0.89800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.865,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.171},{"interface":"front","average":{"1min":1.369,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":1.149,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8660000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.417}]},{"osd":5,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.587,"15min":1.591},"min":{"1min":1.1799999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.95,"5min":2.101,"15min":3.3519999999999999},"last":1.488},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.53,"15min":1.508},"min":{"1min":1.2330000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.9870000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.595}]},{"osd":6,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.492,"15min":1.5089999999999999},"min":{"1min":1.1459999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.8140000000000001,"5min":2.101,"15min":2.7250000000000001},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.6040000000000001,"5min":1.6819999999999999,"15min":1.698},"min":{"1min":1.3080000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.0129999999999999,"5min":2.2109999999999999,"15min":3.1440000000000001},"last":1.5660000000000001}]}]},{"osd":6,"up_from":13,"seq":55834576001,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52164,"kb_used_data":2936,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319676,"statfs":{"total":96636764160,"available":96583348224,"internally_reserved":0,"allocated":3006464,"data_stored":1462730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.2130000000000001,"15min":1.21},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.343,"5min":6.298,"15min":6.298},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.204,"5min":1.331,"15min":1.292},"min":{"1min":0.72199999999999998,"5min":0.68200000000000005,"15min":0.66400000000000003},"max":{"1min":1.607,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":1.377}]},{"osd":1,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.224,"15min":1.212},"min":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.5660000000000001,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.421,"15min":1.3779999999999999},"min":{"1min":0.76200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.7310000000000001,"5min":12.468,"15min":12.468},"last":1.1990000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.3140000000000001,"15min":1.2769999999999999},"min":{"1min":0.77800000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.373,"5min":11.586,"15min":11.586},"last":1.248},{"interface":"front","average":{"1min":1.306,"5min":1.3740000000000001,"15min":1.3680000000000001},"min":{"1min":0.84399999999999997,"5min":0.82599999999999996,"15min":0.77500000000000002},"max":{"1min":1.7629999999999999,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.427}]},{"osd":3,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.425,"15min":1.3460000000000001},"min":{"1min":0.84799999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.859,"5min":12.801,"15min":12.801},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2929999999999999,"15min":1.2769999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.657,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":1.1379999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3899999999999999,"15min":1.3280000000000001},"min":{"1min":0.68700000000000006,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.9510000000000001,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.6220000000000001,"15min":1.5509999999999999},"min":{"1min":1.1180000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.069,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.4630000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.5189999999999999,"15min":1.397},"min":{"1min":0.92500000000000004,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.978,"5min":18.247,"15min":18.247},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.595,"15min":1.5620000000000001},"min":{"1min":0.93799999999999994,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.1240000000000001,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.5049999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.627,"15min":1.5469999999999999},"min":{"1min":1.2070000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.383,"5min":12.589,"15min":12.589},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.661,"5min":1.702,"15min":1.603},"min":{"1min":1.0049999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.2970000000000002,"5min":12.725,"15min":12.725},"last":1.3360000000000001}]}]},{"osd":5,"up_from":13,"seq":55834576001,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29948,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341892,"statfs":{"total":96636764160,"available":96606097408,"internally_reserved":0,"allocated":2998272,"data_stored":1462126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2150000000000001,"15min":1.2629999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.51,"5min":2.661,"15min":2.661},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.204,"5min":1.2769999999999999,"15min":1.339},"min":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7210000000000001,"5min":1.7949999999999999,"15min":1.865},"last":1.113}]},{"osd":1,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.145,"15min":1.1890000000000001},"min":{"1min":0.872,"5min":0.81299999999999994,"15min":0.79100000000000004},"max":{"1min":1.583,"5min":1.704,"15min":1.992},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.079,"5min":1.22,"15min":1.222},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6739999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.427}]},{"osd":2,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.26,"15min":1.236},"min":{"1min":0.73499999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.7450000000000001,"5min":1.952,"15min":1.952},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.236,"5min":1.284,"15min":1.3129999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5840000000000001,"5min":1.8109999999999999,"15min":2.1640000000000001},"last":1.1699999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.319,"15min":1.329},"min":{"1min":0.872,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5920000000000001,"5min":3.012,"15min":3.012},"last":1.21},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.401,"15min":1.4139999999999999},"min":{"1min":0.95399999999999996,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.6639999999999999,"5min":1.9119999999999999,"15min":2.2250000000000001},"last":1.034}]},{"osd":4,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3600000000000001,"15min":1.341},"min":{"1min":0.91300000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.887,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.575},{"interface":"front","average":{"1min":1.413,"5min":1.5529999999999999,"15min":1.5660000000000001},"min":{"1min":0.996,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.6499999999999999,"5min":2.903,"15min":2.903},"last":1.3879999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.4119999999999999,"15min":1.411},"min":{"1min":0.93799999999999994,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.837,"5min":1.8660000000000001,"15min":2.0979999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.486,"15min":1.4810000000000001},"min":{"1min":0.93300000000000005,"5min":0.90700000000000003,"15min":0.89200000000000002},"max":{"1min":2.0329999999999999,"5min":3.073,"15min":3.073},"last":1.673}]},{"osd":7,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.5649999999999999,"15min":1.5429999999999999},"min":{"1min":1.159,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.764,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.3},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.625,"15min":1.6399999999999999},"min":{"1min":1.349,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.9970000000000001,"5min":2.1099999999999999,"15min":2.2930000000000001},"last":1.341}]}]},{"osd":4,"up_from":13,"seq":55834576002,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70160,"kb_used_data":2948,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301680,"statfs":{"total":96636764160,"available":96564920320,"internally_reserved":0,"allocated":3018752,"data_stored":1465075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.167,"15min":1.1799999999999999},"min":{"1min":0.69099999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.2370000000000001,"15min":1.2430000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":1.083}]},{"osd":1,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3069999999999999,"15min":1.3009999999999999},"min":{"1min":0.88800000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.252,"15min":1.252},"min":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":3.0739999999999998,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.5960000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3,"15min":1.3109999999999999},"min":{"1min":0.80500000000000005,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":3.2170000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.4359999999999999,"15min":1.4119999999999999},"min":{"1min":0.90300000000000002,"5min":0.879,"15min":0.872},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.423}]},{"osd":3,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.339,"15min":1.341},"min":{"1min":0.84599999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.516},{"interface":"front","average":{"1min":1.496,"5min":1.4550000000000001,"15min":1.4430000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":3.4220000000000002,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.468}]},{"osd":5,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.5249999999999999,"15min":1.4570000000000001},"min":{"1min":1.1339999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.613,"5min":1.46,"15min":1.4319999999999999},"min":{"1min":1.1519999999999999,"5min":0.85199999999999998,"15min":0.78600000000000003},"max":{"1min":3.7989999999999999,"5min":3.7989999999999999,"15min":3.7989999999999999},"last":1.292}]},{"osd":6,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6679999999999999,"5min":1.5649999999999999,"15min":1.5189999999999999},"min":{"1min":0.98599999999999999,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.776,"5min":1.643,"15min":1.6240000000000001},"min":{"1min":1.0269999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":3.5019999999999998,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.8}]},{"osd":7,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.8089999999999999,"5min":1.613,"15min":1.5860000000000001},"min":{"1min":1.3,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":4.2050000000000001,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":1.7569999999999999},{"interface":"front","average":{"1min":1.8460000000000001,"5min":1.702,"15min":1.6599999999999999},"min":{"1min":1.2370000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":3.7000000000000002,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.6850000000000001}]}]},{"osd":3,"up_from":13,"seq":55834576001,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52160,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319680,"statfs":{"total":96636764160,"available":96583352320,"internally_reserved":0,"allocated":3002368,"data_stored":1462708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.85899999999999999,"15min":0.89700000000000002},"min":{"1min":0.40000000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1339999999999999,"5min":1.478,"15min":1.901},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.997,"15min":1.0569999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1850000000000001,"5min":1.6180000000000001,"15min":1.6679999999999999},"last":0.85399999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.95299999999999996,"15min":0.98399999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.3400000000000001,"5min":1.508,"15min":1.508},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0609999999999999,"15min":1.121},"min":{"1min":0.64900000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5429999999999999,"5min":1.601,"15min":1.7190000000000001},"last":0.91100000000000003}]},{"osd":2,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0580000000000001,"15min":1.1120000000000001},"min":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4199999999999999,"5min":1.694,"15min":1.694},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.161,"15min":1.22},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.7350000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.222}]},{"osd":4,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2989999999999999,"15min":1.3480000000000001},"min":{"1min":0.89400000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.724,"5min":1.806,"15min":1.873},"last":0.995},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3300000000000001,"15min":1.359},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.349}]},{"osd":5,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.389,"15min":1.405},"min":{"1min":1.0549999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8260000000000001,"5min":2.798,"15min":2.798},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.395,"15min":1.4410000000000001},"min":{"1min":0.88800000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.883,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.2889999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.488,"15min":1.5329999999999999},"min":{"1min":1.01,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.911,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4790000000000001,"15min":1.534},"min":{"1min":0.86699999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.0329999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.629}]},{"osd":7,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.518,"15min":1.5580000000000001},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.575,"5min":1.613,"15min":1.669},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.3799999999999999,"5min":2.3799999999999999,"15min":2.5129999999999999},"last":1.7230000000000001}]}]},{"osd":2,"up_from":13,"seq":55834576000,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56320,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315520,"statfs":{"total":96636764160,"available":96579092480,"internally_reserved":0,"allocated":3002368,"data_stored":1462216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.89200000000000002,"15min":0.91600000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.349,"5min":1.532,"15min":1.669},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.99299999999999999,"15min":1.069},"min":{"1min":0.56899999999999995,"5min":0.47699999999999998,"15min":0.32600000000000001},"max":{"1min":1.3460000000000001,"5min":1.6359999999999999,"15min":2.1429999999999998},"last":0.95799999999999996}]},{"osd":1,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0760000000000001,"15min":1.089},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.39200000000000002},"max":{"1min":1.591,"5min":1.645,"15min":1.978},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.085,"15min":1.1379999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.393,"5min":1.609,"15min":1.7949999999999999},"last":1.0680000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0429999999999999,"15min":1.0740000000000001},"min":{"1min":0.499,"5min":0.437,"15min":0.437},"max":{"1min":1.522,"5min":1.651,"15min":1.8560000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.137,"5min":1.2170000000000001,"15min":1.2629999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.6930000000000001,"5min":1.7450000000000001,"15min":1.857},"last":1.379}]},{"osd":4,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3340000000000001,"15min":1.3999999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.73499999999999999},"max":{"1min":1.643,"5min":1.8029999999999999,"15min":2.5059999999999998},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.3220000000000001,"15min":1.339},"min":{"1min":0.88200000000000001,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":1.6499999999999999,"5min":1.825,"15min":1.843},"last":1.2230000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.4139999999999999,"15min":1.4530000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.429,"15min":1.4730000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.885,"5min":1.931,"15min":2.6259999999999999},"last":1.5449999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3899999999999999,"15min":1.3999999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.8260000000000001,"5min":1.9019999999999999,"15min":2.1339999999999999},"last":1.095},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4870000000000001,"15min":1.526},"min":{"1min":0.80500000000000005,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.494}]},{"osd":7,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.5109999999999999,"15min":1.569},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.92500000000000004},"max":{"1min":2.0310000000000001,"5min":2.1469999999999998,"15min":2.8109999999999999},"last":1.615},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4750000000000001,"15min":1.502},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80100000000000005},"max":{"1min":1.952,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.3420000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576000,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64656,"kb_used_data":2948,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307184,"statfs":{"total":96636764160,"available":96570556416,"internally_reserved":0,"allocated":3018752,"data_stored":1465107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.93799999999999994,"15min":0.91700000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":5.117,"5min":5.117,"15min":5.117},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.97299999999999998,"15min":1.0449999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.5069999999999999,"5min":1.5509999999999999,"15min":1.667},"last":0.35899999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.97299999999999998,"15min":1.014},"min":{"1min":0.52600000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2350000000000001,"15min":1.27},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":3.5840000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.90300000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.1779999999999999,"15min":1.202},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.4670000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.1439999999999999,"15min":1.1639999999999999},"min":{"1min":0.48099999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5600000000000001,"5min":1.633,"15min":1.802},"last":0.66400000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.3260000000000001,"15min":1.3009999999999999},"min":{"1min":0.80500000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":5.29,"5min":5.29,"15min":5.29},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.4450000000000001,"15min":1.4259999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":5.0179999999999998,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":0.999}]},{"osd":5,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.397,"15min":1.419},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":6.0979999999999999,"5min":6.0979999999999999,"15min":6.0979999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.415,"15min":1.4159999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.5299999999999998,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.2330000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.49,"15min":1.5029999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":4.0499999999999998,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":1.034},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.474,"15min":1.488},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":3.8799999999999999,"5min":3.8799999999999999,"15min":3.8799999999999999},"last":1.2689999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.577,"15min":1.569},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":4.46,"5min":4.46,"15min":4.46},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.625,"15min":1.613},"min":{"1min":1.0049999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":4.1040000000000001,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":1.135}]}]},{"osd":0,"up_from":13,"seq":55834576001,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52740,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319100,"statfs":{"total":96636764160,"available":96582758400,"internally_reserved":0,"allocated":3596288,"data_stored":2052528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.86099999999999999,"15min":0.88700000000000001},"min":{"1min":0.46800000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2210000000000001,"5min":1.52,"15min":1.619},"last":0.874},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.006,"15min":1.081},"min":{"1min":0.41199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.341,"5min":1.621,"15min":1.76},"last":1.0609999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.075,"15min":1.0800000000000001},"min":{"1min":0.52500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.7349999999999999,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.034,"5min":1.0700000000000001,"15min":1.1299999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":1.583}]},{"osd":3,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.016,"15min":1.0669999999999999},"min":{"1min":0.50600000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1299999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.109},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.1539999999999999,"15min":1.226},"min":{"1min":0.55600000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.3799999999999999,"5min":1.833,"15min":1.9830000000000001},"last":1.2390000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.349,"15min":1.3859999999999999},"min":{"1min":0.78800000000000003,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":3.2799999999999998,"5min":3.2799999999999998,"15min":3.2799999999999998},"last":1.7829999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.292,"15min":1.337},"min":{"1min":0.76100000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.2190000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.357,"15min":1.3720000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0209999999999999,"5min":2.0750000000000002,"15min":2.6669999999999998},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3819999999999999,"15min":1.431},"min":{"1min":0.89800000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.885,"5min":2.2029999999999998,"15min":2.2040000000000002},"last":1.4370000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.4710000000000001,"15min":1.496},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":5.2560000000000002,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":1.53},{"interface":"front","average":{"1min":1.298,"5min":1.429,"15min":1.4790000000000001},"min":{"1min":0.83199999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.804,"5min":2.073,"15min":2.262},"last":1.385}]},{"osd":7,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.704,"5min":1.5429999999999999,"15min":1.5600000000000001},"min":{"1min":0.88700000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.4419999999999999,"15min":1.492},"min":{"1min":0.871,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.623,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.2809999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:25:55.794 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:25:56.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.046+0000 7f1aedeb4640 1 Processor -- start 2023-10-16T11:25:56.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.046+0000 7f1aedeb4640 1 -- start start 2023-10-16T11:25:56.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.048+0000 7f1aedeb4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ae81615f0 0x7f1ae815db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:56.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.048+0000 7f1aedeb4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ae815e3e0 con 0x7f1ae80cd0c0 2023-10-16T11:25:56.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.048+0000 7f1aedeb4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ae80cd4a0 con 0x7f1ae80b0b20 2023-10-16T11:25:56.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.048+0000 7f1aedeb4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae815e100 con 0x7f1ae81615f0 2023-10-16T11:25:56.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.048+0000 7f1ae77fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1ae80cd0c0 0x7f1ae80b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40514/0 (socket says 172.21.15.84:40514) 2023-10-16T11:25:56.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.048+0000 7f1ae77fe640 1 -- 172.21.15.84:0/1767056706 learned_addr learned my addr 172.21.15.84:0/1767056706 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:56.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.049+0000 7f1ae7fff640 1 --2- 172.21.15.84:0/1767056706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ae81615f0 0x7f1ae815db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:56.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.049+0000 7f1ae6ffd640 1 -- 172.21.15.84:0/1767056706 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002971477 0 0) 0x7f1ae815e3e0 con 0x7f1ae80cd0c0 2023-10-16T11:25:56.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.049+0000 7f1ae6ffd640 1 -- 172.21.15.84:0/1767056706 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ad4003520 con 0x7f1ae80cd0c0 2023-10-16T11:25:56.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.049+0000 7f1ae6ffd640 1 -- 172.21.15.84:0/1767056706 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3642211005 0 0) 0x7f1ad4003520 con 0x7f1ae80cd0c0 2023-10-16T11:25:56.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.049+0000 7f1ae6ffd640 1 -- 172.21.15.84:0/1767056706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ae81615f0 msgr2=0x7f1ae815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:56.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.049+0000 7f1ae6ffd640 1 --2- 172.21.15.84:0/1767056706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ae81615f0 0x7f1ae815db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:56.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.049+0000 7f1ae6ffd640 1 -- 172.21.15.84:0/1767056706 >> v1:172.21.15.157:6789/0 conn(0x7f1ae80b0b20 legacy=0x7f1ae815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:56.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.049+0000 7f1ae6ffd640 1 -- 172.21.15.84:0/1767056706 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae8160b10 con 0x7f1ae80cd0c0 2023-10-16T11:25:56.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.049+0000 7f1ae6ffd640 1 -- 172.21.15.84:0/1767056706 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f1ad8001670 con 0x7f1ae80cd0c0 2023-10-16T11:25:56.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.049+0000 7f1ae6ffd640 1 -- 172.21.15.84:0/1767056706 >> v1:172.21.15.84:6789/0 conn(0x7f1ae80cd0c0 legacy=0x7f1ae80b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:56.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.049+0000 7f1ae6ffd640 1 --2- 172.21.15.84:0/1767056706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ae81615f0 0x7f1ad40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:56.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.049+0000 7f1ae6ffd640 1 --2- 172.21.15.84:0/1767056706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ad4003f70 0x7f1ad40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:56.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.049+0000 7f1ae6ffd640 1 --2- 172.21.15.84:0/1767056706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad4004d00 0x7f1ad4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:56.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.049+0000 7f1ae6ffd640 1 -- 172.21.15.84:0/1767056706 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae815e100 con 0x7f1ad4003f70 2023-10-16T11:25:56.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.049+0000 7f1ae6ffd640 1 -- 172.21.15.84:0/1767056706 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad400bb70 con 0x7f1ad4004d00 2023-10-16T11:25:56.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.049+0000 7f1ae6ffd640 1 -- 172.21.15.84:0/1767056706 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad400bd30 con 0x7f1ae81615f0 2023-10-16T11:25:56.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.050+0000 7f1adffff640 1 --2- 172.21.15.84:0/1767056706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ad4003f70 0x7f1ad40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:56.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.050+0000 7f1ae77fe640 1 --2- 172.21.15.84:0/1767056706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ae81615f0 0x7f1ad40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:56.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.050+0000 7f1ae7fff640 1 --2- 172.21.15.84:0/1767056706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad4004d00 0x7f1ad4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:56.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.050+0000 7f1adffff640 1 -- 172.21.15.84:0/1767056706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ae81615f0 msgr2=0x7f1ad40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:56.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.050+0000 7f1adffff640 1 --2- 172.21.15.84:0/1767056706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ae81615f0 0x7f1ad40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:56.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.050+0000 7f1adffff640 1 -- 172.21.15.84:0/1767056706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad4004d00 msgr2=0x7f1ad4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:56.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.050+0000 7f1adffff640 1 --2- 172.21.15.84:0/1767056706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad4004d00 0x7f1ad4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:56.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.050+0000 7f1adffff640 1 -- 172.21.15.84:0/1767056706 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad4003520 con 0x7f1ad4003f70 2023-10-16T11:25:56.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.050+0000 7f1ae7fff640 1 --2- 172.21.15.84:0/1767056706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad4004d00 0x7f1ad4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:25:56.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.052+0000 7f1adffff640 1 --2- 172.21.15.84:0/1767056706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ad4003f70 0x7f1ad40047c0 secure :-1 s=READY pgs=2994 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0001750 tx=0x7f1ad40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abcd391fad98d57 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:56.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.052+0000 7f1ae6ffd640 1 -- 172.21.15.84:0/1767056706 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad00052d0 con 0x7f1ad4003f70 2023-10-16T11:25:56.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.052+0000 7f1ae6ffd640 1 -- 172.21.15.84:0/1767056706 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad002e050 con 0x7f1ad4003f70 2023-10-16T11:25:56.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.052+0000 7f1ae6ffd640 1 -- 172.21.15.84:0/1767056706 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0003830 con 0x7f1ad4003f70 2023-10-16T11:25:56.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.053+0000 7f1aedeb4640 1 -- 172.21.15.84:0/1767056706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ad4003f70 msgr2=0x7f1ad40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:56.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.053+0000 7f1aedeb4640 1 --2- 172.21.15.84:0/1767056706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ad4003f70 0x7f1ad40047c0 secure :-1 s=READY pgs=2994 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0001750 tx=0x7f1ad40057f0 comp rx=0 tx=0).stop 2023-10-16T11:25:56.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.053+0000 7f1ae7fff640 1 -- 172.21.15.84:0/1767056706 reap_dead start 2023-10-16T11:25:56.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.053+0000 7f1aedeb4640 1 -- 172.21.15.84:0/1767056706 shutdown_connections 2023-10-16T11:25:56.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.053+0000 7f1aedeb4640 1 -- 172.21.15.84:0/1767056706 >> 172.21.15.84:0/1767056706 conn(0x7f1ae80cb410 msgr2=0x7f1ae80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:56.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.053+0000 7f1aedeb4640 1 -- 172.21.15.84:0/1767056706 shutdown_connections 2023-10-16T11:25:56.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.053+0000 7f1aedeb4640 1 -- 172.21.15.84:0/1767056706 wait complete. 2023-10-16T11:25:56.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.055+0000 7f1aedeb4640 1 Processor -- start 2023-10-16T11:25:56.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.055+0000 7f1aedeb4640 1 -- start start 2023-10-16T11:25:56.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.056+0000 7f1aedeb4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ad4003f70 0x7f1ae81f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:56.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.056+0000 7f1aedeb4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ae8148dd0 0x7f1ae80afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:56.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.056+0000 7f1aedeb4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ae80b0050 0x7f1ae815ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:56.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.056+0000 7f1aedeb4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae81f5f90 con 0x7f1ad4003f70 2023-10-16T11:25:56.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.056+0000 7f1aedeb4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae80b0560 con 0x7f1ae80b0050 2023-10-16T11:25:56.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.056+0000 7f1aedeb4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae81f6200 con 0x7f1ae8148dd0 2023-10-16T11:25:56.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.056+0000 7f1ae77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ad4003f70 0x7f1ae81f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:56.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.056+0000 7f1ae77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ad4003f70 0x7f1ae81f6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40488/0 (socket says 172.21.15.84:40488) 2023-10-16T11:25:56.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.056+0000 7f1ae77fe640 1 -- 172.21.15.84:0/301489901 learned_addr learned my addr 172.21.15.84:0/301489901 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:56.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.057+0000 7f1ae77fe640 1 -- 172.21.15.84:0/301489901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ae8148dd0 msgr2=0x7f1ae80afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:56.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.057+0000 7f1ae77fe640 1 --2- 172.21.15.84:0/301489901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ae8148dd0 0x7f1ae80afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:56.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.057+0000 7f1ae77fe640 1 -- 172.21.15.84:0/301489901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ae80b0050 msgr2=0x7f1ae815ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:56.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.057+0000 7f1ae77fe640 1 --2- 172.21.15.84:0/301489901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ae80b0050 0x7f1ae815ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:56.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.057+0000 7f1ae77fe640 1 -- 172.21.15.84:0/301489901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae81f64e0 con 0x7f1ad4003f70 2023-10-16T11:25:56.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.057+0000 7f1ae77fe640 1 --2- 172.21.15.84:0/301489901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ad4003f70 0x7f1ae81f6a00 secure :-1 s=READY pgs=2995 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8001050 tx=0x7f1ad800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:56.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.057+0000 7f1ae4ff9640 1 -- 172.21.15.84:0/301489901 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad800eed0 con 0x7f1ad4003f70 2023-10-16T11:25:56.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.057+0000 7f1ae4ff9640 1 -- 172.21.15.84:0/301489901 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad800c7f0 con 0x7f1ad4003f70 2023-10-16T11:25:56.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.057+0000 7f1aedeb4640 1 -- 172.21.15.84:0/301489901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae8149250 con 0x7f1ad4003f70 2023-10-16T11:25:56.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.057+0000 7f1ae4ff9640 1 -- 172.21.15.84:0/301489901 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad800cba0 con 0x7f1ad4003f70 2023-10-16T11:25:56.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.057+0000 7f1aedeb4640 1 -- 172.21.15.84:0/301489901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae814bf80 con 0x7f1ad4003f70 2023-10-16T11:25:56.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.058+0000 7f1aedeb4640 1 -- 172.21.15.84:0/301489901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae81f66d0 con 0x7f1ad4003f70 2023-10-16T11:25:56.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.060+0000 7f1ae4ff9640 1 -- 172.21.15.84:0/301489901 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1ad800cd00 con 0x7f1ad4003f70 2023-10-16T11:25:56.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.060+0000 7f1ae4ff9640 1 --2- 172.21.15.84:0/301489901 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f1ac806b0e0 0x7f1ac806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:56.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.061+0000 7f1ae4ff9640 1 -- 172.21.15.84:0/301489901 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f1ad8089be0 con 0x7f1ad4003f70 2023-10-16T11:25:56.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.061+0000 7f1adffff640 1 --2- 172.21.15.84:0/301489901 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f1ac806b0e0 0x7f1ac806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:56.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.062+0000 7f1adffff640 1 --2- 172.21.15.84:0/301489901 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f1ac806b0e0 0x7f1ac806d5a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0033000 tx=0x7f1ad002f000 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:56.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.065+0000 7f1ae4ff9640 1 -- 172.21.15.84:0/301489901 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1ad8054680 con 0x7f1ad4003f70 2023-10-16T11:25:56.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.255+0000 7f1aedeb4640 1 -- 172.21.15.84:0/301489901 --> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ae81f6370 con 0x7f1ac806b0e0 2023-10-16T11:25:56.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.293+0000 7f1ae4ff9640 1 -- 172.21.15.84:0/301489901 <== mgr.11814 v2:172.21.15.84:6832/2963529201 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+514284 (secure 0 0 0) 0x7f1ae81f6370 con 0x7f1ac806b0e0 2023-10-16T11:25:56.297 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:56.305 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:25:56.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.302+0000 7f1aedeb4640 1 -- 172.21.15.84:0/301489901 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f1ac806b0e0 msgr2=0x7f1ac806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:56.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.302+0000 7f1aedeb4640 1 --2- 172.21.15.84:0/301489901 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f1ac806b0e0 0x7f1ac806d5a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0033000 tx=0x7f1ad002f000 comp rx=0 tx=0).stop 2023-10-16T11:25:56.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.302+0000 7f1aedeb4640 1 -- 172.21.15.84:0/301489901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ad4003f70 msgr2=0x7f1ae81f6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:56.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.302+0000 7f1aedeb4640 1 --2- 172.21.15.84:0/301489901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ad4003f70 0x7f1ae81f6a00 secure :-1 s=READY pgs=2995 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8001050 tx=0x7f1ad800ec80 comp rx=0 tx=0).stop 2023-10-16T11:25:56.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.302+0000 7f1aedeb4640 1 -- 172.21.15.84:0/301489901 shutdown_connections 2023-10-16T11:25:56.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.302+0000 7f1aedeb4640 1 --2- 172.21.15.84:0/301489901 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f1ac806b0e0 0x7f1ac806d5a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:56.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.302+0000 7f1aedeb4640 1 --2- 172.21.15.84:0/301489901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ae80b0050 0x7f1ae815ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:56.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.302+0000 7f1aedeb4640 1 --2- 172.21.15.84:0/301489901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ae8148dd0 0x7f1ae80afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:56.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.302+0000 7f1aedeb4640 1 --2- 172.21.15.84:0/301489901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ad4003f70 0x7f1ae81f6a00 unknown :-1 s=CLOSED pgs=2995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:56.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.302+0000 7f1aedeb4640 1 -- 172.21.15.84:0/301489901 >> 172.21.15.84:0/301489901 conn(0x7f1ae80cb410 msgr2=0x7f1ae815e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:56.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.302+0000 7f1aedeb4640 1 -- 172.21.15.84:0/301489901 shutdown_connections 2023-10-16T11:25:56.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.302+0000 7f1aedeb4640 1 -- 172.21.15.84:0/301489901 wait complete. 2023-10-16T11:25:56.323 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2023-10-16T11:25:55.248077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594023,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":668,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2830,"ondisk_log_size":2830,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":204,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":430888,"kb_used_data":24648,"kb_used_omap":12,"kb_used_meta":406131,"kb_avail":754543832,"statfs":{"total":773094113280,"available":772652883968,"internally_reserved":0,"allocated":25239552,"data_stored":12885054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":415878713},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":987,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.984522"},"pg_stats":[{"pgid":"33.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.273967+0000","last_change":"2023-10-16T11:25:49.273967+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.273967+0000","last_undegraded":"2023-10-16T11:25:49.273967+0000","last_fullsized":"2023-10-16T11:25:49.273967+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3f","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248777+0000","last_change":"2023-10-16T11:25:37.173492+0000","last_active":"2023-10-16T11:25:40.248777+0000","last_peered":"2023-10-16T11:25:40.248777+0000","last_clean":"2023-10-16T11:25:40.248777+0000","last_became_active":"2023-10-16T11:25:37.167017+0000","last_became_peered":"2023-10-16T11:25:37.167017+0000","last_unstale":"2023-10-16T11:25:40.248777+0000","last_undegraded":"2023-10-16T11:25:40.248777+0000","last_fullsized":"2023-10-16T11:25:40.248777+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:07.791499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.270260+0000","last_change":"2023-10-16T11:25:49.270260+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.270260+0000","last_undegraded":"2023-10-16T11:25:49.270260+0000","last_fullsized":"2023-10-16T11:25:49.270260+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.3e","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248717+0000","last_change":"2023-10-16T11:25:37.179117+0000","last_active":"2023-10-16T11:25:40.248717+0000","last_peered":"2023-10-16T11:25:40.248717+0000","last_clean":"2023-10-16T11:25:40.248717+0000","last_became_active":"2023-10-16T11:25:37.178849+0000","last_became_peered":"2023-10-16T11:25:37.178849+0000","last_unstale":"2023-10-16T11:25:40.248717+0000","last_undegraded":"2023-10-16T11:25:40.248717+0000","last_fullsized":"2023-10-16T11:25:40.248717+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:18:57.278249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.273067+0000","last_change":"2023-10-16T11:25:49.273067+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.273067+0000","last_undegraded":"2023-10-16T11:25:49.273067+0000","last_fullsized":"2023-10-16T11:25:49.273067+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3d","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.250094+0000","last_change":"2023-10-16T11:25:37.179065+0000","last_active":"2023-10-16T11:25:40.250094+0000","last_peered":"2023-10-16T11:25:40.250094+0000","last_clean":"2023-10-16T11:25:40.250094+0000","last_became_active":"2023-10-16T11:25:37.178779+0000","last_became_peered":"2023-10-16T11:25:37.178779+0000","last_unstale":"2023-10-16T11:25:40.250094+0000","last_undegraded":"2023-10-16T11:25:40.250094+0000","last_fullsized":"2023-10-16T11:25:40.250094+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:09:47.367715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.271638+0000","last_change":"2023-10-16T11:25:49.271638+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.271638+0000","last_undegraded":"2023-10-16T11:25:49.271638+0000","last_fullsized":"2023-10-16T11:25:49.271638+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3c","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255257+0000","last_change":"2023-10-16T11:25:37.172480+0000","last_active":"2023-10-16T11:25:40.255257+0000","last_peered":"2023-10-16T11:25:40.255257+0000","last_clean":"2023-10-16T11:25:40.255257+0000","last_became_active":"2023-10-16T11:25:37.172192+0000","last_became_peered":"2023-10-16T11:25:37.172192+0000","last_unstale":"2023-10-16T11:25:40.255257+0000","last_undegraded":"2023-10-16T11:25:40.255257+0000","last_fullsized":"2023-10-16T11:25:40.255257+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.272120+0000","last_change":"2023-10-16T11:25:49.272120+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.272120+0000","last_undegraded":"2023-10-16T11:25:49.272120+0000","last_fullsized":"2023-10-16T11:25:49.272120+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3b","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.257086+0000","last_change":"2023-10-16T11:25:37.177505+0000","last_active":"2023-10-16T11:25:40.257086+0000","last_peered":"2023-10-16T11:25:40.257086+0000","last_clean":"2023-10-16T11:25:40.257086+0000","last_became_active":"2023-10-16T11:25:37.174334+0000","last_became_peered":"2023-10-16T11:25:37.174334+0000","last_unstale":"2023-10-16T11:25:40.257086+0000","last_undegraded":"2023-10-16T11:25:40.257086+0000","last_fullsized":"2023-10-16T11:25:40.257086+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.267494+0000","last_change":"2023-10-16T11:25:49.267494+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.267494+0000","last_undegraded":"2023-10-16T11:25:49.267494+0000","last_fullsized":"2023-10-16T11:25:49.267494+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3a","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255230+0000","last_change":"2023-10-16T11:25:37.177781+0000","last_active":"2023-10-16T11:25:40.255230+0000","last_peered":"2023-10-16T11:25:40.255230+0000","last_clean":"2023-10-16T11:25:40.255230+0000","last_became_active":"2023-10-16T11:25:37.177501+0000","last_became_peered":"2023-10-16T11:25:37.177501+0000","last_unstale":"2023-10-16T11:25:40.255230+0000","last_undegraded":"2023-10-16T11:25:40.255230+0000","last_fullsized":"2023-10-16T11:25:40.255230+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.279696+0000","last_change":"2023-10-16T11:25:49.279696+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.279696+0000","last_undegraded":"2023-10-16T11:25:49.279696+0000","last_fullsized":"2023-10-16T11:25:49.279696+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.39","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252215+0000","last_change":"2023-10-16T11:25:37.182745+0000","last_active":"2023-10-16T11:25:40.252215+0000","last_peered":"2023-10-16T11:25:40.252215+0000","last_clean":"2023-10-16T11:25:40.252215+0000","last_became_active":"2023-10-16T11:25:37.182437+0000","last_became_peered":"2023-10-16T11:25:37.182437+0000","last_unstale":"2023-10-16T11:25:40.252215+0000","last_undegraded":"2023-10-16T11:25:40.252215+0000","last_fullsized":"2023-10-16T11:25:40.252215+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.269438+0000","last_change":"2023-10-16T11:25:49.269438+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.269438+0000","last_undegraded":"2023-10-16T11:25:49.269438+0000","last_fullsized":"2023-10-16T11:25:49.269438+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.38","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254686+0000","last_change":"2023-10-16T11:25:37.174929+0000","last_active":"2023-10-16T11:25:40.254686+0000","last_peered":"2023-10-16T11:25:40.254686+0000","last_clean":"2023-10-16T11:25:40.254686+0000","last_became_active":"2023-10-16T11:25:37.172810+0000","last_became_peered":"2023-10-16T11:25:37.172810+0000","last_unstale":"2023-10-16T11:25:40.254686+0000","last_undegraded":"2023-10-16T11:25:40.254686+0000","last_fullsized":"2023-10-16T11:25:40.254686+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.273300+0000","last_change":"2023-10-16T11:25:49.273300+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.273300+0000","last_undegraded":"2023-10-16T11:25:49.273300+0000","last_fullsized":"2023-10-16T11:25:49.273300+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.37","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.259355+0000","last_change":"2023-10-16T11:25:37.177647+0000","last_active":"2023-10-16T11:25:40.259355+0000","last_peered":"2023-10-16T11:25:40.259355+0000","last_clean":"2023-10-16T11:25:40.259355+0000","last_became_active":"2023-10-16T11:25:37.176875+0000","last_became_peered":"2023-10-16T11:25:37.176875+0000","last_unstale":"2023-10-16T11:25:40.259355+0000","last_undegraded":"2023-10-16T11:25:40.259355+0000","last_fullsized":"2023-10-16T11:25:40.259355+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:21.630921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.283150+0000","last_change":"2023-10-16T11:25:49.283150+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.283150+0000","last_undegraded":"2023-10-16T11:25:49.283150+0000","last_fullsized":"2023-10-16T11:25:49.283150+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.36","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252664+0000","last_change":"2023-10-16T11:25:37.172841+0000","last_active":"2023-10-16T11:25:40.252664+0000","last_peered":"2023-10-16T11:25:40.252664+0000","last_clean":"2023-10-16T11:25:40.252664+0000","last_became_active":"2023-10-16T11:25:37.172550+0000","last_became_peered":"2023-10-16T11:25:37.172550+0000","last_unstale":"2023-10-16T11:25:40.252664+0000","last_undegraded":"2023-10-16T11:25:40.252664+0000","last_fullsized":"2023-10-16T11:25:40.252664+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.274776+0000","last_change":"2023-10-16T11:25:49.274776+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.274776+0000","last_undegraded":"2023-10-16T11:25:49.274776+0000","last_fullsized":"2023-10-16T11:25:49.274776+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.35","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249412+0000","last_change":"2023-10-16T11:25:37.181386+0000","last_active":"2023-10-16T11:25:40.249412+0000","last_peered":"2023-10-16T11:25:40.249412+0000","last_clean":"2023-10-16T11:25:40.249412+0000","last_became_active":"2023-10-16T11:25:37.181007+0000","last_became_peered":"2023-10-16T11:25:37.181007+0000","last_unstale":"2023-10-16T11:25:40.249412+0000","last_undegraded":"2023-10-16T11:25:40.249412+0000","last_fullsized":"2023-10-16T11:25:40.249412+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:46.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.277979+0000","last_change":"2023-10-16T11:25:49.277979+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.277979+0000","last_undegraded":"2023-10-16T11:25:49.277979+0000","last_fullsized":"2023-10-16T11:25:49.277979+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.34","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254920+0000","last_change":"2023-10-16T11:25:37.179103+0000","last_active":"2023-10-16T11:25:40.254920+0000","last_peered":"2023-10-16T11:25:40.254920+0000","last_clean":"2023-10-16T11:25:40.254920+0000","last_became_active":"2023-10-16T11:25:37.178445+0000","last_became_peered":"2023-10-16T11:25:37.178445+0000","last_unstale":"2023-10-16T11:25:40.254920+0000","last_undegraded":"2023-10-16T11:25:40.254920+0000","last_fullsized":"2023-10-16T11:25:40.254920+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:38.218704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.269375+0000","last_change":"2023-10-16T11:25:49.269375+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.269375+0000","last_undegraded":"2023-10-16T11:25:49.269375+0000","last_fullsized":"2023-10-16T11:25:49.269375+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.33","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252560+0000","last_change":"2023-10-16T11:25:37.182921+0000","last_active":"2023-10-16T11:25:40.252560+0000","last_peered":"2023-10-16T11:25:40.252560+0000","last_clean":"2023-10-16T11:25:40.252560+0000","last_became_active":"2023-10-16T11:25:37.182603+0000","last_became_peered":"2023-10-16T11:25:37.182603+0000","last_unstale":"2023-10-16T11:25:40.252560+0000","last_undegraded":"2023-10-16T11:25:40.252560+0000","last_fullsized":"2023-10-16T11:25:40.252560+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.283709+0000","last_change":"2023-10-16T11:25:49.283709+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.283709+0000","last_undegraded":"2023-10-16T11:25:49.283709+0000","last_fullsized":"2023-10-16T11:25:49.283709+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.32","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248649+0000","last_change":"2023-10-16T11:25:37.171715+0000","last_active":"2023-10-16T11:25:40.248649+0000","last_peered":"2023-10-16T11:25:40.248649+0000","last_clean":"2023-10-16T11:25:40.248649+0000","last_became_active":"2023-10-16T11:25:37.171421+0000","last_became_peered":"2023-10-16T11:25:37.171421+0000","last_unstale":"2023-10-16T11:25:40.248649+0000","last_undegraded":"2023-10-16T11:25:40.248649+0000","last_fullsized":"2023-10-16T11:25:40.248649+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:10.207081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.266505+0000","last_change":"2023-10-16T11:25:49.266505+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.266505+0000","last_undegraded":"2023-10-16T11:25:49.266505+0000","last_fullsized":"2023-10-16T11:25:49.266505+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.31","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.259204+0000","last_change":"2023-10-16T11:25:37.177928+0000","last_active":"2023-10-16T11:25:40.259204+0000","last_peered":"2023-10-16T11:25:40.259204+0000","last_clean":"2023-10-16T11:25:40.259204+0000","last_became_active":"2023-10-16T11:25:37.177612+0000","last_became_peered":"2023-10-16T11:25:37.177612+0000","last_unstale":"2023-10-16T11:25:40.259204+0000","last_undegraded":"2023-10-16T11:25:40.259204+0000","last_fullsized":"2023-10-16T11:25:40.259204+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:44.826687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.274689+0000","last_change":"2023-10-16T11:25:49.274689+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.274689+0000","last_undegraded":"2023-10-16T11:25:49.274689+0000","last_fullsized":"2023-10-16T11:25:49.274689+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.30","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249967+0000","last_change":"2023-10-16T11:25:37.170292+0000","last_active":"2023-10-16T11:25:40.249967+0000","last_peered":"2023-10-16T11:25:40.249967+0000","last_clean":"2023-10-16T11:25:40.249967+0000","last_became_active":"2023-10-16T11:25:37.169997+0000","last_became_peered":"2023-10-16T11:25:37.169997+0000","last_unstale":"2023-10-16T11:25:40.249967+0000","last_undegraded":"2023-10-16T11:25:40.249967+0000","last_fullsized":"2023-10-16T11:25:40.249967+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:09.329721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.267011+0000","last_change":"2023-10-16T11:25:49.267011+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.267011+0000","last_undegraded":"2023-10-16T11:25:49.267011+0000","last_fullsized":"2023-10-16T11:25:49.267011+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2f","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255135+0000","last_change":"2023-10-16T11:25:37.162923+0000","last_active":"2023-10-16T11:25:40.255135+0000","last_peered":"2023-10-16T11:25:40.255135+0000","last_clean":"2023-10-16T11:25:40.255135+0000","last_became_active":"2023-10-16T11:25:37.162649+0000","last_became_peered":"2023-10-16T11:25:37.162649+0000","last_unstale":"2023-10-16T11:25:40.255135+0000","last_undegraded":"2023-10-16T11:25:40.255135+0000","last_fullsized":"2023-10-16T11:25:40.255135+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:46.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.279612+0000","last_change":"2023-10-16T11:25:49.279612+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.279612+0000","last_undegraded":"2023-10-16T11:25:49.279612+0000","last_fullsized":"2023-10-16T11:25:49.279612+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.2e","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248737+0000","last_change":"2023-10-16T11:25:37.174044+0000","last_active":"2023-10-16T11:25:40.248737+0000","last_peered":"2023-10-16T11:25:40.248737+0000","last_clean":"2023-10-16T11:25:40.248737+0000","last_became_active":"2023-10-16T11:25:37.166694+0000","last_became_peered":"2023-10-16T11:25:37.166694+0000","last_unstale":"2023-10-16T11:25:40.248737+0000","last_undegraded":"2023-10-16T11:25:40.248737+0000","last_fullsized":"2023-10-16T11:25:40.248737+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:23.146288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.271754+0000","last_change":"2023-10-16T11:25:49.271754+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.271754+0000","last_undegraded":"2023-10-16T11:25:49.271754+0000","last_fullsized":"2023-10-16T11:25:49.271754+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2d","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254960+0000","last_change":"2023-10-16T11:25:37.183893+0000","last_active":"2023-10-16T11:25:40.254960+0000","last_peered":"2023-10-16T11:25:40.254960+0000","last_clean":"2023-10-16T11:25:40.254960+0000","last_became_active":"2023-10-16T11:25:37.183586+0000","last_became_peered":"2023-10-16T11:25:37.183586+0000","last_unstale":"2023-10-16T11:25:40.254960+0000","last_undegraded":"2023-10-16T11:25:40.254960+0000","last_fullsized":"2023-10-16T11:25:40.254960+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:43.813878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.274305+0000","last_change":"2023-10-16T11:25:49.274305+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.274305+0000","last_undegraded":"2023-10-16T11:25:49.274305+0000","last_fullsized":"2023-10-16T11:25:49.274305+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2c","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.250037+0000","last_change":"2023-10-16T11:25:37.172984+0000","last_active":"2023-10-16T11:25:40.250037+0000","last_peered":"2023-10-16T11:25:40.250037+0000","last_clean":"2023-10-16T11:25:40.250037+0000","last_became_active":"2023-10-16T11:25:37.172403+0000","last_became_peered":"2023-10-16T11:25:37.172403+0000","last_unstale":"2023-10-16T11:25:40.250037+0000","last_undegraded":"2023-10-16T11:25:40.250037+0000","last_fullsized":"2023-10-16T11:25:40.250037+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:03.912995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.272751+0000","last_change":"2023-10-16T11:25:49.272751+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.272751+0000","last_undegraded":"2023-10-16T11:25:49.272751+0000","last_fullsized":"2023-10-16T11:25:49.272751+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2b","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254876+0000","last_change":"2023-10-16T11:25:37.173026+0000","last_active":"2023-10-16T11:25:40.254876+0000","last_peered":"2023-10-16T11:25:40.254876+0000","last_clean":"2023-10-16T11:25:40.254876+0000","last_became_active":"2023-10-16T11:25:37.172680+0000","last_became_peered":"2023-10-16T11:25:37.172680+0000","last_unstale":"2023-10-16T11:25:40.254876+0000","last_undegraded":"2023-10-16T11:25:40.254876+0000","last_fullsized":"2023-10-16T11:25:40.254876+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:56.314696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.266977+0000","last_change":"2023-10-16T11:25:49.266977+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.266977+0000","last_undegraded":"2023-10-16T11:25:49.266977+0000","last_fullsized":"2023-10-16T11:25:49.266977+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2a","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252432+0000","last_change":"2023-10-16T11:25:37.176870+0000","last_active":"2023-10-16T11:25:40.252432+0000","last_peered":"2023-10-16T11:25:40.252432+0000","last_clean":"2023-10-16T11:25:40.252432+0000","last_became_active":"2023-10-16T11:25:37.175230+0000","last_became_peered":"2023-10-16T11:25:37.175230+0000","last_unstale":"2023-10-16T11:25:40.252432+0000","last_undegraded":"2023-10-16T11:25:40.252432+0000","last_fullsized":"2023-10-16T11:25:40.252432+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.267347+0000","last_change":"2023-10-16T11:25:49.267347+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.267347+0000","last_undegraded":"2023-10-16T11:25:49.267347+0000","last_fullsized":"2023-10-16T11:25:49.267347+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.29","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254808+0000","last_change":"2023-10-16T11:25:37.169637+0000","last_active":"2023-10-16T11:25:40.254808+0000","last_peered":"2023-10-16T11:25:40.254808+0000","last_clean":"2023-10-16T11:25:40.254808+0000","last_became_active":"2023-10-16T11:25:37.169375+0000","last_became_peered":"2023-10-16T11:25:37.169375+0000","last_unstale":"2023-10-16T11:25:40.254808+0000","last_undegraded":"2023-10-16T11:25:40.254808+0000","last_fullsized":"2023-10-16T11:25:40.254808+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:42:23.188876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.272728+0000","last_change":"2023-10-16T11:25:49.272728+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.272728+0000","last_undegraded":"2023-10-16T11:25:49.272728+0000","last_fullsized":"2023-10-16T11:25:49.272728+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.28","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248558+0000","last_change":"2023-10-16T11:25:37.187270+0000","last_active":"2023-10-16T11:25:40.248558+0000","last_peered":"2023-10-16T11:25:40.248558+0000","last_clean":"2023-10-16T11:25:40.248558+0000","last_became_active":"2023-10-16T11:25:37.186962+0000","last_became_peered":"2023-10-16T11:25:37.186962+0000","last_unstale":"2023-10-16T11:25:40.248558+0000","last_undegraded":"2023-10-16T11:25:40.248558+0000","last_fullsized":"2023-10-16T11:25:40.248558+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:56:45.915920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.267168+0000","last_change":"2023-10-16T11:25:49.267168+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.267168+0000","last_undegraded":"2023-10-16T11:25:49.267168+0000","last_fullsized":"2023-10-16T11:25:49.267168+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.27","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255281+0000","last_change":"2023-10-16T11:25:37.177057+0000","last_active":"2023-10-16T11:25:40.255281+0000","last_peered":"2023-10-16T11:25:40.255281+0000","last_clean":"2023-10-16T11:25:40.255281+0000","last_became_active":"2023-10-16T11:25:37.171369+0000","last_became_peered":"2023-10-16T11:25:37.171369+0000","last_unstale":"2023-10-16T11:25:40.255281+0000","last_undegraded":"2023-10-16T11:25:40.255281+0000","last_fullsized":"2023-10-16T11:25:40.255281+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.279039+0000","last_change":"2023-10-16T11:25:49.279039+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.279039+0000","last_undegraded":"2023-10-16T11:25:49.279039+0000","last_fullsized":"2023-10-16T11:25:49.279039+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.26","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252706+0000","last_change":"2023-10-16T11:25:37.182875+0000","last_active":"2023-10-16T11:25:40.252706+0000","last_peered":"2023-10-16T11:25:40.252706+0000","last_clean":"2023-10-16T11:25:40.252706+0000","last_became_active":"2023-10-16T11:25:37.182570+0000","last_became_peered":"2023-10-16T11:25:37.182570+0000","last_unstale":"2023-10-16T11:25:40.252706+0000","last_undegraded":"2023-10-16T11:25:40.252706+0000","last_fullsized":"2023-10-16T11:25:40.252706+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:14.504702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.272764+0000","last_change":"2023-10-16T11:25:49.272764+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.272764+0000","last_undegraded":"2023-10-16T11:25:49.272764+0000","last_fullsized":"2023-10-16T11:25:49.272764+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.25","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249433+0000","last_change":"2023-10-16T11:25:37.178245+0000","last_active":"2023-10-16T11:25:40.249433+0000","last_peered":"2023-10-16T11:25:40.249433+0000","last_clean":"2023-10-16T11:25:40.249433+0000","last_became_active":"2023-10-16T11:25:37.177914+0000","last_became_peered":"2023-10-16T11:25:37.177914+0000","last_unstale":"2023-10-16T11:25:40.249433+0000","last_undegraded":"2023-10-16T11:25:40.249433+0000","last_fullsized":"2023-10-16T11:25:40.249433+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:12.014513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.267216+0000","last_change":"2023-10-16T11:25:49.267216+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.267216+0000","last_undegraded":"2023-10-16T11:25:49.267216+0000","last_fullsized":"2023-10-16T11:25:49.267216+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.24","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248686+0000","last_change":"2023-10-16T11:25:37.170623+0000","last_active":"2023-10-16T11:25:40.248686+0000","last_peered":"2023-10-16T11:25:40.248686+0000","last_clean":"2023-10-16T11:25:40.248686+0000","last_became_active":"2023-10-16T11:25:37.170365+0000","last_became_peered":"2023-10-16T11:25:37.170365+0000","last_unstale":"2023-10-16T11:25:40.248686+0000","last_undegraded":"2023-10-16T11:25:40.248686+0000","last_fullsized":"2023-10-16T11:25:40.248686+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:52.443160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.266326+0000","last_change":"2023-10-16T11:25:49.266326+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.266326+0000","last_undegraded":"2023-10-16T11:25:49.266326+0000","last_fullsized":"2023-10-16T11:25:49.266326+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.23","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249332+0000","last_change":"2023-10-16T11:25:37.178693+0000","last_active":"2023-10-16T11:25:40.249332+0000","last_peered":"2023-10-16T11:25:40.249332+0000","last_clean":"2023-10-16T11:25:40.249332+0000","last_became_active":"2023-10-16T11:25:37.175403+0000","last_became_peered":"2023-10-16T11:25:37.175403+0000","last_unstale":"2023-10-16T11:25:40.249332+0000","last_undegraded":"2023-10-16T11:25:40.249332+0000","last_fullsized":"2023-10-16T11:25:40.249332+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:15:31.572558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.267625+0000","last_change":"2023-10-16T11:25:49.267625+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.267625+0000","last_undegraded":"2023-10-16T11:25:49.267625+0000","last_fullsized":"2023-10-16T11:25:49.267625+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.22","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.257055+0000","last_change":"2023-10-16T11:25:37.181812+0000","last_active":"2023-10-16T11:25:40.257055+0000","last_peered":"2023-10-16T11:25:40.257055+0000","last_clean":"2023-10-16T11:25:40.257055+0000","last_became_active":"2023-10-16T11:25:37.181529+0000","last_became_peered":"2023-10-16T11:25:37.181529+0000","last_unstale":"2023-10-16T11:25:40.257055+0000","last_undegraded":"2023-10-16T11:25:40.257055+0000","last_fullsized":"2023-10-16T11:25:40.257055+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:35.912014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.269415+0000","last_change":"2023-10-16T11:25:49.269415+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.269415+0000","last_undegraded":"2023-10-16T11:25:49.269415+0000","last_fullsized":"2023-10-16T11:25:49.269415+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.21","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255217+0000","last_change":"2023-10-16T11:25:37.166981+0000","last_active":"2023-10-16T11:25:40.255217+0000","last_peered":"2023-10-16T11:25:40.255217+0000","last_clean":"2023-10-16T11:25:40.255217+0000","last_became_active":"2023-10-16T11:25:37.164147+0000","last_became_peered":"2023-10-16T11:25:37.164147+0000","last_unstale":"2023-10-16T11:25:40.255217+0000","last_undegraded":"2023-10-16T11:25:40.255217+0000","last_fullsized":"2023-10-16T11:25:40.255217+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.273227+0000","last_change":"2023-10-16T11:25:49.273227+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.273227+0000","last_undegraded":"2023-10-16T11:25:49.273227+0000","last_fullsized":"2023-10-16T11:25:49.273227+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.20","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248589+0000","last_change":"2023-10-16T11:25:37.178631+0000","last_active":"2023-10-16T11:25:40.248589+0000","last_peered":"2023-10-16T11:25:40.248589+0000","last_clean":"2023-10-16T11:25:40.248589+0000","last_became_active":"2023-10-16T11:25:37.167402+0000","last_became_peered":"2023-10-16T11:25:37.167402+0000","last_unstale":"2023-10-16T11:25:40.248589+0000","last_undegraded":"2023-10-16T11:25:40.248589+0000","last_fullsized":"2023-10-16T11:25:40.248589+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:55.572264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.250139+0000","last_change":"2023-10-16T11:25:37.177706+0000","last_active":"2023-10-16T11:25:40.250139+0000","last_peered":"2023-10-16T11:25:40.250139+0000","last_clean":"2023-10-16T11:25:40.250139+0000","last_became_active":"2023-10-16T11:25:37.176806+0000","last_became_peered":"2023-10-16T11:25:37.176806+0000","last_unstale":"2023-10-16T11:25:40.250139+0000","last_undegraded":"2023-10-16T11:25:40.250139+0000","last_fullsized":"2023-10-16T11:25:40.250139+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:42:32.585642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1e","version":"236'1","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.177232+0000","last_change":"2023-10-16T11:25:36.169327+0000","last_active":"2023-10-16T11:25:38.177232+0000","last_peered":"2023-10-16T11:25:38.177232+0000","last_clean":"2023-10-16T11:25:38.177232+0000","last_became_active":"2023-10-16T11:25:36.169046+0000","last_became_peered":"2023-10-16T11:25:36.169046+0000","last_unstale":"2023-10-16T11:25:38.177232+0000","last_undegraded":"2023-10-16T11:25:38.177232+0000","last_fullsized":"2023-10-16T11:25:38.177232+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:34.899695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249353+0000","last_change":"2023-10-16T11:25:37.173736+0000","last_active":"2023-10-16T11:25:40.249353+0000","last_peered":"2023-10-16T11:25:40.249353+0000","last_clean":"2023-10-16T11:25:40.249353+0000","last_became_active":"2023-10-16T11:25:37.173482+0000","last_became_peered":"2023-10-16T11:25:37.173482+0000","last_unstale":"2023-10-16T11:25:40.249353+0000","last_undegraded":"2023-10-16T11:25:40.249353+0000","last_fullsized":"2023-10-16T11:25:40.249353+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:42:23.188876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1d","version":"236'2","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616465+0000","last_change":"2023-10-16T11:25:36.172861+0000","last_active":"2023-10-16T11:25:38.616465+0000","last_peered":"2023-10-16T11:25:38.616465+0000","last_clean":"2023-10-16T11:25:38.616465+0000","last_became_active":"2023-10-16T11:25:36.172574+0000","last_became_peered":"2023-10-16T11:25:36.172574+0000","last_unstale":"2023-10-16T11:25:38.616465+0000","last_undegraded":"2023-10-16T11:25:38.616465+0000","last_fullsized":"2023-10-16T11:25:38.616465+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:22:17.667795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255008+0000","last_change":"2023-10-16T11:25:37.167451+0000","last_active":"2023-10-16T11:25:40.255008+0000","last_peered":"2023-10-16T11:25:40.255008+0000","last_clean":"2023-10-16T11:25:40.255008+0000","last_became_active":"2023-10-16T11:25:37.167157+0000","last_became_peered":"2023-10-16T11:25:37.167157+0000","last_unstale":"2023-10-16T11:25:40.255008+0000","last_undegraded":"2023-10-16T11:25:40.255008+0000","last_fullsized":"2023-10-16T11:25:40.255008+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:14.504702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1c","version":"236'2","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616883+0000","last_change":"2023-10-16T11:25:36.153273+0000","last_active":"2023-10-16T11:25:38.616883+0000","last_peered":"2023-10-16T11:25:38.616883+0000","last_clean":"2023-10-16T11:25:38.616883+0000","last_became_active":"2023-10-16T11:25:36.152960+0000","last_became_peered":"2023-10-16T11:25:36.152960+0000","last_unstale":"2023-10-16T11:25:38.616883+0000","last_undegraded":"2023-10-16T11:25:38.616883+0000","last_fullsized":"2023-10-16T11:25:38.616883+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:08.218268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249306+0000","last_change":"2023-10-16T11:25:37.174071+0000","last_active":"2023-10-16T11:25:40.249306+0000","last_peered":"2023-10-16T11:25:40.249306+0000","last_clean":"2023-10-16T11:25:40.249306+0000","last_became_active":"2023-10-16T11:25:37.173436+0000","last_became_peered":"2023-10-16T11:25:37.173436+0000","last_unstale":"2023-10-16T11:25:40.249306+0000","last_undegraded":"2023-10-16T11:25:40.249306+0000","last_fullsized":"2023-10-16T11:25:40.249306+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1b","version":"236'1","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.171367+0000","last_change":"2023-10-16T11:25:36.156408+0000","last_active":"2023-10-16T11:25:38.171367+0000","last_peered":"2023-10-16T11:25:38.171367+0000","last_clean":"2023-10-16T11:25:38.171367+0000","last_became_active":"2023-10-16T11:25:36.155786+0000","last_became_peered":"2023-10-16T11:25:36.155786+0000","last_unstale":"2023-10-16T11:25:38.171367+0000","last_undegraded":"2023-10-16T11:25:38.171367+0000","last_fullsized":"2023-10-16T11:25:38.171367+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:01.925277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.259259+0000","last_change":"2023-10-16T11:25:37.179656+0000","last_active":"2023-10-16T11:25:40.259259+0000","last_peered":"2023-10-16T11:25:40.259259+0000","last_clean":"2023-10-16T11:25:40.259259+0000","last_became_active":"2023-10-16T11:25:37.179361+0000","last_became_peered":"2023-10-16T11:25:37.179361+0000","last_unstale":"2023-10-16T11:25:40.259259+0000","last_undegraded":"2023-10-16T11:25:40.259259+0000","last_fullsized":"2023-10-16T11:25:40.259259+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:09.230587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"236'1","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193985+0000","last_change":"2023-10-16T11:25:36.169348+0000","last_active":"2023-10-16T11:25:39.193985+0000","last_peered":"2023-10-16T11:25:39.193985+0000","last_clean":"2023-10-16T11:25:39.193985+0000","last_became_active":"2023-10-16T11:25:36.169053+0000","last_became_peered":"2023-10-16T11:25:36.169053+0000","last_unstale":"2023-10-16T11:25:39.193985+0000","last_undegraded":"2023-10-16T11:25:39.193985+0000","last_fullsized":"2023-10-16T11:25:39.193985+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:05.819502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249986+0000","last_change":"2023-10-16T11:25:37.176966+0000","last_active":"2023-10-16T11:25:40.249986+0000","last_peered":"2023-10-16T11:25:40.249986+0000","last_clean":"2023-10-16T11:25:40.249986+0000","last_became_active":"2023-10-16T11:25:37.174621+0000","last_became_peered":"2023-10-16T11:25:37.174621+0000","last_unstale":"2023-10-16T11:25:40.249986+0000","last_undegraded":"2023-10-16T11:25:40.249986+0000","last_fullsized":"2023-10-16T11:25:40.249986+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:56:09.516122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174368+0000","last_change":"2023-10-16T11:25:36.169635+0000","last_active":"2023-10-16T11:25:38.174368+0000","last_peered":"2023-10-16T11:25:38.174368+0000","last_clean":"2023-10-16T11:25:38.174368+0000","last_became_active":"2023-10-16T11:25:36.169016+0000","last_became_peered":"2023-10-16T11:25:36.169016+0000","last_unstale":"2023-10-16T11:25:38.174368+0000","last_undegraded":"2023-10-16T11:25:38.174368+0000","last_fullsized":"2023-10-16T11:25:38.174368+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:53:06.412234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248609+0000","last_change":"2023-10-16T11:25:37.184475+0000","last_active":"2023-10-16T11:25:40.248609+0000","last_peered":"2023-10-16T11:25:40.248609+0000","last_clean":"2023-10-16T11:25:40.248609+0000","last_became_active":"2023-10-16T11:25:37.179914+0000","last_became_peered":"2023-10-16T11:25:37.179914+0000","last_unstale":"2023-10-16T11:25:40.248609+0000","last_undegraded":"2023-10-16T11:25:40.248609+0000","last_fullsized":"2023-10-16T11:25:40.248609+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:53.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616094+0000","last_change":"2023-10-16T11:25:36.155589+0000","last_active":"2023-10-16T11:25:38.616094+0000","last_peered":"2023-10-16T11:25:38.616094+0000","last_clean":"2023-10-16T11:25:38.616094+0000","last_became_active":"2023-10-16T11:25:36.155087+0000","last_became_peered":"2023-10-16T11:25:36.155087+0000","last_unstale":"2023-10-16T11:25:38.616094+0000","last_undegraded":"2023-10-16T11:25:38.616094+0000","last_fullsized":"2023-10-16T11:25:38.616094+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:52:38.937212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248756+0000","last_change":"2023-10-16T11:25:37.175447+0000","last_active":"2023-10-16T11:25:40.248756+0000","last_peered":"2023-10-16T11:25:40.248756+0000","last_clean":"2023-10-16T11:25:40.248756+0000","last_became_active":"2023-10-16T11:25:37.173563+0000","last_became_peered":"2023-10-16T11:25:37.173563+0000","last_unstale":"2023-10-16T11:25:40.248756+0000","last_undegraded":"2023-10-16T11:25:40.248756+0000","last_fullsized":"2023-10-16T11:25:40.248756+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:54.544135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.198655+0000","last_change":"2023-10-16T11:25:36.169351+0000","last_active":"2023-10-16T11:25:39.198655+0000","last_peered":"2023-10-16T11:25:39.198655+0000","last_clean":"2023-10-16T11:25:39.198655+0000","last_became_active":"2023-10-16T11:25:36.168347+0000","last_became_peered":"2023-10-16T11:25:36.168347+0000","last_unstale":"2023-10-16T11:25:39.198655+0000","last_undegraded":"2023-10-16T11:25:39.198655+0000","last_fullsized":"2023-10-16T11:25:39.198655+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:19.077096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249471+0000","last_change":"2023-10-16T11:25:37.179048+0000","last_active":"2023-10-16T11:25:40.249471+0000","last_peered":"2023-10-16T11:25:40.249471+0000","last_clean":"2023-10-16T11:25:40.249471+0000","last_became_active":"2023-10-16T11:25:37.178751+0000","last_became_peered":"2023-10-16T11:25:37.178751+0000","last_unstale":"2023-10-16T11:25:40.249471+0000","last_undegraded":"2023-10-16T11:25:40.249471+0000","last_fullsized":"2023-10-16T11:25:40.249471+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:48:10.906108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616368+0000","last_change":"2023-10-16T11:25:36.154256+0000","last_active":"2023-10-16T11:25:38.616368+0000","last_peered":"2023-10-16T11:25:38.616368+0000","last_clean":"2023-10-16T11:25:38.616368+0000","last_became_active":"2023-10-16T11:25:36.153421+0000","last_became_peered":"2023-10-16T11:25:36.153421+0000","last_unstale":"2023-10-16T11:25:38.616368+0000","last_undegraded":"2023-10-16T11:25:38.616368+0000","last_fullsized":"2023-10-16T11:25:38.616368+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:09.436508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.250064+0000","last_change":"2023-10-16T11:25:37.188270+0000","last_active":"2023-10-16T11:25:40.250064+0000","last_peered":"2023-10-16T11:25:40.250064+0000","last_clean":"2023-10-16T11:25:40.250064+0000","last_became_active":"2023-10-16T11:25:37.187995+0000","last_became_peered":"2023-10-16T11:25:37.187995+0000","last_unstale":"2023-10-16T11:25:40.250064+0000","last_undegraded":"2023-10-16T11:25:40.250064+0000","last_fullsized":"2023-10-16T11:25:40.250064+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:42:22.542072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.15","version":"236'3","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193813+0000","last_change":"2023-10-16T11:25:36.171645+0000","last_active":"2023-10-16T11:25:39.193813+0000","last_peered":"2023-10-16T11:25:39.193813+0000","last_clean":"2023-10-16T11:25:39.193813+0000","last_became_active":"2023-10-16T11:25:36.170729+0000","last_became_peered":"2023-10-16T11:25:36.170729+0000","last_unstale":"2023-10-16T11:25:39.193813+0000","last_undegraded":"2023-10-16T11:25:39.193813+0000","last_fullsized":"2023-10-16T11:25:39.193813+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:34.362958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252622+0000","last_change":"2023-10-16T11:25:37.163991+0000","last_active":"2023-10-16T11:25:40.252622+0000","last_peered":"2023-10-16T11:25:40.252622+0000","last_clean":"2023-10-16T11:25:40.252622+0000","last_became_active":"2023-10-16T11:25:37.163080+0000","last_became_peered":"2023-10-16T11:25:37.163080+0000","last_unstale":"2023-10-16T11:25:40.252622+0000","last_undegraded":"2023-10-16T11:25:40.252622+0000","last_fullsized":"2023-10-16T11:25:40.252622+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.14","version":"237'8","reported_seq":21,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:46.774610+0000","last_change":"2023-10-16T11:25:36.173259+0000","last_active":"2023-10-16T11:25:46.774610+0000","last_peered":"2023-10-16T11:25:46.774610+0000","last_clean":"2023-10-16T11:25:46.774610+0000","last_became_active":"2023-10-16T11:25:36.172980+0000","last_became_peered":"2023-10-16T11:25:36.172980+0000","last_unstale":"2023-10-16T11:25:46.774610+0000","last_undegraded":"2023-10-16T11:25:46.774610+0000","last_fullsized":"2023-10-16T11:25:46.774610+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:33.782641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252475+0000","last_change":"2023-10-16T11:25:37.166081+0000","last_active":"2023-10-16T11:25:40.252475+0000","last_peered":"2023-10-16T11:25:40.252475+0000","last_clean":"2023-10-16T11:25:40.252475+0000","last_became_active":"2023-10-16T11:25:37.165150+0000","last_became_peered":"2023-10-16T11:25:37.165150+0000","last_unstale":"2023-10-16T11:25:40.252475+0000","last_undegraded":"2023-10-16T11:25:40.252475+0000","last_fullsized":"2023-10-16T11:25:40.252475+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:56.314696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"236'2","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193905+0000","last_change":"2023-10-16T11:25:36.171468+0000","last_active":"2023-10-16T11:25:39.193905+0000","last_peered":"2023-10-16T11:25:39.193905+0000","last_clean":"2023-10-16T11:25:39.193905+0000","last_became_active":"2023-10-16T11:25:36.171236+0000","last_became_peered":"2023-10-16T11:25:36.171236+0000","last_unstale":"2023-10-16T11:25:39.193905+0000","last_undegraded":"2023-10-16T11:25:39.193905+0000","last_fullsized":"2023-10-16T11:25:39.193905+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:17.018903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.259283+0000","last_change":"2023-10-16T11:25:37.178517+0000","last_active":"2023-10-16T11:25:40.259283+0000","last_peered":"2023-10-16T11:25:40.259283+0000","last_clean":"2023-10-16T11:25:40.259283+0000","last_became_active":"2023-10-16T11:25:37.177794+0000","last_became_peered":"2023-10-16T11:25:37.177794+0000","last_unstale":"2023-10-16T11:25:40.259283+0000","last_undegraded":"2023-10-16T11:25:40.259283+0000","last_fullsized":"2023-10-16T11:25:40.259283+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:52:39.949531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616389+0000","last_change":"2023-10-16T11:25:36.162595+0000","last_active":"2023-10-16T11:25:38.616389+0000","last_peered":"2023-10-16T11:25:38.616389+0000","last_clean":"2023-10-16T11:25:38.616389+0000","last_became_active":"2023-10-16T11:25:36.162249+0000","last_became_peered":"2023-10-16T11:25:36.162249+0000","last_unstale":"2023-10-16T11:25:38.616389+0000","last_undegraded":"2023-10-16T11:25:38.616389+0000","last_fullsized":"2023-10-16T11:25:38.616389+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:49.547165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255198+0000","last_change":"2023-10-16T11:25:37.164604+0000","last_active":"2023-10-16T11:25:40.255198+0000","last_peered":"2023-10-16T11:25:40.255198+0000","last_clean":"2023-10-16T11:25:40.255198+0000","last_became_active":"2023-10-16T11:25:37.162365+0000","last_became_peered":"2023-10-16T11:25:37.162365+0000","last_unstale":"2023-10-16T11:25:40.255198+0000","last_undegraded":"2023-10-16T11:25:40.255198+0000","last_fullsized":"2023-10-16T11:25:40.255198+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:35.912014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174625+0000","last_change":"2023-10-16T11:25:36.170014+0000","last_active":"2023-10-16T11:25:38.174625+0000","last_peered":"2023-10-16T11:25:38.174625+0000","last_clean":"2023-10-16T11:25:38.174625+0000","last_became_active":"2023-10-16T11:25:36.169755+0000","last_became_peered":"2023-10-16T11:25:36.169755+0000","last_unstale":"2023-10-16T11:25:38.174625+0000","last_undegraded":"2023-10-16T11:25:38.174625+0000","last_fullsized":"2023-10-16T11:25:38.174625+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:02.119904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252284+0000","last_change":"2023-10-16T11:25:37.178789+0000","last_active":"2023-10-16T11:25:40.252284+0000","last_peered":"2023-10-16T11:25:40.252284+0000","last_clean":"2023-10-16T11:25:40.252284+0000","last_became_active":"2023-10-16T11:25:37.178510+0000","last_became_peered":"2023-10-16T11:25:37.178510+0000","last_unstale":"2023-10-16T11:25:40.252284+0000","last_undegraded":"2023-10-16T11:25:40.252284+0000","last_fullsized":"2023-10-16T11:25:40.252284+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.10","version":"236'8","reported_seq":21,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182764+0000","last_change":"2023-10-16T11:25:36.168889+0000","last_active":"2023-10-16T11:25:38.182764+0000","last_peered":"2023-10-16T11:25:38.182764+0000","last_clean":"2023-10-16T11:25:38.182764+0000","last_became_active":"2023-10-16T11:25:36.168590+0000","last_became_peered":"2023-10-16T11:25:36.168590+0000","last_unstale":"2023-10-16T11:25:38.182764+0000","last_undegraded":"2023-10-16T11:25:38.182764+0000","last_fullsized":"2023-10-16T11:25:38.182764+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:42.801560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255269+0000","last_change":"2023-10-16T11:25:37.175992+0000","last_active":"2023-10-16T11:25:40.255269+0000","last_peered":"2023-10-16T11:25:40.255269+0000","last_clean":"2023-10-16T11:25:40.255269+0000","last_became_active":"2023-10-16T11:25:37.173732+0000","last_became_peered":"2023-10-16T11:25:37.173732+0000","last_unstale":"2023-10-16T11:25:40.255269+0000","last_undegraded":"2023-10-16T11:25:40.255269+0000","last_fullsized":"2023-10-16T11:25:40.255269+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.f","version":"236'1","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.177302+0000","last_change":"2023-10-16T11:25:36.173935+0000","last_active":"2023-10-16T11:25:38.177302+0000","last_peered":"2023-10-16T11:25:38.177302+0000","last_clean":"2023-10-16T11:25:38.177302+0000","last_became_active":"2023-10-16T11:25:36.173510+0000","last_became_peered":"2023-10-16T11:25:36.173510+0000","last_unstale":"2023-10-16T11:25:38.177302+0000","last_undegraded":"2023-10-16T11:25:38.177302+0000","last_fullsized":"2023-10-16T11:25:38.177302+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:19.077096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.250150+0000","last_change":"2023-10-16T11:25:37.177543+0000","last_active":"2023-10-16T11:25:40.250150+0000","last_peered":"2023-10-16T11:25:40.250150+0000","last_clean":"2023-10-16T11:25:40.250150+0000","last_became_active":"2023-10-16T11:25:37.176420+0000","last_became_peered":"2023-10-16T11:25:37.176420+0000","last_unstale":"2023-10-16T11:25:40.250150+0000","last_undegraded":"2023-10-16T11:25:40.250150+0000","last_fullsized":"2023-10-16T11:25:40.250150+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:24.104981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.e","version":"236'2","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616631+0000","last_change":"2023-10-16T11:25:36.170878+0000","last_active":"2023-10-16T11:25:38.616631+0000","last_peered":"2023-10-16T11:25:38.616631+0000","last_clean":"2023-10-16T11:25:38.616631+0000","last_became_active":"2023-10-16T11:25:36.170548+0000","last_became_peered":"2023-10-16T11:25:36.170548+0000","last_unstale":"2023-10-16T11:25:38.616631+0000","last_undegraded":"2023-10-16T11:25:38.616631+0000","last_fullsized":"2023-10-16T11:25:38.616631+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:34:14.932409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.259306+0000","last_change":"2023-10-16T11:25:37.176420+0000","last_active":"2023-10-16T11:25:40.259306+0000","last_peered":"2023-10-16T11:25:40.259306+0000","last_clean":"2023-10-16T11:25:40.259306+0000","last_became_active":"2023-10-16T11:25:37.176101+0000","last_became_peered":"2023-10-16T11:25:37.176101+0000","last_unstale":"2023-10-16T11:25:40.259306+0000","last_undegraded":"2023-10-16T11:25:40.259306+0000","last_fullsized":"2023-10-16T11:25:40.259306+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:10.245184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"236'1","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.175412+0000","last_change":"2023-10-16T11:25:36.171476+0000","last_active":"2023-10-16T11:25:38.175412+0000","last_peered":"2023-10-16T11:25:38.175412+0000","last_clean":"2023-10-16T11:25:38.175412+0000","last_became_active":"2023-10-16T11:25:36.171201+0000","last_became_peered":"2023-10-16T11:25:36.171201+0000","last_unstale":"2023-10-16T11:25:38.175412+0000","last_undegraded":"2023-10-16T11:25:38.175412+0000","last_fullsized":"2023-10-16T11:25:38.175412+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:30.034841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255057+0000","last_change":"2023-10-16T11:25:37.162366+0000","last_active":"2023-10-16T11:25:40.255057+0000","last_peered":"2023-10-16T11:25:40.255057+0000","last_clean":"2023-10-16T11:25:40.255057+0000","last_became_active":"2023-10-16T11:25:37.162032+0000","last_became_peered":"2023-10-16T11:25:37.162032+0000","last_unstale":"2023-10-16T11:25:40.255057+0000","last_undegraded":"2023-10-16T11:25:40.255057+0000","last_fullsized":"2023-10-16T11:25:40.255057+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182815+0000","last_change":"2023-10-16T11:25:36.174246+0000","last_active":"2023-10-16T11:25:38.182815+0000","last_peered":"2023-10-16T11:25:38.182815+0000","last_clean":"2023-10-16T11:25:38.182815+0000","last_became_active":"2023-10-16T11:25:36.173975+0000","last_became_peered":"2023-10-16T11:25:36.173975+0000","last_unstale":"2023-10-16T11:25:38.182815+0000","last_undegraded":"2023-10-16T11:25:38.182815+0000","last_fullsized":"2023-10-16T11:25:38.182815+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:32.130559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254853+0000","last_change":"2023-10-16T11:25:37.168715+0000","last_active":"2023-10-16T11:25:40.254853+0000","last_peered":"2023-10-16T11:25:40.254853+0000","last_clean":"2023-10-16T11:25:40.254853+0000","last_became_active":"2023-10-16T11:25:37.168387+0000","last_became_peered":"2023-10-16T11:25:37.168387+0000","last_unstale":"2023-10-16T11:25:40.254853+0000","last_undegraded":"2023-10-16T11:25:40.254853+0000","last_fullsized":"2023-10-16T11:25:40.254853+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.b","version":"236'2","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616716+0000","last_change":"2023-10-16T11:25:36.174153+0000","last_active":"2023-10-16T11:25:38.616716+0000","last_peered":"2023-10-16T11:25:38.616716+0000","last_clean":"2023-10-16T11:25:38.616716+0000","last_became_active":"2023-10-16T11:25:36.173812+0000","last_became_peered":"2023-10-16T11:25:36.173812+0000","last_unstale":"2023-10-16T11:25:38.616716+0000","last_undegraded":"2023-10-16T11:25:38.616716+0000","last_fullsized":"2023-10-16T11:25:38.616716+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:07:05.931833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248669+0000","last_change":"2023-10-16T11:25:37.175767+0000","last_active":"2023-10-16T11:25:40.248669+0000","last_peered":"2023-10-16T11:25:40.248669+0000","last_clean":"2023-10-16T11:25:40.248669+0000","last_became_active":"2023-10-16T11:25:37.167389+0000","last_became_peered":"2023-10-16T11:25:37.167389+0000","last_unstale":"2023-10-16T11:25:40.248669+0000","last_undegraded":"2023-10-16T11:25:40.248669+0000","last_fullsized":"2023-10-16T11:25:40.248669+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:18.601118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182571+0000","last_change":"2023-10-16T11:25:36.175478+0000","last_active":"2023-10-16T11:25:38.182571+0000","last_peered":"2023-10-16T11:25:38.182571+0000","last_clean":"2023-10-16T11:25:38.182571+0000","last_became_active":"2023-10-16T11:25:36.174935+0000","last_became_peered":"2023-10-16T11:25:36.174935+0000","last_unstale":"2023-10-16T11:25:38.182571+0000","last_undegraded":"2023-10-16T11:25:38.182571+0000","last_fullsized":"2023-10-16T11:25:38.182571+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:14.189489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249282+0000","last_change":"2023-10-16T11:25:37.173034+0000","last_active":"2023-10-16T11:25:40.249282+0000","last_peered":"2023-10-16T11:25:40.249282+0000","last_clean":"2023-10-16T11:25:40.249282+0000","last_became_active":"2023-10-16T11:25:37.172644+0000","last_became_peered":"2023-10-16T11:25:37.172644+0000","last_unstale":"2023-10-16T11:25:40.249282+0000","last_undegraded":"2023-10-16T11:25:40.249282+0000","last_fullsized":"2023-10-16T11:25:40.249282+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174975+0000","last_change":"2023-10-16T11:25:36.163926+0000","last_active":"2023-10-16T11:25:38.174975+0000","last_peered":"2023-10-16T11:25:38.174975+0000","last_clean":"2023-10-16T11:25:38.174975+0000","last_became_active":"2023-10-16T11:25:36.162991+0000","last_became_peered":"2023-10-16T11:25:36.162991+0000","last_unstale":"2023-10-16T11:25:38.174975+0000","last_undegraded":"2023-10-16T11:25:38.174975+0000","last_fullsized":"2023-10-16T11:25:38.174975+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:49.547165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254831+0000","last_change":"2023-10-16T11:25:37.158678+0000","last_active":"2023-10-16T11:25:40.254831+0000","last_peered":"2023-10-16T11:25:40.254831+0000","last_clean":"2023-10-16T11:25:40.254831+0000","last_became_active":"2023-10-16T11:25:37.157389+0000","last_became_peered":"2023-10-16T11:25:37.157389+0000","last_unstale":"2023-10-16T11:25:40.254831+0000","last_undegraded":"2023-10-16T11:25:40.254831+0000","last_fullsized":"2023-10-16T11:25:40.254831+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174650+0000","last_change":"2023-10-16T11:25:36.171876+0000","last_active":"2023-10-16T11:25:38.174650+0000","last_peered":"2023-10-16T11:25:38.174650+0000","last_clean":"2023-10-16T11:25:38.174650+0000","last_became_active":"2023-10-16T11:25:36.169128+0000","last_became_peered":"2023-10-16T11:25:36.169128+0000","last_unstale":"2023-10-16T11:25:38.174650+0000","last_undegraded":"2023-10-16T11:25:38.174650+0000","last_fullsized":"2023-10-16T11:25:38.174650+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:16:18.153133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252751+0000","last_change":"2023-10-16T11:25:37.179119+0000","last_active":"2023-10-16T11:25:40.252751+0000","last_peered":"2023-10-16T11:25:40.252751+0000","last_clean":"2023-10-16T11:25:40.252751+0000","last_became_active":"2023-10-16T11:25:37.175799+0000","last_became_peered":"2023-10-16T11:25:37.175799+0000","last_unstale":"2023-10-16T11:25:40.252751+0000","last_undegraded":"2023-10-16T11:25:40.252751+0000","last_fullsized":"2023-10-16T11:25:40.252751+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"236'2","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.172151+0000","last_change":"2023-10-16T11:25:36.162608+0000","last_active":"2023-10-16T11:25:38.172151+0000","last_peered":"2023-10-16T11:25:38.172151+0000","last_clean":"2023-10-16T11:25:38.172151+0000","last_became_active":"2023-10-16T11:25:36.162336+0000","last_became_peered":"2023-10-16T11:25:36.162336+0000","last_unstale":"2023-10-16T11:25:38.172151+0000","last_undegraded":"2023-10-16T11:25:38.172151+0000","last_fullsized":"2023-10-16T11:25:38.172151+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:19.377028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248788+0000","last_change":"2023-10-16T11:25:37.177560+0000","last_active":"2023-10-16T11:25:40.248788+0000","last_peered":"2023-10-16T11:25:40.248788+0000","last_clean":"2023-10-16T11:25:40.248788+0000","last_became_active":"2023-10-16T11:25:37.176982+0000","last_became_peered":"2023-10-16T11:25:37.176982+0000","last_unstale":"2023-10-16T11:25:40.248788+0000","last_undegraded":"2023-10-16T11:25:40.248788+0000","last_fullsized":"2023-10-16T11:25:40.248788+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:54.497943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"237'1","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:41.776279+0000","last_change":"2023-10-16T11:25:36.174419+0000","last_active":"2023-10-16T11:25:41.776279+0000","last_peered":"2023-10-16T11:25:41.776279+0000","last_clean":"2023-10-16T11:25:41.776279+0000","last_became_active":"2023-10-16T11:25:36.174148+0000","last_became_peered":"2023-10-16T11:25:36.174148+0000","last_unstale":"2023-10-16T11:25:41.776279+0000","last_undegraded":"2023-10-16T11:25:41.776279+0000","last_fullsized":"2023-10-16T11:25:41.776279+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:38.739323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249374+0000","last_change":"2023-10-16T11:25:37.180683+0000","last_active":"2023-10-16T11:25:40.249374+0000","last_peered":"2023-10-16T11:25:40.249374+0000","last_clean":"2023-10-16T11:25:40.249374+0000","last_became_active":"2023-10-16T11:25:37.180393+0000","last_became_peered":"2023-10-16T11:25:37.180393+0000","last_unstale":"2023-10-16T11:25:40.249374+0000","last_undegraded":"2023-10-16T11:25:40.249374+0000","last_fullsized":"2023-10-16T11:25:40.249374+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:38.218704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.175635+0000","last_change":"2023-10-16T11:25:36.164519+0000","last_active":"2023-10-16T11:25:38.175635+0000","last_peered":"2023-10-16T11:25:38.175635+0000","last_clean":"2023-10-16T11:25:38.175635+0000","last_became_active":"2023-10-16T11:25:36.164214+0000","last_became_peered":"2023-10-16T11:25:36.164214+0000","last_unstale":"2023-10-16T11:25:38.175635+0000","last_undegraded":"2023-10-16T11:25:38.175635+0000","last_fullsized":"2023-10-16T11:25:38.175635+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:03.074535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249451+0000","last_change":"2023-10-16T11:25:37.181808+0000","last_active":"2023-10-16T11:25:40.249451+0000","last_peered":"2023-10-16T11:25:40.249451+0000","last_clean":"2023-10-16T11:25:40.249451+0000","last_became_active":"2023-10-16T11:25:37.181575+0000","last_became_peered":"2023-10-16T11:25:37.181575+0000","last_unstale":"2023-10-16T11:25:40.249451+0000","last_undegraded":"2023-10-16T11:25:40.249451+0000","last_fullsized":"2023-10-16T11:25:40.249451+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:17:01.131611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182900+0000","last_change":"2023-10-16T11:25:36.162153+0000","last_active":"2023-10-16T11:25:38.182900+0000","last_peered":"2023-10-16T11:25:38.182900+0000","last_clean":"2023-10-16T11:25:38.182900+0000","last_became_active":"2023-10-16T11:25:36.161842+0000","last_became_peered":"2023-10-16T11:25:36.161842+0000","last_unstale":"2023-10-16T11:25:38.182900+0000","last_undegraded":"2023-10-16T11:25:38.182900+0000","last_fullsized":"2023-10-16T11:25:38.182900+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:55.302377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252517+0000","last_change":"2023-10-16T11:25:37.165420+0000","last_active":"2023-10-16T11:25:40.252517+0000","last_peered":"2023-10-16T11:25:40.252517+0000","last_clean":"2023-10-16T11:25:40.252517+0000","last_became_active":"2023-10-16T11:25:37.164820+0000","last_became_peered":"2023-10-16T11:25:37.164820+0000","last_unstale":"2023-10-16T11:25:40.252517+0000","last_undegraded":"2023-10-16T11:25:40.252517+0000","last_fullsized":"2023-10-16T11:25:40.252517+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:43.813878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.176218+0000","last_change":"2023-10-16T11:25:36.173209+0000","last_active":"2023-10-16T11:25:38.176218+0000","last_peered":"2023-10-16T11:25:38.176218+0000","last_clean":"2023-10-16T11:25:38.176218+0000","last_became_active":"2023-10-16T11:25:36.171799+0000","last_became_peered":"2023-10-16T11:25:36.171799+0000","last_unstale":"2023-10-16T11:25:38.176218+0000","last_undegraded":"2023-10-16T11:25:38.176218+0000","last_fullsized":"2023-10-16T11:25:38.176218+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:19.377028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254897+0000","last_change":"2023-10-16T11:25:37.180243+0000","last_active":"2023-10-16T11:25:40.254897+0000","last_peered":"2023-10-16T11:25:40.254897+0000","last_clean":"2023-10-16T11:25:40.254897+0000","last_became_active":"2023-10-16T11:25:37.179869+0000","last_became_peered":"2023-10-16T11:25:37.179869+0000","last_unstale":"2023-10-16T11:25:40.254897+0000","last_undegraded":"2023-10-16T11:25:40.254897+0000","last_fullsized":"2023-10-16T11:25:40.254897+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:15:31.572558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.171937+0000","last_change":"2023-10-16T11:25:36.169146+0000","last_active":"2023-10-16T11:25:38.171937+0000","last_peered":"2023-10-16T11:25:38.171937+0000","last_clean":"2023-10-16T11:25:38.171937+0000","last_became_active":"2023-10-16T11:25:36.168877+0000","last_became_peered":"2023-10-16T11:25:36.168877+0000","last_unstale":"2023-10-16T11:25:38.171937+0000","last_undegraded":"2023-10-16T11:25:38.171937+0000","last_fullsized":"2023-10-16T11:25:38.171937+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:35.988148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254732+0000","last_change":"2023-10-16T11:25:37.173516+0000","last_active":"2023-10-16T11:25:40.254732+0000","last_peered":"2023-10-16T11:25:40.254732+0000","last_clean":"2023-10-16T11:25:40.254732+0000","last_became_active":"2023-10-16T11:25:37.172964+0000","last_became_peered":"2023-10-16T11:25:37.172964+0000","last_unstale":"2023-10-16T11:25:40.254732+0000","last_undegraded":"2023-10-16T11:25:40.254732+0000","last_fullsized":"2023-10-16T11:25:40.254732+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616128+0000","last_change":"2023-10-16T11:25:36.166442+0000","last_active":"2023-10-16T11:25:38.616128+0000","last_peered":"2023-10-16T11:25:38.616128+0000","last_clean":"2023-10-16T11:25:38.616128+0000","last_became_active":"2023-10-16T11:25:36.166131+0000","last_became_peered":"2023-10-16T11:25:36.166131+0000","last_unstale":"2023-10-16T11:25:38.616128+0000","last_undegraded":"2023-10-16T11:25:38.616128+0000","last_fullsized":"2023-10-16T11:25:38.616128+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:43.814368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248629+0000","last_change":"2023-10-16T11:25:37.183491+0000","last_active":"2023-10-16T11:25:40.248629+0000","last_peered":"2023-10-16T11:25:40.248629+0000","last_clean":"2023-10-16T11:25:40.248629+0000","last_became_active":"2023-10-16T11:25:37.179560+0000","last_became_peered":"2023-10-16T11:25:37.179560+0000","last_unstale":"2023-10-16T11:25:40.248629+0000","last_undegraded":"2023-10-16T11:25:40.248629+0000","last_fullsized":"2023-10-16T11:25:40.248629+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:31.820109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616369+0000","last_change":"2023-10-16T11:25:36.175845+0000","last_active":"2023-10-16T11:25:38.616369+0000","last_peered":"2023-10-16T11:25:38.616369+0000","last_clean":"2023-10-16T11:25:38.616369+0000","last_became_active":"2023-10-16T11:25:36.175390+0000","last_became_peered":"2023-10-16T11:25:36.175390+0000","last_unstale":"2023-10-16T11:25:38.616369+0000","last_undegraded":"2023-10-16T11:25:38.616369+0000","last_fullsized":"2023-10-16T11:25:38.616369+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:09.232865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.257136+0000","last_change":"2023-10-16T11:25:37.178486+0000","last_active":"2023-10-16T11:25:40.257136+0000","last_peered":"2023-10-16T11:25:40.257136+0000","last_clean":"2023-10-16T11:25:40.257136+0000","last_became_active":"2023-10-16T11:25:37.177077+0000","last_became_peered":"2023-10-16T11:25:37.177077+0000","last_unstale":"2023-10-16T11:25:40.257136+0000","last_undegraded":"2023-10-16T11:25:40.257136+0000","last_fullsized":"2023-10-16T11:25:40.257136+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"236'4","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.617092+0000","last_change":"2023-10-16T11:25:36.174205+0000","last_active":"2023-10-16T11:25:38.617092+0000","last_peered":"2023-10-16T11:25:38.617092+0000","last_clean":"2023-10-16T11:25:38.617092+0000","last_became_active":"2023-10-16T11:25:36.173850+0000","last_became_peered":"2023-10-16T11:25:36.173850+0000","last_unstale":"2023-10-16T11:25:38.617092+0000","last_undegraded":"2023-10-16T11:25:38.617092+0000","last_fullsized":"2023-10-16T11:25:38.617092+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:20.618602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"223'2789","reported_seq":3746,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.173478+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:25:38.173478+0000","last_peered":"2023-10-16T11:25:38.173478+0000","last_clean":"2023-10-16T11:25:38.173478+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:25:38.173478+0000","last_undegraded":"2023-10-16T11:25:38.173478+0000","last_fullsized":"2023-10-16T11:25:38.173478+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2789,"log_dups_size":0,"ondisk_log_size":2789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":31,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":30,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2789,"ondisk_log_size":2789,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576001,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52740,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319100,"statfs":{"total":96636764160,"available":96582758400,"internally_reserved":0,"allocated":3596288,"data_stored":2052564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.153,"15min":1.2090000000000001},"min":{"1min":0.71499999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4950000000000001,"5min":1.7549999999999999,"15min":2.3399999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.161,"15min":1.2150000000000001},"min":{"1min":0.623,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.448,"5min":1.6699999999999999,"15min":1.8320000000000001},"last":1.113}]},{"osd":1,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.256,"15min":1.2430000000000001},"min":{"1min":0.84599999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.5029999999999999,"5min":1.913,"15min":1.913},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.171,"5min":1.2210000000000001,"15min":1.264},"min":{"1min":0.67100000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.522,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.0580000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.3,"15min":1.296},"min":{"1min":0.94199999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6879999999999999,"5min":1.8680000000000001,"15min":2.4460000000000002},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.21,"5min":1.3129999999999999,"15min":1.3500000000000001},"min":{"1min":0.94499999999999995,"5min":0.88,"15min":0.88},"max":{"1min":1.5589999999999999,"5min":1.859,"15min":1.974},"last":0.86799999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3520000000000001,"15min":1.3779999999999999},"min":{"1min":0.93700000000000006,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.653,"5min":1.8080000000000001,"15min":1.9099999999999999},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.4370000000000001,"15min":1.454},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.728,"5min":2.0070000000000001,"15min":2.577},"last":1.3460000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4319999999999999,"15min":1.413},"min":{"1min":0.89800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.865,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.171},{"interface":"front","average":{"1min":1.369,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":1.149,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8660000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.417}]},{"osd":5,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.587,"15min":1.591},"min":{"1min":1.1799999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.95,"5min":2.101,"15min":3.3519999999999999},"last":1.488},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.53,"15min":1.508},"min":{"1min":1.2330000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.9870000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.595}]},{"osd":6,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.492,"15min":1.5089999999999999},"min":{"1min":1.1459999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.8140000000000001,"5min":2.101,"15min":2.7250000000000001},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.6040000000000001,"5min":1.6819999999999999,"15min":1.698},"min":{"1min":1.3080000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.0129999999999999,"5min":2.2109999999999999,"15min":3.1440000000000001},"last":1.5660000000000001}]}]},{"osd":6,"up_from":13,"seq":55834576001,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52164,"kb_used_data":2936,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319676,"statfs":{"total":96636764160,"available":96583348224,"internally_reserved":0,"allocated":3006464,"data_stored":1462730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.2130000000000001,"15min":1.21},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.343,"5min":6.298,"15min":6.298},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.204,"5min":1.331,"15min":1.292},"min":{"1min":0.72199999999999998,"5min":0.68200000000000005,"15min":0.66400000000000003},"max":{"1min":1.607,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":1.377}]},{"osd":1,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.224,"15min":1.212},"min":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.5660000000000001,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.421,"15min":1.3779999999999999},"min":{"1min":0.76200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.7310000000000001,"5min":12.468,"15min":12.468},"last":1.1990000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.3140000000000001,"15min":1.2769999999999999},"min":{"1min":0.77800000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.373,"5min":11.586,"15min":11.586},"last":1.248},{"interface":"front","average":{"1min":1.306,"5min":1.3740000000000001,"15min":1.3680000000000001},"min":{"1min":0.84399999999999997,"5min":0.82599999999999996,"15min":0.77500000000000002},"max":{"1min":1.7629999999999999,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.427}]},{"osd":3,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.425,"15min":1.3460000000000001},"min":{"1min":0.84799999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.859,"5min":12.801,"15min":12.801},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2929999999999999,"15min":1.2769999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.657,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":1.1379999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3899999999999999,"15min":1.3280000000000001},"min":{"1min":0.68700000000000006,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.9510000000000001,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.6220000000000001,"15min":1.5509999999999999},"min":{"1min":1.1180000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.069,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.4630000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.5189999999999999,"15min":1.397},"min":{"1min":0.92500000000000004,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.978,"5min":18.247,"15min":18.247},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.595,"15min":1.5620000000000001},"min":{"1min":0.93799999999999994,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.1240000000000001,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.5049999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.627,"15min":1.5469999999999999},"min":{"1min":1.2070000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.383,"5min":12.589,"15min":12.589},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.661,"5min":1.702,"15min":1.603},"min":{"1min":1.0049999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.2970000000000002,"5min":12.725,"15min":12.725},"last":1.3360000000000001}]}]},{"osd":5,"up_from":13,"seq":55834576001,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29948,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341892,"statfs":{"total":96636764160,"available":96606097408,"internally_reserved":0,"allocated":2998272,"data_stored":1462126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2150000000000001,"15min":1.2629999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.51,"5min":2.661,"15min":2.661},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.204,"5min":1.2769999999999999,"15min":1.339},"min":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7210000000000001,"5min":1.7949999999999999,"15min":1.865},"last":1.113}]},{"osd":1,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.145,"15min":1.1890000000000001},"min":{"1min":0.872,"5min":0.81299999999999994,"15min":0.79100000000000004},"max":{"1min":1.583,"5min":1.704,"15min":1.992},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.079,"5min":1.22,"15min":1.222},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6739999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.427}]},{"osd":2,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.26,"15min":1.236},"min":{"1min":0.73499999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.7450000000000001,"5min":1.952,"15min":1.952},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.236,"5min":1.284,"15min":1.3129999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5840000000000001,"5min":1.8109999999999999,"15min":2.1640000000000001},"last":1.1699999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.319,"15min":1.329},"min":{"1min":0.872,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5920000000000001,"5min":3.012,"15min":3.012},"last":1.21},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.401,"15min":1.4139999999999999},"min":{"1min":0.95399999999999996,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.6639999999999999,"5min":1.9119999999999999,"15min":2.2250000000000001},"last":1.034}]},{"osd":4,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3600000000000001,"15min":1.341},"min":{"1min":0.91300000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.887,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.575},{"interface":"front","average":{"1min":1.413,"5min":1.5529999999999999,"15min":1.5660000000000001},"min":{"1min":0.996,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.6499999999999999,"5min":2.903,"15min":2.903},"last":1.3879999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.4119999999999999,"15min":1.411},"min":{"1min":0.93799999999999994,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.837,"5min":1.8660000000000001,"15min":2.0979999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.486,"15min":1.4810000000000001},"min":{"1min":0.93300000000000005,"5min":0.90700000000000003,"15min":0.89200000000000002},"max":{"1min":2.0329999999999999,"5min":3.073,"15min":3.073},"last":1.673}]},{"osd":7,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.5649999999999999,"15min":1.5429999999999999},"min":{"1min":1.159,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.764,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.3},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.625,"15min":1.6399999999999999},"min":{"1min":1.349,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.9970000000000001,"5min":2.1099999999999999,"15min":2.2930000000000001},"last":1.341}]}]},{"osd":4,"up_from":13,"seq":55834576002,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70160,"kb_used_data":2948,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301680,"statfs":{"total":96636764160,"available":96564920320,"internally_reserved":0,"allocated":3018752,"data_stored":1465075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.167,"15min":1.1799999999999999},"min":{"1min":0.69099999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.2370000000000001,"15min":1.2430000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":1.083}]},{"osd":1,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3069999999999999,"15min":1.3009999999999999},"min":{"1min":0.88800000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.252,"15min":1.252},"min":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":3.0739999999999998,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.5960000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3,"15min":1.3109999999999999},"min":{"1min":0.80500000000000005,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":3.2170000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.4359999999999999,"15min":1.4119999999999999},"min":{"1min":0.90300000000000002,"5min":0.879,"15min":0.872},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.423}]},{"osd":3,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.339,"15min":1.341},"min":{"1min":0.84599999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.516},{"interface":"front","average":{"1min":1.496,"5min":1.4550000000000001,"15min":1.4430000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":3.4220000000000002,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.468}]},{"osd":5,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.5249999999999999,"15min":1.4570000000000001},"min":{"1min":1.1339999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.613,"5min":1.46,"15min":1.4319999999999999},"min":{"1min":1.1519999999999999,"5min":0.85199999999999998,"15min":0.78600000000000003},"max":{"1min":3.7989999999999999,"5min":3.7989999999999999,"15min":3.7989999999999999},"last":1.292}]},{"osd":6,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6679999999999999,"5min":1.5649999999999999,"15min":1.5189999999999999},"min":{"1min":0.98599999999999999,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.776,"5min":1.643,"15min":1.6240000000000001},"min":{"1min":1.0269999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":3.5019999999999998,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.8}]},{"osd":7,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.8089999999999999,"5min":1.613,"15min":1.5860000000000001},"min":{"1min":1.3,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":4.2050000000000001,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":1.7569999999999999},{"interface":"front","average":{"1min":1.8460000000000001,"5min":1.702,"15min":1.6599999999999999},"min":{"1min":1.2370000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":3.7000000000000002,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.6850000000000001}]}]},{"osd":3,"up_from":13,"seq":55834576001,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52160,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319680,"statfs":{"total":96636764160,"available":96583352320,"internally_reserved":0,"allocated":3002368,"data_stored":1462708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.85899999999999999,"15min":0.89700000000000002},"min":{"1min":0.40000000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1339999999999999,"5min":1.478,"15min":1.901},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.997,"15min":1.0569999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1850000000000001,"5min":1.6180000000000001,"15min":1.6679999999999999},"last":0.85399999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.95299999999999996,"15min":0.98399999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.3400000000000001,"5min":1.508,"15min":1.508},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0609999999999999,"15min":1.121},"min":{"1min":0.64900000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5429999999999999,"5min":1.601,"15min":1.7190000000000001},"last":0.91100000000000003}]},{"osd":2,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0580000000000001,"15min":1.1120000000000001},"min":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4199999999999999,"5min":1.694,"15min":1.694},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.161,"15min":1.22},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.7350000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.222}]},{"osd":4,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2989999999999999,"15min":1.3480000000000001},"min":{"1min":0.89400000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.724,"5min":1.806,"15min":1.873},"last":0.995},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3300000000000001,"15min":1.359},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.349}]},{"osd":5,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.389,"15min":1.405},"min":{"1min":1.0549999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8260000000000001,"5min":2.798,"15min":2.798},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.395,"15min":1.4410000000000001},"min":{"1min":0.88800000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.883,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.2889999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.488,"15min":1.5329999999999999},"min":{"1min":1.01,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.911,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4790000000000001,"15min":1.534},"min":{"1min":0.86699999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.0329999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.629}]},{"osd":7,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.518,"15min":1.5580000000000001},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.575,"5min":1.613,"15min":1.669},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.3799999999999999,"5min":2.3799999999999999,"15min":2.5129999999999999},"last":1.7230000000000001}]}]},{"osd":2,"up_from":13,"seq":55834576000,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56320,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315520,"statfs":{"total":96636764160,"available":96579092480,"internally_reserved":0,"allocated":3002368,"data_stored":1462216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.89200000000000002,"15min":0.91600000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.349,"5min":1.532,"15min":1.669},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.99299999999999999,"15min":1.069},"min":{"1min":0.56899999999999995,"5min":0.47699999999999998,"15min":0.32600000000000001},"max":{"1min":1.3460000000000001,"5min":1.6359999999999999,"15min":2.1429999999999998},"last":0.95799999999999996}]},{"osd":1,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0760000000000001,"15min":1.089},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.39200000000000002},"max":{"1min":1.591,"5min":1.645,"15min":1.978},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.085,"15min":1.1379999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.393,"5min":1.609,"15min":1.7949999999999999},"last":1.0680000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0429999999999999,"15min":1.0740000000000001},"min":{"1min":0.499,"5min":0.437,"15min":0.437},"max":{"1min":1.522,"5min":1.651,"15min":1.8560000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.137,"5min":1.2170000000000001,"15min":1.2629999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.6930000000000001,"5min":1.7450000000000001,"15min":1.857},"last":1.379}]},{"osd":4,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3340000000000001,"15min":1.3999999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.73499999999999999},"max":{"1min":1.643,"5min":1.8029999999999999,"15min":2.5059999999999998},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.3220000000000001,"15min":1.339},"min":{"1min":0.88200000000000001,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":1.6499999999999999,"5min":1.825,"15min":1.843},"last":1.2230000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.4139999999999999,"15min":1.4530000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.429,"15min":1.4730000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.885,"5min":1.931,"15min":2.6259999999999999},"last":1.5449999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3899999999999999,"15min":1.3999999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.8260000000000001,"5min":1.9019999999999999,"15min":2.1339999999999999},"last":1.095},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4870000000000001,"15min":1.526},"min":{"1min":0.80500000000000005,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.494}]},{"osd":7,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.5109999999999999,"15min":1.569},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.92500000000000004},"max":{"1min":2.0310000000000001,"5min":2.1469999999999998,"15min":2.8109999999999999},"last":1.615},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4750000000000001,"15min":1.502},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80100000000000005},"max":{"1min":1.952,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.3420000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576000,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64656,"kb_used_data":2948,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307184,"statfs":{"total":96636764160,"available":96570556416,"internally_reserved":0,"allocated":3018752,"data_stored":1465107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.93799999999999994,"15min":0.91700000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":5.117,"5min":5.117,"15min":5.117},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.97299999999999998,"15min":1.0449999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.5069999999999999,"5min":1.5509999999999999,"15min":1.667},"last":0.35899999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.97299999999999998,"15min":1.014},"min":{"1min":0.52600000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2350000000000001,"15min":1.27},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":3.5840000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.90300000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.1779999999999999,"15min":1.202},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.4670000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.1439999999999999,"15min":1.1639999999999999},"min":{"1min":0.48099999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5600000000000001,"5min":1.633,"15min":1.802},"last":0.66400000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.3260000000000001,"15min":1.3009999999999999},"min":{"1min":0.80500000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":5.29,"5min":5.29,"15min":5.29},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.4450000000000001,"15min":1.4259999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":5.0179999999999998,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":0.999}]},{"osd":5,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.397,"15min":1.419},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":6.0979999999999999,"5min":6.0979999999999999,"15min":6.0979999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.415,"15min":1.4159999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.5299999999999998,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.2330000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.49,"15min":1.5029999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":4.0499999999999998,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":1.034},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.474,"15min":1.488},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":3.8799999999999999,"5min":3.8799999999999999,"15min":3.8799999999999999},"last":1.2689999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.577,"15min":1.569},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":4.46,"5min":4.46,"15min":4.46},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.625,"15min":1.613},"min":{"1min":1.0049999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":4.1040000000000001,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":1.135}]}]},{"osd":0,"up_from":13,"seq":55834576001,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52740,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319100,"statfs":{"total":96636764160,"available":96582758400,"internally_reserved":0,"allocated":3596288,"data_stored":2052528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.86099999999999999,"15min":0.88700000000000001},"min":{"1min":0.46800000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2210000000000001,"5min":1.52,"15min":1.619},"last":0.874},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.006,"15min":1.081},"min":{"1min":0.41199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.341,"5min":1.621,"15min":1.76},"last":1.0609999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.075,"15min":1.0800000000000001},"min":{"1min":0.52500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.7349999999999999,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.034,"5min":1.0700000000000001,"15min":1.1299999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":1.583}]},{"osd":3,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.016,"15min":1.0669999999999999},"min":{"1min":0.50600000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1299999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.109},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.1539999999999999,"15min":1.226},"min":{"1min":0.55600000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.3799999999999999,"5min":1.833,"15min":1.9830000000000001},"last":1.2390000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.349,"15min":1.3859999999999999},"min":{"1min":0.78800000000000003,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":3.2799999999999998,"5min":3.2799999999999998,"15min":3.2799999999999998},"last":1.7829999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.292,"15min":1.337},"min":{"1min":0.76100000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.2190000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.357,"15min":1.3720000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0209999999999999,"5min":2.0750000000000002,"15min":2.6669999999999998},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3819999999999999,"15min":1.431},"min":{"1min":0.89800000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.885,"5min":2.2029999999999998,"15min":2.2040000000000002},"last":1.4370000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.4710000000000001,"15min":1.496},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":5.2560000000000002,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":1.53},{"interface":"front","average":{"1min":1.298,"5min":1.429,"15min":1.4790000000000001},"min":{"1min":0.83199999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.804,"5min":2.073,"15min":2.262},"last":1.385}]},{"osd":7,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.704,"5min":1.5429999999999999,"15min":1.5600000000000001},"min":{"1min":0.88700000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.4419999999999999,"15min":1.492},"min":{"1min":0.871,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.623,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.2809999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:25:56.333 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:25:56.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.579+0000 7fd175444640 1 Processor -- start 2023-10-16T11:25:56.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.579+0000 7fd175444640 1 -- start start 2023-10-16T11:25:56.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.580+0000 7fd175444640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd1700b0b20 0x7fd170164b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:56.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.580+0000 7fd175444640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17006f7f0 con 0x7fd170159b20 2023-10-16T11:25:56.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.580+0000 7fd175444640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17006c5e0 con 0x7fd170159f00 2023-10-16T11:25:56.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.580+0000 7fd175444640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17006c8c0 con 0x7fd1700b0b20 2023-10-16T11:25:56.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.581+0000 7fd16effd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd170159b20 0x7fd17006bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40528/0 (socket says 172.21.15.84:40528) 2023-10-16T11:25:56.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.581+0000 7fd16effd640 1 -- 172.21.15.84:0/3574871029 learned_addr learned my addr 172.21.15.84:0/3574871029 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:56.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.581+0000 7fd16f7fe640 1 --2- 172.21.15.84:0/3574871029 >> v2:172.21.15.84:3300/0 conn(0x7fd1700b0b20 0x7fd170164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:56.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.581+0000 7fd16dffb640 1 -- 172.21.15.84:0/3574871029 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640975377 0 0) 0x7fd17006f7f0 con 0x7fd170159b20 2023-10-16T11:25:56.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.581+0000 7fd16dffb640 1 -- 172.21.15.84:0/3574871029 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd158003520 con 0x7fd170159b20 2023-10-16T11:25:56.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.581+0000 7fd16f7fe640 1 -- 172.21.15.84:0/3574871029 >> v1:172.21.15.157:6789/0 conn(0x7fd170159f00 legacy=0x7fd1700b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:56.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.581+0000 7fd16f7fe640 1 -- 172.21.15.84:0/3574871029 >> v1:172.21.15.84:6789/0 conn(0x7fd170159b20 legacy=0x7fd17006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:56.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.581+0000 7fd16f7fe640 1 -- 172.21.15.84:0/3574871029 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17006cba0 con 0x7fd1700b0b20 2023-10-16T11:25:56.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.582+0000 7fd16f7fe640 1 --2- 172.21.15.84:0/3574871029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1700b0b20 0x7fd170164b40 secure :-1 s=READY pgs=2997 cs=0 l=1 rev1=1 crypto rx=0x7fd168009d90 tx=0x7fd16802bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf0fa5b37c800f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:56.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.583+0000 7fd16dffb640 1 -- 172.21.15.84:0/3574871029 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16802ccf0 con 0x7fd1700b0b20 2023-10-16T11:25:56.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.583+0000 7fd16dffb640 1 -- 172.21.15.84:0/3574871029 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16802ce50 con 0x7fd1700b0b20 2023-10-16T11:25:56.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.583+0000 7fd16dffb640 1 -- 172.21.15.84:0/3574871029 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168034660 con 0x7fd1700b0b20 2023-10-16T11:25:56.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.583+0000 7fd175444640 1 -- 172.21.15.84:0/3574871029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1700b0b20 msgr2=0x7fd170164b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:56.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.583+0000 7fd175444640 1 --2- 172.21.15.84:0/3574871029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1700b0b20 0x7fd170164b40 secure :-1 s=READY pgs=2997 cs=0 l=1 rev1=1 crypto rx=0x7fd168009d90 tx=0x7fd16802bf40 comp rx=0 tx=0).stop 2023-10-16T11:25:56.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.583+0000 7fd175444640 1 -- 172.21.15.84:0/3574871029 shutdown_connections 2023-10-16T11:25:56.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.583+0000 7fd175444640 1 --2- 172.21.15.84:0/3574871029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1700b0b20 0x7fd170164b40 unknown :-1 s=CLOSED pgs=2997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:56.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.583+0000 7fd175444640 1 -- 172.21.15.84:0/3574871029 >> 172.21.15.84:0/3574871029 conn(0x7fd1700cb410 msgr2=0x7fd1700cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:56.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.584+0000 7fd175444640 1 -- 172.21.15.84:0/3574871029 shutdown_connections 2023-10-16T11:25:56.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.584+0000 7fd175444640 1 -- 172.21.15.84:0/3574871029 wait complete. 2023-10-16T11:25:56.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.585+0000 7fd175444640 1 Processor -- start 2023-10-16T11:25:56.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.585+0000 7fd175444640 1 -- start start 2023-10-16T11:25:56.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.586+0000 7fd175444640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1700b0b20 0x7fd17014c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:56.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.586+0000 7fd175444640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd170159b20 0x7fd17015bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:56.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.586+0000 7fd175444640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd170159f00 0x7fd17015c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:56.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.586+0000 7fd175444640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17015a660 con 0x7fd170159f00 2023-10-16T11:25:56.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.586+0000 7fd175444640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700c5b70 con 0x7fd1700b0b20 2023-10-16T11:25:56.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.586+0000 7fd175444640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700b1370 con 0x7fd170159b20 2023-10-16T11:25:56.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.587+0000 7fd16f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd170159f00 0x7fd17015c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:56.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.587+0000 7fd16f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd170159f00 0x7fd17015c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40514/0 (socket says 172.21.15.84:40514) 2023-10-16T11:25:56.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.587+0000 7fd16f7fe640 1 -- 172.21.15.84:0/2923963123 learned_addr learned my addr 172.21.15.84:0/2923963123 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:56.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.587+0000 7fd16f7fe640 1 -- 172.21.15.84:0/2923963123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd170159b20 msgr2=0x7fd17015bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:56.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.587+0000 7fd16f7fe640 1 --2- 172.21.15.84:0/2923963123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd170159b20 0x7fd17015bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:56.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.587+0000 7fd16f7fe640 1 -- 172.21.15.84:0/2923963123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1700b0b20 msgr2=0x7fd17014c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:56.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.587+0000 7fd16f7fe640 1 --2- 172.21.15.84:0/2923963123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1700b0b20 0x7fd17014c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:56.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.587+0000 7fd16f7fe640 1 -- 172.21.15.84:0/2923963123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168009a70 con 0x7fd170159f00 2023-10-16T11:25:56.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.587+0000 7fd16f7fe640 1 --2- 172.21.15.84:0/2923963123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd170159f00 0x7fd17015c0a0 secure :-1 s=READY pgs=2998 cs=0 l=1 rev1=1 crypto rx=0x7fd168037040 tx=0x7fd168004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:56.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.588+0000 7fd1577fe640 1 -- 172.21.15.84:0/2923963123 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16803d600 con 0x7fd170159f00 2023-10-16T11:25:56.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.588+0000 7fd1577fe640 1 -- 172.21.15.84:0/2923963123 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16803d760 con 0x7fd170159f00 2023-10-16T11:25:56.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.588+0000 7fd1577fe640 1 -- 172.21.15.84:0/2923963123 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16803db10 con 0x7fd170159f00 2023-10-16T11:25:56.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.588+0000 7fd175444640 1 -- 172.21.15.84:0/2923963123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17015c720 con 0x7fd170159f00 2023-10-16T11:25:56.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.588+0000 7fd175444640 1 -- 172.21.15.84:0/2923963123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17015cce0 con 0x7fd170159f00 2023-10-16T11:25:56.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.589+0000 7fd175444640 1 -- 172.21.15.84:0/2923963123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1700b1570 con 0x7fd170159f00 2023-10-16T11:25:56.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.590+0000 7fd1577fe640 1 -- 172.21.15.84:0/2923963123 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd16803b040 con 0x7fd170159f00 2023-10-16T11:25:56.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.591+0000 7fd1577fe640 1 --2- 172.21.15.84:0/2923963123 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd15c06f4f0 0x7fd15c0719b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:56.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.591+0000 7fd1577fe640 1 -- 172.21.15.84:0/2923963123 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7fd1680ac350 con 0x7fd170159f00 2023-10-16T11:25:56.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.591+0000 7fd16effd640 1 --2- 172.21.15.84:0/2923963123 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd15c06f4f0 0x7fd15c0719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:56.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.592+0000 7fd16effd640 1 --2- 172.21.15.84:0/2923963123 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd15c06f4f0 0x7fd15c0719b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd1700c8b10 tx=0x7fd160007930 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:56.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.595+0000 7fd1577fe640 1 -- 172.21.15.84:0/2923963123 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd1700b1570 con 0x7fd170159f00 2023-10-16T11:25:56.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.782+0000 7fd175444640 1 -- 172.21.15.84:0/2923963123 --> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd17006c2f0 con 0x7fd15c06f4f0 2023-10-16T11:25:56.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.819+0000 7fd1577fe640 1 -- 172.21.15.84:0/2923963123 <== mgr.11814 v2:172.21.15.84:6832/2963529201 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+514284 (secure 0 0 0) 0x7fd1600058b0 con 0x7fd15c06f4f0 2023-10-16T11:25:56.829 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:56.832 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:25:56.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.827+0000 7fd175444640 1 -- 172.21.15.84:0/2923963123 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd15c06f4f0 msgr2=0x7fd15c0719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:56.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.827+0000 7fd175444640 1 --2- 172.21.15.84:0/2923963123 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd15c06f4f0 0x7fd15c0719b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd1700c8b10 tx=0x7fd160007930 comp rx=0 tx=0).stop 2023-10-16T11:25:56.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.827+0000 7fd175444640 1 -- 172.21.15.84:0/2923963123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd170159f00 msgr2=0x7fd17015c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:56.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.827+0000 7fd175444640 1 --2- 172.21.15.84:0/2923963123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd170159f00 0x7fd17015c0a0 secure :-1 s=READY pgs=2998 cs=0 l=1 rev1=1 crypto rx=0x7fd168037040 tx=0x7fd168004a40 comp rx=0 tx=0).stop 2023-10-16T11:25:56.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.827+0000 7fd175444640 1 -- 172.21.15.84:0/2923963123 shutdown_connections 2023-10-16T11:25:56.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.827+0000 7fd175444640 1 --2- 172.21.15.84:0/2923963123 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fd15c06f4f0 0x7fd15c0719b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:56.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.827+0000 7fd175444640 1 --2- 172.21.15.84:0/2923963123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd170159f00 0x7fd17015c0a0 unknown :-1 s=CLOSED pgs=2998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:56.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.827+0000 7fd175444640 1 --2- 172.21.15.84:0/2923963123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd170159b20 0x7fd17015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:56.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.827+0000 7fd175444640 1 --2- 172.21.15.84:0/2923963123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1700b0b20 0x7fd17014c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:56.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.827+0000 7fd175444640 1 -- 172.21.15.84:0/2923963123 >> 172.21.15.84:0/2923963123 conn(0x7fd1700cb410 msgr2=0x7fd17006d370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:56.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.827+0000 7fd175444640 1 -- 172.21.15.84:0/2923963123 shutdown_connections 2023-10-16T11:25:56.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:56.827+0000 7fd175444640 1 -- 172.21.15.84:0/2923963123 wait complete. 2023-10-16T11:25:56.849 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2023-10-16T11:25:55.248077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594023,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":668,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2830,"ondisk_log_size":2830,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":204,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":430888,"kb_used_data":24648,"kb_used_omap":12,"kb_used_meta":406131,"kb_avail":754543832,"statfs":{"total":773094113280,"available":772652883968,"internally_reserved":0,"allocated":25239552,"data_stored":12885054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":415878713},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":987,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.984522"},"pg_stats":[{"pgid":"33.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.273967+0000","last_change":"2023-10-16T11:25:49.273967+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.273967+0000","last_undegraded":"2023-10-16T11:25:49.273967+0000","last_fullsized":"2023-10-16T11:25:49.273967+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3f","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248777+0000","last_change":"2023-10-16T11:25:37.173492+0000","last_active":"2023-10-16T11:25:40.248777+0000","last_peered":"2023-10-16T11:25:40.248777+0000","last_clean":"2023-10-16T11:25:40.248777+0000","last_became_active":"2023-10-16T11:25:37.167017+0000","last_became_peered":"2023-10-16T11:25:37.167017+0000","last_unstale":"2023-10-16T11:25:40.248777+0000","last_undegraded":"2023-10-16T11:25:40.248777+0000","last_fullsized":"2023-10-16T11:25:40.248777+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:07.791499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.270260+0000","last_change":"2023-10-16T11:25:49.270260+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.270260+0000","last_undegraded":"2023-10-16T11:25:49.270260+0000","last_fullsized":"2023-10-16T11:25:49.270260+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.3e","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248717+0000","last_change":"2023-10-16T11:25:37.179117+0000","last_active":"2023-10-16T11:25:40.248717+0000","last_peered":"2023-10-16T11:25:40.248717+0000","last_clean":"2023-10-16T11:25:40.248717+0000","last_became_active":"2023-10-16T11:25:37.178849+0000","last_became_peered":"2023-10-16T11:25:37.178849+0000","last_unstale":"2023-10-16T11:25:40.248717+0000","last_undegraded":"2023-10-16T11:25:40.248717+0000","last_fullsized":"2023-10-16T11:25:40.248717+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:18:57.278249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.273067+0000","last_change":"2023-10-16T11:25:49.273067+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.273067+0000","last_undegraded":"2023-10-16T11:25:49.273067+0000","last_fullsized":"2023-10-16T11:25:49.273067+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3d","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.250094+0000","last_change":"2023-10-16T11:25:37.179065+0000","last_active":"2023-10-16T11:25:40.250094+0000","last_peered":"2023-10-16T11:25:40.250094+0000","last_clean":"2023-10-16T11:25:40.250094+0000","last_became_active":"2023-10-16T11:25:37.178779+0000","last_became_peered":"2023-10-16T11:25:37.178779+0000","last_unstale":"2023-10-16T11:25:40.250094+0000","last_undegraded":"2023-10-16T11:25:40.250094+0000","last_fullsized":"2023-10-16T11:25:40.250094+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:09:47.367715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.271638+0000","last_change":"2023-10-16T11:25:49.271638+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.271638+0000","last_undegraded":"2023-10-16T11:25:49.271638+0000","last_fullsized":"2023-10-16T11:25:49.271638+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3c","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255257+0000","last_change":"2023-10-16T11:25:37.172480+0000","last_active":"2023-10-16T11:25:40.255257+0000","last_peered":"2023-10-16T11:25:40.255257+0000","last_clean":"2023-10-16T11:25:40.255257+0000","last_became_active":"2023-10-16T11:25:37.172192+0000","last_became_peered":"2023-10-16T11:25:37.172192+0000","last_unstale":"2023-10-16T11:25:40.255257+0000","last_undegraded":"2023-10-16T11:25:40.255257+0000","last_fullsized":"2023-10-16T11:25:40.255257+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.272120+0000","last_change":"2023-10-16T11:25:49.272120+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.272120+0000","last_undegraded":"2023-10-16T11:25:49.272120+0000","last_fullsized":"2023-10-16T11:25:49.272120+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3b","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.257086+0000","last_change":"2023-10-16T11:25:37.177505+0000","last_active":"2023-10-16T11:25:40.257086+0000","last_peered":"2023-10-16T11:25:40.257086+0000","last_clean":"2023-10-16T11:25:40.257086+0000","last_became_active":"2023-10-16T11:25:37.174334+0000","last_became_peered":"2023-10-16T11:25:37.174334+0000","last_unstale":"2023-10-16T11:25:40.257086+0000","last_undegraded":"2023-10-16T11:25:40.257086+0000","last_fullsized":"2023-10-16T11:25:40.257086+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.267494+0000","last_change":"2023-10-16T11:25:49.267494+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.267494+0000","last_undegraded":"2023-10-16T11:25:49.267494+0000","last_fullsized":"2023-10-16T11:25:49.267494+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3a","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255230+0000","last_change":"2023-10-16T11:25:37.177781+0000","last_active":"2023-10-16T11:25:40.255230+0000","last_peered":"2023-10-16T11:25:40.255230+0000","last_clean":"2023-10-16T11:25:40.255230+0000","last_became_active":"2023-10-16T11:25:37.177501+0000","last_became_peered":"2023-10-16T11:25:37.177501+0000","last_unstale":"2023-10-16T11:25:40.255230+0000","last_undegraded":"2023-10-16T11:25:40.255230+0000","last_fullsized":"2023-10-16T11:25:40.255230+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.279696+0000","last_change":"2023-10-16T11:25:49.279696+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.279696+0000","last_undegraded":"2023-10-16T11:25:49.279696+0000","last_fullsized":"2023-10-16T11:25:49.279696+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.39","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252215+0000","last_change":"2023-10-16T11:25:37.182745+0000","last_active":"2023-10-16T11:25:40.252215+0000","last_peered":"2023-10-16T11:25:40.252215+0000","last_clean":"2023-10-16T11:25:40.252215+0000","last_became_active":"2023-10-16T11:25:37.182437+0000","last_became_peered":"2023-10-16T11:25:37.182437+0000","last_unstale":"2023-10-16T11:25:40.252215+0000","last_undegraded":"2023-10-16T11:25:40.252215+0000","last_fullsized":"2023-10-16T11:25:40.252215+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.269438+0000","last_change":"2023-10-16T11:25:49.269438+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.269438+0000","last_undegraded":"2023-10-16T11:25:49.269438+0000","last_fullsized":"2023-10-16T11:25:49.269438+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.38","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254686+0000","last_change":"2023-10-16T11:25:37.174929+0000","last_active":"2023-10-16T11:25:40.254686+0000","last_peered":"2023-10-16T11:25:40.254686+0000","last_clean":"2023-10-16T11:25:40.254686+0000","last_became_active":"2023-10-16T11:25:37.172810+0000","last_became_peered":"2023-10-16T11:25:37.172810+0000","last_unstale":"2023-10-16T11:25:40.254686+0000","last_undegraded":"2023-10-16T11:25:40.254686+0000","last_fullsized":"2023-10-16T11:25:40.254686+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.273300+0000","last_change":"2023-10-16T11:25:49.273300+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.273300+0000","last_undegraded":"2023-10-16T11:25:49.273300+0000","last_fullsized":"2023-10-16T11:25:49.273300+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.37","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.259355+0000","last_change":"2023-10-16T11:25:37.177647+0000","last_active":"2023-10-16T11:25:40.259355+0000","last_peered":"2023-10-16T11:25:40.259355+0000","last_clean":"2023-10-16T11:25:40.259355+0000","last_became_active":"2023-10-16T11:25:37.176875+0000","last_became_peered":"2023-10-16T11:25:37.176875+0000","last_unstale":"2023-10-16T11:25:40.259355+0000","last_undegraded":"2023-10-16T11:25:40.259355+0000","last_fullsized":"2023-10-16T11:25:40.259355+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:21.630921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.283150+0000","last_change":"2023-10-16T11:25:49.283150+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.283150+0000","last_undegraded":"2023-10-16T11:25:49.283150+0000","last_fullsized":"2023-10-16T11:25:49.283150+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.36","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252664+0000","last_change":"2023-10-16T11:25:37.172841+0000","last_active":"2023-10-16T11:25:40.252664+0000","last_peered":"2023-10-16T11:25:40.252664+0000","last_clean":"2023-10-16T11:25:40.252664+0000","last_became_active":"2023-10-16T11:25:37.172550+0000","last_became_peered":"2023-10-16T11:25:37.172550+0000","last_unstale":"2023-10-16T11:25:40.252664+0000","last_undegraded":"2023-10-16T11:25:40.252664+0000","last_fullsized":"2023-10-16T11:25:40.252664+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.274776+0000","last_change":"2023-10-16T11:25:49.274776+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.274776+0000","last_undegraded":"2023-10-16T11:25:49.274776+0000","last_fullsized":"2023-10-16T11:25:49.274776+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.35","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249412+0000","last_change":"2023-10-16T11:25:37.181386+0000","last_active":"2023-10-16T11:25:40.249412+0000","last_peered":"2023-10-16T11:25:40.249412+0000","last_clean":"2023-10-16T11:25:40.249412+0000","last_became_active":"2023-10-16T11:25:37.181007+0000","last_became_peered":"2023-10-16T11:25:37.181007+0000","last_unstale":"2023-10-16T11:25:40.249412+0000","last_undegraded":"2023-10-16T11:25:40.249412+0000","last_fullsized":"2023-10-16T11:25:40.249412+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:46.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.277979+0000","last_change":"2023-10-16T11:25:49.277979+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.277979+0000","last_undegraded":"2023-10-16T11:25:49.277979+0000","last_fullsized":"2023-10-16T11:25:49.277979+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.34","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254920+0000","last_change":"2023-10-16T11:25:37.179103+0000","last_active":"2023-10-16T11:25:40.254920+0000","last_peered":"2023-10-16T11:25:40.254920+0000","last_clean":"2023-10-16T11:25:40.254920+0000","last_became_active":"2023-10-16T11:25:37.178445+0000","last_became_peered":"2023-10-16T11:25:37.178445+0000","last_unstale":"2023-10-16T11:25:40.254920+0000","last_undegraded":"2023-10-16T11:25:40.254920+0000","last_fullsized":"2023-10-16T11:25:40.254920+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:38.218704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.269375+0000","last_change":"2023-10-16T11:25:49.269375+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.269375+0000","last_undegraded":"2023-10-16T11:25:49.269375+0000","last_fullsized":"2023-10-16T11:25:49.269375+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.33","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252560+0000","last_change":"2023-10-16T11:25:37.182921+0000","last_active":"2023-10-16T11:25:40.252560+0000","last_peered":"2023-10-16T11:25:40.252560+0000","last_clean":"2023-10-16T11:25:40.252560+0000","last_became_active":"2023-10-16T11:25:37.182603+0000","last_became_peered":"2023-10-16T11:25:37.182603+0000","last_unstale":"2023-10-16T11:25:40.252560+0000","last_undegraded":"2023-10-16T11:25:40.252560+0000","last_fullsized":"2023-10-16T11:25:40.252560+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.283709+0000","last_change":"2023-10-16T11:25:49.283709+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.283709+0000","last_undegraded":"2023-10-16T11:25:49.283709+0000","last_fullsized":"2023-10-16T11:25:49.283709+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.32","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248649+0000","last_change":"2023-10-16T11:25:37.171715+0000","last_active":"2023-10-16T11:25:40.248649+0000","last_peered":"2023-10-16T11:25:40.248649+0000","last_clean":"2023-10-16T11:25:40.248649+0000","last_became_active":"2023-10-16T11:25:37.171421+0000","last_became_peered":"2023-10-16T11:25:37.171421+0000","last_unstale":"2023-10-16T11:25:40.248649+0000","last_undegraded":"2023-10-16T11:25:40.248649+0000","last_fullsized":"2023-10-16T11:25:40.248649+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:10.207081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.266505+0000","last_change":"2023-10-16T11:25:49.266505+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.266505+0000","last_undegraded":"2023-10-16T11:25:49.266505+0000","last_fullsized":"2023-10-16T11:25:49.266505+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.31","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.259204+0000","last_change":"2023-10-16T11:25:37.177928+0000","last_active":"2023-10-16T11:25:40.259204+0000","last_peered":"2023-10-16T11:25:40.259204+0000","last_clean":"2023-10-16T11:25:40.259204+0000","last_became_active":"2023-10-16T11:25:37.177612+0000","last_became_peered":"2023-10-16T11:25:37.177612+0000","last_unstale":"2023-10-16T11:25:40.259204+0000","last_undegraded":"2023-10-16T11:25:40.259204+0000","last_fullsized":"2023-10-16T11:25:40.259204+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:44.826687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.274689+0000","last_change":"2023-10-16T11:25:49.274689+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.274689+0000","last_undegraded":"2023-10-16T11:25:49.274689+0000","last_fullsized":"2023-10-16T11:25:49.274689+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.30","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249967+0000","last_change":"2023-10-16T11:25:37.170292+0000","last_active":"2023-10-16T11:25:40.249967+0000","last_peered":"2023-10-16T11:25:40.249967+0000","last_clean":"2023-10-16T11:25:40.249967+0000","last_became_active":"2023-10-16T11:25:37.169997+0000","last_became_peered":"2023-10-16T11:25:37.169997+0000","last_unstale":"2023-10-16T11:25:40.249967+0000","last_undegraded":"2023-10-16T11:25:40.249967+0000","last_fullsized":"2023-10-16T11:25:40.249967+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:09.329721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.267011+0000","last_change":"2023-10-16T11:25:49.267011+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.267011+0000","last_undegraded":"2023-10-16T11:25:49.267011+0000","last_fullsized":"2023-10-16T11:25:49.267011+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2f","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255135+0000","last_change":"2023-10-16T11:25:37.162923+0000","last_active":"2023-10-16T11:25:40.255135+0000","last_peered":"2023-10-16T11:25:40.255135+0000","last_clean":"2023-10-16T11:25:40.255135+0000","last_became_active":"2023-10-16T11:25:37.162649+0000","last_became_peered":"2023-10-16T11:25:37.162649+0000","last_unstale":"2023-10-16T11:25:40.255135+0000","last_undegraded":"2023-10-16T11:25:40.255135+0000","last_fullsized":"2023-10-16T11:25:40.255135+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:46.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.279612+0000","last_change":"2023-10-16T11:25:49.279612+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.279612+0000","last_undegraded":"2023-10-16T11:25:49.279612+0000","last_fullsized":"2023-10-16T11:25:49.279612+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.2e","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248737+0000","last_change":"2023-10-16T11:25:37.174044+0000","last_active":"2023-10-16T11:25:40.248737+0000","last_peered":"2023-10-16T11:25:40.248737+0000","last_clean":"2023-10-16T11:25:40.248737+0000","last_became_active":"2023-10-16T11:25:37.166694+0000","last_became_peered":"2023-10-16T11:25:37.166694+0000","last_unstale":"2023-10-16T11:25:40.248737+0000","last_undegraded":"2023-10-16T11:25:40.248737+0000","last_fullsized":"2023-10-16T11:25:40.248737+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:23.146288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.271754+0000","last_change":"2023-10-16T11:25:49.271754+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.271754+0000","last_undegraded":"2023-10-16T11:25:49.271754+0000","last_fullsized":"2023-10-16T11:25:49.271754+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2d","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254960+0000","last_change":"2023-10-16T11:25:37.183893+0000","last_active":"2023-10-16T11:25:40.254960+0000","last_peered":"2023-10-16T11:25:40.254960+0000","last_clean":"2023-10-16T11:25:40.254960+0000","last_became_active":"2023-10-16T11:25:37.183586+0000","last_became_peered":"2023-10-16T11:25:37.183586+0000","last_unstale":"2023-10-16T11:25:40.254960+0000","last_undegraded":"2023-10-16T11:25:40.254960+0000","last_fullsized":"2023-10-16T11:25:40.254960+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:43.813878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.274305+0000","last_change":"2023-10-16T11:25:49.274305+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.274305+0000","last_undegraded":"2023-10-16T11:25:49.274305+0000","last_fullsized":"2023-10-16T11:25:49.274305+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2c","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.250037+0000","last_change":"2023-10-16T11:25:37.172984+0000","last_active":"2023-10-16T11:25:40.250037+0000","last_peered":"2023-10-16T11:25:40.250037+0000","last_clean":"2023-10-16T11:25:40.250037+0000","last_became_active":"2023-10-16T11:25:37.172403+0000","last_became_peered":"2023-10-16T11:25:37.172403+0000","last_unstale":"2023-10-16T11:25:40.250037+0000","last_undegraded":"2023-10-16T11:25:40.250037+0000","last_fullsized":"2023-10-16T11:25:40.250037+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:03.912995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.272751+0000","last_change":"2023-10-16T11:25:49.272751+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.272751+0000","last_undegraded":"2023-10-16T11:25:49.272751+0000","last_fullsized":"2023-10-16T11:25:49.272751+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2b","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254876+0000","last_change":"2023-10-16T11:25:37.173026+0000","last_active":"2023-10-16T11:25:40.254876+0000","last_peered":"2023-10-16T11:25:40.254876+0000","last_clean":"2023-10-16T11:25:40.254876+0000","last_became_active":"2023-10-16T11:25:37.172680+0000","last_became_peered":"2023-10-16T11:25:37.172680+0000","last_unstale":"2023-10-16T11:25:40.254876+0000","last_undegraded":"2023-10-16T11:25:40.254876+0000","last_fullsized":"2023-10-16T11:25:40.254876+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:56.314696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.266977+0000","last_change":"2023-10-16T11:25:49.266977+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.266977+0000","last_undegraded":"2023-10-16T11:25:49.266977+0000","last_fullsized":"2023-10-16T11:25:49.266977+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2a","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252432+0000","last_change":"2023-10-16T11:25:37.176870+0000","last_active":"2023-10-16T11:25:40.252432+0000","last_peered":"2023-10-16T11:25:40.252432+0000","last_clean":"2023-10-16T11:25:40.252432+0000","last_became_active":"2023-10-16T11:25:37.175230+0000","last_became_peered":"2023-10-16T11:25:37.175230+0000","last_unstale":"2023-10-16T11:25:40.252432+0000","last_undegraded":"2023-10-16T11:25:40.252432+0000","last_fullsized":"2023-10-16T11:25:40.252432+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.267347+0000","last_change":"2023-10-16T11:25:49.267347+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.267347+0000","last_undegraded":"2023-10-16T11:25:49.267347+0000","last_fullsized":"2023-10-16T11:25:49.267347+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.29","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254808+0000","last_change":"2023-10-16T11:25:37.169637+0000","last_active":"2023-10-16T11:25:40.254808+0000","last_peered":"2023-10-16T11:25:40.254808+0000","last_clean":"2023-10-16T11:25:40.254808+0000","last_became_active":"2023-10-16T11:25:37.169375+0000","last_became_peered":"2023-10-16T11:25:37.169375+0000","last_unstale":"2023-10-16T11:25:40.254808+0000","last_undegraded":"2023-10-16T11:25:40.254808+0000","last_fullsized":"2023-10-16T11:25:40.254808+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:42:23.188876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.272728+0000","last_change":"2023-10-16T11:25:49.272728+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.272728+0000","last_undegraded":"2023-10-16T11:25:49.272728+0000","last_fullsized":"2023-10-16T11:25:49.272728+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.28","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248558+0000","last_change":"2023-10-16T11:25:37.187270+0000","last_active":"2023-10-16T11:25:40.248558+0000","last_peered":"2023-10-16T11:25:40.248558+0000","last_clean":"2023-10-16T11:25:40.248558+0000","last_became_active":"2023-10-16T11:25:37.186962+0000","last_became_peered":"2023-10-16T11:25:37.186962+0000","last_unstale":"2023-10-16T11:25:40.248558+0000","last_undegraded":"2023-10-16T11:25:40.248558+0000","last_fullsized":"2023-10-16T11:25:40.248558+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:56:45.915920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.267168+0000","last_change":"2023-10-16T11:25:49.267168+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.267168+0000","last_undegraded":"2023-10-16T11:25:49.267168+0000","last_fullsized":"2023-10-16T11:25:49.267168+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.27","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255281+0000","last_change":"2023-10-16T11:25:37.177057+0000","last_active":"2023-10-16T11:25:40.255281+0000","last_peered":"2023-10-16T11:25:40.255281+0000","last_clean":"2023-10-16T11:25:40.255281+0000","last_became_active":"2023-10-16T11:25:37.171369+0000","last_became_peered":"2023-10-16T11:25:37.171369+0000","last_unstale":"2023-10-16T11:25:40.255281+0000","last_undegraded":"2023-10-16T11:25:40.255281+0000","last_fullsized":"2023-10-16T11:25:40.255281+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.279039+0000","last_change":"2023-10-16T11:25:49.279039+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.279039+0000","last_undegraded":"2023-10-16T11:25:49.279039+0000","last_fullsized":"2023-10-16T11:25:49.279039+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.26","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252706+0000","last_change":"2023-10-16T11:25:37.182875+0000","last_active":"2023-10-16T11:25:40.252706+0000","last_peered":"2023-10-16T11:25:40.252706+0000","last_clean":"2023-10-16T11:25:40.252706+0000","last_became_active":"2023-10-16T11:25:37.182570+0000","last_became_peered":"2023-10-16T11:25:37.182570+0000","last_unstale":"2023-10-16T11:25:40.252706+0000","last_undegraded":"2023-10-16T11:25:40.252706+0000","last_fullsized":"2023-10-16T11:25:40.252706+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:14.504702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.272764+0000","last_change":"2023-10-16T11:25:49.272764+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.272764+0000","last_undegraded":"2023-10-16T11:25:49.272764+0000","last_fullsized":"2023-10-16T11:25:49.272764+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.25","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249433+0000","last_change":"2023-10-16T11:25:37.178245+0000","last_active":"2023-10-16T11:25:40.249433+0000","last_peered":"2023-10-16T11:25:40.249433+0000","last_clean":"2023-10-16T11:25:40.249433+0000","last_became_active":"2023-10-16T11:25:37.177914+0000","last_became_peered":"2023-10-16T11:25:37.177914+0000","last_unstale":"2023-10-16T11:25:40.249433+0000","last_undegraded":"2023-10-16T11:25:40.249433+0000","last_fullsized":"2023-10-16T11:25:40.249433+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:12.014513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.267216+0000","last_change":"2023-10-16T11:25:49.267216+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.267216+0000","last_undegraded":"2023-10-16T11:25:49.267216+0000","last_fullsized":"2023-10-16T11:25:49.267216+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.24","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248686+0000","last_change":"2023-10-16T11:25:37.170623+0000","last_active":"2023-10-16T11:25:40.248686+0000","last_peered":"2023-10-16T11:25:40.248686+0000","last_clean":"2023-10-16T11:25:40.248686+0000","last_became_active":"2023-10-16T11:25:37.170365+0000","last_became_peered":"2023-10-16T11:25:37.170365+0000","last_unstale":"2023-10-16T11:25:40.248686+0000","last_undegraded":"2023-10-16T11:25:40.248686+0000","last_fullsized":"2023-10-16T11:25:40.248686+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:52.443160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.266326+0000","last_change":"2023-10-16T11:25:49.266326+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.266326+0000","last_undegraded":"2023-10-16T11:25:49.266326+0000","last_fullsized":"2023-10-16T11:25:49.266326+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.23","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249332+0000","last_change":"2023-10-16T11:25:37.178693+0000","last_active":"2023-10-16T11:25:40.249332+0000","last_peered":"2023-10-16T11:25:40.249332+0000","last_clean":"2023-10-16T11:25:40.249332+0000","last_became_active":"2023-10-16T11:25:37.175403+0000","last_became_peered":"2023-10-16T11:25:37.175403+0000","last_unstale":"2023-10-16T11:25:40.249332+0000","last_undegraded":"2023-10-16T11:25:40.249332+0000","last_fullsized":"2023-10-16T11:25:40.249332+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:15:31.572558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.267625+0000","last_change":"2023-10-16T11:25:49.267625+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.267625+0000","last_undegraded":"2023-10-16T11:25:49.267625+0000","last_fullsized":"2023-10-16T11:25:49.267625+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.22","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.257055+0000","last_change":"2023-10-16T11:25:37.181812+0000","last_active":"2023-10-16T11:25:40.257055+0000","last_peered":"2023-10-16T11:25:40.257055+0000","last_clean":"2023-10-16T11:25:40.257055+0000","last_became_active":"2023-10-16T11:25:37.181529+0000","last_became_peered":"2023-10-16T11:25:37.181529+0000","last_unstale":"2023-10-16T11:25:40.257055+0000","last_undegraded":"2023-10-16T11:25:40.257055+0000","last_fullsized":"2023-10-16T11:25:40.257055+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:35.912014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.269415+0000","last_change":"2023-10-16T11:25:49.269415+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.269415+0000","last_undegraded":"2023-10-16T11:25:49.269415+0000","last_fullsized":"2023-10-16T11:25:49.269415+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.21","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255217+0000","last_change":"2023-10-16T11:25:37.166981+0000","last_active":"2023-10-16T11:25:40.255217+0000","last_peered":"2023-10-16T11:25:40.255217+0000","last_clean":"2023-10-16T11:25:40.255217+0000","last_became_active":"2023-10-16T11:25:37.164147+0000","last_became_peered":"2023-10-16T11:25:37.164147+0000","last_unstale":"2023-10-16T11:25:40.255217+0000","last_undegraded":"2023-10-16T11:25:40.255217+0000","last_fullsized":"2023-10-16T11:25:40.255217+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":1,"reported_epoch":238,"state":"creating+peering","last_fresh":"2023-10-16T11:25:49.273227+0000","last_change":"2023-10-16T11:25:49.273227+0000","last_active":"2023-10-16T11:25:49.243481+0000","last_peered":"2023-10-16T11:25:49.243481+0000","last_clean":"2023-10-16T11:25:49.243481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:49.273227+0000","last_undegraded":"2023-10-16T11:25:49.273227+0000","last_fullsized":"2023-10-16T11:25:49.273227+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.20","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248589+0000","last_change":"2023-10-16T11:25:37.178631+0000","last_active":"2023-10-16T11:25:40.248589+0000","last_peered":"2023-10-16T11:25:40.248589+0000","last_clean":"2023-10-16T11:25:40.248589+0000","last_became_active":"2023-10-16T11:25:37.167402+0000","last_became_peered":"2023-10-16T11:25:37.167402+0000","last_unstale":"2023-10-16T11:25:40.248589+0000","last_undegraded":"2023-10-16T11:25:40.248589+0000","last_fullsized":"2023-10-16T11:25:40.248589+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:55.572264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.250139+0000","last_change":"2023-10-16T11:25:37.177706+0000","last_active":"2023-10-16T11:25:40.250139+0000","last_peered":"2023-10-16T11:25:40.250139+0000","last_clean":"2023-10-16T11:25:40.250139+0000","last_became_active":"2023-10-16T11:25:37.176806+0000","last_became_peered":"2023-10-16T11:25:37.176806+0000","last_unstale":"2023-10-16T11:25:40.250139+0000","last_undegraded":"2023-10-16T11:25:40.250139+0000","last_fullsized":"2023-10-16T11:25:40.250139+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:42:32.585642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1e","version":"236'1","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.177232+0000","last_change":"2023-10-16T11:25:36.169327+0000","last_active":"2023-10-16T11:25:38.177232+0000","last_peered":"2023-10-16T11:25:38.177232+0000","last_clean":"2023-10-16T11:25:38.177232+0000","last_became_active":"2023-10-16T11:25:36.169046+0000","last_became_peered":"2023-10-16T11:25:36.169046+0000","last_unstale":"2023-10-16T11:25:38.177232+0000","last_undegraded":"2023-10-16T11:25:38.177232+0000","last_fullsized":"2023-10-16T11:25:38.177232+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:34.899695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249353+0000","last_change":"2023-10-16T11:25:37.173736+0000","last_active":"2023-10-16T11:25:40.249353+0000","last_peered":"2023-10-16T11:25:40.249353+0000","last_clean":"2023-10-16T11:25:40.249353+0000","last_became_active":"2023-10-16T11:25:37.173482+0000","last_became_peered":"2023-10-16T11:25:37.173482+0000","last_unstale":"2023-10-16T11:25:40.249353+0000","last_undegraded":"2023-10-16T11:25:40.249353+0000","last_fullsized":"2023-10-16T11:25:40.249353+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:42:23.188876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1d","version":"236'2","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616465+0000","last_change":"2023-10-16T11:25:36.172861+0000","last_active":"2023-10-16T11:25:38.616465+0000","last_peered":"2023-10-16T11:25:38.616465+0000","last_clean":"2023-10-16T11:25:38.616465+0000","last_became_active":"2023-10-16T11:25:36.172574+0000","last_became_peered":"2023-10-16T11:25:36.172574+0000","last_unstale":"2023-10-16T11:25:38.616465+0000","last_undegraded":"2023-10-16T11:25:38.616465+0000","last_fullsized":"2023-10-16T11:25:38.616465+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:22:17.667795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255008+0000","last_change":"2023-10-16T11:25:37.167451+0000","last_active":"2023-10-16T11:25:40.255008+0000","last_peered":"2023-10-16T11:25:40.255008+0000","last_clean":"2023-10-16T11:25:40.255008+0000","last_became_active":"2023-10-16T11:25:37.167157+0000","last_became_peered":"2023-10-16T11:25:37.167157+0000","last_unstale":"2023-10-16T11:25:40.255008+0000","last_undegraded":"2023-10-16T11:25:40.255008+0000","last_fullsized":"2023-10-16T11:25:40.255008+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:14.504702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1c","version":"236'2","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616883+0000","last_change":"2023-10-16T11:25:36.153273+0000","last_active":"2023-10-16T11:25:38.616883+0000","last_peered":"2023-10-16T11:25:38.616883+0000","last_clean":"2023-10-16T11:25:38.616883+0000","last_became_active":"2023-10-16T11:25:36.152960+0000","last_became_peered":"2023-10-16T11:25:36.152960+0000","last_unstale":"2023-10-16T11:25:38.616883+0000","last_undegraded":"2023-10-16T11:25:38.616883+0000","last_fullsized":"2023-10-16T11:25:38.616883+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:08.218268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249306+0000","last_change":"2023-10-16T11:25:37.174071+0000","last_active":"2023-10-16T11:25:40.249306+0000","last_peered":"2023-10-16T11:25:40.249306+0000","last_clean":"2023-10-16T11:25:40.249306+0000","last_became_active":"2023-10-16T11:25:37.173436+0000","last_became_peered":"2023-10-16T11:25:37.173436+0000","last_unstale":"2023-10-16T11:25:40.249306+0000","last_undegraded":"2023-10-16T11:25:40.249306+0000","last_fullsized":"2023-10-16T11:25:40.249306+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1b","version":"236'1","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.171367+0000","last_change":"2023-10-16T11:25:36.156408+0000","last_active":"2023-10-16T11:25:38.171367+0000","last_peered":"2023-10-16T11:25:38.171367+0000","last_clean":"2023-10-16T11:25:38.171367+0000","last_became_active":"2023-10-16T11:25:36.155786+0000","last_became_peered":"2023-10-16T11:25:36.155786+0000","last_unstale":"2023-10-16T11:25:38.171367+0000","last_undegraded":"2023-10-16T11:25:38.171367+0000","last_fullsized":"2023-10-16T11:25:38.171367+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:01.925277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.259259+0000","last_change":"2023-10-16T11:25:37.179656+0000","last_active":"2023-10-16T11:25:40.259259+0000","last_peered":"2023-10-16T11:25:40.259259+0000","last_clean":"2023-10-16T11:25:40.259259+0000","last_became_active":"2023-10-16T11:25:37.179361+0000","last_became_peered":"2023-10-16T11:25:37.179361+0000","last_unstale":"2023-10-16T11:25:40.259259+0000","last_undegraded":"2023-10-16T11:25:40.259259+0000","last_fullsized":"2023-10-16T11:25:40.259259+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:09.230587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"236'1","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193985+0000","last_change":"2023-10-16T11:25:36.169348+0000","last_active":"2023-10-16T11:25:39.193985+0000","last_peered":"2023-10-16T11:25:39.193985+0000","last_clean":"2023-10-16T11:25:39.193985+0000","last_became_active":"2023-10-16T11:25:36.169053+0000","last_became_peered":"2023-10-16T11:25:36.169053+0000","last_unstale":"2023-10-16T11:25:39.193985+0000","last_undegraded":"2023-10-16T11:25:39.193985+0000","last_fullsized":"2023-10-16T11:25:39.193985+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:05.819502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249986+0000","last_change":"2023-10-16T11:25:37.176966+0000","last_active":"2023-10-16T11:25:40.249986+0000","last_peered":"2023-10-16T11:25:40.249986+0000","last_clean":"2023-10-16T11:25:40.249986+0000","last_became_active":"2023-10-16T11:25:37.174621+0000","last_became_peered":"2023-10-16T11:25:37.174621+0000","last_unstale":"2023-10-16T11:25:40.249986+0000","last_undegraded":"2023-10-16T11:25:40.249986+0000","last_fullsized":"2023-10-16T11:25:40.249986+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:56:09.516122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174368+0000","last_change":"2023-10-16T11:25:36.169635+0000","last_active":"2023-10-16T11:25:38.174368+0000","last_peered":"2023-10-16T11:25:38.174368+0000","last_clean":"2023-10-16T11:25:38.174368+0000","last_became_active":"2023-10-16T11:25:36.169016+0000","last_became_peered":"2023-10-16T11:25:36.169016+0000","last_unstale":"2023-10-16T11:25:38.174368+0000","last_undegraded":"2023-10-16T11:25:38.174368+0000","last_fullsized":"2023-10-16T11:25:38.174368+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:53:06.412234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248609+0000","last_change":"2023-10-16T11:25:37.184475+0000","last_active":"2023-10-16T11:25:40.248609+0000","last_peered":"2023-10-16T11:25:40.248609+0000","last_clean":"2023-10-16T11:25:40.248609+0000","last_became_active":"2023-10-16T11:25:37.179914+0000","last_became_peered":"2023-10-16T11:25:37.179914+0000","last_unstale":"2023-10-16T11:25:40.248609+0000","last_undegraded":"2023-10-16T11:25:40.248609+0000","last_fullsized":"2023-10-16T11:25:40.248609+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:53.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616094+0000","last_change":"2023-10-16T11:25:36.155589+0000","last_active":"2023-10-16T11:25:38.616094+0000","last_peered":"2023-10-16T11:25:38.616094+0000","last_clean":"2023-10-16T11:25:38.616094+0000","last_became_active":"2023-10-16T11:25:36.155087+0000","last_became_peered":"2023-10-16T11:25:36.155087+0000","last_unstale":"2023-10-16T11:25:38.616094+0000","last_undegraded":"2023-10-16T11:25:38.616094+0000","last_fullsized":"2023-10-16T11:25:38.616094+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:52:38.937212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248756+0000","last_change":"2023-10-16T11:25:37.175447+0000","last_active":"2023-10-16T11:25:40.248756+0000","last_peered":"2023-10-16T11:25:40.248756+0000","last_clean":"2023-10-16T11:25:40.248756+0000","last_became_active":"2023-10-16T11:25:37.173563+0000","last_became_peered":"2023-10-16T11:25:37.173563+0000","last_unstale":"2023-10-16T11:25:40.248756+0000","last_undegraded":"2023-10-16T11:25:40.248756+0000","last_fullsized":"2023-10-16T11:25:40.248756+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:54.544135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.198655+0000","last_change":"2023-10-16T11:25:36.169351+0000","last_active":"2023-10-16T11:25:39.198655+0000","last_peered":"2023-10-16T11:25:39.198655+0000","last_clean":"2023-10-16T11:25:39.198655+0000","last_became_active":"2023-10-16T11:25:36.168347+0000","last_became_peered":"2023-10-16T11:25:36.168347+0000","last_unstale":"2023-10-16T11:25:39.198655+0000","last_undegraded":"2023-10-16T11:25:39.198655+0000","last_fullsized":"2023-10-16T11:25:39.198655+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:19.077096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249471+0000","last_change":"2023-10-16T11:25:37.179048+0000","last_active":"2023-10-16T11:25:40.249471+0000","last_peered":"2023-10-16T11:25:40.249471+0000","last_clean":"2023-10-16T11:25:40.249471+0000","last_became_active":"2023-10-16T11:25:37.178751+0000","last_became_peered":"2023-10-16T11:25:37.178751+0000","last_unstale":"2023-10-16T11:25:40.249471+0000","last_undegraded":"2023-10-16T11:25:40.249471+0000","last_fullsized":"2023-10-16T11:25:40.249471+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:48:10.906108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616368+0000","last_change":"2023-10-16T11:25:36.154256+0000","last_active":"2023-10-16T11:25:38.616368+0000","last_peered":"2023-10-16T11:25:38.616368+0000","last_clean":"2023-10-16T11:25:38.616368+0000","last_became_active":"2023-10-16T11:25:36.153421+0000","last_became_peered":"2023-10-16T11:25:36.153421+0000","last_unstale":"2023-10-16T11:25:38.616368+0000","last_undegraded":"2023-10-16T11:25:38.616368+0000","last_fullsized":"2023-10-16T11:25:38.616368+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:09.436508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.250064+0000","last_change":"2023-10-16T11:25:37.188270+0000","last_active":"2023-10-16T11:25:40.250064+0000","last_peered":"2023-10-16T11:25:40.250064+0000","last_clean":"2023-10-16T11:25:40.250064+0000","last_became_active":"2023-10-16T11:25:37.187995+0000","last_became_peered":"2023-10-16T11:25:37.187995+0000","last_unstale":"2023-10-16T11:25:40.250064+0000","last_undegraded":"2023-10-16T11:25:40.250064+0000","last_fullsized":"2023-10-16T11:25:40.250064+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:42:22.542072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.15","version":"236'3","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193813+0000","last_change":"2023-10-16T11:25:36.171645+0000","last_active":"2023-10-16T11:25:39.193813+0000","last_peered":"2023-10-16T11:25:39.193813+0000","last_clean":"2023-10-16T11:25:39.193813+0000","last_became_active":"2023-10-16T11:25:36.170729+0000","last_became_peered":"2023-10-16T11:25:36.170729+0000","last_unstale":"2023-10-16T11:25:39.193813+0000","last_undegraded":"2023-10-16T11:25:39.193813+0000","last_fullsized":"2023-10-16T11:25:39.193813+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:34.362958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252622+0000","last_change":"2023-10-16T11:25:37.163991+0000","last_active":"2023-10-16T11:25:40.252622+0000","last_peered":"2023-10-16T11:25:40.252622+0000","last_clean":"2023-10-16T11:25:40.252622+0000","last_became_active":"2023-10-16T11:25:37.163080+0000","last_became_peered":"2023-10-16T11:25:37.163080+0000","last_unstale":"2023-10-16T11:25:40.252622+0000","last_undegraded":"2023-10-16T11:25:40.252622+0000","last_fullsized":"2023-10-16T11:25:40.252622+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.14","version":"237'8","reported_seq":21,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:46.774610+0000","last_change":"2023-10-16T11:25:36.173259+0000","last_active":"2023-10-16T11:25:46.774610+0000","last_peered":"2023-10-16T11:25:46.774610+0000","last_clean":"2023-10-16T11:25:46.774610+0000","last_became_active":"2023-10-16T11:25:36.172980+0000","last_became_peered":"2023-10-16T11:25:36.172980+0000","last_unstale":"2023-10-16T11:25:46.774610+0000","last_undegraded":"2023-10-16T11:25:46.774610+0000","last_fullsized":"2023-10-16T11:25:46.774610+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:33.782641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252475+0000","last_change":"2023-10-16T11:25:37.166081+0000","last_active":"2023-10-16T11:25:40.252475+0000","last_peered":"2023-10-16T11:25:40.252475+0000","last_clean":"2023-10-16T11:25:40.252475+0000","last_became_active":"2023-10-16T11:25:37.165150+0000","last_became_peered":"2023-10-16T11:25:37.165150+0000","last_unstale":"2023-10-16T11:25:40.252475+0000","last_undegraded":"2023-10-16T11:25:40.252475+0000","last_fullsized":"2023-10-16T11:25:40.252475+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:56.314696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"236'2","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:39.193905+0000","last_change":"2023-10-16T11:25:36.171468+0000","last_active":"2023-10-16T11:25:39.193905+0000","last_peered":"2023-10-16T11:25:39.193905+0000","last_clean":"2023-10-16T11:25:39.193905+0000","last_became_active":"2023-10-16T11:25:36.171236+0000","last_became_peered":"2023-10-16T11:25:36.171236+0000","last_unstale":"2023-10-16T11:25:39.193905+0000","last_undegraded":"2023-10-16T11:25:39.193905+0000","last_fullsized":"2023-10-16T11:25:39.193905+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:17.018903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.259283+0000","last_change":"2023-10-16T11:25:37.178517+0000","last_active":"2023-10-16T11:25:40.259283+0000","last_peered":"2023-10-16T11:25:40.259283+0000","last_clean":"2023-10-16T11:25:40.259283+0000","last_became_active":"2023-10-16T11:25:37.177794+0000","last_became_peered":"2023-10-16T11:25:37.177794+0000","last_unstale":"2023-10-16T11:25:40.259283+0000","last_undegraded":"2023-10-16T11:25:40.259283+0000","last_fullsized":"2023-10-16T11:25:40.259283+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:52:39.949531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616389+0000","last_change":"2023-10-16T11:25:36.162595+0000","last_active":"2023-10-16T11:25:38.616389+0000","last_peered":"2023-10-16T11:25:38.616389+0000","last_clean":"2023-10-16T11:25:38.616389+0000","last_became_active":"2023-10-16T11:25:36.162249+0000","last_became_peered":"2023-10-16T11:25:36.162249+0000","last_unstale":"2023-10-16T11:25:38.616389+0000","last_undegraded":"2023-10-16T11:25:38.616389+0000","last_fullsized":"2023-10-16T11:25:38.616389+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:49.547165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255198+0000","last_change":"2023-10-16T11:25:37.164604+0000","last_active":"2023-10-16T11:25:40.255198+0000","last_peered":"2023-10-16T11:25:40.255198+0000","last_clean":"2023-10-16T11:25:40.255198+0000","last_became_active":"2023-10-16T11:25:37.162365+0000","last_became_peered":"2023-10-16T11:25:37.162365+0000","last_unstale":"2023-10-16T11:25:40.255198+0000","last_undegraded":"2023-10-16T11:25:40.255198+0000","last_fullsized":"2023-10-16T11:25:40.255198+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:35.912014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174625+0000","last_change":"2023-10-16T11:25:36.170014+0000","last_active":"2023-10-16T11:25:38.174625+0000","last_peered":"2023-10-16T11:25:38.174625+0000","last_clean":"2023-10-16T11:25:38.174625+0000","last_became_active":"2023-10-16T11:25:36.169755+0000","last_became_peered":"2023-10-16T11:25:36.169755+0000","last_unstale":"2023-10-16T11:25:38.174625+0000","last_undegraded":"2023-10-16T11:25:38.174625+0000","last_fullsized":"2023-10-16T11:25:38.174625+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:02.119904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252284+0000","last_change":"2023-10-16T11:25:37.178789+0000","last_active":"2023-10-16T11:25:40.252284+0000","last_peered":"2023-10-16T11:25:40.252284+0000","last_clean":"2023-10-16T11:25:40.252284+0000","last_became_active":"2023-10-16T11:25:37.178510+0000","last_became_peered":"2023-10-16T11:25:37.178510+0000","last_unstale":"2023-10-16T11:25:40.252284+0000","last_undegraded":"2023-10-16T11:25:40.252284+0000","last_fullsized":"2023-10-16T11:25:40.252284+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.10","version":"236'8","reported_seq":21,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182764+0000","last_change":"2023-10-16T11:25:36.168889+0000","last_active":"2023-10-16T11:25:38.182764+0000","last_peered":"2023-10-16T11:25:38.182764+0000","last_clean":"2023-10-16T11:25:38.182764+0000","last_became_active":"2023-10-16T11:25:36.168590+0000","last_became_peered":"2023-10-16T11:25:36.168590+0000","last_unstale":"2023-10-16T11:25:38.182764+0000","last_undegraded":"2023-10-16T11:25:38.182764+0000","last_fullsized":"2023-10-16T11:25:38.182764+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:42.801560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255269+0000","last_change":"2023-10-16T11:25:37.175992+0000","last_active":"2023-10-16T11:25:40.255269+0000","last_peered":"2023-10-16T11:25:40.255269+0000","last_clean":"2023-10-16T11:25:40.255269+0000","last_became_active":"2023-10-16T11:25:37.173732+0000","last_became_peered":"2023-10-16T11:25:37.173732+0000","last_unstale":"2023-10-16T11:25:40.255269+0000","last_undegraded":"2023-10-16T11:25:40.255269+0000","last_fullsized":"2023-10-16T11:25:40.255269+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.f","version":"236'1","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.177302+0000","last_change":"2023-10-16T11:25:36.173935+0000","last_active":"2023-10-16T11:25:38.177302+0000","last_peered":"2023-10-16T11:25:38.177302+0000","last_clean":"2023-10-16T11:25:38.177302+0000","last_became_active":"2023-10-16T11:25:36.173510+0000","last_became_peered":"2023-10-16T11:25:36.173510+0000","last_unstale":"2023-10-16T11:25:38.177302+0000","last_undegraded":"2023-10-16T11:25:38.177302+0000","last_fullsized":"2023-10-16T11:25:38.177302+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:19.077096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.250150+0000","last_change":"2023-10-16T11:25:37.177543+0000","last_active":"2023-10-16T11:25:40.250150+0000","last_peered":"2023-10-16T11:25:40.250150+0000","last_clean":"2023-10-16T11:25:40.250150+0000","last_became_active":"2023-10-16T11:25:37.176420+0000","last_became_peered":"2023-10-16T11:25:37.176420+0000","last_unstale":"2023-10-16T11:25:40.250150+0000","last_undegraded":"2023-10-16T11:25:40.250150+0000","last_fullsized":"2023-10-16T11:25:40.250150+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:24.104981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.e","version":"236'2","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616631+0000","last_change":"2023-10-16T11:25:36.170878+0000","last_active":"2023-10-16T11:25:38.616631+0000","last_peered":"2023-10-16T11:25:38.616631+0000","last_clean":"2023-10-16T11:25:38.616631+0000","last_became_active":"2023-10-16T11:25:36.170548+0000","last_became_peered":"2023-10-16T11:25:36.170548+0000","last_unstale":"2023-10-16T11:25:38.616631+0000","last_undegraded":"2023-10-16T11:25:38.616631+0000","last_fullsized":"2023-10-16T11:25:38.616631+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:34:14.932409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.259306+0000","last_change":"2023-10-16T11:25:37.176420+0000","last_active":"2023-10-16T11:25:40.259306+0000","last_peered":"2023-10-16T11:25:40.259306+0000","last_clean":"2023-10-16T11:25:40.259306+0000","last_became_active":"2023-10-16T11:25:37.176101+0000","last_became_peered":"2023-10-16T11:25:37.176101+0000","last_unstale":"2023-10-16T11:25:40.259306+0000","last_undegraded":"2023-10-16T11:25:40.259306+0000","last_fullsized":"2023-10-16T11:25:40.259306+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:10.245184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"236'1","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.175412+0000","last_change":"2023-10-16T11:25:36.171476+0000","last_active":"2023-10-16T11:25:38.175412+0000","last_peered":"2023-10-16T11:25:38.175412+0000","last_clean":"2023-10-16T11:25:38.175412+0000","last_became_active":"2023-10-16T11:25:36.171201+0000","last_became_peered":"2023-10-16T11:25:36.171201+0000","last_unstale":"2023-10-16T11:25:38.175412+0000","last_undegraded":"2023-10-16T11:25:38.175412+0000","last_fullsized":"2023-10-16T11:25:38.175412+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:30.034841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.255057+0000","last_change":"2023-10-16T11:25:37.162366+0000","last_active":"2023-10-16T11:25:40.255057+0000","last_peered":"2023-10-16T11:25:40.255057+0000","last_clean":"2023-10-16T11:25:40.255057+0000","last_became_active":"2023-10-16T11:25:37.162032+0000","last_became_peered":"2023-10-16T11:25:37.162032+0000","last_unstale":"2023-10-16T11:25:40.255057+0000","last_undegraded":"2023-10-16T11:25:40.255057+0000","last_fullsized":"2023-10-16T11:25:40.255057+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182815+0000","last_change":"2023-10-16T11:25:36.174246+0000","last_active":"2023-10-16T11:25:38.182815+0000","last_peered":"2023-10-16T11:25:38.182815+0000","last_clean":"2023-10-16T11:25:38.182815+0000","last_became_active":"2023-10-16T11:25:36.173975+0000","last_became_peered":"2023-10-16T11:25:36.173975+0000","last_unstale":"2023-10-16T11:25:38.182815+0000","last_undegraded":"2023-10-16T11:25:38.182815+0000","last_fullsized":"2023-10-16T11:25:38.182815+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:32.130559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254853+0000","last_change":"2023-10-16T11:25:37.168715+0000","last_active":"2023-10-16T11:25:40.254853+0000","last_peered":"2023-10-16T11:25:40.254853+0000","last_clean":"2023-10-16T11:25:40.254853+0000","last_became_active":"2023-10-16T11:25:37.168387+0000","last_became_peered":"2023-10-16T11:25:37.168387+0000","last_unstale":"2023-10-16T11:25:40.254853+0000","last_undegraded":"2023-10-16T11:25:40.254853+0000","last_fullsized":"2023-10-16T11:25:40.254853+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.b","version":"236'2","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616716+0000","last_change":"2023-10-16T11:25:36.174153+0000","last_active":"2023-10-16T11:25:38.616716+0000","last_peered":"2023-10-16T11:25:38.616716+0000","last_clean":"2023-10-16T11:25:38.616716+0000","last_became_active":"2023-10-16T11:25:36.173812+0000","last_became_peered":"2023-10-16T11:25:36.173812+0000","last_unstale":"2023-10-16T11:25:38.616716+0000","last_undegraded":"2023-10-16T11:25:38.616716+0000","last_fullsized":"2023-10-16T11:25:38.616716+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:07:05.931833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248669+0000","last_change":"2023-10-16T11:25:37.175767+0000","last_active":"2023-10-16T11:25:40.248669+0000","last_peered":"2023-10-16T11:25:40.248669+0000","last_clean":"2023-10-16T11:25:40.248669+0000","last_became_active":"2023-10-16T11:25:37.167389+0000","last_became_peered":"2023-10-16T11:25:37.167389+0000","last_unstale":"2023-10-16T11:25:40.248669+0000","last_undegraded":"2023-10-16T11:25:40.248669+0000","last_fullsized":"2023-10-16T11:25:40.248669+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:18.601118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182571+0000","last_change":"2023-10-16T11:25:36.175478+0000","last_active":"2023-10-16T11:25:38.182571+0000","last_peered":"2023-10-16T11:25:38.182571+0000","last_clean":"2023-10-16T11:25:38.182571+0000","last_became_active":"2023-10-16T11:25:36.174935+0000","last_became_peered":"2023-10-16T11:25:36.174935+0000","last_unstale":"2023-10-16T11:25:38.182571+0000","last_undegraded":"2023-10-16T11:25:38.182571+0000","last_fullsized":"2023-10-16T11:25:38.182571+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:14.189489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249282+0000","last_change":"2023-10-16T11:25:37.173034+0000","last_active":"2023-10-16T11:25:40.249282+0000","last_peered":"2023-10-16T11:25:40.249282+0000","last_clean":"2023-10-16T11:25:40.249282+0000","last_became_active":"2023-10-16T11:25:37.172644+0000","last_became_peered":"2023-10-16T11:25:37.172644+0000","last_unstale":"2023-10-16T11:25:40.249282+0000","last_undegraded":"2023-10-16T11:25:40.249282+0000","last_fullsized":"2023-10-16T11:25:40.249282+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174975+0000","last_change":"2023-10-16T11:25:36.163926+0000","last_active":"2023-10-16T11:25:38.174975+0000","last_peered":"2023-10-16T11:25:38.174975+0000","last_clean":"2023-10-16T11:25:38.174975+0000","last_became_active":"2023-10-16T11:25:36.162991+0000","last_became_peered":"2023-10-16T11:25:36.162991+0000","last_unstale":"2023-10-16T11:25:38.174975+0000","last_undegraded":"2023-10-16T11:25:38.174975+0000","last_fullsized":"2023-10-16T11:25:38.174975+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:49.547165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254831+0000","last_change":"2023-10-16T11:25:37.158678+0000","last_active":"2023-10-16T11:25:40.254831+0000","last_peered":"2023-10-16T11:25:40.254831+0000","last_clean":"2023-10-16T11:25:40.254831+0000","last_became_active":"2023-10-16T11:25:37.157389+0000","last_became_peered":"2023-10-16T11:25:37.157389+0000","last_unstale":"2023-10-16T11:25:40.254831+0000","last_undegraded":"2023-10-16T11:25:40.254831+0000","last_fullsized":"2023-10-16T11:25:40.254831+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.174650+0000","last_change":"2023-10-16T11:25:36.171876+0000","last_active":"2023-10-16T11:25:38.174650+0000","last_peered":"2023-10-16T11:25:38.174650+0000","last_clean":"2023-10-16T11:25:38.174650+0000","last_became_active":"2023-10-16T11:25:36.169128+0000","last_became_peered":"2023-10-16T11:25:36.169128+0000","last_unstale":"2023-10-16T11:25:38.174650+0000","last_undegraded":"2023-10-16T11:25:38.174650+0000","last_fullsized":"2023-10-16T11:25:38.174650+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:16:18.153133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252751+0000","last_change":"2023-10-16T11:25:37.179119+0000","last_active":"2023-10-16T11:25:40.252751+0000","last_peered":"2023-10-16T11:25:40.252751+0000","last_clean":"2023-10-16T11:25:40.252751+0000","last_became_active":"2023-10-16T11:25:37.175799+0000","last_became_peered":"2023-10-16T11:25:37.175799+0000","last_unstale":"2023-10-16T11:25:40.252751+0000","last_undegraded":"2023-10-16T11:25:40.252751+0000","last_fullsized":"2023-10-16T11:25:40.252751+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"236'2","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.172151+0000","last_change":"2023-10-16T11:25:36.162608+0000","last_active":"2023-10-16T11:25:38.172151+0000","last_peered":"2023-10-16T11:25:38.172151+0000","last_clean":"2023-10-16T11:25:38.172151+0000","last_became_active":"2023-10-16T11:25:36.162336+0000","last_became_peered":"2023-10-16T11:25:36.162336+0000","last_unstale":"2023-10-16T11:25:38.172151+0000","last_undegraded":"2023-10-16T11:25:38.172151+0000","last_fullsized":"2023-10-16T11:25:38.172151+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:19.377028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248788+0000","last_change":"2023-10-16T11:25:37.177560+0000","last_active":"2023-10-16T11:25:40.248788+0000","last_peered":"2023-10-16T11:25:40.248788+0000","last_clean":"2023-10-16T11:25:40.248788+0000","last_became_active":"2023-10-16T11:25:37.176982+0000","last_became_peered":"2023-10-16T11:25:37.176982+0000","last_unstale":"2023-10-16T11:25:40.248788+0000","last_undegraded":"2023-10-16T11:25:40.248788+0000","last_fullsized":"2023-10-16T11:25:40.248788+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:54.497943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"237'1","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:41.776279+0000","last_change":"2023-10-16T11:25:36.174419+0000","last_active":"2023-10-16T11:25:41.776279+0000","last_peered":"2023-10-16T11:25:41.776279+0000","last_clean":"2023-10-16T11:25:41.776279+0000","last_became_active":"2023-10-16T11:25:36.174148+0000","last_became_peered":"2023-10-16T11:25:36.174148+0000","last_unstale":"2023-10-16T11:25:41.776279+0000","last_undegraded":"2023-10-16T11:25:41.776279+0000","last_fullsized":"2023-10-16T11:25:41.776279+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:38.739323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249374+0000","last_change":"2023-10-16T11:25:37.180683+0000","last_active":"2023-10-16T11:25:40.249374+0000","last_peered":"2023-10-16T11:25:40.249374+0000","last_clean":"2023-10-16T11:25:40.249374+0000","last_became_active":"2023-10-16T11:25:37.180393+0000","last_became_peered":"2023-10-16T11:25:37.180393+0000","last_unstale":"2023-10-16T11:25:40.249374+0000","last_undegraded":"2023-10-16T11:25:40.249374+0000","last_fullsized":"2023-10-16T11:25:40.249374+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:38.218704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.175635+0000","last_change":"2023-10-16T11:25:36.164519+0000","last_active":"2023-10-16T11:25:38.175635+0000","last_peered":"2023-10-16T11:25:38.175635+0000","last_clean":"2023-10-16T11:25:38.175635+0000","last_became_active":"2023-10-16T11:25:36.164214+0000","last_became_peered":"2023-10-16T11:25:36.164214+0000","last_unstale":"2023-10-16T11:25:38.175635+0000","last_undegraded":"2023-10-16T11:25:38.175635+0000","last_fullsized":"2023-10-16T11:25:38.175635+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:03.074535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.249451+0000","last_change":"2023-10-16T11:25:37.181808+0000","last_active":"2023-10-16T11:25:40.249451+0000","last_peered":"2023-10-16T11:25:40.249451+0000","last_clean":"2023-10-16T11:25:40.249451+0000","last_became_active":"2023-10-16T11:25:37.181575+0000","last_became_peered":"2023-10-16T11:25:37.181575+0000","last_unstale":"2023-10-16T11:25:40.249451+0000","last_undegraded":"2023-10-16T11:25:40.249451+0000","last_fullsized":"2023-10-16T11:25:40.249451+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:17:01.131611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.182900+0000","last_change":"2023-10-16T11:25:36.162153+0000","last_active":"2023-10-16T11:25:38.182900+0000","last_peered":"2023-10-16T11:25:38.182900+0000","last_clean":"2023-10-16T11:25:38.182900+0000","last_became_active":"2023-10-16T11:25:36.161842+0000","last_became_peered":"2023-10-16T11:25:36.161842+0000","last_unstale":"2023-10-16T11:25:38.182900+0000","last_undegraded":"2023-10-16T11:25:38.182900+0000","last_fullsized":"2023-10-16T11:25:38.182900+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:55.302377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.252517+0000","last_change":"2023-10-16T11:25:37.165420+0000","last_active":"2023-10-16T11:25:40.252517+0000","last_peered":"2023-10-16T11:25:40.252517+0000","last_clean":"2023-10-16T11:25:40.252517+0000","last_became_active":"2023-10-16T11:25:37.164820+0000","last_became_peered":"2023-10-16T11:25:37.164820+0000","last_unstale":"2023-10-16T11:25:40.252517+0000","last_undegraded":"2023-10-16T11:25:40.252517+0000","last_fullsized":"2023-10-16T11:25:40.252517+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:43.813878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.176218+0000","last_change":"2023-10-16T11:25:36.173209+0000","last_active":"2023-10-16T11:25:38.176218+0000","last_peered":"2023-10-16T11:25:38.176218+0000","last_clean":"2023-10-16T11:25:38.176218+0000","last_became_active":"2023-10-16T11:25:36.171799+0000","last_became_peered":"2023-10-16T11:25:36.171799+0000","last_unstale":"2023-10-16T11:25:38.176218+0000","last_undegraded":"2023-10-16T11:25:38.176218+0000","last_fullsized":"2023-10-16T11:25:38.176218+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:19.377028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254897+0000","last_change":"2023-10-16T11:25:37.180243+0000","last_active":"2023-10-16T11:25:40.254897+0000","last_peered":"2023-10-16T11:25:40.254897+0000","last_clean":"2023-10-16T11:25:40.254897+0000","last_became_active":"2023-10-16T11:25:37.179869+0000","last_became_peered":"2023-10-16T11:25:37.179869+0000","last_unstale":"2023-10-16T11:25:40.254897+0000","last_undegraded":"2023-10-16T11:25:40.254897+0000","last_fullsized":"2023-10-16T11:25:40.254897+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:15:31.572558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.171937+0000","last_change":"2023-10-16T11:25:36.169146+0000","last_active":"2023-10-16T11:25:38.171937+0000","last_peered":"2023-10-16T11:25:38.171937+0000","last_clean":"2023-10-16T11:25:38.171937+0000","last_became_active":"2023-10-16T11:25:36.168877+0000","last_became_peered":"2023-10-16T11:25:36.168877+0000","last_unstale":"2023-10-16T11:25:38.171937+0000","last_undegraded":"2023-10-16T11:25:38.171937+0000","last_fullsized":"2023-10-16T11:25:38.171937+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:35.988148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.254732+0000","last_change":"2023-10-16T11:25:37.173516+0000","last_active":"2023-10-16T11:25:40.254732+0000","last_peered":"2023-10-16T11:25:40.254732+0000","last_clean":"2023-10-16T11:25:40.254732+0000","last_became_active":"2023-10-16T11:25:37.172964+0000","last_became_peered":"2023-10-16T11:25:37.172964+0000","last_unstale":"2023-10-16T11:25:40.254732+0000","last_undegraded":"2023-10-16T11:25:40.254732+0000","last_fullsized":"2023-10-16T11:25:40.254732+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616128+0000","last_change":"2023-10-16T11:25:36.166442+0000","last_active":"2023-10-16T11:25:38.616128+0000","last_peered":"2023-10-16T11:25:38.616128+0000","last_clean":"2023-10-16T11:25:38.616128+0000","last_became_active":"2023-10-16T11:25:36.166131+0000","last_became_peered":"2023-10-16T11:25:36.166131+0000","last_unstale":"2023-10-16T11:25:38.616128+0000","last_undegraded":"2023-10-16T11:25:38.616128+0000","last_fullsized":"2023-10-16T11:25:38.616128+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:43.814368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.248629+0000","last_change":"2023-10-16T11:25:37.183491+0000","last_active":"2023-10-16T11:25:40.248629+0000","last_peered":"2023-10-16T11:25:40.248629+0000","last_clean":"2023-10-16T11:25:40.248629+0000","last_became_active":"2023-10-16T11:25:37.179560+0000","last_became_peered":"2023-10-16T11:25:37.179560+0000","last_unstale":"2023-10-16T11:25:40.248629+0000","last_undegraded":"2023-10-16T11:25:40.248629+0000","last_fullsized":"2023-10-16T11:25:40.248629+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:31.820109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.616369+0000","last_change":"2023-10-16T11:25:36.175845+0000","last_active":"2023-10-16T11:25:38.616369+0000","last_peered":"2023-10-16T11:25:38.616369+0000","last_clean":"2023-10-16T11:25:38.616369+0000","last_became_active":"2023-10-16T11:25:36.175390+0000","last_became_peered":"2023-10-16T11:25:36.175390+0000","last_unstale":"2023-10-16T11:25:38.616369+0000","last_undegraded":"2023-10-16T11:25:38.616369+0000","last_fullsized":"2023-10-16T11:25:38.616369+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:09.232865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:25:50.260374+0000","last_change":"2023-10-16T11:25:50.260374+0000","last_active":"2023-10-16T11:25:50.260374+0000","last_peered":"2023-10-16T11:25:50.260374+0000","last_clean":"2023-10-16T11:25:50.260374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:25:50.260374+0000","last_undegraded":"2023-10-16T11:25:50.260374+0000","last_fullsized":"2023-10-16T11:25:50.260374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:40.257136+0000","last_change":"2023-10-16T11:25:37.178486+0000","last_active":"2023-10-16T11:25:40.257136+0000","last_peered":"2023-10-16T11:25:40.257136+0000","last_clean":"2023-10-16T11:25:40.257136+0000","last_became_active":"2023-10-16T11:25:37.177077+0000","last_became_peered":"2023-10-16T11:25:37.177077+0000","last_unstale":"2023-10-16T11:25:40.257136+0000","last_undegraded":"2023-10-16T11:25:40.257136+0000","last_fullsized":"2023-10-16T11:25:40.257136+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"236'4","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.617092+0000","last_change":"2023-10-16T11:25:36.174205+0000","last_active":"2023-10-16T11:25:38.617092+0000","last_peered":"2023-10-16T11:25:38.617092+0000","last_clean":"2023-10-16T11:25:38.617092+0000","last_became_active":"2023-10-16T11:25:36.173850+0000","last_became_peered":"2023-10-16T11:25:36.173850+0000","last_unstale":"2023-10-16T11:25:38.617092+0000","last_undegraded":"2023-10-16T11:25:38.617092+0000","last_fullsized":"2023-10-16T11:25:38.617092+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:20.618602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"223'2789","reported_seq":3746,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-16T11:25:38.173478+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:25:38.173478+0000","last_peered":"2023-10-16T11:25:38.173478+0000","last_clean":"2023-10-16T11:25:38.173478+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:25:38.173478+0000","last_undegraded":"2023-10-16T11:25:38.173478+0000","last_fullsized":"2023-10-16T11:25:38.173478+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2789,"log_dups_size":0,"ondisk_log_size":2789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":31,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":30,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2789,"ondisk_log_size":2789,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576001,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52740,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319100,"statfs":{"total":96636764160,"available":96582758400,"internally_reserved":0,"allocated":3596288,"data_stored":2052564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.153,"15min":1.2090000000000001},"min":{"1min":0.71499999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4950000000000001,"5min":1.7549999999999999,"15min":2.3399999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.161,"15min":1.2150000000000001},"min":{"1min":0.623,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.448,"5min":1.6699999999999999,"15min":1.8320000000000001},"last":1.113}]},{"osd":1,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.256,"15min":1.2430000000000001},"min":{"1min":0.84599999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.5029999999999999,"5min":1.913,"15min":1.913},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.171,"5min":1.2210000000000001,"15min":1.264},"min":{"1min":0.67100000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.522,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.0580000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.3,"15min":1.296},"min":{"1min":0.94199999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6879999999999999,"5min":1.8680000000000001,"15min":2.4460000000000002},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.21,"5min":1.3129999999999999,"15min":1.3500000000000001},"min":{"1min":0.94499999999999995,"5min":0.88,"15min":0.88},"max":{"1min":1.5589999999999999,"5min":1.859,"15min":1.974},"last":0.86799999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3520000000000001,"15min":1.3779999999999999},"min":{"1min":0.93700000000000006,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.653,"5min":1.8080000000000001,"15min":1.9099999999999999},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.4370000000000001,"15min":1.454},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.728,"5min":2.0070000000000001,"15min":2.577},"last":1.3460000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4319999999999999,"15min":1.413},"min":{"1min":0.89800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.865,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.171},{"interface":"front","average":{"1min":1.369,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":1.149,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8660000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.417}]},{"osd":5,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.587,"15min":1.591},"min":{"1min":1.1799999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.95,"5min":2.101,"15min":3.3519999999999999},"last":1.488},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.53,"15min":1.508},"min":{"1min":1.2330000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.9870000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.595}]},{"osd":6,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.492,"15min":1.5089999999999999},"min":{"1min":1.1459999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.8140000000000001,"5min":2.101,"15min":2.7250000000000001},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.6040000000000001,"5min":1.6819999999999999,"15min":1.698},"min":{"1min":1.3080000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.0129999999999999,"5min":2.2109999999999999,"15min":3.1440000000000001},"last":1.5660000000000001}]}]},{"osd":6,"up_from":13,"seq":55834576001,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52164,"kb_used_data":2936,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319676,"statfs":{"total":96636764160,"available":96583348224,"internally_reserved":0,"allocated":3006464,"data_stored":1462730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.2130000000000001,"15min":1.21},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.343,"5min":6.298,"15min":6.298},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.204,"5min":1.331,"15min":1.292},"min":{"1min":0.72199999999999998,"5min":0.68200000000000005,"15min":0.66400000000000003},"max":{"1min":1.607,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":1.377}]},{"osd":1,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.224,"15min":1.212},"min":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.5660000000000001,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.421,"15min":1.3779999999999999},"min":{"1min":0.76200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.7310000000000001,"5min":12.468,"15min":12.468},"last":1.1990000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.3140000000000001,"15min":1.2769999999999999},"min":{"1min":0.77800000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.373,"5min":11.586,"15min":11.586},"last":1.248},{"interface":"front","average":{"1min":1.306,"5min":1.3740000000000001,"15min":1.3680000000000001},"min":{"1min":0.84399999999999997,"5min":0.82599999999999996,"15min":0.77500000000000002},"max":{"1min":1.7629999999999999,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.427}]},{"osd":3,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.425,"15min":1.3460000000000001},"min":{"1min":0.84799999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.859,"5min":12.801,"15min":12.801},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2929999999999999,"15min":1.2769999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.657,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":1.1379999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3899999999999999,"15min":1.3280000000000001},"min":{"1min":0.68700000000000006,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.9510000000000001,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.6220000000000001,"15min":1.5509999999999999},"min":{"1min":1.1180000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.069,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.4630000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.5189999999999999,"15min":1.397},"min":{"1min":0.92500000000000004,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.978,"5min":18.247,"15min":18.247},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.595,"15min":1.5620000000000001},"min":{"1min":0.93799999999999994,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.1240000000000001,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.5049999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.627,"15min":1.5469999999999999},"min":{"1min":1.2070000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.383,"5min":12.589,"15min":12.589},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.661,"5min":1.702,"15min":1.603},"min":{"1min":1.0049999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.2970000000000002,"5min":12.725,"15min":12.725},"last":1.3360000000000001}]}]},{"osd":5,"up_from":13,"seq":55834576001,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29948,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341892,"statfs":{"total":96636764160,"available":96606097408,"internally_reserved":0,"allocated":2998272,"data_stored":1462126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2150000000000001,"15min":1.2629999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.51,"5min":2.661,"15min":2.661},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.204,"5min":1.2769999999999999,"15min":1.339},"min":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7210000000000001,"5min":1.7949999999999999,"15min":1.865},"last":1.113}]},{"osd":1,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.145,"15min":1.1890000000000001},"min":{"1min":0.872,"5min":0.81299999999999994,"15min":0.79100000000000004},"max":{"1min":1.583,"5min":1.704,"15min":1.992},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.079,"5min":1.22,"15min":1.222},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6739999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.427}]},{"osd":2,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.26,"15min":1.236},"min":{"1min":0.73499999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.7450000000000001,"5min":1.952,"15min":1.952},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.236,"5min":1.284,"15min":1.3129999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5840000000000001,"5min":1.8109999999999999,"15min":2.1640000000000001},"last":1.1699999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.319,"15min":1.329},"min":{"1min":0.872,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5920000000000001,"5min":3.012,"15min":3.012},"last":1.21},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.401,"15min":1.4139999999999999},"min":{"1min":0.95399999999999996,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.6639999999999999,"5min":1.9119999999999999,"15min":2.2250000000000001},"last":1.034}]},{"osd":4,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3600000000000001,"15min":1.341},"min":{"1min":0.91300000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.887,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.575},{"interface":"front","average":{"1min":1.413,"5min":1.5529999999999999,"15min":1.5660000000000001},"min":{"1min":0.996,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.6499999999999999,"5min":2.903,"15min":2.903},"last":1.3879999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.4119999999999999,"15min":1.411},"min":{"1min":0.93799999999999994,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.837,"5min":1.8660000000000001,"15min":2.0979999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.486,"15min":1.4810000000000001},"min":{"1min":0.93300000000000005,"5min":0.90700000000000003,"15min":0.89200000000000002},"max":{"1min":2.0329999999999999,"5min":3.073,"15min":3.073},"last":1.673}]},{"osd":7,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.5649999999999999,"15min":1.5429999999999999},"min":{"1min":1.159,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.764,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.3},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.625,"15min":1.6399999999999999},"min":{"1min":1.349,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.9970000000000001,"5min":2.1099999999999999,"15min":2.2930000000000001},"last":1.341}]}]},{"osd":4,"up_from":13,"seq":55834576002,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70160,"kb_used_data":2948,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301680,"statfs":{"total":96636764160,"available":96564920320,"internally_reserved":0,"allocated":3018752,"data_stored":1465075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.167,"15min":1.1799999999999999},"min":{"1min":0.69099999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.2370000000000001,"15min":1.2430000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":1.083}]},{"osd":1,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3069999999999999,"15min":1.3009999999999999},"min":{"1min":0.88800000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.252,"15min":1.252},"min":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":3.0739999999999998,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.5960000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3,"15min":1.3109999999999999},"min":{"1min":0.80500000000000005,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":3.2170000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.4359999999999999,"15min":1.4119999999999999},"min":{"1min":0.90300000000000002,"5min":0.879,"15min":0.872},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.423}]},{"osd":3,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.339,"15min":1.341},"min":{"1min":0.84599999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.516},{"interface":"front","average":{"1min":1.496,"5min":1.4550000000000001,"15min":1.4430000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":3.4220000000000002,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.468}]},{"osd":5,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.5249999999999999,"15min":1.4570000000000001},"min":{"1min":1.1339999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.613,"5min":1.46,"15min":1.4319999999999999},"min":{"1min":1.1519999999999999,"5min":0.85199999999999998,"15min":0.78600000000000003},"max":{"1min":3.7989999999999999,"5min":3.7989999999999999,"15min":3.7989999999999999},"last":1.292}]},{"osd":6,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6679999999999999,"5min":1.5649999999999999,"15min":1.5189999999999999},"min":{"1min":0.98599999999999999,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.776,"5min":1.643,"15min":1.6240000000000001},"min":{"1min":1.0269999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":3.5019999999999998,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.8}]},{"osd":7,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.8089999999999999,"5min":1.613,"15min":1.5860000000000001},"min":{"1min":1.3,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":4.2050000000000001,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":1.7569999999999999},{"interface":"front","average":{"1min":1.8460000000000001,"5min":1.702,"15min":1.6599999999999999},"min":{"1min":1.2370000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":3.7000000000000002,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.6850000000000001}]}]},{"osd":3,"up_from":13,"seq":55834576001,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52160,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319680,"statfs":{"total":96636764160,"available":96583352320,"internally_reserved":0,"allocated":3002368,"data_stored":1462708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.85899999999999999,"15min":0.89700000000000002},"min":{"1min":0.40000000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1339999999999999,"5min":1.478,"15min":1.901},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.997,"15min":1.0569999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1850000000000001,"5min":1.6180000000000001,"15min":1.6679999999999999},"last":0.85399999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.95299999999999996,"15min":0.98399999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.3400000000000001,"5min":1.508,"15min":1.508},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0609999999999999,"15min":1.121},"min":{"1min":0.64900000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5429999999999999,"5min":1.601,"15min":1.7190000000000001},"last":0.91100000000000003}]},{"osd":2,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0580000000000001,"15min":1.1120000000000001},"min":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4199999999999999,"5min":1.694,"15min":1.694},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.161,"15min":1.22},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.7350000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.222}]},{"osd":4,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2989999999999999,"15min":1.3480000000000001},"min":{"1min":0.89400000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.724,"5min":1.806,"15min":1.873},"last":0.995},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3300000000000001,"15min":1.359},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.349}]},{"osd":5,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.389,"15min":1.405},"min":{"1min":1.0549999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8260000000000001,"5min":2.798,"15min":2.798},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.395,"15min":1.4410000000000001},"min":{"1min":0.88800000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.883,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.2889999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.488,"15min":1.5329999999999999},"min":{"1min":1.01,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.911,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4790000000000001,"15min":1.534},"min":{"1min":0.86699999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.0329999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.629}]},{"osd":7,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.518,"15min":1.5580000000000001},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.575,"5min":1.613,"15min":1.669},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.3799999999999999,"5min":2.3799999999999999,"15min":2.5129999999999999},"last":1.7230000000000001}]}]},{"osd":2,"up_from":13,"seq":55834576000,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56320,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315520,"statfs":{"total":96636764160,"available":96579092480,"internally_reserved":0,"allocated":3002368,"data_stored":1462216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.89200000000000002,"15min":0.91600000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.349,"5min":1.532,"15min":1.669},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.99299999999999999,"15min":1.069},"min":{"1min":0.56899999999999995,"5min":0.47699999999999998,"15min":0.32600000000000001},"max":{"1min":1.3460000000000001,"5min":1.6359999999999999,"15min":2.1429999999999998},"last":0.95799999999999996}]},{"osd":1,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0760000000000001,"15min":1.089},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.39200000000000002},"max":{"1min":1.591,"5min":1.645,"15min":1.978},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.085,"15min":1.1379999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.393,"5min":1.609,"15min":1.7949999999999999},"last":1.0680000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0429999999999999,"15min":1.0740000000000001},"min":{"1min":0.499,"5min":0.437,"15min":0.437},"max":{"1min":1.522,"5min":1.651,"15min":1.8560000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.137,"5min":1.2170000000000001,"15min":1.2629999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.6930000000000001,"5min":1.7450000000000001,"15min":1.857},"last":1.379}]},{"osd":4,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3340000000000001,"15min":1.3999999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.73499999999999999},"max":{"1min":1.643,"5min":1.8029999999999999,"15min":2.5059999999999998},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.3220000000000001,"15min":1.339},"min":{"1min":0.88200000000000001,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":1.6499999999999999,"5min":1.825,"15min":1.843},"last":1.2230000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.4139999999999999,"15min":1.4530000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.429,"15min":1.4730000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.885,"5min":1.931,"15min":2.6259999999999999},"last":1.5449999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3899999999999999,"15min":1.3999999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.8260000000000001,"5min":1.9019999999999999,"15min":2.1339999999999999},"last":1.095},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4870000000000001,"15min":1.526},"min":{"1min":0.80500000000000005,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.494}]},{"osd":7,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.5109999999999999,"15min":1.569},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.92500000000000004},"max":{"1min":2.0310000000000001,"5min":2.1469999999999998,"15min":2.8109999999999999},"last":1.615},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4750000000000001,"15min":1.502},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80100000000000005},"max":{"1min":1.952,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.3420000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576000,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64656,"kb_used_data":2948,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307184,"statfs":{"total":96636764160,"available":96570556416,"internally_reserved":0,"allocated":3018752,"data_stored":1465107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.93799999999999994,"15min":0.91700000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":5.117,"5min":5.117,"15min":5.117},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.97299999999999998,"15min":1.0449999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.5069999999999999,"5min":1.5509999999999999,"15min":1.667},"last":0.35899999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.97299999999999998,"15min":1.014},"min":{"1min":0.52600000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2350000000000001,"15min":1.27},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":3.5840000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.90300000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.1779999999999999,"15min":1.202},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.4670000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.1439999999999999,"15min":1.1639999999999999},"min":{"1min":0.48099999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5600000000000001,"5min":1.633,"15min":1.802},"last":0.66400000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.3260000000000001,"15min":1.3009999999999999},"min":{"1min":0.80500000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":5.29,"5min":5.29,"15min":5.29},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.4450000000000001,"15min":1.4259999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":5.0179999999999998,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":0.999}]},{"osd":5,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.397,"15min":1.419},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":6.0979999999999999,"5min":6.0979999999999999,"15min":6.0979999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.415,"15min":1.4159999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.5299999999999998,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.2330000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.49,"15min":1.5029999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":4.0499999999999998,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":1.034},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.474,"15min":1.488},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":3.8799999999999999,"5min":3.8799999999999999,"15min":3.8799999999999999},"last":1.2689999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:24:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.577,"15min":1.569},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":4.46,"5min":4.46,"15min":4.46},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.625,"15min":1.613},"min":{"1min":1.0049999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":4.1040000000000001,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":1.135}]}]},{"osd":0,"up_from":13,"seq":55834576001,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52740,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319100,"statfs":{"total":96636764160,"available":96582758400,"internally_reserved":0,"allocated":3596288,"data_stored":2052528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.86099999999999999,"15min":0.88700000000000001},"min":{"1min":0.46800000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2210000000000001,"5min":1.52,"15min":1.619},"last":0.874},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.006,"15min":1.081},"min":{"1min":0.41199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.341,"5min":1.621,"15min":1.76},"last":1.0609999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.075,"15min":1.0800000000000001},"min":{"1min":0.52500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.7349999999999999,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.034,"5min":1.0700000000000001,"15min":1.1299999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":1.583}]},{"osd":3,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.016,"15min":1.0669999999999999},"min":{"1min":0.50600000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1299999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.109},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.1539999999999999,"15min":1.226},"min":{"1min":0.55600000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.3799999999999999,"5min":1.833,"15min":1.9830000000000001},"last":1.2390000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.349,"15min":1.3859999999999999},"min":{"1min":0.78800000000000003,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":3.2799999999999998,"5min":3.2799999999999998,"15min":3.2799999999999998},"last":1.7829999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.292,"15min":1.337},"min":{"1min":0.76100000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.2190000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.357,"15min":1.3720000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0209999999999999,"5min":2.0750000000000002,"15min":2.6669999999999998},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3819999999999999,"15min":1.431},"min":{"1min":0.89800000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.885,"5min":2.2029999999999998,"15min":2.2040000000000002},"last":1.4370000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.4710000000000001,"15min":1.496},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":5.2560000000000002,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":1.53},{"interface":"front","average":{"1min":1.298,"5min":1.429,"15min":1.4790000000000001},"min":{"1min":0.83199999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.804,"5min":2.073,"15min":2.262},"last":1.385}]},{"osd":7,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.704,"5min":1.5429999999999999,"15min":1.5600000000000001},"min":{"1min":0.88700000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.4419999999999999,"15min":1.492},"min":{"1min":0.871,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.623,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.2809999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:25:56.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.1 is not active+clean 2023-10-16T11:25:56.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.0 is not active+clean 2023-10-16T11:25:56.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.273967+0000', 'last_change': '2023-10-16T11:25:49.273967+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.273967+0000', 'last_undegraded': '2023-10-16T11:25:49.273967+0000', 'last_fullsized': '2023-10-16T11:25:49.273967+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-16T11:25:56.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.0 is not active+clean 2023-10-16T11:25:56.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.1 is not active+clean 2023-10-16T11:25:56.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.270260+0000', 'last_change': '2023-10-16T11:25:49.270260+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.270260+0000', 'last_undegraded': '2023-10-16T11:25:49.270260+0000', 'last_fullsized': '2023-10-16T11:25:49.270260+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-16T11:25:56.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.3 is not active+clean 2023-10-16T11:25:56.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.2 is not active+clean 2023-10-16T11:25:56.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.273067+0000', 'last_change': '2023-10-16T11:25:49.273067+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.273067+0000', 'last_undegraded': '2023-10-16T11:25:49.273067+0000', 'last_fullsized': '2023-10-16T11:25:49.273067+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-16T11:25:56.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.2 is not active+clean 2023-10-16T11:25:56.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.3 is not active+clean 2023-10-16T11:25:56.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.271638+0000', 'last_change': '2023-10-16T11:25:49.271638+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.271638+0000', 'last_undegraded': '2023-10-16T11:25:49.271638+0000', 'last_fullsized': '2023-10-16T11:25:49.271638+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-16T11:25:56.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.5 is not active+clean 2023-10-16T11:25:56.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.4 is not active+clean 2023-10-16T11:25:56.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.272120+0000', 'last_change': '2023-10-16T11:25:49.272120+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.272120+0000', 'last_undegraded': '2023-10-16T11:25:49.272120+0000', 'last_fullsized': '2023-10-16T11:25:49.272120+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-16T11:25:56.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.4 is not active+clean 2023-10-16T11:25:56.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.5 is not active+clean 2023-10-16T11:25:56.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.267494+0000', 'last_change': '2023-10-16T11:25:49.267494+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.267494+0000', 'last_undegraded': '2023-10-16T11:25:49.267494+0000', 'last_fullsized': '2023-10-16T11:25:49.267494+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-16T11:25:56.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.7 is not active+clean 2023-10-16T11:25:56.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.6 is not active+clean 2023-10-16T11:25:56.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.279696+0000', 'last_change': '2023-10-16T11:25:49.279696+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.279696+0000', 'last_undegraded': '2023-10-16T11:25:49.279696+0000', 'last_fullsized': '2023-10-16T11:25:49.279696+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-16T11:25:56.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.6 is not active+clean 2023-10-16T11:25:56.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.7 is not active+clean 2023-10-16T11:25:56.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.269438+0000', 'last_change': '2023-10-16T11:25:49.269438+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.269438+0000', 'last_undegraded': '2023-10-16T11:25:49.269438+0000', 'last_fullsized': '2023-10-16T11:25:49.269438+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-16T11:25:56.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.9 is not active+clean 2023-10-16T11:25:56.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.8 is not active+clean 2023-10-16T11:25:56.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.273300+0000', 'last_change': '2023-10-16T11:25:49.273300+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.273300+0000', 'last_undegraded': '2023-10-16T11:25:49.273300+0000', 'last_fullsized': '2023-10-16T11:25:49.273300+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-16T11:25:56.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.8 is not active+clean 2023-10-16T11:25:56.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.9 is not active+clean 2023-10-16T11:25:56.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.283150+0000', 'last_change': '2023-10-16T11:25:49.283150+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.283150+0000', 'last_undegraded': '2023-10-16T11:25:49.283150+0000', 'last_fullsized': '2023-10-16T11:25:49.283150+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-16T11:25:56.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.b is not active+clean 2023-10-16T11:25:56.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.a is not active+clean 2023-10-16T11:25:56.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.274776+0000', 'last_change': '2023-10-16T11:25:49.274776+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.274776+0000', 'last_undegraded': '2023-10-16T11:25:49.274776+0000', 'last_fullsized': '2023-10-16T11:25:49.274776+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-16T11:25:56.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.a is not active+clean 2023-10-16T11:25:56.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.b is not active+clean 2023-10-16T11:25:56.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.277979+0000', 'last_change': '2023-10-16T11:25:49.277979+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.277979+0000', 'last_undegraded': '2023-10-16T11:25:49.277979+0000', 'last_fullsized': '2023-10-16T11:25:49.277979+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-16T11:25:56.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.d is not active+clean 2023-10-16T11:25:56.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.c is not active+clean 2023-10-16T11:25:56.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.269375+0000', 'last_change': '2023-10-16T11:25:49.269375+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.269375+0000', 'last_undegraded': '2023-10-16T11:25:49.269375+0000', 'last_fullsized': '2023-10-16T11:25:49.269375+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-16T11:25:56.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.c is not active+clean 2023-10-16T11:25:56.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.d is not active+clean 2023-10-16T11:25:56.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.283709+0000', 'last_change': '2023-10-16T11:25:49.283709+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.283709+0000', 'last_undegraded': '2023-10-16T11:25:49.283709+0000', 'last_fullsized': '2023-10-16T11:25:49.283709+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-16T11:25:56.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.f is not active+clean 2023-10-16T11:25:56.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.e is not active+clean 2023-10-16T11:25:56.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.266505+0000', 'last_change': '2023-10-16T11:25:49.266505+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.266505+0000', 'last_undegraded': '2023-10-16T11:25:49.266505+0000', 'last_fullsized': '2023-10-16T11:25:49.266505+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-16T11:25:56.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.e is not active+clean 2023-10-16T11:25:56.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.f is not active+clean 2023-10-16T11:25:56.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.274689+0000', 'last_change': '2023-10-16T11:25:49.274689+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.274689+0000', 'last_undegraded': '2023-10-16T11:25:49.274689+0000', 'last_fullsized': '2023-10-16T11:25:49.274689+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-16T11:25:56.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.11 is not active+clean 2023-10-16T11:25:56.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.10 is not active+clean 2023-10-16T11:25:56.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.267011+0000', 'last_change': '2023-10-16T11:25:49.267011+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.267011+0000', 'last_undegraded': '2023-10-16T11:25:49.267011+0000', 'last_fullsized': '2023-10-16T11:25:49.267011+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-16T11:25:56.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.10 is not active+clean 2023-10-16T11:25:56.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.11 is not active+clean 2023-10-16T11:25:56.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.279612+0000', 'last_change': '2023-10-16T11:25:49.279612+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.279612+0000', 'last_undegraded': '2023-10-16T11:25:49.279612+0000', 'last_fullsized': '2023-10-16T11:25:49.279612+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-16T11:25:56.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.13 is not active+clean 2023-10-16T11:25:56.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.12 is not active+clean 2023-10-16T11:25:56.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.271754+0000', 'last_change': '2023-10-16T11:25:49.271754+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.271754+0000', 'last_undegraded': '2023-10-16T11:25:49.271754+0000', 'last_fullsized': '2023-10-16T11:25:49.271754+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-16T11:25:56.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.12 is not active+clean 2023-10-16T11:25:56.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.13 is not active+clean 2023-10-16T11:25:56.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.274305+0000', 'last_change': '2023-10-16T11:25:49.274305+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.274305+0000', 'last_undegraded': '2023-10-16T11:25:49.274305+0000', 'last_fullsized': '2023-10-16T11:25:49.274305+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-16T11:25:56.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.15 is not active+clean 2023-10-16T11:25:56.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.14 is not active+clean 2023-10-16T11:25:56.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.272751+0000', 'last_change': '2023-10-16T11:25:49.272751+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.272751+0000', 'last_undegraded': '2023-10-16T11:25:49.272751+0000', 'last_fullsized': '2023-10-16T11:25:49.272751+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-16T11:25:56.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.14 is not active+clean 2023-10-16T11:25:56.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.15 is not active+clean 2023-10-16T11:25:56.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.266977+0000', 'last_change': '2023-10-16T11:25:49.266977+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.266977+0000', 'last_undegraded': '2023-10-16T11:25:49.266977+0000', 'last_fullsized': '2023-10-16T11:25:49.266977+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-16T11:25:56.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.17 is not active+clean 2023-10-16T11:25:56.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.16 is not active+clean 2023-10-16T11:25:56.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.267347+0000', 'last_change': '2023-10-16T11:25:49.267347+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.267347+0000', 'last_undegraded': '2023-10-16T11:25:49.267347+0000', 'last_fullsized': '2023-10-16T11:25:49.267347+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-16T11:25:56.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.16 is not active+clean 2023-10-16T11:25:56.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.17 is not active+clean 2023-10-16T11:25:56.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.272728+0000', 'last_change': '2023-10-16T11:25:49.272728+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.272728+0000', 'last_undegraded': '2023-10-16T11:25:49.272728+0000', 'last_fullsized': '2023-10-16T11:25:49.272728+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-16T11:25:56.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.19 is not active+clean 2023-10-16T11:25:56.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.18 is not active+clean 2023-10-16T11:25:56.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.267168+0000', 'last_change': '2023-10-16T11:25:49.267168+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.267168+0000', 'last_undegraded': '2023-10-16T11:25:49.267168+0000', 'last_fullsized': '2023-10-16T11:25:49.267168+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-16T11:25:56.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.18 is not active+clean 2023-10-16T11:25:56.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.19 is not active+clean 2023-10-16T11:25:56.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.279039+0000', 'last_change': '2023-10-16T11:25:49.279039+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.279039+0000', 'last_undegraded': '2023-10-16T11:25:49.279039+0000', 'last_fullsized': '2023-10-16T11:25:49.279039+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-16T11:25:56.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.1b is not active+clean 2023-10-16T11:25:56.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.1a is not active+clean 2023-10-16T11:25:56.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.272764+0000', 'last_change': '2023-10-16T11:25:49.272764+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.272764+0000', 'last_undegraded': '2023-10-16T11:25:49.272764+0000', 'last_fullsized': '2023-10-16T11:25:49.272764+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-16T11:25:56.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.1a is not active+clean 2023-10-16T11:25:56.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.1b is not active+clean 2023-10-16T11:25:56.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.267216+0000', 'last_change': '2023-10-16T11:25:49.267216+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.267216+0000', 'last_undegraded': '2023-10-16T11:25:49.267216+0000', 'last_fullsized': '2023-10-16T11:25:49.267216+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-16T11:25:56.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.1d is not active+clean 2023-10-16T11:25:56.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.1c is not active+clean 2023-10-16T11:25:56.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.266326+0000', 'last_change': '2023-10-16T11:25:49.266326+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.266326+0000', 'last_undegraded': '2023-10-16T11:25:49.266326+0000', 'last_fullsized': '2023-10-16T11:25:49.266326+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-16T11:25:56.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.1c is not active+clean 2023-10-16T11:25:56.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.1d is not active+clean 2023-10-16T11:25:56.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.267625+0000', 'last_change': '2023-10-16T11:25:49.267625+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.267625+0000', 'last_undegraded': '2023-10-16T11:25:49.267625+0000', 'last_fullsized': '2023-10-16T11:25:49.267625+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-16T11:25:56.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.1f is not active+clean 2023-10-16T11:25:56.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.1e is not active+clean 2023-10-16T11:25:56.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.269415+0000', 'last_change': '2023-10-16T11:25:49.269415+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.269415+0000', 'last_undegraded': '2023-10-16T11:25:49.269415+0000', 'last_fullsized': '2023-10-16T11:25:49.269415+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-16T11:25:56.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.1e is not active+clean 2023-10-16T11:25:56.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 32.1f is not active+clean 2023-10-16T11:25:56.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '32.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 238, 'state': 'creating+peering', 'last_fresh': '2023-10-16T11:25:49.273227+0000', 'last_change': '2023-10-16T11:25:49.273227+0000', 'last_active': '2023-10-16T11:25:49.243481+0000', 'last_peered': '2023-10-16T11:25:49.243481+0000', 'last_clean': '2023-10-16T11:25:49.243481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:49.273227+0000', 'last_undegraded': '2023-10-16T11:25:49.273227+0000', 'last_fullsized': '2023-10-16T11:25:49.273227+0000', 'mapping_epoch': 238, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 238, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:49.243481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-16T11:25:56.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.21 is not active+clean 2023-10-16T11:25:56.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.22 is not active+clean 2023-10-16T11:25:56.913 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.913 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.23 is not active+clean 2023-10-16T11:25:56.914 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.914 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.24 is not active+clean 2023-10-16T11:25:56.914 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.915 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.25 is not active+clean 2023-10-16T11:25:56.915 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.26 is not active+clean 2023-10-16T11:25:56.916 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.27 is not active+clean 2023-10-16T11:25:56.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.28 is not active+clean 2023-10-16T11:25:56.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.29 is not active+clean 2023-10-16T11:25:56.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.2a is not active+clean 2023-10-16T11:25:56.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.2b is not active+clean 2023-10-16T11:25:56.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.2c is not active+clean 2023-10-16T11:25:56.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.2d is not active+clean 2023-10-16T11:25:56.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.2e is not active+clean 2023-10-16T11:25:56.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.2f is not active+clean 2023-10-16T11:25:56.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.30 is not active+clean 2023-10-16T11:25:56.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.31 is not active+clean 2023-10-16T11:25:56.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.32 is not active+clean 2023-10-16T11:25:56.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.33 is not active+clean 2023-10-16T11:25:56.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.927 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.34 is not active+clean 2023-10-16T11:25:56.928 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.928 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.35 is not active+clean 2023-10-16T11:25:56.928 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.929 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.36 is not active+clean 2023-10-16T11:25:56.929 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.929 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.37 is not active+clean 2023-10-16T11:25:56.930 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.930 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.38 is not active+clean 2023-10-16T11:25:56.931 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.931 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.39 is not active+clean 2023-10-16T11:25:56.931 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.932 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.3a is not active+clean 2023-10-16T11:25:56.932 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.933 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.3b is not active+clean 2023-10-16T11:25:56.933 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.933 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.3c is not active+clean 2023-10-16T11:25:56.934 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.934 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.3d is not active+clean 2023-10-16T11:25:56.935 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.935 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.3e is not active+clean 2023-10-16T11:25:56.935 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.936 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.3f is not active+clean 2023-10-16T11:25:56.936 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.937 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.20 is not active+clean 2023-10-16T11:25:56.937 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T11:25:50.260374+0000', 'last_change': '2023-10-16T11:25:50.260374+0000', 'last_active': '2023-10-16T11:25:50.260374+0000', 'last_peered': '2023-10-16T11:25:50.260374+0000', 'last_clean': '2023-10-16T11:25:50.260374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T11:25:50.260374+0000', 'last_undegraded': '2023-10-16T11:25:50.260374+0000', 'last_fullsized': '2023-10-16T11:25:50.260374+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'last_clean_scrub_stamp': '2023-10-16T11:25:50.260374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T11:25:56.938 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-16T11:25:57.105 INFO:tasks.ceph.mds.c.smithi084.stderr:2023-10-16T11:25:57.103+0000 7ff2e8d30640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:25:57.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.158+0000 7f12aa89f640 1 Processor -- start 2023-10-16T11:25:57.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.158+0000 7f12aa89f640 1 -- start start 2023-10-16T11:25:57.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.159+0000 7f12aa89f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12a415ccd0 0x7f12a40c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:57.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.159+0000 7f12aa89f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a40c9700 con 0x7f12a40c92b0 2023-10-16T11:25:57.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.159+0000 7f12aa89f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a40cce00 con 0x7f12a406cd80 2023-10-16T11:25:57.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.159+0000 7f12aa89f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40cd0e0 con 0x7f12a415ccd0 2023-10-16T11:25:57.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.160+0000 7f12aa09e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f12a40c92b0 0x7f12a406b2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40544/0 (socket says 172.21.15.84:40544) 2023-10-16T11:25:57.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.160+0000 7f12aa09e640 1 -- 172.21.15.84:0/3870722781 learned_addr learned my addr 172.21.15.84:0/3870722781 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:57.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.160+0000 7f12a909c640 1 --2- 172.21.15.84:0/3870722781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12a415ccd0 0x7f12a40c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:57.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.160+0000 7f12a889b640 1 -- 172.21.15.84:0/3870722781 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 114378005 0 0) 0x7f129c001490 con 0x7f12a40c92b0 2023-10-16T11:25:57.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.160+0000 7f12a889b640 1 -- 172.21.15.84:0/3870722781 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1290003440 con 0x7f12a40c92b0 2023-10-16T11:25:57.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.161+0000 7f12a889b640 1 -- 172.21.15.84:0/3870722781 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 747655561 0 0) 0x7f1290003440 con 0x7f12a40c92b0 2023-10-16T11:25:57.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.161+0000 7f12a889b640 1 -- 172.21.15.84:0/3870722781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12a415ccd0 msgr2=0x7f12a40c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:57.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.161+0000 7f12a889b640 1 --2- 172.21.15.84:0/3870722781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12a415ccd0 0x7f12a40c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.161+0000 7f12a889b640 1 -- 172.21.15.84:0/3870722781 >> v1:172.21.15.157:6789/0 conn(0x7f12a406cd80 legacy=0x7f12a40cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:57.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.161+0000 7f12a889b640 1 -- 172.21.15.84:0/3870722781 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a40cd3c0 con 0x7f12a40c92b0 2023-10-16T11:25:57.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.161+0000 7f12a889b640 1 -- 172.21.15.84:0/3870722781 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f129c001b80 con 0x7f12a40c92b0 2023-10-16T11:25:57.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.161+0000 7f12a889b640 1 -- 172.21.15.84:0/3870722781 >> v1:172.21.15.84:6789/0 conn(0x7f12a40c92b0 legacy=0x7f12a406b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:57.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.161+0000 7f12a889b640 1 --2- 172.21.15.84:0/3870722781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1290003a90 0x7f1290003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:57.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.161+0000 7f12a889b640 1 --2- 172.21.15.84:0/3870722781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1290008850 0x7f129000acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:57.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.161+0000 7f12a889b640 1 --2- 172.21.15.84:0/3870722781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12a415ccd0 0x7f129000d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:57.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.161+0000 7f12a889b640 1 -- 172.21.15.84:0/3870722781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40cd0e0 con 0x7f1290003a90 2023-10-16T11:25:57.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.161+0000 7f12a889b640 1 -- 172.21.15.84:0/3870722781 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129000dc70 con 0x7f1290008850 2023-10-16T11:25:57.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.161+0000 7f12a889b640 1 -- 172.21.15.84:0/3870722781 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129000de30 con 0x7f12a415ccd0 2023-10-16T11:25:57.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.161+0000 7f12a989d640 1 --2- 172.21.15.84:0/3870722781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1290003a90 0x7f1290003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:57.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.162+0000 7f12a909c640 1 --2- 172.21.15.84:0/3870722781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1290008850 0x7f129000acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:57.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.162+0000 7f12a989d640 1 -- 172.21.15.84:0/3870722781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12a415ccd0 msgr2=0x7f129000d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:57.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.162+0000 7f12a989d640 1 --2- 172.21.15.84:0/3870722781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12a415ccd0 0x7f129000d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.162+0000 7f12a989d640 1 -- 172.21.15.84:0/3870722781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1290008850 msgr2=0x7f129000acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:57.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.162+0000 7f12a989d640 1 --2- 172.21.15.84:0/3870722781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1290008850 0x7f129000acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.162+0000 7f12a989d640 1 -- 172.21.15.84:0/3870722781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1290003440 con 0x7f1290003a90 2023-10-16T11:25:57.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.163+0000 7f12a989d640 1 --2- 172.21.15.84:0/3870722781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1290003a90 0x7f1290003eb0 secure :-1 s=READY pgs=3000 cs=0 l=1 rev1=1 crypto rx=0x7f12a0001750 tx=0x7f12900036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b24354f1fe4c4e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:57.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.163+0000 7f12a889b640 1 -- 172.21.15.84:0/3870722781 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a00052d0 con 0x7f1290003a90 2023-10-16T11:25:57.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.163+0000 7f12a889b640 1 -- 172.21.15.84:0/3870722781 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a002e050 con 0x7f1290003a90 2023-10-16T11:25:57.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.163+0000 7f12a889b640 1 -- 172.21.15.84:0/3870722781 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a00035f0 con 0x7f1290003a90 2023-10-16T11:25:57.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.164+0000 7f12aa89f640 1 -- 172.21.15.84:0/3870722781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1290003a90 msgr2=0x7f1290003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:57.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.164+0000 7f12aa89f640 1 --2- 172.21.15.84:0/3870722781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1290003a90 0x7f1290003eb0 secure :-1 s=READY pgs=3000 cs=0 l=1 rev1=1 crypto rx=0x7f12a0001750 tx=0x7f12900036f0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.164+0000 7f12aa09e640 1 -- 172.21.15.84:0/3870722781 reap_dead start 2023-10-16T11:25:57.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.164+0000 7f12aa89f640 1 -- 172.21.15.84:0/3870722781 shutdown_connections 2023-10-16T11:25:57.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.164+0000 7f12aa89f640 1 -- 172.21.15.84:0/3870722781 >> 172.21.15.84:0/3870722781 conn(0x7f12a40cb1a0 msgr2=0x7f12a40cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:57.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.164+0000 7f12aa89f640 1 -- 172.21.15.84:0/3870722781 shutdown_connections 2023-10-16T11:25:57.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.164+0000 7f12aa89f640 1 -- 172.21.15.84:0/3870722781 wait complete. 2023-10-16T11:25:57.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.166+0000 7f12aa89f640 1 Processor -- start 2023-10-16T11:25:57.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.166+0000 7f12aa89f640 1 -- start start 2023-10-16T11:25:57.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.167+0000 7f12aa89f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1290003a90 0x7f12a40b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:57.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.167+0000 7f12aa89f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12a41529e0 0x7f12a4152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:57.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.167+0000 7f12aa89f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12a40c2250 0x7f12a40c26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:57.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.167+0000 7f12aa89f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40b3970 con 0x7f12a41529e0 2023-10-16T11:25:57.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.167+0000 7f12aa89f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40c5f70 con 0x7f1290003a90 2023-10-16T11:25:57.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.167+0000 7f12aa89f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40b3690 con 0x7f12a40c2250 2023-10-16T11:25:57.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.167+0000 7f12a909c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12a41529e0 0x7f12a4152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:57.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.167+0000 7f12a909c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12a41529e0 0x7f12a4152240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40532/0 (socket says 172.21.15.84:40532) 2023-10-16T11:25:57.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.167+0000 7f12a909c640 1 -- 172.21.15.84:0/3813765148 learned_addr learned my addr 172.21.15.84:0/3813765148 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:57.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.167+0000 7f12aa09e640 1 --2- 172.21.15.84:0/3813765148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12a40c2250 0x7f12a40c26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:57.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.167+0000 7f12a909c640 1 -- 172.21.15.84:0/3813765148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12a40c2250 msgr2=0x7f12a40c26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:57.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.167+0000 7f12a989d640 1 --2- 172.21.15.84:0/3813765148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1290003a90 0x7f12a40b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:57.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.167+0000 7f12a909c640 1 --2- 172.21.15.84:0/3813765148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12a40c2250 0x7f12a40c26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.167+0000 7f12a909c640 1 -- 172.21.15.84:0/3813765148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1290003a90 msgr2=0x7f12a40b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:57.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.167+0000 7f12a909c640 1 --2- 172.21.15.84:0/3813765148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1290003a90 0x7f12a40b30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.168+0000 7f12a909c640 1 -- 172.21.15.84:0/3813765148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a40c5920 con 0x7f12a41529e0 2023-10-16T11:25:57.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.168+0000 7f12a909c640 1 --2- 172.21.15.84:0/3813765148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12a41529e0 0x7f12a4152240 secure :-1 s=READY pgs=3001 cs=0 l=1 rev1=1 crypto rx=0x7f12940108c0 tx=0x7f1294010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:57.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.168+0000 7f129a7fc640 1 -- 172.21.15.84:0/3813765148 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12940095b0 con 0x7f12a41529e0 2023-10-16T11:25:57.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.168+0000 7f129a7fc640 1 -- 172.21.15.84:0/3813765148 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129400eb80 con 0x7f12a41529e0 2023-10-16T11:25:57.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.168+0000 7f129a7fc640 1 -- 172.21.15.84:0/3813765148 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129401c400 con 0x7f12a41529e0 2023-10-16T11:25:57.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.168+0000 7f12aa89f640 1 -- 172.21.15.84:0/3813765148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a40c5c00 con 0x7f12a41529e0 2023-10-16T11:25:57.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.168+0000 7f12aa89f640 1 -- 172.21.15.84:0/3813765148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a4153160 con 0x7f12a41529e0 2023-10-16T11:25:57.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.169+0000 7f12aa89f640 1 -- 172.21.15.84:0/3813765148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a40c6100 con 0x7f12a41529e0 2023-10-16T11:25:57.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.170+0000 7f129a7fc640 1 -- 172.21.15.84:0/3813765148 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f12940143f0 con 0x7f12a41529e0 2023-10-16T11:25:57.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.171+0000 7f129a7fc640 1 --2- 172.21.15.84:0/3813765148 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f128806b0b0 0x7f128806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:57.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.171+0000 7f129a7fc640 1 -- 172.21.15.84:0/3813765148 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f1294089e20 con 0x7f12a41529e0 2023-10-16T11:25:57.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.172+0000 7f12a989d640 1 --2- 172.21.15.84:0/3813765148 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f128806b0b0 0x7f128806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:57.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.172+0000 7f12a989d640 1 --2- 172.21.15.84:0/3813765148 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f128806b0b0 0x7f128806d570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f12a0033000 tx=0x7f12a0009030 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:57.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.176+0000 7f129a7fc640 1 -- 172.21.15.84:0/3813765148 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1294054950 con 0x7f12a41529e0 2023-10-16T11:25:57.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.434+0000 7f12aa89f640 1 -- 172.21.15.84:0/3813765148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f12a404e160 con 0x7f12a41529e0 2023-10-16T11:25:57.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.436+0000 7f129a7fc640 1 -- 172.21.15.84:0/3813765148 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3480 (secure 0 0 0) 0x7f1294021db0 con 0x7f12a41529e0 2023-10-16T11:25:57.438 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:57.438 INFO:teuthology.orchestra.run.smithi084.stdout:{"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":5834,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":241,"num_osds":8,"num_up_osds":8,"osd_up_since":1697449734,"num_in_osds":8,"osd_in_since":1697449726,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97},{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32}],"num_pgs":193,"num_pools":5,"num_objects":24,"data_bytes":594023,"bytes_used":441229312,"bytes_avail":772652883968,"bytes_total":773094113280,"unknown_pgs_ratio":0.33160620927810669,"inactive_pgs_ratio":0.16580310463905334,"write_bytes_sec":171,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":137,"id":15,"up":1,"in":1,"max":1,"id":16,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":16,"rank":0,"name":"c","status":"up:active","gid":12183},{"filesystem_id":15,"rank":0,"name":"a","status":"up:active","gid":12186}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2023-10-16T11:25:23.239106+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-16T11:25:57.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.438+0000 7f12aa89f640 1 -- 172.21.15.84:0/3813765148 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f128806b0b0 msgr2=0x7f128806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:57.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.438+0000 7f12aa89f640 1 --2- 172.21.15.84:0/3813765148 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f128806b0b0 0x7f128806d570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f12a0033000 tx=0x7f12a0009030 comp rx=0 tx=0).stop 2023-10-16T11:25:57.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.438+0000 7f12aa89f640 1 -- 172.21.15.84:0/3813765148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12a41529e0 msgr2=0x7f12a4152240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:57.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.438+0000 7f12aa89f640 1 --2- 172.21.15.84:0/3813765148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12a41529e0 0x7f12a4152240 secure :-1 s=READY pgs=3001 cs=0 l=1 rev1=1 crypto rx=0x7f12940108c0 tx=0x7f1294010d80 comp rx=0 tx=0).stop 2023-10-16T11:25:57.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.438+0000 7f12aa89f640 1 -- 172.21.15.84:0/3813765148 shutdown_connections 2023-10-16T11:25:57.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.438+0000 7f12aa89f640 1 --2- 172.21.15.84:0/3813765148 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f128806b0b0 0x7f128806d570 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.438+0000 7f12aa89f640 1 --2- 172.21.15.84:0/3813765148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12a40c2250 0x7f12a40c26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.438+0000 7f12aa89f640 1 --2- 172.21.15.84:0/3813765148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12a41529e0 0x7f12a4152240 unknown :-1 s=CLOSED pgs=3001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.438+0000 7f12aa89f640 1 --2- 172.21.15.84:0/3813765148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1290003a90 0x7f12a40b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.438+0000 7f12aa89f640 1 -- 172.21.15.84:0/3813765148 >> 172.21.15.84:0/3813765148 conn(0x7f12a40cb1a0 msgr2=0x7f12a415bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:57.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.438+0000 7f12aa89f640 1 -- 172.21.15.84:0/3813765148 shutdown_connections 2023-10-16T11:25:57.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.438+0000 7f12aa89f640 1 -- 172.21.15.84:0/3813765148 wait complete. 2023-10-16T11:25:57.459 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-16T11:25:57.459 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:25:57.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.662+0000 7fb73f341640 1 Processor -- start 2023-10-16T11:25:57.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.663+0000 7fb73f341640 1 -- start start 2023-10-16T11:25:57.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.664+0000 7fb73f341640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb73806c3f0 0x7fb73806c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:57.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.664+0000 7fb73f341640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb73815e0c0 0x7fb7380b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:57.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.664+0000 7fb73f341640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7380cd630 con 0x7fb73815e540 2023-10-16T11:25:57.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.664+0000 7fb73f341640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7380ce1b0 con 0x7fb73815e0c0 2023-10-16T11:25:57.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.664+0000 7fb73f341640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb73806cd10 con 0x7fb73806c3f0 2023-10-16T11:25:57.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.664+0000 7fb73c8dc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb73815e0c0 0x7fb7380b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:57.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.664+0000 7fb73d8de640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb73815e540 0x7fb7380b1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40558/0 (socket says 172.21.15.84:40558) 2023-10-16T11:25:57.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.664+0000 7fb73d8de640 1 -- 172.21.15.84:0/455154478 learned_addr learned my addr 172.21.15.84:0/455154478 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:25:57.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.665+0000 7fb73d0dd640 1 --2- 172.21.15.84:0/455154478 >> v2:172.21.15.157:3300/0 conn(0x7fb73806c3f0 0x7fb73806c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:57.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.665+0000 7fb73c8dc640 1 -- 172.21.15.84:0/455154478 >> v2:172.21.15.157:3300/0 conn(0x7fb73806c3f0 msgr2=0x7fb73806c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:25:57.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.665+0000 7fb73c8dc640 1 --2- 172.21.15.84:0/455154478 >> v2:172.21.15.157:3300/0 conn(0x7fb73806c3f0 0x7fb73806c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.665+0000 7fb73c8dc640 1 -- 172.21.15.84:0/455154478 >> v1:172.21.15.84:6789/0 conn(0x7fb73815e540 legacy=0x7fb7380b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:57.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.665+0000 7fb73c8dc640 1 -- 172.21.15.84:0/455154478 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7380d2170 con 0x7fb73815e0c0 2023-10-16T11:25:57.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.666+0000 7fb73c8dc640 1 --2- 172.21.15.84:0/455154478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73815e0c0 0x7fb7380b4730 secure :-1 s=READY pgs=3003 cs=0 l=1 rev1=1 crypto rx=0x7fb7380d3d00 tx=0x7fb72802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547900d35667d4e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:57.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.666+0000 7fb72ffff640 1 -- 172.21.15.84:0/455154478 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728031830 con 0x7fb73815e0c0 2023-10-16T11:25:57.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.666+0000 7fb72ffff640 1 -- 172.21.15.84:0/455154478 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb728031990 con 0x7fb73815e0c0 2023-10-16T11:25:57.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.666+0000 7fb72ffff640 1 -- 172.21.15.84:0/455154478 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728031d20 con 0x7fb73815e0c0 2023-10-16T11:25:57.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.667+0000 7fb73f341640 1 -- 172.21.15.84:0/455154478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73815e0c0 msgr2=0x7fb7380b4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:57.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.667+0000 7fb73f341640 1 --2- 172.21.15.84:0/455154478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73815e0c0 0x7fb7380b4730 secure :-1 s=READY pgs=3003 cs=0 l=1 rev1=1 crypto rx=0x7fb7380d3d00 tx=0x7fb72802b8d0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.667+0000 7fb73f341640 1 -- 172.21.15.84:0/455154478 shutdown_connections 2023-10-16T11:25:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.667+0000 7fb73f341640 1 --2- 172.21.15.84:0/455154478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73815e0c0 0x7fb7380b4730 unknown :-1 s=CLOSED pgs=3003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.667+0000 7fb73f341640 1 --2- 172.21.15.84:0/455154478 >> v2:172.21.15.157:3300/0 conn(0x7fb73806c3f0 0x7fb73806c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.667+0000 7fb73f341640 1 -- 172.21.15.84:0/455154478 >> 172.21.15.84:0/455154478 conn(0x7fb7380d2cb0 msgr2=0x7fb7380d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.667+0000 7fb73f341640 1 -- 172.21.15.84:0/455154478 shutdown_connections 2023-10-16T11:25:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.667+0000 7fb73f341640 1 -- 172.21.15.84:0/455154478 wait complete. 2023-10-16T11:25:57.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.669+0000 7fb73f341640 1 Processor -- start 2023-10-16T11:25:57.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.669+0000 7fb73f341640 1 -- start start 2023-10-16T11:25:57.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.670+0000 7fb73f341640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73806c3f0 0x7fb738159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:57.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.671+0000 7fb73f341640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb73815e0c0 0x7fb738155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:57.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.671+0000 7fb73f341640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb73815e540 0x7fb7380b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:57.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.671+0000 7fb73f341640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb738159a80 con 0x7fb73806c3f0 2023-10-16T11:25:57.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.671+0000 7fb73f341640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7381584a0 con 0x7fb73815e0c0 2023-10-16T11:25:57.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.671+0000 7fb73f341640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb738158780 con 0x7fb73815e540 2023-10-16T11:25:57.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.671+0000 7fb73d0dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73806c3f0 0x7fb738159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:57.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.671+0000 7fb73d0dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73806c3f0 0x7fb738159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40544/0 (socket says 172.21.15.84:40544) 2023-10-16T11:25:57.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.671+0000 7fb73d0dd640 1 -- 172.21.15.84:0/423600318 learned_addr learned my addr 172.21.15.84:0/423600318 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:25:57.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.671+0000 7fb73d0dd640 1 -- 172.21.15.84:0/423600318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb73815e540 msgr2=0x7fb7380b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:57.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.671+0000 7fb73d0dd640 1 --2- 172.21.15.84:0/423600318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb73815e540 0x7fb7380b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.671+0000 7fb73d0dd640 1 -- 172.21.15.84:0/423600318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb73815e0c0 msgr2=0x7fb738155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:57.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.671+0000 7fb73d0dd640 1 --2- 172.21.15.84:0/423600318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb73815e0c0 0x7fb738155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.671+0000 7fb73d0dd640 1 -- 172.21.15.84:0/423600318 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb728009400 con 0x7fb73806c3f0 2023-10-16T11:25:57.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.672+0000 7fb73d0dd640 1 --2- 172.21.15.84:0/423600318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73806c3f0 0x7fb738159540 secure :-1 s=READY pgs=3004 cs=0 l=1 rev1=1 crypto rx=0x7fb73400d5f0 tx=0x7fb73400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:57.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.672+0000 7fb72dffb640 1 -- 172.21.15.84:0/423600318 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7340125f0 con 0x7fb73806c3f0 2023-10-16T11:25:57.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.672+0000 7fb72dffb640 1 -- 172.21.15.84:0/423600318 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb734012750 con 0x7fb73806c3f0 2023-10-16T11:25:57.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.672+0000 7fb72dffb640 1 -- 172.21.15.84:0/423600318 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734012ae0 con 0x7fb73806c3f0 2023-10-16T11:25:57.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.672+0000 7fb73f341640 1 -- 172.21.15.84:0/423600318 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb738158a60 con 0x7fb73806c3f0 2023-10-16T11:25:57.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.672+0000 7fb73f341640 1 -- 172.21.15.84:0/423600318 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb738159020 con 0x7fb73806c3f0 2023-10-16T11:25:57.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.673+0000 7fb73f341640 1 -- 172.21.15.84:0/423600318 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73804e160 con 0x7fb73806c3f0 2023-10-16T11:25:57.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.674+0000 7fb72dffb640 1 -- 172.21.15.84:0/423600318 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fb734012c40 con 0x7fb73806c3f0 2023-10-16T11:25:57.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.675+0000 7fb72dffb640 1 --2- 172.21.15.84:0/423600318 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fb71c06b0b0 0x7fb71c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:25:57.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.675+0000 7fb72dffb640 1 -- 172.21.15.84:0/423600318 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7fb73408a350 con 0x7fb73806c3f0 2023-10-16T11:25:57.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.675+0000 7fb73c8dc640 1 --2- 172.21.15.84:0/423600318 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fb71c06b0b0 0x7fb71c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:25:57.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.676+0000 7fb73c8dc640 1 --2- 172.21.15.84:0/423600318 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fb71c06b0b0 0x7fb71c06d570 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb728036000 tx=0x7fb728030000 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:25:57.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.679+0000 7fb72dffb640 1 -- 172.21.15.84:0/423600318 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb734054c80 con 0x7fb73806c3f0 2023-10-16T11:25:57.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.865+0000 7fb73f341640 1 -- 172.21.15.84:0/423600318 --> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb7380cdb60 con 0x7fb71c06b0b0 2023-10-16T11:25:57.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.906+0000 7fb72dffb640 1 -- 172.21.15.84:0/423600318 <== mgr.11814 v2:172.21.15.84:6832/2963529201 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+526322 (secure 0 0 0) 0x7fb72803b430 con 0x7fb71c06b0b0 2023-10-16T11:25:57.910 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:25:57.918 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:25:57.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.917+0000 7fb73f341640 1 -- 172.21.15.84:0/423600318 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fb71c06b0b0 msgr2=0x7fb71c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:57.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.917+0000 7fb73f341640 1 --2- 172.21.15.84:0/423600318 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fb71c06b0b0 0x7fb71c06d570 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb728036000 tx=0x7fb728030000 comp rx=0 tx=0).stop 2023-10-16T11:25:57.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.917+0000 7fb73f341640 1 -- 172.21.15.84:0/423600318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73806c3f0 msgr2=0x7fb738159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:25:57.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.917+0000 7fb73f341640 1 --2- 172.21.15.84:0/423600318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73806c3f0 0x7fb738159540 secure :-1 s=READY pgs=3004 cs=0 l=1 rev1=1 crypto rx=0x7fb73400d5f0 tx=0x7fb73400dab0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.917+0000 7fb73f341640 1 -- 172.21.15.84:0/423600318 shutdown_connections 2023-10-16T11:25:57.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.917+0000 7fb73f341640 1 --2- 172.21.15.84:0/423600318 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fb71c06b0b0 0x7fb71c06d570 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.917+0000 7fb73f341640 1 --2- 172.21.15.84:0/423600318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb73815e540 0x7fb7380b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.917+0000 7fb73f341640 1 --2- 172.21.15.84:0/423600318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb73815e0c0 0x7fb738155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.917+0000 7fb73f341640 1 --2- 172.21.15.84:0/423600318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73806c3f0 0x7fb738159540 unknown :-1 s=CLOSED pgs=3004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:25:57.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.917+0000 7fb73f341640 1 -- 172.21.15.84:0/423600318 >> 172.21.15.84:0/423600318 conn(0x7fb7380d2cb0 msgr2=0x7fb73815eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:25:57.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.917+0000 7fb73f341640 1 -- 172.21.15.84:0/423600318 shutdown_connections 2023-10-16T11:25:57.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:25:57.917+0000 7fb73f341640 1 -- 172.21.15.84:0/423600318 wait complete. 2023-10-16T11:25:57.943 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":45,"stamp":"2023-10-16T11:25:57.248846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":712,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2868,"ondisk_log_size":2868,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":435424,"kb_used_data":25000,"kb_used_omap":12,"kb_used_meta":410291,"kb_avail":754539296,"statfs":{"total":773094113280,"available":772648239104,"internally_reserved":0,"allocated":25600000,"data_stored":13066130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":420138553},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.979892"},"pg_stats":[{"pgid":"33.1","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.723012+0000","last_change":"2023-10-16T11:25:51.337046+0000","last_active":"2023-10-16T11:25:52.723012+0000","last_peered":"2023-10-16T11:25:52.723012+0000","last_clean":"2023-10-16T11:25:52.723012+0000","last_became_active":"2023-10-16T11:25:51.336494+0000","last_became_peered":"2023-10-16T11:25:51.336494+0000","last_unstale":"2023-10-16T11:25:52.723012+0000","last_undegraded":"2023-10-16T11:25:52.723012+0000","last_fullsized":"2023-10-16T11:25:52.723012+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:35.603183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.303096+0000","last_change":"2023-10-16T11:25:50.296757+0000","last_active":"2023-10-16T11:25:51.303096+0000","last_peered":"2023-10-16T11:25:51.303096+0000","last_clean":"2023-10-16T11:25:51.303096+0000","last_became_active":"2023-10-16T11:25:50.295979+0000","last_became_peered":"2023-10-16T11:25:50.295979+0000","last_unstale":"2023-10-16T11:25:51.303096+0000","last_undegraded":"2023-10-16T11:25:51.303096+0000","last_fullsized":"2023-10-16T11:25:51.303096+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:49:10.974891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3f","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.314306+0000","last_change":"2023-10-16T11:25:37.173492+0000","last_active":"2023-10-16T11:25:51.314306+0000","last_peered":"2023-10-16T11:25:51.314306+0000","last_clean":"2023-10-16T11:25:51.314306+0000","last_became_active":"2023-10-16T11:25:37.167017+0000","last_became_peered":"2023-10-16T11:25:37.167017+0000","last_unstale":"2023-10-16T11:25:51.314306+0000","last_undegraded":"2023-10-16T11:25:51.314306+0000","last_fullsized":"2023-10-16T11:25:51.314306+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:07.791499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.316237+0000","last_change":"2023-10-16T11:25:51.332747+0000","last_active":"2023-10-16T11:25:52.316237+0000","last_peered":"2023-10-16T11:25:52.316237+0000","last_clean":"2023-10-16T11:25:52.316237+0000","last_became_active":"2023-10-16T11:25:51.332320+0000","last_became_peered":"2023-10-16T11:25:51.332320+0000","last_unstale":"2023-10-16T11:25:52.316237+0000","last_undegraded":"2023-10-16T11:25:52.316237+0000","last_fullsized":"2023-10-16T11:25:52.316237+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:20.974486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.294501+0000","last_change":"2023-10-16T11:25:50.299119+0000","last_active":"2023-10-16T11:25:51.294501+0000","last_peered":"2023-10-16T11:25:51.294501+0000","last_clean":"2023-10-16T11:25:51.294501+0000","last_became_active":"2023-10-16T11:25:50.298813+0000","last_became_peered":"2023-10-16T11:25:50.298813+0000","last_unstale":"2023-10-16T11:25:51.294501+0000","last_undegraded":"2023-10-16T11:25:51.294501+0000","last_fullsized":"2023-10-16T11:25:51.294501+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:17:14.257383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.3e","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.315054+0000","last_change":"2023-10-16T11:25:37.179117+0000","last_active":"2023-10-16T11:25:51.315054+0000","last_peered":"2023-10-16T11:25:51.315054+0000","last_clean":"2023-10-16T11:25:51.315054+0000","last_became_active":"2023-10-16T11:25:37.178849+0000","last_became_peered":"2023-10-16T11:25:37.178849+0000","last_unstale":"2023-10-16T11:25:51.315054+0000","last_undegraded":"2023-10-16T11:25:51.315054+0000","last_fullsized":"2023-10-16T11:25:51.315054+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:18:57.278249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.315426+0000","last_change":"2023-10-16T11:25:51.323314+0000","last_active":"2023-10-16T11:25:52.315426+0000","last_peered":"2023-10-16T11:25:52.315426+0000","last_clean":"2023-10-16T11:25:52.315426+0000","last_became_active":"2023-10-16T11:25:51.313845+0000","last_became_peered":"2023-10-16T11:25:51.313845+0000","last_unstale":"2023-10-16T11:25:52.315426+0000","last_undegraded":"2023-10-16T11:25:52.315426+0000","last_fullsized":"2023-10-16T11:25:52.315426+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:32.173887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.322980+0000","last_change":"2023-10-16T11:25:50.300048+0000","last_active":"2023-10-16T11:25:51.322980+0000","last_peered":"2023-10-16T11:25:51.322980+0000","last_clean":"2023-10-16T11:25:51.322980+0000","last_became_active":"2023-10-16T11:25:50.299723+0000","last_became_peered":"2023-10-16T11:25:50.299723+0000","last_unstale":"2023-10-16T11:25:51.322980+0000","last_undegraded":"2023-10-16T11:25:51.322980+0000","last_fullsized":"2023-10-16T11:25:51.322980+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:24.028667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3d","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.305822+0000","last_change":"2023-10-16T11:25:37.179065+0000","last_active":"2023-10-16T11:25:51.305822+0000","last_peered":"2023-10-16T11:25:51.305822+0000","last_clean":"2023-10-16T11:25:51.305822+0000","last_became_active":"2023-10-16T11:25:37.178779+0000","last_became_peered":"2023-10-16T11:25:37.178779+0000","last_unstale":"2023-10-16T11:25:51.305822+0000","last_undegraded":"2023-10-16T11:25:51.305822+0000","last_fullsized":"2023-10-16T11:25:51.305822+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:09:47.367715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.318974+0000","last_change":"2023-10-16T11:25:51.324420+0000","last_active":"2023-10-16T11:25:52.318974+0000","last_peered":"2023-10-16T11:25:52.318974+0000","last_clean":"2023-10-16T11:25:52.318974+0000","last_became_active":"2023-10-16T11:25:51.324143+0000","last_became_peered":"2023-10-16T11:25:51.324143+0000","last_unstale":"2023-10-16T11:25:52.318974+0000","last_undegraded":"2023-10-16T11:25:52.318974+0000","last_fullsized":"2023-10-16T11:25:52.318974+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:45.962774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317461+0000","last_change":"2023-10-16T11:25:50.290401+0000","last_active":"2023-10-16T11:25:51.317461+0000","last_peered":"2023-10-16T11:25:51.317461+0000","last_clean":"2023-10-16T11:25:51.317461+0000","last_became_active":"2023-10-16T11:25:50.290086+0000","last_became_peered":"2023-10-16T11:25:50.290086+0000","last_unstale":"2023-10-16T11:25:51.317461+0000","last_undegraded":"2023-10-16T11:25:51.317461+0000","last_fullsized":"2023-10-16T11:25:51.317461+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:31:20.394734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3c","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.300992+0000","last_change":"2023-10-16T11:25:37.172480+0000","last_active":"2023-10-16T11:25:51.300992+0000","last_peered":"2023-10-16T11:25:51.300992+0000","last_clean":"2023-10-16T11:25:51.300992+0000","last_became_active":"2023-10-16T11:25:37.172192+0000","last_became_peered":"2023-10-16T11:25:37.172192+0000","last_unstale":"2023-10-16T11:25:51.300992+0000","last_undegraded":"2023-10-16T11:25:51.300992+0000","last_fullsized":"2023-10-16T11:25:51.300992+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.724643+0000","last_change":"2023-10-16T11:25:51.331231+0000","last_active":"2023-10-16T11:25:52.724643+0000","last_peered":"2023-10-16T11:25:52.724643+0000","last_clean":"2023-10-16T11:25:52.724643+0000","last_became_active":"2023-10-16T11:25:51.330927+0000","last_became_peered":"2023-10-16T11:25:51.330927+0000","last_unstale":"2023-10-16T11:25:52.724643+0000","last_undegraded":"2023-10-16T11:25:52.724643+0000","last_fullsized":"2023-10-16T11:25:52.724643+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:25.045561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.295663+0000","last_change":"2023-10-16T11:25:50.296115+0000","last_active":"2023-10-16T11:25:51.295663+0000","last_peered":"2023-10-16T11:25:51.295663+0000","last_clean":"2023-10-16T11:25:51.295663+0000","last_became_active":"2023-10-16T11:25:50.295832+0000","last_became_peered":"2023-10-16T11:25:50.295832+0000","last_unstale":"2023-10-16T11:25:51.295663+0000","last_undegraded":"2023-10-16T11:25:51.295663+0000","last_fullsized":"2023-10-16T11:25:51.295663+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:51:43.054850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3b","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.321786+0000","last_change":"2023-10-16T11:25:37.177505+0000","last_active":"2023-10-16T11:25:51.321786+0000","last_peered":"2023-10-16T11:25:51.321786+0000","last_clean":"2023-10-16T11:25:51.321786+0000","last_became_active":"2023-10-16T11:25:37.174334+0000","last_became_peered":"2023-10-16T11:25:37.174334+0000","last_unstale":"2023-10-16T11:25:51.321786+0000","last_undegraded":"2023-10-16T11:25:51.321786+0000","last_fullsized":"2023-10-16T11:25:51.321786+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.723880+0000","last_change":"2023-10-16T11:25:51.336212+0000","last_active":"2023-10-16T11:25:52.723880+0000","last_peered":"2023-10-16T11:25:52.723880+0000","last_clean":"2023-10-16T11:25:52.723880+0000","last_became_active":"2023-10-16T11:25:51.335751+0000","last_became_peered":"2023-10-16T11:25:51.335751+0000","last_unstale":"2023-10-16T11:25:52.723880+0000","last_undegraded":"2023-10-16T11:25:52.723880+0000","last_fullsized":"2023-10-16T11:25:52.723880+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:32:29.165691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.310930+0000","last_change":"2023-10-16T11:25:50.309435+0000","last_active":"2023-10-16T11:25:51.310930+0000","last_peered":"2023-10-16T11:25:51.310930+0000","last_clean":"2023-10-16T11:25:51.310930+0000","last_became_active":"2023-10-16T11:25:50.308844+0000","last_became_peered":"2023-10-16T11:25:50.308844+0000","last_unstale":"2023-10-16T11:25:51.310930+0000","last_undegraded":"2023-10-16T11:25:51.310930+0000","last_fullsized":"2023-10-16T11:25:51.310930+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:31:20.394734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3a","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.303115+0000","last_change":"2023-10-16T11:25:37.177781+0000","last_active":"2023-10-16T11:25:51.303115+0000","last_peered":"2023-10-16T11:25:51.303115+0000","last_clean":"2023-10-16T11:25:51.303115+0000","last_became_active":"2023-10-16T11:25:37.177501+0000","last_became_peered":"2023-10-16T11:25:37.177501+0000","last_unstale":"2023-10-16T11:25:51.303115+0000","last_undegraded":"2023-10-16T11:25:51.303115+0000","last_fullsized":"2023-10-16T11:25:51.303115+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.791145+0000","last_change":"2023-10-16T11:25:51.308305+0000","last_active":"2023-10-16T11:25:52.791145+0000","last_peered":"2023-10-16T11:25:52.791145+0000","last_clean":"2023-10-16T11:25:52.791145+0000","last_became_active":"2023-10-16T11:25:51.307975+0000","last_became_peered":"2023-10-16T11:25:51.307975+0000","last_unstale":"2023-10-16T11:25:52.791145+0000","last_undegraded":"2023-10-16T11:25:52.791145+0000","last_fullsized":"2023-10-16T11:25:52.791145+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:48:44.730513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.324890+0000","last_change":"2023-10-16T11:25:50.307833+0000","last_active":"2023-10-16T11:25:51.324890+0000","last_peered":"2023-10-16T11:25:51.324890+0000","last_clean":"2023-10-16T11:25:51.324890+0000","last_became_active":"2023-10-16T11:25:50.307526+0000","last_became_peered":"2023-10-16T11:25:50.307526+0000","last_unstale":"2023-10-16T11:25:51.324890+0000","last_undegraded":"2023-10-16T11:25:51.324890+0000","last_fullsized":"2023-10-16T11:25:51.324890+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:55:11.599298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.39","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317697+0000","last_change":"2023-10-16T11:25:37.182745+0000","last_active":"2023-10-16T11:25:51.317697+0000","last_peered":"2023-10-16T11:25:51.317697+0000","last_clean":"2023-10-16T11:25:51.317697+0000","last_became_active":"2023-10-16T11:25:37.182437+0000","last_became_peered":"2023-10-16T11:25:37.182437+0000","last_unstale":"2023-10-16T11:25:51.317697+0000","last_undegraded":"2023-10-16T11:25:51.317697+0000","last_fullsized":"2023-10-16T11:25:51.317697+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.321194+0000","last_change":"2023-10-16T11:25:51.325207+0000","last_active":"2023-10-16T11:25:52.321194+0000","last_peered":"2023-10-16T11:25:52.321194+0000","last_clean":"2023-10-16T11:25:52.321194+0000","last_became_active":"2023-10-16T11:25:51.323968+0000","last_became_peered":"2023-10-16T11:25:51.323968+0000","last_unstale":"2023-10-16T11:25:52.321194+0000","last_undegraded":"2023-10-16T11:25:52.321194+0000","last_fullsized":"2023-10-16T11:25:52.321194+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:06.585825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"240'2","reported_seq":14,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.348098+0000","last_change":"2023-10-16T11:25:50.292129+0000","last_active":"2023-10-16T11:25:51.348098+0000","last_peered":"2023-10-16T11:25:51.348098+0000","last_clean":"2023-10-16T11:25:51.348098+0000","last_became_active":"2023-10-16T11:25:50.291743+0000","last_became_peered":"2023-10-16T11:25:50.291743+0000","last_unstale":"2023-10-16T11:25:51.348098+0000","last_undegraded":"2023-10-16T11:25:51.348098+0000","last_fullsized":"2023-10-16T11:25:51.348098+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:34:31.362688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.38","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.294901+0000","last_change":"2023-10-16T11:25:37.174929+0000","last_active":"2023-10-16T11:25:51.294901+0000","last_peered":"2023-10-16T11:25:51.294901+0000","last_clean":"2023-10-16T11:25:51.294901+0000","last_became_active":"2023-10-16T11:25:37.172810+0000","last_became_peered":"2023-10-16T11:25:37.172810+0000","last_unstale":"2023-10-16T11:25:51.294901+0000","last_undegraded":"2023-10-16T11:25:51.294901+0000","last_fullsized":"2023-10-16T11:25:51.294901+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.315690+0000","last_change":"2023-10-16T11:25:51.335355+0000","last_active":"2023-10-16T11:25:52.315690+0000","last_peered":"2023-10-16T11:25:52.315690+0000","last_clean":"2023-10-16T11:25:52.315690+0000","last_became_active":"2023-10-16T11:25:51.335089+0000","last_became_peered":"2023-10-16T11:25:51.335089+0000","last_unstale":"2023-10-16T11:25:52.315690+0000","last_undegraded":"2023-10-16T11:25:52.315690+0000","last_fullsized":"2023-10-16T11:25:52.315690+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:20.974486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.295492+0000","last_change":"2023-10-16T11:25:50.303972+0000","last_active":"2023-10-16T11:25:51.295492+0000","last_peered":"2023-10-16T11:25:51.295492+0000","last_clean":"2023-10-16T11:25:51.295492+0000","last_became_active":"2023-10-16T11:25:50.302166+0000","last_became_peered":"2023-10-16T11:25:50.302166+0000","last_unstale":"2023-10-16T11:25:51.295492+0000","last_undegraded":"2023-10-16T11:25:51.295492+0000","last_fullsized":"2023-10-16T11:25:51.295492+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:54.968684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.37","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317928+0000","last_change":"2023-10-16T11:25:37.177647+0000","last_active":"2023-10-16T11:25:51.317928+0000","last_peered":"2023-10-16T11:25:51.317928+0000","last_clean":"2023-10-16T11:25:51.317928+0000","last_became_active":"2023-10-16T11:25:37.176875+0000","last_became_peered":"2023-10-16T11:25:37.176875+0000","last_unstale":"2023-10-16T11:25:51.317928+0000","last_undegraded":"2023-10-16T11:25:51.317928+0000","last_fullsized":"2023-10-16T11:25:51.317928+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:21.630921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789782+0000","last_change":"2023-10-16T11:25:51.307779+0000","last_active":"2023-10-16T11:25:52.789782+0000","last_peered":"2023-10-16T11:25:52.789782+0000","last_clean":"2023-10-16T11:25:52.789782+0000","last_became_active":"2023-10-16T11:25:51.307456+0000","last_became_peered":"2023-10-16T11:25:51.307456+0000","last_unstale":"2023-10-16T11:25:52.789782+0000","last_undegraded":"2023-10-16T11:25:52.789782+0000","last_fullsized":"2023-10-16T11:25:52.789782+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:11.229022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.319094+0000","last_change":"2023-10-16T11:25:50.303286+0000","last_active":"2023-10-16T11:25:51.319094+0000","last_peered":"2023-10-16T11:25:51.319094+0000","last_clean":"2023-10-16T11:25:51.319094+0000","last_became_active":"2023-10-16T11:25:50.302625+0000","last_became_peered":"2023-10-16T11:25:50.302625+0000","last_unstale":"2023-10-16T11:25:51.319094+0000","last_undegraded":"2023-10-16T11:25:51.319094+0000","last_fullsized":"2023-10-16T11:25:51.319094+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:42:35.667844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.36","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.313090+0000","last_change":"2023-10-16T11:25:37.172841+0000","last_active":"2023-10-16T11:25:51.313090+0000","last_peered":"2023-10-16T11:25:51.313090+0000","last_clean":"2023-10-16T11:25:51.313090+0000","last_became_active":"2023-10-16T11:25:37.172550+0000","last_became_peered":"2023-10-16T11:25:37.172550+0000","last_unstale":"2023-10-16T11:25:51.313090+0000","last_undegraded":"2023-10-16T11:25:51.313090+0000","last_fullsized":"2023-10-16T11:25:51.313090+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.788520+0000","last_change":"2023-10-16T11:25:51.317443+0000","last_active":"2023-10-16T11:25:52.788520+0000","last_peered":"2023-10-16T11:25:52.788520+0000","last_clean":"2023-10-16T11:25:52.788520+0000","last_became_active":"2023-10-16T11:25:51.316556+0000","last_became_peered":"2023-10-16T11:25:51.316556+0000","last_unstale":"2023-10-16T11:25:52.788520+0000","last_undegraded":"2023-10-16T11:25:52.788520+0000","last_fullsized":"2023-10-16T11:25:52.788520+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:51.178566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.294779+0000","last_change":"2023-10-16T11:25:50.290580+0000","last_active":"2023-10-16T11:25:51.294779+0000","last_peered":"2023-10-16T11:25:51.294779+0000","last_clean":"2023-10-16T11:25:51.294779+0000","last_became_active":"2023-10-16T11:25:50.290044+0000","last_became_peered":"2023-10-16T11:25:50.290044+0000","last_unstale":"2023-10-16T11:25:51.294779+0000","last_undegraded":"2023-10-16T11:25:51.294779+0000","last_fullsized":"2023-10-16T11:25:51.294779+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:34:31.362688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.35","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.320218+0000","last_change":"2023-10-16T11:25:37.181386+0000","last_active":"2023-10-16T11:25:51.320218+0000","last_peered":"2023-10-16T11:25:51.320218+0000","last_clean":"2023-10-16T11:25:51.320218+0000","last_became_active":"2023-10-16T11:25:37.181007+0000","last_became_peered":"2023-10-16T11:25:37.181007+0000","last_unstale":"2023-10-16T11:25:51.320218+0000","last_undegraded":"2023-10-16T11:25:51.320218+0000","last_fullsized":"2023-10-16T11:25:51.320218+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:46.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789452+0000","last_change":"2023-10-16T11:25:51.335795+0000","last_active":"2023-10-16T11:25:52.789452+0000","last_peered":"2023-10-16T11:25:52.789452+0000","last_clean":"2023-10-16T11:25:52.789452+0000","last_became_active":"2023-10-16T11:25:51.335492+0000","last_became_peered":"2023-10-16T11:25:51.335492+0000","last_unstale":"2023-10-16T11:25:52.789452+0000","last_undegraded":"2023-10-16T11:25:52.789452+0000","last_fullsized":"2023-10-16T11:25:52.789452+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:18.229520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"240'2","reported_seq":14,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.343582+0000","last_change":"2023-10-16T11:25:50.308431+0000","last_active":"2023-10-16T11:25:51.343582+0000","last_peered":"2023-10-16T11:25:51.343582+0000","last_clean":"2023-10-16T11:25:51.343582+0000","last_became_active":"2023-10-16T11:25:50.308094+0000","last_became_peered":"2023-10-16T11:25:50.308094+0000","last_unstale":"2023-10-16T11:25:51.343582+0000","last_undegraded":"2023-10-16T11:25:51.343582+0000","last_fullsized":"2023-10-16T11:25:51.343582+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:53:51.538566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.34","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.297827+0000","last_change":"2023-10-16T11:25:37.179103+0000","last_active":"2023-10-16T11:25:51.297827+0000","last_peered":"2023-10-16T11:25:51.297827+0000","last_clean":"2023-10-16T11:25:51.297827+0000","last_became_active":"2023-10-16T11:25:37.178445+0000","last_became_peered":"2023-10-16T11:25:37.178445+0000","last_unstale":"2023-10-16T11:25:51.297827+0000","last_undegraded":"2023-10-16T11:25:51.297827+0000","last_fullsized":"2023-10-16T11:25:51.297827+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:38.218704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.724278+0000","last_change":"2023-10-16T11:25:51.337272+0000","last_active":"2023-10-16T11:25:52.724278+0000","last_peered":"2023-10-16T11:25:52.724278+0000","last_clean":"2023-10-16T11:25:52.724278+0000","last_became_active":"2023-10-16T11:25:51.336260+0000","last_became_peered":"2023-10-16T11:25:51.336260+0000","last_unstale":"2023-10-16T11:25:52.724278+0000","last_undegraded":"2023-10-16T11:25:52.724278+0000","last_fullsized":"2023-10-16T11:25:52.724278+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:34:32.379582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.300778+0000","last_change":"2023-10-16T11:25:50.296580+0000","last_active":"2023-10-16T11:25:51.300778+0000","last_peered":"2023-10-16T11:25:51.300778+0000","last_clean":"2023-10-16T11:25:51.300778+0000","last_became_active":"2023-10-16T11:25:50.296102+0000","last_became_peered":"2023-10-16T11:25:50.296102+0000","last_unstale":"2023-10-16T11:25:51.300778+0000","last_undegraded":"2023-10-16T11:25:51.300778+0000","last_fullsized":"2023-10-16T11:25:51.300778+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:31:20.394734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.33","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.312802+0000","last_change":"2023-10-16T11:25:37.182921+0000","last_active":"2023-10-16T11:25:51.312802+0000","last_peered":"2023-10-16T11:25:51.312802+0000","last_clean":"2023-10-16T11:25:51.312802+0000","last_became_active":"2023-10-16T11:25:37.182603+0000","last_became_peered":"2023-10-16T11:25:37.182603+0000","last_unstale":"2023-10-16T11:25:51.312802+0000","last_undegraded":"2023-10-16T11:25:51.312802+0000","last_fullsized":"2023-10-16T11:25:51.312802+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.319468+0000","last_change":"2023-10-16T11:25:51.320743+0000","last_active":"2023-10-16T11:25:52.319468+0000","last_peered":"2023-10-16T11:25:52.319468+0000","last_clean":"2023-10-16T11:25:52.319468+0000","last_became_active":"2023-10-16T11:25:51.320377+0000","last_became_peered":"2023-10-16T11:25:51.320377+0000","last_unstale":"2023-10-16T11:25:52.319468+0000","last_undegraded":"2023-10-16T11:25:52.319468+0000","last_fullsized":"2023-10-16T11:25:52.319468+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:09.714929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"240'1","reported_seq":13,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.347820+0000","last_change":"2023-10-16T11:25:50.305580+0000","last_active":"2023-10-16T11:25:51.347820+0000","last_peered":"2023-10-16T11:25:51.347820+0000","last_clean":"2023-10-16T11:25:51.347820+0000","last_became_active":"2023-10-16T11:25:50.304710+0000","last_became_peered":"2023-10-16T11:25:50.304710+0000","last_unstale":"2023-10-16T11:25:51.347820+0000","last_undegraded":"2023-10-16T11:25:51.347820+0000","last_fullsized":"2023-10-16T11:25:51.347820+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:37.230753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.32","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.313025+0000","last_change":"2023-10-16T11:25:37.171715+0000","last_active":"2023-10-16T11:25:51.313025+0000","last_peered":"2023-10-16T11:25:51.313025+0000","last_clean":"2023-10-16T11:25:51.313025+0000","last_became_active":"2023-10-16T11:25:37.171421+0000","last_became_peered":"2023-10-16T11:25:37.171421+0000","last_unstale":"2023-10-16T11:25:51.313025+0000","last_undegraded":"2023-10-16T11:25:51.313025+0000","last_fullsized":"2023-10-16T11:25:51.313025+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:10.207081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.442721+0000","last_change":"2023-10-16T11:25:51.341460+0000","last_active":"2023-10-16T11:25:52.442721+0000","last_peered":"2023-10-16T11:25:52.442721+0000","last_clean":"2023-10-16T11:25:52.442721+0000","last_became_active":"2023-10-16T11:25:51.340947+0000","last_became_peered":"2023-10-16T11:25:51.340947+0000","last_unstale":"2023-10-16T11:25:52.442721+0000","last_undegraded":"2023-10-16T11:25:52.442721+0000","last_fullsized":"2023-10-16T11:25:52.442721+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:36.561514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"240'2","reported_seq":14,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.346856+0000","last_change":"2023-10-16T11:25:50.297668+0000","last_active":"2023-10-16T11:25:51.346856+0000","last_peered":"2023-10-16T11:25:51.346856+0000","last_clean":"2023-10-16T11:25:51.346856+0000","last_became_active":"2023-10-16T11:25:50.297397+0000","last_became_peered":"2023-10-16T11:25:50.297397+0000","last_unstale":"2023-10-16T11:25:51.346856+0000","last_undegraded":"2023-10-16T11:25:51.346856+0000","last_fullsized":"2023-10-16T11:25:51.346856+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:53:17.916147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.31","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.322765+0000","last_change":"2023-10-16T11:25:37.177928+0000","last_active":"2023-10-16T11:25:51.322765+0000","last_peered":"2023-10-16T11:25:51.322765+0000","last_clean":"2023-10-16T11:25:51.322765+0000","last_became_active":"2023-10-16T11:25:37.177612+0000","last_became_peered":"2023-10-16T11:25:37.177612+0000","last_unstale":"2023-10-16T11:25:51.322765+0000","last_undegraded":"2023-10-16T11:25:51.322765+0000","last_fullsized":"2023-10-16T11:25:51.322765+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:44.826687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.321638+0000","last_change":"2023-10-16T11:25:51.325376+0000","last_active":"2023-10-16T11:25:52.321638+0000","last_peered":"2023-10-16T11:25:52.321638+0000","last_clean":"2023-10-16T11:25:52.321638+0000","last_became_active":"2023-10-16T11:25:51.323998+0000","last_became_peered":"2023-10-16T11:25:51.323998+0000","last_unstale":"2023-10-16T11:25:52.321638+0000","last_undegraded":"2023-10-16T11:25:52.321638+0000","last_fullsized":"2023-10-16T11:25:52.321638+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:11.420914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"240'1","reported_seq":13,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.343502+0000","last_change":"2023-10-16T11:25:50.304807+0000","last_active":"2023-10-16T11:25:51.343502+0000","last_peered":"2023-10-16T11:25:51.343502+0000","last_clean":"2023-10-16T11:25:51.343502+0000","last_became_active":"2023-10-16T11:25:50.304523+0000","last_became_peered":"2023-10-16T11:25:50.304523+0000","last_unstale":"2023-10-16T11:25:51.343502+0000","last_undegraded":"2023-10-16T11:25:51.343502+0000","last_fullsized":"2023-10-16T11:25:51.343502+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:31.156993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.30","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.302645+0000","last_change":"2023-10-16T11:25:37.170292+0000","last_active":"2023-10-16T11:25:51.302645+0000","last_peered":"2023-10-16T11:25:51.302645+0000","last_clean":"2023-10-16T11:25:51.302645+0000","last_became_active":"2023-10-16T11:25:37.169997+0000","last_became_peered":"2023-10-16T11:25:37.169997+0000","last_unstale":"2023-10-16T11:25:51.302645+0000","last_undegraded":"2023-10-16T11:25:51.302645+0000","last_fullsized":"2023-10-16T11:25:51.302645+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:09.329721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789393+0000","last_change":"2023-10-16T11:25:51.327001+0000","last_active":"2023-10-16T11:25:52.789393+0000","last_peered":"2023-10-16T11:25:52.789393+0000","last_clean":"2023-10-16T11:25:52.789393+0000","last_became_active":"2023-10-16T11:25:51.326746+0000","last_became_peered":"2023-10-16T11:25:51.326746+0000","last_unstale":"2023-10-16T11:25:52.789393+0000","last_undegraded":"2023-10-16T11:25:52.789393+0000","last_fullsized":"2023-10-16T11:25:52.789393+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:19.314057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"240'8","reported_seq":20,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.349111+0000","last_change":"2023-10-16T11:25:50.297630+0000","last_active":"2023-10-16T11:25:51.349111+0000","last_peered":"2023-10-16T11:25:51.349111+0000","last_clean":"2023-10-16T11:25:51.349111+0000","last_became_active":"2023-10-16T11:25:50.295103+0000","last_became_peered":"2023-10-16T11:25:50.295103+0000","last_unstale":"2023-10-16T11:25:51.349111+0000","last_undegraded":"2023-10-16T11:25:51.349111+0000","last_fullsized":"2023-10-16T11:25:51.349111+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:58.688176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2f","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.300068+0000","last_change":"2023-10-16T11:25:37.162923+0000","last_active":"2023-10-16T11:25:51.300068+0000","last_peered":"2023-10-16T11:25:51.300068+0000","last_clean":"2023-10-16T11:25:51.300068+0000","last_became_active":"2023-10-16T11:25:37.162649+0000","last_became_peered":"2023-10-16T11:25:37.162649+0000","last_unstale":"2023-10-16T11:25:51.300068+0000","last_undegraded":"2023-10-16T11:25:51.300068+0000","last_fullsized":"2023-10-16T11:25:51.300068+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:46.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789936+0000","last_change":"2023-10-16T11:25:51.321287+0000","last_active":"2023-10-16T11:25:52.789936+0000","last_peered":"2023-10-16T11:25:52.789936+0000","last_clean":"2023-10-16T11:25:52.789936+0000","last_became_active":"2023-10-16T11:25:51.320555+0000","last_became_peered":"2023-10-16T11:25:51.320555+0000","last_unstale":"2023-10-16T11:25:52.789936+0000","last_undegraded":"2023-10-16T11:25:52.789936+0000","last_fullsized":"2023-10-16T11:25:52.789936+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:15.713642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.322899+0000","last_change":"2023-10-16T11:25:50.306156+0000","last_active":"2023-10-16T11:25:51.322899+0000","last_peered":"2023-10-16T11:25:51.322899+0000","last_clean":"2023-10-16T11:25:51.322899+0000","last_became_active":"2023-10-16T11:25:50.305883+0000","last_became_peered":"2023-10-16T11:25:50.305883+0000","last_unstale":"2023-10-16T11:25:51.322899+0000","last_undegraded":"2023-10-16T11:25:51.322899+0000","last_fullsized":"2023-10-16T11:25:51.322899+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:42:45.711414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.2e","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.315147+0000","last_change":"2023-10-16T11:25:37.174044+0000","last_active":"2023-10-16T11:25:51.315147+0000","last_peered":"2023-10-16T11:25:51.315147+0000","last_clean":"2023-10-16T11:25:51.315147+0000","last_became_active":"2023-10-16T11:25:37.166694+0000","last_became_peered":"2023-10-16T11:25:37.166694+0000","last_unstale":"2023-10-16T11:25:51.315147+0000","last_undegraded":"2023-10-16T11:25:51.315147+0000","last_fullsized":"2023-10-16T11:25:51.315147+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:23.146288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.788098+0000","last_change":"2023-10-16T11:25:51.326577+0000","last_active":"2023-10-16T11:25:52.788098+0000","last_peered":"2023-10-16T11:25:52.788098+0000","last_clean":"2023-10-16T11:25:52.788098+0000","last_became_active":"2023-10-16T11:25:51.326129+0000","last_became_peered":"2023-10-16T11:25:51.326129+0000","last_unstale":"2023-10-16T11:25:52.788098+0000","last_undegraded":"2023-10-16T11:25:52.788098+0000","last_fullsized":"2023-10-16T11:25:52.788098+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:11.767351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.323367+0000","last_change":"2023-10-16T11:25:50.293946+0000","last_active":"2023-10-16T11:25:51.323367+0000","last_peered":"2023-10-16T11:25:51.323367+0000","last_clean":"2023-10-16T11:25:51.323367+0000","last_became_active":"2023-10-16T11:25:50.293183+0000","last_became_peered":"2023-10-16T11:25:50.293183+0000","last_unstale":"2023-10-16T11:25:51.323367+0000","last_undegraded":"2023-10-16T11:25:51.323367+0000","last_fullsized":"2023-10-16T11:25:51.323367+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:31.156993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2d","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.298245+0000","last_change":"2023-10-16T11:25:37.183893+0000","last_active":"2023-10-16T11:25:51.298245+0000","last_peered":"2023-10-16T11:25:51.298245+0000","last_clean":"2023-10-16T11:25:51.298245+0000","last_became_active":"2023-10-16T11:25:37.183586+0000","last_became_peered":"2023-10-16T11:25:37.183586+0000","last_unstale":"2023-10-16T11:25:51.298245+0000","last_undegraded":"2023-10-16T11:25:51.298245+0000","last_fullsized":"2023-10-16T11:25:51.298245+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:43.813878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789843+0000","last_change":"2023-10-16T11:25:51.320733+0000","last_active":"2023-10-16T11:25:52.789843+0000","last_peered":"2023-10-16T11:25:52.789843+0000","last_clean":"2023-10-16T11:25:52.789843+0000","last_became_active":"2023-10-16T11:25:51.320448+0000","last_became_peered":"2023-10-16T11:25:51.320448+0000","last_unstale":"2023-10-16T11:25:52.789843+0000","last_undegraded":"2023-10-16T11:25:52.789843+0000","last_fullsized":"2023-10-16T11:25:52.789843+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:15.782010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"240'2","reported_seq":14,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.348067+0000","last_change":"2023-10-16T11:25:50.298428+0000","last_active":"2023-10-16T11:25:51.348067+0000","last_peered":"2023-10-16T11:25:51.348067+0000","last_clean":"2023-10-16T11:25:51.348067+0000","last_became_active":"2023-10-16T11:25:50.297066+0000","last_became_peered":"2023-10-16T11:25:50.297066+0000","last_unstale":"2023-10-16T11:25:51.348067+0000","last_undegraded":"2023-10-16T11:25:51.348067+0000","last_fullsized":"2023-10-16T11:25:51.348067+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:24.028667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2c","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.301821+0000","last_change":"2023-10-16T11:25:37.172984+0000","last_active":"2023-10-16T11:25:51.301821+0000","last_peered":"2023-10-16T11:25:51.301821+0000","last_clean":"2023-10-16T11:25:51.301821+0000","last_became_active":"2023-10-16T11:25:37.172403+0000","last_became_peered":"2023-10-16T11:25:37.172403+0000","last_unstale":"2023-10-16T11:25:51.301821+0000","last_undegraded":"2023-10-16T11:25:51.301821+0000","last_fullsized":"2023-10-16T11:25:51.301821+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:03.912995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.790904+0000","last_change":"2023-10-16T11:25:51.324490+0000","last_active":"2023-10-16T11:25:52.790904+0000","last_peered":"2023-10-16T11:25:52.790904+0000","last_clean":"2023-10-16T11:25:52.790904+0000","last_became_active":"2023-10-16T11:25:51.324241+0000","last_became_peered":"2023-10-16T11:25:51.324241+0000","last_unstale":"2023-10-16T11:25:52.790904+0000","last_undegraded":"2023-10-16T11:25:52.790904+0000","last_fullsized":"2023-10-16T11:25:52.790904+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:15:02.083129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"240'6","reported_seq":18,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.350676+0000","last_change":"2023-10-16T11:25:50.282643+0000","last_active":"2023-10-16T11:25:51.350676+0000","last_peered":"2023-10-16T11:25:51.350676+0000","last_clean":"2023-10-16T11:25:51.350676+0000","last_became_active":"2023-10-16T11:25:50.282381+0000","last_became_peered":"2023-10-16T11:25:50.282381+0000","last_unstale":"2023-10-16T11:25:51.350676+0000","last_undegraded":"2023-10-16T11:25:51.350676+0000","last_fullsized":"2023-10-16T11:25:51.350676+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:39:01.747122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2b","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.296272+0000","last_change":"2023-10-16T11:25:37.173026+0000","last_active":"2023-10-16T11:25:51.296272+0000","last_peered":"2023-10-16T11:25:51.296272+0000","last_clean":"2023-10-16T11:25:51.296272+0000","last_became_active":"2023-10-16T11:25:37.172680+0000","last_became_peered":"2023-10-16T11:25:37.172680+0000","last_unstale":"2023-10-16T11:25:51.296272+0000","last_undegraded":"2023-10-16T11:25:51.296272+0000","last_fullsized":"2023-10-16T11:25:51.296272+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:56.314696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.319590+0000","last_change":"2023-10-16T11:25:51.306352+0000","last_active":"2023-10-16T11:25:52.319590+0000","last_peered":"2023-10-16T11:25:52.319590+0000","last_clean":"2023-10-16T11:25:52.319590+0000","last_became_active":"2023-10-16T11:25:51.305895+0000","last_became_peered":"2023-10-16T11:25:51.305895+0000","last_unstale":"2023-10-16T11:25:52.319590+0000","last_undegraded":"2023-10-16T11:25:52.319590+0000","last_fullsized":"2023-10-16T11:25:52.319590+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:24.349747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"240'3","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.347853+0000","last_change":"2023-10-16T11:25:50.303987+0000","last_active":"2023-10-16T11:25:51.347853+0000","last_peered":"2023-10-16T11:25:51.347853+0000","last_clean":"2023-10-16T11:25:51.347853+0000","last_became_active":"2023-10-16T11:25:50.303000+0000","last_became_peered":"2023-10-16T11:25:50.303000+0000","last_unstale":"2023-10-16T11:25:51.347853+0000","last_undegraded":"2023-10-16T11:25:51.347853+0000","last_fullsized":"2023-10-16T11:25:51.347853+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:53:25.980767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2a","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.311092+0000","last_change":"2023-10-16T11:25:37.176870+0000","last_active":"2023-10-16T11:25:51.311092+0000","last_peered":"2023-10-16T11:25:51.311092+0000","last_clean":"2023-10-16T11:25:51.311092+0000","last_became_active":"2023-10-16T11:25:37.175230+0000","last_became_peered":"2023-10-16T11:25:37.175230+0000","last_unstale":"2023-10-16T11:25:51.311092+0000","last_undegraded":"2023-10-16T11:25:51.311092+0000","last_fullsized":"2023-10-16T11:25:51.311092+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.790894+0000","last_change":"2023-10-16T11:25:51.317013+0000","last_active":"2023-10-16T11:25:52.790894+0000","last_peered":"2023-10-16T11:25:52.790894+0000","last_clean":"2023-10-16T11:25:52.790894+0000","last_became_active":"2023-10-16T11:25:51.316725+0000","last_became_peered":"2023-10-16T11:25:51.316725+0000","last_unstale":"2023-10-16T11:25:52.790894+0000","last_undegraded":"2023-10-16T11:25:52.790894+0000","last_fullsized":"2023-10-16T11:25:52.790894+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:33:52.278971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.314785+0000","last_change":"2023-10-16T11:25:50.302794+0000","last_active":"2023-10-16T11:25:51.314785+0000","last_peered":"2023-10-16T11:25:51.314785+0000","last_clean":"2023-10-16T11:25:51.314785+0000","last_became_active":"2023-10-16T11:25:50.302353+0000","last_became_peered":"2023-10-16T11:25:50.302353+0000","last_unstale":"2023-10-16T11:25:51.314785+0000","last_undegraded":"2023-10-16T11:25:51.314785+0000","last_fullsized":"2023-10-16T11:25:51.314785+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:12:02.269450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.29","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.293943+0000","last_change":"2023-10-16T11:25:37.169637+0000","last_active":"2023-10-16T11:25:51.293943+0000","last_peered":"2023-10-16T11:25:51.293943+0000","last_clean":"2023-10-16T11:25:51.293943+0000","last_became_active":"2023-10-16T11:25:37.169375+0000","last_became_peered":"2023-10-16T11:25:37.169375+0000","last_unstale":"2023-10-16T11:25:51.293943+0000","last_undegraded":"2023-10-16T11:25:51.293943+0000","last_fullsized":"2023-10-16T11:25:51.293943+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:42:23.188876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789651+0000","last_change":"2023-10-16T11:25:51.327693+0000","last_active":"2023-10-16T11:25:52.789651+0000","last_peered":"2023-10-16T11:25:52.789651+0000","last_clean":"2023-10-16T11:25:52.789651+0000","last_became_active":"2023-10-16T11:25:51.326040+0000","last_became_peered":"2023-10-16T11:25:51.326040+0000","last_unstale":"2023-10-16T11:25:52.789651+0000","last_undegraded":"2023-10-16T11:25:52.789651+0000","last_fullsized":"2023-10-16T11:25:52.789651+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:06.585825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.307532+0000","last_change":"2023-10-16T11:25:50.304100+0000","last_active":"2023-10-16T11:25:51.307532+0000","last_peered":"2023-10-16T11:25:51.307532+0000","last_clean":"2023-10-16T11:25:51.307532+0000","last_became_active":"2023-10-16T11:25:50.303840+0000","last_became_peered":"2023-10-16T11:25:50.303840+0000","last_unstale":"2023-10-16T11:25:51.307532+0000","last_undegraded":"2023-10-16T11:25:51.307532+0000","last_fullsized":"2023-10-16T11:25:51.307532+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:41.793494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.28","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.312147+0000","last_change":"2023-10-16T11:25:37.187270+0000","last_active":"2023-10-16T11:25:51.312147+0000","last_peered":"2023-10-16T11:25:51.312147+0000","last_clean":"2023-10-16T11:25:51.312147+0000","last_became_active":"2023-10-16T11:25:37.186962+0000","last_became_peered":"2023-10-16T11:25:37.186962+0000","last_unstale":"2023-10-16T11:25:51.312147+0000","last_undegraded":"2023-10-16T11:25:51.312147+0000","last_fullsized":"2023-10-16T11:25:51.312147+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:56:45.915920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.788719+0000","last_change":"2023-10-16T11:25:51.326733+0000","last_active":"2023-10-16T11:25:52.788719+0000","last_peered":"2023-10-16T11:25:52.788719+0000","last_clean":"2023-10-16T11:25:52.788719+0000","last_became_active":"2023-10-16T11:25:51.325905+0000","last_became_peered":"2023-10-16T11:25:51.325905+0000","last_unstale":"2023-10-16T11:25:52.788719+0000","last_undegraded":"2023-10-16T11:25:52.788719+0000","last_fullsized":"2023-10-16T11:25:52.788719+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:09.714929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.311169+0000","last_change":"2023-10-16T11:25:50.298538+0000","last_active":"2023-10-16T11:25:51.311169+0000","last_peered":"2023-10-16T11:25:51.311169+0000","last_clean":"2023-10-16T11:25:51.311169+0000","last_became_active":"2023-10-16T11:25:50.298265+0000","last_became_peered":"2023-10-16T11:25:50.298265+0000","last_unstale":"2023-10-16T11:25:51.311169+0000","last_undegraded":"2023-10-16T11:25:51.311169+0000","last_fullsized":"2023-10-16T11:25:51.311169+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:34:31.362688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.27","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.305448+0000","last_change":"2023-10-16T11:25:37.177057+0000","last_active":"2023-10-16T11:25:51.305448+0000","last_peered":"2023-10-16T11:25:51.305448+0000","last_clean":"2023-10-16T11:25:51.305448+0000","last_became_active":"2023-10-16T11:25:37.171369+0000","last_became_peered":"2023-10-16T11:25:37.171369+0000","last_unstale":"2023-10-16T11:25:51.305448+0000","last_undegraded":"2023-10-16T11:25:51.305448+0000","last_fullsized":"2023-10-16T11:25:51.305448+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.787285+0000","last_change":"2023-10-16T11:25:51.323651+0000","last_active":"2023-10-16T11:25:52.787285+0000","last_peered":"2023-10-16T11:25:52.787285+0000","last_clean":"2023-10-16T11:25:52.787285+0000","last_became_active":"2023-10-16T11:25:51.323150+0000","last_became_peered":"2023-10-16T11:25:51.323150+0000","last_unstale":"2023-10-16T11:25:52.787285+0000","last_undegraded":"2023-10-16T11:25:52.787285+0000","last_fullsized":"2023-10-16T11:25:52.787285+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:40.093581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.320872+0000","last_change":"2023-10-16T11:25:50.307251+0000","last_active":"2023-10-16T11:25:51.320872+0000","last_peered":"2023-10-16T11:25:51.320872+0000","last_clean":"2023-10-16T11:25:51.320872+0000","last_became_active":"2023-10-16T11:25:50.306831+0000","last_became_peered":"2023-10-16T11:25:50.306831+0000","last_unstale":"2023-10-16T11:25:51.320872+0000","last_undegraded":"2023-10-16T11:25:51.320872+0000","last_fullsized":"2023-10-16T11:25:51.320872+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:41.271897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.26","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.313225+0000","last_change":"2023-10-16T11:25:37.182875+0000","last_active":"2023-10-16T11:25:51.313225+0000","last_peered":"2023-10-16T11:25:51.313225+0000","last_clean":"2023-10-16T11:25:51.313225+0000","last_became_active":"2023-10-16T11:25:37.182570+0000","last_became_peered":"2023-10-16T11:25:37.182570+0000","last_unstale":"2023-10-16T11:25:51.313225+0000","last_undegraded":"2023-10-16T11:25:51.313225+0000","last_fullsized":"2023-10-16T11:25:51.313225+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:14.504702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.787755+0000","last_change":"2023-10-16T11:25:51.320594+0000","last_active":"2023-10-16T11:25:52.787755+0000","last_peered":"2023-10-16T11:25:52.787755+0000","last_clean":"2023-10-16T11:25:52.787755+0000","last_became_active":"2023-10-16T11:25:51.319783+0000","last_became_peered":"2023-10-16T11:25:51.319783+0000","last_unstale":"2023-10-16T11:25:52.787755+0000","last_undegraded":"2023-10-16T11:25:52.787755+0000","last_fullsized":"2023-10-16T11:25:52.787755+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:32.798332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"240'1","reported_seq":13,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.344852+0000","last_change":"2023-10-16T11:25:50.299291+0000","last_active":"2023-10-16T11:25:51.344852+0000","last_peered":"2023-10-16T11:25:51.344852+0000","last_clean":"2023-10-16T11:25:51.344852+0000","last_became_active":"2023-10-16T11:25:50.298997+0000","last_became_peered":"2023-10-16T11:25:50.298997+0000","last_unstale":"2023-10-16T11:25:51.344852+0000","last_undegraded":"2023-10-16T11:25:51.344852+0000","last_fullsized":"2023-10-16T11:25:51.344852+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:17.289884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.25","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.320279+0000","last_change":"2023-10-16T11:25:37.178245+0000","last_active":"2023-10-16T11:25:51.320279+0000","last_peered":"2023-10-16T11:25:51.320279+0000","last_clean":"2023-10-16T11:25:51.320279+0000","last_became_active":"2023-10-16T11:25:37.177914+0000","last_became_peered":"2023-10-16T11:25:37.177914+0000","last_unstale":"2023-10-16T11:25:51.320279+0000","last_undegraded":"2023-10-16T11:25:51.320279+0000","last_fullsized":"2023-10-16T11:25:51.320279+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:12.014513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.316581+0000","last_change":"2023-10-16T11:25:51.334335+0000","last_active":"2023-10-16T11:25:52.316581+0000","last_peered":"2023-10-16T11:25:52.316581+0000","last_clean":"2023-10-16T11:25:52.316581+0000","last_became_active":"2023-10-16T11:25:51.333152+0000","last_became_peered":"2023-10-16T11:25:51.333152+0000","last_unstale":"2023-10-16T11:25:52.316581+0000","last_undegraded":"2023-10-16T11:25:52.316581+0000","last_fullsized":"2023-10-16T11:25:52.316581+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:49.517942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"240'1","reported_seq":13,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.340669+0000","last_change":"2023-10-16T11:25:50.303326+0000","last_active":"2023-10-16T11:25:51.340669+0000","last_peered":"2023-10-16T11:25:51.340669+0000","last_clean":"2023-10-16T11:25:51.340669+0000","last_became_active":"2023-10-16T11:25:50.300956+0000","last_became_peered":"2023-10-16T11:25:50.300956+0000","last_unstale":"2023-10-16T11:25:51.340669+0000","last_undegraded":"2023-10-16T11:25:51.340669+0000","last_fullsized":"2023-10-16T11:25:51.340669+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:31.156993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.24","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.310834+0000","last_change":"2023-10-16T11:25:37.170623+0000","last_active":"2023-10-16T11:25:51.310834+0000","last_peered":"2023-10-16T11:25:51.310834+0000","last_clean":"2023-10-16T11:25:51.310834+0000","last_became_active":"2023-10-16T11:25:37.170365+0000","last_became_peered":"2023-10-16T11:25:37.170365+0000","last_unstale":"2023-10-16T11:25:51.310834+0000","last_undegraded":"2023-10-16T11:25:51.310834+0000","last_fullsized":"2023-10-16T11:25:51.310834+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:52.443160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.318419+0000","last_change":"2023-10-16T11:25:51.335417+0000","last_active":"2023-10-16T11:25:52.318419+0000","last_peered":"2023-10-16T11:25:52.318419+0000","last_clean":"2023-10-16T11:25:52.318419+0000","last_became_active":"2023-10-16T11:25:51.335099+0000","last_became_peered":"2023-10-16T11:25:51.335099+0000","last_unstale":"2023-10-16T11:25:52.318419+0000","last_undegraded":"2023-10-16T11:25:52.318419+0000","last_fullsized":"2023-10-16T11:25:52.318419+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:57:00.058585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"240'2","reported_seq":14,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.347122+0000","last_change":"2023-10-16T11:25:50.294581+0000","last_active":"2023-10-16T11:25:51.347122+0000","last_peered":"2023-10-16T11:25:51.347122+0000","last_clean":"2023-10-16T11:25:51.347122+0000","last_became_active":"2023-10-16T11:25:50.294303+0000","last_became_peered":"2023-10-16T11:25:50.294303+0000","last_unstale":"2023-10-16T11:25:51.347122+0000","last_undegraded":"2023-10-16T11:25:51.347122+0000","last_fullsized":"2023-10-16T11:25:51.347122+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:49.447144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.23","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317555+0000","last_change":"2023-10-16T11:25:37.178693+0000","last_active":"2023-10-16T11:25:51.317555+0000","last_peered":"2023-10-16T11:25:51.317555+0000","last_clean":"2023-10-16T11:25:51.317555+0000","last_became_active":"2023-10-16T11:25:37.175403+0000","last_became_peered":"2023-10-16T11:25:37.175403+0000","last_unstale":"2023-10-16T11:25:51.317555+0000","last_undegraded":"2023-10-16T11:25:51.317555+0000","last_fullsized":"2023-10-16T11:25:51.317555+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:15:31.572558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.788897+0000","last_change":"2023-10-16T11:25:51.324613+0000","last_active":"2023-10-16T11:25:52.788897+0000","last_peered":"2023-10-16T11:25:52.788897+0000","last_clean":"2023-10-16T11:25:52.788897+0000","last_became_active":"2023-10-16T11:25:51.323228+0000","last_became_peered":"2023-10-16T11:25:51.323228+0000","last_unstale":"2023-10-16T11:25:52.788897+0000","last_undegraded":"2023-10-16T11:25:52.788897+0000","last_fullsized":"2023-10-16T11:25:52.788897+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:14:07.754755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"240'2","reported_seq":14,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.347956+0000","last_change":"2023-10-16T11:25:50.308596+0000","last_active":"2023-10-16T11:25:51.347956+0000","last_peered":"2023-10-16T11:25:51.347956+0000","last_clean":"2023-10-16T11:25:51.347956+0000","last_became_active":"2023-10-16T11:25:50.308237+0000","last_became_peered":"2023-10-16T11:25:50.308237+0000","last_unstale":"2023-10-16T11:25:51.347956+0000","last_undegraded":"2023-10-16T11:25:51.347956+0000","last_fullsized":"2023-10-16T11:25:51.347956+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:16:43.826775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.22","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.324091+0000","last_change":"2023-10-16T11:25:37.181812+0000","last_active":"2023-10-16T11:25:51.324091+0000","last_peered":"2023-10-16T11:25:51.324091+0000","last_clean":"2023-10-16T11:25:51.324091+0000","last_became_active":"2023-10-16T11:25:37.181529+0000","last_became_peered":"2023-10-16T11:25:37.181529+0000","last_unstale":"2023-10-16T11:25:51.324091+0000","last_undegraded":"2023-10-16T11:25:51.324091+0000","last_fullsized":"2023-10-16T11:25:51.324091+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:35.912014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.322190+0000","last_change":"2023-10-16T11:25:51.326648+0000","last_active":"2023-10-16T11:25:52.322190+0000","last_peered":"2023-10-16T11:25:52.322190+0000","last_clean":"2023-10-16T11:25:52.322190+0000","last_became_active":"2023-10-16T11:25:51.314971+0000","last_became_peered":"2023-10-16T11:25:51.314971+0000","last_unstale":"2023-10-16T11:25:52.322190+0000","last_undegraded":"2023-10-16T11:25:52.322190+0000","last_fullsized":"2023-10-16T11:25:52.322190+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:32.743783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"240'1","reported_seq":13,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.346766+0000","last_change":"2023-10-16T11:25:50.290455+0000","last_active":"2023-10-16T11:25:51.346766+0000","last_peered":"2023-10-16T11:25:51.346766+0000","last_clean":"2023-10-16T11:25:51.346766+0000","last_became_active":"2023-10-16T11:25:50.290130+0000","last_became_peered":"2023-10-16T11:25:50.290130+0000","last_unstale":"2023-10-16T11:25:51.346766+0000","last_undegraded":"2023-10-16T11:25:51.346766+0000","last_fullsized":"2023-10-16T11:25:51.346766+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:01:55.571004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.21","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.294716+0000","last_change":"2023-10-16T11:25:37.166981+0000","last_active":"2023-10-16T11:25:51.294716+0000","last_peered":"2023-10-16T11:25:51.294716+0000","last_clean":"2023-10-16T11:25:51.294716+0000","last_became_active":"2023-10-16T11:25:37.164147+0000","last_became_peered":"2023-10-16T11:25:37.164147+0000","last_unstale":"2023-10-16T11:25:51.294716+0000","last_undegraded":"2023-10-16T11:25:51.294716+0000","last_fullsized":"2023-10-16T11:25:51.294716+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.790718+0000","last_change":"2023-10-16T11:25:51.328794+0000","last_active":"2023-10-16T11:25:52.790718+0000","last_peered":"2023-10-16T11:25:52.790718+0000","last_clean":"2023-10-16T11:25:52.790718+0000","last_became_active":"2023-10-16T11:25:51.328364+0000","last_became_peered":"2023-10-16T11:25:51.328364+0000","last_unstale":"2023-10-16T11:25:52.790718+0000","last_undegraded":"2023-10-16T11:25:52.790718+0000","last_fullsized":"2023-10-16T11:25:52.790718+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:21.934164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"240'4","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.348348+0000","last_change":"2023-10-16T11:25:50.300080+0000","last_active":"2023-10-16T11:25:51.348348+0000","last_peered":"2023-10-16T11:25:51.348348+0000","last_clean":"2023-10-16T11:25:51.348348+0000","last_became_active":"2023-10-16T11:25:50.299271+0000","last_became_peered":"2023-10-16T11:25:50.299271+0000","last_unstale":"2023-10-16T11:25:51.348348+0000","last_undegraded":"2023-10-16T11:25:51.348348+0000","last_fullsized":"2023-10-16T11:25:51.348348+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:30:50.501877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.20","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.312252+0000","last_change":"2023-10-16T11:25:37.178631+0000","last_active":"2023-10-16T11:25:51.312252+0000","last_peered":"2023-10-16T11:25:51.312252+0000","last_clean":"2023-10-16T11:25:51.312252+0000","last_became_active":"2023-10-16T11:25:37.167402+0000","last_became_peered":"2023-10-16T11:25:37.167402+0000","last_unstale":"2023-10-16T11:25:51.312252+0000","last_undegraded":"2023-10-16T11:25:51.312252+0000","last_fullsized":"2023-10-16T11:25:51.312252+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:55.572264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.316346+0000","last_change":"2023-10-16T11:25:51.325042+0000","last_active":"2023-10-16T11:25:52.316346+0000","last_peered":"2023-10-16T11:25:52.316346+0000","last_clean":"2023-10-16T11:25:52.316346+0000","last_became_active":"2023-10-16T11:25:51.324731+0000","last_became_peered":"2023-10-16T11:25:51.324731+0000","last_unstale":"2023-10-16T11:25:52.316346+0000","last_undegraded":"2023-10-16T11:25:52.316346+0000","last_fullsized":"2023-10-16T11:25:52.316346+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:48.937625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.308890+0000","last_change":"2023-10-16T11:25:37.177706+0000","last_active":"2023-10-16T11:25:51.308890+0000","last_peered":"2023-10-16T11:25:51.308890+0000","last_clean":"2023-10-16T11:25:51.308890+0000","last_became_active":"2023-10-16T11:25:37.176806+0000","last_became_peered":"2023-10-16T11:25:37.176806+0000","last_unstale":"2023-10-16T11:25:51.308890+0000","last_undegraded":"2023-10-16T11:25:51.308890+0000","last_fullsized":"2023-10-16T11:25:51.308890+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:42:32.585642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1e","version":"236'1","reported_seq":17,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.313596+0000","last_change":"2023-10-16T11:25:36.169327+0000","last_active":"2023-10-16T11:25:51.313596+0000","last_peered":"2023-10-16T11:25:51.313596+0000","last_clean":"2023-10-16T11:25:51.313596+0000","last_became_active":"2023-10-16T11:25:36.169046+0000","last_became_peered":"2023-10-16T11:25:36.169046+0000","last_unstale":"2023-10-16T11:25:51.313596+0000","last_undegraded":"2023-10-16T11:25:51.313596+0000","last_fullsized":"2023-10-16T11:25:51.313596+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:34.899695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.318653+0000","last_change":"2023-10-16T11:25:51.324955+0000","last_active":"2023-10-16T11:25:52.318653+0000","last_peered":"2023-10-16T11:25:52.318653+0000","last_clean":"2023-10-16T11:25:52.318653+0000","last_became_active":"2023-10-16T11:25:51.314480+0000","last_became_peered":"2023-10-16T11:25:51.314480+0000","last_unstale":"2023-10-16T11:25:52.318653+0000","last_undegraded":"2023-10-16T11:25:52.318653+0000","last_fullsized":"2023-10-16T11:25:52.318653+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:57:00.058585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317994+0000","last_change":"2023-10-16T11:25:37.173736+0000","last_active":"2023-10-16T11:25:51.317994+0000","last_peered":"2023-10-16T11:25:51.317994+0000","last_clean":"2023-10-16T11:25:51.317994+0000","last_became_active":"2023-10-16T11:25:37.173482+0000","last_became_peered":"2023-10-16T11:25:37.173482+0000","last_unstale":"2023-10-16T11:25:51.317994+0000","last_undegraded":"2023-10-16T11:25:51.317994+0000","last_fullsized":"2023-10-16T11:25:51.317994+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:42:23.188876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1d","version":"236'2","reported_seq":18,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.306367+0000","last_change":"2023-10-16T11:25:36.172861+0000","last_active":"2023-10-16T11:25:51.306367+0000","last_peered":"2023-10-16T11:25:51.306367+0000","last_clean":"2023-10-16T11:25:51.306367+0000","last_became_active":"2023-10-16T11:25:36.172574+0000","last_became_peered":"2023-10-16T11:25:36.172574+0000","last_unstale":"2023-10-16T11:25:51.306367+0000","last_undegraded":"2023-10-16T11:25:51.306367+0000","last_fullsized":"2023-10-16T11:25:51.306367+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:22:17.667795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789424+0000","last_change":"2023-10-16T11:25:51.317976+0000","last_active":"2023-10-16T11:25:52.789424+0000","last_peered":"2023-10-16T11:25:52.789424+0000","last_clean":"2023-10-16T11:25:52.789424+0000","last_became_active":"2023-10-16T11:25:51.317525+0000","last_became_peered":"2023-10-16T11:25:51.317525+0000","last_unstale":"2023-10-16T11:25:52.789424+0000","last_undegraded":"2023-10-16T11:25:52.789424+0000","last_fullsized":"2023-10-16T11:25:52.789424+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:29.234750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.298248+0000","last_change":"2023-10-16T11:25:37.167451+0000","last_active":"2023-10-16T11:25:51.298248+0000","last_peered":"2023-10-16T11:25:51.298248+0000","last_clean":"2023-10-16T11:25:51.298248+0000","last_became_active":"2023-10-16T11:25:37.167157+0000","last_became_peered":"2023-10-16T11:25:37.167157+0000","last_unstale":"2023-10-16T11:25:51.298248+0000","last_undegraded":"2023-10-16T11:25:51.298248+0000","last_fullsized":"2023-10-16T11:25:51.298248+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:14.504702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1c","version":"236'2","reported_seq":18,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.302706+0000","last_change":"2023-10-16T11:25:36.153273+0000","last_active":"2023-10-16T11:25:51.302706+0000","last_peered":"2023-10-16T11:25:51.302706+0000","last_clean":"2023-10-16T11:25:51.302706+0000","last_became_active":"2023-10-16T11:25:36.152960+0000","last_became_peered":"2023-10-16T11:25:36.152960+0000","last_unstale":"2023-10-16T11:25:51.302706+0000","last_undegraded":"2023-10-16T11:25:51.302706+0000","last_fullsized":"2023-10-16T11:25:51.302706+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:08.218268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.320150+0000","last_change":"2023-10-16T11:25:51.328191+0000","last_active":"2023-10-16T11:25:52.320150+0000","last_peered":"2023-10-16T11:25:52.320150+0000","last_clean":"2023-10-16T11:25:52.320150+0000","last_became_active":"2023-10-16T11:25:51.327805+0000","last_became_peered":"2023-10-16T11:25:51.327805+0000","last_unstale":"2023-10-16T11:25:52.320150+0000","last_undegraded":"2023-10-16T11:25:52.320150+0000","last_fullsized":"2023-10-16T11:25:52.320150+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:37.288954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.323547+0000","last_change":"2023-10-16T11:25:37.174071+0000","last_active":"2023-10-16T11:25:51.323547+0000","last_peered":"2023-10-16T11:25:51.323547+0000","last_clean":"2023-10-16T11:25:51.323547+0000","last_became_active":"2023-10-16T11:25:37.173436+0000","last_became_peered":"2023-10-16T11:25:37.173436+0000","last_unstale":"2023-10-16T11:25:51.323547+0000","last_undegraded":"2023-10-16T11:25:51.323547+0000","last_fullsized":"2023-10-16T11:25:51.323547+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1b","version":"236'1","reported_seq":17,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.303522+0000","last_change":"2023-10-16T11:25:36.156408+0000","last_active":"2023-10-16T11:25:51.303522+0000","last_peered":"2023-10-16T11:25:51.303522+0000","last_clean":"2023-10-16T11:25:51.303522+0000","last_became_active":"2023-10-16T11:25:36.155786+0000","last_became_peered":"2023-10-16T11:25:36.155786+0000","last_unstale":"2023-10-16T11:25:51.303522+0000","last_undegraded":"2023-10-16T11:25:51.303522+0000","last_fullsized":"2023-10-16T11:25:51.303522+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:01.925277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789918+0000","last_change":"2023-10-16T11:25:51.332041+0000","last_active":"2023-10-16T11:25:52.789918+0000","last_peered":"2023-10-16T11:25:52.789918+0000","last_clean":"2023-10-16T11:25:52.789918+0000","last_became_active":"2023-10-16T11:25:51.329122+0000","last_became_peered":"2023-10-16T11:25:51.329122+0000","last_unstale":"2023-10-16T11:25:52.789918+0000","last_undegraded":"2023-10-16T11:25:52.789918+0000","last_fullsized":"2023-10-16T11:25:52.789918+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:32.743783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317182+0000","last_change":"2023-10-16T11:25:37.179656+0000","last_active":"2023-10-16T11:25:51.317182+0000","last_peered":"2023-10-16T11:25:51.317182+0000","last_clean":"2023-10-16T11:25:51.317182+0000","last_became_active":"2023-10-16T11:25:37.179361+0000","last_became_peered":"2023-10-16T11:25:37.179361+0000","last_unstale":"2023-10-16T11:25:51.317182+0000","last_undegraded":"2023-10-16T11:25:51.317182+0000","last_fullsized":"2023-10-16T11:25:51.317182+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:09.230587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"236'1","reported_seq":17,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.313144+0000","last_change":"2023-10-16T11:25:36.169348+0000","last_active":"2023-10-16T11:25:51.313144+0000","last_peered":"2023-10-16T11:25:51.313144+0000","last_clean":"2023-10-16T11:25:51.313144+0000","last_became_active":"2023-10-16T11:25:36.169053+0000","last_became_peered":"2023-10-16T11:25:36.169053+0000","last_unstale":"2023-10-16T11:25:51.313144+0000","last_undegraded":"2023-10-16T11:25:51.313144+0000","last_fullsized":"2023-10-16T11:25:51.313144+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:05.819502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.724196+0000","last_change":"2023-10-16T11:25:51.327481+0000","last_active":"2023-10-16T11:25:52.724196+0000","last_peered":"2023-10-16T11:25:52.724196+0000","last_clean":"2023-10-16T11:25:52.724196+0000","last_became_active":"2023-10-16T11:25:51.327206+0000","last_became_peered":"2023-10-16T11:25:51.327206+0000","last_unstale":"2023-10-16T11:25:52.724196+0000","last_undegraded":"2023-10-16T11:25:52.724196+0000","last_fullsized":"2023-10-16T11:25:52.724196+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:31:21.411627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.302848+0000","last_change":"2023-10-16T11:25:37.176966+0000","last_active":"2023-10-16T11:25:51.302848+0000","last_peered":"2023-10-16T11:25:51.302848+0000","last_clean":"2023-10-16T11:25:51.302848+0000","last_became_active":"2023-10-16T11:25:37.174621+0000","last_became_peered":"2023-10-16T11:25:37.174621+0000","last_unstale":"2023-10-16T11:25:51.302848+0000","last_undegraded":"2023-10-16T11:25:51.302848+0000","last_fullsized":"2023-10-16T11:25:51.302848+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:56:09.516122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317917+0000","last_change":"2023-10-16T11:25:36.169635+0000","last_active":"2023-10-16T11:25:51.317917+0000","last_peered":"2023-10-16T11:25:51.317917+0000","last_clean":"2023-10-16T11:25:51.317917+0000","last_became_active":"2023-10-16T11:25:36.169016+0000","last_became_peered":"2023-10-16T11:25:36.169016+0000","last_unstale":"2023-10-16T11:25:51.317917+0000","last_undegraded":"2023-10-16T11:25:51.317917+0000","last_fullsized":"2023-10-16T11:25:51.317917+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:53:06.412234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.322296+0000","last_change":"2023-10-16T11:25:51.327247+0000","last_active":"2023-10-16T11:25:52.322296+0000","last_peered":"2023-10-16T11:25:52.322296+0000","last_clean":"2023-10-16T11:25:52.322296+0000","last_became_active":"2023-10-16T11:25:51.326955+0000","last_became_peered":"2023-10-16T11:25:51.326955+0000","last_unstale":"2023-10-16T11:25:52.322296+0000","last_undegraded":"2023-10-16T11:25:52.322296+0000","last_fullsized":"2023-10-16T11:25:52.322296+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:59:08.686800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.310236+0000","last_change":"2023-10-16T11:25:37.184475+0000","last_active":"2023-10-16T11:25:51.310236+0000","last_peered":"2023-10-16T11:25:51.310236+0000","last_clean":"2023-10-16T11:25:51.310236+0000","last_became_active":"2023-10-16T11:25:37.179914+0000","last_became_peered":"2023-10-16T11:25:37.179914+0000","last_unstale":"2023-10-16T11:25:51.310236+0000","last_undegraded":"2023-10-16T11:25:51.310236+0000","last_fullsized":"2023-10-16T11:25:51.310236+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:53.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.308698+0000","last_change":"2023-10-16T11:25:36.155589+0000","last_active":"2023-10-16T11:25:51.308698+0000","last_peered":"2023-10-16T11:25:51.308698+0000","last_clean":"2023-10-16T11:25:51.308698+0000","last_became_active":"2023-10-16T11:25:36.155087+0000","last_became_peered":"2023-10-16T11:25:36.155087+0000","last_unstale":"2023-10-16T11:25:51.308698+0000","last_undegraded":"2023-10-16T11:25:51.308698+0000","last_fullsized":"2023-10-16T11:25:51.308698+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:52:38.937212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789190+0000","last_change":"2023-10-16T11:25:51.321470+0000","last_active":"2023-10-16T11:25:52.789190+0000","last_peered":"2023-10-16T11:25:52.789190+0000","last_clean":"2023-10-16T11:25:52.789190+0000","last_became_active":"2023-10-16T11:25:51.320974+0000","last_became_peered":"2023-10-16T11:25:51.320974+0000","last_unstale":"2023-10-16T11:25:52.789190+0000","last_undegraded":"2023-10-16T11:25:52.789190+0000","last_fullsized":"2023-10-16T11:25:52.789190+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:23.241770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.315243+0000","last_change":"2023-10-16T11:25:37.175447+0000","last_active":"2023-10-16T11:25:51.315243+0000","last_peered":"2023-10-16T11:25:51.315243+0000","last_clean":"2023-10-16T11:25:51.315243+0000","last_became_active":"2023-10-16T11:25:37.173563+0000","last_became_peered":"2023-10-16T11:25:37.173563+0000","last_unstale":"2023-10-16T11:25:51.315243+0000","last_undegraded":"2023-10-16T11:25:51.315243+0000","last_fullsized":"2023-10-16T11:25:51.315243+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:54.544135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.301615+0000","last_change":"2023-10-16T11:25:36.169351+0000","last_active":"2023-10-16T11:25:51.301615+0000","last_peered":"2023-10-16T11:25:51.301615+0000","last_clean":"2023-10-16T11:25:51.301615+0000","last_became_active":"2023-10-16T11:25:36.168347+0000","last_became_peered":"2023-10-16T11:25:36.168347+0000","last_unstale":"2023-10-16T11:25:51.301615+0000","last_undegraded":"2023-10-16T11:25:51.301615+0000","last_fullsized":"2023-10-16T11:25:51.301615+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:19.077096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.788244+0000","last_change":"2023-10-16T11:25:51.333596+0000","last_active":"2023-10-16T11:25:52.788244+0000","last_peered":"2023-10-16T11:25:52.788244+0000","last_clean":"2023-10-16T11:25:52.788244+0000","last_became_active":"2023-10-16T11:25:51.333175+0000","last_became_peered":"2023-10-16T11:25:51.333175+0000","last_unstale":"2023-10-16T11:25:52.788244+0000","last_undegraded":"2023-10-16T11:25:52.788244+0000","last_fullsized":"2023-10-16T11:25:52.788244+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:49.517942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.316449+0000","last_change":"2023-10-16T11:25:37.179048+0000","last_active":"2023-10-16T11:25:51.316449+0000","last_peered":"2023-10-16T11:25:51.316449+0000","last_clean":"2023-10-16T11:25:51.316449+0000","last_became_active":"2023-10-16T11:25:37.178751+0000","last_became_peered":"2023-10-16T11:25:37.178751+0000","last_unstale":"2023-10-16T11:25:51.316449+0000","last_undegraded":"2023-10-16T11:25:51.316449+0000","last_fullsized":"2023-10-16T11:25:51.316449+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:48:10.906108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.306108+0000","last_change":"2023-10-16T11:25:36.154256+0000","last_active":"2023-10-16T11:25:51.306108+0000","last_peered":"2023-10-16T11:25:51.306108+0000","last_clean":"2023-10-16T11:25:51.306108+0000","last_became_active":"2023-10-16T11:25:36.153421+0000","last_became_peered":"2023-10-16T11:25:36.153421+0000","last_unstale":"2023-10-16T11:25:51.306108+0000","last_undegraded":"2023-10-16T11:25:51.306108+0000","last_fullsized":"2023-10-16T11:25:51.306108+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:09.436508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.315787+0000","last_change":"2023-10-16T11:25:51.336271+0000","last_active":"2023-10-16T11:25:52.315787+0000","last_peered":"2023-10-16T11:25:52.315787+0000","last_clean":"2023-10-16T11:25:52.315787+0000","last_became_active":"2023-10-16T11:25:51.335956+0000","last_became_peered":"2023-10-16T11:25:51.335956+0000","last_unstale":"2023-10-16T11:25:52.315787+0000","last_undegraded":"2023-10-16T11:25:52.315787+0000","last_fullsized":"2023-10-16T11:25:52.315787+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:48.937625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.302135+0000","last_change":"2023-10-16T11:25:37.188270+0000","last_active":"2023-10-16T11:25:51.302135+0000","last_peered":"2023-10-16T11:25:51.302135+0000","last_clean":"2023-10-16T11:25:51.302135+0000","last_became_active":"2023-10-16T11:25:37.187995+0000","last_became_peered":"2023-10-16T11:25:37.187995+0000","last_unstale":"2023-10-16T11:25:51.302135+0000","last_undegraded":"2023-10-16T11:25:51.302135+0000","last_fullsized":"2023-10-16T11:25:51.302135+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:42:22.542072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.15","version":"236'3","reported_seq":19,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.315149+0000","last_change":"2023-10-16T11:25:36.171645+0000","last_active":"2023-10-16T11:25:51.315149+0000","last_peered":"2023-10-16T11:25:51.315149+0000","last_clean":"2023-10-16T11:25:51.315149+0000","last_became_active":"2023-10-16T11:25:36.170729+0000","last_became_peered":"2023-10-16T11:25:36.170729+0000","last_unstale":"2023-10-16T11:25:51.315149+0000","last_undegraded":"2023-10-16T11:25:51.315149+0000","last_fullsized":"2023-10-16T11:25:51.315149+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:34.362958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.317193+0000","last_change":"2023-10-16T11:25:51.324637+0000","last_active":"2023-10-16T11:25:52.317193+0000","last_peered":"2023-10-16T11:25:52.317193+0000","last_clean":"2023-10-16T11:25:52.317193+0000","last_became_active":"2023-10-16T11:25:51.323712+0000","last_became_peered":"2023-10-16T11:25:51.323712+0000","last_unstale":"2023-10-16T11:25:52.317193+0000","last_undegraded":"2023-10-16T11:25:52.317193+0000","last_fullsized":"2023-10-16T11:25:52.317193+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:57:00.058585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.312295+0000","last_change":"2023-10-16T11:25:37.163991+0000","last_active":"2023-10-16T11:25:51.312295+0000","last_peered":"2023-10-16T11:25:51.312295+0000","last_clean":"2023-10-16T11:25:51.312295+0000","last_became_active":"2023-10-16T11:25:37.163080+0000","last_became_peered":"2023-10-16T11:25:37.163080+0000","last_unstale":"2023-10-16T11:25:51.312295+0000","last_undegraded":"2023-10-16T11:25:51.312295+0000","last_fullsized":"2023-10-16T11:25:51.312295+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.14","version":"237'8","reported_seq":24,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.310933+0000","last_change":"2023-10-16T11:25:36.173259+0000","last_active":"2023-10-16T11:25:51.310933+0000","last_peered":"2023-10-16T11:25:51.310933+0000","last_clean":"2023-10-16T11:25:51.310933+0000","last_became_active":"2023-10-16T11:25:36.172980+0000","last_became_peered":"2023-10-16T11:25:36.172980+0000","last_unstale":"2023-10-16T11:25:51.310933+0000","last_undegraded":"2023-10-16T11:25:51.310933+0000","last_fullsized":"2023-10-16T11:25:51.310933+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:33.782641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.788001+0000","last_change":"2023-10-16T11:25:51.325646+0000","last_active":"2023-10-16T11:25:52.788001+0000","last_peered":"2023-10-16T11:25:52.788001+0000","last_clean":"2023-10-16T11:25:52.788001+0000","last_became_active":"2023-10-16T11:25:51.325326+0000","last_became_peered":"2023-10-16T11:25:51.325326+0000","last_unstale":"2023-10-16T11:25:52.788001+0000","last_undegraded":"2023-10-16T11:25:52.788001+0000","last_fullsized":"2023-10-16T11:25:52.788001+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:01.749517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.313880+0000","last_change":"2023-10-16T11:25:37.166081+0000","last_active":"2023-10-16T11:25:51.313880+0000","last_peered":"2023-10-16T11:25:51.313880+0000","last_clean":"2023-10-16T11:25:51.313880+0000","last_became_active":"2023-10-16T11:25:37.165150+0000","last_became_peered":"2023-10-16T11:25:37.165150+0000","last_unstale":"2023-10-16T11:25:51.313880+0000","last_undegraded":"2023-10-16T11:25:51.313880+0000","last_fullsized":"2023-10-16T11:25:51.313880+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:56.314696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"236'2","reported_seq":18,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.314626+0000","last_change":"2023-10-16T11:25:36.171468+0000","last_active":"2023-10-16T11:25:51.314626+0000","last_peered":"2023-10-16T11:25:51.314626+0000","last_clean":"2023-10-16T11:25:51.314626+0000","last_became_active":"2023-10-16T11:25:36.171236+0000","last_became_peered":"2023-10-16T11:25:36.171236+0000","last_unstale":"2023-10-16T11:25:51.314626+0000","last_undegraded":"2023-10-16T11:25:51.314626+0000","last_fullsized":"2023-10-16T11:25:51.314626+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:17.018903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789810+0000","last_change":"2023-10-16T11:25:51.327239+0000","last_active":"2023-10-16T11:25:52.789810+0000","last_peered":"2023-10-16T11:25:52.789810+0000","last_clean":"2023-10-16T11:25:52.789810+0000","last_became_active":"2023-10-16T11:25:51.326951+0000","last_became_peered":"2023-10-16T11:25:51.326951+0000","last_unstale":"2023-10-16T11:25:52.789810+0000","last_undegraded":"2023-10-16T11:25:52.789810+0000","last_fullsized":"2023-10-16T11:25:52.789810+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:11.420914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317290+0000","last_change":"2023-10-16T11:25:37.178517+0000","last_active":"2023-10-16T11:25:51.317290+0000","last_peered":"2023-10-16T11:25:51.317290+0000","last_clean":"2023-10-16T11:25:51.317290+0000","last_became_active":"2023-10-16T11:25:37.177794+0000","last_became_peered":"2023-10-16T11:25:37.177794+0000","last_unstale":"2023-10-16T11:25:51.317290+0000","last_undegraded":"2023-10-16T11:25:51.317290+0000","last_fullsized":"2023-10-16T11:25:51.317290+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:52:39.949531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.300023+0000","last_change":"2023-10-16T11:25:36.162595+0000","last_active":"2023-10-16T11:25:51.300023+0000","last_peered":"2023-10-16T11:25:51.300023+0000","last_clean":"2023-10-16T11:25:51.300023+0000","last_became_active":"2023-10-16T11:25:36.162249+0000","last_became_peered":"2023-10-16T11:25:36.162249+0000","last_unstale":"2023-10-16T11:25:51.300023+0000","last_undegraded":"2023-10-16T11:25:51.300023+0000","last_fullsized":"2023-10-16T11:25:51.300023+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:49.547165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789988+0000","last_change":"2023-10-16T11:25:51.324884+0000","last_active":"2023-10-16T11:25:52.789988+0000","last_peered":"2023-10-16T11:25:52.789988+0000","last_clean":"2023-10-16T11:25:52.789988+0000","last_became_active":"2023-10-16T11:25:51.320528+0000","last_became_peered":"2023-10-16T11:25:51.320528+0000","last_unstale":"2023-10-16T11:25:52.789988+0000","last_undegraded":"2023-10-16T11:25:52.789988+0000","last_fullsized":"2023-10-16T11:25:52.789988+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:59:08.686800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.293418+0000","last_change":"2023-10-16T11:25:37.164604+0000","last_active":"2023-10-16T11:25:51.293418+0000","last_peered":"2023-10-16T11:25:51.293418+0000","last_clean":"2023-10-16T11:25:51.293418+0000","last_became_active":"2023-10-16T11:25:37.162365+0000","last_became_peered":"2023-10-16T11:25:37.162365+0000","last_unstale":"2023-10-16T11:25:51.293418+0000","last_undegraded":"2023-10-16T11:25:51.293418+0000","last_fullsized":"2023-10-16T11:25:51.293418+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:35.912014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317999+0000","last_change":"2023-10-16T11:25:36.170014+0000","last_active":"2023-10-16T11:25:51.317999+0000","last_peered":"2023-10-16T11:25:51.317999+0000","last_clean":"2023-10-16T11:25:51.317999+0000","last_became_active":"2023-10-16T11:25:36.169755+0000","last_became_peered":"2023-10-16T11:25:36.169755+0000","last_unstale":"2023-10-16T11:25:51.317999+0000","last_undegraded":"2023-10-16T11:25:51.317999+0000","last_fullsized":"2023-10-16T11:25:51.317999+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:02.119904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.791648+0000","last_change":"2023-10-16T11:25:51.316368+0000","last_active":"2023-10-16T11:25:52.791648+0000","last_peered":"2023-10-16T11:25:52.791648+0000","last_clean":"2023-10-16T11:25:52.791648+0000","last_became_active":"2023-10-16T11:25:51.314352+0000","last_became_peered":"2023-10-16T11:25:51.314352+0000","last_unstale":"2023-10-16T11:25:52.791648+0000","last_undegraded":"2023-10-16T11:25:52.791648+0000","last_fullsized":"2023-10-16T11:25:52.791648+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:32:44.818063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317829+0000","last_change":"2023-10-16T11:25:37.178789+0000","last_active":"2023-10-16T11:25:51.317829+0000","last_peered":"2023-10-16T11:25:51.317829+0000","last_clean":"2023-10-16T11:25:51.317829+0000","last_became_active":"2023-10-16T11:25:37.178510+0000","last_became_peered":"2023-10-16T11:25:37.178510+0000","last_unstale":"2023-10-16T11:25:51.317829+0000","last_undegraded":"2023-10-16T11:25:51.317829+0000","last_fullsized":"2023-10-16T11:25:51.317829+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.10","version":"236'8","reported_seq":24,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.315707+0000","last_change":"2023-10-16T11:25:36.168889+0000","last_active":"2023-10-16T11:25:51.315707+0000","last_peered":"2023-10-16T11:25:51.315707+0000","last_clean":"2023-10-16T11:25:51.315707+0000","last_became_active":"2023-10-16T11:25:36.168590+0000","last_became_peered":"2023-10-16T11:25:36.168590+0000","last_unstale":"2023-10-16T11:25:51.315707+0000","last_undegraded":"2023-10-16T11:25:51.315707+0000","last_fullsized":"2023-10-16T11:25:51.315707+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:42.801560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.787261+0000","last_change":"2023-10-16T11:25:51.318378+0000","last_active":"2023-10-16T11:25:52.787261+0000","last_peered":"2023-10-16T11:25:52.787261+0000","last_clean":"2023-10-16T11:25:52.787261+0000","last_became_active":"2023-10-16T11:25:51.318080+0000","last_became_peered":"2023-10-16T11:25:51.318080+0000","last_unstale":"2023-10-16T11:25:52.787261+0000","last_undegraded":"2023-10-16T11:25:52.787261+0000","last_fullsized":"2023-10-16T11:25:52.787261+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:02:43.323091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.300112+0000","last_change":"2023-10-16T11:25:37.175992+0000","last_active":"2023-10-16T11:25:51.300112+0000","last_peered":"2023-10-16T11:25:51.300112+0000","last_clean":"2023-10-16T11:25:51.300112+0000","last_became_active":"2023-10-16T11:25:37.173732+0000","last_became_peered":"2023-10-16T11:25:37.173732+0000","last_unstale":"2023-10-16T11:25:51.300112+0000","last_undegraded":"2023-10-16T11:25:51.300112+0000","last_fullsized":"2023-10-16T11:25:51.300112+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.f","version":"236'1","reported_seq":17,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.316299+0000","last_change":"2023-10-16T11:25:36.173935+0000","last_active":"2023-10-16T11:25:51.316299+0000","last_peered":"2023-10-16T11:25:51.316299+0000","last_clean":"2023-10-16T11:25:51.316299+0000","last_became_active":"2023-10-16T11:25:36.173510+0000","last_became_peered":"2023-10-16T11:25:36.173510+0000","last_unstale":"2023-10-16T11:25:51.316299+0000","last_undegraded":"2023-10-16T11:25:51.316299+0000","last_fullsized":"2023-10-16T11:25:51.316299+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:19.077096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.317141+0000","last_change":"2023-10-16T11:25:51.318620+0000","last_active":"2023-10-16T11:25:52.317141+0000","last_peered":"2023-10-16T11:25:52.317141+0000","last_clean":"2023-10-16T11:25:52.317141+0000","last_became_active":"2023-10-16T11:25:51.311970+0000","last_became_peered":"2023-10-16T11:25:51.311970+0000","last_unstale":"2023-10-16T11:25:52.317141+0000","last_undegraded":"2023-10-16T11:25:52.317141+0000","last_fullsized":"2023-10-16T11:25:52.317141+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:48.937625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.307638+0000","last_change":"2023-10-16T11:25:37.177543+0000","last_active":"2023-10-16T11:25:51.307638+0000","last_peered":"2023-10-16T11:25:51.307638+0000","last_clean":"2023-10-16T11:25:51.307638+0000","last_became_active":"2023-10-16T11:25:37.176420+0000","last_became_peered":"2023-10-16T11:25:37.176420+0000","last_unstale":"2023-10-16T11:25:51.307638+0000","last_undegraded":"2023-10-16T11:25:51.307638+0000","last_fullsized":"2023-10-16T11:25:51.307638+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:24.104981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.e","version":"236'2","reported_seq":18,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.306267+0000","last_change":"2023-10-16T11:25:36.170878+0000","last_active":"2023-10-16T11:25:51.306267+0000","last_peered":"2023-10-16T11:25:51.306267+0000","last_clean":"2023-10-16T11:25:51.306267+0000","last_became_active":"2023-10-16T11:25:36.170548+0000","last_became_peered":"2023-10-16T11:25:36.170548+0000","last_unstale":"2023-10-16T11:25:51.306267+0000","last_undegraded":"2023-10-16T11:25:51.306267+0000","last_fullsized":"2023-10-16T11:25:51.306267+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:34:14.932409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.318304+0000","last_change":"2023-10-16T11:25:51.306315+0000","last_active":"2023-10-16T11:25:52.318304+0000","last_peered":"2023-10-16T11:25:52.318304+0000","last_clean":"2023-10-16T11:25:52.318304+0000","last_became_active":"2023-10-16T11:25:51.305603+0000","last_became_peered":"2023-10-16T11:25:51.305603+0000","last_unstale":"2023-10-16T11:25:52.318304+0000","last_undegraded":"2023-10-16T11:25:52.318304+0000","last_fullsized":"2023-10-16T11:25:52.318304+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:49.517942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317969+0000","last_change":"2023-10-16T11:25:37.176420+0000","last_active":"2023-10-16T11:25:51.317969+0000","last_peered":"2023-10-16T11:25:51.317969+0000","last_clean":"2023-10-16T11:25:51.317969+0000","last_became_active":"2023-10-16T11:25:37.176101+0000","last_became_peered":"2023-10-16T11:25:37.176101+0000","last_unstale":"2023-10-16T11:25:51.317969+0000","last_undegraded":"2023-10-16T11:25:51.317969+0000","last_fullsized":"2023-10-16T11:25:51.317969+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:10.245184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"236'1","reported_seq":17,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.320217+0000","last_change":"2023-10-16T11:25:36.171476+0000","last_active":"2023-10-16T11:25:51.320217+0000","last_peered":"2023-10-16T11:25:51.320217+0000","last_clean":"2023-10-16T11:25:51.320217+0000","last_became_active":"2023-10-16T11:25:36.171201+0000","last_became_peered":"2023-10-16T11:25:36.171201+0000","last_unstale":"2023-10-16T11:25:51.320217+0000","last_undegraded":"2023-10-16T11:25:51.320217+0000","last_fullsized":"2023-10-16T11:25:51.320217+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:30.034841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.319537+0000","last_change":"2023-10-16T11:25:51.323557+0000","last_active":"2023-10-16T11:25:52.319537+0000","last_peered":"2023-10-16T11:25:52.319537+0000","last_clean":"2023-10-16T11:25:52.319537+0000","last_became_active":"2023-10-16T11:25:51.314015+0000","last_became_peered":"2023-10-16T11:25:51.314015+0000","last_unstale":"2023-10-16T11:25:52.319537+0000","last_undegraded":"2023-10-16T11:25:52.319537+0000","last_fullsized":"2023-10-16T11:25:52.319537+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:14:07.754755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.298471+0000","last_change":"2023-10-16T11:25:37.162366+0000","last_active":"2023-10-16T11:25:51.298471+0000","last_peered":"2023-10-16T11:25:51.298471+0000","last_clean":"2023-10-16T11:25:51.298471+0000","last_became_active":"2023-10-16T11:25:37.162032+0000","last_became_peered":"2023-10-16T11:25:37.162032+0000","last_unstale":"2023-10-16T11:25:51.298471+0000","last_undegraded":"2023-10-16T11:25:51.298471+0000","last_fullsized":"2023-10-16T11:25:51.298471+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.318585+0000","last_change":"2023-10-16T11:25:36.174246+0000","last_active":"2023-10-16T11:25:51.318585+0000","last_peered":"2023-10-16T11:25:51.318585+0000","last_clean":"2023-10-16T11:25:51.318585+0000","last_became_active":"2023-10-16T11:25:36.173975+0000","last_became_peered":"2023-10-16T11:25:36.173975+0000","last_unstale":"2023-10-16T11:25:51.318585+0000","last_undegraded":"2023-10-16T11:25:51.318585+0000","last_fullsized":"2023-10-16T11:25:51.318585+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:32.130559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.318259+0000","last_change":"2023-10-16T11:25:51.319530+0000","last_active":"2023-10-16T11:25:52.318259+0000","last_peered":"2023-10-16T11:25:52.318259+0000","last_clean":"2023-10-16T11:25:52.318259+0000","last_became_active":"2023-10-16T11:25:51.319035+0000","last_became_peered":"2023-10-16T11:25:51.319035+0000","last_unstale":"2023-10-16T11:25:52.318259+0000","last_undegraded":"2023-10-16T11:25:52.318259+0000","last_fullsized":"2023-10-16T11:25:52.318259+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:49.517942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.295002+0000","last_change":"2023-10-16T11:25:37.168715+0000","last_active":"2023-10-16T11:25:51.295002+0000","last_peered":"2023-10-16T11:25:51.295002+0000","last_clean":"2023-10-16T11:25:51.295002+0000","last_became_active":"2023-10-16T11:25:37.168387+0000","last_became_peered":"2023-10-16T11:25:37.168387+0000","last_unstale":"2023-10-16T11:25:51.295002+0000","last_undegraded":"2023-10-16T11:25:51.295002+0000","last_fullsized":"2023-10-16T11:25:51.295002+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.b","version":"236'2","reported_seq":18,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.299155+0000","last_change":"2023-10-16T11:25:36.174153+0000","last_active":"2023-10-16T11:25:51.299155+0000","last_peered":"2023-10-16T11:25:51.299155+0000","last_clean":"2023-10-16T11:25:51.299155+0000","last_became_active":"2023-10-16T11:25:36.173812+0000","last_became_peered":"2023-10-16T11:25:36.173812+0000","last_unstale":"2023-10-16T11:25:51.299155+0000","last_undegraded":"2023-10-16T11:25:51.299155+0000","last_fullsized":"2023-10-16T11:25:51.299155+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:07:05.931833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789599+0000","last_change":"2023-10-16T11:25:51.321769+0000","last_active":"2023-10-16T11:25:52.789599+0000","last_peered":"2023-10-16T11:25:52.789599+0000","last_clean":"2023-10-16T11:25:52.789599+0000","last_became_active":"2023-10-16T11:25:51.319324+0000","last_became_peered":"2023-10-16T11:25:51.319324+0000","last_unstale":"2023-10-16T11:25:52.789599+0000","last_undegraded":"2023-10-16T11:25:52.789599+0000","last_fullsized":"2023-10-16T11:25:52.789599+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:37.288954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.314528+0000","last_change":"2023-10-16T11:25:37.175767+0000","last_active":"2023-10-16T11:25:51.314528+0000","last_peered":"2023-10-16T11:25:51.314528+0000","last_clean":"2023-10-16T11:25:51.314528+0000","last_became_active":"2023-10-16T11:25:37.167389+0000","last_became_peered":"2023-10-16T11:25:37.167389+0000","last_unstale":"2023-10-16T11:25:51.314528+0000","last_undegraded":"2023-10-16T11:25:51.314528+0000","last_fullsized":"2023-10-16T11:25:51.314528+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:18.601118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.325646+0000","last_change":"2023-10-16T11:25:36.175478+0000","last_active":"2023-10-16T11:25:51.325646+0000","last_peered":"2023-10-16T11:25:51.325646+0000","last_clean":"2023-10-16T11:25:51.325646+0000","last_became_active":"2023-10-16T11:25:36.174935+0000","last_became_peered":"2023-10-16T11:25:36.174935+0000","last_unstale":"2023-10-16T11:25:51.325646+0000","last_undegraded":"2023-10-16T11:25:51.325646+0000","last_fullsized":"2023-10-16T11:25:51.325646+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:14.189489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789564+0000","last_change":"2023-10-16T11:25:51.328522+0000","last_active":"2023-10-16T11:25:52.789564+0000","last_peered":"2023-10-16T11:25:52.789564+0000","last_clean":"2023-10-16T11:25:52.789564+0000","last_became_active":"2023-10-16T11:25:51.328009+0000","last_became_peered":"2023-10-16T11:25:51.328009+0000","last_unstale":"2023-10-16T11:25:52.789564+0000","last_undegraded":"2023-10-16T11:25:52.789564+0000","last_fullsized":"2023-10-16T11:25:52.789564+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:24.349747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.315573+0000","last_change":"2023-10-16T11:25:37.173034+0000","last_active":"2023-10-16T11:25:51.315573+0000","last_peered":"2023-10-16T11:25:51.315573+0000","last_clean":"2023-10-16T11:25:51.315573+0000","last_became_active":"2023-10-16T11:25:37.172644+0000","last_became_peered":"2023-10-16T11:25:37.172644+0000","last_unstale":"2023-10-16T11:25:51.315573+0000","last_undegraded":"2023-10-16T11:25:51.315573+0000","last_fullsized":"2023-10-16T11:25:51.315573+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.322980+0000","last_change":"2023-10-16T11:25:36.163926+0000","last_active":"2023-10-16T11:25:51.322980+0000","last_peered":"2023-10-16T11:25:51.322980+0000","last_clean":"2023-10-16T11:25:51.322980+0000","last_became_active":"2023-10-16T11:25:36.162991+0000","last_became_peered":"2023-10-16T11:25:36.162991+0000","last_unstale":"2023-10-16T11:25:51.322980+0000","last_undegraded":"2023-10-16T11:25:51.322980+0000","last_fullsized":"2023-10-16T11:25:51.322980+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:49.547165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.317982+0000","last_change":"2023-10-16T11:25:51.329081+0000","last_active":"2023-10-16T11:25:52.317982+0000","last_peered":"2023-10-16T11:25:52.317982+0000","last_clean":"2023-10-16T11:25:52.317982+0000","last_became_active":"2023-10-16T11:25:51.328806+0000","last_became_peered":"2023-10-16T11:25:51.328806+0000","last_unstale":"2023-10-16T11:25:52.317982+0000","last_undegraded":"2023-10-16T11:25:52.317982+0000","last_fullsized":"2023-10-16T11:25:52.317982+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:45.962774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.294181+0000","last_change":"2023-10-16T11:25:37.158678+0000","last_active":"2023-10-16T11:25:51.294181+0000","last_peered":"2023-10-16T11:25:51.294181+0000","last_clean":"2023-10-16T11:25:51.294181+0000","last_became_active":"2023-10-16T11:25:37.157389+0000","last_became_peered":"2023-10-16T11:25:37.157389+0000","last_unstale":"2023-10-16T11:25:51.294181+0000","last_undegraded":"2023-10-16T11:25:51.294181+0000","last_fullsized":"2023-10-16T11:25:51.294181+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.311415+0000","last_change":"2023-10-16T11:25:36.171876+0000","last_active":"2023-10-16T11:25:51.311415+0000","last_peered":"2023-10-16T11:25:51.311415+0000","last_clean":"2023-10-16T11:25:51.311415+0000","last_became_active":"2023-10-16T11:25:36.169128+0000","last_became_peered":"2023-10-16T11:25:36.169128+0000","last_unstale":"2023-10-16T11:25:51.311415+0000","last_undegraded":"2023-10-16T11:25:51.311415+0000","last_fullsized":"2023-10-16T11:25:51.311415+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:16:18.153133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.316284+0000","last_change":"2023-10-16T11:25:51.315843+0000","last_active":"2023-10-16T11:25:52.316284+0000","last_peered":"2023-10-16T11:25:52.316284+0000","last_clean":"2023-10-16T11:25:52.316284+0000","last_became_active":"2023-10-16T11:25:51.314237+0000","last_became_peered":"2023-10-16T11:25:51.314237+0000","last_unstale":"2023-10-16T11:25:52.316284+0000","last_undegraded":"2023-10-16T11:25:52.316284+0000","last_fullsized":"2023-10-16T11:25:52.316284+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:20.974486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.313338+0000","last_change":"2023-10-16T11:25:37.179119+0000","last_active":"2023-10-16T11:25:51.313338+0000","last_peered":"2023-10-16T11:25:51.313338+0000","last_clean":"2023-10-16T11:25:51.313338+0000","last_became_active":"2023-10-16T11:25:37.175799+0000","last_became_peered":"2023-10-16T11:25:37.175799+0000","last_unstale":"2023-10-16T11:25:51.313338+0000","last_undegraded":"2023-10-16T11:25:51.313338+0000","last_fullsized":"2023-10-16T11:25:51.313338+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"236'2","reported_seq":18,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.304109+0000","last_change":"2023-10-16T11:25:36.162608+0000","last_active":"2023-10-16T11:25:51.304109+0000","last_peered":"2023-10-16T11:25:51.304109+0000","last_clean":"2023-10-16T11:25:51.304109+0000","last_became_active":"2023-10-16T11:25:36.162336+0000","last_became_peered":"2023-10-16T11:25:36.162336+0000","last_unstale":"2023-10-16T11:25:51.304109+0000","last_undegraded":"2023-10-16T11:25:51.304109+0000","last_fullsized":"2023-10-16T11:25:51.304109+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:19.377028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.788246+0000","last_change":"2023-10-16T11:25:51.327430+0000","last_active":"2023-10-16T11:25:52.788246+0000","last_peered":"2023-10-16T11:25:52.788246+0000","last_clean":"2023-10-16T11:25:52.788246+0000","last_became_active":"2023-10-16T11:25:51.327100+0000","last_became_peered":"2023-10-16T11:25:51.327100+0000","last_unstale":"2023-10-16T11:25:52.788246+0000","last_undegraded":"2023-10-16T11:25:52.788246+0000","last_fullsized":"2023-10-16T11:25:52.788246+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:57:00.058585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.314519+0000","last_change":"2023-10-16T11:25:37.177560+0000","last_active":"2023-10-16T11:25:51.314519+0000","last_peered":"2023-10-16T11:25:51.314519+0000","last_clean":"2023-10-16T11:25:51.314519+0000","last_became_active":"2023-10-16T11:25:37.176982+0000","last_became_peered":"2023-10-16T11:25:37.176982+0000","last_unstale":"2023-10-16T11:25:51.314519+0000","last_undegraded":"2023-10-16T11:25:51.314519+0000","last_fullsized":"2023-10-16T11:25:51.314519+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:54.497943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"237'1","reported_seq":17,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.325144+0000","last_change":"2023-10-16T11:25:36.174419+0000","last_active":"2023-10-16T11:25:51.325144+0000","last_peered":"2023-10-16T11:25:51.325144+0000","last_clean":"2023-10-16T11:25:51.325144+0000","last_became_active":"2023-10-16T11:25:36.174148+0000","last_became_peered":"2023-10-16T11:25:36.174148+0000","last_unstale":"2023-10-16T11:25:51.325144+0000","last_undegraded":"2023-10-16T11:25:51.325144+0000","last_fullsized":"2023-10-16T11:25:51.325144+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:38.739323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789472+0000","last_change":"2023-10-16T11:25:51.330994+0000","last_active":"2023-10-16T11:25:52.789472+0000","last_peered":"2023-10-16T11:25:52.789472+0000","last_clean":"2023-10-16T11:25:52.789472+0000","last_became_active":"2023-10-16T11:25:51.330677+0000","last_became_peered":"2023-10-16T11:25:51.330677+0000","last_unstale":"2023-10-16T11:25:52.789472+0000","last_undegraded":"2023-10-16T11:25:52.789472+0000","last_fullsized":"2023-10-16T11:25:52.789472+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:45.189825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.318438+0000","last_change":"2023-10-16T11:25:37.180683+0000","last_active":"2023-10-16T11:25:51.318438+0000","last_peered":"2023-10-16T11:25:51.318438+0000","last_clean":"2023-10-16T11:25:51.318438+0000","last_became_active":"2023-10-16T11:25:37.180393+0000","last_became_peered":"2023-10-16T11:25:37.180393+0000","last_unstale":"2023-10-16T11:25:51.318438+0000","last_undegraded":"2023-10-16T11:25:51.318438+0000","last_fullsized":"2023-10-16T11:25:51.318438+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:38.218704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.320357+0000","last_change":"2023-10-16T11:25:36.164519+0000","last_active":"2023-10-16T11:25:51.320357+0000","last_peered":"2023-10-16T11:25:51.320357+0000","last_clean":"2023-10-16T11:25:51.320357+0000","last_became_active":"2023-10-16T11:25:36.164214+0000","last_became_peered":"2023-10-16T11:25:36.164214+0000","last_unstale":"2023-10-16T11:25:51.320357+0000","last_undegraded":"2023-10-16T11:25:51.320357+0000","last_fullsized":"2023-10-16T11:25:51.320357+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:03.074535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.723342+0000","last_change":"2023-10-16T11:25:51.325986+0000","last_active":"2023-10-16T11:25:52.723342+0000","last_peered":"2023-10-16T11:25:52.723342+0000","last_clean":"2023-10-16T11:25:52.723342+0000","last_became_active":"2023-10-16T11:25:51.325725+0000","last_became_peered":"2023-10-16T11:25:51.325725+0000","last_unstale":"2023-10-16T11:25:52.723342+0000","last_undegraded":"2023-10-16T11:25:52.723342+0000","last_fullsized":"2023-10-16T11:25:52.723342+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:01:56.587898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.320310+0000","last_change":"2023-10-16T11:25:37.181808+0000","last_active":"2023-10-16T11:25:51.320310+0000","last_peered":"2023-10-16T11:25:51.320310+0000","last_clean":"2023-10-16T11:25:51.320310+0000","last_became_active":"2023-10-16T11:25:37.181575+0000","last_became_peered":"2023-10-16T11:25:37.181575+0000","last_unstale":"2023-10-16T11:25:51.320310+0000","last_undegraded":"2023-10-16T11:25:51.320310+0000","last_fullsized":"2023-10-16T11:25:51.320310+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:17:01.131611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.318671+0000","last_change":"2023-10-16T11:25:36.162153+0000","last_active":"2023-10-16T11:25:51.318671+0000","last_peered":"2023-10-16T11:25:51.318671+0000","last_clean":"2023-10-16T11:25:51.318671+0000","last_became_active":"2023-10-16T11:25:36.161842+0000","last_became_peered":"2023-10-16T11:25:36.161842+0000","last_unstale":"2023-10-16T11:25:51.318671+0000","last_undegraded":"2023-10-16T11:25:51.318671+0000","last_fullsized":"2023-10-16T11:25:51.318671+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:55.302377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.723428+0000","last_change":"2023-10-16T11:25:51.326939+0000","last_active":"2023-10-16T11:25:52.723428+0000","last_peered":"2023-10-16T11:25:52.723428+0000","last_clean":"2023-10-16T11:25:52.723428+0000","last_became_active":"2023-10-16T11:25:51.326520+0000","last_became_peered":"2023-10-16T11:25:51.326520+0000","last_unstale":"2023-10-16T11:25:52.723428+0000","last_undegraded":"2023-10-16T11:25:52.723428+0000","last_fullsized":"2023-10-16T11:25:52.723428+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:48.107944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.314001+0000","last_change":"2023-10-16T11:25:37.165420+0000","last_active":"2023-10-16T11:25:51.314001+0000","last_peered":"2023-10-16T11:25:51.314001+0000","last_clean":"2023-10-16T11:25:51.314001+0000","last_became_active":"2023-10-16T11:25:37.164820+0000","last_became_peered":"2023-10-16T11:25:37.164820+0000","last_unstale":"2023-10-16T11:25:51.314001+0000","last_undegraded":"2023-10-16T11:25:51.314001+0000","last_fullsized":"2023-10-16T11:25:51.314001+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:43.813878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.312990+0000","last_change":"2023-10-16T11:25:36.173209+0000","last_active":"2023-10-16T11:25:51.312990+0000","last_peered":"2023-10-16T11:25:51.312990+0000","last_clean":"2023-10-16T11:25:51.312990+0000","last_became_active":"2023-10-16T11:25:36.171799+0000","last_became_peered":"2023-10-16T11:25:36.171799+0000","last_unstale":"2023-10-16T11:25:51.312990+0000","last_undegraded":"2023-10-16T11:25:51.312990+0000","last_fullsized":"2023-10-16T11:25:51.312990+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:19.377028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.724296+0000","last_change":"2023-10-16T11:25:51.332559+0000","last_active":"2023-10-16T11:25:52.724296+0000","last_peered":"2023-10-16T11:25:52.724296+0000","last_clean":"2023-10-16T11:25:52.724296+0000","last_became_active":"2023-10-16T11:25:51.332201+0000","last_became_peered":"2023-10-16T11:25:51.332201+0000","last_unstale":"2023-10-16T11:25:52.724296+0000","last_undegraded":"2023-10-16T11:25:52.724296+0000","last_fullsized":"2023-10-16T11:25:52.724296+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:55.985578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.296387+0000","last_change":"2023-10-16T11:25:37.180243+0000","last_active":"2023-10-16T11:25:51.296387+0000","last_peered":"2023-10-16T11:25:51.296387+0000","last_clean":"2023-10-16T11:25:51.296387+0000","last_became_active":"2023-10-16T11:25:37.179869+0000","last_became_peered":"2023-10-16T11:25:37.179869+0000","last_unstale":"2023-10-16T11:25:51.296387+0000","last_undegraded":"2023-10-16T11:25:51.296387+0000","last_fullsized":"2023-10-16T11:25:51.296387+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:15:31.572558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.304062+0000","last_change":"2023-10-16T11:25:36.169146+0000","last_active":"2023-10-16T11:25:51.304062+0000","last_peered":"2023-10-16T11:25:51.304062+0000","last_clean":"2023-10-16T11:25:51.304062+0000","last_became_active":"2023-10-16T11:25:36.168877+0000","last_became_peered":"2023-10-16T11:25:36.168877+0000","last_unstale":"2023-10-16T11:25:51.304062+0000","last_undegraded":"2023-10-16T11:25:51.304062+0000","last_fullsized":"2023-10-16T11:25:51.304062+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:35.988148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.443085+0000","last_change":"2023-10-16T11:25:51.325913+0000","last_active":"2023-10-16T11:25:52.443085+0000","last_peered":"2023-10-16T11:25:52.443085+0000","last_clean":"2023-10-16T11:25:52.443085+0000","last_became_active":"2023-10-16T11:25:51.325577+0000","last_became_peered":"2023-10-16T11:25:51.325577+0000","last_unstale":"2023-10-16T11:25:52.443085+0000","last_undegraded":"2023-10-16T11:25:52.443085+0000","last_fullsized":"2023-10-16T11:25:52.443085+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:22:19.277245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.295096+0000","last_change":"2023-10-16T11:25:37.173516+0000","last_active":"2023-10-16T11:25:51.295096+0000","last_peered":"2023-10-16T11:25:51.295096+0000","last_clean":"2023-10-16T11:25:51.295096+0000","last_became_active":"2023-10-16T11:25:37.172964+0000","last_became_peered":"2023-10-16T11:25:37.172964+0000","last_unstale":"2023-10-16T11:25:51.295096+0000","last_undegraded":"2023-10-16T11:25:51.295096+0000","last_fullsized":"2023-10-16T11:25:51.295096+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.302773+0000","last_change":"2023-10-16T11:25:36.166442+0000","last_active":"2023-10-16T11:25:51.302773+0000","last_peered":"2023-10-16T11:25:51.302773+0000","last_clean":"2023-10-16T11:25:51.302773+0000","last_became_active":"2023-10-16T11:25:36.166131+0000","last_became_peered":"2023-10-16T11:25:36.166131+0000","last_unstale":"2023-10-16T11:25:51.302773+0000","last_undegraded":"2023-10-16T11:25:51.302773+0000","last_fullsized":"2023-10-16T11:25:51.302773+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:43.814368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3f","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.319150+0000","last_change":"2023-10-16T11:25:51.320357+0000","last_active":"2023-10-16T11:25:52.319150+0000","last_peered":"2023-10-16T11:25:52.319150+0000","last_clean":"2023-10-16T11:25:52.319150+0000","last_became_active":"2023-10-16T11:25:51.320053+0000","last_became_peered":"2023-10-16T11:25:51.320053+0000","last_unstale":"2023-10-16T11:25:52.319150+0000","last_undegraded":"2023-10-16T11:25:52.319150+0000","last_fullsized":"2023-10-16T11:25:52.319150+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:45.962774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.310334+0000","last_change":"2023-10-16T11:25:37.183491+0000","last_active":"2023-10-16T11:25:51.310334+0000","last_peered":"2023-10-16T11:25:51.310334+0000","last_clean":"2023-10-16T11:25:51.310334+0000","last_became_active":"2023-10-16T11:25:37.179560+0000","last_became_peered":"2023-10-16T11:25:37.179560+0000","last_unstale":"2023-10-16T11:25:51.310334+0000","last_undegraded":"2023-10-16T11:25:51.310334+0000","last_fullsized":"2023-10-16T11:25:51.310334+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:31.820109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.304486+0000","last_change":"2023-10-16T11:25:36.175845+0000","last_active":"2023-10-16T11:25:51.304486+0000","last_peered":"2023-10-16T11:25:51.304486+0000","last_clean":"2023-10-16T11:25:51.304486+0000","last_became_active":"2023-10-16T11:25:36.175390+0000","last_became_peered":"2023-10-16T11:25:36.175390+0000","last_unstale":"2023-10-16T11:25:51.304486+0000","last_undegraded":"2023-10-16T11:25:51.304486+0000","last_fullsized":"2023-10-16T11:25:51.304486+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:09.232865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.788350+0000","last_change":"2023-10-16T11:25:51.343357+0000","last_active":"2023-10-16T11:25:52.788350+0000","last_peered":"2023-10-16T11:25:52.788350+0000","last_clean":"2023-10-16T11:25:52.788350+0000","last_became_active":"2023-10-16T11:25:51.343044+0000","last_became_peered":"2023-10-16T11:25:51.343044+0000","last_unstale":"2023-10-16T11:25:52.788350+0000","last_undegraded":"2023-10-16T11:25:52.788350+0000","last_fullsized":"2023-10-16T11:25:52.788350+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:45.962774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.325051+0000","last_change":"2023-10-16T11:25:37.178486+0000","last_active":"2023-10-16T11:25:51.325051+0000","last_peered":"2023-10-16T11:25:51.325051+0000","last_clean":"2023-10-16T11:25:51.325051+0000","last_became_active":"2023-10-16T11:25:37.177077+0000","last_became_peered":"2023-10-16T11:25:37.177077+0000","last_unstale":"2023-10-16T11:25:51.325051+0000","last_undegraded":"2023-10-16T11:25:51.325051+0000","last_fullsized":"2023-10-16T11:25:51.325051+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"236'4","reported_seq":20,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.307789+0000","last_change":"2023-10-16T11:25:36.174205+0000","last_active":"2023-10-16T11:25:51.307789+0000","last_peered":"2023-10-16T11:25:51.307789+0000","last_clean":"2023-10-16T11:25:51.307789+0000","last_became_active":"2023-10-16T11:25:36.173850+0000","last_became_peered":"2023-10-16T11:25:36.173850+0000","last_unstale":"2023-10-16T11:25:51.307789+0000","last_undegraded":"2023-10-16T11:25:51.307789+0000","last_fullsized":"2023-10-16T11:25:51.307789+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:20.618602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"223'2789","reported_seq":3749,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.309922+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:25:51.309922+0000","last_peered":"2023-10-16T11:25:51.309922+0000","last_clean":"2023-10-16T11:25:51.309922+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:25:51.309922+0000","last_undegraded":"2023-10-16T11:25:51.309922+0000","last_fullsized":"2023-10-16T11:25:51.309922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2789,"log_dups_size":0,"ondisk_log_size":2789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":31,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":30,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2789,"ondisk_log_size":2789,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576002,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52780,"kb_used_data":3552,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319060,"statfs":{"total":96636764160,"available":96582717440,"internally_reserved":0,"allocated":3637248,"data_stored":2074666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.153,"15min":1.2090000000000001},"min":{"1min":0.71499999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4950000000000001,"5min":1.7549999999999999,"15min":2.3399999999999999},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.161,"15min":1.2150000000000001},"min":{"1min":0.623,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.448,"5min":1.6699999999999999,"15min":1.8320000000000001},"last":1.3180000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.256,"15min":1.2430000000000001},"min":{"1min":0.84599999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.5029999999999999,"5min":1.913,"15min":1.913},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.171,"5min":1.2210000000000001,"15min":1.264},"min":{"1min":0.67100000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.522,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.3520000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.3,"15min":1.296},"min":{"1min":0.94199999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6879999999999999,"5min":1.8680000000000001,"15min":2.4460000000000002},"last":1.206},{"interface":"front","average":{"1min":1.21,"5min":1.3129999999999999,"15min":1.3500000000000001},"min":{"1min":0.94499999999999995,"5min":0.88,"15min":0.88},"max":{"1min":1.5589999999999999,"5min":1.859,"15min":1.974},"last":1.494}]},{"osd":3,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3520000000000001,"15min":1.3779999999999999},"min":{"1min":0.93700000000000006,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.653,"5min":1.8080000000000001,"15min":1.9099999999999999},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.4370000000000001,"15min":1.454},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.728,"5min":2.0070000000000001,"15min":2.577},"last":1.544}]},{"osd":4,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4319999999999999,"15min":1.413},"min":{"1min":0.89800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.865,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.9610000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":1.149,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8660000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.145}]},{"osd":5,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.587,"15min":1.591},"min":{"1min":1.1799999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.95,"5min":2.101,"15min":3.3519999999999999},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.53,"15min":1.508},"min":{"1min":1.2330000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.9870000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.2609999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:24:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.492,"15min":1.5089999999999999},"min":{"1min":1.1459999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.8140000000000001,"5min":2.101,"15min":2.7250000000000001},"last":1.591},{"interface":"front","average":{"1min":1.6040000000000001,"5min":1.6819999999999999,"15min":1.698},"min":{"1min":1.3080000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.0129999999999999,"5min":2.2109999999999999,"15min":3.1440000000000001},"last":1.877}]}]},{"osd":6,"up_from":13,"seq":55834576002,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52216,"kb_used_data":2980,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319624,"statfs":{"total":96636764160,"available":96583294976,"internally_reserved":0,"allocated":3051520,"data_stored":1484874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.2130000000000001,"15min":1.21},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.343,"5min":6.298,"15min":6.298},"last":1.532},{"interface":"front","average":{"1min":1.204,"5min":1.331,"15min":1.292},"min":{"1min":0.72199999999999998,"5min":0.68200000000000005,"15min":0.66400000000000003},"max":{"1min":1.607,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":1.0309999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.224,"15min":1.212},"min":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.5660000000000001,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":1.395},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.421,"15min":1.3779999999999999},"min":{"1min":0.76200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.7310000000000001,"5min":12.468,"15min":12.468},"last":1.208}]},{"osd":2,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.3140000000000001,"15min":1.2769999999999999},"min":{"1min":0.77800000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.373,"5min":11.586,"15min":11.586},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.3740000000000001,"15min":1.3680000000000001},"min":{"1min":0.84399999999999997,"5min":0.82599999999999996,"15min":0.77500000000000002},"max":{"1min":1.7629999999999999,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.6240000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.425,"15min":1.3460000000000001},"min":{"1min":0.84799999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.859,"5min":12.801,"15min":12.801},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2929999999999999,"15min":1.2769999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.657,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":1.155}]},{"osd":4,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3899999999999999,"15min":1.3280000000000001},"min":{"1min":0.68700000000000006,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.9510000000000001,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.6220000000000001,"15min":1.5509999999999999},"min":{"1min":1.1180000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.069,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.353}]},{"osd":5,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.5189999999999999,"15min":1.397},"min":{"1min":0.92500000000000004,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.978,"5min":18.247,"15min":18.247},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.595,"15min":1.5620000000000001},"min":{"1min":0.93799999999999994,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.1240000000000001,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.6879999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.627,"15min":1.5469999999999999},"min":{"1min":1.2070000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.383,"5min":12.589,"15min":12.589},"last":1.5},{"interface":"front","average":{"1min":1.661,"5min":1.702,"15min":1.603},"min":{"1min":1.0049999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.2970000000000002,"5min":12.725,"15min":12.725},"last":1.7190000000000001}]}]},{"osd":5,"up_from":13,"seq":55834576002,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29984,"kb_used_data":2964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341856,"statfs":{"total":96636764160,"available":96606060544,"internally_reserved":0,"allocated":3035136,"data_stored":1484158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2150000000000001,"15min":1.2629999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.51,"5min":2.661,"15min":2.661},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.2769999999999999,"15min":1.339},"min":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7210000000000001,"5min":1.7949999999999999,"15min":1.865},"last":1.343}]},{"osd":1,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.145,"15min":1.1890000000000001},"min":{"1min":0.872,"5min":0.81299999999999994,"15min":0.79100000000000004},"max":{"1min":1.583,"5min":1.704,"15min":1.992},"last":1.7470000000000001},{"interface":"front","average":{"1min":1.079,"5min":1.22,"15min":1.222},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6739999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.669}]},{"osd":2,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.26,"15min":1.236},"min":{"1min":0.73499999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.7450000000000001,"5min":1.952,"15min":1.952},"last":1.204},{"interface":"front","average":{"1min":1.236,"5min":1.284,"15min":1.3129999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5840000000000001,"5min":1.8109999999999999,"15min":2.1640000000000001},"last":1.8660000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.319,"15min":1.329},"min":{"1min":0.872,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5920000000000001,"5min":3.012,"15min":3.012},"last":1.107},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.401,"15min":1.4139999999999999},"min":{"1min":0.95399999999999996,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.6639999999999999,"5min":1.9119999999999999,"15min":2.2250000000000001},"last":2.0129999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3600000000000001,"15min":1.341},"min":{"1min":0.91300000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.887,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.413,"5min":1.5529999999999999,"15min":1.5660000000000001},"min":{"1min":0.996,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.6499999999999999,"5min":2.903,"15min":2.903},"last":1.431}]},{"osd":6,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.4119999999999999,"15min":1.411},"min":{"1min":0.93799999999999994,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.837,"5min":1.8660000000000001,"15min":2.0979999999999999},"last":1.583},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.486,"15min":1.4810000000000001},"min":{"1min":0.93300000000000005,"5min":0.90700000000000003,"15min":0.89200000000000002},"max":{"1min":2.0329999999999999,"5min":3.073,"15min":3.073},"last":1.288}]},{"osd":7,"last update":"Mon Oct 16 11:24:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.5649999999999999,"15min":1.5429999999999999},"min":{"1min":1.159,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.764,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.532},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.625,"15min":1.6399999999999999},"min":{"1min":1.349,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.9970000000000001,"5min":2.1099999999999999,"15min":2.2930000000000001},"last":2.1930000000000001}]}]},{"osd":4,"up_from":13,"seq":55834576003,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70224,"kb_used_data":3004,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301616,"statfs":{"total":96636764160,"available":96564854784,"internally_reserved":0,"allocated":3076096,"data_stored":1488775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.1870000000000001,"15min":1.1839999999999999},"min":{"1min":0.80800000000000005,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.8049999999999999,"5min":2.899,"15min":2.899},"last":1.615},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2170000000000001,"15min":1.2450000000000001},"min":{"1min":0.88400000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.7929999999999999,"5min":3.016,"15min":3.016},"last":1.292}]},{"osd":1,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3049999999999999,"15min":1.298},"min":{"1min":0.84999999999999998,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.887,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.77100000000000002,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.617,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.3400000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.327,"15min":1.3140000000000001},"min":{"1min":1.075,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.169,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4350000000000001,"15min":1.4159999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9870000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.1140000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.3640000000000001,"15min":1.351},"min":{"1min":0.96799999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.0390000000000001,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.47},{"interface":"front","average":{"1min":1.645,"5min":1.4970000000000001,"15min":1.4610000000000001},"min":{"1min":1.216,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.3109999999999999,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.2210000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.5289999999999999,"15min":1.464},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2309999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.504,"5min":1.4810000000000001,"15min":1.4350000000000001},"min":{"1min":0.89500000000000002,"5min":0.85199999999999998,"15min":0.78600000000000003},"max":{"1min":2.0110000000000001,"5min":3.7989999999999999,"15min":3.7989999999999999},"last":1.6539999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.599,"15min":1.534},"min":{"1min":0.98499999999999999,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":2.6459999999999999,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.399},{"interface":"front","average":{"1min":1.752,"5min":1.665,"15min":1.6379999999999999},"min":{"1min":0.77100000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.3599999999999999,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.5920000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.7629999999999999,"5min":1.649,"15min":1.6000000000000001},"min":{"1min":0.74099999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.3570000000000002,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":1.7669999999999999},{"interface":"front","average":{"1min":1.7849999999999999,"5min":1.7270000000000001,"15min":1.675},"min":{"1min":0.98699999999999999,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.464,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.55}]}]},{"osd":3,"up_from":13,"seq":55834576002,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52212,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319628,"statfs":{"total":96636764160,"available":96583299072,"internally_reserved":0,"allocated":3055616,"data_stored":1486934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.85899999999999999,"15min":0.89700000000000002},"min":{"1min":0.40000000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1339999999999999,"5min":1.478,"15min":1.901},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.997,"15min":1.0569999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1850000000000001,"5min":1.6180000000000001,"15min":1.6679999999999999},"last":1.256}]},{"osd":1,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.95299999999999996,"15min":0.98399999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.3400000000000001,"5min":1.508,"15min":1.508},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0609999999999999,"15min":1.121},"min":{"1min":0.64900000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5429999999999999,"5min":1.601,"15min":1.7190000000000001},"last":0.94199999999999995}]},{"osd":2,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0580000000000001,"15min":1.1120000000000001},"min":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4199999999999999,"5min":1.694,"15min":1.694},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.161,"15min":1.22},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.7350000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.0149999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2989999999999999,"15min":1.3480000000000001},"min":{"1min":0.89400000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.724,"5min":1.806,"15min":1.873},"last":1.087},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3300000000000001,"15min":1.359},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.5960000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.389,"15min":1.405},"min":{"1min":1.0549999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8260000000000001,"5min":2.798,"15min":2.798},"last":1.427},{"interface":"front","average":{"1min":1.347,"5min":1.395,"15min":1.4410000000000001},"min":{"1min":0.88800000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.883,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.377}]},{"osd":6,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.488,"15min":1.5329999999999999},"min":{"1min":1.01,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.911,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4790000000000001,"15min":1.534},"min":{"1min":0.86699999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.0329999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.504}]},{"osd":7,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.518,"15min":1.5580000000000001},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.575,"5min":1.613,"15min":1.669},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.3799999999999999,"5min":2.3799999999999999,"15min":2.5129999999999999},"last":1.4710000000000001}]}]},{"osd":2,"up_from":13,"seq":55834576001,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56356,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315484,"statfs":{"total":96636764160,"available":96579055616,"internally_reserved":0,"allocated":3039232,"data_stored":1484248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.89200000000000002,"15min":0.91600000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.349,"5min":1.532,"15min":1.669},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.88,"5min":0.99299999999999999,"15min":1.069},"min":{"1min":0.56899999999999995,"5min":0.47699999999999998,"15min":0.32600000000000001},"max":{"1min":1.3460000000000001,"5min":1.6359999999999999,"15min":2.1429999999999998},"last":1.22}]},{"osd":1,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0760000000000001,"15min":1.089},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.39200000000000002},"max":{"1min":1.591,"5min":1.645,"15min":1.978},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.085,"15min":1.1379999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.393,"5min":1.609,"15min":1.7949999999999999},"last":0.81000000000000005}]},{"osd":3,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0429999999999999,"15min":1.0740000000000001},"min":{"1min":0.499,"5min":0.437,"15min":0.437},"max":{"1min":1.522,"5min":1.651,"15min":1.8560000000000001},"last":1.113},{"interface":"front","average":{"1min":1.137,"5min":1.2170000000000001,"15min":1.2629999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.6930000000000001,"5min":1.7450000000000001,"15min":1.857},"last":1.2569999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3340000000000001,"15min":1.3999999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.73499999999999999},"max":{"1min":1.643,"5min":1.8029999999999999,"15min":2.5059999999999998},"last":1.526},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.3220000000000001,"15min":1.339},"min":{"1min":0.88200000000000001,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":1.6499999999999999,"5min":1.825,"15min":1.843},"last":1.345}]},{"osd":5,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.4139999999999999,"15min":1.4530000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.619},{"interface":"front","average":{"1min":1.371,"5min":1.429,"15min":1.4730000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.885,"5min":1.931,"15min":2.6259999999999999},"last":1.5700000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3899999999999999,"15min":1.3999999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.8260000000000001,"5min":1.9019999999999999,"15min":2.1339999999999999},"last":1.855},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4870000000000001,"15min":1.526},"min":{"1min":0.80500000000000005,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.6990000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.5109999999999999,"15min":1.569},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.92500000000000004},"max":{"1min":2.0310000000000001,"5min":2.1469999999999998,"15min":2.8109999999999999},"last":1.474},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4750000000000001,"15min":1.502},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80100000000000005},"max":{"1min":1.952,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.764}]}]},{"osd":1,"up_from":13,"seq":55834576001,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64708,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307132,"statfs":{"total":96636764160,"available":96570503168,"internally_reserved":0,"allocated":3072000,"data_stored":1487915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.90900000000000003,"15min":0.91300000000000003},"min":{"1min":0.39400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.3520000000000001,"5min":5.117,"15min":5.117},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.93500000000000005,"15min":1.0289999999999999},"min":{"1min":0.35899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.262,"5min":1.5509999999999999,"15min":1.667},"last":1.0609999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.93799999999999994,"15min":1.0029999999999999},"min":{"1min":0.497,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3720000000000001,"5min":2.093,"15min":2.093},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1879999999999999,"15min":1.2589999999999999},"min":{"1min":0.51700000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.456,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.78500000000000003}]},{"osd":3,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.139,"15min":1.1890000000000001},"min":{"1min":0.53800000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.6080000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.103,"15min":1.153},"min":{"1min":0.55800000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.798,"5min":1.798,"15min":1.802},"last":1.252}]},{"osd":4,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.3049999999999999,"15min":1.2949999999999999},"min":{"1min":0.69299999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.978,"5min":5.29,"15min":5.29},"last":1.179},{"interface":"front","average":{"1min":1.327,"5min":1.4159999999999999,"15min":1.421},"min":{"1min":0.82999999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.891,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":1.127}]},{"osd":5,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.373,"15min":1.409},"min":{"1min":0.83799999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.9430000000000001,"5min":6.0979999999999999,"15min":6.0979999999999999},"last":1.9430000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.383,"15min":1.403},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.859,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.1120000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4570000000000001,"15min":1.4890000000000001},"min":{"1min":0.92200000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.923,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":1.8320000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4419999999999999,"15min":1.4810000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.1419999999999999,"5min":3.8799999999999999,"15min":3.8799999999999999},"last":1.2430000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.546,"15min":1.5609999999999999},"min":{"1min":1.0409999999999999,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":2.0219999999999998,"5min":4.46,"15min":4.46},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.512,"5min":1.597,"15min":1.6040000000000001},"min":{"1min":0.99099999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.097,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":2.04}]}]},{"osd":0,"up_from":13,"seq":55834576002,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56944,"kb_used_data":3548,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314896,"statfs":{"total":96636764160,"available":96578453504,"internally_reserved":0,"allocated":3633152,"data_stored":2074560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.86099999999999999,"15min":0.88700000000000001},"min":{"1min":0.46800000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2210000000000001,"5min":1.52,"15min":1.619},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.006,"15min":1.081},"min":{"1min":0.41199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.341,"5min":1.621,"15min":1.76},"last":1.1100000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.075,"15min":1.0800000000000001},"min":{"1min":0.52500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.7349999999999999,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.034,"5min":1.0700000000000001,"15min":1.1299999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":1.0109999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.016,"15min":1.0669999999999999},"min":{"1min":0.50600000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1299999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.1539999999999999,"15min":1.226},"min":{"1min":0.55600000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.3799999999999999,"5min":1.833,"15min":1.9830000000000001},"last":1.0580000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.349,"15min":1.3859999999999999},"min":{"1min":0.78800000000000003,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":3.2799999999999998,"5min":3.2799999999999998,"15min":3.2799999999999998},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.222,"5min":1.292,"15min":1.337},"min":{"1min":0.76100000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.3149999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.357,"15min":1.3720000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0209999999999999,"5min":2.0750000000000002,"15min":2.6669999999999998},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3819999999999999,"15min":1.431},"min":{"1min":0.89800000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.885,"5min":2.2029999999999998,"15min":2.2040000000000002},"last":1.504}]},{"osd":6,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.4710000000000001,"15min":1.496},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":5.2560000000000002,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.429,"15min":1.4790000000000001},"min":{"1min":0.83199999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.804,"5min":2.073,"15min":2.262},"last":1.2929999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.704,"5min":1.5429999999999999,"15min":1.5600000000000001},"min":{"1min":0.88700000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.4419999999999999,"15min":1.492},"min":{"1min":0.871,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.623,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.2330000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:26:00.954 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:26:01.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.179+0000 7f7998e4d640 1 Processor -- start 2023-10-16T11:26:01.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.179+0000 7f7998e4d640 1 -- start start 2023-10-16T11:26:01.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.181+0000 7f7998e4d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f799415cbd0 0x7f7994069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:01.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.181+0000 7f7998e4d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79940b1f10 0x7f799406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:01.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.181+0000 7f7998e4d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7994160cb0 con 0x7f7994160820 2023-10-16T11:26:01.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.181+0000 7f7998e4d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7994160f90 con 0x7f799415cbd0 2023-10-16T11:26:01.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.181+0000 7f7998e4d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7994161270 con 0x7f79940b1f10 2023-10-16T11:26:01.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.181+0000 7f79937fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f799415cbd0 0x7f7994069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:01.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.181+0000 7f79937fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f799415cbd0 0x7f7994069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40556/0 (socket says 172.21.15.84:40556) 2023-10-16T11:26:01.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.181+0000 7f79937fe640 1 -- 172.21.15.84:0/2894212093 learned_addr learned my addr 172.21.15.84:0/2894212093 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:01.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.182+0000 7f79937fe640 1 -- 172.21.15.84:0/2894212093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79940b1f10 msgr2=0x7f799406a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:01.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.182+0000 7f79937fe640 1 --2- 172.21.15.84:0/2894212093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79940b1f10 0x7f799406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.182+0000 7f79937fe640 1 -- 172.21.15.84:0/2894212093 >> v1:172.21.15.157:6789/0 conn(0x7f7994160820 legacy=0x7f79940702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:01.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.182+0000 7f79937fe640 1 -- 172.21.15.84:0/2894212093 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7994161550 con 0x7f799415cbd0 2023-10-16T11:26:01.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.183+0000 7f79937fe640 1 --2- 172.21.15.84:0/2894212093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f799415cbd0 0x7f7994069da0 secure :-1 s=READY pgs=3005 cs=0 l=1 rev1=1 crypto rx=0x7f7984009870 tx=0x7f798402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71d356920c9987e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:01.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.183+0000 7f79927fc640 1 -- 172.21.15.84:0/2894212093 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984031740 con 0x7f799415cbd0 2023-10-16T11:26:01.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.183+0000 7f79927fc640 1 -- 172.21.15.84:0/2894212093 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79840318a0 con 0x7f799415cbd0 2023-10-16T11:26:01.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.183+0000 7f79927fc640 1 -- 172.21.15.84:0/2894212093 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984031c30 con 0x7f799415cbd0 2023-10-16T11:26:01.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.184+0000 7f7998e4d640 1 -- 172.21.15.84:0/2894212093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f799415cbd0 msgr2=0x7f7994069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:01.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.184+0000 7f7998e4d640 1 --2- 172.21.15.84:0/2894212093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f799415cbd0 0x7f7994069da0 secure :-1 s=READY pgs=3005 cs=0 l=1 rev1=1 crypto rx=0x7f7984009870 tx=0x7f798402b7e0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.184+0000 7f7998e4d640 1 -- 172.21.15.84:0/2894212093 shutdown_connections 2023-10-16T11:26:01.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.184+0000 7f7998e4d640 1 --2- 172.21.15.84:0/2894212093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79940b1f10 0x7f799406a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.184+0000 7f7998e4d640 1 --2- 172.21.15.84:0/2894212093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f799415cbd0 0x7f7994069da0 unknown :-1 s=CLOSED pgs=3005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.184+0000 7f7998e4d640 1 -- 172.21.15.84:0/2894212093 >> 172.21.15.84:0/2894212093 conn(0x7f7994065f00 msgr2=0x7f799406aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:01.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.184+0000 7f7998e4d640 1 -- 172.21.15.84:0/2894212093 shutdown_connections 2023-10-16T11:26:01.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.184+0000 7f7998e4d640 1 -- 172.21.15.84:0/2894212093 wait complete. 2023-10-16T11:26:01.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.186+0000 7f7998e4d640 1 Processor -- start 2023-10-16T11:26:01.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.186+0000 7f7998e4d640 1 -- start start 2023-10-16T11:26:01.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.187+0000 7f7998e4d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79940b1f10 0x7f799414d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:01.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.187+0000 7f7998e4d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f799415cbd0 0x7f79940cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:01.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.187+0000 7f7998e4d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7994142fc0 0x7f79941504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:01.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.187+0000 7f7998e4d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7994159560 con 0x7f79940b1f10 2023-10-16T11:26:01.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.187+0000 7f7998e4d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799414d680 con 0x7f7994142fc0 2023-10-16T11:26:01.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.187+0000 7f7998e4d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7994158950 con 0x7f799415cbd0 2023-10-16T11:26:01.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.187+0000 7f79937fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79940b1f10 0x7f799414d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:01.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.187+0000 7f79937fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79940b1f10 0x7f799414d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40562/0 (socket says 172.21.15.84:40562) 2023-10-16T11:26:01.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.187+0000 7f79937fe640 1 -- 172.21.15.84:0/2168110624 learned_addr learned my addr 172.21.15.84:0/2168110624 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:01.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.188+0000 7f79937fe640 1 -- 172.21.15.84:0/2168110624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f799415cbd0 msgr2=0x7f79940cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:01.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.188+0000 7f7992ffd640 1 --2- 172.21.15.84:0/2168110624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f799415cbd0 0x7f79940cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:01.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.188+0000 7f79937fe640 1 --2- 172.21.15.84:0/2168110624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f799415cbd0 0x7f79940cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.188+0000 7f79937fe640 1 -- 172.21.15.84:0/2168110624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7994142fc0 msgr2=0x7f79941504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:01.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.188+0000 7f79937fe640 1 --2- 172.21.15.84:0/2168110624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7994142fc0 0x7f79941504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.188+0000 7f79937fe640 1 -- 172.21.15.84:0/2168110624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79840094d0 con 0x7f79940b1f10 2023-10-16T11:26:01.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.188+0000 7f79937fe640 1 --2- 172.21.15.84:0/2168110624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79940b1f10 0x7f799414d880 secure :-1 s=READY pgs=3006 cs=0 l=1 rev1=1 crypto rx=0x7f7984000c00 tx=0x7f79840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:01.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.188+0000 7f797bfff640 1 -- 172.21.15.84:0/2168110624 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984033c70 con 0x7f79940b1f10 2023-10-16T11:26:01.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.188+0000 7f797bfff640 1 -- 172.21.15.84:0/2168110624 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7984033df0 con 0x7f79940b1f10 2023-10-16T11:26:01.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.188+0000 7f797bfff640 1 -- 172.21.15.84:0/2168110624 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798403d6b0 con 0x7f79940b1f10 2023-10-16T11:26:01.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.188+0000 7f7998e4d640 1 -- 172.21.15.84:0/2168110624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7994158c30 con 0x7f79940b1f10 2023-10-16T11:26:01.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.189+0000 7f7998e4d640 1 -- 172.21.15.84:0/2168110624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79941591f0 con 0x7f79940b1f10 2023-10-16T11:26:01.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.190+0000 7f7998e4d640 1 -- 172.21.15.84:0/2168110624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7994159710 con 0x7f79940b1f10 2023-10-16T11:26:01.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.190+0000 7f797bfff640 1 -- 172.21.15.84:0/2168110624 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f798403a040 con 0x7f79940b1f10 2023-10-16T11:26:01.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.191+0000 7f797bfff640 1 --2- 172.21.15.84:0/2168110624 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f798006b0e0 0x7f798006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:01.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.191+0000 7f797bfff640 1 -- 172.21.15.84:0/2168110624 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f79840abe40 con 0x7f79940b1f10 2023-10-16T11:26:01.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.192+0000 7f7992ffd640 1 --2- 172.21.15.84:0/2168110624 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f798006b0e0 0x7f798006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:01.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.192+0000 7f7992ffd640 1 --2- 172.21.15.84:0/2168110624 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f798006b0e0 0x7f798006d5a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7994151140 tx=0x7f7988008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:01.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.196+0000 7f797bfff640 1 -- 172.21.15.84:0/2168110624 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f79840766f0 con 0x7f79940b1f10 2023-10-16T11:26:01.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.387+0000 7f7998e4d640 1 -- 172.21.15.84:0/2168110624 --> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f799415cfb0 con 0x7f798006b0e0 2023-10-16T11:26:01.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.425+0000 7f797bfff640 1 -- 172.21.15.84:0/2168110624 <== mgr.11814 v2:172.21.15.84:6832/2963529201 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+526392 (secure 0 0 0) 0x7f798800a380 con 0x7f798006b0e0 2023-10-16T11:26:01.435 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:01.439 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:26:01.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.436+0000 7f7998e4d640 1 -- 172.21.15.84:0/2168110624 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f798006b0e0 msgr2=0x7f798006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:01.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.436+0000 7f7998e4d640 1 --2- 172.21.15.84:0/2168110624 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f798006b0e0 0x7f798006d5a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7994151140 tx=0x7f7988008040 comp rx=0 tx=0).stop 2023-10-16T11:26:01.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.436+0000 7f7998e4d640 1 -- 172.21.15.84:0/2168110624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79940b1f10 msgr2=0x7f799414d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:01.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.436+0000 7f7998e4d640 1 --2- 172.21.15.84:0/2168110624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79940b1f10 0x7f799414d880 secure :-1 s=READY pgs=3006 cs=0 l=1 rev1=1 crypto rx=0x7f7984000c00 tx=0x7f79840046c0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.436+0000 7f7998e4d640 1 -- 172.21.15.84:0/2168110624 shutdown_connections 2023-10-16T11:26:01.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.436+0000 7f7998e4d640 1 --2- 172.21.15.84:0/2168110624 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f798006b0e0 0x7f798006d5a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.436+0000 7f7998e4d640 1 --2- 172.21.15.84:0/2168110624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7994142fc0 0x7f79941504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.436+0000 7f7998e4d640 1 --2- 172.21.15.84:0/2168110624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f799415cbd0 0x7f79940cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.436+0000 7f7998e4d640 1 --2- 172.21.15.84:0/2168110624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79940b1f10 0x7f799414d880 unknown :-1 s=CLOSED pgs=3006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.436+0000 7f7998e4d640 1 -- 172.21.15.84:0/2168110624 >> 172.21.15.84:0/2168110624 conn(0x7f7994065f00 msgr2=0x7f7994157930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:01.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.436+0000 7f7998e4d640 1 -- 172.21.15.84:0/2168110624 shutdown_connections 2023-10-16T11:26:01.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.436+0000 7f7998e4d640 1 -- 172.21.15.84:0/2168110624 wait complete. 2023-10-16T11:26:01.462 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2023-10-16T11:26:01.250839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":713,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2869,"ondisk_log_size":2869,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":435424,"kb_used_data":25000,"kb_used_omap":12,"kb_used_meta":410291,"kb_avail":754539296,"statfs":{"total":773094113280,"available":772648239104,"internally_reserved":0,"allocated":25600000,"data_stored":13066130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":420138553},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.962602"},"pg_stats":[{"pgid":"33.1","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.723012+0000","last_change":"2023-10-16T11:25:51.337046+0000","last_active":"2023-10-16T11:25:52.723012+0000","last_peered":"2023-10-16T11:25:52.723012+0000","last_clean":"2023-10-16T11:25:52.723012+0000","last_became_active":"2023-10-16T11:25:51.336494+0000","last_became_peered":"2023-10-16T11:25:51.336494+0000","last_unstale":"2023-10-16T11:25:52.723012+0000","last_undegraded":"2023-10-16T11:25:52.723012+0000","last_fullsized":"2023-10-16T11:25:52.723012+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:35.603183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.303096+0000","last_change":"2023-10-16T11:25:50.296757+0000","last_active":"2023-10-16T11:25:51.303096+0000","last_peered":"2023-10-16T11:25:51.303096+0000","last_clean":"2023-10-16T11:25:51.303096+0000","last_became_active":"2023-10-16T11:25:50.295979+0000","last_became_peered":"2023-10-16T11:25:50.295979+0000","last_unstale":"2023-10-16T11:25:51.303096+0000","last_undegraded":"2023-10-16T11:25:51.303096+0000","last_fullsized":"2023-10-16T11:25:51.303096+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:49:10.974891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3f","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.314306+0000","last_change":"2023-10-16T11:25:37.173492+0000","last_active":"2023-10-16T11:25:51.314306+0000","last_peered":"2023-10-16T11:25:51.314306+0000","last_clean":"2023-10-16T11:25:51.314306+0000","last_became_active":"2023-10-16T11:25:37.167017+0000","last_became_peered":"2023-10-16T11:25:37.167017+0000","last_unstale":"2023-10-16T11:25:51.314306+0000","last_undegraded":"2023-10-16T11:25:51.314306+0000","last_fullsized":"2023-10-16T11:25:51.314306+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:07.791499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.316237+0000","last_change":"2023-10-16T11:25:51.332747+0000","last_active":"2023-10-16T11:25:52.316237+0000","last_peered":"2023-10-16T11:25:52.316237+0000","last_clean":"2023-10-16T11:25:52.316237+0000","last_became_active":"2023-10-16T11:25:51.332320+0000","last_became_peered":"2023-10-16T11:25:51.332320+0000","last_unstale":"2023-10-16T11:25:52.316237+0000","last_undegraded":"2023-10-16T11:25:52.316237+0000","last_fullsized":"2023-10-16T11:25:52.316237+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:20.974486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.294501+0000","last_change":"2023-10-16T11:25:50.299119+0000","last_active":"2023-10-16T11:25:51.294501+0000","last_peered":"2023-10-16T11:25:51.294501+0000","last_clean":"2023-10-16T11:25:51.294501+0000","last_became_active":"2023-10-16T11:25:50.298813+0000","last_became_peered":"2023-10-16T11:25:50.298813+0000","last_unstale":"2023-10-16T11:25:51.294501+0000","last_undegraded":"2023-10-16T11:25:51.294501+0000","last_fullsized":"2023-10-16T11:25:51.294501+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:17:14.257383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.3e","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.315054+0000","last_change":"2023-10-16T11:25:37.179117+0000","last_active":"2023-10-16T11:25:51.315054+0000","last_peered":"2023-10-16T11:25:51.315054+0000","last_clean":"2023-10-16T11:25:51.315054+0000","last_became_active":"2023-10-16T11:25:37.178849+0000","last_became_peered":"2023-10-16T11:25:37.178849+0000","last_unstale":"2023-10-16T11:25:51.315054+0000","last_undegraded":"2023-10-16T11:25:51.315054+0000","last_fullsized":"2023-10-16T11:25:51.315054+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:18:57.278249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.315426+0000","last_change":"2023-10-16T11:25:51.323314+0000","last_active":"2023-10-16T11:25:52.315426+0000","last_peered":"2023-10-16T11:25:52.315426+0000","last_clean":"2023-10-16T11:25:52.315426+0000","last_became_active":"2023-10-16T11:25:51.313845+0000","last_became_peered":"2023-10-16T11:25:51.313845+0000","last_unstale":"2023-10-16T11:25:52.315426+0000","last_undegraded":"2023-10-16T11:25:52.315426+0000","last_fullsized":"2023-10-16T11:25:52.315426+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:32.173887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.322980+0000","last_change":"2023-10-16T11:25:50.300048+0000","last_active":"2023-10-16T11:25:51.322980+0000","last_peered":"2023-10-16T11:25:51.322980+0000","last_clean":"2023-10-16T11:25:51.322980+0000","last_became_active":"2023-10-16T11:25:50.299723+0000","last_became_peered":"2023-10-16T11:25:50.299723+0000","last_unstale":"2023-10-16T11:25:51.322980+0000","last_undegraded":"2023-10-16T11:25:51.322980+0000","last_fullsized":"2023-10-16T11:25:51.322980+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:24.028667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3d","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.305822+0000","last_change":"2023-10-16T11:25:37.179065+0000","last_active":"2023-10-16T11:25:51.305822+0000","last_peered":"2023-10-16T11:25:51.305822+0000","last_clean":"2023-10-16T11:25:51.305822+0000","last_became_active":"2023-10-16T11:25:37.178779+0000","last_became_peered":"2023-10-16T11:25:37.178779+0000","last_unstale":"2023-10-16T11:25:51.305822+0000","last_undegraded":"2023-10-16T11:25:51.305822+0000","last_fullsized":"2023-10-16T11:25:51.305822+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:09:47.367715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.318974+0000","last_change":"2023-10-16T11:25:51.324420+0000","last_active":"2023-10-16T11:25:52.318974+0000","last_peered":"2023-10-16T11:25:52.318974+0000","last_clean":"2023-10-16T11:25:52.318974+0000","last_became_active":"2023-10-16T11:25:51.324143+0000","last_became_peered":"2023-10-16T11:25:51.324143+0000","last_unstale":"2023-10-16T11:25:52.318974+0000","last_undegraded":"2023-10-16T11:25:52.318974+0000","last_fullsized":"2023-10-16T11:25:52.318974+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:45.962774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317461+0000","last_change":"2023-10-16T11:25:50.290401+0000","last_active":"2023-10-16T11:25:51.317461+0000","last_peered":"2023-10-16T11:25:51.317461+0000","last_clean":"2023-10-16T11:25:51.317461+0000","last_became_active":"2023-10-16T11:25:50.290086+0000","last_became_peered":"2023-10-16T11:25:50.290086+0000","last_unstale":"2023-10-16T11:25:51.317461+0000","last_undegraded":"2023-10-16T11:25:51.317461+0000","last_fullsized":"2023-10-16T11:25:51.317461+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:31:20.394734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3c","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.300992+0000","last_change":"2023-10-16T11:25:37.172480+0000","last_active":"2023-10-16T11:25:51.300992+0000","last_peered":"2023-10-16T11:25:51.300992+0000","last_clean":"2023-10-16T11:25:51.300992+0000","last_became_active":"2023-10-16T11:25:37.172192+0000","last_became_peered":"2023-10-16T11:25:37.172192+0000","last_unstale":"2023-10-16T11:25:51.300992+0000","last_undegraded":"2023-10-16T11:25:51.300992+0000","last_fullsized":"2023-10-16T11:25:51.300992+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.724643+0000","last_change":"2023-10-16T11:25:51.331231+0000","last_active":"2023-10-16T11:25:52.724643+0000","last_peered":"2023-10-16T11:25:52.724643+0000","last_clean":"2023-10-16T11:25:52.724643+0000","last_became_active":"2023-10-16T11:25:51.330927+0000","last_became_peered":"2023-10-16T11:25:51.330927+0000","last_unstale":"2023-10-16T11:25:52.724643+0000","last_undegraded":"2023-10-16T11:25:52.724643+0000","last_fullsized":"2023-10-16T11:25:52.724643+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:25.045561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.295663+0000","last_change":"2023-10-16T11:25:50.296115+0000","last_active":"2023-10-16T11:25:51.295663+0000","last_peered":"2023-10-16T11:25:51.295663+0000","last_clean":"2023-10-16T11:25:51.295663+0000","last_became_active":"2023-10-16T11:25:50.295832+0000","last_became_peered":"2023-10-16T11:25:50.295832+0000","last_unstale":"2023-10-16T11:25:51.295663+0000","last_undegraded":"2023-10-16T11:25:51.295663+0000","last_fullsized":"2023-10-16T11:25:51.295663+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:51:43.054850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3b","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.321786+0000","last_change":"2023-10-16T11:25:37.177505+0000","last_active":"2023-10-16T11:25:51.321786+0000","last_peered":"2023-10-16T11:25:51.321786+0000","last_clean":"2023-10-16T11:25:51.321786+0000","last_became_active":"2023-10-16T11:25:37.174334+0000","last_became_peered":"2023-10-16T11:25:37.174334+0000","last_unstale":"2023-10-16T11:25:51.321786+0000","last_undegraded":"2023-10-16T11:25:51.321786+0000","last_fullsized":"2023-10-16T11:25:51.321786+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.723880+0000","last_change":"2023-10-16T11:25:51.336212+0000","last_active":"2023-10-16T11:25:52.723880+0000","last_peered":"2023-10-16T11:25:52.723880+0000","last_clean":"2023-10-16T11:25:52.723880+0000","last_became_active":"2023-10-16T11:25:51.335751+0000","last_became_peered":"2023-10-16T11:25:51.335751+0000","last_unstale":"2023-10-16T11:25:52.723880+0000","last_undegraded":"2023-10-16T11:25:52.723880+0000","last_fullsized":"2023-10-16T11:25:52.723880+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:32:29.165691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.310930+0000","last_change":"2023-10-16T11:25:50.309435+0000","last_active":"2023-10-16T11:25:51.310930+0000","last_peered":"2023-10-16T11:25:51.310930+0000","last_clean":"2023-10-16T11:25:51.310930+0000","last_became_active":"2023-10-16T11:25:50.308844+0000","last_became_peered":"2023-10-16T11:25:50.308844+0000","last_unstale":"2023-10-16T11:25:51.310930+0000","last_undegraded":"2023-10-16T11:25:51.310930+0000","last_fullsized":"2023-10-16T11:25:51.310930+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:31:20.394734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3a","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.303115+0000","last_change":"2023-10-16T11:25:37.177781+0000","last_active":"2023-10-16T11:25:51.303115+0000","last_peered":"2023-10-16T11:25:51.303115+0000","last_clean":"2023-10-16T11:25:51.303115+0000","last_became_active":"2023-10-16T11:25:37.177501+0000","last_became_peered":"2023-10-16T11:25:37.177501+0000","last_unstale":"2023-10-16T11:25:51.303115+0000","last_undegraded":"2023-10-16T11:25:51.303115+0000","last_fullsized":"2023-10-16T11:25:51.303115+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.791145+0000","last_change":"2023-10-16T11:25:51.308305+0000","last_active":"2023-10-16T11:25:52.791145+0000","last_peered":"2023-10-16T11:25:52.791145+0000","last_clean":"2023-10-16T11:25:52.791145+0000","last_became_active":"2023-10-16T11:25:51.307975+0000","last_became_peered":"2023-10-16T11:25:51.307975+0000","last_unstale":"2023-10-16T11:25:52.791145+0000","last_undegraded":"2023-10-16T11:25:52.791145+0000","last_fullsized":"2023-10-16T11:25:52.791145+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:48:44.730513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"241'1","reported_seq":13,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.724059+0000","last_change":"2023-10-16T11:25:50.307833+0000","last_active":"2023-10-16T11:25:52.724059+0000","last_peered":"2023-10-16T11:25:52.724059+0000","last_clean":"2023-10-16T11:25:52.724059+0000","last_became_active":"2023-10-16T11:25:50.307526+0000","last_became_peered":"2023-10-16T11:25:50.307526+0000","last_unstale":"2023-10-16T11:25:52.724059+0000","last_undegraded":"2023-10-16T11:25:52.724059+0000","last_fullsized":"2023-10-16T11:25:52.724059+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:55:11.599298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.39","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317697+0000","last_change":"2023-10-16T11:25:37.182745+0000","last_active":"2023-10-16T11:25:51.317697+0000","last_peered":"2023-10-16T11:25:51.317697+0000","last_clean":"2023-10-16T11:25:51.317697+0000","last_became_active":"2023-10-16T11:25:37.182437+0000","last_became_peered":"2023-10-16T11:25:37.182437+0000","last_unstale":"2023-10-16T11:25:51.317697+0000","last_undegraded":"2023-10-16T11:25:51.317697+0000","last_fullsized":"2023-10-16T11:25:51.317697+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:01.106638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.321194+0000","last_change":"2023-10-16T11:25:51.325207+0000","last_active":"2023-10-16T11:25:52.321194+0000","last_peered":"2023-10-16T11:25:52.321194+0000","last_clean":"2023-10-16T11:25:52.321194+0000","last_became_active":"2023-10-16T11:25:51.323968+0000","last_became_peered":"2023-10-16T11:25:51.323968+0000","last_unstale":"2023-10-16T11:25:52.321194+0000","last_undegraded":"2023-10-16T11:25:52.321194+0000","last_fullsized":"2023-10-16T11:25:52.321194+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:06.585825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"240'2","reported_seq":14,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.348098+0000","last_change":"2023-10-16T11:25:50.292129+0000","last_active":"2023-10-16T11:25:51.348098+0000","last_peered":"2023-10-16T11:25:51.348098+0000","last_clean":"2023-10-16T11:25:51.348098+0000","last_became_active":"2023-10-16T11:25:50.291743+0000","last_became_peered":"2023-10-16T11:25:50.291743+0000","last_unstale":"2023-10-16T11:25:51.348098+0000","last_undegraded":"2023-10-16T11:25:51.348098+0000","last_fullsized":"2023-10-16T11:25:51.348098+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:34:31.362688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.38","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.294901+0000","last_change":"2023-10-16T11:25:37.174929+0000","last_active":"2023-10-16T11:25:51.294901+0000","last_peered":"2023-10-16T11:25:51.294901+0000","last_clean":"2023-10-16T11:25:51.294901+0000","last_became_active":"2023-10-16T11:25:37.172810+0000","last_became_peered":"2023-10-16T11:25:37.172810+0000","last_unstale":"2023-10-16T11:25:51.294901+0000","last_undegraded":"2023-10-16T11:25:51.294901+0000","last_fullsized":"2023-10-16T11:25:51.294901+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.315690+0000","last_change":"2023-10-16T11:25:51.335355+0000","last_active":"2023-10-16T11:25:52.315690+0000","last_peered":"2023-10-16T11:25:52.315690+0000","last_clean":"2023-10-16T11:25:52.315690+0000","last_became_active":"2023-10-16T11:25:51.335089+0000","last_became_peered":"2023-10-16T11:25:51.335089+0000","last_unstale":"2023-10-16T11:25:52.315690+0000","last_undegraded":"2023-10-16T11:25:52.315690+0000","last_fullsized":"2023-10-16T11:25:52.315690+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:20.974486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.295492+0000","last_change":"2023-10-16T11:25:50.303972+0000","last_active":"2023-10-16T11:25:51.295492+0000","last_peered":"2023-10-16T11:25:51.295492+0000","last_clean":"2023-10-16T11:25:51.295492+0000","last_became_active":"2023-10-16T11:25:50.302166+0000","last_became_peered":"2023-10-16T11:25:50.302166+0000","last_unstale":"2023-10-16T11:25:51.295492+0000","last_undegraded":"2023-10-16T11:25:51.295492+0000","last_fullsized":"2023-10-16T11:25:51.295492+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:54.968684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.37","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317928+0000","last_change":"2023-10-16T11:25:37.177647+0000","last_active":"2023-10-16T11:25:51.317928+0000","last_peered":"2023-10-16T11:25:51.317928+0000","last_clean":"2023-10-16T11:25:51.317928+0000","last_became_active":"2023-10-16T11:25:37.176875+0000","last_became_peered":"2023-10-16T11:25:37.176875+0000","last_unstale":"2023-10-16T11:25:51.317928+0000","last_undegraded":"2023-10-16T11:25:51.317928+0000","last_fullsized":"2023-10-16T11:25:51.317928+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:21.630921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789782+0000","last_change":"2023-10-16T11:25:51.307779+0000","last_active":"2023-10-16T11:25:52.789782+0000","last_peered":"2023-10-16T11:25:52.789782+0000","last_clean":"2023-10-16T11:25:52.789782+0000","last_became_active":"2023-10-16T11:25:51.307456+0000","last_became_peered":"2023-10-16T11:25:51.307456+0000","last_unstale":"2023-10-16T11:25:52.789782+0000","last_undegraded":"2023-10-16T11:25:52.789782+0000","last_fullsized":"2023-10-16T11:25:52.789782+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:11.229022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.319094+0000","last_change":"2023-10-16T11:25:50.303286+0000","last_active":"2023-10-16T11:25:51.319094+0000","last_peered":"2023-10-16T11:25:51.319094+0000","last_clean":"2023-10-16T11:25:51.319094+0000","last_became_active":"2023-10-16T11:25:50.302625+0000","last_became_peered":"2023-10-16T11:25:50.302625+0000","last_unstale":"2023-10-16T11:25:51.319094+0000","last_undegraded":"2023-10-16T11:25:51.319094+0000","last_fullsized":"2023-10-16T11:25:51.319094+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:42:35.667844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.36","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.313090+0000","last_change":"2023-10-16T11:25:37.172841+0000","last_active":"2023-10-16T11:25:51.313090+0000","last_peered":"2023-10-16T11:25:51.313090+0000","last_clean":"2023-10-16T11:25:51.313090+0000","last_became_active":"2023-10-16T11:25:37.172550+0000","last_became_peered":"2023-10-16T11:25:37.172550+0000","last_unstale":"2023-10-16T11:25:51.313090+0000","last_undegraded":"2023-10-16T11:25:51.313090+0000","last_fullsized":"2023-10-16T11:25:51.313090+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.788520+0000","last_change":"2023-10-16T11:25:51.317443+0000","last_active":"2023-10-16T11:25:52.788520+0000","last_peered":"2023-10-16T11:25:52.788520+0000","last_clean":"2023-10-16T11:25:52.788520+0000","last_became_active":"2023-10-16T11:25:51.316556+0000","last_became_peered":"2023-10-16T11:25:51.316556+0000","last_unstale":"2023-10-16T11:25:52.788520+0000","last_undegraded":"2023-10-16T11:25:52.788520+0000","last_fullsized":"2023-10-16T11:25:52.788520+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:51.178566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.294779+0000","last_change":"2023-10-16T11:25:50.290580+0000","last_active":"2023-10-16T11:25:51.294779+0000","last_peered":"2023-10-16T11:25:51.294779+0000","last_clean":"2023-10-16T11:25:51.294779+0000","last_became_active":"2023-10-16T11:25:50.290044+0000","last_became_peered":"2023-10-16T11:25:50.290044+0000","last_unstale":"2023-10-16T11:25:51.294779+0000","last_undegraded":"2023-10-16T11:25:51.294779+0000","last_fullsized":"2023-10-16T11:25:51.294779+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:34:31.362688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.35","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.320218+0000","last_change":"2023-10-16T11:25:37.181386+0000","last_active":"2023-10-16T11:25:51.320218+0000","last_peered":"2023-10-16T11:25:51.320218+0000","last_clean":"2023-10-16T11:25:51.320218+0000","last_became_active":"2023-10-16T11:25:37.181007+0000","last_became_peered":"2023-10-16T11:25:37.181007+0000","last_unstale":"2023-10-16T11:25:51.320218+0000","last_undegraded":"2023-10-16T11:25:51.320218+0000","last_fullsized":"2023-10-16T11:25:51.320218+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:46.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789452+0000","last_change":"2023-10-16T11:25:51.335795+0000","last_active":"2023-10-16T11:25:52.789452+0000","last_peered":"2023-10-16T11:25:52.789452+0000","last_clean":"2023-10-16T11:25:52.789452+0000","last_became_active":"2023-10-16T11:25:51.335492+0000","last_became_peered":"2023-10-16T11:25:51.335492+0000","last_unstale":"2023-10-16T11:25:52.789452+0000","last_undegraded":"2023-10-16T11:25:52.789452+0000","last_fullsized":"2023-10-16T11:25:52.789452+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:18.229520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"240'2","reported_seq":14,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.343582+0000","last_change":"2023-10-16T11:25:50.308431+0000","last_active":"2023-10-16T11:25:51.343582+0000","last_peered":"2023-10-16T11:25:51.343582+0000","last_clean":"2023-10-16T11:25:51.343582+0000","last_became_active":"2023-10-16T11:25:50.308094+0000","last_became_peered":"2023-10-16T11:25:50.308094+0000","last_unstale":"2023-10-16T11:25:51.343582+0000","last_undegraded":"2023-10-16T11:25:51.343582+0000","last_fullsized":"2023-10-16T11:25:51.343582+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:53:51.538566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.34","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.297827+0000","last_change":"2023-10-16T11:25:37.179103+0000","last_active":"2023-10-16T11:25:51.297827+0000","last_peered":"2023-10-16T11:25:51.297827+0000","last_clean":"2023-10-16T11:25:51.297827+0000","last_became_active":"2023-10-16T11:25:37.178445+0000","last_became_peered":"2023-10-16T11:25:37.178445+0000","last_unstale":"2023-10-16T11:25:51.297827+0000","last_undegraded":"2023-10-16T11:25:51.297827+0000","last_fullsized":"2023-10-16T11:25:51.297827+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:38.218704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.724278+0000","last_change":"2023-10-16T11:25:51.337272+0000","last_active":"2023-10-16T11:25:52.724278+0000","last_peered":"2023-10-16T11:25:52.724278+0000","last_clean":"2023-10-16T11:25:52.724278+0000","last_became_active":"2023-10-16T11:25:51.336260+0000","last_became_peered":"2023-10-16T11:25:51.336260+0000","last_unstale":"2023-10-16T11:25:52.724278+0000","last_undegraded":"2023-10-16T11:25:52.724278+0000","last_fullsized":"2023-10-16T11:25:52.724278+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:34:32.379582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.300778+0000","last_change":"2023-10-16T11:25:50.296580+0000","last_active":"2023-10-16T11:25:51.300778+0000","last_peered":"2023-10-16T11:25:51.300778+0000","last_clean":"2023-10-16T11:25:51.300778+0000","last_became_active":"2023-10-16T11:25:50.296102+0000","last_became_peered":"2023-10-16T11:25:50.296102+0000","last_unstale":"2023-10-16T11:25:51.300778+0000","last_undegraded":"2023-10-16T11:25:51.300778+0000","last_fullsized":"2023-10-16T11:25:51.300778+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:31:20.394734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.33","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.312802+0000","last_change":"2023-10-16T11:25:37.182921+0000","last_active":"2023-10-16T11:25:51.312802+0000","last_peered":"2023-10-16T11:25:51.312802+0000","last_clean":"2023-10-16T11:25:51.312802+0000","last_became_active":"2023-10-16T11:25:37.182603+0000","last_became_peered":"2023-10-16T11:25:37.182603+0000","last_unstale":"2023-10-16T11:25:51.312802+0000","last_undegraded":"2023-10-16T11:25:51.312802+0000","last_fullsized":"2023-10-16T11:25:51.312802+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.319468+0000","last_change":"2023-10-16T11:25:51.320743+0000","last_active":"2023-10-16T11:25:52.319468+0000","last_peered":"2023-10-16T11:25:52.319468+0000","last_clean":"2023-10-16T11:25:52.319468+0000","last_became_active":"2023-10-16T11:25:51.320377+0000","last_became_peered":"2023-10-16T11:25:51.320377+0000","last_unstale":"2023-10-16T11:25:52.319468+0000","last_undegraded":"2023-10-16T11:25:52.319468+0000","last_fullsized":"2023-10-16T11:25:52.319468+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:09.714929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"240'1","reported_seq":13,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.347820+0000","last_change":"2023-10-16T11:25:50.305580+0000","last_active":"2023-10-16T11:25:51.347820+0000","last_peered":"2023-10-16T11:25:51.347820+0000","last_clean":"2023-10-16T11:25:51.347820+0000","last_became_active":"2023-10-16T11:25:50.304710+0000","last_became_peered":"2023-10-16T11:25:50.304710+0000","last_unstale":"2023-10-16T11:25:51.347820+0000","last_undegraded":"2023-10-16T11:25:51.347820+0000","last_fullsized":"2023-10-16T11:25:51.347820+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:37.230753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.32","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.313025+0000","last_change":"2023-10-16T11:25:37.171715+0000","last_active":"2023-10-16T11:25:51.313025+0000","last_peered":"2023-10-16T11:25:51.313025+0000","last_clean":"2023-10-16T11:25:51.313025+0000","last_became_active":"2023-10-16T11:25:37.171421+0000","last_became_peered":"2023-10-16T11:25:37.171421+0000","last_unstale":"2023-10-16T11:25:51.313025+0000","last_undegraded":"2023-10-16T11:25:51.313025+0000","last_fullsized":"2023-10-16T11:25:51.313025+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:10.207081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.442721+0000","last_change":"2023-10-16T11:25:51.341460+0000","last_active":"2023-10-16T11:25:52.442721+0000","last_peered":"2023-10-16T11:25:52.442721+0000","last_clean":"2023-10-16T11:25:52.442721+0000","last_became_active":"2023-10-16T11:25:51.340947+0000","last_became_peered":"2023-10-16T11:25:51.340947+0000","last_unstale":"2023-10-16T11:25:52.442721+0000","last_undegraded":"2023-10-16T11:25:52.442721+0000","last_fullsized":"2023-10-16T11:25:52.442721+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:36.561514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"240'2","reported_seq":14,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.346856+0000","last_change":"2023-10-16T11:25:50.297668+0000","last_active":"2023-10-16T11:25:51.346856+0000","last_peered":"2023-10-16T11:25:51.346856+0000","last_clean":"2023-10-16T11:25:51.346856+0000","last_became_active":"2023-10-16T11:25:50.297397+0000","last_became_peered":"2023-10-16T11:25:50.297397+0000","last_unstale":"2023-10-16T11:25:51.346856+0000","last_undegraded":"2023-10-16T11:25:51.346856+0000","last_fullsized":"2023-10-16T11:25:51.346856+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:53:17.916147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.31","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.322765+0000","last_change":"2023-10-16T11:25:37.177928+0000","last_active":"2023-10-16T11:25:51.322765+0000","last_peered":"2023-10-16T11:25:51.322765+0000","last_clean":"2023-10-16T11:25:51.322765+0000","last_became_active":"2023-10-16T11:25:37.177612+0000","last_became_peered":"2023-10-16T11:25:37.177612+0000","last_unstale":"2023-10-16T11:25:51.322765+0000","last_undegraded":"2023-10-16T11:25:51.322765+0000","last_fullsized":"2023-10-16T11:25:51.322765+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:44.826687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.321638+0000","last_change":"2023-10-16T11:25:51.325376+0000","last_active":"2023-10-16T11:25:52.321638+0000","last_peered":"2023-10-16T11:25:52.321638+0000","last_clean":"2023-10-16T11:25:52.321638+0000","last_became_active":"2023-10-16T11:25:51.323998+0000","last_became_peered":"2023-10-16T11:25:51.323998+0000","last_unstale":"2023-10-16T11:25:52.321638+0000","last_undegraded":"2023-10-16T11:25:52.321638+0000","last_fullsized":"2023-10-16T11:25:52.321638+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:11.420914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"240'1","reported_seq":13,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.343502+0000","last_change":"2023-10-16T11:25:50.304807+0000","last_active":"2023-10-16T11:25:51.343502+0000","last_peered":"2023-10-16T11:25:51.343502+0000","last_clean":"2023-10-16T11:25:51.343502+0000","last_became_active":"2023-10-16T11:25:50.304523+0000","last_became_peered":"2023-10-16T11:25:50.304523+0000","last_unstale":"2023-10-16T11:25:51.343502+0000","last_undegraded":"2023-10-16T11:25:51.343502+0000","last_fullsized":"2023-10-16T11:25:51.343502+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:31.156993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.30","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.302645+0000","last_change":"2023-10-16T11:25:37.170292+0000","last_active":"2023-10-16T11:25:51.302645+0000","last_peered":"2023-10-16T11:25:51.302645+0000","last_clean":"2023-10-16T11:25:51.302645+0000","last_became_active":"2023-10-16T11:25:37.169997+0000","last_became_peered":"2023-10-16T11:25:37.169997+0000","last_unstale":"2023-10-16T11:25:51.302645+0000","last_undegraded":"2023-10-16T11:25:51.302645+0000","last_fullsized":"2023-10-16T11:25:51.302645+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:09.329721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789393+0000","last_change":"2023-10-16T11:25:51.327001+0000","last_active":"2023-10-16T11:25:52.789393+0000","last_peered":"2023-10-16T11:25:52.789393+0000","last_clean":"2023-10-16T11:25:52.789393+0000","last_became_active":"2023-10-16T11:25:51.326746+0000","last_became_peered":"2023-10-16T11:25:51.326746+0000","last_unstale":"2023-10-16T11:25:52.789393+0000","last_undegraded":"2023-10-16T11:25:52.789393+0000","last_fullsized":"2023-10-16T11:25:52.789393+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:19.314057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"240'8","reported_seq":20,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.349111+0000","last_change":"2023-10-16T11:25:50.297630+0000","last_active":"2023-10-16T11:25:51.349111+0000","last_peered":"2023-10-16T11:25:51.349111+0000","last_clean":"2023-10-16T11:25:51.349111+0000","last_became_active":"2023-10-16T11:25:50.295103+0000","last_became_peered":"2023-10-16T11:25:50.295103+0000","last_unstale":"2023-10-16T11:25:51.349111+0000","last_undegraded":"2023-10-16T11:25:51.349111+0000","last_fullsized":"2023-10-16T11:25:51.349111+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:58.688176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2f","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.300068+0000","last_change":"2023-10-16T11:25:37.162923+0000","last_active":"2023-10-16T11:25:51.300068+0000","last_peered":"2023-10-16T11:25:51.300068+0000","last_clean":"2023-10-16T11:25:51.300068+0000","last_became_active":"2023-10-16T11:25:37.162649+0000","last_became_peered":"2023-10-16T11:25:37.162649+0000","last_unstale":"2023-10-16T11:25:51.300068+0000","last_undegraded":"2023-10-16T11:25:51.300068+0000","last_fullsized":"2023-10-16T11:25:51.300068+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:46.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789936+0000","last_change":"2023-10-16T11:25:51.321287+0000","last_active":"2023-10-16T11:25:52.789936+0000","last_peered":"2023-10-16T11:25:52.789936+0000","last_clean":"2023-10-16T11:25:52.789936+0000","last_became_active":"2023-10-16T11:25:51.320555+0000","last_became_peered":"2023-10-16T11:25:51.320555+0000","last_unstale":"2023-10-16T11:25:52.789936+0000","last_undegraded":"2023-10-16T11:25:52.789936+0000","last_fullsized":"2023-10-16T11:25:52.789936+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:15.713642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.322899+0000","last_change":"2023-10-16T11:25:50.306156+0000","last_active":"2023-10-16T11:25:51.322899+0000","last_peered":"2023-10-16T11:25:51.322899+0000","last_clean":"2023-10-16T11:25:51.322899+0000","last_became_active":"2023-10-16T11:25:50.305883+0000","last_became_peered":"2023-10-16T11:25:50.305883+0000","last_unstale":"2023-10-16T11:25:51.322899+0000","last_undegraded":"2023-10-16T11:25:51.322899+0000","last_fullsized":"2023-10-16T11:25:51.322899+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:42:45.711414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.2e","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.315147+0000","last_change":"2023-10-16T11:25:37.174044+0000","last_active":"2023-10-16T11:25:51.315147+0000","last_peered":"2023-10-16T11:25:51.315147+0000","last_clean":"2023-10-16T11:25:51.315147+0000","last_became_active":"2023-10-16T11:25:37.166694+0000","last_became_peered":"2023-10-16T11:25:37.166694+0000","last_unstale":"2023-10-16T11:25:51.315147+0000","last_undegraded":"2023-10-16T11:25:51.315147+0000","last_fullsized":"2023-10-16T11:25:51.315147+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:23.146288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.788098+0000","last_change":"2023-10-16T11:25:51.326577+0000","last_active":"2023-10-16T11:25:52.788098+0000","last_peered":"2023-10-16T11:25:52.788098+0000","last_clean":"2023-10-16T11:25:52.788098+0000","last_became_active":"2023-10-16T11:25:51.326129+0000","last_became_peered":"2023-10-16T11:25:51.326129+0000","last_unstale":"2023-10-16T11:25:52.788098+0000","last_undegraded":"2023-10-16T11:25:52.788098+0000","last_fullsized":"2023-10-16T11:25:52.788098+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:11.767351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.323367+0000","last_change":"2023-10-16T11:25:50.293946+0000","last_active":"2023-10-16T11:25:51.323367+0000","last_peered":"2023-10-16T11:25:51.323367+0000","last_clean":"2023-10-16T11:25:51.323367+0000","last_became_active":"2023-10-16T11:25:50.293183+0000","last_became_peered":"2023-10-16T11:25:50.293183+0000","last_unstale":"2023-10-16T11:25:51.323367+0000","last_undegraded":"2023-10-16T11:25:51.323367+0000","last_fullsized":"2023-10-16T11:25:51.323367+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:31.156993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2d","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.298245+0000","last_change":"2023-10-16T11:25:37.183893+0000","last_active":"2023-10-16T11:25:51.298245+0000","last_peered":"2023-10-16T11:25:51.298245+0000","last_clean":"2023-10-16T11:25:51.298245+0000","last_became_active":"2023-10-16T11:25:37.183586+0000","last_became_peered":"2023-10-16T11:25:37.183586+0000","last_unstale":"2023-10-16T11:25:51.298245+0000","last_undegraded":"2023-10-16T11:25:51.298245+0000","last_fullsized":"2023-10-16T11:25:51.298245+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:43.813878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789843+0000","last_change":"2023-10-16T11:25:51.320733+0000","last_active":"2023-10-16T11:25:52.789843+0000","last_peered":"2023-10-16T11:25:52.789843+0000","last_clean":"2023-10-16T11:25:52.789843+0000","last_became_active":"2023-10-16T11:25:51.320448+0000","last_became_peered":"2023-10-16T11:25:51.320448+0000","last_unstale":"2023-10-16T11:25:52.789843+0000","last_undegraded":"2023-10-16T11:25:52.789843+0000","last_fullsized":"2023-10-16T11:25:52.789843+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:15.782010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"240'2","reported_seq":14,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.348067+0000","last_change":"2023-10-16T11:25:50.298428+0000","last_active":"2023-10-16T11:25:51.348067+0000","last_peered":"2023-10-16T11:25:51.348067+0000","last_clean":"2023-10-16T11:25:51.348067+0000","last_became_active":"2023-10-16T11:25:50.297066+0000","last_became_peered":"2023-10-16T11:25:50.297066+0000","last_unstale":"2023-10-16T11:25:51.348067+0000","last_undegraded":"2023-10-16T11:25:51.348067+0000","last_fullsized":"2023-10-16T11:25:51.348067+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:24.028667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2c","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.301821+0000","last_change":"2023-10-16T11:25:37.172984+0000","last_active":"2023-10-16T11:25:51.301821+0000","last_peered":"2023-10-16T11:25:51.301821+0000","last_clean":"2023-10-16T11:25:51.301821+0000","last_became_active":"2023-10-16T11:25:37.172403+0000","last_became_peered":"2023-10-16T11:25:37.172403+0000","last_unstale":"2023-10-16T11:25:51.301821+0000","last_undegraded":"2023-10-16T11:25:51.301821+0000","last_fullsized":"2023-10-16T11:25:51.301821+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:03.912995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.790904+0000","last_change":"2023-10-16T11:25:51.324490+0000","last_active":"2023-10-16T11:25:52.790904+0000","last_peered":"2023-10-16T11:25:52.790904+0000","last_clean":"2023-10-16T11:25:52.790904+0000","last_became_active":"2023-10-16T11:25:51.324241+0000","last_became_peered":"2023-10-16T11:25:51.324241+0000","last_unstale":"2023-10-16T11:25:52.790904+0000","last_undegraded":"2023-10-16T11:25:52.790904+0000","last_fullsized":"2023-10-16T11:25:52.790904+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:15:02.083129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"240'6","reported_seq":18,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.350676+0000","last_change":"2023-10-16T11:25:50.282643+0000","last_active":"2023-10-16T11:25:51.350676+0000","last_peered":"2023-10-16T11:25:51.350676+0000","last_clean":"2023-10-16T11:25:51.350676+0000","last_became_active":"2023-10-16T11:25:50.282381+0000","last_became_peered":"2023-10-16T11:25:50.282381+0000","last_unstale":"2023-10-16T11:25:51.350676+0000","last_undegraded":"2023-10-16T11:25:51.350676+0000","last_fullsized":"2023-10-16T11:25:51.350676+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:39:01.747122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2b","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.296272+0000","last_change":"2023-10-16T11:25:37.173026+0000","last_active":"2023-10-16T11:25:51.296272+0000","last_peered":"2023-10-16T11:25:51.296272+0000","last_clean":"2023-10-16T11:25:51.296272+0000","last_became_active":"2023-10-16T11:25:37.172680+0000","last_became_peered":"2023-10-16T11:25:37.172680+0000","last_unstale":"2023-10-16T11:25:51.296272+0000","last_undegraded":"2023-10-16T11:25:51.296272+0000","last_fullsized":"2023-10-16T11:25:51.296272+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:56.314696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.319590+0000","last_change":"2023-10-16T11:25:51.306352+0000","last_active":"2023-10-16T11:25:52.319590+0000","last_peered":"2023-10-16T11:25:52.319590+0000","last_clean":"2023-10-16T11:25:52.319590+0000","last_became_active":"2023-10-16T11:25:51.305895+0000","last_became_peered":"2023-10-16T11:25:51.305895+0000","last_unstale":"2023-10-16T11:25:52.319590+0000","last_undegraded":"2023-10-16T11:25:52.319590+0000","last_fullsized":"2023-10-16T11:25:52.319590+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:24.349747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"240'3","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.347853+0000","last_change":"2023-10-16T11:25:50.303987+0000","last_active":"2023-10-16T11:25:51.347853+0000","last_peered":"2023-10-16T11:25:51.347853+0000","last_clean":"2023-10-16T11:25:51.347853+0000","last_became_active":"2023-10-16T11:25:50.303000+0000","last_became_peered":"2023-10-16T11:25:50.303000+0000","last_unstale":"2023-10-16T11:25:51.347853+0000","last_undegraded":"2023-10-16T11:25:51.347853+0000","last_fullsized":"2023-10-16T11:25:51.347853+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:53:25.980767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2a","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.311092+0000","last_change":"2023-10-16T11:25:37.176870+0000","last_active":"2023-10-16T11:25:51.311092+0000","last_peered":"2023-10-16T11:25:51.311092+0000","last_clean":"2023-10-16T11:25:51.311092+0000","last_became_active":"2023-10-16T11:25:37.175230+0000","last_became_peered":"2023-10-16T11:25:37.175230+0000","last_unstale":"2023-10-16T11:25:51.311092+0000","last_undegraded":"2023-10-16T11:25:51.311092+0000","last_fullsized":"2023-10-16T11:25:51.311092+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.790894+0000","last_change":"2023-10-16T11:25:51.317013+0000","last_active":"2023-10-16T11:25:52.790894+0000","last_peered":"2023-10-16T11:25:52.790894+0000","last_clean":"2023-10-16T11:25:52.790894+0000","last_became_active":"2023-10-16T11:25:51.316725+0000","last_became_peered":"2023-10-16T11:25:51.316725+0000","last_unstale":"2023-10-16T11:25:52.790894+0000","last_undegraded":"2023-10-16T11:25:52.790894+0000","last_fullsized":"2023-10-16T11:25:52.790894+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:33:52.278971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.314785+0000","last_change":"2023-10-16T11:25:50.302794+0000","last_active":"2023-10-16T11:25:51.314785+0000","last_peered":"2023-10-16T11:25:51.314785+0000","last_clean":"2023-10-16T11:25:51.314785+0000","last_became_active":"2023-10-16T11:25:50.302353+0000","last_became_peered":"2023-10-16T11:25:50.302353+0000","last_unstale":"2023-10-16T11:25:51.314785+0000","last_undegraded":"2023-10-16T11:25:51.314785+0000","last_fullsized":"2023-10-16T11:25:51.314785+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:12:02.269450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.29","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.293943+0000","last_change":"2023-10-16T11:25:37.169637+0000","last_active":"2023-10-16T11:25:51.293943+0000","last_peered":"2023-10-16T11:25:51.293943+0000","last_clean":"2023-10-16T11:25:51.293943+0000","last_became_active":"2023-10-16T11:25:37.169375+0000","last_became_peered":"2023-10-16T11:25:37.169375+0000","last_unstale":"2023-10-16T11:25:51.293943+0000","last_undegraded":"2023-10-16T11:25:51.293943+0000","last_fullsized":"2023-10-16T11:25:51.293943+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:42:23.188876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789651+0000","last_change":"2023-10-16T11:25:51.327693+0000","last_active":"2023-10-16T11:25:52.789651+0000","last_peered":"2023-10-16T11:25:52.789651+0000","last_clean":"2023-10-16T11:25:52.789651+0000","last_became_active":"2023-10-16T11:25:51.326040+0000","last_became_peered":"2023-10-16T11:25:51.326040+0000","last_unstale":"2023-10-16T11:25:52.789651+0000","last_undegraded":"2023-10-16T11:25:52.789651+0000","last_fullsized":"2023-10-16T11:25:52.789651+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:06.585825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.307532+0000","last_change":"2023-10-16T11:25:50.304100+0000","last_active":"2023-10-16T11:25:51.307532+0000","last_peered":"2023-10-16T11:25:51.307532+0000","last_clean":"2023-10-16T11:25:51.307532+0000","last_became_active":"2023-10-16T11:25:50.303840+0000","last_became_peered":"2023-10-16T11:25:50.303840+0000","last_unstale":"2023-10-16T11:25:51.307532+0000","last_undegraded":"2023-10-16T11:25:51.307532+0000","last_fullsized":"2023-10-16T11:25:51.307532+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:41.793494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.28","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.312147+0000","last_change":"2023-10-16T11:25:37.187270+0000","last_active":"2023-10-16T11:25:51.312147+0000","last_peered":"2023-10-16T11:25:51.312147+0000","last_clean":"2023-10-16T11:25:51.312147+0000","last_became_active":"2023-10-16T11:25:37.186962+0000","last_became_peered":"2023-10-16T11:25:37.186962+0000","last_unstale":"2023-10-16T11:25:51.312147+0000","last_undegraded":"2023-10-16T11:25:51.312147+0000","last_fullsized":"2023-10-16T11:25:51.312147+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:56:45.915920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.788719+0000","last_change":"2023-10-16T11:25:51.326733+0000","last_active":"2023-10-16T11:25:52.788719+0000","last_peered":"2023-10-16T11:25:52.788719+0000","last_clean":"2023-10-16T11:25:52.788719+0000","last_became_active":"2023-10-16T11:25:51.325905+0000","last_became_peered":"2023-10-16T11:25:51.325905+0000","last_unstale":"2023-10-16T11:25:52.788719+0000","last_undegraded":"2023-10-16T11:25:52.788719+0000","last_fullsized":"2023-10-16T11:25:52.788719+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:09.714929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.311169+0000","last_change":"2023-10-16T11:25:50.298538+0000","last_active":"2023-10-16T11:25:51.311169+0000","last_peered":"2023-10-16T11:25:51.311169+0000","last_clean":"2023-10-16T11:25:51.311169+0000","last_became_active":"2023-10-16T11:25:50.298265+0000","last_became_peered":"2023-10-16T11:25:50.298265+0000","last_unstale":"2023-10-16T11:25:51.311169+0000","last_undegraded":"2023-10-16T11:25:51.311169+0000","last_fullsized":"2023-10-16T11:25:51.311169+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:34:31.362688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.27","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.305448+0000","last_change":"2023-10-16T11:25:37.177057+0000","last_active":"2023-10-16T11:25:51.305448+0000","last_peered":"2023-10-16T11:25:51.305448+0000","last_clean":"2023-10-16T11:25:51.305448+0000","last_became_active":"2023-10-16T11:25:37.171369+0000","last_became_peered":"2023-10-16T11:25:37.171369+0000","last_unstale":"2023-10-16T11:25:51.305448+0000","last_undegraded":"2023-10-16T11:25:51.305448+0000","last_fullsized":"2023-10-16T11:25:51.305448+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.787285+0000","last_change":"2023-10-16T11:25:51.323651+0000","last_active":"2023-10-16T11:25:52.787285+0000","last_peered":"2023-10-16T11:25:52.787285+0000","last_clean":"2023-10-16T11:25:52.787285+0000","last_became_active":"2023-10-16T11:25:51.323150+0000","last_became_peered":"2023-10-16T11:25:51.323150+0000","last_unstale":"2023-10-16T11:25:52.787285+0000","last_undegraded":"2023-10-16T11:25:52.787285+0000","last_fullsized":"2023-10-16T11:25:52.787285+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:40.093581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":12,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.320872+0000","last_change":"2023-10-16T11:25:50.307251+0000","last_active":"2023-10-16T11:25:51.320872+0000","last_peered":"2023-10-16T11:25:51.320872+0000","last_clean":"2023-10-16T11:25:51.320872+0000","last_became_active":"2023-10-16T11:25:50.306831+0000","last_became_peered":"2023-10-16T11:25:50.306831+0000","last_unstale":"2023-10-16T11:25:51.320872+0000","last_undegraded":"2023-10-16T11:25:51.320872+0000","last_fullsized":"2023-10-16T11:25:51.320872+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:41.271897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.26","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.313225+0000","last_change":"2023-10-16T11:25:37.182875+0000","last_active":"2023-10-16T11:25:51.313225+0000","last_peered":"2023-10-16T11:25:51.313225+0000","last_clean":"2023-10-16T11:25:51.313225+0000","last_became_active":"2023-10-16T11:25:37.182570+0000","last_became_peered":"2023-10-16T11:25:37.182570+0000","last_unstale":"2023-10-16T11:25:51.313225+0000","last_undegraded":"2023-10-16T11:25:51.313225+0000","last_fullsized":"2023-10-16T11:25:51.313225+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:14.504702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.787755+0000","last_change":"2023-10-16T11:25:51.320594+0000","last_active":"2023-10-16T11:25:52.787755+0000","last_peered":"2023-10-16T11:25:52.787755+0000","last_clean":"2023-10-16T11:25:52.787755+0000","last_became_active":"2023-10-16T11:25:51.319783+0000","last_became_peered":"2023-10-16T11:25:51.319783+0000","last_unstale":"2023-10-16T11:25:52.787755+0000","last_undegraded":"2023-10-16T11:25:52.787755+0000","last_fullsized":"2023-10-16T11:25:52.787755+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:32.798332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"240'1","reported_seq":13,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.344852+0000","last_change":"2023-10-16T11:25:50.299291+0000","last_active":"2023-10-16T11:25:51.344852+0000","last_peered":"2023-10-16T11:25:51.344852+0000","last_clean":"2023-10-16T11:25:51.344852+0000","last_became_active":"2023-10-16T11:25:50.298997+0000","last_became_peered":"2023-10-16T11:25:50.298997+0000","last_unstale":"2023-10-16T11:25:51.344852+0000","last_undegraded":"2023-10-16T11:25:51.344852+0000","last_fullsized":"2023-10-16T11:25:51.344852+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:17.289884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.25","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.320279+0000","last_change":"2023-10-16T11:25:37.178245+0000","last_active":"2023-10-16T11:25:51.320279+0000","last_peered":"2023-10-16T11:25:51.320279+0000","last_clean":"2023-10-16T11:25:51.320279+0000","last_became_active":"2023-10-16T11:25:37.177914+0000","last_became_peered":"2023-10-16T11:25:37.177914+0000","last_unstale":"2023-10-16T11:25:51.320279+0000","last_undegraded":"2023-10-16T11:25:51.320279+0000","last_fullsized":"2023-10-16T11:25:51.320279+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:12.014513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.316581+0000","last_change":"2023-10-16T11:25:51.334335+0000","last_active":"2023-10-16T11:25:52.316581+0000","last_peered":"2023-10-16T11:25:52.316581+0000","last_clean":"2023-10-16T11:25:52.316581+0000","last_became_active":"2023-10-16T11:25:51.333152+0000","last_became_peered":"2023-10-16T11:25:51.333152+0000","last_unstale":"2023-10-16T11:25:52.316581+0000","last_undegraded":"2023-10-16T11:25:52.316581+0000","last_fullsized":"2023-10-16T11:25:52.316581+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:49.517942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"240'1","reported_seq":13,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.340669+0000","last_change":"2023-10-16T11:25:50.303326+0000","last_active":"2023-10-16T11:25:51.340669+0000","last_peered":"2023-10-16T11:25:51.340669+0000","last_clean":"2023-10-16T11:25:51.340669+0000","last_became_active":"2023-10-16T11:25:50.300956+0000","last_became_peered":"2023-10-16T11:25:50.300956+0000","last_unstale":"2023-10-16T11:25:51.340669+0000","last_undegraded":"2023-10-16T11:25:51.340669+0000","last_fullsized":"2023-10-16T11:25:51.340669+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:31.156993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.24","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.310834+0000","last_change":"2023-10-16T11:25:37.170623+0000","last_active":"2023-10-16T11:25:51.310834+0000","last_peered":"2023-10-16T11:25:51.310834+0000","last_clean":"2023-10-16T11:25:51.310834+0000","last_became_active":"2023-10-16T11:25:37.170365+0000","last_became_peered":"2023-10-16T11:25:37.170365+0000","last_unstale":"2023-10-16T11:25:51.310834+0000","last_undegraded":"2023-10-16T11:25:51.310834+0000","last_fullsized":"2023-10-16T11:25:51.310834+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:52.443160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.318419+0000","last_change":"2023-10-16T11:25:51.335417+0000","last_active":"2023-10-16T11:25:52.318419+0000","last_peered":"2023-10-16T11:25:52.318419+0000","last_clean":"2023-10-16T11:25:52.318419+0000","last_became_active":"2023-10-16T11:25:51.335099+0000","last_became_peered":"2023-10-16T11:25:51.335099+0000","last_unstale":"2023-10-16T11:25:52.318419+0000","last_undegraded":"2023-10-16T11:25:52.318419+0000","last_fullsized":"2023-10-16T11:25:52.318419+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:57:00.058585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"240'2","reported_seq":14,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.347122+0000","last_change":"2023-10-16T11:25:50.294581+0000","last_active":"2023-10-16T11:25:51.347122+0000","last_peered":"2023-10-16T11:25:51.347122+0000","last_clean":"2023-10-16T11:25:51.347122+0000","last_became_active":"2023-10-16T11:25:50.294303+0000","last_became_peered":"2023-10-16T11:25:50.294303+0000","last_unstale":"2023-10-16T11:25:51.347122+0000","last_undegraded":"2023-10-16T11:25:51.347122+0000","last_fullsized":"2023-10-16T11:25:51.347122+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:49.447144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.23","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317555+0000","last_change":"2023-10-16T11:25:37.178693+0000","last_active":"2023-10-16T11:25:51.317555+0000","last_peered":"2023-10-16T11:25:51.317555+0000","last_clean":"2023-10-16T11:25:51.317555+0000","last_became_active":"2023-10-16T11:25:37.175403+0000","last_became_peered":"2023-10-16T11:25:37.175403+0000","last_unstale":"2023-10-16T11:25:51.317555+0000","last_undegraded":"2023-10-16T11:25:51.317555+0000","last_fullsized":"2023-10-16T11:25:51.317555+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:15:31.572558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.788897+0000","last_change":"2023-10-16T11:25:51.324613+0000","last_active":"2023-10-16T11:25:52.788897+0000","last_peered":"2023-10-16T11:25:52.788897+0000","last_clean":"2023-10-16T11:25:52.788897+0000","last_became_active":"2023-10-16T11:25:51.323228+0000","last_became_peered":"2023-10-16T11:25:51.323228+0000","last_unstale":"2023-10-16T11:25:52.788897+0000","last_undegraded":"2023-10-16T11:25:52.788897+0000","last_fullsized":"2023-10-16T11:25:52.788897+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:14:07.754755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"240'2","reported_seq":14,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.347956+0000","last_change":"2023-10-16T11:25:50.308596+0000","last_active":"2023-10-16T11:25:51.347956+0000","last_peered":"2023-10-16T11:25:51.347956+0000","last_clean":"2023-10-16T11:25:51.347956+0000","last_became_active":"2023-10-16T11:25:50.308237+0000","last_became_peered":"2023-10-16T11:25:50.308237+0000","last_unstale":"2023-10-16T11:25:51.347956+0000","last_undegraded":"2023-10-16T11:25:51.347956+0000","last_fullsized":"2023-10-16T11:25:51.347956+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:16:43.826775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.22","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.324091+0000","last_change":"2023-10-16T11:25:37.181812+0000","last_active":"2023-10-16T11:25:51.324091+0000","last_peered":"2023-10-16T11:25:51.324091+0000","last_clean":"2023-10-16T11:25:51.324091+0000","last_became_active":"2023-10-16T11:25:37.181529+0000","last_became_peered":"2023-10-16T11:25:37.181529+0000","last_unstale":"2023-10-16T11:25:51.324091+0000","last_undegraded":"2023-10-16T11:25:51.324091+0000","last_fullsized":"2023-10-16T11:25:51.324091+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:35.912014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.322190+0000","last_change":"2023-10-16T11:25:51.326648+0000","last_active":"2023-10-16T11:25:52.322190+0000","last_peered":"2023-10-16T11:25:52.322190+0000","last_clean":"2023-10-16T11:25:52.322190+0000","last_became_active":"2023-10-16T11:25:51.314971+0000","last_became_peered":"2023-10-16T11:25:51.314971+0000","last_unstale":"2023-10-16T11:25:52.322190+0000","last_undegraded":"2023-10-16T11:25:52.322190+0000","last_fullsized":"2023-10-16T11:25:52.322190+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:32.743783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"240'1","reported_seq":13,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.346766+0000","last_change":"2023-10-16T11:25:50.290455+0000","last_active":"2023-10-16T11:25:51.346766+0000","last_peered":"2023-10-16T11:25:51.346766+0000","last_clean":"2023-10-16T11:25:51.346766+0000","last_became_active":"2023-10-16T11:25:50.290130+0000","last_became_peered":"2023-10-16T11:25:50.290130+0000","last_unstale":"2023-10-16T11:25:51.346766+0000","last_undegraded":"2023-10-16T11:25:51.346766+0000","last_fullsized":"2023-10-16T11:25:51.346766+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:01:55.571004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.21","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.294716+0000","last_change":"2023-10-16T11:25:37.166981+0000","last_active":"2023-10-16T11:25:51.294716+0000","last_peered":"2023-10-16T11:25:51.294716+0000","last_clean":"2023-10-16T11:25:51.294716+0000","last_became_active":"2023-10-16T11:25:37.164147+0000","last_became_peered":"2023-10-16T11:25:37.164147+0000","last_unstale":"2023-10-16T11:25:51.294716+0000","last_undegraded":"2023-10-16T11:25:51.294716+0000","last_fullsized":"2023-10-16T11:25:51.294716+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.790718+0000","last_change":"2023-10-16T11:25:51.328794+0000","last_active":"2023-10-16T11:25:52.790718+0000","last_peered":"2023-10-16T11:25:52.790718+0000","last_clean":"2023-10-16T11:25:52.790718+0000","last_became_active":"2023-10-16T11:25:51.328364+0000","last_became_peered":"2023-10-16T11:25:51.328364+0000","last_unstale":"2023-10-16T11:25:52.790718+0000","last_undegraded":"2023-10-16T11:25:52.790718+0000","last_fullsized":"2023-10-16T11:25:52.790718+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:21.934164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"240'4","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.348348+0000","last_change":"2023-10-16T11:25:50.300080+0000","last_active":"2023-10-16T11:25:51.348348+0000","last_peered":"2023-10-16T11:25:51.348348+0000","last_clean":"2023-10-16T11:25:51.348348+0000","last_became_active":"2023-10-16T11:25:50.299271+0000","last_became_peered":"2023-10-16T11:25:50.299271+0000","last_unstale":"2023-10-16T11:25:51.348348+0000","last_undegraded":"2023-10-16T11:25:51.348348+0000","last_fullsized":"2023-10-16T11:25:51.348348+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:49.243481+0000","last_clean_scrub_stamp":"2023-10-16T11:25:49.243481+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:30:50.501877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.20","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.312252+0000","last_change":"2023-10-16T11:25:37.178631+0000","last_active":"2023-10-16T11:25:51.312252+0000","last_peered":"2023-10-16T11:25:51.312252+0000","last_clean":"2023-10-16T11:25:51.312252+0000","last_became_active":"2023-10-16T11:25:37.167402+0000","last_became_peered":"2023-10-16T11:25:37.167402+0000","last_unstale":"2023-10-16T11:25:51.312252+0000","last_undegraded":"2023-10-16T11:25:51.312252+0000","last_fullsized":"2023-10-16T11:25:51.312252+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:55.572264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.316346+0000","last_change":"2023-10-16T11:25:51.325042+0000","last_active":"2023-10-16T11:25:52.316346+0000","last_peered":"2023-10-16T11:25:52.316346+0000","last_clean":"2023-10-16T11:25:52.316346+0000","last_became_active":"2023-10-16T11:25:51.324731+0000","last_became_peered":"2023-10-16T11:25:51.324731+0000","last_unstale":"2023-10-16T11:25:52.316346+0000","last_undegraded":"2023-10-16T11:25:52.316346+0000","last_fullsized":"2023-10-16T11:25:52.316346+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:48.937625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.308890+0000","last_change":"2023-10-16T11:25:37.177706+0000","last_active":"2023-10-16T11:25:51.308890+0000","last_peered":"2023-10-16T11:25:51.308890+0000","last_clean":"2023-10-16T11:25:51.308890+0000","last_became_active":"2023-10-16T11:25:37.176806+0000","last_became_peered":"2023-10-16T11:25:37.176806+0000","last_unstale":"2023-10-16T11:25:51.308890+0000","last_undegraded":"2023-10-16T11:25:51.308890+0000","last_fullsized":"2023-10-16T11:25:51.308890+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:42:32.585642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1e","version":"236'1","reported_seq":17,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.313596+0000","last_change":"2023-10-16T11:25:36.169327+0000","last_active":"2023-10-16T11:25:51.313596+0000","last_peered":"2023-10-16T11:25:51.313596+0000","last_clean":"2023-10-16T11:25:51.313596+0000","last_became_active":"2023-10-16T11:25:36.169046+0000","last_became_peered":"2023-10-16T11:25:36.169046+0000","last_unstale":"2023-10-16T11:25:51.313596+0000","last_undegraded":"2023-10-16T11:25:51.313596+0000","last_fullsized":"2023-10-16T11:25:51.313596+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:34.899695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.318653+0000","last_change":"2023-10-16T11:25:51.324955+0000","last_active":"2023-10-16T11:25:52.318653+0000","last_peered":"2023-10-16T11:25:52.318653+0000","last_clean":"2023-10-16T11:25:52.318653+0000","last_became_active":"2023-10-16T11:25:51.314480+0000","last_became_peered":"2023-10-16T11:25:51.314480+0000","last_unstale":"2023-10-16T11:25:52.318653+0000","last_undegraded":"2023-10-16T11:25:52.318653+0000","last_fullsized":"2023-10-16T11:25:52.318653+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:57:00.058585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317994+0000","last_change":"2023-10-16T11:25:37.173736+0000","last_active":"2023-10-16T11:25:51.317994+0000","last_peered":"2023-10-16T11:25:51.317994+0000","last_clean":"2023-10-16T11:25:51.317994+0000","last_became_active":"2023-10-16T11:25:37.173482+0000","last_became_peered":"2023-10-16T11:25:37.173482+0000","last_unstale":"2023-10-16T11:25:51.317994+0000","last_undegraded":"2023-10-16T11:25:51.317994+0000","last_fullsized":"2023-10-16T11:25:51.317994+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:42:23.188876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1d","version":"236'2","reported_seq":18,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.306367+0000","last_change":"2023-10-16T11:25:36.172861+0000","last_active":"2023-10-16T11:25:51.306367+0000","last_peered":"2023-10-16T11:25:51.306367+0000","last_clean":"2023-10-16T11:25:51.306367+0000","last_became_active":"2023-10-16T11:25:36.172574+0000","last_became_peered":"2023-10-16T11:25:36.172574+0000","last_unstale":"2023-10-16T11:25:51.306367+0000","last_undegraded":"2023-10-16T11:25:51.306367+0000","last_fullsized":"2023-10-16T11:25:51.306367+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:22:17.667795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789424+0000","last_change":"2023-10-16T11:25:51.317976+0000","last_active":"2023-10-16T11:25:52.789424+0000","last_peered":"2023-10-16T11:25:52.789424+0000","last_clean":"2023-10-16T11:25:52.789424+0000","last_became_active":"2023-10-16T11:25:51.317525+0000","last_became_peered":"2023-10-16T11:25:51.317525+0000","last_unstale":"2023-10-16T11:25:52.789424+0000","last_undegraded":"2023-10-16T11:25:52.789424+0000","last_fullsized":"2023-10-16T11:25:52.789424+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:29.234750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.298248+0000","last_change":"2023-10-16T11:25:37.167451+0000","last_active":"2023-10-16T11:25:51.298248+0000","last_peered":"2023-10-16T11:25:51.298248+0000","last_clean":"2023-10-16T11:25:51.298248+0000","last_became_active":"2023-10-16T11:25:37.167157+0000","last_became_peered":"2023-10-16T11:25:37.167157+0000","last_unstale":"2023-10-16T11:25:51.298248+0000","last_undegraded":"2023-10-16T11:25:51.298248+0000","last_fullsized":"2023-10-16T11:25:51.298248+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:59:14.504702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1c","version":"236'2","reported_seq":18,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.302706+0000","last_change":"2023-10-16T11:25:36.153273+0000","last_active":"2023-10-16T11:25:51.302706+0000","last_peered":"2023-10-16T11:25:51.302706+0000","last_clean":"2023-10-16T11:25:51.302706+0000","last_became_active":"2023-10-16T11:25:36.152960+0000","last_became_peered":"2023-10-16T11:25:36.152960+0000","last_unstale":"2023-10-16T11:25:51.302706+0000","last_undegraded":"2023-10-16T11:25:51.302706+0000","last_fullsized":"2023-10-16T11:25:51.302706+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:08.218268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.320150+0000","last_change":"2023-10-16T11:25:51.328191+0000","last_active":"2023-10-16T11:25:52.320150+0000","last_peered":"2023-10-16T11:25:52.320150+0000","last_clean":"2023-10-16T11:25:52.320150+0000","last_became_active":"2023-10-16T11:25:51.327805+0000","last_became_peered":"2023-10-16T11:25:51.327805+0000","last_unstale":"2023-10-16T11:25:52.320150+0000","last_undegraded":"2023-10-16T11:25:52.320150+0000","last_fullsized":"2023-10-16T11:25:52.320150+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:37.288954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.323547+0000","last_change":"2023-10-16T11:25:37.174071+0000","last_active":"2023-10-16T11:25:51.323547+0000","last_peered":"2023-10-16T11:25:51.323547+0000","last_clean":"2023-10-16T11:25:51.323547+0000","last_became_active":"2023-10-16T11:25:37.173436+0000","last_became_peered":"2023-10-16T11:25:37.173436+0000","last_unstale":"2023-10-16T11:25:51.323547+0000","last_undegraded":"2023-10-16T11:25:51.323547+0000","last_fullsized":"2023-10-16T11:25:51.323547+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1b","version":"236'1","reported_seq":17,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.303522+0000","last_change":"2023-10-16T11:25:36.156408+0000","last_active":"2023-10-16T11:25:51.303522+0000","last_peered":"2023-10-16T11:25:51.303522+0000","last_clean":"2023-10-16T11:25:51.303522+0000","last_became_active":"2023-10-16T11:25:36.155786+0000","last_became_peered":"2023-10-16T11:25:36.155786+0000","last_unstale":"2023-10-16T11:25:51.303522+0000","last_undegraded":"2023-10-16T11:25:51.303522+0000","last_fullsized":"2023-10-16T11:25:51.303522+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:01.925277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789918+0000","last_change":"2023-10-16T11:25:51.332041+0000","last_active":"2023-10-16T11:25:52.789918+0000","last_peered":"2023-10-16T11:25:52.789918+0000","last_clean":"2023-10-16T11:25:52.789918+0000","last_became_active":"2023-10-16T11:25:51.329122+0000","last_became_peered":"2023-10-16T11:25:51.329122+0000","last_unstale":"2023-10-16T11:25:52.789918+0000","last_undegraded":"2023-10-16T11:25:52.789918+0000","last_fullsized":"2023-10-16T11:25:52.789918+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:32.743783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317182+0000","last_change":"2023-10-16T11:25:37.179656+0000","last_active":"2023-10-16T11:25:51.317182+0000","last_peered":"2023-10-16T11:25:51.317182+0000","last_clean":"2023-10-16T11:25:51.317182+0000","last_became_active":"2023-10-16T11:25:37.179361+0000","last_became_peered":"2023-10-16T11:25:37.179361+0000","last_unstale":"2023-10-16T11:25:51.317182+0000","last_undegraded":"2023-10-16T11:25:51.317182+0000","last_fullsized":"2023-10-16T11:25:51.317182+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:09.230587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"236'1","reported_seq":17,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.313144+0000","last_change":"2023-10-16T11:25:36.169348+0000","last_active":"2023-10-16T11:25:51.313144+0000","last_peered":"2023-10-16T11:25:51.313144+0000","last_clean":"2023-10-16T11:25:51.313144+0000","last_became_active":"2023-10-16T11:25:36.169053+0000","last_became_peered":"2023-10-16T11:25:36.169053+0000","last_unstale":"2023-10-16T11:25:51.313144+0000","last_undegraded":"2023-10-16T11:25:51.313144+0000","last_fullsized":"2023-10-16T11:25:51.313144+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:05.819502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.724196+0000","last_change":"2023-10-16T11:25:51.327481+0000","last_active":"2023-10-16T11:25:52.724196+0000","last_peered":"2023-10-16T11:25:52.724196+0000","last_clean":"2023-10-16T11:25:52.724196+0000","last_became_active":"2023-10-16T11:25:51.327206+0000","last_became_peered":"2023-10-16T11:25:51.327206+0000","last_unstale":"2023-10-16T11:25:52.724196+0000","last_undegraded":"2023-10-16T11:25:52.724196+0000","last_fullsized":"2023-10-16T11:25:52.724196+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:31:21.411627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.302848+0000","last_change":"2023-10-16T11:25:37.176966+0000","last_active":"2023-10-16T11:25:51.302848+0000","last_peered":"2023-10-16T11:25:51.302848+0000","last_clean":"2023-10-16T11:25:51.302848+0000","last_became_active":"2023-10-16T11:25:37.174621+0000","last_became_peered":"2023-10-16T11:25:37.174621+0000","last_unstale":"2023-10-16T11:25:51.302848+0000","last_undegraded":"2023-10-16T11:25:51.302848+0000","last_fullsized":"2023-10-16T11:25:51.302848+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:56:09.516122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317917+0000","last_change":"2023-10-16T11:25:36.169635+0000","last_active":"2023-10-16T11:25:51.317917+0000","last_peered":"2023-10-16T11:25:51.317917+0000","last_clean":"2023-10-16T11:25:51.317917+0000","last_became_active":"2023-10-16T11:25:36.169016+0000","last_became_peered":"2023-10-16T11:25:36.169016+0000","last_unstale":"2023-10-16T11:25:51.317917+0000","last_undegraded":"2023-10-16T11:25:51.317917+0000","last_fullsized":"2023-10-16T11:25:51.317917+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:53:06.412234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.322296+0000","last_change":"2023-10-16T11:25:51.327247+0000","last_active":"2023-10-16T11:25:52.322296+0000","last_peered":"2023-10-16T11:25:52.322296+0000","last_clean":"2023-10-16T11:25:52.322296+0000","last_became_active":"2023-10-16T11:25:51.326955+0000","last_became_peered":"2023-10-16T11:25:51.326955+0000","last_unstale":"2023-10-16T11:25:52.322296+0000","last_undegraded":"2023-10-16T11:25:52.322296+0000","last_fullsized":"2023-10-16T11:25:52.322296+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:59:08.686800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.310236+0000","last_change":"2023-10-16T11:25:37.184475+0000","last_active":"2023-10-16T11:25:51.310236+0000","last_peered":"2023-10-16T11:25:51.310236+0000","last_clean":"2023-10-16T11:25:51.310236+0000","last_became_active":"2023-10-16T11:25:37.179914+0000","last_became_peered":"2023-10-16T11:25:37.179914+0000","last_unstale":"2023-10-16T11:25:51.310236+0000","last_undegraded":"2023-10-16T11:25:51.310236+0000","last_fullsized":"2023-10-16T11:25:51.310236+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:53.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.308698+0000","last_change":"2023-10-16T11:25:36.155589+0000","last_active":"2023-10-16T11:25:51.308698+0000","last_peered":"2023-10-16T11:25:51.308698+0000","last_clean":"2023-10-16T11:25:51.308698+0000","last_became_active":"2023-10-16T11:25:36.155087+0000","last_became_peered":"2023-10-16T11:25:36.155087+0000","last_unstale":"2023-10-16T11:25:51.308698+0000","last_undegraded":"2023-10-16T11:25:51.308698+0000","last_fullsized":"2023-10-16T11:25:51.308698+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:52:38.937212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789190+0000","last_change":"2023-10-16T11:25:51.321470+0000","last_active":"2023-10-16T11:25:52.789190+0000","last_peered":"2023-10-16T11:25:52.789190+0000","last_clean":"2023-10-16T11:25:52.789190+0000","last_became_active":"2023-10-16T11:25:51.320974+0000","last_became_peered":"2023-10-16T11:25:51.320974+0000","last_unstale":"2023-10-16T11:25:52.789190+0000","last_undegraded":"2023-10-16T11:25:52.789190+0000","last_fullsized":"2023-10-16T11:25:52.789190+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:23.241770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.315243+0000","last_change":"2023-10-16T11:25:37.175447+0000","last_active":"2023-10-16T11:25:51.315243+0000","last_peered":"2023-10-16T11:25:51.315243+0000","last_clean":"2023-10-16T11:25:51.315243+0000","last_became_active":"2023-10-16T11:25:37.173563+0000","last_became_peered":"2023-10-16T11:25:37.173563+0000","last_unstale":"2023-10-16T11:25:51.315243+0000","last_undegraded":"2023-10-16T11:25:51.315243+0000","last_fullsized":"2023-10-16T11:25:51.315243+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:54.544135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.301615+0000","last_change":"2023-10-16T11:25:36.169351+0000","last_active":"2023-10-16T11:25:51.301615+0000","last_peered":"2023-10-16T11:25:51.301615+0000","last_clean":"2023-10-16T11:25:51.301615+0000","last_became_active":"2023-10-16T11:25:36.168347+0000","last_became_peered":"2023-10-16T11:25:36.168347+0000","last_unstale":"2023-10-16T11:25:51.301615+0000","last_undegraded":"2023-10-16T11:25:51.301615+0000","last_fullsized":"2023-10-16T11:25:51.301615+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:19.077096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.788244+0000","last_change":"2023-10-16T11:25:51.333596+0000","last_active":"2023-10-16T11:25:52.788244+0000","last_peered":"2023-10-16T11:25:52.788244+0000","last_clean":"2023-10-16T11:25:52.788244+0000","last_became_active":"2023-10-16T11:25:51.333175+0000","last_became_peered":"2023-10-16T11:25:51.333175+0000","last_unstale":"2023-10-16T11:25:52.788244+0000","last_undegraded":"2023-10-16T11:25:52.788244+0000","last_fullsized":"2023-10-16T11:25:52.788244+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:49.517942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.316449+0000","last_change":"2023-10-16T11:25:37.179048+0000","last_active":"2023-10-16T11:25:51.316449+0000","last_peered":"2023-10-16T11:25:51.316449+0000","last_clean":"2023-10-16T11:25:51.316449+0000","last_became_active":"2023-10-16T11:25:37.178751+0000","last_became_peered":"2023-10-16T11:25:37.178751+0000","last_unstale":"2023-10-16T11:25:51.316449+0000","last_undegraded":"2023-10-16T11:25:51.316449+0000","last_fullsized":"2023-10-16T11:25:51.316449+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:48:10.906108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.306108+0000","last_change":"2023-10-16T11:25:36.154256+0000","last_active":"2023-10-16T11:25:51.306108+0000","last_peered":"2023-10-16T11:25:51.306108+0000","last_clean":"2023-10-16T11:25:51.306108+0000","last_became_active":"2023-10-16T11:25:36.153421+0000","last_became_peered":"2023-10-16T11:25:36.153421+0000","last_unstale":"2023-10-16T11:25:51.306108+0000","last_undegraded":"2023-10-16T11:25:51.306108+0000","last_fullsized":"2023-10-16T11:25:51.306108+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:09.436508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.315787+0000","last_change":"2023-10-16T11:25:51.336271+0000","last_active":"2023-10-16T11:25:52.315787+0000","last_peered":"2023-10-16T11:25:52.315787+0000","last_clean":"2023-10-16T11:25:52.315787+0000","last_became_active":"2023-10-16T11:25:51.335956+0000","last_became_peered":"2023-10-16T11:25:51.335956+0000","last_unstale":"2023-10-16T11:25:52.315787+0000","last_undegraded":"2023-10-16T11:25:52.315787+0000","last_fullsized":"2023-10-16T11:25:52.315787+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:48.937625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.302135+0000","last_change":"2023-10-16T11:25:37.188270+0000","last_active":"2023-10-16T11:25:51.302135+0000","last_peered":"2023-10-16T11:25:51.302135+0000","last_clean":"2023-10-16T11:25:51.302135+0000","last_became_active":"2023-10-16T11:25:37.187995+0000","last_became_peered":"2023-10-16T11:25:37.187995+0000","last_unstale":"2023-10-16T11:25:51.302135+0000","last_undegraded":"2023-10-16T11:25:51.302135+0000","last_fullsized":"2023-10-16T11:25:51.302135+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:42:22.542072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.15","version":"236'3","reported_seq":19,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.315149+0000","last_change":"2023-10-16T11:25:36.171645+0000","last_active":"2023-10-16T11:25:51.315149+0000","last_peered":"2023-10-16T11:25:51.315149+0000","last_clean":"2023-10-16T11:25:51.315149+0000","last_became_active":"2023-10-16T11:25:36.170729+0000","last_became_peered":"2023-10-16T11:25:36.170729+0000","last_unstale":"2023-10-16T11:25:51.315149+0000","last_undegraded":"2023-10-16T11:25:51.315149+0000","last_fullsized":"2023-10-16T11:25:51.315149+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:34.362958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.317193+0000","last_change":"2023-10-16T11:25:51.324637+0000","last_active":"2023-10-16T11:25:52.317193+0000","last_peered":"2023-10-16T11:25:52.317193+0000","last_clean":"2023-10-16T11:25:52.317193+0000","last_became_active":"2023-10-16T11:25:51.323712+0000","last_became_peered":"2023-10-16T11:25:51.323712+0000","last_unstale":"2023-10-16T11:25:52.317193+0000","last_undegraded":"2023-10-16T11:25:52.317193+0000","last_fullsized":"2023-10-16T11:25:52.317193+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:57:00.058585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.312295+0000","last_change":"2023-10-16T11:25:37.163991+0000","last_active":"2023-10-16T11:25:51.312295+0000","last_peered":"2023-10-16T11:25:51.312295+0000","last_clean":"2023-10-16T11:25:51.312295+0000","last_became_active":"2023-10-16T11:25:37.163080+0000","last_became_peered":"2023-10-16T11:25:37.163080+0000","last_unstale":"2023-10-16T11:25:51.312295+0000","last_undegraded":"2023-10-16T11:25:51.312295+0000","last_fullsized":"2023-10-16T11:25:51.312295+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.14","version":"237'8","reported_seq":24,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.310933+0000","last_change":"2023-10-16T11:25:36.173259+0000","last_active":"2023-10-16T11:25:51.310933+0000","last_peered":"2023-10-16T11:25:51.310933+0000","last_clean":"2023-10-16T11:25:51.310933+0000","last_became_active":"2023-10-16T11:25:36.172980+0000","last_became_peered":"2023-10-16T11:25:36.172980+0000","last_unstale":"2023-10-16T11:25:51.310933+0000","last_undegraded":"2023-10-16T11:25:51.310933+0000","last_fullsized":"2023-10-16T11:25:51.310933+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:33.782641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.788001+0000","last_change":"2023-10-16T11:25:51.325646+0000","last_active":"2023-10-16T11:25:52.788001+0000","last_peered":"2023-10-16T11:25:52.788001+0000","last_clean":"2023-10-16T11:25:52.788001+0000","last_became_active":"2023-10-16T11:25:51.325326+0000","last_became_peered":"2023-10-16T11:25:51.325326+0000","last_unstale":"2023-10-16T11:25:52.788001+0000","last_undegraded":"2023-10-16T11:25:52.788001+0000","last_fullsized":"2023-10-16T11:25:52.788001+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:01.749517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.313880+0000","last_change":"2023-10-16T11:25:37.166081+0000","last_active":"2023-10-16T11:25:51.313880+0000","last_peered":"2023-10-16T11:25:51.313880+0000","last_clean":"2023-10-16T11:25:51.313880+0000","last_became_active":"2023-10-16T11:25:37.165150+0000","last_became_peered":"2023-10-16T11:25:37.165150+0000","last_unstale":"2023-10-16T11:25:51.313880+0000","last_undegraded":"2023-10-16T11:25:51.313880+0000","last_fullsized":"2023-10-16T11:25:51.313880+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:56.314696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"236'2","reported_seq":18,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.314626+0000","last_change":"2023-10-16T11:25:36.171468+0000","last_active":"2023-10-16T11:25:51.314626+0000","last_peered":"2023-10-16T11:25:51.314626+0000","last_clean":"2023-10-16T11:25:51.314626+0000","last_became_active":"2023-10-16T11:25:36.171236+0000","last_became_peered":"2023-10-16T11:25:36.171236+0000","last_unstale":"2023-10-16T11:25:51.314626+0000","last_undegraded":"2023-10-16T11:25:51.314626+0000","last_fullsized":"2023-10-16T11:25:51.314626+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:17.018903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789810+0000","last_change":"2023-10-16T11:25:51.327239+0000","last_active":"2023-10-16T11:25:52.789810+0000","last_peered":"2023-10-16T11:25:52.789810+0000","last_clean":"2023-10-16T11:25:52.789810+0000","last_became_active":"2023-10-16T11:25:51.326951+0000","last_became_peered":"2023-10-16T11:25:51.326951+0000","last_unstale":"2023-10-16T11:25:52.789810+0000","last_undegraded":"2023-10-16T11:25:52.789810+0000","last_fullsized":"2023-10-16T11:25:52.789810+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:11.420914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317290+0000","last_change":"2023-10-16T11:25:37.178517+0000","last_active":"2023-10-16T11:25:51.317290+0000","last_peered":"2023-10-16T11:25:51.317290+0000","last_clean":"2023-10-16T11:25:51.317290+0000","last_became_active":"2023-10-16T11:25:37.177794+0000","last_became_peered":"2023-10-16T11:25:37.177794+0000","last_unstale":"2023-10-16T11:25:51.317290+0000","last_undegraded":"2023-10-16T11:25:51.317290+0000","last_fullsized":"2023-10-16T11:25:51.317290+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:52:39.949531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.300023+0000","last_change":"2023-10-16T11:25:36.162595+0000","last_active":"2023-10-16T11:25:51.300023+0000","last_peered":"2023-10-16T11:25:51.300023+0000","last_clean":"2023-10-16T11:25:51.300023+0000","last_became_active":"2023-10-16T11:25:36.162249+0000","last_became_peered":"2023-10-16T11:25:36.162249+0000","last_unstale":"2023-10-16T11:25:51.300023+0000","last_undegraded":"2023-10-16T11:25:51.300023+0000","last_fullsized":"2023-10-16T11:25:51.300023+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:49.547165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789988+0000","last_change":"2023-10-16T11:25:51.324884+0000","last_active":"2023-10-16T11:25:52.789988+0000","last_peered":"2023-10-16T11:25:52.789988+0000","last_clean":"2023-10-16T11:25:52.789988+0000","last_became_active":"2023-10-16T11:25:51.320528+0000","last_became_peered":"2023-10-16T11:25:51.320528+0000","last_unstale":"2023-10-16T11:25:52.789988+0000","last_undegraded":"2023-10-16T11:25:52.789988+0000","last_fullsized":"2023-10-16T11:25:52.789988+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:59:08.686800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.293418+0000","last_change":"2023-10-16T11:25:37.164604+0000","last_active":"2023-10-16T11:25:51.293418+0000","last_peered":"2023-10-16T11:25:51.293418+0000","last_clean":"2023-10-16T11:25:51.293418+0000","last_became_active":"2023-10-16T11:25:37.162365+0000","last_became_peered":"2023-10-16T11:25:37.162365+0000","last_unstale":"2023-10-16T11:25:51.293418+0000","last_undegraded":"2023-10-16T11:25:51.293418+0000","last_fullsized":"2023-10-16T11:25:51.293418+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:15:35.912014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317999+0000","last_change":"2023-10-16T11:25:36.170014+0000","last_active":"2023-10-16T11:25:51.317999+0000","last_peered":"2023-10-16T11:25:51.317999+0000","last_clean":"2023-10-16T11:25:51.317999+0000","last_became_active":"2023-10-16T11:25:36.169755+0000","last_became_peered":"2023-10-16T11:25:36.169755+0000","last_unstale":"2023-10-16T11:25:51.317999+0000","last_undegraded":"2023-10-16T11:25:51.317999+0000","last_fullsized":"2023-10-16T11:25:51.317999+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:02.119904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.791648+0000","last_change":"2023-10-16T11:25:51.316368+0000","last_active":"2023-10-16T11:25:52.791648+0000","last_peered":"2023-10-16T11:25:52.791648+0000","last_clean":"2023-10-16T11:25:52.791648+0000","last_became_active":"2023-10-16T11:25:51.314352+0000","last_became_peered":"2023-10-16T11:25:51.314352+0000","last_unstale":"2023-10-16T11:25:52.791648+0000","last_undegraded":"2023-10-16T11:25:52.791648+0000","last_fullsized":"2023-10-16T11:25:52.791648+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:32:44.818063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317829+0000","last_change":"2023-10-16T11:25:37.178789+0000","last_active":"2023-10-16T11:25:51.317829+0000","last_peered":"2023-10-16T11:25:51.317829+0000","last_clean":"2023-10-16T11:25:51.317829+0000","last_became_active":"2023-10-16T11:25:37.178510+0000","last_became_peered":"2023-10-16T11:25:37.178510+0000","last_unstale":"2023-10-16T11:25:51.317829+0000","last_undegraded":"2023-10-16T11:25:51.317829+0000","last_fullsized":"2023-10-16T11:25:51.317829+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.10","version":"236'8","reported_seq":24,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.315707+0000","last_change":"2023-10-16T11:25:36.168889+0000","last_active":"2023-10-16T11:25:51.315707+0000","last_peered":"2023-10-16T11:25:51.315707+0000","last_clean":"2023-10-16T11:25:51.315707+0000","last_became_active":"2023-10-16T11:25:36.168590+0000","last_became_peered":"2023-10-16T11:25:36.168590+0000","last_unstale":"2023-10-16T11:25:51.315707+0000","last_undegraded":"2023-10-16T11:25:51.315707+0000","last_fullsized":"2023-10-16T11:25:51.315707+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:42.801560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.787261+0000","last_change":"2023-10-16T11:25:51.318378+0000","last_active":"2023-10-16T11:25:52.787261+0000","last_peered":"2023-10-16T11:25:52.787261+0000","last_clean":"2023-10-16T11:25:52.787261+0000","last_became_active":"2023-10-16T11:25:51.318080+0000","last_became_peered":"2023-10-16T11:25:51.318080+0000","last_unstale":"2023-10-16T11:25:52.787261+0000","last_undegraded":"2023-10-16T11:25:52.787261+0000","last_fullsized":"2023-10-16T11:25:52.787261+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:02:43.323091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.300112+0000","last_change":"2023-10-16T11:25:37.175992+0000","last_active":"2023-10-16T11:25:51.300112+0000","last_peered":"2023-10-16T11:25:51.300112+0000","last_clean":"2023-10-16T11:25:51.300112+0000","last_became_active":"2023-10-16T11:25:37.173732+0000","last_became_peered":"2023-10-16T11:25:37.173732+0000","last_unstale":"2023-10-16T11:25:51.300112+0000","last_undegraded":"2023-10-16T11:25:51.300112+0000","last_fullsized":"2023-10-16T11:25:51.300112+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.f","version":"236'1","reported_seq":17,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.316299+0000","last_change":"2023-10-16T11:25:36.173935+0000","last_active":"2023-10-16T11:25:51.316299+0000","last_peered":"2023-10-16T11:25:51.316299+0000","last_clean":"2023-10-16T11:25:51.316299+0000","last_became_active":"2023-10-16T11:25:36.173510+0000","last_became_peered":"2023-10-16T11:25:36.173510+0000","last_unstale":"2023-10-16T11:25:51.316299+0000","last_undegraded":"2023-10-16T11:25:51.316299+0000","last_fullsized":"2023-10-16T11:25:51.316299+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:19.077096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.317141+0000","last_change":"2023-10-16T11:25:51.318620+0000","last_active":"2023-10-16T11:25:52.317141+0000","last_peered":"2023-10-16T11:25:52.317141+0000","last_clean":"2023-10-16T11:25:52.317141+0000","last_became_active":"2023-10-16T11:25:51.311970+0000","last_became_peered":"2023-10-16T11:25:51.311970+0000","last_unstale":"2023-10-16T11:25:52.317141+0000","last_undegraded":"2023-10-16T11:25:52.317141+0000","last_fullsized":"2023-10-16T11:25:52.317141+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:48.937625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.307638+0000","last_change":"2023-10-16T11:25:37.177543+0000","last_active":"2023-10-16T11:25:51.307638+0000","last_peered":"2023-10-16T11:25:51.307638+0000","last_clean":"2023-10-16T11:25:51.307638+0000","last_became_active":"2023-10-16T11:25:37.176420+0000","last_became_peered":"2023-10-16T11:25:37.176420+0000","last_unstale":"2023-10-16T11:25:51.307638+0000","last_undegraded":"2023-10-16T11:25:51.307638+0000","last_fullsized":"2023-10-16T11:25:51.307638+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:24.104981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.e","version":"236'2","reported_seq":18,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.306267+0000","last_change":"2023-10-16T11:25:36.170878+0000","last_active":"2023-10-16T11:25:51.306267+0000","last_peered":"2023-10-16T11:25:51.306267+0000","last_clean":"2023-10-16T11:25:51.306267+0000","last_became_active":"2023-10-16T11:25:36.170548+0000","last_became_peered":"2023-10-16T11:25:36.170548+0000","last_unstale":"2023-10-16T11:25:51.306267+0000","last_undegraded":"2023-10-16T11:25:51.306267+0000","last_fullsized":"2023-10-16T11:25:51.306267+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:34:14.932409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.318304+0000","last_change":"2023-10-16T11:25:51.306315+0000","last_active":"2023-10-16T11:25:52.318304+0000","last_peered":"2023-10-16T11:25:52.318304+0000","last_clean":"2023-10-16T11:25:52.318304+0000","last_became_active":"2023-10-16T11:25:51.305603+0000","last_became_peered":"2023-10-16T11:25:51.305603+0000","last_unstale":"2023-10-16T11:25:52.318304+0000","last_undegraded":"2023-10-16T11:25:52.318304+0000","last_fullsized":"2023-10-16T11:25:52.318304+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:49.517942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.317969+0000","last_change":"2023-10-16T11:25:37.176420+0000","last_active":"2023-10-16T11:25:51.317969+0000","last_peered":"2023-10-16T11:25:51.317969+0000","last_clean":"2023-10-16T11:25:51.317969+0000","last_became_active":"2023-10-16T11:25:37.176101+0000","last_became_peered":"2023-10-16T11:25:37.176101+0000","last_unstale":"2023-10-16T11:25:51.317969+0000","last_undegraded":"2023-10-16T11:25:51.317969+0000","last_fullsized":"2023-10-16T11:25:51.317969+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:10.245184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"236'1","reported_seq":17,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.320217+0000","last_change":"2023-10-16T11:25:36.171476+0000","last_active":"2023-10-16T11:25:51.320217+0000","last_peered":"2023-10-16T11:25:51.320217+0000","last_clean":"2023-10-16T11:25:51.320217+0000","last_became_active":"2023-10-16T11:25:36.171201+0000","last_became_peered":"2023-10-16T11:25:36.171201+0000","last_unstale":"2023-10-16T11:25:51.320217+0000","last_undegraded":"2023-10-16T11:25:51.320217+0000","last_fullsized":"2023-10-16T11:25:51.320217+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:30.034841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.319537+0000","last_change":"2023-10-16T11:25:51.323557+0000","last_active":"2023-10-16T11:25:52.319537+0000","last_peered":"2023-10-16T11:25:52.319537+0000","last_clean":"2023-10-16T11:25:52.319537+0000","last_became_active":"2023-10-16T11:25:51.314015+0000","last_became_peered":"2023-10-16T11:25:51.314015+0000","last_unstale":"2023-10-16T11:25:52.319537+0000","last_undegraded":"2023-10-16T11:25:52.319537+0000","last_fullsized":"2023-10-16T11:25:52.319537+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:14:07.754755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.298471+0000","last_change":"2023-10-16T11:25:37.162366+0000","last_active":"2023-10-16T11:25:51.298471+0000","last_peered":"2023-10-16T11:25:51.298471+0000","last_clean":"2023-10-16T11:25:51.298471+0000","last_became_active":"2023-10-16T11:25:37.162032+0000","last_became_peered":"2023-10-16T11:25:37.162032+0000","last_unstale":"2023-10-16T11:25:51.298471+0000","last_undegraded":"2023-10-16T11:25:51.298471+0000","last_fullsized":"2023-10-16T11:25:51.298471+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.318585+0000","last_change":"2023-10-16T11:25:36.174246+0000","last_active":"2023-10-16T11:25:51.318585+0000","last_peered":"2023-10-16T11:25:51.318585+0000","last_clean":"2023-10-16T11:25:51.318585+0000","last_became_active":"2023-10-16T11:25:36.173975+0000","last_became_peered":"2023-10-16T11:25:36.173975+0000","last_unstale":"2023-10-16T11:25:51.318585+0000","last_undegraded":"2023-10-16T11:25:51.318585+0000","last_fullsized":"2023-10-16T11:25:51.318585+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:32.130559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.318259+0000","last_change":"2023-10-16T11:25:51.319530+0000","last_active":"2023-10-16T11:25:52.318259+0000","last_peered":"2023-10-16T11:25:52.318259+0000","last_clean":"2023-10-16T11:25:52.318259+0000","last_became_active":"2023-10-16T11:25:51.319035+0000","last_became_peered":"2023-10-16T11:25:51.319035+0000","last_unstale":"2023-10-16T11:25:52.318259+0000","last_undegraded":"2023-10-16T11:25:52.318259+0000","last_fullsized":"2023-10-16T11:25:52.318259+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:49.517942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.295002+0000","last_change":"2023-10-16T11:25:37.168715+0000","last_active":"2023-10-16T11:25:51.295002+0000","last_peered":"2023-10-16T11:25:51.295002+0000","last_clean":"2023-10-16T11:25:51.295002+0000","last_became_active":"2023-10-16T11:25:37.168387+0000","last_became_peered":"2023-10-16T11:25:37.168387+0000","last_unstale":"2023-10-16T11:25:51.295002+0000","last_undegraded":"2023-10-16T11:25:51.295002+0000","last_fullsized":"2023-10-16T11:25:51.295002+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:33.142878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.b","version":"236'2","reported_seq":18,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.299155+0000","last_change":"2023-10-16T11:25:36.174153+0000","last_active":"2023-10-16T11:25:51.299155+0000","last_peered":"2023-10-16T11:25:51.299155+0000","last_clean":"2023-10-16T11:25:51.299155+0000","last_became_active":"2023-10-16T11:25:36.173812+0000","last_became_peered":"2023-10-16T11:25:36.173812+0000","last_unstale":"2023-10-16T11:25:51.299155+0000","last_undegraded":"2023-10-16T11:25:51.299155+0000","last_fullsized":"2023-10-16T11:25:51.299155+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:07:05.931833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789599+0000","last_change":"2023-10-16T11:25:51.321769+0000","last_active":"2023-10-16T11:25:52.789599+0000","last_peered":"2023-10-16T11:25:52.789599+0000","last_clean":"2023-10-16T11:25:52.789599+0000","last_became_active":"2023-10-16T11:25:51.319324+0000","last_became_peered":"2023-10-16T11:25:51.319324+0000","last_unstale":"2023-10-16T11:25:52.789599+0000","last_undegraded":"2023-10-16T11:25:52.789599+0000","last_fullsized":"2023-10-16T11:25:52.789599+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:31:37.288954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.314528+0000","last_change":"2023-10-16T11:25:37.175767+0000","last_active":"2023-10-16T11:25:51.314528+0000","last_peered":"2023-10-16T11:25:51.314528+0000","last_clean":"2023-10-16T11:25:51.314528+0000","last_became_active":"2023-10-16T11:25:37.167389+0000","last_became_peered":"2023-10-16T11:25:37.167389+0000","last_unstale":"2023-10-16T11:25:51.314528+0000","last_undegraded":"2023-10-16T11:25:51.314528+0000","last_fullsized":"2023-10-16T11:25:51.314528+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:18.601118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.325646+0000","last_change":"2023-10-16T11:25:36.175478+0000","last_active":"2023-10-16T11:25:51.325646+0000","last_peered":"2023-10-16T11:25:51.325646+0000","last_clean":"2023-10-16T11:25:51.325646+0000","last_became_active":"2023-10-16T11:25:36.174935+0000","last_became_peered":"2023-10-16T11:25:36.174935+0000","last_unstale":"2023-10-16T11:25:51.325646+0000","last_undegraded":"2023-10-16T11:25:51.325646+0000","last_fullsized":"2023-10-16T11:25:51.325646+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:14.189489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789564+0000","last_change":"2023-10-16T11:25:51.328522+0000","last_active":"2023-10-16T11:25:52.789564+0000","last_peered":"2023-10-16T11:25:52.789564+0000","last_clean":"2023-10-16T11:25:52.789564+0000","last_became_active":"2023-10-16T11:25:51.328009+0000","last_became_peered":"2023-10-16T11:25:51.328009+0000","last_unstale":"2023-10-16T11:25:52.789564+0000","last_undegraded":"2023-10-16T11:25:52.789564+0000","last_fullsized":"2023-10-16T11:25:52.789564+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:24.349747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.315573+0000","last_change":"2023-10-16T11:25:37.173034+0000","last_active":"2023-10-16T11:25:51.315573+0000","last_peered":"2023-10-16T11:25:51.315573+0000","last_clean":"2023-10-16T11:25:51.315573+0000","last_became_active":"2023-10-16T11:25:37.172644+0000","last_became_peered":"2023-10-16T11:25:37.172644+0000","last_unstale":"2023-10-16T11:25:51.315573+0000","last_undegraded":"2023-10-16T11:25:51.315573+0000","last_fullsized":"2023-10-16T11:25:51.315573+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:34.948780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.322980+0000","last_change":"2023-10-16T11:25:36.163926+0000","last_active":"2023-10-16T11:25:51.322980+0000","last_peered":"2023-10-16T11:25:51.322980+0000","last_clean":"2023-10-16T11:25:51.322980+0000","last_became_active":"2023-10-16T11:25:36.162991+0000","last_became_peered":"2023-10-16T11:25:36.162991+0000","last_unstale":"2023-10-16T11:25:51.322980+0000","last_undegraded":"2023-10-16T11:25:51.322980+0000","last_fullsized":"2023-10-16T11:25:51.322980+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:49.547165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.317982+0000","last_change":"2023-10-16T11:25:51.329081+0000","last_active":"2023-10-16T11:25:52.317982+0000","last_peered":"2023-10-16T11:25:52.317982+0000","last_clean":"2023-10-16T11:25:52.317982+0000","last_became_active":"2023-10-16T11:25:51.328806+0000","last_became_peered":"2023-10-16T11:25:51.328806+0000","last_unstale":"2023-10-16T11:25:52.317982+0000","last_undegraded":"2023-10-16T11:25:52.317982+0000","last_fullsized":"2023-10-16T11:25:52.317982+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:45.962774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.294181+0000","last_change":"2023-10-16T11:25:37.158678+0000","last_active":"2023-10-16T11:25:51.294181+0000","last_peered":"2023-10-16T11:25:51.294181+0000","last_clean":"2023-10-16T11:25:51.294181+0000","last_became_active":"2023-10-16T11:25:37.157389+0000","last_became_peered":"2023-10-16T11:25:37.157389+0000","last_unstale":"2023-10-16T11:25:51.294181+0000","last_undegraded":"2023-10-16T11:25:51.294181+0000","last_fullsized":"2023-10-16T11:25:51.294181+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:15.201807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.311415+0000","last_change":"2023-10-16T11:25:36.171876+0000","last_active":"2023-10-16T11:25:51.311415+0000","last_peered":"2023-10-16T11:25:51.311415+0000","last_clean":"2023-10-16T11:25:51.311415+0000","last_became_active":"2023-10-16T11:25:36.169128+0000","last_became_peered":"2023-10-16T11:25:36.169128+0000","last_unstale":"2023-10-16T11:25:51.311415+0000","last_undegraded":"2023-10-16T11:25:51.311415+0000","last_fullsized":"2023-10-16T11:25:51.311415+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:16:18.153133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.316284+0000","last_change":"2023-10-16T11:25:51.315843+0000","last_active":"2023-10-16T11:25:52.316284+0000","last_peered":"2023-10-16T11:25:52.316284+0000","last_clean":"2023-10-16T11:25:52.316284+0000","last_became_active":"2023-10-16T11:25:51.314237+0000","last_became_peered":"2023-10-16T11:25:51.314237+0000","last_unstale":"2023-10-16T11:25:52.316284+0000","last_undegraded":"2023-10-16T11:25:52.316284+0000","last_fullsized":"2023-10-16T11:25:52.316284+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:20.974486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.313338+0000","last_change":"2023-10-16T11:25:37.179119+0000","last_active":"2023-10-16T11:25:51.313338+0000","last_peered":"2023-10-16T11:25:51.313338+0000","last_clean":"2023-10-16T11:25:51.313338+0000","last_became_active":"2023-10-16T11:25:37.175799+0000","last_became_peered":"2023-10-16T11:25:37.175799+0000","last_unstale":"2023-10-16T11:25:51.313338+0000","last_undegraded":"2023-10-16T11:25:51.313338+0000","last_fullsized":"2023-10-16T11:25:51.313338+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:03.691733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"236'2","reported_seq":18,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.304109+0000","last_change":"2023-10-16T11:25:36.162608+0000","last_active":"2023-10-16T11:25:51.304109+0000","last_peered":"2023-10-16T11:25:51.304109+0000","last_clean":"2023-10-16T11:25:51.304109+0000","last_became_active":"2023-10-16T11:25:36.162336+0000","last_became_peered":"2023-10-16T11:25:36.162336+0000","last_unstale":"2023-10-16T11:25:51.304109+0000","last_undegraded":"2023-10-16T11:25:51.304109+0000","last_fullsized":"2023-10-16T11:25:51.304109+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:19.377028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.788246+0000","last_change":"2023-10-16T11:25:51.327430+0000","last_active":"2023-10-16T11:25:52.788246+0000","last_peered":"2023-10-16T11:25:52.788246+0000","last_clean":"2023-10-16T11:25:52.788246+0000","last_became_active":"2023-10-16T11:25:51.327100+0000","last_became_peered":"2023-10-16T11:25:51.327100+0000","last_unstale":"2023-10-16T11:25:52.788246+0000","last_undegraded":"2023-10-16T11:25:52.788246+0000","last_fullsized":"2023-10-16T11:25:52.788246+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:57:00.058585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.314519+0000","last_change":"2023-10-16T11:25:37.177560+0000","last_active":"2023-10-16T11:25:51.314519+0000","last_peered":"2023-10-16T11:25:51.314519+0000","last_clean":"2023-10-16T11:25:51.314519+0000","last_became_active":"2023-10-16T11:25:37.176982+0000","last_became_peered":"2023-10-16T11:25:37.176982+0000","last_unstale":"2023-10-16T11:25:51.314519+0000","last_undegraded":"2023-10-16T11:25:51.314519+0000","last_fullsized":"2023-10-16T11:25:51.314519+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:54.497943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"237'1","reported_seq":17,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.325144+0000","last_change":"2023-10-16T11:25:36.174419+0000","last_active":"2023-10-16T11:25:51.325144+0000","last_peered":"2023-10-16T11:25:51.325144+0000","last_clean":"2023-10-16T11:25:51.325144+0000","last_became_active":"2023-10-16T11:25:36.174148+0000","last_became_peered":"2023-10-16T11:25:36.174148+0000","last_unstale":"2023-10-16T11:25:51.325144+0000","last_undegraded":"2023-10-16T11:25:51.325144+0000","last_fullsized":"2023-10-16T11:25:51.325144+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:38.739323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.789472+0000","last_change":"2023-10-16T11:25:51.330994+0000","last_active":"2023-10-16T11:25:52.789472+0000","last_peered":"2023-10-16T11:25:52.789472+0000","last_clean":"2023-10-16T11:25:52.789472+0000","last_became_active":"2023-10-16T11:25:51.330677+0000","last_became_peered":"2023-10-16T11:25:51.330677+0000","last_unstale":"2023-10-16T11:25:52.789472+0000","last_undegraded":"2023-10-16T11:25:52.789472+0000","last_fullsized":"2023-10-16T11:25:52.789472+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:45.189825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.318438+0000","last_change":"2023-10-16T11:25:37.180683+0000","last_active":"2023-10-16T11:25:51.318438+0000","last_peered":"2023-10-16T11:25:51.318438+0000","last_clean":"2023-10-16T11:25:51.318438+0000","last_became_active":"2023-10-16T11:25:37.180393+0000","last_became_peered":"2023-10-16T11:25:37.180393+0000","last_unstale":"2023-10-16T11:25:51.318438+0000","last_undegraded":"2023-10-16T11:25:51.318438+0000","last_fullsized":"2023-10-16T11:25:51.318438+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:38.218704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.320357+0000","last_change":"2023-10-16T11:25:36.164519+0000","last_active":"2023-10-16T11:25:51.320357+0000","last_peered":"2023-10-16T11:25:51.320357+0000","last_clean":"2023-10-16T11:25:51.320357+0000","last_became_active":"2023-10-16T11:25:36.164214+0000","last_became_peered":"2023-10-16T11:25:36.164214+0000","last_unstale":"2023-10-16T11:25:51.320357+0000","last_undegraded":"2023-10-16T11:25:51.320357+0000","last_fullsized":"2023-10-16T11:25:51.320357+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:03.074535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.723342+0000","last_change":"2023-10-16T11:25:51.325986+0000","last_active":"2023-10-16T11:25:52.723342+0000","last_peered":"2023-10-16T11:25:52.723342+0000","last_clean":"2023-10-16T11:25:52.723342+0000","last_became_active":"2023-10-16T11:25:51.325725+0000","last_became_peered":"2023-10-16T11:25:51.325725+0000","last_unstale":"2023-10-16T11:25:52.723342+0000","last_undegraded":"2023-10-16T11:25:52.723342+0000","last_fullsized":"2023-10-16T11:25:52.723342+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:01:56.587898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.320310+0000","last_change":"2023-10-16T11:25:37.181808+0000","last_active":"2023-10-16T11:25:51.320310+0000","last_peered":"2023-10-16T11:25:51.320310+0000","last_clean":"2023-10-16T11:25:51.320310+0000","last_became_active":"2023-10-16T11:25:37.181575+0000","last_became_peered":"2023-10-16T11:25:37.181575+0000","last_unstale":"2023-10-16T11:25:51.320310+0000","last_undegraded":"2023-10-16T11:25:51.320310+0000","last_fullsized":"2023-10-16T11:25:51.320310+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:17:01.131611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.318671+0000","last_change":"2023-10-16T11:25:36.162153+0000","last_active":"2023-10-16T11:25:51.318671+0000","last_peered":"2023-10-16T11:25:51.318671+0000","last_clean":"2023-10-16T11:25:51.318671+0000","last_became_active":"2023-10-16T11:25:36.161842+0000","last_became_peered":"2023-10-16T11:25:36.161842+0000","last_unstale":"2023-10-16T11:25:51.318671+0000","last_undegraded":"2023-10-16T11:25:51.318671+0000","last_fullsized":"2023-10-16T11:25:51.318671+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:55.302377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.723428+0000","last_change":"2023-10-16T11:25:51.326939+0000","last_active":"2023-10-16T11:25:52.723428+0000","last_peered":"2023-10-16T11:25:52.723428+0000","last_clean":"2023-10-16T11:25:52.723428+0000","last_became_active":"2023-10-16T11:25:51.326520+0000","last_became_peered":"2023-10-16T11:25:51.326520+0000","last_unstale":"2023-10-16T11:25:52.723428+0000","last_undegraded":"2023-10-16T11:25:52.723428+0000","last_fullsized":"2023-10-16T11:25:52.723428+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:48.107944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.314001+0000","last_change":"2023-10-16T11:25:37.165420+0000","last_active":"2023-10-16T11:25:51.314001+0000","last_peered":"2023-10-16T11:25:51.314001+0000","last_clean":"2023-10-16T11:25:51.314001+0000","last_became_active":"2023-10-16T11:25:37.164820+0000","last_became_peered":"2023-10-16T11:25:37.164820+0000","last_unstale":"2023-10-16T11:25:51.314001+0000","last_undegraded":"2023-10-16T11:25:51.314001+0000","last_fullsized":"2023-10-16T11:25:51.314001+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:43.813878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.312990+0000","last_change":"2023-10-16T11:25:36.173209+0000","last_active":"2023-10-16T11:25:51.312990+0000","last_peered":"2023-10-16T11:25:51.312990+0000","last_clean":"2023-10-16T11:25:51.312990+0000","last_became_active":"2023-10-16T11:25:36.171799+0000","last_became_peered":"2023-10-16T11:25:36.171799+0000","last_unstale":"2023-10-16T11:25:51.312990+0000","last_undegraded":"2023-10-16T11:25:51.312990+0000","last_fullsized":"2023-10-16T11:25:51.312990+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:19.377028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.724296+0000","last_change":"2023-10-16T11:25:51.332559+0000","last_active":"2023-10-16T11:25:52.724296+0000","last_peered":"2023-10-16T11:25:52.724296+0000","last_clean":"2023-10-16T11:25:52.724296+0000","last_became_active":"2023-10-16T11:25:51.332201+0000","last_became_peered":"2023-10-16T11:25:51.332201+0000","last_unstale":"2023-10-16T11:25:52.724296+0000","last_undegraded":"2023-10-16T11:25:52.724296+0000","last_fullsized":"2023-10-16T11:25:52.724296+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:55.985578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.296387+0000","last_change":"2023-10-16T11:25:37.180243+0000","last_active":"2023-10-16T11:25:51.296387+0000","last_peered":"2023-10-16T11:25:51.296387+0000","last_clean":"2023-10-16T11:25:51.296387+0000","last_became_active":"2023-10-16T11:25:37.179869+0000","last_became_peered":"2023-10-16T11:25:37.179869+0000","last_unstale":"2023-10-16T11:25:51.296387+0000","last_undegraded":"2023-10-16T11:25:51.296387+0000","last_fullsized":"2023-10-16T11:25:51.296387+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:15:31.572558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.304062+0000","last_change":"2023-10-16T11:25:36.169146+0000","last_active":"2023-10-16T11:25:51.304062+0000","last_peered":"2023-10-16T11:25:51.304062+0000","last_clean":"2023-10-16T11:25:51.304062+0000","last_became_active":"2023-10-16T11:25:36.168877+0000","last_became_peered":"2023-10-16T11:25:36.168877+0000","last_unstale":"2023-10-16T11:25:51.304062+0000","last_undegraded":"2023-10-16T11:25:51.304062+0000","last_fullsized":"2023-10-16T11:25:51.304062+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:35.988148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.443085+0000","last_change":"2023-10-16T11:25:51.325913+0000","last_active":"2023-10-16T11:25:52.443085+0000","last_peered":"2023-10-16T11:25:52.443085+0000","last_clean":"2023-10-16T11:25:52.443085+0000","last_became_active":"2023-10-16T11:25:51.325577+0000","last_became_peered":"2023-10-16T11:25:51.325577+0000","last_unstale":"2023-10-16T11:25:52.443085+0000","last_undegraded":"2023-10-16T11:25:52.443085+0000","last_fullsized":"2023-10-16T11:25:52.443085+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:22:19.277245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.295096+0000","last_change":"2023-10-16T11:25:37.173516+0000","last_active":"2023-10-16T11:25:51.295096+0000","last_peered":"2023-10-16T11:25:51.295096+0000","last_clean":"2023-10-16T11:25:51.295096+0000","last_became_active":"2023-10-16T11:25:37.172964+0000","last_became_peered":"2023-10-16T11:25:37.172964+0000","last_unstale":"2023-10-16T11:25:51.295096+0000","last_undegraded":"2023-10-16T11:25:51.295096+0000","last_fullsized":"2023-10-16T11:25:51.295096+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:53.461764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.302773+0000","last_change":"2023-10-16T11:25:36.166442+0000","last_active":"2023-10-16T11:25:51.302773+0000","last_peered":"2023-10-16T11:25:51.302773+0000","last_clean":"2023-10-16T11:25:51.302773+0000","last_became_active":"2023-10-16T11:25:36.166131+0000","last_became_peered":"2023-10-16T11:25:36.166131+0000","last_unstale":"2023-10-16T11:25:51.302773+0000","last_undegraded":"2023-10-16T11:25:51.302773+0000","last_fullsized":"2023-10-16T11:25:51.302773+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:43.814368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3f","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.319150+0000","last_change":"2023-10-16T11:25:51.320357+0000","last_active":"2023-10-16T11:25:52.319150+0000","last_peered":"2023-10-16T11:25:52.319150+0000","last_clean":"2023-10-16T11:25:52.319150+0000","last_became_active":"2023-10-16T11:25:51.320053+0000","last_became_peered":"2023-10-16T11:25:51.320053+0000","last_unstale":"2023-10-16T11:25:52.319150+0000","last_undegraded":"2023-10-16T11:25:52.319150+0000","last_fullsized":"2023-10-16T11:25:52.319150+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:45.962774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.310334+0000","last_change":"2023-10-16T11:25:37.183491+0000","last_active":"2023-10-16T11:25:51.310334+0000","last_peered":"2023-10-16T11:25:51.310334+0000","last_clean":"2023-10-16T11:25:51.310334+0000","last_became_active":"2023-10-16T11:25:37.179560+0000","last_became_peered":"2023-10-16T11:25:37.179560+0000","last_unstale":"2023-10-16T11:25:51.310334+0000","last_undegraded":"2023-10-16T11:25:51.310334+0000","last_fullsized":"2023-10-16T11:25:51.310334+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:31.820109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":16,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.304486+0000","last_change":"2023-10-16T11:25:36.175845+0000","last_active":"2023-10-16T11:25:51.304486+0000","last_peered":"2023-10-16T11:25:51.304486+0000","last_clean":"2023-10-16T11:25:51.304486+0000","last_became_active":"2023-10-16T11:25:36.175390+0000","last_became_peered":"2023-10-16T11:25:36.175390+0000","last_unstale":"2023-10-16T11:25:51.304486+0000","last_undegraded":"2023-10-16T11:25:51.304486+0000","last_fullsized":"2023-10-16T11:25:51.304486+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:09.232865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-16T11:25:52.788350+0000","last_change":"2023-10-16T11:25:51.343357+0000","last_active":"2023-10-16T11:25:52.788350+0000","last_peered":"2023-10-16T11:25:52.788350+0000","last_clean":"2023-10-16T11:25:52.788350+0000","last_became_active":"2023-10-16T11:25:51.343044+0000","last_became_peered":"2023-10-16T11:25:51.343044+0000","last_unstale":"2023-10-16T11:25:52.788350+0000","last_undegraded":"2023-10-16T11:25:52.788350+0000","last_fullsized":"2023-10-16T11:25:52.788350+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:50.260374+0000","last_clean_scrub_stamp":"2023-10-16T11:25:50.260374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:45.962774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":15,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.325051+0000","last_change":"2023-10-16T11:25:37.178486+0000","last_active":"2023-10-16T11:25:51.325051+0000","last_peered":"2023-10-16T11:25:51.325051+0000","last_clean":"2023-10-16T11:25:51.325051+0000","last_became_active":"2023-10-16T11:25:37.177077+0000","last_became_peered":"2023-10-16T11:25:37.177077+0000","last_unstale":"2023-10-16T11:25:51.325051+0000","last_undegraded":"2023-10-16T11:25:51.325051+0000","last_fullsized":"2023-10-16T11:25:51.325051+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:36.117709+0000","last_clean_scrub_stamp":"2023-10-16T11:25:36.117709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.009773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"236'4","reported_seq":20,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.307789+0000","last_change":"2023-10-16T11:25:36.174205+0000","last_active":"2023-10-16T11:25:51.307789+0000","last_peered":"2023-10-16T11:25:51.307789+0000","last_clean":"2023-10-16T11:25:51.307789+0000","last_became_active":"2023-10-16T11:25:36.173850+0000","last_became_peered":"2023-10-16T11:25:36.173850+0000","last_unstale":"2023-10-16T11:25:51.307789+0000","last_undegraded":"2023-10-16T11:25:51.307789+0000","last_fullsized":"2023-10-16T11:25:51.307789+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:25:35.105390+0000","last_clean_scrub_stamp":"2023-10-16T11:25:35.105390+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:21:20.618602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"223'2789","reported_seq":3749,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-16T11:25:51.309922+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:25:51.309922+0000","last_peered":"2023-10-16T11:25:51.309922+0000","last_clean":"2023-10-16T11:25:51.309922+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:25:51.309922+0000","last_undegraded":"2023-10-16T11:25:51.309922+0000","last_fullsized":"2023-10-16T11:25:51.309922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":2789,"log_dups_size":0,"ondisk_log_size":2789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":31,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":30,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2368,"num_read_kb":4880,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2789,"ondisk_log_size":2789,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576003,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52780,"kb_used_data":3552,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319060,"statfs":{"total":96636764160,"available":96582717440,"internally_reserved":0,"allocated":3637248,"data_stored":2074666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.1160000000000001,"15min":1.1970000000000001},"min":{"1min":0.70299999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4490000000000001,"5min":1.7250000000000001,"15min":2.3399999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.139,"15min":1.202},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.6479999999999999,"5min":1.6699999999999999,"15min":1.8320000000000001},"last":1.5}]},{"osd":1,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.21,"15min":1.2310000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.452,"5min":1.7589999999999999,"15min":1.913},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1970000000000001,"15min":1.2549999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.6799999999999999,"5min":1.726,"15min":1.8600000000000001},"last":1.3879999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2889999999999999,"15min":1.29},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":2.4460000000000002},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.3400000000000001,"15min":1.361},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":3.9249999999999998,"5min":3.9249999999999998,"15min":3.9249999999999998},"last":1.2450000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.359,"15min":1.3799999999999999},"min":{"1min":1.1419999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.399,"15min":1.4379999999999999},"min":{"1min":0.81399999999999995,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.802,"5min":2.0070000000000001,"15min":2.577},"last":1.5740000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.4359999999999999,"15min":1.4079999999999999},"min":{"1min":1.093,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.9610000000000001,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.373,"15min":1.383},"min":{"1min":0.91400000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.1579999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.5740000000000001,"15min":1.583},"min":{"1min":0.81499999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.966,"5min":2.101,"15min":3.3519999999999999},"last":1.45},{"interface":"front","average":{"1min":1.462,"5min":1.516,"15min":1.5009999999999999},"min":{"1min":0.97899999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.7569999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.361}]},{"osd":6,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.502,"15min":1.5029999999999999},"min":{"1min":1.018,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.7250000000000001},"last":1.109},{"interface":"front","average":{"1min":1.635,"5min":1.665,"15min":1.6879999999999999},"min":{"1min":0.76800000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1600000000000001,"5min":2.2109999999999999,"15min":3.1440000000000001},"last":1.665}]}]},{"osd":6,"up_from":13,"seq":55834576003,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52216,"kb_used_data":2980,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319624,"statfs":{"total":96636764160,"available":96583294976,"internally_reserved":0,"allocated":3051520,"data_stored":1484874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.2130000000000001,"15min":1.21},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.343,"5min":6.298,"15min":6.298},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.331,"15min":1.292},"min":{"1min":0.72199999999999998,"5min":0.68200000000000005,"15min":0.66400000000000003},"max":{"1min":1.607,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":1.323}]},{"osd":1,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.224,"15min":1.212},"min":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.5660000000000001,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":1.375},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.421,"15min":1.3779999999999999},"min":{"1min":0.76200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.7310000000000001,"5min":12.468,"15min":12.468},"last":1.1040000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.3140000000000001,"15min":1.2769999999999999},"min":{"1min":0.77800000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.373,"5min":11.586,"15min":11.586},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.3740000000000001,"15min":1.3680000000000001},"min":{"1min":0.84399999999999997,"5min":0.82599999999999996,"15min":0.77500000000000002},"max":{"1min":1.7629999999999999,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.486}]},{"osd":3,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.425,"15min":1.3460000000000001},"min":{"1min":0.84799999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.859,"5min":12.801,"15min":12.801},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2929999999999999,"15min":1.2769999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.657,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":1.2669999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3899999999999999,"15min":1.3280000000000001},"min":{"1min":0.68700000000000006,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.9510000000000001,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.6220000000000001,"15min":1.5509999999999999},"min":{"1min":1.1180000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.069,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.55}]},{"osd":5,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.5189999999999999,"15min":1.397},"min":{"1min":0.92500000000000004,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.978,"5min":18.247,"15min":18.247},"last":1.528},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.595,"15min":1.5620000000000001},"min":{"1min":0.93799999999999994,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.1240000000000001,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.6719999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:25:39 2023","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.627,"15min":1.5469999999999999},"min":{"1min":1.2070000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.383,"5min":12.589,"15min":12.589},"last":1.762},{"interface":"front","average":{"1min":1.661,"5min":1.702,"15min":1.603},"min":{"1min":1.0049999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.2970000000000002,"5min":12.725,"15min":12.725},"last":1.6180000000000001}]}]},{"osd":5,"up_from":13,"seq":55834576003,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29984,"kb_used_data":2964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341856,"statfs":{"total":96636764160,"available":96606060544,"internally_reserved":0,"allocated":3035136,"data_stored":1484158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.2110000000000001,"15min":1.2529999999999999},"min":{"1min":0.749,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.3839999999999999,"5min":2.661,"15min":2.661},"last":1.585},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2749999999999999,"15min":1.327},"min":{"1min":0.76400000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.675,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.49}]},{"osd":1,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.1319999999999999,"15min":1.1779999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.992},"last":1.748},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1970000000000001,"15min":1.22},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.669,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.4159999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.248,"15min":1.228},"min":{"1min":0.73299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.526,"5min":1.952,"15min":1.952},"last":1.121},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.2929999999999999,"15min":1.3109999999999999},"min":{"1min":0.83599999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":2.1640000000000001},"last":1.343}]},{"osd":3,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.302,"15min":1.3220000000000001},"min":{"1min":0.97999999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.6830000000000001,"5min":3.012,"15min":3.012},"last":1.272},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.395,"15min":1.409},"min":{"1min":0.93400000000000005,"5min":0.92700000000000005,"15min":0.89500000000000002},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.2250000000000001},"last":1.627}]},{"osd":4,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3260000000000001,"15min":1.333},"min":{"1min":0.96099999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.575,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.8360000000000001},{"interface":"front","average":{"1min":1.494,"5min":1.544,"15min":1.5609999999999999},"min":{"1min":1.048,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.8009999999999999,"5min":2.903,"15min":2.903},"last":1.667}]},{"osd":6,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.3999999999999999,"15min":1.405},"min":{"1min":1.097,"5min":0.68100000000000005,"15min":0.51000000000000001},"max":{"1min":1.6519999999999999,"5min":1.8660000000000001,"15min":2.0979999999999999},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.458,"15min":1.474},"min":{"1min":0.95999999999999996,"5min":0.90700000000000003,"15min":0.89200000000000002},"max":{"1min":1.758,"5min":3.073,"15min":3.073},"last":1.7110000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.536,"15min":1.538},"min":{"1min":1.0549999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.1150000000000002,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.7789999999999999},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.615,"15min":1.637},"min":{"1min":1.3400000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.2930000000000001},"last":1.8080000000000001}]}]},{"osd":4,"up_from":13,"seq":55834576004,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70224,"kb_used_data":3004,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301616,"statfs":{"total":96636764160,"available":96564854784,"internally_reserved":0,"allocated":3076096,"data_stored":1488775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.1870000000000001,"15min":1.1839999999999999},"min":{"1min":0.80800000000000005,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.8049999999999999,"5min":2.899,"15min":2.899},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2170000000000001,"15min":1.2450000000000001},"min":{"1min":0.88400000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.7929999999999999,"5min":3.016,"15min":3.016},"last":1.22}]},{"osd":1,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3049999999999999,"15min":1.298},"min":{"1min":0.84999999999999998,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.887,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.77100000000000002,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.617,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.2769999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.327,"15min":1.3140000000000001},"min":{"1min":1.075,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.169,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.458},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4350000000000001,"15min":1.4159999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9870000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.54}]},{"osd":3,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.3640000000000001,"15min":1.351},"min":{"1min":0.96799999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.0390000000000001,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.147},{"interface":"front","average":{"1min":1.645,"5min":1.4970000000000001,"15min":1.4610000000000001},"min":{"1min":1.216,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.3109999999999999,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.3640000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.5289999999999999,"15min":1.464},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2309999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.415},{"interface":"front","average":{"1min":1.504,"5min":1.4810000000000001,"15min":1.4350000000000001},"min":{"1min":0.89500000000000002,"5min":0.85199999999999998,"15min":0.78600000000000003},"max":{"1min":2.0110000000000001,"5min":3.7989999999999999,"15min":3.7989999999999999},"last":1.49}]},{"osd":6,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.599,"15min":1.534},"min":{"1min":0.98499999999999999,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":2.6459999999999999,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.611},{"interface":"front","average":{"1min":1.752,"5min":1.665,"15min":1.6379999999999999},"min":{"1min":0.77100000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.3599999999999999,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.7709999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.7629999999999999,"5min":1.649,"15min":1.6000000000000001},"min":{"1min":0.74099999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.3570000000000002,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":1.8029999999999999},{"interface":"front","average":{"1min":1.7849999999999999,"5min":1.7270000000000001,"15min":1.675},"min":{"1min":0.98699999999999999,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.464,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.7250000000000001}]}]},{"osd":3,"up_from":13,"seq":55834576003,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52212,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319628,"statfs":{"total":96636764160,"available":96583299072,"internally_reserved":0,"allocated":3055616,"data_stored":1486934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.85899999999999999,"15min":0.89700000000000002},"min":{"1min":0.40000000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1339999999999999,"5min":1.478,"15min":1.901},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.997,"15min":1.0569999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1850000000000001,"5min":1.6180000000000001,"15min":1.6679999999999999},"last":1.0649999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.95299999999999996,"15min":0.98399999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.3400000000000001,"5min":1.508,"15min":1.508},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0609999999999999,"15min":1.121},"min":{"1min":0.64900000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5429999999999999,"5min":1.601,"15min":1.7190000000000001},"last":1.024}]},{"osd":2,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0580000000000001,"15min":1.1120000000000001},"min":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4199999999999999,"5min":1.694,"15min":1.694},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.161,"15min":1.22},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.7350000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.145}]},{"osd":4,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2989999999999999,"15min":1.3480000000000001},"min":{"1min":0.89400000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.724,"5min":1.806,"15min":1.873},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3300000000000001,"15min":1.359},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.4970000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.389,"15min":1.405},"min":{"1min":1.0549999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8260000000000001,"5min":2.798,"15min":2.798},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.395,"15min":1.4410000000000001},"min":{"1min":0.88800000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.883,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.7909999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.488,"15min":1.5329999999999999},"min":{"1min":1.01,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.911,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.397},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4790000000000001,"15min":1.534},"min":{"1min":0.86699999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.0329999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.587}]},{"osd":7,"last update":"Mon Oct 16 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.518,"15min":1.5580000000000001},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.827},{"interface":"front","average":{"1min":1.575,"5min":1.613,"15min":1.669},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.3799999999999999,"5min":2.3799999999999999,"15min":2.5129999999999999},"last":1.9059999999999999}]}]},{"osd":2,"up_from":13,"seq":55834576002,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56356,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315484,"statfs":{"total":96636764160,"available":96579055616,"internally_reserved":0,"allocated":3039232,"data_stored":1484248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.89200000000000002,"15min":0.91600000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.349,"5min":1.532,"15min":1.669},"last":1.133},{"interface":"front","average":{"1min":0.88,"5min":0.99299999999999999,"15min":1.069},"min":{"1min":0.56899999999999995,"5min":0.47699999999999998,"15min":0.32600000000000001},"max":{"1min":1.3460000000000001,"5min":1.6359999999999999,"15min":2.1429999999999998},"last":0.88800000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0760000000000001,"15min":1.089},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.39200000000000002},"max":{"1min":1.591,"5min":1.645,"15min":1.978},"last":1.4750000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.085,"15min":1.1379999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.393,"5min":1.609,"15min":1.7949999999999999},"last":1.4179999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0429999999999999,"15min":1.0740000000000001},"min":{"1min":0.499,"5min":0.437,"15min":0.437},"max":{"1min":1.522,"5min":1.651,"15min":1.8560000000000001},"last":0.995},{"interface":"front","average":{"1min":1.137,"5min":1.2170000000000001,"15min":1.2629999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.6930000000000001,"5min":1.7450000000000001,"15min":1.857},"last":1.5880000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3340000000000001,"15min":1.3999999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.73499999999999999},"max":{"1min":1.643,"5min":1.8029999999999999,"15min":2.5059999999999998},"last":1.504},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.3220000000000001,"15min":1.339},"min":{"1min":0.88200000000000001,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":1.6499999999999999,"5min":1.825,"15min":1.843},"last":1.2290000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.4139999999999999,"15min":1.4530000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.8620000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.429,"15min":1.4730000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.885,"5min":1.931,"15min":2.6259999999999999},"last":1.6339999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3899999999999999,"15min":1.3999999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.8260000000000001,"5min":1.9019999999999999,"15min":2.1339999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4870000000000001,"15min":1.526},"min":{"1min":0.80500000000000005,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.7929999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:25:08 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.5109999999999999,"15min":1.569},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.92500000000000004},"max":{"1min":2.0310000000000001,"5min":2.1469999999999998,"15min":2.8109999999999999},"last":1.6919999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4750000000000001,"15min":1.502},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80100000000000005},"max":{"1min":1.952,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.3720000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576002,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64708,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307132,"statfs":{"total":96636764160,"available":96570503168,"internally_reserved":0,"allocated":3072000,"data_stored":1487915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.90900000000000003,"15min":0.91300000000000003},"min":{"1min":0.39400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.3520000000000001,"5min":5.117,"15min":5.117},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.93500000000000005,"15min":1.0289999999999999},"min":{"1min":0.35899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.262,"5min":1.5509999999999999,"15min":1.667},"last":0.85899999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.93799999999999994,"15min":1.0029999999999999},"min":{"1min":0.497,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3720000000000001,"5min":2.093,"15min":2.093},"last":1.123},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1879999999999999,"15min":1.2589999999999999},"min":{"1min":0.51700000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.456,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":1.333}]},{"osd":3,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.139,"15min":1.1890000000000001},"min":{"1min":0.53800000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.6080000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.103,"15min":1.153},"min":{"1min":0.55800000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.798,"5min":1.798,"15min":1.802},"last":1.294}]},{"osd":4,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.3049999999999999,"15min":1.2949999999999999},"min":{"1min":0.69299999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.978,"5min":5.29,"15min":5.29},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.327,"5min":1.4159999999999999,"15min":1.421},"min":{"1min":0.82999999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.891,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":1.4430000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.373,"15min":1.409},"min":{"1min":0.83799999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.9430000000000001,"5min":6.0979999999999999,"15min":6.0979999999999999},"last":1.829},{"interface":"front","average":{"1min":1.254,"5min":1.383,"15min":1.403},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.859,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.7410000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4570000000000001,"15min":1.4890000000000001},"min":{"1min":0.92200000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.923,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":1.7},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4419999999999999,"15min":1.4810000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.1419999999999999,"5min":3.8799999999999999,"15min":3.8799999999999999},"last":1.613}]},{"osd":7,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.546,"15min":1.5609999999999999},"min":{"1min":1.0409999999999999,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":2.0219999999999998,"5min":4.46,"15min":4.46},"last":1.774},{"interface":"front","average":{"1min":1.512,"5min":1.597,"15min":1.6040000000000001},"min":{"1min":0.99099999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.097,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":1.897}]}]},{"osd":0,"up_from":13,"seq":55834576003,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56944,"kb_used_data":3548,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314896,"statfs":{"total":96636764160,"available":96578453504,"internally_reserved":0,"allocated":3633152,"data_stored":2074560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.86099999999999999,"15min":0.88700000000000001},"min":{"1min":0.46800000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2210000000000001,"5min":1.52,"15min":1.619},"last":1.157},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.006,"15min":1.081},"min":{"1min":0.41199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.341,"5min":1.621,"15min":1.76},"last":0.86099999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.075,"15min":1.0800000000000001},"min":{"1min":0.52500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.7349999999999999,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":1.339},{"interface":"front","average":{"1min":1.034,"5min":1.0700000000000001,"15min":1.1299999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":1.5}]},{"osd":3,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.016,"15min":1.0669999999999999},"min":{"1min":0.50600000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1299999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.226},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.1539999999999999,"15min":1.226},"min":{"1min":0.55600000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.3799999999999999,"5min":1.833,"15min":1.9830000000000001},"last":0.97399999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.349,"15min":1.3859999999999999},"min":{"1min":0.78800000000000003,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":3.2799999999999998,"5min":3.2799999999999998,"15min":3.2799999999999998},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.222,"5min":1.292,"15min":1.337},"min":{"1min":0.76100000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.5649999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.357,"15min":1.3720000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0209999999999999,"5min":2.0750000000000002,"15min":2.6669999999999998},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3819999999999999,"15min":1.431},"min":{"1min":0.89800000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.885,"5min":2.2029999999999998,"15min":2.2040000000000002},"last":1.6479999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.4710000000000001,"15min":1.496},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":5.2560000000000002,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":1.448},{"interface":"front","average":{"1min":1.298,"5min":1.429,"15min":1.4790000000000001},"min":{"1min":0.83199999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.804,"5min":2.073,"15min":2.262},"last":1.6080000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.704,"5min":1.5429999999999999,"15min":1.5600000000000001},"min":{"1min":0.88700000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.4419999999999999,"15min":1.492},"min":{"1min":0.871,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.623,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.6819999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:26:01.473 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T11:26:01.473 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:26:01.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.727+0000 7f5ad0fa7640 1 Processor -- start 2023-10-16T11:26:01.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.727+0000 7f5ad0fa7640 1 -- start start 2023-10-16T11:26:01.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.729+0000 7f5ad0fa7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5acc0ad970 0x7f5acc0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:01.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.729+0000 7f5ad0fa7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5acc0ccfb0 0x7f5acc0cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:01.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.729+0000 7f5ad0fa7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5acc15b9d0 con 0x7f5acc06c040 2023-10-16T11:26:01.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.729+0000 7f5ad0fa7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5acc15bcb0 con 0x7f5acc0ccfb0 2023-10-16T11:26:01.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.729+0000 7f5ad0fa7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5acc15bf90 con 0x7f5acc0ad970 2023-10-16T11:26:01.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.729+0000 7f5acbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5acc0ccfb0 0x7f5acc0cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:01.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.729+0000 7f5acbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5acc0ccfb0 0x7f5acc0cd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40564/0 (socket says 172.21.15.84:40564) 2023-10-16T11:26:01.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.729+0000 7f5acbfff640 1 -- 172.21.15.84:0/2730035530 learned_addr learned my addr 172.21.15.84:0/2730035530 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:01.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.730+0000 7f5acbfff640 1 -- 172.21.15.84:0/2730035530 >> v2:172.21.15.157:3300/0 conn(0x7f5acc0ad970 msgr2=0x7f5acc0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:01.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.730+0000 7f5acbfff640 1 --2- 172.21.15.84:0/2730035530 >> v2:172.21.15.157:3300/0 conn(0x7f5acc0ad970 0x7f5acc0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.730+0000 7f5acbfff640 1 -- 172.21.15.84:0/2730035530 >> v1:172.21.15.157:6789/0 conn(0x7f5acc06c040 legacy=0x7f5acc06c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:01.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.730+0000 7f5acbfff640 1 -- 172.21.15.84:0/2730035530 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc0cc270 con 0x7f5acc0ccfb0 2023-10-16T11:26:01.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.731+0000 7f5acbfff640 1 --2- 172.21.15.84:0/2730035530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5acc0ccfb0 0x7f5acc0cd460 secure :-1 s=READY pgs=3007 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009e50 tx=0x7f5ac402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c038a1758b012220 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:01.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.731+0000 7f5acaffd640 1 -- 172.21.15.84:0/2730035530 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac402ca80 con 0x7f5acc0ccfb0 2023-10-16T11:26:01.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.731+0000 7f5acaffd640 1 -- 172.21.15.84:0/2730035530 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac402cbe0 con 0x7f5acc0ccfb0 2023-10-16T11:26:01.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.731+0000 7f5acaffd640 1 -- 172.21.15.84:0/2730035530 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac40343f0 con 0x7f5acc0ccfb0 2023-10-16T11:26:01.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.732+0000 7f5ad0fa7640 1 -- 172.21.15.84:0/2730035530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5acc0ccfb0 msgr2=0x7f5acc0cd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:01.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.732+0000 7f5ad0fa7640 1 --2- 172.21.15.84:0/2730035530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5acc0ccfb0 0x7f5acc0cd460 secure :-1 s=READY pgs=3007 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009e50 tx=0x7f5ac402bee0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.732+0000 7f5ad0fa7640 1 -- 172.21.15.84:0/2730035530 shutdown_connections 2023-10-16T11:26:01.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.732+0000 7f5ad0fa7640 1 --2- 172.21.15.84:0/2730035530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5acc0ccfb0 0x7f5acc0cd460 unknown :-1 s=CLOSED pgs=3007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.732+0000 7f5ad0fa7640 1 --2- 172.21.15.84:0/2730035530 >> v2:172.21.15.157:3300/0 conn(0x7f5acc0ad970 0x7f5acc0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.732+0000 7f5ad0fa7640 1 -- 172.21.15.84:0/2730035530 >> 172.21.15.84:0/2730035530 conn(0x7f5acc0ce6a0 msgr2=0x7f5acc0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:01.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.732+0000 7f5ad0fa7640 1 -- 172.21.15.84:0/2730035530 shutdown_connections 2023-10-16T11:26:01.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.732+0000 7f5ad0fa7640 1 -- 172.21.15.84:0/2730035530 wait complete. 2023-10-16T11:26:01.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.734+0000 7f5ad0fa7640 1 Processor -- start 2023-10-16T11:26:01.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.734+0000 7f5ad0fa7640 1 -- start start 2023-10-16T11:26:01.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.735+0000 7f5ad0fa7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5acc06c040 0x7f5acc0ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:01.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.735+0000 7f5ad0fa7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5acc0ad970 0x7f5acc1f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:01.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.735+0000 7f5ad0fa7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5acc0ccfb0 0x7f5acc0b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:01.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.735+0000 7f5ad0fa7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc1f21a0 con 0x7f5acc0ccfb0 2023-10-16T11:26:01.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.735+0000 7f5ad0fa7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0b3440 con 0x7f5acc06c040 2023-10-16T11:26:01.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.735+0000 7f5ad0fa7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0b0fe0 con 0x7f5acc0ad970 2023-10-16T11:26:01.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.735+0000 7f5acbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5acc0ccfb0 0x7f5acc0b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:01.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.735+0000 7f5acbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5acc0ccfb0 0x7f5acc0b0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40578/0 (socket says 172.21.15.84:40578) 2023-10-16T11:26:01.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.735+0000 7f5acbfff640 1 -- 172.21.15.84:0/4113415742 learned_addr learned my addr 172.21.15.84:0/4113415742 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:01.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.735+0000 7f5acb7fe640 1 --2- 172.21.15.84:0/4113415742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5acc06c040 0x7f5acc0ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:01.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.735+0000 7f5ac3fff640 1 --2- 172.21.15.84:0/4113415742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5acc0ad970 0x7f5acc1f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:01.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.736+0000 7f5acbfff640 1 -- 172.21.15.84:0/4113415742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5acc0ad970 msgr2=0x7f5acc1f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:01.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.736+0000 7f5acbfff640 1 --2- 172.21.15.84:0/4113415742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5acc0ad970 0x7f5acc1f1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.736+0000 7f5acbfff640 1 -- 172.21.15.84:0/4113415742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5acc06c040 msgr2=0x7f5acc0ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:01.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.736+0000 7f5acbfff640 1 --2- 172.21.15.84:0/4113415742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5acc06c040 0x7f5acc0ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.736+0000 7f5acbfff640 1 -- 172.21.15.84:0/4113415742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac4009b30 con 0x7f5acc0ccfb0 2023-10-16T11:26:01.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.736+0000 7f5acbfff640 1 --2- 172.21.15.84:0/4113415742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5acc0ccfb0 0x7f5acc0b0aa0 secure :-1 s=READY pgs=3008 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4036040 tx=0x7f5ac4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:01.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.736+0000 7f5ac8ff9640 1 -- 172.21.15.84:0/4113415742 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4040670 con 0x7f5acc0ccfb0 2023-10-16T11:26:01.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.736+0000 7f5ac8ff9640 1 -- 172.21.15.84:0/4113415742 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac40407f0 con 0x7f5acc0ccfb0 2023-10-16T11:26:01.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.736+0000 7f5ac8ff9640 1 -- 172.21.15.84:0/4113415742 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4040ba0 con 0x7f5acc0ccfb0 2023-10-16T11:26:01.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.736+0000 7f5ad0fa7640 1 -- 172.21.15.84:0/4113415742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5acc0b1430 con 0x7f5acc0ccfb0 2023-10-16T11:26:01.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.737+0000 7f5ad0fa7640 1 -- 172.21.15.84:0/4113415742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5acc0b30d0 con 0x7f5acc0ccfb0 2023-10-16T11:26:01.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.738+0000 7f5ad0fa7640 1 -- 172.21.15.84:0/4113415742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5acc06b330 con 0x7f5acc0ccfb0 2023-10-16T11:26:01.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.739+0000 7f5ac8ff9640 1 -- 172.21.15.84:0/4113415742 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5ac402cd50 con 0x7f5acc0ccfb0 2023-10-16T11:26:01.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.740+0000 7f5ac8ff9640 1 --2- 172.21.15.84:0/4113415742 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f5aac06b0e0 0x7f5aac06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:01.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.740+0000 7f5ac8ff9640 1 -- 172.21.15.84:0/4113415742 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f5ac40acc80 con 0x7f5acc0ccfb0 2023-10-16T11:26:01.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.740+0000 7f5acb7fe640 1 --2- 172.21.15.84:0/4113415742 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f5aac06b0e0 0x7f5aac06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:01.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.741+0000 7f5acb7fe640 1 --2- 172.21.15.84:0/4113415742 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f5aac06b0e0 0x7f5aac06d5a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5abc004ea0 tx=0x7f5abc009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:01.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.744+0000 7f5ac8ff9640 1 -- 172.21.15.84:0/4113415742 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5ac4077650 con 0x7f5acc0ccfb0 2023-10-16T11:26:01.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.975+0000 7f5ad0fa7640 1 -- 172.21.15.84:0/4113415742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5acc0b3260 con 0x7f5acc0ccfb0 2023-10-16T11:26:01.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.977+0000 7f5ac8ff9640 1 -- 172.21.15.84:0/4113415742 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+6155 (secure 0 0 0) 0x7f5ac407ac70 con 0x7f5acc0ccfb0 2023-10-16T11:26:01.979 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:01.979 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":137,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2516077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2516077743},{"type":"v1","addr":"172.21.15.157:6833","nonce":2516077743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1527104462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1527104462},{"type":"v1","addr":"172.21.15.157:6835","nonce":1527104462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:25:36.605068+0000","modified":"2023-10-16T11:25:39.675208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12186":{"gid":12186,"name":"a","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/3547503033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3547503033},{"type":"v1","addr":"172.21.15.84:6837","nonce":3547503033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:25:50.739234+0000","modified":"2023-10-16T11:25:53.792071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12183":{"gid":12183,"name":"c","rank":0,"incarnation":133,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/369067539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":369067539},{"type":"v1","addr":"172.21.15.84:6835","nonce":369067539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-16T11:26:01.979 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 137 2023-10-16T11:26:01.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.979+0000 7f5ad0fa7640 1 -- 172.21.15.84:0/4113415742 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f5aac06b0e0 msgr2=0x7f5aac06d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:01.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.979+0000 7f5ad0fa7640 1 --2- 172.21.15.84:0/4113415742 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f5aac06b0e0 0x7f5aac06d5a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5abc004ea0 tx=0x7f5abc009040 comp rx=0 tx=0).stop 2023-10-16T11:26:01.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.979+0000 7f5ad0fa7640 1 -- 172.21.15.84:0/4113415742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5acc0ccfb0 msgr2=0x7f5acc0b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:01.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.979+0000 7f5ad0fa7640 1 --2- 172.21.15.84:0/4113415742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5acc0ccfb0 0x7f5acc0b0aa0 secure :-1 s=READY pgs=3008 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4036040 tx=0x7f5ac4004a40 comp rx=0 tx=0).stop 2023-10-16T11:26:01.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.980+0000 7f5ad0fa7640 1 -- 172.21.15.84:0/4113415742 shutdown_connections 2023-10-16T11:26:01.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.980+0000 7f5ad0fa7640 1 --2- 172.21.15.84:0/4113415742 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f5aac06b0e0 0x7f5aac06d5a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.980+0000 7f5ad0fa7640 1 --2- 172.21.15.84:0/4113415742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5acc0ccfb0 0x7f5acc0b0aa0 unknown :-1 s=CLOSED pgs=3008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.980+0000 7f5ad0fa7640 1 --2- 172.21.15.84:0/4113415742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5acc0ad970 0x7f5acc1f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.980+0000 7f5ad0fa7640 1 --2- 172.21.15.84:0/4113415742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5acc06c040 0x7f5acc0ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:01.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.980+0000 7f5ad0fa7640 1 -- 172.21.15.84:0/4113415742 >> 172.21.15.84:0/4113415742 conn(0x7f5acc0ce6a0 msgr2=0x7f5acc06f700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:01.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.980+0000 7f5ad0fa7640 1 -- 172.21.15.84:0/4113415742 shutdown_connections 2023-10-16T11:26:01.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:01.980+0000 7f5ad0fa7640 1 -- 172.21.15.84:0/4113415742 wait complete. 2023-10-16T11:26:02.402 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 137, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T11:25:50.739234+0000', 'modified': '2023-10-16T11:25:53.792071+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12183}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12183': {'gid': 12183, 'name': 'c', 'rank': 0, 'incarnation': 133, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.84:6835/369067539', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6834', 'nonce': 369067539}, {'type': 'v1', 'addr': '172.21.15.84:6835', 'nonce': 369067539}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [33], 'metadata_pool': 32, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T11:26:02.402 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T11:26:02.403 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-16T11:26:02.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.626+0000 7fcc52acf640 1 Processor -- start 2023-10-16T11:26:02.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.626+0000 7fcc52acf640 1 -- start start 2023-10-16T11:26:02.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.627+0000 7fcc52acf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcc4c15ad70 0x7fcc4c069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.627+0000 7fcc52acf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc4c0c9ca0 0x7fcc4c06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.627+0000 7fcc52acf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc4c0cb030 con 0x7fcc4c0ca3b0 2023-10-16T11:26:02.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.627+0000 7fcc52acf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc4c0c91c0 con 0x7fcc4c15ad70 2023-10-16T11:26:02.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.627+0000 7fcc52acf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0c94a0 con 0x7fcc4c0c9ca0 2023-10-16T11:26:02.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.628+0000 7fcc4bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc4c0c9ca0 0x7fcc4c06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.628+0000 7fcc5086b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcc4c15ad70 0x7fcc4c069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.628+0000 7fcc5106c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcc4c0ca3b0 0x7fcc4c0ca840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:37194/0 (socket says 172.21.15.84:37194) 2023-10-16T11:26:02.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.628+0000 7fcc5106c640 1 -- 172.21.15.84:0/1630205685 learned_addr learned my addr 172.21.15.84:0/1630205685 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:02.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.629+0000 7fcc4bfff640 1 -- 172.21.15.84:0/1630205685 >> v2:172.21.15.157:3300/0 conn(0x7fcc4c15ad70 msgr2=0x7fcc4c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:02.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.629+0000 7fcc4bfff640 1 --2- 172.21.15.84:0/1630205685 >> v2:172.21.15.157:3300/0 conn(0x7fcc4c15ad70 0x7fcc4c069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.629+0000 7fcc4bfff640 1 -- 172.21.15.84:0/1630205685 >> v1:172.21.15.157:6789/0 conn(0x7fcc4c0ca3b0 legacy=0x7fcc4c0ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:02.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.629+0000 7fcc4bfff640 1 -- 172.21.15.84:0/1630205685 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c0c9780 con 0x7fcc4c0c9ca0 2023-10-16T11:26:02.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.630+0000 7fcc4bfff640 1 --2- 172.21.15.84:0/1630205685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc4c0c9ca0 0x7fcc4c06a2e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009d90 tx=0x7fcc3402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cab37cbe70dafa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:02.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.631+0000 7fcc4b7fe640 1 -- 172.21.15.84:0/1630205685 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3402cc50 con 0x7fcc4c0c9ca0 2023-10-16T11:26:02.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.631+0000 7fcc4b7fe640 1 -- 172.21.15.84:0/1630205685 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3402cdb0 con 0x7fcc4c0c9ca0 2023-10-16T11:26:02.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.631+0000 7fcc4b7fe640 1 -- 172.21.15.84:0/1630205685 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc34034660 con 0x7fcc4c0c9ca0 2023-10-16T11:26:02.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.632+0000 7fcc52acf640 1 -- 172.21.15.84:0/1630205685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc4c0c9ca0 msgr2=0x7fcc4c06a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:02.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.632+0000 7fcc52acf640 1 --2- 172.21.15.84:0/1630205685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc4c0c9ca0 0x7fcc4c06a2e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009d90 tx=0x7fcc3402bea0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.632+0000 7fcc52acf640 1 -- 172.21.15.84:0/1630205685 shutdown_connections 2023-10-16T11:26:02.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.632+0000 7fcc52acf640 1 --2- 172.21.15.84:0/1630205685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc4c0c9ca0 0x7fcc4c06a2e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.632+0000 7fcc52acf640 1 --2- 172.21.15.84:0/1630205685 >> v2:172.21.15.157:3300/0 conn(0x7fcc4c15ad70 0x7fcc4c069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.632+0000 7fcc52acf640 1 -- 172.21.15.84:0/1630205685 >> 172.21.15.84:0/1630205685 conn(0x7fcc4c0ce9b0 msgr2=0x7fcc4c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:02.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.632+0000 7fcc52acf640 1 -- 172.21.15.84:0/1630205685 shutdown_connections 2023-10-16T11:26:02.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.632+0000 7fcc52acf640 1 -- 172.21.15.84:0/1630205685 wait complete. 2023-10-16T11:26:02.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.634+0000 7fcc52acf640 1 Processor -- start 2023-10-16T11:26:02.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.634+0000 7fcc52acf640 1 -- start start 2023-10-16T11:26:02.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.635+0000 7fcc52acf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc4c0c9ca0 0x7fcc4c1596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.635+0000 7fcc52acf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc4c0ca3b0 0x7fcc4c0b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.635+0000 7fcc52acf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc4c15ad70 0x7fcc4c0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.635+0000 7fcc52acf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0c67c0 con 0x7fcc4c15ad70 2023-10-16T11:26:02.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.635+0000 7fcc52acf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0c5bb0 con 0x7fcc4c0c9ca0 2023-10-16T11:26:02.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.635+0000 7fcc52acf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0c5e90 con 0x7fcc4c0ca3b0 2023-10-16T11:26:02.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.635+0000 7fcc5106c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc4c15ad70 0x7fcc4c0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.635+0000 7fcc5106c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc4c15ad70 0x7fcc4c0b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40580/0 (socket says 172.21.15.84:40580) 2023-10-16T11:26:02.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.635+0000 7fcc5106c640 1 -- 172.21.15.84:0/649713937 learned_addr learned my addr 172.21.15.84:0/649713937 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:02.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.636+0000 7fcc4bfff640 1 --2- 172.21.15.84:0/649713937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc4c0ca3b0 0x7fcc4c0b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.636+0000 7fcc5086b640 1 --2- 172.21.15.84:0/649713937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc4c0c9ca0 0x7fcc4c1596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.636+0000 7fcc5106c640 1 -- 172.21.15.84:0/649713937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc4c0ca3b0 msgr2=0x7fcc4c0b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:02.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.636+0000 7fcc5106c640 1 --2- 172.21.15.84:0/649713937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc4c0ca3b0 0x7fcc4c0b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.636+0000 7fcc5106c640 1 -- 172.21.15.84:0/649713937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc4c0c9ca0 msgr2=0x7fcc4c1596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:02.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.636+0000 7fcc5106c640 1 --2- 172.21.15.84:0/649713937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc4c0c9ca0 0x7fcc4c1596f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.636+0000 7fcc5106c640 1 -- 172.21.15.84:0/649713937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc34009a70 con 0x7fcc4c15ad70 2023-10-16T11:26:02.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.636+0000 7fcc5106c640 1 --2- 172.21.15.84:0/649713937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc4c15ad70 0x7fcc4c0b2b90 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400c420 tx=0x7fcc4400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:02.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.636+0000 7fcc497fa640 1 -- 172.21.15.84:0/649713937 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc44005ce0 con 0x7fcc4c15ad70 2023-10-16T11:26:02.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.636+0000 7fcc497fa640 1 -- 172.21.15.84:0/649713937 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc44005e40 con 0x7fcc4c15ad70 2023-10-16T11:26:02.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.636+0000 7fcc497fa640 1 -- 172.21.15.84:0/649713937 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440072d0 con 0x7fcc4c15ad70 2023-10-16T11:26:02.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.636+0000 7fcc52acf640 1 -- 172.21.15.84:0/649713937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc4c0c6170 con 0x7fcc4c15ad70 2023-10-16T11:26:02.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.637+0000 7fcc52acf640 1 -- 172.21.15.84:0/649713937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4c0a9120 con 0x7fcc4c15ad70 2023-10-16T11:26:02.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.639+0000 7fcc497fa640 1 -- 172.21.15.84:0/649713937 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fcc440074f0 con 0x7fcc4c15ad70 2023-10-16T11:26:02.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.640+0000 7fcc497fa640 1 --2- 172.21.15.84:0/649713937 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fcc2c06b0b0 0x7fcc2c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.640+0000 7fcc497fa640 1 -- 172.21.15.84:0/649713937 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7fcc440891c0 con 0x7fcc4c15ad70 2023-10-16T11:26:02.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.640+0000 7fcc5086b640 1 --2- 172.21.15.84:0/649713937 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fcc2c06b0b0 0x7fcc2c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.641+0000 7fcc55668740 1 Processor -- start 2023-10-16T11:26:02.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.641+0000 7fcc5086b640 1 --2- 172.21.15.84:0/649713937 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fcc2c06b0b0 0x7fcc2c06d570 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009a30 tx=0x7fcc3c008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:02.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.641+0000 7fcc55668740 1 -- start start 2023-10-16T11:26:02.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.642+0000 7fcc55668740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc55f80 0x55b20bc28e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.642+0000 7fcc55668740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc56360 0x55b20bc58d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.642+0000 7fcc55668740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc293c0 0x55b20bc59250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.642+0000 7fcc55668740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20bb497a0 con 0x55b20bc55f80 2023-10-16T11:26:02.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.642+0000 7fcc55668740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20bb9b740 con 0x55b20bc293c0 2023-10-16T11:26:02.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.642+0000 7fcc55668740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b20bb98e30 con 0x55b20bc56360 2023-10-16T11:26:02.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.642+0000 7fcc5086b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc55f80 0x55b20bc28e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.642+0000 7fcc5086b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc55f80 0x55b20bc28e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40592/0 (socket says 172.21.15.84:40592) 2023-10-16T11:26:02.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.642+0000 7fcc5086b640 1 -- 172.21.15.84:0/4294442950 learned_addr learned my addr 172.21.15.84:0/4294442950 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:02.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.642+0000 7fcc4bfff640 1 --2- 172.21.15.84:0/4294442950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc56360 0x55b20bc58d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.642+0000 7fcc5086b640 1 -- 172.21.15.84:0/4294442950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc56360 msgr2=0x55b20bc58d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:02.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.642+0000 7fcc5086b640 1 --2- 172.21.15.84:0/4294442950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc56360 0x55b20bc58d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.642+0000 7fcc5086b640 1 -- 172.21.15.84:0/4294442950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc293c0 msgr2=0x55b20bc59250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:02.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.642+0000 7fcc5106c640 1 --2- 172.21.15.84:0/4294442950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc293c0 0x55b20bc59250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.642+0000 7fcc5086b640 1 --2- 172.21.15.84:0/4294442950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc293c0 0x55b20bc59250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.642+0000 7fcc5086b640 1 -- 172.21.15.84:0/4294442950 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b20bb58910 con 0x55b20bc55f80 2023-10-16T11:26:02.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.643+0000 7fcc5086b640 1 --2- 172.21.15.84:0/4294442950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc55f80 0x55b20bc28e80 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c011cf0 tx=0x7fcc3c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0ee4f82f7d823 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:02.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.643+0000 7fcc427fc640 1 -- 172.21.15.84:0/4294442950 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c011d20 con 0x55b20bc55f80 2023-10-16T11:26:02.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.643+0000 7fcc427fc640 1 -- 172.21.15.84:0/4294442950 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3c011e80 con 0x55b20bc55f80 2023-10-16T11:26:02.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.643+0000 7fcc427fc640 1 -- 172.21.15.84:0/4294442950 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c021640 con 0x55b20bc55f80 2023-10-16T11:26:02.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.644+0000 7fcc55668740 1 -- 172.21.15.84:0/4294442950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc55f80 msgr2=0x55b20bc28e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:02.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.644+0000 7fcc55668740 1 --2- 172.21.15.84:0/4294442950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc55f80 0x55b20bc28e80 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c011cf0 tx=0x7fcc3c014000 comp rx=0 tx=0).stop 2023-10-16T11:26:02.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.644+0000 7fcc55668740 1 -- 172.21.15.84:0/4294442950 shutdown_connections 2023-10-16T11:26:02.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.644+0000 7fcc55668740 1 --2- 172.21.15.84:0/4294442950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc293c0 0x55b20bc59250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.644+0000 7fcc55668740 1 --2- 172.21.15.84:0/4294442950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc56360 0x55b20bc58d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.644+0000 7fcc55668740 1 --2- 172.21.15.84:0/4294442950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc55f80 0x55b20bc28e80 unknown :-1 s=CLOSED pgs=3010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.644+0000 7fcc55668740 1 -- 172.21.15.84:0/4294442950 >> 172.21.15.84:0/4294442950 conn(0x55b20bc2b190 msgr2=0x55b20bc73d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:02.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.644+0000 7fcc55668740 1 -- 172.21.15.84:0/4294442950 shutdown_connections 2023-10-16T11:26:02.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.644+0000 7fcc55668740 1 -- 172.21.15.84:0/4294442950 wait complete. 2023-10-16T11:26:02.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.645+0000 7fcc55668740 1 -- start start 2023-10-16T11:26:02.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.645+0000 7fcc55668740 1 Processor -- start 2023-10-16T11:26:02.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.646+0000 7fcc55668740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc55f80 0x55b20bc3ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.646+0000 7fcc55668740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc56360 0x55b20bc3f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.646+0000 7fcc55668740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc5c900 0x55b20bc3f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.646+0000 7fcc55668740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20bb94720 con 0x55b20bc5c900 2023-10-16T11:26:02.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.646+0000 7fcc55668740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20bb32970 con 0x55b20bc55f80 2023-10-16T11:26:02.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.646+0000 7fcc55668740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b20bb32610 con 0x55b20bc56360 2023-10-16T11:26:02.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.646+0000 7fcc55668740 10 client.0 _ll_register_callbacks cb 0x55b20bb549d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:26:02.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.646+0000 7fcc5106c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc5c900 0x55b20bc3f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.646+0000 7fcc5106c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc5c900 0x55b20bc3f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40598/0 (socket says 172.21.15.84:40598) 2023-10-16T11:26:02.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.646+0000 7fcc5106c640 1 -- 172.21.15.84:0/1703499601 learned_addr learned my addr 172.21.15.84:0/1703499601 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:02.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.646+0000 7fcc4bfff640 1 --2- 172.21.15.84:0/1703499601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc56360 0x55b20bc3f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.647+0000 7fcc5106c640 1 -- 172.21.15.84:0/1703499601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc56360 msgr2=0x55b20bc3f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:02.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.647+0000 7fcc5106c640 1 --2- 172.21.15.84:0/1703499601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc56360 0x55b20bc3f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.647+0000 7fcc5106c640 1 -- 172.21.15.84:0/1703499601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc55f80 msgr2=0x55b20bc3ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:02.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.647+0000 7fcc5106c640 1 --2- 172.21.15.84:0/1703499601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc55f80 0x55b20bc3ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.647+0000 7fcc5106c640 1 -- 172.21.15.84:0/1703499601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3c0119a0 con 0x55b20bc5c900 2023-10-16T11:26:02.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.647+0000 7fcc5106c640 1 --2- 172.21.15.84:0/1703499601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc5c900 0x55b20bc3f9a0 secure :-1 s=READY pgs=3011 cs=0 l=1 rev1=1 crypto rx=0x7fcc44053e90 tx=0x7fcc440573f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d995861e6900f1e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:02.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.647+0000 7fcc2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T11:26:02.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.647+0000 7fcc2b7fe640 1 -- 172.21.15.84:0/1703499601 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4406f170 con 0x55b20bc5c900 2023-10-16T11:26:02.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.647+0000 7fcc2b7fe640 1 -- 172.21.15.84:0/1703499601 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7fcc44087210 con 0x55b20bc5c900 2023-10-16T11:26:02.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.647+0000 7fcc55668740 1 -- 172.21.15.84:0/1703499601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b20bbbb160 con 0x55b20bc5c900 2023-10-16T11:26:02.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.647+0000 7fcc2b7fe640 1 -- 172.21.15.84:0/1703499601 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=137) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc440873d0 con 0x55b20bc5c900 2023-10-16T11:26:02.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.647+0000 7fcc55668740 10 client.12510 fetch_fsmap learned FSMap version 137 2023-10-16T11:26:02.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.647+0000 7fcc55668740 1 -- 172.21.15.84:0/1703499601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=137,monmap=2+}) v3 -- 0x55b20bb92da0 con 0x55b20bc5c900 2023-10-16T11:26:02.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.648+0000 7fcc2b7fe640 1 -- 172.21.15.84:0/1703499601 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b20bbbb160 con 0x55b20bc5c900 2023-10-16T11:26:02.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.648+0000 7fcc2b7fe640 1 -- 172.21.15.84:0/1703499601 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 137) v1 ==== 3891+0+0 (secure 0 0 0) 0x7fcc44053780 con 0x55b20bc5c900 2023-10-16T11:26:02.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.648+0000 7fcc55668740 10 client.12510 fetch_fsmap finished waiting for FSMap version 137 2023-10-16T11:26:02.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.648+0000 7fcc55668740 10 client.12510 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T11:26:02.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.648+0000 7fcc55668740 20 client.12510 populate_metadata read hostname 'smithi084' 2023-10-16T11:26:02.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.648+0000 7fcc55668740 1 --2- 172.21.15.84:0/1703499601 >> [v2:172.21.15.84:6834/369067539,v1:172.21.15.84:6835/369067539] conn(0x55b20bc57760 0x55b20bc86df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.648+0000 7fcc55668740 4 client.12510 mds_command: new command op to 12183 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T11:26:02.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.648+0000 7fcc55668740 1 -- 172.21.15.84:0/1703499601 --> [v2:172.21.15.84:6834/369067539,v1:172.21.15.84:6835/369067539] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b20bb66cc0 con 0x55b20bc57760 2023-10-16T11:26:02.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.648+0000 7fcc5086b640 1 --2- 172.21.15.84:0/1703499601 >> [v2:172.21.15.84:6834/369067539,v1:172.21.15.84:6835/369067539] conn(0x55b20bc57760 0x55b20bc86df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.649+0000 7fcc5086b640 1 --2- 172.21.15.84:0/1703499601 >> [v2:172.21.15.84:6834/369067539,v1:172.21.15.84:6835/369067539] conn(0x55b20bc57760 0x55b20bc86df0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21a774b66b21846d server_cookie=875c456d58352ee5 in_seq=0 out_seq=0 2023-10-16T11:26:02.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.649+0000 7fcc2b7fe640 10 client.12510 ms_handle_connect on v2:172.21.15.84:6834/369067539 2023-10-16T11:26:02.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.649+0000 7fcc2b7fe640 1 -- 172.21.15.84:0/1703499601 <== mds.0 v2:172.21.15.84:6834/369067539 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b20bb66cc0 con 0x55b20bc57760 2023-10-16T11:26:02.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.649+0000 7fcc2b7fe640 10 client.12510 handle_command_reply: tid=0 2023-10-16T11:26:02.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.649+0000 7fcc55668740 1 client.12510 shutdown 2023-10-16T11:26:02.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.650+0000 7fcc55668740 1 -- 172.21.15.84:0/1703499601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc5c900 msgr2=0x55b20bc3f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:02.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.650+0000 7fcc55668740 1 --2- 172.21.15.84:0/1703499601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc5c900 0x55b20bc3f9a0 secure :-1 s=READY pgs=3011 cs=0 l=1 rev1=1 crypto rx=0x7fcc44053e90 tx=0x7fcc440573f0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.650+0000 7fcc55668740 1 -- 172.21.15.84:0/1703499601 shutdown_connections 2023-10-16T11:26:02.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.650+0000 7fcc55668740 1 --2- 172.21.15.84:0/1703499601 >> [v2:172.21.15.84:6834/369067539,v1:172.21.15.84:6835/369067539] conn(0x55b20bc57760 0x55b20bc86df0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.650+0000 7fcc55668740 1 --2- 172.21.15.84:0/1703499601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc5c900 0x55b20bc3f9a0 unknown :-1 s=CLOSED pgs=3011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.650+0000 7fcc55668740 1 --2- 172.21.15.84:0/1703499601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc56360 0x55b20bc3f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.650+0000 7fcc55668740 1 --2- 172.21.15.84:0/1703499601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc55f80 0x55b20bc3ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.650+0000 7fcc2b7fe640 0 client.12510 ms_handle_reset on v2:172.21.15.84:6834/369067539 2023-10-16T11:26:02.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.650+0000 7fcc55668740 1 -- 172.21.15.84:0/1703499601 >> 172.21.15.84:0/1703499601 conn(0x55b20bc293c0 msgr2=0x55b20bc77220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:02.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.650+0000 7fcc55668740 1 -- 172.21.15.84:0/1703499601 shutdown_connections 2023-10-16T11:26:02.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.650+0000 7fcc55668740 1 -- 172.21.15.84:0/1703499601 wait complete. 2023-10-16T11:26:02.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.650+0000 7fcc55668740 20 client.12510 trim_cache size 0 max 16384 2023-10-16T11:26:02.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.666+0000 7fcc55668740 1 Processor -- start 2023-10-16T11:26:02.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.666+0000 7fcc55668740 1 -- start start 2023-10-16T11:26:02.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.667+0000 7fcc55668740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc55f80 0x55b20bc2eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.667+0000 7fcc55668740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc56360 0x55b20bc56740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.667+0000 7fcc55668740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc5c900 0x55b20bc55330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.667+0000 7fcc55668740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20b8f8f20 con 0x55b20bc5c900 2023-10-16T11:26:02.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.667+0000 7fcc55668740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20bc50d20 con 0x55b20bc56360 2023-10-16T11:26:02.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.667+0000 7fcc55668740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b20bc76e80 con 0x55b20bc55f80 2023-10-16T11:26:02.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.668+0000 7fcc5106c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc5c900 0x55b20bc55330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.668+0000 7fcc5106c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc5c900 0x55b20bc55330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40600/0 (socket says 172.21.15.84:40600) 2023-10-16T11:26:02.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.668+0000 7fcc5106c640 1 -- 172.21.15.84:0/3352344400 learned_addr learned my addr 172.21.15.84:0/3352344400 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:02.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.668+0000 7fcc5086b640 1 --2- 172.21.15.84:0/3352344400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc55f80 0x55b20bc2eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.668+0000 7fcc4bfff640 1 --2- 172.21.15.84:0/3352344400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc56360 0x55b20bc56740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.668+0000 7fcc5106c640 1 -- 172.21.15.84:0/3352344400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc55f80 msgr2=0x55b20bc2eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:02.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.668+0000 7fcc5106c640 1 --2- 172.21.15.84:0/3352344400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc55f80 0x55b20bc2eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.668+0000 7fcc5106c640 1 -- 172.21.15.84:0/3352344400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc56360 msgr2=0x55b20bc56740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:02.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.668+0000 7fcc5106c640 1 --2- 172.21.15.84:0/3352344400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc56360 0x55b20bc56740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.668+0000 7fcc5106c640 1 -- 172.21.15.84:0/3352344400 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b20bb31b80 con 0x55b20bc5c900 2023-10-16T11:26:02.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.669+0000 7fcc5106c640 1 --2- 172.21.15.84:0/3352344400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc5c900 0x55b20bc55330 secure :-1 s=READY pgs=3012 cs=0 l=1 rev1=1 crypto rx=0x7fcc44053e40 tx=0x7fcc44059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0759ed2d248f640 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:02.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.669+0000 7fcc42ffd640 1 -- 172.21.15.84:0/3352344400 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4400e040 con 0x55b20bc5c900 2023-10-16T11:26:02.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.669+0000 7fcc42ffd640 1 -- 172.21.15.84:0/3352344400 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4405ba30 con 0x55b20bc5c900 2023-10-16T11:26:02.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.669+0000 7fcc42ffd640 1 -- 172.21.15.84:0/3352344400 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc44053780 con 0x55b20bc5c900 2023-10-16T11:26:02.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.669+0000 7fcc55668740 1 -- 172.21.15.84:0/3352344400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc5c900 msgr2=0x55b20bc55330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:02.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.669+0000 7fcc55668740 1 --2- 172.21.15.84:0/3352344400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc5c900 0x55b20bc55330 secure :-1 s=READY pgs=3012 cs=0 l=1 rev1=1 crypto rx=0x7fcc44053e40 tx=0x7fcc44059000 comp rx=0 tx=0).stop 2023-10-16T11:26:02.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.669+0000 7fcc55668740 1 -- 172.21.15.84:0/3352344400 shutdown_connections 2023-10-16T11:26:02.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.669+0000 7fcc55668740 1 --2- 172.21.15.84:0/3352344400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc5c900 0x55b20bc55330 unknown :-1 s=CLOSED pgs=3012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.669+0000 7fcc55668740 1 --2- 172.21.15.84:0/3352344400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc56360 0x55b20bc56740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.669+0000 7fcc55668740 1 --2- 172.21.15.84:0/3352344400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc55f80 0x55b20bc2eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.669+0000 7fcc55668740 1 -- 172.21.15.84:0/3352344400 >> 172.21.15.84:0/3352344400 conn(0x55b20bc293c0 msgr2=0x55b20bc77220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:02.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.669+0000 7fcc55668740 1 -- 172.21.15.84:0/3352344400 shutdown_connections 2023-10-16T11:26:02.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.670+0000 7fcc55668740 1 -- 172.21.15.84:0/3352344400 wait complete. 2023-10-16T11:26:02.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.670+0000 7fcc55668740 1 -- start start 2023-10-16T11:26:02.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.670+0000 7fcc55668740 1 Processor -- start 2023-10-16T11:26:02.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.671+0000 7fcc55668740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc55f80 0x55b20bc3c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.671+0000 7fcc55668740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc56360 0x55b20bcc17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.671+0000 7fcc55668740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc5c900 0x55b20bcc1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.671+0000 7fcc55668740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20bc28030 con 0x55b20bc56360 2023-10-16T11:26:02.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.671+0000 7fcc55668740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20bc82890 con 0x55b20bc5c900 2023-10-16T11:26:02.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.671+0000 7fcc55668740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b20bc28800 con 0x55b20bc55f80 2023-10-16T11:26:02.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.671+0000 7fcc55668740 10 client.0 _ll_register_callbacks cb 0x7fcc440011c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:26:02.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.672+0000 7fcc4bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc56360 0x55b20bcc17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.672+0000 7fcc4bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc56360 0x55b20bcc17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40608/0 (socket says 172.21.15.84:40608) 2023-10-16T11:26:02.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.672+0000 7fcc4bfff640 1 -- 172.21.15.84:0/293422534 learned_addr learned my addr 172.21.15.84:0/293422534 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:02.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.672+0000 7fcc5086b640 1 --2- 172.21.15.84:0/293422534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc55f80 0x55b20bc3c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.672+0000 7fcc5106c640 1 --2- 172.21.15.84:0/293422534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc5c900 0x55b20bcc1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.672+0000 7fcc4bfff640 1 -- 172.21.15.84:0/293422534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc55f80 msgr2=0x55b20bc3c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:02.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.672+0000 7fcc4bfff640 1 --2- 172.21.15.84:0/293422534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc55f80 0x55b20bc3c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.672+0000 7fcc4bfff640 1 -- 172.21.15.84:0/293422534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc5c900 msgr2=0x55b20bcc1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:02.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.672+0000 7fcc4bfff640 1 --2- 172.21.15.84:0/293422534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc5c900 0x55b20bcc1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.672+0000 7fcc4bfff640 1 -- 172.21.15.84:0/293422534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b20bb92da0 con 0x55b20bc56360 2023-10-16T11:26:02.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.672+0000 7fcc5106c640 1 --2- 172.21.15.84:0/293422534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc5c900 0x55b20bcc1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:02.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.672+0000 7fcc4bfff640 1 --2- 172.21.15.84:0/293422534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc56360 0x55b20bcc17e0 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009d90 tx=0x7fcc3402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b575e06145e78802 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:02.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.672+0000 7fcc2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T11:26:02.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.673+0000 7fcc2b7fe640 1 -- 172.21.15.84:0/293422534 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc34043040 con 0x55b20bc56360 2023-10-16T11:26:02.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.673+0000 7fcc2b7fe640 1 -- 172.21.15.84:0/293422534 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7fcc3403ed50 con 0x55b20bc56360 2023-10-16T11:26:02.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.673+0000 7fcc55668740 1 -- 172.21.15.84:0/293422534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b20bc4ead0 con 0x55b20bc56360 2023-10-16T11:26:02.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.673+0000 7fcc2b7fe640 1 -- 172.21.15.84:0/293422534 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=137) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc34048440 con 0x55b20bc56360 2023-10-16T11:26:02.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.673+0000 7fcc55668740 10 client.12516 fetch_fsmap learned FSMap version 137 2023-10-16T11:26:02.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.673+0000 7fcc55668740 1 -- 172.21.15.84:0/293422534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=137,monmap=2+}) v3 -- 0x55b20bb57170 con 0x55b20bc56360 2023-10-16T11:26:02.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.673+0000 7fcc2b7fe640 1 -- 172.21.15.84:0/293422534 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b20bc4ead0 con 0x55b20bc56360 2023-10-16T11:26:02.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.673+0000 7fcc2b7fe640 1 -- 172.21.15.84:0/293422534 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 137) v1 ==== 3891+0+0 (secure 0 0 0) 0x7fcc3404e250 con 0x55b20bc56360 2023-10-16T11:26:02.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.673+0000 7fcc55668740 10 client.12516 fetch_fsmap finished waiting for FSMap version 137 2023-10-16T11:26:02.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.673+0000 7fcc55668740 10 client.12516 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T11:26:02.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.673+0000 7fcc55668740 20 client.12516 populate_metadata read hostname 'smithi084' 2023-10-16T11:26:02.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.673+0000 7fcc55668740 1 --2- 172.21.15.84:0/293422534 >> [v2:172.21.15.84:6834/369067539,v1:172.21.15.84:6835/369067539] conn(0x55b20bcdf9f0 0x55b20bcdfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.673+0000 7fcc55668740 4 client.12516 mds_command: new command op to 12183 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T11:26:02.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.673+0000 7fcc55668740 1 -- 172.21.15.84:0/293422534 --> [v2:172.21.15.84:6834/369067539,v1:172.21.15.84:6835/369067539] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fcc3c011e80 con 0x55b20bcdf9f0 2023-10-16T11:26:02.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.674+0000 7fcc5086b640 1 --2- 172.21.15.84:0/293422534 >> [v2:172.21.15.84:6834/369067539,v1:172.21.15.84:6835/369067539] conn(0x55b20bcdf9f0 0x55b20bcdfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.674+0000 7fcc2b7fe640 10 client.12516 ms_handle_connect on v2:172.21.15.84:6834/369067539 2023-10-16T11:26:02.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.674+0000 7fcc5086b640 1 --2- 172.21.15.84:0/293422534 >> [v2:172.21.15.84:6834/369067539,v1:172.21.15.84:6835/369067539] conn(0x55b20bcdf9f0 0x55b20bcdfdd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bdf73dc06e2c87d server_cookie=a724c6e22269dbf7 in_seq=0 out_seq=0 2023-10-16T11:26:02.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.674+0000 7fcc2b7fe640 1 -- 172.21.15.84:0/293422534 <== mds.0 v2:172.21.15.84:6834/369067539 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x7fcc3c011e80 con 0x55b20bcdf9f0 2023-10-16T11:26:02.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.674+0000 7fcc2b7fe640 10 client.12516 handle_command_reply: tid=0 2023-10-16T11:26:02.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.674+0000 7fcc55668740 1 client.12516 shutdown 2023-10-16T11:26:02.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.675+0000 7fcc55668740 1 -- 172.21.15.84:0/293422534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc56360 msgr2=0x55b20bcc17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:02.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.675+0000 7fcc55668740 1 --2- 172.21.15.84:0/293422534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc56360 0x55b20bcc17e0 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009d90 tx=0x7fcc3402e070 comp rx=0 tx=0).stop 2023-10-16T11:26:02.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.675+0000 7fcc55668740 1 -- 172.21.15.84:0/293422534 shutdown_connections 2023-10-16T11:26:02.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.675+0000 7fcc55668740 1 --2- 172.21.15.84:0/293422534 >> [v2:172.21.15.84:6834/369067539,v1:172.21.15.84:6835/369067539] conn(0x55b20bcdf9f0 0x55b20bcdfdd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.675+0000 7fcc55668740 1 --2- 172.21.15.84:0/293422534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b20bc5c900 0x55b20bcc1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.675+0000 7fcc55668740 1 --2- 172.21.15.84:0/293422534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b20bc56360 0x55b20bcc17e0 unknown :-1 s=CLOSED pgs=3013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.675+0000 7fcc55668740 1 --2- 172.21.15.84:0/293422534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55b20bc55f80 0x55b20bc3c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.675+0000 7fcc2b7fe640 0 client.12516 ms_handle_reset on v2:172.21.15.84:6834/369067539 2023-10-16T11:26:02.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.675+0000 7fcc55668740 1 -- 172.21.15.84:0/293422534 >> 172.21.15.84:0/293422534 conn(0x55b20bc293c0 msgr2=0x55b20bc55230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:02.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.675+0000 7fcc55668740 1 -- 172.21.15.84:0/293422534 shutdown_connections 2023-10-16T11:26:02.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.675+0000 7fcc55668740 1 -- 172.21.15.84:0/293422534 wait complete. 2023-10-16T11:26:02.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.675+0000 7fcc55668740 20 client.12516 trim_cache size 0 max 16384 2023-10-16T11:26:02.691 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:26:02.691 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "63fb4d68-f881-4694-b880-d10369105c0e", 2023-10-16T11:26:02.691 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-16T11:26:02.691 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 12183, 2023-10-16T11:26:02.692 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-16T11:26:02.692 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-16T11:26:02.692 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "backup_fs", 2023-10-16T11:26:02.692 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 11.367546099, 2023-10-16T11:26:02.693 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 137, 2023-10-16T11:26:02.693 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 241, 2023-10-16T11:26:02.693 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 241, 2023-10-16T11:26:02.693 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 30.912987376 2023-10-16T11:26:02.693 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:26:02.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.676+0000 7fcc52acf640 1 -- 172.21.15.84:0/649713937 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fcc2c06b0b0 msgr2=0x7fcc2c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:02.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.676+0000 7fcc52acf640 1 --2- 172.21.15.84:0/649713937 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fcc2c06b0b0 0x7fcc2c06d570 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009a30 tx=0x7fcc3c008040 comp rx=0 tx=0).stop 2023-10-16T11:26:02.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.676+0000 7fcc52acf640 1 -- 172.21.15.84:0/649713937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc4c15ad70 msgr2=0x7fcc4c0b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:02.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.676+0000 7fcc52acf640 1 --2- 172.21.15.84:0/649713937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc4c15ad70 0x7fcc4c0b2b90 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400c420 tx=0x7fcc4400c8e0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.676+0000 7fcc52acf640 1 -- 172.21.15.84:0/649713937 shutdown_connections 2023-10-16T11:26:02.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.676+0000 7fcc52acf640 1 --2- 172.21.15.84:0/649713937 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fcc2c06b0b0 0x7fcc2c06d570 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.676+0000 7fcc52acf640 1 --2- 172.21.15.84:0/649713937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc4c15ad70 0x7fcc4c0b2b90 unknown :-1 s=CLOSED pgs=3009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.676+0000 7fcc52acf640 1 --2- 172.21.15.84:0/649713937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc4c0ca3b0 0x7fcc4c0b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.676+0000 7fcc52acf640 1 --2- 172.21.15.84:0/649713937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc4c0c9ca0 0x7fcc4c1596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.676+0000 7fcc52acf640 1 -- 172.21.15.84:0/649713937 >> 172.21.15.84:0/649713937 conn(0x7fcc4c0ce9b0 msgr2=0x7fcc4c0af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:02.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.676+0000 7fcc52acf640 1 -- 172.21.15.84:0/649713937 shutdown_connections 2023-10-16T11:26:02.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.676+0000 7fcc52acf640 1 -- 172.21.15.84:0/649713937 wait complete. 2023-10-16T11:26:02.698 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T11:26:02.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.901+0000 7f1174741640 1 Processor -- start 2023-10-16T11:26:02.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.901+0000 7f1174741640 1 -- start start 2023-10-16T11:26:02.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.902+0000 7f1174741640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116c159e90 0x7f116c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.902+0000 7f1174741640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f116c06c140 0x7f116c06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.902+0000 7f1174741640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f116c070360 con 0x7f116c159ab0 2023-10-16T11:26:02.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.902+0000 7f1174741640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116c06cb10 con 0x7f116c06c140 2023-10-16T11:26:02.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.902+0000 7f1174741640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116c15d890 con 0x7f116c159e90 2023-10-16T11:26:02.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.903+0000 7f1172cde640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f116c06c140 0x7f116c06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.903+0000 7f1172cde640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f116c06c140 0x7f116c06c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40612/0 (socket says 172.21.15.84:40612) 2023-10-16T11:26:02.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.903+0000 7f1172cde640 1 -- 172.21.15.84:0/2012215504 learned_addr learned my addr 172.21.15.84:0/2012215504 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:02.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.903+0000 7f1171cdc640 1 --2- 172.21.15.84:0/2012215504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116c159e90 0x7f116c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.904+0000 7f1172cde640 1 -- 172.21.15.84:0/2012215504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116c159e90 msgr2=0x7f116c0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:02.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.904+0000 7f1172cde640 1 --2- 172.21.15.84:0/2012215504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116c159e90 0x7f116c0ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.904+0000 7f1172cde640 1 -- 172.21.15.84:0/2012215504 >> v1:172.21.15.157:6789/0 conn(0x7f116c159ab0 legacy=0x7f116c06fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:02.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.904+0000 7f1172cde640 1 -- 172.21.15.84:0/2012215504 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c0cd1f0 con 0x7f116c06c140 2023-10-16T11:26:02.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.904+0000 7f1171cdc640 1 --2- 172.21.15.84:0/2012215504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116c159e90 0x7f116c0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:02.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.905+0000 7f1172cde640 1 --2- 172.21.15.84:0/2012215504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f116c06c140 0x7f116c06c5d0 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 crypto rx=0x7f1164009e80 tx=0x7f116402bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4e264b9632f6a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:02.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.905+0000 7f11714db640 1 -- 172.21.15.84:0/2012215504 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116402c930 con 0x7f116c06c140 2023-10-16T11:26:02.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.905+0000 7f11714db640 1 -- 172.21.15.84:0/2012215504 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116402ca90 con 0x7f116c06c140 2023-10-16T11:26:02.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.905+0000 7f11714db640 1 -- 172.21.15.84:0/2012215504 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116402ce20 con 0x7f116c06c140 2023-10-16T11:26:02.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.906+0000 7f1174741640 1 -- 172.21.15.84:0/2012215504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f116c06c140 msgr2=0x7f116c06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:02.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.906+0000 7f1174741640 1 --2- 172.21.15.84:0/2012215504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f116c06c140 0x7f116c06c5d0 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 crypto rx=0x7f1164009e80 tx=0x7f116402bd90 comp rx=0 tx=0).stop 2023-10-16T11:26:02.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.906+0000 7f1174741640 1 -- 172.21.15.84:0/2012215504 shutdown_connections 2023-10-16T11:26:02.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.906+0000 7f1174741640 1 --2- 172.21.15.84:0/2012215504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f116c06c140 0x7f116c06c5d0 unknown :-1 s=CLOSED pgs=3014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.906+0000 7f1174741640 1 --2- 172.21.15.84:0/2012215504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116c159e90 0x7f116c0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.906+0000 7f1174741640 1 -- 172.21.15.84:0/2012215504 >> 172.21.15.84:0/2012215504 conn(0x7f116c0ce9b0 msgr2=0x7f116c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:02.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.906+0000 7f1174741640 1 -- 172.21.15.84:0/2012215504 shutdown_connections 2023-10-16T11:26:02.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.906+0000 7f1174741640 1 -- 172.21.15.84:0/2012215504 wait complete. 2023-10-16T11:26:02.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.908+0000 7f1174741640 1 Processor -- start 2023-10-16T11:26:02.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.908+0000 7f1174741640 1 -- start start 2023-10-16T11:26:02.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.909+0000 7f1174741640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f116c06c140 0x7f116c1570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.909+0000 7f1174741640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f116c159ab0 0x7f116c1e6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.909+0000 7f1174741640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116c159e90 0x7f116c06b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.909+0000 7f1174741640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c157620 con 0x7f116c06c140 2023-10-16T11:26:02.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.909+0000 7f1174741640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c0b1bc0 con 0x7f116c159ab0 2023-10-16T11:26:02.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.909+0000 7f1174741640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116c1e7820 con 0x7f116c159e90 2023-10-16T11:26:02.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.909+0000 7f11724dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f116c06c140 0x7f116c1570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.909+0000 7f11724dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f116c06c140 0x7f116c1570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40628/0 (socket says 172.21.15.84:40628) 2023-10-16T11:26:02.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.909+0000 7f11724dd640 1 -- 172.21.15.84:0/1534485574 learned_addr learned my addr 172.21.15.84:0/1534485574 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:02.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.909+0000 7f11724dd640 1 -- 172.21.15.84:0/1534485574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116c159e90 msgr2=0x7f116c06b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:02.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.909+0000 7f1172cde640 1 --2- 172.21.15.84:0/1534485574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116c159e90 0x7f116c06b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.909+0000 7f1171cdc640 1 --2- 172.21.15.84:0/1534485574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f116c159ab0 0x7f116c1e6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.909+0000 7f11724dd640 1 --2- 172.21.15.84:0/1534485574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116c159e90 0x7f116c06b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.909+0000 7f11724dd640 1 -- 172.21.15.84:0/1534485574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f116c159ab0 msgr2=0x7f116c1e6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:02.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.909+0000 7f11724dd640 1 --2- 172.21.15.84:0/1534485574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f116c159ab0 0x7f116c1e6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:02.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.909+0000 7f11724dd640 1 -- 172.21.15.84:0/1534485574 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1164009b30 con 0x7f116c06c140 2023-10-16T11:26:02.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.910+0000 7f11724dd640 1 --2- 172.21.15.84:0/1534485574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f116c06c140 0x7f116c1570e0 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 crypto rx=0x7f116800b3f0 tx=0x7f116800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:02.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.911+0000 7f1162ffd640 1 -- 172.21.15.84:0/1534485574 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168011830 con 0x7f116c06c140 2023-10-16T11:26:02.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.911+0000 7f1162ffd640 1 -- 172.21.15.84:0/1534485574 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1168011990 con 0x7f116c06c140 2023-10-16T11:26:02.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.911+0000 7f1162ffd640 1 -- 172.21.15.84:0/1534485574 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168011d40 con 0x7f116c06c140 2023-10-16T11:26:02.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.911+0000 7f1174741640 1 -- 172.21.15.84:0/1534485574 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116c1e7b00 con 0x7f116c06c140 2023-10-16T11:26:02.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.911+0000 7f1174741640 1 -- 172.21.15.84:0/1534485574 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116c0b1dd0 con 0x7f116c06c140 2023-10-16T11:26:02.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.913+0000 7f1174741640 1 -- 172.21.15.84:0/1534485574 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116c0cd790 con 0x7f116c06c140 2023-10-16T11:26:02.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.913+0000 7f1162ffd640 1 -- 172.21.15.84:0/1534485574 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f116801a400 con 0x7f116c06c140 2023-10-16T11:26:02.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.915+0000 7f1162ffd640 1 --2- 172.21.15.84:0/1534485574 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f115006b0e0 0x7f115006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:02.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.915+0000 7f1162ffd640 1 -- 172.21.15.84:0/1534485574 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f1168089bc0 con 0x7f116c06c140 2023-10-16T11:26:02.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.915+0000 7f1171cdc640 1 --2- 172.21.15.84:0/1534485574 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f115006b0e0 0x7f115006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:02.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.916+0000 7f1171cdc640 1 --2- 172.21.15.84:0/1534485574 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f115006b0e0 0x7f115006d5a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f116c0b0070 tx=0x7f115c008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:02.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:02.919+0000 7f1162ffd640 1 -- 172.21.15.84:0/1534485574 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f116c0cd790 con 0x7f116c06c140 2023-10-16T11:26:03.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.137+0000 7f1174741640 1 -- 172.21.15.84:0/1534485574 --> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f116c1e8020 con 0x7f115006b0e0 2023-10-16T11:26:03.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.140+0000 7f1162ffd640 1 -- 172.21.15.84:0/1534485574 <== mgr.11814 v2:172.21.15.84:6832/2963529201 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f116c1e8020 con 0x7f115006b0e0 2023-10-16T11:26:03.142 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T11:26:03.142 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:03.143 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephfs" 2023-10-16T11:26:03.143 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:03.143 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:03.143 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "backup_fs" 2023-10-16T11:26:03.143 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:03.144 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T11:26:03.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.142+0000 7f1174741640 1 -- 172.21.15.84:0/1534485574 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f115006b0e0 msgr2=0x7f115006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:03.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.142+0000 7f1174741640 1 --2- 172.21.15.84:0/1534485574 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f115006b0e0 0x7f115006d5a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f116c0b0070 tx=0x7f115c008040 comp rx=0 tx=0).stop 2023-10-16T11:26:03.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.142+0000 7f1174741640 1 -- 172.21.15.84:0/1534485574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f116c06c140 msgr2=0x7f116c1570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:03.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.142+0000 7f1174741640 1 --2- 172.21.15.84:0/1534485574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f116c06c140 0x7f116c1570e0 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 crypto rx=0x7f116800b3f0 tx=0x7f116800b8b0 comp rx=0 tx=0).stop 2023-10-16T11:26:03.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.143+0000 7f1174741640 1 -- 172.21.15.84:0/1534485574 shutdown_connections 2023-10-16T11:26:03.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.143+0000 7f1174741640 1 --2- 172.21.15.84:0/1534485574 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f115006b0e0 0x7f115006d5a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:03.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.143+0000 7f1174741640 1 --2- 172.21.15.84:0/1534485574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f116c159e90 0x7f116c06b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:03.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.143+0000 7f1174741640 1 --2- 172.21.15.84:0/1534485574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f116c159ab0 0x7f116c1e6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:03.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.143+0000 7f1174741640 1 --2- 172.21.15.84:0/1534485574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f116c06c140 0x7f116c1570e0 unknown :-1 s=CLOSED pgs=3015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:03.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.143+0000 7f1174741640 1 -- 172.21.15.84:0/1534485574 >> 172.21.15.84:0/1534485574 conn(0x7f116c0ce9b0 msgr2=0x7f116c158de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:03.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.143+0000 7f1174741640 1 -- 172.21.15.84:0/1534485574 shutdown_connections 2023-10-16T11:26:03.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.143+0000 7f1174741640 1 -- 172.21.15.84:0/1534485574 wait complete. 2023-10-16T11:26:03.163 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T11:26:03.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.381+0000 7f726a244640 1 Processor -- start 2023-10-16T11:26:03.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.381+0000 7f726a244640 1 -- start start 2023-10-16T11:26:03.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.382+0000 7f726a244640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7264159ab0 0x7f72640ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:03.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.382+0000 7f726a244640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f726406c140 0x7f726406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:03.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.382+0000 7f726a244640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7264070360 con 0x7f7264159e90 2023-10-16T11:26:03.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.382+0000 7f726a244640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726406cb10 con 0x7f7264159ab0 2023-10-16T11:26:03.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.382+0000 7f726a244640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726415d890 con 0x7f726406c140 2023-10-16T11:26:03.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.383+0000 7f725bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7264159e90 0x7f726406fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40572/0 (socket says 172.21.15.84:40572) 2023-10-16T11:26:03.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.383+0000 7f725bfff640 1 -- 172.21.15.84:0/4262857792 learned_addr learned my addr 172.21.15.84:0/4262857792 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:03.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.383+0000 7f7262ffd640 1 -- 172.21.15.84:0/4262857792 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1294821772 0 0) 0x7f7264070360 con 0x7f7264159e90 2023-10-16T11:26:03.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.383+0000 7f72637fe640 1 --2- 172.21.15.84:0/4262857792 >> v2:172.21.15.157:3300/0 conn(0x7f7264159ab0 0x7f72640ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:03.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.383+0000 7f7262ffd640 1 -- 172.21.15.84:0/4262857792 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7250005730 con 0x7f7264159e90 2023-10-16T11:26:03.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.383+0000 7f7263fff640 1 --2- 172.21.15.84:0/4262857792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f726406c140 0x7f726406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:03.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.384+0000 7f7262ffd640 1 -- 172.21.15.84:0/4262857792 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3478661258 0 0) 0x7f7250005730 con 0x7f7264159e90 2023-10-16T11:26:03.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.384+0000 7f7262ffd640 1 -- 172.21.15.84:0/4262857792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f726406c140 msgr2=0x7f726406c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:03.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.384+0000 7f7262ffd640 1 --2- 172.21.15.84:0/4262857792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f726406c140 0x7f726406c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:03.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.384+0000 7f7262ffd640 1 -- 172.21.15.84:0/4262857792 >> v2:172.21.15.157:3300/0 conn(0x7f7264159ab0 msgr2=0x7f72640ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:03.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.384+0000 7f7262ffd640 1 --2- 172.21.15.84:0/4262857792 >> v2:172.21.15.157:3300/0 conn(0x7f7264159ab0 0x7f72640ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:03.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.384+0000 7f7262ffd640 1 -- 172.21.15.84:0/4262857792 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72640cd1f0 con 0x7f7264159e90 2023-10-16T11:26:03.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.384+0000 7f7262ffd640 1 -- 172.21.15.84:0/4262857792 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f724c001670 con 0x7f7264159e90 2023-10-16T11:26:03.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.384+0000 7f7262ffd640 1 -- 172.21.15.84:0/4262857792 >> v1:172.21.15.84:6789/0 conn(0x7f7264159e90 legacy=0x7f726406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:03.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.384+0000 7f7262ffd640 1 --2- 172.21.15.84:0/4262857792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f726406c140 0x7f7250005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:03.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.384+0000 7f7262ffd640 1 --2- 172.21.15.84:0/4262857792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7250006150 0x7f72500069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:03.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.384+0000 7f7262ffd640 1 --2- 172.21.15.84:0/4262857792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7250006f10 0x7f7250007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:03.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.384+0000 7f7262ffd640 1 -- 172.21.15.84:0/4262857792 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726406cb10 con 0x7f7250006f10 2023-10-16T11:26:03.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.384+0000 7f7262ffd640 1 -- 172.21.15.84:0/4262857792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726415d890 con 0x7f7250006150 2023-10-16T11:26:03.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.384+0000 7f7262ffd640 1 -- 172.21.15.84:0/4262857792 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725000dd70 con 0x7f726406c140 2023-10-16T11:26:03.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.384+0000 7f72637fe640 1 --2- 172.21.15.84:0/4262857792 >> v2:172.21.15.157:3300/0 conn(0x7f7264159ab0 0x7f72640ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:26:03.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.384+0000 7f7263fff640 1 --2- 172.21.15.84:0/4262857792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7250006f10 0x7f7250007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:03.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.385+0000 7f7263fff640 1 -- 172.21.15.84:0/4262857792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f726406c140 msgr2=0x7f7250005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:03.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.385+0000 7f7263fff640 1 --2- 172.21.15.84:0/4262857792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f726406c140 0x7f7250005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:03.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.385+0000 7f7263fff640 1 -- 172.21.15.84:0/4262857792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7250006150 msgr2=0x7f72500069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:03.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.385+0000 7f7263fff640 1 --2- 172.21.15.84:0/4262857792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7250006150 0x7f72500069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:03.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.385+0000 7f7263fff640 1 -- 172.21.15.84:0/4262857792 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7250005730 con 0x7f7250006f10 2023-10-16T11:26:03.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.386+0000 7f7263fff640 1 --2- 172.21.15.84:0/4262857792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7250006f10 0x7f7250007390 secure :-1 s=READY pgs=3017 cs=0 l=1 rev1=1 crypto rx=0x7f725c009fc0 tx=0x7f7250007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a2e2646eef9aeb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:03.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.386+0000 7f7262ffd640 1 -- 172.21.15.84:0/4262857792 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c02cc60 con 0x7f7250006f10 2023-10-16T11:26:03.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.386+0000 7f7262ffd640 1 -- 172.21.15.84:0/4262857792 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c00f040 con 0x7f7250006f10 2023-10-16T11:26:03.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.386+0000 7f7262ffd640 1 -- 172.21.15.84:0/4262857792 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c034440 con 0x7f7250006f10 2023-10-16T11:26:03.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.386+0000 7f726a244640 1 -- 172.21.15.84:0/4262857792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7250006f10 msgr2=0x7f7250007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:03.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.386+0000 7f726a244640 1 --2- 172.21.15.84:0/4262857792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7250006f10 0x7f7250007390 secure :-1 s=READY pgs=3017 cs=0 l=1 rev1=1 crypto rx=0x7f725c009fc0 tx=0x7f7250007a00 comp rx=0 tx=0).stop 2023-10-16T11:26:03.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.386+0000 7f7263fff640 1 -- 172.21.15.84:0/4262857792 reap_dead start 2023-10-16T11:26:03.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.386+0000 7f726a244640 1 -- 172.21.15.84:0/4262857792 shutdown_connections 2023-10-16T11:26:03.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.386+0000 7f726a244640 1 -- 172.21.15.84:0/4262857792 >> 172.21.15.84:0/4262857792 conn(0x7f72640ce9b0 msgr2=0x7f72640cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:03.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.387+0000 7f726a244640 1 -- 172.21.15.84:0/4262857792 shutdown_connections 2023-10-16T11:26:03.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.387+0000 7f726a244640 1 -- 172.21.15.84:0/4262857792 wait complete. 2023-10-16T11:26:03.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.388+0000 7f726a244640 1 Processor -- start 2023-10-16T11:26:03.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.388+0000 7f726a244640 1 -- start start 2023-10-16T11:26:03.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.389+0000 7f726a244640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7250006f10 0x7f726414bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:03.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.389+0000 7f726a244640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72640c7ee0 0x7f7264158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:03.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.389+0000 7f726a244640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72641592e0 0x7f7264070780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:03.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.389+0000 7f726a244640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72640c6e40 con 0x7f72640c7ee0 2023-10-16T11:26:03.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.389+0000 7f726a244640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72640c7120 con 0x7f72641592e0 2023-10-16T11:26:03.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.389+0000 7f726a244640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72640c7400 con 0x7f7250006f10 2023-10-16T11:26:03.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.390+0000 7f725bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72640c7ee0 0x7f7264158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:03.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.390+0000 7f725bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72640c7ee0 0x7f7264158da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40646/0 (socket says 172.21.15.84:40646) 2023-10-16T11:26:03.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.390+0000 7f725bfff640 1 -- 172.21.15.84:0/228772902 learned_addr learned my addr 172.21.15.84:0/228772902 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:03.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.390+0000 7f72637fe640 1 --2- 172.21.15.84:0/228772902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7250006f10 0x7f726414bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:03.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.390+0000 7f7263fff640 1 --2- 172.21.15.84:0/228772902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72641592e0 0x7f7264070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:03.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.390+0000 7f725bfff640 1 -- 172.21.15.84:0/228772902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7250006f10 msgr2=0x7f726414bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:03.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.390+0000 7f725bfff640 1 --2- 172.21.15.84:0/228772902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7250006f10 0x7f726414bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:03.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.390+0000 7f725bfff640 1 -- 172.21.15.84:0/228772902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72641592e0 msgr2=0x7f7264070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:03.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.390+0000 7f725bfff640 1 --2- 172.21.15.84:0/228772902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72641592e0 0x7f7264070780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:03.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.390+0000 7f725bfff640 1 -- 172.21.15.84:0/228772902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72640c76e0 con 0x7f72640c7ee0 2023-10-16T11:26:03.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.390+0000 7f7263fff640 1 --2- 172.21.15.84:0/228772902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72641592e0 0x7f7264070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:03.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.390+0000 7f725bfff640 1 --2- 172.21.15.84:0/228772902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72640c7ee0 0x7f7264158da0 secure :-1 s=READY pgs=3018 cs=0 l=1 rev1=1 crypto rx=0x7f724c00c770 tx=0x7f724c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:03.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.391+0000 7f7260ff9640 1 -- 172.21.15.84:0/228772902 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c0074e0 con 0x7f72640c7ee0 2023-10-16T11:26:03.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.391+0000 7f7260ff9640 1 -- 172.21.15.84:0/228772902 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c007660 con 0x7f72640c7ee0 2023-10-16T11:26:03.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.391+0000 7f726a244640 1 -- 172.21.15.84:0/228772902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72640c79c0 con 0x7f72640c7ee0 2023-10-16T11:26:03.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.391+0000 7f7260ff9640 1 -- 172.21.15.84:0/228772902 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c004a20 con 0x7f72640c7ee0 2023-10-16T11:26:03.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.391+0000 7f726a244640 1 -- 172.21.15.84:0/228772902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f726414c230 con 0x7f72640c7ee0 2023-10-16T11:26:03.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.392+0000 7f726a244640 1 -- 172.21.15.84:0/228772902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72640c7bb0 con 0x7f72640c7ee0 2023-10-16T11:26:03.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.393+0000 7f7260ff9640 1 -- 172.21.15.84:0/228772902 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 143) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f724c004b80 con 0x7f72640c7ee0 2023-10-16T11:26:03.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.394+0000 7f7260ff9640 1 --2- 172.21.15.84:0/228772902 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f724406b0e0 0x7f724406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:03.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.394+0000 7f7260ff9640 1 -- 172.21.15.84:0/228772902 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f724c089510 con 0x7f72640c7ee0 2023-10-16T11:26:03.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.394+0000 7f72637fe640 1 --2- 172.21.15.84:0/228772902 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f724406b0e0 0x7f724406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:03.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.395+0000 7f72637fe640 1 --2- 172.21.15.84:0/228772902 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f724406b0e0 0x7f724406d5a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7254006fd0 tx=0x7f7254008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:03.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.399+0000 7f7260ff9640 1 -- 172.21.15.84:0/228772902 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f724c053ee0 con 0x7f72640c7ee0 2023-10-16T11:26:03.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.640+0000 7f726a244640 1 -- 172.21.15.84:0/228772902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f726404e160 con 0x7f72640c7ee0 2023-10-16T11:26:03.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.923+0000 7f7260ff9640 1 -- 172.21.15.84:0/228772902 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v144) v1 ==== 92+0+0 (secure 0 0 0) 0x7f724c057500 con 0x7f72640c7ee0 2023-10-16T11:26:03.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.927+0000 7f726a244640 1 -- 172.21.15.84:0/228772902 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f724406b0e0 msgr2=0x7f724406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:03.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.927+0000 7f726a244640 1 --2- 172.21.15.84:0/228772902 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f724406b0e0 0x7f724406d5a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7254006fd0 tx=0x7f7254008040 comp rx=0 tx=0).stop 2023-10-16T11:26:03.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.927+0000 7f726a244640 1 -- 172.21.15.84:0/228772902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72640c7ee0 msgr2=0x7f7264158da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:03.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.927+0000 7f726a244640 1 --2- 172.21.15.84:0/228772902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72640c7ee0 0x7f7264158da0 secure :-1 s=READY pgs=3018 cs=0 l=1 rev1=1 crypto rx=0x7f724c00c770 tx=0x7f724c00cc30 comp rx=0 tx=0).stop 2023-10-16T11:26:03.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.929+0000 7f726a244640 1 -- 172.21.15.84:0/228772902 shutdown_connections 2023-10-16T11:26:03.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.929+0000 7f726a244640 1 --2- 172.21.15.84:0/228772902 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f724406b0e0 0x7f724406d5a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:03.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.929+0000 7f726a244640 1 --2- 172.21.15.84:0/228772902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72641592e0 0x7f7264070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:03.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.929+0000 7f726a244640 1 --2- 172.21.15.84:0/228772902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72640c7ee0 0x7f7264158da0 unknown :-1 s=CLOSED pgs=3018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:03.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.929+0000 7f726a244640 1 --2- 172.21.15.84:0/228772902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7250006f10 0x7f726414bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:03.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.929+0000 7f726a244640 1 -- 172.21.15.84:0/228772902 >> 172.21.15.84:0/228772902 conn(0x7f72640ce9b0 msgr2=0x7f7264156ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:03.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.931+0000 7f726a244640 1 -- 172.21.15.84:0/228772902 shutdown_connections 2023-10-16T11:26:03.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:03.931+0000 7f726a244640 1 -- 172.21.15.84:0/228772902 wait complete. 2023-10-16T11:26:03.953 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T11:26:04.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.150+0000 7f49ce911640 1 Processor -- start 2023-10-16T11:26:04.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.151+0000 7f49ce911640 1 -- start start 2023-10-16T11:26:04.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.152+0000 7f49ce911640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f49c815cd70 0x7f49c80d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:04.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.152+0000 7f49ce911640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c815d150 con 0x7f49c80b8480 2023-10-16T11:26:04.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.152+0000 7f49ce911640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c80d1180 con 0x7f49c80cded0 2023-10-16T11:26:04.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.152+0000 7f49ce911640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c80d1460 con 0x7f49c815cd70 2023-10-16T11:26:04.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.153+0000 7f49cd90f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f49c815cd70 0x7f49c80d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:04.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.153+0000 7f49cd90f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f49c815cd70 0x7f49c80d2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40684/0 (socket says 172.21.15.84:40684) 2023-10-16T11:26:04.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.153+0000 7f49cd90f640 1 -- 172.21.15.84:0/3872957979 learned_addr learned my addr 172.21.15.84:0/3872957979 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:04.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.153+0000 7f49cd90f640 1 -- 172.21.15.84:0/3872957979 >> v1:172.21.15.157:6789/0 conn(0x7f49c80cded0 legacy=0x7f49c80b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:04.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.153+0000 7f49cd90f640 1 -- 172.21.15.84:0/3872957979 >> v1:172.21.15.84:6789/0 conn(0x7f49c80b8480 legacy=0x7f49c80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:04.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.153+0000 7f49cd90f640 1 -- 172.21.15.84:0/3872957979 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c80d1740 con 0x7f49c815cd70 2023-10-16T11:26:04.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.154+0000 7f49cd90f640 1 --2- 172.21.15.84:0/3872957979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c815cd70 0x7f49c80d2220 secure :-1 s=READY pgs=3024 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009870 tx=0x7f49c402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3015b94484164fec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:04.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.155+0000 7f49cc90d640 1 -- 172.21.15.84:0/3872957979 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c402f850 con 0x7f49c815cd70 2023-10-16T11:26:04.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.155+0000 7f49cc90d640 1 -- 172.21.15.84:0/3872957979 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c402f9b0 con 0x7f49c815cd70 2023-10-16T11:26:04.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.155+0000 7f49cc90d640 1 -- 172.21.15.84:0/3872957979 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c402fd40 con 0x7f49c815cd70 2023-10-16T11:26:04.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.155+0000 7f49ce911640 1 -- 172.21.15.84:0/3872957979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c815cd70 msgr2=0x7f49c80d2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:04.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.155+0000 7f49ce911640 1 --2- 172.21.15.84:0/3872957979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c815cd70 0x7f49c80d2220 secure :-1 s=READY pgs=3024 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009870 tx=0x7f49c402ba30 comp rx=0 tx=0).stop 2023-10-16T11:26:04.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.155+0000 7f49ce911640 1 -- 172.21.15.84:0/3872957979 shutdown_connections 2023-10-16T11:26:04.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.155+0000 7f49ce911640 1 --2- 172.21.15.84:0/3872957979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c815cd70 0x7f49c80d2220 unknown :-1 s=CLOSED pgs=3024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.155+0000 7f49ce911640 1 -- 172.21.15.84:0/3872957979 >> 172.21.15.84:0/3872957979 conn(0x7f49c80cf570 msgr2=0x7f49c80cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:04.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.155+0000 7f49ce911640 1 -- 172.21.15.84:0/3872957979 shutdown_connections 2023-10-16T11:26:04.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.155+0000 7f49ce911640 1 -- 172.21.15.84:0/3872957979 wait complete. 2023-10-16T11:26:04.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.157+0000 7f49ce911640 1 Processor -- start 2023-10-16T11:26:04.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.157+0000 7f49ce911640 1 -- start start 2023-10-16T11:26:04.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.158+0000 7f49ce911640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c80b8480 0x7f49c80cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:04.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.158+0000 7f49ce911640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49c80cded0 0x7f49c80aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:04.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.158+0000 7f49ce911640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c815cd70 0x7f49c80b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:04.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.158+0000 7f49ce911640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c80cc620 con 0x7f49c80b8480 2023-10-16T11:26:04.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.158+0000 7f49ce911640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c81515f0 con 0x7f49c815cd70 2023-10-16T11:26:04.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.158+0000 7f49ce911640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c80c9f30 con 0x7f49c80cded0 2023-10-16T11:26:04.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.158+0000 7f49cd90f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c80b8480 0x7f49c80cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:04.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.158+0000 7f49cd90f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c80b8480 0x7f49c80cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40686/0 (socket says 172.21.15.84:40686) 2023-10-16T11:26:04.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.158+0000 7f49cd90f640 1 -- 172.21.15.84:0/603190534 learned_addr learned my addr 172.21.15.84:0/603190534 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:04.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.159+0000 7f49cd90f640 1 -- 172.21.15.84:0/603190534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49c80cded0 msgr2=0x7f49c80aafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:04.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.159+0000 7f49cd90f640 1 --2- 172.21.15.84:0/603190534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49c80cded0 0x7f49c80aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.159+0000 7f49cd90f640 1 -- 172.21.15.84:0/603190534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c815cd70 msgr2=0x7f49c80b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:04.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.159+0000 7f49cd90f640 1 --2- 172.21.15.84:0/603190534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c815cd70 0x7f49c80b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.159+0000 7f49cd90f640 1 -- 172.21.15.84:0/603190534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c40094d0 con 0x7f49c80b8480 2023-10-16T11:26:04.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.159+0000 7f49cd90f640 1 --2- 172.21.15.84:0/603190534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c80b8480 0x7f49c80cc0e0 secure :-1 s=READY pgs=3025 cs=0 l=1 rev1=1 crypto rx=0x7f49c4000c00 tx=0x7f49c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:04.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.159+0000 7f49be7fc640 1 -- 172.21.15.84:0/603190534 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c402f820 con 0x7f49c80b8480 2023-10-16T11:26:04.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.159+0000 7f49be7fc640 1 -- 172.21.15.84:0/603190534 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c4041400 con 0x7f49c80b8480 2023-10-16T11:26:04.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.159+0000 7f49be7fc640 1 -- 172.21.15.84:0/603190534 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c40417d0 con 0x7f49c80b8480 2023-10-16T11:26:04.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.159+0000 7f49ce911640 1 -- 172.21.15.84:0/603190534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c80ca210 con 0x7f49c80b8480 2023-10-16T11:26:04.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.160+0000 7f49ce911640 1 -- 172.21.15.84:0/603190534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c80ca7d0 con 0x7f49c80b8480 2023-10-16T11:26:04.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.161+0000 7f49ce911640 1 -- 172.21.15.84:0/603190534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c804e9b0 con 0x7f49c80b8480 2023-10-16T11:26:04.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.162+0000 7f49be7fc640 1 -- 172.21.15.84:0/603190534 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 144) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f49c4041930 con 0x7f49c80b8480 2023-10-16T11:26:04.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.163+0000 7f49be7fc640 1 --2- 172.21.15.84:0/603190534 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f49ac06b100 0x7f49ac06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:04.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.163+0000 7f49cd10e640 1 -- 172.21.15.84:0/603190534 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f49ac06b100 msgr2=0x7f49ac06d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2963529201 2023-10-16T11:26:04.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.163+0000 7f49be7fc640 1 -- 172.21.15.84:0/603190534 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f49c403c070 con 0x7f49c80b8480 2023-10-16T11:26:04.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.163+0000 7f49cd10e640 1 --2- 172.21.15.84:0/603190534 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f49ac06b100 0x7f49ac06d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:26:04.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.168+0000 7f49be7fc640 1 -- 172.21.15.84:0/603190534 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f49c4077710 con 0x7f49c80b8480 2023-10-16T11:26:04.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.359+0000 7f49ce911640 1 -- 172.21.15.84:0/603190534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f49c815c320 con 0x7f49c80b8480 2023-10-16T11:26:04.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.364+0000 7f49cd10e640 1 -- 172.21.15.84:0/603190534 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f49ac06b100 msgr2=0x7f49ac06d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2963529201 2023-10-16T11:26:04.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.364+0000 7f49cd10e640 1 --2- 172.21.15.84:0/603190534 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f49ac06b100 0x7f49ac06d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:26:04.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.366+0000 7f49be7fc640 1 -- 172.21.15.84:0/603190534 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v83)=0 v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7f49c407ad30 con 0x7f49c80b8480 2023-10-16T11:26:04.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.368+0000 7f49ce911640 1 -- 172.21.15.84:0/603190534 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f49ac06b100 msgr2=0x7f49ac06d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:26:04.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.368+0000 7f49ce911640 1 --2- 172.21.15.84:0/603190534 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f49ac06b100 0x7f49ac06d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.368+0000 7f49ce911640 1 -- 172.21.15.84:0/603190534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c80b8480 msgr2=0x7f49c80cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:04.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.368+0000 7f49ce911640 1 --2- 172.21.15.84:0/603190534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c80b8480 0x7f49c80cc0e0 secure :-1 s=READY pgs=3025 cs=0 l=1 rev1=1 crypto rx=0x7f49c4000c00 tx=0x7f49c4004730 comp rx=0 tx=0).stop 2023-10-16T11:26:04.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.369+0000 7f49ce911640 1 -- 172.21.15.84:0/603190534 shutdown_connections 2023-10-16T11:26:04.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.369+0000 7f49ce911640 1 --2- 172.21.15.84:0/603190534 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f49ac06b100 0x7f49ac06d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.369+0000 7f49ce911640 1 --2- 172.21.15.84:0/603190534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c815cd70 0x7f49c80b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.369+0000 7f49ce911640 1 --2- 172.21.15.84:0/603190534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49c80cded0 0x7f49c80aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.369+0000 7f49ce911640 1 --2- 172.21.15.84:0/603190534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c80b8480 0x7f49c80cc0e0 unknown :-1 s=CLOSED pgs=3025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.369+0000 7f49ce911640 1 -- 172.21.15.84:0/603190534 >> 172.21.15.84:0/603190534 conn(0x7f49c80cf570 msgr2=0x7f49c815d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:04.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.369+0000 7f49ce911640 1 -- 172.21.15.84:0/603190534 shutdown_connections 2023-10-16T11:26:04.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.369+0000 7f49ce911640 1 -- 172.21.15.84:0/603190534 wait complete. 2023-10-16T11:26:04.388 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T11:26:04.457 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:04.455+0000 7f4c2f753040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:26:04.458 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:04.457+0000 7f9c46a32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:26:04.511 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:04.509+0000 7f4c2f753040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:26:04.515 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:04.513+0000 7f9c46a32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:26:04.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.584+0000 7f8eb359e640 1 Processor -- start 2023-10-16T11:26:04.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.584+0000 7f8eb359e640 1 -- start start 2023-10-16T11:26:04.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.586+0000 7f8eb359e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8eb415aaa0 0x7f8eb40b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:04.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.586+0000 7f8eb359e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8eb4070240 0x7f8eb40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:04.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.586+0000 7f8eb359e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb40cad00 con 0x7f8eb40ca8a0 2023-10-16T11:26:04.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.586+0000 7f8eb359e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb4070c10 con 0x7f8eb415aaa0 2023-10-16T11:26:04.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.586+0000 7f8eb359e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb40b2d00 con 0x7f8eb4070240 2023-10-16T11:26:04.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.586+0000 7f8eb259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8eb415aaa0 0x7f8eb40b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:04.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.586+0000 7f8eb259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8eb415aaa0 0x7f8eb40b2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40692/0 (socket says 172.21.15.84:40692) 2023-10-16T11:26:04.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.586+0000 7f8eb259c640 1 -- 172.21.15.84:0/3098223539 learned_addr learned my addr 172.21.15.84:0/3098223539 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:04.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.587+0000 7f8eb259c640 1 -- 172.21.15.84:0/3098223539 >> v2:172.21.15.157:3300/0 conn(0x7f8eb4070240 msgr2=0x7f8eb40706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:04.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.587+0000 7f8eb259c640 1 --2- 172.21.15.84:0/3098223539 >> v2:172.21.15.157:3300/0 conn(0x7f8eb4070240 0x7f8eb40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.587+0000 7f8eb259c640 1 -- 172.21.15.84:0/3098223539 >> v1:172.21.15.157:6789/0 conn(0x7f8eb40ca8a0 legacy=0x7f8eb40b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:04.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.587+0000 7f8eb259c640 1 -- 172.21.15.84:0/3098223539 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb40b2fe0 con 0x7f8eb415aaa0 2023-10-16T11:26:04.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.588+0000 7f8eb259c640 1 --2- 172.21.15.84:0/3098223539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eb415aaa0 0x7f8eb40b2600 secure :-1 s=READY pgs=3026 cs=0 l=1 rev1=1 crypto rx=0x7f8eb40af1c0 tx=0x7f8ea402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e99ac33bb8b268b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:04.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.588+0000 7f8eb1d9b640 1 -- 172.21.15.84:0/3098223539 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea40317e0 con 0x7f8eb415aaa0 2023-10-16T11:26:04.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.588+0000 7f8eb1d9b640 1 -- 172.21.15.84:0/3098223539 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea4031940 con 0x7f8eb415aaa0 2023-10-16T11:26:04.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.588+0000 7f8eb1d9b640 1 -- 172.21.15.84:0/3098223539 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4031cd0 con 0x7f8eb415aaa0 2023-10-16T11:26:04.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.589+0000 7f8eb359e640 1 -- 172.21.15.84:0/3098223539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eb415aaa0 msgr2=0x7f8eb40b2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:04.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.589+0000 7f8eb359e640 1 --2- 172.21.15.84:0/3098223539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eb415aaa0 0x7f8eb40b2600 secure :-1 s=READY pgs=3026 cs=0 l=1 rev1=1 crypto rx=0x7f8eb40af1c0 tx=0x7f8ea402b880 comp rx=0 tx=0).stop 2023-10-16T11:26:04.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.589+0000 7f8eb359e640 1 -- 172.21.15.84:0/3098223539 shutdown_connections 2023-10-16T11:26:04.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.589+0000 7f8eb359e640 1 --2- 172.21.15.84:0/3098223539 >> v2:172.21.15.157:3300/0 conn(0x7f8eb4070240 0x7f8eb40706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.589+0000 7f8eb359e640 1 --2- 172.21.15.84:0/3098223539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eb415aaa0 0x7f8eb40b2600 unknown :-1 s=CLOSED pgs=3026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.589+0000 7f8eb359e640 1 -- 172.21.15.84:0/3098223539 >> 172.21.15.84:0/3098223539 conn(0x7f8eb40cc230 msgr2=0x7f8eb40cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:04.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.589+0000 7f8eb359e640 1 -- 172.21.15.84:0/3098223539 shutdown_connections 2023-10-16T11:26:04.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.589+0000 7f8eb359e640 1 -- 172.21.15.84:0/3098223539 wait complete. 2023-10-16T11:26:04.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.591+0000 7f8eb359e640 1 Processor -- start 2023-10-16T11:26:04.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.591+0000 7f8eb359e640 1 -- start start 2023-10-16T11:26:04.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.592+0000 7f8eb359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eb4070240 0x7f8eb4145780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:04.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.592+0000 7f8eb359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eb40ca8a0 0x7f8eb4148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:04.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.592+0000 7f8eb359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8eb415aaa0 0x7f8eb40c5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:04.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.592+0000 7f8eb359e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb41491f0 con 0x7f8eb40ca8a0 2023-10-16T11:26:04.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.592+0000 7f8eb359e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb4148c30 con 0x7f8eb4070240 2023-10-16T11:26:04.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.592+0000 7f8eb359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb4148f10 con 0x7f8eb415aaa0 2023-10-16T11:26:04.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.592+0000 7f8eabfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eb40ca8a0 0x7f8eb4148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:04.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.592+0000 7f8eabfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eb40ca8a0 0x7f8eb4148220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40706/0 (socket says 172.21.15.84:40706) 2023-10-16T11:26:04.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.592+0000 7f8eabfff640 1 -- 172.21.15.84:0/2424710564 learned_addr learned my addr 172.21.15.84:0/2424710564 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:04.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.592+0000 7f8eb2d9d640 1 --2- 172.21.15.84:0/2424710564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8eb415aaa0 0x7f8eb40c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:04.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.592+0000 7f8eb259c640 1 --2- 172.21.15.84:0/2424710564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eb4070240 0x7f8eb4145780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:04.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.592+0000 7f8eabfff640 1 -- 172.21.15.84:0/2424710564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8eb415aaa0 msgr2=0x7f8eb40c5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:04.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.592+0000 7f8eabfff640 1 --2- 172.21.15.84:0/2424710564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8eb415aaa0 0x7f8eb40c5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.592+0000 7f8eabfff640 1 -- 172.21.15.84:0/2424710564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eb4070240 msgr2=0x7f8eb4145780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:04.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.592+0000 7f8eabfff640 1 --2- 172.21.15.84:0/2424710564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eb4070240 0x7f8eb4145780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.592+0000 7f8eabfff640 1 -- 172.21.15.84:0/2424710564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea40094d0 con 0x7f8eb40ca8a0 2023-10-16T11:26:04.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.592+0000 7f8eb259c640 1 --2- 172.21.15.84:0/2424710564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eb4070240 0x7f8eb4145780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:04.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.593+0000 7f8eabfff640 1 --2- 172.21.15.84:0/2424710564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eb40ca8a0 0x7f8eb4148220 secure :-1 s=READY pgs=3027 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00b3e0 tx=0x7f8e9c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:04.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.593+0000 7f8eaaffd640 1 -- 172.21.15.84:0/2424710564 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c011830 con 0x7f8eb40ca8a0 2023-10-16T11:26:04.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.593+0000 7f8eaaffd640 1 -- 172.21.15.84:0/2424710564 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9c0119b0 con 0x7f8eb40ca8a0 2023-10-16T11:26:04.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.593+0000 7f8eaaffd640 1 -- 172.21.15.84:0/2424710564 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c011d60 con 0x7f8eb40ca8a0 2023-10-16T11:26:04.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.593+0000 7f8eb359e640 1 -- 172.21.15.84:0/2424710564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb4146330 con 0x7f8eb40ca8a0 2023-10-16T11:26:04.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.593+0000 7f8eb359e640 1 -- 172.21.15.84:0/2424710564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb40c9880 con 0x7f8eb40ca8a0 2023-10-16T11:26:04.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.594+0000 7f8eb359e640 1 -- 172.21.15.84:0/2424710564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb4146190 con 0x7f8eb40ca8a0 2023-10-16T11:26:04.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.595+0000 7f8eaaffd640 1 -- 172.21.15.84:0/2424710564 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 144) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f8e9c01a400 con 0x7f8eb40ca8a0 2023-10-16T11:26:04.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.596+0000 7f8eaaffd640 1 --2- 172.21.15.84:0/2424710564 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8e9406b100 0x7f8e9406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:04.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.596+0000 7f8eaaffd640 1 -- 172.21.15.84:0/2424710564 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7f8e9c089f90 con 0x7f8eb40ca8a0 2023-10-16T11:26:04.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.596+0000 7f8eb259c640 1 -- 172.21.15.84:0/2424710564 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8e9406b100 msgr2=0x7f8e9406d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2963529201 2023-10-16T11:26:04.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.596+0000 7f8eb259c640 1 --2- 172.21.15.84:0/2424710564 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8e9406b100 0x7f8e9406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:26:04.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.600+0000 7f8eaaffd640 1 -- 172.21.15.84:0/2424710564 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8e9c0bcfe0 con 0x7f8eb40ca8a0 2023-10-16T11:26:04.638 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:26:04.647 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:26:04.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.793+0000 7f8eb359e640 1 -- 172.21.15.84:0/2424710564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f8eb4146850 con 0x7f8eb40ca8a0 2023-10-16T11:26:04.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.797+0000 7f8eb259c640 1 -- 172.21.15.84:0/2424710564 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8e9406b100 msgr2=0x7f8e9406d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2963529201 2023-10-16T11:26:04.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.797+0000 7f8eb259c640 1 --2- 172.21.15.84:0/2424710564 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8e9406b100 0x7f8e9406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:26:04.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.797+0000 7f8eaaffd640 1 -- 172.21.15.84:0/2424710564 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v84)=0 v84) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8eb4146850 con 0x7f8eb40ca8a0 2023-10-16T11:26:04.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.800+0000 7f8eb359e640 1 -- 172.21.15.84:0/2424710564 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8e9406b100 msgr2=0x7f8e9406d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:26:04.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.800+0000 7f8eb359e640 1 --2- 172.21.15.84:0/2424710564 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8e9406b100 0x7f8e9406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.800+0000 7f8eb359e640 1 -- 172.21.15.84:0/2424710564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eb40ca8a0 msgr2=0x7f8eb4148220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:04.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.800+0000 7f8eb359e640 1 --2- 172.21.15.84:0/2424710564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eb40ca8a0 0x7f8eb4148220 secure :-1 s=READY pgs=3027 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00b3e0 tx=0x7f8e9c00b8a0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.800+0000 7f8eb359e640 1 -- 172.21.15.84:0/2424710564 shutdown_connections 2023-10-16T11:26:04.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.800+0000 7f8eb359e640 1 --2- 172.21.15.84:0/2424710564 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7f8e9406b100 0x7f8e9406d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.800+0000 7f8eb359e640 1 --2- 172.21.15.84:0/2424710564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8eb415aaa0 0x7f8eb40c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.800+0000 7f8eb359e640 1 --2- 172.21.15.84:0/2424710564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eb40ca8a0 0x7f8eb4148220 unknown :-1 s=CLOSED pgs=3027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.800+0000 7f8eb359e640 1 --2- 172.21.15.84:0/2424710564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eb4070240 0x7f8eb4145780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:04.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.800+0000 7f8eb359e640 1 -- 172.21.15.84:0/2424710564 >> 172.21.15.84:0/2424710564 conn(0x7f8eb40cc230 msgr2=0x7f8eb40ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:04.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.800+0000 7f8eb359e640 1 -- 172.21.15.84:0/2424710564 shutdown_connections 2023-10-16T11:26:04.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:04.800+0000 7f8eb359e640 1 -- 172.21.15.84:0/2424710564 wait complete. 2023-10-16T11:26:04.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:26:04.820 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p mandatory_fs_argument_test_dir) 2023-10-16T11:26:04.921 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add mandatory_fs_argument_test_dir 1M 2023-10-16T11:26:04.974 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:04.972+0000 7f4c2f753040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:26:04.988 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:04.986+0000 7f9c46a32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:26:05.026 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:05.024+0000 7f4c2f753040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:26:05.043 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:05.040+0000 7f9c46a32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:26:05.077 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:05.075+0000 7f4c2f753040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:26:05.095 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:05.093+0000 7f9c46a32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:26:05.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.112+0000 7fc358e1a640 1 Processor -- start 2023-10-16T11:26:05.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.112+0000 7fc358e1a640 1 -- start start 2023-10-16T11:26:05.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.114+0000 7fc358e1a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc3540cfe90 0x7fc3540c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:05.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.114+0000 7fc358e1a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3540ca850 0x7fc35415d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:05.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.114+0000 7fc358e1a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc35415a9b0 con 0x7fc354157740 2023-10-16T11:26:05.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.114+0000 7fc358e1a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3540ca420 con 0x7fc3540cfe90 2023-10-16T11:26:05.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.114+0000 7fc358e1a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35415a360 con 0x7fc3540ca850 2023-10-16T11:26:05.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.114+0000 7fc3537fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc354157740 0x7fc3540cf780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40596/0 (socket says 172.21.15.84:40596) 2023-10-16T11:26:05.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.114+0000 7fc3537fe640 1 -- 172.21.15.84:0/281995144 learned_addr learned my addr 172.21.15.84:0/281995144 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:05.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.114+0000 7fc34bfff640 1 --2- 172.21.15.84:0/281995144 >> v2:172.21.15.157:3300/0 conn(0x7fc3540cfe90 0x7fc3540c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:05.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.114+0000 7fc352ffd640 1 -- 172.21.15.84:0/281995144 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777627405 0 0) 0x7fc35415a9b0 con 0x7fc354157740 2023-10-16T11:26:05.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.114+0000 7fc353fff640 1 --2- 172.21.15.84:0/281995144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3540ca850 0x7fc35415d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:05.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.114+0000 7fc352ffd640 1 -- 172.21.15.84:0/281995144 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc340003520 con 0x7fc354157740 2023-10-16T11:26:05.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.115+0000 7fc352ffd640 1 -- 172.21.15.84:0/281995144 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 573469227 0 0) 0x7fc340003520 con 0x7fc354157740 2023-10-16T11:26:05.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.115+0000 7fc352ffd640 1 -- 172.21.15.84:0/281995144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3540ca850 msgr2=0x7fc35415d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:05.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.115+0000 7fc352ffd640 1 --2- 172.21.15.84:0/281995144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3540ca850 0x7fc35415d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:05.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.115+0000 7fc352ffd640 1 -- 172.21.15.84:0/281995144 >> v2:172.21.15.157:3300/0 conn(0x7fc3540cfe90 msgr2=0x7fc3540c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:05.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.115+0000 7fc352ffd640 1 --2- 172.21.15.84:0/281995144 >> v2:172.21.15.157:3300/0 conn(0x7fc3540cfe90 0x7fc3540c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:05.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.115+0000 7fc352ffd640 1 -- 172.21.15.84:0/281995144 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35415a640 con 0x7fc354157740 2023-10-16T11:26:05.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.115+0000 7fc352ffd640 1 -- 172.21.15.84:0/281995144 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc344001670 con 0x7fc354157740 2023-10-16T11:26:05.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.115+0000 7fc352ffd640 1 -- 172.21.15.84:0/281995144 >> v1:172.21.15.84:6789/0 conn(0x7fc354157740 legacy=0x7fc3540cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:05.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.115+0000 7fc352ffd640 1 --2- 172.21.15.84:0/281995144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc340003970 0x7fc340003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:05.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.115+0000 7fc352ffd640 1 --2- 172.21.15.84:0/281995144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3400047c0 0x7fc340004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:05.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.115+0000 7fc352ffd640 1 --2- 172.21.15.84:0/281995144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3540ca850 0x7fc340005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:05.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.115+0000 7fc352ffd640 1 -- 172.21.15.84:0/281995144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3540ca420 con 0x7fc3400047c0 2023-10-16T11:26:05.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.115+0000 7fc352ffd640 1 -- 172.21.15.84:0/281995144 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35415a360 con 0x7fc340003970 2023-10-16T11:26:05.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.115+0000 7fc352ffd640 1 -- 172.21.15.84:0/281995144 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34000b980 con 0x7fc3540ca850 2023-10-16T11:26:05.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.115+0000 7fc34bfff640 1 --2- 172.21.15.84:0/281995144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3400047c0 0x7fc340004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:05.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.116+0000 7fc3537fe640 1 --2- 172.21.15.84:0/281995144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc340003970 0x7fc340003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:05.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.116+0000 7fc34bfff640 1 -- 172.21.15.84:0/281995144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3540ca850 msgr2=0x7fc340005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:05.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.116+0000 7fc34bfff640 1 --2- 172.21.15.84:0/281995144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3540ca850 0x7fc340005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:05.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.116+0000 7fc34bfff640 1 -- 172.21.15.84:0/281995144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc340003970 msgr2=0x7fc340003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:05.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.116+0000 7fc34bfff640 1 --2- 172.21.15.84:0/281995144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc340003970 0x7fc340003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:05.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.116+0000 7fc34bfff640 1 -- 172.21.15.84:0/281995144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc340003520 con 0x7fc3400047c0 2023-10-16T11:26:05.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.116+0000 7fc3537fe640 1 --2- 172.21.15.84:0/281995144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc340003970 0x7fc340003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:05.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.117+0000 7fc34bfff640 1 --2- 172.21.15.84:0/281995144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3400047c0 0x7fc340004c40 secure :-1 s=READY pgs=3029 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00bd60 tx=0x7fc3400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9efa7e68234dbc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:05.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.117+0000 7fc352ffd640 1 -- 172.21.15.84:0/281995144 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c030700 con 0x7fc3400047c0 2023-10-16T11:26:05.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.117+0000 7fc352ffd640 1 -- 172.21.15.84:0/281995144 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33c030860 con 0x7fc3400047c0 2023-10-16T11:26:05.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.117+0000 7fc352ffd640 1 -- 172.21.15.84:0/281995144 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c030bf0 con 0x7fc3400047c0 2023-10-16T11:26:05.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.118+0000 7fc358e1a640 1 -- 172.21.15.84:0/281995144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3400047c0 msgr2=0x7fc340004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:05.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.118+0000 7fc358e1a640 1 --2- 172.21.15.84:0/281995144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3400047c0 0x7fc340004c40 secure :-1 s=READY pgs=3029 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00bd60 tx=0x7fc3400057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:05.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.118+0000 7fc353fff640 1 -- 172.21.15.84:0/281995144 reap_dead start 2023-10-16T11:26:05.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.118+0000 7fc358e1a640 1 -- 172.21.15.84:0/281995144 shutdown_connections 2023-10-16T11:26:05.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.118+0000 7fc358e1a640 1 -- 172.21.15.84:0/281995144 >> 172.21.15.84:0/281995144 conn(0x7fc35406aeb0 msgr2=0x7fc35406b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:05.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.118+0000 7fc358e1a640 1 -- 172.21.15.84:0/281995144 shutdown_connections 2023-10-16T11:26:05.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.118+0000 7fc358e1a640 1 -- 172.21.15.84:0/281995144 wait complete. 2023-10-16T11:26:05.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.120+0000 7fc358e1a640 1 Processor -- start 2023-10-16T11:26:05.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.120+0000 7fc358e1a640 1 -- start start 2023-10-16T11:26:05.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.121+0000 7fc358e1a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3400047c0 0x7fc3540b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:05.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.121+0000 7fc358e1a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3541434e0 0x7fc354158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:05.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.121+0000 7fc358e1a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3540b3030 0x7fc3541590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:05.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.121+0000 7fc358e1a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3540b3570 con 0x7fc3540b3030 2023-10-16T11:26:05.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.121+0000 7fc358e1a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3540b3850 con 0x7fc3400047c0 2023-10-16T11:26:05.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.121+0000 7fc358e1a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3540b3b30 con 0x7fc3541434e0 2023-10-16T11:26:05.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.122+0000 7fc353fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3540b3030 0x7fc3541590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:05.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.122+0000 7fc353fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3540b3030 0x7fc3541590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40722/0 (socket says 172.21.15.84:40722) 2023-10-16T11:26:05.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.122+0000 7fc353fff640 1 -- 172.21.15.84:0/2947934179 learned_addr learned my addr 172.21.15.84:0/2947934179 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:05.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.122+0000 7fc34bfff640 1 --2- 172.21.15.84:0/2947934179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3541434e0 0x7fc354158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:05.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.122+0000 7fc3537fe640 1 --2- 172.21.15.84:0/2947934179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3400047c0 0x7fc3540b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:05.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.122+0000 7fc353fff640 1 -- 172.21.15.84:0/2947934179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3541434e0 msgr2=0x7fc354158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:05.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.122+0000 7fc353fff640 1 --2- 172.21.15.84:0/2947934179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3541434e0 0x7fc354158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:05.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.122+0000 7fc353fff640 1 -- 172.21.15.84:0/2947934179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3400047c0 msgr2=0x7fc3540b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:05.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.122+0000 7fc353fff640 1 --2- 172.21.15.84:0/2947934179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3400047c0 0x7fc3540b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:05.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.122+0000 7fc353fff640 1 -- 172.21.15.84:0/2947934179 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3540b3e10 con 0x7fc3540b3030 2023-10-16T11:26:05.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.122+0000 7fc3537fe640 1 --2- 172.21.15.84:0/2947934179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3400047c0 0x7fc3540b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:05.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.122+0000 7fc34bfff640 1 --2- 172.21.15.84:0/2947934179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3541434e0 0x7fc354158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:05.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.122+0000 7fc353fff640 1 --2- 172.21.15.84:0/2947934179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3540b3030 0x7fc3541590b0 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 crypto rx=0x7fc3540b5240 tx=0x7fc34c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:05.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.123+0000 7fc350ff9640 1 -- 172.21.15.84:0/2947934179 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c010040 con 0x7fc3540b3030 2023-10-16T11:26:05.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.123+0000 7fc350ff9640 1 -- 172.21.15.84:0/2947934179 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c00bab0 con 0x7fc3540b3030 2023-10-16T11:26:05.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.123+0000 7fc358e1a640 1 -- 172.21.15.84:0/2947934179 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3540b40f0 con 0x7fc3540b3030 2023-10-16T11:26:05.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.123+0000 7fc350ff9640 1 -- 172.21.15.84:0/2947934179 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c00be60 con 0x7fc3540b3030 2023-10-16T11:26:05.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.123+0000 7fc358e1a640 1 -- 172.21.15.84:0/2947934179 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3541538f0 con 0x7fc3540b3030 2023-10-16T11:26:05.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.124+0000 7fc358e1a640 1 -- 172.21.15.84:0/2947934179 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3540722b0 con 0x7fc3540b3030 2023-10-16T11:26:05.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.125+0000 7fc350ff9640 1 -- 172.21.15.84:0/2947934179 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 144) v1 ==== 89450+0+0 (secure 0 0 0) 0x7fc34c01e410 con 0x7fc3540b3030 2023-10-16T11:26:05.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.126+0000 7fc350ff9640 1 --2- 172.21.15.84:0/2947934179 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc33406b100 0x7fc33406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:05.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.126+0000 7fc350ff9640 1 -- 172.21.15.84:0/2947934179 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6601+0+0 (secure 0 0 0) 0x7fc34c08a110 con 0x7fc3540b3030 2023-10-16T11:26:05.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.126+0000 7fc3537fe640 1 -- 172.21.15.84:0/2947934179 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc33406b100 msgr2=0x7fc33406d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2963529201 2023-10-16T11:26:05.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.126+0000 7fc3537fe640 1 --2- 172.21.15.84:0/2947934179 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc33406b100 0x7fc33406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:26:05.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.130+0000 7fc350ff9640 1 -- 172.21.15.84:0/2947934179 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc34c054940 con 0x7fc3540b3030 2023-10-16T11:26:05.139 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:05.138+0000 7f4c2f753040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:26:05.161 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:05.160+0000 7f9c46a32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:26:05.188 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:05.186+0000 7f4c2f753040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:26:05.213 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:05.211+0000 7f9c46a32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:26:05.298 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:05.296+0000 7f4c2f753040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:26:05.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.327+0000 7fc3537fe640 1 -- 172.21.15.84:0/2947934179 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc33406b100 msgr2=0x7fc33406d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2963529201 2023-10-16T11:26:05.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.327+0000 7fc3537fe640 1 --2- 172.21.15.84:0/2947934179 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc33406b100 0x7fc33406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:26:05.329 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:05.327+0000 7f9c46a32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:26:05.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.345+0000 7fc358e1a640 1 -- 172.21.15.84:0/2947934179 --> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3540b39c0 con 0x7fc33406b100 2023-10-16T11:26:05.360 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:05.359+0000 7f4c2f753040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:26:05.392 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:05.391+0000 7f9c46a32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:26:05.410 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:05.408+0000 7f4c2f753040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:26:05.443 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:05.441+0000 7f9c46a32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:26:05.467 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:05.465+0000 7f4c2f753040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:26:05.500 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:05.498+0000 7f9c46a32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:26:05.519 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:05.515+0000 7f4c2f753040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:26:05.551 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:05.549+0000 7f9c46a32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:26:05.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.728+0000 7fc3537fe640 1 -- 172.21.15.84:0/2947934179 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc33406b100 msgr2=0x7fc33406d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2963529201 2023-10-16T11:26:05.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:05.728+0000 7fc3537fe640 1 --2- 172.21.15.84:0/2947934179 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc33406b100 0x7fc33406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T11:26:05.896 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:05.894+0000 7f4c2f753040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:26:05.932 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:05.930+0000 7f9c46a32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:26:06.055 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:06.053+0000 7f4c2f753040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:26:06.089 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:06.088+0000 7f9c46a32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:26:06.163 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:06.161+0000 7f4c2f753040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:26:06.197 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:06.196+0000 7f9c46a32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:26:06.213 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:06.211+0000 7f4c2f753040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:26:06.246 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:06.245+0000 7f9c46a32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:26:06.324 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:06.322+0000 7f4c2f753040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:26:06.359 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:06.358+0000 7f9c46a32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:26:06.431 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:06.429+0000 7f4c2f753040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:26:06.467 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:06.466+0000 7f9c46a32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:26:06.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:06.530+0000 7fc3537fe640 1 -- 172.21.15.84:0/2947934179 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc33406b100 msgr2=0x7fc33406d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2963529201 2023-10-16T11:26:06.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:06.530+0000 7fc3537fe640 1 --2- 172.21.15.84:0/2947934179 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc33406b100 0x7fc33406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T11:26:06.588 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:06.586+0000 7f4c2f753040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:26:06.624 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:06.623+0000 7f9c46a32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:26:06.636 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:06.635+0000 7f4c2f753040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:26:06.673 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:06.672+0000 7f9c46a32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:26:06.755 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:06.753+0000 7f4c2f753040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:26:06.794 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:06.793+0000 7f9c46a32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:26:07.571 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:07.569+0000 7f4c2f753040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:26:07.613 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:07.612+0000 7f9c46a32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:26:07.865 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:07.863+0000 7f4c2f753040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:26:07.907 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:07.905+0000 7f9c46a32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:26:07.922 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:07.920+0000 7f4c2f753040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:26:07.963 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:07.962+0000 7f9c46a32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:26:08.036 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:08.034+0000 7f4c2f753040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:26:08.076 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.075+0000 7f9c46a32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:26:08.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:08.091+0000 7fc350ff9640 1 -- 172.21.15.84:0/2947934179 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 145) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fc34c054d10 con 0x7fc3540b3030 2023-10-16T11:26:08.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:08.092+0000 7fc350ff9640 1 -- 172.21.15.84:0/2947934179 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc33406b100 msgr2=0x7fc33406d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:26:08.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:08.092+0000 7fc350ff9640 1 --2- 172.21.15.84:0/2947934179 >> [v2:172.21.15.84:6832/2963529201,v1:172.21.15.84:6833/2963529201] conn(0x7fc33406b100 0x7fc33406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:08.693 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.692+0000 7f9bc2577640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.694 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.692+0000 7f9bc2577640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.694 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.692+0000 7f9bc2577640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.694 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.692+0000 7f9bc2577640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.694 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.692+0000 7f9bc2577640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.694 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.692+0000 7f9bc557d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.695 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.692+0000 7f9bc557d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.695 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.692+0000 7f9bc557d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.695 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.692+0000 7f9bc557d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.695 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.692+0000 7f9bc557d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.710 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.709+0000 7f9bc2577640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.710 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.709+0000 7f9bc2577640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.710 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.709+0000 7f9bc2577640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.711 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.709+0000 7f9bc2577640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.711 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.709+0000 7f9bc2577640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.711 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.709+0000 7f9bc557d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.712 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.710+0000 7f9bc557d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.712 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.710+0000 7f9bc557d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.712 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.710+0000 7f9bc557d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:08.712 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:08.710+0000 7f9bc557d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:09.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.120+0000 7fc350ff9640 1 -- 172.21.15.84:0/2947934179 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 146) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fc34c076e60 con 0x7fc3540b3030 2023-10-16T11:26:09.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.120+0000 7fc350ff9640 1 --2- 172.21.15.84:0/2947934179 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7fc334035800 0x7fc334077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:09.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.120+0000 7fc350ff9640 1 -- 172.21.15.84:0/2947934179 --> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3540b39c0 con 0x7fc334035800 2023-10-16T11:26:09.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.127+0000 7fc3537fe640 1 --2- 172.21.15.84:0/2947934179 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7fc334035800 0x7fc334077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:09.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.134+0000 7fc3537fe640 1 --2- 172.21.15.84:0/2947934179 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7fc334035800 0x7fc334077650 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc344001200 tx=0x7fc344009040 comp rx=0 tx=0).ready entity=mgr.12537 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:09.141 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:09.140+0000 7f9bcc5cb640 -1 mgr.server reply reply (22) Invalid argument filesystem argument is required when there is more than one file system 2023-10-16T11:26:09.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.140+0000 7fc350ff9640 1 -- 172.21.15.84:0/2947934179 <== mgr.12537 v2:172.21.15.84:6832/3938045451 1 ==== mgr_command_reply(tid 0: -22 filesystem argument is required when there is more than one file system) v1 ==== 79+0+0 (secure 0 0 0) 0x7fc3540b39c0 con 0x7fc334035800 2023-10-16T11:26:09.142 INFO:teuthology.orchestra.run.smithi084.stderr:Error EINVAL: filesystem argument is required when there is more than one file system 2023-10-16T11:26:09.162 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-10-16T11:26:09.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:26:09.164 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir mandatory_fs_argument_test_dir) 2023-10-16T11:26:09.218 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T11:26:09.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.420+0000 7f6561ca9640 1 Processor -- start 2023-10-16T11:26:09.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.420+0000 7f6561ca9640 1 -- start start 2023-10-16T11:26:09.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.421+0000 7f6561ca9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f655c15ad70 0x7f655c069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:09.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.421+0000 7f6561ca9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f655c0c9ca0 0x7f655c06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:09.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.422+0000 7f6561ca9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655c0cb030 con 0x7f655c0ca3b0 2023-10-16T11:26:09.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.422+0000 7f6561ca9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655c0c91c0 con 0x7f655c0c9ca0 2023-10-16T11:26:09.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.422+0000 7f6561ca9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655c0c94a0 con 0x7f655c15ad70 2023-10-16T11:26:09.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.422+0000 7f655affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f655c0c9ca0 0x7f655c06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:09.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.422+0000 7f655affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f655c0c9ca0 0x7f655c06a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53750/0 (socket says 172.21.15.84:53750) 2023-10-16T11:26:09.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.422+0000 7f655affd640 1 -- 172.21.15.84:0/2896946012 learned_addr learned my addr 172.21.15.84:0/2896946012 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:09.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.422+0000 7f655b7fe640 1 --2- 172.21.15.84:0/2896946012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f655c15ad70 0x7f655c069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:09.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.422+0000 7f655affd640 1 -- 172.21.15.84:0/2896946012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f655c15ad70 msgr2=0x7f655c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:09.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.422+0000 7f655affd640 1 --2- 172.21.15.84:0/2896946012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f655c15ad70 0x7f655c069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:09.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.422+0000 7f655affd640 1 -- 172.21.15.84:0/2896946012 >> v1:172.21.15.157:6789/0 conn(0x7f655c0ca3b0 legacy=0x7f655c0ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:09.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.422+0000 7f655affd640 1 -- 172.21.15.84:0/2896946012 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c0c9780 con 0x7f655c0c9ca0 2023-10-16T11:26:09.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.422+0000 7f655b7fe640 1 --2- 172.21.15.84:0/2896946012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f655c15ad70 0x7f655c069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:09.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.423+0000 7f655affd640 1 --2- 172.21.15.84:0/2896946012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f655c0c9ca0 0x7f655c06a2e0 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 crypto rx=0x7f6544009870 tx=0x7f654402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1855fbf888cb59f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:09.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.424+0000 7f655a7fc640 1 -- 172.21.15.84:0/2896946012 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544031640 con 0x7f655c0c9ca0 2023-10-16T11:26:09.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.424+0000 7f655a7fc640 1 -- 172.21.15.84:0/2896946012 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65440317a0 con 0x7f655c0c9ca0 2023-10-16T11:26:09.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.424+0000 7f655a7fc640 1 -- 172.21.15.84:0/2896946012 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544031b30 con 0x7f655c0c9ca0 2023-10-16T11:26:09.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.424+0000 7f6561ca9640 1 -- 172.21.15.84:0/2896946012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f655c0c9ca0 msgr2=0x7f655c06a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:09.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.424+0000 7f6561ca9640 1 --2- 172.21.15.84:0/2896946012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f655c0c9ca0 0x7f655c06a2e0 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 crypto rx=0x7f6544009870 tx=0x7f654402b8f0 comp rx=0 tx=0).stop 2023-10-16T11:26:09.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.425+0000 7f6561ca9640 1 -- 172.21.15.84:0/2896946012 shutdown_connections 2023-10-16T11:26:09.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.425+0000 7f6561ca9640 1 --2- 172.21.15.84:0/2896946012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f655c0c9ca0 0x7f655c06a2e0 unknown :-1 s=CLOSED pgs=3047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:09.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.425+0000 7f6561ca9640 1 --2- 172.21.15.84:0/2896946012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f655c15ad70 0x7f655c069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:09.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.425+0000 7f6561ca9640 1 -- 172.21.15.84:0/2896946012 >> 172.21.15.84:0/2896946012 conn(0x7f655c0ce9b0 msgr2=0x7f655c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:09.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.425+0000 7f6561ca9640 1 -- 172.21.15.84:0/2896946012 shutdown_connections 2023-10-16T11:26:09.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.425+0000 7f6561ca9640 1 -- 172.21.15.84:0/2896946012 wait complete. 2023-10-16T11:26:09.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.426+0000 7f6561ca9640 1 Processor -- start 2023-10-16T11:26:09.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.427+0000 7f6561ca9640 1 -- start start 2023-10-16T11:26:09.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.427+0000 7f6561ca9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f655c0c9ca0 0x7f655c1f09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:09.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.427+0000 7f6561ca9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f655c0ca3b0 0x7f655c158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:09.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.427+0000 7f6561ca9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f655c15ad70 0x7f655c0a9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:09.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.427+0000 7f6561ca9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c0c8500 con 0x7f655c0ca3b0 2023-10-16T11:26:09.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.427+0000 7f6561ca9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c0a8520 con 0x7f655c0c9ca0 2023-10-16T11:26:09.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.427+0000 7f6561ca9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655c0a8800 con 0x7f655c15ad70 2023-10-16T11:26:09.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.428+0000 7f655affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f655c0ca3b0 0x7f655c158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:09.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.428+0000 7f655affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f655c0ca3b0 0x7f655c158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53766/0 (socket says 172.21.15.84:53766) 2023-10-16T11:26:09.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.428+0000 7f655affd640 1 -- 172.21.15.84:0/574808264 learned_addr learned my addr 172.21.15.84:0/574808264 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:09.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.428+0000 7f655b7fe640 1 --2- 172.21.15.84:0/574808264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f655c0c9ca0 0x7f655c1f09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:09.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.428+0000 7f655bfff640 1 --2- 172.21.15.84:0/574808264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f655c15ad70 0x7f655c0a9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:09.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.428+0000 7f655affd640 1 -- 172.21.15.84:0/574808264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f655c15ad70 msgr2=0x7f655c0a9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:09.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.428+0000 7f655affd640 1 --2- 172.21.15.84:0/574808264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f655c15ad70 0x7f655c0a9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:09.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.428+0000 7f655affd640 1 -- 172.21.15.84:0/574808264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f655c0c9ca0 msgr2=0x7f655c1f09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:09.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.428+0000 7f655affd640 1 --2- 172.21.15.84:0/574808264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f655c0c9ca0 0x7f655c1f09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:09.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.428+0000 7f655affd640 1 -- 172.21.15.84:0/574808264 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65440094d0 con 0x7f655c0ca3b0 2023-10-16T11:26:09.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.429+0000 7f655affd640 1 --2- 172.21.15.84:0/574808264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f655c0ca3b0 0x7f655c158f80 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7f6544000c00 tx=0x7f65440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:09.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.429+0000 7f6553fff640 1 -- 172.21.15.84:0/574808264 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654403ac60 con 0x7f655c0ca3b0 2023-10-16T11:26:09.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.429+0000 7f6553fff640 1 -- 172.21.15.84:0/574808264 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654403ade0 con 0x7f655c0ca3b0 2023-10-16T11:26:09.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.429+0000 7f6561ca9640 1 -- 172.21.15.84:0/574808264 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655c1594c0 con 0x7f655c0ca3b0 2023-10-16T11:26:09.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.429+0000 7f6553fff640 1 -- 172.21.15.84:0/574808264 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544044630 con 0x7f655c0ca3b0 2023-10-16T11:26:09.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.429+0000 7f6561ca9640 1 -- 172.21.15.84:0/574808264 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655c0b3890 con 0x7f655c0ca3b0 2023-10-16T11:26:09.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.430+0000 7f6561ca9640 1 -- 172.21.15.84:0/574808264 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655c0a89c0 con 0x7f655c0ca3b0 2023-10-16T11:26:09.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.431+0000 7f6553fff640 1 -- 172.21.15.84:0/574808264 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 146) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f654402f050 con 0x7f655c0ca3b0 2023-10-16T11:26:09.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.432+0000 7f6553fff640 1 --2- 172.21.15.84:0/574808264 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f653c06ad10 0x7f653c06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:09.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.432+0000 7f6553fff640 1 -- 172.21.15.84:0/574808264 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7031+0+0 (secure 0 0 0) 0x7f65440ab9d0 con 0x7f655c0ca3b0 2023-10-16T11:26:09.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.433+0000 7f655b7fe640 1 --2- 172.21.15.84:0/574808264 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f653c06ad10 0x7f653c06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:09.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.433+0000 7f655b7fe640 1 --2- 172.21.15.84:0/574808264 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f653c06ad10 0x7f653c06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f654c00a950 tx=0x7f654c008040 comp rx=0 tx=0).ready entity=mgr.12537 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:09.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.437+0000 7f6553fff640 1 -- 172.21.15.84:0/574808264 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f65440768e0 con 0x7f655c0ca3b0 2023-10-16T11:26:09.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:09.680+0000 7f6561ca9640 1 -- 172.21.15.84:0/574808264 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f655c04e160 con 0x7f655c0ca3b0 2023-10-16T11:26:10.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.146+0000 7f6553fff640 1 -- 172.21.15.84:0/574808264 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v147) v1 ==== 93+0+0 (secure 0 0 0) 0x7f6544079f00 con 0x7f655c0ca3b0 2023-10-16T11:26:10.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.151+0000 7f6561ca9640 1 -- 172.21.15.84:0/574808264 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f653c06ad10 msgr2=0x7f653c06d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:10.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.151+0000 7f6561ca9640 1 --2- 172.21.15.84:0/574808264 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f653c06ad10 0x7f653c06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f654c00a950 tx=0x7f654c008040 comp rx=0 tx=0).stop 2023-10-16T11:26:10.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.151+0000 7f6561ca9640 1 -- 172.21.15.84:0/574808264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f655c0ca3b0 msgr2=0x7f655c158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:10.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.151+0000 7f6561ca9640 1 --2- 172.21.15.84:0/574808264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f655c0ca3b0 0x7f655c158f80 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7f6544000c00 tx=0x7f65440046c0 comp rx=0 tx=0).stop 2023-10-16T11:26:10.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.152+0000 7f6561ca9640 1 -- 172.21.15.84:0/574808264 shutdown_connections 2023-10-16T11:26:10.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.152+0000 7f6561ca9640 1 --2- 172.21.15.84:0/574808264 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f653c06ad10 0x7f653c06d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:10.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.152+0000 7f6561ca9640 1 --2- 172.21.15.84:0/574808264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f655c15ad70 0x7f655c0a9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:10.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.152+0000 7f6561ca9640 1 --2- 172.21.15.84:0/574808264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f655c0ca3b0 0x7f655c158f80 unknown :-1 s=CLOSED pgs=3048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:10.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.152+0000 7f6561ca9640 1 --2- 172.21.15.84:0/574808264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f655c0c9ca0 0x7f655c1f09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:10.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.152+0000 7f6561ca9640 1 -- 172.21.15.84:0/574808264 >> 172.21.15.84:0/574808264 conn(0x7f655c0ce9b0 msgr2=0x7f655c15cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:10.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.154+0000 7f6561ca9640 1 -- 172.21.15.84:0/574808264 shutdown_connections 2023-10-16T11:26:10.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.154+0000 7f6561ca9640 1 -- 172.21.15.84:0/574808264 wait complete. 2023-10-16T11:26:10.177 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:26:10.177 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:26:10.279 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:26:10.382 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:26:10.382 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:26:10.383 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:26:10.434 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:26:10.435 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:26:10.525 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:26:10.678 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:10.677+0000 7f2b48613040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:26:10.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.723+0000 7f262b40e640 1 Processor -- start 2023-10-16T11:26:10.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.724+0000 7f262b40e640 1 -- start start 2023-10-16T11:26:10.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.725+0000 7f262b40e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2624159f00 0x7f26240b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:10.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.725+0000 7f262b40e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f262415d880 0x7f26240b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:10.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.725+0000 7f262b40e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f262406f7f0 con 0x7f2624159b20 2023-10-16T11:26:10.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.725+0000 7f262b40e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f262406c5e0 con 0x7f262415d880 2023-10-16T11:26:10.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.725+0000 7f262b40e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f262406c8c0 con 0x7f2624159f00 2023-10-16T11:26:10.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.726+0000 7f26291aa640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2624159b20 0x7f262406bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:46958/0 (socket says 172.21.15.84:46958) 2023-10-16T11:26:10.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.726+0000 7f26291aa640 1 -- 172.21.15.84:0/1726908843 learned_addr learned my addr 172.21.15.84:0/1726908843 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:10.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.726+0000 7f26289a9640 1 --2- 172.21.15.84:0/1726908843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2624159f00 0x7f26240b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:10.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.726+0000 7f26299ab640 1 --2- 172.21.15.84:0/1726908843 >> v2:172.21.15.157:3300/0 conn(0x7f262415d880 0x7f26240b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:10.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.726+0000 7f26299ab640 1 -- 172.21.15.84:0/1726908843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2624159f00 msgr2=0x7f26240b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:10.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.726+0000 7f26299ab640 1 --2- 172.21.15.84:0/1726908843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2624159f00 0x7f26240b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:10.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.726+0000 7f26299ab640 1 -- 172.21.15.84:0/1726908843 >> v1:172.21.15.157:6789/0 conn(0x7f2624159b20 legacy=0x7f262406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:10.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.726+0000 7f26299ab640 1 -- 172.21.15.84:0/1726908843 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262406cba0 con 0x7f262415d880 2023-10-16T11:26:10.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.727+0000 7f26299ab640 1 --2- 172.21.15.84:0/1726908843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262415d880 0x7f26240b0950 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f261c009d90 tx=0x7f261c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b404605246919a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:10.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.728+0000 7f261bfff640 1 -- 172.21.15.84:0/1726908843 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c02cc50 con 0x7f262415d880 2023-10-16T11:26:10.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.728+0000 7f261bfff640 1 -- 172.21.15.84:0/1726908843 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c02cdb0 con 0x7f262415d880 2023-10-16T11:26:10.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.728+0000 7f261bfff640 1 -- 172.21.15.84:0/1726908843 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c034660 con 0x7f262415d880 2023-10-16T11:26:10.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.729+0000 7f262b40e640 1 -- 172.21.15.84:0/1726908843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262415d880 msgr2=0x7f26240b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:10.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.729+0000 7f262b40e640 1 --2- 172.21.15.84:0/1726908843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262415d880 0x7f26240b0950 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f261c009d90 tx=0x7f261c02bea0 comp rx=0 tx=0).stop 2023-10-16T11:26:10.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.729+0000 7f262b40e640 1 -- 172.21.15.84:0/1726908843 shutdown_connections 2023-10-16T11:26:10.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.729+0000 7f262b40e640 1 --2- 172.21.15.84:0/1726908843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262415d880 0x7f26240b0950 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:10.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.729+0000 7f262b40e640 1 --2- 172.21.15.84:0/1726908843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2624159f00 0x7f26240b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:10.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.729+0000 7f262b40e640 1 -- 172.21.15.84:0/1726908843 >> 172.21.15.84:0/1726908843 conn(0x7f26240cb410 msgr2=0x7f26240cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:10.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.729+0000 7f262b40e640 1 -- 172.21.15.84:0/1726908843 shutdown_connections 2023-10-16T11:26:10.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.729+0000 7f262b40e640 1 -- 172.21.15.84:0/1726908843 wait complete. 2023-10-16T11:26:10.733 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:10.730+0000 7fa00ca2c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:26:10.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.730+0000 7f262b40e640 1 Processor -- start 2023-10-16T11:26:10.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.731+0000 7f262b40e640 1 -- start start 2023-10-16T11:26:10.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.731+0000 7f262b40e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2624159b20 0x7f26240afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:10.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.731+0000 7f262b40e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2624159f00 0x7f26240aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:10.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.731+0000 7f262b40e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f262415d880 0x7f2624155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:10.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.731+0000 7f262b40e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26240c5a60 con 0x7f2624159b20 2023-10-16T11:26:10.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.731+0000 7f262b40e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26240ae720 con 0x7f2624159f00 2023-10-16T11:26:10.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.731+0000 7f262b40e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26240aea00 con 0x7f262415d880 2023-10-16T11:26:10.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.732+0000 7f26291aa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2624159b20 0x7f26240afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:10.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.732+0000 7f26291aa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2624159b20 0x7f26240afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53794/0 (socket says 172.21.15.84:53794) 2023-10-16T11:26:10.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.732+0000 7f26291aa640 1 -- 172.21.15.84:0/1475999557 learned_addr learned my addr 172.21.15.84:0/1475999557 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:10.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.732+0000 7f26289a9640 1 --2- 172.21.15.84:0/1475999557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2624159f00 0x7f26240aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:10.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.732+0000 7f26291aa640 1 -- 172.21.15.84:0/1475999557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f262415d880 msgr2=0x7f2624155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:10.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.732+0000 7f26299ab640 1 --2- 172.21.15.84:0/1475999557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f262415d880 0x7f2624155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:10.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.732+0000 7f26291aa640 1 --2- 172.21.15.84:0/1475999557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f262415d880 0x7f2624155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:10.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.732+0000 7f26291aa640 1 -- 172.21.15.84:0/1475999557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2624159f00 msgr2=0x7f26240aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:10.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.732+0000 7f26291aa640 1 --2- 172.21.15.84:0/1475999557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2624159f00 0x7f26240aca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:10.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.732+0000 7f26291aa640 1 -- 172.21.15.84:0/1475999557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c009a70 con 0x7f2624159b20 2023-10-16T11:26:10.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.732+0000 7f26289a9640 1 --2- 172.21.15.84:0/1475999557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2624159f00 0x7f26240aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:10.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.733+0000 7f26291aa640 1 --2- 172.21.15.84:0/1475999557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2624159b20 0x7f26240afd20 secure :-1 s=READY pgs=3053 cs=0 l=1 rev1=1 crypto rx=0x7f262000c3c0 tx=0x7f262000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:10.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.733+0000 7f2619ffb640 1 -- 172.21.15.84:0/1475999557 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2620005ce0 con 0x7f2624159b20 2023-10-16T11:26:10.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.733+0000 7f2619ffb640 1 -- 172.21.15.84:0/1475999557 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2620005e40 con 0x7f2624159b20 2023-10-16T11:26:10.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.733+0000 7f2619ffb640 1 -- 172.21.15.84:0/1475999557 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26200072d0 con 0x7f2624159b20 2023-10-16T11:26:10.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.733+0000 7f262b40e640 1 -- 172.21.15.84:0/1475999557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26240aece0 con 0x7f2624159b20 2023-10-16T11:26:10.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.733+0000 7f262b40e640 1 -- 172.21.15.84:0/1475999557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26240af2a0 con 0x7f2624159b20 2023-10-16T11:26:10.740 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:10.734+0000 7f2b48613040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:26:10.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.734+0000 7f262b40e640 1 -- 172.21.15.84:0/1475999557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26240af430 con 0x7f2624159b20 2023-10-16T11:26:10.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.735+0000 7f2619ffb640 1 -- 172.21.15.84:0/1475999557 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 147) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f26200074f0 con 0x7f2624159b20 2023-10-16T11:26:10.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.737+0000 7f2619ffb640 1 --2- 172.21.15.84:0/1475999557 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f260806b0b0 0x7f260806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:10.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.737+0000 7f26289a9640 1 -- 172.21.15.84:0/1475999557 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f260806b0b0 msgr2=0x7f260806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3938045451 2023-10-16T11:26:10.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.737+0000 7f2619ffb640 1 -- 172.21.15.84:0/1475999557 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7031+0+0 (secure 0 0 0) 0x7f26200894f0 con 0x7f2624159b20 2023-10-16T11:26:10.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.737+0000 7f26289a9640 1 --2- 172.21.15.84:0/1475999557 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f260806b0b0 0x7f260806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:26:10.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.741+0000 7f2619ffb640 1 -- 172.21.15.84:0/1475999557 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2620053d10 con 0x7f2624159b20 2023-10-16T11:26:10.788 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:10.786+0000 7fa00ca2c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:26:10.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.934+0000 7f262b40e640 1 -- 172.21.15.84:0/1475999557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f262404e160 con 0x7f2624159b20 2023-10-16T11:26:10.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.938+0000 7f26289a9640 1 -- 172.21.15.84:0/1475999557 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f260806b0b0 msgr2=0x7f260806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3938045451 2023-10-16T11:26:10.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.938+0000 7f26289a9640 1 --2- 172.21.15.84:0/1475999557 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f260806b0b0 0x7f260806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:26:10.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.938+0000 7f2619ffb640 1 -- 172.21.15.84:0/1475999557 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v242) v1 ==== 74+0+20829 (secure 0 0 0) 0x7f2620057330 con 0x7f2624159b20 2023-10-16T11:26:10.940 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:10.940 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":242,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:26:08.077740+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:25:34.443897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:25:35.635615+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:25:49.001107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:25:49.760637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6833/2963529201":"2023-10-17T11:26:08.077644+0000","172.21.15.84:6832/2963529201":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/2303342316":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/648102622":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/133211355":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1489017743":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1517291402":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1983743360":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1323010124":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/437386817":"2023-10-17T11:26:08.077644+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:26:10.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.940+0000 7f262b40e640 1 -- 172.21.15.84:0/1475999557 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f260806b0b0 msgr2=0x7f260806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:26:10.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.940+0000 7f262b40e640 1 --2- 172.21.15.84:0/1475999557 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f260806b0b0 0x7f260806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:10.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.940+0000 7f262b40e640 1 -- 172.21.15.84:0/1475999557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2624159b20 msgr2=0x7f26240afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:10.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.940+0000 7f262b40e640 1 --2- 172.21.15.84:0/1475999557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2624159b20 0x7f26240afd20 secure :-1 s=READY pgs=3053 cs=0 l=1 rev1=1 crypto rx=0x7f262000c3c0 tx=0x7f262000c880 comp rx=0 tx=0).stop 2023-10-16T11:26:10.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.940+0000 7f262b40e640 1 -- 172.21.15.84:0/1475999557 shutdown_connections 2023-10-16T11:26:10.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.940+0000 7f262b40e640 1 --2- 172.21.15.84:0/1475999557 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f260806b0b0 0x7f260806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:10.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.940+0000 7f262b40e640 1 --2- 172.21.15.84:0/1475999557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f262415d880 0x7f2624155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:10.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.940+0000 7f262b40e640 1 --2- 172.21.15.84:0/1475999557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2624159f00 0x7f26240aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:10.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.940+0000 7f262b40e640 1 --2- 172.21.15.84:0/1475999557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2624159b20 0x7f26240afd20 unknown :-1 s=CLOSED pgs=3053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:10.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.941+0000 7f262b40e640 1 -- 172.21.15.84:0/1475999557 >> 172.21.15.84:0/1475999557 conn(0x7f26240cb410 msgr2=0x7f262406d360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:10.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.941+0000 7f262b40e640 1 -- 172.21.15.84:0/1475999557 shutdown_connections 2023-10-16T11:26:10.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:10.941+0000 7f262b40e640 1 -- 172.21.15.84:0/1475999557 wait complete. 2023-10-16T11:26:10.961 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:25:34.443897+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '236', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:25:35.635615+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '237', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-16T11:25:49.001107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '240', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-16T11:25:49.760637+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '241', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:26:10.962 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:26:11.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.160+0000 7f8b03fff640 1 Processor -- start 2023-10-16T11:26:11.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.160+0000 7f8b03fff640 1 -- start start 2023-10-16T11:26:11.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.161+0000 7f8b03fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8b0415acd0 0x7f8b040b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:11.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.161+0000 7f8b03fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b040cd9b0 0x7f8b040ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:11.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.162+0000 7f8b03fff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b0415b0b0 con 0x7f8b040afcf0 2023-10-16T11:26:11.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.162+0000 7f8b03fff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b040b6730 con 0x7f8b0415acd0 2023-10-16T11:26:11.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.162+0000 7f8b03fff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b040cc6f0 con 0x7f8b040cd9b0 2023-10-16T11:26:11.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.162+0000 7f8b02ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8b0415acd0 0x7f8b040b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:11.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.162+0000 7f8b02ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8b0415acd0 0x7f8b040b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53808/0 (socket says 172.21.15.84:53808) 2023-10-16T11:26:11.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.162+0000 7f8b02ffd640 1 -- 172.21.15.84:0/67583603 learned_addr learned my addr 172.21.15.84:0/67583603 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:11.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.162+0000 7f8b02ffd640 1 -- 172.21.15.84:0/67583603 >> v2:172.21.15.157:3300/0 conn(0x7f8b040cd9b0 msgr2=0x7f8b040ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:11.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.162+0000 7f8b02ffd640 1 --2- 172.21.15.84:0/67583603 >> v2:172.21.15.157:3300/0 conn(0x7f8b040cd9b0 0x7f8b040ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.162+0000 7f8b02ffd640 1 -- 172.21.15.84:0/67583603 >> v1:172.21.15.157:6789/0 conn(0x7f8b040afcf0 legacy=0x7f8b040cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:11.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.162+0000 7f8b02ffd640 1 -- 172.21.15.84:0/67583603 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b040cc9d0 con 0x7f8b0415acd0 2023-10-16T11:26:11.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.163+0000 7f8b02ffd640 1 --2- 172.21.15.84:0/67583603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0415acd0 0x7f8b040b60a0 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009870 tx=0x7f8af402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46cf2051756a5b4a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:11.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.164+0000 7f8b01ffb640 1 -- 172.21.15.84:0/67583603 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af4031870 con 0x7f8b0415acd0 2023-10-16T11:26:11.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.164+0000 7f8b01ffb640 1 -- 172.21.15.84:0/67583603 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af40319d0 con 0x7f8b0415acd0 2023-10-16T11:26:11.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.164+0000 7f8b01ffb640 1 -- 172.21.15.84:0/67583603 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af4031d60 con 0x7f8b0415acd0 2023-10-16T11:26:11.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.164+0000 7f8b03fff640 1 -- 172.21.15.84:0/67583603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0415acd0 msgr2=0x7f8b040b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:11.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.164+0000 7f8b03fff640 1 --2- 172.21.15.84:0/67583603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0415acd0 0x7f8b040b60a0 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009870 tx=0x7f8af402b910 comp rx=0 tx=0).stop 2023-10-16T11:26:11.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.164+0000 7f8b03fff640 1 -- 172.21.15.84:0/67583603 shutdown_connections 2023-10-16T11:26:11.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.164+0000 7f8b03fff640 1 --2- 172.21.15.84:0/67583603 >> v2:172.21.15.157:3300/0 conn(0x7f8b040cd9b0 0x7f8b040ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.164+0000 7f8b03fff640 1 --2- 172.21.15.84:0/67583603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0415acd0 0x7f8b040b60a0 unknown :-1 s=CLOSED pgs=3054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.164+0000 7f8b03fff640 1 -- 172.21.15.84:0/67583603 >> 172.21.15.84:0/67583603 conn(0x7f8b040b23b0 msgr2=0x7f8b040b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:11.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.165+0000 7f8b03fff640 1 -- 172.21.15.84:0/67583603 shutdown_connections 2023-10-16T11:26:11.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.165+0000 7f8b03fff640 1 -- 172.21.15.84:0/67583603 wait complete. 2023-10-16T11:26:11.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.166+0000 7f8b03fff640 1 Processor -- start 2023-10-16T11:26:11.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.167+0000 7f8b03fff640 1 -- start start 2023-10-16T11:26:11.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.168+0000 7f8b03fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b040afcf0 0x7f8b0414de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:11.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.168+0000 7f8b03fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b040cd9b0 0x7f8b040c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:11.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.168+0000 7f8b03fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0415acd0 0x7f8b040ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:11.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.168+0000 7f8b03fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0406c1d0 con 0x7f8b0415acd0 2023-10-16T11:26:11.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.168+0000 7f8b03fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0415c000 con 0x7f8b040afcf0 2023-10-16T11:26:11.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.168+0000 7f8b03fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0406c420 con 0x7f8b040cd9b0 2023-10-16T11:26:11.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.168+0000 7f8b037fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0415acd0 0x7f8b040ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:11.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.168+0000 7f8b037fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0415acd0 0x7f8b040ae360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53818/0 (socket says 172.21.15.84:53818) 2023-10-16T11:26:11.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.168+0000 7f8b037fe640 1 -- 172.21.15.84:0/1210886562 learned_addr learned my addr 172.21.15.84:0/1210886562 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:11.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.168+0000 7f8b02ffd640 1 --2- 172.21.15.84:0/1210886562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b040afcf0 0x7f8b0414de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:11.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.168+0000 7f8b037fe640 1 -- 172.21.15.84:0/1210886562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b040cd9b0 msgr2=0x7f8b040c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:11.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.168+0000 7f8b027fc640 1 --2- 172.21.15.84:0/1210886562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b040cd9b0 0x7f8b040c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:11.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.168+0000 7f8b037fe640 1 --2- 172.21.15.84:0/1210886562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b040cd9b0 0x7f8b040c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.168+0000 7f8b037fe640 1 -- 172.21.15.84:0/1210886562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b040afcf0 msgr2=0x7f8b0414de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:11.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.168+0000 7f8b037fe640 1 --2- 172.21.15.84:0/1210886562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b040afcf0 0x7f8b0414de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.168+0000 7f8b037fe640 1 -- 172.21.15.84:0/1210886562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af40094d0 con 0x7f8b0415acd0 2023-10-16T11:26:11.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.169+0000 7f8b037fe640 1 --2- 172.21.15.84:0/1210886562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0415acd0 0x7f8b040ae360 secure :-1 s=READY pgs=3055 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00b9e0 tx=0x7f8afc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:11.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.169+0000 7f8aeb7fe640 1 -- 172.21.15.84:0/1210886562 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00caa0 con 0x7f8b0415acd0 2023-10-16T11:26:11.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.169+0000 7f8aeb7fe640 1 -- 172.21.15.84:0/1210886562 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8afc00cc00 con 0x7f8b0415acd0 2023-10-16T11:26:11.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.169+0000 7f8aeb7fe640 1 -- 172.21.15.84:0/1210886562 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc014440 con 0x7f8b0415acd0 2023-10-16T11:26:11.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.169+0000 7f8b03fff640 1 -- 172.21.15.84:0/1210886562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0406c700 con 0x7f8b0415acd0 2023-10-16T11:26:11.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.169+0000 7f8b03fff640 1 -- 172.21.15.84:0/1210886562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0406ccc0 con 0x7f8b0415acd0 2023-10-16T11:26:11.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.171+0000 7f8b03fff640 1 -- 172.21.15.84:0/1210886562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b0415c170 con 0x7f8b0415acd0 2023-10-16T11:26:11.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.171+0000 7f8aeb7fe640 1 -- 172.21.15.84:0/1210886562 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 147) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f8afc01d3b0 con 0x7f8b0415acd0 2023-10-16T11:26:11.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.172+0000 7f8aeb7fe640 1 --2- 172.21.15.84:0/1210886562 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f8af006b0b0 0x7f8af006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:11.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.172+0000 7f8aeb7fe640 1 -- 172.21.15.84:0/1210886562 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7031+0+0 (secure 0 0 0) 0x7f8afc089e20 con 0x7f8b0415acd0 2023-10-16T11:26:11.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.172+0000 7f8b02ffd640 1 -- 172.21.15.84:0/1210886562 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f8af006b0b0 msgr2=0x7f8af006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3938045451 2023-10-16T11:26:11.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.172+0000 7f8b02ffd640 1 --2- 172.21.15.84:0/1210886562 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f8af006b0b0 0x7f8af006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:26:11.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.176+0000 7f8aeb7fe640 1 -- 172.21.15.84:0/1210886562 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8afc054630 con 0x7f8b0415acd0 2023-10-16T11:26:11.202 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:11.201+0000 7f2b48613040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:26:11.239 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:11.237+0000 7fa00ca2c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:26:11.256 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:11.255+0000 7f2b48613040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:26:11.291 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:11.289+0000 7fa00ca2c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:26:11.309 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:11.308+0000 7f2b48613040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:26:11.340 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:11.338+0000 7fa00ca2c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:26:11.372 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:11.371+0000 7f2b48613040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:26:11.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.371+0000 7f8b03fff640 1 -- 172.21.15.84:0/1210886562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b0404e9b0 con 0x7f8b0415acd0 2023-10-16T11:26:11.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.372+0000 7f8aeb7fe640 1 -- 172.21.15.84:0/1210886562 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v242) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8afc019020 con 0x7f8b0415acd0 2023-10-16T11:26:11.373 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:26:11.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.373+0000 7f8b02ffd640 1 -- 172.21.15.84:0/1210886562 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f8af006b0b0 msgr2=0x7f8af006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3938045451 2023-10-16T11:26:11.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.373+0000 7f8b02ffd640 1 --2- 172.21.15.84:0/1210886562 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f8af006b0b0 0x7f8af006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:26:11.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.375+0000 7f8b03fff640 1 -- 172.21.15.84:0/1210886562 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f8af006b0b0 msgr2=0x7f8af006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:26:11.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.375+0000 7f8b03fff640 1 --2- 172.21.15.84:0/1210886562 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f8af006b0b0 0x7f8af006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.375+0000 7f8b03fff640 1 -- 172.21.15.84:0/1210886562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0415acd0 msgr2=0x7f8b040ae360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:11.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.375+0000 7f8b03fff640 1 --2- 172.21.15.84:0/1210886562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0415acd0 0x7f8b040ae360 secure :-1 s=READY pgs=3055 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00b9e0 tx=0x7f8afc00bea0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.375+0000 7f8b03fff640 1 -- 172.21.15.84:0/1210886562 shutdown_connections 2023-10-16T11:26:11.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.375+0000 7f8b03fff640 1 --2- 172.21.15.84:0/1210886562 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f8af006b0b0 0x7f8af006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.375+0000 7f8b03fff640 1 --2- 172.21.15.84:0/1210886562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0415acd0 0x7f8b040ae360 unknown :-1 s=CLOSED pgs=3055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.375+0000 7f8b03fff640 1 --2- 172.21.15.84:0/1210886562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b040cd9b0 0x7f8b040c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.375+0000 7f8b03fff640 1 --2- 172.21.15.84:0/1210886562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b040afcf0 0x7f8b0414de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.375+0000 7f8b03fff640 1 -- 172.21.15.84:0/1210886562 >> 172.21.15.84:0/1210886562 conn(0x7f8b040b23b0 msgr2=0x7f8b040cfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:11.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.376+0000 7f8b03fff640 1 -- 172.21.15.84:0/1210886562 shutdown_connections 2023-10-16T11:26:11.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.376+0000 7f8b03fff640 1 -- 172.21.15.84:0/1210886562 wait complete. 2023-10-16T11:26:11.395 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:26:11.401 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:11.399+0000 7fa00ca2c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:26:11.425 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:11.424+0000 7f2b48613040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:26:11.450 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:11.449+0000 7fa00ca2c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:26:11.543 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:11.542+0000 7f2b48613040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:26:11.561 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:11.559+0000 7fa00ca2c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:26:11.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.592+0000 7f146a6d4640 1 Processor -- start 2023-10-16T11:26:11.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.592+0000 7f146a6d4640 1 -- start start 2023-10-16T11:26:11.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.593+0000 7f146a6d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1464070830 0x7f146406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:11.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.593+0000 7f146a6d4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1464070c10 0x7f14640cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:11.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.593+0000 7f146a6d4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1464161740 con 0x7f14640cf5f0 2023-10-16T11:26:11.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.593+0000 7f146a6d4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14640cdbd0 con 0x7f1464070c10 2023-10-16T11:26:11.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.593+0000 7f146a6d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146406b0f0 con 0x7f1464070830 2023-10-16T11:26:11.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.594+0000 7f1463fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1464070830 0x7f146406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:11.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.594+0000 7f1468c71640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1464070c10 0x7f14640cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:11.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.594+0000 7f1463fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1464070830 0x7f146406d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:56264/0 (socket says 172.21.15.84:56264) 2023-10-16T11:26:11.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.594+0000 7f1463fff640 1 -- 172.21.15.84:0/490212404 learned_addr learned my addr 172.21.15.84:0/490212404 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:11.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.595+0000 7f14637fe640 1 -- 172.21.15.84:0/490212404 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238873274 0 0) 0x7f1464161740 con 0x7f14640cf5f0 2023-10-16T11:26:11.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.595+0000 7f14637fe640 1 -- 172.21.15.84:0/490212404 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1450003520 con 0x7f14640cf5f0 2023-10-16T11:26:11.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.595+0000 7f1463fff640 1 -- 172.21.15.84:0/490212404 >> v2:172.21.15.157:3300/0 conn(0x7f1464070c10 msgr2=0x7f14640cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:11.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.595+0000 7f1463fff640 1 --2- 172.21.15.84:0/490212404 >> v2:172.21.15.157:3300/0 conn(0x7f1464070c10 0x7f14640cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.595+0000 7f1463fff640 1 -- 172.21.15.84:0/490212404 >> v1:172.21.15.157:6789/0 conn(0x7f14640cf5f0 legacy=0x7f146406c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:11.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.595+0000 7f1463fff640 1 -- 172.21.15.84:0/490212404 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14641619b0 con 0x7f1464070830 2023-10-16T11:26:11.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.596+0000 7f1463fff640 1 --2- 172.21.15.84:0/490212404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1464070830 0x7f146406d3c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1454009870 tx=0x7f145402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc3141159a8ee72f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:11.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.597+0000 7f14637fe640 1 -- 172.21.15.84:0/490212404 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14540317c0 con 0x7f1464070830 2023-10-16T11:26:11.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.597+0000 7f14637fe640 1 -- 172.21.15.84:0/490212404 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1454031920 con 0x7f1464070830 2023-10-16T11:26:11.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.597+0000 7f14637fe640 1 -- 172.21.15.84:0/490212404 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454031c90 con 0x7f1464070830 2023-10-16T11:26:11.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.597+0000 7f146a6d4640 1 -- 172.21.15.84:0/490212404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1464070830 msgr2=0x7f146406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:11.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.597+0000 7f146a6d4640 1 --2- 172.21.15.84:0/490212404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1464070830 0x7f146406d3c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1454009870 tx=0x7f145402b860 comp rx=0 tx=0).stop 2023-10-16T11:26:11.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.597+0000 7f146a6d4640 1 -- 172.21.15.84:0/490212404 shutdown_connections 2023-10-16T11:26:11.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.597+0000 7f146a6d4640 1 --2- 172.21.15.84:0/490212404 >> v2:172.21.15.157:3300/0 conn(0x7f1464070c10 0x7f14640cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.597+0000 7f146a6d4640 1 --2- 172.21.15.84:0/490212404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1464070830 0x7f146406d3c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.597+0000 7f146a6d4640 1 -- 172.21.15.84:0/490212404 >> 172.21.15.84:0/490212404 conn(0x7f14640cfac0 msgr2=0x7f14640cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:11.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.597+0000 7f146a6d4640 1 -- 172.21.15.84:0/490212404 shutdown_connections 2023-10-16T11:26:11.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.597+0000 7f146a6d4640 1 -- 172.21.15.84:0/490212404 wait complete. 2023-10-16T11:26:11.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.599+0000 7f146a6d4640 1 Processor -- start 2023-10-16T11:26:11.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.599+0000 7f146a6d4640 1 -- start start 2023-10-16T11:26:11.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.600+0000 7f146a6d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1464070830 0x7f1464150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:11.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.600+0000 7f146a6d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1464070c10 0x7f14641f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:11.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.600+0000 7f146a6d4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14640cf5f0 0x7f146414ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:11.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.600+0000 7f146a6d4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14641f6410 con 0x7f14640cf5f0 2023-10-16T11:26:11.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.600+0000 7f146a6d4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14641f6640 con 0x7f1464070c10 2023-10-16T11:26:11.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.600+0000 7f146a6d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14641f6920 con 0x7f1464070830 2023-10-16T11:26:11.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.600+0000 7f1468c71640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14640cf5f0 0x7f146414ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:11.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.601+0000 7f1468c71640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14640cf5f0 0x7f146414ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53822/0 (socket says 172.21.15.84:53822) 2023-10-16T11:26:11.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.601+0000 7f1468c71640 1 -- 172.21.15.84:0/1386313677 learned_addr learned my addr 172.21.15.84:0/1386313677 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:11.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.601+0000 7f145bfff640 1 --2- 172.21.15.84:0/1386313677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1464070c10 0x7f14641f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:11.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.601+0000 7f1463fff640 1 --2- 172.21.15.84:0/1386313677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1464070830 0x7f1464150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:11.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.601+0000 7f1468c71640 1 -- 172.21.15.84:0/1386313677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1464070830 msgr2=0x7f1464150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:11.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.601+0000 7f1468c71640 1 --2- 172.21.15.84:0/1386313677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1464070830 0x7f1464150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.601+0000 7f1468c71640 1 -- 172.21.15.84:0/1386313677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1464070c10 msgr2=0x7f14641f5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:11.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.601+0000 7f1468c71640 1 --2- 172.21.15.84:0/1386313677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1464070c10 0x7f14641f5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.601+0000 7f1468c71640 1 -- 172.21.15.84:0/1386313677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14540094d0 con 0x7f14640cf5f0 2023-10-16T11:26:11.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.601+0000 7f145bfff640 1 --2- 172.21.15.84:0/1386313677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1464070c10 0x7f14641f5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:26:11.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.602+0000 7f1468c71640 1 --2- 172.21.15.84:0/1386313677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14640cf5f0 0x7f146414ce00 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 crypto rx=0x7f145c00efc0 tx=0x7f145c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:11.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.602+0000 7f14617fa640 1 -- 172.21.15.84:0/1386313677 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c014480 con 0x7f14640cf5f0 2023-10-16T11:26:11.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.602+0000 7f14617fa640 1 -- 172.21.15.84:0/1386313677 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c00f040 con 0x7f14640cf5f0 2023-10-16T11:26:11.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.602+0000 7f146a6d4640 1 -- 172.21.15.84:0/1386313677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14641f6c00 con 0x7f14640cf5f0 2023-10-16T11:26:11.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.602+0000 7f14617fa640 1 -- 172.21.15.84:0/1386313677 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c014820 con 0x7f14640cf5f0 2023-10-16T11:26:11.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.602+0000 7f146a6d4640 1 -- 172.21.15.84:0/1386313677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146414d340 con 0x7f14640cf5f0 2023-10-16T11:26:11.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.603+0000 7f146a6d4640 1 -- 172.21.15.84:0/1386313677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146414db30 con 0x7f14640cf5f0 2023-10-16T11:26:11.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.604+0000 7f14617fa640 1 -- 172.21.15.84:0/1386313677 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 147) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f145c014980 con 0x7f14640cf5f0 2023-10-16T11:26:11.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.605+0000 7f14617fa640 1 --2- 172.21.15.84:0/1386313677 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f144406b0e0 0x7f144406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:11.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.605+0000 7f1463fff640 1 -- 172.21.15.84:0/1386313677 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f144406b0e0 msgr2=0x7f144406d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3938045451 2023-10-16T11:26:11.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.605+0000 7f14617fa640 1 -- 172.21.15.84:0/1386313677 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7031+0+0 (secure 0 0 0) 0x7f145c08abf0 con 0x7f14640cf5f0 2023-10-16T11:26:11.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.605+0000 7f1463fff640 1 --2- 172.21.15.84:0/1386313677 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f144406b0e0 0x7f144406d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:26:11.613 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:11.613+0000 7f2b48613040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:26:11.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.613+0000 7f14617fa640 1 -- 172.21.15.84:0/1386313677 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f146414db30 con 0x7f14640cf5f0 2023-10-16T11:26:11.625 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:11.624+0000 7fa00ca2c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:26:11.666 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:11.665+0000 7f2b48613040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:26:11.675 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:11.674+0000 7fa00ca2c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:26:11.725 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:11.724+0000 7f2b48613040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:26:11.732 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:11.730+0000 7fa00ca2c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:26:11.778 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:11.777+0000 7f2b48613040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:26:11.782 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:11.780+0000 7fa00ca2c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:26:11.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.806+0000 7f1463fff640 1 -- 172.21.15.84:0/1386313677 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f144406b0e0 msgr2=0x7f144406d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3938045451 2023-10-16T11:26:11.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.806+0000 7f1463fff640 1 --2- 172.21.15.84:0/1386313677 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f144406b0e0 0x7f144406d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:26:11.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.807+0000 7f146a6d4640 1 -- 172.21.15.84:0/1386313677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f146414db30 con 0x7f14640cf5f0 2023-10-16T11:26:11.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.808+0000 7f14617fa640 1 -- 172.21.15.84:0/1386313677 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v242) v1 ==== 104+0+11 (secure 0 0 0) 0x7f146414db30 con 0x7f14640cf5f0 2023-10-16T11:26:11.809 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:26:11.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.810+0000 7f146a6d4640 1 -- 172.21.15.84:0/1386313677 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f144406b0e0 msgr2=0x7f144406d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:26:11.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.810+0000 7f146a6d4640 1 --2- 172.21.15.84:0/1386313677 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f144406b0e0 0x7f144406d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.810+0000 7f146a6d4640 1 -- 172.21.15.84:0/1386313677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14640cf5f0 msgr2=0x7f146414ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:11.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.810+0000 7f146a6d4640 1 --2- 172.21.15.84:0/1386313677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14640cf5f0 0x7f146414ce00 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 crypto rx=0x7f145c00efc0 tx=0x7f145c00c490 comp rx=0 tx=0).stop 2023-10-16T11:26:11.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.810+0000 7f146a6d4640 1 -- 172.21.15.84:0/1386313677 shutdown_connections 2023-10-16T11:26:11.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.810+0000 7f146a6d4640 1 --2- 172.21.15.84:0/1386313677 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f144406b0e0 0x7f144406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.810+0000 7f146a6d4640 1 --2- 172.21.15.84:0/1386313677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14640cf5f0 0x7f146414ce00 unknown :-1 s=CLOSED pgs=3056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.810+0000 7f146a6d4640 1 --2- 172.21.15.84:0/1386313677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1464070c10 0x7f14641f5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.810+0000 7f146a6d4640 1 --2- 172.21.15.84:0/1386313677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1464070830 0x7f1464150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:11.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.810+0000 7f146a6d4640 1 -- 172.21.15.84:0/1386313677 >> 172.21.15.84:0/1386313677 conn(0x7f14640cfac0 msgr2=0x7f14640cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:11.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.810+0000 7f146a6d4640 1 -- 172.21.15.84:0/1386313677 shutdown_connections 2023-10-16T11:26:11.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:11.810+0000 7f146a6d4640 1 -- 172.21.15.84:0/1386313677 wait complete. 2023-10-16T11:26:11.830 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:26:12.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.087+0000 7f1a08ffc640 1 Processor -- start 2023-10-16T11:26:12.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.087+0000 7f1a08ffc640 1 -- start start 2023-10-16T11:26:12.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.088+0000 7f1a08ffc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1a04159ab0 0x7f1a040ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:12.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.089+0000 7f1a08ffc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a04070360 con 0x7f1a0406c140 2023-10-16T11:26:12.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.089+0000 7f1a08ffc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a040cd1f0 con 0x7f1a04159e90 2023-10-16T11:26:12.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.089+0000 7f1a08ffc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0415d890 con 0x7f1a04159ab0 2023-10-16T11:26:12.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.089+0000 7f1a02d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1a0406c140 0x7f1a0415dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40380/0 (socket says 172.21.15.84:40380) 2023-10-16T11:26:12.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.089+0000 7f1a02d9d640 1 -- 172.21.15.84:0/3299487124 learned_addr learned my addr 172.21.15.84:0/3299487124 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:12.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.089+0000 7f1a0259c640 1 --2- 172.21.15.84:0/3299487124 >> v2:172.21.15.157:3300/0 conn(0x7f1a04159ab0 0x7f1a040ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:12.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.089+0000 7f1a01d9b640 1 -- 172.21.15.84:0/3299487124 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2010584485 0 0) 0x7f1a04070360 con 0x7f1a0406c140 2023-10-16T11:26:12.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.089+0000 7f1a01d9b640 1 -- 172.21.15.84:0/3299487124 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19f0003520 con 0x7f1a0406c140 2023-10-16T11:26:12.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.090+0000 7f1a01d9b640 1 -- 172.21.15.84:0/3299487124 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1785107421 0 0) 0x7f19f0003520 con 0x7f1a0406c140 2023-10-16T11:26:12.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.090+0000 7f1a01d9b640 1 -- 172.21.15.84:0/3299487124 >> v2:172.21.15.157:3300/0 conn(0x7f1a04159ab0 msgr2=0x7f1a040ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:12.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.090+0000 7f1a01d9b640 1 --2- 172.21.15.84:0/3299487124 >> v2:172.21.15.157:3300/0 conn(0x7f1a04159ab0 0x7f1a040ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.090+0000 7f1a01d9b640 1 -- 172.21.15.84:0/3299487124 >> v1:172.21.15.157:6789/0 conn(0x7f1a04159e90 legacy=0x7f1a0406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:12.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.090+0000 7f1a01d9b640 1 -- 172.21.15.84:0/3299487124 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a040cd4d0 con 0x7f1a0406c140 2023-10-16T11:26:12.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.090+0000 7f1a01d9b640 1 -- 172.21.15.84:0/3299487124 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f19fc0019f0 con 0x7f1a0406c140 2023-10-16T11:26:12.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.090+0000 7f1a01d9b640 1 -- 172.21.15.84:0/3299487124 >> v1:172.21.15.84:6789/0 conn(0x7f1a0406c140 legacy=0x7f1a0415dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:12.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.090+0000 7f1a01d9b640 1 --2- 172.21.15.84:0/3299487124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19f0003970 0x7f19f0003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:12.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.090+0000 7f1a01d9b640 1 --2- 172.21.15.84:0/3299487124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f00047c0 0x7f19f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:12.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.090+0000 7f1a01d9b640 1 --2- 172.21.15.84:0/3299487124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0005180 0x7f19f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:12.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.090+0000 7f1a01d9b640 1 -- 172.21.15.84:0/3299487124 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0415d890 con 0x7f19f0005180 2023-10-16T11:26:12.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.090+0000 7f1a01d9b640 1 -- 172.21.15.84:0/3299487124 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f000c0d0 con 0x7f19f00047c0 2023-10-16T11:26:12.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.090+0000 7f1a01d9b640 1 -- 172.21.15.84:0/3299487124 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f000c290 con 0x7f19f0003970 2023-10-16T11:26:12.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.090+0000 7f1a02d9d640 1 --2- 172.21.15.84:0/3299487124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0005180 0x7f19f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:12.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.090+0000 7f19fbfff640 1 --2- 172.21.15.84:0/3299487124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f00047c0 0x7f19f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:12.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.091+0000 7f1a0259c640 1 --2- 172.21.15.84:0/3299487124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19f0003970 0x7f19f0003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:12.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.091+0000 7f1a02d9d640 1 -- 172.21.15.84:0/3299487124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19f0003970 msgr2=0x7f19f0003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:12.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.091+0000 7f1a02d9d640 1 --2- 172.21.15.84:0/3299487124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19f0003970 0x7f19f0003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.091+0000 7f1a02d9d640 1 -- 172.21.15.84:0/3299487124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f00047c0 msgr2=0x7f19f0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:12.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.091+0000 7f1a02d9d640 1 --2- 172.21.15.84:0/3299487124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f00047c0 0x7f19f0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.091+0000 7f1a02d9d640 1 -- 172.21.15.84:0/3299487124 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f0003520 con 0x7f19f0005180 2023-10-16T11:26:12.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.092+0000 7f1a02d9d640 1 -- 172.21.15.84:0/3299487124 reap_dead start 2023-10-16T11:26:12.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.092+0000 7f1a02d9d640 1 --2- 172.21.15.84:0/3299487124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0005180 0x7f19f000b980 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 crypto rx=0x7f19fc00a830 tx=0x7f19f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a2b8c64797620f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:12.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.092+0000 7f1a01d9b640 1 -- 172.21.15.84:0/3299487124 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc02cea0 con 0x7f19f0005180 2023-10-16T11:26:12.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.092+0000 7f1a01d9b640 1 -- 172.21.15.84:0/3299487124 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc02f070 con 0x7f19f0005180 2023-10-16T11:26:12.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.092+0000 7f1a01d9b640 1 -- 172.21.15.84:0/3299487124 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc034890 con 0x7f19f0005180 2023-10-16T11:26:12.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.093+0000 7f1a08ffc640 1 -- 172.21.15.84:0/3299487124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0005180 msgr2=0x7f19f000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:12.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.093+0000 7f1a08ffc640 1 --2- 172.21.15.84:0/3299487124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0005180 0x7f19f000b980 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 crypto rx=0x7f19fc00a830 tx=0x7f19f00057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.093+0000 7f1a08ffc640 1 -- 172.21.15.84:0/3299487124 shutdown_connections 2023-10-16T11:26:12.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.093+0000 7f1a08ffc640 1 --2- 172.21.15.84:0/3299487124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0005180 0x7f19f000b980 unknown :-1 s=CLOSED pgs=3058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.093+0000 7f1a08ffc640 1 -- 172.21.15.84:0/3299487124 >> 172.21.15.84:0/3299487124 conn(0x7f1a040ce9b0 msgr2=0x7f1a040cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:12.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.093+0000 7f1a08ffc640 1 -- 172.21.15.84:0/3299487124 shutdown_connections 2023-10-16T11:26:12.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.093+0000 7f1a08ffc640 1 -- 172.21.15.84:0/3299487124 wait complete. 2023-10-16T11:26:12.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.095+0000 7f1a08ffc640 1 Processor -- start 2023-10-16T11:26:12.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.095+0000 7f1a08ffc640 1 -- start start 2023-10-16T11:26:12.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.096+0000 7f1a08ffc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0005180 0x7f1a0414ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:12.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.096+0000 7f1a08ffc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a040b1a90 0x7f1a040c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:12.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.096+0000 7f1a08ffc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a040c44f0 0x7f1a040c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:12.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.096+0000 7f1a08ffc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a040c4ee0 con 0x7f19f0005180 2023-10-16T11:26:12.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.096+0000 7f1a08ffc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0414e410 con 0x7f1a040b1a90 2023-10-16T11:26:12.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.096+0000 7f1a08ffc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a040c7550 con 0x7f1a040c44f0 2023-10-16T11:26:12.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.096+0000 7f1a0259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0005180 0x7f1a0414ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:12.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.096+0000 7f1a0259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0005180 0x7f1a0414ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53848/0 (socket says 172.21.15.84:53848) 2023-10-16T11:26:12.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.096+0000 7f1a0259c640 1 -- 172.21.15.84:0/828847654 learned_addr learned my addr 172.21.15.84:0/828847654 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:12.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.097+0000 7f1a02d9d640 1 --2- 172.21.15.84:0/828847654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a040c44f0 0x7f1a040c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:12.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.097+0000 7f19fbfff640 1 --2- 172.21.15.84:0/828847654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a040b1a90 0x7f1a040c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:12.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.097+0000 7f1a0259c640 1 -- 172.21.15.84:0/828847654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a040c44f0 msgr2=0x7f1a040c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:12.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.097+0000 7f1a0259c640 1 --2- 172.21.15.84:0/828847654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a040c44f0 0x7f1a040c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.097+0000 7f1a0259c640 1 -- 172.21.15.84:0/828847654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a040b1a90 msgr2=0x7f1a040c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:12.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.097+0000 7f1a0259c640 1 --2- 172.21.15.84:0/828847654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a040b1a90 0x7f1a040c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.097+0000 7f1a0259c640 1 -- 172.21.15.84:0/828847654 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a040c7830 con 0x7f19f0005180 2023-10-16T11:26:12.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.097+0000 7f1a0259c640 1 --2- 172.21.15.84:0/828847654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0005180 0x7f1a0414ded0 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 crypto rx=0x7f19f4011540 tx=0x7f19f4011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:12.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.097+0000 7f19faffd640 1 -- 172.21.15.84:0/828847654 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f4015930 con 0x7f19f0005180 2023-10-16T11:26:12.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.097+0000 7f19faffd640 1 -- 172.21.15.84:0/828847654 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f4015a90 con 0x7f19f0005180 2023-10-16T11:26:12.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.097+0000 7f19faffd640 1 -- 172.21.15.84:0/828847654 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f4015e60 con 0x7f19f0005180 2023-10-16T11:26:12.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.097+0000 7f1a08ffc640 1 -- 172.21.15.84:0/828847654 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a040c7b10 con 0x7f19f0005180 2023-10-16T11:26:12.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.098+0000 7f1a08ffc640 1 -- 172.21.15.84:0/828847654 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a040c80d0 con 0x7f19f0005180 2023-10-16T11:26:12.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.099+0000 7f1a08ffc640 1 -- 172.21.15.84:0/828847654 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0415ac70 con 0x7f19f0005180 2023-10-16T11:26:12.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.100+0000 7f19faffd640 1 -- 172.21.15.84:0/828847654 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 147) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f19f400f460 con 0x7f19f0005180 2023-10-16T11:26:12.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.101+0000 7f19faffd640 1 --2- 172.21.15.84:0/828847654 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f19e406b0b0 0x7f19e406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:12.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.101+0000 7f19fbfff640 1 -- 172.21.15.84:0/828847654 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f19e406b0b0 msgr2=0x7f19e406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3938045451 2023-10-16T11:26:12.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.101+0000 7f19faffd640 1 -- 172.21.15.84:0/828847654 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7031+0+0 (secure 0 0 0) 0x7f19f408a790 con 0x7f19f0005180 2023-10-16T11:26:12.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.101+0000 7f19fbfff640 1 --2- 172.21.15.84:0/828847654 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f19e406b0b0 0x7f19e406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:26:12.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.105+0000 7f19faffd640 1 -- 172.21.15.84:0/828847654 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f19f4054fa0 con 0x7f19f0005180 2023-10-16T11:26:12.166 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:12.165+0000 7fa00ca2c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:26:12.179 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:12.178+0000 7f2b48613040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:26:12.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.301+0000 7f1a08ffc640 1 -- 172.21.15.84:0/828847654 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1a0404e160 con 0x7f19f0005180 2023-10-16T11:26:12.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.302+0000 7f19fbfff640 1 -- 172.21.15.84:0/828847654 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f19e406b0b0 msgr2=0x7f19e406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3938045451 2023-10-16T11:26:12.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.302+0000 7f19fbfff640 1 --2- 172.21.15.84:0/828847654 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f19e406b0b0 0x7f19e406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:26:12.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.302+0000 7f19faffd640 1 -- 172.21.15.84:0/828847654 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v242) v1 ==== 100+0+11 (secure 0 0 0) 0x7f19f40585c0 con 0x7f19f0005180 2023-10-16T11:26:12.304 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:26:12.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.304+0000 7f1a08ffc640 1 -- 172.21.15.84:0/828847654 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f19e406b0b0 msgr2=0x7f19e406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:26:12.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.304+0000 7f1a08ffc640 1 --2- 172.21.15.84:0/828847654 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f19e406b0b0 0x7f19e406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.305+0000 7f1a08ffc640 1 -- 172.21.15.84:0/828847654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0005180 msgr2=0x7f1a0414ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:12.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.305+0000 7f1a08ffc640 1 --2- 172.21.15.84:0/828847654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0005180 0x7f1a0414ded0 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 crypto rx=0x7f19f4011540 tx=0x7f19f4011a00 comp rx=0 tx=0).stop 2023-10-16T11:26:12.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.305+0000 7f1a08ffc640 1 -- 172.21.15.84:0/828847654 shutdown_connections 2023-10-16T11:26:12.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.305+0000 7f1a08ffc640 1 --2- 172.21.15.84:0/828847654 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f19e406b0b0 0x7f19e406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.305+0000 7f1a08ffc640 1 --2- 172.21.15.84:0/828847654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a040c44f0 0x7f1a040c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.305+0000 7f1a08ffc640 1 --2- 172.21.15.84:0/828847654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a040b1a90 0x7f1a040c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.305+0000 7f1a08ffc640 1 --2- 172.21.15.84:0/828847654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0005180 0x7f1a0414ded0 unknown :-1 s=CLOSED pgs=3059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.305+0000 7f1a08ffc640 1 -- 172.21.15.84:0/828847654 >> 172.21.15.84:0/828847654 conn(0x7f1a040ce9b0 msgr2=0x7f1a040ae410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:12.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.305+0000 7f1a08ffc640 1 -- 172.21.15.84:0/828847654 shutdown_connections 2023-10-16T11:26:12.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.305+0000 7f1a08ffc640 1 -- 172.21.15.84:0/828847654 wait complete. 2023-10-16T11:26:12.325 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-16T11:26:12.326 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:12.325+0000 7fa00ca2c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:26:12.348 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:12.347+0000 7f2b48613040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:26:12.435 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:12.433+0000 7fa00ca2c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:26:12.461 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:12.460+0000 7f2b48613040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:26:12.485 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:12.483+0000 7fa00ca2c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:26:12.514 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:12.513+0000 7f2b48613040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:26:12.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.527+0000 7f4e6b516640 1 Processor -- start 2023-10-16T11:26:12.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.527+0000 7f4e6b516640 1 -- start start 2023-10-16T11:26:12.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.528+0000 7f4e6b516640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4e6406d220 0x7f4e640c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:12.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.528+0000 7f4e6b516640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e640ce280 0x7f4e640ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:12.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.528+0000 7f4e6b516640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e640acd70 con 0x7f4e64065ab0 2023-10-16T11:26:12.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.529+0000 7f4e6b516640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e640ac4a0 con 0x7f4e6406d220 2023-10-16T11:26:12.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.529+0000 7f4e6b516640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e64065f40 con 0x7f4e640ce280 2023-10-16T11:26:12.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.529+0000 7f4e69d13640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4e64065ab0 0x7f4e640cdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40382/0 (socket says 172.21.15.84:40382) 2023-10-16T11:26:12.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.529+0000 7f4e69d13640 1 -- 172.21.15.84:0/2820047995 learned_addr learned my addr 172.21.15.84:0/2820047995 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:12.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.529+0000 7f4e6a514640 1 --2- 172.21.15.84:0/2820047995 >> v2:172.21.15.157:3300/0 conn(0x7f4e6406d220 0x7f4e640c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:12.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.529+0000 7f4e69512640 1 -- 172.21.15.84:0/2820047995 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371785606 0 0) 0x7f4e640acd70 con 0x7f4e64065ab0 2023-10-16T11:26:12.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.529+0000 7f4e69512640 1 -- 172.21.15.84:0/2820047995 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e50003520 con 0x7f4e64065ab0 2023-10-16T11:26:12.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.529+0000 7f4e6ad15640 1 --2- 172.21.15.84:0/2820047995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e640ce280 0x7f4e640ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:12.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69512640 1 -- 172.21.15.84:0/2820047995 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3784956341 0 0) 0x7f4e50003520 con 0x7f4e64065ab0 2023-10-16T11:26:12.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69512640 1 -- 172.21.15.84:0/2820047995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e640ce280 msgr2=0x7f4e640ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:12.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69512640 1 --2- 172.21.15.84:0/2820047995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e640ce280 0x7f4e640ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69512640 1 -- 172.21.15.84:0/2820047995 >> v2:172.21.15.157:3300/0 conn(0x7f4e6406d220 msgr2=0x7f4e640c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:12.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69512640 1 --2- 172.21.15.84:0/2820047995 >> v2:172.21.15.157:3300/0 conn(0x7f4e6406d220 0x7f4e640c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69512640 1 -- 172.21.15.84:0/2820047995 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e640ac720 con 0x7f4e64065ab0 2023-10-16T11:26:12.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69512640 1 -- 172.21.15.84:0/2820047995 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f4e54001670 con 0x7f4e64065ab0 2023-10-16T11:26:12.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e6ad15640 1 --2- 172.21.15.84:0/2820047995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e640ce280 0x7f4e640ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:26:12.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69512640 1 -- 172.21.15.84:0/2820047995 >> v1:172.21.15.84:6789/0 conn(0x7f4e64065ab0 legacy=0x7f4e640cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:12.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69512640 1 --2- 172.21.15.84:0/2820047995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e50003970 0x7f4e50003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:12.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69512640 1 --2- 172.21.15.84:0/2820047995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e500047c0 0x7f4e50004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:12.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69512640 1 --2- 172.21.15.84:0/2820047995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e640ce280 0x7f4e50005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:12.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69512640 1 -- 172.21.15.84:0/2820047995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e640ac4a0 con 0x7f4e500047c0 2023-10-16T11:26:12.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69512640 1 -- 172.21.15.84:0/2820047995 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e64065f40 con 0x7f4e50003970 2023-10-16T11:26:12.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69512640 1 -- 172.21.15.84:0/2820047995 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5000b9d0 con 0x7f4e640ce280 2023-10-16T11:26:12.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69d13640 1 --2- 172.21.15.84:0/2820047995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e500047c0 0x7f4e50004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:12.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e6a514640 1 --2- 172.21.15.84:0/2820047995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e50003970 0x7f4e50003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:12.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69d13640 1 -- 172.21.15.84:0/2820047995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e640ce280 msgr2=0x7f4e50005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:12.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69d13640 1 --2- 172.21.15.84:0/2820047995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e640ce280 0x7f4e50005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69d13640 1 -- 172.21.15.84:0/2820047995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e50003970 msgr2=0x7f4e50003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:12.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.530+0000 7f4e69d13640 1 --2- 172.21.15.84:0/2820047995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e50003970 0x7f4e50003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.531+0000 7f4e69d13640 1 -- 172.21.15.84:0/2820047995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e50003520 con 0x7f4e500047c0 2023-10-16T11:26:12.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.531+0000 7f4e69d13640 1 --2- 172.21.15.84:0/2820047995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e500047c0 0x7f4e50004c40 secure :-1 s=READY pgs=3061 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400c1c0 tx=0x7f4e500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4c4fb6b8dc2f41 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:12.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.532+0000 7f4e69512640 1 -- 172.21.15.84:0/2820047995 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e54005ad0 con 0x7f4e500047c0 2023-10-16T11:26:12.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.532+0000 7f4e69512640 1 -- 172.21.15.84:0/2820047995 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e54005c50 con 0x7f4e500047c0 2023-10-16T11:26:12.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.532+0000 7f4e69512640 1 -- 172.21.15.84:0/2820047995 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5402c810 con 0x7f4e500047c0 2023-10-16T11:26:12.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.532+0000 7f4e6b516640 1 -- 172.21.15.84:0/2820047995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e500047c0 msgr2=0x7f4e50004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:12.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.532+0000 7f4e6b516640 1 --2- 172.21.15.84:0/2820047995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e500047c0 0x7f4e50004c40 secure :-1 s=READY pgs=3061 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400c1c0 tx=0x7f4e500057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.532+0000 7f4e6ad15640 1 -- 172.21.15.84:0/2820047995 reap_dead start 2023-10-16T11:26:12.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.532+0000 7f4e6b516640 1 -- 172.21.15.84:0/2820047995 shutdown_connections 2023-10-16T11:26:12.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.532+0000 7f4e6b516640 1 -- 172.21.15.84:0/2820047995 >> 172.21.15.84:0/2820047995 conn(0x7f4e640b0a80 msgr2=0x7f4e640b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:12.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.533+0000 7f4e6b516640 1 -- 172.21.15.84:0/2820047995 shutdown_connections 2023-10-16T11:26:12.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.533+0000 7f4e6b516640 1 -- 172.21.15.84:0/2820047995 wait complete. 2023-10-16T11:26:12.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.534+0000 7f4e6b516640 1 Processor -- start 2023-10-16T11:26:12.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.534+0000 7f4e6b516640 1 -- start start 2023-10-16T11:26:12.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.535+0000 7f4e6b516640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e500047c0 0x7f4e641f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:12.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.535+0000 7f4e6b516640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e640b36d0 0x7f4e6406a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:12.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.535+0000 7f4e6b516640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e64159960 0x7f4e64159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:12.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.535+0000 7f4e6b516640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e641f56f0 con 0x7f4e640b36d0 2023-10-16T11:26:12.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.535+0000 7f4e6b516640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6415a320 con 0x7f4e64159960 2023-10-16T11:26:12.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.535+0000 7f4e6b516640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e641f2330 con 0x7f4e500047c0 2023-10-16T11:26:12.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.536+0000 7f4e69d13640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e640b36d0 0x7f4e6406a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:12.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.536+0000 7f4e69d13640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e640b36d0 0x7f4e6406a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53862/0 (socket says 172.21.15.84:53862) 2023-10-16T11:26:12.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.536+0000 7f4e69d13640 1 -- 172.21.15.84:0/3292849675 learned_addr learned my addr 172.21.15.84:0/3292849675 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:12.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.536+0000 7f4e69d13640 1 -- 172.21.15.84:0/3292849675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e500047c0 msgr2=0x7f4e641f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:12.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.536+0000 7f4e69d13640 1 --2- 172.21.15.84:0/3292849675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e500047c0 0x7f4e641f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.536+0000 7f4e69d13640 1 -- 172.21.15.84:0/3292849675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e64159960 msgr2=0x7f4e64159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:12.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.536+0000 7f4e69d13640 1 --2- 172.21.15.84:0/3292849675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e64159960 0x7f4e64159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.536+0000 7f4e69d13640 1 -- 172.21.15.84:0/3292849675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6406b000 con 0x7f4e640b36d0 2023-10-16T11:26:12.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.536+0000 7f4e69d13640 1 --2- 172.21.15.84:0/3292849675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e640b36d0 0x7f4e6406a760 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 crypto rx=0x7f4e5402c3b0 tx=0x7f4e640acce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:12.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.537+0000 7f4e5affd640 1 -- 172.21.15.84:0/3292849675 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5403c040 con 0x7f4e640b36d0 2023-10-16T11:26:12.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.537+0000 7f4e5affd640 1 -- 172.21.15.84:0/3292849675 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5402dd30 con 0x7f4e640b36d0 2023-10-16T11:26:12.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.537+0000 7f4e5affd640 1 -- 172.21.15.84:0/3292849675 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5402dea0 con 0x7f4e640b36d0 2023-10-16T11:26:12.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.537+0000 7f4e6b516640 1 -- 172.21.15.84:0/3292849675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6406b2e0 con 0x7f4e640b36d0 2023-10-16T11:26:12.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.537+0000 7f4e6b516640 1 -- 172.21.15.84:0/3292849675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6406b8a0 con 0x7f4e640b36d0 2023-10-16T11:26:12.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.538+0000 7f4e6b516640 1 -- 172.21.15.84:0/3292849675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6404e9b0 con 0x7f4e640b36d0 2023-10-16T11:26:12.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.539+0000 7f4e5affd640 1 -- 172.21.15.84:0/3292849675 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 147) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f4e54032070 con 0x7f4e640b36d0 2023-10-16T11:26:12.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.540+0000 7f4e5affd640 1 --2- 172.21.15.84:0/3292849675 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f4e4806b0b0 0x7f4e4806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:12.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.540+0000 7f4e5affd640 1 -- 172.21.15.84:0/3292849675 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7031+0+0 (secure 0 0 0) 0x7f4e540ab9f0 con 0x7f4e640b36d0 2023-10-16T11:26:12.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.540+0000 7f4e6a514640 1 -- 172.21.15.84:0/3292849675 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f4e4806b0b0 msgr2=0x7f4e4806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3938045451 2023-10-16T11:26:12.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.540+0000 7f4e6a514640 1 --2- 172.21.15.84:0/3292849675 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f4e4806b0b0 0x7f4e4806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:26:12.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.544+0000 7f4e5affd640 1 -- 172.21.15.84:0/3292849675 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4e54076200 con 0x7f4e640b36d0 2023-10-16T11:26:12.595 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:12.593+0000 7fa00ca2c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:26:12.628 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:12.627+0000 7f2b48613040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:26:12.702 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:12.700+0000 7fa00ca2c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:26:12.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.737+0000 7f4e6b516640 1 -- 172.21.15.84:0/3292849675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4e640b4f10 con 0x7f4e640b36d0 2023-10-16T11:26:12.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.737+0000 7f4e5affd640 1 -- 172.21.15.84:0/3292849675 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v242) v1 ==== 107+0+11 (secure 0 0 0) 0x7f4e54079820 con 0x7f4e640b36d0 2023-10-16T11:26:12.739 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:26:12.740 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:12.739+0000 7f2b48613040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:26:12.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.740+0000 7f4e6b516640 1 -- 172.21.15.84:0/3292849675 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f4e4806b0b0 msgr2=0x7f4e4806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:26:12.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.740+0000 7f4e6b516640 1 --2- 172.21.15.84:0/3292849675 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f4e4806b0b0 0x7f4e4806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.740+0000 7f4e6b516640 1 -- 172.21.15.84:0/3292849675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e640b36d0 msgr2=0x7f4e6406a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:12.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.740+0000 7f4e6b516640 1 --2- 172.21.15.84:0/3292849675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e640b36d0 0x7f4e6406a760 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 crypto rx=0x7f4e5402c3b0 tx=0x7f4e640acce0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.740+0000 7f4e6b516640 1 -- 172.21.15.84:0/3292849675 shutdown_connections 2023-10-16T11:26:12.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.740+0000 7f4e6b516640 1 --2- 172.21.15.84:0/3292849675 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f4e4806b0b0 0x7f4e4806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.740+0000 7f4e6b516640 1 --2- 172.21.15.84:0/3292849675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e64159960 0x7f4e64159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.740+0000 7f4e6b516640 1 --2- 172.21.15.84:0/3292849675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e640b36d0 0x7f4e6406a760 unknown :-1 s=CLOSED pgs=3062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.740+0000 7f4e6b516640 1 --2- 172.21.15.84:0/3292849675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e500047c0 0x7f4e641f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:12.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.740+0000 7f4e6b516640 1 -- 172.21.15.84:0/3292849675 >> 172.21.15.84:0/3292849675 conn(0x7f4e640b0a80 msgr2=0x7f4e6409a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:12.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.740+0000 7f4e6b516640 1 -- 172.21.15.84:0/3292849675 shutdown_connections 2023-10-16T11:26:12.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:12.740+0000 7f4e6b516640 1 -- 172.21.15.84:0/3292849675 wait complete. 2023-10-16T11:26:12.758 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-16T11:26:12.858 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:12.855+0000 7fa00ca2c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:26:12.907 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:12.905+0000 7fa00ca2c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:26:12.907 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:12.905+0000 7f2b48613040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:26:12.958 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:12.957+0000 7f2b48613040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:26:13.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.005+0000 7f298b59e640 1 Processor -- start 2023-10-16T11:26:13.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.005+0000 7f298b59e640 1 -- start start 2023-10-16T11:26:13.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.007+0000 7f298b59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f298c160c80 0x7f298c161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:13.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.007+0000 7f298b59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f298c0ad7a0 0x7f298c070240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:13.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.007+0000 7f298b59e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f298c0ae1a0 con 0x7f298c0adc00 2023-10-16T11:26:13.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.007+0000 7f298b59e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298c0ae480 con 0x7f298c160c80 2023-10-16T11:26:13.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.007+0000 7f298b59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298c0ae760 con 0x7f298c0ad7a0 2023-10-16T11:26:13.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.008+0000 7f298a59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f298c160c80 0x7f298c161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:13.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.008+0000 7f298a59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f298c160c80 0x7f298c161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:60976/0 (socket says 172.21.15.84:60976) 2023-10-16T11:26:13.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.008+0000 7f298a59c640 1 -- 172.21.15.84:0/3266305124 learned_addr learned my addr 172.21.15.84:0/3266305124 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:13.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.008+0000 7f2983fff640 1 --2- 172.21.15.84:0/3266305124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f298c0ad7a0 0x7f298c070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:13.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.008+0000 7f298a59c640 1 -- 172.21.15.84:0/3266305124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f298c0ad7a0 msgr2=0x7f298c070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:13.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.008+0000 7f298a59c640 1 --2- 172.21.15.84:0/3266305124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f298c0ad7a0 0x7f298c070240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.008+0000 7f298a59c640 1 -- 172.21.15.84:0/3266305124 >> v1:172.21.15.157:6789/0 conn(0x7f298c0adc00 legacy=0x7f298c142400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:13.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.008+0000 7f298a59c640 1 -- 172.21.15.84:0/3266305124 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298c0aea40 con 0x7f298c160c80 2023-10-16T11:26:13.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.010+0000 7f298a59c640 1 --2- 172.21.15.84:0/3266305124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298c160c80 0x7f298c161060 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f297c009870 tx=0x7f297c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e4b0309c43e769f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:13.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.010+0000 7f2989d9b640 1 -- 172.21.15.84:0/3266305124 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c030840 con 0x7f298c160c80 2023-10-16T11:26:13.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.010+0000 7f2989d9b640 1 -- 172.21.15.84:0/3266305124 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c0309a0 con 0x7f298c160c80 2023-10-16T11:26:13.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.010+0000 7f2989d9b640 1 -- 172.21.15.84:0/3266305124 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c030d30 con 0x7f298c160c80 2023-10-16T11:26:13.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.011+0000 7f298b59e640 1 -- 172.21.15.84:0/3266305124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298c160c80 msgr2=0x7f298c161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:13.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.011+0000 7f298b59e640 1 --2- 172.21.15.84:0/3266305124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298c160c80 0x7f298c161060 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f297c009870 tx=0x7f297c02b980 comp rx=0 tx=0).stop 2023-10-16T11:26:13.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.011+0000 7f298b59e640 1 -- 172.21.15.84:0/3266305124 shutdown_connections 2023-10-16T11:26:13.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.011+0000 7f298b59e640 1 --2- 172.21.15.84:0/3266305124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f298c0ad7a0 0x7f298c070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.011+0000 7f298b59e640 1 --2- 172.21.15.84:0/3266305124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298c160c80 0x7f298c161060 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.011+0000 7f298b59e640 1 -- 172.21.15.84:0/3266305124 >> 172.21.15.84:0/3266305124 conn(0x7f298c0cc230 msgr2=0x7f298c0cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:13.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.011+0000 7f298b59e640 1 -- 172.21.15.84:0/3266305124 shutdown_connections 2023-10-16T11:26:13.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.011+0000 7f298b59e640 1 -- 172.21.15.84:0/3266305124 wait complete. 2023-10-16T11:26:13.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.013+0000 7f298b59e640 1 Processor -- start 2023-10-16T11:26:13.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.013+0000 7f298b59e640 1 -- start start 2023-10-16T11:26:13.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.014+0000 7f298b59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298c0ad7a0 0x7f298c0c5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:13.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.014+0000 7f298b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f298c0adc00 0x7f298c1516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:13.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.014+0000 7f298b59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f298c160c80 0x7f298c151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:13.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.014+0000 7f298b59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c15a480 con 0x7f298c0adc00 2023-10-16T11:26:13.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.014+0000 7f298b59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c159f00 con 0x7f298c0ad7a0 2023-10-16T11:26:13.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.014+0000 7f298b59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298c15a1a0 con 0x7f298c160c80 2023-10-16T11:26:13.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.014+0000 7f2983fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f298c0adc00 0x7f298c1516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:13.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.014+0000 7f2983fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f298c0adc00 0x7f298c1516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53878/0 (socket says 172.21.15.84:53878) 2023-10-16T11:26:13.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.014+0000 7f2983fff640 1 -- 172.21.15.84:0/110068719 learned_addr learned my addr 172.21.15.84:0/110068719 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:13.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.014+0000 7f298a59c640 1 --2- 172.21.15.84:0/110068719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298c0ad7a0 0x7f298c0c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:13.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.014+0000 7f298ad9d640 1 --2- 172.21.15.84:0/110068719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f298c160c80 0x7f298c151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:13.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.015+0000 7f2983fff640 1 -- 172.21.15.84:0/110068719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f298c160c80 msgr2=0x7f298c151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:13.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.015+0000 7f2983fff640 1 --2- 172.21.15.84:0/110068719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f298c160c80 0x7f298c151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.015+0000 7f2983fff640 1 -- 172.21.15.84:0/110068719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298c0ad7a0 msgr2=0x7f298c0c5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:13.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.015+0000 7f2983fff640 1 --2- 172.21.15.84:0/110068719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298c0ad7a0 0x7f298c0c5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.015+0000 7f2983fff640 1 -- 172.21.15.84:0/110068719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c0094d0 con 0x7f298c0adc00 2023-10-16T11:26:13.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.015+0000 7f298a59c640 1 --2- 172.21.15.84:0/110068719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298c0ad7a0 0x7f298c0c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:26:13.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.015+0000 7f2983fff640 1 --2- 172.21.15.84:0/110068719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f298c0adc00 0x7f298c1516a0 secure :-1 s=READY pgs=3063 cs=0 l=1 rev1=1 crypto rx=0x7f297400e9b0 tx=0x7f297400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:13.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.015+0000 7f2982ffd640 1 -- 172.21.15.84:0/110068719 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2974009670 con 0x7f298c0adc00 2023-10-16T11:26:13.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.015+0000 7f2982ffd640 1 -- 172.21.15.84:0/110068719 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297400c9a0 con 0x7f298c0adc00 2023-10-16T11:26:13.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.015+0000 7f2982ffd640 1 -- 172.21.15.84:0/110068719 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297400cd50 con 0x7f298c0adc00 2023-10-16T11:26:13.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.015+0000 7f298b59e640 1 -- 172.21.15.84:0/110068719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298c155e10 con 0x7f298c0adc00 2023-10-16T11:26:13.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.015+0000 7f298b59e640 1 -- 172.21.15.84:0/110068719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298c153530 con 0x7f298c0adc00 2023-10-16T11:26:13.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.016+0000 7f298b59e640 1 -- 172.21.15.84:0/110068719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298c155c70 con 0x7f298c0adc00 2023-10-16T11:26:13.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.017+0000 7f2982ffd640 1 -- 172.21.15.84:0/110068719 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 147) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f2974013470 con 0x7f298c0adc00 2023-10-16T11:26:13.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.018+0000 7f2982ffd640 1 --2- 172.21.15.84:0/110068719 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f296c06b0b0 0x7f296c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:13.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.018+0000 7f2982ffd640 1 -- 172.21.15.84:0/110068719 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7031+0+0 (secure 0 0 0) 0x7f297408a220 con 0x7f298c0adc00 2023-10-16T11:26:13.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.018+0000 7f298a59c640 1 -- 172.21.15.84:0/110068719 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f296c06b0b0 msgr2=0x7f296c06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3938045451 2023-10-16T11:26:13.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.019+0000 7f298a59c640 1 --2- 172.21.15.84:0/110068719 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f296c06b0b0 0x7f296c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:26:13.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.023+0000 7f2982ffd640 1 -- 172.21.15.84:0/110068719 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f298c155c70 con 0x7f298c0adc00 2023-10-16T11:26:13.028 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:13.025+0000 7fa00ca2c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:26:13.084 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:13.083+0000 7f2b48613040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:26:13.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.220+0000 7f298a59c640 1 -- 172.21.15.84:0/110068719 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f296c06b0b0 msgr2=0x7f296c06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3938045451 2023-10-16T11:26:13.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.220+0000 7f298a59c640 1 --2- 172.21.15.84:0/110068719 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f296c06b0b0 0x7f296c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:26:13.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.221+0000 7f298b59e640 1 -- 172.21.15.84:0/110068719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f298c155c70 con 0x7f298c0adc00 2023-10-16T11:26:13.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.222+0000 7f2982ffd640 1 -- 172.21.15.84:0/110068719 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v242) v1 ==== 103+0+11 (secure 0 0 0) 0x7f298c155c70 con 0x7f298c0adc00 2023-10-16T11:26:13.223 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:26:13.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.225+0000 7f298b59e640 1 -- 172.21.15.84:0/110068719 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f296c06b0b0 msgr2=0x7f296c06d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:26:13.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.225+0000 7f298b59e640 1 --2- 172.21.15.84:0/110068719 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f296c06b0b0 0x7f296c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.225+0000 7f298b59e640 1 -- 172.21.15.84:0/110068719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f298c0adc00 msgr2=0x7f298c1516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:13.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.225+0000 7f298b59e640 1 --2- 172.21.15.84:0/110068719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f298c0adc00 0x7f298c1516a0 secure :-1 s=READY pgs=3063 cs=0 l=1 rev1=1 crypto rx=0x7f297400e9b0 tx=0x7f297400ee70 comp rx=0 tx=0).stop 2023-10-16T11:26:13.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.225+0000 7f298b59e640 1 -- 172.21.15.84:0/110068719 shutdown_connections 2023-10-16T11:26:13.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.225+0000 7f298b59e640 1 --2- 172.21.15.84:0/110068719 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f296c06b0b0 0x7f296c06d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.225+0000 7f298b59e640 1 --2- 172.21.15.84:0/110068719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f298c160c80 0x7f298c151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.225+0000 7f298b59e640 1 --2- 172.21.15.84:0/110068719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f298c0adc00 0x7f298c1516a0 unknown :-1 s=CLOSED pgs=3063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.225+0000 7f298b59e640 1 --2- 172.21.15.84:0/110068719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298c0ad7a0 0x7f298c0c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.225+0000 7f298b59e640 1 -- 172.21.15.84:0/110068719 >> 172.21.15.84:0/110068719 conn(0x7f298c0cc230 msgr2=0x7f298c0c3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:13.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.225+0000 7f298b59e640 1 -- 172.21.15.84:0/110068719 shutdown_connections 2023-10-16T11:26:13.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.225+0000 7f298b59e640 1 -- 172.21.15.84:0/110068719 wait complete. 2023-10-16T11:26:13.245 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T11:26:13.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.449+0000 7f0772ebb640 1 Processor -- start 2023-10-16T11:26:13.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.449+0000 7f0772ebb640 1 -- start start 2023-10-16T11:26:13.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.451+0000 7f0772ebb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f076c15cc80 0x7f076c0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:13.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.451+0000 7f0772ebb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f076c0cdd60 0x7f076c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:13.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.451+0000 7f0772ebb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f076c071fa0 0x7f076c1607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:13.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.451+0000 7f0772ebb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f076c0d0500 con 0x7f076c0cdd60 2023-10-16T11:26:13.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.451+0000 7f0772ebb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f076c0ce1c0 con 0x7f076c071fa0 2023-10-16T11:26:13.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.451+0000 7f0772ebb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076c0d12f0 con 0x7f076c15cc80 2023-10-16T11:26:13.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.451+0000 7f07716b8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f076c0cdd60 0x7f076c071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:13.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.451+0000 7f07716b8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f076c0cdd60 0x7f076c071a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53894/0 (socket says 172.21.15.84:53894) 2023-10-16T11:26:13.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.451+0000 7f07716b8640 1 -- 172.21.15.84:0/2627781321 learned_addr learned my addr 172.21.15.84:0/2627781321 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:13.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.451+0000 7f07726ba640 1 --2- 172.21.15.84:0/2627781321 >> v2:172.21.15.157:3300/0 conn(0x7f076c071fa0 0x7f076c1607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:13.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.452+0000 7f0771eb9640 1 --2- 172.21.15.84:0/2627781321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f076c15cc80 0x7f076c0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:13.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.452+0000 7f07716b8640 1 -- 172.21.15.84:0/2627781321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f076c15cc80 msgr2=0x7f076c0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:13.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.452+0000 7f07716b8640 1 --2- 172.21.15.84:0/2627781321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f076c15cc80 0x7f076c0cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.452+0000 7f07716b8640 1 -- 172.21.15.84:0/2627781321 >> v2:172.21.15.157:3300/0 conn(0x7f076c071fa0 msgr2=0x7f076c1607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:13.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.452+0000 7f07716b8640 1 --2- 172.21.15.84:0/2627781321 >> v2:172.21.15.157:3300/0 conn(0x7f076c071fa0 0x7f076c1607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.452+0000 7f07716b8640 1 -- 172.21.15.84:0/2627781321 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076c072410 con 0x7f076c0cdd60 2023-10-16T11:26:13.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.453+0000 7f07716b8640 1 --2- 172.21.15.84:0/2627781321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f076c0cdd60 0x7f076c071a60 secure :-1 s=READY pgs=3064 cs=0 l=1 rev1=1 crypto rx=0x7f076c0b27f0 tx=0x7f075c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66370970b5befd1c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:13.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.453+0000 7f0770eb7640 1 -- 172.21.15.84:0/2627781321 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c031870 con 0x7f076c0cdd60 2023-10-16T11:26:13.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.453+0000 7f0770eb7640 1 -- 172.21.15.84:0/2627781321 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c0319d0 con 0x7f076c0cdd60 2023-10-16T11:26:13.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.453+0000 7f0770eb7640 1 -- 172.21.15.84:0/2627781321 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c031d60 con 0x7f076c0cdd60 2023-10-16T11:26:13.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.454+0000 7f0772ebb640 1 -- 172.21.15.84:0/2627781321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f076c0cdd60 msgr2=0x7f076c071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:13.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.454+0000 7f0772ebb640 1 --2- 172.21.15.84:0/2627781321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f076c0cdd60 0x7f076c071a60 secure :-1 s=READY pgs=3064 cs=0 l=1 rev1=1 crypto rx=0x7f076c0b27f0 tx=0x7f075c02b910 comp rx=0 tx=0).stop 2023-10-16T11:26:13.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.454+0000 7f0772ebb640 1 -- 172.21.15.84:0/2627781321 shutdown_connections 2023-10-16T11:26:13.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.454+0000 7f0772ebb640 1 --2- 172.21.15.84:0/2627781321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f076c0cdd60 0x7f076c071a60 unknown :-1 s=CLOSED pgs=3064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.454+0000 7f0772ebb640 1 --2- 172.21.15.84:0/2627781321 >> v2:172.21.15.157:3300/0 conn(0x7f076c071fa0 0x7f076c1607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.454+0000 7f0772ebb640 1 --2- 172.21.15.84:0/2627781321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f076c15cc80 0x7f076c0cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.454+0000 7f0772ebb640 1 -- 172.21.15.84:0/2627781321 >> 172.21.15.84:0/2627781321 conn(0x7f076c065ed0 msgr2=0x7f076c0662e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:13.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.455+0000 7f0772ebb640 1 -- 172.21.15.84:0/2627781321 shutdown_connections 2023-10-16T11:26:13.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.455+0000 7f0772ebb640 1 -- 172.21.15.84:0/2627781321 wait complete. 2023-10-16T11:26:13.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.457+0000 7f0772ebb640 1 Processor -- start 2023-10-16T11:26:13.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.458+0000 7f0772ebb640 1 -- start start 2023-10-16T11:26:13.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.459+0000 7f0772ebb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f076c071fa0 0x7f076c1f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:13.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.459+0000 7f0772ebb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f076c0cdd60 0x7f076c14fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:13.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.459+0000 7f0772ebb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076c15cc80 0x7f076c0ac990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:13.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.459+0000 7f0772ebb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c15d190 con 0x7f076c071fa0 2023-10-16T11:26:13.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.459+0000 7f0772ebb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c0aea70 con 0x7f076c15cc80 2023-10-16T11:26:13.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.459+0000 7f0772ebb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076c0ae790 con 0x7f076c0cdd60 2023-10-16T11:26:13.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.459+0000 7f0771eb9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f076c071fa0 0x7f076c1f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:13.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.459+0000 7f0771eb9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f076c071fa0 0x7f076c1f36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53910/0 (socket says 172.21.15.84:53910) 2023-10-16T11:26:13.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.459+0000 7f0771eb9640 1 -- 172.21.15.84:0/2483010390 learned_addr learned my addr 172.21.15.84:0/2483010390 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:13.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.460+0000 7f07726ba640 1 --2- 172.21.15.84:0/2483010390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076c15cc80 0x7f076c0ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:13.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.460+0000 7f0771eb9640 1 -- 172.21.15.84:0/2483010390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f076c0cdd60 msgr2=0x7f076c14fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:13.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.460+0000 7f0771eb9640 1 --2- 172.21.15.84:0/2483010390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f076c0cdd60 0x7f076c14fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.460+0000 7f0771eb9640 1 -- 172.21.15.84:0/2483010390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076c15cc80 msgr2=0x7f076c0ac990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:13.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.460+0000 7f0771eb9640 1 --2- 172.21.15.84:0/2483010390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076c15cc80 0x7f076c0ac990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.460+0000 7f0771eb9640 1 -- 172.21.15.84:0/2483010390 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c0094d0 con 0x7f076c071fa0 2023-10-16T11:26:13.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.460+0000 7f07726ba640 1 --2- 172.21.15.84:0/2483010390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076c15cc80 0x7f076c0ac990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:13.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.460+0000 7f0771eb9640 1 --2- 172.21.15.84:0/2483010390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f076c071fa0 0x7f076c1f36d0 secure :-1 s=READY pgs=3065 cs=0 l=1 rev1=1 crypto rx=0x7f076800d5f0 tx=0x7f076800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:13.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.460+0000 7f07627fc640 1 -- 172.21.15.84:0/2483010390 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680125f0 con 0x7f076c071fa0 2023-10-16T11:26:13.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.460+0000 7f07627fc640 1 -- 172.21.15.84:0/2483010390 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0768012750 con 0x7f076c071fa0 2023-10-16T11:26:13.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.460+0000 7f07627fc640 1 -- 172.21.15.84:0/2483010390 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768012b00 con 0x7f076c071fa0 2023-10-16T11:26:13.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.460+0000 7f0772ebb640 1 -- 172.21.15.84:0/2483010390 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f076c15d430 con 0x7f076c071fa0 2023-10-16T11:26:13.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.461+0000 7f0772ebb640 1 -- 172.21.15.84:0/2483010390 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f076c150220 con 0x7f076c071fa0 2023-10-16T11:26:13.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.462+0000 7f0772ebb640 1 -- 172.21.15.84:0/2483010390 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f076c04e9b0 con 0x7f076c071fa0 2023-10-16T11:26:13.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.462+0000 7f07627fc640 1 -- 172.21.15.84:0/2483010390 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 147) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f0768012c60 con 0x7f076c071fa0 2023-10-16T11:26:13.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.464+0000 7f07627fc640 1 --2- 172.21.15.84:0/2483010390 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f075006b0b0 0x7f075006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:13.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.464+0000 7f07716b8640 1 -- 172.21.15.84:0/2483010390 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f075006b0b0 msgr2=0x7f075006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3938045451 2023-10-16T11:26:13.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.464+0000 7f07627fc640 1 -- 172.21.15.84:0/2483010390 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7031+0+0 (secure 0 0 0) 0x7f076808a3e0 con 0x7f076c071fa0 2023-10-16T11:26:13.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.464+0000 7f07716b8640 1 --2- 172.21.15.84:0/2483010390 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f075006b0b0 0x7f075006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:26:13.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.467+0000 7f07627fc640 1 -- 172.21.15.84:0/2483010390 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0768054bf0 con 0x7f076c071fa0 2023-10-16T11:26:13.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.665+0000 7f07716b8640 1 -- 172.21.15.84:0/2483010390 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f075006b0b0 msgr2=0x7f075006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3938045451 2023-10-16T11:26:13.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.665+0000 7f07716b8640 1 --2- 172.21.15.84:0/2483010390 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f075006b0b0 0x7f075006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:26:13.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.686+0000 7f0772ebb640 1 -- 172.21.15.84:0/2483010390 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f076c0b6860 con 0x7f076c071fa0 2023-10-16T11:26:13.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.686+0000 7f07627fc640 1 -- 172.21.15.84:0/2483010390 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v242) v1 ==== 81+0+583 (secure 0 0 0) 0x7f0768058210 con 0x7f076c071fa0 2023-10-16T11:26:13.688 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6833/2963529201 2023-10-17T11:26:08.077644+0000 2023-10-16T11:26:13.688 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6832/2963529201 2023-10-17T11:26:08.077644+0000 2023-10-16T11:26:13.688 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2303342316 2023-10-17T11:26:08.077644+0000 2023-10-16T11:26:13.688 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/648102622 2023-10-17T11:26:08.077644+0000 2023-10-16T11:26:13.688 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/133211355 2023-10-17T11:26:08.077644+0000 2023-10-16T11:26:13.689 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1489017743 2023-10-17T11:26:08.077644+0000 2023-10-16T11:26:13.689 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1517291402 2023-10-17T11:26:08.077644+0000 2023-10-16T11:26:13.689 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1983743360 2023-10-17T11:26:08.077644+0000 2023-10-16T11:26:13.689 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1323010124 2023-10-17T11:26:08.077644+0000 2023-10-16T11:26:13.689 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/437386817 2023-10-17T11:26:08.077644+0000 2023-10-16T11:26:13.690 INFO:teuthology.orchestra.run.smithi084.stderr:listed 10 entries 2023-10-16T11:26:13.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.689+0000 7f0772ebb640 1 -- 172.21.15.84:0/2483010390 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f075006b0b0 msgr2=0x7f075006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:26:13.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.689+0000 7f0772ebb640 1 --2- 172.21.15.84:0/2483010390 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f075006b0b0 0x7f075006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.689+0000 7f0772ebb640 1 -- 172.21.15.84:0/2483010390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f076c071fa0 msgr2=0x7f076c1f36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:13.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.689+0000 7f0772ebb640 1 --2- 172.21.15.84:0/2483010390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f076c071fa0 0x7f076c1f36d0 secure :-1 s=READY pgs=3065 cs=0 l=1 rev1=1 crypto rx=0x7f076800d5f0 tx=0x7f076800dab0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.689+0000 7f0772ebb640 1 -- 172.21.15.84:0/2483010390 shutdown_connections 2023-10-16T11:26:13.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.689+0000 7f0772ebb640 1 --2- 172.21.15.84:0/2483010390 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7f075006b0b0 0x7f075006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.689+0000 7f0772ebb640 1 --2- 172.21.15.84:0/2483010390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076c15cc80 0x7f076c0ac990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.689+0000 7f0772ebb640 1 --2- 172.21.15.84:0/2483010390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f076c0cdd60 0x7f076c14fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.689+0000 7f0772ebb640 1 --2- 172.21.15.84:0/2483010390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f076c071fa0 0x7f076c1f36d0 unknown :-1 s=CLOSED pgs=3065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:13.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.689+0000 7f0772ebb640 1 -- 172.21.15.84:0/2483010390 >> 172.21.15.84:0/2483010390 conn(0x7f076c065ed0 msgr2=0x7f076c0b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:13.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.689+0000 7f0772ebb640 1 -- 172.21.15.84:0/2483010390 shutdown_connections 2023-10-16T11:26:13.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:13.689+0000 7f0772ebb640 1 -- 172.21.15.84:0/2483010390 wait complete. 2023-10-16T11:26:13.709 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T11:26:13.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:26:13.710 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T11:26:13.792 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T11:26:13.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:26:13.793 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T11:26:13.837 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:26:13.838 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:26:13.868 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:13.867+0000 7fa00ca2c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:26:13.894 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:26:13.895 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:26:13.895 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:26:13.947 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:26:13.948 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:26:13.948 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:26:13.974 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:13.974+0000 7f2b48613040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:26:14.003 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:26:14.004 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:26:14.179 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:14.177+0000 7fa00ca2c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:26:14.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.208+0000 7fc04b66f640 1 Processor -- start 2023-10-16T11:26:14.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.208+0000 7fc04b66f640 1 -- start start 2023-10-16T11:26:14.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.209+0000 7fc04b66f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc04415aaa0 0x7fc04415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:14.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.209+0000 7fc04b66f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0440ce390 0x7fc044070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:14.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.209+0000 7fc04b66f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04415ed60 con 0x7fc04415ae80 2023-10-16T11:26:14.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.210+0000 7fc04b66f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc04406b810 con 0x7fc04415aaa0 2023-10-16T11:26:14.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.210+0000 7fc04b66f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440707e0 con 0x7fc0440ce390 2023-10-16T11:26:14.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.210+0000 7fc04940b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc04415aaa0 0x7fc04415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:14.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.210+0000 7fc04940b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc04415aaa0 0x7fc04415e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53912/0 (socket says 172.21.15.84:53912) 2023-10-16T11:26:14.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.210+0000 7fc04940b640 1 -- 172.21.15.84:0/708191300 learned_addr learned my addr 172.21.15.84:0/708191300 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:14.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.210+0000 7fc04940b640 1 -- 172.21.15.84:0/708191300 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0440ce390 msgr2=0x7fc044070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:14.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.210+0000 7fc04940b640 1 --2- 172.21.15.84:0/708191300 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0440ce390 0x7fc044070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.210+0000 7fc04940b640 1 -- 172.21.15.84:0/708191300 >> v1:172.21.15.157:6789/0 conn(0x7fc04415ae80 legacy=0x7fc0440cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:14.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.210+0000 7fc04940b640 1 -- 172.21.15.84:0/708191300 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04415ef90 con 0x7fc04415aaa0 2023-10-16T11:26:14.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.211+0000 7fc04940b640 1 --2- 172.21.15.84:0/708191300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc04415aaa0 0x7fc04415e820 secure :-1 s=READY pgs=3066 cs=0 l=1 rev1=1 crypto rx=0x7fc0440cd990 tx=0x7fc04002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2683b522885285 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:14.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.212+0000 7fc03bfff640 1 -- 172.21.15.84:0/708191300 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc040031640 con 0x7fc04415aaa0 2023-10-16T11:26:14.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.212+0000 7fc03bfff640 1 -- 172.21.15.84:0/708191300 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0400317a0 con 0x7fc04415aaa0 2023-10-16T11:26:14.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.212+0000 7fc03bfff640 1 -- 172.21.15.84:0/708191300 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc040031b30 con 0x7fc04415aaa0 2023-10-16T11:26:14.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.212+0000 7fc04b66f640 1 -- 172.21.15.84:0/708191300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc04415aaa0 msgr2=0x7fc04415e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:14.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.212+0000 7fc04b66f640 1 --2- 172.21.15.84:0/708191300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc04415aaa0 0x7fc04415e820 secure :-1 s=READY pgs=3066 cs=0 l=1 rev1=1 crypto rx=0x7fc0440cd990 tx=0x7fc04002b8f0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.212+0000 7fc04b66f640 1 -- 172.21.15.84:0/708191300 shutdown_connections 2023-10-16T11:26:14.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.212+0000 7fc04b66f640 1 --2- 172.21.15.84:0/708191300 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0440ce390 0x7fc044070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.212+0000 7fc04b66f640 1 --2- 172.21.15.84:0/708191300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc04415aaa0 0x7fc04415e820 unknown :-1 s=CLOSED pgs=3066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.212+0000 7fc04b66f640 1 -- 172.21.15.84:0/708191300 >> 172.21.15.84:0/708191300 conn(0x7fc0440cb410 msgr2=0x7fc0440cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:14.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.212+0000 7fc04b66f640 1 -- 172.21.15.84:0/708191300 shutdown_connections 2023-10-16T11:26:14.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.212+0000 7fc04b66f640 1 -- 172.21.15.84:0/708191300 wait complete. 2023-10-16T11:26:14.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.214+0000 7fc04b66f640 1 Processor -- start 2023-10-16T11:26:14.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.214+0000 7fc04b66f640 1 -- start start 2023-10-16T11:26:14.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.215+0000 7fc04b66f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0440ce390 0x7fc044147200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:14.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.215+0000 7fc04b66f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc04415aaa0 0x7fc0440c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:14.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.215+0000 7fc04b66f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc04415ae80 0x7fc0440c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:14.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.215+0000 7fc04b66f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc044147740 con 0x7fc04415aaa0 2023-10-16T11:26:14.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.215+0000 7fc04b66f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440b3850 con 0x7fc0440ce390 2023-10-16T11:26:14.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.215+0000 7fc04b66f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04415cec0 con 0x7fc04415ae80 2023-10-16T11:26:14.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.215+0000 7fc048c0a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc04415aaa0 0x7fc0440c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:14.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.215+0000 7fc048c0a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc04415aaa0 0x7fc0440c74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53916/0 (socket says 172.21.15.84:53916) 2023-10-16T11:26:14.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.215+0000 7fc048c0a640 1 -- 172.21.15.84:0/4082385294 learned_addr learned my addr 172.21.15.84:0/4082385294 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:14.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.216+0000 7fc04940b640 1 --2- 172.21.15.84:0/4082385294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0440ce390 0x7fc044147200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:14.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.216+0000 7fc048c0a640 1 -- 172.21.15.84:0/4082385294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc04415ae80 msgr2=0x7fc0440c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:14.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.216+0000 7fc049c0c640 1 --2- 172.21.15.84:0/4082385294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc04415ae80 0x7fc0440c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:14.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.216+0000 7fc048c0a640 1 --2- 172.21.15.84:0/4082385294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc04415ae80 0x7fc0440c7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.216+0000 7fc048c0a640 1 -- 172.21.15.84:0/4082385294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0440ce390 msgr2=0x7fc044147200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:14.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.216+0000 7fc048c0a640 1 --2- 172.21.15.84:0/4082385294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0440ce390 0x7fc044147200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.216+0000 7fc048c0a640 1 -- 172.21.15.84:0/4082385294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0400094d0 con 0x7fc04415aaa0 2023-10-16T11:26:14.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.216+0000 7fc048c0a640 1 --2- 172.21.15.84:0/4082385294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc04415aaa0 0x7fc0440c74c0 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 crypto rx=0x7fc03400b3c0 tx=0x7fc03400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:14.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.216+0000 7fc039ffb640 1 -- 172.21.15.84:0/4082385294 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034011830 con 0x7fc04415aaa0 2023-10-16T11:26:14.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.216+0000 7fc039ffb640 1 -- 172.21.15.84:0/4082385294 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc034011990 con 0x7fc04415aaa0 2023-10-16T11:26:14.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.216+0000 7fc039ffb640 1 -- 172.21.15.84:0/4082385294 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034011d80 con 0x7fc04415aaa0 2023-10-16T11:26:14.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.216+0000 7fc04b66f640 1 -- 172.21.15.84:0/4082385294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0440c8080 con 0x7fc04415aaa0 2023-10-16T11:26:14.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.216+0000 7fc04b66f640 1 -- 172.21.15.84:0/4082385294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04415c870 con 0x7fc04415aaa0 2023-10-16T11:26:14.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.218+0000 7fc04b66f640 1 -- 172.21.15.84:0/4082385294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc04415c6d0 con 0x7fc04415aaa0 2023-10-16T11:26:14.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.219+0000 7fc039ffb640 1 -- 172.21.15.84:0/4082385294 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 147) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fc03401b460 con 0x7fc04415aaa0 2023-10-16T11:26:14.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.220+0000 7fc039ffb640 1 --2- 172.21.15.84:0/4082385294 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7fc02806b0b0 0x7fc02806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:14.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.220+0000 7fc04940b640 1 -- 172.21.15.84:0/4082385294 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7fc02806b0b0 msgr2=0x7fc02806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3938045451 2023-10-16T11:26:14.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.220+0000 7fc039ffb640 1 -- 172.21.15.84:0/4082385294 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7031+0+0 (secure 0 0 0) 0x7fc034089b70 con 0x7fc04415aaa0 2023-10-16T11:26:14.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.220+0000 7fc04940b640 1 --2- 172.21.15.84:0/4082385294 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7fc02806b0b0 0x7fc02806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:26:14.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.224+0000 7fc039ffb640 1 -- 172.21.15.84:0/4082385294 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc04415c6d0 con 0x7fc04415aaa0 2023-10-16T11:26:14.237 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:14.235+0000 7fa00ca2c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:26:14.297 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.296+0000 7f2b48613040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:26:14.355 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:14.353+0000 7fa00ca2c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:26:14.357 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.356+0000 7f2b48613040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:26:14.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.421+0000 7fc04940b640 1 -- 172.21.15.84:0/4082385294 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7fc02806b0b0 msgr2=0x7fc02806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/3938045451 2023-10-16T11:26:14.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.421+0000 7fc04940b640 1 --2- 172.21.15.84:0/4082385294 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7fc02806b0b0 0x7fc02806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:26:14.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.458+0000 7fc04b66f640 1 -- 172.21.15.84:0/4082385294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc04415c6d0 con 0x7fc04415aaa0 2023-10-16T11:26:14.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.459+0000 7fc039ffb640 1 -- 172.21.15.84:0/4082385294 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+6155 (secure 0 0 0) 0x7fc04415c6d0 con 0x7fc04415aaa0 2023-10-16T11:26:14.463 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:14.463 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":137,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2516077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2516077743},{"type":"v1","addr":"172.21.15.157:6833","nonce":2516077743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1527104462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1527104462},{"type":"v1","addr":"172.21.15.157:6835","nonce":1527104462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:25:36.605068+0000","modified":"2023-10-16T11:25:39.675208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12186":{"gid":12186,"name":"a","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/3547503033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3547503033},{"type":"v1","addr":"172.21.15.84:6837","nonce":3547503033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:25:50.739234+0000","modified":"2023-10-16T11:25:53.792071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12183":{"gid":12183,"name":"c","rank":0,"incarnation":133,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/369067539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":369067539},{"type":"v1","addr":"172.21.15.84:6835","nonce":369067539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-16T11:26:14.465 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 137 2023-10-16T11:26:14.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.462+0000 7fc04b66f640 1 -- 172.21.15.84:0/4082385294 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7fc02806b0b0 msgr2=0x7fc02806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:26:14.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.462+0000 7fc04b66f640 1 --2- 172.21.15.84:0/4082385294 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7fc02806b0b0 0x7fc02806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.462+0000 7fc04b66f640 1 -- 172.21.15.84:0/4082385294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc04415aaa0 msgr2=0x7fc0440c74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:14.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.462+0000 7fc04b66f640 1 --2- 172.21.15.84:0/4082385294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc04415aaa0 0x7fc0440c74c0 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 crypto rx=0x7fc03400b3c0 tx=0x7fc03400b880 comp rx=0 tx=0).stop 2023-10-16T11:26:14.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.462+0000 7fc04b66f640 1 -- 172.21.15.84:0/4082385294 shutdown_connections 2023-10-16T11:26:14.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.462+0000 7fc04b66f640 1 --2- 172.21.15.84:0/4082385294 >> [v2:172.21.15.84:6832/3938045451,v1:172.21.15.84:6833/3938045451] conn(0x7fc02806b0b0 0x7fc02806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.462+0000 7fc04b66f640 1 --2- 172.21.15.84:0/4082385294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc04415ae80 0x7fc0440c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.462+0000 7fc04b66f640 1 --2- 172.21.15.84:0/4082385294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc04415aaa0 0x7fc0440c74c0 unknown :-1 s=CLOSED pgs=3067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.462+0000 7fc04b66f640 1 --2- 172.21.15.84:0/4082385294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0440ce390 0x7fc044147200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.462+0000 7fc04b66f640 1 -- 172.21.15.84:0/4082385294 >> 172.21.15.84:0/4082385294 conn(0x7fc0440cb410 msgr2=0x7fc04406ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:14.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.462+0000 7fc04b66f640 1 -- 172.21.15.84:0/4082385294 shutdown_connections 2023-10-16T11:26:14.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.462+0000 7fc04b66f640 1 -- 172.21.15.84:0/4082385294 wait complete. 2023-10-16T11:26:14.473 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.472+0000 7f2b48613040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:26:14.480 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:26:14.561 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.560+0000 7f2accd51640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.561 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.560+0000 7f2accd51640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.562 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.560+0000 7f2accd51640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.562 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.560+0000 7f2accd51640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.562 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.560+0000 7f2accd51640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.563 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.561+0000 7f2ac8d49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.563 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.561+0000 7f2ac8d49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.563 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.561+0000 7f2ac8d49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.564 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.561+0000 7f2ac8d49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.564 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.561+0000 7f2ac8d49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.575 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.575+0000 7f2accd51640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.576 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.575+0000 7f2accd51640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.576 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.575+0000 7f2accd51640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.576 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.575+0000 7f2accd51640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.576 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.575+0000 7f2accd51640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.577 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.576+0000 7f2ac8d49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.577 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.576+0000 7f2ac8d49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.577 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.576+0000 7f2ac8d49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.578 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.576+0000 7f2ac8d49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.578 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:14.576+0000 7f2ac8d49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:14.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.669+0000 7f90676e0640 1 Processor -- start 2023-10-16T11:26:14.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.669+0000 7f90676e0640 1 -- start start 2023-10-16T11:26:14.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.670+0000 7f90676e0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f906015aaa0 0x7f906015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:14.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.670+0000 7f90676e0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f90600ce390 0x7f9060070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:14.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.670+0000 7f90676e0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906015ed60 con 0x7f906015ae80 2023-10-16T11:26:14.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.670+0000 7f90676e0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f906006b810 con 0x7f906015aaa0 2023-10-16T11:26:14.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.670+0000 7f90676e0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90600707e0 con 0x7f90600ce390 2023-10-16T11:26:14.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.670+0000 7f906547c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f906015aaa0 0x7f906015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:14.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.670+0000 7f906547c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f906015aaa0 0x7f906015e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54064/0 (socket says 172.21.15.84:54064) 2023-10-16T11:26:14.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.670+0000 7f906547c640 1 -- 172.21.15.84:0/2537620426 learned_addr learned my addr 172.21.15.84:0/2537620426 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:14.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.671+0000 7f906547c640 1 -- 172.21.15.84:0/2537620426 >> v2:172.21.15.157:3300/0 conn(0x7f90600ce390 msgr2=0x7f9060070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:14.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.671+0000 7f9065c7d640 1 --2- 172.21.15.84:0/2537620426 >> v2:172.21.15.157:3300/0 conn(0x7f90600ce390 0x7f9060070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:14.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.671+0000 7f906547c640 1 --2- 172.21.15.84:0/2537620426 >> v2:172.21.15.157:3300/0 conn(0x7f90600ce390 0x7f9060070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.671+0000 7f906547c640 1 -- 172.21.15.84:0/2537620426 >> v1:172.21.15.157:6789/0 conn(0x7f906015ae80 legacy=0x7f90600cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:14.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.671+0000 7f906547c640 1 -- 172.21.15.84:0/2537620426 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906015ef90 con 0x7f906015aaa0 2023-10-16T11:26:14.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.672+0000 7f906547c640 1 --2- 172.21.15.84:0/2537620426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f906015aaa0 0x7f906015e820 secure :-1 s=READY pgs=3083 cs=0 l=1 rev1=1 crypto rx=0x7f90600c2490 tx=0x7f905c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67fbfbb800e1d45 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:14.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.672+0000 7f9057fff640 1 -- 172.21.15.84:0/2537620426 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c031870 con 0x7f906015aaa0 2023-10-16T11:26:14.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.672+0000 7f9057fff640 1 -- 172.21.15.84:0/2537620426 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905c0319d0 con 0x7f906015aaa0 2023-10-16T11:26:14.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.672+0000 7f9057fff640 1 -- 172.21.15.84:0/2537620426 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c031d60 con 0x7f906015aaa0 2023-10-16T11:26:14.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.672+0000 7f90676e0640 1 -- 172.21.15.84:0/2537620426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f906015aaa0 msgr2=0x7f906015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:14.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.672+0000 7f90676e0640 1 --2- 172.21.15.84:0/2537620426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f906015aaa0 0x7f906015e820 secure :-1 s=READY pgs=3083 cs=0 l=1 rev1=1 crypto rx=0x7f90600c2490 tx=0x7f905c02b910 comp rx=0 tx=0).stop 2023-10-16T11:26:14.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.673+0000 7f90676e0640 1 -- 172.21.15.84:0/2537620426 shutdown_connections 2023-10-16T11:26:14.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.673+0000 7f90676e0640 1 --2- 172.21.15.84:0/2537620426 >> v2:172.21.15.157:3300/0 conn(0x7f90600ce390 0x7f9060070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.673+0000 7f90676e0640 1 --2- 172.21.15.84:0/2537620426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f906015aaa0 0x7f906015e820 unknown :-1 s=CLOSED pgs=3083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.673+0000 7f90676e0640 1 -- 172.21.15.84:0/2537620426 >> 172.21.15.84:0/2537620426 conn(0x7f90600cb410 msgr2=0x7f90600cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:14.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.673+0000 7f90676e0640 1 -- 172.21.15.84:0/2537620426 shutdown_connections 2023-10-16T11:26:14.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.673+0000 7f90676e0640 1 -- 172.21.15.84:0/2537620426 wait complete. 2023-10-16T11:26:14.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.674+0000 7f90676e0640 1 Processor -- start 2023-10-16T11:26:14.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.675+0000 7f90676e0640 1 -- start start 2023-10-16T11:26:14.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.675+0000 7f90676e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90600ce390 0x7f906015bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:14.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.676+0000 7f90676e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f906015aaa0 0x7f906015db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:14.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.676+0000 7f90676e0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f906015ae80 0x7f9060147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:14.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.676+0000 7f90676e0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906015c020 con 0x7f906015aaa0 2023-10-16T11:26:14.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.676+0000 7f90676e0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906015c2a0 con 0x7f906015ae80 2023-10-16T11:26:14.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.676+0000 7f90676e0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906015c580 con 0x7f90600ce390 2023-10-16T11:26:14.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.676+0000 7f9064c7b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f906015aaa0 0x7f906015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:14.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.676+0000 7f9064c7b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f906015aaa0 0x7f906015db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54080/0 (socket says 172.21.15.84:54080) 2023-10-16T11:26:14.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.676+0000 7f9064c7b640 1 -- 172.21.15.84:0/134852409 learned_addr learned my addr 172.21.15.84:0/134852409 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:14.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.676+0000 7f906547c640 1 --2- 172.21.15.84:0/134852409 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90600ce390 0x7f906015bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:14.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.676+0000 7f9065c7d640 1 --2- 172.21.15.84:0/134852409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f906015ae80 0x7f9060147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:14.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.676+0000 7f9064c7b640 1 -- 172.21.15.84:0/134852409 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90600ce390 msgr2=0x7f906015bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:14.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.676+0000 7f9064c7b640 1 --2- 172.21.15.84:0/134852409 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90600ce390 0x7f906015bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.676+0000 7f9064c7b640 1 -- 172.21.15.84:0/134852409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f906015ae80 msgr2=0x7f9060147240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:14.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.676+0000 7f9064c7b640 1 --2- 172.21.15.84:0/134852409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f906015ae80 0x7f9060147240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.676+0000 7f9064c7b640 1 -- 172.21.15.84:0/134852409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905c0094d0 con 0x7f906015aaa0 2023-10-16T11:26:14.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.677+0000 7f9064c7b640 1 --2- 172.21.15.84:0/134852409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f906015aaa0 0x7f906015db80 secure :-1 s=READY pgs=3084 cs=0 l=1 rev1=1 crypto rx=0x7f905000b3c0 tx=0x7f905000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:14.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.677+0000 7f9055ffb640 1 -- 172.21.15.84:0/134852409 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050011830 con 0x7f906015aaa0 2023-10-16T11:26:14.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.677+0000 7f9055ffb640 1 -- 172.21.15.84:0/134852409 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9050011990 con 0x7f906015aaa0 2023-10-16T11:26:14.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.677+0000 7f9055ffb640 1 -- 172.21.15.84:0/134852409 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050011d60 con 0x7f906015aaa0 2023-10-16T11:26:14.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.677+0000 7f90676e0640 1 -- 172.21.15.84:0/134852409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906015c860 con 0x7f906015aaa0 2023-10-16T11:26:14.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.677+0000 7f90676e0640 1 -- 172.21.15.84:0/134852409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906015ce20 con 0x7f906015aaa0 2023-10-16T11:26:14.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.678+0000 7f90676e0640 1 -- 172.21.15.84:0/134852409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f906004e160 con 0x7f906015aaa0 2023-10-16T11:26:14.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.679+0000 7f9055ffb640 1 -- 172.21.15.84:0/134852409 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 148) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f905001b460 con 0x7f906015aaa0 2023-10-16T11:26:14.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.680+0000 7f9055ffb640 1 -- 172.21.15.84:0/134852409 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7461+0+0 (secure 0 0 0) 0x7f9050089660 con 0x7f906015aaa0 2023-10-16T11:26:14.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.684+0000 7f9055ffb640 1 -- 172.21.15.84:0/134852409 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9050054390 con 0x7f906015aaa0 2023-10-16T11:26:14.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.878+0000 7f90676e0640 1 -- 172.21.15.84:0/134852409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90600cdc60 con 0x7f906015aaa0 2023-10-16T11:26:14.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.882+0000 7f9055ffb640 1 -- 172.21.15.84:0/134852409 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v243) v1 ==== 74+0+21450 (secure 0 0 0) 0x7f90500579b0 con 0x7f906015aaa0 2023-10-16T11:26:14.884 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:14.884 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":243,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:26:14.475537+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:25:34.443897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:25:35.635615+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:25:49.001107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:25:49.760637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2606713455":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/488945617":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/4036998934":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6832/3938045451":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/570042115":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1311117536":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6833/2963529201":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/95207886":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6832/2963529201":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/388205040":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/2303342316":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/648102622":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/133211355":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1263886961":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1489017743":"2023-10-17T11:26:08.077644+0000","172.21.15.84:6833/3938045451":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1517291402":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1983743360":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1323010124":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/437386817":"2023-10-17T11:26:08.077644+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:26:14.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.884+0000 7f90676e0640 1 -- 172.21.15.84:0/134852409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f906015aaa0 msgr2=0x7f906015db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:14.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.884+0000 7f90676e0640 1 --2- 172.21.15.84:0/134852409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f906015aaa0 0x7f906015db80 secure :-1 s=READY pgs=3084 cs=0 l=1 rev1=1 crypto rx=0x7f905000b3c0 tx=0x7f905000b880 comp rx=0 tx=0).stop 2023-10-16T11:26:14.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.884+0000 7f90676e0640 1 -- 172.21.15.84:0/134852409 shutdown_connections 2023-10-16T11:26:14.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.884+0000 7f90676e0640 1 --2- 172.21.15.84:0/134852409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f906015ae80 0x7f9060147240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.884+0000 7f90676e0640 1 --2- 172.21.15.84:0/134852409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f906015aaa0 0x7f906015db80 unknown :-1 s=CLOSED pgs=3084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.884+0000 7f90676e0640 1 --2- 172.21.15.84:0/134852409 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90600ce390 0x7f906015bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:14.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.884+0000 7f90676e0640 1 -- 172.21.15.84:0/134852409 >> 172.21.15.84:0/134852409 conn(0x7f90600cb410 msgr2=0x7f90600cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:14.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.884+0000 7f90676e0640 1 -- 172.21.15.84:0/134852409 shutdown_connections 2023-10-16T11:26:14.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:14.884+0000 7f90676e0640 1 -- 172.21.15.84:0/134852409 wait complete. 2023-10-16T11:26:14.904 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:25:34.443897+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '236', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:25:35.635615+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '237', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-16T11:25:49.001107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '240', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-16T11:25:49.760637+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '241', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:26:14.905 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:26:15.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.113+0000 7f3bbc915640 1 Processor -- start 2023-10-16T11:26:15.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.113+0000 7f3bbc915640 1 -- start start 2023-10-16T11:26:15.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.114+0000 7f3bbc915640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3bb806cca0 0x7f3bb806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:15.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.114+0000 7f3bbc915640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb8164f80 con 0x7f3bb80afbc0 2023-10-16T11:26:15.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.114+0000 7f3bbc915640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb8165260 con 0x7f3bb806ed80 2023-10-16T11:26:15.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.114+0000 7f3bbc915640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb806aeb0 con 0x7f3bb806cca0 2023-10-16T11:26:15.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.115+0000 7f3bb6d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3bb80afbc0 0x7f3bb806c590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40396/0 (socket says 172.21.15.84:40396) 2023-10-16T11:26:15.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.115+0000 7f3bb6d9d640 1 -- 172.21.15.84:0/3334234838 learned_addr learned my addr 172.21.15.84:0/3334234838 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:15.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.115+0000 7f3bb659c640 1 --2- 172.21.15.84:0/3334234838 >> v2:172.21.15.157:3300/0 conn(0x7f3bb806cca0 0x7f3bb806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:15.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.115+0000 7f3bb5d9b640 1 -- 172.21.15.84:0/3334234838 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 47779247 0 0) 0x7f3bb8164f80 con 0x7f3bb80afbc0 2023-10-16T11:26:15.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.115+0000 7f3bb5d9b640 1 -- 172.21.15.84:0/3334234838 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ba4003520 con 0x7f3bb80afbc0 2023-10-16T11:26:15.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.115+0000 7f3bb5d9b640 1 -- 172.21.15.84:0/3334234838 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2036427236 0 0) 0x7f3ba4003520 con 0x7f3bb80afbc0 2023-10-16T11:26:15.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.115+0000 7f3bb5d9b640 1 -- 172.21.15.84:0/3334234838 >> v2:172.21.15.157:3300/0 conn(0x7f3bb806cca0 msgr2=0x7f3bb806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:15.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.115+0000 7f3bb5d9b640 1 --2- 172.21.15.84:0/3334234838 >> v2:172.21.15.157:3300/0 conn(0x7f3bb806cca0 0x7f3bb806f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.115+0000 7f3bb5d9b640 1 -- 172.21.15.84:0/3334234838 >> v1:172.21.15.157:6789/0 conn(0x7f3bb806ed80 legacy=0x7f3bb80ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:15.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.115+0000 7f3bb5d9b640 1 -- 172.21.15.84:0/3334234838 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb8165540 con 0x7f3bb80afbc0 2023-10-16T11:26:15.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.115+0000 7f3bb5d9b640 1 -- 172.21.15.84:0/3334234838 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f3ba80019f0 con 0x7f3bb80afbc0 2023-10-16T11:26:15.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.115+0000 7f3bb5d9b640 1 -- 172.21.15.84:0/3334234838 >> v1:172.21.15.84:6789/0 conn(0x7f3bb80afbc0 legacy=0x7f3bb806c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:15.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.115+0000 7f3bb5d9b640 1 --2- 172.21.15.84:0/3334234838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ba4003970 0x7f3ba4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:15.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.116+0000 7f3bb5d9b640 1 --2- 172.21.15.84:0/3334234838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ba40047c0 0x7f3ba4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:15.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.116+0000 7f3bb5d9b640 1 --2- 172.21.15.84:0/3334234838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ba4005180 0x7f3ba400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:15.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.116+0000 7f3bb5d9b640 1 -- 172.21.15.84:0/3334234838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba400bff0 con 0x7f3ba40047c0 2023-10-16T11:26:15.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.116+0000 7f3bb5d9b640 1 -- 172.21.15.84:0/3334234838 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba400c160 con 0x7f3ba4003970 2023-10-16T11:26:15.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.116+0000 7f3bb5d9b640 1 -- 172.21.15.84:0/3334234838 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba400c320 con 0x7f3ba4005180 2023-10-16T11:26:15.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.116+0000 7f3bb659c640 1 --2- 172.21.15.84:0/3334234838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ba40047c0 0x7f3ba4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:15.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.116+0000 7f3bb659c640 1 -- 172.21.15.84:0/3334234838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ba4005180 msgr2=0x7f3ba400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:15.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.116+0000 7f3bb659c640 1 --2- 172.21.15.84:0/3334234838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ba4005180 0x7f3ba400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.116+0000 7f3bb659c640 1 -- 172.21.15.84:0/3334234838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ba4003970 msgr2=0x7f3ba4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:15.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.116+0000 7f3bb659c640 1 --2- 172.21.15.84:0/3334234838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ba4003970 0x7f3ba4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.116+0000 7f3bb659c640 1 -- 172.21.15.84:0/3334234838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba4003520 con 0x7f3ba40047c0 2023-10-16T11:26:15.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.116+0000 7f3bb759e640 1 -- 172.21.15.84:0/3334234838 reap_dead start 2023-10-16T11:26:15.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.117+0000 7f3bb659c640 1 --2- 172.21.15.84:0/3334234838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ba40047c0 0x7f3ba4004c40 secure :-1 s=READY pgs=3086 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009a90 tx=0x7f3ba40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39a47017cf1a410 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:15.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.117+0000 7f3bb5d9b640 1 -- 172.21.15.84:0/3334234838 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba002c680 con 0x7f3ba40047c0 2023-10-16T11:26:15.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.118+0000 7f3bb5d9b640 1 -- 172.21.15.84:0/3334234838 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba002c7e0 con 0x7f3ba40047c0 2023-10-16T11:26:15.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.118+0000 7f3bb5d9b640 1 -- 172.21.15.84:0/3334234838 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba002cb90 con 0x7f3ba40047c0 2023-10-16T11:26:15.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.118+0000 7f3bbc915640 1 -- 172.21.15.84:0/3334234838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ba40047c0 msgr2=0x7f3ba4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:15.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.118+0000 7f3bbc915640 1 --2- 172.21.15.84:0/3334234838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ba40047c0 0x7f3ba4004c40 secure :-1 s=READY pgs=3086 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009a90 tx=0x7f3ba40057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.118+0000 7f3bbc915640 1 -- 172.21.15.84:0/3334234838 shutdown_connections 2023-10-16T11:26:15.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.118+0000 7f3bbc915640 1 --2- 172.21.15.84:0/3334234838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ba40047c0 0x7f3ba4004c40 unknown :-1 s=CLOSED pgs=3086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.118+0000 7f3bbc915640 1 -- 172.21.15.84:0/3334234838 >> 172.21.15.84:0/3334234838 conn(0x7f3bb80b23b0 msgr2=0x7f3bb80b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:15.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.118+0000 7f3bbc915640 1 -- 172.21.15.84:0/3334234838 shutdown_connections 2023-10-16T11:26:15.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.118+0000 7f3bbc915640 1 -- 172.21.15.84:0/3334234838 wait complete. 2023-10-16T11:26:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.120+0000 7f3bbc915640 1 Processor -- start 2023-10-16T11:26:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.120+0000 7f3bbc915640 1 -- start start 2023-10-16T11:26:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.121+0000 7f3bbc915640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ba40047c0 0x7f3bb81f9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.121+0000 7f3bbc915640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bb80c5d50 0x7f3bb80c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.121+0000 7f3bbc915640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bb8161710 0x7f3bb8161b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.121+0000 7f3bbc915640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb8163010 con 0x7f3bb8161710 2023-10-16T11:26:15.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.121+0000 7f3bbc915640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80c5a60 con 0x7f3ba40047c0 2023-10-16T11:26:15.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.121+0000 7f3bbc915640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80c6710 con 0x7f3bb80c5d50 2023-10-16T11:26:15.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.121+0000 7f3bb759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bb8161710 0x7f3bb8161b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:15.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.121+0000 7f3bb759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bb8161710 0x7f3bb8161b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54108/0 (socket says 172.21.15.84:54108) 2023-10-16T11:26:15.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.121+0000 7f3bb759e640 1 -- 172.21.15.84:0/1041188546 learned_addr learned my addr 172.21.15.84:0/1041188546 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:15.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.122+0000 7f3bb759e640 1 -- 172.21.15.84:0/1041188546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bb80c5d50 msgr2=0x7f3bb80c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:15.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.122+0000 7f3bb659c640 1 --2- 172.21.15.84:0/1041188546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bb80c5d50 0x7f3bb80c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:15.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.122+0000 7f3bb6d9d640 1 --2- 172.21.15.84:0/1041188546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ba40047c0 0x7f3bb81f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:15.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.122+0000 7f3bb759e640 1 --2- 172.21.15.84:0/1041188546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bb80c5d50 0x7f3bb80c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.122+0000 7f3bb759e640 1 -- 172.21.15.84:0/1041188546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ba40047c0 msgr2=0x7f3bb81f9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:15.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.122+0000 7f3bb759e640 1 --2- 172.21.15.84:0/1041188546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ba40047c0 0x7f3bb81f9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.122+0000 7f3bb759e640 1 -- 172.21.15.84:0/1041188546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb81620d0 con 0x7f3bb8161710 2023-10-16T11:26:15.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.122+0000 7f3bb759e640 1 --2- 172.21.15.84:0/1041188546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bb8161710 0x7f3bb8161b90 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80671c0 tx=0x7f3bb000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:15.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.122+0000 7f3baf7fe640 1 -- 172.21.15.84:0/1041188546 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0004220 con 0x7f3bb8161710 2023-10-16T11:26:15.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.122+0000 7f3baf7fe640 1 -- 172.21.15.84:0/1041188546 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb0004380 con 0x7f3bb8161710 2023-10-16T11:26:15.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.122+0000 7f3baf7fe640 1 -- 172.21.15.84:0/1041188546 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0007310 con 0x7f3bb8161710 2023-10-16T11:26:15.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.122+0000 7f3bbc915640 1 -- 172.21.15.84:0/1041188546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb8162390 con 0x7f3bb8161710 2023-10-16T11:26:15.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.122+0000 7f3bbc915640 1 -- 172.21.15.84:0/1041188546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb8162950 con 0x7f3bb8161710 2023-10-16T11:26:15.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.124+0000 7f3bbc915640 1 -- 172.21.15.84:0/1041188546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb804e9b0 con 0x7f3bb8161710 2023-10-16T11:26:15.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.124+0000 7f3baf7fe640 1 -- 172.21.15.84:0/1041188546 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 148) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f3bb0007470 con 0x7f3bb8161710 2023-10-16T11:26:15.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.126+0000 7f3baf7fe640 1 -- 172.21.15.84:0/1041188546 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7461+0+0 (secure 0 0 0) 0x7f3bb00886b0 con 0x7f3bb8161710 2023-10-16T11:26:15.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.130+0000 7f3baf7fe640 1 -- 172.21.15.84:0/1041188546 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3bb00534d0 con 0x7f3bb8161710 2023-10-16T11:26:15.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.322+0000 7f3bbc915640 1 -- 172.21.15.84:0/1041188546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3bb8074220 con 0x7f3bb8161710 2023-10-16T11:26:15.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.323+0000 7f3baf7fe640 1 -- 172.21.15.84:0/1041188546 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v243) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3bb000cea0 con 0x7f3bb8161710 2023-10-16T11:26:15.325 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:26:15.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.326+0000 7f3bbc915640 1 -- 172.21.15.84:0/1041188546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bb8161710 msgr2=0x7f3bb8161b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:15.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.326+0000 7f3bbc915640 1 --2- 172.21.15.84:0/1041188546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bb8161710 0x7f3bb8161b90 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80671c0 tx=0x7f3bb000c820 comp rx=0 tx=0).stop 2023-10-16T11:26:15.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.326+0000 7f3bbc915640 1 -- 172.21.15.84:0/1041188546 shutdown_connections 2023-10-16T11:26:15.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.326+0000 7f3bbc915640 1 --2- 172.21.15.84:0/1041188546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bb8161710 0x7f3bb8161b90 unknown :-1 s=CLOSED pgs=3087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.326+0000 7f3bbc915640 1 --2- 172.21.15.84:0/1041188546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3bb80c5d50 0x7f3bb80c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.326+0000 7f3bbc915640 1 --2- 172.21.15.84:0/1041188546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ba40047c0 0x7f3bb81f9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.326+0000 7f3bbc915640 1 -- 172.21.15.84:0/1041188546 >> 172.21.15.84:0/1041188546 conn(0x7f3bb80b23b0 msgr2=0x7f3bb815f740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:15.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.326+0000 7f3bbc915640 1 -- 172.21.15.84:0/1041188546 shutdown_connections 2023-10-16T11:26:15.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.326+0000 7f3bbc915640 1 -- 172.21.15.84:0/1041188546 wait complete. 2023-10-16T11:26:15.347 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:26:15.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.566+0000 7fc18021f640 1 Processor -- start 2023-10-16T11:26:15.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.566+0000 7fc18021f640 1 -- start start 2023-10-16T11:26:15.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.568+0000 7fc18021f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc17815ad70 0x7fc178069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:15.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.568+0000 7fc18021f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc1780caac0 0x7fc17806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:15.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.568+0000 7fc18021f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1780cb030 con 0x7fc1780c9ca0 2023-10-16T11:26:15.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.568+0000 7fc18021f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1780c91c0 con 0x7fc1780caac0 2023-10-16T11:26:15.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.568+0000 7fc18021f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780c94a0 con 0x7fc17815ad70 2023-10-16T11:26:15.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.568+0000 7fc17d7ba640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc1780c9ca0 0x7fc1780ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40400/0 (socket says 172.21.15.84:40400) 2023-10-16T11:26:15.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.568+0000 7fc17d7ba640 1 -- 172.21.15.84:0/3076728009 learned_addr learned my addr 172.21.15.84:0/3076728009 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:15.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.568+0000 7fc17e7bc640 1 --2- 172.21.15.84:0/3076728009 >> v2:172.21.15.157:3300/0 conn(0x7fc1780caac0 0x7fc17806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:15.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.569+0000 7fc17dfbb640 1 --2- 172.21.15.84:0/3076728009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc17815ad70 0x7fc178069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:15.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.569+0000 7fc17cfb9640 1 -- 172.21.15.84:0/3076728009 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1997378406 0 0) 0x7fc1780cb030 con 0x7fc1780c9ca0 2023-10-16T11:26:15.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.569+0000 7fc17cfb9640 1 -- 172.21.15.84:0/3076728009 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc164003520 con 0x7fc1780c9ca0 2023-10-16T11:26:15.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.569+0000 7fc17cfb9640 1 -- 172.21.15.84:0/3076728009 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2252375895 0 0) 0x7fc164003520 con 0x7fc1780c9ca0 2023-10-16T11:26:15.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.569+0000 7fc17cfb9640 1 -- 172.21.15.84:0/3076728009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc17815ad70 msgr2=0x7fc178069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:15.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.569+0000 7fc17cfb9640 1 --2- 172.21.15.84:0/3076728009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc17815ad70 0x7fc178069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.569+0000 7fc17cfb9640 1 -- 172.21.15.84:0/3076728009 >> v2:172.21.15.157:3300/0 conn(0x7fc1780caac0 msgr2=0x7fc17806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:15.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.569+0000 7fc17cfb9640 1 --2- 172.21.15.84:0/3076728009 >> v2:172.21.15.157:3300/0 conn(0x7fc1780caac0 0x7fc17806a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.569+0000 7fc17cfb9640 1 -- 172.21.15.84:0/3076728009 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1780c9780 con 0x7fc1780c9ca0 2023-10-16T11:26:15.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.569+0000 7fc17cfb9640 1 -- 172.21.15.84:0/3076728009 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc168001670 con 0x7fc1780c9ca0 2023-10-16T11:26:15.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.569+0000 7fc17cfb9640 1 -- 172.21.15.84:0/3076728009 >> v1:172.21.15.84:6789/0 conn(0x7fc1780c9ca0 legacy=0x7fc1780ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:15.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.569+0000 7fc17cfb9640 1 --2- 172.21.15.84:0/3076728009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc17815ad70 0x7fc164003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:15.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.569+0000 7fc17cfb9640 1 --2- 172.21.15.84:0/3076728009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc164003f40 0x7fc1640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:15.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.569+0000 7fc17cfb9640 1 --2- 172.21.15.84:0/3076728009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc164004d00 0x7fc164005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:15.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.569+0000 7fc17cfb9640 1 -- 172.21.15.84:0/3076728009 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780c91c0 con 0x7fc164003f40 2023-10-16T11:26:15.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.569+0000 7fc17cfb9640 1 -- 172.21.15.84:0/3076728009 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780c94a0 con 0x7fc164004d00 2023-10-16T11:26:15.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.569+0000 7fc17cfb9640 1 -- 172.21.15.84:0/3076728009 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16400b980 con 0x7fc17815ad70 2023-10-16T11:26:15.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.570+0000 7fc17d7ba640 1 --2- 172.21.15.84:0/3076728009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc164003f40 0x7fc1640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:15.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.570+0000 7fc17dfbb640 1 --2- 172.21.15.84:0/3076728009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc17815ad70 0x7fc164003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:15.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.570+0000 7fc17e7bc640 1 --2- 172.21.15.84:0/3076728009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc164004d00 0x7fc164005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:15.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.570+0000 7fc17d7ba640 1 -- 172.21.15.84:0/3076728009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc17815ad70 msgr2=0x7fc164003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:15.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.570+0000 7fc17d7ba640 1 --2- 172.21.15.84:0/3076728009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc17815ad70 0x7fc164003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.570+0000 7fc17d7ba640 1 -- 172.21.15.84:0/3076728009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc164004d00 msgr2=0x7fc164005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:15.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.570+0000 7fc17d7ba640 1 --2- 172.21.15.84:0/3076728009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc164004d00 0x7fc164005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.570+0000 7fc17d7ba640 1 -- 172.21.15.84:0/3076728009 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc164003520 con 0x7fc164003f40 2023-10-16T11:26:15.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.570+0000 7fc17e7bc640 1 --2- 172.21.15.84:0/3076728009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc164004d00 0x7fc164005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:15.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.570+0000 7fc17dfbb640 1 --2- 172.21.15.84:0/3076728009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc17815ad70 0x7fc164003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:15.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.571+0000 7fc17d7ba640 1 --2- 172.21.15.84:0/3076728009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc164003f40 0x7fc1640047c0 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 crypto rx=0x7fc16800c190 tx=0x7fc1640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f837b816448d74 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:15.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.571+0000 7fc17cfb9640 1 -- 172.21.15.84:0/3076728009 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc168005ad0 con 0x7fc164003f40 2023-10-16T11:26:15.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.571+0000 7fc17cfb9640 1 -- 172.21.15.84:0/3076728009 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc168005c30 con 0x7fc164003f40 2023-10-16T11:26:15.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.571+0000 7fc17cfb9640 1 -- 172.21.15.84:0/3076728009 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16802c810 con 0x7fc164003f40 2023-10-16T11:26:15.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.572+0000 7fc18021f640 1 -- 172.21.15.84:0/3076728009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc164003f40 msgr2=0x7fc1640047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:15.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.572+0000 7fc18021f640 1 --2- 172.21.15.84:0/3076728009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc164003f40 0x7fc1640047c0 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 crypto rx=0x7fc16800c190 tx=0x7fc1640057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.572+0000 7fc17e7bc640 1 -- 172.21.15.84:0/3076728009 reap_dead start 2023-10-16T11:26:15.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.572+0000 7fc18021f640 1 -- 172.21.15.84:0/3076728009 shutdown_connections 2023-10-16T11:26:15.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.572+0000 7fc18021f640 1 -- 172.21.15.84:0/3076728009 >> 172.21.15.84:0/3076728009 conn(0x7fc1780ce9b0 msgr2=0x7fc1780cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:15.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.572+0000 7fc18021f640 1 -- 172.21.15.84:0/3076728009 shutdown_connections 2023-10-16T11:26:15.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.572+0000 7fc18021f640 1 -- 172.21.15.84:0/3076728009 wait complete. 2023-10-16T11:26:15.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.574+0000 7fc18021f640 1 Processor -- start 2023-10-16T11:26:15.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.574+0000 7fc18021f640 1 -- start start 2023-10-16T11:26:15.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.575+0000 7fc18021f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc164003f40 0x7fc1781f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:15.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.575+0000 7fc18021f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1780a9ab0 0x7fc178158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:15.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.575+0000 7fc18021f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1781f3540 0x7fc1781f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:15.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.575+0000 7fc18021f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1781f3f00 con 0x7fc164003f40 2023-10-16T11:26:15.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.575+0000 7fc18021f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780ad9d0 con 0x7fc1781f3540 2023-10-16T11:26:15.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.575+0000 7fc18021f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780aa8a0 con 0x7fc1780a9ab0 2023-10-16T11:26:15.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.575+0000 7fc17dfbb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc164003f40 0x7fc1781f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:15.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.575+0000 7fc17dfbb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc164003f40 0x7fc1781f4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35132/0 (socket says 172.21.15.84:35132) 2023-10-16T11:26:15.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.575+0000 7fc17dfbb640 1 -- 172.21.15.84:0/2749019864 learned_addr learned my addr 172.21.15.84:0/2749019864 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:15.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.576+0000 7fc17dfbb640 1 -- 172.21.15.84:0/2749019864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1780a9ab0 msgr2=0x7fc178158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:15.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.576+0000 7fc17dfbb640 1 --2- 172.21.15.84:0/2749019864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1780a9ab0 0x7fc178158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.576+0000 7fc17dfbb640 1 -- 172.21.15.84:0/2749019864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1781f3540 msgr2=0x7fc1781f39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:15.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.576+0000 7fc17dfbb640 1 --2- 172.21.15.84:0/2749019864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1781f3540 0x7fc1781f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.576+0000 7fc17dfbb640 1 -- 172.21.15.84:0/2749019864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1780adf70 con 0x7fc164003f40 2023-10-16T11:26:15.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.576+0000 7fc17dfbb640 1 --2- 172.21.15.84:0/2749019864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc164003f40 0x7fc1781f4100 secure :-1 s=READY pgs=3090 cs=0 l=1 rev1=1 crypto rx=0x7fc1740108e0 tx=0x7fc174010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:15.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.576+0000 7fc16e7fc640 1 -- 172.21.15.84:0/2749019864 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17400eba0 con 0x7fc164003f40 2023-10-16T11:26:15.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.576+0000 7fc16e7fc640 1 -- 172.21.15.84:0/2749019864 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17400ed00 con 0x7fc164003f40 2023-10-16T11:26:15.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.576+0000 7fc16e7fc640 1 -- 172.21.15.84:0/2749019864 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17401c600 con 0x7fc164003f40 2023-10-16T11:26:15.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.576+0000 7fc18021f640 1 -- 172.21.15.84:0/2749019864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1780aa2e0 con 0x7fc164003f40 2023-10-16T11:26:15.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.577+0000 7fc18021f640 1 -- 172.21.15.84:0/2749019864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1780adc00 con 0x7fc164003f40 2023-10-16T11:26:15.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.578+0000 7fc18021f640 1 -- 172.21.15.84:0/2749019864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17804e160 con 0x7fc164003f40 2023-10-16T11:26:15.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.579+0000 7fc16e7fc640 1 -- 172.21.15.84:0/2749019864 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 149) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fc17401c760 con 0x7fc164003f40 2023-10-16T11:26:15.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.580+0000 7fc16e7fc640 1 --2- 172.21.15.84:0/2749019864 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc15c06acc0 0x7fc15c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:15.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.580+0000 7fc16e7fc640 1 -- 172.21.15.84:0/2749019864 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7461+0+0 (secure 0 0 0) 0x7fc174089fd0 con 0x7fc164003f40 2023-10-16T11:26:15.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.580+0000 7fc17d7ba640 1 --2- 172.21.15.84:0/2749019864 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc15c06acc0 0x7fc15c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:15.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.581+0000 7fc17d7ba640 1 --2- 172.21.15.84:0/2749019864 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc15c06acc0 0x7fc15c06d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc16800c160 tx=0x7fc16802e000 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:15.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.584+0000 7fc16e7fc640 1 -- 172.21.15.84:0/2749019864 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc174054cc0 con 0x7fc164003f40 2023-10-16T11:26:15.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.777+0000 7fc18021f640 1 -- 172.21.15.84:0/2749019864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc1780cee60 con 0x7fc164003f40 2023-10-16T11:26:15.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.778+0000 7fc16e7fc640 1 -- 172.21.15.84:0/2749019864 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v243) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc1740582e0 con 0x7fc164003f40 2023-10-16T11:26:15.780 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:26:15.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.780+0000 7fc18021f640 1 -- 172.21.15.84:0/2749019864 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc15c06acc0 msgr2=0x7fc15c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:15.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.780+0000 7fc18021f640 1 --2- 172.21.15.84:0/2749019864 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc15c06acc0 0x7fc15c06d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc16800c160 tx=0x7fc16802e000 comp rx=0 tx=0).stop 2023-10-16T11:26:15.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.780+0000 7fc18021f640 1 -- 172.21.15.84:0/2749019864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc164003f40 msgr2=0x7fc1781f4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:15.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.780+0000 7fc18021f640 1 --2- 172.21.15.84:0/2749019864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc164003f40 0x7fc1781f4100 secure :-1 s=READY pgs=3090 cs=0 l=1 rev1=1 crypto rx=0x7fc1740108e0 tx=0x7fc174010da0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.781+0000 7fc18021f640 1 -- 172.21.15.84:0/2749019864 shutdown_connections 2023-10-16T11:26:15.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.781+0000 7fc18021f640 1 --2- 172.21.15.84:0/2749019864 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc15c06acc0 0x7fc15c06d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.781+0000 7fc18021f640 1 --2- 172.21.15.84:0/2749019864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1781f3540 0x7fc1781f39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.781+0000 7fc18021f640 1 --2- 172.21.15.84:0/2749019864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1780a9ab0 0x7fc178158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.781+0000 7fc18021f640 1 --2- 172.21.15.84:0/2749019864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc164003f40 0x7fc1781f4100 unknown :-1 s=CLOSED pgs=3090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:15.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.781+0000 7fc18021f640 1 -- 172.21.15.84:0/2749019864 >> 172.21.15.84:0/2749019864 conn(0x7fc1780ce9b0 msgr2=0x7fc1780b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:15.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.781+0000 7fc18021f640 1 -- 172.21.15.84:0/2749019864 shutdown_connections 2023-10-16T11:26:15.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:15.781+0000 7fc18021f640 1 -- 172.21.15.84:0/2749019864 wait complete. 2023-10-16T11:26:15.802 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:26:16.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.019+0000 7f4140333640 1 Processor -- start 2023-10-16T11:26:16.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.019+0000 7f4140333640 1 -- start start 2023-10-16T11:26:16.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.020+0000 7f4140333640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f413815ae10 0x7f41380ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.020+0000 7f4140333640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f413806d570 0x7f41380caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.020+0000 7f4140333640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41380cf330 0x7f413806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.020+0000 7f4140333640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413806f650 con 0x7f413815ae10 2023-10-16T11:26:16.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.020+0000 7f4140333640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413815b1f0 con 0x7f413806d570 2023-10-16T11:26:16.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.020+0000 7f4140333640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413806bb70 con 0x7f41380cf330 2023-10-16T11:26:16.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.021+0000 7f413e0cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f413815ae10 0x7f41380ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.021+0000 7f413e0cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f413815ae10 0x7f41380ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35144/0 (socket says 172.21.15.84:35144) 2023-10-16T11:26:16.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.021+0000 7f413e0cf640 1 -- 172.21.15.84:0/1027552676 learned_addr learned my addr 172.21.15.84:0/1027552676 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:16.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.021+0000 7f413e0cf640 1 -- 172.21.15.84:0/1027552676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41380cf330 msgr2=0x7f413806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:16.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.021+0000 7f413e0cf640 1 --2- 172.21.15.84:0/1027552676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41380cf330 0x7f413806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.021+0000 7f413e0cf640 1 -- 172.21.15.84:0/1027552676 >> v2:172.21.15.157:3300/0 conn(0x7f413806d570 msgr2=0x7f41380caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:16.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.021+0000 7f413d8ce640 1 --2- 172.21.15.84:0/1027552676 >> v2:172.21.15.157:3300/0 conn(0x7f413806d570 0x7f41380caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.021+0000 7f413e0cf640 1 --2- 172.21.15.84:0/1027552676 >> v2:172.21.15.157:3300/0 conn(0x7f413806d570 0x7f41380caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.021+0000 7f413e0cf640 1 -- 172.21.15.84:0/1027552676 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41380b4190 con 0x7f413815ae10 2023-10-16T11:26:16.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.022+0000 7f413e0cf640 1 --2- 172.21.15.84:0/1027552676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f413815ae10 0x7f41380ca590 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 crypto rx=0x7f41380cf940 tx=0x7f413402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0bb6076f8c37e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:16.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.023+0000 7f413d0cd640 1 -- 172.21.15.84:0/1027552676 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4134031870 con 0x7f413815ae10 2023-10-16T11:26:16.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.023+0000 7f413d0cd640 1 -- 172.21.15.84:0/1027552676 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41340319d0 con 0x7f413815ae10 2023-10-16T11:26:16.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.023+0000 7f413d0cd640 1 -- 172.21.15.84:0/1027552676 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4134031d60 con 0x7f413815ae10 2023-10-16T11:26:16.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.023+0000 7f4140333640 1 -- 172.21.15.84:0/1027552676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f413815ae10 msgr2=0x7f41380ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:16.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.023+0000 7f4140333640 1 --2- 172.21.15.84:0/1027552676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f413815ae10 0x7f41380ca590 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 crypto rx=0x7f41380cf940 tx=0x7f413402b910 comp rx=0 tx=0).stop 2023-10-16T11:26:16.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.023+0000 7f4140333640 1 -- 172.21.15.84:0/1027552676 shutdown_connections 2023-10-16T11:26:16.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.023+0000 7f4140333640 1 --2- 172.21.15.84:0/1027552676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41380cf330 0x7f413806c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.023+0000 7f4140333640 1 --2- 172.21.15.84:0/1027552676 >> v2:172.21.15.157:3300/0 conn(0x7f413806d570 0x7f41380caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.023+0000 7f4140333640 1 --2- 172.21.15.84:0/1027552676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f413815ae10 0x7f41380ca590 unknown :-1 s=CLOSED pgs=3091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.023+0000 7f4140333640 1 -- 172.21.15.84:0/1027552676 >> 172.21.15.84:0/1027552676 conn(0x7f41380ce840 msgr2=0x7f41380cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:16.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.024+0000 7f4140333640 1 -- 172.21.15.84:0/1027552676 shutdown_connections 2023-10-16T11:26:16.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.024+0000 7f4140333640 1 -- 172.21.15.84:0/1027552676 wait complete. 2023-10-16T11:26:16.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.025+0000 7f4140333640 1 Processor -- start 2023-10-16T11:26:16.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.026+0000 7f4140333640 1 -- start start 2023-10-16T11:26:16.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.026+0000 7f4140333640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f413806d570 0x7f41380ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.026+0000 7f4140333640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41380cf330 0x7f41380ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.026+0000 7f4140333640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413815ae10 0x7f41380c2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.026+0000 7f4140333640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41381f37d0 con 0x7f413806d570 2023-10-16T11:26:16.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.026+0000 7f4140333640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41381f2a10 con 0x7f413815ae10 2023-10-16T11:26:16.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.026+0000 7f4140333640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41381f2cf0 con 0x7f41380cf330 2023-10-16T11:26:16.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.027+0000 7f413e0cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f413806d570 0x7f41380ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.027+0000 7f413e0cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f413806d570 0x7f41380ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35158/0 (socket says 172.21.15.84:35158) 2023-10-16T11:26:16.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.027+0000 7f413e0cf640 1 -- 172.21.15.84:0/2148171951 learned_addr learned my addr 172.21.15.84:0/2148171951 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:16.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.027+0000 7f413e0cf640 1 -- 172.21.15.84:0/2148171951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41380cf330 msgr2=0x7f41380ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:16.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.027+0000 7f413e0cf640 1 --2- 172.21.15.84:0/2148171951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41380cf330 0x7f41380ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.027+0000 7f413e0cf640 1 -- 172.21.15.84:0/2148171951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413815ae10 msgr2=0x7f41380c2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:16.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.027+0000 7f413e8d0640 1 --2- 172.21.15.84:0/2148171951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413815ae10 0x7f41380c2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.027+0000 7f413e0cf640 1 --2- 172.21.15.84:0/2148171951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413815ae10 0x7f41380c2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.027+0000 7f413e0cf640 1 -- 172.21.15.84:0/2148171951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41340094d0 con 0x7f413806d570 2023-10-16T11:26:16.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.027+0000 7f413e0cf640 1 --2- 172.21.15.84:0/2148171951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f413806d570 0x7f41380ad8e0 secure :-1 s=READY pgs=3092 cs=0 l=1 rev1=1 crypto rx=0x7f4134000c00 tx=0x7f41340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:16.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.028+0000 7f412effd640 1 -- 172.21.15.84:0/2148171951 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4134031840 con 0x7f413806d570 2023-10-16T11:26:16.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.028+0000 7f412effd640 1 -- 172.21.15.84:0/2148171951 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41340443f0 con 0x7f413806d570 2023-10-16T11:26:16.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.028+0000 7f412effd640 1 -- 172.21.15.84:0/2148171951 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41340447c0 con 0x7f413806d570 2023-10-16T11:26:16.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.028+0000 7f4140333640 1 -- 172.21.15.84:0/2148171951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41381f2fd0 con 0x7f413806d570 2023-10-16T11:26:16.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.028+0000 7f4140333640 1 -- 172.21.15.84:0/2148171951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41380c3cc0 con 0x7f413806d570 2023-10-16T11:26:16.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.029+0000 7f4140333640 1 -- 172.21.15.84:0/2148171951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f413804e160 con 0x7f413806d570 2023-10-16T11:26:16.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.030+0000 7f412effd640 1 -- 172.21.15.84:0/2148171951 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 149) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f4134044920 con 0x7f413806d570 2023-10-16T11:26:16.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.031+0000 7f412effd640 1 --2- 172.21.15.84:0/2148171951 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f411c06ad10 0x7f411c06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.031+0000 7f412effd640 1 -- 172.21.15.84:0/2148171951 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7461+0+0 (secure 0 0 0) 0x7f413402d040 con 0x7f413806d570 2023-10-16T11:26:16.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.031+0000 7f413d8ce640 1 --2- 172.21.15.84:0/2148171951 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f411c06ad10 0x7f411c06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.032+0000 7f413d8ce640 1 --2- 172.21.15.84:0/2148171951 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f411c06ad10 0x7f411c06d1d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f41380cf910 tx=0x7f4128008040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:16.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.036+0000 7f412effd640 1 -- 172.21.15.84:0/2148171951 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4134076fa0 con 0x7f413806d570 2023-10-16T11:26:16.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.233+0000 7f4140333640 1 -- 172.21.15.84:0/2148171951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f413815e1a0 con 0x7f413806d570 2023-10-16T11:26:16.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.234+0000 7f412effd640 1 -- 172.21.15.84:0/2148171951 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v243) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4134035020 con 0x7f413806d570 2023-10-16T11:26:16.236 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:26:16.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.236+0000 7f4140333640 1 -- 172.21.15.84:0/2148171951 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f411c06ad10 msgr2=0x7f411c06d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:16.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.236+0000 7f4140333640 1 --2- 172.21.15.84:0/2148171951 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f411c06ad10 0x7f411c06d1d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f41380cf910 tx=0x7f4128008040 comp rx=0 tx=0).stop 2023-10-16T11:26:16.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.236+0000 7f4140333640 1 -- 172.21.15.84:0/2148171951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f413806d570 msgr2=0x7f41380ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:16.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.236+0000 7f4140333640 1 --2- 172.21.15.84:0/2148171951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f413806d570 0x7f41380ad8e0 secure :-1 s=READY pgs=3092 cs=0 l=1 rev1=1 crypto rx=0x7f4134000c00 tx=0x7f41340046c0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.236+0000 7f4140333640 1 -- 172.21.15.84:0/2148171951 shutdown_connections 2023-10-16T11:26:16.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.236+0000 7f4140333640 1 --2- 172.21.15.84:0/2148171951 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f411c06ad10 0x7f411c06d1d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.236+0000 7f4140333640 1 --2- 172.21.15.84:0/2148171951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413815ae10 0x7f41380c2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.236+0000 7f4140333640 1 --2- 172.21.15.84:0/2148171951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41380cf330 0x7f41380ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.236+0000 7f4140333640 1 --2- 172.21.15.84:0/2148171951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f413806d570 0x7f41380ad8e0 unknown :-1 s=CLOSED pgs=3092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.236+0000 7f4140333640 1 -- 172.21.15.84:0/2148171951 >> 172.21.15.84:0/2148171951 conn(0x7f41380ce840 msgr2=0x7f41380afb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:16.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.236+0000 7f4140333640 1 -- 172.21.15.84:0/2148171951 shutdown_connections 2023-10-16T11:26:16.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.237+0000 7f4140333640 1 -- 172.21.15.84:0/2148171951 wait complete. 2023-10-16T11:26:16.257 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-16T11:26:16.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.474+0000 7f1e17ad0640 1 Processor -- start 2023-10-16T11:26:16.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.474+0000 7f1e17ad0640 1 -- start start 2023-10-16T11:26:16.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.476+0000 7f1e17ad0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1006a4b0 0x7f1e100c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.476+0000 7f1e17ad0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e100ac890 con 0x7f1e100ad350 2023-10-16T11:26:16.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.476+0000 7f1e17ad0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e100ad7a0 con 0x7f1e1015d150 2023-10-16T11:26:16.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.476+0000 7f1e17ad0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1015d530 con 0x7f1e1006a4b0 2023-10-16T11:26:16.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.476+0000 7f1e1506b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1e100ad350 0x7f1e10069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46962/0 (socket says 172.21.15.84:46962) 2023-10-16T11:26:16.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.476+0000 7f1e1506b640 1 -- 172.21.15.84:0/3852986323 learned_addr learned my addr 172.21.15.84:0/3852986323 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:16.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.477+0000 7f1e1606d640 1 --2- 172.21.15.84:0/3852986323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1006a4b0 0x7f1e100c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.477+0000 7f1e1486a640 1 -- 172.21.15.84:0/3852986323 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744081325 0 0) 0x7f1e100ac890 con 0x7f1e100ad350 2023-10-16T11:26:16.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.477+0000 7f1e1486a640 1 -- 172.21.15.84:0/3852986323 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dfc003520 con 0x7f1e100ad350 2023-10-16T11:26:16.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.477+0000 7f1e1486a640 1 -- 172.21.15.84:0/3852986323 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1085081954 0 0) 0x7f1dfc003520 con 0x7f1e100ad350 2023-10-16T11:26:16.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.477+0000 7f1e1486a640 1 -- 172.21.15.84:0/3852986323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1006a4b0 msgr2=0x7f1e100c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:16.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.477+0000 7f1e1486a640 1 --2- 172.21.15.84:0/3852986323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1006a4b0 0x7f1e100c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.477+0000 7f1e1486a640 1 -- 172.21.15.84:0/3852986323 >> v1:172.21.15.157:6789/0 conn(0x7f1e1015d150 legacy=0x7f1e10160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:16.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.477+0000 7f1e1486a640 1 -- 172.21.15.84:0/3852986323 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e10161650 con 0x7f1e100ad350 2023-10-16T11:26:16.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.477+0000 7f1e1486a640 1 -- 172.21.15.84:0/3852986323 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f1e00001670 con 0x7f1e100ad350 2023-10-16T11:26:16.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.477+0000 7f1e1486a640 1 -- 172.21.15.84:0/3852986323 >> v1:172.21.15.84:6789/0 conn(0x7f1e100ad350 legacy=0x7f1e10069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:16.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.477+0000 7f1e1486a640 1 --2- 172.21.15.84:0/3852986323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dfc0039a0 0x7f1dfc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.477+0000 7f1e1486a640 1 --2- 172.21.15.84:0/3852986323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dfc0047c0 0x7f1dfc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.477+0000 7f1e1486a640 1 --2- 172.21.15.84:0/3852986323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1006a4b0 0x7f1dfc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.477+0000 7f1e1486a640 1 -- 172.21.15.84:0/3852986323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1015d530 con 0x7f1dfc0039a0 2023-10-16T11:26:16.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.477+0000 7f1e1486a640 1 -- 172.21.15.84:0/3852986323 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc00ba60 con 0x7f1dfc0047c0 2023-10-16T11:26:16.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.477+0000 7f1e1486a640 1 -- 172.21.15.84:0/3852986323 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc00bc20 con 0x7f1e1006a4b0 2023-10-16T11:26:16.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.478+0000 7f1e1586c640 1 --2- 172.21.15.84:0/3852986323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dfc0039a0 0x7f1dfc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.478+0000 7f1e1506b640 1 --2- 172.21.15.84:0/3852986323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dfc0047c0 0x7f1dfc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.478+0000 7f1e1586c640 1 -- 172.21.15.84:0/3852986323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1006a4b0 msgr2=0x7f1dfc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:16.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.478+0000 7f1e1586c640 1 --2- 172.21.15.84:0/3852986323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1006a4b0 0x7f1dfc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.478+0000 7f1e1586c640 1 -- 172.21.15.84:0/3852986323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dfc0047c0 msgr2=0x7f1dfc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:16.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.478+0000 7f1e1586c640 1 --2- 172.21.15.84:0/3852986323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dfc0047c0 0x7f1dfc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.478+0000 7f1e1586c640 1 -- 172.21.15.84:0/3852986323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc003520 con 0x7f1dfc0039a0 2023-10-16T11:26:16.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.478+0000 7f1e1506b640 1 --2- 172.21.15.84:0/3852986323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dfc0047c0 0x7f1dfc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:16.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.479+0000 7f1e1586c640 1 --2- 172.21.15.84:0/3852986323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dfc0039a0 0x7f1dfc003dc0 secure :-1 s=READY pgs=3094 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c001720 tx=0x7f1dfc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00421a61e6d016f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:16.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.479+0000 7f1e1486a640 1 -- 172.21.15.84:0/3852986323 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c0052d0 con 0x7f1dfc0039a0 2023-10-16T11:26:16.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.479+0000 7f1e1486a640 1 -- 172.21.15.84:0/3852986323 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0c02e050 con 0x7f1dfc0039a0 2023-10-16T11:26:16.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.479+0000 7f1e1486a640 1 -- 172.21.15.84:0/3852986323 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c0035f0 con 0x7f1dfc0039a0 2023-10-16T11:26:16.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.480+0000 7f1e17ad0640 1 -- 172.21.15.84:0/3852986323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dfc0039a0 msgr2=0x7f1dfc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:16.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.480+0000 7f1e17ad0640 1 --2- 172.21.15.84:0/3852986323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dfc0039a0 0x7f1dfc003dc0 secure :-1 s=READY pgs=3094 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c001720 tx=0x7f1dfc0057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.480+0000 7f1e1606d640 1 -- 172.21.15.84:0/3852986323 reap_dead start 2023-10-16T11:26:16.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.480+0000 7f1e17ad0640 1 -- 172.21.15.84:0/3852986323 shutdown_connections 2023-10-16T11:26:16.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.480+0000 7f1e17ad0640 1 -- 172.21.15.84:0/3852986323 >> 172.21.15.84:0/3852986323 conn(0x7f1e100ce9b0 msgr2=0x7f1e100cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:16.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.480+0000 7f1e17ad0640 1 -- 172.21.15.84:0/3852986323 shutdown_connections 2023-10-16T11:26:16.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.480+0000 7f1e17ad0640 1 -- 172.21.15.84:0/3852986323 wait complete. 2023-10-16T11:26:16.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.482+0000 7f1e17ad0640 1 Processor -- start 2023-10-16T11:26:16.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.482+0000 7f1e17ad0640 1 -- start start 2023-10-16T11:26:16.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.483+0000 7f1e17ad0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dfc0039a0 0x7f1e1015d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.483+0000 7f1e17ad0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1015fc90 0x7f1e100ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.483+0000 7f1e17ad0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e100ca6d0 0x7f1e10154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.483+0000 7f1e17ad0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e101604f0 con 0x7f1e100ca6d0 2023-10-16T11:26:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.483+0000 7f1e17ad0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e10160210 con 0x7f1dfc0039a0 2023-10-16T11:26:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.483+0000 7f1e17ad0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e100c2730 con 0x7f1e1015fc90 2023-10-16T11:26:16.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.483+0000 7f1e1606d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e100ca6d0 0x7f1e10154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.483+0000 7f1e1606d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e100ca6d0 0x7f1e10154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35170/0 (socket says 172.21.15.84:35170) 2023-10-16T11:26:16.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.483+0000 7f1e1606d640 1 -- 172.21.15.84:0/2062685901 learned_addr learned my addr 172.21.15.84:0/2062685901 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:16.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.484+0000 7f1e1606d640 1 -- 172.21.15.84:0/2062685901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1015fc90 msgr2=0x7f1e100ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:16.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.484+0000 7f1e1606d640 1 --2- 172.21.15.84:0/2062685901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1015fc90 0x7f1e100ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.484+0000 7f1e1606d640 1 -- 172.21.15.84:0/2062685901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dfc0039a0 msgr2=0x7f1e1015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:16.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.484+0000 7f1e1606d640 1 --2- 172.21.15.84:0/2062685901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dfc0039a0 0x7f1e1015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.484+0000 7f1e1606d640 1 -- 172.21.15.84:0/2062685901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e100c2a10 con 0x7f1e100ca6d0 2023-10-16T11:26:16.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.484+0000 7f1e1606d640 1 --2- 172.21.15.84:0/2062685901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e100ca6d0 0x7f1e10154660 secure :-1 s=READY pgs=3095 cs=0 l=1 rev1=1 crypto rx=0x7f1e100cf870 tx=0x7f1e0800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:16.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.484+0000 7f1e067fc640 1 -- 172.21.15.84:0/2062685901 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08010040 con 0x7f1e100ca6d0 2023-10-16T11:26:16.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.484+0000 7f1e067fc640 1 -- 172.21.15.84:0/2062685901 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0800bae0 con 0x7f1e100ca6d0 2023-10-16T11:26:16.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.484+0000 7f1e067fc640 1 -- 172.21.15.84:0/2062685901 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0800be70 con 0x7f1e100ca6d0 2023-10-16T11:26:16.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.484+0000 7f1e17ad0640 1 -- 172.21.15.84:0/2062685901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e100c2cf0 con 0x7f1e100ca6d0 2023-10-16T11:26:16.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.485+0000 7f1e17ad0640 1 -- 172.21.15.84:0/2062685901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1015dd90 con 0x7f1e100ca6d0 2023-10-16T11:26:16.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.486+0000 7f1e17ad0640 1 -- 172.21.15.84:0/2062685901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e100c2ee0 con 0x7f1e100ca6d0 2023-10-16T11:26:16.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.487+0000 7f1e067fc640 1 -- 172.21.15.84:0/2062685901 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 149) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f1e0801e410 con 0x7f1e100ca6d0 2023-10-16T11:26:16.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.487+0000 7f1e067fc640 1 --2- 172.21.15.84:0/2062685901 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f1df406acc0 0x7f1df406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.488+0000 7f1e067fc640 1 -- 172.21.15.84:0/2062685901 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7461+0+0 (secure 0 0 0) 0x7f1e08089af0 con 0x7f1e100ca6d0 2023-10-16T11:26:16.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.488+0000 7f1e1586c640 1 --2- 172.21.15.84:0/2062685901 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f1df406acc0 0x7f1df406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.488+0000 7f1e1586c640 1 --2- 172.21.15.84:0/2062685901 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f1df406acc0 0x7f1df406d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c033000 tx=0x7f1e0c02f000 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:16.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.492+0000 7f1e067fc640 1 -- 172.21.15.84:0/2062685901 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1e080546a0 con 0x7f1e100ca6d0 2023-10-16T11:26:16.542 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:16.539+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:16.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.688+0000 7f1e17ad0640 1 -- 172.21.15.84:0/2062685901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1e1015dd90 con 0x7f1e100ca6d0 2023-10-16T11:26:16.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.689+0000 7f1e067fc640 1 -- 172.21.15.84:0/2062685901 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v243) v1 ==== 107+0+11 (secure 0 0 0) 0x7f1e1015dd90 con 0x7f1e100ca6d0 2023-10-16T11:26:16.691 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:26:16.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.691+0000 7f1e17ad0640 1 -- 172.21.15.84:0/2062685901 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f1df406acc0 msgr2=0x7f1df406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:16.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.691+0000 7f1e17ad0640 1 --2- 172.21.15.84:0/2062685901 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f1df406acc0 0x7f1df406d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c033000 tx=0x7f1e0c02f000 comp rx=0 tx=0).stop 2023-10-16T11:26:16.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.691+0000 7f1e17ad0640 1 -- 172.21.15.84:0/2062685901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e100ca6d0 msgr2=0x7f1e10154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:16.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.691+0000 7f1e17ad0640 1 --2- 172.21.15.84:0/2062685901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e100ca6d0 0x7f1e10154660 secure :-1 s=READY pgs=3095 cs=0 l=1 rev1=1 crypto rx=0x7f1e100cf870 tx=0x7f1e0800b480 comp rx=0 tx=0).stop 2023-10-16T11:26:16.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.691+0000 7f1e17ad0640 1 -- 172.21.15.84:0/2062685901 shutdown_connections 2023-10-16T11:26:16.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.691+0000 7f1e17ad0640 1 --2- 172.21.15.84:0/2062685901 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f1df406acc0 0x7f1df406d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.691+0000 7f1e17ad0640 1 --2- 172.21.15.84:0/2062685901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e100ca6d0 0x7f1e10154660 unknown :-1 s=CLOSED pgs=3095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.692+0000 7f1e17ad0640 1 --2- 172.21.15.84:0/2062685901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e1015fc90 0x7f1e100ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.692+0000 7f1e17ad0640 1 --2- 172.21.15.84:0/2062685901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dfc0039a0 0x7f1e1015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.692+0000 7f1e17ad0640 1 -- 172.21.15.84:0/2062685901 >> 172.21.15.84:0/2062685901 conn(0x7f1e100ce9b0 msgr2=0x7f1e100704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:16.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.692+0000 7f1e17ad0640 1 -- 172.21.15.84:0/2062685901 shutdown_connections 2023-10-16T11:26:16.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.692+0000 7f1e17ad0640 1 -- 172.21.15.84:0/2062685901 wait complete. 2023-10-16T11:26:16.712 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-16T11:26:16.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.927+0000 7f8a96640640 1 Processor -- start 2023-10-16T11:26:16.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.927+0000 7f8a96640640 1 -- start start 2023-10-16T11:26:16.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.928+0000 7f8a96640640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a9006c040 0x7f8a9006c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.928+0000 7f8a96640640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8a900ad970 0x7f8a9015e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.928+0000 7f8a96640640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a9006c960 con 0x7f8a9015ebc0 2023-10-16T11:26:16.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.929+0000 7f8a96640640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9015b9d0 con 0x7f8a900ad970 2023-10-16T11:26:16.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.929+0000 7f8a96640640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9015bcb0 con 0x7f8a9006c040 2023-10-16T11:26:16.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.929+0000 7f8a95e3f640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8a9015ebc0 0x7f8a900cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46972/0 (socket says 172.21.15.84:46972) 2023-10-16T11:26:16.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.929+0000 7f8a95e3f640 1 -- 172.21.15.84:0/2353583089 learned_addr learned my addr 172.21.15.84:0/2353583089 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:16.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.929+0000 7f8a94e3d640 1 --2- 172.21.15.84:0/2353583089 >> v2:172.21.15.157:3300/0 conn(0x7f8a900ad970 0x7f8a9015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.929+0000 7f8a9563e640 1 --2- 172.21.15.84:0/2353583089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a9006c040 0x7f8a9006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.929+0000 7f8a87fff640 1 -- 172.21.15.84:0/2353583089 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 869274605 0 0) 0x7f8a9006c960 con 0x7f8a9015ebc0 2023-10-16T11:26:16.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.929+0000 7f8a87fff640 1 -- 172.21.15.84:0/2353583089 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a7c003520 con 0x7f8a9015ebc0 2023-10-16T11:26:16.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.930+0000 7f8a87fff640 1 -- 172.21.15.84:0/2353583089 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3765597450 0 0) 0x7f8a7c003520 con 0x7f8a9015ebc0 2023-10-16T11:26:16.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.930+0000 7f8a87fff640 1 -- 172.21.15.84:0/2353583089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a9006c040 msgr2=0x7f8a9006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:16.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.930+0000 7f8a87fff640 1 --2- 172.21.15.84:0/2353583089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a9006c040 0x7f8a9006c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.930+0000 7f8a87fff640 1 -- 172.21.15.84:0/2353583089 >> v2:172.21.15.157:3300/0 conn(0x7f8a900ad970 msgr2=0x7f8a9015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:16.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.930+0000 7f8a87fff640 1 --2- 172.21.15.84:0/2353583089 >> v2:172.21.15.157:3300/0 conn(0x7f8a900ad970 0x7f8a9015e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.930+0000 7f8a87fff640 1 -- 172.21.15.84:0/2353583089 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9015bf90 con 0x7f8a9015ebc0 2023-10-16T11:26:16.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.930+0000 7f8a87fff640 1 -- 172.21.15.84:0/2353583089 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f8a880019f0 con 0x7f8a9015ebc0 2023-10-16T11:26:16.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.930+0000 7f8a87fff640 1 -- 172.21.15.84:0/2353583089 >> v1:172.21.15.84:6789/0 conn(0x7f8a9015ebc0 legacy=0x7f8a900cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:16.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.930+0000 7f8a87fff640 1 --2- 172.21.15.84:0/2353583089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7c003970 0x7f8a7c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.930+0000 7f8a87fff640 1 --2- 172.21.15.84:0/2353583089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c0047c0 0x7f8a7c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.930+0000 7f8a87fff640 1 --2- 172.21.15.84:0/2353583089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a9006c040 0x7f8a7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.930+0000 7f8a87fff640 1 -- 172.21.15.84:0/2353583089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9015b9d0 con 0x7f8a7c003970 2023-10-16T11:26:16.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.930+0000 7f8a87fff640 1 -- 172.21.15.84:0/2353583089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9015bcb0 con 0x7f8a7c0047c0 2023-10-16T11:26:16.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.930+0000 7f8a87fff640 1 -- 172.21.15.84:0/2353583089 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c00b980 con 0x7f8a9006c040 2023-10-16T11:26:16.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.930+0000 7f8a9563e640 1 --2- 172.21.15.84:0/2353583089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7c003970 0x7f8a7c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.931+0000 7f8a94e3d640 1 --2- 172.21.15.84:0/2353583089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c0047c0 0x7f8a7c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.931+0000 7f8a95e3f640 1 --2- 172.21.15.84:0/2353583089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a9006c040 0x7f8a7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.931+0000 7f8a9563e640 1 -- 172.21.15.84:0/2353583089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a9006c040 msgr2=0x7f8a7c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:16.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.931+0000 7f8a9563e640 1 --2- 172.21.15.84:0/2353583089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a9006c040 0x7f8a7c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.931+0000 7f8a9563e640 1 -- 172.21.15.84:0/2353583089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c0047c0 msgr2=0x7f8a7c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:16.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.931+0000 7f8a9563e640 1 --2- 172.21.15.84:0/2353583089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c0047c0 0x7f8a7c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.931+0000 7f8a9563e640 1 -- 172.21.15.84:0/2353583089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c003520 con 0x7f8a7c003970 2023-10-16T11:26:16.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.931+0000 7f8a94e3d640 1 --2- 172.21.15.84:0/2353583089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c0047c0 0x7f8a7c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:16.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.932+0000 7f8a9563e640 1 --2- 172.21.15.84:0/2353583089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7c003970 0x7f8a7c003d90 secure :-1 s=READY pgs=3097 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00ba30 tx=0x7f8a7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759f847020e35945 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:16.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.932+0000 7f8a87fff640 1 -- 172.21.15.84:0/2353583089 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c032640 con 0x7f8a7c003970 2023-10-16T11:26:16.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.932+0000 7f8a87fff640 1 -- 172.21.15.84:0/2353583089 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c0327a0 con 0x7f8a7c003970 2023-10-16T11:26:16.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.932+0000 7f8a87fff640 1 -- 172.21.15.84:0/2353583089 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c032b30 con 0x7f8a7c003970 2023-10-16T11:26:16.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.933+0000 7f8a96640640 1 -- 172.21.15.84:0/2353583089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7c003970 msgr2=0x7f8a7c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:16.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.933+0000 7f8a96640640 1 --2- 172.21.15.84:0/2353583089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a7c003970 0x7f8a7c003d90 secure :-1 s=READY pgs=3097 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00ba30 tx=0x7f8a7c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.933+0000 7f8a95e3f640 1 -- 172.21.15.84:0/2353583089 reap_dead start 2023-10-16T11:26:16.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.933+0000 7f8a96640640 1 -- 172.21.15.84:0/2353583089 shutdown_connections 2023-10-16T11:26:16.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.933+0000 7f8a96640640 1 -- 172.21.15.84:0/2353583089 >> 172.21.15.84:0/2353583089 conn(0x7f8a900ce6a0 msgr2=0x7f8a900ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:16.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.933+0000 7f8a96640640 1 -- 172.21.15.84:0/2353583089 shutdown_connections 2023-10-16T11:26:16.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.933+0000 7f8a96640640 1 -- 172.21.15.84:0/2353583089 wait complete. 2023-10-16T11:26:16.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.935+0000 7f8a96640640 1 Processor -- start 2023-10-16T11:26:16.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.935+0000 7f8a96640640 1 -- start start 2023-10-16T11:26:16.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.936+0000 7f8a96640640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c003970 0x7f8a901584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.936+0000 7f8a96640640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a901f2770 0x7f8a900b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.936+0000 7f8a96640640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a901f2bf0 0x7f8a900ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.936+0000 7f8a96640640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a900c69d0 con 0x7f8a901f2770 2023-10-16T11:26:16.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.936+0000 7f8a96640640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a900c6cb0 con 0x7f8a7c003970 2023-10-16T11:26:16.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.936+0000 7f8a96640640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a900c6f90 con 0x7f8a901f2bf0 2023-10-16T11:26:16.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.936+0000 7f8a94e3d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a901f2770 0x7f8a900b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.936+0000 7f8a94e3d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a901f2770 0x7f8a900b0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35190/0 (socket says 172.21.15.84:35190) 2023-10-16T11:26:16.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.936+0000 7f8a94e3d640 1 -- 172.21.15.84:0/3791201824 learned_addr learned my addr 172.21.15.84:0/3791201824 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:16.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.936+0000 7f8a9563e640 1 --2- 172.21.15.84:0/3791201824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c003970 0x7f8a901584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.936+0000 7f8a94e3d640 1 -- 172.21.15.84:0/3791201824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a901f2bf0 msgr2=0x7f8a900ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:16.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.936+0000 7f8a95e3f640 1 --2- 172.21.15.84:0/3791201824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a901f2bf0 0x7f8a900ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.936+0000 7f8a94e3d640 1 --2- 172.21.15.84:0/3791201824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a901f2bf0 0x7f8a900ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.936+0000 7f8a94e3d640 1 -- 172.21.15.84:0/3791201824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c003970 msgr2=0x7f8a901584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:16.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.936+0000 7f8a94e3d640 1 --2- 172.21.15.84:0/3791201824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c003970 0x7f8a901584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:16.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.936+0000 7f8a94e3d640 1 -- 172.21.15.84:0/3791201824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a900c7270 con 0x7f8a901f2770 2023-10-16T11:26:16.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.937+0000 7f8a94e3d640 1 --2- 172.21.15.84:0/3791201824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a901f2770 0x7f8a900b0790 secure :-1 s=READY pgs=3098 cs=0 l=1 rev1=1 crypto rx=0x7f8a800109d0 tx=0x7f8a80010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:16.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.937+0000 7f8a85ffb640 1 -- 172.21.15.84:0/3791201824 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8000ec20 con 0x7f8a901f2770 2023-10-16T11:26:16.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.937+0000 7f8a85ffb640 1 -- 172.21.15.84:0/3791201824 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8000ed80 con 0x7f8a901f2770 2023-10-16T11:26:16.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.937+0000 7f8a85ffb640 1 -- 172.21.15.84:0/3791201824 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8001c620 con 0x7f8a901f2770 2023-10-16T11:26:16.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.937+0000 7f8a96640640 1 -- 172.21.15.84:0/3791201824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a900c7550 con 0x7f8a901f2770 2023-10-16T11:26:16.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.937+0000 7f8a96640640 1 -- 172.21.15.84:0/3791201824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a901589f0 con 0x7f8a901f2770 2023-10-16T11:26:16.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.939+0000 7f8a96640640 1 -- 172.21.15.84:0/3791201824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a9004e9b0 con 0x7f8a901f2770 2023-10-16T11:26:16.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.939+0000 7f8a85ffb640 1 -- 172.21.15.84:0/3791201824 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 149) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f8a8001c780 con 0x7f8a901f2770 2023-10-16T11:26:16.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.940+0000 7f8a85ffb640 1 --2- 172.21.15.84:0/3791201824 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f8a7406acf0 0x7f8a7406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:16.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.940+0000 7f8a85ffb640 1 -- 172.21.15.84:0/3791201824 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7461+0+0 (secure 0 0 0) 0x7f8a80089e90 con 0x7f8a901f2770 2023-10-16T11:26:16.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.941+0000 7f8a9563e640 1 --2- 172.21.15.84:0/3791201824 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f8a7406acf0 0x7f8a7406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:16.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.941+0000 7f8a9563e640 1 --2- 172.21.15.84:0/3791201824 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f8a7406acf0 0x7f8a7406d1b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00ba00 tx=0x7f8a8c03d040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:16.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:16.945+0000 7f8a85ffb640 1 -- 172.21.15.84:0/3791201824 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8a80054c50 con 0x7f8a901f2770 2023-10-16T11:26:17.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.142+0000 7f8a96640640 1 -- 172.21.15.84:0/3791201824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f8a9006b2b0 con 0x7f8a901f2770 2023-10-16T11:26:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.143+0000 7f8a85ffb640 1 -- 172.21.15.84:0/3791201824 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v243) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8a80058270 con 0x7f8a901f2770 2023-10-16T11:26:17.145 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:26:17.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.145+0000 7f8a96640640 1 -- 172.21.15.84:0/3791201824 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f8a7406acf0 msgr2=0x7f8a7406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:17.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.145+0000 7f8a96640640 1 --2- 172.21.15.84:0/3791201824 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f8a7406acf0 0x7f8a7406d1b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00ba00 tx=0x7f8a8c03d040 comp rx=0 tx=0).stop 2023-10-16T11:26:17.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.145+0000 7f8a96640640 1 -- 172.21.15.84:0/3791201824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a901f2770 msgr2=0x7f8a900b0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:17.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.145+0000 7f8a96640640 1 --2- 172.21.15.84:0/3791201824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a901f2770 0x7f8a900b0790 secure :-1 s=READY pgs=3098 cs=0 l=1 rev1=1 crypto rx=0x7f8a800109d0 tx=0x7f8a80010e90 comp rx=0 tx=0).stop 2023-10-16T11:26:17.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.145+0000 7f8a96640640 1 -- 172.21.15.84:0/3791201824 shutdown_connections 2023-10-16T11:26:17.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.145+0000 7f8a96640640 1 --2- 172.21.15.84:0/3791201824 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f8a7406acf0 0x7f8a7406d1b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.145+0000 7f8a96640640 1 --2- 172.21.15.84:0/3791201824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a901f2bf0 0x7f8a900ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.145+0000 7f8a96640640 1 --2- 172.21.15.84:0/3791201824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a901f2770 0x7f8a900b0790 unknown :-1 s=CLOSED pgs=3098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.145+0000 7f8a96640640 1 --2- 172.21.15.84:0/3791201824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c003970 0x7f8a901584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.145+0000 7f8a96640640 1 -- 172.21.15.84:0/3791201824 >> 172.21.15.84:0/3791201824 conn(0x7f8a900ce6a0 msgr2=0x7f8a900ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:17.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.146+0000 7f8a96640640 1 -- 172.21.15.84:0/3791201824 shutdown_connections 2023-10-16T11:26:17.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.146+0000 7f8a96640640 1 -- 172.21.15.84:0/3791201824 wait complete. 2023-10-16T11:26:17.170 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:26:17.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.379+0000 7fdbd3378640 1 Processor -- start 2023-10-16T11:26:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.379+0000 7fdbd3378640 1 -- start start 2023-10-16T11:26:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.380+0000 7fdbd3378640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdbcc15aaa0 0x7fdbcc15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:17.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.380+0000 7fdbd3378640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdbcc15ae80 0x7fdbcc0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:17.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.380+0000 7fdbd3378640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbcc0ccaa0 0x7fdbcc070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:17.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.380+0000 7fdbd3378640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbcc06b810 con 0x7fdbcc15aaa0 2023-10-16T11:26:17.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.381+0000 7fdbd3378640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbcc15ed60 con 0x7fdbcc15ae80 2023-10-16T11:26:17.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.381+0000 7fdbd3378640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc0707e0 con 0x7fdbcc0ccaa0 2023-10-16T11:26:17.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.381+0000 7fdbd1114640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdbcc15aaa0 0x7fdbcc15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:17.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.381+0000 7fdbd1114640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdbcc15aaa0 0x7fdbcc15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35194/0 (socket says 172.21.15.84:35194) 2023-10-16T11:26:17.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.381+0000 7fdbd1114640 1 -- 172.21.15.84:0/3031587523 learned_addr learned my addr 172.21.15.84:0/3031587523 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:17.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.381+0000 7fdbd1915640 1 --2- 172.21.15.84:0/3031587523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbcc0ccaa0 0x7fdbcc070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:17.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.381+0000 7fdbd0913640 1 --2- 172.21.15.84:0/3031587523 >> v2:172.21.15.157:3300/0 conn(0x7fdbcc15ae80 0x7fdbcc0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:17.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.381+0000 7fdbd1114640 1 -- 172.21.15.84:0/3031587523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbcc0ccaa0 msgr2=0x7fdbcc070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:17.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.382+0000 7fdbd1114640 1 --2- 172.21.15.84:0/3031587523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbcc0ccaa0 0x7fdbcc070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.382+0000 7fdbd1114640 1 -- 172.21.15.84:0/3031587523 >> v2:172.21.15.157:3300/0 conn(0x7fdbcc15ae80 msgr2=0x7fdbcc0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:17.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.382+0000 7fdbd1114640 1 --2- 172.21.15.84:0/3031587523 >> v2:172.21.15.157:3300/0 conn(0x7fdbcc15ae80 0x7fdbcc0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.382+0000 7fdbd1114640 1 -- 172.21.15.84:0/3031587523 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbcc15ef90 con 0x7fdbcc15aaa0 2023-10-16T11:26:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.383+0000 7fdbd1114640 1 --2- 172.21.15.84:0/3031587523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbcc15aaa0 0x7fdbcc15e820 secure :-1 s=READY pgs=3099 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0c59c0 tx=0x7fdbc802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8d2dbb8b38e6d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.383+0000 7fdbc3fff640 1 -- 172.21.15.84:0/3031587523 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8031870 con 0x7fdbcc15aaa0 2023-10-16T11:26:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.383+0000 7fdbc3fff640 1 -- 172.21.15.84:0/3031587523 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc80319d0 con 0x7fdbcc15aaa0 2023-10-16T11:26:17.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.383+0000 7fdbc3fff640 1 -- 172.21.15.84:0/3031587523 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8031d60 con 0x7fdbcc15aaa0 2023-10-16T11:26:17.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.384+0000 7fdbd3378640 1 -- 172.21.15.84:0/3031587523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbcc15aaa0 msgr2=0x7fdbcc15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:17.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.384+0000 7fdbd3378640 1 --2- 172.21.15.84:0/3031587523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbcc15aaa0 0x7fdbcc15e820 secure :-1 s=READY pgs=3099 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0c59c0 tx=0x7fdbc802b910 comp rx=0 tx=0).stop 2023-10-16T11:26:17.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.384+0000 7fdbd3378640 1 -- 172.21.15.84:0/3031587523 shutdown_connections 2023-10-16T11:26:17.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.384+0000 7fdbd3378640 1 --2- 172.21.15.84:0/3031587523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbcc0ccaa0 0x7fdbcc070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.384+0000 7fdbd3378640 1 --2- 172.21.15.84:0/3031587523 >> v2:172.21.15.157:3300/0 conn(0x7fdbcc15ae80 0x7fdbcc0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.384+0000 7fdbd3378640 1 --2- 172.21.15.84:0/3031587523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbcc15aaa0 0x7fdbcc15e820 unknown :-1 s=CLOSED pgs=3099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.384+0000 7fdbd3378640 1 -- 172.21.15.84:0/3031587523 >> 172.21.15.84:0/3031587523 conn(0x7fdbcc0cb410 msgr2=0x7fdbcc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:17.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.384+0000 7fdbd3378640 1 -- 172.21.15.84:0/3031587523 shutdown_connections 2023-10-16T11:26:17.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.384+0000 7fdbd3378640 1 -- 172.21.15.84:0/3031587523 wait complete. 2023-10-16T11:26:17.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.386+0000 7fdbd3378640 1 Processor -- start 2023-10-16T11:26:17.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.386+0000 7fdbd3378640 1 -- start start 2023-10-16T11:26:17.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.387+0000 7fdbd3378640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbcc0ccaa0 0x7fdbcc0c4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:17.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.387+0000 7fdbd3378640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbcc15aaa0 0x7fdbcc15c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:17.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.387+0000 7fdbd3378640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbcc15ae80 0x7fdbcc15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:17.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.387+0000 7fdbd3378640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc159de0 con 0x7fdbcc0ccaa0 2023-10-16T11:26:17.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.387+0000 7fdbd3378640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc149960 con 0x7fdbcc15aaa0 2023-10-16T11:26:17.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.387+0000 7fdbd3378640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc149680 con 0x7fdbcc15ae80 2023-10-16T11:26:17.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.387+0000 7fdbd1114640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbcc0ccaa0 0x7fdbcc0c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:17.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.387+0000 7fdbd1114640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbcc0ccaa0 0x7fdbcc0c4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35206/0 (socket says 172.21.15.84:35206) 2023-10-16T11:26:17.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.387+0000 7fdbd1114640 1 -- 172.21.15.84:0/1647547653 learned_addr learned my addr 172.21.15.84:0/1647547653 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:17.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.387+0000 7fdbd1114640 1 -- 172.21.15.84:0/1647547653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbcc15ae80 msgr2=0x7fdbcc15cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:17.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.387+0000 7fdbd1114640 1 --2- 172.21.15.84:0/1647547653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbcc15ae80 0x7fdbcc15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.387+0000 7fdbd1114640 1 -- 172.21.15.84:0/1647547653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbcc15aaa0 msgr2=0x7fdbcc15c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:17.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.387+0000 7fdbd1114640 1 --2- 172.21.15.84:0/1647547653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbcc15aaa0 0x7fdbcc15c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.387+0000 7fdbd1114640 1 -- 172.21.15.84:0/1647547653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc80094d0 con 0x7fdbcc0ccaa0 2023-10-16T11:26:17.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.388+0000 7fdbd1114640 1 --2- 172.21.15.84:0/1647547653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbcc0ccaa0 0x7fdbcc0c4980 secure :-1 s=READY pgs=3100 cs=0 l=1 rev1=1 crypto rx=0x7fdbc802b950 tx=0x7fdbc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:17.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.388+0000 7fdbc1ffb640 1 -- 172.21.15.84:0/1647547653 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc803ad30 con 0x7fdbcc0ccaa0 2023-10-16T11:26:17.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.388+0000 7fdbc1ffb640 1 -- 172.21.15.84:0/1647547653 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc80443f0 con 0x7fdbcc0ccaa0 2023-10-16T11:26:17.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.388+0000 7fdbd3378640 1 -- 172.21.15.84:0/1647547653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbcc146ad0 con 0x7fdbcc0ccaa0 2023-10-16T11:26:17.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.388+0000 7fdbc1ffb640 1 -- 172.21.15.84:0/1647547653 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80447a0 con 0x7fdbcc0ccaa0 2023-10-16T11:26:17.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.388+0000 7fdbd3378640 1 -- 172.21.15.84:0/1647547653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbcc147090 con 0x7fdbcc0ccaa0 2023-10-16T11:26:17.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.389+0000 7fdbd3378640 1 -- 172.21.15.84:0/1647547653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbcc147280 con 0x7fdbcc0ccaa0 2023-10-16T11:26:17.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.390+0000 7fdbc1ffb640 1 -- 172.21.15.84:0/1647547653 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fdbc804d450 con 0x7fdbcc0ccaa0 2023-10-16T11:26:17.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.392+0000 7fdbc1ffb640 1 --2- 172.21.15.84:0/1647547653 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fdbb006b020 0x7fdbb006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:17.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.392+0000 7fdbc1ffb640 1 -- 172.21.15.84:0/1647547653 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7461+0+0 (secure 0 0 0) 0x7fdbc802d040 con 0x7fdbcc0ccaa0 2023-10-16T11:26:17.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.392+0000 7fdbd0913640 1 --2- 172.21.15.84:0/1647547653 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fdbb006b020 0x7fdbb006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:17.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.393+0000 7fdbd0913640 1 --2- 172.21.15.84:0/1647547653 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fdbb006b020 0x7fdbb006d4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc1f6320 tx=0x7fdbbc008040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:17.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.396+0000 7fdbc1ffb640 1 -- 172.21.15.84:0/1647547653 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdbc8076d70 con 0x7fdbcc0ccaa0 2023-10-16T11:26:17.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.629+0000 7fdbd3378640 1 -- 172.21.15.84:0/1647547653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbcc04e160 con 0x7fdbcc0ccaa0 2023-10-16T11:26:17.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.630+0000 7fdbc1ffb640 1 -- 172.21.15.84:0/1647547653 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+6155 (secure 0 0 0) 0x7fdbc8035090 con 0x7fdbcc0ccaa0 2023-10-16T11:26:17.633 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:17.633 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":137,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2516077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2516077743},{"type":"v1","addr":"172.21.15.157:6833","nonce":2516077743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1527104462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1527104462},{"type":"v1","addr":"172.21.15.157:6835","nonce":1527104462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:25:36.605068+0000","modified":"2023-10-16T11:25:39.675208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12186":{"gid":12186,"name":"a","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/3547503033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3547503033},{"type":"v1","addr":"172.21.15.84:6837","nonce":3547503033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:25:50.739234+0000","modified":"2023-10-16T11:25:53.792071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12183":{"gid":12183,"name":"c","rank":0,"incarnation":133,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/369067539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":369067539},{"type":"v1","addr":"172.21.15.84:6835","nonce":369067539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-16T11:26:17.634 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 137 2023-10-16T11:26:17.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.633+0000 7fdbd3378640 1 -- 172.21.15.84:0/1647547653 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fdbb006b020 msgr2=0x7fdbb006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:17.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.633+0000 7fdbd3378640 1 --2- 172.21.15.84:0/1647547653 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fdbb006b020 0x7fdbb006d4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc1f6320 tx=0x7fdbbc008040 comp rx=0 tx=0).stop 2023-10-16T11:26:17.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.633+0000 7fdbd3378640 1 -- 172.21.15.84:0/1647547653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbcc0ccaa0 msgr2=0x7fdbcc0c4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:17.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.633+0000 7fdbd3378640 1 --2- 172.21.15.84:0/1647547653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbcc0ccaa0 0x7fdbcc0c4980 secure :-1 s=READY pgs=3100 cs=0 l=1 rev1=1 crypto rx=0x7fdbc802b950 tx=0x7fdbc80046c0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.633+0000 7fdbd3378640 1 -- 172.21.15.84:0/1647547653 shutdown_connections 2023-10-16T11:26:17.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.633+0000 7fdbd3378640 1 --2- 172.21.15.84:0/1647547653 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fdbb006b020 0x7fdbb006d4e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.633+0000 7fdbd3378640 1 --2- 172.21.15.84:0/1647547653 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdbcc15ae80 0x7fdbcc15cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.633+0000 7fdbd3378640 1 --2- 172.21.15.84:0/1647547653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbcc15aaa0 0x7fdbcc15c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.633+0000 7fdbd3378640 1 --2- 172.21.15.84:0/1647547653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbcc0ccaa0 0x7fdbcc0c4980 unknown :-1 s=CLOSED pgs=3100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.633+0000 7fdbd3378640 1 -- 172.21.15.84:0/1647547653 >> 172.21.15.84:0/1647547653 conn(0x7fdbcc0cb410 msgr2=0x7fdbcc0b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:17.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.633+0000 7fdbd3378640 1 -- 172.21.15.84:0/1647547653 shutdown_connections 2023-10-16T11:26:17.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.633+0000 7fdbd3378640 1 -- 172.21.15.84:0/1647547653 wait complete. 2023-10-16T11:26:17.647 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:17.643+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:17.653 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:26:17.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.866+0000 7f0027c90640 1 Processor -- start 2023-10-16T11:26:17.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.866+0000 7f0027c90640 1 -- start start 2023-10-16T11:26:17.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.868+0000 7f0027c90640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0020160420 0x7f00200ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:17.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.868+0000 7f0027c90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0020160800 0x7f00200cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:17.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.868+0000 7f0027c90640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f002006e170 con 0x7f00200adc40 2023-10-16T11:26:17.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.868+0000 7f0027c90640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00200cde30 con 0x7f0020160420 2023-10-16T11:26:17.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.868+0000 7f0027c90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00200cd460 con 0x7f0020160800 2023-10-16T11:26:17.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.868+0000 7f0025a2c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0020160420 0x7f00200ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:17.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.868+0000 7f002622d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f00200adc40 0x7f002006adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46982/0 (socket says 172.21.15.84:46982) 2023-10-16T11:26:17.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.868+0000 7f002622d640 1 -- 172.21.15.84:0/3436825468 learned_addr learned my addr 172.21.15.84:0/3436825468 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:17.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.869+0000 7f002522b640 1 --2- 172.21.15.84:0/3436825468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0020160800 0x7f00200cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:17.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.869+0000 7f0025a2c640 1 -- 172.21.15.84:0/3436825468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0020160800 msgr2=0x7f00200cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:17.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.869+0000 7f0025a2c640 1 --2- 172.21.15.84:0/3436825468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0020160800 0x7f00200cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.869+0000 7f0025a2c640 1 -- 172.21.15.84:0/3436825468 >> v1:172.21.15.84:6789/0 conn(0x7f00200adc40 legacy=0x7f002006adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:17.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.869+0000 7f0025a2c640 1 -- 172.21.15.84:0/3436825468 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002006e3c0 con 0x7f0020160420 2023-10-16T11:26:17.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.869+0000 7f002522b640 1 --2- 172.21.15.84:0/3436825468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0020160800 0x7f00200cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:17.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.869+0000 7f0024a2a640 1 -- 172.21.15.84:0/3436825468 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3462581681 0 0) 0x7f002006e170 con 0x7f00200adc40 2023-10-16T11:26:17.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.870+0000 7f0025a2c640 1 --2- 172.21.15.84:0/3436825468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020160420 0x7f00200ae170 secure :-1 s=READY pgs=3102 cs=0 l=1 rev1=1 crypto rx=0x7f001c0097a0 tx=0x7f001c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e235b8cce44832 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:17.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.870+0000 7f0024a2a640 1 -- 172.21.15.84:0/3436825468 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c02d040 con 0x7f0020160420 2023-10-16T11:26:17.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.870+0000 7f0024a2a640 1 -- 172.21.15.84:0/3436825468 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001c031810 con 0x7f0020160420 2023-10-16T11:26:17.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.870+0000 7f0024a2a640 1 -- 172.21.15.84:0/3436825468 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c031ba0 con 0x7f0020160420 2023-10-16T11:26:17.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.871+0000 7f0027c90640 1 -- 172.21.15.84:0/3436825468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020160420 msgr2=0x7f00200ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:17.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.871+0000 7f0027c90640 1 --2- 172.21.15.84:0/3436825468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020160420 0x7f00200ae170 secure :-1 s=READY pgs=3102 cs=0 l=1 rev1=1 crypto rx=0x7f001c0097a0 tx=0x7f001c02b8b0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.871+0000 7f0027c90640 1 -- 172.21.15.84:0/3436825468 shutdown_connections 2023-10-16T11:26:17.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.871+0000 7f0027c90640 1 --2- 172.21.15.84:0/3436825468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0020160800 0x7f00200cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.871+0000 7f0027c90640 1 --2- 172.21.15.84:0/3436825468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020160420 0x7f00200ae170 unknown :-1 s=CLOSED pgs=3102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.871+0000 7f0027c90640 1 -- 172.21.15.84:0/3436825468 >> 172.21.15.84:0/3436825468 conn(0x7f00200b0d30 msgr2=0x7f00200b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:17.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.871+0000 7f0027c90640 1 -- 172.21.15.84:0/3436825468 shutdown_connections 2023-10-16T11:26:17.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.871+0000 7f0027c90640 1 -- 172.21.15.84:0/3436825468 wait complete. 2023-10-16T11:26:17.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.873+0000 7f0027c90640 1 Processor -- start 2023-10-16T11:26:17.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.873+0000 7f0027c90640 1 -- start start 2023-10-16T11:26:17.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.874+0000 7f0027c90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00200adc40 0x7f0020159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:17.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.874+0000 7f0027c90640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020160420 0x7f0020071110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:17.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.874+0000 7f0027c90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0020160800 0x7f0020071650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:17.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.874+0000 7f0027c90640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002015cdd0 con 0x7f0020160420 2023-10-16T11:26:17.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.874+0000 7f0027c90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002015c1c0 con 0x7f0020160800 2023-10-16T11:26:17.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.874+0000 7f0027c90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002015c4a0 con 0x7f00200adc40 2023-10-16T11:26:17.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.874+0000 7f002522b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020160420 0x7f0020071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:17.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.874+0000 7f002522b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020160420 0x7f0020071110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35214/0 (socket says 172.21.15.84:35214) 2023-10-16T11:26:17.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.874+0000 7f002522b640 1 -- 172.21.15.84:0/3264567362 learned_addr learned my addr 172.21.15.84:0/3264567362 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:17.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.875+0000 7f002522b640 1 -- 172.21.15.84:0/3264567362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00200adc40 msgr2=0x7f0020159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:17.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.875+0000 7f002522b640 1 --2- 172.21.15.84:0/3264567362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00200adc40 0x7f0020159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.875+0000 7f002522b640 1 -- 172.21.15.84:0/3264567362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0020160800 msgr2=0x7f0020071650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:17.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.875+0000 7f002522b640 1 --2- 172.21.15.84:0/3264567362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0020160800 0x7f0020071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:17.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.875+0000 7f002522b640 1 -- 172.21.15.84:0/3264567362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c009400 con 0x7f0020160420 2023-10-16T11:26:17.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.875+0000 7f002522b640 1 --2- 172.21.15.84:0/3264567362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020160420 0x7f0020071110 secure :-1 s=READY pgs=3103 cs=0 l=1 rev1=1 crypto rx=0x7f001000c7f0 tx=0x7f001000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:17.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.875+0000 7f00167fc640 1 -- 172.21.15.84:0/3264567362 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010007820 con 0x7f0020160420 2023-10-16T11:26:17.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.875+0000 7f00167fc640 1 -- 172.21.15.84:0/3264567362 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0010007980 con 0x7f0020160420 2023-10-16T11:26:17.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.875+0000 7f0027c90640 1 -- 172.21.15.84:0/3264567362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002015c780 con 0x7f0020160420 2023-10-16T11:26:17.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.875+0000 7f00167fc640 1 -- 172.21.15.84:0/3264567362 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010007cf0 con 0x7f0020160420 2023-10-16T11:26:17.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.875+0000 7f0027c90640 1 -- 172.21.15.84:0/3264567362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002015a3b0 con 0x7f0020160420 2023-10-16T11:26:17.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.876+0000 7f0027c90640 1 -- 172.21.15.84:0/3264567362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002004e160 con 0x7f0020160420 2023-10-16T11:26:17.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.878+0000 7f00167fc640 1 -- 172.21.15.84:0/3264567362 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f00100113f0 con 0x7f0020160420 2023-10-16T11:26:17.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.878+0000 7f00167fc640 1 --2- 172.21.15.84:0/3264567362 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f000406b020 0x7f000406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:17.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.879+0000 7f00167fc640 1 -- 172.21.15.84:0/3264567362 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7461+0+0 (secure 0 0 0) 0x7f001008a430 con 0x7f0020160420 2023-10-16T11:26:17.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.879+0000 7f0025a2c640 1 --2- 172.21.15.84:0/3264567362 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f000406b020 0x7f000406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:17.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.880+0000 7f0025a2c640 1 --2- 172.21.15.84:0/3264567362 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f000406b020 0x7f000406d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f001c036000 tx=0x7f001c030000 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:17.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:17.883+0000 7f00167fc640 1 -- 172.21.15.84:0/3264567362 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0010054a70 con 0x7f0020160420 2023-10-16T11:26:18.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.082+0000 7f0027c90640 1 -- 172.21.15.84:0/3264567362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f002015cf10 con 0x7f0020160420 2023-10-16T11:26:18.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.085+0000 7f00167fc640 1 -- 172.21.15.84:0/3264567362 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v243) v1 ==== 74+0+21450 (secure 0 0 0) 0x7f0010058090 con 0x7f0020160420 2023-10-16T11:26:18.088 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:18.088 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":243,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:26:14.475537+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:25:34.443897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:25:35.635615+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:25:49.001107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:25:49.760637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2606713455":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/488945617":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/4036998934":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6832/3938045451":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/570042115":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1311117536":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6833/2963529201":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/95207886":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6832/2963529201":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/388205040":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/2303342316":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/648102622":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/133211355":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1263886961":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1489017743":"2023-10-17T11:26:08.077644+0000","172.21.15.84:6833/3938045451":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1517291402":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1983743360":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1323010124":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/437386817":"2023-10-17T11:26:08.077644+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:26:18.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.088+0000 7f0027c90640 1 -- 172.21.15.84:0/3264567362 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f000406b020 msgr2=0x7f000406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:18.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.088+0000 7f0027c90640 1 --2- 172.21.15.84:0/3264567362 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f000406b020 0x7f000406d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f001c036000 tx=0x7f001c030000 comp rx=0 tx=0).stop 2023-10-16T11:26:18.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.088+0000 7f0027c90640 1 -- 172.21.15.84:0/3264567362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020160420 msgr2=0x7f0020071110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:18.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.088+0000 7f0027c90640 1 --2- 172.21.15.84:0/3264567362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020160420 0x7f0020071110 secure :-1 s=READY pgs=3103 cs=0 l=1 rev1=1 crypto rx=0x7f001000c7f0 tx=0x7f001000ccb0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.088+0000 7f0027c90640 1 -- 172.21.15.84:0/3264567362 shutdown_connections 2023-10-16T11:26:18.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.088+0000 7f0027c90640 1 --2- 172.21.15.84:0/3264567362 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f000406b020 0x7f000406d4e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.088+0000 7f0027c90640 1 --2- 172.21.15.84:0/3264567362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0020160800 0x7f0020071650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.088+0000 7f0027c90640 1 --2- 172.21.15.84:0/3264567362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020160420 0x7f0020071110 unknown :-1 s=CLOSED pgs=3103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.088+0000 7f0027c90640 1 --2- 172.21.15.84:0/3264567362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00200adc40 0x7f0020159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.088+0000 7f0027c90640 1 -- 172.21.15.84:0/3264567362 >> 172.21.15.84:0/3264567362 conn(0x7f00200b0d30 msgr2=0x7f00200b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:18.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.088+0000 7f0027c90640 1 -- 172.21.15.84:0/3264567362 shutdown_connections 2023-10-16T11:26:18.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.088+0000 7f0027c90640 1 -- 172.21.15.84:0/3264567362 wait complete. 2023-10-16T11:26:18.110 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T11:26:18.111 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:26:18.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.340+0000 7f5597dd5640 1 Processor -- start 2023-10-16T11:26:18.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.340+0000 7f5597dd5640 1 -- start start 2023-10-16T11:26:18.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.341+0000 7f5597dd5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55900b0b20 0x7f55901615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:18.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.342+0000 7f5597dd5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f559015e3e0 con 0x7f55900cd0c0 2023-10-16T11:26:18.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.342+0000 7f5597dd5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55900cd4a0 con 0x7f559015d3f0 2023-10-16T11:26:18.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.342+0000 7f5597dd5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559015e100 con 0x7f55900b0b20 2023-10-16T11:26:18.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.342+0000 7f5595b71640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f55900cd0c0 0x7f55900b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46986/0 (socket says 172.21.15.84:46986) 2023-10-16T11:26:18.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.342+0000 7f5595b71640 1 -- 172.21.15.84:0/2893068067 learned_addr learned my addr 172.21.15.84:0/2893068067 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:18.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.342+0000 7f5595370640 1 --2- 172.21.15.84:0/2893068067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55900b0b20 0x7f55901615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:18.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.342+0000 7f5594b6f640 1 -- 172.21.15.84:0/2893068067 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048443569 0 0) 0x7f559015e3e0 con 0x7f55900cd0c0 2023-10-16T11:26:18.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.342+0000 7f5594b6f640 1 -- 172.21.15.84:0/2893068067 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f557c003520 con 0x7f55900cd0c0 2023-10-16T11:26:18.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.343+0000 7f5594b6f640 1 -- 172.21.15.84:0/2893068067 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3650776319 0 0) 0x7f557c003520 con 0x7f55900cd0c0 2023-10-16T11:26:18.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.343+0000 7f5594b6f640 1 -- 172.21.15.84:0/2893068067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55900b0b20 msgr2=0x7f55901615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:18.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.343+0000 7f5594b6f640 1 --2- 172.21.15.84:0/2893068067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55900b0b20 0x7f55901615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.343+0000 7f5594b6f640 1 -- 172.21.15.84:0/2893068067 >> v1:172.21.15.157:6789/0 conn(0x7f559015d3f0 legacy=0x7f559015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:18.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.343+0000 7f5594b6f640 1 -- 172.21.15.84:0/2893068067 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5590160b10 con 0x7f55900cd0c0 2023-10-16T11:26:18.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.343+0000 7f5594b6f640 1 -- 172.21.15.84:0/2893068067 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f558c001670 con 0x7f55900cd0c0 2023-10-16T11:26:18.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.343+0000 7f5594b6f640 1 -- 172.21.15.84:0/2893068067 >> v1:172.21.15.84:6789/0 conn(0x7f55900cd0c0 legacy=0x7f55900b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:18.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.343+0000 7f5594b6f640 1 --2- 172.21.15.84:0/2893068067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c0039a0 0x7f557c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:18.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.343+0000 7f5594b6f640 1 --2- 172.21.15.84:0/2893068067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55900b0b20 0x7f557c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:18.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.343+0000 7f5594b6f640 1 --2- 172.21.15.84:0/2893068067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f557c004d00 0x7f557c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:18.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.343+0000 7f5594b6f640 1 -- 172.21.15.84:0/2893068067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559015e100 con 0x7f557c004d00 2023-10-16T11:26:18.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.343+0000 7f5594b6f640 1 -- 172.21.15.84:0/2893068067 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557c00bb10 con 0x7f557c0039a0 2023-10-16T11:26:18.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.343+0000 7f5594b6f640 1 -- 172.21.15.84:0/2893068067 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557c00bcd0 con 0x7f55900b0b20 2023-10-16T11:26:18.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.343+0000 7f5596372640 1 --2- 172.21.15.84:0/2893068067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f557c004d00 0x7f557c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:18.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.344+0000 7f5595b71640 1 --2- 172.21.15.84:0/2893068067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c0039a0 0x7f557c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:18.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.344+0000 7f5595370640 1 --2- 172.21.15.84:0/2893068067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55900b0b20 0x7f557c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:18.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.344+0000 7f5596372640 1 -- 172.21.15.84:0/2893068067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55900b0b20 msgr2=0x7f557c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:18.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.344+0000 7f5596372640 1 --2- 172.21.15.84:0/2893068067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55900b0b20 0x7f557c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.344+0000 7f5596372640 1 -- 172.21.15.84:0/2893068067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c0039a0 msgr2=0x7f557c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:18.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.344+0000 7f5596372640 1 --2- 172.21.15.84:0/2893068067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c0039a0 0x7f557c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.344+0000 7f5596372640 1 -- 172.21.15.84:0/2893068067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557c003520 con 0x7f557c004d00 2023-10-16T11:26:18.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.345+0000 7f5596372640 1 --2- 172.21.15.84:0/2893068067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f557c004d00 0x7f557c005180 secure :-1 s=READY pgs=3105 cs=0 l=1 rev1=1 crypto rx=0x7f55880044b0 tx=0x7f557c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25030c9fc257edde server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:18.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.345+0000 7f5594b6f640 1 -- 172.21.15.84:0/2893068067 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588030620 con 0x7f557c004d00 2023-10-16T11:26:18.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.345+0000 7f5594b6f640 1 -- 172.21.15.84:0/2893068067 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5588030780 con 0x7f557c004d00 2023-10-16T11:26:18.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.345+0000 7f5594b6f640 1 -- 172.21.15.84:0/2893068067 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588030b30 con 0x7f557c004d00 2023-10-16T11:26:18.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.346+0000 7f5597dd5640 1 -- 172.21.15.84:0/2893068067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f557c004d00 msgr2=0x7f557c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:18.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.346+0000 7f5597dd5640 1 --2- 172.21.15.84:0/2893068067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f557c004d00 0x7f557c005180 secure :-1 s=READY pgs=3105 cs=0 l=1 rev1=1 crypto rx=0x7f55880044b0 tx=0x7f557c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.346+0000 7f5596372640 1 -- 172.21.15.84:0/2893068067 reap_dead start 2023-10-16T11:26:18.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.346+0000 7f5597dd5640 1 -- 172.21.15.84:0/2893068067 shutdown_connections 2023-10-16T11:26:18.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.346+0000 7f5597dd5640 1 -- 172.21.15.84:0/2893068067 >> 172.21.15.84:0/2893068067 conn(0x7f55900cb410 msgr2=0x7f55900cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:18.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.346+0000 7f5597dd5640 1 -- 172.21.15.84:0/2893068067 shutdown_connections 2023-10-16T11:26:18.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.346+0000 7f5597dd5640 1 -- 172.21.15.84:0/2893068067 wait complete. 2023-10-16T11:26:18.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.348+0000 7f5597dd5640 1 Processor -- start 2023-10-16T11:26:18.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.348+0000 7f5597dd5640 1 -- start start 2023-10-16T11:26:18.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.349+0000 7f5597dd5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f557c004d00 0x7f55900c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:18.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.349+0000 7f5597dd5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55901f28b0 0x7f55900c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:18.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.349+0000 7f5597dd5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55901f60d0 0x7f55901f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:18.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.349+0000 7f5597dd5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55901f38d0 con 0x7f55901f60d0 2023-10-16T11:26:18.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.349+0000 7f5597dd5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55900b3ca0 con 0x7f55901f28b0 2023-10-16T11:26:18.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.349+0000 7f5597dd5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55901f2d50 con 0x7f557c004d00 2023-10-16T11:26:18.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.350+0000 7f5596372640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55901f60d0 0x7f55901f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:18.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.350+0000 7f5596372640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55901f60d0 0x7f55901f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35228/0 (socket says 172.21.15.84:35228) 2023-10-16T11:26:18.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.350+0000 7f5596372640 1 -- 172.21.15.84:0/733674380 learned_addr learned my addr 172.21.15.84:0/733674380 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:18.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.350+0000 7f5595b71640 1 --2- 172.21.15.84:0/733674380 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f557c004d00 0x7f55900c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:18.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.350+0000 7f5595370640 1 --2- 172.21.15.84:0/733674380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55901f28b0 0x7f55900c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:18.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.350+0000 7f5596372640 1 -- 172.21.15.84:0/733674380 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f557c004d00 msgr2=0x7f55900c24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:18.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.350+0000 7f5596372640 1 --2- 172.21.15.84:0/733674380 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f557c004d00 0x7f55900c24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.350+0000 7f5596372640 1 -- 172.21.15.84:0/733674380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55901f28b0 msgr2=0x7f55900c2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:18.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.350+0000 7f5596372640 1 --2- 172.21.15.84:0/733674380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55901f28b0 0x7f55900c2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.350+0000 7f5596372640 1 -- 172.21.15.84:0/733674380 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55901f3030 con 0x7f55901f60d0 2023-10-16T11:26:18.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.351+0000 7f5596372640 1 --2- 172.21.15.84:0/733674380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55901f60d0 0x7f55901f6580 secure :-1 s=READY pgs=3106 cs=0 l=1 rev1=1 crypto rx=0x7f55880045d0 tx=0x7f5588003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:18.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.351+0000 7f55867fc640 1 -- 172.21.15.84:0/733674380 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558803f070 con 0x7f55901f60d0 2023-10-16T11:26:18.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.351+0000 7f55867fc640 1 -- 172.21.15.84:0/733674380 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5588006e00 con 0x7f55901f60d0 2023-10-16T11:26:18.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.351+0000 7f55867fc640 1 -- 172.21.15.84:0/733674380 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588030d00 con 0x7f55901f60d0 2023-10-16T11:26:18.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.351+0000 7f5597dd5640 1 -- 172.21.15.84:0/733674380 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55901f3310 con 0x7f55901f60d0 2023-10-16T11:26:18.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.351+0000 7f5597dd5640 1 -- 172.21.15.84:0/733674380 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55901f6ac0 con 0x7f55901f60d0 2023-10-16T11:26:18.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.352+0000 7f5597dd5640 1 -- 172.21.15.84:0/733674380 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f559004e160 con 0x7f55901f60d0 2023-10-16T11:26:18.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.353+0000 7f55867fc640 1 -- 172.21.15.84:0/733674380 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f5588039450 con 0x7f55901f60d0 2023-10-16T11:26:18.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.354+0000 7f55867fc640 1 --2- 172.21.15.84:0/733674380 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f557406b020 0x7f557406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:18.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.355+0000 7f55867fc640 1 -- 172.21.15.84:0/733674380 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7461+0+0 (secure 0 0 0) 0x7f55880ab980 con 0x7f55901f60d0 2023-10-16T11:26:18.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.355+0000 7f5595b71640 1 --2- 172.21.15.84:0/733674380 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f557406b020 0x7f557406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:18.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.355+0000 7f5595b71640 1 --2- 172.21.15.84:0/733674380 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f557406b020 0x7f557406d4e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f558c001050 tx=0x7f558c009040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:18.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.359+0000 7f55867fc640 1 -- 172.21.15.84:0/733674380 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5588075fb0 con 0x7f55901f60d0 2023-10-16T11:26:18.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.602+0000 7f5597dd5640 1 -- 172.21.15.84:0/733674380 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55901f3830 con 0x7f55901f60d0 2023-10-16T11:26:18.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.603+0000 7f55867fc640 1 -- 172.21.15.84:0/733674380 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+6155 (secure 0 0 0) 0x7f5588049090 con 0x7f55901f60d0 2023-10-16T11:26:18.605 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:18.606 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":137,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2516077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2516077743},{"type":"v1","addr":"172.21.15.157:6833","nonce":2516077743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1527104462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1527104462},{"type":"v1","addr":"172.21.15.157:6835","nonce":1527104462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:25:36.605068+0000","modified":"2023-10-16T11:25:39.675208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12186":{"gid":12186,"name":"a","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/3547503033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3547503033},{"type":"v1","addr":"172.21.15.84:6837","nonce":3547503033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:25:50.739234+0000","modified":"2023-10-16T11:25:53.792071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12183":{"gid":12183,"name":"c","rank":0,"incarnation":133,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/369067539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":369067539},{"type":"v1","addr":"172.21.15.84:6835","nonce":369067539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-16T11:26:18.606 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 137 2023-10-16T11:26:18.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.606+0000 7f5597dd5640 1 -- 172.21.15.84:0/733674380 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f557406b020 msgr2=0x7f557406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:18.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.606+0000 7f5597dd5640 1 --2- 172.21.15.84:0/733674380 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f557406b020 0x7f557406d4e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f558c001050 tx=0x7f558c009040 comp rx=0 tx=0).stop 2023-10-16T11:26:18.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.606+0000 7f5597dd5640 1 -- 172.21.15.84:0/733674380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55901f60d0 msgr2=0x7f55901f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:18.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.606+0000 7f5597dd5640 1 --2- 172.21.15.84:0/733674380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55901f60d0 0x7f55901f6580 secure :-1 s=READY pgs=3106 cs=0 l=1 rev1=1 crypto rx=0x7f55880045d0 tx=0x7f5588003a70 comp rx=0 tx=0).stop 2023-10-16T11:26:18.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.606+0000 7f5597dd5640 1 -- 172.21.15.84:0/733674380 shutdown_connections 2023-10-16T11:26:18.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.606+0000 7f5597dd5640 1 --2- 172.21.15.84:0/733674380 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f557406b020 0x7f557406d4e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.606+0000 7f5597dd5640 1 --2- 172.21.15.84:0/733674380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55901f60d0 0x7f55901f6580 unknown :-1 s=CLOSED pgs=3106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.606+0000 7f5597dd5640 1 --2- 172.21.15.84:0/733674380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55901f28b0 0x7f55900c2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.606+0000 7f5597dd5640 1 --2- 172.21.15.84:0/733674380 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f557c004d00 0x7f55900c24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.606+0000 7f5597dd5640 1 -- 172.21.15.84:0/733674380 >> 172.21.15.84:0/733674380 conn(0x7f55900cb410 msgr2=0x7f55900b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:18.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.606+0000 7f5597dd5640 1 -- 172.21.15.84:0/733674380 shutdown_connections 2023-10-16T11:26:18.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.606+0000 7f5597dd5640 1 -- 172.21.15.84:0/733674380 wait complete. 2023-10-16T11:26:18.627 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:26:18.766 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:18.763+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:18.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.839+0000 7f7cb2333640 1 Processor -- start 2023-10-16T11:26:18.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.839+0000 7f7cb2333640 1 -- start start 2023-10-16T11:26:18.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.841+0000 7f7cb2333640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cac159b20 0x7f7cac06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:18.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.841+0000 7f7cb2333640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7cac159f00 0x7f7cac0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:18.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.841+0000 7f7cb2333640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cac06f7f0 con 0x7f7cac06c120 2023-10-16T11:26:18.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.841+0000 7f7cb2333640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cac06c5e0 con 0x7f7cac159f00 2023-10-16T11:26:18.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.841+0000 7f7cb2333640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac06c8c0 con 0x7f7cac159b20 2023-10-16T11:26:18.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.841+0000 7f7cb08d0640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7cac06c120 0x7f7cac0cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46994/0 (socket says 172.21.15.84:46994) 2023-10-16T11:26:18.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.841+0000 7f7cb08d0640 1 -- 172.21.15.84:0/1390305480 learned_addr learned my addr 172.21.15.84:0/1390305480 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:18.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.842+0000 7f7ca3fff640 1 --2- 172.21.15.84:0/1390305480 >> v2:172.21.15.157:3300/0 conn(0x7f7cac159f00 0x7f7cac0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:18.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.842+0000 7f7cabfff640 1 --2- 172.21.15.84:0/1390305480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cac159b20 0x7f7cac06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:18.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.842+0000 7f7cab7fe640 1 -- 172.21.15.84:0/1390305480 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3651802145 0 0) 0x7f7cac06f7f0 con 0x7f7cac06c120 2023-10-16T11:26:18.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.842+0000 7f7cab7fe640 1 -- 172.21.15.84:0/1390305480 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c98003520 con 0x7f7cac06c120 2023-10-16T11:26:18.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.842+0000 7f7cab7fe640 1 -- 172.21.15.84:0/1390305480 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2306350097 0 0) 0x7f7c98003520 con 0x7f7cac06c120 2023-10-16T11:26:18.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.842+0000 7f7cab7fe640 1 -- 172.21.15.84:0/1390305480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cac159b20 msgr2=0x7f7cac06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:18.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.842+0000 7f7cab7fe640 1 --2- 172.21.15.84:0/1390305480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cac159b20 0x7f7cac06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.842+0000 7f7cab7fe640 1 -- 172.21.15.84:0/1390305480 >> v2:172.21.15.157:3300/0 conn(0x7f7cac159f00 msgr2=0x7f7cac0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:18.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.842+0000 7f7cab7fe640 1 --2- 172.21.15.84:0/1390305480 >> v2:172.21.15.157:3300/0 conn(0x7f7cac159f00 0x7f7cac0b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.842+0000 7f7cab7fe640 1 -- 172.21.15.84:0/1390305480 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac06cba0 con 0x7f7cac06c120 2023-10-16T11:26:18.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.842+0000 7f7cab7fe640 1 -- 172.21.15.84:0/1390305480 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f7ca40019f0 con 0x7f7cac06c120 2023-10-16T11:26:18.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.842+0000 7f7cab7fe640 1 -- 172.21.15.84:0/1390305480 >> v1:172.21.15.84:6789/0 conn(0x7f7cac06c120 legacy=0x7f7cac0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:18.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.842+0000 7f7cab7fe640 1 --2- 172.21.15.84:0/1390305480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c98003970 0x7f7c98003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:18.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.842+0000 7f7cab7fe640 1 --2- 172.21.15.84:0/1390305480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cac159b20 0x7f7c980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:18.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.842+0000 7f7cab7fe640 1 --2- 172.21.15.84:0/1390305480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c98004d00 0x7f7c98005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:18.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.842+0000 7f7cab7fe640 1 -- 172.21.15.84:0/1390305480 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac06c5e0 con 0x7f7c98003970 2023-10-16T11:26:18.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.842+0000 7f7cab7fe640 1 -- 172.21.15.84:0/1390305480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac06c8c0 con 0x7f7c98004d00 2023-10-16T11:26:18.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.842+0000 7f7cab7fe640 1 -- 172.21.15.84:0/1390305480 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9800ba00 con 0x7f7cac159b20 2023-10-16T11:26:18.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.843+0000 7f7cabfff640 1 --2- 172.21.15.84:0/1390305480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c98003970 0x7f7c98003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:18.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.843+0000 7f7cb08d0640 1 --2- 172.21.15.84:0/1390305480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c98004d00 0x7f7c98005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:18.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.843+0000 7f7cabfff640 1 -- 172.21.15.84:0/1390305480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cac159b20 msgr2=0x7f7c980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:18.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.843+0000 7f7cabfff640 1 --2- 172.21.15.84:0/1390305480 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cac159b20 0x7f7c980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.843+0000 7f7cabfff640 1 -- 172.21.15.84:0/1390305480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c98004d00 msgr2=0x7f7c98005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:18.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.843+0000 7f7cabfff640 1 --2- 172.21.15.84:0/1390305480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c98004d00 0x7f7c98005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.843+0000 7f7cabfff640 1 -- 172.21.15.84:0/1390305480 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c98003520 con 0x7f7c98003970 2023-10-16T11:26:18.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.844+0000 7f7cabfff640 1 --2- 172.21.15.84:0/1390305480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c98003970 0x7f7c98003d90 secure :-1 s=READY pgs=3108 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00ba30 tx=0x7f7c980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec24a1f701b52a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:18.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.844+0000 7f7cab7fe640 1 -- 172.21.15.84:0/1390305480 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c032680 con 0x7f7c98003970 2023-10-16T11:26:18.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.844+0000 7f7cab7fe640 1 -- 172.21.15.84:0/1390305480 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c0327e0 con 0x7f7c98003970 2023-10-16T11:26:18.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.844+0000 7f7cab7fe640 1 -- 172.21.15.84:0/1390305480 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c032b70 con 0x7f7c98003970 2023-10-16T11:26:18.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.845+0000 7f7cb2333640 1 -- 172.21.15.84:0/1390305480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c98003970 msgr2=0x7f7c98003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:18.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.845+0000 7f7cb2333640 1 --2- 172.21.15.84:0/1390305480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c98003970 0x7f7c98003d90 secure :-1 s=READY pgs=3108 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00ba30 tx=0x7f7c980057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.845+0000 7f7cb08d0640 1 -- 172.21.15.84:0/1390305480 reap_dead start 2023-10-16T11:26:18.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.845+0000 7f7cb2333640 1 -- 172.21.15.84:0/1390305480 shutdown_connections 2023-10-16T11:26:18.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.845+0000 7f7cb2333640 1 -- 172.21.15.84:0/1390305480 >> 172.21.15.84:0/1390305480 conn(0x7f7cac0cb410 msgr2=0x7f7cac0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:18.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.845+0000 7f7cb2333640 1 -- 172.21.15.84:0/1390305480 shutdown_connections 2023-10-16T11:26:18.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.845+0000 7f7cb2333640 1 -- 172.21.15.84:0/1390305480 wait complete. 2023-10-16T11:26:18.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.847+0000 7f7cb2333640 1 Processor -- start 2023-10-16T11:26:18.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.847+0000 7f7cb2333640 1 -- start start 2023-10-16T11:26:18.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.848+0000 7f7cb2333640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c98003970 0x7f7cac158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:18.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.848+0000 7f7cb2333640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cac0c26d0 0x7f7cac0acef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:18.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.848+0000 7f7cb2333640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cac1f3360 0x7f7cac0b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:18.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.848+0000 7f7cb2333640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac0c2bd0 con 0x7f7c98003970 2023-10-16T11:26:18.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.848+0000 7f7cb2333640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac0c2eb0 con 0x7f7cac0c26d0 2023-10-16T11:26:18.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.848+0000 7f7cb2333640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac0c3190 con 0x7f7cac1f3360 2023-10-16T11:26:18.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.848+0000 7f7cabfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c98003970 0x7f7cac158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:18.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.848+0000 7f7cabfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c98003970 0x7f7cac158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35248/0 (socket says 172.21.15.84:35248) 2023-10-16T11:26:18.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.848+0000 7f7cabfff640 1 -- 172.21.15.84:0/3026480936 learned_addr learned my addr 172.21.15.84:0/3026480936 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:18.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.849+0000 7f7ca3fff640 1 --2- 172.21.15.84:0/3026480936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cac0c26d0 0x7f7cac0acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:18.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.849+0000 7f7cabfff640 1 -- 172.21.15.84:0/3026480936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cac1f3360 msgr2=0x7f7cac0b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:18.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.849+0000 7f7cabfff640 1 --2- 172.21.15.84:0/3026480936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cac1f3360 0x7f7cac0b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.849+0000 7f7cabfff640 1 -- 172.21.15.84:0/3026480936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cac0c26d0 msgr2=0x7f7cac0acef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:18.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.849+0000 7f7cabfff640 1 --2- 172.21.15.84:0/3026480936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cac0c26d0 0x7f7cac0acef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:18.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.849+0000 7f7cabfff640 1 -- 172.21.15.84:0/3026480936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac1f2b60 con 0x7f7c98003970 2023-10-16T11:26:18.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.849+0000 7f7cabfff640 1 --2- 172.21.15.84:0/3026480936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c98003970 0x7f7cac158170 secure :-1 s=READY pgs=3109 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00bb50 tx=0x7f7c9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:18.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.849+0000 7f7ca97fa640 1 -- 172.21.15.84:0/3026480936 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c004220 con 0x7f7c98003970 2023-10-16T11:26:18.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.849+0000 7f7ca97fa640 1 -- 172.21.15.84:0/3026480936 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c004380 con 0x7f7c98003970 2023-10-16T11:26:18.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.849+0000 7f7ca97fa640 1 -- 172.21.15.84:0/3026480936 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c030aa0 con 0x7f7c98003970 2023-10-16T11:26:18.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.849+0000 7f7cb2333640 1 -- 172.21.15.84:0/3026480936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cac1f2e40 con 0x7f7c98003970 2023-10-16T11:26:18.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.850+0000 7f7cb2333640 1 -- 172.21.15.84:0/3026480936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cac0ad430 con 0x7f7c98003970 2023-10-16T11:26:18.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.851+0000 7f7cb2333640 1 -- 172.21.15.84:0/3026480936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cac04e160 con 0x7f7c98003970 2023-10-16T11:26:18.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.852+0000 7f7ca97fa640 1 -- 172.21.15.84:0/3026480936 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f7c9c030c00 con 0x7f7c98003970 2023-10-16T11:26:18.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.853+0000 7f7ca97fa640 1 --2- 172.21.15.84:0/3026480936 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7c8c06b050 0x7f7c8c06d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:18.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.853+0000 7f7ca97fa640 1 -- 172.21.15.84:0/3026480936 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7461+0+0 (secure 0 0 0) 0x7f7c9c0acd30 con 0x7f7c98003970 2023-10-16T11:26:18.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.853+0000 7f7ca3fff640 1 --2- 172.21.15.84:0/3026480936 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7c8c06b050 0x7f7c8c06d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:18.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.854+0000 7f7ca3fff640 1 --2- 172.21.15.84:0/3026480936 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7c8c06b050 0x7f7c8c06d510 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7cac0cdd70 tx=0x7f7c94008040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:18.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:18.857+0000 7f7ca97fa640 1 -- 172.21.15.84:0/3026480936 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7c9c0772d0 con 0x7f7c98003970 2023-10-16T11:26:19.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.099+0000 7f7cb2333640 1 -- 172.21.15.84:0/3026480936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7cac04e160 con 0x7f7c98003970 2023-10-16T11:26:19.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.100+0000 7f7ca97fa640 1 -- 172.21.15.84:0/3026480936 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+6155 (secure 0 0 0) 0x7f7c9c030eb0 con 0x7f7c98003970 2023-10-16T11:26:19.102 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:19.103 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":137,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2516077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2516077743},{"type":"v1","addr":"172.21.15.157:6833","nonce":2516077743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1527104462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1527104462},{"type":"v1","addr":"172.21.15.157:6835","nonce":1527104462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:25:36.605068+0000","modified":"2023-10-16T11:25:39.675208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12186":{"gid":12186,"name":"a","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/3547503033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3547503033},{"type":"v1","addr":"172.21.15.84:6837","nonce":3547503033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:25:50.739234+0000","modified":"2023-10-16T11:25:53.792071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12183":{"gid":12183,"name":"c","rank":0,"incarnation":133,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/369067539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":369067539},{"type":"v1","addr":"172.21.15.84:6835","nonce":369067539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-16T11:26:19.103 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 137 2023-10-16T11:26:19.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.103+0000 7f7cb2333640 1 -- 172.21.15.84:0/3026480936 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7c8c06b050 msgr2=0x7f7c8c06d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:19.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.103+0000 7f7cb2333640 1 --2- 172.21.15.84:0/3026480936 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7c8c06b050 0x7f7c8c06d510 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7cac0cdd70 tx=0x7f7c94008040 comp rx=0 tx=0).stop 2023-10-16T11:26:19.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.103+0000 7f7cb2333640 1 -- 172.21.15.84:0/3026480936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c98003970 msgr2=0x7f7cac158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:19.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.103+0000 7f7cb2333640 1 --2- 172.21.15.84:0/3026480936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c98003970 0x7f7cac158170 secure :-1 s=READY pgs=3109 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00bb50 tx=0x7f7c9c004730 comp rx=0 tx=0).stop 2023-10-16T11:26:19.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.103+0000 7f7cb2333640 1 -- 172.21.15.84:0/3026480936 shutdown_connections 2023-10-16T11:26:19.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.103+0000 7f7cb2333640 1 --2- 172.21.15.84:0/3026480936 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7c8c06b050 0x7f7c8c06d510 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.103+0000 7f7cb2333640 1 --2- 172.21.15.84:0/3026480936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cac1f3360 0x7f7cac0b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.103+0000 7f7cb2333640 1 --2- 172.21.15.84:0/3026480936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cac0c26d0 0x7f7cac0acef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.103+0000 7f7cb2333640 1 --2- 172.21.15.84:0/3026480936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c98003970 0x7f7cac158170 unknown :-1 s=CLOSED pgs=3109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.103+0000 7f7cb2333640 1 -- 172.21.15.84:0/3026480936 >> 172.21.15.84:0/3026480936 conn(0x7f7cac0cb410 msgr2=0x7f7cac0cc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:19.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.103+0000 7f7cb2333640 1 -- 172.21.15.84:0/3026480936 shutdown_connections 2023-10-16T11:26:19.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.103+0000 7f7cb2333640 1 -- 172.21.15.84:0/3026480936 wait complete. 2023-10-16T11:26:19.125 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:26:19.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.341+0000 7f0163528640 1 Processor -- start 2023-10-16T11:26:19.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.341+0000 7f0163528640 1 -- start start 2023-10-16T11:26:19.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.343+0000 7f0163528640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f015c159950 0x7f015c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:19.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.343+0000 7f0163528640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f015c06c810 0x7f015c0708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:19.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.343+0000 7f0163528640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f015c15aef0 con 0x7f015c070450 2023-10-16T11:26:19.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.343+0000 7f0163528640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015c070e20 con 0x7f015c159950 2023-10-16T11:26:19.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.343+0000 7f0163528640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015c15e490 con 0x7f015c06c810 2023-10-16T11:26:19.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.343+0000 7f0162526640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f015c159950 0x7f015c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:19.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.343+0000 7f0162526640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f015c159950 0x7f015c15a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35264/0 (socket says 172.21.15.84:35264) 2023-10-16T11:26:19.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.343+0000 7f0162526640 1 -- 172.21.15.84:0/1639414938 learned_addr learned my addr 172.21.15.84:0/1639414938 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:19.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.343+0000 7f0162d27640 1 --2- 172.21.15.84:0/1639414938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f015c06c810 0x7f015c0708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:19.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.343+0000 7f0162526640 1 -- 172.21.15.84:0/1639414938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f015c06c810 msgr2=0x7f015c0708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:19.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.343+0000 7f0162526640 1 --2- 172.21.15.84:0/1639414938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f015c06c810 0x7f015c0708e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.343+0000 7f0162526640 1 -- 172.21.15.84:0/1639414938 >> v1:172.21.15.84:6789/0 conn(0x7f015c070450 legacy=0x7f015c06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:19.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.344+0000 7f0162526640 1 -- 172.21.15.84:0/1639414938 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c06cc60 con 0x7f015c159950 2023-10-16T11:26:19.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.343+0000 7f0161524640 1 -- 172.21.15.84:0/1639414938 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2613910991 0 0) 0x7f014c001110 con 0x7f015c070450 2023-10-16T11:26:19.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.345+0000 7f0162526640 1 --2- 172.21.15.84:0/1639414938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f015c159950 0x7f015c15a9b0 secure :-1 s=READY pgs=3111 cs=0 l=1 rev1=1 crypto rx=0x7f0158009870 tx=0x7f015802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e3afd44acb9f9e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:19.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.345+0000 7f0161524640 1 -- 172.21.15.84:0/1639414938 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0158031640 con 0x7f015c159950 2023-10-16T11:26:19.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.345+0000 7f0161524640 1 -- 172.21.15.84:0/1639414938 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01580317a0 con 0x7f015c159950 2023-10-16T11:26:19.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.345+0000 7f0161524640 1 -- 172.21.15.84:0/1639414938 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0158031b30 con 0x7f015c159950 2023-10-16T11:26:19.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.345+0000 7f0163528640 1 -- 172.21.15.84:0/1639414938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f015c159950 msgr2=0x7f015c15a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:19.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.345+0000 7f0163528640 1 --2- 172.21.15.84:0/1639414938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f015c159950 0x7f015c15a9b0 secure :-1 s=READY pgs=3111 cs=0 l=1 rev1=1 crypto rx=0x7f0158009870 tx=0x7f015802b8f0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.345+0000 7f0163528640 1 -- 172.21.15.84:0/1639414938 shutdown_connections 2023-10-16T11:26:19.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.345+0000 7f0163528640 1 --2- 172.21.15.84:0/1639414938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f015c06c810 0x7f015c0708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.345+0000 7f0163528640 1 --2- 172.21.15.84:0/1639414938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f015c159950 0x7f015c15a9b0 unknown :-1 s=CLOSED pgs=3111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.345+0000 7f0163528640 1 -- 172.21.15.84:0/1639414938 >> 172.21.15.84:0/1639414938 conn(0x7f015c0ce6a0 msgr2=0x7f015c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:19.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.346+0000 7f0163528640 1 -- 172.21.15.84:0/1639414938 shutdown_connections 2023-10-16T11:26:19.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.346+0000 7f0163528640 1 -- 172.21.15.84:0/1639414938 wait complete. 2023-10-16T11:26:19.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.347+0000 7f0163528640 1 Processor -- start 2023-10-16T11:26:19.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.347+0000 7f0163528640 1 -- start start 2023-10-16T11:26:19.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.348+0000 7f0163528640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f015c06c810 0x7f015c0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:19.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.348+0000 7f0163528640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f015c070450 0x7f015c1f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:19.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.348+0000 7f0163528640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f015c159950 0x7f015c15cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:19.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.348+0000 7f0163528640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c15d2d0 con 0x7f015c159950 2023-10-16T11:26:19.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.348+0000 7f0163528640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c153d40 con 0x7f015c06c810 2023-10-16T11:26:19.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.348+0000 7f0163528640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015c15d4c0 con 0x7f015c070450 2023-10-16T11:26:19.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.349+0000 7f0162d27640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f015c159950 0x7f015c15cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:19.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.349+0000 7f0162d27640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f015c159950 0x7f015c15cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35266/0 (socket says 172.21.15.84:35266) 2023-10-16T11:26:19.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.349+0000 7f0162d27640 1 -- 172.21.15.84:0/2516903368 learned_addr learned my addr 172.21.15.84:0/2516903368 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:19.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.349+0000 7f0162526640 1 --2- 172.21.15.84:0/2516903368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f015c06c810 0x7f015c0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:19.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.349+0000 7f0162d27640 1 -- 172.21.15.84:0/2516903368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f015c070450 msgr2=0x7f015c1f3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:19.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.349+0000 7f0162d27640 1 --2- 172.21.15.84:0/2516903368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f015c070450 0x7f015c1f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.349+0000 7f0162d27640 1 -- 172.21.15.84:0/2516903368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f015c06c810 msgr2=0x7f015c0cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:19.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.349+0000 7f0162d27640 1 --2- 172.21.15.84:0/2516903368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f015c06c810 0x7f015c0cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.349+0000 7f0162d27640 1 -- 172.21.15.84:0/2516903368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01580094d0 con 0x7f015c159950 2023-10-16T11:26:19.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.349+0000 7f0162d27640 1 --2- 172.21.15.84:0/2516903368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f015c159950 0x7f015c15cd90 secure :-1 s=READY pgs=3112 cs=0 l=1 rev1=1 crypto rx=0x7f015400cd70 tx=0x7f015400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:19.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.350+0000 7f0152ffd640 1 -- 172.21.15.84:0/2516903368 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154003b90 con 0x7f015c159950 2023-10-16T11:26:19.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.350+0000 7f0152ffd640 1 -- 172.21.15.84:0/2516903368 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0154003cf0 con 0x7f015c159950 2023-10-16T11:26:19.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.350+0000 7f0152ffd640 1 -- 172.21.15.84:0/2516903368 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154004240 con 0x7f015c159950 2023-10-16T11:26:19.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.350+0000 7f0163528640 1 -- 172.21.15.84:0/2516903368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015c153f50 con 0x7f015c159950 2023-10-16T11:26:19.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.350+0000 7f0163528640 1 -- 172.21.15.84:0/2516903368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015c154510 con 0x7f015c159950 2023-10-16T11:26:19.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.351+0000 7f0163528640 1 -- 172.21.15.84:0/2516903368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015c0cc420 con 0x7f015c159950 2023-10-16T11:26:19.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.352+0000 7f0152ffd640 1 -- 172.21.15.84:0/2516903368 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f01540043a0 con 0x7f015c159950 2023-10-16T11:26:19.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.352+0000 7f0152ffd640 1 --2- 172.21.15.84:0/2516903368 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f014006b020 0x7f014006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:19.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.353+0000 7f0152ffd640 1 -- 172.21.15.84:0/2516903368 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7461+0+0 (secure 0 0 0) 0x7f0154012070 con 0x7f015c159950 2023-10-16T11:26:19.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.353+0000 7f0162526640 1 --2- 172.21.15.84:0/2516903368 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f014006b020 0x7f014006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:19.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.353+0000 7f0162526640 1 --2- 172.21.15.84:0/2516903368 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f014006b020 0x7f014006d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0158036000 tx=0x7f015803c040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:19.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.357+0000 7f0152ffd640 1 -- 172.21.15.84:0/2516903368 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0154055100 con 0x7f015c159950 2023-10-16T11:26:19.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.559+0000 7f0163528640 1 -- 172.21.15.84:0/2516903368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f015c04e9b0 con 0x7f015c159950 2023-10-16T11:26:19.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.563+0000 7f0152ffd640 1 -- 172.21.15.84:0/2516903368 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v243) v1 ==== 74+0+21450 (secure 0 0 0) 0x7f0154058720 con 0x7f015c159950 2023-10-16T11:26:19.565 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:19.566 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":243,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:26:14.475537+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:25:34.443897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:25:35.635615+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:25:49.001107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:25:49.760637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2606713455":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/488945617":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/4036998934":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6832/3938045451":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/570042115":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1311117536":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6833/2963529201":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/95207886":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6832/2963529201":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/388205040":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/2303342316":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/648102622":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/133211355":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1263886961":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1489017743":"2023-10-17T11:26:08.077644+0000","172.21.15.84:6833/3938045451":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1517291402":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1983743360":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1323010124":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/437386817":"2023-10-17T11:26:08.077644+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:26:19.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.565+0000 7f0163528640 1 -- 172.21.15.84:0/2516903368 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f014006b020 msgr2=0x7f014006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:19.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.565+0000 7f0163528640 1 --2- 172.21.15.84:0/2516903368 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f014006b020 0x7f014006d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0158036000 tx=0x7f015803c040 comp rx=0 tx=0).stop 2023-10-16T11:26:19.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.566+0000 7f0163528640 1 -- 172.21.15.84:0/2516903368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f015c159950 msgr2=0x7f015c15cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:19.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.566+0000 7f0163528640 1 --2- 172.21.15.84:0/2516903368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f015c159950 0x7f015c15cd90 secure :-1 s=READY pgs=3112 cs=0 l=1 rev1=1 crypto rx=0x7f015400cd70 tx=0x7f015400a660 comp rx=0 tx=0).stop 2023-10-16T11:26:19.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.566+0000 7f0163528640 1 -- 172.21.15.84:0/2516903368 shutdown_connections 2023-10-16T11:26:19.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.566+0000 7f0163528640 1 --2- 172.21.15.84:0/2516903368 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f014006b020 0x7f014006d4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.566+0000 7f0163528640 1 --2- 172.21.15.84:0/2516903368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f015c159950 0x7f015c15cd90 unknown :-1 s=CLOSED pgs=3112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.566+0000 7f0163528640 1 --2- 172.21.15.84:0/2516903368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f015c070450 0x7f015c1f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.566+0000 7f0163528640 1 --2- 172.21.15.84:0/2516903368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f015c06c810 0x7f015c0cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.566+0000 7f0163528640 1 -- 172.21.15.84:0/2516903368 >> 172.21.15.84:0/2516903368 conn(0x7f015c0ce6a0 msgr2=0x7f015c0aec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:19.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.566+0000 7f0163528640 1 -- 172.21.15.84:0/2516903368 shutdown_connections 2023-10-16T11:26:19.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.566+0000 7f0163528640 1 -- 172.21.15.84:0/2516903368 wait complete. 2023-10-16T11:26:19.587 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T11:26:19.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.789+0000 7fd701ad1640 1 Processor -- start 2023-10-16T11:26:19.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.790+0000 7fd701ad1640 1 -- start start 2023-10-16T11:26:19.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.791+0000 7fd701ad1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6fc0ad970 0x7fd6fc0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:19.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.791+0000 7fd701ad1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd6fc0ccfb0 0x7fd6fc0cd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:19.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.791+0000 7fd701ad1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6fc15b9d0 con 0x7fd6fc06c040 2023-10-16T11:26:19.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.791+0000 7fd701ad1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6fc15bcb0 con 0x7fd6fc0ccfb0 2023-10-16T11:26:19.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.791+0000 7fd701ad1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc15bf90 con 0x7fd6fc0ad970 2023-10-16T11:26:19.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.791+0000 7fd7012d0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd6fc0ccfb0 0x7fd6fc0cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:19.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.791+0000 7fd7012d0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd6fc0ccfb0 0x7fd6fc0cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35282/0 (socket says 172.21.15.84:35282) 2023-10-16T11:26:19.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.791+0000 7fd7012d0640 1 -- 172.21.15.84:0/364521221 learned_addr learned my addr 172.21.15.84:0/364521221 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:19.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.792+0000 7fd6fbfff640 1 --2- 172.21.15.84:0/364521221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6fc0ad970 0x7fd6fc0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:19.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.792+0000 7fd7012d0640 1 -- 172.21.15.84:0/364521221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6fc0ad970 msgr2=0x7fd6fc0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:19.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.792+0000 7fd7012d0640 1 --2- 172.21.15.84:0/364521221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6fc0ad970 0x7fd6fc0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.792+0000 7fd7012d0640 1 -- 172.21.15.84:0/364521221 >> v1:172.21.15.157:6789/0 conn(0x7fd6fc06c040 legacy=0x7fd6fc06c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:19.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.792+0000 7fd7012d0640 1 -- 172.21.15.84:0/364521221 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc0cc270 con 0x7fd6fc0ccfb0 2023-10-16T11:26:19.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.792+0000 7fd6fbfff640 1 --2- 172.21.15.84:0/364521221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6fc0ad970 0x7fd6fc0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:19.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.793+0000 7fd7012d0640 1 --2- 172.21.15.84:0/364521221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc0ccfb0 0x7fd6fc0cd440 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009e80 tx=0x7fd6f402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a0e37448e8d4aa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:19.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.793+0000 7fd6fb7fe640 1 -- 172.21.15.84:0/364521221 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f402ca50 con 0x7fd6fc0ccfb0 2023-10-16T11:26:19.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.793+0000 7fd6fb7fe640 1 -- 172.21.15.84:0/364521221 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f402cbb0 con 0x7fd6fc0ccfb0 2023-10-16T11:26:19.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.793+0000 7fd6fb7fe640 1 -- 172.21.15.84:0/364521221 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40343f0 con 0x7fd6fc0ccfb0 2023-10-16T11:26:19.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.794+0000 7fd701ad1640 1 -- 172.21.15.84:0/364521221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc0ccfb0 msgr2=0x7fd6fc0cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:19.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.794+0000 7fd701ad1640 1 --2- 172.21.15.84:0/364521221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc0ccfb0 0x7fd6fc0cd440 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009e80 tx=0x7fd6f402beb0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.794+0000 7fd701ad1640 1 -- 172.21.15.84:0/364521221 shutdown_connections 2023-10-16T11:26:19.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.794+0000 7fd701ad1640 1 --2- 172.21.15.84:0/364521221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc0ccfb0 0x7fd6fc0cd440 unknown :-1 s=CLOSED pgs=3113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.794+0000 7fd701ad1640 1 --2- 172.21.15.84:0/364521221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6fc0ad970 0x7fd6fc0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.794+0000 7fd701ad1640 1 -- 172.21.15.84:0/364521221 >> 172.21.15.84:0/364521221 conn(0x7fd6fc0ce6a0 msgr2=0x7fd6fc0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:19.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.794+0000 7fd701ad1640 1 -- 172.21.15.84:0/364521221 shutdown_connections 2023-10-16T11:26:19.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.794+0000 7fd701ad1640 1 -- 172.21.15.84:0/364521221 wait complete. 2023-10-16T11:26:19.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.796+0000 7fd701ad1640 1 Processor -- start 2023-10-16T11:26:19.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.796+0000 7fd701ad1640 1 -- start start 2023-10-16T11:26:19.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.797+0000 7fd701ad1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc06c040 0x7fd6fc156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:19.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.797+0000 7fd701ad1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6fc0ad970 0x7fd6fc0c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:19.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.797+0000 7fd701ad1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6fc0ccfb0 0x7fd6fc159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:19.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.797+0000 7fd701ad1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0ae7b0 con 0x7fd6fc06c040 2023-10-16T11:26:19.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.797+0000 7fd701ad1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0ae1f0 con 0x7fd6fc0ad970 2023-10-16T11:26:19.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.797+0000 7fd701ad1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0ae4d0 con 0x7fd6fc0ccfb0 2023-10-16T11:26:19.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.797+0000 7fd700acf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc06c040 0x7fd6fc156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:19.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.797+0000 7fd700acf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc06c040 0x7fd6fc156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35292/0 (socket says 172.21.15.84:35292) 2023-10-16T11:26:19.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.797+0000 7fd700acf640 1 -- 172.21.15.84:0/1771129263 learned_addr learned my addr 172.21.15.84:0/1771129263 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:19.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.798+0000 7fd700acf640 1 -- 172.21.15.84:0/1771129263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6fc0ccfb0 msgr2=0x7fd6fc159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:19.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.798+0000 7fd700acf640 1 --2- 172.21.15.84:0/1771129263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6fc0ccfb0 0x7fd6fc159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.798+0000 7fd700acf640 1 -- 172.21.15.84:0/1771129263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6fc0ad970 msgr2=0x7fd6fc0c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:19.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.798+0000 7fd6fbfff640 1 --2- 172.21.15.84:0/1771129263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6fc0ad970 0x7fd6fc0c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:19.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.798+0000 7fd700acf640 1 --2- 172.21.15.84:0/1771129263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6fc0ad970 0x7fd6fc0c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:19.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.798+0000 7fd700acf640 1 -- 172.21.15.84:0/1771129263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f4009b30 con 0x7fd6fc06c040 2023-10-16T11:26:19.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.798+0000 7fd700acf640 1 --2- 172.21.15.84:0/1771129263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc06c040 0x7fd6fc156c10 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c430 tx=0x7fd6f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:19.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.798+0000 7fd6f97fa640 1 -- 172.21.15.84:0/1771129263 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0011790 con 0x7fd6fc06c040 2023-10-16T11:26:19.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.798+0000 7fd6f97fa640 1 -- 172.21.15.84:0/1771129263 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f00118f0 con 0x7fd6fc06c040 2023-10-16T11:26:19.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.798+0000 7fd6f97fa640 1 -- 172.21.15.84:0/1771129263 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0011ce0 con 0x7fd6fc06c040 2023-10-16T11:26:19.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.798+0000 7fd701ad1640 1 -- 172.21.15.84:0/1771129263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6fc0aecb0 con 0x7fd6fc06c040 2023-10-16T11:26:19.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.798+0000 7fd701ad1640 1 -- 172.21.15.84:0/1771129263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6fc0c8500 con 0x7fd6fc06c040 2023-10-16T11:26:19.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.799+0000 7fd701ad1640 1 -- 172.21.15.84:0/1771129263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6fc06b330 con 0x7fd6fc06c040 2023-10-16T11:26:19.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.800+0000 7fd6f97fa640 1 -- 172.21.15.84:0/1771129263 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fd6f001c460 con 0x7fd6fc06c040 2023-10-16T11:26:19.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.802+0000 7fd6f97fa640 1 --2- 172.21.15.84:0/1771129263 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6e006b020 0x7fd6e006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:19.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.802+0000 7fd6f97fa640 1 -- 172.21.15.84:0/1771129263 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7461+0+0 (secure 0 0 0) 0x7fd6f0089f70 con 0x7fd6fc06c040 2023-10-16T11:26:19.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.802+0000 7fd6fbfff640 1 --2- 172.21.15.84:0/1771129263 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6e006b020 0x7fd6e006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:19.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.803+0000 7fd6fbfff640 1 --2- 172.21.15.84:0/1771129263 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6e006b020 0x7fd6e006d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc157550 tx=0x7fd6e8008040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:19.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:19.807+0000 7fd6f97fa640 1 -- 172.21.15.84:0/1771129263 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd6f00546b0 con 0x7fd6fc06c040 2023-10-16T11:26:19.851 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:19.847+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:20.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.040+0000 7fd701ad1640 1 -- 172.21.15.84:0/1771129263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd6fc04e9b0 con 0x7fd6fc06c040 2023-10-16T11:26:20.043 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-16T11:26:20.040+0000 7fca0c7ff640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:26:20.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.044+0000 7fd6f97fa640 1 -- 172.21.15.84:0/1771129263 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v138) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd6f0057cd0 con 0x7fd6fc06c040 2023-10-16T11:26:20.046 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:26:20.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.047+0000 7fd701ad1640 1 -- 172.21.15.84:0/1771129263 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6e006b020 msgr2=0x7fd6e006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:20.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.047+0000 7fd701ad1640 1 --2- 172.21.15.84:0/1771129263 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6e006b020 0x7fd6e006d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc157550 tx=0x7fd6e8008040 comp rx=0 tx=0).stop 2023-10-16T11:26:20.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.047+0000 7fd701ad1640 1 -- 172.21.15.84:0/1771129263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc06c040 msgr2=0x7fd6fc156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:20.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.047+0000 7fd701ad1640 1 --2- 172.21.15.84:0/1771129263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc06c040 0x7fd6fc156c10 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c430 tx=0x7fd6f000c8f0 comp rx=0 tx=0).stop 2023-10-16T11:26:20.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.048+0000 7fd701ad1640 1 -- 172.21.15.84:0/1771129263 shutdown_connections 2023-10-16T11:26:20.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.048+0000 7fd701ad1640 1 --2- 172.21.15.84:0/1771129263 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6e006b020 0x7fd6e006d4e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:20.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.048+0000 7fd701ad1640 1 --2- 172.21.15.84:0/1771129263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6fc0ccfb0 0x7fd6fc159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:20.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.048+0000 7fd701ad1640 1 --2- 172.21.15.84:0/1771129263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6fc0ad970 0x7fd6fc0c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:20.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.048+0000 7fd701ad1640 1 --2- 172.21.15.84:0/1771129263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6fc06c040 0x7fd6fc156c10 secure :-1 s=CLOSED pgs=3114 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c430 tx=0x7fd6f000c8f0 comp rx=0 tx=0).stop 2023-10-16T11:26:20.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.048+0000 7fd701ad1640 1 -- 172.21.15.84:0/1771129263 >> 172.21.15.84:0/1771129263 conn(0x7fd6fc0ce6a0 msgr2=0x7fd6fc0b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:20.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.048+0000 7fd701ad1640 1 -- 172.21.15.84:0/1771129263 shutdown_connections 2023-10-16T11:26:20.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.048+0000 7fd701ad1640 1 -- 172.21.15.84:0/1771129263 wait complete. 2023-10-16T11:26:20.071 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T11:26:20.136 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T11:26:20.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.262+0000 7f5058e51640 1 Processor -- start 2023-10-16T11:26:20.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.262+0000 7f5058e51640 1 -- start start 2023-10-16T11:26:20.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.263+0000 7f5058e51640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5054070830 0x7f505406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:20.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.263+0000 7f5058e51640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5054161740 con 0x7f50540cf5f0 2023-10-16T11:26:20.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.263+0000 7f5058e51640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505406b0f0 con 0x7f5054070c10 2023-10-16T11:26:20.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.263+0000 7f5058e51640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50541619b0 con 0x7f5054070830 2023-10-16T11:26:20.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.264+0000 7f504bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f50540cf5f0 0x7f505406c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47032/0 (socket says 172.21.15.84:47032) 2023-10-16T11:26:20.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.264+0000 7f504bfff640 1 -- 172.21.15.84:0/1050780587 learned_addr learned my addr 172.21.15.84:0/1050780587 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:20.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.264+0000 7f5051d9b640 1 -- 172.21.15.84:0/1050780587 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125194554 0 0) 0x7f5054161740 con 0x7f50540cf5f0 2023-10-16T11:26:20.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.264+0000 7f5051d9b640 1 -- 172.21.15.84:0/1050780587 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5040003520 con 0x7f50540cf5f0 2023-10-16T11:26:20.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.264+0000 7f505259c640 1 --2- 172.21.15.84:0/1050780587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5054070830 0x7f505406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:20.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.264+0000 7f5051d9b640 1 -- 172.21.15.84:0/1050780587 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 108576583 0 0) 0x7f5040003520 con 0x7f50540cf5f0 2023-10-16T11:26:20.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.264+0000 7f5051d9b640 1 -- 172.21.15.84:0/1050780587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5054070830 msgr2=0x7f505406d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:20.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.264+0000 7f5051d9b640 1 --2- 172.21.15.84:0/1050780587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5054070830 0x7f505406d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:20.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.264+0000 7f5051d9b640 1 -- 172.21.15.84:0/1050780587 >> v1:172.21.15.157:6789/0 conn(0x7f5054070c10 legacy=0x7f50540cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:20.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.264+0000 7f5051d9b640 1 -- 172.21.15.84:0/1050780587 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505406b3d0 con 0x7f50540cf5f0 2023-10-16T11:26:20.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.264+0000 7f5051d9b640 1 -- 172.21.15.84:0/1050780587 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f503c001670 con 0x7f50540cf5f0 2023-10-16T11:26:20.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.264+0000 7f5051d9b640 1 -- 172.21.15.84:0/1050780587 >> v1:172.21.15.84:6789/0 conn(0x7f50540cf5f0 legacy=0x7f505406c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:20.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.264+0000 7f5051d9b640 1 --2- 172.21.15.84:0/1050780587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50400039a0 0x7f5040003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:20.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.265+0000 7f5051d9b640 1 --2- 172.21.15.84:0/1050780587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5054070830 0x7f50400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:20.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.265+0000 7f5051d9b640 1 --2- 172.21.15.84:0/1050780587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5040004d50 0x7f50400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:20.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.265+0000 7f5051d9b640 1 -- 172.21.15.84:0/1050780587 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50541619b0 con 0x7f50400039a0 2023-10-16T11:26:20.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.265+0000 7f5051d9b640 1 -- 172.21.15.84:0/1050780587 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504000bbe0 con 0x7f5040004d50 2023-10-16T11:26:20.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.265+0000 7f5051d9b640 1 -- 172.21.15.84:0/1050780587 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504000bda0 con 0x7f5054070830 2023-10-16T11:26:20.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.265+0000 7f505259c640 1 --2- 172.21.15.84:0/1050780587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50400039a0 0x7f5040003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:20.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.265+0000 7f504bfff640 1 --2- 172.21.15.84:0/1050780587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5054070830 0x7f50400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:20.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.265+0000 7f505259c640 1 -- 172.21.15.84:0/1050780587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5054070830 msgr2=0x7f50400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:20.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.265+0000 7f505259c640 1 --2- 172.21.15.84:0/1050780587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5054070830 0x7f50400047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:20.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.265+0000 7f5052d9d640 1 --2- 172.21.15.84:0/1050780587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5040004d50 0x7f50400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:20.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.265+0000 7f505259c640 1 -- 172.21.15.84:0/1050780587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5040004d50 msgr2=0x7f50400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:20.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.265+0000 7f505259c640 1 --2- 172.21.15.84:0/1050780587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5040004d50 0x7f50400051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:20.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.265+0000 7f505259c640 1 -- 172.21.15.84:0/1050780587 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5040003520 con 0x7f50400039a0 2023-10-16T11:26:20.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.266+0000 7f505259c640 1 --2- 172.21.15.84:0/1050780587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50400039a0 0x7f5040003da0 secure :-1 s=READY pgs=3119 cs=0 l=1 rev1=1 crypto rx=0x7f504400ba00 tx=0x7f50400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba62ebc67f418741 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:20.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.267+0000 7f5051d9b640 1 -- 172.21.15.84:0/1050780587 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044032620 con 0x7f50400039a0 2023-10-16T11:26:20.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.267+0000 7f5051d9b640 1 -- 172.21.15.84:0/1050780587 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5044032780 con 0x7f50400039a0 2023-10-16T11:26:20.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.267+0000 7f5051d9b640 1 -- 172.21.15.84:0/1050780587 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044032b10 con 0x7f50400039a0 2023-10-16T11:26:20.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.267+0000 7f5058e51640 1 -- 172.21.15.84:0/1050780587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50400039a0 msgr2=0x7f5040003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:20.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.267+0000 7f5058e51640 1 --2- 172.21.15.84:0/1050780587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50400039a0 0x7f5040003da0 secure :-1 s=READY pgs=3119 cs=0 l=1 rev1=1 crypto rx=0x7f504400ba00 tx=0x7f50400057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:20.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.267+0000 7f5052d9d640 1 -- 172.21.15.84:0/1050780587 reap_dead start 2023-10-16T11:26:20.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.267+0000 7f5058e51640 1 -- 172.21.15.84:0/1050780587 shutdown_connections 2023-10-16T11:26:20.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.267+0000 7f5058e51640 1 -- 172.21.15.84:0/1050780587 >> 172.21.15.84:0/1050780587 conn(0x7f50540cfac0 msgr2=0x7f50540cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:20.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.267+0000 7f5058e51640 1 -- 172.21.15.84:0/1050780587 shutdown_connections 2023-10-16T11:26:20.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.268+0000 7f5058e51640 1 -- 172.21.15.84:0/1050780587 wait complete. 2023-10-16T11:26:20.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.269+0000 7f5058e51640 1 Processor -- start 2023-10-16T11:26:20.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.269+0000 7f5058e51640 1 -- start start 2023-10-16T11:26:20.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.270+0000 7f5058e51640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50400039a0 0x7f50541f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:20.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.270+0000 7f5058e51640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f505415d9c0 0x7f50540c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:20.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.270+0000 7f5058e51640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f505415de40 0x7f50540b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:20.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.270+0000 7f5058e51640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50540c6f30 con 0x7f505415de40 2023-10-16T11:26:20.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.270+0000 7f5058e51640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50540c7210 con 0x7f50400039a0 2023-10-16T11:26:20.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.270+0000 7f5058e51640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50541f2eb0 con 0x7f505415d9c0 2023-10-16T11:26:20.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.271+0000 7f5052d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f505415de40 0x7f50540b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:20.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.271+0000 7f5052d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f505415de40 0x7f50540b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35340/0 (socket says 172.21.15.84:35340) 2023-10-16T11:26:20.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.271+0000 7f5052d9d640 1 -- 172.21.15.84:0/3093166558 learned_addr learned my addr 172.21.15.84:0/3093166558 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:20.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.271+0000 7f505259c640 1 --2- 172.21.15.84:0/3093166558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50400039a0 0x7f50541f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:20.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.271+0000 7f504bfff640 1 --2- 172.21.15.84:0/3093166558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f505415d9c0 0x7f50540c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:20.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.271+0000 7f5052d9d640 1 -- 172.21.15.84:0/3093166558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f505415d9c0 msgr2=0x7f50540c4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:20.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.271+0000 7f5052d9d640 1 --2- 172.21.15.84:0/3093166558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f505415d9c0 0x7f50540c4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:20.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.271+0000 7f5052d9d640 1 -- 172.21.15.84:0/3093166558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50400039a0 msgr2=0x7f50541f36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:20.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.271+0000 7f5052d9d640 1 --2- 172.21.15.84:0/3093166558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50400039a0 0x7f50541f36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:20.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.271+0000 7f5052d9d640 1 -- 172.21.15.84:0/3093166558 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50541f3190 con 0x7f505415de40 2023-10-16T11:26:20.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.271+0000 7f5052d9d640 1 --2- 172.21.15.84:0/3093166558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f505415de40 0x7f50540b47d0 secure :-1 s=READY pgs=3120 cs=0 l=1 rev1=1 crypto rx=0x7f50540d0720 tx=0x7f504c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:20.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.272+0000 7f504affd640 1 -- 172.21.15.84:0/3093166558 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c004af0 con 0x7f505415de40 2023-10-16T11:26:20.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.272+0000 7f504affd640 1 -- 172.21.15.84:0/3093166558 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504c004c50 con 0x7f505415de40 2023-10-16T11:26:20.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.272+0000 7f504affd640 1 -- 172.21.15.84:0/3093166558 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c005030 con 0x7f505415de40 2023-10-16T11:26:20.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.272+0000 7f5058e51640 1 -- 172.21.15.84:0/3093166558 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50541f2b60 con 0x7f505415de40 2023-10-16T11:26:20.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.272+0000 7f5058e51640 1 -- 172.21.15.84:0/3093166558 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50540c5490 con 0x7f505415de40 2023-10-16T11:26:20.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.273+0000 7f5058e51640 1 -- 172.21.15.84:0/3093166558 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50541f3380 con 0x7f505415de40 2023-10-16T11:26:20.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.274+0000 7f504affd640 1 -- 172.21.15.84:0/3093166558 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f504c005190 con 0x7f505415de40 2023-10-16T11:26:20.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.275+0000 7f504affd640 1 --2- 172.21.15.84:0/3093166558 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f503406b020 0x7f503406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:20.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.275+0000 7f504affd640 1 -- 172.21.15.84:0/3093166558 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 7547+0+0 (secure 0 0 0) 0x7f504c089fe0 con 0x7f505415de40 2023-10-16T11:26:20.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.275+0000 7f505259c640 1 --2- 172.21.15.84:0/3093166558 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f503406b020 0x7f503406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:20.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.276+0000 7f505259c640 1 --2- 172.21.15.84:0/3093166558 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f503406b020 0x7f503406d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5040004740 tx=0x7f5044030000 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:20.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.279+0000 7f504affd640 1 -- 172.21.15.84:0/3093166558 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f504c0545c0 con 0x7f505415de40 2023-10-16T11:26:20.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:20.501+0000 7f5058e51640 1 -- 172.21.15.84:0/3093166558 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f505404e160 con 0x7f505415de40 2023-10-16T11:26:20.998 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:20.995+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:21.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.048+0000 7f504affd640 1 -- 172.21.15.84:0/3093166558 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v139) v1 ==== 104+0+0 (secure 0 0 0) 0x7f504c057be0 con 0x7f505415de40 2023-10-16T11:26:21.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.051+0000 7f5058e51640 1 -- 172.21.15.84:0/3093166558 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f503406b020 msgr2=0x7f503406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:21.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.051+0000 7f5058e51640 1 --2- 172.21.15.84:0/3093166558 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f503406b020 0x7f503406d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5040004740 tx=0x7f5044030000 comp rx=0 tx=0).stop 2023-10-16T11:26:21.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.051+0000 7f5058e51640 1 -- 172.21.15.84:0/3093166558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f505415de40 msgr2=0x7f50540b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:21.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.051+0000 7f5058e51640 1 --2- 172.21.15.84:0/3093166558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f505415de40 0x7f50540b47d0 secure :-1 s=READY pgs=3120 cs=0 l=1 rev1=1 crypto rx=0x7f50540d0720 tx=0x7f504c00dc00 comp rx=0 tx=0).stop 2023-10-16T11:26:21.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.051+0000 7f5058e51640 1 -- 172.21.15.84:0/3093166558 shutdown_connections 2023-10-16T11:26:21.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.051+0000 7f5058e51640 1 --2- 172.21.15.84:0/3093166558 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f503406b020 0x7f503406d4e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:21.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.051+0000 7f5058e51640 1 --2- 172.21.15.84:0/3093166558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f505415de40 0x7f50540b47d0 unknown :-1 s=CLOSED pgs=3120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:21.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.051+0000 7f5058e51640 1 --2- 172.21.15.84:0/3093166558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f505415d9c0 0x7f50540c4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:21.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.051+0000 7f5058e51640 1 --2- 172.21.15.84:0/3093166558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50400039a0 0x7f50541f36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:21.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.052+0000 7f5058e51640 1 -- 172.21.15.84:0/3093166558 >> 172.21.15.84:0/3093166558 conn(0x7f50540cfac0 msgr2=0x7f50540afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:21.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.052+0000 7f5058e51640 1 -- 172.21.15.84:0/3093166558 shutdown_connections 2023-10-16T11:26:21.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.052+0000 7f5058e51640 1 -- 172.21.15.84:0/3093166558 wait complete. 2023-10-16T11:26:21.075 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T11:26:21.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.278+0000 7fe582459640 1 Processor -- start 2023-10-16T11:26:21.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.278+0000 7fe582459640 1 -- start start 2023-10-16T11:26:21.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.280+0000 7fe582459640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe57c1443f0 0x7fe57c06e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:21.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.280+0000 7fe582459640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57c070ff0 con 0x7fe57c0d04e0 2023-10-16T11:26:21.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.280+0000 7fe582459640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57c0703e0 con 0x7fe57c0ca670 2023-10-16T11:26:21.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.280+0000 7fe582459640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe57c0706c0 con 0x7fe57c1443f0 2023-10-16T11:26:21.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.280+0000 7fe581457640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe57c1443f0 0x7fe57c06e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:21.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.280+0000 7fe581457640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe57c1443f0 0x7fe57c06e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35344/0 (socket says 172.21.15.84:35344) 2023-10-16T11:26:21.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.280+0000 7fe581457640 1 -- 172.21.15.84:0/540496229 learned_addr learned my addr 172.21.15.84:0/540496229 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:21.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.280+0000 7fe581457640 1 -- 172.21.15.84:0/540496229 >> v1:172.21.15.157:6789/0 conn(0x7fe57c0ca670 legacy=0x7fe57c0c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:21.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.280+0000 7fe581457640 1 -- 172.21.15.84:0/540496229 >> v1:172.21.15.84:6789/0 conn(0x7fe57c0d04e0 legacy=0x7fe57c0c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:21.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.280+0000 7fe581457640 1 -- 172.21.15.84:0/540496229 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c0709a0 con 0x7fe57c1443f0 2023-10-16T11:26:21.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.280+0000 7fe573fff640 1 -- 172.21.15.84:0/540496229 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317840577 0 0) 0x7fe57c070ff0 con 0x7fe57c0d04e0 2023-10-16T11:26:21.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.281+0000 7fe581457640 1 --2- 172.21.15.84:0/540496229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe57c1443f0 0x7fe57c06e660 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 crypto rx=0x7fe578009870 tx=0x7fe57802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53d92dfc5f789ca server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:21.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.282+0000 7fe573fff640 1 -- 172.21.15.84:0/540496229 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578030860 con 0x7fe57c1443f0 2023-10-16T11:26:21.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.282+0000 7fe573fff640 1 -- 172.21.15.84:0/540496229 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5780309c0 con 0x7fe57c1443f0 2023-10-16T11:26:21.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.282+0000 7fe573fff640 1 -- 172.21.15.84:0/540496229 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578030d50 con 0x7fe57c1443f0 2023-10-16T11:26:21.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.282+0000 7fe582459640 1 -- 172.21.15.84:0/540496229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe57c1443f0 msgr2=0x7fe57c06e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:21.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.282+0000 7fe582459640 1 --2- 172.21.15.84:0/540496229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe57c1443f0 0x7fe57c06e660 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 crypto rx=0x7fe578009870 tx=0x7fe57802b9a0 comp rx=0 tx=0).stop 2023-10-16T11:26:21.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.282+0000 7fe582459640 1 -- 172.21.15.84:0/540496229 shutdown_connections 2023-10-16T11:26:21.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.282+0000 7fe582459640 1 --2- 172.21.15.84:0/540496229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe57c1443f0 0x7fe57c06e660 unknown :-1 s=CLOSED pgs=3122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:21.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.282+0000 7fe582459640 1 -- 172.21.15.84:0/540496229 >> 172.21.15.84:0/540496229 conn(0x7fe57c0cf910 msgr2=0x7fe57c0cfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:21.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.283+0000 7fe582459640 1 -- 172.21.15.84:0/540496229 shutdown_connections 2023-10-16T11:26:21.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.283+0000 7fe582459640 1 -- 172.21.15.84:0/540496229 wait complete. 2023-10-16T11:26:21.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.284+0000 7fe582459640 1 Processor -- start 2023-10-16T11:26:21.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.284+0000 7fe582459640 1 -- start start 2023-10-16T11:26:21.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.285+0000 7fe582459640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe57c0ca670 0x7fe57c15ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:21.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.285+0000 7fe582459640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe57c0d04e0 0x7fe57c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:21.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.285+0000 7fe582459640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe57c1443f0 0x7fe57c15b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:21.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.285+0000 7fe582459640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c0c9450 con 0x7fe57c0d04e0 2023-10-16T11:26:21.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.285+0000 7fe582459640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c0c9730 con 0x7fe57c0ca670 2023-10-16T11:26:21.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.285+0000 7fe582459640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c0c69e0 con 0x7fe57c1443f0 2023-10-16T11:26:21.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.285+0000 7fe580c56640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe57c0d04e0 0x7fe57c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:21.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.285+0000 7fe580c56640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe57c0d04e0 0x7fe57c0b4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35352/0 (socket says 172.21.15.84:35352) 2023-10-16T11:26:21.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.285+0000 7fe580c56640 1 -- 172.21.15.84:0/3704653096 learned_addr learned my addr 172.21.15.84:0/3704653096 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:21.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.286+0000 7fe580c56640 1 -- 172.21.15.84:0/3704653096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe57c1443f0 msgr2=0x7fe57c15b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:21.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.286+0000 7fe580c56640 1 --2- 172.21.15.84:0/3704653096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe57c1443f0 0x7fe57c15b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:21.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.286+0000 7fe580c56640 1 -- 172.21.15.84:0/3704653096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe57c0ca670 msgr2=0x7fe57c15ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:21.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.286+0000 7fe580c56640 1 --2- 172.21.15.84:0/3704653096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe57c0ca670 0x7fe57c15ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:21.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.286+0000 7fe580c56640 1 -- 172.21.15.84:0/3704653096 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5780094d0 con 0x7fe57c0d04e0 2023-10-16T11:26:21.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.286+0000 7fe580c56640 1 --2- 172.21.15.84:0/3704653096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe57c0d04e0 0x7fe57c0b4e10 secure :-1 s=READY pgs=3123 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00c520 tx=0x7fe56c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:21.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.286+0000 7fe571ffb640 1 -- 172.21.15.84:0/3704653096 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c0035f0 con 0x7fe57c0d04e0 2023-10-16T11:26:21.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.286+0000 7fe571ffb640 1 -- 172.21.15.84:0/3704653096 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56c003770 con 0x7fe57c0d04e0 2023-10-16T11:26:21.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.286+0000 7fe571ffb640 1 -- 172.21.15.84:0/3704653096 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c003b20 con 0x7fe57c0d04e0 2023-10-16T11:26:21.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.286+0000 7fe582459640 1 -- 172.21.15.84:0/3704653096 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe57c0c6cc0 con 0x7fe57c0d04e0 2023-10-16T11:26:21.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.286+0000 7fe582459640 1 -- 172.21.15.84:0/3704653096 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe57c0b5350 con 0x7fe57c0d04e0 2023-10-16T11:26:21.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.287+0000 7fe582459640 1 -- 172.21.15.84:0/3704653096 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57c0c71e0 con 0x7fe57c0d04e0 2023-10-16T11:26:21.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.288+0000 7fe571ffb640 1 -- 172.21.15.84:0/3704653096 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fe56c003c80 con 0x7fe57c0d04e0 2023-10-16T11:26:21.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.289+0000 7fe571ffb640 1 --2- 172.21.15.84:0/3704653096 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe56006b020 0x7fe56006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:21.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.290+0000 7fe571ffb640 1 -- 172.21.15.84:0/3704653096 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 7547+0+0 (secure 0 0 0) 0x7fe56c089710 con 0x7fe57c0d04e0 2023-10-16T11:26:21.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.290+0000 7fe581457640 1 --2- 172.21.15.84:0/3704653096 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe56006b020 0x7fe56006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:21.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.290+0000 7fe581457640 1 --2- 172.21.15.84:0/3704653096 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe56006b020 0x7fe56006d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe578036000 tx=0x7fe57803b040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:21.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.293+0000 7fe571ffb640 1 -- 172.21.15.84:0/3704653096 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe57c0c71e0 con 0x7fe57c0d04e0 2023-10-16T11:26:21.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:21.496+0000 7fe582459640 1 -- 172.21.15.84:0/3704653096 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe57c0c71e0 con 0x7fe57c0d04e0 2023-10-16T11:26:22.141 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:22.138+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:22.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.223+0000 7fe571ffb640 1 -- 172.21.15.84:0/3704653096 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v245) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe57c0c71e0 con 0x7fe57c0d04e0 2023-10-16T11:26:22.226 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-16T11:26:22.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.228+0000 7fe582459640 1 -- 172.21.15.84:0/3704653096 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe56006b020 msgr2=0x7fe56006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:22.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.228+0000 7fe582459640 1 --2- 172.21.15.84:0/3704653096 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe56006b020 0x7fe56006d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe578036000 tx=0x7fe57803b040 comp rx=0 tx=0).stop 2023-10-16T11:26:22.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.228+0000 7fe582459640 1 -- 172.21.15.84:0/3704653096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe57c0d04e0 msgr2=0x7fe57c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:22.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.228+0000 7fe582459640 1 --2- 172.21.15.84:0/3704653096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe57c0d04e0 0x7fe57c0b4e10 secure :-1 s=READY pgs=3123 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00c520 tx=0x7fe56c00c9e0 comp rx=0 tx=0).stop 2023-10-16T11:26:22.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.229+0000 7fe582459640 1 -- 172.21.15.84:0/3704653096 shutdown_connections 2023-10-16T11:26:22.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.229+0000 7fe582459640 1 --2- 172.21.15.84:0/3704653096 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe56006b020 0x7fe56006d4e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:22.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.229+0000 7fe582459640 1 --2- 172.21.15.84:0/3704653096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe57c1443f0 0x7fe57c15b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:22.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.229+0000 7fe582459640 1 --2- 172.21.15.84:0/3704653096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe57c0d04e0 0x7fe57c0b4e10 unknown :-1 s=CLOSED pgs=3123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:22.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.229+0000 7fe582459640 1 --2- 172.21.15.84:0/3704653096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe57c0ca670 0x7fe57c15ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:22.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.229+0000 7fe582459640 1 -- 172.21.15.84:0/3704653096 >> 172.21.15.84:0/3704653096 conn(0x7fe57c0cf910 msgr2=0x7fe57c1435b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:22.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.231+0000 7fe582459640 1 -- 172.21.15.84:0/3704653096 shutdown_connections 2023-10-16T11:26:22.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.231+0000 7fe582459640 1 -- 172.21.15.84:0/3704653096 wait complete. 2023-10-16T11:26:22.259 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T11:26:22.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.463+0000 7ffb4359e640 1 Processor -- start 2023-10-16T11:26:22.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.463+0000 7ffb4359e640 1 -- start start 2023-10-16T11:26:22.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.464+0000 7ffb4359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb440ca0a0 0x7ffb4406c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:22.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.464+0000 7ffb4359e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4415d910 con 0x7ffb44159950 2023-10-16T11:26:22.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.464+0000 7ffb4359e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4415dbf0 con 0x7ffb44070b60 2023-10-16T11:26:22.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.464+0000 7ffb4359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4415e490 con 0x7ffb440ca0a0 2023-10-16T11:26:22.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.465+0000 7ffb4259c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ffb44159950 0x7ffb44070450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47040/0 (socket says 172.21.15.84:47040) 2023-10-16T11:26:22.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.465+0000 7ffb4259c640 1 -- 172.21.15.84:0/2107000064 learned_addr learned my addr 172.21.15.84:0/2107000064 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:22.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.465+0000 7ffb42d9d640 1 --2- 172.21.15.84:0/2107000064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb440ca0a0 0x7ffb4406c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:22.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.465+0000 7ffb41d9b640 1 -- 172.21.15.84:0/2107000064 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232331289 0 0) 0x7ffb4415d910 con 0x7ffb44159950 2023-10-16T11:26:22.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.465+0000 7ffb41d9b640 1 -- 172.21.15.84:0/2107000064 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb30003520 con 0x7ffb44159950 2023-10-16T11:26:22.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.465+0000 7ffb41d9b640 1 -- 172.21.15.84:0/2107000064 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1378236496 0 0) 0x7ffb30003520 con 0x7ffb44159950 2023-10-16T11:26:22.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.465+0000 7ffb41d9b640 1 -- 172.21.15.84:0/2107000064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb440ca0a0 msgr2=0x7ffb4406c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:22.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.465+0000 7ffb41d9b640 1 --2- 172.21.15.84:0/2107000064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb440ca0a0 0x7ffb4406c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:22.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.465+0000 7ffb41d9b640 1 -- 172.21.15.84:0/2107000064 >> v1:172.21.15.157:6789/0 conn(0x7ffb44070b60 legacy=0x7ffb4406c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:22.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.465+0000 7ffb41d9b640 1 -- 172.21.15.84:0/2107000064 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4415ded0 con 0x7ffb44159950 2023-10-16T11:26:22.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.466+0000 7ffb41d9b640 1 -- 172.21.15.84:0/2107000064 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ffb34001670 con 0x7ffb44159950 2023-10-16T11:26:22.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.466+0000 7ffb41d9b640 1 -- 172.21.15.84:0/2107000064 >> v1:172.21.15.84:6789/0 conn(0x7ffb44159950 legacy=0x7ffb44070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:22.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.466+0000 7ffb41d9b640 1 --2- 172.21.15.84:0/2107000064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb300039a0 0x7ffb30003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:22.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.466+0000 7ffb41d9b640 1 --2- 172.21.15.84:0/2107000064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb300047c0 0x7ffb30004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:22.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.466+0000 7ffb41d9b640 1 --2- 172.21.15.84:0/2107000064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb440ca0a0 0x7ffb300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:22.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.466+0000 7ffb41d9b640 1 -- 172.21.15.84:0/2107000064 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4415e490 con 0x7ffb300039a0 2023-10-16T11:26:22.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.466+0000 7ffb41d9b640 1 -- 172.21.15.84:0/2107000064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3000dcc0 con 0x7ffb300047c0 2023-10-16T11:26:22.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.466+0000 7ffb41d9b640 1 -- 172.21.15.84:0/2107000064 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3000de80 con 0x7ffb440ca0a0 2023-10-16T11:26:22.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.466+0000 7ffb4259c640 1 --2- 172.21.15.84:0/2107000064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb300039a0 0x7ffb30003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:22.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.466+0000 7ffb4259c640 1 -- 172.21.15.84:0/2107000064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb440ca0a0 msgr2=0x7ffb300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:22.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.466+0000 7ffb4259c640 1 --2- 172.21.15.84:0/2107000064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb440ca0a0 0x7ffb300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:22.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.466+0000 7ffb4259c640 1 -- 172.21.15.84:0/2107000064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb300047c0 msgr2=0x7ffb30004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:22.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.466+0000 7ffb3bfff640 1 --2- 172.21.15.84:0/2107000064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb300047c0 0x7ffb30004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:22.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.466+0000 7ffb4259c640 1 --2- 172.21.15.84:0/2107000064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb300047c0 0x7ffb30004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:22.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.467+0000 7ffb4259c640 1 -- 172.21.15.84:0/2107000064 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb30003520 con 0x7ffb300039a0 2023-10-16T11:26:22.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.468+0000 7ffb4259c640 1 --2- 172.21.15.84:0/2107000064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb300039a0 0x7ffb30003dc0 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 crypto rx=0x7ffb3400c240 tx=0x7ffb300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a2b9fb92386bf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:22.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.468+0000 7ffb41d9b640 1 -- 172.21.15.84:0/2107000064 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb34005ad0 con 0x7ffb300039a0 2023-10-16T11:26:22.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.468+0000 7ffb41d9b640 1 -- 172.21.15.84:0/2107000064 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb34005c30 con 0x7ffb300039a0 2023-10-16T11:26:22.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.468+0000 7ffb41d9b640 1 -- 172.21.15.84:0/2107000064 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3402c8b0 con 0x7ffb300039a0 2023-10-16T11:26:22.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.469+0000 7ffb4359e640 1 -- 172.21.15.84:0/2107000064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb300039a0 msgr2=0x7ffb30003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:22.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.469+0000 7ffb4359e640 1 --2- 172.21.15.84:0/2107000064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb300039a0 0x7ffb30003dc0 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 crypto rx=0x7ffb3400c240 tx=0x7ffb300057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:22.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.469+0000 7ffb42d9d640 1 -- 172.21.15.84:0/2107000064 reap_dead start 2023-10-16T11:26:22.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.469+0000 7ffb4359e640 1 -- 172.21.15.84:0/2107000064 shutdown_connections 2023-10-16T11:26:22.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.469+0000 7ffb4359e640 1 -- 172.21.15.84:0/2107000064 >> 172.21.15.84:0/2107000064 conn(0x7ffb440ce6a0 msgr2=0x7ffb440ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:22.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.469+0000 7ffb4359e640 1 -- 172.21.15.84:0/2107000064 shutdown_connections 2023-10-16T11:26:22.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.469+0000 7ffb4359e640 1 -- 172.21.15.84:0/2107000064 wait complete. 2023-10-16T11:26:22.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.471+0000 7ffb4359e640 1 Processor -- start 2023-10-16T11:26:22.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.471+0000 7ffb4359e640 1 -- start start 2023-10-16T11:26:22.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.472+0000 7ffb4359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb300039a0 0x7ffb44153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:22.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.472+0000 7ffb4359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb440c4b00 0x7ffb4406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:22.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.472+0000 7ffb4359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb44155e30 0x7ffb4406bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:22.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.472+0000 7ffb4359e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb44154e20 con 0x7ffb440c4b00 2023-10-16T11:26:22.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.472+0000 7ffb4359e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb44154160 con 0x7ffb300039a0 2023-10-16T11:26:22.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.472+0000 7ffb4359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb44155070 con 0x7ffb44155e30 2023-10-16T11:26:22.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.472+0000 7ffb3bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb440c4b00 0x7ffb4406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:22.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.472+0000 7ffb3bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb440c4b00 0x7ffb4406bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35362/0 (socket says 172.21.15.84:35362) 2023-10-16T11:26:22.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.472+0000 7ffb3bfff640 1 -- 172.21.15.84:0/1559592885 learned_addr learned my addr 172.21.15.84:0/1559592885 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:22.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.472+0000 7ffb3bfff640 1 -- 172.21.15.84:0/1559592885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb44155e30 msgr2=0x7ffb4406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:22.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.472+0000 7ffb42d9d640 1 --2- 172.21.15.84:0/1559592885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb44155e30 0x7ffb4406bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:22.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.472+0000 7ffb4259c640 1 --2- 172.21.15.84:0/1559592885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb300039a0 0x7ffb44153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:22.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.472+0000 7ffb3bfff640 1 --2- 172.21.15.84:0/1559592885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb44155e30 0x7ffb4406bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:22.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.472+0000 7ffb3bfff640 1 -- 172.21.15.84:0/1559592885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb300039a0 msgr2=0x7ffb44153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:22.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.472+0000 7ffb3bfff640 1 --2- 172.21.15.84:0/1559592885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb300039a0 0x7ffb44153c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:22.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.472+0000 7ffb3bfff640 1 -- 172.21.15.84:0/1559592885 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb44155350 con 0x7ffb440c4b00 2023-10-16T11:26:22.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.473+0000 7ffb3bfff640 1 --2- 172.21.15.84:0/1559592885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb440c4b00 0x7ffb4406bab0 secure :-1 s=READY pgs=3126 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00d4c0 tx=0x7ffb2c00d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:22.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.473+0000 7ffb3affd640 1 -- 172.21.15.84:0/1559592885 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c005480 con 0x7ffb440c4b00 2023-10-16T11:26:22.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.473+0000 7ffb3affd640 1 -- 172.21.15.84:0/1559592885 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c0035f0 con 0x7ffb440c4b00 2023-10-16T11:26:22.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.473+0000 7ffb4359e640 1 -- 172.21.15.84:0/1559592885 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb44155630 con 0x7ffb440c4b00 2023-10-16T11:26:22.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.473+0000 7ffb3affd640 1 -- 172.21.15.84:0/1559592885 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c0039e0 con 0x7ffb440c4b00 2023-10-16T11:26:22.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.473+0000 7ffb4359e640 1 -- 172.21.15.84:0/1559592885 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb44154af0 con 0x7ffb440c4b00 2023-10-16T11:26:22.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.474+0000 7ffb4359e640 1 -- 172.21.15.84:0/1559592885 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4404e9b0 con 0x7ffb440c4b00 2023-10-16T11:26:22.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.475+0000 7ffb3affd640 1 -- 172.21.15.84:0/1559592885 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7ffb2c003b40 con 0x7ffb440c4b00 2023-10-16T11:26:22.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.476+0000 7ffb3affd640 1 --2- 172.21.15.84:0/1559592885 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7ffb2406b050 0x7ffb2406d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:22.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.476+0000 7ffb3affd640 1 -- 172.21.15.84:0/1559592885 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 7109+0+0 (secure 0 0 0) 0x7ffb2c089e20 con 0x7ffb440c4b00 2023-10-16T11:26:22.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.477+0000 7ffb4259c640 1 --2- 172.21.15.84:0/1559592885 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7ffb2406b050 0x7ffb2406d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:22.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.477+0000 7ffb4259c640 1 --2- 172.21.15.84:0/1559592885 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7ffb2406b050 0x7ffb2406d510 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffb34034000 tx=0x7ffb34009030 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:22.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.481+0000 7ffb3affd640 1 -- 172.21.15.84:0/1559592885 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffb2c054650 con 0x7ffb440c4b00 2023-10-16T11:26:22.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:22.681+0000 7ffb4359e640 1 -- 172.21.15.84:0/1559592885 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffb4415d540 con 0x7ffb440c4b00 2023-10-16T11:26:23.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.250+0000 7ffb3affd640 1 -- 172.21.15.84:0/1559592885 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v246) v1 ==== 169+0+0 (secure 0 0 0) 0x7ffb2c057c70 con 0x7ffb440c4b00 2023-10-16T11:26:23.252 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-16T11:26:23.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.254+0000 7ffb4359e640 1 -- 172.21.15.84:0/1559592885 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7ffb2406b050 msgr2=0x7ffb2406d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:23.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.254+0000 7ffb4359e640 1 --2- 172.21.15.84:0/1559592885 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7ffb2406b050 0x7ffb2406d510 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffb34034000 tx=0x7ffb34009030 comp rx=0 tx=0).stop 2023-10-16T11:26:23.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.254+0000 7ffb4359e640 1 -- 172.21.15.84:0/1559592885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb440c4b00 msgr2=0x7ffb4406bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:23.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.254+0000 7ffb4359e640 1 --2- 172.21.15.84:0/1559592885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb440c4b00 0x7ffb4406bab0 secure :-1 s=READY pgs=3126 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00d4c0 tx=0x7ffb2c00d980 comp rx=0 tx=0).stop 2023-10-16T11:26:23.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.254+0000 7ffb4359e640 1 -- 172.21.15.84:0/1559592885 shutdown_connections 2023-10-16T11:26:23.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.254+0000 7ffb4359e640 1 --2- 172.21.15.84:0/1559592885 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7ffb2406b050 0x7ffb2406d510 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:23.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.254+0000 7ffb4359e640 1 --2- 172.21.15.84:0/1559592885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb44155e30 0x7ffb4406bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:23.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.254+0000 7ffb4359e640 1 --2- 172.21.15.84:0/1559592885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb440c4b00 0x7ffb4406bab0 unknown :-1 s=CLOSED pgs=3126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:23.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.254+0000 7ffb4359e640 1 --2- 172.21.15.84:0/1559592885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb300039a0 0x7ffb44153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:23.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.254+0000 7ffb4359e640 1 -- 172.21.15.84:0/1559592885 >> 172.21.15.84:0/1559592885 conn(0x7ffb440ce6a0 msgr2=0x7ffb440ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:23.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.254+0000 7ffb4359e640 1 -- 172.21.15.84:0/1559592885 shutdown_connections 2023-10-16T11:26:23.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.254+0000 7ffb4359e640 1 -- 172.21.15.84:0/1559592885 wait complete. 2023-10-16T11:26:23.284 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:26:23.317 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:23.314+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:23.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.492+0000 7f3fa4ae7640 1 Processor -- start 2023-10-16T11:26:23.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.492+0000 7f3fa4ae7640 1 -- start start 2023-10-16T11:26:23.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.493+0000 7f3fa4ae7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3fa015aaa0 0x7f3fa015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.494+0000 7f3fa4ae7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fa015ae80 0x7f3fa00cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.494+0000 7f3fa4ae7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa015ed60 con 0x7f3fa00ccaa0 2023-10-16T11:26:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.494+0000 7f3fa4ae7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fa006b810 con 0x7f3fa015aaa0 2023-10-16T11:26:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.494+0000 7f3fa4ae7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa00707e0 con 0x7f3fa015ae80 2023-10-16T11:26:23.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.494+0000 7f3f97fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fa015ae80 0x7f3fa00cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:23.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.494+0000 7f3f9e59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3fa015aaa0 0x7f3fa015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:23.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.495+0000 7f3f97fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fa015ae80 0x7f3fa00cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:56384/0 (socket says 172.21.15.84:56384) 2023-10-16T11:26:23.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.495+0000 7f3f9e59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3fa015aaa0 0x7f3fa015e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:46940/0 (socket says 172.21.15.84:46940) 2023-10-16T11:26:23.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.495+0000 7f3f97fff640 1 -- 172.21.15.84:0/1773026348 learned_addr learned my addr 172.21.15.84:0/1773026348 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:23.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.495+0000 7f3f9e59c640 1 -- 172.21.15.84:0/1773026348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fa015ae80 msgr2=0x7f3fa00cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:23.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.495+0000 7f3f9e59c640 1 --2- 172.21.15.84:0/1773026348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fa015ae80 0x7f3fa00cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:23.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.495+0000 7f3f9e59c640 1 -- 172.21.15.84:0/1773026348 >> v1:172.21.15.157:6789/0 conn(0x7f3fa00ccaa0 legacy=0x7f3fa0070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:23.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.495+0000 7f3f9e59c640 1 -- 172.21.15.84:0/1773026348 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa015ef90 con 0x7f3fa015aaa0 2023-10-16T11:26:23.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.495+0000 7f3f97fff640 1 --2- 172.21.15.84:0/1773026348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fa015ae80 0x7f3fa00cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:26:23.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.497+0000 7f3f9e59c640 1 --2- 172.21.15.84:0/1773026348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fa015aaa0 0x7f3fa015e820 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3f900097a0 tx=0x7f3f9002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66544de7383d6afb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:23.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.498+0000 7f3f9dd9b640 1 -- 172.21.15.84:0/1773026348 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90031830 con 0x7f3fa015aaa0 2023-10-16T11:26:23.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.498+0000 7f3f9dd9b640 1 -- 172.21.15.84:0/1773026348 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f90031990 con 0x7f3fa015aaa0 2023-10-16T11:26:23.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.498+0000 7f3f9dd9b640 1 -- 172.21.15.84:0/1773026348 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90031d20 con 0x7f3fa015aaa0 2023-10-16T11:26:23.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.498+0000 7f3fa4ae7640 1 -- 172.21.15.84:0/1773026348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fa015aaa0 msgr2=0x7f3fa015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:23.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.498+0000 7f3fa4ae7640 1 --2- 172.21.15.84:0/1773026348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fa015aaa0 0x7f3fa015e820 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3f900097a0 tx=0x7f3f9002b8d0 comp rx=0 tx=0).stop 2023-10-16T11:26:23.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.499+0000 7f3fa4ae7640 1 -- 172.21.15.84:0/1773026348 shutdown_connections 2023-10-16T11:26:23.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.499+0000 7f3fa4ae7640 1 --2- 172.21.15.84:0/1773026348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fa015ae80 0x7f3fa00cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:23.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.499+0000 7f3fa4ae7640 1 --2- 172.21.15.84:0/1773026348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fa015aaa0 0x7f3fa015e820 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:23.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.499+0000 7f3fa4ae7640 1 -- 172.21.15.84:0/1773026348 >> 172.21.15.84:0/1773026348 conn(0x7f3fa00cb410 msgr2=0x7f3fa00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:23.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.499+0000 7f3fa4ae7640 1 -- 172.21.15.84:0/1773026348 shutdown_connections 2023-10-16T11:26:23.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.499+0000 7f3fa4ae7640 1 -- 172.21.15.84:0/1773026348 wait complete. 2023-10-16T11:26:23.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.501+0000 7f3fa4ae7640 1 Processor -- start 2023-10-16T11:26:23.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.501+0000 7f3fa4ae7640 1 -- start start 2023-10-16T11:26:23.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.502+0000 7f3fa4ae7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fa00ccaa0 0x7f3fa01567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:23.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.502+0000 7f3fa4ae7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fa015aaa0 0x7f3fa0156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:23.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.502+0000 7f3fa4ae7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fa015ae80 0x7f3fa01484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:23.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.502+0000 7f3fa4ae7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa00c4ea0 con 0x7f3fa015aaa0 2023-10-16T11:26:23.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.502+0000 7f3fa4ae7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa015c0d0 con 0x7f3fa015ae80 2023-10-16T11:26:23.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.502+0000 7f3fa4ae7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa00c48e0 con 0x7f3fa00ccaa0 2023-10-16T11:26:23.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.503+0000 7f3f97fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fa015aaa0 0x7f3fa0156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:23.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.503+0000 7f3f97fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fa015aaa0 0x7f3fa0156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35378/0 (socket says 172.21.15.84:35378) 2023-10-16T11:26:23.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.503+0000 7f3f97fff640 1 -- 172.21.15.84:0/2925621339 learned_addr learned my addr 172.21.15.84:0/2925621339 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:23.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.503+0000 7f3f9e59c640 1 --2- 172.21.15.84:0/2925621339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fa00ccaa0 0x7f3fa01567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:23.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.503+0000 7f3f9ed9d640 1 --2- 172.21.15.84:0/2925621339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fa015ae80 0x7f3fa01484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:23.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.503+0000 7f3f97fff640 1 -- 172.21.15.84:0/2925621339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fa00ccaa0 msgr2=0x7f3fa01567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:23.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.503+0000 7f3f97fff640 1 --2- 172.21.15.84:0/2925621339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fa00ccaa0 0x7f3fa01567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:23.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.503+0000 7f3f97fff640 1 -- 172.21.15.84:0/2925621339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fa015ae80 msgr2=0x7f3fa01484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:23.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.503+0000 7f3f97fff640 1 --2- 172.21.15.84:0/2925621339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fa015ae80 0x7f3fa01484e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:23.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.503+0000 7f3f97fff640 1 -- 172.21.15.84:0/2925621339 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f880094d0 con 0x7f3fa015aaa0 2023-10-16T11:26:23.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.504+0000 7f3f97fff640 1 --2- 172.21.15.84:0/2925621339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fa015aaa0 0x7f3fa0156d20 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800e9d0 tx=0x7f3f8800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:23.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.504+0000 7f3f96ffd640 1 -- 172.21.15.84:0/2925621339 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8800ca60 con 0x7f3fa015aaa0 2023-10-16T11:26:23.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.504+0000 7f3f96ffd640 1 -- 172.21.15.84:0/2925621339 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8800cbc0 con 0x7f3fa015aaa0 2023-10-16T11:26:23.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.504+0000 7f3f96ffd640 1 -- 172.21.15.84:0/2925621339 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f88013470 con 0x7f3fa015aaa0 2023-10-16T11:26:23.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.504+0000 7f3fa4ae7640 1 -- 172.21.15.84:0/2925621339 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f90009400 con 0x7f3fa015aaa0 2023-10-16T11:26:23.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.504+0000 7f3fa4ae7640 1 -- 172.21.15.84:0/2925621339 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa015bd60 con 0x7f3fa015aaa0 2023-10-16T11:26:23.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.505+0000 7f3fa4ae7640 1 -- 172.21.15.84:0/2925621339 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa015bbc0 con 0x7f3fa015aaa0 2023-10-16T11:26:23.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.506+0000 7f3f96ffd640 1 -- 172.21.15.84:0/2925621339 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f3f880135d0 con 0x7f3fa015aaa0 2023-10-16T11:26:23.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.507+0000 7f3f96ffd640 1 --2- 172.21.15.84:0/2925621339 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f3f8006b050 0x7f3f8006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:23.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.507+0000 7f3f96ffd640 1 -- 172.21.15.84:0/2925621339 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6695+0+0 (secure 0 0 0) 0x7f3f8808a080 con 0x7f3fa015aaa0 2023-10-16T11:26:23.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.507+0000 7f3f9e59c640 1 --2- 172.21.15.84:0/2925621339 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f3f8006b050 0x7f3f8006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:23.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.508+0000 7f3f9e59c640 1 --2- 172.21.15.84:0/2925621339 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f3f8006b050 0x7f3f8006d510 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3f90036000 tx=0x7f3f9003c040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:23.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.511+0000 7f3f96ffd640 1 -- 172.21.15.84:0/2925621339 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3f880bcfe0 con 0x7f3fa015aaa0 2023-10-16T11:26:23.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.727+0000 7f3fa4ae7640 1 -- 172.21.15.84:0/2925621339 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fa004e160 con 0x7f3fa015aaa0 2023-10-16T11:26:23.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.730+0000 7f3f96ffd640 1 -- 172.21.15.84:0/2925621339 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v246) v1 ==== 74+0+17928 (secure 0 0 0) 0x7f3f88017090 con 0x7f3fa015aaa0 2023-10-16T11:26:23.732 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:23.733 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":246,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:26:23.239524+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:25:49.001107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:25:49.760637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6837/3547503033":"2023-10-17T11:26:20.041719+0000","172.21.15.84:0/2606713455":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/488945617":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/4036998934":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6832/3938045451":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/570042115":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1311117536":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6833/2963529201":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/95207886":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6832/2963529201":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/388205040":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/2303342316":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/648102622":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/133211355":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1263886961":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1489017743":"2023-10-17T11:26:08.077644+0000","172.21.15.84:6833/3938045451":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1517291402":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1983743360":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1323010124":"2023-10-17T11:26:08.077644+0000","172.21.15.84:6836/3547503033":"2023-10-17T11:26:20.041719+0000","172.21.15.84:0/437386817":"2023-10-17T11:26:08.077644+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:26:23.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.733+0000 7f3fa4ae7640 1 -- 172.21.15.84:0/2925621339 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f3f8006b050 msgr2=0x7f3f8006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:23.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.733+0000 7f3fa4ae7640 1 --2- 172.21.15.84:0/2925621339 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f3f8006b050 0x7f3f8006d510 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3f90036000 tx=0x7f3f9003c040 comp rx=0 tx=0).stop 2023-10-16T11:26:23.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.733+0000 7f3fa4ae7640 1 -- 172.21.15.84:0/2925621339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fa015aaa0 msgr2=0x7f3fa0156d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:23.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.733+0000 7f3fa4ae7640 1 --2- 172.21.15.84:0/2925621339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fa015aaa0 0x7f3fa0156d20 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800e9d0 tx=0x7f3f8800ee90 comp rx=0 tx=0).stop 2023-10-16T11:26:23.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.733+0000 7f3fa4ae7640 1 -- 172.21.15.84:0/2925621339 shutdown_connections 2023-10-16T11:26:23.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.733+0000 7f3fa4ae7640 1 --2- 172.21.15.84:0/2925621339 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f3f8006b050 0x7f3f8006d510 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:23.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.733+0000 7f3fa4ae7640 1 --2- 172.21.15.84:0/2925621339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fa015ae80 0x7f3fa01484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:23.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.733+0000 7f3fa4ae7640 1 --2- 172.21.15.84:0/2925621339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fa015aaa0 0x7f3fa0156d20 unknown :-1 s=CLOSED pgs=3127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:23.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.733+0000 7f3fa4ae7640 1 --2- 172.21.15.84:0/2925621339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3fa00ccaa0 0x7f3fa01567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:23.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.733+0000 7f3fa4ae7640 1 -- 172.21.15.84:0/2925621339 >> 172.21.15.84:0/2925621339 conn(0x7f3fa00cb410 msgr2=0x7f3fa00b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:23.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.733+0000 7f3fa4ae7640 1 -- 172.21.15.84:0/2925621339 shutdown_connections 2023-10-16T11:26:23.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:23.733+0000 7f3fa4ae7640 1 -- 172.21.15.84:0/2925621339 wait complete. 2023-10-16T11:26:23.754 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-16T11:25:49.001107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '240', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-16T11:25:49.760637+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '241', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:26:23.755 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:26:24.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.009+0000 7f1c44fff640 1 Processor -- start 2023-10-16T11:26:24.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.010+0000 7f1c44fff640 1 -- start start 2023-10-16T11:26:24.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.011+0000 7f1c44fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1c40070f30 0x7f1c4006aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.011+0000 7f1c44fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1c4006b3e0 0x7f1c400c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.011+0000 7f1c44fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c400cd260 con 0x7f1c40159ad0 2023-10-16T11:26:24.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.011+0000 7f1c44fff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4015de30 con 0x7f1c40070f30 2023-10-16T11:26:24.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.011+0000 7f1c44fff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4015e110 con 0x7f1c4006b3e0 2023-10-16T11:26:24.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.011+0000 7f1c3f7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1c40159ad0 0x7f1c4015d650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47050/0 (socket says 172.21.15.84:47050) 2023-10-16T11:26:24.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.011+0000 7f1c3f7fe640 1 -- 172.21.15.84:0/3383018100 learned_addr learned my addr 172.21.15.84:0/3383018100 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:24.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.011+0000 7f1c3effd640 1 --2- 172.21.15.84:0/3383018100 >> v2:172.21.15.84:3300/0 conn(0x7f1c40070f30 0x7f1c4006aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:24.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.012+0000 7f1c3ffff640 1 --2- 172.21.15.84:0/3383018100 >> v2:172.21.15.157:3300/0 conn(0x7f1c4006b3e0 0x7f1c400c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:24.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.012+0000 7f1c3e7fc640 1 -- 172.21.15.84:0/3383018100 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729137970 0 0) 0x7f1c400cd260 con 0x7f1c40159ad0 2023-10-16T11:26:24.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.012+0000 7f1c3e7fc640 1 -- 172.21.15.84:0/3383018100 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c28005730 con 0x7f1c40159ad0 2023-10-16T11:26:24.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.012+0000 7f1c3effd640 1 -- 172.21.15.84:0/3383018100 >> v2:172.21.15.157:3300/0 conn(0x7f1c4006b3e0 msgr2=0x7f1c400c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:24.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.012+0000 7f1c3effd640 1 --2- 172.21.15.84:0/3383018100 >> v2:172.21.15.157:3300/0 conn(0x7f1c4006b3e0 0x7f1c400c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.012+0000 7f1c3effd640 1 -- 172.21.15.84:0/3383018100 >> v1:172.21.15.84:6789/0 conn(0x7f1c40159ad0 legacy=0x7f1c4015d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:24.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.012+0000 7f1c3effd640 1 -- 172.21.15.84:0/3383018100 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4015e3f0 con 0x7f1c40070f30 2023-10-16T11:26:24.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.013+0000 7f1c3effd640 1 --2- 172.21.15.84:0/3383018100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c40070f30 0x7f1c4006aea0 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 crypto rx=0x7f1c4006d720 tx=0x7f1c3402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb2870984b85b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:24.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.014+0000 7f1c3e7fc640 1 -- 172.21.15.84:0/3383018100 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c34031830 con 0x7f1c40070f30 2023-10-16T11:26:24.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.014+0000 7f1c3e7fc640 1 -- 172.21.15.84:0/3383018100 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c34031990 con 0x7f1c40070f30 2023-10-16T11:26:24.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.014+0000 7f1c3e7fc640 1 -- 172.21.15.84:0/3383018100 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c34031d00 con 0x7f1c40070f30 2023-10-16T11:26:24.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.014+0000 7f1c44fff640 1 -- 172.21.15.84:0/3383018100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c40070f30 msgr2=0x7f1c4006aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:24.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.014+0000 7f1c44fff640 1 --2- 172.21.15.84:0/3383018100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c40070f30 0x7f1c4006aea0 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 crypto rx=0x7f1c4006d720 tx=0x7f1c3402b8d0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.014+0000 7f1c44fff640 1 -- 172.21.15.84:0/3383018100 shutdown_connections 2023-10-16T11:26:24.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.014+0000 7f1c44fff640 1 --2- 172.21.15.84:0/3383018100 >> v2:172.21.15.157:3300/0 conn(0x7f1c4006b3e0 0x7f1c400c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.014+0000 7f1c44fff640 1 --2- 172.21.15.84:0/3383018100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c40070f30 0x7f1c4006aea0 unknown :-1 s=CLOSED pgs=3129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.014+0000 7f1c44fff640 1 -- 172.21.15.84:0/3383018100 >> 172.21.15.84:0/3383018100 conn(0x7f1c400b0170 msgr2=0x7f1c400b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:24.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.015+0000 7f1c44fff640 1 -- 172.21.15.84:0/3383018100 shutdown_connections 2023-10-16T11:26:24.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.015+0000 7f1c44fff640 1 -- 172.21.15.84:0/3383018100 wait complete. 2023-10-16T11:26:24.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.016+0000 7f1c44fff640 1 Processor -- start 2023-10-16T11:26:24.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.016+0000 7f1c44fff640 1 -- start start 2023-10-16T11:26:24.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.017+0000 7f1c44fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c4006b3e0 0x7f1c400c5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.017+0000 7f1c44fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c40070f30 0x7f1c400c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.017+0000 7f1c44fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c40159ad0 0x7f1c4014a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.017+0000 7f1c44fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400c60c0 con 0x7f1c40070f30 2023-10-16T11:26:24.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.017+0000 7f1c44fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4015a8c0 con 0x7f1c4006b3e0 2023-10-16T11:26:24.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.017+0000 7f1c44fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400c8650 con 0x7f1c40159ad0 2023-10-16T11:26:24.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.018+0000 7f1c3effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c40070f30 0x7f1c400c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:24.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.018+0000 7f1c3effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c40070f30 0x7f1c400c8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35400/0 (socket says 172.21.15.84:35400) 2023-10-16T11:26:24.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.018+0000 7f1c3effd640 1 -- 172.21.15.84:0/3270606800 learned_addr learned my addr 172.21.15.84:0/3270606800 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:24.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.018+0000 7f1c3effd640 1 -- 172.21.15.84:0/3270606800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c40159ad0 msgr2=0x7f1c4014a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:24.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.018+0000 7f1c3effd640 1 --2- 172.21.15.84:0/3270606800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c40159ad0 0x7f1c4014a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.018+0000 7f1c3effd640 1 -- 172.21.15.84:0/3270606800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c4006b3e0 msgr2=0x7f1c400c5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:24.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.018+0000 7f1c3effd640 1 --2- 172.21.15.84:0/3270606800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c4006b3e0 0x7f1c400c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.018+0000 7f1c3effd640 1 -- 172.21.15.84:0/3270606800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c34009400 con 0x7f1c40070f30 2023-10-16T11:26:24.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.018+0000 7f1c3effd640 1 --2- 172.21.15.84:0/3270606800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c40070f30 0x7f1c400c8110 secure :-1 s=READY pgs=3130 cs=0 l=1 rev1=1 crypto rx=0x7f1c34000c00 tx=0x7f1c340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:24.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.019+0000 7f1c27fff640 1 -- 172.21.15.84:0/3270606800 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3403a630 con 0x7f1c40070f30 2023-10-16T11:26:24.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.019+0000 7f1c27fff640 1 -- 172.21.15.84:0/3270606800 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3403a7b0 con 0x7f1c40070f30 2023-10-16T11:26:24.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.019+0000 7f1c27fff640 1 -- 172.21.15.84:0/3270606800 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3403ab60 con 0x7f1c40070f30 2023-10-16T11:26:24.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.019+0000 7f1c44fff640 1 -- 172.21.15.84:0/3270606800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c400c5de0 con 0x7f1c40070f30 2023-10-16T11:26:24.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.019+0000 7f1c44fff640 1 -- 172.21.15.84:0/3270606800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4015a550 con 0x7f1c40070f30 2023-10-16T11:26:24.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.020+0000 7f1c44fff640 1 -- 172.21.15.84:0/3270606800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4015a710 con 0x7f1c40070f30 2023-10-16T11:26:24.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.021+0000 7f1c27fff640 1 -- 172.21.15.84:0/3270606800 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f1c3403acc0 con 0x7f1c40070f30 2023-10-16T11:26:24.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.022+0000 7f1c27fff640 1 --2- 172.21.15.84:0/3270606800 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f1c2c06b020 0x7f1c2c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.022+0000 7f1c27fff640 1 -- 172.21.15.84:0/3270606800 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6695+0+0 (secure 0 0 0) 0x7f1c340abc60 con 0x7f1c40070f30 2023-10-16T11:26:24.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.022+0000 7f1c3f7fe640 1 --2- 172.21.15.84:0/3270606800 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f1c2c06b020 0x7f1c2c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:24.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.023+0000 7f1c3f7fe640 1 --2- 172.21.15.84:0/3270606800 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f1c2c06b020 0x7f1c2c06d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1c30000f60 tx=0x7f1c300078f0 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:24.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.026+0000 7f1c27fff640 1 -- 172.21.15.84:0/3270606800 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1c34076590 con 0x7f1c40070f30 2023-10-16T11:26:24.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.226+0000 7f1c44fff640 1 -- 172.21.15.84:0/3270606800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c4004e9b0 con 0x7f1c40070f30 2023-10-16T11:26:24.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.227+0000 7f1c27fff640 1 -- 172.21.15.84:0/3270606800 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v246) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c34048020 con 0x7f1c40070f30 2023-10-16T11:26:24.229 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:26:24.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.229+0000 7f1c44fff640 1 -- 172.21.15.84:0/3270606800 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f1c2c06b020 msgr2=0x7f1c2c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:24.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.230+0000 7f1c44fff640 1 --2- 172.21.15.84:0/3270606800 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f1c2c06b020 0x7f1c2c06d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1c30000f60 tx=0x7f1c300078f0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.230+0000 7f1c44fff640 1 -- 172.21.15.84:0/3270606800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c40070f30 msgr2=0x7f1c400c8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:24.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.230+0000 7f1c44fff640 1 --2- 172.21.15.84:0/3270606800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c40070f30 0x7f1c400c8110 secure :-1 s=READY pgs=3130 cs=0 l=1 rev1=1 crypto rx=0x7f1c34000c00 tx=0x7f1c340046c0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.230+0000 7f1c44fff640 1 -- 172.21.15.84:0/3270606800 shutdown_connections 2023-10-16T11:26:24.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.230+0000 7f1c44fff640 1 --2- 172.21.15.84:0/3270606800 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f1c2c06b020 0x7f1c2c06d4e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.230+0000 7f1c44fff640 1 --2- 172.21.15.84:0/3270606800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c40159ad0 0x7f1c4014a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.230+0000 7f1c44fff640 1 --2- 172.21.15.84:0/3270606800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c40070f30 0x7f1c400c8110 unknown :-1 s=CLOSED pgs=3130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.230+0000 7f1c44fff640 1 --2- 172.21.15.84:0/3270606800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c4006b3e0 0x7f1c400c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.230+0000 7f1c44fff640 1 -- 172.21.15.84:0/3270606800 >> 172.21.15.84:0/3270606800 conn(0x7f1c400b0170 msgr2=0x7f1c4006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:24.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.230+0000 7f1c44fff640 1 -- 172.21.15.84:0/3270606800 shutdown_connections 2023-10-16T11:26:24.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.230+0000 7f1c44fff640 1 -- 172.21.15.84:0/3270606800 wait complete. 2023-10-16T11:26:24.251 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-16T11:26:24.449 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:24.445+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:24.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.475+0000 7f9c4a70a640 1 Processor -- start 2023-10-16T11:26:24.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.475+0000 7f9c4a70a640 1 -- start start 2023-10-16T11:26:24.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.476+0000 7f9c4a70a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9c4406c040 0x7f9c4406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.477+0000 7f9c4a70a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c440ad970 0x7f9c4415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.477+0000 7f9c4a70a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4406c960 con 0x7f9c4415ebc0 2023-10-16T11:26:24.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.477+0000 7f9c4a70a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4415b9d0 con 0x7f9c4406c040 2023-10-16T11:26:24.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.477+0000 7f9c4a70a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4415bcb0 con 0x7f9c440ad970 2023-10-16T11:26:24.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.477+0000 7f9c49708640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9c4406c040 0x7f9c4406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:24.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.477+0000 7f9c49708640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9c4406c040 0x7f9c4406c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35402/0 (socket says 172.21.15.84:35402) 2023-10-16T11:26:24.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.477+0000 7f9c49708640 1 -- 172.21.15.84:0/1610838746 learned_addr learned my addr 172.21.15.84:0/1610838746 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:24.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.477+0000 7f9c49708640 1 -- 172.21.15.84:0/1610838746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c440ad970 msgr2=0x7f9c4415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:24.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.477+0000 7f9c49708640 1 --2- 172.21.15.84:0/1610838746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c440ad970 0x7f9c4415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.477+0000 7f9c49708640 1 -- 172.21.15.84:0/1610838746 >> v1:172.21.15.157:6789/0 conn(0x7f9c4415ebc0 legacy=0x7f9c440cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:24.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.477+0000 7f9c49708640 1 -- 172.21.15.84:0/1610838746 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4415bf90 con 0x7f9c4406c040 2023-10-16T11:26:24.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.478+0000 7f9c49708640 1 --2- 172.21.15.84:0/1610838746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4406c040 0x7f9c4406c420 secure :-1 s=READY pgs=3131 cs=0 l=1 rev1=1 crypto rx=0x7f9c400098a0 tx=0x7f9c4002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c970120894fde6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:24.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.479+0000 7f9c3bfff640 1 -- 172.21.15.84:0/1610838746 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40031640 con 0x7f9c4406c040 2023-10-16T11:26:24.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.479+0000 7f9c3bfff640 1 -- 172.21.15.84:0/1610838746 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c400317a0 con 0x7f9c4406c040 2023-10-16T11:26:24.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.479+0000 7f9c3bfff640 1 -- 172.21.15.84:0/1610838746 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40031b30 con 0x7f9c4406c040 2023-10-16T11:26:24.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.479+0000 7f9c4a70a640 1 -- 172.21.15.84:0/1610838746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4406c040 msgr2=0x7f9c4406c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:24.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.479+0000 7f9c4a70a640 1 --2- 172.21.15.84:0/1610838746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4406c040 0x7f9c4406c420 secure :-1 s=READY pgs=3131 cs=0 l=1 rev1=1 crypto rx=0x7f9c400098a0 tx=0x7f9c4002b8f0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.479+0000 7f9c4a70a640 1 -- 172.21.15.84:0/1610838746 shutdown_connections 2023-10-16T11:26:24.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.479+0000 7f9c4a70a640 1 --2- 172.21.15.84:0/1610838746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c440ad970 0x7f9c4415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.479+0000 7f9c4a70a640 1 --2- 172.21.15.84:0/1610838746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4406c040 0x7f9c4406c420 unknown :-1 s=CLOSED pgs=3131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.479+0000 7f9c4a70a640 1 -- 172.21.15.84:0/1610838746 >> 172.21.15.84:0/1610838746 conn(0x7f9c440ce6a0 msgr2=0x7f9c440ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:24.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.480+0000 7f9c4a70a640 1 -- 172.21.15.84:0/1610838746 shutdown_connections 2023-10-16T11:26:24.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.480+0000 7f9c4a70a640 1 -- 172.21.15.84:0/1610838746 wait complete. 2023-10-16T11:26:24.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.481+0000 7f9c4a70a640 1 Processor -- start 2023-10-16T11:26:24.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.482+0000 7f9c4a70a640 1 -- start start 2023-10-16T11:26:24.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.482+0000 7f9c4a70a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4406c040 0x7f9c440c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.482+0000 7f9c4a70a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c440ad970 0x7f9c440c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.482+0000 7f9c4a70a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c4415ebc0 0x7f9c440c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.482+0000 7f9c4a70a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c44156d20 con 0x7f9c4406c040 2023-10-16T11:26:24.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.482+0000 7f9c4a70a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c440c5cb0 con 0x7f9c440ad970 2023-10-16T11:26:24.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.482+0000 7f9c4a70a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c440c5f50 con 0x7f9c4415ebc0 2023-10-16T11:26:24.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.483+0000 7f9c49708640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4406c040 0x7f9c440c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:24.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.483+0000 7f9c49708640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4406c040 0x7f9c440c5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35410/0 (socket says 172.21.15.84:35410) 2023-10-16T11:26:24.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.483+0000 7f9c49708640 1 -- 172.21.15.84:0/1496643353 learned_addr learned my addr 172.21.15.84:0/1496643353 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:24.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.483+0000 7f9c49708640 1 -- 172.21.15.84:0/1496643353 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c4415ebc0 msgr2=0x7f9c440c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:24.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.483+0000 7f9c49f09640 1 --2- 172.21.15.84:0/1496643353 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c4415ebc0 0x7f9c440c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:24.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.483+0000 7f9c49708640 1 --2- 172.21.15.84:0/1496643353 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c4415ebc0 0x7f9c440c7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.483+0000 7f9c49708640 1 -- 172.21.15.84:0/1496643353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c440ad970 msgr2=0x7f9c440c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:24.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.483+0000 7f9c49708640 1 --2- 172.21.15.84:0/1496643353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c440ad970 0x7f9c440c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.483+0000 7f9c49708640 1 -- 172.21.15.84:0/1496643353 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c400094d0 con 0x7f9c4406c040 2023-10-16T11:26:24.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.483+0000 7f9c49708640 1 --2- 172.21.15.84:0/1496643353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4406c040 0x7f9c440c5770 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 crypto rx=0x7f9c400098a0 tx=0x7f9c400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:24.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.484+0000 7f9c39ffb640 1 -- 172.21.15.84:0/1496643353 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4003ac60 con 0x7f9c4406c040 2023-10-16T11:26:24.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.484+0000 7f9c39ffb640 1 -- 172.21.15.84:0/1496643353 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4003adc0 con 0x7f9c4406c040 2023-10-16T11:26:24.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.484+0000 7f9c39ffb640 1 -- 172.21.15.84:0/1496643353 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40044610 con 0x7f9c4406c040 2023-10-16T11:26:24.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.484+0000 7f9c4a70a640 1 -- 172.21.15.84:0/1496643353 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c440c6230 con 0x7f9c4406c040 2023-10-16T11:26:24.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.484+0000 7f9c4a70a640 1 -- 172.21.15.84:0/1496643353 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c440c67f0 con 0x7f9c4406c040 2023-10-16T11:26:24.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.485+0000 7f9c4a70a640 1 -- 172.21.15.84:0/1496643353 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c4406b330 con 0x7f9c4406c040 2023-10-16T11:26:24.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.486+0000 7f9c39ffb640 1 -- 172.21.15.84:0/1496643353 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f9c4002f050 con 0x7f9c4406c040 2023-10-16T11:26:24.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.487+0000 7f9c39ffb640 1 --2- 172.21.15.84:0/1496643353 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f9c2806b020 0x7f9c2806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.487+0000 7f9c39ffb640 1 -- 172.21.15.84:0/1496643353 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6695+0+0 (secure 0 0 0) 0x7f9c400abde0 con 0x7f9c4406c040 2023-10-16T11:26:24.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.487+0000 7f9c48f07640 1 --2- 172.21.15.84:0/1496643353 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f9c2806b020 0x7f9c2806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:24.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.488+0000 7f9c48f07640 1 --2- 172.21.15.84:0/1496643353 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f9c2806b020 0x7f9c2806d4e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9c44157560 tx=0x7f9c34006c10 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:24.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.491+0000 7f9c39ffb640 1 -- 172.21.15.84:0/1496643353 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9c40076840 con 0x7f9c4406c040 2023-10-16T11:26:24.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.689+0000 7f9c4a70a640 1 -- 172.21.15.84:0/1496643353 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c4404e9b0 con 0x7f9c4406c040 2023-10-16T11:26:24.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.690+0000 7f9c39ffb640 1 -- 172.21.15.84:0/1496643353 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v246) v1 ==== 107+0+11 (secure 0 0 0) 0x7f9c40031e60 con 0x7f9c4406c040 2023-10-16T11:26:24.692 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:26:24.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.692+0000 7f9c4a70a640 1 -- 172.21.15.84:0/1496643353 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f9c2806b020 msgr2=0x7f9c2806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:24.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.692+0000 7f9c4a70a640 1 --2- 172.21.15.84:0/1496643353 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f9c2806b020 0x7f9c2806d4e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9c44157560 tx=0x7f9c34006c10 comp rx=0 tx=0).stop 2023-10-16T11:26:24.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.692+0000 7f9c4a70a640 1 -- 172.21.15.84:0/1496643353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4406c040 msgr2=0x7f9c440c5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:24.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.692+0000 7f9c4a70a640 1 --2- 172.21.15.84:0/1496643353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4406c040 0x7f9c440c5770 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 crypto rx=0x7f9c400098a0 tx=0x7f9c400046c0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.693+0000 7f9c4a70a640 1 -- 172.21.15.84:0/1496643353 shutdown_connections 2023-10-16T11:26:24.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.693+0000 7f9c4a70a640 1 --2- 172.21.15.84:0/1496643353 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f9c2806b020 0x7f9c2806d4e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.693+0000 7f9c4a70a640 1 --2- 172.21.15.84:0/1496643353 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c4415ebc0 0x7f9c440c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.693+0000 7f9c4a70a640 1 --2- 172.21.15.84:0/1496643353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c440ad970 0x7f9c440c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.693+0000 7f9c4a70a640 1 --2- 172.21.15.84:0/1496643353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4406c040 0x7f9c440c5770 unknown :-1 s=CLOSED pgs=3132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.693+0000 7f9c4a70a640 1 -- 172.21.15.84:0/1496643353 >> 172.21.15.84:0/1496643353 conn(0x7f9c440ce6a0 msgr2=0x7f9c4415d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:24.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.693+0000 7f9c4a70a640 1 -- 172.21.15.84:0/1496643353 shutdown_connections 2023-10-16T11:26:24.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.693+0000 7f9c4a70a640 1 -- 172.21.15.84:0/1496643353 wait complete. 2023-10-16T11:26:24.714 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-16T11:26:24.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.937+0000 7fd5e4a89640 1 Processor -- start 2023-10-16T11:26:24.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.937+0000 7fd5e4a89640 1 -- start start 2023-10-16T11:26:24.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.939+0000 7fd5e4a89640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5e00aeef0 0x7fd5e015f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.939+0000 7fd5e4a89640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e006c140 con 0x7fd5e00cbf20 2023-10-16T11:26:24.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.939+0000 7fd5e4a89640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e00af380 con 0x7fd5e015aee0 2023-10-16T11:26:24.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.939+0000 7fd5e4a89640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e015b2c0 con 0x7fd5e00aeef0 2023-10-16T11:26:24.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.939+0000 7fd5d7fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd5e00cbf20 0x7fd5e015e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47054/0 (socket says 172.21.15.84:47054) 2023-10-16T11:26:24.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.939+0000 7fd5d7fff640 1 -- 172.21.15.84:0/1889280899 learned_addr learned my addr 172.21.15.84:0/1889280899 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:24.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.940+0000 7fd5ddd9b640 1 -- 172.21.15.84:0/1889280899 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2993947165 0 0) 0x7fd5e006c140 con 0x7fd5e00cbf20 2023-10-16T11:26:24.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.940+0000 7fd5ddd9b640 1 -- 172.21.15.84:0/1889280899 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5cc003520 con 0x7fd5e00cbf20 2023-10-16T11:26:24.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.940+0000 7fd5ded9d640 1 --2- 172.21.15.84:0/1889280899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5e00aeef0 0x7fd5e015f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:24.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.940+0000 7fd5ddd9b640 1 -- 172.21.15.84:0/1889280899 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2586184691 0 0) 0x7fd5cc003520 con 0x7fd5e00cbf20 2023-10-16T11:26:24.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.940+0000 7fd5ddd9b640 1 -- 172.21.15.84:0/1889280899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5e00aeef0 msgr2=0x7fd5e015f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:24.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.940+0000 7fd5ddd9b640 1 --2- 172.21.15.84:0/1889280899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5e00aeef0 0x7fd5e015f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.940+0000 7fd5ddd9b640 1 -- 172.21.15.84:0/1889280899 >> v1:172.21.15.157:6789/0 conn(0x7fd5e015aee0 legacy=0x7fd5e00cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:24.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.940+0000 7fd5ddd9b640 1 -- 172.21.15.84:0/1889280899 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e006b270 con 0x7fd5e00cbf20 2023-10-16T11:26:24.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.940+0000 7fd5ddd9b640 1 -- 172.21.15.84:0/1889280899 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd5c8001670 con 0x7fd5e00cbf20 2023-10-16T11:26:24.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.940+0000 7fd5ddd9b640 1 -- 172.21.15.84:0/1889280899 >> v1:172.21.15.84:6789/0 conn(0x7fd5e00cbf20 legacy=0x7fd5e015e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:24.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.940+0000 7fd5ddd9b640 1 --2- 172.21.15.84:0/1889280899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5cc0039a0 0x7fd5cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.940+0000 7fd5ddd9b640 1 --2- 172.21.15.84:0/1889280899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5e00aeef0 0x7fd5cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.940+0000 7fd5ddd9b640 1 --2- 172.21.15.84:0/1889280899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5cc004d00 0x7fd5cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.940+0000 7fd5ddd9b640 1 -- 172.21.15.84:0/1889280899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e015b2c0 con 0x7fd5cc004d00 2023-10-16T11:26:24.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.940+0000 7fd5ddd9b640 1 -- 172.21.15.84:0/1889280899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc00bb60 con 0x7fd5cc0039a0 2023-10-16T11:26:24.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.940+0000 7fd5ddd9b640 1 -- 172.21.15.84:0/1889280899 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc00bd20 con 0x7fd5e00aeef0 2023-10-16T11:26:24.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.940+0000 7fd5ded9d640 1 --2- 172.21.15.84:0/1889280899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5cc004d00 0x7fd5cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:24.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.941+0000 7fd5de59c640 1 --2- 172.21.15.84:0/1889280899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5cc0039a0 0x7fd5cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:24.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.941+0000 7fd5d7fff640 1 --2- 172.21.15.84:0/1889280899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5e00aeef0 0x7fd5cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:24.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.941+0000 7fd5ded9d640 1 -- 172.21.15.84:0/1889280899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5e00aeef0 msgr2=0x7fd5cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:24.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.941+0000 7fd5ded9d640 1 --2- 172.21.15.84:0/1889280899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5e00aeef0 0x7fd5cc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.941+0000 7fd5ded9d640 1 -- 172.21.15.84:0/1889280899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5cc0039a0 msgr2=0x7fd5cc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:24.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.941+0000 7fd5ded9d640 1 --2- 172.21.15.84:0/1889280899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5cc0039a0 0x7fd5cc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.941+0000 7fd5ded9d640 1 -- 172.21.15.84:0/1889280899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5cc003520 con 0x7fd5cc004d00 2023-10-16T11:26:24.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.941+0000 7fd5d7fff640 1 --2- 172.21.15.84:0/1889280899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5e00aeef0 0x7fd5cc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:24.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.942+0000 7fd5ded9d640 1 --2- 172.21.15.84:0/1889280899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5cc004d00 0x7fd5cc005180 secure :-1 s=READY pgs=3134 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8007360 tx=0x7fd5cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23411eda83d81ba server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:24.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.942+0000 7fd5ddd9b640 1 -- 172.21.15.84:0/1889280899 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d802ccd0 con 0x7fd5cc004d00 2023-10-16T11:26:24.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.942+0000 7fd5ddd9b640 1 -- 172.21.15.84:0/1889280899 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d802ce30 con 0x7fd5cc004d00 2023-10-16T11:26:24.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.942+0000 7fd5ddd9b640 1 -- 172.21.15.84:0/1889280899 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8034660 con 0x7fd5cc004d00 2023-10-16T11:26:24.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.943+0000 7fd5e4a89640 1 -- 172.21.15.84:0/1889280899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5cc004d00 msgr2=0x7fd5cc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:24.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.943+0000 7fd5e4a89640 1 --2- 172.21.15.84:0/1889280899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5cc004d00 0x7fd5cc005180 secure :-1 s=READY pgs=3134 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8007360 tx=0x7fd5cc0057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.943+0000 7fd5ded9d640 1 -- 172.21.15.84:0/1889280899 reap_dead start 2023-10-16T11:26:24.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.943+0000 7fd5e4a89640 1 -- 172.21.15.84:0/1889280899 shutdown_connections 2023-10-16T11:26:24.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.943+0000 7fd5e4a89640 1 -- 172.21.15.84:0/1889280899 >> 172.21.15.84:0/1889280899 conn(0x7fd5e00cfac0 msgr2=0x7fd5e00cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:24.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.943+0000 7fd5e4a89640 1 -- 172.21.15.84:0/1889280899 shutdown_connections 2023-10-16T11:26:24.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.943+0000 7fd5e4a89640 1 -- 172.21.15.84:0/1889280899 wait complete. 2023-10-16T11:26:24.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.945+0000 7fd5e4a89640 1 Processor -- start 2023-10-16T11:26:24.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.945+0000 7fd5e4a89640 1 -- start start 2023-10-16T11:26:24.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.946+0000 7fd5e4a89640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5cc004d00 0x7fd5e015c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.946+0000 7fd5e4a89640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5e01e45c0 0x7fd5e01e4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.946+0000 7fd5e4a89640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5e015cfa0 0x7fd5e015d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.946+0000 7fd5e4a89640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e015e420 con 0x7fd5cc004d00 2023-10-16T11:26:24.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.946+0000 7fd5e4a89640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e01e4f80 con 0x7fd5e015cfa0 2023-10-16T11:26:24.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.946+0000 7fd5e4a89640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e015d960 con 0x7fd5e01e45c0 2023-10-16T11:26:24.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.946+0000 7fd5de59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5cc004d00 0x7fd5e015c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:24.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.946+0000 7fd5de59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5cc004d00 0x7fd5e015c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35420/0 (socket says 172.21.15.84:35420) 2023-10-16T11:26:24.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.946+0000 7fd5de59c640 1 -- 172.21.15.84:0/2349687456 learned_addr learned my addr 172.21.15.84:0/2349687456 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:24.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.947+0000 7fd5ded9d640 1 --2- 172.21.15.84:0/2349687456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5e015cfa0 0x7fd5e015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:24.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.947+0000 7fd5de59c640 1 -- 172.21.15.84:0/2349687456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5e01e45c0 msgr2=0x7fd5e01e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:24.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.947+0000 7fd5de59c640 1 --2- 172.21.15.84:0/2349687456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5e01e45c0 0x7fd5e01e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.947+0000 7fd5de59c640 1 -- 172.21.15.84:0/2349687456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5e015cfa0 msgr2=0x7fd5e015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:24.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.947+0000 7fd5de59c640 1 --2- 172.21.15.84:0/2349687456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5e015cfa0 0x7fd5e015d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:24.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.947+0000 7fd5de59c640 1 -- 172.21.15.84:0/2349687456 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e015b7b0 con 0x7fd5cc004d00 2023-10-16T11:26:24.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.947+0000 7fd5de59c640 1 --2- 172.21.15.84:0/2349687456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5cc004d00 0x7fd5e015c0c0 secure :-1 s=READY pgs=3135 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000e500 tx=0x7fd5d000e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:24.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.947+0000 7fd5d6ffd640 1 -- 172.21.15.84:0/2349687456 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d00053a0 con 0x7fd5cc004d00 2023-10-16T11:26:24.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.947+0000 7fd5d6ffd640 1 -- 172.21.15.84:0/2349687456 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d00035f0 con 0x7fd5cc004d00 2023-10-16T11:26:24.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.947+0000 7fd5d6ffd640 1 -- 172.21.15.84:0/2349687456 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d00039a0 con 0x7fd5cc004d00 2023-10-16T11:26:24.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.947+0000 7fd5e4a89640 1 -- 172.21.15.84:0/2349687456 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e015db80 con 0x7fd5cc004d00 2023-10-16T11:26:24.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.947+0000 7fd5e4a89640 1 -- 172.21.15.84:0/2349687456 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e015e140 con 0x7fd5cc004d00 2023-10-16T11:26:24.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.949+0000 7fd5e4a89640 1 -- 172.21.15.84:0/2349687456 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e004e160 con 0x7fd5cc004d00 2023-10-16T11:26:24.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.950+0000 7fd5d6ffd640 1 -- 172.21.15.84:0/2349687456 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fd5d0003b00 con 0x7fd5cc004d00 2023-10-16T11:26:24.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.951+0000 7fd5d6ffd640 1 --2- 172.21.15.84:0/2349687456 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd5c006b050 0x7fd5c006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:24.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.951+0000 7fd5d6ffd640 1 -- 172.21.15.84:0/2349687456 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6695+0+0 (secure 0 0 0) 0x7fd5d0089a00 con 0x7fd5cc004d00 2023-10-16T11:26:24.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.951+0000 7fd5d7fff640 1 --2- 172.21.15.84:0/2349687456 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd5c006b050 0x7fd5c006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:24.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.952+0000 7fd5d7fff640 1 --2- 172.21.15.84:0/2349687456 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd5c006b050 0x7fd5c006d510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8001eb0 tx=0x7fd5c8009040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:24.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:24.956+0000 7fd5d6ffd640 1 -- 172.21.15.84:0/2349687456 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd5d00543c0 con 0x7fd5cc004d00 2023-10-16T11:26:25.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.150+0000 7fd5e4a89640 1 -- 172.21.15.84:0/2349687456 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7fd5e00af380 con 0x7fd5cc004d00 2023-10-16T11:26:25.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.151+0000 7fd5d6ffd640 1 -- 172.21.15.84:0/2349687456 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v246) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd5e00af380 con 0x7fd5cc004d00 2023-10-16T11:26:25.153 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:26:25.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.153+0000 7fd5e4a89640 1 -- 172.21.15.84:0/2349687456 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd5c006b050 msgr2=0x7fd5c006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:25.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.153+0000 7fd5e4a89640 1 --2- 172.21.15.84:0/2349687456 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd5c006b050 0x7fd5c006d510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8001eb0 tx=0x7fd5c8009040 comp rx=0 tx=0).stop 2023-10-16T11:26:25.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.153+0000 7fd5e4a89640 1 -- 172.21.15.84:0/2349687456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5cc004d00 msgr2=0x7fd5e015c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:25.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.153+0000 7fd5e4a89640 1 --2- 172.21.15.84:0/2349687456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5cc004d00 0x7fd5e015c0c0 secure :-1 s=READY pgs=3135 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000e500 tx=0x7fd5d000e9c0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.154+0000 7fd5e4a89640 1 -- 172.21.15.84:0/2349687456 shutdown_connections 2023-10-16T11:26:25.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.154+0000 7fd5e4a89640 1 --2- 172.21.15.84:0/2349687456 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd5c006b050 0x7fd5c006d510 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.154+0000 7fd5e4a89640 1 --2- 172.21.15.84:0/2349687456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5e015cfa0 0x7fd5e015d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.154+0000 7fd5e4a89640 1 --2- 172.21.15.84:0/2349687456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5e01e45c0 0x7fd5e01e4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.154+0000 7fd5e4a89640 1 --2- 172.21.15.84:0/2349687456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5cc004d00 0x7fd5e015c0c0 unknown :-1 s=CLOSED pgs=3135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.154+0000 7fd5e4a89640 1 -- 172.21.15.84:0/2349687456 >> 172.21.15.84:0/2349687456 conn(0x7fd5e00cfac0 msgr2=0x7fd5e0158fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:25.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.154+0000 7fd5e4a89640 1 -- 172.21.15.84:0/2349687456 shutdown_connections 2023-10-16T11:26:25.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.154+0000 7fd5e4a89640 1 -- 172.21.15.84:0/2349687456 wait complete. 2023-10-16T11:26:25.174 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:26:25.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.397+0000 7fb8e9661640 1 Processor -- start 2023-10-16T11:26:25.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.397+0000 7fb8e9661640 1 -- start start 2023-10-16T11:26:25.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.398+0000 7fb8e9661640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb8e4159b20 0x7fb8e406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:25.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.398+0000 7fb8e9661640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8e4159f00 0x7fb8e40b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:25.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.398+0000 7fb8e9661640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb8e406c120 0x7fb8e40b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:25.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.398+0000 7fb8e9661640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e406f7f0 con 0x7fb8e406c120 2023-10-16T11:26:25.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.398+0000 7fb8e9661640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e406c5e0 con 0x7fb8e4159b20 2023-10-16T11:26:25.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.398+0000 7fb8e9661640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e406c8c0 con 0x7fb8e4159f00 2023-10-16T11:26:25.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.398+0000 7fb8e37fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb8e406c120 0x7fb8e40b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:25.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.399+0000 7fb8e37fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb8e406c120 0x7fb8e40b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35422/0 (socket says 172.21.15.84:35422) 2023-10-16T11:26:25.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.399+0000 7fb8e37fe640 1 -- 172.21.15.84:0/1490196543 learned_addr learned my addr 172.21.15.84:0/1490196543 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:25.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.399+0000 7fb8e2ffd640 1 --2- 172.21.15.84:0/1490196543 >> v2:172.21.15.157:3300/0 conn(0x7fb8e4159b20 0x7fb8e406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:25.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.399+0000 7fb8e27fc640 1 --2- 172.21.15.84:0/1490196543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8e4159f00 0x7fb8e40b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:25.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.399+0000 7fb8e37fe640 1 -- 172.21.15.84:0/1490196543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8e4159f00 msgr2=0x7fb8e40b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:25.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.399+0000 7fb8e37fe640 1 --2- 172.21.15.84:0/1490196543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8e4159f00 0x7fb8e40b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.399+0000 7fb8e37fe640 1 -- 172.21.15.84:0/1490196543 >> v2:172.21.15.157:3300/0 conn(0x7fb8e4159b20 msgr2=0x7fb8e406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:25.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.399+0000 7fb8e37fe640 1 --2- 172.21.15.84:0/1490196543 >> v2:172.21.15.157:3300/0 conn(0x7fb8e4159b20 0x7fb8e406bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.399+0000 7fb8e37fe640 1 -- 172.21.15.84:0/1490196543 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e406cba0 con 0x7fb8e406c120 2023-10-16T11:26:25.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.400+0000 7fb8e37fe640 1 --2- 172.21.15.84:0/1490196543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e406c120 0x7fb8e40b0950 secure :-1 s=READY pgs=3136 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40c9340 tx=0x7fb8dc02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debf5f0650f5405a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:25.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.401+0000 7fb8e1ffb640 1 -- 172.21.15.84:0/1490196543 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc031860 con 0x7fb8e406c120 2023-10-16T11:26:25.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.401+0000 7fb8e1ffb640 1 -- 172.21.15.84:0/1490196543 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8dc0319c0 con 0x7fb8e406c120 2023-10-16T11:26:25.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.401+0000 7fb8e1ffb640 1 -- 172.21.15.84:0/1490196543 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc031d50 con 0x7fb8e406c120 2023-10-16T11:26:25.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.401+0000 7fb8e9661640 1 -- 172.21.15.84:0/1490196543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e406c120 msgr2=0x7fb8e40b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:25.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.401+0000 7fb8e9661640 1 --2- 172.21.15.84:0/1490196543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e406c120 0x7fb8e40b0950 secure :-1 s=READY pgs=3136 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40c9340 tx=0x7fb8dc02b900 comp rx=0 tx=0).stop 2023-10-16T11:26:25.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.401+0000 7fb8e9661640 1 -- 172.21.15.84:0/1490196543 shutdown_connections 2023-10-16T11:26:25.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.401+0000 7fb8e9661640 1 --2- 172.21.15.84:0/1490196543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e406c120 0x7fb8e40b0950 unknown :-1 s=CLOSED pgs=3136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.401+0000 7fb8e9661640 1 --2- 172.21.15.84:0/1490196543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8e4159f00 0x7fb8e40b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.401+0000 7fb8e9661640 1 --2- 172.21.15.84:0/1490196543 >> v2:172.21.15.157:3300/0 conn(0x7fb8e4159b20 0x7fb8e406bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.401+0000 7fb8e9661640 1 -- 172.21.15.84:0/1490196543 >> 172.21.15.84:0/1490196543 conn(0x7fb8e40cb410 msgr2=0x7fb8e40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:25.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.401+0000 7fb8e9661640 1 -- 172.21.15.84:0/1490196543 shutdown_connections 2023-10-16T11:26:25.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.401+0000 7fb8e9661640 1 -- 172.21.15.84:0/1490196543 wait complete. 2023-10-16T11:26:25.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.403+0000 7fb8e9661640 1 Processor -- start 2023-10-16T11:26:25.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.403+0000 7fb8e9661640 1 -- start start 2023-10-16T11:26:25.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.404+0000 7fb8e9661640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e406c120 0x7fb8e40c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:25.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.404+0000 7fb8e9661640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8e4159b20 0x7fb8e40c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:25.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.404+0000 7fb8e9661640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8e4159f00 0x7fb8e415c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:25.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.404+0000 7fb8e9661640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e414ec90 con 0x7fb8e406c120 2023-10-16T11:26:25.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.404+0000 7fb8e9661640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e414fab0 con 0x7fb8e4159b20 2023-10-16T11:26:25.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.404+0000 7fb8e9661640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e414eea0 con 0x7fb8e4159f00 2023-10-16T11:26:25.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.404+0000 7fb8e2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e406c120 0x7fb8e40c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:25.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.404+0000 7fb8e2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e406c120 0x7fb8e40c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35428/0 (socket says 172.21.15.84:35428) 2023-10-16T11:26:25.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.405+0000 7fb8e2ffd640 1 -- 172.21.15.84:0/751061721 learned_addr learned my addr 172.21.15.84:0/751061721 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:25.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.405+0000 7fb8e2ffd640 1 -- 172.21.15.84:0/751061721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8e4159f00 msgr2=0x7fb8e415c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:25.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.405+0000 7fb8e2ffd640 1 --2- 172.21.15.84:0/751061721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8e4159f00 0x7fb8e415c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.405+0000 7fb8e2ffd640 1 -- 172.21.15.84:0/751061721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8e4159b20 msgr2=0x7fb8e40c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:25.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.405+0000 7fb8e2ffd640 1 --2- 172.21.15.84:0/751061721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8e4159b20 0x7fb8e40c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.405+0000 7fb8e2ffd640 1 -- 172.21.15.84:0/751061721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc0094d0 con 0x7fb8e406c120 2023-10-16T11:26:25.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.405+0000 7fb8e2ffd640 1 --2- 172.21.15.84:0/751061721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e406c120 0x7fb8e40c42d0 secure :-1 s=READY pgs=3137 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4007ef0 tx=0x7fb8d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:25.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.405+0000 7fb8db7fe640 1 -- 172.21.15.84:0/751061721 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4014620 con 0x7fb8e406c120 2023-10-16T11:26:25.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.406+0000 7fb8db7fe640 1 -- 172.21.15.84:0/751061721 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d4014780 con 0x7fb8e406c120 2023-10-16T11:26:25.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.406+0000 7fb8db7fe640 1 -- 172.21.15.84:0/751061721 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4014b30 con 0x7fb8e406c120 2023-10-16T11:26:25.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.406+0000 7fb8e9661640 1 -- 172.21.15.84:0/751061721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e414f180 con 0x7fb8e406c120 2023-10-16T11:26:25.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.406+0000 7fb8e9661640 1 -- 172.21.15.84:0/751061721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e414f740 con 0x7fb8e406c120 2023-10-16T11:26:25.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.407+0000 7fb8e9661640 1 -- 172.21.15.84:0/751061721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e414f900 con 0x7fb8e406c120 2023-10-16T11:26:25.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.408+0000 7fb8db7fe640 1 -- 172.21.15.84:0/751061721 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fb8d4014c90 con 0x7fb8e406c120 2023-10-16T11:26:25.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.409+0000 7fb8db7fe640 1 --2- 172.21.15.84:0/751061721 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fb8c406b020 0x7fb8c406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:25.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.409+0000 7fb8db7fe640 1 -- 172.21.15.84:0/751061721 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6695+0+0 (secure 0 0 0) 0x7fb8d408a720 con 0x7fb8e406c120 2023-10-16T11:26:25.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.409+0000 7fb8e27fc640 1 --2- 172.21.15.84:0/751061721 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fb8c406b020 0x7fb8c406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:25.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.410+0000 7fb8e27fc640 1 --2- 172.21.15.84:0/751061721 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fb8c406b020 0x7fb8c406d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb8e415d120 tx=0x7fb8cc008040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:25.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.413+0000 7fb8db7fe640 1 -- 172.21.15.84:0/751061721 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb8d4055160 con 0x7fb8e406c120 2023-10-16T11:26:25.613 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:25.609+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:25.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.650+0000 7fb8e9661640 1 -- 172.21.15.84:0/751061721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8e404e160 con 0x7fb8e406c120 2023-10-16T11:26:25.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.651+0000 7fb8db7fe640 1 -- 172.21.15.84:0/751061721 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+4933 (secure 0 0 0) 0x7fb8d4058780 con 0x7fb8e406c120 2023-10-16T11:26:25.653 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:25.653 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":139,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2516077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2516077743},{"type":"v1","addr":"172.21.15.157:6833","nonce":2516077743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1527104462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1527104462},{"type":"v1","addr":"172.21.15.157:6835","nonce":1527104462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":12801,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3617906728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3617906728},{"type":"v1","addr":"172.21.15.84:6837","nonce":3617906728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:25:50.739234+0000","modified":"2023-10-16T11:25:53.792071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12183":{"gid":12183,"name":"c","rank":0,"incarnation":133,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/369067539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":369067539},{"type":"v1","addr":"172.21.15.84:6835","nonce":369067539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-16T11:26:25.654 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 139 2023-10-16T11:26:25.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.653+0000 7fb8e9661640 1 -- 172.21.15.84:0/751061721 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fb8c406b020 msgr2=0x7fb8c406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:25.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.653+0000 7fb8e9661640 1 --2- 172.21.15.84:0/751061721 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fb8c406b020 0x7fb8c406d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb8e415d120 tx=0x7fb8cc008040 comp rx=0 tx=0).stop 2023-10-16T11:26:25.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.653+0000 7fb8e9661640 1 -- 172.21.15.84:0/751061721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e406c120 msgr2=0x7fb8e40c42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:25.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.653+0000 7fb8e9661640 1 --2- 172.21.15.84:0/751061721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e406c120 0x7fb8e40c42d0 secure :-1 s=READY pgs=3137 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4007ef0 tx=0x7fb8d400e4b0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.653+0000 7fb8e9661640 1 -- 172.21.15.84:0/751061721 shutdown_connections 2023-10-16T11:26:25.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.653+0000 7fb8e9661640 1 --2- 172.21.15.84:0/751061721 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fb8c406b020 0x7fb8c406d4e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.653+0000 7fb8e9661640 1 --2- 172.21.15.84:0/751061721 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8e4159f00 0x7fb8e415c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.653+0000 7fb8e9661640 1 --2- 172.21.15.84:0/751061721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8e4159b20 0x7fb8e40c4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.654+0000 7fb8e9661640 1 --2- 172.21.15.84:0/751061721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e406c120 0x7fb8e40c42d0 unknown :-1 s=CLOSED pgs=3137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.654+0000 7fb8e9661640 1 -- 172.21.15.84:0/751061721 >> 172.21.15.84:0/751061721 conn(0x7fb8e40cb410 msgr2=0x7fb8e4154470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:25.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.654+0000 7fb8e9661640 1 -- 172.21.15.84:0/751061721 shutdown_connections 2023-10-16T11:26:25.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.654+0000 7fb8e9661640 1 -- 172.21.15.84:0/751061721 wait complete. 2023-10-16T11:26:25.676 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:26:25.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.884+0000 7fea7ac90640 1 Processor -- start 2023-10-16T11:26:25.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.884+0000 7fea7ac90640 1 -- start start 2023-10-16T11:26:25.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.886+0000 7fea7ac90640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fea7406c040 0x7fea7406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:25.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.886+0000 7fea7ac90640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fea740ad970 0x7fea7415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:25.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.886+0000 7fea7ac90640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea7406c960 con 0x7fea7415ebc0 2023-10-16T11:26:25.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.886+0000 7fea7ac90640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea7415b9d0 con 0x7fea740ad970 2023-10-16T11:26:25.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.886+0000 7fea7ac90640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea7415bcb0 con 0x7fea7406c040 2023-10-16T11:26:25.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.886+0000 7fea7948d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fea740ad970 0x7fea7415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:25.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.886+0000 7fea7a48f640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fea7415ebc0 0x7fea740cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60762/0 (socket says 172.21.15.84:60762) 2023-10-16T11:26:25.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.886+0000 7fea7948d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fea740ad970 0x7fea7415e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46642/0 (socket says 172.21.15.84:46642) 2023-10-16T11:26:25.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.886+0000 7fea7a48f640 1 -- 172.21.15.84:0/3676955529 learned_addr learned my addr 172.21.15.84:0/3676955529 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:25.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.887+0000 7fea79c8e640 1 --2- 172.21.15.84:0/3676955529 >> v2:172.21.15.157:3300/0 conn(0x7fea7406c040 0x7fea7406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:25.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.887+0000 7fea7948d640 1 -- 172.21.15.84:0/3676955529 >> v2:172.21.15.157:3300/0 conn(0x7fea7406c040 msgr2=0x7fea7406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:25.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.887+0000 7fea7948d640 1 --2- 172.21.15.84:0/3676955529 >> v2:172.21.15.157:3300/0 conn(0x7fea7406c040 0x7fea7406c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.887+0000 7fea7948d640 1 -- 172.21.15.84:0/3676955529 >> v1:172.21.15.84:6789/0 conn(0x7fea7415ebc0 legacy=0x7fea740cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:25.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.887+0000 7fea7948d640 1 -- 172.21.15.84:0/3676955529 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7415bf90 con 0x7fea740ad970 2023-10-16T11:26:25.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.888+0000 7fea7948d640 1 --2- 172.21.15.84:0/3676955529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea740ad970 0x7fea7415e680 secure :-1 s=READY pgs=3139 cs=0 l=1 rev1=1 crypto rx=0x7fea740cf350 tx=0x7fea6402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce4c3340a05c0f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:25.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.888+0000 7fea78c8c640 1 -- 172.21.15.84:0/3676955529 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea64031830 con 0x7fea740ad970 2023-10-16T11:26:25.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.888+0000 7fea78c8c640 1 -- 172.21.15.84:0/3676955529 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea64031990 con 0x7fea740ad970 2023-10-16T11:26:25.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.888+0000 7fea78c8c640 1 -- 172.21.15.84:0/3676955529 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea64031d20 con 0x7fea740ad970 2023-10-16T11:26:25.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.889+0000 7fea7ac90640 1 -- 172.21.15.84:0/3676955529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea740ad970 msgr2=0x7fea7415e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:25.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.889+0000 7fea7ac90640 1 --2- 172.21.15.84:0/3676955529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea740ad970 0x7fea7415e680 secure :-1 s=READY pgs=3139 cs=0 l=1 rev1=1 crypto rx=0x7fea740cf350 tx=0x7fea6402b8d0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.889+0000 7fea7ac90640 1 -- 172.21.15.84:0/3676955529 shutdown_connections 2023-10-16T11:26:25.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.889+0000 7fea7ac90640 1 --2- 172.21.15.84:0/3676955529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea740ad970 0x7fea7415e680 unknown :-1 s=CLOSED pgs=3139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.889+0000 7fea7ac90640 1 --2- 172.21.15.84:0/3676955529 >> v2:172.21.15.157:3300/0 conn(0x7fea7406c040 0x7fea7406c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.889+0000 7fea7ac90640 1 -- 172.21.15.84:0/3676955529 >> 172.21.15.84:0/3676955529 conn(0x7fea740ce6a0 msgr2=0x7fea740ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:25.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.889+0000 7fea7ac90640 1 -- 172.21.15.84:0/3676955529 shutdown_connections 2023-10-16T11:26:25.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.889+0000 7fea7ac90640 1 -- 172.21.15.84:0/3676955529 wait complete. 2023-10-16T11:26:25.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.891+0000 7fea7ac90640 1 Processor -- start 2023-10-16T11:26:25.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.891+0000 7fea7ac90640 1 -- start start 2023-10-16T11:26:25.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.892+0000 7fea7ac90640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea7406c040 0x7fea740c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:25.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.892+0000 7fea7ac90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea740ad970 0x7fea740ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:25.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.892+0000 7fea7ac90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea7415ebc0 0x7fea74153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:25.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.892+0000 7fea7ac90640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea740ca030 con 0x7fea7406c040 2023-10-16T11:26:25.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.892+0000 7fea7ac90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea740cad40 con 0x7fea7415ebc0 2023-10-16T11:26:25.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.892+0000 7fea7ac90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea740c8490 con 0x7fea740ad970 2023-10-16T11:26:25.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.892+0000 7fea79c8e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea7406c040 0x7fea740c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:25.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.892+0000 7fea79c8e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea7406c040 0x7fea740c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46648/0 (socket says 172.21.15.84:46648) 2023-10-16T11:26:25.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.892+0000 7fea79c8e640 1 -- 172.21.15.84:0/1448326452 learned_addr learned my addr 172.21.15.84:0/1448326452 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:25.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.893+0000 7fea79c8e640 1 -- 172.21.15.84:0/1448326452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea740ad970 msgr2=0x7fea740ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:25.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.893+0000 7fea7a48f640 1 --2- 172.21.15.84:0/1448326452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea7415ebc0 0x7fea74153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:25.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.893+0000 7fea79c8e640 1 --2- 172.21.15.84:0/1448326452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea740ad970 0x7fea740ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.893+0000 7fea79c8e640 1 -- 172.21.15.84:0/1448326452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea7415ebc0 msgr2=0x7fea74153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:25.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.893+0000 7fea79c8e640 1 --2- 172.21.15.84:0/1448326452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea7415ebc0 0x7fea74153720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:25.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.893+0000 7fea79c8e640 1 -- 172.21.15.84:0/1448326452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea64009400 con 0x7fea7406c040 2023-10-16T11:26:25.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.893+0000 7fea79c8e640 1 --2- 172.21.15.84:0/1448326452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea7406c040 0x7fea740c7e20 secure :-1 s=READY pgs=3140 cs=0 l=1 rev1=1 crypto rx=0x7fea7000d5f0 tx=0x7fea7000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:25.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.893+0000 7fea6a7fc640 1 -- 172.21.15.84:0/1448326452 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea700125f0 con 0x7fea7406c040 2023-10-16T11:26:25.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.893+0000 7fea6a7fc640 1 -- 172.21.15.84:0/1448326452 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea70012750 con 0x7fea7406c040 2023-10-16T11:26:25.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.893+0000 7fea6a7fc640 1 -- 172.21.15.84:0/1448326452 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70012b00 con 0x7fea7406c040 2023-10-16T11:26:25.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.893+0000 7fea7ac90640 1 -- 172.21.15.84:0/1448326452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea740c9700 con 0x7fea7406c040 2023-10-16T11:26:25.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.893+0000 7fea7ac90640 1 -- 172.21.15.84:0/1448326452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea740c9cc0 con 0x7fea7406c040 2023-10-16T11:26:25.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.895+0000 7fea7ac90640 1 -- 172.21.15.84:0/1448326452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7406b330 con 0x7fea7406c040 2023-10-16T11:26:25.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.895+0000 7fea6a7fc640 1 -- 172.21.15.84:0/1448326452 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fea70012c60 con 0x7fea7406c040 2023-10-16T11:26:25.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.896+0000 7fea6a7fc640 1 --2- 172.21.15.84:0/1448326452 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fea5806b020 0x7fea5806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:25.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.896+0000 7fea6a7fc640 1 -- 172.21.15.84:0/1448326452 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6695+0+0 (secure 0 0 0) 0x7fea7008a2a0 con 0x7fea7406c040 2023-10-16T11:26:25.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.897+0000 7fea7948d640 1 --2- 172.21.15.84:0/1448326452 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fea5806b020 0x7fea5806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:25.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.897+0000 7fea7948d640 1 --2- 172.21.15.84:0/1448326452 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fea5806b020 0x7fea5806d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fea64036000 tx=0x7fea64030000 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:25.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:25.901+0000 7fea6a7fc640 1 -- 172.21.15.84:0/1448326452 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fea70054c50 con 0x7fea7406c040 2023-10-16T11:26:26.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.099+0000 7fea7ac90640 1 -- 172.21.15.84:0/1448326452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea7404e9b0 con 0x7fea7406c040 2023-10-16T11:26:26.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.102+0000 7fea6a7fc640 1 -- 172.21.15.84:0/1448326452 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v246) v1 ==== 74+0+17928 (secure 0 0 0) 0x7fea70058270 con 0x7fea7406c040 2023-10-16T11:26:26.104 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:26.105 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":246,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:26:23.239524+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:25:49.001107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:25:49.760637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6837/3547503033":"2023-10-17T11:26:20.041719+0000","172.21.15.84:0/2606713455":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/488945617":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/4036998934":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6832/3938045451":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/570042115":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1311117536":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6833/2963529201":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/95207886":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6832/2963529201":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/388205040":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/2303342316":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/648102622":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/133211355":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1263886961":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1489017743":"2023-10-17T11:26:08.077644+0000","172.21.15.84:6833/3938045451":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1517291402":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1983743360":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1323010124":"2023-10-17T11:26:08.077644+0000","172.21.15.84:6836/3547503033":"2023-10-17T11:26:20.041719+0000","172.21.15.84:0/437386817":"2023-10-17T11:26:08.077644+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:26:26.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.105+0000 7fea7ac90640 1 -- 172.21.15.84:0/1448326452 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fea5806b020 msgr2=0x7fea5806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:26.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.105+0000 7fea7ac90640 1 --2- 172.21.15.84:0/1448326452 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fea5806b020 0x7fea5806d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fea64036000 tx=0x7fea64030000 comp rx=0 tx=0).stop 2023-10-16T11:26:26.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.105+0000 7fea7ac90640 1 -- 172.21.15.84:0/1448326452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea7406c040 msgr2=0x7fea740c7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:26.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.105+0000 7fea7ac90640 1 --2- 172.21.15.84:0/1448326452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea7406c040 0x7fea740c7e20 secure :-1 s=READY pgs=3140 cs=0 l=1 rev1=1 crypto rx=0x7fea7000d5f0 tx=0x7fea7000dab0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.105+0000 7fea7ac90640 1 -- 172.21.15.84:0/1448326452 shutdown_connections 2023-10-16T11:26:26.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.105+0000 7fea7ac90640 1 --2- 172.21.15.84:0/1448326452 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fea5806b020 0x7fea5806d4e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.105+0000 7fea7ac90640 1 --2- 172.21.15.84:0/1448326452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea7415ebc0 0x7fea74153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.105+0000 7fea7ac90640 1 --2- 172.21.15.84:0/1448326452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea740ad970 0x7fea740ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.105+0000 7fea7ac90640 1 --2- 172.21.15.84:0/1448326452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea7406c040 0x7fea740c7e20 unknown :-1 s=CLOSED pgs=3140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.105+0000 7fea7ac90640 1 -- 172.21.15.84:0/1448326452 >> 172.21.15.84:0/1448326452 conn(0x7fea740ce6a0 msgr2=0x7fea74159f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:26.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.105+0000 7fea7ac90640 1 -- 172.21.15.84:0/1448326452 shutdown_connections 2023-10-16T11:26:26.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.105+0000 7fea7ac90640 1 -- 172.21.15.84:0/1448326452 wait complete. 2023-10-16T11:26:26.127 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2023-10-16T11:26:26.128 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:26:26.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.349+0000 7f47d018a640 1 Processor -- start 2023-10-16T11:26:26.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.349+0000 7f47d018a640 1 -- start start 2023-10-16T11:26:26.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.350+0000 7f47d018a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f47c8159f00 0x7f47c80b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:26.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.350+0000 7f47d018a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c806f7f0 con 0x7f47c8159b20 2023-10-16T11:26:26.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.350+0000 7f47d018a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c806c5e0 con 0x7f47c815d880 2023-10-16T11:26:26.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.350+0000 7f47d018a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c806c8c0 con 0x7f47c8159f00 2023-10-16T11:26:26.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.351+0000 7f47cdf26640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f47c8159b20 0x7f47c806bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60764/0 (socket says 172.21.15.84:60764) 2023-10-16T11:26:26.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.351+0000 7f47cdf26640 1 -- 172.21.15.84:0/3739468340 learned_addr learned my addr 172.21.15.84:0/3739468340 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:26.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.351+0000 7f47cd725640 1 --2- 172.21.15.84:0/3739468340 >> v2:172.21.15.157:3300/0 conn(0x7f47c8159f00 0x7f47c80b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:26.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.351+0000 7f47ccf24640 1 -- 172.21.15.84:0/3739468340 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982640349 0 0) 0x7f47c806f7f0 con 0x7f47c8159b20 2023-10-16T11:26:26.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.351+0000 7f47ccf24640 1 -- 172.21.15.84:0/3739468340 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47b4003520 con 0x7f47c8159b20 2023-10-16T11:26:26.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.351+0000 7f47ccf24640 1 -- 172.21.15.84:0/3739468340 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 928065327 0 0) 0x7f47b4003520 con 0x7f47c8159b20 2023-10-16T11:26:26.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.351+0000 7f47ccf24640 1 -- 172.21.15.84:0/3739468340 >> v2:172.21.15.157:3300/0 conn(0x7f47c8159f00 msgr2=0x7f47c80b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:26.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.351+0000 7f47ccf24640 1 --2- 172.21.15.84:0/3739468340 >> v2:172.21.15.157:3300/0 conn(0x7f47c8159f00 0x7f47c80b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.352+0000 7f47ccf24640 1 -- 172.21.15.84:0/3739468340 >> v1:172.21.15.157:6789/0 conn(0x7f47c815d880 legacy=0x7f47c806fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:26.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.352+0000 7f47ccf24640 1 -- 172.21.15.84:0/3739468340 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c806cba0 con 0x7f47c8159b20 2023-10-16T11:26:26.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.352+0000 7f47ccf24640 1 -- 172.21.15.84:0/3739468340 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f47c40019f0 con 0x7f47c8159b20 2023-10-16T11:26:26.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.352+0000 7f47ccf24640 1 -- 172.21.15.84:0/3739468340 >> v1:172.21.15.84:6789/0 conn(0x7f47c8159b20 legacy=0x7f47c806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:26.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.352+0000 7f47ccf24640 1 --2- 172.21.15.84:0/3739468340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47b4003970 0x7f47b4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:26.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.352+0000 7f47ccf24640 1 --2- 172.21.15.84:0/3739468340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b40047c0 0x7f47b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:26.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.352+0000 7f47ccf24640 1 --2- 172.21.15.84:0/3739468340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b4005180 0x7f47b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:26.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.352+0000 7f47ccf24640 1 -- 172.21.15.84:0/3739468340 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c806c8c0 con 0x7f47b4005180 2023-10-16T11:26:26.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.352+0000 7f47ccf24640 1 -- 172.21.15.84:0/3739468340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b400c0d0 con 0x7f47b40047c0 2023-10-16T11:26:26.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.352+0000 7f47ccf24640 1 -- 172.21.15.84:0/3739468340 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b400c290 con 0x7f47b4003970 2023-10-16T11:26:26.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.352+0000 7f47ce727640 1 --2- 172.21.15.84:0/3739468340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b4005180 0x7f47b400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:26.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.352+0000 7f47cdf26640 1 --2- 172.21.15.84:0/3739468340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47b4003970 0x7f47b4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:26.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.352+0000 7f47cd725640 1 --2- 172.21.15.84:0/3739468340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b40047c0 0x7f47b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:26.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.353+0000 7f47ce727640 1 -- 172.21.15.84:0/3739468340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47b4003970 msgr2=0x7f47b4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:26.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.353+0000 7f47ce727640 1 --2- 172.21.15.84:0/3739468340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47b4003970 0x7f47b4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.353+0000 7f47ce727640 1 -- 172.21.15.84:0/3739468340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b40047c0 msgr2=0x7f47b4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:26.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.353+0000 7f47ce727640 1 --2- 172.21.15.84:0/3739468340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b40047c0 0x7f47b4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.353+0000 7f47ce727640 1 -- 172.21.15.84:0/3739468340 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b4003520 con 0x7f47b4005180 2023-10-16T11:26:26.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.354+0000 7f47ce727640 1 -- 172.21.15.84:0/3739468340 reap_dead start 2023-10-16T11:26:26.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.354+0000 7f47ce727640 1 --2- 172.21.15.84:0/3739468340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b4005180 0x7f47b400b980 secure :-1 s=READY pgs=3142 cs=0 l=1 rev1=1 crypto rx=0x7f47c0001750 tx=0x7f47b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c344267751655c2e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:26.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.354+0000 7f47ccf24640 1 -- 172.21.15.84:0/3739468340 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c00052d0 con 0x7f47b4005180 2023-10-16T11:26:26.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.354+0000 7f47ccf24640 1 -- 172.21.15.84:0/3739468340 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c002e050 con 0x7f47b4005180 2023-10-16T11:26:26.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.354+0000 7f47ccf24640 1 -- 172.21.15.84:0/3739468340 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c00035f0 con 0x7f47b4005180 2023-10-16T11:26:26.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.355+0000 7f47d018a640 1 -- 172.21.15.84:0/3739468340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b4005180 msgr2=0x7f47b400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:26.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.355+0000 7f47d018a640 1 --2- 172.21.15.84:0/3739468340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b4005180 0x7f47b400b980 secure :-1 s=READY pgs=3142 cs=0 l=1 rev1=1 crypto rx=0x7f47c0001750 tx=0x7f47b40057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.355+0000 7f47d018a640 1 -- 172.21.15.84:0/3739468340 shutdown_connections 2023-10-16T11:26:26.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.355+0000 7f47d018a640 1 --2- 172.21.15.84:0/3739468340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b4005180 0x7f47b400b980 unknown :-1 s=CLOSED pgs=3142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.355+0000 7f47d018a640 1 -- 172.21.15.84:0/3739468340 >> 172.21.15.84:0/3739468340 conn(0x7f47c80cb410 msgr2=0x7f47c80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:26.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.355+0000 7f47d018a640 1 -- 172.21.15.84:0/3739468340 shutdown_connections 2023-10-16T11:26:26.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.355+0000 7f47d018a640 1 -- 172.21.15.84:0/3739468340 wait complete. 2023-10-16T11:26:26.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.357+0000 7f47d018a640 1 Processor -- start 2023-10-16T11:26:26.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.357+0000 7f47d018a640 1 -- start start 2023-10-16T11:26:26.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.358+0000 7f47d018a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b4005180 0x7f47c8149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:26.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.358+0000 7f47d018a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47c815b360 0x7f47c815b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:26.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.358+0000 7f47d018a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c8151910 0x7f47c8151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:26.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.358+0000 7f47d018a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c815caa0 con 0x7f47b4005180 2023-10-16T11:26:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.358+0000 7f47d018a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c81522d0 con 0x7f47c8151910 2023-10-16T11:26:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.358+0000 7f47d018a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c815be20 con 0x7f47c815b360 2023-10-16T11:26:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.358+0000 7f47cdf26640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b4005180 0x7f47c8149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.358+0000 7f47cdf26640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b4005180 0x7f47c8149540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46674/0 (socket says 172.21.15.84:46674) 2023-10-16T11:26:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.358+0000 7f47cdf26640 1 -- 172.21.15.84:0/752036160 learned_addr learned my addr 172.21.15.84:0/752036160 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.359+0000 7f47cd725640 1 --2- 172.21.15.84:0/752036160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47c815b360 0x7f47c815b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.359+0000 7f47ce727640 1 --2- 172.21.15.84:0/752036160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c8151910 0x7f47c8151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.359+0000 7f47cdf26640 1 -- 172.21.15.84:0/752036160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47c815b360 msgr2=0x7f47c815b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.359+0000 7f47cdf26640 1 --2- 172.21.15.84:0/752036160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47c815b360 0x7f47c815b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.359+0000 7f47cdf26640 1 -- 172.21.15.84:0/752036160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c8151910 msgr2=0x7f47c8151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:26.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.359+0000 7f47cdf26640 1 --2- 172.21.15.84:0/752036160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c8151910 0x7f47c8151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.359+0000 7f47cdf26640 1 -- 172.21.15.84:0/752036160 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c815c100 con 0x7f47b4005180 2023-10-16T11:26:26.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.359+0000 7f47cd725640 1 --2- 172.21.15.84:0/752036160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47c815b360 0x7f47c815b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:26.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.359+0000 7f47ce727640 1 --2- 172.21.15.84:0/752036160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c8151910 0x7f47c8151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:26.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.359+0000 7f47cdf26640 1 --2- 172.21.15.84:0/752036160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b4005180 0x7f47c8149540 secure :-1 s=READY pgs=3143 cs=0 l=1 rev1=1 crypto rx=0x7f47c4007a50 tx=0x7f47c400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:26.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.359+0000 7f47be7fc640 1 -- 172.21.15.84:0/752036160 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c400edf0 con 0x7f47b4005180 2023-10-16T11:26:26.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.360+0000 7f47be7fc640 1 -- 172.21.15.84:0/752036160 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c4013420 con 0x7f47b4005180 2023-10-16T11:26:26.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.360+0000 7f47be7fc640 1 -- 172.21.15.84:0/752036160 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c40137d0 con 0x7f47b4005180 2023-10-16T11:26:26.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.360+0000 7f47d018a640 1 -- 172.21.15.84:0/752036160 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c815c3e0 con 0x7f47b4005180 2023-10-16T11:26:26.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.360+0000 7f47d018a640 1 -- 172.21.15.84:0/752036160 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c8149a80 con 0x7f47b4005180 2023-10-16T11:26:26.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.361+0000 7f47d018a640 1 -- 172.21.15.84:0/752036160 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c804e160 con 0x7f47b4005180 2023-10-16T11:26:26.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.362+0000 7f47be7fc640 1 -- 172.21.15.84:0/752036160 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f47c4013930 con 0x7f47b4005180 2023-10-16T11:26:26.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.363+0000 7f47be7fc640 1 --2- 172.21.15.84:0/752036160 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f47ac06b020 0x7f47ac06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:26.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.363+0000 7f47be7fc640 1 -- 172.21.15.84:0/752036160 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6695+0+0 (secure 0 0 0) 0x7f47c408a5d0 con 0x7f47b4005180 2023-10-16T11:26:26.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.363+0000 7f47cd725640 1 --2- 172.21.15.84:0/752036160 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f47ac06b020 0x7f47ac06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:26.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.364+0000 7f47cd725640 1 --2- 172.21.15.84:0/752036160 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f47ac06b020 0x7f47ac06d4e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f47c8146bc0 tx=0x7f47b8008040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:26.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.367+0000 7f47be7fc640 1 -- 172.21.15.84:0/752036160 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f47c4055020 con 0x7f47b4005180 2023-10-16T11:26:26.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.614+0000 7f47d018a640 1 -- 172.21.15.84:0/752036160 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47c8149c10 con 0x7f47b4005180 2023-10-16T11:26:26.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.615+0000 7f47be7fc640 1 -- 172.21.15.84:0/752036160 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+4933 (secure 0 0 0) 0x7f47c8149c10 con 0x7f47b4005180 2023-10-16T11:26:26.617 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:26.618 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":139,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2516077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2516077743},{"type":"v1","addr":"172.21.15.157:6833","nonce":2516077743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1527104462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1527104462},{"type":"v1","addr":"172.21.15.157:6835","nonce":1527104462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":12801,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3617906728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3617906728},{"type":"v1","addr":"172.21.15.84:6837","nonce":3617906728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:25:50.739234+0000","modified":"2023-10-16T11:25:53.792071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12183":{"gid":12183,"name":"c","rank":0,"incarnation":133,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/369067539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":369067539},{"type":"v1","addr":"172.21.15.84:6835","nonce":369067539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-16T11:26:26.618 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 139 2023-10-16T11:26:26.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.618+0000 7f47d018a640 1 -- 172.21.15.84:0/752036160 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f47ac06b020 msgr2=0x7f47ac06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:26.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.618+0000 7f47d018a640 1 --2- 172.21.15.84:0/752036160 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f47ac06b020 0x7f47ac06d4e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f47c8146bc0 tx=0x7f47b8008040 comp rx=0 tx=0).stop 2023-10-16T11:26:26.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.618+0000 7f47d018a640 1 -- 172.21.15.84:0/752036160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b4005180 msgr2=0x7f47c8149540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:26.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.618+0000 7f47d018a640 1 --2- 172.21.15.84:0/752036160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b4005180 0x7f47c8149540 secure :-1 s=READY pgs=3143 cs=0 l=1 rev1=1 crypto rx=0x7f47c4007a50 tx=0x7f47c400e530 comp rx=0 tx=0).stop 2023-10-16T11:26:26.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.618+0000 7f47d018a640 1 -- 172.21.15.84:0/752036160 shutdown_connections 2023-10-16T11:26:26.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.618+0000 7f47d018a640 1 --2- 172.21.15.84:0/752036160 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f47ac06b020 0x7f47ac06d4e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.618+0000 7f47d018a640 1 --2- 172.21.15.84:0/752036160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c8151910 0x7f47c8151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.618+0000 7f47d018a640 1 --2- 172.21.15.84:0/752036160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47c815b360 0x7f47c815b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.618+0000 7f47d018a640 1 --2- 172.21.15.84:0/752036160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b4005180 0x7f47c8149540 unknown :-1 s=CLOSED pgs=3143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.618+0000 7f47d018a640 1 -- 172.21.15.84:0/752036160 >> 172.21.15.84:0/752036160 conn(0x7f47c80cb410 msgr2=0x7f47c806fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:26.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.618+0000 7f47d018a640 1 -- 172.21.15.84:0/752036160 shutdown_connections 2023-10-16T11:26:26.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.618+0000 7f47d018a640 1 -- 172.21.15.84:0/752036160 wait complete. 2023-10-16T11:26:26.639 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:26:26.769 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:26.766+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:26.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.838+0000 7fe9e8168640 1 Processor -- start 2023-10-16T11:26:26.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.838+0000 7fe9e8168640 1 -- start start 2023-10-16T11:26:26.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.839+0000 7fe9e8168640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9e015a970 0x7fe9e006ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:26.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.839+0000 7fe9e8168640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9e015d430 0x7fe9e0070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:26.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.839+0000 7fe9e8168640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe9e0071360 0x7fe9e00cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:26.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.839+0000 7fe9e8168640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e0070660 con 0x7fe9e0071360 2023-10-16T11:26:26.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.839+0000 7fe9e8168640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e015dc60 con 0x7fe9e015d430 2023-10-16T11:26:26.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.839+0000 7fe9e8168640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e015df40 con 0x7fe9e015a970 2023-10-16T11:26:26.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.840+0000 7fe9e6705640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe9e0071360 0x7fe9e00cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:26.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.840+0000 7fe9e6705640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe9e0071360 0x7fe9e00cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46690/0 (socket says 172.21.15.84:46690) 2023-10-16T11:26:26.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.840+0000 7fe9e6705640 1 -- 172.21.15.84:0/2562177306 learned_addr learned my addr 172.21.15.84:0/2562177306 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:26.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.840+0000 7fe9e5f04640 1 --2- 172.21.15.84:0/2562177306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9e015a970 0x7fe9e006ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:26.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.840+0000 7fe9e5703640 1 --2- 172.21.15.84:0/2562177306 >> v2:172.21.15.157:3300/0 conn(0x7fe9e015d430 0x7fe9e0070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:26.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.840+0000 7fe9e6705640 1 -- 172.21.15.84:0/2562177306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9e015a970 msgr2=0x7fe9e006ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:26.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.840+0000 7fe9e6705640 1 --2- 172.21.15.84:0/2562177306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9e015a970 0x7fe9e006ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.840+0000 7fe9e6705640 1 -- 172.21.15.84:0/2562177306 >> v2:172.21.15.157:3300/0 conn(0x7fe9e015d430 msgr2=0x7fe9e0070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:26.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.840+0000 7fe9e6705640 1 --2- 172.21.15.84:0/2562177306 >> v2:172.21.15.157:3300/0 conn(0x7fe9e015d430 0x7fe9e0070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.840+0000 7fe9e6705640 1 -- 172.21.15.84:0/2562177306 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e015e220 con 0x7fe9e0071360 2023-10-16T11:26:26.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.840+0000 7fe9e5703640 1 --2- 172.21.15.84:0/2562177306 >> v2:172.21.15.157:3300/0 conn(0x7fe9e015d430 0x7fe9e0070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:26.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.840+0000 7fe9e5f04640 1 --2- 172.21.15.84:0/2562177306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9e015a970 0x7fe9e006ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:26.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.841+0000 7fe9e6705640 1 --2- 172.21.15.84:0/2562177306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e0071360 0x7fe9e00cd880 secure :-1 s=READY pgs=3144 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0073af0 tx=0x7fe9d802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df2e07bd116b4fe server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:26.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.842+0000 7fe9e4f02640 1 -- 172.21.15.84:0/2562177306 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d802ca70 con 0x7fe9e0071360 2023-10-16T11:26:26.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.842+0000 7fe9e4f02640 1 -- 172.21.15.84:0/2562177306 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d802cbd0 con 0x7fe9e0071360 2023-10-16T11:26:26.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.842+0000 7fe9e4f02640 1 -- 172.21.15.84:0/2562177306 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d80343f0 con 0x7fe9e0071360 2023-10-16T11:26:26.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.842+0000 7fe9e8168640 1 -- 172.21.15.84:0/2562177306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e0071360 msgr2=0x7fe9e00cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:26.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.842+0000 7fe9e8168640 1 --2- 172.21.15.84:0/2562177306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e0071360 0x7fe9e00cd880 secure :-1 s=READY pgs=3144 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0073af0 tx=0x7fe9d802bed0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.842+0000 7fe9e8168640 1 -- 172.21.15.84:0/2562177306 shutdown_connections 2023-10-16T11:26:26.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.842+0000 7fe9e8168640 1 --2- 172.21.15.84:0/2562177306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e0071360 0x7fe9e00cd880 unknown :-1 s=CLOSED pgs=3144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.842+0000 7fe9e8168640 1 --2- 172.21.15.84:0/2562177306 >> v2:172.21.15.157:3300/0 conn(0x7fe9e015d430 0x7fe9e0070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.842+0000 7fe9e8168640 1 --2- 172.21.15.84:0/2562177306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9e015a970 0x7fe9e006ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.842+0000 7fe9e8168640 1 -- 172.21.15.84:0/2562177306 >> 172.21.15.84:0/2562177306 conn(0x7fe9e00cfaa0 msgr2=0x7fe9e00cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:26.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.842+0000 7fe9e8168640 1 -- 172.21.15.84:0/2562177306 shutdown_connections 2023-10-16T11:26:26.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.842+0000 7fe9e8168640 1 -- 172.21.15.84:0/2562177306 wait complete. 2023-10-16T11:26:26.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.844+0000 7fe9e8168640 1 Processor -- start 2023-10-16T11:26:26.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.844+0000 7fe9e8168640 1 -- start start 2023-10-16T11:26:26.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.845+0000 7fe9e8168640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e015a970 0x7fe9e00b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:26.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.845+0000 7fe9e8168640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e015d430 0x7fe9e00b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:26.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.845+0000 7fe9e8168640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9e00af500 0x7fe9e00cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:26.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.845+0000 7fe9e8168640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e0158870 con 0x7fe9e015a970 2023-10-16T11:26:26.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.845+0000 7fe9e8168640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e00b2350 con 0x7fe9e015d430 2023-10-16T11:26:26.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.845+0000 7fe9e8168640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01f69e0 con 0x7fe9e00af500 2023-10-16T11:26:26.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.845+0000 7fe9e5f04640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e015a970 0x7fe9e00b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:26.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.845+0000 7fe9e5f04640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e015a970 0x7fe9e00b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46694/0 (socket says 172.21.15.84:46694) 2023-10-16T11:26:26.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.845+0000 7fe9e5f04640 1 -- 172.21.15.84:0/4180517589 learned_addr learned my addr 172.21.15.84:0/4180517589 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:26.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.846+0000 7fe9e5703640 1 --2- 172.21.15.84:0/4180517589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e015d430 0x7fe9e00b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:26.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.846+0000 7fe9e5f04640 1 -- 172.21.15.84:0/4180517589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9e00af500 msgr2=0x7fe9e00cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:26.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.846+0000 7fe9e5f04640 1 --2- 172.21.15.84:0/4180517589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9e00af500 0x7fe9e00cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.846+0000 7fe9e5f04640 1 -- 172.21.15.84:0/4180517589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e015d430 msgr2=0x7fe9e00b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:26.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.846+0000 7fe9e5f04640 1 --2- 172.21.15.84:0/4180517589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e015d430 0x7fe9e00b36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:26.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.846+0000 7fe9e5f04640 1 -- 172.21.15.84:0/4180517589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d8009b30 con 0x7fe9e015a970 2023-10-16T11:26:26.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.846+0000 7fe9e5f04640 1 --2- 172.21.15.84:0/4180517589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e015a970 0x7fe9e00b0540 secure :-1 s=READY pgs=3145 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00c850 tx=0x7fe9dc00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:26.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.846+0000 7fe9d67fc640 1 -- 172.21.15.84:0/4180517589 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc007a30 con 0x7fe9e015a970 2023-10-16T11:26:26.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.846+0000 7fe9d67fc640 1 -- 172.21.15.84:0/4180517589 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9dc007b90 con 0x7fe9e015a970 2023-10-16T11:26:26.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.846+0000 7fe9e8168640 1 -- 172.21.15.84:0/4180517589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e01f5e60 con 0x7fe9e015a970 2023-10-16T11:26:26.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.846+0000 7fe9d67fc640 1 -- 172.21.15.84:0/4180517589 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc00cea0 con 0x7fe9e015a970 2023-10-16T11:26:26.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.847+0000 7fe9e8168640 1 -- 172.21.15.84:0/4180517589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e01f6420 con 0x7fe9e015a970 2023-10-16T11:26:26.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.848+0000 7fe9e8168640 1 -- 172.21.15.84:0/4180517589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e01f25b0 con 0x7fe9e015a970 2023-10-16T11:26:26.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.849+0000 7fe9d67fc640 1 -- 172.21.15.84:0/4180517589 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fe9dc0113f0 con 0x7fe9e015a970 2023-10-16T11:26:26.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.850+0000 7fe9d67fc640 1 --2- 172.21.15.84:0/4180517589 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe9c406b050 0x7fe9c406d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:26.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.850+0000 7fe9d67fc640 1 -- 172.21.15.84:0/4180517589 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6695+0+0 (secure 0 0 0) 0x7fe9dc089f00 con 0x7fe9e015a970 2023-10-16T11:26:26.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.850+0000 7fe9e5703640 1 --2- 172.21.15.84:0/4180517589 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe9c406b050 0x7fe9c406d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:26.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.851+0000 7fe9e5703640 1 --2- 172.21.15.84:0/4180517589 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe9c406b050 0x7fe9c406d510 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe9e00af0b0 tx=0x7fe9d0008040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:26.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:26.854+0000 7fe9d67fc640 1 -- 172.21.15.84:0/4180517589 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe9dc054830 con 0x7fe9e015a970 2023-10-16T11:26:27.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.101+0000 7fe9e8168640 1 -- 172.21.15.84:0/4180517589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9e004e160 con 0x7fe9e015a970 2023-10-16T11:26:27.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.102+0000 7fe9d67fc640 1 -- 172.21.15.84:0/4180517589 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+4933 (secure 0 0 0) 0x7fe9dc01f090 con 0x7fe9e015a970 2023-10-16T11:26:27.104 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:27.105 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":139,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2516077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2516077743},{"type":"v1","addr":"172.21.15.157:6833","nonce":2516077743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1527104462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1527104462},{"type":"v1","addr":"172.21.15.157:6835","nonce":1527104462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":12801,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3617906728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3617906728},{"type":"v1","addr":"172.21.15.84:6837","nonce":3617906728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:25:50.739234+0000","modified":"2023-10-16T11:25:53.792071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12183":{"gid":12183,"name":"c","rank":0,"incarnation":133,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/369067539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":369067539},{"type":"v1","addr":"172.21.15.84:6835","nonce":369067539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-16T11:26:27.105 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 139 2023-10-16T11:26:27.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.105+0000 7fe9e8168640 1 -- 172.21.15.84:0/4180517589 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe9c406b050 msgr2=0x7fe9c406d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:27.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.105+0000 7fe9e8168640 1 --2- 172.21.15.84:0/4180517589 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe9c406b050 0x7fe9c406d510 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe9e00af0b0 tx=0x7fe9d0008040 comp rx=0 tx=0).stop 2023-10-16T11:26:27.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.105+0000 7fe9e8168640 1 -- 172.21.15.84:0/4180517589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e015a970 msgr2=0x7fe9e00b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:27.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.105+0000 7fe9e8168640 1 --2- 172.21.15.84:0/4180517589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e015a970 0x7fe9e00b0540 secure :-1 s=READY pgs=3145 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00c850 tx=0x7fe9dc00cc50 comp rx=0 tx=0).stop 2023-10-16T11:26:27.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.105+0000 7fe9e8168640 1 -- 172.21.15.84:0/4180517589 shutdown_connections 2023-10-16T11:26:27.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.105+0000 7fe9e8168640 1 --2- 172.21.15.84:0/4180517589 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe9c406b050 0x7fe9c406d510 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.105+0000 7fe9e8168640 1 --2- 172.21.15.84:0/4180517589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9e00af500 0x7fe9e00cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.105+0000 7fe9e8168640 1 --2- 172.21.15.84:0/4180517589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e015d430 0x7fe9e00b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.105+0000 7fe9e8168640 1 --2- 172.21.15.84:0/4180517589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9e015a970 0x7fe9e00b0540 unknown :-1 s=CLOSED pgs=3145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.105+0000 7fe9e8168640 1 -- 172.21.15.84:0/4180517589 >> 172.21.15.84:0/4180517589 conn(0x7fe9e00cfaa0 msgr2=0x7fe9e015aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:27.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.106+0000 7fe9e8168640 1 -- 172.21.15.84:0/4180517589 shutdown_connections 2023-10-16T11:26:27.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.106+0000 7fe9e8168640 1 -- 172.21.15.84:0/4180517589 wait complete. 2023-10-16T11:26:27.125 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:26:27.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.339+0000 7fcd40316640 1 Processor -- start 2023-10-16T11:26:27.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.339+0000 7fcd40316640 1 -- start start 2023-10-16T11:26:27.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.340+0000 7fcd40316640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd3815aaa0 0x7fcd3815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:27.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.340+0000 7fcd40316640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcd380ce390 0x7fcd38070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:27.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.340+0000 7fcd40316640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3815ed60 con 0x7fcd3815ae80 2023-10-16T11:26:27.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.340+0000 7fcd40316640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3806b810 con 0x7fcd380ce390 2023-10-16T11:26:27.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.340+0000 7fcd40316640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd380707e0 con 0x7fcd3815aaa0 2023-10-16T11:26:27.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.341+0000 7fcd3e8b3640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcd380ce390 0x7fcd38070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:27.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.341+0000 7fcd3d8b1640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fcd3815ae80 0x7fcd380cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60768/0 (socket says 172.21.15.84:60768) 2023-10-16T11:26:27.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.341+0000 7fcd3d8b1640 1 -- 172.21.15.84:0/1336710458 learned_addr learned my addr 172.21.15.84:0/1336710458 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:27.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.341+0000 7fcd3e0b2640 1 --2- 172.21.15.84:0/1336710458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd3815aaa0 0x7fcd3815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:27.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.341+0000 7fcd3e8b3640 1 -- 172.21.15.84:0/1336710458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd3815aaa0 msgr2=0x7fcd3815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:27.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.341+0000 7fcd3e8b3640 1 --2- 172.21.15.84:0/1336710458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd3815aaa0 0x7fcd3815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.341+0000 7fcd3e8b3640 1 -- 172.21.15.84:0/1336710458 >> v1:172.21.15.84:6789/0 conn(0x7fcd3815ae80 legacy=0x7fcd380cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:27.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.341+0000 7fcd3e8b3640 1 -- 172.21.15.84:0/1336710458 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3815ef90 con 0x7fcd380ce390 2023-10-16T11:26:27.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.341+0000 7fcd3e0b2640 1 --2- 172.21.15.84:0/1336710458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd3815aaa0 0x7fcd3815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:27.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.342+0000 7fcd3e8b3640 1 --2- 172.21.15.84:0/1336710458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd380ce390 0x7fcd38070f50 secure :-1 s=READY pgs=3147 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009d90 tx=0x7fcd3002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7006d2345f8b5bd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:27.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.343+0000 7fcd3d0b0640 1 -- 172.21.15.84:0/1336710458 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3002cc60 con 0x7fcd380ce390 2023-10-16T11:26:27.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.343+0000 7fcd3d0b0640 1 -- 172.21.15.84:0/1336710458 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3002cdc0 con 0x7fcd380ce390 2023-10-16T11:26:27.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.343+0000 7fcd3d0b0640 1 -- 172.21.15.84:0/1336710458 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd30034660 con 0x7fcd380ce390 2023-10-16T11:26:27.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.343+0000 7fcd40316640 1 -- 172.21.15.84:0/1336710458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd380ce390 msgr2=0x7fcd38070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:27.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.343+0000 7fcd40316640 1 --2- 172.21.15.84:0/1336710458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd380ce390 0x7fcd38070f50 secure :-1 s=READY pgs=3147 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009d90 tx=0x7fcd3002beb0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.343+0000 7fcd40316640 1 -- 172.21.15.84:0/1336710458 shutdown_connections 2023-10-16T11:26:27.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.343+0000 7fcd40316640 1 --2- 172.21.15.84:0/1336710458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd380ce390 0x7fcd38070f50 unknown :-1 s=CLOSED pgs=3147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.343+0000 7fcd40316640 1 --2- 172.21.15.84:0/1336710458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd3815aaa0 0x7fcd3815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.343+0000 7fcd40316640 1 -- 172.21.15.84:0/1336710458 >> 172.21.15.84:0/1336710458 conn(0x7fcd380cb410 msgr2=0x7fcd380cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:27.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.343+0000 7fcd40316640 1 -- 172.21.15.84:0/1336710458 shutdown_connections 2023-10-16T11:26:27.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.343+0000 7fcd40316640 1 -- 172.21.15.84:0/1336710458 wait complete. 2023-10-16T11:26:27.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.345+0000 7fcd40316640 1 Processor -- start 2023-10-16T11:26:27.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.345+0000 7fcd40316640 1 -- start start 2023-10-16T11:26:27.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.346+0000 7fcd40316640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd380ce390 0x7fcd3814f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:27.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.346+0000 7fcd40316640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd3815aaa0 0x7fcd380afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:27.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.346+0000 7fcd40316640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd3815ae80 0x7fcd380c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:27.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.346+0000 7fcd40316640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3814f600 con 0x7fcd3815aaa0 2023-10-16T11:26:27.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.346+0000 7fcd40316640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd380ae520 con 0x7fcd380ce390 2023-10-16T11:26:27.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.346+0000 7fcd40316640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd380b02a0 con 0x7fcd3815ae80 2023-10-16T11:26:27.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.346+0000 7fcd3d8b1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd3815aaa0 0x7fcd380afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:27.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.346+0000 7fcd3d8b1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd3815aaa0 0x7fcd380afd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46710/0 (socket says 172.21.15.84:46710) 2023-10-16T11:26:27.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.346+0000 7fcd3d8b1640 1 -- 172.21.15.84:0/1690820801 learned_addr learned my addr 172.21.15.84:0/1690820801 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:27.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.347+0000 7fcd3d8b1640 1 -- 172.21.15.84:0/1690820801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd3815ae80 msgr2=0x7fcd380c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:27.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.347+0000 7fcd3d8b1640 1 --2- 172.21.15.84:0/1690820801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd3815ae80 0x7fcd380c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.347+0000 7fcd3d8b1640 1 -- 172.21.15.84:0/1690820801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd380ce390 msgr2=0x7fcd3814f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:27.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.347+0000 7fcd3d8b1640 1 --2- 172.21.15.84:0/1690820801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd380ce390 0x7fcd3814f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.347+0000 7fcd3d8b1640 1 -- 172.21.15.84:0/1690820801 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd30009a70 con 0x7fcd3815aaa0 2023-10-16T11:26:27.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.347+0000 7fcd3d8b1640 1 --2- 172.21.15.84:0/1690820801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd3815aaa0 0x7fcd380afd60 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800c4a0 tx=0x7fcd2800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:27.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.347+0000 7fcd2effd640 1 -- 172.21.15.84:0/1690820801 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd28005ce0 con 0x7fcd3815aaa0 2023-10-16T11:26:27.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.347+0000 7fcd2effd640 1 -- 172.21.15.84:0/1690820801 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd28005e40 con 0x7fcd3815aaa0 2023-10-16T11:26:27.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.347+0000 7fcd40316640 1 -- 172.21.15.84:0/1690820801 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3815d820 con 0x7fcd3815aaa0 2023-10-16T11:26:27.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.347+0000 7fcd2effd640 1 -- 172.21.15.84:0/1690820801 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd28007270 con 0x7fcd3815aaa0 2023-10-16T11:26:27.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.347+0000 7fcd40316640 1 -- 172.21.15.84:0/1690820801 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3815dde0 con 0x7fcd3815aaa0 2023-10-16T11:26:27.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.348+0000 7fcd40316640 1 -- 172.21.15.84:0/1690820801 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd3804e160 con 0x7fcd3815aaa0 2023-10-16T11:26:27.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.349+0000 7fcd2effd640 1 -- 172.21.15.84:0/1690820801 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fcd28007490 con 0x7fcd3815aaa0 2023-10-16T11:26:27.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.350+0000 7fcd2effd640 1 --2- 172.21.15.84:0/1690820801 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fcd1c06b020 0x7fcd1c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:27.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.351+0000 7fcd2effd640 1 -- 172.21.15.84:0/1690820801 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6695+0+0 (secure 0 0 0) 0x7fcd28089120 con 0x7fcd3815aaa0 2023-10-16T11:26:27.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.351+0000 7fcd3e0b2640 1 --2- 172.21.15.84:0/1690820801 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fcd1c06b020 0x7fcd1c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:27.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.351+0000 7fcd3e0b2640 1 --2- 172.21.15.84:0/1690820801 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fcd1c06b020 0x7fcd1c06d4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcd34006fd0 tx=0x7fcd34008040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:27.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.354+0000 7fcd2effd640 1 -- 172.21.15.84:0/1690820801 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcd28053c20 con 0x7fcd3815aaa0 2023-10-16T11:26:27.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.550+0000 7fcd40316640 1 -- 172.21.15.84:0/1690820801 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd380ae690 con 0x7fcd3815aaa0 2023-10-16T11:26:27.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.553+0000 7fcd2effd640 1 -- 172.21.15.84:0/1690820801 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v246) v1 ==== 74+0+17928 (secure 0 0 0) 0x7fcd380ae690 con 0x7fcd3815aaa0 2023-10-16T11:26:27.556 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:27.556 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":246,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:26:23.239524+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:25:49.001107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:25:49.760637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6837/3547503033":"2023-10-17T11:26:20.041719+0000","172.21.15.84:0/2606713455":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/488945617":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/4036998934":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6832/3938045451":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/570042115":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1311117536":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6833/2963529201":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/95207886":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6832/2963529201":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/388205040":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/2303342316":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/648102622":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/133211355":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1263886961":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1489017743":"2023-10-17T11:26:08.077644+0000","172.21.15.84:6833/3938045451":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1517291402":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1983743360":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1323010124":"2023-10-17T11:26:08.077644+0000","172.21.15.84:6836/3547503033":"2023-10-17T11:26:20.041719+0000","172.21.15.84:0/437386817":"2023-10-17T11:26:08.077644+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:26:27.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.556+0000 7fcd40316640 1 -- 172.21.15.84:0/1690820801 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fcd1c06b020 msgr2=0x7fcd1c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:27.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.556+0000 7fcd40316640 1 --2- 172.21.15.84:0/1690820801 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fcd1c06b020 0x7fcd1c06d4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcd34006fd0 tx=0x7fcd34008040 comp rx=0 tx=0).stop 2023-10-16T11:26:27.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.556+0000 7fcd40316640 1 -- 172.21.15.84:0/1690820801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd3815aaa0 msgr2=0x7fcd380afd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:27.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.556+0000 7fcd40316640 1 --2- 172.21.15.84:0/1690820801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd3815aaa0 0x7fcd380afd60 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800c4a0 tx=0x7fcd2800c960 comp rx=0 tx=0).stop 2023-10-16T11:26:27.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.556+0000 7fcd40316640 1 -- 172.21.15.84:0/1690820801 shutdown_connections 2023-10-16T11:26:27.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.556+0000 7fcd40316640 1 --2- 172.21.15.84:0/1690820801 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fcd1c06b020 0x7fcd1c06d4e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.556+0000 7fcd40316640 1 --2- 172.21.15.84:0/1690820801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd3815ae80 0x7fcd380c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.556+0000 7fcd40316640 1 --2- 172.21.15.84:0/1690820801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd3815aaa0 0x7fcd380afd60 unknown :-1 s=CLOSED pgs=3148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.556+0000 7fcd40316640 1 --2- 172.21.15.84:0/1690820801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd380ce390 0x7fcd3814f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.556+0000 7fcd40316640 1 -- 172.21.15.84:0/1690820801 >> 172.21.15.84:0/1690820801 conn(0x7fcd380cb410 msgr2=0x7fcd3806d260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:27.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.556+0000 7fcd40316640 1 -- 172.21.15.84:0/1690820801 shutdown_connections 2023-10-16T11:26:27.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.556+0000 7fcd40316640 1 -- 172.21.15.84:0/1690820801 wait complete. 2023-10-16T11:26:27.579 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2023-10-16T11:26:27.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.788+0000 7fe563fff640 1 Processor -- start 2023-10-16T11:26:27.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.788+0000 7fe563fff640 1 -- start start 2023-10-16T11:26:27.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.790+0000 7fe563fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe5640ad970 0x7fe5640cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:27.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.790+0000 7fe563fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe5640ccfb0 0x7fe5640cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:27.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.790+0000 7fe563fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe56415b9d0 con 0x7fe56406c040 2023-10-16T11:26:27.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.790+0000 7fe563fff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe56415bcb0 con 0x7fe5640ad970 2023-10-16T11:26:27.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.790+0000 7fe563fff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe56415bf90 con 0x7fe5640ccfb0 2023-10-16T11:26:27.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.790+0000 7fe55bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe5640ad970 0x7fe5640cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:27.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.790+0000 7fe55bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe5640ad970 0x7fe5640cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46712/0 (socket says 172.21.15.84:46712) 2023-10-16T11:26:27.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.790+0000 7fe562ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe56406c040 0x7fe56406c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60774/0 (socket says 172.21.15.84:60774) 2023-10-16T11:26:27.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.790+0000 7fe55bfff640 1 -- 172.21.15.84:0/1336584344 learned_addr learned my addr 172.21.15.84:0/1336584344 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:27.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.791+0000 7fe5637fe640 1 --2- 172.21.15.84:0/1336584344 >> v2:172.21.15.157:3300/0 conn(0x7fe5640ccfb0 0x7fe5640cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:27.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.791+0000 7fe55bfff640 1 -- 172.21.15.84:0/1336584344 >> v2:172.21.15.157:3300/0 conn(0x7fe5640ccfb0 msgr2=0x7fe5640cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:27.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.791+0000 7fe55bfff640 1 --2- 172.21.15.84:0/1336584344 >> v2:172.21.15.157:3300/0 conn(0x7fe5640ccfb0 0x7fe5640cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.791+0000 7fe55bfff640 1 -- 172.21.15.84:0/1336584344 >> v1:172.21.15.84:6789/0 conn(0x7fe56406c040 legacy=0x7fe56406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:27.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.791+0000 7fe55bfff640 1 -- 172.21.15.84:0/1336584344 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5640cc270 con 0x7fe5640ad970 2023-10-16T11:26:27.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.792+0000 7fe55bfff640 1 --2- 172.21.15.84:0/1336584344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5640ad970 0x7fe5640cca70 secure :-1 s=READY pgs=3150 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009870 tx=0x7fe54c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6017cfc5f6724f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:27.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.792+0000 7fe5627fc640 1 -- 172.21.15.84:0/1336584344 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c030860 con 0x7fe5640ad970 2023-10-16T11:26:27.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.792+0000 7fe5627fc640 1 -- 172.21.15.84:0/1336584344 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c0309c0 con 0x7fe5640ad970 2023-10-16T11:26:27.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.792+0000 7fe5627fc640 1 -- 172.21.15.84:0/1336584344 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c030d50 con 0x7fe5640ad970 2023-10-16T11:26:27.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.793+0000 7fe563fff640 1 -- 172.21.15.84:0/1336584344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5640ad970 msgr2=0x7fe5640cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:27.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.793+0000 7fe563fff640 1 --2- 172.21.15.84:0/1336584344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5640ad970 0x7fe5640cca70 secure :-1 s=READY pgs=3150 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009870 tx=0x7fe54c02b9a0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.793+0000 7fe563fff640 1 -- 172.21.15.84:0/1336584344 shutdown_connections 2023-10-16T11:26:27.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.793+0000 7fe563fff640 1 --2- 172.21.15.84:0/1336584344 >> v2:172.21.15.157:3300/0 conn(0x7fe5640ccfb0 0x7fe5640cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.793+0000 7fe563fff640 1 --2- 172.21.15.84:0/1336584344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5640ad970 0x7fe5640cca70 unknown :-1 s=CLOSED pgs=3150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.793+0000 7fe563fff640 1 -- 172.21.15.84:0/1336584344 >> 172.21.15.84:0/1336584344 conn(0x7fe5640ce6a0 msgr2=0x7fe5640ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:27.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.793+0000 7fe563fff640 1 -- 172.21.15.84:0/1336584344 shutdown_connections 2023-10-16T11:26:27.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.793+0000 7fe563fff640 1 -- 172.21.15.84:0/1336584344 wait complete. 2023-10-16T11:26:27.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.795+0000 7fe563fff640 1 Processor -- start 2023-10-16T11:26:27.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.795+0000 7fe563fff640 1 -- start start 2023-10-16T11:26:27.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.796+0000 7fe563fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe56406c040 0x7fe564153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:27.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.796+0000 7fe563fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5640ad970 0x7fe5640ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:27.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.796+0000 7fe563fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5640ccfb0 0x7fe5640acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:27.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.796+0000 7fe563fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5641540d0 con 0x7fe5640ad970 2023-10-16T11:26:27.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.796+0000 7fe563fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5640b1330 con 0x7fe5640ccfb0 2023-10-16T11:26:27.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.796+0000 7fe563fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5640b1610 con 0x7fe56406c040 2023-10-16T11:26:27.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.796+0000 7fe55bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5640ad970 0x7fe5640ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:27.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.796+0000 7fe55bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5640ad970 0x7fe5640ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46724/0 (socket says 172.21.15.84:46724) 2023-10-16T11:26:27.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.796+0000 7fe55bfff640 1 -- 172.21.15.84:0/2289406415 learned_addr learned my addr 172.21.15.84:0/2289406415 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:27.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.797+0000 7fe55bfff640 1 -- 172.21.15.84:0/2289406415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe56406c040 msgr2=0x7fe564153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:27.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.797+0000 7fe55bfff640 1 --2- 172.21.15.84:0/2289406415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe56406c040 0x7fe564153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.797+0000 7fe55bfff640 1 -- 172.21.15.84:0/2289406415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5640ccfb0 msgr2=0x7fe5640acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:27.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.797+0000 7fe55bfff640 1 --2- 172.21.15.84:0/2289406415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5640ccfb0 0x7fe5640acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:27.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.797+0000 7fe55bfff640 1 -- 172.21.15.84:0/2289406415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c0094d0 con 0x7fe5640ad970 2023-10-16T11:26:27.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.797+0000 7fe55bfff640 1 --2- 172.21.15.84:0/2289406415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5640ad970 0x7fe5640ac890 secure :-1 s=READY pgs=3151 cs=0 l=1 rev1=1 crypto rx=0x7fe54c000c00 tx=0x7fe54c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:27.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.797+0000 7fe55b7fe640 1 -- 172.21.15.84:0/2289406415 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c039c90 con 0x7fe5640ad970 2023-10-16T11:26:27.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.797+0000 7fe55b7fe640 1 -- 172.21.15.84:0/2289406415 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c039df0 con 0x7fe5640ad970 2023-10-16T11:26:27.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.797+0000 7fe55b7fe640 1 -- 172.21.15.84:0/2289406415 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c043640 con 0x7fe5640ad970 2023-10-16T11:26:27.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.797+0000 7fe563fff640 1 -- 172.21.15.84:0/2289406415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5640b18f0 con 0x7fe5640ad970 2023-10-16T11:26:27.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.797+0000 7fe563fff640 1 -- 172.21.15.84:0/2289406415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5640b1eb0 con 0x7fe5640ad970 2023-10-16T11:26:27.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.799+0000 7fe563fff640 1 -- 172.21.15.84:0/2289406415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe56406b330 con 0x7fe5640ad970 2023-10-16T11:26:27.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.800+0000 7fe55b7fe640 1 -- 172.21.15.84:0/2289406415 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fe54c0437a0 con 0x7fe5640ad970 2023-10-16T11:26:27.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.801+0000 7fe55b7fe640 1 --2- 172.21.15.84:0/2289406415 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe54406b020 0x7fe54406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:27.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.801+0000 7fe55b7fe640 1 -- 172.21.15.84:0/2289406415 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6695+0+0 (secure 0 0 0) 0x7fe54c03e070 con 0x7fe5640ad970 2023-10-16T11:26:27.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.801+0000 7fe562ffd640 1 --2- 172.21.15.84:0/2289406415 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe54406b020 0x7fe54406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:27.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.802+0000 7fe562ffd640 1 --2- 172.21.15.84:0/2289406415 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe54406b020 0x7fe54406d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe5540015e0 tx=0x7fe5540079c0 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:27.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:27.805+0000 7fe55b7fe640 1 -- 172.21.15.84:0/2289406415 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe54c077900 con 0x7fe5640ad970 2023-10-16T11:26:27.906 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:27.903+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:28.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.043+0000 7fe563fff640 1 -- 172.21.15.84:0/2289406415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) v1 -- 0x7fe56404e9b0 con 0x7fe5640ad970 2023-10-16T11:26:28.046 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-16T11:26:28.044+0000 7fca0c7ff640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:26:28.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.048+0000 7fe55b7fe640 1 -- 172.21.15.84:0/2289406415 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v140) v1 ==== 167+0+0 (secure 0 0 0) 0x7fe54c07af20 con 0x7fe5640ad970 2023-10-16T11:26:28.051 INFO:teuthology.orchestra.run.smithi084.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:26:28.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.052+0000 7fe563fff640 1 -- 172.21.15.84:0/2289406415 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe54406b020 msgr2=0x7fe54406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:28.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.052+0000 7fe563fff640 1 --2- 172.21.15.84:0/2289406415 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe54406b020 0x7fe54406d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe5540015e0 tx=0x7fe5540079c0 comp rx=0 tx=0).stop 2023-10-16T11:26:28.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.052+0000 7fe563fff640 1 -- 172.21.15.84:0/2289406415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5640ad970 msgr2=0x7fe5640ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:28.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.052+0000 7fe563fff640 1 --2- 172.21.15.84:0/2289406415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5640ad970 0x7fe5640ac890 secure :-1 s=READY pgs=3151 cs=0 l=1 rev1=1 crypto rx=0x7fe54c000c00 tx=0x7fe54c004730 comp rx=0 tx=0).stop 2023-10-16T11:26:28.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.052+0000 7fe563fff640 1 -- 172.21.15.84:0/2289406415 shutdown_connections 2023-10-16T11:26:28.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.052+0000 7fe563fff640 1 --2- 172.21.15.84:0/2289406415 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe54406b020 0x7fe54406d4e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:28.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.052+0000 7fe563fff640 1 --2- 172.21.15.84:0/2289406415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5640ccfb0 0x7fe5640acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:28.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.052+0000 7fe563fff640 1 --2- 172.21.15.84:0/2289406415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5640ad970 0x7fe5640ac890 unknown :-1 s=CLOSED pgs=3151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:28.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.052+0000 7fe563fff640 1 --2- 172.21.15.84:0/2289406415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe56406c040 0x7fe564153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:28.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.052+0000 7fe563fff640 1 -- 172.21.15.84:0/2289406415 >> 172.21.15.84:0/2289406415 conn(0x7fe5640ce6a0 msgr2=0x7fe5640c5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:28.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.052+0000 7fe563fff640 1 -- 172.21.15.84:0/2289406415 shutdown_connections 2023-10-16T11:26:28.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.053+0000 7fe563fff640 1 -- 172.21.15.84:0/2289406415 wait complete. 2023-10-16T11:26:28.064 INFO:tasks.ceph.mds.c.smithi084.stderr: -1800> 2023-10-16T11:25:57.103+0000 7ff2e8d30640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:26:28.075 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2023-10-16T11:26:28.126 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T11:26:28.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.270+0000 7f45567dc640 1 Processor -- start 2023-10-16T11:26:28.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.271+0000 7f45567dc640 1 -- start start 2023-10-16T11:26:28.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.272+0000 7f45567dc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45500b3420 0x7f45500b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:28.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.272+0000 7f45567dc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f45500cfaf0 0x7f45500b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:28.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.272+0000 7f45567dc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45500d3980 con 0x7f45500cf710 2023-10-16T11:26:28.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.272+0000 7f45567dc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45500d3c60 con 0x7f45500cfaf0 2023-10-16T11:26:28.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.272+0000 7f45567dc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45500ce630 con 0x7f45500b3420 2023-10-16T11:26:28.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.273+0000 7f4554d79640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f45500cfaf0 0x7f45500b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:28.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.273+0000 7f4547fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45500b3420 0x7f45500b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:28.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.273+0000 7f4554d79640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f45500cfaf0 0x7f45500b6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:38818/0 (socket says 172.21.15.84:38818) 2023-10-16T11:26:28.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.273+0000 7f4554d79640 1 -- 172.21.15.84:0/3820848853 learned_addr learned my addr 172.21.15.84:0/3820848853 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:28.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.273+0000 7f4547fff640 1 -- 172.21.15.84:0/3820848853 >> v2:172.21.15.157:3300/0 conn(0x7f45500cfaf0 msgr2=0x7f45500b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:28.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.273+0000 7f4547fff640 1 --2- 172.21.15.84:0/3820848853 >> v2:172.21.15.157:3300/0 conn(0x7f45500cfaf0 0x7f45500b6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:28.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.273+0000 7f4547fff640 1 -- 172.21.15.84:0/3820848853 >> v1:172.21.15.157:6789/0 conn(0x7f45500cf710 legacy=0x7f45500b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:28.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.273+0000 7f4547fff640 1 -- 172.21.15.84:0/3820848853 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45500ce910 con 0x7f45500b3420 2023-10-16T11:26:28.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.273+0000 7f4554d79640 1 --2- 172.21.15.84:0/3820848853 >> v2:172.21.15.157:3300/0 conn(0x7f45500cfaf0 0x7f45500b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:26:28.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.275+0000 7f4547fff640 1 --2- 172.21.15.84:0/3820848853 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45500b3420 0x7f45500b5cc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f45380097a0 tx=0x7f453802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c47410bc2e2608a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:28.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.275+0000 7f454f7fe640 1 -- 172.21.15.84:0/3820848853 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538031780 con 0x7f45500b3420 2023-10-16T11:26:28.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.275+0000 7f454f7fe640 1 -- 172.21.15.84:0/3820848853 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45380318e0 con 0x7f45500b3420 2023-10-16T11:26:28.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.275+0000 7f454f7fe640 1 -- 172.21.15.84:0/3820848853 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538031c70 con 0x7f45500b3420 2023-10-16T11:26:28.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.276+0000 7f45567dc640 1 -- 172.21.15.84:0/3820848853 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45500b3420 msgr2=0x7f45500b5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:28.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.276+0000 7f45567dc640 1 --2- 172.21.15.84:0/3820848853 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45500b3420 0x7f45500b5cc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f45380097a0 tx=0x7f453802b820 comp rx=0 tx=0).stop 2023-10-16T11:26:28.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.276+0000 7f45567dc640 1 -- 172.21.15.84:0/3820848853 shutdown_connections 2023-10-16T11:26:28.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.276+0000 7f45567dc640 1 --2- 172.21.15.84:0/3820848853 >> v2:172.21.15.157:3300/0 conn(0x7f45500cfaf0 0x7f45500b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:28.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.276+0000 7f45567dc640 1 --2- 172.21.15.84:0/3820848853 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45500b3420 0x7f45500b5cc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:28.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.276+0000 7f45567dc640 1 -- 172.21.15.84:0/3820848853 >> 172.21.15.84:0/3820848853 conn(0x7f45500d2cc0 msgr2=0x7f45500d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:28.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.276+0000 7f45567dc640 1 -- 172.21.15.84:0/3820848853 shutdown_connections 2023-10-16T11:26:28.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.276+0000 7f45567dc640 1 -- 172.21.15.84:0/3820848853 wait complete. 2023-10-16T11:26:28.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.278+0000 7f45567dc640 1 Processor -- start 2023-10-16T11:26:28.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.278+0000 7f45567dc640 1 -- start start 2023-10-16T11:26:28.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.279+0000 7f45567dc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45500b3420 0x7f455014fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:28.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.279+0000 7f45567dc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45500cf710 0x7f455015a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:28.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.279+0000 7f45567dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45500cfaf0 0x7f45500b7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:28.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.279+0000 7f45567dc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455015d6d0 con 0x7f45500b3420 2023-10-16T11:26:28.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.279+0000 7f45567dc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455015cac0 con 0x7f45500cfaf0 2023-10-16T11:26:28.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.279+0000 7f45567dc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455015cda0 con 0x7f45500cf710 2023-10-16T11:26:28.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.279+0000 7f454ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45500b3420 0x7f455014fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:28.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.279+0000 7f454ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45500b3420 0x7f455014fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46748/0 (socket says 172.21.15.84:46748) 2023-10-16T11:26:28.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.279+0000 7f454ffff640 1 -- 172.21.15.84:0/1255731565 learned_addr learned my addr 172.21.15.84:0/1255731565 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:28.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.280+0000 7f4554d79640 1 --2- 172.21.15.84:0/1255731565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45500cfaf0 0x7f45500b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:28.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.280+0000 7f454ffff640 1 -- 172.21.15.84:0/1255731565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45500cf710 msgr2=0x7f455015a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:28.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.280+0000 7f454ffff640 1 --2- 172.21.15.84:0/1255731565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45500cf710 0x7f455015a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:28.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.280+0000 7f454ffff640 1 -- 172.21.15.84:0/1255731565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45500cfaf0 msgr2=0x7f45500b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:28.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.280+0000 7f454ffff640 1 --2- 172.21.15.84:0/1255731565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45500cfaf0 0x7f45500b7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:28.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.280+0000 7f454ffff640 1 -- 172.21.15.84:0/1255731565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4548009b30 con 0x7f45500b3420 2023-10-16T11:26:28.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.280+0000 7f454ffff640 1 --2- 172.21.15.84:0/1255731565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45500b3420 0x7f455014fe70 secure :-1 s=READY pgs=3154 cs=0 l=1 rev1=1 crypto rx=0x7f454000c450 tx=0x7f454000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:28.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.280+0000 7f454d7fa640 1 -- 172.21.15.84:0/1255731565 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45400052d0 con 0x7f45500b3420 2023-10-16T11:26:28.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.280+0000 7f454d7fa640 1 -- 172.21.15.84:0/1255731565 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4540005450 con 0x7f45500b3420 2023-10-16T11:26:28.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.280+0000 7f45567dc640 1 -- 172.21.15.84:0/1255731565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4538009400 con 0x7f45500b3420 2023-10-16T11:26:28.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.280+0000 7f454d7fa640 1 -- 172.21.15.84:0/1255731565 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4540003880 con 0x7f45500b3420 2023-10-16T11:26:28.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.280+0000 7f45567dc640 1 -- 172.21.15.84:0/1255731565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455015d360 con 0x7f45500b3420 2023-10-16T11:26:28.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.281+0000 7f45567dc640 1 -- 172.21.15.84:0/1255731565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f455004e160 con 0x7f45500b3420 2023-10-16T11:26:28.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.282+0000 7f454d7fa640 1 -- 172.21.15.84:0/1255731565 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f45400039e0 con 0x7f45500b3420 2023-10-16T11:26:28.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.283+0000 7f454d7fa640 1 --2- 172.21.15.84:0/1255731565 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f453006b020 0x7f453006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:28.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.283+0000 7f454d7fa640 1 -- 172.21.15.84:0/1255731565 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f4540089a20 con 0x7f45500b3420 2023-10-16T11:26:28.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.284+0000 7f4547fff640 1 --2- 172.21.15.84:0/1255731565 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f453006b020 0x7f453006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:28.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.284+0000 7f4547fff640 1 --2- 172.21.15.84:0/1255731565 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f453006b020 0x7f453006d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4538036000 tx=0x7f4538030000 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:28.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.288+0000 7f454d7fa640 1 -- 172.21.15.84:0/1255731565 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f45400542f0 con 0x7f45500b3420 2023-10-16T11:26:28.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:28.524+0000 7f45567dc640 1 -- 172.21.15.84:0/1255731565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f45500c9200 con 0x7f45500b3420 2023-10-16T11:26:29.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.051+0000 7f454d7fa640 1 -- 172.21.15.84:0/1255731565 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v141) v1 ==== 107+0+0 (secure 0 0 0) 0x7f4540057910 con 0x7f45500b3420 2023-10-16T11:26:29.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.055+0000 7f45567dc640 1 -- 172.21.15.84:0/1255731565 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f453006b020 msgr2=0x7f453006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:29.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.055+0000 7f45567dc640 1 --2- 172.21.15.84:0/1255731565 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f453006b020 0x7f453006d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4538036000 tx=0x7f4538030000 comp rx=0 tx=0).stop 2023-10-16T11:26:29.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.055+0000 7f45567dc640 1 -- 172.21.15.84:0/1255731565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45500b3420 msgr2=0x7f455014fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:29.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.055+0000 7f45567dc640 1 --2- 172.21.15.84:0/1255731565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45500b3420 0x7f455014fe70 secure :-1 s=READY pgs=3154 cs=0 l=1 rev1=1 crypto rx=0x7f454000c450 tx=0x7f454000c910 comp rx=0 tx=0).stop 2023-10-16T11:26:29.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.055+0000 7f45567dc640 1 -- 172.21.15.84:0/1255731565 shutdown_connections 2023-10-16T11:26:29.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.055+0000 7f45567dc640 1 --2- 172.21.15.84:0/1255731565 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f453006b020 0x7f453006d4e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:29.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.055+0000 7f45567dc640 1 --2- 172.21.15.84:0/1255731565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45500cfaf0 0x7f45500b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:29.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.055+0000 7f45567dc640 1 --2- 172.21.15.84:0/1255731565 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45500cf710 0x7f455015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:29.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.055+0000 7f45567dc640 1 --2- 172.21.15.84:0/1255731565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45500b3420 0x7f455014fe70 unknown :-1 s=CLOSED pgs=3154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:29.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.055+0000 7f45567dc640 1 -- 172.21.15.84:0/1255731565 >> 172.21.15.84:0/1255731565 conn(0x7f45500d2cc0 msgr2=0x7f45500cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:29.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.056+0000 7f45567dc640 1 -- 172.21.15.84:0/1255731565 shutdown_connections 2023-10-16T11:26:29.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.056+0000 7f45567dc640 1 -- 172.21.15.84:0/1255731565 wait complete. 2023-10-16T11:26:29.078 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2023-10-16T11:26:29.135 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:29.132+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:29.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.284+0000 7f1e98133640 1 Processor -- start 2023-10-16T11:26:29.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.284+0000 7f1e98133640 1 -- start start 2023-10-16T11:26:29.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.285+0000 7f1e98133640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e9006afc0 0x7f1e900c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:29.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.285+0000 7f1e98133640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e901596c0 con 0x7f1e901592e0 2023-10-16T11:26:29.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.285+0000 7f1e98133640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9006c6c0 con 0x7f1e900c9470 2023-10-16T11:26:29.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.285+0000 7f1e98133640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e900c9900 con 0x7f1e9006afc0 2023-10-16T11:26:29.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.286+0000 7f1e95ecf640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1e901592e0 0x7f1e9006bea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60776/0 (socket says 172.21.15.84:60776) 2023-10-16T11:26:29.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.286+0000 7f1e95ecf640 1 -- 172.21.15.84:0/1548538191 learned_addr learned my addr 172.21.15.84:0/1548538191 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:29.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.286+0000 7f1e956ce640 1 --2- 172.21.15.84:0/1548538191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e9006afc0 0x7f1e900c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:29.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.286+0000 7f1e94ecd640 1 -- 172.21.15.84:0/1548538191 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557888946 0 0) 0x7f1e901596c0 con 0x7f1e901592e0 2023-10-16T11:26:29.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.286+0000 7f1e94ecd640 1 -- 172.21.15.84:0/1548538191 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e7c003520 con 0x7f1e901592e0 2023-10-16T11:26:29.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.287+0000 7f1e94ecd640 1 -- 172.21.15.84:0/1548538191 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1311120274 0 0) 0x7f1e7c003520 con 0x7f1e901592e0 2023-10-16T11:26:29.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.287+0000 7f1e94ecd640 1 -- 172.21.15.84:0/1548538191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e9006afc0 msgr2=0x7f1e900c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:29.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.287+0000 7f1e94ecd640 1 --2- 172.21.15.84:0/1548538191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e9006afc0 0x7f1e900c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:29.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.287+0000 7f1e94ecd640 1 -- 172.21.15.84:0/1548538191 >> v1:172.21.15.157:6789/0 conn(0x7f1e900c9470 legacy=0x7f1e900cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:29.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.287+0000 7f1e94ecd640 1 -- 172.21.15.84:0/1548538191 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e900cc860 con 0x7f1e901592e0 2023-10-16T11:26:29.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.287+0000 7f1e94ecd640 1 -- 172.21.15.84:0/1548538191 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f1e8c001670 con 0x7f1e901592e0 2023-10-16T11:26:29.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.287+0000 7f1e94ecd640 1 -- 172.21.15.84:0/1548538191 >> v1:172.21.15.84:6789/0 conn(0x7f1e901592e0 legacy=0x7f1e9006bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:29.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.287+0000 7f1e94ecd640 1 --2- 172.21.15.84:0/1548538191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e7c003a10 0x7f1e7c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:29.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.287+0000 7f1e94ecd640 1 --2- 172.21.15.84:0/1548538191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e7c0047c0 0x7f1e7c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:29.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.287+0000 7f1e94ecd640 1 --2- 172.21.15.84:0/1548538191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e9006afc0 0x7f1e7c00ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:29.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.287+0000 7f1e94ecd640 1 -- 172.21.15.84:0/1548538191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e900c9900 con 0x7f1e7c003a10 2023-10-16T11:26:29.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.287+0000 7f1e94ecd640 1 -- 172.21.15.84:0/1548538191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c0054e0 con 0x7f1e7c0047c0 2023-10-16T11:26:29.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.287+0000 7f1e94ecd640 1 -- 172.21.15.84:0/1548538191 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c00cfd0 con 0x7f1e9006afc0 2023-10-16T11:26:29.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.287+0000 7f1e95ecf640 1 --2- 172.21.15.84:0/1548538191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e7c003a10 0x7f1e7c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:29.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.287+0000 7f1e956ce640 1 --2- 172.21.15.84:0/1548538191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e7c0047c0 0x7f1e7c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:29.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.287+0000 7f1e95ecf640 1 -- 172.21.15.84:0/1548538191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e9006afc0 msgr2=0x7f1e7c00ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:29.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.287+0000 7f1e95ecf640 1 --2- 172.21.15.84:0/1548538191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e9006afc0 0x7f1e7c00ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:29.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.287+0000 7f1e95ecf640 1 -- 172.21.15.84:0/1548538191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e7c0047c0 msgr2=0x7f1e7c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:29.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.288+0000 7f1e95ecf640 1 --2- 172.21.15.84:0/1548538191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e7c0047c0 0x7f1e7c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:29.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.288+0000 7f1e95ecf640 1 -- 172.21.15.84:0/1548538191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c003520 con 0x7f1e7c003a10 2023-10-16T11:26:29.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.288+0000 7f1e956ce640 1 --2- 172.21.15.84:0/1548538191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e7c0047c0 0x7f1e7c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:29.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.289+0000 7f1e95ecf640 1 --2- 172.21.15.84:0/1548538191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e7c003a10 0x7f1e7c003e30 secure :-1 s=READY pgs=3156 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00c0e0 tx=0x7f1e7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768cc0004af1837e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:29.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.289+0000 7f1e94ecd640 1 -- 172.21.15.84:0/1548538191 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c005ad0 con 0x7f1e7c003a10 2023-10-16T11:26:29.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.289+0000 7f1e94ecd640 1 -- 172.21.15.84:0/1548538191 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8c005c30 con 0x7f1e7c003a10 2023-10-16T11:26:29.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.289+0000 7f1e94ecd640 1 -- 172.21.15.84:0/1548538191 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c02c7f0 con 0x7f1e7c003a10 2023-10-16T11:26:29.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.289+0000 7f1e98133640 1 -- 172.21.15.84:0/1548538191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e7c003a10 msgr2=0x7f1e7c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:29.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.290+0000 7f1e98133640 1 --2- 172.21.15.84:0/1548538191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e7c003a10 0x7f1e7c003e30 secure :-1 s=READY pgs=3156 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00c0e0 tx=0x7f1e7c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:29.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.290+0000 7f1e966d0640 1 -- 172.21.15.84:0/1548538191 reap_dead start 2023-10-16T11:26:29.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.290+0000 7f1e98133640 1 -- 172.21.15.84:0/1548538191 shutdown_connections 2023-10-16T11:26:29.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.290+0000 7f1e98133640 1 -- 172.21.15.84:0/1548538191 >> 172.21.15.84:0/1548538191 conn(0x7f1e900cb360 msgr2=0x7f1e900cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:29.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.290+0000 7f1e98133640 1 -- 172.21.15.84:0/1548538191 shutdown_connections 2023-10-16T11:26:29.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.290+0000 7f1e98133640 1 -- 172.21.15.84:0/1548538191 wait complete. 2023-10-16T11:26:29.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.292+0000 7f1e98133640 1 Processor -- start 2023-10-16T11:26:29.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.292+0000 7f1e98133640 1 -- start start 2023-10-16T11:26:29.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.293+0000 7f1e98133640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e7c003a10 0x7f1e900cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:29.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.293+0000 7f1e98133640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e901f3ce0 0x7f1e90165220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:29.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.293+0000 7f1e98133640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e901f5910 0x7f1e90070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:29.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.293+0000 7f1e98133640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e901668b0 con 0x7f1e7c003a10 2023-10-16T11:26:29.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.293+0000 7f1e98133640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9015a710 con 0x7f1e901f5910 2023-10-16T11:26:29.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.293+0000 7f1e98133640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e90165760 con 0x7f1e901f3ce0 2023-10-16T11:26:29.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.293+0000 7f1e95ecf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e7c003a10 0x7f1e900cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:29.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.293+0000 7f1e95ecf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e7c003a10 0x7f1e900cec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46772/0 (socket says 172.21.15.84:46772) 2023-10-16T11:26:29.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.293+0000 7f1e95ecf640 1 -- 172.21.15.84:0/399036138 learned_addr learned my addr 172.21.15.84:0/399036138 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:29.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.294+0000 7f1e95ecf640 1 -- 172.21.15.84:0/399036138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e901f3ce0 msgr2=0x7f1e90165220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:29.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.294+0000 7f1e95ecf640 1 --2- 172.21.15.84:0/399036138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e901f3ce0 0x7f1e90165220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:29.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.294+0000 7f1e95ecf640 1 -- 172.21.15.84:0/399036138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e901f5910 msgr2=0x7f1e90070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:29.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.294+0000 7f1e95ecf640 1 --2- 172.21.15.84:0/399036138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e901f5910 0x7f1e90070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:29.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.294+0000 7f1e95ecf640 1 -- 172.21.15.84:0/399036138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e90165a00 con 0x7f1e7c003a10 2023-10-16T11:26:29.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.294+0000 7f1e95ecf640 1 --2- 172.21.15.84:0/399036138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e7c003a10 0x7f1e900cec30 secure :-1 s=READY pgs=3157 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00c0e0 tx=0x7f1e8c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:29.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.295+0000 7f1e867fc640 1 -- 172.21.15.84:0/399036138 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c03c040 con 0x7f1e7c003a10 2023-10-16T11:26:29.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.295+0000 7f1e867fc640 1 -- 172.21.15.84:0/399036138 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8c02dc90 con 0x7f1e7c003a10 2023-10-16T11:26:29.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.295+0000 7f1e98133640 1 -- 172.21.15.84:0/399036138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9015a960 con 0x7f1e7c003a10 2023-10-16T11:26:29.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.295+0000 7f1e867fc640 1 -- 172.21.15.84:0/399036138 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c007070 con 0x7f1e7c003a10 2023-10-16T11:26:29.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.295+0000 7f1e98133640 1 -- 172.21.15.84:0/399036138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9015af20 con 0x7f1e7c003a10 2023-10-16T11:26:29.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.296+0000 7f1e98133640 1 -- 172.21.15.84:0/399036138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e9004e160 con 0x7f1e7c003a10 2023-10-16T11:26:29.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.297+0000 7f1e867fc640 1 -- 172.21.15.84:0/399036138 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f1e8c032070 con 0x7f1e7c003a10 2023-10-16T11:26:29.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.298+0000 7f1e867fc640 1 --2- 172.21.15.84:0/399036138 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f1e7406b020 0x7f1e7406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:29.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.298+0000 7f1e867fc640 1 -- 172.21.15.84:0/399036138 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f1e8c0ab7b0 con 0x7f1e7c003a10 2023-10-16T11:26:29.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.299+0000 7f1e956ce640 1 --2- 172.21.15.84:0/399036138 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f1e7406b020 0x7f1e7406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:29.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.299+0000 7f1e956ce640 1 --2- 172.21.15.84:0/399036138 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f1e7406b020 0x7f1e7406d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1e901f4100 tx=0x7f1e80006e10 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:29.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.303+0000 7f1e867fc640 1 -- 172.21.15.84:0/399036138 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1e8c0761a0 con 0x7f1e7c003a10 2023-10-16T11:26:29.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:29.503+0000 7f1e98133640 1 -- 172.21.15.84:0/399036138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1e90071540 con 0x7f1e7c003a10 2023-10-16T11:26:30.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.316+0000 7f1e867fc640 1 -- 172.21.15.84:0/399036138 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v248) v1 ==== 190+0+0 (secure 0 0 0) 0x7f1e8c0797c0 con 0x7f1e7c003a10 2023-10-16T11:26:30.318 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'backup_fs_metadata' removed 2023-10-16T11:26:30.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.321+0000 7f1e98133640 1 -- 172.21.15.84:0/399036138 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f1e7406b020 msgr2=0x7f1e7406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:30.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.321+0000 7f1e98133640 1 --2- 172.21.15.84:0/399036138 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f1e7406b020 0x7f1e7406d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1e901f4100 tx=0x7f1e80006e10 comp rx=0 tx=0).stop 2023-10-16T11:26:30.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.321+0000 7f1e98133640 1 -- 172.21.15.84:0/399036138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e7c003a10 msgr2=0x7f1e900cec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:30.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.321+0000 7f1e98133640 1 --2- 172.21.15.84:0/399036138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e7c003a10 0x7f1e900cec30 secure :-1 s=READY pgs=3157 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00c0e0 tx=0x7f1e8c001930 comp rx=0 tx=0).stop 2023-10-16T11:26:30.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.321+0000 7f1e98133640 1 -- 172.21.15.84:0/399036138 shutdown_connections 2023-10-16T11:26:30.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.321+0000 7f1e98133640 1 --2- 172.21.15.84:0/399036138 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f1e7406b020 0x7f1e7406d4e0 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1e901f4100 tx=0x7f1e80006e10 comp rx=0 tx=0).stop 2023-10-16T11:26:30.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.321+0000 7f1e98133640 1 --2- 172.21.15.84:0/399036138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e901f5910 0x7f1e90070510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:30.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.322+0000 7f1e98133640 1 --2- 172.21.15.84:0/399036138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e901f3ce0 0x7f1e90165220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:30.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.322+0000 7f1e98133640 1 --2- 172.21.15.84:0/399036138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e7c003a10 0x7f1e900cec30 unknown :-1 s=CLOSED pgs=3157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:30.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.322+0000 7f1e98133640 1 -- 172.21.15.84:0/399036138 >> 172.21.15.84:0/399036138 conn(0x7f1e900cb360 msgr2=0x7f1e90154f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:30.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.323+0000 7f1e98133640 1 -- 172.21.15.84:0/399036138 shutdown_connections 2023-10-16T11:26:30.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.324+0000 7f1e98133640 1 -- 172.21.15.84:0/399036138 wait complete. 2023-10-16T11:26:30.354 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2023-10-16T11:26:30.371 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:30.369+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:30.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.549+0000 7fc379967640 1 Processor -- start 2023-10-16T11:26:30.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.549+0000 7fc379967640 1 -- start start 2023-10-16T11:26:30.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.551+0000 7fc379967640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc3740b8480 0x7fc3740cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:30.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.551+0000 7fc379967640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc3740d0be0 0x7fc3740cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:30.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.551+0000 7fc379967640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37415d150 con 0x7fc37415cd70 2023-10-16T11:26:30.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.551+0000 7fc379967640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3740d1180 con 0x7fc3740d0be0 2023-10-16T11:26:30.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.551+0000 7fc379967640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3740d1460 con 0x7fc3740b8480 2023-10-16T11:26:30.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.551+0000 7fc379166640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc3740d0be0 0x7fc3740cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:30.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.551+0000 7fc379166640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc3740d0be0 0x7fc3740cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46786/0 (socket says 172.21.15.84:46786) 2023-10-16T11:26:30.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.551+0000 7fc379166640 1 -- 172.21.15.84:0/4060238015 learned_addr learned my addr 172.21.15.84:0/4060238015 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:30.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.552+0000 7fc379166640 1 -- 172.21.15.84:0/4060238015 >> v2:172.21.15.157:3300/0 conn(0x7fc3740b8480 msgr2=0x7fc3740cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:30.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.552+0000 7fc379166640 1 --2- 172.21.15.84:0/4060238015 >> v2:172.21.15.157:3300/0 conn(0x7fc3740b8480 0x7fc3740cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:30.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.552+0000 7fc379166640 1 -- 172.21.15.84:0/4060238015 >> v1:172.21.15.157:6789/0 conn(0x7fc37415cd70 legacy=0x7fc3740d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:30.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.552+0000 7fc379166640 1 -- 172.21.15.84:0/4060238015 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3740d1740 con 0x7fc3740d0be0 2023-10-16T11:26:30.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.553+0000 7fc379166640 1 --2- 172.21.15.84:0/4060238015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3740d0be0 0x7fc3740cdd00 secure :-1 s=READY pgs=3158 cs=0 l=1 rev1=1 crypto rx=0x7fc374066a00 tx=0x7fc36c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81f7be1d5813f62 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:30.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.553+0000 7fc3737fe640 1 -- 172.21.15.84:0/4060238015 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c02c9f0 con 0x7fc3740d0be0 2023-10-16T11:26:30.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.553+0000 7fc3737fe640 1 -- 172.21.15.84:0/4060238015 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36c02cb50 con 0x7fc3740d0be0 2023-10-16T11:26:30.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.553+0000 7fc3737fe640 1 -- 172.21.15.84:0/4060238015 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c0343f0 con 0x7fc3740d0be0 2023-10-16T11:26:30.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.554+0000 7fc379967640 1 -- 172.21.15.84:0/4060238015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3740d0be0 msgr2=0x7fc3740cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:30.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.554+0000 7fc379967640 1 --2- 172.21.15.84:0/4060238015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3740d0be0 0x7fc3740cdd00 secure :-1 s=READY pgs=3158 cs=0 l=1 rev1=1 crypto rx=0x7fc374066a00 tx=0x7fc36c02be50 comp rx=0 tx=0).stop 2023-10-16T11:26:30.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.554+0000 7fc379967640 1 -- 172.21.15.84:0/4060238015 shutdown_connections 2023-10-16T11:26:30.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.554+0000 7fc379967640 1 --2- 172.21.15.84:0/4060238015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3740d0be0 0x7fc3740cdd00 unknown :-1 s=CLOSED pgs=3158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:30.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.554+0000 7fc379967640 1 --2- 172.21.15.84:0/4060238015 >> v2:172.21.15.157:3300/0 conn(0x7fc3740b8480 0x7fc3740cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:30.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.554+0000 7fc379967640 1 -- 172.21.15.84:0/4060238015 >> 172.21.15.84:0/4060238015 conn(0x7fc3740cf570 msgr2=0x7fc3740cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:30.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.554+0000 7fc379967640 1 -- 172.21.15.84:0/4060238015 shutdown_connections 2023-10-16T11:26:30.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.554+0000 7fc379967640 1 -- 172.21.15.84:0/4060238015 wait complete. 2023-10-16T11:26:30.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.556+0000 7fc379967640 1 Processor -- start 2023-10-16T11:26:30.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.556+0000 7fc379967640 1 -- start start 2023-10-16T11:26:30.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.557+0000 7fc379967640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3740b8480 0x7fc3740b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:30.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.557+0000 7fc379967640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3740d0be0 0x7fc3740cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:30.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.557+0000 7fc379967640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc37415cd70 0x7fc37415f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:30.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.557+0000 7fc379967640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc374148540 con 0x7fc3740d0be0 2023-10-16T11:26:30.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.557+0000 7fc379967640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3741f53f0 con 0x7fc37415cd70 2023-10-16T11:26:30.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.557+0000 7fc379967640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3741f5640 con 0x7fc3740b8480 2023-10-16T11:26:30.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.558+0000 7fc373fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3740d0be0 0x7fc3740cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:30.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.558+0000 7fc373fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3740d0be0 0x7fc3740cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46794/0 (socket says 172.21.15.84:46794) 2023-10-16T11:26:30.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.558+0000 7fc373fff640 1 -- 172.21.15.84:0/2675294729 learned_addr learned my addr 172.21.15.84:0/2675294729 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:30.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.558+0000 7fc378965640 1 --2- 172.21.15.84:0/2675294729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3740b8480 0x7fc3740b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:30.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.558+0000 7fc379166640 1 --2- 172.21.15.84:0/2675294729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc37415cd70 0x7fc37415f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:30.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.558+0000 7fc373fff640 1 -- 172.21.15.84:0/2675294729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3740b8480 msgr2=0x7fc3740b7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:30.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.558+0000 7fc373fff640 1 --2- 172.21.15.84:0/2675294729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3740b8480 0x7fc3740b7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:30.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.558+0000 7fc373fff640 1 -- 172.21.15.84:0/2675294729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc37415cd70 msgr2=0x7fc37415f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:30.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.558+0000 7fc373fff640 1 --2- 172.21.15.84:0/2675294729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc37415cd70 0x7fc37415f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:30.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.558+0000 7fc373fff640 1 -- 172.21.15.84:0/2675294729 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36c009b30 con 0x7fc3740d0be0 2023-10-16T11:26:30.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.558+0000 7fc379166640 1 --2- 172.21.15.84:0/2675294729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc37415cd70 0x7fc37415f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:30.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.559+0000 7fc373fff640 1 --2- 172.21.15.84:0/2675294729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3740d0be0 0x7fc3740cc310 secure :-1 s=READY pgs=3159 cs=0 l=1 rev1=1 crypto rx=0x7fc36000b410 tx=0x7fc36000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:30.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.559+0000 7fc3717fa640 1 -- 172.21.15.84:0/2675294729 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360011830 con 0x7fc3740d0be0 2023-10-16T11:26:30.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.559+0000 7fc3717fa640 1 -- 172.21.15.84:0/2675294729 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3600119b0 con 0x7fc3740d0be0 2023-10-16T11:26:30.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.559+0000 7fc3717fa640 1 -- 172.21.15.84:0/2675294729 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360011d60 con 0x7fc3740d0be0 2023-10-16T11:26:30.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.559+0000 7fc379967640 1 -- 172.21.15.84:0/2675294729 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3741f5920 con 0x7fc3740d0be0 2023-10-16T11:26:30.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.559+0000 7fc379967640 1 -- 172.21.15.84:0/2675294729 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3741f5ee0 con 0x7fc3740d0be0 2023-10-16T11:26:30.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.560+0000 7fc379967640 1 -- 172.21.15.84:0/2675294729 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37404e9b0 con 0x7fc3740d0be0 2023-10-16T11:26:30.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.562+0000 7fc3717fa640 1 -- 172.21.15.84:0/2675294729 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fc36001a400 con 0x7fc3740d0be0 2023-10-16T11:26:30.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.562+0000 7fc3717fa640 1 --2- 172.21.15.84:0/2675294729 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc35806b020 0x7fc35806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:30.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.563+0000 7fc3717fa640 1 -- 172.21.15.84:0/2675294729 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 6337+0+0 (secure 0 0 0) 0x7fc360089df0 con 0x7fc3740d0be0 2023-10-16T11:26:30.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.563+0000 7fc378965640 1 --2- 172.21.15.84:0/2675294729 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc35806b020 0x7fc35806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:30.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.563+0000 7fc378965640 1 --2- 172.21.15.84:0/2675294729 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc35806b020 0x7fc35806d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc368007fd0 tx=0x7fc368009040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:30.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.567+0000 7fc3717fa640 1 -- 172.21.15.84:0/2675294729 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc360054900 con 0x7fc3740d0be0 2023-10-16T11:26:30.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:30.767+0000 7fc379967640 1 -- 172.21.15.84:0/2675294729 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc374067e30 con 0x7fc3740d0be0 2023-10-16T11:26:31.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.320+0000 7fc3717fa640 1 -- 172.21.15.84:0/2675294729 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v249) v1 ==== 178+0+0 (secure 0 0 0) 0x7fc360015020 con 0x7fc3740d0be0 2023-10-16T11:26:31.323 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'backup_fs_data' removed 2023-10-16T11:26:31.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.325+0000 7fc379967640 1 -- 172.21.15.84:0/2675294729 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc35806b020 msgr2=0x7fc35806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:31.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.325+0000 7fc379967640 1 --2- 172.21.15.84:0/2675294729 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc35806b020 0x7fc35806d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc368007fd0 tx=0x7fc368009040 comp rx=0 tx=0).stop 2023-10-16T11:26:31.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.325+0000 7fc379967640 1 -- 172.21.15.84:0/2675294729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3740d0be0 msgr2=0x7fc3740cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:31.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.325+0000 7fc379967640 1 --2- 172.21.15.84:0/2675294729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3740d0be0 0x7fc3740cc310 secure :-1 s=READY pgs=3159 cs=0 l=1 rev1=1 crypto rx=0x7fc36000b410 tx=0x7fc36000b8d0 comp rx=0 tx=0).stop 2023-10-16T11:26:31.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.329+0000 7fc379967640 1 -- 172.21.15.84:0/2675294729 shutdown_connections 2023-10-16T11:26:31.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.329+0000 7fc379967640 1 --2- 172.21.15.84:0/2675294729 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc35806b020 0x7fc35806d4e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:31.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.329+0000 7fc379967640 1 --2- 172.21.15.84:0/2675294729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc37415cd70 0x7fc37415f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:31.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.329+0000 7fc379967640 1 --2- 172.21.15.84:0/2675294729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3740d0be0 0x7fc3740cc310 unknown :-1 s=CLOSED pgs=3159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:31.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.329+0000 7fc379967640 1 --2- 172.21.15.84:0/2675294729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3740b8480 0x7fc3740b7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:31.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.329+0000 7fc379967640 1 -- 172.21.15.84:0/2675294729 >> 172.21.15.84:0/2675294729 conn(0x7fc3740cf570 msgr2=0x7fc3740d2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:31.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.329+0000 7fc379967640 1 -- 172.21.15.84:0/2675294729 shutdown_connections 2023-10-16T11:26:31.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.329+0000 7fc379967640 1 -- 172.21.15.84:0/2675294729 wait complete. 2023-10-16T11:26:31.354 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T11:26:31.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.595+0000 7f519741a640 1 Processor -- start 2023-10-16T11:26:31.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.596+0000 7f519741a640 1 -- start start 2023-10-16T11:26:31.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.597+0000 7f519741a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f519015c9f0 0x7f51900c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:31.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.597+0000 7f519741a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5190160a90 con 0x7f519015dde0 2023-10-16T11:26:31.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.597+0000 7f519741a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51900cd900 con 0x7f51900c9230 2023-10-16T11:26:31.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.597+0000 7f519741a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51900cdec0 con 0x7f519015c9f0 2023-10-16T11:26:31.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.598+0000 7f5196c19640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f519015dde0 0x7f51900ac890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60778/0 (socket says 172.21.15.84:60778) 2023-10-16T11:26:31.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.598+0000 7f5196c19640 1 -- 172.21.15.84:0/2776720475 learned_addr learned my addr 172.21.15.84:0/2776720475 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:31.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.598+0000 7f5196418640 1 --2- 172.21.15.84:0/2776720475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f519015c9f0 0x7f51900c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:31.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.598+0000 7f5195416640 1 -- 172.21.15.84:0/2776720475 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3896310056 0 0) 0x7f5190160a90 con 0x7f519015dde0 2023-10-16T11:26:31.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.598+0000 7f5195416640 1 -- 172.21.15.84:0/2776720475 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f517c003520 con 0x7f519015dde0 2023-10-16T11:26:31.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.598+0000 7f5195416640 1 -- 172.21.15.84:0/2776720475 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3088217180 0 0) 0x7f517c003520 con 0x7f519015dde0 2023-10-16T11:26:31.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.598+0000 7f5195416640 1 -- 172.21.15.84:0/2776720475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f519015c9f0 msgr2=0x7f51900c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:31.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.598+0000 7f5195416640 1 --2- 172.21.15.84:0/2776720475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f519015c9f0 0x7f51900c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:31.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.598+0000 7f5195416640 1 -- 172.21.15.84:0/2776720475 >> v1:172.21.15.157:6789/0 conn(0x7f51900c9230 legacy=0x7f51900709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:31.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.598+0000 7f5195416640 1 -- 172.21.15.84:0/2776720475 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51900cdbe0 con 0x7f519015dde0 2023-10-16T11:26:31.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.598+0000 7f5195416640 1 -- 172.21.15.84:0/2776720475 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f51880019f0 con 0x7f519015dde0 2023-10-16T11:26:31.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.599+0000 7f5195416640 1 -- 172.21.15.84:0/2776720475 >> v1:172.21.15.84:6789/0 conn(0x7f519015dde0 legacy=0x7f51900ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:31.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.599+0000 7f5195416640 1 --2- 172.21.15.84:0/2776720475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f517c0039a0 0x7f517c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:31.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.599+0000 7f5195416640 1 --2- 172.21.15.84:0/2776720475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f519015c9f0 0x7f517c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:31.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.599+0000 7f5195416640 1 --2- 172.21.15.84:0/2776720475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f517c004d00 0x7f517c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:31.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.599+0000 7f5195416640 1 -- 172.21.15.84:0/2776720475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c00bb60 con 0x7f517c0039a0 2023-10-16T11:26:31.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.599+0000 7f5195416640 1 -- 172.21.15.84:0/2776720475 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c00bca0 con 0x7f517c004d00 2023-10-16T11:26:31.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.599+0000 7f5195416640 1 -- 172.21.15.84:0/2776720475 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517c00be60 con 0x7f519015c9f0 2023-10-16T11:26:31.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.599+0000 7f5196418640 1 --2- 172.21.15.84:0/2776720475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f517c0039a0 0x7f517c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:31.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.599+0000 7f5196c19640 1 --2- 172.21.15.84:0/2776720475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f517c004d00 0x7f517c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:31.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.599+0000 7f5196418640 1 -- 172.21.15.84:0/2776720475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f519015c9f0 msgr2=0x7f517c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:31.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.599+0000 7f5196418640 1 --2- 172.21.15.84:0/2776720475 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f519015c9f0 0x7f517c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:31.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.599+0000 7f5196418640 1 -- 172.21.15.84:0/2776720475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f517c004d00 msgr2=0x7f517c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:31.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.599+0000 7f5196418640 1 --2- 172.21.15.84:0/2776720475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f517c004d00 0x7f517c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:31.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.599+0000 7f5196418640 1 -- 172.21.15.84:0/2776720475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c003520 con 0x7f517c0039a0 2023-10-16T11:26:31.609 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:31.599+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:31.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.600+0000 7f5196418640 1 --2- 172.21.15.84:0/2776720475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f517c0039a0 0x7f517c003dc0 secure :-1 s=READY pgs=3161 cs=0 l=1 rev1=1 crypto rx=0x7f518c00ba30 tx=0x7f517c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad4ec13765bfe60 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:31.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.601+0000 7f5195416640 1 -- 172.21.15.84:0/2776720475 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c032680 con 0x7f517c0039a0 2023-10-16T11:26:31.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.601+0000 7f5195416640 1 -- 172.21.15.84:0/2776720475 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c0327e0 con 0x7f517c0039a0 2023-10-16T11:26:31.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.601+0000 7f5195416640 1 -- 172.21.15.84:0/2776720475 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c032b70 con 0x7f517c0039a0 2023-10-16T11:26:31.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.601+0000 7f519741a640 1 -- 172.21.15.84:0/2776720475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f517c0039a0 msgr2=0x7f517c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:31.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.601+0000 7f519741a640 1 --2- 172.21.15.84:0/2776720475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f517c0039a0 0x7f517c003dc0 secure :-1 s=READY pgs=3161 cs=0 l=1 rev1=1 crypto rx=0x7f518c00ba30 tx=0x7f517c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:31.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.601+0000 7f5196c19640 1 -- 172.21.15.84:0/2776720475 reap_dead start 2023-10-16T11:26:31.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.601+0000 7f519741a640 1 -- 172.21.15.84:0/2776720475 shutdown_connections 2023-10-16T11:26:31.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.601+0000 7f519741a640 1 -- 172.21.15.84:0/2776720475 >> 172.21.15.84:0/2776720475 conn(0x7f51900ce6a0 msgr2=0x7f51900ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:31.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.602+0000 7f519741a640 1 -- 172.21.15.84:0/2776720475 shutdown_connections 2023-10-16T11:26:31.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.602+0000 7f519741a640 1 -- 172.21.15.84:0/2776720475 wait complete. 2023-10-16T11:26:31.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.603+0000 7f519741a640 1 Processor -- start 2023-10-16T11:26:31.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.604+0000 7f519741a640 1 -- start start 2023-10-16T11:26:31.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.604+0000 7f519741a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f517c0039a0 0x7f51900ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:31.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.604+0000 7f519741a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f519006b810 0x7f519006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:31.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.604+0000 7f519741a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51901500c0 0x7f5190150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:31.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.605+0000 7f519741a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5190150a80 con 0x7f519006b810 2023-10-16T11:26:31.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.605+0000 7f519741a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519006c380 con 0x7f51901500c0 2023-10-16T11:26:31.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.605+0000 7f519741a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51900c58a0 con 0x7f517c0039a0 2023-10-16T11:26:31.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.605+0000 7f5195c17640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f519006b810 0x7f519006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:31.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.605+0000 7f5195c17640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f519006b810 0x7f519006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46812/0 (socket says 172.21.15.84:46812) 2023-10-16T11:26:31.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.605+0000 7f5195c17640 1 -- 172.21.15.84:0/4061116511 learned_addr learned my addr 172.21.15.84:0/4061116511 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:31.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.605+0000 7f5196c19640 1 --2- 172.21.15.84:0/4061116511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51901500c0 0x7f5190150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:31.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.605+0000 7f5196418640 1 --2- 172.21.15.84:0/4061116511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f517c0039a0 0x7f51900ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:31.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.605+0000 7f5195c17640 1 -- 172.21.15.84:0/4061116511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f517c0039a0 msgr2=0x7f51900ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:31.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.605+0000 7f5195c17640 1 --2- 172.21.15.84:0/4061116511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f517c0039a0 0x7f51900ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.605+0000 7f5195c17640 1 -- 172.21.15.84:0/4061116511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51901500c0 msgr2=0x7f5190150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.605+0000 7f5195c17640 1 --2- 172.21.15.84:0/4061116511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51901500c0 0x7f5190150540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.605+0000 7f5195c17640 1 -- 172.21.15.84:0/4061116511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51900c5b80 con 0x7f519006b810 2023-10-16T11:26:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.606+0000 7f5195c17640 1 --2- 172.21.15.84:0/4061116511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f519006b810 0x7f519006bc90 secure :-1 s=READY pgs=3162 cs=0 l=1 rev1=1 crypto rx=0x7f518400c530 tx=0x7f518400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:31.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.606+0000 7f5182ffd640 1 -- 172.21.15.84:0/4061116511 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51840052d0 con 0x7f519006b810 2023-10-16T11:26:31.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.606+0000 7f5182ffd640 1 -- 172.21.15.84:0/4061116511 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5184005430 con 0x7f519006b810 2023-10-16T11:26:31.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.606+0000 7f519741a640 1 -- 172.21.15.84:0/4061116511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51900c5e60 con 0x7f519006b810 2023-10-16T11:26:31.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.606+0000 7f5182ffd640 1 -- 172.21.15.84:0/4061116511 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5184012870 con 0x7f519006b810 2023-10-16T11:26:31.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.606+0000 7f519741a640 1 -- 172.21.15.84:0/4061116511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5190150ec0 con 0x7f519006b810 2023-10-16T11:26:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.607+0000 7f519741a640 1 -- 172.21.15.84:0/4061116511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519004e9b0 con 0x7f519006b810 2023-10-16T11:26:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.608+0000 7f5182ffd640 1 -- 172.21.15.84:0/4061116511 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f51840129d0 con 0x7f519006b810 2023-10-16T11:26:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.609+0000 7f5182ffd640 1 --2- 172.21.15.84:0/4061116511 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f517406b050 0x7f517406d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.609+0000 7f5182ffd640 1 -- 172.21.15.84:0/4061116511 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(249..249 src has 1..249) v4 ==== 5917+0+0 (secure 0 0 0) 0x7f5184089550 con 0x7f519006b810 2023-10-16T11:26:31.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.610+0000 7f5196418640 1 --2- 172.21.15.84:0/4061116511 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f517406b050 0x7f517406d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:31.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.610+0000 7f5196418640 1 --2- 172.21.15.84:0/4061116511 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f517406b050 0x7f517406d510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f518c00ba00 tx=0x7f518c03d040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:31.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.614+0000 7f5182ffd640 1 -- 172.21.15.84:0/4061116511 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5184054320 con 0x7f519006b810 2023-10-16T11:26:31.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.818+0000 7f519741a640 1 -- 172.21.15.84:0/4061116511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f51900adac0 con 0x7f519006b810 2023-10-16T11:26:31.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.825+0000 7f5182ffd640 1 -- 172.21.15.84:0/4061116511 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5184057940 con 0x7f519006b810 2023-10-16T11:26:31.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.827+0000 7f519741a640 1 -- 172.21.15.84:0/4061116511 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f517406b050 msgr2=0x7f517406d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:31.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.827+0000 7f519741a640 1 --2- 172.21.15.84:0/4061116511 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f517406b050 0x7f517406d510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f518c00ba00 tx=0x7f518c03d040 comp rx=0 tx=0).stop 2023-10-16T11:26:31.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.828+0000 7f519741a640 1 -- 172.21.15.84:0/4061116511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f519006b810 msgr2=0x7f519006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:31.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.828+0000 7f519741a640 1 --2- 172.21.15.84:0/4061116511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f519006b810 0x7f519006bc90 secure :-1 s=READY pgs=3162 cs=0 l=1 rev1=1 crypto rx=0x7f518400c530 tx=0x7f518400c9f0 comp rx=0 tx=0).stop 2023-10-16T11:26:31.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.828+0000 7f519741a640 1 -- 172.21.15.84:0/4061116511 shutdown_connections 2023-10-16T11:26:31.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.828+0000 7f519741a640 1 --2- 172.21.15.84:0/4061116511 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f517406b050 0x7f517406d510 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:31.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.828+0000 7f519741a640 1 --2- 172.21.15.84:0/4061116511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51901500c0 0x7f5190150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:31.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.828+0000 7f519741a640 1 --2- 172.21.15.84:0/4061116511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f519006b810 0x7f519006bc90 unknown :-1 s=CLOSED pgs=3162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:31.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.828+0000 7f519741a640 1 --2- 172.21.15.84:0/4061116511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f517c0039a0 0x7f51900ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:31.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.828+0000 7f519741a640 1 -- 172.21.15.84:0/4061116511 >> 172.21.15.84:0/4061116511 conn(0x7f51900ce6a0 msgr2=0x7f519015aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.828+0000 7f519741a640 1 -- 172.21.15.84:0/4061116511 shutdown_connections 2023-10-16T11:26:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:31.828+0000 7f519741a640 1 -- 172.21.15.84:0/4061116511 wait complete. 2023-10-16T11:26:31.848 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T11:26:32.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.054+0000 7f0605e32640 1 Processor -- start 2023-10-16T11:26:32.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.054+0000 7f0605e32640 1 -- start start 2023-10-16T11:26:32.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.055+0000 7f0605e32640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06000b8480 0x7f06000cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.055+0000 7f0605e32640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f06000d0be0 0x7f06000cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.055+0000 7f0605e32640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060015d150 con 0x7f060015cd70 2023-10-16T11:26:32.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.055+0000 7f0605e32640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06000d1180 con 0x7f06000d0be0 2023-10-16T11:26:32.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.055+0000 7f0605e32640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06000d1460 con 0x7f06000b8480 2023-10-16T11:26:32.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.056+0000 7f0604e30640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f060015cd70 0x7f06000d2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60794/0 (socket says 172.21.15.84:60794) 2023-10-16T11:26:32.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.056+0000 7f0604e30640 1 -- 172.21.15.84:0/124554058 learned_addr learned my addr 172.21.15.84:0/124554058 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:32.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.056+0000 7f0605631640 1 --2- 172.21.15.84:0/124554058 >> v2:172.21.15.84:3300/0 conn(0x7f06000d0be0 0x7f06000cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.056+0000 7f05ff7fe640 1 -- 172.21.15.84:0/124554058 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130928885 0 0) 0x7f060015d150 con 0x7f060015cd70 2023-10-16T11:26:32.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.056+0000 7f05ff7fe640 1 -- 172.21.15.84:0/124554058 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05ec003520 con 0x7f060015cd70 2023-10-16T11:26:32.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.056+0000 7f05fffff640 1 --2- 172.21.15.84:0/124554058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06000b8480 0x7f06000cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.056+0000 7f0605631640 1 -- 172.21.15.84:0/124554058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06000b8480 msgr2=0x7f06000cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:32.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.056+0000 7f0605631640 1 --2- 172.21.15.84:0/124554058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06000b8480 0x7f06000cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.056+0000 7f0605631640 1 -- 172.21.15.84:0/124554058 >> v1:172.21.15.84:6789/0 conn(0x7f060015cd70 legacy=0x7f06000d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:32.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.056+0000 7f0605631640 1 -- 172.21.15.84:0/124554058 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06000d1740 con 0x7f06000d0be0 2023-10-16T11:26:32.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.056+0000 7f05fffff640 1 --2- 172.21.15.84:0/124554058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06000b8480 0x7f06000cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:32.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.057+0000 7f0605631640 1 --2- 172.21.15.84:0/124554058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06000d0be0 0x7f06000cdd00 secure :-1 s=READY pgs=3164 cs=0 l=1 rev1=1 crypto rx=0x7f05f80097a0 tx=0x7f05f802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02d56bea0825710 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:32.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.058+0000 7f05ff7fe640 1 -- 172.21.15.84:0/124554058 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f802d040 con 0x7f06000d0be0 2023-10-16T11:26:32.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.058+0000 7f05ff7fe640 1 -- 172.21.15.84:0/124554058 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f8031810 con 0x7f06000d0be0 2023-10-16T11:26:32.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.058+0000 7f05ff7fe640 1 -- 172.21.15.84:0/124554058 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f8031b80 con 0x7f06000d0be0 2023-10-16T11:26:32.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.058+0000 7f0605e32640 1 -- 172.21.15.84:0/124554058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06000d0be0 msgr2=0x7f06000cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:32.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.058+0000 7f0605e32640 1 --2- 172.21.15.84:0/124554058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06000d0be0 0x7f06000cdd00 secure :-1 s=READY pgs=3164 cs=0 l=1 rev1=1 crypto rx=0x7f05f80097a0 tx=0x7f05f802b8b0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.058+0000 7f0605e32640 1 -- 172.21.15.84:0/124554058 shutdown_connections 2023-10-16T11:26:32.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.058+0000 7f0605e32640 1 --2- 172.21.15.84:0/124554058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06000d0be0 0x7f06000cdd00 unknown :-1 s=CLOSED pgs=3164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.058+0000 7f0605e32640 1 --2- 172.21.15.84:0/124554058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06000b8480 0x7f06000cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.058+0000 7f0605e32640 1 -- 172.21.15.84:0/124554058 >> 172.21.15.84:0/124554058 conn(0x7f06000cf570 msgr2=0x7f06000cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:32.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.058+0000 7f0605e32640 1 -- 172.21.15.84:0/124554058 shutdown_connections 2023-10-16T11:26:32.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.058+0000 7f0605e32640 1 -- 172.21.15.84:0/124554058 wait complete. 2023-10-16T11:26:32.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.060+0000 7f0605e32640 1 Processor -- start 2023-10-16T11:26:32.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.060+0000 7f0605e32640 1 -- start start 2023-10-16T11:26:32.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.061+0000 7f0605e32640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06000b8480 0x7f06000caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.061+0000 7f0605e32640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06000d0be0 0x7f06000c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.061+0000 7f0605e32640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f060015cd70 0x7f0600159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.061+0000 7f0605e32640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06000cc290 con 0x7f06000d0be0 2023-10-16T11:26:32.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.061+0000 7f0605e32640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06000c8c40 con 0x7f06000b8480 2023-10-16T11:26:32.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.061+0000 7f0605e32640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06000cb610 con 0x7f060015cd70 2023-10-16T11:26:32.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.062+0000 7f05fffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06000d0be0 0x7f06000c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.062+0000 7f05fffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06000d0be0 0x7f06000c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46840/0 (socket says 172.21.15.84:46840) 2023-10-16T11:26:32.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.062+0000 7f05fffff640 1 -- 172.21.15.84:0/1223497963 learned_addr learned my addr 172.21.15.84:0/1223497963 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:32.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.062+0000 7f0605631640 1 --2- 172.21.15.84:0/1223497963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f060015cd70 0x7f0600159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.062+0000 7f0604e30640 1 --2- 172.21.15.84:0/1223497963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06000b8480 0x7f06000caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.062+0000 7f05fffff640 1 -- 172.21.15.84:0/1223497963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f060015cd70 msgr2=0x7f0600159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:32.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.062+0000 7f05fffff640 1 --2- 172.21.15.84:0/1223497963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f060015cd70 0x7f0600159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.062+0000 7f05fffff640 1 -- 172.21.15.84:0/1223497963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06000b8480 msgr2=0x7f06000caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:32.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.062+0000 7f05fffff640 1 --2- 172.21.15.84:0/1223497963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06000b8480 0x7f06000caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.062+0000 7f05fffff640 1 -- 172.21.15.84:0/1223497963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f8009400 con 0x7f06000d0be0 2023-10-16T11:26:32.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.062+0000 7f0605631640 1 --2- 172.21.15.84:0/1223497963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f060015cd70 0x7f0600159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:32.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.062+0000 7f0604e30640 1 --2- 172.21.15.84:0/1223497963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06000b8480 0x7f06000caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:32.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.062+0000 7f05fffff640 1 --2- 172.21.15.84:0/1223497963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06000d0be0 0x7f06000c8700 secure :-1 s=READY pgs=3165 cs=0 l=1 rev1=1 crypto rx=0x7f05f000ce00 tx=0x7f05f000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:32.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.063+0000 7f05fd7fa640 1 -- 172.21.15.84:0/1223497963 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f000f890 con 0x7f06000d0be0 2023-10-16T11:26:32.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.063+0000 7f05fd7fa640 1 -- 172.21.15.84:0/1223497963 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f000f9f0 con 0x7f06000d0be0 2023-10-16T11:26:32.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.063+0000 7f05fd7fa640 1 -- 172.21.15.84:0/1223497963 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f000fd80 con 0x7f06000d0be0 2023-10-16T11:26:32.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.063+0000 7f0605e32640 1 -- 172.21.15.84:0/1223497963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06000cb8f0 con 0x7f06000d0be0 2023-10-16T11:26:32.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.063+0000 7f0605e32640 1 -- 172.21.15.84:0/1223497963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06000cbeb0 con 0x7f06000d0be0 2023-10-16T11:26:32.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.064+0000 7f0605e32640 1 -- 172.21.15.84:0/1223497963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06000cc3d0 con 0x7f06000d0be0 2023-10-16T11:26:32.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.065+0000 7f05fd7fa640 1 -- 172.21.15.84:0/1223497963 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f05f000b040 con 0x7f06000d0be0 2023-10-16T11:26:32.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.066+0000 7f05fd7fa640 1 --2- 172.21.15.84:0/1223497963 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f05e006b020 0x7f05e006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.066+0000 7f05fd7fa640 1 -- 172.21.15.84:0/1223497963 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(249..249 src has 1..249) v4 ==== 5917+0+0 (secure 0 0 0) 0x7f05f008a510 con 0x7f06000d0be0 2023-10-16T11:26:32.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.066+0000 7f0604e30640 1 --2- 172.21.15.84:0/1223497963 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f05e006b020 0x7f05e006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.067+0000 7f0604e30640 1 --2- 172.21.15.84:0/1223497963 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f05e006b020 0x7f05e006d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f05e8005880 tx=0x7f05e8009040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:32.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.071+0000 7f05fd7fa640 1 -- 172.21.15.84:0/1223497963 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f06000cc3d0 con 0x7f06000d0be0 2023-10-16T11:26:32.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.277+0000 7f0605e32640 1 -- 172.21.15.84:0/1223497963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f06000cc3d0 con 0x7f06000d0be0 2023-10-16T11:26:32.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.284+0000 7f05fd7fa640 1 -- 172.21.15.84:0/1223497963 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v86) v1 ==== 120+0+0 (secure 0 0 0) 0x7f06000cc3d0 con 0x7f06000d0be0 2023-10-16T11:26:32.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.287+0000 7f0605e32640 1 -- 172.21.15.84:0/1223497963 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f05e006b020 msgr2=0x7f05e006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:32.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.287+0000 7f0605e32640 1 --2- 172.21.15.84:0/1223497963 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f05e006b020 0x7f05e006d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f05e8005880 tx=0x7f05e8009040 comp rx=0 tx=0).stop 2023-10-16T11:26:32.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.287+0000 7f0605e32640 1 -- 172.21.15.84:0/1223497963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06000d0be0 msgr2=0x7f06000c8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:32.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.287+0000 7f0605e32640 1 --2- 172.21.15.84:0/1223497963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06000d0be0 0x7f06000c8700 secure :-1 s=READY pgs=3165 cs=0 l=1 rev1=1 crypto rx=0x7f05f000ce00 tx=0x7f05f000a650 comp rx=0 tx=0).stop 2023-10-16T11:26:32.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.287+0000 7f0605e32640 1 -- 172.21.15.84:0/1223497963 shutdown_connections 2023-10-16T11:26:32.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.287+0000 7f0605e32640 1 --2- 172.21.15.84:0/1223497963 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f05e006b020 0x7f05e006d4e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.287+0000 7f0605e32640 1 --2- 172.21.15.84:0/1223497963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f060015cd70 0x7f0600159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.287+0000 7f0605e32640 1 --2- 172.21.15.84:0/1223497963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06000d0be0 0x7f06000c8700 unknown :-1 s=CLOSED pgs=3165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.287+0000 7f0605e32640 1 --2- 172.21.15.84:0/1223497963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06000b8480 0x7f06000caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.287+0000 7f0605e32640 1 -- 172.21.15.84:0/1223497963 >> 172.21.15.84:0/1223497963 conn(0x7f06000cf570 msgr2=0x7f06001e7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:32.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.287+0000 7f0605e32640 1 -- 172.21.15.84:0/1223497963 shutdown_connections 2023-10-16T11:26:32.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.288+0000 7f0605e32640 1 -- 172.21.15.84:0/1223497963 wait complete. 2023-10-16T11:26:32.309 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T11:26:32.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.522+0000 7fc7a9c21640 1 Processor -- start 2023-10-16T11:26:32.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.522+0000 7fc7a9c21640 1 -- start start 2023-10-16T11:26:32.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.524+0000 7fc7a9c21640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7a4070f30 0x7fc7a406aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.524+0000 7fc7a9c21640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7a406b3e0 0x7fc7a40c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.524+0000 7fc7a9c21640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a40cd260 con 0x7fc7a4159ad0 2023-10-16T11:26:32.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.524+0000 7fc7a9c21640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a415de30 con 0x7fc7a4070f30 2023-10-16T11:26:32.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.524+0000 7fc7a9c21640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a415e110 con 0x7fc7a406b3e0 2023-10-16T11:26:32.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.524+0000 7fc7a8c1f640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc7a4159ad0 0x7fc7a415d650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60810/0 (socket says 172.21.15.84:60810) 2023-10-16T11:26:32.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.524+0000 7fc7a8c1f640 1 -- 172.21.15.84:0/2412458241 learned_addr learned my addr 172.21.15.84:0/2412458241 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:32.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.524+0000 7fc7a3fff640 1 --2- 172.21.15.84:0/2412458241 >> v2:172.21.15.157:3300/0 conn(0x7fc7a4070f30 0x7fc7a406aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.524+0000 7fc7a9420640 1 --2- 172.21.15.84:0/2412458241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7a406b3e0 0x7fc7a40c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.524+0000 7fc7a37fe640 1 -- 172.21.15.84:0/2412458241 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689056179 0 0) 0x7fc7a40cd260 con 0x7fc7a4159ad0 2023-10-16T11:26:32.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.524+0000 7fc7a37fe640 1 -- 172.21.15.84:0/2412458241 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc794005730 con 0x7fc7a4159ad0 2023-10-16T11:26:32.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.525+0000 7fc7a37fe640 1 -- 172.21.15.84:0/2412458241 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2194022353 0 0) 0x7fc794005730 con 0x7fc7a4159ad0 2023-10-16T11:26:32.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.525+0000 7fc7a37fe640 1 -- 172.21.15.84:0/2412458241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7a406b3e0 msgr2=0x7fc7a40c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:32.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.525+0000 7fc7a37fe640 1 --2- 172.21.15.84:0/2412458241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7a406b3e0 0x7fc7a40c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.525+0000 7fc7a37fe640 1 -- 172.21.15.84:0/2412458241 >> v2:172.21.15.157:3300/0 conn(0x7fc7a4070f30 msgr2=0x7fc7a406aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:32.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.525+0000 7fc7a37fe640 1 --2- 172.21.15.84:0/2412458241 >> v2:172.21.15.157:3300/0 conn(0x7fc7a4070f30 0x7fc7a406aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.525+0000 7fc7a37fe640 1 -- 172.21.15.84:0/2412458241 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a415e3f0 con 0x7fc7a4159ad0 2023-10-16T11:26:32.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.525+0000 7fc7a37fe640 1 -- 172.21.15.84:0/2412458241 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc7980019f0 con 0x7fc7a4159ad0 2023-10-16T11:26:32.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.525+0000 7fc7a3fff640 1 --2- 172.21.15.84:0/2412458241 >> v2:172.21.15.157:3300/0 conn(0x7fc7a4070f30 0x7fc7a406aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:26:32.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.525+0000 7fc7a37fe640 1 -- 172.21.15.84:0/2412458241 >> v1:172.21.15.84:6789/0 conn(0x7fc7a4159ad0 legacy=0x7fc7a415d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:32.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.525+0000 7fc7a37fe640 1 --2- 172.21.15.84:0/2412458241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc794005b80 0x7fc794005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.525+0000 7fc7a37fe640 1 --2- 172.21.15.84:0/2412458241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7a406b3e0 0x7fc7940069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.525+0000 7fc7a37fe640 1 --2- 172.21.15.84:0/2412458241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc794006f10 0x7fc794007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.525+0000 7fc7a37fe640 1 -- 172.21.15.84:0/2412458241 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a415de30 con 0x7fc794006f10 2023-10-16T11:26:32.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.525+0000 7fc7a37fe640 1 -- 172.21.15.84:0/2412458241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a415e110 con 0x7fc794005b80 2023-10-16T11:26:32.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.525+0000 7fc7a37fe640 1 -- 172.21.15.84:0/2412458241 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79400dcc0 con 0x7fc7a406b3e0 2023-10-16T11:26:32.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.525+0000 7fc7a9420640 1 --2- 172.21.15.84:0/2412458241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc794006f10 0x7fc794007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.525+0000 7fc7a8c1f640 1 --2- 172.21.15.84:0/2412458241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc794005b80 0x7fc794005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.526+0000 7fc7a9420640 1 -- 172.21.15.84:0/2412458241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7a406b3e0 msgr2=0x7fc7940069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:32.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.526+0000 7fc7a9420640 1 --2- 172.21.15.84:0/2412458241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7a406b3e0 0x7fc7940069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.526+0000 7fc7a9420640 1 -- 172.21.15.84:0/2412458241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc794005b80 msgr2=0x7fc794005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:32.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.526+0000 7fc7a9420640 1 --2- 172.21.15.84:0/2412458241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc794005b80 0x7fc794005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.526+0000 7fc7a9420640 1 -- 172.21.15.84:0/2412458241 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc794005730 con 0x7fc794006f10 2023-10-16T11:26:32.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.527+0000 7fc7a9420640 1 --2- 172.21.15.84:0/2412458241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc794006f10 0x7fc794007390 secure :-1 s=READY pgs=3167 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00ba30 tx=0x7fc794007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3175f40ba94842 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:32.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.527+0000 7fc7a37fe640 1 -- 172.21.15.84:0/2412458241 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c032680 con 0x7fc794006f10 2023-10-16T11:26:32.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.527+0000 7fc7a37fe640 1 -- 172.21.15.84:0/2412458241 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c0327e0 con 0x7fc794006f10 2023-10-16T11:26:32.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.527+0000 7fc7a37fe640 1 -- 172.21.15.84:0/2412458241 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c032b70 con 0x7fc794006f10 2023-10-16T11:26:32.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.528+0000 7fc7a9c21640 1 -- 172.21.15.84:0/2412458241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc794006f10 msgr2=0x7fc794007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:32.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.528+0000 7fc7a9c21640 1 --2- 172.21.15.84:0/2412458241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc794006f10 0x7fc794007390 secure :-1 s=READY pgs=3167 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00ba30 tx=0x7fc794007a00 comp rx=0 tx=0).stop 2023-10-16T11:26:32.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.528+0000 7fc7a9420640 1 -- 172.21.15.84:0/2412458241 reap_dead start 2023-10-16T11:26:32.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.528+0000 7fc7a9c21640 1 -- 172.21.15.84:0/2412458241 shutdown_connections 2023-10-16T11:26:32.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.528+0000 7fc7a9c21640 1 -- 172.21.15.84:0/2412458241 >> 172.21.15.84:0/2412458241 conn(0x7fc7a40b0170 msgr2=0x7fc7a40b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:32.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.528+0000 7fc7a9c21640 1 -- 172.21.15.84:0/2412458241 shutdown_connections 2023-10-16T11:26:32.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.528+0000 7fc7a9c21640 1 -- 172.21.15.84:0/2412458241 wait complete. 2023-10-16T11:26:32.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.530+0000 7fc7a9c21640 1 Processor -- start 2023-10-16T11:26:32.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.530+0000 7fc7a9c21640 1 -- start start 2023-10-16T11:26:32.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.531+0000 7fc7a9c21640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc794006f10 0x7fc7a4150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.531+0000 7fc7a9c21640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7a415c520 0x7fc7a415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.531+0000 7fc7a9c21640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7a40b3370 0x7fc7a40cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.531+0000 7fc7a9c21640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a40c59d0 con 0x7fc7a40b3370 2023-10-16T11:26:32.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.531+0000 7fc7a9c21640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a415d010 con 0x7fc794006f10 2023-10-16T11:26:32.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.531+0000 7fc7a9c21640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a4150a50 con 0x7fc7a415c520 2023-10-16T11:26:32.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.531+0000 7fc7a9420640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7a40b3370 0x7fc7a40cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.531+0000 7fc7a9420640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7a40b3370 0x7fc7a40cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46856/0 (socket says 172.21.15.84:46856) 2023-10-16T11:26:32.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.531+0000 7fc7a9420640 1 -- 172.21.15.84:0/219298404 learned_addr learned my addr 172.21.15.84:0/219298404 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:32.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.531+0000 7fc7a8c1f640 1 --2- 172.21.15.84:0/219298404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc794006f10 0x7fc7a4150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.531+0000 7fc7a3fff640 1 --2- 172.21.15.84:0/219298404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7a415c520 0x7fc7a415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.532+0000 7fc7a9420640 1 -- 172.21.15.84:0/219298404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7a415c520 msgr2=0x7fc7a415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:32.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.532+0000 7fc7a9420640 1 --2- 172.21.15.84:0/219298404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7a415c520 0x7fc7a415a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.532+0000 7fc7a9420640 1 -- 172.21.15.84:0/219298404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc794006f10 msgr2=0x7fc7a4150250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:32.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.532+0000 7fc7a9420640 1 --2- 172.21.15.84:0/219298404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc794006f10 0x7fc7a4150250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.532+0000 7fc7a9420640 1 -- 172.21.15.84:0/219298404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a4150d30 con 0x7fc7a40b3370 2023-10-16T11:26:32.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.532+0000 7fc7a3fff640 1 --2- 172.21.15.84:0/219298404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7a415c520 0x7fc7a415a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:32.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.532+0000 7fc7a9420640 1 --2- 172.21.15.84:0/219298404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7a40b3370 0x7fc7a40cd8d0 secure :-1 s=READY pgs=3168 cs=0 l=1 rev1=1 crypto rx=0x7fc79c02bf00 tx=0x7fc79c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:32.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.532+0000 7fc7a17fa640 1 -- 172.21.15.84:0/219298404 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c004020 con 0x7fc7a40b3370 2023-10-16T11:26:32.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.532+0000 7fc7a17fa640 1 -- 172.21.15.84:0/219298404 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c004180 con 0x7fc7a40b3370 2023-10-16T11:26:32.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.532+0000 7fc7a17fa640 1 -- 172.21.15.84:0/219298404 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0044f0 con 0x7fc7a40b3370 2023-10-16T11:26:32.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.532+0000 7fc7a9c21640 1 -- 172.21.15.84:0/219298404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a415ca50 con 0x7fc7a40b3370 2023-10-16T11:26:32.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.533+0000 7fc7a9c21640 1 -- 172.21.15.84:0/219298404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a4150790 con 0x7fc7a40b3370 2023-10-16T11:26:32.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.534+0000 7fc7a9c21640 1 -- 172.21.15.84:0/219298404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a40c79d0 con 0x7fc7a40b3370 2023-10-16T11:26:32.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.535+0000 7fc7a17fa640 1 -- 172.21.15.84:0/219298404 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fc79c03b6f0 con 0x7fc7a40b3370 2023-10-16T11:26:32.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.535+0000 7fc7a17fa640 1 --2- 172.21.15.84:0/219298404 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc78806b050 0x7fc78806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.536+0000 7fc7a17fa640 1 -- 172.21.15.84:0/219298404 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(249..249 src has 1..249) v4 ==== 5917+0+0 (secure 0 0 0) 0x7fc79c0abff0 con 0x7fc7a40b3370 2023-10-16T11:26:32.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.536+0000 7fc7a8c1f640 1 --2- 172.21.15.84:0/219298404 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc78806b050 0x7fc78806d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.536+0000 7fc7a8c1f640 1 --2- 172.21.15.84:0/219298404 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc78806b050 0x7fc78806d510 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc7980013d0 tx=0x7fc798009040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:32.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.540+0000 7fc7a17fa640 1 -- 172.21.15.84:0/219298404 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc7a40c79d0 con 0x7fc7a40b3370 2023-10-16T11:26:32.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.734+0000 7fc7a9c21640 1 -- 172.21.15.84:0/219298404 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fc7a40c79d0 con 0x7fc7a40b3370 2023-10-16T11:26:32.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.740+0000 7fc7a17fa640 1 -- 172.21.15.84:0/219298404 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v87) v1 ==== 115+0+0 (secure 0 0 0) 0x7fc7a40c79d0 con 0x7fc7a40b3370 2023-10-16T11:26:32.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.743+0000 7fc7a9c21640 1 -- 172.21.15.84:0/219298404 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc78806b050 msgr2=0x7fc78806d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:32.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.743+0000 7fc7a9c21640 1 --2- 172.21.15.84:0/219298404 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc78806b050 0x7fc78806d510 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc7980013d0 tx=0x7fc798009040 comp rx=0 tx=0).stop 2023-10-16T11:26:32.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.743+0000 7fc7a9c21640 1 -- 172.21.15.84:0/219298404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7a40b3370 msgr2=0x7fc7a40cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:32.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.743+0000 7fc7a9c21640 1 --2- 172.21.15.84:0/219298404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7a40b3370 0x7fc7a40cd8d0 secure :-1 s=READY pgs=3168 cs=0 l=1 rev1=1 crypto rx=0x7fc79c02bf00 tx=0x7fc79c004730 comp rx=0 tx=0).stop 2023-10-16T11:26:32.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.743+0000 7fc7a9c21640 1 -- 172.21.15.84:0/219298404 shutdown_connections 2023-10-16T11:26:32.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.743+0000 7fc7a9c21640 1 --2- 172.21.15.84:0/219298404 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc78806b050 0x7fc78806d510 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.743+0000 7fc7a9c21640 1 --2- 172.21.15.84:0/219298404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7a40b3370 0x7fc7a40cd8d0 unknown :-1 s=CLOSED pgs=3168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.743+0000 7fc7a9c21640 1 --2- 172.21.15.84:0/219298404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7a415c520 0x7fc7a415a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.743+0000 7fc7a9c21640 1 --2- 172.21.15.84:0/219298404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc794006f10 0x7fc7a4150250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.743+0000 7fc7a9c21640 1 -- 172.21.15.84:0/219298404 >> 172.21.15.84:0/219298404 conn(0x7fc7a40b0170 msgr2=0x7fc7a406c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:32.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.743+0000 7fc7a9c21640 1 -- 172.21.15.84:0/219298404 shutdown_connections 2023-10-16T11:26:32.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.743+0000 7fc7a9c21640 1 -- 172.21.15.84:0/219298404 wait complete. 2023-10-16T11:26:32.765 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument' 2023-10-16T11:26:32.810 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:32.807+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:32.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.969+0000 7fb88a361640 1 Processor -- start 2023-10-16T11:26:32.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.970+0000 7fb88a361640 1 -- start start 2023-10-16T11:26:32.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.972+0000 7fb88a361640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb884070fd0 0x7fb8840b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.972+0000 7fb88a361640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb884071450 con 0x7fb8840adf30 2023-10-16T11:26:32.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.972+0000 7fb88a361640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8840cc600 con 0x7fb8840adb50 2023-10-16T11:26:32.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.972+0000 7fb88a361640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8840cc8e0 con 0x7fb884070fd0 2023-10-16T11:26:32.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.972+0000 7fb87bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb884070fd0 0x7fb8840b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.972+0000 7fb87bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb884070fd0 0x7fb8840b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46860/0 (socket says 172.21.15.84:46860) 2023-10-16T11:26:32.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.972+0000 7fb87bfff640 1 -- 172.21.15.84:0/294618685 learned_addr learned my addr 172.21.15.84:0/294618685 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:32.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.972+0000 7fb8888fe640 1 --1- 172.21.15.84:0/294618685 >> v1:172.21.15.84:6789/0 conn(0x7fb8840adf30 0x7fb884164f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60820/0 (socket says 172.21.15.84:60820) 2023-10-16T11:26:32.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.973+0000 7fb87bfff640 1 -- 172.21.15.84:0/294618685 >> v1:172.21.15.157:6789/0 conn(0x7fb8840adb50 legacy=0x7fb88406f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:32.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.973+0000 7fb87bfff640 1 -- 172.21.15.84:0/294618685 >> v1:172.21.15.84:6789/0 conn(0x7fb8840adf30 legacy=0x7fb884164f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:32.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.973+0000 7fb87bfff640 1 -- 172.21.15.84:0/294618685 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8840ccbc0 con 0x7fb884070fd0 2023-10-16T11:26:32.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.974+0000 7fb87bfff640 1 --2- 172.21.15.84:0/294618685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb884070fd0 0x7fb8840b0410 secure :-1 s=READY pgs=3170 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009870 tx=0x7fb86c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d0b51e23050d94 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:32.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.974+0000 7fb8837fe640 1 -- 172.21.15.84:0/294618685 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c030860 con 0x7fb884070fd0 2023-10-16T11:26:32.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.974+0000 7fb8837fe640 1 -- 172.21.15.84:0/294618685 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c0309c0 con 0x7fb884070fd0 2023-10-16T11:26:32.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.974+0000 7fb8837fe640 1 -- 172.21.15.84:0/294618685 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c030d50 con 0x7fb884070fd0 2023-10-16T11:26:32.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.975+0000 7fb88a361640 1 -- 172.21.15.84:0/294618685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb884070fd0 msgr2=0x7fb8840b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:32.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.975+0000 7fb88a361640 1 --2- 172.21.15.84:0/294618685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb884070fd0 0x7fb8840b0410 secure :-1 s=READY pgs=3170 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009870 tx=0x7fb86c02b9a0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.975+0000 7fb88a361640 1 -- 172.21.15.84:0/294618685 shutdown_connections 2023-10-16T11:26:32.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.975+0000 7fb88a361640 1 --2- 172.21.15.84:0/294618685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb884070fd0 0x7fb8840b0410 unknown :-1 s=CLOSED pgs=3170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.975+0000 7fb88a361640 1 -- 172.21.15.84:0/294618685 >> 172.21.15.84:0/294618685 conn(0x7fb8840cb410 msgr2=0x7fb8840cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:32.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.975+0000 7fb88a361640 1 -- 172.21.15.84:0/294618685 shutdown_connections 2023-10-16T11:26:32.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.975+0000 7fb88a361640 1 -- 172.21.15.84:0/294618685 wait complete. 2023-10-16T11:26:32.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.977+0000 7fb88a361640 1 Processor -- start 2023-10-16T11:26:32.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.977+0000 7fb88a361640 1 -- start start 2023-10-16T11:26:32.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.978+0000 7fb88a361640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb884070fd0 0x7fb8841551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.978+0000 7fb88a361640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8840adb50 0x7fb884161a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.978+0000 7fb88a361640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8840adf30 0x7fb884161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.978+0000 7fb88a361640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb884163250 con 0x7fb8840adb50 2023-10-16T11:26:32.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.978+0000 7fb88a361640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8841625d0 con 0x7fb884070fd0 2023-10-16T11:26:32.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.978+0000 7fb88a361640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8841628b0 con 0x7fb8840adf30 2023-10-16T11:26:32.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.978+0000 7fb87bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8840adb50 0x7fb884161a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.979+0000 7fb87bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8840adb50 0x7fb884161a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46874/0 (socket says 172.21.15.84:46874) 2023-10-16T11:26:32.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.979+0000 7fb87bfff640 1 -- 172.21.15.84:0/642429167 learned_addr learned my addr 172.21.15.84:0/642429167 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:32.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.979+0000 7fb883fff640 1 --2- 172.21.15.84:0/642429167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb884070fd0 0x7fb8841551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.979+0000 7fb8888fe640 1 --2- 172.21.15.84:0/642429167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8840adf30 0x7fb884161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.979+0000 7fb87bfff640 1 -- 172.21.15.84:0/642429167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8840adf30 msgr2=0x7fb884161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:32.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.979+0000 7fb87bfff640 1 --2- 172.21.15.84:0/642429167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8840adf30 0x7fb884161fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.979+0000 7fb87bfff640 1 -- 172.21.15.84:0/642429167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb884070fd0 msgr2=0x7fb8841551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:32.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.979+0000 7fb87bfff640 1 --2- 172.21.15.84:0/642429167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb884070fd0 0x7fb8841551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:32.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.979+0000 7fb87bfff640 1 -- 172.21.15.84:0/642429167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c0094d0 con 0x7fb8840adb50 2023-10-16T11:26:32.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.979+0000 7fb87bfff640 1 --2- 172.21.15.84:0/642429167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8840adb50 0x7fb884161a60 secure :-1 s=READY pgs=3171 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009870 tx=0x7fb86c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:32.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.980+0000 7fb8817fa640 1 -- 172.21.15.84:0/642429167 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c030810 con 0x7fb8840adb50 2023-10-16T11:26:32.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.980+0000 7fb8817fa640 1 -- 172.21.15.84:0/642429167 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c039d90 con 0x7fb8840adb50 2023-10-16T11:26:32.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.980+0000 7fb8817fa640 1 -- 172.21.15.84:0/642429167 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c043630 con 0x7fb8840adb50 2023-10-16T11:26:32.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.980+0000 7fb88a361640 1 -- 172.21.15.84:0/642429167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb884162b90 con 0x7fb8840adb50 2023-10-16T11:26:32.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.980+0000 7fb88a361640 1 -- 172.21.15.84:0/642429167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb884155cc0 con 0x7fb8840adb50 2023-10-16T11:26:32.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.981+0000 7fb88a361640 1 -- 172.21.15.84:0/642429167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88404e160 con 0x7fb8840adb50 2023-10-16T11:26:32.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.982+0000 7fb8817fa640 1 -- 172.21.15.84:0/642429167 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fb86c02f3b0 con 0x7fb8840adb50 2023-10-16T11:26:32.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.983+0000 7fb8817fa640 1 --2- 172.21.15.84:0/642429167 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fb86406b020 0x7fb86406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:32.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.983+0000 7fb8817fa640 1 -- 172.21.15.84:0/642429167 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(249..249 src has 1..249) v4 ==== 5917+0+0 (secure 0 0 0) 0x7fb86c0abf50 con 0x7fb8840adb50 2023-10-16T11:26:32.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.983+0000 7fb883fff640 1 --2- 172.21.15.84:0/642429167 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fb86406b020 0x7fb86406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:32.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.984+0000 7fb883fff640 1 --2- 172.21.15.84:0/642429167 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fb86406b020 0x7fb86406d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb874004ea0 tx=0x7fb874009040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:32.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:32.987+0000 7fb8817fa640 1 -- 172.21.15.84:0/642429167 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb86c076cf0 con 0x7fb8840adb50 2023-10-16T11:26:33.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.257+0000 7fb88a361640 1 -- 172.21.15.84:0/642429167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]} v 0) v1 -- 0x7fb8840b4910 con 0x7fb8840adb50 2023-10-16T11:26:33.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.356+0000 7fb8817fa640 1 -- 172.21.15.84:0/642429167 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]}]=0 v3234) v1 ==== 187+0+0 (secure 0 0 0) 0x7fb86c07a310 con 0x7fb8840adb50 2023-10-16T11:26:33.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.360+0000 7fb88a361640 1 -- 172.21.15.84:0/642429167 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fb86406b020 msgr2=0x7fb86406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:33.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.360+0000 7fb88a361640 1 --2- 172.21.15.84:0/642429167 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fb86406b020 0x7fb86406d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb874004ea0 tx=0x7fb874009040 comp rx=0 tx=0).stop 2023-10-16T11:26:33.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.360+0000 7fb88a361640 1 -- 172.21.15.84:0/642429167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8840adb50 msgr2=0x7fb884161a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:33.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.360+0000 7fb88a361640 1 --2- 172.21.15.84:0/642429167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8840adb50 0x7fb884161a60 secure :-1 s=READY pgs=3171 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009870 tx=0x7fb86c004730 comp rx=0 tx=0).stop 2023-10-16T11:26:33.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.360+0000 7fb88a361640 1 -- 172.21.15.84:0/642429167 shutdown_connections 2023-10-16T11:26:33.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.360+0000 7fb88a361640 1 --2- 172.21.15.84:0/642429167 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fb86406b020 0x7fb86406d4e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:33.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.360+0000 7fb88a361640 1 --2- 172.21.15.84:0/642429167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8840adf30 0x7fb884161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:33.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.360+0000 7fb88a361640 1 --2- 172.21.15.84:0/642429167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8840adb50 0x7fb884161a60 unknown :-1 s=CLOSED pgs=3171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:33.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.360+0000 7fb88a361640 1 --2- 172.21.15.84:0/642429167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb884070fd0 0x7fb8841551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:33.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.360+0000 7fb88a361640 1 -- 172.21.15.84:0/642429167 >> 172.21.15.84:0/642429167 conn(0x7fb8840cb410 msgr2=0x7fb884160ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:33.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.361+0000 7fb88a361640 1 -- 172.21.15.84:0/642429167 shutdown_connections 2023-10-16T11:26:33.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.361+0000 7fb88a361640 1 -- 172.21.15.84:0/642429167 wait complete. 2023-10-16T11:26:33.385 INFO:tasks.cephfs_test_runner:Test command fails without --fs argument in presence of multiple fs ... ok 2023-10-16T11:26:33.386 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2023-10-16T11:26:33.386 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-10-16T11:26:33.387 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:26:33.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.643+0000 7ff446bf5640 1 Processor -- start 2023-10-16T11:26:33.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.643+0000 7ff446bf5640 1 -- start start 2023-10-16T11:26:33.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.645+0000 7ff446bf5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff440159f00 0x7ff4400b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:33.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.645+0000 7ff446bf5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff44015d880 0x7ff4400b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:33.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.645+0000 7ff446bf5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44006f7f0 con 0x7ff440159b20 2023-10-16T11:26:33.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.645+0000 7ff446bf5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44006c5e0 con 0x7ff44015d880 2023-10-16T11:26:33.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.645+0000 7ff446bf5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44006c8c0 con 0x7ff440159f00 2023-10-16T11:26:33.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.645+0000 7ff43ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff440159f00 0x7ff4400b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:33.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.645+0000 7ff445192640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff44015d880 0x7ff4400b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:33.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.646+0000 7ff445192640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff44015d880 0x7ff4400b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:38914/0 (socket says 172.21.15.84:38914) 2023-10-16T11:26:33.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.646+0000 7ff445192640 1 -- 172.21.15.84:0/1918107295 learned_addr learned my addr 172.21.15.84:0/1918107295 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:33.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.646+0000 7ff43ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff440159f00 0x7ff4400b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:45938/0 (socket says 172.21.15.84:45938) 2023-10-16T11:26:33.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.646+0000 7ff445192640 1 -- 172.21.15.84:0/1918107295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff440159f00 msgr2=0x7ff4400b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:33.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.646+0000 7ff445192640 1 --2- 172.21.15.84:0/1918107295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff440159f00 0x7ff4400b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:33.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.646+0000 7ff445192640 1 -- 172.21.15.84:0/1918107295 >> v1:172.21.15.157:6789/0 conn(0x7ff440159b20 legacy=0x7ff44006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:33.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.646+0000 7ff445192640 1 -- 172.21.15.84:0/1918107295 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44006cba0 con 0x7ff44015d880 2023-10-16T11:26:33.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.648+0000 7ff445192640 1 --2- 172.21.15.84:0/1918107295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff44015d880 0x7ff4400b0950 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff438009e50 tx=0x7ff43802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ecc431e6d6670bf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:33.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.648+0000 7ff43f7fe640 1 -- 172.21.15.84:0/1918107295 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43802cb00 con 0x7ff44015d880 2023-10-16T11:26:33.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.649+0000 7ff43f7fe640 1 -- 172.21.15.84:0/1918107295 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43802cc60 con 0x7ff44015d880 2023-10-16T11:26:33.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.649+0000 7ff43f7fe640 1 -- 172.21.15.84:0/1918107295 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4380343f0 con 0x7ff44015d880 2023-10-16T11:26:33.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.649+0000 7ff446bf5640 1 -- 172.21.15.84:0/1918107295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff44015d880 msgr2=0x7ff4400b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:33.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.649+0000 7ff446bf5640 1 --2- 172.21.15.84:0/1918107295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff44015d880 0x7ff4400b0950 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff438009e50 tx=0x7ff43802bf60 comp rx=0 tx=0).stop 2023-10-16T11:26:33.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.649+0000 7ff446bf5640 1 -- 172.21.15.84:0/1918107295 shutdown_connections 2023-10-16T11:26:33.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.649+0000 7ff446bf5640 1 --2- 172.21.15.84:0/1918107295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff44015d880 0x7ff4400b0950 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:33.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.649+0000 7ff446bf5640 1 --2- 172.21.15.84:0/1918107295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff440159f00 0x7ff4400b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:33.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.649+0000 7ff446bf5640 1 -- 172.21.15.84:0/1918107295 >> 172.21.15.84:0/1918107295 conn(0x7ff4400cb410 msgr2=0x7ff4400cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:33.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.649+0000 7ff446bf5640 1 -- 172.21.15.84:0/1918107295 shutdown_connections 2023-10-16T11:26:33.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.649+0000 7ff446bf5640 1 -- 172.21.15.84:0/1918107295 wait complete. 2023-10-16T11:26:33.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.651+0000 7ff446bf5640 1 Processor -- start 2023-10-16T11:26:33.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.651+0000 7ff446bf5640 1 -- start start 2023-10-16T11:26:33.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.652+0000 7ff446bf5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff440159b20 0x7ff4400b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:33.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.652+0000 7ff446bf5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff440159f00 0x7ff44015c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:33.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.652+0000 7ff446bf5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff44015d880 0x7ff440157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:33.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.652+0000 7ff446bf5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44015c7f0 con 0x7ff44015d880 2023-10-16T11:26:33.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.652+0000 7ff446bf5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4400b1920 con 0x7ff440159b20 2023-10-16T11:26:33.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.652+0000 7ff446bf5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4400b1c00 con 0x7ff440159f00 2023-10-16T11:26:33.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.653+0000 7ff445192640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff44015d880 0x7ff440157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:33.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.653+0000 7ff445192640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff44015d880 0x7ff440157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46886/0 (socket says 172.21.15.84:46886) 2023-10-16T11:26:33.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.653+0000 7ff445192640 1 -- 172.21.15.84:0/2811139451 learned_addr learned my addr 172.21.15.84:0/2811139451 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:33.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.653+0000 7ff444991640 1 --2- 172.21.15.84:0/2811139451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff440159b20 0x7ff4400b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:33.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.653+0000 7ff43ffff640 1 --2- 172.21.15.84:0/2811139451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff440159f00 0x7ff44015c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:33.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.653+0000 7ff445192640 1 -- 172.21.15.84:0/2811139451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff440159f00 msgr2=0x7ff44015c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:33.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.653+0000 7ff445192640 1 --2- 172.21.15.84:0/2811139451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff440159f00 0x7ff44015c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:33.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.653+0000 7ff445192640 1 -- 172.21.15.84:0/2811139451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff440159b20 msgr2=0x7ff4400b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:33.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.653+0000 7ff445192640 1 --2- 172.21.15.84:0/2811139451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff440159b20 0x7ff4400b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:33.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.653+0000 7ff445192640 1 -- 172.21.15.84:0/2811139451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff438009b30 con 0x7ff44015d880 2023-10-16T11:26:33.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.654+0000 7ff445192640 1 --2- 172.21.15.84:0/2811139451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff44015d880 0x7ff440157fc0 secure :-1 s=READY pgs=3172 cs=0 l=1 rev1=1 crypto rx=0x7ff438009e50 tx=0x7ff438004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:33.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.654+0000 7ff43d7fa640 1 -- 172.21.15.84:0/2811139451 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438009510 con 0x7ff44015d880 2023-10-16T11:26:33.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.654+0000 7ff43d7fa640 1 -- 172.21.15.84:0/2811139451 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43803e450 con 0x7ff44015d880 2023-10-16T11:26:33.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.654+0000 7ff43d7fa640 1 -- 172.21.15.84:0/2811139451 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43803e800 con 0x7ff44015d880 2023-10-16T11:26:33.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.654+0000 7ff446bf5640 1 -- 172.21.15.84:0/2811139451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4400b1ee0 con 0x7ff44015d880 2023-10-16T11:26:33.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.654+0000 7ff446bf5640 1 -- 172.21.15.84:0/2811139451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4400b24a0 con 0x7ff44015d880 2023-10-16T11:26:33.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.655+0000 7ff446bf5640 1 -- 172.21.15.84:0/2811139451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44004e160 con 0x7ff44015d880 2023-10-16T11:26:33.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.656+0000 7ff43d7fa640 1 -- 172.21.15.84:0/2811139451 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7ff43803e960 con 0x7ff44015d880 2023-10-16T11:26:33.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.658+0000 7ff43d7fa640 1 --2- 172.21.15.84:0/2811139451 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7ff42406b020 0x7ff42406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:33.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.658+0000 7ff43d7fa640 1 -- 172.21.15.84:0/2811139451 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(249..249 src has 1..249) v4 ==== 5917+0+0 (secure 0 0 0) 0x7ff4380ac7c0 con 0x7ff44015d880 2023-10-16T11:26:33.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.658+0000 7ff444991640 1 --2- 172.21.15.84:0/2811139451 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7ff42406b020 0x7ff42406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:33.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.659+0000 7ff444991640 1 --2- 172.21.15.84:0/2811139451 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7ff42406b020 0x7ff42406d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff434000f60 tx=0x7ff434009040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:33.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.662+0000 7ff43d7fa640 1 -- 172.21.15.84:0/2811139451 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff438077480 con 0x7ff44015d880 2023-10-16T11:26:33.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.855+0000 7ff446bf5640 1 -- 172.21.15.84:0/2811139451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4400b10b0 con 0x7ff44015d880 2023-10-16T11:26:33.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.857+0000 7ff43d7fa640 1 -- 172.21.15.84:0/2811139451 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v249) v1 ==== 74+0+14388 (secure 0 0 0) 0x7ff43807aaa0 con 0x7ff44015d880 2023-10-16T11:26:33.859 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:33.859 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":249,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:26:31.316920+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6834/369067539":"2023-10-17T11:26:28.045084+0000","172.21.15.84:6837/3547503033":"2023-10-17T11:26:20.041719+0000","172.21.15.84:0/2606713455":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/488945617":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/4036998934":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6832/3938045451":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/570042115":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1311117536":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6833/2963529201":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/95207886":"2023-10-17T11:26:14.475502+0000","172.21.15.84:6832/2963529201":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/388205040":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/2303342316":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/648102622":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/133211355":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1263886961":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1489017743":"2023-10-17T11:26:08.077644+0000","172.21.15.84:6835/369067539":"2023-10-17T11:26:28.045084+0000","172.21.15.84:6833/3938045451":"2023-10-17T11:26:14.475502+0000","172.21.15.84:0/1517291402":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1983743360":"2023-10-17T11:26:08.077644+0000","172.21.15.84:0/1323010124":"2023-10-17T11:26:08.077644+0000","172.21.15.84:6836/3547503033":"2023-10-17T11:26:20.041719+0000","172.21.15.84:0/437386817":"2023-10-17T11:26:08.077644+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:26:33.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.859+0000 7ff446bf5640 1 -- 172.21.15.84:0/2811139451 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7ff42406b020 msgr2=0x7ff42406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:33.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.859+0000 7ff446bf5640 1 --2- 172.21.15.84:0/2811139451 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7ff42406b020 0x7ff42406d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff434000f60 tx=0x7ff434009040 comp rx=0 tx=0).stop 2023-10-16T11:26:33.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.859+0000 7ff446bf5640 1 -- 172.21.15.84:0/2811139451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff44015d880 msgr2=0x7ff440157fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:33.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.859+0000 7ff446bf5640 1 --2- 172.21.15.84:0/2811139451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff44015d880 0x7ff440157fc0 secure :-1 s=READY pgs=3172 cs=0 l=1 rev1=1 crypto rx=0x7ff438009e50 tx=0x7ff438004a40 comp rx=0 tx=0).stop 2023-10-16T11:26:33.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.859+0000 7ff446bf5640 1 -- 172.21.15.84:0/2811139451 shutdown_connections 2023-10-16T11:26:33.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.859+0000 7ff446bf5640 1 --2- 172.21.15.84:0/2811139451 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7ff42406b020 0x7ff42406d4e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:33.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.859+0000 7ff446bf5640 1 --2- 172.21.15.84:0/2811139451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff44015d880 0x7ff440157fc0 unknown :-1 s=CLOSED pgs=3172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:33.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.859+0000 7ff446bf5640 1 --2- 172.21.15.84:0/2811139451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff440159f00 0x7ff44015c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:33.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.859+0000 7ff446bf5640 1 --2- 172.21.15.84:0/2811139451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff440159b20 0x7ff4400b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:33.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.859+0000 7ff446bf5640 1 -- 172.21.15.84:0/2811139451 >> 172.21.15.84:0/2811139451 conn(0x7ff4400cb410 msgr2=0x7ff44006fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:33.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.859+0000 7ff446bf5640 1 -- 172.21.15.84:0/2811139451 shutdown_connections 2023-10-16T11:26:33.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:33.859+0000 7ff446bf5640 1 -- 172.21.15.84:0/2811139451 wait complete. 2023-10-16T11:26:33.879 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:26:33.880 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:26:34.055 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:34.051+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:34.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.096+0000 7fa584ca2640 1 Processor -- start 2023-10-16T11:26:34.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.097+0000 7fa584ca2640 1 -- start start 2023-10-16T11:26:34.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.098+0000 7fa584ca2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa58015ce00 0x7fa580160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:34.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.098+0000 7fa584ca2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa58006ea40 0x7fa580069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:34.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.098+0000 7fa584ca2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa58006a730 con 0x7fa58006a2e0 2023-10-16T11:26:34.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.098+0000 7fa584ca2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5800cdc60 con 0x7fa58006ea40 2023-10-16T11:26:34.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.098+0000 7fa584ca2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5801618b0 con 0x7fa58015ce00 2023-10-16T11:26:34.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.098+0000 7fa577fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa58006ea40 0x7fa580069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:34.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.099+0000 7fa577fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa58006ea40 0x7fa580069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46898/0 (socket says 172.21.15.84:46898) 2023-10-16T11:26:34.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.099+0000 7fa577fff640 1 -- 172.21.15.84:0/1986464019 learned_addr learned my addr 172.21.15.84:0/1986464019 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:34.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.099+0000 7fa57e59c640 1 --2- 172.21.15.84:0/1986464019 >> v2:172.21.15.157:3300/0 conn(0x7fa58015ce00 0x7fa580160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:34.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.099+0000 7fa577fff640 1 -- 172.21.15.84:0/1986464019 >> v2:172.21.15.157:3300/0 conn(0x7fa58015ce00 msgr2=0x7fa580160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:34.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.099+0000 7fa577fff640 1 --2- 172.21.15.84:0/1986464019 >> v2:172.21.15.157:3300/0 conn(0x7fa58015ce00 0x7fa580160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:34.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.099+0000 7fa577fff640 1 -- 172.21.15.84:0/1986464019 >> v1:172.21.15.157:6789/0 conn(0x7fa58006a2e0 legacy=0x7fa580070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:34.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.099+0000 7fa577fff640 1 -- 172.21.15.84:0/1986464019 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa580161010 con 0x7fa58006ea40 2023-10-16T11:26:34.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.100+0000 7fa577fff640 1 --2- 172.21.15.84:0/1986464019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa58006ea40 0x7fa580069da0 secure :-1 s=READY pgs=3173 cs=0 l=1 rev1=1 crypto rx=0x7fa5800cd560 tx=0x7fa56802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2aa63f156683cc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:34.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.101+0000 7fa57dd9b640 1 -- 172.21.15.84:0/1986464019 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5680317e0 con 0x7fa58006ea40 2023-10-16T11:26:34.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.101+0000 7fa57dd9b640 1 -- 172.21.15.84:0/1986464019 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa568031940 con 0x7fa58006ea40 2023-10-16T11:26:34.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.101+0000 7fa57dd9b640 1 -- 172.21.15.84:0/1986464019 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568031cd0 con 0x7fa58006ea40 2023-10-16T11:26:34.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.101+0000 7fa584ca2640 1 -- 172.21.15.84:0/1986464019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa58006ea40 msgr2=0x7fa580069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:34.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.101+0000 7fa584ca2640 1 --2- 172.21.15.84:0/1986464019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa58006ea40 0x7fa580069da0 secure :-1 s=READY pgs=3173 cs=0 l=1 rev1=1 crypto rx=0x7fa5800cd560 tx=0x7fa56802b880 comp rx=0 tx=0).stop 2023-10-16T11:26:34.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.101+0000 7fa584ca2640 1 -- 172.21.15.84:0/1986464019 shutdown_connections 2023-10-16T11:26:34.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.101+0000 7fa584ca2640 1 --2- 172.21.15.84:0/1986464019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa58006ea40 0x7fa580069da0 unknown :-1 s=CLOSED pgs=3173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:34.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.101+0000 7fa584ca2640 1 --2- 172.21.15.84:0/1986464019 >> v2:172.21.15.157:3300/0 conn(0x7fa58015ce00 0x7fa580160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:34.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.101+0000 7fa584ca2640 1 -- 172.21.15.84:0/1986464019 >> 172.21.15.84:0/1986464019 conn(0x7fa5800cc460 msgr2=0x7fa5800cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:34.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.102+0000 7fa584ca2640 1 -- 172.21.15.84:0/1986464019 shutdown_connections 2023-10-16T11:26:34.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.102+0000 7fa584ca2640 1 -- 172.21.15.84:0/1986464019 wait complete. 2023-10-16T11:26:34.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.103+0000 7fa584ca2640 1 Processor -- start 2023-10-16T11:26:34.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.104+0000 7fa584ca2640 1 -- start start 2023-10-16T11:26:34.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.104+0000 7fa584ca2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa58006a2e0 0x7fa5800ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:34.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.104+0000 7fa584ca2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa58006ea40 0x7fa58015af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:34.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.105+0000 7fa584ca2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa58015ce00 0x7fa5800acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:34.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.105+0000 7fa584ca2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58015ec20 con 0x7fa58015ce00 2023-10-16T11:26:34.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.105+0000 7fa584ca2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5800c3850 con 0x7fa58006a2e0 2023-10-16T11:26:34.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.105+0000 7fa584ca2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5800c3b30 con 0x7fa58006ea40 2023-10-16T11:26:34.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.105+0000 7fa57ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa58015ce00 0x7fa5800acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:34.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.105+0000 7fa57ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa58015ce00 0x7fa5800acd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46914/0 (socket says 172.21.15.84:46914) 2023-10-16T11:26:34.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.105+0000 7fa57ed9d640 1 -- 172.21.15.84:0/3900041518 learned_addr learned my addr 172.21.15.84:0/3900041518 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:34.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.105+0000 7fa57e59c640 1 --2- 172.21.15.84:0/3900041518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa58006a2e0 0x7fa5800ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:34.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.105+0000 7fa577fff640 1 --2- 172.21.15.84:0/3900041518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa58006ea40 0x7fa58015af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:34.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.105+0000 7fa57ed9d640 1 -- 172.21.15.84:0/3900041518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa58006ea40 msgr2=0x7fa58015af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:34.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.105+0000 7fa57ed9d640 1 --2- 172.21.15.84:0/3900041518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa58006ea40 0x7fa58015af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:34.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.105+0000 7fa57ed9d640 1 -- 172.21.15.84:0/3900041518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa58006a2e0 msgr2=0x7fa5800ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:34.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.105+0000 7fa57ed9d640 1 --2- 172.21.15.84:0/3900041518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa58006a2e0 0x7fa5800ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:34.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.105+0000 7fa57ed9d640 1 -- 172.21.15.84:0/3900041518 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5680094d0 con 0x7fa58015ce00 2023-10-16T11:26:34.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.105+0000 7fa577fff640 1 --2- 172.21.15.84:0/3900041518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa58006ea40 0x7fa58015af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:34.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.106+0000 7fa57ed9d640 1 --2- 172.21.15.84:0/3900041518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa58015ce00 0x7fa5800acd50 secure :-1 s=READY pgs=3174 cs=0 l=1 rev1=1 crypto rx=0x7fa57800c780 tx=0x7fa57800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:34.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.106+0000 7fa576ffd640 1 -- 172.21.15.84:0/3900041518 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa578003d80 con 0x7fa58015ce00 2023-10-16T11:26:34.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.106+0000 7fa576ffd640 1 -- 172.21.15.84:0/3900041518 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57800ce90 con 0x7fa58015ce00 2023-10-16T11:26:34.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.106+0000 7fa576ffd640 1 -- 172.21.15.84:0/3900041518 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5780052b0 con 0x7fa58015ce00 2023-10-16T11:26:34.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.106+0000 7fa584ca2640 1 -- 172.21.15.84:0/3900041518 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5800c3e10 con 0x7fa58015ce00 2023-10-16T11:26:34.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.106+0000 7fa584ca2640 1 -- 172.21.15.84:0/3900041518 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5800c3490 con 0x7fa58015ce00 2023-10-16T11:26:34.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.107+0000 7fa584ca2640 1 -- 172.21.15.84:0/3900041518 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5800c4050 con 0x7fa58015ce00 2023-10-16T11:26:34.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.108+0000 7fa576ffd640 1 -- 172.21.15.84:0/3900041518 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fa578005410 con 0x7fa58015ce00 2023-10-16T11:26:34.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.109+0000 7fa576ffd640 1 --2- 172.21.15.84:0/3900041518 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fa56006b050 0x7fa56006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:34.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.109+0000 7fa576ffd640 1 -- 172.21.15.84:0/3900041518 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(249..249 src has 1..249) v4 ==== 5917+0+0 (secure 0 0 0) 0x7fa578088ee0 con 0x7fa58015ce00 2023-10-16T11:26:34.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.110+0000 7fa57e59c640 1 --2- 172.21.15.84:0/3900041518 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fa56006b050 0x7fa56006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:34.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.110+0000 7fa57e59c640 1 --2- 172.21.15.84:0/3900041518 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fa56006b050 0x7fa56006d510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa570006fd0 tx=0x7fa570008040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:34.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.114+0000 7fa576ffd640 1 -- 172.21.15.84:0/3900041518 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa5800c4050 con 0x7fa58015ce00 2023-10-16T11:26:34.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.314+0000 7fa584ca2640 1 -- 172.21.15.84:0/3900041518 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5800c4000 con 0x7fa58015ce00 2023-10-16T11:26:34.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.315+0000 7fa576ffd640 1 -- 172.21.15.84:0/3900041518 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v249) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa578017020 con 0x7fa58015ce00 2023-10-16T11:26:34.318 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:26:34.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.318+0000 7fa584ca2640 1 -- 172.21.15.84:0/3900041518 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fa56006b050 msgr2=0x7fa56006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:34.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.318+0000 7fa584ca2640 1 --2- 172.21.15.84:0/3900041518 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fa56006b050 0x7fa56006d510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa570006fd0 tx=0x7fa570008040 comp rx=0 tx=0).stop 2023-10-16T11:26:34.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.318+0000 7fa584ca2640 1 -- 172.21.15.84:0/3900041518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa58015ce00 msgr2=0x7fa5800acd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:34.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.318+0000 7fa584ca2640 1 --2- 172.21.15.84:0/3900041518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa58015ce00 0x7fa5800acd50 secure :-1 s=READY pgs=3174 cs=0 l=1 rev1=1 crypto rx=0x7fa57800c780 tx=0x7fa57800cc40 comp rx=0 tx=0).stop 2023-10-16T11:26:34.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.318+0000 7fa584ca2640 1 -- 172.21.15.84:0/3900041518 shutdown_connections 2023-10-16T11:26:34.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.318+0000 7fa584ca2640 1 --2- 172.21.15.84:0/3900041518 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fa56006b050 0x7fa56006d510 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:34.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.318+0000 7fa584ca2640 1 --2- 172.21.15.84:0/3900041518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa58015ce00 0x7fa5800acd50 unknown :-1 s=CLOSED pgs=3174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:34.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.318+0000 7fa584ca2640 1 --2- 172.21.15.84:0/3900041518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa58006ea40 0x7fa58015af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:34.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.318+0000 7fa584ca2640 1 --2- 172.21.15.84:0/3900041518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa58006a2e0 0x7fa5800ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:34.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.318+0000 7fa584ca2640 1 -- 172.21.15.84:0/3900041518 >> 172.21.15.84:0/3900041518 conn(0x7fa5800cc460 msgr2=0x7fa58006c660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:34.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.318+0000 7fa584ca2640 1 -- 172.21.15.84:0/3900041518 shutdown_connections 2023-10-16T11:26:34.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.318+0000 7fa584ca2640 1 -- 172.21.15.84:0/3900041518 wait complete. 2023-10-16T11:26:34.338 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-10-16T11:26:34.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.537+0000 7f2221090640 1 Processor -- start 2023-10-16T11:26:34.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.537+0000 7f2221090640 1 -- start start 2023-10-16T11:26:34.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.538+0000 7f2221090640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f221c15ae80 0x7f221c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:34.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.539+0000 7f2221090640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f221c0ce390 0x7f221c070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:34.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.539+0000 7f2221090640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f221c15ed60 con 0x7f221c15aaa0 2023-10-16T11:26:34.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.539+0000 7f2221090640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f221c06b810 con 0x7f221c15ae80 2023-10-16T11:26:34.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.539+0000 7f2221090640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221c0707e0 con 0x7f221c0ce390 2023-10-16T11:26:34.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.539+0000 7f221ad9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f221c15aaa0 0x7f221c0cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60830/0 (socket says 172.21.15.84:60830) 2023-10-16T11:26:34.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.539+0000 7f221ad9d640 1 -- 172.21.15.84:0/3945627163 learned_addr learned my addr 172.21.15.84:0/3945627163 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:34.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.539+0000 7f2219d9b640 1 -- 172.21.15.84:0/3945627163 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779908947 0 0) 0x7f221c15ed60 con 0x7f221c15aaa0 2023-10-16T11:26:34.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.539+0000 7f2219d9b640 1 -- 172.21.15.84:0/3945627163 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2204003520 con 0x7f221c15aaa0 2023-10-16T11:26:34.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.539+0000 7f221a59c640 1 --2- 172.21.15.84:0/3945627163 >> v2:172.21.15.157:3300/0 conn(0x7f221c15ae80 0x7f221c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:34.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.539+0000 7f221b59e640 1 --2- 172.21.15.84:0/3945627163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f221c0ce390 0x7f221c070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:34.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f2219d9b640 1 -- 172.21.15.84:0/3945627163 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 753978644 0 0) 0x7f2204003520 con 0x7f221c15aaa0 2023-10-16T11:26:34.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f2219d9b640 1 -- 172.21.15.84:0/3945627163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f221c0ce390 msgr2=0x7f221c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:34.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f2219d9b640 1 --2- 172.21.15.84:0/3945627163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f221c0ce390 0x7f221c070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:34.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f2219d9b640 1 -- 172.21.15.84:0/3945627163 >> v2:172.21.15.157:3300/0 conn(0x7f221c15ae80 msgr2=0x7f221c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:34.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f2219d9b640 1 --2- 172.21.15.84:0/3945627163 >> v2:172.21.15.157:3300/0 conn(0x7f221c15ae80 0x7f221c15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:34.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f2219d9b640 1 -- 172.21.15.84:0/3945627163 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221c15ef90 con 0x7f221c15aaa0 2023-10-16T11:26:34.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f2219d9b640 1 -- 172.21.15.84:0/3945627163 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f220c001690 con 0x7f221c15aaa0 2023-10-16T11:26:34.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f2219d9b640 1 -- 172.21.15.84:0/3945627163 >> v1:172.21.15.84:6789/0 conn(0x7f221c15aaa0 legacy=0x7f221c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:34.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f2219d9b640 1 --2- 172.21.15.84:0/3945627163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f221c0ce390 0x7f2204003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:34.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f2219d9b640 1 --2- 172.21.15.84:0/3945627163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2204003fa0 0x7f22040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:34.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f2219d9b640 1 --2- 172.21.15.84:0/3945627163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2204004d00 0x7f2204005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:34.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f2219d9b640 1 -- 172.21.15.84:0/3945627163 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221c06b810 con 0x7f2204004d00 2023-10-16T11:26:34.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f2219d9b640 1 -- 172.21.15.84:0/3945627163 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221c0707e0 con 0x7f2204003fa0 2023-10-16T11:26:34.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f2219d9b640 1 -- 172.21.15.84:0/3945627163 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220400bb90 con 0x7f221c0ce390 2023-10-16T11:26:34.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f221a59c640 1 --2- 172.21.15.84:0/3945627163 >> v2:172.21.15.157:3300/0 conn(0x7f221c15ae80 0x7f221c15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:34.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f221b59e640 1 --2- 172.21.15.84:0/3945627163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2204004d00 0x7f2204005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:34.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f221b59e640 1 -- 172.21.15.84:0/3945627163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f221c0ce390 msgr2=0x7f2204003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:34.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f221b59e640 1 --2- 172.21.15.84:0/3945627163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f221c0ce390 0x7f2204003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:34.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f221b59e640 1 -- 172.21.15.84:0/3945627163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2204003fa0 msgr2=0x7f22040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:34.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f221b59e640 1 --2- 172.21.15.84:0/3945627163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2204003fa0 0x7f22040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:34.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.540+0000 7f221b59e640 1 -- 172.21.15.84:0/3945627163 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2204003520 con 0x7f2204004d00 2023-10-16T11:26:34.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.541+0000 7f221b59e640 1 --2- 172.21.15.84:0/3945627163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2204004d00 0x7f2204005180 secure :-1 s=READY pgs=3176 cs=0 l=1 rev1=1 crypto rx=0x7f2214007360 tx=0x7f22040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4a8d6fbfed8eff server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:34.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.542+0000 7f2219d9b640 1 -- 172.21.15.84:0/3945627163 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221402ccf0 con 0x7f2204004d00 2023-10-16T11:26:34.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.542+0000 7f2219d9b640 1 -- 172.21.15.84:0/3945627163 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221402ce50 con 0x7f2204004d00 2023-10-16T11:26:34.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.542+0000 7f2219d9b640 1 -- 172.21.15.84:0/3945627163 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2214034660 con 0x7f2204004d00 2023-10-16T11:26:34.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.542+0000 7f2221090640 1 -- 172.21.15.84:0/3945627163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2204004d00 msgr2=0x7f2204005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:34.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.542+0000 7f2221090640 1 --2- 172.21.15.84:0/3945627163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2204004d00 0x7f2204005180 secure :-1 s=READY pgs=3176 cs=0 l=1 rev1=1 crypto rx=0x7f2214007360 tx=0x7f22040057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:34.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.542+0000 7f221b59e640 1 -- 172.21.15.84:0/3945627163 reap_dead start 2023-10-16T11:26:34.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.542+0000 7f2221090640 1 -- 172.21.15.84:0/3945627163 shutdown_connections 2023-10-16T11:26:34.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.542+0000 7f2221090640 1 -- 172.21.15.84:0/3945627163 >> 172.21.15.84:0/3945627163 conn(0x7f221c0cb410 msgr2=0x7f221c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:34.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.542+0000 7f2221090640 1 -- 172.21.15.84:0/3945627163 shutdown_connections 2023-10-16T11:26:34.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.542+0000 7f2221090640 1 -- 172.21.15.84:0/3945627163 wait complete. 2023-10-16T11:26:34.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.544+0000 7f2221090640 1 Processor -- start 2023-10-16T11:26:34.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.545+0000 7f2221090640 1 -- start start 2023-10-16T11:26:34.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.546+0000 7f2221090640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2204004d00 0x7f221c15dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:34.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.546+0000 7f2221090640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221c0c4bd0 0x7f221c1f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:34.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.546+0000 7f2221090640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f221c0c5050 0x7f221c1f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:34.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.546+0000 7f2221090640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221c1f2330 con 0x7f221c0c5050 2023-10-16T11:26:34.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.546+0000 7f2221090640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221c1f3240 con 0x7f221c0c4bd0 2023-10-16T11:26:34.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.546+0000 7f2221090640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221c15d160 con 0x7f2204004d00 2023-10-16T11:26:34.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.546+0000 7f221b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f221c0c5050 0x7f221c1f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:34.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.546+0000 7f221b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f221c0c5050 0x7f221c1f1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46934/0 (socket says 172.21.15.84:46934) 2023-10-16T11:26:34.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.546+0000 7f221b59e640 1 -- 172.21.15.84:0/1059297683 learned_addr learned my addr 172.21.15.84:0/1059297683 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:34.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.546+0000 7f221a59c640 1 --2- 172.21.15.84:0/1059297683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221c0c4bd0 0x7f221c1f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:34.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.546+0000 7f221b59e640 1 -- 172.21.15.84:0/1059297683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2204004d00 msgr2=0x7f221c15dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:34.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.546+0000 7f221b59e640 1 --2- 172.21.15.84:0/1059297683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2204004d00 0x7f221c15dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:34.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.546+0000 7f221b59e640 1 -- 172.21.15.84:0/1059297683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221c0c4bd0 msgr2=0x7f221c1f18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:34.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.546+0000 7f221b59e640 1 --2- 172.21.15.84:0/1059297683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221c0c4bd0 0x7f221c1f18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:34.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.546+0000 7f221b59e640 1 -- 172.21.15.84:0/1059297683 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221c15d440 con 0x7f221c0c5050 2023-10-16T11:26:34.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.547+0000 7f221b59e640 1 --2- 172.21.15.84:0/1059297683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f221c0c5050 0x7f221c1f1df0 secure :-1 s=READY pgs=3177 cs=0 l=1 rev1=1 crypto rx=0x7f221402dfd0 tx=0x7f2214004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:34.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.547+0000 7f22037fe640 1 -- 172.21.15.84:0/1059297683 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221402ccf0 con 0x7f221c0c5050 2023-10-16T11:26:34.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.547+0000 7f22037fe640 1 -- 172.21.15.84:0/1059297683 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221403d400 con 0x7f221c0c5050 2023-10-16T11:26:34.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.547+0000 7f2221090640 1 -- 172.21.15.84:0/1059297683 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f221c15d720 con 0x7f221c0c5050 2023-10-16T11:26:34.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.547+0000 7f22037fe640 1 -- 172.21.15.84:0/1059297683 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221403d7b0 con 0x7f221c0c5050 2023-10-16T11:26:34.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.547+0000 7f2221090640 1 -- 172.21.15.84:0/1059297683 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f221c15e180 con 0x7f221c0c5050 2023-10-16T11:26:34.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.548+0000 7f2221090640 1 -- 172.21.15.84:0/1059297683 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f221c15d910 con 0x7f221c0c5050 2023-10-16T11:26:34.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.549+0000 7f22037fe640 1 -- 172.21.15.84:0/1059297683 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f221403b040 con 0x7f221c0c5050 2023-10-16T11:26:34.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.550+0000 7f22037fe640 1 --2- 172.21.15.84:0/1059297683 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f220806b020 0x7f220806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:34.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.550+0000 7f22037fe640 1 -- 172.21.15.84:0/1059297683 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(249..249 src has 1..249) v4 ==== 5917+0+0 (secure 0 0 0) 0x7f22140ab810 con 0x7f221c0c5050 2023-10-16T11:26:34.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.550+0000 7f221ad9d640 1 --2- 172.21.15.84:0/1059297683 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f220806b020 0x7f220806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:34.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.551+0000 7f221ad9d640 1 --2- 172.21.15.84:0/1059297683 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f220806b020 0x7f220806d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f220c000f60 tx=0x7f220c0015c0 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:34.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.554+0000 7f22037fe640 1 -- 172.21.15.84:0/1059297683 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2214076680 con 0x7f221c0c5050 2023-10-16T11:26:34.785 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:26:34.794 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:26:34.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:34.827+0000 7f2221090640 1 -- 172.21.15.84:0/1059297683 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f221c04e160 con 0x7f221c0c5050 2023-10-16T11:26:35.272 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:35.269+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:35.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.368+0000 7f22037fe640 1 -- 172.21.15.84:0/1059297683 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3236) v1 ==== 158+0+0 (secure 0 0 0) 0x7f22140440e0 con 0x7f221c0c5050 2023-10-16T11:26:35.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.373+0000 7f2221090640 1 -- 172.21.15.84:0/1059297683 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f220806b020 msgr2=0x7f220806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:35.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.373+0000 7f2221090640 1 --2- 172.21.15.84:0/1059297683 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f220806b020 0x7f220806d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f220c000f60 tx=0x7f220c0015c0 comp rx=0 tx=0).stop 2023-10-16T11:26:35.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.373+0000 7f2221090640 1 -- 172.21.15.84:0/1059297683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f221c0c5050 msgr2=0x7f221c1f1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:35.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.373+0000 7f2221090640 1 --2- 172.21.15.84:0/1059297683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f221c0c5050 0x7f221c1f1df0 secure :-1 s=READY pgs=3177 cs=0 l=1 rev1=1 crypto rx=0x7f221402dfd0 tx=0x7f2214004ab0 comp rx=0 tx=0).stop 2023-10-16T11:26:35.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.373+0000 7f2221090640 1 -- 172.21.15.84:0/1059297683 shutdown_connections 2023-10-16T11:26:35.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.373+0000 7f2221090640 1 --2- 172.21.15.84:0/1059297683 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f220806b020 0x7f220806d4e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:35.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.373+0000 7f2221090640 1 --2- 172.21.15.84:0/1059297683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f221c0c5050 0x7f221c1f1df0 unknown :-1 s=CLOSED pgs=3177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:35.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.373+0000 7f2221090640 1 --2- 172.21.15.84:0/1059297683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221c0c4bd0 0x7f221c1f18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:35.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.373+0000 7f2221090640 1 --2- 172.21.15.84:0/1059297683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2204004d00 0x7f221c15dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:35.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.373+0000 7f2221090640 1 -- 172.21.15.84:0/1059297683 >> 172.21.15.84:0/1059297683 conn(0x7f221c0cb410 msgr2=0x7f221c15a090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:35.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.373+0000 7f2221090640 1 -- 172.21.15.84:0/1059297683 shutdown_connections 2023-10-16T11:26:35.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.373+0000 7f2221090640 1 -- 172.21.15.84:0/1059297683 wait complete. 2023-10-16T11:26:35.396 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T11:26:35.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.612+0000 7f66004c2640 1 Processor -- start 2023-10-16T11:26:35.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.612+0000 7f66004c2640 1 -- start start 2023-10-16T11:26:35.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.614+0000 7f66004c2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f8159ab0 0x7f65f80ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:35.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.614+0000 7f66004c2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65f8159e90 0x7f65f806fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:35.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.614+0000 7f66004c2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f80cd1f0 con 0x7f65f806c140 2023-10-16T11:26:35.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.614+0000 7f66004c2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f815d890 con 0x7f65f8159e90 2023-10-16T11:26:35.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.614+0000 7f66004c2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80cd4d0 con 0x7f65f8159ab0 2023-10-16T11:26:35.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.614+0000 7f65fea5f640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f65f806c140 0x7f65f815dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38762/0 (socket says 172.21.15.84:38762) 2023-10-16T11:26:35.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.614+0000 7f65fea5f640 1 -- 172.21.15.84:0/3471026963 learned_addr learned my addr 172.21.15.84:0/3471026963 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:35.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fda5d640 1 --2- 172.21.15.84:0/3471026963 >> v2:172.21.15.157:3300/0 conn(0x7f65f8159e90 0x7f65f806fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:35.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fd25c640 1 -- 172.21.15.84:0/3471026963 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104287796 0 0) 0x7f65f80cd1f0 con 0x7f65f806c140 2023-10-16T11:26:35.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fd25c640 1 -- 172.21.15.84:0/3471026963 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e4003520 con 0x7f65f806c140 2023-10-16T11:26:35.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fe25e640 1 --2- 172.21.15.84:0/3471026963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f8159ab0 0x7f65f80ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:35.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fd25c640 1 -- 172.21.15.84:0/3471026963 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1555178791 0 0) 0x7f65e4003520 con 0x7f65f806c140 2023-10-16T11:26:35.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fd25c640 1 -- 172.21.15.84:0/3471026963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f8159ab0 msgr2=0x7f65f80ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:35.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fd25c640 1 --2- 172.21.15.84:0/3471026963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f8159ab0 0x7f65f80ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:35.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fd25c640 1 -- 172.21.15.84:0/3471026963 >> v2:172.21.15.157:3300/0 conn(0x7f65f8159e90 msgr2=0x7f65f806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:35.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fd25c640 1 --2- 172.21.15.84:0/3471026963 >> v2:172.21.15.157:3300/0 conn(0x7f65f8159e90 0x7f65f806fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:35.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fd25c640 1 -- 172.21.15.84:0/3471026963 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f80cd7b0 con 0x7f65f806c140 2023-10-16T11:26:35.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fd25c640 1 -- 172.21.15.84:0/3471026963 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f65f00019f0 con 0x7f65f806c140 2023-10-16T11:26:35.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fd25c640 1 -- 172.21.15.84:0/3471026963 >> v1:172.21.15.84:6789/0 conn(0x7f65f806c140 legacy=0x7f65f815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:35.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fd25c640 1 --2- 172.21.15.84:0/3471026963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f8159ab0 0x7f65e4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:35.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fd25c640 1 --2- 172.21.15.84:0/3471026963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65e4003f40 0x7f65e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:35.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fd25c640 1 --2- 172.21.15.84:0/3471026963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e4004d00 0x7f65e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:35.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fd25c640 1 -- 172.21.15.84:0/3471026963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f815d890 con 0x7f65e4003f40 2023-10-16T11:26:35.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fd25c640 1 -- 172.21.15.84:0/3471026963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80cd4d0 con 0x7f65e4004d00 2023-10-16T11:26:35.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fd25c640 1 -- 172.21.15.84:0/3471026963 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e400bb60 con 0x7f65f8159ab0 2023-10-16T11:26:35.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.615+0000 7f65fda5d640 1 --2- 172.21.15.84:0/3471026963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65e4003f40 0x7f65e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:35.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.616+0000 7f65fe25e640 1 --2- 172.21.15.84:0/3471026963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f8159ab0 0x7f65e4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:35.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.616+0000 7f65fea5f640 1 --2- 172.21.15.84:0/3471026963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e4004d00 0x7f65e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:35.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.616+0000 7f65fda5d640 1 -- 172.21.15.84:0/3471026963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f8159ab0 msgr2=0x7f65e4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:35.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.616+0000 7f65fda5d640 1 --2- 172.21.15.84:0/3471026963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f8159ab0 0x7f65e4003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:35.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.616+0000 7f65fda5d640 1 -- 172.21.15.84:0/3471026963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e4004d00 msgr2=0x7f65e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:35.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.616+0000 7f65fda5d640 1 --2- 172.21.15.84:0/3471026963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e4004d00 0x7f65e4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:35.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.616+0000 7f65fda5d640 1 -- 172.21.15.84:0/3471026963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e4003520 con 0x7f65e4003f40 2023-10-16T11:26:35.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.616+0000 7f65fea5f640 1 --2- 172.21.15.84:0/3471026963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e4004d00 0x7f65e4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:35.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.617+0000 7f65fda5d640 1 --2- 172.21.15.84:0/3471026963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65e4003f40 0x7f65e40047c0 secure :-1 s=READY pgs=3179 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009ac0 tx=0x7f65e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2a974245b8353f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:35.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.617+0000 7f65fd25c640 1 -- 172.21.15.84:0/3471026963 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e802c710 con 0x7f65e4003f40 2023-10-16T11:26:35.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.617+0000 7f65fd25c640 1 -- 172.21.15.84:0/3471026963 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e802c870 con 0x7f65e4003f40 2023-10-16T11:26:35.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.617+0000 7f65fd25c640 1 -- 172.21.15.84:0/3471026963 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e802cc00 con 0x7f65e4003f40 2023-10-16T11:26:35.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.618+0000 7f66004c2640 1 -- 172.21.15.84:0/3471026963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65e4003f40 msgr2=0x7f65e40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:35.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.618+0000 7f66004c2640 1 --2- 172.21.15.84:0/3471026963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65e4003f40 0x7f65e40047c0 secure :-1 s=READY pgs=3179 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009ac0 tx=0x7f65e40057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:35.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.618+0000 7f65fea5f640 1 -- 172.21.15.84:0/3471026963 reap_dead start 2023-10-16T11:26:35.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.618+0000 7f66004c2640 1 -- 172.21.15.84:0/3471026963 shutdown_connections 2023-10-16T11:26:35.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.618+0000 7f66004c2640 1 -- 172.21.15.84:0/3471026963 >> 172.21.15.84:0/3471026963 conn(0x7f65f80ce9b0 msgr2=0x7f65f80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:35.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.618+0000 7f66004c2640 1 -- 172.21.15.84:0/3471026963 shutdown_connections 2023-10-16T11:26:35.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.618+0000 7f66004c2640 1 -- 172.21.15.84:0/3471026963 wait complete. 2023-10-16T11:26:35.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.620+0000 7f66004c2640 1 Processor -- start 2023-10-16T11:26:35.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.620+0000 7f66004c2640 1 -- start start 2023-10-16T11:26:35.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.621+0000 7f66004c2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e4003f40 0x7f65f814b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:35.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.621+0000 7f66004c2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f80c28f0 0x7f65f80c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:35.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.621+0000 7f66004c2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f80c2d70 0x7f65f80c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:35.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.621+0000 7f66004c2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80c7680 con 0x7f65f80c28f0 2023-10-16T11:26:35.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.621+0000 7f66004c2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80c6ad0 con 0x7f65f80c2d70 2023-10-16T11:26:35.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.621+0000 7f66004c2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80c6ce0 con 0x7f65e4003f40 2023-10-16T11:26:35.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.621+0000 7f65fda5d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f80c28f0 0x7f65f80c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:35.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.621+0000 7f65fda5d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f80c28f0 0x7f65f80c7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51820/0 (socket says 172.21.15.84:51820) 2023-10-16T11:26:35.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.621+0000 7f65fda5d640 1 -- 172.21.15.84:0/4284503125 learned_addr learned my addr 172.21.15.84:0/4284503125 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:35.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.622+0000 7f65fea5f640 1 --2- 172.21.15.84:0/4284503125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f80c2d70 0x7f65f80c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:35.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.622+0000 7f65fda5d640 1 -- 172.21.15.84:0/4284503125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e4003f40 msgr2=0x7f65f814b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:35.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.622+0000 7f65fda5d640 1 --2- 172.21.15.84:0/4284503125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e4003f40 0x7f65f814b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:35.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.622+0000 7f65fda5d640 1 -- 172.21.15.84:0/4284503125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f80c2d70 msgr2=0x7f65f80c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:35.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.622+0000 7f65fda5d640 1 --2- 172.21.15.84:0/4284503125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f80c2d70 0x7f65f80c7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:35.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.622+0000 7f65fda5d640 1 -- 172.21.15.84:0/4284503125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f80c6fc0 con 0x7f65f80c28f0 2023-10-16T11:26:35.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.622+0000 7f65fda5d640 1 --2- 172.21.15.84:0/4284503125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f80c28f0 0x7f65f80c7aa0 secure :-1 s=READY pgs=3180 cs=0 l=1 rev1=1 crypto rx=0x7f65e802c2b0 tx=0x7f65e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:35.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.622+0000 7f65eeffd640 1 -- 172.21.15.84:0/4284503125 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8003810 con 0x7f65f80c28f0 2023-10-16T11:26:35.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.622+0000 7f65eeffd640 1 -- 172.21.15.84:0/4284503125 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e802de90 con 0x7f65f80c28f0 2023-10-16T11:26:35.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.622+0000 7f65eeffd640 1 -- 172.21.15.84:0/4284503125 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8003b70 con 0x7f65f80c28f0 2023-10-16T11:26:35.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.622+0000 7f66004c2640 1 -- 172.21.15.84:0/4284503125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f80c72a0 con 0x7f65f80c28f0 2023-10-16T11:26:35.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.623+0000 7f66004c2640 1 -- 172.21.15.84:0/4284503125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f815a480 con 0x7f65f80c28f0 2023-10-16T11:26:35.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.624+0000 7f66004c2640 1 -- 172.21.15.84:0/4284503125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f80c78f0 con 0x7f65f80c28f0 2023-10-16T11:26:35.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.625+0000 7f65eeffd640 1 -- 172.21.15.84:0/4284503125 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f65e8003cd0 con 0x7f65f80c28f0 2023-10-16T11:26:35.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.626+0000 7f65eeffd640 1 --2- 172.21.15.84:0/4284503125 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f65dc06b020 0x7f65dc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:35.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.626+0000 7f65eeffd640 1 -- 172.21.15.84:0/4284503125 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(249..249 src has 1..249) v4 ==== 5917+0+0 (secure 0 0 0) 0x7f65e80abd20 con 0x7f65f80c28f0 2023-10-16T11:26:35.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.627+0000 7f65fe25e640 1 --2- 172.21.15.84:0/4284503125 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f65dc06b020 0x7f65dc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:35.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.627+0000 7f65fe25e640 1 --2- 172.21.15.84:0/4284503125 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f65dc06b020 0x7f65dc06d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f65f4006f00 tx=0x7f65f4006e10 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:35.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.632+0000 7f65eeffd640 1 -- 172.21.15.84:0/4284503125 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f65e8076b00 con 0x7f65f80c28f0 2023-10-16T11:26:35.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.831+0000 7f66004c2640 1 -- 172.21.15.84:0/4284503125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f65f80c7430 con 0x7f65f80c28f0 2023-10-16T11:26:35.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.838+0000 7f65eeffd640 1 -- 172.21.15.84:0/4284503125 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v88)=0 v88) v1 ==== 122+0+0 (secure 0 0 0) 0x7f65e807a120 con 0x7f65f80c28f0 2023-10-16T11:26:35.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.841+0000 7f66004c2640 1 -- 172.21.15.84:0/4284503125 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f65dc06b020 msgr2=0x7f65dc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:35.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.841+0000 7f66004c2640 1 --2- 172.21.15.84:0/4284503125 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f65dc06b020 0x7f65dc06d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f65f4006f00 tx=0x7f65f4006e10 comp rx=0 tx=0).stop 2023-10-16T11:26:35.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.841+0000 7f66004c2640 1 -- 172.21.15.84:0/4284503125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f80c28f0 msgr2=0x7f65f80c7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:35.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.841+0000 7f66004c2640 1 --2- 172.21.15.84:0/4284503125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f80c28f0 0x7f65f80c7aa0 secure :-1 s=READY pgs=3180 cs=0 l=1 rev1=1 crypto rx=0x7f65e802c2b0 tx=0x7f65e8004730 comp rx=0 tx=0).stop 2023-10-16T11:26:35.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.841+0000 7f66004c2640 1 -- 172.21.15.84:0/4284503125 shutdown_connections 2023-10-16T11:26:35.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.841+0000 7f66004c2640 1 --2- 172.21.15.84:0/4284503125 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f65dc06b020 0x7f65dc06d4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:35.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.841+0000 7f66004c2640 1 --2- 172.21.15.84:0/4284503125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f80c2d70 0x7f65f80c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:35.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.841+0000 7f66004c2640 1 --2- 172.21.15.84:0/4284503125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65f80c28f0 0x7f65f80c7aa0 unknown :-1 s=CLOSED pgs=3180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:35.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.841+0000 7f66004c2640 1 --2- 172.21.15.84:0/4284503125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e4003f40 0x7f65f814b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:35.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.841+0000 7f66004c2640 1 -- 172.21.15.84:0/4284503125 >> 172.21.15.84:0/4284503125 conn(0x7f65f80ce9b0 msgr2=0x7f65f80ae310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:35.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.842+0000 7f66004c2640 1 -- 172.21.15.84:0/4284503125 shutdown_connections 2023-10-16T11:26:35.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:35.842+0000 7f66004c2640 1 -- 172.21.15.84:0/4284503125 wait complete. 2023-10-16T11:26:35.861 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:26:35.862 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:26:35.979 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:26:36.100 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:26:36.101 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:26:36.101 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:26:36.158 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:26:36.159 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:26:36.159 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:26:36.221 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:26:36.222 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:26:36.222 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:26:36.282 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:26:36.283 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:26:36.283 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:26:36.345 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:26:36.345 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T11:26:36.346 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:26:36.492 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:36.488+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:36.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.573+0000 7fa7aa66f640 1 Processor -- start 2023-10-16T11:26:36.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.573+0000 7fa7aa66f640 1 -- start start 2023-10-16T11:26:36.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.575+0000 7fa7aa66f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa7a415ae80 0x7fa7a415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:36.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.575+0000 7fa7aa66f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a415ed60 con 0x7fa7a40ce390 2023-10-16T11:26:36.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.575+0000 7fa7aa66f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a40707e0 con 0x7fa7a415aaa0 2023-10-16T11:26:36.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.575+0000 7fa7aa66f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a406b810 con 0x7fa7a415ae80 2023-10-16T11:26:36.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.575+0000 7fa7a8c0c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa7a40ce390 0x7fa7a4070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38764/0 (socket says 172.21.15.84:38764) 2023-10-16T11:26:36.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.575+0000 7fa7a8c0c640 1 -- 172.21.15.84:0/1014695654 learned_addr learned my addr 172.21.15.84:0/1014695654 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:36.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.575+0000 7fa79bfff640 1 --2- 172.21.15.84:0/1014695654 >> v2:172.21.15.157:3300/0 conn(0x7fa7a415ae80 0x7fa7a415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:36.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.576+0000 7fa7a37fe640 1 -- 172.21.15.84:0/1014695654 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2508139098 0 0) 0x7fa7a415ed60 con 0x7fa7a40ce390 2023-10-16T11:26:36.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.576+0000 7fa7a37fe640 1 -- 172.21.15.84:0/1014695654 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa790003520 con 0x7fa7a40ce390 2023-10-16T11:26:36.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.576+0000 7fa7a37fe640 1 -- 172.21.15.84:0/1014695654 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2903738659 0 0) 0x7fa790003520 con 0x7fa7a40ce390 2023-10-16T11:26:36.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.576+0000 7fa7a37fe640 1 -- 172.21.15.84:0/1014695654 >> v2:172.21.15.157:3300/0 conn(0x7fa7a415ae80 msgr2=0x7fa7a415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:36.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.576+0000 7fa7a37fe640 1 --2- 172.21.15.84:0/1014695654 >> v2:172.21.15.157:3300/0 conn(0x7fa7a415ae80 0x7fa7a415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:36.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.576+0000 7fa7a37fe640 1 -- 172.21.15.84:0/1014695654 >> v1:172.21.15.157:6789/0 conn(0x7fa7a415aaa0 legacy=0x7fa7a40cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:36.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.576+0000 7fa7a37fe640 1 -- 172.21.15.84:0/1014695654 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a415ef90 con 0x7fa7a40ce390 2023-10-16T11:26:36.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.576+0000 7fa7a37fe640 1 -- 172.21.15.84:0/1014695654 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fa79c0019f0 con 0x7fa7a40ce390 2023-10-16T11:26:36.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.576+0000 7fa7a37fe640 1 -- 172.21.15.84:0/1014695654 >> v1:172.21.15.84:6789/0 conn(0x7fa7a40ce390 legacy=0x7fa7a4070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:36.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.576+0000 7fa7a37fe640 1 --2- 172.21.15.84:0/1014695654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7900039c0 0x7fa790003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:36.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.576+0000 7fa7a37fe640 1 --2- 172.21.15.84:0/1014695654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7900047c0 0x7fa790004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:36.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.576+0000 7fa7a37fe640 1 --2- 172.21.15.84:0/1014695654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa790005180 0x7fa79000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:36.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.576+0000 7fa7a37fe640 1 -- 172.21.15.84:0/1014695654 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a406b810 con 0x7fa790005180 2023-10-16T11:26:36.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.576+0000 7fa7a37fe640 1 -- 172.21.15.84:0/1014695654 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79000c080 con 0x7fa7900047c0 2023-10-16T11:26:36.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.576+0000 7fa7a37fe640 1 -- 172.21.15.84:0/1014695654 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79000c240 con 0x7fa7900039c0 2023-10-16T11:26:36.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.577+0000 7fa7a8c0c640 1 --2- 172.21.15.84:0/1014695654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa790005180 0x7fa79000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:36.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.577+0000 7fa79bfff640 1 --2- 172.21.15.84:0/1014695654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7900047c0 0x7fa790004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:36.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.577+0000 7fa7a3fff640 1 --2- 172.21.15.84:0/1014695654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7900039c0 0x7fa790003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:36.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.577+0000 7fa7a8c0c640 1 -- 172.21.15.84:0/1014695654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7900039c0 msgr2=0x7fa790003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:36.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.577+0000 7fa7a8c0c640 1 --2- 172.21.15.84:0/1014695654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7900039c0 0x7fa790003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:36.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.577+0000 7fa7a8c0c640 1 -- 172.21.15.84:0/1014695654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7900047c0 msgr2=0x7fa790004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:36.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.577+0000 7fa7a8c0c640 1 --2- 172.21.15.84:0/1014695654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7900047c0 0x7fa790004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:36.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.577+0000 7fa7a8c0c640 1 -- 172.21.15.84:0/1014695654 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa790003520 con 0x7fa790005180 2023-10-16T11:26:36.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.578+0000 7fa7a8c0c640 1 -- 172.21.15.84:0/1014695654 reap_dead start 2023-10-16T11:26:36.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.578+0000 7fa7a8c0c640 1 --2- 172.21.15.84:0/1014695654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa790005180 0x7fa79000b980 secure :-1 s=READY pgs=3182 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00a750 tx=0x7fa7900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64719ca92c037320 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:36.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.578+0000 7fa7a37fe640 1 -- 172.21.15.84:0/1014695654 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c034630 con 0x7fa790005180 2023-10-16T11:26:36.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.578+0000 7fa7a37fe640 1 -- 172.21.15.84:0/1014695654 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79c02f070 con 0x7fa790005180 2023-10-16T11:26:36.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.578+0000 7fa7a37fe640 1 -- 172.21.15.84:0/1014695654 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0349f0 con 0x7fa790005180 2023-10-16T11:26:36.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.579+0000 7fa7aa66f640 1 -- 172.21.15.84:0/1014695654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa790005180 msgr2=0x7fa79000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:36.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.579+0000 7fa7aa66f640 1 --2- 172.21.15.84:0/1014695654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa790005180 0x7fa79000b980 secure :-1 s=READY pgs=3182 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00a750 tx=0x7fa7900057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:36.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.579+0000 7fa7aa66f640 1 -- 172.21.15.84:0/1014695654 shutdown_connections 2023-10-16T11:26:36.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.579+0000 7fa7aa66f640 1 --2- 172.21.15.84:0/1014695654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa790005180 0x7fa79000b980 unknown :-1 s=CLOSED pgs=3182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:36.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.579+0000 7fa7aa66f640 1 -- 172.21.15.84:0/1014695654 >> 172.21.15.84:0/1014695654 conn(0x7fa7a40cb410 msgr2=0x7fa7a40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:36.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.579+0000 7fa7aa66f640 1 -- 172.21.15.84:0/1014695654 shutdown_connections 2023-10-16T11:26:36.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.579+0000 7fa7aa66f640 1 -- 172.21.15.84:0/1014695654 wait complete. 2023-10-16T11:26:36.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.581+0000 7fa7aa66f640 1 Processor -- start 2023-10-16T11:26:36.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.581+0000 7fa7aa66f640 1 -- start start 2023-10-16T11:26:36.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.582+0000 7fa7aa66f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa790005180 0x7fa7a40b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:36.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.582+0000 7fa7aa66f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7a40b0540 0x7fa7a4152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:36.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.582+0000 7fa7aa66f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a41533c0 0x7fa7a40af370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:36.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.582+0000 7fa7aa66f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4153840 con 0x7fa790005180 2023-10-16T11:26:36.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.582+0000 7fa7aa66f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a40aeee0 con 0x7fa7a41533c0 2023-10-16T11:26:36.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.582+0000 7fa7aa66f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a40ae2d0 con 0x7fa7a40b0540 2023-10-16T11:26:36.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.582+0000 7fa7a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa790005180 0x7fa7a40b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:36.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.582+0000 7fa7a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa790005180 0x7fa7a40b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51828/0 (socket says 172.21.15.84:51828) 2023-10-16T11:26:36.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.582+0000 7fa7a3fff640 1 -- 172.21.15.84:0/1750123238 learned_addr learned my addr 172.21.15.84:0/1750123238 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:36.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.583+0000 7fa7a8c0c640 1 --2- 172.21.15.84:0/1750123238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a41533c0 0x7fa7a40af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:36.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.583+0000 7fa7a3fff640 1 -- 172.21.15.84:0/1750123238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7a40b0540 msgr2=0x7fa7a4152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:36.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.583+0000 7fa7a3fff640 1 --2- 172.21.15.84:0/1750123238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7a40b0540 0x7fa7a4152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:36.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.583+0000 7fa7a3fff640 1 -- 172.21.15.84:0/1750123238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a41533c0 msgr2=0x7fa7a40af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:36.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.583+0000 7fa7a3fff640 1 --2- 172.21.15.84:0/1750123238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a41533c0 0x7fa7a40af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:36.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.583+0000 7fa7a3fff640 1 -- 172.21.15.84:0/1750123238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a40ae5b0 con 0x7fa790005180 2023-10-16T11:26:36.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.583+0000 7fa7a8c0c640 1 --2- 172.21.15.84:0/1750123238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a41533c0 0x7fa7a40af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:36.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.583+0000 7fa7a3fff640 1 --2- 172.21.15.84:0/1750123238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa790005180 0x7fa7a40b0000 secure :-1 s=READY pgs=3183 cs=0 l=1 rev1=1 crypto rx=0x7fa79400d5a0 tx=0x7fa79400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:36.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.583+0000 7fa7a17fa640 1 -- 172.21.15.84:0/1750123238 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794003610 con 0x7fa790005180 2023-10-16T11:26:36.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.584+0000 7fa7a17fa640 1 -- 172.21.15.84:0/1750123238 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa794003770 con 0x7fa790005180 2023-10-16T11:26:36.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.584+0000 7fa7a17fa640 1 -- 172.21.15.84:0/1750123238 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794003b20 con 0x7fa790005180 2023-10-16T11:26:36.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.584+0000 7fa7aa66f640 1 -- 172.21.15.84:0/1750123238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a40ae890 con 0x7fa790005180 2023-10-16T11:26:36.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.584+0000 7fa7aa66f640 1 -- 172.21.15.84:0/1750123238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a4152b10 con 0x7fa790005180 2023-10-16T11:26:36.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.585+0000 7fa7aa66f640 1 -- 172.21.15.84:0/1750123238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a404e160 con 0x7fa790005180 2023-10-16T11:26:36.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.586+0000 7fa7a17fa640 1 -- 172.21.15.84:0/1750123238 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fa794003c80 con 0x7fa790005180 2023-10-16T11:26:36.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.587+0000 7fa7a17fa640 1 --2- 172.21.15.84:0/1750123238 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fa78406b020 0x7fa78406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:36.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.587+0000 7fa7a17fa640 1 -- 172.21.15.84:0/1750123238 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(249..249 src has 1..249) v4 ==== 5917+0+0 (secure 0 0 0) 0x7fa794089940 con 0x7fa790005180 2023-10-16T11:26:36.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.587+0000 7fa79bfff640 1 --2- 172.21.15.84:0/1750123238 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fa78406b020 0x7fa78406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:36.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.588+0000 7fa79bfff640 1 --2- 172.21.15.84:0/1750123238 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fa78406b020 0x7fa78406d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7a415df10 tx=0x7fa78c008040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:36.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.591+0000 7fa7a17fa640 1 -- 172.21.15.84:0/1750123238 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa7940546e0 con 0x7fa790005180 2023-10-16T11:26:36.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.834+0000 7fa7aa66f640 1 -- 172.21.15.84:0/1750123238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7a415e350 con 0x7fa790005180 2023-10-16T11:26:36.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.835+0000 7fa7a17fa640 1 -- 172.21.15.84:0/1750123238 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+3709 (secure 0 0 0) 0x7fa794057d00 con 0x7fa790005180 2023-10-16T11:26:36.838 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:36.838 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":141,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2516077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2516077743},{"type":"v1","addr":"172.21.15.157:6833","nonce":2516077743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1527104462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1527104462},{"type":"v1","addr":"172.21.15.157:6835","nonce":1527104462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":12801,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3617906728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3617906728},{"type":"v1","addr":"172.21.15.84:6837","nonce":3617906728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12897,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/495944676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":495944676},{"type":"v1","addr":"172.21.15.84:6835","nonce":495944676}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":141}],"filesystems":[]} 2023-10-16T11:26:36.838 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 141 2023-10-16T11:26:36.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.838+0000 7fa7aa66f640 1 -- 172.21.15.84:0/1750123238 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fa78406b020 msgr2=0x7fa78406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:36.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.838+0000 7fa7aa66f640 1 --2- 172.21.15.84:0/1750123238 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fa78406b020 0x7fa78406d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7a415df10 tx=0x7fa78c008040 comp rx=0 tx=0).stop 2023-10-16T11:26:36.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.838+0000 7fa7aa66f640 1 -- 172.21.15.84:0/1750123238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa790005180 msgr2=0x7fa7a40b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:36.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.838+0000 7fa7aa66f640 1 --2- 172.21.15.84:0/1750123238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa790005180 0x7fa7a40b0000 secure :-1 s=READY pgs=3183 cs=0 l=1 rev1=1 crypto rx=0x7fa79400d5a0 tx=0x7fa79400da60 comp rx=0 tx=0).stop 2023-10-16T11:26:36.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.838+0000 7fa7aa66f640 1 -- 172.21.15.84:0/1750123238 shutdown_connections 2023-10-16T11:26:36.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.838+0000 7fa7aa66f640 1 --2- 172.21.15.84:0/1750123238 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fa78406b020 0x7fa78406d4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:36.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.838+0000 7fa7aa66f640 1 --2- 172.21.15.84:0/1750123238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a41533c0 0x7fa7a40af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:36.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.838+0000 7fa7aa66f640 1 --2- 172.21.15.84:0/1750123238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7a40b0540 0x7fa7a4152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:36.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.838+0000 7fa7aa66f640 1 --2- 172.21.15.84:0/1750123238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa790005180 0x7fa7a40b0000 unknown :-1 s=CLOSED pgs=3183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:36.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.838+0000 7fa7aa66f640 1 -- 172.21.15.84:0/1750123238 >> 172.21.15.84:0/1750123238 conn(0x7fa7a40cb410 msgr2=0x7fa7a406fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:36.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.838+0000 7fa7aa66f640 1 -- 172.21.15.84:0/1750123238 shutdown_connections 2023-10-16T11:26:36.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:36.838+0000 7fa7aa66f640 1 -- 172.21.15.84:0/1750123238 wait complete. 2023-10-16T11:26:36.860 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T11:26:36.860 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T11:26:36.861 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T11:26:36.861 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:26:36.861 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T11:26:36.861 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T11:26:36.862 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T11:26:36.862 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:26:36.862 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T11:26:36.863 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T11:26:36.863 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T11:26:36.863 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:26:36.864 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T11:26:36.864 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T11:26:36.864 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T11:26:36.864 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:26:36.878 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T11:26:36.878 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T11:26:36.880 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T11:26:36.881 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T11:26:36.886 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T11:26:36.887 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T11:26:36.893 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T11:26:36.894 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T11:26:36.929 INFO:tasks.ceph.mds.c:Started 2023-10-16T11:26:36.930 DEBUG:teuthology.parallel:result is None 2023-10-16T11:26:36.937 INFO:tasks.ceph.mds.a:Started 2023-10-16T11:26:36.938 DEBUG:teuthology.parallel:result is None 2023-10-16T11:26:36.941 INFO:tasks.ceph.mds.d:Started 2023-10-16T11:26:36.942 DEBUG:teuthology.parallel:result is None 2023-10-16T11:26:36.949 INFO:tasks.ceph.mds.b:Started 2023-10-16T11:26:36.950 DEBUG:teuthology.parallel:result is None 2023-10-16T11:26:36.950 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T11:26:37.091 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T11:26:37.094 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T11:26:37.114 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-10-16T11:26:37.115 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T11:26:37.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.143+0000 7f756c10f640 1 Processor -- start 2023-10-16T11:26:37.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.143+0000 7f756c10f640 1 -- start start 2023-10-16T11:26:37.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.144+0000 7f756c10f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7564159e90 0x7f75640ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:37.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.144+0000 7f756c10f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7564070360 con 0x7f7564159ab0 2023-10-16T11:26:37.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.144+0000 7f756c10f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75640cd1f0 con 0x7f756406c140 2023-10-16T11:26:37.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.144+0000 7f756c10f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756415d890 con 0x7f7564159e90 2023-10-16T11:26:37.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.144+0000 7f7569eab640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7564159ab0 0x7f756406fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38766/0 (socket says 172.21.15.84:38766) 2023-10-16T11:26:37.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.144+0000 7f7569eab640 1 -- 172.21.15.84:0/134167173 learned_addr learned my addr 172.21.15.84:0/134167173 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:37.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.145+0000 7f75696aa640 1 --2- 172.21.15.84:0/134167173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7564159e90 0x7f75640ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:37.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.145+0000 7f7568ea9640 1 -- 172.21.15.84:0/134167173 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813712460 0 0) 0x7f7564070360 con 0x7f7564159ab0 2023-10-16T11:26:37.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.145+0000 7f7568ea9640 1 -- 172.21.15.84:0/134167173 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7550003520 con 0x7f7564159ab0 2023-10-16T11:26:37.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.145+0000 7f7568ea9640 1 -- 172.21.15.84:0/134167173 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3497457631 0 0) 0x7f75640cd1f0 con 0x7f756406c140 2023-10-16T11:26:37.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.145+0000 7f7568ea9640 1 -- 172.21.15.84:0/134167173 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7564070360 con 0x7f756406c140 2023-10-16T11:26:37.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.145+0000 7f75696aa640 1 -- 172.21.15.84:0/134167173 >> v1:172.21.15.157:6789/0 conn(0x7f756406c140 legacy=0x7f756415dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:37.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.145+0000 7f75696aa640 1 -- 172.21.15.84:0/134167173 >> v1:172.21.15.84:6789/0 conn(0x7f7564159ab0 legacy=0x7f756406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:37.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.145+0000 7f75696aa640 1 -- 172.21.15.84:0/134167173 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75640cd4d0 con 0x7f7564159e90 2023-10-16T11:26:37.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.146+0000 7f75696aa640 1 --2- 172.21.15.84:0/134167173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7564159e90 0x7f75640ce370 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7554009800 tx=0x7f755402b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7962a3f290a227c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:37.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.147+0000 7f7568ea9640 1 -- 172.21.15.84:0/134167173 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554031770 con 0x7f7564159e90 2023-10-16T11:26:37.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.147+0000 7f7568ea9640 1 -- 172.21.15.84:0/134167173 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75540318d0 con 0x7f7564159e90 2023-10-16T11:26:37.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.147+0000 7f7568ea9640 1 -- 172.21.15.84:0/134167173 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554031c60 con 0x7f7564159e90 2023-10-16T11:26:37.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.147+0000 7f756c10f640 1 -- 172.21.15.84:0/134167173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7564159e90 msgr2=0x7f75640ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:37.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.147+0000 7f756c10f640 1 --2- 172.21.15.84:0/134167173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7564159e90 0x7f75640ce370 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7554009800 tx=0x7f755402b810 comp rx=0 tx=0).stop 2023-10-16T11:26:37.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.148+0000 7f756c10f640 1 -- 172.21.15.84:0/134167173 shutdown_connections 2023-10-16T11:26:37.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.148+0000 7f756c10f640 1 --2- 172.21.15.84:0/134167173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7564159e90 0x7f75640ce370 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:37.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.148+0000 7f756c10f640 1 -- 172.21.15.84:0/134167173 >> 172.21.15.84:0/134167173 conn(0x7f75640ce9b0 msgr2=0x7f75640cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:37.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.148+0000 7f756c10f640 1 -- 172.21.15.84:0/134167173 shutdown_connections 2023-10-16T11:26:37.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.148+0000 7f756c10f640 1 -- 172.21.15.84:0/134167173 wait complete. 2023-10-16T11:26:37.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.149+0000 7f756c10f640 1 Processor -- start 2023-10-16T11:26:37.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.150+0000 7f756c10f640 1 -- start start 2023-10-16T11:26:37.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.150+0000 7f756c10f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756406c140 0x7f75640c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:37.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.150+0000 7f756c10f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7564159ab0 0x7f75640c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:37.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.150+0000 7f756c10f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7564159e90 0x7f7564155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:37.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.150+0000 7f756c10f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75640c9130 con 0x7f7564159ab0 2023-10-16T11:26:37.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.150+0000 7f756c10f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75640c6b50 con 0x7f756406c140 2023-10-16T11:26:37.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.150+0000 7f756c10f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75640acd70 con 0x7f7564159e90 2023-10-16T11:26:37.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.151+0000 7f75696aa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7564159ab0 0x7f75640c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:37.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.151+0000 7f75696aa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7564159ab0 0x7f75640c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51846/0 (socket says 172.21.15.84:51846) 2023-10-16T11:26:37.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.151+0000 7f75696aa640 1 -- 172.21.15.84:0/964284223 learned_addr learned my addr 172.21.15.84:0/964284223 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:37.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.151+0000 7f7569eab640 1 --2- 172.21.15.84:0/964284223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756406c140 0x7f75640c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:37.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.151+0000 7f756a6ac640 1 --2- 172.21.15.84:0/964284223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7564159e90 0x7f7564155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.151+0000 7f75696aa640 1 -- 172.21.15.84:0/964284223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7564159e90 msgr2=0x7f7564155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.151+0000 7f75696aa640 1 --2- 172.21.15.84:0/964284223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7564159e90 0x7f7564155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.151+0000 7f75696aa640 1 -- 172.21.15.84:0/964284223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756406c140 msgr2=0x7f75640c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.151+0000 7f75696aa640 1 --2- 172.21.15.84:0/964284223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756406c140 0x7f75640c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.151+0000 7f75696aa640 1 -- 172.21.15.84:0/964284223 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7554009400 con 0x7f7564159ab0 2023-10-16T11:26:37.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.151+0000 7f756a6ac640 1 --2- 172.21.15.84:0/964284223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7564159e90 0x7f7564155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:26:37.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.152+0000 7f75696aa640 1 --2- 172.21.15.84:0/964284223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7564159ab0 0x7f75640c8a70 secure :-1 s=READY pgs=3187 cs=0 l=1 rev1=1 crypto rx=0x7f7554034040 tx=0x7f75540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:37.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.152+0000 7f755a7fc640 1 -- 172.21.15.84:0/964284223 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554032810 con 0x7f7564159ab0 2023-10-16T11:26:37.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.152+0000 7f755a7fc640 1 -- 172.21.15.84:0/964284223 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7554032970 con 0x7f7564159ab0 2023-10-16T11:26:37.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.152+0000 7f755a7fc640 1 -- 172.21.15.84:0/964284223 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554032d20 con 0x7f7564159ab0 2023-10-16T11:26:37.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.152+0000 7f756c10f640 1 -- 172.21.15.84:0/964284223 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7564155bd0 con 0x7f7564159ab0 2023-10-16T11:26:37.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.152+0000 7f756c10f640 1 -- 172.21.15.84:0/964284223 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75640c6590 con 0x7f7564159ab0 2023-10-16T11:26:37.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.153+0000 7f756c10f640 1 -- 172.21.15.84:0/964284223 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756404e160 con 0x7f7564159ab0 2023-10-16T11:26:37.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.154+0000 7f755a7fc640 1 -- 172.21.15.84:0/964284223 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f755404e440 con 0x7f7564159ab0 2023-10-16T11:26:37.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.155+0000 7f755a7fc640 1 --2- 172.21.15.84:0/964284223 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f754806b020 0x7f754806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:37.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.155+0000 7f755a7fc640 1 -- 172.21.15.84:0/964284223 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(249..249 src has 1..249) v4 ==== 5917+0+0 (secure 0 0 0) 0x7f75540aba60 con 0x7f7564159ab0 2023-10-16T11:26:37.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.156+0000 7f7569eab640 1 --2- 172.21.15.84:0/964284223 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f754806b020 0x7f754806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:37.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.156+0000 7f7569eab640 1 --2- 172.21.15.84:0/964284223 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f754806b020 0x7f754806d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7560000f60 tx=0x7f7560007d70 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:37.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.160+0000 7f755a7fc640 1 -- 172.21.15.84:0/964284223 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7554076610 con 0x7f7564159ab0 2023-10-16T11:26:37.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.376+0000 7f756c10f640 1 -- 172.21.15.84:0/964284223 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f75640ccb60 con 0x7f7564159ab0 2023-10-16T11:26:37.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.393+0000 7f755a7fc640 1 -- 172.21.15.84:0/964284223 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v251) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7554079c30 con 0x7f7564159ab0 2023-10-16T11:26:37.395 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-10-16T11:26:37.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.395+0000 7f756c10f640 1 -- 172.21.15.84:0/964284223 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f754806b020 msgr2=0x7f754806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:37.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.395+0000 7f756c10f640 1 --2- 172.21.15.84:0/964284223 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f754806b020 0x7f754806d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7560000f60 tx=0x7f7560007d70 comp rx=0 tx=0).stop 2023-10-16T11:26:37.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.395+0000 7f756c10f640 1 -- 172.21.15.84:0/964284223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7564159ab0 msgr2=0x7f75640c8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:37.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.395+0000 7f756c10f640 1 --2- 172.21.15.84:0/964284223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7564159ab0 0x7f75640c8a70 secure :-1 s=READY pgs=3187 cs=0 l=1 rev1=1 crypto rx=0x7f7554034040 tx=0x7f75540046c0 comp rx=0 tx=0).stop 2023-10-16T11:26:37.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.395+0000 7f756c10f640 1 -- 172.21.15.84:0/964284223 shutdown_connections 2023-10-16T11:26:37.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.395+0000 7f756c10f640 1 --2- 172.21.15.84:0/964284223 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f754806b020 0x7f754806d4e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:37.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.395+0000 7f756c10f640 1 --2- 172.21.15.84:0/964284223 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7564159e90 0x7f7564155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:37.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.395+0000 7f756c10f640 1 --2- 172.21.15.84:0/964284223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7564159ab0 0x7f75640c8a70 unknown :-1 s=CLOSED pgs=3187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:37.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.395+0000 7f756c10f640 1 --2- 172.21.15.84:0/964284223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f756406c140 0x7f75640c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:37.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.395+0000 7f756c10f640 1 -- 172.21.15.84:0/964284223 >> 172.21.15.84:0/964284223 conn(0x7f75640ce9b0 msgr2=0x7f7564156cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:37.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.395+0000 7f756c10f640 1 -- 172.21.15.84:0/964284223 shutdown_connections 2023-10-16T11:26:37.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.395+0000 7f756c10f640 1 -- 172.21.15.84:0/964284223 wait complete. 2023-10-16T11:26:37.418 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T11:26:37.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.616+0000 7f7c7059e640 1 Processor -- start 2023-10-16T11:26:37.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.616+0000 7f7c7059e640 1 -- start start 2023-10-16T11:26:37.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.618+0000 7f7c7059e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7c680b3070 0x7f7c680cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:37.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.618+0000 7f7c7059e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7c680b3e70 0x7f7c680cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:37.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.618+0000 7f7c7059e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c680b3940 con 0x7f7c680b3450 2023-10-16T11:26:37.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.618+0000 7f7c7059e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c680b45e0 con 0x7f7c680b3070 2023-10-16T11:26:37.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.618+0000 7f7c7059e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c680b3b90 con 0x7f7c680b3e70 2023-10-16T11:26:37.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.618+0000 7f7c6e33a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7c680b3070 0x7f7c680cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:37.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.618+0000 7f7c6e33a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7c680b3070 0x7f7c680cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51858/0 (socket says 172.21.15.84:51858) 2023-10-16T11:26:37.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.618+0000 7f7c6e33a640 1 -- 172.21.15.84:0/1327440509 learned_addr learned my addr 172.21.15.84:0/1327440509 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:37.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.619+0000 7f7c6e33a640 1 -- 172.21.15.84:0/1327440509 >> v2:172.21.15.157:3300/0 conn(0x7f7c680b3e70 msgr2=0x7f7c680cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:37.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.619+0000 7f7c6e33a640 1 --2- 172.21.15.84:0/1327440509 >> v2:172.21.15.157:3300/0 conn(0x7f7c680b3e70 0x7f7c680cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:37.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.619+0000 7f7c6e33a640 1 -- 172.21.15.84:0/1327440509 >> v1:172.21.15.157:6789/0 conn(0x7f7c680b3450 legacy=0x7f7c680c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:37.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.619+0000 7f7c6e33a640 1 -- 172.21.15.84:0/1327440509 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c680d3840 con 0x7f7c680b3070 2023-10-16T11:26:37.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.620+0000 7f7c6e33a640 1 --2- 172.21.15.84:0/1327440509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c680b3070 0x7f7c680cf500 secure :-1 s=READY pgs=3188 cs=0 l=1 rev1=1 crypto rx=0x7f7c680b58e0 tx=0x7f7c6402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f143df5d790333c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:37.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.620+0000 7f7c6d338640 1 -- 172.21.15.84:0/1327440509 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64031870 con 0x7f7c680b3070 2023-10-16T11:26:37.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.620+0000 7f7c6d338640 1 -- 172.21.15.84:0/1327440509 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c640319d0 con 0x7f7c680b3070 2023-10-16T11:26:37.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.620+0000 7f7c6d338640 1 -- 172.21.15.84:0/1327440509 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64031d60 con 0x7f7c680b3070 2023-10-16T11:26:37.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.621+0000 7f7c7059e640 1 -- 172.21.15.84:0/1327440509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c680b3070 msgr2=0x7f7c680cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:37.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.621+0000 7f7c7059e640 1 --2- 172.21.15.84:0/1327440509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c680b3070 0x7f7c680cf500 secure :-1 s=READY pgs=3188 cs=0 l=1 rev1=1 crypto rx=0x7f7c680b58e0 tx=0x7f7c6402b910 comp rx=0 tx=0).stop 2023-10-16T11:26:37.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.621+0000 7f7c7059e640 1 -- 172.21.15.84:0/1327440509 shutdown_connections 2023-10-16T11:26:37.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.621+0000 7f7c7059e640 1 --2- 172.21.15.84:0/1327440509 >> v2:172.21.15.157:3300/0 conn(0x7f7c680b3e70 0x7f7c680cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:37.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.621+0000 7f7c7059e640 1 --2- 172.21.15.84:0/1327440509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c680b3070 0x7f7c680cf500 unknown :-1 s=CLOSED pgs=3188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:37.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.621+0000 7f7c7059e640 1 -- 172.21.15.84:0/1327440509 >> 172.21.15.84:0/1327440509 conn(0x7f7c680d2d20 msgr2=0x7f7c680d3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:37.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.621+0000 7f7c7059e640 1 -- 172.21.15.84:0/1327440509 shutdown_connections 2023-10-16T11:26:37.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.621+0000 7f7c7059e640 1 -- 172.21.15.84:0/1327440509 wait complete. 2023-10-16T11:26:37.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.623+0000 7f7c7059e640 1 Processor -- start 2023-10-16T11:26:37.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.623+0000 7f7c7059e640 1 -- start start 2023-10-16T11:26:37.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.624+0000 7f7c7059e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c680b3070 0x7f7c6809ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:37.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.624+0000 7f7c7059e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c680b3450 0x7f7c6814c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:37.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.624+0000 7f7c7059e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c680b3e70 0x7f7c680b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:37.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.624+0000 7f7c7059e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680cbbb0 con 0x7f7c680b3e70 2023-10-16T11:26:37.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.624+0000 7f7c7059e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6814cd70 con 0x7f7c680b3070 2023-10-16T11:26:37.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.624+0000 7f7c7059e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680cb530 con 0x7f7c680b3450 2023-10-16T11:26:37.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.624+0000 7f7c6eb3b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c680b3e70 0x7f7c680b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:37.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.624+0000 7f7c6eb3b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c680b3e70 0x7f7c680b0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51874/0 (socket says 172.21.15.84:51874) 2023-10-16T11:26:37.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.624+0000 7f7c6eb3b640 1 -- 172.21.15.84:0/351337070 learned_addr learned my addr 172.21.15.84:0/351337070 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:37.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.624+0000 7f7c6e33a640 1 --2- 172.21.15.84:0/351337070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c680b3070 0x7f7c6809ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:37.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.625+0000 7f7c6eb3b640 1 -- 172.21.15.84:0/351337070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c680b3450 msgr2=0x7f7c6814c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:37.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.625+0000 7f7c6db39640 1 --2- 172.21.15.84:0/351337070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c680b3450 0x7f7c6814c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:37.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.625+0000 7f7c6eb3b640 1 --2- 172.21.15.84:0/351337070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c680b3450 0x7f7c6814c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:37.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.625+0000 7f7c6eb3b640 1 -- 172.21.15.84:0/351337070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c680b3070 msgr2=0x7f7c6809ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:37.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.625+0000 7f7c6eb3b640 1 --2- 172.21.15.84:0/351337070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c680b3070 0x7f7c6809ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:37.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.625+0000 7f7c6eb3b640 1 -- 172.21.15.84:0/351337070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c64009470 con 0x7f7c680b3e70 2023-10-16T11:26:37.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.625+0000 7f7c6e33a640 1 --2- 172.21.15.84:0/351337070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c680b3070 0x7f7c6809ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:37.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.625+0000 7f7c6db39640 1 --2- 172.21.15.84:0/351337070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c680b3450 0x7f7c6814c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:37.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.625+0000 7f7c6eb3b640 1 --2- 172.21.15.84:0/351337070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c680b3e70 0x7f7c680b0940 secure :-1 s=READY pgs=3189 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000c780 tx=0x7f7c6000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:37.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.625+0000 7f7c5effd640 1 -- 172.21.15.84:0/351337070 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c60003d80 con 0x7f7c680b3e70 2023-10-16T11:26:37.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.625+0000 7f7c5effd640 1 -- 172.21.15.84:0/351337070 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6000ce90 con 0x7f7c680b3e70 2023-10-16T11:26:37.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.625+0000 7f7c5effd640 1 -- 172.21.15.84:0/351337070 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c600052b0 con 0x7f7c680b3e70 2023-10-16T11:26:37.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.625+0000 7f7c7059e640 1 -- 172.21.15.84:0/351337070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c680cad20 con 0x7f7c680b3e70 2023-10-16T11:26:37.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.625+0000 7f7c7059e640 1 -- 172.21.15.84:0/351337070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c680cb000 con 0x7f7c680b3e70 2023-10-16T11:26:37.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.627+0000 7f7c7059e640 1 -- 172.21.15.84:0/351337070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c680cbcf0 con 0x7f7c680b3e70 2023-10-16T11:26:37.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.628+0000 7f7c5effd640 1 -- 172.21.15.84:0/351337070 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f7c60005410 con 0x7f7c680b3e70 2023-10-16T11:26:37.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.629+0000 7f7c5effd640 1 --2- 172.21.15.84:0/351337070 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7c4c06b050 0x7f7c4c06d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:37.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.629+0000 7f7c5effd640 1 -- 172.21.15.84:0/351337070 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f7c600889f0 con 0x7f7c680b3e70 2023-10-16T11:26:37.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.629+0000 7f7c6e33a640 1 --2- 172.21.15.84:0/351337070 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7c4c06b050 0x7f7c4c06d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:37.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.630+0000 7f7c6e33a640 1 --2- 172.21.15.84:0/351337070 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7c4c06b050 0x7f7c4c06d510 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7c64036000 tx=0x7f7c6403c040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:37.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.633+0000 7f7c5effd640 1 -- 172.21.15.84:0/351337070 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7c680cbcf0 con 0x7f7c680b3e70 2023-10-16T11:26:37.683 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:37.680+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:37.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.896+0000 7f7c7059e640 1 -- 172.21.15.84:0/351337070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7c680cbcf0 con 0x7f7c680b3e70 2023-10-16T11:26:37.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.896+0000 7f7c5effd640 1 -- 172.21.15.84:0/351337070 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v57) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7c680cbcf0 con 0x7f7c680b3e70 2023-10-16T11:26:37.899 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:37.899 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:26:37.899 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-10-16T11:26:37.899 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.900 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-10-16T11:26:37.900 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l6dhjKRAA0pMvY6AK6qtFXDprNhwsqg==", 2023-10-16T11:26:37.900 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.900 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:26:37.900 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:26:37.900 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:26:37.901 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:26:37.901 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.901 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:37.901 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.901 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-10-16T11:26:37.902 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lFpMxEBAA0TLPRt5q3FwwBvMG019cjQ==", 2023-10-16T11:26:37.902 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.902 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:26:37.902 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:26:37.902 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:26:37.903 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:26:37.903 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.903 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:37.903 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.903 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-10-16T11:26:37.904 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l73RBOhAAOMEPxfUAMHhn5Blyvd5oMw==", 2023-10-16T11:26:37.904 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.904 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:26:37.904 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:26:37.905 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:26:37.905 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:26:37.905 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.905 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:37.905 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.905 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-10-16T11:26:37.906 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lcTIRIhAA0+FWm4b4yw77LD3+WKbk7w==", 2023-10-16T11:26:37.906 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.906 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:26:37.906 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:26:37.906 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:26:37.907 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:26:37.907 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.907 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:37.907 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.907 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-10-16T11:26:37.908 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDYBi1l1zBLERAA3YLW3xIYQoRVw7KSkwN3Og==", 2023-10-16T11:26:37.908 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.908 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:26:37.908 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:26:37.908 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:26:37.909 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.909 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:37.909 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.910 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-10-16T11:26:37.910 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDbBi1lFUUxKRAAQbus8SmYprJRmWTjoCznNA==", 2023-10-16T11:26:37.910 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.911 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:26:37.911 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:26:37.912 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:26:37.912 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.912 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:37.912 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.913 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-10-16T11:26:37.913 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDeBi1l+9lBMxAArl6sb/h15uBG/a9lVH40Hg==", 2023-10-16T11:26:37.913 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.913 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:26:37.913 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:26:37.914 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:26:37.914 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.914 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:37.914 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.914 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-10-16T11:26:37.914 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDiBi1l2rnEERAA3oRD71xa0fEOynM8zKuBEg==", 2023-10-16T11:26:37.915 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.915 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:26:37.915 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:26:37.915 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:26:37.915 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.915 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:37.916 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.916 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-10-16T11:26:37.916 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDnBi1lR7oLChAAZto8fDovDbpOd3Fcnc7C6w==", 2023-10-16T11:26:37.916 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.916 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:26:37.917 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:26:37.917 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:26:37.917 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.917 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:37.917 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.918 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-10-16T11:26:37.919 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDqBi1lmjg/JBAAlINKRGBc0/UWJJHMVFw57w==", 2023-10-16T11:26:37.919 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.919 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:26:37.919 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:26:37.920 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:26:37.920 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.920 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:37.920 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.921 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-10-16T11:26:37.921 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDuBi1lu0kMARAAtuSeM8F3yr8CVrc3p1biaQ==", 2023-10-16T11:26:37.921 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.921 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:26:37.921 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:26:37.921 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:26:37.922 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.922 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:37.922 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.922 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-10-16T11:26:37.922 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDxBi1lf6mNGBAAHGbqzJ0fmDxQIsasboHMfg==", 2023-10-16T11:26:37.923 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.923 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:26:37.923 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:26:37.923 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:26:37.923 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.923 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:37.924 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.924 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-10-16T11:26:37.924 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ==", 2023-10-16T11:26:37.924 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.924 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:26:37.925 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow r", 2023-10-16T11:26:37.925 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T11:26:37.925 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.925 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:37.925 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.926 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-10-16T11:26:37.926 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1lHU+4BxAAd8ET8hQ8Y+EUwdTJr9/+Ig==", 2023-10-16T11:26:37.927 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.927 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:26:37.927 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:26:37.927 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:26:37.928 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:26:37.928 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.928 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:37.928 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.929 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-10-16T11:26:37.929 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lA1BOORAAFNyfqgEF34MVR9YX3KBKkQ==", 2023-10-16T11:26:37.929 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.929 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:26:37.929 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:26:37.930 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:26:37.930 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:26:37.930 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.930 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:37.931 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.931 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-10-16T11:26:37.931 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1l848oERAANMfrZz9JrlgNVh7ZFbtnaA==", 2023-10-16T11:26:37.931 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.932 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:26:37.932 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:26:37.932 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:26:37.932 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:26:37.933 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.933 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:37.933 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.933 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-10-16T11:26:37.933 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDRBi1lTAxTGRAAMptxnxy3Qit/cTCXie6CrQ==", 2023-10-16T11:26:37.934 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.934 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:26:37.934 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:26:37.934 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:26:37.934 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:26:37.935 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.935 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:37.935 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.935 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-10-16T11:26:37.935 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lPuJxHRAABAnGPVCrkq0JR4Y6iFGZlw==", 2023-10-16T11:26:37.936 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.936 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:26:37.936 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T11:26:37.936 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:26:37.937 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.937 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:26:37.938 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:37.938 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-10-16T11:26:37.938 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lLxnKEBAA+a/pOHUp7Sg3m4weAust5w==", 2023-10-16T11:26:37.938 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:26:37.938 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:26:37.939 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T11:26:37.939 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:26:37.939 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.939 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:37.940 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-10-16T11:26:37.940 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:26:37.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.899+0000 7f7c7059e640 1 -- 172.21.15.84:0/351337070 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7c4c06b050 msgr2=0x7f7c4c06d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:37.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.899+0000 7f7c7059e640 1 --2- 172.21.15.84:0/351337070 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7c4c06b050 0x7f7c4c06d510 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7c64036000 tx=0x7f7c6403c040 comp rx=0 tx=0).stop 2023-10-16T11:26:37.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.899+0000 7f7c7059e640 1 -- 172.21.15.84:0/351337070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c680b3e70 msgr2=0x7f7c680b0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:37.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.899+0000 7f7c7059e640 1 --2- 172.21.15.84:0/351337070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c680b3e70 0x7f7c680b0940 secure :-1 s=READY pgs=3189 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000c780 tx=0x7f7c6000cc40 comp rx=0 tx=0).stop 2023-10-16T11:26:37.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.899+0000 7f7c7059e640 1 -- 172.21.15.84:0/351337070 shutdown_connections 2023-10-16T11:26:37.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.899+0000 7f7c7059e640 1 --2- 172.21.15.84:0/351337070 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7c4c06b050 0x7f7c4c06d510 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:37.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.899+0000 7f7c7059e640 1 --2- 172.21.15.84:0/351337070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c680b3e70 0x7f7c680b0940 unknown :-1 s=CLOSED pgs=3189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:37.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.899+0000 7f7c7059e640 1 --2- 172.21.15.84:0/351337070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c680b3450 0x7f7c6814c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:37.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.899+0000 7f7c7059e640 1 --2- 172.21.15.84:0/351337070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c680b3070 0x7f7c6809ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:37.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.899+0000 7f7c7059e640 1 -- 172.21.15.84:0/351337070 >> 172.21.15.84:0/351337070 conn(0x7f7c680d2d20 msgr2=0x7f7c680d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:37.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.900+0000 7f7c7059e640 1 -- 172.21.15.84:0/351337070 shutdown_connections 2023-10-16T11:26:37.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:37.900+0000 7f7c7059e640 1 -- 172.21.15.84:0/351337070 wait complete. 2023-10-16T11:26:37.944 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:26:38.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.164+0000 7f874f59e640 1 Processor -- start 2023-10-16T11:26:38.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.165+0000 7f874f59e640 1 -- start start 2023-10-16T11:26:38.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.166+0000 7f874f59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87500b8480 0x7f87500cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:38.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.166+0000 7f874f59e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875015d150 con 0x7f875015cd70 2023-10-16T11:26:38.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.166+0000 7f874f59e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87500d1180 con 0x7f87500d0be0 2023-10-16T11:26:38.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.166+0000 7f874f59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87500d1460 con 0x7f87500b8480 2023-10-16T11:26:38.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.166+0000 7f874e59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f875015cd70 0x7f87500d2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38768/0 (socket says 172.21.15.84:38768) 2023-10-16T11:26:38.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.166+0000 7f874e59c640 1 -- 172.21.15.84:0/2613864691 learned_addr learned my addr 172.21.15.84:0/2613864691 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:38.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.167+0000 7f874dd9b640 1 -- 172.21.15.84:0/2613864691 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1367185484 0 0) 0x7f875015d150 con 0x7f875015cd70 2023-10-16T11:26:38.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.167+0000 7f8745d9b640 1 --2- 172.21.15.84:0/2613864691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87500b8480 0x7f87500cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:38.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.167+0000 7f874dd9b640 1 -- 172.21.15.84:0/2613864691 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f873c003520 con 0x7f875015cd70 2023-10-16T11:26:38.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.167+0000 7f874dd9b640 1 -- 172.21.15.84:0/2613864691 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2840789407 0 0) 0x7f873c003520 con 0x7f875015cd70 2023-10-16T11:26:38.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.167+0000 7f874dd9b640 1 -- 172.21.15.84:0/2613864691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87500b8480 msgr2=0x7f87500cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:38.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.167+0000 7f874dd9b640 1 --2- 172.21.15.84:0/2613864691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87500b8480 0x7f87500cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:38.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.167+0000 7f874dd9b640 1 -- 172.21.15.84:0/2613864691 >> v1:172.21.15.157:6789/0 conn(0x7f87500d0be0 legacy=0x7f87500b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:38.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.167+0000 7f874dd9b640 1 -- 172.21.15.84:0/2613864691 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87500d1740 con 0x7f875015cd70 2023-10-16T11:26:38.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.167+0000 7f874dd9b640 1 -- 172.21.15.84:0/2613864691 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f8740001670 con 0x7f875015cd70 2023-10-16T11:26:38.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.167+0000 7f874dd9b640 1 -- 172.21.15.84:0/2613864691 >> v1:172.21.15.84:6789/0 conn(0x7f875015cd70 legacy=0x7f87500d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:38.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.167+0000 7f874dd9b640 1 --2- 172.21.15.84:0/2613864691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87500b8480 0x7f873c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:38.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.167+0000 7f874dd9b640 1 --2- 172.21.15.84:0/2613864691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f873c003f20 0x7f873c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:38.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.167+0000 7f874dd9b640 1 --2- 172.21.15.84:0/2613864691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f873c004d00 0x7f873c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:38.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.167+0000 7f874dd9b640 1 -- 172.21.15.84:0/2613864691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87500d1460 con 0x7f873c004d00 2023-10-16T11:26:38.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.167+0000 7f874dd9b640 1 -- 172.21.15.84:0/2613864691 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c00ba60 con 0x7f873c003f20 2023-10-16T11:26:38.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.167+0000 7f874dd9b640 1 -- 172.21.15.84:0/2613864691 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873c00bc20 con 0x7f87500b8480 2023-10-16T11:26:38.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.168+0000 7f874ed9d640 1 --2- 172.21.15.84:0/2613864691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f873c004d00 0x7f873c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:38.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.168+0000 7f874e59c640 1 --2- 172.21.15.84:0/2613864691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87500b8480 0x7f873c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:38.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.168+0000 7f8745d9b640 1 --2- 172.21.15.84:0/2613864691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f873c003f20 0x7f873c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:38.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.168+0000 7f874ed9d640 1 -- 172.21.15.84:0/2613864691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87500b8480 msgr2=0x7f873c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:38.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.168+0000 7f874ed9d640 1 --2- 172.21.15.84:0/2613864691 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87500b8480 0x7f873c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:38.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.168+0000 7f874ed9d640 1 -- 172.21.15.84:0/2613864691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f873c003f20 msgr2=0x7f873c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:38.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.168+0000 7f874ed9d640 1 --2- 172.21.15.84:0/2613864691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f873c003f20 0x7f873c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:38.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.168+0000 7f874ed9d640 1 -- 172.21.15.84:0/2613864691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c003520 con 0x7f873c004d00 2023-10-16T11:26:38.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.169+0000 7f874ed9d640 1 --2- 172.21.15.84:0/2613864691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f873c004d00 0x7f873c005180 secure :-1 s=READY pgs=3191 cs=0 l=1 rev1=1 crypto rx=0x7f8748001e50 tx=0x7f873c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363fbaa1caed93c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.169+0000 7f874dd9b640 1 -- 172.21.15.84:0/2613864691 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748003b70 con 0x7f873c004d00 2023-10-16T11:26:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.170+0000 7f874dd9b640 1 -- 172.21.15.84:0/2613864691 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8748003cd0 con 0x7f873c004d00 2023-10-16T11:26:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.170+0000 7f874dd9b640 1 -- 172.21.15.84:0/2613864691 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748005310 con 0x7f873c004d00 2023-10-16T11:26:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.170+0000 7f874f59e640 1 -- 172.21.15.84:0/2613864691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f873c004d00 msgr2=0x7f873c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.170+0000 7f874f59e640 1 --2- 172.21.15.84:0/2613864691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f873c004d00 0x7f873c005180 secure :-1 s=READY pgs=3191 cs=0 l=1 rev1=1 crypto rx=0x7f8748001e50 tx=0x7f873c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.170+0000 7f874ed9d640 1 -- 172.21.15.84:0/2613864691 reap_dead start 2023-10-16T11:26:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.170+0000 7f874f59e640 1 -- 172.21.15.84:0/2613864691 shutdown_connections 2023-10-16T11:26:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.170+0000 7f874f59e640 1 -- 172.21.15.84:0/2613864691 >> 172.21.15.84:0/2613864691 conn(0x7f87500cf570 msgr2=0x7f87500cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.170+0000 7f874f59e640 1 -- 172.21.15.84:0/2613864691 shutdown_connections 2023-10-16T11:26:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.170+0000 7f874f59e640 1 -- 172.21.15.84:0/2613864691 wait complete. 2023-10-16T11:26:38.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.172+0000 7f874f59e640 1 Processor -- start 2023-10-16T11:26:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.172+0000 7f874f59e640 1 -- start start 2023-10-16T11:26:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.173+0000 7f874f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f873c004d00 0x7f8750143850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.173+0000 7f874f59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87501600c0 0x7f87500cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.173+0000 7f874f59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f875015eb30 0x7f87500b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.173+0000 7f874f59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875015f020 con 0x7f873c004d00 2023-10-16T11:26:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.173+0000 7f874f59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875015f300 con 0x7f87501600c0 2023-10-16T11:26:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.173+0000 7f874f59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875015f5e0 con 0x7f875015eb30 2023-10-16T11:26:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.173+0000 7f874e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f873c004d00 0x7f8750143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.173+0000 7f874e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f873c004d00 0x7f8750143850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51898/0 (socket says 172.21.15.84:51898) 2023-10-16T11:26:38.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.173+0000 7f874e59c640 1 -- 172.21.15.84:0/1611600591 learned_addr learned my addr 172.21.15.84:0/1611600591 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:38.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.174+0000 7f874e59c640 1 -- 172.21.15.84:0/1611600591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f875015eb30 msgr2=0x7f87500b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:38.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.174+0000 7f8745d9b640 1 --2- 172.21.15.84:0/1611600591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87501600c0 0x7f87500cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:38.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.174+0000 7f874e59c640 1 --2- 172.21.15.84:0/1611600591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f875015eb30 0x7f87500b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:38.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.174+0000 7f874e59c640 1 -- 172.21.15.84:0/1611600591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87501600c0 msgr2=0x7f87500cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:38.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.174+0000 7f874e59c640 1 --2- 172.21.15.84:0/1611600591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87501600c0 0x7f87500cb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:38.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.174+0000 7f874e59c640 1 -- 172.21.15.84:0/1611600591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875015f8c0 con 0x7f873c004d00 2023-10-16T11:26:38.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.174+0000 7f874e59c640 1 --2- 172.21.15.84:0/1611600591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f873c004d00 0x7f8750143850 secure :-1 s=READY pgs=3192 cs=0 l=1 rev1=1 crypto rx=0x7f874000e870 tx=0x7f874000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:38.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.174+0000 7f87477fe640 1 -- 172.21.15.84:0/1611600591 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874000c850 con 0x7f873c004d00 2023-10-16T11:26:38.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.174+0000 7f87477fe640 1 -- 172.21.15.84:0/1611600591 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874000c9d0 con 0x7f873c004d00 2023-10-16T11:26:38.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.174+0000 7f87477fe640 1 -- 172.21.15.84:0/1611600591 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874000cd80 con 0x7f873c004d00 2023-10-16T11:26:38.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.174+0000 7f874f59e640 1 -- 172.21.15.84:0/1611600591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f875015fba0 con 0x7f873c004d00 2023-10-16T11:26:38.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.174+0000 7f874f59e640 1 -- 172.21.15.84:0/1611600591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87500cb650 con 0x7f873c004d00 2023-10-16T11:26:38.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.176+0000 7f874f59e640 1 -- 172.21.15.84:0/1611600591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87500cd000 con 0x7f873c004d00 2023-10-16T11:26:38.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.177+0000 7f87477fe640 1 -- 172.21.15.84:0/1611600591 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f874001d460 con 0x7f873c004d00 2023-10-16T11:26:38.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.177+0000 7f87477fe640 1 --2- 172.21.15.84:0/1611600591 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f873006b020 0x7f873006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:38.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.177+0000 7f87477fe640 1 -- 172.21.15.84:0/1611600591 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f87400896c0 con 0x7f873c004d00 2023-10-16T11:26:38.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.178+0000 7f8745d9b640 1 --2- 172.21.15.84:0/1611600591 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f873006b020 0x7f873006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:38.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.178+0000 7f8745d9b640 1 --2- 172.21.15.84:0/1611600591 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f873006b020 0x7f873006d4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f87500ce580 tx=0x7f8738008040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:38.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.182+0000 7f87477fe640 1 -- 172.21.15.84:0/1611600591 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8740054790 con 0x7f873c004d00 2023-10-16T11:26:38.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.377+0000 7f874f59e640 1 -- 172.21.15.84:0/1611600591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f875004e9b0 con 0x7f873c004d00 2023-10-16T11:26:38.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.379+0000 7f87477fe640 1 -- 172.21.15.84:0/1611600591 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v253) v1 ==== 74+0+12887 (secure 0 0 0) 0x7f8740057db0 con 0x7f873c004d00 2023-10-16T11:26:38.381 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:38.381 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":253,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:26:37.404077+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:26:38.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.381+0000 7f874f59e640 1 -- 172.21.15.84:0/1611600591 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f873006b020 msgr2=0x7f873006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:38.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.381+0000 7f874f59e640 1 --2- 172.21.15.84:0/1611600591 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f873006b020 0x7f873006d4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f87500ce580 tx=0x7f8738008040 comp rx=0 tx=0).stop 2023-10-16T11:26:38.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.381+0000 7f874f59e640 1 -- 172.21.15.84:0/1611600591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f873c004d00 msgr2=0x7f8750143850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:38.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.381+0000 7f874f59e640 1 --2- 172.21.15.84:0/1611600591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f873c004d00 0x7f8750143850 secure :-1 s=READY pgs=3192 cs=0 l=1 rev1=1 crypto rx=0x7f874000e870 tx=0x7f874000ed30 comp rx=0 tx=0).stop 2023-10-16T11:26:38.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.382+0000 7f874f59e640 1 -- 172.21.15.84:0/1611600591 shutdown_connections 2023-10-16T11:26:38.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.382+0000 7f874f59e640 1 --2- 172.21.15.84:0/1611600591 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f873006b020 0x7f873006d4e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:38.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.382+0000 7f874f59e640 1 --2- 172.21.15.84:0/1611600591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f875015eb30 0x7f87500b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:38.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.382+0000 7f874f59e640 1 --2- 172.21.15.84:0/1611600591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87501600c0 0x7f87500cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:38.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.382+0000 7f874f59e640 1 --2- 172.21.15.84:0/1611600591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f873c004d00 0x7f8750143850 unknown :-1 s=CLOSED pgs=3192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:38.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.382+0000 7f874f59e640 1 -- 172.21.15.84:0/1611600591 >> 172.21.15.84:0/1611600591 conn(0x7f87500cf570 msgr2=0x7f87500d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:38.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.382+0000 7f874f59e640 1 -- 172.21.15.84:0/1611600591 shutdown_connections 2023-10-16T11:26:38.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.382+0000 7f874f59e640 1 -- 172.21.15.84:0/1611600591 wait complete. 2023-10-16T11:26:38.405 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:26:38.406 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:26:38.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.618+0000 7fb883a7c640 1 Processor -- start 2023-10-16T11:26:38.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.619+0000 7fb883a7c640 1 -- start start 2023-10-16T11:26:38.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.620+0000 7fb883a7c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb87c06c850 0x7fb87c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:38.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.620+0000 7fb883a7c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87c070360 con 0x7fb87c159e90 2023-10-16T11:26:38.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.620+0000 7fb883a7c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87c0cd1f0 con 0x7fb87c159ab0 2023-10-16T11:26:38.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.620+0000 7fb883a7c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb87c15d890 con 0x7fb87c06c850 2023-10-16T11:26:38.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.620+0000 7fb881017640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb87c159e90 0x7fb87c06c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38784/0 (socket says 172.21.15.84:38784) 2023-10-16T11:26:38.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.621+0000 7fb882019640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb87c06c850 0x7fb87c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:38.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.621+0000 7fb881017640 1 -- 172.21.15.84:0/3789710530 learned_addr learned my addr 172.21.15.84:0/3789710530 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:38.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.621+0000 7fb882019640 1 -- 172.21.15.84:0/3789710530 >> v1:172.21.15.157:6789/0 conn(0x7fb87c159ab0 legacy=0x7fb87c06fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:38.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.621+0000 7fb882019640 1 -- 172.21.15.84:0/3789710530 >> v1:172.21.15.84:6789/0 conn(0x7fb87c159e90 legacy=0x7fb87c06c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:38.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.621+0000 7fb882019640 1 -- 172.21.15.84:0/3789710530 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c0cd4d0 con 0x7fb87c06c850 2023-10-16T11:26:38.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.622+0000 7fb882019640 1 --2- 172.21.15.84:0/3789710530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87c06c850 0x7fb87c0ce370 secure :-1 s=READY pgs=3194 cs=0 l=1 rev1=1 crypto rx=0x7fb874009d90 tx=0x7fb87402bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e718808456ec15f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:38.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.622+0000 7fb880816640 1 -- 172.21.15.84:0/3789710530 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87402cb40 con 0x7fb87c06c850 2023-10-16T11:26:38.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.622+0000 7fb880816640 1 -- 172.21.15.84:0/3789710530 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87402cca0 con 0x7fb87c06c850 2023-10-16T11:26:38.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.623+0000 7fb880816640 1 -- 172.21.15.84:0/3789710530 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874034660 con 0x7fb87c06c850 2023-10-16T11:26:38.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.623+0000 7fb883a7c640 1 -- 172.21.15.84:0/3789710530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87c06c850 msgr2=0x7fb87c0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:38.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.623+0000 7fb883a7c640 1 --2- 172.21.15.84:0/3789710530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87c06c850 0x7fb87c0ce370 secure :-1 s=READY pgs=3194 cs=0 l=1 rev1=1 crypto rx=0x7fb874009d90 tx=0x7fb87402bd90 comp rx=0 tx=0).stop 2023-10-16T11:26:38.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.623+0000 7fb883a7c640 1 -- 172.21.15.84:0/3789710530 shutdown_connections 2023-10-16T11:26:38.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.623+0000 7fb883a7c640 1 --2- 172.21.15.84:0/3789710530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87c06c850 0x7fb87c0ce370 unknown :-1 s=CLOSED pgs=3194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:38.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.623+0000 7fb883a7c640 1 -- 172.21.15.84:0/3789710530 >> 172.21.15.84:0/3789710530 conn(0x7fb87c0ce9b0 msgr2=0x7fb87c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:38.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.623+0000 7fb883a7c640 1 -- 172.21.15.84:0/3789710530 shutdown_connections 2023-10-16T11:26:38.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.623+0000 7fb883a7c640 1 -- 172.21.15.84:0/3789710530 wait complete. 2023-10-16T11:26:38.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.625+0000 7fb883a7c640 1 Processor -- start 2023-10-16T11:26:38.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.625+0000 7fb883a7c640 1 -- start start 2023-10-16T11:26:38.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.626+0000 7fb883a7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb87c06c850 0x7fb87c0c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:38.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.626+0000 7fb883a7c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87c159ab0 0x7fb87c0c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:38.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.626+0000 7fb883a7c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb87c159e90 0x7fb87c15ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:38.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.626+0000 7fb883a7c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c0c2370 con 0x7fb87c159ab0 2023-10-16T11:26:38.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.626+0000 7fb883a7c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c0c9690 con 0x7fb87c06c850 2023-10-16T11:26:38.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.626+0000 7fb883a7c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c0c5a80 con 0x7fb87c159e90 2023-10-16T11:26:38.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.626+0000 7fb881017640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87c159ab0 0x7fb87c0c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:38.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.627+0000 7fb881017640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87c159ab0 0x7fb87c0c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51916/0 (socket says 172.21.15.84:51916) 2023-10-16T11:26:38.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.627+0000 7fb881017640 1 -- 172.21.15.84:0/1674487860 learned_addr learned my addr 172.21.15.84:0/1674487860 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:38.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.627+0000 7fb881818640 1 --2- 172.21.15.84:0/1674487860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb87c06c850 0x7fb87c0c1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:38.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.627+0000 7fb881017640 1 -- 172.21.15.84:0/1674487860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb87c159e90 msgr2=0x7fb87c15ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:38.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.627+0000 7fb881017640 1 --2- 172.21.15.84:0/1674487860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb87c159e90 0x7fb87c15ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:38.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.627+0000 7fb881017640 1 -- 172.21.15.84:0/1674487860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb87c06c850 msgr2=0x7fb87c0c1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:38.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.627+0000 7fb881017640 1 --2- 172.21.15.84:0/1674487860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb87c06c850 0x7fb87c0c1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:38.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.627+0000 7fb881017640 1 -- 172.21.15.84:0/1674487860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb874009a70 con 0x7fb87c159ab0 2023-10-16T11:26:38.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.627+0000 7fb881017640 1 --2- 172.21.15.84:0/1674487860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87c159ab0 0x7fb87c0c9150 secure :-1 s=READY pgs=3195 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00c400 tx=0x7fb86c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:38.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.627+0000 7fb8727fc640 1 -- 172.21.15.84:0/1674487860 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c005ce0 con 0x7fb87c159ab0 2023-10-16T11:26:38.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.628+0000 7fb8727fc640 1 -- 172.21.15.84:0/1674487860 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c005e40 con 0x7fb87c159ab0 2023-10-16T11:26:38.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.628+0000 7fb883a7c640 1 -- 172.21.15.84:0/1674487860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb87c0c98c0 con 0x7fb87c159ab0 2023-10-16T11:26:38.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.628+0000 7fb8727fc640 1 -- 172.21.15.84:0/1674487860 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c007310 con 0x7fb87c159ab0 2023-10-16T11:26:38.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.628+0000 7fb883a7c640 1 -- 172.21.15.84:0/1674487860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87c0c6020 con 0x7fb87c159ab0 2023-10-16T11:26:38.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.629+0000 7fb883a7c640 1 -- 172.21.15.84:0/1674487860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87c04e160 con 0x7fb87c159ab0 2023-10-16T11:26:38.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.630+0000 7fb8727fc640 1 -- 172.21.15.84:0/1674487860 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fb86c007470 con 0x7fb87c159ab0 2023-10-16T11:26:38.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.631+0000 7fb8727fc640 1 --2- 172.21.15.84:0/1674487860 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fb86006b020 0x7fb86006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:38.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.631+0000 7fb8727fc640 1 -- 172.21.15.84:0/1674487860 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fb86c088c40 con 0x7fb87c159ab0 2023-10-16T11:26:38.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.631+0000 7fb881818640 1 --2- 172.21.15.84:0/1674487860 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fb86006b020 0x7fb86006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:38.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.632+0000 7fb881818640 1 --2- 172.21.15.84:0/1674487860 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fb86006b020 0x7fb86006d4e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb878003d90 tx=0x7fb878009040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:38.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.635+0000 7fb8727fc640 1 -- 172.21.15.84:0/1674487860 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb86c053d00 con 0x7fb87c159ab0 2023-10-16T11:26:38.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.834+0000 7fb883a7c640 1 -- 172.21.15.84:0/1674487860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb87c0ccb60 con 0x7fb87c159ab0 2023-10-16T11:26:38.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.835+0000 7fb8727fc640 1 -- 172.21.15.84:0/1674487860 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v253) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb86c0170a0 con 0x7fb87c159ab0 2023-10-16T11:26:38.837 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:26:38.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.838+0000 7fb883a7c640 1 -- 172.21.15.84:0/1674487860 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fb86006b020 msgr2=0x7fb86006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:38.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.838+0000 7fb883a7c640 1 --2- 172.21.15.84:0/1674487860 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fb86006b020 0x7fb86006d4e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb878003d90 tx=0x7fb878009040 comp rx=0 tx=0).stop 2023-10-16T11:26:38.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.838+0000 7fb883a7c640 1 -- 172.21.15.84:0/1674487860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87c159ab0 msgr2=0x7fb87c0c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:38.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.838+0000 7fb883a7c640 1 --2- 172.21.15.84:0/1674487860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87c159ab0 0x7fb87c0c9150 secure :-1 s=READY pgs=3195 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00c400 tx=0x7fb86c00c8c0 comp rx=0 tx=0).stop 2023-10-16T11:26:38.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.838+0000 7fb883a7c640 1 -- 172.21.15.84:0/1674487860 shutdown_connections 2023-10-16T11:26:38.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.838+0000 7fb883a7c640 1 --2- 172.21.15.84:0/1674487860 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fb86006b020 0x7fb86006d4e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:38.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.838+0000 7fb883a7c640 1 --2- 172.21.15.84:0/1674487860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb87c159e90 0x7fb87c15ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:38.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.838+0000 7fb883a7c640 1 --2- 172.21.15.84:0/1674487860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87c159ab0 0x7fb87c0c9150 unknown :-1 s=CLOSED pgs=3195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:38.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.838+0000 7fb883a7c640 1 --2- 172.21.15.84:0/1674487860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb87c06c850 0x7fb87c0c1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:38.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.838+0000 7fb883a7c640 1 -- 172.21.15.84:0/1674487860 >> 172.21.15.84:0/1674487860 conn(0x7fb87c0ce9b0 msgr2=0x7fb87c0b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:38.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.838+0000 7fb883a7c640 1 -- 172.21.15.84:0/1674487860 shutdown_connections 2023-10-16T11:26:38.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:38.838+0000 7fb883a7c640 1 -- 172.21.15.84:0/1674487860 wait complete. 2023-10-16T11:26:38.860 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T11:26:38.910 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:38.907+0000 7f2ae1e3b640 -1 client.12678: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.12570:a9ee7438-a42a-4b6b-8df1-d3312e55d4e1:v1:172.21.15.84:0/570042115] 2023-10-16T11:26:39.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.071+0000 7f4aaf528640 1 Processor -- start 2023-10-16T11:26:39.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.071+0000 7f4aaf528640 1 -- start start 2023-10-16T11:26:39.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.072+0000 7f4aaf528640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4aa80ad970 0x7f4aa80cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:39.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.072+0000 7f4aaf528640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4aa80ccfb0 0x7f4aa80cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:39.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.072+0000 7f4aaf528640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa815b9d0 con 0x7f4aa806c040 2023-10-16T11:26:39.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.073+0000 7f4aaf528640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa815bcb0 con 0x7f4aa80ad970 2023-10-16T11:26:39.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.073+0000 7f4aaf528640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa815bf90 con 0x7f4aa80ccfb0 2023-10-16T11:26:39.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.073+0000 7f4aae526640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4aa806c040 0x7f4aa806c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38792/0 (socket says 172.21.15.84:38792) 2023-10-16T11:26:39.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.073+0000 7f4aae526640 1 -- 172.21.15.84:0/1974626732 learned_addr learned my addr 172.21.15.84:0/1974626732 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:39.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.073+0000 7f4aadd25640 1 --2- 172.21.15.84:0/1974626732 >> v2:172.21.15.84:3300/0 conn(0x7f4aa80ad970 0x7f4aa80cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:39.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.073+0000 7f4aaed27640 1 --2- 172.21.15.84:0/1974626732 >> v2:172.21.15.157:3300/0 conn(0x7f4aa80ccfb0 0x7f4aa80cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:39.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.073+0000 7f4aad524640 1 -- 172.21.15.84:0/1974626732 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3503483143 0 0) 0x7f4aa815b9d0 con 0x7f4aa806c040 2023-10-16T11:26:39.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.073+0000 7f4aad524640 1 -- 172.21.15.84:0/1974626732 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a94003520 con 0x7f4aa806c040 2023-10-16T11:26:39.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.073+0000 7f4aadd25640 1 -- 172.21.15.84:0/1974626732 >> v2:172.21.15.157:3300/0 conn(0x7f4aa80ccfb0 msgr2=0x7f4aa80cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:39.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.073+0000 7f4aadd25640 1 --2- 172.21.15.84:0/1974626732 >> v2:172.21.15.157:3300/0 conn(0x7f4aa80ccfb0 0x7f4aa80cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:39.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.073+0000 7f4aadd25640 1 -- 172.21.15.84:0/1974626732 >> v1:172.21.15.84:6789/0 conn(0x7f4aa806c040 legacy=0x7f4aa806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:39.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.073+0000 7f4aadd25640 1 -- 172.21.15.84:0/1974626732 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa80cc270 con 0x7f4aa80ad970 2023-10-16T11:26:39.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.075+0000 7f4aadd25640 1 --2- 172.21.15.84:0/1974626732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aa80ad970 0x7f4aa80cca70 secure :-1 s=READY pgs=3197 cs=0 l=1 rev1=1 crypto rx=0x7f4a980097a0 tx=0x7f4a9802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41c41e745e7ed7b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:39.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.075+0000 7f4aad524640 1 -- 172.21.15.84:0/1974626732 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a98031830 con 0x7f4aa80ad970 2023-10-16T11:26:39.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.075+0000 7f4aad524640 1 -- 172.21.15.84:0/1974626732 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a98031990 con 0x7f4aa80ad970 2023-10-16T11:26:39.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.075+0000 7f4aad524640 1 -- 172.21.15.84:0/1974626732 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a98031d20 con 0x7f4aa80ad970 2023-10-16T11:26:39.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.075+0000 7f4aaf528640 1 -- 172.21.15.84:0/1974626732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aa80ad970 msgr2=0x7f4aa80cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:39.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.075+0000 7f4aaf528640 1 --2- 172.21.15.84:0/1974626732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aa80ad970 0x7f4aa80cca70 secure :-1 s=READY pgs=3197 cs=0 l=1 rev1=1 crypto rx=0x7f4a980097a0 tx=0x7f4a9802b8d0 comp rx=0 tx=0).stop 2023-10-16T11:26:39.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.076+0000 7f4aaf528640 1 -- 172.21.15.84:0/1974626732 shutdown_connections 2023-10-16T11:26:39.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.076+0000 7f4aaf528640 1 --2- 172.21.15.84:0/1974626732 >> v2:172.21.15.157:3300/0 conn(0x7f4aa80ccfb0 0x7f4aa80cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:39.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.076+0000 7f4aaf528640 1 --2- 172.21.15.84:0/1974626732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aa80ad970 0x7f4aa80cca70 unknown :-1 s=CLOSED pgs=3197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:39.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.076+0000 7f4aaf528640 1 -- 172.21.15.84:0/1974626732 >> 172.21.15.84:0/1974626732 conn(0x7f4aa80ce6a0 msgr2=0x7f4aa80ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:39.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.076+0000 7f4aaf528640 1 -- 172.21.15.84:0/1974626732 shutdown_connections 2023-10-16T11:26:39.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.076+0000 7f4aaf528640 1 -- 172.21.15.84:0/1974626732 wait complete. 2023-10-16T11:26:39.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.077+0000 7f4aaf528640 1 Processor -- start 2023-10-16T11:26:39.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.077+0000 7f4aaf528640 1 -- start start 2023-10-16T11:26:39.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.078+0000 7f4aaf528640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aa806c040 0x7f4aa80c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:39.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.078+0000 7f4aaf528640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aa80ad970 0x7f4aa80acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:39.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.078+0000 7f4aaf528640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aa80ccfb0 0x7f4aa8156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:39.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.078+0000 7f4aaf528640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa80ca5a0 con 0x7f4aa806c040 2023-10-16T11:26:39.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.078+0000 7f4aaf528640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa80c9230 con 0x7f4aa80ccfb0 2023-10-16T11:26:39.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.078+0000 7f4aaf528640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa80c9500 con 0x7f4aa80ad970 2023-10-16T11:26:39.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.079+0000 7f4aae526640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aa806c040 0x7f4aa80c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:39.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.079+0000 7f4aae526640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aa806c040 0x7f4aa80c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51932/0 (socket says 172.21.15.84:51932) 2023-10-16T11:26:39.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.079+0000 7f4aae526640 1 -- 172.21.15.84:0/3728304315 learned_addr learned my addr 172.21.15.84:0/3728304315 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:39.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.079+0000 7f4aadd25640 1 --2- 172.21.15.84:0/3728304315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aa80ad970 0x7f4aa80acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:39.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.079+0000 7f4aaed27640 1 --2- 172.21.15.84:0/3728304315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aa80ccfb0 0x7f4aa8156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:39.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.079+0000 7f4aae526640 1 -- 172.21.15.84:0/3728304315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aa80ad970 msgr2=0x7f4aa80acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:39.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.079+0000 7f4aae526640 1 --2- 172.21.15.84:0/3728304315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aa80ad970 0x7f4aa80acb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:39.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.079+0000 7f4aae526640 1 -- 172.21.15.84:0/3728304315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aa80ccfb0 msgr2=0x7f4aa8156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:39.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.079+0000 7f4aae526640 1 --2- 172.21.15.84:0/3728304315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aa80ccfb0 0x7f4aa8156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:39.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.079+0000 7f4aae526640 1 -- 172.21.15.84:0/3728304315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a98009400 con 0x7f4aa806c040 2023-10-16T11:26:39.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.079+0000 7f4aaed27640 1 --2- 172.21.15.84:0/3728304315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aa80ccfb0 0x7f4aa8156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:39.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.079+0000 7f4aae526640 1 --2- 172.21.15.84:0/3728304315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aa806c040 0x7f4aa80c8cf0 secure :-1 s=READY pgs=3198 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400c500 tx=0x7f4aa400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:39.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.080+0000 7f4a9effd640 1 -- 172.21.15.84:0/3728304315 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4005ce0 con 0x7f4aa806c040 2023-10-16T11:26:39.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.080+0000 7f4a9effd640 1 -- 172.21.15.84:0/3728304315 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa4005e40 con 0x7f4aa806c040 2023-10-16T11:26:39.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.080+0000 7f4aaf528640 1 -- 172.21.15.84:0/3728304315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa80c97e0 con 0x7f4aa806c040 2023-10-16T11:26:39.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.080+0000 7f4a9effd640 1 -- 172.21.15.84:0/3728304315 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4007330 con 0x7f4aa806c040 2023-10-16T11:26:39.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.080+0000 7f4aaf528640 1 -- 172.21.15.84:0/3728304315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa80c9da0 con 0x7f4aa806c040 2023-10-16T11:26:39.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.081+0000 7f4aaf528640 1 -- 172.21.15.84:0/3728304315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa806b330 con 0x7f4aa806c040 2023-10-16T11:26:39.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.082+0000 7f4a9effd640 1 -- 172.21.15.84:0/3728304315 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f4aa4007550 con 0x7f4aa806c040 2023-10-16T11:26:39.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.083+0000 7f4a9effd640 1 --2- 172.21.15.84:0/3728304315 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f4a8c06b020 0x7f4a8c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:39.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.083+0000 7f4a9effd640 1 -- 172.21.15.84:0/3728304315 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f4aa4088cb0 con 0x7f4aa806c040 2023-10-16T11:26:39.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.083+0000 7f4aadd25640 1 --2- 172.21.15.84:0/3728304315 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f4a8c06b020 0x7f4a8c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:39.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.084+0000 7f4aadd25640 1 --2- 172.21.15.84:0/3728304315 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f4a8c06b020 0x7f4a8c06d4e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4a98036000 tx=0x7f4a9803c040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:39.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.088+0000 7f4a9effd640 1 -- 172.21.15.84:0/3728304315 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4aa4053ec0 con 0x7f4aa806c040 2023-10-16T11:26:39.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.290+0000 7f4aaf528640 1 -- 172.21.15.84:0/3728304315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4aa804e9b0 con 0x7f4aa806c040 2023-10-16T11:26:39.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.291+0000 7f4a9effd640 1 -- 172.21.15.84:0/3728304315 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v253) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4aa4017020 con 0x7f4aa806c040 2023-10-16T11:26:39.293 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:39.294 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T11:26:39.294 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:39.294 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-16T11:26:39.294 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-16T11:26:39.295 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:39.295 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T11:26:39.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.294+0000 7f4aaf528640 1 -- 172.21.15.84:0/3728304315 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f4a8c06b020 msgr2=0x7f4a8c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:39.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.294+0000 7f4aaf528640 1 --2- 172.21.15.84:0/3728304315 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f4a8c06b020 0x7f4a8c06d4e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4a98036000 tx=0x7f4a9803c040 comp rx=0 tx=0).stop 2023-10-16T11:26:39.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.294+0000 7f4aaf528640 1 -- 172.21.15.84:0/3728304315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aa806c040 msgr2=0x7f4aa80c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:39.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.294+0000 7f4aaf528640 1 --2- 172.21.15.84:0/3728304315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aa806c040 0x7f4aa80c8cf0 secure :-1 s=READY pgs=3198 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400c500 tx=0x7f4aa400c9c0 comp rx=0 tx=0).stop 2023-10-16T11:26:39.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.294+0000 7f4aaf528640 1 -- 172.21.15.84:0/3728304315 shutdown_connections 2023-10-16T11:26:39.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.294+0000 7f4aaf528640 1 --2- 172.21.15.84:0/3728304315 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f4a8c06b020 0x7f4a8c06d4e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:39.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.294+0000 7f4aaf528640 1 --2- 172.21.15.84:0/3728304315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aa80ccfb0 0x7f4aa8156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:39.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.294+0000 7f4aaf528640 1 --2- 172.21.15.84:0/3728304315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aa80ad970 0x7f4aa80acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:39.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.294+0000 7f4aaf528640 1 --2- 172.21.15.84:0/3728304315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aa806c040 0x7f4aa80c8cf0 unknown :-1 s=CLOSED pgs=3198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:39.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.294+0000 7f4aaf528640 1 -- 172.21.15.84:0/3728304315 >> 172.21.15.84:0/3728304315 conn(0x7f4aa80ce6a0 msgr2=0x7f4aa80c5770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:39.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.294+0000 7f4aaf528640 1 -- 172.21.15.84:0/3728304315 shutdown_connections 2023-10-16T11:26:39.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.294+0000 7f4aaf528640 1 -- 172.21.15.84:0/3728304315 wait complete. 2023-10-16T11:26:39.315 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T11:26:39.316 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T11:26:39.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.580+0000 7f7a6988a640 1 Processor -- start 2023-10-16T11:26:39.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.581+0000 7f7a6988a640 1 -- start start 2023-10-16T11:26:39.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.582+0000 7f7a6988a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7a64071460 0x7f7a64165570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:39.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.582+0000 7f7a6988a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a640cae70 con 0x7f7a640adec0 2023-10-16T11:26:39.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.582+0000 7f7a6988a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6406b860 con 0x7f7a640c18e0 2023-10-16T11:26:39.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.582+0000 7f7a6988a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a64165de0 con 0x7f7a64071460 2023-10-16T11:26:39.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.582+0000 7f7a68888640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7a640adec0 0x7f7a64164e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38796/0 (socket says 172.21.15.84:38796) 2023-10-16T11:26:39.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.582+0000 7f7a68888640 1 -- 172.21.15.84:0/2526947617 learned_addr learned my addr 172.21.15.84:0/2526947617 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:39.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.583+0000 7f7a637fe640 1 -- 172.21.15.84:0/2526947617 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459196524 0 0) 0x7f7a640cae70 con 0x7f7a640adec0 2023-10-16T11:26:39.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.583+0000 7f7a63fff640 1 --2- 172.21.15.84:0/2526947617 >> v2:172.21.15.157:3300/0 conn(0x7f7a64071460 0x7f7a64165570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:39.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.583+0000 7f7a637fe640 1 -- 172.21.15.84:0/2526947617 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a4c003520 con 0x7f7a640adec0 2023-10-16T11:26:39.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.583+0000 7f7a637fe640 1 -- 172.21.15.84:0/2526947617 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4029176249 0 0) 0x7f7a4c003520 con 0x7f7a640adec0 2023-10-16T11:26:39.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.583+0000 7f7a637fe640 1 -- 172.21.15.84:0/2526947617 >> v2:172.21.15.157:3300/0 conn(0x7f7a64071460 msgr2=0x7f7a64165570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:39.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.583+0000 7f7a637fe640 1 --2- 172.21.15.84:0/2526947617 >> v2:172.21.15.157:3300/0 conn(0x7f7a64071460 0x7f7a64165570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:39.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.583+0000 7f7a637fe640 1 -- 172.21.15.84:0/2526947617 >> v1:172.21.15.157:6789/0 conn(0x7f7a640c18e0 legacy=0x7f7a640c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:39.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.583+0000 7f7a637fe640 1 -- 172.21.15.84:0/2526947617 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6406b580 con 0x7f7a640adec0 2023-10-16T11:26:39.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.583+0000 7f7a637fe640 1 -- 172.21.15.84:0/2526947617 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f7a58001670 con 0x7f7a640adec0 2023-10-16T11:26:39.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.583+0000 7f7a63fff640 1 --2- 172.21.15.84:0/2526947617 >> v2:172.21.15.157:3300/0 conn(0x7f7a64071460 0x7f7a64165570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:26:39.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.583+0000 7f7a637fe640 1 -- 172.21.15.84:0/2526947617 >> v1:172.21.15.84:6789/0 conn(0x7f7a640adec0 legacy=0x7f7a64164e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:39.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.583+0000 7f7a637fe640 1 --2- 172.21.15.84:0/2526947617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a4c003a30 0x7f7a4c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:39.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.583+0000 7f7a637fe640 1 --2- 172.21.15.84:0/2526947617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4c0047c0 0x7f7a4c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:39.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.583+0000 7f7a637fe640 1 --2- 172.21.15.84:0/2526947617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a4c0052a0 0x7f7a4c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:39.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.583+0000 7f7a637fe640 1 -- 172.21.15.84:0/2526947617 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a64165de0 con 0x7f7a4c003a30 2023-10-16T11:26:39.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.583+0000 7f7a637fe640 1 -- 172.21.15.84:0/2526947617 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c00c190 con 0x7f7a4c0052a0 2023-10-16T11:26:39.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.583+0000 7f7a637fe640 1 -- 172.21.15.84:0/2526947617 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c00c350 con 0x7f7a4c0047c0 2023-10-16T11:26:39.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.584+0000 7f7a68888640 1 --2- 172.21.15.84:0/2526947617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a4c003a30 0x7f7a4c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:39.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.584+0000 7f7a63fff640 1 --2- 172.21.15.84:0/2526947617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4c0047c0 0x7f7a4c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:39.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.584+0000 7f7a68888640 1 -- 172.21.15.84:0/2526947617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4c0047c0 msgr2=0x7f7a4c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:39.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.584+0000 7f7a68888640 1 --2- 172.21.15.84:0/2526947617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4c0047c0 0x7f7a4c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:39.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.584+0000 7f7a68888640 1 -- 172.21.15.84:0/2526947617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a4c0052a0 msgr2=0x7f7a4c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:39.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.584+0000 7f7a69089640 1 --2- 172.21.15.84:0/2526947617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a4c0052a0 0x7f7a4c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:39.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.584+0000 7f7a68888640 1 --2- 172.21.15.84:0/2526947617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a4c0052a0 0x7f7a4c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:39.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.584+0000 7f7a68888640 1 -- 172.21.15.84:0/2526947617 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c003520 con 0x7f7a4c003a30 2023-10-16T11:26:39.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.584+0000 7f7a69089640 1 -- 172.21.15.84:0/2526947617 reap_dead start 2023-10-16T11:26:39.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.585+0000 7f7a68888640 1 --2- 172.21.15.84:0/2526947617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a4c003a30 0x7f7a4c003e50 secure :-1 s=READY pgs=3200 cs=0 l=1 rev1=1 crypto rx=0x7f7a58001eb0 tx=0x7f7a4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f537bffd69600572 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:39.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.585+0000 7f7a637fe640 1 -- 172.21.15.84:0/2526947617 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a58005ad0 con 0x7f7a4c003a30 2023-10-16T11:26:39.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.585+0000 7f7a637fe640 1 -- 172.21.15.84:0/2526947617 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a58005c30 con 0x7f7a4c003a30 2023-10-16T11:26:39.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.585+0000 7f7a637fe640 1 -- 172.21.15.84:0/2526947617 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5802ca00 con 0x7f7a4c003a30 2023-10-16T11:26:39.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.586+0000 7f7a6988a640 1 -- 172.21.15.84:0/2526947617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a4c003a30 msgr2=0x7f7a4c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:39.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.586+0000 7f7a6988a640 1 --2- 172.21.15.84:0/2526947617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a4c003a30 0x7f7a4c003e50 secure :-1 s=READY pgs=3200 cs=0 l=1 rev1=1 crypto rx=0x7f7a58001eb0 tx=0x7f7a4c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:39.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.586+0000 7f7a6988a640 1 -- 172.21.15.84:0/2526947617 shutdown_connections 2023-10-16T11:26:39.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.586+0000 7f7a6988a640 1 --2- 172.21.15.84:0/2526947617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a4c003a30 0x7f7a4c003e50 unknown :-1 s=CLOSED pgs=3200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:39.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.586+0000 7f7a6988a640 1 -- 172.21.15.84:0/2526947617 >> 172.21.15.84:0/2526947617 conn(0x7f7a640ce820 msgr2=0x7f7a640cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:39.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.586+0000 7f7a6988a640 1 -- 172.21.15.84:0/2526947617 shutdown_connections 2023-10-16T11:26:39.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.586+0000 7f7a6988a640 1 -- 172.21.15.84:0/2526947617 wait complete. 2023-10-16T11:26:39.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.588+0000 7f7a6988a640 1 Processor -- start 2023-10-16T11:26:39.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.588+0000 7f7a6988a640 1 -- start start 2023-10-16T11:26:39.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.589+0000 7f7a6988a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a4c003a30 0x7f7a641555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:39.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.589+0000 7f7a6988a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a6415ec80 0x7f7a6415f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:39.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.589+0000 7f7a6988a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a640c2320 0x7f7a640c27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:39.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.589+0000 7f7a6988a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a64156200 con 0x7f7a640c2320 2023-10-16T11:26:39.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.589+0000 7f7a6988a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6415d900 con 0x7f7a4c003a30 2023-10-16T11:26:39.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.589+0000 7f7a6988a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6415dbe0 con 0x7f7a6415ec80 2023-10-16T11:26:39.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.590+0000 7f7a69089640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a640c2320 0x7f7a640c27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:39.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.590+0000 7f7a69089640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a640c2320 0x7f7a640c27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51948/0 (socket says 172.21.15.84:51948) 2023-10-16T11:26:39.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.590+0000 7f7a69089640 1 -- 172.21.15.84:0/4250249372 learned_addr learned my addr 172.21.15.84:0/4250249372 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:39.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.590+0000 7f7a63fff640 1 --2- 172.21.15.84:0/4250249372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a6415ec80 0x7f7a6415f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:39.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.590+0000 7f7a68888640 1 --2- 172.21.15.84:0/4250249372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a4c003a30 0x7f7a641555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:39.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.590+0000 7f7a69089640 1 -- 172.21.15.84:0/4250249372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a6415ec80 msgr2=0x7f7a6415f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:39.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.590+0000 7f7a69089640 1 --2- 172.21.15.84:0/4250249372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a6415ec80 0x7f7a6415f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:39.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.590+0000 7f7a69089640 1 -- 172.21.15.84:0/4250249372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a4c003a30 msgr2=0x7f7a641555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:39.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.590+0000 7f7a69089640 1 --2- 172.21.15.84:0/4250249372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a4c003a30 0x7f7a641555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:39.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.590+0000 7f7a69089640 1 -- 172.21.15.84:0/4250249372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6415dec0 con 0x7f7a640c2320 2023-10-16T11:26:39.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.590+0000 7f7a69089640 1 --2- 172.21.15.84:0/4250249372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a640c2320 0x7f7a640c27a0 secure :-1 s=READY pgs=3201 cs=0 l=1 rev1=1 crypto rx=0x7f7a64071d20 tx=0x7f7a5c00d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:39.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.591+0000 7f7a617fa640 1 -- 172.21.15.84:0/4250249372 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c00b070 con 0x7f7a640c2320 2023-10-16T11:26:39.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.591+0000 7f7a617fa640 1 -- 172.21.15.84:0/4250249372 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c003d80 con 0x7f7a640c2320 2023-10-16T11:26:39.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.591+0000 7f7a617fa640 1 -- 172.21.15.84:0/4250249372 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0052f0 con 0x7f7a640c2320 2023-10-16T11:26:39.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.591+0000 7f7a6988a640 1 -- 172.21.15.84:0/4250249372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6415e1a0 con 0x7f7a640c2320 2023-10-16T11:26:39.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.591+0000 7f7a6988a640 1 -- 172.21.15.84:0/4250249372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6415e760 con 0x7f7a640c2320 2023-10-16T11:26:39.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.592+0000 7f7a6988a640 1 -- 172.21.15.84:0/4250249372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6404e9b0 con 0x7f7a640c2320 2023-10-16T11:26:39.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.593+0000 7f7a617fa640 1 -- 172.21.15.84:0/4250249372 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f7a5c005450 con 0x7f7a640c2320 2023-10-16T11:26:39.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.594+0000 7f7a617fa640 1 --2- 172.21.15.84:0/4250249372 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7a5006b050 0x7f7a5006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:39.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.594+0000 7f7a617fa640 1 -- 172.21.15.84:0/4250249372 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f7a5c0890a0 con 0x7f7a640c2320 2023-10-16T11:26:39.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.594+0000 7f7a68888640 1 --2- 172.21.15.84:0/4250249372 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7a5006b050 0x7f7a5006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:39.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.595+0000 7f7a68888640 1 --2- 172.21.15.84:0/4250249372 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7a5006b050 0x7f7a5006d510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7a58034000 tx=0x7f7a58009030 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:39.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.598+0000 7f7a617fa640 1 -- 172.21.15.84:0/4250249372 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7a5c053fc0 con 0x7f7a640c2320 2023-10-16T11:26:39.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:39.803+0000 7f7a6988a640 1 -- 172.21.15.84:0/4250249372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7a640cf730 con 0x7f7a640c2320 2023-10-16T11:26:40.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.416+0000 7f7a617fa640 1 -- 172.21.15.84:0/4250249372 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v254) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7a5c018020 con 0x7f7a640c2320 2023-10-16T11:26:40.419 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-16T11:26:40.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.420+0000 7f7a6988a640 1 -- 172.21.15.84:0/4250249372 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7a5006b050 msgr2=0x7f7a5006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:40.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.420+0000 7f7a6988a640 1 --2- 172.21.15.84:0/4250249372 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7a5006b050 0x7f7a5006d510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7a58034000 tx=0x7f7a58009030 comp rx=0 tx=0).stop 2023-10-16T11:26:40.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.420+0000 7f7a6988a640 1 -- 172.21.15.84:0/4250249372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a640c2320 msgr2=0x7f7a640c27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:40.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.420+0000 7f7a6988a640 1 --2- 172.21.15.84:0/4250249372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a640c2320 0x7f7a640c27a0 secure :-1 s=READY pgs=3201 cs=0 l=1 rev1=1 crypto rx=0x7f7a64071d20 tx=0x7f7a5c00d920 comp rx=0 tx=0).stop 2023-10-16T11:26:40.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.421+0000 7f7a6988a640 1 -- 172.21.15.84:0/4250249372 shutdown_connections 2023-10-16T11:26:40.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.421+0000 7f7a6988a640 1 --2- 172.21.15.84:0/4250249372 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7a5006b050 0x7f7a5006d510 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:40.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.421+0000 7f7a6988a640 1 --2- 172.21.15.84:0/4250249372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a640c2320 0x7f7a640c27a0 unknown :-1 s=CLOSED pgs=3201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:40.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.421+0000 7f7a6988a640 1 --2- 172.21.15.84:0/4250249372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a6415ec80 0x7f7a6415f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:40.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.421+0000 7f7a6988a640 1 --2- 172.21.15.84:0/4250249372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a4c003a30 0x7f7a641555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:40.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.421+0000 7f7a6988a640 1 -- 172.21.15.84:0/4250249372 >> 172.21.15.84:0/4250249372 conn(0x7f7a640ce820 msgr2=0x7f7a6406ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:40.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.421+0000 7f7a6988a640 1 -- 172.21.15.84:0/4250249372 shutdown_connections 2023-10-16T11:26:40.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.421+0000 7f7a6988a640 1 -- 172.21.15.84:0/4250249372 wait complete. 2023-10-16T11:26:40.449 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T11:26:40.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.649+0000 7fa8e4d41640 1 Processor -- start 2023-10-16T11:26:40.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.649+0000 7fa8e4d41640 1 -- start start 2023-10-16T11:26:40.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.651+0000 7fa8e4d41640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8e015ae80 0x7fa8e015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:40.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.651+0000 7fa8e4d41640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa8e00ce390 0x7fa8e0070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:40.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.651+0000 7fa8e4d41640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8e00707e0 con 0x7fa8e015aaa0 2023-10-16T11:26:40.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.651+0000 7fa8e4d41640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e006b7a0 con 0x7fa8e00ce390 2023-10-16T11:26:40.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.651+0000 7fa8e4d41640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e015ef20 con 0x7fa8e015ae80 2023-10-16T11:26:40.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.651+0000 7fa8ded9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa8e00ce390 0x7fa8e0070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:40.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.652+0000 7fa8ddd9b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8e015ae80 0x7fa8e015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:40.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.652+0000 7fa8ded9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa8e00ce390 0x7fa8e0070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:50034/0 (socket says 172.21.15.84:50034) 2023-10-16T11:26:40.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.652+0000 7fa8ded9d640 1 -- 172.21.15.84:0/3795328495 learned_addr learned my addr 172.21.15.84:0/3795328495 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:40.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.652+0000 7fa8ddd9b640 1 -- 172.21.15.84:0/3795328495 >> v2:172.21.15.157:3300/0 conn(0x7fa8e00ce390 msgr2=0x7fa8e0070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:40.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.652+0000 7fa8ddd9b640 1 --2- 172.21.15.84:0/3795328495 >> v2:172.21.15.157:3300/0 conn(0x7fa8e00ce390 0x7fa8e0070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:40.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.652+0000 7fa8ddd9b640 1 -- 172.21.15.84:0/3795328495 >> v1:172.21.15.157:6789/0 conn(0x7fa8e015aaa0 legacy=0x7fa8e00cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:40.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.652+0000 7fa8ddd9b640 1 -- 172.21.15.84:0/3795328495 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e015f200 con 0x7fa8e015ae80 2023-10-16T11:26:40.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.652+0000 7fa8ded9d640 1 --2- 172.21.15.84:0/3795328495 >> v2:172.21.15.157:3300/0 conn(0x7fa8e00ce390 0x7fa8e0070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:26:40.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.654+0000 7fa8ddd9b640 1 --2- 172.21.15.84:0/3795328495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8e015ae80 0x7fa8e015e820 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40097a0 tx=0x7fa8d402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58856411d8c5849a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:40.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.654+0000 7fa8dd59a640 1 -- 172.21.15.84:0/3795328495 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4031780 con 0x7fa8e015ae80 2023-10-16T11:26:40.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.654+0000 7fa8dd59a640 1 -- 172.21.15.84:0/3795328495 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d40318e0 con 0x7fa8e015ae80 2023-10-16T11:26:40.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.654+0000 7fa8dd59a640 1 -- 172.21.15.84:0/3795328495 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4031c70 con 0x7fa8e015ae80 2023-10-16T11:26:40.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.655+0000 7fa8e4d41640 1 -- 172.21.15.84:0/3795328495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8e015ae80 msgr2=0x7fa8e015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:40.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.655+0000 7fa8e4d41640 1 --2- 172.21.15.84:0/3795328495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8e015ae80 0x7fa8e015e820 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40097a0 tx=0x7fa8d402b820 comp rx=0 tx=0).stop 2023-10-16T11:26:40.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.655+0000 7fa8e4d41640 1 -- 172.21.15.84:0/3795328495 shutdown_connections 2023-10-16T11:26:40.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.655+0000 7fa8e4d41640 1 --2- 172.21.15.84:0/3795328495 >> v2:172.21.15.157:3300/0 conn(0x7fa8e00ce390 0x7fa8e0070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:40.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.655+0000 7fa8e4d41640 1 --2- 172.21.15.84:0/3795328495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8e015ae80 0x7fa8e015e820 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:40.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.655+0000 7fa8e4d41640 1 -- 172.21.15.84:0/3795328495 >> 172.21.15.84:0/3795328495 conn(0x7fa8e00cb410 msgr2=0x7fa8e00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:40.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.655+0000 7fa8e4d41640 1 -- 172.21.15.84:0/3795328495 shutdown_connections 2023-10-16T11:26:40.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.655+0000 7fa8e4d41640 1 -- 172.21.15.84:0/3795328495 wait complete. 2023-10-16T11:26:40.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.657+0000 7fa8e4d41640 1 Processor -- start 2023-10-16T11:26:40.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.657+0000 7fa8e4d41640 1 -- start start 2023-10-16T11:26:40.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.658+0000 7fa8e4d41640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8e00ce390 0x7fa8e00c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:40.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.658+0000 7fa8e4d41640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8e015aaa0 0x7fa8e0157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:40.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.658+0000 7fa8e4d41640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8e015ae80 0x7fa8e0155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:40.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.658+0000 7fa8e4d41640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e00c3980 con 0x7fa8e015ae80 2023-10-16T11:26:40.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.658+0000 7fa8e4d41640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e00c8900 con 0x7fa8e015aaa0 2023-10-16T11:26:40.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.658+0000 7fa8e4d41640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e00c8470 con 0x7fa8e00ce390 2023-10-16T11:26:40.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.658+0000 7fa8ded9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8e015ae80 0x7fa8e0155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:40.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.658+0000 7fa8ded9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8e015ae80 0x7fa8e0155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51972/0 (socket says 172.21.15.84:51972) 2023-10-16T11:26:40.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.658+0000 7fa8ded9d640 1 -- 172.21.15.84:0/548946177 learned_addr learned my addr 172.21.15.84:0/548946177 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:40.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.659+0000 7fa8ddd9b640 1 --2- 172.21.15.84:0/548946177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8e015aaa0 0x7fa8e0157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:40.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.659+0000 7fa8de59c640 1 --2- 172.21.15.84:0/548946177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8e00ce390 0x7fa8e00c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:40.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.659+0000 7fa8ded9d640 1 -- 172.21.15.84:0/548946177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8e00ce390 msgr2=0x7fa8e00c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:40.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.659+0000 7fa8ded9d640 1 --2- 172.21.15.84:0/548946177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8e00ce390 0x7fa8e00c6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:40.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.659+0000 7fa8ded9d640 1 -- 172.21.15.84:0/548946177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8e015aaa0 msgr2=0x7fa8e0157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:40.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.659+0000 7fa8ded9d640 1 --2- 172.21.15.84:0/548946177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8e015aaa0 0x7fa8e0157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:40.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.659+0000 7fa8ded9d640 1 -- 172.21.15.84:0/548946177 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d8009b30 con 0x7fa8e015ae80 2023-10-16T11:26:40.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.659+0000 7fa8ded9d640 1 --2- 172.21.15.84:0/548946177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8e015ae80 0x7fa8e0155a70 secure :-1 s=READY pgs=3204 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009f10 tx=0x7fa8d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:40.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.659+0000 7fa8ceffd640 1 -- 172.21.15.84:0/548946177 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8018070 con 0x7fa8e015ae80 2023-10-16T11:26:40.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.660+0000 7fa8ceffd640 1 -- 172.21.15.84:0/548946177 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d800f040 con 0x7fa8e015ae80 2023-10-16T11:26:40.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.660+0000 7fa8e4d41640 1 -- 172.21.15.84:0/548946177 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d4009400 con 0x7fa8e015ae80 2023-10-16T11:26:40.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.660+0000 7fa8ceffd640 1 -- 172.21.15.84:0/548946177 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d800ce60 con 0x7fa8e015ae80 2023-10-16T11:26:40.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.660+0000 7fa8e4d41640 1 -- 172.21.15.84:0/548946177 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e00c7e20 con 0x7fa8e015ae80 2023-10-16T11:26:40.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.661+0000 7fa8e4d41640 1 -- 172.21.15.84:0/548946177 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e00c8600 con 0x7fa8e015ae80 2023-10-16T11:26:40.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.662+0000 7fa8ceffd640 1 -- 172.21.15.84:0/548946177 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fa8d8014480 con 0x7fa8e015ae80 2023-10-16T11:26:40.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.663+0000 7fa8ceffd640 1 --2- 172.21.15.84:0/548946177 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fa8c006b020 0x7fa8c006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:40.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.663+0000 7fa8ceffd640 1 -- 172.21.15.84:0/548946177 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 5298+0+0 (secure 0 0 0) 0x7fa8d8089cc0 con 0x7fa8e015ae80 2023-10-16T11:26:40.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.663+0000 7fa8de59c640 1 --2- 172.21.15.84:0/548946177 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fa8c006b020 0x7fa8c006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:40.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.664+0000 7fa8de59c640 1 --2- 172.21.15.84:0/548946177 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fa8c006b020 0x7fa8c006d4e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0000f60 tx=0x7fa8d0009040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:40.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.668+0000 7fa8ceffd640 1 -- 172.21.15.84:0/548946177 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa8d8090080 con 0x7fa8e015ae80 2023-10-16T11:26:40.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:40.861+0000 7fa8e4d41640 1 -- 172.21.15.84:0/548946177 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa8e015f4b0 con 0x7fa8e015ae80 2023-10-16T11:26:41.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.435+0000 7fa8ceffd640 1 -- 172.21.15.84:0/548946177 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v255) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa8d8054ae0 con 0x7fa8e015ae80 2023-10-16T11:26:41.437 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-16T11:26:41.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.439+0000 7fa8e4d41640 1 -- 172.21.15.84:0/548946177 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fa8c006b020 msgr2=0x7fa8c006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:41.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.439+0000 7fa8e4d41640 1 --2- 172.21.15.84:0/548946177 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fa8c006b020 0x7fa8c006d4e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0000f60 tx=0x7fa8d0009040 comp rx=0 tx=0).stop 2023-10-16T11:26:41.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.439+0000 7fa8e4d41640 1 -- 172.21.15.84:0/548946177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8e015ae80 msgr2=0x7fa8e0155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:41.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.439+0000 7fa8e4d41640 1 --2- 172.21.15.84:0/548946177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8e015ae80 0x7fa8e0155a70 secure :-1 s=READY pgs=3204 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009f10 tx=0x7fa8d800c490 comp rx=0 tx=0).stop 2023-10-16T11:26:41.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.439+0000 7fa8e4d41640 1 -- 172.21.15.84:0/548946177 shutdown_connections 2023-10-16T11:26:41.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.439+0000 7fa8e4d41640 1 --2- 172.21.15.84:0/548946177 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fa8c006b020 0x7fa8c006d4e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:41.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.439+0000 7fa8e4d41640 1 --2- 172.21.15.84:0/548946177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8e015ae80 0x7fa8e0155a70 unknown :-1 s=CLOSED pgs=3204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:41.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.439+0000 7fa8e4d41640 1 --2- 172.21.15.84:0/548946177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8e015aaa0 0x7fa8e0157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:41.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.439+0000 7fa8e4d41640 1 --2- 172.21.15.84:0/548946177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8e00ce390 0x7fa8e00c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:41.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.439+0000 7fa8e4d41640 1 -- 172.21.15.84:0/548946177 >> 172.21.15.84:0/548946177 conn(0x7fa8e00cb410 msgr2=0x7fa8e00b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:41.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.440+0000 7fa8e4d41640 1 -- 172.21.15.84:0/548946177 shutdown_connections 2023-10-16T11:26:41.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.440+0000 7fa8e4d41640 1 -- 172.21.15.84:0/548946177 wait complete. 2023-10-16T11:26:41.472 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T11:26:41.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.656+0000 7f9ed62bb640 1 Processor -- start 2023-10-16T11:26:41.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.656+0000 7f9ed62bb640 1 -- start start 2023-10-16T11:26:41.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.657+0000 7f9ed62bb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9ed006c040 0x7f9ed006c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:41.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.657+0000 7f9ed62bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ed00cda80 0x7f9ed015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:41.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.657+0000 7f9ed62bb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed006c960 con 0x7f9ed00ad970 2023-10-16T11:26:41.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.657+0000 7f9ed62bb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed015b9d0 con 0x7f9ed006c040 2023-10-16T11:26:41.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.657+0000 7f9ed62bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed015bcb0 con 0x7f9ed00cda80 2023-10-16T11:26:41.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.658+0000 7f9ed52b9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9ed006c040 0x7f9ed006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:41.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.658+0000 7f9ed52b9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9ed006c040 0x7f9ed006c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51982/0 (socket says 172.21.15.84:51982) 2023-10-16T11:26:41.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.658+0000 7f9ed52b9640 1 -- 172.21.15.84:0/3832211700 learned_addr learned my addr 172.21.15.84:0/3832211700 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:41.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.658+0000 7f9ed52b9640 1 -- 172.21.15.84:0/3832211700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ed00cda80 msgr2=0x7f9ed015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:41.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.658+0000 7f9ed52b9640 1 --2- 172.21.15.84:0/3832211700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ed00cda80 0x7f9ed015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:41.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.658+0000 7f9ed52b9640 1 -- 172.21.15.84:0/3832211700 >> v1:172.21.15.157:6789/0 conn(0x7f9ed00ad970 legacy=0x7f9ed015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:41.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.658+0000 7f9ed52b9640 1 -- 172.21.15.84:0/3832211700 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed015bf90 con 0x7f9ed006c040 2023-10-16T11:26:41.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.659+0000 7f9ed52b9640 1 --2- 172.21.15.84:0/3832211700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed006c040 0x7f9ed006c420 secure :-1 s=READY pgs=3205 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009870 tx=0x7f9ecc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5665f69126dbabc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:41.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.660+0000 7f9ec7fff640 1 -- 172.21.15.84:0/3832211700 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc031640 con 0x7f9ed006c040 2023-10-16T11:26:41.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.660+0000 7f9ec7fff640 1 -- 172.21.15.84:0/3832211700 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ecc0317a0 con 0x7f9ed006c040 2023-10-16T11:26:41.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.660+0000 7f9ec7fff640 1 -- 172.21.15.84:0/3832211700 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc031b30 con 0x7f9ed006c040 2023-10-16T11:26:41.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.660+0000 7f9ed62bb640 1 -- 172.21.15.84:0/3832211700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed006c040 msgr2=0x7f9ed006c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:41.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.660+0000 7f9ed62bb640 1 --2- 172.21.15.84:0/3832211700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed006c040 0x7f9ed006c420 secure :-1 s=READY pgs=3205 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009870 tx=0x7f9ecc02b8f0 comp rx=0 tx=0).stop 2023-10-16T11:26:41.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.660+0000 7f9ed62bb640 1 -- 172.21.15.84:0/3832211700 shutdown_connections 2023-10-16T11:26:41.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.660+0000 7f9ed62bb640 1 --2- 172.21.15.84:0/3832211700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ed00cda80 0x7f9ed015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:41.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.660+0000 7f9ed62bb640 1 --2- 172.21.15.84:0/3832211700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed006c040 0x7f9ed006c420 unknown :-1 s=CLOSED pgs=3205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:41.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.660+0000 7f9ed62bb640 1 -- 172.21.15.84:0/3832211700 >> 172.21.15.84:0/3832211700 conn(0x7f9ed00ce6a0 msgr2=0x7f9ed00ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:41.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.660+0000 7f9ed62bb640 1 -- 172.21.15.84:0/3832211700 shutdown_connections 2023-10-16T11:26:41.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.660+0000 7f9ed62bb640 1 -- 172.21.15.84:0/3832211700 wait complete. 2023-10-16T11:26:41.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.662+0000 7f9ed62bb640 1 Processor -- start 2023-10-16T11:26:41.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.662+0000 7f9ed62bb640 1 -- start start 2023-10-16T11:26:41.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.663+0000 7f9ed62bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ed006c040 0x7f9ed00c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:41.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.663+0000 7f9ed62bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ed00ad970 0x7f9ed0156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:41.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.663+0000 7f9ed62bb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed00cda80 0x7f9ed00af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:41.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.663+0000 7f9ed62bb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed00c2520 con 0x7f9ed00cda80 2023-10-16T11:26:41.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.663+0000 7f9ed62bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed00c6ee0 con 0x7f9ed006c040 2023-10-16T11:26:41.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.663+0000 7f9ed62bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed00c71c0 con 0x7f9ed00ad970 2023-10-16T11:26:41.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.664+0000 7f9ed5aba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed00cda80 0x7f9ed00af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:41.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.664+0000 7f9ed5aba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed00cda80 0x7f9ed00af1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51994/0 (socket says 172.21.15.84:51994) 2023-10-16T11:26:41.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.664+0000 7f9ed5aba640 1 -- 172.21.15.84:0/495665316 learned_addr learned my addr 172.21.15.84:0/495665316 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:41.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.664+0000 7f9ed52b9640 1 --2- 172.21.15.84:0/495665316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ed006c040 0x7f9ed00c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:41.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.664+0000 7f9ed5aba640 1 -- 172.21.15.84:0/495665316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ed00ad970 msgr2=0x7f9ed0156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:41.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.664+0000 7f9ed5aba640 1 --2- 172.21.15.84:0/495665316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ed00ad970 0x7f9ed0156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:41.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.664+0000 7f9ed5aba640 1 -- 172.21.15.84:0/495665316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ed006c040 msgr2=0x7f9ed00c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:41.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.664+0000 7f9ed5aba640 1 --2- 172.21.15.84:0/495665316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ed006c040 0x7f9ed00c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:41.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.664+0000 7f9ed5aba640 1 -- 172.21.15.84:0/495665316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc0094d0 con 0x7f9ed00cda80 2023-10-16T11:26:41.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.664+0000 7f9ed5aba640 1 --2- 172.21.15.84:0/495665316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed00cda80 0x7f9ed00af1b0 secure :-1 s=READY pgs=3206 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800b970 tx=0x7f9ec800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:41.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.665+0000 7f9ec5ffb640 1 -- 172.21.15.84:0/495665316 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec800cb40 con 0x7f9ed00cda80 2023-10-16T11:26:41.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.665+0000 7f9ec5ffb640 1 -- 172.21.15.84:0/495665316 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec800ccc0 con 0x7f9ed00cda80 2023-10-16T11:26:41.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.665+0000 7f9ec5ffb640 1 -- 172.21.15.84:0/495665316 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8014680 con 0x7f9ed00cda80 2023-10-16T11:26:41.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.665+0000 7f9ed62bb640 1 -- 172.21.15.84:0/495665316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed00c74a0 con 0x7f9ed00cda80 2023-10-16T11:26:41.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.665+0000 7f9ed62bb640 1 -- 172.21.15.84:0/495665316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed00c6ba0 con 0x7f9ed00cda80 2023-10-16T11:26:41.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.666+0000 7f9ed62bb640 1 -- 172.21.15.84:0/495665316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed00c2690 con 0x7f9ed00cda80 2023-10-16T11:26:41.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.667+0000 7f9ec5ffb640 1 -- 172.21.15.84:0/495665316 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f9ec801d3b0 con 0x7f9ed00cda80 2023-10-16T11:26:41.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.668+0000 7f9ec5ffb640 1 --2- 172.21.15.84:0/495665316 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f9eb406b020 0x7f9eb406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:41.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.668+0000 7f9ec5ffb640 1 -- 172.21.15.84:0/495665316 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 5680+0+0 (secure 0 0 0) 0x7f9ec8089cf0 con 0x7f9ed00cda80 2023-10-16T11:26:41.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.669+0000 7f9ed52b9640 1 --2- 172.21.15.84:0/495665316 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f9eb406b020 0x7f9eb406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:41.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.669+0000 7f9ed52b9640 1 --2- 172.21.15.84:0/495665316 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f9eb406b020 0x7f9eb406d4e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc036000 tx=0x7f9ecc03c040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:41.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.672+0000 7f9ec5ffb640 1 -- 172.21.15.84:0/495665316 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9ec8054b40 con 0x7f9ed00cda80 2023-10-16T11:26:41.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:41.905+0000 7f9ed62bb640 1 -- 172.21.15.84:0/495665316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9ed004e9b0 con 0x7f9ed00cda80 2023-10-16T11:26:42.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.467+0000 7f9ec5ffb640 1 -- 172.21.15.84:0/495665316 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 34 and data pool 35 v144) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9ec8019020 con 0x7f9ed00cda80 2023-10-16T11:26:42.472 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 34 and data pool 35 2023-10-16T11:26:42.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.475+0000 7f9ed62bb640 1 -- 172.21.15.84:0/495665316 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f9eb406b020 msgr2=0x7f9eb406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:42.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.475+0000 7f9ed62bb640 1 --2- 172.21.15.84:0/495665316 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f9eb406b020 0x7f9eb406d4e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc036000 tx=0x7f9ecc03c040 comp rx=0 tx=0).stop 2023-10-16T11:26:42.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.475+0000 7f9ed62bb640 1 -- 172.21.15.84:0/495665316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed00cda80 msgr2=0x7f9ed00af1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:42.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.475+0000 7f9ed62bb640 1 --2- 172.21.15.84:0/495665316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed00cda80 0x7f9ed00af1b0 secure :-1 s=READY pgs=3206 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800b970 tx=0x7f9ec800be30 comp rx=0 tx=0).stop 2023-10-16T11:26:42.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.477+0000 7f9ed62bb640 1 -- 172.21.15.84:0/495665316 shutdown_connections 2023-10-16T11:26:42.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.477+0000 7f9ed62bb640 1 --2- 172.21.15.84:0/495665316 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f9eb406b020 0x7f9eb406d4e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:42.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.477+0000 7f9ed62bb640 1 --2- 172.21.15.84:0/495665316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed00cda80 0x7f9ed00af1b0 unknown :-1 s=CLOSED pgs=3206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:42.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.477+0000 7f9ed62bb640 1 --2- 172.21.15.84:0/495665316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ed00ad970 0x7f9ed0156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:42.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.477+0000 7f9ed62bb640 1 --2- 172.21.15.84:0/495665316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ed006c040 0x7f9ed00c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:42.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.477+0000 7f9ed62bb640 1 -- 172.21.15.84:0/495665316 >> 172.21.15.84:0/495665316 conn(0x7f9ed00ce6a0 msgr2=0x7f9ed00ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:42.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.481+0000 7f9ed62bb640 1 -- 172.21.15.84:0/495665316 shutdown_connections 2023-10-16T11:26:42.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.481+0000 7f9ed62bb640 1 -- 172.21.15.84:0/495665316 wait complete. 2023-10-16T11:26:42.508 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:26:42.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.685+0000 7f18e614e640 1 Processor -- start 2023-10-16T11:26:42.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.685+0000 7f18e614e640 1 -- start start 2023-10-16T11:26:42.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.687+0000 7f18e614e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18e006c040 0x7f18e006c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:42.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.687+0000 7f18e614e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18e00cda80 0x7f18e015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:42.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.687+0000 7f18e614e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e006c960 con 0x7f18e00ad970 2023-10-16T11:26:42.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.687+0000 7f18e614e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e015b9d0 con 0x7f18e00cda80 2023-10-16T11:26:42.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.687+0000 7f18e614e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e015bcb0 con 0x7f18e006c040 2023-10-16T11:26:42.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.687+0000 7f18e514c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18e006c040 0x7f18e006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:42.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.687+0000 7f18e594d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18e00cda80 0x7f18e015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:42.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.687+0000 7f18e514c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18e006c040 0x7f18e006c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:60830/0 (socket says 172.21.15.84:60830) 2023-10-16T11:26:42.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.688+0000 7f18e514c640 1 -- 172.21.15.84:0/1912993747 learned_addr learned my addr 172.21.15.84:0/1912993747 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:42.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.688+0000 7f18e514c640 1 -- 172.21.15.84:0/1912993747 >> v2:172.21.15.157:3300/0 conn(0x7f18e00cda80 msgr2=0x7f18e015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:42.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.688+0000 7f18e514c640 1 --2- 172.21.15.84:0/1912993747 >> v2:172.21.15.157:3300/0 conn(0x7f18e00cda80 0x7f18e015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:42.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.688+0000 7f18e514c640 1 -- 172.21.15.84:0/1912993747 >> v1:172.21.15.157:6789/0 conn(0x7f18e00ad970 legacy=0x7f18e015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:42.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.688+0000 7f18e514c640 1 -- 172.21.15.84:0/1912993747 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e015bf90 con 0x7f18e006c040 2023-10-16T11:26:42.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.690+0000 7f18e514c640 1 --2- 172.21.15.84:0/1912993747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18e006c040 0x7f18e006c420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009870 tx=0x7f18dc02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7c5a72743b0a08 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:42.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.690+0000 7f18d7fff640 1 -- 172.21.15.84:0/1912993747 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc031640 con 0x7f18e006c040 2023-10-16T11:26:42.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.690+0000 7f18d7fff640 1 -- 172.21.15.84:0/1912993747 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc0317a0 con 0x7f18e006c040 2023-10-16T11:26:42.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.690+0000 7f18d7fff640 1 -- 172.21.15.84:0/1912993747 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc031b30 con 0x7f18e006c040 2023-10-16T11:26:42.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.691+0000 7f18e614e640 1 -- 172.21.15.84:0/1912993747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18e006c040 msgr2=0x7f18e006c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:42.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.691+0000 7f18e614e640 1 --2- 172.21.15.84:0/1912993747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18e006c040 0x7f18e006c420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009870 tx=0x7f18dc02b8f0 comp rx=0 tx=0).stop 2023-10-16T11:26:42.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.691+0000 7f18e614e640 1 -- 172.21.15.84:0/1912993747 shutdown_connections 2023-10-16T11:26:42.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.691+0000 7f18e614e640 1 --2- 172.21.15.84:0/1912993747 >> v2:172.21.15.157:3300/0 conn(0x7f18e00cda80 0x7f18e015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:42.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.691+0000 7f18e614e640 1 --2- 172.21.15.84:0/1912993747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18e006c040 0x7f18e006c420 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:42.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.691+0000 7f18e614e640 1 -- 172.21.15.84:0/1912993747 >> 172.21.15.84:0/1912993747 conn(0x7f18e00ce6a0 msgr2=0x7f18e00ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:42.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.691+0000 7f18e614e640 1 -- 172.21.15.84:0/1912993747 shutdown_connections 2023-10-16T11:26:42.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.691+0000 7f18e614e640 1 -- 172.21.15.84:0/1912993747 wait complete. 2023-10-16T11:26:42.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.693+0000 7f18e614e640 1 Processor -- start 2023-10-16T11:26:42.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.693+0000 7f18e614e640 1 -- start start 2023-10-16T11:26:42.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.695+0000 7f18e614e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18e006c040 0x7f18e01f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:42.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.695+0000 7f18e614e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18e00ad970 0x7f18e00c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:42.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.695+0000 7f18e614e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18e00cda80 0x7f18e00c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:42.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.695+0000 7f18e614e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e00c71f0 con 0x7f18e00ad970 2023-10-16T11:26:42.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.695+0000 7f18e614e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e00c74d0 con 0x7f18e006c040 2023-10-16T11:26:42.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.695+0000 7f18e614e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e00ae160 con 0x7f18e00cda80 2023-10-16T11:26:42.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.695+0000 7f18e494b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18e00ad970 0x7f18e00c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:42.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.695+0000 7f18e494b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18e00ad970 0x7f18e00c67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52010/0 (socket says 172.21.15.84:52010) 2023-10-16T11:26:42.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.695+0000 7f18e494b640 1 -- 172.21.15.84:0/3832046328 learned_addr learned my addr 172.21.15.84:0/3832046328 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:42.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.695+0000 7f18e594d640 1 --2- 172.21.15.84:0/3832046328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18e00cda80 0x7f18e00c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:42.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.695+0000 7f18e514c640 1 --2- 172.21.15.84:0/3832046328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18e006c040 0x7f18e01f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:42.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.695+0000 7f18e494b640 1 -- 172.21.15.84:0/3832046328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18e00cda80 msgr2=0x7f18e00c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:42.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.695+0000 7f18e494b640 1 --2- 172.21.15.84:0/3832046328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18e00cda80 0x7f18e00c7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:42.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.695+0000 7f18e494b640 1 -- 172.21.15.84:0/3832046328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18e006c040 msgr2=0x7f18e01f21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:42.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.695+0000 7f18e494b640 1 --2- 172.21.15.84:0/3832046328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18e006c040 0x7f18e01f21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:42.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.695+0000 7f18e494b640 1 -- 172.21.15.84:0/3832046328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc0094d0 con 0x7f18e00ad970 2023-10-16T11:26:42.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.696+0000 7f18e494b640 1 --2- 172.21.15.84:0/3832046328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18e00ad970 0x7f18e00c67c0 secure :-1 s=READY pgs=3207 cs=0 l=1 rev1=1 crypto rx=0x7f18d000c430 tx=0x7f18d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:42.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.696+0000 7f18d5ffb640 1 -- 172.21.15.84:0/3832046328 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d0011790 con 0x7f18e00ad970 2023-10-16T11:26:42.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.696+0000 7f18d5ffb640 1 -- 172.21.15.84:0/3832046328 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d00118f0 con 0x7f18e00ad970 2023-10-16T11:26:42.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.696+0000 7f18d5ffb640 1 -- 172.21.15.84:0/3832046328 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d0011ce0 con 0x7f18e00ad970 2023-10-16T11:26:42.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.696+0000 7f18e614e640 1 -- 172.21.15.84:0/3832046328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e00ae440 con 0x7f18e00ad970 2023-10-16T11:26:42.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.696+0000 7f18e614e640 1 -- 172.21.15.84:0/3832046328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e01f2720 con 0x7f18e00ad970 2023-10-16T11:26:42.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.697+0000 7f18e614e640 1 -- 172.21.15.84:0/3832046328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e006b330 con 0x7f18e00ad970 2023-10-16T11:26:42.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.698+0000 7f18d5ffb640 1 -- 172.21.15.84:0/3832046328 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f18d001c460 con 0x7f18e00ad970 2023-10-16T11:26:42.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.699+0000 7f18d5ffb640 1 --2- 172.21.15.84:0/3832046328 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f18c406b020 0x7f18c406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:42.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.700+0000 7f18d5ffb640 1 -- 172.21.15.84:0/3832046328 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5780+0+0 (secure 0 0 0) 0x7f18d0089970 con 0x7f18e00ad970 2023-10-16T11:26:42.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.700+0000 7f18e514c640 1 --2- 172.21.15.84:0/3832046328 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f18c406b020 0x7f18c406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:42.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.700+0000 7f18e514c640 1 --2- 172.21.15.84:0/3832046328 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f18c406b020 0x7f18c406d4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f18dc036000 tx=0x7f18dc03c040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:42.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.704+0000 7f18d5ffb640 1 -- 172.21.15.84:0/3832046328 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f18d00546b0 con 0x7f18e00ad970 2023-10-16T11:26:42.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.896+0000 7f18e614e640 1 -- 172.21.15.84:0/3832046328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18e00ae630 con 0x7f18e00ad970 2023-10-16T11:26:42.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.898+0000 7f18d5ffb640 1 -- 172.21.15.84:0/3832046328 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v256) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f18d0057cd0 con 0x7f18e00ad970 2023-10-16T11:26:42.901 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:42.901 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":256,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:26:41.907374+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:26:39.860053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-16T11:26:40.916519+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/570042115":"2023-10-16T12:26:40.168744+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:26:42.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.901+0000 7f18e614e640 1 -- 172.21.15.84:0/3832046328 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f18c406b020 msgr2=0x7f18c406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:42.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.901+0000 7f18e614e640 1 --2- 172.21.15.84:0/3832046328 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f18c406b020 0x7f18c406d4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f18dc036000 tx=0x7f18dc03c040 comp rx=0 tx=0).stop 2023-10-16T11:26:42.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.901+0000 7f18e614e640 1 -- 172.21.15.84:0/3832046328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18e00ad970 msgr2=0x7f18e00c67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:42.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.901+0000 7f18e614e640 1 --2- 172.21.15.84:0/3832046328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18e00ad970 0x7f18e00c67c0 secure :-1 s=READY pgs=3207 cs=0 l=1 rev1=1 crypto rx=0x7f18d000c430 tx=0x7f18d000c8f0 comp rx=0 tx=0).stop 2023-10-16T11:26:42.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.901+0000 7f18e614e640 1 -- 172.21.15.84:0/3832046328 shutdown_connections 2023-10-16T11:26:42.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.901+0000 7f18e614e640 1 --2- 172.21.15.84:0/3832046328 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f18c406b020 0x7f18c406d4e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:42.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.901+0000 7f18e614e640 1 --2- 172.21.15.84:0/3832046328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18e00cda80 0x7f18e00c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:42.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.901+0000 7f18e614e640 1 --2- 172.21.15.84:0/3832046328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18e00ad970 0x7f18e00c67c0 unknown :-1 s=CLOSED pgs=3207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:42.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.901+0000 7f18e614e640 1 --2- 172.21.15.84:0/3832046328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18e006c040 0x7f18e01f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:42.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.901+0000 7f18e614e640 1 -- 172.21.15.84:0/3832046328 >> 172.21.15.84:0/3832046328 conn(0x7f18e00ce6a0 msgr2=0x7f18e00c8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:42.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.901+0000 7f18e614e640 1 -- 172.21.15.84:0/3832046328 shutdown_connections 2023-10-16T11:26:42.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:42.901+0000 7f18e614e640 1 -- 172.21.15.84:0/3832046328 wait complete. 2023-10-16T11:26:42.922 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:26:43.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.128+0000 7f6394075640 1 Processor -- start 2023-10-16T11:26:43.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.128+0000 7f6394075640 1 -- start start 2023-10-16T11:26:43.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.130+0000 7f6394075640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f638c15aaa0 0x7f638c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:43.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.130+0000 7f6394075640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f638c15ae80 0x7f638c0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:43.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.130+0000 7f6394075640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f638c0ccaa0 0x7f638c070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:43.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.130+0000 7f6394075640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f638c06b810 con 0x7f638c15aaa0 2023-10-16T11:26:43.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.130+0000 7f6394075640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f638c15ed60 con 0x7f638c0ccaa0 2023-10-16T11:26:43.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.130+0000 7f6394075640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638c0707e0 con 0x7f638c15ae80 2023-10-16T11:26:43.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.130+0000 7f6391e11640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f638c15aaa0 0x7f638c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:43.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.130+0000 7f6391e11640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f638c15aaa0 0x7f638c15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52016/0 (socket says 172.21.15.84:52016) 2023-10-16T11:26:43.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.130+0000 7f6391e11640 1 -- 172.21.15.84:0/2924933877 learned_addr learned my addr 172.21.15.84:0/2924933877 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:43.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.131+0000 7f6391610640 1 --2- 172.21.15.84:0/2924933877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f638c15ae80 0x7f638c0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:43.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.131+0000 7f6392612640 1 --2- 172.21.15.84:0/2924933877 >> v2:172.21.15.157:3300/0 conn(0x7f638c0ccaa0 0x7f638c070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:43.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.131+0000 7f6391e11640 1 -- 172.21.15.84:0/2924933877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f638c15ae80 msgr2=0x7f638c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:43.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.131+0000 7f6391e11640 1 --2- 172.21.15.84:0/2924933877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f638c15ae80 0x7f638c0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.131+0000 7f6391e11640 1 -- 172.21.15.84:0/2924933877 >> v2:172.21.15.157:3300/0 conn(0x7f638c0ccaa0 msgr2=0x7f638c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:43.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.131+0000 7f6391e11640 1 --2- 172.21.15.84:0/2924933877 >> v2:172.21.15.157:3300/0 conn(0x7f638c0ccaa0 0x7f638c070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.131+0000 7f6391e11640 1 -- 172.21.15.84:0/2924933877 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c15ef90 con 0x7f638c15aaa0 2023-10-16T11:26:43.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.131+0000 7f6392612640 1 --2- 172.21.15.84:0/2924933877 >> v2:172.21.15.157:3300/0 conn(0x7f638c0ccaa0 0x7f638c070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:43.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.132+0000 7f6391e11640 1 --2- 172.21.15.84:0/2924933877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f638c15aaa0 0x7f638c15e820 secure :-1 s=READY pgs=3208 cs=0 l=1 rev1=1 crypto rx=0x7f638c0ae3c0 tx=0x7f638802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38c49fbd06c6e45 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:43.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.133+0000 7f6390e0f640 1 -- 172.21.15.84:0/2924933877 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388031860 con 0x7f638c15aaa0 2023-10-16T11:26:43.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.133+0000 7f6390e0f640 1 -- 172.21.15.84:0/2924933877 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63880319c0 con 0x7f638c15aaa0 2023-10-16T11:26:43.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.133+0000 7f6390e0f640 1 -- 172.21.15.84:0/2924933877 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388031d50 con 0x7f638c15aaa0 2023-10-16T11:26:43.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.133+0000 7f6394075640 1 -- 172.21.15.84:0/2924933877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f638c15aaa0 msgr2=0x7f638c15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:43.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.133+0000 7f6394075640 1 --2- 172.21.15.84:0/2924933877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f638c15aaa0 0x7f638c15e820 secure :-1 s=READY pgs=3208 cs=0 l=1 rev1=1 crypto rx=0x7f638c0ae3c0 tx=0x7f638802b900 comp rx=0 tx=0).stop 2023-10-16T11:26:43.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.133+0000 7f6394075640 1 -- 172.21.15.84:0/2924933877 shutdown_connections 2023-10-16T11:26:43.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.133+0000 7f6394075640 1 --2- 172.21.15.84:0/2924933877 >> v2:172.21.15.157:3300/0 conn(0x7f638c0ccaa0 0x7f638c070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.133+0000 7f6394075640 1 --2- 172.21.15.84:0/2924933877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f638c15ae80 0x7f638c0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.133+0000 7f6394075640 1 --2- 172.21.15.84:0/2924933877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f638c15aaa0 0x7f638c15e820 unknown :-1 s=CLOSED pgs=3208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.133+0000 7f6394075640 1 -- 172.21.15.84:0/2924933877 >> 172.21.15.84:0/2924933877 conn(0x7f638c0cb410 msgr2=0x7f638c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:43.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.133+0000 7f6394075640 1 -- 172.21.15.84:0/2924933877 shutdown_connections 2023-10-16T11:26:43.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.134+0000 7f6394075640 1 -- 172.21.15.84:0/2924933877 wait complete. 2023-10-16T11:26:43.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.135+0000 7f6394075640 1 Processor -- start 2023-10-16T11:26:43.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.135+0000 7f6394075640 1 -- start start 2023-10-16T11:26:43.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.136+0000 7f6394075640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f638c0ccaa0 0x7f638c1f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:43.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.136+0000 7f6394075640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f638c15aaa0 0x7f638c1f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:43.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.136+0000 7f6394075640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f638c15ae80 0x7f638c1f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:43.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.136+0000 7f6394075640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c0c4f10 con 0x7f638c0ccaa0 2023-10-16T11:26:43.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.136+0000 7f6394075640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c15b4a0 con 0x7f638c15ae80 2023-10-16T11:26:43.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.136+0000 7f6394075640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638c1f2330 con 0x7f638c15aaa0 2023-10-16T11:26:43.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.136+0000 7f6391e11640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f638c0ccaa0 0x7f638c1f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:43.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.137+0000 7f6391e11640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f638c0ccaa0 0x7f638c1f18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52030/0 (socket says 172.21.15.84:52030) 2023-10-16T11:26:43.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.137+0000 7f6391e11640 1 -- 172.21.15.84:0/420204260 learned_addr learned my addr 172.21.15.84:0/420204260 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:43.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.137+0000 7f6391610640 1 --2- 172.21.15.84:0/420204260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f638c15aaa0 0x7f638c1f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:43.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.137+0000 7f6391e11640 1 -- 172.21.15.84:0/420204260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f638c15aaa0 msgr2=0x7f638c1f1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:43.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.137+0000 7f6391e11640 1 --2- 172.21.15.84:0/420204260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f638c15aaa0 0x7f638c1f1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.137+0000 7f6392612640 1 --2- 172.21.15.84:0/420204260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f638c15ae80 0x7f638c1f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:43.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.137+0000 7f6391e11640 1 -- 172.21.15.84:0/420204260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f638c15ae80 msgr2=0x7f638c1f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:43.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.137+0000 7f6391e11640 1 --2- 172.21.15.84:0/420204260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f638c15ae80 0x7f638c1f5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.137+0000 7f6391e11640 1 -- 172.21.15.84:0/420204260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63880094d0 con 0x7f638c0ccaa0 2023-10-16T11:26:43.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.137+0000 7f6391e11640 1 --2- 172.21.15.84:0/420204260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f638c0ccaa0 0x7f638c1f18b0 secure :-1 s=READY pgs=3209 cs=0 l=1 rev1=1 crypto rx=0x7f638802b940 tx=0x7f63880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:43.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.138+0000 7f63827fc640 1 -- 172.21.15.84:0/420204260 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638803ac60 con 0x7f638c0ccaa0 2023-10-16T11:26:43.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.138+0000 7f63827fc640 1 -- 172.21.15.84:0/420204260 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638803adc0 con 0x7f638c0ccaa0 2023-10-16T11:26:43.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.138+0000 7f6394075640 1 -- 172.21.15.84:0/420204260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f638c0c3670 con 0x7f638c0ccaa0 2023-10-16T11:26:43.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.138+0000 7f63827fc640 1 -- 172.21.15.84:0/420204260 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63880445f0 con 0x7f638c0ccaa0 2023-10-16T11:26:43.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.138+0000 7f6394075640 1 -- 172.21.15.84:0/420204260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f638c0c3c30 con 0x7f638c0ccaa0 2023-10-16T11:26:43.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.139+0000 7f6394075640 1 -- 172.21.15.84:0/420204260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638c04e160 con 0x7f638c0ccaa0 2023-10-16T11:26:43.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.140+0000 7f63827fc640 1 -- 172.21.15.84:0/420204260 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 150) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f638804d450 con 0x7f638c0ccaa0 2023-10-16T11:26:43.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.141+0000 7f63827fc640 1 --2- 172.21.15.84:0/420204260 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f637006b020 0x7f637006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:43.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.141+0000 7f63827fc640 1 -- 172.21.15.84:0/420204260 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5780+0+0 (secure 0 0 0) 0x7f63880abf70 con 0x7f638c0ccaa0 2023-10-16T11:26:43.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.141+0000 7f6391610640 1 --2- 172.21.15.84:0/420204260 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f637006b020 0x7f637006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:43.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.142+0000 7f6391610640 1 --2- 172.21.15.84:0/420204260 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f637006b020 0x7f637006d4e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f638c15a1d0 tx=0x7f637c008040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:43.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.145+0000 7f63827fc640 1 -- 172.21.15.84:0/420204260 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f63880defe0 con 0x7f638c0ccaa0 2023-10-16T11:26:43.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.340+0000 7f6394075640 1 -- 172.21.15.84:0/420204260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f638c0cdc60 con 0x7f638c0ccaa0 2023-10-16T11:26:43.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.343+0000 7f63827fc640 1 -- 172.21.15.84:0/420204260 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v256) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f6388035020 con 0x7f638c0ccaa0 2023-10-16T11:26:43.345 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:43.345 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":256,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:26:41.907374+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:26:39.860053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-16T11:26:40.916519+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/570042115":"2023-10-16T12:26:40.168744+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:26:43.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.345+0000 7f6394075640 1 -- 172.21.15.84:0/420204260 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f637006b020 msgr2=0x7f637006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:43.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.345+0000 7f6394075640 1 --2- 172.21.15.84:0/420204260 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f637006b020 0x7f637006d4e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f638c15a1d0 tx=0x7f637c008040 comp rx=0 tx=0).stop 2023-10-16T11:26:43.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.345+0000 7f6394075640 1 -- 172.21.15.84:0/420204260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f638c0ccaa0 msgr2=0x7f638c1f18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:43.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.345+0000 7f6394075640 1 --2- 172.21.15.84:0/420204260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f638c0ccaa0 0x7f638c1f18b0 secure :-1 s=READY pgs=3209 cs=0 l=1 rev1=1 crypto rx=0x7f638802b940 tx=0x7f63880046c0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.346+0000 7f6394075640 1 -- 172.21.15.84:0/420204260 shutdown_connections 2023-10-16T11:26:43.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.346+0000 7f6394075640 1 --2- 172.21.15.84:0/420204260 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f637006b020 0x7f637006d4e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.346+0000 7f6394075640 1 --2- 172.21.15.84:0/420204260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f638c15ae80 0x7f638c1f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.346+0000 7f6394075640 1 --2- 172.21.15.84:0/420204260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f638c15aaa0 0x7f638c1f1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.346+0000 7f6394075640 1 --2- 172.21.15.84:0/420204260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f638c0ccaa0 0x7f638c1f18b0 unknown :-1 s=CLOSED pgs=3209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.346+0000 7f6394075640 1 -- 172.21.15.84:0/420204260 >> 172.21.15.84:0/420204260 conn(0x7f638c0cb410 msgr2=0x7f638c0ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:43.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.346+0000 7f6394075640 1 -- 172.21.15.84:0/420204260 shutdown_connections 2023-10-16T11:26:43.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.346+0000 7f6394075640 1 -- 172.21.15.84:0/420204260 wait complete. 2023-10-16T11:26:43.367 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T11:26:43.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.586+0000 7f70e52e7640 1 Processor -- start 2023-10-16T11:26:43.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.586+0000 7f70e52e7640 1 -- start start 2023-10-16T11:26:43.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.588+0000 7f70e52e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70e015ad70 0x7f70e0069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:43.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.588+0000 7f70e52e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f70e00caac0 0x7f70e006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:43.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.588+0000 7f70e52e7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e00cb030 con 0x7f70e00c9ca0 2023-10-16T11:26:43.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.588+0000 7f70e52e7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e00c91c0 con 0x7f70e00caac0 2023-10-16T11:26:43.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.588+0000 7f70e52e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00c94a0 con 0x7f70e015ad70 2023-10-16T11:26:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.589+0000 7f70d7fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f70e00c9ca0 0x7f70e00ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38804/0 (socket says 172.21.15.84:38804) 2023-10-16T11:26:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.589+0000 7f70d7fff640 1 -- 172.21.15.84:0/2028637955 learned_addr learned my addr 172.21.15.84:0/2028637955 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.589+0000 7f70df7fe640 1 --2- 172.21.15.84:0/2028637955 >> v2:172.21.15.157:3300/0 conn(0x7f70e00caac0 0x7f70e006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.589+0000 7f70deffd640 1 --2- 172.21.15.84:0/2028637955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70e015ad70 0x7f70e0069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:43.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.589+0000 7f70de7fc640 1 -- 172.21.15.84:0/2028637955 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2050178593 0 0) 0x7f70e00cb030 con 0x7f70e00c9ca0 2023-10-16T11:26:43.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.589+0000 7f70de7fc640 1 -- 172.21.15.84:0/2028637955 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70cc003520 con 0x7f70e00c9ca0 2023-10-16T11:26:43.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.590+0000 7f70de7fc640 1 -- 172.21.15.84:0/2028637955 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3363634461 0 0) 0x7f70cc003520 con 0x7f70e00c9ca0 2023-10-16T11:26:43.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.590+0000 7f70de7fc640 1 -- 172.21.15.84:0/2028637955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70e015ad70 msgr2=0x7f70e0069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:43.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.590+0000 7f70de7fc640 1 --2- 172.21.15.84:0/2028637955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70e015ad70 0x7f70e0069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.590+0000 7f70de7fc640 1 -- 172.21.15.84:0/2028637955 >> v2:172.21.15.157:3300/0 conn(0x7f70e00caac0 msgr2=0x7f70e006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:43.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.590+0000 7f70de7fc640 1 --2- 172.21.15.84:0/2028637955 >> v2:172.21.15.157:3300/0 conn(0x7f70e00caac0 0x7f70e006a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.590+0000 7f70de7fc640 1 -- 172.21.15.84:0/2028637955 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e00c9780 con 0x7f70e00c9ca0 2023-10-16T11:26:43.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.590+0000 7f70de7fc640 1 -- 172.21.15.84:0/2028637955 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f70c8001670 con 0x7f70e00c9ca0 2023-10-16T11:26:43.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.590+0000 7f70de7fc640 1 -- 172.21.15.84:0/2028637955 >> v1:172.21.15.84:6789/0 conn(0x7f70e00c9ca0 legacy=0x7f70e00ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:43.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.590+0000 7f70de7fc640 1 --2- 172.21.15.84:0/2028637955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70cc003970 0x7f70cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:43.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.590+0000 7f70de7fc640 1 --2- 172.21.15.84:0/2028637955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70cc0047c0 0x7f70cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:43.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.590+0000 7f70de7fc640 1 --2- 172.21.15.84:0/2028637955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70e015ad70 0x7f70cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:43.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.590+0000 7f70de7fc640 1 -- 172.21.15.84:0/2028637955 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00c91c0 con 0x7f70cc0047c0 2023-10-16T11:26:43.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.590+0000 7f70de7fc640 1 -- 172.21.15.84:0/2028637955 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00c94a0 con 0x7f70cc003970 2023-10-16T11:26:43.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.590+0000 7f70de7fc640 1 -- 172.21.15.84:0/2028637955 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc00b980 con 0x7f70e015ad70 2023-10-16T11:26:43.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.590+0000 7f70d7fff640 1 --2- 172.21.15.84:0/2028637955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70cc0047c0 0x7f70cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:43.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.590+0000 7f70deffd640 1 --2- 172.21.15.84:0/2028637955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70cc003970 0x7f70cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:43.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.591+0000 7f70df7fe640 1 --2- 172.21.15.84:0/2028637955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70e015ad70 0x7f70cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:43.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.591+0000 7f70d7fff640 1 -- 172.21.15.84:0/2028637955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70e015ad70 msgr2=0x7f70cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:43.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.591+0000 7f70d7fff640 1 --2- 172.21.15.84:0/2028637955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70e015ad70 0x7f70cc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.591+0000 7f70d7fff640 1 -- 172.21.15.84:0/2028637955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70cc003970 msgr2=0x7f70cc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:43.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.591+0000 7f70d7fff640 1 --2- 172.21.15.84:0/2028637955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70cc003970 0x7f70cc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.591+0000 7f70d7fff640 1 -- 172.21.15.84:0/2028637955 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc003520 con 0x7f70cc0047c0 2023-10-16T11:26:43.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.592+0000 7f70d7fff640 1 --2- 172.21.15.84:0/2028637955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70cc0047c0 0x7f70cc004c40 secure :-1 s=READY pgs=3211 cs=0 l=1 rev1=1 crypto rx=0x7f70c800c240 tx=0x7f70cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e929db465151f589 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:43.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.592+0000 7f70de7fc640 1 -- 172.21.15.84:0/2028637955 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c802cb00 con 0x7f70cc0047c0 2023-10-16T11:26:43.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.592+0000 7f70de7fc640 1 -- 172.21.15.84:0/2028637955 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c802cc60 con 0x7f70cc0047c0 2023-10-16T11:26:43.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.592+0000 7f70de7fc640 1 -- 172.21.15.84:0/2028637955 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8005af0 con 0x7f70cc0047c0 2023-10-16T11:26:43.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.593+0000 7f70e52e7640 1 -- 172.21.15.84:0/2028637955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70cc0047c0 msgr2=0x7f70cc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:43.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.593+0000 7f70e52e7640 1 --2- 172.21.15.84:0/2028637955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70cc0047c0 0x7f70cc004c40 secure :-1 s=READY pgs=3211 cs=0 l=1 rev1=1 crypto rx=0x7f70c800c240 tx=0x7f70cc0057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.593+0000 7f70df7fe640 1 -- 172.21.15.84:0/2028637955 reap_dead start 2023-10-16T11:26:43.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.593+0000 7f70e52e7640 1 -- 172.21.15.84:0/2028637955 shutdown_connections 2023-10-16T11:26:43.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.593+0000 7f70e52e7640 1 -- 172.21.15.84:0/2028637955 >> 172.21.15.84:0/2028637955 conn(0x7f70e00ce9b0 msgr2=0x7f70e00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:43.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.593+0000 7f70e52e7640 1 -- 172.21.15.84:0/2028637955 shutdown_connections 2023-10-16T11:26:43.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.593+0000 7f70e52e7640 1 -- 172.21.15.84:0/2028637955 wait complete. 2023-10-16T11:26:43.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.595+0000 7f70e52e7640 1 Processor -- start 2023-10-16T11:26:43.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.595+0000 7f70e52e7640 1 -- start start 2023-10-16T11:26:43.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.596+0000 7f70e52e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70cc0047c0 0x7f70e015d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:43.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.596+0000 7f70e52e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70e015be30 0x7f70e015d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:43.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.596+0000 7f70e52e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70e0158aa0 0x7f70e015de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:43.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.596+0000 7f70e52e7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e015cf00 con 0x7f70e015be30 2023-10-16T11:26:43.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.596+0000 7f70e52e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00c8160 con 0x7f70e0158aa0 2023-10-16T11:26:43.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.596+0000 7f70e52e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e015c2f0 con 0x7f70cc0047c0 2023-10-16T11:26:43.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.596+0000 7f70d7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70e015be30 0x7f70e015d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:43.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.596+0000 7f70d7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70e015be30 0x7f70e015d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52056/0 (socket says 172.21.15.84:52056) 2023-10-16T11:26:43.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.596+0000 7f70d7fff640 1 -- 172.21.15.84:0/2873402602 learned_addr learned my addr 172.21.15.84:0/2873402602 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:43.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.597+0000 7f70deffd640 1 --2- 172.21.15.84:0/2873402602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70cc0047c0 0x7f70e015d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:43.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.597+0000 7f70d7fff640 1 -- 172.21.15.84:0/2873402602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70cc0047c0 msgr2=0x7f70e015d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:43.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.597+0000 7f70d7fff640 1 --2- 172.21.15.84:0/2873402602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70cc0047c0 0x7f70e015d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.597+0000 7f70d7fff640 1 -- 172.21.15.84:0/2873402602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70e0158aa0 msgr2=0x7f70e015de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:43.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.597+0000 7f70d7fff640 1 --2- 172.21.15.84:0/2873402602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70e0158aa0 0x7f70e015de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:43.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.597+0000 7f70d7fff640 1 -- 172.21.15.84:0/2873402602 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e015c5d0 con 0x7f70e015be30 2023-10-16T11:26:43.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.597+0000 7f70d7fff640 1 --2- 172.21.15.84:0/2873402602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70e015be30 0x7f70e015d930 secure :-1 s=READY pgs=3212 cs=0 l=1 rev1=1 crypto rx=0x7f70c800c240 tx=0x7f70c8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:43.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.597+0000 7f70d77fe640 1 -- 172.21.15.84:0/2873402602 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8039030 con 0x7f70e015be30 2023-10-16T11:26:43.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.597+0000 7f70d77fe640 1 -- 172.21.15.84:0/2873402602 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c8004bf0 con 0x7f70e015be30 2023-10-16T11:26:43.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.597+0000 7f70e52e7640 1 -- 172.21.15.84:0/2873402602 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e015c8b0 con 0x7f70e015be30 2023-10-16T11:26:43.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.597+0000 7f70d77fe640 1 -- 172.21.15.84:0/2873402602 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c803c040 con 0x7f70e015be30 2023-10-16T11:26:43.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.597+0000 7f70e52e7640 1 -- 172.21.15.84:0/2873402602 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e0159670 con 0x7f70e015be30 2023-10-16T11:26:43.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.599+0000 7f70e52e7640 1 -- 172.21.15.84:0/2873402602 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e004e160 con 0x7f70e015be30 2023-10-16T11:26:43.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.600+0000 7f70d77fe640 1 -- 172.21.15.84:0/2873402602 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 151) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f70c8032070 con 0x7f70e015be30 2023-10-16T11:26:43.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.601+0000 7f70d77fe640 1 --2- 172.21.15.84:0/2873402602 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f70c006b0b0 0x7f70c006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:43.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.601+0000 7f70d77fe640 1 -- 172.21.15.84:0/2873402602 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5780+0+0 (secure 0 0 0) 0x7f70c80aaf50 con 0x7f70e015be30 2023-10-16T11:26:43.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.601+0000 7f70deffd640 1 --2- 172.21.15.84:0/2873402602 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f70c006b0b0 0x7f70c006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:43.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.602+0000 7f70deffd640 1 --2- 172.21.15.84:0/2873402602 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f70c006b0b0 0x7f70c006d570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f70d00097f0 tx=0x7f70d0008040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:43.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.606+0000 7f70d77fe640 1 -- 172.21.15.84:0/2873402602 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f70c8075bb0 con 0x7f70e015be30 2023-10-16T11:26:43.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:43.839+0000 7f70e52e7640 1 -- 172.21.15.84:0/2873402602 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f70e00b3e40 con 0x7f70e015be30 2023-10-16T11:26:44.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.494+0000 7f70d77fe640 1 -- 172.21.15.84:0/2873402602 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v147) v1 ==== 118+0+0 (secure 0 0 0) 0x7f70c80791d0 con 0x7f70e015be30 2023-10-16T11:26:44.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.498+0000 7f70e52e7640 1 -- 172.21.15.84:0/2873402602 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f70c006b0b0 msgr2=0x7f70c006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:44.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.498+0000 7f70e52e7640 1 --2- 172.21.15.84:0/2873402602 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f70c006b0b0 0x7f70c006d570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f70d00097f0 tx=0x7f70d0008040 comp rx=0 tx=0).stop 2023-10-16T11:26:44.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.498+0000 7f70e52e7640 1 -- 172.21.15.84:0/2873402602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70e015be30 msgr2=0x7f70e015d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:44.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.498+0000 7f70e52e7640 1 --2- 172.21.15.84:0/2873402602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70e015be30 0x7f70e015d930 secure :-1 s=READY pgs=3212 cs=0 l=1 rev1=1 crypto rx=0x7f70c800c240 tx=0x7f70c8001a00 comp rx=0 tx=0).stop 2023-10-16T11:26:44.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.499+0000 7f70e52e7640 1 -- 172.21.15.84:0/2873402602 shutdown_connections 2023-10-16T11:26:44.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.499+0000 7f70e52e7640 1 --2- 172.21.15.84:0/2873402602 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f70c006b0b0 0x7f70c006d570 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:44.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.499+0000 7f70e52e7640 1 --2- 172.21.15.84:0/2873402602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70e0158aa0 0x7f70e015de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:44.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.499+0000 7f70e52e7640 1 --2- 172.21.15.84:0/2873402602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70e015be30 0x7f70e015d930 unknown :-1 s=CLOSED pgs=3212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:44.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.499+0000 7f70e52e7640 1 --2- 172.21.15.84:0/2873402602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70cc0047c0 0x7f70e015d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:44.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.499+0000 7f70e52e7640 1 -- 172.21.15.84:0/2873402602 >> 172.21.15.84:0/2873402602 conn(0x7f70e00ce9b0 msgr2=0x7f70e006a190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:44.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.499+0000 7f70e52e7640 1 -- 172.21.15.84:0/2873402602 shutdown_connections 2023-10-16T11:26:44.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.499+0000 7f70e52e7640 1 -- 172.21.15.84:0/2873402602 wait complete. 2023-10-16T11:26:44.522 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T11:26:44.522 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T11:26:44.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.740+0000 7fc04b651640 1 Processor -- start 2023-10-16T11:26:44.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.740+0000 7fc04b651640 1 -- start start 2023-10-16T11:26:44.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.741+0000 7fc04b651640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc04406a4b0 0x7fc0440ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:44.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.741+0000 7fc04b651640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0440cb030 con 0x7fc04415ad70 2023-10-16T11:26:44.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.741+0000 7fc04b651640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0440c91c0 con 0x7fc0440c9ca0 2023-10-16T11:26:44.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.741+0000 7fc04b651640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0440c94a0 con 0x7fc04406a4b0 2023-10-16T11:26:44.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.742+0000 7fc0493ed640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc04415ad70 0x7fc044069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38814/0 (socket says 172.21.15.84:38814) 2023-10-16T11:26:44.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.742+0000 7fc0493ed640 1 -- 172.21.15.84:0/3523767988 learned_addr learned my addr 172.21.15.84:0/3523767988 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:44.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.742+0000 7fc048bec640 1 --2- 172.21.15.84:0/3523767988 >> v2:172.21.15.157:3300/0 conn(0x7fc04406a4b0 0x7fc0440ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:44.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.743+0000 7fc03bfff640 1 -- 172.21.15.84:0/3523767988 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059643258 0 0) 0x7fc0440cb030 con 0x7fc04415ad70 2023-10-16T11:26:44.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.743+0000 7fc03bfff640 1 -- 172.21.15.84:0/3523767988 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc030003520 con 0x7fc04415ad70 2023-10-16T11:26:44.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.743+0000 7fc03bfff640 1 -- 172.21.15.84:0/3523767988 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1744772837 0 0) 0x7fc030003520 con 0x7fc04415ad70 2023-10-16T11:26:44.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.743+0000 7fc03bfff640 1 -- 172.21.15.84:0/3523767988 >> v2:172.21.15.157:3300/0 conn(0x7fc04406a4b0 msgr2=0x7fc0440ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:44.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.743+0000 7fc03bfff640 1 --2- 172.21.15.84:0/3523767988 >> v2:172.21.15.157:3300/0 conn(0x7fc04406a4b0 0x7fc0440ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:44.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.743+0000 7fc03bfff640 1 -- 172.21.15.84:0/3523767988 >> v1:172.21.15.157:6789/0 conn(0x7fc0440c9ca0 legacy=0x7fc0440ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:44.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.743+0000 7fc03bfff640 1 -- 172.21.15.84:0/3523767988 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0440c9780 con 0x7fc04415ad70 2023-10-16T11:26:44.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.743+0000 7fc03bfff640 1 -- 172.21.15.84:0/3523767988 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc040001670 con 0x7fc04415ad70 2023-10-16T11:26:44.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.743+0000 7fc03bfff640 1 -- 172.21.15.84:0/3523767988 >> v1:172.21.15.84:6789/0 conn(0x7fc04415ad70 legacy=0x7fc044069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:44.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.743+0000 7fc03bfff640 1 --2- 172.21.15.84:0/3523767988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc030003970 0x7fc030003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:44.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.743+0000 7fc03bfff640 1 --2- 172.21.15.84:0/3523767988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0300047c0 0x7fc030004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:44.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.743+0000 7fc03bfff640 1 --2- 172.21.15.84:0/3523767988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc030005180 0x7fc03000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:44.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.743+0000 7fc03bfff640 1 -- 172.21.15.84:0/3523767988 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440c94a0 con 0x7fc030005180 2023-10-16T11:26:44.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.743+0000 7fc03bfff640 1 -- 172.21.15.84:0/3523767988 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03000c020 con 0x7fc0300047c0 2023-10-16T11:26:44.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.743+0000 7fc03bfff640 1 -- 172.21.15.84:0/3523767988 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03000c1e0 con 0x7fc030003970 2023-10-16T11:26:44.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.744+0000 7fc049bee640 1 --2- 172.21.15.84:0/3523767988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc030005180 0x7fc03000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:44.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.744+0000 7fc0493ed640 1 --2- 172.21.15.84:0/3523767988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc030003970 0x7fc030003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:44.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.744+0000 7fc048bec640 1 --2- 172.21.15.84:0/3523767988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0300047c0 0x7fc030004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:44.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.744+0000 7fc049bee640 1 -- 172.21.15.84:0/3523767988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc030003970 msgr2=0x7fc030003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:44.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.744+0000 7fc049bee640 1 --2- 172.21.15.84:0/3523767988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc030003970 0x7fc030003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:44.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.744+0000 7fc049bee640 1 -- 172.21.15.84:0/3523767988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0300047c0 msgr2=0x7fc030004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:44.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.744+0000 7fc049bee640 1 --2- 172.21.15.84:0/3523767988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0300047c0 0x7fc030004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:44.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.744+0000 7fc049bee640 1 -- 172.21.15.84:0/3523767988 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc030003520 con 0x7fc030005180 2023-10-16T11:26:44.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.744+0000 7fc048bec640 1 --2- 172.21.15.84:0/3523767988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0300047c0 0x7fc030004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:44.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.745+0000 7fc049bee640 1 -- 172.21.15.84:0/3523767988 reap_dead start 2023-10-16T11:26:44.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.746+0000 7fc049bee640 1 --2- 172.21.15.84:0/3523767988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc030005180 0x7fc03000b980 secure :-1 s=READY pgs=3214 cs=0 l=1 rev1=1 crypto rx=0x7fc03c001e50 tx=0x7fc0300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698b9997f1f4e272 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:44.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.746+0000 7fc03bfff640 1 -- 172.21.15.84:0/3523767988 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c003b70 con 0x7fc030005180 2023-10-16T11:26:44.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.746+0000 7fc03bfff640 1 -- 172.21.15.84:0/3523767988 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c003d10 con 0x7fc030005180 2023-10-16T11:26:44.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.746+0000 7fc03bfff640 1 -- 172.21.15.84:0/3523767988 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c005310 con 0x7fc030005180 2023-10-16T11:26:44.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.746+0000 7fc04b651640 1 -- 172.21.15.84:0/3523767988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc030005180 msgr2=0x7fc03000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:44.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.746+0000 7fc04b651640 1 --2- 172.21.15.84:0/3523767988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc030005180 0x7fc03000b980 secure :-1 s=READY pgs=3214 cs=0 l=1 rev1=1 crypto rx=0x7fc03c001e50 tx=0x7fc0300057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:44.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.747+0000 7fc04b651640 1 -- 172.21.15.84:0/3523767988 shutdown_connections 2023-10-16T11:26:44.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.747+0000 7fc04b651640 1 --2- 172.21.15.84:0/3523767988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc030005180 0x7fc03000b980 unknown :-1 s=CLOSED pgs=3214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:44.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.747+0000 7fc04b651640 1 -- 172.21.15.84:0/3523767988 >> 172.21.15.84:0/3523767988 conn(0x7fc0440ce9b0 msgr2=0x7fc0440cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:44.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.747+0000 7fc04b651640 1 -- 172.21.15.84:0/3523767988 shutdown_connections 2023-10-16T11:26:44.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.747+0000 7fc04b651640 1 -- 172.21.15.84:0/3523767988 wait complete. 2023-10-16T11:26:44.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.749+0000 7fc04b651640 1 Processor -- start 2023-10-16T11:26:44.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.749+0000 7fc04b651640 1 -- start start 2023-10-16T11:26:44.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.750+0000 7fc04b651640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030005180 0x7fc04414be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:44.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.750+0000 7fc04b651640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0440c7c50 0x7fc0440c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:44.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.750+0000 7fc04b651640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0441f2980 0x7fc0441f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:44.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.750+0000 7fc04b651640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440ad440 con 0x7fc0441f2980 2023-10-16T11:26:44.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.750+0000 7fc04b651640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440ac890 con 0x7fc030005180 2023-10-16T11:26:44.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.750+0000 7fc04b651640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440c8610 con 0x7fc0440c7c50 2023-10-16T11:26:44.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.750+0000 7fc049bee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0441f2980 0x7fc0441f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:44.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.750+0000 7fc049bee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0441f2980 0x7fc0441f2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52080/0 (socket says 172.21.15.84:52080) 2023-10-16T11:26:44.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.750+0000 7fc049bee640 1 -- 172.21.15.84:0/2258281862 learned_addr learned my addr 172.21.15.84:0/2258281862 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:44.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.750+0000 7fc048bec640 1 --2- 172.21.15.84:0/2258281862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0440c7c50 0x7fc0440c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:44.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.750+0000 7fc0493ed640 1 --2- 172.21.15.84:0/2258281862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030005180 0x7fc04414be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:44.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.750+0000 7fc049bee640 1 -- 172.21.15.84:0/2258281862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0440c7c50 msgr2=0x7fc0440c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:44.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.750+0000 7fc049bee640 1 --2- 172.21.15.84:0/2258281862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0440c7c50 0x7fc0440c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:44.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.750+0000 7fc049bee640 1 -- 172.21.15.84:0/2258281862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030005180 msgr2=0x7fc04414be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:44.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.750+0000 7fc049bee640 1 --2- 172.21.15.84:0/2258281862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030005180 0x7fc04414be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:44.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.750+0000 7fc049bee640 1 -- 172.21.15.84:0/2258281862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04414c360 con 0x7fc0441f2980 2023-10-16T11:26:44.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.751+0000 7fc049bee640 1 --2- 172.21.15.84:0/2258281862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0441f2980 0x7fc0441f2e00 secure :-1 s=READY pgs=3215 cs=0 l=1 rev1=1 crypto rx=0x7fc03c001f70 tx=0x7fc03c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:44.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.751+0000 7fc039ffb640 1 -- 172.21.15.84:0/2258281862 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c006d50 con 0x7fc0441f2980 2023-10-16T11:26:44.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.751+0000 7fc039ffb640 1 -- 172.21.15.84:0/2258281862 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c003b80 con 0x7fc0441f2980 2023-10-16T11:26:44.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.751+0000 7fc04b651640 1 -- 172.21.15.84:0/2258281862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0440acaa0 con 0x7fc0441f2980 2023-10-16T11:26:44.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.751+0000 7fc039ffb640 1 -- 172.21.15.84:0/2258281862 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c0396c0 con 0x7fc0441f2980 2023-10-16T11:26:44.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.751+0000 7fc04b651640 1 -- 172.21.15.84:0/2258281862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0440ad060 con 0x7fc0441f2980 2023-10-16T11:26:44.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.752+0000 7fc04b651640 1 -- 172.21.15.84:0/2258281862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0440ad220 con 0x7fc0441f2980 2023-10-16T11:26:44.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.753+0000 7fc039ffb640 1 -- 172.21.15.84:0/2258281862 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 151) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fc03c039820 con 0x7fc0441f2980 2023-10-16T11:26:44.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.754+0000 7fc039ffb640 1 --2- 172.21.15.84:0/2258281862 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc02806b0b0 0x7fc02806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:44.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.754+0000 7fc039ffb640 1 -- 172.21.15.84:0/2258281862 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5780+0+0 (secure 0 0 0) 0x7fc03c0ab9f0 con 0x7fc0441f2980 2023-10-16T11:26:44.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.754+0000 7fc0493ed640 1 --2- 172.21.15.84:0/2258281862 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc02806b0b0 0x7fc02806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:44.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.755+0000 7fc0493ed640 1 --2- 172.21.15.84:0/2258281862 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc02806b0b0 0x7fc02806d570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc040001200 tx=0x7fc040001150 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:44.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.759+0000 7fc039ffb640 1 -- 172.21.15.84:0/2258281862 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc03c076690 con 0x7fc0441f2980 2023-10-16T11:26:44.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:44.989+0000 7fc04b651640 1 -- 172.21.15.84:0/2258281862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc04404e160 con 0x7fc0441f2980 2023-10-16T11:26:45.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:45.907+0000 7fc039ffb640 1 -- 172.21.15.84:0/2258281862 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v148) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc03c0480c0 con 0x7fc0441f2980 2023-10-16T11:26:45.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:45.911+0000 7fc04b651640 1 -- 172.21.15.84:0/2258281862 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc02806b0b0 msgr2=0x7fc02806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:45.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:45.911+0000 7fc04b651640 1 --2- 172.21.15.84:0/2258281862 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc02806b0b0 0x7fc02806d570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc040001200 tx=0x7fc040001150 comp rx=0 tx=0).stop 2023-10-16T11:26:45.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:45.911+0000 7fc04b651640 1 -- 172.21.15.84:0/2258281862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0441f2980 msgr2=0x7fc0441f2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:45.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:45.911+0000 7fc04b651640 1 --2- 172.21.15.84:0/2258281862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0441f2980 0x7fc0441f2e00 secure :-1 s=READY pgs=3215 cs=0 l=1 rev1=1 crypto rx=0x7fc03c001f70 tx=0x7fc03c0014a0 comp rx=0 tx=0).stop 2023-10-16T11:26:45.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:45.911+0000 7fc04b651640 1 -- 172.21.15.84:0/2258281862 shutdown_connections 2023-10-16T11:26:45.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:45.911+0000 7fc04b651640 1 --2- 172.21.15.84:0/2258281862 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc02806b0b0 0x7fc02806d570 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:45.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:45.911+0000 7fc04b651640 1 --2- 172.21.15.84:0/2258281862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0441f2980 0x7fc0441f2e00 unknown :-1 s=CLOSED pgs=3215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:45.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:45.911+0000 7fc04b651640 1 --2- 172.21.15.84:0/2258281862 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0440c7c50 0x7fc0440c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:45.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:45.911+0000 7fc04b651640 1 --2- 172.21.15.84:0/2258281862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030005180 0x7fc04414be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:45.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:45.911+0000 7fc04b651640 1 -- 172.21.15.84:0/2258281862 >> 172.21.15.84:0/2258281862 conn(0x7fc0440ce9b0 msgr2=0x7fc0440c5310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:45.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:45.911+0000 7fc04b651640 1 -- 172.21.15.84:0/2258281862 shutdown_connections 2023-10-16T11:26:45.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:45.911+0000 7fc04b651640 1 -- 172.21.15.84:0/2258281862 wait complete. 2023-10-16T11:26:45.932 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T11:26:45.933 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:26:46.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.142+0000 7fd81537c640 1 Processor -- start 2023-10-16T11:26:46.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.142+0000 7fd81537c640 1 -- start start 2023-10-16T11:26:46.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.144+0000 7fd81537c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd81015aaa0 0x7fd81015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:46.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.144+0000 7fd81537c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd8100ce390 0x7fd810070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:46.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.144+0000 7fd81537c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd81015ed60 con 0x7fd81015ae80 2023-10-16T11:26:46.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.144+0000 7fd81537c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd81006b810 con 0x7fd8100ce390 2023-10-16T11:26:46.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.144+0000 7fd81537c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8100707e0 con 0x7fd81015aaa0 2023-10-16T11:26:46.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.144+0000 7fd80effd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd81015aaa0 0x7fd81015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:46.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.145+0000 7fd80f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd8100ce390 0x7fd810070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:46.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.145+0000 7fd80effd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd81015aaa0 0x7fd81015e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:43434/0 (socket says 172.21.15.84:43434) 2023-10-16T11:26:46.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.145+0000 7fd80effd640 1 -- 172.21.15.84:0/63779092 learned_addr learned my addr 172.21.15.84:0/63779092 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:46.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.146+0000 7fd80f7fe640 1 -- 172.21.15.84:0/63779092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd81015aaa0 msgr2=0x7fd81015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:46.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.146+0000 7fd80f7fe640 1 --2- 172.21.15.84:0/63779092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd81015aaa0 0x7fd81015e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.146+0000 7fd80f7fe640 1 -- 172.21.15.84:0/63779092 >> v1:172.21.15.157:6789/0 conn(0x7fd81015ae80 legacy=0x7fd8100cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:46.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.146+0000 7fd80f7fe640 1 -- 172.21.15.84:0/63779092 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81015ef90 con 0x7fd8100ce390 2023-10-16T11:26:46.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.147+0000 7fd80f7fe640 1 --2- 172.21.15.84:0/63779092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8100ce390 0x7fd810070f50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd808009d90 tx=0x7fd80802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ea3c595c9749918 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:46.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.147+0000 7fd80e7fc640 1 -- 172.21.15.84:0/63779092 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80802cc50 con 0x7fd8100ce390 2023-10-16T11:26:46.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.148+0000 7fd80e7fc640 1 -- 172.21.15.84:0/63779092 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80802cdb0 con 0x7fd8100ce390 2023-10-16T11:26:46.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.148+0000 7fd80e7fc640 1 -- 172.21.15.84:0/63779092 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd808034660 con 0x7fd8100ce390 2023-10-16T11:26:46.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.148+0000 7fd81537c640 1 -- 172.21.15.84:0/63779092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8100ce390 msgr2=0x7fd810070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:46.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.148+0000 7fd81537c640 1 --2- 172.21.15.84:0/63779092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8100ce390 0x7fd810070f50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd808009d90 tx=0x7fd80802bea0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.148+0000 7fd81537c640 1 -- 172.21.15.84:0/63779092 shutdown_connections 2023-10-16T11:26:46.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.148+0000 7fd81537c640 1 --2- 172.21.15.84:0/63779092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8100ce390 0x7fd810070f50 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.148+0000 7fd81537c640 1 --2- 172.21.15.84:0/63779092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd81015aaa0 0x7fd81015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.148+0000 7fd81537c640 1 -- 172.21.15.84:0/63779092 >> 172.21.15.84:0/63779092 conn(0x7fd8100cb410 msgr2=0x7fd8100cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:46.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.149+0000 7fd81537c640 1 -- 172.21.15.84:0/63779092 shutdown_connections 2023-10-16T11:26:46.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.149+0000 7fd81537c640 1 -- 172.21.15.84:0/63779092 wait complete. 2023-10-16T11:26:46.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.151+0000 7fd81537c640 1 Processor -- start 2023-10-16T11:26:46.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.151+0000 7fd81537c640 1 -- start start 2023-10-16T11:26:46.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.152+0000 7fd81537c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8100ce390 0x7fd810157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:46.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.152+0000 7fd81537c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd81015aaa0 0x7fd8100c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:46.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.152+0000 7fd81537c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd81015ae80 0x7fd810070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:46.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.152+0000 7fd81537c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8100c51c0 con 0x7fd81015ae80 2023-10-16T11:26:46.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.152+0000 7fd81537c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8100c4c00 con 0x7fd81015aaa0 2023-10-16T11:26:46.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.152+0000 7fd81537c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8100c4ee0 con 0x7fd8100ce390 2023-10-16T11:26:46.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.152+0000 7fd80f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd81015ae80 0x7fd810070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:46.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.152+0000 7fd80f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd81015ae80 0x7fd810070df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51576/0 (socket says 172.21.15.84:51576) 2023-10-16T11:26:46.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.152+0000 7fd80f7fe640 1 -- 172.21.15.84:0/1153408053 learned_addr learned my addr 172.21.15.84:0/1153408053 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:46.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.153+0000 7fd80effd640 1 --2- 172.21.15.84:0/1153408053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8100ce390 0x7fd810157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:46.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.153+0000 7fd807fff640 1 --2- 172.21.15.84:0/1153408053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd81015aaa0 0x7fd8100c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:46.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.153+0000 7fd80f7fe640 1 -- 172.21.15.84:0/1153408053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8100ce390 msgr2=0x7fd810157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:46.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.153+0000 7fd80f7fe640 1 --2- 172.21.15.84:0/1153408053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8100ce390 0x7fd810157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.153+0000 7fd80f7fe640 1 -- 172.21.15.84:0/1153408053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd81015aaa0 msgr2=0x7fd8100c42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:46.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.153+0000 7fd80f7fe640 1 --2- 172.21.15.84:0/1153408053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd81015aaa0 0x7fd8100c42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.153+0000 7fd80f7fe640 1 -- 172.21.15.84:0/1153408053 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd808009a70 con 0x7fd81015ae80 2023-10-16T11:26:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.153+0000 7fd807fff640 1 --2- 172.21.15.84:0/1153408053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd81015aaa0 0x7fd8100c42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.153+0000 7fd80f7fe640 1 --2- 172.21.15.84:0/1153408053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd81015ae80 0x7fd810070df0 secure :-1 s=READY pgs=3216 cs=0 l=1 rev1=1 crypto rx=0x7fd808009d90 tx=0x7fd808004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.153+0000 7fd8077fe640 1 -- 172.21.15.84:0/1153408053 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd808040650 con 0x7fd81015ae80 2023-10-16T11:26:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.154+0000 7fd8077fe640 1 -- 172.21.15.84:0/1153408053 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8080407b0 con 0x7fd81015ae80 2023-10-16T11:26:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.154+0000 7fd81537c640 1 -- 172.21.15.84:0/1153408053 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8100c90f0 con 0x7fd81015ae80 2023-10-16T11:26:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.154+0000 7fd8077fe640 1 -- 172.21.15.84:0/1153408053 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd808040b60 con 0x7fd81015ae80 2023-10-16T11:26:46.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.154+0000 7fd81537c640 1 -- 172.21.15.84:0/1153408053 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8101583e0 con 0x7fd81015ae80 2023-10-16T11:26:46.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.155+0000 7fd81537c640 1 -- 172.21.15.84:0/1153408053 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8100c9610 con 0x7fd81015ae80 2023-10-16T11:26:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.156+0000 7fd8077fe640 1 -- 172.21.15.84:0/1153408053 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 151) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd808040cc0 con 0x7fd81015ae80 2023-10-16T11:26:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.157+0000 7fd8077fe640 1 --2- 172.21.15.84:0/1153408053 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd7f006b0b0 0x7fd7f006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.157+0000 7fd8077fe640 1 -- 172.21.15.84:0/1153408053 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5780+0+0 (secure 0 0 0) 0x7fd8080ac8b0 con 0x7fd81015ae80 2023-10-16T11:26:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.157+0000 7fd80effd640 1 --2- 172.21.15.84:0/1153408053 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd7f006b0b0 0x7fd7f006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.158+0000 7fd80effd640 1 --2- 172.21.15.84:0/1153408053 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd7f006b0b0 0x7fd7f006d570 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd810159c30 tx=0x7fd800008040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:46.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.162+0000 7fd8077fe640 1 -- 172.21.15.84:0/1153408053 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd8100c9610 con 0x7fd81015ae80 2023-10-16T11:26:46.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.393+0000 7fd81537c640 1 -- 172.21.15.84:0/1153408053 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8100c9610 con 0x7fd81015ae80 2023-10-16T11:26:46.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.394+0000 7fd8077fe640 1 -- 172.21.15.84:0/1153408053 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+4929 (secure 0 0 0) 0x7fd8100c9610 con 0x7fd81015ae80 2023-10-16T11:26:46.396 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:46.397 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":148,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/591944719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":591944719},{"type":"v1","addr":"172.21.15.84:6835","nonce":591944719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":142},{"gid":8152,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2371517609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2371517609},{"type":"v1","addr":"172.21.15.84:6837","nonce":2371517609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":143},{"gid":8161,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1079456313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1079456313},{"type":"v1","addr":"172.21.15.157:6834","nonce":1079456313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":143}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:26:41.908585+0000","modified":"2023-10-16T11:26:44.991714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12987":{"gid":12987,"name":"b","rank":0,"incarnation":144,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3899307325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3899307325},{"type":"v1","addr":"172.21.15.157:6835","nonce":3899307325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-16T11:26:46.397 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 148 2023-10-16T11:26:46.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.397+0000 7fd81537c640 1 -- 172.21.15.84:0/1153408053 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd7f006b0b0 msgr2=0x7fd7f006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:46.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.397+0000 7fd81537c640 1 --2- 172.21.15.84:0/1153408053 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd7f006b0b0 0x7fd7f006d570 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd810159c30 tx=0x7fd800008040 comp rx=0 tx=0).stop 2023-10-16T11:26:46.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.397+0000 7fd81537c640 1 -- 172.21.15.84:0/1153408053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd81015ae80 msgr2=0x7fd810070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:46.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.397+0000 7fd81537c640 1 --2- 172.21.15.84:0/1153408053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd81015ae80 0x7fd810070df0 secure :-1 s=READY pgs=3216 cs=0 l=1 rev1=1 crypto rx=0x7fd808009d90 tx=0x7fd808004a40 comp rx=0 tx=0).stop 2023-10-16T11:26:46.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.397+0000 7fd81537c640 1 -- 172.21.15.84:0/1153408053 shutdown_connections 2023-10-16T11:26:46.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.397+0000 7fd81537c640 1 --2- 172.21.15.84:0/1153408053 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd7f006b0b0 0x7fd7f006d570 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.397+0000 7fd81537c640 1 --2- 172.21.15.84:0/1153408053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd81015ae80 0x7fd810070df0 unknown :-1 s=CLOSED pgs=3216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.397+0000 7fd81537c640 1 --2- 172.21.15.84:0/1153408053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd81015aaa0 0x7fd8100c42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.397+0000 7fd81537c640 1 --2- 172.21.15.84:0/1153408053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8100ce390 0x7fd810157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.397+0000 7fd81537c640 1 -- 172.21.15.84:0/1153408053 >> 172.21.15.84:0/1153408053 conn(0x7fd8100cb410 msgr2=0x7fd8101589f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:46.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.397+0000 7fd81537c640 1 -- 172.21.15.84:0/1153408053 shutdown_connections 2023-10-16T11:26:46.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.397+0000 7fd81537c640 1 -- 172.21.15.84:0/1153408053 wait complete. 2023-10-16T11:26:46.416 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:26:46.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.628+0000 7fead2bc0640 1 Processor -- start 2023-10-16T11:26:46.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.628+0000 7fead2bc0640 1 -- start start 2023-10-16T11:26:46.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.629+0000 7fead2bc0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feacc159b20 0x7feacc06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:46.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.629+0000 7fead2bc0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feacc159f00 0x7feacc0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:46.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.629+0000 7fead2bc0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feacc06f7f0 con 0x7feacc06c120 2023-10-16T11:26:46.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.629+0000 7fead2bc0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feacc06c5e0 con 0x7feacc159f00 2023-10-16T11:26:46.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.629+0000 7fead2bc0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feacc06c8c0 con 0x7feacc159b20 2023-10-16T11:26:46.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.629+0000 7feacbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feacc159f00 0x7feacc0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:46.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.629+0000 7feacbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feacc159f00 0x7feacc0b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51578/0 (socket says 172.21.15.84:51578) 2023-10-16T11:26:46.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.630+0000 7feacbfff640 1 -- 172.21.15.84:0/2331292251 learned_addr learned my addr 172.21.15.84:0/2331292251 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:46.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.630+0000 7fead095c640 1 --2- 172.21.15.84:0/2331292251 >> v2:172.21.15.157:3300/0 conn(0x7feacc159b20 0x7feacc06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:46.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.630+0000 7feacbfff640 1 -- 172.21.15.84:0/2331292251 >> v2:172.21.15.157:3300/0 conn(0x7feacc159b20 msgr2=0x7feacc06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:46.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.630+0000 7feacbfff640 1 --2- 172.21.15.84:0/2331292251 >> v2:172.21.15.157:3300/0 conn(0x7feacc159b20 0x7feacc06bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.630+0000 7feacbfff640 1 -- 172.21.15.84:0/2331292251 >> v1:172.21.15.157:6789/0 conn(0x7feacc06c120 legacy=0x7feacc0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:46.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.630+0000 7feacbfff640 1 -- 172.21.15.84:0/2331292251 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feacc06cba0 con 0x7feacc159f00 2023-10-16T11:26:46.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.631+0000 7feacbfff640 1 --2- 172.21.15.84:0/2331292251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc159f00 0x7feacc0b0410 secure :-1 s=READY pgs=3217 cs=0 l=1 rev1=1 crypto rx=0x7feacc0c9340 tx=0x7feabc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cdbd20ed43f63c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:46.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.631+0000 7feacb7fe640 1 -- 172.21.15.84:0/2331292251 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc02ca80 con 0x7feacc159f00 2023-10-16T11:26:46.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.631+0000 7feacb7fe640 1 -- 172.21.15.84:0/2331292251 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc02cbe0 con 0x7feacc159f00 2023-10-16T11:26:46.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.631+0000 7feacb7fe640 1 -- 172.21.15.84:0/2331292251 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc0343f0 con 0x7feacc159f00 2023-10-16T11:26:46.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.632+0000 7fead2bc0640 1 -- 172.21.15.84:0/2331292251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc159f00 msgr2=0x7feacc0b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:46.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.632+0000 7fead2bc0640 1 --2- 172.21.15.84:0/2331292251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc159f00 0x7feacc0b0410 secure :-1 s=READY pgs=3217 cs=0 l=1 rev1=1 crypto rx=0x7feacc0c9340 tx=0x7feabc02bee0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.632+0000 7fead2bc0640 1 -- 172.21.15.84:0/2331292251 shutdown_connections 2023-10-16T11:26:46.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.632+0000 7fead2bc0640 1 --2- 172.21.15.84:0/2331292251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc159f00 0x7feacc0b0410 unknown :-1 s=CLOSED pgs=3217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.632+0000 7fead2bc0640 1 --2- 172.21.15.84:0/2331292251 >> v2:172.21.15.157:3300/0 conn(0x7feacc159b20 0x7feacc06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.632+0000 7fead2bc0640 1 -- 172.21.15.84:0/2331292251 >> 172.21.15.84:0/2331292251 conn(0x7feacc0cb410 msgr2=0x7feacc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:46.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.632+0000 7fead2bc0640 1 -- 172.21.15.84:0/2331292251 shutdown_connections 2023-10-16T11:26:46.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.632+0000 7fead2bc0640 1 -- 172.21.15.84:0/2331292251 wait complete. 2023-10-16T11:26:46.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.634+0000 7fead2bc0640 1 Processor -- start 2023-10-16T11:26:46.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.634+0000 7fead2bc0640 1 -- start start 2023-10-16T11:26:46.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.635+0000 7fead2bc0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc06c120 0x7feacc0c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:46.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.635+0000 7fead2bc0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feacc159b20 0x7feacc1f6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:46.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.635+0000 7fead2bc0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feacc159f00 0x7feacc0b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:46.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.635+0000 7fead2bc0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc0c4ab0 con 0x7feacc06c120 2023-10-16T11:26:46.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.635+0000 7fead2bc0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc1f6da0 con 0x7feacc159b20 2023-10-16T11:26:46.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.635+0000 7fead2bc0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feacc0c3e30 con 0x7feacc159f00 2023-10-16T11:26:46.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.635+0000 7fead095c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc06c120 0x7feacc0c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:46.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.635+0000 7fead095c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc06c120 0x7feacc0c4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51588/0 (socket says 172.21.15.84:51588) 2023-10-16T11:26:46.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.635+0000 7fead095c640 1 -- 172.21.15.84:0/2487649393 learned_addr learned my addr 172.21.15.84:0/2487649393 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:46.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.636+0000 7fead095c640 1 -- 172.21.15.84:0/2487649393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feacc159f00 msgr2=0x7feacc0b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:46.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.636+0000 7fead095c640 1 --2- 172.21.15.84:0/2487649393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feacc159f00 0x7feacc0b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.636+0000 7fead095c640 1 -- 172.21.15.84:0/2487649393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feacc159b20 msgr2=0x7feacc1f6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:46.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.636+0000 7fead095c640 1 --2- 172.21.15.84:0/2487649393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feacc159b20 0x7feacc1f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.636+0000 7fead095c640 1 -- 172.21.15.84:0/2487649393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc009b30 con 0x7feacc06c120 2023-10-16T11:26:46.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.636+0000 7fead095c640 1 --2- 172.21.15.84:0/2487649393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc06c120 0x7feacc0c4ed0 secure :-1 s=READY pgs=3218 cs=0 l=1 rev1=1 crypto rx=0x7feab400c7f0 tx=0x7feab400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:46.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.636+0000 7feac97fa640 1 -- 172.21.15.84:0/2487649393 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4004220 con 0x7feacc06c120 2023-10-16T11:26:46.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.636+0000 7feac97fa640 1 -- 172.21.15.84:0/2487649393 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab40043a0 con 0x7feacc06c120 2023-10-16T11:26:46.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.636+0000 7feac97fa640 1 -- 172.21.15.84:0/2487649393 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4007860 con 0x7feacc06c120 2023-10-16T11:26:46.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.636+0000 7fead2bc0640 1 -- 172.21.15.84:0/2487649393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feacc0c4110 con 0x7feacc06c120 2023-10-16T11:26:46.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.636+0000 7fead2bc0640 1 -- 172.21.15.84:0/2487649393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feacc0c46d0 con 0x7feacc06c120 2023-10-16T11:26:46.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.637+0000 7fead2bc0640 1 -- 172.21.15.84:0/2487649393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feacc04e160 con 0x7feacc06c120 2023-10-16T11:26:46.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.638+0000 7feac97fa640 1 -- 172.21.15.84:0/2487649393 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 151) v1 ==== 89433+0+0 (secure 0 0 0) 0x7feab4007bd0 con 0x7feacc06c120 2023-10-16T11:26:46.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.639+0000 7feac97fa640 1 --2- 172.21.15.84:0/2487649393 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7feaac06b0b0 0x7feaac06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:46.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.639+0000 7feac97fa640 1 -- 172.21.15.84:0/2487649393 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5780+0+0 (secure 0 0 0) 0x7feab4089b40 con 0x7feacc06c120 2023-10-16T11:26:46.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.640+0000 7feacbfff640 1 --2- 172.21.15.84:0/2487649393 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7feaac06b0b0 0x7feaac06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:46.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.640+0000 7feacbfff640 1 --2- 172.21.15.84:0/2487649393 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7feaac06b0b0 0x7feaac06d570 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feabc035000 tx=0x7feabc02e000 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:46.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.643+0000 7feac97fa640 1 -- 172.21.15.84:0/2487649393 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7feab4054730 con 0x7feacc06c120 2023-10-16T11:26:46.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.837+0000 7fead2bc0640 1 -- 172.21.15.84:0/2487649393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feacc0c5440 con 0x7feacc06c120 2023-10-16T11:26:46.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.840+0000 7feac97fa640 1 -- 172.21.15.84:0/2487649393 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v257) v1 ==== 74+0+16607 (secure 0 0 0) 0x7feab401e020 con 0x7feacc06c120 2023-10-16T11:26:46.843 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:46.843 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":257,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:26:43.439554+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:26:39.860053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-16T11:26:40.916519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/570042115":"2023-10-16T12:26:40.168744+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:26:46.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.843+0000 7fead2bc0640 1 -- 172.21.15.84:0/2487649393 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7feaac06b0b0 msgr2=0x7feaac06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:46.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.843+0000 7fead2bc0640 1 --2- 172.21.15.84:0/2487649393 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7feaac06b0b0 0x7feaac06d570 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feabc035000 tx=0x7feabc02e000 comp rx=0 tx=0).stop 2023-10-16T11:26:46.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.843+0000 7fead2bc0640 1 -- 172.21.15.84:0/2487649393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc06c120 msgr2=0x7feacc0c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:46.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.843+0000 7fead2bc0640 1 --2- 172.21.15.84:0/2487649393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc06c120 0x7feacc0c4ed0 secure :-1 s=READY pgs=3218 cs=0 l=1 rev1=1 crypto rx=0x7feab400c7f0 tx=0x7feab400ccb0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.843+0000 7fead2bc0640 1 -- 172.21.15.84:0/2487649393 shutdown_connections 2023-10-16T11:26:46.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.843+0000 7fead2bc0640 1 --2- 172.21.15.84:0/2487649393 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7feaac06b0b0 0x7feaac06d570 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.843+0000 7fead2bc0640 1 --2- 172.21.15.84:0/2487649393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feacc159f00 0x7feacc0b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.843+0000 7fead2bc0640 1 --2- 172.21.15.84:0/2487649393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feacc159b20 0x7feacc1f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.843+0000 7fead2bc0640 1 --2- 172.21.15.84:0/2487649393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feacc06c120 0x7feacc0c4ed0 unknown :-1 s=CLOSED pgs=3218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:46.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.843+0000 7fead2bc0640 1 -- 172.21.15.84:0/2487649393 >> 172.21.15.84:0/2487649393 conn(0x7feacc0cb410 msgr2=0x7feacc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:46.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.844+0000 7fead2bc0640 1 -- 172.21.15.84:0/2487649393 shutdown_connections 2023-10-16T11:26:46.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:46.844+0000 7fead2bc0640 1 -- 172.21.15.84:0/2487649393 wait complete. 2023-10-16T11:26:46.863 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T11:26:46.864 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:26:47.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.052+0000 7fe369ba9640 1 Processor -- start 2023-10-16T11:26:47.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.053+0000 7fe369ba9640 1 -- start start 2023-10-16T11:26:47.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.054+0000 7fe369ba9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe36415ae80 0x7fe36415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:47.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.054+0000 7fe369ba9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36415ed60 con 0x7fe3640ce390 2023-10-16T11:26:47.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.054+0000 7fe369ba9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3640707e0 con 0x7fe36415aaa0 2023-10-16T11:26:47.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.054+0000 7fe369ba9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36406b810 con 0x7fe36415ae80 2023-10-16T11:26:47.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.054+0000 7fe363fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe3640ce390 0x7fe364070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40944/0 (socket says 172.21.15.84:40944) 2023-10-16T11:26:47.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.054+0000 7fe363fff640 1 -- 172.21.15.84:0/4066388752 learned_addr learned my addr 172.21.15.84:0/4066388752 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:47.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.055+0000 7fe35bfff640 1 --2- 172.21.15.84:0/4066388752 >> v2:172.21.15.157:3300/0 conn(0x7fe36415ae80 0x7fe36415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:47.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.055+0000 7fe362ffd640 1 -- 172.21.15.84:0/4066388752 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283501561 0 0) 0x7fe36415ed60 con 0x7fe3640ce390 2023-10-16T11:26:47.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.055+0000 7fe362ffd640 1 -- 172.21.15.84:0/4066388752 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe350003520 con 0x7fe3640ce390 2023-10-16T11:26:47.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.055+0000 7fe362ffd640 1 -- 172.21.15.84:0/4066388752 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1080576170 0 0) 0x7fe350003520 con 0x7fe3640ce390 2023-10-16T11:26:47.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.055+0000 7fe362ffd640 1 -- 172.21.15.84:0/4066388752 >> v2:172.21.15.157:3300/0 conn(0x7fe36415ae80 msgr2=0x7fe36415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:47.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.055+0000 7fe362ffd640 1 --2- 172.21.15.84:0/4066388752 >> v2:172.21.15.157:3300/0 conn(0x7fe36415ae80 0x7fe36415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.055+0000 7fe362ffd640 1 -- 172.21.15.84:0/4066388752 >> v1:172.21.15.157:6789/0 conn(0x7fe36415aaa0 legacy=0x7fe3640cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:47.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.055+0000 7fe362ffd640 1 -- 172.21.15.84:0/4066388752 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36415ef90 con 0x7fe3640ce390 2023-10-16T11:26:47.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.055+0000 7fe362ffd640 1 -- 172.21.15.84:0/4066388752 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fe35c0019f0 con 0x7fe3640ce390 2023-10-16T11:26:47.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.056+0000 7fe362ffd640 1 -- 172.21.15.84:0/4066388752 >> v1:172.21.15.84:6789/0 conn(0x7fe3640ce390 legacy=0x7fe364070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:47.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.056+0000 7fe362ffd640 1 --2- 172.21.15.84:0/4066388752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe350003970 0x7fe350003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:47.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.056+0000 7fe362ffd640 1 --2- 172.21.15.84:0/4066388752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3500047c0 0x7fe350004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:47.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.056+0000 7fe362ffd640 1 --2- 172.21.15.84:0/4066388752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe350005180 0x7fe35000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:47.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.056+0000 7fe362ffd640 1 -- 172.21.15.84:0/4066388752 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36406b810 con 0x7fe3500047c0 2023-10-16T11:26:47.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.056+0000 7fe362ffd640 1 -- 172.21.15.84:0/4066388752 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35000c020 con 0x7fe350003970 2023-10-16T11:26:47.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.056+0000 7fe362ffd640 1 -- 172.21.15.84:0/4066388752 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35000c1e0 con 0x7fe350005180 2023-10-16T11:26:47.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.056+0000 7fe35bfff640 1 --2- 172.21.15.84:0/4066388752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3500047c0 0x7fe350004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:47.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.056+0000 7fe35bfff640 1 -- 172.21.15.84:0/4066388752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe350005180 msgr2=0x7fe35000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:47.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.056+0000 7fe35bfff640 1 --2- 172.21.15.84:0/4066388752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe350005180 0x7fe35000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.056+0000 7fe35bfff640 1 -- 172.21.15.84:0/4066388752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe350003970 msgr2=0x7fe350003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:47.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.056+0000 7fe35bfff640 1 --2- 172.21.15.84:0/4066388752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe350003970 0x7fe350003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.056+0000 7fe35bfff640 1 -- 172.21.15.84:0/4066388752 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe350003520 con 0x7fe3500047c0 2023-10-16T11:26:47.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.057+0000 7fe363fff640 1 -- 172.21.15.84:0/4066388752 reap_dead start 2023-10-16T11:26:47.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.058+0000 7fe35bfff640 1 --2- 172.21.15.84:0/4066388752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3500047c0 0x7fe350004c40 secure :-1 s=READY pgs=3220 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009a90 tx=0x7fe3500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9b76c8e8bd577 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:47.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.058+0000 7fe362ffd640 1 -- 172.21.15.84:0/4066388752 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c02c6b0 con 0x7fe3500047c0 2023-10-16T11:26:47.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.058+0000 7fe362ffd640 1 -- 172.21.15.84:0/4066388752 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c02c810 con 0x7fe3500047c0 2023-10-16T11:26:47.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.058+0000 7fe362ffd640 1 -- 172.21.15.84:0/4066388752 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c02cba0 con 0x7fe3500047c0 2023-10-16T11:26:47.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.058+0000 7fe369ba9640 1 -- 172.21.15.84:0/4066388752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3500047c0 msgr2=0x7fe350004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:47.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.058+0000 7fe369ba9640 1 --2- 172.21.15.84:0/4066388752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3500047c0 0x7fe350004c40 secure :-1 s=READY pgs=3220 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009a90 tx=0x7fe3500057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.059+0000 7fe369ba9640 1 -- 172.21.15.84:0/4066388752 shutdown_connections 2023-10-16T11:26:47.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.059+0000 7fe369ba9640 1 --2- 172.21.15.84:0/4066388752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3500047c0 0x7fe350004c40 unknown :-1 s=CLOSED pgs=3220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.059+0000 7fe369ba9640 1 -- 172.21.15.84:0/4066388752 >> 172.21.15.84:0/4066388752 conn(0x7fe3640cb410 msgr2=0x7fe3640cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:47.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.059+0000 7fe369ba9640 1 -- 172.21.15.84:0/4066388752 shutdown_connections 2023-10-16T11:26:47.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.059+0000 7fe369ba9640 1 -- 172.21.15.84:0/4066388752 wait complete. 2023-10-16T11:26:47.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.060+0000 7fe369ba9640 1 Processor -- start 2023-10-16T11:26:47.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.061+0000 7fe369ba9640 1 -- start start 2023-10-16T11:26:47.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.062+0000 7fe369ba9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3500047c0 0x7fe36414af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:47.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.062+0000 7fe369ba9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3641580d0 0x7fe3640cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:47.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.062+0000 7fe369ba9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3641e6160 0x7fe3641e65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:47.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.062+0000 7fe369ba9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3641e4f80 con 0x7fe3641e6160 2023-10-16T11:26:47.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.062+0000 7fe369ba9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3640cc9f0 con 0x7fe3500047c0 2023-10-16T11:26:47.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.062+0000 7fe369ba9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3641e6b20 con 0x7fe3641580d0 2023-10-16T11:26:47.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.062+0000 7fe363fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3641e6160 0x7fe3641e65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:47.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.062+0000 7fe363fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3641e6160 0x7fe3641e65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51602/0 (socket says 172.21.15.84:51602) 2023-10-16T11:26:47.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.062+0000 7fe363fff640 1 -- 172.21.15.84:0/3990890271 learned_addr learned my addr 172.21.15.84:0/3990890271 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:47.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.062+0000 7fe3637fe640 1 --2- 172.21.15.84:0/3990890271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3500047c0 0x7fe36414af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:47.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.062+0000 7fe35bfff640 1 --2- 172.21.15.84:0/3990890271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3641580d0 0x7fe3640cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:47.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.062+0000 7fe363fff640 1 -- 172.21.15.84:0/3990890271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3641580d0 msgr2=0x7fe3640cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:47.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.062+0000 7fe363fff640 1 --2- 172.21.15.84:0/3990890271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3641580d0 0x7fe3640cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.062+0000 7fe363fff640 1 -- 172.21.15.84:0/3990890271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3500047c0 msgr2=0x7fe36414af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:47.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.062+0000 7fe363fff640 1 --2- 172.21.15.84:0/3990890271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3500047c0 0x7fe36414af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.062+0000 7fe363fff640 1 -- 172.21.15.84:0/3990890271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3641e5a00 con 0x7fe3641e6160 2023-10-16T11:26:47.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.063+0000 7fe363fff640 1 --2- 172.21.15.84:0/3990890271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3641e6160 0x7fe3641e65e0 secure :-1 s=READY pgs=3221 cs=0 l=1 rev1=1 crypto rx=0x7fe35c000f80 tx=0x7fe35c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:47.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.063+0000 7fe360ff9640 1 -- 172.21.15.84:0/3990890271 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c00cea0 con 0x7fe3641e6160 2023-10-16T11:26:47.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.063+0000 7fe360ff9640 1 -- 172.21.15.84:0/3990890271 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c00a610 con 0x7fe3641e6160 2023-10-16T11:26:47.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.063+0000 7fe360ff9640 1 -- 172.21.15.84:0/3990890271 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c00a9a0 con 0x7fe3641e6160 2023-10-16T11:26:47.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.063+0000 7fe369ba9640 1 -- 172.21.15.84:0/3990890271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3641e5160 con 0x7fe3641e6160 2023-10-16T11:26:47.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.063+0000 7fe369ba9640 1 -- 172.21.15.84:0/3990890271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3641e5720 con 0x7fe3641e6160 2023-10-16T11:26:47.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.064+0000 7fe369ba9640 1 -- 172.21.15.84:0/3990890271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36404e160 con 0x7fe3641e6160 2023-10-16T11:26:47.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.065+0000 7fe360ff9640 1 -- 172.21.15.84:0/3990890271 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 151) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe35c00ab00 con 0x7fe3641e6160 2023-10-16T11:26:47.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.067+0000 7fe360ff9640 1 --2- 172.21.15.84:0/3990890271 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe34406b0e0 0x7fe34406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:47.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.067+0000 7fe360ff9640 1 -- 172.21.15.84:0/3990890271 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5780+0+0 (secure 0 0 0) 0x7fe35c089160 con 0x7fe3641e6160 2023-10-16T11:26:47.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.067+0000 7fe3637fe640 1 --2- 172.21.15.84:0/3990890271 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe34406b0e0 0x7fe34406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:47.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.068+0000 7fe3637fe640 1 --2- 172.21.15.84:0/3990890271 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe34406b0e0 0x7fe34406d5a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe354001650 tx=0x7fe354009040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:47.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.071+0000 7fe360ff9640 1 -- 172.21.15.84:0/3990890271 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe35c0577c0 con 0x7fe3641e6160 2023-10-16T11:26:47.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.259+0000 7fe369ba9640 1 -- 172.21.15.84:0/3990890271 --> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe36415f270 con 0x7fe34406b0e0 2023-10-16T11:26:47.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.286+0000 7fe360ff9640 1 -- 172.21.15.84:0/3990890271 <== mgr.12624 v2:172.21.15.84:6832/2330196255 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287426 (secure 0 0 0) 0x7fe3540050d0 con 0x7fe34406b0e0 2023-10-16T11:26:47.293 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:47.294 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:26:47.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.292+0000 7fe369ba9640 1 -- 172.21.15.84:0/3990890271 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe34406b0e0 msgr2=0x7fe34406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:47.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.292+0000 7fe369ba9640 1 --2- 172.21.15.84:0/3990890271 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe34406b0e0 0x7fe34406d5a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe354001650 tx=0x7fe354009040 comp rx=0 tx=0).stop 2023-10-16T11:26:47.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.292+0000 7fe369ba9640 1 -- 172.21.15.84:0/3990890271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3641e6160 msgr2=0x7fe3641e65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:47.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.292+0000 7fe369ba9640 1 --2- 172.21.15.84:0/3990890271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3641e6160 0x7fe3641e65e0 secure :-1 s=READY pgs=3221 cs=0 l=1 rev1=1 crypto rx=0x7fe35c000f80 tx=0x7fe35c00ccb0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.293+0000 7fe369ba9640 1 -- 172.21.15.84:0/3990890271 shutdown_connections 2023-10-16T11:26:47.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.293+0000 7fe369ba9640 1 --2- 172.21.15.84:0/3990890271 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fe34406b0e0 0x7fe34406d5a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.293+0000 7fe369ba9640 1 --2- 172.21.15.84:0/3990890271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3641e6160 0x7fe3641e65e0 unknown :-1 s=CLOSED pgs=3221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.293+0000 7fe369ba9640 1 --2- 172.21.15.84:0/3990890271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3641580d0 0x7fe3640cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.293+0000 7fe369ba9640 1 --2- 172.21.15.84:0/3990890271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3500047c0 0x7fe36414af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.293+0000 7fe369ba9640 1 -- 172.21.15.84:0/3990890271 >> 172.21.15.84:0/3990890271 conn(0x7fe3640cb410 msgr2=0x7fe36406cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:47.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.293+0000 7fe369ba9640 1 -- 172.21.15.84:0/3990890271 shutdown_connections 2023-10-16T11:26:47.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.293+0000 7fe369ba9640 1 -- 172.21.15.84:0/3990890271 wait complete. 2023-10-16T11:26:47.314 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-10-16T11:26:46.510759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2495,"num_read_kb":5117,"num_write":687,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4841,"ondisk_log_size":4841,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":455040,"kb_used_data":26564,"kb_used_omap":12,"kb_used_meta":428339,"kb_avail":754519680,"statfs":{"total":773094113280,"available":772628152320,"internally_reserved":0,"allocated":27201536,"data_stored":13895419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":438619705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":94,"num_read_kb":206,"num_write":64,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001770"},"pg_stats":[{"pgid":"35.3f","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.485834+0000","last_change":"2023-10-16T11:26:42.485177+0000","last_active":"2023-10-16T11:26:42.485834+0000","last_peered":"2023-10-16T11:26:42.485834+0000","last_clean":"2023-10-16T11:26:42.485834+0000","last_became_active":"2023-10-16T11:26:42.484764+0000","last_became_peered":"2023-10-16T11:26:42.484764+0000","last_unstale":"2023-10-16T11:26:42.485834+0000","last_undegraded":"2023-10-16T11:26:42.485834+0000","last_fullsized":"2023-10-16T11:26:42.485834+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:48:34.990016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.3e","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.448974+0000","last_change":"2023-10-16T11:26:42.477252+0000","last_active":"2023-10-16T11:26:43.448974+0000","last_peered":"2023-10-16T11:26:43.448974+0000","last_clean":"2023-10-16T11:26:43.448974+0000","last_became_active":"2023-10-16T11:26:42.476616+0000","last_became_peered":"2023-10-16T11:26:42.476616+0000","last_unstale":"2023-10-16T11:26:43.448974+0000","last_undegraded":"2023-10-16T11:26:43.448974+0000","last_fullsized":"2023-10-16T11:26:43.448974+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:48:35.570379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.3d","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.448731+0000","last_change":"2023-10-16T11:26:42.472467+0000","last_active":"2023-10-16T11:26:43.448731+0000","last_peered":"2023-10-16T11:26:43.448731+0000","last_clean":"2023-10-16T11:26:43.448731+0000","last_became_active":"2023-10-16T11:26:42.472067+0000","last_became_peered":"2023-10-16T11:26:42.472067+0000","last_unstale":"2023-10-16T11:26:43.448731+0000","last_undegraded":"2023-10-16T11:26:43.448731+0000","last_fullsized":"2023-10-16T11:26:43.448731+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:32.729558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.3c","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.488295+0000","last_change":"2023-10-16T11:26:42.473214+0000","last_active":"2023-10-16T11:26:42.488295+0000","last_peered":"2023-10-16T11:26:42.488295+0000","last_clean":"2023-10-16T11:26:42.488295+0000","last_became_active":"2023-10-16T11:26:42.472966+0000","last_became_peered":"2023-10-16T11:26:42.472966+0000","last_unstale":"2023-10-16T11:26:42.488295+0000","last_undegraded":"2023-10-16T11:26:42.488295+0000","last_fullsized":"2023-10-16T11:26:42.488295+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:53.818915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.3b","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.448246+0000","last_change":"2023-10-16T11:26:42.468516+0000","last_active":"2023-10-16T11:26:43.448246+0000","last_peered":"2023-10-16T11:26:43.448246+0000","last_clean":"2023-10-16T11:26:43.448246+0000","last_became_active":"2023-10-16T11:26:42.467300+0000","last_became_peered":"2023-10-16T11:26:42.467300+0000","last_unstale":"2023-10-16T11:26:43.448246+0000","last_undegraded":"2023-10-16T11:26:43.448246+0000","last_fullsized":"2023-10-16T11:26:43.448246+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:22:02.870114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.3a","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.448107+0000","last_change":"2023-10-16T11:26:42.468662+0000","last_active":"2023-10-16T11:26:43.448107+0000","last_peered":"2023-10-16T11:26:43.448107+0000","last_clean":"2023-10-16T11:26:43.448107+0000","last_became_active":"2023-10-16T11:26:42.466694+0000","last_became_peered":"2023-10-16T11:26:42.466694+0000","last_unstale":"2023-10-16T11:26:43.448107+0000","last_undegraded":"2023-10-16T11:26:43.448107+0000","last_fullsized":"2023-10-16T11:26:43.448107+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:11:12.445109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.39","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.013584+0000","last_change":"2023-10-16T11:26:42.498390+0000","last_active":"2023-10-16T11:26:44.013584+0000","last_peered":"2023-10-16T11:26:44.013584+0000","last_clean":"2023-10-16T11:26:44.013584+0000","last_became_active":"2023-10-16T11:26:42.497340+0000","last_became_peered":"2023-10-16T11:26:42.497340+0000","last_unstale":"2023-10-16T11:26:44.013584+0000","last_undegraded":"2023-10-16T11:26:44.013584+0000","last_fullsized":"2023-10-16T11:26:44.013584+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:07.920676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.38","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.013710+0000","last_change":"2023-10-16T11:26:42.483168+0000","last_active":"2023-10-16T11:26:44.013710+0000","last_peered":"2023-10-16T11:26:44.013710+0000","last_clean":"2023-10-16T11:26:44.013710+0000","last_became_active":"2023-10-16T11:26:42.481102+0000","last_became_peered":"2023-10-16T11:26:42.481102+0000","last_unstale":"2023-10-16T11:26:44.013710+0000","last_undegraded":"2023-10-16T11:26:44.013710+0000","last_fullsized":"2023-10-16T11:26:44.013710+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:55.860376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.37","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.446267+0000","last_change":"2023-10-16T11:26:42.460799+0000","last_active":"2023-10-16T11:26:43.446267+0000","last_peered":"2023-10-16T11:26:43.446267+0000","last_clean":"2023-10-16T11:26:43.446267+0000","last_became_active":"2023-10-16T11:26:42.460209+0000","last_became_peered":"2023-10-16T11:26:42.460209+0000","last_unstale":"2023-10-16T11:26:43.446267+0000","last_undegraded":"2023-10-16T11:26:43.446267+0000","last_fullsized":"2023-10-16T11:26:43.446267+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:52.907745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.36","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.488163+0000","last_change":"2023-10-16T11:26:42.480358+0000","last_active":"2023-10-16T11:26:42.488163+0000","last_peered":"2023-10-16T11:26:42.488163+0000","last_clean":"2023-10-16T11:26:42.488163+0000","last_became_active":"2023-10-16T11:26:42.480004+0000","last_became_peered":"2023-10-16T11:26:42.480004+0000","last_unstale":"2023-10-16T11:26:42.488163+0000","last_undegraded":"2023-10-16T11:26:42.488163+0000","last_fullsized":"2023-10-16T11:26:42.488163+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:56:20.149184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.35","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.057906+0000","last_change":"2023-10-16T11:26:42.489625+0000","last_active":"2023-10-16T11:26:44.057906+0000","last_peered":"2023-10-16T11:26:44.057906+0000","last_clean":"2023-10-16T11:26:44.057906+0000","last_became_active":"2023-10-16T11:26:42.488751+0000","last_became_peered":"2023-10-16T11:26:42.488751+0000","last_unstale":"2023-10-16T11:26:44.057906+0000","last_undegraded":"2023-10-16T11:26:44.057906+0000","last_fullsized":"2023-10-16T11:26:44.057906+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:05.823763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.34","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.479147+0000","last_change":"2023-10-16T11:26:42.477091+0000","last_active":"2023-10-16T11:26:42.479147+0000","last_peered":"2023-10-16T11:26:42.479147+0000","last_clean":"2023-10-16T11:26:42.479147+0000","last_became_active":"2023-10-16T11:26:42.476019+0000","last_became_peered":"2023-10-16T11:26:42.476019+0000","last_unstale":"2023-10-16T11:26:42.479147+0000","last_undegraded":"2023-10-16T11:26:42.479147+0000","last_fullsized":"2023-10-16T11:26:42.479147+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:44:41.622265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.33","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.481313+0000","last_change":"2023-10-16T11:26:42.478109+0000","last_active":"2023-10-16T11:26:42.481313+0000","last_peered":"2023-10-16T11:26:42.481313+0000","last_clean":"2023-10-16T11:26:42.481313+0000","last_became_active":"2023-10-16T11:26:42.477759+0000","last_became_peered":"2023-10-16T11:26:42.477759+0000","last_unstale":"2023-10-16T11:26:42.481313+0000","last_undegraded":"2023-10-16T11:26:42.481313+0000","last_fullsized":"2023-10-16T11:26:42.481313+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:40.299496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.32","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.494543+0000","last_change":"2023-10-16T11:26:42.491655+0000","last_active":"2023-10-16T11:26:42.494543+0000","last_peered":"2023-10-16T11:26:42.494543+0000","last_clean":"2023-10-16T11:26:42.494543+0000","last_became_active":"2023-10-16T11:26:42.491347+0000","last_became_peered":"2023-10-16T11:26:42.491347+0000","last_unstale":"2023-10-16T11:26:42.494543+0000","last_undegraded":"2023-10-16T11:26:42.494543+0000","last_fullsized":"2023-10-16T11:26:42.494543+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:39:35.732428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.31","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.013983+0000","last_change":"2023-10-16T11:26:42.479023+0000","last_active":"2023-10-16T11:26:44.013983+0000","last_peered":"2023-10-16T11:26:44.013983+0000","last_clean":"2023-10-16T11:26:44.013983+0000","last_became_active":"2023-10-16T11:26:42.478494+0000","last_became_peered":"2023-10-16T11:26:42.478494+0000","last_unstale":"2023-10-16T11:26:44.013983+0000","last_undegraded":"2023-10-16T11:26:44.013983+0000","last_fullsized":"2023-10-16T11:26:44.013983+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:02:44.250353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.30","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.013598+0000","last_change":"2023-10-16T11:26:42.474989+0000","last_active":"2023-10-16T11:26:44.013598+0000","last_peered":"2023-10-16T11:26:44.013598+0000","last_clean":"2023-10-16T11:26:44.013598+0000","last_became_active":"2023-10-16T11:26:42.474697+0000","last_became_peered":"2023-10-16T11:26:42.474697+0000","last_unstale":"2023-10-16T11:26:44.013598+0000","last_undegraded":"2023-10-16T11:26:44.013598+0000","last_fullsized":"2023-10-16T11:26:44.013598+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:11.816797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.2f","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.446435+0000","last_change":"2023-10-16T11:26:42.477897+0000","last_active":"2023-10-16T11:26:43.446435+0000","last_peered":"2023-10-16T11:26:43.446435+0000","last_clean":"2023-10-16T11:26:43.446435+0000","last_became_active":"2023-10-16T11:26:42.477574+0000","last_became_peered":"2023-10-16T11:26:42.477574+0000","last_unstale":"2023-10-16T11:26:43.446435+0000","last_undegraded":"2023-10-16T11:26:43.446435+0000","last_fullsized":"2023-10-16T11:26:43.446435+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:02.925578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.2e","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.488004+0000","last_change":"2023-10-16T11:26:42.482627+0000","last_active":"2023-10-16T11:26:42.488004+0000","last_peered":"2023-10-16T11:26:42.488004+0000","last_clean":"2023-10-16T11:26:42.488004+0000","last_became_active":"2023-10-16T11:26:42.482369+0000","last_became_peered":"2023-10-16T11:26:42.482369+0000","last_unstale":"2023-10-16T11:26:42.488004+0000","last_undegraded":"2023-10-16T11:26:42.488004+0000","last_fullsized":"2023-10-16T11:26:42.488004+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:35.685825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.2d","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.013675+0000","last_change":"2023-10-16T11:26:42.496202+0000","last_active":"2023-10-16T11:26:44.013675+0000","last_peered":"2023-10-16T11:26:44.013675+0000","last_clean":"2023-10-16T11:26:44.013675+0000","last_became_active":"2023-10-16T11:26:42.495910+0000","last_became_peered":"2023-10-16T11:26:42.495910+0000","last_unstale":"2023-10-16T11:26:44.013675+0000","last_undegraded":"2023-10-16T11:26:44.013675+0000","last_fullsized":"2023-10-16T11:26:44.013675+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:29:18.145001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.2c","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.013602+0000","last_change":"2023-10-16T11:26:42.483449+0000","last_active":"2023-10-16T11:26:44.013602+0000","last_peered":"2023-10-16T11:26:44.013602+0000","last_clean":"2023-10-16T11:26:44.013602+0000","last_became_active":"2023-10-16T11:26:42.482770+0000","last_became_peered":"2023-10-16T11:26:42.482770+0000","last_unstale":"2023-10-16T11:26:44.013602+0000","last_undegraded":"2023-10-16T11:26:44.013602+0000","last_fullsized":"2023-10-16T11:26:44.013602+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:36.348053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.2b","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.481696+0000","last_change":"2023-10-16T11:26:42.478965+0000","last_active":"2023-10-16T11:26:42.481696+0000","last_peered":"2023-10-16T11:26:42.481696+0000","last_clean":"2023-10-16T11:26:42.481696+0000","last_became_active":"2023-10-16T11:26:42.478648+0000","last_became_peered":"2023-10-16T11:26:42.478648+0000","last_unstale":"2023-10-16T11:26:42.481696+0000","last_undegraded":"2023-10-16T11:26:42.481696+0000","last_fullsized":"2023-10-16T11:26:42.481696+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:39:35.732428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.2a","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.448286+0000","last_change":"2023-10-16T11:26:42.487879+0000","last_active":"2023-10-16T11:26:43.448286+0000","last_peered":"2023-10-16T11:26:43.448286+0000","last_clean":"2023-10-16T11:26:43.448286+0000","last_became_active":"2023-10-16T11:26:42.487472+0000","last_became_peered":"2023-10-16T11:26:42.487472+0000","last_unstale":"2023-10-16T11:26:43.448286+0000","last_undegraded":"2023-10-16T11:26:43.448286+0000","last_fullsized":"2023-10-16T11:26:43.448286+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:52.117654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.29","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.449755+0000","last_change":"2023-10-16T11:26:42.480757+0000","last_active":"2023-10-16T11:26:43.449755+0000","last_peered":"2023-10-16T11:26:43.449755+0000","last_clean":"2023-10-16T11:26:43.449755+0000","last_became_active":"2023-10-16T11:26:42.480439+0000","last_became_peered":"2023-10-16T11:26:42.480439+0000","last_unstale":"2023-10-16T11:26:43.449755+0000","last_undegraded":"2023-10-16T11:26:43.449755+0000","last_fullsized":"2023-10-16T11:26:43.449755+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:25:33.957694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.28","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.447914+0000","last_change":"2023-10-16T11:26:42.493819+0000","last_active":"2023-10-16T11:26:43.447914+0000","last_peered":"2023-10-16T11:26:43.447914+0000","last_clean":"2023-10-16T11:26:43.447914+0000","last_became_active":"2023-10-16T11:26:42.493540+0000","last_became_peered":"2023-10-16T11:26:42.493540+0000","last_unstale":"2023-10-16T11:26:43.447914+0000","last_undegraded":"2023-10-16T11:26:43.447914+0000","last_fullsized":"2023-10-16T11:26:43.447914+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:34:39.047026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.27","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.450341+0000","last_change":"2023-10-16T11:26:42.478717+0000","last_active":"2023-10-16T11:26:43.450341+0000","last_peered":"2023-10-16T11:26:43.450341+0000","last_clean":"2023-10-16T11:26:43.450341+0000","last_became_active":"2023-10-16T11:26:42.478340+0000","last_became_peered":"2023-10-16T11:26:42.478340+0000","last_unstale":"2023-10-16T11:26:43.450341+0000","last_undegraded":"2023-10-16T11:26:43.450341+0000","last_fullsized":"2023-10-16T11:26:43.450341+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:28.887145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.26","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.481376+0000","last_change":"2023-10-16T11:26:42.477429+0000","last_active":"2023-10-16T11:26:42.481376+0000","last_peered":"2023-10-16T11:26:42.481376+0000","last_clean":"2023-10-16T11:26:42.481376+0000","last_became_active":"2023-10-16T11:26:42.475792+0000","last_became_peered":"2023-10-16T11:26:42.475792+0000","last_unstale":"2023-10-16T11:26:42.481376+0000","last_undegraded":"2023-10-16T11:26:42.481376+0000","last_fullsized":"2023-10-16T11:26:42.481376+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:34:50.863297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.25","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.014068+0000","last_change":"2023-10-16T11:26:42.495035+0000","last_active":"2023-10-16T11:26:44.014068+0000","last_peered":"2023-10-16T11:26:44.014068+0000","last_clean":"2023-10-16T11:26:44.014068+0000","last_became_active":"2023-10-16T11:26:42.491700+0000","last_became_peered":"2023-10-16T11:26:42.491700+0000","last_unstale":"2023-10-16T11:26:44.014068+0000","last_undegraded":"2023-10-16T11:26:44.014068+0000","last_fullsized":"2023-10-16T11:26:44.014068+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:45.250424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.24","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.465251+0000","last_change":"2023-10-16T11:26:42.459535+0000","last_active":"2023-10-16T11:26:42.465251+0000","last_peered":"2023-10-16T11:26:42.465251+0000","last_clean":"2023-10-16T11:26:42.465251+0000","last_became_active":"2023-10-16T11:26:42.458870+0000","last_became_peered":"2023-10-16T11:26:42.458870+0000","last_unstale":"2023-10-16T11:26:42.465251+0000","last_undegraded":"2023-10-16T11:26:42.465251+0000","last_fullsized":"2023-10-16T11:26:42.465251+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:52:06.837078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.23","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.014022+0000","last_change":"2023-10-16T11:26:42.496413+0000","last_active":"2023-10-16T11:26:44.014022+0000","last_peered":"2023-10-16T11:26:44.014022+0000","last_clean":"2023-10-16T11:26:44.014022+0000","last_became_active":"2023-10-16T11:26:42.496103+0000","last_became_peered":"2023-10-16T11:26:42.496103+0000","last_unstale":"2023-10-16T11:26:44.014022+0000","last_undegraded":"2023-10-16T11:26:44.014022+0000","last_fullsized":"2023-10-16T11:26:44.014022+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:14.531480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.21","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.014313+0000","last_change":"2023-10-16T11:26:42.497225+0000","last_active":"2023-10-16T11:26:44.014313+0000","last_peered":"2023-10-16T11:26:44.014313+0000","last_clean":"2023-10-16T11:26:44.014313+0000","last_became_active":"2023-10-16T11:26:42.496582+0000","last_became_peered":"2023-10-16T11:26:42.496582+0000","last_unstale":"2023-10-16T11:26:44.014313+0000","last_undegraded":"2023-10-16T11:26:44.014313+0000","last_fullsized":"2023-10-16T11:26:44.014313+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:19:15.546077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.20","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.445526+0000","last_change":"2023-10-16T11:26:42.458807+0000","last_active":"2023-10-16T11:26:43.445526+0000","last_peered":"2023-10-16T11:26:43.445526+0000","last_clean":"2023-10-16T11:26:43.445526+0000","last_became_active":"2023-10-16T11:26:42.457861+0000","last_became_peered":"2023-10-16T11:26:42.457861+0000","last_unstale":"2023-10-16T11:26:43.445526+0000","last_undegraded":"2023-10-16T11:26:43.445526+0000","last_fullsized":"2023-10-16T11:26:43.445526+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:18:41.473131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1e","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.494383+0000","last_change":"2023-10-16T11:26:42.492923+0000","last_active":"2023-10-16T11:26:42.494383+0000","last_peered":"2023-10-16T11:26:42.494383+0000","last_clean":"2023-10-16T11:26:42.494383+0000","last_became_active":"2023-10-16T11:26:42.491679+0000","last_became_peered":"2023-10-16T11:26:42.491679+0000","last_unstale":"2023-10-16T11:26:42.494383+0000","last_undegraded":"2023-10-16T11:26:42.494383+0000","last_fullsized":"2023-10-16T11:26:42.494383+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:56:20.149184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1f","version":"256'4","reported_seq":15,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.495620+0000","last_change":"2023-10-16T11:26:41.472041+0000","last_active":"2023-10-16T11:26:42.495620+0000","last_peered":"2023-10-16T11:26:42.495620+0000","last_clean":"2023-10-16T11:26:42.495620+0000","last_became_active":"2023-10-16T11:26:41.471494+0000","last_became_peered":"2023-10-16T11:26:41.471494+0000","last_unstale":"2023-10-16T11:26:42.495620+0000","last_undegraded":"2023-10-16T11:26:42.495620+0000","last_fullsized":"2023-10-16T11:26:42.495620+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:33.468817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1f","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.058325+0000","last_change":"2023-10-16T11:26:42.485850+0000","last_active":"2023-10-16T11:26:44.058325+0000","last_peered":"2023-10-16T11:26:44.058325+0000","last_clean":"2023-10-16T11:26:44.058325+0000","last_became_active":"2023-10-16T11:26:42.485492+0000","last_became_peered":"2023-10-16T11:26:42.485492+0000","last_unstale":"2023-10-16T11:26:44.058325+0000","last_undegraded":"2023-10-16T11:26:44.058325+0000","last_fullsized":"2023-10-16T11:26:44.058325+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:54:36.826735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.1e","version":"256'1","reported_seq":13,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.493251+0000","last_change":"2023-10-16T11:26:41.463055+0000","last_active":"2023-10-16T11:26:42.493251+0000","last_peered":"2023-10-16T11:26:42.493251+0000","last_clean":"2023-10-16T11:26:42.493251+0000","last_became_active":"2023-10-16T11:26:41.462240+0000","last_became_peered":"2023-10-16T11:26:41.462240+0000","last_unstale":"2023-10-16T11:26:42.493251+0000","last_undegraded":"2023-10-16T11:26:42.493251+0000","last_fullsized":"2023-10-16T11:26:42.493251+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:48:33.977514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1c","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.448426+0000","last_change":"2023-10-16T11:26:42.461151+0000","last_active":"2023-10-16T11:26:43.448426+0000","last_peered":"2023-10-16T11:26:43.448426+0000","last_clean":"2023-10-16T11:26:43.448426+0000","last_became_active":"2023-10-16T11:26:42.460058+0000","last_became_peered":"2023-10-16T11:26:42.460058+0000","last_unstale":"2023-10-16T11:26:43.448426+0000","last_undegraded":"2023-10-16T11:26:43.448426+0000","last_fullsized":"2023-10-16T11:26:43.448426+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:02:25.324063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1d","version":"256'2","reported_seq":14,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.496583+0000","last_change":"2023-10-16T11:26:41.472513+0000","last_active":"2023-10-16T11:26:42.496583+0000","last_peered":"2023-10-16T11:26:42.496583+0000","last_clean":"2023-10-16T11:26:42.496583+0000","last_became_active":"2023-10-16T11:26:41.472233+0000","last_became_peered":"2023-10-16T11:26:41.472233+0000","last_unstale":"2023-10-16T11:26:42.496583+0000","last_undegraded":"2023-10-16T11:26:42.496583+0000","last_fullsized":"2023-10-16T11:26:42.496583+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:44:40.609763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1d","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.448887+0000","last_change":"2023-10-16T11:26:42.461915+0000","last_active":"2023-10-16T11:26:43.448887+0000","last_peered":"2023-10-16T11:26:43.448887+0000","last_clean":"2023-10-16T11:26:43.448887+0000","last_became_active":"2023-10-16T11:26:42.461183+0000","last_became_peered":"2023-10-16T11:26:42.461183+0000","last_unstale":"2023-10-16T11:26:43.448887+0000","last_undegraded":"2023-10-16T11:26:43.448887+0000","last_fullsized":"2023-10-16T11:26:43.448887+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:54:30.752990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"256'2","reported_seq":14,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.502802+0000","last_change":"2023-10-16T11:26:41.461721+0000","last_active":"2023-10-16T11:26:42.502802+0000","last_peered":"2023-10-16T11:26:42.502802+0000","last_clean":"2023-10-16T11:26:42.502802+0000","last_became_active":"2023-10-16T11:26:41.461381+0000","last_became_peered":"2023-10-16T11:26:41.461381+0000","last_unstale":"2023-10-16T11:26:42.502802+0000","last_undegraded":"2023-10-16T11:26:42.502802+0000","last_fullsized":"2023-10-16T11:26:42.502802+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:01.913076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.1a","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.450069+0000","last_change":"2023-10-16T11:26:42.472572+0000","last_active":"2023-10-16T11:26:43.450069+0000","last_peered":"2023-10-16T11:26:43.450069+0000","last_clean":"2023-10-16T11:26:43.450069+0000","last_became_active":"2023-10-16T11:26:42.472143+0000","last_became_peered":"2023-10-16T11:26:42.472143+0000","last_unstale":"2023-10-16T11:26:43.450069+0000","last_undegraded":"2023-10-16T11:26:43.450069+0000","last_fullsized":"2023-10-16T11:26:43.450069+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:32:11.353644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1b","version":"256'1","reported_seq":13,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.498815+0000","last_change":"2023-10-16T11:26:41.466268+0000","last_active":"2023-10-16T11:26:42.498815+0000","last_peered":"2023-10-16T11:26:42.498815+0000","last_clean":"2023-10-16T11:26:42.498815+0000","last_became_active":"2023-10-16T11:26:41.465918+0000","last_became_peered":"2023-10-16T11:26:41.465918+0000","last_unstale":"2023-10-16T11:26:42.498815+0000","last_undegraded":"2023-10-16T11:26:42.498815+0000","last_fullsized":"2023-10-16T11:26:42.498815+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:15.325814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.1b","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.014413+0000","last_change":"2023-10-16T11:26:42.482948+0000","last_active":"2023-10-16T11:26:44.014413+0000","last_peered":"2023-10-16T11:26:44.014413+0000","last_clean":"2023-10-16T11:26:44.014413+0000","last_became_active":"2023-10-16T11:26:42.481415+0000","last_became_peered":"2023-10-16T11:26:42.481415+0000","last_unstale":"2023-10-16T11:26:44.014413+0000","last_undegraded":"2023-10-16T11:26:44.014413+0000","last_fullsized":"2023-10-16T11:26:44.014413+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:22:26.931814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1a","version":"256'1","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.467576+0000","last_change":"2023-10-16T11:26:41.470192+0000","last_active":"2023-10-16T11:26:42.467576+0000","last_peered":"2023-10-16T11:26:42.467576+0000","last_clean":"2023-10-16T11:26:42.467576+0000","last_became_active":"2023-10-16T11:26:41.469882+0000","last_became_peered":"2023-10-16T11:26:41.469882+0000","last_unstale":"2023-10-16T11:26:42.467576+0000","last_undegraded":"2023-10-16T11:26:42.467576+0000","last_fullsized":"2023-10-16T11:26:42.467576+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:13.411479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.18","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.445598+0000","last_change":"2023-10-16T11:26:42.462722+0000","last_active":"2023-10-16T11:26:43.445598+0000","last_peered":"2023-10-16T11:26:43.445598+0000","last_clean":"2023-10-16T11:26:43.445598+0000","last_became_active":"2023-10-16T11:26:42.459747+0000","last_became_peered":"2023-10-16T11:26:42.459747+0000","last_unstale":"2023-10-16T11:26:43.445598+0000","last_undegraded":"2023-10-16T11:26:43.445598+0000","last_fullsized":"2023-10-16T11:26:43.445598+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:34.481319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.474658+0000","last_change":"2023-10-16T11:26:41.473722+0000","last_active":"2023-10-16T11:26:42.474658+0000","last_peered":"2023-10-16T11:26:42.474658+0000","last_clean":"2023-10-16T11:26:42.474658+0000","last_became_active":"2023-10-16T11:26:41.473402+0000","last_became_peered":"2023-10-16T11:26:41.473402+0000","last_unstale":"2023-10-16T11:26:42.474658+0000","last_undegraded":"2023-10-16T11:26:42.474658+0000","last_fullsized":"2023-10-16T11:26:42.474658+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:12:53.432070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.19","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.487717+0000","last_change":"2023-10-16T11:26:42.476892+0000","last_active":"2023-10-16T11:26:42.487717+0000","last_peered":"2023-10-16T11:26:42.487717+0000","last_clean":"2023-10-16T11:26:42.487717+0000","last_became_active":"2023-10-16T11:26:42.476613+0000","last_became_peered":"2023-10-16T11:26:42.476613+0000","last_unstale":"2023-10-16T11:26:42.487717+0000","last_undegraded":"2023-10-16T11:26:42.487717+0000","last_fullsized":"2023-10-16T11:26:42.487717+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:07.002504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.470425+0000","last_change":"2023-10-16T11:26:41.461932+0000","last_active":"2023-10-16T11:26:42.470425+0000","last_peered":"2023-10-16T11:26:42.470425+0000","last_clean":"2023-10-16T11:26:42.470425+0000","last_became_active":"2023-10-16T11:26:41.461519+0000","last_became_peered":"2023-10-16T11:26:41.461519+0000","last_unstale":"2023-10-16T11:26:42.470425+0000","last_undegraded":"2023-10-16T11:26:42.470425+0000","last_fullsized":"2023-10-16T11:26:42.470425+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:05:59.649006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.16","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.487894+0000","last_change":"2023-10-16T11:26:42.477778+0000","last_active":"2023-10-16T11:26:42.487894+0000","last_peered":"2023-10-16T11:26:42.487894+0000","last_clean":"2023-10-16T11:26:42.487894+0000","last_became_active":"2023-10-16T11:26:42.474986+0000","last_became_peered":"2023-10-16T11:26:42.474986+0000","last_unstale":"2023-10-16T11:26:42.487894+0000","last_undegraded":"2023-10-16T11:26:42.487894+0000","last_fullsized":"2023-10-16T11:26:42.487894+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:13.852158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.452129+0000","last_change":"2023-10-16T11:26:41.470674+0000","last_active":"2023-10-16T11:26:42.452129+0000","last_peered":"2023-10-16T11:26:42.452129+0000","last_clean":"2023-10-16T11:26:42.452129+0000","last_became_active":"2023-10-16T11:26:41.470381+0000","last_became_peered":"2023-10-16T11:26:41.470381+0000","last_unstale":"2023-10-16T11:26:42.452129+0000","last_undegraded":"2023-10-16T11:26:42.452129+0000","last_fullsized":"2023-10-16T11:26:42.452129+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:17.143386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.17","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.488152+0000","last_change":"2023-10-16T11:26:42.482653+0000","last_active":"2023-10-16T11:26:42.488152+0000","last_peered":"2023-10-16T11:26:42.488152+0000","last_clean":"2023-10-16T11:26:42.488152+0000","last_became_active":"2023-10-16T11:26:42.482384+0000","last_became_peered":"2023-10-16T11:26:42.482384+0000","last_unstale":"2023-10-16T11:26:42.488152+0000","last_undegraded":"2023-10-16T11:26:42.488152+0000","last_fullsized":"2023-10-16T11:26:42.488152+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:17:36.001896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.463153+0000","last_change":"2023-10-16T11:26:41.464349+0000","last_active":"2023-10-16T11:26:42.463153+0000","last_peered":"2023-10-16T11:26:42.463153+0000","last_clean":"2023-10-16T11:26:42.463153+0000","last_became_active":"2023-10-16T11:26:41.464050+0000","last_became_peered":"2023-10-16T11:26:41.464050+0000","last_unstale":"2023-10-16T11:26:42.463153+0000","last_undegraded":"2023-10-16T11:26:42.463153+0000","last_fullsized":"2023-10-16T11:26:42.463153+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:44:19.380476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.14","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.058038+0000","last_change":"2023-10-16T11:26:42.479550+0000","last_active":"2023-10-16T11:26:44.058038+0000","last_peered":"2023-10-16T11:26:44.058038+0000","last_clean":"2023-10-16T11:26:44.058038+0000","last_became_active":"2023-10-16T11:26:42.479264+0000","last_became_peered":"2023-10-16T11:26:42.479264+0000","last_unstale":"2023-10-16T11:26:44.058038+0000","last_undegraded":"2023-10-16T11:26:44.058038+0000","last_fullsized":"2023-10-16T11:26:44.058038+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:23.444814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.15","version":"256'3","reported_seq":15,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.499634+0000","last_change":"2023-10-16T11:26:41.455346+0000","last_active":"2023-10-16T11:26:42.499634+0000","last_peered":"2023-10-16T11:26:42.499634+0000","last_clean":"2023-10-16T11:26:42.499634+0000","last_became_active":"2023-10-16T11:26:41.455005+0000","last_became_peered":"2023-10-16T11:26:41.455005+0000","last_unstale":"2023-10-16T11:26:42.499634+0000","last_undegraded":"2023-10-16T11:26:42.499634+0000","last_fullsized":"2023-10-16T11:26:42.499634+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:39:34.719926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.15","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.477208+0000","last_change":"2023-10-16T11:26:42.472647+0000","last_active":"2023-10-16T11:26:42.477208+0000","last_peered":"2023-10-16T11:26:42.477208+0000","last_clean":"2023-10-16T11:26:42.477208+0000","last_became_active":"2023-10-16T11:26:42.471549+0000","last_became_peered":"2023-10-16T11:26:42.471549+0000","last_unstale":"2023-10-16T11:26:42.477208+0000","last_undegraded":"2023-10-16T11:26:42.477208+0000","last_fullsized":"2023-10-16T11:26:42.477208+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:44:41.622265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.14","version":"256'6","reported_seq":18,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.501699+0000","last_change":"2023-10-16T11:26:41.469567+0000","last_active":"2023-10-16T11:26:42.501699+0000","last_peered":"2023-10-16T11:26:42.501699+0000","last_clean":"2023-10-16T11:26:42.501699+0000","last_became_active":"2023-10-16T11:26:41.469190+0000","last_became_peered":"2023-10-16T11:26:41.469190+0000","last_unstale":"2023-10-16T11:26:42.501699+0000","last_undegraded":"2023-10-16T11:26:42.501699+0000","last_fullsized":"2023-10-16T11:26:42.501699+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:50:17.705254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.12","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.014304+0000","last_change":"2023-10-16T11:26:42.482507+0000","last_active":"2023-10-16T11:26:44.014304+0000","last_peered":"2023-10-16T11:26:44.014304+0000","last_clean":"2023-10-16T11:26:44.014304+0000","last_became_active":"2023-10-16T11:26:42.480558+0000","last_became_peered":"2023-10-16T11:26:42.480558+0000","last_unstale":"2023-10-16T11:26:44.014304+0000","last_undegraded":"2023-10-16T11:26:44.014304+0000","last_fullsized":"2023-10-16T11:26:44.014304+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:09.387747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.13","version":"256'2","reported_seq":14,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.498622+0000","last_change":"2023-10-16T11:26:41.459360+0000","last_active":"2023-10-16T11:26:42.498622+0000","last_peered":"2023-10-16T11:26:42.498622+0000","last_clean":"2023-10-16T11:26:42.498622+0000","last_became_active":"2023-10-16T11:26:41.459076+0000","last_became_peered":"2023-10-16T11:26:41.459076+0000","last_unstale":"2023-10-16T11:26:42.498622+0000","last_undegraded":"2023-10-16T11:26:42.498622+0000","last_fullsized":"2023-10-16T11:26:42.498622+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:21.456069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.13","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.492101+0000","last_change":"2023-10-16T11:26:42.488436+0000","last_active":"2023-10-16T11:26:42.492101+0000","last_peered":"2023-10-16T11:26:42.492101+0000","last_clean":"2023-10-16T11:26:42.492101+0000","last_became_active":"2023-10-16T11:26:42.488148+0000","last_became_peered":"2023-10-16T11:26:42.488148+0000","last_unstale":"2023-10-16T11:26:42.492101+0000","last_undegraded":"2023-10-16T11:26:42.492101+0000","last_fullsized":"2023-10-16T11:26:42.492101+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:05:35.518664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.466683+0000","last_change":"2023-10-16T11:26:41.473074+0000","last_active":"2023-10-16T11:26:42.466683+0000","last_peered":"2023-10-16T11:26:42.466683+0000","last_clean":"2023-10-16T11:26:42.466683+0000","last_became_active":"2023-10-16T11:26:41.472333+0000","last_became_peered":"2023-10-16T11:26:41.472333+0000","last_unstale":"2023-10-16T11:26:42.466683+0000","last_undegraded":"2023-10-16T11:26:42.466683+0000","last_fullsized":"2023-10-16T11:26:42.466683+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:04:26.707239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.10","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.057265+0000","last_change":"2023-10-16T11:26:42.489797+0000","last_active":"2023-10-16T11:26:44.057265+0000","last_peered":"2023-10-16T11:26:44.057265+0000","last_clean":"2023-10-16T11:26:44.057265+0000","last_became_active":"2023-10-16T11:26:42.489497+0000","last_became_peered":"2023-10-16T11:26:42.489497+0000","last_unstale":"2023-10-16T11:26:44.057265+0000","last_undegraded":"2023-10-16T11:26:44.057265+0000","last_fullsized":"2023-10-16T11:26:44.057265+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:33:35.976291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.449349+0000","last_change":"2023-10-16T11:26:41.469759+0000","last_active":"2023-10-16T11:26:42.449349+0000","last_peered":"2023-10-16T11:26:42.449349+0000","last_clean":"2023-10-16T11:26:42.449349+0000","last_became_active":"2023-10-16T11:26:41.469472+0000","last_became_peered":"2023-10-16T11:26:41.469472+0000","last_unstale":"2023-10-16T11:26:42.449349+0000","last_undegraded":"2023-10-16T11:26:42.449349+0000","last_fullsized":"2023-10-16T11:26:42.449349+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:48:33.977514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.11","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.014508+0000","last_change":"2023-10-16T11:26:42.478827+0000","last_active":"2023-10-16T11:26:44.014508+0000","last_peered":"2023-10-16T11:26:44.014508+0000","last_clean":"2023-10-16T11:26:44.014508+0000","last_became_active":"2023-10-16T11:26:42.478081+0000","last_became_peered":"2023-10-16T11:26:42.478081+0000","last_unstale":"2023-10-16T11:26:44.014508+0000","last_undegraded":"2023-10-16T11:26:44.014508+0000","last_fullsized":"2023-10-16T11:26:44.014508+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:23:23.981006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.10","version":"256'8","reported_seq":20,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.501896+0000","last_change":"2023-10-16T11:26:41.465360+0000","last_active":"2023-10-16T11:26:42.501896+0000","last_peered":"2023-10-16T11:26:42.501896+0000","last_clean":"2023-10-16T11:26:42.501896+0000","last_became_active":"2023-10-16T11:26:41.465065+0000","last_became_peered":"2023-10-16T11:26:41.465065+0000","last_unstale":"2023-10-16T11:26:42.501896+0000","last_undegraded":"2023-10-16T11:26:42.501896+0000","last_fullsized":"2023-10-16T11:26:42.501896+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:35:26.885012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.e","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.450025+0000","last_change":"2023-10-16T11:26:42.470088+0000","last_active":"2023-10-16T11:26:43.450025+0000","last_peered":"2023-10-16T11:26:43.450025+0000","last_clean":"2023-10-16T11:26:43.450025+0000","last_became_active":"2023-10-16T11:26:42.469799+0000","last_became_peered":"2023-10-16T11:26:42.469799+0000","last_unstale":"2023-10-16T11:26:43.450025+0000","last_undegraded":"2023-10-16T11:26:43.450025+0000","last_fullsized":"2023-10-16T11:26:43.450025+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:54:49.226718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"256'1","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.451515+0000","last_change":"2023-10-16T11:26:41.470130+0000","last_active":"2023-10-16T11:26:42.451515+0000","last_peered":"2023-10-16T11:26:42.451515+0000","last_clean":"2023-10-16T11:26:42.451515+0000","last_became_active":"2023-10-16T11:26:41.469725+0000","last_became_peered":"2023-10-16T11:26:41.469725+0000","last_unstale":"2023-10-16T11:26:42.451515+0000","last_undegraded":"2023-10-16T11:26:42.451515+0000","last_fullsized":"2023-10-16T11:26:42.451515+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:12:53.432070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.f","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.448743+0000","last_change":"2023-10-16T11:26:42.488062+0000","last_active":"2023-10-16T11:26:43.448743+0000","last_peered":"2023-10-16T11:26:43.448743+0000","last_clean":"2023-10-16T11:26:43.448743+0000","last_became_active":"2023-10-16T11:26:42.487772+0000","last_became_peered":"2023-10-16T11:26:42.487772+0000","last_unstale":"2023-10-16T11:26:43.448743+0000","last_undegraded":"2023-10-16T11:26:43.448743+0000","last_fullsized":"2023-10-16T11:26:43.448743+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:58:06.446678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.e","version":"256'2","reported_seq":14,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.493684+0000","last_change":"2023-10-16T11:26:41.473393+0000","last_active":"2023-10-16T11:26:42.493684+0000","last_peered":"2023-10-16T11:26:42.493684+0000","last_clean":"2023-10-16T11:26:42.493684+0000","last_became_active":"2023-10-16T11:26:41.473133+0000","last_became_peered":"2023-10-16T11:26:41.473133+0000","last_unstale":"2023-10-16T11:26:42.493684+0000","last_undegraded":"2023-10-16T11:26:42.493684+0000","last_fullsized":"2023-10-16T11:26:42.493684+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:01:53.010279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.c","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.488666+0000","last_change":"2023-10-16T11:26:42.483173+0000","last_active":"2023-10-16T11:26:42.488666+0000","last_peered":"2023-10-16T11:26:42.488666+0000","last_clean":"2023-10-16T11:26:42.488666+0000","last_became_active":"2023-10-16T11:26:42.482420+0000","last_became_peered":"2023-10-16T11:26:42.482420+0000","last_unstale":"2023-10-16T11:26:42.488666+0000","last_undegraded":"2023-10-16T11:26:42.488666+0000","last_fullsized":"2023-10-16T11:26:42.488666+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:46:50.175836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.d","version":"256'1","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.448859+0000","last_change":"2023-10-16T11:26:41.472146+0000","last_active":"2023-10-16T11:26:42.448859+0000","last_peered":"2023-10-16T11:26:42.448859+0000","last_clean":"2023-10-16T11:26:42.448859+0000","last_became_active":"2023-10-16T11:26:41.471866+0000","last_became_peered":"2023-10-16T11:26:41.471866+0000","last_unstale":"2023-10-16T11:26:42.448859+0000","last_undegraded":"2023-10-16T11:26:42.448859+0000","last_fullsized":"2023-10-16T11:26:42.448859+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:39.286994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.d","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.449033+0000","last_change":"2023-10-16T11:26:42.476458+0000","last_active":"2023-10-16T11:26:43.449033+0000","last_peered":"2023-10-16T11:26:43.449033+0000","last_clean":"2023-10-16T11:26:43.449033+0000","last_became_active":"2023-10-16T11:26:42.476137+0000","last_became_peered":"2023-10-16T11:26:42.476137+0000","last_unstale":"2023-10-16T11:26:43.449033+0000","last_undegraded":"2023-10-16T11:26:43.449033+0000","last_fullsized":"2023-10-16T11:26:43.449033+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:49.829385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.448474+0000","last_change":"2023-10-16T11:26:41.474290+0000","last_active":"2023-10-16T11:26:42.448474+0000","last_peered":"2023-10-16T11:26:42.448474+0000","last_clean":"2023-10-16T11:26:42.448474+0000","last_became_active":"2023-10-16T11:26:41.473903+0000","last_became_peered":"2023-10-16T11:26:41.473903+0000","last_unstale":"2023-10-16T11:26:42.448474+0000","last_undegraded":"2023-10-16T11:26:42.448474+0000","last_fullsized":"2023-10-16T11:26:42.448474+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:34:49.850795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.a","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.445770+0000","last_change":"2023-10-16T11:26:42.461725+0000","last_active":"2023-10-16T11:26:43.445770+0000","last_peered":"2023-10-16T11:26:43.445770+0000","last_clean":"2023-10-16T11:26:43.445770+0000","last_became_active":"2023-10-16T11:26:42.458470+0000","last_became_peered":"2023-10-16T11:26:42.458470+0000","last_unstale":"2023-10-16T11:26:43.445770+0000","last_undegraded":"2023-10-16T11:26:43.445770+0000","last_fullsized":"2023-10-16T11:26:43.445770+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:47:31.251808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.b","version":"256'2","reported_seq":13,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.493775+0000","last_change":"2023-10-16T11:26:41.454416+0000","last_active":"2023-10-16T11:26:42.493775+0000","last_peered":"2023-10-16T11:26:42.493775+0000","last_clean":"2023-10-16T11:26:42.493775+0000","last_became_active":"2023-10-16T11:26:41.454134+0000","last_became_peered":"2023-10-16T11:26:41.454134+0000","last_unstale":"2023-10-16T11:26:42.493775+0000","last_undegraded":"2023-10-16T11:26:42.493775+0000","last_fullsized":"2023-10-16T11:26:42.493775+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:18:40.460629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.b","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.487549+0000","last_change":"2023-10-16T11:26:42.476361+0000","last_active":"2023-10-16T11:26:42.487549+0000","last_peered":"2023-10-16T11:26:42.487549+0000","last_clean":"2023-10-16T11:26:42.487549+0000","last_became_active":"2023-10-16T11:26:42.475996+0000","last_became_peered":"2023-10-16T11:26:42.475996+0000","last_unstale":"2023-10-16T11:26:42.487549+0000","last_undegraded":"2023-10-16T11:26:42.487549+0000","last_fullsized":"2023-10-16T11:26:42.487549+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:10:29.358302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.454470+0000","last_change":"2023-10-16T11:26:41.471717+0000","last_active":"2023-10-16T11:26:42.454470+0000","last_peered":"2023-10-16T11:26:42.454470+0000","last_clean":"2023-10-16T11:26:42.454470+0000","last_became_active":"2023-10-16T11:26:41.471354+0000","last_became_peered":"2023-10-16T11:26:41.471354+0000","last_unstale":"2023-10-16T11:26:42.454470+0000","last_undegraded":"2023-10-16T11:26:42.454470+0000","last_fullsized":"2023-10-16T11:26:42.454470+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:05:53.729578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.8","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.492444+0000","last_change":"2023-10-16T11:26:42.490549+0000","last_active":"2023-10-16T11:26:42.492444+0000","last_peered":"2023-10-16T11:26:42.492444+0000","last_clean":"2023-10-16T11:26:42.492444+0000","last_became_active":"2023-10-16T11:26:42.489090+0000","last_became_peered":"2023-10-16T11:26:42.489090+0000","last_unstale":"2023-10-16T11:26:42.492444+0000","last_undegraded":"2023-10-16T11:26:42.492444+0000","last_fullsized":"2023-10-16T11:26:42.492444+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:18:47.880771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.449418+0000","last_change":"2023-10-16T11:26:41.473560+0000","last_active":"2023-10-16T11:26:42.449418+0000","last_peered":"2023-10-16T11:26:42.449418+0000","last_clean":"2023-10-16T11:26:42.449418+0000","last_became_active":"2023-10-16T11:26:41.472581+0000","last_became_peered":"2023-10-16T11:26:41.472581+0000","last_unstale":"2023-10-16T11:26:42.449418+0000","last_undegraded":"2023-10-16T11:26:42.449418+0000","last_fullsized":"2023-10-16T11:26:42.449418+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:05:59.649006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.9","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.484248+0000","last_change":"2023-10-16T11:26:42.483715+0000","last_active":"2023-10-16T11:26:42.484248+0000","last_peered":"2023-10-16T11:26:42.484248+0000","last_clean":"2023-10-16T11:26:42.484248+0000","last_became_active":"2023-10-16T11:26:42.483451+0000","last_became_peered":"2023-10-16T11:26:42.483451+0000","last_unstale":"2023-10-16T11:26:42.484248+0000","last_undegraded":"2023-10-16T11:26:42.484248+0000","last_fullsized":"2023-10-16T11:26:42.484248+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:53.818915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.470635+0000","last_change":"2023-10-16T11:26:41.466050+0000","last_active":"2023-10-16T11:26:42.470635+0000","last_peered":"2023-10-16T11:26:42.470635+0000","last_clean":"2023-10-16T11:26:42.470635+0000","last_became_active":"2023-10-16T11:26:41.465785+0000","last_became_peered":"2023-10-16T11:26:41.465785+0000","last_unstale":"2023-10-16T11:26:42.470635+0000","last_undegraded":"2023-10-16T11:26:42.470635+0000","last_fullsized":"2023-10-16T11:26:42.470635+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:17:34.989394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.6","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.013987+0000","last_change":"2023-10-16T11:26:42.490458+0000","last_active":"2023-10-16T11:26:44.013987+0000","last_peered":"2023-10-16T11:26:44.013987+0000","last_clean":"2023-10-16T11:26:44.013987+0000","last_became_active":"2023-10-16T11:26:42.490031+0000","last_became_peered":"2023-10-16T11:26:42.490031+0000","last_unstale":"2023-10-16T11:26:44.013987+0000","last_undegraded":"2023-10-16T11:26:44.013987+0000","last_fullsized":"2023-10-16T11:26:44.013987+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:12.245044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.7","version":"256'2","reported_seq":14,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.501289+0000","last_change":"2023-10-16T11:26:41.475372+0000","last_active":"2023-10-16T11:26:42.501289+0000","last_peered":"2023-10-16T11:26:42.501289+0000","last_clean":"2023-10-16T11:26:42.501289+0000","last_became_active":"2023-10-16T11:26:41.475008+0000","last_became_peered":"2023-10-16T11:26:41.475008+0000","last_unstale":"2023-10-16T11:26:42.501289+0000","last_undegraded":"2023-10-16T11:26:42.501289+0000","last_fullsized":"2023-10-16T11:26:42.501289+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:46:01.374748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.7","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.488195+0000","last_change":"2023-10-16T11:26:42.479530+0000","last_active":"2023-10-16T11:26:42.488195+0000","last_peered":"2023-10-16T11:26:42.488195+0000","last_clean":"2023-10-16T11:26:42.488195+0000","last_became_active":"2023-10-16T11:26:42.479206+0000","last_became_peered":"2023-10-16T11:26:42.479206+0000","last_unstale":"2023-10-16T11:26:42.488195+0000","last_undegraded":"2023-10-16T11:26:42.488195+0000","last_fullsized":"2023-10-16T11:26:42.488195+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:32:11.699728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.471557+0000","last_change":"2023-10-16T11:26:41.471074+0000","last_active":"2023-10-16T11:26:42.471557+0000","last_peered":"2023-10-16T11:26:42.471557+0000","last_clean":"2023-10-16T11:26:42.471557+0000","last_became_active":"2023-10-16T11:26:41.469399+0000","last_became_peered":"2023-10-16T11:26:41.469399+0000","last_unstale":"2023-10-16T11:26:42.471557+0000","last_undegraded":"2023-10-16T11:26:42.471557+0000","last_fullsized":"2023-10-16T11:26:42.471557+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:52:05.824577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.4","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.014002+0000","last_change":"2023-10-16T11:26:42.493533+0000","last_active":"2023-10-16T11:26:44.014002+0000","last_peered":"2023-10-16T11:26:44.014002+0000","last_clean":"2023-10-16T11:26:44.014002+0000","last_became_active":"2023-10-16T11:26:42.493244+0000","last_became_peered":"2023-10-16T11:26:42.493244+0000","last_unstale":"2023-10-16T11:26:44.014002+0000","last_undegraded":"2023-10-16T11:26:44.014002+0000","last_fullsized":"2023-10-16T11:26:44.014002+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:53:50.127580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.466387+0000","last_change":"2023-10-16T11:26:41.469347+0000","last_active":"2023-10-16T11:26:42.466387+0000","last_peered":"2023-10-16T11:26:42.466387+0000","last_clean":"2023-10-16T11:26:42.466387+0000","last_became_active":"2023-10-16T11:26:41.469012+0000","last_became_peered":"2023-10-16T11:26:41.469012+0000","last_unstale":"2023-10-16T11:26:42.466387+0000","last_undegraded":"2023-10-16T11:26:42.466387+0000","last_fullsized":"2023-10-16T11:26:42.466387+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:34:42.424697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.5","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.449204+0000","last_change":"2023-10-16T11:26:42.457693+0000","last_active":"2023-10-16T11:26:43.449204+0000","last_peered":"2023-10-16T11:26:43.449204+0000","last_clean":"2023-10-16T11:26:43.449204+0000","last_became_active":"2023-10-16T11:26:42.457094+0000","last_became_peered":"2023-10-16T11:26:42.457094+0000","last_unstale":"2023-10-16T11:26:43.449204+0000","last_undegraded":"2023-10-16T11:26:43.449204+0000","last_fullsized":"2023-10-16T11:26:43.449204+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:04:15.156293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.465322+0000","last_change":"2023-10-16T11:26:41.469699+0000","last_active":"2023-10-16T11:26:42.465322+0000","last_peered":"2023-10-16T11:26:42.465322+0000","last_clean":"2023-10-16T11:26:42.465322+0000","last_became_active":"2023-10-16T11:26:41.469373+0000","last_became_peered":"2023-10-16T11:26:41.469373+0000","last_unstale":"2023-10-16T11:26:42.465322+0000","last_undegraded":"2023-10-16T11:26:42.465322+0000","last_fullsized":"2023-10-16T11:26:42.465322+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:53:05.859368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.2","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.445993+0000","last_change":"2023-10-16T11:26:42.462890+0000","last_active":"2023-10-16T11:26:43.445993+0000","last_peered":"2023-10-16T11:26:43.445993+0000","last_clean":"2023-10-16T11:26:43.445993+0000","last_became_active":"2023-10-16T11:26:42.462368+0000","last_became_peered":"2023-10-16T11:26:42.462368+0000","last_unstale":"2023-10-16T11:26:43.445993+0000","last_undegraded":"2023-10-16T11:26:43.445993+0000","last_fullsized":"2023-10-16T11:26:43.445993+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:23.956560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.483323+0000","last_change":"2023-10-16T11:26:41.462500+0000","last_active":"2023-10-16T11:26:42.483323+0000","last_peered":"2023-10-16T11:26:42.483323+0000","last_clean":"2023-10-16T11:26:42.483323+0000","last_became_active":"2023-10-16T11:26:41.461669+0000","last_became_peered":"2023-10-16T11:26:41.461669+0000","last_unstale":"2023-10-16T11:26:42.483323+0000","last_undegraded":"2023-10-16T11:26:42.483323+0000","last_fullsized":"2023-10-16T11:26:42.483323+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:51.895243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.3","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.492080+0000","last_change":"2023-10-16T11:26:42.486393+0000","last_active":"2023-10-16T11:26:42.492080+0000","last_peered":"2023-10-16T11:26:42.492080+0000","last_clean":"2023-10-16T11:26:42.492080+0000","last_became_active":"2023-10-16T11:26:42.478616+0000","last_became_peered":"2023-10-16T11:26:42.478616+0000","last_unstale":"2023-10-16T11:26:42.492080+0000","last_undegraded":"2023-10-16T11:26:42.492080+0000","last_fullsized":"2023-10-16T11:26:42.492080+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:26:16.887346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.475669+0000","last_change":"2023-10-16T11:26:41.459862+0000","last_active":"2023-10-16T11:26:42.475669+0000","last_peered":"2023-10-16T11:26:42.475669+0000","last_clean":"2023-10-16T11:26:42.475669+0000","last_became_active":"2023-10-16T11:26:41.459567+0000","last_became_peered":"2023-10-16T11:26:41.459567+0000","last_unstale":"2023-10-16T11:26:42.475669+0000","last_undegraded":"2023-10-16T11:26:42.475669+0000","last_fullsized":"2023-10-16T11:26:42.475669+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:54:09.078766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.0","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.487912+0000","last_change":"2023-10-16T11:26:42.477532+0000","last_active":"2023-10-16T11:26:42.487912+0000","last_peered":"2023-10-16T11:26:42.487912+0000","last_clean":"2023-10-16T11:26:42.487912+0000","last_became_active":"2023-10-16T11:26:42.477268+0000","last_became_peered":"2023-10-16T11:26:42.477268+0000","last_unstale":"2023-10-16T11:26:42.487912+0000","last_undegraded":"2023-10-16T11:26:42.487912+0000","last_fullsized":"2023-10-16T11:26:42.487912+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:24.858930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.474804+0000","last_change":"2023-10-16T11:26:41.463237+0000","last_active":"2023-10-16T11:26:42.474804+0000","last_peered":"2023-10-16T11:26:42.474804+0000","last_clean":"2023-10-16T11:26:42.474804+0000","last_became_active":"2023-10-16T11:26:41.462972+0000","last_became_peered":"2023-10-16T11:26:41.462972+0000","last_unstale":"2023-10-16T11:26:42.474804+0000","last_undegraded":"2023-10-16T11:26:42.474804+0000","last_fullsized":"2023-10-16T11:26:42.474804+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:17.143386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.495180+0000","last_change":"2023-10-16T11:26:42.492352+0000","last_active":"2023-10-16T11:26:42.495180+0000","last_peered":"2023-10-16T11:26:42.495180+0000","last_clean":"2023-10-16T11:26:42.495180+0000","last_became_active":"2023-10-16T11:26:42.490917+0000","last_became_peered":"2023-10-16T11:26:42.490917+0000","last_unstale":"2023-10-16T11:26:42.495180+0000","last_undegraded":"2023-10-16T11:26:42.495180+0000","last_fullsized":"2023-10-16T11:26:42.495180+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:14.423981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.451400+0000","last_change":"2023-10-16T11:26:41.463453+0000","last_active":"2023-10-16T11:26:42.451400+0000","last_peered":"2023-10-16T11:26:42.451400+0000","last_clean":"2023-10-16T11:26:42.451400+0000","last_became_active":"2023-10-16T11:26:41.462536+0000","last_became_peered":"2023-10-16T11:26:41.462536+0000","last_unstale":"2023-10-16T11:26:42.451400+0000","last_undegraded":"2023-10-16T11:26:42.451400+0000","last_fullsized":"2023-10-16T11:26:42.451400+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:32:24.083977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.22","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.057542+0000","last_change":"2023-10-16T11:26:42.490965+0000","last_active":"2023-10-16T11:26:44.057542+0000","last_peered":"2023-10-16T11:26:44.057542+0000","last_clean":"2023-10-16T11:26:44.057542+0000","last_became_active":"2023-10-16T11:26:42.489788+0000","last_became_peered":"2023-10-16T11:26:42.489788+0000","last_unstale":"2023-10-16T11:26:44.057542+0000","last_undegraded":"2023-10-16T11:26:44.057542+0000","last_fullsized":"2023-10-16T11:26:44.057542+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:44:01.049493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"254'4803","reported_seq":5829,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.462104+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:26:42.462104+0000","last_peered":"2023-10-16T11:26:42.462104+0000","last_clean":"2023-10-16T11:26:42.462104+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:26:42.462104+0000","last_undegraded":"2023-10-16T11:26:42.462104+0000","last_fullsized":"2023-10-16T11:26:42.462104+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":4803,"log_dups_size":0,"ondisk_log_size":4803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2495,"num_read_kb":5117,"num_write":643,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":35,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":34,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2495,"num_read_kb":5117,"num_write":643,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4803,"ondisk_log_size":4803,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576011,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61304,"kb_used_data":3752,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310536,"statfs":{"total":96636764160,"available":96573988864,"internally_reserved":0,"allocated":3842048,"data_stored":2181490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.1160000000000001,"15min":1.1970000000000001},"min":{"1min":0.70299999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4490000000000001,"5min":1.7250000000000001,"15min":2.3399999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.139,"15min":1.202},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.6479999999999999,"5min":1.6699999999999999,"15min":1.8320000000000001},"last":0.95499999999999996}]},{"osd":1,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.21,"15min":1.2310000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.452,"5min":1.7589999999999999,"15min":1.913},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1970000000000001,"15min":1.2549999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.6799999999999999,"5min":1.726,"15min":1.8600000000000001},"last":1.163}]},{"osd":2,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2889999999999999,"15min":1.29},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":2.4460000000000002},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.3400000000000001,"15min":1.361},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":3.9249999999999998,"5min":3.9249999999999998,"15min":3.9249999999999998},"last":1.0589999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.359,"15min":1.3799999999999999},"min":{"1min":1.1419999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.399,"15min":1.4379999999999999},"min":{"1min":0.81399999999999995,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.802,"5min":2.0070000000000001,"15min":2.577},"last":1.484}]},{"osd":4,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.4359999999999999,"15min":1.4079999999999999},"min":{"1min":1.093,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.9610000000000001,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.204},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.373,"15min":1.383},"min":{"1min":0.91400000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.25}]},{"osd":5,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.5740000000000001,"15min":1.583},"min":{"1min":0.81499999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.966,"5min":2.101,"15min":3.3519999999999999},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.462,"5min":1.516,"15min":1.5009999999999999},"min":{"1min":0.97899999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.7569999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.1240000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.502,"15min":1.5029999999999999},"min":{"1min":1.018,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.7250000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.635,"5min":1.665,"15min":1.6879999999999999},"min":{"1min":0.76800000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1600000000000001,"5min":2.2109999999999999,"15min":3.1440000000000001},"last":1.601}]}]},{"osd":6,"up_from":13,"seq":55834576011,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52412,"kb_used_data":3176,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319428,"statfs":{"total":96636764160,"available":96583094272,"internally_reserved":0,"allocated":3252224,"data_stored":1591212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.1779999999999999,"15min":1.206},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.5700000000000001,"5min":6.298,"15min":6.298},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.3129999999999999,"15min":1.278},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6140000000000001,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":1.202}]},{"osd":1,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.69799999999999995,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.24,"5min":1.395,"15min":1.365},"min":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.956,"5min":12.468,"15min":12.468},"last":0.97699999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.286,"15min":1.262},"min":{"1min":0.82899999999999996,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4870000000000001,"5min":11.586,"15min":11.586},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.3799999999999999,"15min":1.3720000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.77500000000000002},"max":{"1min":1.8600000000000001,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":0.79300000000000004}]},{"osd":3,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.4199999999999999,"15min":1.3440000000000001},"min":{"1min":0.86299999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.925,"5min":12.801,"15min":12.801},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.2809999999999999,"15min":1.2649999999999999},"min":{"1min":0.88100000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5589999999999999,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":1.5589999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.417,"15min":1.3320000000000001},"min":{"1min":0.76300000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.806,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.417},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.6359999999999999,"15min":1.548},"min":{"1min":1.109,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.3069999999999999,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.607}]},{"osd":5,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.5469999999999999,"15min":1.3999999999999999},"min":{"1min":0.86199999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.8979999999999999,"5min":18.247,"15min":18.247},"last":1.274},{"interface":"front","average":{"1min":1.617,"5min":1.625,"15min":1.5660000000000001},"min":{"1min":0.85899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.923,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.6839999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.6779999999999999,"5min":1.6819999999999999,"15min":1.5529999999999999},"min":{"1min":0.92500000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.4660000000000002,"5min":12.589,"15min":12.589},"last":1.738},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.7110000000000001,"15min":1.6020000000000001},"min":{"1min":1.1779999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.2050000000000001,"5min":12.725,"15min":12.725},"last":1.3740000000000001}]}]},{"osd":5,"up_from":13,"seq":55834576011,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48252,"kb_used_data":3180,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323588,"statfs":{"total":96636764160,"available":96587354112,"internally_reserved":0,"allocated":3256320,"data_stored":1587015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.2110000000000001,"15min":1.2529999999999999},"min":{"1min":0.749,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.3839999999999999,"5min":2.661,"15min":2.661},"last":1.446},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2749999999999999,"15min":1.327},"min":{"1min":0.76400000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.675,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.1619999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.1319999999999999,"15min":1.1779999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.992},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1970000000000001,"15min":1.22},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.669,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.5960000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.248,"15min":1.228},"min":{"1min":0.73299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.526,"5min":1.952,"15min":1.952},"last":1.329},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.2929999999999999,"15min":1.3109999999999999},"min":{"1min":0.83599999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":2.1640000000000001},"last":1.389}]},{"osd":3,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.302,"15min":1.3220000000000001},"min":{"1min":0.97999999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.6830000000000001,"5min":3.012,"15min":3.012},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.395,"15min":1.409},"min":{"1min":0.93400000000000005,"5min":0.92700000000000005,"15min":0.89500000000000002},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.2250000000000001},"last":1.51}]},{"osd":4,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3260000000000001,"15min":1.333},"min":{"1min":0.96099999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.575,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.494,"5min":1.544,"15min":1.5609999999999999},"min":{"1min":1.048,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.8009999999999999,"5min":2.903,"15min":2.903},"last":1.6439999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.3999999999999999,"15min":1.405},"min":{"1min":1.097,"5min":0.68100000000000005,"15min":0.51000000000000001},"max":{"1min":1.6519999999999999,"5min":1.8660000000000001,"15min":2.0979999999999999},"last":1.673},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.458,"15min":1.474},"min":{"1min":0.95999999999999996,"5min":0.90700000000000003,"15min":0.89200000000000002},"max":{"1min":1.758,"5min":3.073,"15min":3.073},"last":1.218}]},{"osd":7,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.536,"15min":1.538},"min":{"1min":1.0549999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.1150000000000002,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.556},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.615,"15min":1.637},"min":{"1min":1.3400000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.2930000000000001},"last":1.748}]}]},{"osd":4,"up_from":13,"seq":55834576012,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70380,"kb_used_data":3168,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301460,"statfs":{"total":96636764160,"available":96564695040,"internally_reserved":0,"allocated":3244032,"data_stored":1584843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.1870000000000001,"15min":1.1839999999999999},"min":{"1min":0.80800000000000005,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.8049999999999999,"5min":2.899,"15min":2.899},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2170000000000001,"15min":1.2450000000000001},"min":{"1min":0.88400000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.7929999999999999,"5min":3.016,"15min":3.016},"last":1.091}]},{"osd":1,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3049999999999999,"15min":1.298},"min":{"1min":0.84999999999999998,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.887,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.77100000000000002,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.617,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.018}]},{"osd":2,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.327,"15min":1.3140000000000001},"min":{"1min":1.075,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.169,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4350000000000001,"15min":1.4159999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9870000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.417}]},{"osd":3,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.3640000000000001,"15min":1.351},"min":{"1min":0.96799999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.0390000000000001,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.365},{"interface":"front","average":{"1min":1.645,"5min":1.4970000000000001,"15min":1.4610000000000001},"min":{"1min":1.216,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.3109999999999999,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.794}]},{"osd":5,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.5289999999999999,"15min":1.464},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2309999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.681},{"interface":"front","average":{"1min":1.504,"5min":1.4810000000000001,"15min":1.4350000000000001},"min":{"1min":0.89500000000000002,"5min":0.85199999999999998,"15min":0.78600000000000003},"max":{"1min":2.0110000000000001,"5min":3.7989999999999999,"15min":3.7989999999999999},"last":1.464}]},{"osd":6,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.599,"15min":1.534},"min":{"1min":0.98499999999999999,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":2.6459999999999999,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.9159999999999999},{"interface":"front","average":{"1min":1.752,"5min":1.665,"15min":1.6379999999999999},"min":{"1min":0.77100000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.3599999999999999,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.76}]},{"osd":7,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.7629999999999999,"5min":1.649,"15min":1.6000000000000001},"min":{"1min":0.74099999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.3570000000000002,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.7849999999999999,"5min":1.7270000000000001,"15min":1.675},"min":{"1min":0.98699999999999999,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.464,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":2.0219999999999998}]}]},{"osd":3,"up_from":13,"seq":55834576011,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52420,"kb_used_data":3184,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319420,"statfs":{"total":96636764160,"available":96583086080,"internally_reserved":0,"allocated":3260416,"data_stored":1592734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.83199999999999996,"15min":0.88900000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.2430000000000001,"5min":1.478,"15min":1.901},"last":1.4730000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.95599999999999996,"15min":1.046},"min":{"1min":0.45100000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.581,"5min":1.581,"15min":1.6679999999999999},"last":1.0880000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.92700000000000005,"15min":0.97599999999999998},"min":{"1min":0.39600000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.3380000000000001,"5min":1.508,"15min":1.508},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.89100000000000001,"5min":1.016,"15min":1.1020000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.246,"5min":1.601,"15min":1.7190000000000001},"last":1.038}]},{"osd":2,"last update":"Mon Oct 16 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.042,"15min":1.105},"min":{"1min":0.52400000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5660000000000001,"5min":1.694,"15min":1.694},"last":1.3180000000000001},{"interface":"front","average":{"1min":0.999,"5min":1.117,"15min":1.2030000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.542,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.492}]},{"osd":4,"last update":"Mon Oct 16 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2529999999999999,"15min":1.335},"min":{"1min":0.71199999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.821,"5min":1.821,"15min":1.873},"last":1.411},{"interface":"front","average":{"1min":1.204,"5min":1.2989999999999999,"15min":1.349},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5960000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.3700000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.341,"15min":1.393},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9199999999999999,"5min":2.798,"15min":2.798},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.371,"15min":1.429},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.8440000000000001,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.4319999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.456,"15min":1.52},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.901,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.351},{"interface":"front","average":{"1min":1.405,"5min":1.4630000000000001,"15min":1.522},"min":{"1min":0.96299999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.7929999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.5589999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.496,"15min":1.55},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.827,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.617},{"interface":"front","average":{"1min":1.498,"5min":1.5800000000000001,"15min":1.657},"min":{"1min":1.0309999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.9059999999999999,"5min":2.3799999999999999,"15min":2.4649999999999999},"last":1.7090000000000001}]}]},{"osd":2,"up_from":13,"seq":55834576010,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56564,"kb_used_data":3176,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315276,"statfs":{"total":96636764160,"available":96578842624,"internally_reserved":0,"allocated":3252224,"data_stored":1591212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88900000000000001,"15min":0.90900000000000003},"min":{"1min":0.498,"5min":0.314,"15min":0.314},"max":{"1min":1.4870000000000001,"5min":1.532,"15min":1.6399999999999999},"last":1.3},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.97799999999999998,"15min":1.0580000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.32600000000000001},"max":{"1min":1.5449999999999999,"5min":1.6359999999999999,"15min":2.1429999999999998},"last":0.81999999999999995}]},{"osd":1,"last update":"Mon Oct 16 11:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0640000000000001,"15min":1.0780000000000001},"min":{"1min":0.57399999999999995,"5min":0.42699999999999999,"15min":0.39200000000000002},"max":{"1min":1.4750000000000001,"5min":1.645,"15min":1.978},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.048,"15min":1.123},"min":{"1min":0.58999999999999997,"5min":0.438,"15min":0.438},"max":{"1min":1.4179999999999999,"5min":1.609,"15min":1.7949999999999999},"last":0.95099999999999996}]},{"osd":3,"last update":"Mon Oct 16 11:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.006,"15min":1.0629999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4510000000000001,"5min":1.651,"15min":1.8560000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.089,"5min":1.1739999999999999,"15min":1.2509999999999999},"min":{"1min":0.61399999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.5880000000000001,"5min":1.7450000000000001,"15min":1.857},"last":1.0049999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:26:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3140000000000001,"15min":1.391},"min":{"1min":0.95099999999999996,"5min":0.78600000000000003,"15min":0.73499999999999999},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.5059999999999998},"last":1.244},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.2709999999999999,"15min":1.321},"min":{"1min":0.878,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":1.345,"5min":1.825,"15min":1.843},"last":1.1339999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:26:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.4059999999999999,"15min":1.4419999999999999},"min":{"1min":0.89200000000000002,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.895,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.425,"15min":1.464},"min":{"1min":0.96399999999999997,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.323,"5min":2.323,"15min":2.6259999999999999},"last":1.4590000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:26:08 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.3740000000000001,"15min":1.3899999999999999},"min":{"1min":0.83199999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.855,"5min":1.9019999999999999,"15min":2.1339999999999999},"last":1.341},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4810000000000001,"15min":1.5189999999999999},"min":{"1min":0.81499999999999995,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.7929999999999999,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.506}]},{"osd":7,"last update":"Mon Oct 16 11:26:08 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.5069999999999999,"15min":1.5589999999999999},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92500000000000004},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.8109999999999999},"last":1.611},{"interface":"front","average":{"1min":1.389,"5min":1.452,"15min":1.492},"min":{"1min":0.88400000000000001,"5min":0.80200000000000005,"15min":0.80100000000000005},"max":{"1min":1.768,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.5620000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576010,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48244,"kb_used_data":3176,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323596,"statfs":{"total":96636764160,"available":96587362304,"internally_reserved":0,"allocated":3252224,"data_stored":1591156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.90900000000000003,"15min":0.91300000000000003},"min":{"1min":0.39400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.3520000000000001,"5min":5.117,"15min":5.117},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.93500000000000005,"15min":1.0289999999999999},"min":{"1min":0.35899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.262,"5min":1.5509999999999999,"15min":1.667},"last":0.45500000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.93799999999999994,"15min":1.0029999999999999},"min":{"1min":0.497,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3720000000000001,"5min":2.093,"15min":2.093},"last":0.33100000000000002},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1879999999999999,"15min":1.2589999999999999},"min":{"1min":0.51700000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.456,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.498}]},{"osd":3,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.139,"15min":1.1890000000000001},"min":{"1min":0.53800000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.6080000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.103,"15min":1.153},"min":{"1min":0.55800000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.798,"5min":1.798,"15min":1.802},"last":0.54100000000000004}]},{"osd":4,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.3049999999999999,"15min":1.2949999999999999},"min":{"1min":0.69299999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.978,"5min":5.29,"15min":5.29},"last":1.079},{"interface":"front","average":{"1min":1.327,"5min":1.4159999999999999,"15min":1.421},"min":{"1min":0.82999999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.891,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":0.78300000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.373,"15min":1.409},"min":{"1min":0.83799999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.9430000000000001,"5min":6.0979999999999999,"15min":6.0979999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.383,"15min":1.403},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.859,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":0.90400000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4570000000000001,"15min":1.4890000000000001},"min":{"1min":0.92200000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.923,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4419999999999999,"15min":1.4810000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.1419999999999999,"5min":3.8799999999999999,"15min":3.8799999999999999},"last":0.70599999999999996}]},{"osd":7,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.546,"15min":1.5609999999999999},"min":{"1min":1.0409999999999999,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":2.0219999999999998,"5min":4.46,"15min":4.46},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.512,"5min":1.597,"15min":1.6040000000000001},"min":{"1min":0.99099999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.097,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":1.024}]}]},{"osd":0,"up_from":13,"seq":55834576011,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65464,"kb_used_data":3752,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306376,"statfs":{"total":96636764160,"available":96569729024,"internally_reserved":0,"allocated":3842048,"data_stored":2175757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.86099999999999999,"15min":0.88700000000000001},"min":{"1min":0.46800000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2210000000000001,"5min":1.52,"15min":1.619},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.006,"15min":1.081},"min":{"1min":0.41199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.341,"5min":1.621,"15min":1.76},"last":1.518}]},{"osd":2,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.075,"15min":1.0800000000000001},"min":{"1min":0.52500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.7349999999999999,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":1.038},{"interface":"front","average":{"1min":1.034,"5min":1.0700000000000001,"15min":1.1299999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.97899999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.016,"15min":1.0669999999999999},"min":{"1min":0.50600000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1299999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.1539999999999999,"15min":1.226},"min":{"1min":0.55600000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.3799999999999999,"5min":1.833,"15min":1.9830000000000001},"last":1.3799999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.349,"15min":1.3859999999999999},"min":{"1min":0.78800000000000003,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":3.2799999999999998,"5min":3.2799999999999998,"15min":3.2799999999999998},"last":1.272},{"interface":"front","average":{"1min":1.222,"5min":1.292,"15min":1.337},"min":{"1min":0.76100000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.5920000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.357,"15min":1.3720000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0209999999999999,"5min":2.0750000000000002,"15min":2.6669999999999998},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3819999999999999,"15min":1.431},"min":{"1min":0.89800000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.885,"5min":2.2029999999999998,"15min":2.2040000000000002},"last":1.155}]},{"osd":6,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.4710000000000001,"15min":1.496},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":5.2560000000000002,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":1.462},{"interface":"front","average":{"1min":1.298,"5min":1.429,"15min":1.4790000000000001},"min":{"1min":0.83199999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.804,"5min":2.073,"15min":2.262},"last":1.6499999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.704,"5min":1.5429999999999999,"15min":1.5600000000000001},"min":{"1min":0.88700000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.425},{"interface":"front","average":{"1min":1.278,"5min":1.4419999999999999,"15min":1.492},"min":{"1min":0.871,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.623,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.6950000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:26:47.320 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:26:47.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.570+0000 7feeb2747640 1 Processor -- start 2023-10-16T11:26:47.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.571+0000 7feeb2747640 1 -- start start 2023-10-16T11:26:47.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.572+0000 7feeb2747640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeac15c9f0 0x7feeac0c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:47.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.572+0000 7feeb2747640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feeac15dde0 0x7feeac0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:47.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.572+0000 7feeb2747640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeac160a90 con 0x7feeac0c9230 2023-10-16T11:26:47.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.572+0000 7feeb2747640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeac0cdec0 con 0x7feeac15dde0 2023-10-16T11:26:47.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.572+0000 7feeb2747640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0cd900 con 0x7feeac15c9f0 2023-10-16T11:26:47.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.573+0000 7feeb1f46640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feeac15dde0 0x7feeac0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:47.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.573+0000 7feeb1745640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeac15c9f0 0x7feeac0c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:47.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.573+0000 7feeb1f46640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feeac15dde0 0x7feeac0ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:38504/0 (socket says 172.21.15.84:38504) 2023-10-16T11:26:47.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.573+0000 7feeb1f46640 1 -- 172.21.15.84:0/1593485570 learned_addr learned my addr 172.21.15.84:0/1593485570 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:47.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.574+0000 7feeb1f46640 1 -- 172.21.15.84:0/1593485570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeac15c9f0 msgr2=0x7feeac0c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:47.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.574+0000 7feeb1f46640 1 --2- 172.21.15.84:0/1593485570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeac15c9f0 0x7feeac0c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.574+0000 7feeb1f46640 1 -- 172.21.15.84:0/1593485570 >> v1:172.21.15.157:6789/0 conn(0x7feeac0c9230 legacy=0x7feeac0709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:47.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.574+0000 7feeb1f46640 1 -- 172.21.15.84:0/1593485570 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac0cdbe0 con 0x7feeac15dde0 2023-10-16T11:26:47.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.574+0000 7feeb1745640 1 --2- 172.21.15.84:0/1593485570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeac15c9f0 0x7feeac0c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:26:47.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.575+0000 7feeb1f46640 1 --2- 172.21.15.84:0/1593485570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeac15dde0 0x7feeac0ac890 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7feea4009e50 tx=0x7feea402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3810a10c3738afec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:47.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.576+0000 7feea3fff640 1 -- 172.21.15.84:0/1593485570 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea402cb00 con 0x7feeac15dde0 2023-10-16T11:26:47.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.576+0000 7feea3fff640 1 -- 172.21.15.84:0/1593485570 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea402cc60 con 0x7feeac15dde0 2023-10-16T11:26:47.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.576+0000 7feea3fff640 1 -- 172.21.15.84:0/1593485570 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea40343f0 con 0x7feeac15dde0 2023-10-16T11:26:47.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.577+0000 7feeb2747640 1 -- 172.21.15.84:0/1593485570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeac15dde0 msgr2=0x7feeac0ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:47.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.577+0000 7feeb2747640 1 --2- 172.21.15.84:0/1593485570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeac15dde0 0x7feeac0ac890 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7feea4009e50 tx=0x7feea402bf60 comp rx=0 tx=0).stop 2023-10-16T11:26:47.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.577+0000 7feeb2747640 1 -- 172.21.15.84:0/1593485570 shutdown_connections 2023-10-16T11:26:47.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.577+0000 7feeb2747640 1 --2- 172.21.15.84:0/1593485570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeac15dde0 0x7feeac0ac890 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.577+0000 7feeb2747640 1 --2- 172.21.15.84:0/1593485570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeac15c9f0 0x7feeac0c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.577+0000 7feeb2747640 1 -- 172.21.15.84:0/1593485570 >> 172.21.15.84:0/1593485570 conn(0x7feeac0ce6a0 msgr2=0x7feeac0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:47.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.577+0000 7feeb2747640 1 -- 172.21.15.84:0/1593485570 shutdown_connections 2023-10-16T11:26:47.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.577+0000 7feeb2747640 1 -- 172.21.15.84:0/1593485570 wait complete. 2023-10-16T11:26:47.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.579+0000 7feeb2747640 1 Processor -- start 2023-10-16T11:26:47.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.579+0000 7feeb2747640 1 -- start start 2023-10-16T11:26:47.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.580+0000 7feeb2747640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeac0c9230 0x7feeac142920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:47.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.580+0000 7feeb2747640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeac15c9f0 0x7feeac143300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:47.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.580+0000 7feeb2747640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeac15dde0 0x7feeac06adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:47.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.580+0000 7feeb2747640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0c66f0 con 0x7feeac15dde0 2023-10-16T11:26:47.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.580+0000 7feeb2747640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac06b840 con 0x7feeac0c9230 2023-10-16T11:26:47.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.580+0000 7feeb2747640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac06bb20 con 0x7feeac15c9f0 2023-10-16T11:26:47.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.581+0000 7feeb1f46640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeac15dde0 0x7feeac06adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:47.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.581+0000 7feeb1f46640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeac15dde0 0x7feeac06adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51608/0 (socket says 172.21.15.84:51608) 2023-10-16T11:26:47.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.581+0000 7feeb1f46640 1 -- 172.21.15.84:0/2719079825 learned_addr learned my addr 172.21.15.84:0/2719079825 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:47.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.581+0000 7feeb1745640 1 --2- 172.21.15.84:0/2719079825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeac0c9230 0x7feeac142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:47.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.581+0000 7feeb0f44640 1 --2- 172.21.15.84:0/2719079825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeac15c9f0 0x7feeac143300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:47.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.581+0000 7feeb1f46640 1 -- 172.21.15.84:0/2719079825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeac15c9f0 msgr2=0x7feeac143300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:47.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.581+0000 7feeb1f46640 1 --2- 172.21.15.84:0/2719079825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeac15c9f0 0x7feeac143300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.581+0000 7feeb1f46640 1 -- 172.21.15.84:0/2719079825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeac0c9230 msgr2=0x7feeac142920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:47.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.581+0000 7feeb1f46640 1 --2- 172.21.15.84:0/2719079825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeac0c9230 0x7feeac142920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.581+0000 7feeb1f46640 1 -- 172.21.15.84:0/2719079825 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea8009400 con 0x7feeac15dde0 2023-10-16T11:26:47.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.581+0000 7feeb1745640 1 --2- 172.21.15.84:0/2719079825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeac0c9230 0x7feeac142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:47.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.582+0000 7feeb1f46640 1 --2- 172.21.15.84:0/2719079825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeac15dde0 0x7feeac06adf0 secure :-1 s=READY pgs=3222 cs=0 l=1 rev1=1 crypto rx=0x7feea4009e50 tx=0x7feea4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:47.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.582+0000 7feea1ffb640 1 -- 172.21.15.84:0/2719079825 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4009510 con 0x7feeac15dde0 2023-10-16T11:26:47.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.582+0000 7feea1ffb640 1 -- 172.21.15.84:0/2719079825 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea403e450 con 0x7feeac15dde0 2023-10-16T11:26:47.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.582+0000 7feeb2747640 1 -- 172.21.15.84:0/2719079825 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea4009b30 con 0x7feeac15dde0 2023-10-16T11:26:47.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.582+0000 7feea1ffb640 1 -- 172.21.15.84:0/2719079825 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea403e800 con 0x7feeac15dde0 2023-10-16T11:26:47.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.582+0000 7feeb2747640 1 -- 172.21.15.84:0/2719079825 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeac06c0e0 con 0x7feeac15dde0 2023-10-16T11:26:47.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.583+0000 7feeb2747640 1 -- 172.21.15.84:0/2719079825 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeac04e9b0 con 0x7feeac15dde0 2023-10-16T11:26:47.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.584+0000 7feea1ffb640 1 -- 172.21.15.84:0/2719079825 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 151) v1 ==== 89433+0+0 (secure 0 0 0) 0x7feea403e960 con 0x7feeac15dde0 2023-10-16T11:26:47.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.585+0000 7feea1ffb640 1 --2- 172.21.15.84:0/2719079825 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fee9006b0b0 0x7fee9006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:47.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.585+0000 7feea1ffb640 1 -- 172.21.15.84:0/2719079825 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5780+0+0 (secure 0 0 0) 0x7feea40ac840 con 0x7feeac15dde0 2023-10-16T11:26:47.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.585+0000 7feeb1745640 1 --2- 172.21.15.84:0/2719079825 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fee9006b0b0 0x7fee9006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:47.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.586+0000 7feeb1745640 1 --2- 172.21.15.84:0/2719079825 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fee9006b0b0 0x7fee9006d570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feea8009c50 tx=0x7feea8008040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:47.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.589+0000 7feea1ffb640 1 -- 172.21.15.84:0/2719079825 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7feea40774a0 con 0x7feeac15dde0 2023-10-16T11:26:47.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.777+0000 7feeb2747640 1 -- 172.21.15.84:0/2719079825 --> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feeac0704c0 con 0x7fee9006b0b0 2023-10-16T11:26:47.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.790+0000 7feea1ffb640 1 -- 172.21.15.84:0/2719079825 <== mgr.12624 v2:172.21.15.84:6832/2330196255 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287426 (secure 0 0 0) 0x7feea8010e20 con 0x7fee9006b0b0 2023-10-16T11:26:47.796 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:47.798 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:26:47.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.796+0000 7feeb2747640 1 -- 172.21.15.84:0/2719079825 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fee9006b0b0 msgr2=0x7fee9006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:47.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.796+0000 7feeb2747640 1 --2- 172.21.15.84:0/2719079825 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fee9006b0b0 0x7fee9006d570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feea8009c50 tx=0x7feea8008040 comp rx=0 tx=0).stop 2023-10-16T11:26:47.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.796+0000 7feeb2747640 1 -- 172.21.15.84:0/2719079825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeac15dde0 msgr2=0x7feeac06adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:47.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.796+0000 7feeb2747640 1 --2- 172.21.15.84:0/2719079825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeac15dde0 0x7feeac06adf0 secure :-1 s=READY pgs=3222 cs=0 l=1 rev1=1 crypto rx=0x7feea4009e50 tx=0x7feea4004a40 comp rx=0 tx=0).stop 2023-10-16T11:26:47.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.797+0000 7feeb2747640 1 -- 172.21.15.84:0/2719079825 shutdown_connections 2023-10-16T11:26:47.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.797+0000 7feeb2747640 1 --2- 172.21.15.84:0/2719079825 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fee9006b0b0 0x7fee9006d570 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.797+0000 7feeb2747640 1 --2- 172.21.15.84:0/2719079825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feeac15dde0 0x7feeac06adf0 unknown :-1 s=CLOSED pgs=3222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.797+0000 7feeb2747640 1 --2- 172.21.15.84:0/2719079825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feeac15c9f0 0x7feeac143300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.797+0000 7feeb2747640 1 --2- 172.21.15.84:0/2719079825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeac0c9230 0x7feeac142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:47.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.797+0000 7feeb2747640 1 -- 172.21.15.84:0/2719079825 >> 172.21.15.84:0/2719079825 conn(0x7feeac0ce6a0 msgr2=0x7feeac15e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:47.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.797+0000 7feeb2747640 1 -- 172.21.15.84:0/2719079825 shutdown_connections 2023-10-16T11:26:47.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:47.797+0000 7feeb2747640 1 -- 172.21.15.84:0/2719079825 wait complete. 2023-10-16T11:26:47.817 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-10-16T11:26:46.510759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2495,"num_read_kb":5117,"num_write":687,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4841,"ondisk_log_size":4841,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":455040,"kb_used_data":26564,"kb_used_omap":12,"kb_used_meta":428339,"kb_avail":754519680,"statfs":{"total":773094113280,"available":772628152320,"internally_reserved":0,"allocated":27201536,"data_stored":13895419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":438619705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":94,"num_read_kb":206,"num_write":64,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001770"},"pg_stats":[{"pgid":"35.3f","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.485834+0000","last_change":"2023-10-16T11:26:42.485177+0000","last_active":"2023-10-16T11:26:42.485834+0000","last_peered":"2023-10-16T11:26:42.485834+0000","last_clean":"2023-10-16T11:26:42.485834+0000","last_became_active":"2023-10-16T11:26:42.484764+0000","last_became_peered":"2023-10-16T11:26:42.484764+0000","last_unstale":"2023-10-16T11:26:42.485834+0000","last_undegraded":"2023-10-16T11:26:42.485834+0000","last_fullsized":"2023-10-16T11:26:42.485834+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:48:34.990016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.3e","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.448974+0000","last_change":"2023-10-16T11:26:42.477252+0000","last_active":"2023-10-16T11:26:43.448974+0000","last_peered":"2023-10-16T11:26:43.448974+0000","last_clean":"2023-10-16T11:26:43.448974+0000","last_became_active":"2023-10-16T11:26:42.476616+0000","last_became_peered":"2023-10-16T11:26:42.476616+0000","last_unstale":"2023-10-16T11:26:43.448974+0000","last_undegraded":"2023-10-16T11:26:43.448974+0000","last_fullsized":"2023-10-16T11:26:43.448974+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:48:35.570379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.3d","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.448731+0000","last_change":"2023-10-16T11:26:42.472467+0000","last_active":"2023-10-16T11:26:43.448731+0000","last_peered":"2023-10-16T11:26:43.448731+0000","last_clean":"2023-10-16T11:26:43.448731+0000","last_became_active":"2023-10-16T11:26:42.472067+0000","last_became_peered":"2023-10-16T11:26:42.472067+0000","last_unstale":"2023-10-16T11:26:43.448731+0000","last_undegraded":"2023-10-16T11:26:43.448731+0000","last_fullsized":"2023-10-16T11:26:43.448731+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:32.729558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.3c","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.488295+0000","last_change":"2023-10-16T11:26:42.473214+0000","last_active":"2023-10-16T11:26:42.488295+0000","last_peered":"2023-10-16T11:26:42.488295+0000","last_clean":"2023-10-16T11:26:42.488295+0000","last_became_active":"2023-10-16T11:26:42.472966+0000","last_became_peered":"2023-10-16T11:26:42.472966+0000","last_unstale":"2023-10-16T11:26:42.488295+0000","last_undegraded":"2023-10-16T11:26:42.488295+0000","last_fullsized":"2023-10-16T11:26:42.488295+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:53.818915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.3b","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.448246+0000","last_change":"2023-10-16T11:26:42.468516+0000","last_active":"2023-10-16T11:26:43.448246+0000","last_peered":"2023-10-16T11:26:43.448246+0000","last_clean":"2023-10-16T11:26:43.448246+0000","last_became_active":"2023-10-16T11:26:42.467300+0000","last_became_peered":"2023-10-16T11:26:42.467300+0000","last_unstale":"2023-10-16T11:26:43.448246+0000","last_undegraded":"2023-10-16T11:26:43.448246+0000","last_fullsized":"2023-10-16T11:26:43.448246+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:22:02.870114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.3a","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.448107+0000","last_change":"2023-10-16T11:26:42.468662+0000","last_active":"2023-10-16T11:26:43.448107+0000","last_peered":"2023-10-16T11:26:43.448107+0000","last_clean":"2023-10-16T11:26:43.448107+0000","last_became_active":"2023-10-16T11:26:42.466694+0000","last_became_peered":"2023-10-16T11:26:42.466694+0000","last_unstale":"2023-10-16T11:26:43.448107+0000","last_undegraded":"2023-10-16T11:26:43.448107+0000","last_fullsized":"2023-10-16T11:26:43.448107+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:11:12.445109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.39","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.013584+0000","last_change":"2023-10-16T11:26:42.498390+0000","last_active":"2023-10-16T11:26:44.013584+0000","last_peered":"2023-10-16T11:26:44.013584+0000","last_clean":"2023-10-16T11:26:44.013584+0000","last_became_active":"2023-10-16T11:26:42.497340+0000","last_became_peered":"2023-10-16T11:26:42.497340+0000","last_unstale":"2023-10-16T11:26:44.013584+0000","last_undegraded":"2023-10-16T11:26:44.013584+0000","last_fullsized":"2023-10-16T11:26:44.013584+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:07.920676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.38","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.013710+0000","last_change":"2023-10-16T11:26:42.483168+0000","last_active":"2023-10-16T11:26:44.013710+0000","last_peered":"2023-10-16T11:26:44.013710+0000","last_clean":"2023-10-16T11:26:44.013710+0000","last_became_active":"2023-10-16T11:26:42.481102+0000","last_became_peered":"2023-10-16T11:26:42.481102+0000","last_unstale":"2023-10-16T11:26:44.013710+0000","last_undegraded":"2023-10-16T11:26:44.013710+0000","last_fullsized":"2023-10-16T11:26:44.013710+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:55.860376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.37","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.446267+0000","last_change":"2023-10-16T11:26:42.460799+0000","last_active":"2023-10-16T11:26:43.446267+0000","last_peered":"2023-10-16T11:26:43.446267+0000","last_clean":"2023-10-16T11:26:43.446267+0000","last_became_active":"2023-10-16T11:26:42.460209+0000","last_became_peered":"2023-10-16T11:26:42.460209+0000","last_unstale":"2023-10-16T11:26:43.446267+0000","last_undegraded":"2023-10-16T11:26:43.446267+0000","last_fullsized":"2023-10-16T11:26:43.446267+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:52.907745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.36","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.488163+0000","last_change":"2023-10-16T11:26:42.480358+0000","last_active":"2023-10-16T11:26:42.488163+0000","last_peered":"2023-10-16T11:26:42.488163+0000","last_clean":"2023-10-16T11:26:42.488163+0000","last_became_active":"2023-10-16T11:26:42.480004+0000","last_became_peered":"2023-10-16T11:26:42.480004+0000","last_unstale":"2023-10-16T11:26:42.488163+0000","last_undegraded":"2023-10-16T11:26:42.488163+0000","last_fullsized":"2023-10-16T11:26:42.488163+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:56:20.149184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.35","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.057906+0000","last_change":"2023-10-16T11:26:42.489625+0000","last_active":"2023-10-16T11:26:44.057906+0000","last_peered":"2023-10-16T11:26:44.057906+0000","last_clean":"2023-10-16T11:26:44.057906+0000","last_became_active":"2023-10-16T11:26:42.488751+0000","last_became_peered":"2023-10-16T11:26:42.488751+0000","last_unstale":"2023-10-16T11:26:44.057906+0000","last_undegraded":"2023-10-16T11:26:44.057906+0000","last_fullsized":"2023-10-16T11:26:44.057906+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:05.823763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.34","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.479147+0000","last_change":"2023-10-16T11:26:42.477091+0000","last_active":"2023-10-16T11:26:42.479147+0000","last_peered":"2023-10-16T11:26:42.479147+0000","last_clean":"2023-10-16T11:26:42.479147+0000","last_became_active":"2023-10-16T11:26:42.476019+0000","last_became_peered":"2023-10-16T11:26:42.476019+0000","last_unstale":"2023-10-16T11:26:42.479147+0000","last_undegraded":"2023-10-16T11:26:42.479147+0000","last_fullsized":"2023-10-16T11:26:42.479147+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:44:41.622265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.33","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.481313+0000","last_change":"2023-10-16T11:26:42.478109+0000","last_active":"2023-10-16T11:26:42.481313+0000","last_peered":"2023-10-16T11:26:42.481313+0000","last_clean":"2023-10-16T11:26:42.481313+0000","last_became_active":"2023-10-16T11:26:42.477759+0000","last_became_peered":"2023-10-16T11:26:42.477759+0000","last_unstale":"2023-10-16T11:26:42.481313+0000","last_undegraded":"2023-10-16T11:26:42.481313+0000","last_fullsized":"2023-10-16T11:26:42.481313+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:40.299496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.32","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.494543+0000","last_change":"2023-10-16T11:26:42.491655+0000","last_active":"2023-10-16T11:26:42.494543+0000","last_peered":"2023-10-16T11:26:42.494543+0000","last_clean":"2023-10-16T11:26:42.494543+0000","last_became_active":"2023-10-16T11:26:42.491347+0000","last_became_peered":"2023-10-16T11:26:42.491347+0000","last_unstale":"2023-10-16T11:26:42.494543+0000","last_undegraded":"2023-10-16T11:26:42.494543+0000","last_fullsized":"2023-10-16T11:26:42.494543+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:39:35.732428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.31","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.013983+0000","last_change":"2023-10-16T11:26:42.479023+0000","last_active":"2023-10-16T11:26:44.013983+0000","last_peered":"2023-10-16T11:26:44.013983+0000","last_clean":"2023-10-16T11:26:44.013983+0000","last_became_active":"2023-10-16T11:26:42.478494+0000","last_became_peered":"2023-10-16T11:26:42.478494+0000","last_unstale":"2023-10-16T11:26:44.013983+0000","last_undegraded":"2023-10-16T11:26:44.013983+0000","last_fullsized":"2023-10-16T11:26:44.013983+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:02:44.250353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.30","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.013598+0000","last_change":"2023-10-16T11:26:42.474989+0000","last_active":"2023-10-16T11:26:44.013598+0000","last_peered":"2023-10-16T11:26:44.013598+0000","last_clean":"2023-10-16T11:26:44.013598+0000","last_became_active":"2023-10-16T11:26:42.474697+0000","last_became_peered":"2023-10-16T11:26:42.474697+0000","last_unstale":"2023-10-16T11:26:44.013598+0000","last_undegraded":"2023-10-16T11:26:44.013598+0000","last_fullsized":"2023-10-16T11:26:44.013598+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:11.816797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.2f","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.446435+0000","last_change":"2023-10-16T11:26:42.477897+0000","last_active":"2023-10-16T11:26:43.446435+0000","last_peered":"2023-10-16T11:26:43.446435+0000","last_clean":"2023-10-16T11:26:43.446435+0000","last_became_active":"2023-10-16T11:26:42.477574+0000","last_became_peered":"2023-10-16T11:26:42.477574+0000","last_unstale":"2023-10-16T11:26:43.446435+0000","last_undegraded":"2023-10-16T11:26:43.446435+0000","last_fullsized":"2023-10-16T11:26:43.446435+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:02.925578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.2e","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.488004+0000","last_change":"2023-10-16T11:26:42.482627+0000","last_active":"2023-10-16T11:26:42.488004+0000","last_peered":"2023-10-16T11:26:42.488004+0000","last_clean":"2023-10-16T11:26:42.488004+0000","last_became_active":"2023-10-16T11:26:42.482369+0000","last_became_peered":"2023-10-16T11:26:42.482369+0000","last_unstale":"2023-10-16T11:26:42.488004+0000","last_undegraded":"2023-10-16T11:26:42.488004+0000","last_fullsized":"2023-10-16T11:26:42.488004+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:35.685825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.2d","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.013675+0000","last_change":"2023-10-16T11:26:42.496202+0000","last_active":"2023-10-16T11:26:44.013675+0000","last_peered":"2023-10-16T11:26:44.013675+0000","last_clean":"2023-10-16T11:26:44.013675+0000","last_became_active":"2023-10-16T11:26:42.495910+0000","last_became_peered":"2023-10-16T11:26:42.495910+0000","last_unstale":"2023-10-16T11:26:44.013675+0000","last_undegraded":"2023-10-16T11:26:44.013675+0000","last_fullsized":"2023-10-16T11:26:44.013675+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:29:18.145001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.2c","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.013602+0000","last_change":"2023-10-16T11:26:42.483449+0000","last_active":"2023-10-16T11:26:44.013602+0000","last_peered":"2023-10-16T11:26:44.013602+0000","last_clean":"2023-10-16T11:26:44.013602+0000","last_became_active":"2023-10-16T11:26:42.482770+0000","last_became_peered":"2023-10-16T11:26:42.482770+0000","last_unstale":"2023-10-16T11:26:44.013602+0000","last_undegraded":"2023-10-16T11:26:44.013602+0000","last_fullsized":"2023-10-16T11:26:44.013602+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:36.348053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.2b","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.481696+0000","last_change":"2023-10-16T11:26:42.478965+0000","last_active":"2023-10-16T11:26:42.481696+0000","last_peered":"2023-10-16T11:26:42.481696+0000","last_clean":"2023-10-16T11:26:42.481696+0000","last_became_active":"2023-10-16T11:26:42.478648+0000","last_became_peered":"2023-10-16T11:26:42.478648+0000","last_unstale":"2023-10-16T11:26:42.481696+0000","last_undegraded":"2023-10-16T11:26:42.481696+0000","last_fullsized":"2023-10-16T11:26:42.481696+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:39:35.732428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.2a","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.448286+0000","last_change":"2023-10-16T11:26:42.487879+0000","last_active":"2023-10-16T11:26:43.448286+0000","last_peered":"2023-10-16T11:26:43.448286+0000","last_clean":"2023-10-16T11:26:43.448286+0000","last_became_active":"2023-10-16T11:26:42.487472+0000","last_became_peered":"2023-10-16T11:26:42.487472+0000","last_unstale":"2023-10-16T11:26:43.448286+0000","last_undegraded":"2023-10-16T11:26:43.448286+0000","last_fullsized":"2023-10-16T11:26:43.448286+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:52.117654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.29","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.449755+0000","last_change":"2023-10-16T11:26:42.480757+0000","last_active":"2023-10-16T11:26:43.449755+0000","last_peered":"2023-10-16T11:26:43.449755+0000","last_clean":"2023-10-16T11:26:43.449755+0000","last_became_active":"2023-10-16T11:26:42.480439+0000","last_became_peered":"2023-10-16T11:26:42.480439+0000","last_unstale":"2023-10-16T11:26:43.449755+0000","last_undegraded":"2023-10-16T11:26:43.449755+0000","last_fullsized":"2023-10-16T11:26:43.449755+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:25:33.957694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.28","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.447914+0000","last_change":"2023-10-16T11:26:42.493819+0000","last_active":"2023-10-16T11:26:43.447914+0000","last_peered":"2023-10-16T11:26:43.447914+0000","last_clean":"2023-10-16T11:26:43.447914+0000","last_became_active":"2023-10-16T11:26:42.493540+0000","last_became_peered":"2023-10-16T11:26:42.493540+0000","last_unstale":"2023-10-16T11:26:43.447914+0000","last_undegraded":"2023-10-16T11:26:43.447914+0000","last_fullsized":"2023-10-16T11:26:43.447914+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:34:39.047026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.27","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.450341+0000","last_change":"2023-10-16T11:26:42.478717+0000","last_active":"2023-10-16T11:26:43.450341+0000","last_peered":"2023-10-16T11:26:43.450341+0000","last_clean":"2023-10-16T11:26:43.450341+0000","last_became_active":"2023-10-16T11:26:42.478340+0000","last_became_peered":"2023-10-16T11:26:42.478340+0000","last_unstale":"2023-10-16T11:26:43.450341+0000","last_undegraded":"2023-10-16T11:26:43.450341+0000","last_fullsized":"2023-10-16T11:26:43.450341+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:28.887145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.26","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.481376+0000","last_change":"2023-10-16T11:26:42.477429+0000","last_active":"2023-10-16T11:26:42.481376+0000","last_peered":"2023-10-16T11:26:42.481376+0000","last_clean":"2023-10-16T11:26:42.481376+0000","last_became_active":"2023-10-16T11:26:42.475792+0000","last_became_peered":"2023-10-16T11:26:42.475792+0000","last_unstale":"2023-10-16T11:26:42.481376+0000","last_undegraded":"2023-10-16T11:26:42.481376+0000","last_fullsized":"2023-10-16T11:26:42.481376+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:34:50.863297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.25","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.014068+0000","last_change":"2023-10-16T11:26:42.495035+0000","last_active":"2023-10-16T11:26:44.014068+0000","last_peered":"2023-10-16T11:26:44.014068+0000","last_clean":"2023-10-16T11:26:44.014068+0000","last_became_active":"2023-10-16T11:26:42.491700+0000","last_became_peered":"2023-10-16T11:26:42.491700+0000","last_unstale":"2023-10-16T11:26:44.014068+0000","last_undegraded":"2023-10-16T11:26:44.014068+0000","last_fullsized":"2023-10-16T11:26:44.014068+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:45.250424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.24","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.465251+0000","last_change":"2023-10-16T11:26:42.459535+0000","last_active":"2023-10-16T11:26:42.465251+0000","last_peered":"2023-10-16T11:26:42.465251+0000","last_clean":"2023-10-16T11:26:42.465251+0000","last_became_active":"2023-10-16T11:26:42.458870+0000","last_became_peered":"2023-10-16T11:26:42.458870+0000","last_unstale":"2023-10-16T11:26:42.465251+0000","last_undegraded":"2023-10-16T11:26:42.465251+0000","last_fullsized":"2023-10-16T11:26:42.465251+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:52:06.837078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.23","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.014022+0000","last_change":"2023-10-16T11:26:42.496413+0000","last_active":"2023-10-16T11:26:44.014022+0000","last_peered":"2023-10-16T11:26:44.014022+0000","last_clean":"2023-10-16T11:26:44.014022+0000","last_became_active":"2023-10-16T11:26:42.496103+0000","last_became_peered":"2023-10-16T11:26:42.496103+0000","last_unstale":"2023-10-16T11:26:44.014022+0000","last_undegraded":"2023-10-16T11:26:44.014022+0000","last_fullsized":"2023-10-16T11:26:44.014022+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:14.531480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.21","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.014313+0000","last_change":"2023-10-16T11:26:42.497225+0000","last_active":"2023-10-16T11:26:44.014313+0000","last_peered":"2023-10-16T11:26:44.014313+0000","last_clean":"2023-10-16T11:26:44.014313+0000","last_became_active":"2023-10-16T11:26:42.496582+0000","last_became_peered":"2023-10-16T11:26:42.496582+0000","last_unstale":"2023-10-16T11:26:44.014313+0000","last_undegraded":"2023-10-16T11:26:44.014313+0000","last_fullsized":"2023-10-16T11:26:44.014313+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:19:15.546077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.20","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.445526+0000","last_change":"2023-10-16T11:26:42.458807+0000","last_active":"2023-10-16T11:26:43.445526+0000","last_peered":"2023-10-16T11:26:43.445526+0000","last_clean":"2023-10-16T11:26:43.445526+0000","last_became_active":"2023-10-16T11:26:42.457861+0000","last_became_peered":"2023-10-16T11:26:42.457861+0000","last_unstale":"2023-10-16T11:26:43.445526+0000","last_undegraded":"2023-10-16T11:26:43.445526+0000","last_fullsized":"2023-10-16T11:26:43.445526+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:18:41.473131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1e","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.494383+0000","last_change":"2023-10-16T11:26:42.492923+0000","last_active":"2023-10-16T11:26:42.494383+0000","last_peered":"2023-10-16T11:26:42.494383+0000","last_clean":"2023-10-16T11:26:42.494383+0000","last_became_active":"2023-10-16T11:26:42.491679+0000","last_became_peered":"2023-10-16T11:26:42.491679+0000","last_unstale":"2023-10-16T11:26:42.494383+0000","last_undegraded":"2023-10-16T11:26:42.494383+0000","last_fullsized":"2023-10-16T11:26:42.494383+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:56:20.149184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1f","version":"256'4","reported_seq":15,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.495620+0000","last_change":"2023-10-16T11:26:41.472041+0000","last_active":"2023-10-16T11:26:42.495620+0000","last_peered":"2023-10-16T11:26:42.495620+0000","last_clean":"2023-10-16T11:26:42.495620+0000","last_became_active":"2023-10-16T11:26:41.471494+0000","last_became_peered":"2023-10-16T11:26:41.471494+0000","last_unstale":"2023-10-16T11:26:42.495620+0000","last_undegraded":"2023-10-16T11:26:42.495620+0000","last_fullsized":"2023-10-16T11:26:42.495620+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:33.468817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1f","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.058325+0000","last_change":"2023-10-16T11:26:42.485850+0000","last_active":"2023-10-16T11:26:44.058325+0000","last_peered":"2023-10-16T11:26:44.058325+0000","last_clean":"2023-10-16T11:26:44.058325+0000","last_became_active":"2023-10-16T11:26:42.485492+0000","last_became_peered":"2023-10-16T11:26:42.485492+0000","last_unstale":"2023-10-16T11:26:44.058325+0000","last_undegraded":"2023-10-16T11:26:44.058325+0000","last_fullsized":"2023-10-16T11:26:44.058325+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:54:36.826735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.1e","version":"256'1","reported_seq":13,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.493251+0000","last_change":"2023-10-16T11:26:41.463055+0000","last_active":"2023-10-16T11:26:42.493251+0000","last_peered":"2023-10-16T11:26:42.493251+0000","last_clean":"2023-10-16T11:26:42.493251+0000","last_became_active":"2023-10-16T11:26:41.462240+0000","last_became_peered":"2023-10-16T11:26:41.462240+0000","last_unstale":"2023-10-16T11:26:42.493251+0000","last_undegraded":"2023-10-16T11:26:42.493251+0000","last_fullsized":"2023-10-16T11:26:42.493251+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:48:33.977514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1c","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.448426+0000","last_change":"2023-10-16T11:26:42.461151+0000","last_active":"2023-10-16T11:26:43.448426+0000","last_peered":"2023-10-16T11:26:43.448426+0000","last_clean":"2023-10-16T11:26:43.448426+0000","last_became_active":"2023-10-16T11:26:42.460058+0000","last_became_peered":"2023-10-16T11:26:42.460058+0000","last_unstale":"2023-10-16T11:26:43.448426+0000","last_undegraded":"2023-10-16T11:26:43.448426+0000","last_fullsized":"2023-10-16T11:26:43.448426+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:02:25.324063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1d","version":"256'2","reported_seq":14,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.496583+0000","last_change":"2023-10-16T11:26:41.472513+0000","last_active":"2023-10-16T11:26:42.496583+0000","last_peered":"2023-10-16T11:26:42.496583+0000","last_clean":"2023-10-16T11:26:42.496583+0000","last_became_active":"2023-10-16T11:26:41.472233+0000","last_became_peered":"2023-10-16T11:26:41.472233+0000","last_unstale":"2023-10-16T11:26:42.496583+0000","last_undegraded":"2023-10-16T11:26:42.496583+0000","last_fullsized":"2023-10-16T11:26:42.496583+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:44:40.609763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1d","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.448887+0000","last_change":"2023-10-16T11:26:42.461915+0000","last_active":"2023-10-16T11:26:43.448887+0000","last_peered":"2023-10-16T11:26:43.448887+0000","last_clean":"2023-10-16T11:26:43.448887+0000","last_became_active":"2023-10-16T11:26:42.461183+0000","last_became_peered":"2023-10-16T11:26:42.461183+0000","last_unstale":"2023-10-16T11:26:43.448887+0000","last_undegraded":"2023-10-16T11:26:43.448887+0000","last_fullsized":"2023-10-16T11:26:43.448887+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:54:30.752990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"256'2","reported_seq":14,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.502802+0000","last_change":"2023-10-16T11:26:41.461721+0000","last_active":"2023-10-16T11:26:42.502802+0000","last_peered":"2023-10-16T11:26:42.502802+0000","last_clean":"2023-10-16T11:26:42.502802+0000","last_became_active":"2023-10-16T11:26:41.461381+0000","last_became_peered":"2023-10-16T11:26:41.461381+0000","last_unstale":"2023-10-16T11:26:42.502802+0000","last_undegraded":"2023-10-16T11:26:42.502802+0000","last_fullsized":"2023-10-16T11:26:42.502802+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:01.913076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.1a","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.450069+0000","last_change":"2023-10-16T11:26:42.472572+0000","last_active":"2023-10-16T11:26:43.450069+0000","last_peered":"2023-10-16T11:26:43.450069+0000","last_clean":"2023-10-16T11:26:43.450069+0000","last_became_active":"2023-10-16T11:26:42.472143+0000","last_became_peered":"2023-10-16T11:26:42.472143+0000","last_unstale":"2023-10-16T11:26:43.450069+0000","last_undegraded":"2023-10-16T11:26:43.450069+0000","last_fullsized":"2023-10-16T11:26:43.450069+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:32:11.353644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1b","version":"256'1","reported_seq":13,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.498815+0000","last_change":"2023-10-16T11:26:41.466268+0000","last_active":"2023-10-16T11:26:42.498815+0000","last_peered":"2023-10-16T11:26:42.498815+0000","last_clean":"2023-10-16T11:26:42.498815+0000","last_became_active":"2023-10-16T11:26:41.465918+0000","last_became_peered":"2023-10-16T11:26:41.465918+0000","last_unstale":"2023-10-16T11:26:42.498815+0000","last_undegraded":"2023-10-16T11:26:42.498815+0000","last_fullsized":"2023-10-16T11:26:42.498815+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:15.325814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.1b","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.014413+0000","last_change":"2023-10-16T11:26:42.482948+0000","last_active":"2023-10-16T11:26:44.014413+0000","last_peered":"2023-10-16T11:26:44.014413+0000","last_clean":"2023-10-16T11:26:44.014413+0000","last_became_active":"2023-10-16T11:26:42.481415+0000","last_became_peered":"2023-10-16T11:26:42.481415+0000","last_unstale":"2023-10-16T11:26:44.014413+0000","last_undegraded":"2023-10-16T11:26:44.014413+0000","last_fullsized":"2023-10-16T11:26:44.014413+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:22:26.931814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1a","version":"256'1","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.467576+0000","last_change":"2023-10-16T11:26:41.470192+0000","last_active":"2023-10-16T11:26:42.467576+0000","last_peered":"2023-10-16T11:26:42.467576+0000","last_clean":"2023-10-16T11:26:42.467576+0000","last_became_active":"2023-10-16T11:26:41.469882+0000","last_became_peered":"2023-10-16T11:26:41.469882+0000","last_unstale":"2023-10-16T11:26:42.467576+0000","last_undegraded":"2023-10-16T11:26:42.467576+0000","last_fullsized":"2023-10-16T11:26:42.467576+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:13.411479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.18","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.445598+0000","last_change":"2023-10-16T11:26:42.462722+0000","last_active":"2023-10-16T11:26:43.445598+0000","last_peered":"2023-10-16T11:26:43.445598+0000","last_clean":"2023-10-16T11:26:43.445598+0000","last_became_active":"2023-10-16T11:26:42.459747+0000","last_became_peered":"2023-10-16T11:26:42.459747+0000","last_unstale":"2023-10-16T11:26:43.445598+0000","last_undegraded":"2023-10-16T11:26:43.445598+0000","last_fullsized":"2023-10-16T11:26:43.445598+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:34.481319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.474658+0000","last_change":"2023-10-16T11:26:41.473722+0000","last_active":"2023-10-16T11:26:42.474658+0000","last_peered":"2023-10-16T11:26:42.474658+0000","last_clean":"2023-10-16T11:26:42.474658+0000","last_became_active":"2023-10-16T11:26:41.473402+0000","last_became_peered":"2023-10-16T11:26:41.473402+0000","last_unstale":"2023-10-16T11:26:42.474658+0000","last_undegraded":"2023-10-16T11:26:42.474658+0000","last_fullsized":"2023-10-16T11:26:42.474658+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:12:53.432070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.19","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.487717+0000","last_change":"2023-10-16T11:26:42.476892+0000","last_active":"2023-10-16T11:26:42.487717+0000","last_peered":"2023-10-16T11:26:42.487717+0000","last_clean":"2023-10-16T11:26:42.487717+0000","last_became_active":"2023-10-16T11:26:42.476613+0000","last_became_peered":"2023-10-16T11:26:42.476613+0000","last_unstale":"2023-10-16T11:26:42.487717+0000","last_undegraded":"2023-10-16T11:26:42.487717+0000","last_fullsized":"2023-10-16T11:26:42.487717+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:07.002504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.470425+0000","last_change":"2023-10-16T11:26:41.461932+0000","last_active":"2023-10-16T11:26:42.470425+0000","last_peered":"2023-10-16T11:26:42.470425+0000","last_clean":"2023-10-16T11:26:42.470425+0000","last_became_active":"2023-10-16T11:26:41.461519+0000","last_became_peered":"2023-10-16T11:26:41.461519+0000","last_unstale":"2023-10-16T11:26:42.470425+0000","last_undegraded":"2023-10-16T11:26:42.470425+0000","last_fullsized":"2023-10-16T11:26:42.470425+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:05:59.649006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.16","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.487894+0000","last_change":"2023-10-16T11:26:42.477778+0000","last_active":"2023-10-16T11:26:42.487894+0000","last_peered":"2023-10-16T11:26:42.487894+0000","last_clean":"2023-10-16T11:26:42.487894+0000","last_became_active":"2023-10-16T11:26:42.474986+0000","last_became_peered":"2023-10-16T11:26:42.474986+0000","last_unstale":"2023-10-16T11:26:42.487894+0000","last_undegraded":"2023-10-16T11:26:42.487894+0000","last_fullsized":"2023-10-16T11:26:42.487894+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:13.852158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.452129+0000","last_change":"2023-10-16T11:26:41.470674+0000","last_active":"2023-10-16T11:26:42.452129+0000","last_peered":"2023-10-16T11:26:42.452129+0000","last_clean":"2023-10-16T11:26:42.452129+0000","last_became_active":"2023-10-16T11:26:41.470381+0000","last_became_peered":"2023-10-16T11:26:41.470381+0000","last_unstale":"2023-10-16T11:26:42.452129+0000","last_undegraded":"2023-10-16T11:26:42.452129+0000","last_fullsized":"2023-10-16T11:26:42.452129+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:17.143386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.17","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.488152+0000","last_change":"2023-10-16T11:26:42.482653+0000","last_active":"2023-10-16T11:26:42.488152+0000","last_peered":"2023-10-16T11:26:42.488152+0000","last_clean":"2023-10-16T11:26:42.488152+0000","last_became_active":"2023-10-16T11:26:42.482384+0000","last_became_peered":"2023-10-16T11:26:42.482384+0000","last_unstale":"2023-10-16T11:26:42.488152+0000","last_undegraded":"2023-10-16T11:26:42.488152+0000","last_fullsized":"2023-10-16T11:26:42.488152+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:17:36.001896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.463153+0000","last_change":"2023-10-16T11:26:41.464349+0000","last_active":"2023-10-16T11:26:42.463153+0000","last_peered":"2023-10-16T11:26:42.463153+0000","last_clean":"2023-10-16T11:26:42.463153+0000","last_became_active":"2023-10-16T11:26:41.464050+0000","last_became_peered":"2023-10-16T11:26:41.464050+0000","last_unstale":"2023-10-16T11:26:42.463153+0000","last_undegraded":"2023-10-16T11:26:42.463153+0000","last_fullsized":"2023-10-16T11:26:42.463153+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:44:19.380476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.14","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.058038+0000","last_change":"2023-10-16T11:26:42.479550+0000","last_active":"2023-10-16T11:26:44.058038+0000","last_peered":"2023-10-16T11:26:44.058038+0000","last_clean":"2023-10-16T11:26:44.058038+0000","last_became_active":"2023-10-16T11:26:42.479264+0000","last_became_peered":"2023-10-16T11:26:42.479264+0000","last_unstale":"2023-10-16T11:26:44.058038+0000","last_undegraded":"2023-10-16T11:26:44.058038+0000","last_fullsized":"2023-10-16T11:26:44.058038+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:23.444814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.15","version":"256'3","reported_seq":15,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.499634+0000","last_change":"2023-10-16T11:26:41.455346+0000","last_active":"2023-10-16T11:26:42.499634+0000","last_peered":"2023-10-16T11:26:42.499634+0000","last_clean":"2023-10-16T11:26:42.499634+0000","last_became_active":"2023-10-16T11:26:41.455005+0000","last_became_peered":"2023-10-16T11:26:41.455005+0000","last_unstale":"2023-10-16T11:26:42.499634+0000","last_undegraded":"2023-10-16T11:26:42.499634+0000","last_fullsized":"2023-10-16T11:26:42.499634+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:39:34.719926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.15","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.477208+0000","last_change":"2023-10-16T11:26:42.472647+0000","last_active":"2023-10-16T11:26:42.477208+0000","last_peered":"2023-10-16T11:26:42.477208+0000","last_clean":"2023-10-16T11:26:42.477208+0000","last_became_active":"2023-10-16T11:26:42.471549+0000","last_became_peered":"2023-10-16T11:26:42.471549+0000","last_unstale":"2023-10-16T11:26:42.477208+0000","last_undegraded":"2023-10-16T11:26:42.477208+0000","last_fullsized":"2023-10-16T11:26:42.477208+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:44:41.622265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.14","version":"256'6","reported_seq":18,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.501699+0000","last_change":"2023-10-16T11:26:41.469567+0000","last_active":"2023-10-16T11:26:42.501699+0000","last_peered":"2023-10-16T11:26:42.501699+0000","last_clean":"2023-10-16T11:26:42.501699+0000","last_became_active":"2023-10-16T11:26:41.469190+0000","last_became_peered":"2023-10-16T11:26:41.469190+0000","last_unstale":"2023-10-16T11:26:42.501699+0000","last_undegraded":"2023-10-16T11:26:42.501699+0000","last_fullsized":"2023-10-16T11:26:42.501699+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:50:17.705254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.12","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.014304+0000","last_change":"2023-10-16T11:26:42.482507+0000","last_active":"2023-10-16T11:26:44.014304+0000","last_peered":"2023-10-16T11:26:44.014304+0000","last_clean":"2023-10-16T11:26:44.014304+0000","last_became_active":"2023-10-16T11:26:42.480558+0000","last_became_peered":"2023-10-16T11:26:42.480558+0000","last_unstale":"2023-10-16T11:26:44.014304+0000","last_undegraded":"2023-10-16T11:26:44.014304+0000","last_fullsized":"2023-10-16T11:26:44.014304+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:09.387747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.13","version":"256'2","reported_seq":14,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.498622+0000","last_change":"2023-10-16T11:26:41.459360+0000","last_active":"2023-10-16T11:26:42.498622+0000","last_peered":"2023-10-16T11:26:42.498622+0000","last_clean":"2023-10-16T11:26:42.498622+0000","last_became_active":"2023-10-16T11:26:41.459076+0000","last_became_peered":"2023-10-16T11:26:41.459076+0000","last_unstale":"2023-10-16T11:26:42.498622+0000","last_undegraded":"2023-10-16T11:26:42.498622+0000","last_fullsized":"2023-10-16T11:26:42.498622+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:21.456069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.13","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.492101+0000","last_change":"2023-10-16T11:26:42.488436+0000","last_active":"2023-10-16T11:26:42.492101+0000","last_peered":"2023-10-16T11:26:42.492101+0000","last_clean":"2023-10-16T11:26:42.492101+0000","last_became_active":"2023-10-16T11:26:42.488148+0000","last_became_peered":"2023-10-16T11:26:42.488148+0000","last_unstale":"2023-10-16T11:26:42.492101+0000","last_undegraded":"2023-10-16T11:26:42.492101+0000","last_fullsized":"2023-10-16T11:26:42.492101+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:05:35.518664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.466683+0000","last_change":"2023-10-16T11:26:41.473074+0000","last_active":"2023-10-16T11:26:42.466683+0000","last_peered":"2023-10-16T11:26:42.466683+0000","last_clean":"2023-10-16T11:26:42.466683+0000","last_became_active":"2023-10-16T11:26:41.472333+0000","last_became_peered":"2023-10-16T11:26:41.472333+0000","last_unstale":"2023-10-16T11:26:42.466683+0000","last_undegraded":"2023-10-16T11:26:42.466683+0000","last_fullsized":"2023-10-16T11:26:42.466683+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:04:26.707239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.10","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.057265+0000","last_change":"2023-10-16T11:26:42.489797+0000","last_active":"2023-10-16T11:26:44.057265+0000","last_peered":"2023-10-16T11:26:44.057265+0000","last_clean":"2023-10-16T11:26:44.057265+0000","last_became_active":"2023-10-16T11:26:42.489497+0000","last_became_peered":"2023-10-16T11:26:42.489497+0000","last_unstale":"2023-10-16T11:26:44.057265+0000","last_undegraded":"2023-10-16T11:26:44.057265+0000","last_fullsized":"2023-10-16T11:26:44.057265+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:33:35.976291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.449349+0000","last_change":"2023-10-16T11:26:41.469759+0000","last_active":"2023-10-16T11:26:42.449349+0000","last_peered":"2023-10-16T11:26:42.449349+0000","last_clean":"2023-10-16T11:26:42.449349+0000","last_became_active":"2023-10-16T11:26:41.469472+0000","last_became_peered":"2023-10-16T11:26:41.469472+0000","last_unstale":"2023-10-16T11:26:42.449349+0000","last_undegraded":"2023-10-16T11:26:42.449349+0000","last_fullsized":"2023-10-16T11:26:42.449349+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:48:33.977514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.11","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.014508+0000","last_change":"2023-10-16T11:26:42.478827+0000","last_active":"2023-10-16T11:26:44.014508+0000","last_peered":"2023-10-16T11:26:44.014508+0000","last_clean":"2023-10-16T11:26:44.014508+0000","last_became_active":"2023-10-16T11:26:42.478081+0000","last_became_peered":"2023-10-16T11:26:42.478081+0000","last_unstale":"2023-10-16T11:26:44.014508+0000","last_undegraded":"2023-10-16T11:26:44.014508+0000","last_fullsized":"2023-10-16T11:26:44.014508+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:23:23.981006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.10","version":"256'8","reported_seq":20,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.501896+0000","last_change":"2023-10-16T11:26:41.465360+0000","last_active":"2023-10-16T11:26:42.501896+0000","last_peered":"2023-10-16T11:26:42.501896+0000","last_clean":"2023-10-16T11:26:42.501896+0000","last_became_active":"2023-10-16T11:26:41.465065+0000","last_became_peered":"2023-10-16T11:26:41.465065+0000","last_unstale":"2023-10-16T11:26:42.501896+0000","last_undegraded":"2023-10-16T11:26:42.501896+0000","last_fullsized":"2023-10-16T11:26:42.501896+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:35:26.885012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.e","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.450025+0000","last_change":"2023-10-16T11:26:42.470088+0000","last_active":"2023-10-16T11:26:43.450025+0000","last_peered":"2023-10-16T11:26:43.450025+0000","last_clean":"2023-10-16T11:26:43.450025+0000","last_became_active":"2023-10-16T11:26:42.469799+0000","last_became_peered":"2023-10-16T11:26:42.469799+0000","last_unstale":"2023-10-16T11:26:43.450025+0000","last_undegraded":"2023-10-16T11:26:43.450025+0000","last_fullsized":"2023-10-16T11:26:43.450025+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:54:49.226718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"256'1","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.451515+0000","last_change":"2023-10-16T11:26:41.470130+0000","last_active":"2023-10-16T11:26:42.451515+0000","last_peered":"2023-10-16T11:26:42.451515+0000","last_clean":"2023-10-16T11:26:42.451515+0000","last_became_active":"2023-10-16T11:26:41.469725+0000","last_became_peered":"2023-10-16T11:26:41.469725+0000","last_unstale":"2023-10-16T11:26:42.451515+0000","last_undegraded":"2023-10-16T11:26:42.451515+0000","last_fullsized":"2023-10-16T11:26:42.451515+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:12:53.432070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.f","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.448743+0000","last_change":"2023-10-16T11:26:42.488062+0000","last_active":"2023-10-16T11:26:43.448743+0000","last_peered":"2023-10-16T11:26:43.448743+0000","last_clean":"2023-10-16T11:26:43.448743+0000","last_became_active":"2023-10-16T11:26:42.487772+0000","last_became_peered":"2023-10-16T11:26:42.487772+0000","last_unstale":"2023-10-16T11:26:43.448743+0000","last_undegraded":"2023-10-16T11:26:43.448743+0000","last_fullsized":"2023-10-16T11:26:43.448743+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:58:06.446678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.e","version":"256'2","reported_seq":14,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.493684+0000","last_change":"2023-10-16T11:26:41.473393+0000","last_active":"2023-10-16T11:26:42.493684+0000","last_peered":"2023-10-16T11:26:42.493684+0000","last_clean":"2023-10-16T11:26:42.493684+0000","last_became_active":"2023-10-16T11:26:41.473133+0000","last_became_peered":"2023-10-16T11:26:41.473133+0000","last_unstale":"2023-10-16T11:26:42.493684+0000","last_undegraded":"2023-10-16T11:26:42.493684+0000","last_fullsized":"2023-10-16T11:26:42.493684+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:01:53.010279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.c","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.488666+0000","last_change":"2023-10-16T11:26:42.483173+0000","last_active":"2023-10-16T11:26:42.488666+0000","last_peered":"2023-10-16T11:26:42.488666+0000","last_clean":"2023-10-16T11:26:42.488666+0000","last_became_active":"2023-10-16T11:26:42.482420+0000","last_became_peered":"2023-10-16T11:26:42.482420+0000","last_unstale":"2023-10-16T11:26:42.488666+0000","last_undegraded":"2023-10-16T11:26:42.488666+0000","last_fullsized":"2023-10-16T11:26:42.488666+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:46:50.175836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.d","version":"256'1","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.448859+0000","last_change":"2023-10-16T11:26:41.472146+0000","last_active":"2023-10-16T11:26:42.448859+0000","last_peered":"2023-10-16T11:26:42.448859+0000","last_clean":"2023-10-16T11:26:42.448859+0000","last_became_active":"2023-10-16T11:26:41.471866+0000","last_became_peered":"2023-10-16T11:26:41.471866+0000","last_unstale":"2023-10-16T11:26:42.448859+0000","last_undegraded":"2023-10-16T11:26:42.448859+0000","last_fullsized":"2023-10-16T11:26:42.448859+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:39.286994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.d","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.449033+0000","last_change":"2023-10-16T11:26:42.476458+0000","last_active":"2023-10-16T11:26:43.449033+0000","last_peered":"2023-10-16T11:26:43.449033+0000","last_clean":"2023-10-16T11:26:43.449033+0000","last_became_active":"2023-10-16T11:26:42.476137+0000","last_became_peered":"2023-10-16T11:26:42.476137+0000","last_unstale":"2023-10-16T11:26:43.449033+0000","last_undegraded":"2023-10-16T11:26:43.449033+0000","last_fullsized":"2023-10-16T11:26:43.449033+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:49.829385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.448474+0000","last_change":"2023-10-16T11:26:41.474290+0000","last_active":"2023-10-16T11:26:42.448474+0000","last_peered":"2023-10-16T11:26:42.448474+0000","last_clean":"2023-10-16T11:26:42.448474+0000","last_became_active":"2023-10-16T11:26:41.473903+0000","last_became_peered":"2023-10-16T11:26:41.473903+0000","last_unstale":"2023-10-16T11:26:42.448474+0000","last_undegraded":"2023-10-16T11:26:42.448474+0000","last_fullsized":"2023-10-16T11:26:42.448474+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:34:49.850795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.a","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.445770+0000","last_change":"2023-10-16T11:26:42.461725+0000","last_active":"2023-10-16T11:26:43.445770+0000","last_peered":"2023-10-16T11:26:43.445770+0000","last_clean":"2023-10-16T11:26:43.445770+0000","last_became_active":"2023-10-16T11:26:42.458470+0000","last_became_peered":"2023-10-16T11:26:42.458470+0000","last_unstale":"2023-10-16T11:26:43.445770+0000","last_undegraded":"2023-10-16T11:26:43.445770+0000","last_fullsized":"2023-10-16T11:26:43.445770+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:47:31.251808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.b","version":"256'2","reported_seq":13,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.493775+0000","last_change":"2023-10-16T11:26:41.454416+0000","last_active":"2023-10-16T11:26:42.493775+0000","last_peered":"2023-10-16T11:26:42.493775+0000","last_clean":"2023-10-16T11:26:42.493775+0000","last_became_active":"2023-10-16T11:26:41.454134+0000","last_became_peered":"2023-10-16T11:26:41.454134+0000","last_unstale":"2023-10-16T11:26:42.493775+0000","last_undegraded":"2023-10-16T11:26:42.493775+0000","last_fullsized":"2023-10-16T11:26:42.493775+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:18:40.460629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.b","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.487549+0000","last_change":"2023-10-16T11:26:42.476361+0000","last_active":"2023-10-16T11:26:42.487549+0000","last_peered":"2023-10-16T11:26:42.487549+0000","last_clean":"2023-10-16T11:26:42.487549+0000","last_became_active":"2023-10-16T11:26:42.475996+0000","last_became_peered":"2023-10-16T11:26:42.475996+0000","last_unstale":"2023-10-16T11:26:42.487549+0000","last_undegraded":"2023-10-16T11:26:42.487549+0000","last_fullsized":"2023-10-16T11:26:42.487549+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:10:29.358302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.454470+0000","last_change":"2023-10-16T11:26:41.471717+0000","last_active":"2023-10-16T11:26:42.454470+0000","last_peered":"2023-10-16T11:26:42.454470+0000","last_clean":"2023-10-16T11:26:42.454470+0000","last_became_active":"2023-10-16T11:26:41.471354+0000","last_became_peered":"2023-10-16T11:26:41.471354+0000","last_unstale":"2023-10-16T11:26:42.454470+0000","last_undegraded":"2023-10-16T11:26:42.454470+0000","last_fullsized":"2023-10-16T11:26:42.454470+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:05:53.729578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.8","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.492444+0000","last_change":"2023-10-16T11:26:42.490549+0000","last_active":"2023-10-16T11:26:42.492444+0000","last_peered":"2023-10-16T11:26:42.492444+0000","last_clean":"2023-10-16T11:26:42.492444+0000","last_became_active":"2023-10-16T11:26:42.489090+0000","last_became_peered":"2023-10-16T11:26:42.489090+0000","last_unstale":"2023-10-16T11:26:42.492444+0000","last_undegraded":"2023-10-16T11:26:42.492444+0000","last_fullsized":"2023-10-16T11:26:42.492444+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:18:47.880771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.449418+0000","last_change":"2023-10-16T11:26:41.473560+0000","last_active":"2023-10-16T11:26:42.449418+0000","last_peered":"2023-10-16T11:26:42.449418+0000","last_clean":"2023-10-16T11:26:42.449418+0000","last_became_active":"2023-10-16T11:26:41.472581+0000","last_became_peered":"2023-10-16T11:26:41.472581+0000","last_unstale":"2023-10-16T11:26:42.449418+0000","last_undegraded":"2023-10-16T11:26:42.449418+0000","last_fullsized":"2023-10-16T11:26:42.449418+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:05:59.649006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.9","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.484248+0000","last_change":"2023-10-16T11:26:42.483715+0000","last_active":"2023-10-16T11:26:42.484248+0000","last_peered":"2023-10-16T11:26:42.484248+0000","last_clean":"2023-10-16T11:26:42.484248+0000","last_became_active":"2023-10-16T11:26:42.483451+0000","last_became_peered":"2023-10-16T11:26:42.483451+0000","last_unstale":"2023-10-16T11:26:42.484248+0000","last_undegraded":"2023-10-16T11:26:42.484248+0000","last_fullsized":"2023-10-16T11:26:42.484248+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:53.818915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.470635+0000","last_change":"2023-10-16T11:26:41.466050+0000","last_active":"2023-10-16T11:26:42.470635+0000","last_peered":"2023-10-16T11:26:42.470635+0000","last_clean":"2023-10-16T11:26:42.470635+0000","last_became_active":"2023-10-16T11:26:41.465785+0000","last_became_peered":"2023-10-16T11:26:41.465785+0000","last_unstale":"2023-10-16T11:26:42.470635+0000","last_undegraded":"2023-10-16T11:26:42.470635+0000","last_fullsized":"2023-10-16T11:26:42.470635+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:17:34.989394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.6","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.013987+0000","last_change":"2023-10-16T11:26:42.490458+0000","last_active":"2023-10-16T11:26:44.013987+0000","last_peered":"2023-10-16T11:26:44.013987+0000","last_clean":"2023-10-16T11:26:44.013987+0000","last_became_active":"2023-10-16T11:26:42.490031+0000","last_became_peered":"2023-10-16T11:26:42.490031+0000","last_unstale":"2023-10-16T11:26:44.013987+0000","last_undegraded":"2023-10-16T11:26:44.013987+0000","last_fullsized":"2023-10-16T11:26:44.013987+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:12.245044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.7","version":"256'2","reported_seq":14,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.501289+0000","last_change":"2023-10-16T11:26:41.475372+0000","last_active":"2023-10-16T11:26:42.501289+0000","last_peered":"2023-10-16T11:26:42.501289+0000","last_clean":"2023-10-16T11:26:42.501289+0000","last_became_active":"2023-10-16T11:26:41.475008+0000","last_became_peered":"2023-10-16T11:26:41.475008+0000","last_unstale":"2023-10-16T11:26:42.501289+0000","last_undegraded":"2023-10-16T11:26:42.501289+0000","last_fullsized":"2023-10-16T11:26:42.501289+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:46:01.374748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.7","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.488195+0000","last_change":"2023-10-16T11:26:42.479530+0000","last_active":"2023-10-16T11:26:42.488195+0000","last_peered":"2023-10-16T11:26:42.488195+0000","last_clean":"2023-10-16T11:26:42.488195+0000","last_became_active":"2023-10-16T11:26:42.479206+0000","last_became_peered":"2023-10-16T11:26:42.479206+0000","last_unstale":"2023-10-16T11:26:42.488195+0000","last_undegraded":"2023-10-16T11:26:42.488195+0000","last_fullsized":"2023-10-16T11:26:42.488195+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:32:11.699728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.471557+0000","last_change":"2023-10-16T11:26:41.471074+0000","last_active":"2023-10-16T11:26:42.471557+0000","last_peered":"2023-10-16T11:26:42.471557+0000","last_clean":"2023-10-16T11:26:42.471557+0000","last_became_active":"2023-10-16T11:26:41.469399+0000","last_became_peered":"2023-10-16T11:26:41.469399+0000","last_unstale":"2023-10-16T11:26:42.471557+0000","last_undegraded":"2023-10-16T11:26:42.471557+0000","last_fullsized":"2023-10-16T11:26:42.471557+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:52:05.824577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.4","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.014002+0000","last_change":"2023-10-16T11:26:42.493533+0000","last_active":"2023-10-16T11:26:44.014002+0000","last_peered":"2023-10-16T11:26:44.014002+0000","last_clean":"2023-10-16T11:26:44.014002+0000","last_became_active":"2023-10-16T11:26:42.493244+0000","last_became_peered":"2023-10-16T11:26:42.493244+0000","last_unstale":"2023-10-16T11:26:44.014002+0000","last_undegraded":"2023-10-16T11:26:44.014002+0000","last_fullsized":"2023-10-16T11:26:44.014002+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:53:50.127580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.466387+0000","last_change":"2023-10-16T11:26:41.469347+0000","last_active":"2023-10-16T11:26:42.466387+0000","last_peered":"2023-10-16T11:26:42.466387+0000","last_clean":"2023-10-16T11:26:42.466387+0000","last_became_active":"2023-10-16T11:26:41.469012+0000","last_became_peered":"2023-10-16T11:26:41.469012+0000","last_unstale":"2023-10-16T11:26:42.466387+0000","last_undegraded":"2023-10-16T11:26:42.466387+0000","last_fullsized":"2023-10-16T11:26:42.466387+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:34:42.424697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.5","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.449204+0000","last_change":"2023-10-16T11:26:42.457693+0000","last_active":"2023-10-16T11:26:43.449204+0000","last_peered":"2023-10-16T11:26:43.449204+0000","last_clean":"2023-10-16T11:26:43.449204+0000","last_became_active":"2023-10-16T11:26:42.457094+0000","last_became_peered":"2023-10-16T11:26:42.457094+0000","last_unstale":"2023-10-16T11:26:43.449204+0000","last_undegraded":"2023-10-16T11:26:43.449204+0000","last_fullsized":"2023-10-16T11:26:43.449204+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:04:15.156293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.465322+0000","last_change":"2023-10-16T11:26:41.469699+0000","last_active":"2023-10-16T11:26:42.465322+0000","last_peered":"2023-10-16T11:26:42.465322+0000","last_clean":"2023-10-16T11:26:42.465322+0000","last_became_active":"2023-10-16T11:26:41.469373+0000","last_became_peered":"2023-10-16T11:26:41.469373+0000","last_unstale":"2023-10-16T11:26:42.465322+0000","last_undegraded":"2023-10-16T11:26:42.465322+0000","last_fullsized":"2023-10-16T11:26:42.465322+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:53:05.859368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.2","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:43.445993+0000","last_change":"2023-10-16T11:26:42.462890+0000","last_active":"2023-10-16T11:26:43.445993+0000","last_peered":"2023-10-16T11:26:43.445993+0000","last_clean":"2023-10-16T11:26:43.445993+0000","last_became_active":"2023-10-16T11:26:42.462368+0000","last_became_peered":"2023-10-16T11:26:42.462368+0000","last_unstale":"2023-10-16T11:26:43.445993+0000","last_undegraded":"2023-10-16T11:26:43.445993+0000","last_fullsized":"2023-10-16T11:26:43.445993+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:23.956560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.483323+0000","last_change":"2023-10-16T11:26:41.462500+0000","last_active":"2023-10-16T11:26:42.483323+0000","last_peered":"2023-10-16T11:26:42.483323+0000","last_clean":"2023-10-16T11:26:42.483323+0000","last_became_active":"2023-10-16T11:26:41.461669+0000","last_became_peered":"2023-10-16T11:26:41.461669+0000","last_unstale":"2023-10-16T11:26:42.483323+0000","last_undegraded":"2023-10-16T11:26:42.483323+0000","last_fullsized":"2023-10-16T11:26:42.483323+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:51.895243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.3","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.492080+0000","last_change":"2023-10-16T11:26:42.486393+0000","last_active":"2023-10-16T11:26:42.492080+0000","last_peered":"2023-10-16T11:26:42.492080+0000","last_clean":"2023-10-16T11:26:42.492080+0000","last_became_active":"2023-10-16T11:26:42.478616+0000","last_became_peered":"2023-10-16T11:26:42.478616+0000","last_unstale":"2023-10-16T11:26:42.492080+0000","last_undegraded":"2023-10-16T11:26:42.492080+0000","last_fullsized":"2023-10-16T11:26:42.492080+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:26:16.887346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.475669+0000","last_change":"2023-10-16T11:26:41.459862+0000","last_active":"2023-10-16T11:26:42.475669+0000","last_peered":"2023-10-16T11:26:42.475669+0000","last_clean":"2023-10-16T11:26:42.475669+0000","last_became_active":"2023-10-16T11:26:41.459567+0000","last_became_peered":"2023-10-16T11:26:41.459567+0000","last_unstale":"2023-10-16T11:26:42.475669+0000","last_undegraded":"2023-10-16T11:26:42.475669+0000","last_fullsized":"2023-10-16T11:26:42.475669+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:54:09.078766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.0","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.487912+0000","last_change":"2023-10-16T11:26:42.477532+0000","last_active":"2023-10-16T11:26:42.487912+0000","last_peered":"2023-10-16T11:26:42.487912+0000","last_clean":"2023-10-16T11:26:42.487912+0000","last_became_active":"2023-10-16T11:26:42.477268+0000","last_became_peered":"2023-10-16T11:26:42.477268+0000","last_unstale":"2023-10-16T11:26:42.487912+0000","last_undegraded":"2023-10-16T11:26:42.487912+0000","last_fullsized":"2023-10-16T11:26:42.487912+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:24.858930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.474804+0000","last_change":"2023-10-16T11:26:41.463237+0000","last_active":"2023-10-16T11:26:42.474804+0000","last_peered":"2023-10-16T11:26:42.474804+0000","last_clean":"2023-10-16T11:26:42.474804+0000","last_became_active":"2023-10-16T11:26:41.462972+0000","last_became_peered":"2023-10-16T11:26:41.462972+0000","last_unstale":"2023-10-16T11:26:42.474804+0000","last_undegraded":"2023-10-16T11:26:42.474804+0000","last_fullsized":"2023-10-16T11:26:42.474804+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:17.143386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1","version":"0'0","reported_seq":9,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.495180+0000","last_change":"2023-10-16T11:26:42.492352+0000","last_active":"2023-10-16T11:26:42.495180+0000","last_peered":"2023-10-16T11:26:42.495180+0000","last_clean":"2023-10-16T11:26:42.495180+0000","last_became_active":"2023-10-16T11:26:42.490917+0000","last_became_peered":"2023-10-16T11:26:42.490917+0000","last_unstale":"2023-10-16T11:26:42.495180+0000","last_undegraded":"2023-10-16T11:26:42.495180+0000","last_fullsized":"2023-10-16T11:26:42.495180+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:14.423981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.451400+0000","last_change":"2023-10-16T11:26:41.463453+0000","last_active":"2023-10-16T11:26:42.451400+0000","last_peered":"2023-10-16T11:26:42.451400+0000","last_clean":"2023-10-16T11:26:42.451400+0000","last_became_active":"2023-10-16T11:26:41.462536+0000","last_became_peered":"2023-10-16T11:26:41.462536+0000","last_unstale":"2023-10-16T11:26:42.451400+0000","last_undegraded":"2023-10-16T11:26:42.451400+0000","last_fullsized":"2023-10-16T11:26:42.451400+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:40.406100+0000","last_clean_scrub_stamp":"2023-10-16T11:26:40.406100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:32:24.083977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.22","version":"0'0","reported_seq":11,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-16T11:26:44.057542+0000","last_change":"2023-10-16T11:26:42.490965+0000","last_active":"2023-10-16T11:26:44.057542+0000","last_peered":"2023-10-16T11:26:44.057542+0000","last_clean":"2023-10-16T11:26:44.057542+0000","last_became_active":"2023-10-16T11:26:42.489788+0000","last_became_peered":"2023-10-16T11:26:42.489788+0000","last_unstale":"2023-10-16T11:26:44.057542+0000","last_undegraded":"2023-10-16T11:26:44.057542+0000","last_fullsized":"2023-10-16T11:26:44.057542+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:26:41.418602+0000","last_clean_scrub_stamp":"2023-10-16T11:26:41.418602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:44:01.049493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"254'4803","reported_seq":5829,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:26:42.462104+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:26:42.462104+0000","last_peered":"2023-10-16T11:26:42.462104+0000","last_clean":"2023-10-16T11:26:42.462104+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:26:42.462104+0000","last_undegraded":"2023-10-16T11:26:42.462104+0000","last_fullsized":"2023-10-16T11:26:42.462104+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":4803,"log_dups_size":0,"ondisk_log_size":4803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2495,"num_read_kb":5117,"num_write":643,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":35,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":34,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2495,"num_read_kb":5117,"num_write":643,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4803,"ondisk_log_size":4803,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576011,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61304,"kb_used_data":3752,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310536,"statfs":{"total":96636764160,"available":96573988864,"internally_reserved":0,"allocated":3842048,"data_stored":2181490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.1160000000000001,"15min":1.1970000000000001},"min":{"1min":0.70299999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4490000000000001,"5min":1.7250000000000001,"15min":2.3399999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.139,"15min":1.202},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.6479999999999999,"5min":1.6699999999999999,"15min":1.8320000000000001},"last":0.95499999999999996}]},{"osd":1,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.21,"15min":1.2310000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.452,"5min":1.7589999999999999,"15min":1.913},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1970000000000001,"15min":1.2549999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.6799999999999999,"5min":1.726,"15min":1.8600000000000001},"last":1.163}]},{"osd":2,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2889999999999999,"15min":1.29},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":2.4460000000000002},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.3400000000000001,"15min":1.361},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":3.9249999999999998,"5min":3.9249999999999998,"15min":3.9249999999999998},"last":1.0589999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.359,"15min":1.3799999999999999},"min":{"1min":1.1419999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.399,"15min":1.4379999999999999},"min":{"1min":0.81399999999999995,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.802,"5min":2.0070000000000001,"15min":2.577},"last":1.484}]},{"osd":4,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.4359999999999999,"15min":1.4079999999999999},"min":{"1min":1.093,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.9610000000000001,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.204},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.373,"15min":1.383},"min":{"1min":0.91400000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.25}]},{"osd":5,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.5740000000000001,"15min":1.583},"min":{"1min":0.81499999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.966,"5min":2.101,"15min":3.3519999999999999},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.462,"5min":1.516,"15min":1.5009999999999999},"min":{"1min":0.97899999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.7569999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.1240000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:25:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.502,"15min":1.5029999999999999},"min":{"1min":1.018,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.7250000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.635,"5min":1.665,"15min":1.6879999999999999},"min":{"1min":0.76800000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1600000000000001,"5min":2.2109999999999999,"15min":3.1440000000000001},"last":1.601}]}]},{"osd":6,"up_from":13,"seq":55834576011,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52412,"kb_used_data":3176,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319428,"statfs":{"total":96636764160,"available":96583094272,"internally_reserved":0,"allocated":3252224,"data_stored":1591212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.1779999999999999,"15min":1.206},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.5700000000000001,"5min":6.298,"15min":6.298},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.3129999999999999,"15min":1.278},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6140000000000001,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":1.202}]},{"osd":1,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.69799999999999995,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.24,"5min":1.395,"15min":1.365},"min":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.956,"5min":12.468,"15min":12.468},"last":0.97699999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.286,"15min":1.262},"min":{"1min":0.82899999999999996,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4870000000000001,"5min":11.586,"15min":11.586},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.3799999999999999,"15min":1.3720000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.77500000000000002},"max":{"1min":1.8600000000000001,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":0.79300000000000004}]},{"osd":3,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.4199999999999999,"15min":1.3440000000000001},"min":{"1min":0.86299999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.925,"5min":12.801,"15min":12.801},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.2809999999999999,"15min":1.2649999999999999},"min":{"1min":0.88100000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5589999999999999,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":1.5589999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.417,"15min":1.3320000000000001},"min":{"1min":0.76300000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.806,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.417},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.6359999999999999,"15min":1.548},"min":{"1min":1.109,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.3069999999999999,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.607}]},{"osd":5,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.5469999999999999,"15min":1.3999999999999999},"min":{"1min":0.86199999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.8979999999999999,"5min":18.247,"15min":18.247},"last":1.274},{"interface":"front","average":{"1min":1.617,"5min":1.625,"15min":1.5660000000000001},"min":{"1min":0.85899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.923,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.6839999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.6779999999999999,"5min":1.6819999999999999,"15min":1.5529999999999999},"min":{"1min":0.92500000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.4660000000000002,"5min":12.589,"15min":12.589},"last":1.738},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.7110000000000001,"15min":1.6020000000000001},"min":{"1min":1.1779999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.2050000000000001,"5min":12.725,"15min":12.725},"last":1.3740000000000001}]}]},{"osd":5,"up_from":13,"seq":55834576011,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48252,"kb_used_data":3180,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323588,"statfs":{"total":96636764160,"available":96587354112,"internally_reserved":0,"allocated":3256320,"data_stored":1587015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.2110000000000001,"15min":1.2529999999999999},"min":{"1min":0.749,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.3839999999999999,"5min":2.661,"15min":2.661},"last":1.446},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2749999999999999,"15min":1.327},"min":{"1min":0.76400000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.675,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.1619999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.1319999999999999,"15min":1.1779999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.992},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1970000000000001,"15min":1.22},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.669,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.5960000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.248,"15min":1.228},"min":{"1min":0.73299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.526,"5min":1.952,"15min":1.952},"last":1.329},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.2929999999999999,"15min":1.3109999999999999},"min":{"1min":0.83599999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":2.1640000000000001},"last":1.389}]},{"osd":3,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.302,"15min":1.3220000000000001},"min":{"1min":0.97999999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.6830000000000001,"5min":3.012,"15min":3.012},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.395,"15min":1.409},"min":{"1min":0.93400000000000005,"5min":0.92700000000000005,"15min":0.89500000000000002},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.2250000000000001},"last":1.51}]},{"osd":4,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3260000000000001,"15min":1.333},"min":{"1min":0.96099999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.575,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.494,"5min":1.544,"15min":1.5609999999999999},"min":{"1min":1.048,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.8009999999999999,"5min":2.903,"15min":2.903},"last":1.6439999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.3999999999999999,"15min":1.405},"min":{"1min":1.097,"5min":0.68100000000000005,"15min":0.51000000000000001},"max":{"1min":1.6519999999999999,"5min":1.8660000000000001,"15min":2.0979999999999999},"last":1.673},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.458,"15min":1.474},"min":{"1min":0.95999999999999996,"5min":0.90700000000000003,"15min":0.89200000000000002},"max":{"1min":1.758,"5min":3.073,"15min":3.073},"last":1.218}]},{"osd":7,"last update":"Mon Oct 16 11:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.536,"15min":1.538},"min":{"1min":1.0549999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.1150000000000002,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.556},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.615,"15min":1.637},"min":{"1min":1.3400000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.2930000000000001},"last":1.748}]}]},{"osd":4,"up_from":13,"seq":55834576012,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70380,"kb_used_data":3168,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301460,"statfs":{"total":96636764160,"available":96564695040,"internally_reserved":0,"allocated":3244032,"data_stored":1584843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.1870000000000001,"15min":1.1839999999999999},"min":{"1min":0.80800000000000005,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.8049999999999999,"5min":2.899,"15min":2.899},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2170000000000001,"15min":1.2450000000000001},"min":{"1min":0.88400000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.7929999999999999,"5min":3.016,"15min":3.016},"last":1.091}]},{"osd":1,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3049999999999999,"15min":1.298},"min":{"1min":0.84999999999999998,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.887,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.77100000000000002,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.617,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.018}]},{"osd":2,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.327,"15min":1.3140000000000001},"min":{"1min":1.075,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.169,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4350000000000001,"15min":1.4159999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9870000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.417}]},{"osd":3,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.3640000000000001,"15min":1.351},"min":{"1min":0.96799999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.0390000000000001,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.365},{"interface":"front","average":{"1min":1.645,"5min":1.4970000000000001,"15min":1.4610000000000001},"min":{"1min":1.216,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.3109999999999999,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.794}]},{"osd":5,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.5289999999999999,"15min":1.464},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2309999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.681},{"interface":"front","average":{"1min":1.504,"5min":1.4810000000000001,"15min":1.4350000000000001},"min":{"1min":0.89500000000000002,"5min":0.85199999999999998,"15min":0.78600000000000003},"max":{"1min":2.0110000000000001,"5min":3.7989999999999999,"15min":3.7989999999999999},"last":1.464}]},{"osd":6,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.599,"15min":1.534},"min":{"1min":0.98499999999999999,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":2.6459999999999999,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.9159999999999999},{"interface":"front","average":{"1min":1.752,"5min":1.665,"15min":1.6379999999999999},"min":{"1min":0.77100000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.3599999999999999,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.76}]},{"osd":7,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.7629999999999999,"5min":1.649,"15min":1.6000000000000001},"min":{"1min":0.74099999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.3570000000000002,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.7849999999999999,"5min":1.7270000000000001,"15min":1.675},"min":{"1min":0.98699999999999999,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.464,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":2.0219999999999998}]}]},{"osd":3,"up_from":13,"seq":55834576011,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52420,"kb_used_data":3184,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319420,"statfs":{"total":96636764160,"available":96583086080,"internally_reserved":0,"allocated":3260416,"data_stored":1592734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.83199999999999996,"15min":0.88900000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.2430000000000001,"5min":1.478,"15min":1.901},"last":1.4730000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.95599999999999996,"15min":1.046},"min":{"1min":0.45100000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.581,"5min":1.581,"15min":1.6679999999999999},"last":1.0880000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.92700000000000005,"15min":0.97599999999999998},"min":{"1min":0.39600000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.3380000000000001,"5min":1.508,"15min":1.508},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.89100000000000001,"5min":1.016,"15min":1.1020000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.246,"5min":1.601,"15min":1.7190000000000001},"last":1.038}]},{"osd":2,"last update":"Mon Oct 16 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.042,"15min":1.105},"min":{"1min":0.52400000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5660000000000001,"5min":1.694,"15min":1.694},"last":1.3180000000000001},{"interface":"front","average":{"1min":0.999,"5min":1.117,"15min":1.2030000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.542,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.492}]},{"osd":4,"last update":"Mon Oct 16 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2529999999999999,"15min":1.335},"min":{"1min":0.71199999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.821,"5min":1.821,"15min":1.873},"last":1.411},{"interface":"front","average":{"1min":1.204,"5min":1.2989999999999999,"15min":1.349},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5960000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.3700000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.341,"15min":1.393},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9199999999999999,"5min":2.798,"15min":2.798},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.371,"15min":1.429},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.8440000000000001,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.4319999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.456,"15min":1.52},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.901,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.351},{"interface":"front","average":{"1min":1.405,"5min":1.4630000000000001,"15min":1.522},"min":{"1min":0.96299999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.7929999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.5589999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.496,"15min":1.55},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.827,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.617},{"interface":"front","average":{"1min":1.498,"5min":1.5800000000000001,"15min":1.657},"min":{"1min":1.0309999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.9059999999999999,"5min":2.3799999999999999,"15min":2.4649999999999999},"last":1.7090000000000001}]}]},{"osd":2,"up_from":13,"seq":55834576010,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56564,"kb_used_data":3176,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315276,"statfs":{"total":96636764160,"available":96578842624,"internally_reserved":0,"allocated":3252224,"data_stored":1591212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88900000000000001,"15min":0.90900000000000003},"min":{"1min":0.498,"5min":0.314,"15min":0.314},"max":{"1min":1.4870000000000001,"5min":1.532,"15min":1.6399999999999999},"last":1.3},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.97799999999999998,"15min":1.0580000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.32600000000000001},"max":{"1min":1.5449999999999999,"5min":1.6359999999999999,"15min":2.1429999999999998},"last":0.81999999999999995}]},{"osd":1,"last update":"Mon Oct 16 11:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0640000000000001,"15min":1.0780000000000001},"min":{"1min":0.57399999999999995,"5min":0.42699999999999999,"15min":0.39200000000000002},"max":{"1min":1.4750000000000001,"5min":1.645,"15min":1.978},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.048,"15min":1.123},"min":{"1min":0.58999999999999997,"5min":0.438,"15min":0.438},"max":{"1min":1.4179999999999999,"5min":1.609,"15min":1.7949999999999999},"last":0.95099999999999996}]},{"osd":3,"last update":"Mon Oct 16 11:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.006,"15min":1.0629999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4510000000000001,"5min":1.651,"15min":1.8560000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.089,"5min":1.1739999999999999,"15min":1.2509999999999999},"min":{"1min":0.61399999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.5880000000000001,"5min":1.7450000000000001,"15min":1.857},"last":1.0049999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:26:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3140000000000001,"15min":1.391},"min":{"1min":0.95099999999999996,"5min":0.78600000000000003,"15min":0.73499999999999999},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.5059999999999998},"last":1.244},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.2709999999999999,"15min":1.321},"min":{"1min":0.878,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":1.345,"5min":1.825,"15min":1.843},"last":1.1339999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:26:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.4059999999999999,"15min":1.4419999999999999},"min":{"1min":0.89200000000000002,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.895,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.425,"15min":1.464},"min":{"1min":0.96399999999999997,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.323,"5min":2.323,"15min":2.6259999999999999},"last":1.4590000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:26:08 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.3740000000000001,"15min":1.3899999999999999},"min":{"1min":0.83199999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.855,"5min":1.9019999999999999,"15min":2.1339999999999999},"last":1.341},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4810000000000001,"15min":1.5189999999999999},"min":{"1min":0.81499999999999995,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.7929999999999999,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.506}]},{"osd":7,"last update":"Mon Oct 16 11:26:08 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.5069999999999999,"15min":1.5589999999999999},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92500000000000004},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.8109999999999999},"last":1.611},{"interface":"front","average":{"1min":1.389,"5min":1.452,"15min":1.492},"min":{"1min":0.88400000000000001,"5min":0.80200000000000005,"15min":0.80100000000000005},"max":{"1min":1.768,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.5620000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576010,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48244,"kb_used_data":3176,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323596,"statfs":{"total":96636764160,"available":96587362304,"internally_reserved":0,"allocated":3252224,"data_stored":1591156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.90900000000000003,"15min":0.91300000000000003},"min":{"1min":0.39400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.3520000000000001,"5min":5.117,"15min":5.117},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.93500000000000005,"15min":1.0289999999999999},"min":{"1min":0.35899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.262,"5min":1.5509999999999999,"15min":1.667},"last":0.45500000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.93799999999999994,"15min":1.0029999999999999},"min":{"1min":0.497,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3720000000000001,"5min":2.093,"15min":2.093},"last":0.33100000000000002},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1879999999999999,"15min":1.2589999999999999},"min":{"1min":0.51700000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.456,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.498}]},{"osd":3,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.139,"15min":1.1890000000000001},"min":{"1min":0.53800000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.6080000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.103,"15min":1.153},"min":{"1min":0.55800000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.798,"5min":1.798,"15min":1.802},"last":0.54100000000000004}]},{"osd":4,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.3049999999999999,"15min":1.2949999999999999},"min":{"1min":0.69299999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.978,"5min":5.29,"15min":5.29},"last":1.079},{"interface":"front","average":{"1min":1.327,"5min":1.4159999999999999,"15min":1.421},"min":{"1min":0.82999999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.891,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":0.78300000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.373,"15min":1.409},"min":{"1min":0.83799999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.9430000000000001,"5min":6.0979999999999999,"15min":6.0979999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.383,"15min":1.403},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.859,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":0.90400000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4570000000000001,"15min":1.4890000000000001},"min":{"1min":0.92200000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.923,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4419999999999999,"15min":1.4810000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.1419999999999999,"5min":3.8799999999999999,"15min":3.8799999999999999},"last":0.70599999999999996}]},{"osd":7,"last update":"Mon Oct 16 11:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.546,"15min":1.5609999999999999},"min":{"1min":1.0409999999999999,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":2.0219999999999998,"5min":4.46,"15min":4.46},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.512,"5min":1.597,"15min":1.6040000000000001},"min":{"1min":0.99099999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.097,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":1.024}]}]},{"osd":0,"up_from":13,"seq":55834576011,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65464,"kb_used_data":3752,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306376,"statfs":{"total":96636764160,"available":96569729024,"internally_reserved":0,"allocated":3842048,"data_stored":2175757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.86099999999999999,"15min":0.88700000000000001},"min":{"1min":0.46800000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2210000000000001,"5min":1.52,"15min":1.619},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.006,"15min":1.081},"min":{"1min":0.41199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.341,"5min":1.621,"15min":1.76},"last":1.518}]},{"osd":2,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.075,"15min":1.0800000000000001},"min":{"1min":0.52500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.7349999999999999,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":1.038},{"interface":"front","average":{"1min":1.034,"5min":1.0700000000000001,"15min":1.1299999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.97899999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.016,"15min":1.0669999999999999},"min":{"1min":0.50600000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1299999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.1539999999999999,"15min":1.226},"min":{"1min":0.55600000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.3799999999999999,"5min":1.833,"15min":1.9830000000000001},"last":1.3799999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.349,"15min":1.3859999999999999},"min":{"1min":0.78800000000000003,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":3.2799999999999998,"5min":3.2799999999999998,"15min":3.2799999999999998},"last":1.272},{"interface":"front","average":{"1min":1.222,"5min":1.292,"15min":1.337},"min":{"1min":0.76100000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.5920000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.357,"15min":1.3720000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0209999999999999,"5min":2.0750000000000002,"15min":2.6669999999999998},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3819999999999999,"15min":1.431},"min":{"1min":0.89800000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.885,"5min":2.2029999999999998,"15min":2.2040000000000002},"last":1.155}]},{"osd":6,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.4710000000000001,"15min":1.496},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":5.2560000000000002,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":1.462},{"interface":"front","average":{"1min":1.298,"5min":1.429,"15min":1.4790000000000001},"min":{"1min":0.83199999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.804,"5min":2.073,"15min":2.262},"last":1.6499999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":1.704,"5min":1.5429999999999999,"15min":1.5600000000000001},"min":{"1min":0.88700000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.425},{"interface":"front","average":{"1min":1.278,"5min":1.4419999999999999,"15min":1.492},"min":{"1min":0.871,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.623,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.6950000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:26:47.823 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T11:26:47.823 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T11:26:48.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.088+0000 7f2c81741640 1 Processor -- start 2023-10-16T11:26:48.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.088+0000 7f2c81741640 1 -- start start 2023-10-16T11:26:48.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.089+0000 7f2c81741640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c159f00 0x7f2c7c0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:48.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.089+0000 7f2c81741640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2c7c15d8c0 0x7f2c7c0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:48.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.089+0000 7f2c81741640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7c06f890 con 0x7f2c7c159b20 2023-10-16T11:26:48.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.089+0000 7f2c81741640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c7c070d30 con 0x7f2c7c15d8c0 2023-10-16T11:26:48.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.089+0000 7f2c81741640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c06c650 con 0x7f2c7c159f00 2023-10-16T11:26:48.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.090+0000 7f2c7b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2c7c15d8c0 0x7f2c7c0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:48.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.090+0000 7f2c7b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2c7c15d8c0 0x7f2c7c0b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51620/0 (socket says 172.21.15.84:51620) 2023-10-16T11:26:48.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.090+0000 7f2c7b7fe640 1 -- 172.21.15.84:0/2584800185 learned_addr learned my addr 172.21.15.84:0/2584800185 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:48.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.090+0000 7f2c7a7fc640 1 --2- 172.21.15.84:0/2584800185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c159f00 0x7f2c7c0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:48.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.090+0000 7f2c7b7fe640 1 -- 172.21.15.84:0/2584800185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c159f00 msgr2=0x7f2c7c0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:48.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.090+0000 7f2c7b7fe640 1 --2- 172.21.15.84:0/2584800185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c159f00 0x7f2c7c0b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.090+0000 7f2c7b7fe640 1 -- 172.21.15.84:0/2584800185 >> v1:172.21.15.157:6789/0 conn(0x7f2c7c159b20 legacy=0x7f2c7c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:48.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.090+0000 7f2c7b7fe640 1 -- 172.21.15.84:0/2584800185 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7c06c930 con 0x7f2c7c15d8c0 2023-10-16T11:26:48.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.091+0000 7f2c7b7fe640 1 --2- 172.21.15.84:0/2584800185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c15d8c0 0x7f2c7c0b0950 secure :-1 s=READY pgs=3223 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009e50 tx=0x7f2c7402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5790d7786e2524f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:48.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.092+0000 7f2c79ffb640 1 -- 172.21.15.84:0/2584800185 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7402cb00 con 0x7f2c7c15d8c0 2023-10-16T11:26:48.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.092+0000 7f2c79ffb640 1 -- 172.21.15.84:0/2584800185 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7402cc60 con 0x7f2c7c15d8c0 2023-10-16T11:26:48.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.092+0000 7f2c79ffb640 1 -- 172.21.15.84:0/2584800185 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c740343f0 con 0x7f2c7c15d8c0 2023-10-16T11:26:48.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.092+0000 7f2c81741640 1 -- 172.21.15.84:0/2584800185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c15d8c0 msgr2=0x7f2c7c0b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:48.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.092+0000 7f2c81741640 1 --2- 172.21.15.84:0/2584800185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c15d8c0 0x7f2c7c0b0950 secure :-1 s=READY pgs=3223 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009e50 tx=0x7f2c7402bf60 comp rx=0 tx=0).stop 2023-10-16T11:26:48.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.092+0000 7f2c81741640 1 -- 172.21.15.84:0/2584800185 shutdown_connections 2023-10-16T11:26:48.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.092+0000 7f2c81741640 1 --2- 172.21.15.84:0/2584800185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c15d8c0 0x7f2c7c0b0950 unknown :-1 s=CLOSED pgs=3223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.092+0000 7f2c81741640 1 --2- 172.21.15.84:0/2584800185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c159f00 0x7f2c7c0b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.092+0000 7f2c81741640 1 -- 172.21.15.84:0/2584800185 >> 172.21.15.84:0/2584800185 conn(0x7f2c7c0cb410 msgr2=0x7f2c7c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:48.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.092+0000 7f2c81741640 1 -- 172.21.15.84:0/2584800185 shutdown_connections 2023-10-16T11:26:48.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.092+0000 7f2c81741640 1 -- 172.21.15.84:0/2584800185 wait complete. 2023-10-16T11:26:48.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.094+0000 7f2c81741640 1 Processor -- start 2023-10-16T11:26:48.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.094+0000 7f2c81741640 1 -- start start 2023-10-16T11:26:48.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.095+0000 7f2c81741640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c7c159b20 0x7f2c7c151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:48.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.095+0000 7f2c81741640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c159f00 0x7f2c7c0b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:48.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.095+0000 7f2c81741640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c15d8c0 0x7f2c7c0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:48.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.095+0000 7f2c81741640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c1524b0 con 0x7f2c7c159f00 2023-10-16T11:26:48.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.095+0000 7f2c81741640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c0b4000 con 0x7f2c7c159b20 2023-10-16T11:26:48.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.095+0000 7f2c81741640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c151f50 con 0x7f2c7c15d8c0 2023-10-16T11:26:48.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.095+0000 7f2c7a7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c159f00 0x7f2c7c0b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:48.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.095+0000 7f2c7a7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c159f00 0x7f2c7c0b2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51636/0 (socket says 172.21.15.84:51636) 2023-10-16T11:26:48.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.096+0000 7f2c7a7fc640 1 -- 172.21.15.84:0/2071192818 learned_addr learned my addr 172.21.15.84:0/2071192818 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:48.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.096+0000 7f2c7a7fc640 1 -- 172.21.15.84:0/2071192818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c15d8c0 msgr2=0x7f2c7c0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:48.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.096+0000 7f2c7a7fc640 1 --2- 172.21.15.84:0/2071192818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c15d8c0 0x7f2c7c0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.096+0000 7f2c7a7fc640 1 -- 172.21.15.84:0/2071192818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c7c159b20 msgr2=0x7f2c7c151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:48.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.096+0000 7f2c7affd640 1 --2- 172.21.15.84:0/2071192818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c7c159b20 0x7f2c7c151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:48.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.096+0000 7f2c7a7fc640 1 --2- 172.21.15.84:0/2071192818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c7c159b20 0x7f2c7c151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.096+0000 7f2c7a7fc640 1 -- 172.21.15.84:0/2071192818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c74009b30 con 0x7f2c7c159f00 2023-10-16T11:26:48.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.096+0000 7f2c7a7fc640 1 --2- 172.21.15.84:0/2071192818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c159f00 0x7f2c7c0b2590 secure :-1 s=READY pgs=3224 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000d7d0 tx=0x7f2c7000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:48.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.096+0000 7f2c637fe640 1 -- 172.21.15.84:0/2071192818 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7000b6c0 con 0x7f2c7c159f00 2023-10-16T11:26:48.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.097+0000 7f2c637fe640 1 -- 172.21.15.84:0/2071192818 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7000b820 con 0x7f2c7c159f00 2023-10-16T11:26:48.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.097+0000 7f2c81741640 1 -- 172.21.15.84:0/2071192818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7c0b1580 con 0x7f2c7c159f00 2023-10-16T11:26:48.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.097+0000 7f2c637fe640 1 -- 172.21.15.84:0/2071192818 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7000bbb0 con 0x7f2c7c159f00 2023-10-16T11:26:48.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.097+0000 7f2c81741640 1 -- 172.21.15.84:0/2071192818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7c0b1ab0 con 0x7f2c7c159f00 2023-10-16T11:26:48.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.098+0000 7f2c81741640 1 -- 172.21.15.84:0/2071192818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c7c04e160 con 0x7f2c7c159f00 2023-10-16T11:26:48.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.099+0000 7f2c637fe640 1 -- 172.21.15.84:0/2071192818 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 151) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f2c7000bd10 con 0x7f2c7c159f00 2023-10-16T11:26:48.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.099+0000 7f2c637fe640 1 --2- 172.21.15.84:0/2071192818 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f2c6806b0b0 0x7f2c6806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:48.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.100+0000 7f2c637fe640 1 -- 172.21.15.84:0/2071192818 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5780+0+0 (secure 0 0 0) 0x7f2c7008a090 con 0x7f2c7c159f00 2023-10-16T11:26:48.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.100+0000 7f2c7affd640 1 --2- 172.21.15.84:0/2071192818 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f2c6806b0b0 0x7f2c6806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:48.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.101+0000 7f2c7affd640 1 --2- 172.21.15.84:0/2071192818 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f2c6806b0b0 0x7f2c6806d570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c000f60 tx=0x7f2c6c009040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:48.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.104+0000 7f2c637fe640 1 -- 172.21.15.84:0/2071192818 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2c70054cf0 con 0x7f2c7c159f00 2023-10-16T11:26:48.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.366+0000 7f2c81741640 1 -- 172.21.15.84:0/2071192818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2c7c15d570 con 0x7f2c7c159f00 2023-10-16T11:26:48.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.372+0000 7f2c637fe640 1 -- 172.21.15.84:0/2071192818 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v58) v1 ==== 190+0+147 (secure 0 0 0) 0x7f2c70058310 con 0x7f2c7c159f00 2023-10-16T11:26:48.374 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2023-10-16T11:26:48.375 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ== 2023-10-16T11:26:48.375 INFO:teuthology.orchestra.run.smithi084.stdout: caps mds = "allow" 2023-10-16T11:26:48.375 INFO:teuthology.orchestra.run.smithi084.stdout: caps mon = "allow r" 2023-10-16T11:26:48.375 INFO:teuthology.orchestra.run.smithi084.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T11:26:48.376 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-16T11:26:48.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.374+0000 7f2c81741640 1 -- 172.21.15.84:0/2071192818 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f2c6806b0b0 msgr2=0x7f2c6806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:48.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.374+0000 7f2c81741640 1 --2- 172.21.15.84:0/2071192818 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f2c6806b0b0 0x7f2c6806d570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c000f60 tx=0x7f2c6c009040 comp rx=0 tx=0).stop 2023-10-16T11:26:48.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.374+0000 7f2c81741640 1 -- 172.21.15.84:0/2071192818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c159f00 msgr2=0x7f2c7c0b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:48.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.374+0000 7f2c81741640 1 --2- 172.21.15.84:0/2071192818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c159f00 0x7f2c7c0b2590 secure :-1 s=READY pgs=3224 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000d7d0 tx=0x7f2c7000dc90 comp rx=0 tx=0).stop 2023-10-16T11:26:48.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.374+0000 7f2c81741640 1 -- 172.21.15.84:0/2071192818 shutdown_connections 2023-10-16T11:26:48.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.374+0000 7f2c81741640 1 --2- 172.21.15.84:0/2071192818 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f2c6806b0b0 0x7f2c6806d570 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.374+0000 7f2c81741640 1 --2- 172.21.15.84:0/2071192818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c7c15d8c0 0x7f2c7c0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.374+0000 7f2c81741640 1 --2- 172.21.15.84:0/2071192818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c7c159f00 0x7f2c7c0b2590 unknown :-1 s=CLOSED pgs=3224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.374+0000 7f2c81741640 1 --2- 172.21.15.84:0/2071192818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c7c159b20 0x7f2c7c151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.374+0000 7f2c81741640 1 -- 172.21.15.84:0/2071192818 >> 172.21.15.84:0/2071192818 conn(0x7f2c7c0cb410 msgr2=0x7f2c7c0c7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:48.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.375+0000 7f2c81741640 1 -- 172.21.15.84:0/2071192818 shutdown_connections 2023-10-16T11:26:48.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.375+0000 7f2c81741640 1 -- 172.21.15.84:0/2071192818 wait complete. 2023-10-16T11:26:48.395 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:26:48.412 INFO:tasks.ceph.mds.b.smithi157.stderr:2023-10-16T11:26:48.410+0000 7f65c66dc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:26:48.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.599+0000 7f17af075640 1 Processor -- start 2023-10-16T11:26:48.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.600+0000 7f17af075640 1 -- start start 2023-10-16T11:26:48.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.601+0000 7f17af075640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a8159b20 0x7f17a806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:48.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.601+0000 7f17af075640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a806f7f0 con 0x7f17a8159f00 2023-10-16T11:26:48.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.601+0000 7f17af075640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a806c5e0 con 0x7f17a80b0b20 2023-10-16T11:26:48.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.601+0000 7f17af075640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a806c8c0 con 0x7f17a8159b20 2023-10-16T11:26:48.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.601+0000 7f17a7fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f17a8159f00 0x7f17a80b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40954/0 (socket says 172.21.15.84:40954) 2023-10-16T11:26:48.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.601+0000 7f17a7fff640 1 -- 172.21.15.84:0/1644954393 learned_addr learned my addr 172.21.15.84:0/1644954393 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:48.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.602+0000 7f17ace11640 1 --2- 172.21.15.84:0/1644954393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a8159b20 0x7f17a806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:48.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.602+0000 7f17a77fe640 1 -- 172.21.15.84:0/1644954393 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810942567 0 0) 0x7f17a806f7f0 con 0x7f17a8159f00 2023-10-16T11:26:48.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.602+0000 7f17a77fe640 1 -- 172.21.15.84:0/1644954393 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1798003520 con 0x7f17a8159f00 2023-10-16T11:26:48.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.602+0000 7f17a77fe640 1 -- 172.21.15.84:0/1644954393 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2114650990 0 0) 0x7f1798003520 con 0x7f17a8159f00 2023-10-16T11:26:48.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.602+0000 7f17a77fe640 1 -- 172.21.15.84:0/1644954393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a8159b20 msgr2=0x7f17a806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:48.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.602+0000 7f17a77fe640 1 --2- 172.21.15.84:0/1644954393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a8159b20 0x7f17a806bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.602+0000 7f17a77fe640 1 -- 172.21.15.84:0/1644954393 >> v1:172.21.15.157:6789/0 conn(0x7f17a80b0b20 legacy=0x7f17a806fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:48.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.602+0000 7f17a77fe640 1 -- 172.21.15.84:0/1644954393 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a806cba0 con 0x7f17a8159f00 2023-10-16T11:26:48.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.602+0000 7f17a77fe640 1 -- 172.21.15.84:0/1644954393 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f1794001670 con 0x7f17a8159f00 2023-10-16T11:26:48.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.602+0000 7f17a77fe640 1 -- 172.21.15.84:0/1644954393 >> v1:172.21.15.84:6789/0 conn(0x7f17a8159f00 legacy=0x7f17a80b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:48.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.602+0000 7f17a77fe640 1 --2- 172.21.15.84:0/1644954393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17980039a0 0x7f1798003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:48.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.602+0000 7f17a77fe640 1 --2- 172.21.15.84:0/1644954393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a8159b20 0x7f17980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:48.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.602+0000 7f17a77fe640 1 --2- 172.21.15.84:0/1644954393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1798004d00 0x7f1798005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:48.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.602+0000 7f17a77fe640 1 -- 172.21.15.84:0/1644954393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a806c8c0 con 0x7f17980039a0 2023-10-16T11:26:48.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.602+0000 7f17a77fe640 1 -- 172.21.15.84:0/1644954393 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179800ba30 con 0x7f1798004d00 2023-10-16T11:26:48.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.602+0000 7f17a77fe640 1 -- 172.21.15.84:0/1644954393 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179800bbf0 con 0x7f17a8159b20 2023-10-16T11:26:48.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.602+0000 7f17ace11640 1 --2- 172.21.15.84:0/1644954393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17980039a0 0x7f1798003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:48.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.603+0000 7f17a7fff640 1 --2- 172.21.15.84:0/1644954393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a8159b20 0x7f17980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:48.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.603+0000 7f17ace11640 1 -- 172.21.15.84:0/1644954393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a8159b20 msgr2=0x7f17980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:48.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.603+0000 7f17ace11640 1 --2- 172.21.15.84:0/1644954393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a8159b20 0x7f17980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.603+0000 7f17ace11640 1 -- 172.21.15.84:0/1644954393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1798004d00 msgr2=0x7f1798005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:48.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.603+0000 7f17ace11640 1 --2- 172.21.15.84:0/1644954393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1798004d00 0x7f1798005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.603+0000 7f17ace11640 1 -- 172.21.15.84:0/1644954393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1798003520 con 0x7f17980039a0 2023-10-16T11:26:48.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.603+0000 7f17a7fff640 1 --2- 172.21.15.84:0/1644954393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a8159b20 0x7f17980047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:48.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.604+0000 7f17ace11640 1 --2- 172.21.15.84:0/1644954393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17980039a0 0x7f1798003dc0 secure :-1 s=READY pgs=3226 cs=0 l=1 rev1=1 crypto rx=0x7f179c00bd30 tx=0x7f17980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336715671bc09ccc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:48.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.604+0000 7f17a77fe640 1 -- 172.21.15.84:0/1644954393 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c030690 con 0x7f17980039a0 2023-10-16T11:26:48.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.604+0000 7f17a77fe640 1 -- 172.21.15.84:0/1644954393 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179c0307f0 con 0x7f17980039a0 2023-10-16T11:26:48.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.604+0000 7f17a77fe640 1 -- 172.21.15.84:0/1644954393 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c030b80 con 0x7f17980039a0 2023-10-16T11:26:48.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.605+0000 7f17af075640 1 -- 172.21.15.84:0/1644954393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17980039a0 msgr2=0x7f1798003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:48.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.605+0000 7f17af075640 1 --2- 172.21.15.84:0/1644954393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17980039a0 0x7f1798003dc0 secure :-1 s=READY pgs=3226 cs=0 l=1 rev1=1 crypto rx=0x7f179c00bd30 tx=0x7f17980057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.605+0000 7f17ad612640 1 -- 172.21.15.84:0/1644954393 reap_dead start 2023-10-16T11:26:48.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.605+0000 7f17af075640 1 -- 172.21.15.84:0/1644954393 shutdown_connections 2023-10-16T11:26:48.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.605+0000 7f17af075640 1 -- 172.21.15.84:0/1644954393 >> 172.21.15.84:0/1644954393 conn(0x7f17a80cb410 msgr2=0x7f17a80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:48.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.605+0000 7f17af075640 1 -- 172.21.15.84:0/1644954393 shutdown_connections 2023-10-16T11:26:48.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.605+0000 7f17af075640 1 -- 172.21.15.84:0/1644954393 wait complete. 2023-10-16T11:26:48.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.607+0000 7f17af075640 1 Processor -- start 2023-10-16T11:26:48.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.607+0000 7f17af075640 1 -- start start 2023-10-16T11:26:48.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.608+0000 7f17af075640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17980039a0 0x7f17a80c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:48.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.608+0000 7f17af075640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17a814bee0 0x7f17a80ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:48.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.608+0000 7f17af075640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a80acdd0 0x7f17a80b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:48.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.608+0000 7f17af075640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a80ad250 con 0x7f17a814bee0 2023-10-16T11:26:48.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.608+0000 7f17af075640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a814c360 con 0x7f17980039a0 2023-10-16T11:26:48.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.608+0000 7f17af075640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a80c2640 con 0x7f17a80acdd0 2023-10-16T11:26:48.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.608+0000 7f17a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17a814bee0 0x7f17a80ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:48.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.608+0000 7f17a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17a814bee0 0x7f17a80ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51666/0 (socket says 172.21.15.84:51666) 2023-10-16T11:26:48.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.608+0000 7f17a7fff640 1 -- 172.21.15.84:0/3832872798 learned_addr learned my addr 172.21.15.84:0/3832872798 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:48.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.609+0000 7f17ace11640 1 --2- 172.21.15.84:0/3832872798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17980039a0 0x7f17a80c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:48.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.609+0000 7f17ad612640 1 --2- 172.21.15.84:0/3832872798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a80acdd0 0x7f17a80b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:48.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.609+0000 7f17a7fff640 1 -- 172.21.15.84:0/3832872798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a80acdd0 msgr2=0x7f17a80b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:48.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.609+0000 7f17a7fff640 1 --2- 172.21.15.84:0/3832872798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a80acdd0 0x7f17a80b34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.609+0000 7f17a7fff640 1 -- 172.21.15.84:0/3832872798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17980039a0 msgr2=0x7f17a80c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:48.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.609+0000 7f17a7fff640 1 --2- 172.21.15.84:0/3832872798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17980039a0 0x7f17a80c3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.609+0000 7f17a7fff640 1 -- 172.21.15.84:0/3832872798 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a80c2920 con 0x7f17a814bee0 2023-10-16T11:26:48.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.609+0000 7f17ace11640 1 --2- 172.21.15.84:0/3832872798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17980039a0 0x7f17a80c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:48.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.609+0000 7f17a7fff640 1 --2- 172.21.15.84:0/3832872798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17a814bee0 0x7f17a80ac890 secure :-1 s=READY pgs=3227 cs=0 l=1 rev1=1 crypto rx=0x7f179400d810 tx=0x7f179400dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:48.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.609+0000 7f17a57fa640 1 -- 172.21.15.84:0/3832872798 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17940074e0 con 0x7f17a814bee0 2023-10-16T11:26:48.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.609+0000 7f17a57fa640 1 -- 172.21.15.84:0/3832872798 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1794007640 con 0x7f17a814bee0 2023-10-16T11:26:48.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.609+0000 7f17af075640 1 -- 172.21.15.84:0/3832872798 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a80c2c00 con 0x7f17a814bee0 2023-10-16T11:26:48.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.609+0000 7f17a57fa640 1 -- 172.21.15.84:0/3832872798 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17940049e0 con 0x7f17a814bee0 2023-10-16T11:26:48.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.609+0000 7f17af075640 1 -- 172.21.15.84:0/3832872798 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a80b3a20 con 0x7f17a814bee0 2023-10-16T11:26:48.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.611+0000 7f17af075640 1 -- 172.21.15.84:0/3832872798 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a806ce60 con 0x7f17a814bee0 2023-10-16T11:26:48.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.612+0000 7f17a57fa640 1 -- 172.21.15.84:0/3832872798 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 151) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1794004b40 con 0x7f17a814bee0 2023-10-16T11:26:48.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.612+0000 7f17a57fa640 1 --2- 172.21.15.84:0/3832872798 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f178c06b0e0 0x7f178c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:48.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.613+0000 7f17a57fa640 1 -- 172.21.15.84:0/3832872798 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5780+0+0 (secure 0 0 0) 0x7f1794089550 con 0x7f17a814bee0 2023-10-16T11:26:48.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.613+0000 7f17ace11640 1 --2- 172.21.15.84:0/3832872798 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f178c06b0e0 0x7f178c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:48.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.613+0000 7f17ace11640 1 --2- 172.21.15.84:0/3832872798 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f178c06b0e0 0x7f178c06d5a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f179c039000 tx=0x7f179c008040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:48.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.617+0000 7f17a57fa640 1 -- 172.21.15.84:0/3832872798 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f17a806ce60 con 0x7f17a814bee0 2023-10-16T11:26:48.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.842+0000 7f17af075640 1 -- 172.21.15.84:0/3832872798 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17a806ce60 con 0x7f17a814bee0 2023-10-16T11:26:48.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.843+0000 7f17a57fa640 1 -- 172.21.15.84:0/3832872798 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+4929 (secure 0 0 0) 0x7f17a806ce60 con 0x7f17a814bee0 2023-10-16T11:26:48.845 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:48.846 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":148,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/591944719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":591944719},{"type":"v1","addr":"172.21.15.84:6835","nonce":591944719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":142},{"gid":8152,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2371517609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2371517609},{"type":"v1","addr":"172.21.15.84:6837","nonce":2371517609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":143},{"gid":8161,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1079456313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1079456313},{"type":"v1","addr":"172.21.15.157:6834","nonce":1079456313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":143}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:26:41.908585+0000","modified":"2023-10-16T11:26:44.991714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12987":{"gid":12987,"name":"b","rank":0,"incarnation":144,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3899307325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3899307325},{"type":"v1","addr":"172.21.15.157:6835","nonce":3899307325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-16T11:26:48.846 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 148 2023-10-16T11:26:48.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.845+0000 7f17af075640 1 -- 172.21.15.84:0/3832872798 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f178c06b0e0 msgr2=0x7f178c06d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:48.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.845+0000 7f17af075640 1 --2- 172.21.15.84:0/3832872798 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f178c06b0e0 0x7f178c06d5a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f179c039000 tx=0x7f179c008040 comp rx=0 tx=0).stop 2023-10-16T11:26:48.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.845+0000 7f17af075640 1 -- 172.21.15.84:0/3832872798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17a814bee0 msgr2=0x7f17a80ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:48.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.845+0000 7f17af075640 1 --2- 172.21.15.84:0/3832872798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17a814bee0 0x7f17a80ac890 secure :-1 s=READY pgs=3227 cs=0 l=1 rev1=1 crypto rx=0x7f179400d810 tx=0x7f179400dcd0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.846+0000 7f17af075640 1 -- 172.21.15.84:0/3832872798 shutdown_connections 2023-10-16T11:26:48.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.846+0000 7f17af075640 1 --2- 172.21.15.84:0/3832872798 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f178c06b0e0 0x7f178c06d5a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.846+0000 7f17af075640 1 --2- 172.21.15.84:0/3832872798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17a80acdd0 0x7f17a80b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.846+0000 7f17af075640 1 --2- 172.21.15.84:0/3832872798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17a814bee0 0x7f17a80ac890 unknown :-1 s=CLOSED pgs=3227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.846+0000 7f17af075640 1 --2- 172.21.15.84:0/3832872798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17980039a0 0x7f17a80c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:48.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.846+0000 7f17af075640 1 -- 172.21.15.84:0/3832872798 >> 172.21.15.84:0/3832872798 conn(0x7f17a80cb410 msgr2=0x7f17a806fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:48.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.846+0000 7f17af075640 1 -- 172.21.15.84:0/3832872798 shutdown_connections 2023-10-16T11:26:48.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:48.846+0000 7f17af075640 1 -- 172.21.15.84:0/3832872798 wait complete. 2023-10-16T11:26:49.268 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 148, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T11:26:41.908585+0000', 'modified': '2023-10-16T11:26:44.991714+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12987}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12987': {'gid': 12987, 'name': 'b', 'rank': 0, 'incarnation': 144, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/3899307325', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6833', 'nonce': 3899307325}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 3899307325}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [35], 'metadata_pool': 34, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T11:26:49.269 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T11:26:49.269 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-16T11:26:49.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.492+0000 7f7e95e4b640 1 Processor -- start 2023-10-16T11:26:49.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.493+0000 7f7e95e4b640 1 -- start start 2023-10-16T11:26:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.494+0000 7f7e95e4b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e900cf710 0x7f7e900b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.494+0000 7f7e95e4b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e900d18c0 con 0x7f7e900cfaf0 2023-10-16T11:26:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.494+0000 7f7e95e4b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e900d3980 con 0x7f7e900b6cd0 2023-10-16T11:26:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.494+0000 7f7e95e4b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900d3c60 con 0x7f7e900cf710 2023-10-16T11:26:49.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.494+0000 7f7e87fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7e900cfaf0 0x7f7e900d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40964/0 (socket says 172.21.15.84:40964) 2023-10-16T11:26:49.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.494+0000 7f7e87fff640 1 -- 172.21.15.84:0/1178433699 learned_addr learned my addr 172.21.15.84:0/1178433699 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:49.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.495+0000 7f7e8f7fe640 1 --2- 172.21.15.84:0/1178433699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e900cf710 0x7f7e900b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.495+0000 7f7e8effd640 1 -- 172.21.15.84:0/1178433699 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922386935 0 0) 0x7f7e900d18c0 con 0x7f7e900cfaf0 2023-10-16T11:26:49.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.495+0000 7f7e8effd640 1 -- 172.21.15.84:0/1178433699 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e7c003520 con 0x7f7e900cfaf0 2023-10-16T11:26:49.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.495+0000 7f7e8effd640 1 -- 172.21.15.84:0/1178433699 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2527826449 0 0) 0x7f7e7c003520 con 0x7f7e900cfaf0 2023-10-16T11:26:49.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.495+0000 7f7e8effd640 1 -- 172.21.15.84:0/1178433699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e900cf710 msgr2=0x7f7e900b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:49.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.495+0000 7f7e8effd640 1 --2- 172.21.15.84:0/1178433699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e900cf710 0x7f7e900b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.495+0000 7f7e8effd640 1 -- 172.21.15.84:0/1178433699 >> v1:172.21.15.157:6789/0 conn(0x7f7e900b6cd0 legacy=0x7f7e900d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:49.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.495+0000 7f7e8effd640 1 -- 172.21.15.84:0/1178433699 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e900ce630 con 0x7f7e900cfaf0 2023-10-16T11:26:49.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.495+0000 7f7e8effd640 1 -- 172.21.15.84:0/1178433699 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f7e78001670 con 0x7f7e900cfaf0 2023-10-16T11:26:49.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.495+0000 7f7e8effd640 1 -- 172.21.15.84:0/1178433699 >> v1:172.21.15.84:6789/0 conn(0x7f7e900cfaf0 legacy=0x7f7e900d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:49.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.495+0000 7f7e8effd640 1 --2- 172.21.15.84:0/1178433699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e7c0039a0 0x7f7e7c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.495+0000 7f7e8effd640 1 --2- 172.21.15.84:0/1178433699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e900cf710 0x7f7e7c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.495+0000 7f7e8effd640 1 --2- 172.21.15.84:0/1178433699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c004d00 0x7f7e7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.495+0000 7f7e8effd640 1 -- 172.21.15.84:0/1178433699 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900d3c60 con 0x7f7e7c0039a0 2023-10-16T11:26:49.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.495+0000 7f7e8effd640 1 -- 172.21.15.84:0/1178433699 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c00bc00 con 0x7f7e7c004d00 2023-10-16T11:26:49.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.495+0000 7f7e8effd640 1 -- 172.21.15.84:0/1178433699 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c00bdc0 con 0x7f7e900cf710 2023-10-16T11:26:49.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.495+0000 7f7e8f7fe640 1 --2- 172.21.15.84:0/1178433699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e7c0039a0 0x7f7e7c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.496+0000 7f7e87fff640 1 --2- 172.21.15.84:0/1178433699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e900cf710 0x7f7e7c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.496+0000 7f7e8ffff640 1 --2- 172.21.15.84:0/1178433699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c004d00 0x7f7e7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.496+0000 7f7e8f7fe640 1 -- 172.21.15.84:0/1178433699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e900cf710 msgr2=0x7f7e7c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:49.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.496+0000 7f7e8f7fe640 1 --2- 172.21.15.84:0/1178433699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e900cf710 0x7f7e7c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.496+0000 7f7e8f7fe640 1 -- 172.21.15.84:0/1178433699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c004d00 msgr2=0x7f7e7c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:49.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.496+0000 7f7e8f7fe640 1 --2- 172.21.15.84:0/1178433699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c004d00 0x7f7e7c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.496+0000 7f7e8f7fe640 1 -- 172.21.15.84:0/1178433699 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7c003520 con 0x7f7e7c0039a0 2023-10-16T11:26:49.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.497+0000 7f7e8f7fe640 1 --2- 172.21.15.84:0/1178433699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e7c0039a0 0x7f7e7c003dc0 secure :-1 s=READY pgs=3229 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000ba10 tx=0x7f7e7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88b5aa127aba1ac server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:49.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.497+0000 7f7e8effd640 1 -- 172.21.15.84:0/1178433699 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80005b90 con 0x7f7e7c0039a0 2023-10-16T11:26:49.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.497+0000 7f7e8effd640 1 -- 172.21.15.84:0/1178433699 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e80005cf0 con 0x7f7e7c0039a0 2023-10-16T11:26:49.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.497+0000 7f7e8effd640 1 -- 172.21.15.84:0/1178433699 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80006270 con 0x7f7e7c0039a0 2023-10-16T11:26:49.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.498+0000 7f7e95e4b640 1 -- 172.21.15.84:0/1178433699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e7c0039a0 msgr2=0x7f7e7c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:49.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.498+0000 7f7e95e4b640 1 --2- 172.21.15.84:0/1178433699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e7c0039a0 0x7f7e7c003dc0 secure :-1 s=READY pgs=3229 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000ba10 tx=0x7f7e7c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.498+0000 7f7e8ffff640 1 -- 172.21.15.84:0/1178433699 reap_dead start 2023-10-16T11:26:49.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.498+0000 7f7e95e4b640 1 -- 172.21.15.84:0/1178433699 shutdown_connections 2023-10-16T11:26:49.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.498+0000 7f7e95e4b640 1 -- 172.21.15.84:0/1178433699 >> 172.21.15.84:0/1178433699 conn(0x7f7e900d2cc0 msgr2=0x7f7e900d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:49.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.498+0000 7f7e95e4b640 1 -- 172.21.15.84:0/1178433699 shutdown_connections 2023-10-16T11:26:49.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.498+0000 7f7e95e4b640 1 -- 172.21.15.84:0/1178433699 wait complete. 2023-10-16T11:26:49.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.500+0000 7f7e95e4b640 1 Processor -- start 2023-10-16T11:26:49.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.500+0000 7f7e95e4b640 1 -- start start 2023-10-16T11:26:49.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.501+0000 7f7e95e4b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e7c0039a0 0x7f7e901596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.501+0000 7f7e95e4b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e90158130 0x7f7e9015f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.501+0000 7f7e95e4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9015da70 0x7f7e90152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.501+0000 7f7e95e4b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e90159c10 con 0x7f7e7c0039a0 2023-10-16T11:26:49.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.501+0000 7f7e95e4b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e90158630 con 0x7f7e9015da70 2023-10-16T11:26:49.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.501+0000 7f7e95e4b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e90158910 con 0x7f7e90158130 2023-10-16T11:26:49.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.501+0000 7f7e8f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e7c0039a0 0x7f7e901596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.501+0000 7f7e8f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e7c0039a0 0x7f7e901596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51676/0 (socket says 172.21.15.84:51676) 2023-10-16T11:26:49.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.501+0000 7f7e8f7fe640 1 -- 172.21.15.84:0/2662676347 learned_addr learned my addr 172.21.15.84:0/2662676347 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:49.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.502+0000 7f7e87fff640 1 --2- 172.21.15.84:0/2662676347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e90158130 0x7f7e9015f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.502+0000 7f7e8ffff640 1 --2- 172.21.15.84:0/2662676347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9015da70 0x7f7e90152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.502+0000 7f7e8f7fe640 1 -- 172.21.15.84:0/2662676347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e90158130 msgr2=0x7f7e9015f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:49.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.502+0000 7f7e8f7fe640 1 --2- 172.21.15.84:0/2662676347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e90158130 0x7f7e9015f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.502+0000 7f7e8f7fe640 1 -- 172.21.15.84:0/2662676347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9015da70 msgr2=0x7f7e90152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:49.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.502+0000 7f7e8f7fe640 1 --2- 172.21.15.84:0/2662676347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9015da70 0x7f7e90152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.502+0000 7f7e8f7fe640 1 -- 172.21.15.84:0/2662676347 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e90158bf0 con 0x7f7e7c0039a0 2023-10-16T11:26:49.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.502+0000 7f7e8ffff640 1 --2- 172.21.15.84:0/2662676347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9015da70 0x7f7e90152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:49.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.502+0000 7f7e8f7fe640 1 --2- 172.21.15.84:0/2662676347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e7c0039a0 0x7f7e901596d0 secure :-1 s=READY pgs=3230 cs=0 l=1 rev1=1 crypto rx=0x7f7e80034040 tx=0x7f7e80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:49.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.502+0000 7f7e8cff9640 1 -- 172.21.15.84:0/2662676347 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80047680 con 0x7f7e7c0039a0 2023-10-16T11:26:49.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.503+0000 7f7e8cff9640 1 -- 172.21.15.84:0/2662676347 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e80040070 con 0x7f7e7c0039a0 2023-10-16T11:26:49.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.503+0000 7f7e95e4b640 1 -- 172.21.15.84:0/2662676347 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e90158ed0 con 0x7f7e7c0039a0 2023-10-16T11:26:49.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.503+0000 7f7e8cff9640 1 -- 172.21.15.84:0/2662676347 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e800479e0 con 0x7f7e7c0039a0 2023-10-16T11:26:49.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.503+0000 7f7e95e4b640 1 -- 172.21.15.84:0/2662676347 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9015fe80 con 0x7f7e7c0039a0 2023-10-16T11:26:49.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.505+0000 7f7e8cff9640 1 -- 172.21.15.84:0/2662676347 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 151) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f7e80047b40 con 0x7f7e7c0039a0 2023-10-16T11:26:49.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.506+0000 7f7e8cff9640 1 --2- 172.21.15.84:0/2662676347 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7e7006b0e0 0x7f7e7006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.506+0000 7f7e8cff9640 1 -- 172.21.15.84:0/2662676347 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5780+0+0 (secure 0 0 0) 0x7f7e80042070 con 0x7f7e7c0039a0 2023-10-16T11:26:49.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.506+0000 7f7e87fff640 1 --2- 172.21.15.84:0/2662676347 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7e7006b0e0 0x7f7e7006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.506+0000 7f7e989db740 1 Processor -- start 2023-10-16T11:26:49.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.506+0000 7f7e87fff640 1 --2- 172.21.15.84:0/2662676347 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7e7006b0e0 0x7f7e7006d5a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e78000ff0 tx=0x7f7e78009040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:49.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.507+0000 7f7e989db740 1 -- start start 2023-10-16T11:26:49.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.507+0000 7f7e989db740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6df80 0x560f86f40e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.507+0000 7f7e989db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f86f6e360 0x560f86f70d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.507+0000 7f7e989db740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560f86f413c0 0x560f86f71250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.507+0000 7f7e989db740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f86e617a0 con 0x560f86f6df80 2023-10-16T11:26:49.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.508+0000 7f7e989db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f86eb3740 con 0x560f86f6e360 2023-10-16T11:26:49.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.508+0000 7f7e989db740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f86eb0e30 con 0x560f86f413c0 2023-10-16T11:26:49.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.508+0000 7f7e8f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6df80 0x560f86f40e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.508+0000 7f7e8f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6df80 0x560f86f40e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51692/0 (socket says 172.21.15.84:51692) 2023-10-16T11:26:49.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.508+0000 7f7e8f7fe640 1 -- 172.21.15.84:0/2408946461 learned_addr learned my addr 172.21.15.84:0/2408946461 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:49.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.508+0000 7f7e8f7fe640 1 -- 172.21.15.84:0/2408946461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560f86f413c0 msgr2=0x560f86f71250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:49.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.508+0000 7f7e8f7fe640 1 --2- 172.21.15.84:0/2408946461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560f86f413c0 0x560f86f71250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.508+0000 7f7e87fff640 1 --2- 172.21.15.84:0/2408946461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f86f6e360 0x560f86f70d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.508+0000 7f7e8f7fe640 1 -- 172.21.15.84:0/2408946461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f86f6e360 msgr2=0x560f86f70d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:49.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.508+0000 7f7e8f7fe640 1 --2- 172.21.15.84:0/2408946461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f86f6e360 0x560f86f70d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.508+0000 7f7e8f7fe640 1 -- 172.21.15.84:0/2408946461 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f86e70910 con 0x560f86f6df80 2023-10-16T11:26:49.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.508+0000 7f7e8f7fe640 1 --2- 172.21.15.84:0/2408946461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6df80 0x560f86f40e80 secure :-1 s=READY pgs=3231 cs=0 l=1 rev1=1 crypto rx=0x7f7e80079f60 tx=0x7f7e800463e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c391d9642a6a5c00 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:49.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.509+0000 7f7e857fa640 1 -- 172.21.15.84:0/2408946461 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8007a260 con 0x560f86f6df80 2023-10-16T11:26:49.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.509+0000 7f7e857fa640 1 -- 172.21.15.84:0/2408946461 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8007e3b0 con 0x560f86f6df80 2023-10-16T11:26:49.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.509+0000 7f7e857fa640 1 -- 172.21.15.84:0/2408946461 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80082560 con 0x560f86f6df80 2023-10-16T11:26:49.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.509+0000 7f7e989db740 1 -- 172.21.15.84:0/2408946461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6df80 msgr2=0x560f86f40e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:49.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.509+0000 7f7e989db740 1 --2- 172.21.15.84:0/2408946461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6df80 0x560f86f40e80 secure :-1 s=READY pgs=3231 cs=0 l=1 rev1=1 crypto rx=0x7f7e80079f60 tx=0x7f7e800463e0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.509+0000 7f7e989db740 1 -- 172.21.15.84:0/2408946461 shutdown_connections 2023-10-16T11:26:49.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.509+0000 7f7e989db740 1 --2- 172.21.15.84:0/2408946461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560f86f413c0 0x560f86f71250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.509+0000 7f7e989db740 1 --2- 172.21.15.84:0/2408946461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f86f6e360 0x560f86f70d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.509+0000 7f7e989db740 1 --2- 172.21.15.84:0/2408946461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6df80 0x560f86f40e80 unknown :-1 s=CLOSED pgs=3231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.509+0000 7f7e989db740 1 -- 172.21.15.84:0/2408946461 >> 172.21.15.84:0/2408946461 conn(0x560f86f43190 msgr2=0x560f86f8bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:49.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.509+0000 7f7e989db740 1 -- 172.21.15.84:0/2408946461 shutdown_connections 2023-10-16T11:26:49.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.509+0000 7f7e989db740 1 -- 172.21.15.84:0/2408946461 wait complete. 2023-10-16T11:26:49.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.510+0000 7f7e989db740 1 -- start start 2023-10-16T11:26:49.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.510+0000 7f7e989db740 1 Processor -- start 2023-10-16T11:26:49.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.511+0000 7f7e989db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f86f6df80 0x560f86f8b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.511+0000 7f7e989db740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6e360 0x560f86f8b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.511+0000 7f7e989db740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560f86f85e70 0x560f86f8bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.511+0000 7f7e989db740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f86eac720 con 0x560f86f6e360 2023-10-16T11:26:49.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.511+0000 7f7e989db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f86e4a970 con 0x560f86f6df80 2023-10-16T11:26:49.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.511+0000 7f7e989db740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f86e4a610 con 0x560f86f85e70 2023-10-16T11:26:49.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.511+0000 7f7e989db740 10 client.0 _ll_register_callbacks cb 0x560f86e6c9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:26:49.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.512+0000 7f7e87fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6e360 0x560f86f8b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.512+0000 7f7e87fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6e360 0x560f86f8b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51694/0 (socket says 172.21.15.84:51694) 2023-10-16T11:26:49.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.512+0000 7f7e87fff640 1 -- 172.21.15.84:0/1660261425 learned_addr learned my addr 172.21.15.84:0/1660261425 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:49.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.512+0000 7f7e87fff640 1 -- 172.21.15.84:0/1660261425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560f86f85e70 msgr2=0x560f86f8bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:49.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.512+0000 7f7e87fff640 1 --2- 172.21.15.84:0/1660261425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560f86f85e70 0x560f86f8bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.512+0000 7f7e87fff640 1 -- 172.21.15.84:0/1660261425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f86f6df80 msgr2=0x560f86f8b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:49.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.512+0000 7f7e8f7fe640 1 --2- 172.21.15.84:0/1660261425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f86f6df80 0x560f86f8b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.512+0000 7f7e87fff640 1 --2- 172.21.15.84:0/1660261425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f86f6df80 0x560f86f8b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.512+0000 7f7e87fff640 1 -- 172.21.15.84:0/1660261425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9015fe80 con 0x560f86f6e360 2023-10-16T11:26:49.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.512+0000 7f7e87fff640 1 --2- 172.21.15.84:0/1660261425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6e360 0x560f86f8b9f0 secure :-1 s=READY pgs=3232 cs=0 l=1 rev1=1 crypto rx=0x7f7e78001660 tx=0x7f7e78001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e2094186169101 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:49.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.512+0000 7f7e6e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T11:26:49.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.513+0000 7f7e6e7fc640 1 -- 172.21.15.84:0/1660261425 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800c050 con 0x560f86f6e360 2023-10-16T11:26:49.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.513+0000 7f7e6e7fc640 1 -- 172.21.15.84:0/1660261425 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(257..257 src has 1..257) v4 ==== 5780+0+0 (secure 0 0 0) 0x7f7e78021ad0 con 0x560f86f6e360 2023-10-16T11:26:49.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.513+0000 7f7e989db740 1 -- 172.21.15.84:0/1660261425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f86ed3160 con 0x560f86f6e360 2023-10-16T11:26:49.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.513+0000 7f7e6e7fc640 1 -- 172.21.15.84:0/1660261425 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e78021c90 con 0x560f86f6e360 2023-10-16T11:26:49.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.513+0000 7f7e989db740 10 client.13107 fetch_fsmap learned FSMap version 148 2023-10-16T11:26:49.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.513+0000 7f7e989db740 1 -- 172.21.15.84:0/1660261425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=148,monmap=2+}) v3 -- 0x560f86eaada0 con 0x560f86f6e360 2023-10-16T11:26:49.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.513+0000 7f7e6e7fc640 1 -- 172.21.15.84:0/1660261425 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560f86ed3160 con 0x560f86f6e360 2023-10-16T11:26:49.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.513+0000 7f7e6e7fc640 1 -- 172.21.15.84:0/1660261425 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 148) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7e78021420 con 0x560f86f6e360 2023-10-16T11:26:49.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.513+0000 7f7e989db740 10 client.13107 fetch_fsmap finished waiting for FSMap version 148 2023-10-16T11:26:49.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.513+0000 7f7e989db740 10 client.13107 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T11:26:49.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.513+0000 7f7e989db740 20 client.13107 populate_metadata read hostname 'smithi084' 2023-10-16T11:26:49.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.513+0000 7f7e989db740 1 --2- 172.21.15.84:0/1660261425 >> [v2:172.21.15.157:6833/3899307325,v1:172.21.15.157:6835/3899307325] conn(0x560f86f5cfc0 0x560f86f5e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.513+0000 7f7e989db740 4 client.13107 mds_command: new command op to 12987 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T11:26:49.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.513+0000 7f7e989db740 1 -- 172.21.15.84:0/1660261425 --> [v2:172.21.15.157:6833/3899307325,v1:172.21.15.157:6835/3899307325] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f86e7ecc0 con 0x560f86f5cfc0 2023-10-16T11:26:49.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.514+0000 7f7e8f7fe640 1 --2- 172.21.15.84:0/1660261425 >> [v2:172.21.15.157:6833/3899307325,v1:172.21.15.157:6835/3899307325] conn(0x560f86f5cfc0 0x560f86f5e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.515+0000 7f7e8f7fe640 1 --2- 172.21.15.84:0/1660261425 >> [v2:172.21.15.157:6833/3899307325,v1:172.21.15.157:6835/3899307325] conn(0x560f86f5cfc0 0x560f86f5e000 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6288dbaa03f92a75 server_cookie=d91aa16f625227ea in_seq=0 out_seq=0 2023-10-16T11:26:49.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.515+0000 7f7e6e7fc640 10 client.13107 ms_handle_connect on v2:172.21.15.157:6833/3899307325 2023-10-16T11:26:49.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.517+0000 7f7e6e7fc640 1 -- 172.21.15.84:0/1660261425 <== mds.0 v2:172.21.15.157:6833/3899307325 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560f86e7ecc0 con 0x560f86f5cfc0 2023-10-16T11:26:49.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.517+0000 7f7e6e7fc640 10 client.13107 handle_command_reply: tid=0 2023-10-16T11:26:49.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.517+0000 7f7e989db740 1 client.13107 shutdown 2023-10-16T11:26:49.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.517+0000 7f7e989db740 1 -- 172.21.15.84:0/1660261425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6e360 msgr2=0x560f86f8b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:49.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.517+0000 7f7e989db740 1 --2- 172.21.15.84:0/1660261425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6e360 0x560f86f8b9f0 secure :-1 s=READY pgs=3232 cs=0 l=1 rev1=1 crypto rx=0x7f7e78001660 tx=0x7f7e78001930 comp rx=0 tx=0).stop 2023-10-16T11:26:49.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.517+0000 7f7e989db740 1 -- 172.21.15.84:0/1660261425 shutdown_connections 2023-10-16T11:26:49.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.517+0000 7f7e989db740 1 --2- 172.21.15.84:0/1660261425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560f86f85e70 0x560f86f8bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.517+0000 7f7e989db740 1 --2- 172.21.15.84:0/1660261425 >> [v2:172.21.15.157:6833/3899307325,v1:172.21.15.157:6835/3899307325] conn(0x560f86f5cfc0 0x560f86f5e000 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.517+0000 7f7e989db740 1 --2- 172.21.15.84:0/1660261425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6e360 0x560f86f8b9f0 unknown :-1 s=CLOSED pgs=3232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.517+0000 7f7e989db740 1 --2- 172.21.15.84:0/1660261425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f86f6df80 0x560f86f8b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.517+0000 7f7e6e7fc640 0 client.13107 ms_handle_reset on v2:172.21.15.157:6833/3899307325 2023-10-16T11:26:49.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.517+0000 7f7e989db740 1 -- 172.21.15.84:0/1660261425 >> 172.21.15.84:0/1660261425 conn(0x560f86f413c0 msgr2=0x560f86f8f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:49.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.517+0000 7f7e989db740 1 -- 172.21.15.84:0/1660261425 shutdown_connections 2023-10-16T11:26:49.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.517+0000 7f7e989db740 1 -- 172.21.15.84:0/1660261425 wait complete. 2023-10-16T11:26:49.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.518+0000 7f7e989db740 20 client.13107 trim_cache size 0 max 16384 2023-10-16T11:26:49.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.534+0000 7f7e989db740 1 Processor -- start 2023-10-16T11:26:49.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.534+0000 7f7e989db740 1 -- start start 2023-10-16T11:26:49.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.535+0000 7f7e989db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f86f6df80 0x560f86f8b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.535+0000 7f7e989db740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6e360 0x560f86f8a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.535+0000 7f7e989db740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560f86f85e70 0x560f86f70e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.535+0000 7f7e989db740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f86c10f20 con 0x560f86f6e360 2023-10-16T11:26:49.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.535+0000 7f7e989db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f86f9e940 con 0x560f86f6df80 2023-10-16T11:26:49.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.535+0000 7f7e989db740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f86e6bf10 con 0x560f86f85e70 2023-10-16T11:26:49.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.535+0000 7f7e87fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6e360 0x560f86f8a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.535+0000 7f7e87fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6e360 0x560f86f8a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51706/0 (socket says 172.21.15.84:51706) 2023-10-16T11:26:49.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.535+0000 7f7e87fff640 1 -- 172.21.15.84:0/3025926378 learned_addr learned my addr 172.21.15.84:0/3025926378 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:49.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.535+0000 7f7e87fff640 1 -- 172.21.15.84:0/3025926378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560f86f85e70 msgr2=0x560f86f70e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:49.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.535+0000 7f7e87fff640 1 --2- 172.21.15.84:0/3025926378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560f86f85e70 0x560f86f70e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.535+0000 7f7e87fff640 1 -- 172.21.15.84:0/3025926378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f86f6df80 msgr2=0x560f86f8b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:49.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.535+0000 7f7e87fff640 1 --2- 172.21.15.84:0/3025926378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f86f6df80 0x560f86f8b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.535+0000 7f7e87fff640 1 -- 172.21.15.84:0/3025926378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f86e49b80 con 0x560f86f6e360 2023-10-16T11:26:49.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.536+0000 7f7e87fff640 1 --2- 172.21.15.84:0/3025926378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6e360 0x560f86f8a990 secure :-1 s=READY pgs=3233 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800ef90 tx=0x7f7e7801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5889c3f4176f06 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:49.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.536+0000 7f7e85ffb640 1 -- 172.21.15.84:0/3025926378 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800c070 con 0x560f86f6e360 2023-10-16T11:26:49.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.536+0000 7f7e85ffb640 1 -- 172.21.15.84:0/3025926378 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7801b050 con 0x560f86f6e360 2023-10-16T11:26:49.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.536+0000 7f7e85ffb640 1 -- 172.21.15.84:0/3025926378 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e78021ba0 con 0x560f86f6e360 2023-10-16T11:26:49.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.536+0000 7f7e989db740 1 -- 172.21.15.84:0/3025926378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6e360 msgr2=0x560f86f8a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:49.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.536+0000 7f7e989db740 1 --2- 172.21.15.84:0/3025926378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6e360 0x560f86f8a990 secure :-1 s=READY pgs=3233 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800ef90 tx=0x7f7e7801e000 comp rx=0 tx=0).stop 2023-10-16T11:26:49.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.536+0000 7f7e989db740 1 -- 172.21.15.84:0/3025926378 shutdown_connections 2023-10-16T11:26:49.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.536+0000 7f7e989db740 1 --2- 172.21.15.84:0/3025926378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560f86f85e70 0x560f86f70e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.536+0000 7f7e989db740 1 --2- 172.21.15.84:0/3025926378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6e360 0x560f86f8a990 unknown :-1 s=CLOSED pgs=3233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.536+0000 7f7e989db740 1 --2- 172.21.15.84:0/3025926378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f86f6df80 0x560f86f8b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.536+0000 7f7e989db740 1 -- 172.21.15.84:0/3025926378 >> 172.21.15.84:0/3025926378 conn(0x560f86f413c0 msgr2=0x560f86f47f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:49.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.537+0000 7f7e989db740 1 -- 172.21.15.84:0/3025926378 shutdown_connections 2023-10-16T11:26:49.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.537+0000 7f7e989db740 1 -- 172.21.15.84:0/3025926378 wait complete. 2023-10-16T11:26:49.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.537+0000 7f7e989db740 1 -- start start 2023-10-16T11:26:49.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.537+0000 7f7e989db740 1 Processor -- start 2023-10-16T11:26:49.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.538+0000 7f7e989db740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6df80 0x560f86ff3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.538+0000 7f7e989db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f86f6e360 0x560f86f44b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.538+0000 7f7e989db740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560f86f85e70 0x560f86f45060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.538+0000 7f7e989db740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f86f71df0 con 0x560f86f6df80 2023-10-16T11:26:49.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.538+0000 7f7e989db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f86f88e50 con 0x560f86f6e360 2023-10-16T11:26:49.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.538+0000 7f7e989db740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f86f8dd20 con 0x560f86f85e70 2023-10-16T11:26:49.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.538+0000 7f7e989db740 10 client.0 _ll_register_callbacks cb 0x7f7e78001d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:26:49.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.539+0000 7f7e8f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6df80 0x560f86ff3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.539+0000 7f7e8f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6df80 0x560f86ff3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51722/0 (socket says 172.21.15.84:51722) 2023-10-16T11:26:49.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.539+0000 7f7e8f7fe640 1 -- 172.21.15.84:0/1117387154 learned_addr learned my addr 172.21.15.84:0/1117387154 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:49.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.539+0000 7f7e8f7fe640 1 -- 172.21.15.84:0/1117387154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560f86f85e70 msgr2=0x560f86f45060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:49.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.539+0000 7f7e8f7fe640 1 --2- 172.21.15.84:0/1117387154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560f86f85e70 0x560f86f45060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.539+0000 7f7e8f7fe640 1 -- 172.21.15.84:0/1117387154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f86f6e360 msgr2=0x560f86f44b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:49.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.539+0000 7f7e8f7fe640 1 --2- 172.21.15.84:0/1117387154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f86f6e360 0x560f86f44b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.539+0000 7f7e8f7fe640 1 -- 172.21.15.84:0/1117387154 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f86eaada0 con 0x560f86f6df80 2023-10-16T11:26:49.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.539+0000 7f7e8f7fe640 1 --2- 172.21.15.84:0/1117387154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6df80 0x560f86ff3940 secure :-1 s=READY pgs=3234 cs=0 l=1 rev1=1 crypto rx=0x7f7e8002bf90 tx=0x7f7e80000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab5c9c31ece7bc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:49.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.539+0000 7f7e6e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T11:26:49.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.539+0000 7f7e6e7fc640 1 -- 172.21.15.84:0/1117387154 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8007a260 con 0x560f86f6df80 2023-10-16T11:26:49.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.540+0000 7f7e6e7fc640 1 -- 172.21.15.84:0/1117387154 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(257..257 src has 1..257) v4 ==== 5780+0+0 (secure 0 0 0) 0x7f7e80082560 con 0x560f86f6df80 2023-10-16T11:26:49.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.540+0000 7f7e989db740 1 -- 172.21.15.84:0/1117387154 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e8007e3b0 con 0x560f86f6df80 2023-10-16T11:26:49.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.540+0000 7f7e6e7fc640 1 -- 172.21.15.84:0/1117387154 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e80078600 con 0x560f86f6df80 2023-10-16T11:26:49.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.540+0000 7f7e989db740 10 client.13113 fetch_fsmap learned FSMap version 148 2023-10-16T11:26:49.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.540+0000 7f7e989db740 1 -- 172.21.15.84:0/1117387154 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=148,monmap=2+}) v3 -- 0x560f86e6f170 con 0x560f86f6df80 2023-10-16T11:26:49.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.540+0000 7f7e6e7fc640 1 -- 172.21.15.84:0/1117387154 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8007e3b0 con 0x560f86f6df80 2023-10-16T11:26:49.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.540+0000 7f7e6e7fc640 1 -- 172.21.15.84:0/1117387154 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 148) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7e80076970 con 0x560f86f6df80 2023-10-16T11:26:49.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.540+0000 7f7e989db740 10 client.13113 fetch_fsmap finished waiting for FSMap version 148 2023-10-16T11:26:49.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.540+0000 7f7e989db740 10 client.13113 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T11:26:49.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.540+0000 7f7e989db740 20 client.13113 populate_metadata read hostname 'smithi084' 2023-10-16T11:26:49.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.540+0000 7f7e989db740 1 --2- 172.21.15.84:0/1117387154 >> [v2:172.21.15.157:6833/3899307325,v1:172.21.15.157:6835/3899307325] conn(0x560f86f9a9a0 0x560f86f9ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.540+0000 7f7e989db740 4 client.13113 mds_command: new command op to 12987 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T11:26:49.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.540+0000 7f7e989db740 1 -- 172.21.15.84:0/1117387154 --> [v2:172.21.15.157:6833/3899307325,v1:172.21.15.157:6835/3899307325] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560f86ea5700 con 0x560f86f9a9a0 2023-10-16T11:26:49.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.541+0000 7f7e87fff640 1 --2- 172.21.15.84:0/1117387154 >> [v2:172.21.15.157:6833/3899307325,v1:172.21.15.157:6835/3899307325] conn(0x560f86f9a9a0 0x560f86f9ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.542+0000 7f7e6e7fc640 10 client.13113 ms_handle_connect on v2:172.21.15.157:6833/3899307325 2023-10-16T11:26:49.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.542+0000 7f7e87fff640 1 --2- 172.21.15.84:0/1117387154 >> [v2:172.21.15.157:6833/3899307325,v1:172.21.15.157:6835/3899307325] conn(0x560f86f9a9a0 0x560f86f9ad80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1beb4ad779504715 server_cookie=2a6024167242a0c5 in_seq=0 out_seq=0 2023-10-16T11:26:49.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.542+0000 7f7e6e7fc640 1 -- 172.21.15.84:0/1117387154 <== mds.0 v2:172.21.15.157:6833/3899307325 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x560f86ea5700 con 0x560f86f9a9a0 2023-10-16T11:26:49.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.542+0000 7f7e6e7fc640 10 client.13113 handle_command_reply: tid=0 2023-10-16T11:26:49.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.542+0000 7f7e989db740 1 client.13113 shutdown 2023-10-16T11:26:49.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.543+0000 7f7e989db740 1 -- 172.21.15.84:0/1117387154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6df80 msgr2=0x560f86ff3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:49.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.543+0000 7f7e989db740 1 --2- 172.21.15.84:0/1117387154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6df80 0x560f86ff3940 secure :-1 s=READY pgs=3234 cs=0 l=1 rev1=1 crypto rx=0x7f7e8002bf90 tx=0x7f7e80000bc0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.543+0000 7f7e989db740 1 -- 172.21.15.84:0/1117387154 shutdown_connections 2023-10-16T11:26:49.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.543+0000 7f7e989db740 1 --2- 172.21.15.84:0/1117387154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x560f86f85e70 0x560f86f45060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.543+0000 7f7e989db740 1 --2- 172.21.15.84:0/1117387154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f86f6e360 0x560f86f44b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.543+0000 7f7e989db740 1 --2- 172.21.15.84:0/1117387154 >> [v2:172.21.15.157:6833/3899307325,v1:172.21.15.157:6835/3899307325] conn(0x560f86f9a9a0 0x560f86f9ad80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.543+0000 7f7e989db740 1 --2- 172.21.15.84:0/1117387154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f86f6df80 0x560f86ff3940 unknown :-1 s=CLOSED pgs=3234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.543+0000 7f7e6e7fc640 0 client.13113 ms_handle_reset on v2:172.21.15.157:6833/3899307325 2023-10-16T11:26:49.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.543+0000 7f7e989db740 1 -- 172.21.15.84:0/1117387154 >> 172.21.15.84:0/1117387154 conn(0x560f86f413c0 msgr2=0x560f86f8f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:49.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.543+0000 7f7e989db740 1 -- 172.21.15.84:0/1117387154 shutdown_connections 2023-10-16T11:26:49.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.543+0000 7f7e989db740 1 -- 172.21.15.84:0/1117387154 wait complete. 2023-10-16T11:26:49.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.543+0000 7f7e989db740 20 client.13113 trim_cache size 0 max 16384 2023-10-16T11:26:49.558 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:26:49.559 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "63fb4d68-f881-4694-b880-d10369105c0e", 2023-10-16T11:26:49.559 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-16T11:26:49.559 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 12987, 2023-10-16T11:26:49.559 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-16T11:26:49.559 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-16T11:26:49.560 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-16T11:26:49.560 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 7.0704712680000004, 2023-10-16T11:26:49.560 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 148, 2023-10-16T11:26:49.560 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 257, 2023-10-16T11:26:49.560 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 257, 2023-10-16T11:26:49.561 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 12.425252958 2023-10-16T11:26:49.561 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:26:49.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.544+0000 7f7e95e4b640 1 -- 172.21.15.84:0/2662676347 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7e7006b0e0 msgr2=0x7f7e7006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:49.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.544+0000 7f7e95e4b640 1 --2- 172.21.15.84:0/2662676347 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7e7006b0e0 0x7f7e7006d5a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e78000ff0 tx=0x7f7e78009040 comp rx=0 tx=0).stop 2023-10-16T11:26:49.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.544+0000 7f7e95e4b640 1 -- 172.21.15.84:0/2662676347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e7c0039a0 msgr2=0x7f7e901596d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:49.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.544+0000 7f7e95e4b640 1 --2- 172.21.15.84:0/2662676347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e7c0039a0 0x7f7e901596d0 secure :-1 s=READY pgs=3230 cs=0 l=1 rev1=1 crypto rx=0x7f7e80034040 tx=0x7f7e80004730 comp rx=0 tx=0).stop 2023-10-16T11:26:49.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.544+0000 7f7e95e4b640 1 -- 172.21.15.84:0/2662676347 shutdown_connections 2023-10-16T11:26:49.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.544+0000 7f7e95e4b640 1 --2- 172.21.15.84:0/2662676347 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f7e7006b0e0 0x7f7e7006d5a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.544+0000 7f7e95e4b640 1 --2- 172.21.15.84:0/2662676347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9015da70 0x7f7e90152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.544+0000 7f7e95e4b640 1 --2- 172.21.15.84:0/2662676347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e90158130 0x7f7e9015f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.544+0000 7f7e95e4b640 1 --2- 172.21.15.84:0/2662676347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e7c0039a0 0x7f7e901596d0 unknown :-1 s=CLOSED pgs=3230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.544+0000 7f7e95e4b640 1 -- 172.21.15.84:0/2662676347 >> 172.21.15.84:0/2662676347 conn(0x7f7e900d2cc0 msgr2=0x7f7e900b7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:49.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.544+0000 7f7e95e4b640 1 -- 172.21.15.84:0/2662676347 shutdown_connections 2023-10-16T11:26:49.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.544+0000 7f7e95e4b640 1 -- 172.21.15.84:0/2662676347 wait complete. 2023-10-16T11:26:49.565 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T11:26:49.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.762+0000 7f543d0e7640 1 Processor -- start 2023-10-16T11:26:49.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.762+0000 7f543d0e7640 1 -- start start 2023-10-16T11:26:49.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.763+0000 7f543d0e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f543815ae80 0x7f543815e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.763+0000 7f543d0e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54380ce390 0x7f5438070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.764+0000 7f543d0e7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543815ed60 con 0x7f543815aaa0 2023-10-16T11:26:49.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.764+0000 7f543d0e7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f543806b810 con 0x7f54380ce390 2023-10-16T11:26:49.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.764+0000 7f543d0e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54380707e0 con 0x7f543815ae80 2023-10-16T11:26:49.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.764+0000 7f543759e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54380ce390 0x7f5438070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.764+0000 7f542ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f543815ae80 0x7f543815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.764+0000 7f543759e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54380ce390 0x7f5438070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:38594/0 (socket says 172.21.15.84:38594) 2023-10-16T11:26:49.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.764+0000 7f543759e640 1 -- 172.21.15.84:0/1206201228 learned_addr learned my addr 172.21.15.84:0/1206201228 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:49.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.765+0000 7f543759e640 1 -- 172.21.15.84:0/1206201228 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f543815ae80 msgr2=0x7f543815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:49.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.765+0000 7f543759e640 1 --2- 172.21.15.84:0/1206201228 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f543815ae80 0x7f543815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.765+0000 7f543759e640 1 -- 172.21.15.84:0/1206201228 >> v1:172.21.15.157:6789/0 conn(0x7f543815aaa0 legacy=0x7f54380cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:49.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.765+0000 7f543759e640 1 -- 172.21.15.84:0/1206201228 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543815ef90 con 0x7f54380ce390 2023-10-16T11:26:49.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.766+0000 7f543759e640 1 --2- 172.21.15.84:0/1206201228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54380ce390 0x7f5438070f50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f5430009e50 tx=0x7f543002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65729697f631003e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:49.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.767+0000 7f543659c640 1 -- 172.21.15.84:0/1206201228 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543002cb00 con 0x7f54380ce390 2023-10-16T11:26:49.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.767+0000 7f543659c640 1 -- 172.21.15.84:0/1206201228 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543002cc60 con 0x7f54380ce390 2023-10-16T11:26:49.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.767+0000 7f543659c640 1 -- 172.21.15.84:0/1206201228 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54300343f0 con 0x7f54380ce390 2023-10-16T11:26:49.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.767+0000 7f543d0e7640 1 -- 172.21.15.84:0/1206201228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54380ce390 msgr2=0x7f5438070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:49.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.767+0000 7f543d0e7640 1 --2- 172.21.15.84:0/1206201228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54380ce390 0x7f5438070f50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f5430009e50 tx=0x7f543002bf60 comp rx=0 tx=0).stop 2023-10-16T11:26:49.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.768+0000 7f543d0e7640 1 -- 172.21.15.84:0/1206201228 shutdown_connections 2023-10-16T11:26:49.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.768+0000 7f543d0e7640 1 --2- 172.21.15.84:0/1206201228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54380ce390 0x7f5438070f50 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.768+0000 7f543d0e7640 1 --2- 172.21.15.84:0/1206201228 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f543815ae80 0x7f543815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.768+0000 7f543d0e7640 1 -- 172.21.15.84:0/1206201228 >> 172.21.15.84:0/1206201228 conn(0x7f54380cb410 msgr2=0x7f54380cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:49.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.768+0000 7f543d0e7640 1 -- 172.21.15.84:0/1206201228 shutdown_connections 2023-10-16T11:26:49.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.768+0000 7f543d0e7640 1 -- 172.21.15.84:0/1206201228 wait complete. 2023-10-16T11:26:49.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.770+0000 7f543d0e7640 1 Processor -- start 2023-10-16T11:26:49.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.770+0000 7f543d0e7640 1 -- start start 2023-10-16T11:26:49.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.771+0000 7f543d0e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54380ce390 0x7f5438159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.771+0000 7f543d0e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f543815aaa0 0x7f54381f10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.771+0000 7f543d0e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f543815ae80 0x7f54381f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.771+0000 7f543d0e7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5438159da0 con 0x7f543815aaa0 2023-10-16T11:26:49.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.771+0000 7f543d0e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543815dd60 con 0x7f54380ce390 2023-10-16T11:26:49.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.771+0000 7f543d0e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54381f0680 con 0x7f543815ae80 2023-10-16T11:26:49.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.772+0000 7f542ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f543815aaa0 0x7f54381f10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.772+0000 7f542ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f543815aaa0 0x7f54381f10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51726/0 (socket says 172.21.15.84:51726) 2023-10-16T11:26:49.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.772+0000 7f542ffff640 1 -- 172.21.15.84:0/2059289756 learned_addr learned my addr 172.21.15.84:0/2059289756 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:49.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.772+0000 7f542ffff640 1 -- 172.21.15.84:0/2059289756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f543815ae80 msgr2=0x7f54381f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:49.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.772+0000 7f542ffff640 1 --2- 172.21.15.84:0/2059289756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f543815ae80 0x7f54381f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.772+0000 7f5436d9d640 1 --2- 172.21.15.84:0/2059289756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54380ce390 0x7f5438159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.772+0000 7f542ffff640 1 -- 172.21.15.84:0/2059289756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54380ce390 msgr2=0x7f5438159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:49.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.772+0000 7f542ffff640 1 --2- 172.21.15.84:0/2059289756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54380ce390 0x7f5438159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:49.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.772+0000 7f542ffff640 1 -- 172.21.15.84:0/2059289756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5430009b30 con 0x7f543815aaa0 2023-10-16T11:26:49.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.772+0000 7f5436d9d640 1 --2- 172.21.15.84:0/2059289756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54380ce390 0x7f5438159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:49.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.772+0000 7f542ffff640 1 --2- 172.21.15.84:0/2059289756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f543815aaa0 0x7f54381f10b0 secure :-1 s=READY pgs=3235 cs=0 l=1 rev1=1 crypto rx=0x7f542000e9e0 tx=0x7f542000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:49.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.773+0000 7f542f7fe640 1 -- 172.21.15.84:0/2059289756 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5420009670 con 0x7f543815aaa0 2023-10-16T11:26:49.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.773+0000 7f542f7fe640 1 -- 172.21.15.84:0/2059289756 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542000c9d0 con 0x7f543815aaa0 2023-10-16T11:26:49.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.773+0000 7f542f7fe640 1 -- 172.21.15.84:0/2059289756 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542000cd80 con 0x7f543815aaa0 2023-10-16T11:26:49.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.773+0000 7f543d0e7640 1 -- 172.21.15.84:0/2059289756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54381f15f0 con 0x7f543815aaa0 2023-10-16T11:26:49.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.773+0000 7f543d0e7640 1 -- 172.21.15.84:0/2059289756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54381f49a0 con 0x7f543815aaa0 2023-10-16T11:26:49.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.774+0000 7f543d0e7640 1 -- 172.21.15.84:0/2059289756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54381f4b90 con 0x7f543815aaa0 2023-10-16T11:26:49.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.775+0000 7f542f7fe640 1 -- 172.21.15.84:0/2059289756 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 151) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5420013470 con 0x7f543815aaa0 2023-10-16T11:26:49.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.776+0000 7f542f7fe640 1 --2- 172.21.15.84:0/2059289756 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f541806b0e0 0x7f541806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:49.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.776+0000 7f542f7fe640 1 -- 172.21.15.84:0/2059289756 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5780+0+0 (secure 0 0 0) 0x7f5420089c30 con 0x7f543815aaa0 2023-10-16T11:26:49.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.776+0000 7f5436d9d640 1 --2- 172.21.15.84:0/2059289756 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f541806b0e0 0x7f541806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:49.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.777+0000 7f5436d9d640 1 --2- 172.21.15.84:0/2059289756 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f541806b0e0 0x7f541806d5a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5428000f60 tx=0x7f5428009040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:49.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.780+0000 7f542f7fe640 1 -- 172.21.15.84:0/2059289756 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5420054930 con 0x7f543815aaa0 2023-10-16T11:26:49.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.998+0000 7f543d0e7640 1 -- 172.21.15.84:0/2059289756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f543804e160 con 0x7f543815aaa0 2023-10-16T11:26:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:49.998+0000 7f542f7fe640 1 -- 172.21.15.84:0/2059289756 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v148) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5420017070 con 0x7f543815aaa0 2023-10-16T11:26:50.000 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T11:26:50.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:50.000+0000 7f543d0e7640 1 -- 172.21.15.84:0/2059289756 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f541806b0e0 msgr2=0x7f541806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:50.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:50.000+0000 7f543d0e7640 1 --2- 172.21.15.84:0/2059289756 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f541806b0e0 0x7f541806d5a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5428000f60 tx=0x7f5428009040 comp rx=0 tx=0).stop 2023-10-16T11:26:50.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:50.000+0000 7f543d0e7640 1 -- 172.21.15.84:0/2059289756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f543815aaa0 msgr2=0x7f54381f10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:50.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:50.000+0000 7f543d0e7640 1 --2- 172.21.15.84:0/2059289756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f543815aaa0 0x7f54381f10b0 secure :-1 s=READY pgs=3235 cs=0 l=1 rev1=1 crypto rx=0x7f542000e9e0 tx=0x7f542000eea0 comp rx=0 tx=0).stop 2023-10-16T11:26:50.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:50.001+0000 7f543d0e7640 1 -- 172.21.15.84:0/2059289756 shutdown_connections 2023-10-16T11:26:50.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:50.001+0000 7f543d0e7640 1 --2- 172.21.15.84:0/2059289756 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f541806b0e0 0x7f541806d5a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:50.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:50.001+0000 7f543d0e7640 1 --2- 172.21.15.84:0/2059289756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f543815ae80 0x7f54381f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:50.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:50.001+0000 7f543d0e7640 1 --2- 172.21.15.84:0/2059289756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f543815aaa0 0x7f54381f10b0 unknown :-1 s=CLOSED pgs=3235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:50.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:50.001+0000 7f543d0e7640 1 --2- 172.21.15.84:0/2059289756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54380ce390 0x7f5438159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:50.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:50.001+0000 7f543d0e7640 1 -- 172.21.15.84:0/2059289756 >> 172.21.15.84:0/2059289756 conn(0x7f54380cb410 msgr2=0x7f543806f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:50.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:50.001+0000 7f543d0e7640 1 -- 172.21.15.84:0/2059289756 shutdown_connections 2023-10-16T11:26:50.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:50.001+0000 7f543d0e7640 1 -- 172.21.15.84:0/2059289756 wait complete. 2023-10-16T11:26:50.020 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T11:26:50.020 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T11:26:50.021 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-16T11:26:50.021 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T11:26:50.022 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T11:26:50.022 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T11:26:50.023 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T11:26:50.024 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T11:26:50.024 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:26:50.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:26:50.025 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-16T11:26:50.076 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T11:26:50.076 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T11:26:50.076 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T11:26:50.076 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:26:50.077 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T11:26:50.077 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:26:50.077 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:26:50.077 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:26:50.077 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-16T11:26:50.078 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:26:50.078 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:26:50.078 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-16T11:26:50.078 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:26:50.078 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:26:50.079 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T11:26:50.079 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:26:50.079 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T11:26:50.079 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 36541sec preferred_lft 36541sec 2023-10-16T11:26:50.079 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-16T11:26:50.080 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:26:50.080 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T11:26:50.080 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:26:50.080 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T11:26:50.080 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:26:50.081 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::20a8:eaff:fe9d:5cea/64 scope link 2023-10-16T11:26:50.081 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:26:50.081 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:26:50.081 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T11:26:50.081 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1854:13ff:fe47:be92/64 scope link 2023-10-16T11:26:50.082 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:26:50.082 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:26:50.082 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether ee:d5:41:a3:a5:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T11:26:50.082 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ecd5:41ff:fea3:a570/64 scope link 2023-10-16T11:26:50.082 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:26:50.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:26:50.084 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T11:26:50.173 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-16T11:26:50.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:26:50.178 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-16T11:26:50.235 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-16T11:26:50.235 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T11:26:50.235 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T11:26:50.236 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T11:26:50.236 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T11:26:50.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:26:50.237 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T11:26:50.237 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T11:26:50.238 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T11:26:50.238 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T11:26:50.239 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T11:26:50.239 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T11:26:50.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:26:50.432 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T11:26:50.487 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T11:26:50.488 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T11:26:50.489 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:26:50.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:26:50.489 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-16T11:26:50.575 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:26:50.576 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T11:26:50.629 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:26:50.630 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:26:50.630 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:26:50.685 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-16T11:26:50.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:26:50.686 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:26:50.823 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:26:50.823 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T11:26:50.858 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:26:50.858 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T11:26:50.858 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T11:26:50.858 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with new device syntax: 0@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T11:26:50.858 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T11:26:50.861 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-16T11:26:50.861 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2023-10-16T11:26:50.861 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T11:26:50.861 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T11:26:50.862 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T11:26:50.864 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T11:26:50.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:26:50.865 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:26:50.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:26:50.953 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T11:26:51.033 INFO:teuthology.orchestra.run.smithi084.stdout:client8242 2023-10-16T11:26:51.033 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:26:51.034 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:26:51.091 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T11:26:51.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.300+0000 7f617593e640 1 Processor -- start 2023-10-16T11:26:51.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.300+0000 7f617593e640 1 -- start start 2023-10-16T11:26:51.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.302+0000 7f617593e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61700addb0 0x7f61700caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:51.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.302+0000 7f617593e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f61700c9000 0x7f6170069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:51.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.302+0000 7f617593e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61700cb140 con 0x7f617006a2e0 2023-10-16T11:26:51.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.302+0000 7f617593e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6170161a50 con 0x7f61700c9000 2023-10-16T11:26:51.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.302+0000 7f617593e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617006a750 con 0x7f61700addb0 2023-10-16T11:26:51.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.303+0000 7f616e7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f61700c9000 0x7f6170069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:51.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.303+0000 7f616e7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f61700c9000 0x7f6170069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51728/0 (socket says 172.21.15.84:51728) 2023-10-16T11:26:51.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.303+0000 7f616e7fc640 1 -- 172.21.15.84:0/1789901130 learned_addr learned my addr 172.21.15.84:0/1789901130 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:51.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.303+0000 7f616effd640 1 --2- 172.21.15.84:0/1789901130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61700addb0 0x7f61700caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:51.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.303+0000 7f616e7fc640 1 -- 172.21.15.84:0/1789901130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61700addb0 msgr2=0x7f61700caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:51.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.303+0000 7f616e7fc640 1 --2- 172.21.15.84:0/1789901130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61700addb0 0x7f61700caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:51.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.304+0000 7f616e7fc640 1 -- 172.21.15.84:0/1789901130 >> v1:172.21.15.157:6789/0 conn(0x7f617006a2e0 legacy=0x7f61700cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:51.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.304+0000 7f616e7fc640 1 -- 172.21.15.84:0/1789901130 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617006b030 con 0x7f61700c9000 2023-10-16T11:26:51.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.305+0000 7f616e7fc640 1 --2- 172.21.15.84:0/1789901130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61700c9000 0x7f6170069da0 secure :-1 s=READY pgs=3236 cs=0 l=1 rev1=1 crypto rx=0x7f61580098a0 tx=0x7f615802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbbd814abe04bcf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:51.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.305+0000 7f616dffb640 1 -- 172.21.15.84:0/1789901130 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158030840 con 0x7f61700c9000 2023-10-16T11:26:51.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.305+0000 7f616dffb640 1 -- 172.21.15.84:0/1789901130 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61580309a0 con 0x7f61700c9000 2023-10-16T11:26:51.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.305+0000 7f616dffb640 1 -- 172.21.15.84:0/1789901130 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158030d30 con 0x7f61700c9000 2023-10-16T11:26:51.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.306+0000 7f617593e640 1 -- 172.21.15.84:0/1789901130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61700c9000 msgr2=0x7f6170069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:51.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.306+0000 7f617593e640 1 --2- 172.21.15.84:0/1789901130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61700c9000 0x7f6170069da0 secure :-1 s=READY pgs=3236 cs=0 l=1 rev1=1 crypto rx=0x7f61580098a0 tx=0x7f615802b980 comp rx=0 tx=0).stop 2023-10-16T11:26:51.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.306+0000 7f617593e640 1 -- 172.21.15.84:0/1789901130 shutdown_connections 2023-10-16T11:26:51.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.306+0000 7f617593e640 1 --2- 172.21.15.84:0/1789901130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61700c9000 0x7f6170069da0 unknown :-1 s=CLOSED pgs=3236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:51.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.306+0000 7f617593e640 1 --2- 172.21.15.84:0/1789901130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61700addb0 0x7f61700caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:51.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.306+0000 7f617593e640 1 -- 172.21.15.84:0/1789901130 >> 172.21.15.84:0/1789901130 conn(0x7f61700ce9b0 msgr2=0x7f61700cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:51.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.306+0000 7f617593e640 1 -- 172.21.15.84:0/1789901130 shutdown_connections 2023-10-16T11:26:51.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.306+0000 7f617593e640 1 -- 172.21.15.84:0/1789901130 wait complete. 2023-10-16T11:26:51.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.308+0000 7f617593e640 1 Processor -- start 2023-10-16T11:26:51.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.308+0000 7f617593e640 1 -- start start 2023-10-16T11:26:51.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.309+0000 7f617593e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f617006a2e0 0x7f6170158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:51.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.309+0000 7f617593e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61700addb0 0x7f617015dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:51.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.309+0000 7f617593e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61700c9000 0x7f617015e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:51.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.309+0000 7f617593e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6170163690 con 0x7f61700c9000 2023-10-16T11:26:51.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.309+0000 7f617593e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61701594f0 con 0x7f617006a2e0 2023-10-16T11:26:51.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.309+0000 7f617593e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61700c6f10 con 0x7f61700addb0 2023-10-16T11:26:51.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.309+0000 7f616f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61700c9000 0x7f617015e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:51.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.309+0000 7f616f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61700c9000 0x7f617015e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51734/0 (socket says 172.21.15.84:51734) 2023-10-16T11:26:51.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.309+0000 7f616f7fe640 1 -- 172.21.15.84:0/1640866687 learned_addr learned my addr 172.21.15.84:0/1640866687 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:51.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.310+0000 7f616e7fc640 1 --2- 172.21.15.84:0/1640866687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61700addb0 0x7f617015dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:51.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.310+0000 7f616f7fe640 1 -- 172.21.15.84:0/1640866687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61700addb0 msgr2=0x7f617015dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.310+0000 7f616effd640 1 --2- 172.21.15.84:0/1640866687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f617006a2e0 0x7f6170158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.310+0000 7f616f7fe640 1 --2- 172.21.15.84:0/1640866687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61700addb0 0x7f617015dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.310+0000 7f616f7fe640 1 -- 172.21.15.84:0/1640866687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f617006a2e0 msgr2=0x7f6170158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.310+0000 7f616f7fe640 1 --2- 172.21.15.84:0/1640866687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f617006a2e0 0x7f6170158fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.310+0000 7f616f7fe640 1 -- 172.21.15.84:0/1640866687 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61580094d0 con 0x7f61700c9000 2023-10-16T11:26:51.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.310+0000 7f616f7fe640 1 --2- 172.21.15.84:0/1640866687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61700c9000 0x7f617015e2b0 secure :-1 s=READY pgs=3237 cs=0 l=1 rev1=1 crypto rx=0x7f6168001eb0 tx=0x7f616800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:51.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.310+0000 7f6167fff640 1 -- 172.21.15.84:0/1640866687 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61680052d0 con 0x7f61700c9000 2023-10-16T11:26:51.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.310+0000 7f6167fff640 1 -- 172.21.15.84:0/1640866687 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6168005430 con 0x7f61700c9000 2023-10-16T11:26:51.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.310+0000 7f6167fff640 1 -- 172.21.15.84:0/1640866687 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168003810 con 0x7f61700c9000 2023-10-16T11:26:51.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.310+0000 7f617593e640 1 -- 172.21.15.84:0/1640866687 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61700c71f0 con 0x7f61700c9000 2023-10-16T11:26:51.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.311+0000 7f617593e640 1 -- 172.21.15.84:0/1640866687 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61701632b0 con 0x7f61700c9000 2023-10-16T11:26:51.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.312+0000 7f617593e640 1 -- 172.21.15.84:0/1640866687 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f617004e160 con 0x7f61700c9000 2023-10-16T11:26:51.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.313+0000 7f6167fff640 1 -- 172.21.15.84:0/1640866687 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 151) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f6168003970 con 0x7f61700c9000 2023-10-16T11:26:51.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.314+0000 7f6167fff640 1 --2- 172.21.15.84:0/1640866687 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f615006b0b0 0x7f615006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:51.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.314+0000 7f6167fff640 1 -- 172.21.15.84:0/1640866687 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5780+0+0 (secure 0 0 0) 0x7f6168088f70 con 0x7f61700c9000 2023-10-16T11:26:51.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.314+0000 7f616effd640 1 --2- 172.21.15.84:0/1640866687 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f615006b0b0 0x7f615006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:51.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.315+0000 7f616effd640 1 --2- 172.21.15.84:0/1640866687 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f615006b0b0 0x7f615006d570 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f61701600a0 tx=0x7f616000b040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:51.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.319+0000 7f6167fff640 1 -- 172.21.15.84:0/1640866687 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6168053d30 con 0x7f61700c9000 2023-10-16T11:26:51.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.536+0000 7f617593e640 1 -- 172.21.15.84:0/1640866687 --> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f617006b5f0 con 0x7f615006b0b0 2023-10-16T11:26:51.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.539+0000 7f6167fff640 1 -- 172.21.15.84:0/1640866687 <== mgr.12624 v2:172.21.15.84:6832/2330196255 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6160016420 con 0x7f615006b0b0 2023-10-16T11:26:51.541 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T11:26:51.541 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:26:51.541 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephfs" 2023-10-16T11:26:51.541 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:26:51.542 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T11:26:51.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.541+0000 7f617593e640 1 -- 172.21.15.84:0/1640866687 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f615006b0b0 msgr2=0x7f615006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:51.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.541+0000 7f617593e640 1 --2- 172.21.15.84:0/1640866687 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f615006b0b0 0x7f615006d570 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f61701600a0 tx=0x7f616000b040 comp rx=0 tx=0).stop 2023-10-16T11:26:51.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.541+0000 7f617593e640 1 -- 172.21.15.84:0/1640866687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61700c9000 msgr2=0x7f617015e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:51.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.541+0000 7f617593e640 1 --2- 172.21.15.84:0/1640866687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61700c9000 0x7f617015e2b0 secure :-1 s=READY pgs=3237 cs=0 l=1 rev1=1 crypto rx=0x7f6168001eb0 tx=0x7f616800c890 comp rx=0 tx=0).stop 2023-10-16T11:26:51.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.541+0000 7f617593e640 1 -- 172.21.15.84:0/1640866687 shutdown_connections 2023-10-16T11:26:51.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.541+0000 7f617593e640 1 --2- 172.21.15.84:0/1640866687 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f615006b0b0 0x7f615006d570 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:51.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.541+0000 7f617593e640 1 --2- 172.21.15.84:0/1640866687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61700c9000 0x7f617015e2b0 unknown :-1 s=CLOSED pgs=3237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:51.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.541+0000 7f617593e640 1 --2- 172.21.15.84:0/1640866687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61700addb0 0x7f617015dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:51.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.541+0000 7f617593e640 1 --2- 172.21.15.84:0/1640866687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f617006a2e0 0x7f6170158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:51.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.542+0000 7f617593e640 1 -- 172.21.15.84:0/1640866687 >> 172.21.15.84:0/1640866687 conn(0x7f61700ce9b0 msgr2=0x7f61700ae240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:51.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.542+0000 7f617593e640 1 -- 172.21.15.84:0/1640866687 shutdown_connections 2023-10-16T11:26:51.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.542+0000 7f617593e640 1 -- 172.21.15.84:0/1640866687 wait complete. 2023-10-16T11:26:51.562 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T11:26:51.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.776+0000 7f92e34e6640 1 Processor -- start 2023-10-16T11:26:51.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.777+0000 7f92e34e6640 1 -- start start 2023-10-16T11:26:51.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.778+0000 7f92e34e6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f92dc15c9f0 0x7f92dc0c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:51.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.778+0000 7f92e34e6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92dc160a90 con 0x7f92dc0c9230 2023-10-16T11:26:51.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.778+0000 7f92e34e6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92dc0cd900 con 0x7f92dc15dde0 2023-10-16T11:26:51.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.778+0000 7f92e34e6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92dc0cdec0 con 0x7f92dc15c9f0 2023-10-16T11:26:51.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.779+0000 7f92e1ce3640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f92dc0c9230 0x7f92dc0709e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40968/0 (socket says 172.21.15.84:40968) 2023-10-16T11:26:51.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.779+0000 7f92e1ce3640 1 -- 172.21.15.84:0/1354417741 learned_addr learned my addr 172.21.15.84:0/1354417741 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:51.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.779+0000 7f92e14e2640 1 -- 172.21.15.84:0/1354417741 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407397397 0 0) 0x7f92dc160a90 con 0x7f92dc0c9230 2023-10-16T11:26:51.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.779+0000 7f92e14e2640 1 -- 172.21.15.84:0/1354417741 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92c8003520 con 0x7f92dc0c9230 2023-10-16T11:26:51.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.779+0000 7f92e24e4640 1 --2- 172.21.15.84:0/1354417741 >> v2:172.21.15.157:3300/0 conn(0x7f92dc15c9f0 0x7f92dc0c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:51.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.779+0000 7f92e14e2640 1 -- 172.21.15.84:0/1354417741 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4206454719 0 0) 0x7f92c8003520 con 0x7f92dc0c9230 2023-10-16T11:26:51.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.779+0000 7f92e14e2640 1 -- 172.21.15.84:0/1354417741 >> v2:172.21.15.157:3300/0 conn(0x7f92dc15c9f0 msgr2=0x7f92dc0c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:51.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.779+0000 7f92e14e2640 1 --2- 172.21.15.84:0/1354417741 >> v2:172.21.15.157:3300/0 conn(0x7f92dc15c9f0 0x7f92dc0c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:51.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.779+0000 7f92e14e2640 1 -- 172.21.15.84:0/1354417741 >> v1:172.21.15.157:6789/0 conn(0x7f92dc15dde0 legacy=0x7f92dc0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:51.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.779+0000 7f92e14e2640 1 -- 172.21.15.84:0/1354417741 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc0cdbe0 con 0x7f92dc0c9230 2023-10-16T11:26:51.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.779+0000 7f92e14e2640 1 -- 172.21.15.84:0/1354417741 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f92cc001670 con 0x7f92dc0c9230 2023-10-16T11:26:51.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.780+0000 7f92e14e2640 1 -- 172.21.15.84:0/1354417741 >> v1:172.21.15.84:6789/0 conn(0x7f92dc0c9230 legacy=0x7f92dc0709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:51.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.780+0000 7f92e14e2640 1 --2- 172.21.15.84:0/1354417741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92c80039c0 0x7f92c8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:51.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.780+0000 7f92e14e2640 1 --2- 172.21.15.84:0/1354417741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92c80047c0 0x7f92c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:51.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.780+0000 7f92e14e2640 1 --2- 172.21.15.84:0/1354417741 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92c80051d0 0x7f92c800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:51.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.780+0000 7f92e14e2640 1 -- 172.21.15.84:0/1354417741 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c800c0f0 con 0x7f92c80039c0 2023-10-16T11:26:51.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.780+0000 7f92e14e2640 1 -- 172.21.15.84:0/1354417741 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c800c260 con 0x7f92c80047c0 2023-10-16T11:26:51.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.780+0000 7f92e14e2640 1 -- 172.21.15.84:0/1354417741 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c800c420 con 0x7f92c80051d0 2023-10-16T11:26:51.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.780+0000 7f92e24e4640 1 --2- 172.21.15.84:0/1354417741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92c80039c0 0x7f92c8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:51.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.780+0000 7f92e24e4640 1 -- 172.21.15.84:0/1354417741 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92c80051d0 msgr2=0x7f92c800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:51.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.780+0000 7f92e24e4640 1 --2- 172.21.15.84:0/1354417741 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92c80051d0 0x7f92c800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:51.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.780+0000 7f92e24e4640 1 -- 172.21.15.84:0/1354417741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92c80047c0 msgr2=0x7f92c8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:51.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.780+0000 7f92e24e4640 1 --2- 172.21.15.84:0/1354417741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92c80047c0 0x7f92c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:51.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.780+0000 7f92e24e4640 1 -- 172.21.15.84:0/1354417741 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c8003520 con 0x7f92c80039c0 2023-10-16T11:26:51.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.780+0000 7f92e2ce5640 1 -- 172.21.15.84:0/1354417741 reap_dead start 2023-10-16T11:26:51.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.781+0000 7f92e24e4640 1 --2- 172.21.15.84:0/1354417741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92c80039c0 0x7f92c8003de0 secure :-1 s=READY pgs=3239 cs=0 l=1 rev1=1 crypto rx=0x7f92d800ba30 tx=0x7f92c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bed2e10f12c20e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:51.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.782+0000 7f92e14e2640 1 -- 172.21.15.84:0/1354417741 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d8032640 con 0x7f92c80039c0 2023-10-16T11:26:51.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.782+0000 7f92e14e2640 1 -- 172.21.15.84:0/1354417741 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d80327a0 con 0x7f92c80039c0 2023-10-16T11:26:51.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.782+0000 7f92e14e2640 1 -- 172.21.15.84:0/1354417741 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d8032b30 con 0x7f92c80039c0 2023-10-16T11:26:51.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.782+0000 7f92e34e6640 1 -- 172.21.15.84:0/1354417741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92c80039c0 msgr2=0x7f92c8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:51.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.782+0000 7f92e34e6640 1 --2- 172.21.15.84:0/1354417741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92c80039c0 0x7f92c8003de0 secure :-1 s=READY pgs=3239 cs=0 l=1 rev1=1 crypto rx=0x7f92d800ba30 tx=0x7f92c80057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:51.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.782+0000 7f92e34e6640 1 -- 172.21.15.84:0/1354417741 shutdown_connections 2023-10-16T11:26:51.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.782+0000 7f92e34e6640 1 --2- 172.21.15.84:0/1354417741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92c80039c0 0x7f92c8003de0 unknown :-1 s=CLOSED pgs=3239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:51.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.782+0000 7f92e34e6640 1 -- 172.21.15.84:0/1354417741 >> 172.21.15.84:0/1354417741 conn(0x7f92dc0ce6a0 msgr2=0x7f92dc0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:51.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.782+0000 7f92e34e6640 1 -- 172.21.15.84:0/1354417741 shutdown_connections 2023-10-16T11:26:51.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.783+0000 7f92e34e6640 1 -- 172.21.15.84:0/1354417741 wait complete. 2023-10-16T11:26:51.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.784+0000 7f92e34e6640 1 Processor -- start 2023-10-16T11:26:51.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.784+0000 7f92e34e6640 1 -- start start 2023-10-16T11:26:51.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.785+0000 7f92e34e6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92c80039c0 0x7f92dc14d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:51.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.785+0000 7f92e34e6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92dc069da0 0x7f92dc1f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:51.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.785+0000 7f92e34e6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92dc06a220 0x7f92dc1f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:51.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.785+0000 7f92e34e6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc1f6b60 con 0x7f92dc069da0 2023-10-16T11:26:51.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.785+0000 7f92e34e6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc14d9f0 con 0x7f92c80039c0 2023-10-16T11:26:51.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.785+0000 7f92e34e6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc1588b0 con 0x7f92dc06a220 2023-10-16T11:26:51.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.786+0000 7f92e1ce3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92dc069da0 0x7f92dc1f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:51.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.786+0000 7f92e1ce3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92dc069da0 0x7f92dc1f5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51742/0 (socket says 172.21.15.84:51742) 2023-10-16T11:26:51.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.786+0000 7f92e1ce3640 1 -- 172.21.15.84:0/4287450151 learned_addr learned my addr 172.21.15.84:0/4287450151 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:51.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.786+0000 7f92e24e4640 1 --2- 172.21.15.84:0/4287450151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92c80039c0 0x7f92dc14d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:51.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.786+0000 7f92e2ce5640 1 --2- 172.21.15.84:0/4287450151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92dc06a220 0x7f92dc1f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:51.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.786+0000 7f92e1ce3640 1 -- 172.21.15.84:0/4287450151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92dc06a220 msgr2=0x7f92dc1f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:51.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.786+0000 7f92e1ce3640 1 --2- 172.21.15.84:0/4287450151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92dc06a220 0x7f92dc1f6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:51.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.786+0000 7f92e1ce3640 1 -- 172.21.15.84:0/4287450151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92c80039c0 msgr2=0x7f92dc14d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:51.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.786+0000 7f92e1ce3640 1 --2- 172.21.15.84:0/4287450151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92c80039c0 0x7f92dc14d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:51.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.786+0000 7f92e1ce3640 1 -- 172.21.15.84:0/4287450151 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc158b90 con 0x7f92dc069da0 2023-10-16T11:26:51.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.786+0000 7f92e24e4640 1 --2- 172.21.15.84:0/4287450151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92c80039c0 0x7f92dc14d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:51.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.787+0000 7f92e1ce3640 1 --2- 172.21.15.84:0/4287450151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92dc069da0 0x7f92dc1f5d00 secure :-1 s=READY pgs=3240 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00c700 tx=0x7f92cc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:51.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.787+0000 7f92d2ffd640 1 -- 172.21.15.84:0/4287450151 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc0072d0 con 0x7f92dc069da0 2023-10-16T11:26:51.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.787+0000 7f92d2ffd640 1 -- 172.21.15.84:0/4287450151 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92cc007430 con 0x7f92dc069da0 2023-10-16T11:26:51.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.787+0000 7f92d2ffd640 1 -- 172.21.15.84:0/4287450151 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc0047e0 con 0x7f92dc069da0 2023-10-16T11:26:51.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.787+0000 7f92e34e6640 1 -- 172.21.15.84:0/4287450151 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92dc158e70 con 0x7f92dc069da0 2023-10-16T11:26:51.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.787+0000 7f92e34e6640 1 -- 172.21.15.84:0/4287450151 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92dc159430 con 0x7f92dc069da0 2023-10-16T11:26:51.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.788+0000 7f92e34e6640 1 -- 172.21.15.84:0/4287450151 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92dc04e9b0 con 0x7f92dc069da0 2023-10-16T11:26:51.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.789+0000 7f92d2ffd640 1 -- 172.21.15.84:0/4287450151 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 151) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f92cc004940 con 0x7f92dc069da0 2023-10-16T11:26:51.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.790+0000 7f92d2ffd640 1 --2- 172.21.15.84:0/4287450151 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f92c006b0e0 0x7f92c006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:51.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.790+0000 7f92d2ffd640 1 -- 172.21.15.84:0/4287450151 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5780+0+0 (secure 0 0 0) 0x7f92cc088f70 con 0x7f92dc069da0 2023-10-16T11:26:51.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.791+0000 7f92e24e4640 1 --2- 172.21.15.84:0/4287450151 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f92c006b0e0 0x7f92c006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:51.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.791+0000 7f92e24e4640 1 --2- 172.21.15.84:0/4287450151 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f92c006b0e0 0x7f92c006d5a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f92d800ba00 tx=0x7f92d803d040 comp rx=0 tx=0).ready entity=mgr.12624 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:51.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:51.795+0000 7f92d2ffd640 1 -- 172.21.15.84:0/4287450151 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f92cc053bd0 con 0x7f92dc069da0 2023-10-16T11:26:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:52.037+0000 7f92e34e6640 1 -- 172.21.15.84:0/4287450151 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f92dc0adac0 con 0x7f92dc069da0 2023-10-16T11:26:52.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:52.951+0000 7f92d2ffd640 1 -- 172.21.15.84:0/4287450151 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v152) v1 ==== 92+0+0 (secure 0 0 0) 0x7f92cc018020 con 0x7f92dc069da0 2023-10-16T11:26:52.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:52.955+0000 7f92e34e6640 1 -- 172.21.15.84:0/4287450151 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f92c006b0e0 msgr2=0x7f92c006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:52.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:52.955+0000 7f92e34e6640 1 --2- 172.21.15.84:0/4287450151 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f92c006b0e0 0x7f92c006d5a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f92d800ba00 tx=0x7f92d803d040 comp rx=0 tx=0).stop 2023-10-16T11:26:52.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:52.955+0000 7f92e34e6640 1 -- 172.21.15.84:0/4287450151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92dc069da0 msgr2=0x7f92dc1f5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:52.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:52.955+0000 7f92e34e6640 1 --2- 172.21.15.84:0/4287450151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92dc069da0 0x7f92dc1f5d00 secure :-1 s=READY pgs=3240 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00c700 tx=0x7f92cc00cbc0 comp rx=0 tx=0).stop 2023-10-16T11:26:52.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:52.955+0000 7f92e34e6640 1 -- 172.21.15.84:0/4287450151 shutdown_connections 2023-10-16T11:26:52.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:52.955+0000 7f92e34e6640 1 --2- 172.21.15.84:0/4287450151 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f92c006b0e0 0x7f92c006d5a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:52.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:52.955+0000 7f92e34e6640 1 --2- 172.21.15.84:0/4287450151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f92dc06a220 0x7f92dc1f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:52.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:52.955+0000 7f92e34e6640 1 --2- 172.21.15.84:0/4287450151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92dc069da0 0x7f92dc1f5d00 unknown :-1 s=CLOSED pgs=3240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:52.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:52.955+0000 7f92e34e6640 1 --2- 172.21.15.84:0/4287450151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92c80039c0 0x7f92dc14d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:52.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:52.955+0000 7f92e34e6640 1 -- 172.21.15.84:0/4287450151 >> 172.21.15.84:0/4287450151 conn(0x7f92dc0ce6a0 msgr2=0x7f92dc0c1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:52.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:52.956+0000 7f92e34e6640 1 -- 172.21.15.84:0/4287450151 shutdown_connections 2023-10-16T11:26:52.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:52.956+0000 7f92e34e6640 1 -- 172.21.15.84:0/4287450151 wait complete. 2023-10-16T11:26:52.981 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T11:26:53.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.171+0000 7fc766c3f640 1 Processor -- start 2023-10-16T11:26:53.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.171+0000 7fc766c3f640 1 -- start start 2023-10-16T11:26:53.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.172+0000 7fc766c3f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc76015aee0 0x7fc7600aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:53.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.172+0000 7fc766c3f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc7600cb810 0x7fc7600cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:53.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.172+0000 7fc766c3f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc760069da0 0x7fc76006a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:53.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.172+0000 7fc766c3f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc76006c140 con 0x7fc7600cb810 2023-10-16T11:26:53.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.172+0000 7fc766c3f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc76015b2c0 con 0x7fc76015aee0 2023-10-16T11:26:53.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.172+0000 7fc766c3f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76006b270 con 0x7fc760069da0 2023-10-16T11:26:53.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.173+0000 7fc75ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc7600cb810 0x7fc7600cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:53.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.173+0000 7fc75ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc7600cb810 0x7fc7600cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51768/0 (socket says 172.21.15.84:51768) 2023-10-16T11:26:53.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.173+0000 7fc75ffff640 1 -- 172.21.15.84:0/1238251916 learned_addr learned my addr 172.21.15.84:0/1238251916 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:53.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.173+0000 7fc7649db640 1 --2- 172.21.15.84:0/1238251916 >> v2:172.21.15.157:3300/0 conn(0x7fc76015aee0 0x7fc7600aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:53.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.173+0000 7fc7651dc640 1 --2- 172.21.15.84:0/1238251916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc760069da0 0x7fc76006a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:53.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.173+0000 7fc75ffff640 1 -- 172.21.15.84:0/1238251916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc760069da0 msgr2=0x7fc76006a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:53.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.173+0000 7fc75ffff640 1 --2- 172.21.15.84:0/1238251916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc760069da0 0x7fc76006a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.173+0000 7fc75ffff640 1 -- 172.21.15.84:0/1238251916 >> v2:172.21.15.157:3300/0 conn(0x7fc76015aee0 msgr2=0x7fc7600aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:53.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.173+0000 7fc75ffff640 1 --2- 172.21.15.84:0/1238251916 >> v2:172.21.15.157:3300/0 conn(0x7fc76015aee0 0x7fc7600aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.173+0000 7fc75ffff640 1 -- 172.21.15.84:0/1238251916 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76006daf0 con 0x7fc7600cb810 2023-10-16T11:26:53.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.175+0000 7fc75ffff640 1 --2- 172.21.15.84:0/1238251916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7600cb810 0x7fc7600cbc70 secure :-1 s=READY pgs=3244 cs=0 l=1 rev1=1 crypto rx=0x7fc7600cf8c0 tx=0x7fc74c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a042a36cece67470 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:53.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.176+0000 7fc75f7fe640 1 -- 172.21.15.84:0/1238251916 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0317e0 con 0x7fc7600cb810 2023-10-16T11:26:53.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.176+0000 7fc75f7fe640 1 -- 172.21.15.84:0/1238251916 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c031940 con 0x7fc7600cb810 2023-10-16T11:26:53.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.176+0000 7fc75f7fe640 1 -- 172.21.15.84:0/1238251916 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c031cd0 con 0x7fc7600cb810 2023-10-16T11:26:53.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.176+0000 7fc766c3f640 1 -- 172.21.15.84:0/1238251916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7600cb810 msgr2=0x7fc7600cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:53.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.176+0000 7fc766c3f640 1 --2- 172.21.15.84:0/1238251916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7600cb810 0x7fc7600cbc70 secure :-1 s=READY pgs=3244 cs=0 l=1 rev1=1 crypto rx=0x7fc7600cf8c0 tx=0x7fc74c02b880 comp rx=0 tx=0).stop 2023-10-16T11:26:53.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.176+0000 7fc766c3f640 1 -- 172.21.15.84:0/1238251916 shutdown_connections 2023-10-16T11:26:53.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.177+0000 7fc766c3f640 1 --2- 172.21.15.84:0/1238251916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7600cb810 0x7fc7600cbc70 unknown :-1 s=CLOSED pgs=3244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.177+0000 7fc766c3f640 1 --2- 172.21.15.84:0/1238251916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc760069da0 0x7fc76006a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.177+0000 7fc766c3f640 1 --2- 172.21.15.84:0/1238251916 >> v2:172.21.15.157:3300/0 conn(0x7fc76015aee0 0x7fc7600aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.177+0000 7fc766c3f640 1 -- 172.21.15.84:0/1238251916 >> 172.21.15.84:0/1238251916 conn(0x7fc7600cfac0 msgr2=0x7fc7600cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:53.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.177+0000 7fc766c3f640 1 -- 172.21.15.84:0/1238251916 shutdown_connections 2023-10-16T11:26:53.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.177+0000 7fc766c3f640 1 -- 172.21.15.84:0/1238251916 wait complete. 2023-10-16T11:26:53.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.178+0000 7fc766c3f640 1 Processor -- start 2023-10-16T11:26:53.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.179+0000 7fc766c3f640 1 -- start start 2023-10-16T11:26:53.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.179+0000 7fc766c3f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc760069da0 0x7fc76014e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:53.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.180+0000 7fc766c3f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7600cb810 0x7fc76014b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:53.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.180+0000 7fc766c3f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76015aee0 0x7fc7600b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:53.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.180+0000 7fc766c3f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76014ec30 con 0x7fc760069da0 2023-10-16T11:26:53.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.180+0000 7fc766c3f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76014dca0 con 0x7fc7600cb810 2023-10-16T11:26:53.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.180+0000 7fc766c3f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7600b4340 con 0x7fc76015aee0 2023-10-16T11:26:53.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.180+0000 7fc7649db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc760069da0 0x7fc76014e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:53.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.180+0000 7fc7649db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc760069da0 0x7fc76014e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51774/0 (socket says 172.21.15.84:51774) 2023-10-16T11:26:53.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.180+0000 7fc7649db640 1 -- 172.21.15.84:0/2415764556 learned_addr learned my addr 172.21.15.84:0/2415764556 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:53.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.180+0000 7fc7651dc640 1 --2- 172.21.15.84:0/2415764556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76015aee0 0x7fc7600b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:53.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.180+0000 7fc75ffff640 1 --2- 172.21.15.84:0/2415764556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7600cb810 0x7fc76014b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:53.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.180+0000 7fc7649db640 1 -- 172.21.15.84:0/2415764556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76015aee0 msgr2=0x7fc7600b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:53.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.180+0000 7fc7649db640 1 --2- 172.21.15.84:0/2415764556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76015aee0 0x7fc7600b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.180+0000 7fc7649db640 1 -- 172.21.15.84:0/2415764556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7600cb810 msgr2=0x7fc76014b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:53.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.180+0000 7fc7649db640 1 --2- 172.21.15.84:0/2415764556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7600cb810 0x7fc76014b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.180+0000 7fc7649db640 1 -- 172.21.15.84:0/2415764556 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74c0094d0 con 0x7fc760069da0 2023-10-16T11:26:53.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.180+0000 7fc75ffff640 1 --2- 172.21.15.84:0/2415764556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7600cb810 0x7fc76014b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:53.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.180+0000 7fc7651dc640 1 --2- 172.21.15.84:0/2415764556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76015aee0 0x7fc7600b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:53.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.181+0000 7fc7649db640 1 --2- 172.21.15.84:0/2415764556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc760069da0 0x7fc76014e6f0 secure :-1 s=READY pgs=3245 cs=0 l=1 rev1=1 crypto rx=0x7fc75400d800 tx=0x7fc75400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:53.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.181+0000 7fc75d7fa640 1 -- 172.21.15.84:0/2415764556 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75400b6c0 con 0x7fc760069da0 2023-10-16T11:26:53.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.181+0000 7fc75d7fa640 1 -- 172.21.15.84:0/2415764556 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75400b820 con 0x7fc760069da0 2023-10-16T11:26:53.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.181+0000 7fc75d7fa640 1 -- 172.21.15.84:0/2415764556 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75400bbb0 con 0x7fc760069da0 2023-10-16T11:26:53.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.181+0000 7fc766c3f640 1 -- 172.21.15.84:0/2415764556 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7600b4620 con 0x7fc760069da0 2023-10-16T11:26:53.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.181+0000 7fc766c3f640 1 -- 172.21.15.84:0/2415764556 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76014e1d0 con 0x7fc760069da0 2023-10-16T11:26:53.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.182+0000 7fc766c3f640 1 -- 172.21.15.84:0/2415764556 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76015ce30 con 0x7fc760069da0 2023-10-16T11:26:53.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.183+0000 7fc75d7fa640 1 -- 172.21.15.84:0/2415764556 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 152) v1 ==== 89450+0+0 (secure 0 0 0) 0x7fc75400bd10 con 0x7fc760069da0 2023-10-16T11:26:53.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.184+0000 7fc75d7fa640 1 --2- 172.21.15.84:0/2415764556 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc74406b100 0x7fc74406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:53.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.184+0000 7fc75d7fa640 1 -- 172.21.15.84:0/2415764556 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5780+0+0 (secure 0 0 0) 0x7fc75408a0a0 con 0x7fc760069da0 2023-10-16T11:26:53.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.184+0000 7fc75ffff640 1 -- 172.21.15.84:0/2415764556 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc74406b100 msgr2=0x7fc74406d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2330196255 2023-10-16T11:26:53.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.184+0000 7fc75ffff640 1 --2- 172.21.15.84:0/2415764556 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc74406b100 0x7fc74406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:26:53.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.193+0000 7fc75d7fa640 1 -- 172.21.15.84:0/2415764556 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc754054d90 con 0x7fc760069da0 2023-10-16T11:26:53.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.385+0000 7fc75ffff640 1 -- 172.21.15.84:0/2415764556 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc74406b100 msgr2=0x7fc74406d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2330196255 2023-10-16T11:26:53.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.385+0000 7fc75ffff640 1 --2- 172.21.15.84:0/2415764556 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc74406b100 0x7fc74406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:26:53.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.386+0000 7fc766c3f640 1 -- 172.21.15.84:0/2415764556 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fc76004e160 con 0x7fc760069da0 2023-10-16T11:26:53.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.390+0000 7fc75d7fa640 1 -- 172.21.15.84:0/2415764556 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v89)=0 v89) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc7540583b0 con 0x7fc760069da0 2023-10-16T11:26:53.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.393+0000 7fc766c3f640 1 -- 172.21.15.84:0/2415764556 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc74406b100 msgr2=0x7fc74406d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:26:53.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.393+0000 7fc766c3f640 1 --2- 172.21.15.84:0/2415764556 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc74406b100 0x7fc74406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.393+0000 7fc766c3f640 1 -- 172.21.15.84:0/2415764556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc760069da0 msgr2=0x7fc76014e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:53.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.393+0000 7fc766c3f640 1 --2- 172.21.15.84:0/2415764556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc760069da0 0x7fc76014e6f0 secure :-1 s=READY pgs=3245 cs=0 l=1 rev1=1 crypto rx=0x7fc75400d800 tx=0x7fc75400dcc0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.393+0000 7fc766c3f640 1 -- 172.21.15.84:0/2415764556 shutdown_connections 2023-10-16T11:26:53.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.393+0000 7fc766c3f640 1 --2- 172.21.15.84:0/2415764556 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fc74406b100 0x7fc74406d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.393+0000 7fc766c3f640 1 --2- 172.21.15.84:0/2415764556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76015aee0 0x7fc7600b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.393+0000 7fc766c3f640 1 --2- 172.21.15.84:0/2415764556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7600cb810 0x7fc76014b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.393+0000 7fc766c3f640 1 --2- 172.21.15.84:0/2415764556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc760069da0 0x7fc76014e6f0 unknown :-1 s=CLOSED pgs=3245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.393+0000 7fc766c3f640 1 -- 172.21.15.84:0/2415764556 >> 172.21.15.84:0/2415764556 conn(0x7fc7600cfac0 msgr2=0x7fc7600b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:53.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.393+0000 7fc766c3f640 1 -- 172.21.15.84:0/2415764556 shutdown_connections 2023-10-16T11:26:53.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.393+0000 7fc766c3f640 1 -- 172.21.15.84:0/2415764556 wait complete. 2023-10-16T11:26:53.413 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T11:26:53.474 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:53.472+0000 7f48aa42b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:26:53.503 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:53.502+0000 7fec13614040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:26:53.528 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:53.526+0000 7f48aa42b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:26:53.560 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:53.558+0000 7fec13614040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:26:53.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.617+0000 7f51aa640640 1 Processor -- start 2023-10-16T11:26:53.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.617+0000 7f51aa640640 1 -- start start 2023-10-16T11:26:53.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.618+0000 7f51aa640640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51a415cd70 0x7f51a40d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:53.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.618+0000 7f51aa640640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f51a40b8480 0x7f51a40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:53.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.618+0000 7f51aa640640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a415d150 con 0x7f51a40d0be0 2023-10-16T11:26:53.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.618+0000 7f51aa640640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a40d1180 con 0x7f51a40b8480 2023-10-16T11:26:53.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.618+0000 7f51aa640640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a40d1460 con 0x7f51a415cd70 2023-10-16T11:26:53.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.619+0000 7f51a8e3d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f51a40b8480 0x7f51a40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:53.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.619+0000 7f51a8e3d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f51a40b8480 0x7f51a40cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51782/0 (socket says 172.21.15.84:51782) 2023-10-16T11:26:53.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.619+0000 7f51a8e3d640 1 -- 172.21.15.84:0/3416312244 learned_addr learned my addr 172.21.15.84:0/3416312244 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:53.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.619+0000 7f51a963e640 1 --2- 172.21.15.84:0/3416312244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51a415cd70 0x7f51a40d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:53.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.619+0000 7f51a8e3d640 1 -- 172.21.15.84:0/3416312244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51a415cd70 msgr2=0x7f51a40d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:53.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.619+0000 7f51a8e3d640 1 --2- 172.21.15.84:0/3416312244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51a415cd70 0x7f51a40d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.619+0000 7f51a8e3d640 1 -- 172.21.15.84:0/3416312244 >> v1:172.21.15.157:6789/0 conn(0x7f51a40d0be0 legacy=0x7f51a40b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:53.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.619+0000 7f51a8e3d640 1 -- 172.21.15.84:0/3416312244 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a40d1740 con 0x7f51a40b8480 2023-10-16T11:26:53.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.620+0000 7f51a8e3d640 1 --2- 172.21.15.84:0/3416312244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51a40b8480 0x7f51a40cd7c0 secure :-1 s=READY pgs=3246 cs=0 l=1 rev1=1 crypto rx=0x7f5194009870 tx=0x7f519402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b66987a0bd294a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:53.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.620+0000 7f519bfff640 1 -- 172.21.15.84:0/3416312244 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194030840 con 0x7f51a40b8480 2023-10-16T11:26:53.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.621+0000 7f519bfff640 1 -- 172.21.15.84:0/3416312244 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51940309a0 con 0x7f51a40b8480 2023-10-16T11:26:53.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.621+0000 7f519bfff640 1 -- 172.21.15.84:0/3416312244 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194030d30 con 0x7f51a40b8480 2023-10-16T11:26:53.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.621+0000 7f51aa640640 1 -- 172.21.15.84:0/3416312244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51a40b8480 msgr2=0x7f51a40cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:53.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.621+0000 7f51aa640640 1 --2- 172.21.15.84:0/3416312244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51a40b8480 0x7f51a40cd7c0 secure :-1 s=READY pgs=3246 cs=0 l=1 rev1=1 crypto rx=0x7f5194009870 tx=0x7f519402b980 comp rx=0 tx=0).stop 2023-10-16T11:26:53.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.621+0000 7f51aa640640 1 -- 172.21.15.84:0/3416312244 shutdown_connections 2023-10-16T11:26:53.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.621+0000 7f51aa640640 1 --2- 172.21.15.84:0/3416312244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51a40b8480 0x7f51a40cd7c0 unknown :-1 s=CLOSED pgs=3246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.621+0000 7f51aa640640 1 --2- 172.21.15.84:0/3416312244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51a415cd70 0x7f51a40d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.621+0000 7f51aa640640 1 -- 172.21.15.84:0/3416312244 >> 172.21.15.84:0/3416312244 conn(0x7f51a40cf570 msgr2=0x7f51a40cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:53.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.621+0000 7f51aa640640 1 -- 172.21.15.84:0/3416312244 shutdown_connections 2023-10-16T11:26:53.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.621+0000 7f51aa640640 1 -- 172.21.15.84:0/3416312244 wait complete. 2023-10-16T11:26:53.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.624+0000 7f51aa640640 1 Processor -- start 2023-10-16T11:26:53.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.624+0000 7f51aa640640 1 -- start start 2023-10-16T11:26:53.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.626+0000 7f51aa640640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51a40b8480 0x7f51a40c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:53.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.626+0000 7f51aa640640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51a40d0be0 0x7f51a41548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:53.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.626+0000 7f51aa640640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51a415cd70 0x7f51a4147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:53.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.626+0000 7f51aa640640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a40cc340 con 0x7f51a415cd70 2023-10-16T11:26:53.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.626+0000 7f51aa640640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a40cb6c0 con 0x7f51a40d0be0 2023-10-16T11:26:53.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.626+0000 7f51aa640640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a40cb9a0 con 0x7f51a40b8480 2023-10-16T11:26:53.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.626+0000 7f51a9e3f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51a415cd70 0x7f51a4147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:53.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.626+0000 7f51a9e3f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51a415cd70 0x7f51a4147d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51796/0 (socket says 172.21.15.84:51796) 2023-10-16T11:26:53.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.626+0000 7f51a9e3f640 1 -- 172.21.15.84:0/1196620227 learned_addr learned my addr 172.21.15.84:0/1196620227 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:53.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.626+0000 7f51a8e3d640 1 --2- 172.21.15.84:0/1196620227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51a40d0be0 0x7f51a41548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:53.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.626+0000 7f51a963e640 1 --2- 172.21.15.84:0/1196620227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51a40b8480 0x7f51a40c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:53.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.627+0000 7f51a9e3f640 1 -- 172.21.15.84:0/1196620227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51a40b8480 msgr2=0x7f51a40c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:53.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.627+0000 7f51a9e3f640 1 --2- 172.21.15.84:0/1196620227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51a40b8480 0x7f51a40c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.627+0000 7f51a9e3f640 1 -- 172.21.15.84:0/1196620227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51a40d0be0 msgr2=0x7f51a41548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:53.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.627+0000 7f51a9e3f640 1 --2- 172.21.15.84:0/1196620227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51a40d0be0 0x7f51a41548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.627+0000 7f51a9e3f640 1 -- 172.21.15.84:0/1196620227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51940094d0 con 0x7f51a415cd70 2023-10-16T11:26:53.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.627+0000 7f51a9e3f640 1 --2- 172.21.15.84:0/1196620227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51a415cd70 0x7f51a4147d40 secure :-1 s=READY pgs=3247 cs=0 l=1 rev1=1 crypto rx=0x7f519c00c730 tx=0x7f519c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:53.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.627+0000 7f5199ffb640 1 -- 172.21.15.84:0/1196620227 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c003b70 con 0x7f51a415cd70 2023-10-16T11:26:53.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.628+0000 7f5199ffb640 1 -- 172.21.15.84:0/1196620227 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519c003cf0 con 0x7f51a415cd70 2023-10-16T11:26:53.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.628+0000 7f51aa640640 1 -- 172.21.15.84:0/1196620227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a40cbc80 con 0x7f51a415cd70 2023-10-16T11:26:53.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.628+0000 7f5199ffb640 1 -- 172.21.15.84:0/1196620227 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c005310 con 0x7f51a415cd70 2023-10-16T11:26:53.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.628+0000 7f51aa640640 1 -- 172.21.15.84:0/1196620227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a4155480 con 0x7f51a415cd70 2023-10-16T11:26:53.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.629+0000 7f51aa640640 1 -- 172.21.15.84:0/1196620227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a40cc480 con 0x7f51a415cd70 2023-10-16T11:26:53.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.630+0000 7f5199ffb640 1 -- 172.21.15.84:0/1196620227 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 152) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f519c005680 con 0x7f51a415cd70 2023-10-16T11:26:53.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.631+0000 7f5199ffb640 1 --2- 172.21.15.84:0/1196620227 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f518806b100 0x7f518806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:53.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.631+0000 7f51a963e640 1 -- 172.21.15.84:0/1196620227 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f518806b100 msgr2=0x7f518806d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2330196255 2023-10-16T11:26:53.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.631+0000 7f5199ffb640 1 -- 172.21.15.84:0/1196620227 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5780+0+0 (secure 0 0 0) 0x7f519c089b20 con 0x7f51a415cd70 2023-10-16T11:26:53.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.631+0000 7f51a963e640 1 --2- 172.21.15.84:0/1196620227 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f518806b100 0x7f518806d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:26:53.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.635+0000 7f5199ffb640 1 -- 172.21.15.84:0/1196620227 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f51a40cc480 con 0x7f51a415cd70 2023-10-16T11:26:53.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.824+0000 7f51aa640640 1 -- 172.21.15.84:0/1196620227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f51a40cc480 con 0x7f51a415cd70 2023-10-16T11:26:53.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.830+0000 7f5199ffb640 1 -- 172.21.15.84:0/1196620227 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v90)=0 v90) v1 ==== 133+0+0 (secure 0 0 0) 0x7f51a40cc480 con 0x7f51a415cd70 2023-10-16T11:26:53.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.832+0000 7f51a963e640 1 -- 172.21.15.84:0/1196620227 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f518806b100 msgr2=0x7f518806d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2330196255 2023-10-16T11:26:53.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.832+0000 7f51a963e640 1 --2- 172.21.15.84:0/1196620227 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f518806b100 0x7f518806d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:26:53.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.832+0000 7f51aa640640 1 -- 172.21.15.84:0/1196620227 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f518806b100 msgr2=0x7f518806d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:26:53.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.832+0000 7f51aa640640 1 --2- 172.21.15.84:0/1196620227 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f518806b100 0x7f518806d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.832+0000 7f51aa640640 1 -- 172.21.15.84:0/1196620227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51a415cd70 msgr2=0x7f51a4147d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:53.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.832+0000 7f51aa640640 1 --2- 172.21.15.84:0/1196620227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51a415cd70 0x7f51a4147d40 secure :-1 s=READY pgs=3247 cs=0 l=1 rev1=1 crypto rx=0x7f519c00c730 tx=0x7f519c00cbf0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.832+0000 7f51aa640640 1 -- 172.21.15.84:0/1196620227 shutdown_connections 2023-10-16T11:26:53.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.832+0000 7f51aa640640 1 --2- 172.21.15.84:0/1196620227 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7f518806b100 0x7f518806d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.832+0000 7f51aa640640 1 --2- 172.21.15.84:0/1196620227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51a415cd70 0x7f51a4147d40 unknown :-1 s=CLOSED pgs=3247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.832+0000 7f51aa640640 1 --2- 172.21.15.84:0/1196620227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51a40d0be0 0x7f51a41548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.832+0000 7f51aa640640 1 --2- 172.21.15.84:0/1196620227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51a40b8480 0x7f51a40c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:53.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.832+0000 7f51aa640640 1 -- 172.21.15.84:0/1196620227 >> 172.21.15.84:0/1196620227 conn(0x7f51a40cf570 msgr2=0x7f51a406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:53.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.833+0000 7f51aa640640 1 -- 172.21.15.84:0/1196620227 shutdown_connections 2023-10-16T11:26:53.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:53.833+0000 7f51aa640640 1 -- 172.21.15.84:0/1196620227 wait complete. 2023-10-16T11:26:53.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:26:53.854 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-16T11:26:53.911 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-16T11:26:53.981 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:53.979+0000 7f48aa42b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:26:54.034 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:54.032+0000 7f48aa42b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:26:54.045 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:54.043+0000 7fec13614040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:26:54.084 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:54.082+0000 7f48aa42b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:26:54.102 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:54.100+0000 7fec13614040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:26:54.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.111+0000 7fd717e44640 1 Processor -- start 2023-10-16T11:26:54.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.111+0000 7fd717e44640 1 -- start start 2023-10-16T11:26:54.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.113+0000 7fd717e44640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd710070830 0x7fd71006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:54.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.113+0000 7fd717e44640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd7100cf5f0 0x7fd71006c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:54.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.113+0000 7fd717e44640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd710070c10 0x7fd7100cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:54.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.113+0000 7fd717e44640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd710161740 con 0x7fd7100cf5f0 2023-10-16T11:26:54.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.113+0000 7fd717e44640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71006c680 con 0x7fd710070830 2023-10-16T11:26:54.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.113+0000 7fd717e44640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100cdbd0 con 0x7fd710070c10 2023-10-16T11:26:54.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.113+0000 7fd7153df640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd7100cf5f0 0x7fd71006c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:54.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.113+0000 7fd7153df640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd7100cf5f0 0x7fd71006c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51804/0 (socket says 172.21.15.84:51804) 2023-10-16T11:26:54.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.113+0000 7fd7153df640 1 -- 172.21.15.84:0/3934345046 learned_addr learned my addr 172.21.15.84:0/3934345046 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:54.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.113+0000 7fd715be0640 1 --2- 172.21.15.84:0/3934345046 >> v2:172.21.15.157:3300/0 conn(0x7fd710070830 0x7fd71006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:54.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.113+0000 7fd7163e1640 1 --2- 172.21.15.84:0/3934345046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd710070c10 0x7fd7100cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:54.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.114+0000 7fd7153df640 1 -- 172.21.15.84:0/3934345046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd710070c10 msgr2=0x7fd7100cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:54.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.114+0000 7fd7153df640 1 --2- 172.21.15.84:0/3934345046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd710070c10 0x7fd7100cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:54.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.114+0000 7fd7153df640 1 -- 172.21.15.84:0/3934345046 >> v2:172.21.15.157:3300/0 conn(0x7fd710070830 msgr2=0x7fd71006d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:54.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.114+0000 7fd7153df640 1 --2- 172.21.15.84:0/3934345046 >> v2:172.21.15.157:3300/0 conn(0x7fd710070830 0x7fd71006d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:54.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.114+0000 7fd7153df640 1 -- 172.21.15.84:0/3934345046 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71006b0f0 con 0x7fd7100cf5f0 2023-10-16T11:26:54.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.115+0000 7fd7153df640 1 --2- 172.21.15.84:0/3934345046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7100cf5f0 0x7fd71006c140 secure :-1 s=READY pgs=3248 cs=0 l=1 rev1=1 crypto rx=0x7fd71006c010 tx=0x7fd70002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a866238ef1cf2ee server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:54.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.115+0000 7fd714bde640 1 -- 172.21.15.84:0/3934345046 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700031750 con 0x7fd7100cf5f0 2023-10-16T11:26:54.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.115+0000 7fd714bde640 1 -- 172.21.15.84:0/3934345046 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7000318b0 con 0x7fd7100cf5f0 2023-10-16T11:26:54.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.115+0000 7fd714bde640 1 -- 172.21.15.84:0/3934345046 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700031c40 con 0x7fd7100cf5f0 2023-10-16T11:26:54.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.116+0000 7fd717e44640 1 -- 172.21.15.84:0/3934345046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7100cf5f0 msgr2=0x7fd71006c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:54.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.116+0000 7fd717e44640 1 --2- 172.21.15.84:0/3934345046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7100cf5f0 0x7fd71006c140 secure :-1 s=READY pgs=3248 cs=0 l=1 rev1=1 crypto rx=0x7fd71006c010 tx=0x7fd70002b7f0 comp rx=0 tx=0).stop 2023-10-16T11:26:54.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.116+0000 7fd717e44640 1 -- 172.21.15.84:0/3934345046 shutdown_connections 2023-10-16T11:26:54.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.116+0000 7fd717e44640 1 --2- 172.21.15.84:0/3934345046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7100cf5f0 0x7fd71006c140 unknown :-1 s=CLOSED pgs=3248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:54.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.116+0000 7fd717e44640 1 --2- 172.21.15.84:0/3934345046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd710070c10 0x7fd7100cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:54.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.116+0000 7fd717e44640 1 --2- 172.21.15.84:0/3934345046 >> v2:172.21.15.157:3300/0 conn(0x7fd710070830 0x7fd71006d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:54.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.116+0000 7fd717e44640 1 -- 172.21.15.84:0/3934345046 >> 172.21.15.84:0/3934345046 conn(0x7fd7100cfac0 msgr2=0x7fd7100cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:54.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.116+0000 7fd717e44640 1 -- 172.21.15.84:0/3934345046 shutdown_connections 2023-10-16T11:26:54.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.116+0000 7fd717e44640 1 -- 172.21.15.84:0/3934345046 wait complete. 2023-10-16T11:26:54.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.118+0000 7fd717e44640 1 Processor -- start 2023-10-16T11:26:54.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.118+0000 7fd717e44640 1 -- start start 2023-10-16T11:26:54.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.119+0000 7fd717e44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd710070830 0x7fd71014ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:54.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.119+0000 7fd717e44640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd710070c10 0x7fd71014aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:54.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.119+0000 7fd717e44640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7100cf5f0 0x7fd7100c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:54.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.119+0000 7fd717e44640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71014d340 con 0x7fd710070830 2023-10-16T11:26:54.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.119+0000 7fd717e44640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71014b3e0 con 0x7fd710070c10 2023-10-16T11:26:54.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.119+0000 7fd717e44640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd710147b50 con 0x7fd7100cf5f0 2023-10-16T11:26:54.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.119+0000 7fd715be0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd710070830 0x7fd71014ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:54.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.119+0000 7fd715be0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd710070830 0x7fd71014ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51808/0 (socket says 172.21.15.84:51808) 2023-10-16T11:26:54.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.119+0000 7fd715be0640 1 -- 172.21.15.84:0/42042727 learned_addr learned my addr 172.21.15.84:0/42042727 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:54.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.119+0000 7fd7153df640 1 --2- 172.21.15.84:0/42042727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd710070c10 0x7fd71014aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:54.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.119+0000 7fd7163e1640 1 --2- 172.21.15.84:0/42042727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7100cf5f0 0x7fd7100c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:54.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.119+0000 7fd715be0640 1 -- 172.21.15.84:0/42042727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7100cf5f0 msgr2=0x7fd7100c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:54.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.119+0000 7fd715be0640 1 --2- 172.21.15.84:0/42042727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7100cf5f0 0x7fd7100c85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:54.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.119+0000 7fd715be0640 1 -- 172.21.15.84:0/42042727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd710070c10 msgr2=0x7fd71014aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:54.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.119+0000 7fd715be0640 1 --2- 172.21.15.84:0/42042727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd710070c10 0x7fd71014aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:54.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.119+0000 7fd715be0640 1 -- 172.21.15.84:0/42042727 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7000094d0 con 0x7fd710070830 2023-10-16T11:26:54.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.119+0000 7fd7153df640 1 --2- 172.21.15.84:0/42042727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd710070c10 0x7fd71014aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:26:54.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.120+0000 7fd715be0640 1 --2- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd710070830 0x7fd71014ce00 secure :-1 s=READY pgs=3249 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00e870 tx=0x7fd70c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:54.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.120+0000 7fd7067fc640 1 -- 172.21.15.84:0/42042727 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c00ee60 con 0x7fd710070830 2023-10-16T11:26:54.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.120+0000 7fd7067fc640 1 -- 172.21.15.84:0/42042727 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70c00c940 con 0x7fd710070830 2023-10-16T11:26:54.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.120+0000 7fd7067fc640 1 -- 172.21.15.84:0/42042727 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c00ccf0 con 0x7fd710070830 2023-10-16T11:26:54.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.120+0000 7fd717e44640 1 -- 172.21.15.84:0/42042727 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd710147e30 con 0x7fd710070830 2023-10-16T11:26:54.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.120+0000 7fd717e44640 1 -- 172.21.15.84:0/42042727 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7101483f0 con 0x7fd710070830 2023-10-16T11:26:54.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.121+0000 7fd717e44640 1 -- 172.21.15.84:0/42042727 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71004e160 con 0x7fd710070830 2023-10-16T11:26:54.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.123+0000 7fd7067fc640 1 -- 172.21.15.84:0/42042727 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 152) v1 ==== 89450+0+0 (secure 0 0 0) 0x7fd70c013470 con 0x7fd710070830 2023-10-16T11:26:54.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.124+0000 7fd7067fc640 1 --2- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6f406b130 0x7fd6f406d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:54.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.124+0000 7fd7067fc640 1 -- 172.21.15.84:0/42042727 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5780+0+0 (secure 0 0 0) 0x7fd70c089a90 con 0x7fd710070830 2023-10-16T11:26:54.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.124+0000 7fd7153df640 1 -- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6f406b130 msgr2=0x7fd6f406d5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2330196255 2023-10-16T11:26:54.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.124+0000 7fd7153df640 1 --2- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6f406b130 0x7fd6f406d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:26:54.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.128+0000 7fd7067fc640 1 -- 172.21.15.84:0/42042727 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd70c054820 con 0x7fd710070830 2023-10-16T11:26:54.147 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:54.145+0000 7f48aa42b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:26:54.158 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:54.156+0000 7fec13614040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:26:54.196 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:54.194+0000 7f48aa42b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:26:54.221 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:54.219+0000 7fec13614040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:26:54.273 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:54.271+0000 7fec13614040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:26:54.308 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:54.306+0000 7f48aa42b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:26:54.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.325+0000 7fd7153df640 1 -- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6f406b130 msgr2=0x7fd6f406d5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2330196255 2023-10-16T11:26:54.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.325+0000 7fd7153df640 1 --2- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6f406b130 0x7fd6f406d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:26:54.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.342+0000 7fd717e44640 1 -- 172.21.15.84:0/42042727 --> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7101619b0 con 0x7fd6f406b130 2023-10-16T11:26:54.370 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:54.368+0000 7f48aa42b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:26:54.387 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:54.386+0000 7fec13614040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:26:54.419 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:54.417+0000 7f48aa42b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:26:54.452 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:54.450+0000 7fec13614040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:26:54.477 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:54.475+0000 7f48aa42b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:26:54.502 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:54.500+0000 7fec13614040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:26:54.528 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:54.526+0000 7f48aa42b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:26:54.558 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:54.556+0000 7fec13614040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:26:54.609 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:54.607+0000 7fec13614040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:26:54.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.726+0000 7fd7153df640 1 -- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6f406b130 msgr2=0x7fd6f406d5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2330196255 2023-10-16T11:26:54.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:54.726+0000 7fd7153df640 1 --2- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6f406b130 0x7fd6f406d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T11:26:54.908 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:54.906+0000 7f48aa42b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:26:54.993 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:54.991+0000 7fec13614040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:26:55.066 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:55.064+0000 7f48aa42b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:26:55.152 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:55.150+0000 7fec13614040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:26:55.174 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:55.172+0000 7f48aa42b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:26:55.224 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:55.222+0000 7f48aa42b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:26:55.260 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:55.257+0000 7fec13614040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:26:55.310 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:55.308+0000 7fec13614040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:26:55.330 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:55.329+0000 7f48aa42b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:26:55.417 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:55.415+0000 7fec13614040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:26:55.437 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:55.435+0000 7f48aa42b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:26:55.526 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:55.524+0000 7fec13614040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:26:55.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:55.527+0000 7fd7153df640 1 -- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6f406b130 msgr2=0x7fd6f406d5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2330196255 2023-10-16T11:26:55.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:55.527+0000 7fd7153df640 1 --2- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6f406b130 0x7fd6f406d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T11:26:55.594 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:55.592+0000 7f48aa42b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:26:55.643 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:55.641+0000 7f48aa42b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:26:55.684 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:55.682+0000 7fec13614040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:26:55.735 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:55.733+0000 7fec13614040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:26:55.762 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:55.760+0000 7f48aa42b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:26:55.855 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:55.852+0000 7fec13614040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:26:56.610 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:56.608+0000 7f48aa42b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:26:56.684 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:56.682+0000 7fec13614040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:26:56.930 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:56.928+0000 7f48aa42b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:26:56.978 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:56.975+0000 7fec13614040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:26:56.987 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:56.985+0000 7f48aa42b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:26:57.034 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:57.032+0000 7fec13614040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:26:57.100 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:26:57.098+0000 7f48aa42b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:26:57.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:57.129+0000 7fd7153df640 1 -- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6f406b130 msgr2=0x7fd6f406d5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2330196255 2023-10-16T11:26:57.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:57.129+0000 7fd7153df640 1 --2- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6f406b130 0x7fd6f406d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T11:26:57.149 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:57.147+0000 7fec13614040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:26:57.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:57.164+0000 7fd7067fc640 1 -- 172.21.15.84:0/42042727 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 153) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fd70c054bf0 con 0x7fd710070830 2023-10-16T11:26:57.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:57.165+0000 7fd7067fc640 1 -- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6f406b130 msgr2=0x7fd6f406d5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:26:57.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:57.165+0000 7fd7067fc640 1 --2- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:6832/2330196255,v1:172.21.15.84:6833/2330196255] conn(0x7fd6f406b130 0x7fd6f406d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:57.514 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:57.512+0000 7feb8fe1e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:57.514 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:57.512+0000 7feb8fe1e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:57.515 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:57.512+0000 7feb8fe1e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:57.515 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:57.512+0000 7feb8fe1e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:57.515 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:57.512+0000 7feb8fe1e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:57.515 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:57.513+0000 7feb91e62640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:57.516 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:57.513+0000 7feb91e62640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:57.516 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:57.513+0000 7feb91e62640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:57.516 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:57.513+0000 7feb91e62640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:57.516 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:26:57.513+0000 7feb91e62640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:26:58.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.192+0000 7fd7067fc640 1 -- 172.21.15.84:0/42042727 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 154) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fd70c056290 con 0x7fd710070830 2023-10-16T11:26:58.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.193+0000 7fd7067fc640 1 --2- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7fd6f4035830 0x7fd6f4075110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:58.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.194+0000 7fd7067fc640 1 -- 172.21.15.84:0/42042727 --> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6f40283e0 con 0x7fd6f4035830 2023-10-16T11:26:58.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.196+0000 7fd7153df640 1 --2- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7fd6f4035830 0x7fd6f4075110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:58.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.203+0000 7fd7153df640 1 --2- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7fd6f4035830 0x7fd6f4075110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd700035000 tx=0x7fd700008030 comp rx=0 tx=0).ready entity=mgr.13143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:58.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.407+0000 7fd7067fc640 1 -- 172.21.15.84:0/42042727 <== mgr.13143 v2:172.21.15.84:6832/2988856492 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd6f40283e0 con 0x7fd6f4035830 2023-10-16T11:26:58.409 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path snap_test_dir1 2023-10-16T11:26:58.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.410+0000 7fd717e44640 1 -- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7fd6f4035830 msgr2=0x7fd6f4075110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:58.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.410+0000 7fd717e44640 1 --2- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7fd6f4035830 0x7fd6f4075110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd700035000 tx=0x7fd700008030 comp rx=0 tx=0).stop 2023-10-16T11:26:58.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.410+0000 7fd717e44640 1 -- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd710070830 msgr2=0x7fd71014ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:58.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.410+0000 7fd717e44640 1 --2- 172.21.15.84:0/42042727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd710070830 0x7fd71014ce00 secure :-1 s=READY pgs=3249 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00e870 tx=0x7fd70c00ed30 comp rx=0 tx=0).stop 2023-10-16T11:26:58.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.410+0000 7fd7163e1640 1 -- 172.21.15.84:0/42042727 reap_dead start 2023-10-16T11:26:58.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.410+0000 7fd717e44640 1 -- 172.21.15.84:0/42042727 shutdown_connections 2023-10-16T11:26:58.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.410+0000 7fd717e44640 1 -- 172.21.15.84:0/42042727 >> 172.21.15.84:0/42042727 conn(0x7fd7100cfac0 msgr2=0x7fd7100ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:58.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.410+0000 7fd717e44640 1 -- 172.21.15.84:0/42042727 shutdown_connections 2023-10-16T11:26:58.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.411+0000 7fd717e44640 1 -- 172.21.15.84:0/42042727 wait complete. 2023-10-16T11:26:58.433 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-16T11:26:58.433 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1m 2023-10-16T11:26:58.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.644+0000 7ffb0604b640 1 Processor -- start 2023-10-16T11:26:58.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.644+0000 7ffb0604b640 1 -- start start 2023-10-16T11:26:58.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.646+0000 7ffb0604b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb000c1d30 0x7ffb00165aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:58.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.646+0000 7ffb0604b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffb000c8fa0 0x7ffb000cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:58.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.646+0000 7ffb0604b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb00165610 con 0x7ffb000b0360 2023-10-16T11:26:58.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.646+0000 7ffb0604b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb000ce370 con 0x7ffb000c8fa0 2023-10-16T11:26:58.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.646+0000 7ffb0604b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb000ce650 con 0x7ffb000c1d30 2023-10-16T11:26:58.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.646+0000 7ffafffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ffb000b0360 0x7ffb0006afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35660/0 (socket says 172.21.15.84:35660) 2023-10-16T11:26:58.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.646+0000 7ffafffff640 1 -- 172.21.15.84:0/4149588937 learned_addr learned my addr 172.21.15.84:0/4149588937 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:58.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.647+0000 7ffaf7fff640 1 --2- 172.21.15.84:0/4149588937 >> v2:172.21.15.157:3300/0 conn(0x7ffb000c8fa0 0x7ffb000cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:58.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.647+0000 7ffaff7fe640 1 --2- 172.21.15.84:0/4149588937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb000c1d30 0x7ffb00165aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:58.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.647+0000 7ffafeffd640 1 -- 172.21.15.84:0/4149588937 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592576706 0 0) 0x7ffb00165610 con 0x7ffb000b0360 2023-10-16T11:26:58.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.647+0000 7ffafeffd640 1 -- 172.21.15.84:0/4149588937 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaec003520 con 0x7ffb000b0360 2023-10-16T11:26:58.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.647+0000 7ffafeffd640 1 -- 172.21.15.84:0/4149588937 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4175274154 0 0) 0x7ffaec003520 con 0x7ffb000b0360 2023-10-16T11:26:58.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.647+0000 7ffafeffd640 1 -- 172.21.15.84:0/4149588937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb000c1d30 msgr2=0x7ffb00165aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:58.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.647+0000 7ffafeffd640 1 --2- 172.21.15.84:0/4149588937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb000c1d30 0x7ffb00165aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:58.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.647+0000 7ffafeffd640 1 -- 172.21.15.84:0/4149588937 >> v2:172.21.15.157:3300/0 conn(0x7ffb000c8fa0 msgr2=0x7ffb000cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:58.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.647+0000 7ffafeffd640 1 --2- 172.21.15.84:0/4149588937 >> v2:172.21.15.157:3300/0 conn(0x7ffb000c8fa0 0x7ffb000cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:58.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.647+0000 7ffafeffd640 1 -- 172.21.15.84:0/4149588937 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb00164fc0 con 0x7ffb000b0360 2023-10-16T11:26:58.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.647+0000 7ffafeffd640 1 -- 172.21.15.84:0/4149588937 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ffaf80019f0 con 0x7ffb000b0360 2023-10-16T11:26:58.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.647+0000 7ffafeffd640 1 -- 172.21.15.84:0/4149588937 >> v1:172.21.15.84:6789/0 conn(0x7ffb000b0360 legacy=0x7ffb0006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:58.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.647+0000 7ffafeffd640 1 --2- 172.21.15.84:0/4149588937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb000c1d30 0x7ffaec0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:58.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.647+0000 7ffafeffd640 1 --2- 172.21.15.84:0/4149588937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaec003f60 0x7ffaec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:58.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.647+0000 7ffafeffd640 1 --2- 172.21.15.84:0/4149588937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffaec004d00 0x7ffaec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:58.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.647+0000 7ffafeffd640 1 -- 172.21.15.84:0/4149588937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb000ce370 con 0x7ffaec004d00 2023-10-16T11:26:58.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.648+0000 7ffafeffd640 1 -- 172.21.15.84:0/4149588937 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb000ce650 con 0x7ffaec003f60 2023-10-16T11:26:58.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.648+0000 7ffafeffd640 1 -- 172.21.15.84:0/4149588937 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec00b9b0 con 0x7ffb000c1d30 2023-10-16T11:26:58.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.648+0000 7ffafffff640 1 --2- 172.21.15.84:0/4149588937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffaec004d00 0x7ffaec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:58.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.648+0000 7ffaff7fe640 1 --2- 172.21.15.84:0/4149588937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb000c1d30 0x7ffaec0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:58.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.648+0000 7ffaf7fff640 1 --2- 172.21.15.84:0/4149588937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaec003f60 0x7ffaec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:58.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.648+0000 7ffafffff640 1 -- 172.21.15.84:0/4149588937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb000c1d30 msgr2=0x7ffaec0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:58.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.648+0000 7ffafffff640 1 --2- 172.21.15.84:0/4149588937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb000c1d30 0x7ffaec0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:58.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.648+0000 7ffafffff640 1 -- 172.21.15.84:0/4149588937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaec003f60 msgr2=0x7ffaec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:58.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.648+0000 7ffafffff640 1 --2- 172.21.15.84:0/4149588937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaec003f60 0x7ffaec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:58.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.648+0000 7ffafffff640 1 -- 172.21.15.84:0/4149588937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaec003520 con 0x7ffaec004d00 2023-10-16T11:26:58.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.649+0000 7ffafffff640 1 --2- 172.21.15.84:0/4149588937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffaec004d00 0x7ffaec005180 secure :-1 s=READY pgs=3264 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800a860 tx=0x7ffaec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2b2a1e270b3898 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:58.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.649+0000 7ffafeffd640 1 -- 172.21.15.84:0/4149588937 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8034630 con 0x7ffaec004d00 2023-10-16T11:26:58.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.649+0000 7ffafeffd640 1 -- 172.21.15.84:0/4149588937 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf802f070 con 0x7ffaec004d00 2023-10-16T11:26:58.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.650+0000 7ffafeffd640 1 -- 172.21.15.84:0/4149588937 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8034a30 con 0x7ffaec004d00 2023-10-16T11:26:58.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.650+0000 7ffb0604b640 1 -- 172.21.15.84:0/4149588937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffaec004d00 msgr2=0x7ffaec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:58.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.650+0000 7ffb0604b640 1 --2- 172.21.15.84:0/4149588937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffaec004d00 0x7ffaec005180 secure :-1 s=READY pgs=3264 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800a860 tx=0x7ffaec0057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:58.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.650+0000 7ffafffff640 1 -- 172.21.15.84:0/4149588937 reap_dead start 2023-10-16T11:26:58.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.650+0000 7ffb0604b640 1 -- 172.21.15.84:0/4149588937 shutdown_connections 2023-10-16T11:26:58.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.650+0000 7ffb0604b640 1 -- 172.21.15.84:0/4149588937 >> 172.21.15.84:0/4149588937 conn(0x7ffb000cb360 msgr2=0x7ffb000cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:58.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.650+0000 7ffb0604b640 1 -- 172.21.15.84:0/4149588937 shutdown_connections 2023-10-16T11:26:58.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.650+0000 7ffb0604b640 1 -- 172.21.15.84:0/4149588937 wait complete. 2023-10-16T11:26:58.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.652+0000 7ffb0604b640 1 Processor -- start 2023-10-16T11:26:58.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.652+0000 7ffb0604b640 1 -- start start 2023-10-16T11:26:58.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.653+0000 7ffb0604b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaec004d00 0x7ffb0006f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:58.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.653+0000 7ffb0604b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb000c4e60 0x7ffb001fa130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:58.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.653+0000 7ffb0604b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb001549b0 0x7ffb00154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:58.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.653+0000 7ffb0604b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb001fb060 con 0x7ffb000c4e60 2023-10-16T11:26:58.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.653+0000 7ffb0604b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb00155380 con 0x7ffaec004d00 2023-10-16T11:26:58.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.653+0000 7ffb0604b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb001faaa0 con 0x7ffb001549b0 2023-10-16T11:26:58.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.654+0000 7ffaf7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb000c4e60 0x7ffb001fa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.654+0000 7ffaf7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb000c4e60 0x7ffb001fa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42390/0 (socket says 172.21.15.84:42390) 2023-10-16T11:26:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.654+0000 7ffaf7fff640 1 -- 172.21.15.84:0/1351079209 learned_addr learned my addr 172.21.15.84:0/1351079209 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.654+0000 7ffaff7fe640 1 --2- 172.21.15.84:0/1351079209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaec004d00 0x7ffb0006f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.654+0000 7ffafffff640 1 --2- 172.21.15.84:0/1351079209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb001549b0 0x7ffb00154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.654+0000 7ffaf7fff640 1 -- 172.21.15.84:0/1351079209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb001549b0 msgr2=0x7ffb00154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.654+0000 7ffaf7fff640 1 --2- 172.21.15.84:0/1351079209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb001549b0 0x7ffb00154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.654+0000 7ffaf7fff640 1 -- 172.21.15.84:0/1351079209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaec004d00 msgr2=0x7ffb0006f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.654+0000 7ffaf7fff640 1 --2- 172.21.15.84:0/1351079209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaec004d00 0x7ffb0006f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.654+0000 7ffaf7fff640 1 -- 172.21.15.84:0/1351079209 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb001fad80 con 0x7ffb000c4e60 2023-10-16T11:26:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.654+0000 7ffaf7fff640 1 --2- 172.21.15.84:0/1351079209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb000c4e60 0x7ffb001fa130 secure :-1 s=READY pgs=3265 cs=0 l=1 rev1=1 crypto rx=0x7ffae8011600 tx=0x7ffae8011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.655+0000 7ffafcff9640 1 -- 172.21.15.84:0/1351079209 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae800f8c0 con 0x7ffb000c4e60 2023-10-16T11:26:58.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.655+0000 7ffafcff9640 1 -- 172.21.15.84:0/1351079209 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae800fa20 con 0x7ffb000c4e60 2023-10-16T11:26:58.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.655+0000 7ffafcff9640 1 -- 172.21.15.84:0/1351079209 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae800fdd0 con 0x7ffb000c4e60 2023-10-16T11:26:58.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.655+0000 7ffb0604b640 1 -- 172.21.15.84:0/1351079209 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb001f7e10 con 0x7ffb000c4e60 2023-10-16T11:26:58.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.655+0000 7ffb0604b640 1 -- 172.21.15.84:0/1351079209 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0006fb40 con 0x7ffb000c4e60 2023-10-16T11:26:58.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.656+0000 7ffb0604b640 1 -- 172.21.15.84:0/1351079209 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb001f7c50 con 0x7ffb000c4e60 2023-10-16T11:26:58.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.657+0000 7ffafcff9640 1 -- 172.21.15.84:0/1351079209 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 154) v1 ==== 89084+0+0 (secure 0 0 0) 0x7ffae8015460 con 0x7ffb000c4e60 2023-10-16T11:26:58.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.658+0000 7ffafcff9640 1 --2- 172.21.15.84:0/1351079209 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7ffae006ad10 0x7ffae006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:58.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.658+0000 7ffafcff9640 1 -- 172.21.15.84:0/1351079209 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6210+0+0 (secure 0 0 0) 0x7ffae8089dc0 con 0x7ffb000c4e60 2023-10-16T11:26:58.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.658+0000 7ffaff7fe640 1 --2- 172.21.15.84:0/1351079209 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7ffae006ad10 0x7ffae006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:58.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.659+0000 7ffaff7fe640 1 --2- 172.21.15.84:0/1351079209 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7ffae006ad10 0x7ffae006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009a10 tx=0x7ffaf0008040 comp rx=0 tx=0).ready entity=mgr.13143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:58.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.663+0000 7ffafcff9640 1 -- 172.21.15.84:0/1351079209 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffae8054ee0 con 0x7ffb000c4e60 2023-10-16T11:26:58.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:58.876+0000 7ffb0604b640 1 -- 172.21.15.84:0/1351079209 --> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb001652a0 con 0x7ffae006ad10 2023-10-16T11:26:59.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.058+0000 7ffafcff9640 1 -- 172.21.15.84:0/1351079209 <== mgr.13143 v2:172.21.15.84:6832/2988856492 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7ffaf0013a50 con 0x7ffae006ad10 2023-10-16T11:26:59.061 INFO:teuthology.orchestra.run.smithi084.stdout:Retention added to path snap_test_dir1 2023-10-16T11:26:59.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.062+0000 7ffb0604b640 1 -- 172.21.15.84:0/1351079209 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7ffae006ad10 msgr2=0x7ffae006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:59.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.062+0000 7ffb0604b640 1 --2- 172.21.15.84:0/1351079209 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7ffae006ad10 0x7ffae006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009a10 tx=0x7ffaf0008040 comp rx=0 tx=0).stop 2023-10-16T11:26:59.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.062+0000 7ffb0604b640 1 -- 172.21.15.84:0/1351079209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb000c4e60 msgr2=0x7ffb001fa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:59.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.062+0000 7ffb0604b640 1 --2- 172.21.15.84:0/1351079209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb000c4e60 0x7ffb001fa130 secure :-1 s=READY pgs=3265 cs=0 l=1 rev1=1 crypto rx=0x7ffae8011600 tx=0x7ffae8011ac0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.062+0000 7ffb0604b640 1 -- 172.21.15.84:0/1351079209 shutdown_connections 2023-10-16T11:26:59.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.062+0000 7ffb0604b640 1 --2- 172.21.15.84:0/1351079209 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7ffae006ad10 0x7ffae006d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.062+0000 7ffb0604b640 1 --2- 172.21.15.84:0/1351079209 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb001549b0 0x7ffb00154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.062+0000 7ffb0604b640 1 --2- 172.21.15.84:0/1351079209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb000c4e60 0x7ffb001fa130 unknown :-1 s=CLOSED pgs=3265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.062+0000 7ffb0604b640 1 --2- 172.21.15.84:0/1351079209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaec004d00 0x7ffb0006f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.062+0000 7ffb0604b640 1 -- 172.21.15.84:0/1351079209 >> 172.21.15.84:0/1351079209 conn(0x7ffb000cb360 msgr2=0x7ffb000ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:59.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.062+0000 7ffb0604b640 1 -- 172.21.15.84:0/1351079209 shutdown_connections 2023-10-16T11:26:59.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.063+0000 7ffb0604b640 1 -- 172.21.15.84:0/1351079209 wait complete. 2023-10-16T11:26:59.086 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-10-16T11:26:59.087 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-16-11_27_00 in ~1s... 2023-10-16T11:26:59.087 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [{'m': 1}] 2023-10-16T11:26:59.087 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-16T11:26:59.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.299+0000 7f4b381c9640 1 Processor -- start 2023-10-16T11:26:59.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.300+0000 7f4b381c9640 1 -- start start 2023-10-16T11:26:59.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.301+0000 7f4b381c9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4b300ad350 0x7f4b30069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:59.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.301+0000 7f4b381c9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4b3006a2e0 0x7f4b300c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:59.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.301+0000 7f4b381c9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b3006a730 con 0x7f4b3015d150 2023-10-16T11:26:59.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.301+0000 7f4b381c9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b3015d530 con 0x7f4b300ad350 2023-10-16T11:26:59.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.301+0000 7f4b381c9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b300ac890 con 0x7f4b3006a2e0 2023-10-16T11:26:59.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.301+0000 7f4b35764640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4b300ad350 0x7f4b30069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:59.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.302+0000 7f4b35764640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4b300ad350 0x7f4b30069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42400/0 (socket says 172.21.15.84:42400) 2023-10-16T11:26:59.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.302+0000 7f4b35764640 1 -- 172.21.15.84:0/348487276 learned_addr learned my addr 172.21.15.84:0/348487276 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:59.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.302+0000 7f4b36766640 1 --2- 172.21.15.84:0/348487276 >> v2:172.21.15.157:3300/0 conn(0x7f4b3006a2e0 0x7f4b300c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:59.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.302+0000 7f4b35764640 1 -- 172.21.15.84:0/348487276 >> v2:172.21.15.157:3300/0 conn(0x7f4b3006a2e0 msgr2=0x7f4b300c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:59.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.302+0000 7f4b35764640 1 --2- 172.21.15.84:0/348487276 >> v2:172.21.15.157:3300/0 conn(0x7f4b3006a2e0 0x7f4b300c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.302+0000 7f4b35764640 1 -- 172.21.15.84:0/348487276 >> v1:172.21.15.157:6789/0 conn(0x7f4b3015d150 legacy=0x7f4b30160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:59.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.302+0000 7f4b35764640 1 -- 172.21.15.84:0/348487276 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b300ad7a0 con 0x7f4b300ad350 2023-10-16T11:26:59.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.303+0000 7f4b35764640 1 --2- 172.21.15.84:0/348487276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b300ad350 0x7f4b30069da0 secure :-1 s=READY pgs=3266 cs=0 l=1 rev1=1 crypto rx=0x7f4b300adde0 tx=0x7f4b2402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12efeca9bf75bc38 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:59.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.304+0000 7f4b34f63640 1 -- 172.21.15.84:0/348487276 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b24030860 con 0x7f4b300ad350 2023-10-16T11:26:59.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.304+0000 7f4b34f63640 1 -- 172.21.15.84:0/348487276 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b240309c0 con 0x7f4b300ad350 2023-10-16T11:26:59.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.304+0000 7f4b34f63640 1 -- 172.21.15.84:0/348487276 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b24030d50 con 0x7f4b300ad350 2023-10-16T11:26:59.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.304+0000 7f4b381c9640 1 -- 172.21.15.84:0/348487276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b300ad350 msgr2=0x7f4b30069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:59.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.304+0000 7f4b381c9640 1 --2- 172.21.15.84:0/348487276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b300ad350 0x7f4b30069da0 secure :-1 s=READY pgs=3266 cs=0 l=1 rev1=1 crypto rx=0x7f4b300adde0 tx=0x7f4b2402b9a0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.304+0000 7f4b381c9640 1 -- 172.21.15.84:0/348487276 shutdown_connections 2023-10-16T11:26:59.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.304+0000 7f4b381c9640 1 --2- 172.21.15.84:0/348487276 >> v2:172.21.15.157:3300/0 conn(0x7f4b3006a2e0 0x7f4b300c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.304+0000 7f4b381c9640 1 --2- 172.21.15.84:0/348487276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b300ad350 0x7f4b30069da0 unknown :-1 s=CLOSED pgs=3266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.304+0000 7f4b381c9640 1 -- 172.21.15.84:0/348487276 >> 172.21.15.84:0/348487276 conn(0x7f4b300ce9b0 msgr2=0x7f4b300cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:59.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.305+0000 7f4b381c9640 1 -- 172.21.15.84:0/348487276 shutdown_connections 2023-10-16T11:26:59.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.305+0000 7f4b381c9640 1 -- 172.21.15.84:0/348487276 wait complete. 2023-10-16T11:26:59.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.306+0000 7f4b381c9640 1 Processor -- start 2023-10-16T11:26:59.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.307+0000 7f4b381c9640 1 -- start start 2023-10-16T11:26:59.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.308+0000 7f4b381c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b3006a2e0 0x7f4b3015fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:59.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.308+0000 7f4b381c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b300ad350 0x7f4b300c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:59.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.308+0000 7f4b381c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b3015d150 0x7f4b300c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:59.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.308+0000 7f4b381c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b301f5ef0 con 0x7f4b3015d150 2023-10-16T11:26:59.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.308+0000 7f4b381c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b30160420 con 0x7f4b3006a2e0 2023-10-16T11:26:59.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.308+0000 7f4b381c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b300c86f0 con 0x7f4b300ad350 2023-10-16T11:26:59.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.308+0000 7f4b36766640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b3015d150 0x7f4b300c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:59.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.308+0000 7f4b36766640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b3015d150 0x7f4b300c81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42426/0 (socket says 172.21.15.84:42426) 2023-10-16T11:26:59.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.308+0000 7f4b36766640 1 -- 172.21.15.84:0/4191112479 learned_addr learned my addr 172.21.15.84:0/4191112479 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:59.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.308+0000 7f4b35f65640 1 --2- 172.21.15.84:0/4191112479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b3006a2e0 0x7f4b3015fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:59.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.308+0000 7f4b35764640 1 --2- 172.21.15.84:0/4191112479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b300ad350 0x7f4b300c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:59.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.309+0000 7f4b36766640 1 -- 172.21.15.84:0/4191112479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b300ad350 msgr2=0x7f4b300c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:59.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.309+0000 7f4b36766640 1 --2- 172.21.15.84:0/4191112479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b300ad350 0x7f4b300c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.309+0000 7f4b36766640 1 -- 172.21.15.84:0/4191112479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b3006a2e0 msgr2=0x7f4b3015fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:59.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.309+0000 7f4b36766640 1 --2- 172.21.15.84:0/4191112479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b3006a2e0 0x7f4b3015fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.309+0000 7f4b36766640 1 -- 172.21.15.84:0/4191112479 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b240094d0 con 0x7f4b3015d150 2023-10-16T11:26:59.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.309+0000 7f4b35764640 1 --2- 172.21.15.84:0/4191112479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b300ad350 0x7f4b300c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:26:59.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.309+0000 7f4b36766640 1 --2- 172.21.15.84:0/4191112479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b3015d150 0x7f4b300c81b0 secure :-1 s=READY pgs=3269 cs=0 l=1 rev1=1 crypto rx=0x7f4b28007e70 tx=0x7f4b2800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:59.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.309+0000 7f4b227fc640 1 -- 172.21.15.84:0/4191112479 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b28014620 con 0x7f4b3015d150 2023-10-16T11:26:59.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.309+0000 7f4b227fc640 1 -- 172.21.15.84:0/4191112479 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b280147a0 con 0x7f4b3015d150 2023-10-16T11:26:59.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.309+0000 7f4b227fc640 1 -- 172.21.15.84:0/4191112479 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b28014b50 con 0x7f4b3015d150 2023-10-16T11:26:59.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.309+0000 7f4b381c9640 1 -- 172.21.15.84:0/4191112479 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b300c2d00 con 0x7f4b3015d150 2023-10-16T11:26:59.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.309+0000 7f4b381c9640 1 -- 172.21.15.84:0/4191112479 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b300c35a0 con 0x7f4b3015d150 2023-10-16T11:26:59.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.311+0000 7f4b381c9640 1 -- 172.21.15.84:0/4191112479 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b3004e160 con 0x7f4b3015d150 2023-10-16T11:26:59.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.312+0000 7f4b227fc640 1 -- 172.21.15.84:0/4191112479 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 155) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f4b28014cb0 con 0x7f4b3015d150 2023-10-16T11:26:59.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.313+0000 7f4b227fc640 1 --2- 172.21.15.84:0/4191112479 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f4b1406afe0 0x7f4b1406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:59.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.313+0000 7f4b227fc640 1 -- 172.21.15.84:0/4191112479 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6210+0+0 (secure 0 0 0) 0x7f4b2808a460 con 0x7f4b3015d150 2023-10-16T11:26:59.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.313+0000 7f4b35f65640 1 --2- 172.21.15.84:0/4191112479 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f4b1406afe0 0x7f4b1406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:59.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.314+0000 7f4b35f65640 1 --2- 172.21.15.84:0/4191112479 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f4b1406afe0 0x7f4b1406d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c000f60 tx=0x7f4b2c009040 comp rx=0 tx=0).ready entity=mgr.13143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:59.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.317+0000 7f4b227fc640 1 -- 172.21.15.84:0/4191112479 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4b28055160 con 0x7f4b3015d150 2023-10-16T11:26:59.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.532+0000 7f4b381c9640 1 -- 172.21.15.84:0/4191112479 --> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4b30161650 con 0x7f4b1406afe0 2023-10-16T11:26:59.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.549+0000 7f4b227fc640 1 -- 172.21.15.84:0/4191112479 <== mgr.13143 v2:172.21.15.84:6832/2988856492 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f4b2c0012c0 con 0x7f4b1406afe0 2023-10-16T11:26:59.551 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:26:59.552 INFO:teuthology.orchestra.run.smithi084.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2023-10-16T11:26:59.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.552+0000 7f4b381c9640 1 -- 172.21.15.84:0/4191112479 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f4b1406afe0 msgr2=0x7f4b1406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:59.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.552+0000 7f4b381c9640 1 --2- 172.21.15.84:0/4191112479 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f4b1406afe0 0x7f4b1406d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c000f60 tx=0x7f4b2c009040 comp rx=0 tx=0).stop 2023-10-16T11:26:59.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.552+0000 7f4b381c9640 1 -- 172.21.15.84:0/4191112479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b3015d150 msgr2=0x7f4b300c81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:59.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.552+0000 7f4b381c9640 1 --2- 172.21.15.84:0/4191112479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b3015d150 0x7f4b300c81b0 secure :-1 s=READY pgs=3269 cs=0 l=1 rev1=1 crypto rx=0x7f4b28007e70 tx=0x7f4b2800e4b0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.552+0000 7f4b381c9640 1 -- 172.21.15.84:0/4191112479 shutdown_connections 2023-10-16T11:26:59.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.552+0000 7f4b381c9640 1 --2- 172.21.15.84:0/4191112479 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f4b1406afe0 0x7f4b1406d4a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.552+0000 7f4b381c9640 1 --2- 172.21.15.84:0/4191112479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b3015d150 0x7f4b300c81b0 unknown :-1 s=CLOSED pgs=3269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.552+0000 7f4b381c9640 1 --2- 172.21.15.84:0/4191112479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b300ad350 0x7f4b300c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.552+0000 7f4b381c9640 1 --2- 172.21.15.84:0/4191112479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b3006a2e0 0x7f4b3015fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.552+0000 7f4b381c9640 1 -- 172.21.15.84:0/4191112479 >> 172.21.15.84:0/4191112479 conn(0x7f4b300ce9b0 msgr2=0x7f4b3015b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:59.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.552+0000 7f4b381c9640 1 -- 172.21.15.84:0/4191112479 shutdown_connections 2023-10-16T11:26:59.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.552+0000 7f4b381c9640 1 -- 172.21.15.84:0/4191112479 wait complete. 2023-10-16T11:26:59.573 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2023-10-16T11:26:59.574 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{'m': 1}]} 2023-10-16T11:26:59.574 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-16T11:26:59.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.774+0000 7f981f11d640 1 Processor -- start 2023-10-16T11:26:59.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.774+0000 7f981f11d640 1 -- start start 2023-10-16T11:26:59.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.776+0000 7f981f11d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f981806c850 0x7f98180ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:59.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.776+0000 7f981f11d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9818070360 con 0x7f9818159e90 2023-10-16T11:26:59.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.776+0000 7f981f11d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98180cd1f0 con 0x7f9818159ab0 2023-10-16T11:26:59.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.776+0000 7f981f11d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981815d890 con 0x7f981806c850 2023-10-16T11:26:59.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.776+0000 7f9817fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9818159e90 0x7f981806c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35666/0 (socket says 172.21.15.84:35666) 2023-10-16T11:26:59.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.776+0000 7f9817fff640 1 -- 172.21.15.84:0/101791360 learned_addr learned my addr 172.21.15.84:0/101791360 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:26:59.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.776+0000 7f98177fe640 1 -- 172.21.15.84:0/101791360 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719873019 0 0) 0x7f9818070360 con 0x7f9818159e90 2023-10-16T11:26:59.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.776+0000 7f981d6ba640 1 --2- 172.21.15.84:0/101791360 >> v2:172.21.15.157:3300/0 conn(0x7f981806c850 0x7f98180ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:59.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.776+0000 7f98177fe640 1 -- 172.21.15.84:0/101791360 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9808003520 con 0x7f9818159e90 2023-10-16T11:26:59.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.777+0000 7f98177fe640 1 -- 172.21.15.84:0/101791360 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 960727694 0 0) 0x7f9808003520 con 0x7f9818159e90 2023-10-16T11:26:59.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.777+0000 7f98177fe640 1 -- 172.21.15.84:0/101791360 >> v2:172.21.15.157:3300/0 conn(0x7f981806c850 msgr2=0x7f98180ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:59.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.777+0000 7f98177fe640 1 --2- 172.21.15.84:0/101791360 >> v2:172.21.15.157:3300/0 conn(0x7f981806c850 0x7f98180ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.777+0000 7f98177fe640 1 -- 172.21.15.84:0/101791360 >> v1:172.21.15.157:6789/0 conn(0x7f9818159ab0 legacy=0x7f981806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:59.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.777+0000 7f98177fe640 1 -- 172.21.15.84:0/101791360 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98180cd4d0 con 0x7f9818159e90 2023-10-16T11:26:59.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.777+0000 7f98177fe640 1 -- 172.21.15.84:0/101791360 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f9804001690 con 0x7f9818159e90 2023-10-16T11:26:59.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.777+0000 7f98177fe640 1 -- 172.21.15.84:0/101791360 >> v1:172.21.15.84:6789/0 conn(0x7f9818159e90 legacy=0x7f981806c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:59.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.777+0000 7f98177fe640 1 --2- 172.21.15.84:0/101791360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98080039c0 0x7f9808003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:59.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.777+0000 7f98177fe640 1 --2- 172.21.15.84:0/101791360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98080047c0 0x7f9808004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:59.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.777+0000 7f98177fe640 1 --2- 172.21.15.84:0/101791360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9808005380 0x7f980800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:59.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.777+0000 7f98177fe640 1 -- 172.21.15.84:0/101791360 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981815d890 con 0x7f98080039c0 2023-10-16T11:26:59.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.777+0000 7f98177fe640 1 -- 172.21.15.84:0/101791360 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980800e470 con 0x7f9808005380 2023-10-16T11:26:59.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.777+0000 7f98177fe640 1 -- 172.21.15.84:0/101791360 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980800e630 con 0x7f98080047c0 2023-10-16T11:26:59.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.777+0000 7f981ceb9640 1 --2- 172.21.15.84:0/101791360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98080039c0 0x7f9808003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:59.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.778+0000 7f981ceb9640 1 -- 172.21.15.84:0/101791360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98080047c0 msgr2=0x7f9808004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:59.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.778+0000 7f9817fff640 1 --2- 172.21.15.84:0/101791360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98080047c0 0x7f9808004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:59.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.778+0000 7f981d6ba640 1 --2- 172.21.15.84:0/101791360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9808005380 0x7f980800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:59.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.778+0000 7f981ceb9640 1 --2- 172.21.15.84:0/101791360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98080047c0 0x7f9808004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.778+0000 7f981ceb9640 1 -- 172.21.15.84:0/101791360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9808005380 msgr2=0x7f980800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:26:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.778+0000 7f981ceb9640 1 --2- 172.21.15.84:0/101791360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9808005380 0x7f980800db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.778+0000 7f981ceb9640 1 -- 172.21.15.84:0/101791360 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9808003520 con 0x7f98080039c0 2023-10-16T11:26:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.778+0000 7f981d6ba640 1 -- 172.21.15.84:0/101791360 reap_dead start 2023-10-16T11:26:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.779+0000 7f981ceb9640 1 --2- 172.21.15.84:0/101791360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98080039c0 0x7f9808003de0 secure :-1 s=READY pgs=3271 cs=0 l=1 rev1=1 crypto rx=0x7f980c001750 tx=0x7f98080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cbd2e859dab860 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.779+0000 7f98177fe640 1 -- 172.21.15.84:0/101791360 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c0052d0 con 0x7f98080039c0 2023-10-16T11:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.779+0000 7f98177fe640 1 -- 172.21.15.84:0/101791360 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980c02e050 con 0x7f98080039c0 2023-10-16T11:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.779+0000 7f98177fe640 1 -- 172.21.15.84:0/101791360 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c003830 con 0x7f98080039c0 2023-10-16T11:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.780+0000 7f981f11d640 1 -- 172.21.15.84:0/101791360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98080039c0 msgr2=0x7f9808003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.780+0000 7f981f11d640 1 --2- 172.21.15.84:0/101791360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98080039c0 0x7f9808003de0 secure :-1 s=READY pgs=3271 cs=0 l=1 rev1=1 crypto rx=0x7f980c001750 tx=0x7f98080057f0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.780+0000 7f981f11d640 1 -- 172.21.15.84:0/101791360 shutdown_connections 2023-10-16T11:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.780+0000 7f981f11d640 1 --2- 172.21.15.84:0/101791360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98080039c0 0x7f9808003de0 unknown :-1 s=CLOSED pgs=3271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.780+0000 7f981f11d640 1 -- 172.21.15.84:0/101791360 >> 172.21.15.84:0/101791360 conn(0x7f98180ce9b0 msgr2=0x7f98180cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.780+0000 7f981f11d640 1 -- 172.21.15.84:0/101791360 shutdown_connections 2023-10-16T11:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.780+0000 7f981f11d640 1 -- 172.21.15.84:0/101791360 wait complete. 2023-10-16T11:26:59.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.782+0000 7f981f11d640 1 Processor -- start 2023-10-16T11:26:59.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.782+0000 7f981f11d640 1 -- start start 2023-10-16T11:26:59.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.783+0000 7f981f11d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98080039c0 0x7f98180c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:59.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.783+0000 7f981f11d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98180c3f00 0x7f98180c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:59.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.783+0000 7f981f11d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9818148d90 0x7f9818149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:59.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.783+0000 7f981f11d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98180c5010 con 0x7f9818148d90 2023-10-16T11:26:59.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.783+0000 7f981f11d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9818149740 con 0x7f98080039c0 2023-10-16T11:26:59.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.783+0000 7f981f11d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98180c4400 con 0x7f98180c3f00 2023-10-16T11:26:59.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.783+0000 7f981d6ba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9818148d90 0x7f9818149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:59.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.783+0000 7f981d6ba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9818148d90 0x7f9818149200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42450/0 (socket says 172.21.15.84:42450) 2023-10-16T11:26:59.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.783+0000 7f981d6ba640 1 -- 172.21.15.84:0/1575508906 learned_addr learned my addr 172.21.15.84:0/1575508906 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:26:59.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.783+0000 7f981ceb9640 1 --2- 172.21.15.84:0/1575508906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98080039c0 0x7f98180c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:59.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.783+0000 7f981d6ba640 1 -- 172.21.15.84:0/1575508906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98180c3f00 msgr2=0x7f98180c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:59.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.783+0000 7f981d6ba640 1 --2- 172.21.15.84:0/1575508906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98180c3f00 0x7f98180c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.783+0000 7f981d6ba640 1 -- 172.21.15.84:0/1575508906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98080039c0 msgr2=0x7f98180c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:26:59.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.783+0000 7f981d6ba640 1 --2- 172.21.15.84:0/1575508906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98080039c0 0x7f98180c6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:26:59.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.783+0000 7f981d6ba640 1 -- 172.21.15.84:0/1575508906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98180c46e0 con 0x7f9818148d90 2023-10-16T11:26:59.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.784+0000 7f981d6ba640 1 --2- 172.21.15.84:0/1575508906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9818148d90 0x7f9818149200 secure :-1 s=READY pgs=3272 cs=0 l=1 rev1=1 crypto rx=0x7f98180b15a0 tx=0x7f981000e510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:59.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.784+0000 7f98157fa640 1 -- 172.21.15.84:0/1575508906 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9810007e40 con 0x7f9818148d90 2023-10-16T11:26:59.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.784+0000 7f98157fa640 1 -- 172.21.15.84:0/1575508906 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981000f040 con 0x7f9818148d90 2023-10-16T11:26:59.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.784+0000 7f98157fa640 1 -- 172.21.15.84:0/1575508906 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9810003bd0 con 0x7f9818148d90 2023-10-16T11:26:59.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.784+0000 7f981f11d640 1 -- 172.21.15.84:0/1575508906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98180c49c0 con 0x7f9818148d90 2023-10-16T11:26:59.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.784+0000 7f981f11d640 1 -- 172.21.15.84:0/1575508906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98180c7340 con 0x7f9818148d90 2023-10-16T11:26:59.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.785+0000 7f981f11d640 1 -- 172.21.15.84:0/1575508906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98180c51c0 con 0x7f9818148d90 2023-10-16T11:26:59.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.786+0000 7f98157fa640 1 -- 172.21.15.84:0/1575508906 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 155) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f9810003d30 con 0x7f9818148d90 2023-10-16T11:26:59.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.788+0000 7f98157fa640 1 --2- 172.21.15.84:0/1575508906 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f97fc06b010 0x7f97fc06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:26:59.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.788+0000 7f98157fa640 1 -- 172.21.15.84:0/1575508906 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6210+0+0 (secure 0 0 0) 0x7f981008a060 con 0x7f9818148d90 2023-10-16T11:26:59.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.788+0000 7f981ceb9640 1 --2- 172.21.15.84:0/1575508906 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f97fc06b010 0x7f97fc06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:26:59.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.789+0000 7f981ceb9640 1 --2- 172.21.15.84:0/1575508906 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f97fc06b010 0x7f97fc06d4d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f980c033000 tx=0x7f980c009030 comp rx=0 tx=0).ready entity=mgr.13143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:26:59.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:26:59.792+0000 7f98157fa640 1 -- 172.21.15.84:0/1575508906 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f98180c51c0 con 0x7f9818148d90 2023-10-16T11:27:00.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.009+0000 7f981f11d640 1 -- 172.21.15.84:0/1575508906 --> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f98180cda90 con 0x7f97fc06b010 2023-10-16T11:27:00.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.223+0000 7f98157fa640 1 -- 172.21.15.84:0/1575508906 <== mgr.13143 v2:172.21.15.84:6832/2988856492 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f980c005480 con 0x7f97fc06b010 2023-10-16T11:27:00.226 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule removed for path snap_test_dir1 2023-10-16T11:27:00.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.227+0000 7f981f11d640 1 -- 172.21.15.84:0/1575508906 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f97fc06b010 msgr2=0x7f97fc06d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:00.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.227+0000 7f981f11d640 1 --2- 172.21.15.84:0/1575508906 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f97fc06b010 0x7f97fc06d4d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f980c033000 tx=0x7f980c009030 comp rx=0 tx=0).stop 2023-10-16T11:27:00.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.227+0000 7f981f11d640 1 -- 172.21.15.84:0/1575508906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9818148d90 msgr2=0x7f9818149200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:00.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.227+0000 7f981f11d640 1 --2- 172.21.15.84:0/1575508906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9818148d90 0x7f9818149200 secure :-1 s=READY pgs=3272 cs=0 l=1 rev1=1 crypto rx=0x7f98180b15a0 tx=0x7f981000e510 comp rx=0 tx=0).stop 2023-10-16T11:27:00.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.227+0000 7f981f11d640 1 -- 172.21.15.84:0/1575508906 shutdown_connections 2023-10-16T11:27:00.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.227+0000 7f981f11d640 1 --2- 172.21.15.84:0/1575508906 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f97fc06b010 0x7f97fc06d4d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:00.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.227+0000 7f981f11d640 1 --2- 172.21.15.84:0/1575508906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9818148d90 0x7f9818149200 unknown :-1 s=CLOSED pgs=3272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:00.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.227+0000 7f981f11d640 1 --2- 172.21.15.84:0/1575508906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98180c3f00 0x7f98180c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:00.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.227+0000 7f981f11d640 1 --2- 172.21.15.84:0/1575508906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98080039c0 0x7f98180c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:00.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.227+0000 7f981f11d640 1 -- 172.21.15.84:0/1575508906 >> 172.21.15.84:0/1575508906 conn(0x7f98180ce9b0 msgr2=0x7f98180ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:00.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.228+0000 7f981f11d640 1 -- 172.21.15.84:0/1575508906 shutdown_connections 2023-10-16T11:27:00.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.228+0000 7f981f11d640 1 -- 172.21.15.84:0/1575508906 wait complete. 2023-10-16T11:27:00.251 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-16T11:27:00.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:00.252 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:27:00.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:00.309 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-16T11:27:00.324 INFO:tasks.ceph.mgr.y.smithi084.stderr:Exception in thread Thread-5: 2023-10-16T11:27:00.324 INFO:tasks.ceph.mgr.y.smithi084.stderr:Traceback (most recent call last): 2023-10-16T11:27:00.325 INFO:tasks.ceph.mgr.y.smithi084.stderr: File "/usr/lib64/python3.9/threading.py", line 980, in _bootstrap_inner 2023-10-16T11:27:00.336 INFO:tasks.ceph.mgr.y.smithi084.stderr: self.run() 2023-10-16T11:27:00.337 INFO:tasks.ceph.mgr.y.smithi084.stderr: File "/usr/lib64/python3.9/threading.py", line 1306, in run 2023-10-16T11:27:00.337 INFO:tasks.ceph.mgr.y.smithi084.stderr: self.function(*self.args, **self.kwargs) 2023-10-16T11:27:00.337 INFO:tasks.ceph.mgr.y.smithi084.stderr: File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line 355, in create_scheduled_snapshot 2023-10-16T11:27:00.338 INFO:tasks.ceph.mgr.y.smithi084.stderr: self.prune_snapshots(sched) 2023-10-16T11:27:00.338 INFO:tasks.ceph.mgr.y.smithi084.stderr:UnboundLocalError: local variable 'sched' referenced before assignment 2023-10-16T11:27:00.362 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T11:27:00.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.626+0000 7f83f4d67640 1 Processor -- start 2023-10-16T11:27:00.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.626+0000 7f83f4d67640 1 -- start start 2023-10-16T11:27:00.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.627+0000 7f83f4d67640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83f00ad970 0x7f83f00cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:00.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.627+0000 7f83f4d67640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f015b9d0 con 0x7f83f00ccfb0 2023-10-16T11:27:00.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.628+0000 7f83f4d67640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f015bcb0 con 0x7f83f006c040 2023-10-16T11:27:00.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.628+0000 7f83f4d67640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f015bf90 con 0x7f83f00ad970 2023-10-16T11:27:00.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.628+0000 7f83effff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f83f00ccfb0 0x7f83f00cd460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35674/0 (socket says 172.21.15.84:35674) 2023-10-16T11:27:00.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.628+0000 7f83effff640 1 -- 172.21.15.84:0/1202406297 learned_addr learned my addr 172.21.15.84:0/1202406297 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:00.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.628+0000 7f83e6ffd640 1 --2- 172.21.15.84:0/1202406297 >> v2:172.21.15.84:3300/0 conn(0x7f83f00ad970 0x7f83f00cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:00.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.628+0000 7f83eeffd640 1 -- 172.21.15.84:0/1202406297 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599188386 0 0) 0x7f83f015b9d0 con 0x7f83f00ccfb0 2023-10-16T11:27:00.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.628+0000 7f83eeffd640 1 -- 172.21.15.84:0/1202406297 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83dc003520 con 0x7f83f00ccfb0 2023-10-16T11:27:00.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.628+0000 7f83e6ffd640 1 -- 172.21.15.84:0/1202406297 >> v1:172.21.15.157:6789/0 conn(0x7f83f006c040 legacy=0x7f83f006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:00.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.628+0000 7f83e6ffd640 1 -- 172.21.15.84:0/1202406297 >> v1:172.21.15.84:6789/0 conn(0x7f83f00ccfb0 legacy=0x7f83f00cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:00.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.628+0000 7f83e6ffd640 1 -- 172.21.15.84:0/1202406297 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f00cc270 con 0x7f83f00ad970 2023-10-16T11:27:00.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.630+0000 7f83e6ffd640 1 --2- 172.21.15.84:0/1202406297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f00ad970 0x7f83f00cca70 secure :-1 s=READY pgs=3274 cs=0 l=1 rev1=1 crypto rx=0x7f83d80097d0 tx=0x7f83d802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807be8030b8ce934 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:00.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.630+0000 7f83eeffd640 1 -- 172.21.15.84:0/1202406297 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d8030610 con 0x7f83f00ad970 2023-10-16T11:27:00.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.630+0000 7f83eeffd640 1 -- 172.21.15.84:0/1202406297 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d8030770 con 0x7f83f00ad970 2023-10-16T11:27:00.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.630+0000 7f83eeffd640 1 -- 172.21.15.84:0/1202406297 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d8030b00 con 0x7f83f00ad970 2023-10-16T11:27:00.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.630+0000 7f83f4d67640 1 -- 172.21.15.84:0/1202406297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f00ad970 msgr2=0x7f83f00cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:00.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.630+0000 7f83f4d67640 1 --2- 172.21.15.84:0/1202406297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f00ad970 0x7f83f00cca70 secure :-1 s=READY pgs=3274 cs=0 l=1 rev1=1 crypto rx=0x7f83d80097d0 tx=0x7f83d802b960 comp rx=0 tx=0).stop 2023-10-16T11:27:00.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.631+0000 7f83f4d67640 1 -- 172.21.15.84:0/1202406297 shutdown_connections 2023-10-16T11:27:00.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.631+0000 7f83f4d67640 1 --2- 172.21.15.84:0/1202406297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f00ad970 0x7f83f00cca70 unknown :-1 s=CLOSED pgs=3274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:00.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.631+0000 7f83f4d67640 1 -- 172.21.15.84:0/1202406297 >> 172.21.15.84:0/1202406297 conn(0x7f83f00ce6a0 msgr2=0x7f83f00ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:00.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.631+0000 7f83f4d67640 1 -- 172.21.15.84:0/1202406297 shutdown_connections 2023-10-16T11:27:00.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.631+0000 7f83f4d67640 1 -- 172.21.15.84:0/1202406297 wait complete. 2023-10-16T11:27:00.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.632+0000 7f83f4d67640 1 Processor -- start 2023-10-16T11:27:00.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.633+0000 7f83f4d67640 1 -- start start 2023-10-16T11:27:00.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.633+0000 7f83f4d67640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83f006c040 0x7f83f00a8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:00.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.633+0000 7f83f4d67640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f00ad970 0x7f83f00b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:00.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.633+0000 7f83f4d67640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f00ccfb0 0x7f83f00a9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:00.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.633+0000 7f83f4d67640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00a89c0 con 0x7f83f00ad970 2023-10-16T11:27:00.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.633+0000 7f83f4d67640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00b23b0 con 0x7f83f00ccfb0 2023-10-16T11:27:00.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.633+0000 7f83f4d67640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00b2690 con 0x7f83f006c040 2023-10-16T11:27:00.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.634+0000 7f83e6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f00ad970 0x7f83f00b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:00.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.634+0000 7f83e6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f00ad970 0x7f83f00b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42462/0 (socket says 172.21.15.84:42462) 2023-10-16T11:27:00.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.634+0000 7f83e6ffd640 1 -- 172.21.15.84:0/4027707176 learned_addr learned my addr 172.21.15.84:0/4027707176 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:00.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.634+0000 7f83ef7fe640 1 --2- 172.21.15.84:0/4027707176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83f006c040 0x7f83f00a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:00.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.634+0000 7f83effff640 1 --2- 172.21.15.84:0/4027707176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f00ccfb0 0x7f83f00a9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:00.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.634+0000 7f83e6ffd640 1 -- 172.21.15.84:0/4027707176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83f006c040 msgr2=0x7f83f00a8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:00.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.634+0000 7f83e6ffd640 1 --2- 172.21.15.84:0/4027707176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83f006c040 0x7f83f00a8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:00.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.634+0000 7f83e6ffd640 1 -- 172.21.15.84:0/4027707176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f00ccfb0 msgr2=0x7f83f00a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:00.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.634+0000 7f83e6ffd640 1 --2- 172.21.15.84:0/4027707176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f00ccfb0 0x7f83f00a9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:00.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.634+0000 7f83e6ffd640 1 -- 172.21.15.84:0/4027707176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d8009400 con 0x7f83f00ad970 2023-10-16T11:27:00.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.634+0000 7f83ef7fe640 1 --2- 172.21.15.84:0/4027707176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83f006c040 0x7f83f00a8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:00.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.634+0000 7f83effff640 1 --2- 172.21.15.84:0/4027707176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f00ccfb0 0x7f83f00a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:00.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.634+0000 7f83e6ffd640 1 --2- 172.21.15.84:0/4027707176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f00ad970 0x7f83f00b3450 secure :-1 s=READY pgs=3275 cs=0 l=1 rev1=1 crypto rx=0x7f83d802bf70 tx=0x7f83d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:00.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.635+0000 7f83ecff9640 1 -- 172.21.15.84:0/4027707176 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d803e040 con 0x7f83f00ad970 2023-10-16T11:27:00.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.635+0000 7f83ecff9640 1 -- 172.21.15.84:0/4027707176 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d8039cb0 con 0x7f83f00ad970 2023-10-16T11:27:00.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.635+0000 7f83f4d67640 1 -- 172.21.15.84:0/4027707176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f00b2970 con 0x7f83f00ad970 2023-10-16T11:27:00.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.635+0000 7f83ecff9640 1 -- 172.21.15.84:0/4027707176 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d8044630 con 0x7f83f00ad970 2023-10-16T11:27:00.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.635+0000 7f83f4d67640 1 -- 172.21.15.84:0/4027707176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f00b2f30 con 0x7f83f00ad970 2023-10-16T11:27:00.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.636+0000 7f83f4d67640 1 -- 172.21.15.84:0/4027707176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f00b3120 con 0x7f83f00ad970 2023-10-16T11:27:00.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.637+0000 7f83ecff9640 1 -- 172.21.15.84:0/4027707176 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 155) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f83d804d3b0 con 0x7f83f00ad970 2023-10-16T11:27:00.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.638+0000 7f83ecff9640 1 --2- 172.21.15.84:0/4027707176 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f83d006afe0 0x7f83d006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:00.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.638+0000 7f83ecff9640 1 -- 172.21.15.84:0/4027707176 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6210+0+0 (secure 0 0 0) 0x7f83d80abf90 con 0x7f83f00ad970 2023-10-16T11:27:00.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.638+0000 7f83ef7fe640 1 --2- 172.21.15.84:0/4027707176 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f83d006afe0 0x7f83d006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:00.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.639+0000 7f83ef7fe640 1 --2- 172.21.15.84:0/4027707176 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f83d006afe0 0x7f83d006d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f83f00ad4a0 tx=0x7f83e0009040 comp rx=0 tx=0).ready entity=mgr.13143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:00.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.643+0000 7f83ecff9640 1 -- 172.21.15.84:0/4027707176 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f83d8076c10 con 0x7f83f00ad970 2023-10-16T11:27:00.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:00.884+0000 7f83f4d67640 1 -- 172.21.15.84:0/4027707176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f83f004e9b0 con 0x7f83f00ad970 2023-10-16T11:27:01.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.237+0000 7f83ecff9640 1 -- 172.21.15.84:0/4027707176 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v156) v1 ==== 93+0+0 (secure 0 0 0) 0x7f83d807a230 con 0x7f83f00ad970 2023-10-16T11:27:01.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.241+0000 7f83f4d67640 1 -- 172.21.15.84:0/4027707176 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f83d006afe0 msgr2=0x7f83d006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:01.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.241+0000 7f83f4d67640 1 --2- 172.21.15.84:0/4027707176 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f83d006afe0 0x7f83d006d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f83f00ad4a0 tx=0x7f83e0009040 comp rx=0 tx=0).stop 2023-10-16T11:27:01.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.241+0000 7f83f4d67640 1 -- 172.21.15.84:0/4027707176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f00ad970 msgr2=0x7f83f00b3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:01.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.241+0000 7f83f4d67640 1 --2- 172.21.15.84:0/4027707176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f00ad970 0x7f83f00b3450 secure :-1 s=READY pgs=3275 cs=0 l=1 rev1=1 crypto rx=0x7f83d802bf70 tx=0x7f83d8004730 comp rx=0 tx=0).stop 2023-10-16T11:27:01.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.243+0000 7f83f4d67640 1 -- 172.21.15.84:0/4027707176 shutdown_connections 2023-10-16T11:27:01.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.243+0000 7f83f4d67640 1 --2- 172.21.15.84:0/4027707176 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f83d006afe0 0x7f83d006d4a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:01.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.243+0000 7f83f4d67640 1 --2- 172.21.15.84:0/4027707176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f00ccfb0 0x7f83f00a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:01.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.243+0000 7f83f4d67640 1 --2- 172.21.15.84:0/4027707176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f00ad970 0x7f83f00b3450 unknown :-1 s=CLOSED pgs=3275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:01.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.243+0000 7f83f4d67640 1 --2- 172.21.15.84:0/4027707176 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83f006c040 0x7f83f00a8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:01.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.243+0000 7f83f4d67640 1 -- 172.21.15.84:0/4027707176 >> 172.21.15.84:0/4027707176 conn(0x7f83f00ce6a0 msgr2=0x7f83f006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:01.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.243+0000 7f83f4d67640 1 -- 172.21.15.84:0/4027707176 shutdown_connections 2023-10-16T11:27:01.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.243+0000 7f83f4d67640 1 -- 172.21.15.84:0/4027707176 wait complete. 2023-10-16T11:27:01.395 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:27:01.395 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:27:01.515 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:27:01.619 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:27:01.619 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:27:01.620 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:01.674 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:27:01.675 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:01.728 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:27:01.769 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:01.767+0000 7f7879a33040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:27:01.812 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:01.810+0000 7f7a5dc15040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:27:01.825 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:01.825+0000 7f7879a33040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:27:01.866 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:01.864+0000 7f7a5dc15040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:27:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.924+0000 7f5aad059640 1 Processor -- start 2023-10-16T11:27:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.924+0000 7f5aad059640 1 -- start start 2023-10-16T11:27:01.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.925+0000 7f5aad059640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5aa806c040 0x7f5aa806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:01.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.925+0000 7f5aad059640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5aa80ad970 0x7f5aa815e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:01.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.925+0000 7f5aad059640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5aa815ebc0 0x7f5aa815f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:01.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.925+0000 7f5aad059640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa806c960 con 0x7f5aa815ebc0 2023-10-16T11:27:01.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.925+0000 7f5aad059640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa815b9d0 con 0x7f5aa80ad970 2023-10-16T11:27:01.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.925+0000 7f5aad059640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa815bcb0 con 0x7f5aa806c040 2023-10-16T11:27:01.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.926+0000 7f5aac858640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5aa815ebc0 0x7f5aa815f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:01.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.926+0000 7f5aac858640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5aa815ebc0 0x7f5aa815f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42482/0 (socket says 172.21.15.84:42482) 2023-10-16T11:27:01.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.926+0000 7f5aac858640 1 -- 172.21.15.84:0/1030393561 learned_addr learned my addr 172.21.15.84:0/1030393561 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:01.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.926+0000 7f5a9ffff640 1 --2- 172.21.15.84:0/1030393561 >> v2:172.21.15.157:3300/0 conn(0x7f5aa80ad970 0x7f5aa815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:01.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.926+0000 7f5aa7fff640 1 --2- 172.21.15.84:0/1030393561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5aa806c040 0x7f5aa806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:01.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.926+0000 7f5aac858640 1 -- 172.21.15.84:0/1030393561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5aa806c040 msgr2=0x7f5aa806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:01.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.926+0000 7f5aac858640 1 --2- 172.21.15.84:0/1030393561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5aa806c040 0x7f5aa806c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:01.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.926+0000 7f5aac858640 1 -- 172.21.15.84:0/1030393561 >> v2:172.21.15.157:3300/0 conn(0x7f5aa80ad970 msgr2=0x7f5aa815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:01.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.926+0000 7f5aac858640 1 --2- 172.21.15.84:0/1030393561 >> v2:172.21.15.157:3300/0 conn(0x7f5aa80ad970 0x7f5aa815e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:01.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.926+0000 7f5aac858640 1 -- 172.21.15.84:0/1030393561 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa815bf90 con 0x7f5aa815ebc0 2023-10-16T11:27:01.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.926+0000 7f5a9ffff640 1 --2- 172.21.15.84:0/1030393561 >> v2:172.21.15.157:3300/0 conn(0x7f5aa80ad970 0x7f5aa815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:01.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.926+0000 7f5aa7fff640 1 --2- 172.21.15.84:0/1030393561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5aa806c040 0x7f5aa806c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:01.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.927+0000 7f5aac858640 1 --2- 172.21.15.84:0/1030393561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5aa815ebc0 0x7f5aa815f050 secure :-1 s=READY pgs=3281 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0009e50 tx=0x7f5aa002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04d2d73d493ef81 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:01.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.928+0000 7f5aa77fe640 1 -- 172.21.15.84:0/1030393561 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa002ca80 con 0x7f5aa815ebc0 2023-10-16T11:27:01.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.928+0000 7f5aa77fe640 1 -- 172.21.15.84:0/1030393561 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa002cbe0 con 0x7f5aa815ebc0 2023-10-16T11:27:01.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.928+0000 7f5aa77fe640 1 -- 172.21.15.84:0/1030393561 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00343f0 con 0x7f5aa815ebc0 2023-10-16T11:27:01.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.928+0000 7f5aad059640 1 -- 172.21.15.84:0/1030393561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5aa815ebc0 msgr2=0x7f5aa815f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:01.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.928+0000 7f5aad059640 1 --2- 172.21.15.84:0/1030393561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5aa815ebc0 0x7f5aa815f050 secure :-1 s=READY pgs=3281 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0009e50 tx=0x7f5aa002bee0 comp rx=0 tx=0).stop 2023-10-16T11:27:01.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.928+0000 7f5aad059640 1 -- 172.21.15.84:0/1030393561 shutdown_connections 2023-10-16T11:27:01.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.928+0000 7f5aad059640 1 --2- 172.21.15.84:0/1030393561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5aa815ebc0 0x7f5aa815f050 unknown :-1 s=CLOSED pgs=3281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:01.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.928+0000 7f5aad059640 1 --2- 172.21.15.84:0/1030393561 >> v2:172.21.15.157:3300/0 conn(0x7f5aa80ad970 0x7f5aa815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:01.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.928+0000 7f5aad059640 1 --2- 172.21.15.84:0/1030393561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5aa806c040 0x7f5aa806c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:01.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.928+0000 7f5aad059640 1 -- 172.21.15.84:0/1030393561 >> 172.21.15.84:0/1030393561 conn(0x7f5aa80ce6a0 msgr2=0x7f5aa80ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:01.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.928+0000 7f5aad059640 1 -- 172.21.15.84:0/1030393561 shutdown_connections 2023-10-16T11:27:01.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.928+0000 7f5aad059640 1 -- 172.21.15.84:0/1030393561 wait complete. 2023-10-16T11:27:01.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.930+0000 7f5aad059640 1 Processor -- start 2023-10-16T11:27:01.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.930+0000 7f5aad059640 1 -- start start 2023-10-16T11:27:01.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.931+0000 7f5aad059640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5aa806c040 0x7f5aa80c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:01.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.931+0000 7f5aad059640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa80ad970 0x7f5aa8152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:01.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.931+0000 7f5aad059640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5aa815ebc0 0x7f5aa80c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:01.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.931+0000 7f5aad059640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa80c7cc0 con 0x7f5aa815ebc0 2023-10-16T11:27:01.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.931+0000 7f5aad059640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa80c9c80 con 0x7f5aa80ad970 2023-10-16T11:27:01.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.931+0000 7f5aad059640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa80c8cf0 con 0x7f5aa806c040 2023-10-16T11:27:01.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.932+0000 7f5aac858640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5aa815ebc0 0x7f5aa80c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:01.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.932+0000 7f5aac858640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5aa815ebc0 0x7f5aa80c6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42484/0 (socket says 172.21.15.84:42484) 2023-10-16T11:27:01.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.932+0000 7f5aac858640 1 -- 172.21.15.84:0/4187053908 learned_addr learned my addr 172.21.15.84:0/4187053908 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:01.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.932+0000 7f5aa7fff640 1 --2- 172.21.15.84:0/4187053908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5aa806c040 0x7f5aa80c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:01.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.932+0000 7f5a9ffff640 1 --2- 172.21.15.84:0/4187053908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa80ad970 0x7f5aa8152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:01.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.932+0000 7f5aac858640 1 -- 172.21.15.84:0/4187053908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5aa806c040 msgr2=0x7f5aa80c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:01.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.932+0000 7f5aac858640 1 --2- 172.21.15.84:0/4187053908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5aa806c040 0x7f5aa80c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:01.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.932+0000 7f5aac858640 1 -- 172.21.15.84:0/4187053908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa80ad970 msgr2=0x7f5aa8152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:01.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.932+0000 7f5aac858640 1 --2- 172.21.15.84:0/4187053908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa80ad970 0x7f5aa8152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:01.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.932+0000 7f5aac858640 1 -- 172.21.15.84:0/4187053908 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa0009b30 con 0x7f5aa815ebc0 2023-10-16T11:27:01.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.932+0000 7f5a9ffff640 1 --2- 172.21.15.84:0/4187053908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa80ad970 0x7f5aa8152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:01.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.933+0000 7f5aac858640 1 --2- 172.21.15.84:0/4187053908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5aa815ebc0 0x7f5aa80c6cb0 secure :-1 s=READY pgs=3282 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0009e50 tx=0x7f5aa0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:01.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.933+0000 7f5aa57fa640 1 -- 172.21.15.84:0/4187053908 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0040670 con 0x7f5aa815ebc0 2023-10-16T11:27:01.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.933+0000 7f5aa57fa640 1 -- 172.21.15.84:0/4187053908 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa00407f0 con 0x7f5aa815ebc0 2023-10-16T11:27:01.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.933+0000 7f5aa57fa640 1 -- 172.21.15.84:0/4187053908 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0040ba0 con 0x7f5aa815ebc0 2023-10-16T11:27:01.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.933+0000 7f5aad059640 1 -- 172.21.15.84:0/4187053908 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa80c92b0 con 0x7f5aa815ebc0 2023-10-16T11:27:01.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.933+0000 7f5aad059640 1 -- 172.21.15.84:0/4187053908 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa80c79e0 con 0x7f5aa815ebc0 2023-10-16T11:27:01.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.934+0000 7f5aad059640 1 -- 172.21.15.84:0/4187053908 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa806b330 con 0x7f5aa815ebc0 2023-10-16T11:27:01.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.936+0000 7f5aa57fa640 1 -- 172.21.15.84:0/4187053908 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 156) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f5aa002cd50 con 0x7f5aa815ebc0 2023-10-16T11:27:01.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.936+0000 7f5aa57fa640 1 --2- 172.21.15.84:0/4187053908 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f5a8806b020 0x7f5a8806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:01.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.937+0000 7f5aa57fa640 1 -- 172.21.15.84:0/4187053908 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6210+0+0 (secure 0 0 0) 0x7f5aa00acbc0 con 0x7f5aa815ebc0 2023-10-16T11:27:01.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.937+0000 7f5aa7fff640 1 -- 172.21.15.84:0/4187053908 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f5a8806b020 msgr2=0x7f5a8806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2988856492 2023-10-16T11:27:01.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.937+0000 7f5aa7fff640 1 --2- 172.21.15.84:0/4187053908 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f5a8806b020 0x7f5a8806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:01.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:01.941+0000 7f5aa57fa640 1 -- 172.21.15.84:0/4187053908 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5aa00779a0 con 0x7f5aa815ebc0 2023-10-16T11:27:02.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.132+0000 7f5aad059640 1 -- 172.21.15.84:0/4187053908 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5aa8153350 con 0x7f5aa815ebc0 2023-10-16T11:27:02.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.133+0000 7f5aa57fa640 1 -- 172.21.15.84:0/4187053908 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v258) v1 ==== 74+0+17231 (secure 0 0 0) 0x7f5aa0009510 con 0x7f5aa815ebc0 2023-10-16T11:27:02.134 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:02.135 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":258,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:26:57.151442+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:26:39.860053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-16T11:26:40.916519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/907959269":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/1863310536":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/847549599":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/1934386371":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/3540589884":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/2370234804":"2023-10-17T11:26:57.151307+0000","172.21.15.84:6833/2330196255":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/1650008535":"2023-10-17T11:26:57.151307+0000","172.21.15.84:6832/2330196255":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/2440426242":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/570042115":"2023-10-16T12:26:40.168744+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:27:02.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.135+0000 7f5aad059640 1 -- 172.21.15.84:0/4187053908 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f5a8806b020 msgr2=0x7f5a8806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:02.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.136+0000 7f5aad059640 1 --2- 172.21.15.84:0/4187053908 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f5a8806b020 0x7f5a8806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.136+0000 7f5aad059640 1 -- 172.21.15.84:0/4187053908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5aa815ebc0 msgr2=0x7f5aa80c6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:02.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.136+0000 7f5aad059640 1 --2- 172.21.15.84:0/4187053908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5aa815ebc0 0x7f5aa80c6cb0 secure :-1 s=READY pgs=3282 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0009e50 tx=0x7f5aa0004a40 comp rx=0 tx=0).stop 2023-10-16T11:27:02.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.136+0000 7f5aad059640 1 -- 172.21.15.84:0/4187053908 shutdown_connections 2023-10-16T11:27:02.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.136+0000 7f5aad059640 1 --2- 172.21.15.84:0/4187053908 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f5a8806b020 0x7f5a8806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.136+0000 7f5aad059640 1 --2- 172.21.15.84:0/4187053908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5aa815ebc0 0x7f5aa80c6cb0 unknown :-1 s=CLOSED pgs=3282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.136+0000 7f5aad059640 1 --2- 172.21.15.84:0/4187053908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa80ad970 0x7f5aa8152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.136+0000 7f5aad059640 1 --2- 172.21.15.84:0/4187053908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5aa806c040 0x7f5aa80c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.136+0000 7f5aad059640 1 -- 172.21.15.84:0/4187053908 >> 172.21.15.84:0/4187053908 conn(0x7f5aa80ce6a0 msgr2=0x7f5aa8069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:02.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.136+0000 7f5aad059640 1 -- 172.21.15.84:0/4187053908 shutdown_connections 2023-10-16T11:27:02.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.136+0000 7f5aad059640 1 -- 172.21.15.84:0/4187053908 wait complete. 2023-10-16T11:27:02.155 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 34, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:26:39.860053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '256', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 35, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:26:40.916519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '257', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:27:02.156 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:27:02.286 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:02.285+0000 7f7879a33040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:27:02.317 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:02.315+0000 7f7a5dc15040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:27:02.340 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:02.339+0000 7f7879a33040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:27:02.369 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:02.367+0000 7f7a5dc15040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:27:02.391 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:02.390+0000 7f7879a33040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:27:02.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.404+0000 7f6667846640 1 Processor -- start 2023-10-16T11:27:02.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.405+0000 7f6667846640 1 -- start start 2023-10-16T11:27:02.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.406+0000 7f6667846640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f66600caf80 0x7f66600ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:02.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.406+0000 7f6667846640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66600c9930 con 0x7f6660065ab0 2023-10-16T11:27:02.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.406+0000 7f6667846640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66600c9c10 con 0x7f66600b4e50 2023-10-16T11:27:02.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.406+0000 7f6667846640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f666015b1e0 con 0x7f66600caf80 2023-10-16T11:27:02.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.406+0000 7f6664de1640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6660065ab0 0x7f66600cb360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35684/0 (socket says 172.21.15.84:35684) 2023-10-16T11:27:02.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.406+0000 7f6664de1640 1 -- 172.21.15.84:0/88334956 learned_addr learned my addr 172.21.15.84:0/88334956 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:02.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.407+0000 7f66655e2640 1 --2- 172.21.15.84:0/88334956 >> v2:172.21.15.157:3300/0 conn(0x7f66600caf80 0x7f66600ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:02.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.407+0000 7f6657fff640 1 -- 172.21.15.84:0/88334956 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217972366 0 0) 0x7f66600c9930 con 0x7f6660065ab0 2023-10-16T11:27:02.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.407+0000 7f6657fff640 1 -- 172.21.15.84:0/88334956 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f664c003520 con 0x7f6660065ab0 2023-10-16T11:27:02.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.407+0000 7f6657fff640 1 -- 172.21.15.84:0/88334956 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1475377263 0 0) 0x7f664c003520 con 0x7f6660065ab0 2023-10-16T11:27:02.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.407+0000 7f6657fff640 1 -- 172.21.15.84:0/88334956 >> v2:172.21.15.157:3300/0 conn(0x7f66600caf80 msgr2=0x7f66600ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:02.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.407+0000 7f6657fff640 1 --2- 172.21.15.84:0/88334956 >> v2:172.21.15.157:3300/0 conn(0x7f66600caf80 0x7f66600ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.407+0000 7f6657fff640 1 -- 172.21.15.84:0/88334956 >> v1:172.21.15.157:6789/0 conn(0x7f66600b4e50 legacy=0x7f6660069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:02.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.407+0000 7f6657fff640 1 -- 172.21.15.84:0/88334956 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66600c9ef0 con 0x7f6660065ab0 2023-10-16T11:27:02.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.407+0000 7f6657fff640 1 -- 172.21.15.84:0/88334956 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f6650001670 con 0x7f6660065ab0 2023-10-16T11:27:02.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.407+0000 7f6657fff640 1 -- 172.21.15.84:0/88334956 >> v1:172.21.15.84:6789/0 conn(0x7f6660065ab0 legacy=0x7f66600cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:02.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.407+0000 7f6657fff640 1 --2- 172.21.15.84:0/88334956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f664c003970 0x7f664c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:02.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.407+0000 7f6657fff640 1 --2- 172.21.15.84:0/88334956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f664c0047c0 0x7f664c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:02.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.407+0000 7f6657fff640 1 --2- 172.21.15.84:0/88334956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f664c005180 0x7f664c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:02.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.407+0000 7f6657fff640 1 -- 172.21.15.84:0/88334956 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c00c0f0 con 0x7f664c005180 2023-10-16T11:27:02.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.407+0000 7f6657fff640 1 -- 172.21.15.84:0/88334956 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c00c260 con 0x7f664c003970 2023-10-16T11:27:02.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.407+0000 7f6657fff640 1 -- 172.21.15.84:0/88334956 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664c00c420 con 0x7f664c0047c0 2023-10-16T11:27:02.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.408+0000 7f6665de3640 1 --2- 172.21.15.84:0/88334956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f664c005180 0x7f664c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:02.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.408+0000 7f6664de1640 1 --2- 172.21.15.84:0/88334956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f664c0047c0 0x7f664c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:02.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.408+0000 7f66655e2640 1 --2- 172.21.15.84:0/88334956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f664c003970 0x7f664c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:02.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.408+0000 7f6665de3640 1 -- 172.21.15.84:0/88334956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f664c0047c0 msgr2=0x7f664c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:02.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.408+0000 7f6665de3640 1 --2- 172.21.15.84:0/88334956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f664c0047c0 0x7f664c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.408+0000 7f6665de3640 1 -- 172.21.15.84:0/88334956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f664c003970 msgr2=0x7f664c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:02.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.408+0000 7f6665de3640 1 --2- 172.21.15.84:0/88334956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f664c003970 0x7f664c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.408+0000 7f6665de3640 1 -- 172.21.15.84:0/88334956 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c003520 con 0x7f664c005180 2023-10-16T11:27:02.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.409+0000 7f6665de3640 1 -- 172.21.15.84:0/88334956 reap_dead start 2023-10-16T11:27:02.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.409+0000 7f6665de3640 1 --2- 172.21.15.84:0/88334956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f664c005180 0x7f664c00b980 secure :-1 s=READY pgs=3284 cs=0 l=1 rev1=1 crypto rx=0x7f6658001e50 tx=0x7f664c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172b794cf1a3d0ad server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:02.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.409+0000 7f6657fff640 1 -- 172.21.15.84:0/88334956 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658003b70 con 0x7f664c005180 2023-10-16T11:27:02.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.409+0000 7f6657fff640 1 -- 172.21.15.84:0/88334956 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6658003d10 con 0x7f664c005180 2023-10-16T11:27:02.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.409+0000 7f6657fff640 1 -- 172.21.15.84:0/88334956 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658005310 con 0x7f664c005180 2023-10-16T11:27:02.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.410+0000 7f6667846640 1 -- 172.21.15.84:0/88334956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f664c005180 msgr2=0x7f664c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:02.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.410+0000 7f6667846640 1 --2- 172.21.15.84:0/88334956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f664c005180 0x7f664c00b980 secure :-1 s=READY pgs=3284 cs=0 l=1 rev1=1 crypto rx=0x7f6658001e50 tx=0x7f664c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.410+0000 7f6667846640 1 -- 172.21.15.84:0/88334956 shutdown_connections 2023-10-16T11:27:02.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.410+0000 7f6667846640 1 --2- 172.21.15.84:0/88334956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f664c005180 0x7f664c00b980 unknown :-1 s=CLOSED pgs=3284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.410+0000 7f6667846640 1 -- 172.21.15.84:0/88334956 >> 172.21.15.84:0/88334956 conn(0x7f666006bc10 msgr2=0x7f666006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:02.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.410+0000 7f6667846640 1 -- 172.21.15.84:0/88334956 shutdown_connections 2023-10-16T11:27:02.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.410+0000 7f6667846640 1 -- 172.21.15.84:0/88334956 wait complete. 2023-10-16T11:27:02.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.412+0000 7f6667846640 1 Processor -- start 2023-10-16T11:27:02.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.412+0000 7f6667846640 1 -- start start 2023-10-16T11:27:02.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.413+0000 7f6667846640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f664c005180 0x7f66600c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:02.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.413+0000 7f6667846640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66600adbb0 0x7f66600ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:02.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.413+0000 7f6667846640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66600ac890 0x7f66600acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:02.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.413+0000 7f6667846640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66600cdbe0 con 0x7f664c005180 2023-10-16T11:27:02.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.413+0000 7f6667846640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66600cde30 con 0x7f66600ac890 2023-10-16T11:27:02.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.413+0000 7f6667846640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66600ce110 con 0x7f66600adbb0 2023-10-16T11:27:02.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.413+0000 7f66655e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f664c005180 0x7f66600c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:02.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.413+0000 7f66655e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f664c005180 0x7f66600c4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42494/0 (socket says 172.21.15.84:42494) 2023-10-16T11:27:02.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.413+0000 7f66655e2640 1 -- 172.21.15.84:0/715994383 learned_addr learned my addr 172.21.15.84:0/715994383 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:02.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.414+0000 7f6664de1640 1 --2- 172.21.15.84:0/715994383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66600adbb0 0x7f66600ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:02.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.414+0000 7f6665de3640 1 --2- 172.21.15.84:0/715994383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66600ac890 0x7f66600acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:02.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.414+0000 7f66655e2640 1 -- 172.21.15.84:0/715994383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66600adbb0 msgr2=0x7f66600ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:02.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.414+0000 7f66655e2640 1 --2- 172.21.15.84:0/715994383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66600adbb0 0x7f66600ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.414+0000 7f66655e2640 1 -- 172.21.15.84:0/715994383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66600ac890 msgr2=0x7f66600acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:02.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.414+0000 7f66655e2640 1 --2- 172.21.15.84:0/715994383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66600ac890 0x7f66600acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.414+0000 7f66655e2640 1 -- 172.21.15.84:0/715994383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66600ce3f0 con 0x7f664c005180 2023-10-16T11:27:02.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.414+0000 7f6665de3640 1 --2- 172.21.15.84:0/715994383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66600ac890 0x7f66600acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:02.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.414+0000 7f66655e2640 1 --2- 172.21.15.84:0/715994383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f664c005180 0x7f66600c4520 secure :-1 s=READY pgs=3285 cs=0 l=1 rev1=1 crypto rx=0x7f665c011540 tx=0x7f665c011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:02.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.414+0000 7f6655ffb640 1 -- 172.21.15.84:0/715994383 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c015930 con 0x7f664c005180 2023-10-16T11:27:02.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.414+0000 7f6655ffb640 1 -- 172.21.15.84:0/715994383 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c015a90 con 0x7f664c005180 2023-10-16T11:27:02.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.414+0000 7f6655ffb640 1 -- 172.21.15.84:0/715994383 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c015e60 con 0x7f664c005180 2023-10-16T11:27:02.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.414+0000 7f6667846640 1 -- 172.21.15.84:0/715994383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66600ce6d0 con 0x7f664c005180 2023-10-16T11:27:02.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.415+0000 7f6667846640 1 -- 172.21.15.84:0/715994383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66600c4a60 con 0x7f664c005180 2023-10-16T11:27:02.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.416+0000 7f6667846640 1 -- 172.21.15.84:0/715994383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f666004e160 con 0x7f664c005180 2023-10-16T11:27:02.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.417+0000 7f6655ffb640 1 -- 172.21.15.84:0/715994383 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 156) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f665c00f460 con 0x7f664c005180 2023-10-16T11:27:02.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.417+0000 7f6655ffb640 1 --2- 172.21.15.84:0/715994383 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f664406b020 0x7f664406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:02.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.418+0000 7f6655ffb640 1 -- 172.21.15.84:0/715994383 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6210+0+0 (secure 0 0 0) 0x7f665c08a420 con 0x7f664c005180 2023-10-16T11:27:02.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.418+0000 7f6664de1640 1 -- 172.21.15.84:0/715994383 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f664406b020 msgr2=0x7f664406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2988856492 2023-10-16T11:27:02.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.418+0000 7f6664de1640 1 --2- 172.21.15.84:0/715994383 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f664406b020 0x7f664406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:02.427 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:02.418+0000 7f7a5dc15040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:27:02.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.422+0000 7f6655ffb640 1 -- 172.21.15.84:0/715994383 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f665c054fb0 con 0x7f664c005180 2023-10-16T11:27:02.456 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:02.455+0000 7f7879a33040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:27:02.481 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:02.479+0000 7f7a5dc15040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:27:02.505 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:02.504+0000 7f7879a33040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:27:02.530 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:02.528+0000 7f7a5dc15040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:27:02.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.614+0000 7f6667846640 1 -- 172.21.15.84:0/715994383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66600c6fe0 con 0x7f664c005180 2023-10-16T11:27:02.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.615+0000 7f6655ffb640 1 -- 172.21.15.84:0/715994383 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v258) v1 ==== 93+0+10 (secure 0 0 0) 0x7f665c0585d0 con 0x7f664c005180 2023-10-16T11:27:02.616 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:27:02.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.618+0000 7f6667846640 1 -- 172.21.15.84:0/715994383 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f664406b020 msgr2=0x7f664406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:02.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.618+0000 7f6667846640 1 --2- 172.21.15.84:0/715994383 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f664406b020 0x7f664406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.620 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:02.618+0000 7f7879a33040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:27:02.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.618+0000 7f6667846640 1 -- 172.21.15.84:0/715994383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f664c005180 msgr2=0x7f66600c4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:02.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.618+0000 7f6667846640 1 --2- 172.21.15.84:0/715994383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f664c005180 0x7f66600c4520 secure :-1 s=READY pgs=3285 cs=0 l=1 rev1=1 crypto rx=0x7f665c011540 tx=0x7f665c011a00 comp rx=0 tx=0).stop 2023-10-16T11:27:02.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.618+0000 7f6667846640 1 -- 172.21.15.84:0/715994383 shutdown_connections 2023-10-16T11:27:02.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.618+0000 7f6667846640 1 --2- 172.21.15.84:0/715994383 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f664406b020 0x7f664406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.618+0000 7f6667846640 1 --2- 172.21.15.84:0/715994383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66600ac890 0x7f66600acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.618+0000 7f6667846640 1 --2- 172.21.15.84:0/715994383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66600adbb0 0x7f66600ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.618+0000 7f6667846640 1 --2- 172.21.15.84:0/715994383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f664c005180 0x7f66600c4520 unknown :-1 s=CLOSED pgs=3285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.618+0000 7f6667846640 1 -- 172.21.15.84:0/715994383 >> 172.21.15.84:0/715994383 conn(0x7f666006bc10 msgr2=0x7f66600b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:02.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.618+0000 7f6667846640 1 -- 172.21.15.84:0/715994383 shutdown_connections 2023-10-16T11:27:02.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.618+0000 7f6667846640 1 -- 172.21.15.84:0/715994383 wait complete. 2023-10-16T11:27:02.636 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:27:02.642 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:02.640+0000 7f7a5dc15040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:27:02.685 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:02.684+0000 7f7879a33040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:27:02.704 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:02.703+0000 7f7a5dc15040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:27:02.737 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:02.736+0000 7f7879a33040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:27:02.755 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:02.753+0000 7f7a5dc15040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:27:02.795 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:02.794+0000 7f7879a33040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:27:02.811 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:02.809+0000 7f7a5dc15040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:27:02.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.837+0000 7f1cab283640 1 Processor -- start 2023-10-16T11:27:02.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.837+0000 7f1cab283640 1 -- start start 2023-10-16T11:27:02.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.838+0000 7f1cab283640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1ca40cf5f0 0x7f1ca406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:02.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.839+0000 7f1cab283640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca4161740 con 0x7f1ca4070c10 2023-10-16T11:27:02.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.839+0000 7f1cab283640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca406b0f0 con 0x7f1ca4070830 2023-10-16T11:27:02.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.839+0000 7f1cab283640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca41619b0 con 0x7f1ca40cf5f0 2023-10-16T11:27:02.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.839+0000 7f1ca9820640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1ca4070c10 0x7f1ca40cd690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35686/0 (socket says 172.21.15.84:35686) 2023-10-16T11:27:02.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.839+0000 7f1ca9820640 1 -- 172.21.15.84:0/4052947597 learned_addr learned my addr 172.21.15.84:0/4052947597 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:02.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.839+0000 7f1ca881e640 1 --2- 172.21.15.84:0/4052947597 >> v2:172.21.15.157:3300/0 conn(0x7f1ca40cf5f0 0x7f1ca406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:02.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.839+0000 7f1c9bfff640 1 -- 172.21.15.84:0/4052947597 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126546259 0 0) 0x7f1ca4161740 con 0x7f1ca4070c10 2023-10-16T11:27:02.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.839+0000 7f1c9bfff640 1 -- 172.21.15.84:0/4052947597 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c90003520 con 0x7f1ca4070c10 2023-10-16T11:27:02.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.840+0000 7f1c9bfff640 1 -- 172.21.15.84:0/4052947597 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315616779 0 0) 0x7f1ca406b0f0 con 0x7f1ca4070830 2023-10-16T11:27:02.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.840+0000 7f1c9bfff640 1 -- 172.21.15.84:0/4052947597 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ca4161740 con 0x7f1ca4070830 2023-10-16T11:27:02.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.840+0000 7f1c9bfff640 1 -- 172.21.15.84:0/4052947597 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2309095015 0 0) 0x7f1c90003520 con 0x7f1ca4070c10 2023-10-16T11:27:02.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.840+0000 7f1c9bfff640 1 -- 172.21.15.84:0/4052947597 >> v2:172.21.15.157:3300/0 conn(0x7f1ca40cf5f0 msgr2=0x7f1ca406d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:02.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.840+0000 7f1c9bfff640 1 --2- 172.21.15.84:0/4052947597 >> v2:172.21.15.157:3300/0 conn(0x7f1ca40cf5f0 0x7f1ca406d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.840+0000 7f1c9bfff640 1 -- 172.21.15.84:0/4052947597 >> v1:172.21.15.157:6789/0 conn(0x7f1ca4070830 legacy=0x7f1ca406c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:02.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.840+0000 7f1c9bfff640 1 -- 172.21.15.84:0/4052947597 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca406b3d0 con 0x7f1ca4070c10 2023-10-16T11:27:02.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.840+0000 7f1c9bfff640 1 -- 172.21.15.84:0/4052947597 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f1c9c001970 con 0x7f1ca4070c10 2023-10-16T11:27:02.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.840+0000 7f1c9bfff640 1 -- 172.21.15.84:0/4052947597 >> v1:172.21.15.84:6789/0 conn(0x7f1ca4070c10 legacy=0x7f1ca40cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:02.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.840+0000 7f1c9bfff640 1 --2- 172.21.15.84:0/4052947597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c90003b50 0x7f1c90003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:02.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.840+0000 7f1c9bfff640 1 --2- 172.21.15.84:0/4052947597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c900047c0 0x7f1c90004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:02.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.840+0000 7f1c9bfff640 1 --2- 172.21.15.84:0/4052947597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c90005240 0x7f1c9000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:02.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.840+0000 7f1c9bfff640 1 -- 172.21.15.84:0/4052947597 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca41619b0 con 0x7f1c900047c0 2023-10-16T11:27:02.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.840+0000 7f1c9bfff640 1 -- 172.21.15.84:0/4052947597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9000c180 con 0x7f1c90005240 2023-10-16T11:27:02.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.840+0000 7f1c9bfff640 1 -- 172.21.15.84:0/4052947597 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9000c340 con 0x7f1c90003b50 2023-10-16T11:27:02.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.840+0000 7f1ca881e640 1 --2- 172.21.15.84:0/4052947597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c900047c0 0x7f1c90004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:02.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.841+0000 7f1ca901f640 1 --2- 172.21.15.84:0/4052947597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c90003b50 0x7f1c90003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:02.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.841+0000 7f1ca9820640 1 --2- 172.21.15.84:0/4052947597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c90005240 0x7f1c9000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:02.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.841+0000 7f1ca881e640 1 -- 172.21.15.84:0/4052947597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c90003b50 msgr2=0x7f1c90003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:02.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.841+0000 7f1ca881e640 1 --2- 172.21.15.84:0/4052947597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c90003b50 0x7f1c90003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.841+0000 7f1ca881e640 1 -- 172.21.15.84:0/4052947597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c90005240 msgr2=0x7f1c9000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:02.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.841+0000 7f1ca881e640 1 --2- 172.21.15.84:0/4052947597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c90005240 0x7f1c9000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.841+0000 7f1ca881e640 1 -- 172.21.15.84:0/4052947597 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c90003520 con 0x7f1c900047c0 2023-10-16T11:27:02.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.841+0000 7f1ca9820640 1 --2- 172.21.15.84:0/4052947597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c90005240 0x7f1c9000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:02.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.841+0000 7f1ca9820640 1 -- 172.21.15.84:0/4052947597 reap_dead start 2023-10-16T11:27:02.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.842+0000 7f1ca881e640 1 --2- 172.21.15.84:0/4052947597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c900047c0 0x7f1c90004c40 secure :-1 s=READY pgs=3287 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009ac0 tx=0x7f1c900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4c5802cb1d7418 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:02.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.843+0000 7f1c9bfff640 1 -- 172.21.15.84:0/4052947597 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9402c6b0 con 0x7f1c900047c0 2023-10-16T11:27:02.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.843+0000 7f1c9bfff640 1 -- 172.21.15.84:0/4052947597 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9402c810 con 0x7f1c900047c0 2023-10-16T11:27:02.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.843+0000 7f1c9bfff640 1 -- 172.21.15.84:0/4052947597 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9402cba0 con 0x7f1c900047c0 2023-10-16T11:27:02.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.843+0000 7f1cab283640 1 -- 172.21.15.84:0/4052947597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c900047c0 msgr2=0x7f1c90004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:02.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.843+0000 7f1cab283640 1 --2- 172.21.15.84:0/4052947597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c900047c0 0x7f1c90004c40 secure :-1 s=READY pgs=3287 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009ac0 tx=0x7f1c900057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.843+0000 7f1cab283640 1 -- 172.21.15.84:0/4052947597 shutdown_connections 2023-10-16T11:27:02.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.843+0000 7f1cab283640 1 --2- 172.21.15.84:0/4052947597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c900047c0 0x7f1c90004c40 unknown :-1 s=CLOSED pgs=3287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.843+0000 7f1cab283640 1 -- 172.21.15.84:0/4052947597 >> 172.21.15.84:0/4052947597 conn(0x7f1ca40cfac0 msgr2=0x7f1ca40cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:02.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.843+0000 7f1cab283640 1 -- 172.21.15.84:0/4052947597 shutdown_connections 2023-10-16T11:27:02.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.843+0000 7f1cab283640 1 -- 172.21.15.84:0/4052947597 wait complete. 2023-10-16T11:27:02.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.845+0000 7f1cab283640 1 Processor -- start 2023-10-16T11:27:02.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.845+0000 7f1cab283640 1 -- start start 2023-10-16T11:27:02.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.846+0000 7f1cab283640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c900047c0 0x7f1ca41f3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:02.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.846+0000 7f1cab283640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ca40b4e40 0x7f1ca41f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:02.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.846+0000 7f1cab283640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca415be40 0x7f1ca415c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:02.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.846+0000 7f1cab283640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca415c830 con 0x7f1ca40b4e40 2023-10-16T11:27:02.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.846+0000 7f1cab283640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca41f3aa0 con 0x7f1ca415be40 2023-10-16T11:27:02.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.846+0000 7f1cab283640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca415d2b0 con 0x7f1c900047c0 2023-10-16T11:27:02.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.846+0000 7f1ca881e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ca40b4e40 0x7f1ca41f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:02.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.846+0000 7f1ca881e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ca40b4e40 0x7f1ca41f3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42520/0 (socket says 172.21.15.84:42520) 2023-10-16T11:27:02.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.846+0000 7f1ca881e640 1 -- 172.21.15.84:0/3969300109 learned_addr learned my addr 172.21.15.84:0/3969300109 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:02.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.847+0000 7f1ca901f640 1 --2- 172.21.15.84:0/3969300109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c900047c0 0x7f1ca41f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:02.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.847+0000 7f1ca9820640 1 --2- 172.21.15.84:0/3969300109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca415be40 0x7f1ca415c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:02.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.847+0000 7f1ca881e640 1 -- 172.21.15.84:0/3969300109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c900047c0 msgr2=0x7f1ca41f3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:02.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.847+0000 7f1ca881e640 1 --2- 172.21.15.84:0/3969300109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c900047c0 0x7f1ca41f3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.847+0000 7f1ca881e640 1 -- 172.21.15.84:0/3969300109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca415be40 msgr2=0x7f1ca415c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:02.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.847+0000 7f1ca881e640 1 --2- 172.21.15.84:0/3969300109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca415be40 0x7f1ca415c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:02.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.847+0000 7f1ca881e640 1 -- 172.21.15.84:0/3969300109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca415d590 con 0x7f1ca40b4e40 2023-10-16T11:27:02.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.847+0000 7f1ca9820640 1 --2- 172.21.15.84:0/3969300109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca415be40 0x7f1ca415c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:02.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.847+0000 7f1ca881e640 1 --2- 172.21.15.84:0/3969300109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ca40b4e40 0x7f1ca41f3ca0 secure :-1 s=READY pgs=3288 cs=0 l=1 rev1=1 crypto rx=0x7f1c9402c250 tx=0x7f1c94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:02.857 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:02.847+0000 7f7879a33040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:27:02.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.847+0000 7f1c99ffb640 1 -- 172.21.15.84:0/3969300109 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94003810 con 0x7f1ca40b4e40 2023-10-16T11:27:02.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.848+0000 7f1c99ffb640 1 -- 172.21.15.84:0/3969300109 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c94003970 con 0x7f1ca40b4e40 2023-10-16T11:27:02.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.848+0000 7f1c99ffb640 1 -- 172.21.15.84:0/3969300109 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94003d00 con 0x7f1ca40b4e40 2023-10-16T11:27:02.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.848+0000 7f1cab283640 1 -- 172.21.15.84:0/3969300109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca415d870 con 0x7f1ca40b4e40 2023-10-16T11:27:02.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.848+0000 7f1cab283640 1 -- 172.21.15.84:0/3969300109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca415de30 con 0x7f1ca40b4e40 2023-10-16T11:27:02.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.849+0000 7f1cab283640 1 -- 172.21.15.84:0/3969300109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca415dfc0 con 0x7f1ca40b4e40 2023-10-16T11:27:02.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.850+0000 7f1c99ffb640 1 -- 172.21.15.84:0/3969300109 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 156) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f1c94034440 con 0x7f1ca40b4e40 2023-10-16T11:27:02.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.851+0000 7f1c99ffb640 1 --2- 172.21.15.84:0/3969300109 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f1c8806b020 0x7f1c8806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:02.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.851+0000 7f1c99ffb640 1 -- 172.21.15.84:0/3969300109 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6210+0+0 (secure 0 0 0) 0x7f1c940abdd0 con 0x7f1ca40b4e40 2023-10-16T11:27:02.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.851+0000 7f1ca901f640 1 -- 172.21.15.84:0/3969300109 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f1c8806b020 msgr2=0x7f1c8806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2988856492 2023-10-16T11:27:02.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.851+0000 7f1ca901f640 1 --2- 172.21.15.84:0/3969300109 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f1c8806b020 0x7f1c8806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:02.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:02.855+0000 7f1c99ffb640 1 -- 172.21.15.84:0/3969300109 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1c94076a10 con 0x7f1ca40b4e40 2023-10-16T11:27:02.862 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:02.861+0000 7f7a5dc15040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:27:03.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.048+0000 7f1cab283640 1 -- 172.21.15.84:0/3969300109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ca404e160 con 0x7f1ca40b4e40 2023-10-16T11:27:03.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.049+0000 7f1c99ffb640 1 -- 172.21.15.84:0/3969300109 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v258) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1c9404c020 con 0x7f1ca40b4e40 2023-10-16T11:27:03.050 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:27:03.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.052+0000 7f1ca901f640 1 -- 172.21.15.84:0/3969300109 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f1c8806b020 msgr2=0x7f1c8806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2988856492 2023-10-16T11:27:03.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.052+0000 7f1ca901f640 1 --2- 172.21.15.84:0/3969300109 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f1c8806b020 0x7f1c8806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:27:03.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.053+0000 7f1cab283640 1 -- 172.21.15.84:0/3969300109 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f1c8806b020 msgr2=0x7f1c8806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:03.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.053+0000 7f1cab283640 1 --2- 172.21.15.84:0/3969300109 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f1c8806b020 0x7f1c8806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.053+0000 7f1cab283640 1 -- 172.21.15.84:0/3969300109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ca40b4e40 msgr2=0x7f1ca41f3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:03.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.053+0000 7f1cab283640 1 --2- 172.21.15.84:0/3969300109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ca40b4e40 0x7f1ca41f3ca0 secure :-1 s=READY pgs=3288 cs=0 l=1 rev1=1 crypto rx=0x7f1c9402c250 tx=0x7f1c94004730 comp rx=0 tx=0).stop 2023-10-16T11:27:03.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.053+0000 7f1cab283640 1 -- 172.21.15.84:0/3969300109 shutdown_connections 2023-10-16T11:27:03.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.053+0000 7f1cab283640 1 --2- 172.21.15.84:0/3969300109 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f1c8806b020 0x7f1c8806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.053+0000 7f1cab283640 1 --2- 172.21.15.84:0/3969300109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca415be40 0x7f1ca415c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.053+0000 7f1cab283640 1 --2- 172.21.15.84:0/3969300109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ca40b4e40 0x7f1ca41f3ca0 unknown :-1 s=CLOSED pgs=3288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.053+0000 7f1cab283640 1 --2- 172.21.15.84:0/3969300109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c900047c0 0x7f1ca41f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.053+0000 7f1cab283640 1 -- 172.21.15.84:0/3969300109 >> 172.21.15.84:0/3969300109 conn(0x7f1ca40cfac0 msgr2=0x7f1ca40ad260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:03.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.053+0000 7f1cab283640 1 -- 172.21.15.84:0/3969300109 shutdown_connections 2023-10-16T11:27:03.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.053+0000 7f1cab283640 1 -- 172.21.15.84:0/3969300109 wait complete. 2023-10-16T11:27:03.080 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:27:03.243 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:03.241+0000 7f7a5dc15040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:27:03.247 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:03.246+0000 7f7879a33040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:27:03.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.289+0000 7f3ce9a15640 1 Processor -- start 2023-10-16T11:27:03.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.289+0000 7f3ce9a15640 1 -- start start 2023-10-16T11:27:03.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.290+0000 7f3ce9a15640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3ce4159ab0 0x7f3ce40ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:03.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.290+0000 7f3ce9a15640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ce406c140 0x7f3ce406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:03.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.290+0000 7f3ce9a15640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce4070360 con 0x7f3ce4159e90 2023-10-16T11:27:03.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.290+0000 7f3ce9a15640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce406cb10 con 0x7f3ce4159ab0 2023-10-16T11:27:03.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.290+0000 7f3ce9a15640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce415d890 con 0x7f3ce406c140 2023-10-16T11:27:03.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.291+0000 7f3ce2ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3ce4159ab0 0x7f3ce40ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:03.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.291+0000 7f3ce27fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3ce4159e90 0x7f3ce406fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:54410/0 (socket says 172.21.15.84:54410) 2023-10-16T11:27:03.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.291+0000 7f3ce27fc640 1 -- 172.21.15.84:0/1216251847 learned_addr learned my addr 172.21.15.84:0/1216251847 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:03.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.291+0000 7f3ce37fe640 1 --2- 172.21.15.84:0/1216251847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ce406c140 0x7f3ce406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:03.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.292+0000 7f3ce2ffd640 1 -- 172.21.15.84:0/1216251847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ce406c140 msgr2=0x7f3ce406c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:03.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.292+0000 7f3ce2ffd640 1 --2- 172.21.15.84:0/1216251847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ce406c140 0x7f3ce406c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.292+0000 7f3ce2ffd640 1 -- 172.21.15.84:0/1216251847 >> v1:172.21.15.157:6789/0 conn(0x7f3ce4159e90 legacy=0x7f3ce406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:03.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.292+0000 7f3ce2ffd640 1 -- 172.21.15.84:0/1216251847 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce40cd1f0 con 0x7f3ce4159ab0 2023-10-16T11:27:03.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.293+0000 7f3ce2ffd640 1 --2- 172.21.15.84:0/1216251847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce4159ab0 0x7f3ce40ce370 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40097a0 tx=0x7f3cd402d9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=743847d16506b684 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:03.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.293+0000 7f3ce1ffb640 1 -- 172.21.15.84:0/1216251847 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4032870 con 0x7f3ce4159ab0 2023-10-16T11:27:03.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.293+0000 7f3ce1ffb640 1 -- 172.21.15.84:0/1216251847 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd40329d0 con 0x7f3ce4159ab0 2023-10-16T11:27:03.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.293+0000 7f3ce1ffb640 1 -- 172.21.15.84:0/1216251847 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4032d60 con 0x7f3ce4159ab0 2023-10-16T11:27:03.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.294+0000 7f3ce9a15640 1 -- 172.21.15.84:0/1216251847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce4159ab0 msgr2=0x7f3ce40ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:03.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.294+0000 7f3ce9a15640 1 --2- 172.21.15.84:0/1216251847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce4159ab0 0x7f3ce40ce370 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40097a0 tx=0x7f3cd402d9b0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.294+0000 7f3ce9a15640 1 -- 172.21.15.84:0/1216251847 shutdown_connections 2023-10-16T11:27:03.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.294+0000 7f3ce9a15640 1 --2- 172.21.15.84:0/1216251847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ce406c140 0x7f3ce406c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.294+0000 7f3ce9a15640 1 --2- 172.21.15.84:0/1216251847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce4159ab0 0x7f3ce40ce370 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.294+0000 7f3ce9a15640 1 -- 172.21.15.84:0/1216251847 >> 172.21.15.84:0/1216251847 conn(0x7f3ce40ce9b0 msgr2=0x7f3ce40cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:03.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.294+0000 7f3ce9a15640 1 -- 172.21.15.84:0/1216251847 shutdown_connections 2023-10-16T11:27:03.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.294+0000 7f3ce9a15640 1 -- 172.21.15.84:0/1216251847 wait complete. 2023-10-16T11:27:03.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.296+0000 7f3ce9a15640 1 Processor -- start 2023-10-16T11:27:03.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.296+0000 7f3ce9a15640 1 -- start start 2023-10-16T11:27:03.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.297+0000 7f3ce9a15640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce406c140 0x7f3ce406b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:03.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.297+0000 7f3ce9a15640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ce4159ab0 0x7f3ce41552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:03.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.297+0000 7f3ce9a15640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ce4159e90 0x7f3ce40b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:03.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.297+0000 7f3ce9a15640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce406b0c0 con 0x7f3ce4159ab0 2023-10-16T11:27:03.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.297+0000 7f3ce9a15640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce40acd90 con 0x7f3ce406c140 2023-10-16T11:27:03.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.297+0000 7f3ce9a15640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce40ad000 con 0x7f3ce4159e90 2023-10-16T11:27:03.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.297+0000 7f3ce27fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ce4159ab0 0x7f3ce41552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:03.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.297+0000 7f3ce27fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ce4159ab0 0x7f3ce41552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42532/0 (socket says 172.21.15.84:42532) 2023-10-16T11:27:03.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.297+0000 7f3ce27fc640 1 -- 172.21.15.84:0/1987145237 learned_addr learned my addr 172.21.15.84:0/1987145237 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:03.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.297+0000 7f3ce2ffd640 1 --2- 172.21.15.84:0/1987145237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce406c140 0x7f3ce406b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:03.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.297+0000 7f3ce37fe640 1 --2- 172.21.15.84:0/1987145237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ce4159e90 0x7f3ce40b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:03.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.298+0000 7f3ce27fc640 1 -- 172.21.15.84:0/1987145237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ce4159e90 msgr2=0x7f3ce40b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:03.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.298+0000 7f3ce27fc640 1 --2- 172.21.15.84:0/1987145237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ce4159e90 0x7f3ce40b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.298+0000 7f3ce27fc640 1 -- 172.21.15.84:0/1987145237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce406c140 msgr2=0x7f3ce406b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:03.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.298+0000 7f3ce27fc640 1 --2- 172.21.15.84:0/1987145237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce406c140 0x7f3ce406b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.298+0000 7f3ce27fc640 1 -- 172.21.15.84:0/1987145237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd4009400 con 0x7f3ce4159ab0 2023-10-16T11:27:03.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.298+0000 7f3ce2ffd640 1 --2- 172.21.15.84:0/1987145237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce406c140 0x7f3ce406b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:27:03.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.298+0000 7f3ce27fc640 1 --2- 172.21.15.84:0/1987145237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ce4159ab0 0x7f3ce41552d0 secure :-1 s=READY pgs=3289 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800c4a0 tx=0x7f3cd800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:03.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.298+0000 7f3ccbfff640 1 -- 172.21.15.84:0/1987145237 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8005ce0 con 0x7f3ce4159ab0 2023-10-16T11:27:03.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.298+0000 7f3ccbfff640 1 -- 172.21.15.84:0/1987145237 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd8005e40 con 0x7f3ce4159ab0 2023-10-16T11:27:03.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.298+0000 7f3ccbfff640 1 -- 172.21.15.84:0/1987145237 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8007270 con 0x7f3ce4159ab0 2023-10-16T11:27:03.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.298+0000 7f3ce9a15640 1 -- 172.21.15.84:0/1987145237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce40ad2e0 con 0x7f3ce4159ab0 2023-10-16T11:27:03.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.298+0000 7f3ce9a15640 1 -- 172.21.15.84:0/1987145237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce4155810 con 0x7f3ce4159ab0 2023-10-16T11:27:03.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.299+0000 7f3ce9a15640 1 -- 172.21.15.84:0/1987145237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce404e160 con 0x7f3ce4159ab0 2023-10-16T11:27:03.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.300+0000 7f3ccbfff640 1 -- 172.21.15.84:0/1987145237 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 156) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f3cd8007490 con 0x7f3ce4159ab0 2023-10-16T11:27:03.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.301+0000 7f3ccbfff640 1 --2- 172.21.15.84:0/1987145237 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f3cd006b020 0x7f3cd006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:03.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.301+0000 7f3ccbfff640 1 -- 172.21.15.84:0/1987145237 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6210+0+0 (secure 0 0 0) 0x7f3cd8088fe0 con 0x7f3ce4159ab0 2023-10-16T11:27:03.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.301+0000 7f3ce2ffd640 1 -- 172.21.15.84:0/1987145237 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f3cd006b020 msgr2=0x7f3cd006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2988856492 2023-10-16T11:27:03.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.301+0000 7f3ce2ffd640 1 --2- 172.21.15.84:0/1987145237 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f3cd006b020 0x7f3cd006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:03.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.305+0000 7f3ccbfff640 1 -- 172.21.15.84:0/1987145237 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3cd8053c20 con 0x7f3ce4159ab0 2023-10-16T11:27:03.404 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:03.402+0000 7f7a5dc15040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:27:03.412 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:03.411+0000 7f7879a33040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:27:03.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.496+0000 7f3ce9a15640 1 -- 172.21.15.84:0/1987145237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ce40cda60 con 0x7f3ce4159ab0 2023-10-16T11:27:03.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.497+0000 7f3ccbfff640 1 -- 172.21.15.84:0/1987145237 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v258) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3cd8017020 con 0x7f3ce4159ab0 2023-10-16T11:27:03.498 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:27:03.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.499+0000 7f3ce9a15640 1 -- 172.21.15.84:0/1987145237 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f3cd006b020 msgr2=0x7f3cd006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:03.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.499+0000 7f3ce9a15640 1 --2- 172.21.15.84:0/1987145237 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f3cd006b020 0x7f3cd006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.499+0000 7f3ce9a15640 1 -- 172.21.15.84:0/1987145237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ce4159ab0 msgr2=0x7f3ce41552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:03.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.499+0000 7f3ce9a15640 1 --2- 172.21.15.84:0/1987145237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ce4159ab0 0x7f3ce41552d0 secure :-1 s=READY pgs=3289 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800c4a0 tx=0x7f3cd800c960 comp rx=0 tx=0).stop 2023-10-16T11:27:03.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.499+0000 7f3ce9a15640 1 -- 172.21.15.84:0/1987145237 shutdown_connections 2023-10-16T11:27:03.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.499+0000 7f3ce9a15640 1 --2- 172.21.15.84:0/1987145237 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f3cd006b020 0x7f3cd006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.499+0000 7f3ce9a15640 1 --2- 172.21.15.84:0/1987145237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ce4159e90 0x7f3ce40b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.499+0000 7f3ce9a15640 1 --2- 172.21.15.84:0/1987145237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ce4159ab0 0x7f3ce41552d0 unknown :-1 s=CLOSED pgs=3289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.499+0000 7f3ce9a15640 1 --2- 172.21.15.84:0/1987145237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce406c140 0x7f3ce406b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.499+0000 7f3ce9a15640 1 -- 172.21.15.84:0/1987145237 >> 172.21.15.84:0/1987145237 conn(0x7f3ce40ce9b0 msgr2=0x7f3ce40b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:03.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.500+0000 7f3ce9a15640 1 -- 172.21.15.84:0/1987145237 shutdown_connections 2023-10-16T11:27:03.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.500+0000 7f3ce9a15640 1 -- 172.21.15.84:0/1987145237 wait complete. 2023-10-16T11:27:03.512 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:03.510+0000 7f7a5dc15040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:27:03.519 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T11:27:03.521 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:03.520+0000 7f7879a33040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:27:03.565 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:03.563+0000 7f7a5dc15040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:27:03.575 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:03.574+0000 7f7879a33040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:27:03.673 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:03.671+0000 7f7a5dc15040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:27:03.687 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:03.686+0000 7f7879a33040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:27:03.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.713+0000 7fe32feec640 1 Processor -- start 2023-10-16T11:27:03.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.714+0000 7fe32feec640 1 -- start start 2023-10-16T11:27:03.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.715+0000 7fe32feec640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe32806a4b0 0x7fe3280ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:03.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.715+0000 7fe32feec640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe3280c9ca0 0x7fe3280ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:03.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.715+0000 7fe32feec640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3280cb030 con 0x7fe32815ad70 2023-10-16T11:27:03.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.715+0000 7fe32feec640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3280cae30 con 0x7fe3280c9ca0 2023-10-16T11:27:03.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.715+0000 7fe32feec640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3280c91c0 con 0x7fe32806a4b0 2023-10-16T11:27:03.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.716+0000 7fe32e489640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe3280c9ca0 0x7fe3280ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:03.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.716+0000 7fe32e489640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe3280c9ca0 0x7fe3280ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42542/0 (socket says 172.21.15.84:42542) 2023-10-16T11:27:03.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.716+0000 7fe32e489640 1 -- 172.21.15.84:0/2404848605 learned_addr learned my addr 172.21.15.84:0/2404848605 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:03.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.716+0000 7fe32d487640 1 --2- 172.21.15.84:0/2404848605 >> v2:172.21.15.157:3300/0 conn(0x7fe32806a4b0 0x7fe3280ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:03.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.716+0000 7fe32e489640 1 -- 172.21.15.84:0/2404848605 >> v2:172.21.15.157:3300/0 conn(0x7fe32806a4b0 msgr2=0x7fe3280ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:03.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.716+0000 7fe32e489640 1 --2- 172.21.15.84:0/2404848605 >> v2:172.21.15.157:3300/0 conn(0x7fe32806a4b0 0x7fe3280ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.716+0000 7fe32e489640 1 -- 172.21.15.84:0/2404848605 >> v1:172.21.15.157:6789/0 conn(0x7fe32815ad70 legacy=0x7fe328069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:03.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.716+0000 7fe32e489640 1 -- 172.21.15.84:0/2404848605 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3280c94a0 con 0x7fe3280c9ca0 2023-10-16T11:27:03.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.717+0000 7fe32e489640 1 --2- 172.21.15.84:0/2404848605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3280c9ca0 0x7fe3280ca8f0 secure :-1 s=READY pgs=3290 cs=0 l=1 rev1=1 crypto rx=0x7fe3280ce870 tx=0x7fe32002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce59b6258e273b7e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:03.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.717+0000 7fe32cc86640 1 -- 172.21.15.84:0/2404848605 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32002ca80 con 0x7fe3280c9ca0 2023-10-16T11:27:03.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.717+0000 7fe32cc86640 1 -- 172.21.15.84:0/2404848605 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32002cbe0 con 0x7fe3280c9ca0 2023-10-16T11:27:03.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.717+0000 7fe32cc86640 1 -- 172.21.15.84:0/2404848605 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3200343f0 con 0x7fe3280c9ca0 2023-10-16T11:27:03.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.718+0000 7fe32feec640 1 -- 172.21.15.84:0/2404848605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3280c9ca0 msgr2=0x7fe3280ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:03.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.718+0000 7fe32feec640 1 --2- 172.21.15.84:0/2404848605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3280c9ca0 0x7fe3280ca8f0 secure :-1 s=READY pgs=3290 cs=0 l=1 rev1=1 crypto rx=0x7fe3280ce870 tx=0x7fe32002bee0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.718+0000 7fe32feec640 1 -- 172.21.15.84:0/2404848605 shutdown_connections 2023-10-16T11:27:03.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.718+0000 7fe32feec640 1 --2- 172.21.15.84:0/2404848605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3280c9ca0 0x7fe3280ca8f0 unknown :-1 s=CLOSED pgs=3290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.718+0000 7fe32feec640 1 --2- 172.21.15.84:0/2404848605 >> v2:172.21.15.157:3300/0 conn(0x7fe32806a4b0 0x7fe3280ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.718+0000 7fe32feec640 1 -- 172.21.15.84:0/2404848605 >> 172.21.15.84:0/2404848605 conn(0x7fe3280ce9b0 msgr2=0x7fe3280cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:03.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.718+0000 7fe32feec640 1 -- 172.21.15.84:0/2404848605 shutdown_connections 2023-10-16T11:27:03.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.718+0000 7fe32feec640 1 -- 172.21.15.84:0/2404848605 wait complete. 2023-10-16T11:27:03.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.720+0000 7fe32feec640 1 Processor -- start 2023-10-16T11:27:03.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.720+0000 7fe32feec640 1 -- start start 2023-10-16T11:27:03.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.721+0000 7fe32feec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe32806a4b0 0x7fe3280c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:03.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.721+0000 7fe32feec640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3280c9ca0 0x7fe3280c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:03.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.721+0000 7fe32feec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe32815ad70 0x7fe3280c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:03.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.721+0000 7fe32feec640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3280c69d0 con 0x7fe32815ad70 2023-10-16T11:27:03.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.721+0000 7fe32feec640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3281f03f0 con 0x7fe3280c9ca0 2023-10-16T11:27:03.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.721+0000 7fe32feec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3280c6c00 con 0x7fe32806a4b0 2023-10-16T11:27:03.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.721+0000 7fe32e489640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe32815ad70 0x7fe3280c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:03.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.721+0000 7fe32e489640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe32815ad70 0x7fe3280c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42546/0 (socket says 172.21.15.84:42546) 2023-10-16T11:27:03.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.721+0000 7fe32e489640 1 -- 172.21.15.84:0/1502471164 learned_addr learned my addr 172.21.15.84:0/1502471164 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:03.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.722+0000 7fe32dc88640 1 --2- 172.21.15.84:0/1502471164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe32806a4b0 0x7fe3280c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:03.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.722+0000 7fe32e489640 1 -- 172.21.15.84:0/1502471164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe32806a4b0 msgr2=0x7fe3280c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:03.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.722+0000 7fe32e489640 1 --2- 172.21.15.84:0/1502471164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe32806a4b0 0x7fe3280c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.722+0000 7fe32e489640 1 -- 172.21.15.84:0/1502471164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3280c9ca0 msgr2=0x7fe3280c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:03.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.722+0000 7fe32e489640 1 --2- 172.21.15.84:0/1502471164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3280c9ca0 0x7fe3280c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.722+0000 7fe32e489640 1 -- 172.21.15.84:0/1502471164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe320009b30 con 0x7fe32815ad70 2023-10-16T11:27:03.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.722+0000 7fe32e489640 1 --2- 172.21.15.84:0/1502471164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe32815ad70 0x7fe3280c8360 secure :-1 s=READY pgs=3291 cs=0 l=1 rev1=1 crypto rx=0x7fe320036040 tx=0x7fe320004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:03.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.722+0000 7fe31e7fc640 1 -- 172.21.15.84:0/1502471164 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe320009510 con 0x7fe32815ad70 2023-10-16T11:27:03.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.722+0000 7fe31e7fc640 1 -- 172.21.15.84:0/1502471164 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32003e450 con 0x7fe32815ad70 2023-10-16T11:27:03.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.722+0000 7fe32feec640 1 -- 172.21.15.84:0/1502471164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3280c6ee0 con 0x7fe32815ad70 2023-10-16T11:27:03.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.722+0000 7fe31e7fc640 1 -- 172.21.15.84:0/1502471164 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32003e7e0 con 0x7fe32815ad70 2023-10-16T11:27:03.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.722+0000 7fe32feec640 1 -- 172.21.15.84:0/1502471164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3280c74a0 con 0x7fe32815ad70 2023-10-16T11:27:03.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.724+0000 7fe32feec640 1 -- 172.21.15.84:0/1502471164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3281f05f0 con 0x7fe32815ad70 2023-10-16T11:27:03.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.724+0000 7fe31e7fc640 1 -- 172.21.15.84:0/1502471164 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 156) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fe32002cd50 con 0x7fe32815ad70 2023-10-16T11:27:03.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.725+0000 7fe31e7fc640 1 --2- 172.21.15.84:0/1502471164 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7fe30c06b020 0x7fe30c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:03.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.725+0000 7fe31e7fc640 1 -- 172.21.15.84:0/1502471164 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6210+0+0 (secure 0 0 0) 0x7fe3200ac630 con 0x7fe32815ad70 2023-10-16T11:27:03.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.725+0000 7fe32dc88640 1 -- 172.21.15.84:0/1502471164 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7fe30c06b020 msgr2=0x7fe30c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2988856492 2023-10-16T11:27:03.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.725+0000 7fe32dc88640 1 --2- 172.21.15.84:0/1502471164 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7fe30c06b020 0x7fe30c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:03.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.730+0000 7fe31e7fc640 1 -- 172.21.15.84:0/1502471164 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe3281f05f0 con 0x7fe32815ad70 2023-10-16T11:27:03.783 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:03.781+0000 7f7a5dc15040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:27:03.800 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:03.799+0000 7f7879a33040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:27:03.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.926+0000 7fe32dc88640 1 -- 172.21.15.84:0/1502471164 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7fe30c06b020 msgr2=0x7fe30c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2988856492 2023-10-16T11:27:03.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.926+0000 7fe32dc88640 1 --2- 172.21.15.84:0/1502471164 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7fe30c06b020 0x7fe30c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:27:03.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.934+0000 7fe32feec640 1 -- 172.21.15.84:0/1502471164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe3281f05b0 con 0x7fe32815ad70 2023-10-16T11:27:03.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.935+0000 7fe31e7fc640 1 -- 172.21.15.84:0/1502471164 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v258) v1 ==== 81+0+641 (secure 0 0 0) 0x7fe320077380 con 0x7fe32815ad70 2023-10-16T11:27:03.936 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/907959269 2023-10-17T11:26:57.151307+0000 2023-10-16T11:27:03.936 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1863310536 2023-10-17T11:26:57.151307+0000 2023-10-16T11:27:03.936 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/847549599 2023-10-17T11:26:57.151307+0000 2023-10-16T11:27:03.937 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1934386371 2023-10-17T11:26:57.151307+0000 2023-10-16T11:27:03.937 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3540589884 2023-10-17T11:26:57.151307+0000 2023-10-16T11:27:03.937 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2370234804 2023-10-17T11:26:57.151307+0000 2023-10-16T11:27:03.937 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6833/2330196255 2023-10-17T11:26:57.151307+0000 2023-10-16T11:27:03.937 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1650008535 2023-10-17T11:26:57.151307+0000 2023-10-16T11:27:03.938 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6832/2330196255 2023-10-17T11:26:57.151307+0000 2023-10-16T11:27:03.938 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2440426242 2023-10-17T11:26:57.151307+0000 2023-10-16T11:27:03.938 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/570042115 2023-10-16T12:26:40.168744+0000 2023-10-16T11:27:03.939 INFO:teuthology.orchestra.run.smithi084.stderr:listed 11 entries 2023-10-16T11:27:03.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.937+0000 7fe32feec640 1 -- 172.21.15.84:0/1502471164 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7fe30c06b020 msgr2=0x7fe30c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:03.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.937+0000 7fe32feec640 1 --2- 172.21.15.84:0/1502471164 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7fe30c06b020 0x7fe30c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.937+0000 7fe32feec640 1 -- 172.21.15.84:0/1502471164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe32815ad70 msgr2=0x7fe3280c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:03.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.937+0000 7fe32feec640 1 --2- 172.21.15.84:0/1502471164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe32815ad70 0x7fe3280c8360 secure :-1 s=READY pgs=3291 cs=0 l=1 rev1=1 crypto rx=0x7fe320036040 tx=0x7fe320004a40 comp rx=0 tx=0).stop 2023-10-16T11:27:03.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.937+0000 7fe32feec640 1 -- 172.21.15.84:0/1502471164 shutdown_connections 2023-10-16T11:27:03.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.937+0000 7fe32feec640 1 --2- 172.21.15.84:0/1502471164 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7fe30c06b020 0x7fe30c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.937+0000 7fe32feec640 1 --2- 172.21.15.84:0/1502471164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe32815ad70 0x7fe3280c8360 unknown :-1 s=CLOSED pgs=3291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.937+0000 7fe32feec640 1 --2- 172.21.15.84:0/1502471164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3280c9ca0 0x7fe3280c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.937+0000 7fe32feec640 1 --2- 172.21.15.84:0/1502471164 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe32806a4b0 0x7fe3280c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:03.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.937+0000 7fe32feec640 1 -- 172.21.15.84:0/1502471164 >> 172.21.15.84:0/1502471164 conn(0x7fe3280ce9b0 msgr2=0x7fe32815d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:03.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.938+0000 7fe32feec640 1 -- 172.21.15.84:0/1502471164 shutdown_connections 2023-10-16T11:27:03.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:03.938+0000 7fe32feec640 1 -- 172.21.15.84:0/1502471164 wait complete. 2023-10-16T11:27:03.942 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:03.939+0000 7f7a5dc15040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:27:03.956 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T11:27:03.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:03.957 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T11:27:03.963 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:03.961+0000 7f7879a33040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:27:03.990 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:03.988+0000 7f7a5dc15040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:27:04.015 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:04.014+0000 7f7879a33040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:27:04.051 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T11:27:04.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:04.053 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T11:27:04.098 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:27:04.099 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:04.112 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:04.110+0000 7f7a5dc15040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:27:04.142 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:04.141+0000 7f7879a33040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:27:04.154 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:27:04.155 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:27:04.155 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:04.207 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:27:04.208 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:27:04.208 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:04.264 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:27:04.265 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:27:04.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.473+0000 7f997afc7640 1 Processor -- start 2023-10-16T11:27:04.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.474+0000 7f997afc7640 1 -- start start 2023-10-16T11:27:04.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.475+0000 7f997afc7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f997415ae80 0x7f997415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:04.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.475+0000 7f997afc7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f99740ce390 0x7f9974070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:04.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.475+0000 7f997afc7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997415ed60 con 0x7f997415aaa0 2023-10-16T11:27:04.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.475+0000 7f997afc7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997406b810 con 0x7f997415ae80 2023-10-16T11:27:04.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.475+0000 7f997afc7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99740707e0 con 0x7f99740ce390 2023-10-16T11:27:04.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.475+0000 7f9973fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f997415ae80 0x7f997415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:04.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.475+0000 7f9973fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f997415ae80 0x7f997415e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42562/0 (socket says 172.21.15.84:42562) 2023-10-16T11:27:04.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.475+0000 7f9973fff640 1 -- 172.21.15.84:0/419289927 learned_addr learned my addr 172.21.15.84:0/419289927 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:04.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.476+0000 7f9979564640 1 --2- 172.21.15.84:0/419289927 >> v2:172.21.15.157:3300/0 conn(0x7f99740ce390 0x7f9974070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:04.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.476+0000 7f9973fff640 1 -- 172.21.15.84:0/419289927 >> v2:172.21.15.157:3300/0 conn(0x7f99740ce390 msgr2=0x7f9974070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:04.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.476+0000 7f9973fff640 1 --2- 172.21.15.84:0/419289927 >> v2:172.21.15.157:3300/0 conn(0x7f99740ce390 0x7f9974070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:04.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.476+0000 7f9973fff640 1 -- 172.21.15.84:0/419289927 >> v1:172.21.15.84:6789/0 conn(0x7f997415aaa0 legacy=0x7f99740cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:04.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.476+0000 7f9973fff640 1 -- 172.21.15.84:0/419289927 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997415ef90 con 0x7f997415ae80 2023-10-16T11:27:04.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.476+0000 7f99737fe640 1 -- 172.21.15.84:0/419289927 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811505657 0 0) 0x7f997415ed60 con 0x7f997415aaa0 2023-10-16T11:27:04.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.476+0000 7f9979564640 1 --2- 172.21.15.84:0/419289927 >> v2:172.21.15.157:3300/0 conn(0x7f99740ce390 0x7f9974070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:04.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.477+0000 7f9973fff640 1 --2- 172.21.15.84:0/419289927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997415ae80 0x7f997415e820 secure :-1 s=READY pgs=3293 cs=0 l=1 rev1=1 crypto rx=0x7f99740c2490 tx=0x7f996002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa96399b4efaf2e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:04.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.477+0000 7f99737fe640 1 -- 172.21.15.84:0/419289927 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9960030860 con 0x7f997415ae80 2023-10-16T11:27:04.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.477+0000 7f99737fe640 1 -- 172.21.15.84:0/419289927 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99600309c0 con 0x7f997415ae80 2023-10-16T11:27:04.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.477+0000 7f99737fe640 1 -- 172.21.15.84:0/419289927 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9960030d50 con 0x7f997415ae80 2023-10-16T11:27:04.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.478+0000 7f997afc7640 1 -- 172.21.15.84:0/419289927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997415ae80 msgr2=0x7f997415e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:04.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.478+0000 7f997afc7640 1 --2- 172.21.15.84:0/419289927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997415ae80 0x7f997415e820 secure :-1 s=READY pgs=3293 cs=0 l=1 rev1=1 crypto rx=0x7f99740c2490 tx=0x7f996002b9a0 comp rx=0 tx=0).stop 2023-10-16T11:27:04.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.478+0000 7f997afc7640 1 -- 172.21.15.84:0/419289927 shutdown_connections 2023-10-16T11:27:04.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.478+0000 7f997afc7640 1 --2- 172.21.15.84:0/419289927 >> v2:172.21.15.157:3300/0 conn(0x7f99740ce390 0x7f9974070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:04.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.478+0000 7f997afc7640 1 --2- 172.21.15.84:0/419289927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997415ae80 0x7f997415e820 unknown :-1 s=CLOSED pgs=3293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:04.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.478+0000 7f997afc7640 1 -- 172.21.15.84:0/419289927 >> 172.21.15.84:0/419289927 conn(0x7f99740cb410 msgr2=0x7f99740cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:04.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.478+0000 7f997afc7640 1 -- 172.21.15.84:0/419289927 shutdown_connections 2023-10-16T11:27:04.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.478+0000 7f997afc7640 1 -- 172.21.15.84:0/419289927 wait complete. 2023-10-16T11:27:04.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.480+0000 7f997afc7640 1 Processor -- start 2023-10-16T11:27:04.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.480+0000 7f997afc7640 1 -- start start 2023-10-16T11:27:04.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.481+0000 7f997afc7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99740ce390 0x7f99740c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:04.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.481+0000 7f997afc7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997415aaa0 0x7f99741f2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:04.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.481+0000 7f997afc7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997415ae80 0x7f99740c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:04.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.481+0000 7f997afc7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99741f2ec0 con 0x7f997415ae80 2023-10-16T11:27:04.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.481+0000 7f997afc7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99741f31a0 con 0x7f99740ce390 2023-10-16T11:27:04.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.481+0000 7f997afc7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99741f3480 con 0x7f997415aaa0 2023-10-16T11:27:04.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.481+0000 7f9979564640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997415ae80 0x7f99740c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:04.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.481+0000 7f9979564640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997415ae80 0x7f99740c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42572/0 (socket says 172.21.15.84:42572) 2023-10-16T11:27:04.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.481+0000 7f9979564640 1 -- 172.21.15.84:0/3040938222 learned_addr learned my addr 172.21.15.84:0/3040938222 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:04.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.481+0000 7f9973fff640 1 --2- 172.21.15.84:0/3040938222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997415aaa0 0x7f99741f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:04.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.481+0000 7f9979564640 1 -- 172.21.15.84:0/3040938222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997415aaa0 msgr2=0x7f99741f2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:04.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.481+0000 7f9979564640 1 --2- 172.21.15.84:0/3040938222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997415aaa0 0x7f99741f2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:04.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.481+0000 7f9979564640 1 -- 172.21.15.84:0/3040938222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99740ce390 msgr2=0x7f99740c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:04.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.481+0000 7f9979564640 1 --2- 172.21.15.84:0/3040938222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99740ce390 0x7f99740c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:04.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.481+0000 7f9979564640 1 -- 172.21.15.84:0/3040938222 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99600094d0 con 0x7f997415ae80 2023-10-16T11:27:04.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.482+0000 7f9979564640 1 --2- 172.21.15.84:0/3040938222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997415ae80 0x7f99740c97e0 secure :-1 s=READY pgs=3294 cs=0 l=1 rev1=1 crypto rx=0x7f996c00cef0 tx=0x7f996c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:04.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.482+0000 7f99717fa640 1 -- 172.21.15.84:0/3040938222 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00d840 con 0x7f997415ae80 2023-10-16T11:27:04.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.482+0000 7f99717fa640 1 -- 172.21.15.84:0/3040938222 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c00d9c0 con 0x7f997415ae80 2023-10-16T11:27:04.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.482+0000 7f997afc7640 1 -- 172.21.15.84:0/3040938222 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99740c8fe0 con 0x7f997415ae80 2023-10-16T11:27:04.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.482+0000 7f99717fa640 1 -- 172.21.15.84:0/3040938222 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00dd70 con 0x7f997415ae80 2023-10-16T11:27:04.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.482+0000 7f997afc7640 1 -- 172.21.15.84:0/3040938222 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99740c7670 con 0x7f997415ae80 2023-10-16T11:27:04.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.483+0000 7f997afc7640 1 -- 172.21.15.84:0/3040938222 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99740c9500 con 0x7f997415ae80 2023-10-16T11:27:04.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.484+0000 7f99717fa640 1 -- 172.21.15.84:0/3040938222 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 156) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f996c01e3b0 con 0x7f997415ae80 2023-10-16T11:27:04.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.485+0000 7f99717fa640 1 --2- 172.21.15.84:0/3040938222 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f995806b020 0x7f995806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:04.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.485+0000 7f9978d63640 1 -- 172.21.15.84:0/3040938222 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f995806b020 msgr2=0x7f995806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2988856492 2023-10-16T11:27:04.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.485+0000 7f99717fa640 1 -- 172.21.15.84:0/3040938222 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6210+0+0 (secure 0 0 0) 0x7f996c08a610 con 0x7f997415ae80 2023-10-16T11:27:04.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.485+0000 7f9978d63640 1 --2- 172.21.15.84:0/3040938222 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f995806b020 0x7f995806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:04.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.489+0000 7f99717fa640 1 -- 172.21.15.84:0/3040938222 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f99740c9500 con 0x7f997415ae80 2023-10-16T11:27:04.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.686+0000 7f9978d63640 1 -- 172.21.15.84:0/3040938222 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f995806b020 msgr2=0x7f995806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2988856492 2023-10-16T11:27:04.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.686+0000 7f9978d63640 1 --2- 172.21.15.84:0/3040938222 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f995806b020 0x7f995806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:27:04.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.722+0000 7f997afc7640 1 -- 172.21.15.84:0/3040938222 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99740c9500 con 0x7f997415ae80 2023-10-16T11:27:04.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.722+0000 7f99717fa640 1 -- 172.21.15.84:0/3040938222 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+4929 (secure 0 0 0) 0x7f99740c9500 con 0x7f997415ae80 2023-10-16T11:27:04.724 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:04.724 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":148,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/591944719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":591944719},{"type":"v1","addr":"172.21.15.84:6835","nonce":591944719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":142},{"gid":8152,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2371517609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2371517609},{"type":"v1","addr":"172.21.15.84:6837","nonce":2371517609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":143},{"gid":8161,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1079456313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1079456313},{"type":"v1","addr":"172.21.15.157:6834","nonce":1079456313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":143}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:26:41.908585+0000","modified":"2023-10-16T11:26:44.991714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12987":{"gid":12987,"name":"b","rank":0,"incarnation":144,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3899307325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3899307325},{"type":"v1","addr":"172.21.15.157:6835","nonce":3899307325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-16T11:27:04.724 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 148 2023-10-16T11:27:04.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.725+0000 7f997afc7640 1 -- 172.21.15.84:0/3040938222 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f995806b020 msgr2=0x7f995806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:04.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.725+0000 7f997afc7640 1 --2- 172.21.15.84:0/3040938222 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f995806b020 0x7f995806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:04.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.725+0000 7f997afc7640 1 -- 172.21.15.84:0/3040938222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997415ae80 msgr2=0x7f99740c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:04.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.725+0000 7f997afc7640 1 --2- 172.21.15.84:0/3040938222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997415ae80 0x7f99740c97e0 secure :-1 s=READY pgs=3294 cs=0 l=1 rev1=1 crypto rx=0x7f996c00cef0 tx=0x7f996c00a710 comp rx=0 tx=0).stop 2023-10-16T11:27:04.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.725+0000 7f997afc7640 1 -- 172.21.15.84:0/3040938222 shutdown_connections 2023-10-16T11:27:04.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.725+0000 7f997afc7640 1 --2- 172.21.15.84:0/3040938222 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f995806b020 0x7f995806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:04.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.725+0000 7f997afc7640 1 --2- 172.21.15.84:0/3040938222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997415ae80 0x7f99740c97e0 unknown :-1 s=CLOSED pgs=3294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:04.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.725+0000 7f997afc7640 1 --2- 172.21.15.84:0/3040938222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997415aaa0 0x7f99741f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:04.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.725+0000 7f997afc7640 1 --2- 172.21.15.84:0/3040938222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99740ce390 0x7f99740c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:04.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.725+0000 7f997afc7640 1 -- 172.21.15.84:0/3040938222 >> 172.21.15.84:0/3040938222 conn(0x7f99740cb410 msgr2=0x7f99740cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:04.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.725+0000 7f997afc7640 1 -- 172.21.15.84:0/3040938222 shutdown_connections 2023-10-16T11:27:04.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.725+0000 7f997afc7640 1 -- 172.21.15.84:0/3040938222 wait complete. 2023-10-16T11:27:04.743 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:27:04.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.930+0000 7f2e01d01640 1 Processor -- start 2023-10-16T11:27:04.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.931+0000 7f2e01d01640 1 -- start start 2023-10-16T11:27:04.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.933+0000 7f2e01d01640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2dfc159b20 0x7f2dfc06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:04.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.933+0000 7f2e01d01640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dfc0b0b20 0x7f2dfc06fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:04.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.933+0000 7f2e01d01640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dfc06f7f0 con 0x7f2dfc159f00 2023-10-16T11:27:04.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.933+0000 7f2e01d01640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dfc06c5e0 con 0x7f2dfc159b20 2023-10-16T11:27:04.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.933+0000 7f2e01d01640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc06c8c0 con 0x7f2dfc0b0b20 2023-10-16T11:27:04.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.933+0000 7f2df3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2dfc159f00 0x7f2dfc0b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35698/0 (socket says 172.21.15.84:35698) 2023-10-16T11:27:04.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.933+0000 7f2df3fff640 1 -- 172.21.15.84:0/1903127966 learned_addr learned my addr 172.21.15.84:0/1903127966 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:04.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.933+0000 7f2dfb7fe640 1 --2- 172.21.15.84:0/1903127966 >> v2:172.21.15.157:3300/0 conn(0x7f2dfc159b20 0x7f2dfc06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:04.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.933+0000 7f2dfbfff640 1 --2- 172.21.15.84:0/1903127966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dfc0b0b20 0x7f2dfc06fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:04.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.934+0000 7f2dfaffd640 1 -- 172.21.15.84:0/1903127966 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2918612965 0 0) 0x7f2dfc06f7f0 con 0x7f2dfc159f00 2023-10-16T11:27:04.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.934+0000 7f2dfaffd640 1 -- 172.21.15.84:0/1903127966 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2de8003520 con 0x7f2dfc159f00 2023-10-16T11:27:04.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.934+0000 7f2dfb7fe640 1 -- 172.21.15.84:0/1903127966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dfc0b0b20 msgr2=0x7f2dfc06fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:04.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.934+0000 7f2dfb7fe640 1 --2- 172.21.15.84:0/1903127966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dfc0b0b20 0x7f2dfc06fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:04.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.934+0000 7f2dfb7fe640 1 -- 172.21.15.84:0/1903127966 >> v1:172.21.15.84:6789/0 conn(0x7f2dfc159f00 legacy=0x7f2dfc0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:04.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.934+0000 7f2dfaffd640 1 -- 172.21.15.84:0/1903127966 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2248826240 0 0) 0x7f2de8003520 con 0x7f2dfc159f00 2023-10-16T11:27:04.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.934+0000 7f2dfb7fe640 1 -- 172.21.15.84:0/1903127966 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dfc06cba0 con 0x7f2dfc159b20 2023-10-16T11:27:04.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.935+0000 7f2dfb7fe640 1 --2- 172.21.15.84:0/1903127966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dfc159b20 0x7f2dfc06bbe0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009d90 tx=0x7f2dec02e0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27ba7edcead46eef server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:04.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.936+0000 7f2dfaffd640 1 -- 172.21.15.84:0/1903127966 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec02ee60 con 0x7f2dfc159b20 2023-10-16T11:27:04.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.936+0000 7f2dfaffd640 1 -- 172.21.15.84:0/1903127966 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dec036440 con 0x7f2dfc159b20 2023-10-16T11:27:04.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.936+0000 7f2dfaffd640 1 -- 172.21.15.84:0/1903127966 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec0367b0 con 0x7f2dfc159b20 2023-10-16T11:27:04.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.937+0000 7f2e01d01640 1 -- 172.21.15.84:0/1903127966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dfc159b20 msgr2=0x7f2dfc06bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:04.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.937+0000 7f2e01d01640 1 --2- 172.21.15.84:0/1903127966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dfc159b20 0x7f2dfc06bbe0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009d90 tx=0x7f2dec02e0b0 comp rx=0 tx=0).stop 2023-10-16T11:27:04.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.937+0000 7f2e01d01640 1 -- 172.21.15.84:0/1903127966 shutdown_connections 2023-10-16T11:27:04.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.937+0000 7f2e01d01640 1 --2- 172.21.15.84:0/1903127966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dfc0b0b20 0x7f2dfc06fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:04.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.937+0000 7f2e01d01640 1 --2- 172.21.15.84:0/1903127966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dfc159b20 0x7f2dfc06bbe0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:04.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.937+0000 7f2e01d01640 1 -- 172.21.15.84:0/1903127966 >> 172.21.15.84:0/1903127966 conn(0x7f2dfc0cb410 msgr2=0x7f2dfc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:04.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.937+0000 7f2e01d01640 1 -- 172.21.15.84:0/1903127966 shutdown_connections 2023-10-16T11:27:04.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.937+0000 7f2e01d01640 1 -- 172.21.15.84:0/1903127966 wait complete. 2023-10-16T11:27:04.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.939+0000 7f2e01d01640 1 Processor -- start 2023-10-16T11:27:04.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.939+0000 7f2e01d01640 1 -- start start 2023-10-16T11:27:04.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.940+0000 7f2e01d01640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dfc0b0b20 0x7f2dfc0c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:04.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.940+0000 7f2e01d01640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dfc159b20 0x7f2dfc0c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:04.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.940+0000 7f2e01d01640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2dfc159f00 0x7f2dfc1f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:04.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.940+0000 7f2e01d01640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc1f3d00 con 0x7f2dfc159f00 2023-10-16T11:27:04.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.940+0000 7f2e01d01640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc0c3e60 con 0x7f2dfc0b0b20 2023-10-16T11:27:04.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.940+0000 7f2e01d01640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc0c2b10 con 0x7f2dfc159b20 2023-10-16T11:27:04.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.940+0000 7f2dfbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2dfc159f00 0x7f2dfc1f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:04.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.940+0000 7f2dfbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2dfc159f00 0x7f2dfc1f37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42582/0 (socket says 172.21.15.84:42582) 2023-10-16T11:27:04.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.940+0000 7f2dfbfff640 1 -- 172.21.15.84:0/2205940532 learned_addr learned my addr 172.21.15.84:0/2205940532 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:04.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.940+0000 7f2df3fff640 1 --2- 172.21.15.84:0/2205940532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dfc159b20 0x7f2dfc0c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:04.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.940+0000 7f2dfb7fe640 1 --2- 172.21.15.84:0/2205940532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dfc0b0b20 0x7f2dfc0c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:04.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.941+0000 7f2dfbfff640 1 -- 172.21.15.84:0/2205940532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dfc159b20 msgr2=0x7f2dfc0c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:04.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.941+0000 7f2dfbfff640 1 --2- 172.21.15.84:0/2205940532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dfc159b20 0x7f2dfc0c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:04.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.941+0000 7f2dfbfff640 1 -- 172.21.15.84:0/2205940532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dfc0b0b20 msgr2=0x7f2dfc0c24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:04.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.941+0000 7f2dfbfff640 1 --2- 172.21.15.84:0/2205940532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dfc0b0b20 0x7f2dfc0c24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:04.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.941+0000 7f2dfbfff640 1 -- 172.21.15.84:0/2205940532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dec009a70 con 0x7f2dfc159f00 2023-10-16T11:27:04.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.941+0000 7f2dfbfff640 1 --2- 172.21.15.84:0/2205940532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2dfc159f00 0x7f2dfc1f37c0 secure :-1 s=READY pgs=3296 cs=0 l=1 rev1=1 crypto rx=0x7f2df400e850 tx=0x7f2df400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:04.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.941+0000 7f2df8ff9640 1 -- 172.21.15.84:0/2205940532 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df4009630 con 0x7f2dfc159f00 2023-10-16T11:27:04.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.941+0000 7f2df8ff9640 1 -- 172.21.15.84:0/2205940532 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df400c940 con 0x7f2dfc159f00 2023-10-16T11:27:04.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.941+0000 7f2e01d01640 1 -- 172.21.15.84:0/2205940532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dfc0c2df0 con 0x7f2dfc159f00 2023-10-16T11:27:04.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.941+0000 7f2df8ff9640 1 -- 172.21.15.84:0/2205940532 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df400ccf0 con 0x7f2dfc159f00 2023-10-16T11:27:04.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.941+0000 7f2e01d01640 1 -- 172.21.15.84:0/2205940532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dfc0c33b0 con 0x7f2dfc159f00 2023-10-16T11:27:04.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.943+0000 7f2e01d01640 1 -- 172.21.15.84:0/2205940532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dfc0af4a0 con 0x7f2dfc159f00 2023-10-16T11:27:04.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.945+0000 7f2df8ff9640 1 -- 172.21.15.84:0/2205940532 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 156) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f2df4013470 con 0x7f2dfc159f00 2023-10-16T11:27:04.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.946+0000 7f2df8ff9640 1 --2- 172.21.15.84:0/2205940532 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f2ddc06b050 0x7f2ddc06d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:04.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.946+0000 7f2df8ff9640 1 -- 172.21.15.84:0/2205940532 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6210+0+0 (secure 0 0 0) 0x7f2df4089cb0 con 0x7f2dfc159f00 2023-10-16T11:27:04.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.946+0000 7f2dfb7fe640 1 -- 172.21.15.84:0/2205940532 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f2ddc06b050 msgr2=0x7f2ddc06d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2988856492 2023-10-16T11:27:04.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.946+0000 7f2dfb7fe640 1 --2- 172.21.15.84:0/2205940532 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f2ddc06b050 0x7f2ddc06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:04.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:04.951+0000 7f2df8ff9640 1 -- 172.21.15.84:0/2205940532 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2df4054970 con 0x7f2dfc159f00 2023-10-16T11:27:04.966 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:27:04.967 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:04.964+0000 7f7a5dc15040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:27:04.973 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:27:04.995 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:04.994+0000 7f7879a33040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:27:05.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.144+0000 7f2e01d01640 1 -- 172.21.15.84:0/2205940532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2dfc0ae100 con 0x7f2dfc159f00 2023-10-16T11:27:05.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.146+0000 7f2df8ff9640 1 -- 172.21.15.84:0/2205940532 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v258) v1 ==== 74+0+17231 (secure 0 0 0) 0x7f2df4021090 con 0x7f2dfc159f00 2023-10-16T11:27:05.147 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:05.148 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":258,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:26:57.151442+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:26:39.860053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-16T11:26:40.916519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/907959269":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/1863310536":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/847549599":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/1934386371":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/3540589884":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/2370234804":"2023-10-17T11:26:57.151307+0000","172.21.15.84:6833/2330196255":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/1650008535":"2023-10-17T11:26:57.151307+0000","172.21.15.84:6832/2330196255":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/2440426242":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/570042115":"2023-10-16T12:26:40.168744+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:27:05.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.147+0000 7f2dfb7fe640 1 -- 172.21.15.84:0/2205940532 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f2ddc06b050 msgr2=0x7f2ddc06d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2988856492 2023-10-16T11:27:05.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.147+0000 7f2dfb7fe640 1 --2- 172.21.15.84:0/2205940532 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f2ddc06b050 0x7f2ddc06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:27:05.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.148+0000 7f2e01d01640 1 -- 172.21.15.84:0/2205940532 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f2ddc06b050 msgr2=0x7f2ddc06d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:05.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.148+0000 7f2e01d01640 1 --2- 172.21.15.84:0/2205940532 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f2ddc06b050 0x7f2ddc06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.148+0000 7f2e01d01640 1 -- 172.21.15.84:0/2205940532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2dfc159f00 msgr2=0x7f2dfc1f37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:05.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.148+0000 7f2e01d01640 1 --2- 172.21.15.84:0/2205940532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2dfc159f00 0x7f2dfc1f37c0 secure :-1 s=READY pgs=3296 cs=0 l=1 rev1=1 crypto rx=0x7f2df400e850 tx=0x7f2df400ed10 comp rx=0 tx=0).stop 2023-10-16T11:27:05.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.149+0000 7f2e01d01640 1 -- 172.21.15.84:0/2205940532 shutdown_connections 2023-10-16T11:27:05.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.149+0000 7f2e01d01640 1 --2- 172.21.15.84:0/2205940532 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f2ddc06b050 0x7f2ddc06d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.149+0000 7f2e01d01640 1 --2- 172.21.15.84:0/2205940532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2dfc159f00 0x7f2dfc1f37c0 unknown :-1 s=CLOSED pgs=3296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.149+0000 7f2e01d01640 1 --2- 172.21.15.84:0/2205940532 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2dfc159b20 0x7f2dfc0c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.149+0000 7f2e01d01640 1 --2- 172.21.15.84:0/2205940532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dfc0b0b20 0x7f2dfc0c24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.149+0000 7f2e01d01640 1 -- 172.21.15.84:0/2205940532 >> 172.21.15.84:0/2205940532 conn(0x7f2dfc0cb410 msgr2=0x7f2dfc15d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:05.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.149+0000 7f2e01d01640 1 -- 172.21.15.84:0/2205940532 shutdown_connections 2023-10-16T11:27:05.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.149+0000 7f2e01d01640 1 -- 172.21.15.84:0/2205940532 wait complete. 2023-10-16T11:27:05.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 34, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:26:39.860053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '256', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 35, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:26:40.916519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '257', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:27:05.169 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:27:05.276 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:05.274+0000 7f7a5dc15040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:27:05.310 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:05.309+0000 7f7879a33040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:27:05.333 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:05.331+0000 7f7a5dc15040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:27:05.369 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:05.368+0000 7f7879a33040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:27:05.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.378+0000 7f85fbf68640 1 Processor -- start 2023-10-16T11:27:05.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.378+0000 7f85fbf68640 1 -- start start 2023-10-16T11:27:05.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.379+0000 7f85fbf68640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f85f4070830 0x7f85f406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:05.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.379+0000 7f85fbf68640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85f4070c10 0x7f85f40cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:05.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.379+0000 7f85fbf68640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f4161740 con 0x7f85f40cf5f0 2023-10-16T11:27:05.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.379+0000 7f85fbf68640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f40cdbd0 con 0x7f85f4070830 2023-10-16T11:27:05.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.379+0000 7f85fbf68640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f406b0f0 con 0x7f85f4070c10 2023-10-16T11:27:05.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.380+0000 7f85f9503640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f85f40cf5f0 0x7f85f406c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35702/0 (socket says 172.21.15.84:35702) 2023-10-16T11:27:05.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.380+0000 7f85f9503640 1 -- 172.21.15.84:0/4117037026 learned_addr learned my addr 172.21.15.84:0/4117037026 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:05.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.380+0000 7f85fa505640 1 --2- 172.21.15.84:0/4117037026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85f4070c10 0x7f85f40cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:05.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.380+0000 7f85f9d04640 1 --2- 172.21.15.84:0/4117037026 >> v2:172.21.15.157:3300/0 conn(0x7f85f4070830 0x7f85f406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:05.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.380+0000 7f85f8d02640 1 -- 172.21.15.84:0/4117037026 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383564889 0 0) 0x7f85f4161740 con 0x7f85f40cf5f0 2023-10-16T11:27:05.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.380+0000 7f85f8d02640 1 -- 172.21.15.84:0/4117037026 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85e0003520 con 0x7f85f40cf5f0 2023-10-16T11:27:05.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.381+0000 7f85f8d02640 1 -- 172.21.15.84:0/4117037026 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2057158259 0 0) 0x7f85e0003520 con 0x7f85f40cf5f0 2023-10-16T11:27:05.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.381+0000 7f85f8d02640 1 -- 172.21.15.84:0/4117037026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85f4070c10 msgr2=0x7f85f40cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:05.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.381+0000 7f85f8d02640 1 --2- 172.21.15.84:0/4117037026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85f4070c10 0x7f85f40cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.381+0000 7f85f8d02640 1 -- 172.21.15.84:0/4117037026 >> v2:172.21.15.157:3300/0 conn(0x7f85f4070830 msgr2=0x7f85f406d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:05.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.381+0000 7f85f8d02640 1 --2- 172.21.15.84:0/4117037026 >> v2:172.21.15.157:3300/0 conn(0x7f85f4070830 0x7f85f406d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.381+0000 7f85f8d02640 1 -- 172.21.15.84:0/4117037026 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f41619b0 con 0x7f85f40cf5f0 2023-10-16T11:27:05.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.381+0000 7f85f8d02640 1 -- 172.21.15.84:0/4117037026 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f85e4001670 con 0x7f85f40cf5f0 2023-10-16T11:27:05.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.381+0000 7f85f8d02640 1 -- 172.21.15.84:0/4117037026 >> v1:172.21.15.84:6789/0 conn(0x7f85f40cf5f0 legacy=0x7f85f406c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:05.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.381+0000 7f85f8d02640 1 --2- 172.21.15.84:0/4117037026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85e0003970 0x7f85e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:05.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.381+0000 7f85f8d02640 1 --2- 172.21.15.84:0/4117037026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85f4070c10 0x7f85e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:05.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.381+0000 7f85f8d02640 1 --2- 172.21.15.84:0/4117037026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e0004d00 0x7f85e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:05.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.381+0000 7f85f8d02640 1 -- 172.21.15.84:0/4117037026 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f40cdbd0 con 0x7f85e0004d00 2023-10-16T11:27:05.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.381+0000 7f85f8d02640 1 -- 172.21.15.84:0/4117037026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f406b0f0 con 0x7f85e0003970 2023-10-16T11:27:05.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.381+0000 7f85f8d02640 1 -- 172.21.15.84:0/4117037026 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e000ba00 con 0x7f85f4070c10 2023-10-16T11:27:05.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.381+0000 7f85fa505640 1 --2- 172.21.15.84:0/4117037026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e0004d00 0x7f85e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:05.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.381+0000 7f85f9503640 1 --2- 172.21.15.84:0/4117037026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85f4070c10 0x7f85e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:05.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.381+0000 7f85f9d04640 1 --2- 172.21.15.84:0/4117037026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85e0003970 0x7f85e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:05.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.382+0000 7f85fa505640 1 -- 172.21.15.84:0/4117037026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85f4070c10 msgr2=0x7f85e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:05.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.382+0000 7f85fa505640 1 --2- 172.21.15.84:0/4117037026 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85f4070c10 0x7f85e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.382+0000 7f85fa505640 1 -- 172.21.15.84:0/4117037026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85e0003970 msgr2=0x7f85e0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:05.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.382+0000 7f85fa505640 1 --2- 172.21.15.84:0/4117037026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85e0003970 0x7f85e0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.382+0000 7f85fa505640 1 -- 172.21.15.84:0/4117037026 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e0003520 con 0x7f85e0004d00 2023-10-16T11:27:05.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.382+0000 7f85f9d04640 1 --2- 172.21.15.84:0/4117037026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85e0003970 0x7f85e0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:05.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.383+0000 7f85fa505640 1 --2- 172.21.15.84:0/4117037026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e0004d00 0x7f85e0005180 secure :-1 s=READY pgs=3298 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009fc0 tx=0x7f85e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6257077ef63466d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:05.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.383+0000 7f85f8d02640 1 -- 172.21.15.84:0/4117037026 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec02cc20 con 0x7f85e0004d00 2023-10-16T11:27:05.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.383+0000 7f85f8d02640 1 -- 172.21.15.84:0/4117037026 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ec00f040 con 0x7f85e0004d00 2023-10-16T11:27:05.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.383+0000 7f85f8d02640 1 -- 172.21.15.84:0/4117037026 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec034440 con 0x7f85e0004d00 2023-10-16T11:27:05.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.383+0000 7f85fbf68640 1 -- 172.21.15.84:0/4117037026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e0004d00 msgr2=0x7f85e0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:05.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.383+0000 7f85fbf68640 1 --2- 172.21.15.84:0/4117037026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e0004d00 0x7f85e0005180 secure :-1 s=READY pgs=3298 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009fc0 tx=0x7f85e00057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.383+0000 7f85fa505640 1 -- 172.21.15.84:0/4117037026 reap_dead start 2023-10-16T11:27:05.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.384+0000 7f85fbf68640 1 -- 172.21.15.84:0/4117037026 shutdown_connections 2023-10-16T11:27:05.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.384+0000 7f85fbf68640 1 -- 172.21.15.84:0/4117037026 >> 172.21.15.84:0/4117037026 conn(0x7f85f40cfac0 msgr2=0x7f85f40cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:05.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.384+0000 7f85fbf68640 1 -- 172.21.15.84:0/4117037026 shutdown_connections 2023-10-16T11:27:05.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.384+0000 7f85fbf68640 1 -- 172.21.15.84:0/4117037026 wait complete. 2023-10-16T11:27:05.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.385+0000 7f85fbf68640 1 Processor -- start 2023-10-16T11:27:05.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.385+0000 7f85fbf68640 1 -- start start 2023-10-16T11:27:05.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.386+0000 7f85fbf68640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e0004d00 0x7f85f414e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.386+0000 7f85fbf68640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85f415eb90 0x7f85f415f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.386+0000 7f85fbf68640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f40ca340 0x7f85f40ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.386+0000 7f85fbf68640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f415b690 con 0x7f85e0004d00 2023-10-16T11:27:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.386+0000 7f85fbf68640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f415f550 con 0x7f85f40ca340 2023-10-16T11:27:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.386+0000 7f85fbf68640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f40cbbf0 con 0x7f85f415eb90 2023-10-16T11:27:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.387+0000 7f85f9d04640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e0004d00 0x7f85f414e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.387+0000 7f85f9d04640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e0004d00 0x7f85f414e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42602/0 (socket says 172.21.15.84:42602) 2023-10-16T11:27:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.387+0000 7f85f9d04640 1 -- 172.21.15.84:0/4158091942 learned_addr learned my addr 172.21.15.84:0/4158091942 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:05.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.387+0000 7f85f9503640 1 --2- 172.21.15.84:0/4158091942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85f415eb90 0x7f85f415f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:05.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.387+0000 7f85fa505640 1 --2- 172.21.15.84:0/4158091942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f40ca340 0x7f85f40ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:05.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.387+0000 7f85f9d04640 1 -- 172.21.15.84:0/4158091942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85f415eb90 msgr2=0x7f85f415f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:05.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.387+0000 7f85f9d04640 1 --2- 172.21.15.84:0/4158091942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85f415eb90 0x7f85f415f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.387+0000 7f85f9d04640 1 -- 172.21.15.84:0/4158091942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f40ca340 msgr2=0x7f85f40ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:05.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.387+0000 7f85f9d04640 1 --2- 172.21.15.84:0/4158091942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f40ca340 0x7f85f40ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.387+0000 7f85f9d04640 1 -- 172.21.15.84:0/4158091942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f40cafe0 con 0x7f85e0004d00 2023-10-16T11:27:05.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.387+0000 7f85f9d04640 1 --2- 172.21.15.84:0/4158091942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e0004d00 0x7f85f414e040 secure :-1 s=READY pgs=3299 cs=0 l=1 rev1=1 crypto rx=0x7f85f0010a00 tx=0x7f85f0010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:05.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.388+0000 7f85ea7fc640 1 -- 172.21.15.84:0/4158091942 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f000ebc0 con 0x7f85e0004d00 2023-10-16T11:27:05.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.388+0000 7f85ea7fc640 1 -- 172.21.15.84:0/4158091942 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f000ed20 con 0x7f85e0004d00 2023-10-16T11:27:05.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.388+0000 7f85ea7fc640 1 -- 172.21.15.84:0/4158091942 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f001c620 con 0x7f85e0004d00 2023-10-16T11:27:05.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.388+0000 7f85fbf68640 1 -- 172.21.15.84:0/4158091942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f40cb2c0 con 0x7f85e0004d00 2023-10-16T11:27:05.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.388+0000 7f85fbf68640 1 -- 172.21.15.84:0/4158091942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f40cb880 con 0x7f85e0004d00 2023-10-16T11:27:05.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.389+0000 7f85fbf68640 1 -- 172.21.15.84:0/4158091942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f40cba40 con 0x7f85e0004d00 2023-10-16T11:27:05.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.390+0000 7f85ea7fc640 1 -- 172.21.15.84:0/4158091942 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 156) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f85f001c780 con 0x7f85e0004d00 2023-10-16T11:27:05.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.391+0000 7f85ea7fc640 1 --2- 172.21.15.84:0/4158091942 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f85d806b050 0x7f85d806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:05.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.391+0000 7f85ea7fc640 1 -- 172.21.15.84:0/4158091942 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6210+0+0 (secure 0 0 0) 0x7f85f008a0d0 con 0x7f85e0004d00 2023-10-16T11:27:05.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.391+0000 7f85f9503640 1 -- 172.21.15.84:0/4158091942 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f85d806b050 msgr2=0x7f85d806d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/2988856492 2023-10-16T11:27:05.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.391+0000 7f85f9503640 1 --2- 172.21.15.84:0/4158091942 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f85d806b050 0x7f85d806d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:05.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.399+0000 7f85ea7fc640 1 -- 172.21.15.84:0/4158091942 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f85f0054d90 con 0x7f85e0004d00 2023-10-16T11:27:05.448 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:05.446+0000 7f7a5dc15040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:27:05.493 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:05.492+0000 7f7879a33040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:27:05.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.506+0000 7f85ea7fc640 1 -- 172.21.15.84:0/4158091942 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 157) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f85f0055160 con 0x7f85e0004d00 2023-10-16T11:27:05.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.507+0000 7f85ea7fc640 1 -- 172.21.15.84:0/4158091942 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f85d806b050 msgr2=0x7f85d806d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:05.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.507+0000 7f85ea7fc640 1 --2- 172.21.15.84:0/4158091942 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f85d806b050 0x7f85d806d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.576 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:05.574+0000 7f77fb4b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:05.576 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:05.574+0000 7f77fb4b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:05.576 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:05.574+0000 7f77fb4b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:05.576 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:05.574+0000 7f77fb4b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:05.577 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:05.574+0000 7f77fb4b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:05.577 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:05.577+0000 7f77fe4ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:05.578 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:05.577+0000 7f77fe4ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:05.578 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:05.577+0000 7f77fe4ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:05.578 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:05.577+0000 7f77fe4ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:05.578 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:05.577+0000 7f77fe4ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:05.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.620+0000 7f85fbf68640 1 -- 172.21.15.84:0/4158091942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f85f404e160 con 0x7f85e0004d00 2023-10-16T11:27:05.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.620+0000 7f85ea7fc640 1 -- 172.21.15.84:0/4158091942 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v259) v1 ==== 93+0+10 (secure 0 0 0) 0x7f85f0086590 con 0x7f85e0004d00 2023-10-16T11:27:05.621 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:27:05.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.623+0000 7f85fbf68640 1 -- 172.21.15.84:0/4158091942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e0004d00 msgr2=0x7f85f414e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:05.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.623+0000 7f85fbf68640 1 --2- 172.21.15.84:0/4158091942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e0004d00 0x7f85f414e040 secure :-1 s=READY pgs=3299 cs=0 l=1 rev1=1 crypto rx=0x7f85f0010a00 tx=0x7f85f0010ec0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.623+0000 7f85fbf68640 1 -- 172.21.15.84:0/4158091942 shutdown_connections 2023-10-16T11:27:05.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.623+0000 7f85fbf68640 1 --2- 172.21.15.84:0/4158091942 >> [v2:172.21.15.84:6832/2988856492,v1:172.21.15.84:6833/2988856492] conn(0x7f85d806b050 0x7f85d806d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.623+0000 7f85fbf68640 1 --2- 172.21.15.84:0/4158091942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f40ca340 0x7f85f40ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.623+0000 7f85fbf68640 1 --2- 172.21.15.84:0/4158091942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85f415eb90 0x7f85f415f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.623+0000 7f85fbf68640 1 --2- 172.21.15.84:0/4158091942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e0004d00 0x7f85f414e040 unknown :-1 s=CLOSED pgs=3299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.623+0000 7f85fbf68640 1 -- 172.21.15.84:0/4158091942 >> 172.21.15.84:0/4158091942 conn(0x7f85f40cfac0 msgr2=0x7f85f40b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:05.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.623+0000 7f85fbf68640 1 -- 172.21.15.84:0/4158091942 shutdown_connections 2023-10-16T11:27:05.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.623+0000 7f85fbf68640 1 -- 172.21.15.84:0/4158091942 wait complete. 2023-10-16T11:27:05.640 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:27:05.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.826+0000 7f4e495e2640 1 Processor -- start 2023-10-16T11:27:05.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.826+0000 7f4e495e2640 1 -- start start 2023-10-16T11:27:05.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.828+0000 7f4e495e2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4e4415d150 0x7f4e440ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:05.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.828+0000 7f4e495e2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4e4406a4b0 0x7f4e44160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:05.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.828+0000 7f4e495e2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e440ac890 con 0x7f4e44160b90 2023-10-16T11:27:05.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.828+0000 7f4e495e2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4415d530 con 0x7f4e4415d150 2023-10-16T11:27:05.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.828+0000 7f4e495e2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e44161650 con 0x7f4e4406a4b0 2023-10-16T11:27:05.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.828+0000 7f4e42ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4e4415d150 0x7f4e440ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:05.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.828+0000 7f4e3bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4e44160b90 0x7f4e44069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57272/0 (socket says 172.21.15.84:57272) 2023-10-16T11:27:05.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.828+0000 7f4e3bfff640 1 -- 172.21.15.84:0/507158696 learned_addr learned my addr 172.21.15.84:0/507158696 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:05.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.829+0000 7f4e437fe640 1 --2- 172.21.15.84:0/507158696 >> v2:172.21.15.157:3300/0 conn(0x7f4e4406a4b0 0x7f4e44160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:05.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.829+0000 7f4e42ffd640 1 -- 172.21.15.84:0/507158696 >> v2:172.21.15.157:3300/0 conn(0x7f4e4406a4b0 msgr2=0x7f4e44160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:05.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.829+0000 7f4e42ffd640 1 --2- 172.21.15.84:0/507158696 >> v2:172.21.15.157:3300/0 conn(0x7f4e4406a4b0 0x7f4e44160fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.829+0000 7f4e42ffd640 1 -- 172.21.15.84:0/507158696 >> v1:172.21.15.84:6789/0 conn(0x7f4e44160b90 legacy=0x7f4e44069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:05.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.829+0000 7f4e427fc640 1 -- 172.21.15.84:0/507158696 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888617647 0 0) 0x7f4e440ac890 con 0x7f4e44160b90 2023-10-16T11:27:05.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.829+0000 7f4e42ffd640 1 -- 172.21.15.84:0/507158696 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e44161930 con 0x7f4e4415d150 2023-10-16T11:27:05.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.830+0000 7f4e42ffd640 1 --2- 172.21.15.84:0/507158696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e4415d150 0x7f4e440ad350 secure :-1 s=READY pgs=3312 cs=0 l=1 rev1=1 crypto rx=0x7f4e440adde0 tx=0x7f4e3402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4072128d00794fd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:05.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.831+0000 7f4e427fc640 1 -- 172.21.15.84:0/507158696 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3402e070 con 0x7f4e4415d150 2023-10-16T11:27:05.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.831+0000 7f4e427fc640 1 -- 172.21.15.84:0/507158696 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e340315a0 con 0x7f4e4415d150 2023-10-16T11:27:05.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.831+0000 7f4e427fc640 1 -- 172.21.15.84:0/507158696 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34031930 con 0x7f4e4415d150 2023-10-16T11:27:05.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.831+0000 7f4e495e2640 1 -- 172.21.15.84:0/507158696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e4415d150 msgr2=0x7f4e440ad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:05.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.831+0000 7f4e495e2640 1 --2- 172.21.15.84:0/507158696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e4415d150 0x7f4e440ad350 secure :-1 s=READY pgs=3312 cs=0 l=1 rev1=1 crypto rx=0x7f4e440adde0 tx=0x7f4e3402b840 comp rx=0 tx=0).stop 2023-10-16T11:27:05.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.831+0000 7f4e495e2640 1 -- 172.21.15.84:0/507158696 shutdown_connections 2023-10-16T11:27:05.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.831+0000 7f4e495e2640 1 --2- 172.21.15.84:0/507158696 >> v2:172.21.15.157:3300/0 conn(0x7f4e4406a4b0 0x7f4e44160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.831+0000 7f4e495e2640 1 --2- 172.21.15.84:0/507158696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e4415d150 0x7f4e440ad350 unknown :-1 s=CLOSED pgs=3312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.831+0000 7f4e495e2640 1 -- 172.21.15.84:0/507158696 >> 172.21.15.84:0/507158696 conn(0x7f4e440ce9b0 msgr2=0x7f4e440cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:05.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.831+0000 7f4e495e2640 1 -- 172.21.15.84:0/507158696 shutdown_connections 2023-10-16T11:27:05.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.831+0000 7f4e495e2640 1 -- 172.21.15.84:0/507158696 wait complete. 2023-10-16T11:27:05.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.833+0000 7f4e495e2640 1 Processor -- start 2023-10-16T11:27:05.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.833+0000 7f4e495e2640 1 -- start start 2023-10-16T11:27:05.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.834+0000 7f4e495e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e4406a4b0 0x7f4e44154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:05.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.834+0000 7f4e495e2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e4415d150 0x7f4e440b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:05.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.834+0000 7f4e495e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e44160b90 0x7f4e440b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:05.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.834+0000 7f4e495e2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e441554f0 con 0x7f4e4406a4b0 2023-10-16T11:27:05.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.834+0000 7f4e495e2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e440ae240 con 0x7f4e44160b90 2023-10-16T11:27:05.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.834+0000 7f4e495e2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e440c6c00 con 0x7f4e4415d150 2023-10-16T11:27:05.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.834+0000 7f4e42ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e4406a4b0 0x7f4e44154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:05.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.835+0000 7f4e42ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e4406a4b0 0x7f4e44154fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43984/0 (socket says 172.21.15.84:43984) 2023-10-16T11:27:05.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.835+0000 7f4e42ffd640 1 -- 172.21.15.84:0/776231550 learned_addr learned my addr 172.21.15.84:0/776231550 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:05.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.835+0000 7f4e437fe640 1 --2- 172.21.15.84:0/776231550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e44160b90 0x7f4e440b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:05.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.835+0000 7f4e3bfff640 1 --2- 172.21.15.84:0/776231550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e4415d150 0x7f4e440b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:05.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.835+0000 7f4e42ffd640 1 -- 172.21.15.84:0/776231550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e4415d150 msgr2=0x7f4e440b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:05.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.835+0000 7f4e42ffd640 1 --2- 172.21.15.84:0/776231550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e4415d150 0x7f4e440b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.835+0000 7f4e42ffd640 1 -- 172.21.15.84:0/776231550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e44160b90 msgr2=0x7f4e440b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:05.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.835+0000 7f4e42ffd640 1 --2- 172.21.15.84:0/776231550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e44160b90 0x7f4e440b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:05.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.835+0000 7f4e42ffd640 1 -- 172.21.15.84:0/776231550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e34009400 con 0x7f4e4406a4b0 2023-10-16T11:27:05.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.835+0000 7f4e437fe640 1 --2- 172.21.15.84:0/776231550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e44160b90 0x7f4e440b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:05.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.835+0000 7f4e42ffd640 1 --2- 172.21.15.84:0/776231550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e4406a4b0 0x7f4e44154fb0 secure :-1 s=READY pgs=3313 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009890 tx=0x7f4e340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:05.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.836+0000 7f4e3b7fe640 1 -- 172.21.15.84:0/776231550 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34032840 con 0x7f4e4406a4b0 2023-10-16T11:27:05.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.836+0000 7f4e3b7fe640 1 -- 172.21.15.84:0/776231550 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e340329a0 con 0x7f4e4406a4b0 2023-10-16T11:27:05.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.836+0000 7f4e3b7fe640 1 -- 172.21.15.84:0/776231550 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34032d50 con 0x7f4e4406a4b0 2023-10-16T11:27:05.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.836+0000 7f4e495e2640 1 -- 172.21.15.84:0/776231550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e440c5ff0 con 0x7f4e4406a4b0 2023-10-16T11:27:05.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.836+0000 7f4e495e2640 1 -- 172.21.15.84:0/776231550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e440c65b0 con 0x7f4e4406a4b0 2023-10-16T11:27:05.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.837+0000 7f4e495e2640 1 -- 172.21.15.84:0/776231550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4404e160 con 0x7f4e4406a4b0 2023-10-16T11:27:05.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.838+0000 7f4e3b7fe640 1 -- 172.21.15.84:0/776231550 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 157) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f4e3404d3f0 con 0x7f4e4406a4b0 2023-10-16T11:27:05.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.839+0000 7f4e3b7fe640 1 -- 172.21.15.84:0/776231550 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f4e340ab0a0 con 0x7f4e4406a4b0 2023-10-16T11:27:05.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:05.843+0000 7f4e3b7fe640 1 -- 172.21.15.84:0/776231550 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4e34076130 con 0x7f4e4406a4b0 2023-10-16T11:27:06.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.021+0000 7f4e495e2640 1 -- 172.21.15.84:0/776231550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4e440c8cc0 con 0x7f4e4406a4b0 2023-10-16T11:27:06.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.022+0000 7f4e3b7fe640 1 -- 172.21.15.84:0/776231550 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v259) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4e34079750 con 0x7f4e4406a4b0 2023-10-16T11:27:06.023 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:27:06.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.024+0000 7f4e495e2640 1 -- 172.21.15.84:0/776231550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e4406a4b0 msgr2=0x7f4e44154fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:06.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.024+0000 7f4e495e2640 1 --2- 172.21.15.84:0/776231550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e4406a4b0 0x7f4e44154fb0 secure :-1 s=READY pgs=3313 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009890 tx=0x7f4e340046c0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.025+0000 7f4e495e2640 1 -- 172.21.15.84:0/776231550 shutdown_connections 2023-10-16T11:27:06.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.025+0000 7f4e495e2640 1 --2- 172.21.15.84:0/776231550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e44160b90 0x7f4e440b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.025+0000 7f4e495e2640 1 --2- 172.21.15.84:0/776231550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e4415d150 0x7f4e440b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.025+0000 7f4e495e2640 1 --2- 172.21.15.84:0/776231550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e4406a4b0 0x7f4e44154fb0 unknown :-1 s=CLOSED pgs=3313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.025+0000 7f4e495e2640 1 -- 172.21.15.84:0/776231550 >> 172.21.15.84:0/776231550 conn(0x7f4e440ce9b0 msgr2=0x7f4e440710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:06.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.025+0000 7f4e495e2640 1 -- 172.21.15.84:0/776231550 shutdown_connections 2023-10-16T11:27:06.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.025+0000 7f4e495e2640 1 -- 172.21.15.84:0/776231550 wait complete. 2023-10-16T11:27:06.042 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:27:06.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.249+0000 7f842c9ef640 1 Processor -- start 2023-10-16T11:27:06.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.249+0000 7f842c9ef640 1 -- start start 2023-10-16T11:27:06.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.250+0000 7f842c9ef640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8428160c80 0x7f8428161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:06.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.250+0000 7f842c9ef640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8428070950 0x7f8428140400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:06.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.250+0000 7f842c9ef640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84280ae1a0 con 0x7f84280ad7a0 2023-10-16T11:27:06.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.250+0000 7f842c9ef640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84280ae480 con 0x7f8428160c80 2023-10-16T11:27:06.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.250+0000 7f842c9ef640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84280ae760 con 0x7f8428070950 2023-10-16T11:27:06.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.251+0000 7f841ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f84280ad7a0 0x7f8428070240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57286/0 (socket says 172.21.15.84:57286) 2023-10-16T11:27:06.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.251+0000 7f841ffff640 1 -- 172.21.15.84:0/3312913465 learned_addr learned my addr 172.21.15.84:0/3312913465 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:06.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.251+0000 7f8427fff640 1 --2- 172.21.15.84:0/3312913465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8428070950 0x7f8428140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:06.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.251+0000 7f84277fe640 1 --2- 172.21.15.84:0/3312913465 >> v2:172.21.15.157:3300/0 conn(0x7f8428160c80 0x7f8428161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:06.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.252+0000 7f8427fff640 1 -- 172.21.15.84:0/3312913465 >> v2:172.21.15.157:3300/0 conn(0x7f8428160c80 msgr2=0x7f8428161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:06.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.252+0000 7f8427fff640 1 --2- 172.21.15.84:0/3312913465 >> v2:172.21.15.157:3300/0 conn(0x7f8428160c80 0x7f8428161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.252+0000 7f8427fff640 1 -- 172.21.15.84:0/3312913465 >> v1:172.21.15.84:6789/0 conn(0x7f84280ad7a0 legacy=0x7f8428070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:06.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.252+0000 7f8427fff640 1 -- 172.21.15.84:0/3312913465 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84280aea40 con 0x7f8428070950 2023-10-16T11:27:06.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.254+0000 7f8427fff640 1 --2- 172.21.15.84:0/3312913465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8428070950 0x7f8428140400 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8420009d90 tx=0x7f842002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e925806995915fa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:06.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.257+0000 7f8426ffd640 1 -- 172.21.15.84:0/3312913465 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842002cc50 con 0x7f8428070950 2023-10-16T11:27:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.258+0000 7f8426ffd640 1 -- 172.21.15.84:0/3312913465 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842002cdb0 con 0x7f8428070950 2023-10-16T11:27:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.258+0000 7f8426ffd640 1 -- 172.21.15.84:0/3312913465 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420034660 con 0x7f8428070950 2023-10-16T11:27:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.258+0000 7f842c9ef640 1 -- 172.21.15.84:0/3312913465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8428070950 msgr2=0x7f8428140400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.258+0000 7f842c9ef640 1 --2- 172.21.15.84:0/3312913465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8428070950 0x7f8428140400 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8420009d90 tx=0x7f842002bea0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.258+0000 7f842c9ef640 1 -- 172.21.15.84:0/3312913465 shutdown_connections 2023-10-16T11:27:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.258+0000 7f842c9ef640 1 --2- 172.21.15.84:0/3312913465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8428070950 0x7f8428140400 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.258+0000 7f842c9ef640 1 --2- 172.21.15.84:0/3312913465 >> v2:172.21.15.157:3300/0 conn(0x7f8428160c80 0x7f8428161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.258+0000 7f842c9ef640 1 -- 172.21.15.84:0/3312913465 >> 172.21.15.84:0/3312913465 conn(0x7f84280cc230 msgr2=0x7f84280cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:06.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.258+0000 7f842c9ef640 1 -- 172.21.15.84:0/3312913465 shutdown_connections 2023-10-16T11:27:06.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.258+0000 7f842c9ef640 1 -- 172.21.15.84:0/3312913465 wait complete. 2023-10-16T11:27:06.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.260+0000 7f842c9ef640 1 Processor -- start 2023-10-16T11:27:06.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.260+0000 7f842c9ef640 1 -- start start 2023-10-16T11:27:06.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.261+0000 7f842c9ef640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8428070950 0x7f84281f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:06.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.261+0000 7f842c9ef640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84280ad7a0 0x7f8428151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:06.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.261+0000 7f842c9ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8428160c80 0x7f842815f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:06.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.261+0000 7f842c9ef640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84280c5300 con 0x7f84280ad7a0 2023-10-16T11:27:06.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.261+0000 7f842c9ef640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84280c4780 con 0x7f8428160c80 2023-10-16T11:27:06.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.261+0000 7f842c9ef640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84280c4a60 con 0x7f8428070950 2023-10-16T11:27:06.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.261+0000 7f841ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84280ad7a0 0x7f8428151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:06.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.261+0000 7f841ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84280ad7a0 0x7f8428151040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43992/0 (socket says 172.21.15.84:43992) 2023-10-16T11:27:06.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.261+0000 7f841ffff640 1 -- 172.21.15.84:0/221550838 learned_addr learned my addr 172.21.15.84:0/221550838 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:06.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.262+0000 7f841ffff640 1 -- 172.21.15.84:0/221550838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8428070950 msgr2=0x7f84281f24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:06.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.262+0000 7f84277fe640 1 --2- 172.21.15.84:0/221550838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8428070950 0x7f84281f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:06.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.262+0000 7f8427fff640 1 --2- 172.21.15.84:0/221550838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8428160c80 0x7f842815f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:06.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.262+0000 7f841ffff640 1 --2- 172.21.15.84:0/221550838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8428070950 0x7f84281f24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.262+0000 7f841ffff640 1 -- 172.21.15.84:0/221550838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8428160c80 msgr2=0x7f842815f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:06.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.262+0000 7f841ffff640 1 --2- 172.21.15.84:0/221550838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8428160c80 0x7f842815f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.262+0000 7f841ffff640 1 -- 172.21.15.84:0/221550838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8420009a70 con 0x7f84280ad7a0 2023-10-16T11:27:06.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.262+0000 7f841ffff640 1 --2- 172.21.15.84:0/221550838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84280ad7a0 0x7f8428151040 secure :-1 s=READY pgs=3315 cs=0 l=1 rev1=1 crypto rx=0x7f841000c400 tx=0x7f841000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:06.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.262+0000 7f8424ff9640 1 -- 172.21.15.84:0/221550838 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410005ce0 con 0x7f84280ad7a0 2023-10-16T11:27:06.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.262+0000 7f8424ff9640 1 -- 172.21.15.84:0/221550838 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8410005e40 con 0x7f84280ad7a0 2023-10-16T11:27:06.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.262+0000 7f8424ff9640 1 -- 172.21.15.84:0/221550838 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84100072d0 con 0x7f84280ad7a0 2023-10-16T11:27:06.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.263+0000 7f842c9ef640 1 -- 172.21.15.84:0/221550838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84280c4d40 con 0x7f84280ad7a0 2023-10-16T11:27:06.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.263+0000 7f842c9ef640 1 -- 172.21.15.84:0/221550838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8428151580 con 0x7f84280ad7a0 2023-10-16T11:27:06.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.264+0000 7f842c9ef640 1 -- 172.21.15.84:0/221550838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84280b12f0 con 0x7f84280ad7a0 2023-10-16T11:27:06.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.265+0000 7f8424ff9640 1 -- 172.21.15.84:0/221550838 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 157) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f8410007430 con 0x7f84280ad7a0 2023-10-16T11:27:06.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.265+0000 7f8424ff9640 1 -- 172.21.15.84:0/221550838 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f8410088ad0 con 0x7f84280ad7a0 2023-10-16T11:27:06.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.270+0000 7f8424ff9640 1 -- 172.21.15.84:0/221550838 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f84280b12f0 con 0x7f84280ad7a0 2023-10-16T11:27:06.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.448+0000 7f842c9ef640 1 -- 172.21.15.84:0/221550838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f84280b12f0 con 0x7f84280ad7a0 2023-10-16T11:27:06.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.448+0000 7f8424ff9640 1 -- 172.21.15.84:0/221550838 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v259) v1 ==== 100+0+11 (secure 0 0 0) 0x7f84280b12f0 con 0x7f84280ad7a0 2023-10-16T11:27:06.450 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:27:06.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.451+0000 7f842c9ef640 1 -- 172.21.15.84:0/221550838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84280ad7a0 msgr2=0x7f8428151040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:06.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.451+0000 7f842c9ef640 1 --2- 172.21.15.84:0/221550838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84280ad7a0 0x7f8428151040 secure :-1 s=READY pgs=3315 cs=0 l=1 rev1=1 crypto rx=0x7f841000c400 tx=0x7f841000c8c0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.452+0000 7f842c9ef640 1 -- 172.21.15.84:0/221550838 shutdown_connections 2023-10-16T11:27:06.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.452+0000 7f842c9ef640 1 --2- 172.21.15.84:0/221550838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8428160c80 0x7f842815f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.452+0000 7f842c9ef640 1 --2- 172.21.15.84:0/221550838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84280ad7a0 0x7f8428151040 unknown :-1 s=CLOSED pgs=3315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.452+0000 7f842c9ef640 1 --2- 172.21.15.84:0/221550838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8428070950 0x7f84281f24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.452+0000 7f842c9ef640 1 -- 172.21.15.84:0/221550838 >> 172.21.15.84:0/221550838 conn(0x7f84280cc230 msgr2=0x7f84281415c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:06.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.452+0000 7f842c9ef640 1 -- 172.21.15.84:0/221550838 shutdown_connections 2023-10-16T11:27:06.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.452+0000 7f842c9ef640 1 -- 172.21.15.84:0/221550838 wait complete. 2023-10-16T11:27:06.470 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:27:06.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.670+0000 7fa6d43f0640 1 Processor -- start 2023-10-16T11:27:06.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.670+0000 7fa6d43f0640 1 -- start start 2023-10-16T11:27:06.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.671+0000 7fa6d43f0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa6cc159f00 0x7fa6cc0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:06.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.671+0000 7fa6d43f0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6cc15d880 0x7fa6cc0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:06.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.671+0000 7fa6d43f0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6cc06f7f0 con 0x7fa6cc159b20 2023-10-16T11:27:06.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.671+0000 7fa6d43f0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6cc06c5e0 con 0x7fa6cc159f00 2023-10-16T11:27:06.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.671+0000 7fa6d43f0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc06c8c0 con 0x7fa6cc15d880 2023-10-16T11:27:06.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.672+0000 7fa6d218c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa6cc159b20 0x7fa6cc06bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57296/0 (socket says 172.21.15.84:57296) 2023-10-16T11:27:06.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.672+0000 7fa6d218c640 1 -- 172.21.15.84:0/3897258533 learned_addr learned my addr 172.21.15.84:0/3897258533 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:06.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.672+0000 7fa6d198b640 1 --2- 172.21.15.84:0/3897258533 >> v2:172.21.15.84:3300/0 conn(0x7fa6cc159f00 0x7fa6cc0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:06.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.672+0000 7fa6d198b640 1 -- 172.21.15.84:0/3897258533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6cc15d880 msgr2=0x7fa6cc0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:06.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.672+0000 7fa6d298d640 1 --2- 172.21.15.84:0/3897258533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6cc15d880 0x7fa6cc0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:06.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.672+0000 7fa6d198b640 1 --2- 172.21.15.84:0/3897258533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6cc15d880 0x7fa6cc0b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.672+0000 7fa6d198b640 1 -- 172.21.15.84:0/3897258533 >> v1:172.21.15.84:6789/0 conn(0x7fa6cc159b20 legacy=0x7fa6cc06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:06.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.672+0000 7fa6d198b640 1 -- 172.21.15.84:0/3897258533 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6cc06cba0 con 0x7fa6cc159f00 2023-10-16T11:27:06.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.672+0000 7fa6d118a640 1 -- 172.21.15.84:0/3897258533 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282515658 0 0) 0x7fa6cc06f7f0 con 0x7fa6cc159b20 2023-10-16T11:27:06.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.672+0000 7fa6d298d640 1 --2- 172.21.15.84:0/3897258533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6cc15d880 0x7fa6cc0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:06.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.673+0000 7fa6d198b640 1 --2- 172.21.15.84:0/3897258533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc159f00 0x7fa6cc0b0410 secure :-1 s=READY pgs=3317 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0097a0 tx=0x7fa6bc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd54e6bc6cb5611 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:06.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.673+0000 7fa6d118a640 1 -- 172.21.15.84:0/3897258533 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc02d040 con 0x7fa6cc159f00 2023-10-16T11:27:06.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.674+0000 7fa6d118a640 1 -- 172.21.15.84:0/3897258533 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc031810 con 0x7fa6cc159f00 2023-10-16T11:27:06.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.674+0000 7fa6d118a640 1 -- 172.21.15.84:0/3897258533 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc031ba0 con 0x7fa6cc159f00 2023-10-16T11:27:06.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.674+0000 7fa6d43f0640 1 -- 172.21.15.84:0/3897258533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc159f00 msgr2=0x7fa6cc0b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:06.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.674+0000 7fa6d43f0640 1 --2- 172.21.15.84:0/3897258533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc159f00 0x7fa6cc0b0410 secure :-1 s=READY pgs=3317 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0097a0 tx=0x7fa6bc02b8b0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.674+0000 7fa6d43f0640 1 -- 172.21.15.84:0/3897258533 shutdown_connections 2023-10-16T11:27:06.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.674+0000 7fa6d43f0640 1 --2- 172.21.15.84:0/3897258533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6cc15d880 0x7fa6cc0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.674+0000 7fa6d43f0640 1 --2- 172.21.15.84:0/3897258533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc159f00 0x7fa6cc0b0410 unknown :-1 s=CLOSED pgs=3317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.674+0000 7fa6d43f0640 1 -- 172.21.15.84:0/3897258533 >> 172.21.15.84:0/3897258533 conn(0x7fa6cc0cb410 msgr2=0x7fa6cc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:06.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.674+0000 7fa6d43f0640 1 -- 172.21.15.84:0/3897258533 shutdown_connections 2023-10-16T11:27:06.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.674+0000 7fa6d43f0640 1 -- 172.21.15.84:0/3897258533 wait complete. 2023-10-16T11:27:06.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.676+0000 7fa6d43f0640 1 Processor -- start 2023-10-16T11:27:06.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.676+0000 7fa6d43f0640 1 -- start start 2023-10-16T11:27:06.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.677+0000 7fa6d43f0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6cc159b20 0x7fa6cc1570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:06.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.677+0000 7fa6d43f0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6cc159f00 0x7fa6cc151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:06.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.677+0000 7fa6d43f0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc15d880 0x7fa6cc15caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:06.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.677+0000 7fa6d43f0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc151da0 con 0x7fa6cc15d880 2023-10-16T11:27:06.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.677+0000 7fa6d43f0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc1576a0 con 0x7fa6cc159b20 2023-10-16T11:27:06.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.677+0000 7fa6d43f0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc157980 con 0x7fa6cc159f00 2023-10-16T11:27:06.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.677+0000 7fa6d298d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc15d880 0x7fa6cc15caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:06.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.677+0000 7fa6d298d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc15d880 0x7fa6cc15caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44002/0 (socket says 172.21.15.84:44002) 2023-10-16T11:27:06.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.677+0000 7fa6d298d640 1 -- 172.21.15.84:0/2610417308 learned_addr learned my addr 172.21.15.84:0/2610417308 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:06.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.678+0000 7fa6d298d640 1 -- 172.21.15.84:0/2610417308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6cc159f00 msgr2=0x7fa6cc151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:06.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.678+0000 7fa6d298d640 1 --2- 172.21.15.84:0/2610417308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6cc159f00 0x7fa6cc151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.678+0000 7fa6d298d640 1 -- 172.21.15.84:0/2610417308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6cc159b20 msgr2=0x7fa6cc1570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:06.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.678+0000 7fa6d298d640 1 --2- 172.21.15.84:0/2610417308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6cc159b20 0x7fa6cc1570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.678+0000 7fa6d298d640 1 -- 172.21.15.84:0/2610417308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc009400 con 0x7fa6cc15d880 2023-10-16T11:27:06.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.678+0000 7fa6d298d640 1 --2- 172.21.15.84:0/2610417308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc15d880 0x7fa6cc15caf0 secure :-1 s=READY pgs=3318 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400cd50 tx=0x7fa6c400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:06.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.678+0000 7fa6c2ffd640 1 -- 172.21.15.84:0/2610417308 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c400f680 con 0x7fa6cc15d880 2023-10-16T11:27:06.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.678+0000 7fa6c2ffd640 1 -- 172.21.15.84:0/2610417308 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c400f800 con 0x7fa6cc15d880 2023-10-16T11:27:06.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.678+0000 7fa6c2ffd640 1 -- 172.21.15.84:0/2610417308 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c400fbb0 con 0x7fa6cc15d880 2023-10-16T11:27:06.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.678+0000 7fa6d43f0640 1 -- 172.21.15.84:0/2610417308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6cc157c60 con 0x7fa6cc15d880 2023-10-16T11:27:06.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.678+0000 7fa6d43f0640 1 -- 172.21.15.84:0/2610417308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6cc158220 con 0x7fa6cc15d880 2023-10-16T11:27:06.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.680+0000 7fa6d43f0640 1 -- 172.21.15.84:0/2610417308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6cc158410 con 0x7fa6cc15d880 2023-10-16T11:27:06.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.680+0000 7fa6c2ffd640 1 -- 172.21.15.84:0/2610417308 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 158) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fa6c400b040 con 0x7fa6cc15d880 2023-10-16T11:27:06.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.681+0000 7fa6c2ffd640 1 --2- 172.21.15.84:0/2610417308 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fa6b006acc0 0x7fa6b006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:06.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.681+0000 7fa6c2ffd640 1 -- 172.21.15.84:0/2610417308 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fa6c4089fb0 con 0x7fa6cc15d880 2023-10-16T11:27:06.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.681+0000 7fa6d218c640 1 --2- 172.21.15.84:0/2610417308 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fa6b006acc0 0x7fa6b006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:06.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.682+0000 7fa6d218c640 1 --2- 172.21.15.84:0/2610417308 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fa6b006acc0 0x7fa6b006d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8007fd0 tx=0x7fa6c8007920 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:06.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.685+0000 7fa6c2ffd640 1 -- 172.21.15.84:0/2610417308 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa6c40bcfe0 con 0x7fa6cc15d880 2023-10-16T11:27:06.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.897+0000 7fa6d43f0640 1 -- 172.21.15.84:0/2610417308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6cc04e160 con 0x7fa6cc15d880 2023-10-16T11:27:06.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.898+0000 7fa6c2ffd640 1 -- 172.21.15.84:0/2610417308 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+4929 (secure 0 0 0) 0x7fa6c401d090 con 0x7fa6cc15d880 2023-10-16T11:27:06.900 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:06.901 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":148,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/591944719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":591944719},{"type":"v1","addr":"172.21.15.84:6835","nonce":591944719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":142},{"gid":8152,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2371517609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2371517609},{"type":"v1","addr":"172.21.15.84:6837","nonce":2371517609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":143},{"gid":8161,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1079456313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1079456313},{"type":"v1","addr":"172.21.15.157:6834","nonce":1079456313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":143}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:26:41.908585+0000","modified":"2023-10-16T11:26:44.991714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12987":{"gid":12987,"name":"b","rank":0,"incarnation":144,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3899307325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3899307325},{"type":"v1","addr":"172.21.15.157:6835","nonce":3899307325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-16T11:27:06.901 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 148 2023-10-16T11:27:06.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.901+0000 7fa6d43f0640 1 -- 172.21.15.84:0/2610417308 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fa6b006acc0 msgr2=0x7fa6b006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:06.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.901+0000 7fa6d43f0640 1 --2- 172.21.15.84:0/2610417308 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fa6b006acc0 0x7fa6b006d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8007fd0 tx=0x7fa6c8007920 comp rx=0 tx=0).stop 2023-10-16T11:27:06.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.901+0000 7fa6d43f0640 1 -- 172.21.15.84:0/2610417308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc15d880 msgr2=0x7fa6cc15caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:06.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.901+0000 7fa6d43f0640 1 --2- 172.21.15.84:0/2610417308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc15d880 0x7fa6cc15caf0 secure :-1 s=READY pgs=3318 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400cd50 tx=0x7fa6c400a650 comp rx=0 tx=0).stop 2023-10-16T11:27:06.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.901+0000 7fa6d43f0640 1 -- 172.21.15.84:0/2610417308 shutdown_connections 2023-10-16T11:27:06.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.901+0000 7fa6d43f0640 1 --2- 172.21.15.84:0/2610417308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc15d880 0x7fa6cc15caf0 unknown :-1 s=CLOSED pgs=3318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.901+0000 7fa6d43f0640 1 --2- 172.21.15.84:0/2610417308 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fa6b006acc0 0x7fa6b006d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.901+0000 7fa6d43f0640 1 --2- 172.21.15.84:0/2610417308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6cc159f00 0x7fa6cc151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.901+0000 7fa6d43f0640 1 --2- 172.21.15.84:0/2610417308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6cc159b20 0x7fa6cc1570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:06.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.901+0000 7fa6d43f0640 1 -- 172.21.15.84:0/2610417308 >> 172.21.15.84:0/2610417308 conn(0x7fa6cc0cb410 msgr2=0x7fa6cc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:06.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.901+0000 7fa6d43f0640 1 -- 172.21.15.84:0/2610417308 shutdown_connections 2023-10-16T11:27:06.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:06.901+0000 7fa6d43f0640 1 -- 172.21.15.84:0/2610417308 wait complete. 2023-10-16T11:27:06.919 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:27:07.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.122+0000 7f61610de640 1 Processor -- start 2023-10-16T11:27:07.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.123+0000 7f61610de640 1 -- start start 2023-10-16T11:27:07.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.124+0000 7f61610de640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f615c15ae80 0x7f615c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:07.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.124+0000 7f61610de640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f615c0ce390 0x7f615c070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:07.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.125+0000 7f61610de640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f615c15ed60 con 0x7f615c15aaa0 2023-10-16T11:27:07.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.125+0000 7f61610de640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f615c06b810 con 0x7f615c15ae80 2023-10-16T11:27:07.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.125+0000 7f61610de640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615c0707e0 con 0x7f615c0ce390 2023-10-16T11:27:07.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.125+0000 7f615ad9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f615c15aaa0 0x7f615c0cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57302/0 (socket says 172.21.15.84:57302) 2023-10-16T11:27:07.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.125+0000 7f615ad9d640 1 -- 172.21.15.84:0/2998280668 learned_addr learned my addr 172.21.15.84:0/2998280668 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:07.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.125+0000 7f615a59c640 1 -- 172.21.15.84:0/2998280668 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782385166 0 0) 0x7f615c15ed60 con 0x7f615c15aaa0 2023-10-16T11:27:07.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.125+0000 7f615b59e640 1 --2- 172.21.15.84:0/2998280668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f615c0ce390 0x7f615c070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:07.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.125+0000 7f615a59c640 1 -- 172.21.15.84:0/2998280668 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6148003520 con 0x7f615c15aaa0 2023-10-16T11:27:07.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.125+0000 7f615259c640 1 --2- 172.21.15.84:0/2998280668 >> v2:172.21.15.157:3300/0 conn(0x7f615c15ae80 0x7f615c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:07.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.125+0000 7f615a59c640 1 -- 172.21.15.84:0/2998280668 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3252613607 0 0) 0x7f6148003520 con 0x7f615c15aaa0 2023-10-16T11:27:07.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.125+0000 7f615a59c640 1 -- 172.21.15.84:0/2998280668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f615c0ce390 msgr2=0x7f615c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:07.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.125+0000 7f615a59c640 1 --2- 172.21.15.84:0/2998280668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f615c0ce390 0x7f615c070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.125+0000 7f615a59c640 1 -- 172.21.15.84:0/2998280668 >> v2:172.21.15.157:3300/0 conn(0x7f615c15ae80 msgr2=0x7f615c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:07.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.125+0000 7f615a59c640 1 --2- 172.21.15.84:0/2998280668 >> v2:172.21.15.157:3300/0 conn(0x7f615c15ae80 0x7f615c15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.126+0000 7f615a59c640 1 -- 172.21.15.84:0/2998280668 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615c15ef90 con 0x7f615c15aaa0 2023-10-16T11:27:07.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.126+0000 7f615a59c640 1 -- 172.21.15.84:0/2998280668 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f614c001a10 con 0x7f615c15aaa0 2023-10-16T11:27:07.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.126+0000 7f615a59c640 1 -- 172.21.15.84:0/2998280668 >> v1:172.21.15.84:6789/0 conn(0x7f615c15aaa0 legacy=0x7f615c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:07.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.126+0000 7f615a59c640 1 --2- 172.21.15.84:0/2998280668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f615c0ce390 0x7f6148003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:07.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.126+0000 7f615a59c640 1 --2- 172.21.15.84:0/2998280668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6148003fa0 0x7f61480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:07.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.126+0000 7f615a59c640 1 --2- 172.21.15.84:0/2998280668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148004d00 0x7f6148005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:07.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.126+0000 7f615a59c640 1 -- 172.21.15.84:0/2998280668 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c06b810 con 0x7f6148004d00 2023-10-16T11:27:07.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.126+0000 7f615a59c640 1 -- 172.21.15.84:0/2998280668 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c0707e0 con 0x7f6148003fa0 2023-10-16T11:27:07.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.126+0000 7f615a59c640 1 -- 172.21.15.84:0/2998280668 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614800bb90 con 0x7f615c0ce390 2023-10-16T11:27:07.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.126+0000 7f615b59e640 1 --2- 172.21.15.84:0/2998280668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148004d00 0x7f6148005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:07.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.126+0000 7f615b59e640 1 -- 172.21.15.84:0/2998280668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f615c0ce390 msgr2=0x7f6148003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:07.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.126+0000 7f615b59e640 1 --2- 172.21.15.84:0/2998280668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f615c0ce390 0x7f6148003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.126+0000 7f615b59e640 1 -- 172.21.15.84:0/2998280668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6148003fa0 msgr2=0x7f61480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:07.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.126+0000 7f615b59e640 1 --2- 172.21.15.84:0/2998280668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6148003fa0 0x7f61480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.126+0000 7f615b59e640 1 -- 172.21.15.84:0/2998280668 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6148003520 con 0x7f6148004d00 2023-10-16T11:27:07.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.127+0000 7f615b59e640 1 --2- 172.21.15.84:0/2998280668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148004d00 0x7f6148005180 secure :-1 s=READY pgs=3320 cs=0 l=1 rev1=1 crypto rx=0x7f615400ba30 tx=0x7f61480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d25d35b48185118 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:07.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.128+0000 7f615a59c640 1 -- 172.21.15.84:0/2998280668 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154032680 con 0x7f6148004d00 2023-10-16T11:27:07.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.128+0000 7f615a59c640 1 -- 172.21.15.84:0/2998280668 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61540327e0 con 0x7f6148004d00 2023-10-16T11:27:07.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.128+0000 7f615a59c640 1 -- 172.21.15.84:0/2998280668 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154032b70 con 0x7f6148004d00 2023-10-16T11:27:07.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.128+0000 7f61610de640 1 -- 172.21.15.84:0/2998280668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148004d00 msgr2=0x7f6148005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:07.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.128+0000 7f61610de640 1 --2- 172.21.15.84:0/2998280668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148004d00 0x7f6148005180 secure :-1 s=READY pgs=3320 cs=0 l=1 rev1=1 crypto rx=0x7f615400ba30 tx=0x7f61480057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.128+0000 7f615b59e640 1 -- 172.21.15.84:0/2998280668 reap_dead start 2023-10-16T11:27:07.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.128+0000 7f61610de640 1 -- 172.21.15.84:0/2998280668 shutdown_connections 2023-10-16T11:27:07.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.128+0000 7f61610de640 1 -- 172.21.15.84:0/2998280668 >> 172.21.15.84:0/2998280668 conn(0x7f615c0cb410 msgr2=0x7f615c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:07.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.128+0000 7f61610de640 1 -- 172.21.15.84:0/2998280668 shutdown_connections 2023-10-16T11:27:07.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.128+0000 7f61610de640 1 -- 172.21.15.84:0/2998280668 wait complete. 2023-10-16T11:27:07.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.130+0000 7f61610de640 1 Processor -- start 2023-10-16T11:27:07.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.130+0000 7f61610de640 1 -- start start 2023-10-16T11:27:07.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.131+0000 7f61610de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148004d00 0x7f615c158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:07.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.131+0000 7f61610de640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f615c0ae540 0x7f615c1574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:07.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.131+0000 7f61610de640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f615c0c5000 0x7f615c157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:07.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.131+0000 7f61610de640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c0af070 con 0x7f6148004d00 2023-10-16T11:27:07.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.131+0000 7f61610de640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c15c7d0 con 0x7f615c0c5000 2023-10-16T11:27:07.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.131+0000 7f61610de640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615c14f1d0 con 0x7f615c0ae540 2023-10-16T11:27:07.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.132+0000 7f615ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148004d00 0x7f615c158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:07.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.132+0000 7f615ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148004d00 0x7f615c158230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44030/0 (socket says 172.21.15.84:44030) 2023-10-16T11:27:07.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.132+0000 7f615ad9d640 1 -- 172.21.15.84:0/2475744064 learned_addr learned my addr 172.21.15.84:0/2475744064 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:07.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.132+0000 7f615ad9d640 1 -- 172.21.15.84:0/2475744064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f615c0ae540 msgr2=0x7f615c1574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:07.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.132+0000 7f615259c640 1 --2- 172.21.15.84:0/2475744064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f615c0ae540 0x7f615c1574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:07.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.132+0000 7f615ad9d640 1 --2- 172.21.15.84:0/2475744064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f615c0ae540 0x7f615c1574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.132+0000 7f615ad9d640 1 -- 172.21.15.84:0/2475744064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f615c0c5000 msgr2=0x7f615c157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:07.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.132+0000 7f615b59e640 1 --2- 172.21.15.84:0/2475744064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f615c0c5000 0x7f615c157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:07.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.132+0000 7f615ad9d640 1 --2- 172.21.15.84:0/2475744064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f615c0c5000 0x7f615c157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.132+0000 7f615ad9d640 1 -- 172.21.15.84:0/2475744064 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615c0aeab0 con 0x7f6148004d00 2023-10-16T11:27:07.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.132+0000 7f615ad9d640 1 --2- 172.21.15.84:0/2475744064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148004d00 0x7f615c158230 secure :-1 s=READY pgs=3321 cs=0 l=1 rev1=1 crypto rx=0x7f614c00cad0 tx=0x7f614c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:07.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.133+0000 7f6153fff640 1 -- 172.21.15.84:0/2475744064 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c00a7c0 con 0x7f6148004d00 2023-10-16T11:27:07.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.133+0000 7f6153fff640 1 -- 172.21.15.84:0/2475744064 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c00a920 con 0x7f6148004d00 2023-10-16T11:27:07.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.133+0000 7f6153fff640 1 -- 172.21.15.84:0/2475744064 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c00acd0 con 0x7f6148004d00 2023-10-16T11:27:07.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.133+0000 7f61610de640 1 -- 172.21.15.84:0/2475744064 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f615c0aed90 con 0x7f6148004d00 2023-10-16T11:27:07.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.133+0000 7f61610de640 1 -- 172.21.15.84:0/2475744064 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f615c0c24c0 con 0x7f6148004d00 2023-10-16T11:27:07.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.134+0000 7f61610de640 1 -- 172.21.15.84:0/2475744064 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f615c04e160 con 0x7f6148004d00 2023-10-16T11:27:07.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.135+0000 7f6153fff640 1 -- 172.21.15.84:0/2475744064 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 158) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f614c0070d0 con 0x7f6148004d00 2023-10-16T11:27:07.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.136+0000 7f6153fff640 1 --2- 172.21.15.84:0/2475744064 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f613c06ad40 0x7f613c06d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:07.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.136+0000 7f6153fff640 1 -- 172.21.15.84:0/2475744064 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f614c089250 con 0x7f6148004d00 2023-10-16T11:27:07.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.136+0000 7f615259c640 1 --2- 172.21.15.84:0/2475744064 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f613c06ad40 0x7f613c06d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:07.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.137+0000 7f615259c640 1 --2- 172.21.15.84:0/2475744064 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f613c06ad40 0x7f613c06d200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f615c1594a0 tx=0x7f6144006d30 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:07.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.140+0000 7f6153fff640 1 -- 172.21.15.84:0/2475744064 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f614c054270 con 0x7f6148004d00 2023-10-16T11:27:07.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.329+0000 7f61610de640 1 -- 172.21.15.84:0/2475744064 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f615c0c8cc0 con 0x7f6148004d00 2023-10-16T11:27:07.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.332+0000 7f6153fff640 1 -- 172.21.15.84:0/2475744064 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v259) v1 ==== 74+0+17793 (secure 0 0 0) 0x7f614c018090 con 0x7f6148004d00 2023-10-16T11:27:07.334 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:07.335 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":259,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:27:05.495073+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:26:39.860053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-16T11:26:40.916519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1625282261":"2023-10-17T11:27:05.494979+0000","172.21.15.84:6833/2988856492":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/765814920":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/2899443560":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/3002614854":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/2731742209":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/907959269":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/1863310536":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/847549599":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/972005011":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/1934386371":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/3540589884":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/2370234804":"2023-10-17T11:26:57.151307+0000","172.21.15.84:6833/2330196255":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/1650008535":"2023-10-17T11:26:57.151307+0000","172.21.15.84:6832/2330196255":"2023-10-17T11:26:57.151307+0000","172.21.15.84:6832/2988856492":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/3247488662":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/2440426242":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/570042115":"2023-10-16T12:26:40.168744+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:27:07.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.335+0000 7f61610de640 1 -- 172.21.15.84:0/2475744064 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f613c06ad40 msgr2=0x7f613c06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:07.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.335+0000 7f61610de640 1 --2- 172.21.15.84:0/2475744064 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f613c06ad40 0x7f613c06d200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f615c1594a0 tx=0x7f6144006d30 comp rx=0 tx=0).stop 2023-10-16T11:27:07.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.335+0000 7f61610de640 1 -- 172.21.15.84:0/2475744064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148004d00 msgr2=0x7f615c158230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:07.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.335+0000 7f61610de640 1 --2- 172.21.15.84:0/2475744064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148004d00 0x7f615c158230 secure :-1 s=READY pgs=3321 cs=0 l=1 rev1=1 crypto rx=0x7f614c00cad0 tx=0x7f614c00cf90 comp rx=0 tx=0).stop 2023-10-16T11:27:07.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.335+0000 7f61610de640 1 -- 172.21.15.84:0/2475744064 shutdown_connections 2023-10-16T11:27:07.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.335+0000 7f61610de640 1 --2- 172.21.15.84:0/2475744064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f615c0c5000 0x7f615c157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.335+0000 7f61610de640 1 --2- 172.21.15.84:0/2475744064 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f613c06ad40 0x7f613c06d200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.335+0000 7f61610de640 1 --2- 172.21.15.84:0/2475744064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f615c0ae540 0x7f615c1574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.335+0000 7f61610de640 1 --2- 172.21.15.84:0/2475744064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148004d00 0x7f615c158230 unknown :-1 s=CLOSED pgs=3321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.335+0000 7f61610de640 1 -- 172.21.15.84:0/2475744064 >> 172.21.15.84:0/2475744064 conn(0x7f615c0cb410 msgr2=0x7f615c1588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:07.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.336+0000 7f61610de640 1 -- 172.21.15.84:0/2475744064 shutdown_connections 2023-10-16T11:27:07.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.336+0000 7f61610de640 1 -- 172.21.15.84:0/2475744064 wait complete. 2023-10-16T11:27:07.352 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T11:27:07.353 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:27:07.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.549+0000 7fcb2e11d640 1 Processor -- start 2023-10-16T11:27:07.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.550+0000 7fcb2e11d640 1 -- start start 2023-10-16T11:27:07.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.551+0000 7fcb2e11d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcb2806b5c0 0x7fcb280cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:07.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.551+0000 7fcb2e11d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb280b24a0 0x7fcb280b2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:07.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.551+0000 7fcb2e11d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb280d0180 con 0x7fcb280afef0 2023-10-16T11:27:07.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.551+0000 7fcb2e11d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb280b2e50 con 0x7fcb2806b5c0 2023-10-16T11:27:07.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.551+0000 7fcb2e11d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb280cfea0 con 0x7fcb280b24a0 2023-10-16T11:27:07.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.551+0000 7fcb277fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fcb280afef0 0x7fcb2806aeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57316/0 (socket says 172.21.15.84:57316) 2023-10-16T11:27:07.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.551+0000 7fcb277fe640 1 -- 172.21.15.84:0/1168468574 learned_addr learned my addr 172.21.15.84:0/1168468574 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:07.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.552+0000 7fcb26ffd640 1 --2- 172.21.15.84:0/1168468574 >> v2:172.21.15.157:3300/0 conn(0x7fcb2806b5c0 0x7fcb280cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:07.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.552+0000 7fcb267fc640 1 -- 172.21.15.84:0/1168468574 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2907867717 0 0) 0x7fcb280d0180 con 0x7fcb280afef0 2023-10-16T11:27:07.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.552+0000 7fcb27fff640 1 --2- 172.21.15.84:0/1168468574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb280b24a0 0x7fcb280b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:07.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.552+0000 7fcb267fc640 1 -- 172.21.15.84:0/1168468574 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb14003520 con 0x7fcb280afef0 2023-10-16T11:27:07.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.552+0000 7fcb267fc640 1 -- 172.21.15.84:0/1168468574 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2374409279 0 0) 0x7fcb14003520 con 0x7fcb280afef0 2023-10-16T11:27:07.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.552+0000 7fcb267fc640 1 -- 172.21.15.84:0/1168468574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb280b24a0 msgr2=0x7fcb280b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:07.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.552+0000 7fcb267fc640 1 --2- 172.21.15.84:0/1168468574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb280b24a0 0x7fcb280b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.552+0000 7fcb267fc640 1 -- 172.21.15.84:0/1168468574 >> v2:172.21.15.157:3300/0 conn(0x7fcb2806b5c0 msgr2=0x7fcb280cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:07.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.552+0000 7fcb267fc640 1 --2- 172.21.15.84:0/1168468574 >> v2:172.21.15.157:3300/0 conn(0x7fcb2806b5c0 0x7fcb280cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.552+0000 7fcb267fc640 1 -- 172.21.15.84:0/1168468574 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb280b04d0 con 0x7fcb280afef0 2023-10-16T11:27:07.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.552+0000 7fcb267fc640 1 -- 172.21.15.84:0/1168468574 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fcb100017f0 con 0x7fcb280afef0 2023-10-16T11:27:07.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.552+0000 7fcb27fff640 1 --2- 172.21.15.84:0/1168468574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb280b24a0 0x7fcb280b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:27:07.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.552+0000 7fcb267fc640 1 -- 172.21.15.84:0/1168468574 >> v1:172.21.15.84:6789/0 conn(0x7fcb280afef0 legacy=0x7fcb2806aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:07.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.552+0000 7fcb267fc640 1 --2- 172.21.15.84:0/1168468574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb14003970 0x7fcb14003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:07.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.553+0000 7fcb267fc640 1 --2- 172.21.15.84:0/1168468574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb140047c0 0x7fcb14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:07.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.553+0000 7fcb267fc640 1 --2- 172.21.15.84:0/1168468574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb280b24a0 0x7fcb14005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:07.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.553+0000 7fcb267fc640 1 -- 172.21.15.84:0/1168468574 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb280b2e50 con 0x7fcb140047c0 2023-10-16T11:27:07.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.553+0000 7fcb267fc640 1 -- 172.21.15.84:0/1168468574 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb280cfea0 con 0x7fcb14003970 2023-10-16T11:27:07.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.553+0000 7fcb267fc640 1 -- 172.21.15.84:0/1168468574 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1400ba80 con 0x7fcb280b24a0 2023-10-16T11:27:07.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.553+0000 7fcb26ffd640 1 --2- 172.21.15.84:0/1168468574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb140047c0 0x7fcb14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:07.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.553+0000 7fcb26ffd640 1 -- 172.21.15.84:0/1168468574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb280b24a0 msgr2=0x7fcb14005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:07.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.553+0000 7fcb26ffd640 1 --2- 172.21.15.84:0/1168468574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb280b24a0 0x7fcb14005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.553+0000 7fcb26ffd640 1 -- 172.21.15.84:0/1168468574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb14003970 msgr2=0x7fcb14003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:07.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.553+0000 7fcb26ffd640 1 --2- 172.21.15.84:0/1168468574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb14003970 0x7fcb14003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.553+0000 7fcb26ffd640 1 -- 172.21.15.84:0/1168468574 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb14003520 con 0x7fcb140047c0 2023-10-16T11:27:07.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.554+0000 7fcb26ffd640 1 --2- 172.21.15.84:0/1168468574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb140047c0 0x7fcb14004c40 secure :-1 s=READY pgs=3323 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009f90 tx=0x7fcb140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ab630d573b96d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:07.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.554+0000 7fcb267fc640 1 -- 172.21.15.84:0/1168468574 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1802cbf0 con 0x7fcb140047c0 2023-10-16T11:27:07.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.554+0000 7fcb267fc640 1 -- 172.21.15.84:0/1168468574 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1800f040 con 0x7fcb140047c0 2023-10-16T11:27:07.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.555+0000 7fcb267fc640 1 -- 172.21.15.84:0/1168468574 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18034440 con 0x7fcb140047c0 2023-10-16T11:27:07.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.555+0000 7fcb2e11d640 1 -- 172.21.15.84:0/1168468574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb140047c0 msgr2=0x7fcb14004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:07.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.555+0000 7fcb2e11d640 1 --2- 172.21.15.84:0/1168468574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb140047c0 0x7fcb14004c40 secure :-1 s=READY pgs=3323 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009f90 tx=0x7fcb140057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.555+0000 7fcb27fff640 1 -- 172.21.15.84:0/1168468574 reap_dead start 2023-10-16T11:27:07.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.555+0000 7fcb2e11d640 1 -- 172.21.15.84:0/1168468574 shutdown_connections 2023-10-16T11:27:07.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.555+0000 7fcb2e11d640 1 -- 172.21.15.84:0/1168468574 >> 172.21.15.84:0/1168468574 conn(0x7fcb280d0ad0 msgr2=0x7fcb280d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:07.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.555+0000 7fcb2e11d640 1 -- 172.21.15.84:0/1168468574 shutdown_connections 2023-10-16T11:27:07.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.555+0000 7fcb2e11d640 1 -- 172.21.15.84:0/1168468574 wait complete. 2023-10-16T11:27:07.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.557+0000 7fcb2e11d640 1 Processor -- start 2023-10-16T11:27:07.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.557+0000 7fcb2e11d640 1 -- start start 2023-10-16T11:27:07.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.558+0000 7fcb2e11d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb140047c0 0x7fcb280c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:07.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.558+0000 7fcb2e11d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb280b3820 0x7fcb280c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:07.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.558+0000 7fcb2e11d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb280c69c0 0x7fcb280b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:07.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.558+0000 7fcb2e11d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb280b6980 con 0x7fcb140047c0 2023-10-16T11:27:07.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.558+0000 7fcb2e11d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb280c56c0 con 0x7fcb280b3820 2023-10-16T11:27:07.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.558+0000 7fcb2e11d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb280c59a0 con 0x7fcb280c69c0 2023-10-16T11:27:07.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.558+0000 7fcb277fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb140047c0 0x7fcb280c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:07.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.558+0000 7fcb277fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb140047c0 0x7fcb280c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44046/0 (socket says 172.21.15.84:44046) 2023-10-16T11:27:07.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.558+0000 7fcb277fe640 1 -- 172.21.15.84:0/3385369906 learned_addr learned my addr 172.21.15.84:0/3385369906 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:07.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.559+0000 7fcb27fff640 1 --2- 172.21.15.84:0/3385369906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb280c69c0 0x7fcb280b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:07.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.559+0000 7fcb277fe640 1 -- 172.21.15.84:0/3385369906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb280c69c0 msgr2=0x7fcb280b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:07.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.559+0000 7fcb26ffd640 1 --2- 172.21.15.84:0/3385369906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb280b3820 0x7fcb280c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:07.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.559+0000 7fcb277fe640 1 --2- 172.21.15.84:0/3385369906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb280c69c0 0x7fcb280b4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.559+0000 7fcb277fe640 1 -- 172.21.15.84:0/3385369906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb280b3820 msgr2=0x7fcb280c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:07.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.559+0000 7fcb277fe640 1 --2- 172.21.15.84:0/3385369906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb280b3820 0x7fcb280c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.559+0000 7fcb277fe640 1 -- 172.21.15.84:0/3385369906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb280c5c80 con 0x7fcb140047c0 2023-10-16T11:27:07.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.559+0000 7fcb26ffd640 1 --2- 172.21.15.84:0/3385369906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb280b3820 0x7fcb280c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:07.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.559+0000 7fcb277fe640 1 --2- 172.21.15.84:0/3385369906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb140047c0 0x7fcb280c8040 secure :-1 s=READY pgs=3324 cs=0 l=1 rev1=1 crypto rx=0x7fcb10000ef0 tx=0x7fcb1000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:07.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.559+0000 7fcb1ffff640 1 -- 172.21.15.84:0/3385369906 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10007480 con 0x7fcb140047c0 2023-10-16T11:27:07.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.559+0000 7fcb1ffff640 1 -- 172.21.15.84:0/3385369906 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb100075e0 con 0x7fcb140047c0 2023-10-16T11:27:07.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.559+0000 7fcb1ffff640 1 -- 172.21.15.84:0/3385369906 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb100059a0 con 0x7fcb140047c0 2023-10-16T11:27:07.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.559+0000 7fcb2e11d640 1 -- 172.21.15.84:0/3385369906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb280c5f60 con 0x7fcb140047c0 2023-10-16T11:27:07.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.560+0000 7fcb2e11d640 1 -- 172.21.15.84:0/3385369906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb281f23c0 con 0x7fcb140047c0 2023-10-16T11:27:07.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.561+0000 7fcb2e11d640 1 -- 172.21.15.84:0/3385369906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb280c6150 con 0x7fcb140047c0 2023-10-16T11:27:07.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.561+0000 7fcb1ffff640 1 -- 172.21.15.84:0/3385369906 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 158) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fcb10005b00 con 0x7fcb140047c0 2023-10-16T11:27:07.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.562+0000 7fcb1ffff640 1 --2- 172.21.15.84:0/3385369906 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fcb0806acc0 0x7fcb0806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:07.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.563+0000 7fcb1ffff640 1 -- 172.21.15.84:0/3385369906 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fcb10088f00 con 0x7fcb140047c0 2023-10-16T11:27:07.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.563+0000 7fcb26ffd640 1 --2- 172.21.15.84:0/3385369906 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fcb0806acc0 0x7fcb0806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:07.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.563+0000 7fcb26ffd640 1 --2- 172.21.15.84:0/3385369906 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fcb0806acc0 0x7fcb0806d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcb18036000 tx=0x7fcb1802e000 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:07.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.567+0000 7fcb1ffff640 1 -- 172.21.15.84:0/3385369906 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcb10053ea0 con 0x7fcb140047c0 2023-10-16T11:27:07.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.807+0000 7fcb2e11d640 1 -- 172.21.15.84:0/3385369906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb2804e160 con 0x7fcb140047c0 2023-10-16T11:27:07.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.808+0000 7fcb1ffff640 1 -- 172.21.15.84:0/3385369906 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+4929 (secure 0 0 0) 0x7fcb10017070 con 0x7fcb140047c0 2023-10-16T11:27:07.810 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:07.810 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":148,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/591944719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":591944719},{"type":"v1","addr":"172.21.15.84:6835","nonce":591944719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":142},{"gid":8152,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2371517609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2371517609},{"type":"v1","addr":"172.21.15.84:6837","nonce":2371517609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":143},{"gid":8161,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1079456313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1079456313},{"type":"v1","addr":"172.21.15.157:6834","nonce":1079456313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":143}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:26:41.908585+0000","modified":"2023-10-16T11:26:44.991714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12987":{"gid":12987,"name":"b","rank":0,"incarnation":144,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3899307325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3899307325},{"type":"v1","addr":"172.21.15.157:6835","nonce":3899307325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-16T11:27:07.810 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 148 2023-10-16T11:27:07.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.811+0000 7fcb2e11d640 1 -- 172.21.15.84:0/3385369906 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fcb0806acc0 msgr2=0x7fcb0806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:07.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.811+0000 7fcb2e11d640 1 --2- 172.21.15.84:0/3385369906 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fcb0806acc0 0x7fcb0806d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcb18036000 tx=0x7fcb1802e000 comp rx=0 tx=0).stop 2023-10-16T11:27:07.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.811+0000 7fcb2e11d640 1 -- 172.21.15.84:0/3385369906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb140047c0 msgr2=0x7fcb280c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:07.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.811+0000 7fcb2e11d640 1 --2- 172.21.15.84:0/3385369906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb140047c0 0x7fcb280c8040 secure :-1 s=READY pgs=3324 cs=0 l=1 rev1=1 crypto rx=0x7fcb10000ef0 tx=0x7fcb1000cc00 comp rx=0 tx=0).stop 2023-10-16T11:27:07.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.811+0000 7fcb2e11d640 1 -- 172.21.15.84:0/3385369906 shutdown_connections 2023-10-16T11:27:07.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.811+0000 7fcb2e11d640 1 --2- 172.21.15.84:0/3385369906 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fcb0806acc0 0x7fcb0806d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.811+0000 7fcb2e11d640 1 --2- 172.21.15.84:0/3385369906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb280c69c0 0x7fcb280b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.811+0000 7fcb2e11d640 1 --2- 172.21.15.84:0/3385369906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb280b3820 0x7fcb280c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.811+0000 7fcb2e11d640 1 --2- 172.21.15.84:0/3385369906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb140047c0 0x7fcb280c8040 unknown :-1 s=CLOSED pgs=3324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:07.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.811+0000 7fcb2e11d640 1 -- 172.21.15.84:0/3385369906 >> 172.21.15.84:0/3385369906 conn(0x7fcb280d0ad0 msgr2=0x7fcb280af2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:07.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.811+0000 7fcb2e11d640 1 -- 172.21.15.84:0/3385369906 shutdown_connections 2023-10-16T11:27:07.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:07.811+0000 7fcb2e11d640 1 -- 172.21.15.84:0/3385369906 wait complete. 2023-10-16T11:27:07.831 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:27:08.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.044+0000 7f0315861640 1 Processor -- start 2023-10-16T11:27:08.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.044+0000 7f0315861640 1 -- start start 2023-10-16T11:27:08.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.046+0000 7f0315861640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f031015ae80 0x7f031015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:08.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.046+0000 7f0315861640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f03100ce390 0x7f0310070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:08.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.046+0000 7f0315861640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f031015ed60 con 0x7f031015aaa0 2023-10-16T11:27:08.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.046+0000 7f0315861640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031006b810 con 0x7f03100ce390 2023-10-16T11:27:08.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.046+0000 7f0315861640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03100707e0 con 0x7f031015ae80 2023-10-16T11:27:08.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.046+0000 7f030f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f03100ce390 0x7f0310070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:08.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.046+0000 7f030f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f03100ce390 0x7f0310070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44068/0 (socket says 172.21.15.84:44068) 2023-10-16T11:27:08.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.046+0000 7f030f7fe640 1 -- 172.21.15.84:0/2990627954 learned_addr learned my addr 172.21.15.84:0/2990627954 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:08.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.046+0000 7f03067fc640 1 --2- 172.21.15.84:0/2990627954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f031015ae80 0x7f031015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:08.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.047+0000 7f030f7fe640 1 -- 172.21.15.84:0/2990627954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f031015ae80 msgr2=0x7f031015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:08.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.047+0000 7f030f7fe640 1 --2- 172.21.15.84:0/2990627954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f031015ae80 0x7f031015e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.047+0000 7f030f7fe640 1 -- 172.21.15.84:0/2990627954 >> v1:172.21.15.157:6789/0 conn(0x7f031015aaa0 legacy=0x7f03100cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:08.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.047+0000 7f030f7fe640 1 -- 172.21.15.84:0/2990627954 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031015ef90 con 0x7f03100ce390 2023-10-16T11:27:08.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.048+0000 7f030f7fe640 1 --2- 172.21.15.84:0/2990627954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03100ce390 0x7f0310070f50 secure :-1 s=READY pgs=3327 cs=0 l=1 rev1=1 crypto rx=0x7f0308009870 tx=0x7f030802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1062f88161fbc35 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:08.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.048+0000 7f030e7fc640 1 -- 172.21.15.84:0/2990627954 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308031640 con 0x7f03100ce390 2023-10-16T11:27:08.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.048+0000 7f030e7fc640 1 -- 172.21.15.84:0/2990627954 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03080317a0 con 0x7f03100ce390 2023-10-16T11:27:08.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.048+0000 7f030e7fc640 1 -- 172.21.15.84:0/2990627954 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308031b30 con 0x7f03100ce390 2023-10-16T11:27:08.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.049+0000 7f0315861640 1 -- 172.21.15.84:0/2990627954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03100ce390 msgr2=0x7f0310070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:08.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.049+0000 7f0315861640 1 --2- 172.21.15.84:0/2990627954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03100ce390 0x7f0310070f50 secure :-1 s=READY pgs=3327 cs=0 l=1 rev1=1 crypto rx=0x7f0308009870 tx=0x7f030802b8f0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.049+0000 7f0315861640 1 -- 172.21.15.84:0/2990627954 shutdown_connections 2023-10-16T11:27:08.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.049+0000 7f0315861640 1 --2- 172.21.15.84:0/2990627954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03100ce390 0x7f0310070f50 unknown :-1 s=CLOSED pgs=3327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.049+0000 7f0315861640 1 --2- 172.21.15.84:0/2990627954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f031015ae80 0x7f031015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.049+0000 7f0315861640 1 -- 172.21.15.84:0/2990627954 >> 172.21.15.84:0/2990627954 conn(0x7f03100cb410 msgr2=0x7f03100cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:08.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.049+0000 7f0315861640 1 -- 172.21.15.84:0/2990627954 shutdown_connections 2023-10-16T11:27:08.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.049+0000 7f0315861640 1 -- 172.21.15.84:0/2990627954 wait complete. 2023-10-16T11:27:08.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.050+0000 7f0315861640 1 Processor -- start 2023-10-16T11:27:08.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.051+0000 7f0315861640 1 -- start start 2023-10-16T11:27:08.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.051+0000 7f0315861640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03100ce390 0x7f031014aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:08.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.052+0000 7f0315861640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f031015aaa0 0x7f03100c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:08.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.052+0000 7f0315861640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f031015ae80 0x7f03100c25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:08.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.052+0000 7f0315861640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031014b410 con 0x7f031015ae80 2023-10-16T11:27:08.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.052+0000 7f0315861640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031014b6f0 con 0x7f03100ce390 2023-10-16T11:27:08.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.052+0000 7f0315861640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031014b9d0 con 0x7f031015aaa0 2023-10-16T11:27:08.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.052+0000 7f030f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f031015ae80 0x7f03100c25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:08.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.052+0000 7f030f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f031015ae80 0x7f03100c25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44070/0 (socket says 172.21.15.84:44070) 2023-10-16T11:27:08.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.052+0000 7f030f7fe640 1 -- 172.21.15.84:0/888120159 learned_addr learned my addr 172.21.15.84:0/888120159 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:08.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.052+0000 7f03067fc640 1 --2- 172.21.15.84:0/888120159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f031015aaa0 0x7f03100c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:08.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.052+0000 7f030effd640 1 --2- 172.21.15.84:0/888120159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03100ce390 0x7f031014aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:08.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.052+0000 7f030f7fe640 1 -- 172.21.15.84:0/888120159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f031015aaa0 msgr2=0x7f03100c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:08.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.052+0000 7f030f7fe640 1 --2- 172.21.15.84:0/888120159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f031015aaa0 0x7f03100c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.052+0000 7f030f7fe640 1 -- 172.21.15.84:0/888120159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03100ce390 msgr2=0x7f031014aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:08.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.052+0000 7f030f7fe640 1 --2- 172.21.15.84:0/888120159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03100ce390 0x7f031014aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.052+0000 7f030f7fe640 1 -- 172.21.15.84:0/888120159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03080094d0 con 0x7f031015ae80 2023-10-16T11:27:08.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.052+0000 7f03067fc640 1 --2- 172.21.15.84:0/888120159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f031015aaa0 0x7f03100c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:08.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.052+0000 7f030effd640 1 --2- 172.21.15.84:0/888120159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03100ce390 0x7f031014aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:08.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.053+0000 7f030f7fe640 1 --2- 172.21.15.84:0/888120159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f031015ae80 0x7f03100c25c0 secure :-1 s=READY pgs=3328 cs=0 l=1 rev1=1 crypto rx=0x7f030802b930 tx=0x7f03080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:08.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.053+0000 7f030cff9640 1 -- 172.21.15.84:0/888120159 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030803ac60 con 0x7f031015ae80 2023-10-16T11:27:08.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.053+0000 7f030cff9640 1 -- 172.21.15.84:0/888120159 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030803adc0 con 0x7f031015ae80 2023-10-16T11:27:08.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.053+0000 7f0315861640 1 -- 172.21.15.84:0/888120159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031014bcb0 con 0x7f031015ae80 2023-10-16T11:27:08.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.053+0000 7f030cff9640 1 -- 172.21.15.84:0/888120159 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308044630 con 0x7f031015ae80 2023-10-16T11:27:08.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.053+0000 7f0315861640 1 -- 172.21.15.84:0/888120159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f031014c270 con 0x7f031015ae80 2023-10-16T11:27:08.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.054+0000 7f0315861640 1 -- 172.21.15.84:0/888120159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f031014c460 con 0x7f031015ae80 2023-10-16T11:27:08.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.055+0000 7f030cff9640 1 -- 172.21.15.84:0/888120159 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 158) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f030802f050 con 0x7f031015ae80 2023-10-16T11:27:08.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.056+0000 7f030cff9640 1 --2- 172.21.15.84:0/888120159 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f02f006acc0 0x7f02f006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:08.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.056+0000 7f030cff9640 1 -- 172.21.15.84:0/888120159 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f03080ab4d0 con 0x7f031015ae80 2023-10-16T11:27:08.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.056+0000 7f030effd640 1 --2- 172.21.15.84:0/888120159 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f02f006acc0 0x7f02f006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:08.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.057+0000 7f030effd640 1 --2- 172.21.15.84:0/888120159 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f02f006acc0 0x7f02f006d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0300004520 tx=0x7f0300009040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:08.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.061+0000 7f030cff9640 1 -- 172.21.15.84:0/888120159 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0308076470 con 0x7f031015ae80 2023-10-16T11:27:08.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.267+0000 7f030cff9640 1 -- 172.21.15.84:0/888120159 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 159) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f0308076840 con 0x7f031015ae80 2023-10-16T11:27:08.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.296+0000 7f0315861640 1 -- 172.21.15.84:0/888120159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f031014bb10 con 0x7f031015ae80 2023-10-16T11:27:08.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.297+0000 7f030cff9640 1 -- 172.21.15.84:0/888120159 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+4929 (secure 0 0 0) 0x7f031014bb10 con 0x7f031015ae80 2023-10-16T11:27:08.298 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:08.299 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":148,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/591944719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":591944719},{"type":"v1","addr":"172.21.15.84:6835","nonce":591944719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":142},{"gid":8152,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2371517609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2371517609},{"type":"v1","addr":"172.21.15.84:6837","nonce":2371517609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":143},{"gid":8161,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1079456313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1079456313},{"type":"v1","addr":"172.21.15.157:6834","nonce":1079456313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":143}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:26:41.908585+0000","modified":"2023-10-16T11:26:44.991714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12987":{"gid":12987,"name":"b","rank":0,"incarnation":144,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3899307325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3899307325},{"type":"v1","addr":"172.21.15.157:6835","nonce":3899307325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-16T11:27:08.299 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 148 2023-10-16T11:27:08.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.299+0000 7f0315861640 1 -- 172.21.15.84:0/888120159 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f02f006acc0 msgr2=0x7f02f006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:08.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.300+0000 7f0315861640 1 --2- 172.21.15.84:0/888120159 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f02f006acc0 0x7f02f006d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0300004520 tx=0x7f0300009040 comp rx=0 tx=0).stop 2023-10-16T11:27:08.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.300+0000 7f0315861640 1 -- 172.21.15.84:0/888120159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f031015ae80 msgr2=0x7f03100c25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:08.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.300+0000 7f0315861640 1 --2- 172.21.15.84:0/888120159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f031015ae80 0x7f03100c25c0 secure :-1 s=READY pgs=3328 cs=0 l=1 rev1=1 crypto rx=0x7f030802b930 tx=0x7f03080046c0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.300+0000 7f0315861640 1 -- 172.21.15.84:0/888120159 shutdown_connections 2023-10-16T11:27:08.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.300+0000 7f0315861640 1 --2- 172.21.15.84:0/888120159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f031015ae80 0x7f03100c25c0 unknown :-1 s=CLOSED pgs=3328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.300+0000 7f0315861640 1 --2- 172.21.15.84:0/888120159 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f02f006acc0 0x7f02f006d180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.300+0000 7f0315861640 1 --2- 172.21.15.84:0/888120159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f031015aaa0 0x7f03100c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.300+0000 7f0315861640 1 --2- 172.21.15.84:0/888120159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03100ce390 0x7f031014aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.300+0000 7f0315861640 1 -- 172.21.15.84:0/888120159 >> 172.21.15.84:0/888120159 conn(0x7f03100cb410 msgr2=0x7f03100b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:08.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.300+0000 7f0315861640 1 -- 172.21.15.84:0/888120159 shutdown_connections 2023-10-16T11:27:08.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.300+0000 7f0315861640 1 -- 172.21.15.84:0/888120159 wait complete. 2023-10-16T11:27:08.320 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:27:08.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.525+0000 7fe97a741640 1 Processor -- start 2023-10-16T11:27:08.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.526+0000 7fe97a741640 1 -- start start 2023-10-16T11:27:08.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.527+0000 7fe97a741640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9740b0b20 0x7fe9741615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:08.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.527+0000 7fe97a741640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe97415d3f0 0x7fe97415d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:08.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.527+0000 7fe97a741640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe97415e3e0 con 0x7fe9740cd0c0 2023-10-16T11:27:08.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.527+0000 7fe97a741640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9740cd4a0 con 0x7fe97415d3f0 2023-10-16T11:27:08.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.527+0000 7fe97a741640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe97415e100 con 0x7fe9740b0b20 2023-10-16T11:27:08.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.527+0000 7fe978cde640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe97415d3f0 0x7fe97415d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:08.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.527+0000 7fe973fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe9740cd0c0 0x7fe9740b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57332/0 (socket says 172.21.15.84:57332) 2023-10-16T11:27:08.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.527+0000 7fe973fff640 1 -- 172.21.15.84:0/319938125 learned_addr learned my addr 172.21.15.84:0/319938125 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:08.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.528+0000 7fe96bfff640 1 --2- 172.21.15.84:0/319938125 >> v2:172.21.15.157:3300/0 conn(0x7fe9740b0b20 0x7fe9741615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:08.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.528+0000 7fe978cde640 1 -- 172.21.15.84:0/319938125 >> v2:172.21.15.157:3300/0 conn(0x7fe9740b0b20 msgr2=0x7fe9741615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:08.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.528+0000 7fe978cde640 1 --2- 172.21.15.84:0/319938125 >> v2:172.21.15.157:3300/0 conn(0x7fe9740b0b20 0x7fe9741615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.528+0000 7fe978cde640 1 -- 172.21.15.84:0/319938125 >> v1:172.21.15.84:6789/0 conn(0x7fe9740cd0c0 legacy=0x7fe9740b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:08.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.528+0000 7fe978cde640 1 -- 172.21.15.84:0/319938125 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe974160b10 con 0x7fe97415d3f0 2023-10-16T11:27:08.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.528+0000 7fe9737fe640 1 -- 172.21.15.84:0/319938125 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2556036898 0 0) 0x7fe97415e3e0 con 0x7fe9740cd0c0 2023-10-16T11:27:08.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.529+0000 7fe978cde640 1 --2- 172.21.15.84:0/319938125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe97415d3f0 0x7fe97415d820 secure :-1 s=READY pgs=3330 cs=0 l=1 rev1=1 crypto rx=0x7fe97415afc0 tx=0x7fe96c02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55f03c54c6892f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:08.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.529+0000 7fe9737fe640 1 -- 172.21.15.84:0/319938125 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c02cbe0 con 0x7fe97415d3f0 2023-10-16T11:27:08.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.529+0000 7fe9737fe640 1 -- 172.21.15.84:0/319938125 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96c02cd40 con 0x7fe97415d3f0 2023-10-16T11:27:08.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.529+0000 7fe9737fe640 1 -- 172.21.15.84:0/319938125 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c034660 con 0x7fe97415d3f0 2023-10-16T11:27:08.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.530+0000 7fe97a741640 1 -- 172.21.15.84:0/319938125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe97415d3f0 msgr2=0x7fe97415d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:08.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.530+0000 7fe97a741640 1 --2- 172.21.15.84:0/319938125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe97415d3f0 0x7fe97415d820 secure :-1 s=READY pgs=3330 cs=0 l=1 rev1=1 crypto rx=0x7fe97415afc0 tx=0x7fe96c02be30 comp rx=0 tx=0).stop 2023-10-16T11:27:08.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.530+0000 7fe97a741640 1 -- 172.21.15.84:0/319938125 shutdown_connections 2023-10-16T11:27:08.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.530+0000 7fe97a741640 1 --2- 172.21.15.84:0/319938125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe97415d3f0 0x7fe97415d820 unknown :-1 s=CLOSED pgs=3330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.530+0000 7fe97a741640 1 --2- 172.21.15.84:0/319938125 >> v2:172.21.15.157:3300/0 conn(0x7fe9740b0b20 0x7fe9741615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.530+0000 7fe97a741640 1 -- 172.21.15.84:0/319938125 >> 172.21.15.84:0/319938125 conn(0x7fe9740cb410 msgr2=0x7fe9740cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:08.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.530+0000 7fe97a741640 1 -- 172.21.15.84:0/319938125 shutdown_connections 2023-10-16T11:27:08.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.530+0000 7fe97a741640 1 -- 172.21.15.84:0/319938125 wait complete. 2023-10-16T11:27:08.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.532+0000 7fe97a741640 1 Processor -- start 2023-10-16T11:27:08.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.532+0000 7fe97a741640 1 -- start start 2023-10-16T11:27:08.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.533+0000 7fe97a741640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9740b0b20 0x7fe9740c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.533+0000 7fe97a741640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9740cd0c0 0x7fe9741f3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.533+0000 7fe97a741640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe97415d3f0 0x7fe9741f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.533+0000 7fe97a741640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9740c7bb0 con 0x7fe9740cd0c0 2023-10-16T11:27:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.533+0000 7fe97a741640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9740c3d60 con 0x7fe97415d3f0 2023-10-16T11:27:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.533+0000 7fe97a741640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9740c4040 con 0x7fe9740b0b20 2023-10-16T11:27:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.533+0000 7fe96bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9740cd0c0 0x7fe9741f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.533+0000 7fe96bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9740cd0c0 0x7fe9741f3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44088/0 (socket says 172.21.15.84:44088) 2023-10-16T11:27:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.533+0000 7fe96bfff640 1 -- 172.21.15.84:0/3246757285 learned_addr learned my addr 172.21.15.84:0/3246757285 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.534+0000 7fe96bfff640 1 -- 172.21.15.84:0/3246757285 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9740b0b20 msgr2=0x7fe9740c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:08.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.534+0000 7fe96bfff640 1 --2- 172.21.15.84:0/3246757285 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9740b0b20 0x7fe9740c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.534+0000 7fe96bfff640 1 -- 172.21.15.84:0/3246757285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe97415d3f0 msgr2=0x7fe9741f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:08.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.534+0000 7fe96bfff640 1 --2- 172.21.15.84:0/3246757285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe97415d3f0 0x7fe9741f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.534+0000 7fe96bfff640 1 -- 172.21.15.84:0/3246757285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c009a70 con 0x7fe9740cd0c0 2023-10-16T11:27:08.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.534+0000 7fe96bfff640 1 --2- 172.21.15.84:0/3246757285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9740cd0c0 0x7fe9741f3720 secure :-1 s=READY pgs=3331 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00d5d0 tx=0x7fe95c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:08.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.534+0000 7fe9717fa640 1 -- 172.21.15.84:0/3246757285 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c0125f0 con 0x7fe9740cd0c0 2023-10-16T11:27:08.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.534+0000 7fe9717fa640 1 -- 172.21.15.84:0/3246757285 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c012770 con 0x7fe9740cd0c0 2023-10-16T11:27:08.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.534+0000 7fe9717fa640 1 -- 172.21.15.84:0/3246757285 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c012b20 con 0x7fe9740cd0c0 2023-10-16T11:27:08.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.534+0000 7fe97a741640 1 -- 172.21.15.84:0/3246757285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9740c4320 con 0x7fe9740cd0c0 2023-10-16T11:27:08.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.534+0000 7fe97a741640 1 -- 172.21.15.84:0/3246757285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9741f3c60 con 0x7fe9740cd0c0 2023-10-16T11:27:08.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.535+0000 7fe97a741640 1 -- 172.21.15.84:0/3246757285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9741f3df0 con 0x7fe9740cd0c0 2023-10-16T11:27:08.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.536+0000 7fe9717fa640 1 -- 172.21.15.84:0/3246757285 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 159) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fe95c012c80 con 0x7fe9740cd0c0 2023-10-16T11:27:08.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.537+0000 7fe9717fa640 1 --2- 172.21.15.84:0/3246757285 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fe95406af00 0x7fe95406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:08.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.537+0000 7fe9717fa640 1 -- 172.21.15.84:0/3246757285 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fe95c089ea0 con 0x7fe9740cd0c0 2023-10-16T11:27:08.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.537+0000 7fe973fff640 1 --2- 172.21.15.84:0/3246757285 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fe95406af00 0x7fe95406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:08.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.538+0000 7fe973fff640 1 --2- 172.21.15.84:0/3246757285 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fe95406af00 0x7fe95406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe964007fd0 tx=0x7fe964007960 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:08.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.542+0000 7fe9717fa640 1 -- 172.21.15.84:0/3246757285 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe95c054c70 con 0x7fe9740cd0c0 2023-10-16T11:27:08.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.734+0000 7fe97a741640 1 -- 172.21.15.84:0/3246757285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe97404e160 con 0x7fe9740cd0c0 2023-10-16T11:27:08.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.736+0000 7fe9717fa640 1 -- 172.21.15.84:0/3246757285 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v259) v1 ==== 74+0+17793 (secure 0 0 0) 0x7fe95c058290 con 0x7fe9740cd0c0 2023-10-16T11:27:08.737 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:08.738 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":259,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:27:05.495073+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:26:39.860053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-16T11:26:40.916519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1625282261":"2023-10-17T11:27:05.494979+0000","172.21.15.84:6833/2988856492":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/765814920":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/2899443560":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/3002614854":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/2731742209":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/907959269":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/1863310536":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/847549599":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/972005011":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/1934386371":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/3540589884":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/2370234804":"2023-10-17T11:26:57.151307+0000","172.21.15.84:6833/2330196255":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/1650008535":"2023-10-17T11:26:57.151307+0000","172.21.15.84:6832/2330196255":"2023-10-17T11:26:57.151307+0000","172.21.15.84:6832/2988856492":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/3247488662":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/2440426242":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/570042115":"2023-10-16T12:26:40.168744+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:27:08.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.738+0000 7fe97a741640 1 -- 172.21.15.84:0/3246757285 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fe95406af00 msgr2=0x7fe95406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:08.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.738+0000 7fe97a741640 1 --2- 172.21.15.84:0/3246757285 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fe95406af00 0x7fe95406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe964007fd0 tx=0x7fe964007960 comp rx=0 tx=0).stop 2023-10-16T11:27:08.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.738+0000 7fe97a741640 1 -- 172.21.15.84:0/3246757285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9740cd0c0 msgr2=0x7fe9741f3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:08.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.738+0000 7fe97a741640 1 --2- 172.21.15.84:0/3246757285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9740cd0c0 0x7fe9741f3720 secure :-1 s=READY pgs=3331 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00d5d0 tx=0x7fe95c00da90 comp rx=0 tx=0).stop 2023-10-16T11:27:08.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.739+0000 7fe97a741640 1 -- 172.21.15.84:0/3246757285 shutdown_connections 2023-10-16T11:27:08.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.739+0000 7fe97a741640 1 --2- 172.21.15.84:0/3246757285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe97415d3f0 0x7fe9741f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.739+0000 7fe97a741640 1 --2- 172.21.15.84:0/3246757285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9740cd0c0 0x7fe9741f3720 unknown :-1 s=CLOSED pgs=3331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.739+0000 7fe97a741640 1 --2- 172.21.15.84:0/3246757285 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fe95406af00 0x7fe95406d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.739+0000 7fe97a741640 1 --2- 172.21.15.84:0/3246757285 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9740b0b20 0x7fe9740c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.739+0000 7fe97a741640 1 -- 172.21.15.84:0/3246757285 >> 172.21.15.84:0/3246757285 conn(0x7fe9740cb410 msgr2=0x7fe9740ca9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:08.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.739+0000 7fe97a741640 1 -- 172.21.15.84:0/3246757285 shutdown_connections 2023-10-16T11:27:08.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.739+0000 7fe97a741640 1 -- 172.21.15.84:0/3246757285 wait complete. 2023-10-16T11:27:08.759 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T11:27:08.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.968+0000 7fee8d213640 1 Processor -- start 2023-10-16T11:27:08.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.968+0000 7fee8d213640 1 -- start start 2023-10-16T11:27:08.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.969+0000 7fee8d213640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fee880ad970 0x7fee880cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:08.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.969+0000 7fee8d213640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fee880ccfb0 0x7fee880cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:08.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.969+0000 7fee8d213640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee8815b9d0 con 0x7fee8806c040 2023-10-16T11:27:08.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.969+0000 7fee8d213640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee8815bcb0 con 0x7fee880ccfb0 2023-10-16T11:27:08.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.969+0000 7fee8d213640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee8815bf90 con 0x7fee880ad970 2023-10-16T11:27:08.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.969+0000 7fee87fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fee8806c040 0x7fee8806c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57346/0 (socket says 172.21.15.84:57346) 2023-10-16T11:27:08.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.969+0000 7fee87fff640 1 -- 172.21.15.84:0/1989778201 learned_addr learned my addr 172.21.15.84:0/1989778201 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:08.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.969+0000 7fee8ca12640 1 --2- 172.21.15.84:0/1989778201 >> v2:172.21.15.84:3300/0 conn(0x7fee880ccfb0 0x7fee880cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:08.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.970+0000 7fee7ffff640 1 --2- 172.21.15.84:0/1989778201 >> v2:172.21.15.157:3300/0 conn(0x7fee880ad970 0x7fee880cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:08.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.970+0000 7fee877fe640 1 -- 172.21.15.84:0/1989778201 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392720643 0 0) 0x7fee8815b9d0 con 0x7fee8806c040 2023-10-16T11:27:08.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.970+0000 7fee877fe640 1 -- 172.21.15.84:0/1989778201 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee74003520 con 0x7fee8806c040 2023-10-16T11:27:08.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.970+0000 7fee8ca12640 1 -- 172.21.15.84:0/1989778201 >> v2:172.21.15.157:3300/0 conn(0x7fee880ad970 msgr2=0x7fee880cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:08.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.970+0000 7fee8ca12640 1 --2- 172.21.15.84:0/1989778201 >> v2:172.21.15.157:3300/0 conn(0x7fee880ad970 0x7fee880cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.970+0000 7fee8ca12640 1 -- 172.21.15.84:0/1989778201 >> v1:172.21.15.84:6789/0 conn(0x7fee8806c040 legacy=0x7fee8806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:08.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.970+0000 7fee8ca12640 1 -- 172.21.15.84:0/1989778201 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee880cc270 con 0x7fee880ccfb0 2023-10-16T11:27:08.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.971+0000 7fee8ca12640 1 --2- 172.21.15.84:0/1989778201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee880ccfb0 0x7fee880cd460 secure :-1 s=READY pgs=3333 cs=0 l=1 rev1=1 crypto rx=0x7fee80009d90 tx=0x7fee8002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995e305307f6d9da server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:08.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.971+0000 7fee877fe640 1 -- 172.21.15.84:0/1989778201 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8002cc60 con 0x7fee880ccfb0 2023-10-16T11:27:08.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.971+0000 7fee877fe640 1 -- 172.21.15.84:0/1989778201 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8002cdc0 con 0x7fee880ccfb0 2023-10-16T11:27:08.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.971+0000 7fee877fe640 1 -- 172.21.15.84:0/1989778201 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee80034660 con 0x7fee880ccfb0 2023-10-16T11:27:08.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.972+0000 7fee8d213640 1 -- 172.21.15.84:0/1989778201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee880ccfb0 msgr2=0x7fee880cd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:08.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.972+0000 7fee8d213640 1 --2- 172.21.15.84:0/1989778201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee880ccfb0 0x7fee880cd460 secure :-1 s=READY pgs=3333 cs=0 l=1 rev1=1 crypto rx=0x7fee80009d90 tx=0x7fee8002beb0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.972+0000 7fee8d213640 1 -- 172.21.15.84:0/1989778201 shutdown_connections 2023-10-16T11:27:08.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.972+0000 7fee8d213640 1 --2- 172.21.15.84:0/1989778201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee880ccfb0 0x7fee880cd460 unknown :-1 s=CLOSED pgs=3333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.972+0000 7fee8d213640 1 --2- 172.21.15.84:0/1989778201 >> v2:172.21.15.157:3300/0 conn(0x7fee880ad970 0x7fee880cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.972+0000 7fee8d213640 1 -- 172.21.15.84:0/1989778201 >> 172.21.15.84:0/1989778201 conn(0x7fee880ce6a0 msgr2=0x7fee880ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:08.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.972+0000 7fee8d213640 1 -- 172.21.15.84:0/1989778201 shutdown_connections 2023-10-16T11:27:08.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.972+0000 7fee8d213640 1 -- 172.21.15.84:0/1989778201 wait complete. 2023-10-16T11:27:08.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.974+0000 7fee8d213640 1 Processor -- start 2023-10-16T11:27:08.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.974+0000 7fee8d213640 1 -- start start 2023-10-16T11:27:08.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.975+0000 7fee8d213640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee8806c040 0x7fee880af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:08.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.975+0000 7fee8d213640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee880ad970 0x7fee88155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:08.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.975+0000 7fee8d213640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee880ccfb0 0x7fee880b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:08.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.975+0000 7fee8d213640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee880af820 con 0x7fee8806c040 2023-10-16T11:27:08.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.975+0000 7fee8d213640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee88155ec0 con 0x7fee880ad970 2023-10-16T11:27:08.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.975+0000 7fee8d213640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee880b2cd0 con 0x7fee880ccfb0 2023-10-16T11:27:08.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.975+0000 7fee87fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee8806c040 0x7fee880af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:08.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.975+0000 7fee87fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee8806c040 0x7fee880af2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44100/0 (socket says 172.21.15.84:44100) 2023-10-16T11:27:08.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.975+0000 7fee87fff640 1 -- 172.21.15.84:0/706905999 learned_addr learned my addr 172.21.15.84:0/706905999 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:08.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.975+0000 7fee87fff640 1 -- 172.21.15.84:0/706905999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee880ccfb0 msgr2=0x7fee880b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:08.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.975+0000 7fee87fff640 1 --2- 172.21.15.84:0/706905999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee880ccfb0 0x7fee880b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.975+0000 7fee87fff640 1 -- 172.21.15.84:0/706905999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee880ad970 msgr2=0x7fee88155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:08.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.975+0000 7fee87fff640 1 --2- 172.21.15.84:0/706905999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee880ad970 0x7fee88155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:08.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.975+0000 7fee87fff640 1 -- 172.21.15.84:0/706905999 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee80009a70 con 0x7fee8806c040 2023-10-16T11:27:08.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.976+0000 7fee87fff640 1 --2- 172.21.15.84:0/706905999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee8806c040 0x7fee880af2e0 secure :-1 s=READY pgs=3334 cs=0 l=1 rev1=1 crypto rx=0x7fee7800c560 tx=0x7fee7800ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:08.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.976+0000 7fee857fa640 1 -- 172.21.15.84:0/706905999 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78005ce0 con 0x7fee8806c040 2023-10-16T11:27:08.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.976+0000 7fee857fa640 1 -- 172.21.15.84:0/706905999 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee78005e40 con 0x7fee8806c040 2023-10-16T11:27:08.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.976+0000 7fee857fa640 1 -- 172.21.15.84:0/706905999 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee780072b0 con 0x7fee8806c040 2023-10-16T11:27:08.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.976+0000 7fee8d213640 1 -- 172.21.15.84:0/706905999 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee880b2f00 con 0x7fee8806c040 2023-10-16T11:27:08.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.976+0000 7fee8d213640 1 -- 172.21.15.84:0/706905999 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee880b34c0 con 0x7fee8806c040 2023-10-16T11:27:08.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.977+0000 7fee8d213640 1 -- 172.21.15.84:0/706905999 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee8806b330 con 0x7fee8806c040 2023-10-16T11:27:08.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.978+0000 7fee857fa640 1 -- 172.21.15.84:0/706905999 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 159) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fee78007410 con 0x7fee8806c040 2023-10-16T11:27:08.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.979+0000 7fee857fa640 1 --2- 172.21.15.84:0/706905999 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fee6806af00 0x7fee6806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:08.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.979+0000 7fee857fa640 1 -- 172.21.15.84:0/706905999 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fee78089080 con 0x7fee8806c040 2023-10-16T11:27:08.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.980+0000 7fee7ffff640 1 --2- 172.21.15.84:0/706905999 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fee6806af00 0x7fee6806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:08.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.980+0000 7fee7ffff640 1 --2- 172.21.15.84:0/706905999 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fee6806af00 0x7fee6806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fee880c8750 tx=0x7fee70008040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:08.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:08.984+0000 7fee857fa640 1 -- 172.21.15.84:0/706905999 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fee780bbfe0 con 0x7fee8806c040 2023-10-16T11:27:09.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.212+0000 7fee8d213640 1 -- 172.21.15.84:0/706905999 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fee8804e9b0 con 0x7fee8806c040 2023-10-16T11:27:09.214 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-16T11:27:09.213+0000 7fca0c7ff640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:27:09.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.217+0000 7fee857fa640 1 -- 172.21.15.84:0/706905999 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v149) v1 ==== 161+0+0 (secure 0 0 0) 0x7fee78017020 con 0x7fee8806c040 2023-10-16T11:27:09.218 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:27:09.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.220+0000 7fee8d213640 1 -- 172.21.15.84:0/706905999 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fee6806af00 msgr2=0x7fee6806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:09.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.220+0000 7fee8d213640 1 --2- 172.21.15.84:0/706905999 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fee6806af00 0x7fee6806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fee880c8750 tx=0x7fee70008040 comp rx=0 tx=0).stop 2023-10-16T11:27:09.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.220+0000 7fee8d213640 1 -- 172.21.15.84:0/706905999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee8806c040 msgr2=0x7fee880af2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:09.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.220+0000 7fee8d213640 1 --2- 172.21.15.84:0/706905999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee8806c040 0x7fee880af2e0 secure :-1 s=READY pgs=3334 cs=0 l=1 rev1=1 crypto rx=0x7fee7800c560 tx=0x7fee7800ca20 comp rx=0 tx=0).stop 2023-10-16T11:27:09.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.221+0000 7fee8d213640 1 -- 172.21.15.84:0/706905999 shutdown_connections 2023-10-16T11:27:09.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.221+0000 7fee8d213640 1 --2- 172.21.15.84:0/706905999 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fee6806af00 0x7fee6806d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:09.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.221+0000 7fee8d213640 1 --2- 172.21.15.84:0/706905999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee880ccfb0 0x7fee880b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:09.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.221+0000 7fee8d213640 1 --2- 172.21.15.84:0/706905999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee880ad970 0x7fee88155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:09.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.221+0000 7fee8d213640 1 --2- 172.21.15.84:0/706905999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee8806c040 0x7fee880af2e0 unknown :-1 s=CLOSED pgs=3334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:09.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.221+0000 7fee8d213640 1 -- 172.21.15.84:0/706905999 >> 172.21.15.84:0/706905999 conn(0x7fee880ce6a0 msgr2=0x7fee880ca1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:09.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.221+0000 7fee8d213640 1 -- 172.21.15.84:0/706905999 shutdown_connections 2023-10-16T11:27:09.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.221+0000 7fee8d213640 1 -- 172.21.15.84:0/706905999 wait complete. 2023-10-16T11:27:09.241 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T11:27:09.250 INFO:tasks.ceph.mds.b.smithi157.stderr: -3905> 2023-10-16T11:26:48.410+0000 7f65c66dc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:27:09.339 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T11:27:09.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.450+0000 7faa70c6f640 1 Processor -- start 2023-10-16T11:27:09.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.450+0000 7faa70c6f640 1 -- start start 2023-10-16T11:27:09.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.452+0000 7faa70c6f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c070830 0x7faa6c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:09.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.452+0000 7faa70c6f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faa6c0cf5f0 0x7faa6c06c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:09.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.452+0000 7faa70c6f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6c161740 con 0x7faa6c070c10 2023-10-16T11:27:09.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.452+0000 7faa70c6f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6c06c680 con 0x7faa6c0cf5f0 2023-10-16T11:27:09.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.452+0000 7faa70c6f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c06b0f0 con 0x7faa6c070830 2023-10-16T11:27:09.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.453+0000 7faa6a59c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c070830 0x7faa6c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:09.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.453+0000 7faa69d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faa6c0cf5f0 0x7faa6c06c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:09.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.453+0000 7faa69d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faa6c0cf5f0 0x7faa6c06c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:56958/0 (socket says 172.21.15.84:56958) 2023-10-16T11:27:09.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.453+0000 7faa6ad9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7faa6c070c10 0x7faa6c0cd690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:44140/0 (socket says 172.21.15.84:44140) 2023-10-16T11:27:09.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.453+0000 7faa69d9b640 1 -- 172.21.15.84:0/4095169533 learned_addr learned my addr 172.21.15.84:0/4095169533 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:09.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.454+0000 7faa69d9b640 1 -- 172.21.15.84:0/4095169533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c070830 msgr2=0x7faa6c06d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:09.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.454+0000 7faa69d9b640 1 --2- 172.21.15.84:0/4095169533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c070830 0x7faa6c06d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:09.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.454+0000 7faa69d9b640 1 -- 172.21.15.84:0/4095169533 >> v1:172.21.15.157:6789/0 conn(0x7faa6c070c10 legacy=0x7faa6c0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:09.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.454+0000 7faa69d9b640 1 -- 172.21.15.84:0/4095169533 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c1619b0 con 0x7faa6c0cf5f0 2023-10-16T11:27:09.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.455+0000 7faa69d9b640 1 --2- 172.21.15.84:0/4095169533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6c0cf5f0 0x7faa6c06c140 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7faa54009870 tx=0x7faa5402b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e17872c5cf98af1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:09.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.455+0000 7faa6959a640 1 -- 172.21.15.84:0/4095169533 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa540317c0 con 0x7faa6c0cf5f0 2023-10-16T11:27:09.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.456+0000 7faa6959a640 1 -- 172.21.15.84:0/4095169533 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa54031920 con 0x7faa6c0cf5f0 2023-10-16T11:27:09.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.456+0000 7faa6959a640 1 -- 172.21.15.84:0/4095169533 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54031cb0 con 0x7faa6c0cf5f0 2023-10-16T11:27:09.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.456+0000 7faa70c6f640 1 -- 172.21.15.84:0/4095169533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6c0cf5f0 msgr2=0x7faa6c06c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:09.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.456+0000 7faa70c6f640 1 --2- 172.21.15.84:0/4095169533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6c0cf5f0 0x7faa6c06c140 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7faa54009870 tx=0x7faa5402b860 comp rx=0 tx=0).stop 2023-10-16T11:27:09.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.456+0000 7faa70c6f640 1 -- 172.21.15.84:0/4095169533 shutdown_connections 2023-10-16T11:27:09.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.456+0000 7faa70c6f640 1 --2- 172.21.15.84:0/4095169533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6c0cf5f0 0x7faa6c06c140 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:09.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.456+0000 7faa70c6f640 1 --2- 172.21.15.84:0/4095169533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c070830 0x7faa6c06d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:09.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.456+0000 7faa70c6f640 1 -- 172.21.15.84:0/4095169533 >> 172.21.15.84:0/4095169533 conn(0x7faa6c0cfac0 msgr2=0x7faa6c0cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:09.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.456+0000 7faa70c6f640 1 -- 172.21.15.84:0/4095169533 shutdown_connections 2023-10-16T11:27:09.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.456+0000 7faa70c6f640 1 -- 172.21.15.84:0/4095169533 wait complete. 2023-10-16T11:27:09.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.458+0000 7faa70c6f640 1 Processor -- start 2023-10-16T11:27:09.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.458+0000 7faa70c6f640 1 -- start start 2023-10-16T11:27:09.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.459+0000 7faa70c6f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c070830 0x7faa6c157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:09.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.459+0000 7faa70c6f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6c070c10 0x7faa6c0ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:09.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.459+0000 7faa70c6f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c0cf5f0 0x7faa6c0adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:09.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.459+0000 7faa70c6f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0c54d0 con 0x7faa6c0cf5f0 2023-10-16T11:27:09.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.459+0000 7faa70c6f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0c57b0 con 0x7faa6c070c10 2023-10-16T11:27:09.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.459+0000 7faa70c6f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0c5a90 con 0x7faa6c070830 2023-10-16T11:27:09.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.459+0000 7faa6ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c0cf5f0 0x7faa6c0adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:09.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.460+0000 7faa6ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c0cf5f0 0x7faa6c0adbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44112/0 (socket says 172.21.15.84:44112) 2023-10-16T11:27:09.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.460+0000 7faa6ad9d640 1 -- 172.21.15.84:0/341305575 learned_addr learned my addr 172.21.15.84:0/341305575 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:09.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.460+0000 7faa69d9b640 1 --2- 172.21.15.84:0/341305575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6c070c10 0x7faa6c0ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:09.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.460+0000 7faa6a59c640 1 --2- 172.21.15.84:0/341305575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c070830 0x7faa6c157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:09.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.460+0000 7faa6ad9d640 1 -- 172.21.15.84:0/341305575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c070830 msgr2=0x7faa6c157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:09.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.460+0000 7faa6ad9d640 1 --2- 172.21.15.84:0/341305575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c070830 0x7faa6c157160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:09.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.460+0000 7faa6ad9d640 1 -- 172.21.15.84:0/341305575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6c070c10 msgr2=0x7faa6c0ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:09.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.460+0000 7faa6ad9d640 1 --2- 172.21.15.84:0/341305575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6c070c10 0x7faa6c0ad6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:09.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.460+0000 7faa6ad9d640 1 -- 172.21.15.84:0/341305575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa540094d0 con 0x7faa6c0cf5f0 2023-10-16T11:27:09.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.460+0000 7faa6ad9d640 1 --2- 172.21.15.84:0/341305575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c0cf5f0 0x7faa6c0adbf0 secure :-1 s=READY pgs=3336 cs=0 l=1 rev1=1 crypto rx=0x7faa6400c400 tx=0x7faa6400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:09.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.460+0000 7faa62ffd640 1 -- 172.21.15.84:0/341305575 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa640074e0 con 0x7faa6c0cf5f0 2023-10-16T11:27:09.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.461+0000 7faa62ffd640 1 -- 172.21.15.84:0/341305575 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa64007640 con 0x7faa6c0cf5f0 2023-10-16T11:27:09.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.461+0000 7faa70c6f640 1 -- 172.21.15.84:0/341305575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa6c0c5d70 con 0x7faa6c0cf5f0 2023-10-16T11:27:09.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.461+0000 7faa62ffd640 1 -- 172.21.15.84:0/341305575 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa640049e0 con 0x7faa6c0cf5f0 2023-10-16T11:27:09.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.461+0000 7faa70c6f640 1 -- 172.21.15.84:0/341305575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa6c0b1ee0 con 0x7faa6c0cf5f0 2023-10-16T11:27:09.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.462+0000 7faa70c6f640 1 -- 172.21.15.84:0/341305575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6c0c6290 con 0x7faa6c0cf5f0 2023-10-16T11:27:09.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.463+0000 7faa62ffd640 1 -- 172.21.15.84:0/341305575 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 159) v1 ==== 89275+0+0 (secure 0 0 0) 0x7faa64004b40 con 0x7faa6c0cf5f0 2023-10-16T11:27:09.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.463+0000 7faa62ffd640 1 --2- 172.21.15.84:0/341305575 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7faa4c06af00 0x7faa4c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:09.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.464+0000 7faa62ffd640 1 -- 172.21.15.84:0/341305575 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 6683+0+0 (secure 0 0 0) 0x7faa64089560 con 0x7faa6c0cf5f0 2023-10-16T11:27:09.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.464+0000 7faa6a59c640 1 --2- 172.21.15.84:0/341305575 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7faa4c06af00 0x7faa4c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:09.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.464+0000 7faa6a59c640 1 --2- 172.21.15.84:0/341305575 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7faa4c06af00 0x7faa4c06d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009fd0 tx=0x7faa5c00b040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:09.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.468+0000 7faa62ffd640 1 -- 172.21.15.84:0/341305575 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7faa6c0c6290 con 0x7faa6c0cf5f0 2023-10-16T11:27:09.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:09.702+0000 7faa70c6f640 1 -- 172.21.15.84:0/341305575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7faa6c0c6290 con 0x7faa6c0cf5f0 2023-10-16T11:27:10.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.287+0000 7faa62ffd640 1 -- 172.21.15.84:0/341305575 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v150) v1 ==== 104+0+0 (secure 0 0 0) 0x7faa6c0c6290 con 0x7faa6c0cf5f0 2023-10-16T11:27:10.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.291+0000 7faa70c6f640 1 -- 172.21.15.84:0/341305575 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7faa4c06af00 msgr2=0x7faa4c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:10.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.291+0000 7faa70c6f640 1 --2- 172.21.15.84:0/341305575 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7faa4c06af00 0x7faa4c06d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009fd0 tx=0x7faa5c00b040 comp rx=0 tx=0).stop 2023-10-16T11:27:10.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.291+0000 7faa70c6f640 1 -- 172.21.15.84:0/341305575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c0cf5f0 msgr2=0x7faa6c0adbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:10.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.291+0000 7faa70c6f640 1 --2- 172.21.15.84:0/341305575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c0cf5f0 0x7faa6c0adbf0 secure :-1 s=READY pgs=3336 cs=0 l=1 rev1=1 crypto rx=0x7faa6400c400 tx=0x7faa6400c8c0 comp rx=0 tx=0).stop 2023-10-16T11:27:10.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.292+0000 7faa70c6f640 1 -- 172.21.15.84:0/341305575 shutdown_connections 2023-10-16T11:27:10.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.292+0000 7faa70c6f640 1 --2- 172.21.15.84:0/341305575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c0cf5f0 0x7faa6c0adbf0 unknown :-1 s=CLOSED pgs=3336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:10.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.292+0000 7faa70c6f640 1 --2- 172.21.15.84:0/341305575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6c070c10 0x7faa6c0ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:10.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.292+0000 7faa70c6f640 1 --2- 172.21.15.84:0/341305575 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7faa4c06af00 0x7faa4c06d3c0 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009fd0 tx=0x7faa5c00b040 comp rx=0 tx=0).stop 2023-10-16T11:27:10.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.292+0000 7faa70c6f640 1 --2- 172.21.15.84:0/341305575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c070830 0x7faa6c157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:10.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.292+0000 7faa70c6f640 1 -- 172.21.15.84:0/341305575 >> 172.21.15.84:0/341305575 conn(0x7faa6c0cfac0 msgr2=0x7faa6c1601f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:10.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.292+0000 7faa70c6f640 1 -- 172.21.15.84:0/341305575 shutdown_connections 2023-10-16T11:27:10.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.292+0000 7faa70c6f640 1 -- 172.21.15.84:0/341305575 wait complete. 2023-10-16T11:27:10.317 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T11:27:10.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.527+0000 7ffae4940640 1 Processor -- start 2023-10-16T11:27:10.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.528+0000 7ffae4940640 1 -- start start 2023-10-16T11:27:10.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.529+0000 7ffae4940640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae0160c80 0x7ffae0161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:10.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.529+0000 7ffae4940640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffae0070950 0x7ffae0140400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:10.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.529+0000 7ffae4940640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae00ae1a0 con 0x7ffae00ad7a0 2023-10-16T11:27:10.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.529+0000 7ffae4940640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae00ae480 con 0x7ffae0070950 2023-10-16T11:27:10.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.529+0000 7ffae4940640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae00ae760 con 0x7ffae0160c80 2023-10-16T11:27:10.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.530+0000 7ffaded9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae0160c80 0x7ffae0161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:10.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.530+0000 7ffaded9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae0160c80 0x7ffae0161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:57758/0 (socket says 172.21.15.84:57758) 2023-10-16T11:27:10.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.530+0000 7ffaded9d640 1 -- 172.21.15.84:0/3764692454 learned_addr learned my addr 172.21.15.84:0/3764692454 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:10.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.530+0000 7ffadf59e640 1 --2- 172.21.15.84:0/3764692454 >> v2:172.21.15.157:3300/0 conn(0x7ffae0070950 0x7ffae0140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:10.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.531+0000 7ffadf59e640 1 -- 172.21.15.84:0/3764692454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae0160c80 msgr2=0x7ffae0161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:10.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.531+0000 7ffadf59e640 1 --2- 172.21.15.84:0/3764692454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae0160c80 0x7ffae0161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:10.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.531+0000 7ffadf59e640 1 -- 172.21.15.84:0/3764692454 >> v1:172.21.15.157:6789/0 conn(0x7ffae00ad7a0 legacy=0x7ffae0070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:10.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.531+0000 7ffadf59e640 1 -- 172.21.15.84:0/3764692454 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae00aea40 con 0x7ffae0070950 2023-10-16T11:27:10.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.531+0000 7ffaded9d640 1 --2- 172.21.15.84:0/3764692454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae0160c80 0x7ffae0161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:27:10.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.532+0000 7ffadf59e640 1 --2- 172.21.15.84:0/3764692454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae0070950 0x7ffae0140400 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ffad80097a0 tx=0x7ffad802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44626f9e23936a6a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:10.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.532+0000 7ffaddd9b640 1 -- 172.21.15.84:0/3764692454 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad802d040 con 0x7ffae0070950 2023-10-16T11:27:10.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.533+0000 7ffaddd9b640 1 -- 172.21.15.84:0/3764692454 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad8031810 con 0x7ffae0070950 2023-10-16T11:27:10.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.533+0000 7ffaddd9b640 1 -- 172.21.15.84:0/3764692454 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad8031ba0 con 0x7ffae0070950 2023-10-16T11:27:10.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.533+0000 7ffae4940640 1 -- 172.21.15.84:0/3764692454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae0070950 msgr2=0x7ffae0140400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:10.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.533+0000 7ffae4940640 1 --2- 172.21.15.84:0/3764692454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae0070950 0x7ffae0140400 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ffad80097a0 tx=0x7ffad802b8b0 comp rx=0 tx=0).stop 2023-10-16T11:27:10.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.533+0000 7ffae4940640 1 -- 172.21.15.84:0/3764692454 shutdown_connections 2023-10-16T11:27:10.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.533+0000 7ffae4940640 1 --2- 172.21.15.84:0/3764692454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae0070950 0x7ffae0140400 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:10.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.533+0000 7ffae4940640 1 --2- 172.21.15.84:0/3764692454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae0160c80 0x7ffae0161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:10.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.533+0000 7ffae4940640 1 -- 172.21.15.84:0/3764692454 >> 172.21.15.84:0/3764692454 conn(0x7ffae00cc230 msgr2=0x7ffae00cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:10.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.533+0000 7ffae4940640 1 -- 172.21.15.84:0/3764692454 shutdown_connections 2023-10-16T11:27:10.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.533+0000 7ffae4940640 1 -- 172.21.15.84:0/3764692454 wait complete. 2023-10-16T11:27:10.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.535+0000 7ffae4940640 1 Processor -- start 2023-10-16T11:27:10.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.535+0000 7ffae4940640 1 -- start start 2023-10-16T11:27:10.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.536+0000 7ffae4940640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae0070950 0x7ffae0142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:10.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.536+0000 7ffae4940640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae00ad7a0 0x7ffae01f5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:10.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.536+0000 7ffae4940640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae0160c80 0x7ffae00ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:10.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.536+0000 7ffae4940640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae006dd60 con 0x7ffae0070950 2023-10-16T11:27:10.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.536+0000 7ffae4940640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae0142d80 con 0x7ffae00ad7a0 2023-10-16T11:27:10.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.536+0000 7ffae4940640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae0143060 con 0x7ffae0160c80 2023-10-16T11:27:10.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.536+0000 7ffaded9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae0070950 0x7ffae0142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:10.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.536+0000 7ffaded9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae0070950 0x7ffae0142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44118/0 (socket says 172.21.15.84:44118) 2023-10-16T11:27:10.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.537+0000 7ffaded9d640 1 -- 172.21.15.84:0/3639231655 learned_addr learned my addr 172.21.15.84:0/3639231655 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:10.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.537+0000 7ffadf59e640 1 --2- 172.21.15.84:0/3639231655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae0160c80 0x7ffae00ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:10.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.537+0000 7ffade59c640 1 --2- 172.21.15.84:0/3639231655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae00ad7a0 0x7ffae01f5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:10.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.537+0000 7ffaded9d640 1 -- 172.21.15.84:0/3639231655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae0160c80 msgr2=0x7ffae00ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:10.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.537+0000 7ffaded9d640 1 --2- 172.21.15.84:0/3639231655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae0160c80 0x7ffae00ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:10.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.537+0000 7ffaded9d640 1 -- 172.21.15.84:0/3639231655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae00ad7a0 msgr2=0x7ffae01f5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:10.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.537+0000 7ffaded9d640 1 --2- 172.21.15.84:0/3639231655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae00ad7a0 0x7ffae01f5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:10.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.537+0000 7ffaded9d640 1 -- 172.21.15.84:0/3639231655 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac80094d0 con 0x7ffae0070950 2023-10-16T11:27:10.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.538+0000 7ffaded9d640 1 --2- 172.21.15.84:0/3639231655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae0070950 0x7ffae0142680 secure :-1 s=READY pgs=3337 cs=0 l=1 rev1=1 crypto rx=0x7ffac800e9b0 tx=0x7ffac800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:10.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.538+0000 7ffad77fe640 1 -- 172.21.15.84:0/3639231655 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac800c9a0 con 0x7ffae0070950 2023-10-16T11:27:10.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.538+0000 7ffae4940640 1 -- 172.21.15.84:0/3639231655 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad8009400 con 0x7ffae0070950 2023-10-16T11:27:10.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.538+0000 7ffad77fe640 1 -- 172.21.15.84:0/3639231655 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac800cb00 con 0x7ffae0070950 2023-10-16T11:27:10.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.538+0000 7ffad77fe640 1 -- 172.21.15.84:0/3639231655 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac800ce70 con 0x7ffae0070950 2023-10-16T11:27:10.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.538+0000 7ffae4940640 1 -- 172.21.15.84:0/3639231655 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae0143620 con 0x7ffae0070950 2023-10-16T11:27:10.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.539+0000 7ffae4940640 1 -- 172.21.15.84:0/3639231655 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae004e9b0 con 0x7ffae0070950 2023-10-16T11:27:10.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.540+0000 7ffad77fe640 1 -- 172.21.15.84:0/3639231655 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ffac8013470 con 0x7ffae0070950 2023-10-16T11:27:10.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.541+0000 7ffad77fe640 1 --2- 172.21.15.84:0/3639231655 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7ffac006af90 0x7ffac006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:10.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.541+0000 7ffad77fe640 1 -- 172.21.15.84:0/3639231655 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 6683+0+0 (secure 0 0 0) 0x7ffac808a0a0 con 0x7ffae0070950 2023-10-16T11:27:10.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.541+0000 7ffade59c640 1 --2- 172.21.15.84:0/3639231655 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7ffac006af90 0x7ffac006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:10.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.542+0000 7ffade59c640 1 --2- 172.21.15.84:0/3639231655 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7ffac006af90 0x7ffac006d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffae0146430 tx=0x7ffad0009040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:10.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.546+0000 7ffad77fe640 1 -- 172.21.15.84:0/3639231655 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffac8054d00 con 0x7ffae0070950 2023-10-16T11:27:10.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:10.740+0000 7ffae4940640 1 -- 172.21.15.84:0/3639231655 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffae0142ec0 con 0x7ffae0070950 2023-10-16T11:27:11.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.290+0000 7ffad77fe640 1 -- 172.21.15.84:0/3639231655 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v261) v1 ==== 181+0+0 (secure 0 0 0) 0x7ffae0142ec0 con 0x7ffae0070950 2023-10-16T11:27:11.292 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-16T11:27:11.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.293+0000 7ffae4940640 1 -- 172.21.15.84:0/3639231655 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7ffac006af90 msgr2=0x7ffac006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:11.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.293+0000 7ffae4940640 1 --2- 172.21.15.84:0/3639231655 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7ffac006af90 0x7ffac006d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffae0146430 tx=0x7ffad0009040 comp rx=0 tx=0).stop 2023-10-16T11:27:11.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.293+0000 7ffae4940640 1 -- 172.21.15.84:0/3639231655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae0070950 msgr2=0x7ffae0142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:11.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.293+0000 7ffae4940640 1 --2- 172.21.15.84:0/3639231655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae0070950 0x7ffae0142680 secure :-1 s=READY pgs=3337 cs=0 l=1 rev1=1 crypto rx=0x7ffac800e9b0 tx=0x7ffac800ee70 comp rx=0 tx=0).stop 2023-10-16T11:27:11.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.293+0000 7ffae4940640 1 -- 172.21.15.84:0/3639231655 shutdown_connections 2023-10-16T11:27:11.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.293+0000 7ffae4940640 1 --2- 172.21.15.84:0/3639231655 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7ffac006af90 0x7ffac006d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:11.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.293+0000 7ffae4940640 1 --2- 172.21.15.84:0/3639231655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae0160c80 0x7ffae00ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:11.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.293+0000 7ffae4940640 1 --2- 172.21.15.84:0/3639231655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae00ad7a0 0x7ffae01f5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:11.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.293+0000 7ffae4940640 1 --2- 172.21.15.84:0/3639231655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffae0070950 0x7ffae0142680 unknown :-1 s=CLOSED pgs=3337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:11.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.293+0000 7ffae4940640 1 -- 172.21.15.84:0/3639231655 >> 172.21.15.84:0/3639231655 conn(0x7ffae00cc230 msgr2=0x7ffae015f910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:11.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.294+0000 7ffae4940640 1 -- 172.21.15.84:0/3639231655 shutdown_connections 2023-10-16T11:27:11.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.294+0000 7ffae4940640 1 -- 172.21.15.84:0/3639231655 wait complete. 2023-10-16T11:27:11.319 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T11:27:11.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.533+0000 7f777bfff640 1 Processor -- start 2023-10-16T11:27:11.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.534+0000 7f777bfff640 1 -- start start 2023-10-16T11:27:11.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.535+0000 7f777bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f777c0b8480 0x7f777c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:11.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.535+0000 7f777bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777c0d0be0 0x7f777c0cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:11.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.535+0000 7f777bfff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777c15d150 con 0x7f777c15cd70 2023-10-16T11:27:11.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.535+0000 7f777bfff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777c0d1180 con 0x7f777c0b8480 2023-10-16T11:27:11.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.535+0000 7f777bfff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777c0d1460 con 0x7f777c0d0be0 2023-10-16T11:27:11.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.536+0000 7f777b7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777c0d0be0 0x7f777c0cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:11.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.536+0000 7f777b7fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777c0d0be0 0x7f777c0cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:57770/0 (socket says 172.21.15.84:57770) 2023-10-16T11:27:11.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.536+0000 7f777b7fe640 1 -- 172.21.15.84:0/2439914483 learned_addr learned my addr 172.21.15.84:0/2439914483 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:11.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.536+0000 7f777a7fc640 1 --2- 172.21.15.84:0/2439914483 >> v2:172.21.15.157:3300/0 conn(0x7f777c0b8480 0x7f777c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:11.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.537+0000 7f777b7fe640 1 -- 172.21.15.84:0/2439914483 >> v2:172.21.15.157:3300/0 conn(0x7f777c0b8480 msgr2=0x7f777c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:11.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.537+0000 7f777b7fe640 1 --2- 172.21.15.84:0/2439914483 >> v2:172.21.15.157:3300/0 conn(0x7f777c0b8480 0x7f777c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:11.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.537+0000 7f777b7fe640 1 -- 172.21.15.84:0/2439914483 >> v1:172.21.15.157:6789/0 conn(0x7f777c15cd70 legacy=0x7f777c0d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:11.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.537+0000 7f777b7fe640 1 -- 172.21.15.84:0/2439914483 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777c0d1740 con 0x7f777c0d0be0 2023-10-16T11:27:11.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.538+0000 7f777b7fe640 1 --2- 172.21.15.84:0/2439914483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777c0d0be0 0x7f777c0cdd00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7774009870 tx=0x7f777402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c0e80ed082ae9f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:11.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.539+0000 7f7779ffb640 1 -- 172.21.15.84:0/2439914483 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7774030840 con 0x7f777c0d0be0 2023-10-16T11:27:11.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.539+0000 7f7779ffb640 1 -- 172.21.15.84:0/2439914483 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77740309a0 con 0x7f777c0d0be0 2023-10-16T11:27:11.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.539+0000 7f7779ffb640 1 -- 172.21.15.84:0/2439914483 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7774030d30 con 0x7f777c0d0be0 2023-10-16T11:27:11.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.540+0000 7f777bfff640 1 -- 172.21.15.84:0/2439914483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777c0d0be0 msgr2=0x7f777c0cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:11.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.540+0000 7f777bfff640 1 --2- 172.21.15.84:0/2439914483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777c0d0be0 0x7f777c0cdd00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7774009870 tx=0x7f777402b980 comp rx=0 tx=0).stop 2023-10-16T11:27:11.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.540+0000 7f777bfff640 1 -- 172.21.15.84:0/2439914483 shutdown_connections 2023-10-16T11:27:11.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.540+0000 7f777bfff640 1 --2- 172.21.15.84:0/2439914483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777c0d0be0 0x7f777c0cdd00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:11.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.540+0000 7f777bfff640 1 --2- 172.21.15.84:0/2439914483 >> v2:172.21.15.157:3300/0 conn(0x7f777c0b8480 0x7f777c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:11.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.540+0000 7f777bfff640 1 -- 172.21.15.84:0/2439914483 >> 172.21.15.84:0/2439914483 conn(0x7f777c0cf570 msgr2=0x7f777c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:11.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.540+0000 7f777bfff640 1 -- 172.21.15.84:0/2439914483 shutdown_connections 2023-10-16T11:27:11.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.540+0000 7f777bfff640 1 -- 172.21.15.84:0/2439914483 wait complete. 2023-10-16T11:27:11.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.541+0000 7f777bfff640 1 Processor -- start 2023-10-16T11:27:11.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.542+0000 7f777bfff640 1 -- start start 2023-10-16T11:27:11.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.543+0000 7f777bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c0b8480 0x7f777c15f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:11.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.543+0000 7f777bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f777c0d0be0 0x7f777c15fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:11.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.543+0000 7f777bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777c15cd70 0x7f777c1f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:11.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.543+0000 7f777bfff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c15ebf0 con 0x7f777c0b8480 2023-10-16T11:27:11.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.543+0000 7f777bfff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c0cca10 con 0x7f777c0d0be0 2023-10-16T11:27:11.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.543+0000 7f777bfff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777c15e2c0 con 0x7f777c15cd70 2023-10-16T11:27:11.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.543+0000 7f777affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c0b8480 0x7f777c15f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:11.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.543+0000 7f777affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c0b8480 0x7f777c15f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44128/0 (socket says 172.21.15.84:44128) 2023-10-16T11:27:11.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.543+0000 7f777affd640 1 -- 172.21.15.84:0/1168944665 learned_addr learned my addr 172.21.15.84:0/1168944665 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:11.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.543+0000 7f777b7fe640 1 --2- 172.21.15.84:0/1168944665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777c15cd70 0x7f777c1f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:11.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.543+0000 7f777a7fc640 1 --2- 172.21.15.84:0/1168944665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f777c0d0be0 0x7f777c15fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:11.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.543+0000 7f777affd640 1 -- 172.21.15.84:0/1168944665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777c15cd70 msgr2=0x7f777c1f5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:11.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.543+0000 7f777affd640 1 --2- 172.21.15.84:0/1168944665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777c15cd70 0x7f777c1f5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:11.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.543+0000 7f777affd640 1 -- 172.21.15.84:0/1168944665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f777c0d0be0 msgr2=0x7f777c15fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:11.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.543+0000 7f777affd640 1 --2- 172.21.15.84:0/1168944665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f777c0d0be0 0x7f777c15fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:11.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.543+0000 7f777affd640 1 -- 172.21.15.84:0/1168944665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77740094d0 con 0x7f777c0b8480 2023-10-16T11:27:11.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.543+0000 7f777b7fe640 1 --2- 172.21.15.84:0/1168944665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777c15cd70 0x7f777c1f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:11.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.544+0000 7f777affd640 1 --2- 172.21.15.84:0/1168944665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c0b8480 0x7f777c15f4c0 secure :-1 s=READY pgs=3338 cs=0 l=1 rev1=1 crypto rx=0x7f776c00c400 tx=0x7f776c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:11.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.544+0000 7f77637fe640 1 -- 172.21.15.84:0/1168944665 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c0052d0 con 0x7f777c0b8480 2023-10-16T11:27:11.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.544+0000 7f77637fe640 1 -- 172.21.15.84:0/1168944665 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776c005450 con 0x7f777c0b8480 2023-10-16T11:27:11.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.544+0000 7f77637fe640 1 -- 172.21.15.84:0/1168944665 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c007260 con 0x7f777c0b8480 2023-10-16T11:27:11.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.544+0000 7f777bfff640 1 -- 172.21.15.84:0/1168944665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f777c15e5a0 con 0x7f777c0b8480 2023-10-16T11:27:11.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.544+0000 7f777bfff640 1 -- 172.21.15.84:0/1168944665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f777c155cd0 con 0x7f777c0b8480 2023-10-16T11:27:11.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.545+0000 7f777bfff640 1 -- 172.21.15.84:0/1168944665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777c153a30 con 0x7f777c0b8480 2023-10-16T11:27:11.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.546+0000 7f77637fe640 1 -- 172.21.15.84:0/1168944665 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f776c0073c0 con 0x7f777c0b8480 2023-10-16T11:27:11.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.547+0000 7f77637fe640 1 --2- 172.21.15.84:0/1168944665 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f776806af90 0x7f776806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:11.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.547+0000 7f77637fe640 1 -- 172.21.15.84:0/1168944665 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f776c088de0 con 0x7f777c0b8480 2023-10-16T11:27:11.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.548+0000 7f777a7fc640 1 --2- 172.21.15.84:0/1168944665 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f776806af90 0x7f776806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:11.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.548+0000 7f777a7fc640 1 --2- 172.21.15.84:0/1168944665 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f776806af90 0x7f776806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f777c1f5b60 tx=0x7f777000b040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:11.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.552+0000 7f77637fe640 1 -- 172.21.15.84:0/1168944665 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f776c053b60 con 0x7f777c0b8480 2023-10-16T11:27:11.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:11.745+0000 7f777bfff640 1 -- 172.21.15.84:0/1168944665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f777c04e9b0 con 0x7f777c0b8480 2023-10-16T11:27:12.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.305+0000 7f77637fe640 1 -- 172.21.15.84:0/1168944665 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v262) v1 ==== 169+0+0 (secure 0 0 0) 0x7f776c057180 con 0x7f777c0b8480 2023-10-16T11:27:12.308 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-16T11:27:12.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.310+0000 7f777bfff640 1 -- 172.21.15.84:0/1168944665 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f776806af90 msgr2=0x7f776806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:12.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.310+0000 7f777bfff640 1 --2- 172.21.15.84:0/1168944665 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f776806af90 0x7f776806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f777c1f5b60 tx=0x7f777000b040 comp rx=0 tx=0).stop 2023-10-16T11:27:12.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.310+0000 7f777bfff640 1 -- 172.21.15.84:0/1168944665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c0b8480 msgr2=0x7f777c15f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:12.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.310+0000 7f777bfff640 1 --2- 172.21.15.84:0/1168944665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c0b8480 0x7f777c15f4c0 secure :-1 s=READY pgs=3338 cs=0 l=1 rev1=1 crypto rx=0x7f776c00c400 tx=0x7f776c00c8c0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.311+0000 7f777bfff640 1 -- 172.21.15.84:0/1168944665 shutdown_connections 2023-10-16T11:27:12.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.311+0000 7f777bfff640 1 --2- 172.21.15.84:0/1168944665 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f776806af90 0x7f776806d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.311+0000 7f777bfff640 1 --2- 172.21.15.84:0/1168944665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777c15cd70 0x7f777c1f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.311+0000 7f777bfff640 1 --2- 172.21.15.84:0/1168944665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f777c0d0be0 0x7f777c15fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.311+0000 7f777bfff640 1 --2- 172.21.15.84:0/1168944665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c0b8480 0x7f777c15f4c0 unknown :-1 s=CLOSED pgs=3338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.311+0000 7f777bfff640 1 -- 172.21.15.84:0/1168944665 >> 172.21.15.84:0/1168944665 conn(0x7f777c0cf570 msgr2=0x7f777c0b64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:12.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.312+0000 7f777bfff640 1 -- 172.21.15.84:0/1168944665 shutdown_connections 2023-10-16T11:27:12.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.313+0000 7f777bfff640 1 -- 172.21.15.84:0/1168944665 wait complete. 2023-10-16T11:27:12.340 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T11:27:12.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.537+0000 7f6244733640 1 Processor -- start 2023-10-16T11:27:12.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.537+0000 7f6244733640 1 -- start start 2023-10-16T11:27:12.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.539+0000 7f6244733640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f623c06b660 0x7f623c06ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:12.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.539+0000 7f6244733640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f623c0c9860 0x7f623c0ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:12.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.539+0000 7f6244733640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f623c0cc6b0 con 0x7f623c1428c0 2023-10-16T11:27:12.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.539+0000 7f6244733640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f623c1599b0 con 0x7f623c06b660 2023-10-16T11:27:12.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.539+0000 7f6244733640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f623c06c290 con 0x7f623c0c9860 2023-10-16T11:27:12.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.539+0000 7f62424cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f623c06b660 0x7f623c06ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:12.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.539+0000 7f62424cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f623c06b660 0x7f623c06ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44130/0 (socket says 172.21.15.84:44130) 2023-10-16T11:27:12.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.539+0000 7f62424cf640 1 -- 172.21.15.84:0/2878595891 learned_addr learned my addr 172.21.15.84:0/2878595891 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:12.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.540+0000 7f6242cd0640 1 --2- 172.21.15.84:0/2878595891 >> v2:172.21.15.157:3300/0 conn(0x7f623c0c9860 0x7f623c0ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:12.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.540+0000 7f62424cf640 1 -- 172.21.15.84:0/2878595891 >> v2:172.21.15.157:3300/0 conn(0x7f623c0c9860 msgr2=0x7f623c0ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:12.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.540+0000 7f62424cf640 1 --2- 172.21.15.84:0/2878595891 >> v2:172.21.15.157:3300/0 conn(0x7f623c0c9860 0x7f623c0ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.540+0000 7f62424cf640 1 -- 172.21.15.84:0/2878595891 >> v1:172.21.15.84:6789/0 conn(0x7f623c1428c0 legacy=0x7f623c0c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:12.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.540+0000 7f62424cf640 1 -- 172.21.15.84:0/2878595891 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c06c570 con 0x7f623c06b660 2023-10-16T11:27:12.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.541+0000 7f62424cf640 1 --2- 172.21.15.84:0/2878595891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f623c06b660 0x7f623c06ba40 secure :-1 s=READY pgs=3340 cs=0 l=1 rev1=1 crypto rx=0x7f623c0b0430 tx=0x7f623802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b75699224f6ea14 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:12.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.541+0000 7f62414cd640 1 -- 172.21.15.84:0/2878595891 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6238030860 con 0x7f623c06b660 2023-10-16T11:27:12.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.541+0000 7f62414cd640 1 -- 172.21.15.84:0/2878595891 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62380309c0 con 0x7f623c06b660 2023-10-16T11:27:12.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.541+0000 7f62414cd640 1 -- 172.21.15.84:0/2878595891 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6238030d50 con 0x7f623c06b660 2023-10-16T11:27:12.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.542+0000 7f6244733640 1 -- 172.21.15.84:0/2878595891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f623c06b660 msgr2=0x7f623c06ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:12.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.542+0000 7f6244733640 1 --2- 172.21.15.84:0/2878595891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f623c06b660 0x7f623c06ba40 secure :-1 s=READY pgs=3340 cs=0 l=1 rev1=1 crypto rx=0x7f623c0b0430 tx=0x7f623802b9a0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.542+0000 7f6244733640 1 -- 172.21.15.84:0/2878595891 shutdown_connections 2023-10-16T11:27:12.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.542+0000 7f6244733640 1 --2- 172.21.15.84:0/2878595891 >> v2:172.21.15.157:3300/0 conn(0x7f623c0c9860 0x7f623c0ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.542+0000 7f6244733640 1 --2- 172.21.15.84:0/2878595891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f623c06b660 0x7f623c06ba40 unknown :-1 s=CLOSED pgs=3340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.542+0000 7f6244733640 1 -- 172.21.15.84:0/2878595891 >> 172.21.15.84:0/2878595891 conn(0x7f623c0ce9b0 msgr2=0x7f623c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:12.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.542+0000 7f6244733640 1 -- 172.21.15.84:0/2878595891 shutdown_connections 2023-10-16T11:27:12.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.542+0000 7f6244733640 1 -- 172.21.15.84:0/2878595891 wait complete. 2023-10-16T11:27:12.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.544+0000 7f6244733640 1 Processor -- start 2023-10-16T11:27:12.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.544+0000 7f6244733640 1 -- start start 2023-10-16T11:27:12.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.545+0000 7f6244733640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f623c06b660 0x7f623c159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:12.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.545+0000 7f6244733640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f623c0c9860 0x7f623c0af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:12.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.545+0000 7f6244733640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f623c1428c0 0x7f623c14e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:12.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.545+0000 7f6244733640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c1684e0 con 0x7f623c1428c0 2023-10-16T11:27:12.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.545+0000 7f6244733640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c15a060 con 0x7f623c06b660 2023-10-16T11:27:12.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.545+0000 7f6244733640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623c167db0 con 0x7f623c0c9860 2023-10-16T11:27:12.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.545+0000 7f6242cd0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f623c1428c0 0x7f623c14e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:12.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.545+0000 7f6242cd0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f623c1428c0 0x7f623c14e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44142/0 (socket says 172.21.15.84:44142) 2023-10-16T11:27:12.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.545+0000 7f6242cd0640 1 -- 172.21.15.84:0/3465291684 learned_addr learned my addr 172.21.15.84:0/3465291684 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:12.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.546+0000 7f62424cf640 1 --2- 172.21.15.84:0/3465291684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f623c06b660 0x7f623c159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:12.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.546+0000 7f6241cce640 1 --2- 172.21.15.84:0/3465291684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f623c0c9860 0x7f623c0af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:12.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.546+0000 7f6242cd0640 1 -- 172.21.15.84:0/3465291684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f623c0c9860 msgr2=0x7f623c0af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:12.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.546+0000 7f6242cd0640 1 --2- 172.21.15.84:0/3465291684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f623c0c9860 0x7f623c0af510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.546+0000 7f6242cd0640 1 -- 172.21.15.84:0/3465291684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f623c06b660 msgr2=0x7f623c159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:12.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.546+0000 7f6242cd0640 1 --2- 172.21.15.84:0/3465291684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f623c06b660 0x7f623c159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.546+0000 7f6242cd0640 1 -- 172.21.15.84:0/3465291684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62380094d0 con 0x7f623c1428c0 2023-10-16T11:27:12.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.546+0000 7f6242cd0640 1 --2- 172.21.15.84:0/3465291684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f623c1428c0 0x7f623c14e0e0 secure :-1 s=READY pgs=3341 cs=0 l=1 rev1=1 crypto rx=0x7f6234007e30 tx=0x7f623400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:12.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.546+0000 7f6232ffd640 1 -- 172.21.15.84:0/3465291684 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234014620 con 0x7f623c1428c0 2023-10-16T11:27:12.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.546+0000 7f6232ffd640 1 -- 172.21.15.84:0/3465291684 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623400ee90 con 0x7f623c1428c0 2023-10-16T11:27:12.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.546+0000 7f6232ffd640 1 -- 172.21.15.84:0/3465291684 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62340149c0 con 0x7f623c1428c0 2023-10-16T11:27:12.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.546+0000 7f6244733640 1 -- 172.21.15.84:0/3465291684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623c14f7f0 con 0x7f623c1428c0 2023-10-16T11:27:12.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.547+0000 7f6244733640 1 -- 172.21.15.84:0/3465291684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623c14fdb0 con 0x7f623c1428c0 2023-10-16T11:27:12.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.548+0000 7f6244733640 1 -- 172.21.15.84:0/3465291684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623c06f470 con 0x7f623c1428c0 2023-10-16T11:27:12.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.549+0000 7f6232ffd640 1 -- 172.21.15.84:0/3465291684 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6234014b20 con 0x7f623c1428c0 2023-10-16T11:27:12.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.549+0000 7f6232ffd640 1 --2- 172.21.15.84:0/3465291684 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f622006af90 0x7f622006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:12.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.550+0000 7f6232ffd640 1 -- 172.21.15.84:0/3465291684 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 5831+0+0 (secure 0 0 0) 0x7f623408a0e0 con 0x7f623c1428c0 2023-10-16T11:27:12.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.550+0000 7f62424cf640 1 --2- 172.21.15.84:0/3465291684 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f622006af90 0x7f622006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:12.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.551+0000 7f62424cf640 1 --2- 172.21.15.84:0/3465291684 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f622006af90 0x7f622006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6238036000 tx=0x7f623803b040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:12.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.554+0000 7f6232ffd640 1 -- 172.21.15.84:0/3465291684 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6234055090 con 0x7f623c1428c0 2023-10-16T11:27:12.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.749+0000 7f6244733640 1 -- 172.21.15.84:0/3465291684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f623c04e160 con 0x7f623c1428c0 2023-10-16T11:27:12.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.757+0000 7f6232ffd640 1 -- 172.21.15.84:0/3465291684 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v91) v1 ==== 104+0+0 (secure 0 0 0) 0x7f62340586b0 con 0x7f623c1428c0 2023-10-16T11:27:12.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.760+0000 7f6244733640 1 -- 172.21.15.84:0/3465291684 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f622006af90 msgr2=0x7f622006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:12.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.760+0000 7f6244733640 1 --2- 172.21.15.84:0/3465291684 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f622006af90 0x7f622006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6238036000 tx=0x7f623803b040 comp rx=0 tx=0).stop 2023-10-16T11:27:12.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.760+0000 7f6244733640 1 -- 172.21.15.84:0/3465291684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f623c1428c0 msgr2=0x7f623c14e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:12.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.760+0000 7f6244733640 1 --2- 172.21.15.84:0/3465291684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f623c1428c0 0x7f623c14e0e0 secure :-1 s=READY pgs=3341 cs=0 l=1 rev1=1 crypto rx=0x7f6234007e30 tx=0x7f623400e4b0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.760+0000 7f6244733640 1 -- 172.21.15.84:0/3465291684 shutdown_connections 2023-10-16T11:27:12.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.760+0000 7f6244733640 1 --2- 172.21.15.84:0/3465291684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f623c1428c0 0x7f623c14e0e0 unknown :-1 s=CLOSED pgs=3341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.760+0000 7f6244733640 1 --2- 172.21.15.84:0/3465291684 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f622006af90 0x7f622006d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.760+0000 7f6244733640 1 --2- 172.21.15.84:0/3465291684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f623c0c9860 0x7f623c0af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.760+0000 7f6244733640 1 --2- 172.21.15.84:0/3465291684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f623c06b660 0x7f623c159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.760+0000 7f6244733640 1 -- 172.21.15.84:0/3465291684 >> 172.21.15.84:0/3465291684 conn(0x7f623c0ce9b0 msgr2=0x7f623c0ae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:12.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.760+0000 7f6244733640 1 -- 172.21.15.84:0/3465291684 shutdown_connections 2023-10-16T11:27:12.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.760+0000 7f6244733640 1 -- 172.21.15.84:0/3465291684 wait complete. 2023-10-16T11:27:12.780 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T11:27:12.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.984+0000 7f7407565640 1 Processor -- start 2023-10-16T11:27:12.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.984+0000 7f7407565640 1 -- start start 2023-10-16T11:27:12.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.986+0000 7f7407565640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7400159ab0 0x7f74000ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:12.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.986+0000 7f7407565640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f740006c140 0x7f740006c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:12.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.986+0000 7f7407565640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740006cad0 con 0x7f7400159e90 2023-10-16T11:27:12.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.986+0000 7f7407565640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f740015d890 con 0x7f7400159ab0 2023-10-16T11:27:12.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.986+0000 7f7407565640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7400070360 con 0x7f740006c140 2023-10-16T11:27:12.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.986+0000 7f7405301640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7400159ab0 0x7f74000ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:12.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.986+0000 7f7405301640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7400159ab0 0x7f74000ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44150/0 (socket says 172.21.15.84:44150) 2023-10-16T11:27:12.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.986+0000 7f7405301640 1 -- 172.21.15.84:0/276520441 learned_addr learned my addr 172.21.15.84:0/276520441 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:12.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.986+0000 7f7405301640 1 -- 172.21.15.84:0/276520441 >> v2:172.21.15.157:3300/0 conn(0x7f740006c140 msgr2=0x7f740006c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:12.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.987+0000 7f7405301640 1 --2- 172.21.15.84:0/276520441 >> v2:172.21.15.157:3300/0 conn(0x7f740006c140 0x7f740006c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.987+0000 7f7405301640 1 -- 172.21.15.84:0/276520441 >> v1:172.21.15.84:6789/0 conn(0x7f7400159e90 legacy=0x7f740006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:12.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.987+0000 7f7405301640 1 -- 172.21.15.84:0/276520441 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74000cd1f0 con 0x7f7400159ab0 2023-10-16T11:27:12.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.988+0000 7f7405301640 1 --2- 172.21.15.84:0/276520441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7400159ab0 0x7f74000ce370 secure :-1 s=READY pgs=3343 cs=0 l=1 rev1=1 crypto rx=0x7f74000adcd0 tx=0x7f73fc02da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae4a7b287eb9dfe server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:12.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.988+0000 7f73f7fff640 1 -- 172.21.15.84:0/276520441 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc031820 con 0x7f7400159ab0 2023-10-16T11:27:12.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.988+0000 7f73f7fff640 1 -- 172.21.15.84:0/276520441 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73fc031980 con 0x7f7400159ab0 2023-10-16T11:27:12.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.988+0000 7f73f7fff640 1 -- 172.21.15.84:0/276520441 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc031d10 con 0x7f7400159ab0 2023-10-16T11:27:12.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.989+0000 7f7407565640 1 -- 172.21.15.84:0/276520441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7400159ab0 msgr2=0x7f74000ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:12.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.989+0000 7f7407565640 1 --2- 172.21.15.84:0/276520441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7400159ab0 0x7f74000ce370 secure :-1 s=READY pgs=3343 cs=0 l=1 rev1=1 crypto rx=0x7f74000adcd0 tx=0x7f73fc02da00 comp rx=0 tx=0).stop 2023-10-16T11:27:12.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.989+0000 7f7407565640 1 -- 172.21.15.84:0/276520441 shutdown_connections 2023-10-16T11:27:12.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.989+0000 7f7407565640 1 --2- 172.21.15.84:0/276520441 >> v2:172.21.15.157:3300/0 conn(0x7f740006c140 0x7f740006c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.989+0000 7f7407565640 1 --2- 172.21.15.84:0/276520441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7400159ab0 0x7f74000ce370 unknown :-1 s=CLOSED pgs=3343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.989+0000 7f7407565640 1 -- 172.21.15.84:0/276520441 >> 172.21.15.84:0/276520441 conn(0x7f74000ce9b0 msgr2=0x7f74000cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:12.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.989+0000 7f7407565640 1 -- 172.21.15.84:0/276520441 shutdown_connections 2023-10-16T11:27:12.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.989+0000 7f7407565640 1 -- 172.21.15.84:0/276520441 wait complete. 2023-10-16T11:27:12.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.991+0000 7f7407565640 1 Processor -- start 2023-10-16T11:27:12.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.991+0000 7f7407565640 1 -- start start 2023-10-16T11:27:12.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.992+0000 7f7407565640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740006c140 0x7f740015cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:12.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.992+0000 7f7407565640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7400159ab0 0x7f7400156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:12.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.992+0000 7f7407565640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7400159e90 0x7f74000c3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:12.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.992+0000 7f7407565640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74000c2cb0 con 0x7f7400159ab0 2023-10-16T11:27:12.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.992+0000 7f7407565640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740015bc80 con 0x7f740006c140 2023-10-16T11:27:12.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.992+0000 7f7407565640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740015bf60 con 0x7f7400159e90 2023-10-16T11:27:12.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.992+0000 7f7404b00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7400159ab0 0x7f7400156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:12.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.992+0000 7f7404b00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7400159ab0 0x7f7400156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44166/0 (socket says 172.21.15.84:44166) 2023-10-16T11:27:12.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.992+0000 7f7404b00640 1 -- 172.21.15.84:0/246680975 learned_addr learned my addr 172.21.15.84:0/246680975 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:12.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.992+0000 7f7404b00640 1 -- 172.21.15.84:0/246680975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7400159e90 msgr2=0x7f74000c3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:12.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.992+0000 7f7404b00640 1 --2- 172.21.15.84:0/246680975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7400159e90 0x7f74000c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.992+0000 7f7404b00640 1 -- 172.21.15.84:0/246680975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740006c140 msgr2=0x7f740015cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:12.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.992+0000 7f7404b00640 1 --2- 172.21.15.84:0/246680975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740006c140 0x7f740015cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:12.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.992+0000 7f7404b00640 1 -- 172.21.15.84:0/246680975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc0094d0 con 0x7f7400159ab0 2023-10-16T11:27:12.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.993+0000 7f7404b00640 1 --2- 172.21.15.84:0/246680975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7400159ab0 0x7f7400156d30 secure :-1 s=READY pgs=3344 cs=0 l=1 rev1=1 crypto rx=0x7f73f000c430 tx=0x7f73f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:12.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.993+0000 7f73f5ffb640 1 -- 172.21.15.84:0/246680975 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0011790 con 0x7f7400159ab0 2023-10-16T11:27:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.993+0000 7f73f5ffb640 1 -- 172.21.15.84:0/246680975 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f00118f0 con 0x7f7400159ab0 2023-10-16T11:27:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.993+0000 7f73f5ffb640 1 -- 172.21.15.84:0/246680975 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0011ce0 con 0x7f7400159ab0 2023-10-16T11:27:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.993+0000 7f7407565640 1 -- 172.21.15.84:0/246680975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740015c240 con 0x7f7400159ab0 2023-10-16T11:27:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.993+0000 7f7407565640 1 -- 172.21.15.84:0/246680975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740015c800 con 0x7f7400159ab0 2023-10-16T11:27:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.994+0000 7f7407565640 1 -- 172.21.15.84:0/246680975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740004e160 con 0x7f7400159ab0 2023-10-16T11:27:13.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.995+0000 7f73f5ffb640 1 -- 172.21.15.84:0/246680975 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f73f001c460 con 0x7f7400159ab0 2023-10-16T11:27:13.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.996+0000 7f73f5ffb640 1 --2- 172.21.15.84:0/246680975 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f73e406af90 0x7f73e406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:13.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.996+0000 7f73f5ffb640 1 -- 172.21.15.84:0/246680975 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 5831+0+0 (secure 0 0 0) 0x7f73f0089870 con 0x7f7400159ab0 2023-10-16T11:27:13.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.997+0000 7f7405301640 1 --2- 172.21.15.84:0/246680975 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f73e406af90 0x7f73e406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:13.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:12.997+0000 7f7405301640 1 --2- 172.21.15.84:0/246680975 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f73e406af90 0x7f73e406d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f73fc038000 tx=0x7f73fc032000 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:13.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.001+0000 7f73f5ffb640 1 -- 172.21.15.84:0/246680975 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f73f00546f0 con 0x7f7400159ab0 2023-10-16T11:27:13.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.192+0000 7f7407565640 1 -- 172.21.15.84:0/246680975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f74000ccb60 con 0x7f7400159ab0 2023-10-16T11:27:13.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.197+0000 7f73f5ffb640 1 -- 172.21.15.84:0/246680975 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v92) v1 ==== 120+0+0 (secure 0 0 0) 0x7f73f0057d10 con 0x7f7400159ab0 2023-10-16T11:27:13.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.200+0000 7f7407565640 1 -- 172.21.15.84:0/246680975 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f73e406af90 msgr2=0x7f73e406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:13.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.200+0000 7f7407565640 1 --2- 172.21.15.84:0/246680975 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f73e406af90 0x7f73e406d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f73fc038000 tx=0x7f73fc032000 comp rx=0 tx=0).stop 2023-10-16T11:27:13.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.200+0000 7f7407565640 1 -- 172.21.15.84:0/246680975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7400159ab0 msgr2=0x7f7400156d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:13.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.200+0000 7f7407565640 1 --2- 172.21.15.84:0/246680975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7400159ab0 0x7f7400156d30 secure :-1 s=READY pgs=3344 cs=0 l=1 rev1=1 crypto rx=0x7f73f000c430 tx=0x7f73f000c8f0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.200+0000 7f7407565640 1 -- 172.21.15.84:0/246680975 shutdown_connections 2023-10-16T11:27:13.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.200+0000 7f7407565640 1 --2- 172.21.15.84:0/246680975 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f73e406af90 0x7f73e406d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.200+0000 7f7407565640 1 --2- 172.21.15.84:0/246680975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7400159e90 0x7f74000c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.200+0000 7f7407565640 1 --2- 172.21.15.84:0/246680975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7400159ab0 0x7f7400156d30 unknown :-1 s=CLOSED pgs=3344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.200+0000 7f7407565640 1 --2- 172.21.15.84:0/246680975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f740006c140 0x7f740015cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.200+0000 7f7407565640 1 -- 172.21.15.84:0/246680975 >> 172.21.15.84:0/246680975 conn(0x7f74000ce9b0 msgr2=0x7f74000b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:13.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.201+0000 7f7407565640 1 -- 172.21.15.84:0/246680975 shutdown_connections 2023-10-16T11:27:13.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.201+0000 7f7407565640 1 -- 172.21.15.84:0/246680975 wait complete. 2023-10-16T11:27:13.221 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T11:27:13.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.432+0000 7f6b660bb640 1 Processor -- start 2023-10-16T11:27:13.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.432+0000 7f6b660bb640 1 -- start start 2023-10-16T11:27:13.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.434+0000 7f6b660bb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b60159f00 0x7f6b600b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:13.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.434+0000 7f6b660bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b6015d8c0 0x7f6b600b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:13.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.434+0000 7f6b660bb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6006f890 con 0x7f6b60159b20 2023-10-16T11:27:13.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.434+0000 7f6b660bb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b60070d30 con 0x7f6b60159f00 2023-10-16T11:27:13.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.434+0000 7f6b660bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6006c650 con 0x7f6b6015d8c0 2023-10-16T11:27:13.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.434+0000 7f6b5f7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6b60159b20 0x7f6b6006bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57370/0 (socket says 172.21.15.84:57370) 2023-10-16T11:27:13.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.434+0000 7f6b5f7fe640 1 -- 172.21.15.84:0/2499469799 learned_addr learned my addr 172.21.15.84:0/2499469799 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:13.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5e7fc640 1 -- 172.21.15.84:0/2499469799 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 119310668 0 0) 0x7f6b6006f890 con 0x7f6b60159b20 2023-10-16T11:27:13.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5e7fc640 1 -- 172.21.15.84:0/2499469799 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b4c003520 con 0x7f6b60159b20 2023-10-16T11:27:13.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5effd640 1 --2- 172.21.15.84:0/2499469799 >> v2:172.21.15.157:3300/0 conn(0x7f6b60159f00 0x7f6b600b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:13.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5ffff640 1 --2- 172.21.15.84:0/2499469799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b6015d8c0 0x7f6b600b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:13.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5e7fc640 1 -- 172.21.15.84:0/2499469799 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2484091938 0 0) 0x7f6b4c003520 con 0x7f6b60159b20 2023-10-16T11:27:13.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5e7fc640 1 -- 172.21.15.84:0/2499469799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b6015d8c0 msgr2=0x7f6b600b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:13.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5e7fc640 1 --2- 172.21.15.84:0/2499469799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b6015d8c0 0x7f6b600b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5e7fc640 1 -- 172.21.15.84:0/2499469799 >> v2:172.21.15.157:3300/0 conn(0x7f6b60159f00 msgr2=0x7f6b600b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:13.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5e7fc640 1 --2- 172.21.15.84:0/2499469799 >> v2:172.21.15.157:3300/0 conn(0x7f6b60159f00 0x7f6b600b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5e7fc640 1 -- 172.21.15.84:0/2499469799 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6006c930 con 0x7f6b60159b20 2023-10-16T11:27:13.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5e7fc640 1 -- 172.21.15.84:0/2499469799 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f6b500019f0 con 0x7f6b60159b20 2023-10-16T11:27:13.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5e7fc640 1 -- 172.21.15.84:0/2499469799 >> v1:172.21.15.84:6789/0 conn(0x7f6b60159b20 legacy=0x7f6b6006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:13.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5e7fc640 1 --2- 172.21.15.84:0/2499469799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b6015d8c0 0x7f6b4c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:13.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5e7fc640 1 --2- 172.21.15.84:0/2499469799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b4c003fa0 0x7f6b4c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:13.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5e7fc640 1 --2- 172.21.15.84:0/2499469799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b4c004d00 0x7f6b4c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:13.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5e7fc640 1 -- 172.21.15.84:0/2499469799 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6006c650 con 0x7f6b4c004d00 2023-10-16T11:27:13.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5e7fc640 1 -- 172.21.15.84:0/2499469799 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c00dd20 con 0x7f6b4c003fa0 2023-10-16T11:27:13.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5e7fc640 1 -- 172.21.15.84:0/2499469799 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c00dee0 con 0x7f6b6015d8c0 2023-10-16T11:27:13.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.435+0000 7f6b5ffff640 1 --2- 172.21.15.84:0/2499469799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b4c004d00 0x7f6b4c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:13.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.436+0000 7f6b5ffff640 1 -- 172.21.15.84:0/2499469799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b6015d8c0 msgr2=0x7f6b4c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:13.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.436+0000 7f6b5ffff640 1 --2- 172.21.15.84:0/2499469799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b6015d8c0 0x7f6b4c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.436+0000 7f6b5effd640 1 --2- 172.21.15.84:0/2499469799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b4c003fa0 0x7f6b4c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:13.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.436+0000 7f6b5ffff640 1 -- 172.21.15.84:0/2499469799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b4c003fa0 msgr2=0x7f6b4c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:13.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.436+0000 7f6b5ffff640 1 --2- 172.21.15.84:0/2499469799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b4c003fa0 0x7f6b4c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.436+0000 7f6b5ffff640 1 -- 172.21.15.84:0/2499469799 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c003520 con 0x7f6b4c004d00 2023-10-16T11:27:13.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.437+0000 7f6b5ffff640 1 --2- 172.21.15.84:0/2499469799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b4c004d00 0x7f6b4c005180 secure :-1 s=READY pgs=3346 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800a970 tx=0x7f6b4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a019672d49b371 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:13.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.437+0000 7f6b5e7fc640 1 -- 172.21.15.84:0/2499469799 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58030620 con 0x7f6b4c004d00 2023-10-16T11:27:13.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.437+0000 7f6b5e7fc640 1 -- 172.21.15.84:0/2499469799 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b58030780 con 0x7f6b4c004d00 2023-10-16T11:27:13.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.437+0000 7f6b5e7fc640 1 -- 172.21.15.84:0/2499469799 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58030b30 con 0x7f6b4c004d00 2023-10-16T11:27:13.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.438+0000 7f6b660bb640 1 -- 172.21.15.84:0/2499469799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b4c004d00 msgr2=0x7f6b4c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:13.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.438+0000 7f6b660bb640 1 --2- 172.21.15.84:0/2499469799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b4c004d00 0x7f6b4c005180 secure :-1 s=READY pgs=3346 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800a970 tx=0x7f6b4c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.438+0000 7f6b5ffff640 1 -- 172.21.15.84:0/2499469799 reap_dead start 2023-10-16T11:27:13.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.438+0000 7f6b660bb640 1 -- 172.21.15.84:0/2499469799 shutdown_connections 2023-10-16T11:27:13.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.438+0000 7f6b660bb640 1 -- 172.21.15.84:0/2499469799 >> 172.21.15.84:0/2499469799 conn(0x7f6b600cb410 msgr2=0x7f6b600cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:13.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.438+0000 7f6b660bb640 1 -- 172.21.15.84:0/2499469799 shutdown_connections 2023-10-16T11:27:13.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.438+0000 7f6b660bb640 1 -- 172.21.15.84:0/2499469799 wait complete. 2023-10-16T11:27:13.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.439+0000 7f6b660bb640 1 Processor -- start 2023-10-16T11:27:13.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.440+0000 7f6b660bb640 1 -- start start 2023-10-16T11:27:13.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.440+0000 7f6b660bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b4c004d00 0x7f6b601f3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:13.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.441+0000 7f6b660bb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b600c7720 0x7f6b601f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:13.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.441+0000 7f6b660bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b601f6300 0x7f6b600cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:13.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.441+0000 7f6b660bb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b601f6880 con 0x7f6b600c7720 2023-10-16T11:27:13.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.441+0000 7f6b660bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b601f6b60 con 0x7f6b4c004d00 2023-10-16T11:27:13.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.441+0000 7f6b660bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b601f2b00 con 0x7f6b601f6300 2023-10-16T11:27:13.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.441+0000 7f6b5effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b600c7720 0x7f6b601f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:13.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.441+0000 7f6b5effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b600c7720 0x7f6b601f5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44182/0 (socket says 172.21.15.84:44182) 2023-10-16T11:27:13.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.441+0000 7f6b5effd640 1 -- 172.21.15.84:0/169671021 learned_addr learned my addr 172.21.15.84:0/169671021 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:13.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.441+0000 7f6b5effd640 1 -- 172.21.15.84:0/169671021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b601f6300 msgr2=0x7f6b600cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:13.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.441+0000 7f6b5effd640 1 --2- 172.21.15.84:0/169671021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b601f6300 0x7f6b600cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.441+0000 7f6b5effd640 1 -- 172.21.15.84:0/169671021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b4c004d00 msgr2=0x7f6b601f3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:13.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.441+0000 7f6b5effd640 1 --2- 172.21.15.84:0/169671021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b4c004d00 0x7f6b601f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.441+0000 7f6b5effd640 1 -- 172.21.15.84:0/169671021 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b601f2de0 con 0x7f6b600c7720 2023-10-16T11:27:13.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.441+0000 7f6b5effd640 1 --2- 172.21.15.84:0/169671021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b600c7720 0x7f6b601f5dc0 secure :-1 s=READY pgs=3347 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800e7e0 tx=0x7f6b4800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:13.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.442+0000 7f6b57fff640 1 -- 172.21.15.84:0/169671021 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4800c650 con 0x7f6b600c7720 2023-10-16T11:27:13.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.442+0000 7f6b57fff640 1 -- 172.21.15.84:0/169671021 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4800c7b0 con 0x7f6b600c7720 2023-10-16T11:27:13.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.442+0000 7f6b57fff640 1 -- 172.21.15.84:0/169671021 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4800cb20 con 0x7f6b600c7720 2023-10-16T11:27:13.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.442+0000 7f6b660bb640 1 -- 172.21.15.84:0/169671021 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b601f27d0 con 0x7f6b600c7720 2023-10-16T11:27:13.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.442+0000 7f6b660bb640 1 -- 172.21.15.84:0/169671021 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b60154930 con 0x7f6b600c7720 2023-10-16T11:27:13.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.443+0000 7f6b660bb640 1 -- 172.21.15.84:0/169671021 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b601f2f70 con 0x7f6b600c7720 2023-10-16T11:27:13.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.444+0000 7f6b57fff640 1 -- 172.21.15.84:0/169671021 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6b4800cc80 con 0x7f6b600c7720 2023-10-16T11:27:13.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.445+0000 7f6b57fff640 1 --2- 172.21.15.84:0/169671021 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f6b4006afc0 0x7f6b4006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:13.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.445+0000 7f6b57fff640 1 -- 172.21.15.84:0/169671021 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 5831+0+0 (secure 0 0 0) 0x7f6b48089690 con 0x7f6b600c7720 2023-10-16T11:27:13.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.445+0000 7f6b5f7fe640 1 --2- 172.21.15.84:0/169671021 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f6b4006afc0 0x7f6b4006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:13.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.446+0000 7f6b5f7fe640 1 --2- 172.21.15.84:0/169671021 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f6b4006afc0 0x7f6b4006d480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6b50001fa0 tx=0x7f6b500018c0 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:13.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.449+0000 7f6b57fff640 1 -- 172.21.15.84:0/169671021 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6b48054490 con 0x7f6b600c7720 2023-10-16T11:27:13.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.628+0000 7f6b660bb640 1 -- 172.21.15.84:0/169671021 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f6b6004e160 con 0x7f6b600c7720 2023-10-16T11:27:13.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.633+0000 7f6b57fff640 1 -- 172.21.15.84:0/169671021 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v93) v1 ==== 115+0+0 (secure 0 0 0) 0x7f6b48057ab0 con 0x7f6b600c7720 2023-10-16T11:27:13.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.635+0000 7f6b660bb640 1 -- 172.21.15.84:0/169671021 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f6b4006afc0 msgr2=0x7f6b4006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:13.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.635+0000 7f6b660bb640 1 --2- 172.21.15.84:0/169671021 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f6b4006afc0 0x7f6b4006d480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6b50001fa0 tx=0x7f6b500018c0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.635+0000 7f6b660bb640 1 -- 172.21.15.84:0/169671021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b600c7720 msgr2=0x7f6b601f5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:13.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.635+0000 7f6b660bb640 1 --2- 172.21.15.84:0/169671021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b600c7720 0x7f6b601f5dc0 secure :-1 s=READY pgs=3347 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800e7e0 tx=0x7f6b4800eca0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.636+0000 7f6b660bb640 1 -- 172.21.15.84:0/169671021 shutdown_connections 2023-10-16T11:27:13.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.636+0000 7f6b660bb640 1 --2- 172.21.15.84:0/169671021 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f6b4006afc0 0x7f6b4006d480 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.636+0000 7f6b660bb640 1 --2- 172.21.15.84:0/169671021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b601f6300 0x7f6b600cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.636+0000 7f6b660bb640 1 --2- 172.21.15.84:0/169671021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b600c7720 0x7f6b601f5dc0 unknown :-1 s=CLOSED pgs=3347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.636+0000 7f6b660bb640 1 --2- 172.21.15.84:0/169671021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b4c004d00 0x7f6b601f3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.636+0000 7f6b660bb640 1 -- 172.21.15.84:0/169671021 >> 172.21.15.84:0/169671021 conn(0x7f6b600cb410 msgr2=0x7f6b601576f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:13.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.636+0000 7f6b660bb640 1 -- 172.21.15.84:0/169671021 shutdown_connections 2023-10-16T11:27:13.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.636+0000 7f6b660bb640 1 -- 172.21.15.84:0/169671021 wait complete. 2023-10-16T11:27:13.656 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-10-16T11:27:13.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.867+0000 7fb8d122d640 1 Processor -- start 2023-10-16T11:27:13.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.868+0000 7fb8d122d640 1 -- start start 2023-10-16T11:27:13.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.869+0000 7fb8d122d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb8cc0ce7e0 0x7fb8cc0719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:13.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.869+0000 7fb8d122d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb8cc15ce90 0x7fb8cc06adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:13.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.869+0000 7fb8d122d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8cc15d450 con 0x7fb8cc06b2f0 2023-10-16T11:27:13.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.869+0000 7fb8d122d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8cc06b760 con 0x7fb8cc0ce7e0 2023-10-16T11:27:13.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.869+0000 7fb8d122d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8cc15d730 con 0x7fb8cc15ce90 2023-10-16T11:27:13.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.869+0000 7fb8cad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb8cc0ce7e0 0x7fb8cc0719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:13.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.869+0000 7fb8cad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb8cc0ce7e0 0x7fb8cc0719f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44188/0 (socket says 172.21.15.84:44188) 2023-10-16T11:27:13.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.869+0000 7fb8cad9d640 1 -- 172.21.15.84:0/2010718877 learned_addr learned my addr 172.21.15.84:0/2010718877 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:13.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.870+0000 7fb8c3fff640 1 --2- 172.21.15.84:0/2010718877 >> v2:172.21.15.157:3300/0 conn(0x7fb8cc15ce90 0x7fb8cc06adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:13.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.870+0000 7fb8cad9d640 1 -- 172.21.15.84:0/2010718877 >> v2:172.21.15.157:3300/0 conn(0x7fb8cc15ce90 msgr2=0x7fb8cc06adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:13.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.870+0000 7fb8cad9d640 1 --2- 172.21.15.84:0/2010718877 >> v2:172.21.15.157:3300/0 conn(0x7fb8cc15ce90 0x7fb8cc06adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.870+0000 7fb8cad9d640 1 -- 172.21.15.84:0/2010718877 >> v1:172.21.15.84:6789/0 conn(0x7fb8cc06b2f0 legacy=0x7fb8cc0ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:13.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.870+0000 7fb8cad9d640 1 -- 172.21.15.84:0/2010718877 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8cc15da10 con 0x7fb8cc0ce7e0 2023-10-16T11:27:13.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.871+0000 7fb8cad9d640 1 --2- 172.21.15.84:0/2010718877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8cc0ce7e0 0x7fb8cc0719f0 secure :-1 s=READY pgs=3349 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0712f0 tx=0x7fb8bc02cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30aef3be994da5cd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:13.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.871+0000 7fb8ca59c640 1 -- 172.21.15.84:0/2010718877 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc02dba0 con 0x7fb8cc0ce7e0 2023-10-16T11:27:13.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.871+0000 7fb8ca59c640 1 -- 172.21.15.84:0/2010718877 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8bc02dd00 con 0x7fb8cc0ce7e0 2023-10-16T11:27:13.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.871+0000 7fb8ca59c640 1 -- 172.21.15.84:0/2010718877 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc035660 con 0x7fb8cc0ce7e0 2023-10-16T11:27:13.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.872+0000 7fb8d122d640 1 -- 172.21.15.84:0/2010718877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8cc0ce7e0 msgr2=0x7fb8cc0719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:13.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.872+0000 7fb8d122d640 1 --2- 172.21.15.84:0/2010718877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8cc0ce7e0 0x7fb8cc0719f0 secure :-1 s=READY pgs=3349 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0712f0 tx=0x7fb8bc02cdf0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.872+0000 7fb8d122d640 1 -- 172.21.15.84:0/2010718877 shutdown_connections 2023-10-16T11:27:13.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.872+0000 7fb8d122d640 1 --2- 172.21.15.84:0/2010718877 >> v2:172.21.15.157:3300/0 conn(0x7fb8cc15ce90 0x7fb8cc06adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.872+0000 7fb8d122d640 1 --2- 172.21.15.84:0/2010718877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8cc0ce7e0 0x7fb8cc0719f0 unknown :-1 s=CLOSED pgs=3349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.872+0000 7fb8d122d640 1 -- 172.21.15.84:0/2010718877 >> 172.21.15.84:0/2010718877 conn(0x7fb8cc0b0c80 msgr2=0x7fb8cc0b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:13.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.872+0000 7fb8d122d640 1 -- 172.21.15.84:0/2010718877 shutdown_connections 2023-10-16T11:27:13.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.872+0000 7fb8d122d640 1 -- 172.21.15.84:0/2010718877 wait complete. 2023-10-16T11:27:13.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.874+0000 7fb8d122d640 1 Processor -- start 2023-10-16T11:27:13.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.874+0000 7fb8d122d640 1 -- start start 2023-10-16T11:27:13.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.875+0000 7fb8d122d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8cc06b2f0 0x7fb8cc1575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:13.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.875+0000 7fb8d122d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8cc0ce7e0 0x7fb8cc0c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:13.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.875+0000 7fb8d122d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8cc15ce90 0x7fb8cc0af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:13.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.875+0000 7fb8d122d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc157b30 con 0x7fb8cc06b2f0 2023-10-16T11:27:13.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.875+0000 7fb8d122d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc1f3f00 con 0x7fb8cc15ce90 2023-10-16T11:27:13.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.875+0000 7fb8d122d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc1f4d60 con 0x7fb8cc0ce7e0 2023-10-16T11:27:13.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.875+0000 7fb8cad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8cc06b2f0 0x7fb8cc1575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:13.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.875+0000 7fb8cad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8cc06b2f0 0x7fb8cc1575f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44190/0 (socket says 172.21.15.84:44190) 2023-10-16T11:27:13.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.875+0000 7fb8cad9d640 1 -- 172.21.15.84:0/887429103 learned_addr learned my addr 172.21.15.84:0/887429103 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:13.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.875+0000 7fb8cad9d640 1 -- 172.21.15.84:0/887429103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8cc0ce7e0 msgr2=0x7fb8cc0c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:13.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.875+0000 7fb8cad9d640 1 --2- 172.21.15.84:0/887429103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8cc0ce7e0 0x7fb8cc0c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.875+0000 7fb8cad9d640 1 -- 172.21.15.84:0/887429103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8cc15ce90 msgr2=0x7fb8cc0af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:13.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.876+0000 7fb8cad9d640 1 --2- 172.21.15.84:0/887429103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8cc15ce90 0x7fb8cc0af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:13.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.876+0000 7fb8cad9d640 1 -- 172.21.15.84:0/887429103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8bc00a670 con 0x7fb8cc06b2f0 2023-10-16T11:27:13.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.876+0000 7fb8cad9d640 1 --2- 172.21.15.84:0/887429103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8cc06b2f0 0x7fb8cc1575f0 secure :-1 s=READY pgs=3350 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc02ce30 tx=0x7fb8bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:13.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.876+0000 7fb8c37fe640 1 -- 172.21.15.84:0/887429103 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc03d800 con 0x7fb8cc06b2f0 2023-10-16T11:27:13.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.876+0000 7fb8c37fe640 1 -- 172.21.15.84:0/887429103 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8bc03d960 con 0x7fb8cc06b2f0 2023-10-16T11:27:13.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.876+0000 7fb8c37fe640 1 -- 172.21.15.84:0/887429103 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc03dd10 con 0x7fb8cc06b2f0 2023-10-16T11:27:13.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.876+0000 7fb8d122d640 1 -- 172.21.15.84:0/887429103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8cc1f4150 con 0x7fb8cc06b2f0 2023-10-16T11:27:13.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.876+0000 7fb8d122d640 1 -- 172.21.15.84:0/887429103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8cc1f4710 con 0x7fb8cc06b2f0 2023-10-16T11:27:13.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.877+0000 7fb8d122d640 1 -- 172.21.15.84:0/887429103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8cc04e160 con 0x7fb8cc06b2f0 2023-10-16T11:27:13.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.878+0000 7fb8c37fe640 1 -- 172.21.15.84:0/887429103 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb8bc03f3e0 con 0x7fb8cc06b2f0 2023-10-16T11:27:13.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.879+0000 7fb8c37fe640 1 --2- 172.21.15.84:0/887429103 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fb8ac06af90 0x7fb8ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:13.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.879+0000 7fb8c37fe640 1 -- 172.21.15.84:0/887429103 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 5831+0+0 (secure 0 0 0) 0x7fb8bc0abb30 con 0x7fb8cc06b2f0 2023-10-16T11:27:13.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.879+0000 7fb8c3fff640 1 --2- 172.21.15.84:0/887429103 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fb8ac06af90 0x7fb8ac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:13.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.880+0000 7fb8c3fff640 1 --2- 172.21.15.84:0/887429103 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fb8ac06af90 0x7fb8ac06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4006fd0 tx=0x7fb8b4008040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:13.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:13.883+0000 7fb8c37fe640 1 -- 172.21.15.84:0/887429103 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb8bc0769b0 con 0x7fb8cc06b2f0 2023-10-16T11:27:14.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.146+0000 7fb8d122d640 1 -- 172.21.15.84:0/887429103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7fb8cc06e310 con 0x7fb8cc06b2f0 2023-10-16T11:27:14.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.332+0000 7fb8c37fe640 1 -- 172.21.15.84:0/887429103 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3272) v1 ==== 155+0+0 (secure 0 0 0) 0x7fb8bc0450e0 con 0x7fb8cc06b2f0 2023-10-16T11:27:14.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.336+0000 7fb8d122d640 1 -- 172.21.15.84:0/887429103 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fb8ac06af90 msgr2=0x7fb8ac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:14.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.336+0000 7fb8d122d640 1 --2- 172.21.15.84:0/887429103 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fb8ac06af90 0x7fb8ac06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4006fd0 tx=0x7fb8b4008040 comp rx=0 tx=0).stop 2023-10-16T11:27:14.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.336+0000 7fb8d122d640 1 -- 172.21.15.84:0/887429103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8cc06b2f0 msgr2=0x7fb8cc1575f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:14.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.336+0000 7fb8d122d640 1 --2- 172.21.15.84:0/887429103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8cc06b2f0 0x7fb8cc1575f0 secure :-1 s=READY pgs=3350 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc02ce30 tx=0x7fb8bc0046c0 comp rx=0 tx=0).stop 2023-10-16T11:27:14.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.336+0000 7fb8d122d640 1 -- 172.21.15.84:0/887429103 shutdown_connections 2023-10-16T11:27:14.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.336+0000 7fb8d122d640 1 --2- 172.21.15.84:0/887429103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8cc15ce90 0x7fb8cc0af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:14.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.336+0000 7fb8d122d640 1 --2- 172.21.15.84:0/887429103 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fb8ac06af90 0x7fb8ac06d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:14.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.336+0000 7fb8d122d640 1 --2- 172.21.15.84:0/887429103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8cc0ce7e0 0x7fb8cc0c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:14.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.336+0000 7fb8d122d640 1 --2- 172.21.15.84:0/887429103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8cc06b2f0 0x7fb8cc1575f0 unknown :-1 s=CLOSED pgs=3350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:14.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.336+0000 7fb8d122d640 1 -- 172.21.15.84:0/887429103 >> 172.21.15.84:0/887429103 conn(0x7fb8cc0b0c80 msgr2=0x7fb8cc0c2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:14.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.336+0000 7fb8d122d640 1 -- 172.21.15.84:0/887429103 shutdown_connections 2023-10-16T11:27:14.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.336+0000 7fb8d122d640 1 -- 172.21.15.84:0/887429103 wait complete. 2023-10-16T11:27:14.358 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2023-10-16T11:27:14.359 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_subvol_and_group_arguments (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) Test subvol and group arguments to snap-schedule cli. 2023-10-16T11:27:14.359 INFO:tasks.cephfs_test_runner:test_snap_schedule_subvol_and_group_arguments (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2023-10-16T11:27:14.360 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:27:14.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.574+0000 7f09a9cf7640 1 Processor -- start 2023-10-16T11:27:14.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.574+0000 7f09a9cf7640 1 -- start start 2023-10-16T11:27:14.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.576+0000 7f09a9cf7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f09a415aaa0 0x7f09a415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:14.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.576+0000 7f09a9cf7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a415ed60 con 0x7f09a40ce390 2023-10-16T11:27:14.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.576+0000 7f09a9cf7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a40707e0 con 0x7f09a415ae80 2023-10-16T11:27:14.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.576+0000 7f09a9cf7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a406b810 con 0x7f09a415aaa0 2023-10-16T11:27:14.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.576+0000 7f09a37fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f09a415aaa0 0x7f09a415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:14.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.576+0000 7f09a37fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f09a415aaa0 0x7f09a415e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44206/0 (socket says 172.21.15.84:44206) 2023-10-16T11:27:14.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.576+0000 7f09a37fe640 1 -- 172.21.15.84:0/2333898308 learned_addr learned my addr 172.21.15.84:0/2333898308 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:14.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.577+0000 7f09a37fe640 1 -- 172.21.15.84:0/2333898308 >> v1:172.21.15.157:6789/0 conn(0x7f09a415ae80 legacy=0x7f09a40cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:14.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.577+0000 7f09a37fe640 1 -- 172.21.15.84:0/2333898308 >> v1:172.21.15.84:6789/0 conn(0x7f09a40ce390 legacy=0x7f09a4070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:14.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.577+0000 7f09a37fe640 1 -- 172.21.15.84:0/2333898308 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a415ef90 con 0x7f09a415aaa0 2023-10-16T11:27:14.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.579+0000 7f09a37fe640 1 --2- 172.21.15.84:0/2333898308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a415aaa0 0x7f09a415e820 secure :-1 s=READY pgs=3352 cs=0 l=1 rev1=1 crypto rx=0x7f0994009e50 tx=0x7f099402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d4be079abf145c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:14.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.579+0000 7f09a27fc640 1 -- 172.21.15.84:0/2333898308 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099402cbb0 con 0x7f09a415aaa0 2023-10-16T11:27:14.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.579+0000 7f09a27fc640 1 -- 172.21.15.84:0/2333898308 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099402cd10 con 0x7f09a415aaa0 2023-10-16T11:27:14.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.579+0000 7f09a27fc640 1 -- 172.21.15.84:0/2333898308 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994034610 con 0x7f09a415aaa0 2023-10-16T11:27:14.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.580+0000 7f09a9cf7640 1 -- 172.21.15.84:0/2333898308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a415aaa0 msgr2=0x7f09a415e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:14.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.580+0000 7f09a9cf7640 1 --2- 172.21.15.84:0/2333898308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a415aaa0 0x7f09a415e820 secure :-1 s=READY pgs=3352 cs=0 l=1 rev1=1 crypto rx=0x7f0994009e50 tx=0x7f099402c010 comp rx=0 tx=0).stop 2023-10-16T11:27:14.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.580+0000 7f09a9cf7640 1 -- 172.21.15.84:0/2333898308 shutdown_connections 2023-10-16T11:27:14.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.580+0000 7f09a9cf7640 1 --2- 172.21.15.84:0/2333898308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a415aaa0 0x7f09a415e820 unknown :-1 s=CLOSED pgs=3352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:14.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.580+0000 7f09a9cf7640 1 -- 172.21.15.84:0/2333898308 >> 172.21.15.84:0/2333898308 conn(0x7f09a40cb410 msgr2=0x7f09a40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:14.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.580+0000 7f09a9cf7640 1 -- 172.21.15.84:0/2333898308 shutdown_connections 2023-10-16T11:27:14.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.580+0000 7f09a9cf7640 1 -- 172.21.15.84:0/2333898308 wait complete. 2023-10-16T11:27:14.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.581+0000 7f09a9cf7640 1 Processor -- start 2023-10-16T11:27:14.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.582+0000 7f09a9cf7640 1 -- start start 2023-10-16T11:27:14.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.582+0000 7f09a9cf7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a40ce390 0x7f09a4155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:14.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.582+0000 7f09a9cf7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09a415aaa0 0x7f09a415dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:14.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.582+0000 7f09a9cf7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a415ae80 0x7f09a415bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:14.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.582+0000 7f09a9cf7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41564c0 con 0x7f09a415ae80 2023-10-16T11:27:14.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.582+0000 7f09a9cf7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40c3bb0 con 0x7f09a40ce390 2023-10-16T11:27:14.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.582+0000 7f09a9cf7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41f0c20 con 0x7f09a415aaa0 2023-10-16T11:27:14.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.583+0000 7f09a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a415ae80 0x7f09a415bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:14.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.583+0000 7f09a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a415ae80 0x7f09a415bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44210/0 (socket says 172.21.15.84:44210) 2023-10-16T11:27:14.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.583+0000 7f09a3fff640 1 -- 172.21.15.84:0/3645451504 learned_addr learned my addr 172.21.15.84:0/3645451504 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:14.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.583+0000 7f09a3fff640 1 -- 172.21.15.84:0/3645451504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09a415aaa0 msgr2=0x7f09a415dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:14.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.583+0000 7f09a3fff640 1 --2- 172.21.15.84:0/3645451504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09a415aaa0 0x7f09a415dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:14.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.583+0000 7f09a3fff640 1 -- 172.21.15.84:0/3645451504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a40ce390 msgr2=0x7f09a4155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:14.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.583+0000 7f09a37fe640 1 --2- 172.21.15.84:0/3645451504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a40ce390 0x7f09a4155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:14.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.583+0000 7f09a3fff640 1 --2- 172.21.15.84:0/3645451504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a40ce390 0x7f09a4155f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:14.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.583+0000 7f09a3fff640 1 -- 172.21.15.84:0/3645451504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0994009b30 con 0x7f09a415ae80 2023-10-16T11:27:14.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.584+0000 7f09a3fff640 1 --2- 172.21.15.84:0/3645451504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a415ae80 0x7f09a415bd30 secure :-1 s=READY pgs=3353 cs=0 l=1 rev1=1 crypto rx=0x7f099c001eb0 tx=0x7f099c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:14.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.584+0000 7f098bfff640 1 -- 172.21.15.84:0/3645451504 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c0052d0 con 0x7f09a415ae80 2023-10-16T11:27:14.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.584+0000 7f098bfff640 1 -- 172.21.15.84:0/3645451504 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099c005430 con 0x7f09a415ae80 2023-10-16T11:27:14.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.584+0000 7f098bfff640 1 -- 172.21.15.84:0/3645451504 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c003810 con 0x7f09a415ae80 2023-10-16T11:27:14.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.584+0000 7f09a9cf7640 1 -- 172.21.15.84:0/3645451504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a41f0f00 con 0x7f09a415ae80 2023-10-16T11:27:14.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.584+0000 7f09a9cf7640 1 -- 172.21.15.84:0/3645451504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a41f14c0 con 0x7f09a415ae80 2023-10-16T11:27:14.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.585+0000 7f09a9cf7640 1 -- 172.21.15.84:0/3645451504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a404e160 con 0x7f09a415ae80 2023-10-16T11:27:14.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.586+0000 7f098bfff640 1 -- 172.21.15.84:0/3645451504 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f099c003970 con 0x7f09a415ae80 2023-10-16T11:27:14.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.587+0000 7f098bfff640 1 --2- 172.21.15.84:0/3645451504 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f099006af90 0x7f099006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:14.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.587+0000 7f098bfff640 1 -- 172.21.15.84:0/3645451504 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 5831+0+0 (secure 0 0 0) 0x7f099c088de0 con 0x7f09a415ae80 2023-10-16T11:27:14.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.587+0000 7f09a37fe640 1 --2- 172.21.15.84:0/3645451504 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f099006af90 0x7f099006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:14.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.588+0000 7f09a37fe640 1 --2- 172.21.15.84:0/3645451504 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f099006af90 0x7f099006d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0994035000 tx=0x7f0994030000 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:14.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.591+0000 7f098bfff640 1 -- 172.21.15.84:0/3645451504 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f099c053d00 con 0x7f09a415ae80 2023-10-16T11:27:14.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.782+0000 7f09a9cf7640 1 -- 172.21.15.84:0/3645451504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09a40c3cf0 con 0x7f09a415ae80 2023-10-16T11:27:14.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.784+0000 7f098bfff640 1 -- 172.21.15.84:0/3645451504 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v262) v1 ==== 74+0+14265 (secure 0 0 0) 0x7f099c017090 con 0x7f09a415ae80 2023-10-16T11:27:14.786 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:14.786 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":262,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:27:12.302976+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/3899307325":"2023-10-17T11:27:09.213507+0000","172.21.15.84:0/1625282261":"2023-10-17T11:27:05.494979+0000","172.21.15.84:6833/2988856492":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/765814920":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/2899443560":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/3002614854":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/2731742209":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/907959269":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/1863310536":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/847549599":"2023-10-17T11:26:57.151307+0000","172.21.15.157:6833/3899307325":"2023-10-17T11:27:09.213507+0000","172.21.15.84:0/972005011":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/1934386371":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/3540589884":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/2370234804":"2023-10-17T11:26:57.151307+0000","172.21.15.84:6833/2330196255":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/1650008535":"2023-10-17T11:26:57.151307+0000","172.21.15.84:6832/2330196255":"2023-10-17T11:26:57.151307+0000","172.21.15.84:6832/2988856492":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/3247488662":"2023-10-17T11:27:05.494979+0000","172.21.15.84:0/2440426242":"2023-10-17T11:26:57.151307+0000","172.21.15.84:0/570042115":"2023-10-16T12:26:40.168744+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:27:14.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.786+0000 7f09a9cf7640 1 -- 172.21.15.84:0/3645451504 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f099006af90 msgr2=0x7f099006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:14.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.786+0000 7f09a9cf7640 1 --2- 172.21.15.84:0/3645451504 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f099006af90 0x7f099006d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0994035000 tx=0x7f0994030000 comp rx=0 tx=0).stop 2023-10-16T11:27:14.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.786+0000 7f09a9cf7640 1 -- 172.21.15.84:0/3645451504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a415ae80 msgr2=0x7f09a415bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:14.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.786+0000 7f09a9cf7640 1 --2- 172.21.15.84:0/3645451504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a415ae80 0x7f09a415bd30 secure :-1 s=READY pgs=3353 cs=0 l=1 rev1=1 crypto rx=0x7f099c001eb0 tx=0x7f099c00c890 comp rx=0 tx=0).stop 2023-10-16T11:27:14.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.786+0000 7f09a9cf7640 1 -- 172.21.15.84:0/3645451504 shutdown_connections 2023-10-16T11:27:14.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.786+0000 7f09a9cf7640 1 --2- 172.21.15.84:0/3645451504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a415ae80 0x7f09a415bd30 unknown :-1 s=CLOSED pgs=3353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:14.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.786+0000 7f09a9cf7640 1 --2- 172.21.15.84:0/3645451504 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f099006af90 0x7f099006d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:14.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.786+0000 7f09a9cf7640 1 --2- 172.21.15.84:0/3645451504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09a415aaa0 0x7f09a415dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:14.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.786+0000 7f09a9cf7640 1 --2- 172.21.15.84:0/3645451504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a40ce390 0x7f09a4155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:14.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.786+0000 7f09a9cf7640 1 -- 172.21.15.84:0/3645451504 >> 172.21.15.84:0/3645451504 conn(0x7f09a40cb410 msgr2=0x7f09a40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:14.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.786+0000 7f09a9cf7640 1 -- 172.21.15.84:0/3645451504 shutdown_connections 2023-10-16T11:27:14.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:14.786+0000 7f09a9cf7640 1 -- 172.21.15.84:0/3645451504 wait complete. 2023-10-16T11:27:14.806 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:27:14.806 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:27:15.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.014+0000 7f88c359e640 1 Processor -- start 2023-10-16T11:27:15.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.014+0000 7f88c359e640 1 -- start start 2023-10-16T11:27:15.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.016+0000 7f88c359e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f88c40b3ec0 0x7f88c40cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:15.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.016+0000 7f88c359e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f88c40b0ea0 0x7f88c40cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:15.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.016+0000 7f88c359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88c40b43c0 0x7f88c415a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:15.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.016+0000 7f88c359e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c406ca00 con 0x7f88c40b3ec0 2023-10-16T11:27:15.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.016+0000 7f88c359e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c406be30 con 0x7f88c40b0ea0 2023-10-16T11:27:15.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.016+0000 7f88c359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c40ada10 con 0x7f88c40b43c0 2023-10-16T11:27:15.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.016+0000 7f88c259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f88c40b3ec0 0x7f88c40cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:15.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.016+0000 7f88c259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f88c40b3ec0 0x7f88c40cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44226/0 (socket says 172.21.15.84:44226) 2023-10-16T11:27:15.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.016+0000 7f88c259c640 1 -- 172.21.15.84:0/1755975589 learned_addr learned my addr 172.21.15.84:0/1755975589 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:15.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.016+0000 7f88c259c640 1 -- 172.21.15.84:0/1755975589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88c40b43c0 msgr2=0x7f88c415a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:15.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.016+0000 7f88c259c640 1 --2- 172.21.15.84:0/1755975589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88c40b43c0 0x7f88c415a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:15.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.016+0000 7f88c259c640 1 -- 172.21.15.84:0/1755975589 >> v2:172.21.15.157:3300/0 conn(0x7f88c40b0ea0 msgr2=0x7f88c40cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:15.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.016+0000 7f88c259c640 1 --2- 172.21.15.84:0/1755975589 >> v2:172.21.15.157:3300/0 conn(0x7f88c40b0ea0 0x7f88c40cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:15.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.016+0000 7f88c259c640 1 -- 172.21.15.84:0/1755975589 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c406c060 con 0x7f88c40b3ec0 2023-10-16T11:27:15.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.017+0000 7f88c259c640 1 --2- 172.21.15.84:0/1755975589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88c40b3ec0 0x7f88c40cd260 secure :-1 s=READY pgs=3354 cs=0 l=1 rev1=1 crypto rx=0x7f88c4067810 tx=0x7f88b402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe6fcdbac2c7da1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:15.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.018+0000 7f88c1d9b640 1 -- 172.21.15.84:0/1755975589 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b40317e0 con 0x7f88c40b3ec0 2023-10-16T11:27:15.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.018+0000 7f88c1d9b640 1 -- 172.21.15.84:0/1755975589 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b4031940 con 0x7f88c40b3ec0 2023-10-16T11:27:15.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.018+0000 7f88c1d9b640 1 -- 172.21.15.84:0/1755975589 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4031cd0 con 0x7f88c40b3ec0 2023-10-16T11:27:15.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.019+0000 7f88c359e640 1 -- 172.21.15.84:0/1755975589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88c40b3ec0 msgr2=0x7f88c40cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:15.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.019+0000 7f88c359e640 1 --2- 172.21.15.84:0/1755975589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88c40b3ec0 0x7f88c40cd260 secure :-1 s=READY pgs=3354 cs=0 l=1 rev1=1 crypto rx=0x7f88c4067810 tx=0x7f88b402b880 comp rx=0 tx=0).stop 2023-10-16T11:27:15.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.019+0000 7f88c359e640 1 -- 172.21.15.84:0/1755975589 shutdown_connections 2023-10-16T11:27:15.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.019+0000 7f88c359e640 1 --2- 172.21.15.84:0/1755975589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88c40b43c0 0x7f88c415a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:15.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.019+0000 7f88c359e640 1 --2- 172.21.15.84:0/1755975589 >> v2:172.21.15.157:3300/0 conn(0x7f88c40b0ea0 0x7f88c40cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:15.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.019+0000 7f88c359e640 1 --2- 172.21.15.84:0/1755975589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88c40b3ec0 0x7f88c40cd260 unknown :-1 s=CLOSED pgs=3354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:15.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.019+0000 7f88c359e640 1 -- 172.21.15.84:0/1755975589 >> 172.21.15.84:0/1755975589 conn(0x7f88c40b0170 msgr2=0x7f88c40b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:15.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.019+0000 7f88c359e640 1 -- 172.21.15.84:0/1755975589 shutdown_connections 2023-10-16T11:27:15.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.019+0000 7f88c359e640 1 -- 172.21.15.84:0/1755975589 wait complete. 2023-10-16T11:27:15.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.020+0000 7f88c359e640 1 Processor -- start 2023-10-16T11:27:15.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.021+0000 7f88c359e640 1 -- start start 2023-10-16T11:27:15.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.021+0000 7f88c359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88c40b0ea0 0x7f88c40c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:15.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.022+0000 7f88c359e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88c40b3ec0 0x7f88c414eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:15.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.022+0000 7f88c359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88c40b43c0 0x7f88c40c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:15.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.022+0000 7f88c359e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c40c55b0 con 0x7f88c40b0ea0 2023-10-16T11:27:15.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.022+0000 7f88c359e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c41593f0 con 0x7f88c40b43c0 2023-10-16T11:27:15.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.022+0000 7f88c359e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c4158670 con 0x7f88c40b3ec0 2023-10-16T11:27:15.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.022+0000 7f88c259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88c40b0ea0 0x7f88c40c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:15.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.022+0000 7f88c259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88c40b0ea0 0x7f88c40c25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44232/0 (socket says 172.21.15.84:44232) 2023-10-16T11:27:15.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.022+0000 7f88c259c640 1 -- 172.21.15.84:0/2393335594 learned_addr learned my addr 172.21.15.84:0/2393335594 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:15.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.022+0000 7f88c259c640 1 -- 172.21.15.84:0/2393335594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88c40b3ec0 msgr2=0x7f88c414eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:15.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.022+0000 7f88c259c640 1 --2- 172.21.15.84:0/2393335594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88c40b3ec0 0x7f88c414eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:15.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.022+0000 7f88c259c640 1 -- 172.21.15.84:0/2393335594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88c40b43c0 msgr2=0x7f88c40c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:15.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.022+0000 7f88c259c640 1 --2- 172.21.15.84:0/2393335594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88c40b43c0 0x7f88c40c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:15.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.022+0000 7f88c259c640 1 -- 172.21.15.84:0/2393335594 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b40094d0 con 0x7f88c40b0ea0 2023-10-16T11:27:15.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.023+0000 7f88c259c640 1 --2- 172.21.15.84:0/2393335594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88c40b0ea0 0x7f88c40c25b0 secure :-1 s=READY pgs=3355 cs=0 l=1 rev1=1 crypto rx=0x7f88b402b8c0 tx=0x7f88b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:15.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.023+0000 7f88baffd640 1 -- 172.21.15.84:0/2393335594 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b40329c0 con 0x7f88c40b0ea0 2023-10-16T11:27:15.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.023+0000 7f88baffd640 1 -- 172.21.15.84:0/2393335594 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b4032b40 con 0x7f88c40b0ea0 2023-10-16T11:27:15.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.023+0000 7f88c359e640 1 -- 172.21.15.84:0/2393335594 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c4158910 con 0x7f88c40b0ea0 2023-10-16T11:27:15.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.023+0000 7f88baffd640 1 -- 172.21.15.84:0/2393335594 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b40453f0 con 0x7f88c40b0ea0 2023-10-16T11:27:15.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.023+0000 7f88c359e640 1 -- 172.21.15.84:0/2393335594 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c4158ed0 con 0x7f88c40b0ea0 2023-10-16T11:27:15.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.024+0000 7f88c359e640 1 -- 172.21.15.84:0/2393335594 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c41590c0 con 0x7f88c40b0ea0 2023-10-16T11:27:15.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.025+0000 7f88baffd640 1 -- 172.21.15.84:0/2393335594 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f88b4045550 con 0x7f88c40b0ea0 2023-10-16T11:27:15.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.027+0000 7f88baffd640 1 --2- 172.21.15.84:0/2393335594 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f88a406af90 0x7f88a406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:15.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.027+0000 7f88baffd640 1 -- 172.21.15.84:0/2393335594 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 5831+0+0 (secure 0 0 0) 0x7f88b40abdd0 con 0x7f88c40b0ea0 2023-10-16T11:27:15.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.027+0000 7f88bbfff640 1 --2- 172.21.15.84:0/2393335594 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f88a406af90 0x7f88a406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:15.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.027+0000 7f88bbfff640 1 --2- 172.21.15.84:0/2393335594 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f88a406af90 0x7f88a406d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f88c40c2af0 tx=0x7f88ac006bf0 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:15.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.031+0000 7f88baffd640 1 -- 172.21.15.84:0/2393335594 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f88b4076d00 con 0x7f88c40b0ea0 2023-10-16T11:27:15.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.222+0000 7f88c359e640 1 -- 172.21.15.84:0/2393335594 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88c404e9b0 con 0x7f88c40b0ea0 2023-10-16T11:27:15.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.223+0000 7f88baffd640 1 -- 172.21.15.84:0/2393335594 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v262) v1 ==== 93+0+10 (secure 0 0 0) 0x7f88b40ad0a0 con 0x7f88c40b0ea0 2023-10-16T11:27:15.225 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:27:15.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.225+0000 7f88c359e640 1 -- 172.21.15.84:0/2393335594 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f88a406af90 msgr2=0x7f88a406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:15.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.225+0000 7f88c359e640 1 --2- 172.21.15.84:0/2393335594 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f88a406af90 0x7f88a406d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f88c40c2af0 tx=0x7f88ac006bf0 comp rx=0 tx=0).stop 2023-10-16T11:27:15.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.226+0000 7f88c359e640 1 -- 172.21.15.84:0/2393335594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88c40b0ea0 msgr2=0x7f88c40c25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:15.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.226+0000 7f88c359e640 1 --2- 172.21.15.84:0/2393335594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88c40b0ea0 0x7f88c40c25b0 secure :-1 s=READY pgs=3355 cs=0 l=1 rev1=1 crypto rx=0x7f88b402b8c0 tx=0x7f88b40046c0 comp rx=0 tx=0).stop 2023-10-16T11:27:15.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.226+0000 7f88c359e640 1 -- 172.21.15.84:0/2393335594 shutdown_connections 2023-10-16T11:27:15.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.226+0000 7f88c359e640 1 --2- 172.21.15.84:0/2393335594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88c40b43c0 0x7f88c40c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:15.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.226+0000 7f88c359e640 1 --2- 172.21.15.84:0/2393335594 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f88a406af90 0x7f88a406d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:15.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.226+0000 7f88c359e640 1 --2- 172.21.15.84:0/2393335594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88c40b3ec0 0x7f88c414eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:15.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.226+0000 7f88c359e640 1 --2- 172.21.15.84:0/2393335594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88c40b0ea0 0x7f88c40c25b0 unknown :-1 s=CLOSED pgs=3355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:15.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.226+0000 7f88c359e640 1 -- 172.21.15.84:0/2393335594 >> 172.21.15.84:0/2393335594 conn(0x7f88c40b0170 msgr2=0x7f88c40c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:15.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.226+0000 7f88c359e640 1 -- 172.21.15.84:0/2393335594 shutdown_connections 2023-10-16T11:27:15.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.226+0000 7f88c359e640 1 -- 172.21.15.84:0/2393335594 wait complete. 2023-10-16T11:27:15.247 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments' 2023-10-16T11:27:15.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.465+0000 7fe5dc27c640 1 Processor -- start 2023-10-16T11:27:15.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.466+0000 7fe5dc27c640 1 -- start start 2023-10-16T11:27:15.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.467+0000 7fe5dc27c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe5d406b8c0 0x7fe5d40cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:15.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.467+0000 7fe5dc27c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe5d40b0b90 0x7fe5d415d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:15.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.467+0000 7fe5dc27c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d40cf940 con 0x7fe5d406bca0 2023-10-16T11:27:15.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.467+0000 7fe5dc27c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d40ce990 con 0x7fe5d406b8c0 2023-10-16T11:27:15.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.467+0000 7fe5dc27c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d40d0310 con 0x7fe5d40b0b90 2023-10-16T11:27:15.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.467+0000 7fe5da018640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe5d406b8c0 0x7fe5d40cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:15.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.467+0000 7fe5da018640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe5d406b8c0 0x7fe5d40cfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44242/0 (socket says 172.21.15.84:44242) 2023-10-16T11:27:15.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.467+0000 7fe5da018640 1 -- 172.21.15.84:0/3759475028 learned_addr learned my addr 172.21.15.84:0/3759475028 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:15.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.468+0000 7fe5da018640 1 -- 172.21.15.84:0/3759475028 >> v2:172.21.15.157:3300/0 conn(0x7fe5d40b0b90 msgr2=0x7fe5d415d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:15.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.468+0000 7fe5da018640 1 --2- 172.21.15.84:0/3759475028 >> v2:172.21.15.157:3300/0 conn(0x7fe5d40b0b90 0x7fe5d415d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:15.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.468+0000 7fe5da018640 1 -- 172.21.15.84:0/3759475028 >> v1:172.21.15.157:6789/0 conn(0x7fe5d406bca0 legacy=0x7fe5d40b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:15.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.468+0000 7fe5da018640 1 -- 172.21.15.84:0/3759475028 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d415e290 con 0x7fe5d406b8c0 2023-10-16T11:27:15.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.469+0000 7fe5da018640 1 --2- 172.21.15.84:0/3759475028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5d406b8c0 0x7fe5d40cfdd0 secure :-1 s=READY pgs=3356 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4066a90 tx=0x7fe5c802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26969e1e3aaba0f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:15.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.469+0000 7fe5d9016640 1 -- 172.21.15.84:0/3759475028 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c802cad0 con 0x7fe5d406b8c0 2023-10-16T11:27:15.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.469+0000 7fe5d9016640 1 -- 172.21.15.84:0/3759475028 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c802cc30 con 0x7fe5d406b8c0 2023-10-16T11:27:15.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.469+0000 7fe5d9016640 1 -- 172.21.15.84:0/3759475028 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80343f0 con 0x7fe5d406b8c0 2023-10-16T11:27:15.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.470+0000 7fe5dc27c640 1 -- 172.21.15.84:0/3759475028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5d406b8c0 msgr2=0x7fe5d40cfdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:15.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.470+0000 7fe5dc27c640 1 --2- 172.21.15.84:0/3759475028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5d406b8c0 0x7fe5d40cfdd0 secure :-1 s=READY pgs=3356 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4066a90 tx=0x7fe5c802bf30 comp rx=0 tx=0).stop 2023-10-16T11:27:15.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.470+0000 7fe5dc27c640 1 -- 172.21.15.84:0/3759475028 shutdown_connections 2023-10-16T11:27:15.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.470+0000 7fe5dc27c640 1 --2- 172.21.15.84:0/3759475028 >> v2:172.21.15.157:3300/0 conn(0x7fe5d40b0b90 0x7fe5d415d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:15.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.470+0000 7fe5dc27c640 1 --2- 172.21.15.84:0/3759475028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5d406b8c0 0x7fe5d40cfdd0 unknown :-1 s=CLOSED pgs=3356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:15.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.470+0000 7fe5dc27c640 1 -- 172.21.15.84:0/3759475028 >> 172.21.15.84:0/3759475028 conn(0x7fe5d4099ea0 msgr2=0x7fe5d40d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:15.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.470+0000 7fe5dc27c640 1 -- 172.21.15.84:0/3759475028 shutdown_connections 2023-10-16T11:27:15.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.470+0000 7fe5dc27c640 1 -- 172.21.15.84:0/3759475028 wait complete. 2023-10-16T11:27:15.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.472+0000 7fe5dc27c640 1 Processor -- start 2023-10-16T11:27:15.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.473+0000 7fe5dc27c640 1 -- start start 2023-10-16T11:27:15.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.474+0000 7fe5dc27c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5d406b8c0 0x7fe5d40c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:15.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.474+0000 7fe5dc27c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d406bca0 0x7fe5d40ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:15.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.474+0000 7fe5dc27c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5d40b0b90 0x7fe5d40c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:15.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.474+0000 7fe5dc27c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d40c5580 con 0x7fe5d406b8c0 2023-10-16T11:27:15.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.474+0000 7fe5dc27c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d40ce030 con 0x7fe5d406bca0 2023-10-16T11:27:15.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.474+0000 7fe5dc27c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d40c4ce0 con 0x7fe5d40b0b90 2023-10-16T11:27:15.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.474+0000 7fe5da018640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5d406b8c0 0x7fe5d40c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:15.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.474+0000 7fe5da018640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5d406b8c0 0x7fe5d40c4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44244/0 (socket says 172.21.15.84:44244) 2023-10-16T11:27:15.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.474+0000 7fe5da018640 1 -- 172.21.15.84:0/3851942010 learned_addr learned my addr 172.21.15.84:0/3851942010 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:15.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.475+0000 7fe5da018640 1 -- 172.21.15.84:0/3851942010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5d40b0b90 msgr2=0x7fe5d40c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:15.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.475+0000 7fe5da018640 1 --2- 172.21.15.84:0/3851942010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5d40b0b90 0x7fe5d40c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:15.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.475+0000 7fe5da018640 1 -- 172.21.15.84:0/3851942010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d406bca0 msgr2=0x7fe5d40ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:15.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.475+0000 7fe5da018640 1 --2- 172.21.15.84:0/3851942010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d406bca0 0x7fe5d40ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:15.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.475+0000 7fe5da018640 1 -- 172.21.15.84:0/3851942010 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c8009b30 con 0x7fe5d406b8c0 2023-10-16T11:27:15.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.475+0000 7fe5da018640 1 --2- 172.21.15.84:0/3851942010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5d406b8c0 0x7fe5d40c4650 secure :-1 s=READY pgs=3357 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80093e0 tx=0x7fe5c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:15.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.475+0000 7fe5d2ffd640 1 -- 172.21.15.84:0/3851942010 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c803e450 con 0x7fe5d406b8c0 2023-10-16T11:27:15.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.475+0000 7fe5d2ffd640 1 -- 172.21.15.84:0/3851942010 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c803e5b0 con 0x7fe5d406b8c0 2023-10-16T11:27:15.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.475+0000 7fe5d2ffd640 1 -- 172.21.15.84:0/3851942010 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c803e960 con 0x7fe5d406b8c0 2023-10-16T11:27:15.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.475+0000 7fe5dc27c640 1 -- 172.21.15.84:0/3851942010 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d40c4fc0 con 0x7fe5d406b8c0 2023-10-16T11:27:15.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.476+0000 7fe5dc27c640 1 -- 172.21.15.84:0/3851942010 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d41f2b60 con 0x7fe5d406b8c0 2023-10-16T11:27:15.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.477+0000 7fe5dc27c640 1 -- 172.21.15.84:0/3851942010 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d41f29c0 con 0x7fe5d406b8c0 2023-10-16T11:27:15.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.478+0000 7fe5d2ffd640 1 -- 172.21.15.84:0/3851942010 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe5c803eac0 con 0x7fe5d406b8c0 2023-10-16T11:27:15.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.479+0000 7fe5d2ffd640 1 --2- 172.21.15.84:0/3851942010 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fe5b806af90 0x7fe5b806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:15.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.479+0000 7fe5d2ffd640 1 -- 172.21.15.84:0/3851942010 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 5831+0+0 (secure 0 0 0) 0x7fe5c80ac8d0 con 0x7fe5d406b8c0 2023-10-16T11:27:15.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.479+0000 7fe5d9817640 1 --2- 172.21.15.84:0/3851942010 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fe5b806af90 0x7fe5b806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:15.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.480+0000 7fe5d9817640 1 --2- 172.21.15.84:0/3851942010 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fe5b806af90 0x7fe5b806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe5d406c3b0 tx=0x7fe5c0007990 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:15.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.484+0000 7fe5d2ffd640 1 -- 172.21.15.84:0/3851942010 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe5d41f29c0 con 0x7fe5d406b8c0 2023-10-16T11:27:15.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:15.745+0000 7fe5dc27c640 1 -- 172.21.15.84:0/3851942010 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments"]} v 0) v1 -- 0x7fe5d41f29c0 con 0x7fe5d406b8c0 2023-10-16T11:27:16.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.338+0000 7fe5d2ffd640 1 -- 172.21.15.84:0/3851942010 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments"]}]=0 v3274) v1 ==== 201+0+0 (secure 0 0 0) 0x7fe5d41f29c0 con 0x7fe5d406b8c0 2023-10-16T11:27:16.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.341+0000 7fe5dc27c640 1 -- 172.21.15.84:0/3851942010 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fe5b806af90 msgr2=0x7fe5b806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:16.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.341+0000 7fe5dc27c640 1 --2- 172.21.15.84:0/3851942010 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fe5b806af90 0x7fe5b806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe5d406c3b0 tx=0x7fe5c0007990 comp rx=0 tx=0).stop 2023-10-16T11:27:16.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.341+0000 7fe5dc27c640 1 -- 172.21.15.84:0/3851942010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5d406b8c0 msgr2=0x7fe5d40c4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:16.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.341+0000 7fe5dc27c640 1 --2- 172.21.15.84:0/3851942010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5d406b8c0 0x7fe5d40c4650 secure :-1 s=READY pgs=3357 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80093e0 tx=0x7fe5c8004a40 comp rx=0 tx=0).stop 2023-10-16T11:27:16.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.341+0000 7fe5dc27c640 1 -- 172.21.15.84:0/3851942010 shutdown_connections 2023-10-16T11:27:16.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.341+0000 7fe5dc27c640 1 --2- 172.21.15.84:0/3851942010 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fe5b806af90 0x7fe5b806d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:16.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.341+0000 7fe5dc27c640 1 --2- 172.21.15.84:0/3851942010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5d40b0b90 0x7fe5d40c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:16.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.341+0000 7fe5dc27c640 1 --2- 172.21.15.84:0/3851942010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d406bca0 0x7fe5d40ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:16.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.341+0000 7fe5dc27c640 1 --2- 172.21.15.84:0/3851942010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5d406b8c0 0x7fe5d40c4650 unknown :-1 s=CLOSED pgs=3357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:16.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.341+0000 7fe5dc27c640 1 -- 172.21.15.84:0/3851942010 >> 172.21.15.84:0/3851942010 conn(0x7fe5d4099ea0 msgr2=0x7fe5d415e570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:16.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.342+0000 7fe5dc27c640 1 -- 172.21.15.84:0/3851942010 shutdown_connections 2023-10-16T11:27:16.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.342+0000 7fe5dc27c640 1 -- 172.21.15.84:0/3851942010 wait complete. 2023-10-16T11:27:16.363 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T11:27:16.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.575+0000 7fc85d067640 1 Processor -- start 2023-10-16T11:27:16.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.575+0000 7fc85d067640 1 -- start start 2023-10-16T11:27:16.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.577+0000 7fc85d067640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc85806c040 0x7fc85806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:16.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.577+0000 7fc85d067640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc8580ad970 0x7fc85815e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:16.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.577+0000 7fc85d067640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc85806c960 con 0x7fc85815ebc0 2023-10-16T11:27:16.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.577+0000 7fc85d067640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85815b9d0 con 0x7fc85806c040 2023-10-16T11:27:16.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.577+0000 7fc85d067640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85815bcb0 con 0x7fc8580ad970 2023-10-16T11:27:16.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.577+0000 7fc857fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc85806c040 0x7fc85806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:16.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.577+0000 7fc857fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc85806c040 0x7fc85806c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46652/0 (socket says 172.21.15.84:46652) 2023-10-16T11:27:16.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.577+0000 7fc857fff640 1 -- 172.21.15.84:0/3525411713 learned_addr learned my addr 172.21.15.84:0/3525411713 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:16.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.578+0000 7fc8577fe640 1 --2- 172.21.15.84:0/3525411713 >> v2:172.21.15.157:3300/0 conn(0x7fc8580ad970 0x7fc85815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:16.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.578+0000 7fc857fff640 1 -- 172.21.15.84:0/3525411713 >> v2:172.21.15.157:3300/0 conn(0x7fc8580ad970 msgr2=0x7fc85815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:16.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.578+0000 7fc857fff640 1 --2- 172.21.15.84:0/3525411713 >> v2:172.21.15.157:3300/0 conn(0x7fc8580ad970 0x7fc85815e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:16.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.578+0000 7fc857fff640 1 -- 172.21.15.84:0/3525411713 >> v1:172.21.15.157:6789/0 conn(0x7fc85815ebc0 legacy=0x7fc8580cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:16.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.578+0000 7fc857fff640 1 -- 172.21.15.84:0/3525411713 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85815bf90 con 0x7fc85806c040 2023-10-16T11:27:16.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.579+0000 7fc857fff640 1 --2- 172.21.15.84:0/3525411713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc85806c040 0x7fc85806c420 secure :-1 s=READY pgs=3358 cs=0 l=1 rev1=1 crypto rx=0x7fc8580cb060 tx=0x7fc84802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9a0fe6c4aeef83 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:16.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.580+0000 7fc856ffd640 1 -- 172.21.15.84:0/3525411713 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848031870 con 0x7fc85806c040 2023-10-16T11:27:16.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.580+0000 7fc856ffd640 1 -- 172.21.15.84:0/3525411713 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8480319d0 con 0x7fc85806c040 2023-10-16T11:27:16.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.580+0000 7fc856ffd640 1 -- 172.21.15.84:0/3525411713 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848031d60 con 0x7fc85806c040 2023-10-16T11:27:16.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.581+0000 7fc85d067640 1 -- 172.21.15.84:0/3525411713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc85806c040 msgr2=0x7fc85806c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:16.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.581+0000 7fc85d067640 1 --2- 172.21.15.84:0/3525411713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc85806c040 0x7fc85806c420 secure :-1 s=READY pgs=3358 cs=0 l=1 rev1=1 crypto rx=0x7fc8580cb060 tx=0x7fc84802b910 comp rx=0 tx=0).stop 2023-10-16T11:27:16.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.581+0000 7fc85d067640 1 -- 172.21.15.84:0/3525411713 shutdown_connections 2023-10-16T11:27:16.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.581+0000 7fc85d067640 1 --2- 172.21.15.84:0/3525411713 >> v2:172.21.15.157:3300/0 conn(0x7fc8580ad970 0x7fc85815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:16.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.581+0000 7fc85d067640 1 --2- 172.21.15.84:0/3525411713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc85806c040 0x7fc85806c420 unknown :-1 s=CLOSED pgs=3358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:16.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.581+0000 7fc85d067640 1 -- 172.21.15.84:0/3525411713 >> 172.21.15.84:0/3525411713 conn(0x7fc8580ce6a0 msgr2=0x7fc8580ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:16.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.581+0000 7fc85d067640 1 -- 172.21.15.84:0/3525411713 shutdown_connections 2023-10-16T11:27:16.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.581+0000 7fc85d067640 1 -- 172.21.15.84:0/3525411713 wait complete. 2023-10-16T11:27:16.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.583+0000 7fc85d067640 1 Processor -- start 2023-10-16T11:27:16.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.583+0000 7fc85d067640 1 -- start start 2023-10-16T11:27:16.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.584+0000 7fc85d067640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc85806c040 0x7fc8581edcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:16.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.584+0000 7fc85d067640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8580ad970 0x7fc8580aca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:16.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.584+0000 7fc85d067640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc85815ebc0 0x7fc8580acf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:16.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.584+0000 7fc85d067640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc858158bb0 con 0x7fc85815ebc0 2023-10-16T11:27:16.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.584+0000 7fc85d067640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc858157df0 con 0x7fc8580ad970 2023-10-16T11:27:16.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.584+0000 7fc85d067640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8581580d0 con 0x7fc85806c040 2023-10-16T11:27:16.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.584+0000 7fc85c866640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc85815ebc0 0x7fc8580acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:16.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.584+0000 7fc85c866640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc85815ebc0 0x7fc8580acf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46666/0 (socket says 172.21.15.84:46666) 2023-10-16T11:27:16.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.584+0000 7fc85c866640 1 -- 172.21.15.84:0/810706190 learned_addr learned my addr 172.21.15.84:0/810706190 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:16.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.585+0000 7fc8577fe640 1 --2- 172.21.15.84:0/810706190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8580ad970 0x7fc8580aca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:16.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.585+0000 7fc857fff640 1 --2- 172.21.15.84:0/810706190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc85806c040 0x7fc8581edcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:16.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.585+0000 7fc85c866640 1 -- 172.21.15.84:0/810706190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc85806c040 msgr2=0x7fc8581edcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:16.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.585+0000 7fc85c866640 1 --2- 172.21.15.84:0/810706190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc85806c040 0x7fc8581edcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:16.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.585+0000 7fc85c866640 1 -- 172.21.15.84:0/810706190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8580ad970 msgr2=0x7fc8580aca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:16.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.585+0000 7fc85c866640 1 --2- 172.21.15.84:0/810706190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8580ad970 0x7fc8580aca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:16.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.585+0000 7fc85c866640 1 -- 172.21.15.84:0/810706190 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8480094d0 con 0x7fc85815ebc0 2023-10-16T11:27:16.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.585+0000 7fc8577fe640 1 --2- 172.21.15.84:0/810706190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8580ad970 0x7fc8580aca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:16.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.585+0000 7fc85c866640 1 --2- 172.21.15.84:0/810706190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc85815ebc0 0x7fc8580acf40 secure :-1 s=READY pgs=3359 cs=0 l=1 rev1=1 crypto rx=0x7fc85000c780 tx=0x7fc85000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:16.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.585+0000 7fc854ff9640 1 -- 172.21.15.84:0/810706190 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850004b60 con 0x7fc85815ebc0 2023-10-16T11:27:16.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.585+0000 7fc854ff9640 1 -- 172.21.15.84:0/810706190 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85000ce90 con 0x7fc85815ebc0 2023-10-16T11:27:16.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.585+0000 7fc854ff9640 1 -- 172.21.15.84:0/810706190 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85000a850 con 0x7fc85815ebc0 2023-10-16T11:27:16.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.585+0000 7fc85d067640 1 -- 172.21.15.84:0/810706190 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8581583b0 con 0x7fc85815ebc0 2023-10-16T11:27:16.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.586+0000 7fc85d067640 1 -- 172.21.15.84:0/810706190 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc858157aa0 con 0x7fc85815ebc0 2023-10-16T11:27:16.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.587+0000 7fc85d067640 1 -- 172.21.15.84:0/810706190 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85806b330 con 0x7fc85815ebc0 2023-10-16T11:27:16.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.588+0000 7fc854ff9640 1 -- 172.21.15.84:0/810706190 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc85000a9b0 con 0x7fc85815ebc0 2023-10-16T11:27:16.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.589+0000 7fc854ff9640 1 --2- 172.21.15.84:0/810706190 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fc84406afc0 0x7fc84406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:16.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.589+0000 7fc854ff9640 1 -- 172.21.15.84:0/810706190 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 5831+0+0 (secure 0 0 0) 0x7fc850089980 con 0x7fc85815ebc0 2023-10-16T11:27:16.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.589+0000 7fc857fff640 1 --2- 172.21.15.84:0/810706190 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fc84406afc0 0x7fc84406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:16.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.590+0000 7fc857fff640 1 --2- 172.21.15.84:0/810706190 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fc84406afc0 0x7fc84406d480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc848036000 tx=0x7fc84803c040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:16.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.596+0000 7fc854ff9640 1 -- 172.21.15.84:0/810706190 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc8500548a0 con 0x7fc85815ebc0 2023-10-16T11:27:16.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.785+0000 7fc85d067640 1 -- 172.21.15.84:0/810706190 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc8581588d0 con 0x7fc85815ebc0 2023-10-16T11:27:16.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.792+0000 7fc854ff9640 1 -- 172.21.15.84:0/810706190 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v94)=0 v94) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc8581588d0 con 0x7fc85815ebc0 2023-10-16T11:27:16.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.794+0000 7fc85d067640 1 -- 172.21.15.84:0/810706190 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fc84406afc0 msgr2=0x7fc84406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:16.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.794+0000 7fc85d067640 1 --2- 172.21.15.84:0/810706190 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fc84406afc0 0x7fc84406d480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc848036000 tx=0x7fc84803c040 comp rx=0 tx=0).stop 2023-10-16T11:27:16.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.794+0000 7fc85d067640 1 -- 172.21.15.84:0/810706190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc85815ebc0 msgr2=0x7fc8580acf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:16.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.795+0000 7fc85d067640 1 --2- 172.21.15.84:0/810706190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc85815ebc0 0x7fc8580acf40 secure :-1 s=READY pgs=3359 cs=0 l=1 rev1=1 crypto rx=0x7fc85000c780 tx=0x7fc85000cc40 comp rx=0 tx=0).stop 2023-10-16T11:27:16.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.795+0000 7fc85d067640 1 -- 172.21.15.84:0/810706190 shutdown_connections 2023-10-16T11:27:16.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.795+0000 7fc85d067640 1 --2- 172.21.15.84:0/810706190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc85815ebc0 0x7fc8580acf40 unknown :-1 s=CLOSED pgs=3359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:16.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.795+0000 7fc85d067640 1 --2- 172.21.15.84:0/810706190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8580ad970 0x7fc8580aca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:16.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.795+0000 7fc85d067640 1 --2- 172.21.15.84:0/810706190 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fc84406afc0 0x7fc84406d480 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:16.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.795+0000 7fc85d067640 1 --2- 172.21.15.84:0/810706190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc85806c040 0x7fc8581edcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:16.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.795+0000 7fc85d067640 1 -- 172.21.15.84:0/810706190 >> 172.21.15.84:0/810706190 conn(0x7fc8580ce6a0 msgr2=0x7fc8580c5770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:16.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.795+0000 7fc85d067640 1 -- 172.21.15.84:0/810706190 shutdown_connections 2023-10-16T11:27:16.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:16.795+0000 7fc85d067640 1 -- 172.21.15.84:0/810706190 wait complete. 2023-10-16T11:27:16.814 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:27:16.815 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:27:16.928 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:27:17.054 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:27:17.054 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:27:17.055 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:17.114 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:27:17.114 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:27:17.114 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:17.174 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:27:17.175 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:27:17.175 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:17.235 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:27:17.236 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:27:17.236 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:17.296 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:27:17.297 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T11:27:17.297 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:27:17.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.515+0000 7f21292e0640 1 Processor -- start 2023-10-16T11:27:17.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.515+0000 7f21292e0640 1 -- start start 2023-10-16T11:27:17.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.517+0000 7f21292e0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f21241593b0 0x7f212406f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:17.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.517+0000 7f21292e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21240cc920 0x7f21240ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:17.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.517+0000 7f21292e0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2124159790 con 0x7f21240caa10 2023-10-16T11:27:17.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.517+0000 7f21292e0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21240c9b20 con 0x7f21241593b0 2023-10-16T11:27:17.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.517+0000 7f21292e0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21240c9e00 con 0x7f21240cc920 2023-10-16T11:27:17.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.517+0000 7f2122ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f21241593b0 0x7f212406f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:17.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.517+0000 7f2122ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f21241593b0 0x7f212406f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46678/0 (socket says 172.21.15.84:46678) 2023-10-16T11:27:17.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.517+0000 7f2122ffd640 1 -- 172.21.15.84:0/1620767211 learned_addr learned my addr 172.21.15.84:0/1620767211 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:17.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.518+0000 7f21237fe640 1 --2- 172.21.15.84:0/1620767211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21240cc920 0x7f21240ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:17.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.518+0000 7f2122ffd640 1 -- 172.21.15.84:0/1620767211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21240cc920 msgr2=0x7f21240ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:17.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.518+0000 7f2122ffd640 1 --2- 172.21.15.84:0/1620767211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21240cc920 0x7f21240ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:17.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.518+0000 7f2122ffd640 1 -- 172.21.15.84:0/1620767211 >> v1:172.21.15.157:6789/0 conn(0x7f21240caa10 legacy=0x7f212406fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:17.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.518+0000 7f2122ffd640 1 -- 172.21.15.84:0/1620767211 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21240ca0e0 con 0x7f21241593b0 2023-10-16T11:27:17.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.519+0000 7f2122ffd640 1 --2- 172.21.15.84:0/1620767211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21241593b0 0x7f212406f550 secure :-1 s=READY pgs=3360 cs=0 l=1 rev1=1 crypto rx=0x7f2114009870 tx=0x7f211402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ee8e80abe4ad8e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:17.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.519+0000 7f2121ffb640 1 -- 172.21.15.84:0/1620767211 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2114031740 con 0x7f21241593b0 2023-10-16T11:27:17.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.519+0000 7f2121ffb640 1 -- 172.21.15.84:0/1620767211 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21140318a0 con 0x7f21241593b0 2023-10-16T11:27:17.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.519+0000 7f2121ffb640 1 -- 172.21.15.84:0/1620767211 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2114031c30 con 0x7f21241593b0 2023-10-16T11:27:17.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.520+0000 7f21292e0640 1 -- 172.21.15.84:0/1620767211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21241593b0 msgr2=0x7f212406f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:17.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.520+0000 7f21292e0640 1 --2- 172.21.15.84:0/1620767211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21241593b0 0x7f212406f550 secure :-1 s=READY pgs=3360 cs=0 l=1 rev1=1 crypto rx=0x7f2114009870 tx=0x7f211402b7e0 comp rx=0 tx=0).stop 2023-10-16T11:27:17.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.520+0000 7f21292e0640 1 -- 172.21.15.84:0/1620767211 shutdown_connections 2023-10-16T11:27:17.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.520+0000 7f21292e0640 1 --2- 172.21.15.84:0/1620767211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21240cc920 0x7f21240ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:17.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.520+0000 7f21292e0640 1 --2- 172.21.15.84:0/1620767211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21241593b0 0x7f212406f550 unknown :-1 s=CLOSED pgs=3360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:17.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.520+0000 7f21292e0640 1 -- 172.21.15.84:0/1620767211 >> 172.21.15.84:0/1620767211 conn(0x7f21240cb410 msgr2=0x7f21240cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:17.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.520+0000 7f21292e0640 1 -- 172.21.15.84:0/1620767211 shutdown_connections 2023-10-16T11:27:17.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.520+0000 7f21292e0640 1 -- 172.21.15.84:0/1620767211 wait complete. 2023-10-16T11:27:17.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.521+0000 7f21292e0640 1 Processor -- start 2023-10-16T11:27:17.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.522+0000 7f21292e0640 1 -- start start 2023-10-16T11:27:17.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.522+0000 7f21292e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21240caa10 0x7f212406bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:17.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.522+0000 7f21292e0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21240cc920 0x7f21240c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:17.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.523+0000 7f21292e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21241593b0 0x7f21240c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:17.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.523+0000 7f21292e0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2124159920 con 0x7f21240caa10 2023-10-16T11:27:17.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.523+0000 7f21292e0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2124159c00 con 0x7f21240cc920 2023-10-16T11:27:17.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.523+0000 7f21292e0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2124159ee0 con 0x7f21241593b0 2023-10-16T11:27:17.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.523+0000 7f2122ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21240caa10 0x7f212406bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:17.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.523+0000 7f2122ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21240caa10 0x7f212406bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46692/0 (socket says 172.21.15.84:46692) 2023-10-16T11:27:17.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.523+0000 7f2122ffd640 1 -- 172.21.15.84:0/2442050692 learned_addr learned my addr 172.21.15.84:0/2442050692 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:17.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.523+0000 7f21227fc640 1 --2- 172.21.15.84:0/2442050692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21240cc920 0x7f21240c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:17.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.523+0000 7f2122ffd640 1 -- 172.21.15.84:0/2442050692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21241593b0 msgr2=0x7f21240c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:17.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.523+0000 7f21237fe640 1 --2- 172.21.15.84:0/2442050692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21241593b0 0x7f21240c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:17.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.523+0000 7f2122ffd640 1 --2- 172.21.15.84:0/2442050692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21241593b0 0x7f21240c81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:17.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.523+0000 7f2122ffd640 1 -- 172.21.15.84:0/2442050692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21240cc920 msgr2=0x7f21240c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:17.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.523+0000 7f2122ffd640 1 --2- 172.21.15.84:0/2442050692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21240cc920 0x7f21240c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:17.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.523+0000 7f2122ffd640 1 -- 172.21.15.84:0/2442050692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21140094d0 con 0x7f21240caa10 2023-10-16T11:27:17.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.523+0000 7f21227fc640 1 --2- 172.21.15.84:0/2442050692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21240cc920 0x7f21240c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:17.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.523+0000 7f21237fe640 1 --2- 172.21.15.84:0/2442050692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21241593b0 0x7f21240c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:17.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.524+0000 7f2122ffd640 1 --2- 172.21.15.84:0/2442050692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21240caa10 0x7f212406bc00 secure :-1 s=READY pgs=3361 cs=0 l=1 rev1=1 crypto rx=0x7f2114009870 tx=0x7f21140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:17.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.524+0000 7f211b7fe640 1 -- 172.21.15.84:0/2442050692 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2114033c70 con 0x7f21240caa10 2023-10-16T11:27:17.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.524+0000 7f211b7fe640 1 -- 172.21.15.84:0/2442050692 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2114033df0 con 0x7f21240caa10 2023-10-16T11:27:17.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.524+0000 7f211b7fe640 1 -- 172.21.15.84:0/2442050692 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211403d6b0 con 0x7f21240caa10 2023-10-16T11:27:17.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.524+0000 7f21292e0640 1 -- 172.21.15.84:0/2442050692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f212415a1c0 con 0x7f21240caa10 2023-10-16T11:27:17.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.524+0000 7f21292e0640 1 -- 172.21.15.84:0/2442050692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212414c210 con 0x7f21240caa10 2023-10-16T11:27:17.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.525+0000 7f21292e0640 1 -- 172.21.15.84:0/2442050692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f212414c3a0 con 0x7f21240caa10 2023-10-16T11:27:17.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.526+0000 7f211b7fe640 1 -- 172.21.15.84:0/2442050692 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f211403a040 con 0x7f21240caa10 2023-10-16T11:27:17.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.527+0000 7f211b7fe640 1 --2- 172.21.15.84:0/2442050692 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f210406afc0 0x7f210406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:17.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.527+0000 7f211b7fe640 1 -- 172.21.15.84:0/2442050692 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 5831+0+0 (secure 0 0 0) 0x7f21140ab8b0 con 0x7f21240caa10 2023-10-16T11:27:17.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.528+0000 7f21227fc640 1 --2- 172.21.15.84:0/2442050692 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f210406afc0 0x7f210406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:17.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.528+0000 7f21227fc640 1 --2- 172.21.15.84:0/2442050692 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f210406afc0 0x7f210406d480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f21240c4ec0 tx=0x7f210c008040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:17.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.531+0000 7f211b7fe640 1 -- 172.21.15.84:0/2442050692 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f21140766b0 con 0x7f21240caa10 2023-10-16T11:27:17.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.761+0000 7f21292e0640 1 -- 172.21.15.84:0/2442050692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f212404e160 con 0x7f21240caa10 2023-10-16T11:27:17.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.762+0000 7f211b7fe640 1 -- 172.21.15.84:0/2442050692 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) v1 ==== 95+0+3707 (secure 0 0 0) 0x7f211404ee90 con 0x7f21240caa10 2023-10-16T11:27:17.765 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:17.765 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":150,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/591944719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":591944719},{"type":"v1","addr":"172.21.15.84:6835","nonce":591944719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":142},{"gid":7421,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2732456137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2732456137},{"type":"v1","addr":"172.21.15.157:6835","nonce":2732456137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":150},{"gid":8152,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2371517609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2371517609},{"type":"v1","addr":"172.21.15.84:6837","nonce":2371517609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":143},{"gid":8161,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1079456313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1079456313},{"type":"v1","addr":"172.21.15.157:6834","nonce":1079456313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":143}],"filesystems":[]} 2023-10-16T11:27:17.765 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 150 2023-10-16T11:27:17.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.765+0000 7f21292e0640 1 -- 172.21.15.84:0/2442050692 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f210406afc0 msgr2=0x7f210406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:17.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.765+0000 7f21292e0640 1 --2- 172.21.15.84:0/2442050692 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f210406afc0 0x7f210406d480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f21240c4ec0 tx=0x7f210c008040 comp rx=0 tx=0).stop 2023-10-16T11:27:17.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.765+0000 7f21292e0640 1 -- 172.21.15.84:0/2442050692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21240caa10 msgr2=0x7f212406bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:17.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.765+0000 7f21292e0640 1 --2- 172.21.15.84:0/2442050692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21240caa10 0x7f212406bc00 secure :-1 s=READY pgs=3361 cs=0 l=1 rev1=1 crypto rx=0x7f2114009870 tx=0x7f21140046c0 comp rx=0 tx=0).stop 2023-10-16T11:27:17.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.765+0000 7f21292e0640 1 -- 172.21.15.84:0/2442050692 shutdown_connections 2023-10-16T11:27:17.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.765+0000 7f21292e0640 1 --2- 172.21.15.84:0/2442050692 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f210406afc0 0x7f210406d480 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:17.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.765+0000 7f21292e0640 1 --2- 172.21.15.84:0/2442050692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21241593b0 0x7f21240c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:17.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.765+0000 7f21292e0640 1 --2- 172.21.15.84:0/2442050692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21240cc920 0x7f21240c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:17.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.765+0000 7f21292e0640 1 --2- 172.21.15.84:0/2442050692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21240caa10 0x7f212406bc00 unknown :-1 s=CLOSED pgs=3361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:17.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.765+0000 7f21292e0640 1 -- 172.21.15.84:0/2442050692 >> 172.21.15.84:0/2442050692 conn(0x7f21240cb410 msgr2=0x7f21240cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:17.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.765+0000 7f21292e0640 1 -- 172.21.15.84:0/2442050692 shutdown_connections 2023-10-16T11:27:17.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:17.765+0000 7f21292e0640 1 -- 172.21.15.84:0/2442050692 wait complete. 2023-10-16T11:27:17.787 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T11:27:17.787 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T11:27:17.788 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T11:27:17.788 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:27:17.788 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T11:27:17.788 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T11:27:17.789 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T11:27:17.789 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:27:17.789 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T11:27:17.790 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T11:27:17.790 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T11:27:17.790 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:27:17.790 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T11:27:17.791 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T11:27:17.791 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T11:27:17.791 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:27:17.807 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T11:27:17.807 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T11:27:17.808 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T11:27:17.808 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T11:27:17.811 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T11:27:17.812 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T11:27:17.813 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T11:27:17.813 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T11:27:17.855 INFO:tasks.ceph.mds.c:Started 2023-10-16T11:27:17.856 DEBUG:teuthology.parallel:result is None 2023-10-16T11:27:17.861 INFO:tasks.ceph.mds.b:Started 2023-10-16T11:27:17.862 DEBUG:teuthology.parallel:result is None 2023-10-16T11:27:17.863 INFO:tasks.ceph.mds.a:Started 2023-10-16T11:27:17.863 DEBUG:teuthology.parallel:result is None 2023-10-16T11:27:17.869 INFO:tasks.ceph.mds.d:Started 2023-10-16T11:27:17.870 DEBUG:teuthology.parallel:result is None 2023-10-16T11:27:17.870 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T11:27:18.013 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-16T11:27:18.018 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T11:27:18.025 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2023-10-16T11:27:18.025 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2023-10-16T11:27:18.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.055+0000 7fd026533640 1 Processor -- start 2023-10-16T11:27:18.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.056+0000 7fd026533640 1 -- start start 2023-10-16T11:27:18.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.057+0000 7fd026533640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0200caac0 0x7fd0200ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:18.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.057+0000 7fd026533640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0200cb030 con 0x7fd02015ad70 2023-10-16T11:27:18.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.057+0000 7fd026533640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0200c91c0 con 0x7fd02006a4b0 2023-10-16T11:27:18.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.057+0000 7fd026533640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0200c94a0 con 0x7fd0200caac0 2023-10-16T11:27:18.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.057+0000 7fd01ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd02015ad70 0x7fd020069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55294/0 (socket says 172.21.15.84:55294) 2023-10-16T11:27:18.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.057+0000 7fd01ffff640 1 -- 172.21.15.84:0/3539186130 learned_addr learned my addr 172.21.15.84:0/3539186130 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:18.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.057+0000 7fd024ad0640 1 --2- 172.21.15.84:0/3539186130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0200caac0 0x7fd0200ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:18.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.058+0000 7fd01f7fe640 1 -- 172.21.15.84:0/3539186130 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336172953 0 0) 0x7fd0200cb030 con 0x7fd02015ad70 2023-10-16T11:27:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.058+0000 7fd01f7fe640 1 -- 172.21.15.84:0/3539186130 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd00c003520 con 0x7fd02015ad70 2023-10-16T11:27:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.058+0000 7fd01f7fe640 1 -- 172.21.15.84:0/3539186130 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578481442 0 0) 0x7fd0200c91c0 con 0x7fd02006a4b0 2023-10-16T11:27:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.058+0000 7fd024ad0640 1 -- 172.21.15.84:0/3539186130 >> v1:172.21.15.157:6789/0 conn(0x7fd02006a4b0 legacy=0x7fd0200ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.058+0000 7fd024ad0640 1 -- 172.21.15.84:0/3539186130 >> v1:172.21.15.84:6789/0 conn(0x7fd02015ad70 legacy=0x7fd020069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.058+0000 7fd024ad0640 1 -- 172.21.15.84:0/3539186130 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0200c9780 con 0x7fd0200caac0 2023-10-16T11:27:18.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.059+0000 7fd024ad0640 1 --2- 172.21.15.84:0/3539186130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0200caac0 0x7fd0200ad4d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd018009dc0 tx=0x7fd01802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21e1e84265278f21 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:18.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.059+0000 7fd01f7fe640 1 -- 172.21.15.84:0/3539186130 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01802ccd0 con 0x7fd0200caac0 2023-10-16T11:27:18.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.059+0000 7fd01f7fe640 1 -- 172.21.15.84:0/3539186130 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01802ce30 con 0x7fd0200caac0 2023-10-16T11:27:18.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.059+0000 7fd01f7fe640 1 -- 172.21.15.84:0/3539186130 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018034660 con 0x7fd0200caac0 2023-10-16T11:27:18.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.060+0000 7fd026533640 1 -- 172.21.15.84:0/3539186130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0200caac0 msgr2=0x7fd0200ad4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:18.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.060+0000 7fd026533640 1 --2- 172.21.15.84:0/3539186130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0200caac0 0x7fd0200ad4d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd018009dc0 tx=0x7fd01802bf20 comp rx=0 tx=0).stop 2023-10-16T11:27:18.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.060+0000 7fd026533640 1 -- 172.21.15.84:0/3539186130 shutdown_connections 2023-10-16T11:27:18.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.060+0000 7fd026533640 1 --2- 172.21.15.84:0/3539186130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0200caac0 0x7fd0200ad4d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:18.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.060+0000 7fd026533640 1 -- 172.21.15.84:0/3539186130 >> 172.21.15.84:0/3539186130 conn(0x7fd0200ce9b0 msgr2=0x7fd0200cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:18.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.060+0000 7fd026533640 1 -- 172.21.15.84:0/3539186130 shutdown_connections 2023-10-16T11:27:18.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.060+0000 7fd026533640 1 -- 172.21.15.84:0/3539186130 wait complete. 2023-10-16T11:27:18.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.062+0000 7fd026533640 1 Processor -- start 2023-10-16T11:27:18.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.062+0000 7fd026533640 1 -- start start 2023-10-16T11:27:18.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.063+0000 7fd026533640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd02006a4b0 0x7fd02014b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:18.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.063+0000 7fd026533640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0200caac0 0x7fd0200c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:18.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.063+0000 7fd026533640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd02015ad70 0x7fd02015da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:18.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.063+0000 7fd026533640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02015c0d0 con 0x7fd02015ad70 2023-10-16T11:27:18.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.063+0000 7fd026533640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02014bc50 con 0x7fd0200caac0 2023-10-16T11:27:18.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.063+0000 7fd026533640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02015b450 con 0x7fd02006a4b0 2023-10-16T11:27:18.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.063+0000 7fd024ad0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd02015ad70 0x7fd02015da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:18.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.063+0000 7fd024ad0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd02015ad70 0x7fd02015da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46730/0 (socket says 172.21.15.84:46730) 2023-10-16T11:27:18.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.063+0000 7fd024ad0640 1 -- 172.21.15.84:0/1573596402 learned_addr learned my addr 172.21.15.84:0/1573596402 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:18.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.063+0000 7fd01ffff640 1 --2- 172.21.15.84:0/1573596402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd02006a4b0 0x7fd02014b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:18.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.063+0000 7fd017fff640 1 --2- 172.21.15.84:0/1573596402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0200caac0 0x7fd0200c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:18.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.064+0000 7fd024ad0640 1 -- 172.21.15.84:0/1573596402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd02006a4b0 msgr2=0x7fd02014b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:18.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.064+0000 7fd024ad0640 1 --2- 172.21.15.84:0/1573596402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd02006a4b0 0x7fd02014b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:18.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.064+0000 7fd024ad0640 1 -- 172.21.15.84:0/1573596402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0200caac0 msgr2=0x7fd0200c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:18.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.064+0000 7fd024ad0640 1 --2- 172.21.15.84:0/1573596402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0200caac0 0x7fd0200c4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:18.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.064+0000 7fd024ad0640 1 -- 172.21.15.84:0/1573596402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd018009a70 con 0x7fd02015ad70 2023-10-16T11:27:18.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.064+0000 7fd024ad0640 1 --2- 172.21.15.84:0/1573596402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd02015ad70 0x7fd02015da00 secure :-1 s=READY pgs=3369 cs=0 l=1 rev1=1 crypto rx=0x7fd018009ee0 tx=0x7fd018004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:18.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.064+0000 7fd01d7fa640 1 -- 172.21.15.84:0/1573596402 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018003b90 con 0x7fd02015ad70 2023-10-16T11:27:18.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.064+0000 7fd01d7fa640 1 -- 172.21.15.84:0/1573596402 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd018003cf0 con 0x7fd02015ad70 2023-10-16T11:27:18.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.064+0000 7fd026533640 1 -- 172.21.15.84:0/1573596402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02015b730 con 0x7fd02015ad70 2023-10-16T11:27:18.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.064+0000 7fd01d7fa640 1 -- 172.21.15.84:0/1573596402 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018004270 con 0x7fd02015ad70 2023-10-16T11:27:18.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.064+0000 7fd026533640 1 -- 172.21.15.84:0/1573596402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd02015bcf0 con 0x7fd02015ad70 2023-10-16T11:27:18.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.065+0000 7fd026533640 1 -- 172.21.15.84:0/1573596402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd02015c210 con 0x7fd02015ad70 2023-10-16T11:27:18.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.066+0000 7fd01d7fa640 1 -- 172.21.15.84:0/1573596402 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd01803b040 con 0x7fd02015ad70 2023-10-16T11:27:18.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.067+0000 7fd01d7fa640 1 --2- 172.21.15.84:0/1573596402 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd00006af90 0x7fd00006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:18.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.067+0000 7fd01d7fa640 1 -- 172.21.15.84:0/1573596402 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 5831+0+0 (secure 0 0 0) 0x7fd0180abac0 con 0x7fd02015ad70 2023-10-16T11:27:18.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.068+0000 7fd01ffff640 1 --2- 172.21.15.84:0/1573596402 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd00006af90 0x7fd00006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:18.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.068+0000 7fd01ffff640 1 --2- 172.21.15.84:0/1573596402 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd00006af90 0x7fd00006d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd010005880 tx=0x7fd010009040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:18.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.072+0000 7fd01d7fa640 1 -- 172.21.15.84:0/1573596402 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd02015c210 con 0x7fd02015ad70 2023-10-16T11:27:18.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.278+0000 7fd026533640 1 -- 172.21.15.84:0/1573596402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd02015c210 con 0x7fd02015ad70 2023-10-16T11:27:18.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.362+0000 7fd01d7fa640 1 -- 172.21.15.84:0/1573596402 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v264) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd02015c210 con 0x7fd02015ad70 2023-10-16T11:27:18.364 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-10-16T11:27:18.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.366+0000 7fd026533640 1 -- 172.21.15.84:0/1573596402 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd00006af90 msgr2=0x7fd00006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:18.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.366+0000 7fd026533640 1 --2- 172.21.15.84:0/1573596402 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd00006af90 0x7fd00006d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd010005880 tx=0x7fd010009040 comp rx=0 tx=0).stop 2023-10-16T11:27:18.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.366+0000 7fd026533640 1 -- 172.21.15.84:0/1573596402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd02015ad70 msgr2=0x7fd02015da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:18.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.366+0000 7fd026533640 1 --2- 172.21.15.84:0/1573596402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd02015ad70 0x7fd02015da00 secure :-1 s=READY pgs=3369 cs=0 l=1 rev1=1 crypto rx=0x7fd018009ee0 tx=0x7fd018004a40 comp rx=0 tx=0).stop 2023-10-16T11:27:18.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.366+0000 7fd026533640 1 -- 172.21.15.84:0/1573596402 shutdown_connections 2023-10-16T11:27:18.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.366+0000 7fd026533640 1 --2- 172.21.15.84:0/1573596402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd02015ad70 0x7fd02015da00 unknown :-1 s=CLOSED pgs=3369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:18.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.366+0000 7fd026533640 1 --2- 172.21.15.84:0/1573596402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0200caac0 0x7fd0200c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:18.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.366+0000 7fd026533640 1 --2- 172.21.15.84:0/1573596402 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd00006af90 0x7fd00006d450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:18.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.366+0000 7fd026533640 1 --2- 172.21.15.84:0/1573596402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd02006a4b0 0x7fd02014b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:18.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.366+0000 7fd026533640 1 -- 172.21.15.84:0/1573596402 >> 172.21.15.84:0/1573596402 conn(0x7fd0200ce9b0 msgr2=0x7fd0200ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:18.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.366+0000 7fd026533640 1 -- 172.21.15.84:0/1573596402 shutdown_connections 2023-10-16T11:27:18.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.366+0000 7fd026533640 1 -- 172.21.15.84:0/1573596402 wait complete. 2023-10-16T11:27:18.389 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T11:27:18.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.592+0000 7fd148a61640 1 Processor -- start 2023-10-16T11:27:18.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.592+0000 7fd148a61640 1 -- start start 2023-10-16T11:27:18.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.593+0000 7fd148a61640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd1441593b0 0x7fd14406f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:18.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.593+0000 7fd148a61640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd1440caa10 0x7fd14406fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:18.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.593+0000 7fd148a61640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd144159790 con 0x7fd14406ffd0 2023-10-16T11:27:18.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.593+0000 7fd148a61640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1440c9b20 con 0x7fd1440caa10 2023-10-16T11:27:18.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.593+0000 7fd148a61640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1440c9e00 con 0x7fd1441593b0 2023-10-16T11:27:18.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.594+0000 7fd141d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd1440caa10 0x7fd14406fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:18.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.594+0000 7fd141d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd1440caa10 0x7fd14406fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46740/0 (socket says 172.21.15.84:46740) 2023-10-16T11:27:18.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.594+0000 7fd141d9b640 1 -- 172.21.15.84:0/3577363648 learned_addr learned my addr 172.21.15.84:0/3577363648 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:18.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.594+0000 7fd141d9b640 1 -- 172.21.15.84:0/3577363648 >> v2:172.21.15.157:3300/0 conn(0x7fd1441593b0 msgr2=0x7fd14406f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:18.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.594+0000 7fd141d9b640 1 --2- 172.21.15.84:0/3577363648 >> v2:172.21.15.157:3300/0 conn(0x7fd1441593b0 0x7fd14406f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:18.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.594+0000 7fd141d9b640 1 -- 172.21.15.84:0/3577363648 >> v1:172.21.15.157:6789/0 conn(0x7fd14406ffd0 legacy=0x7fd14415ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:18.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.594+0000 7fd141d9b640 1 -- 172.21.15.84:0/3577363648 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1440ca0e0 con 0x7fd1440caa10 2023-10-16T11:27:18.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.595+0000 7fd141d9b640 1 --2- 172.21.15.84:0/3577363648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1440caa10 0x7fd14406fa90 secure :-1 s=READY pgs=3370 cs=0 l=1 rev1=1 crypto rx=0x7fd1440ce8f0 tx=0x7fd13402bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4440a0e8cd3ad80 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:18.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.595+0000 7fd14159a640 1 -- 172.21.15.84:0/3577363648 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13402c960 con 0x7fd1440caa10 2023-10-16T11:27:18.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.595+0000 7fd14159a640 1 -- 172.21.15.84:0/3577363648 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13402cac0 con 0x7fd1440caa10 2023-10-16T11:27:18.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.596+0000 7fd14159a640 1 -- 172.21.15.84:0/3577363648 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13402ce50 con 0x7fd1440caa10 2023-10-16T11:27:18.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.596+0000 7fd148a61640 1 -- 172.21.15.84:0/3577363648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1440caa10 msgr2=0x7fd14406fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:18.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.596+0000 7fd148a61640 1 --2- 172.21.15.84:0/3577363648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1440caa10 0x7fd14406fa90 secure :-1 s=READY pgs=3370 cs=0 l=1 rev1=1 crypto rx=0x7fd1440ce8f0 tx=0x7fd13402bdc0 comp rx=0 tx=0).stop 2023-10-16T11:27:18.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.596+0000 7fd148a61640 1 -- 172.21.15.84:0/3577363648 shutdown_connections 2023-10-16T11:27:18.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.596+0000 7fd148a61640 1 --2- 172.21.15.84:0/3577363648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1440caa10 0x7fd14406fa90 unknown :-1 s=CLOSED pgs=3370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:18.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.596+0000 7fd148a61640 1 --2- 172.21.15.84:0/3577363648 >> v2:172.21.15.157:3300/0 conn(0x7fd1441593b0 0x7fd14406f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:18.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.596+0000 7fd148a61640 1 -- 172.21.15.84:0/3577363648 >> 172.21.15.84:0/3577363648 conn(0x7fd1440cb410 msgr2=0x7fd1440cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:18.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.596+0000 7fd148a61640 1 -- 172.21.15.84:0/3577363648 shutdown_connections 2023-10-16T11:27:18.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.596+0000 7fd148a61640 1 -- 172.21.15.84:0/3577363648 wait complete. 2023-10-16T11:27:18.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.598+0000 7fd148a61640 1 Processor -- start 2023-10-16T11:27:18.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.598+0000 7fd148a61640 1 -- start start 2023-10-16T11:27:18.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.599+0000 7fd148a61640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd14406ffd0 0x7fd1440c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:18.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.599+0000 7fd148a61640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1440caa10 0x7fd14415a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:18.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.599+0000 7fd148a61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1441593b0 0x7fd14406c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:18.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.599+0000 7fd148a61640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14415a8e0 con 0x7fd1440caa10 2023-10-16T11:27:18.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.599+0000 7fd148a61640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14415bc00 con 0x7fd14406ffd0 2023-10-16T11:27:18.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.599+0000 7fd148a61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14415ab60 con 0x7fd1441593b0 2023-10-16T11:27:18.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.599+0000 7fd141d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1440caa10 0x7fd14415a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:18.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.599+0000 7fd141d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1440caa10 0x7fd14415a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46746/0 (socket says 172.21.15.84:46746) 2023-10-16T11:27:18.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.599+0000 7fd141d9b640 1 -- 172.21.15.84:0/2887568341 learned_addr learned my addr 172.21.15.84:0/2887568341 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:18.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.600+0000 7fd14259c640 1 --2- 172.21.15.84:0/2887568341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd14406ffd0 0x7fd1440c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:18.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.600+0000 7fd141d9b640 1 -- 172.21.15.84:0/2887568341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1441593b0 msgr2=0x7fd14406c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:18.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.600+0000 7fd142d9d640 1 --2- 172.21.15.84:0/2887568341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1441593b0 0x7fd14406c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:18.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.600+0000 7fd141d9b640 1 --2- 172.21.15.84:0/2887568341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1441593b0 0x7fd14406c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:18.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.600+0000 7fd141d9b640 1 -- 172.21.15.84:0/2887568341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd14406ffd0 msgr2=0x7fd1440c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:18.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.600+0000 7fd141d9b640 1 --2- 172.21.15.84:0/2887568341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd14406ffd0 0x7fd1440c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:18.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.600+0000 7fd141d9b640 1 -- 172.21.15.84:0/2887568341 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd134009b30 con 0x7fd1440caa10 2023-10-16T11:27:18.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.600+0000 7fd14259c640 1 --2- 172.21.15.84:0/2887568341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd14406ffd0 0x7fd1440c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:18.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.600+0000 7fd141d9b640 1 --2- 172.21.15.84:0/2887568341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1440caa10 0x7fd14415a3a0 secure :-1 s=READY pgs=3371 cs=0 l=1 rev1=1 crypto rx=0x7fd134000f80 tx=0x7fd134004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:18.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.601+0000 7fd13affd640 1 -- 172.21.15.84:0/2887568341 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13403e450 con 0x7fd1440caa10 2023-10-16T11:27:18.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.601+0000 7fd13affd640 1 -- 172.21.15.84:0/2887568341 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13403e5b0 con 0x7fd1440caa10 2023-10-16T11:27:18.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.601+0000 7fd13affd640 1 -- 172.21.15.84:0/2887568341 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13403e960 con 0x7fd1440caa10 2023-10-16T11:27:18.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.601+0000 7fd148a61640 1 -- 172.21.15.84:0/2887568341 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14415ae40 con 0x7fd1440caa10 2023-10-16T11:27:18.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.601+0000 7fd148a61640 1 -- 172.21.15.84:0/2887568341 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14415b400 con 0x7fd1440caa10 2023-10-16T11:27:18.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.602+0000 7fd148a61640 1 -- 172.21.15.84:0/2887568341 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14404e160 con 0x7fd1440caa10 2023-10-16T11:27:18.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.603+0000 7fd13affd640 1 -- 172.21.15.84:0/2887568341 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd13403eac0 con 0x7fd1440caa10 2023-10-16T11:27:18.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.604+0000 7fd13affd640 1 --2- 172.21.15.84:0/2887568341 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd12406af90 0x7fd12406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:18.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.604+0000 7fd13affd640 1 -- 172.21.15.84:0/2887568341 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fd1340ac510 con 0x7fd1440caa10 2023-10-16T11:27:18.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.604+0000 7fd14259c640 1 --2- 172.21.15.84:0/2887568341 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd12406af90 0x7fd12406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:18.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.605+0000 7fd14259c640 1 --2- 172.21.15.84:0/2887568341 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd12406af90 0x7fd12406d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd12c006fd0 tx=0x7fd12c008040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:18.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.608+0000 7fd13affd640 1 -- 172.21.15.84:0/2887568341 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd134077740 con 0x7fd1440caa10 2023-10-16T11:27:18.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.860+0000 7fd148a61640 1 -- 172.21.15.84:0/2887568341 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd1440b4910 con 0x7fd1440caa10 2023-10-16T11:27:18.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.861+0000 7fd13affd640 1 -- 172.21.15.84:0/2887568341 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v58) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd13407ad60 con 0x7fd1440caa10 2023-10-16T11:27:18.863 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:18.863 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:27:18.863 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-10-16T11:27:18.864 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.864 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-10-16T11:27:18.864 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l6dhjKRAA0pMvY6AK6qtFXDprNhwsqg==", 2023-10-16T11:27:18.864 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.864 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:27:18.865 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:27:18.865 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:27:18.865 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:27:18.865 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.865 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:27:18.866 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.866 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-10-16T11:27:18.866 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lFpMxEBAA0TLPRt5q3FwwBvMG019cjQ==", 2023-10-16T11:27:18.866 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.866 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:27:18.867 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:27:18.867 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:27:18.867 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:27:18.867 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.867 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:27:18.868 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.868 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-10-16T11:27:18.868 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1l73RBOhAAOMEPxfUAMHhn5Blyvd5oMw==", 2023-10-16T11:27:18.868 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.868 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:27:18.869 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:27:18.869 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:27:18.869 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:27:18.869 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.869 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:27:18.870 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.870 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-10-16T11:27:18.870 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lcTIRIhAA0+FWm4b4yw77LD3+WKbk7w==", 2023-10-16T11:27:18.870 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.870 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:27:18.871 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:27:18.871 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:27:18.871 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:27:18.871 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.871 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:27:18.872 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.872 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-10-16T11:27:18.872 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDYBi1l1zBLERAA3YLW3xIYQoRVw7KSkwN3Og==", 2023-10-16T11:27:18.872 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.872 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:27:18.873 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:27:18.873 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:27:18.873 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.873 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:27:18.873 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.874 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-10-16T11:27:18.874 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDbBi1lFUUxKRAAQbus8SmYprJRmWTjoCznNA==", 2023-10-16T11:27:18.874 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.874 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:27:18.874 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:27:18.875 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:27:18.875 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.875 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:27:18.875 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.875 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-10-16T11:27:18.876 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDeBi1l+9lBMxAArl6sb/h15uBG/a9lVH40Hg==", 2023-10-16T11:27:18.876 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.876 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:27:18.876 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:27:18.876 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:27:18.877 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.877 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:27:18.877 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.877 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-10-16T11:27:18.877 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDiBi1l2rnEERAA3oRD71xa0fEOynM8zKuBEg==", 2023-10-16T11:27:18.878 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.878 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:27:18.878 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:27:18.878 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:27:18.878 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.879 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:27:18.879 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.879 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-10-16T11:27:18.879 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDnBi1lR7oLChAAZto8fDovDbpOd3Fcnc7C6w==", 2023-10-16T11:27:18.879 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.879 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:27:18.880 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:27:18.880 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:27:18.880 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.880 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:27:18.880 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.881 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-10-16T11:27:18.881 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDqBi1lmjg/JBAAlINKRGBc0/UWJJHMVFw57w==", 2023-10-16T11:27:18.881 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.881 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:27:18.882 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:27:18.882 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:27:18.882 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.882 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:27:18.882 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.883 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-10-16T11:27:18.883 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDuBi1lu0kMARAAtuSeM8F3yr8CVrc3p1biaQ==", 2023-10-16T11:27:18.883 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.883 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:27:18.883 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:27:18.884 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:27:18.884 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.884 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:27:18.884 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.884 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-10-16T11:27:18.885 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDxBi1lf6mNGBAAHGbqzJ0fmDxQIsasboHMfg==", 2023-10-16T11:27:18.885 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.885 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-16T11:27:18.885 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-16T11:27:18.885 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:27:18.886 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.886 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:27:18.886 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.886 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-10-16T11:27:18.886 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ==", 2023-10-16T11:27:18.887 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.887 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:27:18.887 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow r", 2023-10-16T11:27:18.887 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T11:27:18.887 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.888 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:27:18.888 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.888 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-10-16T11:27:18.888 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1lHU+4BxAAd8ET8hQ8Y+EUwdTJr9/+Ig==", 2023-10-16T11:27:18.888 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.889 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:27:18.889 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:27:18.889 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:27:18.889 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:27:18.889 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.890 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:27:18.890 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.890 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-10-16T11:27:18.890 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDTBi1lA1BOORAAFNyfqgEF34MVR9YX3KBKkQ==", 2023-10-16T11:27:18.890 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.891 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:27:18.891 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:27:18.891 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:27:18.891 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:27:18.891 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.892 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:27:18.892 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.892 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-10-16T11:27:18.892 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDUBi1l848oERAANMfrZz9JrlgNVh7ZFbtnaA==", 2023-10-16T11:27:18.892 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.893 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-16T11:27:18.893 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-16T11:27:18.893 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-16T11:27:18.893 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-16T11:27:18.893 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.893 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:27:18.894 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.894 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-10-16T11:27:18.894 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDRBi1lTAxTGRAAMptxnxy3Qit/cTCXie6CrQ==", 2023-10-16T11:27:18.894 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.895 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:27:18.895 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-16T11:27:18.895 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-16T11:27:18.895 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:27:18.895 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.896 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:27:18.896 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.896 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-10-16T11:27:18.896 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lPuJxHRAABAnGPVCrkq0JR4Y6iFGZlw==", 2023-10-16T11:27:18.896 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.897 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:27:18.897 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T11:27:18.897 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:27:18.897 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.897 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-16T11:27:18.898 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:18.898 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-10-16T11:27:18.898 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQDSBi1lLxnKEBAA+a/pOHUp7Sg3m4weAust5w==", 2023-10-16T11:27:18.898 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-16T11:27:18.898 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-16T11:27:18.899 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-16T11:27:18.899 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-16T11:27:18.899 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.899 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:18.899 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-10-16T11:27:18.900 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:27:18.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.864+0000 7fd148a61640 1 -- 172.21.15.84:0/2887568341 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd12406af90 msgr2=0x7fd12406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:18.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.864+0000 7fd148a61640 1 --2- 172.21.15.84:0/2887568341 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd12406af90 0x7fd12406d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd12c006fd0 tx=0x7fd12c008040 comp rx=0 tx=0).stop 2023-10-16T11:27:18.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.864+0000 7fd148a61640 1 -- 172.21.15.84:0/2887568341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1440caa10 msgr2=0x7fd14415a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:18.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.864+0000 7fd148a61640 1 --2- 172.21.15.84:0/2887568341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1440caa10 0x7fd14415a3a0 secure :-1 s=READY pgs=3371 cs=0 l=1 rev1=1 crypto rx=0x7fd134000f80 tx=0x7fd134004a40 comp rx=0 tx=0).stop 2023-10-16T11:27:18.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.864+0000 7fd148a61640 1 -- 172.21.15.84:0/2887568341 shutdown_connections 2023-10-16T11:27:18.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.864+0000 7fd148a61640 1 --2- 172.21.15.84:0/2887568341 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd12406af90 0x7fd12406d450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:18.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.864+0000 7fd148a61640 1 --2- 172.21.15.84:0/2887568341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd1441593b0 0x7fd14406c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:18.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.864+0000 7fd148a61640 1 --2- 172.21.15.84:0/2887568341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1440caa10 0x7fd14415a3a0 unknown :-1 s=CLOSED pgs=3371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:18.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.864+0000 7fd148a61640 1 --2- 172.21.15.84:0/2887568341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd14406ffd0 0x7fd1440c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:18.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.864+0000 7fd148a61640 1 -- 172.21.15.84:0/2887568341 >> 172.21.15.84:0/2887568341 conn(0x7fd1440cb410 msgr2=0x7fd144155f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:18.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.864+0000 7fd148a61640 1 -- 172.21.15.84:0/2887568341 shutdown_connections 2023-10-16T11:27:18.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:18.864+0000 7fd148a61640 1 -- 172.21.15.84:0/2887568341 wait complete. 2023-10-16T11:27:18.904 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:27:19.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.112+0000 7fdd25f4e640 1 Processor -- start 2023-10-16T11:27:19.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.112+0000 7fdd25f4e640 1 -- start start 2023-10-16T11:27:19.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.113+0000 7fdd25f4e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdd201593b0 0x7fdd2006f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:19.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.113+0000 7fdd25f4e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd200caa10 0x7fdd2006fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:19.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.113+0000 7fdd25f4e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd20159790 con 0x7fdd2006ffd0 2023-10-16T11:27:19.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.113+0000 7fdd25f4e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd200c9b20 con 0x7fdd201593b0 2023-10-16T11:27:19.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.113+0000 7fdd25f4e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd200c9e00 con 0x7fdd200caa10 2023-10-16T11:27:19.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.113+0000 7fdd1f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdd201593b0 0x7fdd2006f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:19.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.113+0000 7fdd1f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdd201593b0 0x7fdd2006f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46752/0 (socket says 172.21.15.84:46752) 2023-10-16T11:27:19.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.113+0000 7fdd1f7fe640 1 -- 172.21.15.84:0/1861882666 learned_addr learned my addr 172.21.15.84:0/1861882666 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:19.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.114+0000 7fdd1f7fe640 1 -- 172.21.15.84:0/1861882666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd200caa10 msgr2=0x7fdd2006fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:19.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.114+0000 7fdd17fff640 1 --2- 172.21.15.84:0/1861882666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd200caa10 0x7fdd2006fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:19.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.114+0000 7fdd1f7fe640 1 --2- 172.21.15.84:0/1861882666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd200caa10 0x7fdd2006fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.114+0000 7fdd1f7fe640 1 -- 172.21.15.84:0/1861882666 >> v1:172.21.15.157:6789/0 conn(0x7fdd2006ffd0 legacy=0x7fdd2015ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:19.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.114+0000 7fdd1f7fe640 1 -- 172.21.15.84:0/1861882666 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd200ca0e0 con 0x7fdd201593b0 2023-10-16T11:27:19.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.115+0000 7fdd1f7fe640 1 --2- 172.21.15.84:0/1861882666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd201593b0 0x7fdd2006f550 secure :-1 s=READY pgs=3372 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009870 tx=0x7fdd1002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc2f8abff9c97f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:19.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.115+0000 7fdd1effd640 1 -- 172.21.15.84:0/1861882666 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd10031740 con 0x7fdd201593b0 2023-10-16T11:27:19.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.115+0000 7fdd1effd640 1 -- 172.21.15.84:0/1861882666 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd100318a0 con 0x7fdd201593b0 2023-10-16T11:27:19.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.115+0000 7fdd1effd640 1 -- 172.21.15.84:0/1861882666 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd10031c30 con 0x7fdd201593b0 2023-10-16T11:27:19.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.116+0000 7fdd25f4e640 1 -- 172.21.15.84:0/1861882666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd201593b0 msgr2=0x7fdd2006f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:19.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.116+0000 7fdd25f4e640 1 --2- 172.21.15.84:0/1861882666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd201593b0 0x7fdd2006f550 secure :-1 s=READY pgs=3372 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009870 tx=0x7fdd1002b7e0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.116+0000 7fdd25f4e640 1 -- 172.21.15.84:0/1861882666 shutdown_connections 2023-10-16T11:27:19.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.116+0000 7fdd25f4e640 1 --2- 172.21.15.84:0/1861882666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd200caa10 0x7fdd2006fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.116+0000 7fdd25f4e640 1 --2- 172.21.15.84:0/1861882666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd201593b0 0x7fdd2006f550 unknown :-1 s=CLOSED pgs=3372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.116+0000 7fdd25f4e640 1 -- 172.21.15.84:0/1861882666 >> 172.21.15.84:0/1861882666 conn(0x7fdd200cb410 msgr2=0x7fdd200cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:19.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.116+0000 7fdd25f4e640 1 -- 172.21.15.84:0/1861882666 shutdown_connections 2023-10-16T11:27:19.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.116+0000 7fdd25f4e640 1 -- 172.21.15.84:0/1861882666 wait complete. 2023-10-16T11:27:19.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.118+0000 7fdd25f4e640 1 Processor -- start 2023-10-16T11:27:19.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.118+0000 7fdd25f4e640 1 -- start start 2023-10-16T11:27:19.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.119+0000 7fdd25f4e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd2006ffd0 0x7fdd200acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:19.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.119+0000 7fdd25f4e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd200caa10 0x7fdd200c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:19.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.119+0000 7fdd25f4e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd201593b0 0x7fdd201f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:19.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.119+0000 7fdd25f4e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd200b2b60 con 0x7fdd201593b0 2023-10-16T11:27:19.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.119+0000 7fdd25f4e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd200b2570 con 0x7fdd200caa10 2023-10-16T11:27:19.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.119+0000 7fdd25f4e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd201f3ca0 con 0x7fdd2006ffd0 2023-10-16T11:27:19.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.119+0000 7fdd1ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd201593b0 0x7fdd201f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:19.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.119+0000 7fdd1ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd201593b0 0x7fdd201f3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46766/0 (socket says 172.21.15.84:46766) 2023-10-16T11:27:19.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.119+0000 7fdd1ffff640 1 -- 172.21.15.84:0/3147054551 learned_addr learned my addr 172.21.15.84:0/3147054551 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:19.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.119+0000 7fdd1f7fe640 1 --2- 172.21.15.84:0/3147054551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd2006ffd0 0x7fdd200acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:19.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.119+0000 7fdd17fff640 1 --2- 172.21.15.84:0/3147054551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd200caa10 0x7fdd200c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:19.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.119+0000 7fdd1ffff640 1 -- 172.21.15.84:0/3147054551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd2006ffd0 msgr2=0x7fdd200acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:19.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.119+0000 7fdd1ffff640 1 --2- 172.21.15.84:0/3147054551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd2006ffd0 0x7fdd200acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.119+0000 7fdd1ffff640 1 -- 172.21.15.84:0/3147054551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd200caa10 msgr2=0x7fdd200c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:19.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.119+0000 7fdd1ffff640 1 --2- 172.21.15.84:0/3147054551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd200caa10 0x7fdd200c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.119+0000 7fdd1ffff640 1 -- 172.21.15.84:0/3147054551 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd100094d0 con 0x7fdd201593b0 2023-10-16T11:27:19.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.120+0000 7fdd1ffff640 1 --2- 172.21.15.84:0/3147054551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd201593b0 0x7fdd201f3250 secure :-1 s=READY pgs=3373 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800b970 tx=0x7fdd1800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:19.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.120+0000 7fdd1cff9640 1 -- 172.21.15.84:0/3147054551 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1800cb40 con 0x7fdd201593b0 2023-10-16T11:27:19.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.120+0000 7fdd1cff9640 1 -- 172.21.15.84:0/3147054551 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1800cca0 con 0x7fdd201593b0 2023-10-16T11:27:19.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.120+0000 7fdd25f4e640 1 -- 172.21.15.84:0/3147054551 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd201f3f80 con 0x7fdd201593b0 2023-10-16T11:27:19.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.120+0000 7fdd1cff9640 1 -- 172.21.15.84:0/3147054551 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd18014680 con 0x7fdd201593b0 2023-10-16T11:27:19.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.120+0000 7fdd25f4e640 1 -- 172.21.15.84:0/3147054551 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd201f6a00 con 0x7fdd201593b0 2023-10-16T11:27:19.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.121+0000 7fdd25f4e640 1 -- 172.21.15.84:0/3147054551 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd200b2d10 con 0x7fdd201593b0 2023-10-16T11:27:19.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.122+0000 7fdd1cff9640 1 -- 172.21.15.84:0/3147054551 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdd1801d3b0 con 0x7fdd201593b0 2023-10-16T11:27:19.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.124+0000 7fdd1cff9640 1 --2- 172.21.15.84:0/3147054551 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fdd0006afc0 0x7fdd0006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:19.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.124+0000 7fdd1cff9640 1 -- 172.21.15.84:0/3147054551 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fdd18089270 con 0x7fdd201593b0 2023-10-16T11:27:19.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.124+0000 7fdd1f7fe640 1 --2- 172.21.15.84:0/3147054551 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fdd0006afc0 0x7fdd0006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:19.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.125+0000 7fdd1f7fe640 1 --2- 172.21.15.84:0/3147054551 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fdd0006afc0 0x7fdd0006d480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdd10035000 tx=0x7fdd10008030 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:19.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.128+0000 7fdd1cff9640 1 -- 172.21.15.84:0/3147054551 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdd200b2d10 con 0x7fdd201593b0 2023-10-16T11:27:19.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.321+0000 7fdd25f4e640 1 -- 172.21.15.84:0/3147054551 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd200b2d10 con 0x7fdd201593b0 2023-10-16T11:27:19.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.323+0000 7fdd1cff9640 1 -- 172.21.15.84:0/3147054551 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v266) v1 ==== 74+0+12887 (secure 0 0 0) 0x7fdd200b2d10 con 0x7fdd201593b0 2023-10-16T11:27:19.325 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:19.325 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":266,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:27:18.375277+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:27:19.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.325+0000 7fdd25f4e640 1 -- 172.21.15.84:0/3147054551 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fdd0006afc0 msgr2=0x7fdd0006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:19.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.325+0000 7fdd25f4e640 1 --2- 172.21.15.84:0/3147054551 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fdd0006afc0 0x7fdd0006d480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdd10035000 tx=0x7fdd10008030 comp rx=0 tx=0).stop 2023-10-16T11:27:19.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.325+0000 7fdd25f4e640 1 -- 172.21.15.84:0/3147054551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd201593b0 msgr2=0x7fdd201f3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:19.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.325+0000 7fdd25f4e640 1 --2- 172.21.15.84:0/3147054551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd201593b0 0x7fdd201f3250 secure :-1 s=READY pgs=3373 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800b970 tx=0x7fdd1800be30 comp rx=0 tx=0).stop 2023-10-16T11:27:19.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.326+0000 7fdd25f4e640 1 -- 172.21.15.84:0/3147054551 shutdown_connections 2023-10-16T11:27:19.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.326+0000 7fdd25f4e640 1 --2- 172.21.15.84:0/3147054551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd201593b0 0x7fdd201f3250 unknown :-1 s=CLOSED pgs=3373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.326+0000 7fdd25f4e640 1 --2- 172.21.15.84:0/3147054551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd200caa10 0x7fdd200c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.326+0000 7fdd25f4e640 1 --2- 172.21.15.84:0/3147054551 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fdd0006afc0 0x7fdd0006d480 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.326+0000 7fdd25f4e640 1 --2- 172.21.15.84:0/3147054551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd2006ffd0 0x7fdd200acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.326+0000 7fdd25f4e640 1 -- 172.21.15.84:0/3147054551 >> 172.21.15.84:0/3147054551 conn(0x7fdd200cb410 msgr2=0x7fdd200aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:19.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.326+0000 7fdd25f4e640 1 -- 172.21.15.84:0/3147054551 shutdown_connections 2023-10-16T11:27:19.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.326+0000 7fdd25f4e640 1 -- 172.21.15.84:0/3147054551 wait complete. 2023-10-16T11:27:19.347 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:27:19.347 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:27:19.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.559+0000 7fc82cbf5640 1 Processor -- start 2023-10-16T11:27:19.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.559+0000 7fc82cbf5640 1 -- start start 2023-10-16T11:27:19.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.561+0000 7fc82cbf5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc82815d150 0x7fc8280ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:19.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.561+0000 7fc82cbf5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8280ac890 con 0x7fc82806a4b0 2023-10-16T11:27:19.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.561+0000 7fc82cbf5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc828161650 con 0x7fc828160b90 2023-10-16T11:27:19.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.561+0000 7fc82cbf5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82815d530 con 0x7fc82815d150 2023-10-16T11:27:19.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.561+0000 7fc82659c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc82815d150 0x7fc8280ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:19.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.561+0000 7fc826d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc82806a4b0 0x7fc8280c9000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55300/0 (socket says 172.21.15.84:55300) 2023-10-16T11:27:19.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.561+0000 7fc826d9d640 1 -- 172.21.15.84:0/339554996 learned_addr learned my addr 172.21.15.84:0/339554996 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:19.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.562+0000 7fc82659c640 1 -- 172.21.15.84:0/339554996 >> v1:172.21.15.157:6789/0 conn(0x7fc828160b90 legacy=0x7fc828069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:19.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.562+0000 7fc82659c640 1 -- 172.21.15.84:0/339554996 >> v1:172.21.15.84:6789/0 conn(0x7fc82806a4b0 legacy=0x7fc8280c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:19.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.562+0000 7fc82659c640 1 -- 172.21.15.84:0/339554996 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc828161930 con 0x7fc82815d150 2023-10-16T11:27:19.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.562+0000 7fc82559a640 1 -- 172.21.15.84:0/339554996 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483755483 0 0) 0x7fc8280ac890 con 0x7fc82806a4b0 2023-10-16T11:27:19.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.563+0000 7fc82659c640 1 --2- 172.21.15.84:0/339554996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc82815d150 0x7fc8280ad350 secure :-1 s=READY pgs=3375 cs=0 l=1 rev1=1 crypto rx=0x7fc8180097a0 tx=0x7fc81802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c57b516732ba1a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:19.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.563+0000 7fc82559a640 1 -- 172.21.15.84:0/339554996 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818031830 con 0x7fc82815d150 2023-10-16T11:27:19.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.563+0000 7fc82559a640 1 -- 172.21.15.84:0/339554996 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc818031990 con 0x7fc82815d150 2023-10-16T11:27:19.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.563+0000 7fc82559a640 1 -- 172.21.15.84:0/339554996 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818031d20 con 0x7fc82815d150 2023-10-16T11:27:19.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.564+0000 7fc82cbf5640 1 -- 172.21.15.84:0/339554996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc82815d150 msgr2=0x7fc8280ad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:19.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.564+0000 7fc82cbf5640 1 --2- 172.21.15.84:0/339554996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc82815d150 0x7fc8280ad350 secure :-1 s=READY pgs=3375 cs=0 l=1 rev1=1 crypto rx=0x7fc8180097a0 tx=0x7fc81802b8d0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.564+0000 7fc82cbf5640 1 -- 172.21.15.84:0/339554996 shutdown_connections 2023-10-16T11:27:19.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.564+0000 7fc82cbf5640 1 --2- 172.21.15.84:0/339554996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc82815d150 0x7fc8280ad350 unknown :-1 s=CLOSED pgs=3375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.564+0000 7fc82cbf5640 1 -- 172.21.15.84:0/339554996 >> 172.21.15.84:0/339554996 conn(0x7fc8280ce9b0 msgr2=0x7fc8280cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:19.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.564+0000 7fc82cbf5640 1 -- 172.21.15.84:0/339554996 shutdown_connections 2023-10-16T11:27:19.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.564+0000 7fc82cbf5640 1 -- 172.21.15.84:0/339554996 wait complete. 2023-10-16T11:27:19.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.566+0000 7fc82cbf5640 1 Processor -- start 2023-10-16T11:27:19.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.566+0000 7fc82cbf5640 1 -- start start 2023-10-16T11:27:19.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.567+0000 7fc82cbf5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc82806a4b0 0x7fc82814edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:19.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.567+0000 7fc82cbf5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc82815d150 0x7fc82815f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:19.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.567+0000 7fc82cbf5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc828160b90 0x7fc8281e8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:19.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.567+0000 7fc82cbf5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280c6fe0 con 0x7fc82806a4b0 2023-10-16T11:27:19.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.567+0000 7fc82cbf5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280c6b50 con 0x7fc828160b90 2023-10-16T11:27:19.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.567+0000 7fc82cbf5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280c5f40 con 0x7fc82815d150 2023-10-16T11:27:19.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.567+0000 7fc82659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc82806a4b0 0x7fc82814edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:19.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.567+0000 7fc82659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc82806a4b0 0x7fc82814edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46790/0 (socket says 172.21.15.84:46790) 2023-10-16T11:27:19.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.567+0000 7fc82659c640 1 -- 172.21.15.84:0/762496967 learned_addr learned my addr 172.21.15.84:0/762496967 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:19.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.567+0000 7fc82659c640 1 -- 172.21.15.84:0/762496967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc82815d150 msgr2=0x7fc82815f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:19.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.567+0000 7fc82659c640 1 --2- 172.21.15.84:0/762496967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc82815d150 0x7fc82815f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.567+0000 7fc82659c640 1 -- 172.21.15.84:0/762496967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc828160b90 msgr2=0x7fc8281e8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:19.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.567+0000 7fc82659c640 1 --2- 172.21.15.84:0/762496967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc828160b90 0x7fc8281e8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.567+0000 7fc82659c640 1 -- 172.21.15.84:0/762496967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818009400 con 0x7fc82806a4b0 2023-10-16T11:27:19.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.568+0000 7fc82659c640 1 --2- 172.21.15.84:0/762496967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc82806a4b0 0x7fc82814edc0 secure :-1 s=READY pgs=3376 cs=0 l=1 rev1=1 crypto rx=0x7fc8180098c0 tx=0x7fc8180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:19.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.568+0000 7fc81effd640 1 -- 172.21.15.84:0/762496967 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81803ba70 con 0x7fc82806a4b0 2023-10-16T11:27:19.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.568+0000 7fc81effd640 1 -- 172.21.15.84:0/762496967 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81803bbd0 con 0x7fc82806a4b0 2023-10-16T11:27:19.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.568+0000 7fc82cbf5640 1 -- 172.21.15.84:0/762496967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8280c6220 con 0x7fc82806a4b0 2023-10-16T11:27:19.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.568+0000 7fc81effd640 1 -- 172.21.15.84:0/762496967 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8180453f0 con 0x7fc82806a4b0 2023-10-16T11:27:19.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.568+0000 7fc82cbf5640 1 -- 172.21.15.84:0/762496967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8280c67e0 con 0x7fc82806a4b0 2023-10-16T11:27:19.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.569+0000 7fc82cbf5640 1 -- 172.21.15.84:0/762496967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc82804e160 con 0x7fc82806a4b0 2023-10-16T11:27:19.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.570+0000 7fc81effd640 1 -- 172.21.15.84:0/762496967 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc818045550 con 0x7fc82806a4b0 2023-10-16T11:27:19.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.572+0000 7fc81effd640 1 --2- 172.21.15.84:0/762496967 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fc80806af90 0x7fc80806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:19.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.572+0000 7fc81effd640 1 -- 172.21.15.84:0/762496967 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fc8180ab840 con 0x7fc82806a4b0 2023-10-16T11:27:19.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.572+0000 7fc825d9b640 1 --2- 172.21.15.84:0/762496967 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fc80806af90 0x7fc80806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:19.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.573+0000 7fc825d9b640 1 --2- 172.21.15.84:0/762496967 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fc80806af90 0x7fc80806d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc82815f520 tx=0x7fc8100079b0 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:19.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.576+0000 7fc81effd640 1 -- 172.21.15.84:0/762496967 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc818076b10 con 0x7fc82806a4b0 2023-10-16T11:27:19.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.769+0000 7fc82cbf5640 1 -- 172.21.15.84:0/762496967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc8280ade10 con 0x7fc82806a4b0 2023-10-16T11:27:19.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.770+0000 7fc81effd640 1 -- 172.21.15.84:0/762496967 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v266) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc81807a130 con 0x7fc82806a4b0 2023-10-16T11:27:19.772 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:27:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.773+0000 7fc82cbf5640 1 -- 172.21.15.84:0/762496967 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fc80806af90 msgr2=0x7fc80806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.773+0000 7fc82cbf5640 1 --2- 172.21.15.84:0/762496967 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fc80806af90 0x7fc80806d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc82815f520 tx=0x7fc8100079b0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.773+0000 7fc82cbf5640 1 -- 172.21.15.84:0/762496967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc82806a4b0 msgr2=0x7fc82814edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.773+0000 7fc82cbf5640 1 --2- 172.21.15.84:0/762496967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc82806a4b0 0x7fc82814edc0 secure :-1 s=READY pgs=3376 cs=0 l=1 rev1=1 crypto rx=0x7fc8180098c0 tx=0x7fc8180046c0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.773+0000 7fc82cbf5640 1 -- 172.21.15.84:0/762496967 shutdown_connections 2023-10-16T11:27:19.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.773+0000 7fc82cbf5640 1 --2- 172.21.15.84:0/762496967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc828160b90 0x7fc8281e8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.773+0000 7fc82cbf5640 1 --2- 172.21.15.84:0/762496967 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fc80806af90 0x7fc80806d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.773+0000 7fc82cbf5640 1 --2- 172.21.15.84:0/762496967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc82815d150 0x7fc82815f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.773+0000 7fc82cbf5640 1 --2- 172.21.15.84:0/762496967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc82806a4b0 0x7fc82814edc0 unknown :-1 s=CLOSED pgs=3376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:19.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.773+0000 7fc82cbf5640 1 -- 172.21.15.84:0/762496967 >> 172.21.15.84:0/762496967 conn(0x7fc8280ce9b0 msgr2=0x7fc8280ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:19.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.773+0000 7fc82cbf5640 1 -- 172.21.15.84:0/762496967 shutdown_connections 2023-10-16T11:27:19.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.773+0000 7fc82cbf5640 1 -- 172.21.15.84:0/762496967 wait complete. 2023-10-16T11:27:19.793 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T11:27:19.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.994+0000 7ff435690640 1 Processor -- start 2023-10-16T11:27:19.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.994+0000 7ff435690640 1 -- start start 2023-10-16T11:27:19.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.995+0000 7ff435690640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4300b0b20 0x7ff4301615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:19.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.995+0000 7ff435690640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43015e3e0 con 0x7ff43015d3f0 2023-10-16T11:27:19.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.995+0000 7ff435690640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4300cd4a0 con 0x7ff4300cd0c0 2023-10-16T11:27:19.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.995+0000 7ff435690640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43015e100 con 0x7ff4300b0b20 2023-10-16T11:27:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.996+0000 7ff42f7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff43015d3f0 0x7ff43015d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55304/0 (socket says 172.21.15.84:55304) 2023-10-16T11:27:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.996+0000 7ff42f7fe640 1 -- 172.21.15.84:0/3438515964 learned_addr learned my addr 172.21.15.84:0/3438515964 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.996+0000 7ff427fff640 1 --2- 172.21.15.84:0/3438515964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4300b0b20 0x7ff4301615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:20.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.996+0000 7ff42e7fc640 1 -- 172.21.15.84:0/3438515964 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1650605669 0 0) 0x7ff43015e3e0 con 0x7ff43015d3f0 2023-10-16T11:27:20.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.996+0000 7ff42e7fc640 1 -- 172.21.15.84:0/3438515964 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff41c003520 con 0x7ff43015d3f0 2023-10-16T11:27:20.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff42e7fc640 1 -- 172.21.15.84:0/3438515964 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 278488068 0 0) 0x7ff41c003520 con 0x7ff43015d3f0 2023-10-16T11:27:20.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff42e7fc640 1 -- 172.21.15.84:0/3438515964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4300b0b20 msgr2=0x7ff4301615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:20.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff42e7fc640 1 --2- 172.21.15.84:0/3438515964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4300b0b20 0x7ff4301615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:20.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff42e7fc640 1 -- 172.21.15.84:0/3438515964 >> v1:172.21.15.157:6789/0 conn(0x7ff4300cd0c0 legacy=0x7ff4300b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:20.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff42e7fc640 1 -- 172.21.15.84:0/3438515964 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff430160b10 con 0x7ff43015d3f0 2023-10-16T11:27:20.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff42e7fc640 1 -- 172.21.15.84:0/3438515964 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7ff4280019f0 con 0x7ff43015d3f0 2023-10-16T11:27:20.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff42e7fc640 1 -- 172.21.15.84:0/3438515964 >> v1:172.21.15.84:6789/0 conn(0x7ff43015d3f0 legacy=0x7ff43015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:20.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff42e7fc640 1 --2- 172.21.15.84:0/3438515964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff41c0039a0 0x7ff41c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:20.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff42e7fc640 1 --2- 172.21.15.84:0/3438515964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4300b0b20 0x7ff41c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:20.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff42e7fc640 1 --2- 172.21.15.84:0/3438515964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff41c004d00 0x7ff41c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:20.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff42e7fc640 1 -- 172.21.15.84:0/3438515964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43015e100 con 0x7ff41c0039a0 2023-10-16T11:27:20.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff42e7fc640 1 -- 172.21.15.84:0/3438515964 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c00bb40 con 0x7ff41c004d00 2023-10-16T11:27:20.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff42e7fc640 1 -- 172.21.15.84:0/3438515964 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c00bd00 con 0x7ff4300b0b20 2023-10-16T11:27:20.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff42effd640 1 --2- 172.21.15.84:0/3438515964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff41c0039a0 0x7ff41c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:20.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff427fff640 1 --2- 172.21.15.84:0/3438515964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4300b0b20 0x7ff41c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:20.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff42effd640 1 -- 172.21.15.84:0/3438515964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4300b0b20 msgr2=0x7ff41c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:20.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff42effd640 1 --2- 172.21.15.84:0/3438515964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4300b0b20 0x7ff41c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:20.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff42effd640 1 -- 172.21.15.84:0/3438515964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff41c004d00 msgr2=0x7ff41c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:20.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.997+0000 7ff42effd640 1 --2- 172.21.15.84:0/3438515964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff41c004d00 0x7ff41c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:20.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.998+0000 7ff42effd640 1 -- 172.21.15.84:0/3438515964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c003520 con 0x7ff41c0039a0 2023-10-16T11:27:20.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.999+0000 7ff42effd640 1 --2- 172.21.15.84:0/3438515964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff41c0039a0 0x7ff41c003dc0 secure :-1 s=READY pgs=3378 cs=0 l=1 rev1=1 crypto rx=0x7ff420001750 tx=0x7ff41c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a353bb9e8f390c0a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:20.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.999+0000 7ff42e7fc640 1 -- 172.21.15.84:0/3438515964 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4200052d0 con 0x7ff41c0039a0 2023-10-16T11:27:20.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.999+0000 7ff42e7fc640 1 -- 172.21.15.84:0/3438515964 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42002e050 con 0x7ff41c0039a0 2023-10-16T11:27:20.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.999+0000 7ff42e7fc640 1 -- 172.21.15.84:0/3438515964 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4200035f0 con 0x7ff41c0039a0 2023-10-16T11:27:20.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.999+0000 7ff435690640 1 -- 172.21.15.84:0/3438515964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff41c0039a0 msgr2=0x7ff41c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:20.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.999+0000 7ff435690640 1 --2- 172.21.15.84:0/3438515964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff41c0039a0 0x7ff41c003dc0 secure :-1 s=READY pgs=3378 cs=0 l=1 rev1=1 crypto rx=0x7ff420001750 tx=0x7ff41c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:20.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:19.999+0000 7ff42f7fe640 1 -- 172.21.15.84:0/3438515964 reap_dead start 2023-10-16T11:27:20.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.000+0000 7ff435690640 1 -- 172.21.15.84:0/3438515964 shutdown_connections 2023-10-16T11:27:20.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.000+0000 7ff435690640 1 -- 172.21.15.84:0/3438515964 >> 172.21.15.84:0/3438515964 conn(0x7ff4300cb410 msgr2=0x7ff4300cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:20.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.000+0000 7ff435690640 1 -- 172.21.15.84:0/3438515964 shutdown_connections 2023-10-16T11:27:20.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.000+0000 7ff435690640 1 -- 172.21.15.84:0/3438515964 wait complete. 2023-10-16T11:27:20.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.001+0000 7ff435690640 1 Processor -- start 2023-10-16T11:27:20.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.002+0000 7ff435690640 1 -- start start 2023-10-16T11:27:20.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.002+0000 7ff435690640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff41c0039a0 0x7ff4301f2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:20.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.002+0000 7ff435690640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4300aef00 0x7ff4300af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:20.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.003+0000 7ff435690640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff43015ef40 0x7ff43015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:20.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.003+0000 7ff435690640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4301585d0 con 0x7ff43015ef40 2023-10-16T11:27:20.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.003+0000 7ff435690640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4300af8c0 con 0x7ff4300aef00 2023-10-16T11:27:20.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.003+0000 7ff435690640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4301607d0 con 0x7ff41c0039a0 2023-10-16T11:27:20.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.003+0000 7ff42f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff43015ef40 0x7ff43015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:20.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.003+0000 7ff42f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff43015ef40 0x7ff43015f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46808/0 (socket says 172.21.15.84:46808) 2023-10-16T11:27:20.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.003+0000 7ff42f7fe640 1 -- 172.21.15.84:0/3880187195 learned_addr learned my addr 172.21.15.84:0/3880187195 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:20.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.003+0000 7ff427fff640 1 --2- 172.21.15.84:0/3880187195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4300aef00 0x7ff4300af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:20.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.003+0000 7ff42effd640 1 --2- 172.21.15.84:0/3880187195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff41c0039a0 0x7ff4301f2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:20.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.003+0000 7ff42f7fe640 1 -- 172.21.15.84:0/3880187195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff41c0039a0 msgr2=0x7ff4301f2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:20.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.003+0000 7ff42f7fe640 1 --2- 172.21.15.84:0/3880187195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff41c0039a0 0x7ff4301f2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:20.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.003+0000 7ff42f7fe640 1 -- 172.21.15.84:0/3880187195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4300aef00 msgr2=0x7ff4300af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:20.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.003+0000 7ff42f7fe640 1 --2- 172.21.15.84:0/3880187195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4300aef00 0x7ff4300af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:20.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.003+0000 7ff42f7fe640 1 -- 172.21.15.84:0/3880187195 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43015f900 con 0x7ff43015ef40 2023-10-16T11:27:20.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.004+0000 7ff42f7fe640 1 --2- 172.21.15.84:0/3880187195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff43015ef40 0x7ff43015f3c0 secure :-1 s=READY pgs=3379 cs=0 l=1 rev1=1 crypto rx=0x7ff43015d7d0 tx=0x7ff42800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:20.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.004+0000 7ff4277fe640 1 -- 172.21.15.84:0/3880187195 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff428007c00 con 0x7ff43015ef40 2023-10-16T11:27:20.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.004+0000 7ff4277fe640 1 -- 172.21.15.84:0/3880187195 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42800ceb0 con 0x7ff43015ef40 2023-10-16T11:27:20.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.004+0000 7ff435690640 1 -- 172.21.15.84:0/3880187195 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43015fbc0 con 0x7ff43015ef40 2023-10-16T11:27:20.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.004+0000 7ff4277fe640 1 -- 172.21.15.84:0/3880187195 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42800a830 con 0x7ff43015ef40 2023-10-16T11:27:20.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.004+0000 7ff435690640 1 -- 172.21.15.84:0/3880187195 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff430160180 con 0x7ff43015ef40 2023-10-16T11:27:20.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.005+0000 7ff435690640 1 -- 172.21.15.84:0/3880187195 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4300b1550 con 0x7ff43015ef40 2023-10-16T11:27:20.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.006+0000 7ff4277fe640 1 -- 172.21.15.84:0/3880187195 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff42800a990 con 0x7ff43015ef40 2023-10-16T11:27:20.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.007+0000 7ff4277fe640 1 --2- 172.21.15.84:0/3880187195 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7ff41006af90 0x7ff41006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:20.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.007+0000 7ff4277fe640 1 -- 172.21.15.84:0/3880187195 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 4885+0+0 (secure 0 0 0) 0x7ff4280890d0 con 0x7ff43015ef40 2023-10-16T11:27:20.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.008+0000 7ff42effd640 1 --2- 172.21.15.84:0/3880187195 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7ff41006af90 0x7ff41006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:20.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.008+0000 7ff42effd640 1 --2- 172.21.15.84:0/3880187195 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7ff41006af90 0x7ff41006d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff420033000 tx=0x7ff420009030 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:20.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.012+0000 7ff4277fe640 1 -- 172.21.15.84:0/3880187195 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff4300b1550 con 0x7ff43015ef40 2023-10-16T11:27:20.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.207+0000 7ff435690640 1 -- 172.21.15.84:0/3880187195 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff4300b1550 con 0x7ff43015ef40 2023-10-16T11:27:20.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.208+0000 7ff4277fe640 1 -- 172.21.15.84:0/3880187195 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v266) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff4300b1550 con 0x7ff43015ef40 2023-10-16T11:27:20.211 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:20.211 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T11:27:20.211 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:20.211 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-16T11:27:20.211 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-16T11:27:20.212 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:20.212 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T11:27:20.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.211+0000 7ff435690640 1 -- 172.21.15.84:0/3880187195 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7ff41006af90 msgr2=0x7ff41006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:20.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.211+0000 7ff435690640 1 --2- 172.21.15.84:0/3880187195 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7ff41006af90 0x7ff41006d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff420033000 tx=0x7ff420009030 comp rx=0 tx=0).stop 2023-10-16T11:27:20.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.211+0000 7ff435690640 1 -- 172.21.15.84:0/3880187195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff43015ef40 msgr2=0x7ff43015f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:20.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.211+0000 7ff435690640 1 --2- 172.21.15.84:0/3880187195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff43015ef40 0x7ff43015f3c0 secure :-1 s=READY pgs=3379 cs=0 l=1 rev1=1 crypto rx=0x7ff43015d7d0 tx=0x7ff42800cc60 comp rx=0 tx=0).stop 2023-10-16T11:27:20.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.211+0000 7ff435690640 1 -- 172.21.15.84:0/3880187195 shutdown_connections 2023-10-16T11:27:20.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.211+0000 7ff435690640 1 --2- 172.21.15.84:0/3880187195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff43015ef40 0x7ff43015f3c0 unknown :-1 s=CLOSED pgs=3379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:20.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.211+0000 7ff435690640 1 --2- 172.21.15.84:0/3880187195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4300aef00 0x7ff4300af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:20.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.211+0000 7ff435690640 1 --2- 172.21.15.84:0/3880187195 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7ff41006af90 0x7ff41006d450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:20.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.211+0000 7ff435690640 1 --2- 172.21.15.84:0/3880187195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff41c0039a0 0x7ff4301f2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:20.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.211+0000 7ff435690640 1 -- 172.21.15.84:0/3880187195 >> 172.21.15.84:0/3880187195 conn(0x7ff4300cb410 msgr2=0x7ff4300c9420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:20.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.211+0000 7ff435690640 1 -- 172.21.15.84:0/3880187195 shutdown_connections 2023-10-16T11:27:20.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.211+0000 7ff435690640 1 -- 172.21.15.84:0/3880187195 wait complete. 2023-10-16T11:27:20.232 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T11:27:20.232 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T11:27:20.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.450+0000 7f5b97fff640 1 Processor -- start 2023-10-16T11:27:20.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.451+0000 7f5b97fff640 1 -- start start 2023-10-16T11:27:20.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.452+0000 7f5b97fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b980cd920 0x7f5b98167c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:20.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.452+0000 7f5b97fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b980cc480 con 0x7f5b980ac890 2023-10-16T11:27:20.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.452+0000 7f5b97fff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b980c9cc0 con 0x7f5b980c9880 2023-10-16T11:27:20.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.452+0000 7f5b97fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b980adfe0 con 0x7f5b980cd920 2023-10-16T11:27:20.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.452+0000 7f5b96ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f5b980ac890 0x7f5b980acc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55306/0 (socket says 172.21.15.84:55306) 2023-10-16T11:27:20.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.453+0000 7f5b96ffd640 1 -- 172.21.15.84:0/104393316 learned_addr learned my addr 172.21.15.84:0/104393316 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:20.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.453+0000 7f5b977fe640 1 --2- 172.21.15.84:0/104393316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b980cd920 0x7f5b98167c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:20.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.453+0000 7f5b967fc640 1 -- 172.21.15.84:0/104393316 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834191475 0 0) 0x7f5b980cc480 con 0x7f5b980ac890 2023-10-16T11:27:20.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.453+0000 7f5b967fc640 1 -- 172.21.15.84:0/104393316 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b84003520 con 0x7f5b980ac890 2023-10-16T11:27:20.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.453+0000 7f5b967fc640 1 -- 172.21.15.84:0/104393316 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3562162517 0 0) 0x7f5b84003520 con 0x7f5b980ac890 2023-10-16T11:27:20.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.453+0000 7f5b967fc640 1 -- 172.21.15.84:0/104393316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b980cd920 msgr2=0x7f5b98167c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:20.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.453+0000 7f5b967fc640 1 --2- 172.21.15.84:0/104393316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b980cd920 0x7f5b98167c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:20.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.453+0000 7f5b967fc640 1 -- 172.21.15.84:0/104393316 >> v1:172.21.15.157:6789/0 conn(0x7f5b980c9880 legacy=0x7f5b9815d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:20.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.453+0000 7f5b967fc640 1 -- 172.21.15.84:0/104393316 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9815d8c0 con 0x7f5b980ac890 2023-10-16T11:27:20.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.453+0000 7f5b967fc640 1 -- 172.21.15.84:0/104393316 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f5b88001670 con 0x7f5b980ac890 2023-10-16T11:27:20.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.453+0000 7f5b967fc640 1 -- 172.21.15.84:0/104393316 >> v1:172.21.15.84:6789/0 conn(0x7f5b980ac890 legacy=0x7f5b980acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:20.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.454+0000 7f5b967fc640 1 --2- 172.21.15.84:0/104393316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b84003a10 0x7f5b84003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:20.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.454+0000 7f5b967fc640 1 --2- 172.21.15.84:0/104393316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b980cd920 0x7f5b840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:20.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.454+0000 7f5b967fc640 1 --2- 172.21.15.84:0/104393316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b84004d00 0x7f5b84005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:20.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.454+0000 7f5b967fc640 1 -- 172.21.15.84:0/104393316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b980adfe0 con 0x7f5b84004d00 2023-10-16T11:27:20.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.454+0000 7f5b967fc640 1 -- 172.21.15.84:0/104393316 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8400bac0 con 0x7f5b84003a10 2023-10-16T11:27:20.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.454+0000 7f5b967fc640 1 -- 172.21.15.84:0/104393316 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8400bc80 con 0x7f5b980cd920 2023-10-16T11:27:20.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.454+0000 7f5b977fe640 1 --2- 172.21.15.84:0/104393316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b84004d00 0x7f5b84005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:20.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.454+0000 7f5b977fe640 1 -- 172.21.15.84:0/104393316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b980cd920 msgr2=0x7f5b840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:20.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.454+0000 7f5b977fe640 1 --2- 172.21.15.84:0/104393316 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b980cd920 0x7f5b840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:20.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.454+0000 7f5b977fe640 1 -- 172.21.15.84:0/104393316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b84003a10 msgr2=0x7f5b84003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:20.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.454+0000 7f5b977fe640 1 --2- 172.21.15.84:0/104393316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b84003a10 0x7f5b84003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:20.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.454+0000 7f5b977fe640 1 -- 172.21.15.84:0/104393316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b84003520 con 0x7f5b84004d00 2023-10-16T11:27:20.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.455+0000 7f5b977fe640 1 --2- 172.21.15.84:0/104393316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b84004d00 0x7f5b84005180 secure :-1 s=READY pgs=3381 cs=0 l=1 rev1=1 crypto rx=0x7f5b980cddb0 tx=0x7f5b840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a28df871a5433a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:20.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.456+0000 7f5b967fc640 1 -- 172.21.15.84:0/104393316 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9002cbd0 con 0x7f5b84004d00 2023-10-16T11:27:20.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.456+0000 7f5b967fc640 1 -- 172.21.15.84:0/104393316 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9000f040 con 0x7f5b84004d00 2023-10-16T11:27:20.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.456+0000 7f5b967fc640 1 -- 172.21.15.84:0/104393316 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90034440 con 0x7f5b84004d00 2023-10-16T11:27:20.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.456+0000 7f5b97fff640 1 -- 172.21.15.84:0/104393316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b84004d00 msgr2=0x7f5b84005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:20.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.456+0000 7f5b97fff640 1 --2- 172.21.15.84:0/104393316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b84004d00 0x7f5b84005180 secure :-1 s=READY pgs=3381 cs=0 l=1 rev1=1 crypto rx=0x7f5b980cddb0 tx=0x7f5b840057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:20.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.456+0000 7f5b977fe640 1 -- 172.21.15.84:0/104393316 reap_dead start 2023-10-16T11:27:20.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.456+0000 7f5b97fff640 1 -- 172.21.15.84:0/104393316 shutdown_connections 2023-10-16T11:27:20.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.456+0000 7f5b97fff640 1 -- 172.21.15.84:0/104393316 >> 172.21.15.84:0/104393316 conn(0x7f5b980ce820 msgr2=0x7f5b980cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:20.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.456+0000 7f5b97fff640 1 -- 172.21.15.84:0/104393316 shutdown_connections 2023-10-16T11:27:20.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.456+0000 7f5b97fff640 1 -- 172.21.15.84:0/104393316 wait complete. 2023-10-16T11:27:20.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.458+0000 7f5b97fff640 1 Processor -- start 2023-10-16T11:27:20.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.458+0000 7f5b97fff640 1 -- start start 2023-10-16T11:27:20.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.459+0000 7f5b97fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b84004d00 0x7f5b981568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:20.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.459+0000 7f5b97fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b980c6890 0x7f5b980c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:20.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.459+0000 7f5b97fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b98159c90 0x7f5b9815a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:20.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.459+0000 7f5b97fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b98156e30 con 0x7f5b980c6890 2023-10-16T11:27:20.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.459+0000 7f5b97fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b980b2620 con 0x7f5b84004d00 2023-10-16T11:27:20.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.459+0000 7f5b97fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b980b28c0 con 0x7f5b98159c90 2023-10-16T11:27:20.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.459+0000 7f5b8ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b980c6890 0x7f5b980c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:20.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.460+0000 7f5b8ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b980c6890 0x7f5b980c6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46830/0 (socket says 172.21.15.84:46830) 2023-10-16T11:27:20.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.460+0000 7f5b8ffff640 1 -- 172.21.15.84:0/3564774060 learned_addr learned my addr 172.21.15.84:0/3564774060 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:20.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.460+0000 7f5b96ffd640 1 --2- 172.21.15.84:0/3564774060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b84004d00 0x7f5b981568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:20.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.460+0000 7f5b8ffff640 1 -- 172.21.15.84:0/3564774060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b98159c90 msgr2=0x7f5b9815a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:20.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.460+0000 7f5b8ffff640 1 --2- 172.21.15.84:0/3564774060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b98159c90 0x7f5b9815a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:20.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.460+0000 7f5b8ffff640 1 -- 172.21.15.84:0/3564774060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b84004d00 msgr2=0x7f5b981568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:20.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.460+0000 7f5b8ffff640 1 --2- 172.21.15.84:0/3564774060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b84004d00 0x7f5b981568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:20.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.460+0000 7f5b8ffff640 1 -- 172.21.15.84:0/3564774060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b980b2ba0 con 0x7f5b980c6890 2023-10-16T11:27:20.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.460+0000 7f5b8ffff640 1 --2- 172.21.15.84:0/3564774060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b980c6890 0x7f5b980c6d10 secure :-1 s=READY pgs=3382 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000c450 tx=0x7f5b8000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:20.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.460+0000 7f5b8f7fe640 1 -- 172.21.15.84:0/3564774060 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b800052d0 con 0x7f5b980c6890 2023-10-16T11:27:20.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.460+0000 7f5b8f7fe640 1 -- 172.21.15.84:0/3564774060 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b80005430 con 0x7f5b980c6890 2023-10-16T11:27:20.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.460+0000 7f5b97fff640 1 -- 172.21.15.84:0/3564774060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b980b2e80 con 0x7f5b980c6890 2023-10-16T11:27:20.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.460+0000 7f5b8f7fe640 1 -- 172.21.15.84:0/3564774060 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80012870 con 0x7f5b980c6890 2023-10-16T11:27:20.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.461+0000 7f5b97fff640 1 -- 172.21.15.84:0/3564774060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b981f6fa0 con 0x7f5b980c6890 2023-10-16T11:27:20.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.462+0000 7f5b97fff640 1 -- 172.21.15.84:0/3564774060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9804e9b0 con 0x7f5b980c6890 2023-10-16T11:27:20.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.463+0000 7f5b8f7fe640 1 -- 172.21.15.84:0/3564774060 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5b800129d0 con 0x7f5b980c6890 2023-10-16T11:27:20.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.464+0000 7f5b8f7fe640 1 --2- 172.21.15.84:0/3564774060 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f5b7806af90 0x7f5b7806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:20.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.464+0000 7f5b8f7fe640 1 -- 172.21.15.84:0/3564774060 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f5b80088f00 con 0x7f5b980c6890 2023-10-16T11:27:20.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.464+0000 7f5b96ffd640 1 --2- 172.21.15.84:0/3564774060 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f5b7806af90 0x7f5b7806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:20.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.465+0000 7f5b96ffd640 1 --2- 172.21.15.84:0/3564774060 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f5b7806af90 0x7f5b7806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5b880079b0 tx=0x7f5b88009040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:20.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.468+0000 7f5b8f7fe640 1 -- 172.21.15.84:0/3564774060 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5b80054260 con 0x7f5b980c6890 2023-10-16T11:27:20.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:20.657+0000 7f5b97fff640 1 -- 172.21.15.84:0/3564774060 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5b980ce1f0 con 0x7f5b980c6890 2023-10-16T11:27:21.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.394+0000 7f5b8f7fe640 1 -- 172.21.15.84:0/3564774060 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v267) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5b80057880 con 0x7f5b980c6890 2023-10-16T11:27:21.397 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-16T11:27:21.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.398+0000 7f5b97fff640 1 -- 172.21.15.84:0/3564774060 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f5b7806af90 msgr2=0x7f5b7806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:21.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.398+0000 7f5b97fff640 1 --2- 172.21.15.84:0/3564774060 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f5b7806af90 0x7f5b7806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5b880079b0 tx=0x7f5b88009040 comp rx=0 tx=0).stop 2023-10-16T11:27:21.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.398+0000 7f5b97fff640 1 -- 172.21.15.84:0/3564774060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b980c6890 msgr2=0x7f5b980c6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:21.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.398+0000 7f5b97fff640 1 --2- 172.21.15.84:0/3564774060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b980c6890 0x7f5b980c6d10 secure :-1 s=READY pgs=3382 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000c450 tx=0x7f5b8000c910 comp rx=0 tx=0).stop 2023-10-16T11:27:21.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.398+0000 7f5b97fff640 1 -- 172.21.15.84:0/3564774060 shutdown_connections 2023-10-16T11:27:21.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.398+0000 7f5b97fff640 1 --2- 172.21.15.84:0/3564774060 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f5b7806af90 0x7f5b7806d450 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5b880079b0 tx=0x7f5b88009040 comp rx=0 tx=0).stop 2023-10-16T11:27:21.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.398+0000 7f5b97fff640 1 --2- 172.21.15.84:0/3564774060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b98159c90 0x7f5b9815a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:21.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.398+0000 7f5b97fff640 1 --2- 172.21.15.84:0/3564774060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b980c6890 0x7f5b980c6d10 unknown :-1 s=CLOSED pgs=3382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:21.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.398+0000 7f5b97fff640 1 --2- 172.21.15.84:0/3564774060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b84004d00 0x7f5b981568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:21.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.398+0000 7f5b97fff640 1 -- 172.21.15.84:0/3564774060 >> 172.21.15.84:0/3564774060 conn(0x7f5b980ce820 msgr2=0x7f5b98158d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:21.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.402+0000 7f5b97fff640 1 -- 172.21.15.84:0/3564774060 shutdown_connections 2023-10-16T11:27:21.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.402+0000 7f5b97fff640 1 -- 172.21.15.84:0/3564774060 wait complete. 2023-10-16T11:27:21.428 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T11:27:21.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.633+0000 7f5f07c75640 1 Processor -- start 2023-10-16T11:27:21.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.633+0000 7f5f07c75640 1 -- start start 2023-10-16T11:27:21.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.635+0000 7f5f07c75640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f000b3070 0x7f5f000cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:21.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.635+0000 7f5f07c75640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5f000d1b10 0x7f5f000cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:21.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.635+0000 7f5f07c75640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f000b3940 con 0x7f5f000b3e70 2023-10-16T11:27:21.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.635+0000 7f5f07c75640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f000b45e0 con 0x7f5f000d1b10 2023-10-16T11:27:21.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.635+0000 7f5f07c75640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000b3b90 con 0x7f5f000b3070 2023-10-16T11:27:21.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.635+0000 7f5f05210640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f5f000b3e70 0x7f5f000c6870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55312/0 (socket says 172.21.15.84:55312) 2023-10-16T11:27:21.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.635+0000 7f5f05210640 1 -- 172.21.15.84:0/382651516 learned_addr learned my addr 172.21.15.84:0/382651516 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:21.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.636+0000 7f5f05a11640 1 --2- 172.21.15.84:0/382651516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f000b3070 0x7f5f000cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:21.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.636+0000 7f5f06212640 1 --2- 172.21.15.84:0/382651516 >> v2:172.21.15.157:3300/0 conn(0x7f5f000d1b10 0x7f5f000cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:21.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.636+0000 7f5f04a0f640 1 -- 172.21.15.84:0/382651516 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2136993691 0 0) 0x7f5f000b3940 con 0x7f5f000b3e70 2023-10-16T11:27:21.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.636+0000 7f5f04a0f640 1 -- 172.21.15.84:0/382651516 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5eec003520 con 0x7f5f000b3e70 2023-10-16T11:27:21.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.636+0000 7f5f04a0f640 1 -- 172.21.15.84:0/382651516 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1142965837 0 0) 0x7f5eec003520 con 0x7f5f000b3e70 2023-10-16T11:27:21.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.636+0000 7f5f04a0f640 1 -- 172.21.15.84:0/382651516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f000b3070 msgr2=0x7f5f000cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:21.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.636+0000 7f5f04a0f640 1 --2- 172.21.15.84:0/382651516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f000b3070 0x7f5f000cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:21.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.636+0000 7f5f04a0f640 1 -- 172.21.15.84:0/382651516 >> v2:172.21.15.157:3300/0 conn(0x7f5f000d1b10 msgr2=0x7f5f000cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:21.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.636+0000 7f5f04a0f640 1 --2- 172.21.15.84:0/382651516 >> v2:172.21.15.157:3300/0 conn(0x7f5f000d1b10 0x7f5f000cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:21.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.636+0000 7f5f04a0f640 1 -- 172.21.15.84:0/382651516 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f000d3840 con 0x7f5f000b3e70 2023-10-16T11:27:21.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.636+0000 7f5f04a0f640 1 -- 172.21.15.84:0/382651516 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f5ef0001670 con 0x7f5f000b3e70 2023-10-16T11:27:21.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.637+0000 7f5f04a0f640 1 -- 172.21.15.84:0/382651516 >> v1:172.21.15.84:6789/0 conn(0x7f5f000b3e70 legacy=0x7f5f000c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:21.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.637+0000 7f5f04a0f640 1 --2- 172.21.15.84:0/382651516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5eec003970 0x7f5eec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:21.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.637+0000 7f5f04a0f640 1 --2- 172.21.15.84:0/382651516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f000b3070 0x7f5eec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:21.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.637+0000 7f5f04a0f640 1 --2- 172.21.15.84:0/382651516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5eec004d00 0x7f5eec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:21.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.637+0000 7f5f04a0f640 1 -- 172.21.15.84:0/382651516 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000b45e0 con 0x7f5eec004d00 2023-10-16T11:27:21.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.637+0000 7f5f04a0f640 1 -- 172.21.15.84:0/382651516 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000b3b90 con 0x7f5eec003970 2023-10-16T11:27:21.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.637+0000 7f5f04a0f640 1 -- 172.21.15.84:0/382651516 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec00b980 con 0x7f5f000b3070 2023-10-16T11:27:21.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.637+0000 7f5f06212640 1 --2- 172.21.15.84:0/382651516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5eec004d00 0x7f5eec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:21.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.637+0000 7f5f05210640 1 --2- 172.21.15.84:0/382651516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f000b3070 0x7f5eec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:21.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.637+0000 7f5f05a11640 1 --2- 172.21.15.84:0/382651516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5eec003970 0x7f5eec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:21.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.637+0000 7f5f06212640 1 -- 172.21.15.84:0/382651516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f000b3070 msgr2=0x7f5eec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:21.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.637+0000 7f5f06212640 1 --2- 172.21.15.84:0/382651516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f000b3070 0x7f5eec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:21.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.637+0000 7f5f06212640 1 -- 172.21.15.84:0/382651516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5eec003970 msgr2=0x7f5eec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:21.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.637+0000 7f5f06212640 1 --2- 172.21.15.84:0/382651516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5eec003970 0x7f5eec003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:21.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.637+0000 7f5f06212640 1 -- 172.21.15.84:0/382651516 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec003520 con 0x7f5eec004d00 2023-10-16T11:27:21.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.637+0000 7f5f05a11640 1 --2- 172.21.15.84:0/382651516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5eec003970 0x7f5eec003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:21.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.637+0000 7f5f05210640 1 --2- 172.21.15.84:0/382651516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f000b3070 0x7f5eec0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:21.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.638+0000 7f5f06212640 1 --2- 172.21.15.84:0/382651516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5eec004d00 0x7f5eec005180 secure :-1 s=READY pgs=3384 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009fc0 tx=0x7f5eec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d141d6172efb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:21.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.639+0000 7f5f04a0f640 1 -- 172.21.15.84:0/382651516 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef802cc60 con 0x7f5eec004d00 2023-10-16T11:27:21.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.639+0000 7f5f04a0f640 1 -- 172.21.15.84:0/382651516 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef800f040 con 0x7f5eec004d00 2023-10-16T11:27:21.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.639+0000 7f5f04a0f640 1 -- 172.21.15.84:0/382651516 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8034440 con 0x7f5eec004d00 2023-10-16T11:27:21.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.639+0000 7f5f07c75640 1 -- 172.21.15.84:0/382651516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5eec004d00 msgr2=0x7f5eec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:21.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.639+0000 7f5f07c75640 1 --2- 172.21.15.84:0/382651516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5eec004d00 0x7f5eec005180 secure :-1 s=READY pgs=3384 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009fc0 tx=0x7f5eec0057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:21.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.639+0000 7f5f06212640 1 -- 172.21.15.84:0/382651516 reap_dead start 2023-10-16T11:27:21.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.639+0000 7f5f07c75640 1 -- 172.21.15.84:0/382651516 shutdown_connections 2023-10-16T11:27:21.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.639+0000 7f5f07c75640 1 -- 172.21.15.84:0/382651516 >> 172.21.15.84:0/382651516 conn(0x7f5f000d2d20 msgr2=0x7f5f000d3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:21.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.639+0000 7f5f07c75640 1 -- 172.21.15.84:0/382651516 shutdown_connections 2023-10-16T11:27:21.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.640+0000 7f5f07c75640 1 -- 172.21.15.84:0/382651516 wait complete. 2023-10-16T11:27:21.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.641+0000 7f5f07c75640 1 Processor -- start 2023-10-16T11:27:21.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.641+0000 7f5f07c75640 1 -- start start 2023-10-16T11:27:21.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.642+0000 7f5f07c75640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5eec004d00 0x7f5f000af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:21.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.642+0000 7f5f07c75640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f0015a120 0x7f5f000afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:21.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.642+0000 7f5f07c75640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f0015a900 0x7f5f0015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:21.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.642+0000 7f5f07c75640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0015b8b0 con 0x7f5f0015a900 2023-10-16T11:27:21.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.642+0000 7f5f07c75640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000ca3e0 con 0x7f5f0015a120 2023-10-16T11:27:21.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.642+0000 7f5f07c75640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000ca6c0 con 0x7f5eec004d00 2023-10-16T11:27:21.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.643+0000 7f5f06212640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f0015a900 0x7f5f0015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:21.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.643+0000 7f5f06212640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f0015a900 0x7f5f0015adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46848/0 (socket says 172.21.15.84:46848) 2023-10-16T11:27:21.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.643+0000 7f5f06212640 1 -- 172.21.15.84:0/2604315109 learned_addr learned my addr 172.21.15.84:0/2604315109 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:21.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.643+0000 7f5f05a11640 1 --2- 172.21.15.84:0/2604315109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5eec004d00 0x7f5f000af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:21.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.643+0000 7f5f05210640 1 --2- 172.21.15.84:0/2604315109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f0015a120 0x7f5f000afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:21.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.643+0000 7f5f06212640 1 -- 172.21.15.84:0/2604315109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5eec004d00 msgr2=0x7f5f000af8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:21.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.643+0000 7f5f06212640 1 --2- 172.21.15.84:0/2604315109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5eec004d00 0x7f5f000af8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:21.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.643+0000 7f5f06212640 1 -- 172.21.15.84:0/2604315109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f0015a120 msgr2=0x7f5f000afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:21.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.643+0000 7f5f06212640 1 --2- 172.21.15.84:0/2604315109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f0015a120 0x7f5f000afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:21.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.643+0000 7f5f06212640 1 -- 172.21.15.84:0/2604315109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f000ca9a0 con 0x7f5f0015a900 2023-10-16T11:27:21.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.643+0000 7f5f05210640 1 --2- 172.21.15.84:0/2604315109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f0015a120 0x7f5f000afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:21.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.644+0000 7f5f06212640 1 --2- 172.21.15.84:0/2604315109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f0015a900 0x7f5f0015adb0 secure :-1 s=READY pgs=3385 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8037040 tx=0x7f5ef8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:21.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.644+0000 7f5ef67fc640 1 -- 172.21.15.84:0/2604315109 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8043070 con 0x7f5f0015a900 2023-10-16T11:27:21.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.644+0000 7f5ef67fc640 1 -- 172.21.15.84:0/2604315109 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef802ce50 con 0x7f5f0015a900 2023-10-16T11:27:21.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.644+0000 7f5ef67fc640 1 -- 172.21.15.84:0/2604315109 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8034d90 con 0x7f5f0015a900 2023-10-16T11:27:21.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.644+0000 7f5f07c75640 1 -- 172.21.15.84:0/2604315109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f000cac80 con 0x7f5f0015a900 2023-10-16T11:27:21.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.644+0000 7f5f07c75640 1 -- 172.21.15.84:0/2604315109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0015b4d0 con 0x7f5f0015a900 2023-10-16T11:27:21.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.645+0000 7f5f07c75640 1 -- 172.21.15.84:0/2604315109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f000cd050 con 0x7f5f0015a900 2023-10-16T11:27:21.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.646+0000 7f5ef67fc640 1 -- 172.21.15.84:0/2604315109 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5ef803e440 con 0x7f5f0015a900 2023-10-16T11:27:21.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.647+0000 7f5ef67fc640 1 --2- 172.21.15.84:0/2604315109 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f5ee406afc0 0x7f5ee406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:21.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.647+0000 7f5ef67fc640 1 -- 172.21.15.84:0/2604315109 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(267..267 src has 1..267) v4 ==== 5255+0+0 (secure 0 0 0) 0x7f5ef80aaf40 con 0x7f5f0015a900 2023-10-16T11:27:21.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.647+0000 7f5f05a11640 1 --2- 172.21.15.84:0/2604315109 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f5ee406afc0 0x7f5ee406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:21.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.648+0000 7f5f05a11640 1 --2- 172.21.15.84:0/2604315109 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f5ee406afc0 0x7f5ee406d480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009a90 tx=0x7f5efc008040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:21.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.652+0000 7f5ef67fc640 1 -- 172.21.15.84:0/2604315109 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5ef8076130 con 0x7f5f0015a900 2023-10-16T11:27:21.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:21.844+0000 7f5f07c75640 1 -- 172.21.15.84:0/2604315109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5f0004e160 con 0x7f5f0015a900 2023-10-16T11:27:22.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.401+0000 7f5ef67fc640 1 -- 172.21.15.84:0/2604315109 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v268) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5ef8079750 con 0x7f5f0015a900 2023-10-16T11:27:22.403 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-16T11:27:22.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.406+0000 7f5f07c75640 1 -- 172.21.15.84:0/2604315109 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f5ee406afc0 msgr2=0x7f5ee406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:22.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.406+0000 7f5f07c75640 1 --2- 172.21.15.84:0/2604315109 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f5ee406afc0 0x7f5ee406d480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009a90 tx=0x7f5efc008040 comp rx=0 tx=0).stop 2023-10-16T11:27:22.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.406+0000 7f5f07c75640 1 -- 172.21.15.84:0/2604315109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f0015a900 msgr2=0x7f5f0015adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:22.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.406+0000 7f5f07c75640 1 --2- 172.21.15.84:0/2604315109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f0015a900 0x7f5f0015adb0 secure :-1 s=READY pgs=3385 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8037040 tx=0x7f5ef8009750 comp rx=0 tx=0).stop 2023-10-16T11:27:22.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.406+0000 7f5f07c75640 1 -- 172.21.15.84:0/2604315109 shutdown_connections 2023-10-16T11:27:22.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.406+0000 7f5f07c75640 1 --2- 172.21.15.84:0/2604315109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f0015a900 0x7f5f0015adb0 unknown :-1 s=CLOSED pgs=3385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:22.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.406+0000 7f5f07c75640 1 --2- 172.21.15.84:0/2604315109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f0015a120 0x7f5f000afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:22.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.406+0000 7f5f07c75640 1 --2- 172.21.15.84:0/2604315109 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f5ee406afc0 0x7f5ee406d480 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:22.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.406+0000 7f5f07c75640 1 --2- 172.21.15.84:0/2604315109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5eec004d00 0x7f5f000af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:22.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.406+0000 7f5f07c75640 1 -- 172.21.15.84:0/2604315109 >> 172.21.15.84:0/2604315109 conn(0x7f5f000d2d20 msgr2=0x7f5f0015cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:22.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.409+0000 7f5f07c75640 1 -- 172.21.15.84:0/2604315109 shutdown_connections 2023-10-16T11:27:22.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.409+0000 7f5f07c75640 1 -- 172.21.15.84:0/2604315109 wait complete. 2023-10-16T11:27:22.445 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T11:27:22.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.629+0000 7fcf70e6d640 1 Processor -- start 2023-10-16T11:27:22.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.629+0000 7fcf70e6d640 1 -- start start 2023-10-16T11:27:22.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.630+0000 7fcf70e6d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcf6c06c040 0x7fcf6c06c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:22.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.630+0000 7fcf70e6d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf6c0ad970 0x7fcf6c15e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:22.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.630+0000 7fcf70e6d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf6c06c960 con 0x7fcf6c15ebc0 2023-10-16T11:27:22.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.630+0000 7fcf70e6d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf6c15b9d0 con 0x7fcf6c06c040 2023-10-16T11:27:22.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.630+0000 7fcf70e6d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c15bcb0 con 0x7fcf6c0ad970 2023-10-16T11:27:22.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.631+0000 7fcf6b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcf6c06c040 0x7fcf6c06c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:22.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.631+0000 7fcf6b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcf6c06c040 0x7fcf6c06c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46850/0 (socket says 172.21.15.84:46850) 2023-10-16T11:27:22.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.631+0000 7fcf6b7fe640 1 -- 172.21.15.84:0/3027080132 learned_addr learned my addr 172.21.15.84:0/3027080132 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:22.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.631+0000 7fcf63fff640 1 --2- 172.21.15.84:0/3027080132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf6c0ad970 0x7fcf6c15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:22.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.631+0000 7fcf6b7fe640 1 -- 172.21.15.84:0/3027080132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf6c0ad970 msgr2=0x7fcf6c15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:22.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.631+0000 7fcf6b7fe640 1 --2- 172.21.15.84:0/3027080132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf6c0ad970 0x7fcf6c15e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:22.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.631+0000 7fcf6b7fe640 1 -- 172.21.15.84:0/3027080132 >> v1:172.21.15.157:6789/0 conn(0x7fcf6c15ebc0 legacy=0x7fcf6c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:22.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.631+0000 7fcf6b7fe640 1 -- 172.21.15.84:0/3027080132 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c15bf90 con 0x7fcf6c06c040 2023-10-16T11:27:22.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.631+0000 7fcf63fff640 1 --2- 172.21.15.84:0/3027080132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf6c0ad970 0x7fcf6c15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:22.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.632+0000 7fcf6b7fe640 1 --2- 172.21.15.84:0/3027080132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf6c06c040 0x7fcf6c06c420 secure :-1 s=READY pgs=3386 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0098a0 tx=0x7fcf5c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95dda11e3b859396 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:22.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.632+0000 7fcf6affd640 1 -- 172.21.15.84:0/3027080132 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c031640 con 0x7fcf6c06c040 2023-10-16T11:27:22.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.633+0000 7fcf6affd640 1 -- 172.21.15.84:0/3027080132 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5c0317a0 con 0x7fcf6c06c040 2023-10-16T11:27:22.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.633+0000 7fcf6affd640 1 -- 172.21.15.84:0/3027080132 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c031b30 con 0x7fcf6c06c040 2023-10-16T11:27:22.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.633+0000 7fcf70e6d640 1 -- 172.21.15.84:0/3027080132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf6c06c040 msgr2=0x7fcf6c06c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:22.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.633+0000 7fcf70e6d640 1 --2- 172.21.15.84:0/3027080132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf6c06c040 0x7fcf6c06c420 secure :-1 s=READY pgs=3386 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0098a0 tx=0x7fcf5c02b8f0 comp rx=0 tx=0).stop 2023-10-16T11:27:22.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.633+0000 7fcf70e6d640 1 -- 172.21.15.84:0/3027080132 shutdown_connections 2023-10-16T11:27:22.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.633+0000 7fcf70e6d640 1 --2- 172.21.15.84:0/3027080132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf6c0ad970 0x7fcf6c15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:22.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.633+0000 7fcf70e6d640 1 --2- 172.21.15.84:0/3027080132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf6c06c040 0x7fcf6c06c420 unknown :-1 s=CLOSED pgs=3386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:22.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.633+0000 7fcf70e6d640 1 -- 172.21.15.84:0/3027080132 >> 172.21.15.84:0/3027080132 conn(0x7fcf6c0ce6a0 msgr2=0x7fcf6c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:22.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.633+0000 7fcf70e6d640 1 -- 172.21.15.84:0/3027080132 shutdown_connections 2023-10-16T11:27:22.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.633+0000 7fcf70e6d640 1 -- 172.21.15.84:0/3027080132 wait complete. 2023-10-16T11:27:22.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.635+0000 7fcf70e6d640 1 Processor -- start 2023-10-16T11:27:22.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.635+0000 7fcf70e6d640 1 -- start start 2023-10-16T11:27:22.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.636+0000 7fcf70e6d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf6c06c040 0x7fcf6c0afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:22.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.636+0000 7fcf70e6d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf6c0ad970 0x7fcf6c0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:22.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.636+0000 7fcf70e6d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf6c15ebc0 0x7fcf6c0acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:22.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.636+0000 7fcf70e6d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c0c5770 con 0x7fcf6c15ebc0 2023-10-16T11:27:22.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.636+0000 7fcf70e6d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c0c6010 con 0x7fcf6c0ad970 2023-10-16T11:27:22.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.636+0000 7fcf70e6d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c0b0440 con 0x7fcf6c06c040 2023-10-16T11:27:22.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.636+0000 7fcf6bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf6c15ebc0 0x7fcf6c0acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:22.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.637+0000 7fcf6bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf6c15ebc0 0x7fcf6c0acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46858/0 (socket says 172.21.15.84:46858) 2023-10-16T11:27:22.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.637+0000 7fcf6bfff640 1 -- 172.21.15.84:0/2444663541 learned_addr learned my addr 172.21.15.84:0/2444663541 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:22.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.637+0000 7fcf6b7fe640 1 --2- 172.21.15.84:0/2444663541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf6c06c040 0x7fcf6c0afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:22.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.637+0000 7fcf63fff640 1 --2- 172.21.15.84:0/2444663541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf6c0ad970 0x7fcf6c0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:22.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.637+0000 7fcf6bfff640 1 -- 172.21.15.84:0/2444663541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf6c06c040 msgr2=0x7fcf6c0afa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:22.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.637+0000 7fcf6bfff640 1 --2- 172.21.15.84:0/2444663541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf6c06c040 0x7fcf6c0afa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:22.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.637+0000 7fcf6bfff640 1 -- 172.21.15.84:0/2444663541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf6c0ad970 msgr2=0x7fcf6c0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:22.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.637+0000 7fcf6bfff640 1 --2- 172.21.15.84:0/2444663541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf6c0ad970 0x7fcf6c0ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:22.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.637+0000 7fcf6bfff640 1 -- 172.21.15.84:0/2444663541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c0094d0 con 0x7fcf6c15ebc0 2023-10-16T11:27:22.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.637+0000 7fcf63fff640 1 --2- 172.21.15.84:0/2444663541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf6c0ad970 0x7fcf6c0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:22.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.637+0000 7fcf6bfff640 1 --2- 172.21.15.84:0/2444663541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf6c15ebc0 0x7fcf6c0acdd0 secure :-1 s=READY pgs=3387 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400c780 tx=0x7fcf6400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:22.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.638+0000 7fcf68ff9640 1 -- 172.21.15.84:0/2444663541 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64004b60 con 0x7fcf6c15ebc0 2023-10-16T11:27:22.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.638+0000 7fcf68ff9640 1 -- 172.21.15.84:0/2444663541 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6400ce90 con 0x7fcf6c15ebc0 2023-10-16T11:27:22.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.638+0000 7fcf70e6d640 1 -- 172.21.15.84:0/2444663541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6c0c59c0 con 0x7fcf6c15ebc0 2023-10-16T11:27:22.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.638+0000 7fcf68ff9640 1 -- 172.21.15.84:0/2444663541 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6400a850 con 0x7fcf6c15ebc0 2023-10-16T11:27:22.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.638+0000 7fcf70e6d640 1 -- 172.21.15.84:0/2444663541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf6c0ca6b0 con 0x7fcf6c15ebc0 2023-10-16T11:27:22.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.639+0000 7fcf70e6d640 1 -- 172.21.15.84:0/2444663541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf6c06b330 con 0x7fcf6c15ebc0 2023-10-16T11:27:22.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.641+0000 7fcf68ff9640 1 -- 172.21.15.84:0/2444663541 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcf64006b50 con 0x7fcf6c15ebc0 2023-10-16T11:27:22.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.642+0000 7fcf68ff9640 1 --2- 172.21.15.84:0/2444663541 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fcf4c06afc0 0x7fcf4c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:22.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.642+0000 7fcf68ff9640 1 -- 172.21.15.84:0/2444663541 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 5637+0+0 (secure 0 0 0) 0x7fcf64089fd0 con 0x7fcf6c15ebc0 2023-10-16T11:27:22.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.642+0000 7fcf6b7fe640 1 --2- 172.21.15.84:0/2444663541 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fcf4c06afc0 0x7fcf4c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:22.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.643+0000 7fcf6b7fe640 1 --2- 172.21.15.84:0/2444663541 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fcf4c06afc0 0x7fcf4c06d480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c036000 tx=0x7fcf5c03c040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:22.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.646+0000 7fcf68ff9640 1 -- 172.21.15.84:0/2444663541 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcf64055090 con 0x7fcf6c15ebc0 2023-10-16T11:27:22.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:22.868+0000 7fcf70e6d640 1 -- 172.21.15.84:0/2444663541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fcf6c0c6180 con 0x7fcf6c15ebc0 2023-10-16T11:27:23.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.458+0000 7fcf68ff9640 1 -- 172.21.15.84:0/2444663541 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 36 and data pool 37 v153) v1 ==== 174+0+0 (secure 0 0 0) 0x7fcf640586b0 con 0x7fcf6c15ebc0 2023-10-16T11:27:23.465 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 36 and data pool 37 2023-10-16T11:27:23.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.467+0000 7fcf70e6d640 1 -- 172.21.15.84:0/2444663541 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fcf4c06afc0 msgr2=0x7fcf4c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:23.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.467+0000 7fcf70e6d640 1 --2- 172.21.15.84:0/2444663541 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fcf4c06afc0 0x7fcf4c06d480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c036000 tx=0x7fcf5c03c040 comp rx=0 tx=0).stop 2023-10-16T11:27:23.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.467+0000 7fcf70e6d640 1 -- 172.21.15.84:0/2444663541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf6c15ebc0 msgr2=0x7fcf6c0acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:23.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.467+0000 7fcf70e6d640 1 --2- 172.21.15.84:0/2444663541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf6c15ebc0 0x7fcf6c0acdd0 secure :-1 s=READY pgs=3387 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400c780 tx=0x7fcf6400cc40 comp rx=0 tx=0).stop 2023-10-16T11:27:23.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.472+0000 7fcf70e6d640 1 -- 172.21.15.84:0/2444663541 shutdown_connections 2023-10-16T11:27:23.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.472+0000 7fcf70e6d640 1 --2- 172.21.15.84:0/2444663541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf6c15ebc0 0x7fcf6c0acdd0 unknown :-1 s=CLOSED pgs=3387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:23.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.472+0000 7fcf70e6d640 1 --2- 172.21.15.84:0/2444663541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf6c0ad970 0x7fcf6c0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:23.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.472+0000 7fcf70e6d640 1 --2- 172.21.15.84:0/2444663541 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fcf4c06afc0 0x7fcf4c06d480 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:23.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.472+0000 7fcf70e6d640 1 --2- 172.21.15.84:0/2444663541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf6c06c040 0x7fcf6c0afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:23.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.472+0000 7fcf70e6d640 1 -- 172.21.15.84:0/2444663541 >> 172.21.15.84:0/2444663541 conn(0x7fcf6c0ce6a0 msgr2=0x7fcf6c15f010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:23.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.474+0000 7fcf70e6d640 1 -- 172.21.15.84:0/2444663541 shutdown_connections 2023-10-16T11:27:23.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.474+0000 7fcf70e6d640 1 -- 172.21.15.84:0/2444663541 wait complete. 2023-10-16T11:27:23.501 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:27:23.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.674+0000 7fd678e4b640 1 Processor -- start 2023-10-16T11:27:23.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.674+0000 7fd678e4b640 1 -- start start 2023-10-16T11:27:23.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.675+0000 7fd678e4b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd674159b20 0x7fd67406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:23.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.676+0000 7fd678e4b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67406f7f0 con 0x7fd6740b0b20 2023-10-16T11:27:23.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.676+0000 7fd678e4b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67406c5e0 con 0x7fd674159f00 2023-10-16T11:27:23.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.676+0000 7fd678e4b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67406c8c0 con 0x7fd674159b20 2023-10-16T11:27:23.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.676+0000 7fd672d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd6740b0b20 0x7fd67406fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55324/0 (socket says 172.21.15.84:55324) 2023-10-16T11:27:23.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.676+0000 7fd672d9d640 1 -- 172.21.15.84:0/4265152880 learned_addr learned my addr 172.21.15.84:0/4265152880 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:23.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.676+0000 7fd67259c640 1 --2- 172.21.15.84:0/4265152880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd674159b20 0x7fd67406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:23.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.676+0000 7fd671d9b640 1 -- 172.21.15.84:0/4265152880 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2687015632 0 0) 0x7fd67406f7f0 con 0x7fd6740b0b20 2023-10-16T11:27:23.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.676+0000 7fd671d9b640 1 -- 172.21.15.84:0/4265152880 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd660005730 con 0x7fd6740b0b20 2023-10-16T11:27:23.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.677+0000 7fd671d9b640 1 -- 172.21.15.84:0/4265152880 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 295192516 0 0) 0x7fd660005730 con 0x7fd6740b0b20 2023-10-16T11:27:23.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.677+0000 7fd671d9b640 1 -- 172.21.15.84:0/4265152880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd674159b20 msgr2=0x7fd67406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:23.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.677+0000 7fd671d9b640 1 --2- 172.21.15.84:0/4265152880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd674159b20 0x7fd67406bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:23.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.677+0000 7fd671d9b640 1 -- 172.21.15.84:0/4265152880 >> v1:172.21.15.157:6789/0 conn(0x7fd674159f00 legacy=0x7fd6740b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:23.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.677+0000 7fd671d9b640 1 -- 172.21.15.84:0/4265152880 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67406cba0 con 0x7fd6740b0b20 2023-10-16T11:27:23.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.677+0000 7fd671d9b640 1 -- 172.21.15.84:0/4265152880 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd66c001670 con 0x7fd6740b0b20 2023-10-16T11:27:23.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.677+0000 7fd671d9b640 1 -- 172.21.15.84:0/4265152880 >> v1:172.21.15.84:6789/0 conn(0x7fd6740b0b20 legacy=0x7fd67406fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:23.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.677+0000 7fd671d9b640 1 --2- 172.21.15.84:0/4265152880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd674159b20 0x7fd660005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:23.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.677+0000 7fd671d9b640 1 --2- 172.21.15.84:0/4265152880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6600061c0 0x7fd6600069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:23.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.677+0000 7fd671d9b640 1 --2- 172.21.15.84:0/4265152880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd660006f10 0x7fd660007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:23.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.677+0000 7fd671d9b640 1 -- 172.21.15.84:0/4265152880 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67406c8c0 con 0x7fd660006f10 2023-10-16T11:27:23.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.677+0000 7fd671d9b640 1 -- 172.21.15.84:0/4265152880 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66000dd20 con 0x7fd6600061c0 2023-10-16T11:27:23.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.677+0000 7fd671d9b640 1 -- 172.21.15.84:0/4265152880 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66000dee0 con 0x7fd674159b20 2023-10-16T11:27:23.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.677+0000 7fd672d9d640 1 --2- 172.21.15.84:0/4265152880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd660006f10 0x7fd660007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:23.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.677+0000 7fd67259c640 1 --2- 172.21.15.84:0/4265152880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd674159b20 0x7fd660005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:23.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.678+0000 7fd669d9b640 1 --2- 172.21.15.84:0/4265152880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6600061c0 0x7fd6600069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:23.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.678+0000 7fd672d9d640 1 -- 172.21.15.84:0/4265152880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd674159b20 msgr2=0x7fd660005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:23.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.678+0000 7fd672d9d640 1 --2- 172.21.15.84:0/4265152880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd674159b20 0x7fd660005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:23.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.678+0000 7fd672d9d640 1 -- 172.21.15.84:0/4265152880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6600061c0 msgr2=0x7fd6600069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:23.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.678+0000 7fd672d9d640 1 --2- 172.21.15.84:0/4265152880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6600061c0 0x7fd6600069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:23.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.678+0000 7fd672d9d640 1 -- 172.21.15.84:0/4265152880 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd660005730 con 0x7fd660006f10 2023-10-16T11:27:23.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.678+0000 7fd669d9b640 1 --2- 172.21.15.84:0/4265152880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6600061c0 0x7fd6600069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:23.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.679+0000 7fd672d9d640 1 --2- 172.21.15.84:0/4265152880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd660006f10 0x7fd660007390 secure :-1 s=READY pgs=3389 cs=0 l=1 rev1=1 crypto rx=0x7fd66c001eb0 tx=0x7fd660007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0681bdb8492e21 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:23.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.679+0000 7fd671d9b640 1 -- 172.21.15.84:0/4265152880 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c005ad0 con 0x7fd660006f10 2023-10-16T11:27:23.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.679+0000 7fd671d9b640 1 -- 172.21.15.84:0/4265152880 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66c005c50 con 0x7fd660006f10 2023-10-16T11:27:23.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.679+0000 7fd671d9b640 1 -- 172.21.15.84:0/4265152880 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c02c850 con 0x7fd660006f10 2023-10-16T11:27:23.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.679+0000 7fd678e4b640 1 -- 172.21.15.84:0/4265152880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd660006f10 msgr2=0x7fd660007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:23.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.679+0000 7fd678e4b640 1 --2- 172.21.15.84:0/4265152880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd660006f10 0x7fd660007390 secure :-1 s=READY pgs=3389 cs=0 l=1 rev1=1 crypto rx=0x7fd66c001eb0 tx=0x7fd660007a00 comp rx=0 tx=0).stop 2023-10-16T11:27:23.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.679+0000 7fd672d9d640 1 -- 172.21.15.84:0/4265152880 reap_dead start 2023-10-16T11:27:23.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.679+0000 7fd678e4b640 1 -- 172.21.15.84:0/4265152880 shutdown_connections 2023-10-16T11:27:23.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.679+0000 7fd678e4b640 1 -- 172.21.15.84:0/4265152880 >> 172.21.15.84:0/4265152880 conn(0x7fd6740cb410 msgr2=0x7fd6740cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:23.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.680+0000 7fd678e4b640 1 -- 172.21.15.84:0/4265152880 shutdown_connections 2023-10-16T11:27:23.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.680+0000 7fd678e4b640 1 -- 172.21.15.84:0/4265152880 wait complete. 2023-10-16T11:27:23.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.681+0000 7fd678e4b640 1 Processor -- start 2023-10-16T11:27:23.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.681+0000 7fd678e4b640 1 -- start start 2023-10-16T11:27:23.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.682+0000 7fd678e4b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd660006f10 0x7fd6740c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:23.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.682+0000 7fd678e4b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd674151f70 0x7fd6740c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:23.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.682+0000 7fd678e4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6740c7a10 0x7fd6740c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:23.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.682+0000 7fd678e4b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6740c8a80 con 0x7fd660006f10 2023-10-16T11:27:23.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.682+0000 7fd678e4b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6740c38e0 con 0x7fd6740c7a10 2023-10-16T11:27:23.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.682+0000 7fd678e4b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6740c6e50 con 0x7fd674151f70 2023-10-16T11:27:23.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.682+0000 7fd67259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd660006f10 0x7fd6740c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:23.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.682+0000 7fd67259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd660006f10 0x7fd6740c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46876/0 (socket says 172.21.15.84:46876) 2023-10-16T11:27:23.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.682+0000 7fd67259c640 1 -- 172.21.15.84:0/3896043514 learned_addr learned my addr 172.21.15.84:0/3896043514 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:23.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.683+0000 7fd67259c640 1 -- 172.21.15.84:0/3896043514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd674151f70 msgr2=0x7fd6740c74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:23.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.683+0000 7fd67259c640 1 --2- 172.21.15.84:0/3896043514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd674151f70 0x7fd6740c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:23.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.683+0000 7fd67259c640 1 -- 172.21.15.84:0/3896043514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6740c7a10 msgr2=0x7fd6740c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:23.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.683+0000 7fd67259c640 1 --2- 172.21.15.84:0/3896043514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6740c7a10 0x7fd6740c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:23.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.683+0000 7fd67259c640 1 -- 172.21.15.84:0/3896043514 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6740c6b70 con 0x7fd660006f10 2023-10-16T11:27:23.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.683+0000 7fd67259c640 1 --2- 172.21.15.84:0/3896043514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd660006f10 0x7fd6740c6470 secure :-1 s=READY pgs=3390 cs=0 l=1 rev1=1 crypto rx=0x7fd664009d10 tx=0x7fd664007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:23.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.683+0000 7fd66b7fe640 1 -- 172.21.15.84:0/3896043514 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664014610 con 0x7fd660006f10 2023-10-16T11:27:23.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.683+0000 7fd66b7fe640 1 -- 172.21.15.84:0/3896043514 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66400f050 con 0x7fd660006f10 2023-10-16T11:27:23.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.683+0000 7fd66b7fe640 1 -- 172.21.15.84:0/3896043514 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664014990 con 0x7fd660006f10 2023-10-16T11:27:23.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.683+0000 7fd678e4b640 1 -- 172.21.15.84:0/3896043514 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6740c3b10 con 0x7fd660006f10 2023-10-16T11:27:23.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.683+0000 7fd678e4b640 1 -- 172.21.15.84:0/3896043514 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6740c40d0 con 0x7fd660006f10 2023-10-16T11:27:23.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.684+0000 7fd678e4b640 1 -- 172.21.15.84:0/3896043514 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6740c4290 con 0x7fd660006f10 2023-10-16T11:27:23.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.685+0000 7fd66b7fe640 1 -- 172.21.15.84:0/3896043514 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd664014af0 con 0x7fd660006f10 2023-10-16T11:27:23.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.686+0000 7fd66b7fe640 1 --2- 172.21.15.84:0/3896043514 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd65406af90 0x7fd65406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:23.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.686+0000 7fd66b7fe640 1 -- 172.21.15.84:0/3896043514 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(269..269 src has 1..269) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd66408a1e0 con 0x7fd660006f10 2023-10-16T11:27:23.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.686+0000 7fd669d9b640 1 --2- 172.21.15.84:0/3896043514 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd65406af90 0x7fd65406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:23.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.687+0000 7fd669d9b640 1 --2- 172.21.15.84:0/3896043514 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd65406af90 0x7fd65406d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6740b48c0 tx=0x7fd65c009040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:23.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.691+0000 7fd66b7fe640 1 -- 172.21.15.84:0/3896043514 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd664054fb0 con 0x7fd660006f10 2023-10-16T11:27:23.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.868+0000 7fd678e4b640 1 -- 172.21.15.84:0/3896043514 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd67404e160 con 0x7fd660006f10 2023-10-16T11:27:23.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.871+0000 7fd66b7fe640 1 -- 172.21.15.84:0/3896043514 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v269) v1 ==== 74+0+16560 (secure 0 0 0) 0x7fd664003e40 con 0x7fd660006f10 2023-10-16T11:27:23.873 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:23.873 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":269,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:27:22.870565+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:27:20.716240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"269","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_data","create_time":"2023-10-16T11:27:21.905553+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"269","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:27:23.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.873+0000 7fd678e4b640 1 -- 172.21.15.84:0/3896043514 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd65406af90 msgr2=0x7fd65406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:23.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.873+0000 7fd678e4b640 1 --2- 172.21.15.84:0/3896043514 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd65406af90 0x7fd65406d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6740b48c0 tx=0x7fd65c009040 comp rx=0 tx=0).stop 2023-10-16T11:27:23.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.873+0000 7fd678e4b640 1 -- 172.21.15.84:0/3896043514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd660006f10 msgr2=0x7fd6740c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:23.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.873+0000 7fd678e4b640 1 --2- 172.21.15.84:0/3896043514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd660006f10 0x7fd6740c6470 secure :-1 s=READY pgs=3390 cs=0 l=1 rev1=1 crypto rx=0x7fd664009d10 tx=0x7fd664007700 comp rx=0 tx=0).stop 2023-10-16T11:27:23.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.873+0000 7fd678e4b640 1 -- 172.21.15.84:0/3896043514 shutdown_connections 2023-10-16T11:27:23.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.873+0000 7fd678e4b640 1 --2- 172.21.15.84:0/3896043514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6740c7a10 0x7fd6740c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:23.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.873+0000 7fd678e4b640 1 --2- 172.21.15.84:0/3896043514 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd65406af90 0x7fd65406d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:23.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.873+0000 7fd678e4b640 1 --2- 172.21.15.84:0/3896043514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd674151f70 0x7fd6740c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:23.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.873+0000 7fd678e4b640 1 --2- 172.21.15.84:0/3896043514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd660006f10 0x7fd6740c6470 unknown :-1 s=CLOSED pgs=3390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:23.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.873+0000 7fd678e4b640 1 -- 172.21.15.84:0/3896043514 >> 172.21.15.84:0/3896043514 conn(0x7fd6740cb410 msgr2=0x7fd6740ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:23.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.873+0000 7fd678e4b640 1 -- 172.21.15.84:0/3896043514 shutdown_connections 2023-10-16T11:27:23.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:23.873+0000 7fd678e4b640 1 -- 172.21.15.84:0/3896043514 wait complete. 2023-10-16T11:27:23.893 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:27:24.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.099+0000 7f2ac6089640 1 Processor -- start 2023-10-16T11:27:24.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.099+0000 7f2ac6089640 1 -- start start 2023-10-16T11:27:24.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.100+0000 7f2ac6089640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ac015aaa0 0x7f2ac015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:24.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.100+0000 7f2ac6089640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2ac015ae80 0x7f2ac00cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:24.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.100+0000 7f2ac6089640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ac00ccaa0 0x7f2ac0070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:24.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.100+0000 7f2ac6089640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ac006b810 con 0x7f2ac015ae80 2023-10-16T11:27:24.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.100+0000 7f2ac6089640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ac015ed60 con 0x7f2ac00ccaa0 2023-10-16T11:27:24.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.100+0000 7f2ac6089640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac00707e0 con 0x7f2ac015aaa0 2023-10-16T11:27:24.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.100+0000 7f2ab7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2ac015ae80 0x7f2ac00cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:24.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.100+0000 7f2ab7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2ac015ae80 0x7f2ac00cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46888/0 (socket says 172.21.15.84:46888) 2023-10-16T11:27:24.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.100+0000 7f2ab7fff640 1 -- 172.21.15.84:0/2221145738 learned_addr learned my addr 172.21.15.84:0/2221145738 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:24.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.101+0000 7f2ab7fff640 1 -- 172.21.15.84:0/2221145738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ac015aaa0 msgr2=0x7f2ac015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:24.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.101+0000 7f2ab7fff640 1 --2- 172.21.15.84:0/2221145738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ac015aaa0 0x7f2ac015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.101+0000 7f2ab7fff640 1 -- 172.21.15.84:0/2221145738 >> v2:172.21.15.157:3300/0 conn(0x7f2ac00ccaa0 msgr2=0x7f2ac0070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:24.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.101+0000 7f2ab7fff640 1 --2- 172.21.15.84:0/2221145738 >> v2:172.21.15.157:3300/0 conn(0x7f2ac00ccaa0 0x7f2ac0070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.101+0000 7f2ab7fff640 1 -- 172.21.15.84:0/2221145738 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac015ef90 con 0x7f2ac015ae80 2023-10-16T11:27:24.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.102+0000 7f2ab7fff640 1 --2- 172.21.15.84:0/2221145738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac015ae80 0x7f2ac00cc560 secure :-1 s=READY pgs=3391 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00ae3c0 tx=0x7f2aa802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406026f3cfbb2584 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:24.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.102+0000 7f2abeffd640 1 -- 172.21.15.84:0/2221145738 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8031870 con 0x7f2ac015ae80 2023-10-16T11:27:24.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.102+0000 7f2abeffd640 1 -- 172.21.15.84:0/2221145738 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa80319d0 con 0x7f2ac015ae80 2023-10-16T11:27:24.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.102+0000 7f2abeffd640 1 -- 172.21.15.84:0/2221145738 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8031d60 con 0x7f2ac015ae80 2023-10-16T11:27:24.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.103+0000 7f2ac6089640 1 -- 172.21.15.84:0/2221145738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac015ae80 msgr2=0x7f2ac00cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:24.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.103+0000 7f2ac6089640 1 --2- 172.21.15.84:0/2221145738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac015ae80 0x7f2ac00cc560 secure :-1 s=READY pgs=3391 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00ae3c0 tx=0x7f2aa802b910 comp rx=0 tx=0).stop 2023-10-16T11:27:24.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.103+0000 7f2ac6089640 1 -- 172.21.15.84:0/2221145738 shutdown_connections 2023-10-16T11:27:24.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.103+0000 7f2ac6089640 1 --2- 172.21.15.84:0/2221145738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac015ae80 0x7f2ac00cc560 unknown :-1 s=CLOSED pgs=3391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.103+0000 7f2ac6089640 1 --2- 172.21.15.84:0/2221145738 >> v2:172.21.15.157:3300/0 conn(0x7f2ac00ccaa0 0x7f2ac0070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.103+0000 7f2ac6089640 1 --2- 172.21.15.84:0/2221145738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ac015aaa0 0x7f2ac015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.103+0000 7f2ac6089640 1 -- 172.21.15.84:0/2221145738 >> 172.21.15.84:0/2221145738 conn(0x7f2ac00cb410 msgr2=0x7f2ac00cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:24.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.103+0000 7f2ac6089640 1 -- 172.21.15.84:0/2221145738 shutdown_connections 2023-10-16T11:27:24.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.103+0000 7f2ac6089640 1 -- 172.21.15.84:0/2221145738 wait complete. 2023-10-16T11:27:24.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.104+0000 7f2ac6089640 1 Processor -- start 2023-10-16T11:27:24.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.104+0000 7f2ac6089640 1 -- start start 2023-10-16T11:27:24.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.105+0000 7f2ac6089640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac00ccaa0 0x7f2ac0159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:24.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.105+0000 7f2ac6089640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ac015aaa0 0x7f2ac00c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:24.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.105+0000 7f2ac6089640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ac015ae80 0x7f2ac00c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:24.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.105+0000 7f2ac6089640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac00c8d30 con 0x7f2ac00ccaa0 2023-10-16T11:27:24.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.105+0000 7f2ac6089640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac00c9010 con 0x7f2ac015aaa0 2023-10-16T11:27:24.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.105+0000 7f2ac6089640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac00c92f0 con 0x7f2ac015ae80 2023-10-16T11:27:24.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.106+0000 7f2abf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac00ccaa0 0x7f2ac0159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:24.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.106+0000 7f2abf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac00ccaa0 0x7f2ac0159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46896/0 (socket says 172.21.15.84:46896) 2023-10-16T11:27:24.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.106+0000 7f2abf7fe640 1 -- 172.21.15.84:0/1057430005 learned_addr learned my addr 172.21.15.84:0/1057430005 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:24.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.106+0000 7f2abf7fe640 1 -- 172.21.15.84:0/1057430005 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ac015ae80 msgr2=0x7f2ac00c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:24.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.106+0000 7f2abf7fe640 1 --2- 172.21.15.84:0/1057430005 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ac015ae80 0x7f2ac00c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.106+0000 7f2abf7fe640 1 -- 172.21.15.84:0/1057430005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ac015aaa0 msgr2=0x7f2ac00c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:24.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.106+0000 7f2abf7fe640 1 --2- 172.21.15.84:0/1057430005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ac015aaa0 0x7f2ac00c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.106+0000 7f2abf7fe640 1 -- 172.21.15.84:0/1057430005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa80094d0 con 0x7f2ac00ccaa0 2023-10-16T11:27:24.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.106+0000 7f2abf7fe640 1 --2- 172.21.15.84:0/1057430005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac00ccaa0 0x7f2ac0159ee0 secure :-1 s=READY pgs=3392 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000b410 tx=0x7f2ab000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:24.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.106+0000 7f2abcff9640 1 -- 172.21.15.84:0/1057430005 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0011830 con 0x7f2ac00ccaa0 2023-10-16T11:27:24.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.107+0000 7f2abcff9640 1 -- 172.21.15.84:0/1057430005 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab00119b0 con 0x7f2ac00ccaa0 2023-10-16T11:27:24.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.107+0000 7f2abcff9640 1 -- 172.21.15.84:0/1057430005 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0011d60 con 0x7f2ac00ccaa0 2023-10-16T11:27:24.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.107+0000 7f2ac6089640 1 -- 172.21.15.84:0/1057430005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac00c95d0 con 0x7f2ac00ccaa0 2023-10-16T11:27:24.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.107+0000 7f2ac6089640 1 -- 172.21.15.84:0/1057430005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac00c8a20 con 0x7f2ac00ccaa0 2023-10-16T11:27:24.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.108+0000 7f2ac6089640 1 -- 172.21.15.84:0/1057430005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac00c97c0 con 0x7f2ac00ccaa0 2023-10-16T11:27:24.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.109+0000 7f2abcff9640 1 -- 172.21.15.84:0/1057430005 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2ab001a400 con 0x7f2ac00ccaa0 2023-10-16T11:27:24.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.110+0000 7f2abcff9640 1 --2- 172.21.15.84:0/1057430005 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f2aa006afe0 0x7f2aa006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:24.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.110+0000 7f2abcff9640 1 -- 172.21.15.84:0/1057430005 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(269..269 src has 1..269) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f2ab0089820 con 0x7f2ac00ccaa0 2023-10-16T11:27:24.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.110+0000 7f2ab7fff640 1 --2- 172.21.15.84:0/1057430005 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f2aa006afe0 0x7f2aa006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:24.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.111+0000 7f2ab7fff640 1 --2- 172.21.15.84:0/1057430005 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f2aa006afe0 0x7f2aa006d4a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8036000 tx=0x7f2aa8030000 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:24.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.114+0000 7f2abcff9640 1 -- 172.21.15.84:0/1057430005 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2ab0054880 con 0x7f2ac00ccaa0 2023-10-16T11:27:24.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.306+0000 7f2ac6089640 1 -- 172.21.15.84:0/1057430005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ac004e160 con 0x7f2ac00ccaa0 2023-10-16T11:27:24.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.309+0000 7f2abcff9640 1 -- 172.21.15.84:0/1057430005 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v269) v1 ==== 74+0+16560 (secure 0 0 0) 0x7f2ab0015090 con 0x7f2ac00ccaa0 2023-10-16T11:27:24.312 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:24.312 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":269,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:27:22.870565+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:27:20.716240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"269","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_data","create_time":"2023-10-16T11:27:21.905553+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"269","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:27:24.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.312+0000 7f2ac6089640 1 -- 172.21.15.84:0/1057430005 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f2aa006afe0 msgr2=0x7f2aa006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.312+0000 7f2ac6089640 1 --2- 172.21.15.84:0/1057430005 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f2aa006afe0 0x7f2aa006d4a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8036000 tx=0x7f2aa8030000 comp rx=0 tx=0).stop 2023-10-16T11:27:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.312+0000 7f2ac6089640 1 -- 172.21.15.84:0/1057430005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac00ccaa0 msgr2=0x7f2ac0159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.312+0000 7f2ac6089640 1 --2- 172.21.15.84:0/1057430005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac00ccaa0 0x7f2ac0159ee0 secure :-1 s=READY pgs=3392 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000b410 tx=0x7f2ab000b8d0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.312+0000 7f2ac6089640 1 -- 172.21.15.84:0/1057430005 shutdown_connections 2023-10-16T11:27:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.312+0000 7f2ac6089640 1 --2- 172.21.15.84:0/1057430005 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f2aa006afe0 0x7f2aa006d4a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.312+0000 7f2ac6089640 1 --2- 172.21.15.84:0/1057430005 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ac015ae80 0x7f2ac00c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.312+0000 7f2ac6089640 1 --2- 172.21.15.84:0/1057430005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ac015aaa0 0x7f2ac00c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.312+0000 7f2ac6089640 1 --2- 172.21.15.84:0/1057430005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ac00ccaa0 0x7f2ac0159ee0 unknown :-1 s=CLOSED pgs=3392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.312+0000 7f2ac6089640 1 -- 172.21.15.84:0/1057430005 >> 172.21.15.84:0/1057430005 conn(0x7f2ac00cb410 msgr2=0x7f2ac00c44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:24.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.313+0000 7f2ac6089640 1 -- 172.21.15.84:0/1057430005 shutdown_connections 2023-10-16T11:27:24.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.313+0000 7f2ac6089640 1 -- 172.21.15.84:0/1057430005 wait complete. 2023-10-16T11:27:24.334 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T11:27:24.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.553+0000 7f06625dc640 1 Processor -- start 2023-10-16T11:27:24.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.553+0000 7f06625dc640 1 -- start start 2023-10-16T11:27:24.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.554+0000 7f06625dc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f065c159ab0 0x7f065c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:24.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.554+0000 7f06625dc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f065c06c140 0x7f065c06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:24.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.554+0000 7f06625dc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f065c070360 con 0x7f065c159e90 2023-10-16T11:27:24.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.554+0000 7f06625dc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f065c06cb10 con 0x7f065c159ab0 2023-10-16T11:27:24.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.554+0000 7f06625dc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c15d890 con 0x7f065c06c140 2023-10-16T11:27:24.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.555+0000 7f065b7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f065c159e90 0x7f065c06fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55328/0 (socket says 172.21.15.84:55328) 2023-10-16T11:27:24.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.555+0000 7f065b7fe640 1 -- 172.21.15.84:0/3118470386 learned_addr learned my addr 172.21.15.84:0/3118470386 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:24.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.555+0000 7f0660b79640 1 --2- 172.21.15.84:0/3118470386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f065c06c140 0x7f065c06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:24.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.555+0000 7f065bfff640 1 --2- 172.21.15.84:0/3118470386 >> v2:172.21.15.157:3300/0 conn(0x7f065c159ab0 0x7f065c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:24.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.555+0000 7f065affd640 1 -- 172.21.15.84:0/3118470386 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667120556 0 0) 0x7f065c070360 con 0x7f065c159e90 2023-10-16T11:27:24.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.555+0000 7f065affd640 1 -- 172.21.15.84:0/3118470386 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0644003520 con 0x7f065c159e90 2023-10-16T11:27:24.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.556+0000 7f065affd640 1 -- 172.21.15.84:0/3118470386 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 37294125 0 0) 0x7f0644003520 con 0x7f065c159e90 2023-10-16T11:27:24.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.556+0000 7f065affd640 1 -- 172.21.15.84:0/3118470386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f065c06c140 msgr2=0x7f065c06c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:24.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.556+0000 7f065affd640 1 --2- 172.21.15.84:0/3118470386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f065c06c140 0x7f065c06c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.556+0000 7f065affd640 1 -- 172.21.15.84:0/3118470386 >> v2:172.21.15.157:3300/0 conn(0x7f065c159ab0 msgr2=0x7f065c0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:24.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.556+0000 7f065affd640 1 --2- 172.21.15.84:0/3118470386 >> v2:172.21.15.157:3300/0 conn(0x7f065c159ab0 0x7f065c0ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.556+0000 7f065affd640 1 -- 172.21.15.84:0/3118470386 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c0cd1f0 con 0x7f065c159e90 2023-10-16T11:27:24.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.556+0000 7f065affd640 1 -- 172.21.15.84:0/3118470386 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f0650001670 con 0x7f065c159e90 2023-10-16T11:27:24.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.556+0000 7f065affd640 1 -- 172.21.15.84:0/3118470386 >> v1:172.21.15.84:6789/0 conn(0x7f065c159e90 legacy=0x7f065c06fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:24.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.556+0000 7f065affd640 1 --2- 172.21.15.84:0/3118470386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0644003970 0x7f0644003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:24.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.556+0000 7f065affd640 1 --2- 172.21.15.84:0/3118470386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f065c06c140 0x7f06440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:24.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.556+0000 7f065affd640 1 --2- 172.21.15.84:0/3118470386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0644004d00 0x7f0644005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:24.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.556+0000 7f065affd640 1 -- 172.21.15.84:0/3118470386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c06cb10 con 0x7f0644003970 2023-10-16T11:27:24.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.556+0000 7f065affd640 1 -- 172.21.15.84:0/3118470386 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c15d890 con 0x7f0644004d00 2023-10-16T11:27:24.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.556+0000 7f065affd640 1 -- 172.21.15.84:0/3118470386 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064400ba00 con 0x7f065c06c140 2023-10-16T11:27:24.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.556+0000 7f065bfff640 1 --2- 172.21.15.84:0/3118470386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0644003970 0x7f0644003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:24.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.557+0000 7f065bfff640 1 -- 172.21.15.84:0/3118470386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f065c06c140 msgr2=0x7f06440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:24.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.557+0000 7f065bfff640 1 --2- 172.21.15.84:0/3118470386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f065c06c140 0x7f06440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.557+0000 7f065bfff640 1 -- 172.21.15.84:0/3118470386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0644004d00 msgr2=0x7f0644005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:24.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.557+0000 7f065bfff640 1 --2- 172.21.15.84:0/3118470386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0644004d00 0x7f0644005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.557+0000 7f065bfff640 1 -- 172.21.15.84:0/3118470386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0644003520 con 0x7f0644003970 2023-10-16T11:27:24.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.558+0000 7f065bfff640 1 --2- 172.21.15.84:0/3118470386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0644003970 0x7f0644003d90 secure :-1 s=READY pgs=3394 cs=0 l=1 rev1=1 crypto rx=0x7f064c009ac0 tx=0x7f06440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292b3f8608029c43 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:24.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.558+0000 7f065affd640 1 -- 172.21.15.84:0/3118470386 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c02c710 con 0x7f0644003970 2023-10-16T11:27:24.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.558+0000 7f065affd640 1 -- 172.21.15.84:0/3118470386 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c02c870 con 0x7f0644003970 2023-10-16T11:27:24.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.558+0000 7f065affd640 1 -- 172.21.15.84:0/3118470386 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c02cc00 con 0x7f0644003970 2023-10-16T11:27:24.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.558+0000 7f06625dc640 1 -- 172.21.15.84:0/3118470386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0644003970 msgr2=0x7f0644003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:24.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.558+0000 7f06625dc640 1 --2- 172.21.15.84:0/3118470386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0644003970 0x7f0644003d90 secure :-1 s=READY pgs=3394 cs=0 l=1 rev1=1 crypto rx=0x7f064c009ac0 tx=0x7f06440057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.559+0000 7f0660b79640 1 -- 172.21.15.84:0/3118470386 reap_dead start 2023-10-16T11:27:24.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.559+0000 7f06625dc640 1 -- 172.21.15.84:0/3118470386 shutdown_connections 2023-10-16T11:27:24.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.559+0000 7f06625dc640 1 -- 172.21.15.84:0/3118470386 >> 172.21.15.84:0/3118470386 conn(0x7f065c0ce9b0 msgr2=0x7f065c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:24.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.559+0000 7f06625dc640 1 -- 172.21.15.84:0/3118470386 shutdown_connections 2023-10-16T11:27:24.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.559+0000 7f06625dc640 1 -- 172.21.15.84:0/3118470386 wait complete. 2023-10-16T11:27:24.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.560+0000 7f06625dc640 1 Processor -- start 2023-10-16T11:27:24.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.561+0000 7f06625dc640 1 -- start start 2023-10-16T11:27:24.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.561+0000 7f06625dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0644003970 0x7f065c158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:24.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.561+0000 7f06625dc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065c1519b0 0x7f065c15c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:24.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.561+0000 7f06625dc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f065c0c1600 0x7f065c15a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:24.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.562+0000 7f06625dc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c15d230 con 0x7f065c1519b0 2023-10-16T11:27:24.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.562+0000 7f06625dc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c1580b0 con 0x7f0644003970 2023-10-16T11:27:24.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.562+0000 7f06625dc640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c158390 con 0x7f065c0c1600 2023-10-16T11:27:24.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.562+0000 7f065b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065c1519b0 0x7f065c15c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:24.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.562+0000 7f065b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065c1519b0 0x7f065c15c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46922/0 (socket says 172.21.15.84:46922) 2023-10-16T11:27:24.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.562+0000 7f065b7fe640 1 -- 172.21.15.84:0/409579093 learned_addr learned my addr 172.21.15.84:0/409579093 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:24.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.562+0000 7f065bfff640 1 --2- 172.21.15.84:0/409579093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0644003970 0x7f065c158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:24.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.562+0000 7f065b7fe640 1 -- 172.21.15.84:0/409579093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f065c0c1600 msgr2=0x7f065c15a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:24.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.562+0000 7f065b7fe640 1 --2- 172.21.15.84:0/409579093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f065c0c1600 0x7f065c15a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.562+0000 7f065b7fe640 1 -- 172.21.15.84:0/409579093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0644003970 msgr2=0x7f065c158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:24.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.562+0000 7f065b7fe640 1 --2- 172.21.15.84:0/409579093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0644003970 0x7f065c158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:24.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.562+0000 7f065b7fe640 1 -- 172.21.15.84:0/409579093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c158670 con 0x7f065c1519b0 2023-10-16T11:27:24.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.562+0000 7f065b7fe640 1 --2- 172.21.15.84:0/409579093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065c1519b0 0x7f065c15c9c0 secure :-1 s=READY pgs=3395 cs=0 l=1 rev1=1 crypto rx=0x7f065000c7a0 tx=0x7f065000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:24.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.563+0000 7f0658ff9640 1 -- 172.21.15.84:0/409579093 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06500074e0 con 0x7f065c1519b0 2023-10-16T11:27:24.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.563+0000 7f0658ff9640 1 -- 172.21.15.84:0/409579093 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0650007640 con 0x7f065c1519b0 2023-10-16T11:27:24.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.563+0000 7f06625dc640 1 -- 172.21.15.84:0/409579093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065c158950 con 0x7f065c1519b0 2023-10-16T11:27:24.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.563+0000 7f0658ff9640 1 -- 172.21.15.84:0/409579093 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650004a20 con 0x7f065c1519b0 2023-10-16T11:27:24.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.563+0000 7f06625dc640 1 -- 172.21.15.84:0/409579093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065c15cf00 con 0x7f065c1519b0 2023-10-16T11:27:24.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.564+0000 7f06625dc640 1 -- 172.21.15.84:0/409579093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065c158b70 con 0x7f065c1519b0 2023-10-16T11:27:24.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.565+0000 7f0658ff9640 1 -- 172.21.15.84:0/409579093 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0650004b80 con 0x7f065c1519b0 2023-10-16T11:27:24.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.566+0000 7f0658ff9640 1 --2- 172.21.15.84:0/409579093 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f064806af90 0x7f064806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:24.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.566+0000 7f0658ff9640 1 -- 172.21.15.84:0/409579093 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0650088bd0 con 0x7f065c1519b0 2023-10-16T11:27:24.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.566+0000 7f065bfff640 1 --2- 172.21.15.84:0/409579093 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f064806af90 0x7f064806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:24.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.567+0000 7f065bfff640 1 --2- 172.21.15.84:0/409579093 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f064806af90 0x7f064806d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f064c036000 tx=0x7f064c030000 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:24.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.570+0000 7f0658ff9640 1 -- 172.21.15.84:0/409579093 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0650053ab0 con 0x7f065c1519b0 2023-10-16T11:27:24.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:24.799+0000 7f06625dc640 1 -- 172.21.15.84:0/409579093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f065c04e160 con 0x7f065c1519b0 2023-10-16T11:27:25.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.480+0000 7f0658ff9640 1 -- 172.21.15.84:0/409579093 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v156) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0650018020 con 0x7f065c1519b0 2023-10-16T11:27:25.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.484+0000 7f06625dc640 1 -- 172.21.15.84:0/409579093 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f064806af90 msgr2=0x7f064806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:25.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.484+0000 7f06625dc640 1 --2- 172.21.15.84:0/409579093 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f064806af90 0x7f064806d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f064c036000 tx=0x7f064c030000 comp rx=0 tx=0).stop 2023-10-16T11:27:25.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.484+0000 7f06625dc640 1 -- 172.21.15.84:0/409579093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065c1519b0 msgr2=0x7f065c15c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:25.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.484+0000 7f06625dc640 1 --2- 172.21.15.84:0/409579093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065c1519b0 0x7f065c15c9c0 secure :-1 s=READY pgs=3395 cs=0 l=1 rev1=1 crypto rx=0x7f065000c7a0 tx=0x7f065000cc60 comp rx=0 tx=0).stop 2023-10-16T11:27:25.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.484+0000 7f06625dc640 1 -- 172.21.15.84:0/409579093 shutdown_connections 2023-10-16T11:27:25.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.484+0000 7f06625dc640 1 --2- 172.21.15.84:0/409579093 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f064806af90 0x7f064806d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:25.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.484+0000 7f06625dc640 1 --2- 172.21.15.84:0/409579093 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f065c0c1600 0x7f065c15a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:25.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.484+0000 7f06625dc640 1 --2- 172.21.15.84:0/409579093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065c1519b0 0x7f065c15c9c0 unknown :-1 s=CLOSED pgs=3395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:25.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.484+0000 7f06625dc640 1 --2- 172.21.15.84:0/409579093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0644003970 0x7f065c158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:25.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.484+0000 7f06625dc640 1 -- 172.21.15.84:0/409579093 >> 172.21.15.84:0/409579093 conn(0x7f065c0ce9b0 msgr2=0x7f065c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:25.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.484+0000 7f06625dc640 1 -- 172.21.15.84:0/409579093 shutdown_connections 2023-10-16T11:27:25.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.485+0000 7f06625dc640 1 -- 172.21.15.84:0/409579093 wait complete. 2023-10-16T11:27:25.506 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T11:27:25.507 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T11:27:25.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.723+0000 7f2927ac9640 1 Processor -- start 2023-10-16T11:27:25.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.723+0000 7f2927ac9640 1 -- start start 2023-10-16T11:27:25.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.724+0000 7f2927ac9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f292015ae10 0x7f29200ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:25.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.725+0000 7f2927ac9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f292015b1f0 con 0x7f29200cf330 2023-10-16T11:27:25.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.725+0000 7f2927ac9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f292006ce20 con 0x7f292006d570 2023-10-16T11:27:25.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.725+0000 7f2927ac9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f292006f650 con 0x7f292015ae10 2023-10-16T11:27:25.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.725+0000 7f2926066640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f29200cf330 0x7f292006c710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43298/0 (socket says 172.21.15.84:43298) 2023-10-16T11:27:25.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.725+0000 7f2926066640 1 -- 172.21.15.84:0/2833570997 learned_addr learned my addr 172.21.15.84:0/2833570997 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:25.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.725+0000 7f2924863640 1 -- 172.21.15.84:0/2833570997 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726246508 0 0) 0x7f2918001490 con 0x7f29200cf330 2023-10-16T11:27:25.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.725+0000 7f2925865640 1 --2- 172.21.15.84:0/2833570997 >> v2:172.21.15.157:3300/0 conn(0x7f292015ae10 0x7f29200ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:25.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.725+0000 7f2924863640 1 -- 172.21.15.84:0/2833570997 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f290c003520 con 0x7f29200cf330 2023-10-16T11:27:25.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.726+0000 7f2924863640 1 -- 172.21.15.84:0/2833570997 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1122837191 0 0) 0x7f290c003520 con 0x7f29200cf330 2023-10-16T11:27:25.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.726+0000 7f2924863640 1 -- 172.21.15.84:0/2833570997 >> v2:172.21.15.157:3300/0 conn(0x7f292015ae10 msgr2=0x7f29200ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:25.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.726+0000 7f2924863640 1 --2- 172.21.15.84:0/2833570997 >> v2:172.21.15.157:3300/0 conn(0x7f292015ae10 0x7f29200ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:25.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.726+0000 7f2924863640 1 -- 172.21.15.84:0/2833570997 >> v1:172.21.15.157:6789/0 conn(0x7f292006d570 legacy=0x7f292006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:25.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.726+0000 7f2924863640 1 -- 172.21.15.84:0/2833570997 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292006bb70 con 0x7f29200cf330 2023-10-16T11:27:25.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.726+0000 7f2924863640 1 -- 172.21.15.84:0/2833570997 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f2918001b80 con 0x7f29200cf330 2023-10-16T11:27:25.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.726+0000 7f2924863640 1 -- 172.21.15.84:0/2833570997 >> v1:172.21.15.84:6789/0 conn(0x7f29200cf330 legacy=0x7f292006c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:25.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.726+0000 7f2924863640 1 --2- 172.21.15.84:0/2833570997 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f290c003970 0x7f290c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:25.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.726+0000 7f2924863640 1 --2- 172.21.15.84:0/2833570997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f290c0047c0 0x7f290c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:25.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.726+0000 7f2924863640 1 --2- 172.21.15.84:0/2833570997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c005180 0x7f290c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:25.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.726+0000 7f2924863640 1 -- 172.21.15.84:0/2833570997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c00bff0 con 0x7f290c005180 2023-10-16T11:27:25.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.726+0000 7f2924863640 1 -- 172.21.15.84:0/2833570997 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c00c160 con 0x7f290c0047c0 2023-10-16T11:27:25.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.726+0000 7f2924863640 1 -- 172.21.15.84:0/2833570997 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290c00c320 con 0x7f290c003970 2023-10-16T11:27:25.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.726+0000 7f2926066640 1 --2- 172.21.15.84:0/2833570997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c005180 0x7f290c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:25.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.727+0000 7f2925865640 1 --2- 172.21.15.84:0/2833570997 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f290c003970 0x7f290c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:25.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.727+0000 7f2925064640 1 --2- 172.21.15.84:0/2833570997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f290c0047c0 0x7f290c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:25.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.727+0000 7f2926066640 1 -- 172.21.15.84:0/2833570997 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f290c003970 msgr2=0x7f290c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:25.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.727+0000 7f2926066640 1 --2- 172.21.15.84:0/2833570997 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f290c003970 0x7f290c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:25.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.727+0000 7f2926066640 1 -- 172.21.15.84:0/2833570997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f290c0047c0 msgr2=0x7f290c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:25.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.727+0000 7f2926066640 1 --2- 172.21.15.84:0/2833570997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f290c0047c0 0x7f290c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:25.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.727+0000 7f2926066640 1 -- 172.21.15.84:0/2833570997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c003520 con 0x7f290c005180 2023-10-16T11:27:25.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.727+0000 7f2925865640 1 --2- 172.21.15.84:0/2833570997 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f290c003970 0x7f290c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:25.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.727+0000 7f2925064640 1 --2- 172.21.15.84:0/2833570997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f290c0047c0 0x7f290c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:25.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.728+0000 7f2926066640 1 -- 172.21.15.84:0/2833570997 reap_dead start 2023-10-16T11:27:25.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.728+0000 7f2926066640 1 --2- 172.21.15.84:0/2833570997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c005180 0x7f290c00b980 secure :-1 s=READY pgs=3397 cs=0 l=1 rev1=1 crypto rx=0x7f291800a830 tx=0x7f290c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2a69e75677dae1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:25.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.728+0000 7f2924863640 1 -- 172.21.15.84:0/2833570997 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918034630 con 0x7f290c005180 2023-10-16T11:27:25.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.728+0000 7f2924863640 1 -- 172.21.15.84:0/2833570997 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291802f070 con 0x7f290c005180 2023-10-16T11:27:25.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.728+0000 7f2924863640 1 -- 172.21.15.84:0/2833570997 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29180349f0 con 0x7f290c005180 2023-10-16T11:27:25.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.729+0000 7f2927ac9640 1 -- 172.21.15.84:0/2833570997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c005180 msgr2=0x7f290c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:25.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.729+0000 7f2927ac9640 1 --2- 172.21.15.84:0/2833570997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c005180 0x7f290c00b980 secure :-1 s=READY pgs=3397 cs=0 l=1 rev1=1 crypto rx=0x7f291800a830 tx=0x7f290c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:25.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.729+0000 7f2927ac9640 1 -- 172.21.15.84:0/2833570997 shutdown_connections 2023-10-16T11:27:25.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.729+0000 7f2927ac9640 1 --2- 172.21.15.84:0/2833570997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c005180 0x7f290c00b980 unknown :-1 s=CLOSED pgs=3397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:25.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.729+0000 7f2927ac9640 1 -- 172.21.15.84:0/2833570997 >> 172.21.15.84:0/2833570997 conn(0x7f29200ce840 msgr2=0x7f29200cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:25.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.729+0000 7f2927ac9640 1 -- 172.21.15.84:0/2833570997 shutdown_connections 2023-10-16T11:27:25.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.729+0000 7f2927ac9640 1 -- 172.21.15.84:0/2833570997 wait complete. 2023-10-16T11:27:25.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.731+0000 7f2927ac9640 1 Processor -- start 2023-10-16T11:27:25.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.731+0000 7f2927ac9640 1 -- start start 2023-10-16T11:27:25.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.732+0000 7f2927ac9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c005180 0x7f292014d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:25.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.732+0000 7f2927ac9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f292014b470 0x7f292014b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:25.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.732+0000 7f2927ac9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f292014a670 0x7f292014aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:25.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.732+0000 7f2927ac9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292014be10 con 0x7f290c005180 2023-10-16T11:27:25.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.732+0000 7f2927ac9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292014e380 con 0x7f292014a670 2023-10-16T11:27:25.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.732+0000 7f2927ac9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292014c030 con 0x7f292014b470 2023-10-16T11:27:25.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.732+0000 7f2925865640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c005180 0x7f292014d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:25.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.732+0000 7f2925865640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c005180 0x7f292014d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42384/0 (socket says 172.21.15.84:42384) 2023-10-16T11:27:25.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.732+0000 7f2925865640 1 -- 172.21.15.84:0/4045581231 learned_addr learned my addr 172.21.15.84:0/4045581231 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:25.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.733+0000 7f2925064640 1 --2- 172.21.15.84:0/4045581231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f292014b470 0x7f292014b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:25.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.733+0000 7f2926066640 1 --2- 172.21.15.84:0/4045581231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f292014a670 0x7f292014aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:25.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.733+0000 7f2925865640 1 -- 172.21.15.84:0/4045581231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f292014b470 msgr2=0x7f292014b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:25.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.733+0000 7f2925865640 1 --2- 172.21.15.84:0/4045581231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f292014b470 0x7f292014b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:25.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.733+0000 7f2925865640 1 -- 172.21.15.84:0/4045581231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f292014a670 msgr2=0x7f292014aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:25.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.733+0000 7f2925865640 1 --2- 172.21.15.84:0/4045581231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f292014a670 0x7f292014aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:25.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.733+0000 7f2925865640 1 -- 172.21.15.84:0/4045581231 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292014c310 con 0x7f290c005180 2023-10-16T11:27:25.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.733+0000 7f2925064640 1 --2- 172.21.15.84:0/4045581231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f292014b470 0x7f292014b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:25.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.733+0000 7f2925865640 1 --2- 172.21.15.84:0/4045581231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c005180 0x7f292014d770 secure :-1 s=READY pgs=3398 cs=0 l=1 rev1=1 crypto rx=0x7f291c0108b0 tx=0x7f291c010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:25.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.733+0000 7f29167fc640 1 -- 172.21.15.84:0/4045581231 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c00eb60 con 0x7f290c005180 2023-10-16T11:27:25.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.733+0000 7f29167fc640 1 -- 172.21.15.84:0/4045581231 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291c00ecc0 con 0x7f290c005180 2023-10-16T11:27:25.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.733+0000 7f29167fc640 1 -- 172.21.15.84:0/4045581231 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c01c400 con 0x7f290c005180 2023-10-16T11:27:25.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.733+0000 7f2927ac9640 1 -- 172.21.15.84:0/4045581231 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f292014c5f0 con 0x7f290c005180 2023-10-16T11:27:25.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.733+0000 7f2927ac9640 1 -- 172.21.15.84:0/4045581231 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292014cbb0 con 0x7f290c005180 2023-10-16T11:27:25.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.734+0000 7f2927ac9640 1 -- 172.21.15.84:0/4045581231 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f292004e160 con 0x7f290c005180 2023-10-16T11:27:25.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.736+0000 7f29167fc640 1 -- 172.21.15.84:0/4045581231 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f291c01c560 con 0x7f290c005180 2023-10-16T11:27:25.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.737+0000 7f29167fc640 1 --2- 172.21.15.84:0/4045581231 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f290406af90 0x7f290406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:25.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.737+0000 7f29167fc640 1 -- 172.21.15.84:0/4045581231 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f291c089d80 con 0x7f290c005180 2023-10-16T11:27:25.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.737+0000 7f2925064640 1 --2- 172.21.15.84:0/4045581231 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f290406af90 0x7f290406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:25.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.738+0000 7f2925064640 1 --2- 172.21.15.84:0/4045581231 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f290406af90 0x7f290406d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f292015e420 tx=0x7f2910009040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:25.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.743+0000 7f29167fc640 1 -- 172.21.15.84:0/4045581231 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f291c054db0 con 0x7f290c005180 2023-10-16T11:27:25.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:25.973+0000 7f2927ac9640 1 -- 172.21.15.84:0/4045581231 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f292006f8c0 con 0x7f290c005180 2023-10-16T11:27:26.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.490+0000 7f29167fc640 1 -- 172.21.15.84:0/4045581231 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v157) v1 ==== 122+0+0 (secure 0 0 0) 0x7f291c0583d0 con 0x7f290c005180 2023-10-16T11:27:26.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.494+0000 7f2927ac9640 1 -- 172.21.15.84:0/4045581231 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f290406af90 msgr2=0x7f290406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:26.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.494+0000 7f2927ac9640 1 --2- 172.21.15.84:0/4045581231 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f290406af90 0x7f290406d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f292015e420 tx=0x7f2910009040 comp rx=0 tx=0).stop 2023-10-16T11:27:26.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.494+0000 7f2927ac9640 1 -- 172.21.15.84:0/4045581231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c005180 msgr2=0x7f292014d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:26.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.494+0000 7f2927ac9640 1 --2- 172.21.15.84:0/4045581231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c005180 0x7f292014d770 secure :-1 s=READY pgs=3398 cs=0 l=1 rev1=1 crypto rx=0x7f291c0108b0 tx=0x7f291c010d70 comp rx=0 tx=0).stop 2023-10-16T11:27:26.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.494+0000 7f2927ac9640 1 -- 172.21.15.84:0/4045581231 shutdown_connections 2023-10-16T11:27:26.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.494+0000 7f2927ac9640 1 --2- 172.21.15.84:0/4045581231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f292014a670 0x7f292014aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:26.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.494+0000 7f2927ac9640 1 --2- 172.21.15.84:0/4045581231 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f290406af90 0x7f290406d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:26.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.494+0000 7f2927ac9640 1 --2- 172.21.15.84:0/4045581231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f292014b470 0x7f292014b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:26.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.494+0000 7f2927ac9640 1 --2- 172.21.15.84:0/4045581231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f290c005180 0x7f292014d770 unknown :-1 s=CLOSED pgs=3398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:26.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.494+0000 7f2927ac9640 1 -- 172.21.15.84:0/4045581231 >> 172.21.15.84:0/4045581231 conn(0x7f29200ce840 msgr2=0x7f29200cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:26.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.494+0000 7f2927ac9640 1 -- 172.21.15.84:0/4045581231 shutdown_connections 2023-10-16T11:27:26.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.494+0000 7f2927ac9640 1 -- 172.21.15.84:0/4045581231 wait complete. 2023-10-16T11:27:26.519 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T11:27:26.519 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:27:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.735+0000 7f1aa8bf0640 1 Processor -- start 2023-10-16T11:27:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.735+0000 7f1aa8bf0640 1 -- start start 2023-10-16T11:27:26.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.736+0000 7f1aa8bf0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa4159f00 0x7f1aa40b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:26.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.736+0000 7f1aa8bf0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa406f7f0 con 0x7f1aa415d880 2023-10-16T11:27:26.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.737+0000 7f1aa8bf0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa406c5e0 con 0x7f1aa4159b20 2023-10-16T11:27:26.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.737+0000 7f1aa8bf0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa406c8c0 con 0x7f1aa4159f00 2023-10-16T11:27:26.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.737+0000 7f1aa2d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1aa415d880 0x7f1aa406fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43312/0 (socket says 172.21.15.84:43312) 2023-10-16T11:27:26.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.737+0000 7f1aa2d9d640 1 -- 172.21.15.84:0/2714769581 learned_addr learned my addr 172.21.15.84:0/2714769581 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:26.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.737+0000 7f1a99d9b640 1 --2- 172.21.15.84:0/2714769581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa4159f00 0x7f1aa40b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:26.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.737+0000 7f1aa1d9b640 1 -- 172.21.15.84:0/2714769581 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924514145 0 0) 0x7f1aa406f7f0 con 0x7f1aa415d880 2023-10-16T11:27:26.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.737+0000 7f1aa1d9b640 1 -- 172.21.15.84:0/2714769581 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a84005730 con 0x7f1aa415d880 2023-10-16T11:27:26.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.738+0000 7f1aa1d9b640 1 -- 172.21.15.84:0/2714769581 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2773156281 0 0) 0x7f1a84005730 con 0x7f1aa415d880 2023-10-16T11:27:26.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.738+0000 7f1aa1d9b640 1 -- 172.21.15.84:0/2714769581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa4159f00 msgr2=0x7f1aa40b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:26.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.738+0000 7f1aa1d9b640 1 --2- 172.21.15.84:0/2714769581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa4159f00 0x7f1aa40b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:26.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.738+0000 7f1aa1d9b640 1 -- 172.21.15.84:0/2714769581 >> v1:172.21.15.157:6789/0 conn(0x7f1aa4159b20 legacy=0x7f1aa406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:26.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.738+0000 7f1aa1d9b640 1 -- 172.21.15.84:0/2714769581 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa406cba0 con 0x7f1aa415d880 2023-10-16T11:27:26.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.738+0000 7f1aa1d9b640 1 -- 172.21.15.84:0/2714769581 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f1a9c0019f0 con 0x7f1aa415d880 2023-10-16T11:27:26.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.738+0000 7f1aa1d9b640 1 -- 172.21.15.84:0/2714769581 >> v1:172.21.15.84:6789/0 conn(0x7f1aa415d880 legacy=0x7f1aa406fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:26.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.738+0000 7f1aa1d9b640 1 --2- 172.21.15.84:0/2714769581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a84005bb0 0x7f1a84005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:26.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.738+0000 7f1aa1d9b640 1 --2- 172.21.15.84:0/2714769581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a840069d0 0x7f1a84006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:26.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.738+0000 7f1aa1d9b640 1 --2- 172.21.15.84:0/2714769581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa4159f00 0x7f1a84007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:26.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.738+0000 7f1aa1d9b640 1 -- 172.21.15.84:0/2714769581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa406c8c0 con 0x7f1a840069d0 2023-10-16T11:27:26.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.738+0000 7f1aa1d9b640 1 -- 172.21.15.84:0/2714769581 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8400dd20 con 0x7f1a84005bb0 2023-10-16T11:27:26.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.738+0000 7f1aa1d9b640 1 -- 172.21.15.84:0/2714769581 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8400dee0 con 0x7f1aa4159f00 2023-10-16T11:27:26.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.738+0000 7f1a99d9b640 1 --2- 172.21.15.84:0/2714769581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a840069d0 0x7f1a84006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:26.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.739+0000 7f1aa2d9d640 1 --2- 172.21.15.84:0/2714769581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa4159f00 0x7f1a84007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:26.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.739+0000 7f1a99d9b640 1 -- 172.21.15.84:0/2714769581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa4159f00 msgr2=0x7f1a84007420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:26.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.739+0000 7f1aa259c640 1 --2- 172.21.15.84:0/2714769581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a84005bb0 0x7f1a84005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:26.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.739+0000 7f1a99d9b640 1 --2- 172.21.15.84:0/2714769581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1aa4159f00 0x7f1a84007420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:26.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.739+0000 7f1a99d9b640 1 -- 172.21.15.84:0/2714769581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a84005bb0 msgr2=0x7f1a84005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:26.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.739+0000 7f1a99d9b640 1 --2- 172.21.15.84:0/2714769581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a84005bb0 0x7f1a84005fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:26.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.739+0000 7f1a99d9b640 1 -- 172.21.15.84:0/2714769581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a84005730 con 0x7f1a840069d0 2023-10-16T11:27:26.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.740+0000 7f1a99d9b640 1 --2- 172.21.15.84:0/2714769581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a840069d0 0x7f1a84006e50 secure :-1 s=READY pgs=3400 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00ba00 tx=0x7f1a84007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66cc804a5cc4a514 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:26.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.740+0000 7f1aa1d9b640 1 -- 172.21.15.84:0/2714769581 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c032620 con 0x7f1a840069d0 2023-10-16T11:27:26.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.740+0000 7f1aa1d9b640 1 -- 172.21.15.84:0/2714769581 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c032780 con 0x7f1a840069d0 2023-10-16T11:27:26.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.740+0000 7f1aa1d9b640 1 -- 172.21.15.84:0/2714769581 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c032b10 con 0x7f1a840069d0 2023-10-16T11:27:26.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.741+0000 7f1aa8bf0640 1 -- 172.21.15.84:0/2714769581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a840069d0 msgr2=0x7f1a84006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:26.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.741+0000 7f1aa8bf0640 1 --2- 172.21.15.84:0/2714769581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a840069d0 0x7f1a84006e50 secure :-1 s=READY pgs=3400 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00ba00 tx=0x7f1a84007a00 comp rx=0 tx=0).stop 2023-10-16T11:27:26.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.741+0000 7f1aa2d9d640 1 -- 172.21.15.84:0/2714769581 reap_dead start 2023-10-16T11:27:26.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.741+0000 7f1aa8bf0640 1 -- 172.21.15.84:0/2714769581 shutdown_connections 2023-10-16T11:27:26.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.741+0000 7f1aa8bf0640 1 -- 172.21.15.84:0/2714769581 >> 172.21.15.84:0/2714769581 conn(0x7f1aa40cb410 msgr2=0x7f1aa40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:26.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.741+0000 7f1aa8bf0640 1 -- 172.21.15.84:0/2714769581 shutdown_connections 2023-10-16T11:27:26.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.741+0000 7f1aa8bf0640 1 -- 172.21.15.84:0/2714769581 wait complete. 2023-10-16T11:27:26.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.743+0000 7f1aa8bf0640 1 Processor -- start 2023-10-16T11:27:26.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.743+0000 7f1aa8bf0640 1 -- start start 2023-10-16T11:27:26.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.744+0000 7f1aa8bf0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a840069d0 0x7f1aa415c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:26.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.744+0000 7f1aa8bf0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1aa415a310 0x7f1aa41f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:26.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.744+0000 7f1aa8bf0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa40b3770 0x7f1aa40b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:26.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.744+0000 7f1aa8bf0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa415b490 con 0x7f1aa415a310 2023-10-16T11:27:26.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.744+0000 7f1aa8bf0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa415cd70 con 0x7f1aa40b3770 2023-10-16T11:27:26.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.744+0000 7f1aa8bf0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40b4140 con 0x7f1a840069d0 2023-10-16T11:27:26.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.744+0000 7f1a99d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1aa415a310 0x7f1aa41f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:26.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.744+0000 7f1a99d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1aa415a310 0x7f1aa41f5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42408/0 (socket says 172.21.15.84:42408) 2023-10-16T11:27:26.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.744+0000 7f1a99d9b640 1 -- 172.21.15.84:0/3052048511 learned_addr learned my addr 172.21.15.84:0/3052048511 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:26.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.745+0000 7f1a99d9b640 1 -- 172.21.15.84:0/3052048511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a840069d0 msgr2=0x7f1aa415c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:26.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.745+0000 7f1a99d9b640 1 --2- 172.21.15.84:0/3052048511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a840069d0 0x7f1aa415c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:26.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.745+0000 7f1a99d9b640 1 -- 172.21.15.84:0/3052048511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa40b3770 msgr2=0x7f1aa40b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:26.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.745+0000 7f1a99d9b640 1 --2- 172.21.15.84:0/3052048511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa40b3770 0x7f1aa40b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:26.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.745+0000 7f1a99d9b640 1 -- 172.21.15.84:0/3052048511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa415a810 con 0x7f1aa415a310 2023-10-16T11:27:26.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.745+0000 7f1a99d9b640 1 --2- 172.21.15.84:0/3052048511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1aa415a310 0x7f1aa41f5950 secure :-1 s=READY pgs=3401 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00ba00 tx=0x7f1a8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:26.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.745+0000 7f1a9b7fe640 1 -- 172.21.15.84:0/3052048511 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c004020 con 0x7f1aa415a310 2023-10-16T11:27:26.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.745+0000 7f1a9b7fe640 1 -- 172.21.15.84:0/3052048511 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c004180 con 0x7f1aa415a310 2023-10-16T11:27:26.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.745+0000 7f1a9b7fe640 1 -- 172.21.15.84:0/3052048511 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c0044f0 con 0x7f1aa415a310 2023-10-16T11:27:26.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.745+0000 7f1aa8bf0640 1 -- 172.21.15.84:0/3052048511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa415aaf0 con 0x7f1aa415a310 2023-10-16T11:27:26.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.745+0000 7f1aa8bf0640 1 -- 172.21.15.84:0/3052048511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa415b0b0 con 0x7f1aa415a310 2023-10-16T11:27:26.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.746+0000 7f1aa8bf0640 1 -- 172.21.15.84:0/3052048511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa406c270 con 0x7f1aa415a310 2023-10-16T11:27:26.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.748+0000 7f1a9b7fe640 1 -- 172.21.15.84:0/3052048511 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1a8c003610 con 0x7f1aa415a310 2023-10-16T11:27:26.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.749+0000 7f1a9b7fe640 1 --2- 172.21.15.84:0/3052048511 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f1a8806afc0 0x7f1a8806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:26.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.749+0000 7f1a9b7fe640 1 -- 172.21.15.84:0/3052048511 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1a8c0abc00 con 0x7f1aa415a310 2023-10-16T11:27:26.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.749+0000 7f1aa259c640 1 --2- 172.21.15.84:0/3052048511 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f1a8806afc0 0x7f1a8806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:26.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.750+0000 7f1aa259c640 1 --2- 172.21.15.84:0/3052048511 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f1a8806afc0 0x7f1a8806d480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40c5580 tx=0x7f1a94009040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:26.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.753+0000 7f1a9b7fe640 1 -- 172.21.15.84:0/3052048511 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1a8c0b2080 con 0x7f1aa415a310 2023-10-16T11:27:26.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.982+0000 7f1aa8bf0640 1 -- 172.21.15.84:0/3052048511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1aa406c270 con 0x7f1aa415a310 2023-10-16T11:27:26.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.983+0000 7f1a9b7fe640 1 -- 172.21.15.84:0/3052048511 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+4933 (secure 0 0 0) 0x7f1a8c076a70 con 0x7f1aa415a310 2023-10-16T11:27:26.986 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:26.987 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":157,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7457,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3193159532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3193159532},{"type":"v1","addr":"172.21.15.157:6835","nonce":3193159532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":152},{"gid":8431,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1728897149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1728897149},{"type":"v1","addr":"172.21.15.157:6834","nonce":1728897149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":152},{"gid":13476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3409165899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3409165899},{"type":"v1","addr":"172.21.15.84:6835","nonce":3409165899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":157,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:27:22.871058+0000","modified":"2023-10-16T11:27:25.974995+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":13482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13482":{"gid":13482,"name":"a","rank":0,"incarnation":153,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2806813701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2806813701},{"type":"v1","addr":"172.21.15.84:6837","nonce":2806813701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[37],"metadata_pool":36,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":18}]} 2023-10-16T11:27:26.988 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 157 2023-10-16T11:27:26.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.986+0000 7f1aa8bf0640 1 -- 172.21.15.84:0/3052048511 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f1a8806afc0 msgr2=0x7f1a8806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:26.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.986+0000 7f1aa8bf0640 1 --2- 172.21.15.84:0/3052048511 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f1a8806afc0 0x7f1a8806d480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40c5580 tx=0x7f1a94009040 comp rx=0 tx=0).stop 2023-10-16T11:27:26.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.986+0000 7f1aa8bf0640 1 -- 172.21.15.84:0/3052048511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1aa415a310 msgr2=0x7f1aa41f5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:26.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.986+0000 7f1aa8bf0640 1 --2- 172.21.15.84:0/3052048511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1aa415a310 0x7f1aa41f5950 secure :-1 s=READY pgs=3401 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00ba00 tx=0x7f1a8c004730 comp rx=0 tx=0).stop 2023-10-16T11:27:26.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.986+0000 7f1aa8bf0640 1 -- 172.21.15.84:0/3052048511 shutdown_connections 2023-10-16T11:27:26.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.986+0000 7f1aa8bf0640 1 --2- 172.21.15.84:0/3052048511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1aa40b3770 0x7f1aa40b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:26.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.986+0000 7f1aa8bf0640 1 --2- 172.21.15.84:0/3052048511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1aa415a310 0x7f1aa41f5950 unknown :-1 s=CLOSED pgs=3401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:26.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.986+0000 7f1aa8bf0640 1 --2- 172.21.15.84:0/3052048511 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f1a8806afc0 0x7f1a8806d480 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:26.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.986+0000 7f1aa8bf0640 1 --2- 172.21.15.84:0/3052048511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a840069d0 0x7f1aa415c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:26.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.986+0000 7f1aa8bf0640 1 -- 172.21.15.84:0/3052048511 >> 172.21.15.84:0/3052048511 conn(0x7f1aa40cb410 msgr2=0x7f1aa40ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:26.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.986+0000 7f1aa8bf0640 1 -- 172.21.15.84:0/3052048511 shutdown_connections 2023-10-16T11:27:26.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:26.986+0000 7f1aa8bf0640 1 -- 172.21.15.84:0/3052048511 wait complete. 2023-10-16T11:27:27.008 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:27:27.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.222+0000 7f16df989640 1 Processor -- start 2023-10-16T11:27:27.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.222+0000 7f16df989640 1 -- start start 2023-10-16T11:27:27.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.223+0000 7f16df989640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f16d80cd0c0 0x7f16d81615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:27.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.224+0000 7f16df989640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d815e3e0 con 0x7f16d815db00 2023-10-16T11:27:27.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.224+0000 7f16df989640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d80cd4a0 con 0x7f16d80b0410 2023-10-16T11:27:27.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.224+0000 7f16df989640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d815e100 con 0x7f16d80cd0c0 2023-10-16T11:27:27.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.224+0000 7f16ddf26640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f16d815db00 0x7f16d806f550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43328/0 (socket says 172.21.15.84:43328) 2023-10-16T11:27:27.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.224+0000 7f16ddf26640 1 -- 172.21.15.84:0/419084210 learned_addr learned my addr 172.21.15.84:0/419084210 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:27.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.224+0000 7f16cbfff640 1 -- 172.21.15.84:0/419084210 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234408738 0 0) 0x7f16d815e3e0 con 0x7f16d815db00 2023-10-16T11:27:27.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.224+0000 7f16cbfff640 1 -- 172.21.15.84:0/419084210 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16c4003520 con 0x7f16d815db00 2023-10-16T11:27:27.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.224+0000 7f16dd725640 1 --2- 172.21.15.84:0/419084210 >> v2:172.21.15.157:3300/0 conn(0x7f16d80cd0c0 0x7f16d81615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:27.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.225+0000 7f16cbfff640 1 -- 172.21.15.84:0/419084210 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 656910362 0 0) 0x7f16c4003520 con 0x7f16d815db00 2023-10-16T11:27:27.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.225+0000 7f16cbfff640 1 -- 172.21.15.84:0/419084210 >> v2:172.21.15.157:3300/0 conn(0x7f16d80cd0c0 msgr2=0x7f16d81615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:27.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.225+0000 7f16cbfff640 1 --2- 172.21.15.84:0/419084210 >> v2:172.21.15.157:3300/0 conn(0x7f16d80cd0c0 0x7f16d81615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.225+0000 7f16cbfff640 1 -- 172.21.15.84:0/419084210 >> v1:172.21.15.157:6789/0 conn(0x7f16d80b0410 legacy=0x7f16d815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:27.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.225+0000 7f16cbfff640 1 -- 172.21.15.84:0/419084210 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d8160b10 con 0x7f16d815db00 2023-10-16T11:27:27.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.225+0000 7f16cbfff640 1 -- 172.21.15.84:0/419084210 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f16d00019f0 con 0x7f16d815db00 2023-10-16T11:27:27.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.225+0000 7f16cbfff640 1 -- 172.21.15.84:0/419084210 >> v1:172.21.15.84:6789/0 conn(0x7f16d815db00 legacy=0x7f16d806f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:27.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.225+0000 7f16cbfff640 1 --2- 172.21.15.84:0/419084210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16c4003970 0x7f16c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:27.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.225+0000 7f16cbfff640 1 --2- 172.21.15.84:0/419084210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c40047c0 0x7f16c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:27.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.225+0000 7f16cbfff640 1 --2- 172.21.15.84:0/419084210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c4005180 0x7f16c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:27.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.225+0000 7f16cbfff640 1 -- 172.21.15.84:0/419084210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d815e100 con 0x7f16c4005180 2023-10-16T11:27:27.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.225+0000 7f16cbfff640 1 -- 172.21.15.84:0/419084210 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c400c120 con 0x7f16c40047c0 2023-10-16T11:27:27.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.225+0000 7f16cbfff640 1 -- 172.21.15.84:0/419084210 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c400c2e0 con 0x7f16c4003970 2023-10-16T11:27:27.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.225+0000 7f16ddf26640 1 --2- 172.21.15.84:0/419084210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c4005180 0x7f16c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:27.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.226+0000 7f16dd725640 1 --2- 172.21.15.84:0/419084210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16c4003970 0x7f16c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:27.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.226+0000 7f16dcf24640 1 --2- 172.21.15.84:0/419084210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c40047c0 0x7f16c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:27.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.226+0000 7f16ddf26640 1 -- 172.21.15.84:0/419084210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16c4003970 msgr2=0x7f16c4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:27.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.226+0000 7f16ddf26640 1 --2- 172.21.15.84:0/419084210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16c4003970 0x7f16c4003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.226+0000 7f16ddf26640 1 -- 172.21.15.84:0/419084210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c40047c0 msgr2=0x7f16c4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:27.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.226+0000 7f16ddf26640 1 --2- 172.21.15.84:0/419084210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c40047c0 0x7f16c4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.226+0000 7f16ddf26640 1 -- 172.21.15.84:0/419084210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c4003520 con 0x7f16c4005180 2023-10-16T11:27:27.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.226+0000 7f16dcf24640 1 --2- 172.21.15.84:0/419084210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c40047c0 0x7f16c4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:27.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.227+0000 7f16ddf26640 1 -- 172.21.15.84:0/419084210 reap_dead start 2023-10-16T11:27:27.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.227+0000 7f16ddf26640 1 --2- 172.21.15.84:0/419084210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c4005180 0x7f16c400b980 secure :-1 s=READY pgs=3403 cs=0 l=1 rev1=1 crypto rx=0x7f16d000a830 tx=0x7f16c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ab9fa7b651940b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:27.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.227+0000 7f16cbfff640 1 -- 172.21.15.84:0/419084210 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d00072d0 con 0x7f16c4005180 2023-10-16T11:27:27.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.227+0000 7f16cbfff640 1 -- 172.21.15.84:0/419084210 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d002f070 con 0x7f16c4005180 2023-10-16T11:27:27.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.227+0000 7f16cbfff640 1 -- 172.21.15.84:0/419084210 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d00076b0 con 0x7f16c4005180 2023-10-16T11:27:27.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.228+0000 7f16df989640 1 -- 172.21.15.84:0/419084210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c4005180 msgr2=0x7f16c400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:27.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.228+0000 7f16df989640 1 --2- 172.21.15.84:0/419084210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c4005180 0x7f16c400b980 secure :-1 s=READY pgs=3403 cs=0 l=1 rev1=1 crypto rx=0x7f16d000a830 tx=0x7f16c40057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.228+0000 7f16df989640 1 -- 172.21.15.84:0/419084210 shutdown_connections 2023-10-16T11:27:27.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.228+0000 7f16df989640 1 --2- 172.21.15.84:0/419084210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c4005180 0x7f16c400b980 unknown :-1 s=CLOSED pgs=3403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.228+0000 7f16df989640 1 -- 172.21.15.84:0/419084210 >> 172.21.15.84:0/419084210 conn(0x7f16d80cb410 msgr2=0x7f16d80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:27.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.228+0000 7f16df989640 1 -- 172.21.15.84:0/419084210 shutdown_connections 2023-10-16T11:27:27.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.228+0000 7f16df989640 1 -- 172.21.15.84:0/419084210 wait complete. 2023-10-16T11:27:27.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.230+0000 7f16df989640 1 Processor -- start 2023-10-16T11:27:27.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.230+0000 7f16df989640 1 -- start start 2023-10-16T11:27:27.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.231+0000 7f16df989640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16c4005180 0x7f16d8156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:27.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.231+0000 7f16df989640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16d80c4f00 0x7f16d80b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:27.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.231+0000 7f16df989640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d815f5e0 0x7f16d815fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:27.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.231+0000 7f16df989640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d815ffa0 con 0x7f16d815f5e0 2023-10-16T11:27:27.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.231+0000 7f16df989640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80b2ce0 con 0x7f16d80c4f00 2023-10-16T11:27:27.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.231+0000 7f16df989640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d8156d70 con 0x7f16c4005180 2023-10-16T11:27:27.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.231+0000 7f16ddf26640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d815f5e0 0x7f16d815fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:27.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.231+0000 7f16ddf26640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d815f5e0 0x7f16d815fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42412/0 (socket says 172.21.15.84:42412) 2023-10-16T11:27:27.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.231+0000 7f16ddf26640 1 -- 172.21.15.84:0/3172024496 learned_addr learned my addr 172.21.15.84:0/3172024496 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:27.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.231+0000 7f16dd725640 1 --2- 172.21.15.84:0/3172024496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16c4005180 0x7f16d8156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:27.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.231+0000 7f16dcf24640 1 --2- 172.21.15.84:0/3172024496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16d80c4f00 0x7f16d80b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:27.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.232+0000 7f16ddf26640 1 -- 172.21.15.84:0/3172024496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16c4005180 msgr2=0x7f16d8156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:27.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.232+0000 7f16ddf26640 1 --2- 172.21.15.84:0/3172024496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16c4005180 0x7f16d8156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.232+0000 7f16ddf26640 1 -- 172.21.15.84:0/3172024496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16d80c4f00 msgr2=0x7f16d80b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:27.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.232+0000 7f16ddf26640 1 --2- 172.21.15.84:0/3172024496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16d80c4f00 0x7f16d80b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.232+0000 7f16ddf26640 1 -- 172.21.15.84:0/3172024496 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d80b20d0 con 0x7f16d815f5e0 2023-10-16T11:27:27.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.232+0000 7f16dd725640 1 --2- 172.21.15.84:0/3172024496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16c4005180 0x7f16d8156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:27.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.232+0000 7f16dcf24640 1 --2- 172.21.15.84:0/3172024496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16d80c4f00 0x7f16d80b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:27.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.232+0000 7f16ddf26640 1 --2- 172.21.15.84:0/3172024496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d815f5e0 0x7f16d815fa60 secure :-1 s=READY pgs=3404 cs=0 l=1 rev1=1 crypto rx=0x7f16d002cc40 tx=0x7f16d0001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:27.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.232+0000 7f16c9ffb640 1 -- 172.21.15.84:0/3172024496 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d0001930 con 0x7f16d815f5e0 2023-10-16T11:27:27.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.232+0000 7f16c9ffb640 1 -- 172.21.15.84:0/3172024496 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d0001250 con 0x7f16d815f5e0 2023-10-16T11:27:27.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.232+0000 7f16c9ffb640 1 -- 172.21.15.84:0/3172024496 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d0004da0 con 0x7f16d815f5e0 2023-10-16T11:27:27.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.232+0000 7f16df989640 1 -- 172.21.15.84:0/3172024496 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d80b23b0 con 0x7f16d815f5e0 2023-10-16T11:27:27.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.232+0000 7f16df989640 1 -- 172.21.15.84:0/3172024496 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d80b2970 con 0x7f16d815f5e0 2023-10-16T11:27:27.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.233+0000 7f16df989640 1 -- 172.21.15.84:0/3172024496 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d804e160 con 0x7f16d815f5e0 2023-10-16T11:27:27.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.234+0000 7f16c9ffb640 1 -- 172.21.15.84:0/3172024496 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f16d0031030 con 0x7f16d815f5e0 2023-10-16T11:27:27.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.235+0000 7f16c9ffb640 1 --2- 172.21.15.84:0/3172024496 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f16bc06afc0 0x7f16bc06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:27.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.235+0000 7f16c9ffb640 1 -- 172.21.15.84:0/3172024496 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f16d00aaab0 con 0x7f16d815f5e0 2023-10-16T11:27:27.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.235+0000 7f16dd725640 1 --2- 172.21.15.84:0/3172024496 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f16bc06afc0 0x7f16bc06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:27.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.236+0000 7f16dd725640 1 --2- 172.21.15.84:0/3172024496 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f16bc06afc0 0x7f16bc06d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f16d8157280 tx=0x7f16d4008040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:27.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.239+0000 7f16c9ffb640 1 -- 172.21.15.84:0/3172024496 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f16d0075920 con 0x7f16d815f5e0 2023-10-16T11:27:27.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.435+0000 7f16df989640 1 -- 172.21.15.84:0/3172024496 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16d80b4910 con 0x7f16d815f5e0 2023-10-16T11:27:27.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.437+0000 7f16c9ffb640 1 -- 172.21.15.84:0/3172024496 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v270) v1 ==== 74+0+16547 (secure 0 0 0) 0x7f16d0043020 con 0x7f16d815f5e0 2023-10-16T11:27:27.439 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:27.440 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":270,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:27:24.433803+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:27:20.716240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"269","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_data","create_time":"2023-10-16T11:27:21.905553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"270","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:27:27.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.439+0000 7f16df989640 1 -- 172.21.15.84:0/3172024496 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f16bc06afc0 msgr2=0x7f16bc06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:27.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.439+0000 7f16df989640 1 --2- 172.21.15.84:0/3172024496 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f16bc06afc0 0x7f16bc06d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f16d8157280 tx=0x7f16d4008040 comp rx=0 tx=0).stop 2023-10-16T11:27:27.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.439+0000 7f16df989640 1 -- 172.21.15.84:0/3172024496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d815f5e0 msgr2=0x7f16d815fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:27.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.439+0000 7f16df989640 1 --2- 172.21.15.84:0/3172024496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d815f5e0 0x7f16d815fa60 secure :-1 s=READY pgs=3404 cs=0 l=1 rev1=1 crypto rx=0x7f16d002cc40 tx=0x7f16d0001d80 comp rx=0 tx=0).stop 2023-10-16T11:27:27.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.439+0000 7f16df989640 1 -- 172.21.15.84:0/3172024496 shutdown_connections 2023-10-16T11:27:27.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.439+0000 7f16df989640 1 --2- 172.21.15.84:0/3172024496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d815f5e0 0x7f16d815fa60 unknown :-1 s=CLOSED pgs=3404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.439+0000 7f16df989640 1 --2- 172.21.15.84:0/3172024496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16d80c4f00 0x7f16d80b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.439+0000 7f16df989640 1 --2- 172.21.15.84:0/3172024496 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f16bc06afc0 0x7f16bc06d480 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.439+0000 7f16df989640 1 --2- 172.21.15.84:0/3172024496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16c4005180 0x7f16d8156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.439+0000 7f16df989640 1 -- 172.21.15.84:0/3172024496 >> 172.21.15.84:0/3172024496 conn(0x7f16d80cb410 msgr2=0x7f16d80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:27.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.440+0000 7f16df989640 1 -- 172.21.15.84:0/3172024496 shutdown_connections 2023-10-16T11:27:27.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.440+0000 7f16df989640 1 -- 172.21.15.84:0/3172024496 wait complete. 2023-10-16T11:27:27.461 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T11:27:27.462 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:27:27.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.663+0000 7fd83750f640 1 Processor -- start 2023-10-16T11:27:27.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.663+0000 7fd83750f640 1 -- start start 2023-10-16T11:27:27.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.665+0000 7fd83750f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd83015aaa0 0x7fd83015e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:27.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.665+0000 7fd83750f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83015ed60 con 0x7fd83015ae80 2023-10-16T11:27:27.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.665+0000 7fd83750f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8300707e0 con 0x7fd8300ce390 2023-10-16T11:27:27.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.665+0000 7fd83750f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83006b810 con 0x7fd83015aaa0 2023-10-16T11:27:27.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.665+0000 7fd834aaa640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd83015ae80 0x7fd8300cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43340/0 (socket says 172.21.15.84:43340) 2023-10-16T11:27:27.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.665+0000 7fd834aaa640 1 -- 172.21.15.84:0/1661558806 learned_addr learned my addr 172.21.15.84:0/1661558806 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:27.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.665+0000 7fd827fff640 1 -- 172.21.15.84:0/1661558806 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5833814 0 0) 0x7fd83015ed60 con 0x7fd83015ae80 2023-10-16T11:27:27.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.665+0000 7fd827fff640 1 -- 172.21.15.84:0/1661558806 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd81c003520 con 0x7fd83015ae80 2023-10-16T11:27:27.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.665+0000 7fd8352ab640 1 --2- 172.21.15.84:0/1661558806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd83015aaa0 0x7fd83015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:27.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.666+0000 7fd827fff640 1 -- 172.21.15.84:0/1661558806 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2562175876 0 0) 0x7fd81c003520 con 0x7fd83015ae80 2023-10-16T11:27:27.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.666+0000 7fd827fff640 1 -- 172.21.15.84:0/1661558806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd83015aaa0 msgr2=0x7fd83015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:27.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.666+0000 7fd827fff640 1 --2- 172.21.15.84:0/1661558806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd83015aaa0 0x7fd83015e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.666+0000 7fd827fff640 1 -- 172.21.15.84:0/1661558806 >> v1:172.21.15.157:6789/0 conn(0x7fd8300ce390 legacy=0x7fd830070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:27.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.666+0000 7fd827fff640 1 -- 172.21.15.84:0/1661558806 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83015ef90 con 0x7fd83015ae80 2023-10-16T11:27:27.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.666+0000 7fd827fff640 1 -- 172.21.15.84:0/1661558806 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd820001670 con 0x7fd83015ae80 2023-10-16T11:27:27.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.666+0000 7fd827fff640 1 -- 172.21.15.84:0/1661558806 >> v1:172.21.15.84:6789/0 conn(0x7fd83015ae80 legacy=0x7fd8300cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:27.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.666+0000 7fd827fff640 1 --2- 172.21.15.84:0/1661558806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd83015aaa0 0x7fd81c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:27.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.666+0000 7fd827fff640 1 --2- 172.21.15.84:0/1661558806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd81c003fd0 0x7fd81c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:27.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.666+0000 7fd827fff640 1 --2- 172.21.15.84:0/1661558806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd81c004d00 0x7fd81c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:27.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.666+0000 7fd827fff640 1 -- 172.21.15.84:0/1661558806 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83006b810 con 0x7fd81c004d00 2023-10-16T11:27:27.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.666+0000 7fd827fff640 1 -- 172.21.15.84:0/1661558806 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c00bb40 con 0x7fd81c003fd0 2023-10-16T11:27:27.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.666+0000 7fd827fff640 1 -- 172.21.15.84:0/1661558806 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c00bd00 con 0x7fd83015aaa0 2023-10-16T11:27:27.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.666+0000 7fd835aac640 1 --2- 172.21.15.84:0/1661558806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd81c004d00 0x7fd81c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:27.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.667+0000 7fd8352ab640 1 --2- 172.21.15.84:0/1661558806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd83015aaa0 0x7fd81c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:27.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.667+0000 7fd835aac640 1 -- 172.21.15.84:0/1661558806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd83015aaa0 msgr2=0x7fd81c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:27.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.667+0000 7fd835aac640 1 --2- 172.21.15.84:0/1661558806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd83015aaa0 0x7fd81c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.667+0000 7fd835aac640 1 -- 172.21.15.84:0/1661558806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd81c003fd0 msgr2=0x7fd81c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:27.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.667+0000 7fd835aac640 1 --2- 172.21.15.84:0/1661558806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd81c003fd0 0x7fd81c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.667+0000 7fd835aac640 1 -- 172.21.15.84:0/1661558806 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c003520 con 0x7fd81c004d00 2023-10-16T11:27:27.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.667+0000 7fd8352ab640 1 --2- 172.21.15.84:0/1661558806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd83015aaa0 0x7fd81c0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:27.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.668+0000 7fd835aac640 1 --2- 172.21.15.84:0/1661558806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd81c004d00 0x7fd81c005180 secure :-1 s=READY pgs=3406 cs=0 l=1 rev1=1 crypto rx=0x7fd828001e50 tx=0x7fd81c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83be7d260f1cf1aa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:27.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.668+0000 7fd827fff640 1 -- 172.21.15.84:0/1661558806 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828003b70 con 0x7fd81c004d00 2023-10-16T11:27:27.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.668+0000 7fd827fff640 1 -- 172.21.15.84:0/1661558806 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd828003cd0 con 0x7fd81c004d00 2023-10-16T11:27:27.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.668+0000 7fd827fff640 1 -- 172.21.15.84:0/1661558806 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828005310 con 0x7fd81c004d00 2023-10-16T11:27:27.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.668+0000 7fd83750f640 1 -- 172.21.15.84:0/1661558806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd81c004d00 msgr2=0x7fd81c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:27.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.668+0000 7fd83750f640 1 --2- 172.21.15.84:0/1661558806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd81c004d00 0x7fd81c005180 secure :-1 s=READY pgs=3406 cs=0 l=1 rev1=1 crypto rx=0x7fd828001e50 tx=0x7fd81c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.669+0000 7fd835aac640 1 -- 172.21.15.84:0/1661558806 reap_dead start 2023-10-16T11:27:27.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.669+0000 7fd83750f640 1 -- 172.21.15.84:0/1661558806 shutdown_connections 2023-10-16T11:27:27.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.669+0000 7fd83750f640 1 -- 172.21.15.84:0/1661558806 >> 172.21.15.84:0/1661558806 conn(0x7fd8300cb410 msgr2=0x7fd8300cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:27.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.669+0000 7fd83750f640 1 -- 172.21.15.84:0/1661558806 shutdown_connections 2023-10-16T11:27:27.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.669+0000 7fd83750f640 1 -- 172.21.15.84:0/1661558806 wait complete. 2023-10-16T11:27:27.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.670+0000 7fd83750f640 1 Processor -- start 2023-10-16T11:27:27.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.671+0000 7fd83750f640 1 -- start start 2023-10-16T11:27:27.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.671+0000 7fd83750f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd830145a30 0x7fd830147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:27.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.671+0000 7fd83750f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd830148fe0 0x7fd830146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:27.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.671+0000 7fd83750f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8300b3580 0x7fd8300b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:27.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.671+0000 7fd83750f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300b3f50 con 0x7fd8300b3580 2023-10-16T11:27:27.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.672+0000 7fd83750f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd830146ba0 con 0x7fd830145a30 2023-10-16T11:27:27.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.672+0000 7fd83750f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd830146e80 con 0x7fd830148fe0 2023-10-16T11:27:27.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.672+0000 7fd835aac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8300b3580 0x7fd8300b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:27.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.672+0000 7fd835aac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8300b3580 0x7fd8300b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42428/0 (socket says 172.21.15.84:42428) 2023-10-16T11:27:27.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.672+0000 7fd835aac640 1 -- 172.21.15.84:0/3477946138 learned_addr learned my addr 172.21.15.84:0/3477946138 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:27.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.672+0000 7fd8352ab640 1 --2- 172.21.15.84:0/3477946138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd830145a30 0x7fd830147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:27.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.672+0000 7fd834aaa640 1 --2- 172.21.15.84:0/3477946138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd830148fe0 0x7fd830146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:27.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.672+0000 7fd835aac640 1 -- 172.21.15.84:0/3477946138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd830148fe0 msgr2=0x7fd830146230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:27.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.672+0000 7fd835aac640 1 --2- 172.21.15.84:0/3477946138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd830148fe0 0x7fd830146230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.672+0000 7fd835aac640 1 -- 172.21.15.84:0/3477946138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd830145a30 msgr2=0x7fd830147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:27.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.672+0000 7fd835aac640 1 --2- 172.21.15.84:0/3477946138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd830145a30 0x7fd830147c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.672+0000 7fd835aac640 1 -- 172.21.15.84:0/3477946138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd830147160 con 0x7fd8300b3580 2023-10-16T11:27:27.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.672+0000 7fd834aaa640 1 --2- 172.21.15.84:0/3477946138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd830148fe0 0x7fd830146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:27.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.673+0000 7fd835aac640 1 --2- 172.21.15.84:0/3477946138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8300b3580 0x7fd8300b3a10 secure :-1 s=READY pgs=3407 cs=0 l=1 rev1=1 crypto rx=0x7fd828001f70 tx=0x7fd8280014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:27.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.673+0000 7fd825ffb640 1 -- 172.21.15.84:0/3477946138 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82803e070 con 0x7fd8300b3580 2023-10-16T11:27:27.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.673+0000 7fd825ffb640 1 -- 172.21.15.84:0/3477946138 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd828007db0 con 0x7fd8300b3580 2023-10-16T11:27:27.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.673+0000 7fd825ffb640 1 -- 172.21.15.84:0/3477946138 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828006d70 con 0x7fd8300b3580 2023-10-16T11:27:27.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.673+0000 7fd83750f640 1 -- 172.21.15.84:0/3477946138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd830147440 con 0x7fd8300b3580 2023-10-16T11:27:27.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.673+0000 7fd83750f640 1 -- 172.21.15.84:0/3477946138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd830145e10 con 0x7fd8300b3580 2023-10-16T11:27:27.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.674+0000 7fd83750f640 1 -- 172.21.15.84:0/3477946138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd830145fa0 con 0x7fd8300b3580 2023-10-16T11:27:27.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.675+0000 7fd825ffb640 1 -- 172.21.15.84:0/3477946138 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd828007030 con 0x7fd8300b3580 2023-10-16T11:27:27.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.676+0000 7fd825ffb640 1 --2- 172.21.15.84:0/3477946138 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd81406af90 0x7fd81406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:27.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.676+0000 7fd825ffb640 1 -- 172.21.15.84:0/3477946138 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd8280aa9e0 con 0x7fd8300b3580 2023-10-16T11:27:27.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.676+0000 7fd8352ab640 1 --2- 172.21.15.84:0/3477946138 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd81406af90 0x7fd81406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:27.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.677+0000 7fd8352ab640 1 --2- 172.21.15.84:0/3477946138 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd81406af90 0x7fd81406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd8301f6570 tx=0x7fd82c008040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:27.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.680+0000 7fd825ffb640 1 -- 172.21.15.84:0/3477946138 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd8280758e0 con 0x7fd8300b3580 2023-10-16T11:27:27.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.868+0000 7fd83750f640 1 -- 172.21.15.84:0/3477946138 --> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd83015f550 con 0x7fd81406af90 2023-10-16T11:27:27.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.881+0000 7fd825ffb640 1 -- 172.21.15.84:0/3477946138 <== mgr.13239 v2:172.21.15.84:6832/731888425 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287615 (secure 0 0 0) 0x7fd82c00ce40 con 0x7fd81406af90 2023-10-16T11:27:27.887 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:27.888 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:27:27.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.886+0000 7fd83750f640 1 -- 172.21.15.84:0/3477946138 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd81406af90 msgr2=0x7fd81406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:27.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.886+0000 7fd83750f640 1 --2- 172.21.15.84:0/3477946138 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd81406af90 0x7fd81406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd8301f6570 tx=0x7fd82c008040 comp rx=0 tx=0).stop 2023-10-16T11:27:27.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.886+0000 7fd83750f640 1 -- 172.21.15.84:0/3477946138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8300b3580 msgr2=0x7fd8300b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:27.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.886+0000 7fd83750f640 1 --2- 172.21.15.84:0/3477946138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8300b3580 0x7fd8300b3a10 secure :-1 s=READY pgs=3407 cs=0 l=1 rev1=1 crypto rx=0x7fd828001f70 tx=0x7fd8280014a0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.886+0000 7fd83750f640 1 -- 172.21.15.84:0/3477946138 shutdown_connections 2023-10-16T11:27:27.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.886+0000 7fd83750f640 1 --2- 172.21.15.84:0/3477946138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8300b3580 0x7fd8300b3a10 unknown :-1 s=CLOSED pgs=3407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.886+0000 7fd83750f640 1 --2- 172.21.15.84:0/3477946138 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fd81406af90 0x7fd81406d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.886+0000 7fd83750f640 1 --2- 172.21.15.84:0/3477946138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd830148fe0 0x7fd830146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.886+0000 7fd83750f640 1 --2- 172.21.15.84:0/3477946138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd830145a30 0x7fd830147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:27.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.887+0000 7fd83750f640 1 -- 172.21.15.84:0/3477946138 >> 172.21.15.84:0/3477946138 conn(0x7fd8300cb410 msgr2=0x7fd8300cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:27.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.887+0000 7fd83750f640 1 -- 172.21.15.84:0/3477946138 shutdown_connections 2023-10-16T11:27:27.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:27.887+0000 7fd83750f640 1 -- 172.21.15.84:0/3477946138 wait complete. 2023-10-16T11:27:27.907 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-16T11:27:27.528674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2687,"num_read_kb":5529,"num_write":727,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4863,"ondisk_log_size":4863,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":479984,"kb_used_data":27900,"kb_used_omap":12,"kb_used_meta":451955,"kb_avail":754494736,"statfs":{"total":773094113280,"available":772602609664,"internally_reserved":0,"allocated":28569600,"data_stored":14561452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":462802489},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001059"},"pg_stats":[{"pgid":"37.3f","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.746978+0000","last_change":"2023-10-16T11:27:23.468831+0000","last_active":"2023-10-16T11:27:25.746978+0000","last_peered":"2023-10-16T11:27:25.746978+0000","last_clean":"2023-10-16T11:27:25.746978+0000","last_became_active":"2023-10-16T11:27:23.468594+0000","last_became_peered":"2023-10-16T11:27:23.468594+0000","last_unstale":"2023-10-16T11:27:25.746978+0000","last_undegraded":"2023-10-16T11:27:25.746978+0000","last_fullsized":"2023-10-16T11:27:25.746978+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:32:14.401834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.3e","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.443450+0000","last_change":"2023-10-16T11:27:23.467568+0000","last_active":"2023-10-16T11:27:24.443450+0000","last_peered":"2023-10-16T11:27:24.443450+0000","last_clean":"2023-10-16T11:27:24.443450+0000","last_became_active":"2023-10-16T11:27:23.467213+0000","last_became_peered":"2023-10-16T11:27:23.467213+0000","last_unstale":"2023-10-16T11:27:24.443450+0000","last_undegraded":"2023-10-16T11:27:24.443450+0000","last_fullsized":"2023-10-16T11:27:24.443450+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:02.508652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.3d","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.440663+0000","last_change":"2023-10-16T11:27:23.454799+0000","last_active":"2023-10-16T11:27:24.440663+0000","last_peered":"2023-10-16T11:27:24.440663+0000","last_clean":"2023-10-16T11:27:24.440663+0000","last_became_active":"2023-10-16T11:27:23.454408+0000","last_became_peered":"2023-10-16T11:27:23.454408+0000","last_unstale":"2023-10-16T11:27:24.440663+0000","last_undegraded":"2023-10-16T11:27:24.440663+0000","last_fullsized":"2023-10-16T11:27:24.440663+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:01.719295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.3c","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.440471+0000","last_change":"2023-10-16T11:27:23.455745+0000","last_active":"2023-10-16T11:27:24.440471+0000","last_peered":"2023-10-16T11:27:24.440471+0000","last_clean":"2023-10-16T11:27:24.440471+0000","last_became_active":"2023-10-16T11:27:23.454456+0000","last_became_peered":"2023-10-16T11:27:23.454456+0000","last_unstale":"2023-10-16T11:27:24.440471+0000","last_undegraded":"2023-10-16T11:27:24.440471+0000","last_fullsized":"2023-10-16T11:27:24.440471+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:09.556639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.3b","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.440683+0000","last_change":"2023-10-16T11:27:23.471412+0000","last_active":"2023-10-16T11:27:24.440683+0000","last_peered":"2023-10-16T11:27:24.440683+0000","last_clean":"2023-10-16T11:27:24.440683+0000","last_became_active":"2023-10-16T11:27:23.470036+0000","last_became_peered":"2023-10-16T11:27:23.470036+0000","last_unstale":"2023-10-16T11:27:24.440683+0000","last_undegraded":"2023-10-16T11:27:24.440683+0000","last_fullsized":"2023-10-16T11:27:24.440683+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:52.119044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.3a","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.741736+0000","last_change":"2023-10-16T11:27:23.476060+0000","last_active":"2023-10-16T11:27:25.741736+0000","last_peered":"2023-10-16T11:27:25.741736+0000","last_clean":"2023-10-16T11:27:25.741736+0000","last_became_active":"2023-10-16T11:27:23.474212+0000","last_became_peered":"2023-10-16T11:27:23.474212+0000","last_unstale":"2023-10-16T11:27:25.741736+0000","last_undegraded":"2023-10-16T11:27:25.741736+0000","last_fullsized":"2023-10-16T11:27:25.741736+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:40.261473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.39","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.744540+0000","last_change":"2023-10-16T11:27:23.460009+0000","last_active":"2023-10-16T11:27:25.744540+0000","last_peered":"2023-10-16T11:27:25.744540+0000","last_clean":"2023-10-16T11:27:25.744540+0000","last_became_active":"2023-10-16T11:27:23.459356+0000","last_became_peered":"2023-10-16T11:27:23.459356+0000","last_unstale":"2023-10-16T11:27:25.744540+0000","last_undegraded":"2023-10-16T11:27:25.744540+0000","last_fullsized":"2023-10-16T11:27:25.744540+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:55.507612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.38","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.531882+0000","last_change":"2023-10-16T11:27:23.458731+0000","last_active":"2023-10-16T11:27:25.531882+0000","last_peered":"2023-10-16T11:27:25.531882+0000","last_clean":"2023-10-16T11:27:25.531882+0000","last_became_active":"2023-10-16T11:27:23.458425+0000","last_became_peered":"2023-10-16T11:27:23.458425+0000","last_unstale":"2023-10-16T11:27:25.531882+0000","last_undegraded":"2023-10-16T11:27:25.531882+0000","last_fullsized":"2023-10-16T11:27:25.531882+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:00:42.986177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.37","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.443257+0000","last_change":"2023-10-16T11:27:23.468135+0000","last_active":"2023-10-16T11:27:24.443257+0000","last_peered":"2023-10-16T11:27:24.443257+0000","last_clean":"2023-10-16T11:27:24.443257+0000","last_became_active":"2023-10-16T11:27:23.467862+0000","last_became_peered":"2023-10-16T11:27:23.467862+0000","last_unstale":"2023-10-16T11:27:24.443257+0000","last_undegraded":"2023-10-16T11:27:24.443257+0000","last_fullsized":"2023-10-16T11:27:24.443257+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:54:48.482099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.36","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.745976+0000","last_change":"2023-10-16T11:27:23.460169+0000","last_active":"2023-10-16T11:27:25.745976+0000","last_peered":"2023-10-16T11:27:25.745976+0000","last_clean":"2023-10-16T11:27:25.745976+0000","last_became_active":"2023-10-16T11:27:23.459597+0000","last_became_peered":"2023-10-16T11:27:23.459597+0000","last_unstale":"2023-10-16T11:27:25.745976+0000","last_undegraded":"2023-10-16T11:27:25.745976+0000","last_fullsized":"2023-10-16T11:27:25.745976+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:56.725852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.35","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.532297+0000","last_change":"2023-10-16T11:27:23.470722+0000","last_active":"2023-10-16T11:27:25.532297+0000","last_peered":"2023-10-16T11:27:25.532297+0000","last_clean":"2023-10-16T11:27:25.532297+0000","last_became_active":"2023-10-16T11:27:23.470462+0000","last_became_peered":"2023-10-16T11:27:23.470462+0000","last_unstale":"2023-10-16T11:27:25.532297+0000","last_undegraded":"2023-10-16T11:27:25.532297+0000","last_fullsized":"2023-10-16T11:27:25.532297+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:36.797546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.34","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.471995+0000","last_change":"2023-10-16T11:27:23.466997+0000","last_active":"2023-10-16T11:27:24.471995+0000","last_peered":"2023-10-16T11:27:24.471995+0000","last_clean":"2023-10-16T11:27:24.471995+0000","last_became_active":"2023-10-16T11:27:23.466671+0000","last_became_peered":"2023-10-16T11:27:23.466671+0000","last_unstale":"2023-10-16T11:27:24.471995+0000","last_undegraded":"2023-10-16T11:27:24.471995+0000","last_fullsized":"2023-10-16T11:27:24.471995+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:57.314448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.33","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.746072+0000","last_change":"2023-10-16T11:27:23.479624+0000","last_active":"2023-10-16T11:27:25.746072+0000","last_peered":"2023-10-16T11:27:25.746072+0000","last_clean":"2023-10-16T11:27:25.746072+0000","last_became_active":"2023-10-16T11:27:23.479327+0000","last_became_peered":"2023-10-16T11:27:23.479327+0000","last_unstale":"2023-10-16T11:27:25.746072+0000","last_undegraded":"2023-10-16T11:27:25.746072+0000","last_fullsized":"2023-10-16T11:27:25.746072+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:57:41.546314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.32","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.442918+0000","last_change":"2023-10-16T11:27:23.461838+0000","last_active":"2023-10-16T11:27:24.442918+0000","last_peered":"2023-10-16T11:27:24.442918+0000","last_clean":"2023-10-16T11:27:24.442918+0000","last_became_active":"2023-10-16T11:27:23.460967+0000","last_became_peered":"2023-10-16T11:27:23.460967+0000","last_unstale":"2023-10-16T11:27:24.442918+0000","last_undegraded":"2023-10-16T11:27:24.442918+0000","last_fullsized":"2023-10-16T11:27:24.442918+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:09.863277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.31","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.440134+0000","last_change":"2023-10-16T11:27:23.485618+0000","last_active":"2023-10-16T11:27:24.440134+0000","last_peered":"2023-10-16T11:27:24.440134+0000","last_clean":"2023-10-16T11:27:24.440134+0000","last_became_active":"2023-10-16T11:27:23.485294+0000","last_became_peered":"2023-10-16T11:27:23.485294+0000","last_unstale":"2023-10-16T11:27:24.440134+0000","last_undegraded":"2023-10-16T11:27:24.440134+0000","last_fullsized":"2023-10-16T11:27:24.440134+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:30:41.982564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.30","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.470496+0000","last_change":"2023-10-16T11:27:23.484857+0000","last_active":"2023-10-16T11:27:24.470496+0000","last_peered":"2023-10-16T11:27:24.470496+0000","last_clean":"2023-10-16T11:27:24.470496+0000","last_became_active":"2023-10-16T11:27:23.484530+0000","last_became_peered":"2023-10-16T11:27:23.484530+0000","last_unstale":"2023-10-16T11:27:24.470496+0000","last_undegraded":"2023-10-16T11:27:24.470496+0000","last_fullsized":"2023-10-16T11:27:24.470496+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:52:11.968883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.2f","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.746838+0000","last_change":"2023-10-16T11:27:23.482344+0000","last_active":"2023-10-16T11:27:25.746838+0000","last_peered":"2023-10-16T11:27:25.746838+0000","last_clean":"2023-10-16T11:27:25.746838+0000","last_became_active":"2023-10-16T11:27:23.480710+0000","last_became_peered":"2023-10-16T11:27:23.480710+0000","last_unstale":"2023-10-16T11:27:25.746838+0000","last_undegraded":"2023-10-16T11:27:25.746838+0000","last_fullsized":"2023-10-16T11:27:25.746838+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:44.126144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.2e","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.746056+0000","last_change":"2023-10-16T11:27:23.478674+0000","last_active":"2023-10-16T11:27:25.746056+0000","last_peered":"2023-10-16T11:27:25.746056+0000","last_clean":"2023-10-16T11:27:25.746056+0000","last_became_active":"2023-10-16T11:27:23.477899+0000","last_became_peered":"2023-10-16T11:27:23.477899+0000","last_unstale":"2023-10-16T11:27:25.746056+0000","last_undegraded":"2023-10-16T11:27:25.746056+0000","last_fullsized":"2023-10-16T11:27:25.746056+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:50.441137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.2d","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.444227+0000","last_change":"2023-10-16T11:27:23.473911+0000","last_active":"2023-10-16T11:27:24.444227+0000","last_peered":"2023-10-16T11:27:24.444227+0000","last_clean":"2023-10-16T11:27:24.444227+0000","last_became_active":"2023-10-16T11:27:23.473610+0000","last_became_peered":"2023-10-16T11:27:23.473610+0000","last_unstale":"2023-10-16T11:27:24.444227+0000","last_undegraded":"2023-10-16T11:27:24.444227+0000","last_fullsized":"2023-10-16T11:27:24.444227+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:43.093225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.2c","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.443015+0000","last_change":"2023-10-16T11:27:23.454345+0000","last_active":"2023-10-16T11:27:24.443015+0000","last_peered":"2023-10-16T11:27:24.443015+0000","last_clean":"2023-10-16T11:27:24.443015+0000","last_became_active":"2023-10-16T11:27:23.453919+0000","last_became_peered":"2023-10-16T11:27:23.453919+0000","last_unstale":"2023-10-16T11:27:24.443015+0000","last_undegraded":"2023-10-16T11:27:24.443015+0000","last_fullsized":"2023-10-16T11:27:24.443015+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:12:31.482145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.2b","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.532145+0000","last_change":"2023-10-16T11:27:23.469571+0000","last_active":"2023-10-16T11:27:25.532145+0000","last_peered":"2023-10-16T11:27:25.532145+0000","last_clean":"2023-10-16T11:27:25.532145+0000","last_became_active":"2023-10-16T11:27:23.469316+0000","last_became_peered":"2023-10-16T11:27:23.469316+0000","last_unstale":"2023-10-16T11:27:25.532145+0000","last_undegraded":"2023-10-16T11:27:25.532145+0000","last_fullsized":"2023-10-16T11:27:25.532145+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:04:13.703694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.2a","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.440293+0000","last_change":"2023-10-16T11:27:23.467378+0000","last_active":"2023-10-16T11:27:24.440293+0000","last_peered":"2023-10-16T11:27:24.440293+0000","last_clean":"2023-10-16T11:27:24.440293+0000","last_became_active":"2023-10-16T11:27:23.465993+0000","last_became_peered":"2023-10-16T11:27:23.465993+0000","last_unstale":"2023-10-16T11:27:24.440293+0000","last_undegraded":"2023-10-16T11:27:24.440293+0000","last_fullsized":"2023-10-16T11:27:24.440293+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:16.206103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.29","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.745327+0000","last_change":"2023-10-16T11:27:23.479466+0000","last_active":"2023-10-16T11:27:25.745327+0000","last_peered":"2023-10-16T11:27:25.745327+0000","last_clean":"2023-10-16T11:27:25.745327+0000","last_became_active":"2023-10-16T11:27:23.479159+0000","last_became_peered":"2023-10-16T11:27:23.479159+0000","last_unstale":"2023-10-16T11:27:25.745327+0000","last_undegraded":"2023-10-16T11:27:25.745327+0000","last_fullsized":"2023-10-16T11:27:25.745327+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:01.719295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.28","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.442432+0000","last_change":"2023-10-16T11:27:23.464430+0000","last_active":"2023-10-16T11:27:24.442432+0000","last_peered":"2023-10-16T11:27:24.442432+0000","last_clean":"2023-10-16T11:27:24.442432+0000","last_became_active":"2023-10-16T11:27:23.462771+0000","last_became_peered":"2023-10-16T11:27:23.462771+0000","last_unstale":"2023-10-16T11:27:24.442432+0000","last_undegraded":"2023-10-16T11:27:24.442432+0000","last_fullsized":"2023-10-16T11:27:24.442432+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:32:52.329776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.27","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.747180+0000","last_change":"2023-10-16T11:27:23.481050+0000","last_active":"2023-10-16T11:27:25.747180+0000","last_peered":"2023-10-16T11:27:25.747180+0000","last_clean":"2023-10-16T11:27:25.747180+0000","last_became_active":"2023-10-16T11:27:23.480560+0000","last_became_peered":"2023-10-16T11:27:23.480560+0000","last_unstale":"2023-10-16T11:27:25.747180+0000","last_undegraded":"2023-10-16T11:27:25.747180+0000","last_fullsized":"2023-10-16T11:27:25.747180+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:32:23.653130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.26","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.746332+0000","last_change":"2023-10-16T11:27:23.472694+0000","last_active":"2023-10-16T11:27:25.746332+0000","last_peered":"2023-10-16T11:27:25.746332+0000","last_clean":"2023-10-16T11:27:25.746332+0000","last_became_active":"2023-10-16T11:27:23.472023+0000","last_became_peered":"2023-10-16T11:27:23.472023+0000","last_unstale":"2023-10-16T11:27:25.746332+0000","last_undegraded":"2023-10-16T11:27:25.746332+0000","last_fullsized":"2023-10-16T11:27:25.746332+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:36:02.221752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.25","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.471970+0000","last_change":"2023-10-16T11:27:23.473453+0000","last_active":"2023-10-16T11:27:24.471970+0000","last_peered":"2023-10-16T11:27:24.471970+0000","last_clean":"2023-10-16T11:27:24.471970+0000","last_became_active":"2023-10-16T11:27:23.473188+0000","last_became_peered":"2023-10-16T11:27:23.473188+0000","last_unstale":"2023-10-16T11:27:24.471970+0000","last_undegraded":"2023-10-16T11:27:24.471970+0000","last_fullsized":"2023-10-16T11:27:24.471970+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:51:45.291096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.23","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.741943+0000","last_change":"2023-10-16T11:27:23.477433+0000","last_active":"2023-10-16T11:27:25.741943+0000","last_peered":"2023-10-16T11:27:25.741943+0000","last_clean":"2023-10-16T11:27:25.741943+0000","last_became_active":"2023-10-16T11:27:23.477143+0000","last_became_peered":"2023-10-16T11:27:23.477143+0000","last_unstale":"2023-10-16T11:27:25.741943+0000","last_undegraded":"2023-10-16T11:27:25.741943+0000","last_fullsized":"2023-10-16T11:27:25.741943+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:03:33.273585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.22","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.745446+0000","last_change":"2023-10-16T11:27:23.482959+0000","last_active":"2023-10-16T11:27:25.745446+0000","last_peered":"2023-10-16T11:27:25.745446+0000","last_clean":"2023-10-16T11:27:25.745446+0000","last_became_active":"2023-10-16T11:27:23.482693+0000","last_became_peered":"2023-10-16T11:27:23.482693+0000","last_unstale":"2023-10-16T11:27:25.745446+0000","last_undegraded":"2023-10-16T11:27:25.745446+0000","last_fullsized":"2023-10-16T11:27:25.745446+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:52.119044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.21","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.442386+0000","last_change":"2023-10-16T11:27:23.468429+0000","last_active":"2023-10-16T11:27:24.442386+0000","last_peered":"2023-10-16T11:27:24.442386+0000","last_clean":"2023-10-16T11:27:24.442386+0000","last_became_active":"2023-10-16T11:27:23.467356+0000","last_became_peered":"2023-10-16T11:27:23.467356+0000","last_unstale":"2023-10-16T11:27:24.442386+0000","last_undegraded":"2023-10-16T11:27:24.442386+0000","last_fullsized":"2023-10-16T11:27:24.442386+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:30.202850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.20","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.439794+0000","last_change":"2023-10-16T11:27:23.455961+0000","last_active":"2023-10-16T11:27:24.439794+0000","last_peered":"2023-10-16T11:27:24.439794+0000","last_clean":"2023-10-16T11:27:24.439794+0000","last_became_active":"2023-10-16T11:27:23.454819+0000","last_became_peered":"2023-10-16T11:27:23.454819+0000","last_unstale":"2023-10-16T11:27:24.439794+0000","last_undegraded":"2023-10-16T11:27:24.439794+0000","last_fullsized":"2023-10-16T11:27:24.439794+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:22:05.724817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.1e","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.471403+0000","last_change":"2023-10-16T11:27:23.474396+0000","last_active":"2023-10-16T11:27:24.471403+0000","last_peered":"2023-10-16T11:27:24.471403+0000","last_clean":"2023-10-16T11:27:24.471403+0000","last_became_active":"2023-10-16T11:27:23.473949+0000","last_became_peered":"2023-10-16T11:27:23.473949+0000","last_unstale":"2023-10-16T11:27:24.471403+0000","last_undegraded":"2023-10-16T11:27:24.471403+0000","last_fullsized":"2023-10-16T11:27:24.471403+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:25:43.689973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.1f","version":"269'4","reported_seq":16,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.488760+0000","last_change":"2023-10-16T11:27:22.434039+0000","last_active":"2023-10-16T11:27:23.488760+0000","last_peered":"2023-10-16T11:27:23.488760+0000","last_clean":"2023-10-16T11:27:23.488760+0000","last_became_active":"2023-10-16T11:27:22.433533+0000","last_became_peered":"2023-10-16T11:27:22.433533+0000","last_unstale":"2023-10-16T11:27:23.488760+0000","last_undegraded":"2023-10-16T11:27:23.488760+0000","last_fullsized":"2023-10-16T11:27:23.488760+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:23:50.394436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.1f","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.443404+0000","last_change":"2023-10-16T11:27:23.486773+0000","last_active":"2023-10-16T11:27:24.443404+0000","last_peered":"2023-10-16T11:27:24.443404+0000","last_clean":"2023-10-16T11:27:24.443404+0000","last_became_active":"2023-10-16T11:27:23.485146+0000","last_became_peered":"2023-10-16T11:27:23.485146+0000","last_unstale":"2023-10-16T11:27:24.443404+0000","last_undegraded":"2023-10-16T11:27:24.443404+0000","last_fullsized":"2023-10-16T11:27:24.443404+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:56:18.831831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.1e","version":"269'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.487395+0000","last_change":"2023-10-16T11:27:22.430210+0000","last_active":"2023-10-16T11:27:23.487395+0000","last_peered":"2023-10-16T11:27:23.487395+0000","last_clean":"2023-10-16T11:27:23.487395+0000","last_became_active":"2023-10-16T11:27:22.429926+0000","last_became_peered":"2023-10-16T11:27:22.429926+0000","last_unstale":"2023-10-16T11:27:23.487395+0000","last_undegraded":"2023-10-16T11:27:23.487395+0000","last_fullsized":"2023-10-16T11:27:23.487395+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:08.846109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.1c","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.742121+0000","last_change":"2023-10-16T11:27:23.479224+0000","last_active":"2023-10-16T11:27:25.742121+0000","last_peered":"2023-10-16T11:27:25.742121+0000","last_clean":"2023-10-16T11:27:25.742121+0000","last_became_active":"2023-10-16T11:27:23.478884+0000","last_became_peered":"2023-10-16T11:27:23.478884+0000","last_unstale":"2023-10-16T11:27:25.742121+0000","last_undegraded":"2023-10-16T11:27:25.742121+0000","last_fullsized":"2023-10-16T11:27:25.742121+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:22:05.724817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.1d","version":"269'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.488222+0000","last_change":"2023-10-16T11:27:22.439649+0000","last_active":"2023-10-16T11:27:23.488222+0000","last_peered":"2023-10-16T11:27:23.488222+0000","last_clean":"2023-10-16T11:27:23.488222+0000","last_became_active":"2023-10-16T11:27:22.439385+0000","last_became_peered":"2023-10-16T11:27:22.439385+0000","last_unstale":"2023-10-16T11:27:23.488222+0000","last_undegraded":"2023-10-16T11:27:23.488222+0000","last_fullsized":"2023-10-16T11:27:23.488222+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:56:17.814662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.1d","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.532606+0000","last_change":"2023-10-16T11:27:23.474658+0000","last_active":"2023-10-16T11:27:25.532606+0000","last_peered":"2023-10-16T11:27:25.532606+0000","last_clean":"2023-10-16T11:27:25.532606+0000","last_became_active":"2023-10-16T11:27:23.469718+0000","last_became_peered":"2023-10-16T11:27:23.469718+0000","last_unstale":"2023-10-16T11:27:25.532606+0000","last_undegraded":"2023-10-16T11:27:25.532606+0000","last_fullsized":"2023-10-16T11:27:25.532606+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:00:07.418959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1c","version":"269'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.486877+0000","last_change":"2023-10-16T11:27:22.427045+0000","last_active":"2023-10-16T11:27:23.486877+0000","last_peered":"2023-10-16T11:27:23.486877+0000","last_clean":"2023-10-16T11:27:23.486877+0000","last_became_active":"2023-10-16T11:27:22.426728+0000","last_became_peered":"2023-10-16T11:27:22.426728+0000","last_unstale":"2023-10-16T11:27:23.486877+0000","last_undegraded":"2023-10-16T11:27:23.486877+0000","last_fullsized":"2023-10-16T11:27:23.486877+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:39.244304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.1a","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.532299+0000","last_change":"2023-10-16T11:27:23.457525+0000","last_active":"2023-10-16T11:27:25.532299+0000","last_peered":"2023-10-16T11:27:25.532299+0000","last_clean":"2023-10-16T11:27:25.532299+0000","last_became_active":"2023-10-16T11:27:23.456872+0000","last_became_peered":"2023-10-16T11:27:23.456872+0000","last_unstale":"2023-10-16T11:27:25.532299+0000","last_undegraded":"2023-10-16T11:27:25.532299+0000","last_fullsized":"2023-10-16T11:27:25.532299+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:00.798477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1b","version":"269'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.486399+0000","last_change":"2023-10-16T11:27:22.440153+0000","last_active":"2023-10-16T11:27:23.486399+0000","last_peered":"2023-10-16T11:27:23.486399+0000","last_clean":"2023-10-16T11:27:23.486399+0000","last_became_active":"2023-10-16T11:27:22.439862+0000","last_became_peered":"2023-10-16T11:27:22.439862+0000","last_unstale":"2023-10-16T11:27:23.486399+0000","last_undegraded":"2023-10-16T11:27:23.486399+0000","last_fullsized":"2023-10-16T11:27:23.486399+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:26:25.704295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.1b","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.442971+0000","last_change":"2023-10-16T11:27:23.469778+0000","last_active":"2023-10-16T11:27:24.442971+0000","last_peered":"2023-10-16T11:27:24.442971+0000","last_clean":"2023-10-16T11:27:24.442971+0000","last_became_active":"2023-10-16T11:27:23.469529+0000","last_became_peered":"2023-10-16T11:27:23.469529+0000","last_unstale":"2023-10-16T11:27:24.442971+0000","last_undegraded":"2023-10-16T11:27:24.442971+0000","last_fullsized":"2023-10-16T11:27:24.442971+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:06:02.521146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1a","version":"269'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.484787+0000","last_change":"2023-10-16T11:27:22.435228+0000","last_active":"2023-10-16T11:27:23.484787+0000","last_peered":"2023-10-16T11:27:23.484787+0000","last_clean":"2023-10-16T11:27:23.484787+0000","last_became_active":"2023-10-16T11:27:22.434922+0000","last_became_peered":"2023-10-16T11:27:22.434922+0000","last_unstale":"2023-10-16T11:27:23.484787+0000","last_undegraded":"2023-10-16T11:27:23.484787+0000","last_fullsized":"2023-10-16T11:27:23.484787+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:15.188934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.18","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.744508+0000","last_change":"2023-10-16T11:27:23.485335+0000","last_active":"2023-10-16T11:27:25.744508+0000","last_peered":"2023-10-16T11:27:25.744508+0000","last_clean":"2023-10-16T11:27:25.744508+0000","last_became_active":"2023-10-16T11:27:23.483862+0000","last_became_peered":"2023-10-16T11:27:23.483862+0000","last_unstale":"2023-10-16T11:27:25.744508+0000","last_undegraded":"2023-10-16T11:27:25.744508+0000","last_fullsized":"2023-10-16T11:27:25.744508+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:23:16.110019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.19","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.450091+0000","last_change":"2023-10-16T11:27:22.442373+0000","last_active":"2023-10-16T11:27:23.450091+0000","last_peered":"2023-10-16T11:27:23.450091+0000","last_clean":"2023-10-16T11:27:23.450091+0000","last_became_active":"2023-10-16T11:27:22.441763+0000","last_became_peered":"2023-10-16T11:27:22.441763+0000","last_unstale":"2023-10-16T11:27:23.450091+0000","last_undegraded":"2023-10-16T11:27:23.450091+0000","last_fullsized":"2023-10-16T11:27:23.450091+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:37:05.780888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.19","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.741955+0000","last_change":"2023-10-16T11:27:23.474956+0000","last_active":"2023-10-16T11:27:25.741955+0000","last_peered":"2023-10-16T11:27:25.741955+0000","last_clean":"2023-10-16T11:27:25.741955+0000","last_became_active":"2023-10-16T11:27:23.474685+0000","last_became_peered":"2023-10-16T11:27:23.474685+0000","last_unstale":"2023-10-16T11:27:25.741955+0000","last_undegraded":"2023-10-16T11:27:25.741955+0000","last_fullsized":"2023-10-16T11:27:25.741955+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:43.093225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.18","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.462560+0000","last_change":"2023-10-16T11:27:22.432937+0000","last_active":"2023-10-16T11:27:23.462560+0000","last_peered":"2023-10-16T11:27:23.462560+0000","last_clean":"2023-10-16T11:27:23.462560+0000","last_became_active":"2023-10-16T11:27:22.432719+0000","last_became_peered":"2023-10-16T11:27:22.432719+0000","last_unstale":"2023-10-16T11:27:23.462560+0000","last_undegraded":"2023-10-16T11:27:23.462560+0000","last_fullsized":"2023-10-16T11:27:23.462560+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:02.427534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.16","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.742245+0000","last_change":"2023-10-16T11:27:23.473098+0000","last_active":"2023-10-16T11:27:25.742245+0000","last_peered":"2023-10-16T11:27:25.742245+0000","last_clean":"2023-10-16T11:27:25.742245+0000","last_became_active":"2023-10-16T11:27:23.472855+0000","last_became_peered":"2023-10-16T11:27:23.472855+0000","last_unstale":"2023-10-16T11:27:25.742245+0000","last_undegraded":"2023-10-16T11:27:25.742245+0000","last_fullsized":"2023-10-16T11:27:25.742245+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:09.556639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.17","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.466216+0000","last_change":"2023-10-16T11:27:22.434526+0000","last_active":"2023-10-16T11:27:23.466216+0000","last_peered":"2023-10-16T11:27:23.466216+0000","last_clean":"2023-10-16T11:27:23.466216+0000","last_became_active":"2023-10-16T11:27:22.433489+0000","last_became_peered":"2023-10-16T11:27:22.433489+0000","last_unstale":"2023-10-16T11:27:23.466216+0000","last_undegraded":"2023-10-16T11:27:23.466216+0000","last_fullsized":"2023-10-16T11:27:23.466216+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:13.916657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.17","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.443755+0000","last_change":"2023-10-16T11:27:23.474492+0000","last_active":"2023-10-16T11:27:24.443755+0000","last_peered":"2023-10-16T11:27:24.443755+0000","last_clean":"2023-10-16T11:27:24.443755+0000","last_became_active":"2023-10-16T11:27:23.474228+0000","last_became_peered":"2023-10-16T11:27:23.474228+0000","last_unstale":"2023-10-16T11:27:24.443755+0000","last_undegraded":"2023-10-16T11:27:24.443755+0000","last_fullsized":"2023-10-16T11:27:24.443755+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:34.898375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.16","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.465874+0000","last_change":"2023-10-16T11:27:22.435414+0000","last_active":"2023-10-16T11:27:23.465874+0000","last_peered":"2023-10-16T11:27:23.465874+0000","last_clean":"2023-10-16T11:27:23.465874+0000","last_became_active":"2023-10-16T11:27:22.434556+0000","last_became_peered":"2023-10-16T11:27:22.434556+0000","last_unstale":"2023-10-16T11:27:23.465874+0000","last_undegraded":"2023-10-16T11:27:23.465874+0000","last_fullsized":"2023-10-16T11:27:23.465874+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:02.427534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.14","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.440813+0000","last_change":"2023-10-16T11:27:23.466931+0000","last_active":"2023-10-16T11:27:24.440813+0000","last_peered":"2023-10-16T11:27:24.440813+0000","last_clean":"2023-10-16T11:27:24.440813+0000","last_became_active":"2023-10-16T11:27:23.466635+0000","last_became_peered":"2023-10-16T11:27:23.466635+0000","last_unstale":"2023-10-16T11:27:24.440813+0000","last_undegraded":"2023-10-16T11:27:24.440813+0000","last_fullsized":"2023-10-16T11:27:24.440813+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:57:41.546314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.15","version":"269'3","reported_seq":15,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.489036+0000","last_change":"2023-10-16T11:27:22.428021+0000","last_active":"2023-10-16T11:27:23.489036+0000","last_peered":"2023-10-16T11:27:23.489036+0000","last_clean":"2023-10-16T11:27:23.489036+0000","last_became_active":"2023-10-16T11:27:22.427744+0000","last_became_peered":"2023-10-16T11:27:22.427744+0000","last_unstale":"2023-10-16T11:27:23.489036+0000","last_undegraded":"2023-10-16T11:27:23.489036+0000","last_fullsized":"2023-10-16T11:27:23.489036+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:29.185681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.15","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.742198+0000","last_change":"2023-10-16T11:27:23.472980+0000","last_active":"2023-10-16T11:27:25.742198+0000","last_peered":"2023-10-16T11:27:25.742198+0000","last_clean":"2023-10-16T11:27:25.742198+0000","last_became_active":"2023-10-16T11:27:23.471883+0000","last_became_peered":"2023-10-16T11:27:23.471883+0000","last_unstale":"2023-10-16T11:27:25.742198+0000","last_undegraded":"2023-10-16T11:27:25.742198+0000","last_fullsized":"2023-10-16T11:27:25.742198+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:16.206103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.14","version":"269'6","reported_seq":18,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.491027+0000","last_change":"2023-10-16T11:27:22.440362+0000","last_active":"2023-10-16T11:27:23.491027+0000","last_peered":"2023-10-16T11:27:23.491027+0000","last_clean":"2023-10-16T11:27:23.491027+0000","last_became_active":"2023-10-16T11:27:22.439632+0000","last_became_peered":"2023-10-16T11:27:22.439632+0000","last_unstale":"2023-10-16T11:27:23.491027+0000","last_undegraded":"2023-10-16T11:27:23.491027+0000","last_fullsized":"2023-10-16T11:27:23.491027+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:39.244304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.12","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.443685+0000","last_change":"2023-10-16T11:27:23.470789+0000","last_active":"2023-10-16T11:27:24.443685+0000","last_peered":"2023-10-16T11:27:24.443685+0000","last_clean":"2023-10-16T11:27:24.443685+0000","last_became_active":"2023-10-16T11:27:23.470511+0000","last_became_peered":"2023-10-16T11:27:23.470511+0000","last_unstale":"2023-10-16T11:27:24.443685+0000","last_undegraded":"2023-10-16T11:27:24.443685+0000","last_fullsized":"2023-10-16T11:27:24.443685+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:40.261473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.13","version":"269'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.488139+0000","last_change":"2023-10-16T11:27:22.420724+0000","last_active":"2023-10-16T11:27:23.488139+0000","last_peered":"2023-10-16T11:27:23.488139+0000","last_clean":"2023-10-16T11:27:23.488139+0000","last_became_active":"2023-10-16T11:27:22.416656+0000","last_became_peered":"2023-10-16T11:27:22.416656+0000","last_unstale":"2023-10-16T11:27:23.488139+0000","last_undegraded":"2023-10-16T11:27:23.488139+0000","last_fullsized":"2023-10-16T11:27:23.488139+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:23.396162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.13","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.745838+0000","last_change":"2023-10-16T11:27:23.468059+0000","last_active":"2023-10-16T11:27:25.745838+0000","last_peered":"2023-10-16T11:27:25.745838+0000","last_clean":"2023-10-16T11:27:25.745838+0000","last_became_active":"2023-10-16T11:27:23.467796+0000","last_became_peered":"2023-10-16T11:27:23.467796+0000","last_unstale":"2023-10-16T11:27:25.745838+0000","last_undegraded":"2023-10-16T11:27:25.745838+0000","last_fullsized":"2023-10-16T11:27:25.745838+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:24:04.957139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.12","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.444463+0000","last_change":"2023-10-16T11:27:22.418954+0000","last_active":"2023-10-16T11:27:23.444463+0000","last_peered":"2023-10-16T11:27:23.444463+0000","last_clean":"2023-10-16T11:27:23.444463+0000","last_became_active":"2023-10-16T11:27:22.418291+0000","last_became_peered":"2023-10-16T11:27:22.418291+0000","last_unstale":"2023-10-16T11:27:23.444463+0000","last_undegraded":"2023-10-16T11:27:23.444463+0000","last_fullsized":"2023-10-16T11:27:23.444463+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:05:07.678705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.10","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.470564+0000","last_change":"2023-10-16T11:27:23.467760+0000","last_active":"2023-10-16T11:27:24.470564+0000","last_peered":"2023-10-16T11:27:24.470564+0000","last_clean":"2023-10-16T11:27:24.470564+0000","last_became_active":"2023-10-16T11:27:23.466581+0000","last_became_peered":"2023-10-16T11:27:23.466581+0000","last_unstale":"2023-10-16T11:27:24.470564+0000","last_undegraded":"2023-10-16T11:27:24.470564+0000","last_fullsized":"2023-10-16T11:27:24.470564+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:24.699653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.11","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.467774+0000","last_change":"2023-10-16T11:27:22.432941+0000","last_active":"2023-10-16T11:27:23.467774+0000","last_peered":"2023-10-16T11:27:23.467774+0000","last_clean":"2023-10-16T11:27:23.467774+0000","last_became_active":"2023-10-16T11:27:22.432633+0000","last_became_peered":"2023-10-16T11:27:22.432633+0000","last_unstale":"2023-10-16T11:27:23.467774+0000","last_undegraded":"2023-10-16T11:27:23.467774+0000","last_fullsized":"2023-10-16T11:27:23.467774+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:33.881206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.11","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.744898+0000","last_change":"2023-10-16T11:27:23.470371+0000","last_active":"2023-10-16T11:27:25.744898+0000","last_peered":"2023-10-16T11:27:25.744898+0000","last_clean":"2023-10-16T11:27:25.744898+0000","last_became_active":"2023-10-16T11:27:23.469794+0000","last_became_peered":"2023-10-16T11:27:23.469794+0000","last_unstale":"2023-10-16T11:27:25.744898+0000","last_undegraded":"2023-10-16T11:27:25.744898+0000","last_fullsized":"2023-10-16T11:27:25.744898+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:54:26.226556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.10","version":"269'8","reported_seq":20,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.491124+0000","last_change":"2023-10-16T11:27:22.435285+0000","last_active":"2023-10-16T11:27:23.491124+0000","last_peered":"2023-10-16T11:27:23.491124+0000","last_clean":"2023-10-16T11:27:23.491124+0000","last_became_active":"2023-10-16T11:27:22.434837+0000","last_became_peered":"2023-10-16T11:27:22.434837+0000","last_unstale":"2023-10-16T11:27:23.491124+0000","last_undegraded":"2023-10-16T11:27:23.491124+0000","last_fullsized":"2023-10-16T11:27:23.491124+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:57:54.775978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.e","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.441263+0000","last_change":"2023-10-16T11:27:23.478549+0000","last_active":"2023-10-16T11:27:24.441263+0000","last_peered":"2023-10-16T11:27:24.441263+0000","last_clean":"2023-10-16T11:27:24.441263+0000","last_became_active":"2023-10-16T11:27:23.478282+0000","last_became_peered":"2023-10-16T11:27:23.478282+0000","last_unstale":"2023-10-16T11:27:24.441263+0000","last_undegraded":"2023-10-16T11:27:24.441263+0000","last_fullsized":"2023-10-16T11:27:24.441263+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:44.126144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.f","version":"269'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.487257+0000","last_change":"2023-10-16T11:27:22.426688+0000","last_active":"2023-10-16T11:27:23.487257+0000","last_peered":"2023-10-16T11:27:23.487257+0000","last_clean":"2023-10-16T11:27:23.487257+0000","last_became_active":"2023-10-16T11:27:22.426400+0000","last_became_peered":"2023-10-16T11:27:22.426400+0000","last_unstale":"2023-10-16T11:27:23.487257+0000","last_undegraded":"2023-10-16T11:27:23.487257+0000","last_fullsized":"2023-10-16T11:27:23.487257+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:08:03.403778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.f","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.472365+0000","last_change":"2023-10-16T11:27:23.481822+0000","last_active":"2023-10-16T11:27:24.472365+0000","last_peered":"2023-10-16T11:27:24.472365+0000","last_clean":"2023-10-16T11:27:24.472365+0000","last_became_active":"2023-10-16T11:27:23.481451+0000","last_became_peered":"2023-10-16T11:27:23.481451+0000","last_unstale":"2023-10-16T11:27:24.472365+0000","last_undegraded":"2023-10-16T11:27:24.472365+0000","last_fullsized":"2023-10-16T11:27:24.472365+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:02:34.998913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.e","version":"269'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.487751+0000","last_change":"2023-10-16T11:27:22.435582+0000","last_active":"2023-10-16T11:27:23.487751+0000","last_peered":"2023-10-16T11:27:23.487751+0000","last_clean":"2023-10-16T11:27:23.487751+0000","last_became_active":"2023-10-16T11:27:22.435098+0000","last_became_peered":"2023-10-16T11:27:22.435098+0000","last_unstale":"2023-10-16T11:27:23.487751+0000","last_undegraded":"2023-10-16T11:27:23.487751+0000","last_fullsized":"2023-10-16T11:27:23.487751+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:05:07.678705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.c","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.742166+0000","last_change":"2023-10-16T11:27:23.472000+0000","last_active":"2023-10-16T11:27:25.742166+0000","last_peered":"2023-10-16T11:27:25.742166+0000","last_clean":"2023-10-16T11:27:25.742166+0000","last_became_active":"2023-10-16T11:27:23.469423+0000","last_became_peered":"2023-10-16T11:27:23.469423+0000","last_unstale":"2023-10-16T11:27:25.742166+0000","last_undegraded":"2023-10-16T11:27:25.742166+0000","last_fullsized":"2023-10-16T11:27:25.742166+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:30:41.982564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.d","version":"269'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.487807+0000","last_change":"2023-10-16T11:27:22.434669+0000","last_active":"2023-10-16T11:27:23.487807+0000","last_peered":"2023-10-16T11:27:23.487807+0000","last_clean":"2023-10-16T11:27:23.487807+0000","last_became_active":"2023-10-16T11:27:22.433720+0000","last_became_peered":"2023-10-16T11:27:22.433720+0000","last_unstale":"2023-10-16T11:27:23.487807+0000","last_undegraded":"2023-10-16T11:27:23.487807+0000","last_fullsized":"2023-10-16T11:27:23.487807+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:57.154827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.d","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.444077+0000","last_change":"2023-10-16T11:27:23.469154+0000","last_active":"2023-10-16T11:27:24.444077+0000","last_peered":"2023-10-16T11:27:24.444077+0000","last_clean":"2023-10-16T11:27:24.444077+0000","last_became_active":"2023-10-16T11:27:23.468885+0000","last_became_peered":"2023-10-16T11:27:23.468885+0000","last_unstale":"2023-10-16T11:27:24.444077+0000","last_undegraded":"2023-10-16T11:27:24.444077+0000","last_fullsized":"2023-10-16T11:27:24.444077+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:03:33.273585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.c","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.465428+0000","last_change":"2023-10-16T11:27:22.438800+0000","last_active":"2023-10-16T11:27:23.465428+0000","last_peered":"2023-10-16T11:27:23.465428+0000","last_clean":"2023-10-16T11:27:23.465428+0000","last_became_active":"2023-10-16T11:27:22.438513+0000","last_became_peered":"2023-10-16T11:27:22.438513+0000","last_unstale":"2023-10-16T11:27:23.465428+0000","last_undegraded":"2023-10-16T11:27:23.465428+0000","last_fullsized":"2023-10-16T11:27:23.465428+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:32:51.312608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.a","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.471207+0000","last_change":"2023-10-16T11:27:23.486689+0000","last_active":"2023-10-16T11:27:24.471207+0000","last_peered":"2023-10-16T11:27:24.471207+0000","last_clean":"2023-10-16T11:27:24.471207+0000","last_became_active":"2023-10-16T11:27:23.484418+0000","last_became_peered":"2023-10-16T11:27:23.484418+0000","last_unstale":"2023-10-16T11:27:24.471207+0000","last_undegraded":"2023-10-16T11:27:24.471207+0000","last_fullsized":"2023-10-16T11:27:24.471207+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:33:06.072611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.b","version":"269'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.489077+0000","last_change":"2023-10-16T11:27:22.427350+0000","last_active":"2023-10-16T11:27:23.489077+0000","last_peered":"2023-10-16T11:27:23.489077+0000","last_clean":"2023-10-16T11:27:23.489077+0000","last_became_active":"2023-10-16T11:27:22.427089+0000","last_became_peered":"2023-10-16T11:27:22.427089+0000","last_unstale":"2023-10-16T11:27:23.489077+0000","last_undegraded":"2023-10-16T11:27:23.489077+0000","last_fullsized":"2023-10-16T11:27:23.489077+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:33.881206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.b","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.443349+0000","last_change":"2023-10-16T11:27:23.467686+0000","last_active":"2023-10-16T11:27:24.443349+0000","last_peered":"2023-10-16T11:27:24.443349+0000","last_clean":"2023-10-16T11:27:24.443349+0000","last_became_active":"2023-10-16T11:27:23.467408+0000","last_became_peered":"2023-10-16T11:27:23.467408+0000","last_unstale":"2023-10-16T11:27:24.443349+0000","last_undegraded":"2023-10-16T11:27:24.443349+0000","last_fullsized":"2023-10-16T11:27:24.443349+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:00.149551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.a","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.450086+0000","last_change":"2023-10-16T11:27:22.427630+0000","last_active":"2023-10-16T11:27:23.450086+0000","last_peered":"2023-10-16T11:27:23.450086+0000","last_clean":"2023-10-16T11:27:23.450086+0000","last_became_active":"2023-10-16T11:27:22.426731+0000","last_became_peered":"2023-10-16T11:27:22.426731+0000","last_unstale":"2023-10-16T11:27:23.450086+0000","last_undegraded":"2023-10-16T11:27:23.450086+0000","last_fullsized":"2023-10-16T11:27:23.450086+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:16:33.200320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.8","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.531722+0000","last_change":"2023-10-16T11:27:23.460647+0000","last_active":"2023-10-16T11:27:25.531722+0000","last_peered":"2023-10-16T11:27:25.531722+0000","last_clean":"2023-10-16T11:27:25.531722+0000","last_became_active":"2023-10-16T11:27:23.460185+0000","last_became_peered":"2023-10-16T11:27:23.460185+0000","last_unstale":"2023-10-16T11:27:25.531722+0000","last_undegraded":"2023-10-16T11:27:25.531722+0000","last_fullsized":"2023-10-16T11:27:25.531722+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:30.306508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.9","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.462457+0000","last_change":"2023-10-16T11:27:22.439130+0000","last_active":"2023-10-16T11:27:23.462457+0000","last_peered":"2023-10-16T11:27:23.462457+0000","last_clean":"2023-10-16T11:27:23.462457+0000","last_became_active":"2023-10-16T11:27:22.438098+0000","last_became_peered":"2023-10-16T11:27:22.438098+0000","last_unstale":"2023-10-16T11:27:23.462457+0000","last_undegraded":"2023-10-16T11:27:23.462457+0000","last_fullsized":"2023-10-16T11:27:23.462457+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:36:07.856477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.9","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.531769+0000","last_change":"2023-10-16T11:27:23.480112+0000","last_active":"2023-10-16T11:27:25.531769+0000","last_peered":"2023-10-16T11:27:25.531769+0000","last_clean":"2023-10-16T11:27:25.531769+0000","last_became_active":"2023-10-16T11:27:23.479844+0000","last_became_peered":"2023-10-16T11:27:23.479844+0000","last_unstale":"2023-10-16T11:27:25.531769+0000","last_undegraded":"2023-10-16T11:27:25.531769+0000","last_fullsized":"2023-10-16T11:27:25.531769+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:00:54.115844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.8","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.451508+0000","last_change":"2023-10-16T11:27:22.424674+0000","last_active":"2023-10-16T11:27:23.451508+0000","last_peered":"2023-10-16T11:27:23.451508+0000","last_clean":"2023-10-16T11:27:23.451508+0000","last_became_active":"2023-10-16T11:27:22.424065+0000","last_became_peered":"2023-10-16T11:27:22.424065+0000","last_unstale":"2023-10-16T11:27:23.451508+0000","last_undegraded":"2023-10-16T11:27:23.451508+0000","last_fullsized":"2023-10-16T11:27:23.451508+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:45.782726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.6","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.443606+0000","last_change":"2023-10-16T11:27:23.455323+0000","last_active":"2023-10-16T11:27:24.443606+0000","last_peered":"2023-10-16T11:27:24.443606+0000","last_clean":"2023-10-16T11:27:24.443606+0000","last_became_active":"2023-10-16T11:27:23.454550+0000","last_became_peered":"2023-10-16T11:27:23.454550+0000","last_unstale":"2023-10-16T11:27:24.443606+0000","last_undegraded":"2023-10-16T11:27:24.443606+0000","last_fullsized":"2023-10-16T11:27:24.443606+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:09.933592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.7","version":"269'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.488039+0000","last_change":"2023-10-16T11:27:22.427506+0000","last_active":"2023-10-16T11:27:23.488039+0000","last_peered":"2023-10-16T11:27:23.488039+0000","last_clean":"2023-10-16T11:27:23.488039+0000","last_became_active":"2023-10-16T11:27:22.426863+0000","last_became_peered":"2023-10-16T11:27:22.426863+0000","last_unstale":"2023-10-16T11:27:23.488039+0000","last_undegraded":"2023-10-16T11:27:23.488039+0000","last_fullsized":"2023-10-16T11:27:23.488039+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:57.154827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.7","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.747515+0000","last_change":"2023-10-16T11:27:23.480903+0000","last_active":"2023-10-16T11:27:25.747515+0000","last_peered":"2023-10-16T11:27:25.747515+0000","last_clean":"2023-10-16T11:27:25.747515+0000","last_became_active":"2023-10-16T11:27:23.480347+0000","last_became_peered":"2023-10-16T11:27:23.480347+0000","last_unstale":"2023-10-16T11:27:25.747515+0000","last_undegraded":"2023-10-16T11:27:25.747515+0000","last_fullsized":"2023-10-16T11:27:25.747515+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:28:14.944747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.6","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.450228+0000","last_change":"2023-10-16T11:27:22.432595+0000","last_active":"2023-10-16T11:27:23.450228+0000","last_peered":"2023-10-16T11:27:23.450228+0000","last_clean":"2023-10-16T11:27:23.450228+0000","last_became_active":"2023-10-16T11:27:22.432255+0000","last_became_peered":"2023-10-16T11:27:22.432255+0000","last_unstale":"2023-10-16T11:27:23.450228+0000","last_undegraded":"2023-10-16T11:27:23.450228+0000","last_fullsized":"2023-10-16T11:27:23.450228+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:15.847704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.4","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.443294+0000","last_change":"2023-10-16T11:27:23.476219+0000","last_active":"2023-10-16T11:27:24.443294+0000","last_peered":"2023-10-16T11:27:24.443294+0000","last_clean":"2023-10-16T11:27:24.443294+0000","last_became_active":"2023-10-16T11:27:23.475943+0000","last_became_peered":"2023-10-16T11:27:23.475943+0000","last_unstale":"2023-10-16T11:27:24.443294+0000","last_undegraded":"2023-10-16T11:27:24.443294+0000","last_fullsized":"2023-10-16T11:27:24.443294+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:58.171995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.5","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.445325+0000","last_change":"2023-10-16T11:27:22.434467+0000","last_active":"2023-10-16T11:27:23.445325+0000","last_peered":"2023-10-16T11:27:23.445325+0000","last_clean":"2023-10-16T11:27:23.445325+0000","last_became_active":"2023-10-16T11:27:22.433992+0000","last_became_peered":"2023-10-16T11:27:22.433992+0000","last_unstale":"2023-10-16T11:27:23.445325+0000","last_undegraded":"2023-10-16T11:27:23.445325+0000","last_fullsized":"2023-10-16T11:27:23.445325+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:32:51.312608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.5","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.745544+0000","last_change":"2023-10-16T11:27:23.470590+0000","last_active":"2023-10-16T11:27:25.745544+0000","last_peered":"2023-10-16T11:27:25.745544+0000","last_clean":"2023-10-16T11:27:25.745544+0000","last_became_active":"2023-10-16T11:27:23.466649+0000","last_became_peered":"2023-10-16T11:27:23.466649+0000","last_unstale":"2023-10-16T11:27:25.745544+0000","last_undegraded":"2023-10-16T11:27:25.745544+0000","last_fullsized":"2023-10-16T11:27:25.745544+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:23:07.907946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.4","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.456950+0000","last_change":"2023-10-16T11:27:22.439371+0000","last_active":"2023-10-16T11:27:23.456950+0000","last_peered":"2023-10-16T11:27:23.456950+0000","last_clean":"2023-10-16T11:27:23.456950+0000","last_became_active":"2023-10-16T11:27:22.439092+0000","last_became_peered":"2023-10-16T11:27:22.439092+0000","last_unstale":"2023-10-16T11:27:23.456950+0000","last_undegraded":"2023-10-16T11:27:23.456950+0000","last_fullsized":"2023-10-16T11:27:23.456950+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:54:50.109436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.2","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.741901+0000","last_change":"2023-10-16T11:27:23.473928+0000","last_active":"2023-10-16T11:27:25.741901+0000","last_peered":"2023-10-16T11:27:25.741901+0000","last_clean":"2023-10-16T11:27:25.741901+0000","last_became_active":"2023-10-16T11:27:23.473681+0000","last_became_peered":"2023-10-16T11:27:23.473681+0000","last_unstale":"2023-10-16T11:27:25.741901+0000","last_undegraded":"2023-10-16T11:27:25.741901+0000","last_fullsized":"2023-10-16T11:27:25.741901+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:34.898375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.3","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.454385+0000","last_change":"2023-10-16T11:27:22.441927+0000","last_active":"2023-10-16T11:27:23.454385+0000","last_peered":"2023-10-16T11:27:23.454385+0000","last_clean":"2023-10-16T11:27:23.454385+0000","last_became_active":"2023-10-16T11:27:22.441607+0000","last_became_peered":"2023-10-16T11:27:22.441607+0000","last_unstale":"2023-10-16T11:27:23.454385+0000","last_undegraded":"2023-10-16T11:27:23.454385+0000","last_fullsized":"2023-10-16T11:27:23.454385+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:41.763300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.3","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.440771+0000","last_change":"2023-10-16T11:27:23.471757+0000","last_active":"2023-10-16T11:27:24.440771+0000","last_peered":"2023-10-16T11:27:24.440771+0000","last_clean":"2023-10-16T11:27:24.440771+0000","last_became_active":"2023-10-16T11:27:23.469662+0000","last_became_peered":"2023-10-16T11:27:23.469662+0000","last_unstale":"2023-10-16T11:27:24.440771+0000","last_undegraded":"2023-10-16T11:27:24.440771+0000","last_fullsized":"2023-10-16T11:27:24.440771+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:50.441137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.2","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.444768+0000","last_change":"2023-10-16T11:27:22.434794+0000","last_active":"2023-10-16T11:27:23.444768+0000","last_peered":"2023-10-16T11:27:23.444768+0000","last_clean":"2023-10-16T11:27:23.444768+0000","last_became_active":"2023-10-16T11:27:22.434535+0000","last_became_peered":"2023-10-16T11:27:22.434535+0000","last_unstale":"2023-10-16T11:27:23.444768+0000","last_undegraded":"2023-10-16T11:27:23.444768+0000","last_fullsized":"2023-10-16T11:27:23.444768+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:13.916657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.0","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.531935+0000","last_change":"2023-10-16T11:27:23.470420+0000","last_active":"2023-10-16T11:27:25.531935+0000","last_peered":"2023-10-16T11:27:25.531935+0000","last_clean":"2023-10-16T11:27:25.531935+0000","last_became_active":"2023-10-16T11:27:23.470185+0000","last_became_peered":"2023-10-16T11:27:23.470185+0000","last_unstale":"2023-10-16T11:27:25.531935+0000","last_undegraded":"2023-10-16T11:27:25.531935+0000","last_fullsized":"2023-10-16T11:27:25.531935+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:03:13.636592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.464850+0000","last_change":"2023-10-16T11:27:22.436919+0000","last_active":"2023-10-16T11:27:23.464850+0000","last_peered":"2023-10-16T11:27:23.464850+0000","last_clean":"2023-10-16T11:27:23.464850+0000","last_became_active":"2023-10-16T11:27:22.434273+0000","last_became_peered":"2023-10-16T11:27:22.434273+0000","last_unstale":"2023-10-16T11:27:23.464850+0000","last_undegraded":"2023-10-16T11:27:23.464850+0000","last_fullsized":"2023-10-16T11:27:23.464850+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:36:07.856477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.1","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.745268+0000","last_change":"2023-10-16T11:27:23.470169+0000","last_active":"2023-10-16T11:27:25.745268+0000","last_peered":"2023-10-16T11:27:25.745268+0000","last_clean":"2023-10-16T11:27:25.745268+0000","last_became_active":"2023-10-16T11:27:23.469310+0000","last_became_peered":"2023-10-16T11:27:23.469310+0000","last_unstale":"2023-10-16T11:27:25.745268+0000","last_undegraded":"2023-10-16T11:27:25.745268+0000","last_fullsized":"2023-10-16T11:27:25.745268+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:19:56.522209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.0","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.442094+0000","last_change":"2023-10-16T11:27:22.434884+0000","last_active":"2023-10-16T11:27:23.442094+0000","last_peered":"2023-10-16T11:27:23.442094+0000","last_clean":"2023-10-16T11:27:23.442094+0000","last_became_active":"2023-10-16T11:27:22.434345+0000","last_became_peered":"2023-10-16T11:27:22.434345+0000","last_unstale":"2023-10-16T11:27:23.442094+0000","last_undegraded":"2023-10-16T11:27:23.442094+0000","last_fullsized":"2023-10-16T11:27:23.442094+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:16:49.172852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.24","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.440644+0000","last_change":"2023-10-16T11:27:23.463422+0000","last_active":"2023-10-16T11:27:24.440644+0000","last_peered":"2023-10-16T11:27:24.440644+0000","last_clean":"2023-10-16T11:27:24.440644+0000","last_became_active":"2023-10-16T11:27:23.463089+0000","last_became_peered":"2023-10-16T11:27:23.463089+0000","last_unstale":"2023-10-16T11:27:24.440644+0000","last_undegraded":"2023-10-16T11:27:24.440644+0000","last_fullsized":"2023-10-16T11:27:24.440644+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:23:16.110019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"259'4825","reported_seq":5918,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.470715+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:27:24.470715+0000","last_peered":"2023-10-16T11:27:24.470715+0000","last_clean":"2023-10-16T11:27:24.470715+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:27:24.470715+0000","last_undegraded":"2023-10-16T11:27:24.470715+0000","last_fullsized":"2023-10-16T11:27:24.470715+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":4825,"log_dups_size":0,"ondisk_log_size":4825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2687,"num_read_kb":5529,"num_write":683,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":37,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":36,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2687,"num_read_kb":5529,"num_write":683,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4825,"ondisk_log_size":4825,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576018,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79512,"kb_used_data":3916,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94292328,"statfs":{"total":96636764160,"available":96555343872,"internally_reserved":0,"allocated":4009984,"data_stored":2263146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":77396427},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1140000000000001,"15min":1.1859999999999999},"min":{"1min":0.64500000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.673,"5min":1.7250000000000001,"15min":2.3399999999999999},"last":1.655},{"interface":"front","average":{"1min":1.109,"5min":1.1259999999999999,"15min":1.1930000000000001},"min":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.633,"5min":1.6699999999999999,"15min":1.71},"last":0.93400000000000005}]},{"osd":1,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2010000000000001,"15min":1.2290000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1719999999999999,"15min":1.2470000000000001},"min":{"1min":0.75600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.6100000000000001,"5min":1.6799999999999999,"15min":1.8600000000000001},"last":1.1100000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2749999999999999,"15min":1.286},"min":{"1min":0.71799999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8640000000000001,"5min":1.9039999999999999,"15min":2.4460000000000002},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.3140000000000001,"15min":1.349},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.5389999999999999,"5min":3.9249999999999998,"15min":3.9249999999999998},"last":1.3029999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.339,"15min":1.371},"min":{"1min":0.85999999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8169999999999999,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.389,"15min":1.4339999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.016,"5min":2.016,"15min":2.577},"last":1.355}]},{"osd":4,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.419,"15min":1.4039999999999999},"min":{"1min":0.64800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.113,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.379,"15min":1.3779999999999999},"min":{"1min":0.65600000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.756,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.472}]},{"osd":5,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.5669999999999999,"15min":1.5780000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":3.3519999999999999},"last":1.7909999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.534,"15min":1.498},"min":{"1min":0.62,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.0880000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.591}]},{"osd":6,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.488,"15min":1.4970000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.907,"5min":2.2469999999999999,"15min":2.7250000000000001},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.633,"5min":1.6639999999999999,"15min":1.6859999999999999},"min":{"1min":0.749,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.323,"5min":2.323,"15min":3.1440000000000001},"last":1.73}]}]},{"osd":6,"up_from":13,"seq":55834576018,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52580,"kb_used_data":3348,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319260,"statfs":{"total":96636764160,"available":96582922240,"internally_reserved":0,"allocated":3428352,"data_stored":1674390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.1779999999999999,"15min":1.206},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.5700000000000001,"5min":6.298,"15min":6.298},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.3129999999999999,"15min":1.278},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6140000000000001,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":1.1000000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.69799999999999995,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.24,"5min":1.395,"15min":1.365},"min":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.956,"5min":12.468,"15min":12.468},"last":1.385}]},{"osd":2,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.286,"15min":1.262},"min":{"1min":0.82899999999999996,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4870000000000001,"5min":11.586,"15min":11.586},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.3799999999999999,"15min":1.3720000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.77500000000000002},"max":{"1min":1.8600000000000001,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.3540000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.4199999999999999,"15min":1.3440000000000001},"min":{"1min":0.86299999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.925,"5min":12.801,"15min":12.801},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.177,"5min":1.2809999999999999,"15min":1.2649999999999999},"min":{"1min":0.88100000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5589999999999999,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":1.157}]},{"osd":4,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.417,"15min":1.3320000000000001},"min":{"1min":0.76300000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.806,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.046},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.6359999999999999,"15min":1.548},"min":{"1min":1.109,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.3069999999999999,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.216}]},{"osd":5,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.5469999999999999,"15min":1.3999999999999999},"min":{"1min":0.86199999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.8979999999999999,"5min":18.247,"15min":18.247},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.617,"5min":1.625,"15min":1.5660000000000001},"min":{"1min":0.85899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.923,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.597}]},{"osd":7,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.6779999999999999,"5min":1.6819999999999999,"15min":1.5529999999999999},"min":{"1min":0.92500000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.4660000000000002,"5min":12.589,"15min":12.589},"last":1.528},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.7110000000000001,"15min":1.6020000000000001},"min":{"1min":1.1779999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.2050000000000001,"5min":12.725,"15min":12.725},"last":1.5600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834576018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52572,"kb_used_data":3340,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319268,"statfs":{"total":96636764160,"available":96582930432,"internally_reserved":0,"allocated":3420160,"data_stored":1668152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.1519999999999999,"15min":1.234},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.585,"5min":1.585,"15min":2.661},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.248,"15min":1.3169999999999999},"min":{"1min":0.75600000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.583,"5min":1.7909999999999999,"15min":1.7949999999999999},"last":0.96299999999999997}]},{"osd":1,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1279999999999999,"15min":1.175},"min":{"1min":0.82199999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.748,"5min":1.748,"15min":1.992},"last":0.878},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.1919999999999999,"15min":1.228},"min":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":2.9399999999999999},"last":0.81799999999999995}]},{"osd":2,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2470000000000001,"15min":1.2390000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.274},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2869999999999999,"15min":1.3009999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7829999999999999,"5min":1.8660000000000001,"15min":2.1640000000000001},"last":1.3799999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.29,"15min":1.3169999999999999},"min":{"1min":0.82199999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.121,"5min":2.121,"15min":3.012},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.429,"5min":1.387,"15min":1.411},"min":{"1min":1.0880000000000001,"5min":0.92700000000000005,"15min":0.89500000000000002},"max":{"1min":1.9299999999999999,"5min":2.0129999999999999,"15min":2.2250000000000001},"last":1.2}]},{"osd":4,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.3280000000000001,"15min":1.345},"min":{"1min":0.80300000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.8660000000000001},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.605,"5min":1.5429999999999999,"15min":1.5620000000000001},"min":{"1min":1.125,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.903},"last":1.544}]},{"osd":6,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.4059999999999999,"15min":1.417},"min":{"1min":0.95099999999999996,"5min":0.68100000000000005,"15min":0.51000000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0979999999999999},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.53,"5min":1.4379999999999999,"15min":1.4770000000000001},"min":{"1min":1.004,"5min":0.90700000000000003,"15min":0.89200000000000002},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":3.073},"last":1.302}]},{"osd":7,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.552,"15min":1.548},"min":{"1min":1.228,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":3.1160000000000001},"last":1.8069999999999999},{"interface":"front","average":{"1min":1.702,"5min":1.619,"15min":1.639},"min":{"1min":1.2809999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.161,"5min":2.1930000000000001,"15min":2.2930000000000001},"last":1.6200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834576019,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88604,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94283236,"statfs":{"total":96636764160,"available":96546033664,"internally_reserved":0,"allocated":3424256,"data_stored":1673430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":87292358},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.151,"15min":1.1739999999999999},"min":{"1min":0.73799999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.615,"5min":2.899,"15min":2.899},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.296,"5min":1.236,"15min":1.254},"min":{"1min":0.77500000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.784,"5min":3.016,"15min":3.016},"last":1.252}]},{"osd":1,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.28,"15min":1.298},"min":{"1min":0.83999999999999997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6799999999999999,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.111},{"interface":"front","average":{"1min":1.089,"5min":1.228,"15min":1.25},"min":{"1min":0.84499999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.4299999999999999,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.429}]},{"osd":2,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8300000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.157},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4370000000000001,"15min":1.4199999999999999},"min":{"1min":0.90500000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.748,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.591}]},{"osd":3,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.351,"15min":1.349},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.7669999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.452,"5min":1.4930000000000001,"15min":1.4610000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.9930000000000001,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.494}]},{"osd":5,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":1.542,"15min":1.4790000000000001},"min":{"1min":1.0549999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.915,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.375},{"interface":"front","average":{"1min":1.444,"5min":1.4650000000000001,"15min":1.4370000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.9410000000000001,"5min":3.7989999999999999,"15min":3.7989999999999999},"last":1.3080000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.6020000000000001,"15min":1.5349999999999999},"min":{"1min":0.85699999999999998,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":2.0779999999999998,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.643},{"interface":"front","average":{"1min":1.724,"5min":1.679,"15min":1.6459999999999999},"min":{"1min":1.268,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.944,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.694}]},{"osd":7,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":1.647,"15min":1.607},"min":{"1min":0.73699999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.1949999999999998,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.702,"5min":1.7190000000000001,"15min":1.6799999999999999},"min":{"1min":1.026,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.1419999999999999,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.7589999999999999}]}]},{"osd":3,"up_from":13,"seq":55834576018,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52572,"kb_used_data":3340,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319268,"statfs":{"total":96636764160,"available":96582930432,"internally_reserved":0,"allocated":3420160,"data_stored":1673360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81699999999999995,"15min":0.88200000000000001},"min":{"1min":0.33800000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.901},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.89200000000000002,"15min":1.0249999999999999},"min":{"1min":0.443,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1919999999999999,"5min":1.581,"15min":1.6679999999999999},"last":0.316}]},{"osd":1,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.88800000000000001,"15min":0.95899999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3420000000000001,"5min":1.508,"15min":1.508},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.94899999999999995,"15min":1.087},"min":{"1min":0.51300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4770000000000001,"5min":1.601,"15min":1.7190000000000001},"last":0.435}]},{"osd":2,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.99199999999999999,"15min":1.087},"min":{"1min":0.56299999999999994,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.524,"5min":1.5660000000000001,"15min":1.694},"last":0.495},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.079,"15min":1.1870000000000001},"min":{"1min":0.48499999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.597,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.61199999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.23,"15min":1.3240000000000001},"min":{"1min":0.72499999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6839999999999999,"5min":1.821,"15min":1.873},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2949999999999999,"15min":1.3440000000000001},"min":{"1min":0.80900000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.681,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.98299999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.3009999999999999,"15min":1.385},"min":{"1min":0.69299999999999995,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.7749999999999999,"5min":2.798,"15min":2.798},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3149999999999999,"15min":1.4179999999999999},"min":{"1min":0.74199999999999999,"5min":0.63,"15min":0.63},"max":{"1min":1.772,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":0.95299999999999996}]},{"osd":6,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4330000000000001,"15min":1.51},"min":{"1min":0.93000000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.9970000000000001,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.413,"15min":1.506},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9350000000000001,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.111}]},{"osd":7,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.46,"15min":1.544},"min":{"1min":0.96799999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.8380000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.5329999999999999,"15min":1.6459999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.952,"5min":2.3799999999999999,"15min":2.4649999999999999},"last":0.94299999999999995}]}]},{"osd":2,"up_from":13,"seq":55834576017,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56740,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315100,"statfs":{"total":96636764160,"available":96578662400,"internally_reserved":0,"allocated":3424256,"data_stored":1672902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.85799999999999998,"15min":0.89900000000000002},"min":{"1min":0.39800000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.3200000000000001,"5min":1.532,"15min":1.6399999999999999},"last":1.177},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.92100000000000004,"15min":1.0369999999999999},"min":{"1min":0.53400000000000003,"5min":0.40500000000000003,"15min":0.32600000000000001},"max":{"1min":1.258,"5min":1.6359999999999999,"15min":2.1429999999999998},"last":0.92900000000000005}]},{"osd":1,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.046,"15min":1.075},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39200000000000002},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.02,"15min":1.1120000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.246}]},{"osd":3,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.96099999999999997,"15min":1.0469999999999999},"min":{"1min":0.52300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.452,"5min":1.522,"15min":1.8560000000000001},"last":1.02},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1639999999999999,"15min":1.242},"min":{"1min":0.58199999999999996,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.0459999999999998,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":1.458}]},{"osd":4,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.306,"15min":1.3819999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.8079999999999998,"5min":3.8079999999999998,"15min":3.8079999999999998},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.2490000000000001,"15min":1.3140000000000001},"min":{"1min":0.79700000000000004,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.2889999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.397,"15min":1.431},"min":{"1min":0.77900000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3839999999999999,"15min":1.4470000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0499999999999998,"5min":2.323,"15min":2.6259999999999999},"last":1.508}]},{"osd":6,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3560000000000001,"15min":1.381},"min":{"1min":0.85099999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.4830000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":1.131},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.464,"15min":1.5089999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":1.5329999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.4910000000000001,"15min":1.5489999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.742,"5min":3.742,"15min":3.742},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.4339999999999999,"15min":1.4810000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":1.367}]}]},{"osd":1,"up_from":13,"seq":55834576017,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48412,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323428,"statfs":{"total":96636764160,"available":96587190272,"internally_reserved":0,"allocated":3424256,"data_stored":1672834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.89300000000000002,"15min":0.91000000000000003},"min":{"1min":0.40699999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4039999999999999,"5min":5.117,"15min":5.117},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.89300000000000002,"15min":1.0029999999999999},"min":{"1min":0.45500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.325,"5min":1.5509999999999999,"15min":1.667},"last":0.82399999999999995}]},{"osd":2,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.91300000000000003,"15min":0.98899999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.651,"5min":2.093,"15min":2.093},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.149,"15min":1.242},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2400000000000002,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.872}]},{"osd":3,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.125,"15min":1.179},"min":{"1min":0.56999999999999995,"5min":0.439,"15min":0.439},"max":{"1min":1.8879999999999999,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.05,"15min":1.1399999999999999},"min":{"1min":0.52700000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5860000000000001,"5min":1.798,"15min":1.802},"last":1.238}]},{"osd":4,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.2749999999999999,"15min":1.2849999999999999},"min":{"1min":0.70299999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.978,"5min":5.29,"15min":5.29},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.403,"15min":1.409},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.0129999999999999,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":1.5129999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.3400000000000001,"15min":1.3979999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.829,"5min":6.0979999999999999,"15min":6.0979999999999999},"last":1.673},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.3420000000000001,"15min":1.3899999999999999},"min":{"1min":0.74099999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7410000000000001,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.5569999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.429,"15min":1.482},"min":{"1min":0.82399999999999995,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8979999999999999,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":1.7789999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.393,"15min":1.4630000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.948,"5min":3.8799999999999999,"15min":3.8799999999999999},"last":1.393}]},{"osd":7,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.518,"15min":1.5509999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.157,"5min":4.46,"15min":4.46},"last":1.722},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.5349999999999999,"15min":1.589},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.0259999999999998,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":1.8799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834576018,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48992,"kb_used_data":3924,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322848,"statfs":{"total":96636764160,"available":96586596352,"internally_reserved":0,"allocated":4018176,"data_stored":2263238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85499999999999998,"15min":0.88500000000000001},"min":{"1min":0.44900000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2350000000000001,"5min":1.502,"15min":1.619},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.98699999999999999,"15min":1.0740000000000001},"min":{"1min":0.40000000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.518,"5min":1.621,"15min":1.76},"last":0.91500000000000004}]},{"osd":2,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0640000000000001,"15min":1.0820000000000001},"min":{"1min":0.51300000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.569,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.996,"5min":1.036,"15min":1.1180000000000001},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.7649999999999999,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":1.05}]},{"osd":3,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0089999999999999,"15min":1.0640000000000001},"min":{"1min":0.621,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.5189999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.131,"15min":1.2170000000000001},"min":{"1min":0.59099999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.45,"5min":1.833,"15min":1.833},"last":1.2090000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.3149999999999999,"15min":1.377},"min":{"1min":0.93000000000000005,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.538,"5min":3.2799999999999998,"15min":3.2799999999999998},"last":1.284},{"interface":"front","average":{"1min":1.286,"5min":1.2869999999999999,"15min":1.3320000000000001},"min":{"1min":0.80000000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.752,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.6140000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.3109999999999999,"15min":1.3660000000000001},"min":{"1min":0.76000000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.605,"5min":2.0209999999999999,"15min":2.6669999999999998},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.478,"5min":1.385,"15min":1.4339999999999999},"min":{"1min":0.877,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.498}]},{"osd":6,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4570000000000001,"15min":1.4930000000000001},"min":{"1min":0.78900000000000003,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.1309999999999998,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":1.427},{"interface":"front","average":{"1min":1.363,"5min":1.4019999999999999,"15min":1.468},"min":{"1min":0.90200000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7310000000000001,"5min":2.073,"15min":2.262},"last":1.1639999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5309999999999999,"15min":1.5549999999999999},"min":{"1min":1.0429999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7869999999999999,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.419,"15min":1.486},"min":{"1min":0.98399999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7869999999999999,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.5309999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:27:27.913 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:27:28.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.164+0000 7facc32e6640 1 Processor -- start 2023-10-16T11:27:28.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.164+0000 7facc32e6640 1 -- start start 2023-10-16T11:27:28.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.166+0000 7facc32e6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7facbc15cc80 0x7facbc0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:28.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.166+0000 7facc32e6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facbc0cdd60 0x7facbc071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:28.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.166+0000 7facc32e6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7facbc071fa0 0x7facbc1607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:28.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.166+0000 7facc32e6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facbc0d0500 con 0x7facbc071fa0 2023-10-16T11:27:28.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.166+0000 7facc32e6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facbc0ce1c0 con 0x7facbc15cc80 2023-10-16T11:27:28.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.166+0000 7facc32e6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facbc0d12f0 con 0x7facbc0cdd60 2023-10-16T11:27:28.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.166+0000 7facc2ae5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7facbc071fa0 0x7facbc1607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:28.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.166+0000 7facc2ae5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7facbc071fa0 0x7facbc1607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42430/0 (socket says 172.21.15.84:42430) 2023-10-16T11:27:28.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.166+0000 7facc2ae5640 1 -- 172.21.15.84:0/1782456927 learned_addr learned my addr 172.21.15.84:0/1782456927 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:28.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.166+0000 7facc22e4640 1 --2- 172.21.15.84:0/1782456927 >> v2:172.21.15.157:3300/0 conn(0x7facbc15cc80 0x7facbc0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:28.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.166+0000 7facc1ae3640 1 --2- 172.21.15.84:0/1782456927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facbc0cdd60 0x7facbc071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:28.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.166+0000 7facc2ae5640 1 -- 172.21.15.84:0/1782456927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facbc0cdd60 msgr2=0x7facbc071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:28.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.166+0000 7facc2ae5640 1 --2- 172.21.15.84:0/1782456927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facbc0cdd60 0x7facbc071a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.166+0000 7facc2ae5640 1 -- 172.21.15.84:0/1782456927 >> v2:172.21.15.157:3300/0 conn(0x7facbc15cc80 msgr2=0x7facbc0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:28.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.167+0000 7facc2ae5640 1 --2- 172.21.15.84:0/1782456927 >> v2:172.21.15.157:3300/0 conn(0x7facbc15cc80 0x7facbc0cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.167+0000 7facc2ae5640 1 -- 172.21.15.84:0/1782456927 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facbc072410 con 0x7facbc071fa0 2023-10-16T11:27:28.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.168+0000 7facc2ae5640 1 --2- 172.21.15.84:0/1782456927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facbc071fa0 0x7facbc1607e0 secure :-1 s=READY pgs=3408 cs=0 l=1 rev1=1 crypto rx=0x7facbc0ce370 tx=0x7facb402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd8a399085a5948 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:28.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.168+0000 7facc12e2640 1 -- 172.21.15.84:0/1782456927 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb402ca80 con 0x7facbc071fa0 2023-10-16T11:27:28.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.168+0000 7facc12e2640 1 -- 172.21.15.84:0/1782456927 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb402cbe0 con 0x7facbc071fa0 2023-10-16T11:27:28.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.168+0000 7facc12e2640 1 -- 172.21.15.84:0/1782456927 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb40343f0 con 0x7facbc071fa0 2023-10-16T11:27:28.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.168+0000 7facc32e6640 1 -- 172.21.15.84:0/1782456927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facbc071fa0 msgr2=0x7facbc1607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:28.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.168+0000 7facc32e6640 1 --2- 172.21.15.84:0/1782456927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facbc071fa0 0x7facbc1607e0 secure :-1 s=READY pgs=3408 cs=0 l=1 rev1=1 crypto rx=0x7facbc0ce370 tx=0x7facb402bee0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.169+0000 7facc32e6640 1 -- 172.21.15.84:0/1782456927 shutdown_connections 2023-10-16T11:27:28.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.169+0000 7facc32e6640 1 --2- 172.21.15.84:0/1782456927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facbc071fa0 0x7facbc1607e0 unknown :-1 s=CLOSED pgs=3408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.169+0000 7facc32e6640 1 --2- 172.21.15.84:0/1782456927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facbc0cdd60 0x7facbc071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.169+0000 7facc32e6640 1 --2- 172.21.15.84:0/1782456927 >> v2:172.21.15.157:3300/0 conn(0x7facbc15cc80 0x7facbc0cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.169+0000 7facc32e6640 1 -- 172.21.15.84:0/1782456927 >> 172.21.15.84:0/1782456927 conn(0x7facbc065ed0 msgr2=0x7facbc0662e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:28.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.169+0000 7facc32e6640 1 -- 172.21.15.84:0/1782456927 shutdown_connections 2023-10-16T11:27:28.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.169+0000 7facc32e6640 1 -- 172.21.15.84:0/1782456927 wait complete. 2023-10-16T11:27:28.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.171+0000 7facc32e6640 1 Processor -- start 2023-10-16T11:27:28.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.171+0000 7facc32e6640 1 -- start start 2023-10-16T11:27:28.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.172+0000 7facc32e6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facbc071fa0 0x7facbc0b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:28.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.172+0000 7facc32e6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facbc0cdd60 0x7facbc15f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:28.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.172+0000 7facc32e6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facbc15cc80 0x7facbc15f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:28.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.172+0000 7facc32e6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facbc0cc130 con 0x7facbc15cc80 2023-10-16T11:27:28.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.172+0000 7facc32e6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facbc0b4f30 con 0x7facbc071fa0 2023-10-16T11:27:28.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.172+0000 7facc32e6640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facbc0cb890 con 0x7facbc0cdd60 2023-10-16T11:27:28.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.172+0000 7facc2ae5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facbc15cc80 0x7facbc15f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:28.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.172+0000 7facc2ae5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facbc15cc80 0x7facbc15f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42438/0 (socket says 172.21.15.84:42438) 2023-10-16T11:27:28.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.172+0000 7facc2ae5640 1 -- 172.21.15.84:0/243509998 learned_addr learned my addr 172.21.15.84:0/243509998 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:28.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.173+0000 7facc1ae3640 1 --2- 172.21.15.84:0/243509998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facbc0cdd60 0x7facbc15f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:28.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.173+0000 7facc2ae5640 1 -- 172.21.15.84:0/243509998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facbc0cdd60 msgr2=0x7facbc15f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:28.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.173+0000 7facc22e4640 1 --2- 172.21.15.84:0/243509998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facbc071fa0 0x7facbc0b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:28.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.173+0000 7facc2ae5640 1 --2- 172.21.15.84:0/243509998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facbc0cdd60 0x7facbc15f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.173+0000 7facc2ae5640 1 -- 172.21.15.84:0/243509998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facbc071fa0 msgr2=0x7facbc0b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:28.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.173+0000 7facc2ae5640 1 --2- 172.21.15.84:0/243509998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facbc071fa0 0x7facbc0b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.173+0000 7facc2ae5640 1 -- 172.21.15.84:0/243509998 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb4009b30 con 0x7facbc15cc80 2023-10-16T11:27:28.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.173+0000 7facc2ae5640 1 --2- 172.21.15.84:0/243509998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facbc15cc80 0x7facbc15f770 secure :-1 s=READY pgs=3409 cs=0 l=1 rev1=1 crypto rx=0x7facb4036040 tx=0x7facb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:28.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.173+0000 7facb2ffd640 1 -- 172.21.15.84:0/243509998 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb4009510 con 0x7facbc15cc80 2023-10-16T11:27:28.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.173+0000 7facb2ffd640 1 -- 172.21.15.84:0/243509998 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb403e450 con 0x7facbc15cc80 2023-10-16T11:27:28.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.173+0000 7facb2ffd640 1 -- 172.21.15.84:0/243509998 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb403e800 con 0x7facbc15cc80 2023-10-16T11:27:28.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.173+0000 7facc32e6640 1 -- 172.21.15.84:0/243509998 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facbc0cbb70 con 0x7facbc15cc80 2023-10-16T11:27:28.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.174+0000 7facc32e6640 1 -- 172.21.15.84:0/243509998 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facbc0c79b0 con 0x7facbc15cc80 2023-10-16T11:27:28.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.175+0000 7facc32e6640 1 -- 172.21.15.84:0/243509998 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facbc0b4d00 con 0x7facbc15cc80 2023-10-16T11:27:28.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.176+0000 7facb2ffd640 1 -- 172.21.15.84:0/243509998 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7facb402cd50 con 0x7facbc15cc80 2023-10-16T11:27:28.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.177+0000 7facb2ffd640 1 --2- 172.21.15.84:0/243509998 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7faca006af90 0x7faca006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:28.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.177+0000 7facb2ffd640 1 -- 172.21.15.84:0/243509998 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 5737+0+0 (secure 0 0 0) 0x7facb40ac310 con 0x7facbc15cc80 2023-10-16T11:27:28.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.177+0000 7facc22e4640 1 --2- 172.21.15.84:0/243509998 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7faca006af90 0x7faca006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:28.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.177+0000 7facc22e4640 1 --2- 172.21.15.84:0/243509998 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7faca006af90 0x7faca006d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7facb800b740 tx=0x7facb8008040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:28.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.181+0000 7facb2ffd640 1 -- 172.21.15.84:0/243509998 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7facbc0b4d00 con 0x7facbc15cc80 2023-10-16T11:27:28.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.368+0000 7facc32e6640 1 -- 172.21.15.84:0/243509998 --> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7facbc0d09c0 con 0x7faca006af90 2023-10-16T11:27:28.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.391+0000 7facb2ffd640 1 -- 172.21.15.84:0/243509998 <== mgr.13239 v2:172.21.15.84:6832/731888425 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287615 (secure 0 0 0) 0x7facb800c040 con 0x7faca006af90 2023-10-16T11:27:28.396 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:28.398 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:27:28.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.397+0000 7facc32e6640 1 -- 172.21.15.84:0/243509998 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7faca006af90 msgr2=0x7faca006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:28.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.397+0000 7facc32e6640 1 --2- 172.21.15.84:0/243509998 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7faca006af90 0x7faca006d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7facb800b740 tx=0x7facb8008040 comp rx=0 tx=0).stop 2023-10-16T11:27:28.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.397+0000 7facc32e6640 1 -- 172.21.15.84:0/243509998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facbc15cc80 msgr2=0x7facbc15f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:28.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.397+0000 7facc32e6640 1 --2- 172.21.15.84:0/243509998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facbc15cc80 0x7facbc15f770 secure :-1 s=READY pgs=3409 cs=0 l=1 rev1=1 crypto rx=0x7facb4036040 tx=0x7facb4004a40 comp rx=0 tx=0).stop 2023-10-16T11:27:28.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.398+0000 7facc32e6640 1 -- 172.21.15.84:0/243509998 shutdown_connections 2023-10-16T11:27:28.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.398+0000 7facc32e6640 1 --2- 172.21.15.84:0/243509998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facbc15cc80 0x7facbc15f770 unknown :-1 s=CLOSED pgs=3409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.398+0000 7facc32e6640 1 --2- 172.21.15.84:0/243509998 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7faca006af90 0x7faca006d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.398+0000 7facc32e6640 1 --2- 172.21.15.84:0/243509998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facbc0cdd60 0x7facbc15f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.398+0000 7facc32e6640 1 --2- 172.21.15.84:0/243509998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facbc071fa0 0x7facbc0b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.398+0000 7facc32e6640 1 -- 172.21.15.84:0/243509998 >> 172.21.15.84:0/243509998 conn(0x7facbc065ed0 msgr2=0x7facbc0ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:28.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.398+0000 7facc32e6640 1 -- 172.21.15.84:0/243509998 shutdown_connections 2023-10-16T11:27:28.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.398+0000 7facc32e6640 1 -- 172.21.15.84:0/243509998 wait complete. 2023-10-16T11:27:28.418 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-16T11:27:27.528674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2687,"num_read_kb":5529,"num_write":727,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4863,"ondisk_log_size":4863,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":479984,"kb_used_data":27900,"kb_used_omap":12,"kb_used_meta":451955,"kb_avail":754494736,"statfs":{"total":773094113280,"available":772602609664,"internally_reserved":0,"allocated":28569600,"data_stored":14561452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":462802489},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001059"},"pg_stats":[{"pgid":"37.3f","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.746978+0000","last_change":"2023-10-16T11:27:23.468831+0000","last_active":"2023-10-16T11:27:25.746978+0000","last_peered":"2023-10-16T11:27:25.746978+0000","last_clean":"2023-10-16T11:27:25.746978+0000","last_became_active":"2023-10-16T11:27:23.468594+0000","last_became_peered":"2023-10-16T11:27:23.468594+0000","last_unstale":"2023-10-16T11:27:25.746978+0000","last_undegraded":"2023-10-16T11:27:25.746978+0000","last_fullsized":"2023-10-16T11:27:25.746978+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:32:14.401834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.3e","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.443450+0000","last_change":"2023-10-16T11:27:23.467568+0000","last_active":"2023-10-16T11:27:24.443450+0000","last_peered":"2023-10-16T11:27:24.443450+0000","last_clean":"2023-10-16T11:27:24.443450+0000","last_became_active":"2023-10-16T11:27:23.467213+0000","last_became_peered":"2023-10-16T11:27:23.467213+0000","last_unstale":"2023-10-16T11:27:24.443450+0000","last_undegraded":"2023-10-16T11:27:24.443450+0000","last_fullsized":"2023-10-16T11:27:24.443450+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:02.508652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.3d","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.440663+0000","last_change":"2023-10-16T11:27:23.454799+0000","last_active":"2023-10-16T11:27:24.440663+0000","last_peered":"2023-10-16T11:27:24.440663+0000","last_clean":"2023-10-16T11:27:24.440663+0000","last_became_active":"2023-10-16T11:27:23.454408+0000","last_became_peered":"2023-10-16T11:27:23.454408+0000","last_unstale":"2023-10-16T11:27:24.440663+0000","last_undegraded":"2023-10-16T11:27:24.440663+0000","last_fullsized":"2023-10-16T11:27:24.440663+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:01.719295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.3c","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.440471+0000","last_change":"2023-10-16T11:27:23.455745+0000","last_active":"2023-10-16T11:27:24.440471+0000","last_peered":"2023-10-16T11:27:24.440471+0000","last_clean":"2023-10-16T11:27:24.440471+0000","last_became_active":"2023-10-16T11:27:23.454456+0000","last_became_peered":"2023-10-16T11:27:23.454456+0000","last_unstale":"2023-10-16T11:27:24.440471+0000","last_undegraded":"2023-10-16T11:27:24.440471+0000","last_fullsized":"2023-10-16T11:27:24.440471+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:09.556639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.3b","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.440683+0000","last_change":"2023-10-16T11:27:23.471412+0000","last_active":"2023-10-16T11:27:24.440683+0000","last_peered":"2023-10-16T11:27:24.440683+0000","last_clean":"2023-10-16T11:27:24.440683+0000","last_became_active":"2023-10-16T11:27:23.470036+0000","last_became_peered":"2023-10-16T11:27:23.470036+0000","last_unstale":"2023-10-16T11:27:24.440683+0000","last_undegraded":"2023-10-16T11:27:24.440683+0000","last_fullsized":"2023-10-16T11:27:24.440683+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:52.119044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.3a","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.741736+0000","last_change":"2023-10-16T11:27:23.476060+0000","last_active":"2023-10-16T11:27:25.741736+0000","last_peered":"2023-10-16T11:27:25.741736+0000","last_clean":"2023-10-16T11:27:25.741736+0000","last_became_active":"2023-10-16T11:27:23.474212+0000","last_became_peered":"2023-10-16T11:27:23.474212+0000","last_unstale":"2023-10-16T11:27:25.741736+0000","last_undegraded":"2023-10-16T11:27:25.741736+0000","last_fullsized":"2023-10-16T11:27:25.741736+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:40.261473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.39","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.744540+0000","last_change":"2023-10-16T11:27:23.460009+0000","last_active":"2023-10-16T11:27:25.744540+0000","last_peered":"2023-10-16T11:27:25.744540+0000","last_clean":"2023-10-16T11:27:25.744540+0000","last_became_active":"2023-10-16T11:27:23.459356+0000","last_became_peered":"2023-10-16T11:27:23.459356+0000","last_unstale":"2023-10-16T11:27:25.744540+0000","last_undegraded":"2023-10-16T11:27:25.744540+0000","last_fullsized":"2023-10-16T11:27:25.744540+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:55.507612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.38","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.531882+0000","last_change":"2023-10-16T11:27:23.458731+0000","last_active":"2023-10-16T11:27:25.531882+0000","last_peered":"2023-10-16T11:27:25.531882+0000","last_clean":"2023-10-16T11:27:25.531882+0000","last_became_active":"2023-10-16T11:27:23.458425+0000","last_became_peered":"2023-10-16T11:27:23.458425+0000","last_unstale":"2023-10-16T11:27:25.531882+0000","last_undegraded":"2023-10-16T11:27:25.531882+0000","last_fullsized":"2023-10-16T11:27:25.531882+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:00:42.986177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.37","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.443257+0000","last_change":"2023-10-16T11:27:23.468135+0000","last_active":"2023-10-16T11:27:24.443257+0000","last_peered":"2023-10-16T11:27:24.443257+0000","last_clean":"2023-10-16T11:27:24.443257+0000","last_became_active":"2023-10-16T11:27:23.467862+0000","last_became_peered":"2023-10-16T11:27:23.467862+0000","last_unstale":"2023-10-16T11:27:24.443257+0000","last_undegraded":"2023-10-16T11:27:24.443257+0000","last_fullsized":"2023-10-16T11:27:24.443257+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:54:48.482099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.36","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.745976+0000","last_change":"2023-10-16T11:27:23.460169+0000","last_active":"2023-10-16T11:27:25.745976+0000","last_peered":"2023-10-16T11:27:25.745976+0000","last_clean":"2023-10-16T11:27:25.745976+0000","last_became_active":"2023-10-16T11:27:23.459597+0000","last_became_peered":"2023-10-16T11:27:23.459597+0000","last_unstale":"2023-10-16T11:27:25.745976+0000","last_undegraded":"2023-10-16T11:27:25.745976+0000","last_fullsized":"2023-10-16T11:27:25.745976+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:56.725852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.35","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.532297+0000","last_change":"2023-10-16T11:27:23.470722+0000","last_active":"2023-10-16T11:27:25.532297+0000","last_peered":"2023-10-16T11:27:25.532297+0000","last_clean":"2023-10-16T11:27:25.532297+0000","last_became_active":"2023-10-16T11:27:23.470462+0000","last_became_peered":"2023-10-16T11:27:23.470462+0000","last_unstale":"2023-10-16T11:27:25.532297+0000","last_undegraded":"2023-10-16T11:27:25.532297+0000","last_fullsized":"2023-10-16T11:27:25.532297+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:36.797546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.34","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.471995+0000","last_change":"2023-10-16T11:27:23.466997+0000","last_active":"2023-10-16T11:27:24.471995+0000","last_peered":"2023-10-16T11:27:24.471995+0000","last_clean":"2023-10-16T11:27:24.471995+0000","last_became_active":"2023-10-16T11:27:23.466671+0000","last_became_peered":"2023-10-16T11:27:23.466671+0000","last_unstale":"2023-10-16T11:27:24.471995+0000","last_undegraded":"2023-10-16T11:27:24.471995+0000","last_fullsized":"2023-10-16T11:27:24.471995+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:57.314448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.33","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.746072+0000","last_change":"2023-10-16T11:27:23.479624+0000","last_active":"2023-10-16T11:27:25.746072+0000","last_peered":"2023-10-16T11:27:25.746072+0000","last_clean":"2023-10-16T11:27:25.746072+0000","last_became_active":"2023-10-16T11:27:23.479327+0000","last_became_peered":"2023-10-16T11:27:23.479327+0000","last_unstale":"2023-10-16T11:27:25.746072+0000","last_undegraded":"2023-10-16T11:27:25.746072+0000","last_fullsized":"2023-10-16T11:27:25.746072+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:57:41.546314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.32","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.442918+0000","last_change":"2023-10-16T11:27:23.461838+0000","last_active":"2023-10-16T11:27:24.442918+0000","last_peered":"2023-10-16T11:27:24.442918+0000","last_clean":"2023-10-16T11:27:24.442918+0000","last_became_active":"2023-10-16T11:27:23.460967+0000","last_became_peered":"2023-10-16T11:27:23.460967+0000","last_unstale":"2023-10-16T11:27:24.442918+0000","last_undegraded":"2023-10-16T11:27:24.442918+0000","last_fullsized":"2023-10-16T11:27:24.442918+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:09.863277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.31","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.440134+0000","last_change":"2023-10-16T11:27:23.485618+0000","last_active":"2023-10-16T11:27:24.440134+0000","last_peered":"2023-10-16T11:27:24.440134+0000","last_clean":"2023-10-16T11:27:24.440134+0000","last_became_active":"2023-10-16T11:27:23.485294+0000","last_became_peered":"2023-10-16T11:27:23.485294+0000","last_unstale":"2023-10-16T11:27:24.440134+0000","last_undegraded":"2023-10-16T11:27:24.440134+0000","last_fullsized":"2023-10-16T11:27:24.440134+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:30:41.982564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.30","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.470496+0000","last_change":"2023-10-16T11:27:23.484857+0000","last_active":"2023-10-16T11:27:24.470496+0000","last_peered":"2023-10-16T11:27:24.470496+0000","last_clean":"2023-10-16T11:27:24.470496+0000","last_became_active":"2023-10-16T11:27:23.484530+0000","last_became_peered":"2023-10-16T11:27:23.484530+0000","last_unstale":"2023-10-16T11:27:24.470496+0000","last_undegraded":"2023-10-16T11:27:24.470496+0000","last_fullsized":"2023-10-16T11:27:24.470496+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:52:11.968883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.2f","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.746838+0000","last_change":"2023-10-16T11:27:23.482344+0000","last_active":"2023-10-16T11:27:25.746838+0000","last_peered":"2023-10-16T11:27:25.746838+0000","last_clean":"2023-10-16T11:27:25.746838+0000","last_became_active":"2023-10-16T11:27:23.480710+0000","last_became_peered":"2023-10-16T11:27:23.480710+0000","last_unstale":"2023-10-16T11:27:25.746838+0000","last_undegraded":"2023-10-16T11:27:25.746838+0000","last_fullsized":"2023-10-16T11:27:25.746838+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:44.126144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.2e","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.746056+0000","last_change":"2023-10-16T11:27:23.478674+0000","last_active":"2023-10-16T11:27:25.746056+0000","last_peered":"2023-10-16T11:27:25.746056+0000","last_clean":"2023-10-16T11:27:25.746056+0000","last_became_active":"2023-10-16T11:27:23.477899+0000","last_became_peered":"2023-10-16T11:27:23.477899+0000","last_unstale":"2023-10-16T11:27:25.746056+0000","last_undegraded":"2023-10-16T11:27:25.746056+0000","last_fullsized":"2023-10-16T11:27:25.746056+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:50.441137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.2d","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.444227+0000","last_change":"2023-10-16T11:27:23.473911+0000","last_active":"2023-10-16T11:27:24.444227+0000","last_peered":"2023-10-16T11:27:24.444227+0000","last_clean":"2023-10-16T11:27:24.444227+0000","last_became_active":"2023-10-16T11:27:23.473610+0000","last_became_peered":"2023-10-16T11:27:23.473610+0000","last_unstale":"2023-10-16T11:27:24.444227+0000","last_undegraded":"2023-10-16T11:27:24.444227+0000","last_fullsized":"2023-10-16T11:27:24.444227+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:43.093225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.2c","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.443015+0000","last_change":"2023-10-16T11:27:23.454345+0000","last_active":"2023-10-16T11:27:24.443015+0000","last_peered":"2023-10-16T11:27:24.443015+0000","last_clean":"2023-10-16T11:27:24.443015+0000","last_became_active":"2023-10-16T11:27:23.453919+0000","last_became_peered":"2023-10-16T11:27:23.453919+0000","last_unstale":"2023-10-16T11:27:24.443015+0000","last_undegraded":"2023-10-16T11:27:24.443015+0000","last_fullsized":"2023-10-16T11:27:24.443015+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:12:31.482145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.2b","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.532145+0000","last_change":"2023-10-16T11:27:23.469571+0000","last_active":"2023-10-16T11:27:25.532145+0000","last_peered":"2023-10-16T11:27:25.532145+0000","last_clean":"2023-10-16T11:27:25.532145+0000","last_became_active":"2023-10-16T11:27:23.469316+0000","last_became_peered":"2023-10-16T11:27:23.469316+0000","last_unstale":"2023-10-16T11:27:25.532145+0000","last_undegraded":"2023-10-16T11:27:25.532145+0000","last_fullsized":"2023-10-16T11:27:25.532145+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:04:13.703694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.2a","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.440293+0000","last_change":"2023-10-16T11:27:23.467378+0000","last_active":"2023-10-16T11:27:24.440293+0000","last_peered":"2023-10-16T11:27:24.440293+0000","last_clean":"2023-10-16T11:27:24.440293+0000","last_became_active":"2023-10-16T11:27:23.465993+0000","last_became_peered":"2023-10-16T11:27:23.465993+0000","last_unstale":"2023-10-16T11:27:24.440293+0000","last_undegraded":"2023-10-16T11:27:24.440293+0000","last_fullsized":"2023-10-16T11:27:24.440293+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:16.206103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.29","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.745327+0000","last_change":"2023-10-16T11:27:23.479466+0000","last_active":"2023-10-16T11:27:25.745327+0000","last_peered":"2023-10-16T11:27:25.745327+0000","last_clean":"2023-10-16T11:27:25.745327+0000","last_became_active":"2023-10-16T11:27:23.479159+0000","last_became_peered":"2023-10-16T11:27:23.479159+0000","last_unstale":"2023-10-16T11:27:25.745327+0000","last_undegraded":"2023-10-16T11:27:25.745327+0000","last_fullsized":"2023-10-16T11:27:25.745327+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:01.719295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.28","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.442432+0000","last_change":"2023-10-16T11:27:23.464430+0000","last_active":"2023-10-16T11:27:24.442432+0000","last_peered":"2023-10-16T11:27:24.442432+0000","last_clean":"2023-10-16T11:27:24.442432+0000","last_became_active":"2023-10-16T11:27:23.462771+0000","last_became_peered":"2023-10-16T11:27:23.462771+0000","last_unstale":"2023-10-16T11:27:24.442432+0000","last_undegraded":"2023-10-16T11:27:24.442432+0000","last_fullsized":"2023-10-16T11:27:24.442432+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:32:52.329776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.27","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.747180+0000","last_change":"2023-10-16T11:27:23.481050+0000","last_active":"2023-10-16T11:27:25.747180+0000","last_peered":"2023-10-16T11:27:25.747180+0000","last_clean":"2023-10-16T11:27:25.747180+0000","last_became_active":"2023-10-16T11:27:23.480560+0000","last_became_peered":"2023-10-16T11:27:23.480560+0000","last_unstale":"2023-10-16T11:27:25.747180+0000","last_undegraded":"2023-10-16T11:27:25.747180+0000","last_fullsized":"2023-10-16T11:27:25.747180+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:32:23.653130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.26","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.746332+0000","last_change":"2023-10-16T11:27:23.472694+0000","last_active":"2023-10-16T11:27:25.746332+0000","last_peered":"2023-10-16T11:27:25.746332+0000","last_clean":"2023-10-16T11:27:25.746332+0000","last_became_active":"2023-10-16T11:27:23.472023+0000","last_became_peered":"2023-10-16T11:27:23.472023+0000","last_unstale":"2023-10-16T11:27:25.746332+0000","last_undegraded":"2023-10-16T11:27:25.746332+0000","last_fullsized":"2023-10-16T11:27:25.746332+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:36:02.221752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.25","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.471970+0000","last_change":"2023-10-16T11:27:23.473453+0000","last_active":"2023-10-16T11:27:24.471970+0000","last_peered":"2023-10-16T11:27:24.471970+0000","last_clean":"2023-10-16T11:27:24.471970+0000","last_became_active":"2023-10-16T11:27:23.473188+0000","last_became_peered":"2023-10-16T11:27:23.473188+0000","last_unstale":"2023-10-16T11:27:24.471970+0000","last_undegraded":"2023-10-16T11:27:24.471970+0000","last_fullsized":"2023-10-16T11:27:24.471970+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:51:45.291096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.23","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.741943+0000","last_change":"2023-10-16T11:27:23.477433+0000","last_active":"2023-10-16T11:27:25.741943+0000","last_peered":"2023-10-16T11:27:25.741943+0000","last_clean":"2023-10-16T11:27:25.741943+0000","last_became_active":"2023-10-16T11:27:23.477143+0000","last_became_peered":"2023-10-16T11:27:23.477143+0000","last_unstale":"2023-10-16T11:27:25.741943+0000","last_undegraded":"2023-10-16T11:27:25.741943+0000","last_fullsized":"2023-10-16T11:27:25.741943+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:03:33.273585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.22","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.745446+0000","last_change":"2023-10-16T11:27:23.482959+0000","last_active":"2023-10-16T11:27:25.745446+0000","last_peered":"2023-10-16T11:27:25.745446+0000","last_clean":"2023-10-16T11:27:25.745446+0000","last_became_active":"2023-10-16T11:27:23.482693+0000","last_became_peered":"2023-10-16T11:27:23.482693+0000","last_unstale":"2023-10-16T11:27:25.745446+0000","last_undegraded":"2023-10-16T11:27:25.745446+0000","last_fullsized":"2023-10-16T11:27:25.745446+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:52.119044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.21","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.442386+0000","last_change":"2023-10-16T11:27:23.468429+0000","last_active":"2023-10-16T11:27:24.442386+0000","last_peered":"2023-10-16T11:27:24.442386+0000","last_clean":"2023-10-16T11:27:24.442386+0000","last_became_active":"2023-10-16T11:27:23.467356+0000","last_became_peered":"2023-10-16T11:27:23.467356+0000","last_unstale":"2023-10-16T11:27:24.442386+0000","last_undegraded":"2023-10-16T11:27:24.442386+0000","last_fullsized":"2023-10-16T11:27:24.442386+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:30.202850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.20","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.439794+0000","last_change":"2023-10-16T11:27:23.455961+0000","last_active":"2023-10-16T11:27:24.439794+0000","last_peered":"2023-10-16T11:27:24.439794+0000","last_clean":"2023-10-16T11:27:24.439794+0000","last_became_active":"2023-10-16T11:27:23.454819+0000","last_became_peered":"2023-10-16T11:27:23.454819+0000","last_unstale":"2023-10-16T11:27:24.439794+0000","last_undegraded":"2023-10-16T11:27:24.439794+0000","last_fullsized":"2023-10-16T11:27:24.439794+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:22:05.724817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.1e","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.471403+0000","last_change":"2023-10-16T11:27:23.474396+0000","last_active":"2023-10-16T11:27:24.471403+0000","last_peered":"2023-10-16T11:27:24.471403+0000","last_clean":"2023-10-16T11:27:24.471403+0000","last_became_active":"2023-10-16T11:27:23.473949+0000","last_became_peered":"2023-10-16T11:27:23.473949+0000","last_unstale":"2023-10-16T11:27:24.471403+0000","last_undegraded":"2023-10-16T11:27:24.471403+0000","last_fullsized":"2023-10-16T11:27:24.471403+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:25:43.689973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.1f","version":"269'4","reported_seq":16,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.488760+0000","last_change":"2023-10-16T11:27:22.434039+0000","last_active":"2023-10-16T11:27:23.488760+0000","last_peered":"2023-10-16T11:27:23.488760+0000","last_clean":"2023-10-16T11:27:23.488760+0000","last_became_active":"2023-10-16T11:27:22.433533+0000","last_became_peered":"2023-10-16T11:27:22.433533+0000","last_unstale":"2023-10-16T11:27:23.488760+0000","last_undegraded":"2023-10-16T11:27:23.488760+0000","last_fullsized":"2023-10-16T11:27:23.488760+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:23:50.394436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.1f","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.443404+0000","last_change":"2023-10-16T11:27:23.486773+0000","last_active":"2023-10-16T11:27:24.443404+0000","last_peered":"2023-10-16T11:27:24.443404+0000","last_clean":"2023-10-16T11:27:24.443404+0000","last_became_active":"2023-10-16T11:27:23.485146+0000","last_became_peered":"2023-10-16T11:27:23.485146+0000","last_unstale":"2023-10-16T11:27:24.443404+0000","last_undegraded":"2023-10-16T11:27:24.443404+0000","last_fullsized":"2023-10-16T11:27:24.443404+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:56:18.831831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.1e","version":"269'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.487395+0000","last_change":"2023-10-16T11:27:22.430210+0000","last_active":"2023-10-16T11:27:23.487395+0000","last_peered":"2023-10-16T11:27:23.487395+0000","last_clean":"2023-10-16T11:27:23.487395+0000","last_became_active":"2023-10-16T11:27:22.429926+0000","last_became_peered":"2023-10-16T11:27:22.429926+0000","last_unstale":"2023-10-16T11:27:23.487395+0000","last_undegraded":"2023-10-16T11:27:23.487395+0000","last_fullsized":"2023-10-16T11:27:23.487395+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:08.846109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.1c","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.742121+0000","last_change":"2023-10-16T11:27:23.479224+0000","last_active":"2023-10-16T11:27:25.742121+0000","last_peered":"2023-10-16T11:27:25.742121+0000","last_clean":"2023-10-16T11:27:25.742121+0000","last_became_active":"2023-10-16T11:27:23.478884+0000","last_became_peered":"2023-10-16T11:27:23.478884+0000","last_unstale":"2023-10-16T11:27:25.742121+0000","last_undegraded":"2023-10-16T11:27:25.742121+0000","last_fullsized":"2023-10-16T11:27:25.742121+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:22:05.724817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.1d","version":"269'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.488222+0000","last_change":"2023-10-16T11:27:22.439649+0000","last_active":"2023-10-16T11:27:23.488222+0000","last_peered":"2023-10-16T11:27:23.488222+0000","last_clean":"2023-10-16T11:27:23.488222+0000","last_became_active":"2023-10-16T11:27:22.439385+0000","last_became_peered":"2023-10-16T11:27:22.439385+0000","last_unstale":"2023-10-16T11:27:23.488222+0000","last_undegraded":"2023-10-16T11:27:23.488222+0000","last_fullsized":"2023-10-16T11:27:23.488222+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:56:17.814662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.1d","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.532606+0000","last_change":"2023-10-16T11:27:23.474658+0000","last_active":"2023-10-16T11:27:25.532606+0000","last_peered":"2023-10-16T11:27:25.532606+0000","last_clean":"2023-10-16T11:27:25.532606+0000","last_became_active":"2023-10-16T11:27:23.469718+0000","last_became_peered":"2023-10-16T11:27:23.469718+0000","last_unstale":"2023-10-16T11:27:25.532606+0000","last_undegraded":"2023-10-16T11:27:25.532606+0000","last_fullsized":"2023-10-16T11:27:25.532606+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:00:07.418959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1c","version":"269'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.486877+0000","last_change":"2023-10-16T11:27:22.427045+0000","last_active":"2023-10-16T11:27:23.486877+0000","last_peered":"2023-10-16T11:27:23.486877+0000","last_clean":"2023-10-16T11:27:23.486877+0000","last_became_active":"2023-10-16T11:27:22.426728+0000","last_became_peered":"2023-10-16T11:27:22.426728+0000","last_unstale":"2023-10-16T11:27:23.486877+0000","last_undegraded":"2023-10-16T11:27:23.486877+0000","last_fullsized":"2023-10-16T11:27:23.486877+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:39.244304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.1a","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.532299+0000","last_change":"2023-10-16T11:27:23.457525+0000","last_active":"2023-10-16T11:27:25.532299+0000","last_peered":"2023-10-16T11:27:25.532299+0000","last_clean":"2023-10-16T11:27:25.532299+0000","last_became_active":"2023-10-16T11:27:23.456872+0000","last_became_peered":"2023-10-16T11:27:23.456872+0000","last_unstale":"2023-10-16T11:27:25.532299+0000","last_undegraded":"2023-10-16T11:27:25.532299+0000","last_fullsized":"2023-10-16T11:27:25.532299+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:00.798477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1b","version":"269'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.486399+0000","last_change":"2023-10-16T11:27:22.440153+0000","last_active":"2023-10-16T11:27:23.486399+0000","last_peered":"2023-10-16T11:27:23.486399+0000","last_clean":"2023-10-16T11:27:23.486399+0000","last_became_active":"2023-10-16T11:27:22.439862+0000","last_became_peered":"2023-10-16T11:27:22.439862+0000","last_unstale":"2023-10-16T11:27:23.486399+0000","last_undegraded":"2023-10-16T11:27:23.486399+0000","last_fullsized":"2023-10-16T11:27:23.486399+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:26:25.704295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.1b","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.442971+0000","last_change":"2023-10-16T11:27:23.469778+0000","last_active":"2023-10-16T11:27:24.442971+0000","last_peered":"2023-10-16T11:27:24.442971+0000","last_clean":"2023-10-16T11:27:24.442971+0000","last_became_active":"2023-10-16T11:27:23.469529+0000","last_became_peered":"2023-10-16T11:27:23.469529+0000","last_unstale":"2023-10-16T11:27:24.442971+0000","last_undegraded":"2023-10-16T11:27:24.442971+0000","last_fullsized":"2023-10-16T11:27:24.442971+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:06:02.521146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1a","version":"269'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.484787+0000","last_change":"2023-10-16T11:27:22.435228+0000","last_active":"2023-10-16T11:27:23.484787+0000","last_peered":"2023-10-16T11:27:23.484787+0000","last_clean":"2023-10-16T11:27:23.484787+0000","last_became_active":"2023-10-16T11:27:22.434922+0000","last_became_peered":"2023-10-16T11:27:22.434922+0000","last_unstale":"2023-10-16T11:27:23.484787+0000","last_undegraded":"2023-10-16T11:27:23.484787+0000","last_fullsized":"2023-10-16T11:27:23.484787+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:15.188934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.18","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.744508+0000","last_change":"2023-10-16T11:27:23.485335+0000","last_active":"2023-10-16T11:27:25.744508+0000","last_peered":"2023-10-16T11:27:25.744508+0000","last_clean":"2023-10-16T11:27:25.744508+0000","last_became_active":"2023-10-16T11:27:23.483862+0000","last_became_peered":"2023-10-16T11:27:23.483862+0000","last_unstale":"2023-10-16T11:27:25.744508+0000","last_undegraded":"2023-10-16T11:27:25.744508+0000","last_fullsized":"2023-10-16T11:27:25.744508+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:23:16.110019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.19","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.450091+0000","last_change":"2023-10-16T11:27:22.442373+0000","last_active":"2023-10-16T11:27:23.450091+0000","last_peered":"2023-10-16T11:27:23.450091+0000","last_clean":"2023-10-16T11:27:23.450091+0000","last_became_active":"2023-10-16T11:27:22.441763+0000","last_became_peered":"2023-10-16T11:27:22.441763+0000","last_unstale":"2023-10-16T11:27:23.450091+0000","last_undegraded":"2023-10-16T11:27:23.450091+0000","last_fullsized":"2023-10-16T11:27:23.450091+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:37:05.780888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.19","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.741955+0000","last_change":"2023-10-16T11:27:23.474956+0000","last_active":"2023-10-16T11:27:25.741955+0000","last_peered":"2023-10-16T11:27:25.741955+0000","last_clean":"2023-10-16T11:27:25.741955+0000","last_became_active":"2023-10-16T11:27:23.474685+0000","last_became_peered":"2023-10-16T11:27:23.474685+0000","last_unstale":"2023-10-16T11:27:25.741955+0000","last_undegraded":"2023-10-16T11:27:25.741955+0000","last_fullsized":"2023-10-16T11:27:25.741955+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:43.093225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.18","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.462560+0000","last_change":"2023-10-16T11:27:22.432937+0000","last_active":"2023-10-16T11:27:23.462560+0000","last_peered":"2023-10-16T11:27:23.462560+0000","last_clean":"2023-10-16T11:27:23.462560+0000","last_became_active":"2023-10-16T11:27:22.432719+0000","last_became_peered":"2023-10-16T11:27:22.432719+0000","last_unstale":"2023-10-16T11:27:23.462560+0000","last_undegraded":"2023-10-16T11:27:23.462560+0000","last_fullsized":"2023-10-16T11:27:23.462560+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:02.427534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.16","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.742245+0000","last_change":"2023-10-16T11:27:23.473098+0000","last_active":"2023-10-16T11:27:25.742245+0000","last_peered":"2023-10-16T11:27:25.742245+0000","last_clean":"2023-10-16T11:27:25.742245+0000","last_became_active":"2023-10-16T11:27:23.472855+0000","last_became_peered":"2023-10-16T11:27:23.472855+0000","last_unstale":"2023-10-16T11:27:25.742245+0000","last_undegraded":"2023-10-16T11:27:25.742245+0000","last_fullsized":"2023-10-16T11:27:25.742245+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:09.556639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.17","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.466216+0000","last_change":"2023-10-16T11:27:22.434526+0000","last_active":"2023-10-16T11:27:23.466216+0000","last_peered":"2023-10-16T11:27:23.466216+0000","last_clean":"2023-10-16T11:27:23.466216+0000","last_became_active":"2023-10-16T11:27:22.433489+0000","last_became_peered":"2023-10-16T11:27:22.433489+0000","last_unstale":"2023-10-16T11:27:23.466216+0000","last_undegraded":"2023-10-16T11:27:23.466216+0000","last_fullsized":"2023-10-16T11:27:23.466216+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:13.916657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.17","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.443755+0000","last_change":"2023-10-16T11:27:23.474492+0000","last_active":"2023-10-16T11:27:24.443755+0000","last_peered":"2023-10-16T11:27:24.443755+0000","last_clean":"2023-10-16T11:27:24.443755+0000","last_became_active":"2023-10-16T11:27:23.474228+0000","last_became_peered":"2023-10-16T11:27:23.474228+0000","last_unstale":"2023-10-16T11:27:24.443755+0000","last_undegraded":"2023-10-16T11:27:24.443755+0000","last_fullsized":"2023-10-16T11:27:24.443755+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:34.898375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.16","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.465874+0000","last_change":"2023-10-16T11:27:22.435414+0000","last_active":"2023-10-16T11:27:23.465874+0000","last_peered":"2023-10-16T11:27:23.465874+0000","last_clean":"2023-10-16T11:27:23.465874+0000","last_became_active":"2023-10-16T11:27:22.434556+0000","last_became_peered":"2023-10-16T11:27:22.434556+0000","last_unstale":"2023-10-16T11:27:23.465874+0000","last_undegraded":"2023-10-16T11:27:23.465874+0000","last_fullsized":"2023-10-16T11:27:23.465874+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:02.427534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.14","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.440813+0000","last_change":"2023-10-16T11:27:23.466931+0000","last_active":"2023-10-16T11:27:24.440813+0000","last_peered":"2023-10-16T11:27:24.440813+0000","last_clean":"2023-10-16T11:27:24.440813+0000","last_became_active":"2023-10-16T11:27:23.466635+0000","last_became_peered":"2023-10-16T11:27:23.466635+0000","last_unstale":"2023-10-16T11:27:24.440813+0000","last_undegraded":"2023-10-16T11:27:24.440813+0000","last_fullsized":"2023-10-16T11:27:24.440813+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:57:41.546314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.15","version":"269'3","reported_seq":15,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.489036+0000","last_change":"2023-10-16T11:27:22.428021+0000","last_active":"2023-10-16T11:27:23.489036+0000","last_peered":"2023-10-16T11:27:23.489036+0000","last_clean":"2023-10-16T11:27:23.489036+0000","last_became_active":"2023-10-16T11:27:22.427744+0000","last_became_peered":"2023-10-16T11:27:22.427744+0000","last_unstale":"2023-10-16T11:27:23.489036+0000","last_undegraded":"2023-10-16T11:27:23.489036+0000","last_fullsized":"2023-10-16T11:27:23.489036+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:29.185681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.15","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.742198+0000","last_change":"2023-10-16T11:27:23.472980+0000","last_active":"2023-10-16T11:27:25.742198+0000","last_peered":"2023-10-16T11:27:25.742198+0000","last_clean":"2023-10-16T11:27:25.742198+0000","last_became_active":"2023-10-16T11:27:23.471883+0000","last_became_peered":"2023-10-16T11:27:23.471883+0000","last_unstale":"2023-10-16T11:27:25.742198+0000","last_undegraded":"2023-10-16T11:27:25.742198+0000","last_fullsized":"2023-10-16T11:27:25.742198+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:16.206103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.14","version":"269'6","reported_seq":18,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.491027+0000","last_change":"2023-10-16T11:27:22.440362+0000","last_active":"2023-10-16T11:27:23.491027+0000","last_peered":"2023-10-16T11:27:23.491027+0000","last_clean":"2023-10-16T11:27:23.491027+0000","last_became_active":"2023-10-16T11:27:22.439632+0000","last_became_peered":"2023-10-16T11:27:22.439632+0000","last_unstale":"2023-10-16T11:27:23.491027+0000","last_undegraded":"2023-10-16T11:27:23.491027+0000","last_fullsized":"2023-10-16T11:27:23.491027+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:39.244304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.12","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.443685+0000","last_change":"2023-10-16T11:27:23.470789+0000","last_active":"2023-10-16T11:27:24.443685+0000","last_peered":"2023-10-16T11:27:24.443685+0000","last_clean":"2023-10-16T11:27:24.443685+0000","last_became_active":"2023-10-16T11:27:23.470511+0000","last_became_peered":"2023-10-16T11:27:23.470511+0000","last_unstale":"2023-10-16T11:27:24.443685+0000","last_undegraded":"2023-10-16T11:27:24.443685+0000","last_fullsized":"2023-10-16T11:27:24.443685+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:40.261473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.13","version":"269'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.488139+0000","last_change":"2023-10-16T11:27:22.420724+0000","last_active":"2023-10-16T11:27:23.488139+0000","last_peered":"2023-10-16T11:27:23.488139+0000","last_clean":"2023-10-16T11:27:23.488139+0000","last_became_active":"2023-10-16T11:27:22.416656+0000","last_became_peered":"2023-10-16T11:27:22.416656+0000","last_unstale":"2023-10-16T11:27:23.488139+0000","last_undegraded":"2023-10-16T11:27:23.488139+0000","last_fullsized":"2023-10-16T11:27:23.488139+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:23.396162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.13","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.745838+0000","last_change":"2023-10-16T11:27:23.468059+0000","last_active":"2023-10-16T11:27:25.745838+0000","last_peered":"2023-10-16T11:27:25.745838+0000","last_clean":"2023-10-16T11:27:25.745838+0000","last_became_active":"2023-10-16T11:27:23.467796+0000","last_became_peered":"2023-10-16T11:27:23.467796+0000","last_unstale":"2023-10-16T11:27:25.745838+0000","last_undegraded":"2023-10-16T11:27:25.745838+0000","last_fullsized":"2023-10-16T11:27:25.745838+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:24:04.957139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.12","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.444463+0000","last_change":"2023-10-16T11:27:22.418954+0000","last_active":"2023-10-16T11:27:23.444463+0000","last_peered":"2023-10-16T11:27:23.444463+0000","last_clean":"2023-10-16T11:27:23.444463+0000","last_became_active":"2023-10-16T11:27:22.418291+0000","last_became_peered":"2023-10-16T11:27:22.418291+0000","last_unstale":"2023-10-16T11:27:23.444463+0000","last_undegraded":"2023-10-16T11:27:23.444463+0000","last_fullsized":"2023-10-16T11:27:23.444463+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:05:07.678705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.10","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.470564+0000","last_change":"2023-10-16T11:27:23.467760+0000","last_active":"2023-10-16T11:27:24.470564+0000","last_peered":"2023-10-16T11:27:24.470564+0000","last_clean":"2023-10-16T11:27:24.470564+0000","last_became_active":"2023-10-16T11:27:23.466581+0000","last_became_peered":"2023-10-16T11:27:23.466581+0000","last_unstale":"2023-10-16T11:27:24.470564+0000","last_undegraded":"2023-10-16T11:27:24.470564+0000","last_fullsized":"2023-10-16T11:27:24.470564+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:24.699653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.11","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.467774+0000","last_change":"2023-10-16T11:27:22.432941+0000","last_active":"2023-10-16T11:27:23.467774+0000","last_peered":"2023-10-16T11:27:23.467774+0000","last_clean":"2023-10-16T11:27:23.467774+0000","last_became_active":"2023-10-16T11:27:22.432633+0000","last_became_peered":"2023-10-16T11:27:22.432633+0000","last_unstale":"2023-10-16T11:27:23.467774+0000","last_undegraded":"2023-10-16T11:27:23.467774+0000","last_fullsized":"2023-10-16T11:27:23.467774+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:33.881206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.11","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.744898+0000","last_change":"2023-10-16T11:27:23.470371+0000","last_active":"2023-10-16T11:27:25.744898+0000","last_peered":"2023-10-16T11:27:25.744898+0000","last_clean":"2023-10-16T11:27:25.744898+0000","last_became_active":"2023-10-16T11:27:23.469794+0000","last_became_peered":"2023-10-16T11:27:23.469794+0000","last_unstale":"2023-10-16T11:27:25.744898+0000","last_undegraded":"2023-10-16T11:27:25.744898+0000","last_fullsized":"2023-10-16T11:27:25.744898+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:54:26.226556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.10","version":"269'8","reported_seq":20,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.491124+0000","last_change":"2023-10-16T11:27:22.435285+0000","last_active":"2023-10-16T11:27:23.491124+0000","last_peered":"2023-10-16T11:27:23.491124+0000","last_clean":"2023-10-16T11:27:23.491124+0000","last_became_active":"2023-10-16T11:27:22.434837+0000","last_became_peered":"2023-10-16T11:27:22.434837+0000","last_unstale":"2023-10-16T11:27:23.491124+0000","last_undegraded":"2023-10-16T11:27:23.491124+0000","last_fullsized":"2023-10-16T11:27:23.491124+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:57:54.775978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.e","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.441263+0000","last_change":"2023-10-16T11:27:23.478549+0000","last_active":"2023-10-16T11:27:24.441263+0000","last_peered":"2023-10-16T11:27:24.441263+0000","last_clean":"2023-10-16T11:27:24.441263+0000","last_became_active":"2023-10-16T11:27:23.478282+0000","last_became_peered":"2023-10-16T11:27:23.478282+0000","last_unstale":"2023-10-16T11:27:24.441263+0000","last_undegraded":"2023-10-16T11:27:24.441263+0000","last_fullsized":"2023-10-16T11:27:24.441263+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:44.126144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.f","version":"269'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.487257+0000","last_change":"2023-10-16T11:27:22.426688+0000","last_active":"2023-10-16T11:27:23.487257+0000","last_peered":"2023-10-16T11:27:23.487257+0000","last_clean":"2023-10-16T11:27:23.487257+0000","last_became_active":"2023-10-16T11:27:22.426400+0000","last_became_peered":"2023-10-16T11:27:22.426400+0000","last_unstale":"2023-10-16T11:27:23.487257+0000","last_undegraded":"2023-10-16T11:27:23.487257+0000","last_fullsized":"2023-10-16T11:27:23.487257+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:08:03.403778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.f","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.472365+0000","last_change":"2023-10-16T11:27:23.481822+0000","last_active":"2023-10-16T11:27:24.472365+0000","last_peered":"2023-10-16T11:27:24.472365+0000","last_clean":"2023-10-16T11:27:24.472365+0000","last_became_active":"2023-10-16T11:27:23.481451+0000","last_became_peered":"2023-10-16T11:27:23.481451+0000","last_unstale":"2023-10-16T11:27:24.472365+0000","last_undegraded":"2023-10-16T11:27:24.472365+0000","last_fullsized":"2023-10-16T11:27:24.472365+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:02:34.998913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.e","version":"269'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.487751+0000","last_change":"2023-10-16T11:27:22.435582+0000","last_active":"2023-10-16T11:27:23.487751+0000","last_peered":"2023-10-16T11:27:23.487751+0000","last_clean":"2023-10-16T11:27:23.487751+0000","last_became_active":"2023-10-16T11:27:22.435098+0000","last_became_peered":"2023-10-16T11:27:22.435098+0000","last_unstale":"2023-10-16T11:27:23.487751+0000","last_undegraded":"2023-10-16T11:27:23.487751+0000","last_fullsized":"2023-10-16T11:27:23.487751+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:05:07.678705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.c","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.742166+0000","last_change":"2023-10-16T11:27:23.472000+0000","last_active":"2023-10-16T11:27:25.742166+0000","last_peered":"2023-10-16T11:27:25.742166+0000","last_clean":"2023-10-16T11:27:25.742166+0000","last_became_active":"2023-10-16T11:27:23.469423+0000","last_became_peered":"2023-10-16T11:27:23.469423+0000","last_unstale":"2023-10-16T11:27:25.742166+0000","last_undegraded":"2023-10-16T11:27:25.742166+0000","last_fullsized":"2023-10-16T11:27:25.742166+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:30:41.982564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.d","version":"269'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.487807+0000","last_change":"2023-10-16T11:27:22.434669+0000","last_active":"2023-10-16T11:27:23.487807+0000","last_peered":"2023-10-16T11:27:23.487807+0000","last_clean":"2023-10-16T11:27:23.487807+0000","last_became_active":"2023-10-16T11:27:22.433720+0000","last_became_peered":"2023-10-16T11:27:22.433720+0000","last_unstale":"2023-10-16T11:27:23.487807+0000","last_undegraded":"2023-10-16T11:27:23.487807+0000","last_fullsized":"2023-10-16T11:27:23.487807+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:57.154827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.d","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.444077+0000","last_change":"2023-10-16T11:27:23.469154+0000","last_active":"2023-10-16T11:27:24.444077+0000","last_peered":"2023-10-16T11:27:24.444077+0000","last_clean":"2023-10-16T11:27:24.444077+0000","last_became_active":"2023-10-16T11:27:23.468885+0000","last_became_peered":"2023-10-16T11:27:23.468885+0000","last_unstale":"2023-10-16T11:27:24.444077+0000","last_undegraded":"2023-10-16T11:27:24.444077+0000","last_fullsized":"2023-10-16T11:27:24.444077+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:03:33.273585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.c","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.465428+0000","last_change":"2023-10-16T11:27:22.438800+0000","last_active":"2023-10-16T11:27:23.465428+0000","last_peered":"2023-10-16T11:27:23.465428+0000","last_clean":"2023-10-16T11:27:23.465428+0000","last_became_active":"2023-10-16T11:27:22.438513+0000","last_became_peered":"2023-10-16T11:27:22.438513+0000","last_unstale":"2023-10-16T11:27:23.465428+0000","last_undegraded":"2023-10-16T11:27:23.465428+0000","last_fullsized":"2023-10-16T11:27:23.465428+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:32:51.312608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.a","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.471207+0000","last_change":"2023-10-16T11:27:23.486689+0000","last_active":"2023-10-16T11:27:24.471207+0000","last_peered":"2023-10-16T11:27:24.471207+0000","last_clean":"2023-10-16T11:27:24.471207+0000","last_became_active":"2023-10-16T11:27:23.484418+0000","last_became_peered":"2023-10-16T11:27:23.484418+0000","last_unstale":"2023-10-16T11:27:24.471207+0000","last_undegraded":"2023-10-16T11:27:24.471207+0000","last_fullsized":"2023-10-16T11:27:24.471207+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:33:06.072611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.b","version":"269'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.489077+0000","last_change":"2023-10-16T11:27:22.427350+0000","last_active":"2023-10-16T11:27:23.489077+0000","last_peered":"2023-10-16T11:27:23.489077+0000","last_clean":"2023-10-16T11:27:23.489077+0000","last_became_active":"2023-10-16T11:27:22.427089+0000","last_became_peered":"2023-10-16T11:27:22.427089+0000","last_unstale":"2023-10-16T11:27:23.489077+0000","last_undegraded":"2023-10-16T11:27:23.489077+0000","last_fullsized":"2023-10-16T11:27:23.489077+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:33.881206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.b","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.443349+0000","last_change":"2023-10-16T11:27:23.467686+0000","last_active":"2023-10-16T11:27:24.443349+0000","last_peered":"2023-10-16T11:27:24.443349+0000","last_clean":"2023-10-16T11:27:24.443349+0000","last_became_active":"2023-10-16T11:27:23.467408+0000","last_became_peered":"2023-10-16T11:27:23.467408+0000","last_unstale":"2023-10-16T11:27:24.443349+0000","last_undegraded":"2023-10-16T11:27:24.443349+0000","last_fullsized":"2023-10-16T11:27:24.443349+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:00.149551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.a","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.450086+0000","last_change":"2023-10-16T11:27:22.427630+0000","last_active":"2023-10-16T11:27:23.450086+0000","last_peered":"2023-10-16T11:27:23.450086+0000","last_clean":"2023-10-16T11:27:23.450086+0000","last_became_active":"2023-10-16T11:27:22.426731+0000","last_became_peered":"2023-10-16T11:27:22.426731+0000","last_unstale":"2023-10-16T11:27:23.450086+0000","last_undegraded":"2023-10-16T11:27:23.450086+0000","last_fullsized":"2023-10-16T11:27:23.450086+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:16:33.200320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.8","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.531722+0000","last_change":"2023-10-16T11:27:23.460647+0000","last_active":"2023-10-16T11:27:25.531722+0000","last_peered":"2023-10-16T11:27:25.531722+0000","last_clean":"2023-10-16T11:27:25.531722+0000","last_became_active":"2023-10-16T11:27:23.460185+0000","last_became_peered":"2023-10-16T11:27:23.460185+0000","last_unstale":"2023-10-16T11:27:25.531722+0000","last_undegraded":"2023-10-16T11:27:25.531722+0000","last_fullsized":"2023-10-16T11:27:25.531722+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:30.306508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.9","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.462457+0000","last_change":"2023-10-16T11:27:22.439130+0000","last_active":"2023-10-16T11:27:23.462457+0000","last_peered":"2023-10-16T11:27:23.462457+0000","last_clean":"2023-10-16T11:27:23.462457+0000","last_became_active":"2023-10-16T11:27:22.438098+0000","last_became_peered":"2023-10-16T11:27:22.438098+0000","last_unstale":"2023-10-16T11:27:23.462457+0000","last_undegraded":"2023-10-16T11:27:23.462457+0000","last_fullsized":"2023-10-16T11:27:23.462457+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:36:07.856477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.9","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.531769+0000","last_change":"2023-10-16T11:27:23.480112+0000","last_active":"2023-10-16T11:27:25.531769+0000","last_peered":"2023-10-16T11:27:25.531769+0000","last_clean":"2023-10-16T11:27:25.531769+0000","last_became_active":"2023-10-16T11:27:23.479844+0000","last_became_peered":"2023-10-16T11:27:23.479844+0000","last_unstale":"2023-10-16T11:27:25.531769+0000","last_undegraded":"2023-10-16T11:27:25.531769+0000","last_fullsized":"2023-10-16T11:27:25.531769+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:00:54.115844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.8","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.451508+0000","last_change":"2023-10-16T11:27:22.424674+0000","last_active":"2023-10-16T11:27:23.451508+0000","last_peered":"2023-10-16T11:27:23.451508+0000","last_clean":"2023-10-16T11:27:23.451508+0000","last_became_active":"2023-10-16T11:27:22.424065+0000","last_became_peered":"2023-10-16T11:27:22.424065+0000","last_unstale":"2023-10-16T11:27:23.451508+0000","last_undegraded":"2023-10-16T11:27:23.451508+0000","last_fullsized":"2023-10-16T11:27:23.451508+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:45.782726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.6","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.443606+0000","last_change":"2023-10-16T11:27:23.455323+0000","last_active":"2023-10-16T11:27:24.443606+0000","last_peered":"2023-10-16T11:27:24.443606+0000","last_clean":"2023-10-16T11:27:24.443606+0000","last_became_active":"2023-10-16T11:27:23.454550+0000","last_became_peered":"2023-10-16T11:27:23.454550+0000","last_unstale":"2023-10-16T11:27:24.443606+0000","last_undegraded":"2023-10-16T11:27:24.443606+0000","last_fullsized":"2023-10-16T11:27:24.443606+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:09.933592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.7","version":"269'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.488039+0000","last_change":"2023-10-16T11:27:22.427506+0000","last_active":"2023-10-16T11:27:23.488039+0000","last_peered":"2023-10-16T11:27:23.488039+0000","last_clean":"2023-10-16T11:27:23.488039+0000","last_became_active":"2023-10-16T11:27:22.426863+0000","last_became_peered":"2023-10-16T11:27:22.426863+0000","last_unstale":"2023-10-16T11:27:23.488039+0000","last_undegraded":"2023-10-16T11:27:23.488039+0000","last_fullsized":"2023-10-16T11:27:23.488039+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:57.154827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.7","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.747515+0000","last_change":"2023-10-16T11:27:23.480903+0000","last_active":"2023-10-16T11:27:25.747515+0000","last_peered":"2023-10-16T11:27:25.747515+0000","last_clean":"2023-10-16T11:27:25.747515+0000","last_became_active":"2023-10-16T11:27:23.480347+0000","last_became_peered":"2023-10-16T11:27:23.480347+0000","last_unstale":"2023-10-16T11:27:25.747515+0000","last_undegraded":"2023-10-16T11:27:25.747515+0000","last_fullsized":"2023-10-16T11:27:25.747515+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:28:14.944747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.6","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.450228+0000","last_change":"2023-10-16T11:27:22.432595+0000","last_active":"2023-10-16T11:27:23.450228+0000","last_peered":"2023-10-16T11:27:23.450228+0000","last_clean":"2023-10-16T11:27:23.450228+0000","last_became_active":"2023-10-16T11:27:22.432255+0000","last_became_peered":"2023-10-16T11:27:22.432255+0000","last_unstale":"2023-10-16T11:27:23.450228+0000","last_undegraded":"2023-10-16T11:27:23.450228+0000","last_fullsized":"2023-10-16T11:27:23.450228+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:15.847704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.4","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.443294+0000","last_change":"2023-10-16T11:27:23.476219+0000","last_active":"2023-10-16T11:27:24.443294+0000","last_peered":"2023-10-16T11:27:24.443294+0000","last_clean":"2023-10-16T11:27:24.443294+0000","last_became_active":"2023-10-16T11:27:23.475943+0000","last_became_peered":"2023-10-16T11:27:23.475943+0000","last_unstale":"2023-10-16T11:27:24.443294+0000","last_undegraded":"2023-10-16T11:27:24.443294+0000","last_fullsized":"2023-10-16T11:27:24.443294+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:58.171995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.5","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.445325+0000","last_change":"2023-10-16T11:27:22.434467+0000","last_active":"2023-10-16T11:27:23.445325+0000","last_peered":"2023-10-16T11:27:23.445325+0000","last_clean":"2023-10-16T11:27:23.445325+0000","last_became_active":"2023-10-16T11:27:22.433992+0000","last_became_peered":"2023-10-16T11:27:22.433992+0000","last_unstale":"2023-10-16T11:27:23.445325+0000","last_undegraded":"2023-10-16T11:27:23.445325+0000","last_fullsized":"2023-10-16T11:27:23.445325+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:32:51.312608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.5","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.745544+0000","last_change":"2023-10-16T11:27:23.470590+0000","last_active":"2023-10-16T11:27:25.745544+0000","last_peered":"2023-10-16T11:27:25.745544+0000","last_clean":"2023-10-16T11:27:25.745544+0000","last_became_active":"2023-10-16T11:27:23.466649+0000","last_became_peered":"2023-10-16T11:27:23.466649+0000","last_unstale":"2023-10-16T11:27:25.745544+0000","last_undegraded":"2023-10-16T11:27:25.745544+0000","last_fullsized":"2023-10-16T11:27:25.745544+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:23:07.907946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.4","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.456950+0000","last_change":"2023-10-16T11:27:22.439371+0000","last_active":"2023-10-16T11:27:23.456950+0000","last_peered":"2023-10-16T11:27:23.456950+0000","last_clean":"2023-10-16T11:27:23.456950+0000","last_became_active":"2023-10-16T11:27:22.439092+0000","last_became_peered":"2023-10-16T11:27:22.439092+0000","last_unstale":"2023-10-16T11:27:23.456950+0000","last_undegraded":"2023-10-16T11:27:23.456950+0000","last_fullsized":"2023-10-16T11:27:23.456950+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:54:50.109436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.2","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.741901+0000","last_change":"2023-10-16T11:27:23.473928+0000","last_active":"2023-10-16T11:27:25.741901+0000","last_peered":"2023-10-16T11:27:25.741901+0000","last_clean":"2023-10-16T11:27:25.741901+0000","last_became_active":"2023-10-16T11:27:23.473681+0000","last_became_peered":"2023-10-16T11:27:23.473681+0000","last_unstale":"2023-10-16T11:27:25.741901+0000","last_undegraded":"2023-10-16T11:27:25.741901+0000","last_fullsized":"2023-10-16T11:27:25.741901+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:34.898375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.3","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.454385+0000","last_change":"2023-10-16T11:27:22.441927+0000","last_active":"2023-10-16T11:27:23.454385+0000","last_peered":"2023-10-16T11:27:23.454385+0000","last_clean":"2023-10-16T11:27:23.454385+0000","last_became_active":"2023-10-16T11:27:22.441607+0000","last_became_peered":"2023-10-16T11:27:22.441607+0000","last_unstale":"2023-10-16T11:27:23.454385+0000","last_undegraded":"2023-10-16T11:27:23.454385+0000","last_fullsized":"2023-10-16T11:27:23.454385+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:41.763300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.3","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.440771+0000","last_change":"2023-10-16T11:27:23.471757+0000","last_active":"2023-10-16T11:27:24.440771+0000","last_peered":"2023-10-16T11:27:24.440771+0000","last_clean":"2023-10-16T11:27:24.440771+0000","last_became_active":"2023-10-16T11:27:23.469662+0000","last_became_peered":"2023-10-16T11:27:23.469662+0000","last_unstale":"2023-10-16T11:27:24.440771+0000","last_undegraded":"2023-10-16T11:27:24.440771+0000","last_fullsized":"2023-10-16T11:27:24.440771+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:50.441137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.2","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.444768+0000","last_change":"2023-10-16T11:27:22.434794+0000","last_active":"2023-10-16T11:27:23.444768+0000","last_peered":"2023-10-16T11:27:23.444768+0000","last_clean":"2023-10-16T11:27:23.444768+0000","last_became_active":"2023-10-16T11:27:22.434535+0000","last_became_peered":"2023-10-16T11:27:22.434535+0000","last_unstale":"2023-10-16T11:27:23.444768+0000","last_undegraded":"2023-10-16T11:27:23.444768+0000","last_fullsized":"2023-10-16T11:27:23.444768+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:13.916657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.0","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.531935+0000","last_change":"2023-10-16T11:27:23.470420+0000","last_active":"2023-10-16T11:27:25.531935+0000","last_peered":"2023-10-16T11:27:25.531935+0000","last_clean":"2023-10-16T11:27:25.531935+0000","last_became_active":"2023-10-16T11:27:23.470185+0000","last_became_peered":"2023-10-16T11:27:23.470185+0000","last_unstale":"2023-10-16T11:27:25.531935+0000","last_undegraded":"2023-10-16T11:27:25.531935+0000","last_fullsized":"2023-10-16T11:27:25.531935+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:03:13.636592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.464850+0000","last_change":"2023-10-16T11:27:22.436919+0000","last_active":"2023-10-16T11:27:23.464850+0000","last_peered":"2023-10-16T11:27:23.464850+0000","last_clean":"2023-10-16T11:27:23.464850+0000","last_became_active":"2023-10-16T11:27:22.434273+0000","last_became_peered":"2023-10-16T11:27:22.434273+0000","last_unstale":"2023-10-16T11:27:23.464850+0000","last_undegraded":"2023-10-16T11:27:23.464850+0000","last_fullsized":"2023-10-16T11:27:23.464850+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:36:07.856477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.1","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:25.745268+0000","last_change":"2023-10-16T11:27:23.470169+0000","last_active":"2023-10-16T11:27:25.745268+0000","last_peered":"2023-10-16T11:27:25.745268+0000","last_clean":"2023-10-16T11:27:25.745268+0000","last_became_active":"2023-10-16T11:27:23.469310+0000","last_became_peered":"2023-10-16T11:27:23.469310+0000","last_unstale":"2023-10-16T11:27:25.745268+0000","last_undegraded":"2023-10-16T11:27:25.745268+0000","last_fullsized":"2023-10-16T11:27:25.745268+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:19:56.522209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.0","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:27:23.442094+0000","last_change":"2023-10-16T11:27:22.434884+0000","last_active":"2023-10-16T11:27:23.442094+0000","last_peered":"2023-10-16T11:27:23.442094+0000","last_clean":"2023-10-16T11:27:23.442094+0000","last_became_active":"2023-10-16T11:27:22.434345+0000","last_became_peered":"2023-10-16T11:27:22.434345+0000","last_unstale":"2023-10-16T11:27:23.442094+0000","last_undegraded":"2023-10-16T11:27:23.442094+0000","last_fullsized":"2023-10-16T11:27:23.442094+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:21.377565+0000","last_clean_scrub_stamp":"2023-10-16T11:27:21.377565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:16:49.172852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.24","version":"0'0","reported_seq":11,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.440644+0000","last_change":"2023-10-16T11:27:23.463422+0000","last_active":"2023-10-16T11:27:24.440644+0000","last_peered":"2023-10-16T11:27:24.440644+0000","last_clean":"2023-10-16T11:27:24.440644+0000","last_became_active":"2023-10-16T11:27:23.463089+0000","last_became_peered":"2023-10-16T11:27:23.463089+0000","last_unstale":"2023-10-16T11:27:24.440644+0000","last_undegraded":"2023-10-16T11:27:24.440644+0000","last_fullsized":"2023-10-16T11:27:24.440644+0000","mapping_epoch":268,"log_start":"0'0","ondisk_log_start":"0'0","created":268,"last_epoch_clean":269,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:27:22.394734+0000","last_clean_scrub_stamp":"2023-10-16T11:27:22.394734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:23:16.110019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"259'4825","reported_seq":5918,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:27:24.470715+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:27:24.470715+0000","last_peered":"2023-10-16T11:27:24.470715+0000","last_clean":"2023-10-16T11:27:24.470715+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:27:24.470715+0000","last_undegraded":"2023-10-16T11:27:24.470715+0000","last_fullsized":"2023-10-16T11:27:24.470715+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":4825,"log_dups_size":0,"ondisk_log_size":4825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2687,"num_read_kb":5529,"num_write":683,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":37,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":36,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2687,"num_read_kb":5529,"num_write":683,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4825,"ondisk_log_size":4825,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576018,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79512,"kb_used_data":3916,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94292328,"statfs":{"total":96636764160,"available":96555343872,"internally_reserved":0,"allocated":4009984,"data_stored":2263146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":77396427},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1140000000000001,"15min":1.1859999999999999},"min":{"1min":0.64500000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.673,"5min":1.7250000000000001,"15min":2.3399999999999999},"last":1.655},{"interface":"front","average":{"1min":1.109,"5min":1.1259999999999999,"15min":1.1930000000000001},"min":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.633,"5min":1.6699999999999999,"15min":1.71},"last":0.93400000000000005}]},{"osd":1,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2010000000000001,"15min":1.2290000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1719999999999999,"15min":1.2470000000000001},"min":{"1min":0.75600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.6100000000000001,"5min":1.6799999999999999,"15min":1.8600000000000001},"last":1.1100000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2749999999999999,"15min":1.286},"min":{"1min":0.71799999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8640000000000001,"5min":1.9039999999999999,"15min":2.4460000000000002},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.3140000000000001,"15min":1.349},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.5389999999999999,"5min":3.9249999999999998,"15min":3.9249999999999998},"last":1.3029999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.339,"15min":1.371},"min":{"1min":0.85999999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8169999999999999,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.389,"15min":1.4339999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.016,"5min":2.016,"15min":2.577},"last":1.355}]},{"osd":4,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.419,"15min":1.4039999999999999},"min":{"1min":0.64800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.113,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.379,"15min":1.3779999999999999},"min":{"1min":0.65600000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.756,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.472}]},{"osd":5,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.5669999999999999,"15min":1.5780000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":3.3519999999999999},"last":1.7909999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.534,"15min":1.498},"min":{"1min":0.62,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.0880000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.591}]},{"osd":6,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.488,"15min":1.4970000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.907,"5min":2.2469999999999999,"15min":2.7250000000000001},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.633,"5min":1.6639999999999999,"15min":1.6859999999999999},"min":{"1min":0.749,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.323,"5min":2.323,"15min":3.1440000000000001},"last":1.73}]}]},{"osd":6,"up_from":13,"seq":55834576018,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52580,"kb_used_data":3348,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319260,"statfs":{"total":96636764160,"available":96582922240,"internally_reserved":0,"allocated":3428352,"data_stored":1674390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.1779999999999999,"15min":1.206},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.5700000000000001,"5min":6.298,"15min":6.298},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.3129999999999999,"15min":1.278},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6140000000000001,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":1.1000000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.69799999999999995,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.24,"5min":1.395,"15min":1.365},"min":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.956,"5min":12.468,"15min":12.468},"last":1.385}]},{"osd":2,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.286,"15min":1.262},"min":{"1min":0.82899999999999996,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4870000000000001,"5min":11.586,"15min":11.586},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.3799999999999999,"15min":1.3720000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.77500000000000002},"max":{"1min":1.8600000000000001,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.3540000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.4199999999999999,"15min":1.3440000000000001},"min":{"1min":0.86299999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.925,"5min":12.801,"15min":12.801},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.177,"5min":1.2809999999999999,"15min":1.2649999999999999},"min":{"1min":0.88100000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5589999999999999,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":1.157}]},{"osd":4,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.417,"15min":1.3320000000000001},"min":{"1min":0.76300000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.806,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.046},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.6359999999999999,"15min":1.548},"min":{"1min":1.109,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.3069999999999999,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.216}]},{"osd":5,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.5469999999999999,"15min":1.3999999999999999},"min":{"1min":0.86199999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.8979999999999999,"5min":18.247,"15min":18.247},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.617,"5min":1.625,"15min":1.5660000000000001},"min":{"1min":0.85899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.923,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.597}]},{"osd":7,"last update":"Mon Oct 16 11:26:44 2023","interfaces":[{"interface":"back","average":{"1min":1.6779999999999999,"5min":1.6819999999999999,"15min":1.5529999999999999},"min":{"1min":0.92500000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.4660000000000002,"5min":12.589,"15min":12.589},"last":1.528},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.7110000000000001,"15min":1.6020000000000001},"min":{"1min":1.1779999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.2050000000000001,"5min":12.725,"15min":12.725},"last":1.5600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834576018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52572,"kb_used_data":3340,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319268,"statfs":{"total":96636764160,"available":96582930432,"internally_reserved":0,"allocated":3420160,"data_stored":1668152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.1519999999999999,"15min":1.234},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.585,"5min":1.585,"15min":2.661},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.248,"15min":1.3169999999999999},"min":{"1min":0.75600000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.583,"5min":1.7909999999999999,"15min":1.7949999999999999},"last":0.96299999999999997}]},{"osd":1,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1279999999999999,"15min":1.175},"min":{"1min":0.82199999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.748,"5min":1.748,"15min":1.992},"last":0.878},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.1919999999999999,"15min":1.228},"min":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":2.9399999999999999},"last":0.81799999999999995}]},{"osd":2,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2470000000000001,"15min":1.2390000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.274},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2869999999999999,"15min":1.3009999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7829999999999999,"5min":1.8660000000000001,"15min":2.1640000000000001},"last":1.3799999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.29,"15min":1.3169999999999999},"min":{"1min":0.82199999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.121,"5min":2.121,"15min":3.012},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.429,"5min":1.387,"15min":1.411},"min":{"1min":1.0880000000000001,"5min":0.92700000000000005,"15min":0.89500000000000002},"max":{"1min":1.9299999999999999,"5min":2.0129999999999999,"15min":2.2250000000000001},"last":1.2}]},{"osd":4,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.3280000000000001,"15min":1.345},"min":{"1min":0.80300000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.8660000000000001},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.605,"5min":1.5429999999999999,"15min":1.5620000000000001},"min":{"1min":1.125,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.903},"last":1.544}]},{"osd":6,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.4059999999999999,"15min":1.417},"min":{"1min":0.95099999999999996,"5min":0.68100000000000005,"15min":0.51000000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0979999999999999},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.53,"5min":1.4379999999999999,"15min":1.4770000000000001},"min":{"1min":1.004,"5min":0.90700000000000003,"15min":0.89200000000000002},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":3.073},"last":1.302}]},{"osd":7,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.552,"15min":1.548},"min":{"1min":1.228,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":3.1160000000000001},"last":1.8069999999999999},{"interface":"front","average":{"1min":1.702,"5min":1.619,"15min":1.639},"min":{"1min":1.2809999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.161,"5min":2.1930000000000001,"15min":2.2930000000000001},"last":1.6200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834576019,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88604,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94283236,"statfs":{"total":96636764160,"available":96546033664,"internally_reserved":0,"allocated":3424256,"data_stored":1673430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":87292358},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.151,"15min":1.1739999999999999},"min":{"1min":0.73799999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.615,"5min":2.899,"15min":2.899},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.296,"5min":1.236,"15min":1.254},"min":{"1min":0.77500000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.784,"5min":3.016,"15min":3.016},"last":1.252}]},{"osd":1,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.28,"15min":1.298},"min":{"1min":0.83999999999999997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6799999999999999,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.111},{"interface":"front","average":{"1min":1.089,"5min":1.228,"15min":1.25},"min":{"1min":0.84499999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.4299999999999999,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.429}]},{"osd":2,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8300000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.157},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4370000000000001,"15min":1.4199999999999999},"min":{"1min":0.90500000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.748,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.591}]},{"osd":3,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.351,"15min":1.349},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.7669999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.452,"5min":1.4930000000000001,"15min":1.4610000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.9930000000000001,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.494}]},{"osd":5,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":1.542,"15min":1.4790000000000001},"min":{"1min":1.0549999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.915,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.375},{"interface":"front","average":{"1min":1.444,"5min":1.4650000000000001,"15min":1.4370000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.9410000000000001,"5min":3.7989999999999999,"15min":3.7989999999999999},"last":1.3080000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.6020000000000001,"15min":1.5349999999999999},"min":{"1min":0.85699999999999998,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":2.0779999999999998,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.643},{"interface":"front","average":{"1min":1.724,"5min":1.679,"15min":1.6459999999999999},"min":{"1min":1.268,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.944,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.694}]},{"osd":7,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":1.647,"15min":1.607},"min":{"1min":0.73699999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.1949999999999998,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.702,"5min":1.7190000000000001,"15min":1.6799999999999999},"min":{"1min":1.026,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.1419999999999999,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.7589999999999999}]}]},{"osd":3,"up_from":13,"seq":55834576018,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52572,"kb_used_data":3340,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319268,"statfs":{"total":96636764160,"available":96582930432,"internally_reserved":0,"allocated":3420160,"data_stored":1673360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81699999999999995,"15min":0.88200000000000001},"min":{"1min":0.33800000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.901},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.89200000000000002,"15min":1.0249999999999999},"min":{"1min":0.443,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1919999999999999,"5min":1.581,"15min":1.6679999999999999},"last":0.316}]},{"osd":1,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.88800000000000001,"15min":0.95899999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3420000000000001,"5min":1.508,"15min":1.508},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.94899999999999995,"15min":1.087},"min":{"1min":0.51300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4770000000000001,"5min":1.601,"15min":1.7190000000000001},"last":0.435}]},{"osd":2,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.99199999999999999,"15min":1.087},"min":{"1min":0.56299999999999994,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.524,"5min":1.5660000000000001,"15min":1.694},"last":0.495},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.079,"15min":1.1870000000000001},"min":{"1min":0.48499999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.597,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.61199999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.23,"15min":1.3240000000000001},"min":{"1min":0.72499999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6839999999999999,"5min":1.821,"15min":1.873},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2949999999999999,"15min":1.3440000000000001},"min":{"1min":0.80900000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.681,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.98299999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.3009999999999999,"15min":1.385},"min":{"1min":0.69299999999999995,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.7749999999999999,"5min":2.798,"15min":2.798},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3149999999999999,"15min":1.4179999999999999},"min":{"1min":0.74199999999999999,"5min":0.63,"15min":0.63},"max":{"1min":1.772,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":0.95299999999999996}]},{"osd":6,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4330000000000001,"15min":1.51},"min":{"1min":0.93000000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.9970000000000001,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.413,"15min":1.506},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9350000000000001,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.111}]},{"osd":7,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.46,"15min":1.544},"min":{"1min":0.96799999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.8380000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.5329999999999999,"15min":1.6459999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.952,"5min":2.3799999999999999,"15min":2.4649999999999999},"last":0.94299999999999995}]}]},{"osd":2,"up_from":13,"seq":55834576017,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56740,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315100,"statfs":{"total":96636764160,"available":96578662400,"internally_reserved":0,"allocated":3424256,"data_stored":1672902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.85799999999999998,"15min":0.89900000000000002},"min":{"1min":0.39800000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.3200000000000001,"5min":1.532,"15min":1.6399999999999999},"last":1.177},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.92100000000000004,"15min":1.0369999999999999},"min":{"1min":0.53400000000000003,"5min":0.40500000000000003,"15min":0.32600000000000001},"max":{"1min":1.258,"5min":1.6359999999999999,"15min":2.1429999999999998},"last":0.92900000000000005}]},{"osd":1,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.046,"15min":1.075},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39200000000000002},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.02,"15min":1.1120000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.246}]},{"osd":3,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.96099999999999997,"15min":1.0469999999999999},"min":{"1min":0.52300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.452,"5min":1.522,"15min":1.8560000000000001},"last":1.02},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1639999999999999,"15min":1.242},"min":{"1min":0.58199999999999996,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.0459999999999998,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":1.458}]},{"osd":4,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.306,"15min":1.3819999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.8079999999999998,"5min":3.8079999999999998,"15min":3.8079999999999998},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.2490000000000001,"15min":1.3140000000000001},"min":{"1min":0.79700000000000004,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.2889999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.397,"15min":1.431},"min":{"1min":0.77900000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3839999999999999,"15min":1.4470000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0499999999999998,"5min":2.323,"15min":2.6259999999999999},"last":1.508}]},{"osd":6,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3560000000000001,"15min":1.381},"min":{"1min":0.85099999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.4830000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":1.131},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.464,"15min":1.5089999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":1.5329999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.4910000000000001,"15min":1.5489999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.742,"5min":3.742,"15min":3.742},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.4339999999999999,"15min":1.4810000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":1.367}]}]},{"osd":1,"up_from":13,"seq":55834576017,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48412,"kb_used_data":3344,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323428,"statfs":{"total":96636764160,"available":96587190272,"internally_reserved":0,"allocated":3424256,"data_stored":1672834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.89300000000000002,"15min":0.91000000000000003},"min":{"1min":0.40699999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4039999999999999,"5min":5.117,"15min":5.117},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.89300000000000002,"15min":1.0029999999999999},"min":{"1min":0.45500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.325,"5min":1.5509999999999999,"15min":1.667},"last":0.82399999999999995}]},{"osd":2,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.91300000000000003,"15min":0.98899999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.651,"5min":2.093,"15min":2.093},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.149,"15min":1.242},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2400000000000002,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.872}]},{"osd":3,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.125,"15min":1.179},"min":{"1min":0.56999999999999995,"5min":0.439,"15min":0.439},"max":{"1min":1.8879999999999999,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.05,"15min":1.1399999999999999},"min":{"1min":0.52700000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5860000000000001,"5min":1.798,"15min":1.802},"last":1.238}]},{"osd":4,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.2749999999999999,"15min":1.2849999999999999},"min":{"1min":0.70299999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.978,"5min":5.29,"15min":5.29},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.403,"15min":1.409},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.0129999999999999,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":1.5129999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.3400000000000001,"15min":1.3979999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.829,"5min":6.0979999999999999,"15min":6.0979999999999999},"last":1.673},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.3420000000000001,"15min":1.3899999999999999},"min":{"1min":0.74099999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7410000000000001,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.5569999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.429,"15min":1.482},"min":{"1min":0.82399999999999995,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8979999999999999,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":1.7789999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.393,"15min":1.4630000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.948,"5min":3.8799999999999999,"15min":3.8799999999999999},"last":1.393}]},{"osd":7,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.518,"15min":1.5509999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.157,"5min":4.46,"15min":4.46},"last":1.722},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.5349999999999999,"15min":1.589},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.0259999999999998,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":1.8799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834576018,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48992,"kb_used_data":3924,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322848,"statfs":{"total":96636764160,"available":96586596352,"internally_reserved":0,"allocated":4018176,"data_stored":2263238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85499999999999998,"15min":0.88500000000000001},"min":{"1min":0.44900000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2350000000000001,"5min":1.502,"15min":1.619},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.98699999999999999,"15min":1.0740000000000001},"min":{"1min":0.40000000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.518,"5min":1.621,"15min":1.76},"last":0.91500000000000004}]},{"osd":2,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0640000000000001,"15min":1.0820000000000001},"min":{"1min":0.51300000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.569,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.996,"5min":1.036,"15min":1.1180000000000001},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.7649999999999999,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":1.05}]},{"osd":3,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0089999999999999,"15min":1.0640000000000001},"min":{"1min":0.621,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.5189999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.131,"15min":1.2170000000000001},"min":{"1min":0.59099999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.45,"5min":1.833,"15min":1.833},"last":1.2090000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.3149999999999999,"15min":1.377},"min":{"1min":0.93000000000000005,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.538,"5min":3.2799999999999998,"15min":3.2799999999999998},"last":1.284},{"interface":"front","average":{"1min":1.286,"5min":1.2869999999999999,"15min":1.3320000000000001},"min":{"1min":0.80000000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.752,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.6140000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.3109999999999999,"15min":1.3660000000000001},"min":{"1min":0.76000000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.605,"5min":2.0209999999999999,"15min":2.6669999999999998},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.478,"5min":1.385,"15min":1.4339999999999999},"min":{"1min":0.877,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.498}]},{"osd":6,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4570000000000001,"15min":1.4930000000000001},"min":{"1min":0.78900000000000003,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.1309999999999998,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":1.427},{"interface":"front","average":{"1min":1.363,"5min":1.4019999999999999,"15min":1.468},"min":{"1min":0.90200000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7310000000000001,"5min":2.073,"15min":2.262},"last":1.1639999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5309999999999999,"15min":1.5549999999999999},"min":{"1min":1.0429999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7869999999999999,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.419,"15min":1.486},"min":{"1min":0.98399999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7869999999999999,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.5309999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:27:28.424 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T11:27:28.424 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T11:27:28.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.680+0000 7f4f694e0640 1 Processor -- start 2023-10-16T11:27:28.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.680+0000 7f4f694e0640 1 -- start start 2023-10-16T11:27:28.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.682+0000 7f4f694e0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4f640ccfc0 0x7f4f640b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:28.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.682+0000 7f4f694e0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4f641615d0 0x7f4f640b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:28.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.682+0000 7f4f694e0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6415e3c0 con 0x7f4f6415d3f0 2023-10-16T11:27:28.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.682+0000 7f4f694e0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6415e0e0 con 0x7f4f640ccfc0 2023-10-16T11:27:28.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.682+0000 7f4f694e0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f64160af0 con 0x7f4f641615d0 2023-10-16T11:27:28.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.683+0000 7f4f62ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4f640ccfc0 0x7f4f640b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:28.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.683+0000 7f4f62ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4f640ccfc0 0x7f4f640b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42444/0 (socket says 172.21.15.84:42444) 2023-10-16T11:27:28.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.683+0000 7f4f62ffd640 1 -- 172.21.15.84:0/3651454445 learned_addr learned my addr 172.21.15.84:0/3651454445 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:28.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.683+0000 7f4f62ffd640 1 -- 172.21.15.84:0/3651454445 >> v2:172.21.15.157:3300/0 conn(0x7f4f641615d0 msgr2=0x7f4f640b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:28.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.683+0000 7f4f62ffd640 1 --2- 172.21.15.84:0/3651454445 >> v2:172.21.15.157:3300/0 conn(0x7f4f641615d0 0x7f4f640b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.683+0000 7f4f62ffd640 1 -- 172.21.15.84:0/3651454445 >> v1:172.21.15.157:6789/0 conn(0x7f4f6415d3f0 legacy=0x7f4f6415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:28.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.683+0000 7f4f62ffd640 1 -- 172.21.15.84:0/3651454445 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f64160dd0 con 0x7f4f640ccfc0 2023-10-16T11:27:28.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.684+0000 7f4f62ffd640 1 --2- 172.21.15.84:0/3651454445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f640ccfc0 0x7f4f640b0410 secure :-1 s=READY pgs=3410 cs=0 l=1 rev1=1 crypto rx=0x7f4f64160910 tx=0x7f4f5402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fdde898a42da4b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:28.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.684+0000 7f4f627fc640 1 -- 172.21.15.84:0/3651454445 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f540317e0 con 0x7f4f640ccfc0 2023-10-16T11:27:28.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.685+0000 7f4f627fc640 1 -- 172.21.15.84:0/3651454445 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f54031940 con 0x7f4f640ccfc0 2023-10-16T11:27:28.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.685+0000 7f4f627fc640 1 -- 172.21.15.84:0/3651454445 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54031cd0 con 0x7f4f640ccfc0 2023-10-16T11:27:28.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.685+0000 7f4f694e0640 1 -- 172.21.15.84:0/3651454445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f640ccfc0 msgr2=0x7f4f640b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:28.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.685+0000 7f4f694e0640 1 --2- 172.21.15.84:0/3651454445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f640ccfc0 0x7f4f640b0410 secure :-1 s=READY pgs=3410 cs=0 l=1 rev1=1 crypto rx=0x7f4f64160910 tx=0x7f4f5402b880 comp rx=0 tx=0).stop 2023-10-16T11:27:28.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.685+0000 7f4f694e0640 1 -- 172.21.15.84:0/3651454445 shutdown_connections 2023-10-16T11:27:28.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.685+0000 7f4f694e0640 1 --2- 172.21.15.84:0/3651454445 >> v2:172.21.15.157:3300/0 conn(0x7f4f641615d0 0x7f4f640b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.685+0000 7f4f694e0640 1 --2- 172.21.15.84:0/3651454445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f640ccfc0 0x7f4f640b0410 unknown :-1 s=CLOSED pgs=3410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.685+0000 7f4f694e0640 1 -- 172.21.15.84:0/3651454445 >> 172.21.15.84:0/3651454445 conn(0x7f4f640cb410 msgr2=0x7f4f640cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:28.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.686+0000 7f4f694e0640 1 -- 172.21.15.84:0/3651454445 shutdown_connections 2023-10-16T11:27:28.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.686+0000 7f4f694e0640 1 -- 172.21.15.84:0/3651454445 wait complete. 2023-10-16T11:27:28.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.688+0000 7f4f694e0640 1 Processor -- start 2023-10-16T11:27:28.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.688+0000 7f4f694e0640 1 -- start start 2023-10-16T11:27:28.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.689+0000 7f4f694e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f640ccfc0 0x7f4f640b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:28.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.689+0000 7f4f694e0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f6415d3f0 0x7f4f64154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:28.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.689+0000 7f4f694e0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f641615d0 0x7f4f6415c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:28.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.689+0000 7f4f694e0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6415c5e0 con 0x7f4f640ccfc0 2023-10-16T11:27:28.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.689+0000 7f4f694e0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640b2660 con 0x7f4f6415d3f0 2023-10-16T11:27:28.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.689+0000 7f4f694e0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640b2940 con 0x7f4f641615d0 2023-10-16T11:27:28.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.689+0000 7f4f62ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f640ccfc0 0x7f4f640b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:28.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.690+0000 7f4f62ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f640ccfc0 0x7f4f640b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42454/0 (socket says 172.21.15.84:42454) 2023-10-16T11:27:28.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.690+0000 7f4f62ffd640 1 -- 172.21.15.84:0/3868747570 learned_addr learned my addr 172.21.15.84:0/3868747570 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:28.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.690+0000 7f4f637fe640 1 --2- 172.21.15.84:0/3868747570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f641615d0 0x7f4f6415c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:28.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.690+0000 7f4f62ffd640 1 -- 172.21.15.84:0/3868747570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f641615d0 msgr2=0x7f4f6415c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:28.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.690+0000 7f4f62ffd640 1 --2- 172.21.15.84:0/3868747570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f641615d0 0x7f4f6415c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.690+0000 7f4f62ffd640 1 -- 172.21.15.84:0/3868747570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f6415d3f0 msgr2=0x7f4f64154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:28.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.690+0000 7f4f62ffd640 1 --2- 172.21.15.84:0/3868747570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f6415d3f0 0x7f4f64154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.690+0000 7f4f62ffd640 1 -- 172.21.15.84:0/3868747570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f540094d0 con 0x7f4f640ccfc0 2023-10-16T11:27:28.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.690+0000 7f4f62ffd640 1 --2- 172.21.15.84:0/3868747570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f640ccfc0 0x7f4f640b1f60 secure :-1 s=READY pgs=3411 cs=0 l=1 rev1=1 crypto rx=0x7f4f5402b8c0 tx=0x7f4f540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:28.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.690+0000 7f4f5b7fe640 1 -- 172.21.15.84:0/3868747570 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54032940 con 0x7f4f640ccfc0 2023-10-16T11:27:28.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.691+0000 7f4f5b7fe640 1 -- 172.21.15.84:0/3868747570 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f54032ac0 con 0x7f4f640ccfc0 2023-10-16T11:27:28.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.691+0000 7f4f5b7fe640 1 -- 172.21.15.84:0/3868747570 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54032e70 con 0x7f4f640ccfc0 2023-10-16T11:27:28.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.691+0000 7f4f694e0640 1 -- 172.21.15.84:0/3868747570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f640b2c20 con 0x7f4f640ccfc0 2023-10-16T11:27:28.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.691+0000 7f4f694e0640 1 -- 172.21.15.84:0/3868747570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f640b31e0 con 0x7f4f640ccfc0 2023-10-16T11:27:28.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.692+0000 7f4f694e0640 1 -- 172.21.15.84:0/3868747570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f640709e0 con 0x7f4f640ccfc0 2023-10-16T11:27:28.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.693+0000 7f4f5b7fe640 1 -- 172.21.15.84:0/3868747570 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4f540453f0 con 0x7f4f640ccfc0 2023-10-16T11:27:28.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.693+0000 7f4f5b7fe640 1 --2- 172.21.15.84:0/3868747570 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f4f4406af90 0x7f4f4406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:28.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.694+0000 7f4f5b7fe640 1 -- 172.21.15.84:0/3868747570 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4f540abe00 con 0x7f4f640ccfc0 2023-10-16T11:27:28.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.694+0000 7f4f5bfff640 1 --2- 172.21.15.84:0/3868747570 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f4f4406af90 0x7f4f4406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:28.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.694+0000 7f4f5bfff640 1 --2- 172.21.15.84:0/3868747570 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f4f4406af90 0x7f4f4406d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4f64070680 tx=0x7f4f4c006c00 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:28.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.698+0000 7f4f5b7fe640 1 -- 172.21.15.84:0/3868747570 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4f54076d80 con 0x7f4f640ccfc0 2023-10-16T11:27:28.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.960+0000 7f4f694e0640 1 -- 172.21.15.84:0/3868747570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4f640709e0 con 0x7f4f640ccfc0 2023-10-16T11:27:28.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.965+0000 7f4f5b7fe640 1 -- 172.21.15.84:0/3868747570 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v59) v1 ==== 190+0+147 (secure 0 0 0) 0x7f4f5407a3a0 con 0x7f4f640ccfc0 2023-10-16T11:27:28.967 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2023-10-16T11:27:28.968 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQDTBi1lglobLxAASJzlj7+4uUfhHkEkY4hDbQ== 2023-10-16T11:27:28.968 INFO:teuthology.orchestra.run.smithi084.stdout: caps mds = "allow" 2023-10-16T11:27:28.968 INFO:teuthology.orchestra.run.smithi084.stdout: caps mon = "allow r" 2023-10-16T11:27:28.968 INFO:teuthology.orchestra.run.smithi084.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T11:27:28.969 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-16T11:27:28.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.968+0000 7f4f694e0640 1 -- 172.21.15.84:0/3868747570 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f4f4406af90 msgr2=0x7f4f4406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:28.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.968+0000 7f4f694e0640 1 --2- 172.21.15.84:0/3868747570 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f4f4406af90 0x7f4f4406d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4f64070680 tx=0x7f4f4c006c00 comp rx=0 tx=0).stop 2023-10-16T11:27:28.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.968+0000 7f4f694e0640 1 -- 172.21.15.84:0/3868747570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f640ccfc0 msgr2=0x7f4f640b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:28.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.968+0000 7f4f694e0640 1 --2- 172.21.15.84:0/3868747570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f640ccfc0 0x7f4f640b1f60 secure :-1 s=READY pgs=3411 cs=0 l=1 rev1=1 crypto rx=0x7f4f5402b8c0 tx=0x7f4f540046c0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.968+0000 7f4f694e0640 1 -- 172.21.15.84:0/3868747570 shutdown_connections 2023-10-16T11:27:28.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.968+0000 7f4f694e0640 1 --2- 172.21.15.84:0/3868747570 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f4f4406af90 0x7f4f4406d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.968+0000 7f4f694e0640 1 --2- 172.21.15.84:0/3868747570 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f641615d0 0x7f4f6415c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.968+0000 7f4f694e0640 1 --2- 172.21.15.84:0/3868747570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f6415d3f0 0x7f4f64154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.968+0000 7f4f694e0640 1 --2- 172.21.15.84:0/3868747570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f640ccfc0 0x7f4f640b1f60 unknown :-1 s=CLOSED pgs=3411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:28.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.968+0000 7f4f694e0640 1 -- 172.21.15.84:0/3868747570 >> 172.21.15.84:0/3868747570 conn(0x7f4f640cb410 msgr2=0x7f4f6406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:28.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.968+0000 7f4f694e0640 1 -- 172.21.15.84:0/3868747570 shutdown_connections 2023-10-16T11:27:28.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:28.968+0000 7f4f694e0640 1 -- 172.21.15.84:0/3868747570 wait complete. 2023-10-16T11:27:28.990 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:27:29.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.203+0000 7fe23b251640 1 Processor -- start 2023-10-16T11:27:29.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.203+0000 7fe23b251640 1 -- start start 2023-10-16T11:27:29.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.205+0000 7fe23b251640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe2340b0b20 0x7fe234164b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:29.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.205+0000 7fe23b251640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe23406f7f0 con 0x7fe234159f00 2023-10-16T11:27:29.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.205+0000 7fe23b251640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe23406c5e0 con 0x7fe234159b20 2023-10-16T11:27:29.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.205+0000 7fe23b251640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe23406c8c0 con 0x7fe2340b0b20 2023-10-16T11:27:29.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.205+0000 7fe2397ee640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe2340b0b20 0x7fe234164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:29.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.205+0000 7fe233fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe234159f00 0x7fe2340b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43342/0 (socket says 172.21.15.84:43342) 2023-10-16T11:27:29.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.205+0000 7fe233fff640 1 -- 172.21.15.84:0/190483407 learned_addr learned my addr 172.21.15.84:0/190483407 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:29.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.206+0000 7fe2397ee640 1 -- 172.21.15.84:0/190483407 >> v1:172.21.15.157:6789/0 conn(0x7fe234159b20 legacy=0x7fe23406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:29.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.206+0000 7fe2397ee640 1 -- 172.21.15.84:0/190483407 >> v1:172.21.15.84:6789/0 conn(0x7fe234159f00 legacy=0x7fe2340b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:29.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.206+0000 7fe2397ee640 1 -- 172.21.15.84:0/190483407 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23406cba0 con 0x7fe2340b0b20 2023-10-16T11:27:29.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.207+0000 7fe2397ee640 1 --2- 172.21.15.84:0/190483407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2340b0b20 0x7fe234164b40 secure :-1 s=READY pgs=3413 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009d90 tx=0x7fe22c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38357a6b9d4e2aa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:29.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.207+0000 7fe2337fe640 1 -- 172.21.15.84:0/190483407 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c02cc60 con 0x7fe2340b0b20 2023-10-16T11:27:29.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.207+0000 7fe2337fe640 1 -- 172.21.15.84:0/190483407 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c02cdc0 con 0x7fe2340b0b20 2023-10-16T11:27:29.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.207+0000 7fe2337fe640 1 -- 172.21.15.84:0/190483407 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c034660 con 0x7fe2340b0b20 2023-10-16T11:27:29.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.208+0000 7fe23b251640 1 -- 172.21.15.84:0/190483407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2340b0b20 msgr2=0x7fe234164b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:29.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.208+0000 7fe23b251640 1 --2- 172.21.15.84:0/190483407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2340b0b20 0x7fe234164b40 secure :-1 s=READY pgs=3413 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009d90 tx=0x7fe22c02beb0 comp rx=0 tx=0).stop 2023-10-16T11:27:29.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.208+0000 7fe23b251640 1 -- 172.21.15.84:0/190483407 shutdown_connections 2023-10-16T11:27:29.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.208+0000 7fe23b251640 1 --2- 172.21.15.84:0/190483407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2340b0b20 0x7fe234164b40 unknown :-1 s=CLOSED pgs=3413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:29.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.208+0000 7fe23b251640 1 -- 172.21.15.84:0/190483407 >> 172.21.15.84:0/190483407 conn(0x7fe2340cb410 msgr2=0x7fe2340cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:29.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.208+0000 7fe23b251640 1 -- 172.21.15.84:0/190483407 shutdown_connections 2023-10-16T11:27:29.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.208+0000 7fe23b251640 1 -- 172.21.15.84:0/190483407 wait complete. 2023-10-16T11:27:29.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.210+0000 7fe23b251640 1 Processor -- start 2023-10-16T11:27:29.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.210+0000 7fe23b251640 1 -- start start 2023-10-16T11:27:29.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.211+0000 7fe23b251640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2340b0b20 0x7fe2341549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:29.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.211+0000 7fe23b251640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe234159b20 0x7fe234153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:29.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.211+0000 7fe23b251640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe234159f00 0x7fe2340b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:29.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.211+0000 7fe23b251640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2340c36b0 con 0x7fe234159f00 2023-10-16T11:27:29.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.211+0000 7fe23b251640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2340c4070 con 0x7fe234159b20 2023-10-16T11:27:29.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.211+0000 7fe23b251640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe234154200 con 0x7fe2340b0b20 2023-10-16T11:27:29.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.212+0000 7fe2397ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe234159f00 0x7fe2340b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:29.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.212+0000 7fe2397ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe234159f00 0x7fe2340b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42474/0 (socket says 172.21.15.84:42474) 2023-10-16T11:27:29.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.212+0000 7fe2397ee640 1 -- 172.21.15.84:0/3452792941 learned_addr learned my addr 172.21.15.84:0/3452792941 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:29.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.212+0000 7fe238fed640 1 --2- 172.21.15.84:0/3452792941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2340b0b20 0x7fe2341549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:29.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.212+0000 7fe233fff640 1 --2- 172.21.15.84:0/3452792941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe234159b20 0x7fe234153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:29.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.212+0000 7fe2397ee640 1 -- 172.21.15.84:0/3452792941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2340b0b20 msgr2=0x7fe2341549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:29.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.212+0000 7fe2397ee640 1 --2- 172.21.15.84:0/3452792941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2340b0b20 0x7fe2341549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:29.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.212+0000 7fe2397ee640 1 -- 172.21.15.84:0/3452792941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe234159b20 msgr2=0x7fe234153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:29.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.212+0000 7fe2397ee640 1 --2- 172.21.15.84:0/3452792941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe234159b20 0x7fe234153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:29.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.212+0000 7fe2397ee640 1 -- 172.21.15.84:0/3452792941 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c009a70 con 0x7fe234159f00 2023-10-16T11:27:29.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.212+0000 7fe233fff640 1 --2- 172.21.15.84:0/3452792941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe234159b20 0x7fe234153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:29.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.213+0000 7fe2397ee640 1 --2- 172.21.15.84:0/3452792941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe234159f00 0x7fe2340b3d10 secure :-1 s=READY pgs=3414 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009d90 tx=0x7fe22c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:29.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.213+0000 7fe2317fa640 1 -- 172.21.15.84:0/3452792941 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c03e610 con 0x7fe234159f00 2023-10-16T11:27:29.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.213+0000 7fe2317fa640 1 -- 172.21.15.84:0/3452792941 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c03e790 con 0x7fe234159f00 2023-10-16T11:27:29.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.213+0000 7fe2317fa640 1 -- 172.21.15.84:0/3452792941 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c03eb40 con 0x7fe234159f00 2023-10-16T11:27:29.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.213+0000 7fe23b251640 1 -- 172.21.15.84:0/3452792941 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2340c33d0 con 0x7fe234159f00 2023-10-16T11:27:29.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.213+0000 7fe23b251640 1 -- 172.21.15.84:0/3452792941 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2340c3d00 con 0x7fe234159f00 2023-10-16T11:27:29.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.214+0000 7fe23b251640 1 -- 172.21.15.84:0/3452792941 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2340c3880 con 0x7fe234159f00 2023-10-16T11:27:29.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.215+0000 7fe2317fa640 1 -- 172.21.15.84:0/3452792941 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe22c03b050 con 0x7fe234159f00 2023-10-16T11:27:29.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.216+0000 7fe2317fa640 1 --2- 172.21.15.84:0/3452792941 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fe21806afc0 0x7fe21806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:29.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.217+0000 7fe2317fa640 1 -- 172.21.15.84:0/3452792941 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe22c0ab7d0 con 0x7fe234159f00 2023-10-16T11:27:29.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.217+0000 7fe238fed640 1 --2- 172.21.15.84:0/3452792941 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fe21806afc0 0x7fe21806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:29.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.217+0000 7fe238fed640 1 --2- 172.21.15.84:0/3452792941 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fe21806afc0 0x7fe21806d480 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe228004ea0 tx=0x7fe228009040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:29.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.221+0000 7fe2317fa640 1 -- 172.21.15.84:0/3452792941 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe2340c3880 con 0x7fe234159f00 2023-10-16T11:27:29.376 INFO:tasks.ceph.mds.a.smithi084.stderr:2023-10-16T11:27:29.373+0000 7fbe1a45b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:27:29.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.434+0000 7fe23b251640 1 -- 172.21.15.84:0/3452792941 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2340c3880 con 0x7fe234159f00 2023-10-16T11:27:29.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.435+0000 7fe2317fa640 1 -- 172.21.15.84:0/3452792941 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+4933 (secure 0 0 0) 0x7fe2340c3880 con 0x7fe234159f00 2023-10-16T11:27:29.437 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:29.438 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":157,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7457,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3193159532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3193159532},{"type":"v1","addr":"172.21.15.157:6835","nonce":3193159532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":152},{"gid":8431,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1728897149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1728897149},{"type":"v1","addr":"172.21.15.157:6834","nonce":1728897149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":152},{"gid":13476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3409165899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3409165899},{"type":"v1","addr":"172.21.15.84:6835","nonce":3409165899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":157,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:27:22.871058+0000","modified":"2023-10-16T11:27:25.974995+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":13482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13482":{"gid":13482,"name":"a","rank":0,"incarnation":153,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2806813701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2806813701},{"type":"v1","addr":"172.21.15.84:6837","nonce":2806813701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[37],"metadata_pool":36,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":18}]} 2023-10-16T11:27:29.438 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 157 2023-10-16T11:27:29.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.438+0000 7fe23b251640 1 -- 172.21.15.84:0/3452792941 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fe21806afc0 msgr2=0x7fe21806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:29.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.438+0000 7fe23b251640 1 --2- 172.21.15.84:0/3452792941 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fe21806afc0 0x7fe21806d480 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe228004ea0 tx=0x7fe228009040 comp rx=0 tx=0).stop 2023-10-16T11:27:29.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.438+0000 7fe23b251640 1 -- 172.21.15.84:0/3452792941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe234159f00 msgr2=0x7fe2340b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:29.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.438+0000 7fe23b251640 1 --2- 172.21.15.84:0/3452792941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe234159f00 0x7fe2340b3d10 secure :-1 s=READY pgs=3414 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009d90 tx=0x7fe22c004a40 comp rx=0 tx=0).stop 2023-10-16T11:27:29.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.438+0000 7fe23b251640 1 -- 172.21.15.84:0/3452792941 shutdown_connections 2023-10-16T11:27:29.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.438+0000 7fe23b251640 1 --2- 172.21.15.84:0/3452792941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe234159f00 0x7fe2340b3d10 unknown :-1 s=CLOSED pgs=3414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:29.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.438+0000 7fe23b251640 1 --2- 172.21.15.84:0/3452792941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe234159b20 0x7fe234153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:29.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.438+0000 7fe23b251640 1 --2- 172.21.15.84:0/3452792941 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fe21806afc0 0x7fe21806d480 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:29.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.438+0000 7fe23b251640 1 --2- 172.21.15.84:0/3452792941 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2340b0b20 0x7fe2341549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:29.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.438+0000 7fe23b251640 1 -- 172.21.15.84:0/3452792941 >> 172.21.15.84:0/3452792941 conn(0x7fe2340cb410 msgr2=0x7fe23406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:29.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.438+0000 7fe23b251640 1 -- 172.21.15.84:0/3452792941 shutdown_connections 2023-10-16T11:27:29.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:29.438+0000 7fe23b251640 1 -- 172.21.15.84:0/3452792941 wait complete. 2023-10-16T11:27:29.861 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 157, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T11:27:22.871058+0000', 'modified': '2023-10-16T11:27:25.974995+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 13482}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_13482': {'gid': 13482, 'name': 'a', 'rank': 0, 'incarnation': 153, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.84:6837/2806813701', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6836', 'nonce': 2806813701}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 2806813701}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [37], 'metadata_pool': 36, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T11:27:29.861 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T11:27:29.862 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-16T11:27:30.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.086+0000 7fc9bf039640 1 Processor -- start 2023-10-16T11:27:30.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.086+0000 7fc9bf039640 1 -- start start 2023-10-16T11:27:30.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.087+0000 7fc9bf039640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc9b815c9f0 0x7fc9b80c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.087+0000 7fc9bf039640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9b815dde0 0x7fc9b80ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.087+0000 7fc9bf039640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b8160a90 con 0x7fc9b80c9230 2023-10-16T11:27:30.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.087+0000 7fc9bf039640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9b80cdec0 con 0x7fc9b815c9f0 2023-10-16T11:27:30.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.087+0000 7fc9bf039640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b80cd900 con 0x7fc9b815dde0 2023-10-16T11:27:30.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.088+0000 7fc9be838640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9b815dde0 0x7fc9b80ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.088+0000 7fc9be037640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc9b815c9f0 0x7fc9b80c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.088+0000 7fc9bd836640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc9b80c9230 0x7fc9b80709e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:50952/0 (socket says 172.21.15.84:50952) 2023-10-16T11:27:30.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.088+0000 7fc9bd836640 1 -- 172.21.15.84:0/3700851752 learned_addr learned my addr 172.21.15.84:0/3700851752 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:30.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.089+0000 7fc9be037640 1 -- 172.21.15.84:0/3700851752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9b815dde0 msgr2=0x7fc9b80ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:30.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.089+0000 7fc9be037640 1 --2- 172.21.15.84:0/3700851752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9b815dde0 0x7fc9b80ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.089+0000 7fc9be037640 1 -- 172.21.15.84:0/3700851752 >> v1:172.21.15.157:6789/0 conn(0x7fc9b80c9230 legacy=0x7fc9b80709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:30.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.089+0000 7fc9be037640 1 -- 172.21.15.84:0/3700851752 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b80cdbe0 con 0x7fc9b815c9f0 2023-10-16T11:27:30.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.090+0000 7fc9be037640 1 --2- 172.21.15.84:0/3700851752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b815c9f0 0x7fc9b80c8cf0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc9b40097a0 tx=0x7fc9b402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7c2aa871b8c1fe3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:30.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.091+0000 7fc9bd035640 1 -- 172.21.15.84:0/3700851752 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b402d040 con 0x7fc9b815c9f0 2023-10-16T11:27:30.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.091+0000 7fc9bd035640 1 -- 172.21.15.84:0/3700851752 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b4031810 con 0x7fc9b815c9f0 2023-10-16T11:27:30.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.091+0000 7fc9bd035640 1 -- 172.21.15.84:0/3700851752 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4031ba0 con 0x7fc9b815c9f0 2023-10-16T11:27:30.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.091+0000 7fc9bf039640 1 -- 172.21.15.84:0/3700851752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b815c9f0 msgr2=0x7fc9b80c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:30.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.091+0000 7fc9bf039640 1 --2- 172.21.15.84:0/3700851752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b815c9f0 0x7fc9b80c8cf0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc9b40097a0 tx=0x7fc9b402b8b0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.091+0000 7fc9bf039640 1 -- 172.21.15.84:0/3700851752 shutdown_connections 2023-10-16T11:27:30.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.091+0000 7fc9bf039640 1 --2- 172.21.15.84:0/3700851752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9b815dde0 0x7fc9b80ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.091+0000 7fc9bf039640 1 --2- 172.21.15.84:0/3700851752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b815c9f0 0x7fc9b80c8cf0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.091+0000 7fc9bf039640 1 -- 172.21.15.84:0/3700851752 >> 172.21.15.84:0/3700851752 conn(0x7fc9b80ce6a0 msgr2=0x7fc9b80ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:30.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.091+0000 7fc9bf039640 1 -- 172.21.15.84:0/3700851752 shutdown_connections 2023-10-16T11:27:30.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.091+0000 7fc9bf039640 1 -- 172.21.15.84:0/3700851752 wait complete. 2023-10-16T11:27:30.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.093+0000 7fc9bf039640 1 Processor -- start 2023-10-16T11:27:30.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.093+0000 7fc9bf039640 1 -- start start 2023-10-16T11:27:30.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.094+0000 7fc9bf039640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b80c9230 0x7fc9b814e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.094+0000 7fc9bf039640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9b815c9f0 0x7fc9b814e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.094+0000 7fc9bf039640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9b815dde0 0x7fc9b806c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.094+0000 7fc9bf039640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b80c6300 con 0x7fc9b815c9f0 2023-10-16T11:27:30.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.094+0000 7fc9bf039640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b80c65e0 con 0x7fc9b80c9230 2023-10-16T11:27:30.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.094+0000 7fc9bf039640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b80c68c0 con 0x7fc9b815dde0 2023-10-16T11:27:30.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.094+0000 7fc9bd836640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9b815c9f0 0x7fc9b814e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.095+0000 7fc9bd836640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9b815c9f0 0x7fc9b814e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42476/0 (socket says 172.21.15.84:42476) 2023-10-16T11:27:30.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.095+0000 7fc9bd836640 1 -- 172.21.15.84:0/283324763 learned_addr learned my addr 172.21.15.84:0/283324763 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:30.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.095+0000 7fc9be838640 1 --2- 172.21.15.84:0/283324763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9b815dde0 0x7fc9b806c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.095+0000 7fc9be037640 1 --2- 172.21.15.84:0/283324763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b80c9230 0x7fc9b814e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.095+0000 7fc9bd836640 1 -- 172.21.15.84:0/283324763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9b815dde0 msgr2=0x7fc9b806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:30.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.095+0000 7fc9bd836640 1 --2- 172.21.15.84:0/283324763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9b815dde0 0x7fc9b806c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.095+0000 7fc9bd836640 1 -- 172.21.15.84:0/283324763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b80c9230 msgr2=0x7fc9b814e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:30.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.095+0000 7fc9bd836640 1 --2- 172.21.15.84:0/283324763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b80c9230 0x7fc9b814e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.095+0000 7fc9bd836640 1 -- 172.21.15.84:0/283324763 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b4009400 con 0x7fc9b815c9f0 2023-10-16T11:27:30.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.095+0000 7fc9bd836640 1 --2- 172.21.15.84:0/283324763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9b815c9f0 0x7fc9b814e790 secure :-1 s=READY pgs=3415 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800c400 tx=0x7fc9a800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:30.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.096+0000 7fc9aeffd640 1 -- 172.21.15.84:0/283324763 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8005ce0 con 0x7fc9b815c9f0 2023-10-16T11:27:30.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.096+0000 7fc9aeffd640 1 -- 172.21.15.84:0/283324763 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a8005e40 con 0x7fc9b815c9f0 2023-10-16T11:27:30.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.096+0000 7fc9aeffd640 1 -- 172.21.15.84:0/283324763 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a80072d0 con 0x7fc9b815c9f0 2023-10-16T11:27:30.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.096+0000 7fc9bf039640 1 -- 172.21.15.84:0/283324763 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b80c6ba0 con 0x7fc9b815c9f0 2023-10-16T11:27:30.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.096+0000 7fc9bf039640 1 -- 172.21.15.84:0/283324763 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b814ede0 con 0x7fc9b815c9f0 2023-10-16T11:27:30.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.098+0000 7fc9aeffd640 1 -- 172.21.15.84:0/283324763 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc9a8007430 con 0x7fc9b815c9f0 2023-10-16T11:27:30.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.099+0000 7fc9aeffd640 1 --2- 172.21.15.84:0/283324763 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fc99c06afc0 0x7fc99c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.099+0000 7fc9aeffd640 1 -- 172.21.15.84:0/283324763 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc9a8088df0 con 0x7fc9b815c9f0 2023-10-16T11:27:30.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.099+0000 7fc9be037640 1 --2- 172.21.15.84:0/283324763 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fc99c06afc0 0x7fc99c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.100+0000 7fc9c2e30740 1 Processor -- start 2023-10-16T11:27:30.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.100+0000 7fc9c2e30740 1 -- start start 2023-10-16T11:27:30.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.100+0000 7fc9be037640 1 --2- 172.21.15.84:0/283324763 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fc99c06afc0 0x7fc99c06d480 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4036000 tx=0x7fc9b403c040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:30.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.101+0000 7fc9c2e30740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558aeb4cb170 0x558aeb49e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.101+0000 7fc9c2e30740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb550 0x558aeb4cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.101+0000 7fc9c2e30740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb49e5b0 0x558aeb4ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.101+0000 7fc9c2e30740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aeb3be7a0 con 0x558aeb49e5b0 2023-10-16T11:27:30.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.101+0000 7fc9c2e30740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aeb410740 con 0x558aeb4cb550 2023-10-16T11:27:30.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.101+0000 7fc9c2e30740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x558aeb40de30 con 0x558aeb4cb170 2023-10-16T11:27:30.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.101+0000 7fc9be838640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb49e5b0 0x558aeb4ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.101+0000 7fc9be838640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb49e5b0 0x558aeb4ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42490/0 (socket says 172.21.15.84:42490) 2023-10-16T11:27:30.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.101+0000 7fc9be838640 1 -- 172.21.15.84:0/1595511469 learned_addr learned my addr 172.21.15.84:0/1595511469 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:30.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.101+0000 7fc9be838640 1 -- 172.21.15.84:0/1595511469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558aeb4cb170 msgr2=0x558aeb49e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:30.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.101+0000 7fc9be838640 1 --2- 172.21.15.84:0/1595511469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558aeb4cb170 0x558aeb49e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.101+0000 7fc9be838640 1 -- 172.21.15.84:0/1595511469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb550 msgr2=0x558aeb4cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:30.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.101+0000 7fc9be838640 1 --2- 172.21.15.84:0/1595511469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb550 0x558aeb4cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.101+0000 7fc9be838640 1 -- 172.21.15.84:0/1595511469 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558aeb3cd910 con 0x558aeb49e5b0 2023-10-16T11:27:30.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.102+0000 7fc9be838640 1 --2- 172.21.15.84:0/1595511469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb49e5b0 0x558aeb4ce440 secure :-1 s=READY pgs=3416 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0007490 tx=0x7fc9b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73eb6aaef9a622bc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:30.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.102+0000 7fc99bfff640 1 -- 172.21.15.84:0/1595511469 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b000e030 con 0x558aeb49e5b0 2023-10-16T11:27:30.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.102+0000 7fc99bfff640 1 -- 172.21.15.84:0/1595511469 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b0015600 con 0x558aeb49e5b0 2023-10-16T11:27:30.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.102+0000 7fc99bfff640 1 -- 172.21.15.84:0/1595511469 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0009c40 con 0x558aeb49e5b0 2023-10-16T11:27:30.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.102+0000 7fc9c2e30740 1 -- 172.21.15.84:0/1595511469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb49e5b0 msgr2=0x558aeb4ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:30.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.102+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/1595511469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb49e5b0 0x558aeb4ce440 secure :-1 s=READY pgs=3416 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0007490 tx=0x7fc9b0009750 comp rx=0 tx=0).stop 2023-10-16T11:27:30.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.103+0000 7fc9c2e30740 1 -- 172.21.15.84:0/1595511469 shutdown_connections 2023-10-16T11:27:30.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.103+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/1595511469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb49e5b0 0x558aeb4ce440 unknown :-1 s=CLOSED pgs=3416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.103+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/1595511469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb550 0x558aeb4cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.103+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/1595511469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558aeb4cb170 0x558aeb49e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.103+0000 7fc9c2e30740 1 -- 172.21.15.84:0/1595511469 >> 172.21.15.84:0/1595511469 conn(0x558aeb4a0380 msgr2=0x558aeb4e8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:30.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.103+0000 7fc9c2e30740 1 -- 172.21.15.84:0/1595511469 shutdown_connections 2023-10-16T11:27:30.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.103+0000 7fc9c2e30740 1 -- 172.21.15.84:0/1595511469 wait complete. 2023-10-16T11:27:30.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.103+0000 7fc9c2e30740 1 -- start start 2023-10-16T11:27:30.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.104+0000 7fc9c2e30740 1 Processor -- start 2023-10-16T11:27:30.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.105+0000 7fc9c2e30740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb170 0x558aeb4f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.105+0000 7fc9c2e30740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4cb550 0x558aeb4c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.105+0000 7fc9c2e30740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558aeb4e2040 0x558aeb4e24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.105+0000 7fc9c2e30740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aeb409720 con 0x558aeb4cb550 2023-10-16T11:27:30.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.105+0000 7fc9c2e30740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aeb3a7970 con 0x558aeb4cb170 2023-10-16T11:27:30.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.105+0000 7fc9c2e30740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x558aeb3a7610 con 0x558aeb4e2040 2023-10-16T11:27:30.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.105+0000 7fc9c2e30740 10 client.0 _ll_register_callbacks cb 0x558aeb3c99d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:27:30.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.105+0000 7fc9bd836640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4cb550 0x558aeb4c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.105+0000 7fc9bd836640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4cb550 0x558aeb4c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42496/0 (socket says 172.21.15.84:42496) 2023-10-16T11:27:30.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.105+0000 7fc9bd836640 1 -- 172.21.15.84:0/2241033511 learned_addr learned my addr 172.21.15.84:0/2241033511 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:30.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.105+0000 7fc9be838640 1 --2- 172.21.15.84:0/2241033511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558aeb4e2040 0x558aeb4e24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.105+0000 7fc9be037640 1 --2- 172.21.15.84:0/2241033511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb170 0x558aeb4f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.105+0000 7fc9bd836640 1 -- 172.21.15.84:0/2241033511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558aeb4e2040 msgr2=0x558aeb4e24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:30.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.105+0000 7fc9bd836640 1 --2- 172.21.15.84:0/2241033511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558aeb4e2040 0x558aeb4e24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.105+0000 7fc9bd836640 1 -- 172.21.15.84:0/2241033511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb170 msgr2=0x558aeb4f8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:30.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.105+0000 7fc9bd836640 1 --2- 172.21.15.84:0/2241033511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb170 0x558aeb4f8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.105+0000 7fc9bd836640 1 -- 172.21.15.84:0/2241033511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b0009da0 con 0x558aeb4cb550 2023-10-16T11:27:30.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.106+0000 7fc9bd836640 1 --2- 172.21.15.84:0/2241033511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4cb550 0x558aeb4c8db0 secure :-1 s=READY pgs=3417 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8087140 tx=0x7fc9a8057ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e23d0994a1aa281 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:30.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.106+0000 7fc9997fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T11:27:30.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.106+0000 7fc9997fa640 1 -- 172.21.15.84:0/2241033511 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a80540a0 con 0x558aeb4cb550 2023-10-16T11:27:30.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.106+0000 7fc9997fa640 1 -- 172.21.15.84:0/2241033511 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(270..270 src has 1..270) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc9a8054200 con 0x558aeb4cb550 2023-10-16T11:27:30.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.106+0000 7fc9c2e30740 1 -- 172.21.15.84:0/2241033511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558aeb430160 con 0x558aeb4cb550 2023-10-16T11:27:30.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.106+0000 7fc9997fa640 1 -- 172.21.15.84:0/2241033511 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=157) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9a805f800 con 0x558aeb4cb550 2023-10-16T11:27:30.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.106+0000 7fc9c2e30740 10 client.13611 fetch_fsmap learned FSMap version 157 2023-10-16T11:27:30.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.106+0000 7fc9c2e30740 1 -- 172.21.15.84:0/2241033511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=157,monmap=2+}) v3 -- 0x558aeb407da0 con 0x558aeb4cb550 2023-10-16T11:27:30.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.106+0000 7fc9997fa640 1 -- 172.21.15.84:0/2241033511 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558aeb430160 con 0x558aeb4cb550 2023-10-16T11:27:30.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.106+0000 7fc9997fa640 1 -- 172.21.15.84:0/2241033511 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 157) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc9a8053780 con 0x558aeb4cb550 2023-10-16T11:27:30.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.107+0000 7fc9c2e30740 10 client.13611 fetch_fsmap finished waiting for FSMap version 157 2023-10-16T11:27:30.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.107+0000 7fc9c2e30740 10 client.13611 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T11:27:30.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.107+0000 7fc9c2e30740 20 client.13611 populate_metadata read hostname 'smithi084' 2023-10-16T11:27:30.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.107+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/2241033511 >> [v2:172.21.15.84:6836/2806813701,v1:172.21.15.84:6837/2806813701] conn(0x558aeb4d0470 0x558aeb4e09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.107+0000 7fc9c2e30740 4 client.13611 mds_command: new command op to 13482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T11:27:30.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.107+0000 7fc9c2e30740 1 -- 172.21.15.84:0/2241033511 --> [v2:172.21.15.84:6836/2806813701,v1:172.21.15.84:6837/2806813701] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558aeb3dbcc0 con 0x558aeb4d0470 2023-10-16T11:27:30.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.107+0000 7fc9be037640 1 --2- 172.21.15.84:0/2241033511 >> [v2:172.21.15.84:6836/2806813701,v1:172.21.15.84:6837/2806813701] conn(0x558aeb4d0470 0x558aeb4e09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.107+0000 7fc9be037640 1 --2- 172.21.15.84:0/2241033511 >> [v2:172.21.15.84:6836/2806813701,v1:172.21.15.84:6837/2806813701] conn(0x558aeb4d0470 0x558aeb4e09e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=827084954860a1d server_cookie=3d4e96a9efd14d53 in_seq=0 out_seq=0 2023-10-16T11:27:30.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.107+0000 7fc9997fa640 10 client.13611 ms_handle_connect on v2:172.21.15.84:6836/2806813701 2023-10-16T11:27:30.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.108+0000 7fc9997fa640 1 -- 172.21.15.84:0/2241033511 <== mds.0 v2:172.21.15.84:6836/2806813701 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558aeb3dbcc0 con 0x558aeb4d0470 2023-10-16T11:27:30.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.108+0000 7fc9997fa640 10 client.13611 handle_command_reply: tid=0 2023-10-16T11:27:30.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.108+0000 7fc9c2e30740 1 client.13611 shutdown 2023-10-16T11:27:30.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.108+0000 7fc9c2e30740 1 -- 172.21.15.84:0/2241033511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4cb550 msgr2=0x558aeb4c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:30.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.108+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/2241033511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4cb550 0x558aeb4c8db0 secure :-1 s=READY pgs=3417 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8087140 tx=0x7fc9a8057ad0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.108+0000 7fc9c2e30740 1 -- 172.21.15.84:0/2241033511 shutdown_connections 2023-10-16T11:27:30.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.108+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/2241033511 >> [v2:172.21.15.84:6836/2806813701,v1:172.21.15.84:6837/2806813701] conn(0x558aeb4d0470 0x558aeb4e09e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.108+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/2241033511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558aeb4e2040 0x558aeb4e24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.108+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/2241033511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4cb550 0x558aeb4c8db0 unknown :-1 s=CLOSED pgs=3417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.108+0000 7fc9997fa640 0 client.13611 ms_handle_reset on v2:172.21.15.84:6836/2806813701 2023-10-16T11:27:30.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.108+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/2241033511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb170 0x558aeb4f8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.108+0000 7fc9c2e30740 1 -- 172.21.15.84:0/2241033511 >> 172.21.15.84:0/2241033511 conn(0x558aeb49e5b0 msgr2=0x558aeb4a2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:30.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.109+0000 7fc9c2e30740 1 -- 172.21.15.84:0/2241033511 shutdown_connections 2023-10-16T11:27:30.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.109+0000 7fc9c2e30740 1 -- 172.21.15.84:0/2241033511 wait complete. 2023-10-16T11:27:30.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.109+0000 7fc9c2e30740 20 client.13611 trim_cache size 0 max 16384 2023-10-16T11:27:30.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.125+0000 7fc9c2e30740 1 Processor -- start 2023-10-16T11:27:30.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.125+0000 7fc9c2e30740 1 -- start start 2023-10-16T11:27:30.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.126+0000 7fc9c2e30740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4cb170 0x558aeb4e1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.126+0000 7fc9c2e30740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb550 0x558aeb4d1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.126+0000 7fc9c2e30740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558aeb4e2040 0x558aeb4a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.126+0000 7fc9c2e30740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aeb16df20 con 0x558aeb4cb170 2023-10-16T11:27:30.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.126+0000 7fc9c2e30740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aeb50a980 con 0x558aeb4cb550 2023-10-16T11:27:30.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.126+0000 7fc9c2e30740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x558aeb4f8ac0 con 0x558aeb4e2040 2023-10-16T11:27:30.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.126+0000 7fc9be037640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4cb170 0x558aeb4e1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.126+0000 7fc9be037640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4cb170 0x558aeb4e1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42510/0 (socket says 172.21.15.84:42510) 2023-10-16T11:27:30.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.126+0000 7fc9be037640 1 -- 172.21.15.84:0/1803423033 learned_addr learned my addr 172.21.15.84:0/1803423033 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:30.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.127+0000 7fc9be037640 1 -- 172.21.15.84:0/1803423033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558aeb4e2040 msgr2=0x558aeb4a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:30.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.127+0000 7fc9be037640 1 --2- 172.21.15.84:0/1803423033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558aeb4e2040 0x558aeb4a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.127+0000 7fc9be037640 1 -- 172.21.15.84:0/1803423033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb550 msgr2=0x558aeb4d1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:30.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.127+0000 7fc9be037640 1 --2- 172.21.15.84:0/1803423033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb550 0x558aeb4d1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.127+0000 7fc9be037640 1 -- 172.21.15.84:0/1803423033 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558aeb3a6b80 con 0x558aeb4cb170 2023-10-16T11:27:30.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.127+0000 7fc9be037640 1 --2- 172.21.15.84:0/1803423033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4cb170 0x558aeb4e1590 secure :-1 s=READY pgs=3418 cs=0 l=1 rev1=1 crypto rx=0x7fc9b402bec0 tx=0x7fc9b402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90f09f8f076bc1d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:30.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.127+0000 7fc9ace01640 1 -- 172.21.15.84:0/1803423033 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b404a040 con 0x558aeb4cb170 2023-10-16T11:27:30.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.127+0000 7fc9ace01640 1 -- 172.21.15.84:0/1803423033 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b403ac80 con 0x558aeb4cb170 2023-10-16T11:27:30.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.127+0000 7fc9ace01640 1 -- 172.21.15.84:0/1803423033 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b404f900 con 0x558aeb4cb170 2023-10-16T11:27:30.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.128+0000 7fc9c2e30740 1 -- 172.21.15.84:0/1803423033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4cb170 msgr2=0x558aeb4e1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:30.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.128+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/1803423033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4cb170 0x558aeb4e1590 secure :-1 s=READY pgs=3418 cs=0 l=1 rev1=1 crypto rx=0x7fc9b402bec0 tx=0x7fc9b402e000 comp rx=0 tx=0).stop 2023-10-16T11:27:30.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.128+0000 7fc9c2e30740 1 -- 172.21.15.84:0/1803423033 shutdown_connections 2023-10-16T11:27:30.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.128+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/1803423033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558aeb4e2040 0x558aeb4a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.128+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/1803423033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb550 0x558aeb4d1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.128+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/1803423033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4cb170 0x558aeb4e1590 unknown :-1 s=CLOSED pgs=3418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.128+0000 7fc9c2e30740 1 -- 172.21.15.84:0/1803423033 >> 172.21.15.84:0/1803423033 conn(0x558aeb49e5b0 msgr2=0x558aeb4d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.128+0000 7fc9c2e30740 1 -- 172.21.15.84:0/1803423033 shutdown_connections 2023-10-16T11:27:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.128+0000 7fc9c2e30740 1 -- 172.21.15.84:0/1803423033 wait complete. 2023-10-16T11:27:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.129+0000 7fc9c2e30740 1 -- start start 2023-10-16T11:27:30.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.129+0000 7fc9c2e30740 1 Processor -- start 2023-10-16T11:27:30.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.130+0000 7fc9c2e30740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb170 0x558aeb54bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.130+0000 7fc9c2e30740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558aeb4cb550 0x558aeb4ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.130+0000 7fc9c2e30740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4e2040 0x558aeb551be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.130+0000 7fc9c2e30740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aeb4cdd50 con 0x558aeb4e2040 2023-10-16T11:27:30.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.130+0000 7fc9c2e30740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aeb552720 con 0x558aeb4cb170 2023-10-16T11:27:30.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.130+0000 7fc9c2e30740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x558aeb4cf680 con 0x558aeb4cb550 2023-10-16T11:27:30.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.130+0000 7fc9c2e30740 10 client.0 _ll_register_callbacks cb 0x7fc9a8053cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:27:30.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.130+0000 7fc9be838640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4e2040 0x558aeb551be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.130+0000 7fc9be838640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4e2040 0x558aeb551be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42526/0 (socket says 172.21.15.84:42526) 2023-10-16T11:27:30.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.130+0000 7fc9be838640 1 -- 172.21.15.84:0/2545577148 learned_addr learned my addr 172.21.15.84:0/2545577148 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:30.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.131+0000 7fc9be037640 1 --2- 172.21.15.84:0/2545577148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb170 0x558aeb54bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.131+0000 7fc9bd836640 1 --2- 172.21.15.84:0/2545577148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558aeb4cb550 0x558aeb4ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.131+0000 7fc9be838640 1 -- 172.21.15.84:0/2545577148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558aeb4cb550 msgr2=0x558aeb4ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:30.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.131+0000 7fc9be838640 1 --2- 172.21.15.84:0/2545577148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558aeb4cb550 0x558aeb4ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.131+0000 7fc9be838640 1 -- 172.21.15.84:0/2545577148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb170 msgr2=0x558aeb54bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:30.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.131+0000 7fc9be838640 1 --2- 172.21.15.84:0/2545577148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb170 0x558aeb54bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.131+0000 7fc9be838640 1 -- 172.21.15.84:0/2545577148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b80cdbe0 con 0x558aeb4e2040 2023-10-16T11:27:30.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.131+0000 7fc9be037640 1 --2- 172.21.15.84:0/2545577148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb170 0x558aeb54bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:30.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.131+0000 7fc9be838640 1 --2- 172.21.15.84:0/2545577148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4e2040 0x558aeb551be0 secure :-1 s=READY pgs=3419 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0018040 tx=0x7fc9b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eacd317edfa80eb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:30.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.131+0000 7fc9997fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-16T11:27:30.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.131+0000 7fc9997fa640 1 -- 172.21.15.84:0/2545577148 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b000e070 con 0x558aeb4e2040 2023-10-16T11:27:30.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.131+0000 7fc9997fa640 1 -- 172.21.15.84:0/2545577148 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(270..270 src has 1..270) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc9b000f070 con 0x558aeb4e2040 2023-10-16T11:27:30.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.131+0000 7fc9c2e30740 1 -- 172.21.15.84:0/2545577148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9b0015600 con 0x558aeb4e2040 2023-10-16T11:27:30.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.132+0000 7fc9997fa640 1 -- 172.21.15.84:0/2545577148 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=157) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9b0009c20 con 0x558aeb4e2040 2023-10-16T11:27:30.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.132+0000 7fc9c2e30740 10 client.13617 fetch_fsmap learned FSMap version 157 2023-10-16T11:27:30.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.132+0000 7fc9c2e30740 1 -- 172.21.15.84:0/2545577148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=157,monmap=2+}) v3 -- 0x558aeb3cc170 con 0x558aeb4e2040 2023-10-16T11:27:30.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.132+0000 7fc9997fa640 1 -- 172.21.15.84:0/2545577148 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b0015600 con 0x558aeb4e2040 2023-10-16T11:27:30.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.132+0000 7fc9997fa640 1 -- 172.21.15.84:0/2545577148 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 157) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc9b0014070 con 0x558aeb4e2040 2023-10-16T11:27:30.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.132+0000 7fc9c2e30740 10 client.13617 fetch_fsmap finished waiting for FSMap version 157 2023-10-16T11:27:30.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.132+0000 7fc9c2e30740 10 client.13617 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T11:27:30.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.132+0000 7fc9c2e30740 20 client.13617 populate_metadata read hostname 'smithi084' 2023-10-16T11:27:30.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.132+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/2545577148 >> [v2:172.21.15.84:6836/2806813701,v1:172.21.15.84:6837/2806813701] conn(0x558aeb4deb90 0x558aeb4c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.132+0000 7fc9c2e30740 4 client.13617 mds_command: new command op to 13482 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T11:27:30.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.132+0000 7fc9c2e30740 1 -- 172.21.15.84:0/2545577148 --> [v2:172.21.15.84:6836/2806813701,v1:172.21.15.84:6837/2806813701] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558aeb402700 con 0x558aeb4deb90 2023-10-16T11:27:30.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.132+0000 7fc9be037640 1 --2- 172.21.15.84:0/2545577148 >> [v2:172.21.15.84:6836/2806813701,v1:172.21.15.84:6837/2806813701] conn(0x558aeb4deb90 0x558aeb4c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.133+0000 7fc9be037640 1 --2- 172.21.15.84:0/2545577148 >> [v2:172.21.15.84:6836/2806813701,v1:172.21.15.84:6837/2806813701] conn(0x558aeb4deb90 0x558aeb4c7a60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=610ae0671dfb3e8f server_cookie=c768851220520a3c in_seq=0 out_seq=0 2023-10-16T11:27:30.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.133+0000 7fc9997fa640 10 client.13617 ms_handle_connect on v2:172.21.15.84:6836/2806813701 2023-10-16T11:27:30.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.133+0000 7fc9997fa640 1 -- 172.21.15.84:0/2545577148 <== mds.0 v2:172.21.15.84:6836/2806813701 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x558aeb402700 con 0x558aeb4deb90 2023-10-16T11:27:30.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.133+0000 7fc9997fa640 10 client.13617 handle_command_reply: tid=0 2023-10-16T11:27:30.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.133+0000 7fc9c2e30740 1 client.13617 shutdown 2023-10-16T11:27:30.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.133+0000 7fc9c2e30740 1 -- 172.21.15.84:0/2545577148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4e2040 msgr2=0x558aeb551be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:30.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.133+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/2545577148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4e2040 0x558aeb551be0 secure :-1 s=READY pgs=3419 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0018040 tx=0x7fc9b0004a40 comp rx=0 tx=0).stop 2023-10-16T11:27:30.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.133+0000 7fc9c2e30740 1 -- 172.21.15.84:0/2545577148 shutdown_connections 2023-10-16T11:27:30.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.133+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/2545577148 >> [v2:172.21.15.84:6836/2806813701,v1:172.21.15.84:6837/2806813701] conn(0x558aeb4deb90 0x558aeb4c7a60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.133+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/2545577148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558aeb4e2040 0x558aeb551be0 unknown :-1 s=CLOSED pgs=3419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.133+0000 7fc9997fa640 0 client.13617 ms_handle_reset on v2:172.21.15.84:6836/2806813701 2023-10-16T11:27:30.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.133+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/2545577148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558aeb4cb550 0x558aeb4ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.133+0000 7fc9c2e30740 1 --2- 172.21.15.84:0/2545577148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558aeb4cb170 0x558aeb54bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.133+0000 7fc9c2e30740 1 -- 172.21.15.84:0/2545577148 >> 172.21.15.84:0/2545577148 conn(0x558aeb49e5b0 msgr2=0x558aeb4d0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:30.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.134+0000 7fc9c2e30740 1 -- 172.21.15.84:0/2545577148 shutdown_connections 2023-10-16T11:27:30.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.134+0000 7fc9c2e30740 1 -- 172.21.15.84:0/2545577148 wait complete. 2023-10-16T11:27:30.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.134+0000 7fc9c2e30740 20 client.13617 trim_cache size 0 max 16384 2023-10-16T11:27:30.149 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:27:30.149 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "63fb4d68-f881-4694-b880-d10369105c0e", 2023-10-16T11:27:30.150 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-16T11:27:30.150 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 13482, 2023-10-16T11:27:30.150 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-16T11:27:30.150 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-16T11:27:30.151 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-16T11:27:30.151 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 6.6739112489999997, 2023-10-16T11:27:30.151 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 157, 2023-10-16T11:27:30.151 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 270, 2023-10-16T11:27:30.152 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 270, 2023-10-16T11:27:30.152 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 12.111718313000001 2023-10-16T11:27:30.152 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:27:30.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.134+0000 7fc9bf039640 1 -- 172.21.15.84:0/283324763 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fc99c06afc0 msgr2=0x7fc99c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:30.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.134+0000 7fc9bf039640 1 --2- 172.21.15.84:0/283324763 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fc99c06afc0 0x7fc99c06d480 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4036000 tx=0x7fc9b403c040 comp rx=0 tx=0).stop 2023-10-16T11:27:30.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.134+0000 7fc9bf039640 1 -- 172.21.15.84:0/283324763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9b815c9f0 msgr2=0x7fc9b814e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:30.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.134+0000 7fc9bf039640 1 --2- 172.21.15.84:0/283324763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9b815c9f0 0x7fc9b814e790 secure :-1 s=READY pgs=3415 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800c400 tx=0x7fc9a800c8c0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.134+0000 7fc9bf039640 1 -- 172.21.15.84:0/283324763 shutdown_connections 2023-10-16T11:27:30.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.134+0000 7fc9bf039640 1 --2- 172.21.15.84:0/283324763 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fc99c06afc0 0x7fc99c06d480 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.134+0000 7fc9bf039640 1 --2- 172.21.15.84:0/283324763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9b815dde0 0x7fc9b806c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.134+0000 7fc9bf039640 1 --2- 172.21.15.84:0/283324763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9b815c9f0 0x7fc9b814e790 unknown :-1 s=CLOSED pgs=3415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.134+0000 7fc9bf039640 1 --2- 172.21.15.84:0/283324763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b80c9230 0x7fc9b814e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.134+0000 7fc9bf039640 1 -- 172.21.15.84:0/283324763 >> 172.21.15.84:0/283324763 conn(0x7fc9b80ce6a0 msgr2=0x7fc9b80ca530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:30.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.135+0000 7fc9bf039640 1 -- 172.21.15.84:0/283324763 shutdown_connections 2023-10-16T11:27:30.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.135+0000 7fc9bf039640 1 -- 172.21.15.84:0/283324763 wait complete. 2023-10-16T11:27:30.156 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T11:27:30.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.353+0000 7f95712cf640 1 Processor -- start 2023-10-16T11:27:30.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.354+0000 7f95712cf640 1 -- start start 2023-10-16T11:27:30.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.355+0000 7f95712cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f956c15aaa0 0x7f956c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.355+0000 7f95712cf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c15ae80 0x7f956c0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.355+0000 7f95712cf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f956c15ed60 con 0x7f956c0ccaa0 2023-10-16T11:27:30.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.355+0000 7f95712cf640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f956c06b810 con 0x7f956c15aaa0 2023-10-16T11:27:30.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.355+0000 7f95712cf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956c0707e0 con 0x7f956c15ae80 2023-10-16T11:27:30.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.355+0000 7f956affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f956c15aaa0 0x7f956c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.356+0000 7f956b7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f956c0ccaa0 0x7f956c070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43348/0 (socket says 172.21.15.84:43348) 2023-10-16T11:27:30.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.356+0000 7f956b7fe640 1 -- 172.21.15.84:0/1412208262 learned_addr learned my addr 172.21.15.84:0/1412208262 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:30.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.356+0000 7f956a7fc640 1 --2- 172.21.15.84:0/1412208262 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c15ae80 0x7f956c0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.356+0000 7f956affd640 1 -- 172.21.15.84:0/1412208262 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c15ae80 msgr2=0x7f956c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:30.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.356+0000 7f956affd640 1 --2- 172.21.15.84:0/1412208262 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c15ae80 0x7f956c0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.356+0000 7f956affd640 1 -- 172.21.15.84:0/1412208262 >> v1:172.21.15.84:6789/0 conn(0x7f956c0ccaa0 legacy=0x7f956c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:30.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.356+0000 7f9569ffb640 1 -- 172.21.15.84:0/1412208262 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3180589834 0 0) 0x7f956c15ed60 con 0x7f956c0ccaa0 2023-10-16T11:27:30.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.356+0000 7f956affd640 1 -- 172.21.15.84:0/1412208262 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956c15ef90 con 0x7f956c15aaa0 2023-10-16T11:27:30.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.356+0000 7f956a7fc640 1 --2- 172.21.15.84:0/1412208262 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c15ae80 0x7f956c0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:30.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.357+0000 7f956affd640 1 --2- 172.21.15.84:0/1412208262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f956c15aaa0 0x7f956c15e820 secure :-1 s=READY pgs=3421 cs=0 l=1 rev1=1 crypto rx=0x7f955c009d90 tx=0x7f955c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb4a53af0c5a743 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:30.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.357+0000 7f9569ffb640 1 -- 172.21.15.84:0/1412208262 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c02cc50 con 0x7f956c15aaa0 2023-10-16T11:27:30.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.357+0000 7f9569ffb640 1 -- 172.21.15.84:0/1412208262 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955c02cdb0 con 0x7f956c15aaa0 2023-10-16T11:27:30.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.357+0000 7f9569ffb640 1 -- 172.21.15.84:0/1412208262 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c034660 con 0x7f956c15aaa0 2023-10-16T11:27:30.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.358+0000 7f95712cf640 1 -- 172.21.15.84:0/1412208262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f956c15aaa0 msgr2=0x7f956c15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:30.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.358+0000 7f95712cf640 1 --2- 172.21.15.84:0/1412208262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f956c15aaa0 0x7f956c15e820 secure :-1 s=READY pgs=3421 cs=0 l=1 rev1=1 crypto rx=0x7f955c009d90 tx=0x7f955c02bea0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.358+0000 7f95712cf640 1 -- 172.21.15.84:0/1412208262 shutdown_connections 2023-10-16T11:27:30.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.358+0000 7f95712cf640 1 --2- 172.21.15.84:0/1412208262 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c15ae80 0x7f956c0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.358+0000 7f95712cf640 1 --2- 172.21.15.84:0/1412208262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f956c15aaa0 0x7f956c15e820 unknown :-1 s=CLOSED pgs=3421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.358+0000 7f95712cf640 1 -- 172.21.15.84:0/1412208262 >> 172.21.15.84:0/1412208262 conn(0x7f956c0cb410 msgr2=0x7f956c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:30.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.358+0000 7f95712cf640 1 -- 172.21.15.84:0/1412208262 shutdown_connections 2023-10-16T11:27:30.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.358+0000 7f95712cf640 1 -- 172.21.15.84:0/1412208262 wait complete. 2023-10-16T11:27:30.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.360+0000 7f95712cf640 1 Processor -- start 2023-10-16T11:27:30.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.360+0000 7f95712cf640 1 -- start start 2023-10-16T11:27:30.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.361+0000 7f95712cf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f956c0ccaa0 0x7f956c15a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.361+0000 7f95712cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f956c15aaa0 0x7f956c0c3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.361+0000 7f95712cf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c15ae80 0x7f956c0c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.361+0000 7f95712cf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956c0c6720 con 0x7f956c15aaa0 2023-10-16T11:27:30.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.361+0000 7f95712cf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956c0c6a00 con 0x7f956c0ccaa0 2023-10-16T11:27:30.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.361+0000 7f95712cf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956c0c6ce0 con 0x7f956c15ae80 2023-10-16T11:27:30.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.361+0000 7f956a7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f956c15aaa0 0x7f956c0c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.361+0000 7f956a7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f956c15aaa0 0x7f956c0c3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42558/0 (socket says 172.21.15.84:42558) 2023-10-16T11:27:30.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.361+0000 7f956a7fc640 1 -- 172.21.15.84:0/1129062391 learned_addr learned my addr 172.21.15.84:0/1129062391 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:30.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.362+0000 7f956affd640 1 --2- 172.21.15.84:0/1129062391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f956c0ccaa0 0x7f956c15a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.362+0000 7f956a7fc640 1 -- 172.21.15.84:0/1129062391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c15ae80 msgr2=0x7f956c0c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:30.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.362+0000 7f956a7fc640 1 --2- 172.21.15.84:0/1129062391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c15ae80 0x7f956c0c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.362+0000 7f956a7fc640 1 -- 172.21.15.84:0/1129062391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f956c0ccaa0 msgr2=0x7f956c15a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:30.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.362+0000 7f956a7fc640 1 --2- 172.21.15.84:0/1129062391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f956c0ccaa0 0x7f956c15a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.362+0000 7f956a7fc640 1 -- 172.21.15.84:0/1129062391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c009a70 con 0x7f956c15aaa0 2023-10-16T11:27:30.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.362+0000 7f956a7fc640 1 --2- 172.21.15.84:0/1129062391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f956c15aaa0 0x7f956c0c3970 secure :-1 s=READY pgs=3422 cs=0 l=1 rev1=1 crypto rx=0x7f955400c7f0 tx=0x7f955400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:30.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.362+0000 7f95637fe640 1 -- 172.21.15.84:0/1129062391 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554007a30 con 0x7f956c15aaa0 2023-10-16T11:27:30.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.362+0000 7f95637fe640 1 -- 172.21.15.84:0/1129062391 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9554007b90 con 0x7f956c15aaa0 2023-10-16T11:27:30.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.362+0000 7f95637fe640 1 -- 172.21.15.84:0/1129062391 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955400ce40 con 0x7f956c15aaa0 2023-10-16T11:27:30.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.362+0000 7f95712cf640 1 -- 172.21.15.84:0/1129062391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f956c15c730 con 0x7f956c15aaa0 2023-10-16T11:27:30.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.363+0000 7f95712cf640 1 -- 172.21.15.84:0/1129062391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f956c15a5b0 con 0x7f956c15aaa0 2023-10-16T11:27:30.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.364+0000 7f95712cf640 1 -- 172.21.15.84:0/1129062391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f956c15c950 con 0x7f956c15aaa0 2023-10-16T11:27:30.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.365+0000 7f95637fe640 1 -- 172.21.15.84:0/1129062391 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f95540113f0 con 0x7f956c15aaa0 2023-10-16T11:27:30.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.366+0000 7f95637fe640 1 --2- 172.21.15.84:0/1129062391 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f954c06afc0 0x7f954c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:30.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.366+0000 7f95637fe640 1 -- 172.21.15.84:0/1129062391 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9554089980 con 0x7f956c15aaa0 2023-10-16T11:27:30.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.366+0000 7f956affd640 1 --2- 172.21.15.84:0/1129062391 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f954c06afc0 0x7f954c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:30.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.367+0000 7f956affd640 1 --2- 172.21.15.84:0/1129062391 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f954c06afc0 0x7f954c06d480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f955c036000 tx=0x7f955c030000 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:30.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.371+0000 7f95637fe640 1 -- 172.21.15.84:0/1129062391 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9554054880 con 0x7f956c15aaa0 2023-10-16T11:27:30.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.608+0000 7f95712cf640 1 -- 172.21.15.84:0/1129062391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f956c04e160 con 0x7f956c15aaa0 2023-10-16T11:27:30.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.609+0000 7f95637fe640 1 -- 172.21.15.84:0/1129062391 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v157) v1 ==== 53+0+73 (secure 0 0 0) 0x7f955401f070 con 0x7f956c15aaa0 2023-10-16T11:27:30.611 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T11:27:30.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.611+0000 7f95712cf640 1 -- 172.21.15.84:0/1129062391 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f954c06afc0 msgr2=0x7f954c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:30.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.611+0000 7f95712cf640 1 --2- 172.21.15.84:0/1129062391 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f954c06afc0 0x7f954c06d480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f955c036000 tx=0x7f955c030000 comp rx=0 tx=0).stop 2023-10-16T11:27:30.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.611+0000 7f95712cf640 1 -- 172.21.15.84:0/1129062391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f956c15aaa0 msgr2=0x7f956c0c3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:30.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.611+0000 7f95712cf640 1 --2- 172.21.15.84:0/1129062391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f956c15aaa0 0x7f956c0c3970 secure :-1 s=READY pgs=3422 cs=0 l=1 rev1=1 crypto rx=0x7f955400c7f0 tx=0x7f955400cbf0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.611+0000 7f95712cf640 1 -- 172.21.15.84:0/1129062391 shutdown_connections 2023-10-16T11:27:30.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.611+0000 7f95712cf640 1 --2- 172.21.15.84:0/1129062391 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f954c06afc0 0x7f954c06d480 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.612+0000 7f95712cf640 1 --2- 172.21.15.84:0/1129062391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f956c15ae80 0x7f956c0c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.612+0000 7f95712cf640 1 --2- 172.21.15.84:0/1129062391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f956c15aaa0 0x7f956c0c3970 unknown :-1 s=CLOSED pgs=3422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.612+0000 7f95712cf640 1 --2- 172.21.15.84:0/1129062391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f956c0ccaa0 0x7f956c15a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:30.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.612+0000 7f95712cf640 1 -- 172.21.15.84:0/1129062391 >> 172.21.15.84:0/1129062391 conn(0x7f956c0cb410 msgr2=0x7f956c0b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:30.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.612+0000 7f95712cf640 1 -- 172.21.15.84:0/1129062391 shutdown_connections 2023-10-16T11:27:30.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:30.612+0000 7f95712cf640 1 -- 172.21.15.84:0/1129062391 wait complete. 2023-10-16T11:27:30.632 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T11:27:30.633 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T11:27:30.633 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-16T11:27:30.634 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T11:27:30.634 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T11:27:30.634 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T11:27:30.635 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T11:27:30.635 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T11:27:30.635 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:27:30.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:30.636 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-16T11:27:30.689 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T11:27:30.690 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T11:27:30.691 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T11:27:30.691 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:27:30.692 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-16T11:27:30.692 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:27:30.692 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:27:30.692 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:27:30.692 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-16T11:27:30.692 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:27:30.693 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:27:30.693 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-16T11:27:30.693 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:27:30.693 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:27:30.693 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T11:27:30.694 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:27:30.694 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T11:27:30.694 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 36500sec preferred_lft 36500sec 2023-10-16T11:27:30.694 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-16T11:27:30.694 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:27:30.695 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T11:27:30.695 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:27:30.695 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T11:27:30.695 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:27:30.695 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::20a8:eaff:fe9d:5cea/64 scope link 2023-10-16T11:27:30.696 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:27:30.696 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:27:30.696 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 1a:54:13:47:be:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T11:27:30.696 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1854:13ff:fe47:be92/64 scope link 2023-10-16T11:27:30.696 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:27:30.697 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:27:30.697 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether ee:d5:41:a3:a5:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T11:27:30.697 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ecd5:41ff:fea3:a570/64 scope link 2023-10-16T11:27:30.697 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:27:30.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:30.698 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T11:27:30.788 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-16T11:27:30.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:30.793 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-16T11:27:30.847 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-16T11:27:30.847 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T11:27:30.848 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T11:27:30.848 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T11:27:30.848 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T11:27:30.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:30.850 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-16T11:27:30.850 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-16T11:27:30.850 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T11:27:30.851 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T11:27:30.851 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T11:27:30.851 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-16T11:27:31.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:31.038 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T11:27:31.088 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T11:27:31.089 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T11:27:31.089 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:27:31.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:31.090 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-16T11:27:31.187 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:27:31.188 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T11:27:31.247 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:27:31.248 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:27:31.248 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:27:31.299 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-16T11:27:31.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:31.300 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:27:31.434 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:27:31.434 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T11:27:31.473 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-16T11:27:31.473 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-16T11:27:31.473 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.157:3300,172.21.15.157:3301" 2023-10-16T11:27:31.474 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with new device syntax: 0@63fb4d68-f881-4694-b880-d10369105c0e.cephfs=/ 2023-10-16T11:27:31.474 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.84:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2023-10-16T11:27:31.475 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-16T11:27:31.475 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2023-10-16T11:27:31.475 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-16T11:27:31.476 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2023-10-16T11:27:31.476 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-16T11:27:31.482 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T11:27:31.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:31.483 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:27:31.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:31.570 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T11:27:31.632 INFO:teuthology.orchestra.run.smithi084.stdout:client8488 2023-10-16T11:27:31.633 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:27:31.633 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:31.683 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T11:27:31.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.941+0000 7f7a6759e640 1 Processor -- start 2023-10-16T11:27:31.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.942+0000 7f7a6759e640 1 -- start start 2023-10-16T11:27:31.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.943+0000 7f7a6759e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a6815cd70 0x7f7a680d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:31.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.943+0000 7f7a6759e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7a680cded0 0x7f7a680b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:31.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.943+0000 7f7a6759e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6815d150 con 0x7f7a680b8480 2023-10-16T11:27:31.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.943+0000 7f7a6759e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a680d1180 con 0x7f7a680cded0 2023-10-16T11:27:31.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.943+0000 7f7a6759e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680d1460 con 0x7f7a6815cd70 2023-10-16T11:27:31.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.944+0000 7f7a65d9b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7a680b8480 0x7f7a680cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43352/0 (socket says 172.21.15.84:43352) 2023-10-16T11:27:31.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.944+0000 7f7a65d9b640 1 -- 172.21.15.84:0/664733100 learned_addr learned my addr 172.21.15.84:0/664733100 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:31.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.944+0000 7f7a6659c640 1 --2- 172.21.15.84:0/664733100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a6815cd70 0x7f7a680d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:31.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.944+0000 7f7a6559a640 1 -- 172.21.15.84:0/664733100 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540934933 0 0) 0x7f7a6815d150 con 0x7f7a680b8480 2023-10-16T11:27:31.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.944+0000 7f7a6559a640 1 -- 172.21.15.84:0/664733100 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a54003520 con 0x7f7a680b8480 2023-10-16T11:27:31.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.944+0000 7f7a66d9d640 1 --2- 172.21.15.84:0/664733100 >> v2:172.21.15.157:3300/0 conn(0x7f7a680cded0 0x7f7a680b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:31.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.944+0000 7f7a6559a640 1 -- 172.21.15.84:0/664733100 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2576090782 0 0) 0x7f7a54003520 con 0x7f7a680b8480 2023-10-16T11:27:31.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.944+0000 7f7a6559a640 1 -- 172.21.15.84:0/664733100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a6815cd70 msgr2=0x7f7a680d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:31.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.944+0000 7f7a6559a640 1 --2- 172.21.15.84:0/664733100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a6815cd70 0x7f7a680d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:31.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.944+0000 7f7a6559a640 1 -- 172.21.15.84:0/664733100 >> v2:172.21.15.157:3300/0 conn(0x7f7a680cded0 msgr2=0x7f7a680b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:31.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.944+0000 7f7a6559a640 1 --2- 172.21.15.84:0/664733100 >> v2:172.21.15.157:3300/0 conn(0x7f7a680cded0 0x7f7a680b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:31.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.944+0000 7f7a6559a640 1 -- 172.21.15.84:0/664733100 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a680d1740 con 0x7f7a680b8480 2023-10-16T11:27:31.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.944+0000 7f7a6559a640 1 -- 172.21.15.84:0/664733100 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f7a50001670 con 0x7f7a680b8480 2023-10-16T11:27:31.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.944+0000 7f7a6559a640 1 -- 172.21.15.84:0/664733100 >> v1:172.21.15.84:6789/0 conn(0x7f7a680b8480 legacy=0x7f7a680cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:31.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.944+0000 7f7a6559a640 1 --2- 172.21.15.84:0/664733100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a54003970 0x7f7a54003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:31.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.945+0000 7f7a6559a640 1 --2- 172.21.15.84:0/664733100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a540047c0 0x7f7a54004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:31.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.945+0000 7f7a6559a640 1 --2- 172.21.15.84:0/664733100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a6815cd70 0x7f7a54005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:31.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.945+0000 7f7a6559a640 1 -- 172.21.15.84:0/664733100 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680d1180 con 0x7f7a540047c0 2023-10-16T11:27:31.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.945+0000 7f7a6559a640 1 -- 172.21.15.84:0/664733100 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680d1460 con 0x7f7a54003970 2023-10-16T11:27:31.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.945+0000 7f7a6559a640 1 -- 172.21.15.84:0/664733100 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5400b9d0 con 0x7f7a6815cd70 2023-10-16T11:27:31.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.945+0000 7f7a65d9b640 1 --2- 172.21.15.84:0/664733100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a540047c0 0x7f7a54004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:31.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.945+0000 7f7a6659c640 1 --2- 172.21.15.84:0/664733100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a54003970 0x7f7a54003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:31.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.945+0000 7f7a66d9d640 1 --2- 172.21.15.84:0/664733100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a6815cd70 0x7f7a54005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:31.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.945+0000 7f7a65d9b640 1 -- 172.21.15.84:0/664733100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a6815cd70 msgr2=0x7f7a54005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:31.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.945+0000 7f7a65d9b640 1 --2- 172.21.15.84:0/664733100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a6815cd70 0x7f7a54005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:31.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.945+0000 7f7a65d9b640 1 -- 172.21.15.84:0/664733100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a54003970 msgr2=0x7f7a54003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:31.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.945+0000 7f7a65d9b640 1 --2- 172.21.15.84:0/664733100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a54003970 0x7f7a54003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:31.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.945+0000 7f7a65d9b640 1 -- 172.21.15.84:0/664733100 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a54003520 con 0x7f7a540047c0 2023-10-16T11:27:31.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.946+0000 7f7a65d9b640 1 --2- 172.21.15.84:0/664733100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a540047c0 0x7f7a54004c40 secure :-1 s=READY pgs=3424 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000c160 tx=0x7f7a540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577bd2a826987e8b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:31.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.947+0000 7f7a6559a640 1 -- 172.21.15.84:0/664733100 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50005ad0 con 0x7f7a540047c0 2023-10-16T11:27:31.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.947+0000 7f7a6559a640 1 -- 172.21.15.84:0/664733100 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a50005c30 con 0x7f7a540047c0 2023-10-16T11:27:31.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.947+0000 7f7a6559a640 1 -- 172.21.15.84:0/664733100 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5002c810 con 0x7f7a540047c0 2023-10-16T11:27:31.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.947+0000 7f7a6759e640 1 -- 172.21.15.84:0/664733100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a540047c0 msgr2=0x7f7a54004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:31.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.947+0000 7f7a6759e640 1 --2- 172.21.15.84:0/664733100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a540047c0 0x7f7a54004c40 secure :-1 s=READY pgs=3424 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000c160 tx=0x7f7a540057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:31.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.947+0000 7f7a66d9d640 1 -- 172.21.15.84:0/664733100 reap_dead start 2023-10-16T11:27:31.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.947+0000 7f7a6759e640 1 -- 172.21.15.84:0/664733100 shutdown_connections 2023-10-16T11:27:31.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.947+0000 7f7a6759e640 1 -- 172.21.15.84:0/664733100 >> 172.21.15.84:0/664733100 conn(0x7f7a680cf570 msgr2=0x7f7a680cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:31.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.947+0000 7f7a6759e640 1 -- 172.21.15.84:0/664733100 shutdown_connections 2023-10-16T11:27:31.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.947+0000 7f7a6759e640 1 -- 172.21.15.84:0/664733100 wait complete. 2023-10-16T11:27:31.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.949+0000 7f7a6759e640 1 Processor -- start 2023-10-16T11:27:31.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.949+0000 7f7a6759e640 1 -- start start 2023-10-16T11:27:31.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.950+0000 7f7a6759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a540047c0 0x7f7a680ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:31.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.950+0000 7f7a6759e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a680cbcc0 0x7f7a680b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:31.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.950+0000 7f7a6759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a6814fed0 0x7f7a680b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:31.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.950+0000 7f7a6759e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680b1790 con 0x7f7a6814fed0 2023-10-16T11:27:31.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.950+0000 7f7a6759e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680cac20 con 0x7f7a540047c0 2023-10-16T11:27:31.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.950+0000 7f7a6759e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680caf00 con 0x7f7a680cbcc0 2023-10-16T11:27:31.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.950+0000 7f7a66d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a6814fed0 0x7f7a680b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:31.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.951+0000 7f7a66d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a6814fed0 0x7f7a680b3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42578/0 (socket says 172.21.15.84:42578) 2023-10-16T11:27:31.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.951+0000 7f7a66d9d640 1 -- 172.21.15.84:0/3576787155 learned_addr learned my addr 172.21.15.84:0/3576787155 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:31.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.951+0000 7f7a6659c640 1 --2- 172.21.15.84:0/3576787155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a540047c0 0x7f7a680ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:31.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.951+0000 7f7a65d9b640 1 --2- 172.21.15.84:0/3576787155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a680cbcc0 0x7f7a680b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:31.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.951+0000 7f7a66d9d640 1 -- 172.21.15.84:0/3576787155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a680cbcc0 msgr2=0x7f7a680b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:31.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.951+0000 7f7a66d9d640 1 --2- 172.21.15.84:0/3576787155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a680cbcc0 0x7f7a680b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:31.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.951+0000 7f7a66d9d640 1 -- 172.21.15.84:0/3576787155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a540047c0 msgr2=0x7f7a680ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:31.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.951+0000 7f7a66d9d640 1 --2- 172.21.15.84:0/3576787155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a540047c0 0x7f7a680ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:31.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.951+0000 7f7a66d9d640 1 -- 172.21.15.84:0/3576787155 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a680cb1e0 con 0x7f7a6814fed0 2023-10-16T11:27:31.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.951+0000 7f7a66d9d640 1 --2- 172.21.15.84:0/3576787155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a6814fed0 0x7f7a680b3970 secure :-1 s=READY pgs=3425 cs=0 l=1 rev1=1 crypto rx=0x7f7a680b56a0 tx=0x7f7a6000e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:32.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.951+0000 7f7a5effd640 1 -- 172.21.15.84:0/3576787155 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6000c070 con 0x7f7a6814fed0 2023-10-16T11:27:32.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.952+0000 7f7a5effd640 1 -- 172.21.15.84:0/3576787155 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a60004020 con 0x7f7a6814fed0 2023-10-16T11:27:32.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.952+0000 7f7a5effd640 1 -- 172.21.15.84:0/3576787155 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60004410 con 0x7f7a6814fed0 2023-10-16T11:27:32.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.952+0000 7f7a6759e640 1 -- 172.21.15.84:0/3576787155 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a680cb4c0 con 0x7f7a6814fed0 2023-10-16T11:27:32.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.952+0000 7f7a6759e640 1 -- 172.21.15.84:0/3576787155 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a68150350 con 0x7f7a6814fed0 2023-10-16T11:27:32.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.953+0000 7f7a6759e640 1 -- 172.21.15.84:0/3576787155 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6804e9b0 con 0x7f7a6814fed0 2023-10-16T11:27:32.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.954+0000 7f7a5effd640 1 -- 172.21.15.84:0/3576787155 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7a60003610 con 0x7f7a6814fed0 2023-10-16T11:27:32.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.955+0000 7f7a5effd640 1 --2- 172.21.15.84:0/3576787155 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f7a4806af90 0x7f7a4806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:32.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.955+0000 7f7a5effd640 1 -- 172.21.15.84:0/3576787155 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f7a60088bc0 con 0x7f7a6814fed0 2023-10-16T11:27:32.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.955+0000 7f7a6659c640 1 --2- 172.21.15.84:0/3576787155 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f7a4806af90 0x7f7a4806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:32.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.955+0000 7f7a6659c640 1 --2- 172.21.15.84:0/3576787155 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f7a4806af90 0x7f7a4806d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a58007440 tx=0x7f7a5800b040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:32.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:31.959+0000 7f7a5effd640 1 -- 172.21.15.84:0/3576787155 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7a60053990 con 0x7f7a6814fed0 2023-10-16T11:27:32.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.176+0000 7f7a6759e640 1 -- 172.21.15.84:0/3576787155 --> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a680d1d00 con 0x7f7a4806af90 2023-10-16T11:27:32.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.179+0000 7f7a5effd640 1 -- 172.21.15.84:0/3576787155 <== mgr.13239 v2:172.21.15.84:6832/731888425 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f7a58003990 con 0x7f7a4806af90 2023-10-16T11:27:32.181 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-16T11:27:32.181 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-16T11:27:32.182 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephfs" 2023-10-16T11:27:32.182 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-16T11:27:32.182 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-16T11:27:32.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.182+0000 7f7a6759e640 1 -- 172.21.15.84:0/3576787155 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f7a4806af90 msgr2=0x7f7a4806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:32.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.182+0000 7f7a6759e640 1 --2- 172.21.15.84:0/3576787155 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f7a4806af90 0x7f7a4806d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a58007440 tx=0x7f7a5800b040 comp rx=0 tx=0).stop 2023-10-16T11:27:32.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.182+0000 7f7a6759e640 1 -- 172.21.15.84:0/3576787155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a6814fed0 msgr2=0x7f7a680b3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:32.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.182+0000 7f7a6759e640 1 --2- 172.21.15.84:0/3576787155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a6814fed0 0x7f7a680b3970 secure :-1 s=READY pgs=3425 cs=0 l=1 rev1=1 crypto rx=0x7f7a680b56a0 tx=0x7f7a6000e830 comp rx=0 tx=0).stop 2023-10-16T11:27:32.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.182+0000 7f7a6759e640 1 -- 172.21.15.84:0/3576787155 shutdown_connections 2023-10-16T11:27:32.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.182+0000 7f7a6759e640 1 --2- 172.21.15.84:0/3576787155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a6814fed0 0x7f7a680b3970 unknown :-1 s=CLOSED pgs=3425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:32.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.182+0000 7f7a6759e640 1 --2- 172.21.15.84:0/3576787155 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f7a4806af90 0x7f7a4806d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:32.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.182+0000 7f7a6759e640 1 --2- 172.21.15.84:0/3576787155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a680cbcc0 0x7f7a680b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:32.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.182+0000 7f7a6759e640 1 --2- 172.21.15.84:0/3576787155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a540047c0 0x7f7a680ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:32.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.182+0000 7f7a6759e640 1 -- 172.21.15.84:0/3576787155 >> 172.21.15.84:0/3576787155 conn(0x7f7a680cf570 msgr2=0x7f7a680cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:32.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.182+0000 7f7a6759e640 1 -- 172.21.15.84:0/3576787155 shutdown_connections 2023-10-16T11:27:32.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.182+0000 7f7a6759e640 1 -- 172.21.15.84:0/3576787155 wait complete. 2023-10-16T11:27:32.201 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T11:27:32.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.415+0000 7f89aacb4640 1 Processor -- start 2023-10-16T11:27:32.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.415+0000 7f89aacb4640 1 -- start start 2023-10-16T11:27:32.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.417+0000 7f89aacb4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f89a4159ab0 0x7f89a40ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:32.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.417+0000 7f89aacb4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a4159e90 0x7f89a406fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:32.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.417+0000 7f89aacb4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a40cd1f0 con 0x7f89a406c140 2023-10-16T11:27:32.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.417+0000 7f89aacb4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a415d890 con 0x7f89a4159ab0 2023-10-16T11:27:32.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.417+0000 7f89aacb4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a40cd4d0 con 0x7f89a4159e90 2023-10-16T11:27:32.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.417+0000 7f899bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a4159e90 0x7f89a406fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:32.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.417+0000 7f89a8a50640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f89a4159ab0 0x7f89a40ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:32.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.418+0000 7f89a8a50640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f89a4159ab0 0x7f89a40ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:41834/0 (socket says 172.21.15.84:41834) 2023-10-16T11:27:32.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.418+0000 7f89a8a50640 1 -- 172.21.15.84:0/1234739248 learned_addr learned my addr 172.21.15.84:0/1234739248 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:32.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.418+0000 7f89a9251640 1 --1- 172.21.15.84:0/1234739248 >> v1:172.21.15.157:6789/0 conn(0x7f89a406c140 0x7f89a415dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:50964/0 (socket says 172.21.15.84:50964) 2023-10-16T11:27:32.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.418+0000 7f89a8a50640 1 -- 172.21.15.84:0/1234739248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a4159e90 msgr2=0x7f89a406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:32.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.418+0000 7f89a8a50640 1 --2- 172.21.15.84:0/1234739248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a4159e90 0x7f89a406fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:32.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.418+0000 7f89a8a50640 1 -- 172.21.15.84:0/1234739248 >> v1:172.21.15.157:6789/0 conn(0x7f89a406c140 legacy=0x7f89a415dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:32.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.418+0000 7f89a8a50640 1 -- 172.21.15.84:0/1234739248 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a40cd7b0 con 0x7f89a4159ab0 2023-10-16T11:27:32.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.420+0000 7f89a8a50640 1 --2- 172.21.15.84:0/1234739248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a4159ab0 0x7f89a40ce370 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f89a00098a0 tx=0x7f89a002b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59d6c9dfe1655d80 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:32.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.420+0000 7f899b7fe640 1 -- 172.21.15.84:0/1234739248 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0032640 con 0x7f89a4159ab0 2023-10-16T11:27:32.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.421+0000 7f899b7fe640 1 -- 172.21.15.84:0/1234739248 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a002e070 con 0x7f89a4159ab0 2023-10-16T11:27:32.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.421+0000 7f899b7fe640 1 -- 172.21.15.84:0/1234739248 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a00329c0 con 0x7f89a4159ab0 2023-10-16T11:27:32.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.421+0000 7f89aacb4640 1 -- 172.21.15.84:0/1234739248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a4159ab0 msgr2=0x7f89a40ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:32.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.421+0000 7f89aacb4640 1 --2- 172.21.15.84:0/1234739248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a4159ab0 0x7f89a40ce370 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f89a00098a0 tx=0x7f89a002b850 comp rx=0 tx=0).stop 2023-10-16T11:27:32.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.421+0000 7f89aacb4640 1 -- 172.21.15.84:0/1234739248 shutdown_connections 2023-10-16T11:27:32.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.421+0000 7f89aacb4640 1 --2- 172.21.15.84:0/1234739248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a4159e90 0x7f89a406fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:32.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.421+0000 7f89aacb4640 1 --2- 172.21.15.84:0/1234739248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a4159ab0 0x7f89a40ce370 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:32.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.421+0000 7f89aacb4640 1 -- 172.21.15.84:0/1234739248 >> 172.21.15.84:0/1234739248 conn(0x7f89a40ce9b0 msgr2=0x7f89a40cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:32.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.421+0000 7f89aacb4640 1 -- 172.21.15.84:0/1234739248 shutdown_connections 2023-10-16T11:27:32.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.421+0000 7f89aacb4640 1 -- 172.21.15.84:0/1234739248 wait complete. 2023-10-16T11:27:32.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.423+0000 7f89aacb4640 1 Processor -- start 2023-10-16T11:27:32.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.423+0000 7f89aacb4640 1 -- start start 2023-10-16T11:27:32.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.424+0000 7f89aacb4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a406c140 0x7f89a40c1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:32.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.424+0000 7f89aacb4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a4159ab0 0x7f89a415af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:32.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.424+0000 7f89aacb4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89a4159e90 0x7f89a40c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:32.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.424+0000 7f89aacb4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a415d0b0 con 0x7f89a4159e90 2023-10-16T11:27:32.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.424+0000 7f89aacb4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a415c430 con 0x7f89a406c140 2023-10-16T11:27:32.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.424+0000 7f89aacb4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a415c710 con 0x7f89a4159ab0 2023-10-16T11:27:32.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.425+0000 7f89a9251640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89a4159e90 0x7f89a40c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:32.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.425+0000 7f89a9251640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89a4159e90 0x7f89a40c44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42584/0 (socket says 172.21.15.84:42584) 2023-10-16T11:27:32.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.425+0000 7f89a9251640 1 -- 172.21.15.84:0/1671302999 learned_addr learned my addr 172.21.15.84:0/1671302999 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:32.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.425+0000 7f89a8a50640 1 --2- 172.21.15.84:0/1671302999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a406c140 0x7f89a40c1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:32.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.425+0000 7f899bfff640 1 --2- 172.21.15.84:0/1671302999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a4159ab0 0x7f89a415af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:32.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.425+0000 7f89a9251640 1 -- 172.21.15.84:0/1671302999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a4159ab0 msgr2=0x7f89a415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:32.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.425+0000 7f89a9251640 1 --2- 172.21.15.84:0/1671302999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a4159ab0 0x7f89a415af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:32.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.425+0000 7f89a9251640 1 -- 172.21.15.84:0/1671302999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a406c140 msgr2=0x7f89a40c1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:32.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.425+0000 7f89a9251640 1 --2- 172.21.15.84:0/1671302999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a406c140 0x7f89a40c1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:32.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.425+0000 7f89a9251640 1 -- 172.21.15.84:0/1671302999 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a00094d0 con 0x7f89a4159e90 2023-10-16T11:27:32.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.426+0000 7f89a9251640 1 --2- 172.21.15.84:0/1671302999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89a4159e90 0x7f89a40c44f0 secure :-1 s=READY pgs=3426 cs=0 l=1 rev1=1 crypto rx=0x7f899c00caa0 tx=0x7f899c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:32.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.426+0000 7f89997fa640 1 -- 172.21.15.84:0/1671302999 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c00a6d0 con 0x7f89a4159e90 2023-10-16T11:27:32.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.426+0000 7f89997fa640 1 -- 172.21.15.84:0/1671302999 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899c00a830 con 0x7f89a4159e90 2023-10-16T11:27:32.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.426+0000 7f89997fa640 1 -- 172.21.15.84:0/1671302999 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c00aba0 con 0x7f89a4159e90 2023-10-16T11:27:32.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.426+0000 7f89aacb4640 1 -- 172.21.15.84:0/1671302999 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a415c9f0 con 0x7f89a4159e90 2023-10-16T11:27:32.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.426+0000 7f89aacb4640 1 -- 172.21.15.84:0/1671302999 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a40c21d0 con 0x7f89a4159e90 2023-10-16T11:27:32.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.427+0000 7f89aacb4640 1 -- 172.21.15.84:0/1671302999 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a415d1f0 con 0x7f89a4159e90 2023-10-16T11:27:32.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.428+0000 7f89997fa640 1 -- 172.21.15.84:0/1671302999 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 160) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f899c00ad00 con 0x7f89a4159e90 2023-10-16T11:27:32.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.429+0000 7f89997fa640 1 --2- 172.21.15.84:0/1671302999 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f899006af90 0x7f899006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:32.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.429+0000 7f89997fa640 1 -- 172.21.15.84:0/1671302999 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f899c089050 con 0x7f89a4159e90 2023-10-16T11:27:32.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.429+0000 7f89a8a50640 1 --2- 172.21.15.84:0/1671302999 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f899006af90 0x7f899006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:32.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.430+0000 7f89a8a50640 1 --2- 172.21.15.84:0/1671302999 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f899006af90 0x7f899006d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89a0036000 tx=0x7f89a003d040 comp rx=0 tx=0).ready entity=mgr.13239 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:32.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.433+0000 7f89997fa640 1 -- 172.21.15.84:0/1671302999 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f89a415d1f0 con 0x7f89a4159e90 2023-10-16T11:27:32.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.676+0000 7f89aacb4640 1 -- 172.21.15.84:0/1671302999 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f89a415d1f0 con 0x7f89a4159e90 2023-10-16T11:27:32.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.936+0000 7f89997fa640 1 -- 172.21.15.84:0/1671302999 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v161) v1 ==== 92+0+0 (secure 0 0 0) 0x7f89a415d1f0 con 0x7f89a4159e90 2023-10-16T11:27:32.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.939+0000 7f89aacb4640 1 -- 172.21.15.84:0/1671302999 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f899006af90 msgr2=0x7f899006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:32.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.939+0000 7f89aacb4640 1 --2- 172.21.15.84:0/1671302999 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f899006af90 0x7f899006d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89a0036000 tx=0x7f89a003d040 comp rx=0 tx=0).stop 2023-10-16T11:27:32.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.939+0000 7f89aacb4640 1 -- 172.21.15.84:0/1671302999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89a4159e90 msgr2=0x7f89a40c44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:32.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.939+0000 7f89aacb4640 1 --2- 172.21.15.84:0/1671302999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89a4159e90 0x7f89a40c44f0 secure :-1 s=READY pgs=3426 cs=0 l=1 rev1=1 crypto rx=0x7f899c00caa0 tx=0x7f899c00cf60 comp rx=0 tx=0).stop 2023-10-16T11:27:32.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.940+0000 7f89aacb4640 1 -- 172.21.15.84:0/1671302999 shutdown_connections 2023-10-16T11:27:32.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.940+0000 7f89aacb4640 1 --2- 172.21.15.84:0/1671302999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89a4159e90 0x7f89a40c44f0 unknown :-1 s=CLOSED pgs=3426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:32.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.940+0000 7f89aacb4640 1 --2- 172.21.15.84:0/1671302999 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f899006af90 0x7f899006d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:32.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.940+0000 7f89aacb4640 1 --2- 172.21.15.84:0/1671302999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89a4159ab0 0x7f89a415af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:32.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.940+0000 7f89aacb4640 1 --2- 172.21.15.84:0/1671302999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a406c140 0x7f89a40c1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:32.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.940+0000 7f89aacb4640 1 -- 172.21.15.84:0/1671302999 >> 172.21.15.84:0/1671302999 conn(0x7f89a40ce9b0 msgr2=0x7f89a40b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:32.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.940+0000 7f89aacb4640 1 -- 172.21.15.84:0/1671302999 shutdown_connections 2023-10-16T11:27:32.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:32.941+0000 7f89aacb4640 1 -- 172.21.15.84:0/1671302999 wait complete. 2023-10-16T11:27:32.973 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T11:27:33.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.163+0000 7f640d7b0640 1 Processor -- start 2023-10-16T11:27:33.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.163+0000 7f640d7b0640 1 -- start start 2023-10-16T11:27:33.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.164+0000 7f640d7b0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f640815d330 0x7f64081618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:33.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.164+0000 7f640d7b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64080661c0 0x7f64080ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:33.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.164+0000 7f640d7b0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64080d0890 con 0x7f640806c7d0 2023-10-16T11:27:33.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.164+0000 7f640d7b0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f640806b1e0 con 0x7f640815d330 2023-10-16T11:27:33.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.164+0000 7f640d7b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64080d05b0 con 0x7f64080661c0 2023-10-16T11:27:33.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.165+0000 7f6406ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f640815d330 0x7f64081618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:33.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.165+0000 7f6406ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f640815d330 0x7f64081618b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.84:41870/0 (socket says 172.21.15.84:41870) 2023-10-16T11:27:33.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.165+0000 7f6406ffd640 1 -- 172.21.15.84:0/4216454982 learned_addr learned my addr 172.21.15.84:0/4216454982 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:33.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.165+0000 7f64077fe640 1 --2- 172.21.15.84:0/4216454982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64080661c0 0x7f64080ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:33.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.166+0000 7f6406ffd640 1 -- 172.21.15.84:0/4216454982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64080661c0 msgr2=0x7f64080ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:33.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.166+0000 7f6406ffd640 1 --2- 172.21.15.84:0/4216454982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64080661c0 0x7f64080ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.166+0000 7f6406ffd640 1 -- 172.21.15.84:0/4216454982 >> v1:172.21.15.157:6789/0 conn(0x7f640806c7d0 legacy=0x7f6408065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:33.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.166+0000 7f6406ffd640 1 -- 172.21.15.84:0/4216454982 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6408160dd0 con 0x7f640815d330 2023-10-16T11:27:33.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.167+0000 7f6406ffd640 1 --2- 172.21.15.84:0/4216454982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f640815d330 0x7f64081618b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009870 tx=0x7f63f802b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d23ce8e9f0df66a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:33.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.167+0000 7f64067fc640 1 -- 172.21.15.84:0/4216454982 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f80317c0 con 0x7f640815d330 2023-10-16T11:27:33.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.167+0000 7f64067fc640 1 -- 172.21.15.84:0/4216454982 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f8031920 con 0x7f640815d330 2023-10-16T11:27:33.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.167+0000 7f64067fc640 1 -- 172.21.15.84:0/4216454982 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8031cb0 con 0x7f640815d330 2023-10-16T11:27:33.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.168+0000 7f640d7b0640 1 -- 172.21.15.84:0/4216454982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f640815d330 msgr2=0x7f64081618b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:33.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.168+0000 7f640d7b0640 1 --2- 172.21.15.84:0/4216454982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f640815d330 0x7f64081618b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009870 tx=0x7f63f802b860 comp rx=0 tx=0).stop 2023-10-16T11:27:33.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.168+0000 7f640d7b0640 1 -- 172.21.15.84:0/4216454982 shutdown_connections 2023-10-16T11:27:33.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.168+0000 7f640d7b0640 1 --2- 172.21.15.84:0/4216454982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64080661c0 0x7f64080ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.168+0000 7f640d7b0640 1 --2- 172.21.15.84:0/4216454982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f640815d330 0x7f64081618b0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.168+0000 7f640d7b0640 1 -- 172.21.15.84:0/4216454982 >> 172.21.15.84:0/4216454982 conn(0x7f64080d0ab0 msgr2=0x7f64080d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:33.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.168+0000 7f640d7b0640 1 -- 172.21.15.84:0/4216454982 shutdown_connections 2023-10-16T11:27:33.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.168+0000 7f640d7b0640 1 -- 172.21.15.84:0/4216454982 wait complete. 2023-10-16T11:27:33.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.170+0000 7f640d7b0640 1 Processor -- start 2023-10-16T11:27:33.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.170+0000 7f640d7b0640 1 -- start start 2023-10-16T11:27:33.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.171+0000 7f640d7b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64080661c0 0x7f64080c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:33.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.171+0000 7f640d7b0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f640806c7d0 0x7f64081f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:33.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.171+0000 7f640d7b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f640815d330 0x7f64080cec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:33.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.171+0000 7f640d7b0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64080c9fa0 con 0x7f640806c7d0 2023-10-16T11:27:33.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.171+0000 7f640d7b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64081f7010 con 0x7f640815d330 2023-10-16T11:27:33.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.171+0000 7f640d7b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6408159870 con 0x7f64080661c0 2023-10-16T11:27:33.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.171+0000 7f63fffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f640806c7d0 0x7f64081f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:33.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.171+0000 7f63fffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f640806c7d0 0x7f64081f6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42612/0 (socket says 172.21.15.84:42612) 2023-10-16T11:27:33.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.171+0000 7f63fffff640 1 -- 172.21.15.84:0/226082516 learned_addr learned my addr 172.21.15.84:0/226082516 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:33.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.171+0000 7f64077fe640 1 --2- 172.21.15.84:0/226082516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f640815d330 0x7f64080cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:33.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.171+0000 7f6406ffd640 1 --2- 172.21.15.84:0/226082516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64080661c0 0x7f64080c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:33.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.172+0000 7f63fffff640 1 -- 172.21.15.84:0/226082516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64080661c0 msgr2=0x7f64080c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:33.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.172+0000 7f63fffff640 1 --2- 172.21.15.84:0/226082516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64080661c0 0x7f64080c99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.172+0000 7f63fffff640 1 -- 172.21.15.84:0/226082516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f640815d330 msgr2=0x7f64080cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:33.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.172+0000 7f63fffff640 1 --2- 172.21.15.84:0/226082516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f640815d330 0x7f64080cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.172+0000 7f63fffff640 1 -- 172.21.15.84:0/226082516 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f80094d0 con 0x7f640806c7d0 2023-10-16T11:27:33.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.172+0000 7f63fffff640 1 --2- 172.21.15.84:0/226082516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f640806c7d0 0x7f64081f6ad0 secure :-1 s=READY pgs=3431 cs=0 l=1 rev1=1 crypto rx=0x7f63f000b3f0 tx=0x7f63f000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:33.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.172+0000 7f63ff7fe640 1 -- 172.21.15.84:0/226082516 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0011830 con 0x7f640806c7d0 2023-10-16T11:27:33.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.172+0000 7f63ff7fe640 1 -- 172.21.15.84:0/226082516 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f0011990 con 0x7f640806c7d0 2023-10-16T11:27:33.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.172+0000 7f63ff7fe640 1 -- 172.21.15.84:0/226082516 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0011d40 con 0x7f640806c7d0 2023-10-16T11:27:33.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.172+0000 7f640d7b0640 1 -- 172.21.15.84:0/226082516 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6408159b10 con 0x7f640806c7d0 2023-10-16T11:27:33.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.172+0000 7f640d7b0640 1 -- 172.21.15.84:0/226082516 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f640815a0d0 con 0x7f640806c7d0 2023-10-16T11:27:33.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.173+0000 7f640d7b0640 1 -- 172.21.15.84:0/226082516 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f640804e160 con 0x7f640806c7d0 2023-10-16T11:27:33.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.175+0000 7f63ff7fe640 1 -- 172.21.15.84:0/226082516 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 161) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f63f001a400 con 0x7f640806c7d0 2023-10-16T11:27:33.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.175+0000 7f63ff7fe640 1 --2- 172.21.15.84:0/226082516 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f63e806afe0 0x7f63e806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:33.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.176+0000 7f6406ffd640 1 -- 172.21.15.84:0/226082516 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f63e806afe0 msgr2=0x7f63e806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/731888425 2023-10-16T11:27:33.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.176+0000 7f63ff7fe640 1 -- 172.21.15.84:0/226082516 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f63f0089620 con 0x7f640806c7d0 2023-10-16T11:27:33.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.176+0000 7f6406ffd640 1 --2- 172.21.15.84:0/226082516 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f63e806afe0 0x7f63e806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:33.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.184+0000 7f63ff7fe640 1 -- 172.21.15.84:0/226082516 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f63f00545a0 con 0x7f640806c7d0 2023-10-16T11:27:33.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.371+0000 7f640d7b0640 1 -- 172.21.15.84:0/226082516 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f64080ca900 con 0x7f640806c7d0 2023-10-16T11:27:33.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.376+0000 7f63ff7fe640 1 -- 172.21.15.84:0/226082516 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v95)=0 v95) v1 ==== 138+0+0 (secure 0 0 0) 0x7f63f0057bc0 con 0x7f640806c7d0 2023-10-16T11:27:33.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.377+0000 7f6406ffd640 1 -- 172.21.15.84:0/226082516 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f63e806afe0 msgr2=0x7f63e806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/731888425 2023-10-16T11:27:33.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.377+0000 7f6406ffd640 1 --2- 172.21.15.84:0/226082516 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f63e806afe0 0x7f63e806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:27:33.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.379+0000 7f640d7b0640 1 -- 172.21.15.84:0/226082516 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f63e806afe0 msgr2=0x7f63e806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:33.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.379+0000 7f640d7b0640 1 --2- 172.21.15.84:0/226082516 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f63e806afe0 0x7f63e806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.379+0000 7f640d7b0640 1 -- 172.21.15.84:0/226082516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f640806c7d0 msgr2=0x7f64081f6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:33.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.379+0000 7f640d7b0640 1 --2- 172.21.15.84:0/226082516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f640806c7d0 0x7f64081f6ad0 secure :-1 s=READY pgs=3431 cs=0 l=1 rev1=1 crypto rx=0x7f63f000b3f0 tx=0x7f63f000b8b0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.379+0000 7f640d7b0640 1 -- 172.21.15.84:0/226082516 shutdown_connections 2023-10-16T11:27:33.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.379+0000 7f640d7b0640 1 --2- 172.21.15.84:0/226082516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f640815d330 0x7f64080cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.379+0000 7f640d7b0640 1 --2- 172.21.15.84:0/226082516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f640806c7d0 0x7f64081f6ad0 unknown :-1 s=CLOSED pgs=3431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.379+0000 7f640d7b0640 1 --2- 172.21.15.84:0/226082516 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f63e806afe0 0x7f63e806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.379+0000 7f640d7b0640 1 --2- 172.21.15.84:0/226082516 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64080661c0 0x7f64080c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.379+0000 7f640d7b0640 1 -- 172.21.15.84:0/226082516 >> 172.21.15.84:0/226082516 conn(0x7f64080d0ab0 msgr2=0x7f640806b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:33.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.379+0000 7f640d7b0640 1 -- 172.21.15.84:0/226082516 shutdown_connections 2023-10-16T11:27:33.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.379+0000 7f640d7b0640 1 -- 172.21.15.84:0/226082516 wait complete. 2023-10-16T11:27:33.399 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T11:27:33.479 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:33.478+0000 7fc0c2030040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:27:33.485 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:33.483+0000 7faa51e2b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:27:33.537 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:33.534+0000 7fc0c2030040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:27:33.544 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:33.542+0000 7faa51e2b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:27:33.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.603+0000 7fbbecf1d640 1 Processor -- start 2023-10-16T11:27:33.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.603+0000 7fbbecf1d640 1 -- start start 2023-10-16T11:27:33.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.604+0000 7fbbecf1d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbbe8070c10 0x7fbbe806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:33.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.604+0000 7fbbecf1d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe8161740 con 0x7fbbe80cf5f0 2023-10-16T11:27:33.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.604+0000 7fbbecf1d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe806b0f0 con 0x7fbbe8070830 2023-10-16T11:27:33.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.604+0000 7fbbecf1d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe81619b0 con 0x7fbbe8070c10 2023-10-16T11:27:33.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.605+0000 7fbbe6d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbbe8070c10 0x7fbbe806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:33.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.605+0000 7fbbe5d9b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbbe80cf5f0 0x7fbbe80cd690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43370/0 (socket says 172.21.15.84:43370) 2023-10-16T11:27:33.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.605+0000 7fbbe5d9b640 1 -- 172.21.15.84:0/3984764099 learned_addr learned my addr 172.21.15.84:0/3984764099 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:33.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.605+0000 7fbbe6d9d640 1 -- 172.21.15.84:0/3984764099 >> v1:172.21.15.157:6789/0 conn(0x7fbbe8070830 legacy=0x7fbbe806c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:33.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.605+0000 7fbbe6d9d640 1 -- 172.21.15.84:0/3984764099 >> v1:172.21.15.84:6789/0 conn(0x7fbbe80cf5f0 legacy=0x7fbbe80cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:33.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.605+0000 7fbbe6d9d640 1 -- 172.21.15.84:0/3984764099 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe806b3d0 con 0x7fbbe8070c10 2023-10-16T11:27:33.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.605+0000 7fbbe559a640 1 -- 172.21.15.84:0/3984764099 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3641231771 0 0) 0x7fbbe8161740 con 0x7fbbe80cf5f0 2023-10-16T11:27:33.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.606+0000 7fbbe6d9d640 1 --2- 172.21.15.84:0/3984764099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbe8070c10 0x7fbbe806d3c0 secure :-1 s=READY pgs=3433 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009d90 tx=0x7fbbe002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74929f88ba06f86c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:33.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.607+0000 7fbbe559a640 1 -- 172.21.15.84:0/3984764099 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe002cbd0 con 0x7fbbe8070c10 2023-10-16T11:27:33.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.607+0000 7fbbe559a640 1 -- 172.21.15.84:0/3984764099 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe002cd30 con 0x7fbbe8070c10 2023-10-16T11:27:33.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.607+0000 7fbbe559a640 1 -- 172.21.15.84:0/3984764099 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe002cea0 con 0x7fbbe8070c10 2023-10-16T11:27:33.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.607+0000 7fbbecf1d640 1 -- 172.21.15.84:0/3984764099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbe8070c10 msgr2=0x7fbbe806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:33.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.607+0000 7fbbecf1d640 1 --2- 172.21.15.84:0/3984764099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbe8070c10 0x7fbbe806d3c0 secure :-1 s=READY pgs=3433 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009d90 tx=0x7fbbe002be20 comp rx=0 tx=0).stop 2023-10-16T11:27:33.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.607+0000 7fbbecf1d640 1 -- 172.21.15.84:0/3984764099 shutdown_connections 2023-10-16T11:27:33.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.607+0000 7fbbecf1d640 1 --2- 172.21.15.84:0/3984764099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbe8070c10 0x7fbbe806d3c0 unknown :-1 s=CLOSED pgs=3433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.607+0000 7fbbecf1d640 1 -- 172.21.15.84:0/3984764099 >> 172.21.15.84:0/3984764099 conn(0x7fbbe80cfac0 msgr2=0x7fbbe80cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:33.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.608+0000 7fbbecf1d640 1 -- 172.21.15.84:0/3984764099 shutdown_connections 2023-10-16T11:27:33.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.608+0000 7fbbecf1d640 1 -- 172.21.15.84:0/3984764099 wait complete. 2023-10-16T11:27:33.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.610+0000 7fbbecf1d640 1 Processor -- start 2023-10-16T11:27:33.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.610+0000 7fbbecf1d640 1 -- start start 2023-10-16T11:27:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.612+0000 7fbbecf1d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbe8070830 0x7fbbe80c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.612+0000 7fbbecf1d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbe8070c10 0x7fbbe815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.612+0000 7fbbecf1d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbe80cf5f0 0x7fbbe814fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:33.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.612+0000 7fbbecf1d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80af6e0 con 0x7fbbe8070830 2023-10-16T11:27:33.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.612+0000 7fbbecf1d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80c53c0 con 0x7fbbe8070c10 2023-10-16T11:27:33.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.612+0000 7fbbecf1d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80c4b20 con 0x7fbbe80cf5f0 2023-10-16T11:27:33.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.612+0000 7fbbe659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbe8070830 0x7fbbe80c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:33.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.612+0000 7fbbe659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbe8070830 0x7fbbe80c42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42638/0 (socket says 172.21.15.84:42638) 2023-10-16T11:27:33.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.612+0000 7fbbe659c640 1 -- 172.21.15.84:0/3402554138 learned_addr learned my addr 172.21.15.84:0/3402554138 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:33.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.612+0000 7fbbe5d9b640 1 --2- 172.21.15.84:0/3402554138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbe8070c10 0x7fbbe815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:33.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.613+0000 7fbbe6d9d640 1 --2- 172.21.15.84:0/3402554138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbe80cf5f0 0x7fbbe814fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:33.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.613+0000 7fbbe659c640 1 -- 172.21.15.84:0/3402554138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbe80cf5f0 msgr2=0x7fbbe814fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:33.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.613+0000 7fbbe659c640 1 --2- 172.21.15.84:0/3402554138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbe80cf5f0 0x7fbbe814fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.613+0000 7fbbe659c640 1 -- 172.21.15.84:0/3402554138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbe8070c10 msgr2=0x7fbbe815ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:33.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.613+0000 7fbbe659c640 1 --2- 172.21.15.84:0/3402554138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbe8070c10 0x7fbbe815ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.613+0000 7fbbe659c640 1 -- 172.21.15.84:0/3402554138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe0009a70 con 0x7fbbe8070830 2023-10-16T11:27:33.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.613+0000 7fbbe659c640 1 --2- 172.21.15.84:0/3402554138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbe8070830 0x7fbbe80c42b0 secure :-1 s=READY pgs=3434 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800c450 tx=0x7fbbd800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:33.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.613+0000 7fbbd6ffd640 1 -- 172.21.15.84:0/3402554138 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd80052d0 con 0x7fbbe8070830 2023-10-16T11:27:33.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.613+0000 7fbbd6ffd640 1 -- 172.21.15.84:0/3402554138 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd8005430 con 0x7fbbe8070830 2023-10-16T11:27:33.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.613+0000 7fbbecf1d640 1 -- 172.21.15.84:0/3402554138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe80c4e00 con 0x7fbbe8070830 2023-10-16T11:27:33.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.613+0000 7fbbd6ffd640 1 -- 172.21.15.84:0/3402554138 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8012870 con 0x7fbbe8070830 2023-10-16T11:27:33.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.613+0000 7fbbecf1d640 1 -- 172.21.15.84:0/3402554138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe80c47f0 con 0x7fbbe8070830 2023-10-16T11:27:33.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.615+0000 7fbbecf1d640 1 -- 172.21.15.84:0/3402554138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe804e160 con 0x7fbbe8070830 2023-10-16T11:27:33.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.615+0000 7fbbd6ffd640 1 -- 172.21.15.84:0/3402554138 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 161) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fbbd80129d0 con 0x7fbbe8070830 2023-10-16T11:27:33.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.617+0000 7fbbd6ffd640 1 --2- 172.21.15.84:0/3402554138 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fbbc806afe0 0x7fbbc806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:33.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.617+0000 7fbbe5d9b640 1 -- 172.21.15.84:0/3402554138 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fbbc806afe0 msgr2=0x7fbbc806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/731888425 2023-10-16T11:27:33.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.617+0000 7fbbd6ffd640 1 -- 172.21.15.84:0/3402554138 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fbbd8089360 con 0x7fbbe8070830 2023-10-16T11:27:33.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.617+0000 7fbbe5d9b640 1 --2- 172.21.15.84:0/3402554138 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fbbc806afe0 0x7fbbc806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:33.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.621+0000 7fbbd6ffd640 1 -- 172.21.15.84:0/3402554138 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbbd80542d0 con 0x7fbbe8070830 2023-10-16T11:27:33.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.818+0000 7fbbe5d9b640 1 -- 172.21.15.84:0/3402554138 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fbbc806afe0 msgr2=0x7fbbc806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/731888425 2023-10-16T11:27:33.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.818+0000 7fbbe5d9b640 1 --2- 172.21.15.84:0/3402554138 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fbbc806afe0 0x7fbbc806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:27:33.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.828+0000 7fbbecf1d640 1 -- 172.21.15.84:0/3402554138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fbbe8160b70 con 0x7fbbe8070830 2023-10-16T11:27:33.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.833+0000 7fbbd6ffd640 1 -- 172.21.15.84:0/3402554138 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v96)=0 v96) v1 ==== 133+0+0 (secure 0 0 0) 0x7fbbd80578f0 con 0x7fbbe8070830 2023-10-16T11:27:33.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.835+0000 7fbbecf1d640 1 -- 172.21.15.84:0/3402554138 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fbbc806afe0 msgr2=0x7fbbc806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:33.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.835+0000 7fbbecf1d640 1 --2- 172.21.15.84:0/3402554138 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fbbc806afe0 0x7fbbc806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.835+0000 7fbbecf1d640 1 -- 172.21.15.84:0/3402554138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbe8070830 msgr2=0x7fbbe80c42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:33.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.835+0000 7fbbecf1d640 1 --2- 172.21.15.84:0/3402554138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbe8070830 0x7fbbe80c42b0 secure :-1 s=READY pgs=3434 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800c450 tx=0x7fbbd800c910 comp rx=0 tx=0).stop 2023-10-16T11:27:33.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.835+0000 7fbbecf1d640 1 -- 172.21.15.84:0/3402554138 shutdown_connections 2023-10-16T11:27:33.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.835+0000 7fbbecf1d640 1 --2- 172.21.15.84:0/3402554138 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7fbbc806afe0 0x7fbbc806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.835+0000 7fbbecf1d640 1 --2- 172.21.15.84:0/3402554138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbe80cf5f0 0x7fbbe814fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.835+0000 7fbbecf1d640 1 --2- 172.21.15.84:0/3402554138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbe8070c10 0x7fbbe815ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.835+0000 7fbbecf1d640 1 --2- 172.21.15.84:0/3402554138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbe8070830 0x7fbbe80c42b0 unknown :-1 s=CLOSED pgs=3434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:33.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.836+0000 7fbbecf1d640 1 -- 172.21.15.84:0/3402554138 >> 172.21.15.84:0/3402554138 conn(0x7fbbe80cfac0 msgr2=0x7fbbe80cd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:33.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.836+0000 7fbbecf1d640 1 -- 172.21.15.84:0/3402554138 shutdown_connections 2023-10-16T11:27:33.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:33.836+0000 7fbbecf1d640 1 -- 172.21.15.84:0/3402554138 wait complete. 2023-10-16T11:27:33.855 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create a sv1 2023-10-16T11:27:33.995 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:33.994+0000 7faa51e2b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:27:34.027 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:34.025+0000 7fc0c2030040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:27:34.048 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:34.046+0000 7faa51e2b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:27:34.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.060+0000 7f0f2631d640 1 Processor -- start 2023-10-16T11:27:34.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.061+0000 7f0f2631d640 1 -- start start 2023-10-16T11:27:34.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.062+0000 7f0f2631d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0f2006a4b0 0x7f0f200ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:34.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.062+0000 7f0f2631d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f200cb030 con 0x7f0f200c9ca0 2023-10-16T11:27:34.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.062+0000 7f0f2631d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f200c91c0 con 0x7f0f2015ad70 2023-10-16T11:27:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.062+0000 7f0f2631d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f200c94a0 con 0x7f0f2006a4b0 2023-10-16T11:27:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.062+0000 7f0f248ba640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0f200c9ca0 0x7f0f200ad4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43372/0 (socket says 172.21.15.84:43372) 2023-10-16T11:27:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.062+0000 7f0f248ba640 1 -- 172.21.15.84:0/2947969084 learned_addr learned my addr 172.21.15.84:0/2947969084 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:34.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.063+0000 7f0f17fff640 1 --2- 172.21.15.84:0/2947969084 >> v2:172.21.15.157:3300/0 conn(0x7f0f2006a4b0 0x7f0f200ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:34.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.063+0000 7f0f1f7fe640 1 -- 172.21.15.84:0/2947969084 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054122526 0 0) 0x7f0f200cb030 con 0x7f0f200c9ca0 2023-10-16T11:27:34.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.063+0000 7f0f1f7fe640 1 -- 172.21.15.84:0/2947969084 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f0c003520 con 0x7f0f200c9ca0 2023-10-16T11:27:34.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.063+0000 7f0f1f7fe640 1 -- 172.21.15.84:0/2947969084 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 741257923 0 0) 0x7f0f0c003520 con 0x7f0f200c9ca0 2023-10-16T11:27:34.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.063+0000 7f0f1f7fe640 1 -- 172.21.15.84:0/2947969084 >> v2:172.21.15.157:3300/0 conn(0x7f0f2006a4b0 msgr2=0x7f0f200ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:34.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.063+0000 7f0f1f7fe640 1 --2- 172.21.15.84:0/2947969084 >> v2:172.21.15.157:3300/0 conn(0x7f0f2006a4b0 0x7f0f200ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:34.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.063+0000 7f0f1f7fe640 1 -- 172.21.15.84:0/2947969084 >> v1:172.21.15.157:6789/0 conn(0x7f0f2015ad70 legacy=0x7f0f20069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:34.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.063+0000 7f0f1f7fe640 1 -- 172.21.15.84:0/2947969084 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f200c9780 con 0x7f0f200c9ca0 2023-10-16T11:27:34.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.063+0000 7f0f1f7fe640 1 -- 172.21.15.84:0/2947969084 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f0f180019f0 con 0x7f0f200c9ca0 2023-10-16T11:27:34.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.063+0000 7f0f1f7fe640 1 -- 172.21.15.84:0/2947969084 >> v1:172.21.15.84:6789/0 conn(0x7f0f200c9ca0 legacy=0x7f0f200ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:34.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.063+0000 7f0f1f7fe640 1 --2- 172.21.15.84:0/2947969084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f0c0039c0 0x7f0f0c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:34.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.063+0000 7f0f1f7fe640 1 --2- 172.21.15.84:0/2947969084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f0c0047c0 0x7f0f0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:34.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.063+0000 7f0f1f7fe640 1 --2- 172.21.15.84:0/2947969084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c005180 0x7f0f0c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:34.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.063+0000 7f0f1f7fe640 1 -- 172.21.15.84:0/2947969084 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f200c94a0 con 0x7f0f0c005180 2023-10-16T11:27:34.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.063+0000 7f0f1f7fe640 1 -- 172.21.15.84:0/2947969084 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c00c080 con 0x7f0f0c0039c0 2023-10-16T11:27:34.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.063+0000 7f0f1f7fe640 1 -- 172.21.15.84:0/2947969084 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c00c240 con 0x7f0f0c0047c0 2023-10-16T11:27:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.064+0000 7f0f248ba640 1 --2- 172.21.15.84:0/2947969084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c005180 0x7f0f0c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.064+0000 7f0f1ffff640 1 --2- 172.21.15.84:0/2947969084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f0c0039c0 0x7f0f0c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.064+0000 7f0f17fff640 1 --2- 172.21.15.84:0/2947969084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f0c0047c0 0x7f0f0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.064+0000 7f0f248ba640 1 -- 172.21.15.84:0/2947969084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f0c0047c0 msgr2=0x7f0f0c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.064+0000 7f0f248ba640 1 --2- 172.21.15.84:0/2947969084 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f0c0047c0 0x7f0f0c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:34.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.064+0000 7f0f248ba640 1 -- 172.21.15.84:0/2947969084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f0c0039c0 msgr2=0x7f0f0c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:34.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.064+0000 7f0f248ba640 1 --2- 172.21.15.84:0/2947969084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f0c0039c0 0x7f0f0c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:34.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.064+0000 7f0f248ba640 1 -- 172.21.15.84:0/2947969084 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c003520 con 0x7f0f0c005180 2023-10-16T11:27:34.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.065+0000 7f0f248ba640 1 -- 172.21.15.84:0/2947969084 reap_dead start 2023-10-16T11:27:34.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.065+0000 7f0f248ba640 1 --2- 172.21.15.84:0/2947969084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c005180 0x7f0f0c00b980 secure :-1 s=READY pgs=3436 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800a760 tx=0x7f0f0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d343b116b54d03b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:34.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.065+0000 7f0f1f7fe640 1 -- 172.21.15.84:0/2947969084 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f18034630 con 0x7f0f0c005180 2023-10-16T11:27:34.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.065+0000 7f0f1f7fe640 1 -- 172.21.15.84:0/2947969084 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1802f070 con 0x7f0f0c005180 2023-10-16T11:27:34.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.065+0000 7f0f1f7fe640 1 -- 172.21.15.84:0/2947969084 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f18034a10 con 0x7f0f0c005180 2023-10-16T11:27:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.066+0000 7f0f2631d640 1 -- 172.21.15.84:0/2947969084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c005180 msgr2=0x7f0f0c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.066+0000 7f0f2631d640 1 --2- 172.21.15.84:0/2947969084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c005180 0x7f0f0c00b980 secure :-1 s=READY pgs=3436 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800a760 tx=0x7f0f0c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.066+0000 7f0f2631d640 1 -- 172.21.15.84:0/2947969084 shutdown_connections 2023-10-16T11:27:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.066+0000 7f0f2631d640 1 --2- 172.21.15.84:0/2947969084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c005180 0x7f0f0c00b980 unknown :-1 s=CLOSED pgs=3436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.066+0000 7f0f2631d640 1 -- 172.21.15.84:0/2947969084 >> 172.21.15.84:0/2947969084 conn(0x7f0f200ce9b0 msgr2=0x7f0f200cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:34.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.066+0000 7f0f2631d640 1 -- 172.21.15.84:0/2947969084 shutdown_connections 2023-10-16T11:27:34.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.066+0000 7f0f2631d640 1 -- 172.21.15.84:0/2947969084 wait complete. 2023-10-16T11:27:34.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.068+0000 7f0f2631d640 1 Processor -- start 2023-10-16T11:27:34.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.068+0000 7f0f2631d640 1 -- start start 2023-10-16T11:27:34.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.069+0000 7f0f2631d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c005180 0x7f0f20158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:34.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.069+0000 7f0f2631d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f200b0480 0x7f0f200b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:34.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.069+0000 7f0f2631d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f200cd9e0 0x7f0f200cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:34.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.069+0000 7f0f2631d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f200b36f0 con 0x7f0f0c005180 2023-10-16T11:27:34.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.069+0000 7f0f2631d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f200ccf90 con 0x7f0f200b0480 2023-10-16T11:27:34.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.069+0000 7f0f2631d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f200b3900 con 0x7f0f200cd9e0 2023-10-16T11:27:34.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.069+0000 7f0f1ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c005180 0x7f0f20158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:34.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.069+0000 7f0f1ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c005180 0x7f0f20158070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42648/0 (socket says 172.21.15.84:42648) 2023-10-16T11:27:34.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.069+0000 7f0f1ffff640 1 -- 172.21.15.84:0/2426210642 learned_addr learned my addr 172.21.15.84:0/2426210642 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:34.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.070+0000 7f0f17fff640 1 --2- 172.21.15.84:0/2426210642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f200b0480 0x7f0f200b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:34.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.070+0000 7f0f248ba640 1 --2- 172.21.15.84:0/2426210642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f200cd9e0 0x7f0f200cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:34.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.070+0000 7f0f1ffff640 1 -- 172.21.15.84:0/2426210642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f200cd9e0 msgr2=0x7f0f200cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:34.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.070+0000 7f0f1ffff640 1 --2- 172.21.15.84:0/2426210642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f200cd9e0 0x7f0f200cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:34.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.070+0000 7f0f1ffff640 1 -- 172.21.15.84:0/2426210642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f200b0480 msgr2=0x7f0f200b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:34.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.070+0000 7f0f1ffff640 1 --2- 172.21.15.84:0/2426210642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f200b0480 0x7f0f200b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:34.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.070+0000 7f0f1ffff640 1 -- 172.21.15.84:0/2426210642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f200b3be0 con 0x7f0f0c005180 2023-10-16T11:27:34.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.070+0000 7f0f248ba640 1 --2- 172.21.15.84:0/2426210642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f200cd9e0 0x7f0f200cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:34.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.070+0000 7f0f1ffff640 1 --2- 172.21.15.84:0/2426210642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f0c005180 0x7f0f20158070 secure :-1 s=READY pgs=3437 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000e4b0 tx=0x7f0f1000e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:34.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.070+0000 7f0f1d7fa640 1 -- 172.21.15.84:0/2426210642 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f100052e0 con 0x7f0f0c005180 2023-10-16T11:27:34.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.070+0000 7f0f1d7fa640 1 -- 172.21.15.84:0/2426210642 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f10005440 con 0x7f0f0c005180 2023-10-16T11:27:34.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.070+0000 7f0f1d7fa640 1 -- 172.21.15.84:0/2426210642 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10003830 con 0x7f0f0c005180 2023-10-16T11:27:34.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.070+0000 7f0f2631d640 1 -- 172.21.15.84:0/2426210642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f200cd1e0 con 0x7f0f0c005180 2023-10-16T11:27:34.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.071+0000 7f0f2631d640 1 -- 172.21.15.84:0/2426210642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f201585b0 con 0x7f0f0c005180 2023-10-16T11:27:34.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.072+0000 7f0f2631d640 1 -- 172.21.15.84:0/2426210642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2006c9f0 con 0x7f0f0c005180 2023-10-16T11:27:34.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.073+0000 7f0f1d7fa640 1 -- 172.21.15.84:0/2426210642 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 161) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f0f10003990 con 0x7f0f0c005180 2023-10-16T11:27:34.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.073+0000 7f0f1d7fa640 1 --2- 172.21.15.84:0/2426210642 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f0f0006afe0 0x7f0f0006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.074+0000 7f0f17fff640 1 -- 172.21.15.84:0/2426210642 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f0f0006afe0 msgr2=0x7f0f0006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/731888425 2023-10-16T11:27:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.074+0000 7f0f1d7fa640 1 -- 172.21.15.84:0/2426210642 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0f100894e0 con 0x7f0f0c005180 2023-10-16T11:27:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.074+0000 7f0f17fff640 1 --2- 172.21.15.84:0/2426210642 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f0f0006afe0 0x7f0f0006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.078+0000 7f0f1d7fa640 1 -- 172.21.15.84:0/2426210642 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0f10054450 con 0x7f0f0c005180 2023-10-16T11:27:34.084 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:34.083+0000 7fc0c2030040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:27:34.099 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:34.097+0000 7faa51e2b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:27:34.137 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:34.135+0000 7fc0c2030040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:27:34.159 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:34.158+0000 7faa51e2b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:27:34.200 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:34.198+0000 7fc0c2030040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:27:34.208 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:34.206+0000 7faa51e2b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:27:34.250 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:34.248+0000 7fc0c2030040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:27:34.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.275+0000 7f0f17fff640 1 -- 172.21.15.84:0/2426210642 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f0f0006afe0 msgr2=0x7f0f0006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/731888425 2023-10-16T11:27:34.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.275+0000 7f0f17fff640 1 --2- 172.21.15.84:0/2426210642 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f0f0006afe0 0x7f0f0006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:27:34.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.296+0000 7f0f2631d640 1 -- 172.21.15.84:0/2426210642 --> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "a", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f20069da0 con 0x7f0f0006afe0 2023-10-16T11:27:34.320 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:34.318+0000 7faa51e2b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:27:34.365 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:34.363+0000 7fc0c2030040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:27:34.382 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:34.381+0000 7faa51e2b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:27:34.427 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:34.425+0000 7fc0c2030040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:27:34.432 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:34.431+0000 7faa51e2b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:27:34.477 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:34.475+0000 7fc0c2030040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:27:34.490 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:34.488+0000 7faa51e2b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:27:34.534 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:34.532+0000 7fc0c2030040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:27:34.540 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:34.539+0000 7faa51e2b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:27:34.585 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:34.583+0000 7fc0c2030040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:27:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.676+0000 7f0f17fff640 1 -- 172.21.15.84:0/2426210642 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f0f0006afe0 msgr2=0x7f0f0006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/731888425 2023-10-16T11:27:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:34.676+0000 7f0f17fff640 1 --2- 172.21.15.84:0/2426210642 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f0f0006afe0 0x7f0f0006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T11:27:34.918 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:34.917+0000 7faa51e2b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:27:34.965 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:34.963+0000 7fc0c2030040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:27:35.081 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:35.080+0000 7faa51e2b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:27:35.096 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:27:35.105 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:27:35.125 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:35.123+0000 7fc0c2030040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:27:35.194 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:35.193+0000 7faa51e2b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:27:35.239 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:35.237+0000 7fc0c2030040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:27:35.246 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:35.245+0000 7faa51e2b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:27:35.289 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:35.287+0000 7fc0c2030040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:27:35.355 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:35.354+0000 7faa51e2b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:27:35.397 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:35.395+0000 7fc0c2030040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:27:35.462 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:35.462+0000 7faa51e2b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:27:35.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:35.477+0000 7f0f17fff640 1 -- 172.21.15.84:0/2426210642 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f0f0006afe0 msgr2=0x7f0f0006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/731888425 2023-10-16T11:27:35.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:35.477+0000 7f0f17fff640 1 --2- 172.21.15.84:0/2426210642 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f0f0006afe0 0x7f0f0006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T11:27:35.504 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:35.501+0000 7fc0c2030040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:27:35.619 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:35.618+0000 7faa51e2b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:27:35.661 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:35.658+0000 7fc0c2030040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:27:35.669 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:35.668+0000 7faa51e2b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:27:35.709 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:35.707+0000 7fc0c2030040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:27:35.786 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:35.785+0000 7faa51e2b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:27:35.830 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:35.828+0000 7fc0c2030040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:27:36.606 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:36.605+0000 7faa51e2b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:27:36.665 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:36.662+0000 7fc0c2030040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:27:36.899 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:36.898+0000 7faa51e2b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:27:36.956 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:36.955+0000 7faa51e2b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:27:36.957 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:36.955+0000 7fc0c2030040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:27:37.014 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:37.012+0000 7fc0c2030040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:27:37.069 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:37.069+0000 7faa51e2b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:27:37.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:37.079+0000 7f0f17fff640 1 -- 172.21.15.84:0/2426210642 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f0f0006afe0 msgr2=0x7f0f0006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/731888425 2023-10-16T11:27:37.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:37.079+0000 7f0f17fff640 1 --2- 172.21.15.84:0/2426210642 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f0f0006afe0 0x7f0f0006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T11:27:37.129 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:37.126+0000 7fc0c2030040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:27:37.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:37.135+0000 7f0f1d7fa640 1 -- 172.21.15.84:0/2426210642 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 162) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f0f10054820 con 0x7f0f0c005180 2023-10-16T11:27:37.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:37.156+0000 7f0f1d7fa640 1 -- 172.21.15.84:0/2426210642 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 163) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f0f10057600 con 0x7f0f0c005180 2023-10-16T11:27:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:37.156+0000 7f0f1d7fa640 1 -- 172.21.15.84:0/2426210642 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f0f0006afe0 msgr2=0x7f0f0006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:37.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:37.156+0000 7f0f1d7fa640 1 --2- 172.21.15.84:0/2426210642 >> [v2:172.21.15.84:6832/731888425,v1:172.21.15.84:6833/731888425] conn(0x7f0f0006afe0 0x7f0f0006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:37.458 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:37.456+0000 7fc03e3c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:37.459 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:37.456+0000 7fc03e3c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:37.459 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:37.456+0000 7fc03e3c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:37.459 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:37.456+0000 7fc03e3c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:37.459 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:37.456+0000 7fc03e3c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:37.460 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:37.457+0000 7fc040c08640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:37.460 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:37.457+0000 7fc040c08640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:37.460 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:37.457+0000 7fc040c08640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:37.461 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:37.457+0000 7fc040c08640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:37.461 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:37.457+0000 7fc040c08640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:38.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.163+0000 7f0f1d7fa640 1 -- 172.21.15.84:0/2426210642 <== mon.0 v2:172.21.15.84:3300/0 9 ==== mgrmap(e 164) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f0f100550f0 con 0x7f0f0c005180 2023-10-16T11:27:38.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.164+0000 7f0f1d7fa640 1 --2- 172.21.15.84:0/2426210642 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f0f0001d1c0 0x7f0f00074f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:38.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.164+0000 7f0f1d7fa640 1 -- 172.21.15.84:0/2426210642 --> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "a", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f0001d5a0 con 0x7f0f0001d1c0 2023-10-16T11:27:38.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.172+0000 7f0f17fff640 1 --2- 172.21.15.84:0/2426210642 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f0f0001d1c0 0x7f0f00074f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.180+0000 7f0f17fff640 1 --2- 172.21.15.84:0/2426210642 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f0f0001d1c0 0x7f0f00074f00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0f200c8ce0 tx=0x7f0f08008040 comp rx=0 tx=0).ready entity=mgr.13647 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:38.186 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:38.184+0000 7fc03b3bd640 -1 mgr.server reply reply (2) No such file or directory FS 'a' not found 2023-10-16T11:27:38.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.184+0000 7f0f1d7fa640 1 -- 172.21.15.84:0/2426210642 <== mgr.13647 v2:172.21.15.84:6832/129259578 1 ==== mgr_command_reply(tid 0: -2 FS 'a' not found) v1 ==== 24+0+0 (secure 0 0 0) 0x7f0f0001d5a0 con 0x7f0f0001d1c0 2023-10-16T11:27:38.187 INFO:teuthology.orchestra.run.smithi084.stderr:Error ENOENT: FS 'a' not found 2023-10-16T11:27:38.200 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T11:27:38.201 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T11:27:38.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.436+0000 7f0e5fe21640 1 Processor -- start 2023-10-16T11:27:38.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.436+0000 7f0e5fe21640 1 -- start start 2023-10-16T11:27:38.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.437+0000 7f0e5fe21640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0e580caac0 0x7f0e580ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:38.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.437+0000 7f0e5fe21640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e580cb030 con 0x7f0e5815ad70 2023-10-16T11:27:38.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.437+0000 7f0e5fe21640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e580c91c0 con 0x7f0e5806a4b0 2023-10-16T11:27:38.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.438+0000 7f0e5fe21640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e580c94a0 con 0x7f0e580caac0 2023-10-16T11:27:38.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.438+0000 7f0e5dbbd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0e5815ad70 0x7f0e58069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52434/0 (socket says 172.21.15.84:52434) 2023-10-16T11:27:38.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.438+0000 7f0e5dbbd640 1 -- 172.21.15.84:0/134704131 learned_addr learned my addr 172.21.15.84:0/134704131 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:38.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.438+0000 7f0e5e3be640 1 --2- 172.21.15.84:0/134704131 >> v2:172.21.15.157:3300/0 conn(0x7f0e580caac0 0x7f0e580ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:38.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.438+0000 7f0e5cbbb640 1 -- 172.21.15.84:0/134704131 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290157421 0 0) 0x7f0e580cb030 con 0x7f0e5815ad70 2023-10-16T11:27:38.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.438+0000 7f0e5cbbb640 1 -- 172.21.15.84:0/134704131 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e44003520 con 0x7f0e5815ad70 2023-10-16T11:27:38.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.439+0000 7f0e5cbbb640 1 -- 172.21.15.84:0/134704131 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 712131668 0 0) 0x7f0e44003520 con 0x7f0e5815ad70 2023-10-16T11:27:38.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.439+0000 7f0e5cbbb640 1 -- 172.21.15.84:0/134704131 >> v2:172.21.15.157:3300/0 conn(0x7f0e580caac0 msgr2=0x7f0e580ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:38.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.439+0000 7f0e5cbbb640 1 --2- 172.21.15.84:0/134704131 >> v2:172.21.15.157:3300/0 conn(0x7f0e580caac0 0x7f0e580ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:38.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.439+0000 7f0e5cbbb640 1 -- 172.21.15.84:0/134704131 >> v1:172.21.15.157:6789/0 conn(0x7f0e5806a4b0 legacy=0x7f0e580ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:38.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.439+0000 7f0e5cbbb640 1 -- 172.21.15.84:0/134704131 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e580c9780 con 0x7f0e5815ad70 2023-10-16T11:27:38.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.439+0000 7f0e5cbbb640 1 -- 172.21.15.84:0/134704131 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f0e54001670 con 0x7f0e5815ad70 2023-10-16T11:27:38.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.439+0000 7f0e5cbbb640 1 -- 172.21.15.84:0/134704131 >> v1:172.21.15.84:6789/0 conn(0x7f0e5815ad70 legacy=0x7f0e58069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:38.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.439+0000 7f0e5cbbb640 1 --2- 172.21.15.84:0/134704131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e440039c0 0x7f0e44003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:38.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.439+0000 7f0e5cbbb640 1 --2- 172.21.15.84:0/134704131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e440047c0 0x7f0e44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:38.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.439+0000 7f0e5cbbb640 1 --2- 172.21.15.84:0/134704131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e44005180 0x7f0e4400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:38.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.439+0000 7f0e5cbbb640 1 -- 172.21.15.84:0/134704131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e580c94a0 con 0x7f0e44005180 2023-10-16T11:27:38.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.439+0000 7f0e5cbbb640 1 -- 172.21.15.84:0/134704131 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4400c050 con 0x7f0e440039c0 2023-10-16T11:27:38.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.439+0000 7f0e5cbbb640 1 -- 172.21.15.84:0/134704131 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4400c210 con 0x7f0e440047c0 2023-10-16T11:27:38.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.439+0000 7f0e5e3be640 1 --2- 172.21.15.84:0/134704131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e44005180 0x7f0e4400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:38.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.440+0000 7f0e5e3be640 1 -- 172.21.15.84:0/134704131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e440047c0 msgr2=0x7f0e44004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:38.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.440+0000 7f0e5e3be640 1 --2- 172.21.15.84:0/134704131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e440047c0 0x7f0e44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:38.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.440+0000 7f0e5e3be640 1 -- 172.21.15.84:0/134704131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e440039c0 msgr2=0x7f0e44003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:38.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.440+0000 7f0e5e3be640 1 --2- 172.21.15.84:0/134704131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e440039c0 0x7f0e44003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:38.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.440+0000 7f0e5e3be640 1 -- 172.21.15.84:0/134704131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e44003520 con 0x7f0e44005180 2023-10-16T11:27:38.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.441+0000 7f0e5e3be640 1 -- 172.21.15.84:0/134704131 reap_dead start 2023-10-16T11:27:38.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.441+0000 7f0e5e3be640 1 --2- 172.21.15.84:0/134704131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e44005180 0x7f0e4400b980 secure :-1 s=READY pgs=3452 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009fc0 tx=0x7f0e440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126e4afac440721d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:38.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.441+0000 7f0e5cbbb640 1 -- 172.21.15.84:0/134704131 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5002cc60 con 0x7f0e44005180 2023-10-16T11:27:38.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.441+0000 7f0e5cbbb640 1 -- 172.21.15.84:0/134704131 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5000f040 con 0x7f0e44005180 2023-10-16T11:27:38.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.441+0000 7f0e5cbbb640 1 -- 172.21.15.84:0/134704131 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50003990 con 0x7f0e44005180 2023-10-16T11:27:38.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.442+0000 7f0e5fe21640 1 -- 172.21.15.84:0/134704131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e44005180 msgr2=0x7f0e4400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:38.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.442+0000 7f0e5fe21640 1 --2- 172.21.15.84:0/134704131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e44005180 0x7f0e4400b980 secure :-1 s=READY pgs=3452 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009fc0 tx=0x7f0e440057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:38.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.442+0000 7f0e5fe21640 1 -- 172.21.15.84:0/134704131 shutdown_connections 2023-10-16T11:27:38.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.442+0000 7f0e5fe21640 1 --2- 172.21.15.84:0/134704131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e44005180 0x7f0e4400b980 unknown :-1 s=CLOSED pgs=3452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:38.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.442+0000 7f0e5fe21640 1 -- 172.21.15.84:0/134704131 >> 172.21.15.84:0/134704131 conn(0x7f0e580ce9b0 msgr2=0x7f0e580cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:38.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.442+0000 7f0e5fe21640 1 -- 172.21.15.84:0/134704131 shutdown_connections 2023-10-16T11:27:38.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.442+0000 7f0e5fe21640 1 -- 172.21.15.84:0/134704131 wait complete. 2023-10-16T11:27:38.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.444+0000 7f0e5fe21640 1 Processor -- start 2023-10-16T11:27:38.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.444+0000 7f0e5fe21640 1 -- start start 2023-10-16T11:27:38.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.445+0000 7f0e5fe21640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e44005180 0x7f0e580ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:38.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.445+0000 7f0e5fe21640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e580b2c50 0x7f0e580c3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:38.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.445+0000 7f0e5fe21640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e58158b10 0x7f0e580c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:38.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.445+0000 7f0e5fe21640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e580c3580 con 0x7f0e58158b10 2023-10-16T11:27:38.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.445+0000 7f0e5fe21640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e580b0e50 con 0x7f0e580b2c50 2023-10-16T11:27:38.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.445+0000 7f0e5fe21640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e580b30d0 con 0x7f0e44005180 2023-10-16T11:27:38.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.445+0000 7f0e5e3be640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e58158b10 0x7f0e580c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:38.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.445+0000 7f0e5e3be640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e58158b10 0x7f0e580c4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45416/0 (socket says 172.21.15.84:45416) 2023-10-16T11:27:38.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.445+0000 7f0e5e3be640 1 -- 172.21.15.84:0/3316302140 learned_addr learned my addr 172.21.15.84:0/3316302140 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:38.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.445+0000 7f0e5e3be640 1 -- 172.21.15.84:0/3316302140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e44005180 msgr2=0x7f0e580ad3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:38.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.446+0000 7f0e5e3be640 1 --2- 172.21.15.84:0/3316302140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e44005180 0x7f0e580ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:38.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.446+0000 7f0e5e3be640 1 -- 172.21.15.84:0/3316302140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e580b2c50 msgr2=0x7f0e580c3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:38.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.446+0000 7f0e5e3be640 1 --2- 172.21.15.84:0/3316302140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e580b2c50 0x7f0e580c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:38.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.446+0000 7f0e5e3be640 1 -- 172.21.15.84:0/3316302140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e580b3350 con 0x7f0e58158b10 2023-10-16T11:27:38.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.446+0000 7f0e5e3be640 1 --2- 172.21.15.84:0/3316302140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e58158b10 0x7f0e580c4060 secure :-1 s=READY pgs=3453 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000e200 tx=0x7f0e50009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:38.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.446+0000 7f0e4e7fc640 1 -- 172.21.15.84:0/3316302140 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50035df0 con 0x7f0e58158b10 2023-10-16T11:27:38.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.446+0000 7f0e4e7fc640 1 -- 172.21.15.84:0/3316302140 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e50003990 con 0x7f0e58158b10 2023-10-16T11:27:38.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.446+0000 7f0e5fe21640 1 -- 172.21.15.84:0/3316302140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e580b0520 con 0x7f0e58158b10 2023-10-16T11:27:38.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.446+0000 7f0e4e7fc640 1 -- 172.21.15.84:0/3316302140 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50003d40 con 0x7f0e58158b10 2023-10-16T11:27:38.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.446+0000 7f0e5fe21640 1 -- 172.21.15.84:0/3316302140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e580b0ae0 con 0x7f0e58158b10 2023-10-16T11:27:38.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.448+0000 7f0e5fe21640 1 -- 172.21.15.84:0/3316302140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e5804e160 con 0x7f0e58158b10 2023-10-16T11:27:38.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.448+0000 7f0e4e7fc640 1 -- 172.21.15.84:0/3316302140 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 164) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f0e50004030 con 0x7f0e58158b10 2023-10-16T11:27:38.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.450+0000 7f0e4e7fc640 1 --2- 172.21.15.84:0/3316302140 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f0e3c06ad10 0x7f0e3c06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:38.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.450+0000 7f0e4e7fc640 1 -- 172.21.15.84:0/3316302140 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f0e500ac110 con 0x7f0e58158b10 2023-10-16T11:27:38.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.450+0000 7f0e5dbbd640 1 --2- 172.21.15.84:0/3316302140 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f0e3c06ad10 0x7f0e3c06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:38.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.451+0000 7f0e5dbbd640 1 --2- 172.21.15.84:0/3316302140 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f0e3c06ad10 0x7f0e3c06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0e581f6af0 tx=0x7f0e54001570 comp rx=0 tx=0).ready entity=mgr.13647 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:38.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.455+0000 7f0e4e7fc640 1 -- 172.21.15.84:0/3316302140 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0e50077380 con 0x7f0e58158b10 2023-10-16T11:27:38.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:38.693+0000 7f0e5fe21640 1 -- 172.21.15.84:0/3316302140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f0e580b1000 con 0x7f0e58158b10 2023-10-16T11:27:39.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.190+0000 7f0e4e7fc640 1 -- 172.21.15.84:0/3316302140 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v165) v1 ==== 93+0+0 (secure 0 0 0) 0x7f0e580b1000 con 0x7f0e58158b10 2023-10-16T11:27:39.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.194+0000 7f0e5fe21640 1 -- 172.21.15.84:0/3316302140 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f0e3c06ad10 msgr2=0x7f0e3c06d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:39.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.194+0000 7f0e5fe21640 1 --2- 172.21.15.84:0/3316302140 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f0e3c06ad10 0x7f0e3c06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0e581f6af0 tx=0x7f0e54001570 comp rx=0 tx=0).stop 2023-10-16T11:27:39.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.194+0000 7f0e5fe21640 1 -- 172.21.15.84:0/3316302140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e58158b10 msgr2=0x7f0e580c4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:39.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.194+0000 7f0e5fe21640 1 --2- 172.21.15.84:0/3316302140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e58158b10 0x7f0e580c4060 secure :-1 s=READY pgs=3453 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000e200 tx=0x7f0e50009750 comp rx=0 tx=0).stop 2023-10-16T11:27:39.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.195+0000 7f0e5fe21640 1 -- 172.21.15.84:0/3316302140 shutdown_connections 2023-10-16T11:27:39.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.195+0000 7f0e5fe21640 1 --2- 172.21.15.84:0/3316302140 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f0e3c06ad10 0x7f0e3c06d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:39.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.195+0000 7f0e5fe21640 1 --2- 172.21.15.84:0/3316302140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e58158b10 0x7f0e580c4060 unknown :-1 s=CLOSED pgs=3453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:39.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.195+0000 7f0e5fe21640 1 --2- 172.21.15.84:0/3316302140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e580b2c50 0x7f0e580c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:39.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.195+0000 7f0e5fe21640 1 --2- 172.21.15.84:0/3316302140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e44005180 0x7f0e580ad3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:39.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.195+0000 7f0e5fe21640 1 -- 172.21.15.84:0/3316302140 >> 172.21.15.84:0/3316302140 conn(0x7f0e580ce9b0 msgr2=0x7f0e580b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:39.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.195+0000 7f0e5fe21640 1 -- 172.21.15.84:0/3316302140 shutdown_connections 2023-10-16T11:27:39.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.196+0000 7f0e5fe21640 1 -- 172.21.15.84:0/3316302140 wait complete. 2023-10-16T11:27:39.219 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:27:39.220 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:27:39.310 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:27:39.403 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:27:39.404 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:27:39.404 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:39.456 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:27:39.456 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:39.547 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:27:39.710 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:39.708+0000 7f338283b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:27:39.721 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:39.721+0000 7f950da0f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:27:39.765 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:39.763+0000 7f338283b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:27:39.777 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:39.776+0000 7f950da0f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:27:39.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.795+0000 7f2ba0697640 1 Processor -- start 2023-10-16T11:27:39.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.795+0000 7f2ba0697640 1 -- start start 2023-10-16T11:27:39.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.797+0000 7f2ba0697640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2b980b3070 0x7f2b980cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:39.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.797+0000 7f2ba0697640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b980b3e70 0x7f2b980cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:39.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.797+0000 7f2ba0697640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b980b3940 con 0x7f2b980b3450 2023-10-16T11:27:39.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.797+0000 7f2ba0697640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b980b45e0 con 0x7f2b980b3070 2023-10-16T11:27:39.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.797+0000 7f2ba0697640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b980b3b90 con 0x7f2b980b3e70 2023-10-16T11:27:39.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.798+0000 7f2b9e433640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2b980b3070 0x7f2b980cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:39.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.798+0000 7f2b9e433640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2b980b3070 0x7f2b980cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45440/0 (socket says 172.21.15.84:45440) 2023-10-16T11:27:39.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.798+0000 7f2b9e433640 1 -- 172.21.15.84:0/2118514915 learned_addr learned my addr 172.21.15.84:0/2118514915 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:39.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.798+0000 7f2b9dc32640 1 --2- 172.21.15.84:0/2118514915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b980b3e70 0x7f2b980cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:39.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.798+0000 7f2b9e433640 1 -- 172.21.15.84:0/2118514915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b980b3e70 msgr2=0x7f2b980cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:39.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.798+0000 7f2b9e433640 1 --2- 172.21.15.84:0/2118514915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b980b3e70 0x7f2b980cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:39.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.798+0000 7f2b9e433640 1 -- 172.21.15.84:0/2118514915 >> v1:172.21.15.157:6789/0 conn(0x7f2b980b3450 legacy=0x7f2b980c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:39.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.798+0000 7f2b9e433640 1 -- 172.21.15.84:0/2118514915 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b980d3840 con 0x7f2b980b3070 2023-10-16T11:27:39.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.798+0000 7f2b9dc32640 1 --2- 172.21.15.84:0/2118514915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b980b3e70 0x7f2b980cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:39.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.799+0000 7f2b9e433640 1 --2- 172.21.15.84:0/2118514915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b980b3070 0x7f2b980cf500 secure :-1 s=READY pgs=3456 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009860 tx=0x7f2b9402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8f379416d988ec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:39.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.800+0000 7f2b9d431640 1 -- 172.21.15.84:0/2118514915 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94031640 con 0x7f2b980b3070 2023-10-16T11:27:39.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.800+0000 7f2b9d431640 1 -- 172.21.15.84:0/2118514915 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b940317a0 con 0x7f2b980b3070 2023-10-16T11:27:39.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.800+0000 7f2b9d431640 1 -- 172.21.15.84:0/2118514915 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94031b30 con 0x7f2b980b3070 2023-10-16T11:27:39.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.800+0000 7f2ba0697640 1 -- 172.21.15.84:0/2118514915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b980b3070 msgr2=0x7f2b980cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:39.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.800+0000 7f2ba0697640 1 --2- 172.21.15.84:0/2118514915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b980b3070 0x7f2b980cf500 secure :-1 s=READY pgs=3456 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009860 tx=0x7f2b9402b8f0 comp rx=0 tx=0).stop 2023-10-16T11:27:39.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.800+0000 7f2ba0697640 1 -- 172.21.15.84:0/2118514915 shutdown_connections 2023-10-16T11:27:39.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.800+0000 7f2ba0697640 1 --2- 172.21.15.84:0/2118514915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b980b3e70 0x7f2b980cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:39.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.800+0000 7f2ba0697640 1 --2- 172.21.15.84:0/2118514915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b980b3070 0x7f2b980cf500 unknown :-1 s=CLOSED pgs=3456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:39.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.800+0000 7f2ba0697640 1 -- 172.21.15.84:0/2118514915 >> 172.21.15.84:0/2118514915 conn(0x7f2b980d2d20 msgr2=0x7f2b980d3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:39.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.801+0000 7f2ba0697640 1 -- 172.21.15.84:0/2118514915 shutdown_connections 2023-10-16T11:27:39.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.801+0000 7f2ba0697640 1 -- 172.21.15.84:0/2118514915 wait complete. 2023-10-16T11:27:39.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.802+0000 7f2ba0697640 1 Processor -- start 2023-10-16T11:27:39.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.802+0000 7f2ba0697640 1 -- start start 2023-10-16T11:27:39.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.803+0000 7f2ba0697640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b980b3070 0x7f2b9815c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:39.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.803+0000 7f2ba0697640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b980b3450 0x7f2b980cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:39.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.803+0000 7f2ba0697640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b980b3e70 0x7f2b9806a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:39.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.803+0000 7f2ba0697640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b980b28e0 con 0x7f2b980b3070 2023-10-16T11:27:39.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.803+0000 7f2ba0697640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b980b65f0 con 0x7f2b980b3e70 2023-10-16T11:27:39.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.803+0000 7f2ba0697640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b980b2040 con 0x7f2b980b3450 2023-10-16T11:27:39.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.803+0000 7f2b9e433640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b980b3070 0x7f2b9815c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:39.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.804+0000 7f2b9e433640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b980b3070 0x7f2b9815c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45448/0 (socket says 172.21.15.84:45448) 2023-10-16T11:27:39.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.804+0000 7f2b9e433640 1 -- 172.21.15.84:0/477353778 learned_addr learned my addr 172.21.15.84:0/477353778 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:39.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.804+0000 7f2b9ec34640 1 --2- 172.21.15.84:0/477353778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b980b3e70 0x7f2b9806a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:39.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.804+0000 7f2b9dc32640 1 --2- 172.21.15.84:0/477353778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b980b3450 0x7f2b980cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:39.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.804+0000 7f2b9e433640 1 -- 172.21.15.84:0/477353778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b980b3450 msgr2=0x7f2b980cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:39.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.804+0000 7f2b9e433640 1 --2- 172.21.15.84:0/477353778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b980b3450 0x7f2b980cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:39.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.804+0000 7f2b9e433640 1 -- 172.21.15.84:0/477353778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b980b3e70 msgr2=0x7f2b9806a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:39.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.804+0000 7f2b9e433640 1 --2- 172.21.15.84:0/477353778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b980b3e70 0x7f2b9806a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:39.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.804+0000 7f2b9e433640 1 -- 172.21.15.84:0/477353778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b94009470 con 0x7f2b980b3070 2023-10-16T11:27:39.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.804+0000 7f2b9ec34640 1 --2- 172.21.15.84:0/477353778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b980b3e70 0x7f2b9806a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:27:39.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.804+0000 7f2b9dc32640 1 --2- 172.21.15.84:0/477353778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b980b3450 0x7f2b980cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:39.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.804+0000 7f2b9e433640 1 --2- 172.21.15.84:0/477353778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b980b3070 0x7f2b9815c2c0 secure :-1 s=READY pgs=3457 cs=0 l=1 rev1=1 crypto rx=0x7f2b94000c00 tx=0x7f2b940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:39.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.805+0000 7f2b8effd640 1 -- 172.21.15.84:0/477353778 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9403ac60 con 0x7f2b980b3070 2023-10-16T11:27:39.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.805+0000 7f2b8effd640 1 -- 172.21.15.84:0/477353778 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9403ade0 con 0x7f2b980b3070 2023-10-16T11:27:39.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.805+0000 7f2ba0697640 1 -- 172.21.15.84:0/477353778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b980b2320 con 0x7f2b980b3070 2023-10-16T11:27:39.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.805+0000 7f2b8effd640 1 -- 172.21.15.84:0/477353778 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94044630 con 0x7f2b980b3070 2023-10-16T11:27:39.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.805+0000 7f2ba0697640 1 -- 172.21.15.84:0/477353778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9815c800 con 0x7f2b980b3070 2023-10-16T11:27:39.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.806+0000 7f2ba0697640 1 -- 172.21.15.84:0/477353778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b980b6450 con 0x7f2b980b3070 2023-10-16T11:27:39.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.807+0000 7f2b8effd640 1 -- 172.21.15.84:0/477353778 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f2b9402f050 con 0x7f2b980b3070 2023-10-16T11:27:39.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.808+0000 7f2b8effd640 1 --2- 172.21.15.84:0/477353778 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f2b7c06af90 0x7f2b7c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:39.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.808+0000 7f2b8effd640 1 -- 172.21.15.84:0/477353778 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f2b940abb90 con 0x7f2b980b3070 2023-10-16T11:27:39.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.808+0000 7f2b9dc32640 1 -- 172.21.15.84:0/477353778 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f2b7c06af90 msgr2=0x7f2b7c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/129259578 2023-10-16T11:27:39.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.808+0000 7f2b9dc32640 1 --2- 172.21.15.84:0/477353778 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f2b7c06af90 0x7f2b7c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:39.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:39.812+0000 7f2b8effd640 1 -- 172.21.15.84:0/477353778 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2b980b6450 con 0x7f2b980b3070 2023-10-16T11:27:40.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.006+0000 7f2ba0697640 1 -- 172.21.15.84:0/477353778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b980b6450 con 0x7f2b980b3070 2023-10-16T11:27:40.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.008+0000 7f2b8effd640 1 -- 172.21.15.84:0/477353778 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v271) v1 ==== 74+0+17044 (secure 0 0 0) 0x7f2b980b6450 con 0x7f2b980b3070 2023-10-16T11:27:40.010 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:40.011 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":271,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:27:37.140352+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:27:20.716240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"269","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_data","create_time":"2023-10-16T11:27:21.905553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"270","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1788498161":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/3634760901":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/3842621119":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/40770371":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/1147031855":"2023-10-17T11:27:37.140228+0000","172.21.15.84:6833/731888425":"2023-10-17T11:27:37.140228+0000","172.21.15.84:6832/731888425":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/2353059609":"2023-10-17T11:27:37.140228+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:27:40.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.009+0000 7f2b9dc32640 1 -- 172.21.15.84:0/477353778 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f2b7c06af90 msgr2=0x7f2b7c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/129259578 2023-10-16T11:27:40.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.009+0000 7f2b9dc32640 1 --2- 172.21.15.84:0/477353778 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f2b7c06af90 0x7f2b7c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:27:40.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.010+0000 7f2ba0697640 1 -- 172.21.15.84:0/477353778 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f2b7c06af90 msgr2=0x7f2b7c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:40.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.010+0000 7f2ba0697640 1 --2- 172.21.15.84:0/477353778 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f2b7c06af90 0x7f2b7c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.010+0000 7f2ba0697640 1 -- 172.21.15.84:0/477353778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b980b3070 msgr2=0x7f2b9815c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:40.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.010+0000 7f2ba0697640 1 --2- 172.21.15.84:0/477353778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b980b3070 0x7f2b9815c2c0 secure :-1 s=READY pgs=3457 cs=0 l=1 rev1=1 crypto rx=0x7f2b94000c00 tx=0x7f2b940046c0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.010+0000 7f2ba0697640 1 -- 172.21.15.84:0/477353778 shutdown_connections 2023-10-16T11:27:40.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.010+0000 7f2ba0697640 1 --2- 172.21.15.84:0/477353778 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f2b7c06af90 0x7f2b7c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.010+0000 7f2ba0697640 1 --2- 172.21.15.84:0/477353778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b980b3e70 0x7f2b9806a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.010+0000 7f2ba0697640 1 --2- 172.21.15.84:0/477353778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b980b3450 0x7f2b980cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.010+0000 7f2ba0697640 1 --2- 172.21.15.84:0/477353778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b980b3070 0x7f2b9815c2c0 unknown :-1 s=CLOSED pgs=3457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.010+0000 7f2ba0697640 1 -- 172.21.15.84:0/477353778 >> 172.21.15.84:0/477353778 conn(0x7f2b980d2d20 msgr2=0x7f2b981605d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:40.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.011+0000 7f2ba0697640 1 -- 172.21.15.84:0/477353778 shutdown_connections 2023-10-16T11:27:40.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.011+0000 7f2ba0697640 1 -- 172.21.15.84:0/477353778 wait complete. 2023-10-16T11:27:40.030 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 36, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:27:20.716240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '269', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 37, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:27:21.905553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '270', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:27:40.031 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:27:40.234 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:40.233+0000 7f950da0f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:27:40.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.234+0000 7fb5630b4640 1 Processor -- start 2023-10-16T11:27:40.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.235+0000 7fb5630b4640 1 -- start start 2023-10-16T11:27:40.237 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:40.235+0000 7f338283b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:27:40.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.236+0000 7fb5630b4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb55c0caac0 0x7fb55c0ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:40.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.236+0000 7fb5630b4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55c0cb030 con 0x7fb55c15ad70 2023-10-16T11:27:40.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.236+0000 7fb5630b4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55c0c91c0 con 0x7fb55c06a4b0 2023-10-16T11:27:40.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.236+0000 7fb5630b4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c0c94a0 con 0x7fb55c0caac0 2023-10-16T11:27:40.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.236+0000 7fb560e50640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb55c15ad70 0x7fb55c069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52460/0 (socket says 172.21.15.84:52460) 2023-10-16T11:27:40.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.236+0000 7fb560e50640 1 -- 172.21.15.84:0/784552341 learned_addr learned my addr 172.21.15.84:0/784552341 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:40.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.237+0000 7fb561651640 1 --2- 172.21.15.84:0/784552341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb55c0caac0 0x7fb55c0ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:40.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.237+0000 7fb55b7fe640 1 -- 172.21.15.84:0/784552341 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913203629 0 0) 0x7fb55c0cb030 con 0x7fb55c15ad70 2023-10-16T11:27:40.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.237+0000 7fb55b7fe640 1 -- 172.21.15.84:0/784552341 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb53c003520 con 0x7fb55c15ad70 2023-10-16T11:27:40.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.237+0000 7fb55b7fe640 1 -- 172.21.15.84:0/784552341 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 988497922 0 0) 0x7fb53c003520 con 0x7fb55c15ad70 2023-10-16T11:27:40.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.237+0000 7fb55b7fe640 1 -- 172.21.15.84:0/784552341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb55c0caac0 msgr2=0x7fb55c0ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:40.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.237+0000 7fb55b7fe640 1 --2- 172.21.15.84:0/784552341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb55c0caac0 0x7fb55c0ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.237+0000 7fb55b7fe640 1 -- 172.21.15.84:0/784552341 >> v1:172.21.15.157:6789/0 conn(0x7fb55c06a4b0 legacy=0x7fb55c0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:40.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.237+0000 7fb55b7fe640 1 -- 172.21.15.84:0/784552341 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c0c9780 con 0x7fb55c15ad70 2023-10-16T11:27:40.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.237+0000 7fb55b7fe640 1 -- 172.21.15.84:0/784552341 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fb54c001670 con 0x7fb55c15ad70 2023-10-16T11:27:40.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.237+0000 7fb55b7fe640 1 -- 172.21.15.84:0/784552341 >> v1:172.21.15.84:6789/0 conn(0x7fb55c15ad70 legacy=0x7fb55c069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:40.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.237+0000 7fb55b7fe640 1 --2- 172.21.15.84:0/784552341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb53c0039a0 0x7fb53c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:40.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.237+0000 7fb55b7fe640 1 --2- 172.21.15.84:0/784552341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb53c0047c0 0x7fb53c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:40.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.238+0000 7fb55b7fe640 1 --2- 172.21.15.84:0/784552341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb55c0caac0 0x7fb53c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:40.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.238+0000 7fb55b7fe640 1 -- 172.21.15.84:0/784552341 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c0c94a0 con 0x7fb53c0039a0 2023-10-16T11:27:40.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.238+0000 7fb55b7fe640 1 -- 172.21.15.84:0/784552341 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c00ba60 con 0x7fb53c0047c0 2023-10-16T11:27:40.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.238+0000 7fb55b7fe640 1 -- 172.21.15.84:0/784552341 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c00bc20 con 0x7fb55c0caac0 2023-10-16T11:27:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.238+0000 7fb560e50640 1 --2- 172.21.15.84:0/784552341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb53c0039a0 0x7fb53c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.238+0000 7fb561651640 1 --2- 172.21.15.84:0/784552341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb55c0caac0 0x7fb53c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.238+0000 7fb55bfff640 1 --2- 172.21.15.84:0/784552341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb53c0047c0 0x7fb53c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.238+0000 7fb560e50640 1 -- 172.21.15.84:0/784552341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb55c0caac0 msgr2=0x7fb53c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.238+0000 7fb560e50640 1 --2- 172.21.15.84:0/784552341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb55c0caac0 0x7fb53c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.238+0000 7fb560e50640 1 -- 172.21.15.84:0/784552341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb53c0047c0 msgr2=0x7fb53c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.238+0000 7fb560e50640 1 --2- 172.21.15.84:0/784552341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb53c0047c0 0x7fb53c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.238+0000 7fb560e50640 1 -- 172.21.15.84:0/784552341 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53c003520 con 0x7fb53c0039a0 2023-10-16T11:27:40.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.238+0000 7fb55bfff640 1 --2- 172.21.15.84:0/784552341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb53c0047c0 0x7fb53c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:40.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.239+0000 7fb560e50640 1 --2- 172.21.15.84:0/784552341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb53c0039a0 0x7fb53c003dc0 secure :-1 s=READY pgs=3459 cs=0 l=1 rev1=1 crypto rx=0x7fb54c001eb0 tx=0x7fb53c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1748896ad949d377 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:40.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.240+0000 7fb55b7fe640 1 -- 172.21.15.84:0/784552341 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c005ad0 con 0x7fb53c0039a0 2023-10-16T11:27:40.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.240+0000 7fb55b7fe640 1 -- 172.21.15.84:0/784552341 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c005c30 con 0x7fb53c0039a0 2023-10-16T11:27:40.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.240+0000 7fb55b7fe640 1 -- 172.21.15.84:0/784552341 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c02c7c0 con 0x7fb53c0039a0 2023-10-16T11:27:40.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.240+0000 7fb5630b4640 1 -- 172.21.15.84:0/784552341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb53c0039a0 msgr2=0x7fb53c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:40.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.240+0000 7fb5630b4640 1 --2- 172.21.15.84:0/784552341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb53c0039a0 0x7fb53c003dc0 secure :-1 s=READY pgs=3459 cs=0 l=1 rev1=1 crypto rx=0x7fb54c001eb0 tx=0x7fb53c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.240+0000 7fb561651640 1 -- 172.21.15.84:0/784552341 reap_dead start 2023-10-16T11:27:40.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.240+0000 7fb5630b4640 1 -- 172.21.15.84:0/784552341 shutdown_connections 2023-10-16T11:27:40.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.240+0000 7fb5630b4640 1 -- 172.21.15.84:0/784552341 >> 172.21.15.84:0/784552341 conn(0x7fb55c0ce9b0 msgr2=0x7fb55c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:40.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.240+0000 7fb5630b4640 1 -- 172.21.15.84:0/784552341 shutdown_connections 2023-10-16T11:27:40.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.241+0000 7fb5630b4640 1 -- 172.21.15.84:0/784552341 wait complete. 2023-10-16T11:27:40.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.242+0000 7fb5630b4640 1 Processor -- start 2023-10-16T11:27:40.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.242+0000 7fb5630b4640 1 -- start start 2023-10-16T11:27:40.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.243+0000 7fb5630b4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb53c0039a0 0x7fb55c0a86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:40.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.243+0000 7fb5630b4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb55c0b0520 0x7fb55c0b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:40.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.243+0000 7fb5630b4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb55c0b0ee0 0x7fb55c1f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:40.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.243+0000 7fb5630b4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c0c2b10 con 0x7fb53c0039a0 2023-10-16T11:27:40.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.243+0000 7fb5630b4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c0c2d80 con 0x7fb55c0b0520 2023-10-16T11:27:40.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.243+0000 7fb5630b4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c0c3060 con 0x7fb55c0b0ee0 2023-10-16T11:27:40.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.244+0000 7fb560e50640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb53c0039a0 0x7fb55c0a86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:40.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.244+0000 7fb560e50640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb53c0039a0 0x7fb55c0a86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45452/0 (socket says 172.21.15.84:45452) 2023-10-16T11:27:40.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.244+0000 7fb560e50640 1 -- 172.21.15.84:0/1764583171 learned_addr learned my addr 172.21.15.84:0/1764583171 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:40.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.244+0000 7fb561651640 1 --2- 172.21.15.84:0/1764583171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb55c0b0ee0 0x7fb55c1f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:40.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.244+0000 7fb55bfff640 1 --2- 172.21.15.84:0/1764583171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb55c0b0520 0x7fb55c0b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:40.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.244+0000 7fb560e50640 1 -- 172.21.15.84:0/1764583171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb55c0b0ee0 msgr2=0x7fb55c1f36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:40.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.244+0000 7fb560e50640 1 --2- 172.21.15.84:0/1764583171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb55c0b0ee0 0x7fb55c1f36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.244+0000 7fb560e50640 1 -- 172.21.15.84:0/1764583171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb55c0b0520 msgr2=0x7fb55c0b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:40.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.244+0000 7fb560e50640 1 --2- 172.21.15.84:0/1764583171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb55c0b0520 0x7fb55c0b09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.244+0000 7fb560e50640 1 -- 172.21.15.84:0/1764583171 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c0c3340 con 0x7fb53c0039a0 2023-10-16T11:27:40.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.244+0000 7fb560e50640 1 --2- 172.21.15.84:0/1764583171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb53c0039a0 0x7fb55c0a86f0 secure :-1 s=READY pgs=3460 cs=0 l=1 rev1=1 crypto rx=0x7fb54c02c360 tx=0x7fb54c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:40.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.245+0000 7fb5597fa640 1 -- 172.21.15.84:0/1764583171 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c03c040 con 0x7fb53c0039a0 2023-10-16T11:27:40.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.245+0000 7fb5597fa640 1 -- 172.21.15.84:0/1764583171 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c02cbd0 con 0x7fb53c0039a0 2023-10-16T11:27:40.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.245+0000 7fb5630b4640 1 -- 172.21.15.84:0/1764583171 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55c0c3620 con 0x7fb53c0039a0 2023-10-16T11:27:40.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.245+0000 7fb5597fa640 1 -- 172.21.15.84:0/1764583171 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c007070 con 0x7fb53c0039a0 2023-10-16T11:27:40.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.245+0000 7fb5630b4640 1 -- 172.21.15.84:0/1764583171 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55c0a8e10 con 0x7fb53c0039a0 2023-10-16T11:27:40.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.246+0000 7fb5630b4640 1 -- 172.21.15.84:0/1764583171 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb55c04e160 con 0x7fb53c0039a0 2023-10-16T11:27:40.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.247+0000 7fb5597fa640 1 -- 172.21.15.84:0/1764583171 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fb54c032070 con 0x7fb53c0039a0 2023-10-16T11:27:40.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.248+0000 7fb5597fa640 1 --2- 172.21.15.84:0/1764583171 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7fb54006afc0 0x7fb54006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:40.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.249+0000 7fb5597fa640 1 -- 172.21.15.84:0/1764583171 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6081+0+0 (secure 0 0 0) 0x7fb54c0aa8b0 con 0x7fb53c0039a0 2023-10-16T11:27:40.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.249+0000 7fb55bfff640 1 -- 172.21.15.84:0/1764583171 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7fb54006afc0 msgr2=0x7fb54006d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/129259578 2023-10-16T11:27:40.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.249+0000 7fb55bfff640 1 --2- 172.21.15.84:0/1764583171 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7fb54006afc0 0x7fb54006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:40.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.253+0000 7fb5597fa640 1 -- 172.21.15.84:0/1764583171 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb54c0756d0 con 0x7fb53c0039a0 2023-10-16T11:27:40.288 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:40.287+0000 7f950da0f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:27:40.293 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:40.291+0000 7f338283b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:27:40.339 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:40.337+0000 7f950da0f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:27:40.345 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:40.343+0000 7f338283b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:27:40.400 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:40.398+0000 7f950da0f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:27:40.408 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:40.406+0000 7f338283b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:27:40.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.445+0000 7fb5630b4640 1 -- 172.21.15.84:0/1764583171 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb55c0a8e10 con 0x7fb53c0039a0 2023-10-16T11:27:40.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.445+0000 7fb5597fa640 1 -- 172.21.15.84:0/1764583171 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v271) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb55c0a8e10 con 0x7fb53c0039a0 2023-10-16T11:27:40.448 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:27:40.449 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:40.447+0000 7f950da0f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:27:40.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.448+0000 7fb5630b4640 1 -- 172.21.15.84:0/1764583171 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7fb54006afc0 msgr2=0x7fb54006d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:40.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.448+0000 7fb5630b4640 1 --2- 172.21.15.84:0/1764583171 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7fb54006afc0 0x7fb54006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.448+0000 7fb5630b4640 1 -- 172.21.15.84:0/1764583171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb53c0039a0 msgr2=0x7fb55c0a86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:40.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.448+0000 7fb5630b4640 1 --2- 172.21.15.84:0/1764583171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb53c0039a0 0x7fb55c0a86f0 secure :-1 s=READY pgs=3460 cs=0 l=1 rev1=1 crypto rx=0x7fb54c02c360 tx=0x7fb54c001930 comp rx=0 tx=0).stop 2023-10-16T11:27:40.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.448+0000 7fb5630b4640 1 -- 172.21.15.84:0/1764583171 shutdown_connections 2023-10-16T11:27:40.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.448+0000 7fb5630b4640 1 --2- 172.21.15.84:0/1764583171 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7fb54006afc0 0x7fb54006d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.448+0000 7fb5630b4640 1 --2- 172.21.15.84:0/1764583171 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb55c0b0ee0 0x7fb55c1f36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.448+0000 7fb5630b4640 1 --2- 172.21.15.84:0/1764583171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb55c0b0520 0x7fb55c0b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.448+0000 7fb5630b4640 1 --2- 172.21.15.84:0/1764583171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb53c0039a0 0x7fb55c0a86f0 unknown :-1 s=CLOSED pgs=3460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.448+0000 7fb5630b4640 1 -- 172.21.15.84:0/1764583171 >> 172.21.15.84:0/1764583171 conn(0x7fb55c0ce9b0 msgr2=0x7fb55c06b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:40.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.448+0000 7fb5630b4640 1 -- 172.21.15.84:0/1764583171 shutdown_connections 2023-10-16T11:27:40.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.448+0000 7fb5630b4640 1 -- 172.21.15.84:0/1764583171 wait complete. 2023-10-16T11:27:40.459 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:40.458+0000 7f338283b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:27:40.468 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:27:40.560 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:40.558+0000 7f950da0f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:27:40.580 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:40.578+0000 7f338283b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:27:40.622 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:40.620+0000 7f950da0f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:27:40.645 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:40.643+0000 7f338283b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:27:40.672 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:40.670+0000 7f950da0f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:27:40.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.675+0000 7faa21b87640 1 Processor -- start 2023-10-16T11:27:40.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.675+0000 7faa21b87640 1 -- start start 2023-10-16T11:27:40.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.676+0000 7faa21b87640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faa1c15c9f0 0x7faa1c0c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:40.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.676+0000 7faa21b87640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa1c0c9230 0x7faa1c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:40.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.676+0000 7faa21b87640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1c160a90 con 0x7faa1c070f20 2023-10-16T11:27:40.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.676+0000 7faa21b87640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c0cdec0 con 0x7faa1c15c9f0 2023-10-16T11:27:40.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.676+0000 7faa21b87640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c0cd900 con 0x7faa1c0c9230 2023-10-16T11:27:40.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.676+0000 7faa21386640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7faa1c070f20 0x7faa1c0ac890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52468/0 (socket says 172.21.15.84:52468) 2023-10-16T11:27:40.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa21386640 1 -- 172.21.15.84:0/860381043 learned_addr learned my addr 172.21.15.84:0/860381043 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:40.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa1bfff640 1 --2- 172.21.15.84:0/860381043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa1c0c9230 0x7faa1c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:40.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa20b85640 1 --2- 172.21.15.84:0/860381043 >> v2:172.21.15.157:3300/0 conn(0x7faa1c15c9f0 0x7faa1c0c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:40.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa1b7fe640 1 -- 172.21.15.84:0/860381043 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408556681 0 0) 0x7faa1c160a90 con 0x7faa1c070f20 2023-10-16T11:27:40.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa1b7fe640 1 -- 172.21.15.84:0/860381043 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa0c003520 con 0x7faa1c070f20 2023-10-16T11:27:40.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa1b7fe640 1 -- 172.21.15.84:0/860381043 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2351077605 0 0) 0x7faa0c003520 con 0x7faa1c070f20 2023-10-16T11:27:40.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa1b7fe640 1 -- 172.21.15.84:0/860381043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa1c0c9230 msgr2=0x7faa1c0709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:40.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa1b7fe640 1 --2- 172.21.15.84:0/860381043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa1c0c9230 0x7faa1c0709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa1b7fe640 1 -- 172.21.15.84:0/860381043 >> v2:172.21.15.157:3300/0 conn(0x7faa1c15c9f0 msgr2=0x7faa1c0c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:40.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa1b7fe640 1 --2- 172.21.15.84:0/860381043 >> v2:172.21.15.157:3300/0 conn(0x7faa1c15c9f0 0x7faa1c0c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa1b7fe640 1 -- 172.21.15.84:0/860381043 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c0cdbe0 con 0x7faa1c070f20 2023-10-16T11:27:40.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa1b7fe640 1 -- 172.21.15.84:0/860381043 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7faa140019f0 con 0x7faa1c070f20 2023-10-16T11:27:40.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa1b7fe640 1 -- 172.21.15.84:0/860381043 >> v1:172.21.15.84:6789/0 conn(0x7faa1c070f20 legacy=0x7faa1c0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:40.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa1b7fe640 1 --2- 172.21.15.84:0/860381043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa1c0c9230 0x7faa0c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:40.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa1b7fe640 1 --2- 172.21.15.84:0/860381043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa0c003f40 0x7faa0c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:40.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa1b7fe640 1 --2- 172.21.15.84:0/860381043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa0c004d00 0x7faa0c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:40.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa1b7fe640 1 -- 172.21.15.84:0/860381043 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c0cd900 con 0x7faa0c004d00 2023-10-16T11:27:40.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa1b7fe640 1 -- 172.21.15.84:0/860381043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c00b980 con 0x7faa0c003f40 2023-10-16T11:27:40.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.677+0000 7faa1b7fe640 1 -- 172.21.15.84:0/860381043 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c00bac0 con 0x7faa1c0c9230 2023-10-16T11:27:40.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.678+0000 7faa21386640 1 --2- 172.21.15.84:0/860381043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa0c004d00 0x7faa0c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:40.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.678+0000 7faa1bfff640 1 --2- 172.21.15.84:0/860381043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa0c003f40 0x7faa0c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:40.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.678+0000 7faa20b85640 1 --2- 172.21.15.84:0/860381043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa1c0c9230 0x7faa0c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:40.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.678+0000 7faa21386640 1 -- 172.21.15.84:0/860381043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa1c0c9230 msgr2=0x7faa0c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:40.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.678+0000 7faa21386640 1 --2- 172.21.15.84:0/860381043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa1c0c9230 0x7faa0c003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.678+0000 7faa21386640 1 -- 172.21.15.84:0/860381043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa0c003f40 msgr2=0x7faa0c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:40.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.678+0000 7faa21386640 1 --2- 172.21.15.84:0/860381043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa0c003f40 0x7faa0c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.678+0000 7faa21386640 1 -- 172.21.15.84:0/860381043 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c003520 con 0x7faa0c004d00 2023-10-16T11:27:40.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.679+0000 7faa21386640 1 --2- 172.21.15.84:0/860381043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa0c004d00 0x7faa0c005180 secure :-1 s=READY pgs=3462 cs=0 l=1 rev1=1 crypto rx=0x7faa1400a7c0 tx=0x7faa0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2663b99ba0cee420 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:40.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.680+0000 7faa1b7fe640 1 -- 172.21.15.84:0/860381043 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14034630 con 0x7faa0c004d00 2023-10-16T11:27:40.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.680+0000 7faa1b7fe640 1 -- 172.21.15.84:0/860381043 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1402f070 con 0x7faa0c004d00 2023-10-16T11:27:40.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.680+0000 7faa1b7fe640 1 -- 172.21.15.84:0/860381043 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14034a30 con 0x7faa0c004d00 2023-10-16T11:27:40.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.680+0000 7faa21b87640 1 -- 172.21.15.84:0/860381043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa0c004d00 msgr2=0x7faa0c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:40.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.680+0000 7faa21b87640 1 --2- 172.21.15.84:0/860381043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa0c004d00 0x7faa0c005180 secure :-1 s=READY pgs=3462 cs=0 l=1 rev1=1 crypto rx=0x7faa1400a7c0 tx=0x7faa0c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.680+0000 7faa21386640 1 -- 172.21.15.84:0/860381043 reap_dead start 2023-10-16T11:27:40.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.680+0000 7faa21b87640 1 -- 172.21.15.84:0/860381043 shutdown_connections 2023-10-16T11:27:40.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.680+0000 7faa21b87640 1 -- 172.21.15.84:0/860381043 >> 172.21.15.84:0/860381043 conn(0x7faa1c0ce6a0 msgr2=0x7faa1c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:40.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.680+0000 7faa21b87640 1 -- 172.21.15.84:0/860381043 shutdown_connections 2023-10-16T11:27:40.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.680+0000 7faa21b87640 1 -- 172.21.15.84:0/860381043 wait complete. 2023-10-16T11:27:40.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.682+0000 7faa21b87640 1 Processor -- start 2023-10-16T11:27:40.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.682+0000 7faa21b87640 1 -- start start 2023-10-16T11:27:40.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.683+0000 7faa21b87640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa0c004d00 0x7faa1c15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:40.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.683+0000 7faa21b87640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa1c153e60 0x7faa1c15f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:40.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.683+0000 7faa21b87640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1c14fe70 0x7faa1c0c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:40.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.683+0000 7faa21b87640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c06c530 con 0x7faa0c004d00 2023-10-16T11:27:40.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.683+0000 7faa21b87640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c15c250 con 0x7faa1c14fe70 2023-10-16T11:27:40.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.683+0000 7faa21b87640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c0af550 con 0x7faa1c153e60 2023-10-16T11:27:40.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.683+0000 7faa20b85640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa0c004d00 0x7faa1c15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:40.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.683+0000 7faa20b85640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa0c004d00 0x7faa1c15efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45468/0 (socket says 172.21.15.84:45468) 2023-10-16T11:27:40.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.683+0000 7faa20b85640 1 -- 172.21.15.84:0/761442457 learned_addr learned my addr 172.21.15.84:0/761442457 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:40.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.684+0000 7faa21386640 1 --2- 172.21.15.84:0/761442457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1c14fe70 0x7faa1c0c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:40.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.684+0000 7faa1bfff640 1 --2- 172.21.15.84:0/761442457 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa1c153e60 0x7faa1c15f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:40.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.684+0000 7faa20b85640 1 -- 172.21.15.84:0/761442457 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa1c153e60 msgr2=0x7faa1c15f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:40.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.684+0000 7faa20b85640 1 --2- 172.21.15.84:0/761442457 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa1c153e60 0x7faa1c15f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.684+0000 7faa20b85640 1 -- 172.21.15.84:0/761442457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1c14fe70 msgr2=0x7faa1c0c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:40.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.684+0000 7faa20b85640 1 --2- 172.21.15.84:0/761442457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1c14fe70 0x7faa1c0c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.684+0000 7faa20b85640 1 -- 172.21.15.84:0/761442457 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c06c150 con 0x7faa0c004d00 2023-10-16T11:27:40.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.684+0000 7faa1bfff640 1 --2- 172.21.15.84:0/761442457 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa1c153e60 0x7faa1c15f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:40.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.684+0000 7faa21386640 1 --2- 172.21.15.84:0/761442457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1c14fe70 0x7faa1c0c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:40.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.684+0000 7faa20b85640 1 --2- 172.21.15.84:0/761442457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa0c004d00 0x7faa1c15efa0 secure :-1 s=READY pgs=3463 cs=0 l=1 rev1=1 crypto rx=0x7faa100108e0 tx=0x7faa10010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:40.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.684+0000 7faa197fa640 1 -- 172.21.15.84:0/761442457 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1000eb60 con 0x7faa0c004d00 2023-10-16T11:27:40.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.684+0000 7faa197fa640 1 -- 172.21.15.84:0/761442457 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1000ecc0 con 0x7faa0c004d00 2023-10-16T11:27:40.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.684+0000 7faa197fa640 1 -- 172.21.15.84:0/761442457 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1001c400 con 0x7faa0c004d00 2023-10-16T11:27:40.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.684+0000 7faa21b87640 1 -- 172.21.15.84:0/761442457 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c15bee0 con 0x7faa0c004d00 2023-10-16T11:27:40.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.685+0000 7faa21b87640 1 -- 172.21.15.84:0/761442457 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c06cb30 con 0x7faa0c004d00 2023-10-16T11:27:40.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.686+0000 7faa21b87640 1 -- 172.21.15.84:0/761442457 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1c06c2e0 con 0x7faa0c004d00 2023-10-16T11:27:40.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.687+0000 7faa197fa640 1 -- 172.21.15.84:0/761442457 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7faa1001c560 con 0x7faa0c004d00 2023-10-16T11:27:40.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.688+0000 7faa197fa640 1 --2- 172.21.15.84:0/761442457 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7faa0006afc0 0x7faa0006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:40.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.688+0000 7faa1bfff640 1 -- 172.21.15.84:0/761442457 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7faa0006afc0 msgr2=0x7faa0006d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/129259578 2023-10-16T11:27:40.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.688+0000 7faa197fa640 1 -- 172.21.15.84:0/761442457 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6081+0+0 (secure 0 0 0) 0x7faa10089f40 con 0x7faa0c004d00 2023-10-16T11:27:40.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.688+0000 7faa1bfff640 1 --2- 172.21.15.84:0/761442457 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7faa0006afc0 0x7faa0006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:40.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.693+0000 7faa197fa640 1 -- 172.21.15.84:0/761442457 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7faa10054ea0 con 0x7faa0c004d00 2023-10-16T11:27:40.699 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:40.698+0000 7f338283b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:27:40.732 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:40.729+0000 7f950da0f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:27:40.759 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:40.757+0000 7f338283b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:27:40.782 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:40.780+0000 7f950da0f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:27:40.811 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:40.809+0000 7f338283b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:27:40.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.885+0000 7faa21b87640 1 -- 172.21.15.84:0/761442457 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faa1c04e9b0 con 0x7faa0c004d00 2023-10-16T11:27:40.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.886+0000 7faa197fa640 1 -- 172.21.15.84:0/761442457 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v271) v1 ==== 104+0+11 (secure 0 0 0) 0x7faa100584c0 con 0x7faa0c004d00 2023-10-16T11:27:40.888 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:27:40.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.889+0000 7faa1bfff640 1 -- 172.21.15.84:0/761442457 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7faa0006afc0 msgr2=0x7faa0006d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/129259578 2023-10-16T11:27:40.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.889+0000 7faa1bfff640 1 --2- 172.21.15.84:0/761442457 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7faa0006afc0 0x7faa0006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:27:40.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.889+0000 7faa21b87640 1 -- 172.21.15.84:0/761442457 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7faa0006afc0 msgr2=0x7faa0006d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:40.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.889+0000 7faa21b87640 1 --2- 172.21.15.84:0/761442457 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7faa0006afc0 0x7faa0006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.889+0000 7faa21b87640 1 -- 172.21.15.84:0/761442457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa0c004d00 msgr2=0x7faa1c15efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:40.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.889+0000 7faa21b87640 1 --2- 172.21.15.84:0/761442457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa0c004d00 0x7faa1c15efa0 secure :-1 s=READY pgs=3463 cs=0 l=1 rev1=1 crypto rx=0x7faa100108e0 tx=0x7faa10010da0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.889+0000 7faa21b87640 1 -- 172.21.15.84:0/761442457 shutdown_connections 2023-10-16T11:27:40.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.889+0000 7faa21b87640 1 --2- 172.21.15.84:0/761442457 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7faa0006afc0 0x7faa0006d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.889+0000 7faa21b87640 1 --2- 172.21.15.84:0/761442457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1c14fe70 0x7faa1c0c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.889+0000 7faa21b87640 1 --2- 172.21.15.84:0/761442457 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa1c153e60 0x7faa1c15f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.889+0000 7faa21b87640 1 --2- 172.21.15.84:0/761442457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa0c004d00 0x7faa1c15efa0 unknown :-1 s=CLOSED pgs=3463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:40.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.889+0000 7faa21b87640 1 -- 172.21.15.84:0/761442457 >> 172.21.15.84:0/761442457 conn(0x7faa1c0ce6a0 msgr2=0x7faa1c0ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:40.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.889+0000 7faa21b87640 1 -- 172.21.15.84:0/761442457 shutdown_connections 2023-10-16T11:27:40.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:40.889+0000 7faa21b87640 1 -- 172.21.15.84:0/761442457 wait complete. 2023-10-16T11:27:40.909 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:27:41.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.112+0000 7f712b1a0640 1 Processor -- start 2023-10-16T11:27:41.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.112+0000 7f712b1a0640 1 -- start start 2023-10-16T11:27:41.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.114+0000 7f712b1a0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7124070260 0x7f71240c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:41.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.114+0000 7f712b1a0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71240cdb70 0x7f71240ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:41.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.114+0000 7f712b1a0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71240c9e30 con 0x7f712415ce30 2023-10-16T11:27:41.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.114+0000 7f712b1a0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71240ce560 con 0x7f7124070260 2023-10-16T11:27:41.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.114+0000 7f712b1a0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71240ca110 con 0x7f71240cdb70 2023-10-16T11:27:41.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.114+0000 7f712999d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7124070260 0x7f71240c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:41.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.114+0000 7f712999d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7124070260 0x7f71240c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45480/0 (socket says 172.21.15.84:45480) 2023-10-16T11:27:41.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.114+0000 7f712999d640 1 -- 172.21.15.84:0/2312778590 learned_addr learned my addr 172.21.15.84:0/2312778590 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:41.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.115+0000 7f712a99f640 1 --2- 172.21.15.84:0/2312778590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71240cdb70 0x7f71240ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:41.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.115+0000 7f712999d640 1 -- 172.21.15.84:0/2312778590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71240cdb70 msgr2=0x7f71240ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:41.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.115+0000 7f712999d640 1 --2- 172.21.15.84:0/2312778590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71240cdb70 0x7f71240ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.115+0000 7f712999d640 1 -- 172.21.15.84:0/2312778590 >> v1:172.21.15.157:6789/0 conn(0x7f712415ce30 legacy=0x7f71240caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:41.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.115+0000 7f712999d640 1 -- 172.21.15.84:0/2312778590 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71240ca3f0 con 0x7f7124070260 2023-10-16T11:27:41.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.116+0000 7f712999d640 1 --2- 172.21.15.84:0/2312778590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7124070260 0x7f71240c95e0 secure :-1 s=READY pgs=3464 cs=0 l=1 rev1=1 crypto rx=0x7f7114009870 tx=0x7f711402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a723affd93625069 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:41.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.116+0000 7f712919c640 1 -- 172.21.15.84:0/2312778590 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7114031640 con 0x7f7124070260 2023-10-16T11:27:41.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.116+0000 7f712919c640 1 -- 172.21.15.84:0/2312778590 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71140317a0 con 0x7f7124070260 2023-10-16T11:27:41.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.116+0000 7f712919c640 1 -- 172.21.15.84:0/2312778590 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7114031b30 con 0x7f7124070260 2023-10-16T11:27:41.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.117+0000 7f712b1a0640 1 -- 172.21.15.84:0/2312778590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7124070260 msgr2=0x7f71240c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:41.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.117+0000 7f712b1a0640 1 --2- 172.21.15.84:0/2312778590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7124070260 0x7f71240c95e0 secure :-1 s=READY pgs=3464 cs=0 l=1 rev1=1 crypto rx=0x7f7114009870 tx=0x7f711402b8f0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.117+0000 7f712b1a0640 1 -- 172.21.15.84:0/2312778590 shutdown_connections 2023-10-16T11:27:41.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.117+0000 7f712b1a0640 1 --2- 172.21.15.84:0/2312778590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71240cdb70 0x7f71240ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.117+0000 7f712b1a0640 1 --2- 172.21.15.84:0/2312778590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7124070260 0x7f71240c95e0 unknown :-1 s=CLOSED pgs=3464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.117+0000 7f712b1a0640 1 -- 172.21.15.84:0/2312778590 >> 172.21.15.84:0/2312778590 conn(0x7f71240b0a80 msgr2=0x7f71240b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:41.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.117+0000 7f712b1a0640 1 -- 172.21.15.84:0/2312778590 shutdown_connections 2023-10-16T11:27:41.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.117+0000 7f712b1a0640 1 -- 172.21.15.84:0/2312778590 wait complete. 2023-10-16T11:27:41.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.119+0000 7f712b1a0640 1 Processor -- start 2023-10-16T11:27:41.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.119+0000 7f712b1a0640 1 -- start start 2023-10-16T11:27:41.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.120+0000 7f712b1a0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7124070260 0x7f712414e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:41.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.121+0000 7f712b1a0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71240cdb70 0x7f712406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:41.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.121+0000 7f712b1a0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f712415ce30 0x7f71240ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:41.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.121+0000 7f712b1a0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712406b970 con 0x7f7124070260 2023-10-16T11:27:41.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.121+0000 7f712b1a0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712406cba0 con 0x7f712415ce30 2023-10-16T11:27:41.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.121+0000 7f712b1a0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712406b1f0 con 0x7f71240cdb70 2023-10-16T11:27:41.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.121+0000 7f712a19e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7124070260 0x7f712414e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:41.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.121+0000 7f712a19e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7124070260 0x7f712414e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45496/0 (socket says 172.21.15.84:45496) 2023-10-16T11:27:41.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.121+0000 7f712a19e640 1 -- 172.21.15.84:0/180925807 learned_addr learned my addr 172.21.15.84:0/180925807 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:41.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.121+0000 7f712999d640 1 --2- 172.21.15.84:0/180925807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71240cdb70 0x7f712406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:41.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.121+0000 7f712a99f640 1 --2- 172.21.15.84:0/180925807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f712415ce30 0x7f71240ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:41.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.121+0000 7f712a19e640 1 -- 172.21.15.84:0/180925807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71240cdb70 msgr2=0x7f712406c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:41.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.121+0000 7f712a19e640 1 --2- 172.21.15.84:0/180925807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71240cdb70 0x7f712406c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.121+0000 7f712a19e640 1 -- 172.21.15.84:0/180925807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f712415ce30 msgr2=0x7f71240ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:41.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.121+0000 7f712a19e640 1 --2- 172.21.15.84:0/180925807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f712415ce30 0x7f71240ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.121+0000 7f712a19e640 1 -- 172.21.15.84:0/180925807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71140094d0 con 0x7f7124070260 2023-10-16T11:27:41.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.122+0000 7f712a19e640 1 --2- 172.21.15.84:0/180925807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7124070260 0x7f712414e7c0 secure :-1 s=READY pgs=3465 cs=0 l=1 rev1=1 crypto rx=0x7f712000c400 tx=0x7f712000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:41.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.122+0000 7f711affd640 1 -- 172.21.15.84:0/180925807 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71200052d0 con 0x7f7124070260 2023-10-16T11:27:41.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.122+0000 7f711affd640 1 -- 172.21.15.84:0/180925807 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7120005430 con 0x7f7124070260 2023-10-16T11:27:41.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.122+0000 7f712b1a0640 1 -- 172.21.15.84:0/180925807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712406b4d0 con 0x7f7124070260 2023-10-16T11:27:41.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.122+0000 7f711affd640 1 -- 172.21.15.84:0/180925807 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120003830 con 0x7f7124070260 2023-10-16T11:27:41.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.122+0000 7f712b1a0640 1 -- 172.21.15.84:0/180925807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712406be60 con 0x7f7124070260 2023-10-16T11:27:41.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.123+0000 7f712b1a0640 1 -- 172.21.15.84:0/180925807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712404e9b0 con 0x7f7124070260 2023-10-16T11:27:41.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.124+0000 7f711affd640 1 -- 172.21.15.84:0/180925807 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f7120003990 con 0x7f7124070260 2023-10-16T11:27:41.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.125+0000 7f711affd640 1 --2- 172.21.15.84:0/180925807 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f710806afc0 0x7f710806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:41.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.125+0000 7f711affd640 1 -- 172.21.15.84:0/180925807 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f7120088e30 con 0x7f7124070260 2023-10-16T11:27:41.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.125+0000 7f712999d640 1 -- 172.21.15.84:0/180925807 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f710806afc0 msgr2=0x7f710806d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/129259578 2023-10-16T11:27:41.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.125+0000 7f712999d640 1 --2- 172.21.15.84:0/180925807 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f710806afc0 0x7f710806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:41.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.129+0000 7f711affd640 1 -- 172.21.15.84:0/180925807 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7120053c50 con 0x7f7124070260 2023-10-16T11:27:41.168 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:41.166+0000 7f950da0f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:27:41.211 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:41.209+0000 7f338283b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:27:41.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.314+0000 7f712b1a0640 1 -- 172.21.15.84:0/180925807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7124065d60 con 0x7f7124070260 2023-10-16T11:27:41.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.315+0000 7f711affd640 1 -- 172.21.15.84:0/180925807 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v271) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7120017020 con 0x7f7124070260 2023-10-16T11:27:41.318 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:27:41.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.317+0000 7f712b1a0640 1 -- 172.21.15.84:0/180925807 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f710806afc0 msgr2=0x7f710806d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:41.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.317+0000 7f712b1a0640 1 --2- 172.21.15.84:0/180925807 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f710806afc0 0x7f710806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.317+0000 7f712b1a0640 1 -- 172.21.15.84:0/180925807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7124070260 msgr2=0x7f712414e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:41.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.317+0000 7f712b1a0640 1 --2- 172.21.15.84:0/180925807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7124070260 0x7f712414e7c0 secure :-1 s=READY pgs=3465 cs=0 l=1 rev1=1 crypto rx=0x7f712000c400 tx=0x7f712000c8c0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.318+0000 7f712b1a0640 1 -- 172.21.15.84:0/180925807 shutdown_connections 2023-10-16T11:27:41.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.318+0000 7f712b1a0640 1 --2- 172.21.15.84:0/180925807 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f710806afc0 0x7f710806d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.318+0000 7f712b1a0640 1 --2- 172.21.15.84:0/180925807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f712415ce30 0x7f71240ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.318+0000 7f712b1a0640 1 --2- 172.21.15.84:0/180925807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71240cdb70 0x7f712406c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.318+0000 7f712b1a0640 1 --2- 172.21.15.84:0/180925807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7124070260 0x7f712414e7c0 unknown :-1 s=CLOSED pgs=3465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.318+0000 7f712b1a0640 1 -- 172.21.15.84:0/180925807 >> 172.21.15.84:0/180925807 conn(0x7f71240b0a80 msgr2=0x7f71240b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:41.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.318+0000 7f712b1a0640 1 -- 172.21.15.84:0/180925807 shutdown_connections 2023-10-16T11:27:41.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.318+0000 7f712b1a0640 1 -- 172.21.15.84:0/180925807 wait complete. 2023-10-16T11:27:41.330 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:41.328+0000 7f950da0f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:27:41.337 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T11:27:41.378 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:41.376+0000 7f338283b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:27:41.441 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:41.438+0000 7f950da0f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:27:41.490 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:41.489+0000 7f950da0f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:27:41.491 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:41.489+0000 7f338283b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:27:41.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.534+0000 7f8370814640 1 Processor -- start 2023-10-16T11:27:41.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.534+0000 7f8370814640 1 -- start start 2023-10-16T11:27:41.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.535+0000 7f8370814640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f836c15cd70 0x7f836c0d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:41.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.535+0000 7f8370814640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f836c0b8480 0x7f836c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:41.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.535+0000 7f8370814640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f836c0d0be0 0x7f836c0cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:41.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.535+0000 7f8370814640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f836c15d150 con 0x7f836c15cd70 2023-10-16T11:27:41.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.535+0000 7f8370814640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f836c0d1180 con 0x7f836c0d0be0 2023-10-16T11:27:41.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.535+0000 7f8370814640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836c0d1460 con 0x7f836c0b8480 2023-10-16T11:27:41.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.535+0000 7f836ad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f836c15cd70 0x7f836c0d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:41.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.535+0000 7f836ad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f836c15cd70 0x7f836c0d2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45506/0 (socket says 172.21.15.84:45506) 2023-10-16T11:27:41.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.535+0000 7f836ad9d640 1 -- 172.21.15.84:0/2658906213 learned_addr learned my addr 172.21.15.84:0/2658906213 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:41.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.536+0000 7f836a59c640 1 --2- 172.21.15.84:0/2658906213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f836c0b8480 0x7f836c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:41.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.536+0000 7f836b59e640 1 --2- 172.21.15.84:0/2658906213 >> v2:172.21.15.157:3300/0 conn(0x7f836c0d0be0 0x7f836c0cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:41.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.536+0000 7f836ad9d640 1 -- 172.21.15.84:0/2658906213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f836c0b8480 msgr2=0x7f836c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:41.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.536+0000 7f836ad9d640 1 --2- 172.21.15.84:0/2658906213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f836c0b8480 0x7f836c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.536+0000 7f836ad9d640 1 -- 172.21.15.84:0/2658906213 >> v2:172.21.15.157:3300/0 conn(0x7f836c0d0be0 msgr2=0x7f836c0cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:41.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.536+0000 7f836ad9d640 1 --2- 172.21.15.84:0/2658906213 >> v2:172.21.15.157:3300/0 conn(0x7f836c0d0be0 0x7f836c0cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.536+0000 7f836ad9d640 1 -- 172.21.15.84:0/2658906213 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836c0d1740 con 0x7f836c15cd70 2023-10-16T11:27:41.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.536+0000 7f836b59e640 1 --2- 172.21.15.84:0/2658906213 >> v2:172.21.15.157:3300/0 conn(0x7f836c0d0be0 0x7f836c0cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:41.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.537+0000 7f836ad9d640 1 --2- 172.21.15.84:0/2658906213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f836c15cd70 0x7f836c0d2220 secure :-1 s=READY pgs=3466 cs=0 l=1 rev1=1 crypto rx=0x7f836c0b2520 tx=0x7f835c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df7587afbc7476a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:41.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.537+0000 7f8369d9b640 1 -- 172.21.15.84:0/2658906213 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c031870 con 0x7f836c15cd70 2023-10-16T11:27:41.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.537+0000 7f8369d9b640 1 -- 172.21.15.84:0/2658906213 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835c0319d0 con 0x7f836c15cd70 2023-10-16T11:27:41.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.537+0000 7f8369d9b640 1 -- 172.21.15.84:0/2658906213 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c031d60 con 0x7f836c15cd70 2023-10-16T11:27:41.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.538+0000 7f8370814640 1 -- 172.21.15.84:0/2658906213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f836c15cd70 msgr2=0x7f836c0d2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:41.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.538+0000 7f8370814640 1 --2- 172.21.15.84:0/2658906213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f836c15cd70 0x7f836c0d2220 secure :-1 s=READY pgs=3466 cs=0 l=1 rev1=1 crypto rx=0x7f836c0b2520 tx=0x7f835c02b910 comp rx=0 tx=0).stop 2023-10-16T11:27:41.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.538+0000 7f8370814640 1 -- 172.21.15.84:0/2658906213 shutdown_connections 2023-10-16T11:27:41.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.538+0000 7f8370814640 1 --2- 172.21.15.84:0/2658906213 >> v2:172.21.15.157:3300/0 conn(0x7f836c0d0be0 0x7f836c0cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.538+0000 7f8370814640 1 --2- 172.21.15.84:0/2658906213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f836c0b8480 0x7f836c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.538+0000 7f8370814640 1 --2- 172.21.15.84:0/2658906213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f836c15cd70 0x7f836c0d2220 unknown :-1 s=CLOSED pgs=3466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.538+0000 7f8370814640 1 -- 172.21.15.84:0/2658906213 >> 172.21.15.84:0/2658906213 conn(0x7f836c0cf570 msgr2=0x7f836c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:41.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.538+0000 7f8370814640 1 -- 172.21.15.84:0/2658906213 shutdown_connections 2023-10-16T11:27:41.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.538+0000 7f8370814640 1 -- 172.21.15.84:0/2658906213 wait complete. 2023-10-16T11:27:41.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.540+0000 7f8370814640 1 Processor -- start 2023-10-16T11:27:41.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.540+0000 7f8370814640 1 -- start start 2023-10-16T11:27:41.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.541+0000 7f8370814640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f836c0b8480 0x7f836c0ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:41.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.541+0000 7f8370814640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f836c0d0be0 0x7f836c15f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:41.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.541+0000 7f8370814640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f836c15cd70 0x7f836c0c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:41.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.541+0000 7f8370814640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c0c74a0 con 0x7f836c15cd70 2023-10-16T11:27:41.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.541+0000 7f8370814640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c0c76b0 con 0x7f836c0b8480 2023-10-16T11:27:41.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.541+0000 7f8370814640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836c0c7990 con 0x7f836c0d0be0 2023-10-16T11:27:41.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.541+0000 7f836b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f836c15cd70 0x7f836c0c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:41.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.541+0000 7f836b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f836c15cd70 0x7f836c0c8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45510/0 (socket says 172.21.15.84:45510) 2023-10-16T11:27:41.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.541+0000 7f836b59e640 1 -- 172.21.15.84:0/525824126 learned_addr learned my addr 172.21.15.84:0/525824126 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:41.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.541+0000 7f836a59c640 1 --2- 172.21.15.84:0/525824126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f836c0d0be0 0x7f836c15f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:41.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.541+0000 7f836ad9d640 1 --2- 172.21.15.84:0/525824126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f836c0b8480 0x7f836c0ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:41.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.541+0000 7f836b59e640 1 -- 172.21.15.84:0/525824126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f836c0d0be0 msgr2=0x7f836c15f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:41.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.541+0000 7f836b59e640 1 --2- 172.21.15.84:0/525824126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f836c0d0be0 0x7f836c15f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.541+0000 7f836b59e640 1 -- 172.21.15.84:0/525824126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f836c0b8480 msgr2=0x7f836c0ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:41.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.542+0000 7f836b59e640 1 --2- 172.21.15.84:0/525824126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f836c0b8480 0x7f836c0ca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.542+0000 7f836b59e640 1 -- 172.21.15.84:0/525824126 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835c0094d0 con 0x7f836c15cd70 2023-10-16T11:27:41.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.542+0000 7f836ad9d640 1 --2- 172.21.15.84:0/525824126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f836c0b8480 0x7f836c0ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:27:41.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.542+0000 7f836b59e640 1 --2- 172.21.15.84:0/525824126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f836c15cd70 0x7f836c0c8750 secure :-1 s=READY pgs=3467 cs=0 l=1 rev1=1 crypto rx=0x7f836400ce40 tx=0x7f836400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:41.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.542+0000 7f83537fe640 1 -- 172.21.15.84:0/525824126 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836400d840 con 0x7f836c15cd70 2023-10-16T11:27:41.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.542+0000 7f83537fe640 1 -- 172.21.15.84:0/525824126 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836400d9a0 con 0x7f836c15cd70 2023-10-16T11:27:41.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.542+0000 7f83537fe640 1 -- 172.21.15.84:0/525824126 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836400dd50 con 0x7f836c15cd70 2023-10-16T11:27:41.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.542+0000 7f8370814640 1 -- 172.21.15.84:0/525824126 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836c0c7c70 con 0x7f836c15cd70 2023-10-16T11:27:41.565 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:41.542+0000 7f338283b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:27:41.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.542+0000 7f8370814640 1 -- 172.21.15.84:0/525824126 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836c0c8230 con 0x7f836c15cd70 2023-10-16T11:27:41.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.543+0000 7f8370814640 1 -- 172.21.15.84:0/525824126 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f836c0c8420 con 0x7f836c15cd70 2023-10-16T11:27:41.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.544+0000 7f83537fe640 1 -- 172.21.15.84:0/525824126 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f836401e3b0 con 0x7f836c15cd70 2023-10-16T11:27:41.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.545+0000 7f83537fe640 1 --2- 172.21.15.84:0/525824126 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f835806af90 0x7f835806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:41.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.545+0000 7f83537fe640 1 -- 172.21.15.84:0/525824126 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f836408a290 con 0x7f836c15cd70 2023-10-16T11:27:41.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.545+0000 7f836ad9d640 1 -- 172.21.15.84:0/525824126 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f835806af90 msgr2=0x7f835806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/129259578 2023-10-16T11:27:41.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.545+0000 7f836ad9d640 1 --2- 172.21.15.84:0/525824126 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f835806af90 0x7f835806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:41.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.549+0000 7f83537fe640 1 -- 172.21.15.84:0/525824126 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f83640550c0 con 0x7f836c15cd70 2023-10-16T11:27:41.600 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:41.598+0000 7f950da0f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:27:41.654 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:41.652+0000 7f338283b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:27:41.707 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:41.705+0000 7f950da0f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:27:41.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.746+0000 7f836ad9d640 1 -- 172.21.15.84:0/525824126 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f835806af90 msgr2=0x7f835806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/129259578 2023-10-16T11:27:41.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.746+0000 7f836ad9d640 1 --2- 172.21.15.84:0/525824126 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f835806af90 0x7f835806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:27:41.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.751+0000 7f8370814640 1 -- 172.21.15.84:0/525824126 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f836c04e9b0 con 0x7f836c15cd70 2023-10-16T11:27:41.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.751+0000 7f83537fe640 1 -- 172.21.15.84:0/525824126 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v271) v1 ==== 80+0+466 (secure 0 0 0) 0x7f83640586e0 con 0x7f836c15cd70 2023-10-16T11:27:41.754 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1788498161 2023-10-17T11:27:37.140228+0000 2023-10-16T11:27:41.754 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3634760901 2023-10-17T11:27:37.140228+0000 2023-10-16T11:27:41.754 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3842621119 2023-10-17T11:27:37.140228+0000 2023-10-16T11:27:41.754 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/40770371 2023-10-17T11:27:37.140228+0000 2023-10-16T11:27:41.755 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1147031855 2023-10-17T11:27:37.140228+0000 2023-10-16T11:27:41.755 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6833/731888425 2023-10-17T11:27:37.140228+0000 2023-10-16T11:27:41.755 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6832/731888425 2023-10-17T11:27:37.140228+0000 2023-10-16T11:27:41.756 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2353059609 2023-10-17T11:27:37.140228+0000 2023-10-16T11:27:41.757 INFO:teuthology.orchestra.run.smithi084.stderr:listed 8 entries 2023-10-16T11:27:41.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.754+0000 7f8370814640 1 -- 172.21.15.84:0/525824126 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f835806af90 msgr2=0x7f835806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:41.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.754+0000 7f8370814640 1 --2- 172.21.15.84:0/525824126 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f835806af90 0x7f835806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.754+0000 7f8370814640 1 -- 172.21.15.84:0/525824126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f836c15cd70 msgr2=0x7f836c0c8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:41.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.754+0000 7f8370814640 1 --2- 172.21.15.84:0/525824126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f836c15cd70 0x7f836c0c8750 secure :-1 s=READY pgs=3467 cs=0 l=1 rev1=1 crypto rx=0x7f836400ce40 tx=0x7f836400a710 comp rx=0 tx=0).stop 2023-10-16T11:27:41.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.754+0000 7f8370814640 1 -- 172.21.15.84:0/525824126 shutdown_connections 2023-10-16T11:27:41.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.754+0000 7f8370814640 1 --2- 172.21.15.84:0/525824126 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f835806af90 0x7f835806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.754+0000 7f8370814640 1 --2- 172.21.15.84:0/525824126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f836c15cd70 0x7f836c0c8750 unknown :-1 s=CLOSED pgs=3467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.754+0000 7f8370814640 1 --2- 172.21.15.84:0/525824126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f836c0d0be0 0x7f836c15f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.754+0000 7f8370814640 1 --2- 172.21.15.84:0/525824126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f836c0b8480 0x7f836c0ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:41.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.754+0000 7f8370814640 1 -- 172.21.15.84:0/525824126 >> 172.21.15.84:0/525824126 conn(0x7f836c0cf570 msgr2=0x7f836c0cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:41.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.754+0000 7f8370814640 1 -- 172.21.15.84:0/525824126 shutdown_connections 2023-10-16T11:27:41.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:41.755+0000 7f8370814640 1 -- 172.21.15.84:0/525824126 wait complete. 2023-10-16T11:27:41.763 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:41.761+0000 7f338283b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:27:41.774 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T11:27:41.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:41.775 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T11:27:41.864 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:41.862+0000 7f950da0f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:27:41.915 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:41.913+0000 7f950da0f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:27:41.917 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T11:27:41.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:41.918 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T11:27:41.927 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:41.925+0000 7f338283b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:27:41.962 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:27:41.963 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:41.979 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:41.978+0000 7f338283b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:27:42.018 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:27:42.018 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:27:42.019 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:42.041 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:42.040+0000 7f950da0f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:27:42.072 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:27:42.073 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:27:42.073 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:42.107 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:42.104+0000 7f338283b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:27:42.127 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:27:42.128 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:27:42.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.332+0000 7f3b062ec640 1 Processor -- start 2023-10-16T11:27:42.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.332+0000 7f3b062ec640 1 -- start start 2023-10-16T11:27:42.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.333+0000 7f3b062ec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b0006c040 0x7f3b0006c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:42.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.333+0000 7f3b062ec640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b000cda80 0x7f3b0015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:42.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.333+0000 7f3b062ec640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0006c960 con 0x7f3b000ad970 2023-10-16T11:27:42.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.333+0000 7f3b062ec640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b0015b9d0 con 0x7f3b000cda80 2023-10-16T11:27:42.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.334+0000 7f3b062ec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0015bcb0 con 0x7f3b0006c040 2023-10-16T11:27:42.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.334+0000 7f3b04ae9640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3b000ad970 0x7f3b0015e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52472/0 (socket says 172.21.15.84:52472) 2023-10-16T11:27:42.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.334+0000 7f3b04ae9640 1 -- 172.21.15.84:0/262061630 learned_addr learned my addr 172.21.15.84:0/262061630 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:42.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.334+0000 7f3b052ea640 1 --2- 172.21.15.84:0/262061630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b0006c040 0x7f3b0006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:42.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.334+0000 7f3af7fff640 1 -- 172.21.15.84:0/262061630 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508386590 0 0) 0x7f3b0006c960 con 0x7f3b000ad970 2023-10-16T11:27:42.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.334+0000 7f3af7fff640 1 -- 172.21.15.84:0/262061630 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aec003520 con 0x7f3b000ad970 2023-10-16T11:27:42.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.334+0000 7f3b05aeb640 1 --2- 172.21.15.84:0/262061630 >> v2:172.21.15.157:3300/0 conn(0x7f3b000cda80 0x7f3b0015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:42.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.335+0000 7f3af7fff640 1 -- 172.21.15.84:0/262061630 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 132180643 0 0) 0x7f3aec003520 con 0x7f3b000ad970 2023-10-16T11:27:42.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.335+0000 7f3af7fff640 1 -- 172.21.15.84:0/262061630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b0006c040 msgr2=0x7f3b0006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:42.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.335+0000 7f3af7fff640 1 --2- 172.21.15.84:0/262061630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b0006c040 0x7f3b0006c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:42.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.335+0000 7f3af7fff640 1 -- 172.21.15.84:0/262061630 >> v2:172.21.15.157:3300/0 conn(0x7f3b000cda80 msgr2=0x7f3b0015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:42.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.335+0000 7f3af7fff640 1 --2- 172.21.15.84:0/262061630 >> v2:172.21.15.157:3300/0 conn(0x7f3b000cda80 0x7f3b0015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:42.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.335+0000 7f3af7fff640 1 -- 172.21.15.84:0/262061630 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0015bf90 con 0x7f3b000ad970 2023-10-16T11:27:42.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.335+0000 7f3af7fff640 1 -- 172.21.15.84:0/262061630 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f3af0001670 con 0x7f3b000ad970 2023-10-16T11:27:42.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.335+0000 7f3af7fff640 1 -- 172.21.15.84:0/262061630 >> v1:172.21.15.84:6789/0 conn(0x7f3b000ad970 legacy=0x7f3b0015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:42.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.335+0000 7f3af7fff640 1 --2- 172.21.15.84:0/262061630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aec003970 0x7f3aec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:42.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.335+0000 7f3af7fff640 1 --2- 172.21.15.84:0/262061630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b0006c040 0x7f3aec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:42.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.335+0000 7f3af7fff640 1 --2- 172.21.15.84:0/262061630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3aec004d00 0x7f3aec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:42.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.335+0000 7f3af7fff640 1 -- 172.21.15.84:0/262061630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0015b9d0 con 0x7f3aec003970 2023-10-16T11:27:42.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.335+0000 7f3af7fff640 1 -- 172.21.15.84:0/262061630 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0015bcb0 con 0x7f3aec004d00 2023-10-16T11:27:42.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.335+0000 7f3af7fff640 1 -- 172.21.15.84:0/262061630 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec00b980 con 0x7f3b0006c040 2023-10-16T11:27:42.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.335+0000 7f3b052ea640 1 --2- 172.21.15.84:0/262061630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aec003970 0x7f3aec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:42.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.336+0000 7f3b04ae9640 1 --2- 172.21.15.84:0/262061630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b0006c040 0x7f3aec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:42.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.336+0000 7f3b052ea640 1 -- 172.21.15.84:0/262061630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b0006c040 msgr2=0x7f3aec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:42.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.336+0000 7f3b052ea640 1 --2- 172.21.15.84:0/262061630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b0006c040 0x7f3aec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:42.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.336+0000 7f3b05aeb640 1 --2- 172.21.15.84:0/262061630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3aec004d00 0x7f3aec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.336+0000 7f3b052ea640 1 -- 172.21.15.84:0/262061630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3aec004d00 msgr2=0x7f3aec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.336+0000 7f3b052ea640 1 --2- 172.21.15.84:0/262061630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3aec004d00 0x7f3aec005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.336+0000 7f3b052ea640 1 -- 172.21.15.84:0/262061630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec003520 con 0x7f3aec003970 2023-10-16T11:27:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.336+0000 7f3b04ae9640 1 --2- 172.21.15.84:0/262061630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b0006c040 0x7f3aec0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.337+0000 7f3b052ea640 1 --2- 172.21.15.84:0/262061630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aec003970 0x7f3aec003d90 secure :-1 s=READY pgs=3469 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009ac0 tx=0x7f3aec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de94043c14f7a3ca server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.337+0000 7f3af7fff640 1 -- 172.21.15.84:0/262061630 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc02c6b0 con 0x7f3aec003970 2023-10-16T11:27:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.337+0000 7f3af7fff640 1 -- 172.21.15.84:0/262061630 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3afc02c810 con 0x7f3aec003970 2023-10-16T11:27:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.337+0000 7f3af7fff640 1 -- 172.21.15.84:0/262061630 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc02cba0 con 0x7f3aec003970 2023-10-16T11:27:42.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.338+0000 7f3b062ec640 1 -- 172.21.15.84:0/262061630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aec003970 msgr2=0x7f3aec003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:42.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.338+0000 7f3b062ec640 1 --2- 172.21.15.84:0/262061630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aec003970 0x7f3aec003d90 secure :-1 s=READY pgs=3469 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009ac0 tx=0x7f3aec0057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:42.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.338+0000 7f3b05aeb640 1 -- 172.21.15.84:0/262061630 reap_dead start 2023-10-16T11:27:42.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.338+0000 7f3b062ec640 1 -- 172.21.15.84:0/262061630 shutdown_connections 2023-10-16T11:27:42.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.338+0000 7f3b062ec640 1 -- 172.21.15.84:0/262061630 >> 172.21.15.84:0/262061630 conn(0x7f3b000ce6a0 msgr2=0x7f3b000ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:42.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.338+0000 7f3b062ec640 1 -- 172.21.15.84:0/262061630 shutdown_connections 2023-10-16T11:27:42.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.338+0000 7f3b062ec640 1 -- 172.21.15.84:0/262061630 wait complete. 2023-10-16T11:27:42.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.340+0000 7f3b062ec640 1 Processor -- start 2023-10-16T11:27:42.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.340+0000 7f3b062ec640 1 -- start start 2023-10-16T11:27:42.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.341+0000 7f3b062ec640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3aec003970 0x7f3b000c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:42.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.341+0000 7f3b062ec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b001f1fb0 0x7f3b00150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:42.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.341+0000 7f3b062ec640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b000ca050 0x7f3b00151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:42.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.341+0000 7f3b062ec640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b000c8d90 con 0x7f3b001f1fb0 2023-10-16T11:27:42.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.341+0000 7f3b062ec640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b000c9030 con 0x7f3aec003970 2023-10-16T11:27:42.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.341+0000 7f3b062ec640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b000c9310 con 0x7f3b000ca050 2023-10-16T11:27:42.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.341+0000 7f3b04ae9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b001f1fb0 0x7f3b00150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:42.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.341+0000 7f3b04ae9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b001f1fb0 0x7f3b00150d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45530/0 (socket says 172.21.15.84:45530) 2023-10-16T11:27:42.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.341+0000 7f3b04ae9640 1 -- 172.21.15.84:0/3469260408 learned_addr learned my addr 172.21.15.84:0/3469260408 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:42.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.341+0000 7f3b04ae9640 1 -- 172.21.15.84:0/3469260408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b000ca050 msgr2=0x7f3b00151250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:42.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.341+0000 7f3b04ae9640 1 --2- 172.21.15.84:0/3469260408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b000ca050 0x7f3b00151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:42.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.341+0000 7f3b04ae9640 1 -- 172.21.15.84:0/3469260408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3aec003970 msgr2=0x7f3b000c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:42.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.341+0000 7f3b04ae9640 1 --2- 172.21.15.84:0/3469260408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3aec003970 0x7f3b000c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:42.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.341+0000 7f3b04ae9640 1 -- 172.21.15.84:0/3469260408 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b000c95f0 con 0x7f3b001f1fb0 2023-10-16T11:27:42.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.342+0000 7f3b04ae9640 1 --2- 172.21.15.84:0/3469260408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b001f1fb0 0x7f3b00150d10 secure :-1 s=READY pgs=3470 cs=0 l=1 rev1=1 crypto rx=0x7f3af000d770 tx=0x7f3af000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:42.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.342+0000 7f3af5ffb640 1 -- 172.21.15.84:0/3469260408 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af00074e0 con 0x7f3b001f1fb0 2023-10-16T11:27:42.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.342+0000 7f3af5ffb640 1 -- 172.21.15.84:0/3469260408 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af0007640 con 0x7f3b001f1fb0 2023-10-16T11:27:42.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.342+0000 7f3af5ffb640 1 -- 172.21.15.84:0/3469260408 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af0004a20 con 0x7f3b001f1fb0 2023-10-16T11:27:42.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.342+0000 7f3b062ec640 1 -- 172.21.15.84:0/3469260408 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b000c6460 con 0x7f3b001f1fb0 2023-10-16T11:27:42.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.342+0000 7f3b062ec640 1 -- 172.21.15.84:0/3469260408 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b000aca70 con 0x7f3b001f1fb0 2023-10-16T11:27:42.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.343+0000 7f3b062ec640 1 -- 172.21.15.84:0/3469260408 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0006b330 con 0x7f3b001f1fb0 2023-10-16T11:27:42.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.345+0000 7f3af5ffb640 1 -- 172.21.15.84:0/3469260408 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f3af0004b80 con 0x7f3b001f1fb0 2023-10-16T11:27:42.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.346+0000 7f3af5ffb640 1 --2- 172.21.15.84:0/3469260408 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f3ae406afc0 0x7f3ae406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:42.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.346+0000 7f3af5ffb640 1 -- 172.21.15.84:0/3469260408 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f3af00892a0 con 0x7f3b001f1fb0 2023-10-16T11:27:42.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.346+0000 7f3b052ea640 1 -- 172.21.15.84:0/3469260408 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f3ae406afc0 msgr2=0x7f3ae406d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/129259578 2023-10-16T11:27:42.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.346+0000 7f3b052ea640 1 --2- 172.21.15.84:0/3469260408 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f3ae406afc0 0x7f3ae406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:42.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.350+0000 7f3af5ffb640 1 -- 172.21.15.84:0/3469260408 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3af0053f10 con 0x7f3b001f1fb0 2023-10-16T11:27:42.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.547+0000 7f3b052ea640 1 -- 172.21.15.84:0/3469260408 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f3ae406afc0 msgr2=0x7f3ae406d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/129259578 2023-10-16T11:27:42.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.547+0000 7f3b052ea640 1 --2- 172.21.15.84:0/3469260408 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f3ae406afc0 0x7f3ae406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:27:42.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.578+0000 7f3b062ec640 1 -- 172.21.15.84:0/3469260408 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b000c65f0 con 0x7f3b001f1fb0 2023-10-16T11:27:42.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.579+0000 7f3af5ffb640 1 -- 172.21.15.84:0/3469260408 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+4933 (secure 0 0 0) 0x7f3af0057530 con 0x7f3b001f1fb0 2023-10-16T11:27:42.581 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:42.582 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":157,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7457,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3193159532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3193159532},{"type":"v1","addr":"172.21.15.157:6835","nonce":3193159532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":152},{"gid":8431,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1728897149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1728897149},{"type":"v1","addr":"172.21.15.157:6834","nonce":1728897149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":152},{"gid":13476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3409165899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3409165899},{"type":"v1","addr":"172.21.15.84:6835","nonce":3409165899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":157,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:27:22.871058+0000","modified":"2023-10-16T11:27:25.974995+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":13482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13482":{"gid":13482,"name":"a","rank":0,"incarnation":153,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2806813701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2806813701},{"type":"v1","addr":"172.21.15.84:6837","nonce":2806813701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[37],"metadata_pool":36,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":18}]} 2023-10-16T11:27:42.582 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 157 2023-10-16T11:27:42.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.582+0000 7f3b062ec640 1 -- 172.21.15.84:0/3469260408 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f3ae406afc0 msgr2=0x7f3ae406d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:42.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.582+0000 7f3b062ec640 1 --2- 172.21.15.84:0/3469260408 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f3ae406afc0 0x7f3ae406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:42.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.582+0000 7f3b062ec640 1 -- 172.21.15.84:0/3469260408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b001f1fb0 msgr2=0x7f3b00150d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:42.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.582+0000 7f3b062ec640 1 --2- 172.21.15.84:0/3469260408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b001f1fb0 0x7f3b00150d10 secure :-1 s=READY pgs=3470 cs=0 l=1 rev1=1 crypto rx=0x7f3af000d770 tx=0x7f3af000dc30 comp rx=0 tx=0).stop 2023-10-16T11:27:42.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.582+0000 7f3b062ec640 1 -- 172.21.15.84:0/3469260408 shutdown_connections 2023-10-16T11:27:42.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.582+0000 7f3b062ec640 1 --2- 172.21.15.84:0/3469260408 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f3ae406afc0 0x7f3ae406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:42.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.582+0000 7f3b062ec640 1 --2- 172.21.15.84:0/3469260408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b000ca050 0x7f3b00151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:42.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.582+0000 7f3b062ec640 1 --2- 172.21.15.84:0/3469260408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b001f1fb0 0x7f3b00150d10 unknown :-1 s=CLOSED pgs=3470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:42.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.582+0000 7f3b062ec640 1 --2- 172.21.15.84:0/3469260408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3aec003970 0x7f3b000c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:42.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.583+0000 7f3b062ec640 1 -- 172.21.15.84:0/3469260408 >> 172.21.15.84:0/3469260408 conn(0x7f3b000ce6a0 msgr2=0x7f3b0015c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:42.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.583+0000 7f3b062ec640 1 -- 172.21.15.84:0/3469260408 shutdown_connections 2023-10-16T11:27:42.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.583+0000 7f3b062ec640 1 -- 172.21.15.84:0/3469260408 wait complete. 2023-10-16T11:27:42.602 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:27:42.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.792+0000 7f141d0f2640 1 Processor -- start 2023-10-16T11:27:42.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.793+0000 7f141d0f2640 1 -- start start 2023-10-16T11:27:42.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.794+0000 7f141d0f2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f14180b3070 0x7f14180cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:42.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.794+0000 7f141d0f2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14180b3940 con 0x7f14180d1b10 2023-10-16T11:27:42.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.794+0000 7f141d0f2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14180b45e0 con 0x7f14180b3e70 2023-10-16T11:27:42.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.794+0000 7f141d0f2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14180b3b90 con 0x7f14180b3070 2023-10-16T11:27:42.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.794+0000 7f1416d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f14180b3070 0x7f14180cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:42.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.794+0000 7f1416d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f14180b3070 0x7f14180cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45538/0 (socket says 172.21.15.84:45538) 2023-10-16T11:27:42.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.794+0000 7f141759e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f14180d1b10 0x7f14180c6f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52488/0 (socket says 172.21.15.84:52488) 2023-10-16T11:27:42.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.794+0000 7f1416d9d640 1 -- 172.21.15.84:0/702065103 learned_addr learned my addr 172.21.15.84:0/702065103 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:42.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.795+0000 7f1416d9d640 1 -- 172.21.15.84:0/702065103 >> v1:172.21.15.157:6789/0 conn(0x7f14180b3e70 legacy=0x7f14180c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:42.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.795+0000 7f1416d9d640 1 -- 172.21.15.84:0/702065103 >> v1:172.21.15.84:6789/0 conn(0x7f14180d1b10 legacy=0x7f14180c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:42.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.795+0000 7f1416d9d640 1 -- 172.21.15.84:0/702065103 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14180d3840 con 0x7f14180b3070 2023-10-16T11:27:42.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.796+0000 7f1416d9d640 1 --2- 172.21.15.84:0/702065103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14180b3070 0x7f14180cf500 secure :-1 s=READY pgs=3472 cs=0 l=1 rev1=1 crypto rx=0x7f1408009850 tx=0x7f140802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0ff97bc94b80d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:42.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.796+0000 7f141659c640 1 -- 172.21.15.84:0/702065103 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408004230 con 0x7f14180b3070 2023-10-16T11:27:42.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.796+0000 7f141659c640 1 -- 172.21.15.84:0/702065103 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1408004390 con 0x7f14180b3070 2023-10-16T11:27:42.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.796+0000 7f141659c640 1 -- 172.21.15.84:0/702065103 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408030840 con 0x7f14180b3070 2023-10-16T11:27:42.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.797+0000 7f141d0f2640 1 -- 172.21.15.84:0/702065103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14180b3070 msgr2=0x7f14180cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:42.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.797+0000 7f141d0f2640 1 --2- 172.21.15.84:0/702065103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14180b3070 0x7f14180cf500 secure :-1 s=READY pgs=3472 cs=0 l=1 rev1=1 crypto rx=0x7f1408009850 tx=0x7f140802b980 comp rx=0 tx=0).stop 2023-10-16T11:27:42.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.797+0000 7f141d0f2640 1 -- 172.21.15.84:0/702065103 shutdown_connections 2023-10-16T11:27:42.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.797+0000 7f141d0f2640 1 --2- 172.21.15.84:0/702065103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14180b3070 0x7f14180cf500 unknown :-1 s=CLOSED pgs=3472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:42.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.797+0000 7f141d0f2640 1 -- 172.21.15.84:0/702065103 >> 172.21.15.84:0/702065103 conn(0x7f14180d2d20 msgr2=0x7f14180d3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:42.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.797+0000 7f141d0f2640 1 -- 172.21.15.84:0/702065103 shutdown_connections 2023-10-16T11:27:42.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.797+0000 7f141d0f2640 1 -- 172.21.15.84:0/702065103 wait complete. 2023-10-16T11:27:42.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.799+0000 7f141d0f2640 1 Processor -- start 2023-10-16T11:27:42.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.799+0000 7f141d0f2640 1 -- start start 2023-10-16T11:27:42.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.800+0000 7f141d0f2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14180b3070 0x7f1418151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:42.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.800+0000 7f141d0f2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14180b3e70 0x7f141815fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:42.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.800+0000 7f141d0f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14180d1b10 0x7f1418160430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:42.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.800+0000 7f141d0f2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14180b22b0 con 0x7f14180b3070 2023-10-16T11:27:42.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.800+0000 7f141d0f2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1418160970 con 0x7f14180d1b10 2023-10-16T11:27:42.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.800+0000 7f141d0f2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14180b1730 con 0x7f14180b3e70 2023-10-16T11:27:42.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.800+0000 7f1416d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14180b3070 0x7f1418151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:42.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.800+0000 7f1416d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14180b3070 0x7f1418151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45548/0 (socket says 172.21.15.84:45548) 2023-10-16T11:27:42.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.800+0000 7f1416d9d640 1 -- 172.21.15.84:0/3432964935 learned_addr learned my addr 172.21.15.84:0/3432964935 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:42.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.801+0000 7f141759e640 1 --2- 172.21.15.84:0/3432964935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14180d1b10 0x7f1418160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:42.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.801+0000 7f140ffff640 1 --2- 172.21.15.84:0/3432964935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14180b3e70 0x7f141815fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:42.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.801+0000 7f1416d9d640 1 -- 172.21.15.84:0/3432964935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14180b3e70 msgr2=0x7f141815fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:42.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.801+0000 7f1416d9d640 1 --2- 172.21.15.84:0/3432964935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14180b3e70 0x7f141815fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:42.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.801+0000 7f1416d9d640 1 -- 172.21.15.84:0/3432964935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14180d1b10 msgr2=0x7f1418160430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:42.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.801+0000 7f1416d9d640 1 --2- 172.21.15.84:0/3432964935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14180d1b10 0x7f1418160430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:42.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.801+0000 7f1416d9d640 1 -- 172.21.15.84:0/3432964935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1408009460 con 0x7f14180b3070 2023-10-16T11:27:42.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.801+0000 7f1416d9d640 1 --2- 172.21.15.84:0/3432964935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14180b3070 0x7f1418151d60 secure :-1 s=READY pgs=3473 cs=0 l=1 rev1=1 crypto rx=0x7f1408009970 tx=0x7f14080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:42.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.802+0000 7f140f7fe640 1 -- 172.21.15.84:0/3432964935 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140803e070 con 0x7f14180b3070 2023-10-16T11:27:42.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.802+0000 7f140f7fe640 1 -- 172.21.15.84:0/3432964935 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1408004500 con 0x7f14180b3070 2023-10-16T11:27:42.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.802+0000 7f140f7fe640 1 -- 172.21.15.84:0/3432964935 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408030e40 con 0x7f14180b3070 2023-10-16T11:27:42.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.802+0000 7f141d0f2640 1 -- 172.21.15.84:0/3432964935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14180b1a10 con 0x7f14180b3070 2023-10-16T11:27:42.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.802+0000 7f141d0f2640 1 -- 172.21.15.84:0/3432964935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14180b1fd0 con 0x7f14180b3070 2023-10-16T11:27:42.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.803+0000 7f141d0f2640 1 -- 172.21.15.84:0/3432964935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14180cadd0 con 0x7f14180b3070 2023-10-16T11:27:42.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.804+0000 7f140f7fe640 1 -- 172.21.15.84:0/3432964935 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f1408003610 con 0x7f14180b3070 2023-10-16T11:27:42.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.805+0000 7f140f7fe640 1 --2- 172.21.15.84:0/3432964935 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f13f806afc0 0x7f13f806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:42.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.805+0000 7f140f7fe640 1 -- 172.21.15.84:0/3432964935 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f14080ab840 con 0x7f14180b3070 2023-10-16T11:27:42.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.805+0000 7f140ffff640 1 -- 172.21.15.84:0/3432964935 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f13f806afc0 msgr2=0x7f13f806d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/129259578 2023-10-16T11:27:42.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.805+0000 7f140ffff640 1 --2- 172.21.15.84:0/3432964935 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f13f806afc0 0x7f13f806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:42.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.809+0000 7f140f7fe640 1 -- 172.21.15.84:0/3432964935 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f14180cadd0 con 0x7f14180b3070 2023-10-16T11:27:42.876 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:42.874+0000 7f950da0f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:27:42.964 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:42.962+0000 7f338283b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:27:42.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.997+0000 7f141d0f2640 1 -- 172.21.15.84:0/3432964935 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14180cadd0 con 0x7f14180b3070 2023-10-16T11:27:43.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:42.999+0000 7f140f7fe640 1 -- 172.21.15.84:0/3432964935 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v271) v1 ==== 74+0+17044 (secure 0 0 0) 0x7f14180cadd0 con 0x7f14180b3070 2023-10-16T11:27:43.002 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:43.002 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":271,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:27:37.140352+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:27:20.716240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"269","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_data","create_time":"2023-10-16T11:27:21.905553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"270","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1788498161":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/3634760901":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/3842621119":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/40770371":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/1147031855":"2023-10-17T11:27:37.140228+0000","172.21.15.84:6833/731888425":"2023-10-17T11:27:37.140228+0000","172.21.15.84:6832/731888425":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/2353059609":"2023-10-17T11:27:37.140228+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:27:43.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.003+0000 7f141d0f2640 1 -- 172.21.15.84:0/3432964935 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f13f806afc0 msgr2=0x7f13f806d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:43.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.003+0000 7f141d0f2640 1 --2- 172.21.15.84:0/3432964935 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f13f806afc0 0x7f13f806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.003+0000 7f141d0f2640 1 -- 172.21.15.84:0/3432964935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14180b3070 msgr2=0x7f1418151d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:43.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.003+0000 7f141d0f2640 1 --2- 172.21.15.84:0/3432964935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14180b3070 0x7f1418151d60 secure :-1 s=READY pgs=3473 cs=0 l=1 rev1=1 crypto rx=0x7f1408009970 tx=0x7f14080046c0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.003+0000 7f141d0f2640 1 -- 172.21.15.84:0/3432964935 shutdown_connections 2023-10-16T11:27:43.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.003+0000 7f141d0f2640 1 --2- 172.21.15.84:0/3432964935 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f13f806afc0 0x7f13f806d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.003+0000 7f141d0f2640 1 --2- 172.21.15.84:0/3432964935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14180d1b10 0x7f1418160430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.003+0000 7f141d0f2640 1 --2- 172.21.15.84:0/3432964935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14180b3e70 0x7f141815fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.003+0000 7f141d0f2640 1 --2- 172.21.15.84:0/3432964935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14180b3070 0x7f1418151d60 unknown :-1 s=CLOSED pgs=3473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.003+0000 7f141d0f2640 1 -- 172.21.15.84:0/3432964935 >> 172.21.15.84:0/3432964935 conn(0x7f14180d2d20 msgr2=0x7f1418152eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:43.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.003+0000 7f141d0f2640 1 -- 172.21.15.84:0/3432964935 shutdown_connections 2023-10-16T11:27:43.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.003+0000 7f141d0f2640 1 -- 172.21.15.84:0/3432964935 wait complete. 2023-10-16T11:27:43.030 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:48:55.657982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 36, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:27:20.716240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '269', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 37, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:27:21.905553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '270', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:27:43.031 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:27:43.175 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:43.173+0000 7f950da0f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:27:43.235 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:43.233+0000 7f950da0f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:27:43.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.233+0000 7f8dd0d8e640 1 Processor -- start 2023-10-16T11:27:43.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.234+0000 7f8dd0d8e640 1 -- start start 2023-10-16T11:27:43.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.235+0000 7f8dd0d8e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8dcc15ad10 0x7f8dcc0b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:43.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.235+0000 7f8dd0d8e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8dcc0ce250 0x7f8dcc09d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:43.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.235+0000 7f8dd0d8e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dcc070c40 con 0x7f8dcc065cc0 2023-10-16T11:27:43.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.235+0000 7f8dd0d8e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dcc066100 con 0x7f8dcc15ad10 2023-10-16T11:27:43.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.235+0000 7f8dd0d8e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dcc06e4c0 con 0x7f8dcc0ce250 2023-10-16T11:27:43.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.235+0000 7f8dcb7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8dcc15ad10 0x7f8dcc0b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:43.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.235+0000 7f8dcb7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8dcc15ad10 0x7f8dcc0b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45554/0 (socket says 172.21.15.84:45554) 2023-10-16T11:27:43.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.235+0000 7f8dcb7fe640 1 -- 172.21.15.84:0/1952998999 learned_addr learned my addr 172.21.15.84:0/1952998999 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:43.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.236+0000 7f8dcbfff640 1 --2- 172.21.15.84:0/1952998999 >> v2:172.21.15.157:3300/0 conn(0x7f8dcc0ce250 0x7f8dcc09d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:43.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.236+0000 7f8dcb7fe640 1 -- 172.21.15.84:0/1952998999 >> v2:172.21.15.157:3300/0 conn(0x7f8dcc0ce250 msgr2=0x7f8dcc09d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:43.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.236+0000 7f8dcb7fe640 1 --2- 172.21.15.84:0/1952998999 >> v2:172.21.15.157:3300/0 conn(0x7f8dcc0ce250 0x7f8dcc09d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.236+0000 7f8dcb7fe640 1 -- 172.21.15.84:0/1952998999 >> v1:172.21.15.84:6789/0 conn(0x7f8dcc065cc0 legacy=0x7f8dcc0cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:43.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.236+0000 7f8dcb7fe640 1 -- 172.21.15.84:0/1952998999 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dcc0b49c0 con 0x7f8dcc15ad10 2023-10-16T11:27:43.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.237+0000 7f8dcb7fe640 1 --2- 172.21.15.84:0/1952998999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dcc15ad10 0x7f8dcc0b1db0 secure :-1 s=READY pgs=3475 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0b1940 tx=0x7f8dbc02b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2d361fb1c6826e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:43.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.237+0000 7f8dcaffd640 1 -- 172.21.15.84:0/1952998999 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc030870 con 0x7f8dcc15ad10 2023-10-16T11:27:43.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.237+0000 7f8dcaffd640 1 -- 172.21.15.84:0/1952998999 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc0309d0 con 0x7f8dcc15ad10 2023-10-16T11:27:43.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.237+0000 7f8dcaffd640 1 -- 172.21.15.84:0/1952998999 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc030d60 con 0x7f8dcc15ad10 2023-10-16T11:27:43.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.238+0000 7f8dd0d8e640 1 -- 172.21.15.84:0/1952998999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dcc15ad10 msgr2=0x7f8dcc0b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:43.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.238+0000 7f8dd0d8e640 1 --2- 172.21.15.84:0/1952998999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dcc15ad10 0x7f8dcc0b1db0 secure :-1 s=READY pgs=3475 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0b1940 tx=0x7f8dbc02b9b0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.238+0000 7f8dd0d8e640 1 -- 172.21.15.84:0/1952998999 shutdown_connections 2023-10-16T11:27:43.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.238+0000 7f8dd0d8e640 1 --2- 172.21.15.84:0/1952998999 >> v2:172.21.15.157:3300/0 conn(0x7f8dcc0ce250 0x7f8dcc09d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.238+0000 7f8dd0d8e640 1 --2- 172.21.15.84:0/1952998999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dcc15ad10 0x7f8dcc0b1db0 unknown :-1 s=CLOSED pgs=3475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.238+0000 7f8dd0d8e640 1 -- 172.21.15.84:0/1952998999 >> 172.21.15.84:0/1952998999 conn(0x7f8dcc0b0a50 msgr2=0x7f8dcc0b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:43.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.238+0000 7f8dd0d8e640 1 -- 172.21.15.84:0/1952998999 shutdown_connections 2023-10-16T11:27:43.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.238+0000 7f8dd0d8e640 1 -- 172.21.15.84:0/1952998999 wait complete. 2023-10-16T11:27:43.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.239+0000 7f8dd0d8e640 1 Processor -- start 2023-10-16T11:27:43.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.240+0000 7f8dd0d8e640 1 -- start start 2023-10-16T11:27:43.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.240+0000 7f8dd0d8e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8dcc065cc0 0x7f8dcc15b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:43.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.240+0000 7f8dd0d8e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dcc0ce250 0x7f8dcc0ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:43.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.240+0000 7f8dd0d8e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8dcc15ad10 0x7f8dcc157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:43.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.240+0000 7f8dd0d8e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc0c95b0 con 0x7f8dcc0ce250 2023-10-16T11:27:43.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.240+0000 7f8dd0d8e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc1f6110 con 0x7f8dcc15ad10 2023-10-16T11:27:43.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.240+0000 7f8dd0d8e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc1f63f0 con 0x7f8dcc065cc0 2023-10-16T11:27:43.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.241+0000 7f8dc3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dcc0ce250 0x7f8dcc0ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:43.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.241+0000 7f8dc3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dcc0ce250 0x7f8dcc0ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45558/0 (socket says 172.21.15.84:45558) 2023-10-16T11:27:43.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.241+0000 7f8dc3fff640 1 -- 172.21.15.84:0/1233329575 learned_addr learned my addr 172.21.15.84:0/1233329575 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:43.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.241+0000 7f8dcb7fe640 1 --2- 172.21.15.84:0/1233329575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8dcc065cc0 0x7f8dcc15b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:43.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.241+0000 7f8dcbfff640 1 --2- 172.21.15.84:0/1233329575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8dcc15ad10 0x7f8dcc157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:43.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.241+0000 7f8dc3fff640 1 -- 172.21.15.84:0/1233329575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8dcc065cc0 msgr2=0x7f8dcc15b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:43.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.241+0000 7f8dc3fff640 1 --2- 172.21.15.84:0/1233329575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8dcc065cc0 0x7f8dcc15b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.241+0000 7f8dc3fff640 1 -- 172.21.15.84:0/1233329575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8dcc15ad10 msgr2=0x7f8dcc157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:43.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.241+0000 7f8dc3fff640 1 --2- 172.21.15.84:0/1233329575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8dcc15ad10 0x7f8dcc157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.241+0000 7f8dc3fff640 1 -- 172.21.15.84:0/1233329575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc0094d0 con 0x7f8dcc0ce250 2023-10-16T11:27:43.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.241+0000 7f8dcbfff640 1 --2- 172.21.15.84:0/1233329575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8dcc15ad10 0x7f8dcc157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:43.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.242+0000 7f8dc3fff640 1 --2- 172.21.15.84:0/1233329575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dcc0ce250 0x7f8dcc0ade90 secure :-1 s=READY pgs=3476 cs=0 l=1 rev1=1 crypto rx=0x7f8db400c450 tx=0x7f8db400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:43.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.242+0000 7f8dc8ff9640 1 -- 172.21.15.84:0/1233329575 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db40052d0 con 0x7f8dcc0ce250 2023-10-16T11:27:43.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.242+0000 7f8dc8ff9640 1 -- 172.21.15.84:0/1233329575 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db4005430 con 0x7f8dcc0ce250 2023-10-16T11:27:43.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.242+0000 7f8dd0d8e640 1 -- 172.21.15.84:0/1233329575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dcc1f66d0 con 0x7f8dcc0ce250 2023-10-16T11:27:43.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.242+0000 7f8dc8ff9640 1 -- 172.21.15.84:0/1233329575 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db4012870 con 0x7f8dcc0ce250 2023-10-16T11:27:43.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.242+0000 7f8dd0d8e640 1 -- 172.21.15.84:0/1233329575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dcc15b860 con 0x7f8dcc0ce250 2023-10-16T11:27:43.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.243+0000 7f8dd0d8e640 1 -- 172.21.15.84:0/1233329575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dcc15ba20 con 0x7f8dcc0ce250 2023-10-16T11:27:43.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.244+0000 7f8dc8ff9640 1 -- 172.21.15.84:0/1233329575 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f8db40129d0 con 0x7f8dcc0ce250 2023-10-16T11:27:43.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.245+0000 7f8dc8ff9640 1 --2- 172.21.15.84:0/1233329575 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f8dac06afc0 0x7f8dac06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:43.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.245+0000 7f8dcb7fe640 1 -- 172.21.15.84:0/1233329575 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f8dac06afc0 msgr2=0x7f8dac06d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6832/129259578 2023-10-16T11:27:43.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.245+0000 7f8dc8ff9640 1 -- 172.21.15.84:0/1233329575 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f8db4089480 con 0x7f8dcc0ce250 2023-10-16T11:27:43.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.245+0000 7f8dcb7fe640 1 --2- 172.21.15.84:0/1233329575 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f8dac06afc0 0x7f8dac06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:27:43.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.250+0000 7f8dc8ff9640 1 -- 172.21.15.84:0/1233329575 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8dcc15ba20 con 0x7f8dcc0ce250 2023-10-16T11:27:43.293 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:43.291+0000 7f338283b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:27:43.350 INFO:tasks.ceph.mgr.x.smithi157.stderr:2023-10-16T11:27:43.348+0000 7f950da0f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:27:43.351 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:43.349+0000 7f338283b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:27:43.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.432+0000 7f8dd0d8e640 1 -- 172.21.15.84:0/1233329575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8dcc15b9f0 con 0x7f8dcc0ce250 2023-10-16T11:27:43.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.433+0000 7f8dc8ff9640 1 -- 172.21.15.84:0/1233329575 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v271) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8db4054200 con 0x7f8dcc0ce250 2023-10-16T11:27:43.435 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-16T11:27:43.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.436+0000 7f8dd0d8e640 1 -- 172.21.15.84:0/1233329575 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f8dac06afc0 msgr2=0x7f8dac06d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:27:43.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.436+0000 7f8dd0d8e640 1 --2- 172.21.15.84:0/1233329575 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f8dac06afc0 0x7f8dac06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.436+0000 7f8dd0d8e640 1 -- 172.21.15.84:0/1233329575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dcc0ce250 msgr2=0x7f8dcc0ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:43.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.436+0000 7f8dd0d8e640 1 --2- 172.21.15.84:0/1233329575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dcc0ce250 0x7f8dcc0ade90 secure :-1 s=READY pgs=3476 cs=0 l=1 rev1=1 crypto rx=0x7f8db400c450 tx=0x7f8db400c910 comp rx=0 tx=0).stop 2023-10-16T11:27:43.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.436+0000 7f8dd0d8e640 1 -- 172.21.15.84:0/1233329575 shutdown_connections 2023-10-16T11:27:43.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.436+0000 7f8dd0d8e640 1 --2- 172.21.15.84:0/1233329575 >> [v2:172.21.15.84:6832/129259578,v1:172.21.15.84:6833/129259578] conn(0x7f8dac06afc0 0x7f8dac06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.437+0000 7f8dd0d8e640 1 --2- 172.21.15.84:0/1233329575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8dcc15ad10 0x7f8dcc157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.437+0000 7f8dd0d8e640 1 --2- 172.21.15.84:0/1233329575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dcc0ce250 0x7f8dcc0ade90 unknown :-1 s=CLOSED pgs=3476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.437+0000 7f8dd0d8e640 1 --2- 172.21.15.84:0/1233329575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8dcc065cc0 0x7f8dcc15b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.437+0000 7f8dd0d8e640 1 -- 172.21.15.84:0/1233329575 >> 172.21.15.84:0/1233329575 conn(0x7f8dcc0b0a50 msgr2=0x7f8dcc06fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:43.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.437+0000 7f8dd0d8e640 1 -- 172.21.15.84:0/1233329575 shutdown_connections 2023-10-16T11:27:43.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.437+0000 7f8dd0d8e640 1 -- 172.21.15.84:0/1233329575 wait complete. 2023-10-16T11:27:43.455 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:27:43.467 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:43.466+0000 7f338283b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:27:43.542 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:43.540+0000 7f3307c48640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:43.543 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:43.540+0000 7f3307c48640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:43.543 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:43.540+0000 7f3307c48640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:43.543 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:43.540+0000 7f3307c48640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:43.544 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:43.540+0000 7f3307c48640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:43.544 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:43.542+0000 7f3303c40640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:43.544 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:43.542+0000 7f3303c40640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:43.545 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:43.542+0000 7f3303c40640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:43.545 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:43.542+0000 7f3303c40640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:43.545 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-16T11:27:43.542+0000 7f3303c40640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:27:43.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.646+0000 7fcd55615640 1 Processor -- start 2023-10-16T11:27:43.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.646+0000 7fcd55615640 1 -- start start 2023-10-16T11:27:43.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.647+0000 7fcd55615640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcd5015af80 0x7fcd500cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:43.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.647+0000 7fcd55615640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd50065ab0 0x7fcd50065f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:43.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.647+0000 7fcd55615640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5015eff0 con 0x7fcd5006faf0 2023-10-16T11:27:43.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.647+0000 7fcd55615640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd50066480 con 0x7fcd5015af80 2023-10-16T11:27:43.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.647+0000 7fcd55615640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd500d0930 con 0x7fcd50065ab0 2023-10-16T11:27:43.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.648+0000 7fcd4effd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcd5015af80 0x7fcd500cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:43.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.648+0000 7fcd4effd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcd5015af80 0x7fcd500cb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45652/0 (socket says 172.21.15.84:45652) 2023-10-16T11:27:43.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.648+0000 7fcd4effd640 1 -- 172.21.15.84:0/2130468073 learned_addr learned my addr 172.21.15.84:0/2130468073 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:43.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.648+0000 7fcd4f7fe640 1 --2- 172.21.15.84:0/2130468073 >> v2:172.21.15.157:3300/0 conn(0x7fcd50065ab0 0x7fcd50065f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:43.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.648+0000 7fcd4effd640 1 -- 172.21.15.84:0/2130468073 >> v2:172.21.15.157:3300/0 conn(0x7fcd50065ab0 msgr2=0x7fcd50065f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:43.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.648+0000 7fcd4effd640 1 --2- 172.21.15.84:0/2130468073 >> v2:172.21.15.157:3300/0 conn(0x7fcd50065ab0 0x7fcd50065f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.648+0000 7fcd4effd640 1 -- 172.21.15.84:0/2130468073 >> v1:172.21.15.157:6789/0 conn(0x7fcd5006faf0 legacy=0x7fcd5015f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:43.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.648+0000 7fcd4effd640 1 -- 172.21.15.84:0/2130468073 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5015ecc0 con 0x7fcd5015af80 2023-10-16T11:27:43.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.648+0000 7fcd4f7fe640 1 --2- 172.21.15.84:0/2130468073 >> v2:172.21.15.157:3300/0 conn(0x7fcd50065ab0 0x7fcd50065f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:43.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.649+0000 7fcd4effd640 1 --2- 172.21.15.84:0/2130468073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd5015af80 0x7fcd500cb1c0 secure :-1 s=READY pgs=3488 cs=0 l=1 rev1=1 crypto rx=0x7fcd500b6c00 tx=0x7fcd4002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291adfa22235384a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:43.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.649+0000 7fcd4dffb640 1 -- 172.21.15.84:0/2130468073 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd400317d0 con 0x7fcd5015af80 2023-10-16T11:27:43.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.649+0000 7fcd4dffb640 1 -- 172.21.15.84:0/2130468073 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd40031930 con 0x7fcd5015af80 2023-10-16T11:27:43.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.649+0000 7fcd4dffb640 1 -- 172.21.15.84:0/2130468073 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40031cc0 con 0x7fcd5015af80 2023-10-16T11:27:43.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.650+0000 7fcd55615640 1 -- 172.21.15.84:0/2130468073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd5015af80 msgr2=0x7fcd500cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:43.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.650+0000 7fcd55615640 1 --2- 172.21.15.84:0/2130468073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd5015af80 0x7fcd500cb1c0 secure :-1 s=READY pgs=3488 cs=0 l=1 rev1=1 crypto rx=0x7fcd500b6c00 tx=0x7fcd4002b870 comp rx=0 tx=0).stop 2023-10-16T11:27:43.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.650+0000 7fcd55615640 1 -- 172.21.15.84:0/2130468073 shutdown_connections 2023-10-16T11:27:43.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.650+0000 7fcd55615640 1 --2- 172.21.15.84:0/2130468073 >> v2:172.21.15.157:3300/0 conn(0x7fcd50065ab0 0x7fcd50065f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.650+0000 7fcd55615640 1 --2- 172.21.15.84:0/2130468073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd5015af80 0x7fcd500cb1c0 unknown :-1 s=CLOSED pgs=3488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.650+0000 7fcd55615640 1 -- 172.21.15.84:0/2130468073 >> 172.21.15.84:0/2130468073 conn(0x7fcd500d0ab0 msgr2=0x7fcd500d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:43.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.650+0000 7fcd55615640 1 -- 172.21.15.84:0/2130468073 shutdown_connections 2023-10-16T11:27:43.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.650+0000 7fcd55615640 1 -- 172.21.15.84:0/2130468073 wait complete. 2023-10-16T11:27:43.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.652+0000 7fcd55615640 1 Processor -- start 2023-10-16T11:27:43.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.652+0000 7fcd55615640 1 -- start start 2023-10-16T11:27:43.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.653+0000 7fcd55615640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd50065ab0 0x7fcd5015de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:43.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.653+0000 7fcd55615640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd5006faf0 0x7fcd50155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:43.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.653+0000 7fcd55615640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5015af80 0x7fcd500ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:43.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.653+0000 7fcd55615640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500c56d0 con 0x7fcd50065ab0 2023-10-16T11:27:43.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.653+0000 7fcd55615640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500c6530 con 0x7fcd5015af80 2023-10-16T11:27:43.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.653+0000 7fcd55615640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500c5920 con 0x7fcd5006faf0 2023-10-16T11:27:43.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.653+0000 7fcd4effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd50065ab0 0x7fcd5015de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:43.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.653+0000 7fcd4effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd50065ab0 0x7fcd5015de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45660/0 (socket says 172.21.15.84:45660) 2023-10-16T11:27:43.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.653+0000 7fcd4effd640 1 -- 172.21.15.84:0/936645002 learned_addr learned my addr 172.21.15.84:0/936645002 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:43.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.653+0000 7fcd4effd640 1 -- 172.21.15.84:0/936645002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd5006faf0 msgr2=0x7fcd50155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:43.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.653+0000 7fcd4effd640 1 --2- 172.21.15.84:0/936645002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd5006faf0 0x7fcd50155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.653+0000 7fcd4effd640 1 -- 172.21.15.84:0/936645002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5015af80 msgr2=0x7fcd500ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:43.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.654+0000 7fcd4effd640 1 --2- 172.21.15.84:0/936645002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5015af80 0x7fcd500ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.654+0000 7fcd4effd640 1 -- 172.21.15.84:0/936645002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd400094d0 con 0x7fcd50065ab0 2023-10-16T11:27:43.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.654+0000 7fcd4effd640 1 --2- 172.21.15.84:0/936645002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd50065ab0 0x7fcd5015de20 secure :-1 s=READY pgs=3489 cs=0 l=1 rev1=1 crypto rx=0x7fcd40000c00 tx=0x7fcd400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:43.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.654+0000 7fcd477fe640 1 -- 172.21.15.84:0/936645002 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd400329b0 con 0x7fcd50065ab0 2023-10-16T11:27:43.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.654+0000 7fcd477fe640 1 -- 172.21.15.84:0/936645002 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd40032b30 con 0x7fcd50065ab0 2023-10-16T11:27:43.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.654+0000 7fcd477fe640 1 -- 172.21.15.84:0/936645002 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd400453f0 con 0x7fcd50065ab0 2023-10-16T11:27:43.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.654+0000 7fcd55615640 1 -- 172.21.15.84:0/936645002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd500c5c00 con 0x7fcd50065ab0 2023-10-16T11:27:43.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.654+0000 7fcd55615640 1 -- 172.21.15.84:0/936645002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd500c61c0 con 0x7fcd50065ab0 2023-10-16T11:27:43.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.655+0000 7fcd55615640 1 -- 172.21.15.84:0/936645002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5004e160 con 0x7fcd50065ab0 2023-10-16T11:27:43.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.656+0000 7fcd477fe640 1 -- 172.21.15.84:0/936645002 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 166) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fcd40030070 con 0x7fcd50065ab0 2023-10-16T11:27:43.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.657+0000 7fcd477fe640 1 -- 172.21.15.84:0/936645002 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(272..272 src has 1..272) v4 ==== 6425+0+0 (secure 0 0 0) 0x7fcd400ab020 con 0x7fcd50065ab0 2023-10-16T11:27:43.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.661+0000 7fcd477fe640 1 -- 172.21.15.84:0/936645002 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcd400761c0 con 0x7fcd50065ab0 2023-10-16T11:27:43.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.838+0000 7fcd55615640 1 -- 172.21.15.84:0/936645002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcd5004e160 con 0x7fcd50065ab0 2023-10-16T11:27:43.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.839+0000 7fcd477fe640 1 -- 172.21.15.84:0/936645002 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v272) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcd400797e0 con 0x7fcd50065ab0 2023-10-16T11:27:43.841 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-16T11:27:43.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.841+0000 7fcd55615640 1 -- 172.21.15.84:0/936645002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd50065ab0 msgr2=0x7fcd5015de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:43.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.841+0000 7fcd55615640 1 --2- 172.21.15.84:0/936645002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd50065ab0 0x7fcd5015de20 secure :-1 s=READY pgs=3489 cs=0 l=1 rev1=1 crypto rx=0x7fcd40000c00 tx=0x7fcd400046c0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.841+0000 7fcd55615640 1 -- 172.21.15.84:0/936645002 shutdown_connections 2023-10-16T11:27:43.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.841+0000 7fcd55615640 1 --2- 172.21.15.84:0/936645002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5015af80 0x7fcd500ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.841+0000 7fcd55615640 1 --2- 172.21.15.84:0/936645002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd5006faf0 0x7fcd50155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.841+0000 7fcd55615640 1 --2- 172.21.15.84:0/936645002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd50065ab0 0x7fcd5015de20 unknown :-1 s=CLOSED pgs=3489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:43.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.841+0000 7fcd55615640 1 -- 172.21.15.84:0/936645002 >> 172.21.15.84:0/936645002 conn(0x7fcd500d0ab0 msgr2=0x7fcd500c3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:43.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.841+0000 7fcd55615640 1 -- 172.21.15.84:0/936645002 shutdown_connections 2023-10-16T11:27:43.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:43.841+0000 7fcd55615640 1 -- 172.21.15.84:0/936645002 wait complete. 2023-10-16T11:27:43.862 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:27:44.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.115+0000 7fc49bf6c640 1 Processor -- start 2023-10-16T11:27:44.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.115+0000 7fc49bf6c640 1 -- start start 2023-10-16T11:27:44.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.117+0000 7fc49bf6c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4940addb0 0x7fc4940caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:44.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.117+0000 7fc49bf6c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc4940c9000 0x7fc494069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:44.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.117+0000 7fc49bf6c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc49406a6f0 con 0x7fc49406a2e0 2023-10-16T11:27:44.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.117+0000 7fc49bf6c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc494161a50 con 0x7fc4940c9000 2023-10-16T11:27:44.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.117+0000 7fc49bf6c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4940cb140 con 0x7fc4940addb0 2023-10-16T11:27:44.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.117+0000 7fc499507640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc4940c9000 0x7fc494069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:44.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.117+0000 7fc49a509640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc49406a2e0 0x7fc4940cc580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52516/0 (socket says 172.21.15.84:52516) 2023-10-16T11:27:44.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.117+0000 7fc499507640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc4940c9000 0x7fc494069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45662/0 (socket says 172.21.15.84:45662) 2023-10-16T11:27:44.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.117+0000 7fc49a509640 1 -- 172.21.15.84:0/1879389584 learned_addr learned my addr 172.21.15.84:0/1879389584 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:44.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.117+0000 7fc499507640 1 -- 172.21.15.84:0/1879389584 >> v2:172.21.15.157:3300/0 conn(0x7fc4940addb0 msgr2=0x7fc4940caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:44.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.117+0000 7fc499d08640 1 --2- 172.21.15.84:0/1879389584 >> v2:172.21.15.157:3300/0 conn(0x7fc4940addb0 0x7fc4940caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:44.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.118+0000 7fc499507640 1 --2- 172.21.15.84:0/1879389584 >> v2:172.21.15.157:3300/0 conn(0x7fc4940addb0 0x7fc4940caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.118+0000 7fc499507640 1 -- 172.21.15.84:0/1879389584 >> v1:172.21.15.84:6789/0 conn(0x7fc49406a2e0 legacy=0x7fc4940cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:44.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.118+0000 7fc499507640 1 -- 172.21.15.84:0/1879389584 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49406b030 con 0x7fc4940c9000 2023-10-16T11:27:44.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.118+0000 7fc498d06640 1 -- 172.21.15.84:0/1879389584 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1521516967 0 0) 0x7fc49406a6f0 con 0x7fc49406a2e0 2023-10-16T11:27:44.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.119+0000 7fc499507640 1 --2- 172.21.15.84:0/1879389584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4940c9000 0x7fc494069da0 secure :-1 s=READY pgs=3491 cs=0 l=1 rev1=1 crypto rx=0x7fc4940b15a0 tx=0x7fc48402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17510acb365f50 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:44.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.119+0000 7fc498d06640 1 -- 172.21.15.84:0/1879389584 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc484031830 con 0x7fc4940c9000 2023-10-16T11:27:44.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.119+0000 7fc498d06640 1 -- 172.21.15.84:0/1879389584 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc484031990 con 0x7fc4940c9000 2023-10-16T11:27:44.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.119+0000 7fc498d06640 1 -- 172.21.15.84:0/1879389584 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc484031d20 con 0x7fc4940c9000 2023-10-16T11:27:44.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.119+0000 7fc49bf6c640 1 -- 172.21.15.84:0/1879389584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4940c9000 msgr2=0x7fc494069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:44.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.119+0000 7fc49bf6c640 1 --2- 172.21.15.84:0/1879389584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4940c9000 0x7fc494069da0 secure :-1 s=READY pgs=3491 cs=0 l=1 rev1=1 crypto rx=0x7fc4940b15a0 tx=0x7fc48402b8d0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.119+0000 7fc49bf6c640 1 -- 172.21.15.84:0/1879389584 shutdown_connections 2023-10-16T11:27:44.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.119+0000 7fc49bf6c640 1 --2- 172.21.15.84:0/1879389584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4940c9000 0x7fc494069da0 unknown :-1 s=CLOSED pgs=3491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.119+0000 7fc49bf6c640 1 --2- 172.21.15.84:0/1879389584 >> v2:172.21.15.157:3300/0 conn(0x7fc4940addb0 0x7fc4940caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.119+0000 7fc49bf6c640 1 -- 172.21.15.84:0/1879389584 >> 172.21.15.84:0/1879389584 conn(0x7fc4940ce9b0 msgr2=0x7fc4940cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:44.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.120+0000 7fc49bf6c640 1 -- 172.21.15.84:0/1879389584 shutdown_connections 2023-10-16T11:27:44.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.120+0000 7fc49bf6c640 1 -- 172.21.15.84:0/1879389584 wait complete. 2023-10-16T11:27:44.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.121+0000 7fc49bf6c640 1 Processor -- start 2023-10-16T11:27:44.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.121+0000 7fc49bf6c640 1 -- start start 2023-10-16T11:27:44.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.122+0000 7fc49bf6c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49406a2e0 0x7fc4940c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:44.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.122+0000 7fc49bf6c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4940addb0 0x7fc4940c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:44.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.122+0000 7fc49bf6c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4940c9000 0x7fc49415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:44.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.122+0000 7fc49bf6c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940c78e0 con 0x7fc4940c9000 2023-10-16T11:27:44.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.122+0000 7fc49bf6c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49415d5e0 con 0x7fc4940addb0 2023-10-16T11:27:44.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.122+0000 7fc49bf6c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940c6110 con 0x7fc49406a2e0 2023-10-16T11:27:44.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.123+0000 7fc49a509640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4940c9000 0x7fc49415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:44.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.123+0000 7fc49a509640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4940c9000 0x7fc49415d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45672/0 (socket says 172.21.15.84:45672) 2023-10-16T11:27:44.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.123+0000 7fc49a509640 1 -- 172.21.15.84:0/3896028472 learned_addr learned my addr 172.21.15.84:0/3896028472 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:44.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.123+0000 7fc49a509640 1 -- 172.21.15.84:0/3896028472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49406a2e0 msgr2=0x7fc4940c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:44.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.123+0000 7fc49a509640 1 --2- 172.21.15.84:0/3896028472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49406a2e0 0x7fc4940c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.123+0000 7fc49a509640 1 -- 172.21.15.84:0/3896028472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4940addb0 msgr2=0x7fc4940c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:44.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.123+0000 7fc499507640 1 --2- 172.21.15.84:0/3896028472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4940addb0 0x7fc4940c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:44.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.123+0000 7fc49a509640 1 --2- 172.21.15.84:0/3896028472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4940addb0 0x7fc4940c73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.123+0000 7fc49a509640 1 -- 172.21.15.84:0/3896028472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc484009400 con 0x7fc4940c9000 2023-10-16T11:27:44.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.123+0000 7fc49a509640 1 --2- 172.21.15.84:0/3896028472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4940c9000 0x7fc49415d0a0 secure :-1 s=READY pgs=3492 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00c7a0 tx=0x7fc48c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:44.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.123+0000 7fc48a7fc640 1 -- 172.21.15.84:0/3896028472 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c00a640 con 0x7fc4940c9000 2023-10-16T11:27:44.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.124+0000 7fc48a7fc640 1 -- 172.21.15.84:0/3896028472 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48c00a7a0 con 0x7fc4940c9000 2023-10-16T11:27:44.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.124+0000 7fc48a7fc640 1 -- 172.21.15.84:0/3896028472 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c00abd0 con 0x7fc4940c9000 2023-10-16T11:27:44.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.124+0000 7fc49bf6c640 1 -- 172.21.15.84:0/3896028472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4940c63f0 con 0x7fc4940c9000 2023-10-16T11:27:44.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.124+0000 7fc49bf6c640 1 -- 172.21.15.84:0/3896028472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4940c69b0 con 0x7fc4940c9000 2023-10-16T11:27:44.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.125+0000 7fc49bf6c640 1 -- 172.21.15.84:0/3896028472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc49404e160 con 0x7fc4940c9000 2023-10-16T11:27:44.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.126+0000 7fc48a7fc640 1 -- 172.21.15.84:0/3896028472 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 166) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fc48c00ad30 con 0x7fc4940c9000 2023-10-16T11:27:44.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.126+0000 7fc48a7fc640 1 -- 172.21.15.84:0/3896028472 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(272..272 src has 1..272) v4 ==== 6425+0+0 (secure 0 0 0) 0x7fc48c088a10 con 0x7fc4940c9000 2023-10-16T11:27:44.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.130+0000 7fc48a7fc640 1 -- 172.21.15.84:0/3896028472 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc48c053b50 con 0x7fc4940c9000 2023-10-16T11:27:44.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.310+0000 7fc49bf6c640 1 -- 172.21.15.84:0/3896028472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc49406c5b0 con 0x7fc4940c9000 2023-10-16T11:27:44.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.310+0000 7fc48a7fc640 1 -- 172.21.15.84:0/3896028472 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v272) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc48c018070 con 0x7fc4940c9000 2023-10-16T11:27:44.312 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-16T11:27:44.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.313+0000 7fc49bf6c640 1 -- 172.21.15.84:0/3896028472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4940c9000 msgr2=0x7fc49415d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:44.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.313+0000 7fc49bf6c640 1 --2- 172.21.15.84:0/3896028472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4940c9000 0x7fc49415d0a0 secure :-1 s=READY pgs=3492 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00c7a0 tx=0x7fc48c00cc60 comp rx=0 tx=0).stop 2023-10-16T11:27:44.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.313+0000 7fc49bf6c640 1 -- 172.21.15.84:0/3896028472 shutdown_connections 2023-10-16T11:27:44.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.313+0000 7fc49bf6c640 1 --2- 172.21.15.84:0/3896028472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4940c9000 0x7fc49415d0a0 unknown :-1 s=CLOSED pgs=3492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.313+0000 7fc49bf6c640 1 --2- 172.21.15.84:0/3896028472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4940addb0 0x7fc4940c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.313+0000 7fc49bf6c640 1 --2- 172.21.15.84:0/3896028472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49406a2e0 0x7fc4940c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.313+0000 7fc49bf6c640 1 -- 172.21.15.84:0/3896028472 >> 172.21.15.84:0/3896028472 conn(0x7fc4940ce9b0 msgr2=0x7fc4940ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:44.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.313+0000 7fc49bf6c640 1 -- 172.21.15.84:0/3896028472 shutdown_connections 2023-10-16T11:27:44.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.313+0000 7fc49bf6c640 1 -- 172.21.15.84:0/3896028472 wait complete. 2023-10-16T11:27:44.331 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:27:44.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.538+0000 7f33d12e5640 1 Processor -- start 2023-10-16T11:27:44.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.538+0000 7f33d12e5640 1 -- start start 2023-10-16T11:27:44.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.539+0000 7f33d12e5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33cc0ad970 0x7f33cc0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:44.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.539+0000 7f33d12e5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f33cc0ccfb0 0x7f33cc0cd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:44.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.539+0000 7f33d12e5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33cc15b9d0 con 0x7f33cc06c040 2023-10-16T11:27:44.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.539+0000 7f33d12e5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33cc15bcb0 con 0x7f33cc0ccfb0 2023-10-16T11:27:44.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.539+0000 7f33d12e5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc15bf90 con 0x7f33cc0ad970 2023-10-16T11:27:44.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.540+0000 7f33cbfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f33cc06c040 0x7f33cc06c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52522/0 (socket says 172.21.15.84:52522) 2023-10-16T11:27:44.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.540+0000 7f33cbfff640 1 -- 172.21.15.84:0/2591921885 learned_addr learned my addr 172.21.15.84:0/2591921885 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:44.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.540+0000 7f33c3fff640 1 --2- 172.21.15.84:0/2591921885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33cc0ad970 0x7f33cc0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:44.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.540+0000 7f33d0ae4640 1 --2- 172.21.15.84:0/2591921885 >> v2:172.21.15.157:3300/0 conn(0x7f33cc0ccfb0 0x7f33cc0cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:44.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.540+0000 7f33cb7fe640 1 -- 172.21.15.84:0/2591921885 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 931153033 0 0) 0x7f33cc15b9d0 con 0x7f33cc06c040 2023-10-16T11:27:44.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.540+0000 7f33cb7fe640 1 -- 172.21.15.84:0/2591921885 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33b8003520 con 0x7f33cc06c040 2023-10-16T11:27:44.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.540+0000 7f33cb7fe640 1 -- 172.21.15.84:0/2591921885 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3228258398 0 0) 0x7f33b8003520 con 0x7f33cc06c040 2023-10-16T11:27:44.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.540+0000 7f33cb7fe640 1 -- 172.21.15.84:0/2591921885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33cc0ad970 msgr2=0x7f33cc0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:44.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.540+0000 7f33cb7fe640 1 --2- 172.21.15.84:0/2591921885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33cc0ad970 0x7f33cc0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.540+0000 7f33cb7fe640 1 -- 172.21.15.84:0/2591921885 >> v2:172.21.15.157:3300/0 conn(0x7f33cc0ccfb0 msgr2=0x7f33cc0cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:44.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.540+0000 7f33cb7fe640 1 --2- 172.21.15.84:0/2591921885 >> v2:172.21.15.157:3300/0 conn(0x7f33cc0ccfb0 0x7f33cc0cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.540+0000 7f33cb7fe640 1 -- 172.21.15.84:0/2591921885 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33cc0cc270 con 0x7f33cc06c040 2023-10-16T11:27:44.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.540+0000 7f33cb7fe640 1 -- 172.21.15.84:0/2591921885 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f33bc001670 con 0x7f33cc06c040 2023-10-16T11:27:44.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.540+0000 7f33d0ae4640 1 --2- 172.21.15.84:0/2591921885 >> v2:172.21.15.157:3300/0 conn(0x7f33cc0ccfb0 0x7f33cc0cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:27:44.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.540+0000 7f33cb7fe640 1 -- 172.21.15.84:0/2591921885 >> v1:172.21.15.84:6789/0 conn(0x7f33cc06c040 legacy=0x7f33cc06c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:44.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.540+0000 7f33cb7fe640 1 --2- 172.21.15.84:0/2591921885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33b8003970 0x7f33b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:44.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.540+0000 7f33cb7fe640 1 --2- 172.21.15.84:0/2591921885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b80047c0 0x7f33b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:44.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.541+0000 7f33cb7fe640 1 --2- 172.21.15.84:0/2591921885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33cc0ad970 0x7f33b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:44.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.541+0000 7f33cb7fe640 1 -- 172.21.15.84:0/2591921885 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc15bcb0 con 0x7f33b8003970 2023-10-16T11:27:44.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.541+0000 7f33cb7fe640 1 -- 172.21.15.84:0/2591921885 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc15bf90 con 0x7f33b80047c0 2023-10-16T11:27:44.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.541+0000 7f33cb7fe640 1 -- 172.21.15.84:0/2591921885 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b800ba00 con 0x7f33cc0ad970 2023-10-16T11:27:44.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.541+0000 7f33cbfff640 1 --2- 172.21.15.84:0/2591921885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33b8003970 0x7f33b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:44.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.541+0000 7f33c3fff640 1 --2- 172.21.15.84:0/2591921885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b80047c0 0x7f33b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:44.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.541+0000 7f33cbfff640 1 -- 172.21.15.84:0/2591921885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33cc0ad970 msgr2=0x7f33b8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:44.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.541+0000 7f33cbfff640 1 --2- 172.21.15.84:0/2591921885 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33cc0ad970 0x7f33b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.541+0000 7f33cbfff640 1 -- 172.21.15.84:0/2591921885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b80047c0 msgr2=0x7f33b8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:44.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.541+0000 7f33cbfff640 1 --2- 172.21.15.84:0/2591921885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b80047c0 0x7f33b8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.541+0000 7f33cbfff640 1 -- 172.21.15.84:0/2591921885 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b8003520 con 0x7f33b8003970 2023-10-16T11:27:44.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.542+0000 7f33cbfff640 1 --2- 172.21.15.84:0/2591921885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33b8003970 0x7f33b8003d90 secure :-1 s=READY pgs=3494 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00c240 tx=0x7f33b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cb29ddf9270b27 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:44.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.542+0000 7f33cb7fe640 1 -- 172.21.15.84:0/2591921885 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc005ad0 con 0x7f33b8003970 2023-10-16T11:27:44.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.543+0000 7f33cb7fe640 1 -- 172.21.15.84:0/2591921885 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33bc005c30 con 0x7f33b8003970 2023-10-16T11:27:44.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.543+0000 7f33cb7fe640 1 -- 172.21.15.84:0/2591921885 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc02c8b0 con 0x7f33b8003970 2023-10-16T11:27:44.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.543+0000 7f33d12e5640 1 -- 172.21.15.84:0/2591921885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33b8003970 msgr2=0x7f33b8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:44.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.543+0000 7f33d12e5640 1 --2- 172.21.15.84:0/2591921885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33b8003970 0x7f33b8003d90 secure :-1 s=READY pgs=3494 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00c240 tx=0x7f33b80057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.543+0000 7f33d0ae4640 1 -- 172.21.15.84:0/2591921885 reap_dead start 2023-10-16T11:27:44.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.543+0000 7f33d12e5640 1 -- 172.21.15.84:0/2591921885 shutdown_connections 2023-10-16T11:27:44.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.543+0000 7f33d12e5640 1 -- 172.21.15.84:0/2591921885 >> 172.21.15.84:0/2591921885 conn(0x7f33cc0ce6a0 msgr2=0x7f33cc0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:44.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.543+0000 7f33d12e5640 1 -- 172.21.15.84:0/2591921885 shutdown_connections 2023-10-16T11:27:44.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.543+0000 7f33d12e5640 1 -- 172.21.15.84:0/2591921885 wait complete. 2023-10-16T11:27:44.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.545+0000 7f33d12e5640 1 Processor -- start 2023-10-16T11:27:44.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.545+0000 7f33d12e5640 1 -- start start 2023-10-16T11:27:44.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.546+0000 7f33d12e5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33b8003970 0x7f33cc0ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:44.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.546+0000 7f33d12e5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33cc0c7e80 0x7f33cc0af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:44.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.546+0000 7f33d12e5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33cc153740 0x7f33cc15d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:44.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.546+0000 7f33d12e5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc0af420 con 0x7f33b8003970 2023-10-16T11:27:44.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.546+0000 7f33d12e5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc0ae7a0 con 0x7f33cc0c7e80 2023-10-16T11:27:44.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.546+0000 7f33d12e5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc0aea80 con 0x7f33cc153740 2023-10-16T11:27:44.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.546+0000 7f33cbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33b8003970 0x7f33cc0ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:44.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.546+0000 7f33cbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33b8003970 0x7f33cc0ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45696/0 (socket says 172.21.15.84:45696) 2023-10-16T11:27:44.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.546+0000 7f33cbfff640 1 -- 172.21.15.84:0/3239634167 learned_addr learned my addr 172.21.15.84:0/3239634167 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:44.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.547+0000 7f33cbfff640 1 -- 172.21.15.84:0/3239634167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33cc153740 msgr2=0x7f33cc15d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:44.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.547+0000 7f33cbfff640 1 --2- 172.21.15.84:0/3239634167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33cc153740 0x7f33cc15d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.547+0000 7f33cbfff640 1 -- 172.21.15.84:0/3239634167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33cc0c7e80 msgr2=0x7f33cc0af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:44.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.547+0000 7f33cbfff640 1 --2- 172.21.15.84:0/3239634167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33cc0c7e80 0x7f33cc0af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.547+0000 7f33cbfff640 1 -- 172.21.15.84:0/3239634167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33cc0aed60 con 0x7f33b8003970 2023-10-16T11:27:44.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.547+0000 7f33cbfff640 1 --2- 172.21.15.84:0/3239634167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33b8003970 0x7f33cc0ae100 secure :-1 s=READY pgs=3495 cs=0 l=1 rev1=1 crypto rx=0x7f33bc001930 tx=0x7f33bc001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:44.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.547+0000 7f33c97fa640 1 -- 172.21.15.84:0/3239634167 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc03c040 con 0x7f33b8003970 2023-10-16T11:27:44.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.547+0000 7f33c97fa640 1 -- 172.21.15.84:0/3239634167 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33bc02ccc0 con 0x7f33b8003970 2023-10-16T11:27:44.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.547+0000 7f33d12e5640 1 -- 172.21.15.84:0/3239634167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33cc0af040 con 0x7f33b8003970 2023-10-16T11:27:44.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.547+0000 7f33c97fa640 1 -- 172.21.15.84:0/3239634167 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc007290 con 0x7f33b8003970 2023-10-16T11:27:44.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.547+0000 7f33d12e5640 1 -- 172.21.15.84:0/3239634167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33cc0c8300 con 0x7f33b8003970 2023-10-16T11:27:44.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.548+0000 7f33d12e5640 1 -- 172.21.15.84:0/3239634167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33cc04e9b0 con 0x7f33b8003970 2023-10-16T11:27:44.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.549+0000 7f33c97fa640 1 -- 172.21.15.84:0/3239634167 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 167) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f33bc032070 con 0x7f33b8003970 2023-10-16T11:27:44.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.550+0000 7f33c97fa640 1 --2- 172.21.15.84:0/3239634167 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f33ac06acc0 0x7f33ac06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:44.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.550+0000 7f33c97fa640 1 -- 172.21.15.84:0/3239634167 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(272..272 src has 1..272) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f33bc0aa950 con 0x7f33b8003970 2023-10-16T11:27:44.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.550+0000 7f33c3fff640 1 --2- 172.21.15.84:0/3239634167 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f33ac06acc0 0x7f33ac06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:44.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.551+0000 7f33c3fff640 1 --2- 172.21.15.84:0/3239634167 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f33ac06acc0 0x7f33ac06d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f33cc15dda0 tx=0x7f33b4008040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:44.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.555+0000 7f33c97fa640 1 -- 172.21.15.84:0/3239634167 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f33bc075a70 con 0x7f33b8003970 2023-10-16T11:27:44.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.766+0000 7f33d12e5640 1 -- 172.21.15.84:0/3239634167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33cc06b2b0 con 0x7f33b8003970 2023-10-16T11:27:44.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.768+0000 7f33c97fa640 1 -- 172.21.15.84:0/3239634167 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+4933 (secure 0 0 0) 0x7f33bc079090 con 0x7f33b8003970 2023-10-16T11:27:44.770 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:44.770 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":157,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7457,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3193159532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3193159532},{"type":"v1","addr":"172.21.15.157:6835","nonce":3193159532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":152},{"gid":8431,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1728897149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1728897149},{"type":"v1","addr":"172.21.15.157:6834","nonce":1728897149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":152},{"gid":13476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3409165899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3409165899},{"type":"v1","addr":"172.21.15.84:6835","nonce":3409165899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":157,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:27:22.871058+0000","modified":"2023-10-16T11:27:25.974995+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":13482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13482":{"gid":13482,"name":"a","rank":0,"incarnation":153,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2806813701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2806813701},{"type":"v1","addr":"172.21.15.84:6837","nonce":2806813701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[37],"metadata_pool":36,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":18}]} 2023-10-16T11:27:44.770 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 157 2023-10-16T11:27:44.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.770+0000 7f33d12e5640 1 -- 172.21.15.84:0/3239634167 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f33ac06acc0 msgr2=0x7f33ac06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:44.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.770+0000 7f33d12e5640 1 --2- 172.21.15.84:0/3239634167 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f33ac06acc0 0x7f33ac06d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f33cc15dda0 tx=0x7f33b4008040 comp rx=0 tx=0).stop 2023-10-16T11:27:44.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.770+0000 7f33d12e5640 1 -- 172.21.15.84:0/3239634167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33b8003970 msgr2=0x7f33cc0ae100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:44.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.770+0000 7f33d12e5640 1 --2- 172.21.15.84:0/3239634167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33b8003970 0x7f33cc0ae100 secure :-1 s=READY pgs=3495 cs=0 l=1 rev1=1 crypto rx=0x7f33bc001930 tx=0x7f33bc001a00 comp rx=0 tx=0).stop 2023-10-16T11:27:44.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.770+0000 7f33d12e5640 1 -- 172.21.15.84:0/3239634167 shutdown_connections 2023-10-16T11:27:44.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.770+0000 7f33d12e5640 1 --2- 172.21.15.84:0/3239634167 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f33ac06acc0 0x7f33ac06d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.770+0000 7f33d12e5640 1 --2- 172.21.15.84:0/3239634167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33cc153740 0x7f33cc15d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.770+0000 7f33d12e5640 1 --2- 172.21.15.84:0/3239634167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33cc0c7e80 0x7f33cc0af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.770+0000 7f33d12e5640 1 --2- 172.21.15.84:0/3239634167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33b8003970 0x7f33cc0ae100 unknown :-1 s=CLOSED pgs=3495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:44.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.770+0000 7f33d12e5640 1 -- 172.21.15.84:0/3239634167 >> 172.21.15.84:0/3239634167 conn(0x7f33cc0ce6a0 msgr2=0x7f33cc06ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:44.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.770+0000 7f33d12e5640 1 -- 172.21.15.84:0/3239634167 shutdown_connections 2023-10-16T11:27:44.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:44.770+0000 7f33d12e5640 1 -- 172.21.15.84:0/3239634167 wait complete. 2023-10-16T11:27:44.791 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:27:45.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.001+0000 7f172bf48640 1 Processor -- start 2023-10-16T11:27:45.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.002+0000 7f172bf48640 1 -- start start 2023-10-16T11:27:45.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.003+0000 7f172bf48640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172415aaa0 0x7f172415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:45.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.003+0000 7f172bf48640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f17240ce390 0x7f1724070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:45.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.003+0000 7f172bf48640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f172415ed60 con 0x7f172415ae80 2023-10-16T11:27:45.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.003+0000 7f172bf48640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f172406b810 con 0x7f17240ce390 2023-10-16T11:27:45.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.003+0000 7f172bf48640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17240707e0 con 0x7f172415aaa0 2023-10-16T11:27:45.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.003+0000 7f172a4e5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f17240ce390 0x7f1724070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:45.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.003+0000 7f17294e3640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f172415ae80 0x7f17240cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52536/0 (socket says 172.21.15.84:52536) 2023-10-16T11:27:45.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.003+0000 7f17294e3640 1 -- 172.21.15.84:0/495650484 learned_addr learned my addr 172.21.15.84:0/495650484 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:45.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.004+0000 7f1729ce4640 1 --2- 172.21.15.84:0/495650484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172415aaa0 0x7f172415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:45.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.004+0000 7f172a4e5640 1 -- 172.21.15.84:0/495650484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172415aaa0 msgr2=0x7f172415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:45.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.004+0000 7f172a4e5640 1 --2- 172.21.15.84:0/495650484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172415aaa0 0x7f172415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.004+0000 7f172a4e5640 1 -- 172.21.15.84:0/495650484 >> v1:172.21.15.84:6789/0 conn(0x7f172415ae80 legacy=0x7f17240cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:45.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.004+0000 7f172a4e5640 1 -- 172.21.15.84:0/495650484 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172415ef90 con 0x7f17240ce390 2023-10-16T11:27:45.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.004+0000 7f1728ce2640 1 -- 172.21.15.84:0/495650484 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 483502217 0 0) 0x7f172415ed60 con 0x7f172415ae80 2023-10-16T11:27:45.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.004+0000 7f1729ce4640 1 --2- 172.21.15.84:0/495650484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172415aaa0 0x7f172415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:45.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.005+0000 7f172a4e5640 1 --2- 172.21.15.84:0/495650484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17240ce390 0x7f1724070f50 secure :-1 s=READY pgs=3497 cs=0 l=1 rev1=1 crypto rx=0x7f171c009d90 tx=0x7f171c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da85d64da75d4d30 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:45.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.005+0000 7f1728ce2640 1 -- 172.21.15.84:0/495650484 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c02cc50 con 0x7f17240ce390 2023-10-16T11:27:45.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.005+0000 7f1728ce2640 1 -- 172.21.15.84:0/495650484 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171c02cdb0 con 0x7f17240ce390 2023-10-16T11:27:45.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.005+0000 7f1728ce2640 1 -- 172.21.15.84:0/495650484 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c034660 con 0x7f17240ce390 2023-10-16T11:27:45.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.006+0000 7f172bf48640 1 -- 172.21.15.84:0/495650484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17240ce390 msgr2=0x7f1724070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:45.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.006+0000 7f172bf48640 1 --2- 172.21.15.84:0/495650484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17240ce390 0x7f1724070f50 secure :-1 s=READY pgs=3497 cs=0 l=1 rev1=1 crypto rx=0x7f171c009d90 tx=0x7f171c02bea0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.006+0000 7f172bf48640 1 -- 172.21.15.84:0/495650484 shutdown_connections 2023-10-16T11:27:45.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.006+0000 7f172bf48640 1 --2- 172.21.15.84:0/495650484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17240ce390 0x7f1724070f50 unknown :-1 s=CLOSED pgs=3497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.006+0000 7f172bf48640 1 --2- 172.21.15.84:0/495650484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172415aaa0 0x7f172415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.006+0000 7f172bf48640 1 -- 172.21.15.84:0/495650484 >> 172.21.15.84:0/495650484 conn(0x7f17240cb410 msgr2=0x7f17240cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:45.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.006+0000 7f172bf48640 1 -- 172.21.15.84:0/495650484 shutdown_connections 2023-10-16T11:27:45.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.006+0000 7f172bf48640 1 -- 172.21.15.84:0/495650484 wait complete. 2023-10-16T11:27:45.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.008+0000 7f172bf48640 1 Processor -- start 2023-10-16T11:27:45.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.008+0000 7f172bf48640 1 -- start start 2023-10-16T11:27:45.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.009+0000 7f172bf48640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17240ce390 0x7f172414db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:45.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.009+0000 7f172bf48640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172415aaa0 0x7f17240c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:45.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.009+0000 7f172bf48640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172415ae80 0x7f172406cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:45.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.009+0000 7f172bf48640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17240c3a70 con 0x7f17240ce390 2023-10-16T11:27:45.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.009+0000 7f172bf48640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17240c2e60 con 0x7f172415aaa0 2023-10-16T11:27:45.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.009+0000 7f172bf48640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17240c3140 con 0x7f172415ae80 2023-10-16T11:27:45.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.009+0000 7f1729ce4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17240ce390 0x7f172414db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:45.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.009+0000 7f1729ce4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17240ce390 0x7f172414db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45718/0 (socket says 172.21.15.84:45718) 2023-10-16T11:27:45.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.009+0000 7f1729ce4640 1 -- 172.21.15.84:0/317255314 learned_addr learned my addr 172.21.15.84:0/317255314 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:45.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.010+0000 7f1729ce4640 1 -- 172.21.15.84:0/317255314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172415ae80 msgr2=0x7f172406cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:45.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.010+0000 7f1729ce4640 1 --2- 172.21.15.84:0/317255314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172415ae80 0x7f172406cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.010+0000 7f1729ce4640 1 -- 172.21.15.84:0/317255314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172415aaa0 msgr2=0x7f17240c24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:45.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.010+0000 7f1729ce4640 1 --2- 172.21.15.84:0/317255314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172415aaa0 0x7f17240c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.010+0000 7f1729ce4640 1 -- 172.21.15.84:0/317255314 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c009a70 con 0x7f17240ce390 2023-10-16T11:27:45.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.010+0000 7f1729ce4640 1 --2- 172.21.15.84:0/317255314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17240ce390 0x7f172414db60 secure :-1 s=READY pgs=3498 cs=0 l=1 rev1=1 crypto rx=0x7f172000c7f0 tx=0x7f172000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:45.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.010+0000 7f171a7fc640 1 -- 172.21.15.84:0/317255314 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720007a30 con 0x7f17240ce390 2023-10-16T11:27:45.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.010+0000 7f171a7fc640 1 -- 172.21.15.84:0/317255314 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1720007b90 con 0x7f17240ce390 2023-10-16T11:27:45.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.010+0000 7f172bf48640 1 -- 172.21.15.84:0/317255314 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17240c3420 con 0x7f17240ce390 2023-10-16T11:27:45.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.010+0000 7f171a7fc640 1 -- 172.21.15.84:0/317255314 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172000ce40 con 0x7f17240ce390 2023-10-16T11:27:45.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.010+0000 7f172bf48640 1 -- 172.21.15.84:0/317255314 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172414e0a0 con 0x7f17240ce390 2023-10-16T11:27:45.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.011+0000 7f172bf48640 1 -- 172.21.15.84:0/317255314 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172404e160 con 0x7f17240ce390 2023-10-16T11:27:45.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.012+0000 7f171a7fc640 1 -- 172.21.15.84:0/317255314 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 167) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f17200113f0 con 0x7f17240ce390 2023-10-16T11:27:45.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.013+0000 7f171a7fc640 1 --2- 172.21.15.84:0/317255314 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f170806acc0 0x7f170806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:45.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.013+0000 7f171a7fc640 1 -- 172.21.15.84:0/317255314 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(272..272 src has 1..272) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f1720089650 con 0x7f17240ce390 2023-10-16T11:27:45.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.013+0000 7f17294e3640 1 --2- 172.21.15.84:0/317255314 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f170806acc0 0x7f170806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:45.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.014+0000 7f17294e3640 1 --2- 172.21.15.84:0/317255314 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f170806acc0 0x7f170806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1714007fd0 tx=0x7f1714007920 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:45.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.018+0000 7f171a7fc640 1 -- 172.21.15.84:0/317255314 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1720054770 con 0x7f17240ce390 2023-10-16T11:27:45.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.213+0000 7f172bf48640 1 -- 172.21.15.84:0/317255314 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f172415ecd0 con 0x7f17240ce390 2023-10-16T11:27:45.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.216+0000 7f171a7fc640 1 -- 172.21.15.84:0/317255314 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v272) v1 ==== 74+0+17543 (secure 0 0 0) 0x7f1720057d90 con 0x7f17240ce390 2023-10-16T11:27:45.219 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:45.219 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":272,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:27:43.469311+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:27:20.716240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"269","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_data","create_time":"2023-10-16T11:27:21.905553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"270","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2893482429":"2023-10-17T11:27:43.469263+0000","172.21.15.84:0/1321046441":"2023-10-17T11:27:43.469263+0000","172.21.15.84:0/822007176":"2023-10-17T11:27:43.469263+0000","172.21.15.84:0/3876279166":"2023-10-17T11:27:43.469263+0000","172.21.15.84:6833/129259578":"2023-10-17T11:27:43.469263+0000","172.21.15.84:0/2238142758":"2023-10-17T11:27:43.469263+0000","172.21.15.84:0/1788498161":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/3634760901":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/3509607392":"2023-10-17T11:27:43.469263+0000","172.21.15.84:0/3842621119":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/40770371":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/1147031855":"2023-10-17T11:27:37.140228+0000","172.21.15.84:6832/129259578":"2023-10-17T11:27:43.469263+0000","172.21.15.84:6833/731888425":"2023-10-17T11:27:37.140228+0000","172.21.15.84:6832/731888425":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/2353059609":"2023-10-17T11:27:37.140228+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:27:45.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.219+0000 7f172bf48640 1 -- 172.21.15.84:0/317255314 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f170806acc0 msgr2=0x7f170806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:45.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.219+0000 7f172bf48640 1 --2- 172.21.15.84:0/317255314 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f170806acc0 0x7f170806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1714007fd0 tx=0x7f1714007920 comp rx=0 tx=0).stop 2023-10-16T11:27:45.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.219+0000 7f172bf48640 1 -- 172.21.15.84:0/317255314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17240ce390 msgr2=0x7f172414db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:45.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.219+0000 7f172bf48640 1 --2- 172.21.15.84:0/317255314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17240ce390 0x7f172414db60 secure :-1 s=READY pgs=3498 cs=0 l=1 rev1=1 crypto rx=0x7f172000c7f0 tx=0x7f172000cbf0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.219+0000 7f172bf48640 1 -- 172.21.15.84:0/317255314 shutdown_connections 2023-10-16T11:27:45.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.219+0000 7f172bf48640 1 --2- 172.21.15.84:0/317255314 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f170806acc0 0x7f170806d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.219+0000 7f172bf48640 1 --2- 172.21.15.84:0/317255314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f172415ae80 0x7f172406cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.219+0000 7f172bf48640 1 --2- 172.21.15.84:0/317255314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172415aaa0 0x7f17240c24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.219+0000 7f172bf48640 1 --2- 172.21.15.84:0/317255314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17240ce390 0x7f172414db60 unknown :-1 s=CLOSED pgs=3498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.219+0000 7f172bf48640 1 -- 172.21.15.84:0/317255314 >> 172.21.15.84:0/317255314 conn(0x7f17240cb410 msgr2=0x7f1724159590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:45.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.219+0000 7f172bf48640 1 -- 172.21.15.84:0/317255314 shutdown_connections 2023-10-16T11:27:45.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.220+0000 7f172bf48640 1 -- 172.21.15.84:0/317255314 wait complete. 2023-10-16T11:27:45.241 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T11:27:45.241 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:27:45.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.457+0000 7f33a6e8a640 1 Processor -- start 2023-10-16T11:27:45.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.457+0000 7f33a6e8a640 1 -- start start 2023-10-16T11:27:45.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.459+0000 7f33a6e8a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f33a006c040 0x7f33a006c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:45.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.459+0000 7f33a6e8a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a00ad970 0x7f33a015e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:45.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.459+0000 7f33a6e8a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a006c960 con 0x7f33a015ebc0 2023-10-16T11:27:45.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.459+0000 7f33a6e8a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a015b9d0 con 0x7f33a006c040 2023-10-16T11:27:45.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.459+0000 7f33a6e8a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a015bcb0 con 0x7f33a00ad970 2023-10-16T11:27:45.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.459+0000 7f33a5e88640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f33a006c040 0x7f33a006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:45.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.459+0000 7f33a6689640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f33a015ebc0 0x7f33a00cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52540/0 (socket says 172.21.15.84:52540) 2023-10-16T11:27:45.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.459+0000 7f33a6689640 1 -- 172.21.15.84:0/3266267403 learned_addr learned my addr 172.21.15.84:0/3266267403 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:45.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.460+0000 7f33a5687640 1 --2- 172.21.15.84:0/3266267403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a00ad970 0x7f33a015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:45.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.460+0000 7f33a5e88640 1 -- 172.21.15.84:0/3266267403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a00ad970 msgr2=0x7f33a015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:45.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.460+0000 7f33a4e86640 1 -- 172.21.15.84:0/3266267403 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529917939 0 0) 0x7f33a006c960 con 0x7f33a015ebc0 2023-10-16T11:27:45.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.460+0000 7f33a5e88640 1 --2- 172.21.15.84:0/3266267403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a00ad970 0x7f33a015e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.460+0000 7f33a5e88640 1 -- 172.21.15.84:0/3266267403 >> v1:172.21.15.84:6789/0 conn(0x7f33a015ebc0 legacy=0x7f33a00cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:45.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.460+0000 7f33a5e88640 1 -- 172.21.15.84:0/3266267403 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a015bf90 con 0x7f33a006c040 2023-10-16T11:27:45.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.460+0000 7f33a5687640 1 --2- 172.21.15.84:0/3266267403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a00ad970 0x7f33a015e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:45.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.461+0000 7f33a5e88640 1 --2- 172.21.15.84:0/3266267403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a006c040 0x7f33a006c420 secure :-1 s=READY pgs=3500 cs=0 l=1 rev1=1 crypto rx=0x7f339c0097d0 tx=0x7f339c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a429581d9c0d469b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:45.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.461+0000 7f33a4e86640 1 -- 172.21.15.84:0/3266267403 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c031800 con 0x7f33a006c040 2023-10-16T11:27:45.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.461+0000 7f33a4e86640 1 -- 172.21.15.84:0/3266267403 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c031960 con 0x7f33a006c040 2023-10-16T11:27:45.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.461+0000 7f33a4e86640 1 -- 172.21.15.84:0/3266267403 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c031cf0 con 0x7f33a006c040 2023-10-16T11:27:45.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.462+0000 7f33a6e8a640 1 -- 172.21.15.84:0/3266267403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a006c040 msgr2=0x7f33a006c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:45.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.462+0000 7f33a6e8a640 1 --2- 172.21.15.84:0/3266267403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a006c040 0x7f33a006c420 secure :-1 s=READY pgs=3500 cs=0 l=1 rev1=1 crypto rx=0x7f339c0097d0 tx=0x7f339c02b8a0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.462+0000 7f33a6e8a640 1 -- 172.21.15.84:0/3266267403 shutdown_connections 2023-10-16T11:27:45.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.462+0000 7f33a6e8a640 1 --2- 172.21.15.84:0/3266267403 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a00ad970 0x7f33a015e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.462+0000 7f33a6e8a640 1 --2- 172.21.15.84:0/3266267403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a006c040 0x7f33a006c420 unknown :-1 s=CLOSED pgs=3500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.462+0000 7f33a6e8a640 1 -- 172.21.15.84:0/3266267403 >> 172.21.15.84:0/3266267403 conn(0x7f33a00ce6a0 msgr2=0x7f33a00ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:45.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.462+0000 7f33a6e8a640 1 -- 172.21.15.84:0/3266267403 shutdown_connections 2023-10-16T11:27:45.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.462+0000 7f33a6e8a640 1 -- 172.21.15.84:0/3266267403 wait complete. 2023-10-16T11:27:45.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.463+0000 7f33a6e8a640 1 Processor -- start 2023-10-16T11:27:45.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.464+0000 7f33a6e8a640 1 -- start start 2023-10-16T11:27:45.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.464+0000 7f33a6e8a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a006c040 0x7f33a01f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:45.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.465+0000 7f33a6e8a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00ad970 0x7f33a0156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:45.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.465+0000 7f33a6e8a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a015ebc0 0x7f33a00c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:45.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.465+0000 7f33a6e8a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a01f2630 con 0x7f33a00ad970 2023-10-16T11:27:45.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.465+0000 7f33a6e8a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a01f2910 con 0x7f33a006c040 2023-10-16T11:27:45.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.465+0000 7f33a6e8a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a01f2bf0 con 0x7f33a015ebc0 2023-10-16T11:27:45.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.465+0000 7f33a5687640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00ad970 0x7f33a0156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:45.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.465+0000 7f33a5687640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00ad970 0x7f33a0156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45744/0 (socket says 172.21.15.84:45744) 2023-10-16T11:27:45.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.465+0000 7f33a5687640 1 -- 172.21.15.84:0/3756221582 learned_addr learned my addr 172.21.15.84:0/3756221582 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:45.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.465+0000 7f33a5e88640 1 --2- 172.21.15.84:0/3756221582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a006c040 0x7f33a01f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:45.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.465+0000 7f33a6689640 1 --2- 172.21.15.84:0/3756221582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a015ebc0 0x7f33a00c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:45.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.465+0000 7f33a5687640 1 -- 172.21.15.84:0/3756221582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a015ebc0 msgr2=0x7f33a00c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:45.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.465+0000 7f33a5687640 1 --2- 172.21.15.84:0/3756221582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a015ebc0 0x7f33a00c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.465+0000 7f33a5687640 1 -- 172.21.15.84:0/3756221582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a006c040 msgr2=0x7f33a01f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:45.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.465+0000 7f33a5687640 1 --2- 172.21.15.84:0/3756221582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a006c040 0x7f33a01f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.465+0000 7f33a5687640 1 -- 172.21.15.84:0/3756221582 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339c009400 con 0x7f33a00ad970 2023-10-16T11:27:45.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.466+0000 7f33a5687640 1 --2- 172.21.15.84:0/3756221582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00ad970 0x7f33a0156f70 secure :-1 s=READY pgs=3501 cs=0 l=1 rev1=1 crypto rx=0x7f339000c7f0 tx=0x7f339000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:45.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.466+0000 7f33967fc640 1 -- 172.21.15.84:0/3756221582 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3390007820 con 0x7f33a00ad970 2023-10-16T11:27:45.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.466+0000 7f33967fc640 1 -- 172.21.15.84:0/3756221582 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3390007980 con 0x7f33a00ad970 2023-10-16T11:27:45.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.466+0000 7f33967fc640 1 -- 172.21.15.84:0/3756221582 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3390007cf0 con 0x7f33a00ad970 2023-10-16T11:27:45.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.466+0000 7f33a6e8a640 1 -- 172.21.15.84:0/3756221582 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a01f2ed0 con 0x7f33a00ad970 2023-10-16T11:27:45.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.466+0000 7f33a6e8a640 1 -- 172.21.15.84:0/3756221582 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a01f22e0 con 0x7f33a00ad970 2023-10-16T11:27:45.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.467+0000 7f33a6e8a640 1 -- 172.21.15.84:0/3756221582 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a006b330 con 0x7f33a00ad970 2023-10-16T11:27:45.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.468+0000 7f33967fc640 1 -- 172.21.15.84:0/3756221582 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 167) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f33900113f0 con 0x7f33a00ad970 2023-10-16T11:27:45.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.469+0000 7f33967fc640 1 --2- 172.21.15.84:0/3756221582 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f338406acc0 0x7f338406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:45.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.469+0000 7f33967fc640 1 -- 172.21.15.84:0/3756221582 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(272..272 src has 1..272) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f3390089910 con 0x7f33a00ad970 2023-10-16T11:27:45.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.470+0000 7f33a5e88640 1 --2- 172.21.15.84:0/3756221582 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f338406acc0 0x7f338406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:45.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.470+0000 7f33a5e88640 1 --2- 172.21.15.84:0/3756221582 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f338406acc0 0x7f338406d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f339c036000 tx=0x7f339c03c040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:45.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.474+0000 7f33967fc640 1 -- 172.21.15.84:0/3756221582 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3390054960 con 0x7f33a00ad970 2023-10-16T11:27:45.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.701+0000 7f33a6e8a640 1 -- 172.21.15.84:0/3756221582 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33a004e9b0 con 0x7f33a00ad970 2023-10-16T11:27:45.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.702+0000 7f33967fc640 1 -- 172.21.15.84:0/3756221582 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+4933 (secure 0 0 0) 0x7f3390057f80 con 0x7f33a00ad970 2023-10-16T11:27:45.705 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:45.705 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":157,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7457,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3193159532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3193159532},{"type":"v1","addr":"172.21.15.157:6835","nonce":3193159532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":152},{"gid":8431,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1728897149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1728897149},{"type":"v1","addr":"172.21.15.157:6834","nonce":1728897149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":152},{"gid":13476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3409165899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3409165899},{"type":"v1","addr":"172.21.15.84:6835","nonce":3409165899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":157,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:27:22.871058+0000","modified":"2023-10-16T11:27:25.974995+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":13482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13482":{"gid":13482,"name":"a","rank":0,"incarnation":153,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2806813701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2806813701},{"type":"v1","addr":"172.21.15.84:6837","nonce":2806813701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[37],"metadata_pool":36,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":18}]} 2023-10-16T11:27:45.705 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 157 2023-10-16T11:27:45.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.705+0000 7f33a6e8a640 1 -- 172.21.15.84:0/3756221582 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f338406acc0 msgr2=0x7f338406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:45.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.705+0000 7f33a6e8a640 1 --2- 172.21.15.84:0/3756221582 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f338406acc0 0x7f338406d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f339c036000 tx=0x7f339c03c040 comp rx=0 tx=0).stop 2023-10-16T11:27:45.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.705+0000 7f33a6e8a640 1 -- 172.21.15.84:0/3756221582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00ad970 msgr2=0x7f33a0156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:45.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.705+0000 7f33a6e8a640 1 --2- 172.21.15.84:0/3756221582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00ad970 0x7f33a0156f70 secure :-1 s=READY pgs=3501 cs=0 l=1 rev1=1 crypto rx=0x7f339000c7f0 tx=0x7f339000ccb0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.705+0000 7f33a6e8a640 1 -- 172.21.15.84:0/3756221582 shutdown_connections 2023-10-16T11:27:45.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.705+0000 7f33a6e8a640 1 --2- 172.21.15.84:0/3756221582 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f338406acc0 0x7f338406d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.705+0000 7f33a6e8a640 1 --2- 172.21.15.84:0/3756221582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a015ebc0 0x7f33a00c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.705+0000 7f33a6e8a640 1 --2- 172.21.15.84:0/3756221582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a00ad970 0x7f33a0156f70 unknown :-1 s=CLOSED pgs=3501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.705+0000 7f33a6e8a640 1 --2- 172.21.15.84:0/3756221582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a006c040 0x7f33a01f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.705+0000 7f33a6e8a640 1 -- 172.21.15.84:0/3756221582 >> 172.21.15.84:0/3756221582 conn(0x7f33a00ce6a0 msgr2=0x7f33a00c9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:45.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.705+0000 7f33a6e8a640 1 -- 172.21.15.84:0/3756221582 shutdown_connections 2023-10-16T11:27:45.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.705+0000 7f33a6e8a640 1 -- 172.21.15.84:0/3756221582 wait complete. 2023-10-16T11:27:45.724 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:27:45.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.926+0000 7f15b2997640 1 Processor -- start 2023-10-16T11:27:45.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.927+0000 7f15b2997640 1 -- start start 2023-10-16T11:27:45.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.928+0000 7f15b2997640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f15ac15aaa0 0x7f15ac15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:45.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.928+0000 7f15b2997640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f15ac0ce390 0x7f15ac070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:45.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.928+0000 7f15b2997640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15ac15ed60 con 0x7f15ac15ae80 2023-10-16T11:27:45.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.928+0000 7f15b2997640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ac06b810 con 0x7f15ac15aaa0 2023-10-16T11:27:45.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.928+0000 7f15b2997640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ac0707e0 con 0x7f15ac0ce390 2023-10-16T11:27:45.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.929+0000 7f15abfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f15ac15aaa0 0x7f15ac15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:45.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.929+0000 7f15abfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f15ac15aaa0 0x7f15ac15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36812/0 (socket says 172.21.15.84:36812) 2023-10-16T11:27:45.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.929+0000 7f15abfff640 1 -- 172.21.15.84:0/2364217583 learned_addr learned my addr 172.21.15.84:0/2364217583 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:45.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.929+0000 7f15b0f34640 1 --2- 172.21.15.84:0/2364217583 >> v2:172.21.15.157:3300/0 conn(0x7f15ac0ce390 0x7f15ac070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:45.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.929+0000 7f15abfff640 1 -- 172.21.15.84:0/2364217583 >> v2:172.21.15.157:3300/0 conn(0x7f15ac0ce390 msgr2=0x7f15ac070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:45.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.929+0000 7f15abfff640 1 --2- 172.21.15.84:0/2364217583 >> v2:172.21.15.157:3300/0 conn(0x7f15ac0ce390 0x7f15ac070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.929+0000 7f15abfff640 1 -- 172.21.15.84:0/2364217583 >> v1:172.21.15.157:6789/0 conn(0x7f15ac15ae80 legacy=0x7f15ac0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:45.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.929+0000 7f15abfff640 1 -- 172.21.15.84:0/2364217583 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac15ef90 con 0x7f15ac15aaa0 2023-10-16T11:27:45.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.931+0000 7f15abfff640 1 --2- 172.21.15.84:0/2364217583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15ac15aaa0 0x7f15ac15e820 secure :-1 s=READY pgs=3504 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0c2490 tx=0x7f159c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11656b18ed1f13b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:45.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.931+0000 7f15aaffd640 1 -- 172.21.15.84:0/2364217583 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c031870 con 0x7f15ac15aaa0 2023-10-16T11:27:45.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.931+0000 7f15aaffd640 1 -- 172.21.15.84:0/2364217583 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159c0319d0 con 0x7f15ac15aaa0 2023-10-16T11:27:45.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.931+0000 7f15aaffd640 1 -- 172.21.15.84:0/2364217583 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c031d60 con 0x7f15ac15aaa0 2023-10-16T11:27:45.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.932+0000 7f15b2997640 1 -- 172.21.15.84:0/2364217583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15ac15aaa0 msgr2=0x7f15ac15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.932+0000 7f15b2997640 1 --2- 172.21.15.84:0/2364217583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15ac15aaa0 0x7f15ac15e820 secure :-1 s=READY pgs=3504 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0c2490 tx=0x7f159c02b910 comp rx=0 tx=0).stop 2023-10-16T11:27:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.932+0000 7f15b2997640 1 -- 172.21.15.84:0/2364217583 shutdown_connections 2023-10-16T11:27:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.932+0000 7f15b2997640 1 --2- 172.21.15.84:0/2364217583 >> v2:172.21.15.157:3300/0 conn(0x7f15ac0ce390 0x7f15ac070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.932+0000 7f15b2997640 1 --2- 172.21.15.84:0/2364217583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15ac15aaa0 0x7f15ac15e820 unknown :-1 s=CLOSED pgs=3504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.932+0000 7f15b2997640 1 -- 172.21.15.84:0/2364217583 >> 172.21.15.84:0/2364217583 conn(0x7f15ac0cb410 msgr2=0x7f15ac0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.932+0000 7f15b2997640 1 -- 172.21.15.84:0/2364217583 shutdown_connections 2023-10-16T11:27:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.932+0000 7f15b2997640 1 -- 172.21.15.84:0/2364217583 wait complete. 2023-10-16T11:27:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.934+0000 7f15b2997640 1 Processor -- start 2023-10-16T11:27:45.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.934+0000 7f15b2997640 1 -- start start 2023-10-16T11:27:45.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.935+0000 7f15b2997640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15ac0ce390 0x7f15ac155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:45.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.935+0000 7f15b2997640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15ac15aaa0 0x7f15ac15dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:45.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.935+0000 7f15b2997640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15ac15ae80 0x7f15ac1f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:45.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.935+0000 7f15b2997640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac156450 con 0x7f15ac15aaa0 2023-10-16T11:27:45.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.935+0000 7f15b2997640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac15b690 con 0x7f15ac0ce390 2023-10-16T11:27:45.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.935+0000 7f15b2997640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac15b970 con 0x7f15ac15ae80 2023-10-16T11:27:45.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.935+0000 7f15ab7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15ac15aaa0 0x7f15ac15dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:45.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.935+0000 7f15ab7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15ac15aaa0 0x7f15ac15dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36820/0 (socket says 172.21.15.84:36820) 2023-10-16T11:27:45.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.935+0000 7f15ab7fe640 1 -- 172.21.15.84:0/590690605 learned_addr learned my addr 172.21.15.84:0/590690605 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:45.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.935+0000 7f15abfff640 1 --2- 172.21.15.84:0/590690605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15ac0ce390 0x7f15ac155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:45.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.935+0000 7f15ab7fe640 1 -- 172.21.15.84:0/590690605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15ac15ae80 msgr2=0x7f15ac1f39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:45.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.935+0000 7f15ab7fe640 1 --2- 172.21.15.84:0/590690605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15ac15ae80 0x7f15ac1f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.935+0000 7f15ab7fe640 1 -- 172.21.15.84:0/590690605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15ac0ce390 msgr2=0x7f15ac155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:45.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.935+0000 7f15ab7fe640 1 --2- 172.21.15.84:0/590690605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15ac0ce390 0x7f15ac155f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:45.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.935+0000 7f15ab7fe640 1 -- 172.21.15.84:0/590690605 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159c0094d0 con 0x7f15ac15aaa0 2023-10-16T11:27:45.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.936+0000 7f15ab7fe640 1 --2- 172.21.15.84:0/590690605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15ac15aaa0 0x7f15ac15dd50 secure :-1 s=READY pgs=3505 cs=0 l=1 rev1=1 crypto rx=0x7f15a000b3c0 tx=0x7f15a000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:45.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.936+0000 7f15a8ff9640 1 -- 172.21.15.84:0/590690605 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a0011830 con 0x7f15ac15aaa0 2023-10-16T11:27:45.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.936+0000 7f15a8ff9640 1 -- 172.21.15.84:0/590690605 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a0011990 con 0x7f15ac15aaa0 2023-10-16T11:27:45.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.936+0000 7f15a8ff9640 1 -- 172.21.15.84:0/590690605 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a0011d80 con 0x7f15ac15aaa0 2023-10-16T11:27:45.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.936+0000 7f15b2997640 1 -- 172.21.15.84:0/590690605 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ac15bc50 con 0x7f15ac15aaa0 2023-10-16T11:27:45.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.936+0000 7f15b2997640 1 -- 172.21.15.84:0/590690605 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ac15c210 con 0x7f15ac15aaa0 2023-10-16T11:27:45.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.937+0000 7f15b2997640 1 -- 172.21.15.84:0/590690605 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ac15c430 con 0x7f15ac15aaa0 2023-10-16T11:27:45.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.938+0000 7f15a8ff9640 1 -- 172.21.15.84:0/590690605 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 167) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f15a001b460 con 0x7f15ac15aaa0 2023-10-16T11:27:45.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.939+0000 7f15a8ff9640 1 --2- 172.21.15.84:0/590690605 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f159806acc0 0x7f159806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:45.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.939+0000 7f15a8ff9640 1 -- 172.21.15.84:0/590690605 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(272..272 src has 1..272) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f15a0089100 con 0x7f15ac15aaa0 2023-10-16T11:27:45.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.939+0000 7f15abfff640 1 --2- 172.21.15.84:0/590690605 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f159806acc0 0x7f159806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:45.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.940+0000 7f15abfff640 1 --2- 172.21.15.84:0/590690605 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f159806acc0 0x7f159806d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f159c036000 tx=0x7f159c03c040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:45.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:45.943+0000 7f15a8ff9640 1 -- 172.21.15.84:0/590690605 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f15a0054220 con 0x7f15ac15aaa0 2023-10-16T11:27:46.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.158+0000 7f15b2997640 1 -- 172.21.15.84:0/590690605 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15ac04e160 con 0x7f15ac15aaa0 2023-10-16T11:27:46.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.159+0000 7f15a8ff9640 1 -- 172.21.15.84:0/590690605 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+4933 (secure 0 0 0) 0x7f15a0057840 con 0x7f15ac15aaa0 2023-10-16T11:27:46.161 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:46.161 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":157,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7457,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3193159532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3193159532},{"type":"v1","addr":"172.21.15.157:6835","nonce":3193159532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":152},{"gid":8431,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1728897149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1728897149},{"type":"v1","addr":"172.21.15.157:6834","nonce":1728897149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":152},{"gid":13476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3409165899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3409165899},{"type":"v1","addr":"172.21.15.84:6835","nonce":3409165899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":157,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:27:22.871058+0000","modified":"2023-10-16T11:27:25.974995+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":13482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13482":{"gid":13482,"name":"a","rank":0,"incarnation":153,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2806813701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2806813701},{"type":"v1","addr":"172.21.15.84:6837","nonce":2806813701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[37],"metadata_pool":36,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":18}]} 2023-10-16T11:27:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 157 2023-10-16T11:27:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.161+0000 7f15b2997640 1 -- 172.21.15.84:0/590690605 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f159806acc0 msgr2=0x7f159806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.161+0000 7f15b2997640 1 --2- 172.21.15.84:0/590690605 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f159806acc0 0x7f159806d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f159c036000 tx=0x7f159c03c040 comp rx=0 tx=0).stop 2023-10-16T11:27:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.161+0000 7f15b2997640 1 -- 172.21.15.84:0/590690605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15ac15aaa0 msgr2=0x7f15ac15dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.161+0000 7f15b2997640 1 --2- 172.21.15.84:0/590690605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15ac15aaa0 0x7f15ac15dd50 secure :-1 s=READY pgs=3505 cs=0 l=1 rev1=1 crypto rx=0x7f15a000b3c0 tx=0x7f15a000b880 comp rx=0 tx=0).stop 2023-10-16T11:27:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.161+0000 7f15b2997640 1 -- 172.21.15.84:0/590690605 shutdown_connections 2023-10-16T11:27:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.161+0000 7f15b2997640 1 --2- 172.21.15.84:0/590690605 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f159806acc0 0x7f159806d180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.161+0000 7f15b2997640 1 --2- 172.21.15.84:0/590690605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15ac15ae80 0x7f15ac1f39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.161+0000 7f15b2997640 1 --2- 172.21.15.84:0/590690605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15ac15aaa0 0x7f15ac15dd50 unknown :-1 s=CLOSED pgs=3505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.161+0000 7f15b2997640 1 --2- 172.21.15.84:0/590690605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15ac0ce390 0x7f15ac155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.161+0000 7f15b2997640 1 -- 172.21.15.84:0/590690605 >> 172.21.15.84:0/590690605 conn(0x7f15ac0cb410 msgr2=0x7f15ac0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:46.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.161+0000 7f15b2997640 1 -- 172.21.15.84:0/590690605 shutdown_connections 2023-10-16T11:27:46.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.161+0000 7f15b2997640 1 -- 172.21.15.84:0/590690605 wait complete. 2023-10-16T11:27:46.181 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:27:46.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.383+0000 7fc9fe80d640 1 Processor -- start 2023-10-16T11:27:46.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.383+0000 7fc9fe80d640 1 -- start start 2023-10-16T11:27:46.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.384+0000 7fc9fe80d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc9f80ad970 0x7fc9f80cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:46.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.384+0000 7fc9fe80d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9f80ccfb0 0x7fc9f80cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:46.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.384+0000 7fc9fe80d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f815b9d0 con 0x7fc9f806c040 2023-10-16T11:27:46.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.384+0000 7fc9fe80d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f815bcb0 con 0x7fc9f80ad970 2023-10-16T11:27:46.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.384+0000 7fc9fe80d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f815bf90 con 0x7fc9f80ccfb0 2023-10-16T11:27:46.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.385+0000 7fc9fd00a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc9f80ad970 0x7fc9f80cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:46.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.385+0000 7fc9fd00a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc9f80ad970 0x7fc9f80cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36822/0 (socket says 172.21.15.84:36822) 2023-10-16T11:27:46.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.385+0000 7fc9fd00a640 1 -- 172.21.15.84:0/3834930520 learned_addr learned my addr 172.21.15.84:0/3834930520 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:46.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.385+0000 7fc9fe00c640 1 --2- 172.21.15.84:0/3834930520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9f80ccfb0 0x7fc9f80cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:46.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.385+0000 7fc9fd00a640 1 -- 172.21.15.84:0/3834930520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9f80ccfb0 msgr2=0x7fc9f80cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:46.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.385+0000 7fc9fd00a640 1 --2- 172.21.15.84:0/3834930520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9f80ccfb0 0x7fc9f80cd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.385+0000 7fc9fd00a640 1 -- 172.21.15.84:0/3834930520 >> v1:172.21.15.157:6789/0 conn(0x7fc9f806c040 legacy=0x7fc9f806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:46.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.385+0000 7fc9fd00a640 1 -- 172.21.15.84:0/3834930520 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f80cc270 con 0x7fc9f80ad970 2023-10-16T11:27:46.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.386+0000 7fc9fd00a640 1 --2- 172.21.15.84:0/3834930520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f80ad970 0x7fc9f80cca70 secure :-1 s=READY pgs=3506 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8009870 tx=0x7fc9e802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7817fd24fda73c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:46.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.387+0000 7fc9fc809640 1 -- 172.21.15.84:0/3834930520 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8031640 con 0x7fc9f80ad970 2023-10-16T11:27:46.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.387+0000 7fc9fc809640 1 -- 172.21.15.84:0/3834930520 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e80317a0 con 0x7fc9f80ad970 2023-10-16T11:27:46.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.387+0000 7fc9fc809640 1 -- 172.21.15.84:0/3834930520 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8031b30 con 0x7fc9f80ad970 2023-10-16T11:27:46.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.387+0000 7fc9fe80d640 1 -- 172.21.15.84:0/3834930520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f80ad970 msgr2=0x7fc9f80cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:46.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.387+0000 7fc9fe80d640 1 --2- 172.21.15.84:0/3834930520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f80ad970 0x7fc9f80cca70 secure :-1 s=READY pgs=3506 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8009870 tx=0x7fc9e802b8f0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.387+0000 7fc9fe80d640 1 -- 172.21.15.84:0/3834930520 shutdown_connections 2023-10-16T11:27:46.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.387+0000 7fc9fe80d640 1 --2- 172.21.15.84:0/3834930520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9f80ccfb0 0x7fc9f80cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.387+0000 7fc9fe80d640 1 --2- 172.21.15.84:0/3834930520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f80ad970 0x7fc9f80cca70 unknown :-1 s=CLOSED pgs=3506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.387+0000 7fc9fe80d640 1 -- 172.21.15.84:0/3834930520 >> 172.21.15.84:0/3834930520 conn(0x7fc9f80ce6a0 msgr2=0x7fc9f80ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:46.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.388+0000 7fc9fe80d640 1 -- 172.21.15.84:0/3834930520 shutdown_connections 2023-10-16T11:27:46.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.388+0000 7fc9fe80d640 1 -- 172.21.15.84:0/3834930520 wait complete. 2023-10-16T11:27:46.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.390+0000 7fc9fe80d640 1 Processor -- start 2023-10-16T11:27:46.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.390+0000 7fc9fe80d640 1 -- start start 2023-10-16T11:27:46.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.391+0000 7fc9fe80d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9f806c040 0x7fc9f814ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:46.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.391+0000 7fc9fe80d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f80ad970 0x7fc9f80b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:46.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.391+0000 7fc9fe80d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9f80ccfb0 0x7fc9f80c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:46.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.391+0000 7fc9fe80d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80ae2a0 con 0x7fc9f80ad970 2023-10-16T11:27:46.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.391+0000 7fc9fe80d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80ae580 con 0x7fc9f806c040 2023-10-16T11:27:46.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.391+0000 7fc9fe80d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80ae860 con 0x7fc9f80ccfb0 2023-10-16T11:27:46.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.391+0000 7fc9fd00a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f80ad970 0x7fc9f80b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:46.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.391+0000 7fc9fd00a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f80ad970 0x7fc9f80b0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36824/0 (socket says 172.21.15.84:36824) 2023-10-16T11:27:46.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.391+0000 7fc9fd00a640 1 -- 172.21.15.84:0/3969640212 learned_addr learned my addr 172.21.15.84:0/3969640212 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:46.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.392+0000 7fc9fd80b640 1 --2- 172.21.15.84:0/3969640212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9f806c040 0x7fc9f814ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:46.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.392+0000 7fc9fd00a640 1 -- 172.21.15.84:0/3969640212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9f80ccfb0 msgr2=0x7fc9f80c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:46.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.392+0000 7fc9fd00a640 1 --2- 172.21.15.84:0/3969640212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9f80ccfb0 0x7fc9f80c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.392+0000 7fc9fd00a640 1 -- 172.21.15.84:0/3969640212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9f806c040 msgr2=0x7fc9f814ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:46.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.392+0000 7fc9fd00a640 1 --2- 172.21.15.84:0/3969640212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9f806c040 0x7fc9f814ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.392+0000 7fc9fd00a640 1 -- 172.21.15.84:0/3969640212 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e80094d0 con 0x7fc9f80ad970 2023-10-16T11:27:46.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.392+0000 7fc9fd00a640 1 --2- 172.21.15.84:0/3969640212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f80ad970 0x7fc9f80b0710 secure :-1 s=READY pgs=3507 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80097c0 tx=0x7fc9e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:46.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.392+0000 7fc9ee7fc640 1 -- 172.21.15.84:0/3969640212 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e803ad20 con 0x7fc9f80ad970 2023-10-16T11:27:46.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.392+0000 7fc9ee7fc640 1 -- 172.21.15.84:0/3969640212 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e803ae80 con 0x7fc9f80ad970 2023-10-16T11:27:46.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.392+0000 7fc9fe80d640 1 -- 172.21.15.84:0/3969640212 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f80aeb40 con 0x7fc9f80ad970 2023-10-16T11:27:46.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.393+0000 7fc9ee7fc640 1 -- 172.21.15.84:0/3969640212 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8044630 con 0x7fc9f80ad970 2023-10-16T11:27:46.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.393+0000 7fc9fe80d640 1 -- 172.21.15.84:0/3969640212 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f80af060 con 0x7fc9f80ad970 2023-10-16T11:27:46.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.394+0000 7fc9fe80d640 1 -- 172.21.15.84:0/3969640212 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f806b330 con 0x7fc9f80ad970 2023-10-16T11:27:46.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.395+0000 7fc9ee7fc640 1 -- 172.21.15.84:0/3969640212 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 168) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fc9e802f050 con 0x7fc9f80ad970 2023-10-16T11:27:46.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.396+0000 7fc9ee7fc640 1 --2- 172.21.15.84:0/3969640212 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fc9dc06af00 0x7fc9dc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:46.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.396+0000 7fc9ee7fc640 1 -- 172.21.15.84:0/3969640212 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(272..272 src has 1..272) v4 ==== 6425+0+0 (secure 0 0 0) 0x7fc9e80abbe0 con 0x7fc9f80ad970 2023-10-16T11:27:46.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.396+0000 7fc9fd80b640 1 --2- 172.21.15.84:0/3969640212 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fc9dc06af00 0x7fc9dc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:46.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.397+0000 7fc9fd80b640 1 --2- 172.21.15.84:0/3969640212 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fc9dc06af00 0x7fc9dc06d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4003d90 tx=0x7fc9f4009040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:46.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.400+0000 7fc9ee7fc640 1 -- 172.21.15.84:0/3969640212 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc9e80767c0 con 0x7fc9f80ad970 2023-10-16T11:27:46.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.598+0000 7fc9fe80d640 1 -- 172.21.15.84:0/3969640212 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9f80ae3e0 con 0x7fc9f80ad970 2023-10-16T11:27:46.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.601+0000 7fc9ee7fc640 1 -- 172.21.15.84:0/3969640212 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v272) v1 ==== 74+0+17543 (secure 0 0 0) 0x7fc9f80ae3e0 con 0x7fc9f80ad970 2023-10-16T11:27:46.603 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:46.603 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":272,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:27:43.469311+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:27:20.716240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"269","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_data","create_time":"2023-10-16T11:27:21.905553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"270","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2893482429":"2023-10-17T11:27:43.469263+0000","172.21.15.84:0/1321046441":"2023-10-17T11:27:43.469263+0000","172.21.15.84:0/822007176":"2023-10-17T11:27:43.469263+0000","172.21.15.84:0/3876279166":"2023-10-17T11:27:43.469263+0000","172.21.15.84:6833/129259578":"2023-10-17T11:27:43.469263+0000","172.21.15.84:0/2238142758":"2023-10-17T11:27:43.469263+0000","172.21.15.84:0/1788498161":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/3634760901":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/3509607392":"2023-10-17T11:27:43.469263+0000","172.21.15.84:0/3842621119":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/40770371":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/1147031855":"2023-10-17T11:27:37.140228+0000","172.21.15.84:6832/129259578":"2023-10-17T11:27:43.469263+0000","172.21.15.84:6833/731888425":"2023-10-17T11:27:37.140228+0000","172.21.15.84:6832/731888425":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/2353059609":"2023-10-17T11:27:37.140228+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:27:46.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.603+0000 7fc9fe80d640 1 -- 172.21.15.84:0/3969640212 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fc9dc06af00 msgr2=0x7fc9dc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:46.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.603+0000 7fc9fe80d640 1 --2- 172.21.15.84:0/3969640212 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fc9dc06af00 0x7fc9dc06d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4003d90 tx=0x7fc9f4009040 comp rx=0 tx=0).stop 2023-10-16T11:27:46.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.603+0000 7fc9fe80d640 1 -- 172.21.15.84:0/3969640212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f80ad970 msgr2=0x7fc9f80b0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:46.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.603+0000 7fc9fe80d640 1 --2- 172.21.15.84:0/3969640212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f80ad970 0x7fc9f80b0710 secure :-1 s=READY pgs=3507 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80097c0 tx=0x7fc9e80046c0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.603+0000 7fc9fe80d640 1 -- 172.21.15.84:0/3969640212 shutdown_connections 2023-10-16T11:27:46.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.604+0000 7fc9fe80d640 1 --2- 172.21.15.84:0/3969640212 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fc9dc06af00 0x7fc9dc06d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.604+0000 7fc9fe80d640 1 --2- 172.21.15.84:0/3969640212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9f80ccfb0 0x7fc9f80c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.604+0000 7fc9fe80d640 1 --2- 172.21.15.84:0/3969640212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9f80ad970 0x7fc9f80b0710 unknown :-1 s=CLOSED pgs=3507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.604+0000 7fc9fe80d640 1 --2- 172.21.15.84:0/3969640212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9f806c040 0x7fc9f814ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.604+0000 7fc9fe80d640 1 -- 172.21.15.84:0/3969640212 >> 172.21.15.84:0/3969640212 conn(0x7fc9f80ce6a0 msgr2=0x7fc9f815ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:46.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.604+0000 7fc9fe80d640 1 -- 172.21.15.84:0/3969640212 shutdown_connections 2023-10-16T11:27:46.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.604+0000 7fc9fe80d640 1 -- 172.21.15.84:0/3969640212 wait complete. 2023-10-16T11:27:46.623 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T11:27:46.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.821+0000 7fbb491c9640 1 Processor -- start 2023-10-16T11:27:46.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.822+0000 7fbb491c9640 1 -- start start 2023-10-16T11:27:46.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.823+0000 7fbb491c9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbb440addb0 0x7fbb440caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:46.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.823+0000 7fbb491c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb440c9000 0x7fbb44069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:46.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.823+0000 7fbb491c9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb440cb140 con 0x7fbb4406a2e0 2023-10-16T11:27:46.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.823+0000 7fbb491c9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb44161a50 con 0x7fbb440addb0 2023-10-16T11:27:46.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.823+0000 7fbb491c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4406a750 con 0x7fbb440c9000 2023-10-16T11:27:46.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.824+0000 7fbb4359e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbb4406a2e0 0x7fbb440cc580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54136/0 (socket says 172.21.15.84:54136) 2023-10-16T11:27:46.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.824+0000 7fbb4359e640 1 -- 172.21.15.84:0/1159358753 learned_addr learned my addr 172.21.15.84:0/1159358753 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:46.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.824+0000 7fbb4259c640 1 --2- 172.21.15.84:0/1159358753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb440c9000 0x7fbb44069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:46.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.824+0000 7fbb42d9d640 1 --2- 172.21.15.84:0/1159358753 >> v2:172.21.15.157:3300/0 conn(0x7fbb440addb0 0x7fbb440caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:46.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.824+0000 7fbb41d9b640 1 -- 172.21.15.84:0/1159358753 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959340157 0 0) 0x7fbb440cb140 con 0x7fbb4406a2e0 2023-10-16T11:27:46.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.824+0000 7fbb41d9b640 1 -- 172.21.15.84:0/1159358753 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb30003520 con 0x7fbb4406a2e0 2023-10-16T11:27:46.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.824+0000 7fbb41d9b640 1 -- 172.21.15.84:0/1159358753 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1000719766 0 0) 0x7fbb30003520 con 0x7fbb4406a2e0 2023-10-16T11:27:46.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb41d9b640 1 -- 172.21.15.84:0/1159358753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb440c9000 msgr2=0x7fbb44069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:46.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb41d9b640 1 --2- 172.21.15.84:0/1159358753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb440c9000 0x7fbb44069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb41d9b640 1 -- 172.21.15.84:0/1159358753 >> v2:172.21.15.157:3300/0 conn(0x7fbb440addb0 msgr2=0x7fbb440caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:46.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb41d9b640 1 --2- 172.21.15.84:0/1159358753 >> v2:172.21.15.157:3300/0 conn(0x7fbb440addb0 0x7fbb440caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb41d9b640 1 -- 172.21.15.84:0/1159358753 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4406b030 con 0x7fbb4406a2e0 2023-10-16T11:27:46.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb41d9b640 1 -- 172.21.15.84:0/1159358753 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fbb3c001670 con 0x7fbb4406a2e0 2023-10-16T11:27:46.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb42d9d640 1 --2- 172.21.15.84:0/1159358753 >> v2:172.21.15.157:3300/0 conn(0x7fbb440addb0 0x7fbb440caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:27:46.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb41d9b640 1 -- 172.21.15.84:0/1159358753 >> v1:172.21.15.84:6789/0 conn(0x7fbb4406a2e0 legacy=0x7fbb440cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:46.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb41d9b640 1 --2- 172.21.15.84:0/1159358753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb440c9000 0x7fbb300039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:46.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb41d9b640 1 --2- 172.21.15.84:0/1159358753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb30003f20 0x7fbb300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:46.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb41d9b640 1 --2- 172.21.15.84:0/1159358753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb30004d00 0x7fbb30005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:46.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb41d9b640 1 -- 172.21.15.84:0/1159358753 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb44161a50 con 0x7fbb30004d00 2023-10-16T11:27:46.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb41d9b640 1 -- 172.21.15.84:0/1159358753 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4406a750 con 0x7fbb30003f20 2023-10-16T11:27:46.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb41d9b640 1 -- 172.21.15.84:0/1159358753 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3000ba00 con 0x7fbb440c9000 2023-10-16T11:27:46.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb4359e640 1 --2- 172.21.15.84:0/1159358753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb30004d00 0x7fbb30005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:46.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb4359e640 1 -- 172.21.15.84:0/1159358753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb440c9000 msgr2=0x7fbb300039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:46.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb4359e640 1 --2- 172.21.15.84:0/1159358753 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb440c9000 0x7fbb300039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb4359e640 1 -- 172.21.15.84:0/1159358753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb30003f20 msgr2=0x7fbb300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:46.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb4359e640 1 --2- 172.21.15.84:0/1159358753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb30003f20 0x7fbb300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.825+0000 7fbb4359e640 1 -- 172.21.15.84:0/1159358753 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb30003520 con 0x7fbb30004d00 2023-10-16T11:27:46.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.826+0000 7fbb4359e640 1 --2- 172.21.15.84:0/1159358753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb30004d00 0x7fbb30005180 secure :-1 s=READY pgs=3509 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c240 tx=0x7fbb300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185c993d6d2f1eeb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:46.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.827+0000 7fbb41d9b640 1 -- 172.21.15.84:0/1159358753 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c005ad0 con 0x7fbb30004d00 2023-10-16T11:27:46.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.827+0000 7fbb41d9b640 1 -- 172.21.15.84:0/1159358753 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3c005c50 con 0x7fbb30004d00 2023-10-16T11:27:46.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.827+0000 7fbb41d9b640 1 -- 172.21.15.84:0/1159358753 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c02c8b0 con 0x7fbb30004d00 2023-10-16T11:27:46.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.827+0000 7fbb491c9640 1 -- 172.21.15.84:0/1159358753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb30004d00 msgr2=0x7fbb30005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:46.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.827+0000 7fbb491c9640 1 --2- 172.21.15.84:0/1159358753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb30004d00 0x7fbb30005180 secure :-1 s=READY pgs=3509 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c240 tx=0x7fbb300057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.827+0000 7fbb4359e640 1 -- 172.21.15.84:0/1159358753 reap_dead start 2023-10-16T11:27:46.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.827+0000 7fbb491c9640 1 -- 172.21.15.84:0/1159358753 shutdown_connections 2023-10-16T11:27:46.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.827+0000 7fbb491c9640 1 -- 172.21.15.84:0/1159358753 >> 172.21.15.84:0/1159358753 conn(0x7fbb440ce9b0 msgr2=0x7fbb440cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:46.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.827+0000 7fbb491c9640 1 -- 172.21.15.84:0/1159358753 shutdown_connections 2023-10-16T11:27:46.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.827+0000 7fbb491c9640 1 -- 172.21.15.84:0/1159358753 wait complete. 2023-10-16T11:27:46.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.829+0000 7fbb491c9640 1 Processor -- start 2023-10-16T11:27:46.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.829+0000 7fbb491c9640 1 -- start start 2023-10-16T11:27:46.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.830+0000 7fbb491c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb30004d00 0x7fbb440c5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:46.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.830+0000 7fbb491c9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb441fa540 0x7fbb440b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:46.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.830+0000 7fbb491c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb440b3ce0 0x7fbb440c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:46.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.830+0000 7fbb491c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb440b4160 con 0x7fbb440b3ce0 2023-10-16T11:27:46.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.830+0000 7fbb491c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb440c56a0 con 0x7fbb30004d00 2023-10-16T11:27:46.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.830+0000 7fbb491c9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb44162ec0 con 0x7fbb441fa540 2023-10-16T11:27:46.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.830+0000 7fbb4359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb440b3ce0 0x7fbb440c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:46.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.830+0000 7fbb4359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb440b3ce0 0x7fbb440c6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36850/0 (socket says 172.21.15.84:36850) 2023-10-16T11:27:46.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.830+0000 7fbb4359e640 1 -- 172.21.15.84:0/2791189824 learned_addr learned my addr 172.21.15.84:0/2791189824 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:46.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.831+0000 7fbb42d9d640 1 --2- 172.21.15.84:0/2791189824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb30004d00 0x7fbb440c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:46.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.831+0000 7fbb4259c640 1 --2- 172.21.15.84:0/2791189824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb441fa540 0x7fbb440b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:46.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.831+0000 7fbb4359e640 1 -- 172.21.15.84:0/2791189824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb441fa540 msgr2=0x7fbb440b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:46.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.831+0000 7fbb4359e640 1 --2- 172.21.15.84:0/2791189824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb441fa540 0x7fbb440b37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.831+0000 7fbb4359e640 1 -- 172.21.15.84:0/2791189824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb30004d00 msgr2=0x7fbb440c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:46.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.831+0000 7fbb4359e640 1 --2- 172.21.15.84:0/2791189824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb30004d00 0x7fbb440c5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:46.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.831+0000 7fbb4359e640 1 -- 172.21.15.84:0/2791189824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb44162900 con 0x7fbb440b3ce0 2023-10-16T11:27:46.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.831+0000 7fbb42d9d640 1 --2- 172.21.15.84:0/2791189824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb30004d00 0x7fbb440c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:46.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.831+0000 7fbb4359e640 1 --2- 172.21.15.84:0/2791189824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb440b3ce0 0x7fbb440c6bb0 secure :-1 s=READY pgs=3510 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c360 tx=0x7fbb3c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:46.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.831+0000 7fbb3b7fe640 1 -- 172.21.15.84:0/2791189824 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c03c040 con 0x7fbb440b3ce0 2023-10-16T11:27:46.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.831+0000 7fbb3b7fe640 1 -- 172.21.15.84:0/2791189824 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3c02ccc0 con 0x7fbb440b3ce0 2023-10-16T11:27:46.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.831+0000 7fbb491c9640 1 -- 172.21.15.84:0/2791189824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb44162be0 con 0x7fbb440b3ce0 2023-10-16T11:27:46.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.831+0000 7fbb3b7fe640 1 -- 172.21.15.84:0/2791189824 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c0072b0 con 0x7fbb440b3ce0 2023-10-16T11:27:46.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.831+0000 7fbb491c9640 1 -- 172.21.15.84:0/2791189824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb441fa020 con 0x7fbb440b3ce0 2023-10-16T11:27:46.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.832+0000 7fbb491c9640 1 -- 172.21.15.84:0/2791189824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb4404e160 con 0x7fbb440b3ce0 2023-10-16T11:27:46.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.833+0000 7fbb3b7fe640 1 -- 172.21.15.84:0/2791189824 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 168) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fbb3c032070 con 0x7fbb440b3ce0 2023-10-16T11:27:46.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.834+0000 7fbb3b7fe640 1 --2- 172.21.15.84:0/2791189824 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fbb2406af00 0x7fbb2406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:46.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.834+0000 7fbb3b7fe640 1 -- 172.21.15.84:0/2791189824 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(272..272 src has 1..272) v4 ==== 6425+0+0 (secure 0 0 0) 0x7fbb3c0aaa40 con 0x7fbb440b3ce0 2023-10-16T11:27:46.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.835+0000 7fbb42d9d640 1 --2- 172.21.15.84:0/2791189824 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fbb2406af00 0x7fbb2406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:46.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.835+0000 7fbb42d9d640 1 --2- 172.21.15.84:0/2791189824 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fbb2406af00 0x7fbb2406d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00d940 tx=0x7fbb2c008040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:46.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:46.839+0000 7fbb3b7fe640 1 -- 172.21.15.84:0/2791189824 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbb3c075850 con 0x7fbb440b3ce0 2023-10-16T11:27:47.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.050+0000 7fbb491c9640 1 -- 172.21.15.84:0/2791189824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbb4406c5b0 con 0x7fbb440b3ce0 2023-10-16T11:27:47.052 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-16T11:27:47.051+0000 7fca0c7ff640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:27:47.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.054+0000 7fbb3b7fe640 1 -- 172.21.15.84:0/2791189824 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v158) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbb3c046090 con 0x7fbb440b3ce0 2023-10-16T11:27:47.056 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:27:47.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.057+0000 7fbb491c9640 1 -- 172.21.15.84:0/2791189824 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fbb2406af00 msgr2=0x7fbb2406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:47.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.057+0000 7fbb491c9640 1 --2- 172.21.15.84:0/2791189824 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fbb2406af00 0x7fbb2406d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00d940 tx=0x7fbb2c008040 comp rx=0 tx=0).stop 2023-10-16T11:27:47.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.057+0000 7fbb491c9640 1 -- 172.21.15.84:0/2791189824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb440b3ce0 msgr2=0x7fbb440c6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:47.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.057+0000 7fbb491c9640 1 --2- 172.21.15.84:0/2791189824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb440b3ce0 0x7fbb440c6bb0 secure :-1 s=READY pgs=3510 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c360 tx=0x7fbb3c02e070 comp rx=0 tx=0).stop 2023-10-16T11:27:47.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.058+0000 7fbb491c9640 1 -- 172.21.15.84:0/2791189824 shutdown_connections 2023-10-16T11:27:47.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.058+0000 7fbb491c9640 1 --2- 172.21.15.84:0/2791189824 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fbb2406af00 0x7fbb2406d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:47.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.058+0000 7fbb491c9640 1 --2- 172.21.15.84:0/2791189824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb440b3ce0 0x7fbb440c6bb0 unknown :-1 s=CLOSED pgs=3510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:47.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.058+0000 7fbb491c9640 1 --2- 172.21.15.84:0/2791189824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb441fa540 0x7fbb440b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:47.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.058+0000 7fbb491c9640 1 --2- 172.21.15.84:0/2791189824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb30004d00 0x7fbb440c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:47.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.058+0000 7fbb491c9640 1 -- 172.21.15.84:0/2791189824 >> 172.21.15.84:0/2791189824 conn(0x7fbb440ce9b0 msgr2=0x7fbb440af770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:47.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.058+0000 7fbb491c9640 1 -- 172.21.15.84:0/2791189824 shutdown_connections 2023-10-16T11:27:47.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.060+0000 7fbb491c9640 1 -- 172.21.15.84:0/2791189824 wait complete. 2023-10-16T11:27:47.069 INFO:tasks.ceph.mds.a.smithi084.stderr: -2603> 2023-10-16T11:27:29.373+0000 7fbe1a45b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:27:47.079 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T11:27:47.128 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-16T11:27:47.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.266+0000 7fee53fff640 1 Processor -- start 2023-10-16T11:27:47.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.266+0000 7fee53fff640 1 -- start start 2023-10-16T11:27:47.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.267+0000 7fee53fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fee54159950 0x7fee5415a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:47.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.267+0000 7fee53fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5415aef0 con 0x7fee54070450 2023-10-16T11:27:47.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.267+0000 7fee53fff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5415d910 con 0x7fee5406c810 2023-10-16T11:27:47.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.267+0000 7fee53fff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee5415e490 con 0x7fee54159950 2023-10-16T11:27:47.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.268+0000 7fee4a7fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fee54070450 0x7fee5406c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54144/0 (socket says 172.21.15.84:54144) 2023-10-16T11:27:47.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.268+0000 7fee4a7fc640 1 -- 172.21.15.84:0/89972074 learned_addr learned my addr 172.21.15.84:0/89972074 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:47.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.268+0000 7fee527fc640 1 -- 172.21.15.84:0/89972074 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828981575 0 0) 0x7fee3c001110 con 0x7fee54070450 2023-10-16T11:27:47.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.268+0000 7fee527fc640 1 -- 172.21.15.84:0/89972074 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee40003520 con 0x7fee54070450 2023-10-16T11:27:47.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.268+0000 7fee52ffd640 1 --2- 172.21.15.84:0/89972074 >> v2:172.21.15.157:3300/0 conn(0x7fee54159950 0x7fee5415a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:47.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.268+0000 7fee527fc640 1 -- 172.21.15.84:0/89972074 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 828427866 0 0) 0x7fee40003520 con 0x7fee54070450 2023-10-16T11:27:47.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.268+0000 7fee527fc640 1 -- 172.21.15.84:0/89972074 >> v2:172.21.15.157:3300/0 conn(0x7fee54159950 msgr2=0x7fee5415a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:47.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.268+0000 7fee527fc640 1 --2- 172.21.15.84:0/89972074 >> v2:172.21.15.157:3300/0 conn(0x7fee54159950 0x7fee5415a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:47.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.268+0000 7fee527fc640 1 -- 172.21.15.84:0/89972074 >> v1:172.21.15.157:6789/0 conn(0x7fee5406c810 legacy=0x7fee540c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:47.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.268+0000 7fee527fc640 1 -- 172.21.15.84:0/89972074 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5415dbf0 con 0x7fee54070450 2023-10-16T11:27:47.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.268+0000 7fee527fc640 1 -- 172.21.15.84:0/89972074 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fee3c001800 con 0x7fee54070450 2023-10-16T11:27:47.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.268+0000 7fee527fc640 1 -- 172.21.15.84:0/89972074 >> v1:172.21.15.84:6789/0 conn(0x7fee54070450 legacy=0x7fee5406c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:47.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.269+0000 7fee527fc640 1 --2- 172.21.15.84:0/89972074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee400039c0 0x7fee40003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:47.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.269+0000 7fee527fc640 1 --2- 172.21.15.84:0/89972074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400047c0 0x7fee40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:47.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.269+0000 7fee527fc640 1 --2- 172.21.15.84:0/89972074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee40005250 0x7fee4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:47.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.269+0000 7fee527fc640 1 -- 172.21.15.84:0/89972074 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5415e490 con 0x7fee400047c0 2023-10-16T11:27:47.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.269+0000 7fee527fc640 1 -- 172.21.15.84:0/89972074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4000c1a0 con 0x7fee400039c0 2023-10-16T11:27:47.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.269+0000 7fee527fc640 1 -- 172.21.15.84:0/89972074 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4000c360 con 0x7fee40005250 2023-10-16T11:27:47.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.269+0000 7fee4a7fc640 1 --2- 172.21.15.84:0/89972074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400047c0 0x7fee40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:47.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.269+0000 7fee52ffd640 1 --2- 172.21.15.84:0/89972074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee400039c0 0x7fee40003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:47.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.269+0000 7fee537fe640 1 --2- 172.21.15.84:0/89972074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee40005250 0x7fee4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:47.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.269+0000 7fee4a7fc640 1 -- 172.21.15.84:0/89972074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee40005250 msgr2=0x7fee4000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:47.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.269+0000 7fee4a7fc640 1 --2- 172.21.15.84:0/89972074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee40005250 0x7fee4000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:47.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.269+0000 7fee4a7fc640 1 -- 172.21.15.84:0/89972074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee400039c0 msgr2=0x7fee40003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:47.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.269+0000 7fee4a7fc640 1 --2- 172.21.15.84:0/89972074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee400039c0 0x7fee40003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:47.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.269+0000 7fee4a7fc640 1 -- 172.21.15.84:0/89972074 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee40003520 con 0x7fee400047c0 2023-10-16T11:27:47.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.269+0000 7fee52ffd640 1 --2- 172.21.15.84:0/89972074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee400039c0 0x7fee40003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:47.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.269+0000 7fee537fe640 1 -- 172.21.15.84:0/89972074 reap_dead start 2023-10-16T11:27:47.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.270+0000 7fee4a7fc640 1 --2- 172.21.15.84:0/89972074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400047c0 0x7fee40004c40 secure :-1 s=READY pgs=3515 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00c110 tx=0x7fee400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca4e69a49af3b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:47.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.271+0000 7fee527fc640 1 -- 172.21.15.84:0/89972074 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c005ad0 con 0x7fee400047c0 2023-10-16T11:27:47.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.271+0000 7fee527fc640 1 -- 172.21.15.84:0/89972074 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3c005c30 con 0x7fee400047c0 2023-10-16T11:27:47.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.271+0000 7fee527fc640 1 -- 172.21.15.84:0/89972074 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c02c7e0 con 0x7fee400047c0 2023-10-16T11:27:47.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.271+0000 7fee53fff640 1 -- 172.21.15.84:0/89972074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400047c0 msgr2=0x7fee40004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:47.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.271+0000 7fee53fff640 1 --2- 172.21.15.84:0/89972074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400047c0 0x7fee40004c40 secure :-1 s=READY pgs=3515 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00c110 tx=0x7fee400057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:47.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.271+0000 7fee53fff640 1 -- 172.21.15.84:0/89972074 shutdown_connections 2023-10-16T11:27:47.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.271+0000 7fee53fff640 1 --2- 172.21.15.84:0/89972074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400047c0 0x7fee40004c40 unknown :-1 s=CLOSED pgs=3515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:47.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.271+0000 7fee53fff640 1 -- 172.21.15.84:0/89972074 >> 172.21.15.84:0/89972074 conn(0x7fee540ce6a0 msgr2=0x7fee540ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:47.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.271+0000 7fee53fff640 1 -- 172.21.15.84:0/89972074 shutdown_connections 2023-10-16T11:27:47.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.271+0000 7fee53fff640 1 -- 172.21.15.84:0/89972074 wait complete. 2023-10-16T11:27:47.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.273+0000 7fee53fff640 1 Processor -- start 2023-10-16T11:27:47.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.273+0000 7fee53fff640 1 -- start start 2023-10-16T11:27:47.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.274+0000 7fee53fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400047c0 0x7fee540b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:47.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.274+0000 7fee53fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee540c7970 0x7fee540ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:47.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.274+0000 7fee53fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee54152830 0x7fee540ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:47.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.274+0000 7fee53fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee540c8190 con 0x7fee400047c0 2023-10-16T11:27:47.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.274+0000 7fee53fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee540c8930 con 0x7fee54152830 2023-10-16T11:27:47.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.274+0000 7fee53fff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee540c8370 con 0x7fee540c7970 2023-10-16T11:27:47.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.274+0000 7fee52ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400047c0 0x7fee540b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:47.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.274+0000 7fee52ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400047c0 0x7fee540b36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36880/0 (socket says 172.21.15.84:36880) 2023-10-16T11:27:47.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.274+0000 7fee52ffd640 1 -- 172.21.15.84:0/2449111018 learned_addr learned my addr 172.21.15.84:0/2449111018 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:47.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.275+0000 7fee52ffd640 1 -- 172.21.15.84:0/2449111018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee540c7970 msgr2=0x7fee540ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:47.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.275+0000 7fee52ffd640 1 --2- 172.21.15.84:0/2449111018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee540c7970 0x7fee540ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:47.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.275+0000 7fee52ffd640 1 -- 172.21.15.84:0/2449111018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee54152830 msgr2=0x7fee540ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:47.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.275+0000 7fee52ffd640 1 --2- 172.21.15.84:0/2449111018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee54152830 0x7fee540ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:47.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.275+0000 7fee52ffd640 1 -- 172.21.15.84:0/2449111018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee540c8650 con 0x7fee400047c0 2023-10-16T11:27:47.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.275+0000 7fee52ffd640 1 --2- 172.21.15.84:0/2449111018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400047c0 0x7fee540b36c0 secure :-1 s=READY pgs=3516 cs=0 l=1 rev1=1 crypto rx=0x7fee4400e810 tx=0x7fee4400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:47.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.275+0000 7fee50ff9640 1 -- 172.21.15.84:0/2449111018 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4400c870 con 0x7fee400047c0 2023-10-16T11:27:47.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.275+0000 7fee50ff9640 1 -- 172.21.15.84:0/2449111018 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4400c9d0 con 0x7fee400047c0 2023-10-16T11:27:47.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.275+0000 7fee50ff9640 1 -- 172.21.15.84:0/2449111018 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4400cd40 con 0x7fee400047c0 2023-10-16T11:27:47.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.275+0000 7fee53fff640 1 -- 172.21.15.84:0/2449111018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee540b2ec0 con 0x7fee400047c0 2023-10-16T11:27:47.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.275+0000 7fee53fff640 1 -- 172.21.15.84:0/2449111018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee540adbf0 con 0x7fee400047c0 2023-10-16T11:27:47.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.276+0000 7fee53fff640 1 -- 172.21.15.84:0/2449111018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee540b2d20 con 0x7fee400047c0 2023-10-16T11:27:47.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.277+0000 7fee50ff9640 1 -- 172.21.15.84:0/2449111018 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 168) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fee44013410 con 0x7fee400047c0 2023-10-16T11:27:47.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.278+0000 7fee50ff9640 1 --2- 172.21.15.84:0/2449111018 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fee3406af00 0x7fee3406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:47.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.278+0000 7fee50ff9640 1 -- 172.21.15.84:0/2449111018 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(273..273 src has 1..273) v4 ==== 6511+0+0 (secure 0 0 0) 0x7fee44089930 con 0x7fee400047c0 2023-10-16T11:27:47.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.278+0000 7fee4a7fc640 1 --2- 172.21.15.84:0/2449111018 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fee3406af00 0x7fee3406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:47.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.279+0000 7fee4a7fc640 1 --2- 172.21.15.84:0/2449111018 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fee3406af00 0x7fee3406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fee3c034000 tx=0x7fee3c039040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:47.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.282+0000 7fee50ff9640 1 -- 172.21.15.84:0/2449111018 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fee540b2d20 con 0x7fee400047c0 2023-10-16T11:27:47.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:47.517+0000 7fee53fff640 1 -- 172.21.15.84:0/2449111018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fee540b2d20 con 0x7fee400047c0 2023-10-16T11:27:48.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.057+0000 7fee50ff9640 1 -- 172.21.15.84:0/2449111018 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v159) v1 ==== 104+0+0 (secure 0 0 0) 0x7fee540b2d20 con 0x7fee400047c0 2023-10-16T11:27:48.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.061+0000 7fee53fff640 1 -- 172.21.15.84:0/2449111018 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fee3406af00 msgr2=0x7fee3406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:48.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.061+0000 7fee53fff640 1 --2- 172.21.15.84:0/2449111018 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fee3406af00 0x7fee3406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fee3c034000 tx=0x7fee3c039040 comp rx=0 tx=0).stop 2023-10-16T11:27:48.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.061+0000 7fee53fff640 1 -- 172.21.15.84:0/2449111018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400047c0 msgr2=0x7fee540b36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:48.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.061+0000 7fee53fff640 1 --2- 172.21.15.84:0/2449111018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400047c0 0x7fee540b36c0 secure :-1 s=READY pgs=3516 cs=0 l=1 rev1=1 crypto rx=0x7fee4400e810 tx=0x7fee4400ecd0 comp rx=0 tx=0).stop 2023-10-16T11:27:48.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.061+0000 7fee53fff640 1 -- 172.21.15.84:0/2449111018 shutdown_connections 2023-10-16T11:27:48.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.061+0000 7fee53fff640 1 --2- 172.21.15.84:0/2449111018 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fee3406af00 0x7fee3406d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:48.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.061+0000 7fee53fff640 1 --2- 172.21.15.84:0/2449111018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee54152830 0x7fee540ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:48.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.061+0000 7fee53fff640 1 --2- 172.21.15.84:0/2449111018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee540c7970 0x7fee540ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:48.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.061+0000 7fee53fff640 1 --2- 172.21.15.84:0/2449111018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400047c0 0x7fee540b36c0 unknown :-1 s=CLOSED pgs=3516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:48.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.061+0000 7fee53fff640 1 -- 172.21.15.84:0/2449111018 >> 172.21.15.84:0/2449111018 conn(0x7fee540ce6a0 msgr2=0x7fee5406ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:48.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.062+0000 7fee53fff640 1 -- 172.21.15.84:0/2449111018 shutdown_connections 2023-10-16T11:27:48.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.062+0000 7fee53fff640 1 -- 172.21.15.84:0/2449111018 wait complete. 2023-10-16T11:27:48.085 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T11:27:48.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.306+0000 7f1c4b59e640 1 Processor -- start 2023-10-16T11:27:48.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.306+0000 7f1c4b59e640 1 -- start start 2023-10-16T11:27:48.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.307+0000 7f1c4b59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c4c159950 0x7f1c4c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:48.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.307+0000 7f1c4b59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1c4c070450 0x7f1c4c0708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:48.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.307+0000 7f1c4b59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1c4c06c100 0x7f1c4c06c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:48.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.307+0000 7f1c4b59e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4c15e490 con 0x7f1c4c06c100 2023-10-16T11:27:48.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.307+0000 7f1c4b59e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4c070df0 con 0x7f1c4c070450 2023-10-16T11:27:48.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.308+0000 7f1c4b59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c15aef0 con 0x7f1c4c159950 2023-10-16T11:27:48.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.308+0000 7f1c4ad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1c4c06c100 0x7f1c4c06c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:48.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.308+0000 7f1c4ad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1c4c06c100 0x7f1c4c06c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36886/0 (socket says 172.21.15.84:36886) 2023-10-16T11:27:48.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.308+0000 7f1c4ad9d640 1 -- 172.21.15.84:0/884015931 learned_addr learned my addr 172.21.15.84:0/884015931 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:48.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.308+0000 7f1c4a59c640 1 --2- 172.21.15.84:0/884015931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c4c159950 0x7f1c4c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:48.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.308+0000 7f1c49d9b640 1 --2- 172.21.15.84:0/884015931 >> v2:172.21.15.157:3300/0 conn(0x7f1c4c070450 0x7f1c4c0708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:48.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.308+0000 7f1c4ad9d640 1 -- 172.21.15.84:0/884015931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c4c159950 msgr2=0x7f1c4c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:48.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.308+0000 7f1c4ad9d640 1 --2- 172.21.15.84:0/884015931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c4c159950 0x7f1c4c15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:48.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.308+0000 7f1c4ad9d640 1 -- 172.21.15.84:0/884015931 >> v2:172.21.15.157:3300/0 conn(0x7f1c4c070450 msgr2=0x7f1c4c0708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:48.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.308+0000 7f1c4ad9d640 1 --2- 172.21.15.84:0/884015931 >> v2:172.21.15.157:3300/0 conn(0x7f1c4c070450 0x7f1c4c0708b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:48.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.308+0000 7f1c4ad9d640 1 -- 172.21.15.84:0/884015931 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c15d910 con 0x7f1c4c06c100 2023-10-16T11:27:48.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.309+0000 7f1c4ad9d640 1 --2- 172.21.15.84:0/884015931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c4c06c100 0x7f1c4c06c570 secure :-1 s=READY pgs=3517 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0719e0 tx=0x7f1c4402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d7527d95766495 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:48.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.310+0000 7f1c4959a640 1 -- 172.21.15.84:0/884015931 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4402c9e0 con 0x7f1c4c06c100 2023-10-16T11:27:48.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.310+0000 7f1c4959a640 1 -- 172.21.15.84:0/884015931 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4402cb40 con 0x7f1c4c06c100 2023-10-16T11:27:48.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.310+0000 7f1c4959a640 1 -- 172.21.15.84:0/884015931 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c440343f0 con 0x7f1c4c06c100 2023-10-16T11:27:48.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.310+0000 7f1c4b59e640 1 -- 172.21.15.84:0/884015931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c4c06c100 msgr2=0x7f1c4c06c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:48.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.310+0000 7f1c4b59e640 1 --2- 172.21.15.84:0/884015931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c4c06c100 0x7f1c4c06c570 secure :-1 s=READY pgs=3517 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0719e0 tx=0x7f1c4402be40 comp rx=0 tx=0).stop 2023-10-16T11:27:48.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.311+0000 7f1c4b59e640 1 -- 172.21.15.84:0/884015931 shutdown_connections 2023-10-16T11:27:48.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.311+0000 7f1c4b59e640 1 --2- 172.21.15.84:0/884015931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c4c06c100 0x7f1c4c06c570 unknown :-1 s=CLOSED pgs=3517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:48.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.311+0000 7f1c4b59e640 1 --2- 172.21.15.84:0/884015931 >> v2:172.21.15.157:3300/0 conn(0x7f1c4c070450 0x7f1c4c0708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:48.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.311+0000 7f1c4b59e640 1 --2- 172.21.15.84:0/884015931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c4c159950 0x7f1c4c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:48.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.311+0000 7f1c4b59e640 1 -- 172.21.15.84:0/884015931 >> 172.21.15.84:0/884015931 conn(0x7f1c4c0ce6a0 msgr2=0x7f1c4c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:48.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.311+0000 7f1c4b59e640 1 -- 172.21.15.84:0/884015931 shutdown_connections 2023-10-16T11:27:48.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.311+0000 7f1c4b59e640 1 -- 172.21.15.84:0/884015931 wait complete. 2023-10-16T11:27:48.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.312+0000 7f1c4b59e640 1 Processor -- start 2023-10-16T11:27:48.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.312+0000 7f1c4b59e640 1 -- start start 2023-10-16T11:27:48.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.313+0000 7f1c4b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c4c06c100 0x7f1c4c0b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:48.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.313+0000 7f1c4b59e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c4c070450 0x7f1c4c0c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:48.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.313+0000 7f1c4b59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c4c159950 0x7f1c4c15c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:48.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.313+0000 7f1c4b59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0c2340 con 0x7f1c4c06c100 2023-10-16T11:27:48.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.313+0000 7f1c4b59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0b3980 con 0x7f1c4c159950 2023-10-16T11:27:48.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.313+0000 7f1c4b59e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0b3c20 con 0x7f1c4c070450 2023-10-16T11:27:48.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.313+0000 7f1c4a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c4c06c100 0x7f1c4c0b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:48.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.314+0000 7f1c4a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c4c06c100 0x7f1c4c0b12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36892/0 (socket says 172.21.15.84:36892) 2023-10-16T11:27:48.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.314+0000 7f1c4a59c640 1 -- 172.21.15.84:0/1841653251 learned_addr learned my addr 172.21.15.84:0/1841653251 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:48.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.314+0000 7f1c4a59c640 1 -- 172.21.15.84:0/1841653251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c4c070450 msgr2=0x7f1c4c0c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:48.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.314+0000 7f1c4a59c640 1 --2- 172.21.15.84:0/1841653251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c4c070450 0x7f1c4c0c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:48.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.314+0000 7f1c4a59c640 1 -- 172.21.15.84:0/1841653251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c4c159950 msgr2=0x7f1c4c15c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:48.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.314+0000 7f1c4a59c640 1 --2- 172.21.15.84:0/1841653251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c4c159950 0x7f1c4c15c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:48.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.314+0000 7f1c4a59c640 1 -- 172.21.15.84:0/1841653251 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c44009b30 con 0x7f1c4c06c100 2023-10-16T11:27:48.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.314+0000 7f1c4a59c640 1 --2- 172.21.15.84:0/1841653251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c4c06c100 0x7f1c4c0b12c0 secure :-1 s=READY pgs=3518 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00d800 tx=0x7f1c3c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:48.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.314+0000 7f1c3affd640 1 -- 172.21.15.84:0/1841653251 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c00b6c0 con 0x7f1c4c06c100 2023-10-16T11:27:48.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.315+0000 7f1c3affd640 1 -- 172.21.15.84:0/1841653251 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3c00b820 con 0x7f1c4c06c100 2023-10-16T11:27:48.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.315+0000 7f1c3affd640 1 -- 172.21.15.84:0/1841653251 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c00bbb0 con 0x7f1c4c06c100 2023-10-16T11:27:48.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.315+0000 7f1c4b59e640 1 -- 172.21.15.84:0/1841653251 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4c0b3f00 con 0x7f1c4c06c100 2023-10-16T11:27:48.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.315+0000 7f1c4b59e640 1 -- 172.21.15.84:0/1841653251 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4c0b1800 con 0x7f1c4c06c100 2023-10-16T11:27:48.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.316+0000 7f1c4b59e640 1 -- 172.21.15.84:0/1841653251 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4c04e9b0 con 0x7f1c4c06c100 2023-10-16T11:27:48.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.317+0000 7f1c3affd640 1 -- 172.21.15.84:0/1841653251 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1c3c00bd10 con 0x7f1c4c06c100 2023-10-16T11:27:48.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.318+0000 7f1c3affd640 1 --2- 172.21.15.84:0/1841653251 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f1c2c06af90 0x7f1c2c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:48.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.318+0000 7f1c3affd640 1 -- 172.21.15.84:0/1841653251 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(273..273 src has 1..273) v4 ==== 6511+0+0 (secure 0 0 0) 0x7f1c3c089ed0 con 0x7f1c4c06c100 2023-10-16T11:27:48.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.318+0000 7f1c49d9b640 1 --2- 172.21.15.84:0/1841653251 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f1c2c06af90 0x7f1c2c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:48.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.319+0000 7f1c49d9b640 1 --2- 172.21.15.84:0/1841653251 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f1c2c06af90 0x7f1c2c06d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0b0910 tx=0x7f1c40008040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:48.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.322+0000 7f1c3affd640 1 -- 172.21.15.84:0/1841653251 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1c3c054be0 con 0x7f1c4c06c100 2023-10-16T11:27:48.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:48.515+0000 7f1c4b59e640 1 -- 172.21.15.84:0/1841653251 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1c4c0b4620 con 0x7f1c4c06c100 2023-10-16T11:27:49.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.241+0000 7f1c3affd640 1 -- 172.21.15.84:0/1841653251 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v274) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1c3c058200 con 0x7f1c4c06c100 2023-10-16T11:27:49.244 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-16T11:27:49.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.247+0000 7f1c4b59e640 1 -- 172.21.15.84:0/1841653251 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f1c2c06af90 msgr2=0x7f1c2c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:49.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.247+0000 7f1c4b59e640 1 --2- 172.21.15.84:0/1841653251 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f1c2c06af90 0x7f1c2c06d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0b0910 tx=0x7f1c40008040 comp rx=0 tx=0).stop 2023-10-16T11:27:49.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.247+0000 7f1c4b59e640 1 -- 172.21.15.84:0/1841653251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c4c06c100 msgr2=0x7f1c4c0b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:49.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.247+0000 7f1c4b59e640 1 --2- 172.21.15.84:0/1841653251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c4c06c100 0x7f1c4c0b12c0 secure :-1 s=READY pgs=3518 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00d800 tx=0x7f1c3c00dcc0 comp rx=0 tx=0).stop 2023-10-16T11:27:49.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.247+0000 7f1c4b59e640 1 -- 172.21.15.84:0/1841653251 shutdown_connections 2023-10-16T11:27:49.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.247+0000 7f1c4b59e640 1 --2- 172.21.15.84:0/1841653251 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f1c2c06af90 0x7f1c2c06d450 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0b0910 tx=0x7f1c40008040 comp rx=0 tx=0).stop 2023-10-16T11:27:49.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.247+0000 7f1c4b59e640 1 --2- 172.21.15.84:0/1841653251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c4c159950 0x7f1c4c15c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:49.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.247+0000 7f1c4b59e640 1 --2- 172.21.15.84:0/1841653251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c4c070450 0x7f1c4c0c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:49.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.247+0000 7f1c4b59e640 1 --2- 172.21.15.84:0/1841653251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c4c06c100 0x7f1c4c0b12c0 unknown :-1 s=CLOSED pgs=3518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:49.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.247+0000 7f1c4b59e640 1 -- 172.21.15.84:0/1841653251 >> 172.21.15.84:0/1841653251 conn(0x7f1c4c0ce6a0 msgr2=0x7f1c4c150d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:49.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.247+0000 7f1c4b59e640 1 -- 172.21.15.84:0/1841653251 shutdown_connections 2023-10-16T11:27:49.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.247+0000 7f1c4b59e640 1 -- 172.21.15.84:0/1841653251 wait complete. 2023-10-16T11:27:49.275 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T11:27:49.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.482+0000 7fbb0ffff640 1 Processor -- start 2023-10-16T11:27:49.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.482+0000 7fbb0ffff640 1 -- start start 2023-10-16T11:27:49.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.484+0000 7fbb0ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb1015c9f0 0x7fbb100c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:49.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.484+0000 7fbb0ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbb1015dde0 0x7fbb100ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:49.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.484+0000 7fbb0ffff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb10160a90 con 0x7fbb100c9230 2023-10-16T11:27:49.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.484+0000 7fbb0ffff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb100cdec0 con 0x7fbb1015dde0 2023-10-16T11:27:49.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.484+0000 7fbb0ffff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100cd900 con 0x7fbb1015c9f0 2023-10-16T11:27:49.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.485+0000 7fbb0effd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb1015c9f0 0x7fbb100c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:49.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.485+0000 7fbb0effd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb1015c9f0 0x7fbb100c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.84:55272/0 (socket says 172.21.15.84:55272) 2023-10-16T11:27:49.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.485+0000 7fbb0effd640 1 -- 172.21.15.84:0/1323419701 learned_addr learned my addr 172.21.15.84:0/1323419701 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:49.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.485+0000 7fbb0f7fe640 1 --2- 172.21.15.84:0/1323419701 >> v2:172.21.15.157:3300/0 conn(0x7fbb1015dde0 0x7fbb100ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:49.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.486+0000 7fbb0e7fc640 1 -- 172.21.15.84:0/1323419701 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547697603 0 0) 0x7fbb10160a90 con 0x7fbb100c9230 2023-10-16T11:27:49.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.486+0000 7fbb0effd640 1 -- 172.21.15.84:0/1323419701 >> v2:172.21.15.157:3300/0 conn(0x7fbb1015dde0 msgr2=0x7fbb100ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:49.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.486+0000 7fbb0effd640 1 --2- 172.21.15.84:0/1323419701 >> v2:172.21.15.157:3300/0 conn(0x7fbb1015dde0 0x7fbb100ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:49.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.486+0000 7fbb0effd640 1 -- 172.21.15.84:0/1323419701 >> v1:172.21.15.157:6789/0 conn(0x7fbb100c9230 legacy=0x7fbb100709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:49.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.486+0000 7fbb0effd640 1 -- 172.21.15.84:0/1323419701 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb100cdbe0 con 0x7fbb1015c9f0 2023-10-16T11:27:49.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.487+0000 7fbb0effd640 1 --2- 172.21.15.84:0/1323419701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb1015c9f0 0x7fbb100c8cf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009870 tx=0x7fbb0002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1af7f33cad262c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:49.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.488+0000 7fbb0e7fc640 1 -- 172.21.15.84:0/1323419701 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb00030840 con 0x7fbb1015c9f0 2023-10-16T11:27:49.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.488+0000 7fbb0e7fc640 1 -- 172.21.15.84:0/1323419701 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb000309a0 con 0x7fbb1015c9f0 2023-10-16T11:27:49.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.488+0000 7fbb0e7fc640 1 -- 172.21.15.84:0/1323419701 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb00030d30 con 0x7fbb1015c9f0 2023-10-16T11:27:49.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.488+0000 7fbb0ffff640 1 -- 172.21.15.84:0/1323419701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb1015c9f0 msgr2=0x7fbb100c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:49.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.488+0000 7fbb0ffff640 1 --2- 172.21.15.84:0/1323419701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb1015c9f0 0x7fbb100c8cf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009870 tx=0x7fbb0002b980 comp rx=0 tx=0).stop 2023-10-16T11:27:49.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.488+0000 7fbb0ffff640 1 -- 172.21.15.84:0/1323419701 shutdown_connections 2023-10-16T11:27:49.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.488+0000 7fbb0ffff640 1 --2- 172.21.15.84:0/1323419701 >> v2:172.21.15.157:3300/0 conn(0x7fbb1015dde0 0x7fbb100ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:49.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.488+0000 7fbb0ffff640 1 --2- 172.21.15.84:0/1323419701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb1015c9f0 0x7fbb100c8cf0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:49.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.488+0000 7fbb0ffff640 1 -- 172.21.15.84:0/1323419701 >> 172.21.15.84:0/1323419701 conn(0x7fbb100ce6a0 msgr2=0x7fbb100ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:49.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.489+0000 7fbb0ffff640 1 -- 172.21.15.84:0/1323419701 shutdown_connections 2023-10-16T11:27:49.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.489+0000 7fbb0ffff640 1 -- 172.21.15.84:0/1323419701 wait complete. 2023-10-16T11:27:49.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.490+0000 7fbb0ffff640 1 Processor -- start 2023-10-16T11:27:49.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.490+0000 7fbb0ffff640 1 -- start start 2023-10-16T11:27:49.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.491+0000 7fbb0ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb100c9230 0x7fbb100c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.491+0000 7fbb0ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb1015c9f0 0x7fbb1014e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.491+0000 7fbb0ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb1015dde0 0x7fbb100c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.491+0000 7fbb0ffff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1006a950 con 0x7fbb1015c9f0 2023-10-16T11:27:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.491+0000 7fbb0ffff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1006ac30 con 0x7fbb100c9230 2023-10-16T11:27:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.491+0000 7fbb0ffff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1006af10 con 0x7fbb1015dde0 2023-10-16T11:27:49.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.492+0000 7fbb067fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb1015c9f0 0x7fbb1014e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:49.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.492+0000 7fbb067fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb1015c9f0 0x7fbb1014e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36904/0 (socket says 172.21.15.84:36904) 2023-10-16T11:27:49.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.492+0000 7fbb067fc640 1 -- 172.21.15.84:0/658297311 learned_addr learned my addr 172.21.15.84:0/658297311 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:49.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.492+0000 7fbb0f7fe640 1 --2- 172.21.15.84:0/658297311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb1015dde0 0x7fbb100c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:49.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.492+0000 7fbb0effd640 1 --2- 172.21.15.84:0/658297311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb100c9230 0x7fbb100c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:49.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.492+0000 7fbb067fc640 1 -- 172.21.15.84:0/658297311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb1015dde0 msgr2=0x7fbb100c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:49.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.492+0000 7fbb067fc640 1 --2- 172.21.15.84:0/658297311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb1015dde0 0x7fbb100c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:49.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.492+0000 7fbb067fc640 1 -- 172.21.15.84:0/658297311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb100c9230 msgr2=0x7fbb100c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:49.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.492+0000 7fbb067fc640 1 --2- 172.21.15.84:0/658297311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb100c9230 0x7fbb100c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:49.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.492+0000 7fbb067fc640 1 -- 172.21.15.84:0/658297311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb000094d0 con 0x7fbb1015c9f0 2023-10-16T11:27:49.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.492+0000 7fbb0effd640 1 --2- 172.21.15.84:0/658297311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb100c9230 0x7fbb100c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:49.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.493+0000 7fbb067fc640 1 --2- 172.21.15.84:0/658297311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb1015c9f0 0x7fbb1014e9f0 secure :-1 s=READY pgs=3519 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800c4b0 tx=0x7fbaf800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:49.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.493+0000 7fbb07fff640 1 -- 172.21.15.84:0/658297311 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8005480 con 0x7fbb1015c9f0 2023-10-16T11:27:49.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.493+0000 7fbb07fff640 1 -- 172.21.15.84:0/658297311 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf80035f0 con 0x7fbb1015c9f0 2023-10-16T11:27:49.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.493+0000 7fbb07fff640 1 -- 172.21.15.84:0/658297311 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf80039e0 con 0x7fbb1015c9f0 2023-10-16T11:27:49.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.493+0000 7fbb0ffff640 1 -- 172.21.15.84:0/658297311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1006b1f0 con 0x7fbb1015c9f0 2023-10-16T11:27:49.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.493+0000 7fbb0ffff640 1 -- 172.21.15.84:0/658297311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb100c72a0 con 0x7fbb1015c9f0 2023-10-16T11:27:49.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.494+0000 7fbb0ffff640 1 -- 172.21.15.84:0/658297311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1004e9b0 con 0x7fbb1015c9f0 2023-10-16T11:27:49.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.495+0000 7fbb07fff640 1 -- 172.21.15.84:0/658297311 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbaf8003b40 con 0x7fbb1015c9f0 2023-10-16T11:27:49.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.496+0000 7fbb07fff640 1 --2- 172.21.15.84:0/658297311 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fbaf006af90 0x7fbaf006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:49.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.496+0000 7fbb07fff640 1 -- 172.21.15.84:0/658297311 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbaf8088f90 con 0x7fbb1015c9f0 2023-10-16T11:27:49.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.496+0000 7fbb0effd640 1 --2- 172.21.15.84:0/658297311 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fbaf006af90 0x7fbaf006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:49.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.497+0000 7fbb0effd640 1 --2- 172.21.15.84:0/658297311 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fbaf006af90 0x7fbaf006d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbb00036000 tx=0x7fbb0003b040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:49.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.500+0000 7fbb07fff640 1 -- 172.21.15.84:0/658297311 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbaf8053d20 con 0x7fbb1015c9f0 2023-10-16T11:27:49.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:49.692+0000 7fbb0ffff640 1 -- 172.21.15.84:0/658297311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbb1006aa90 con 0x7fbb1015c9f0 2023-10-16T11:27:50.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.266+0000 7fbb07fff640 1 -- 172.21.15.84:0/658297311 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v275) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbb1006aa90 con 0x7fbb1015c9f0 2023-10-16T11:27:50.269 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-16T11:27:50.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.272+0000 7fbb0ffff640 1 -- 172.21.15.84:0/658297311 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fbaf006af90 msgr2=0x7fbaf006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:50.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.272+0000 7fbb0ffff640 1 --2- 172.21.15.84:0/658297311 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fbaf006af90 0x7fbaf006d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbb00036000 tx=0x7fbb0003b040 comp rx=0 tx=0).stop 2023-10-16T11:27:50.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.272+0000 7fbb0ffff640 1 -- 172.21.15.84:0/658297311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb1015c9f0 msgr2=0x7fbb1014e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:50.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.272+0000 7fbb0ffff640 1 --2- 172.21.15.84:0/658297311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb1015c9f0 0x7fbb1014e9f0 secure :-1 s=READY pgs=3519 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800c4b0 tx=0x7fbaf800c970 comp rx=0 tx=0).stop 2023-10-16T11:27:50.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.272+0000 7fbb0ffff640 1 -- 172.21.15.84:0/658297311 shutdown_connections 2023-10-16T11:27:50.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.272+0000 7fbb0ffff640 1 --2- 172.21.15.84:0/658297311 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fbaf006af90 0x7fbaf006d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.272+0000 7fbb0ffff640 1 --2- 172.21.15.84:0/658297311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb1015dde0 0x7fbb100c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.272+0000 7fbb0ffff640 1 --2- 172.21.15.84:0/658297311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb1015c9f0 0x7fbb1014e9f0 unknown :-1 s=CLOSED pgs=3519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.272+0000 7fbb0ffff640 1 --2- 172.21.15.84:0/658297311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb100c9230 0x7fbb100c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.272+0000 7fbb0ffff640 1 -- 172.21.15.84:0/658297311 >> 172.21.15.84:0/658297311 conn(0x7fbb100ce6a0 msgr2=0x7fbb100cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:50.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.273+0000 7fbb0ffff640 1 -- 172.21.15.84:0/658297311 shutdown_connections 2023-10-16T11:27:50.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.276+0000 7fbb0ffff640 1 -- 172.21.15.84:0/658297311 wait complete. 2023-10-16T11:27:50.304 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T11:27:50.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.496+0000 7f72295b0640 1 Processor -- start 2023-10-16T11:27:50.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.496+0000 7f72295b0640 1 -- start start 2023-10-16T11:27:50.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.497+0000 7f72295b0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7224159e90 0x7f72240ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:50.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.498+0000 7f72295b0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7224070360 con 0x7f7224159ab0 2023-10-16T11:27:50.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.498+0000 7f72295b0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72240cd1f0 con 0x7f722406c140 2023-10-16T11:27:50.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.498+0000 7f72295b0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f722415d890 con 0x7f7224159e90 2023-10-16T11:27:50.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.498+0000 7f7222ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7224159ab0 0x7f722406fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54160/0 (socket says 172.21.15.84:54160) 2023-10-16T11:27:50.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.498+0000 7f7222ffd640 1 -- 172.21.15.84:0/2614237372 learned_addr learned my addr 172.21.15.84:0/2614237372 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:50.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.498+0000 7f7221ffb640 1 -- 172.21.15.84:0/2614237372 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 333265760 0 0) 0x7f7224070360 con 0x7f7224159ab0 2023-10-16T11:27:50.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.498+0000 7f7221ffb640 1 -- 172.21.15.84:0/2614237372 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f720c003520 con 0x7f7224159ab0 2023-10-16T11:27:50.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.499+0000 7f7221ffb640 1 -- 172.21.15.84:0/2614237372 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3483017865 0 0) 0x7f720c003520 con 0x7f7224159ab0 2023-10-16T11:27:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.499+0000 7f7221ffb640 1 -- 172.21.15.84:0/2614237372 >> v2:172.21.15.157:3300/0 conn(0x7f7224159e90 msgr2=0x7f72240ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.499+0000 7f7221ffb640 1 --2- 172.21.15.84:0/2614237372 >> v2:172.21.15.157:3300/0 conn(0x7f7224159e90 0x7f72240ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.499+0000 7f7221ffb640 1 -- 172.21.15.84:0/2614237372 >> v1:172.21.15.157:6789/0 conn(0x7f722406c140 legacy=0x7f722415dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.499+0000 7f7221ffb640 1 -- 172.21.15.84:0/2614237372 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72240cd4d0 con 0x7f7224159ab0 2023-10-16T11:27:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.499+0000 7f7221ffb640 1 -- 172.21.15.84:0/2614237372 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f7214001670 con 0x7f7224159ab0 2023-10-16T11:27:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.499+0000 7f7221ffb640 1 -- 172.21.15.84:0/2614237372 >> v1:172.21.15.84:6789/0 conn(0x7f7224159ab0 legacy=0x7f722406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.499+0000 7f7221ffb640 1 --2- 172.21.15.84:0/2614237372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f720c003970 0x7f720c003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.499+0000 7f7221ffb640 1 --2- 172.21.15.84:0/2614237372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720c0047c0 0x7f720c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.499+0000 7f7221ffb640 1 --2- 172.21.15.84:0/2614237372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f720c005380 0x7f720c00b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.499+0000 7f7221ffb640 1 -- 172.21.15.84:0/2614237372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722415d890 con 0x7f720c0047c0 2023-10-16T11:27:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.499+0000 7f7221ffb640 1 -- 172.21.15.84:0/2614237372 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c00c2c0 con 0x7f720c005380 2023-10-16T11:27:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.499+0000 7f7221ffb640 1 -- 172.21.15.84:0/2614237372 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720c00e690 con 0x7f720c003970 2023-10-16T11:27:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.499+0000 7f72227fc640 1 --2- 172.21.15.84:0/2614237372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720c0047c0 0x7f720c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.499+0000 7f7222ffd640 1 --2- 172.21.15.84:0/2614237372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f720c003970 0x7f720c003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.500+0000 7f72227fc640 1 -- 172.21.15.84:0/2614237372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f720c003970 msgr2=0x7f720c003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.500+0000 7f72227fc640 1 --2- 172.21.15.84:0/2614237372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f720c003970 0x7f720c003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.500+0000 7f72227fc640 1 -- 172.21.15.84:0/2614237372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f720c005380 msgr2=0x7f720c00b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.500+0000 7f72237fe640 1 --2- 172.21.15.84:0/2614237372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f720c005380 0x7f720c00b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.500+0000 7f72227fc640 1 --2- 172.21.15.84:0/2614237372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f720c005380 0x7f720c00b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.500+0000 7f72227fc640 1 -- 172.21.15.84:0/2614237372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c003520 con 0x7f720c0047c0 2023-10-16T11:27:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.500+0000 7f72237fe640 1 --2- 172.21.15.84:0/2614237372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f720c005380 0x7f720c00b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.500+0000 7f72237fe640 1 -- 172.21.15.84:0/2614237372 reap_dead start 2023-10-16T11:27:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.500+0000 7f7222ffd640 1 --2- 172.21.15.84:0/2614237372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f720c003970 0x7f720c003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.501+0000 7f72227fc640 1 --2- 172.21.15.84:0/2614237372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720c0047c0 0x7f720c004c40 secure :-1 s=READY pgs=3521 cs=0 l=1 rev1=1 crypto rx=0x7f7218009860 tx=0x7f720c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6faf1dfe64f9a1df server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.501+0000 7f7221ffb640 1 -- 172.21.15.84:0/2614237372 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218032600 con 0x7f720c0047c0 2023-10-16T11:27:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.501+0000 7f7221ffb640 1 -- 172.21.15.84:0/2614237372 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721802e050 con 0x7f720c0047c0 2023-10-16T11:27:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.501+0000 7f7221ffb640 1 -- 172.21.15.84:0/2614237372 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72180329c0 con 0x7f720c0047c0 2023-10-16T11:27:50.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.502+0000 7f72295b0640 1 -- 172.21.15.84:0/2614237372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720c0047c0 msgr2=0x7f720c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:50.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.502+0000 7f72295b0640 1 --2- 172.21.15.84:0/2614237372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720c0047c0 0x7f720c004c40 secure :-1 s=READY pgs=3521 cs=0 l=1 rev1=1 crypto rx=0x7f7218009860 tx=0x7f720c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.502+0000 7f72295b0640 1 -- 172.21.15.84:0/2614237372 shutdown_connections 2023-10-16T11:27:50.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.502+0000 7f72295b0640 1 --2- 172.21.15.84:0/2614237372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f720c0047c0 0x7f720c004c40 unknown :-1 s=CLOSED pgs=3521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.502+0000 7f72295b0640 1 -- 172.21.15.84:0/2614237372 >> 172.21.15.84:0/2614237372 conn(0x7f72240ce9b0 msgr2=0x7f72240cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:50.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.502+0000 7f72295b0640 1 -- 172.21.15.84:0/2614237372 shutdown_connections 2023-10-16T11:27:50.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.502+0000 7f72295b0640 1 -- 172.21.15.84:0/2614237372 wait complete. 2023-10-16T11:27:50.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.504+0000 7f72295b0640 1 Processor -- start 2023-10-16T11:27:50.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.504+0000 7f72295b0640 1 -- start start 2023-10-16T11:27:50.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.505+0000 7f72295b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f720c0047c0 0x7f7224158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:50.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.505+0000 7f72295b0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72240ad2f0 0x7f72240c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:50.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.505+0000 7f72295b0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72240b3aa0 0x7f72240b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:50.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.505+0000 7f72295b0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72240b15d0 con 0x7f72240b3aa0 2023-10-16T11:27:50.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.505+0000 7f72295b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72240b2110 con 0x7f720c0047c0 2023-10-16T11:27:50.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.505+0000 7f72295b0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72240b17e0 con 0x7f72240ad2f0 2023-10-16T11:27:50.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.505+0000 7f72237fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72240b3aa0 0x7f72240b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:50.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.505+0000 7f72237fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72240b3aa0 0x7f72240b3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36926/0 (socket says 172.21.15.84:36926) 2023-10-16T11:27:50.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.505+0000 7f72237fe640 1 -- 172.21.15.84:0/3497350606 learned_addr learned my addr 172.21.15.84:0/3497350606 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:50.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.506+0000 7f72227fc640 1 --2- 172.21.15.84:0/3497350606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72240ad2f0 0x7f72240c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:50.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.506+0000 7f7222ffd640 1 --2- 172.21.15.84:0/3497350606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f720c0047c0 0x7f7224158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:50.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.506+0000 7f72237fe640 1 -- 172.21.15.84:0/3497350606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72240ad2f0 msgr2=0x7f72240c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:50.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.506+0000 7f72237fe640 1 --2- 172.21.15.84:0/3497350606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72240ad2f0 0x7f72240c4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.506+0000 7f72237fe640 1 -- 172.21.15.84:0/3497350606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f720c0047c0 msgr2=0x7f7224158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:50.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.506+0000 7f72237fe640 1 --2- 172.21.15.84:0/3497350606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f720c0047c0 0x7f7224158740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.506+0000 7f72237fe640 1 -- 172.21.15.84:0/3497350606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72240b1ac0 con 0x7f72240b3aa0 2023-10-16T11:27:50.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.506+0000 7f72237fe640 1 --2- 172.21.15.84:0/3497350606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72240b3aa0 0x7f72240b3f10 secure :-1 s=READY pgs=3522 cs=0 l=1 rev1=1 crypto rx=0x7f72240b15a0 tx=0x7f721c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:50.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.506+0000 7f720b7fe640 1 -- 172.21.15.84:0/3497350606 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c006d50 con 0x7f72240b3aa0 2023-10-16T11:27:50.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.506+0000 7f720b7fe640 1 -- 172.21.15.84:0/3497350606 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c0048d0 con 0x7f72240b3aa0 2023-10-16T11:27:50.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.506+0000 7f720b7fe640 1 -- 172.21.15.84:0/3497350606 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c004c80 con 0x7f72240b3aa0 2023-10-16T11:27:50.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.506+0000 7f72295b0640 1 -- 172.21.15.84:0/3497350606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72240b1da0 con 0x7f72240b3aa0 2023-10-16T11:27:50.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.507+0000 7f72295b0640 1 -- 172.21.15.84:0/3497350606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72240c5320 con 0x7f72240b3aa0 2023-10-16T11:27:50.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.508+0000 7f72295b0640 1 -- 172.21.15.84:0/3497350606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72240b22a0 con 0x7f72240b3aa0 2023-10-16T11:27:50.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.508+0000 7f720b7fe640 1 -- 172.21.15.84:0/3497350606 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f721c005070 con 0x7f72240b3aa0 2023-10-16T11:27:50.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.509+0000 7f720b7fe640 1 --2- 172.21.15.84:0/3497350606 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f721006af90 0x7f721006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:50.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.509+0000 7f720b7fe640 1 -- 172.21.15.84:0/3497350606 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f721c089410 con 0x7f72240b3aa0 2023-10-16T11:27:50.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.510+0000 7f7222ffd640 1 --2- 172.21.15.84:0/3497350606 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f721006af90 0x7f721006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:50.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.510+0000 7f7222ffd640 1 --2- 172.21.15.84:0/3497350606 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f721006af90 0x7f721006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7214001eb0 tx=0x7f7214009040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:50.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.514+0000 7f720b7fe640 1 -- 172.21.15.84:0/3497350606 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f721c054360 con 0x7f72240b3aa0 2023-10-16T11:27:50.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.712+0000 7f72295b0640 1 -- 172.21.15.84:0/3497350606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f72240cda60 con 0x7f72240b3aa0 2023-10-16T11:27:50.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.719+0000 7f720b7fe640 1 -- 172.21.15.84:0/3497350606 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v97) v1 ==== 104+0+0 (secure 0 0 0) 0x7f721c057980 con 0x7f72240b3aa0 2023-10-16T11:27:50.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.721+0000 7f72295b0640 1 -- 172.21.15.84:0/3497350606 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f721006af90 msgr2=0x7f721006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:50.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.721+0000 7f72295b0640 1 --2- 172.21.15.84:0/3497350606 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f721006af90 0x7f721006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7214001eb0 tx=0x7f7214009040 comp rx=0 tx=0).stop 2023-10-16T11:27:50.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.721+0000 7f72295b0640 1 -- 172.21.15.84:0/3497350606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72240b3aa0 msgr2=0x7f72240b3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:50.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.721+0000 7f72295b0640 1 --2- 172.21.15.84:0/3497350606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72240b3aa0 0x7f72240b3f10 secure :-1 s=READY pgs=3522 cs=0 l=1 rev1=1 crypto rx=0x7f72240b15a0 tx=0x7f721c00dc00 comp rx=0 tx=0).stop 2023-10-16T11:27:50.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.721+0000 7f72295b0640 1 -- 172.21.15.84:0/3497350606 shutdown_connections 2023-10-16T11:27:50.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.721+0000 7f72295b0640 1 --2- 172.21.15.84:0/3497350606 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f721006af90 0x7f721006d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.721+0000 7f72295b0640 1 --2- 172.21.15.84:0/3497350606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72240b3aa0 0x7f72240b3f10 unknown :-1 s=CLOSED pgs=3522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.721+0000 7f72295b0640 1 --2- 172.21.15.84:0/3497350606 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72240ad2f0 0x7f72240c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.721+0000 7f72295b0640 1 --2- 172.21.15.84:0/3497350606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f720c0047c0 0x7f7224158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.721+0000 7f72295b0640 1 -- 172.21.15.84:0/3497350606 >> 172.21.15.84:0/3497350606 conn(0x7f72240ce9b0 msgr2=0x7f722406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:50.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.722+0000 7f72295b0640 1 -- 172.21.15.84:0/3497350606 shutdown_connections 2023-10-16T11:27:50.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.722+0000 7f72295b0640 1 -- 172.21.15.84:0/3497350606 wait complete. 2023-10-16T11:27:50.743 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T11:27:50.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.958+0000 7f2346fa9640 1 Processor -- start 2023-10-16T11:27:50.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.958+0000 7f2346fa9640 1 -- start start 2023-10-16T11:27:50.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.960+0000 7f2346fa9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f23400addb0 0x7f23400caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:50.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.960+0000 7f2346fa9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23400cb140 con 0x7f234006a4b0 2023-10-16T11:27:50.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.960+0000 7f2346fa9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f234006b030 con 0x7f23400c9000 2023-10-16T11:27:50.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.960+0000 7f2346fa9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2340161a50 con 0x7f23400addb0 2023-10-16T11:27:50.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.960+0000 7f2345546640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f234006a4b0 0x7f23401650d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54166/0 (socket says 172.21.15.84:54166) 2023-10-16T11:27:50.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.960+0000 7f2345546640 1 -- 172.21.15.84:0/4050876845 learned_addr learned my addr 172.21.15.84:0/4050876845 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:50.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.961+0000 7f23377fe640 1 -- 172.21.15.84:0/4050876845 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3175233511 0 0) 0x7f23400cb140 con 0x7f234006a4b0 2023-10-16T11:27:50.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.961+0000 7f23377fe640 1 -- 172.21.15.84:0/4050876845 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2328003520 con 0x7f234006a4b0 2023-10-16T11:27:50.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.961+0000 7f2344d45640 1 --2- 172.21.15.84:0/4050876845 >> v2:172.21.15.157:3300/0 conn(0x7f23400addb0 0x7f23400caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:50.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.961+0000 7f23377fe640 1 -- 172.21.15.84:0/4050876845 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2222009069 0 0) 0x7f2328003520 con 0x7f234006a4b0 2023-10-16T11:27:50.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.961+0000 7f23377fe640 1 -- 172.21.15.84:0/4050876845 >> v2:172.21.15.157:3300/0 conn(0x7f23400addb0 msgr2=0x7f23400caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:50.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.961+0000 7f23377fe640 1 --2- 172.21.15.84:0/4050876845 >> v2:172.21.15.157:3300/0 conn(0x7f23400addb0 0x7f23400caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.961+0000 7f23377fe640 1 -- 172.21.15.84:0/4050876845 >> v1:172.21.15.157:6789/0 conn(0x7f23400c9000 legacy=0x7f2340069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:50.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.961+0000 7f23377fe640 1 -- 172.21.15.84:0/4050876845 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234006b310 con 0x7f234006a4b0 2023-10-16T11:27:50.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.961+0000 7f23377fe640 1 -- 172.21.15.84:0/4050876845 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f23380019f0 con 0x7f234006a4b0 2023-10-16T11:27:50.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.961+0000 7f23377fe640 1 -- 172.21.15.84:0/4050876845 >> v1:172.21.15.84:6789/0 conn(0x7f234006a4b0 legacy=0x7f23401650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:50.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.961+0000 7f23377fe640 1 --2- 172.21.15.84:0/4050876845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2328003970 0x7f2328003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:50.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.961+0000 7f23377fe640 1 --2- 172.21.15.84:0/4050876845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23280047c0 0x7f2328004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:50.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.961+0000 7f23377fe640 1 --2- 172.21.15.84:0/4050876845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2328005180 0x7f232800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:50.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.961+0000 7f23377fe640 1 -- 172.21.15.84:0/4050876845 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2340161a50 con 0x7f23280047c0 2023-10-16T11:27:50.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.961+0000 7f23377fe640 1 -- 172.21.15.84:0/4050876845 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232800c0d0 con 0x7f2328005180 2023-10-16T11:27:50.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.961+0000 7f23377fe640 1 -- 172.21.15.84:0/4050876845 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232800c290 con 0x7f2328003970 2023-10-16T11:27:50.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.962+0000 7f2337fff640 1 --2- 172.21.15.84:0/4050876845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23280047c0 0x7f2328004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:50.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.962+0000 7f2344d45640 1 --2- 172.21.15.84:0/4050876845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2328003970 0x7f2328003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:50.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.962+0000 7f2345546640 1 --2- 172.21.15.84:0/4050876845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2328005180 0x7f232800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:50.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.962+0000 7f2337fff640 1 -- 172.21.15.84:0/4050876845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2328003970 msgr2=0x7f2328003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:50.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.962+0000 7f2337fff640 1 --2- 172.21.15.84:0/4050876845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2328003970 0x7f2328003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.962+0000 7f2337fff640 1 -- 172.21.15.84:0/4050876845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2328005180 msgr2=0x7f232800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:50.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.962+0000 7f2337fff640 1 --2- 172.21.15.84:0/4050876845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2328005180 0x7f232800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.962+0000 7f2337fff640 1 -- 172.21.15.84:0/4050876845 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2328003520 con 0x7f23280047c0 2023-10-16T11:27:50.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.962+0000 7f2345546640 1 -- 172.21.15.84:0/4050876845 reap_dead start 2023-10-16T11:27:50.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.963+0000 7f2337fff640 1 --2- 172.21.15.84:0/4050876845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23280047c0 0x7f2328004c40 secure :-1 s=READY pgs=3524 cs=0 l=1 rev1=1 crypto rx=0x7f2330001750 tx=0x7f23280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4795f61a6e874e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:50.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.964+0000 7f23377fe640 1 -- 172.21.15.84:0/4050876845 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23300052d0 con 0x7f23280047c0 2023-10-16T11:27:50.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.964+0000 7f23377fe640 1 -- 172.21.15.84:0/4050876845 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233002e050 con 0x7f23280047c0 2023-10-16T11:27:50.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.964+0000 7f23377fe640 1 -- 172.21.15.84:0/4050876845 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2330003830 con 0x7f23280047c0 2023-10-16T11:27:50.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.964+0000 7f2346fa9640 1 -- 172.21.15.84:0/4050876845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23280047c0 msgr2=0x7f2328004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:50.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.964+0000 7f2346fa9640 1 --2- 172.21.15.84:0/4050876845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23280047c0 0x7f2328004c40 secure :-1 s=READY pgs=3524 cs=0 l=1 rev1=1 crypto rx=0x7f2330001750 tx=0x7f23280057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.964+0000 7f2346fa9640 1 -- 172.21.15.84:0/4050876845 shutdown_connections 2023-10-16T11:27:50.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.964+0000 7f2346fa9640 1 --2- 172.21.15.84:0/4050876845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23280047c0 0x7f2328004c40 unknown :-1 s=CLOSED pgs=3524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.964+0000 7f2346fa9640 1 -- 172.21.15.84:0/4050876845 >> 172.21.15.84:0/4050876845 conn(0x7f23400ce9b0 msgr2=0x7f23400cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:50.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.964+0000 7f2346fa9640 1 -- 172.21.15.84:0/4050876845 shutdown_connections 2023-10-16T11:27:50.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.964+0000 7f2346fa9640 1 -- 172.21.15.84:0/4050876845 wait complete. 2023-10-16T11:27:50.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.966+0000 7f2346fa9640 1 Processor -- start 2023-10-16T11:27:50.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.966+0000 7f2346fa9640 1 -- start start 2023-10-16T11:27:50.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.967+0000 7f2346fa9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23280047c0 0x7f234015fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:50.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.967+0000 7f2346fa9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23400c5a80 0x7f23400c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:50.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.967+0000 7f2346fa9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23400b0420 0x7f23400b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:50.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.967+0000 7f2346fa9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23400b0de0 con 0x7f23400b0420 2023-10-16T11:27:50.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.967+0000 7f2346fa9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23400c65f0 con 0x7f23280047c0 2023-10-16T11:27:50.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.967+0000 7f2346fa9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23400c68d0 con 0x7f23400c5a80 2023-10-16T11:27:50.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.968+0000 7f2345546640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23400b0420 0x7f23400b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:50.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.968+0000 7f2345546640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23400b0420 0x7f23400b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36934/0 (socket says 172.21.15.84:36934) 2023-10-16T11:27:50.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.968+0000 7f2345546640 1 -- 172.21.15.84:0/3326322290 learned_addr learned my addr 172.21.15.84:0/3326322290 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:50.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.968+0000 7f2337fff640 1 --2- 172.21.15.84:0/3326322290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23400c5a80 0x7f23400c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:50.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.968+0000 7f2344d45640 1 --2- 172.21.15.84:0/3326322290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23280047c0 0x7f234015fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:50.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.968+0000 7f2345546640 1 -- 172.21.15.84:0/3326322290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23400c5a80 msgr2=0x7f23400c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:50.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.968+0000 7f2345546640 1 --2- 172.21.15.84:0/3326322290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23400c5a80 0x7f23400c5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.968+0000 7f2345546640 1 -- 172.21.15.84:0/3326322290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23280047c0 msgr2=0x7f234015fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:50.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.968+0000 7f2345546640 1 --2- 172.21.15.84:0/3326322290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23280047c0 0x7f234015fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:50.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.968+0000 7f2345546640 1 -- 172.21.15.84:0/3326322290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23400c6bb0 con 0x7f23400b0420 2023-10-16T11:27:50.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.968+0000 7f2344d45640 1 --2- 172.21.15.84:0/3326322290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23280047c0 0x7f234015fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:50.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.968+0000 7f2345546640 1 --2- 172.21.15.84:0/3326322290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23400b0420 0x7f23400b08a0 secure :-1 s=READY pgs=3525 cs=0 l=1 rev1=1 crypto rx=0x7f2338000f80 tx=0x7f233800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:50.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.969+0000 7f23357fa640 1 -- 172.21.15.84:0/3326322290 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338004d60 con 0x7f23400b0420 2023-10-16T11:27:50.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.969+0000 7f23357fa640 1 -- 172.21.15.84:0/3326322290 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233800bb80 con 0x7f23400b0420 2023-10-16T11:27:50.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.969+0000 7f23357fa640 1 -- 172.21.15.84:0/3326322290 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338007070 con 0x7f23400b0420 2023-10-16T11:27:50.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.969+0000 7f2346fa9640 1 -- 172.21.15.84:0/3326322290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23400c6e90 con 0x7f23400b0420 2023-10-16T11:27:50.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.969+0000 7f2346fa9640 1 -- 172.21.15.84:0/3326322290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2340162be0 con 0x7f23400b0420 2023-10-16T11:27:50.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.970+0000 7f2346fa9640 1 -- 172.21.15.84:0/3326322290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23400c6a10 con 0x7f23400b0420 2023-10-16T11:27:50.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.971+0000 7f23357fa640 1 -- 172.21.15.84:0/3326322290 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f23380071d0 con 0x7f23400b0420 2023-10-16T11:27:50.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.972+0000 7f23357fa640 1 --2- 172.21.15.84:0/3326322290 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f232c06af90 0x7f232c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:50.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.972+0000 7f23357fa640 1 -- 172.21.15.84:0/3326322290 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f2338089710 con 0x7f23400b0420 2023-10-16T11:27:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.972+0000 7f2344d45640 1 --2- 172.21.15.84:0/3326322290 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f232c06af90 0x7f232c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.973+0000 7f2344d45640 1 --2- 172.21.15.84:0/3326322290 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f232c06af90 0x7f232c06d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2340069740 tx=0x7f233c008040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:50.976+0000 7f23357fa640 1 -- 172.21.15.84:0/3326322290 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f23400c6a10 con 0x7f23400b0420 2023-10-16T11:27:51.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.168+0000 7f2346fa9640 1 -- 172.21.15.84:0/3326322290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f23400c6a10 con 0x7f23400b0420 2023-10-16T11:27:51.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.174+0000 7f23357fa640 1 -- 172.21.15.84:0/3326322290 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v98) v1 ==== 120+0+0 (secure 0 0 0) 0x7f23400c6a10 con 0x7f23400b0420 2023-10-16T11:27:51.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.177+0000 7f2346fa9640 1 -- 172.21.15.84:0/3326322290 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f232c06af90 msgr2=0x7f232c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:51.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.177+0000 7f2346fa9640 1 --2- 172.21.15.84:0/3326322290 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f232c06af90 0x7f232c06d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2340069740 tx=0x7f233c008040 comp rx=0 tx=0).stop 2023-10-16T11:27:51.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.177+0000 7f2346fa9640 1 -- 172.21.15.84:0/3326322290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23400b0420 msgr2=0x7f23400b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:51.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.177+0000 7f2346fa9640 1 --2- 172.21.15.84:0/3326322290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23400b0420 0x7f23400b08a0 secure :-1 s=READY pgs=3525 cs=0 l=1 rev1=1 crypto rx=0x7f2338000f80 tx=0x7f233800b4a0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.177+0000 7f2346fa9640 1 -- 172.21.15.84:0/3326322290 shutdown_connections 2023-10-16T11:27:51.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.177+0000 7f2346fa9640 1 --2- 172.21.15.84:0/3326322290 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f232c06af90 0x7f232c06d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.177+0000 7f2346fa9640 1 --2- 172.21.15.84:0/3326322290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23400b0420 0x7f23400b08a0 unknown :-1 s=CLOSED pgs=3525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.177+0000 7f2346fa9640 1 --2- 172.21.15.84:0/3326322290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23400c5a80 0x7f23400c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.177+0000 7f2346fa9640 1 --2- 172.21.15.84:0/3326322290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23280047c0 0x7f234015fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.177+0000 7f2346fa9640 1 -- 172.21.15.84:0/3326322290 >> 172.21.15.84:0/3326322290 conn(0x7f23400ce9b0 msgr2=0x7f23400c44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:51.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.178+0000 7f2346fa9640 1 -- 172.21.15.84:0/3326322290 shutdown_connections 2023-10-16T11:27:51.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.178+0000 7f2346fa9640 1 -- 172.21.15.84:0/3326322290 wait complete. 2023-10-16T11:27:51.199 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T11:27:51.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.405+0000 7fd35e6df640 1 Processor -- start 2023-10-16T11:27:51.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.406+0000 7fd35e6df640 1 -- start start 2023-10-16T11:27:51.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.407+0000 7fd35e6df640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3580cd970 0x7fd35815aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:51.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.407+0000 7fd35e6df640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35806be30 con 0x7fd3580b0ea0 2023-10-16T11:27:51.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.407+0000 7fd35e6df640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3580ada10 con 0x7fd3580b3ec0 2023-10-16T11:27:51.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.407+0000 7fd35e6df640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35806ca00 con 0x7fd3580cd970 2023-10-16T11:27:51.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.408+0000 7fd35cedc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd3580b0ea0 0x7fd35815a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54168/0 (socket says 172.21.15.84:54168) 2023-10-16T11:27:51.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.408+0000 7fd35cedc640 1 -- 172.21.15.84:0/110874756 learned_addr learned my addr 172.21.15.84:0/110874756 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:51.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.408+0000 7fd35dede640 1 --2- 172.21.15.84:0/110874756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3580cd970 0x7fd35815aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:51.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.408+0000 7fd34ffff640 1 -- 172.21.15.84:0/110874756 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2086842458 0 0) 0x7fd35806be30 con 0x7fd3580b0ea0 2023-10-16T11:27:51.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.408+0000 7fd34ffff640 1 -- 172.21.15.84:0/110874756 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd344003520 con 0x7fd3580b0ea0 2023-10-16T11:27:51.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.409+0000 7fd34ffff640 1 -- 172.21.15.84:0/110874756 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224100367 0 0) 0x7fd3580ada10 con 0x7fd3580b3ec0 2023-10-16T11:27:51.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.409+0000 7fd34ffff640 1 -- 172.21.15.84:0/110874756 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd35806be30 con 0x7fd3580b3ec0 2023-10-16T11:27:51.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.409+0000 7fd34ffff640 1 -- 172.21.15.84:0/110874756 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1987907108 0 0) 0x7fd344003520 con 0x7fd3580b0ea0 2023-10-16T11:27:51.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.409+0000 7fd34ffff640 1 -- 172.21.15.84:0/110874756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3580cd970 msgr2=0x7fd35815aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:51.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.409+0000 7fd34ffff640 1 --2- 172.21.15.84:0/110874756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3580cd970 0x7fd35815aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.409+0000 7fd34ffff640 1 -- 172.21.15.84:0/110874756 >> v1:172.21.15.157:6789/0 conn(0x7fd3580b3ec0 legacy=0x7fd3580cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:51.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.409+0000 7fd34ffff640 1 -- 172.21.15.84:0/110874756 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35806c060 con 0x7fd3580b0ea0 2023-10-16T11:27:51.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.409+0000 7fd34ffff640 1 -- 172.21.15.84:0/110874756 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd348001670 con 0x7fd3580b0ea0 2023-10-16T11:27:51.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.409+0000 7fd34ffff640 1 -- 172.21.15.84:0/110874756 >> v1:172.21.15.84:6789/0 conn(0x7fd3580b0ea0 legacy=0x7fd35815a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:51.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.409+0000 7fd34ffff640 1 --2- 172.21.15.84:0/110874756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd344003b30 0x7fd344003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:51.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.409+0000 7fd34ffff640 1 --2- 172.21.15.84:0/110874756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3580cd970 0x7fd3440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:51.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.409+0000 7fd34ffff640 1 --2- 172.21.15.84:0/110874756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd344004d00 0x7fd344005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:51.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.409+0000 7fd34ffff640 1 -- 172.21.15.84:0/110874756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35806ca00 con 0x7fd344004d00 2023-10-16T11:27:51.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.409+0000 7fd34ffff640 1 -- 172.21.15.84:0/110874756 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34400bab0 con 0x7fd344003b30 2023-10-16T11:27:51.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.409+0000 7fd34ffff640 1 -- 172.21.15.84:0/110874756 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34400bc70 con 0x7fd3580cd970 2023-10-16T11:27:51.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.409+0000 7fd35dede640 1 --2- 172.21.15.84:0/110874756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd344004d00 0x7fd344005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:51.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.409+0000 7fd35cedc640 1 --2- 172.21.15.84:0/110874756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3580cd970 0x7fd3440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:51.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.409+0000 7fd35d6dd640 1 --2- 172.21.15.84:0/110874756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd344003b30 0x7fd344003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:51.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.410+0000 7fd35dede640 1 -- 172.21.15.84:0/110874756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3580cd970 msgr2=0x7fd3440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:51.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.410+0000 7fd35dede640 1 --2- 172.21.15.84:0/110874756 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3580cd970 0x7fd3440047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.410+0000 7fd35dede640 1 -- 172.21.15.84:0/110874756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd344003b30 msgr2=0x7fd344003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:51.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.410+0000 7fd35dede640 1 --2- 172.21.15.84:0/110874756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd344003b30 0x7fd344003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.410+0000 7fd35dede640 1 -- 172.21.15.84:0/110874756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd344003520 con 0x7fd344004d00 2023-10-16T11:27:51.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.411+0000 7fd35dede640 1 --2- 172.21.15.84:0/110874756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd344004d00 0x7fd344005180 secure :-1 s=READY pgs=3527 cs=0 l=1 rev1=1 crypto rx=0x7fd350009f90 tx=0x7fd3440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7103d82122b718b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:51.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.411+0000 7fd34ffff640 1 -- 172.21.15.84:0/110874756 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35002cbd0 con 0x7fd344004d00 2023-10-16T11:27:51.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.411+0000 7fd34ffff640 1 -- 172.21.15.84:0/110874756 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35000f040 con 0x7fd344004d00 2023-10-16T11:27:51.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.411+0000 7fd34ffff640 1 -- 172.21.15.84:0/110874756 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd350034440 con 0x7fd344004d00 2023-10-16T11:27:51.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.412+0000 7fd35e6df640 1 -- 172.21.15.84:0/110874756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd344004d00 msgr2=0x7fd344005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:51.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.412+0000 7fd35e6df640 1 --2- 172.21.15.84:0/110874756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd344004d00 0x7fd344005180 secure :-1 s=READY pgs=3527 cs=0 l=1 rev1=1 crypto rx=0x7fd350009f90 tx=0x7fd3440057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.412+0000 7fd35dede640 1 -- 172.21.15.84:0/110874756 reap_dead start 2023-10-16T11:27:51.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.412+0000 7fd35e6df640 1 -- 172.21.15.84:0/110874756 shutdown_connections 2023-10-16T11:27:51.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.412+0000 7fd35e6df640 1 -- 172.21.15.84:0/110874756 >> 172.21.15.84:0/110874756 conn(0x7fd3580b0170 msgr2=0x7fd3580b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:51.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.412+0000 7fd35e6df640 1 -- 172.21.15.84:0/110874756 shutdown_connections 2023-10-16T11:27:51.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.412+0000 7fd35e6df640 1 -- 172.21.15.84:0/110874756 wait complete. 2023-10-16T11:27:51.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.414+0000 7fd35e6df640 1 Processor -- start 2023-10-16T11:27:51.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.414+0000 7fd35e6df640 1 -- start start 2023-10-16T11:27:51.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.415+0000 7fd35e6df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd344004d00 0x7fd3580aad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:51.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.415+0000 7fd35e6df640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3580c21b0 0x7fd3580c2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:51.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.415+0000 7fd35e6df640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd35814d7e0 0x7fd35814dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:51.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.415+0000 7fd35e6df640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3580c2c70 con 0x7fd3580c21b0 2023-10-16T11:27:51.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.415+0000 7fd35e6df640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3580c2f50 con 0x7fd344004d00 2023-10-16T11:27:51.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.415+0000 7fd35e6df640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3580c3230 con 0x7fd35814d7e0 2023-10-16T11:27:51.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.415+0000 7fd35cedc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3580c21b0 0x7fd3580c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:51.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.415+0000 7fd35cedc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3580c21b0 0x7fd3580c2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36944/0 (socket says 172.21.15.84:36944) 2023-10-16T11:27:51.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.415+0000 7fd35cedc640 1 -- 172.21.15.84:0/2522482556 learned_addr learned my addr 172.21.15.84:0/2522482556 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:51.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.415+0000 7fd35cedc640 1 -- 172.21.15.84:0/2522482556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd35814d7e0 msgr2=0x7fd35814dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:51.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.415+0000 7fd35cedc640 1 --2- 172.21.15.84:0/2522482556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd35814d7e0 0x7fd35814dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.415+0000 7fd35cedc640 1 -- 172.21.15.84:0/2522482556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd344004d00 msgr2=0x7fd3580aad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:51.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.415+0000 7fd35cedc640 1 --2- 172.21.15.84:0/2522482556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd344004d00 0x7fd3580aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.415+0000 7fd35cedc640 1 -- 172.21.15.84:0/2522482556 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3580c3510 con 0x7fd3580c21b0 2023-10-16T11:27:51.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.416+0000 7fd35cedc640 1 --2- 172.21.15.84:0/2522482556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3580c21b0 0x7fd3580c2630 secure :-1 s=READY pgs=3528 cs=0 l=1 rev1=1 crypto rx=0x7fd34800e700 tx=0x7fd34800eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:51.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.416+0000 7fd34dffb640 1 -- 172.21.15.84:0/2522482556 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34800c5f0 con 0x7fd3580c21b0 2023-10-16T11:27:51.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.416+0000 7fd34dffb640 1 -- 172.21.15.84:0/2522482556 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34800c750 con 0x7fd3580c21b0 2023-10-16T11:27:51.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.416+0000 7fd34dffb640 1 -- 172.21.15.84:0/2522482556 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34800cb00 con 0x7fd3580c21b0 2023-10-16T11:27:51.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.416+0000 7fd35e6df640 1 -- 172.21.15.84:0/2522482556 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3580c37f0 con 0x7fd3580c21b0 2023-10-16T11:27:51.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.416+0000 7fd35e6df640 1 -- 172.21.15.84:0/2522482556 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3580ab280 con 0x7fd3580c21b0 2023-10-16T11:27:51.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.417+0000 7fd35e6df640 1 -- 172.21.15.84:0/2522482556 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3580c39e0 con 0x7fd3580c21b0 2023-10-16T11:27:51.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.418+0000 7fd34dffb640 1 -- 172.21.15.84:0/2522482556 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd34800cc60 con 0x7fd3580c21b0 2023-10-16T11:27:51.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.419+0000 7fd34dffb640 1 --2- 172.21.15.84:0/2522482556 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fd33c06af60 0x7fd33c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:51.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.419+0000 7fd34dffb640 1 -- 172.21.15.84:0/2522482556 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fd348089600 con 0x7fd3580c21b0 2023-10-16T11:27:51.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.419+0000 7fd35d6dd640 1 --2- 172.21.15.84:0/2522482556 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fd33c06af60 0x7fd33c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:51.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.420+0000 7fd35d6dd640 1 --2- 172.21.15.84:0/2522482556 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fd33c06af60 0x7fd33c06d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd354007fd0 tx=0x7fd354009040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:51.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.423+0000 7fd34dffb640 1 -- 172.21.15.84:0/2522482556 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd3480545c0 con 0x7fd3580c21b0 2023-10-16T11:27:51.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.615+0000 7fd35e6df640 1 -- 172.21.15.84:0/2522482556 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fd35804e9b0 con 0x7fd3580c21b0 2023-10-16T11:27:51.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.621+0000 7fd34dffb640 1 -- 172.21.15.84:0/2522482556 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v99) v1 ==== 115+0+0 (secure 0 0 0) 0x7fd348057be0 con 0x7fd3580c21b0 2023-10-16T11:27:51.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.624+0000 7fd35e6df640 1 -- 172.21.15.84:0/2522482556 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fd33c06af60 msgr2=0x7fd33c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:51.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.624+0000 7fd35e6df640 1 --2- 172.21.15.84:0/2522482556 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fd33c06af60 0x7fd33c06d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd354007fd0 tx=0x7fd354009040 comp rx=0 tx=0).stop 2023-10-16T11:27:51.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.624+0000 7fd35e6df640 1 -- 172.21.15.84:0/2522482556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3580c21b0 msgr2=0x7fd3580c2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:51.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.624+0000 7fd35e6df640 1 --2- 172.21.15.84:0/2522482556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3580c21b0 0x7fd3580c2630 secure :-1 s=READY pgs=3528 cs=0 l=1 rev1=1 crypto rx=0x7fd34800e700 tx=0x7fd34800eb00 comp rx=0 tx=0).stop 2023-10-16T11:27:51.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.624+0000 7fd35e6df640 1 -- 172.21.15.84:0/2522482556 shutdown_connections 2023-10-16T11:27:51.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.624+0000 7fd35e6df640 1 --2- 172.21.15.84:0/2522482556 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fd33c06af60 0x7fd33c06d420 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.624+0000 7fd35e6df640 1 --2- 172.21.15.84:0/2522482556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd35814d7e0 0x7fd35814dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.624+0000 7fd35e6df640 1 --2- 172.21.15.84:0/2522482556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3580c21b0 0x7fd3580c2630 unknown :-1 s=CLOSED pgs=3528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.624+0000 7fd35e6df640 1 --2- 172.21.15.84:0/2522482556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd344004d00 0x7fd3580aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.624+0000 7fd35e6df640 1 -- 172.21.15.84:0/2522482556 >> 172.21.15.84:0/2522482556 conn(0x7fd3580b0170 msgr2=0x7fd3580b12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:51.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.624+0000 7fd35e6df640 1 -- 172.21.15.84:0/2522482556 shutdown_connections 2023-10-16T11:27:51.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.624+0000 7fd35e6df640 1 -- 172.21.15.84:0/2522482556 wait complete. 2023-10-16T11:27:51.644 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments' 2023-10-16T11:27:51.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.853+0000 7ff7c5a2e640 1 Processor -- start 2023-10-16T11:27:51.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.853+0000 7ff7c5a2e640 1 -- start start 2023-10-16T11:27:51.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.855+0000 7ff7c5a2e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff7c01573e0 0x7ff7c00ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:51.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.855+0000 7ff7c5a2e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c0159e70 0x7ff7c015a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:51.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.855+0000 7ff7c5a2e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff7c0070240 0x7ff7c00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:51.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.855+0000 7ff7c5a2e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c0070c10 con 0x7ff7c01573e0 2023-10-16T11:27:51.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.855+0000 7ff7c5a2e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c00ca3a0 con 0x7ff7c0070240 2023-10-16T11:27:51.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.855+0000 7ff7c5a2e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c00cad70 con 0x7ff7c0159e70 2023-10-16T11:27:51.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.855+0000 7ff7c4a2c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff7c01573e0 0x7ff7c00ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:51.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.855+0000 7ff7c4a2c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff7c01573e0 0x7ff7c00ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36948/0 (socket says 172.21.15.84:36948) 2023-10-16T11:27:51.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.855+0000 7ff7c4a2c640 1 -- 172.21.15.84:0/2379258391 learned_addr learned my addr 172.21.15.84:0/2379258391 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:51.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.856+0000 7ff7c522d640 1 --2- 172.21.15.84:0/2379258391 >> v2:172.21.15.157:3300/0 conn(0x7ff7c0070240 0x7ff7c00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:51.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.856+0000 7ff7bffff640 1 --2- 172.21.15.84:0/2379258391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c0159e70 0x7ff7c015a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:51.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.856+0000 7ff7c4a2c640 1 -- 172.21.15.84:0/2379258391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c0159e70 msgr2=0x7ff7c015a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:51.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.856+0000 7ff7c4a2c640 1 --2- 172.21.15.84:0/2379258391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c0159e70 0x7ff7c015a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.856+0000 7ff7c4a2c640 1 -- 172.21.15.84:0/2379258391 >> v2:172.21.15.157:3300/0 conn(0x7ff7c0070240 msgr2=0x7ff7c00706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:51.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.856+0000 7ff7c4a2c640 1 --2- 172.21.15.84:0/2379258391 >> v2:172.21.15.157:3300/0 conn(0x7ff7c0070240 0x7ff7c00706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.856+0000 7ff7c4a2c640 1 -- 172.21.15.84:0/2379258391 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c00c9d70 con 0x7ff7c01573e0 2023-10-16T11:27:51.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.856+0000 7ff7c522d640 1 --2- 172.21.15.84:0/2379258391 >> v2:172.21.15.157:3300/0 conn(0x7ff7c0070240 0x7ff7c00706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:51.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.856+0000 7ff7bffff640 1 --2- 172.21.15.84:0/2379258391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c0159e70 0x7ff7c015a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:51.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.857+0000 7ff7c4a2c640 1 --2- 172.21.15.84:0/2379258391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7c01573e0 0x7ff7c00ca830 secure :-1 s=READY pgs=3529 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00b56e0 tx=0x7ff7b402e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86f6a28d38a753a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:51.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.857+0000 7ff7bf7fe640 1 -- 172.21.15.84:0/2379258391 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b402ee80 con 0x7ff7c01573e0 2023-10-16T11:27:51.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.857+0000 7ff7bf7fe640 1 -- 172.21.15.84:0/2379258391 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b4036440 con 0x7ff7c01573e0 2023-10-16T11:27:51.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.857+0000 7ff7bf7fe640 1 -- 172.21.15.84:0/2379258391 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40367b0 con 0x7ff7c01573e0 2023-10-16T11:27:51.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.858+0000 7ff7c5a2e640 1 -- 172.21.15.84:0/2379258391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7c01573e0 msgr2=0x7ff7c00ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:51.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.858+0000 7ff7c5a2e640 1 --2- 172.21.15.84:0/2379258391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7c01573e0 0x7ff7c00ca830 secure :-1 s=READY pgs=3529 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00b56e0 tx=0x7ff7b402e0d0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.858+0000 7ff7c5a2e640 1 -- 172.21.15.84:0/2379258391 shutdown_connections 2023-10-16T11:27:51.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.858+0000 7ff7c5a2e640 1 --2- 172.21.15.84:0/2379258391 >> v2:172.21.15.157:3300/0 conn(0x7ff7c0070240 0x7ff7c00706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.858+0000 7ff7c5a2e640 1 --2- 172.21.15.84:0/2379258391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c0159e70 0x7ff7c015a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.858+0000 7ff7c5a2e640 1 --2- 172.21.15.84:0/2379258391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7c01573e0 0x7ff7c00ca830 unknown :-1 s=CLOSED pgs=3529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.858+0000 7ff7c5a2e640 1 -- 172.21.15.84:0/2379258391 >> 172.21.15.84:0/2379258391 conn(0x7ff7c00cc230 msgr2=0x7ff7c00cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:51.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.858+0000 7ff7c5a2e640 1 -- 172.21.15.84:0/2379258391 shutdown_connections 2023-10-16T11:27:51.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.858+0000 7ff7c5a2e640 1 -- 172.21.15.84:0/2379258391 wait complete. 2023-10-16T11:27:51.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.860+0000 7ff7c5a2e640 1 Processor -- start 2023-10-16T11:27:51.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.860+0000 7ff7c5a2e640 1 -- start start 2023-10-16T11:27:51.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.861+0000 7ff7c5a2e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c0070240 0x7ff7c00b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:51.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.861+0000 7ff7c5a2e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c01573e0 0x7ff7c00c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:51.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.861+0000 7ff7c5a2e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7c0159e70 0x7ff7c01f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:51.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.861+0000 7ff7c5a2e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c00c7e70 con 0x7ff7c0159e70 2023-10-16T11:27:51.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.861+0000 7ff7c5a2e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c00c72a0 con 0x7ff7c01573e0 2023-10-16T11:27:51.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.861+0000 7ff7c5a2e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c00c7540 con 0x7ff7c0070240 2023-10-16T11:27:51.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.861+0000 7ff7c522d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7c0159e70 0x7ff7c01f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:51.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.861+0000 7ff7c522d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7c0159e70 0x7ff7c01f1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36962/0 (socket says 172.21.15.84:36962) 2023-10-16T11:27:51.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.861+0000 7ff7c522d640 1 -- 172.21.15.84:0/1825308230 learned_addr learned my addr 172.21.15.84:0/1825308230 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:51.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.862+0000 7ff7c4a2c640 1 --2- 172.21.15.84:0/1825308230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c0070240 0x7ff7c00b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:51.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.862+0000 7ff7c522d640 1 -- 172.21.15.84:0/1825308230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c0070240 msgr2=0x7ff7c00b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:51.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.862+0000 7ff7c522d640 1 --2- 172.21.15.84:0/1825308230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c0070240 0x7ff7c00b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.862+0000 7ff7c522d640 1 -- 172.21.15.84:0/1825308230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c01573e0 msgr2=0x7ff7c00c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.862+0000 7ff7c522d640 1 --2- 172.21.15.84:0/1825308230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c01573e0 0x7ff7c00c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.862+0000 7ff7c522d640 1 -- 172.21.15.84:0/1825308230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b400b770 con 0x7ff7c0159e70 2023-10-16T11:27:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.862+0000 7ff7c522d640 1 --2- 172.21.15.84:0/1825308230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7c0159e70 0x7ff7c01f1f10 secure :-1 s=READY pgs=3530 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800cd10 tx=0x7ff7b800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.862+0000 7ff7bd7fa640 1 -- 172.21.15.84:0/1825308230 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b800f680 con 0x7ff7c0159e70 2023-10-16T11:27:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.862+0000 7ff7bd7fa640 1 -- 172.21.15.84:0/1825308230 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b800f800 con 0x7ff7c0159e70 2023-10-16T11:27:51.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.862+0000 7ff7bd7fa640 1 -- 172.21.15.84:0/1825308230 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b800fbb0 con 0x7ff7c0159e70 2023-10-16T11:27:51.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.862+0000 7ff7c5a2e640 1 -- 172.21.15.84:0/1825308230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c00c7820 con 0x7ff7c0159e70 2023-10-16T11:27:51.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.863+0000 7ff7c5a2e640 1 -- 172.21.15.84:0/1825308230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c00b05a0 con 0x7ff7c0159e70 2023-10-16T11:27:51.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.863+0000 7ff7c5a2e640 1 -- 172.21.15.84:0/1825308230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c00c8020 con 0x7ff7c0159e70 2023-10-16T11:27:51.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.865+0000 7ff7bd7fa640 1 -- 172.21.15.84:0/1825308230 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff7b800b040 con 0x7ff7c0159e70 2023-10-16T11:27:51.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.865+0000 7ff7bd7fa640 1 --2- 172.21.15.84:0/1825308230 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7ff7a406af90 0x7ff7a406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:51.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.866+0000 7ff7bd7fa640 1 -- 172.21.15.84:0/1825308230 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7ff7b808a280 con 0x7ff7c0159e70 2023-10-16T11:27:51.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.866+0000 7ff7c4a2c640 1 --2- 172.21.15.84:0/1825308230 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7ff7a406af90 0x7ff7a406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:51.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.867+0000 7ff7c4a2c640 1 --2- 172.21.15.84:0/1825308230 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7ff7a406af90 0x7ff7a406d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4038000 tx=0x7ff7b4008040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:51.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:51.870+0000 7ff7bd7fa640 1 -- 172.21.15.84:0/1825308230 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff7c00c8020 con 0x7ff7c0159e70 2023-10-16T11:27:52.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:52.143+0000 7ff7c5a2e640 1 -- 172.21.15.84:0/1825308230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments"]} v 0) v1 -- 0x7ff7c004e9b0 con 0x7ff7c0159e70 2023-10-16T11:27:52.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:52.279+0000 7ff7bd7fa640 1 -- 172.21.15.84:0/1825308230 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments"]}]=0 v3306) v1 ==== 198+0+0 (secure 0 0 0) 0x7ff7b80550a0 con 0x7ff7c0159e70 2023-10-16T11:27:52.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:52.283+0000 7ff7c5a2e640 1 -- 172.21.15.84:0/1825308230 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7ff7a406af90 msgr2=0x7ff7a406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:52.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:52.283+0000 7ff7c5a2e640 1 --2- 172.21.15.84:0/1825308230 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7ff7a406af90 0x7ff7a406d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4038000 tx=0x7ff7b4008040 comp rx=0 tx=0).stop 2023-10-16T11:27:52.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:52.283+0000 7ff7c5a2e640 1 -- 172.21.15.84:0/1825308230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7c0159e70 msgr2=0x7ff7c01f1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:52.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:52.283+0000 7ff7c5a2e640 1 --2- 172.21.15.84:0/1825308230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7c0159e70 0x7ff7c01f1f10 secure :-1 s=READY pgs=3530 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800cd10 tx=0x7ff7b800a650 comp rx=0 tx=0).stop 2023-10-16T11:27:52.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:52.283+0000 7ff7c5a2e640 1 -- 172.21.15.84:0/1825308230 shutdown_connections 2023-10-16T11:27:52.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:52.283+0000 7ff7c5a2e640 1 --2- 172.21.15.84:0/1825308230 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7ff7a406af90 0x7ff7a406d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:52.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:52.283+0000 7ff7c5a2e640 1 --2- 172.21.15.84:0/1825308230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7c0159e70 0x7ff7c01f1f10 unknown :-1 s=CLOSED pgs=3530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:52.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:52.283+0000 7ff7c5a2e640 1 --2- 172.21.15.84:0/1825308230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7c01573e0 0x7ff7c00c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:52.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:52.283+0000 7ff7c5a2e640 1 --2- 172.21.15.84:0/1825308230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff7c0070240 0x7ff7c00b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:52.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:52.283+0000 7ff7c5a2e640 1 -- 172.21.15.84:0/1825308230 >> 172.21.15.84:0/1825308230 conn(0x7ff7c00cc230 msgr2=0x7ff7c00ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:52.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:52.283+0000 7ff7c5a2e640 1 -- 172.21.15.84:0/1825308230 shutdown_connections 2023-10-16T11:27:52.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:52.283+0000 7ff7c5a2e640 1 -- 172.21.15.84:0/1825308230 wait complete. 2023-10-16T11:27:52.308 INFO:tasks.cephfs_test_runner:Test subvol and group arguments to snap-schedule cli. ... ERROR 2023-10-16T11:27:52.308 INFO:tasks.cephfs_test_runner: 2023-10-16T11:27:52.308 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-16T11:27:52.309 INFO:tasks.cephfs_test_runner:ERROR: test_snap_schedule_subvol_and_group_arguments (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2023-10-16T11:27:52.309 INFO:tasks.cephfs_test_runner:Test subvol and group arguments to snap-schedule cli. 2023-10-16T11:27:52.309 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-16T11:27:52.310 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-16T11:27:52.310 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/cephfs/test_snap_schedules.py", line 585, in test_snap_schedule_subvol_and_group_arguments 2023-10-16T11:27:52.310 INFO:tasks.cephfs_test_runner: self._fs_cmd('subvolume', 'create', 'a', 'sv1') 2023-10-16T11:27:52.310 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/cephfs/test_snap_schedules.py", line 39, in _fs_cmd 2023-10-16T11:27:52.311 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2023-10-16T11:27:52.311 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2023-10-16T11:27:52.311 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2023-10-16T11:27:52.311 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2023-10-16T11:27:52.312 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2023-10-16T11:27:52.312 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd 2023-10-16T11:27:52.312 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-16T11:27:52.312 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run 2023-10-16T11:27:52.313 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-16T11:27:52.313 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run 2023-10-16T11:27:52.313 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-16T11:27:52.313 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait 2023-10-16T11:27:52.314 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-16T11:27:52.314 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-16T11:27:52.314 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-16T11:27:52.314 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi084 with status 2: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create a sv1' 2023-10-16T11:27:52.315 INFO:tasks.cephfs_test_runner: 2023-10-16T11:27:52.315 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-16T11:27:52.315 INFO:tasks.cephfs_test_runner:Ran 15 tests in 5890.203s 2023-10-16T11:27:52.316 INFO:tasks.cephfs_test_runner: 2023-10-16T11:27:52.316 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-10-16T11:27:52.316 INFO:tasks.cephfs_test_runner: 2023-10-16T11:27:52.316 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-16T11:27:52.317 INFO:tasks.cephfs_test_runner:ERROR: test_snap_schedule_subvol_and_group_arguments (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2023-10-16T11:27:52.317 INFO:tasks.cephfs_test_runner:Test subvol and group arguments to snap-schedule cli. 2023-10-16T11:27:52.317 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-16T11:27:52.317 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-16T11:27:52.318 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/cephfs/test_snap_schedules.py", line 585, in test_snap_schedule_subvol_and_group_arguments 2023-10-16T11:27:52.318 INFO:tasks.cephfs_test_runner: self._fs_cmd('subvolume', 'create', 'a', 'sv1') 2023-10-16T11:27:52.318 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/cephfs/test_snap_schedules.py", line 39, in _fs_cmd 2023-10-16T11:27:52.318 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2023-10-16T11:27:52.319 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2023-10-16T11:27:52.319 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2023-10-16T11:27:52.319 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2023-10-16T11:27:52.319 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2023-10-16T11:27:52.320 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd 2023-10-16T11:27:52.320 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-16T11:27:52.320 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run 2023-10-16T11:27:52.320 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-16T11:27:52.321 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run 2023-10-16T11:27:52.321 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-16T11:27:52.321 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait 2023-10-16T11:27:52.321 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-16T11:27:52.322 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-16T11:27:52.322 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-16T11:27:52.322 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi084 with status 2: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create a sv1' 2023-10-16T11:27:52.322 INFO:tasks.cephfs_test_runner: 2023-10-16T11:27:52.323 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_snap_schedule_subvol_and_group_arguments (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2023-10-16T11:27:52.388 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=6c4b54827f9f4f5ebf24f205fad28a20 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_snap_schedule_subvol_and_group_arguments (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2023-10-16T11:27:52.390 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-16T11:27:52.402 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-16T11:27:52.409 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-16T11:27:52.410 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:27:52.410 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:52.473 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:27:52.474 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-16T11:27:52.474 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:52.529 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T11:27:52.529 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:27:52.530 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:52.590 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:27:52.591 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-16T11:27:52.591 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:27:52.646 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T11:27:52.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:52.646 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T11:27:52.705 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T11:27:52.705 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T11:27:52.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:52.707 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T11:27:52.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:52.807 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T11:27:52.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:52.906 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link delete ceph-brx 2023-10-16T11:27:53.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:53.047 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-16T11:27:53.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:53.093 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link delete ceph-brx 2023-10-16T11:27:53.230 INFO:teuthology.orchestra.run.smithi084.stderr:Cannot find device "ceph-brx" 2023-10-16T11:27:53.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T11:27:53.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:53.235 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-10-16T11:27:53.295 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-16T11:27:53.295 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-16T11:27:53.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:53.297 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T11:27:53.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:53.394 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T11:27:53.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:53.482 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2023-10-16T11:27:53.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:53.607 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-10-16T11:27:53.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:27:53.654 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2023-10-16T11:27:53.790 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2023-10-16T11:27:53.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T11:27:53.794 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-16T11:27:53.804 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-16T11:27:53.804 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:27:54.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.015+0000 7fe5bbb1d640 1 Processor -- start 2023-10-16T11:27:54.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.015+0000 7fe5bbb1d640 1 -- start start 2023-10-16T11:27:54.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.016+0000 7fe5bbb1d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe5b4159b20 0x7fe5b406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.016+0000 7fe5bbb1d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe5b4159f00 0x7fe5b40b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.016+0000 7fe5bbb1d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b406f7f0 con 0x7fe5b406c120 2023-10-16T11:27:54.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.016+0000 7fe5bbb1d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b406c5e0 con 0x7fe5b4159b20 2023-10-16T11:27:54.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.016+0000 7fe5bbb1d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b406c8c0 con 0x7fe5b4159f00 2023-10-16T11:27:54.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.016+0000 7fe5b98b9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe5b4159b20 0x7fe5b406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.017+0000 7fe5b98b9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe5b4159b20 0x7fe5b406bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36968/0 (socket says 172.21.15.84:36968) 2023-10-16T11:27:54.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.017+0000 7fe5b98b9640 1 -- 172.21.15.84:0/3545273083 learned_addr learned my addr 172.21.15.84:0/3545273083 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:54.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.017+0000 7fe5b98b9640 1 -- 172.21.15.84:0/3545273083 >> v2:172.21.15.157:3300/0 conn(0x7fe5b4159f00 msgr2=0x7fe5b40b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:54.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.017+0000 7fe5b98b9640 1 --2- 172.21.15.84:0/3545273083 >> v2:172.21.15.157:3300/0 conn(0x7fe5b4159f00 0x7fe5b40b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.017+0000 7fe5b98b9640 1 -- 172.21.15.84:0/3545273083 >> v1:172.21.15.157:6789/0 conn(0x7fe5b406c120 legacy=0x7fe5b40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:54.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.017+0000 7fe5b98b9640 1 -- 172.21.15.84:0/3545273083 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b406cba0 con 0x7fe5b4159b20 2023-10-16T11:27:54.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.018+0000 7fe5b98b9640 1 --2- 172.21.15.84:0/3545273083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5b4159b20 0x7fe5b406bbe0 secure :-1 s=READY pgs=3531 cs=0 l=1 rev1=1 crypto rx=0x7fe5b415d770 tx=0x7fe5b002e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c6df689bcfb5b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:54.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.019+0000 7fe5b88b7640 1 -- 172.21.15.84:0/3545273083 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b002eea0 con 0x7fe5b4159b20 2023-10-16T11:27:54.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.019+0000 7fe5b88b7640 1 -- 172.21.15.84:0/3545273083 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b0036440 con 0x7fe5b4159b20 2023-10-16T11:27:54.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.019+0000 7fe5b88b7640 1 -- 172.21.15.84:0/3545273083 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b00367d0 con 0x7fe5b4159b20 2023-10-16T11:27:54.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.019+0000 7fe5bbb1d640 1 -- 172.21.15.84:0/3545273083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5b4159b20 msgr2=0x7fe5b406bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:54.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.019+0000 7fe5bbb1d640 1 --2- 172.21.15.84:0/3545273083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5b4159b20 0x7fe5b406bbe0 secure :-1 s=READY pgs=3531 cs=0 l=1 rev1=1 crypto rx=0x7fe5b415d770 tx=0x7fe5b002e0f0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.019+0000 7fe5bbb1d640 1 -- 172.21.15.84:0/3545273083 shutdown_connections 2023-10-16T11:27:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.019+0000 7fe5bbb1d640 1 --2- 172.21.15.84:0/3545273083 >> v2:172.21.15.157:3300/0 conn(0x7fe5b4159f00 0x7fe5b40b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.019+0000 7fe5bbb1d640 1 --2- 172.21.15.84:0/3545273083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5b4159b20 0x7fe5b406bbe0 unknown :-1 s=CLOSED pgs=3531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.019+0000 7fe5bbb1d640 1 -- 172.21.15.84:0/3545273083 >> 172.21.15.84:0/3545273083 conn(0x7fe5b40cb410 msgr2=0x7fe5b40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:54.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.019+0000 7fe5bbb1d640 1 -- 172.21.15.84:0/3545273083 shutdown_connections 2023-10-16T11:27:54.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.020+0000 7fe5bbb1d640 1 -- 172.21.15.84:0/3545273083 wait complete. 2023-10-16T11:27:54.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.021+0000 7fe5bbb1d640 1 Processor -- start 2023-10-16T11:27:54.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.021+0000 7fe5bbb1d640 1 -- start start 2023-10-16T11:27:54.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.022+0000 7fe5bbb1d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b406c120 0x7fe5b4158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.022+0000 7fe5bbb1d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5b4159b20 0x7fe5b415a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.022+0000 7fe5bbb1d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5b4159f00 0x7fe5b40c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.022+0000 7fe5bbb1d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b40c24c0 con 0x7fe5b4159b20 2023-10-16T11:27:54.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.022+0000 7fe5bbb1d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b415c7e0 con 0x7fe5b4159f00 2023-10-16T11:27:54.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.022+0000 7fe5bbb1d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b40c3000 con 0x7fe5b406c120 2023-10-16T11:27:54.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.023+0000 7fe5b90b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5b4159b20 0x7fe5b415a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.023+0000 7fe5b90b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5b4159b20 0x7fe5b415a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36976/0 (socket says 172.21.15.84:36976) 2023-10-16T11:27:54.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.023+0000 7fe5b90b8640 1 -- 172.21.15.84:0/3946935864 learned_addr learned my addr 172.21.15.84:0/3946935864 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:54.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.023+0000 7fe5b98b9640 1 --2- 172.21.15.84:0/3946935864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b406c120 0x7fe5b4158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.023+0000 7fe5b90b8640 1 -- 172.21.15.84:0/3946935864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b406c120 msgr2=0x7fe5b4158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:54.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.023+0000 7fe5ba0ba640 1 --2- 172.21.15.84:0/3946935864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5b4159f00 0x7fe5b40c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.023+0000 7fe5b90b8640 1 --2- 172.21.15.84:0/3946935864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b406c120 0x7fe5b4158850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.023+0000 7fe5b90b8640 1 -- 172.21.15.84:0/3946935864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5b4159f00 msgr2=0x7fe5b40c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:54.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.023+0000 7fe5b90b8640 1 --2- 172.21.15.84:0/3946935864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5b4159f00 0x7fe5b40c6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.023+0000 7fe5b90b8640 1 -- 172.21.15.84:0/3946935864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b000b770 con 0x7fe5b4159b20 2023-10-16T11:27:54.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.023+0000 7fe5b90b8640 1 --2- 172.21.15.84:0/3946935864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5b4159b20 0x7fe5b415a460 secure :-1 s=READY pgs=3532 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400b480 tx=0x7fe5a400b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:54.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.024+0000 7fe5aa7fc640 1 -- 172.21.15.84:0/3946935864 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4010870 con 0x7fe5b4159b20 2023-10-16T11:27:54.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.024+0000 7fe5aa7fc640 1 -- 172.21.15.84:0/3946935864 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a40109d0 con 0x7fe5b4159b20 2023-10-16T11:27:54.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.024+0000 7fe5bbb1d640 1 -- 172.21.15.84:0/3946935864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b40c26d0 con 0x7fe5b4159b20 2023-10-16T11:27:54.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.024+0000 7fe5aa7fc640 1 -- 172.21.15.84:0/3946935864 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4010d80 con 0x7fe5b4159b20 2023-10-16T11:27:54.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.024+0000 7fe5bbb1d640 1 -- 172.21.15.84:0/3946935864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b40c2c90 con 0x7fe5b4159b20 2023-10-16T11:27:54.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.025+0000 7fe5bbb1d640 1 -- 172.21.15.84:0/3946935864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b40c3170 con 0x7fe5b4159b20 2023-10-16T11:27:54.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.026+0000 7fe5aa7fc640 1 -- 172.21.15.84:0/3946935864 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe5a400f3c0 con 0x7fe5b4159b20 2023-10-16T11:27:54.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.027+0000 7fe5aa7fc640 1 --2- 172.21.15.84:0/3946935864 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fe59806af90 0x7fe59806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.027+0000 7fe5aa7fc640 1 -- 172.21.15.84:0/3946935864 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fe5a40897d0 con 0x7fe5b4159b20 2023-10-16T11:27:54.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.027+0000 7fe5b98b9640 1 --2- 172.21.15.84:0/3946935864 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fe59806af90 0x7fe59806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.028+0000 7fe5b98b9640 1 --2- 172.21.15.84:0/3946935864 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fe59806af90 0x7fe59806d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0038000 tx=0x7fe5b0032000 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:54.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.032+0000 7fe5aa7fc640 1 -- 172.21.15.84:0/3946935864 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe5a4054790 con 0x7fe5b4159b20 2023-10-16T11:27:54.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.219+0000 7fe5bbb1d640 1 -- 172.21.15.84:0/3946935864 --> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5b406ce80 con 0x7fe59806af90 2023-10-16T11:27:54.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.225+0000 7fe5aa7fc640 1 -- 172.21.15.84:0/3946935864 <== mgr.13719 v2:172.21.15.84:6832/3003264401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48584 (secure 0 0 0) 0x7fe5b0036a60 con 0x7fe59806af90 2023-10-16T11:27:54.228 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:54.228 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:27:54.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.228+0000 7fe5bbb1d640 1 -- 172.21.15.84:0/3946935864 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fe59806af90 msgr2=0x7fe59806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:54.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.228+0000 7fe5bbb1d640 1 --2- 172.21.15.84:0/3946935864 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fe59806af90 0x7fe59806d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0038000 tx=0x7fe5b0032000 comp rx=0 tx=0).stop 2023-10-16T11:27:54.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.228+0000 7fe5bbb1d640 1 -- 172.21.15.84:0/3946935864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5b4159b20 msgr2=0x7fe5b415a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:54.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.228+0000 7fe5bbb1d640 1 --2- 172.21.15.84:0/3946935864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5b4159b20 0x7fe5b415a460 secure :-1 s=READY pgs=3532 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400b480 tx=0x7fe5a400b940 comp rx=0 tx=0).stop 2023-10-16T11:27:54.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.228+0000 7fe5bbb1d640 1 -- 172.21.15.84:0/3946935864 shutdown_connections 2023-10-16T11:27:54.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.228+0000 7fe5bbb1d640 1 --2- 172.21.15.84:0/3946935864 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fe59806af90 0x7fe59806d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.228+0000 7fe5bbb1d640 1 --2- 172.21.15.84:0/3946935864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5b4159f00 0x7fe5b40c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.228+0000 7fe5bbb1d640 1 --2- 172.21.15.84:0/3946935864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5b4159b20 0x7fe5b415a460 unknown :-1 s=CLOSED pgs=3532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.228+0000 7fe5bbb1d640 1 --2- 172.21.15.84:0/3946935864 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5b406c120 0x7fe5b4158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.228+0000 7fe5bbb1d640 1 -- 172.21.15.84:0/3946935864 >> 172.21.15.84:0/3946935864 conn(0x7fe5b40cb410 msgr2=0x7fe5b40ae3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:54.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.228+0000 7fe5bbb1d640 1 -- 172.21.15.84:0/3946935864 shutdown_connections 2023-10-16T11:27:54.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.228+0000 7fe5bbb1d640 1 -- 172.21.15.84:0/3946935864 wait complete. 2023-10-16T11:27:54.249 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-10-16T11:27:53.491942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2783,"num_read_kb":5735,"num_write":703,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4836,"ondisk_log_size":4836,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":481304,"kb_used_data":29240,"kb_used_omap":12,"kb_used_meta":451955,"kb_avail":754493416,"statfs":{"total":773094113280,"available":772601257984,"internally_reserved":0,"allocated":29941760,"data_stored":15788895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":462802489},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-530474,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":10,"num_read_kb":133,"num_write":-143,"num_write_kb":-1330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.435533"},"pg_stats":[{"pgid":"1.0","version":"272'4836","reported_seq":5961,"reported_epoch":273,"state":"active+clean","last_fresh":"2023-10-16T11:27:47.446875+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:27:47.446875+0000","last_peered":"2023-10-16T11:27:47.446875+0000","last_clean":"2023-10-16T11:27:47.446875+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:27:47.446875+0000","last_undegraded":"2023-10-16T11:27:47.446875+0000","last_fullsized":"2023-10-16T11:27:47.446875+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":4836,"log_dups_size":0,"ondisk_log_size":4836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2783,"num_read_kb":5735,"num_write":703,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2783,"num_read_kb":5735,"num_write":703,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4836,"ondisk_log_size":4836,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576022,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79932,"kb_used_data":4336,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291908,"statfs":{"total":96636764160,"available":96554913792,"internally_reserved":0,"allocated":4440064,"data_stored":2676753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":77396427},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1140000000000001,"15min":1.1859999999999999},"min":{"1min":0.64500000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.673,"5min":1.7250000000000001,"15min":2.3399999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.109,"5min":1.1259999999999999,"15min":1.1930000000000001},"min":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.633,"5min":1.6699999999999999,"15min":1.71},"last":1.8380000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2010000000000001,"15min":1.2290000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1719999999999999,"15min":1.2470000000000001},"min":{"1min":0.75600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.6100000000000001,"5min":1.6799999999999999,"15min":1.8600000000000001},"last":0.98399999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2749999999999999,"15min":1.286},"min":{"1min":0.71799999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8640000000000001,"5min":1.9039999999999999,"15min":2.4460000000000002},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.3140000000000001,"15min":1.349},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.5389999999999999,"5min":3.9249999999999998,"15min":3.9249999999999998},"last":1.304}]},{"osd":3,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.339,"15min":1.371},"min":{"1min":0.85999999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8169999999999999,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.389,"15min":1.4339999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.016,"5min":2.016,"15min":2.577},"last":1.542}]},{"osd":4,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.419,"15min":1.4039999999999999},"min":{"1min":0.64800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.113,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.345,"5min":1.379,"15min":1.3779999999999999},"min":{"1min":0.65600000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.756,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":2.161}]},{"osd":5,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.5669999999999999,"15min":1.5780000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":3.3519999999999999},"last":2.105},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.534,"15min":1.498},"min":{"1min":0.62,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.0880000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":2.2400000000000002}]},{"osd":6,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.488,"15min":1.4970000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.907,"5min":2.2469999999999999,"15min":2.7250000000000001},"last":1.792},{"interface":"front","average":{"1min":1.633,"5min":1.6639999999999999,"15min":1.6859999999999999},"min":{"1min":0.749,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.323,"5min":2.323,"15min":3.1440000000000001},"last":1.98}]}]},{"osd":6,"up_from":13,"seq":55834576022,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52616,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319224,"statfs":{"total":96636764160,"available":96582885376,"internally_reserved":0,"allocated":3469312,"data_stored":1699888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1779999999999999,"15min":1.202},"min":{"1min":0.69399999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.538,"5min":6.298,"15min":6.298},"last":1.177},{"interface":"front","average":{"1min":1.159,"5min":1.2989999999999999,"15min":1.27},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.597,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":1.3939999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.198,"15min":1.204},"min":{"1min":0.72299999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7969999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3740000000000001,"15min":1.361},"min":{"1min":0.749,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.714,"5min":12.468,"15min":12.468},"last":1.337}]},{"osd":2,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.2549999999999999,"15min":1.254},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.639,"5min":11.586,"15min":11.586},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.385,"15min":1.371},"min":{"1min":0.87,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7170000000000001,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.367}]},{"osd":3,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.4139999999999999,"15min":1.337},"min":{"1min":0.68100000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.766,"5min":12.801,"15min":12.801},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.254,"15min":1.2589999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.7969999999999999,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":1.099}]},{"osd":4,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4370000000000001,"15min":1.341},"min":{"1min":1.0429999999999999,"5min":0.65400000000000003,"15min":0.49099999999999999},"max":{"1min":1.931,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.51},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.6519999999999999,"15min":1.5569999999999999},"min":{"1min":0.70399999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.97,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.48}]},{"osd":5,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.55,"15min":1.3979999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.9319999999999999,"5min":18.247,"15min":18.247},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.6279999999999999,"15min":1.569},"min":{"1min":0.76700000000000002,"5min":0.76200000000000001,"15min":0.60899999999999999},"max":{"1min":2.085,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.45}]},{"osd":7,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.6990000000000001,"15min":1.5580000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.59199999999999997},"max":{"1min":1.9870000000000001,"5min":12.589,"15min":12.589},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.6990000000000001,"15min":1.597},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0249999999999999,"5min":12.725,"15min":12.725},"last":1.306}]}]},{"osd":5,"up_from":13,"seq":55834576022,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52620,"kb_used_data":3392,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319220,"statfs":{"total":96636764160,"available":96582881280,"internally_reserved":0,"allocated":3473408,"data_stored":1698119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.1519999999999999,"15min":1.234},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.585,"5min":1.585,"15min":2.661},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.248,"15min":1.3169999999999999},"min":{"1min":0.75600000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.583,"5min":1.7909999999999999,"15min":1.7949999999999999},"last":1.6799999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1279999999999999,"15min":1.175},"min":{"1min":0.82199999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.748,"5min":1.748,"15min":1.992},"last":1.014},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.1919999999999999,"15min":1.228},"min":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":2.9399999999999999},"last":1.5369999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2470000000000001,"15min":1.2390000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2869999999999999,"15min":1.3009999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7829999999999999,"5min":1.8660000000000001,"15min":2.1640000000000001},"last":1.198}]},{"osd":3,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.29,"15min":1.3169999999999999},"min":{"1min":0.82199999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.121,"5min":2.121,"15min":3.012},"last":1.738},{"interface":"front","average":{"1min":1.429,"5min":1.387,"15min":1.411},"min":{"1min":1.0880000000000001,"5min":0.92700000000000005,"15min":0.89500000000000002},"max":{"1min":1.9299999999999999,"5min":2.0129999999999999,"15min":2.2250000000000001},"last":0.876}]},{"osd":4,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.3280000000000001,"15min":1.345},"min":{"1min":0.80300000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.8660000000000001},"last":1.383},{"interface":"front","average":{"1min":1.605,"5min":1.5429999999999999,"15min":1.5620000000000001},"min":{"1min":1.125,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.903},"last":1.792}]},{"osd":6,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.4059999999999999,"15min":1.417},"min":{"1min":0.95099999999999996,"5min":0.68100000000000005,"15min":0.51000000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0979999999999999},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.53,"5min":1.4379999999999999,"15min":1.4770000000000001},"min":{"1min":1.004,"5min":0.90700000000000003,"15min":0.89200000000000002},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":3.073},"last":1.7030000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.552,"15min":1.548},"min":{"1min":1.228,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":3.1160000000000001},"last":1.593},{"interface":"front","average":{"1min":1.702,"5min":1.619,"15min":1.639},"min":{"1min":1.2809999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.161,"5min":2.1930000000000001,"15min":2.2930000000000001},"last":1.4259999999999999}]}]},{"osd":4,"up_from":13,"seq":55834576023,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88772,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94283068,"statfs":{"total":96636764160,"available":96545861632,"internally_reserved":0,"allocated":3596288,"data_stored":1825421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":87292358},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.151,"15min":1.1739999999999999},"min":{"1min":0.73799999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.615,"5min":2.899,"15min":2.899},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.296,"5min":1.236,"15min":1.254},"min":{"1min":0.77500000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.784,"5min":3.016,"15min":3.016},"last":0.76600000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.28,"15min":1.298},"min":{"1min":0.83999999999999997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6799999999999999,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.647},{"interface":"front","average":{"1min":1.089,"5min":1.228,"15min":1.25},"min":{"1min":0.84499999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.4299999999999999,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.2010000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8300000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4370000000000001,"15min":1.4199999999999999},"min":{"1min":0.90500000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.748,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.1299999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.351,"15min":1.349},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.7669999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.452,"5min":1.4930000000000001,"15min":1.4610000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.9930000000000001,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.837}]},{"osd":5,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":1.542,"15min":1.4790000000000001},"min":{"1min":1.0549999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.915,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.861},{"interface":"front","average":{"1min":1.444,"5min":1.4650000000000001,"15min":1.4370000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.9410000000000001,"5min":3.7989999999999999,"15min":3.7989999999999999},"last":1.333}]},{"osd":6,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.6020000000000001,"15min":1.5349999999999999},"min":{"1min":0.85699999999999998,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":2.0779999999999998,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":2.016},{"interface":"front","average":{"1min":1.724,"5min":1.679,"15min":1.6459999999999999},"min":{"1min":1.268,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.944,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.9670000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":1.647,"15min":1.607},"min":{"1min":0.73699999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.1949999999999998,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":1.425},{"interface":"front","average":{"1min":1.702,"5min":1.7190000000000001,"15min":1.6799999999999999},"min":{"1min":1.026,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.1419999999999999,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.913}]}]},{"osd":3,"up_from":13,"seq":55834576022,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52604,"kb_used_data":3376,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319236,"statfs":{"total":96636764160,"available":96582897664,"internally_reserved":0,"allocated":3457024,"data_stored":1693751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81699999999999995,"15min":0.88200000000000001},"min":{"1min":0.33800000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.901},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.89200000000000002,"15min":1.0249999999999999},"min":{"1min":0.443,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1919999999999999,"5min":1.581,"15min":1.6679999999999999},"last":1.3859999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.88800000000000001,"15min":0.95899999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3420000000000001,"5min":1.508,"15min":1.508},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.94899999999999995,"15min":1.087},"min":{"1min":0.51300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4770000000000001,"5min":1.601,"15min":1.7190000000000001},"last":1.1120000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.99199999999999999,"15min":1.087},"min":{"1min":0.56299999999999994,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.524,"5min":1.5660000000000001,"15min":1.694},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.079,"15min":1.1870000000000001},"min":{"1min":0.48499999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.597,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.341}]},{"osd":4,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.23,"15min":1.3240000000000001},"min":{"1min":0.72499999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6839999999999999,"5min":1.821,"15min":1.873},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2949999999999999,"15min":1.3440000000000001},"min":{"1min":0.80900000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.681,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.1719999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.3009999999999999,"15min":1.385},"min":{"1min":0.69299999999999995,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.7749999999999999,"5min":2.798,"15min":2.798},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3149999999999999,"15min":1.4179999999999999},"min":{"1min":0.74199999999999999,"5min":0.63,"15min":0.63},"max":{"1min":1.772,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.2609999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4330000000000001,"15min":1.51},"min":{"1min":0.93000000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.9970000000000001,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.413,"15min":1.506},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9350000000000001,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.45}]},{"osd":7,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.46,"15min":1.544},"min":{"1min":0.96799999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.8380000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.478},{"interface":"front","average":{"1min":1.48,"5min":1.5329999999999999,"15min":1.6459999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.952,"5min":2.3799999999999999,"15min":2.4649999999999999},"last":1.5349999999999999}]}]},{"osd":2,"up_from":13,"seq":55834576021,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56768,"kb_used_data":3380,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315072,"statfs":{"total":96636764160,"available":96578633728,"internally_reserved":0,"allocated":3461120,"data_stored":1693293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.85799999999999998,"15min":0.89900000000000002},"min":{"1min":0.39800000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.3200000000000001,"5min":1.532,"15min":1.6399999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.92100000000000004,"15min":1.0369999999999999},"min":{"1min":0.53400000000000003,"5min":0.40500000000000003,"15min":0.32600000000000001},"max":{"1min":1.258,"5min":1.6359999999999999,"15min":2.1429999999999998},"last":0.86299999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.046,"15min":1.075},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39200000000000002},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.02,"15min":1.1120000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.2030000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.96099999999999997,"15min":1.0469999999999999},"min":{"1min":0.52300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.452,"5min":1.522,"15min":1.8560000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1639999999999999,"15min":1.242},"min":{"1min":0.58199999999999996,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.0459999999999998,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":1.3240000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.306,"15min":1.3819999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.8079999999999998,"5min":3.8079999999999998,"15min":3.8079999999999998},"last":1.54},{"interface":"front","average":{"1min":1.236,"5min":1.2490000000000001,"15min":1.3140000000000001},"min":{"1min":0.79700000000000004,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.256}]},{"osd":5,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.397,"15min":1.431},"min":{"1min":0.77900000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.391},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3839999999999999,"15min":1.4470000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0499999999999998,"5min":2.323,"15min":2.6259999999999999},"last":1.6180000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3560000000000001,"15min":1.381},"min":{"1min":0.85099999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.4830000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.464,"15min":1.5089999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":1.5840000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.4910000000000001,"15min":1.5489999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.742,"5min":3.742,"15min":3.742},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.4339999999999999,"15min":1.4810000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":1.4379999999999999}]}]},{"osd":1,"up_from":13,"seq":55834576021,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48832,"kb_used_data":3764,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323008,"statfs":{"total":96636764160,"available":96586760192,"internally_reserved":0,"allocated":3854336,"data_stored":2086441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.89300000000000002,"15min":0.91000000000000003},"min":{"1min":0.40699999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4039999999999999,"5min":5.117,"15min":5.117},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.89300000000000002,"15min":1.0029999999999999},"min":{"1min":0.45500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.325,"5min":1.5509999999999999,"15min":1.667},"last":0.79200000000000004}]},{"osd":2,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.91300000000000003,"15min":0.98899999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.651,"5min":2.093,"15min":2.093},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.149,"15min":1.242},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2400000000000002,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":1.1699999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.125,"15min":1.179},"min":{"1min":0.56999999999999995,"5min":0.439,"15min":0.439},"max":{"1min":1.8879999999999999,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.05,"15min":1.1399999999999999},"min":{"1min":0.52700000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5860000000000001,"5min":1.798,"15min":1.802},"last":1.1830000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.2749999999999999,"15min":1.2849999999999999},"min":{"1min":0.70299999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.978,"5min":5.29,"15min":5.29},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.403,"15min":1.409},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.0129999999999999,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":1.1220000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.3400000000000001,"15min":1.3979999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.829,"5min":6.0979999999999999,"15min":6.0979999999999999},"last":1.409},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.3420000000000001,"15min":1.3899999999999999},"min":{"1min":0.74099999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7410000000000001,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.2509999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.429,"15min":1.482},"min":{"1min":0.82399999999999995,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8979999999999999,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":1.552},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.393,"15min":1.4630000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.948,"5min":3.8799999999999999,"15min":3.8799999999999999},"last":1.3660000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.518,"15min":1.5509999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.157,"5min":4.46,"15min":4.46},"last":1.298},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.5349999999999999,"15min":1.589},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.0259999999999998,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":1.4910000000000001}]}]},{"osd":0,"up_from":13,"seq":55834576022,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49160,"kb_used_data":4092,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322680,"statfs":{"total":96636764160,"available":96586424320,"internally_reserved":0,"allocated":4190208,"data_stored":2415229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85499999999999998,"15min":0.88500000000000001},"min":{"1min":0.44900000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2350000000000001,"5min":1.502,"15min":1.619},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.98699999999999999,"15min":1.0740000000000001},"min":{"1min":0.40000000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.518,"5min":1.621,"15min":1.76},"last":1.3300000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0640000000000001,"15min":1.0820000000000001},"min":{"1min":0.51300000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.569,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.996,"5min":1.036,"15min":1.1180000000000001},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.7649999999999999,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":1.014}]},{"osd":3,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0089999999999999,"15min":1.0640000000000001},"min":{"1min":0.621,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.5189999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.131,"15min":1.2170000000000001},"min":{"1min":0.59099999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.45,"5min":1.833,"15min":1.833},"last":1.2669999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.3149999999999999,"15min":1.377},"min":{"1min":0.93000000000000005,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.538,"5min":3.2799999999999998,"15min":3.2799999999999998},"last":1.546},{"interface":"front","average":{"1min":1.286,"5min":1.2869999999999999,"15min":1.3320000000000001},"min":{"1min":0.80000000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.752,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.23}]},{"osd":5,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.3109999999999999,"15min":1.3660000000000001},"min":{"1min":0.76000000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.605,"5min":2.0209999999999999,"15min":2.6669999999999998},"last":1.099},{"interface":"front","average":{"1min":1.478,"5min":1.385,"15min":1.4339999999999999},"min":{"1min":0.877,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.4359999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4570000000000001,"15min":1.4930000000000001},"min":{"1min":0.78900000000000003,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.1309999999999998,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.363,"5min":1.4019999999999999,"15min":1.468},"min":{"1min":0.90200000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7310000000000001,"5min":2.073,"15min":2.262},"last":1.613}]},{"osd":7,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5309999999999999,"15min":1.5549999999999999},"min":{"1min":1.0429999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7869999999999999,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.419,"15min":1.486},"min":{"1min":0.98399999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7869999999999999,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.653}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:27:54.251 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:27:54.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.463+0000 7ff2aeb33640 1 Processor -- start 2023-10-16T11:27:54.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.463+0000 7ff2aeb33640 1 -- start start 2023-10-16T11:27:54.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.465+0000 7ff2aeb33640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff2a815adc0 0x7ff2a80cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.465+0000 7ff2aeb33640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2a80ad990 0x7ff2a815f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.465+0000 7ff2aeb33640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff2a8070c70 0x7ff2a80710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.465+0000 7ff2aeb33640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a80b4380 con 0x7ff2a815adc0 2023-10-16T11:27:54.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.465+0000 7ff2aeb33640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a80b4620 con 0x7ff2a8070c70 2023-10-16T11:27:54.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.465+0000 7ff2aeb33640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a815e9b0 con 0x7ff2a80ad990 2023-10-16T11:27:54.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.465+0000 7ff2ac8cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff2a815adc0 0x7ff2a80cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.465+0000 7ff2ac8cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff2a815adc0 0x7ff2a80cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36978/0 (socket says 172.21.15.84:36978) 2023-10-16T11:27:54.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.465+0000 7ff2ac8cf640 1 -- 172.21.15.84:0/2209693528 learned_addr learned my addr 172.21.15.84:0/2209693528 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:54.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.466+0000 7ff2a7fff640 1 --2- 172.21.15.84:0/2209693528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2a80ad990 0x7ff2a815f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.466+0000 7ff2ad0d0640 1 --2- 172.21.15.84:0/2209693528 >> v2:172.21.15.157:3300/0 conn(0x7ff2a8070c70 0x7ff2a80710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.466+0000 7ff2ac8cf640 1 -- 172.21.15.84:0/2209693528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2a80ad990 msgr2=0x7ff2a815f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:54.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.466+0000 7ff2ac8cf640 1 --2- 172.21.15.84:0/2209693528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2a80ad990 0x7ff2a815f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.466+0000 7ff2ac8cf640 1 -- 172.21.15.84:0/2209693528 >> v2:172.21.15.157:3300/0 conn(0x7ff2a8070c70 msgr2=0x7ff2a80710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:54.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.466+0000 7ff2ac8cf640 1 --2- 172.21.15.84:0/2209693528 >> v2:172.21.15.157:3300/0 conn(0x7ff2a8070c70 0x7ff2a80710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.466+0000 7ff2ac8cf640 1 -- 172.21.15.84:0/2209693528 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a815ec90 con 0x7ff2a815adc0 2023-10-16T11:27:54.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.466+0000 7ff2ad0d0640 1 --2- 172.21.15.84:0/2209693528 >> v2:172.21.15.157:3300/0 conn(0x7ff2a8070c70 0x7ff2a80710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:27:54.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.468+0000 7ff2ac8cf640 1 --2- 172.21.15.84:0/2209693528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2a815adc0 0x7ff2a80cf380 secure :-1 s=READY pgs=3533 cs=0 l=1 rev1=1 crypto rx=0x7ff2a806f3f0 tx=0x7ff29c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b1654f542277ae server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:54.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.468+0000 7ff2a77fe640 1 -- 172.21.15.84:0/2209693528 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c031870 con 0x7ff2a815adc0 2023-10-16T11:27:54.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.468+0000 7ff2a77fe640 1 -- 172.21.15.84:0/2209693528 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c0319d0 con 0x7ff2a815adc0 2023-10-16T11:27:54.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.468+0000 7ff2a77fe640 1 -- 172.21.15.84:0/2209693528 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c031d60 con 0x7ff2a815adc0 2023-10-16T11:27:54.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.469+0000 7ff2aeb33640 1 -- 172.21.15.84:0/2209693528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2a815adc0 msgr2=0x7ff2a80cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:54.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.469+0000 7ff2aeb33640 1 --2- 172.21.15.84:0/2209693528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2a815adc0 0x7ff2a80cf380 secure :-1 s=READY pgs=3533 cs=0 l=1 rev1=1 crypto rx=0x7ff2a806f3f0 tx=0x7ff29c02b910 comp rx=0 tx=0).stop 2023-10-16T11:27:54.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.469+0000 7ff2aeb33640 1 -- 172.21.15.84:0/2209693528 shutdown_connections 2023-10-16T11:27:54.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.469+0000 7ff2aeb33640 1 --2- 172.21.15.84:0/2209693528 >> v2:172.21.15.157:3300/0 conn(0x7ff2a8070c70 0x7ff2a80710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.469+0000 7ff2aeb33640 1 --2- 172.21.15.84:0/2209693528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2a80ad990 0x7ff2a815f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.469+0000 7ff2aeb33640 1 --2- 172.21.15.84:0/2209693528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2a815adc0 0x7ff2a80cf380 unknown :-1 s=CLOSED pgs=3533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.469+0000 7ff2aeb33640 1 -- 172.21.15.84:0/2209693528 >> 172.21.15.84:0/2209693528 conn(0x7ff2a80ce860 msgr2=0x7ff2a80cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:54.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.469+0000 7ff2aeb33640 1 -- 172.21.15.84:0/2209693528 shutdown_connections 2023-10-16T11:27:54.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.469+0000 7ff2aeb33640 1 -- 172.21.15.84:0/2209693528 wait complete. 2023-10-16T11:27:54.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.471+0000 7ff2aeb33640 1 Processor -- start 2023-10-16T11:27:54.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.471+0000 7ff2aeb33640 1 -- start start 2023-10-16T11:27:54.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.472+0000 7ff2aeb33640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2a8070c70 0x7ff2a80c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.472+0000 7ff2aeb33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a80ad990 0x7ff2a80ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.472+0000 7ff2aeb33640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2a815adc0 0x7ff2a814e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.472+0000 7ff2aeb33640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80c9290 con 0x7ff2a815adc0 2023-10-16T11:27:54.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.472+0000 7ff2aeb33640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80c9570 con 0x7ff2a80ad990 2023-10-16T11:27:54.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.472+0000 7ff2aeb33640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80c9850 con 0x7ff2a8070c70 2023-10-16T11:27:54.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.472+0000 7ff2ad0d0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2a815adc0 0x7ff2a814e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.472+0000 7ff2ad0d0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2a815adc0 0x7ff2a814e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36990/0 (socket says 172.21.15.84:36990) 2023-10-16T11:27:54.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.473+0000 7ff2ad0d0640 1 -- 172.21.15.84:0/2670409599 learned_addr learned my addr 172.21.15.84:0/2670409599 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:54.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.473+0000 7ff2ac8cf640 1 --2- 172.21.15.84:0/2670409599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2a8070c70 0x7ff2a80c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.473+0000 7ff2a7fff640 1 --2- 172.21.15.84:0/2670409599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a80ad990 0x7ff2a80ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.473+0000 7ff2ad0d0640 1 -- 172.21.15.84:0/2670409599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2a8070c70 msgr2=0x7ff2a80c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:54.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.473+0000 7ff2ad0d0640 1 --2- 172.21.15.84:0/2670409599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2a8070c70 0x7ff2a80c81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.473+0000 7ff2ad0d0640 1 -- 172.21.15.84:0/2670409599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a80ad990 msgr2=0x7ff2a80ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:54.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.473+0000 7ff2ad0d0640 1 --2- 172.21.15.84:0/2670409599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a80ad990 0x7ff2a80ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.473+0000 7ff2ad0d0640 1 -- 172.21.15.84:0/2670409599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c0094d0 con 0x7ff2a815adc0 2023-10-16T11:27:54.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.473+0000 7ff2ad0d0640 1 --2- 172.21.15.84:0/2670409599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2a815adc0 0x7ff2a814e700 secure :-1 s=READY pgs=3534 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0007e70 tx=0x7ff2a000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:54.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.473+0000 7ff2a57fa640 1 -- 172.21.15.84:0/2670409599 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0003b90 con 0x7ff2a815adc0 2023-10-16T11:27:54.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.474+0000 7ff2a57fa640 1 -- 172.21.15.84:0/2670409599 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a000f040 con 0x7ff2a815adc0 2023-10-16T11:27:54.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.474+0000 7ff2a57fa640 1 -- 172.21.15.84:0/2670409599 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a000de80 con 0x7ff2a815adc0 2023-10-16T11:27:54.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.474+0000 7ff2aeb33640 1 -- 172.21.15.84:0/2670409599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a80c9b30 con 0x7ff2a815adc0 2023-10-16T11:27:54.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.474+0000 7ff2aeb33640 1 -- 172.21.15.84:0/2670409599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a80c86f0 con 0x7ff2a815adc0 2023-10-16T11:27:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.475+0000 7ff2aeb33640 1 -- 172.21.15.84:0/2670409599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a80ca050 con 0x7ff2a815adc0 2023-10-16T11:27:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.476+0000 7ff2a57fa640 1 -- 172.21.15.84:0/2670409599 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff2a0004030 con 0x7ff2a815adc0 2023-10-16T11:27:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.477+0000 7ff2a57fa640 1 --2- 172.21.15.84:0/2670409599 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7ff28c06af90 0x7ff28c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.477+0000 7ff2a57fa640 1 -- 172.21.15.84:0/2670409599 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7ff2a0089c20 con 0x7ff2a815adc0 2023-10-16T11:27:54.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.477+0000 7ff2ac8cf640 1 --2- 172.21.15.84:0/2670409599 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7ff28c06af90 0x7ff28c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.478+0000 7ff2ac8cf640 1 --2- 172.21.15.84:0/2670409599 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7ff28c06af90 0x7ff28c06d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff29c036000 tx=0x7ff29c03c040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:54.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.481+0000 7ff2a57fa640 1 -- 172.21.15.84:0/2670409599 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff2a80ca050 con 0x7ff2a815adc0 2023-10-16T11:27:54.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.671+0000 7ff2aeb33640 1 -- 172.21.15.84:0/2670409599 --> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff2a80c99c0 con 0x7ff28c06af90 2023-10-16T11:27:54.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.677+0000 7ff2a57fa640 1 -- 172.21.15.84:0/2670409599 <== mgr.13719 v2:172.21.15.84:6832/3003264401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48584 (secure 0 0 0) 0x7ff2a80c99c0 con 0x7ff28c06af90 2023-10-16T11:27:54.680 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:54.680 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:27:54.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.680+0000 7ff2aeb33640 1 -- 172.21.15.84:0/2670409599 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7ff28c06af90 msgr2=0x7ff28c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:54.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.680+0000 7ff2aeb33640 1 --2- 172.21.15.84:0/2670409599 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7ff28c06af90 0x7ff28c06d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff29c036000 tx=0x7ff29c03c040 comp rx=0 tx=0).stop 2023-10-16T11:27:54.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.680+0000 7ff2aeb33640 1 -- 172.21.15.84:0/2670409599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2a815adc0 msgr2=0x7ff2a814e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:54.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.680+0000 7ff2aeb33640 1 --2- 172.21.15.84:0/2670409599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2a815adc0 0x7ff2a814e700 secure :-1 s=READY pgs=3534 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0007e70 tx=0x7ff2a000d4a0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.680+0000 7ff2aeb33640 1 -- 172.21.15.84:0/2670409599 shutdown_connections 2023-10-16T11:27:54.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.680+0000 7ff2aeb33640 1 --2- 172.21.15.84:0/2670409599 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7ff28c06af90 0x7ff28c06d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.680+0000 7ff2aeb33640 1 --2- 172.21.15.84:0/2670409599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2a815adc0 0x7ff2a814e700 unknown :-1 s=CLOSED pgs=3534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.680+0000 7ff2aeb33640 1 --2- 172.21.15.84:0/2670409599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a80ad990 0x7ff2a80ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.680+0000 7ff2aeb33640 1 --2- 172.21.15.84:0/2670409599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff2a8070c70 0x7ff2a80c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.680+0000 7ff2aeb33640 1 -- 172.21.15.84:0/2670409599 >> 172.21.15.84:0/2670409599 conn(0x7ff2a80ce860 msgr2=0x7ff2a814f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:54.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.680+0000 7ff2aeb33640 1 -- 172.21.15.84:0/2670409599 shutdown_connections 2023-10-16T11:27:54.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.680+0000 7ff2aeb33640 1 -- 172.21.15.84:0/2670409599 wait complete. 2023-10-16T11:27:54.701 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-10-16T11:27:53.491942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2783,"num_read_kb":5735,"num_write":703,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4836,"ondisk_log_size":4836,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":481304,"kb_used_data":29240,"kb_used_omap":12,"kb_used_meta":451955,"kb_avail":754493416,"statfs":{"total":773094113280,"available":772601257984,"internally_reserved":0,"allocated":29941760,"data_stored":15788895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":462802489},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-530474,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":10,"num_read_kb":133,"num_write":-143,"num_write_kb":-1330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.435533"},"pg_stats":[{"pgid":"1.0","version":"272'4836","reported_seq":5961,"reported_epoch":273,"state":"active+clean","last_fresh":"2023-10-16T11:27:47.446875+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:27:47.446875+0000","last_peered":"2023-10-16T11:27:47.446875+0000","last_clean":"2023-10-16T11:27:47.446875+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:27:47.446875+0000","last_undegraded":"2023-10-16T11:27:47.446875+0000","last_fullsized":"2023-10-16T11:27:47.446875+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":4836,"log_dups_size":0,"ondisk_log_size":4836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2783,"num_read_kb":5735,"num_write":703,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2783,"num_read_kb":5735,"num_write":703,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4836,"ondisk_log_size":4836,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576022,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79932,"kb_used_data":4336,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291908,"statfs":{"total":96636764160,"available":96554913792,"internally_reserved":0,"allocated":4440064,"data_stored":2676753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":77396427},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1140000000000001,"15min":1.1859999999999999},"min":{"1min":0.64500000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.673,"5min":1.7250000000000001,"15min":2.3399999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.109,"5min":1.1259999999999999,"15min":1.1930000000000001},"min":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.633,"5min":1.6699999999999999,"15min":1.71},"last":1.8380000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2010000000000001,"15min":1.2290000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1719999999999999,"15min":1.2470000000000001},"min":{"1min":0.75600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.6100000000000001,"5min":1.6799999999999999,"15min":1.8600000000000001},"last":0.98399999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2749999999999999,"15min":1.286},"min":{"1min":0.71799999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8640000000000001,"5min":1.9039999999999999,"15min":2.4460000000000002},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.3140000000000001,"15min":1.349},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.5389999999999999,"5min":3.9249999999999998,"15min":3.9249999999999998},"last":1.304}]},{"osd":3,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.339,"15min":1.371},"min":{"1min":0.85999999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8169999999999999,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.389,"15min":1.4339999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.016,"5min":2.016,"15min":2.577},"last":1.542}]},{"osd":4,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.419,"15min":1.4039999999999999},"min":{"1min":0.64800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.113,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.345,"5min":1.379,"15min":1.3779999999999999},"min":{"1min":0.65600000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.756,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":2.161}]},{"osd":5,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.5669999999999999,"15min":1.5780000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":3.3519999999999999},"last":2.105},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.534,"15min":1.498},"min":{"1min":0.62,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.0880000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":2.2400000000000002}]},{"osd":6,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.488,"15min":1.4970000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.907,"5min":2.2469999999999999,"15min":2.7250000000000001},"last":1.792},{"interface":"front","average":{"1min":1.633,"5min":1.6639999999999999,"15min":1.6859999999999999},"min":{"1min":0.749,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.323,"5min":2.323,"15min":3.1440000000000001},"last":1.98}]}]},{"osd":6,"up_from":13,"seq":55834576022,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52616,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319224,"statfs":{"total":96636764160,"available":96582885376,"internally_reserved":0,"allocated":3469312,"data_stored":1699888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1779999999999999,"15min":1.202},"min":{"1min":0.69399999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.538,"5min":6.298,"15min":6.298},"last":1.177},{"interface":"front","average":{"1min":1.159,"5min":1.2989999999999999,"15min":1.27},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.597,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":1.3939999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.198,"15min":1.204},"min":{"1min":0.72299999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7969999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3740000000000001,"15min":1.361},"min":{"1min":0.749,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.714,"5min":12.468,"15min":12.468},"last":1.337}]},{"osd":2,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.2549999999999999,"15min":1.254},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.639,"5min":11.586,"15min":11.586},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.385,"15min":1.371},"min":{"1min":0.87,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7170000000000001,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.367}]},{"osd":3,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.4139999999999999,"15min":1.337},"min":{"1min":0.68100000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.766,"5min":12.801,"15min":12.801},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.254,"15min":1.2589999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.7969999999999999,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":1.099}]},{"osd":4,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4370000000000001,"15min":1.341},"min":{"1min":1.0429999999999999,"5min":0.65400000000000003,"15min":0.49099999999999999},"max":{"1min":1.931,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.51},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.6519999999999999,"15min":1.5569999999999999},"min":{"1min":0.70399999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.97,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.48}]},{"osd":5,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.55,"15min":1.3979999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.9319999999999999,"5min":18.247,"15min":18.247},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.6279999999999999,"15min":1.569},"min":{"1min":0.76700000000000002,"5min":0.76200000000000001,"15min":0.60899999999999999},"max":{"1min":2.085,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.45}]},{"osd":7,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.6990000000000001,"15min":1.5580000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.59199999999999997},"max":{"1min":1.9870000000000001,"5min":12.589,"15min":12.589},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.6990000000000001,"15min":1.597},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0249999999999999,"5min":12.725,"15min":12.725},"last":1.306}]}]},{"osd":5,"up_from":13,"seq":55834576022,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52620,"kb_used_data":3392,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319220,"statfs":{"total":96636764160,"available":96582881280,"internally_reserved":0,"allocated":3473408,"data_stored":1698119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.1519999999999999,"15min":1.234},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.585,"5min":1.585,"15min":2.661},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.248,"15min":1.3169999999999999},"min":{"1min":0.75600000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.583,"5min":1.7909999999999999,"15min":1.7949999999999999},"last":1.6799999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1279999999999999,"15min":1.175},"min":{"1min":0.82199999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.748,"5min":1.748,"15min":1.992},"last":1.014},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.1919999999999999,"15min":1.228},"min":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":2.9399999999999999},"last":1.5369999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2470000000000001,"15min":1.2390000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2869999999999999,"15min":1.3009999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7829999999999999,"5min":1.8660000000000001,"15min":2.1640000000000001},"last":1.198}]},{"osd":3,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.29,"15min":1.3169999999999999},"min":{"1min":0.82199999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.121,"5min":2.121,"15min":3.012},"last":1.738},{"interface":"front","average":{"1min":1.429,"5min":1.387,"15min":1.411},"min":{"1min":1.0880000000000001,"5min":0.92700000000000005,"15min":0.89500000000000002},"max":{"1min":1.9299999999999999,"5min":2.0129999999999999,"15min":2.2250000000000001},"last":0.876}]},{"osd":4,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.3280000000000001,"15min":1.345},"min":{"1min":0.80300000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.8660000000000001},"last":1.383},{"interface":"front","average":{"1min":1.605,"5min":1.5429999999999999,"15min":1.5620000000000001},"min":{"1min":1.125,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.903},"last":1.792}]},{"osd":6,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.4059999999999999,"15min":1.417},"min":{"1min":0.95099999999999996,"5min":0.68100000000000005,"15min":0.51000000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0979999999999999},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.53,"5min":1.4379999999999999,"15min":1.4770000000000001},"min":{"1min":1.004,"5min":0.90700000000000003,"15min":0.89200000000000002},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":3.073},"last":1.7030000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.552,"15min":1.548},"min":{"1min":1.228,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":3.1160000000000001},"last":1.593},{"interface":"front","average":{"1min":1.702,"5min":1.619,"15min":1.639},"min":{"1min":1.2809999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.161,"5min":2.1930000000000001,"15min":2.2930000000000001},"last":1.4259999999999999}]}]},{"osd":4,"up_from":13,"seq":55834576023,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88772,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94283068,"statfs":{"total":96636764160,"available":96545861632,"internally_reserved":0,"allocated":3596288,"data_stored":1825421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":87292358},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.151,"15min":1.1739999999999999},"min":{"1min":0.73799999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.615,"5min":2.899,"15min":2.899},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.296,"5min":1.236,"15min":1.254},"min":{"1min":0.77500000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.784,"5min":3.016,"15min":3.016},"last":0.76600000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.28,"15min":1.298},"min":{"1min":0.83999999999999997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6799999999999999,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.647},{"interface":"front","average":{"1min":1.089,"5min":1.228,"15min":1.25},"min":{"1min":0.84499999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.4299999999999999,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.2010000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8300000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4370000000000001,"15min":1.4199999999999999},"min":{"1min":0.90500000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.748,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.1299999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.351,"15min":1.349},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.7669999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.452,"5min":1.4930000000000001,"15min":1.4610000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.9930000000000001,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.837}]},{"osd":5,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":1.542,"15min":1.4790000000000001},"min":{"1min":1.0549999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.915,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.861},{"interface":"front","average":{"1min":1.444,"5min":1.4650000000000001,"15min":1.4370000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.9410000000000001,"5min":3.7989999999999999,"15min":3.7989999999999999},"last":1.333}]},{"osd":6,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.6020000000000001,"15min":1.5349999999999999},"min":{"1min":0.85699999999999998,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":2.0779999999999998,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":2.016},{"interface":"front","average":{"1min":1.724,"5min":1.679,"15min":1.6459999999999999},"min":{"1min":1.268,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.944,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.9670000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":1.647,"15min":1.607},"min":{"1min":0.73699999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.1949999999999998,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":1.425},{"interface":"front","average":{"1min":1.702,"5min":1.7190000000000001,"15min":1.6799999999999999},"min":{"1min":1.026,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.1419999999999999,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.913}]}]},{"osd":3,"up_from":13,"seq":55834576022,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52604,"kb_used_data":3376,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319236,"statfs":{"total":96636764160,"available":96582897664,"internally_reserved":0,"allocated":3457024,"data_stored":1693751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81699999999999995,"15min":0.88200000000000001},"min":{"1min":0.33800000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.901},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.89200000000000002,"15min":1.0249999999999999},"min":{"1min":0.443,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1919999999999999,"5min":1.581,"15min":1.6679999999999999},"last":1.3859999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.88800000000000001,"15min":0.95899999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3420000000000001,"5min":1.508,"15min":1.508},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.94899999999999995,"15min":1.087},"min":{"1min":0.51300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4770000000000001,"5min":1.601,"15min":1.7190000000000001},"last":1.1120000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.99199999999999999,"15min":1.087},"min":{"1min":0.56299999999999994,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.524,"5min":1.5660000000000001,"15min":1.694},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.079,"15min":1.1870000000000001},"min":{"1min":0.48499999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.597,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.341}]},{"osd":4,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.23,"15min":1.3240000000000001},"min":{"1min":0.72499999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6839999999999999,"5min":1.821,"15min":1.873},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2949999999999999,"15min":1.3440000000000001},"min":{"1min":0.80900000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.681,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.1719999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.3009999999999999,"15min":1.385},"min":{"1min":0.69299999999999995,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.7749999999999999,"5min":2.798,"15min":2.798},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3149999999999999,"15min":1.4179999999999999},"min":{"1min":0.74199999999999999,"5min":0.63,"15min":0.63},"max":{"1min":1.772,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.2609999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4330000000000001,"15min":1.51},"min":{"1min":0.93000000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.9970000000000001,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.413,"15min":1.506},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9350000000000001,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.45}]},{"osd":7,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.46,"15min":1.544},"min":{"1min":0.96799999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.8380000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.478},{"interface":"front","average":{"1min":1.48,"5min":1.5329999999999999,"15min":1.6459999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.952,"5min":2.3799999999999999,"15min":2.4649999999999999},"last":1.5349999999999999}]}]},{"osd":2,"up_from":13,"seq":55834576021,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56768,"kb_used_data":3380,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315072,"statfs":{"total":96636764160,"available":96578633728,"internally_reserved":0,"allocated":3461120,"data_stored":1693293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.85799999999999998,"15min":0.89900000000000002},"min":{"1min":0.39800000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.3200000000000001,"5min":1.532,"15min":1.6399999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.92100000000000004,"15min":1.0369999999999999},"min":{"1min":0.53400000000000003,"5min":0.40500000000000003,"15min":0.32600000000000001},"max":{"1min":1.258,"5min":1.6359999999999999,"15min":2.1429999999999998},"last":0.86299999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.046,"15min":1.075},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39200000000000002},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.02,"15min":1.1120000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.2030000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.96099999999999997,"15min":1.0469999999999999},"min":{"1min":0.52300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.452,"5min":1.522,"15min":1.8560000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1639999999999999,"15min":1.242},"min":{"1min":0.58199999999999996,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.0459999999999998,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":1.3240000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.306,"15min":1.3819999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.8079999999999998,"5min":3.8079999999999998,"15min":3.8079999999999998},"last":1.54},{"interface":"front","average":{"1min":1.236,"5min":1.2490000000000001,"15min":1.3140000000000001},"min":{"1min":0.79700000000000004,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.256}]},{"osd":5,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.397,"15min":1.431},"min":{"1min":0.77900000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.391},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3839999999999999,"15min":1.4470000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0499999999999998,"5min":2.323,"15min":2.6259999999999999},"last":1.6180000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3560000000000001,"15min":1.381},"min":{"1min":0.85099999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.4830000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.464,"15min":1.5089999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":1.5840000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.4910000000000001,"15min":1.5489999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.742,"5min":3.742,"15min":3.742},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.4339999999999999,"15min":1.4810000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":1.4379999999999999}]}]},{"osd":1,"up_from":13,"seq":55834576021,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48832,"kb_used_data":3764,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323008,"statfs":{"total":96636764160,"available":96586760192,"internally_reserved":0,"allocated":3854336,"data_stored":2086441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.89300000000000002,"15min":0.91000000000000003},"min":{"1min":0.40699999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4039999999999999,"5min":5.117,"15min":5.117},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.89300000000000002,"15min":1.0029999999999999},"min":{"1min":0.45500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.325,"5min":1.5509999999999999,"15min":1.667},"last":0.79200000000000004}]},{"osd":2,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.91300000000000003,"15min":0.98899999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.651,"5min":2.093,"15min":2.093},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.149,"15min":1.242},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2400000000000002,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":1.1699999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.125,"15min":1.179},"min":{"1min":0.56999999999999995,"5min":0.439,"15min":0.439},"max":{"1min":1.8879999999999999,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.05,"15min":1.1399999999999999},"min":{"1min":0.52700000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5860000000000001,"5min":1.798,"15min":1.802},"last":1.1830000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.2749999999999999,"15min":1.2849999999999999},"min":{"1min":0.70299999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.978,"5min":5.29,"15min":5.29},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.403,"15min":1.409},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.0129999999999999,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":1.1220000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.3400000000000001,"15min":1.3979999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.829,"5min":6.0979999999999999,"15min":6.0979999999999999},"last":1.409},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.3420000000000001,"15min":1.3899999999999999},"min":{"1min":0.74099999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7410000000000001,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.2509999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.429,"15min":1.482},"min":{"1min":0.82399999999999995,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8979999999999999,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":1.552},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.393,"15min":1.4630000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.948,"5min":3.8799999999999999,"15min":3.8799999999999999},"last":1.3660000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.518,"15min":1.5509999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.157,"5min":4.46,"15min":4.46},"last":1.298},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.5349999999999999,"15min":1.589},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.0259999999999998,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":1.4910000000000001}]}]},{"osd":0,"up_from":13,"seq":55834576022,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49160,"kb_used_data":4092,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322680,"statfs":{"total":96636764160,"available":96586424320,"internally_reserved":0,"allocated":4190208,"data_stored":2415229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85499999999999998,"15min":0.88500000000000001},"min":{"1min":0.44900000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2350000000000001,"5min":1.502,"15min":1.619},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.98699999999999999,"15min":1.0740000000000001},"min":{"1min":0.40000000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.518,"5min":1.621,"15min":1.76},"last":1.3300000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0640000000000001,"15min":1.0820000000000001},"min":{"1min":0.51300000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.569,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.996,"5min":1.036,"15min":1.1180000000000001},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.7649999999999999,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":1.014}]},{"osd":3,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0089999999999999,"15min":1.0640000000000001},"min":{"1min":0.621,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.5189999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.131,"15min":1.2170000000000001},"min":{"1min":0.59099999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.45,"5min":1.833,"15min":1.833},"last":1.2669999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.3149999999999999,"15min":1.377},"min":{"1min":0.93000000000000005,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.538,"5min":3.2799999999999998,"15min":3.2799999999999998},"last":1.546},{"interface":"front","average":{"1min":1.286,"5min":1.2869999999999999,"15min":1.3320000000000001},"min":{"1min":0.80000000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.752,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.23}]},{"osd":5,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.3109999999999999,"15min":1.3660000000000001},"min":{"1min":0.76000000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.605,"5min":2.0209999999999999,"15min":2.6669999999999998},"last":1.099},{"interface":"front","average":{"1min":1.478,"5min":1.385,"15min":1.4339999999999999},"min":{"1min":0.877,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.4359999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4570000000000001,"15min":1.4930000000000001},"min":{"1min":0.78900000000000003,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.1309999999999998,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.363,"5min":1.4019999999999999,"15min":1.468},"min":{"1min":0.90200000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7310000000000001,"5min":2.073,"15min":2.262},"last":1.613}]},{"osd":7,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5309999999999999,"15min":1.5549999999999999},"min":{"1min":1.0429999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7869999999999999,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.419,"15min":1.486},"min":{"1min":0.98399999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7869999999999999,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.653}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:27:54.703 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-16T11:27:54.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:27:54.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.902+0000 7fd303261640 1 Processor -- start 2023-10-16T11:27:54.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.902+0000 7fd303261640 1 -- start start 2023-10-16T11:27:54.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.903+0000 7fd303261640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd2fc15aaa0 0x7fd2fc15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.903+0000 7fd303261640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2fc15ed60 con 0x7fd2fc15ae80 2023-10-16T11:27:54.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.904+0000 7fd303261640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2fc0707e0 con 0x7fd2fc0ce390 2023-10-16T11:27:54.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.904+0000 7fd303261640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2fc06b810 con 0x7fd2fc15aaa0 2023-10-16T11:27:54.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.904+0000 7fd2fbfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd2fc15ae80 0x7fd2fc0cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54178/0 (socket says 172.21.15.84:54178) 2023-10-16T11:27:54.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.904+0000 7fd2fbfff640 1 -- 172.21.15.84:0/890931555 learned_addr learned my addr 172.21.15.84:0/890931555 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:54.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.904+0000 7fd2fb7fe640 1 -- 172.21.15.84:0/890931555 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306608800 0 0) 0x7fd2fc15ed60 con 0x7fd2fc15ae80 2023-10-16T11:27:54.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.904+0000 7fd2fb7fe640 1 -- 172.21.15.84:0/890931555 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2ec003520 con 0x7fd2fc15ae80 2023-10-16T11:27:54.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.905+0000 7fd300ffd640 1 --2- 172.21.15.84:0/890931555 >> v2:172.21.15.157:3300/0 conn(0x7fd2fc15aaa0 0x7fd2fc15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.905+0000 7fd2fb7fe640 1 -- 172.21.15.84:0/890931555 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3258191679 0 0) 0x7fd2ec003520 con 0x7fd2fc15ae80 2023-10-16T11:27:54.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.905+0000 7fd2fb7fe640 1 -- 172.21.15.84:0/890931555 >> v2:172.21.15.157:3300/0 conn(0x7fd2fc15aaa0 msgr2=0x7fd2fc15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:54.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.905+0000 7fd2fb7fe640 1 --2- 172.21.15.84:0/890931555 >> v2:172.21.15.157:3300/0 conn(0x7fd2fc15aaa0 0x7fd2fc15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.905+0000 7fd2fb7fe640 1 -- 172.21.15.84:0/890931555 >> v1:172.21.15.157:6789/0 conn(0x7fd2fc0ce390 legacy=0x7fd2fc070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:54.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.905+0000 7fd2fb7fe640 1 -- 172.21.15.84:0/890931555 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc15ef90 con 0x7fd2fc15ae80 2023-10-16T11:27:54.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.905+0000 7fd2fb7fe640 1 -- 172.21.15.84:0/890931555 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd2e8001670 con 0x7fd2fc15ae80 2023-10-16T11:27:54.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.905+0000 7fd2fb7fe640 1 -- 172.21.15.84:0/890931555 >> v1:172.21.15.84:6789/0 conn(0x7fd2fc15ae80 legacy=0x7fd2fc0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:54.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.905+0000 7fd2fb7fe640 1 --2- 172.21.15.84:0/890931555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec0039c0 0x7fd2ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.905+0000 7fd2fb7fe640 1 --2- 172.21.15.84:0/890931555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2ec0047c0 0x7fd2ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.905+0000 7fd2fb7fe640 1 --2- 172.21.15.84:0/890931555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2ec0051d0 0x7fd2ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.905+0000 7fd2fb7fe640 1 -- 172.21.15.84:0/890931555 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc06b810 con 0x7fd2ec0051d0 2023-10-16T11:27:54.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.905+0000 7fd2fb7fe640 1 -- 172.21.15.84:0/890931555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec00c120 con 0x7fd2ec0039c0 2023-10-16T11:27:54.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.905+0000 7fd2fb7fe640 1 -- 172.21.15.84:0/890931555 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec00c2e0 con 0x7fd2ec0047c0 2023-10-16T11:27:54.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.905+0000 7fd3017fe640 1 --2- 172.21.15.84:0/890931555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2ec0051d0 0x7fd2ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.906+0000 7fd2fbfff640 1 --2- 172.21.15.84:0/890931555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2ec0047c0 0x7fd2ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.906+0000 7fd3017fe640 1 -- 172.21.15.84:0/890931555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2ec0047c0 msgr2=0x7fd2ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:54.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.906+0000 7fd3017fe640 1 --2- 172.21.15.84:0/890931555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2ec0047c0 0x7fd2ec004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.906+0000 7fd3017fe640 1 -- 172.21.15.84:0/890931555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec0039c0 msgr2=0x7fd2ec003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:54.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.906+0000 7fd300ffd640 1 --2- 172.21.15.84:0/890931555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec0039c0 0x7fd2ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.906+0000 7fd3017fe640 1 --2- 172.21.15.84:0/890931555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec0039c0 0x7fd2ec003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.906+0000 7fd3017fe640 1 -- 172.21.15.84:0/890931555 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec003520 con 0x7fd2ec0051d0 2023-10-16T11:27:54.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.906+0000 7fd300ffd640 1 --2- 172.21.15.84:0/890931555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec0039c0 0x7fd2ec003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:54.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.907+0000 7fd3017fe640 1 -- 172.21.15.84:0/890931555 reap_dead start 2023-10-16T11:27:54.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.907+0000 7fd3017fe640 1 --2- 172.21.15.84:0/890931555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2ec0051d0 0x7fd2ec00b980 secure :-1 s=READY pgs=3536 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4001e50 tx=0x7fd2ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefe06e792f9361b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:54.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.907+0000 7fd2fb7fe640 1 -- 172.21.15.84:0/890931555 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4003b70 con 0x7fd2ec0051d0 2023-10-16T11:27:54.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.907+0000 7fd2fb7fe640 1 -- 172.21.15.84:0/890931555 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f4003d10 con 0x7fd2ec0051d0 2023-10-16T11:27:54.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.907+0000 7fd2fb7fe640 1 -- 172.21.15.84:0/890931555 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4005310 con 0x7fd2ec0051d0 2023-10-16T11:27:54.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.908+0000 7fd303261640 1 -- 172.21.15.84:0/890931555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2ec0051d0 msgr2=0x7fd2ec00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:54.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.908+0000 7fd303261640 1 --2- 172.21.15.84:0/890931555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2ec0051d0 0x7fd2ec00b980 secure :-1 s=READY pgs=3536 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4001e50 tx=0x7fd2ec0057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.908+0000 7fd303261640 1 -- 172.21.15.84:0/890931555 shutdown_connections 2023-10-16T11:27:54.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.908+0000 7fd303261640 1 --2- 172.21.15.84:0/890931555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2ec0051d0 0x7fd2ec00b980 unknown :-1 s=CLOSED pgs=3536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.908+0000 7fd303261640 1 -- 172.21.15.84:0/890931555 >> 172.21.15.84:0/890931555 conn(0x7fd2fc0cb410 msgr2=0x7fd2fc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:54.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.908+0000 7fd303261640 1 -- 172.21.15.84:0/890931555 shutdown_connections 2023-10-16T11:27:54.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.908+0000 7fd303261640 1 -- 172.21.15.84:0/890931555 wait complete. 2023-10-16T11:27:54.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.910+0000 7fd303261640 1 Processor -- start 2023-10-16T11:27:54.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.910+0000 7fd303261640 1 -- start start 2023-10-16T11:27:54.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.911+0000 7fd303261640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2ec0051d0 0x7fd2fc0c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.911+0000 7fd303261640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2fc1578b0 0x7fd2fc1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.911+0000 7fd303261640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2fc0ac890 0x7fd2fc0acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.911+0000 7fd303261640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc1f6ca0 con 0x7fd2fc1578b0 2023-10-16T11:27:54.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.911+0000 7fd303261640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc0ad250 con 0x7fd2fc0ac890 2023-10-16T11:27:54.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.911+0000 7fd303261640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc157710 con 0x7fd2ec0051d0 2023-10-16T11:27:54.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.911+0000 7fd2fbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2fc1578b0 0x7fd2fc1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.911+0000 7fd2fbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2fc1578b0 0x7fd2fc1571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37004/0 (socket says 172.21.15.84:37004) 2023-10-16T11:27:54.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.911+0000 7fd2fbfff640 1 -- 172.21.15.84:0/291767939 learned_addr learned my addr 172.21.15.84:0/291767939 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:54.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.911+0000 7fd300ffd640 1 --2- 172.21.15.84:0/291767939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2ec0051d0 0x7fd2fc0c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.911+0000 7fd2fbfff640 1 -- 172.21.15.84:0/291767939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2ec0051d0 msgr2=0x7fd2fc0c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:54.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.911+0000 7fd2fbfff640 1 --2- 172.21.15.84:0/291767939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2ec0051d0 0x7fd2fc0c8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.911+0000 7fd2fbfff640 1 -- 172.21.15.84:0/291767939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2fc0ac890 msgr2=0x7fd2fc0acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:54.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.911+0000 7fd2fbfff640 1 --2- 172.21.15.84:0/291767939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2fc0ac890 0x7fd2fc0acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:54.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.911+0000 7fd2fbfff640 1 -- 172.21.15.84:0/291767939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc1f6020 con 0x7fd2fc1578b0 2023-10-16T11:27:54.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.912+0000 7fd2fbfff640 1 --2- 172.21.15.84:0/291767939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2fc1578b0 0x7fd2fc1571d0 secure :-1 s=READY pgs=3537 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800d7d0 tx=0x7fd2e800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:54.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.912+0000 7fd2f97fa640 1 -- 172.21.15.84:0/291767939 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80072d0 con 0x7fd2fc1578b0 2023-10-16T11:27:54.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.912+0000 7fd2f97fa640 1 -- 172.21.15.84:0/291767939 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e8007430 con 0x7fd2fc1578b0 2023-10-16T11:27:54.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.912+0000 7fd2f97fa640 1 -- 172.21.15.84:0/291767939 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80047e0 con 0x7fd2fc1578b0 2023-10-16T11:27:54.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.912+0000 7fd303261640 1 -- 172.21.15.84:0/291767939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2fc1f6300 con 0x7fd2fc1578b0 2023-10-16T11:27:54.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.912+0000 7fd303261640 1 -- 172.21.15.84:0/291767939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2fc1f68c0 con 0x7fd2fc1578b0 2023-10-16T11:27:54.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.913+0000 7fd303261640 1 -- 172.21.15.84:0/291767939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2fc1f6a80 con 0x7fd2fc1578b0 2023-10-16T11:27:54.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.914+0000 7fd2f97fa640 1 -- 172.21.15.84:0/291767939 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd2e8004940 con 0x7fd2fc1578b0 2023-10-16T11:27:54.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.915+0000 7fd2f97fa640 1 --2- 172.21.15.84:0/291767939 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fd2e006af90 0x7fd2e006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:54.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.915+0000 7fd2f97fa640 1 -- 172.21.15.84:0/291767939 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fd2e8089390 con 0x7fd2fc1578b0 2023-10-16T11:27:54.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.915+0000 7fd300ffd640 1 --2- 172.21.15.84:0/291767939 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fd2e006af90 0x7fd2e006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:54.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.916+0000 7fd300ffd640 1 --2- 172.21.15.84:0/291767939 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fd2e006af90 0x7fd2e006d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0006fd0 tx=0x7fd2f0008040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:54.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:54.919+0000 7fd2f97fa640 1 -- 172.21.15.84:0/291767939 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd2e8057d80 con 0x7fd2fc1578b0 2023-10-16T11:27:55.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.108+0000 7fd303261640 1 -- 172.21.15.84:0/291767939 --> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd2fc15f550 con 0x7fd2e006af90 2023-10-16T11:27:55.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.114+0000 7fd2f97fa640 1 -- 172.21.15.84:0/291767939 <== mgr.13719 v2:172.21.15.84:6832/3003264401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48584 (secure 0 0 0) 0x7fd2f00095b0 con 0x7fd2e006af90 2023-10-16T11:27:55.117 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:55.118 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:27:55.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.117+0000 7fd303261640 1 -- 172.21.15.84:0/291767939 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fd2e006af90 msgr2=0x7fd2e006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:55.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.117+0000 7fd303261640 1 --2- 172.21.15.84:0/291767939 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fd2e006af90 0x7fd2e006d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0006fd0 tx=0x7fd2f0008040 comp rx=0 tx=0).stop 2023-10-16T11:27:55.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.117+0000 7fd303261640 1 -- 172.21.15.84:0/291767939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2fc1578b0 msgr2=0x7fd2fc1571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:55.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.117+0000 7fd303261640 1 --2- 172.21.15.84:0/291767939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2fc1578b0 0x7fd2fc1571d0 secure :-1 s=READY pgs=3537 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800d7d0 tx=0x7fd2e800dc90 comp rx=0 tx=0).stop 2023-10-16T11:27:55.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.118+0000 7fd303261640 1 -- 172.21.15.84:0/291767939 shutdown_connections 2023-10-16T11:27:55.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.118+0000 7fd303261640 1 --2- 172.21.15.84:0/291767939 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fd2e006af90 0x7fd2e006d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:55.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.118+0000 7fd303261640 1 --2- 172.21.15.84:0/291767939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2fc0ac890 0x7fd2fc0acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:55.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.118+0000 7fd303261640 1 --2- 172.21.15.84:0/291767939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2fc1578b0 0x7fd2fc1571d0 unknown :-1 s=CLOSED pgs=3537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:55.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.118+0000 7fd303261640 1 --2- 172.21.15.84:0/291767939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2ec0051d0 0x7fd2fc0c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:55.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.118+0000 7fd303261640 1 -- 172.21.15.84:0/291767939 >> 172.21.15.84:0/291767939 conn(0x7fd2fc0cb410 msgr2=0x7fd2fc15a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:55.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.118+0000 7fd303261640 1 -- 172.21.15.84:0/291767939 shutdown_connections 2023-10-16T11:27:55.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.118+0000 7fd303261640 1 -- 172.21.15.84:0/291767939 wait complete. 2023-10-16T11:27:55.139 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-10-16T11:27:53.491942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2783,"num_read_kb":5735,"num_write":703,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4836,"ondisk_log_size":4836,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":481304,"kb_used_data":29240,"kb_used_omap":12,"kb_used_meta":451955,"kb_avail":754493416,"statfs":{"total":773094113280,"available":772601257984,"internally_reserved":0,"allocated":29941760,"data_stored":15788895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":462802489},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-530474,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":10,"num_read_kb":133,"num_write":-143,"num_write_kb":-1330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.435533"},"pg_stats":[{"pgid":"1.0","version":"272'4836","reported_seq":5961,"reported_epoch":273,"state":"active+clean","last_fresh":"2023-10-16T11:27:47.446875+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:27:47.446875+0000","last_peered":"2023-10-16T11:27:47.446875+0000","last_clean":"2023-10-16T11:27:47.446875+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:27:47.446875+0000","last_undegraded":"2023-10-16T11:27:47.446875+0000","last_fullsized":"2023-10-16T11:27:47.446875+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":4836,"log_dups_size":0,"ondisk_log_size":4836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2783,"num_read_kb":5735,"num_write":703,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2783,"num_read_kb":5735,"num_write":703,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4836,"ondisk_log_size":4836,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576022,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79932,"kb_used_data":4336,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291908,"statfs":{"total":96636764160,"available":96554913792,"internally_reserved":0,"allocated":4440064,"data_stored":2676753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":77396427},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1140000000000001,"15min":1.1859999999999999},"min":{"1min":0.64500000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.673,"5min":1.7250000000000001,"15min":2.3399999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.109,"5min":1.1259999999999999,"15min":1.1930000000000001},"min":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.633,"5min":1.6699999999999999,"15min":1.71},"last":1.8380000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2010000000000001,"15min":1.2290000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1719999999999999,"15min":1.2470000000000001},"min":{"1min":0.75600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.6100000000000001,"5min":1.6799999999999999,"15min":1.8600000000000001},"last":0.98399999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2749999999999999,"15min":1.286},"min":{"1min":0.71799999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8640000000000001,"5min":1.9039999999999999,"15min":2.4460000000000002},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.3140000000000001,"15min":1.349},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.5389999999999999,"5min":3.9249999999999998,"15min":3.9249999999999998},"last":1.304}]},{"osd":3,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.339,"15min":1.371},"min":{"1min":0.85999999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8169999999999999,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.389,"15min":1.4339999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.016,"5min":2.016,"15min":2.577},"last":1.542}]},{"osd":4,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.419,"15min":1.4039999999999999},"min":{"1min":0.64800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.113,"5min":2.1549999999999998,"15min":2.6779999999999999},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.345,"5min":1.379,"15min":1.3779999999999999},"min":{"1min":0.65600000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.756,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":2.161}]},{"osd":5,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.5669999999999999,"15min":1.5780000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":3.3519999999999999},"last":2.105},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.534,"15min":1.498},"min":{"1min":0.62,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.0880000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":2.2400000000000002}]},{"osd":6,"last update":"Mon Oct 16 11:26:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.488,"15min":1.4970000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.907,"5min":2.2469999999999999,"15min":2.7250000000000001},"last":1.792},{"interface":"front","average":{"1min":1.633,"5min":1.6639999999999999,"15min":1.6859999999999999},"min":{"1min":0.749,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.323,"5min":2.323,"15min":3.1440000000000001},"last":1.98}]}]},{"osd":6,"up_from":13,"seq":55834576022,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52616,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319224,"statfs":{"total":96636764160,"available":96582885376,"internally_reserved":0,"allocated":3469312,"data_stored":1699888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1779999999999999,"15min":1.202},"min":{"1min":0.69399999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.538,"5min":6.298,"15min":6.298},"last":1.177},{"interface":"front","average":{"1min":1.159,"5min":1.2989999999999999,"15min":1.27},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.597,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":1.3939999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.198,"15min":1.204},"min":{"1min":0.72299999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7969999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3740000000000001,"15min":1.361},"min":{"1min":0.749,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.714,"5min":12.468,"15min":12.468},"last":1.337}]},{"osd":2,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.2549999999999999,"15min":1.254},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.639,"5min":11.586,"15min":11.586},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.385,"15min":1.371},"min":{"1min":0.87,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7170000000000001,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.367}]},{"osd":3,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.4139999999999999,"15min":1.337},"min":{"1min":0.68100000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.766,"5min":12.801,"15min":12.801},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.254,"15min":1.2589999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.7969999999999999,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":1.099}]},{"osd":4,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4370000000000001,"15min":1.341},"min":{"1min":1.0429999999999999,"5min":0.65400000000000003,"15min":0.49099999999999999},"max":{"1min":1.931,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.51},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.6519999999999999,"15min":1.5569999999999999},"min":{"1min":0.70399999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.97,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.48}]},{"osd":5,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.55,"15min":1.3979999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.9319999999999999,"5min":18.247,"15min":18.247},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.6279999999999999,"15min":1.569},"min":{"1min":0.76700000000000002,"5min":0.76200000000000001,"15min":0.60899999999999999},"max":{"1min":2.085,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.45}]},{"osd":7,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.6990000000000001,"15min":1.5580000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.59199999999999997},"max":{"1min":1.9870000000000001,"5min":12.589,"15min":12.589},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.6990000000000001,"15min":1.597},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0249999999999999,"5min":12.725,"15min":12.725},"last":1.306}]}]},{"osd":5,"up_from":13,"seq":55834576022,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52620,"kb_used_data":3392,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319220,"statfs":{"total":96636764160,"available":96582881280,"internally_reserved":0,"allocated":3473408,"data_stored":1698119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.1519999999999999,"15min":1.234},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.585,"5min":1.585,"15min":2.661},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.248,"15min":1.3169999999999999},"min":{"1min":0.75600000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.583,"5min":1.7909999999999999,"15min":1.7949999999999999},"last":1.6799999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1279999999999999,"15min":1.175},"min":{"1min":0.82199999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.748,"5min":1.748,"15min":1.992},"last":1.014},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.1919999999999999,"15min":1.228},"min":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":2.9399999999999999},"last":1.5369999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2470000000000001,"15min":1.2390000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2869999999999999,"15min":1.3009999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7829999999999999,"5min":1.8660000000000001,"15min":2.1640000000000001},"last":1.198}]},{"osd":3,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.29,"15min":1.3169999999999999},"min":{"1min":0.82199999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.121,"5min":2.121,"15min":3.012},"last":1.738},{"interface":"front","average":{"1min":1.429,"5min":1.387,"15min":1.411},"min":{"1min":1.0880000000000001,"5min":0.92700000000000005,"15min":0.89500000000000002},"max":{"1min":1.9299999999999999,"5min":2.0129999999999999,"15min":2.2250000000000001},"last":0.876}]},{"osd":4,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.3280000000000001,"15min":1.345},"min":{"1min":0.80300000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.8660000000000001},"last":1.383},{"interface":"front","average":{"1min":1.605,"5min":1.5429999999999999,"15min":1.5620000000000001},"min":{"1min":1.125,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.903},"last":1.792}]},{"osd":6,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.4059999999999999,"15min":1.417},"min":{"1min":0.95099999999999996,"5min":0.68100000000000005,"15min":0.51000000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0979999999999999},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.53,"5min":1.4379999999999999,"15min":1.4770000000000001},"min":{"1min":1.004,"5min":0.90700000000000003,"15min":0.89200000000000002},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":3.073},"last":1.7030000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.552,"15min":1.548},"min":{"1min":1.228,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":3.1160000000000001},"last":1.593},{"interface":"front","average":{"1min":1.702,"5min":1.619,"15min":1.639},"min":{"1min":1.2809999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.161,"5min":2.1930000000000001,"15min":2.2930000000000001},"last":1.4259999999999999}]}]},{"osd":4,"up_from":13,"seq":55834576023,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88772,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94283068,"statfs":{"total":96636764160,"available":96545861632,"internally_reserved":0,"allocated":3596288,"data_stored":1825421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":87292358},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.151,"15min":1.1739999999999999},"min":{"1min":0.73799999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.615,"5min":2.899,"15min":2.899},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.296,"5min":1.236,"15min":1.254},"min":{"1min":0.77500000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.784,"5min":3.016,"15min":3.016},"last":0.76600000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.28,"15min":1.298},"min":{"1min":0.83999999999999997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6799999999999999,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.647},{"interface":"front","average":{"1min":1.089,"5min":1.228,"15min":1.25},"min":{"1min":0.84499999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.4299999999999999,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.2010000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8300000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4370000000000001,"15min":1.4199999999999999},"min":{"1min":0.90500000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.748,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.1299999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.351,"15min":1.349},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.7669999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.452,"5min":1.4930000000000001,"15min":1.4610000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.9930000000000001,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.837}]},{"osd":5,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":1.542,"15min":1.4790000000000001},"min":{"1min":1.0549999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.915,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.861},{"interface":"front","average":{"1min":1.444,"5min":1.4650000000000001,"15min":1.4370000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.9410000000000001,"5min":3.7989999999999999,"15min":3.7989999999999999},"last":1.333}]},{"osd":6,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.6020000000000001,"15min":1.5349999999999999},"min":{"1min":0.85699999999999998,"5min":0.82099999999999995,"15min":0.78000000000000003},"max":{"1min":2.0779999999999998,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":2.016},{"interface":"front","average":{"1min":1.724,"5min":1.679,"15min":1.6459999999999999},"min":{"1min":1.268,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.944,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.9670000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":1.647,"15min":1.607},"min":{"1min":0.73699999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.1949999999999998,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":1.425},{"interface":"front","average":{"1min":1.702,"5min":1.7190000000000001,"15min":1.6799999999999999},"min":{"1min":1.026,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.1419999999999999,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.913}]}]},{"osd":3,"up_from":13,"seq":55834576022,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52604,"kb_used_data":3376,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319236,"statfs":{"total":96636764160,"available":96582897664,"internally_reserved":0,"allocated":3457024,"data_stored":1693751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81699999999999995,"15min":0.88200000000000001},"min":{"1min":0.33800000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.901},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.89200000000000002,"15min":1.0249999999999999},"min":{"1min":0.443,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1919999999999999,"5min":1.581,"15min":1.6679999999999999},"last":1.3859999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.88800000000000001,"15min":0.95899999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3420000000000001,"5min":1.508,"15min":1.508},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.94899999999999995,"15min":1.087},"min":{"1min":0.51300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4770000000000001,"5min":1.601,"15min":1.7190000000000001},"last":1.1120000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.99199999999999999,"15min":1.087},"min":{"1min":0.56299999999999994,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.524,"5min":1.5660000000000001,"15min":1.694},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.079,"15min":1.1870000000000001},"min":{"1min":0.48499999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.597,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.341}]},{"osd":4,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.23,"15min":1.3240000000000001},"min":{"1min":0.72499999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6839999999999999,"5min":1.821,"15min":1.873},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2949999999999999,"15min":1.3440000000000001},"min":{"1min":0.80900000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.681,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.1719999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.3009999999999999,"15min":1.385},"min":{"1min":0.69299999999999995,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.7749999999999999,"5min":2.798,"15min":2.798},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3149999999999999,"15min":1.4179999999999999},"min":{"1min":0.74199999999999999,"5min":0.63,"15min":0.63},"max":{"1min":1.772,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.2609999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4330000000000001,"15min":1.51},"min":{"1min":0.93000000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.9970000000000001,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.413,"15min":1.506},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9350000000000001,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.45}]},{"osd":7,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.46,"15min":1.544},"min":{"1min":0.96799999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.8380000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.478},{"interface":"front","average":{"1min":1.48,"5min":1.5329999999999999,"15min":1.6459999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.952,"5min":2.3799999999999999,"15min":2.4649999999999999},"last":1.5349999999999999}]}]},{"osd":2,"up_from":13,"seq":55834576021,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56768,"kb_used_data":3380,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315072,"statfs":{"total":96636764160,"available":96578633728,"internally_reserved":0,"allocated":3461120,"data_stored":1693293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.85799999999999998,"15min":0.89900000000000002},"min":{"1min":0.39800000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.3200000000000001,"5min":1.532,"15min":1.6399999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.92100000000000004,"15min":1.0369999999999999},"min":{"1min":0.53400000000000003,"5min":0.40500000000000003,"15min":0.32600000000000001},"max":{"1min":1.258,"5min":1.6359999999999999,"15min":2.1429999999999998},"last":0.86299999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.046,"15min":1.075},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39200000000000002},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.02,"15min":1.1120000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.2030000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.96099999999999997,"15min":1.0469999999999999},"min":{"1min":0.52300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.452,"5min":1.522,"15min":1.8560000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1639999999999999,"15min":1.242},"min":{"1min":0.58199999999999996,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.0459999999999998,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":1.3240000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.306,"15min":1.3819999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.8079999999999998,"5min":3.8079999999999998,"15min":3.8079999999999998},"last":1.54},{"interface":"front","average":{"1min":1.236,"5min":1.2490000000000001,"15min":1.3140000000000001},"min":{"1min":0.79700000000000004,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.256}]},{"osd":5,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.397,"15min":1.431},"min":{"1min":0.77900000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.391},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3839999999999999,"15min":1.4470000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0499999999999998,"5min":2.323,"15min":2.6259999999999999},"last":1.6180000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3560000000000001,"15min":1.381},"min":{"1min":0.85099999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.4830000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.464,"15min":1.5089999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":1.5840000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.4910000000000001,"15min":1.5489999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.742,"5min":3.742,"15min":3.742},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.4339999999999999,"15min":1.4810000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":1.4379999999999999}]}]},{"osd":1,"up_from":13,"seq":55834576021,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48832,"kb_used_data":3764,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323008,"statfs":{"total":96636764160,"available":96586760192,"internally_reserved":0,"allocated":3854336,"data_stored":2086441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.89300000000000002,"15min":0.91000000000000003},"min":{"1min":0.40699999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4039999999999999,"5min":5.117,"15min":5.117},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.89300000000000002,"15min":1.0029999999999999},"min":{"1min":0.45500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.325,"5min":1.5509999999999999,"15min":1.667},"last":0.79200000000000004}]},{"osd":2,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.91300000000000003,"15min":0.98899999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.651,"5min":2.093,"15min":2.093},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.149,"15min":1.242},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2400000000000002,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":1.1699999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.125,"15min":1.179},"min":{"1min":0.56999999999999995,"5min":0.439,"15min":0.439},"max":{"1min":1.8879999999999999,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.05,"15min":1.1399999999999999},"min":{"1min":0.52700000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5860000000000001,"5min":1.798,"15min":1.802},"last":1.1830000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.2749999999999999,"15min":1.2849999999999999},"min":{"1min":0.70299999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.978,"5min":5.29,"15min":5.29},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.403,"15min":1.409},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.0129999999999999,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":1.1220000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.3400000000000001,"15min":1.3979999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.829,"5min":6.0979999999999999,"15min":6.0979999999999999},"last":1.409},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.3420000000000001,"15min":1.3899999999999999},"min":{"1min":0.74099999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7410000000000001,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.2509999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.429,"15min":1.482},"min":{"1min":0.82399999999999995,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8979999999999999,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":1.552},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.393,"15min":1.4630000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.948,"5min":3.8799999999999999,"15min":3.8799999999999999},"last":1.3660000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.518,"15min":1.5509999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.157,"5min":4.46,"15min":4.46},"last":1.298},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.5349999999999999,"15min":1.589},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.0259999999999998,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":1.4910000000000001}]}]},{"osd":0,"up_from":13,"seq":55834576022,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49160,"kb_used_data":4092,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322680,"statfs":{"total":96636764160,"available":96586424320,"internally_reserved":0,"allocated":4190208,"data_stored":2415229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85499999999999998,"15min":0.88500000000000001},"min":{"1min":0.44900000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2350000000000001,"5min":1.502,"15min":1.619},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.98699999999999999,"15min":1.0740000000000001},"min":{"1min":0.40000000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.518,"5min":1.621,"15min":1.76},"last":1.3300000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0640000000000001,"15min":1.0820000000000001},"min":{"1min":0.51300000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.569,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.996,"5min":1.036,"15min":1.1180000000000001},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.7649999999999999,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":1.014}]},{"osd":3,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0089999999999999,"15min":1.0640000000000001},"min":{"1min":0.621,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.5189999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.131,"15min":1.2170000000000001},"min":{"1min":0.59099999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.45,"5min":1.833,"15min":1.833},"last":1.2669999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.3149999999999999,"15min":1.377},"min":{"1min":0.93000000000000005,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.538,"5min":3.2799999999999998,"15min":3.2799999999999998},"last":1.546},{"interface":"front","average":{"1min":1.286,"5min":1.2869999999999999,"15min":1.3320000000000001},"min":{"1min":0.80000000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.752,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.23}]},{"osd":5,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.3109999999999999,"15min":1.3660000000000001},"min":{"1min":0.76000000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.605,"5min":2.0209999999999999,"15min":2.6669999999999998},"last":1.099},{"interface":"front","average":{"1min":1.478,"5min":1.385,"15min":1.4339999999999999},"min":{"1min":0.877,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.4359999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4570000000000001,"15min":1.4930000000000001},"min":{"1min":0.78900000000000003,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.1309999999999998,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.363,"5min":1.4019999999999999,"15min":1.468},"min":{"1min":0.90200000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7310000000000001,"5min":2.073,"15min":2.262},"last":1.613}]},{"osd":7,"last update":"Mon Oct 16 11:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5309999999999999,"15min":1.5549999999999999},"min":{"1min":1.0429999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7869999999999999,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.419,"15min":1.486},"min":{"1min":0.98399999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7869999999999999,"5min":2.1240000000000001,"15min":2.8199999999999998},"last":1.653}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:27:55.140 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:27:55.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.351+0000 7f6903c61640 1 Processor -- start 2023-10-16T11:27:55.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.351+0000 7f6903c61640 1 -- start start 2023-10-16T11:27:55.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.353+0000 7f6903c61640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f68fc15aaa0 0x7f68fc15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:55.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.353+0000 7f6903c61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68fc15ae80 0x7f68fc0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:55.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.353+0000 7f6903c61640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68fc15ed60 con 0x7f68fc0ccaa0 2023-10-16T11:27:55.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.353+0000 7f6903c61640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68fc06b810 con 0x7f68fc15aaa0 2023-10-16T11:27:55.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.353+0000 7f6903c61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0707e0 con 0x7f68fc15ae80 2023-10-16T11:27:55.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.353+0000 7f69019fd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f68fc15aaa0 0x7f68fc15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:55.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.353+0000 7f69019fd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f68fc15aaa0 0x7f68fc15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37012/0 (socket says 172.21.15.84:37012) 2023-10-16T11:27:55.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.353+0000 7f69019fd640 1 -- 172.21.15.84:0/2397730286 learned_addr learned my addr 172.21.15.84:0/2397730286 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:55.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.354+0000 7f69019fd640 1 -- 172.21.15.84:0/2397730286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68fc15ae80 msgr2=0x7f68fc0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:55.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.354+0000 7f69019fd640 1 --2- 172.21.15.84:0/2397730286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68fc15ae80 0x7f68fc0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:55.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.354+0000 7f69019fd640 1 -- 172.21.15.84:0/2397730286 >> v1:172.21.15.84:6789/0 conn(0x7f68fc0ccaa0 legacy=0x7f68fc070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:55.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.354+0000 7f69019fd640 1 -- 172.21.15.84:0/2397730286 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc15ef90 con 0x7f68fc15aaa0 2023-10-16T11:27:55.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.355+0000 7f69019fd640 1 --2- 172.21.15.84:0/2397730286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68fc15aaa0 0x7f68fc15e820 secure :-1 s=READY pgs=3539 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009870 tx=0x7f68f802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca155eca59b265d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:55.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.355+0000 7f69009fb640 1 -- 172.21.15.84:0/2397730286 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8030840 con 0x7f68fc15aaa0 2023-10-16T11:27:55.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.355+0000 7f69009fb640 1 -- 172.21.15.84:0/2397730286 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f80309a0 con 0x7f68fc15aaa0 2023-10-16T11:27:55.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.355+0000 7f69009fb640 1 -- 172.21.15.84:0/2397730286 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8030d30 con 0x7f68fc15aaa0 2023-10-16T11:27:55.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.356+0000 7f6903c61640 1 -- 172.21.15.84:0/2397730286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68fc15aaa0 msgr2=0x7f68fc15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:55.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.356+0000 7f6903c61640 1 --2- 172.21.15.84:0/2397730286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68fc15aaa0 0x7f68fc15e820 secure :-1 s=READY pgs=3539 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009870 tx=0x7f68f802b980 comp rx=0 tx=0).stop 2023-10-16T11:27:55.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.356+0000 7f6903c61640 1 -- 172.21.15.84:0/2397730286 shutdown_connections 2023-10-16T11:27:55.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.356+0000 7f6903c61640 1 --2- 172.21.15.84:0/2397730286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68fc15ae80 0x7f68fc0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:55.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.356+0000 7f6903c61640 1 --2- 172.21.15.84:0/2397730286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68fc15aaa0 0x7f68fc15e820 unknown :-1 s=CLOSED pgs=3539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:55.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.356+0000 7f6903c61640 1 -- 172.21.15.84:0/2397730286 >> 172.21.15.84:0/2397730286 conn(0x7f68fc0cb410 msgr2=0x7f68fc0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:55.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.356+0000 7f6903c61640 1 -- 172.21.15.84:0/2397730286 shutdown_connections 2023-10-16T11:27:55.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.356+0000 7f6903c61640 1 -- 172.21.15.84:0/2397730286 wait complete. 2023-10-16T11:27:55.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.358+0000 7f6903c61640 1 Processor -- start 2023-10-16T11:27:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.358+0000 7f6903c61640 1 -- start start 2023-10-16T11:27:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.359+0000 7f6903c61640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68fc0ccaa0 0x7f68fc1f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.359+0000 7f6903c61640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68fc15aaa0 0x7f68fc0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.359+0000 7f6903c61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68fc15ae80 0x7f68fc0acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.359+0000 7f6903c61640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0c2f70 con 0x7f68fc15aaa0 2023-10-16T11:27:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.359+0000 7f6903c61640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0c2640 con 0x7f68fc0ccaa0 2023-10-16T11:27:55.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.359+0000 7f6903c61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0c2920 con 0x7f68fc15ae80 2023-10-16T11:27:55.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.359+0000 7f69011fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68fc15aaa0 0x7f68fc0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:55.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.359+0000 7f69011fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68fc15aaa0 0x7f68fc0ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37018/0 (socket says 172.21.15.84:37018) 2023-10-16T11:27:55.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.359+0000 7f69011fc640 1 -- 172.21.15.84:0/1672049080 learned_addr learned my addr 172.21.15.84:0/1672049080 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:55.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.359+0000 7f69019fd640 1 --2- 172.21.15.84:0/1672049080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68fc0ccaa0 0x7f68fc1f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:55.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.359+0000 7f69021fe640 1 --2- 172.21.15.84:0/1672049080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68fc15ae80 0x7f68fc0acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:55.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.359+0000 7f69011fc640 1 -- 172.21.15.84:0/1672049080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68fc15ae80 msgr2=0x7f68fc0acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:55.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.359+0000 7f69011fc640 1 --2- 172.21.15.84:0/1672049080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68fc15ae80 0x7f68fc0acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:55.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.359+0000 7f69011fc640 1 -- 172.21.15.84:0/1672049080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68fc0ccaa0 msgr2=0x7f68fc1f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:55.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.359+0000 7f69011fc640 1 --2- 172.21.15.84:0/1672049080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68fc0ccaa0 0x7f68fc1f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:55.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.359+0000 7f69011fc640 1 -- 172.21.15.84:0/1672049080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f80094d0 con 0x7f68fc15aaa0 2023-10-16T11:27:55.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.360+0000 7f69011fc640 1 --2- 172.21.15.84:0/1672049080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68fc15aaa0 0x7f68fc0ac890 secure :-1 s=READY pgs=3540 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00b4a0 tx=0x7f68ec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:55.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.360+0000 7f68f27fc640 1 -- 172.21.15.84:0/1672049080 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec010610 con 0x7f68fc15aaa0 2023-10-16T11:27:55.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.360+0000 7f68f27fc640 1 -- 172.21.15.84:0/1672049080 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec010770 con 0x7f68fc15aaa0 2023-10-16T11:27:55.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.360+0000 7f68f27fc640 1 -- 172.21.15.84:0/1672049080 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec010b20 con 0x7f68fc15aaa0 2023-10-16T11:27:55.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.360+0000 7f6903c61640 1 -- 172.21.15.84:0/1672049080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68fc0c2c00 con 0x7f68fc15aaa0 2023-10-16T11:27:55.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.360+0000 7f6903c61640 1 -- 172.21.15.84:0/1672049080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68fc146a10 con 0x7f68fc15aaa0 2023-10-16T11:27:55.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.361+0000 7f6903c61640 1 -- 172.21.15.84:0/1672049080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68fc0c3120 con 0x7f68fc15aaa0 2023-10-16T11:27:55.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.362+0000 7f68f27fc640 1 -- 172.21.15.84:0/1672049080 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f68ec010c80 con 0x7f68fc15aaa0 2023-10-16T11:27:55.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.363+0000 7f68f27fc640 1 --2- 172.21.15.84:0/1672049080 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f68e006afc0 0x7f68e006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:55.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.363+0000 7f68f27fc640 1 -- 172.21.15.84:0/1672049080 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f68ec089330 con 0x7f68fc15aaa0 2023-10-16T11:27:55.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.363+0000 7f69019fd640 1 --2- 172.21.15.84:0/1672049080 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f68e006afc0 0x7f68e006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:55.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.364+0000 7f69019fd640 1 --2- 172.21.15.84:0/1672049080 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f68e006afc0 0x7f68e006d480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f68f8036000 tx=0x7f68f803b040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:55.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.369+0000 7f68f27fc640 1 -- 172.21.15.84:0/1672049080 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f68fc0c3120 con 0x7f68fc15aaa0 2023-10-16T11:27:55.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.563+0000 7f6903c61640 1 -- 172.21.15.84:0/1672049080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68fc04e160 con 0x7f68fc15aaa0 2023-10-16T11:27:55.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.564+0000 7f68f27fc640 1 -- 172.21.15.84:0/1672049080 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v275) v1 ==== 74+0+14013 (secure 0 0 0) 0x7f68ec054300 con 0x7f68fc15aaa0 2023-10-16T11:27:55.566 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:27:55.567 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":275,"fsid":"63fb4d68-f881-4694-b880-d10369105c0e","created":"2023-10-16T09:48:42.748674+0000","modified":"2023-10-16T11:27:50.255596+0000","last_up_change":"2023-10-16T09:48:54.329282+0000","last_in_change":"2023-10-16T09:48:46.163691+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:48:55.657982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e56c068-81b6-4d45-ae23-abd5ca91bf25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6801","nonce":2077434097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6803","nonce":2077434097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6807","nonce":2077434097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2077434097},{"type":"v1","addr":"172.21.15.84:6805","nonce":2077434097}]},"public_addr":"172.21.15.84:6801/2077434097","cluster_addr":"172.21.15.84:6803/2077434097","heartbeat_back_addr":"172.21.15.84:6807/2077434097","heartbeat_front_addr":"172.21.15.84:6805/2077434097","state":["exists","up"]},{"osd":1,"uuid":"306aa004-b7d8-4c60-b346-925524c4d79f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6809","nonce":805077222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6811","nonce":805077222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6815","nonce":805077222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":805077222},{"type":"v1","addr":"172.21.15.84:6813","nonce":805077222}]},"public_addr":"172.21.15.84:6809/805077222","cluster_addr":"172.21.15.84:6811/805077222","heartbeat_back_addr":"172.21.15.84:6815/805077222","heartbeat_front_addr":"172.21.15.84:6813/805077222","state":["exists","up"]},{"osd":2,"uuid":"807fffa0-14de-4fac-bfc9-ef94715af8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6821","nonce":338331808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6825","nonce":338331808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6831","nonce":338331808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":338331808},{"type":"v1","addr":"172.21.15.84:6829","nonce":338331808}]},"public_addr":"172.21.15.84:6821/338331808","cluster_addr":"172.21.15.84:6825/338331808","heartbeat_back_addr":"172.21.15.84:6831/338331808","heartbeat_front_addr":"172.21.15.84:6829/338331808","state":["exists","up"]},{"osd":3,"uuid":"300bbe2b-e619-4d94-963f-1ba6e567f666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6817","nonce":1933430277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6820","nonce":1933430277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6827","nonce":1933430277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1933430277},{"type":"v1","addr":"172.21.15.84:6824","nonce":1933430277}]},"public_addr":"172.21.15.84:6817/1933430277","cluster_addr":"172.21.15.84:6820/1933430277","heartbeat_back_addr":"172.21.15.84:6827/1933430277","heartbeat_front_addr":"172.21.15.84:6824/1933430277","state":["exists","up"]},{"osd":4,"uuid":"9067ece2-38cc-475c-981a-fe25c90cf9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6809","nonce":2704622262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6811","nonce":2704622262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6815","nonce":2704622262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2704622262},{"type":"v1","addr":"172.21.15.157:6813","nonce":2704622262}]},"public_addr":"172.21.15.157:6809/2704622262","cluster_addr":"172.21.15.157:6811/2704622262","heartbeat_back_addr":"172.21.15.157:6815/2704622262","heartbeat_front_addr":"172.21.15.157:6813/2704622262","state":["exists","up"]},{"osd":5,"uuid":"796e804a-e843-4356-8e30-252ce0dcc18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6801","nonce":3654970329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6803","nonce":3654970329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6807","nonce":3654970329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3654970329},{"type":"v1","addr":"172.21.15.157:6805","nonce":3654970329}]},"public_addr":"172.21.15.157:6801/3654970329","cluster_addr":"172.21.15.157:6803/3654970329","heartbeat_back_addr":"172.21.15.157:6807/3654970329","heartbeat_front_addr":"172.21.15.157:6805/3654970329","state":["exists","up"]},{"osd":6,"uuid":"38f95207-b5e2-4d7f-8d92-002ab5421e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6817","nonce":899624983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6820","nonce":899624983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6828","nonce":899624983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":899624983},{"type":"v1","addr":"172.21.15.157:6824","nonce":899624983}]},"public_addr":"172.21.15.157:6817/899624983","cluster_addr":"172.21.15.157:6820/899624983","heartbeat_back_addr":"172.21.15.157:6828/899624983","heartbeat_front_addr":"172.21.15.157:6824/899624983","state":["exists","up"]},{"osd":7,"uuid":"4f0b5649-ac0f-4deb-bb8d-cf3107fe891b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":268,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6819","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6821","nonce":1752891116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6823","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6825","nonce":1752891116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6831","nonce":1752891116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6827","nonce":1752891116},{"type":"v1","addr":"172.21.15.157:6829","nonce":1752891116}]},"public_addr":"172.21.15.157:6821/1752891116","cluster_addr":"172.21.15.157:6825/1752891116","heartbeat_back_addr":"172.21.15.157:6831/1752891116","heartbeat_front_addr":"172.21.15.157:6829/1752891116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.433230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.434057+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.556833+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.565349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.514285+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.249445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.544329+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:48:51.560865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/2893482429":"2023-10-17T11:27:43.469263+0000","172.21.15.84:0/1321046441":"2023-10-17T11:27:43.469263+0000","172.21.15.84:0/822007176":"2023-10-17T11:27:43.469263+0000","172.21.15.84:0/3876279166":"2023-10-17T11:27:43.469263+0000","172.21.15.84:6833/129259578":"2023-10-17T11:27:43.469263+0000","172.21.15.84:6836/2806813701":"2023-10-17T11:27:47.051839+0000","172.21.15.84:0/2238142758":"2023-10-17T11:27:43.469263+0000","172.21.15.84:6837/2806813701":"2023-10-17T11:27:47.051839+0000","172.21.15.84:0/1788498161":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/3634760901":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/3509607392":"2023-10-17T11:27:43.469263+0000","172.21.15.84:0/3842621119":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/40770371":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/1147031855":"2023-10-17T11:27:37.140228+0000","172.21.15.84:6832/129259578":"2023-10-17T11:27:43.469263+0000","172.21.15.84:6833/731888425":"2023-10-17T11:27:37.140228+0000","172.21.15.84:6832/731888425":"2023-10-17T11:27:37.140228+0000","172.21.15.84:0/2353059609":"2023-10-17T11:27:37.140228+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:27:55.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.567+0000 7f6903c61640 1 -- 172.21.15.84:0/1672049080 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f68e006afc0 msgr2=0x7f68e006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:55.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.567+0000 7f6903c61640 1 --2- 172.21.15.84:0/1672049080 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f68e006afc0 0x7f68e006d480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f68f8036000 tx=0x7f68f803b040 comp rx=0 tx=0).stop 2023-10-16T11:27:55.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.567+0000 7f6903c61640 1 -- 172.21.15.84:0/1672049080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68fc15aaa0 msgr2=0x7f68fc0ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:55.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.567+0000 7f6903c61640 1 --2- 172.21.15.84:0/1672049080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68fc15aaa0 0x7f68fc0ac890 secure :-1 s=READY pgs=3540 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00b4a0 tx=0x7f68ec00b960 comp rx=0 tx=0).stop 2023-10-16T11:27:55.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.567+0000 7f6903c61640 1 -- 172.21.15.84:0/1672049080 shutdown_connections 2023-10-16T11:27:55.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.567+0000 7f6903c61640 1 --2- 172.21.15.84:0/1672049080 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f68e006afc0 0x7f68e006d480 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:55.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.567+0000 7f6903c61640 1 --2- 172.21.15.84:0/1672049080 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68fc15ae80 0x7f68fc0acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:55.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.567+0000 7f6903c61640 1 --2- 172.21.15.84:0/1672049080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68fc15aaa0 0x7f68fc0ac890 unknown :-1 s=CLOSED pgs=3540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:55.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.567+0000 7f6903c61640 1 --2- 172.21.15.84:0/1672049080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68fc0ccaa0 0x7f68fc1f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:55.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.567+0000 7f6903c61640 1 -- 172.21.15.84:0/1672049080 >> 172.21.15.84:0/1672049080 conn(0x7f68fc0cb410 msgr2=0x7f68fc0cefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:55.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.567+0000 7f6903c61640 1 -- 172.21.15.84:0/1672049080 shutdown_connections 2023-10-16T11:27:55.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:55.567+0000 7f6903c61640 1 -- 172.21.15.84:0/1672049080 wait complete. 2023-10-16T11:27:56.588 INFO:tasks.ceph:Scrubbing osd.0 2023-10-16T11:27:56.589 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-16T11:27:56.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.811+0000 7fddff81f640 1 Processor -- start 2023-10-16T11:27:56.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.812+0000 7fddff81f640 1 -- start start 2023-10-16T11:27:56.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.813+0000 7fddff81f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fddf815ac10 0x7fddf806c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:56.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.813+0000 7fddff81f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddf80b0360 0x7fddf80b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:56.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.813+0000 7fddff81f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fddf80ca510 0x7fddf80ca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:56.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.813+0000 7fddff81f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf80b0d00 con 0x7fddf815ac10 2023-10-16T11:27:56.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.813+0000 7fddff81f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf815b310 con 0x7fddf80ca510 2023-10-16T11:27:56.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.813+0000 7fddff81f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf806f640 con 0x7fddf80b0360 2023-10-16T11:27:56.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.813+0000 7fddfd5bb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fddf815ac10 0x7fddf806c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:56.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.813+0000 7fddfd5bb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fddf815ac10 0x7fddf806c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52314/0 (socket says 172.21.15.84:52314) 2023-10-16T11:27:56.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.813+0000 7fddfd5bb640 1 -- 172.21.15.84:0/3390975837 learned_addr learned my addr 172.21.15.84:0/3390975837 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:56.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.813+0000 7fddfddbc640 1 --2- 172.21.15.84:0/3390975837 >> v2:172.21.15.157:3300/0 conn(0x7fddf80ca510 0x7fddf80ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:56.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.813+0000 7fddfcdba640 1 --2- 172.21.15.84:0/3390975837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddf80b0360 0x7fddf80b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:56.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.814+0000 7fddfd5bb640 1 -- 172.21.15.84:0/3390975837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddf80b0360 msgr2=0x7fddf80b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:56.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.814+0000 7fddfd5bb640 1 --2- 172.21.15.84:0/3390975837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddf80b0360 0x7fddf80b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:56.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.814+0000 7fddfd5bb640 1 -- 172.21.15.84:0/3390975837 >> v2:172.21.15.157:3300/0 conn(0x7fddf80ca510 msgr2=0x7fddf80ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:56.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.814+0000 7fddfd5bb640 1 --2- 172.21.15.84:0/3390975837 >> v2:172.21.15.157:3300/0 conn(0x7fddf80ca510 0x7fddf80ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:56.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.814+0000 7fddfd5bb640 1 -- 172.21.15.84:0/3390975837 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf815b5b0 con 0x7fddf815ac10 2023-10-16T11:27:56.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.814+0000 7fddfddbc640 1 --2- 172.21.15.84:0/3390975837 >> v2:172.21.15.157:3300/0 conn(0x7fddf80ca510 0x7fddf80ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:27:56.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.815+0000 7fddfd5bb640 1 --2- 172.21.15.84:0/3390975837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddf815ac10 0x7fddf806c730 secure :-1 s=READY pgs=3541 cs=0 l=1 rev1=1 crypto rx=0x7fddf80cc3c0 tx=0x7fdde802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129e39bfd94a4640 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:56.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.815+0000 7fddf7fff640 1 -- 172.21.15.84:0/3390975837 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde802c9f0 con 0x7fddf815ac10 2023-10-16T11:27:56.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.815+0000 7fddf7fff640 1 -- 172.21.15.84:0/3390975837 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde802cb50 con 0x7fddf815ac10 2023-10-16T11:27:56.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.815+0000 7fddf7fff640 1 -- 172.21.15.84:0/3390975837 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde80343f0 con 0x7fddf815ac10 2023-10-16T11:27:56.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.815+0000 7fddff81f640 1 -- 172.21.15.84:0/3390975837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddf815ac10 msgr2=0x7fddf806c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:56.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.816+0000 7fddff81f640 1 --2- 172.21.15.84:0/3390975837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddf815ac10 0x7fddf806c730 secure :-1 s=READY pgs=3541 cs=0 l=1 rev1=1 crypto rx=0x7fddf80cc3c0 tx=0x7fdde802be50 comp rx=0 tx=0).stop 2023-10-16T11:27:56.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.816+0000 7fddff81f640 1 -- 172.21.15.84:0/3390975837 shutdown_connections 2023-10-16T11:27:56.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.816+0000 7fddff81f640 1 --2- 172.21.15.84:0/3390975837 >> v2:172.21.15.157:3300/0 conn(0x7fddf80ca510 0x7fddf80ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:56.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.816+0000 7fddff81f640 1 --2- 172.21.15.84:0/3390975837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddf80b0360 0x7fddf80b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:56.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.816+0000 7fddff81f640 1 --2- 172.21.15.84:0/3390975837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddf815ac10 0x7fddf806c730 unknown :-1 s=CLOSED pgs=3541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:56.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.816+0000 7fddff81f640 1 -- 172.21.15.84:0/3390975837 >> 172.21.15.84:0/3390975837 conn(0x7fddf80cb360 msgr2=0x7fddf80cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:56.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.816+0000 7fddff81f640 1 -- 172.21.15.84:0/3390975837 shutdown_connections 2023-10-16T11:27:56.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.816+0000 7fddff81f640 1 -- 172.21.15.84:0/3390975837 wait complete. 2023-10-16T11:27:56.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.817+0000 7fddff81f640 1 Processor -- start 2023-10-16T11:27:56.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.818+0000 7fddff81f640 1 -- start start 2023-10-16T11:27:56.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.818+0000 7fddff81f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf80b0360 0x7fddf80c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:56.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.818+0000 7fddff81f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddf80ca510 0x7fddf80ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:56.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.818+0000 7fddff81f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddf815ac10 0x7fddf81f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:56.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.818+0000 7fddff81f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf80c5fa0 con 0x7fddf80ca510 2023-10-16T11:27:56.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.818+0000 7fddff81f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf80c59b0 con 0x7fddf80b0360 2023-10-16T11:27:56.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.818+0000 7fddff81f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf81f4270 con 0x7fddf815ac10 2023-10-16T11:27:56.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.819+0000 7fddfcdba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddf80ca510 0x7fddf80ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:56.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.819+0000 7fddfcdba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddf80ca510 0x7fddf80ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52316/0 (socket says 172.21.15.84:52316) 2023-10-16T11:27:56.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.819+0000 7fddfcdba640 1 -- 172.21.15.84:0/1251382144 learned_addr learned my addr 172.21.15.84:0/1251382144 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:56.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.819+0000 7fddfd5bb640 1 --2- 172.21.15.84:0/1251382144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf80b0360 0x7fddf80c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:56.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.819+0000 7fddfddbc640 1 --2- 172.21.15.84:0/1251382144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddf815ac10 0x7fddf81f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:56.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.819+0000 7fddfcdba640 1 -- 172.21.15.84:0/1251382144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddf815ac10 msgr2=0x7fddf81f3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:56.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.819+0000 7fddfcdba640 1 --2- 172.21.15.84:0/1251382144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fddf815ac10 0x7fddf81f3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:56.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.819+0000 7fddfcdba640 1 -- 172.21.15.84:0/1251382144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf80b0360 msgr2=0x7fddf80c63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:56.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.819+0000 7fddfcdba640 1 --2- 172.21.15.84:0/1251382144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf80b0360 0x7fddf80c63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:56.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.819+0000 7fddfcdba640 1 -- 172.21.15.84:0/1251382144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde8009b30 con 0x7fddf80ca510 2023-10-16T11:27:56.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.819+0000 7fddfd5bb640 1 --2- 172.21.15.84:0/1251382144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddf80b0360 0x7fddf80c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:56.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.819+0000 7fddfcdba640 1 --2- 172.21.15.84:0/1251382144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddf80ca510 0x7fddf80ceac0 secure :-1 s=READY pgs=3542 cs=0 l=1 rev1=1 crypto rx=0x7fddec00d800 tx=0x7fddec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:56.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.820+0000 7fddf5ffb640 1 -- 172.21.15.84:0/1251382144 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec0096a0 con 0x7fddf80ca510 2023-10-16T11:27:56.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.820+0000 7fddf5ffb640 1 -- 172.21.15.84:0/1251382144 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec009800 con 0x7fddf80ca510 2023-10-16T11:27:56.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.820+0000 7fddf5ffb640 1 -- 172.21.15.84:0/1251382144 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec009990 con 0x7fddf80ca510 2023-10-16T11:27:56.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.820+0000 7fddff81f640 1 -- 172.21.15.84:0/1251382144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf81f4550 con 0x7fddf80ca510 2023-10-16T11:27:56.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.820+0000 7fddff81f640 1 -- 172.21.15.84:0/1251382144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf80c6900 con 0x7fddf80ca510 2023-10-16T11:27:56.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.821+0000 7fddff81f640 1 -- 172.21.15.84:0/1251382144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fddf815bb70 con 0x7fddf80ca510 2023-10-16T11:27:56.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.822+0000 7fddf5ffb640 1 -- 172.21.15.84:0/1251382144 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fddec009af0 con 0x7fddf80ca510 2023-10-16T11:27:56.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.823+0000 7fddf5ffb640 1 --2- 172.21.15.84:0/1251382144 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fdde406f3a0 0x7fdde4071860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:56.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.823+0000 7fddf5ffb640 1 -- 172.21.15.84:0/1251382144 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fddec089c80 con 0x7fddf80ca510 2023-10-16T11:27:56.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.823+0000 7fddfd5bb640 1 --2- 172.21.15.84:0/1251382144 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fdde406f3a0 0x7fdde4071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:56.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.823+0000 7fddf5ffb640 1 --2- 172.21.15.84:0/1251382144 >> [v2:172.21.15.84:6800/2077434097,v1:172.21.15.84:6801/2077434097] conn(0x7fdde40757f0 0x7fdde4077c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:56.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.823+0000 7fddf5ffb640 1 -- 172.21.15.84:0/1251382144 --> [v2:172.21.15.84:6800/2077434097,v1:172.21.15.84:6801/2077434097] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdde40782e0 con 0x7fdde40757f0 2023-10-16T11:27:56.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.823+0000 7fddf5ffb640 1 -- 172.21.15.84:0/1251382144 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=275) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddec08e220 con 0x7fddf80ca510 2023-10-16T11:27:56.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.823+0000 7fddfddbc640 1 --2- 172.21.15.84:0/1251382144 >> [v2:172.21.15.84:6800/2077434097,v1:172.21.15.84:6801/2077434097] conn(0x7fdde40757f0 0x7fdde4077c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:56.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.824+0000 7fddfd5bb640 1 --2- 172.21.15.84:0/1251382144 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fdde406f3a0 0x7fdde4071860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdde8035000 tx=0x7fdde8030000 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:56.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.824+0000 7fddfddbc640 1 --2- 172.21.15.84:0/1251382144 >> [v2:172.21.15.84:6800/2077434097,v1:172.21.15.84:6801/2077434097] conn(0x7fdde40757f0 0x7fdde4077c10 crc :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:56.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.825+0000 7fddf5ffb640 1 -- 172.21.15.84:0/1251382144 <== osd.0 v2:172.21.15.84:6800/2077434097 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fdde40782e0 con 0x7fdde40757f0 2023-10-16T11:27:56.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.839+0000 7fddff81f640 1 -- 172.21.15.84:0/1251382144 --> [v2:172.21.15.84:6800/2077434097,v1:172.21.15.84:6801/2077434097] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fddf815b890 con 0x7fdde40757f0 2023-10-16T11:27:56.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.839+0000 7fddf5ffb640 1 -- 172.21.15.84:0/1251382144 <== osd.0 v2:172.21.15.84:6800/2077434097 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fddf0009340 con 0x7fdde40757f0 2023-10-16T11:27:56.841 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:27:56.841 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T11:27:56.841 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:27:56.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.839+0000 7fddff81f640 1 -- 172.21.15.84:0/1251382144 >> [v2:172.21.15.84:6800/2077434097,v1:172.21.15.84:6801/2077434097] conn(0x7fdde40757f0 msgr2=0x7fdde4077c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:56.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.839+0000 7fddff81f640 1 --2- 172.21.15.84:0/1251382144 >> [v2:172.21.15.84:6800/2077434097,v1:172.21.15.84:6801/2077434097] conn(0x7fdde40757f0 0x7fdde4077c10 crc :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:56.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.839+0000 7fddff81f640 1 -- 172.21.15.84:0/1251382144 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fdde406f3a0 msgr2=0x7fdde4071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:56.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.839+0000 7fddff81f640 1 --2- 172.21.15.84:0/1251382144 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fdde406f3a0 0x7fdde4071860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdde8035000 tx=0x7fdde8030000 comp rx=0 tx=0).stop 2023-10-16T11:27:56.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.839+0000 7fddff81f640 1 -- 172.21.15.84:0/1251382144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddf80ca510 msgr2=0x7fddf80ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:56.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.839+0000 7fddff81f640 1 --2- 172.21.15.84:0/1251382144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddf80ca510 0x7fddf80ceac0 secure :-1 s=READY pgs=3542 cs=0 l=1 rev1=1 crypto rx=0x7fddec00d800 tx=0x7fddec00dcc0 comp rx=0 tx=0).stop 2023-10-16T11:27:56.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.839+0000 7fddfddbc640 1 -- 172.21.15.84:0/1251382144 reap_dead start 2023-10-16T11:27:56.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.839+0000 7fddff81f640 1 -- 172.21.15.84:0/1251382144 shutdown_connections 2023-10-16T11:27:56.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.839+0000 7fddff81f640 1 -- 172.21.15.84:0/1251382144 >> 172.21.15.84:0/1251382144 conn(0x7fddf80cb360 msgr2=0x7fddf80ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:56.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.839+0000 7fddff81f640 1 -- 172.21.15.84:0/1251382144 shutdown_connections 2023-10-16T11:27:56.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:56.839+0000 7fddff81f640 1 -- 172.21.15.84:0/1251382144 wait complete. 2023-10-16T11:27:56.855 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-16T11:27:57.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.061+0000 7fecf1a61640 1 Processor -- start 2023-10-16T11:27:57.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.061+0000 7fecf1a61640 1 -- start start 2023-10-16T11:27:57.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.063+0000 7fecf1a61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec0cf5f0 0x7fecec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.063+0000 7fecf1a61640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecec161740 con 0x7fecec070c10 2023-10-16T11:27:57.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.063+0000 7fecf1a61640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecec06b0f0 con 0x7fecec070830 2023-10-16T11:27:57.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.063+0000 7fecf1a61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec1619b0 con 0x7fecec0cf5f0 2023-10-16T11:27:57.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.063+0000 7feceb7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fecec070c10 0x7fecec0cd690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60504/0 (socket says 172.21.15.84:60504) 2023-10-16T11:27:57.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.063+0000 7feceb7fe640 1 -- 172.21.15.84:0/2100284913 learned_addr learned my addr 172.21.15.84:0/2100284913 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:57.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.063+0000 7fece27fc640 1 --2- 172.21.15.84:0/2100284913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec0cf5f0 0x7fecec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.063+0000 7fecea7fc640 1 -- 172.21.15.84:0/2100284913 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804995900 0 0) 0x7fecec161740 con 0x7fecec070c10 2023-10-16T11:27:57.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.063+0000 7fecea7fc640 1 -- 172.21.15.84:0/2100284913 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecd8003520 con 0x7fecec070c10 2023-10-16T11:27:57.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.064+0000 7fecea7fc640 1 -- 172.21.15.84:0/2100284913 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2409122422 0 0) 0x7fecd8003520 con 0x7fecec070c10 2023-10-16T11:27:57.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.064+0000 7fecea7fc640 1 -- 172.21.15.84:0/2100284913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec0cf5f0 msgr2=0x7fecec06d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:57.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.064+0000 7fecea7fc640 1 --2- 172.21.15.84:0/2100284913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec0cf5f0 0x7fecec06d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.064+0000 7fecea7fc640 1 -- 172.21.15.84:0/2100284913 >> v1:172.21.15.157:6789/0 conn(0x7fecec070830 legacy=0x7fecec06c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.064+0000 7fecea7fc640 1 -- 172.21.15.84:0/2100284913 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec06b3d0 con 0x7fecec070c10 2023-10-16T11:27:57.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.064+0000 7fecea7fc640 1 -- 172.21.15.84:0/2100284913 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fece40019f0 con 0x7fecec070c10 2023-10-16T11:27:57.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.064+0000 7fecea7fc640 1 -- 172.21.15.84:0/2100284913 >> v1:172.21.15.84:6789/0 conn(0x7fecec070c10 legacy=0x7fecec0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.064+0000 7fecea7fc640 1 --2- 172.21.15.84:0/2100284913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec0cf5f0 0x7fecd80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.064+0000 7fecea7fc640 1 --2- 172.21.15.84:0/2100284913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd8003ee0 0x7fecd80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.064+0000 7fecea7fc640 1 --2- 172.21.15.84:0/2100284913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8004d00 0x7fecd8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.064+0000 7fecea7fc640 1 -- 172.21.15.84:0/2100284913 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec1619b0 con 0x7fecd8003ee0 2023-10-16T11:27:57.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.064+0000 7fecea7fc640 1 -- 172.21.15.84:0/2100284913 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd800bb40 con 0x7fecd8004d00 2023-10-16T11:27:57.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.064+0000 7fecea7fc640 1 -- 172.21.15.84:0/2100284913 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd800bd00 con 0x7fecec0cf5f0 2023-10-16T11:27:57.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.064+0000 7fece27fc640 1 --2- 172.21.15.84:0/2100284913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd8003ee0 0x7fecd80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.065+0000 7feceaffd640 1 --2- 172.21.15.84:0/2100284913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec0cf5f0 0x7fecd80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.065+0000 7feceb7fe640 1 --2- 172.21.15.84:0/2100284913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8004d00 0x7fecd8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.065+0000 7fece27fc640 1 -- 172.21.15.84:0/2100284913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec0cf5f0 msgr2=0x7fecd80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:57.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.065+0000 7fece27fc640 1 --2- 172.21.15.84:0/2100284913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec0cf5f0 0x7fecd80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.065+0000 7fece27fc640 1 -- 172.21.15.84:0/2100284913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8004d00 msgr2=0x7fecd8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:57.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.065+0000 7fece27fc640 1 --2- 172.21.15.84:0/2100284913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8004d00 0x7fecd8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.065+0000 7fece27fc640 1 -- 172.21.15.84:0/2100284913 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd8003520 con 0x7fecd8003ee0 2023-10-16T11:27:57.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.065+0000 7feceb7fe640 1 --2- 172.21.15.84:0/2100284913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8004d00 0x7fecd8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:57.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.066+0000 7fece27fc640 1 --2- 172.21.15.84:0/2100284913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd8003ee0 0x7fecd80047c0 secure :-1 s=READY pgs=3544 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009a90 tx=0x7fecd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b425d7fef07ab2b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:57.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.066+0000 7fecea7fc640 1 -- 172.21.15.84:0/2100284913 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd402c680 con 0x7fecd8003ee0 2023-10-16T11:27:57.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.066+0000 7fecea7fc640 1 -- 172.21.15.84:0/2100284913 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd402c7e0 con 0x7fecd8003ee0 2023-10-16T11:27:57.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.066+0000 7fecea7fc640 1 -- 172.21.15.84:0/2100284913 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd402cb70 con 0x7fecd8003ee0 2023-10-16T11:27:57.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.067+0000 7fecf1a61640 1 -- 172.21.15.84:0/2100284913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd8003ee0 msgr2=0x7fecd80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.067+0000 7fecf1a61640 1 --2- 172.21.15.84:0/2100284913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd8003ee0 0x7fecd80047c0 secure :-1 s=READY pgs=3544 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009a90 tx=0x7fecd80057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.067+0000 7feceb7fe640 1 -- 172.21.15.84:0/2100284913 reap_dead start 2023-10-16T11:27:57.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.067+0000 7fecf1a61640 1 -- 172.21.15.84:0/2100284913 shutdown_connections 2023-10-16T11:27:57.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.067+0000 7fecf1a61640 1 -- 172.21.15.84:0/2100284913 >> 172.21.15.84:0/2100284913 conn(0x7fecec0cfac0 msgr2=0x7fecec0cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:57.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.067+0000 7fecf1a61640 1 -- 172.21.15.84:0/2100284913 shutdown_connections 2023-10-16T11:27:57.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.067+0000 7fecf1a61640 1 -- 172.21.15.84:0/2100284913 wait complete. 2023-10-16T11:27:57.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.068+0000 7fecf1a61640 1 Processor -- start 2023-10-16T11:27:57.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.069+0000 7fecf1a61640 1 -- start start 2023-10-16T11:27:57.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.069+0000 7fecf1a61640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8003ee0 0x7fecec0c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.069+0000 7fecf1a61640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecec14a190 0x7fecec15e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.069+0000 7fecf1a61640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec15eea0 0x7fecec158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.070+0000 7fecf1a61640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec15f320 con 0x7fecec14a190 2023-10-16T11:27:57.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.070+0000 7fecf1a61640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0b2970 con 0x7fecd8003ee0 2023-10-16T11:27:57.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.070+0000 7fecf1a61640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0b2c50 con 0x7fecec15eea0 2023-10-16T11:27:57.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.070+0000 7fece27fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecec14a190 0x7fecec15e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.070+0000 7fece27fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecec14a190 0x7fecec15e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52336/0 (socket says 172.21.15.84:52336) 2023-10-16T11:27:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.070+0000 7fece27fc640 1 -- 172.21.15.84:0/261813408 learned_addr learned my addr 172.21.15.84:0/261813408 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.070+0000 7feceaffd640 1 --2- 172.21.15.84:0/261813408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8003ee0 0x7fecec0c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.070+0000 7feceb7fe640 1 --2- 172.21.15.84:0/261813408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec15eea0 0x7fecec158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.070+0000 7fece27fc640 1 -- 172.21.15.84:0/261813408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec15eea0 msgr2=0x7fecec158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.070+0000 7fece27fc640 1 --2- 172.21.15.84:0/261813408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec15eea0 0x7fecec158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.070+0000 7fece27fc640 1 -- 172.21.15.84:0/261813408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8003ee0 msgr2=0x7fecec0c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.070+0000 7fece27fc640 1 --2- 172.21.15.84:0/261813408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8003ee0 0x7fecec0c58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.070+0000 7fece27fc640 1 -- 172.21.15.84:0/261813408 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec0b2f30 con 0x7fecec14a190 2023-10-16T11:27:57.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.071+0000 7fece27fc640 1 --2- 172.21.15.84:0/261813408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecec14a190 0x7fecec15e960 secure :-1 s=READY pgs=3545 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009a90 tx=0x7fecd4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:57.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.071+0000 7fece8ff9640 1 -- 172.21.15.84:0/261813408 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd402dea0 con 0x7fecec14a190 2023-10-16T11:27:57.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.071+0000 7fece8ff9640 1 -- 172.21.15.84:0/261813408 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd4003810 con 0x7fecec14a190 2023-10-16T11:27:57.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.071+0000 7fece8ff9640 1 -- 172.21.15.84:0/261813408 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4003ba0 con 0x7fecec14a190 2023-10-16T11:27:57.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.071+0000 7fecf1a61640 1 -- 172.21.15.84:0/261813408 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecec0b3210 con 0x7fecec14a190 2023-10-16T11:27:57.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.071+0000 7fecf1a61640 1 -- 172.21.15.84:0/261813408 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecec14a610 con 0x7fecec14a190 2023-10-16T11:27:57.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.072+0000 7fecf1a61640 1 -- 172.21.15.84:0/261813408 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecec0b2d90 con 0x7fecec14a190 2023-10-16T11:27:57.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.073+0000 7fece8ff9640 1 -- 172.21.15.84:0/261813408 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fecd4003d00 con 0x7fecec14a190 2023-10-16T11:27:57.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.074+0000 7fece8ff9640 1 --2- 172.21.15.84:0/261813408 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7feccc06af90 0x7feccc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.074+0000 7fece8ff9640 1 -- 172.21.15.84:0/261813408 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fecd40ab6d0 con 0x7fecec14a190 2023-10-16T11:27:57.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.074+0000 7feceaffd640 1 --2- 172.21.15.84:0/261813408 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7feccc06af90 0x7feccc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.075+0000 7feceaffd640 1 --2- 172.21.15.84:0/261813408 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7feccc06af90 0x7feccc06d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fecdc005540 tx=0x7fecdc007c50 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:57.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.078+0000 7fece8ff9640 1 -- 172.21.15.84:0/261813408 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fecec0b2d90 con 0x7fecec14a190 2023-10-16T11:27:57.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.269+0000 7fecf1a61640 1 -- 172.21.15.84:0/261813408 --> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fecec0cf060 con 0x7feccc06af90 2023-10-16T11:27:57.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.270+0000 7fece8ff9640 1 -- 172.21.15.84:0/261813408 <== mgr.13719 v2:172.21.15.84:6832/3003264401 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fecdc0014d0 con 0x7feccc06af90 2023-10-16T11:27:57.272 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 0 to deep-scrub 2023-10-16T11:27:57.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.272+0000 7fecf1a61640 1 -- 172.21.15.84:0/261813408 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7feccc06af90 msgr2=0x7feccc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.272+0000 7fecf1a61640 1 --2- 172.21.15.84:0/261813408 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7feccc06af90 0x7feccc06d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fecdc005540 tx=0x7fecdc007c50 comp rx=0 tx=0).stop 2023-10-16T11:27:57.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.272+0000 7fecf1a61640 1 -- 172.21.15.84:0/261813408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecec14a190 msgr2=0x7fecec15e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.272+0000 7fecf1a61640 1 --2- 172.21.15.84:0/261813408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecec14a190 0x7fecec15e960 secure :-1 s=READY pgs=3545 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009a90 tx=0x7fecd4004730 comp rx=0 tx=0).stop 2023-10-16T11:27:57.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.273+0000 7fecf1a61640 1 -- 172.21.15.84:0/261813408 shutdown_connections 2023-10-16T11:27:57.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.273+0000 7fecf1a61640 1 --2- 172.21.15.84:0/261813408 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7feccc06af90 0x7feccc06d450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.273+0000 7fecf1a61640 1 --2- 172.21.15.84:0/261813408 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec15eea0 0x7fecec158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.273+0000 7fecf1a61640 1 --2- 172.21.15.84:0/261813408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecec14a190 0x7fecec15e960 unknown :-1 s=CLOSED pgs=3545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.273+0000 7fecf1a61640 1 --2- 172.21.15.84:0/261813408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd8003ee0 0x7fecec0c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.273+0000 7fecf1a61640 1 -- 172.21.15.84:0/261813408 >> 172.21.15.84:0/261813408 conn(0x7fecec0cfac0 msgr2=0x7fecec06b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:57.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.273+0000 7fecf1a61640 1 -- 172.21.15.84:0/261813408 shutdown_connections 2023-10-16T11:27:57.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.273+0000 7fecf1a61640 1 -- 172.21.15.84:0/261813408 wait complete. 2023-10-16T11:27:57.293 INFO:tasks.ceph:Scrubbing osd.1 2023-10-16T11:27:57.293 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-16T11:27:57.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.504+0000 7f995aa21640 1 Processor -- start 2023-10-16T11:27:57.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.504+0000 7f995aa21640 1 -- start start 2023-10-16T11:27:57.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f995aa21640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99540c9300 0x7f995406c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f995aa21640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995415dfb0 con 0x7f99541599c0 2023-10-16T11:27:57.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f995aa21640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995415e290 con 0x7f9954159da0 2023-10-16T11:27:57.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f995aa21640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99540c9740 con 0x7f99540c9300 2023-10-16T11:27:57.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9953fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f99541599c0 0x7f995415d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60512/0 (socket says 172.21.15.84:60512) 2023-10-16T11:27:57.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9953fff640 1 -- 172.21.15.84:0/2019305425 learned_addr learned my addr 172.21.15.84:0/2019305425 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:57.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9958fbe640 1 --2- 172.21.15.84:0/2019305425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99540c9300 0x7f995406c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9952ffd640 1 -- 172.21.15.84:0/2019305425 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910608964 0 0) 0x7f995415dfb0 con 0x7f99541599c0 2023-10-16T11:27:57.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9952ffd640 1 -- 172.21.15.84:0/2019305425 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f993c003520 con 0x7f99541599c0 2023-10-16T11:27:57.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9952ffd640 1 -- 172.21.15.84:0/2019305425 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 14513673 0 0) 0x7f993c003520 con 0x7f99541599c0 2023-10-16T11:27:57.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9952ffd640 1 -- 172.21.15.84:0/2019305425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99540c9300 msgr2=0x7f995406c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:57.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9952ffd640 1 --2- 172.21.15.84:0/2019305425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99540c9300 0x7f995406c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9952ffd640 1 -- 172.21.15.84:0/2019305425 >> v1:172.21.15.157:6789/0 conn(0x7f9954159da0 legacy=0x7f99540cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9952ffd640 1 -- 172.21.15.84:0/2019305425 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995415e570 con 0x7f99541599c0 2023-10-16T11:27:57.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9952ffd640 1 -- 172.21.15.84:0/2019305425 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f9944001670 con 0x7f99541599c0 2023-10-16T11:27:57.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9952ffd640 1 -- 172.21.15.84:0/2019305425 >> v1:172.21.15.84:6789/0 conn(0x7f99541599c0 legacy=0x7f995415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9952ffd640 1 --2- 172.21.15.84:0/2019305425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f993c003a10 0x7f993c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9952ffd640 1 --2- 172.21.15.84:0/2019305425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f993c0047c0 0x7f993c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9952ffd640 1 --2- 172.21.15.84:0/2019305425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99540c9300 0x7f993c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9952ffd640 1 -- 172.21.15.84:0/2019305425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c00ba80 con 0x7f993c003a10 2023-10-16T11:27:57.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9952ffd640 1 -- 172.21.15.84:0/2019305425 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c00bbf0 con 0x7f993c0047c0 2023-10-16T11:27:57.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9952ffd640 1 -- 172.21.15.84:0/2019305425 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993c00bdb0 con 0x7f99540c9300 2023-10-16T11:27:57.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.506+0000 7f9953fff640 1 --2- 172.21.15.84:0/2019305425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f993c003a10 0x7f993c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.507+0000 7f9958fbe640 1 --2- 172.21.15.84:0/2019305425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99540c9300 0x7f993c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.507+0000 7f99537fe640 1 --2- 172.21.15.84:0/2019305425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f993c0047c0 0x7f993c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.507+0000 7f9953fff640 1 -- 172.21.15.84:0/2019305425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99540c9300 msgr2=0x7f993c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:57.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.507+0000 7f9953fff640 1 --2- 172.21.15.84:0/2019305425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99540c9300 0x7f993c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.507+0000 7f9953fff640 1 -- 172.21.15.84:0/2019305425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f993c0047c0 msgr2=0x7f993c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:57.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.507+0000 7f9953fff640 1 --2- 172.21.15.84:0/2019305425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f993c0047c0 0x7f993c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.507+0000 7f9953fff640 1 -- 172.21.15.84:0/2019305425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c003520 con 0x7f993c003a10 2023-10-16T11:27:57.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.507+0000 7f99537fe640 1 --2- 172.21.15.84:0/2019305425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f993c0047c0 0x7f993c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:57.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.508+0000 7f9953fff640 1 --2- 172.21.15.84:0/2019305425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f993c003a10 0x7f993c003e30 secure :-1 s=READY pgs=3547 cs=0 l=1 rev1=1 crypto rx=0x7f994400c240 tx=0x7f993c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa6dfdeaace532e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:57.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.508+0000 7f9952ffd640 1 -- 172.21.15.84:0/2019305425 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944005ad0 con 0x7f993c003a10 2023-10-16T11:27:57.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.508+0000 7f9952ffd640 1 -- 172.21.15.84:0/2019305425 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9944005c50 con 0x7f993c003a10 2023-10-16T11:27:57.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.508+0000 7f9952ffd640 1 -- 172.21.15.84:0/2019305425 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994402c8b0 con 0x7f993c003a10 2023-10-16T11:27:57.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.509+0000 7f995aa21640 1 -- 172.21.15.84:0/2019305425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f993c003a10 msgr2=0x7f993c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.509+0000 7f995aa21640 1 --2- 172.21.15.84:0/2019305425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f993c003a10 0x7f993c003e30 secure :-1 s=READY pgs=3547 cs=0 l=1 rev1=1 crypto rx=0x7f994400c240 tx=0x7f993c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.509+0000 7f9958fbe640 1 -- 172.21.15.84:0/2019305425 reap_dead start 2023-10-16T11:27:57.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.509+0000 7f995aa21640 1 -- 172.21.15.84:0/2019305425 shutdown_connections 2023-10-16T11:27:57.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.509+0000 7f995aa21640 1 -- 172.21.15.84:0/2019305425 >> 172.21.15.84:0/2019305425 conn(0x7f99540cb360 msgr2=0x7f99540cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:57.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.509+0000 7f995aa21640 1 -- 172.21.15.84:0/2019305425 shutdown_connections 2023-10-16T11:27:57.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.509+0000 7f995aa21640 1 -- 172.21.15.84:0/2019305425 wait complete. 2023-10-16T11:27:57.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.511+0000 7f995aa21640 1 Processor -- start 2023-10-16T11:27:57.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.511+0000 7f995aa21640 1 -- start start 2023-10-16T11:27:57.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.513+0000 7f995aa21640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f993c003a10 0x7f99541f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.513+0000 7f995aa21640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99540cbd50 0x7f99540acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.513+0000 7f995aa21640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99541557c0 0x7f9954155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.513+0000 7f995aa21640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99541f6610 con 0x7f993c003a10 2023-10-16T11:27:57.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.513+0000 7f995aa21640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540b1500 con 0x7f99541557c0 2023-10-16T11:27:57.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.513+0000 7f995aa21640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9954156190 con 0x7f99540cbd50 2023-10-16T11:27:57.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.513+0000 7f9953fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f993c003a10 0x7f99541f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.513+0000 7f9953fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f993c003a10 0x7f99541f60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52354/0 (socket says 172.21.15.84:52354) 2023-10-16T11:27:57.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.513+0000 7f9953fff640 1 -- 172.21.15.84:0/1479249007 learned_addr learned my addr 172.21.15.84:0/1479249007 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:57.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.513+0000 7f9958fbe640 1 --2- 172.21.15.84:0/1479249007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99541557c0 0x7f9954155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.513+0000 7f99537fe640 1 --2- 172.21.15.84:0/1479249007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99540cbd50 0x7f99540acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.513+0000 7f9953fff640 1 -- 172.21.15.84:0/1479249007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99540cbd50 msgr2=0x7f99540acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.513+0000 7f9953fff640 1 --2- 172.21.15.84:0/1479249007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99540cbd50 0x7f99540acb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.513+0000 7f9953fff640 1 -- 172.21.15.84:0/1479249007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99541557c0 msgr2=0x7f9954155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.513+0000 7f9953fff640 1 --2- 172.21.15.84:0/1479249007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99541557c0 0x7f9954155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.513+0000 7f9953fff640 1 -- 172.21.15.84:0/1479249007 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995415c900 con 0x7f993c003a10 2023-10-16T11:27:57.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.513+0000 7f99537fe640 1 --2- 172.21.15.84:0/1479249007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99540cbd50 0x7f99540acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:27:57.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.514+0000 7f9953fff640 1 --2- 172.21.15.84:0/1479249007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f993c003a10 0x7f99541f60d0 secure :-1 s=READY pgs=3548 cs=0 l=1 rev1=1 crypto rx=0x7f994402c450 tx=0x7f9944001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:57.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.514+0000 7f9950ff9640 1 -- 172.21.15.84:0/1479249007 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944039070 con 0x7f993c003a10 2023-10-16T11:27:57.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.514+0000 7f9950ff9640 1 -- 172.21.15.84:0/1479249007 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994402ccc0 con 0x7f993c003a10 2023-10-16T11:27:57.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.514+0000 7f9950ff9640 1 -- 172.21.15.84:0/1479249007 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994403c040 con 0x7f993c003a10 2023-10-16T11:27:57.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.514+0000 7f995aa21640 1 -- 172.21.15.84:0/1479249007 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995415cbc0 con 0x7f993c003a10 2023-10-16T11:27:57.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.514+0000 7f995aa21640 1 -- 172.21.15.84:0/1479249007 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99540b1750 con 0x7f993c003a10 2023-10-16T11:27:57.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.515+0000 7f995aa21640 1 -- 172.21.15.84:0/1479249007 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f995406fa50 con 0x7f993c003a10 2023-10-16T11:27:57.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.516+0000 7f9950ff9640 1 -- 172.21.15.84:0/1479249007 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9944032070 con 0x7f993c003a10 2023-10-16T11:27:57.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.517+0000 7f9950ff9640 1 --2- 172.21.15.84:0/1479249007 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f994006af90 0x7f994006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.517+0000 7f9950ff9640 1 -- 172.21.15.84:0/1479249007 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f99440aaa10 con 0x7f993c003a10 2023-10-16T11:27:57.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.517+0000 7f9950ff9640 1 --2- 172.21.15.84:0/1479249007 >> [v2:172.21.15.84:6808/805077222,v1:172.21.15.84:6809/805077222] conn(0x7f99400713e0 0x7f9940073800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.517+0000 7f99537fe640 1 --2- 172.21.15.84:0/1479249007 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f994006af90 0x7f994006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.517+0000 7f9950ff9640 1 -- 172.21.15.84:0/1479249007 --> [v2:172.21.15.84:6808/805077222,v1:172.21.15.84:6809/805077222] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9940073ed0 con 0x7f99400713e0 2023-10-16T11:27:57.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.517+0000 7f9950ff9640 1 -- 172.21.15.84:0/1479249007 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=275) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99440af2a0 con 0x7f993c003a10 2023-10-16T11:27:57.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.518+0000 7f9958fbe640 1 --2- 172.21.15.84:0/1479249007 >> [v2:172.21.15.84:6808/805077222,v1:172.21.15.84:6809/805077222] conn(0x7f99400713e0 0x7f9940073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.518+0000 7f99537fe640 1 --2- 172.21.15.84:0/1479249007 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f994006af90 0x7f994006d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9954159330 tx=0x7f9948009040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:57.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.518+0000 7f9958fbe640 1 --2- 172.21.15.84:0/1479249007 >> [v2:172.21.15.84:6808/805077222,v1:172.21.15.84:6809/805077222] conn(0x7f99400713e0 0x7f9940073800 crc :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:57.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.519+0000 7f9950ff9640 1 -- 172.21.15.84:0/1479249007 <== osd.1 v2:172.21.15.84:6808/805077222 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f9940073ed0 con 0x7f99400713e0 2023-10-16T11:27:57.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.533+0000 7f995aa21640 1 -- 172.21.15.84:0/1479249007 --> [v2:172.21.15.84:6808/805077222,v1:172.21.15.84:6809/805077222] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f995406fda0 con 0x7f99400713e0 2023-10-16T11:27:57.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.534+0000 7f9950ff9640 1 -- 172.21.15.84:0/1479249007 <== osd.1 v2:172.21.15.84:6808/805077222 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f994c013040 con 0x7f99400713e0 2023-10-16T11:27:57.536 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:27:57.536 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T11:27:57.537 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:27:57.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.534+0000 7f995aa21640 1 -- 172.21.15.84:0/1479249007 >> [v2:172.21.15.84:6808/805077222,v1:172.21.15.84:6809/805077222] conn(0x7f99400713e0 msgr2=0x7f9940073800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.534+0000 7f995aa21640 1 --2- 172.21.15.84:0/1479249007 >> [v2:172.21.15.84:6808/805077222,v1:172.21.15.84:6809/805077222] conn(0x7f99400713e0 0x7f9940073800 crc :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.534+0000 7f995aa21640 1 -- 172.21.15.84:0/1479249007 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f994006af90 msgr2=0x7f994006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.534+0000 7f995aa21640 1 --2- 172.21.15.84:0/1479249007 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f994006af90 0x7f994006d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9954159330 tx=0x7f9948009040 comp rx=0 tx=0).stop 2023-10-16T11:27:57.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.534+0000 7f995aa21640 1 -- 172.21.15.84:0/1479249007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f993c003a10 msgr2=0x7f99541f60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.534+0000 7f995aa21640 1 --2- 172.21.15.84:0/1479249007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f993c003a10 0x7f99541f60d0 secure :-1 s=READY pgs=3548 cs=0 l=1 rev1=1 crypto rx=0x7f994402c450 tx=0x7f9944001a00 comp rx=0 tx=0).stop 2023-10-16T11:27:57.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.534+0000 7f9958fbe640 1 -- 172.21.15.84:0/1479249007 reap_dead start 2023-10-16T11:27:57.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.534+0000 7f995aa21640 1 -- 172.21.15.84:0/1479249007 shutdown_connections 2023-10-16T11:27:57.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.534+0000 7f995aa21640 1 -- 172.21.15.84:0/1479249007 >> 172.21.15.84:0/1479249007 conn(0x7f99540cb360 msgr2=0x7f99540c8130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:57.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.534+0000 7f995aa21640 1 -- 172.21.15.84:0/1479249007 shutdown_connections 2023-10-16T11:27:57.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.534+0000 7f995aa21640 1 -- 172.21.15.84:0/1479249007 wait complete. 2023-10-16T11:27:57.553 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-16T11:27:57.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.758+0000 7f65eea5d640 1 Processor -- start 2023-10-16T11:27:57.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.759+0000 7f65eea5d640 1 -- start start 2023-10-16T11:27:57.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.760+0000 7f65eea5d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65e806a4b0 0x7f65e80ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.760+0000 7f65eea5d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e80cb030 con 0x7f65e80c9ca0 2023-10-16T11:27:57.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.760+0000 7f65eea5d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e80c91c0 con 0x7f65e815ad70 2023-10-16T11:27:57.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.760+0000 7f65eea5d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e80c94a0 con 0x7f65e806a4b0 2023-10-16T11:27:57.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.760+0000 7f65ecffa640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f65e80c9ca0 0x7f65e80ad4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60518/0 (socket says 172.21.15.84:60518) 2023-10-16T11:27:57.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.761+0000 7f65ecffa640 1 -- 172.21.15.84:0/1428985788 learned_addr learned my addr 172.21.15.84:0/1428985788 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:57.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.761+0000 7f65e77fe640 1 --2- 172.21.15.84:0/1428985788 >> v2:172.21.15.157:3300/0 conn(0x7f65e806a4b0 0x7f65e80ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.761+0000 7f65e6ffd640 1 -- 172.21.15.84:0/1428985788 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881609595 0 0) 0x7f65e80cb030 con 0x7f65e80c9ca0 2023-10-16T11:27:57.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.761+0000 7f65e6ffd640 1 -- 172.21.15.84:0/1428985788 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65d0003520 con 0x7f65e80c9ca0 2023-10-16T11:27:57.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.761+0000 7f65e6ffd640 1 -- 172.21.15.84:0/1428985788 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 600641240 0 0) 0x7f65d0003520 con 0x7f65e80c9ca0 2023-10-16T11:27:57.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.761+0000 7f65e6ffd640 1 -- 172.21.15.84:0/1428985788 >> v2:172.21.15.157:3300/0 conn(0x7f65e806a4b0 msgr2=0x7f65e80ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:57.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.761+0000 7f65e6ffd640 1 --2- 172.21.15.84:0/1428985788 >> v2:172.21.15.157:3300/0 conn(0x7f65e806a4b0 0x7f65e80ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.761+0000 7f65e6ffd640 1 -- 172.21.15.84:0/1428985788 >> v1:172.21.15.157:6789/0 conn(0x7f65e815ad70 legacy=0x7f65e8069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.761+0000 7f65e6ffd640 1 -- 172.21.15.84:0/1428985788 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e80c9780 con 0x7f65e80c9ca0 2023-10-16T11:27:57.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.761+0000 7f65e6ffd640 1 -- 172.21.15.84:0/1428985788 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f65e0001670 con 0x7f65e80c9ca0 2023-10-16T11:27:57.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.761+0000 7f65e6ffd640 1 -- 172.21.15.84:0/1428985788 >> v1:172.21.15.84:6789/0 conn(0x7f65e80c9ca0 legacy=0x7f65e80ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.761+0000 7f65e6ffd640 1 --2- 172.21.15.84:0/1428985788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65d0003970 0x7f65d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.761+0000 7f65e6ffd640 1 --2- 172.21.15.84:0/1428985788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65d00047c0 0x7f65d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.761+0000 7f65e6ffd640 1 --2- 172.21.15.84:0/1428985788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65d0005180 0x7f65d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.761+0000 7f65e6ffd640 1 -- 172.21.15.84:0/1428985788 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e80c94a0 con 0x7f65d00047c0 2023-10-16T11:27:57.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.762+0000 7f65e6ffd640 1 -- 172.21.15.84:0/1428985788 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d000c020 con 0x7f65d0003970 2023-10-16T11:27:57.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.762+0000 7f65e6ffd640 1 -- 172.21.15.84:0/1428985788 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d000c1e0 con 0x7f65d0005180 2023-10-16T11:27:57.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.762+0000 7f65e77fe640 1 --2- 172.21.15.84:0/1428985788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65d00047c0 0x7f65d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.762+0000 7f65ecffa640 1 --2- 172.21.15.84:0/1428985788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65d0005180 0x7f65d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.762+0000 7f65e7fff640 1 --2- 172.21.15.84:0/1428985788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65d0003970 0x7f65d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.762+0000 7f65e77fe640 1 -- 172.21.15.84:0/1428985788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65d0005180 msgr2=0x7f65d000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:57.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.762+0000 7f65e77fe640 1 --2- 172.21.15.84:0/1428985788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65d0005180 0x7f65d000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.762+0000 7f65e77fe640 1 -- 172.21.15.84:0/1428985788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65d0003970 msgr2=0x7f65d0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:57.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.762+0000 7f65e77fe640 1 --2- 172.21.15.84:0/1428985788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65d0003970 0x7f65d0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.762+0000 7f65e77fe640 1 -- 172.21.15.84:0/1428985788 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d0003520 con 0x7f65d00047c0 2023-10-16T11:27:57.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.762+0000 7f65e7fff640 1 --2- 172.21.15.84:0/1428985788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65d0003970 0x7f65d0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:57.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.762+0000 7f65ecffa640 1 -- 172.21.15.84:0/1428985788 reap_dead start 2023-10-16T11:27:57.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.763+0000 7f65e77fe640 1 --2- 172.21.15.84:0/1428985788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65d00047c0 0x7f65d0004c40 secure :-1 s=READY pgs=3550 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009ac0 tx=0x7f65d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43aa9954dbc7dbee server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:57.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.764+0000 7f65e6ffd640 1 -- 172.21.15.84:0/1428985788 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc02c6b0 con 0x7f65d00047c0 2023-10-16T11:27:57.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.764+0000 7f65e6ffd640 1 -- 172.21.15.84:0/1428985788 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc02c810 con 0x7f65d00047c0 2023-10-16T11:27:57.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.764+0000 7f65e6ffd640 1 -- 172.21.15.84:0/1428985788 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc02cba0 con 0x7f65d00047c0 2023-10-16T11:27:57.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.764+0000 7f65eea5d640 1 -- 172.21.15.84:0/1428985788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65d00047c0 msgr2=0x7f65d0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.764+0000 7f65eea5d640 1 --2- 172.21.15.84:0/1428985788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65d00047c0 0x7f65d0004c40 secure :-1 s=READY pgs=3550 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009ac0 tx=0x7f65d00057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.764+0000 7f65eea5d640 1 -- 172.21.15.84:0/1428985788 shutdown_connections 2023-10-16T11:27:57.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.764+0000 7f65eea5d640 1 --2- 172.21.15.84:0/1428985788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65d00047c0 0x7f65d0004c40 unknown :-1 s=CLOSED pgs=3550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.764+0000 7f65eea5d640 1 -- 172.21.15.84:0/1428985788 >> 172.21.15.84:0/1428985788 conn(0x7f65e80ce9b0 msgr2=0x7f65e80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:57.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.764+0000 7f65eea5d640 1 -- 172.21.15.84:0/1428985788 shutdown_connections 2023-10-16T11:27:57.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.764+0000 7f65eea5d640 1 -- 172.21.15.84:0/1428985788 wait complete. 2023-10-16T11:27:57.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.766+0000 7f65eea5d640 1 Processor -- start 2023-10-16T11:27:57.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.766+0000 7f65eea5d640 1 -- start start 2023-10-16T11:27:57.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.767+0000 7f65eea5d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65d00047c0 0x7f65e81f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.767+0000 7f65eea5d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e81513d0 0x7f65e80cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.767+0000 7f65eea5d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e81f2a70 0x7f65e8150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.767+0000 7f65eea5d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e80b0520 con 0x7f65d00047c0 2023-10-16T11:27:57.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.767+0000 7f65eea5d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e80b07c0 con 0x7f65e81f2a70 2023-10-16T11:27:57.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.767+0000 7f65eea5d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e80b0aa0 con 0x7f65e81513d0 2023-10-16T11:27:57.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.767+0000 7f65e7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65d00047c0 0x7f65e81f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.767+0000 7f65e7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65d00047c0 0x7f65e81f3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52372/0 (socket says 172.21.15.84:52372) 2023-10-16T11:27:57.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.768+0000 7f65e7fff640 1 -- 172.21.15.84:0/1663124271 learned_addr learned my addr 172.21.15.84:0/1663124271 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:57.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.768+0000 7f65e7fff640 1 -- 172.21.15.84:0/1663124271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e81513d0 msgr2=0x7f65e80cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.768+0000 7f65e7fff640 1 --2- 172.21.15.84:0/1663124271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e81513d0 0x7f65e80cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.768+0000 7f65e7fff640 1 -- 172.21.15.84:0/1663124271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e81f2a70 msgr2=0x7f65e8150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.768+0000 7f65e7fff640 1 --2- 172.21.15.84:0/1663124271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e81f2a70 0x7f65e8150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.768+0000 7f65e7fff640 1 -- 172.21.15.84:0/1663124271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e80b0d80 con 0x7f65d00047c0 2023-10-16T11:27:57.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.768+0000 7f65e7fff640 1 --2- 172.21.15.84:0/1663124271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65d00047c0 0x7f65e81f3460 secure :-1 s=READY pgs=3551 cs=0 l=1 rev1=1 crypto rx=0x7f65d800d810 tx=0x7f65d800dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:57.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.768+0000 7f65e4ff9640 1 -- 172.21.15.84:0/1663124271 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d8003990 con 0x7f65d00047c0 2023-10-16T11:27:57.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.768+0000 7f65e4ff9640 1 -- 172.21.15.84:0/1663124271 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d8003b10 con 0x7f65d00047c0 2023-10-16T11:27:57.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.768+0000 7f65e4ff9640 1 -- 172.21.15.84:0/1663124271 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d800b6b0 con 0x7f65d00047c0 2023-10-16T11:27:57.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.769+0000 7f65eea5d640 1 -- 172.21.15.84:0/1663124271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e80b1060 con 0x7f65d00047c0 2023-10-16T11:27:57.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.769+0000 7f65eea5d640 1 -- 172.21.15.84:0/1663124271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e80cdae0 con 0x7f65d00047c0 2023-10-16T11:27:57.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.770+0000 7f65eea5d640 1 -- 172.21.15.84:0/1663124271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e804e160 con 0x7f65d00047c0 2023-10-16T11:27:57.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.771+0000 7f65e4ff9640 1 -- 172.21.15.84:0/1663124271 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f65d800b810 con 0x7f65d00047c0 2023-10-16T11:27:57.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.772+0000 7f65e4ff9640 1 --2- 172.21.15.84:0/1663124271 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f65d406b010 0x7f65d406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:57.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.772+0000 7f65e4ff9640 1 -- 172.21.15.84:0/1663124271 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f65d8089390 con 0x7f65d00047c0 2023-10-16T11:27:57.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.772+0000 7f65e77fe640 1 --2- 172.21.15.84:0/1663124271 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f65d406b010 0x7f65d406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:57.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.773+0000 7f65e77fe640 1 --2- 172.21.15.84:0/1663124271 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f65d406b010 0x7f65d406d4d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f65dc036000 tx=0x7f65dc02e000 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:57.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.776+0000 7f65e4ff9640 1 -- 172.21.15.84:0/1663124271 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f65d80541a0 con 0x7f65d00047c0 2023-10-16T11:27:57.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.966+0000 7f65eea5d640 1 -- 172.21.15.84:0/1663124271 --> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f65e8069da0 con 0x7f65d406b010 2023-10-16T11:27:57.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.967+0000 7f65e4ff9640 1 -- 172.21.15.84:0/1663124271 <== mgr.13719 v2:172.21.15.84:6832/3003264401 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f65dc004140 con 0x7f65d406b010 2023-10-16T11:27:57.968 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 1 to deep-scrub 2023-10-16T11:27:57.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.969+0000 7f65eea5d640 1 -- 172.21.15.84:0/1663124271 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f65d406b010 msgr2=0x7f65d406d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.969+0000 7f65eea5d640 1 --2- 172.21.15.84:0/1663124271 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f65d406b010 0x7f65d406d4d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f65dc036000 tx=0x7f65dc02e000 comp rx=0 tx=0).stop 2023-10-16T11:27:57.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.969+0000 7f65eea5d640 1 -- 172.21.15.84:0/1663124271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65d00047c0 msgr2=0x7f65e81f3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:57.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.969+0000 7f65eea5d640 1 --2- 172.21.15.84:0/1663124271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65d00047c0 0x7f65e81f3460 secure :-1 s=READY pgs=3551 cs=0 l=1 rev1=1 crypto rx=0x7f65d800d810 tx=0x7f65d800dcd0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.969+0000 7f65eea5d640 1 -- 172.21.15.84:0/1663124271 shutdown_connections 2023-10-16T11:27:57.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.969+0000 7f65eea5d640 1 --2- 172.21.15.84:0/1663124271 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f65d406b010 0x7f65d406d4d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.969+0000 7f65eea5d640 1 --2- 172.21.15.84:0/1663124271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65e81f2a70 0x7f65e8150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.969+0000 7f65eea5d640 1 --2- 172.21.15.84:0/1663124271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e81513d0 0x7f65e80cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.969+0000 7f65eea5d640 1 --2- 172.21.15.84:0/1663124271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65d00047c0 0x7f65e81f3460 unknown :-1 s=CLOSED pgs=3551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:57.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.969+0000 7f65eea5d640 1 -- 172.21.15.84:0/1663124271 >> 172.21.15.84:0/1663124271 conn(0x7f65e80ce9b0 msgr2=0x7f65e815c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:57.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.969+0000 7f65eea5d640 1 -- 172.21.15.84:0/1663124271 shutdown_connections 2023-10-16T11:27:57.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:57.970+0000 7f65eea5d640 1 -- 172.21.15.84:0/1663124271 wait complete. 2023-10-16T11:27:57.989 INFO:tasks.ceph:Scrubbing osd.2 2023-10-16T11:27:57.989 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-16T11:27:58.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.203+0000 7fa3324bb640 1 Processor -- start 2023-10-16T11:27:58.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.204+0000 7fa3324bb640 1 -- start start 2023-10-16T11:27:58.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.205+0000 7fa3324bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa32c15cf60 0x7fa32c06f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:58.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.205+0000 7fa3324bb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa32c0b0360 0x7fa32c0b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:58.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.205+0000 7fa3324bb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa32c0c9620 con 0x7fa32c06f9d0 2023-10-16T11:27:58.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.205+0000 7fa3324bb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32c15d340 con 0x7fa32c0b0360 2023-10-16T11:27:58.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.205+0000 7fa3324bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c0cc860 con 0x7fa32c15cf60 2023-10-16T11:27:58.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.206+0000 7fa330a58640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa32c0b0360 0x7fa32c0b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:58.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.206+0000 7fa330a58640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa32c0b0360 0x7fa32c0b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52382/0 (socket says 172.21.15.84:52382) 2023-10-16T11:27:58.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.206+0000 7fa330a58640 1 -- 172.21.15.84:0/3970154827 learned_addr learned my addr 172.21.15.84:0/3970154827 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:58.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.206+0000 7fa32bfff640 1 --2- 172.21.15.84:0/3970154827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa32c15cf60 0x7fa32c06f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:58.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.206+0000 7fa330a58640 1 -- 172.21.15.84:0/3970154827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa32c15cf60 msgr2=0x7fa32c06f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:58.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.206+0000 7fa330a58640 1 --2- 172.21.15.84:0/3970154827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa32c15cf60 0x7fa32c06f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.206+0000 7fa330a58640 1 -- 172.21.15.84:0/3970154827 >> v1:172.21.15.157:6789/0 conn(0x7fa32c06f9d0 legacy=0x7fa32c0c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:58.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.206+0000 7fa330a58640 1 -- 172.21.15.84:0/3970154827 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c0ad4a0 con 0x7fa32c0b0360 2023-10-16T11:27:58.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.207+0000 7fa330a58640 1 --2- 172.21.15.84:0/3970154827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa32c0b0360 0x7fa32c0b07f0 secure :-1 s=READY pgs=3552 cs=0 l=1 rev1=1 crypto rx=0x7fa324009e50 tx=0x7fa32402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98e137a770cb025 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:58.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.207+0000 7fa32b7fe640 1 -- 172.21.15.84:0/3970154827 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32402ca70 con 0x7fa32c0b0360 2023-10-16T11:27:58.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.207+0000 7fa32b7fe640 1 -- 172.21.15.84:0/3970154827 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32402cbd0 con 0x7fa32c0b0360 2023-10-16T11:27:58.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.207+0000 7fa32b7fe640 1 -- 172.21.15.84:0/3970154827 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3240343f0 con 0x7fa32c0b0360 2023-10-16T11:27:58.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.208+0000 7fa3324bb640 1 -- 172.21.15.84:0/3970154827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa32c0b0360 msgr2=0x7fa32c0b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.208+0000 7fa3324bb640 1 --2- 172.21.15.84:0/3970154827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa32c0b0360 0x7fa32c0b07f0 secure :-1 s=READY pgs=3552 cs=0 l=1 rev1=1 crypto rx=0x7fa324009e50 tx=0x7fa32402bed0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.208+0000 7fa3324bb640 1 -- 172.21.15.84:0/3970154827 shutdown_connections 2023-10-16T11:27:58.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.208+0000 7fa3324bb640 1 --2- 172.21.15.84:0/3970154827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa32c0b0360 0x7fa32c0b07f0 unknown :-1 s=CLOSED pgs=3552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.208+0000 7fa3324bb640 1 --2- 172.21.15.84:0/3970154827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa32c15cf60 0x7fa32c06f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.208+0000 7fa3324bb640 1 -- 172.21.15.84:0/3970154827 >> 172.21.15.84:0/3970154827 conn(0x7fa32c0cb360 msgr2=0x7fa32c0cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:58.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.208+0000 7fa3324bb640 1 -- 172.21.15.84:0/3970154827 shutdown_connections 2023-10-16T11:27:58.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.208+0000 7fa3324bb640 1 -- 172.21.15.84:0/3970154827 wait complete. 2023-10-16T11:27:58.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.210+0000 7fa3324bb640 1 Processor -- start 2023-10-16T11:27:58.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.210+0000 7fa3324bb640 1 -- start start 2023-10-16T11:27:58.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.211+0000 7fa3324bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa32c06f9d0 0x7fa32c15fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:58.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.211+0000 7fa3324bb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa32c0b0360 0x7fa32c14fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:58.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.211+0000 7fa3324bb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa32c15cf60 0x7fa32c0b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:58.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.211+0000 7fa3324bb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c0c5170 con 0x7fa32c15cf60 2023-10-16T11:27:58.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.211+0000 7fa3324bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c0c2410 con 0x7fa32c06f9d0 2023-10-16T11:27:58.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.211+0000 7fa3324bb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c0c26b0 con 0x7fa32c0b0360 2023-10-16T11:27:58.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.212+0000 7fa330a58640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa32c15cf60 0x7fa32c0b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:58.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.212+0000 7fa330a58640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa32c15cf60 0x7fa32c0b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52390/0 (socket says 172.21.15.84:52390) 2023-10-16T11:27:58.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.212+0000 7fa330a58640 1 -- 172.21.15.84:0/1901428352 learned_addr learned my addr 172.21.15.84:0/1901428352 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:58.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.212+0000 7fa323fff640 1 --2- 172.21.15.84:0/1901428352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa32c0b0360 0x7fa32c14fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:58.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.212+0000 7fa32bfff640 1 --2- 172.21.15.84:0/1901428352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa32c06f9d0 0x7fa32c15fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:58.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.212+0000 7fa330a58640 1 -- 172.21.15.84:0/1901428352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa32c0b0360 msgr2=0x7fa32c14fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.212+0000 7fa330a58640 1 --2- 172.21.15.84:0/1901428352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa32c0b0360 0x7fa32c14fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.212+0000 7fa330a58640 1 -- 172.21.15.84:0/1901428352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa32c06f9d0 msgr2=0x7fa32c15fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.212+0000 7fa330a58640 1 --2- 172.21.15.84:0/1901428352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa32c06f9d0 0x7fa32c15fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.212+0000 7fa330a58640 1 -- 172.21.15.84:0/1901428352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa324009b30 con 0x7fa32c15cf60 2023-10-16T11:27:58.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.212+0000 7fa330a58640 1 --2- 172.21.15.84:0/1901428352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa32c15cf60 0x7fa32c0b3380 secure :-1 s=READY pgs=3553 cs=0 l=1 rev1=1 crypto rx=0x7fa324036040 tx=0x7fa324004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:58.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.213+0000 7fa3297fa640 1 -- 172.21.15.84:0/1901428352 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324009510 con 0x7fa32c15cf60 2023-10-16T11:27:58.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.213+0000 7fa3297fa640 1 -- 172.21.15.84:0/1901428352 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32403e450 con 0x7fa32c15cf60 2023-10-16T11:27:58.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.213+0000 7fa3297fa640 1 -- 172.21.15.84:0/1901428352 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32403e7e0 con 0x7fa32c15cf60 2023-10-16T11:27:58.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.213+0000 7fa3324bb640 1 -- 172.21.15.84:0/1901428352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32c0c2990 con 0x7fa32c15cf60 2023-10-16T11:27:58.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.213+0000 7fa3324bb640 1 -- 172.21.15.84:0/1901428352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c0c4e00 con 0x7fa32c15cf60 2023-10-16T11:27:58.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.214+0000 7fa3324bb640 1 -- 172.21.15.84:0/1901428352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa32c0c92b0 con 0x7fa32c15cf60 2023-10-16T11:27:58.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.215+0000 7fa3297fa640 1 -- 172.21.15.84:0/1901428352 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa32403e940 con 0x7fa32c15cf60 2023-10-16T11:27:58.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.216+0000 7fa3297fa640 1 --2- 172.21.15.84:0/1901428352 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fa30c06af90 0x7fa30c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:58.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.216+0000 7fa3297fa640 1 -- 172.21.15.84:0/1901428352 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fa3240ac520 con 0x7fa32c15cf60 2023-10-16T11:27:58.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.216+0000 7fa3297fa640 1 --2- 172.21.15.84:0/1901428352 >> [v2:172.21.15.84:6818/338331808,v1:172.21.15.84:6821/338331808] conn(0x7fa30c0713e0 0x7fa30c073800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:58.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.216+0000 7fa3297fa640 1 -- 172.21.15.84:0/1901428352 --> [v2:172.21.15.84:6818/338331808,v1:172.21.15.84:6821/338331808] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa30c073ed0 con 0x7fa30c0713e0 2023-10-16T11:27:58.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.216+0000 7fa3297fa640 1 -- 172.21.15.84:0/1901428352 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=275) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3240aeb00 con 0x7fa32c15cf60 2023-10-16T11:27:58.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.216+0000 7fa32bfff640 1 --2- 172.21.15.84:0/1901428352 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fa30c06af90 0x7fa30c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:58.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.216+0000 7fa323fff640 1 --2- 172.21.15.84:0/1901428352 >> [v2:172.21.15.84:6818/338331808,v1:172.21.15.84:6821/338331808] conn(0x7fa30c0713e0 0x7fa30c073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:58.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.217+0000 7fa32bfff640 1 --2- 172.21.15.84:0/1901428352 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fa30c06af90 0x7fa30c06d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa31c006fd0 tx=0x7fa31c008040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:58.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.217+0000 7fa323fff640 1 --2- 172.21.15.84:0/1901428352 >> [v2:172.21.15.84:6818/338331808,v1:172.21.15.84:6821/338331808] conn(0x7fa30c0713e0 0x7fa30c073800 crc :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:58.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.217+0000 7fa3297fa640 1 -- 172.21.15.84:0/1901428352 <== osd.2 v2:172.21.15.84:6818/338331808 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fa30c073ed0 con 0x7fa30c0713e0 2023-10-16T11:27:58.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.231+0000 7fa3324bb640 1 -- 172.21.15.84:0/1901428352 --> [v2:172.21.15.84:6818/338331808,v1:172.21.15.84:6821/338331808] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa32c0c8fd0 con 0x7fa30c0713e0 2023-10-16T11:27:58.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.232+0000 7fa3297fa640 1 -- 172.21.15.84:0/1901428352 <== osd.2 v2:172.21.15.84:6818/338331808 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa31400a380 con 0x7fa30c0713e0 2023-10-16T11:27:58.233 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:27:58.233 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T11:27:58.234 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:27:58.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.232+0000 7fa3324bb640 1 -- 172.21.15.84:0/1901428352 >> [v2:172.21.15.84:6818/338331808,v1:172.21.15.84:6821/338331808] conn(0x7fa30c0713e0 msgr2=0x7fa30c073800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.232+0000 7fa3324bb640 1 --2- 172.21.15.84:0/1901428352 >> [v2:172.21.15.84:6818/338331808,v1:172.21.15.84:6821/338331808] conn(0x7fa30c0713e0 0x7fa30c073800 crc :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.232+0000 7fa3324bb640 1 -- 172.21.15.84:0/1901428352 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fa30c06af90 msgr2=0x7fa30c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.232+0000 7fa3324bb640 1 --2- 172.21.15.84:0/1901428352 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fa30c06af90 0x7fa30c06d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa31c006fd0 tx=0x7fa31c008040 comp rx=0 tx=0).stop 2023-10-16T11:27:58.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.232+0000 7fa3324bb640 1 -- 172.21.15.84:0/1901428352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa32c15cf60 msgr2=0x7fa32c0b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.232+0000 7fa3324bb640 1 --2- 172.21.15.84:0/1901428352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa32c15cf60 0x7fa32c0b3380 secure :-1 s=READY pgs=3553 cs=0 l=1 rev1=1 crypto rx=0x7fa324036040 tx=0x7fa324004a40 comp rx=0 tx=0).stop 2023-10-16T11:27:58.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.232+0000 7fa330a58640 1 -- 172.21.15.84:0/1901428352 reap_dead start 2023-10-16T11:27:58.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.233+0000 7fa3324bb640 1 -- 172.21.15.84:0/1901428352 shutdown_connections 2023-10-16T11:27:58.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.233+0000 7fa3324bb640 1 -- 172.21.15.84:0/1901428352 >> 172.21.15.84:0/1901428352 conn(0x7fa32c0cb360 msgr2=0x7fa32c0ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:58.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.233+0000 7fa3324bb640 1 -- 172.21.15.84:0/1901428352 shutdown_connections 2023-10-16T11:27:58.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.233+0000 7fa3324bb640 1 -- 172.21.15.84:0/1901428352 wait complete. 2023-10-16T11:27:58.251 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-16T11:27:58.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.462+0000 7f83a0d3a640 1 Processor -- start 2023-10-16T11:27:58.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.463+0000 7f83a0d3a640 1 -- start start 2023-10-16T11:27:58.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.464+0000 7f83a0d3a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f839c15ad70 0x7f839c069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:58.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.464+0000 7f83a0d3a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f839c0cb030 con 0x7f839c0c9ca0 2023-10-16T11:27:58.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.464+0000 7f83a0d3a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f839c0c91c0 con 0x7f839c0caac0 2023-10-16T11:27:58.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.464+0000 7f83a0d3a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f839c0c94a0 con 0x7f839c15ad70 2023-10-16T11:27:58.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.464+0000 7f839a59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f839c15ad70 0x7f839c069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:58.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.464+0000 7f8393fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f839c0c9ca0 0x7f839c0ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60532/0 (socket says 172.21.15.84:60532) 2023-10-16T11:27:58.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.464+0000 7f8393fff640 1 -- 172.21.15.84:0/2695286879 learned_addr learned my addr 172.21.15.84:0/2695286879 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:58.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.465+0000 7f839a59c640 1 -- 172.21.15.84:0/2695286879 >> v1:172.21.15.157:6789/0 conn(0x7f839c0caac0 legacy=0x7f839c0ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:58.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.465+0000 7f839a59c640 1 -- 172.21.15.84:0/2695286879 >> v1:172.21.15.84:6789/0 conn(0x7f839c0c9ca0 legacy=0x7f839c0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.465+0000 7f839a59c640 1 -- 172.21.15.84:0/2695286879 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839c0c9780 con 0x7f839c15ad70 2023-10-16T11:27:58.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.465+0000 7f8399d9b640 1 -- 172.21.15.84:0/2695286879 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247764377 0 0) 0x7f839c0cb030 con 0x7f839c0c9ca0 2023-10-16T11:27:58.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.466+0000 7f839a59c640 1 --2- 172.21.15.84:0/2695286879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c15ad70 0x7f839c069da0 secure :-1 s=READY pgs=3555 cs=0 l=1 rev1=1 crypto rx=0x7f838c0097a0 tx=0x7f838c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd398773c30af45 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:58.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.466+0000 7f8399d9b640 1 -- 172.21.15.84:0/2695286879 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c031830 con 0x7f839c15ad70 2023-10-16T11:27:58.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.466+0000 7f8399d9b640 1 -- 172.21.15.84:0/2695286879 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838c031990 con 0x7f839c15ad70 2023-10-16T11:27:58.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.466+0000 7f8399d9b640 1 -- 172.21.15.84:0/2695286879 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c031d20 con 0x7f839c15ad70 2023-10-16T11:27:58.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.467+0000 7f83a0d3a640 1 -- 172.21.15.84:0/2695286879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c15ad70 msgr2=0x7f839c069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.467+0000 7f83a0d3a640 1 --2- 172.21.15.84:0/2695286879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c15ad70 0x7f839c069da0 secure :-1 s=READY pgs=3555 cs=0 l=1 rev1=1 crypto rx=0x7f838c0097a0 tx=0x7f838c02b8d0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.467+0000 7f83a0d3a640 1 -- 172.21.15.84:0/2695286879 shutdown_connections 2023-10-16T11:27:58.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.467+0000 7f83a0d3a640 1 --2- 172.21.15.84:0/2695286879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c15ad70 0x7f839c069da0 unknown :-1 s=CLOSED pgs=3555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.467+0000 7f83a0d3a640 1 -- 172.21.15.84:0/2695286879 >> 172.21.15.84:0/2695286879 conn(0x7f839c0ce9b0 msgr2=0x7f839c0cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:58.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.467+0000 7f83a0d3a640 1 -- 172.21.15.84:0/2695286879 shutdown_connections 2023-10-16T11:27:58.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.467+0000 7f83a0d3a640 1 -- 172.21.15.84:0/2695286879 wait complete. 2023-10-16T11:27:58.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.469+0000 7f83a0d3a640 1 Processor -- start 2023-10-16T11:27:58.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.469+0000 7f83a0d3a640 1 -- start start 2023-10-16T11:27:58.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.470+0000 7f83a0d3a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f839c0c9ca0 0x7f839c148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:58.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.470+0000 7f83a0d3a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c0caac0 0x7f839c0ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:58.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.470+0000 7f83a0d3a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f839c15ad70 0x7f839c0c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:58.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.470+0000 7f83a0d3a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c0ac930 con 0x7f839c0caac0 2023-10-16T11:27:58.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.470+0000 7f83a0d3a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c0acbf0 con 0x7f839c15ad70 2023-10-16T11:27:58.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.470+0000 7f83a0d3a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839c0aced0 con 0x7f839c0c9ca0 2023-10-16T11:27:58.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.470+0000 7f8393fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c0caac0 0x7f839c0ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:58.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.470+0000 7f8393fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c0caac0 0x7f839c0ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52404/0 (socket says 172.21.15.84:52404) 2023-10-16T11:27:58.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.470+0000 7f8393fff640 1 -- 172.21.15.84:0/2321877808 learned_addr learned my addr 172.21.15.84:0/2321877808 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:58.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.470+0000 7f839ad9d640 1 --2- 172.21.15.84:0/2321877808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f839c15ad70 0x7f839c0c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:58.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.470+0000 7f8393fff640 1 -- 172.21.15.84:0/2321877808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f839c0c9ca0 msgr2=0x7f839c148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.470+0000 7f8393fff640 1 --2- 172.21.15.84:0/2321877808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f839c0c9ca0 0x7f839c148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.470+0000 7f8393fff640 1 -- 172.21.15.84:0/2321877808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f839c15ad70 msgr2=0x7f839c0c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.470+0000 7f8393fff640 1 --2- 172.21.15.84:0/2321877808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f839c15ad70 0x7f839c0c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.470+0000 7f8393fff640 1 -- 172.21.15.84:0/2321877808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c009400 con 0x7f839c0caac0 2023-10-16T11:27:58.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.471+0000 7f8393fff640 1 --2- 172.21.15.84:0/2321877808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c0caac0 0x7f839c0ad9b0 secure :-1 s=READY pgs=3556 cs=0 l=1 rev1=1 crypto rx=0x7f838400c4d0 tx=0x7f838400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:58.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.471+0000 7f8392ffd640 1 -- 172.21.15.84:0/2321877808 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83840074e0 con 0x7f839c0caac0 2023-10-16T11:27:58.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.471+0000 7f8392ffd640 1 -- 172.21.15.84:0/2321877808 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8384007660 con 0x7f839c0caac0 2023-10-16T11:27:58.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.471+0000 7f8392ffd640 1 -- 172.21.15.84:0/2321877808 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8384004a20 con 0x7f839c0caac0 2023-10-16T11:27:58.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.471+0000 7f83a0d3a640 1 -- 172.21.15.84:0/2321877808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f839c0ad1b0 con 0x7f839c0caac0 2023-10-16T11:27:58.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.471+0000 7f83a0d3a640 1 -- 172.21.15.84:0/2321877808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f839c1492c0 con 0x7f839c0caac0 2023-10-16T11:27:58.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.472+0000 7f83a0d3a640 1 -- 172.21.15.84:0/2321877808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f839c0ad6d0 con 0x7f839c0caac0 2023-10-16T11:27:58.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.473+0000 7f8392ffd640 1 -- 172.21.15.84:0/2321877808 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8384004b80 con 0x7f839c0caac0 2023-10-16T11:27:58.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.474+0000 7f8392ffd640 1 --2- 172.21.15.84:0/2321877808 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f837c06afc0 0x7f837c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:58.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.474+0000 7f8392ffd640 1 -- 172.21.15.84:0/2321877808 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f83840889b0 con 0x7f839c0caac0 2023-10-16T11:27:58.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.474+0000 7f839a59c640 1 --2- 172.21.15.84:0/2321877808 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f837c06afc0 0x7f837c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:58.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.475+0000 7f839a59c640 1 --2- 172.21.15.84:0/2321877808 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f837c06afc0 0x7f837c06d480 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f838c036000 tx=0x7f838c030000 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:58.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.478+0000 7f8392ffd640 1 -- 172.21.15.84:0/2321877808 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f839c0ad6d0 con 0x7f839c0caac0 2023-10-16T11:27:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.669+0000 7f83a0d3a640 1 -- 172.21.15.84:0/2321877808 --> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f839c06a2e0 con 0x7f837c06afc0 2023-10-16T11:27:58.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.669+0000 7f8392ffd640 1 -- 172.21.15.84:0/2321877808 <== mgr.13719 v2:172.21.15.84:6832/3003264401 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f838c031b00 con 0x7f837c06afc0 2023-10-16T11:27:58.671 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 2 to deep-scrub 2023-10-16T11:27:58.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.672+0000 7f83a0d3a640 1 -- 172.21.15.84:0/2321877808 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f837c06afc0 msgr2=0x7f837c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.672+0000 7f83a0d3a640 1 --2- 172.21.15.84:0/2321877808 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f837c06afc0 0x7f837c06d480 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f838c036000 tx=0x7f838c030000 comp rx=0 tx=0).stop 2023-10-16T11:27:58.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.672+0000 7f83a0d3a640 1 -- 172.21.15.84:0/2321877808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c0caac0 msgr2=0x7f839c0ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.672+0000 7f83a0d3a640 1 --2- 172.21.15.84:0/2321877808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c0caac0 0x7f839c0ad9b0 secure :-1 s=READY pgs=3556 cs=0 l=1 rev1=1 crypto rx=0x7f838400c4d0 tx=0x7f838400c990 comp rx=0 tx=0).stop 2023-10-16T11:27:58.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.672+0000 7f83a0d3a640 1 -- 172.21.15.84:0/2321877808 shutdown_connections 2023-10-16T11:27:58.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.672+0000 7f83a0d3a640 1 --2- 172.21.15.84:0/2321877808 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f837c06afc0 0x7f837c06d480 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.672+0000 7f83a0d3a640 1 --2- 172.21.15.84:0/2321877808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f839c15ad70 0x7f839c0c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.672+0000 7f83a0d3a640 1 --2- 172.21.15.84:0/2321877808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c0caac0 0x7f839c0ad9b0 unknown :-1 s=CLOSED pgs=3556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.672+0000 7f83a0d3a640 1 --2- 172.21.15.84:0/2321877808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f839c0c9ca0 0x7f839c148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.672+0000 7f83a0d3a640 1 -- 172.21.15.84:0/2321877808 >> 172.21.15.84:0/2321877808 conn(0x7f839c0ce9b0 msgr2=0x7f839c0b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:58.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.672+0000 7f83a0d3a640 1 -- 172.21.15.84:0/2321877808 shutdown_connections 2023-10-16T11:27:58.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.672+0000 7f83a0d3a640 1 -- 172.21.15.84:0/2321877808 wait complete. 2023-10-16T11:27:58.692 INFO:tasks.ceph:Scrubbing osd.3 2023-10-16T11:27:58.692 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-16T11:27:58.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.911+0000 7fa23c015640 1 Processor -- start 2023-10-16T11:27:58.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.911+0000 7fa23c015640 1 -- start start 2023-10-16T11:27:58.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.913+0000 7fa23c015640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa23415cf50 0x7fa2340d2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:58.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.913+0000 7fa23c015640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2340d1420 con 0x7fa2340b4750 2023-10-16T11:27:58.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.913+0000 7fa23c015640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa23415d330 con 0x7fa2340b8a10 2023-10-16T11:27:58.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.913+0000 7fa23c015640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2340d1680 con 0x7fa23415cf50 2023-10-16T11:27:58.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.913+0000 7fa239db1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa23415cf50 0x7fa2340d2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:58.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.913+0000 7fa239db1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa23415cf50 0x7fa2340d2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52420/0 (socket says 172.21.15.84:52420) 2023-10-16T11:27:58.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.913+0000 7fa239db1640 1 -- 172.21.15.84:0/3536700473 learned_addr learned my addr 172.21.15.84:0/3536700473 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:58.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.913+0000 7fa23a5b2640 1 --1- 172.21.15.84:0/3536700473 >> v1:172.21.15.84:6789/0 conn(0x7fa2340b4750 0x7fa2340b4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60534/0 (socket says 172.21.15.84:60534) 2023-10-16T11:27:58.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.914+0000 7fa239db1640 1 -- 172.21.15.84:0/3536700473 >> v1:172.21.15.157:6789/0 conn(0x7fa2340b8a10 legacy=0x7fa2340d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:58.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.914+0000 7fa239db1640 1 -- 172.21.15.84:0/3536700473 >> v1:172.21.15.84:6789/0 conn(0x7fa2340b4750 legacy=0x7fa2340b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.914+0000 7fa239db1640 1 -- 172.21.15.84:0/3536700473 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340d1960 con 0x7fa23415cf50 2023-10-16T11:27:58.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.915+0000 7fa239db1640 1 --2- 172.21.15.84:0/3536700473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa23415cf50 0x7fa2340d2720 secure :-1 s=READY pgs=3558 cs=0 l=1 rev1=1 crypto rx=0x7fa224009e50 tx=0x7fa22402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713f422c1b28262 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:58.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.915+0000 7fa238daf640 1 -- 172.21.15.84:0/3536700473 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22402cb10 con 0x7fa23415cf50 2023-10-16T11:27:58.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.915+0000 7fa238daf640 1 -- 172.21.15.84:0/3536700473 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22402cc70 con 0x7fa23415cf50 2023-10-16T11:27:58.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.915+0000 7fa238daf640 1 -- 172.21.15.84:0/3536700473 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2240343f0 con 0x7fa23415cf50 2023-10-16T11:27:58.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.916+0000 7fa23c015640 1 -- 172.21.15.84:0/3536700473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa23415cf50 msgr2=0x7fa2340d2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.916+0000 7fa23c015640 1 --2- 172.21.15.84:0/3536700473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa23415cf50 0x7fa2340d2720 secure :-1 s=READY pgs=3558 cs=0 l=1 rev1=1 crypto rx=0x7fa224009e50 tx=0x7fa22402bf70 comp rx=0 tx=0).stop 2023-10-16T11:27:58.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.916+0000 7fa23c015640 1 -- 172.21.15.84:0/3536700473 shutdown_connections 2023-10-16T11:27:58.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.916+0000 7fa23c015640 1 --2- 172.21.15.84:0/3536700473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa23415cf50 0x7fa2340d2720 unknown :-1 s=CLOSED pgs=3558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.916+0000 7fa23c015640 1 -- 172.21.15.84:0/3536700473 >> 172.21.15.84:0/3536700473 conn(0x7fa2340cf750 msgr2=0x7fa2340cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:58.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.916+0000 7fa23c015640 1 -- 172.21.15.84:0/3536700473 shutdown_connections 2023-10-16T11:27:58.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.916+0000 7fa23c015640 1 -- 172.21.15.84:0/3536700473 wait complete. 2023-10-16T11:27:58.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.918+0000 7fa23c015640 1 Processor -- start 2023-10-16T11:27:58.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.918+0000 7fa23c015640 1 -- start start 2023-10-16T11:27:58.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.919+0000 7fa23c015640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2340b4750 0x7fa2340c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:58.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.919+0000 7fa23c015640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2340b8a10 0x7fa23415d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:58.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.919+0000 7fa23c015640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa23415cf50 0x7fa23415b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:58.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.919+0000 7fa23c015640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23415df00 con 0x7fa2340b4750 2023-10-16T11:27:58.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.919+0000 7fa23c015640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23415e1e0 con 0x7fa23415cf50 2023-10-16T11:27:58.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.919+0000 7fa23c015640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23415e4c0 con 0x7fa2340b8a10 2023-10-16T11:27:58.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.919+0000 7fa239db1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2340b4750 0x7fa2340c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:58.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.919+0000 7fa239db1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2340b4750 0x7fa2340c8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52422/0 (socket says 172.21.15.84:52422) 2023-10-16T11:27:58.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.919+0000 7fa239db1640 1 -- 172.21.15.84:0/1209062549 learned_addr learned my addr 172.21.15.84:0/1209062549 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:58.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.920+0000 7fa23a5b2640 1 --2- 172.21.15.84:0/1209062549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa23415cf50 0x7fa23415b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:58.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.920+0000 7fa239db1640 1 -- 172.21.15.84:0/1209062549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2340b8a10 msgr2=0x7fa23415d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.920+0000 7fa2395b0640 1 --2- 172.21.15.84:0/1209062549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2340b8a10 0x7fa23415d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:58.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.920+0000 7fa239db1640 1 --2- 172.21.15.84:0/1209062549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2340b8a10 0x7fa23415d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.920+0000 7fa239db1640 1 -- 172.21.15.84:0/1209062549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa23415cf50 msgr2=0x7fa23415b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.920+0000 7fa239db1640 1 --2- 172.21.15.84:0/1209062549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa23415cf50 0x7fa23415b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.920+0000 7fa239db1640 1 -- 172.21.15.84:0/1209062549 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa224009b30 con 0x7fa2340b4750 2023-10-16T11:27:58.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.920+0000 7fa23a5b2640 1 --2- 172.21.15.84:0/1209062549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa23415cf50 0x7fa23415b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:58.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.920+0000 7fa239db1640 1 --2- 172.21.15.84:0/1209062549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2340b4750 0x7fa2340c8910 secure :-1 s=READY pgs=3559 cs=0 l=1 rev1=1 crypto rx=0x7fa224009e50 tx=0x7fa224004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:58.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.920+0000 7fa2327fc640 1 -- 172.21.15.84:0/1209062549 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22403e450 con 0x7fa2340b4750 2023-10-16T11:27:58.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.920+0000 7fa2327fc640 1 -- 172.21.15.84:0/1209062549 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22403e5f0 con 0x7fa2340b4750 2023-10-16T11:27:58.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.920+0000 7fa2327fc640 1 -- 172.21.15.84:0/1209062549 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22403e9a0 con 0x7fa2340b4750 2023-10-16T11:27:58.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.920+0000 7fa23c015640 1 -- 172.21.15.84:0/1209062549 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2340c9e40 con 0x7fa2340b4750 2023-10-16T11:27:58.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.921+0000 7fa23c015640 1 -- 172.21.15.84:0/1209062549 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23415fc90 con 0x7fa2340b4750 2023-10-16T11:27:58.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.921+0000 7fa23c015640 1 -- 172.21.15.84:0/1209062549 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa23415fee0 con 0x7fa2340b4750 2023-10-16T11:27:58.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.923+0000 7fa2327fc640 1 -- 172.21.15.84:0/1209062549 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa22403eb00 con 0x7fa2340b4750 2023-10-16T11:27:58.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.924+0000 7fa2327fc640 1 --2- 172.21.15.84:0/1209062549 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fa22006af90 0x7fa22006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:58.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.924+0000 7fa2327fc640 1 -- 172.21.15.84:0/1209062549 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fa2240acda0 con 0x7fa2340b4750 2023-10-16T11:27:58.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.924+0000 7fa2395b0640 1 --2- 172.21.15.84:0/1209062549 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fa22006af90 0x7fa22006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:58.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.924+0000 7fa2327fc640 1 --2- 172.21.15.84:0/1209062549 >> [v2:172.21.15.84:6816/1933430277,v1:172.21.15.84:6817/1933430277] conn(0x7fa2200713e0 0x7fa220073800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:58.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.924+0000 7fa2327fc640 1 -- 172.21.15.84:0/1209062549 --> [v2:172.21.15.84:6816/1933430277,v1:172.21.15.84:6817/1933430277] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa220073ed0 con 0x7fa2200713e0 2023-10-16T11:27:58.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.924+0000 7fa2327fc640 1 -- 172.21.15.84:0/1209062549 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=275) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2240ad110 con 0x7fa2340b4750 2023-10-16T11:27:58.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.924+0000 7fa23a5b2640 1 --2- 172.21.15.84:0/1209062549 >> [v2:172.21.15.84:6816/1933430277,v1:172.21.15.84:6817/1933430277] conn(0x7fa2200713e0 0x7fa220073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:58.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.925+0000 7fa2395b0640 1 --2- 172.21.15.84:0/1209062549 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fa22006af90 0x7fa22006d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa2340b80c0 tx=0x7fa228009040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:58.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.925+0000 7fa23a5b2640 1 --2- 172.21.15.84:0/1209062549 >> [v2:172.21.15.84:6816/1933430277,v1:172.21.15.84:6817/1933430277] conn(0x7fa2200713e0 0x7fa220073800 crc :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:58.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.926+0000 7fa2327fc640 1 -- 172.21.15.84:0/1209062549 <== osd.3 v2:172.21.15.84:6816/1933430277 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fa220073ed0 con 0x7fa2200713e0 2023-10-16T11:27:58.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.940+0000 7fa23c015640 1 -- 172.21.15.84:0/1209062549 --> [v2:172.21.15.84:6816/1933430277,v1:172.21.15.84:6817/1933430277] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa23415e350 con 0x7fa2200713e0 2023-10-16T11:27:58.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.940+0000 7fa2327fc640 1 -- 172.21.15.84:0/1209062549 <== osd.3 v2:172.21.15.84:6816/1933430277 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa23415e350 con 0x7fa2200713e0 2023-10-16T11:27:58.942 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:27:58.942 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T11:27:58.942 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:27:58.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.941+0000 7fa23c015640 1 -- 172.21.15.84:0/1209062549 >> [v2:172.21.15.84:6816/1933430277,v1:172.21.15.84:6817/1933430277] conn(0x7fa2200713e0 msgr2=0x7fa220073800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.941+0000 7fa23c015640 1 --2- 172.21.15.84:0/1209062549 >> [v2:172.21.15.84:6816/1933430277,v1:172.21.15.84:6817/1933430277] conn(0x7fa2200713e0 0x7fa220073800 crc :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:58.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.941+0000 7fa23c015640 1 -- 172.21.15.84:0/1209062549 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fa22006af90 msgr2=0x7fa22006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.941+0000 7fa23c015640 1 --2- 172.21.15.84:0/1209062549 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fa22006af90 0x7fa22006d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa2340b80c0 tx=0x7fa228009040 comp rx=0 tx=0).stop 2023-10-16T11:27:58.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.941+0000 7fa23c015640 1 -- 172.21.15.84:0/1209062549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2340b4750 msgr2=0x7fa2340c8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:58.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.941+0000 7fa23c015640 1 --2- 172.21.15.84:0/1209062549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2340b4750 0x7fa2340c8910 secure :-1 s=READY pgs=3559 cs=0 l=1 rev1=1 crypto rx=0x7fa224009e50 tx=0x7fa224004a40 comp rx=0 tx=0).stop 2023-10-16T11:27:58.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.941+0000 7fa23a5b2640 1 -- 172.21.15.84:0/1209062549 reap_dead start 2023-10-16T11:27:58.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.941+0000 7fa23c015640 1 -- 172.21.15.84:0/1209062549 shutdown_connections 2023-10-16T11:27:58.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.941+0000 7fa23c015640 1 -- 172.21.15.84:0/1209062549 >> 172.21.15.84:0/1209062549 conn(0x7fa2340cf750 msgr2=0x7fa2340b5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:58.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.941+0000 7fa23c015640 1 -- 172.21.15.84:0/1209062549 shutdown_connections 2023-10-16T11:27:58.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:58.941+0000 7fa23c015640 1 -- 172.21.15.84:0/1209062549 wait complete. 2023-10-16T11:27:58.959 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-16T11:27:59.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.164+0000 7f3b1ffff640 1 Processor -- start 2023-10-16T11:27:59.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.164+0000 7f3b1ffff640 1 -- start start 2023-10-16T11:27:59.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.166+0000 7f3b1ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b2015c9f0 0x7f3b200c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.166+0000 7f3b1ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3b2015dde0 0x7f3b200ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.166+0000 7f3b1ffff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b20160a90 con 0x7f3b200c9230 2023-10-16T11:27:59.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.166+0000 7f3b1ffff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b200cdec0 con 0x7f3b2015dde0 2023-10-16T11:27:59.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.166+0000 7f3b1ffff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b200cd900 con 0x7f3b2015c9f0 2023-10-16T11:27:59.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.167+0000 7f3b1f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3b2015dde0 0x7f3b200ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.167+0000 7f3b167fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3b200c9230 0x7f3b200709e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60548/0 (socket says 172.21.15.84:60548) 2023-10-16T11:27:59.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.167+0000 7f3b167fc640 1 -- 172.21.15.84:0/1825818866 learned_addr learned my addr 172.21.15.84:0/1825818866 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:59.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.167+0000 7f3b1effd640 1 --2- 172.21.15.84:0/1825818866 >> v2:172.21.15.157:3300/0 conn(0x7f3b2015c9f0 0x7f3b200c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.167+0000 7f3b1f7fe640 1 -- 172.21.15.84:0/1825818866 >> v2:172.21.15.157:3300/0 conn(0x7f3b2015c9f0 msgr2=0x7f3b200c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:59.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.167+0000 7f3b1f7fe640 1 --2- 172.21.15.84:0/1825818866 >> v2:172.21.15.157:3300/0 conn(0x7f3b2015c9f0 0x7f3b200c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.167+0000 7f3b1f7fe640 1 -- 172.21.15.84:0/1825818866 >> v1:172.21.15.84:6789/0 conn(0x7f3b200c9230 legacy=0x7f3b200709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:59.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.167+0000 7f3b1f7fe640 1 -- 172.21.15.84:0/1825818866 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b200cdbe0 con 0x7f3b2015dde0 2023-10-16T11:27:59.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.167+0000 7f3b1e7fc640 1 -- 172.21.15.84:0/1825818866 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 927972739 0 0) 0x7f3b20160a90 con 0x7f3b200c9230 2023-10-16T11:27:59.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.168+0000 7f3b1f7fe640 1 --2- 172.21.15.84:0/1825818866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2015dde0 0x7f3b200ac890 secure :-1 s=READY pgs=3561 cs=0 l=1 rev1=1 crypto rx=0x7f3b200ad4e0 tx=0x7f3b1802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04acc2f9c1f0c6f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:59.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.168+0000 7f3b1e7fc640 1 -- 172.21.15.84:0/1825818866 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1802cc70 con 0x7f3b2015dde0 2023-10-16T11:27:59.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.168+0000 7f3b1e7fc640 1 -- 172.21.15.84:0/1825818866 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1802cdd0 con 0x7f3b2015dde0 2023-10-16T11:27:59.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.168+0000 7f3b1e7fc640 1 -- 172.21.15.84:0/1825818866 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18034660 con 0x7f3b2015dde0 2023-10-16T11:27:59.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.169+0000 7f3b1ffff640 1 -- 172.21.15.84:0/1825818866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2015dde0 msgr2=0x7f3b200ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:59.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.169+0000 7f3b1ffff640 1 --2- 172.21.15.84:0/1825818866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2015dde0 0x7f3b200ac890 secure :-1 s=READY pgs=3561 cs=0 l=1 rev1=1 crypto rx=0x7f3b200ad4e0 tx=0x7f3b1802bec0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.169+0000 7f3b1ffff640 1 -- 172.21.15.84:0/1825818866 shutdown_connections 2023-10-16T11:27:59.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.169+0000 7f3b1ffff640 1 --2- 172.21.15.84:0/1825818866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2015dde0 0x7f3b200ac890 unknown :-1 s=CLOSED pgs=3561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.169+0000 7f3b1ffff640 1 --2- 172.21.15.84:0/1825818866 >> v2:172.21.15.157:3300/0 conn(0x7f3b2015c9f0 0x7f3b200c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.169+0000 7f3b1ffff640 1 -- 172.21.15.84:0/1825818866 >> 172.21.15.84:0/1825818866 conn(0x7f3b200ce6a0 msgr2=0x7f3b200ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:59.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.169+0000 7f3b1ffff640 1 -- 172.21.15.84:0/1825818866 shutdown_connections 2023-10-16T11:27:59.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.169+0000 7f3b1ffff640 1 -- 172.21.15.84:0/1825818866 wait complete. 2023-10-16T11:27:59.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.171+0000 7f3b1ffff640 1 Processor -- start 2023-10-16T11:27:59.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.171+0000 7f3b1ffff640 1 -- start start 2023-10-16T11:27:59.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.172+0000 7f3b1ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b200c9230 0x7f3b2014da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.172+0000 7f3b1ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2015c9f0 0x7f3b200c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.172+0000 7f3b1ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2015dde0 0x7f3b200c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.172+0000 7f3b1ffff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200c6dd0 con 0x7f3b2015c9f0 2023-10-16T11:27:59.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.172+0000 7f3b1ffff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200c6150 con 0x7f3b2015dde0 2023-10-16T11:27:59.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.172+0000 7f3b1ffff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200c6430 con 0x7f3b200c9230 2023-10-16T11:27:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.172+0000 7f3b167fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2015c9f0 0x7f3b200c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.172+0000 7f3b167fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2015c9f0 0x7f3b200c5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52440/0 (socket says 172.21.15.84:52440) 2023-10-16T11:27:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.172+0000 7f3b167fc640 1 -- 172.21.15.84:0/4204014879 learned_addr learned my addr 172.21.15.84:0/4204014879 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.173+0000 7f3b1effd640 1 --2- 172.21.15.84:0/4204014879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b200c9230 0x7f3b2014da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.173+0000 7f3b1f7fe640 1 --2- 172.21.15.84:0/4204014879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2015dde0 0x7f3b200c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.173+0000 7f3b167fc640 1 -- 172.21.15.84:0/4204014879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b200c9230 msgr2=0x7f3b2014da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:59.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.173+0000 7f3b167fc640 1 --2- 172.21.15.84:0/4204014879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b200c9230 0x7f3b2014da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.173+0000 7f3b167fc640 1 -- 172.21.15.84:0/4204014879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2015dde0 msgr2=0x7f3b200c43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:59.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.173+0000 7f3b167fc640 1 --2- 172.21.15.84:0/4204014879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2015dde0 0x7f3b200c43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.173+0000 7f3b167fc640 1 -- 172.21.15.84:0/4204014879 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b18009a70 con 0x7f3b2015c9f0 2023-10-16T11:27:59.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.173+0000 7f3b1f7fe640 1 --2- 172.21.15.84:0/4204014879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2015dde0 0x7f3b200c43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:27:59.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.173+0000 7f3b167fc640 1 --2- 172.21.15.84:0/4204014879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2015c9f0 0x7f3b200c5770 secure :-1 s=READY pgs=3562 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800c4d0 tx=0x7f3b0800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:59.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.173+0000 7f3b17fff640 1 -- 172.21.15.84:0/4204014879 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080074e0 con 0x7f3b2015c9f0 2023-10-16T11:27:59.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.173+0000 7f3b17fff640 1 -- 172.21.15.84:0/4204014879 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b08007660 con 0x7f3b2015c9f0 2023-10-16T11:27:59.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.173+0000 7f3b17fff640 1 -- 172.21.15.84:0/4204014879 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b08004a20 con 0x7f3b2015c9f0 2023-10-16T11:27:59.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.173+0000 7f3b1ffff640 1 -- 172.21.15.84:0/4204014879 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b200c6710 con 0x7f3b2015c9f0 2023-10-16T11:27:59.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.174+0000 7f3b1ffff640 1 -- 172.21.15.84:0/4204014879 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2014dfc0 con 0x7f3b2015c9f0 2023-10-16T11:27:59.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.175+0000 7f3b1ffff640 1 -- 172.21.15.84:0/4204014879 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2004e9b0 con 0x7f3b2015c9f0 2023-10-16T11:27:59.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.176+0000 7f3b17fff640 1 -- 172.21.15.84:0/4204014879 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3b08004b80 con 0x7f3b2015c9f0 2023-10-16T11:27:59.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.177+0000 7f3b17fff640 1 --2- 172.21.15.84:0/4204014879 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f3b0006af90 0x7f3b0006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.177+0000 7f3b17fff640 1 -- 172.21.15.84:0/4204014879 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f3b080888f0 con 0x7f3b2015c9f0 2023-10-16T11:27:59.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.177+0000 7f3b1effd640 1 --2- 172.21.15.84:0/4204014879 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f3b0006af90 0x7f3b0006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.178+0000 7f3b1effd640 1 --2- 172.21.15.84:0/4204014879 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f3b0006af90 0x7f3b0006d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b10006fd0 tx=0x7f3b10008040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:59.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.181+0000 7f3b17fff640 1 -- 172.21.15.84:0/4204014879 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3b080538c0 con 0x7f3b2015c9f0 2023-10-16T11:27:59.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.373+0000 7f3b1ffff640 1 -- 172.21.15.84:0/4204014879 --> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b200704c0 con 0x7f3b0006af90 2023-10-16T11:27:59.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.373+0000 7f3b17fff640 1 -- 172.21.15.84:0/4204014879 <== mgr.13719 v2:172.21.15.84:6832/3003264401 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3b10009d80 con 0x7f3b0006af90 2023-10-16T11:27:59.375 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 3 to deep-scrub 2023-10-16T11:27:59.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.376+0000 7f3b1ffff640 1 -- 172.21.15.84:0/4204014879 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f3b0006af90 msgr2=0x7f3b0006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:59.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.376+0000 7f3b1ffff640 1 --2- 172.21.15.84:0/4204014879 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f3b0006af90 0x7f3b0006d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b10006fd0 tx=0x7f3b10008040 comp rx=0 tx=0).stop 2023-10-16T11:27:59.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.376+0000 7f3b1ffff640 1 -- 172.21.15.84:0/4204014879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2015c9f0 msgr2=0x7f3b200c5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:59.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.376+0000 7f3b1ffff640 1 --2- 172.21.15.84:0/4204014879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2015c9f0 0x7f3b200c5770 secure :-1 s=READY pgs=3562 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800c4d0 tx=0x7f3b0800c990 comp rx=0 tx=0).stop 2023-10-16T11:27:59.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.376+0000 7f3b1ffff640 1 -- 172.21.15.84:0/4204014879 shutdown_connections 2023-10-16T11:27:59.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.376+0000 7f3b1ffff640 1 --2- 172.21.15.84:0/4204014879 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f3b0006af90 0x7f3b0006d450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.376+0000 7f3b1ffff640 1 --2- 172.21.15.84:0/4204014879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2015dde0 0x7f3b200c43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.376+0000 7f3b1ffff640 1 --2- 172.21.15.84:0/4204014879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b2015c9f0 0x7f3b200c5770 unknown :-1 s=CLOSED pgs=3562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.376+0000 7f3b1ffff640 1 --2- 172.21.15.84:0/4204014879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b200c9230 0x7f3b2014da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.376+0000 7f3b1ffff640 1 -- 172.21.15.84:0/4204014879 >> 172.21.15.84:0/4204014879 conn(0x7f3b200ce6a0 msgr2=0x7f3b2015f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:59.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.376+0000 7f3b1ffff640 1 -- 172.21.15.84:0/4204014879 shutdown_connections 2023-10-16T11:27:59.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.376+0000 7f3b1ffff640 1 -- 172.21.15.84:0/4204014879 wait complete. 2023-10-16T11:27:59.396 INFO:tasks.ceph:Scrubbing osd.4 2023-10-16T11:27:59.397 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-16T11:27:59.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.612+0000 7f90890cf640 1 Processor -- start 2023-10-16T11:27:59.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.612+0000 7f90890cf640 1 -- start start 2023-10-16T11:27:59.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.614+0000 7f90890cf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f908415cf60 0x7f908406f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.614+0000 7f90890cf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90840b0360 0x7f90840b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.614+0000 7f90890cf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90840c9620 con 0x7f908406f9d0 2023-10-16T11:27:59.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.614+0000 7f90890cf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f908415d340 con 0x7f908415cf60 2023-10-16T11:27:59.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.614+0000 7f90890cf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90840cc860 con 0x7f90840b0360 2023-10-16T11:27:59.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.614+0000 7f908259c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f908406f9d0 0x7f90840c9ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60556/0 (socket says 172.21.15.84:60556) 2023-10-16T11:27:59.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.614+0000 7f908259c640 1 -- 172.21.15.84:0/3667592219 learned_addr learned my addr 172.21.15.84:0/3667592219 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:59.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.615+0000 7f9082d9d640 1 --2- 172.21.15.84:0/3667592219 >> v2:172.21.15.157:3300/0 conn(0x7f908415cf60 0x7f908406f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.615+0000 7f908359e640 1 --2- 172.21.15.84:0/3667592219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90840b0360 0x7f90840b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.615+0000 7f9081d9b640 1 -- 172.21.15.84:0/3667592219 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388645984 0 0) 0x7f90840c9620 con 0x7f908406f9d0 2023-10-16T11:27:59.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.615+0000 7f9081d9b640 1 -- 172.21.15.84:0/3667592219 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9070003520 con 0x7f908406f9d0 2023-10-16T11:27:59.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.615+0000 7f9081d9b640 1 -- 172.21.15.84:0/3667592219 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3047653844 0 0) 0x7f9070003520 con 0x7f908406f9d0 2023-10-16T11:27:59.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.615+0000 7f9081d9b640 1 -- 172.21.15.84:0/3667592219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90840b0360 msgr2=0x7f90840b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:59.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.615+0000 7f9081d9b640 1 --2- 172.21.15.84:0/3667592219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90840b0360 0x7f90840b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.615+0000 7f9081d9b640 1 -- 172.21.15.84:0/3667592219 >> v2:172.21.15.157:3300/0 conn(0x7f908415cf60 msgr2=0x7f908406f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:59.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.615+0000 7f9081d9b640 1 --2- 172.21.15.84:0/3667592219 >> v2:172.21.15.157:3300/0 conn(0x7f908415cf60 0x7f908406f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.615+0000 7f9081d9b640 1 -- 172.21.15.84:0/3667592219 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90840ad4a0 con 0x7f908406f9d0 2023-10-16T11:27:59.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.615+0000 7f9081d9b640 1 -- 172.21.15.84:0/3667592219 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f906c001670 con 0x7f908406f9d0 2023-10-16T11:27:59.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.615+0000 7f9081d9b640 1 -- 172.21.15.84:0/3667592219 >> v1:172.21.15.84:6789/0 conn(0x7f908406f9d0 legacy=0x7f90840c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:59.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.615+0000 7f9081d9b640 1 --2- 172.21.15.84:0/3667592219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90840b0360 0x7f90700039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.615+0000 7f9081d9b640 1 --2- 172.21.15.84:0/3667592219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9070003fb0 0x7f90700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.615+0000 7f9081d9b640 1 --2- 172.21.15.84:0/3667592219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9070004d00 0x7f9070005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.615+0000 7f9081d9b640 1 -- 172.21.15.84:0/3667592219 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90840cc860 con 0x7f9070003fb0 2023-10-16T11:27:59.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.615+0000 7f9081d9b640 1 -- 172.21.15.84:0/3667592219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907000b980 con 0x7f9070004d00 2023-10-16T11:27:59.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.615+0000 7f9081d9b640 1 -- 172.21.15.84:0/3667592219 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907000bb40 con 0x7f90840b0360 2023-10-16T11:27:59.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.616+0000 7f908259c640 1 --2- 172.21.15.84:0/3667592219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9070003fb0 0x7f90700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.616+0000 7f908359e640 1 --2- 172.21.15.84:0/3667592219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9070004d00 0x7f9070005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.616+0000 7f9082d9d640 1 --2- 172.21.15.84:0/3667592219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90840b0360 0x7f90700039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.616+0000 7f908259c640 1 -- 172.21.15.84:0/3667592219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90840b0360 msgr2=0x7f90700039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:59.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.616+0000 7f908259c640 1 --2- 172.21.15.84:0/3667592219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90840b0360 0x7f90700039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.616+0000 7f908259c640 1 -- 172.21.15.84:0/3667592219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9070004d00 msgr2=0x7f9070005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:59.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.616+0000 7f908259c640 1 --2- 172.21.15.84:0/3667592219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9070004d00 0x7f9070005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.616+0000 7f908259c640 1 -- 172.21.15.84:0/3667592219 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9070003520 con 0x7f9070003fb0 2023-10-16T11:27:59.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.617+0000 7f908259c640 1 --2- 172.21.15.84:0/3667592219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9070003fb0 0x7f90700047c0 secure :-1 s=READY pgs=3564 cs=0 l=1 rev1=1 crypto rx=0x7f906c00c0e0 tx=0x7f90700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70041cf47c033752 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:59.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.617+0000 7f9081d9b640 1 -- 172.21.15.84:0/3667592219 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c005ad0 con 0x7f9070003fb0 2023-10-16T11:27:59.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.617+0000 7f9081d9b640 1 -- 172.21.15.84:0/3667592219 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906c005c30 con 0x7f9070003fb0 2023-10-16T11:27:59.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.617+0000 7f9081d9b640 1 -- 172.21.15.84:0/3667592219 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c02ca50 con 0x7f9070003fb0 2023-10-16T11:27:59.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.618+0000 7f90890cf640 1 -- 172.21.15.84:0/3667592219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9070003fb0 msgr2=0x7f90700047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:59.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.618+0000 7f90890cf640 1 --2- 172.21.15.84:0/3667592219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9070003fb0 0x7f90700047c0 secure :-1 s=READY pgs=3564 cs=0 l=1 rev1=1 crypto rx=0x7f906c00c0e0 tx=0x7f90700057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.618+0000 7f908359e640 1 -- 172.21.15.84:0/3667592219 reap_dead start 2023-10-16T11:27:59.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.618+0000 7f90890cf640 1 -- 172.21.15.84:0/3667592219 shutdown_connections 2023-10-16T11:27:59.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.618+0000 7f90890cf640 1 -- 172.21.15.84:0/3667592219 >> 172.21.15.84:0/3667592219 conn(0x7f90840cb360 msgr2=0x7f90840cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:59.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.618+0000 7f90890cf640 1 -- 172.21.15.84:0/3667592219 shutdown_connections 2023-10-16T11:27:59.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.618+0000 7f90890cf640 1 -- 172.21.15.84:0/3667592219 wait complete. 2023-10-16T11:27:59.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.620+0000 7f90890cf640 1 Processor -- start 2023-10-16T11:27:59.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.620+0000 7f90890cf640 1 -- start start 2023-10-16T11:27:59.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.621+0000 7f90890cf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9070003fb0 0x7f9084070c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.621+0000 7f90890cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908415ff40 0x7f90841f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.621+0000 7f90890cf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f908415d340 0x7f908415d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.621+0000 7f90890cf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908415dd00 con 0x7f908415ff40 2023-10-16T11:27:59.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.621+0000 7f90890cf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9084160770 con 0x7f908415d340 2023-10-16T11:27:59.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.621+0000 7f90890cf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9084160a50 con 0x7f9070003fb0 2023-10-16T11:27:59.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.621+0000 7f908259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908415ff40 0x7f90841f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.621+0000 7f908259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908415ff40 0x7f90841f1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52460/0 (socket says 172.21.15.84:52460) 2023-10-16T11:27:59.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.621+0000 7f908259c640 1 -- 172.21.15.84:0/108825668 learned_addr learned my addr 172.21.15.84:0/108825668 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:59.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.622+0000 7f9082d9d640 1 --2- 172.21.15.84:0/108825668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9070003fb0 0x7f9084070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.622+0000 7f908359e640 1 --2- 172.21.15.84:0/108825668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f908415d340 0x7f908415d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.622+0000 7f908259c640 1 -- 172.21.15.84:0/108825668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9070003fb0 msgr2=0x7f9084070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:59.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.622+0000 7f908259c640 1 --2- 172.21.15.84:0/108825668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9070003fb0 0x7f9084070c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.622+0000 7f908259c640 1 -- 172.21.15.84:0/108825668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f908415d340 msgr2=0x7f908415d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:59.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.622+0000 7f908259c640 1 --2- 172.21.15.84:0/108825668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f908415d340 0x7f908415d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.622+0000 7f908259c640 1 -- 172.21.15.84:0/108825668 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9084160d30 con 0x7f908415ff40 2023-10-16T11:27:59.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.622+0000 7f908259c640 1 --2- 172.21.15.84:0/108825668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908415ff40 0x7f90841f1fd0 secure :-1 s=READY pgs=3565 cs=0 l=1 rev1=1 crypto rx=0x7f906c02c3b0 tx=0x7f906c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:59.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.622+0000 7f907b7fe640 1 -- 172.21.15.84:0/108825668 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c039070 con 0x7f908415ff40 2023-10-16T11:27:59.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.622+0000 7f907b7fe640 1 -- 172.21.15.84:0/108825668 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906c02cc20 con 0x7f908415ff40 2023-10-16T11:27:59.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.622+0000 7f907b7fe640 1 -- 172.21.15.84:0/108825668 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c03c040 con 0x7f908415ff40 2023-10-16T11:27:59.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.622+0000 7f90890cf640 1 -- 172.21.15.84:0/108825668 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9084161010 con 0x7f908415ff40 2023-10-16T11:27:59.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.623+0000 7f90890cf640 1 -- 172.21.15.84:0/108825668 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9084161810 con 0x7f908415ff40 2023-10-16T11:27:59.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.624+0000 7f90797fa640 1 -- 172.21.15.84:0/108825668 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f905c000fc0 con 0x7f908415ff40 2023-10-16T11:27:59.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.625+0000 7f907b7fe640 1 -- 172.21.15.84:0/108825668 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f906c032070 con 0x7f908415ff40 2023-10-16T11:27:59.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.625+0000 7f907b7fe640 1 --2- 172.21.15.84:0/108825668 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f906406afc0 0x7f906406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.625+0000 7f907b7fe640 1 -- 172.21.15.84:0/108825668 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f906c0aac00 con 0x7f908415ff40 2023-10-16T11:27:59.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.626+0000 7f9082d9d640 1 --2- 172.21.15.84:0/108825668 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f906406afc0 0x7f906406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.626+0000 7f907b7fe640 1 --2- 172.21.15.84:0/108825668 >> [v2:172.21.15.157:6808/2704622262,v1:172.21.15.157:6809/2704622262] conn(0x7f9064071410 0x7f9064073830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.626+0000 7f907b7fe640 1 -- 172.21.15.84:0/108825668 --> [v2:172.21.15.157:6808/2704622262,v1:172.21.15.157:6809/2704622262] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9064073f00 con 0x7f9064071410 2023-10-16T11:27:59.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.626+0000 7f907b7fe640 1 -- 172.21.15.84:0/108825668 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=275) v2 ==== 24+0+0 (secure 0 0 0) 0x7f906c0af2a0 con 0x7f908415ff40 2023-10-16T11:27:59.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.626+0000 7f9082d9d640 1 --2- 172.21.15.84:0/108825668 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f906406afc0 0x7f906406d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9074007380 tx=0x7f907400b040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:59.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.626+0000 7f908359e640 1 --2- 172.21.15.84:0/108825668 >> [v2:172.21.15.157:6808/2704622262,v1:172.21.15.157:6809/2704622262] conn(0x7f9064071410 0x7f9064073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.627+0000 7f908359e640 1 --2- 172.21.15.84:0/108825668 >> [v2:172.21.15.157:6808/2704622262,v1:172.21.15.157:6809/2704622262] conn(0x7f9064071410 0x7f9064073830 crc :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:59.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.629+0000 7f907b7fe640 1 -- 172.21.15.84:0/108825668 <== osd.4 v2:172.21.15.157:6808/2704622262 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f9064073f00 con 0x7f9064071410 2023-10-16T11:27:59.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.644+0000 7f90797fa640 1 -- 172.21.15.84:0/108825668 --> [v2:172.21.15.157:6808/2704622262,v1:172.21.15.157:6809/2704622262] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f905c002c50 con 0x7f9064071410 2023-10-16T11:27:59.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.645+0000 7f907b7fe640 1 -- 172.21.15.84:0/108825668 <== osd.4 v2:172.21.15.157:6808/2704622262 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f905c002c50 con 0x7f9064071410 2023-10-16T11:27:59.646 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:27:59.646 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T11:27:59.647 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:27:59.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.645+0000 7f90797fa640 1 -- 172.21.15.84:0/108825668 >> [v2:172.21.15.157:6808/2704622262,v1:172.21.15.157:6809/2704622262] conn(0x7f9064071410 msgr2=0x7f9064073830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:59.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.645+0000 7f90797fa640 1 --2- 172.21.15.84:0/108825668 >> [v2:172.21.15.157:6808/2704622262,v1:172.21.15.157:6809/2704622262] conn(0x7f9064071410 0x7f9064073830 crc :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.645+0000 7f90797fa640 1 -- 172.21.15.84:0/108825668 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f906406afc0 msgr2=0x7f906406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:59.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.645+0000 7f90797fa640 1 --2- 172.21.15.84:0/108825668 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f906406afc0 0x7f906406d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9074007380 tx=0x7f907400b040 comp rx=0 tx=0).stop 2023-10-16T11:27:59.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.645+0000 7f90797fa640 1 -- 172.21.15.84:0/108825668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908415ff40 msgr2=0x7f90841f1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:59.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.645+0000 7f90797fa640 1 --2- 172.21.15.84:0/108825668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908415ff40 0x7f90841f1fd0 secure :-1 s=READY pgs=3565 cs=0 l=1 rev1=1 crypto rx=0x7f906c02c3b0 tx=0x7f906c001930 comp rx=0 tx=0).stop 2023-10-16T11:27:59.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.645+0000 7f908359e640 1 -- 172.21.15.84:0/108825668 reap_dead start 2023-10-16T11:27:59.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.645+0000 7f90797fa640 1 -- 172.21.15.84:0/108825668 shutdown_connections 2023-10-16T11:27:59.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.645+0000 7f90797fa640 1 -- 172.21.15.84:0/108825668 >> 172.21.15.84:0/108825668 conn(0x7f90840cb360 msgr2=0x7f908406c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:59.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.645+0000 7f90797fa640 1 -- 172.21.15.84:0/108825668 shutdown_connections 2023-10-16T11:27:59.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.646+0000 7f90797fa640 1 -- 172.21.15.84:0/108825668 wait complete. 2023-10-16T11:27:59.662 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-16T11:27:59.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.869+0000 7fd8a49db640 1 Processor -- start 2023-10-16T11:27:59.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.869+0000 7fd8a49db640 1 -- start start 2023-10-16T11:27:59.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.871+0000 7fd8a49db640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a0070b60 0x7fd8a015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.871+0000 7fd8a49db640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd8a006c100 0x7fd8a006c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.871+0000 7fd8a49db640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a015aef0 con 0x7fd8a0159950 2023-10-16T11:27:59.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.871+0000 7fd8a49db640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8a015e490 con 0x7fd8a006c100 2023-10-16T11:27:59.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.871+0000 7fd8a49db640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a015d910 con 0x7fd8a0070b60 2023-10-16T11:27:59.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.871+0000 7fd89ed9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd8a0159950 0x7fd8a0070450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60568/0 (socket says 172.21.15.84:60568) 2023-10-16T11:27:59.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.871+0000 7fd89ed9d640 1 -- 172.21.15.84:0/886895596 learned_addr learned my addr 172.21.15.84:0/886895596 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:27:59.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.871+0000 7fd89f59e640 1 --2- 172.21.15.84:0/886895596 >> v2:172.21.15.84:3300/0 conn(0x7fd8a006c100 0x7fd8a006c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.871+0000 7fd897fff640 1 --2- 172.21.15.84:0/886895596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a0070b60 0x7fd8a015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.871+0000 7fd89e59c640 1 -- 172.21.15.84:0/886895596 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4142477077 0 0) 0x7fd890001110 con 0x7fd8a0159950 2023-10-16T11:27:59.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.871+0000 7fd89e59c640 1 -- 172.21.15.84:0/886895596 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd88c003520 con 0x7fd8a0159950 2023-10-16T11:27:59.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.872+0000 7fd89e59c640 1 -- 172.21.15.84:0/886895596 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3301696453 0 0) 0x7fd88c003520 con 0x7fd8a0159950 2023-10-16T11:27:59.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.872+0000 7fd89e59c640 1 -- 172.21.15.84:0/886895596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a0070b60 msgr2=0x7fd8a015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:59.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.872+0000 7fd89e59c640 1 --2- 172.21.15.84:0/886895596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a0070b60 0x7fd8a015a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.872+0000 7fd89e59c640 1 -- 172.21.15.84:0/886895596 >> v2:172.21.15.84:3300/0 conn(0x7fd8a006c100 msgr2=0x7fd8a006c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:59.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.872+0000 7fd89e59c640 1 --2- 172.21.15.84:0/886895596 >> v2:172.21.15.84:3300/0 conn(0x7fd8a006c100 0x7fd8a006c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.872+0000 7fd89e59c640 1 -- 172.21.15.84:0/886895596 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a015dbf0 con 0x7fd8a0159950 2023-10-16T11:27:59.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.872+0000 7fd89e59c640 1 -- 172.21.15.84:0/886895596 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fd890001800 con 0x7fd8a0159950 2023-10-16T11:27:59.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.872+0000 7fd89e59c640 1 -- 172.21.15.84:0/886895596 >> v1:172.21.15.84:6789/0 conn(0x7fd8a0159950 legacy=0x7fd8a0070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:59.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.872+0000 7fd89e59c640 1 --2- 172.21.15.84:0/886895596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd88c0039a0 0x7fd88c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.872+0000 7fd89e59c640 1 --2- 172.21.15.84:0/886895596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd88c0047c0 0x7fd88c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.872+0000 7fd89e59c640 1 --2- 172.21.15.84:0/886895596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a0070b60 0x7fd88c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.872+0000 7fd89e59c640 1 -- 172.21.15.84:0/886895596 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a015e490 con 0x7fd88c0039a0 2023-10-16T11:27:59.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.872+0000 7fd89e59c640 1 -- 172.21.15.84:0/886895596 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a015d910 con 0x7fd88c0047c0 2023-10-16T11:27:59.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.872+0000 7fd89e59c640 1 -- 172.21.15.84:0/886895596 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c00ba30 con 0x7fd8a0070b60 2023-10-16T11:27:59.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.872+0000 7fd89f59e640 1 --2- 172.21.15.84:0/886895596 >> v2:172.21.15.84:3300/0 conn(0x7fd8a006c100 0x7fd8a006c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:27:59.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.872+0000 7fd89ed9d640 1 --2- 172.21.15.84:0/886895596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd88c0039a0 0x7fd88c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.873+0000 7fd897fff640 1 --2- 172.21.15.84:0/886895596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd88c0047c0 0x7fd88c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.873+0000 7fd89ed9d640 1 -- 172.21.15.84:0/886895596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a0070b60 msgr2=0x7fd88c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:59.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.873+0000 7fd89ed9d640 1 --2- 172.21.15.84:0/886895596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a0070b60 0x7fd88c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.873+0000 7fd89ed9d640 1 -- 172.21.15.84:0/886895596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd88c0047c0 msgr2=0x7fd88c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:27:59.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.873+0000 7fd89ed9d640 1 --2- 172.21.15.84:0/886895596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd88c0047c0 0x7fd88c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.873+0000 7fd89ed9d640 1 -- 172.21.15.84:0/886895596 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c003520 con 0x7fd88c0039a0 2023-10-16T11:27:59.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.874+0000 7fd89ed9d640 1 --2- 172.21.15.84:0/886895596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd88c0039a0 0x7fd88c003dc0 secure :-1 s=READY pgs=3567 cs=0 l=1 rev1=1 crypto rx=0x7fd890007fc0 tx=0x7fd88c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e988caa483ecb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:59.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.874+0000 7fd89e59c640 1 -- 172.21.15.84:0/886895596 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd890005ad0 con 0x7fd88c0039a0 2023-10-16T11:27:59.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.874+0000 7fd89e59c640 1 -- 172.21.15.84:0/886895596 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd890005c50 con 0x7fd88c0039a0 2023-10-16T11:27:59.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.874+0000 7fd89e59c640 1 -- 172.21.15.84:0/886895596 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89002c7e0 con 0x7fd88c0039a0 2023-10-16T11:27:59.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.875+0000 7fd8a49db640 1 -- 172.21.15.84:0/886895596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd88c0039a0 msgr2=0x7fd88c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:59.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.875+0000 7fd8a49db640 1 --2- 172.21.15.84:0/886895596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd88c0039a0 0x7fd88c003dc0 secure :-1 s=READY pgs=3567 cs=0 l=1 rev1=1 crypto rx=0x7fd890007fc0 tx=0x7fd88c0057f0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.875+0000 7fd89f59e640 1 -- 172.21.15.84:0/886895596 reap_dead start 2023-10-16T11:27:59.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.875+0000 7fd8a49db640 1 -- 172.21.15.84:0/886895596 shutdown_connections 2023-10-16T11:27:59.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.875+0000 7fd8a49db640 1 -- 172.21.15.84:0/886895596 >> 172.21.15.84:0/886895596 conn(0x7fd8a00ce6a0 msgr2=0x7fd8a00ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:27:59.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.875+0000 7fd8a49db640 1 -- 172.21.15.84:0/886895596 shutdown_connections 2023-10-16T11:27:59.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.875+0000 7fd8a49db640 1 -- 172.21.15.84:0/886895596 wait complete. 2023-10-16T11:27:59.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.877+0000 7fd8a49db640 1 Processor -- start 2023-10-16T11:27:59.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.877+0000 7fd8a49db640 1 -- start start 2023-10-16T11:27:59.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.879+0000 7fd8a49db640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd88c0039a0 0x7fd8a00b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.879+0000 7fd8a49db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a014e170 0x7fd8a00b0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.879+0000 7fd8a49db640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a015ba70 0x7fd8a014d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.879+0000 7fd8a49db640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a00c2de0 con 0x7fd8a014e170 2023-10-16T11:27:59.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.879+0000 7fd8a49db640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a00c30c0 con 0x7fd88c0039a0 2023-10-16T11:27:59.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.879+0000 7fd8a49db640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a00c33a0 con 0x7fd8a015ba70 2023-10-16T11:27:59.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.879+0000 7fd897fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a014e170 0x7fd8a00b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.879+0000 7fd897fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a014e170 0x7fd8a00b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52494/0 (socket says 172.21.15.84:52494) 2023-10-16T11:27:59.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.879+0000 7fd897fff640 1 -- 172.21.15.84:0/2223298616 learned_addr learned my addr 172.21.15.84:0/2223298616 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:27:59.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.879+0000 7fd89f59e640 1 --2- 172.21.15.84:0/2223298616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a015ba70 0x7fd8a014d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.879+0000 7fd89ed9d640 1 --2- 172.21.15.84:0/2223298616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd88c0039a0 0x7fd8a00b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.879+0000 7fd897fff640 1 -- 172.21.15.84:0/2223298616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a015ba70 msgr2=0x7fd8a014d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:59.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.879+0000 7fd897fff640 1 --2- 172.21.15.84:0/2223298616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a015ba70 0x7fd8a014d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.879+0000 7fd897fff640 1 -- 172.21.15.84:0/2223298616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd88c0039a0 msgr2=0x7fd8a00b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:27:59.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.879+0000 7fd897fff640 1 --2- 172.21.15.84:0/2223298616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd88c0039a0 0x7fd8a00b0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:27:59.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.879+0000 7fd897fff640 1 -- 172.21.15.84:0/2223298616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a00c3680 con 0x7fd8a014e170 2023-10-16T11:27:59.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.880+0000 7fd897fff640 1 --2- 172.21.15.84:0/2223298616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a014e170 0x7fd8a00b0690 secure :-1 s=READY pgs=3568 cs=0 l=1 rev1=1 crypto rx=0x7fd888010860 tx=0x7fd888010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:59.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.880+0000 7fd8977fe640 1 -- 172.21.15.84:0/2223298616 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd888009610 con 0x7fd8a014e170 2023-10-16T11:27:59.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.880+0000 7fd8977fe640 1 -- 172.21.15.84:0/2223298616 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88800eb60 con 0x7fd8a014e170 2023-10-16T11:27:59.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.880+0000 7fd8977fe640 1 -- 172.21.15.84:0/2223298616 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88801c400 con 0x7fd8a014e170 2023-10-16T11:27:59.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.880+0000 7fd8a49db640 1 -- 172.21.15.84:0/2223298616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a00c3960 con 0x7fd8a014e170 2023-10-16T11:27:59.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.880+0000 7fd8a49db640 1 -- 172.21.15.84:0/2223298616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a00c4060 con 0x7fd8a014e170 2023-10-16T11:27:59.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.881+0000 7fd8a49db640 1 -- 172.21.15.84:0/2223298616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a004e9b0 con 0x7fd8a014e170 2023-10-16T11:27:59.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.882+0000 7fd8977fe640 1 -- 172.21.15.84:0/2223298616 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd88801c560 con 0x7fd8a014e170 2023-10-16T11:27:59.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.884+0000 7fd8977fe640 1 --2- 172.21.15.84:0/2223298616 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fd88006af90 0x7fd88006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:27:59.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.884+0000 7fd8977fe640 1 -- 172.21.15.84:0/2223298616 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fd888089c70 con 0x7fd8a014e170 2023-10-16T11:27:59.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.884+0000 7fd89ed9d640 1 --2- 172.21.15.84:0/2223298616 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fd88006af90 0x7fd88006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:27:59.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.885+0000 7fd89ed9d640 1 --2- 172.21.15.84:0/2223298616 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fd88006af90 0x7fd88006d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd890034000 tx=0x7fd890009030 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:27:59.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:27:59.888+0000 7fd8977fe640 1 -- 172.21.15.84:0/2223298616 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd888054d80 con 0x7fd8a014e170 2023-10-16T11:28:00.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.082+0000 7fd8a49db640 1 -- 172.21.15.84:0/2223298616 --> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8a00c3230 con 0x7fd88006af90 2023-10-16T11:28:00.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.083+0000 7fd8977fe640 1 -- 172.21.15.84:0/2223298616 <== mgr.13719 v2:172.21.15.84:6832/3003264401 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd8a00c3230 con 0x7fd88006af90 2023-10-16T11:28:00.084 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 4 to deep-scrub 2023-10-16T11:28:00.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.085+0000 7fd8a49db640 1 -- 172.21.15.84:0/2223298616 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fd88006af90 msgr2=0x7fd88006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:00.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.085+0000 7fd8a49db640 1 --2- 172.21.15.84:0/2223298616 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fd88006af90 0x7fd88006d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd890034000 tx=0x7fd890009030 comp rx=0 tx=0).stop 2023-10-16T11:28:00.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.085+0000 7fd8a49db640 1 -- 172.21.15.84:0/2223298616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a014e170 msgr2=0x7fd8a00b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:00.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.085+0000 7fd8a49db640 1 --2- 172.21.15.84:0/2223298616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a014e170 0x7fd8a00b0690 secure :-1 s=READY pgs=3568 cs=0 l=1 rev1=1 crypto rx=0x7fd888010860 tx=0x7fd888010d20 comp rx=0 tx=0).stop 2023-10-16T11:28:00.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.086+0000 7fd8a49db640 1 -- 172.21.15.84:0/2223298616 shutdown_connections 2023-10-16T11:28:00.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.086+0000 7fd8a49db640 1 --2- 172.21.15.84:0/2223298616 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fd88006af90 0x7fd88006d450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.086+0000 7fd8a49db640 1 --2- 172.21.15.84:0/2223298616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8a015ba70 0x7fd8a014d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.086+0000 7fd8a49db640 1 --2- 172.21.15.84:0/2223298616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a014e170 0x7fd8a00b0690 unknown :-1 s=CLOSED pgs=3568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.086+0000 7fd8a49db640 1 --2- 172.21.15.84:0/2223298616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd88c0039a0 0x7fd8a00b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.086+0000 7fd8a49db640 1 -- 172.21.15.84:0/2223298616 >> 172.21.15.84:0/2223298616 conn(0x7fd8a00ce6a0 msgr2=0x7fd8a00ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:00.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.086+0000 7fd8a49db640 1 -- 172.21.15.84:0/2223298616 shutdown_connections 2023-10-16T11:28:00.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.086+0000 7fd8a49db640 1 -- 172.21.15.84:0/2223298616 wait complete. 2023-10-16T11:28:00.104 INFO:tasks.ceph:Scrubbing osd.5 2023-10-16T11:28:00.105 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-16T11:28:00.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.320+0000 7fcb4ada9640 1 Processor -- start 2023-10-16T11:28:00.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.320+0000 7fcb4ada9640 1 -- start start 2023-10-16T11:28:00.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.322+0000 7fcb4ada9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcb440cda30 0x7fcb440ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:00.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.322+0000 7fcb4ada9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb440cb890 con 0x7fcb4415af90 2023-10-16T11:28:00.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.322+0000 7fcb4ada9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb440cbac0 con 0x7fcb440cf3d0 2023-10-16T11:28:00.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.322+0000 7fcb4ada9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb440cef80 con 0x7fcb440cda30 2023-10-16T11:28:00.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.322+0000 7fcb48b45640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fcb4415af90 0x7fcb440cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60576/0 (socket says 172.21.15.84:60576) 2023-10-16T11:28:00.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.322+0000 7fcb48b45640 1 -- 172.21.15.84:0/2649586338 learned_addr learned my addr 172.21.15.84:0/2649586338 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:28:00.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.322+0000 7fcb437fe640 1 -- 172.21.15.84:0/2649586338 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3928164535 0 0) 0x7fcb440cb890 con 0x7fcb4415af90 2023-10-16T11:28:00.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.322+0000 7fcb437fe640 1 -- 172.21.15.84:0/2649586338 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb34003520 con 0x7fcb4415af90 2023-10-16T11:28:00.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb43fff640 1 --2- 172.21.15.84:0/2649586338 >> v2:172.21.15.157:3300/0 conn(0x7fcb440cda30 0x7fcb440ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:00.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb437fe640 1 -- 172.21.15.84:0/2649586338 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3168094081 0 0) 0x7fcb34003520 con 0x7fcb4415af90 2023-10-16T11:28:00.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb437fe640 1 -- 172.21.15.84:0/2649586338 >> v2:172.21.15.157:3300/0 conn(0x7fcb440cda30 msgr2=0x7fcb440ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:00.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb437fe640 1 --2- 172.21.15.84:0/2649586338 >> v2:172.21.15.157:3300/0 conn(0x7fcb440cda30 0x7fcb440ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb437fe640 1 -- 172.21.15.84:0/2649586338 >> v1:172.21.15.157:6789/0 conn(0x7fcb440cf3d0 legacy=0x7fcb440afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:00.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb437fe640 1 -- 172.21.15.84:0/2649586338 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb440cbda0 con 0x7fcb4415af90 2023-10-16T11:28:00.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb437fe640 1 -- 172.21.15.84:0/2649586338 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fcb38001670 con 0x7fcb4415af90 2023-10-16T11:28:00.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb437fe640 1 -- 172.21.15.84:0/2649586338 >> v1:172.21.15.84:6789/0 conn(0x7fcb4415af90 legacy=0x7fcb440cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:00.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb437fe640 1 --2- 172.21.15.84:0/2649586338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb340039e0 0x7fcb34003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:00.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb437fe640 1 --2- 172.21.15.84:0/2649586338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb340047c0 0x7fcb34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:00.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb437fe640 1 --2- 172.21.15.84:0/2649586338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb34005380 0x7fcb3400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:00.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb437fe640 1 -- 172.21.15.84:0/2649586338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb440cef80 con 0x7fcb340039e0 2023-10-16T11:28:00.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb437fe640 1 -- 172.21.15.84:0/2649586338 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3400c1f0 con 0x7fcb34005380 2023-10-16T11:28:00.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb437fe640 1 -- 172.21.15.84:0/2649586338 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3400c3b0 con 0x7fcb340047c0 2023-10-16T11:28:00.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb43fff640 1 --2- 172.21.15.84:0/2649586338 >> v2:172.21.15.157:3300/0 conn(0x7fcb440cda30 0x7fcb440ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:28:00.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb48b45640 1 --2- 172.21.15.84:0/2649586338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb340039e0 0x7fcb34003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:00.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb48b45640 1 -- 172.21.15.84:0/2649586338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb340047c0 msgr2=0x7fcb34004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:00.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb48b45640 1 --2- 172.21.15.84:0/2649586338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb340047c0 0x7fcb34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb48b45640 1 -- 172.21.15.84:0/2649586338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb34005380 msgr2=0x7fcb3400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:00.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb48b45640 1 --2- 172.21.15.84:0/2649586338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb34005380 0x7fcb3400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.323+0000 7fcb48b45640 1 -- 172.21.15.84:0/2649586338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb34003520 con 0x7fcb340039e0 2023-10-16T11:28:00.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.324+0000 7fcb49346640 1 -- 172.21.15.84:0/2649586338 reap_dead start 2023-10-16T11:28:00.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.325+0000 7fcb48b45640 1 --2- 172.21.15.84:0/2649586338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb340039e0 0x7fcb34003de0 secure :-1 s=READY pgs=3570 cs=0 l=1 rev1=1 crypto rx=0x7fcb3800c210 tx=0x7fcb340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b69a3e61b69c8e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:00.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.325+0000 7fcb437fe640 1 -- 172.21.15.84:0/2649586338 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3802cae0 con 0x7fcb340039e0 2023-10-16T11:28:00.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.325+0000 7fcb437fe640 1 -- 172.21.15.84:0/2649586338 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3802cc60 con 0x7fcb340039e0 2023-10-16T11:28:00.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.325+0000 7fcb437fe640 1 -- 172.21.15.84:0/2649586338 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb38005b10 con 0x7fcb340039e0 2023-10-16T11:28:00.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.325+0000 7fcb4ada9640 1 -- 172.21.15.84:0/2649586338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb340039e0 msgr2=0x7fcb34003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:00.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.325+0000 7fcb4ada9640 1 --2- 172.21.15.84:0/2649586338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb340039e0 0x7fcb34003de0 secure :-1 s=READY pgs=3570 cs=0 l=1 rev1=1 crypto rx=0x7fcb3800c210 tx=0x7fcb340057f0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.325+0000 7fcb4ada9640 1 -- 172.21.15.84:0/2649586338 shutdown_connections 2023-10-16T11:28:00.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.325+0000 7fcb4ada9640 1 --2- 172.21.15.84:0/2649586338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb340039e0 0x7fcb34003de0 unknown :-1 s=CLOSED pgs=3570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.325+0000 7fcb4ada9640 1 -- 172.21.15.84:0/2649586338 >> 172.21.15.84:0/2649586338 conn(0x7fcb44099e30 msgr2=0x7fcb440d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:00.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.326+0000 7fcb4ada9640 1 -- 172.21.15.84:0/2649586338 shutdown_connections 2023-10-16T11:28:00.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.326+0000 7fcb4ada9640 1 -- 172.21.15.84:0/2649586338 wait complete. 2023-10-16T11:28:00.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.327+0000 7fcb4ada9640 1 Processor -- start 2023-10-16T11:28:00.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.327+0000 7fcb4ada9640 1 -- start start 2023-10-16T11:28:00.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.328+0000 7fcb4ada9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb340039e0 0x7fcb440c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:00.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.328+0000 7fcb4ada9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb4406b3c0 0x7fcb44154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:00.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.328+0000 7fcb4ada9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb441585f0 0x7fcb44156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:00.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.328+0000 7fcb4ada9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb440c5f00 con 0x7fcb441585f0 2023-10-16T11:28:00.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.328+0000 7fcb4ada9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb440c67a0 con 0x7fcb4406b3c0 2023-10-16T11:28:00.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.328+0000 7fcb4ada9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4406b8a0 con 0x7fcb340039e0 2023-10-16T11:28:00.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.329+0000 7fcb49346640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb441585f0 0x7fcb44156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:00.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.329+0000 7fcb49346640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb441585f0 0x7fcb44156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52512/0 (socket says 172.21.15.84:52512) 2023-10-16T11:28:00.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.329+0000 7fcb49346640 1 -- 172.21.15.84:0/2355440738 learned_addr learned my addr 172.21.15.84:0/2355440738 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:28:00.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.329+0000 7fcb49346640 1 -- 172.21.15.84:0/2355440738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb340039e0 msgr2=0x7fcb440c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:00.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.329+0000 7fcb49346640 1 --2- 172.21.15.84:0/2355440738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb340039e0 0x7fcb440c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.329+0000 7fcb49346640 1 -- 172.21.15.84:0/2355440738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb4406b3c0 msgr2=0x7fcb44154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:00.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.329+0000 7fcb43fff640 1 --2- 172.21.15.84:0/2355440738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb4406b3c0 0x7fcb44154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:00.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.329+0000 7fcb49346640 1 --2- 172.21.15.84:0/2355440738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb4406b3c0 0x7fcb44154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.329+0000 7fcb49346640 1 -- 172.21.15.84:0/2355440738 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb440c6150 con 0x7fcb441585f0 2023-10-16T11:28:00.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.329+0000 7fcb49346640 1 --2- 172.21.15.84:0/2355440738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb441585f0 0x7fcb44156600 secure :-1 s=READY pgs=3571 cs=0 l=1 rev1=1 crypto rx=0x7fcb440d03c0 tx=0x7fcb3c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:00.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.330+0000 7fcb417fa640 1 -- 172.21.15.84:0/2355440738 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c007070 con 0x7fcb441585f0 2023-10-16T11:28:00.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.330+0000 7fcb417fa640 1 -- 172.21.15.84:0/2355440738 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3c0071f0 con 0x7fcb441585f0 2023-10-16T11:28:00.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.330+0000 7fcb417fa640 1 -- 172.21.15.84:0/2355440738 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c0075a0 con 0x7fcb441585f0 2023-10-16T11:28:00.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.330+0000 7fcb4ada9640 1 -- 172.21.15.84:0/2355440738 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb440c6430 con 0x7fcb441585f0 2023-10-16T11:28:00.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.330+0000 7fcb4ada9640 1 -- 172.21.15.84:0/2355440738 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb44158c40 con 0x7fcb441585f0 2023-10-16T11:28:00.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.331+0000 7fcb4ada9640 1 -- 172.21.15.84:0/2355440738 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcb440cc360 con 0x7fcb441585f0 2023-10-16T11:28:00.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.332+0000 7fcb417fa640 1 -- 172.21.15.84:0/2355440738 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcb3c007700 con 0x7fcb441585f0 2023-10-16T11:28:00.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.333+0000 7fcb417fa640 1 --2- 172.21.15.84:0/2355440738 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fcb2806afc0 0x7fcb2806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:00.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.333+0000 7fcb417fa640 1 -- 172.21.15.84:0/2355440738 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fcb3c0884d0 con 0x7fcb441585f0 2023-10-16T11:28:00.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.333+0000 7fcb417fa640 1 --2- 172.21.15.84:0/2355440738 >> [v2:172.21.15.157:6800/3654970329,v1:172.21.15.157:6801/3654970329] conn(0x7fcb28071410 0x7fcb28073830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:00.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.333+0000 7fcb417fa640 1 -- 172.21.15.84:0/2355440738 --> [v2:172.21.15.157:6800/3654970329,v1:172.21.15.157:6801/3654970329] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb28073f00 con 0x7fcb28071410 2023-10-16T11:28:00.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.333+0000 7fcb417fa640 1 -- 172.21.15.84:0/2355440738 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=275) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb3c08ab90 con 0x7fcb441585f0 2023-10-16T11:28:00.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.333+0000 7fcb48b45640 1 --2- 172.21.15.84:0/2355440738 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fcb2806afc0 0x7fcb2806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:00.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.334+0000 7fcb43fff640 1 --2- 172.21.15.84:0/2355440738 >> [v2:172.21.15.157:6800/3654970329,v1:172.21.15.157:6801/3654970329] conn(0x7fcb28071410 0x7fcb28073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:00.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.334+0000 7fcb48b45640 1 --2- 172.21.15.84:0/2355440738 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fcb2806afc0 0x7fcb2806d480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcb38034000 tx=0x7fcb38039040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:00.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.335+0000 7fcb43fff640 1 --2- 172.21.15.84:0/2355440738 >> [v2:172.21.15.157:6800/3654970329,v1:172.21.15.157:6801/3654970329] conn(0x7fcb28071410 0x7fcb28073830 crc :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:00.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.337+0000 7fcb417fa640 1 -- 172.21.15.84:0/2355440738 <== osd.5 v2:172.21.15.157:6800/3654970329 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fcb28073f00 con 0x7fcb28071410 2023-10-16T11:28:00.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.351+0000 7fcb4ada9640 1 -- 172.21.15.84:0/2355440738 --> [v2:172.21.15.157:6800/3654970329,v1:172.21.15.157:6801/3654970329] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcb440cc080 con 0x7fcb28071410 2023-10-16T11:28:00.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.352+0000 7fcb417fa640 1 -- 172.21.15.84:0/2355440738 <== osd.5 v2:172.21.15.157:6800/3654970329 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fcb3000c040 con 0x7fcb28071410 2023-10-16T11:28:00.353 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:28:00.353 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T11:28:00.354 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:28:00.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.352+0000 7fcb4ada9640 1 -- 172.21.15.84:0/2355440738 >> [v2:172.21.15.157:6800/3654970329,v1:172.21.15.157:6801/3654970329] conn(0x7fcb28071410 msgr2=0x7fcb28073830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:00.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.352+0000 7fcb4ada9640 1 --2- 172.21.15.84:0/2355440738 >> [v2:172.21.15.157:6800/3654970329,v1:172.21.15.157:6801/3654970329] conn(0x7fcb28071410 0x7fcb28073830 crc :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.352+0000 7fcb4ada9640 1 -- 172.21.15.84:0/2355440738 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fcb2806afc0 msgr2=0x7fcb2806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:00.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.352+0000 7fcb4ada9640 1 --2- 172.21.15.84:0/2355440738 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fcb2806afc0 0x7fcb2806d480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcb38034000 tx=0x7fcb38039040 comp rx=0 tx=0).stop 2023-10-16T11:28:00.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.352+0000 7fcb4ada9640 1 -- 172.21.15.84:0/2355440738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb441585f0 msgr2=0x7fcb44156600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:00.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.352+0000 7fcb4ada9640 1 --2- 172.21.15.84:0/2355440738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb441585f0 0x7fcb44156600 secure :-1 s=READY pgs=3571 cs=0 l=1 rev1=1 crypto rx=0x7fcb440d03c0 tx=0x7fcb3c00c8b0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.352+0000 7fcb49346640 1 -- 172.21.15.84:0/2355440738 reap_dead start 2023-10-16T11:28:00.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.352+0000 7fcb4ada9640 1 -- 172.21.15.84:0/2355440738 shutdown_connections 2023-10-16T11:28:00.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.352+0000 7fcb4ada9640 1 -- 172.21.15.84:0/2355440738 >> 172.21.15.84:0/2355440738 conn(0x7fcb44099e30 msgr2=0x7fcb440b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:00.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.352+0000 7fcb4ada9640 1 -- 172.21.15.84:0/2355440738 shutdown_connections 2023-10-16T11:28:00.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.353+0000 7fcb4ada9640 1 -- 172.21.15.84:0/2355440738 wait complete. 2023-10-16T11:28:00.370 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-16T11:28:00.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.583+0000 7fb2d5da5640 1 Processor -- start 2023-10-16T11:28:00.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.583+0000 7fb2d5da5640 1 -- start start 2023-10-16T11:28:00.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.584+0000 7fb2d5da5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb2d00caac0 0x7fb2d00ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:00.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.585+0000 7fb2d5da5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d00cb030 con 0x7fb2d015ad70 2023-10-16T11:28:00.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.585+0000 7fb2d5da5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d00c91c0 con 0x7fb2d006a4b0 2023-10-16T11:28:00.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.585+0000 7fb2d5da5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d00c94a0 con 0x7fb2d00caac0 2023-10-16T11:28:00.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.585+0000 7fb2cf7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb2d015ad70 0x7fb2d0069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60578/0 (socket says 172.21.15.84:60578) 2023-10-16T11:28:00.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.585+0000 7fb2cf7fe640 1 -- 172.21.15.84:0/2422302698 learned_addr learned my addr 172.21.15.84:0/2422302698 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:28:00.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.585+0000 7fb2cffff640 1 --2- 172.21.15.84:0/2422302698 >> v2:172.21.15.84:3300/0 conn(0x7fb2d00caac0 0x7fb2d00ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:00.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.585+0000 7fb2cffff640 1 -- 172.21.15.84:0/2422302698 >> v1:172.21.15.157:6789/0 conn(0x7fb2d006a4b0 legacy=0x7fb2d00ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:00.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.585+0000 7fb2cffff640 1 -- 172.21.15.84:0/2422302698 >> v1:172.21.15.84:6789/0 conn(0x7fb2d015ad70 legacy=0x7fb2d0069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:00.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.585+0000 7fb2cffff640 1 -- 172.21.15.84:0/2422302698 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d00c9780 con 0x7fb2d00caac0 2023-10-16T11:28:00.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.585+0000 7fb2ce7fc640 1 -- 172.21.15.84:0/2422302698 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332562988 0 0) 0x7fb2d00cb030 con 0x7fb2d015ad70 2023-10-16T11:28:00.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.586+0000 7fb2cffff640 1 --2- 172.21.15.84:0/2422302698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2d00caac0 0x7fb2d00ad4d0 secure :-1 s=READY pgs=3573 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009d90 tx=0x7fb2c802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d8045daeae19c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:00.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.587+0000 7fb2ce7fc640 1 -- 172.21.15.84:0/2422302698 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c802cc60 con 0x7fb2d00caac0 2023-10-16T11:28:00.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.587+0000 7fb2ce7fc640 1 -- 172.21.15.84:0/2422302698 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c802cdc0 con 0x7fb2d00caac0 2023-10-16T11:28:00.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.587+0000 7fb2ce7fc640 1 -- 172.21.15.84:0/2422302698 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8034660 con 0x7fb2d00caac0 2023-10-16T11:28:00.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.587+0000 7fb2d5da5640 1 -- 172.21.15.84:0/2422302698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2d00caac0 msgr2=0x7fb2d00ad4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:00.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.587+0000 7fb2d5da5640 1 --2- 172.21.15.84:0/2422302698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2d00caac0 0x7fb2d00ad4d0 secure :-1 s=READY pgs=3573 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009d90 tx=0x7fb2c802beb0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.588+0000 7fb2d5da5640 1 -- 172.21.15.84:0/2422302698 shutdown_connections 2023-10-16T11:28:00.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.588+0000 7fb2d5da5640 1 --2- 172.21.15.84:0/2422302698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2d00caac0 0x7fb2d00ad4d0 unknown :-1 s=CLOSED pgs=3573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.588+0000 7fb2d5da5640 1 -- 172.21.15.84:0/2422302698 >> 172.21.15.84:0/2422302698 conn(0x7fb2d00ce9b0 msgr2=0x7fb2d00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:00.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.588+0000 7fb2d5da5640 1 -- 172.21.15.84:0/2422302698 shutdown_connections 2023-10-16T11:28:00.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.588+0000 7fb2d5da5640 1 -- 172.21.15.84:0/2422302698 wait complete. 2023-10-16T11:28:00.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.589+0000 7fb2d5da5640 1 Processor -- start 2023-10-16T11:28:00.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.590+0000 7fb2d5da5640 1 -- start start 2023-10-16T11:28:00.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.590+0000 7fb2d5da5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d006a4b0 0x7fb2d014ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:00.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.591+0000 7fb2d5da5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d00caac0 0x7fb2d00c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:00.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.591+0000 7fb2d5da5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2d015ad70 0x7fb2d01590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:00.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.591+0000 7fb2d5da5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d00addd0 con 0x7fb2d015ad70 2023-10-16T11:28:00.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.591+0000 7fb2d5da5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d00ae090 con 0x7fb2d00caac0 2023-10-16T11:28:00.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.591+0000 7fb2d5da5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d00c4570 con 0x7fb2d006a4b0 2023-10-16T11:28:00.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.591+0000 7fb2cffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2d015ad70 0x7fb2d01590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:00.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.591+0000 7fb2cffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2d015ad70 0x7fb2d01590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52530/0 (socket says 172.21.15.84:52530) 2023-10-16T11:28:00.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.591+0000 7fb2cffff640 1 -- 172.21.15.84:0/2715556919 learned_addr learned my addr 172.21.15.84:0/2715556919 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:28:00.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.591+0000 7fb2cf7fe640 1 --2- 172.21.15.84:0/2715556919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d006a4b0 0x7fb2d014ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:00.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.591+0000 7fb2ceffd640 1 --2- 172.21.15.84:0/2715556919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d00caac0 0x7fb2d00c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:00.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.591+0000 7fb2cffff640 1 -- 172.21.15.84:0/2715556919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d006a4b0 msgr2=0x7fb2d014ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:00.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.591+0000 7fb2cffff640 1 --2- 172.21.15.84:0/2715556919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d006a4b0 0x7fb2d014ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.591+0000 7fb2cffff640 1 -- 172.21.15.84:0/2715556919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d00caac0 msgr2=0x7fb2d00c34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:00.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.591+0000 7fb2cffff640 1 --2- 172.21.15.84:0/2715556919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d00caac0 0x7fb2d00c34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.591+0000 7fb2cffff640 1 -- 172.21.15.84:0/2715556919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c8009a70 con 0x7fb2d015ad70 2023-10-16T11:28:00.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.592+0000 7fb2cffff640 1 --2- 172.21.15.84:0/2715556919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2d015ad70 0x7fb2d01590e0 secure :-1 s=READY pgs=3574 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009d90 tx=0x7fb2c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:00.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.592+0000 7fb2b7fff640 1 -- 172.21.15.84:0/2715556919 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c803e610 con 0x7fb2d015ad70 2023-10-16T11:28:00.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.592+0000 7fb2b7fff640 1 -- 172.21.15.84:0/2715556919 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c803e790 con 0x7fb2d015ad70 2023-10-16T11:28:00.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.592+0000 7fb2b7fff640 1 -- 172.21.15.84:0/2715556919 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c803eb40 con 0x7fb2d015ad70 2023-10-16T11:28:00.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.592+0000 7fb2d5da5640 1 -- 172.21.15.84:0/2715556919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d00c3a30 con 0x7fb2d015ad70 2023-10-16T11:28:00.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.592+0000 7fb2d5da5640 1 -- 172.21.15.84:0/2715556919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d00c3fb0 con 0x7fb2d015ad70 2023-10-16T11:28:00.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.593+0000 7fb2d5da5640 1 -- 172.21.15.84:0/2715556919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d00ae240 con 0x7fb2d015ad70 2023-10-16T11:28:00.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.594+0000 7fb2b7fff640 1 -- 172.21.15.84:0/2715556919 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb2c803b050 con 0x7fb2d015ad70 2023-10-16T11:28:00.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.595+0000 7fb2b7fff640 1 --2- 172.21.15.84:0/2715556919 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fb2bc06af90 0x7fb2bc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:00.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.595+0000 7fb2b7fff640 1 -- 172.21.15.84:0/2715556919 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fb2c80ac270 con 0x7fb2d015ad70 2023-10-16T11:28:00.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.596+0000 7fb2cf7fe640 1 --2- 172.21.15.84:0/2715556919 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fb2bc06af90 0x7fb2bc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:00.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.596+0000 7fb2cf7fe640 1 --2- 172.21.15.84:0/2715556919 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fb2bc06af90 0x7fb2bc06d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0000f60 tx=0x7fb2c0007c00 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:00.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.600+0000 7fb2b7fff640 1 -- 172.21.15.84:0/2715556919 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb2c80771a0 con 0x7fb2d015ad70 2023-10-16T11:28:00.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.792+0000 7fb2d5da5640 1 -- 172.21.15.84:0/2715556919 --> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2d00c9ca0 con 0x7fb2bc06af90 2023-10-16T11:28:00.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.793+0000 7fb2b7fff640 1 -- 172.21.15.84:0/2715556919 <== mgr.13719 v2:172.21.15.84:6832/3003264401 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb2c0007270 con 0x7fb2bc06af90 2023-10-16T11:28:00.794 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 5 to deep-scrub 2023-10-16T11:28:00.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.796+0000 7fb2d5da5640 1 -- 172.21.15.84:0/2715556919 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fb2bc06af90 msgr2=0x7fb2bc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:00.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.796+0000 7fb2d5da5640 1 --2- 172.21.15.84:0/2715556919 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fb2bc06af90 0x7fb2bc06d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0000f60 tx=0x7fb2c0007c00 comp rx=0 tx=0).stop 2023-10-16T11:28:00.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.796+0000 7fb2d5da5640 1 -- 172.21.15.84:0/2715556919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2d015ad70 msgr2=0x7fb2d01590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:00.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.796+0000 7fb2d5da5640 1 --2- 172.21.15.84:0/2715556919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2d015ad70 0x7fb2d01590e0 secure :-1 s=READY pgs=3574 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009d90 tx=0x7fb2c8004a40 comp rx=0 tx=0).stop 2023-10-16T11:28:00.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.796+0000 7fb2d5da5640 1 -- 172.21.15.84:0/2715556919 shutdown_connections 2023-10-16T11:28:00.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.796+0000 7fb2d5da5640 1 --2- 172.21.15.84:0/2715556919 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fb2bc06af90 0x7fb2bc06d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.796+0000 7fb2d5da5640 1 --2- 172.21.15.84:0/2715556919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2d015ad70 0x7fb2d01590e0 unknown :-1 s=CLOSED pgs=3574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.796+0000 7fb2d5da5640 1 --2- 172.21.15.84:0/2715556919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d00caac0 0x7fb2d00c34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.796+0000 7fb2d5da5640 1 --2- 172.21.15.84:0/2715556919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d006a4b0 0x7fb2d014ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:00.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.796+0000 7fb2d5da5640 1 -- 172.21.15.84:0/2715556919 >> 172.21.15.84:0/2715556919 conn(0x7fb2d00ce9b0 msgr2=0x7fb2d0069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:00.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.796+0000 7fb2d5da5640 1 -- 172.21.15.84:0/2715556919 shutdown_connections 2023-10-16T11:28:00.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:00.796+0000 7fb2d5da5640 1 -- 172.21.15.84:0/2715556919 wait complete. 2023-10-16T11:28:00.815 INFO:tasks.ceph:Scrubbing osd.6 2023-10-16T11:28:00.816 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-16T11:28:01.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.032+0000 7fbbc27e9640 1 Processor -- start 2023-10-16T11:28:01.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.032+0000 7fbbc27e9640 1 -- start start 2023-10-16T11:28:01.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.034+0000 7fbbc27e9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbbc0ad970 0x7fbbbc0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.034+0000 7fbbc27e9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbbbc0ccfb0 0x7fbbbc0cd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.034+0000 7fbbc27e9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbbc15b9d0 con 0x7fbbbc06c040 2023-10-16T11:28:01.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.034+0000 7fbbc27e9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbbc15bcb0 con 0x7fbbbc0ccfb0 2023-10-16T11:28:01.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.034+0000 7fbbc27e9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc15bf90 con 0x7fbbbc0ad970 2023-10-16T11:28:01.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.035+0000 7fbbc0fe6640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbbc0ad970 0x7fbbbc0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.035+0000 7fbbc1fe8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbbbc0ccfb0 0x7fbbbc0cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.035+0000 7fbbc17e7640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fbbbc06c040 0x7fbbbc06c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.84:47900/0 (socket says 172.21.15.84:47900) 2023-10-16T11:28:01.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.035+0000 7fbbc17e7640 1 -- 172.21.15.84:0/1546637299 learned_addr learned my addr 172.21.15.84:0/1546637299 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:28:01.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.035+0000 7fbbc1fe8640 1 -- 172.21.15.84:0/1546637299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbbc0ad970 msgr2=0x7fbbbc0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:01.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.035+0000 7fbbc1fe8640 1 --2- 172.21.15.84:0/1546637299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbbc0ad970 0x7fbbbc0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.036+0000 7fbbc1fe8640 1 -- 172.21.15.84:0/1546637299 >> v1:172.21.15.157:6789/0 conn(0x7fbbbc06c040 legacy=0x7fbbbc06c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.036+0000 7fbbc1fe8640 1 -- 172.21.15.84:0/1546637299 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbbc0cc270 con 0x7fbbbc0ccfb0 2023-10-16T11:28:01.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.036+0000 7fbbc0fe6640 1 --2- 172.21.15.84:0/1546637299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbbc0ad970 0x7fbbbc0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:28:01.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.037+0000 7fbbc1fe8640 1 --2- 172.21.15.84:0/1546637299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbbc0ccfb0 0x7fbbbc0cd440 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009dc0 tx=0x7fbbb402be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56dc58415cdc6721 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:01.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.037+0000 7fbbaffff640 1 -- 172.21.15.84:0/1546637299 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb402cc40 con 0x7fbbbc0ccfb0 2023-10-16T11:28:01.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.037+0000 7fbbaffff640 1 -- 172.21.15.84:0/1546637299 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb402cda0 con 0x7fbbbc0ccfb0 2023-10-16T11:28:01.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.037+0000 7fbbaffff640 1 -- 172.21.15.84:0/1546637299 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4034660 con 0x7fbbbc0ccfb0 2023-10-16T11:28:01.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.038+0000 7fbbc27e9640 1 -- 172.21.15.84:0/1546637299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbbc0ccfb0 msgr2=0x7fbbbc0cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.038+0000 7fbbc27e9640 1 --2- 172.21.15.84:0/1546637299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbbc0ccfb0 0x7fbbbc0cd440 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009dc0 tx=0x7fbbb402be90 comp rx=0 tx=0).stop 2023-10-16T11:28:01.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.038+0000 7fbbc27e9640 1 -- 172.21.15.84:0/1546637299 shutdown_connections 2023-10-16T11:28:01.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.038+0000 7fbbc27e9640 1 --2- 172.21.15.84:0/1546637299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbbc0ccfb0 0x7fbbbc0cd440 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.038+0000 7fbbc27e9640 1 --2- 172.21.15.84:0/1546637299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbbc0ad970 0x7fbbbc0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.038+0000 7fbbc27e9640 1 -- 172.21.15.84:0/1546637299 >> 172.21.15.84:0/1546637299 conn(0x7fbbbc0ce6a0 msgr2=0x7fbbbc0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:01.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.038+0000 7fbbc27e9640 1 -- 172.21.15.84:0/1546637299 shutdown_connections 2023-10-16T11:28:01.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.038+0000 7fbbc27e9640 1 -- 172.21.15.84:0/1546637299 wait complete. 2023-10-16T11:28:01.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.040+0000 7fbbc27e9640 1 Processor -- start 2023-10-16T11:28:01.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.040+0000 7fbbc27e9640 1 -- start start 2023-10-16T11:28:01.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.041+0000 7fbbc27e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbbc06c040 0x7fbbbc14edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.041+0000 7fbbc27e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbbc0ad970 0x7fbbbc0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.041+0000 7fbbc27e9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbbc0ccfb0 0x7fbbbc0acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.041+0000 7fbbc27e9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc1f69b0 con 0x7fbbbc06c040 2023-10-16T11:28:01.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.041+0000 7fbbc27e9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc152a90 con 0x7fbbbc0ad970 2023-10-16T11:28:01.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.041+0000 7fbbc27e9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc14f330 con 0x7fbbbc0ccfb0 2023-10-16T11:28:01.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.041+0000 7fbbc17e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbbc06c040 0x7fbbbc14edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.041+0000 7fbbc17e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbbc06c040 0x7fbbbc14edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52540/0 (socket says 172.21.15.84:52540) 2023-10-16T11:28:01.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.041+0000 7fbbc17e7640 1 -- 172.21.15.84:0/2353897652 learned_addr learned my addr 172.21.15.84:0/2353897652 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:28:01.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.042+0000 7fbbc1fe8640 1 --2- 172.21.15.84:0/2353897652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbbc0ccfb0 0x7fbbbc0acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.042+0000 7fbbc17e7640 1 -- 172.21.15.84:0/2353897652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbbc0ccfb0 msgr2=0x7fbbbc0acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.042+0000 7fbbc17e7640 1 --2- 172.21.15.84:0/2353897652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbbc0ccfb0 0x7fbbbc0acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.042+0000 7fbbc0fe6640 1 --2- 172.21.15.84:0/2353897652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbbc0ad970 0x7fbbbc0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.042+0000 7fbbc17e7640 1 -- 172.21.15.84:0/2353897652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbbc0ad970 msgr2=0x7fbbbc0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.042+0000 7fbbc17e7640 1 --2- 172.21.15.84:0/2353897652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbbc0ad970 0x7fbbbc0ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.042+0000 7fbbc17e7640 1 -- 172.21.15.84:0/2353897652 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb0009400 con 0x7fbbbc06c040 2023-10-16T11:28:01.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.042+0000 7fbbc17e7640 1 --2- 172.21.15.84:0/2353897652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbbc06c040 0x7fbbbc14edf0 secure :-1 s=READY pgs=3575 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800c4a0 tx=0x7fbbb800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:01.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.042+0000 7fbbadffb640 1 -- 172.21.15.84:0/2353897652 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8005ce0 con 0x7fbbbc06c040 2023-10-16T11:28:01.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.042+0000 7fbbadffb640 1 -- 172.21.15.84:0/2353897652 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb8005e40 con 0x7fbbbc06c040 2023-10-16T11:28:01.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.042+0000 7fbbadffb640 1 -- 172.21.15.84:0/2353897652 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8007270 con 0x7fbbbc06c040 2023-10-16T11:28:01.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.042+0000 7fbbc27e9640 1 -- 172.21.15.84:0/2353897652 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb4009a70 con 0x7fbbbc06c040 2023-10-16T11:28:01.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.043+0000 7fbbc27e9640 1 -- 172.21.15.84:0/2353897652 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbbc0c7130 con 0x7fbbbc06c040 2023-10-16T11:28:01.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.043+0000 7fbbc27e9640 1 -- 172.21.15.84:0/2353897652 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbbbc0cc550 con 0x7fbbbc06c040 2023-10-16T11:28:01.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.045+0000 7fbbadffb640 1 -- 172.21.15.84:0/2353897652 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbbb80073d0 con 0x7fbbbc06c040 2023-10-16T11:28:01.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.046+0000 7fbbadffb640 1 --2- 172.21.15.84:0/2353897652 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fbba006af90 0x7fbba006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.046+0000 7fbbadffb640 1 -- 172.21.15.84:0/2353897652 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fbbb8088bf0 con 0x7fbbbc06c040 2023-10-16T11:28:01.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.046+0000 7fbbadffb640 1 --2- 172.21.15.84:0/2353897652 >> [v2:172.21.15.157:6816/899624983,v1:172.21.15.157:6817/899624983] conn(0x7fbba00713e0 0x7fbba0073800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.046+0000 7fbbadffb640 1 -- 172.21.15.84:0/2353897652 --> [v2:172.21.15.157:6816/899624983,v1:172.21.15.157:6817/899624983] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbba0073ed0 con 0x7fbba00713e0 2023-10-16T11:28:01.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.046+0000 7fbbadffb640 1 -- 172.21.15.84:0/2353897652 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=275) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbb808d2a0 con 0x7fbbbc06c040 2023-10-16T11:28:01.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.046+0000 7fbbc0fe6640 1 --2- 172.21.15.84:0/2353897652 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fbba006af90 0x7fbba006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.047+0000 7fbbc1fe8640 1 --2- 172.21.15.84:0/2353897652 >> [v2:172.21.15.157:6816/899624983,v1:172.21.15.157:6817/899624983] conn(0x7fbba00713e0 0x7fbba0073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.047+0000 7fbbc0fe6640 1 --2- 172.21.15.84:0/2353897652 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fbba006af90 0x7fbba006d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc1542f0 tx=0x7fbbb0008040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:01.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.047+0000 7fbbc1fe8640 1 --2- 172.21.15.84:0/2353897652 >> [v2:172.21.15.157:6816/899624983,v1:172.21.15.157:6817/899624983] conn(0x7fbba00713e0 0x7fbba0073800 crc :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:01.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.050+0000 7fbbadffb640 1 -- 172.21.15.84:0/2353897652 <== osd.6 v2:172.21.15.157:6816/899624983 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fbba0073ed0 con 0x7fbba00713e0 2023-10-16T11:28:01.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.064+0000 7fbbc27e9640 1 -- 172.21.15.84:0/2353897652 --> [v2:172.21.15.157:6816/899624983,v1:172.21.15.157:6817/899624983] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbbbc04e9b0 con 0x7fbba00713e0 2023-10-16T11:28:01.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.064+0000 7fbbadffb640 1 -- 172.21.15.84:0/2353897652 <== osd.6 v2:172.21.15.157:6816/899624983 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fbbbc04e9b0 con 0x7fbba00713e0 2023-10-16T11:28:01.066 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:28:01.066 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T11:28:01.066 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:28:01.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.065+0000 7fbbc27e9640 1 -- 172.21.15.84:0/2353897652 >> [v2:172.21.15.157:6816/899624983,v1:172.21.15.157:6817/899624983] conn(0x7fbba00713e0 msgr2=0x7fbba0073800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.065+0000 7fbbc27e9640 1 --2- 172.21.15.84:0/2353897652 >> [v2:172.21.15.157:6816/899624983,v1:172.21.15.157:6817/899624983] conn(0x7fbba00713e0 0x7fbba0073800 crc :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.065+0000 7fbbc27e9640 1 -- 172.21.15.84:0/2353897652 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fbba006af90 msgr2=0x7fbba006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.065+0000 7fbbc27e9640 1 --2- 172.21.15.84:0/2353897652 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fbba006af90 0x7fbba006d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc1542f0 tx=0x7fbbb0008040 comp rx=0 tx=0).stop 2023-10-16T11:28:01.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.065+0000 7fbbc27e9640 1 -- 172.21.15.84:0/2353897652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbbc06c040 msgr2=0x7fbbbc14edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.065+0000 7fbbc27e9640 1 --2- 172.21.15.84:0/2353897652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbbc06c040 0x7fbbbc14edf0 secure :-1 s=READY pgs=3575 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800c4a0 tx=0x7fbbb800c960 comp rx=0 tx=0).stop 2023-10-16T11:28:01.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.065+0000 7fbbc1fe8640 1 -- 172.21.15.84:0/2353897652 reap_dead start 2023-10-16T11:28:01.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.065+0000 7fbbc27e9640 1 -- 172.21.15.84:0/2353897652 shutdown_connections 2023-10-16T11:28:01.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.065+0000 7fbbc27e9640 1 -- 172.21.15.84:0/2353897652 >> 172.21.15.84:0/2353897652 conn(0x7fbbbc0ce6a0 msgr2=0x7fbbbc159d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:01.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.065+0000 7fbbc27e9640 1 -- 172.21.15.84:0/2353897652 shutdown_connections 2023-10-16T11:28:01.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.065+0000 7fbbc27e9640 1 -- 172.21.15.84:0/2353897652 wait complete. 2023-10-16T11:28:01.082 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-16T11:28:01.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.290+0000 7fa84bfff640 1 Processor -- start 2023-10-16T11:28:01.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.290+0000 7fa84bfff640 1 -- start start 2023-10-16T11:28:01.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.292+0000 7fa84bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa84c0ac890 0x7fa84c0accc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.292+0000 7fa84bfff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84c160a90 con 0x7fa84c15dde0 2023-10-16T11:28:01.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.292+0000 7fa84bfff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84c0ad200 con 0x7fa84c15c9f0 2023-10-16T11:28:01.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.292+0000 7fa84bfff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa84c0cdec0 con 0x7fa84c0ac890 2023-10-16T11:28:01.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.293+0000 7fa8427fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa84c15dde0 0x7fa84c0709e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60590/0 (socket says 172.21.15.84:60590) 2023-10-16T11:28:01.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.293+0000 7fa8427fc640 1 -- 172.21.15.84:0/3378026767 learned_addr learned my addr 172.21.15.84:0/3378026767 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:28:01.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.293+0000 7fa84b7fe640 1 --2- 172.21.15.84:0/3378026767 >> v2:172.21.15.157:3300/0 conn(0x7fa84c0ac890 0x7fa84c0accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.293+0000 7fa84a7fc640 1 -- 172.21.15.84:0/3378026767 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783695141 0 0) 0x7fa84c160a90 con 0x7fa84c15dde0 2023-10-16T11:28:01.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.293+0000 7fa84a7fc640 1 -- 172.21.15.84:0/3378026767 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa838003520 con 0x7fa84c15dde0 2023-10-16T11:28:01.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.294+0000 7fa84a7fc640 1 -- 172.21.15.84:0/3378026767 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 346170191 0 0) 0x7fa838003520 con 0x7fa84c15dde0 2023-10-16T11:28:01.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.294+0000 7fa84a7fc640 1 -- 172.21.15.84:0/3378026767 >> v2:172.21.15.157:3300/0 conn(0x7fa84c0ac890 msgr2=0x7fa84c0accc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:01.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.294+0000 7fa84a7fc640 1 --2- 172.21.15.84:0/3378026767 >> v2:172.21.15.157:3300/0 conn(0x7fa84c0ac890 0x7fa84c0accc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.294+0000 7fa84a7fc640 1 -- 172.21.15.84:0/3378026767 >> v1:172.21.15.157:6789/0 conn(0x7fa84c15c9f0 legacy=0x7fa84c0c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.294+0000 7fa84a7fc640 1 -- 172.21.15.84:0/3378026767 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c0cd900 con 0x7fa84c15dde0 2023-10-16T11:28:01.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.294+0000 7fa84a7fc640 1 -- 172.21.15.84:0/3378026767 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fa834001670 con 0x7fa84c15dde0 2023-10-16T11:28:01.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.294+0000 7fa84a7fc640 1 -- 172.21.15.84:0/3378026767 >> v1:172.21.15.84:6789/0 conn(0x7fa84c15dde0 legacy=0x7fa84c0709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.294+0000 7fa84a7fc640 1 --2- 172.21.15.84:0/3378026767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa838003970 0x7fa838003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.294+0000 7fa84a7fc640 1 --2- 172.21.15.84:0/3378026767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8380047c0 0x7fa838004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.294+0000 7fa84a7fc640 1 --2- 172.21.15.84:0/3378026767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa838005180 0x7fa83800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.294+0000 7fa84a7fc640 1 -- 172.21.15.84:0/3378026767 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83800c050 con 0x7fa838003970 2023-10-16T11:28:01.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.294+0000 7fa84a7fc640 1 -- 172.21.15.84:0/3378026767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83800c190 con 0x7fa8380047c0 2023-10-16T11:28:01.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.294+0000 7fa84a7fc640 1 -- 172.21.15.84:0/3378026767 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83800c350 con 0x7fa838005180 2023-10-16T11:28:01.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.294+0000 7fa84affd640 1 --2- 172.21.15.84:0/3378026767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa838003970 0x7fa838003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.295+0000 7fa84b7fe640 1 --2- 172.21.15.84:0/3378026767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa838005180 0x7fa83800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.295+0000 7fa8427fc640 1 --2- 172.21.15.84:0/3378026767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8380047c0 0x7fa838004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.295+0000 7fa84affd640 1 -- 172.21.15.84:0/3378026767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa838005180 msgr2=0x7fa83800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:01.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.295+0000 7fa84affd640 1 --2- 172.21.15.84:0/3378026767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa838005180 0x7fa83800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.295+0000 7fa84affd640 1 -- 172.21.15.84:0/3378026767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8380047c0 msgr2=0x7fa838004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:01.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.295+0000 7fa84affd640 1 --2- 172.21.15.84:0/3378026767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8380047c0 0x7fa838004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.295+0000 7fa84affd640 1 -- 172.21.15.84:0/3378026767 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa838003520 con 0x7fa838003970 2023-10-16T11:28:01.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.295+0000 7fa8427fc640 1 --2- 172.21.15.84:0/3378026767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8380047c0 0x7fa838004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:28:01.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.295+0000 7fa84b7fe640 1 --2- 172.21.15.84:0/3378026767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa838005180 0x7fa83800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:28:01.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.295+0000 7fa84b7fe640 1 -- 172.21.15.84:0/3378026767 reap_dead start 2023-10-16T11:28:01.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.297+0000 7fa84affd640 1 --2- 172.21.15.84:0/3378026767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa838003970 0x7fa838003db0 secure :-1 s=READY pgs=3577 cs=0 l=1 rev1=1 crypto rx=0x7fa83c001750 tx=0x7fa8380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d593d0fe60b98da server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:01.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.297+0000 7fa84a7fc640 1 -- 172.21.15.84:0/3378026767 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c0052d0 con 0x7fa838003970 2023-10-16T11:28:01.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.297+0000 7fa84a7fc640 1 -- 172.21.15.84:0/3378026767 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83c02e050 con 0x7fa838003970 2023-10-16T11:28:01.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.297+0000 7fa84a7fc640 1 -- 172.21.15.84:0/3378026767 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c0035f0 con 0x7fa838003970 2023-10-16T11:28:01.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.297+0000 7fa84bfff640 1 -- 172.21.15.84:0/3378026767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa838003970 msgr2=0x7fa838003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.297+0000 7fa84bfff640 1 --2- 172.21.15.84:0/3378026767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa838003970 0x7fa838003db0 secure :-1 s=READY pgs=3577 cs=0 l=1 rev1=1 crypto rx=0x7fa83c001750 tx=0x7fa8380057f0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.298+0000 7fa84bfff640 1 -- 172.21.15.84:0/3378026767 shutdown_connections 2023-10-16T11:28:01.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.298+0000 7fa84bfff640 1 --2- 172.21.15.84:0/3378026767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa838003970 0x7fa838003db0 unknown :-1 s=CLOSED pgs=3577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.298+0000 7fa84bfff640 1 -- 172.21.15.84:0/3378026767 >> 172.21.15.84:0/3378026767 conn(0x7fa84c0ce6a0 msgr2=0x7fa84c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:01.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.298+0000 7fa84bfff640 1 -- 172.21.15.84:0/3378026767 shutdown_connections 2023-10-16T11:28:01.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.298+0000 7fa84bfff640 1 -- 172.21.15.84:0/3378026767 wait complete. 2023-10-16T11:28:01.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.299+0000 7fa84bfff640 1 Processor -- start 2023-10-16T11:28:01.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.300+0000 7fa84bfff640 1 -- start start 2023-10-16T11:28:01.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.300+0000 7fa84bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa838003970 0x7fa84c0c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.300+0000 7fa84bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84c06b460 0x7fa84c0c4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.300+0000 7fa84bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa84c0c4dd0 0x7fa84c0ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.301+0000 7fa84bfff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c06c550 con 0x7fa84c0c4dd0 2023-10-16T11:28:01.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.301+0000 7fa84bfff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c06b940 con 0x7fa84c06b460 2023-10-16T11:28:01.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.301+0000 7fa84bfff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c06bc20 con 0x7fa838003970 2023-10-16T11:28:01.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.301+0000 7fa84b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa84c0c4dd0 0x7fa84c0ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.301+0000 7fa84b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa84c0c4dd0 0x7fa84c0ae9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52558/0 (socket says 172.21.15.84:52558) 2023-10-16T11:28:01.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.301+0000 7fa84b7fe640 1 -- 172.21.15.84:0/324851560 learned_addr learned my addr 172.21.15.84:0/324851560 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:28:01.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.301+0000 7fa84affd640 1 --2- 172.21.15.84:0/324851560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa838003970 0x7fa84c0c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.301+0000 7fa84b7fe640 1 -- 172.21.15.84:0/324851560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa838003970 msgr2=0x7fa84c0c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.301+0000 7fa84b7fe640 1 --2- 172.21.15.84:0/324851560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa838003970 0x7fa84c0c0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.301+0000 7fa84b7fe640 1 -- 172.21.15.84:0/324851560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84c06b460 msgr2=0x7fa84c0c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.301+0000 7fa84b7fe640 1 --2- 172.21.15.84:0/324851560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84c06b460 0x7fa84c0c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.301+0000 7fa84b7fe640 1 -- 172.21.15.84:0/324851560 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c06bf00 con 0x7fa84c0c4dd0 2023-10-16T11:28:01.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.302+0000 7fa84b7fe640 1 --2- 172.21.15.84:0/324851560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa84c0c4dd0 0x7fa84c0ae9a0 secure :-1 s=READY pgs=3578 cs=0 l=1 rev1=1 crypto rx=0x7fa84c06d350 tx=0x7fa84400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:01.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.302+0000 7fa843fff640 1 -- 172.21.15.84:0/324851560 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844016620 con 0x7fa84c0c4dd0 2023-10-16T11:28:01.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.302+0000 7fa843fff640 1 -- 172.21.15.84:0/324851560 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa844016780 con 0x7fa84c0c4dd0 2023-10-16T11:28:01.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.302+0000 7fa843fff640 1 -- 172.21.15.84:0/324851560 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844016af0 con 0x7fa84c0c4dd0 2023-10-16T11:28:01.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.302+0000 7fa84bfff640 1 -- 172.21.15.84:0/324851560 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84c06c1e0 con 0x7fa84c0c4dd0 2023-10-16T11:28:01.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.302+0000 7fa84bfff640 1 -- 172.21.15.84:0/324851560 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84c06cbc0 con 0x7fa84c0c4dd0 2023-10-16T11:28:01.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.303+0000 7fa84bfff640 1 -- 172.21.15.84:0/324851560 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84c04e9b0 con 0x7fa84c0c4dd0 2023-10-16T11:28:01.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.304+0000 7fa843fff640 1 -- 172.21.15.84:0/324851560 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa844016c50 con 0x7fa84c0c4dd0 2023-10-16T11:28:01.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.305+0000 7fa843fff640 1 --2- 172.21.15.84:0/324851560 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fa82c06af90 0x7fa82c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.305+0000 7fa843fff640 1 -- 172.21.15.84:0/324851560 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fa84408a9d0 con 0x7fa84c0c4dd0 2023-10-16T11:28:01.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.305+0000 7fa84affd640 1 --2- 172.21.15.84:0/324851560 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fa82c06af90 0x7fa82c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.306+0000 7fa84affd640 1 --2- 172.21.15.84:0/324851560 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fa82c06af90 0x7fa82c06d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa83c001750 tx=0x7fa83c009030 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:01.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.309+0000 7fa843fff640 1 -- 172.21.15.84:0/324851560 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa8440557f0 con 0x7fa84c0c4dd0 2023-10-16T11:28:01.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.504+0000 7fa84bfff640 1 -- 172.21.15.84:0/324851560 --> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fa84c06a070 con 0x7fa82c06af90 2023-10-16T11:28:01.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.505+0000 7fa843fff640 1 -- 172.21.15.84:0/324851560 <== mgr.13719 v2:172.21.15.84:6832/3003264401 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa84c06a070 con 0x7fa82c06af90 2023-10-16T11:28:01.506 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 6 to deep-scrub 2023-10-16T11:28:01.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.507+0000 7fa84bfff640 1 -- 172.21.15.84:0/324851560 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fa82c06af90 msgr2=0x7fa82c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.507+0000 7fa84bfff640 1 --2- 172.21.15.84:0/324851560 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fa82c06af90 0x7fa82c06d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa83c001750 tx=0x7fa83c009030 comp rx=0 tx=0).stop 2023-10-16T11:28:01.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.507+0000 7fa84bfff640 1 -- 172.21.15.84:0/324851560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa84c0c4dd0 msgr2=0x7fa84c0ae9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.507+0000 7fa84bfff640 1 --2- 172.21.15.84:0/324851560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa84c0c4dd0 0x7fa84c0ae9a0 secure :-1 s=READY pgs=3578 cs=0 l=1 rev1=1 crypto rx=0x7fa84c06d350 tx=0x7fa84400f520 comp rx=0 tx=0).stop 2023-10-16T11:28:01.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.507+0000 7fa84bfff640 1 -- 172.21.15.84:0/324851560 shutdown_connections 2023-10-16T11:28:01.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.507+0000 7fa84bfff640 1 --2- 172.21.15.84:0/324851560 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fa82c06af90 0x7fa82c06d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.507+0000 7fa84bfff640 1 --2- 172.21.15.84:0/324851560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa84c0c4dd0 0x7fa84c0ae9a0 unknown :-1 s=CLOSED pgs=3578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.507+0000 7fa84bfff640 1 --2- 172.21.15.84:0/324851560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84c06b460 0x7fa84c0c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.507+0000 7fa84bfff640 1 --2- 172.21.15.84:0/324851560 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa838003970 0x7fa84c0c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.507+0000 7fa84bfff640 1 -- 172.21.15.84:0/324851560 >> 172.21.15.84:0/324851560 conn(0x7fa84c0ce6a0 msgr2=0x7fa84c15b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:01.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.508+0000 7fa84bfff640 1 -- 172.21.15.84:0/324851560 shutdown_connections 2023-10-16T11:28:01.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.508+0000 7fa84bfff640 1 -- 172.21.15.84:0/324851560 wait complete. 2023-10-16T11:28:01.525 INFO:tasks.ceph:Scrubbing osd.7 2023-10-16T11:28:01.526 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-16T11:28:01.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.741+0000 7f97917ef640 1 Processor -- start 2023-10-16T11:28:01.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.741+0000 7f97917ef640 1 -- start start 2023-10-16T11:28:01.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.743+0000 7f97917ef640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f978c0c9060 0x7f978c06f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.743+0000 7f97917ef640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f978c0c9440 0x7f978c06f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.743+0000 7f97917ef640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f978c0ca560 con 0x7f978c06c9e0 2023-10-16T11:28:01.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.743+0000 7f97917ef640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f978c0c9950 con 0x7f978c0c9060 2023-10-16T11:28:01.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.743+0000 7f97917ef640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0c9c30 con 0x7f978c0c9440 2023-10-16T11:28:01.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.743+0000 7f9783fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f978c06c9e0 0x7f978c164d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60598/0 (socket says 172.21.15.84:60598) 2023-10-16T11:28:01.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.743+0000 7f9783fff640 1 -- 172.21.15.84:0/303967054 learned_addr learned my addr 172.21.15.84:0/303967054 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:28:01.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.744+0000 7f978b7fe640 1 -- 172.21.15.84:0/303967054 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940622098 0 0) 0x7f978c0ca560 con 0x7f978c06c9e0 2023-10-16T11:28:01.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.744+0000 7f9790fee640 1 --2- 172.21.15.84:0/303967054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f978c0c9440 0x7f978c06f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.744+0000 7f978bfff640 1 --2- 172.21.15.84:0/303967054 >> v2:172.21.15.157:3300/0 conn(0x7f978c0c9060 0x7f978c06f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.744+0000 7f978b7fe640 1 -- 172.21.15.84:0/303967054 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9778003520 con 0x7f978c06c9e0 2023-10-16T11:28:01.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.744+0000 7f978b7fe640 1 -- 172.21.15.84:0/303967054 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2756942536 0 0) 0x7f9778003520 con 0x7f978c06c9e0 2023-10-16T11:28:01.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.744+0000 7f978b7fe640 1 -- 172.21.15.84:0/303967054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f978c0c9440 msgr2=0x7f978c06f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:01.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.744+0000 7f978b7fe640 1 --2- 172.21.15.84:0/303967054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f978c0c9440 0x7f978c06f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.744+0000 7f978b7fe640 1 -- 172.21.15.84:0/303967054 >> v2:172.21.15.157:3300/0 conn(0x7f978c0c9060 msgr2=0x7f978c06f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:01.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.744+0000 7f978b7fe640 1 --2- 172.21.15.84:0/303967054 >> v2:172.21.15.157:3300/0 conn(0x7f978c0c9060 0x7f978c06f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.744+0000 7f978b7fe640 1 -- 172.21.15.84:0/303967054 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c0c9f10 con 0x7f978c06c9e0 2023-10-16T11:28:01.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.744+0000 7f978b7fe640 1 -- 172.21.15.84:0/303967054 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f9774001670 con 0x7f978c06c9e0 2023-10-16T11:28:01.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.744+0000 7f978b7fe640 1 -- 172.21.15.84:0/303967054 >> v1:172.21.15.84:6789/0 conn(0x7f978c06c9e0 legacy=0x7f978c164d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.744+0000 7f978b7fe640 1 --2- 172.21.15.84:0/303967054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9778003970 0x7f9778003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.744+0000 7f978b7fe640 1 --2- 172.21.15.84:0/303967054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f978c0c9440 0x7f97780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.744+0000 7f978b7fe640 1 --2- 172.21.15.84:0/303967054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9778004d00 0x7f9778005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.744+0000 7f978b7fe640 1 -- 172.21.15.84:0/303967054 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0c9950 con 0x7f9778004d00 2023-10-16T11:28:01.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.744+0000 7f978b7fe640 1 -- 172.21.15.84:0/303967054 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0c9c30 con 0x7f9778003970 2023-10-16T11:28:01.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.744+0000 7f978b7fe640 1 -- 172.21.15.84:0/303967054 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977800ba30 con 0x7f978c0c9440 2023-10-16T11:28:01.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.745+0000 7f9790fee640 1 --2- 172.21.15.84:0/303967054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9778004d00 0x7f9778005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.745+0000 7f978bfff640 1 --2- 172.21.15.84:0/303967054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9778003970 0x7f9778003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.745+0000 7f9783fff640 1 --2- 172.21.15.84:0/303967054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f978c0c9440 0x7f97780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.745+0000 7f9790fee640 1 -- 172.21.15.84:0/303967054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f978c0c9440 msgr2=0x7f97780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:01.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.745+0000 7f9790fee640 1 --2- 172.21.15.84:0/303967054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f978c0c9440 0x7f97780047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.745+0000 7f9790fee640 1 -- 172.21.15.84:0/303967054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9778003970 msgr2=0x7f9778003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:01.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.745+0000 7f9790fee640 1 --2- 172.21.15.84:0/303967054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9778003970 0x7f9778003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.745+0000 7f9790fee640 1 -- 172.21.15.84:0/303967054 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9778003520 con 0x7f9778004d00 2023-10-16T11:28:01.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.745+0000 7f978bfff640 1 --2- 172.21.15.84:0/303967054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9778003970 0x7f9778003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:28:01.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.746+0000 7f9790fee640 1 --2- 172.21.15.84:0/303967054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9778004d00 0x7f9778005180 secure :-1 s=READY pgs=3580 cs=0 l=1 rev1=1 crypto rx=0x7f9784007360 tx=0x7f97780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1978271d58bfc2ce server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:01.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.746+0000 7f978b7fe640 1 -- 172.21.15.84:0/303967054 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978402cc80 con 0x7f9778004d00 2023-10-16T11:28:01.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.746+0000 7f978b7fe640 1 -- 172.21.15.84:0/303967054 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978402cde0 con 0x7f9778004d00 2023-10-16T11:28:01.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.746+0000 7f978b7fe640 1 -- 172.21.15.84:0/303967054 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9784034680 con 0x7f9778004d00 2023-10-16T11:28:01.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.747+0000 7f97917ef640 1 -- 172.21.15.84:0/303967054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9778004d00 msgr2=0x7f9778005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.747+0000 7f97917ef640 1 --2- 172.21.15.84:0/303967054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9778004d00 0x7f9778005180 secure :-1 s=READY pgs=3580 cs=0 l=1 rev1=1 crypto rx=0x7f9784007360 tx=0x7f97780057f0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.747+0000 7f9790fee640 1 -- 172.21.15.84:0/303967054 reap_dead start 2023-10-16T11:28:01.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.747+0000 7f97917ef640 1 -- 172.21.15.84:0/303967054 shutdown_connections 2023-10-16T11:28:01.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.747+0000 7f97917ef640 1 -- 172.21.15.84:0/303967054 >> 172.21.15.84:0/303967054 conn(0x7f978c0ce6a0 msgr2=0x7f978c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:01.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.747+0000 7f97917ef640 1 -- 172.21.15.84:0/303967054 shutdown_connections 2023-10-16T11:28:01.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.747+0000 7f97917ef640 1 -- 172.21.15.84:0/303967054 wait complete. 2023-10-16T11:28:01.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.749+0000 7f97917ef640 1 Processor -- start 2023-10-16T11:28:01.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.749+0000 7f97917ef640 1 -- start start 2023-10-16T11:28:01.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.750+0000 7f97917ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9778004d00 0x7f978c15fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.750+0000 7f97917ef640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f978c0b0d10 0x7f978c0c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.750+0000 7f97917ef640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f978c0b1190 0x7f978c0c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.750+0000 7f97917ef640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c161790 con 0x7f978c0b1190 2023-10-16T11:28:01.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.750+0000 7f97917ef640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c160290 con 0x7f9778004d00 2023-10-16T11:28:01.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.750+0000 7f97917ef640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978c160b80 con 0x7f978c0b0d10 2023-10-16T11:28:01.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.750+0000 7f9790fee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f978c0b1190 0x7f978c0c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.750+0000 7f9790fee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f978c0b1190 0x7f978c0c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52586/0 (socket says 172.21.15.84:52586) 2023-10-16T11:28:01.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.750+0000 7f9790fee640 1 -- 172.21.15.84:0/4287061242 learned_addr learned my addr 172.21.15.84:0/4287061242 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:28:01.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.751+0000 7f9790fee640 1 -- 172.21.15.84:0/4287061242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f978c0b0d10 msgr2=0x7f978c0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.751+0000 7f9790fee640 1 --2- 172.21.15.84:0/4287061242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f978c0b0d10 0x7f978c0c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.751+0000 7f9790fee640 1 -- 172.21.15.84:0/4287061242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9778004d00 msgr2=0x7f978c15fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.751+0000 7f9790fee640 1 --2- 172.21.15.84:0/4287061242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9778004d00 0x7f978c15fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.751+0000 7f9790fee640 1 -- 172.21.15.84:0/4287061242 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c160e60 con 0x7f978c0b1190 2023-10-16T11:28:01.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.751+0000 7f9790fee640 1 --2- 172.21.15.84:0/4287061242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f978c0b1190 0x7f978c0c7e80 secure :-1 s=READY pgs=3581 cs=0 l=1 rev1=1 crypto rx=0x7f9784037040 tx=0x7f9784004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:01.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.751+0000 7f97897fa640 1 -- 172.21.15.84:0/4287061242 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978403e640 con 0x7f978c0b1190 2023-10-16T11:28:01.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.751+0000 7f97897fa640 1 -- 172.21.15.84:0/4287061242 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978403e7a0 con 0x7f978c0b1190 2023-10-16T11:28:01.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.751+0000 7f97897fa640 1 -- 172.21.15.84:0/4287061242 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978403eb50 con 0x7f978c0b1190 2023-10-16T11:28:01.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.751+0000 7f97917ef640 1 -- 172.21.15.84:0/4287061242 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f978c161140 con 0x7f978c0b1190 2023-10-16T11:28:01.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.752+0000 7f97917ef640 1 -- 172.21.15.84:0/4287061242 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978c0b0660 con 0x7f978c0b1190 2023-10-16T11:28:01.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.752+0000 7f97917ef640 1 -- 172.21.15.84:0/4287061242 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f978c0c1f10 con 0x7f978c0b1190 2023-10-16T11:28:01.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.753+0000 7f97897fa640 1 -- 172.21.15.84:0/4287061242 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f978403ecb0 con 0x7f978c0b1190 2023-10-16T11:28:01.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.754+0000 7f97897fa640 1 --2- 172.21.15.84:0/4287061242 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f976c06af90 0x7f976c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.754+0000 7f97897fa640 1 -- 172.21.15.84:0/4287061242 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f97840abb40 con 0x7f978c0b1190 2023-10-16T11:28:01.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.755+0000 7f97897fa640 1 --2- 172.21.15.84:0/4287061242 >> [v2:172.21.15.157:6819/1752891116,v1:172.21.15.157:6821/1752891116] conn(0x7f976c0713e0 0x7f976c073800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:01.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.755+0000 7f978bfff640 1 --2- 172.21.15.84:0/4287061242 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f976c06af90 0x7f976c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.755+0000 7f97897fa640 1 -- 172.21.15.84:0/4287061242 --> [v2:172.21.15.157:6819/1752891116,v1:172.21.15.157:6821/1752891116] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f976c073ed0 con 0x7f976c0713e0 2023-10-16T11:28:01.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.755+0000 7f97897fa640 1 -- 172.21.15.84:0/4287061242 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=275) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97840adea0 con 0x7f978c0b1190 2023-10-16T11:28:01.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.755+0000 7f978bfff640 1 --2- 172.21.15.84:0/4287061242 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f976c06af90 0x7f976c06d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f977c006fd0 tx=0x7f977c0093d0 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:01.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.755+0000 7f9783fff640 1 --2- 172.21.15.84:0/4287061242 >> [v2:172.21.15.157:6819/1752891116,v1:172.21.15.157:6821/1752891116] conn(0x7f976c0713e0 0x7f976c073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:01.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.756+0000 7f9783fff640 1 --2- 172.21.15.84:0/4287061242 >> [v2:172.21.15.157:6819/1752891116,v1:172.21.15.157:6821/1752891116] conn(0x7f976c0713e0 0x7f976c073800 crc :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:01.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.758+0000 7f97897fa640 1 -- 172.21.15.84:0/4287061242 <== osd.7 v2:172.21.15.157:6819/1752891116 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f976c073ed0 con 0x7f976c0713e0 2023-10-16T11:28:01.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.772+0000 7f97917ef640 1 -- 172.21.15.84:0/4287061242 --> [v2:172.21.15.157:6819/1752891116,v1:172.21.15.157:6821/1752891116] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f978c0ca1f0 con 0x7f976c0713e0 2023-10-16T11:28:01.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.773+0000 7f97897fa640 1 -- 172.21.15.84:0/4287061242 <== osd.7 v2:172.21.15.157:6819/1752891116 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9774007b40 con 0x7f976c0713e0 2023-10-16T11:28:01.775 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-16T11:28:01.775 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T11:28:01.775 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-16T11:28:01.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.774+0000 7f97917ef640 1 -- 172.21.15.84:0/4287061242 >> [v2:172.21.15.157:6819/1752891116,v1:172.21.15.157:6821/1752891116] conn(0x7f976c0713e0 msgr2=0x7f976c073800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.774+0000 7f97917ef640 1 --2- 172.21.15.84:0/4287061242 >> [v2:172.21.15.157:6819/1752891116,v1:172.21.15.157:6821/1752891116] conn(0x7f976c0713e0 0x7f976c073800 crc :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.774+0000 7f97917ef640 1 -- 172.21.15.84:0/4287061242 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f976c06af90 msgr2=0x7f976c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.774+0000 7f97917ef640 1 --2- 172.21.15.84:0/4287061242 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f976c06af90 0x7f976c06d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f977c006fd0 tx=0x7f977c0093d0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.774+0000 7f97917ef640 1 -- 172.21.15.84:0/4287061242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f978c0b1190 msgr2=0x7f978c0c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:01.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.774+0000 7f97917ef640 1 --2- 172.21.15.84:0/4287061242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f978c0b1190 0x7f978c0c7e80 secure :-1 s=READY pgs=3581 cs=0 l=1 rev1=1 crypto rx=0x7f9784037040 tx=0x7f9784004ab0 comp rx=0 tx=0).stop 2023-10-16T11:28:01.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.774+0000 7f9790fee640 1 -- 172.21.15.84:0/4287061242 reap_dead start 2023-10-16T11:28:01.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.774+0000 7f97917ef640 1 -- 172.21.15.84:0/4287061242 shutdown_connections 2023-10-16T11:28:01.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.774+0000 7f97917ef640 1 -- 172.21.15.84:0/4287061242 >> 172.21.15.84:0/4287061242 conn(0x7f978c0ce6a0 msgr2=0x7f978c06fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:01.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.774+0000 7f97917ef640 1 -- 172.21.15.84:0/4287061242 shutdown_connections 2023-10-16T11:28:01.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.774+0000 7f97917ef640 1 -- 172.21.15.84:0/4287061242 wait complete. 2023-10-16T11:28:01.793 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-16T11:28:01.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.998+0000 7f817781f640 1 Processor -- start 2023-10-16T11:28:02.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:01.999+0000 7f817781f640 1 -- start start 2023-10-16T11:28:02.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.000+0000 7f817781f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f817015ad70 0x7f8170069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:02.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.000+0000 7f817781f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f81700caac0 0x7f817006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:02.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.000+0000 7f817781f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81700cb030 con 0x7f81700c9ca0 2023-10-16T11:28:02.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.000+0000 7f817781f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81700c91c0 con 0x7f817015ad70 2023-10-16T11:28:02.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.000+0000 7f817781f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81700c94a0 con 0x7f81700caac0 2023-10-16T11:28:02.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.000+0000 7f81755bb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f817015ad70 0x7f8170069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:02.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.000+0000 7f8174dba640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f81700c9ca0 0x7f81700ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60602/0 (socket says 172.21.15.84:60602) 2023-10-16T11:28:02.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.000+0000 7f8174dba640 1 -- 172.21.15.84:0/3389061639 learned_addr learned my addr 172.21.15.84:0/3389061639 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:28:02.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.001+0000 7f8175dbc640 1 --2- 172.21.15.84:0/3389061639 >> v2:172.21.15.157:3300/0 conn(0x7f81700caac0 0x7f817006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:02.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.001+0000 7f81755bb640 1 -- 172.21.15.84:0/3389061639 >> v2:172.21.15.157:3300/0 conn(0x7f81700caac0 msgr2=0x7f817006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:02.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.001+0000 7f81755bb640 1 --2- 172.21.15.84:0/3389061639 >> v2:172.21.15.157:3300/0 conn(0x7f81700caac0 0x7f817006a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.001+0000 7f81755bb640 1 -- 172.21.15.84:0/3389061639 >> v1:172.21.15.84:6789/0 conn(0x7f81700c9ca0 legacy=0x7f81700ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:02.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.001+0000 7f8167fff640 1 -- 172.21.15.84:0/3389061639 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961171531 0 0) 0x7f81700cb030 con 0x7f81700c9ca0 2023-10-16T11:28:02.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.001+0000 7f81755bb640 1 -- 172.21.15.84:0/3389061639 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81700c9780 con 0x7f817015ad70 2023-10-16T11:28:02.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.002+0000 7f81755bb640 1 --2- 172.21.15.84:0/3389061639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f817015ad70 0x7f8170069da0 secure :-1 s=READY pgs=3583 cs=0 l=1 rev1=1 crypto rx=0x7f817006d090 tx=0x7f816c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb603d410934871 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:02.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.003+0000 7f8167fff640 1 -- 172.21.15.84:0/3389061639 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c031830 con 0x7f817015ad70 2023-10-16T11:28:02.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.003+0000 7f8167fff640 1 -- 172.21.15.84:0/3389061639 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816c031990 con 0x7f817015ad70 2023-10-16T11:28:02.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.003+0000 7f8167fff640 1 -- 172.21.15.84:0/3389061639 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c031d20 con 0x7f817015ad70 2023-10-16T11:28:02.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.003+0000 7f817781f640 1 -- 172.21.15.84:0/3389061639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f817015ad70 msgr2=0x7f8170069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:02.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.003+0000 7f817781f640 1 --2- 172.21.15.84:0/3389061639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f817015ad70 0x7f8170069da0 secure :-1 s=READY pgs=3583 cs=0 l=1 rev1=1 crypto rx=0x7f817006d090 tx=0x7f816c02b8d0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.004+0000 7f817781f640 1 -- 172.21.15.84:0/3389061639 shutdown_connections 2023-10-16T11:28:02.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.004+0000 7f817781f640 1 --2- 172.21.15.84:0/3389061639 >> v2:172.21.15.157:3300/0 conn(0x7f81700caac0 0x7f817006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.004+0000 7f817781f640 1 --2- 172.21.15.84:0/3389061639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f817015ad70 0x7f8170069da0 unknown :-1 s=CLOSED pgs=3583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.004+0000 7f817781f640 1 -- 172.21.15.84:0/3389061639 >> 172.21.15.84:0/3389061639 conn(0x7f81700ce9b0 msgr2=0x7f81700cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:02.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.004+0000 7f817781f640 1 -- 172.21.15.84:0/3389061639 shutdown_connections 2023-10-16T11:28:02.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.004+0000 7f817781f640 1 -- 172.21.15.84:0/3389061639 wait complete. 2023-10-16T11:28:02.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.005+0000 7f817781f640 1 Processor -- start 2023-10-16T11:28:02.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.006+0000 7f817781f640 1 -- start start 2023-10-16T11:28:02.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.006+0000 7f817781f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81700c9ca0 0x7f81700c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:02.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.006+0000 7f817781f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81700caac0 0x7f81700c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:02.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.006+0000 7f817781f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f817015ad70 0x7f8170150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:02.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.006+0000 7f817781f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81700c6270 con 0x7f81700caac0 2023-10-16T11:28:02.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.006+0000 7f817781f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817015cff0 con 0x7f81700c9ca0 2023-10-16T11:28:02.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.006+0000 7f817781f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817015d2d0 con 0x7f817015ad70 2023-10-16T11:28:02.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.007+0000 7f8174dba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81700caac0 0x7f81700c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:02.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.007+0000 7f8174dba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81700caac0 0x7f81700c6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52606/0 (socket says 172.21.15.84:52606) 2023-10-16T11:28:02.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.007+0000 7f8174dba640 1 -- 172.21.15.84:0/2976809912 learned_addr learned my addr 172.21.15.84:0/2976809912 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:28:02.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.007+0000 7f81755bb640 1 --2- 172.21.15.84:0/2976809912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81700c9ca0 0x7f81700c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:02.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.007+0000 7f8175dbc640 1 --2- 172.21.15.84:0/2976809912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f817015ad70 0x7f8170150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:02.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.007+0000 7f8174dba640 1 -- 172.21.15.84:0/2976809912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f817015ad70 msgr2=0x7f8170150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:02.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.007+0000 7f8174dba640 1 --2- 172.21.15.84:0/2976809912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f817015ad70 0x7f8170150a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.007+0000 7f8174dba640 1 -- 172.21.15.84:0/2976809912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81700c9ca0 msgr2=0x7f81700c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:02.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.007+0000 7f8174dba640 1 --2- 172.21.15.84:0/2976809912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81700c9ca0 0x7f81700c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.007+0000 7f8174dba640 1 -- 172.21.15.84:0/2976809912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c009400 con 0x7f81700caac0 2023-10-16T11:28:02.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.008+0000 7f8174dba640 1 --2- 172.21.15.84:0/2976809912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81700caac0 0x7f81700c6be0 secure :-1 s=READY pgs=3584 cs=0 l=1 rev1=1 crypto rx=0x7f816000c4d0 tx=0x7f816000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:02.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.008+0000 7f8165ffb640 1 -- 172.21.15.84:0/2976809912 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81600074e0 con 0x7f81700caac0 2023-10-16T11:28:02.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.008+0000 7f8165ffb640 1 -- 172.21.15.84:0/2976809912 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8160007660 con 0x7f81700caac0 2023-10-16T11:28:02.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.008+0000 7f8165ffb640 1 -- 172.21.15.84:0/2976809912 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160004a20 con 0x7f81700caac0 2023-10-16T11:28:02.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.008+0000 7f817781f640 1 -- 172.21.15.84:0/2976809912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817015d5b0 con 0x7f81700caac0 2023-10-16T11:28:02.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.008+0000 7f817781f640 1 -- 172.21.15.84:0/2976809912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817015db70 con 0x7f81700caac0 2023-10-16T11:28:02.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.009+0000 7f817781f640 1 -- 172.21.15.84:0/2976809912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81700ad4b0 con 0x7f81700caac0 2023-10-16T11:28:02.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.010+0000 7f8165ffb640 1 -- 172.21.15.84:0/2976809912 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8160004b80 con 0x7f81700caac0 2023-10-16T11:28:02.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.011+0000 7f8165ffb640 1 --2- 172.21.15.84:0/2976809912 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f815406af90 0x7f815406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:02.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.011+0000 7f8165ffb640 1 -- 172.21.15.84:0/2976809912 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f8160088900 con 0x7f81700caac0 2023-10-16T11:28:02.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.012+0000 7f81755bb640 1 --2- 172.21.15.84:0/2976809912 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f815406af90 0x7f815406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:02.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.012+0000 7f81755bb640 1 --2- 172.21.15.84:0/2976809912 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f815406af90 0x7f815406d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f816c036000 tx=0x7f816c03c040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:02.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.016+0000 7f8165ffb640 1 -- 172.21.15.84:0/2976809912 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f81700ad4b0 con 0x7f81700caac0 2023-10-16T11:28:02.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.208+0000 7f817781f640 1 -- 172.21.15.84:0/2976809912 --> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f81700ad4b0 con 0x7f815406af90 2023-10-16T11:28:02.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.209+0000 7f8165ffb640 1 -- 172.21.15.84:0/2976809912 <== mgr.13719 v2:172.21.15.84:6832/3003264401 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f81700ad4b0 con 0x7f815406af90 2023-10-16T11:28:02.211 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 7 to deep-scrub 2023-10-16T11:28:02.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.212+0000 7f817781f640 1 -- 172.21.15.84:0/2976809912 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f815406af90 msgr2=0x7f815406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:02.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.212+0000 7f817781f640 1 --2- 172.21.15.84:0/2976809912 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f815406af90 0x7f815406d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f816c036000 tx=0x7f816c03c040 comp rx=0 tx=0).stop 2023-10-16T11:28:02.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.212+0000 7f817781f640 1 -- 172.21.15.84:0/2976809912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81700caac0 msgr2=0x7f81700c6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:02.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.212+0000 7f817781f640 1 --2- 172.21.15.84:0/2976809912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81700caac0 0x7f81700c6be0 secure :-1 s=READY pgs=3584 cs=0 l=1 rev1=1 crypto rx=0x7f816000c4d0 tx=0x7f816000c990 comp rx=0 tx=0).stop 2023-10-16T11:28:02.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.212+0000 7f817781f640 1 -- 172.21.15.84:0/2976809912 shutdown_connections 2023-10-16T11:28:02.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.212+0000 7f817781f640 1 --2- 172.21.15.84:0/2976809912 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f815406af90 0x7f815406d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.212+0000 7f817781f640 1 --2- 172.21.15.84:0/2976809912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f817015ad70 0x7f8170150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.212+0000 7f817781f640 1 --2- 172.21.15.84:0/2976809912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81700caac0 0x7f81700c6be0 unknown :-1 s=CLOSED pgs=3584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.212+0000 7f817781f640 1 --2- 172.21.15.84:0/2976809912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81700c9ca0 0x7f81700c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.212+0000 7f817781f640 1 -- 172.21.15.84:0/2976809912 >> 172.21.15.84:0/2976809912 conn(0x7f81700ce9b0 msgr2=0x7f81700ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:02.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.212+0000 7f817781f640 1 -- 172.21.15.84:0/2976809912 shutdown_connections 2023-10-16T11:28:02.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.212+0000 7f817781f640 1 -- 172.21.15.84:0/2976809912 wait complete. 2023-10-16T11:28:02.231 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:28:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.443+0000 7fc4296e7640 1 Processor -- start 2023-10-16T11:28:02.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.444+0000 7fc4296e7640 1 -- start start 2023-10-16T11:28:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.445+0000 7fc4296e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc42415ae80 0x7fc42415e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.445+0000 7fc4296e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4240ce390 0x7fc424070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.445+0000 7fc4296e7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc42415ed60 con 0x7fc42415aaa0 2023-10-16T11:28:02.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.445+0000 7fc4296e7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc42406b810 con 0x7fc4240ce390 2023-10-16T11:28:02.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.445+0000 7fc4296e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240707e0 con 0x7fc42415ae80 2023-10-16T11:28:02.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.446+0000 7fc422ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc42415aaa0 0x7fc4240cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60616/0 (socket says 172.21.15.84:60616) 2023-10-16T11:28:02.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.446+0000 7fc422ffd640 1 -- 172.21.15.84:0/2112863649 learned_addr learned my addr 172.21.15.84:0/2112863649 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:28:02.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.446+0000 7fc41a7fc640 1 --2- 172.21.15.84:0/2112863649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc42415ae80 0x7fc42415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:02.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.446+0000 7fc4237fe640 1 --2- 172.21.15.84:0/2112863649 >> v2:172.21.15.157:3300/0 conn(0x7fc4240ce390 0x7fc424070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:02.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.446+0000 7fc4227fc640 1 -- 172.21.15.84:0/2112863649 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 717757633 0 0) 0x7fc42415ed60 con 0x7fc42415aaa0 2023-10-16T11:28:02.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.447+0000 7fc4227fc640 1 -- 172.21.15.84:0/2112863649 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc410003520 con 0x7fc42415aaa0 2023-10-16T11:28:02.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.447+0000 7fc4227fc640 1 -- 172.21.15.84:0/2112863649 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2975333694 0 0) 0x7fc410003520 con 0x7fc42415aaa0 2023-10-16T11:28:02.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.447+0000 7fc4227fc640 1 -- 172.21.15.84:0/2112863649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc42415ae80 msgr2=0x7fc42415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:02.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.447+0000 7fc4227fc640 1 --2- 172.21.15.84:0/2112863649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc42415ae80 0x7fc42415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.447+0000 7fc4227fc640 1 -- 172.21.15.84:0/2112863649 >> v2:172.21.15.157:3300/0 conn(0x7fc4240ce390 msgr2=0x7fc424070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:02.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.447+0000 7fc4227fc640 1 --2- 172.21.15.84:0/2112863649 >> v2:172.21.15.157:3300/0 conn(0x7fc4240ce390 0x7fc424070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.447+0000 7fc4227fc640 1 -- 172.21.15.84:0/2112863649 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42415ef90 con 0x7fc42415aaa0 2023-10-16T11:28:02.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.447+0000 7fc4227fc640 1 -- 172.21.15.84:0/2112863649 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7fc414001670 con 0x7fc42415aaa0 2023-10-16T11:28:02.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.447+0000 7fc4227fc640 1 -- 172.21.15.84:0/2112863649 >> v1:172.21.15.84:6789/0 conn(0x7fc42415aaa0 legacy=0x7fc4240cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:02.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.447+0000 7fc4227fc640 1 --2- 172.21.15.84:0/2112863649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc410003970 0x7fc410003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:02.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.447+0000 7fc4227fc640 1 --2- 172.21.15.84:0/2112863649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc42415ae80 0x7fc4100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:02.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.447+0000 7fc4227fc640 1 --2- 172.21.15.84:0/2112863649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc410004d00 0x7fc410005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:02.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.447+0000 7fc4227fc640 1 -- 172.21.15.84:0/2112863649 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42406b810 con 0x7fc410003970 2023-10-16T11:28:02.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.447+0000 7fc4227fc640 1 -- 172.21.15.84:0/2112863649 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240707e0 con 0x7fc410004d00 2023-10-16T11:28:02.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.447+0000 7fc4227fc640 1 -- 172.21.15.84:0/2112863649 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41000b980 con 0x7fc42415ae80 2023-10-16T11:28:02.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.447+0000 7fc422ffd640 1 --2- 172.21.15.84:0/2112863649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc410003970 0x7fc410003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.448+0000 7fc41a7fc640 1 --2- 172.21.15.84:0/2112863649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc42415ae80 0x7fc4100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.448+0000 7fc422ffd640 1 -- 172.21.15.84:0/2112863649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc42415ae80 msgr2=0x7fc4100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.448+0000 7fc422ffd640 1 --2- 172.21.15.84:0/2112863649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc42415ae80 0x7fc4100047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.448+0000 7fc422ffd640 1 -- 172.21.15.84:0/2112863649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc410004d00 msgr2=0x7fc410005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:02.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.448+0000 7fc422ffd640 1 --2- 172.21.15.84:0/2112863649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc410004d00 0x7fc410005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.448+0000 7fc422ffd640 1 -- 172.21.15.84:0/2112863649 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc410003520 con 0x7fc410003970 2023-10-16T11:28:02.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.449+0000 7fc422ffd640 1 --2- 172.21.15.84:0/2112863649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc410003970 0x7fc410003d90 secure :-1 s=READY pgs=3586 cs=0 l=1 rev1=1 crypto rx=0x7fc41400c2c0 tx=0x7fc4100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b453c7e0ff82b033 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:02.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.450+0000 7fc4227fc640 1 -- 172.21.15.84:0/2112863649 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc414005ad0 con 0x7fc410003970 2023-10-16T11:28:02.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.450+0000 7fc4227fc640 1 -- 172.21.15.84:0/2112863649 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc414005c30 con 0x7fc410003970 2023-10-16T11:28:02.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.450+0000 7fc4227fc640 1 -- 172.21.15.84:0/2112863649 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41402c8b0 con 0x7fc410003970 2023-10-16T11:28:02.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.450+0000 7fc4296e7640 1 -- 172.21.15.84:0/2112863649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc410003970 msgr2=0x7fc410003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:02.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.450+0000 7fc4296e7640 1 --2- 172.21.15.84:0/2112863649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc410003970 0x7fc410003d90 secure :-1 s=READY pgs=3586 cs=0 l=1 rev1=1 crypto rx=0x7fc41400c2c0 tx=0x7fc4100057f0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.450+0000 7fc4237fe640 1 -- 172.21.15.84:0/2112863649 reap_dead start 2023-10-16T11:28:02.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.450+0000 7fc4296e7640 1 -- 172.21.15.84:0/2112863649 shutdown_connections 2023-10-16T11:28:02.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.450+0000 7fc4296e7640 1 -- 172.21.15.84:0/2112863649 >> 172.21.15.84:0/2112863649 conn(0x7fc4240cb410 msgr2=0x7fc4240cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:02.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.450+0000 7fc4296e7640 1 -- 172.21.15.84:0/2112863649 shutdown_connections 2023-10-16T11:28:02.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.450+0000 7fc4296e7640 1 -- 172.21.15.84:0/2112863649 wait complete. 2023-10-16T11:28:02.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.452+0000 7fc4296e7640 1 Processor -- start 2023-10-16T11:28:02.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.452+0000 7fc4296e7640 1 -- start start 2023-10-16T11:28:02.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.453+0000 7fc4296e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc424155ba0 0x7fc4240c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:02.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.453+0000 7fc4296e7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4240c3a30 0x7fc424156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:02.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.453+0000 7fc4296e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc424156b90 0x7fc424149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:02.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.453+0000 7fc4296e7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc424157010 con 0x7fc424155ba0 2023-10-16T11:28:02.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.453+0000 7fc4296e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240c4b80 con 0x7fc424156b90 2023-10-16T11:28:02.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.453+0000 7fc4296e7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc424155f80 con 0x7fc4240c3a30 2023-10-16T11:28:02.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.454+0000 7fc422ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc424155ba0 0x7fc4240c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:02.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.454+0000 7fc422ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc424155ba0 0x7fc4240c34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52618/0 (socket says 172.21.15.84:52618) 2023-10-16T11:28:02.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.454+0000 7fc422ffd640 1 -- 172.21.15.84:0/3748664538 learned_addr learned my addr 172.21.15.84:0/3748664538 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:28:02.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.454+0000 7fc422ffd640 1 -- 172.21.15.84:0/3748664538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4240c3a30 msgr2=0x7fc424156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:02.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.454+0000 7fc422ffd640 1 --2- 172.21.15.84:0/3748664538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4240c3a30 0x7fc424156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.454+0000 7fc422ffd640 1 -- 172.21.15.84:0/3748664538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc424156b90 msgr2=0x7fc424149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:02.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.454+0000 7fc4237fe640 1 --2- 172.21.15.84:0/3748664538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc424156b90 0x7fc424149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:02.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.454+0000 7fc422ffd640 1 --2- 172.21.15.84:0/3748664538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc424156b90 0x7fc424149be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.454+0000 7fc422ffd640 1 -- 172.21.15.84:0/3748664538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4240c3f70 con 0x7fc424155ba0 2023-10-16T11:28:02.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.454+0000 7fc422ffd640 1 --2- 172.21.15.84:0/3748664538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc424155ba0 0x7fc4240c34f0 secure :-1 s=READY pgs=3587 cs=0 l=1 rev1=1 crypto rx=0x7fc414004840 tx=0x7fc414001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:02.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.455+0000 7fc41bfff640 1 -- 172.21.15.84:0/3748664538 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41403c040 con 0x7fc424155ba0 2023-10-16T11:28:02.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.455+0000 7fc41bfff640 1 -- 172.21.15.84:0/3748664538 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41402cdc0 con 0x7fc424155ba0 2023-10-16T11:28:02.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.455+0000 7fc41bfff640 1 -- 172.21.15.84:0/3748664538 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4140072b0 con 0x7fc424155ba0 2023-10-16T11:28:02.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.455+0000 7fc4296e7640 1 -- 172.21.15.84:0/3748664538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4240c4250 con 0x7fc424155ba0 2023-10-16T11:28:02.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.455+0000 7fc4296e7640 1 -- 172.21.15.84:0/3748664538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4240c4810 con 0x7fc424155ba0 2023-10-16T11:28:02.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.456+0000 7fc4296e7640 1 -- 172.21.15.84:0/3748664538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4240c49d0 con 0x7fc424155ba0 2023-10-16T11:28:02.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.457+0000 7fc41bfff640 1 -- 172.21.15.84:0/3748664538 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc414032070 con 0x7fc424155ba0 2023-10-16T11:28:02.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.458+0000 7fc41bfff640 1 --2- 172.21.15.84:0/3748664538 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fc40406afc0 0x7fc40406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:02.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.458+0000 7fc41bfff640 1 -- 172.21.15.84:0/3748664538 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fc4140aa950 con 0x7fc424155ba0 2023-10-16T11:28:02.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.458+0000 7fc41a7fc640 1 --2- 172.21.15.84:0/3748664538 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fc40406afc0 0x7fc40406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:02.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.459+0000 7fc41a7fc640 1 --2- 172.21.15.84:0/3748664538 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fc40406afc0 0x7fc40406d480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc4241f5fc0 tx=0x7fc40c008040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:02.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.462+0000 7fc41bfff640 1 -- 172.21.15.84:0/3748664538 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc414075880 con 0x7fc424155ba0 2023-10-16T11:28:02.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.650+0000 7fc4296e7640 1 -- 172.21.15.84:0/3748664538 --> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc42415f550 con 0x7fc40406afc0 2023-10-16T11:28:02.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.655+0000 7fc41bfff640 1 -- 172.21.15.84:0/3748664538 <== mgr.13719 v2:172.21.15.84:6832/3003264401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48521 (secure 0 0 0) 0x7fc40c0095c0 con 0x7fc40406afc0 2023-10-16T11:28:02.656 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:28:02.657 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:28:02.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.657+0000 7fc4296e7640 1 -- 172.21.15.84:0/3748664538 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fc40406afc0 msgr2=0x7fc40406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:02.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.657+0000 7fc4296e7640 1 --2- 172.21.15.84:0/3748664538 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fc40406afc0 0x7fc40406d480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc4241f5fc0 tx=0x7fc40c008040 comp rx=0 tx=0).stop 2023-10-16T11:28:02.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.657+0000 7fc4296e7640 1 -- 172.21.15.84:0/3748664538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc424155ba0 msgr2=0x7fc4240c34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:02.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.657+0000 7fc4296e7640 1 --2- 172.21.15.84:0/3748664538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc424155ba0 0x7fc4240c34f0 secure :-1 s=READY pgs=3587 cs=0 l=1 rev1=1 crypto rx=0x7fc414004840 tx=0x7fc414001a00 comp rx=0 tx=0).stop 2023-10-16T11:28:02.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.658+0000 7fc4296e7640 1 -- 172.21.15.84:0/3748664538 shutdown_connections 2023-10-16T11:28:02.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.658+0000 7fc4296e7640 1 --2- 172.21.15.84:0/3748664538 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fc40406afc0 0x7fc40406d480 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.658+0000 7fc4296e7640 1 --2- 172.21.15.84:0/3748664538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc424156b90 0x7fc424149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.658+0000 7fc4296e7640 1 --2- 172.21.15.84:0/3748664538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4240c3a30 0x7fc424156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.658+0000 7fc4296e7640 1 --2- 172.21.15.84:0/3748664538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc424155ba0 0x7fc4240c34f0 unknown :-1 s=CLOSED pgs=3587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:02.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.658+0000 7fc4296e7640 1 -- 172.21.15.84:0/3748664538 >> 172.21.15.84:0/3748664538 conn(0x7fc4240cb410 msgr2=0x7fc424070df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:02.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.658+0000 7fc4296e7640 1 -- 172.21.15.84:0/3748664538 shutdown_connections 2023-10-16T11:28:02.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:02.658+0000 7fc4296e7640 1 -- 172.21.15.84:0/3748664538 wait complete. 2023-10-16T11:28:02.677 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2023-10-16T11:28:01.494066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2783,"num_read_kb":5735,"num_write":703,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4836,"ondisk_log_size":4836,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":480476,"kb_used_data":28392,"kb_used_omap":12,"kb_used_meta":451955,"kb_avail":754494244,"statfs":{"total":773094113280,"available":772602105856,"internally_reserved":0,"allocated":29073408,"data_stored":14830088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":462802489},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.225241"},"pg_stats":[{"pgid":"1.0","version":"272'4836","reported_seq":5962,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-16T11:27:49.476743+0000","last_change":"2023-10-16T09:48:57.578325+0000","last_active":"2023-10-16T11:27:49.476743+0000","last_peered":"2023-10-16T11:27:49.476743+0000","last_clean":"2023-10-16T11:27:49.476743+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:27:49.476743+0000","last_undegraded":"2023-10-16T11:27:49.476743+0000","last_fullsized":"2023-10-16T11:27:49.476743+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:48:56.337130+0000","last_clean_scrub_stamp":"2023-10-16T09:48:56.337130+0000","objects_scrubbed":0,"log_size":4836,"log_dups_size":0,"ondisk_log_size":4836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:46.682897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2783,"num_read_kb":5735,"num_write":703,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2783,"num_read_kb":5735,"num_write":703,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4836,"ondisk_log_size":4836,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576024,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79580,"kb_used_data":3984,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94292260,"statfs":{"total":96636764160,"available":96555274240,"internally_reserved":0,"allocated":4079616,"data_stored":2296537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":77396427},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.115,"15min":1.1719999999999999},"min":{"1min":0.65300000000000002,"5min":0.64500000000000002,"15min":0.44400000000000001},"max":{"1min":1.655,"5min":1.7250000000000001,"15min":2.3399999999999999},"last":1.095},{"interface":"front","average":{"1min":1.03,"5min":1.1220000000000001,"15min":1.1779999999999999},"min":{"1min":0.63100000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.63100000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:27:58 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.1699999999999999,"15min":1.226},"min":{"1min":0.755,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.655,"5min":1.96,"15min":1.96},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.161,"15min":1.2370000000000001},"min":{"1min":0.72799999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.5089999999999999,"5min":1.6799999999999999,"15min":1.8600000000000001},"last":0.73299999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:27:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.2330000000000001,"15min":1.276},"min":{"1min":0.69199999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6279999999999999,"5min":1.9039999999999999,"15min":2.4460000000000002},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.278,"15min":1.331},"min":{"1min":0.85499999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.5429999999999999,"5min":3.9249999999999998,"15min":3.9249999999999998},"last":0.90000000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:27:58 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.339,"15min":1.365},"min":{"1min":0.97999999999999998,"5min":0.85999999999999999,"15min":0.67100000000000004},"max":{"1min":1.681,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.054},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3640000000000001,"15min":1.431},"min":{"1min":1.004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.7569999999999999,"5min":2.016,"15min":2.577},"last":1.488}]},{"osd":4,"last update":"Mon Oct 16 11:27:58 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.4139999999999999,"15min":1.4079999999999999},"min":{"1min":1.0680000000000001,"5min":0.64800000000000002,"15min":0.48999999999999999},"max":{"1min":1.9830000000000001,"5min":2.113,"15min":2.6779999999999999},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.397,"15min":1.387},"min":{"1min":0.97699999999999998,"5min":0.65600000000000003,"15min":0.51200000000000001},"max":{"1min":2.335,"5min":2.335,"15min":2.335},"last":1.266}]},{"osd":5,"last update":"Mon Oct 16 11:27:58 2023","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.5600000000000001,"15min":1.583},"min":{"1min":0.91800000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.4159999999999999,"5min":2.4329999999999998,"15min":3.3519999999999999},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.53,"15min":1.5049999999999999},"min":{"1min":1.133,"5min":0.62,"15min":0.60499999999999998},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.3460000000000001},"last":1.167}]},{"osd":6,"last update":"Mon Oct 16 11:27:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.488,"15min":1.5009999999999999},"min":{"1min":1.1479999999999999,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.0739999999999998,"5min":2.2469999999999999,"15min":2.7250000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.694,"5min":1.657,"15min":1.6899999999999999},"min":{"1min":1.1599999999999999,"5min":0.749,"15min":0.65000000000000002},"max":{"1min":2.2599999999999998,"5min":2.323,"15min":3.1440000000000001},"last":1.728}]}]},{"osd":6,"up_from":13,"seq":55834576024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52636,"kb_used_data":3404,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319204,"statfs":{"total":96636764160,"available":96582864896,"internally_reserved":0,"allocated":3485696,"data_stored":1706169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1779999999999999,"15min":1.202},"min":{"1min":0.69399999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.538,"5min":6.298,"15min":6.298},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.159,"5min":1.2989999999999999,"15min":1.27},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.597,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":0.71899999999999997}]},{"osd":1,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.198,"15min":1.204},"min":{"1min":0.72299999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7969999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3740000000000001,"15min":1.361},"min":{"1min":0.749,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.714,"5min":12.468,"15min":12.468},"last":0.96899999999999997}]},{"osd":2,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.2549999999999999,"15min":1.254},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.639,"5min":11.586,"15min":11.586},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.337,"5min":1.385,"15min":1.371},"min":{"1min":0.87,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7170000000000001,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.1499999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.4139999999999999,"15min":1.337},"min":{"1min":0.68100000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.766,"5min":12.801,"15min":12.801},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.135,"5min":1.254,"15min":1.2589999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.7969999999999999,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":0.91700000000000004}]},{"osd":4,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4370000000000001,"15min":1.341},"min":{"1min":1.0429999999999999,"5min":0.65400000000000003,"15min":0.49099999999999999},"max":{"1min":1.931,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.6519999999999999,"15min":1.5569999999999999},"min":{"1min":0.70399999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.97,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.109}]},{"osd":5,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.55,"15min":1.3979999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.9319999999999999,"5min":18.247,"15min":18.247},"last":1.054},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.6279999999999999,"15min":1.569},"min":{"1min":0.76700000000000002,"5min":0.76200000000000001,"15min":0.60899999999999999},"max":{"1min":2.085,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.23}]},{"osd":7,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.6990000000000001,"15min":1.5580000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.59199999999999997},"max":{"1min":1.9870000000000001,"5min":12.589,"15min":12.589},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.6990000000000001,"15min":1.597},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0249999999999999,"5min":12.725,"15min":12.725},"last":1.3080000000000001}]}]},{"osd":5,"up_from":13,"seq":55834576024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52636,"kb_used_data":3404,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319204,"statfs":{"total":96636764160,"available":96582864896,"internally_reserved":0,"allocated":3485696,"data_stored":1706169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.1519999999999999,"15min":1.234},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.585,"5min":1.585,"15min":2.661},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.248,"15min":1.3169999999999999},"min":{"1min":0.75600000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.583,"5min":1.7909999999999999,"15min":1.7949999999999999},"last":1.145}]},{"osd":1,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1279999999999999,"15min":1.175},"min":{"1min":0.82199999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.748,"5min":1.748,"15min":1.992},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.1919999999999999,"15min":1.228},"min":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":2.9399999999999999},"last":0.86499999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2470000000000001,"15min":1.2390000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.282},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2869999999999999,"15min":1.3009999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7829999999999999,"5min":1.8660000000000001,"15min":2.1640000000000001},"last":0.79400000000000004}]},{"osd":3,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.29,"15min":1.3169999999999999},"min":{"1min":0.82199999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.121,"5min":2.121,"15min":3.012},"last":1.599},{"interface":"front","average":{"1min":1.429,"5min":1.387,"15min":1.411},"min":{"1min":1.0880000000000001,"5min":0.92700000000000005,"15min":0.89500000000000002},"max":{"1min":1.9299999999999999,"5min":2.0129999999999999,"15min":2.2250000000000001},"last":0.84099999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.3280000000000001,"15min":1.345},"min":{"1min":0.80300000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.8660000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.605,"5min":1.5429999999999999,"15min":1.5620000000000001},"min":{"1min":1.125,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.903},"last":1.3700000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.4059999999999999,"15min":1.417},"min":{"1min":0.95099999999999996,"5min":0.68100000000000005,"15min":0.51000000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0979999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.53,"5min":1.4379999999999999,"15min":1.4770000000000001},"min":{"1min":1.004,"5min":0.90700000000000003,"15min":0.89200000000000002},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":3.073},"last":0.97899999999999998}]},{"osd":7,"last update":"Mon Oct 16 11:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.552,"15min":1.548},"min":{"1min":1.228,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":3.1160000000000001},"last":1.504},{"interface":"front","average":{"1min":1.702,"5min":1.619,"15min":1.639},"min":{"1min":1.2809999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.161,"5min":2.1930000000000001,"15min":2.2930000000000001},"last":0.74199999999999999}]}]},{"osd":4,"up_from":13,"seq":55834576025,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":88664,"kb_used_data":3404,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94283176,"statfs":{"total":96636764160,"available":96545972224,"internally_reserved":0,"allocated":3485696,"data_stored":1706169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":87292358},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1559999999999999,"15min":1.167},"min":{"1min":0.628,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.9970000000000001,"5min":2.899,"15min":2.899},"last":1.26},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.2290000000000001,"15min":1.2430000000000001},"min":{"1min":0.66200000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.411,"5min":3.016,"15min":3.016},"last":0.97899999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:27:55 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.28,"15min":1.2929999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.70599999999999996},"max":{"1min":2.5129999999999999,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.2190000000000001,"15min":1.2350000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.7190000000000001,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.0389999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:27:55 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.298,"15min":1.306},"min":{"1min":0.77100000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7629999999999999,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.627},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.431,"15min":1.4159999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.395,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.5880000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:27:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.321,"15min":1.335},"min":{"1min":0.80300000000000005,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.4039999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.196},{"interface":"front","average":{"1min":1.429,"5min":1.4990000000000001,"15min":1.458},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.837,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.8149999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:27:55 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.542,"15min":1.4810000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.6440000000000001,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.4650000000000001,"15min":1.4370000000000001},"min":{"1min":0.61699999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.95,"5min":3.7989999999999999,"15min":3.7989999999999999},"last":1.528}]},{"osd":6,"last update":"Mon Oct 16 11:27:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.609,"15min":1.5349999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.016,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.665,"5min":1.6830000000000001,"15min":1.6479999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.8319999999999999,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.736}]},{"osd":7,"last update":"Mon Oct 16 11:27:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.6519999999999999,"15min":1.6020000000000001},"min":{"1min":0.78400000000000003,"5min":0.73699999999999999,"15min":0.72299999999999998},"max":{"1min":2.351,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.647,"5min":1.7230000000000001,"15min":1.677},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.2599999999999998,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.889}]}]},{"osd":3,"up_from":13,"seq":55834576024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52636,"kb_used_data":3404,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319204,"statfs":{"total":96636764160,"available":96582864896,"internally_reserved":0,"allocated":3485696,"data_stored":1706169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81699999999999995,"15min":0.88200000000000001},"min":{"1min":0.33800000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.901},"last":0.752},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.89200000000000002,"15min":1.0249999999999999},"min":{"1min":0.443,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1919999999999999,"5min":1.581,"15min":1.6679999999999999},"last":0.67600000000000005}]},{"osd":1,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.88800000000000001,"15min":0.95899999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3420000000000001,"5min":1.508,"15min":1.508},"last":1.2509999999999999},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.94899999999999995,"15min":1.087},"min":{"1min":0.51300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4770000000000001,"5min":1.601,"15min":1.7190000000000001},"last":1.113}]},{"osd":2,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.99199999999999999,"15min":1.087},"min":{"1min":0.56299999999999994,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.524,"5min":1.5660000000000001,"15min":1.694},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.079,"15min":1.1870000000000001},"min":{"1min":0.48499999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.597,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.0649999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.23,"15min":1.3240000000000001},"min":{"1min":0.72499999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6839999999999999,"5min":1.821,"15min":1.873},"last":1.482},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2949999999999999,"15min":1.3440000000000001},"min":{"1min":0.80900000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.681,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.149}]},{"osd":5,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.3009999999999999,"15min":1.385},"min":{"1min":0.69299999999999995,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.7749999999999999,"5min":2.798,"15min":2.798},"last":1.349},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3149999999999999,"15min":1.4179999999999999},"min":{"1min":0.74199999999999999,"5min":0.63,"15min":0.63},"max":{"1min":1.772,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.579}]},{"osd":6,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4330000000000001,"15min":1.51},"min":{"1min":0.93000000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.9970000000000001,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.23},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.413,"15min":1.506},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9350000000000001,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.6579999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.46,"15min":1.544},"min":{"1min":0.96799999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.8380000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.5329999999999999,"15min":1.6459999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.952,"5min":2.3799999999999999,"15min":2.4649999999999999},"last":1.548}]}]},{"osd":2,"up_from":13,"seq":55834576023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56796,"kb_used_data":3404,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315044,"statfs":{"total":96636764160,"available":96578605056,"internally_reserved":0,"allocated":3485696,"data_stored":1706169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.85799999999999998,"15min":0.89900000000000002},"min":{"1min":0.39800000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.3200000000000001,"5min":1.532,"15min":1.6399999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.92100000000000004,"15min":1.0369999999999999},"min":{"1min":0.53400000000000003,"5min":0.40500000000000003,"15min":0.32600000000000001},"max":{"1min":1.258,"5min":1.6359999999999999,"15min":2.1429999999999998},"last":0.92400000000000004}]},{"osd":1,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.046,"15min":1.075},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39200000000000002},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":1.179},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.02,"15min":1.1120000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.77400000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.96099999999999997,"15min":1.0469999999999999},"min":{"1min":0.52300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.452,"5min":1.522,"15min":1.8560000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1639999999999999,"15min":1.242},"min":{"1min":0.58199999999999996,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.0459999999999998,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":1.3320000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.306,"15min":1.3819999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.8079999999999998,"5min":3.8079999999999998,"15min":3.8079999999999998},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.2490000000000001,"15min":1.3140000000000001},"min":{"1min":0.79700000000000004,"5min":0.77000000000000002,"15min":0.748},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.288}]},{"osd":5,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.397,"15min":1.431},"min":{"1min":0.77900000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.53},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3839999999999999,"15min":1.4470000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0499999999999998,"5min":2.323,"15min":2.6259999999999999},"last":1.4119999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3560000000000001,"15min":1.381},"min":{"1min":0.85099999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.4830000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":1.486},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.464,"15min":1.5089999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":1.9159999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.4910000000000001,"15min":1.5489999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.742,"5min":3.742,"15min":3.742},"last":1.462},{"interface":"front","average":{"1min":1.431,"5min":1.4339999999999999,"15min":1.4810000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":1.6439999999999999}]}]},{"osd":1,"up_from":13,"seq":55834576023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48472,"kb_used_data":3404,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323368,"statfs":{"total":96636764160,"available":96587128832,"internally_reserved":0,"allocated":3485696,"data_stored":1706169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.89300000000000002,"15min":0.91000000000000003},"min":{"1min":0.40699999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4039999999999999,"5min":5.117,"15min":5.117},"last":1.544},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.89300000000000002,"15min":1.0029999999999999},"min":{"1min":0.45500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.325,"5min":1.5509999999999999,"15min":1.667},"last":0.82699999999999996}]},{"osd":2,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.91300000000000003,"15min":0.98899999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.651,"5min":2.093,"15min":2.093},"last":1.157},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.149,"15min":1.242},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2400000000000002,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":1.5960000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.125,"15min":1.179},"min":{"1min":0.56999999999999995,"5min":0.439,"15min":0.439},"max":{"1min":1.8879999999999999,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.647},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.05,"15min":1.1399999999999999},"min":{"1min":0.52700000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5860000000000001,"5min":1.798,"15min":1.802},"last":1.0600000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.2749999999999999,"15min":1.2849999999999999},"min":{"1min":0.70299999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.978,"5min":5.29,"15min":5.29},"last":1.292},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.403,"15min":1.409},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.0129999999999999,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":1.321}]},{"osd":5,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.3400000000000001,"15min":1.3979999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.829,"5min":6.0979999999999999,"15min":6.0979999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.3420000000000001,"15min":1.3899999999999999},"min":{"1min":0.74099999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7410000000000001,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.4199999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.429,"15min":1.482},"min":{"1min":0.82399999999999995,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8979999999999999,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.393,"15min":1.4630000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.948,"5min":3.8799999999999999,"15min":3.8799999999999999},"last":1.51}]},{"osd":7,"last update":"Mon Oct 16 11:26:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.518,"15min":1.5509999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.157,"5min":4.46,"15min":4.46},"last":1.478},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.5349999999999999,"15min":1.589},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.0259999999999998,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":1.377}]}]},{"osd":0,"up_from":13,"seq":55834576024,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49056,"kb_used_data":3984,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322784,"statfs":{"total":96636764160,"available":96586530816,"internally_reserved":0,"allocated":4079616,"data_stored":2296537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.85999999999999999,"15min":0.879},"min":{"1min":0.46000000000000002,"5min":0.42699999999999999,"15min":0.34000000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.95199999999999996,"15min":1.0509999999999999},"min":{"1min":0.44,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":1.3300000000000001,"5min":1.621,"15min":1.76},"last":0.40500000000000003}]},{"osd":2,"last update":"Mon Oct 16 11:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":1.0469999999999999,"15min":1.069},"min":{"1min":0.47999999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4490000000000001,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.91400000000000003,"5min":1.026,"15min":1.099},"min":{"1min":0.48899999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5569999999999999,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.55700000000000005}]},{"osd":3,"last update":"Mon Oct 16 11:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.99299999999999999,"15min":1.054},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.44900000000000001},"max":{"1min":1.4430000000000001,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1299999999999999,"15min":1.21},"min":{"1min":0.59099999999999997,"5min":0.55600000000000005,"15min":0.48999999999999999},"max":{"1min":1.7869999999999999,"5min":1.833,"15min":1.833},"last":0.47899999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.2989999999999999,"15min":1.365},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.6659999999999999,"5min":3.2799999999999998,"15min":3.2799999999999998},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.288,"15min":1.325},"min":{"1min":0.83699999999999997,"5min":0.76100000000000001,"15min":0.71499999999999997},"max":{"1min":1.8580000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.98799999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.292,"15min":1.3560000000000001},"min":{"1min":0.61599999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0030000000000001,"5min":2.0209999999999999,"15min":2.6669999999999998},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.3,"5min":1.3799999999999999,"15min":1.4239999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.70499999999999996},"max":{"1min":1.8420000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":0.72399999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.4430000000000001,"15min":1.4790000000000001},"min":{"1min":0.84099999999999997,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.831,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.3959999999999999,"15min":1.4610000000000001},"min":{"1min":0.90500000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.9019999999999999,"5min":2.073,"15min":2.262},"last":1.1899999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.5309999999999999,"15min":1.5429999999999999},"min":{"1min":1.0409999999999999,"5min":0.88700000000000001,"15min":0.72099999999999997},"max":{"1min":2.0009999999999999,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.4059999999999999,"15min":1.4770000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.60299999999999998},"max":{"1min":1.925,"5min":1.9550000000000001,"15min":2.8199999999999998},"last":0.96499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:28:02.681 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-16T09:48:56.337130+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=9, tm_min=48, tm_sec=56, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=11, tm_min=27, tm_sec=55, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T11:28:02.681 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-16T11:28:05.216 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:28:05.226 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:28:22.682 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:28:22.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.906+0000 7fea26e47640 1 Processor -- start 2023-10-16T11:28:22.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.906+0000 7fea26e47640 1 -- start start 2023-10-16T11:28:22.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.907+0000 7fea26e47640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fea20070b60 0x7fea2015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:22.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.907+0000 7fea26e47640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea2006c100 0x7fea2006c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:22.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.907+0000 7fea26e47640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2015aef0 con 0x7fea20159950 2023-10-16T11:28:22.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.908+0000 7fea26e47640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea2015e490 con 0x7fea20070b60 2023-10-16T11:28:22.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.908+0000 7fea26e47640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2015d910 con 0x7fea2006c100 2023-10-16T11:28:22.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.908+0000 7fea25644640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fea20070b60 0x7fea2015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:22.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.908+0000 7fea25e45640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fea20159950 0x7fea20070450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34400/0 (socket says 172.21.15.84:34400) 2023-10-16T11:28:22.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.908+0000 7fea25644640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fea20070b60 0x7fea2015a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37850/0 (socket says 172.21.15.84:37850) 2023-10-16T11:28:22.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.908+0000 7fea25e45640 1 -- 172.21.15.84:0/202997448 learned_addr learned my addr 172.21.15.84:0/202997448 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:28:22.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.908+0000 7fea25644640 1 -- 172.21.15.84:0/202997448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea2006c100 msgr2=0x7fea2006c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:22.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.908+0000 7fea25644640 1 --2- 172.21.15.84:0/202997448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea2006c100 0x7fea2006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:22.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.908+0000 7fea25644640 1 -- 172.21.15.84:0/202997448 >> v1:172.21.15.84:6789/0 conn(0x7fea20159950 legacy=0x7fea20070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:22.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.909+0000 7fea25644640 1 -- 172.21.15.84:0/202997448 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2015dbf0 con 0x7fea20070b60 2023-10-16T11:28:22.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.908+0000 7fea24e43640 1 -- 172.21.15.84:0/202997448 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226114346 0 0) 0x7fea1c001110 con 0x7fea20159950 2023-10-16T11:28:22.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.910+0000 7fea25644640 1 --2- 172.21.15.84:0/202997448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea20070b60 0x7fea2015a9b0 secure :-1 s=READY pgs=3589 cs=0 l=1 rev1=1 crypto rx=0x7fea100097d0 tx=0x7fea1002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5066cfd65a143093 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:22.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.910+0000 7fea24e43640 1 -- 172.21.15.84:0/202997448 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea10031780 con 0x7fea20070b60 2023-10-16T11:28:22.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.910+0000 7fea24e43640 1 -- 172.21.15.84:0/202997448 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea100318e0 con 0x7fea20070b60 2023-10-16T11:28:22.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.910+0000 7fea24e43640 1 -- 172.21.15.84:0/202997448 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea10031c70 con 0x7fea20070b60 2023-10-16T11:28:22.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.910+0000 7fea26e47640 1 -- 172.21.15.84:0/202997448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea20070b60 msgr2=0x7fea2015a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:22.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.910+0000 7fea26e47640 1 --2- 172.21.15.84:0/202997448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea20070b60 0x7fea2015a9b0 secure :-1 s=READY pgs=3589 cs=0 l=1 rev1=1 crypto rx=0x7fea100097d0 tx=0x7fea1002b820 comp rx=0 tx=0).stop 2023-10-16T11:28:22.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.911+0000 7fea26e47640 1 -- 172.21.15.84:0/202997448 shutdown_connections 2023-10-16T11:28:22.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.911+0000 7fea26e47640 1 --2- 172.21.15.84:0/202997448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea2006c100 0x7fea2006c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:22.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.911+0000 7fea26e47640 1 --2- 172.21.15.84:0/202997448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea20070b60 0x7fea2015a9b0 unknown :-1 s=CLOSED pgs=3589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:22.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.911+0000 7fea26e47640 1 -- 172.21.15.84:0/202997448 >> 172.21.15.84:0/202997448 conn(0x7fea200ce6a0 msgr2=0x7fea200ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:22.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.911+0000 7fea26e47640 1 -- 172.21.15.84:0/202997448 shutdown_connections 2023-10-16T11:28:22.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.911+0000 7fea26e47640 1 -- 172.21.15.84:0/202997448 wait complete. 2023-10-16T11:28:22.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.912+0000 7fea26e47640 1 Processor -- start 2023-10-16T11:28:22.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.912+0000 7fea26e47640 1 -- start start 2023-10-16T11:28:22.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.913+0000 7fea26e47640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2006c100 0x7fea2014ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:22.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.913+0000 7fea26e47640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea20070b60 0x7fea2014b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:22.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.913+0000 7fea26e47640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea20159950 0x7fea200ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:22.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.913+0000 7fea26e47640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2014b940 con 0x7fea20159950 2023-10-16T11:28:22.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.913+0000 7fea26e47640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea200c45f0 con 0x7fea2006c100 2023-10-16T11:28:22.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.913+0000 7fea26e47640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea200c48d0 con 0x7fea20070b60 2023-10-16T11:28:22.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.914+0000 7fea26646640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea20159950 0x7fea200ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:22.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.914+0000 7fea26646640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea20159950 0x7fea200ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37858/0 (socket says 172.21.15.84:37858) 2023-10-16T11:28:22.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.914+0000 7fea26646640 1 -- 172.21.15.84:0/2792505289 learned_addr learned my addr 172.21.15.84:0/2792505289 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:28:22.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.914+0000 7fea25644640 1 --2- 172.21.15.84:0/2792505289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea20070b60 0x7fea2014b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:22.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.914+0000 7fea25e45640 1 --2- 172.21.15.84:0/2792505289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2006c100 0x7fea2014ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:22.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.914+0000 7fea26646640 1 -- 172.21.15.84:0/2792505289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea20070b60 msgr2=0x7fea2014b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:22.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.914+0000 7fea26646640 1 --2- 172.21.15.84:0/2792505289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea20070b60 0x7fea2014b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.914+0000 7fea26646640 1 -- 172.21.15.84:0/2792505289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2006c100 msgr2=0x7fea2014ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.914+0000 7fea26646640 1 --2- 172.21.15.84:0/2792505289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2006c100 0x7fea2014ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.914+0000 7fea26646640 1 -- 172.21.15.84:0/2792505289 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea10009400 con 0x7fea20159950 2023-10-16T11:28:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.915+0000 7fea26646640 1 --2- 172.21.15.84:0/2792505289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea20159950 0x7fea200ac890 secure :-1 s=READY pgs=3590 cs=0 l=1 rev1=1 crypto rx=0x7fea1800b970 tx=0x7fea1800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:22.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.915+0000 7fea167fc640 1 -- 172.21.15.84:0/2792505289 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1800cb40 con 0x7fea20159950 2023-10-16T11:28:22.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.915+0000 7fea167fc640 1 -- 172.21.15.84:0/2792505289 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1800ccc0 con 0x7fea20159950 2023-10-16T11:28:22.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.915+0000 7fea167fc640 1 -- 172.21.15.84:0/2792505289 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18014680 con 0x7fea20159950 2023-10-16T11:28:22.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.915+0000 7fea26e47640 1 -- 172.21.15.84:0/2792505289 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea200c4bb0 con 0x7fea20159950 2023-10-16T11:28:22.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.915+0000 7fea26e47640 1 -- 172.21.15.84:0/2792505289 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea200c5170 con 0x7fea20159950 2023-10-16T11:28:22.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.916+0000 7fea26e47640 1 -- 172.21.15.84:0/2792505289 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2004e9b0 con 0x7fea20159950 2023-10-16T11:28:22.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.917+0000 7fea167fc640 1 -- 172.21.15.84:0/2792505289 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fea1801d3b0 con 0x7fea20159950 2023-10-16T11:28:22.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.918+0000 7fea167fc640 1 --2- 172.21.15.84:0/2792505289 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fea0406af90 0x7fea0406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:22.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.918+0000 7fea167fc640 1 -- 172.21.15.84:0/2792505289 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fea18089780 con 0x7fea20159950 2023-10-16T11:28:22.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.918+0000 7fea25e45640 1 --2- 172.21.15.84:0/2792505289 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fea0406af90 0x7fea0406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:22.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.919+0000 7fea25e45640 1 --2- 172.21.15.84:0/2792505289 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fea0406af90 0x7fea0406d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fea1c000f60 tx=0x7fea1c007c00 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:22.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:22.923+0000 7fea167fc640 1 -- 172.21.15.84:0/2792505289 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fea180546b0 con 0x7fea20159950 2023-10-16T11:28:23.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.112+0000 7fea26e47640 1 -- 172.21.15.84:0/2792505289 --> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fea2015e1b0 con 0x7fea0406af90 2023-10-16T11:28:23.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.118+0000 7fea167fc640 1 -- 172.21.15.84:0/2792505289 <== mgr.13719 v2:172.21.15.84:6832/3003264401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48319 (secure 0 0 0) 0x7fea1c007270 con 0x7fea0406af90 2023-10-16T11:28:23.121 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:28:23.121 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-16T11:28:23.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.121+0000 7fea26e47640 1 -- 172.21.15.84:0/2792505289 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fea0406af90 msgr2=0x7fea0406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:23.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.121+0000 7fea26e47640 1 --2- 172.21.15.84:0/2792505289 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fea0406af90 0x7fea0406d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fea1c000f60 tx=0x7fea1c007c00 comp rx=0 tx=0).stop 2023-10-16T11:28:23.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.121+0000 7fea26e47640 1 -- 172.21.15.84:0/2792505289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea20159950 msgr2=0x7fea200ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:23.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.121+0000 7fea26e47640 1 --2- 172.21.15.84:0/2792505289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea20159950 0x7fea200ac890 secure :-1 s=READY pgs=3590 cs=0 l=1 rev1=1 crypto rx=0x7fea1800b970 tx=0x7fea1800be30 comp rx=0 tx=0).stop 2023-10-16T11:28:23.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.121+0000 7fea26e47640 1 -- 172.21.15.84:0/2792505289 shutdown_connections 2023-10-16T11:28:23.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.121+0000 7fea26e47640 1 --2- 172.21.15.84:0/2792505289 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7fea0406af90 0x7fea0406d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.121+0000 7fea26e47640 1 --2- 172.21.15.84:0/2792505289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea20159950 0x7fea200ac890 unknown :-1 s=CLOSED pgs=3590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.121+0000 7fea26e47640 1 --2- 172.21.15.84:0/2792505289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea20070b60 0x7fea2014b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.121+0000 7fea26e47640 1 --2- 172.21.15.84:0/2792505289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2006c100 0x7fea2014ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.121+0000 7fea26e47640 1 -- 172.21.15.84:0/2792505289 >> 172.21.15.84:0/2792505289 conn(0x7fea200ce6a0 msgr2=0x7fea20070450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:23.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.121+0000 7fea26e47640 1 -- 172.21.15.84:0/2792505289 shutdown_connections 2023-10-16T11:28:23.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.121+0000 7fea26e47640 1 -- 172.21.15.84:0/2792505289 wait complete. 2023-10-16T11:28:23.143 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-16T11:28:21.499576+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2783,"num_read_kb":5735,"num_write":703,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4837,"ondisk_log_size":4837,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":480464,"kb_used_data":28392,"kb_used_omap":12,"kb_used_meta":451955,"kb_avail":754494256,"statfs":{"total":773094113280,"available":772602118144,"internally_reserved":0,"allocated":29073408,"data_stored":14830088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":462802489},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003432"},"pg_stats":[{"pgid":"1.0","version":"275'4837","reported_seq":5972,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-16T11:28:03.347313+0000","last_change":"2023-10-16T11:28:03.347446+0000","last_active":"2023-10-16T11:28:03.347313+0000","last_peered":"2023-10-16T11:28:03.347313+0000","last_clean":"2023-10-16T11:28:03.347313+0000","last_became_active":"2023-10-16T09:48:57.575783+0000","last_became_peered":"2023-10-16T09:48:57.575783+0000","last_unstale":"2023-10-16T11:28:03.347313+0000","last_undegraded":"2023-10-16T11:28:03.347313+0000","last_fullsized":"2023-10-16T11:28:03.347313+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'4837","last_scrub_stamp":"2023-10-16T11:28:03.347101+0000","last_deep_scrub":"275'4837","last_deep_scrub_stamp":"2023-10-16T11:28:03.347101+0000","last_clean_scrub_stamp":"2023-10-16T11:28:03.347101+0000","objects_scrubbed":2,"log_size":4837,"log_dups_size":0,"ondisk_log_size":4837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:05:37.084792+0000","scrub_duration":0.014424302999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2783,"num_read_kb":5735,"num_write":703,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2783,"num_read_kb":5735,"num_write":703,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4837,"ondisk_log_size":4837,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576028,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79576,"kb_used_data":3984,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94292264,"statfs":{"total":96636764160,"available":96555278336,"internally_reserved":0,"allocated":4079616,"data_stored":2296537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":77396427},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.115,"15min":1.1719999999999999},"min":{"1min":0.65300000000000002,"5min":0.64500000000000002,"15min":0.44400000000000001},"max":{"1min":1.655,"5min":1.7250000000000001,"15min":2.3399999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.03,"5min":1.1220000000000001,"15min":1.1779999999999999},"min":{"1min":0.63100000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":1.619}]},{"osd":1,"last update":"Mon Oct 16 11:27:58 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.1699999999999999,"15min":1.226},"min":{"1min":0.755,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.655,"5min":1.96,"15min":1.96},"last":1.52},{"interface":"front","average":{"1min":1.147,"5min":1.161,"15min":1.2370000000000001},"min":{"1min":0.72799999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.5089999999999999,"5min":1.6799999999999999,"15min":1.8600000000000001},"last":1.3959999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:27:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.2330000000000001,"15min":1.276},"min":{"1min":0.69199999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6279999999999999,"5min":1.9039999999999999,"15min":2.4460000000000002},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.278,"15min":1.331},"min":{"1min":0.85499999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.5429999999999999,"5min":3.9249999999999998,"15min":3.9249999999999998},"last":1.2809999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:27:58 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.339,"15min":1.365},"min":{"1min":0.97999999999999998,"5min":0.85999999999999999,"15min":0.67100000000000004},"max":{"1min":1.681,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":1.077},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3640000000000001,"15min":1.431},"min":{"1min":1.004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.7569999999999999,"5min":2.016,"15min":2.577},"last":1.4259999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:27:58 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.4139999999999999,"15min":1.4079999999999999},"min":{"1min":1.0680000000000001,"5min":0.64800000000000002,"15min":0.48999999999999999},"max":{"1min":1.9830000000000001,"5min":2.113,"15min":2.6779999999999999},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.397,"15min":1.387},"min":{"1min":0.97699999999999998,"5min":0.65600000000000003,"15min":0.51200000000000001},"max":{"1min":2.335,"5min":2.335,"15min":2.335},"last":1.2330000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:27:58 2023","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.5600000000000001,"15min":1.583},"min":{"1min":0.91800000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.4159999999999999,"5min":2.4329999999999998,"15min":3.3519999999999999},"last":1.339},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.53,"15min":1.5049999999999999},"min":{"1min":1.133,"5min":0.62,"15min":0.60499999999999998},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.3460000000000001},"last":1.107}]},{"osd":6,"last update":"Mon Oct 16 11:27:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.488,"15min":1.5009999999999999},"min":{"1min":1.1479999999999999,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.0739999999999998,"5min":2.2469999999999999,"15min":2.7250000000000001},"last":1.46},{"interface":"front","average":{"1min":1.694,"5min":1.657,"15min":1.6899999999999999},"min":{"1min":1.1599999999999999,"5min":0.749,"15min":0.65000000000000002},"max":{"1min":2.2599999999999998,"5min":2.323,"15min":3.1440000000000001},"last":1.5589999999999999}]}]},{"osd":6,"up_from":13,"seq":55834576028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52636,"kb_used_data":3404,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319204,"statfs":{"total":96636764160,"available":96582864896,"internally_reserved":0,"allocated":3485696,"data_stored":1706169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1779999999999999,"15min":1.202},"min":{"1min":0.69399999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.538,"5min":6.298,"15min":6.298},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.2989999999999999,"15min":1.27},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.597,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":1.4299999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.198,"15min":1.204},"min":{"1min":0.72299999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7969999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":1.236},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3740000000000001,"15min":1.361},"min":{"1min":0.749,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.714,"5min":12.468,"15min":12.468},"last":1.319}]},{"osd":2,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.2549999999999999,"15min":1.254},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.639,"5min":11.586,"15min":11.586},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.337,"5min":1.385,"15min":1.371},"min":{"1min":0.87,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7170000000000001,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":1.476}]},{"osd":3,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.4139999999999999,"15min":1.337},"min":{"1min":0.68100000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.766,"5min":12.801,"15min":12.801},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.135,"5min":1.254,"15min":1.2589999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.7969999999999999,"5min":9.1539999999999999,"15min":9.1539999999999999},"last":1.2749999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4370000000000001,"15min":1.341},"min":{"1min":1.0429999999999999,"5min":0.65400000000000003,"15min":0.49099999999999999},"max":{"1min":1.931,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":1.577},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.6519999999999999,"15min":1.5569999999999999},"min":{"1min":0.70399999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.97,"5min":13.981999999999999,"15min":13.981999999999999},"last":0.83499999999999996}]},{"osd":5,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.55,"15min":1.3979999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.9319999999999999,"5min":18.247,"15min":18.247},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.6279999999999999,"15min":1.569},"min":{"1min":0.76700000000000002,"5min":0.76200000000000001,"15min":0.60899999999999999},"max":{"1min":2.085,"5min":6.4039999999999999,"15min":6.4039999999999999},"last":1.653}]},{"osd":7,"last update":"Mon Oct 16 11:27:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.6990000000000001,"15min":1.5580000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.59199999999999997},"max":{"1min":1.9870000000000001,"5min":12.589,"15min":12.589},"last":1.103},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.6990000000000001,"15min":1.597},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0249999999999999,"5min":12.725,"15min":12.725},"last":1.5229999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52636,"kb_used_data":3404,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319204,"statfs":{"total":96636764160,"available":96582864896,"internally_reserved":0,"allocated":3485696,"data_stored":1706169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.1160000000000001,"15min":1.2190000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":2.661},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.2150000000000001,"15min":1.2989999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.6839999999999999,"5min":1.7909999999999999,"15min":1.7949999999999999},"last":1.3899999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.117,"15min":1.1599999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.585,"5min":1.748,"15min":1.992},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.165,"15min":1.2250000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.613,"5min":1.8049999999999999,"15min":2.9399999999999999},"last":1.083}]},{"osd":2,"last update":"Mon Oct 16 11:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.226,"15min":1.232},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.915,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.036},{"interface":"front","average":{"1min":1.204,"5min":1.2729999999999999,"15min":1.2949999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.544,"5min":1.8660000000000001,"15min":2.1640000000000001},"last":1.2649999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3129999999999999,"15min":1.3109999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.57499999999999996},"max":{"1min":1.8380000000000001,"5min":2.121,"15min":3.012},"last":1.546},{"interface":"front","average":{"1min":1.282,"5min":1.375,"15min":1.3999999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7170000000000001,"5min":2.0129999999999999,"15min":2.2250000000000001},"last":1.3600000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.341,"15min":1.337},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.50700000000000001},"max":{"1min":1.8540000000000001,"5min":2.0459999999999998,"15min":2.8660000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.585,"5min":1.5549999999999999,"15min":1.5569999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.351,"5min":2.3540000000000001,"15min":2.903},"last":1.7509999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.431,"15min":1.4239999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.51000000000000001},"max":{"1min":1.9059999999999999,"5min":2.0859999999999999,"15min":2.0979999999999999},"last":1.6899999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.466,"15min":1.474},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":3.073},"last":1.47}]},{"osd":7,"last update":"Mon Oct 16 11:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.734,"5min":1.6100000000000001,"15min":1.5600000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.25,"5min":2.2749999999999999,"15min":3.1160000000000001},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.643,"15min":1.6379999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2930000000000001},"last":1.591}]}]},{"osd":4,"up_from":13,"seq":55834576029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":88664,"kb_used_data":3404,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94283176,"statfs":{"total":96636764160,"available":96545972224,"internally_reserved":0,"allocated":3485696,"data_stored":1706169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":87292358},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1559999999999999,"15min":1.167},"min":{"1min":0.628,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.9970000000000001,"5min":2.899,"15min":2.899},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.2290000000000001,"15min":1.2430000000000001},"min":{"1min":0.66200000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.411,"5min":3.016,"15min":3.016},"last":1.0409999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:27:55 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.28,"15min":1.2929999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.70599999999999996},"max":{"1min":2.5129999999999999,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.603},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.2190000000000001,"15min":1.2350000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.7190000000000001,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.4139999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:27:55 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.298,"15min":1.306},"min":{"1min":0.77100000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7629999999999999,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.181},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.431,"15min":1.4159999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.395,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.4410000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:27:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.321,"15min":1.335},"min":{"1min":0.80300000000000005,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.4039999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.429,"5min":1.4990000000000001,"15min":1.458},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.837,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.3600000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:27:55 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.542,"15min":1.4810000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.6440000000000001,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.296},{"interface":"front","average":{"1min":1.395,"5min":1.4650000000000001,"15min":1.4370000000000001},"min":{"1min":0.61699999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.95,"5min":3.7989999999999999,"15min":3.7989999999999999},"last":1.242}]},{"osd":6,"last update":"Mon Oct 16 11:27:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.609,"15min":1.5349999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.016,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.54},{"interface":"front","average":{"1min":1.665,"5min":1.6830000000000001,"15min":1.6479999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.8319999999999999,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":1.7230000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:27:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.6519999999999999,"15min":1.6020000000000001},"min":{"1min":0.78400000000000003,"5min":0.73699999999999999,"15min":0.72299999999999998},"max":{"1min":2.351,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":1.581},{"interface":"front","average":{"1min":1.647,"5min":1.7230000000000001,"15min":1.677},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.2599999999999998,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.762}]}]},{"osd":3,"up_from":13,"seq":55834576028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52636,"kb_used_data":3404,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319204,"statfs":{"total":96636764160,"available":96582864896,"internally_reserved":0,"allocated":3485696,"data_stored":1706169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.81299999999999994,"15min":0.87},"min":{"1min":0.35999999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.3160000000000001,"5min":1.5880000000000001,"15min":1.901},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.874,"5min":0.88800000000000001,"15min":1.008},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.3859999999999999,"5min":1.581,"15min":1.6220000000000001},"last":1.1060000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.872,"15min":0.94599999999999995},"min":{"1min":0.41499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.478,"5min":1.508,"15min":1.508},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.95999999999999996,"15min":1.0760000000000001},"min":{"1min":0.42399999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.5600000000000001,"5min":1.601,"15min":1.7190000000000001},"last":1.476}]},{"osd":2,"last update":"Mon Oct 16 11:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.98499999999999999,"15min":1.0660000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.3859999999999999,"5min":1.5660000000000001,"15min":1.694},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.081,"15min":1.177},"min":{"1min":0.503,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5700000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.429}]},{"osd":4,"last update":"Mon Oct 16 11:28:19 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.2310000000000001,"15min":1.3109999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.7549999999999999,"5min":1.821,"15min":1.873},"last":1.165},{"interface":"front","average":{"1min":1.173,"5min":1.2729999999999999,"15min":1.333},"min":{"1min":0.67800000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.746,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.3169999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:28:19 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3160000000000001,"15min":1.373},"min":{"1min":0.69499999999999995,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.875,"5min":2.798,"15min":2.798},"last":1.383},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.3080000000000001,"15min":1.401},"min":{"1min":0.81100000000000005,"5min":0.63,"15min":0.63},"max":{"1min":1.611,"5min":2.1280000000000001,"15min":2.1579999999999999},"last":1.518}]},{"osd":6,"last update":"Mon Oct 16 11:28:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.4359999999999999,"15min":1.494},"min":{"1min":0.76900000000000002,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.988,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.413,"15min":1.492},"min":{"1min":0.75800000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9299999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":1.645}]},{"osd":7,"last update":"Mon Oct 16 11:28:19 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.4450000000000001,"15min":1.526},"min":{"1min":0.879,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.2719999999999998,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.5169999999999999,"15min":1.627},"min":{"1min":0.92200000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.3399999999999999,"5min":2.3799999999999999,"15min":2.4649999999999999},"last":1.698}]}]},{"osd":2,"up_from":13,"seq":55834576027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56796,"kb_used_data":3404,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315044,"statfs":{"total":96636764160,"available":96578605056,"internally_reserved":0,"allocated":3485696,"data_stored":1706169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.86199999999999999,"15min":0.90100000000000002},"min":{"1min":0.375,"5min":0.314,"15min":0.314},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.90600000000000003,"15min":1.022},"min":{"1min":0.439,"5min":0.40500000000000003,"15min":0.32600000000000001},"max":{"1min":1.3129999999999999,"5min":1.5449999999999999,"15min":2.1429999999999998},"last":0.97699999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:28:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0489999999999999,"15min":1.0720000000000001},"min":{"1min":0.45800000000000002,"5min":0.40000000000000002,"15min":0.39200000000000002},"max":{"1min":1.3959999999999999,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.02,"15min":1.1000000000000001},"min":{"1min":0.53400000000000003,"5min":0.435,"15min":0.435},"max":{"1min":1.6040000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.0800000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:28:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.95899999999999996,"15min":1.034},"min":{"1min":0.439,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.417,"5min":1.522,"15min":1.8560000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.139,"5min":1.163,"15min":1.23},"min":{"1min":0.5,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.1019999999999999,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":1.1739999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:28:11 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.2889999999999999,"15min":1.369},"min":{"1min":0.83499999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.831,"5min":3.8079999999999998,"15min":3.8079999999999998},"last":1.7869999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.216,"15min":1.2969999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.748},"max":{"1min":1.5620000000000001,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.1200000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:28:11 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3839999999999999,"15min":1.429},"min":{"1min":0.80000000000000004,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.7669999999999999,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.375,"15min":1.4410000000000001},"min":{"1min":0.755,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0249999999999999,"5min":2.323,"15min":2.6259999999999999},"last":1.7230000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:28:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.3660000000000001,"15min":1.3799999999999999},"min":{"1min":0.80600000000000005,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":5.6070000000000002,"5min":5.6070000000000002,"15min":5.6070000000000002},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.4790000000000001,"15min":1.51},"min":{"1min":0.76400000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.351,"5min":2.899,"15min":2.899},"last":1.105}]},{"osd":7,"last update":"Mon Oct 16 11:28:11 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.4870000000000001,"15min":1.542},"min":{"1min":0.89200000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.806,"5min":3.742,"15min":3.742},"last":1.649},{"interface":"front","average":{"1min":1.365,"5min":1.4179999999999999,"15min":1.462},"min":{"1min":0.93899999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.7390000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":1.3919999999999999}]}]},{"osd":1,"up_from":13,"seq":55834576027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48472,"kb_used_data":3404,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323368,"statfs":{"total":96636764160,"available":96587128832,"internally_reserved":0,"allocated":3485696,"data_stored":1706169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.85999999999999999,"15min":0.90200000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.544,"5min":5.117,"15min":5.117},"last":1.4530000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.83599999999999997,"15min":0.97999999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.077,"5min":1.5069999999999999,"15min":1.633},"last":0.82899999999999996}]},{"osd":2,"last update":"Mon Oct 16 11:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.878,"15min":0.97199999999999998},"min":{"1min":0.46800000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2330000000000001,"5min":2.093,"15min":2.093},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.877,"5min":1.083,"15min":1.218},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5960000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":1.3109999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.0669999999999999,"15min":1.1619999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.647,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.369},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.0089999999999999,"15min":1.125},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2849999999999999,"5min":1.798,"15min":1.802},"last":1.161}]},{"osd":4,"last update":"Mon Oct 16 11:27:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.248,"15min":1.2709999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.51100000000000001},"max":{"1min":1.78,"5min":5.29,"15min":5.29},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.3460000000000001,"15min":1.397},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.734,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":1.202}]},{"osd":5,"last update":"Mon Oct 16 11:27:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.3080000000000001,"15min":1.3799999999999999},"min":{"1min":0.72899999999999998,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6870000000000001,"5min":6.0979999999999999,"15min":6.0979999999999999},"last":1.577},{"interface":"front","average":{"1min":1.238,"5min":1.3120000000000001,"15min":1.3799999999999999},"min":{"1min":0.73999999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8660000000000001,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.6459999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:27:59 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3939999999999999,"15min":1.468},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.3839999999999999,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.3580000000000001,"15min":1.448},"min":{"1min":0.80600000000000005,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.6220000000000001,"5min":3.8799999999999999,"15min":3.8799999999999999},"last":1.526}]},{"osd":7,"last update":"Mon Oct 16 11:27:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.468,"15min":1.536},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.4329999999999998,"5min":4.46,"15min":4.46},"last":2.0209999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.476,"15min":1.571},"min":{"1min":0.79800000000000004,"5min":0.753,"15min":0.753},"max":{"1min":1.8799999999999999,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":1.8160000000000001}]}]},{"osd":0,"up_from":13,"seq":55834576028,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49048,"kb_used_data":3984,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322792,"statfs":{"total":96636764160,"available":96586539008,"internally_reserved":0,"allocated":4079616,"data_stored":2296537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.85999999999999999,"15min":0.879},"min":{"1min":0.46000000000000002,"5min":0.42699999999999999,"15min":0.34000000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.3200000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.95199999999999996,"15min":1.0509999999999999},"min":{"1min":0.44,"5min":0.40000000000000002,"15min":0.38200000000000001},"max":{"1min":1.3300000000000001,"5min":1.621,"15min":1.76},"last":0.76500000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":1.0469999999999999,"15min":1.069},"min":{"1min":0.47999999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4490000000000001,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.91400000000000003,"5min":1.026,"15min":1.099},"min":{"1min":0.48899999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5569999999999999,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.96999999999999997}]},{"osd":3,"last update":"Mon Oct 16 11:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.99299999999999999,"15min":1.054},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.44900000000000001},"max":{"1min":1.4430000000000001,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1299999999999999,"15min":1.21},"min":{"1min":0.59099999999999997,"5min":0.55600000000000005,"15min":0.48999999999999999},"max":{"1min":1.7869999999999999,"5min":1.833,"15min":1.833},"last":1.5660000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.2989999999999999,"15min":1.365},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.6659999999999999,"5min":3.2799999999999998,"15min":3.2799999999999998},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.288,"15min":1.325},"min":{"1min":0.83699999999999997,"5min":0.76100000000000001,"15min":0.71499999999999997},"max":{"1min":1.8580000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.3640000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.292,"15min":1.3560000000000001},"min":{"1min":0.61599999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0030000000000001,"5min":2.0209999999999999,"15min":2.6669999999999998},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.3799999999999999,"15min":1.4239999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.70499999999999996},"max":{"1min":1.8420000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.194}]},{"osd":6,"last update":"Mon Oct 16 11:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.4430000000000001,"15min":1.4790000000000001},"min":{"1min":0.84099999999999997,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.831,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.3959999999999999,"15min":1.4610000000000001},"min":{"1min":0.90500000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.9019999999999999,"5min":2.073,"15min":2.262},"last":1.2330000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.5309999999999999,"15min":1.5429999999999999},"min":{"1min":1.0409999999999999,"5min":0.88700000000000001,"15min":0.72099999999999997},"max":{"1min":2.0009999999999999,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.4059999999999999,"15min":1.4770000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.60299999999999998},"max":{"1min":1.925,"5min":1.9550000000000001,"15min":2.8199999999999998},"last":1.5169999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:28:23.145 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-16T11:28:23.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.344+0000 7f463eec9640 1 Processor -- start 2023-10-16T11:28:23.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.344+0000 7f463eec9640 1 -- start start 2023-10-16T11:28:23.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.346+0000 7f463eec9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f463806a4b0 0x7f46380ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:23.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.346+0000 7f463eec9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46380c9ca0 0x7f46380ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:23.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.346+0000 7f463eec9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46380cb030 con 0x7f463815ad70 2023-10-16T11:28:23.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.346+0000 7f463eec9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46380cae30 con 0x7f463806a4b0 2023-10-16T11:28:23.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.346+0000 7f463eec9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46380c91c0 con 0x7f46380c9ca0 2023-10-16T11:28:23.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.346+0000 7f463cc65640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f463815ad70 0x7f4638069da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34406/0 (socket says 172.21.15.84:34406) 2023-10-16T11:28:23.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.346+0000 7f463cc65640 1 -- 172.21.15.84:0/1030642927 learned_addr learned my addr 172.21.15.84:0/1030642927 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-16T11:28:23.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.347+0000 7f463d466640 1 --2- 172.21.15.84:0/1030642927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46380c9ca0 0x7f46380ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:23.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.347+0000 7f46377fe640 1 -- 172.21.15.84:0/1030642927 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957028994 0 0) 0x7f46380cb030 con 0x7f463815ad70 2023-10-16T11:28:23.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.347+0000 7f46377fe640 1 -- 172.21.15.84:0/1030642927 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4628003520 con 0x7f463815ad70 2023-10-16T11:28:23.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.347+0000 7f4637fff640 1 --2- 172.21.15.84:0/1030642927 >> v2:172.21.15.157:3300/0 conn(0x7f463806a4b0 0x7f46380ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:23.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.347+0000 7f46377fe640 1 -- 172.21.15.84:0/1030642927 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3484952133 0 0) 0x7f4628003520 con 0x7f463815ad70 2023-10-16T11:28:23.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.347+0000 7f46377fe640 1 -- 172.21.15.84:0/1030642927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46380c9ca0 msgr2=0x7f46380ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:23.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.347+0000 7f46377fe640 1 --2- 172.21.15.84:0/1030642927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46380c9ca0 0x7f46380ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.347+0000 7f46377fe640 1 -- 172.21.15.84:0/1030642927 >> v2:172.21.15.157:3300/0 conn(0x7f463806a4b0 msgr2=0x7f46380ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:23.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.347+0000 7f46377fe640 1 --2- 172.21.15.84:0/1030642927 >> v2:172.21.15.157:3300/0 conn(0x7f463806a4b0 0x7f46380ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.347+0000 7f46377fe640 1 -- 172.21.15.84:0/1030642927 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46380c94a0 con 0x7f463815ad70 2023-10-16T11:28:23.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.347+0000 7f46377fe640 1 -- 172.21.15.84:0/1030642927 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2710956704 0 0) 0x7f462c001670 con 0x7f463815ad70 2023-10-16T11:28:23.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.347+0000 7f46377fe640 1 -- 172.21.15.84:0/1030642927 >> v1:172.21.15.84:6789/0 conn(0x7f463815ad70 legacy=0x7f4638069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:23.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.347+0000 7f46377fe640 1 --2- 172.21.15.84:0/1030642927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4628003970 0x7f4628003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:23.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.347+0000 7f46377fe640 1 --2- 172.21.15.84:0/1030642927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46380c9ca0 0x7f46280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:23.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.347+0000 7f46377fe640 1 --2- 172.21.15.84:0/1030642927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4628004d00 0x7f4628005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:23.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.347+0000 7f46377fe640 1 -- 172.21.15.84:0/1030642927 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46380cae30 con 0x7f4628004d00 2023-10-16T11:28:23.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.347+0000 7f46377fe640 1 -- 172.21.15.84:0/1030642927 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46380c91c0 con 0x7f4628003970 2023-10-16T11:28:23.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.347+0000 7f46377fe640 1 -- 172.21.15.84:0/1030642927 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462800ba00 con 0x7f46380c9ca0 2023-10-16T11:28:23.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.348+0000 7f463d466640 1 --2- 172.21.15.84:0/1030642927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4628004d00 0x7f4628005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:23.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.348+0000 7f4637fff640 1 --2- 172.21.15.84:0/1030642927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46380c9ca0 0x7f46280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:23.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.348+0000 7f463d466640 1 -- 172.21.15.84:0/1030642927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46380c9ca0 msgr2=0x7f46280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:23.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.348+0000 7f463cc65640 1 --2- 172.21.15.84:0/1030642927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4628003970 0x7f4628003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:23.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.348+0000 7f463d466640 1 --2- 172.21.15.84:0/1030642927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46380c9ca0 0x7f46280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.348+0000 7f463d466640 1 -- 172.21.15.84:0/1030642927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4628003970 msgr2=0x7f4628003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:23.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.348+0000 7f463d466640 1 --2- 172.21.15.84:0/1030642927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4628003970 0x7f4628003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.348+0000 7f463d466640 1 -- 172.21.15.84:0/1030642927 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4628003520 con 0x7f4628004d00 2023-10-16T11:28:23.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.348+0000 7f4637fff640 1 --2- 172.21.15.84:0/1030642927 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46380c9ca0 0x7f46280047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:28:23.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.349+0000 7f463d466640 1 --2- 172.21.15.84:0/1030642927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4628004d00 0x7f4628005180 secure :-1 s=READY pgs=3592 cs=0 l=1 rev1=1 crypto rx=0x7f4630009fc0 tx=0x7f46280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed3598d6feca2db server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:23.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.349+0000 7f46377fe640 1 -- 172.21.15.84:0/1030642927 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463002cc20 con 0x7f4628004d00 2023-10-16T11:28:23.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.350+0000 7f46377fe640 1 -- 172.21.15.84:0/1030642927 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463000f040 con 0x7f4628004d00 2023-10-16T11:28:23.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.350+0000 7f46377fe640 1 -- 172.21.15.84:0/1030642927 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630034440 con 0x7f4628004d00 2023-10-16T11:28:23.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.350+0000 7f463eec9640 1 -- 172.21.15.84:0/1030642927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4628004d00 msgr2=0x7f4628005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:23.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.350+0000 7f463eec9640 1 --2- 172.21.15.84:0/1030642927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4628004d00 0x7f4628005180 secure :-1 s=READY pgs=3592 cs=0 l=1 rev1=1 crypto rx=0x7f4630009fc0 tx=0x7f46280057f0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.350+0000 7f463d466640 1 -- 172.21.15.84:0/1030642927 reap_dead start 2023-10-16T11:28:23.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.350+0000 7f463eec9640 1 -- 172.21.15.84:0/1030642927 shutdown_connections 2023-10-16T11:28:23.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.350+0000 7f463eec9640 1 -- 172.21.15.84:0/1030642927 >> 172.21.15.84:0/1030642927 conn(0x7f46380ce9b0 msgr2=0x7f46380cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:23.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.350+0000 7f463eec9640 1 -- 172.21.15.84:0/1030642927 shutdown_connections 2023-10-16T11:28:23.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.350+0000 7f463eec9640 1 -- 172.21.15.84:0/1030642927 wait complete. 2023-10-16T11:28:23.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.352+0000 7f463eec9640 1 Processor -- start 2023-10-16T11:28:23.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.352+0000 7f463eec9640 1 -- start start 2023-10-16T11:28:23.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.353+0000 7f463eec9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4628004d00 0x7f46380aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:23.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.353+0000 7f463eec9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f463815d420 0x7f463815d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:23.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.353+0000 7f463eec9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f463814e5d0 0x7f463814ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:23.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.353+0000 7f463eec9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46380ad640 con 0x7f4628004d00 2023-10-16T11:28:23.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.353+0000 7f463eec9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463815ddc0 con 0x7f463815d420 2023-10-16T11:28:23.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.353+0000 7f463eec9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46380ac9c0 con 0x7f463814e5d0 2023-10-16T11:28:23.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.353+0000 7f463cc65640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4628004d00 0x7f46380aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:23.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.353+0000 7f463cc65640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4628004d00 0x7f46380aefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37886/0 (socket says 172.21.15.84:37886) 2023-10-16T11:28:23.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.353+0000 7f463cc65640 1 -- 172.21.15.84:0/886173018 learned_addr learned my addr 172.21.15.84:0/886173018 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:28:23.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.354+0000 7f463cc65640 1 -- 172.21.15.84:0/886173018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f463814e5d0 msgr2=0x7f463814ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:23.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.354+0000 7f463cc65640 1 --2- 172.21.15.84:0/886173018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f463814e5d0 0x7f463814ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.354+0000 7f463cc65640 1 -- 172.21.15.84:0/886173018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f463815d420 msgr2=0x7f463815d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:23.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.354+0000 7f463cc65640 1 --2- 172.21.15.84:0/886173018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f463815d420 0x7f463815d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.354+0000 7f463cc65640 1 -- 172.21.15.84:0/886173018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46380acca0 con 0x7f4628004d00 2023-10-16T11:28:23.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.354+0000 7f463cc65640 1 --2- 172.21.15.84:0/886173018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4628004d00 0x7f46380aefe0 secure :-1 s=READY pgs=3593 cs=0 l=1 rev1=1 crypto rx=0x7f462c00d810 tx=0x7f462c00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:23.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.354+0000 7f46357fa640 1 -- 172.21.15.84:0/886173018 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c0074e0 con 0x7f4628004d00 2023-10-16T11:28:23.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.354+0000 7f46357fa640 1 -- 172.21.15.84:0/886173018 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c007640 con 0x7f4628004d00 2023-10-16T11:28:23.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.354+0000 7f46357fa640 1 -- 172.21.15.84:0/886173018 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c004a20 con 0x7f4628004d00 2023-10-16T11:28:23.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.354+0000 7f463eec9640 1 -- 172.21.15.84:0/886173018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46380acf80 con 0x7f4628004d00 2023-10-16T11:28:23.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.355+0000 7f463eec9640 1 -- 172.21.15.84:0/886173018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463814fa30 con 0x7f4628004d00 2023-10-16T11:28:23.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.356+0000 7f463eec9640 1 -- 172.21.15.84:0/886173018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46380ad780 con 0x7f4628004d00 2023-10-16T11:28:23.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.357+0000 7f46357fa640 1 -- 172.21.15.84:0/886173018 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f462c004b80 con 0x7f4628004d00 2023-10-16T11:28:23.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.357+0000 7f46357fa640 1 --2- 172.21.15.84:0/886173018 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f461c06af90 0x7f461c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:23.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.358+0000 7f46357fa640 1 -- 172.21.15.84:0/886173018 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f462c089170 con 0x7f4628004d00 2023-10-16T11:28:23.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.358+0000 7f4637fff640 1 --2- 172.21.15.84:0/886173018 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f461c06af90 0x7f461c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:23.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.358+0000 7f4637fff640 1 --2- 172.21.15.84:0/886173018 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f461c06af90 0x7f461c06d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f46380b0330 tx=0x7f46240093d0 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:23.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.362+0000 7f46357fa640 1 -- 172.21.15.84:0/886173018 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f46380ad780 con 0x7f4628004d00 2023-10-16T11:28:23.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.555+0000 7f463eec9640 1 -- 172.21.15.84:0/886173018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f46380ad780 con 0x7f4628004d00 2023-10-16T11:28:23.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.562+0000 7f46357fa640 1 -- 172.21.15.84:0/886173018 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v100)=0 v100) v1 ==== 123+0+0 (secure 0 0 0) 0x7f46380ad780 con 0x7f4628004d00 2023-10-16T11:28:23.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.565+0000 7f46357fa640 1 -- 172.21.15.84:0/886173018 <== mon.0 v2:172.21.15.84:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f462c054020 con 0x7f4628004d00 2023-10-16T11:28:23.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.566+0000 7f463eec9640 1 -- 172.21.15.84:0/886173018 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f461c06af90 msgr2=0x7f461c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:23.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.566+0000 7f463eec9640 1 --2- 172.21.15.84:0/886173018 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f461c06af90 0x7f461c06d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f46380b0330 tx=0x7f46240093d0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.566+0000 7f463eec9640 1 -- 172.21.15.84:0/886173018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4628004d00 msgr2=0x7f46380aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:23.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.566+0000 7f463eec9640 1 --2- 172.21.15.84:0/886173018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4628004d00 0x7f46380aefe0 secure :-1 s=READY pgs=3593 cs=0 l=1 rev1=1 crypto rx=0x7f462c00d810 tx=0x7f462c00dcd0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.566+0000 7f463eec9640 1 -- 172.21.15.84:0/886173018 shutdown_connections 2023-10-16T11:28:23.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.566+0000 7f463eec9640 1 --2- 172.21.15.84:0/886173018 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f461c06af90 0x7f461c06d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.566+0000 7f463eec9640 1 --2- 172.21.15.84:0/886173018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f463814e5d0 0x7f463814ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.567+0000 7f463eec9640 1 --2- 172.21.15.84:0/886173018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f463815d420 0x7f463815d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.567+0000 7f463eec9640 1 --2- 172.21.15.84:0/886173018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4628004d00 0x7f46380aefe0 unknown :-1 s=CLOSED pgs=3593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.567+0000 7f463eec9640 1 -- 172.21.15.84:0/886173018 >> 172.21.15.84:0/886173018 conn(0x7f46380ce9b0 msgr2=0x7f46380c7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:23.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.567+0000 7f463eec9640 1 -- 172.21.15.84:0/886173018 shutdown_connections 2023-10-16T11:28:23.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.567+0000 7f463eec9640 1 -- 172.21.15.84:0/886173018 wait complete. 2023-10-16T11:28:23.587 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T11:28:23.588 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:28:23.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.794+0000 7f4ab1690640 1 Processor -- start 2023-10-16T11:28:23.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.794+0000 7f4ab1690640 1 -- start start 2023-10-16T11:28:23.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.795+0000 7f4ab1690640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aac159950 0x7f4aac15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:23.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.795+0000 7f4ab1690640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4aac070450 0x7f4aac0708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:23.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.795+0000 7f4ab1690640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4aac06c100 0x7f4aac06c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:23.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.796+0000 7f4ab1690640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aac15e490 con 0x7f4aac06c100 2023-10-16T11:28:23.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.796+0000 7f4ab1690640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aac070df0 con 0x7f4aac070450 2023-10-16T11:28:23.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.796+0000 7f4ab1690640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac15aef0 con 0x7f4aac159950 2023-10-16T11:28:23.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.796+0000 7f4ab0e8f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4aac06c100 0x7f4aac06c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:23.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.796+0000 7f4ab0e8f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4aac06c100 0x7f4aac06c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37902/0 (socket says 172.21.15.84:37902) 2023-10-16T11:28:23.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.796+0000 7f4ab0e8f640 1 -- 172.21.15.84:0/2619560426 learned_addr learned my addr 172.21.15.84:0/2619560426 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:28:23.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.796+0000 7f4aab7fe640 1 --2- 172.21.15.84:0/2619560426 >> v2:172.21.15.157:3300/0 conn(0x7f4aac070450 0x7f4aac0708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:23.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.796+0000 7f4aabfff640 1 --2- 172.21.15.84:0/2619560426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aac159950 0x7f4aac15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:23.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.796+0000 7f4ab0e8f640 1 -- 172.21.15.84:0/2619560426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aac159950 msgr2=0x7f4aac15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:23.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.796+0000 7f4ab0e8f640 1 --2- 172.21.15.84:0/2619560426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aac159950 0x7f4aac15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.796+0000 7f4ab0e8f640 1 -- 172.21.15.84:0/2619560426 >> v2:172.21.15.157:3300/0 conn(0x7f4aac070450 msgr2=0x7f4aac0708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:28:23.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.796+0000 7f4ab0e8f640 1 --2- 172.21.15.84:0/2619560426 >> v2:172.21.15.157:3300/0 conn(0x7f4aac070450 0x7f4aac0708b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.796+0000 7f4ab0e8f640 1 -- 172.21.15.84:0/2619560426 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac15d910 con 0x7f4aac06c100 2023-10-16T11:28:23.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.796+0000 7f4aabfff640 1 --2- 172.21.15.84:0/2619560426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aac159950 0x7f4aac15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:28:23.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.798+0000 7f4ab0e8f640 1 --2- 172.21.15.84:0/2619560426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aac06c100 0x7f4aac06c570 secure :-1 s=READY pgs=3594 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0719e0 tx=0x7f4aa402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca07287a8fcbc2a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:23.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.798+0000 7f4aaaffd640 1 -- 172.21.15.84:0/2619560426 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa402c9f0 con 0x7f4aac06c100 2023-10-16T11:28:23.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.798+0000 7f4aaaffd640 1 -- 172.21.15.84:0/2619560426 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4aa402cb50 con 0x7f4aac06c100 2023-10-16T11:28:23.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.798+0000 7f4aaaffd640 1 -- 172.21.15.84:0/2619560426 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa40343f0 con 0x7f4aac06c100 2023-10-16T11:28:23.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.798+0000 7f4ab1690640 1 -- 172.21.15.84:0/2619560426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aac06c100 msgr2=0x7f4aac06c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:23.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.798+0000 7f4ab1690640 1 --2- 172.21.15.84:0/2619560426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aac06c100 0x7f4aac06c570 secure :-1 s=READY pgs=3594 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0719e0 tx=0x7f4aa402be50 comp rx=0 tx=0).stop 2023-10-16T11:28:23.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.799+0000 7f4ab1690640 1 -- 172.21.15.84:0/2619560426 shutdown_connections 2023-10-16T11:28:23.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.799+0000 7f4ab1690640 1 --2- 172.21.15.84:0/2619560426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aac06c100 0x7f4aac06c570 unknown :-1 s=CLOSED pgs=3594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.799+0000 7f4ab1690640 1 --2- 172.21.15.84:0/2619560426 >> v2:172.21.15.157:3300/0 conn(0x7f4aac070450 0x7f4aac0708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.799+0000 7f4ab1690640 1 --2- 172.21.15.84:0/2619560426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aac159950 0x7f4aac15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.799+0000 7f4ab1690640 1 -- 172.21.15.84:0/2619560426 >> 172.21.15.84:0/2619560426 conn(0x7f4aac0ce6a0 msgr2=0x7f4aac0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:23.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.799+0000 7f4ab1690640 1 -- 172.21.15.84:0/2619560426 shutdown_connections 2023-10-16T11:28:23.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.799+0000 7f4ab1690640 1 -- 172.21.15.84:0/2619560426 wait complete. 2023-10-16T11:28:23.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.801+0000 7f4ab1690640 1 Processor -- start 2023-10-16T11:28:23.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.801+0000 7f4ab1690640 1 -- start start 2023-10-16T11:28:23.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.802+0000 7f4ab1690640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aac06c100 0x7f4aac0c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:23.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.802+0000 7f4ab1690640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aac070450 0x7f4aac1554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:23.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.802+0000 7f4ab1690640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aac159950 0x7f4aac155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:23.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.802+0000 7f4ab1690640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac158c30 con 0x7f4aac06c100 2023-10-16T11:28:23.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.802+0000 7f4ab1690640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac155f50 con 0x7f4aac159950 2023-10-16T11:28:23.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.802+0000 7f4ab1690640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac157fb0 con 0x7f4aac070450 2023-10-16T11:28:23.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.802+0000 7f4aabfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aac06c100 0x7f4aac0c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:23.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.802+0000 7f4aabfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aac06c100 0x7f4aac0c2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37904/0 (socket says 172.21.15.84:37904) 2023-10-16T11:28:23.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.802+0000 7f4aabfff640 1 -- 172.21.15.84:0/2850334416 learned_addr learned my addr 172.21.15.84:0/2850334416 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-16T11:28:23.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.802+0000 7f4aabfff640 1 -- 172.21.15.84:0/2850334416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aac070450 msgr2=0x7f4aac1554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:23.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.802+0000 7f4aab7fe640 1 --2- 172.21.15.84:0/2850334416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aac070450 0x7f4aac1554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:23.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.802+0000 7f4ab0e8f640 1 --2- 172.21.15.84:0/2850334416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aac159950 0x7f4aac155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:23.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.802+0000 7f4aabfff640 1 --2- 172.21.15.84:0/2850334416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aac070450 0x7f4aac1554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.802+0000 7f4aabfff640 1 -- 172.21.15.84:0/2850334416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aac159950 msgr2=0x7f4aac155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:23.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.802+0000 7f4aabfff640 1 --2- 172.21.15.84:0/2850334416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aac159950 0x7f4aac155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:23.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.802+0000 7f4aabfff640 1 -- 172.21.15.84:0/2850334416 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa4009b30 con 0x7f4aac06c100 2023-10-16T11:28:23.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.803+0000 7f4aabfff640 1 --2- 172.21.15.84:0/2850334416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aac06c100 0x7f4aac0c2be0 secure :-1 s=READY pgs=3595 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00c7f0 tx=0x7f4a9c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:23.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.803+0000 7f4aa8ff9640 1 -- 172.21.15.84:0/2850334416 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c007a30 con 0x7f4aac06c100 2023-10-16T11:28:23.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.803+0000 7f4aa8ff9640 1 -- 172.21.15.84:0/2850334416 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4a9c007b90 con 0x7f4aac06c100 2023-10-16T11:28:23.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.803+0000 7f4ab1690640 1 -- 172.21.15.84:0/2850334416 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aac158290 con 0x7f4aac06c100 2023-10-16T11:28:23.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.803+0000 7f4aa8ff9640 1 -- 172.21.15.84:0/2850334416 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c00ce40 con 0x7f4aac06c100 2023-10-16T11:28:23.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.803+0000 7f4ab1690640 1 -- 172.21.15.84:0/2850334416 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aac158850 con 0x7f4aac06c100 2023-10-16T11:28:23.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.804+0000 7f4ab1690640 1 -- 172.21.15.84:0/2850334416 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aac04e9b0 con 0x7f4aac06c100 2023-10-16T11:28:23.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.805+0000 7f4aa8ff9640 1 -- 172.21.15.84:0/2850334416 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4a9c0113f0 con 0x7f4aac06c100 2023-10-16T11:28:23.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.806+0000 7f4aa8ff9640 1 --2- 172.21.15.84:0/2850334416 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f4a9806af90 0x7f4a9806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:28:23.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.806+0000 7f4aa8ff9640 1 -- 172.21.15.84:0/2850334416 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f4a9c089a10 con 0x7f4aac06c100 2023-10-16T11:28:23.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.807+0000 7f4aab7fe640 1 --2- 172.21.15.84:0/2850334416 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f4a9806af90 0x7f4a9806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:28:23.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.807+0000 7f4aab7fe640 1 --2- 172.21.15.84:0/2850334416 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f4a9806af90 0x7f4a9806d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4aac06daf0 tx=0x7f4aa0008040 comp rx=0 tx=0).ready entity=mgr.13719 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:28:23.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:23.810+0000 7f4aa8ff9640 1 -- 172.21.15.84:0/2850334416 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4a9c0548c0 con 0x7f4aac06c100 2023-10-16T11:28:24.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:24.046+0000 7f4ab1690640 1 -- 172.21.15.84:0/2850334416 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4aac0b4620 con 0x7f4aac06c100 2023-10-16T11:28:24.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:24.047+0000 7f4aa8ff9640 1 -- 172.21.15.84:0/2850334416 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 159 v159) v1 ==== 95+0+3712 (secure 0 0 0) 0x7f4a9c01f090 con 0x7f4aac06c100 2023-10-16T11:28:24.049 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-16T11:28:24.049 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":159,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7457,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3193159532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3193159532},{"type":"v1","addr":"172.21.15.157:6835","nonce":3193159532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":152},{"gid":8431,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1728897149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1728897149},{"type":"v1","addr":"172.21.15.157:6834","nonce":1728897149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":152},{"gid":13476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3409165899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3409165899},{"type":"v1","addr":"172.21.15.84:6835","nonce":3409165899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":151},{"gid":13875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/1762534548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":1762534548},{"type":"v1","addr":"172.21.15.84:6837","nonce":1762534548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":159}],"filesystems":[]} 2023-10-16T11:28:24.050 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 159 2023-10-16T11:28:24.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:24.049+0000 7f4ab1690640 1 -- 172.21.15.84:0/2850334416 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f4a9806af90 msgr2=0x7f4a9806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:24.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:24.049+0000 7f4ab1690640 1 --2- 172.21.15.84:0/2850334416 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f4a9806af90 0x7f4a9806d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4aac06daf0 tx=0x7f4aa0008040 comp rx=0 tx=0).stop 2023-10-16T11:28:24.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:24.049+0000 7f4ab1690640 1 -- 172.21.15.84:0/2850334416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aac06c100 msgr2=0x7f4aac0c2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:28:24.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:24.049+0000 7f4ab1690640 1 --2- 172.21.15.84:0/2850334416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aac06c100 0x7f4aac0c2be0 secure :-1 s=READY pgs=3595 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00c7f0 tx=0x7f4a9c00cbf0 comp rx=0 tx=0).stop 2023-10-16T11:28:24.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:24.050+0000 7f4ab1690640 1 -- 172.21.15.84:0/2850334416 shutdown_connections 2023-10-16T11:28:24.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:24.050+0000 7f4ab1690640 1 --2- 172.21.15.84:0/2850334416 >> [v2:172.21.15.84:6832/3003264401,v1:172.21.15.84:6833/3003264401] conn(0x7f4a9806af90 0x7f4a9806d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:24.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:24.050+0000 7f4ab1690640 1 --2- 172.21.15.84:0/2850334416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4aac159950 0x7f4aac155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:24.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:24.050+0000 7f4ab1690640 1 --2- 172.21.15.84:0/2850334416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4aac070450 0x7f4aac1554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:24.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:24.050+0000 7f4ab1690640 1 --2- 172.21.15.84:0/2850334416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aac06c100 0x7f4aac0c2be0 unknown :-1 s=CLOSED pgs=3595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:28:24.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:24.050+0000 7f4ab1690640 1 -- 172.21.15.84:0/2850334416 >> 172.21.15.84:0/2850334416 conn(0x7f4aac0ce6a0 msgr2=0x7f4aac0c9310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:28:24.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:24.050+0000 7f4ab1690640 1 -- 172.21.15.84:0/2850334416 shutdown_connections 2023-10-16T11:28:24.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-16T11:28:24.050+0000 7f4ab1690640 1 -- 172.21.15.84:0/2850334416 wait complete. 2023-10-16T11:28:24.071 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-16T11:28:24.071 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-16T11:28:24.072 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T11:28:24.072 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:28:24.091 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T11:28:24.092 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T11:28:24.092 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:28:24.157 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T11:28:24.158 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T11:28:24.158 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:28:24.187 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T11:28:24.188 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T11:28:24.188 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:28:24.256 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T11:28:24.257 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-16T11:28:24.257 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-16T11:28:24.258 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:28:24.348 INFO:tasks.ceph.osd.0:Stopped 2023-10-16T11:28:24.349 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-16T11:28:24.349 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:28:24.431 INFO:tasks.ceph.osd.1:Stopped 2023-10-16T11:28:24.432 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-16T11:28:24.432 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:28:24.511 INFO:tasks.ceph.osd.2:Stopped 2023-10-16T11:28:24.511 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-16T11:28:24.512 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:28:24.604 INFO:tasks.ceph.osd.3:Stopped 2023-10-16T11:28:24.605 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-16T11:28:24.605 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:28:24.676 INFO:tasks.ceph.osd.4:Stopped 2023-10-16T11:28:24.677 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-16T11:28:24.677 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:28:24.767 INFO:tasks.ceph.osd.5:Stopped 2023-10-16T11:28:24.768 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-16T11:28:24.768 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:28:24.852 INFO:tasks.ceph.osd.6:Stopped 2023-10-16T11:28:24.853 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-16T11:28:24.853 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:28:24.918 INFO:tasks.ceph.osd.7:Stopped 2023-10-16T11:28:24.918 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-16T11:28:24.919 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-16T11:28:24.919 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:28:24.996 INFO:tasks.ceph.mgr.y:Stopped 2023-10-16T11:28:24.997 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-16T11:28:24.997 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:28:28.549 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2023-10-16T11:28:31.099 INFO:tasks.ceph.mgr.x:Stopped 2023-10-16T11:28:31.100 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-16T11:28:31.100 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-16T11:28:31.101 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:28:31.185 INFO:tasks.ceph.mon.a:Stopped 2023-10-16T11:28:31.185 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-16T11:28:31.186 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:28:31.248 INFO:tasks.ceph.mon.b:Stopped 2023-10-16T11:28:31.249 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-16T11:28:31.249 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:28:31.315 INFO:tasks.ceph.mon.c:Stopped 2023-10-16T11:28:31.316 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-16T11:28:31.316 DEBUG:teuthology.orchestra.run.smithi084:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-16T11:28:31.693 INFO:teuthology.orchestra.run.smithi084.stdout:2023-10-16T10:07:06.139299+0000 mon.a (mon.0) 1183 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-10-16T11:28:31.695 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-16T11:28:31.695 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T11:28:31.696 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-16T11:28:31.951 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T11:28:31.952 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-16T11:28:32.135 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T11:28:32.136 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-16T11:28:32.309 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T11:28:32.309 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-16T11:28:32.480 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi157.front.sepia.ceph.com 2023-10-16T11:28:32.481 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-16T11:28:32.738 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi157.front.sepia.ceph.com 2023-10-16T11:28:32.738 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-16T11:28:32.885 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi157.front.sepia.ceph.com 2023-10-16T11:28:32.886 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-16T11:28:33.031 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi157.front.sepia.ceph.com 2023-10-16T11:28:33.031 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-16T11:28:33.228 INFO:tasks.ceph:Archiving mon data... 2023-10-16T11:28:33.229 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428968/data/mon.a.tgz 2023-10-16T11:28:33.230 DEBUG:teuthology.orchestra.run.smithi084:> mktemp 2023-10-16T11:28:33.291 INFO:teuthology.orchestra.run.smithi084.stdout:/tmp/tmp.5vqIxc3wlB 2023-10-16T11:28:33.292 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.5vqIxc3wlB 2023-10-16T11:28:33.551 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2023-10-16T11:28:34.025 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0666 /tmp/tmp.5vqIxc3wlB 2023-10-16T11:28:34.177 DEBUG:teuthology.orchestra.remote:smithi084:/tmp/tmp.5vqIxc3wlB is 4MB 2023-10-16T11:28:34.344 DEBUG:teuthology.orchestra.run.smithi084:> rm -fr /tmp/tmp.5vqIxc3wlB 2023-10-16T11:28:34.405 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428968/data/mon.b.tgz 2023-10-16T11:28:34.405 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2023-10-16T11:28:34.467 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.x1jOlACjNT 2023-10-16T11:28:34.467 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.x1jOlACjNT 2023-10-16T11:28:35.266 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.x1jOlACjNT 2023-10-16T11:28:35.367 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:28:35.376 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:28:35.422 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.x1jOlACjNT is 4MB 2023-10-16T11:28:35.582 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.x1jOlACjNT 2023-10-16T11:28:35.640 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428968/data/mon.c.tgz 2023-10-16T11:28:35.640 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2023-10-16T11:28:35.690 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.bJdhV4Shcn 2023-10-16T11:28:35.691 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.bJdhV4Shcn 2023-10-16T11:28:36.378 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.bJdhV4Shcn 2023-10-16T11:28:36.531 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.bJdhV4Shcn is 4MB 2023-10-16T11:28:36.705 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.bJdhV4Shcn 2023-10-16T11:28:36.768 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-16T11:28:36.769 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-16T11:28:36.778 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-16T11:28:36.882 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-16T11:28:36.883 DEBUG:teuthology.orchestra.run.smithi084:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-16T11:28:36.892 DEBUG:teuthology.orchestra.run.smithi157:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-16T11:28:36.988 INFO:teuthology.orchestra.run.smithi084.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-16T11:28:36.997 INFO:teuthology.orchestra.run.smithi157.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-16T11:28:37.010 INFO:tasks.ceph:Archiving crash dumps... 2023-10-16T11:28:37.012 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428968/remote/smithi084/crash 2023-10-16T11:28:37.013 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-16T11:28:37.107 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428968/remote/smithi157/crash 2023-10-16T11:28:37.109 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-16T11:28:37.211 INFO:tasks.ceph:Shutting down logrotate 2023-10-16T11:28:37.212 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:28:37.221 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:28:37.349 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-16T11:28:37.447 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-16T11:28:37.551 INFO:tasks.ceph:Compressing logs... 2023-10-16T11:28:37.552 DEBUG:teuthology.orchestra.run.smithi084:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-16T11:28:37.561 DEBUG:teuthology.orchestra.run.smithi157:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-16T11:28:37.649 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135050.log 2023-10-16T11:28:37.650 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131233.log 2023-10-16T11:28:37.652 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139291.log 2023-10-16T11:28:37.652 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.135050.log.gz 2023-10-16T11:28:37.653 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.131233.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.131233.log.gz 2023-10-16T11:28:37.653 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124178.log 2023-10-16T11:28:37.654 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139291.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.139291.log.gz 2023-10-16T11:28:37.654 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94370.log 2023-10-16T11:28:37.655 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.124178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147422.log 2023-10-16T11:28:37.656 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.124178.log.gz 2023-10-16T11:28:37.656 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-16T11:28:37.657 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162940.log 2023-10-16T11:28:37.658 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94370.log.gz 2023-10-16T11:28:37.658 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147422.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.147422.log.gz 2023-10-16T11:28:37.658 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120629.log 2023-10-16T11:28:37.666 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87602.log 2023-10-16T11:28:37.666 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87562.log 2023-10-16T11:28:37.667 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.87602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87482.log 2023-10-16T11:28:37.667 INFO:teuthology.orchestra.run.smithi157.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87602.log.gz 2023-10-16T11:28:37.667 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.87562.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87562.log.gz 2023-10-16T11:28:37.667 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.92424.log 2023-10-16T11:28:37.668 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.87482.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-16T11:28:37.668 INFO:teuthology.orchestra.run.smithi157.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.87482.log.gz 2023-10-16T11:28:37.668 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.3.92424.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.92424.log.gz 2023-10-16T11:28:37.668 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-16T11:28:37.669 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87722.log 2023-10-16T11:28:37.672 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-16T11:28:37.672 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.87722.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.87722.log.gz 2023-10-16T11:28:37.672 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87522.log 2023-10-16T11:28:37.672 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-16T11:28:37.673 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.87522.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-16T11:28:37.673 INFO:teuthology.orchestra.run.smithi157.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.87522.log.gz 2023-10-16T11:28:37.674 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.162940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115814.log 2023-10-16T11:28:37.674 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.162940.log.gz 2023-10-16T11:28:37.675 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.120629.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.120629.log.gz 2023-10-16T11:28:37.675 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145180.log 2023-10-16T11:28:37.675 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.115814.log: gzip 88.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.115814.log.gz --verbose 2023-10-16T11:28:37.675 INFO:teuthology.orchestra.run.smithi084.stderr: -- /var/log/ceph/ceph-client.admin.99072.log 2023-10-16T11:28:37.676 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99401.log 2023-10-16T11:28:37.676 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.145180.log.gz 2023-10-16T11:28:37.676 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115617.log 2023-10-16T11:28:37.677 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99072.log.gz 2023-10-16T11:28:37.677 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99401.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99401.log.gz 2023-10-16T11:28:37.677 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135197.log 2023-10-16T11:28:37.677 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.115617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102676.log 2023-10-16T11:28:37.678 INFO:teuthology.orchestra.run.smithi084.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.115617.log.gz 2023-10-16T11:28:37.678 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135197.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.135197.log.gz 2023-10-16T11:28:37.678 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158303.log 2023-10-16T11:28:37.679 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.102676.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.102676.log.gz 2023-10-16T11:28:37.679 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115894.log 2023-10-16T11:28:37.679 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137948.log 88.9% -- replaced with /var/log/ceph/ceph-client.admin.158303.log.gz 2023-10-16T11:28:37.679 INFO:teuthology.orchestra.run.smithi084.stderr: 2023-10-16T11:28:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.115894.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115894.log.gz 2023-10-16T11:28:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161880.log 2023-10-16T11:28:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137948.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137948.log.gz 2023-10-16T11:28:37.680 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-16T11:28:37.681 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161880.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.161880.log.gz 2023-10-16T11:28:37.681 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137426.log 2023-10-16T11:28:37.687 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-16T11:28:37.687 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-16T11:28:37.687 INFO:teuthology.orchestra.run.smithi157.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-16T11:28:37.688 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87642.log 2023-10-16T11:28:37.688 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.87642.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.87642.log.gz 2023-10-16T11:28:37.688 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.92031.log 2023-10-16T11:28:37.688 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.92031.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-16T11:28:37.689 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mon.b.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.92031.log.gz 2023-10-16T11:28:37.689 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87682.log 2023-10-16T11:28:37.691 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162567.log 2023-10-16T11:28:37.692 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137426.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.137426.log.gz 2023-10-16T11:28:37.692 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143671.log 2023-10-16T11:28:37.692 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.162567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93688.log 2023-10-16T11:28:37.693 INFO:teuthology.orchestra.run.smithi084.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.162567.log.gz 2023-10-16T11:28:37.693 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143671.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.143671.log.gz 2023-10-16T11:28:37.693 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154144.log 2023-10-16T11:28:37.693 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144402.log 2023-10-16T11:28:37.694 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.93688.log.gz 2023-10-16T11:28:37.694 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154144.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.154144.log.gz 2023-10-16T11:28:37.694 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153099.log 2023-10-16T11:28:37.695 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152721.log 2023-10-16T11:28:37.695 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144402.log.gz 2023-10-16T11:28:37.695 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153099.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.153099.log.gz 2023-10-16T11:28:37.695 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154937.log 2023-10-16T11:28:37.696 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111859.log 2023-10-16T11:28:37.696 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.152721.log.gz 2023-10-16T11:28:37.696 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154937.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.154937.log.gz 2023-10-16T11:28:37.696 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165552.log 2023-10-16T11:28:37.697 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.111859.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.111859.log.gz 2023-10-16T11:28:37.697 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156417.log 2023-10-16T11:28:37.697 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.165552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115974.log 2023-10-16T11:28:37.697 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.165552.log.gz 2023-10-16T11:28:37.698 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156417.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.156417.log.gz 2023-10-16T11:28:37.698 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152376.log 2023-10-16T11:28:37.698 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.115974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153649.log 2023-10-16T11:28:37.699 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115974.log.gz 2023-10-16T11:28:37.700 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-16T11:28:37.700 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-client.admin.87682.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87682.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87442.log 2023-10-16T11:28:37.701 INFO:teuthology.orchestra.run.smithi157.stderr: 2023-10-16T11:28:37.701 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-16T11:28:37.704 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152376.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.152376.log.gz 2023-10-16T11:28:37.704 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100396.log 2023-10-16T11:28:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153649.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.153649.log.gz 2023-10-16T11:28:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157678.log 2023-10-16T11:28:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.100396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143399.log 2023-10-16T11:28:37.705 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100396.log.gz 2023-10-16T11:28:37.706 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157678.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.157678.log.gz 2023-10-16T11:28:37.706 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125874.log 2023-10-16T11:28:37.706 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137718.log 2023-10-16T11:28:37.707 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.143399.log.gz 2023-10-16T11:28:37.707 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125874.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125874.log.gz 2023-10-16T11:28:37.707 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159829.log 2023-10-16T11:28:37.707 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137718.log: gzip 87.4% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.137718.log.gz -- 2023-10-16T11:28:37.708 INFO:teuthology.orchestra.run.smithi084.stderr: /var/log/ceph/ceph-client.admin.128857.log 2023-10-16T11:28:37.708 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.159829.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.159829.log.gz 2023-10-16T11:28:37.708 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152805.log 2023-10-16T11:28:37.709 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.128857.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128857.log.gz 2023-10-16T11:28:37.709 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161540.log 2023-10-16T11:28:37.709 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152805.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.152805.log.gz 2023-10-16T11:28:37.709 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155371.log 2023-10-16T11:28:37.710 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.87442.log: 88.1%/var/log/ceph/ceph-mon.c.log: -- replaced with /var/log/ceph/ceph-client.admin.87442.log.gz 2023-10-16T11:28:37.711 INFO:teuthology.orchestra.run.smithi157.stderr: 94.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-16T11:28:37.713 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161540.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.161540.log.gz 2023-10-16T11:28:37.713 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165384.log 2023-10-16T11:28:37.714 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123666.log 2023-10-16T11:28:37.714 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.155371.log.gz 2023-10-16T11:28:37.714 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.165384.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.165384.log.gz 2023-10-16T11:28:37.714 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121094.log 2023-10-16T11:28:37.715 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.123666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131060.log 2023-10-16T11:28:37.715 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123666.log.gz 2023-10-16T11:28:37.715 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121094.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.121094.log.gz 2023-10-16T11:28:37.716 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165626.log 2023-10-16T11:28:37.716 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.131060.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.131060.log.gz 2023-10-16T11:28:37.716 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102844.log 2023-10-16T11:28:37.716 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.165626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115178.log 2023-10-16T11:28:37.717 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.165626.log.gz 2023-10-16T11:28:37.717 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.102844.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102844.log.gz 2023-10-16T11:28:37.717 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125176.log 2023-10-16T11:28:37.719 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.115178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138631.log 2023-10-16T11:28:37.720 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115178.log.gz 2023-10-16T11:28:37.720 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138775.log 2023-10-16T11:28:37.720 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125176.log.gz 2023-10-16T11:28:37.720 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162898.log 2023-10-16T11:28:37.721 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.138631.log.gz 2023-10-16T11:28:37.721 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138775.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138775.log.gz 2023-10-16T11:28:37.721 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120832.log 2023-10-16T11:28:37.722 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.162898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104188.log 2023-10-16T11:28:37.722 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.162898.log.gz 2023-10-16T11:28:37.722 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.120832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103307.log 2023-10-16T11:28:37.724 INFO:teuthology.orchestra.run.smithi084.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.120832.log.gz 2023-10-16T11:28:37.724 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.104188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162357.log 2023-10-16T11:28:37.724 INFO:teuthology.orchestra.run.smithi084.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.104188.log.gz 2023-10-16T11:28:37.725 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.103307.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103307.log.gz 2023-10-16T11:28:37.725 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96764.log 2023-10-16T11:28:37.725 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.162357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96680.log 2023-10-16T11:28:37.726 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.162357.log.gz 2023-10-16T11:28:37.726 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.96764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96806.log 2023-10-16T11:28:37.727 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96764.log.gz 2023-10-16T11:28:37.728 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.96680.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96680.log.gz 2023-10-16T11:28:37.728 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154853.log 2023-10-16T11:28:37.728 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.96806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128815.log 2023-10-16T11:28:37.729 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96806.log.gz 2023-10-16T11:28:37.729 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101418.log 2023-10-16T11:28:37.729 INFO:teuthology.orchestra.run.smithi084.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.154853.log.gz 2023-10-16T11:28:37.730 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.128815.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.128815.log.gz 2023-10-16T11:28:37.730 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98633.log 2023-10-16T11:28:37.731 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.101418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155797.log 2023-10-16T11:28:37.732 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101418.log.gz 2023-10-16T11:28:37.732 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95113.log 2023-10-16T11:28:37.732 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98633.log.gz 2023-10-16T11:28:37.733 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155797.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.155797.log.gz 2023-10-16T11:28:37.733 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161372.log 2023-10-16T11:28:37.734 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.95113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115220.log 2023-10-16T11:28:37.734 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95113.log.gz 2023-10-16T11:28:37.735 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155329.log 2023-10-16T11:28:37.735 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.161372.log.gz 2023-10-16T11:28:37.736 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.115220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113987.log 2023-10-16T11:28:37.737 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115220.log.gz 2023-10-16T11:28:37.737 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155329.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.155329.log.gz 2023-10-16T11:28:37.738 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147338.log 2023-10-16T11:28:37.739 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.113987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114104.log 2023-10-16T11:28:37.739 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113987.log.gz 2023-10-16T11:28:37.740 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107532.log 2023-10-16T11:28:37.740 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.147338.log.gz 2023-10-16T11:28:37.741 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.114104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149459.log 2023-10-16T11:28:37.741 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.114104.log.gz 2023-10-16T11:28:37.742 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.107532.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107532.log.gz 2023-10-16T11:28:37.742 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102802.log 2023-10-16T11:28:37.743 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125081.log 2023-10-16T11:28:37.743 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.149459.log.gz 2023-10-16T11:28:37.744 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.102802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130048.log 2023-10-16T11:28:37.744 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102802.log.gz 2023-10-16T11:28:37.745 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125081.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125081.log.gz 2023-10-16T11:28:37.746 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129307.log 2023-10-16T11:28:37.747 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.130048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164207.log 2023-10-16T11:28:37.747 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.130048.log.gz 2023-10-16T11:28:37.748 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130958.log 2023-10-16T11:28:37.748 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.129307.log.gz 2023-10-16T11:28:37.749 INFO:teuthology.orchestra.run.smithi157.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-16T11:28:37.750 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.164207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95365.log 2023-10-16T11:28:37.750 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.164207.log.gz 2023-10-16T11:28:37.751 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.130958.log: gzip -5 88.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.130958.log.gz 2023-10-16T11:28:37.751 INFO:teuthology.orchestra.run.smithi084.stderr: /var/log/ceph/ceph-client.admin.153882.log 2023-10-16T11:28:37.752 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.95365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136818.log 2023-10-16T11:28:37.752 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95365.log.gz 2023-10-16T11:28:37.753 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87265.log 2023-10-16T11:28:37.753 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.153882.log.gz 2023-10-16T11:28:37.754 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92742.log 2023-10-16T11:28:37.754 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.136818.log.gz 2023-10-16T11:28:37.755 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.87265.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.87265.log.gz 2023-10-16T11:28:37.755 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128540.log 2023-10-16T11:28:37.756 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.92742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113817.log 2023-10-16T11:28:37.756 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.92742.log.gz 2023-10-16T11:28:37.757 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.128540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111134.log 2023-10-16T11:28:37.757 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.128540.log.gz 2023-10-16T11:28:37.758 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.113817.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113817.log.gz 2023-10-16T11:28:37.758 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157636.log 2023-10-16T11:28:37.759 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.111134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152324.log 2023-10-16T11:28:37.759 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111134.log.gz 2023-10-16T11:28:37.760 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98003.log 2023-10-16T11:28:37.761 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.157636.log.gz 2023-10-16T11:28:37.761 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161078.log 2023-10-16T11:28:37.762 INFO:teuthology.orchestra.run.smithi084.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.152324.log.gz 2023-10-16T11:28:37.763 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98003.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98003.log.gz 2023-10-16T11:28:37.763 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107957.log 2023-10-16T11:28:37.764 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129630.log 2023-10-16T11:28:37.764 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.161078.log.gz 2023-10-16T11:28:37.765 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.107957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108305.log 2023-10-16T11:28:37.765 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107957.log.gz 2023-10-16T11:28:37.766 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114726.log 2023-10-16T11:28:37.767 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129630.log.gz 2023-10-16T11:28:37.767 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.108305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165048.log 2023-10-16T11:28:37.768 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108305.log.gz 2023-10-16T11:28:37.768 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.114726.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.114726.log.gz -5 2023-10-16T11:28:37.769 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.135833.log 2023-10-16T11:28:37.769 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.165048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139929.log 2023-10-16T11:28:37.770 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.165048.log.gz 2023-10-16T11:28:37.771 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155245.log 2023-10-16T11:28:37.771 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.135833.log.gz 2023-10-16T11:28:37.772 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139929.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.139929.log.gz 2023-10-16T11:28:37.772 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157426.log 2023-10-16T11:28:37.773 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147914.log 2023-10-16T11:28:37.774 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155245.log.gz 2023-10-16T11:28:37.774 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157426.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.157426.log.gz 2023-10-16T11:28:37.775 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115659.log 2023-10-16T11:28:37.775 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155117.log 2023-10-16T11:28:37.776 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.147914.log.gz 2023-10-16T11:28:37.777 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.115659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128994.log 2023-10-16T11:28:37.777 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115659.log.gz 2023-10-16T11:28:37.778 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155117.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.155117.log.gz -5 2023-10-16T11:28:37.778 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.146604.log 2023-10-16T11:28:37.779 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.128994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135665.log 2023-10-16T11:28:37.779 INFO:teuthology.orchestra.run.smithi084.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.128994.log.gz 2023-10-16T11:28:37.780 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93517.log 2023-10-16T11:28:37.781 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.146604.log.gz 2023-10-16T11:28:37.781 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117834.log 2023-10-16T11:28:37.782 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.135665.log.gz 2023-10-16T11:28:37.783 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93517.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93517.log.gz 2023-10-16T11:28:37.783 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96848.log 2023-10-16T11:28:37.784 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.117834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158686.log 2023-10-16T11:28:37.784 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117834.log.gz 2023-10-16T11:28:37.785 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.96848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161330.log 2023-10-16T11:28:37.785 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96848.log.gz 2023-10-16T11:28:37.786 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158686.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.158686.log.gz 2023-10-16T11:28:37.786 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148772.log 2023-10-16T11:28:37.787 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102886.log 2023-10-16T11:28:37.787 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161330.log.gz 2023-10-16T11:28:37.788 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120237.log 2023-10-16T11:28:37.788 INFO:teuthology.orchestra.run.smithi084.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.148772.log.gz 2023-10-16T11:28:37.789 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.102886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149842.log 2023-10-16T11:28:37.790 INFO:teuthology.orchestra.run.smithi084.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.102886.log.gz 2023-10-16T11:28:37.790 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.120237.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120237.log.gz 2023-10-16T11:28:37.791 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153141.log 2023-10-16T11:28:37.792 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99910.log 2023-10-16T11:28:37.792 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.149842.log.gz 2023-10-16T11:28:37.792 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111996.log 2023-10-16T11:28:37.793 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.153141.log.gz 2023-10-16T11:28:37.794 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152889.log 2023-10-16T11:28:37.794 INFO:teuthology.orchestra.run.smithi084.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99910.log.gz 2023-10-16T11:28:37.795 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.111996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112519.log 2023-10-16T11:28:37.795 INFO:teuthology.orchestra.run.smithi084.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111996.log.gz 2023-10-16T11:28:37.796 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131001.log 2023-10-16T11:28:37.796 INFO:teuthology.orchestra.run.smithi084.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.152889.log.gz 2023-10-16T11:28:37.797 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.112519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125531.log 2023-10-16T11:28:37.798 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112519.log.gz 2023-10-16T11:28:37.798 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.131001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100781.log 2023-10-16T11:28:37.799 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.131001.log.gz 2023-10-16T11:28:37.799 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113354.log 87.7% 2023-10-16T11:28:37.800 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.125531.log.gz 2023-10-16T11:28:37.801 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.100781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138168.log 2023-10-16T11:28:37.801 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100781.log.gz 2023-10-16T11:28:37.802 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.113354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113270.log 2023-10-16T11:28:37.802 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113354.log.gz 2023-10-16T11:28:37.803 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126642.log 2023-10-16T11:28:37.803 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.138168.log.gz 2023-10-16T11:28:37.804 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.113270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109016.log 2023-10-16T11:28:37.804 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113270.log.gz 2023-10-16T11:28:37.805 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.126642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143713.log 2023-10-16T11:28:37.805 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.126642.log.gz 2023-10-16T11:28:37.806 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.109016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101858.log 2023-10-16T11:28:37.806 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109016.log.gz 2023-10-16T11:28:37.808 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143713.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.143713.log.gz 2023-10-16T11:28:37.808 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130520.log 2023-10-16T11:28:37.809 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.101858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108041.log 2023-10-16T11:28:37.809 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101858.log.gz 2023-10-16T11:28:37.810 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.130520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125699.log 2023-10-16T11:28:37.811 INFO:teuthology.orchestra.run.smithi084.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.130520.log.gz 2023-10-16T11:28:37.811 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.108041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98345.log 2023-10-16T11:28:37.812 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108041.log.gz 2023-10-16T11:28:37.812 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125699.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125699.log.gz 2023-10-16T11:28:37.813 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139709.log 2023-10-16T11:28:37.814 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156129.log 2023-10-16T11:28:37.814 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98345.log.gz 2023-10-16T11:28:37.815 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155203.log 2023-10-16T11:28:37.815 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.139709.log.gz 2023-10-16T11:28:37.816 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123840.log 2023-10-16T11:28:37.816 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.156129.log.gz 2023-10-16T11:28:37.817 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127866.log 2023-10-16T11:28:37.817 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.155203.log.gz 2023-10-16T11:28:37.818 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.123840.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.123840.log.gz 2023-10-16T11:28:37.819 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97268.log 2023-10-16T11:28:37.820 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.127866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116578.log 2023-10-16T11:28:37.820 INFO:teuthology.orchestra.run.smithi084.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.127866.log.gz 2023-10-16T11:28:37.821 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157342.log 2023-10-16T11:28:37.821 INFO:teuthology.orchestra.run.smithi084.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97268.log.gz 2023-10-16T11:28:37.822 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.116578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122217.log 2023-10-16T11:28:37.822 INFO:teuthology.orchestra.run.smithi084.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116578.log.gz 2023-10-16T11:28:37.823 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157342.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.157342.log.gz 2023-10-16T11:28:37.824 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123549.log 2023-10-16T11:28:37.824 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121000.log 2023-10-16T11:28:37.825 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.122217.log.gz 2023-10-16T11:28:37.826 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.123549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147830.log 2023-10-16T11:28:37.826 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.123549.log.gz 2023-10-16T11:28:37.827 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121000.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.121000.log.gz 2023-10-16T11:28:37.827 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154354.log 2023-10-16T11:28:37.828 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126474.log 2023-10-16T11:28:37.828 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.147830.log.gz 2023-10-16T11:28:37.829 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162483.log 2023-10-16T11:28:37.829 INFO:teuthology.orchestra.run.smithi084.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.154354.log.gz 2023-10-16T11:28:37.830 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.126474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156606.log 2023-10-16T11:28:37.831 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.126474.log.gz 2023-10-16T11:28:37.831 INFO:teuthology.orchestra.run.smithi157.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-16T11:28:37.832 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.162483.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.162483.log.gz 2023-10-16T11:28:37.832 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163660.log 2023-10-16T11:28:37.833 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152072.log 2023-10-16T11:28:37.833 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.156606.log.gz 2023-10-16T11:28:37.834 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.163660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162273.log 2023-10-16T11:28:37.834 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.163660.log.gz 2023-10-16T11:28:37.835 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144447.log 2023-10-16T11:28:37.835 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.152072.log.gz 2023-10-16T11:28:37.836 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.162273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163986.log 2023-10-16T11:28:37.837 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.162273.log.gz 2023-10-16T11:28:37.837 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144447.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.144447.log.gz -5 2023-10-16T11:28:37.838 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.106029.log 2023-10-16T11:28:37.838 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.163986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143525.log 2023-10-16T11:28:37.839 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.163986.log.gz 2023-10-16T11:28:37.839 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.106029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133521.log 2023-10-16T11:28:37.840 INFO:teuthology.orchestra.run.smithi084.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106029.log.gz 2023-10-16T11:28:37.841 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143525.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.143525.log.gz 2023-10-16T11:28:37.841 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147464.log 2023-10-16T11:28:37.842 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.133521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140929.log 2023-10-16T11:28:37.842 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.133521.log.gz 2023-10-16T11:28:37.843 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.162820.log 2023-10-16T11:28:37.843 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.147464.log.gz 2023-10-16T11:28:37.844 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.140929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147506.log 2023-10-16T11:28:37.844 INFO:teuthology.orchestra.run.smithi084.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.140929.log.gz 2023-10-16T11:28:37.845 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.162820.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.162820.log.gz 2023-10-16T11:28:37.845 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108517.log 2023-10-16T11:28:37.846 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130098.log 2023-10-16T11:28:37.847 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.147506.log.gz 2023-10-16T11:28:37.847 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.108517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138032.log 2023-10-16T11:28:37.848 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108517.log.gz 2023-10-16T11:28:37.848 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.130098.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.130098.log.gz 2023-10-16T11:28:37.849 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.164070.log 2023-10-16T11:28:37.850 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99278.log 2023-10-16T11:28:37.850 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.138032.log.gz 2023-10-16T11:28:37.851 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.164070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128698.log 2023-10-16T11:28:37.851 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.164070.log.gz 2023-10-16T11:28:37.852 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99278.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99278.log.gz 2023-10-16T11:28:37.852 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94727.log 2023-10-16T11:28:37.853 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.128698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114915.log 2023-10-16T11:28:37.853 INFO:teuthology.orchestra.run.smithi084.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.128698.log.gz 2023-10-16T11:28:37.854 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115041.log 2023-10-16T11:28:37.855 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94727.log.gz 2023-10-16T11:28:37.855 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.114915.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114915.log.gz 2023-10-16T11:28:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165132.log 2023-10-16T11:28:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.115041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139845.log 2023-10-16T11:28:37.857 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115041.log.gz 2023-10-16T11:28:37.858 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.165132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142991.log 2023-10-16T11:28:37.858 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.165132.log.gz 2023-10-16T11:28:37.859 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136903.log 2023-10-16T11:28:37.859 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139845.log.gz 2023-10-16T11:28:37.860 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157720.log 2023-10-16T11:28:37.860 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.142991.log.gz 2023-10-16T11:28:37.861 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136903.log: gzip -5 88.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.136903.log.gz /var/log/ceph/ceph-client.admin.98507.log 2023-10-16T11:28:37.861 INFO:teuthology.orchestra.run.smithi084.stderr: 2023-10-16T11:28:37.862 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120874.log 2023-10-16T11:28:37.866 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.157720.log.gz 2023-10-16T11:28:37.866 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161582.log 2023-10-16T11:28:37.867 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98507.log.gz 2023-10-16T11:28:37.867 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.120874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115083.log 2023-10-16T11:28:37.867 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120874.log.gz 2023-10-16T11:28:37.868 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162051.log 2023-10-16T11:28:37.868 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.161582.log.gz 2023-10-16T11:28:37.869 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.115083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93475.log 2023-10-16T11:28:37.869 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115083.log.gz 2023-10-16T11:28:37.870 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.162051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121954.log 2023-10-16T11:28:37.870 INFO:teuthology.orchestra.run.smithi084.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.162051.log.gz 2023-10-16T11:28:37.870 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93475.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93475.log.gz 2023-10-16T11:28:37.871 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139028.log 2023-10-16T11:28:37.871 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121954.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.121954.log.gz 2023-10-16T11:28:37.872 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147096.log 2023-10-16T11:28:37.872 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139028.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.139028.log.gz 2023-10-16T11:28:37.873 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.119677.log 2023-10-16T11:28:37.873 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147096.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.147096.log.gz 2023-10-16T11:28:37.874 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159662.log 2023-10-16T11:28:37.874 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119677.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119677.log.gz 2023-10-16T11:28:37.875 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126432.log 2023-10-16T11:28:37.875 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.159662.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.159662.log.gz 2023-10-16T11:28:37.876 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92508.log 2023-10-16T11:28:37.877 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.126432.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.126432.log.gz 2023-10-16T11:28:37.877 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118480.log 2023-10-16T11:28:37.878 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.92508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152973.log 88.8% 2023-10-16T11:28:37.878 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.92508.log.gz 2023-10-16T11:28:37.879 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114453.log 2023-10-16T11:28:37.879 INFO:teuthology.orchestra.run.smithi084.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118480.log.gz 2023-10-16T11:28:37.880 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152973.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152973.log.gz 2023-10-16T11:28:37.880 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119509.log 2023-10-16T11:28:37.881 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.114453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108083.log 2023-10-16T11:28:37.881 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114453.log.gz 2023-10-16T11:28:37.882 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119509.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.119509.log.gz 2023-10-16T11:28:37.882 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.124766.log 2023-10-16T11:28:37.883 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.108083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102718.log 2023-10-16T11:28:37.883 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108083.log.gz 2023-10-16T11:28:37.884 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.124766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158478.log 2023-10-16T11:28:37.884 INFO:teuthology.orchestra.run.smithi084.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.124766.log.gz 2023-10-16T11:28:37.885 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.102718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145497.log 2023-10-16T11:28:37.885 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102718.log.gz 2023-10-16T11:28:37.886 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158478.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.158478.log.gz 2023-10-16T11:28:37.886 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157510.log 2023-10-16T11:28:37.887 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94019.log 2023-10-16T11:28:37.888 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.145497.log.gz 2023-10-16T11:28:37.888 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161965.log 2023-10-16T11:28:37.888 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.157510.log.gz 2023-10-16T11:28:37.889 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161456.log 2023-10-16T11:28:37.889 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94019.log.gz 2023-10-16T11:28:37.890 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161965.log: gzip -5 88.8% --verbose -- replaced with /var/log/ceph/ceph-client.admin.161965.log.gz -- 2023-10-16T11:28:37.890 INFO:teuthology.orchestra.run.smithi084.stderr: /var/log/ceph/ceph-client.admin.97520.log 2023-10-16T11:28:37.891 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149968.log 2023-10-16T11:28:37.891 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.161456.log.gz 2023-10-16T11:28:37.892 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116232.log 2023-10-16T11:28:37.892 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97520.log.gz 2023-10-16T11:28:37.893 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95323.log 2023-10-16T11:28:37.894 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.149968.log.gz 2023-10-16T11:28:37.894 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.116232.log: gzip -5 87.9% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.116232.log.gz 2023-10-16T11:28:37.894 INFO:teuthology.orchestra.run.smithi084.stderr: /var/log/ceph/ceph-client.admin.164310.log 2023-10-16T11:28:37.895 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.95323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140756.log 2023-10-16T11:28:37.895 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95323.log.gz 2023-10-16T11:28:37.896 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.164310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143357.log 2023-10-16T11:28:37.897 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.164310.log.gz 2023-10-16T11:28:37.897 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.140756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108559.log 2023-10-16T11:28:37.897 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.140756.log.gz 2023-10-16T11:28:37.898 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143357.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143357.log.gz 2023-10-16T11:28:37.899 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149026.log 2023-10-16T11:28:37.899 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.108559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135707.log 2023-10-16T11:28:37.900 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108559.log.gz 2023-10-16T11:28:37.901 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149026.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.149026.log.gz 2023-10-16T11:28:37.901 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151977.log 2023-10-16T11:28:37.902 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135707.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.135707.log.gz 2023-10-16T11:28:37.902 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152469.log 2023-10-16T11:28:37.903 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101643.log 2023-10-16T11:28:37.903 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.151977.log.gz 2023-10-16T11:28:37.904 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136472.log 2023-10-16T11:28:37.904 INFO:teuthology.orchestra.run.smithi084.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.152469.log.gz 2023-10-16T11:28:37.905 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.101643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.140452.log 2023-10-16T11:28:37.905 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101643.log.gz 2023-10-16T11:28:37.906 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144096.log 2023-10-16T11:28:37.907 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.136472.log.gz 2023-10-16T11:28:37.907 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.140452.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.140452.log.gz 2023-10-16T11:28:37.907 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165300.log 2023-10-16T11:28:37.908 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144096.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.144096.log.gz 2023-10-16T11:28:37.908 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140638.log 2023-10-16T11:28:37.909 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.165300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143212.log 2023-10-16T11:28:37.909 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.165300.log.gz 2023-10-16T11:28:37.910 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.140638.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.140638.log.gz 2023-10-16T11:28:37.911 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118233.log 2023-10-16T11:28:37.912 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143212.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.143212.log.gz 2023-10-16T11:28:37.912 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94580.log 2023-10-16T11:28:37.912 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118233.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118233.log.gz 2023-10-16T11:28:37.913 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113226.log 2023-10-16T11:28:37.913 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94580.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94580.log.gz 2023-10-16T11:28:37.914 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127743.log 2023-10-16T11:28:37.915 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.113226.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113226.log.gz 2023-10-16T11:28:37.915 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156501.log 2023-10-16T11:28:37.916 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.127743.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.127743.log.gz 2023-10-16T11:28:37.916 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128108.log 2023-10-16T11:28:37.917 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156501.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.156501.log.gz 2023-10-16T11:28:37.917 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122420.log 2023-10-16T11:28:37.918 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.128108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94945.log 2023-10-16T11:28:37.918 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128108.log.gz 2023-10-16T11:28:37.919 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122420.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.122420.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111439.log 2023-10-16T11:28:37.919 INFO:teuthology.orchestra.run.smithi084.stderr: 2023-10-16T11:28:37.920 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144224.log 2023-10-16T11:28:37.920 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94945.log.gz 2023-10-16T11:28:37.921 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.111439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108601.log 2023-10-16T11:28:37.921 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111439.log.gz 2023-10-16T11:28:37.922 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153692.log 2023-10-16T11:28:37.922 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.144224.log.gz 2023-10-16T11:28:37.923 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.108601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96722.log 2023-10-16T11:28:37.924 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153692.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108601.log.gz 2023-10-16T11:28:37.924 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.153692.log.gz 2023-10-16T11:28:37.925 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142571.log 2023-10-16T11:28:37.925 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.96722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137906.log 2023-10-16T11:28:37.926 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96722.log.gz 2023-10-16T11:28:37.926 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135299.log 2023-10-16T11:28:37.927 INFO:teuthology.orchestra.run.smithi084.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142571.log.gz 2023-10-16T11:28:37.928 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117876.log 2023-10-16T11:28:37.928 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137906.log.gz 2023-10-16T11:28:37.929 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135299.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.135299.log.gz 2023-10-16T11:28:37.929 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100330.log 2023-10-16T11:28:37.930 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.117876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136650.log 2023-10-16T11:28:37.930 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.117876.log.gz 2023-10-16T11:28:37.931 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.100330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98219.log 2023-10-16T11:28:37.931 INFO:teuthology.orchestra.run.smithi084.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.100330.log.gz 2023-10-16T11:28:37.932 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136650.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.136650.log.gz 2023-10-16T11:28:37.932 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127948.log 2023-10-16T11:28:37.933 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117603.log 2023-10-16T11:28:37.934 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98219.log.gz 2023-10-16T11:28:37.935 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.127948.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.127948.log.gz 2023-10-16T11:28:37.935 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103248.log 2023-10-16T11:28:37.936 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.117603.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.117603.log.gz 2023-10-16T11:28:37.936 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106999.log 2023-10-16T11:28:37.937 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.103248.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103248.log.gz 2023-10-16T11:28:37.937 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157029.log 2023-10-16T11:28:37.938 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.106999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98549.log 2023-10-16T11:28:37.939 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106999.log.gz 2023-10-16T11:28:37.940 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157029.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.157029.log.gz 2023-10-16T11:28:37.940 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137787.log 2023-10-16T11:28:37.941 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98549.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98549.log.gz 2023-10-16T11:28:37.941 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145114.log 2023-10-16T11:28:37.942 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94690.log 2023-10-16T11:28:37.943 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137787.log.gz 2023-10-16T11:28:37.943 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145114.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.145114.log.gz -5 2023-10-16T11:28:37.944 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.121701.log 2023-10-16T11:28:37.944 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156987.log 2023-10-16T11:28:37.945 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94690.log.gz 2023-10-16T11:28:37.945 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136257.log 2023-10-16T11:28:37.946 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.121701.log.gz 2023-10-16T11:28:37.947 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126124.log 2023-10-16T11:28:37.947 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.156987.log.gz 2023-10-16T11:28:37.947 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136257.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.136257.log.gz 2023-10-16T11:28:37.948 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103425.log 2023-10-16T11:28:37.949 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.126124.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.126124.log.gz 2023-10-16T11:28:37.949 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.101686.log 2023-10-16T11:28:37.950 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.103425.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103425.log.gz 2023-10-16T11:28:37.950 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143755.log 2023-10-16T11:28:37.951 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.101686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157239.log 2023-10-16T11:28:37.952 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101686.log.gz 2023-10-16T11:28:37.953 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143755.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.143755.log.gz 2023-10-16T11:28:37.953 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113438.log 2023-10-16T11:28:37.954 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157239.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.157239.log.gz 2023-10-16T11:28:37.954 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104451.log 2023-10-16T11:28:37.955 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.113438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154102.log 2023-10-16T11:28:37.955 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.113438.log.gz 2023-10-16T11:28:37.956 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.104451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114673.log 2023-10-16T11:28:37.957 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104451.log.gz 2023-10-16T11:28:37.958 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154102.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.154102.log.gz 2023-10-16T11:28:37.958 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107178.log 2023-10-16T11:28:37.959 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.114673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93698.log 2023-10-16T11:28:37.959 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114673.log.gz 2023-10-16T11:28:37.960 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.107178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125405.log 2023-10-16T11:28:37.960 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107178.log.gz 2023-10-16T11:28:37.961 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125657.log 2023-10-16T11:28:37.961 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93698.log.gz 2023-10-16T11:28:37.963 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93349.log 2023-10-16T11:28:37.963 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.125405.log.gz 2023-10-16T11:28:37.964 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125657.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125657.log.gz 2023-10-16T11:28:37.964 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158601.log 2023-10-16T11:28:37.965 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93349.log: gzip -5 --verbose 87.5% -- -- replaced with /var/log/ceph/ceph-client.admin.93349.log.gz /var/log/ceph/ceph-client.admin.153932.log 2023-10-16T11:28:37.965 INFO:teuthology.orchestra.run.smithi084.stderr: 2023-10-16T11:28:37.966 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158601.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.158601.log.gz 2023-10-16T11:28:37.966 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.122897.log 2023-10-16T11:28:37.967 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153932.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.153932.log.gz 2023-10-16T11:28:37.968 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97058.log 2023-10-16T11:28:37.968 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106726.log 2023-10-16T11:28:37.969 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.122897.log.gz 2023-10-16T11:28:37.970 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97058.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97058.log.gz 2023-10-16T11:28:37.970 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159355.log 2023-10-16T11:28:37.971 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.106726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118775.log 2023-10-16T11:28:37.972 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106726.log.gz 2023-10-16T11:28:37.972 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.159355.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159355.log.gz 2023-10-16T11:28:37.973 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136428.log 2023-10-16T11:28:37.974 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118775.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.118775.log.gz 2023-10-16T11:28:37.974 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126082.log 2023-10-16T11:28:37.975 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121557.log 2023-10-16T11:28:37.975 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.136428.log.gz 2023-10-16T11:28:37.976 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.126082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117645.log 88.7% 2023-10-16T11:28:37.976 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.126082.log.gz 2023-10-16T11:28:37.977 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143627.log 2023-10-16T11:28:37.977 INFO:teuthology.orchestra.run.smithi084.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.121557.log.gz 2023-10-16T11:28:37.978 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.117645.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117645.log.gz 2023-10-16T11:28:37.978 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152847.log 2023-10-16T11:28:37.979 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107220.log 2023-10-16T11:28:37.980 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.143627.log.gz 2023-10-16T11:28:37.980 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154016.log 2023-10-16T11:28:37.980 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.152847.log.gz 2023-10-16T11:28:37.981 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.107220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118438.log 2023-10-16T11:28:37.982 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107220.log.gz 2023-10-16T11:28:37.982 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154016.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.154016.log.gz -5 2023-10-16T11:28:37.982 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.94726.log 2023-10-16T11:28:37.983 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160098.log 2023-10-16T11:28:37.984 INFO:teuthology.orchestra.run.smithi084.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.118438.log.gz 2023-10-16T11:28:37.984 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147643.log 2023-10-16T11:28:37.985 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94726.log.gz 2023-10-16T11:28:37.985 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.160098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102970.log 2023-10-16T11:28:37.986 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.160098.log.gz 2023-10-16T11:28:37.987 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162095.log 2023-10-16T11:28:37.987 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.147643.log.gz 2023-10-16T11:28:37.988 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.102970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95281.log 2023-10-16T11:28:37.988 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102970.log.gz 2023-10-16T11:28:37.989 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.162095.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.162095.log.gz 2023-10-16T11:28:37.989 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98423.log 2023-10-16T11:28:37.990 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.95281.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95281.log.gz 2023-10-16T11:28:37.991 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118023.log 2023-10-16T11:28:37.991 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136300.log 2023-10-16T11:28:37.992 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98423.log.gz 2023-10-16T11:28:37.993 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112656.log 2023-10-16T11:28:37.993 INFO:teuthology.orchestra.run.smithi084.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.118023.log.gz 2023-10-16T11:28:37.994 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151127.log 2023-10-16T11:28:37.994 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136300.log.gz 2023-10-16T11:28:37.995 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.112656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157594.log 2023-10-16T11:28:37.995 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.112656.log.gz 2023-10-16T11:28:37.996 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151127.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.151127.log.gz 2023-10-16T11:28:37.996 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165342.log 2023-10-16T11:28:37.997 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157594.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.157594.log.gz 2023-10-16T11:28:37.997 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164796.log 2023-10-16T11:28:37.998 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.165342.log: gzip -5 87.3% --verbose -- replaced with /var/log/ceph/ceph-client.admin.165342.log.gz -- 2023-10-16T11:28:37.998 INFO:teuthology.orchestra.run.smithi084.stderr: /var/log/ceph/ceph-client.0.144994.log 2023-10-16T11:28:37.999 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.164796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164964.log 2023-10-16T11:28:37.999 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.164796.log.gz 2023-10-16T11:28:38.000 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.144994.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.144994.log.gz 2023-10-16T11:28:38.001 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160983.log 2023-10-16T11:28:38.002 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.164964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140157.log 2023-10-16T11:28:38.002 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.164964.log.gz 2023-10-16T11:28:38.003 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.160983.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.160983.log.gz 2023-10-16T11:28:38.003 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156249.log 2023-10-16T11:28:38.004 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.140157.log: gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.140157.log.gz --verbose 2023-10-16T11:28:38.004 INFO:teuthology.orchestra.run.smithi084.stderr: -- /var/log/ceph/ceph-client.admin.149068.log 2023-10-16T11:28:38.005 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154186.log 2023-10-16T11:28:38.005 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.156249.log.gz 2023-10-16T11:28:38.006 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148638.log 2023-10-16T11:28:38.007 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.149068.log.gz 2023-10-16T11:28:38.007 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158117.log 2023-10-16T11:28:38.008 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.154186.log.gz 2023-10-16T11:28:38.008 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148638.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.148638.log.gz 2023-10-16T11:28:38.009 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129036.log 2023-10-16T11:28:38.009 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101358.log 2023-10-16T11:28:38.010 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.158117.log.gz 2023-10-16T11:28:38.011 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144699.log 2023-10-16T11:28:38.011 INFO:teuthology.orchestra.run.smithi084.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.129036.log.gz 2023-10-16T11:28:38.012 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.101358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148083.log 2023-10-16T11:28:38.012 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101358.log.gz 2023-10-16T11:28:38.013 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152763.log 2023-10-16T11:28:38.013 INFO:teuthology.orchestra.run.smithi084.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.144699.log.gz 2023-10-16T11:28:38.014 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136860.log 2023-10-16T11:28:38.014 INFO:teuthology.orchestra.run.smithi084.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.148083.log.gz 2023-10-16T11:28:38.015 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138860.log 2023-10-16T11:28:38.015 INFO:teuthology.orchestra.run.smithi084.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.152763.log.gz 2023-10-16T11:28:38.016 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129876.log 2023-10-16T11:28:38.017 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.136860.log.gz 2023-10-16T11:28:38.017 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157468.log 2023-10-16T11:28:38.017 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.138860.log.gz 2023-10-16T11:28:38.018 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152156.log 2023-10-16T11:28:38.019 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.129876.log.gz 2023-10-16T11:28:38.019 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164352.log 2023-10-16T11:28:38.019 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.157468.log.gz 2023-10-16T11:28:38.020 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152553.log 2023-10-16T11:28:38.020 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.152156.log.gz 2023-10-16T11:28:38.021 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.164352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113396.log 2023-10-16T11:28:38.021 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164352.log.gz 2023-10-16T11:28:38.022 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125134.log 2023-10-16T11:28:38.022 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.152553.log.gz 2023-10-16T11:28:38.023 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.113396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155916.log 2023-10-16T11:28:38.023 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113396.log.gz 2023-10-16T11:28:38.024 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147872.log 2023-10-16T11:28:38.024 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125134.log.gz 2023-10-16T11:28:38.025 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155916.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.155916.log.gz 2023-10-16T11:28:38.026 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151186.log 2023-10-16T11:28:38.027 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147872.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.147872.log.gz 2023-10-16T11:28:38.027 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105503.log 2023-10-16T11:28:38.028 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129833.log 2023-10-16T11:28:38.028 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.151186.log.gz 2023-10-16T11:28:38.029 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.105503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125489.log 2023-10-16T11:28:38.029 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105503.log.gz 2023-10-16T11:28:38.030 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129833.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129833.log.gz -5 2023-10-16T11:28:38.030 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.119803.log 2023-10-16T11:28:38.031 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97352.log 2023-10-16T11:28:38.032 INFO:teuthology.orchestra.run.smithi084.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.125489.log.gz 2023-10-16T11:28:38.033 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120337.log 2023-10-16T11:28:38.033 INFO:teuthology.orchestra.run.smithi084.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.119803.log.gz 2023-10-16T11:28:38.034 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135917.log 2023-10-16T11:28:38.034 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97352.log.gz 2023-10-16T11:28:38.035 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.120337.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.120337.log.gz 2023-10-16T11:28:38.035 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152282.log 2023-10-16T11:28:38.036 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137276.log 2023-10-16T11:28:38.037 INFO:teuthology.orchestra.run.smithi084.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.135917.log.gz 2023-10-16T11:28:38.037 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103189.log 2023-10-16T11:28:38.038 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152282.log.gz 2023-10-16T11:28:38.039 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137276.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.137276.log.gz 2023-10-16T11:28:38.039 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136566.log 2023-10-16T11:28:38.040 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.103189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130478.log 2023-10-16T11:28:38.040 INFO:teuthology.orchestra.run.smithi084.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.103189.log.gz 2023-10-16T11:28:38.041 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165510.log 2023-10-16T11:28:38.041 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.136566.log.gz 2023-10-16T11:28:38.042 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.130478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112435.log 2023-10-16T11:28:38.042 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.130478.log.gz 2023-10-16T11:28:38.043 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.165510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116016.log 2023-10-16T11:28:38.043 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.165510.log.gz 2023-10-16T11:28:38.044 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.112435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101298.log 2023-10-16T11:28:38.044 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112435.log.gz 2023-10-16T11:28:38.045 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.116016.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116016.log.gz 2023-10-16T11:28:38.046 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159121.log 2023-10-16T11:28:38.046 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.101298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103012.log 2023-10-16T11:28:38.047 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101298.log.gz 2023-10-16T11:28:38.047 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.159121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129791.log 2023-10-16T11:28:38.048 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.159121.log.gz 2023-10-16T11:28:38.049 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.103012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124874.log 2023-10-16T11:28:38.049 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103012.log.gz 2023-10-16T11:28:38.050 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122505.log 2023-10-16T11:28:38.050 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.129791.log.gz 2023-10-16T11:28:38.051 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.124874.log: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.124874.log.gz 2023-10-16T11:28:38.051 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.155075.log 2023-10-16T11:28:38.052 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99780.log 2023-10-16T11:28:38.053 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122505.log.gz 2023-10-16T11:28:38.053 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108263.log 2023-10-16T11:28:38.054 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.155075.log.gz 2023-10-16T11:28:38.054 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143296.log 2023-10-16T11:28:38.055 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99780.log.gz 2023-10-16T11:28:38.055 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.108263.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108263.log.gz 2023-10-16T11:28:38.056 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141877.log 2023-10-16T11:28:38.056 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124535.log 2023-10-16T11:28:38.057 INFO:teuthology.orchestra.run.smithi084.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143296.log.gz 2023-10-16T11:28:38.058 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141877.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141877.log.gz 2023-10-16T11:28:38.058 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107262.log 2023-10-16T11:28:38.059 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.124535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105240.log 2023-10-16T11:28:38.059 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124535.log.gz 2023-10-16T11:28:38.060 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.107262.log: gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.107262.log.gz --verbose 2023-10-16T11:28:38.060 INFO:teuthology.orchestra.run.smithi084.stderr: -- /var/log/ceph/ceph-client.admin.120528.log 2023-10-16T11:28:38.061 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.105240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148041.log 2023-10-16T11:28:38.061 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.105240.log.gz 2023-10-16T11:28:38.062 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.120528.log: gzip -5 87.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.120528.log.gz -- 2023-10-16T11:28:38.063 INFO:teuthology.orchestra.run.smithi084.stderr: /var/log/ceph/ceph-client.admin.103925.log 2023-10-16T11:28:38.063 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148041.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.148041.log.gz 2023-10-16T11:28:38.064 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113522.log 2023-10-16T11:28:38.065 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.103925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112225.log 2023-10-16T11:28:38.065 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103925.log.gz 2023-10-16T11:28:38.066 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.113522.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113522.log.gz 2023-10-16T11:28:38.066 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.159542.log 2023-10-16T11:28:38.067 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.112225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140031.log 2023-10-16T11:28:38.067 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.112225.log.gz 2023-10-16T11:28:38.068 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.159542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.159542.log.gz 2023-10-16T11:28:38.068 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.149363.log 2023-10-16T11:28:38.069 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.140031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160668.log 2023-10-16T11:28:38.069 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140031.log.gz 2023-10-16T11:28:38.070 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.149363.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.149363.log.gz 2023-10-16T11:28:38.073 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97436.log 2023-10-16T11:28:38.075 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.160668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150454.log 88.9% 2023-10-16T11:28:38.075 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.160668.log.gz 2023-10-16T11:28:38.075 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99612.log 2023-10-16T11:28:38.075 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97436.log.gz 2023-10-16T11:28:38.076 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150454.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.150454.log.gz 2023-10-16T11:28:38.076 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124671.log 2023-10-16T11:28:38.076 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115719.log 2023-10-16T11:28:38.076 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99612.log.gz 2023-10-16T11:28:38.078 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.124671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149884.log 2023-10-16T11:28:38.078 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124671.log.gz 2023-10-16T11:28:38.078 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.115719.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115719.log.gz 2023-10-16T11:28:38.078 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102634.log 2023-10-16T11:28:38.079 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149884.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.149884.log.gz 2023-10-16T11:28:38.079 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160941.log 2023-10-16T11:28:38.080 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.102634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99994.log 2023-10-16T11:28:38.080 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102634.log.gz 2023-10-16T11:28:38.081 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.160941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158559.log 2023-10-16T11:28:38.081 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.160941.log.gz 2023-10-16T11:28:38.081 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157907.log 2023-10-16T11:28:38.081 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99994.log.gz 2023-10-16T11:28:38.082 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158559.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.158559.log.gz 2023-10-16T11:28:38.083 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157762.log 2023-10-16T11:28:38.083 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162441.log 2023-10-16T11:28:38.084 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.157907.log.gz 2023-10-16T11:28:38.084 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157762.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.157762.log.gz 2023-10-16T11:28:38.085 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165006.log 2023-10-16T11:28:38.085 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.162441.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.162441.log.gz 2023-10-16T11:28:38.086 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129139.log 2023-10-16T11:28:38.086 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.165006.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.165006.log.gz 2023-10-16T11:28:38.087 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138818.log 2023-10-16T11:28:38.087 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129139.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129139.log.gz 2023-10-16T11:28:38.088 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-16T11:28:38.089 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115491.log 2023-10-16T11:28:38.089 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138818.log.gz 2023-10-16T11:28:38.089 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108751.log 2023-10-16T11:28:38.090 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.115491.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115491.log.gz 2023-10-16T11:28:38.091 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142613.log 2023-10-16T11:28:38.092 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.108751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118733.log 2023-10-16T11:28:38.092 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108751.log.gz/var/log/ceph/ceph-client.admin.142613.log: 2023-10-16T11:28:38.093 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.142613.log.gz 2023-10-16T11:28:38.093 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130893.log 2023-10-16T11:28:38.095 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158869.log 2023-10-16T11:28:38.095 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118733.log.gz 90.0% 2023-10-16T11:28:38.095 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-16T11:28:38.096 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.130893.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.130893.log.gz 2023-10-16T11:28:38.096 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112038.log 2023-10-16T11:28:38.097 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158869.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.158869.log.gz 2023-10-16T11:28:38.097 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107574.log 2023-10-16T11:28:38.098 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.112038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164028.log 2023-10-16T11:28:38.099 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112038.log.gz 2023-10-16T11:28:38.100 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.107574.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107574.log.gz 2023-10-16T11:28:38.100 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97673.log 2023-10-16T11:28:38.101 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.164028.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.164028.log.gz 2023-10-16T11:28:38.101 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122591.log 2023-10-16T11:28:38.102 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97673.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97673.log.gz 2023-10-16T11:28:38.102 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95071.log 2023-10-16T11:28:38.103 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122591.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122591.log.gz 2023-10-16T11:28:38.103 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158995.log 2023-10-16T11:28:38.104 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.95071.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.95071.log.gz 2023-10-16T11:28:38.104 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.116188.log 2023-10-16T11:28:38.105 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129096.log 2023-10-16T11:28:38.105 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.158995.log.gz 2023-10-16T11:28:38.106 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.116188.log: gzip -5 --verbose 88.8% -- -- replaced with /var/log/ceph/ceph-client.admin.116188.log.gz 2023-10-16T11:28:38.106 INFO:teuthology.orchestra.run.smithi084.stderr: /var/log/ceph/ceph-client.admin.157991.log 2023-10-16T11:28:38.107 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129096.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.129096.log.gz 2023-10-16T11:28:38.108 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151851.log 2023-10-16T11:28:38.108 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122038.log 2023-10-16T11:28:38.109 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.157991.log.gz 2023-10-16T11:28:38.109 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145454.log 2023-10-16T11:28:38.109 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.151851.log.gz 2023-10-16T11:28:38.110 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143033.log 2023-10-16T11:28:38.111 INFO:teuthology.orchestra.run.smithi084.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.122038.log.gz 2023-10-16T11:28:38.111 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144807.log 2023-10-16T11:28:38.112 INFO:teuthology.orchestra.run.smithi084.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.145454.log.gz 2023-10-16T11:28:38.112 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148900.log 2023-10-16T11:28:38.113 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.143033.log.gz 2023-10-16T11:28:38.113 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129349.log 2023-10-16T11:28:38.114 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.144807.log.gz 2023-10-16T11:28:38.114 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94033.log 2023-10-16T11:28:38.115 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.148900.log.gz 2023-10-16T11:28:38.115 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103130.log 2023-10-16T11:28:38.116 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.129349.log.gz 2023-10-16T11:28:38.117 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99486.log 2023-10-16T11:28:38.117 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94033.log.gz 2023-10-16T11:28:38.118 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.103130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127015.log 2023-10-16T11:28:38.118 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103130.log.gz 2023-10-16T11:28:38.119 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99486.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.94364.log -- replaced with /var/log/ceph/ceph-client.admin.99486.log.gz 2023-10-16T11:28:38.119 INFO:teuthology.orchestra.run.smithi084.stderr: 2023-10-16T11:28:38.120 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.127015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148220.log 2023-10-16T11:28:38.120 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.127015.log.gz 2023-10-16T11:28:38.121 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140265.log 2023-10-16T11:28:38.121 INFO:teuthology.orchestra.run.smithi084.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94364.log.gz 2023-10-16T11:28:38.122 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165258.log 2023-10-16T11:28:38.122 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.148220.log.gz 2023-10-16T11:28:38.123 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.140265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128353.log 2023-10-16T11:28:38.123 INFO:teuthology.orchestra.run.smithi084.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.140265.log.gz 2023-10-16T11:28:38.124 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.165258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148508.log 2023-10-16T11:28:38.124 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.165258.log.gz 2023-10-16T11:28:38.125 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.128353.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.128353.log.gz 2023-10-16T11:28:38.125 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98177.log 2023-10-16T11:28:38.126 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122813.log 2023-10-16T11:28:38.126 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.148508.log.gz 2023-10-16T11:28:38.127 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98177.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98177.log.gz 2023-10-16T11:28:38.127 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152679.log 2023-10-16T11:28:38.128 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107136.log 2023-10-16T11:28:38.128 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122813.log.gz 2023-10-16T11:28:38.129 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152679.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.152679.log.gz 2023-10-16T11:28:38.129 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144741.log 2023-10-16T11:28:38.130 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.107136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158033.log 2023-10-16T11:28:38.130 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107136.log.gz 2023-10-16T11:28:38.131 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97757.log 2023-10-16T11:28:38.131 INFO:teuthology.orchestra.run.smithi084.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.144741.log.gz 2023-10-16T11:28:38.132 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158033.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.158033.log.gz 2023-10-16T11:28:38.133 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121336.log 2023-10-16T11:28:38.133 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155731.log 2023-10-16T11:28:38.134 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97757.log.gz 2023-10-16T11:28:38.135 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121336.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.121336.log.gz 2023-10-16T11:28:38.135 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143170.log 2023-10-16T11:28:38.136 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143483.log 2023-10-16T11:28:38.136 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.155731.log.gz 2023-10-16T11:28:38.137 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143170.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.143170.log.gz 2023-10-16T11:28:38.137 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144318.log 2023-10-16T11:28:38.138 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143483.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.143483.log.gz 2023-10-16T11:28:38.138 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.96437.log 2023-10-16T11:28:38.139 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144318.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144318.log.gz 2023-10-16T11:28:38.139 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144615.log 2023-10-16T11:28:38.140 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.2.96437.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.96437.log.gz 2023-10-16T11:28:38.140 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128899.log 2023-10-16T11:28:38.141 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144615.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.144615.log.gz 2023-10-16T11:28:38.142 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160045.log 2023-10-16T11:28:38.142 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.128899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163524.log 2023-10-16T11:28:38.143 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128899.log.gz 2023-10-16T11:28:38.143 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.160045.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.160045.log.gzgzip 2023-10-16T11:28:38.144 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.164123.log 2023-10-16T11:28:38.144 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.163524.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.163524.log.gz 2023-10-16T11:28:38.145 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94685.log 2023-10-16T11:28:38.145 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.164123.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.164123.log.gz 2023-10-16T11:28:38.146 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138221.log 2023-10-16T11:28:38.147 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94685.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94685.log.gz 2023-10-16T11:28:38.147 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137676.log 2023-10-16T11:28:38.148 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138221.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.138221.log.gz 2023-10-16T11:28:38.148 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161757.log 2023-10-16T11:28:38.149 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163482.log 2023-10-16T11:28:38.149 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137676.log.gz 2023-10-16T11:28:38.150 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161757.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.161757.log.gz 2023-10-16T11:28:38.150 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148423.log 2023-10-16T11:28:38.151 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.163482.log: gzip 89.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.163482.log.gz 2023-10-16T11:28:38.151 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.136776.log 2023-10-16T11:28:38.152 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148423.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.148423.log.gz 2023-10-16T11:28:38.152 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151042.log 2023-10-16T11:28:38.153 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136776.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136776.log.gz 2023-10-16T11:28:38.153 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.154607.log 2023-10-16T11:28:38.154 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122547.log 2023-10-16T11:28:38.155 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.151042.log.gz 2023-10-16T11:28:38.155 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.154607.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.154607.log.gz 2023-10-16T11:28:38.156 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122080.log 2023-10-16T11:28:38.156 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122547.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.122547.log.gz -5 2023-10-16T11:28:38.157 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.117077.log 2023-10-16T11:28:38.158 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107430.log 2023-10-16T11:28:38.158 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.122080.log.gz 2023-10-16T11:28:38.159 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.117077.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117077.log.gz 2023-10-16T11:28:38.159 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123083.log 2023-10-16T11:28:38.160 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.107430.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107430.log.gz 2023-10-16T11:28:38.160 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106537.log 2023-10-16T11:28:38.160 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.123083.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123083.log.gz -5 2023-10-16T11:28:38.161 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.115280.log 2023-10-16T11:28:38.162 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.106537.log: gzip -5 --verbose -- 89.0% /var/log/ceph/ceph-client.admin.97394.log -- replaced with /var/log/ceph/ceph-client.admin.106537.log.gz 2023-10-16T11:28:38.162 INFO:teuthology.orchestra.run.smithi084.stderr: 2023-10-16T11:28:38.163 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.115280.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115280.log.gz 2023-10-16T11:28:38.163 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92794.log 2023-10-16T11:28:38.164 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97394.log: gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.97394.log.gz 2023-10-16T11:28:38.164 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.164436.log 2023-10-16T11:28:38.165 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.92794.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92794.log.gz 2023-10-16T11:28:38.165 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92685.log 2023-10-16T11:28:38.166 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.164436.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.140680.log 2023-10-16T11:28:38.166 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.164436.log.gz 2023-10-16T11:28:38.167 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.92685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125615.log 2023-10-16T11:28:38.167 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92685.log.gz 2023-10-16T11:28:38.168 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.140680.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140680.log.gz 2023-10-16T11:28:38.169 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149627.log 2023-10-16T11:28:38.169 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125615.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.125615.log.gz 2023-10-16T11:28:38.170 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148984.log 2023-10-16T11:28:38.170 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158728.log 2023-10-16T11:28:38.171 INFO:teuthology.orchestra.run.smithi084.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.149627.log.gz 2023-10-16T11:28:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148984.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.148984.log.gz 2023-10-16T11:28:38.172 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96570.log 2023-10-16T11:28:38.173 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158728.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.158728.log.gz 2023-10-16T11:28:38.173 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.139887.log 2023-10-16T11:28:38.174 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.96570.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96570.log.gz 2023-10-16T11:28:38.174 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99654.log 2023-10-16T11:28:38.175 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139887.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.139887.log.gz 2023-10-16T11:28:38.176 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147746.log 2023-10-16T11:28:38.176 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99654.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.99654.log.gz 2023-10-16T11:28:38.177 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125218.log 2023-10-16T11:28:38.177 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156903.log 2023-10-16T11:28:38.178 INFO:teuthology.orchestra.run.smithi084.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.147746.log.gz 2023-10-16T11:28:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125218.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.125218.log.gz 2023-10-16T11:28:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112818.log 2023-10-16T11:28:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111628.log 2023-10-16T11:28:38.180 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0%/var/log/ceph/ceph-client.admin.112818.log: -- replaced with /var/log/ceph/ceph-client.admin.156903.log.gz 2023-10-16T11:28:38.181 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112818.log.gz 2023-10-16T11:28:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160461.log 2023-10-16T11:28:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.111628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107999.log 2023-10-16T11:28:38.183 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111628.log.gz 2023-10-16T11:28:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.160461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106453.log 2023-10-16T11:28:38.184 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.160461.log.gz 2023-10-16T11:28:38.185 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.107999.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107999.log.gz 2023-10-16T11:28:38.185 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100120.log 2023-10-16T11:28:38.186 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.106453.log: gzip -5 88.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.106453.log.gz 2023-10-16T11:28:38.186 INFO:teuthology.orchestra.run.smithi084.stderr: /var/log/ceph/ceph-client.admin.162229.log 2023-10-16T11:28:38.187 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.100120.log: gzip -5 89.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.100120.log.gz -- 2023-10-16T11:28:38.187 INFO:teuthology.orchestra.run.smithi084.stderr: /var/log/ceph/ceph-client.admin.135959.log 2023-10-16T11:28:38.188 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.162229.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.162229.log.gz 2023-10-16T11:28:38.188 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142487.log 2023-10-16T11:28:38.189 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135959.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.135959.log.gz 2023-10-16T11:28:38.189 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122635.log 2023-10-16T11:28:38.190 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142487.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.142487.log.gz 2023-10-16T11:28:38.190 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163048.log 2023-10-16T11:28:38.191 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122635.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.122635.log.gz 2023-10-16T11:28:38.191 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94359.log 2023-10-16T11:28:38.192 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.163048.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.163048.log.gz 2023-10-16T11:28:38.192 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116494.log 2023-10-16T11:28:38.193 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94359.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94359.log.gz 2023-10-16T11:28:38.194 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150538.log 2023-10-16T11:28:38.194 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.116494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124493.log 2023-10-16T11:28:38.195 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116494.log.gz 2023-10-16T11:28:38.196 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123607.log 2023-10-16T11:28:38.196 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.150538.log.gz 2023-10-16T11:28:38.197 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.124493.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.124493.log.gz 2023-10-16T11:28:38.197 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136945.log 2023-10-16T11:28:38.198 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.123607.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.123607.log.gz 2023-10-16T11:28:38.198 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148730.log 2023-10-16T11:28:38.199 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111775.log 2023-10-16T11:28:38.199 INFO:teuthology.orchestra.run.smithi084.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.136945.log.gz 2023-10-16T11:28:38.200 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148730.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.148730.log.gz 2023-10-16T11:28:38.200 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98943.log 2023-10-16T11:28:38.201 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.111775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93690.log 2023-10-16T11:28:38.202 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111775.log.gz 2023-10-16T11:28:38.203 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98943.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98943.log.gz 2023-10-16T11:28:38.203 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118191.log 2023-10-16T11:28:38.204 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93690.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93690.log.gz 2023-10-16T11:28:38.204 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107490.log 2023-10-16T11:28:38.205 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118191.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118191.log.gz 2023-10-16T11:28:38.205 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106579.log 2023-10-16T11:28:38.206 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.107490.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107490.log.gz 2023-10-16T11:28:38.206 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163566.log 2023-10-16T11:28:38.207 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.106579.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106579.log.gz 2023-10-16T11:28:38.208 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158953.log 2023-10-16T11:28:38.208 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.163566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147191.log 2023-10-16T11:28:38.209 INFO:teuthology.orchestra.run.smithi084.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.163566.log.gz 2023-10-16T11:28:38.209 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129517.log 2023-10-16T11:28:38.210 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.158953.log.gz 2023-10-16T11:28:38.211 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151365.log 2023-10-16T11:28:38.211 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147191.log.gz 2023-10-16T11:28:38.212 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129517.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129517.log.gz 2023-10-16T11:28:38.212 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164712.log 2023-10-16T11:28:38.213 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154981.log 2023-10-16T11:28:38.214 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.151365.log.gz 2023-10-16T11:28:38.214 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.164712.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164712.log.gz 2023-10-16T11:28:38.215 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119635.log 2023-10-16T11:28:38.216 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139801.log 2023-10-16T11:28:38.216 INFO:teuthology.orchestra.run.smithi084.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.154981.log.gz 2023-10-16T11:28:38.217 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119635.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119635.log.gz 2023-10-16T11:28:38.217 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117398.log 2023-10-16T11:28:38.219 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139801.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.139801.log.gz 2023-10-16T11:28:38.219 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116704.log 2023-10-16T11:28:38.220 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.117398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101815.log 2023-10-16T11:28:38.221 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117398.log.gz 2023-10-16T11:28:38.221 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.116704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114579.log 2023-10-16T11:28:38.222 INFO:teuthology.orchestra.run.smithi084.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.116704.log.gz 2023-10-16T11:28:38.223 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.101815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121420.log 2023-10-16T11:28:38.223 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101815.log.gz 2023-10-16T11:28:38.224 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.114579.log: gzip -5 --verbose -- 89.2% /var/log/ceph/ceph-client.admin.93223.log -- replaced with /var/log/ceph/ceph-client.admin.114579.log.gz 2023-10-16T11:28:38.224 INFO:teuthology.orchestra.run.smithi084.stderr: 2023-10-16T11:28:38.225 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121420.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.121420.log.gz 2023-10-16T11:28:38.226 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97562.log 2023-10-16T11:28:38.227 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93699.log 2023-10-16T11:28:38.227 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93223.log.gz 2023-10-16T11:28:38.228 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97562.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97562.log.gz 2023-10-16T11:28:38.228 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155287.log 2023-10-16T11:28:38.229 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135008.log 2023-10-16T11:28:38.230 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93699.log.gz 2023-10-16T11:28:38.231 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100823.log 2023-10-16T11:28:38.231 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.155287.log.gz 2023-10-16T11:28:38.232 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135008.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.135008.log.gz 2023-10-16T11:28:38.232 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96890.log 2023-10-16T11:28:38.233 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.100823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153776.log 2023-10-16T11:28:38.233 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100823.log.gz 2023-10-16T11:28:38.234 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.96890.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96890.log.gz 2023-10-16T11:28:38.234 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165666.log 2023-10-16T11:28:38.235 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160335.log 2023-10-16T11:28:38.236 INFO:teuthology.orchestra.run.smithi084.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.153776.log.gz 2023-10-16T11:28:38.236 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.165666.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.165666.log.gz 2023-10-16T11:28:38.237 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116410.log 2023-10-16T11:28:38.238 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.160335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118107.log 2023-10-16T11:28:38.239 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.160335.log.gz 2023-10-16T11:28:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.116410.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116410.log.gz 2023-10-16T11:28:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160174.log 2023-10-16T11:28:38.241 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94013.log 2023-10-16T11:28:38.241 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118107.log.gz 2023-10-16T11:28:38.242 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.160174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99824.log 2023-10-16T11:28:38.243 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.160174.log.gz 2023-10-16T11:28:38.243 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94013.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94013.log.gz 2023-10-16T11:28:38.244 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127503.log 2023-10-16T11:28:38.244 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108391.log 2023-10-16T11:28:38.245 INFO:teuthology.orchestra.run.smithi084.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99824.log.gz 2023-10-16T11:28:38.246 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.127503.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.127503.log.gz 2023-10-16T11:28:38.246 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125363.log 2023-10-16T11:28:38.247 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.108391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148300.log 2023-10-16T11:28:38.248 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108391.log.gz 2023-10-16T11:28:38.248 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125363.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.125363.log.gz 2023-10-16T11:28:38.249 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148858.log 2023-10-16T11:28:38.250 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161246.log 2023-10-16T11:28:38.250 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.148300.log.gz 2023-10-16T11:28:38.251 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148858.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.148858.log.gz 2023-10-16T11:28:38.251 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-16T11:28:38.252 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138074.log 2023-10-16T11:28:38.253 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.161246.log.gz 2023-10-16T11:28:38.254 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115323.log 2023-10-16T11:28:38.255 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138074.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.138074.log.gz 2023-10-16T11:28:38.255 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148594.log 2023-10-16T11:28:38.256 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.115323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93693.log 2023-10-16T11:28:38.257 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115323.log.gz 2023-10-16T11:28:38.258 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148594.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.148594.log.gz 2023-10-16T11:28:38.258 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162399.log 2023-10-16T11:28:38.259 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118649.log 2023-10-16T11:28:38.260 INFO:teuthology.orchestra.run.smithi084.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.93693.log.gz 2023-10-16T11:28:38.260 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.162399.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162399.log.gz 2023-10-16T11:28:38.261 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156945.log 2023-10-16T11:28:38.262 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144268.log 2023-10-16T11:28:38.263 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118649.log.gz 2023-10-16T11:28:38.264 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156945.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.156945.log.gz 2023-10-16T11:28:38.264 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138715.log 2023-10-16T11:28:38.265 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103484.log 2023-10-16T11:28:38.266 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144268.log.gz 2023-10-16T11:28:38.267 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138715.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.138715.log.gz 2023-10-16T11:28:38.267 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153183.log 2023-10-16T11:28:38.268 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.103484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121462.log 2023-10-16T11:28:38.269 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103484.log.gz 2023-10-16T11:28:38.270 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153183.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.153183.log.gz 2023-10-16T11:28:38.270 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150346.log 2023-10-16T11:28:38.271 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94724.log 2023-10-16T11:28:38.272 INFO:teuthology.orchestra.run.smithi084.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.121462.log.gz 2023-10-16T11:28:38.273 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150346.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.150346.log.gz 2023-10-16T11:28:38.273 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121378.log 2023-10-16T11:28:38.274 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103620.log 2023-10-16T11:28:38.275 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94724.log.gz 2023-10-16T11:28:38.276 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121378.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.121378.log.gz 2023-10-16T11:28:38.276 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136384.log 2023-10-16T11:28:38.278 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.103620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119719.log 2023-10-16T11:28:38.278 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103620.log.gz 2023-10-16T11:28:38.279 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136384.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.136384.log.gz 2023-10-16T11:28:38.279 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126516.log 2023-10-16T11:28:38.280 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122378.log 2023-10-16T11:28:38.281 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119719.log.gz 2023-10-16T11:28:38.282 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.126516.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126516.log.gz 2023-10-16T11:28:38.282 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156291.log 2023-10-16T11:28:38.283 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162187.log 2023-10-16T11:28:38.283 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122378.log.gz 2023-10-16T11:28:38.284 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156291.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.156291.log.gz 2023-10-16T11:28:38.284 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151407.log 2023-10-16T11:28:38.285 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.162187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109441.log 2023-10-16T11:28:38.285 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.162187.log.gz 2023-10-16T11:28:38.286 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151407.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.151407.log.gz 2023-10-16T11:28:38.287 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144489.log 2023-10-16T11:28:38.288 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.109441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160293.log 2023-10-16T11:28:38.288 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109441.log.gz 2023-10-16T11:28:38.289 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144489.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.144489.log.gz 2023-10-16T11:28:38.289 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117227.log 2023-10-16T11:28:38.290 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.160293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126600.log 2023-10-16T11:28:38.290 INFO:teuthology.orchestra.run.smithi084.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.160293.log.gz 2023-10-16T11:28:38.291 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.117227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120187.log 2023-10-16T11:28:38.291 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117227.log.gz 2023-10-16T11:28:38.292 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.126600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129265.log 2023-10-16T11:28:38.293 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126600.log.gz 2023-10-16T11:28:38.294 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.120187.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.120187.log.gz 2023-10-16T11:28:38.294 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146918.log 2023-10-16T11:28:38.295 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137485.log 2023-10-16T11:28:38.295 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.129265.log.gz 2023-10-16T11:28:38.296 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127907.log 2023-10-16T11:28:38.296 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146918.log.gz 2023-10-16T11:28:38.297 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107041.log 2023-10-16T11:28:38.297 INFO:teuthology.orchestra.run.smithi084.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.137485.log.gz 2023-10-16T11:28:38.298 INFO:teuthology.orchestra.run.smithi157.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-16T11:28:38.299 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.127907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129475.log 2023-10-16T11:28:38.299 INFO:teuthology.orchestra.run.smithi084.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.127907.log.gz 2023-10-16T11:28:38.300 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.107041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163440.log 2023-10-16T11:28:38.300 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107041.log.gz 2023-10-16T11:28:38.301 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108125.log 2023-10-16T11:28:38.301 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129475.log.gz 2023-10-16T11:28:38.302 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.163440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117138.log 2023-10-16T11:28:38.302 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.163440.log.gz 2023-10-16T11:28:38.303 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.108125.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108125.log.gz 2023-10-16T11:28:38.304 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129391.log 2023-10-16T11:28:38.304 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.117138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139452.log 2023-10-16T11:28:38.305 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117138.log.gz 2023-10-16T11:28:38.306 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129391.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129391.log.gz 2023-10-16T11:28:38.306 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160857.log 2023-10-16T11:28:38.307 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162007.log 2023-10-16T11:28:38.307 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139452.log.gz 2023-10-16T11:28:38.308 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.160857.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.160857.log.gz 2023-10-16T11:28:38.309 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125955.log 2023-10-16T11:28:38.309 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.162007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113312.log 2023-10-16T11:28:38.310 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.162007.log.gz 2023-10-16T11:28:38.310 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125955.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125955.log.gz 2023-10-16T11:28:38.311 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155455.log 2023-10-16T11:28:38.312 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.113312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153838.log 2023-10-16T11:28:38.312 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113312.log.gz 2023-10-16T11:28:38.313 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147002.log 2023-10-16T11:28:38.313 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.155455.log.gz 2023-10-16T11:28:38.314 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120916.log 2023-10-16T11:28:38.315 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.153838.log.gz 2023-10-16T11:28:38.315 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124271.log 2023-10-16T11:28:38.316 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.147002.log.gz 2023-10-16T11:28:38.317 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.120916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116144.log 2023-10-16T11:28:38.317 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120916.log.gz 2023-10-16T11:28:38.318 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.124271.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124271.log.gz 2023-10-16T11:28:38.318 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137990.log 2023-10-16T11:28:38.319 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.116144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128311.log 2023-10-16T11:28:38.319 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116144.log.gz 2023-10-16T11:28:38.320 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137990.log: gzip -5 88.6% --verbose -- replaced with /var/log/ceph/ceph-client.admin.137990.log.gz -- 2023-10-16T11:28:38.321 INFO:teuthology.orchestra.run.smithi084.stderr: /var/log/ceph/ceph-client.admin.121189.log 2023-10-16T11:28:38.321 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.128311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104714.log 2023-10-16T11:28:38.322 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128311.log.gz 2023-10-16T11:28:38.323 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129960.log 2023-10-16T11:28:38.323 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121189.log.gz 2023-10-16T11:28:38.324 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.104714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112477.log 2023-10-16T11:28:38.324 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104714.log.gz 2023-10-16T11:28:38.325 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129960.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129960.log.gz 2023-10-16T11:28:38.326 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87010.log 2023-10-16T11:28:38.327 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.112477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122463.log 2023-10-16T11:28:38.327 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112477.log.gz 2023-10-16T11:28:38.328 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.87010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154727.log 2023-10-16T11:28:38.328 INFO:teuthology.orchestra.run.smithi084.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.87010.log.gz 2023-10-16T11:28:38.329 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165090.log 2023-10-16T11:28:38.330 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122463.log.gz 2023-10-16T11:28:38.331 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107304.log 2023-10-16T11:28:38.331 INFO:teuthology.orchestra.run.smithi084.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.154727.log.gz 2023-10-16T11:28:38.332 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.165090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97184.log 2023-10-16T11:28:38.332 INFO:teuthology.orchestra.run.smithi084.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.165090.log.gz 2023-10-16T11:28:38.333 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.107304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114537.log 2023-10-16T11:28:38.333 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107304.log.gz 2023-10-16T11:28:38.334 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97184.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97184.log.gz 2023-10-16T11:28:38.334 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121786.log 2023-10-16T11:28:38.335 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.114537.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114537.log.gz 2023-10-16T11:28:38.336 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98675.log 2023-10-16T11:28:38.337 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121786.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.121786.log.gz 2023-10-16T11:28:38.337 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162525.log 2023-10-16T11:28:38.338 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164922.log 2023-10-16T11:28:38.338 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98675.log.gz 2023-10-16T11:28:38.339 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.162525.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.162525.log.gz 2023-10-16T11:28:38.339 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161036.log 2023-10-16T11:28:38.340 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.164922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140073.log 2023-10-16T11:28:38.341 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.164922.log.gz 2023-10-16T11:28:38.341 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161036.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.161036.log.gz 2023-10-16T11:28:38.342 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124391.log 2023-10-16T11:28:38.343 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.140073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94371.log 2023-10-16T11:28:38.343 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.140073.log.gz 2023-10-16T11:28:38.344 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.124391.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.124391.log.gz 2023-10-16T11:28:38.344 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130851.log 2023-10-16T11:28:38.345 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152240.log 2023-10-16T11:28:38.345 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94371.log.gz 2023-10-16T11:28:38.346 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.130851.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.130851.log.gz 2023-10-16T11:28:38.347 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101478.log 2023-10-16T11:28:38.348 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152240.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.152240.log.gz 2023-10-16T11:28:38.348 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135436.log 2023-10-16T11:28:38.349 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.101478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155539.log 2023-10-16T11:28:38.349 INFO:teuthology.orchestra.run.smithi084.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101478.log.gz 2023-10-16T11:28:38.350 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113124.log 2023-10-16T11:28:38.350 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.135436.log.gz 2023-10-16T11:28:38.351 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155539.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.155539.log.gz 2023-10-16T11:28:38.351 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159770.log 2023-10-16T11:28:38.352 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.113124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160815.log 2023-10-16T11:28:38.353 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113124.log.gz 2023-10-16T11:28:38.353 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.159770.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.159770.log.gz 2023-10-16T11:28:38.354 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149800.log 2023-10-16T11:28:38.355 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.160815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158075.log 2023-10-16T11:28:38.355 INFO:teuthology.orchestra.run.smithi084.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.160815.log.gz 2023-10-16T11:28:38.356 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108213.log 2023-10-16T11:28:38.356 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.149800.log.gz 2023-10-16T11:28:38.357 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112183.log 2023-10-16T11:28:38.357 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.158075.log.gz 2023-10-16T11:28:38.358 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.108213.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108213.log.gz 2023-10-16T11:28:38.358 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111397.log 2023-10-16T11:28:38.360 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.112183.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112183.log.gz 2023-10-16T11:28:38.360 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126262.log 2023-10-16T11:28:38.361 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.111397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119761.log 2023-10-16T11:28:38.361 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111397.log.gz 2023-10-16T11:28:38.362 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.126262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112561.log 2023-10-16T11:28:38.362 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126262.log.gz 2023-10-16T11:28:38.363 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119761.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119761.log.gz 2023-10-16T11:28:38.363 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159163.log 2023-10-16T11:28:38.364 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.112561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160002.log 2023-10-16T11:28:38.365 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112561.log.gz 2023-10-16T11:28:38.365 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.159163.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159163.log.gz 2023-10-16T11:28:38.366 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155839.log 2023-10-16T11:28:38.367 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.160002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161120.log 2023-10-16T11:28:38.367 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.160002.log.gz 2023-10-16T11:28:38.368 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155839.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.155839.log.gz 2023-10-16T11:28:38.368 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158398.log 2023-10-16T11:28:38.369 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161120.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.161120.log.gz 2023-10-16T11:28:38.370 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98801.log 2023-10-16T11:28:38.370 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155623.log 2023-10-16T11:28:38.371 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.158398.log.gz 2023-10-16T11:28:38.372 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98801.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98801.log.gz 2023-10-16T11:28:38.372 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161162.log 2023-10-16T11:28:38.373 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137011.log 2023-10-16T11:28:38.373 INFO:teuthology.orchestra.run.smithi084.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.155623.log.gz 2023-10-16T11:28:38.374 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149176.log 2023-10-16T11:28:38.374 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.161162.log.gz 2023-10-16T11:28:38.375 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137011.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.137011.log.gz 2023-10-16T11:28:38.375 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160555.log 2023-10-16T11:28:38.376 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152931.log 2023-10-16T11:28:38.376 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149176.log.gz 2023-10-16T11:28:38.377 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.160555.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.160555.log.gz 2023-10-16T11:28:38.378 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152595.log 2023-10-16T11:28:38.379 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152931.log: gzip -5 --verbose -- 87.8% /var/log/ceph/ceph-client.0.96001.log 2023-10-16T11:28:38.379 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.152931.log.gz 2023-10-16T11:28:38.380 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115575.log 2023-10-16T11:28:38.380 INFO:teuthology.orchestra.run.smithi084.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.152595.log.gz 2023-10-16T11:28:38.380 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.96001.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.96001.log.gz 2023-10-16T11:28:38.381 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153267.log 2023-10-16T11:28:38.382 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.115575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118395.log 2023-10-16T11:28:38.382 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115575.log.gz 2023-10-16T11:28:38.383 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153267.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.153267.log.gz 2023-10-16T11:28:38.383 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111586.log 2023-10-16T11:28:38.384 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147380.log 2023-10-16T11:28:38.384 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.118395.log.gz 2023-10-16T11:28:38.385 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.111586.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111586.log.gz 2023-10-16T11:28:38.386 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127784.log 2023-10-16T11:28:38.387 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121147.log 2023-10-16T11:28:38.387 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.147380.log.gz 2023-10-16T11:28:38.388 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.127784.log: gzip -5 87.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.127784.log.gz 2023-10-16T11:28:38.388 INFO:teuthology.orchestra.run.smithi084.stderr: /var/log/ceph/ceph-client.admin.113030.log 2023-10-16T11:28:38.389 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162633.log 2023-10-16T11:28:38.389 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.121147.log.gz 2023-10-16T11:28:38.390 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.113030.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.113030.log.gz 2023-10-16T11:28:38.390 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159247.log 2023-10-16T11:28:38.391 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.162633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143567.log 2023-10-16T11:28:38.392 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.162633.log.gz 2023-10-16T11:28:38.392 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.159247.log: gzip -5 --verbose 88.9% -- -- replaced with /var/log/ceph/ceph-client.admin.159247.log.gz /var/log/ceph/ceph-client.admin.130268.log 2023-10-16T11:28:38.393 INFO:teuthology.orchestra.run.smithi084.stderr: 2023-10-16T11:28:38.394 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134735.log 2023-10-16T11:28:38.394 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.143567.log.gz 2023-10-16T11:28:38.395 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.130268.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.130268.log.gz 2023-10-16T11:28:38.395 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94674.log 2023-10-16T11:28:38.396 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.134735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.126937.log 2023-10-16T11:28:38.396 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.134735.log.gz 2023-10-16T11:28:38.397 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94674.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94674.log.gz 2023-10-16T11:28:38.398 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97100.log 2023-10-16T11:28:38.398 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.126937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136522.log 2023-10-16T11:28:38.399 INFO:teuthology.orchestra.run.smithi084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.126937.log.gz 2023-10-16T11:28:38.400 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148816.log 2023-10-16T11:28:38.400 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97100.log.gz 2023-10-16T11:28:38.401 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136522.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.136522.log.gz 2023-10-16T11:28:38.401 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155025.log 2023-10-16T11:28:38.402 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148816.log: gzip -5 88.6% --verbose -- replaced with /var/log/ceph/ceph-client.admin.148816.log.gz -- 2023-10-16T11:28:38.402 INFO:teuthology.orchestra.run.smithi084.stderr: /var/log/ceph/ceph-client.admin.126684.log 2023-10-16T11:28:38.403 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154060.log 2023-10-16T11:28:38.403 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.155025.log.gz 2023-10-16T11:28:38.404 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.126684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108169.log 2023-10-16T11:28:38.405 INFO:teuthology.orchestra.run.smithi084.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.126684.log.gz 2023-10-16T11:28:38.406 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154060.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.154060.log.gz 2023-10-16T11:28:38.406 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106768.log 2023-10-16T11:28:38.407 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.108169.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108169.log.gz 2023-10-16T11:28:38.407 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143930.log 2023-10-16T11:28:38.408 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.106768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162990.log 2023-10-16T11:28:38.409 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106768.log.gz 2023-10-16T11:28:38.409 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129710.log 2023-10-16T11:28:38.410 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.143930.log.gz 2023-10-16T11:28:38.411 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.162990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115407.log 2023-10-16T11:28:38.411 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.162990.log.gz 2023-10-16T11:28:38.412 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129710.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.129710.log.gz 2023-10-16T11:28:38.412 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.93391.log 2023-10-16T11:28:38.413 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.115407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93307.log 2023-10-16T11:28:38.413 INFO:teuthology.orchestra.run.smithi084.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.115407.log.gz 2023-10-16T11:28:38.414 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126040.log 2023-10-16T11:28:38.415 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93391.log.gz 2023-10-16T11:28:38.415 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109166.log 2023-10-16T11:28:38.416 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93307.log.gz 2023-10-16T11:28:38.416 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.126040.log: gzip -5 --verbose 88.4% -- -- replaced with /var/log/ceph/ceph-client.admin.126040.log.gz /var/log/ceph/ceph-client.admin.157384.log 2023-10-16T11:28:38.417 INFO:teuthology.orchestra.run.smithi084.stderr: 2023-10-16T11:28:38.418 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.109166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128489.log 2023-10-16T11:28:38.418 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109166.log.gz 2023-10-16T11:28:38.419 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159205.log 2023-10-16T11:28:38.419 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.157384.log.gz 2023-10-16T11:28:38.420 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.128489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142326.log 2023-10-16T11:28:38.420 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128489.log.gz 2023-10-16T11:28:38.421 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.159205.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.159205.log.gz 2023-10-16T11:28:38.422 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108643.log 2023-10-16T11:28:38.422 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136608.log 2023-10-16T11:28:38.423 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.142326.log.gz 2023-10-16T11:28:38.423 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.108643.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.108643.log.gzgzip 2023-10-16T11:28:38.424 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.125573.log 2023-10-16T11:28:38.425 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126390.log 2023-10-16T11:28:38.425 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136608.log.gz 2023-10-16T11:28:38.426 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125573.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.125573.log.gz -5 2023-10-16T11:28:38.426 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.120295.log 2023-10-16T11:28:38.427 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.126390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155689.log 2023-10-16T11:28:38.427 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126390.log.gz 2023-10-16T11:28:38.428 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.120295.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120295.log.gz 2023-10-16T11:28:38.429 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119242.log 2023-10-16T11:28:38.429 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135875.log 2023-10-16T11:28:38.430 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.155689.log.gz 2023-10-16T11:28:38.431 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135341.log 2023-10-16T11:28:38.431 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.119242.log.gz 2023-10-16T11:28:38.432 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124722.log 2023-10-16T11:28:38.432 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135875.log.gz 2023-10-16T11:28:38.433 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112267.log 2023-10-16T11:28:38.433 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.135341.log.gz 2023-10-16T11:28:38.434 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.124722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163398.log 2023-10-16T11:28:38.435 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.124722.log.gz 2023-10-16T11:28:38.435 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.112267.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.112267.log.gz -5 2023-10-16T11:28:38.436 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.99198.log 2023-10-16T11:28:38.437 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.163398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136134.log 2023-10-16T11:28:38.437 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.163398.log.gz 2023-10-16T11:28:38.438 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99198.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.96932.log 2023-10-16T11:28:38.438 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99198.log.gz 2023-10-16T11:28:38.439 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95155.log 2023-10-16T11:28:38.439 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.136134.log.gz 2023-10-16T11:28:38.440 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.96932.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96932.log.gz 2023-10-16T11:28:38.440 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130226.log 2023-10-16T11:28:38.441 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.95155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150262.log 2023-10-16T11:28:38.442 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95155.log.gz 2023-10-16T11:28:38.442 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.130226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157552.log 2023-10-16T11:28:38.443 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.130226.log.gz 2023-10-16T11:28:38.444 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97016.log 2023-10-16T11:28:38.444 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.150262.log.gz 2023-10-16T11:28:38.445 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144180.log 2023-10-16T11:28:38.445 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.157552.log.gz 2023-10-16T11:28:38.446 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113074.log 2023-10-16T11:28:38.446 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97016.log.gz 2023-10-16T11:28:38.447 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144180.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-16T11:28:38.448 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144180.log.gz 2023-10-16T11:28:38.448 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.113074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122297.log 2023-10-16T11:28:38.449 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113074.log.gz 2023-10-16T11:28:38.449 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99696.log 2023-10-16T11:28:38.451 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122297.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122297.log.gz 2023-10-16T11:28:38.452 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125270.log 2023-10-16T11:28:38.453 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99696.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.99696.log.gz 2023-10-16T11:28:38.453 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119423.log 2023-10-16T11:28:38.454 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100706.log 2023-10-16T11:28:38.455 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125270.log.gz 2023-10-16T11:28:38.455 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119423.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119423.log.gz 2023-10-16T11:28:38.456 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145072.log 2023-10-16T11:28:38.457 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.100706.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100706.log.gz 2023-10-16T11:28:38.457 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116536.log 2023-10-16T11:28:38.458 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153057.log 2023-10-16T11:28:38.459 INFO:teuthology.orchestra.run.smithi084.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.145072.log.gz 2023-10-16T11:28:38.460 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.116536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116366.log 2023-10-16T11:28:38.460 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116536.log.gz 2023-10-16T11:28:38.461 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153057.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.153057.log.gz 2023-10-16T11:28:38.462 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155413.log 2023-10-16T11:28:38.463 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.116366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149926.log 2023-10-16T11:28:38.463 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116366.log.gz 2023-10-16T11:28:38.465 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155413.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.155413.log.gz 2023-10-16T11:28:38.465 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114999.log 2023-10-16T11:28:38.466 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149926.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.149926.log.gz 2023-10-16T11:28:38.466 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123023.log 2023-10-16T11:28:38.467 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.114999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134819.log 2023-10-16T11:28:38.468 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114999.log.gz 2023-10-16T11:28:38.469 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.123023.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123023.log.gz -5 2023-10-16T11:28:38.469 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.117035.log 2023-10-16T11:28:38.471 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.134819.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134819.log.gz 2023-10-16T11:28:38.471 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100288.log 2023-10-16T11:28:38.472 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.117035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98303.log 2023-10-16T11:28:38.472 INFO:teuthology.orchestra.run.smithi084.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.117035.log.gz 2023-10-16T11:28:38.473 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.100288.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100288.log.gz 2023-10-16T11:28:38.474 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118065.log 2023-10-16T11:28:38.475 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98303.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.98303.log.gz 2023-10-16T11:28:38.475 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126212.log 2023-10-16T11:28:38.476 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116102.log 2023-10-16T11:28:38.477 INFO:teuthology.orchestra.run.smithi084.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118065.log.gz 2023-10-16T11:28:38.478 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.126212.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.126212.log.gz 2023-10-16T11:28:38.478 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101987.log 2023-10-16T11:28:38.479 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.116102.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116102.log.gz 2023-10-16T11:28:38.481 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113937.log 2023-10-16T11:28:38.482 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.101987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147149.log 2023-10-16T11:28:38.482 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101987.log.gz 2023-10-16T11:28:38.483 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.113937.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113937.log.gz 2023-10-16T11:28:38.483 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150136.log 2023-10-16T11:28:38.484 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114495.log 2023-10-16T11:28:38.484 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147149.log.gz 2023-10-16T11:28:38.485 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94327.log 2023-10-16T11:28:38.486 INFO:teuthology.orchestra.run.smithi084.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.150136.log.gz 2023-10-16T11:28:38.487 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.114495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127183.log 2023-10-16T11:28:38.487 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114495.log.gz 2023-10-16T11:28:38.488 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94327.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94327.log.gz 2023-10-16T11:28:38.489 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99029.log 2023-10-16T11:28:38.490 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.127183.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127183.log.gz 2023-10-16T11:28:38.490 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138902.log 2023-10-16T11:28:38.491 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125321.log 2023-10-16T11:28:38.492 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99029.log.gz 2023-10-16T11:28:38.493 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138902.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.138902.log.gz 2023-10-16T11:28:38.493 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123191.log 2023-10-16T11:28:38.494 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125321.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125321.log.gz 2023-10-16T11:28:38.495 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159289.log 2023-10-16T11:28:38.496 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.123191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142802.log 2023-10-16T11:28:38.496 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.123191.log.gz 2023-10-16T11:28:38.497 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.159289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142529.log 2023-10-16T11:28:38.498 INFO:teuthology.orchestra.run.smithi084.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.159289.log.gz 2023-10-16T11:28:38.499 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121828.log 2023-10-16T11:28:38.499 INFO:teuthology.orchestra.run.smithi084.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.142802.log.gz 2023-10-16T11:28:38.500 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121744.log 2023-10-16T11:28:38.501 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.142529.log.gz 2023-10-16T11:28:38.502 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121828.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.121828.log.gz 2023-10-16T11:28:38.502 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93433.log 2023-10-16T11:28:38.503 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121744.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.121744.log.gz 2023-10-16T11:28:38.503 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128773.log 2023-10-16T11:28:38.505 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138263.log 2023-10-16T11:28:38.505 INFO:teuthology.orchestra.run.smithi084.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93433.log.gz 2023-10-16T11:28:38.506 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.128773.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.128773.log.gz 2023-10-16T11:28:38.506 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135623.log 2023-10-16T11:28:38.508 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109398.log 2023-10-16T11:28:38.508 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138263.log.gz 2023-10-16T11:28:38.510 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98261.log 2023-10-16T11:28:38.510 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.135623.log.gz 2023-10-16T11:28:38.511 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.109398.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109398.log.gz 2023-10-16T11:28:38.511 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106673.log 2023-10-16T11:28:38.512 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130394.log 2023-10-16T11:28:38.513 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98261.log.gz 2023-10-16T11:28:38.514 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.106673.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106673.log.gz 2023-10-16T11:28:38.514 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156819.log 2023-10-16T11:28:38.516 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.130394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99114.log 2023-10-16T11:28:38.516 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.130394.log.gz 2023-10-16T11:28:38.517 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119381.log 2023-10-16T11:28:38.518 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.156819.log.gz 2023-10-16T11:28:38.518 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100204.log 2023-10-16T11:28:38.519 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99114.log.gz 2023-10-16T11:28:38.520 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119381.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119381.log.gz 2023-10-16T11:28:38.520 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106915.log 2023-10-16T11:28:38.522 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.100204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117185.log 2023-10-16T11:28:38.522 INFO:teuthology.orchestra.run.smithi084.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.100204.log.gz 2023-10-16T11:28:38.524 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.106915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142949.log 2023-10-16T11:28:38.524 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106915.log.gz 2023-10-16T11:28:38.525 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.117185.log: gzip -5 --verbose -- 88.2% /var/log/ceph/ceph-client.admin.98717.log -- replaced with /var/log/ceph/ceph-client.admin.117185.log.gz 2023-10-16T11:28:38.525 INFO:teuthology.orchestra.run.smithi084.stderr: 2023-10-16T11:28:38.526 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162145.log 2023-10-16T11:28:38.527 INFO:teuthology.orchestra.run.smithi084.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.142949.log.gz 2023-10-16T11:28:38.528 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98717.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.98717.log.gz -5 2023-10-16T11:28:38.528 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.128584.log 2023-10-16T11:28:38.529 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.162145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118817.log 2023-10-16T11:28:38.530 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.162145.log.gz 2023-10-16T11:28:38.531 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.128584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93181.log 2023-10-16T11:28:38.531 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128584.log.gz 2023-10-16T11:28:38.532 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139494.log 2023-10-16T11:28:38.533 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.118817.log.gz 2023-10-16T11:28:38.534 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93181.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93181.log.gz 2023-10-16T11:28:38.534 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118565.log 2023-10-16T11:28:38.535 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139494.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.139494.log.gz 2023-10-16T11:28:38.535 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130436.log 2023-10-16T11:28:38.536 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148688.log 2023-10-16T11:28:38.537 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118565.log.gz 2023-10-16T11:28:38.538 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.130436.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.130436.log.gz 2023-10-16T11:28:38.538 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148125.log 2023-10-16T11:28:38.539 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148688.log: 89.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.148688.log.gz 2023-10-16T11:28:38.540 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.161677.log 2023-10-16T11:28:38.541 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111355.log 2023-10-16T11:28:38.541 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.148125.log.gz 2023-10-16T11:28:38.542 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161677.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.161677.log.gz 2023-10-16T11:28:38.543 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151680.log 2023-10-16T11:28:38.544 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.111355.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111355.log.gz 2023-10-16T11:28:38.544 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147956.log 2023-10-16T11:28:38.545 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135394.log 2023-10-16T11:28:38.545 INFO:teuthology.orchestra.run.smithi084.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.151680.log.gz 2023-10-16T11:28:38.547 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147956.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.154312.log 2023-10-16T11:28:38.547 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.147956.log.gz 2023-10-16T11:28:38.548 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135394.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.135394.log.gz 2023-10-16T11:28:38.548 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103561.log 2023-10-16T11:28:38.549 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98987.log 2023-10-16T11:28:38.550 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.154312.log.gz 2023-10-16T11:28:38.551 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.103561.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103561.log.gz 2023-10-16T11:28:38.551 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106411.log 2023-10-16T11:28:38.552 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98987.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98987.log.gz 2023-10-16T11:28:38.553 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150304.log 2023-10-16T11:28:38.554 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.106411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101772.log 2023-10-16T11:28:38.554 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106411.log.gz 2023-10-16T11:28:38.555 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150304.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150304.log.gz 2023-10-16T11:28:38.556 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154270.log 2023-10-16T11:28:38.557 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.101772.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101772.log.gz 2023-10-16T11:28:38.557 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156459.log 2023-10-16T11:28:38.559 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95407.log 2023-10-16T11:28:38.559 INFO:teuthology.orchestra.run.smithi084.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.154270.log.gz 2023-10-16T11:28:38.560 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156459.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.156459.log.gz 2023-10-16T11:28:38.561 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118523.log 2023-10-16T11:28:38.562 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.95407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117519.log 2023-10-16T11:28:38.562 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95407.log.gz 2023-10-16T11:28:38.563 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118523.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118523.log.gz 2023-10-16T11:28:38.564 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97841.log 2023-10-16T11:28:38.565 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.117519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153225.log 2023-10-16T11:28:38.565 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117519.log.gz 2023-10-16T11:28:38.566 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97841.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97841.log.gz 2023-10-16T11:28:38.567 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163885.log 2023-10-16T11:28:38.567 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153225.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.153225.log.gz 2023-10-16T11:28:38.568 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149501.log 2023-10-16T11:28:38.569 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.163885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118992.log 2023-10-16T11:28:38.570 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163885.log.gz 2023-10-16T11:28:38.571 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149501.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.149501.log.gz 2023-10-16T11:28:38.571 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114768.log 2023-10-16T11:28:38.572 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118992.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118992.log.gz 2023-10-16T11:28:38.573 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163090.log 2023-10-16T11:28:38.573 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.114768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158159.log 2023-10-16T11:28:38.574 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114768.log.gz 2023-10-16T11:28:38.575 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.163090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98087.log 2023-10-16T11:28:38.576 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.163090.log.gz 2023-10-16T11:28:38.577 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158159.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.158159.log.gz 2023-10-16T11:28:38.577 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116770.log 2023-10-16T11:28:38.578 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109225.log 2023-10-16T11:28:38.579 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98087.log.gz 2023-10-16T11:28:38.579 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.116770.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116770.log.gz 2023-10-16T11:28:38.580 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94903.log 2023-10-16T11:28:38.581 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.109225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136342.log 2023-10-16T11:28:38.582 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109225.log.gz 2023-10-16T11:28:38.583 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94903.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94903.log.gz 2023-10-16T11:28:38.583 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151491.log 2023-10-16T11:28:38.584 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135239.log 2023-10-16T11:28:38.584 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136342.log.gz 2023-10-16T11:28:38.586 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151491.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.151491.log.gz 2023-10-16T11:28:38.586 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138673.log 2023-10-16T11:28:38.587 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135239.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.135239.log.gz 2023-10-16T11:28:38.587 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100162.log 2023-10-16T11:28:38.588 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108347.log 2023-10-16T11:28:38.589 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.138673.log.gz 2023-10-16T11:28:38.590 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.100162.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100162.log.gz 2023-10-16T11:28:38.590 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161838.log 2023-10-16T11:28:38.591 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.108347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157155.log 2023-10-16T11:28:38.592 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108347.log.gz 2023-10-16T11:28:38.593 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161838.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.161838.log.gz 2023-10-16T11:28:38.593 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150663.log 2023-10-16T11:28:38.595 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139371.log 2023-10-16T11:28:38.595 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.157155.log.gz 2023-10-16T11:28:38.596 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150663.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.150663.log.gz 2023-10-16T11:28:38.596 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115533.log 2023-10-16T11:28:38.597 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158201.log 2023-10-16T11:28:38.598 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.139371.log.gz 2023-10-16T11:28:38.599 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.115533.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.115533.log.gz 2023-10-16T11:28:38.599 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.95449.log 2023-10-16T11:28:38.600 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158201.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.158201.log.gz 2023-10-16T11:28:38.601 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161204.log 2023-10-16T11:28:38.602 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.95449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124997.log 2023-10-16T11:28:38.602 INFO:teuthology.orchestra.run.smithi084.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95449.log.gz 2023-10-16T11:28:38.603 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161204.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.161204.log.gz 2023-10-16T11:28:38.604 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142758.log 2023-10-16T11:28:38.605 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.124997.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124997.log.gz 2023-10-16T11:28:38.605 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129433.log 2023-10-16T11:28:38.606 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126558.log 2023-10-16T11:28:38.606 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.142758.log.gz 2023-10-16T11:28:38.608 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129433.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.129433.log.gz 2023-10-16T11:28:38.608 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93139.log 2023-10-16T11:28:38.609 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.126558.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.126558.log.gz 2023-10-16T11:28:38.609 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143254.log 2023-10-16T11:28:38.610 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138986.log 2023-10-16T11:28:38.611 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93139.log.gz 2023-10-16T11:28:38.612 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143254.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.143254.log.gz 2023-10-16T11:28:38.612 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165174.log 2023-10-16T11:28:38.614 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113895.log 2023-10-16T11:28:38.614 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138986.log.gz 2023-10-16T11:28:38.615 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.165174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93011.log 2023-10-16T11:28:38.616 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.165174.log.gz 2023-10-16T11:28:38.617 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.113895.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113895.log.gz 2023-10-16T11:28:38.617 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164754.log 2023-10-16T11:28:38.618 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157299.log 2023-10-16T11:28:38.619 INFO:teuthology.orchestra.run.smithi084.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93011.log.gz 2023-10-16T11:28:38.619 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.164754.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.164754.log.gz 2023-10-16T11:28:38.620 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100662.log 2023-10-16T11:28:38.621 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157299.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.157299.log.gz 2023-10-16T11:28:38.621 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114411.log 2023-10-16T11:28:38.622 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.100662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112860.log 2023-10-16T11:28:38.623 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100662.log.gz 2023-10-16T11:28:38.624 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.114411.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114411.log.gz 2023-10-16T11:28:38.624 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155159.log 2023-10-16T11:28:38.625 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.112860.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112860.log.gz 2023-10-16T11:28:38.626 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139971.log 2023-10-16T11:28:38.627 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115136.log 2023-10-16T11:28:38.627 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.155159.log.gz 2023-10-16T11:28:38.628 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139971.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.139971.log.gz 2023-10-16T11:28:38.629 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144573.log 2023-10-16T11:28:38.630 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.115136.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115136.log.gz 2023-10-16T11:28:38.630 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111271.log 2023-10-16T11:28:38.631 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150220.log 2023-10-16T11:28:38.631 INFO:teuthology.orchestra.run.smithi084.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.144573.log.gz 2023-10-16T11:28:38.633 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.111271.log: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.111271.log.gz 2023-10-16T11:28:38.633 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.118149.log 2023-10-16T11:28:38.634 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134955.log 2023-10-16T11:28:38.635 INFO:teuthology.orchestra.run.smithi084.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150220.log.gz 2023-10-16T11:28:38.636 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118149.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118149.log.gz 2023-10-16T11:28:38.636 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149585.log 2023-10-16T11:28:38.637 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.134955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120671.log 2023-10-16T11:28:38.638 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.134955.log.gz 2023-10-16T11:28:38.639 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149585.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.149585.log.gz 2023-10-16T11:28:38.639 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101238.log 2023-10-16T11:28:38.641 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.120671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151085.log 2023-10-16T11:28:38.641 INFO:teuthology.orchestra.run.smithi084.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.120671.log.gz 2023-10-16T11:28:38.642 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.101238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164478.log 2023-10-16T11:28:38.643 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101238.log.gz 2023-10-16T11:28:38.644 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151085.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.151085.log.gz 2023-10-16T11:28:38.644 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.104977.log 2023-10-16T11:28:38.645 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.164478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128269.log 2023-10-16T11:28:38.645 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.164478.log.gz 2023-10-16T11:28:38.647 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.104977.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104977.log.gz 2023-10-16T11:28:38.647 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114045.log 2023-10-16T11:28:38.648 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.128269.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.128269.log.gz 2023-10-16T11:28:38.648 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152511.log 2023-10-16T11:28:38.649 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.114045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141448.log 2023-10-16T11:28:38.650 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114045.log.gz 2023-10-16T11:28:38.651 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152511.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.152511.log.gz 2023-10-16T11:28:38.651 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122939.log 2023-10-16T11:28:38.652 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141448.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.141448.log.gz 2023-10-16T11:28:38.653 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155497.log 2023-10-16T11:28:38.654 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146834.log 2023-10-16T11:28:38.654 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.122939.log.gz 2023-10-16T11:28:38.655 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155497.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.155497.log.gz 2023-10-16T11:28:38.656 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101944.log 2023-10-16T11:28:38.657 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155581.log 2023-10-16T11:28:38.658 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.146834.log.gz 2023-10-16T11:28:38.659 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.101944.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101944.log.gz 2023-10-16T11:28:38.659 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105766.log 2023-10-16T11:28:38.660 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155581.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-16T11:28:38.660 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.155581.log.gz 2023-10-16T11:28:38.661 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.105766.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105766.log.gz 2023-10-16T11:28:38.662 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122981.log 2023-10-16T11:28:38.663 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122685.log 2023-10-16T11:28:38.665 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134693.log 2023-10-16T11:28:38.665 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122981.log.gz 2023-10-16T11:28:38.666 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164165.log 2023-10-16T11:28:38.666 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.122685.log.gz 2023-10-16T11:28:38.668 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.134693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138452.log 2023-10-16T11:28:38.668 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.134693.log.gz 2023-10-16T11:28:38.669 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.164165.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.164165.log.gz 2023-10-16T11:28:38.669 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144011.log 2023-10-16T11:28:38.671 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138452.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.138452.log.gz 2023-10-16T11:28:38.671 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153309.log 2023-10-16T11:28:38.672 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141978.log 2023-10-16T11:28:38.672 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.144011.log.gz 2023-10-16T11:28:38.674 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138410.log 2023-10-16T11:28:38.674 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.153309.log.gz 2023-10-16T11:28:38.675 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121912.log 2023-10-16T11:28:38.675 INFO:teuthology.orchestra.run.smithi084.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.141978.log.gz 2023-10-16T11:28:38.676 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119158.log 2023-10-16T11:28:38.677 INFO:teuthology.orchestra.run.smithi084.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138410.log.gz 2023-10-16T11:28:38.678 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161288.log 2023-10-16T11:28:38.679 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.121912.log.gz 2023-10-16T11:28:38.680 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120713.log 2023-10-16T11:28:38.680 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119158.log.gz 2023-10-16T11:28:38.681 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.130773.log 2023-10-16T11:28:38.681 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.161288.log.gz 2023-10-16T11:28:38.682 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.120713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119593.log 2023-10-16T11:28:38.683 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120713.log.gz 2023-10-16T11:28:38.683 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.130773.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.130773.log.gz 2023-10-16T11:28:38.686 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108433.log 2023-10-16T11:28:38.686 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148466.log 2023-10-16T11:28:38.687 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.119593.log.gz 2023-10-16T11:28:38.688 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.108433.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.108433.log.gz -5 2023-10-16T11:28:38.688 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.152637.log 2023-10-16T11:28:38.689 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98843.log 2023-10-16T11:28:38.689 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.148466.log.gz 2023-10-16T11:28:38.690 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112140.log 2023-10-16T11:28:38.690 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.152637.log.gz 2023-10-16T11:28:38.691 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99444.log 2023-10-16T11:28:38.692 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98843.log.gz 2023-10-16T11:28:38.693 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.112140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127683.log 87.7% 2023-10-16T11:28:38.693 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.112140.log.gz 2023-10-16T11:28:38.694 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103662.log 2023-10-16T11:28:38.695 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99444.log.gz 2023-10-16T11:28:38.696 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.127683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159079.log 2023-10-16T11:28:38.696 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127683.log.gz 2023-10-16T11:28:38.697 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.103662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122122.log 2023-10-16T11:28:38.697 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103662.log.gz 2023-10-16T11:28:38.699 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.159079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136734.log 2023-10-16T11:28:38.699 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159079.log.gz 2023-10-16T11:28:38.700 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118335.log 2023-10-16T11:28:38.701 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122122.log.gz 2023-10-16T11:28:38.701 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136054.log 2023-10-16T11:28:38.702 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.136734.log.gz 2023-10-16T11:28:38.703 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141936.log 2023-10-16T11:28:38.703 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118335.log.gz 2023-10-16T11:28:38.704 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117781.log 2023-10-16T11:28:38.705 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.136054.log.gz 2023-10-16T11:28:38.706 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121870.log 2023-10-16T11:28:38.706 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.141936.log.gz 2023-10-16T11:28:38.707 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.117781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97478.log 2023-10-16T11:28:38.707 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117781.log.gz 2023-10-16T11:28:38.709 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119115.log 2023-10-16T11:28:38.709 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.121870.log.gz 2023-10-16T11:28:38.710 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99156.log 2023-10-16T11:28:38.711 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97478.log.gz 2023-10-16T11:28:38.712 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122769.log 2023-10-16T11:28:38.712 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119115.log.gz 2023-10-16T11:28:38.713 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161498.log 2023-10-16T11:28:38.713 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99156.log.gz 2023-10-16T11:28:38.714 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122769.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.122769.log.gz 2023-10-16T11:28:38.715 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97799.log 2023-10-16T11:28:38.716 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125794.log 2023-10-16T11:28:38.717 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.161498.log.gz 2023-10-16T11:28:38.718 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150861.log 2023-10-16T11:28:38.718 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97799.log.gz 2023-10-16T11:28:38.719 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125794.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125794.log.gz 2023-10-16T11:28:38.719 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136215.log 2023-10-16T11:28:38.721 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.123378.log 2023-10-16T11:28:38.721 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150861.log.gz 2023-10-16T11:28:38.722 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136215.log: gzip -5 --verbose 87.4% -- /var/log/ceph/ceph-client.admin.143441.log -- replaced with /var/log/ceph/ceph-client.admin.136215.log.gz 2023-10-16T11:28:38.722 INFO:teuthology.orchestra.run.smithi084.stderr: 2023-10-16T11:28:38.723 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.123378.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.123378.log.gz 2023-10-16T11:28:38.724 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139579.log 2023-10-16T11:28:38.725 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130004.log 2023-10-16T11:28:38.725 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.143441.log.gz 2023-10-16T11:28:38.727 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102928.log 2023-10-16T11:28:38.727 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139579.log.gz 2023-10-16T11:28:38.728 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.130004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115449.log 2023-10-16T11:28:38.729 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.130004.log.gz 2023-10-16T11:28:38.730 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.102928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111901.log 2023-10-16T11:28:38.730 INFO:teuthology.orchestra.run.smithi084.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.102928.log.gz 2023-10-16T11:28:38.732 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.115449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140199.log 2023-10-16T11:28:38.732 INFO:teuthology.orchestra.run.smithi084.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.115449.log.gz 2023-10-16T11:28:38.733 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.111901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94356.log 2023-10-16T11:28:38.733 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111901.log.gz 2023-10-16T11:28:38.734 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.140199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154420.log 2023-10-16T11:28:38.734 INFO:teuthology.orchestra.run.smithi084.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.140199.log.gz 2023-10-16T11:28:38.735 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100899.log 2023-10-16T11:28:38.735 INFO:teuthology.orchestra.run.smithi084.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94356.log.gz 2023-10-16T11:28:38.736 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121504.log 2023-10-16T11:28:38.737 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.154420.log.gz 2023-10-16T11:28:38.739 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.100899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96638.log 2023-10-16T11:28:38.739 INFO:teuthology.orchestra.run.smithi084.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.100899.log.gz 2023-10-16T11:28:38.740 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163712.log 2023-10-16T11:28:38.740 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.121504.log.gz 2023-10-16T11:28:38.741 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.96638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152114.log 2023-10-16T11:28:38.741 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96638.log.gz 2023-10-16T11:28:38.742 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.163712.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163712.log.gz 2023-10-16T11:28:38.742 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164838.log 2023-10-16T11:28:38.744 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101089.log 2023-10-16T11:28:38.744 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.152114.log.gz 2023-10-16T11:28:38.745 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.164838.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.164838.log.gz -5 2023-10-16T11:28:38.745 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.150052.log 2023-10-16T11:28:38.747 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.101089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151893.log 2023-10-16T11:28:38.747 INFO:teuthology.orchestra.run.smithi084.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101089.log.gz 2023-10-16T11:28:38.748 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123125.log 2023-10-16T11:28:38.749 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.150052.log.gz 2023-10-16T11:28:38.750 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148381.log 2023-10-16T11:28:38.750 INFO:teuthology.orchestra.run.smithi084.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.151893.log.gz 2023-10-16T11:28:38.751 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.123125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112944.log 2023-10-16T11:28:38.751 INFO:teuthology.orchestra.run.smithi084.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.123125.log.gz 2023-10-16T11:28:38.753 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148381.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.148381.log.gz 2023-10-16T11:28:38.753 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134532.log 2023-10-16T11:28:38.754 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.112944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158783.log 2023-10-16T11:28:38.754 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112944.log.gz 2023-10-16T11:28:38.755 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.134532.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.134532.log.gz --verbose 2023-10-16T11:28:38.755 INFO:teuthology.orchestra.run.smithi084.stderr: -- /var/log/ceph/ceph-client.admin.116662.log 2023-10-16T11:28:38.757 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158827.log 2023-10-16T11:28:38.757 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.158783.log.gz 2023-10-16T11:28:38.758 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.116662.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.116662.log.gz 2023-10-16T11:28:38.759 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.164267.log 2023-10-16T11:28:38.760 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94987.log 2023-10-16T11:28:38.760 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.158827.log.gz 2023-10-16T11:28:38.761 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.164267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152019.log 2023-10-16T11:28:38.762 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.164267.log.gz 2023-10-16T11:28:38.763 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137318.log 2023-10-16T11:28:38.763 INFO:teuthology.orchestra.run.smithi084.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.94987.log.gz 2023-10-16T11:28:38.764 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159037.log 2023-10-16T11:28:38.765 INFO:teuthology.orchestra.run.smithi084.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.152019.log.gz 2023-10-16T11:28:38.766 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127563.log 2023-10-16T11:28:38.766 INFO:teuthology.orchestra.run.smithi084.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.137318.log.gz 2023-10-16T11:28:38.767 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.159037.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.159037.log.gz -5 2023-10-16T11:28:38.767 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.128395.log 2023-10-16T11:28:38.768 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.127563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152198.log 2023-10-16T11:28:38.769 INFO:teuthology.orchestra.run.smithi084.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.127563.log.gz 2023-10-16T11:28:38.770 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.128395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102030.log 2023-10-16T11:28:38.770 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128395.log.gz 2023-10-16T11:28:38.771 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148942.log 2023-10-16T11:28:38.772 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.152198.log.gz 2023-10-16T11:28:38.773 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.102030.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.102030.log.gz 2023-10-16T11:28:38.773 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.99952.log 2023-10-16T11:28:38.774 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165216.log 2023-10-16T11:28:38.775 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.148942.log.gz 2023-10-16T11:28:38.776 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108475.log 2023-10-16T11:28:38.776 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99952.log.gz 2023-10-16T11:28:38.777 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.165216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109058.log 2023-10-16T11:28:38.777 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.165216.log.gz 2023-10-16T11:28:38.779 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.108475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124955.log 2023-10-16T11:28:38.779 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108475.log.gz 2023-10-16T11:28:38.780 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.109058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107875.log 2023-10-16T11:28:38.780 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109058.log.gz 2023-10-16T11:28:38.782 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.124955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94028.log 2023-10-16T11:28:38.782 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124955.log.gz 2023-10-16T11:28:38.783 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.107875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98045.log 2023-10-16T11:28:38.783 INFO:teuthology.orchestra.run.smithi084.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107875.log.gz 2023-10-16T11:28:38.784 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95197.log 2023-10-16T11:28:38.785 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94028.log.gz 2023-10-16T11:28:38.786 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160899.log 2023-10-16T11:28:38.786 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98045.log.gz 2023-10-16T11:28:38.787 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.95197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153351.log 2023-10-16T11:28:38.788 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95197.log.gz 2023-10-16T11:28:38.789 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.160899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100036.log 2023-10-16T11:28:38.789 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.160899.log.gz 2023-10-16T11:28:38.790 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143075.log 2023-10-16T11:28:38.790 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.153351.log.gz 2023-10-16T11:28:38.792 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.100036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107616.log 2023-10-16T11:28:38.792 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100036.log.gz 2023-10-16T11:28:38.793 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93694.log 2023-10-16T11:28:38.793 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.143075.log.gz 2023-10-16T11:28:38.794 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.107616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130140.log 2023-10-16T11:28:38.795 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107616.log.gz 2023-10-16T11:28:38.796 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93694.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-16T11:28:38.796 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93694.log.gz 2023-10-16T11:28:38.797 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.130140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135749.log 2023-10-16T11:28:38.798 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.130140.log.gz 2023-10-16T11:28:38.799 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97310.log 2023-10-16T11:28:38.801 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151449.log 2023-10-16T11:28:38.801 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.135749.log.gz 2023-10-16T11:28:38.802 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97310.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97310.log.gz 2023-10-16T11:28:38.802 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118912.log 2023-10-16T11:28:38.804 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160419.log 2023-10-16T11:28:38.804 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.151449.log.gz 2023-10-16T11:28:38.806 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127125.log 2023-10-16T11:28:38.806 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118912.log.gz 2023-10-16T11:28:38.807 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.160419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165426.log 2023-10-16T11:28:38.807 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.160419.log.gz 2023-10-16T11:28:38.808 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.127125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125447.log 2023-10-16T11:28:38.809 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127125.log.gz 2023-10-16T11:28:38.810 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.165426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125039.log 2023-10-16T11:28:38.810 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.165426.log.gz 2023-10-16T11:28:38.812 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111943.log 2023-10-16T11:28:38.812 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.125447.log.gz 2023-10-16T11:28:38.813 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.125039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147788.log 2023-10-16T11:28:38.814 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125039.log.gz 2023-10-16T11:28:38.815 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.111943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126168.log 2023-10-16T11:28:38.815 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111943.log.gz 2023-10-16T11:28:38.816 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96136.log 2023-10-16T11:28:38.817 INFO:teuthology.orchestra.run.smithi084.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.147788.log.gz 2023-10-16T11:28:38.817 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.126168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156861.log 2023-10-16T11:28:38.818 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126168.log.gz 2023-10-16T11:28:38.819 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.96136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149543.log 2023-10-16T11:28:38.819 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96136.log.gz 2023-10-16T11:28:38.820 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136692.log 2023-10-16T11:28:38.821 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.156861.log.gz 2023-10-16T11:28:38.822 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122727.log 2023-10-16T11:28:38.822 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149543.log.gz 2023-10-16T11:28:38.823 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130586.log 2023-10-16T11:28:38.824 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.136692.log.gz 2023-10-16T11:28:38.826 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122727.log: gzip -5 89.2% -- replaced with /var/log/ceph/ceph-client.admin.122727.log.gz 2023-10-16T11:28:38.826 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.134777.log 2023-10-16T11:28:38.826 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.130586.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.130586.log.gz 2023-10-16T11:28:38.827 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138944.log 2023-10-16T11:28:38.828 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.134777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92864.log 2023-10-16T11:28:38.828 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.134777.log.gz 2023-10-16T11:28:38.829 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138944.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138944.log.gz 2023-10-16T11:28:38.830 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111533.log 2023-10-16T11:28:38.831 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.92864.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94365.log 2023-10-16T11:28:38.831 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.92864.log.gz 2023-10-16T11:28:38.833 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.111533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165708.log 2023-10-16T11:28:38.833 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111533.log.gz 2023-10-16T11:28:38.834 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154895.log 2023-10-16T11:28:38.834 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94365.log.gz 2023-10-16T11:28:38.835 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.165708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163279.log 2023-10-16T11:28:38.836 INFO:teuthology.orchestra.run.smithi084.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.165708.log.gz 2023-10-16T11:28:38.836 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154895.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154895.log.gz 2023-10-16T11:28:38.837 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93055.log 2023-10-16T11:28:38.839 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.163279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107083.log 2023-10-16T11:28:38.839 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.163279.log.gz 2023-10-16T11:28:38.840 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93055.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161923.log 2023-10-16T11:28:38.840 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93055.log.gz 2023-10-16T11:28:38.841 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.107083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147559.log 2023-10-16T11:28:38.842 INFO:teuthology.orchestra.run.smithi084.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107083.log.gz 2023-10-16T11:28:38.843 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161923.log: gzip -5 87.3% -- replaced with /var/log/ceph/ceph-client.admin.161923.log.gz 2023-10-16T11:28:38.844 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.112393.log 2023-10-16T11:28:38.844 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147559.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147559.log.gz 2023-10-16T11:28:38.845 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147693.log 2023-10-16T11:28:38.846 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.112393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127825.log 2023-10-16T11:28:38.847 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112393.log.gz 2023-10-16T11:28:38.848 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127623.log 2023-10-16T11:28:38.848 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.147693.log.gz 2023-10-16T11:28:38.849 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.127825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92604.log 2023-10-16T11:28:38.849 INFO:teuthology.orchestra.run.smithi084.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.127825.log.gz 2023-10-16T11:28:38.851 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.127623.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.127623.log.gz -5 2023-10-16T11:28:38.851 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.100078.log 2023-10-16T11:28:38.852 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.92604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164394.log 2023-10-16T11:28:38.853 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.92604.log.gz 2023-10-16T11:28:38.854 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.100078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154685.log 2023-10-16T11:28:38.854 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100078.log.gz 2023-10-16T11:28:38.855 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.164394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144138.log 2023-10-16T11:28:38.856 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.164394.log.gz 2023-10-16T11:28:38.857 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154769.log 2023-10-16T11:28:38.857 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.154685.log.gz 2023-10-16T11:28:38.858 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95029.log 2023-10-16T11:28:38.859 INFO:teuthology.orchestra.run.smithi084.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.144138.log.gz 2023-10-16T11:28:38.859 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139665.log 87.3% 2023-10-16T11:28:38.860 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.154769.log.gz 2023-10-16T11:28:38.861 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.95029.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145281.log 2023-10-16T11:28:38.862 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95029.log.gz 2023-10-16T11:28:38.863 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119551.log 2023-10-16T11:28:38.863 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.139665.log.gz 2023-10-16T11:28:38.864 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97961.log 2023-10-16T11:28:38.864 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145281.log.gz 2023-10-16T11:28:38.866 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98591.log 2023-10-16T11:28:38.866 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119551.log.gz 2023-10-16T11:28:38.867 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135478.log 2023-10-16T11:28:38.867 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97961.log.gz 2023-10-16T11:28:38.868 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163755.log 2023-10-16T11:28:38.869 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98591.log.gz 2023-10-16T11:28:38.870 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108685.log 2023-10-16T11:28:38.870 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.135478.log.gz 2023-10-16T11:28:38.871 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.163755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152427.log 2023-10-16T11:28:38.872 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.163755.log.gz 2023-10-16T11:28:38.873 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.108685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112986.log 2023-10-16T11:28:38.873 INFO:teuthology.orchestra.run.smithi084.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.108685.log.gz 2023-10-16T11:28:38.874 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113630.log 2023-10-16T11:28:38.875 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.152427.log.gz 2023-10-16T11:28:38.876 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.112986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99359.log 2023-10-16T11:28:38.876 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112986.log.gz 2023-10-16T11:28:38.877 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.113630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97919.log 2023-10-16T11:28:38.878 INFO:teuthology.orchestra.run.smithi084.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.113630.log.gz 2023-10-16T11:28:38.879 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99359.log: gzip -5 88.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.99359.log.gz /var/log/ceph/ceph-client.admin.95533.log 2023-10-16T11:28:38.879 INFO:teuthology.orchestra.run.smithi084.stderr: 2023-10-16T11:28:38.880 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150178.log 2023-10-16T11:28:38.880 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97919.log.gz 2023-10-16T11:28:38.882 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.95533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116452.log 2023-10-16T11:28:38.882 INFO:teuthology.orchestra.run.smithi084.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.95533.log.gz 2023-10-16T11:28:38.884 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150178.log: gzip -5 --verbose -- 89.1% -- replaced with /var/log/ceph/ceph-client.admin.150178.log.gz 2023-10-16T11:28:38.884 INFO:teuthology.orchestra.run.smithi084.stderr: /var/log/ceph/ceph-client.admin.129223.log 2023-10-16T11:28:38.885 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.116452.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116452.log.gz 2023-10-16T11:28:38.885 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151935.log 2023-10-16T11:28:38.886 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114292.log 2023-10-16T11:28:38.889 INFO:teuthology.orchestra.run.smithi084.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.129223.log.gz 2023-10-16T11:28:38.890 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151935.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.151935.log.gz 2023-10-16T11:28:38.890 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118691.log 2023-10-16T11:28:38.891 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.114292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94368.log 2023-10-16T11:28:38.891 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114292.log.gz 2023-10-16T11:28:38.892 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118691.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-16T11:28:38.892 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.118691.log.gz 2023-10-16T11:28:38.892 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94368.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.80949.log 2023-10-16T11:28:38.893 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94368.log.gz 2023-10-16T11:28:38.894 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93097.log 2023-10-16T11:28:38.895 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph.tmp-client.admin.80949.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.80949.log.gz 2023-10-16T11:28:38.896 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107795.log 2023-10-16T11:28:38.897 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93097.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93097.log.gz 2023-10-16T11:28:38.898 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144657.log 2023-10-16T11:28:38.899 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.107795.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107795.log.gz 2023-10-16T11:28:38.900 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159712.log 2023-10-16T11:28:38.901 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144657.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144657.log.gz 2023-10-16T11:28:38.902 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120958.log 2023-10-16T11:28:38.902 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.159712.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.159712.log.gz 2023-10-16T11:28:38.904 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107700.log 2023-10-16T11:28:38.904 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.120958.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120958.log.gz 2023-10-16T11:28:38.905 INFO:teuthology.orchestra.run.smithi084.stderr: 89.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-16T11:28:38.906 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157804.log 2023-10-16T11:28:38.907 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.107700.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107700.log.gz 2023-10-16T11:28:38.907 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156753.log 2023-10-16T11:28:38.908 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157804.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.157804.log.gzgzip 2023-10-16T11:28:38.909 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.100246.log 2023-10-16T11:28:38.910 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160377.log 2023-10-16T11:28:38.910 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.156753.log.gz 2023-10-16T11:28:38.911 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.100246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119467.log 2023-10-16T11:28:38.912 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100246.log.gz 2023-10-16T11:28:38.913 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.160377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161414.log 2023-10-16T11:28:38.913 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.160377.log.gz 2023-10-16T11:28:38.914 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118293.log 2023-10-16T11:28:38.914 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119467.log.gz 2023-10-16T11:28:38.916 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.161414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129181.log 2023-10-16T11:28:38.916 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161414.log.gz 2023-10-16T11:28:38.917 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146876.log 2023-10-16T11:28:38.917 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118293.log.gz 2023-10-16T11:28:38.919 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151575.log 2023-10-16T11:28:38.919 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129181.log.gz 2023-10-16T11:28:38.920 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107346.log 2023-10-16T11:28:38.920 INFO:teuthology.orchestra.run.smithi084.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.146876.log.gz 2023-10-16T11:28:38.921 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151575.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.151575.log.gz -5 2023-10-16T11:28:38.922 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.151533.log 2023-10-16T11:28:38.923 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.107346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135791.log 2023-10-16T11:28:38.923 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107346.log.gz 2023-10-16T11:28:38.924 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151533.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.151533.log.gz 2023-10-16T11:28:38.925 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135581.log 2023-10-16T11:28:38.925 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135791.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.135791.log.gz 2023-10-16T11:28:38.926 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-16T11:28:38.927 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.135581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93265.log 2023-10-16T11:28:38.928 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.135581.log.gz 2023-10-16T11:28:38.930 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.93265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139196.log 2023-10-16T11:28:38.931 INFO:teuthology.orchestra.run.smithi084.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93265.log.gz 2023-10-16T11:28:38.932 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119200.log 2023-10-16T11:28:38.933 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139196.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.139196.log.gz 2023-10-16T11:28:38.934 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109124.log 2023-10-16T11:28:38.935 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119200.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119200.log.gz 2023-10-16T11:28:38.936 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-16T11:28:38.937 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.109124.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.109124.log.gz 2023-10-16T11:28:38.938 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164880.log 2023-10-16T11:28:38.940 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.164880.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.164880.log.gz 2023-10-16T11:28:38.941 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94031.log 2023-10-16T11:28:38.944 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111024.log 2023-10-16T11:28:38.945 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94031.log.gz 2023-10-16T11:28:38.947 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112776.log 2023-10-16T11:28:38.949 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.111024.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111024.log.gz 2023-10-16T11:28:38.952 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.112776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112902.log 2023-10-16T11:28:38.953 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112776.log.gz 2023-10-16T11:28:38.955 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.112902.log: -- /var/log/ceph/ceph-client.admin.150904.log 2023-10-16T11:28:38.957 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112902.log.gz 2023-10-16T11:28:38.958 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151000.log 2023-10-16T11:28:38.962 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150904.log: /var/log/ceph/ceph-client.admin.151000.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113480.log 2023-10-16T11:28:38.963 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.150904.log.gz 2023-10-16T11:28:38.964 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.151000.log.gz 2023-10-16T11:28:38.965 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119869.log/var/log/ceph/ceph-client.admin.113480.log: 2023-10-16T11:28:38.967 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92959.log 2023-10-16T11:28:38.970 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119869.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119869.log.gz 2023-10-16T11:28:38.971 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.92959.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113480.log.gz 2023-10-16T11:28:38.972 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.92959.log.gz 2023-10-16T11:28:38.973 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114957.log 2023-10-16T11:28:38.974 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140115.log 2023-10-16T11:28:38.976 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.114957.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114957.log.gz 2023-10-16T11:28:38.978 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138494.log 2023-10-16T11:28:38.982 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.140115.log: /var/log/ceph/ceph-client.admin.138494.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101158.log 2023-10-16T11:28:38.983 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.140115.log.gz 2023-10-16T11:28:38.986 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6%/var/log/ceph/ceph-client.admin.101158.log: -- replaced with /var/log/ceph/ceph-client.admin.138494.log.gz 2023-10-16T11:28:38.986 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116324.log 2023-10-16T11:28:38.989 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.116324.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101158.log.gz 2023-10-16T11:28:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119330.log 2023-10-16T11:28:38.991 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93696.log 2023-10-16T11:28:38.994 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119330.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116324.log.gz 2023-10-16T11:28:38.995 INFO:teuthology.orchestra.run.smithi084.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119330.log.gz 2023-10-16T11:28:38.997 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107388.log 2023-10-16T11:28:39.001 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.93696.log: /var/log/ceph/ceph-client.admin.107388.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107658.log 2023-10-16T11:28:39.002 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93696.log.gz 2023-10-16T11:28:39.004 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2%/var/log/ceph/ceph-client.admin.107658.log: -- replaced with /var/log/ceph/ceph-client.admin.107388.log.gz 2023-10-16T11:28:39.005 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126304.log 2023-10-16T11:28:39.008 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.126304.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107658.log.gz 2023-10-16T11:28:39.009 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137384.log 2023-10-16T11:28:39.011 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144053.log 2023-10-16T11:28:39.012 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137384.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.126304.log.gz 2023-10-16T11:28:39.012 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.137384.log.gz 2023-10-16T11:28:39.015 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153015.log 2023-10-16T11:28:39.015 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144053.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144053.log.gz 2023-10-16T11:28:39.016 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96974.log 2023-10-16T11:28:39.020 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153015.log: /var/log/ceph/ceph-client.admin.96974.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.153015.log.gz 2023-10-16T11:28:39.020 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99570.log 2023-10-16T11:28:39.022 INFO:teuthology.orchestra.run.smithi084.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150094.log 2023-10-16T11:28:39.022 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96974.log.gz 2023-10-16T11:28:39.024 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99570.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99570.log.gz 2023-10-16T11:28:39.026 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147999.log 2023-10-16T11:28:39.027 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150094.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.150094.log.gz 2023-10-16T11:28:39.028 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116282.log 2023-10-16T11:28:39.030 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147999.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.147999.log.gz 2023-10-16T11:28:39.033 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.116282.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116282.log.gz 2023-10-16T11:28:39.034 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103071.log 2023-10-16T11:28:39.036 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99528.log 2023-10-16T11:28:39.038 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.103071.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103071.log.gz 2023-10-16T11:28:39.040 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93686.log 2023-10-16T11:28:39.044 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0%/var/log/ceph/ceph-client.admin.93686.log: -- replaced with /var/log/ceph/ceph-client.admin.99528.log.gz 2023-10-16T11:28:39.045 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128227.log 2023-10-16T11:28:39.048 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.128227.log: 88.1%gzip -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.93686.log.gz -- /var/log/ceph/ceph-client.admin.159620.log 2023-10-16T11:28:39.048 INFO:teuthology.orchestra.run.smithi084.stderr: 2023-10-16T11:28:39.050 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.128227.log.gz 2023-10-16T11:28:39.051 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146697.log 2023-10-16T11:28:39.056 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146697.log: /var/log/ceph/ceph-client.admin.159620.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118607.log 2023-10-16T11:28:39.056 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.159620.log.gz 2023-10-16T11:28:39.057 INFO:teuthology.orchestra.run.smithi084.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.146697.log.gz 2023-10-16T11:28:39.059 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149711.log 2023-10-16T11:28:39.063 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158644.log 2023-10-16T11:28:39.065 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.118607.log: 88.5% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118607.log.gz 2023-10-16T11:28:39.066 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108938.log 2023-10-16T11:28:39.066 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.149711.log.gz 2023-10-16T11:28:39.067 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158911.log 2023-10-16T11:28:39.068 INFO:teuthology.orchestra.run.smithi084.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.158644.log.gz 2023-10-16T11:28:39.071 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.108938.log: /var/log/ceph/ceph-client.admin.158911.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.108938.log.gz 2023-10-16T11:28:39.072 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119286.log 2023-10-16T11:28:39.075 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119286.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.158911.log.gz 2023-10-16T11:28:39.076 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112351.log 2023-10-16T11:28:39.079 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.112351.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154811.log 2023-10-16T11:28:39.080 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.119286.log.gz 2023-10-16T11:28:39.081 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112351.log.gz 2023-10-16T11:28:39.082 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131276.log 2023-10-16T11:28:39.086 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.131276.log: /var/log/ceph/ceph-client.admin.154811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122855.log 2023-10-16T11:28:39.088 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.154811.log.gz 2023-10-16T11:28:39.089 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.131276.log.gz 2023-10-16T11:28:39.093 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126346.log 2023-10-16T11:28:39.093 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.126346.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126346.log.gz 2023-10-16T11:28:39.093 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157197.log 2023-10-16T11:28:39.097 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.122855.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122855.log.gz 2023-10-16T11:28:39.098 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154228.log 2023-10-16T11:28:39.100 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4%/var/log/ceph/ceph-client.admin.154228.log: -- replaced with /var/log/ceph/ceph-client.admin.157197.log.gz 2023-10-16T11:28:39.102 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134913.log 2023-10-16T11:28:39.102 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.154228.log.gz 2023-10-16T11:28:39.104 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147601.log 2023-10-16T11:28:39.109 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.134913.log: /var/log/ceph/ceph-client.admin.147601.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106495.log 2023-10-16T11:28:39.109 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.134913.log.gz 2023-10-16T11:28:39.110 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.147601.log.gz 2023-10-16T11:28:39.112 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124451.log 2023-10-16T11:28:39.116 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.124451.log: /var/log/ceph/ceph-client.admin.106495.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111313.log 2023-10-16T11:28:39.117 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106495.log.gz 2023-10-16T11:28:39.118 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124451.log.gz 2023-10-16T11:28:39.119 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149110.log 2023-10-16T11:28:39.123 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.111313.log: /var/log/ceph/ceph-client.admin.149110.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111313.log.gz 2023-10-16T11:28:39.125 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123457.log 2023-10-16T11:28:39.126 INFO:teuthology.orchestra.run.smithi084.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.149110.log.gz 2023-10-16T11:28:39.128 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150010.log/var/log/ceph/ceph-client.admin.123457.log: 2023-10-16T11:28:39.132 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150010.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123457.log.gz 2023-10-16T11:28:39.133 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87393.log 2023-10-16T11:28:39.134 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.150010.log.gz 2023-10-16T11:28:39.135 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116620.log 2023-10-16T11:28:39.139 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.116620.log: /var/log/ceph/ceph-client.admin.87393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103366.log 2023-10-16T11:28:39.141 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.87393.log.gz 2023-10-16T11:28:39.142 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.116620.log.gz 2023-10-16T11:28:39.143 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157949.log 2023-10-16T11:28:39.144 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.103366.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103366.log.gz 2023-10-16T11:28:39.145 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163944.log 2023-10-16T11:28:39.147 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.157949.log.gz 2023-10-16T11:28:39.148 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101729.log 2023-10-16T11:28:39.153 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.163944.log: /var/log/ceph/ceph-client.admin.101729.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99868.log 2023-10-16T11:28:39.153 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.163944.log.gz 2023-10-16T11:28:39.156 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.99868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140572.log 2023-10-16T11:28:39.157 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101729.log.gz 2023-10-16T11:28:39.157 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99868.log.gz 2023-10-16T11:28:39.159 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142707.log 2023-10-16T11:28:39.163 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.140572.log: /var/log/ceph/ceph-client.admin.142707.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.140572.log.gz 2023-10-16T11:28:39.164 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119073.log 2023-10-16T11:28:39.165 INFO:teuthology.orchestra.run.smithi084.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142707.log.gz 2023-10-16T11:28:39.166 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139621.log 2023-10-16T11:28:39.166 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.119073.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119073.log.gz 2023-10-16T11:28:39.168 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112309.log 2023-10-16T11:28:39.170 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.139621.log.gz 2023-10-16T11:28:39.171 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148550.log 2023-10-16T11:28:39.176 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.112309.log: /var/log/ceph/ceph-client.admin.148550.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143850.log 2023-10-16T11:28:39.177 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.112309.log.gz 2023-10-16T11:28:39.178 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.148550.log.gz 2023-10-16T11:28:39.180 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150388.log 2023-10-16T11:28:39.183 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150388.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.143850.log.gz 2023-10-16T11:28:39.184 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138578.log 2023-10-16T11:28:39.186 INFO:teuthology.orchestra.run.smithi084.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.150388.log.gz 2023-10-16T11:28:39.188 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162315.log 2023-10-16T11:28:39.191 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.162315.log: /var/log/ceph/ceph-client.admin.138578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139070.log 2023-10-16T11:28:39.193 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145222.log 2023-10-16T11:28:39.193 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138578.log.gz 2023-10-16T11:28:39.194 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.162315.log.gz 2023-10-16T11:28:39.197 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145222.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.145222.log.gz 2023-10-16T11:28:39.199 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139070.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.139070.log.gz 2023-10-16T11:28:39.199 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117687.log 2023-10-16T11:28:39.201 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.117687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127432.log 2023-10-16T11:28:39.203 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117687.log.gz 2023-10-16T11:28:39.206 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.127432.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127432.log.gz 2023-10-16T11:28:39.207 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142445.log 2023-10-16T11:28:39.207 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142445.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.142445.log.gz 2023-10-16T11:28:39.208 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150958.log 2023-10-16T11:28:39.211 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143117.log 2023-10-16T11:28:39.212 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150958.log.gz 2023-10-16T11:28:39.213 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101600.log 2023-10-16T11:28:39.216 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97226.log 2023-10-16T11:28:39.216 INFO:teuthology.orchestra.run.smithi084.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.143117.log.gz 2023-10-16T11:28:39.218 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.101600.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101600.log.gz 2023-10-16T11:28:39.219 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149669.log 2023-10-16T11:28:39.224 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97226.log: /var/log/ceph/ceph-client.admin.149669.log: 88.7%gzip -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.97226.log.gz /var/log/ceph/ceph-client.admin.111817.log 2023-10-16T11:28:39.224 INFO:teuthology.orchestra.run.smithi084.stderr: 2023-10-16T11:28:39.225 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.149669.log.gz 2023-10-16T11:28:39.227 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.111817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130184.log 2023-10-16T11:28:39.229 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156375.log 2023-10-16T11:28:39.230 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6%/var/log/ceph/ceph-client.admin.130184.log: -- replaced with /var/log/ceph/ceph-client.admin.111817.log.gz 2023-10-16T11:28:39.231 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130184.log.gz 2023-10-16T11:28:39.233 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144531.log 2023-10-16T11:28:39.236 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144531.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.156375.log.gz 2023-10-16T11:28:39.237 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124577.log 2023-10-16T11:28:39.238 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144531.log.gz 2023-10-16T11:28:39.238 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.120057.log 2023-10-16T11:28:39.240 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.124577.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124577.log.gz 2023-10-16T11:28:39.241 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92917.log 2023-10-16T11:28:39.244 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.120057.log: /var/log/ceph/ceph-client.admin.92917.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.120057.log.gz 2023-10-16T11:28:39.245 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92917.log.gz 2023-10-16T11:28:39.246 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165468.log 2023-10-16T11:28:39.249 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.165468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158243.log 2023-10-16T11:28:39.250 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.165468.log.gz 2023-10-16T11:28:39.252 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98885.log 2023-10-16T11:28:39.256 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158243.log: /var/log/ceph/ceph-client.admin.98885.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156333.log 2023-10-16T11:28:39.257 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.158243.log.gz 2023-10-16T11:28:39.258 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98885.log.gz 2023-10-16T11:28:39.260 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94034.log 2023-10-16T11:28:39.264 INFO:teuthology.orchestra.run.smithi084.stderr:gzip/var/log/ceph/ceph-client.admin.156333.log: 88.4% -5 --verbose -- /var/log/ceph/ceph-client.admin.164520.log 2023-10-16T11:28:39.264 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.156333.log.gz 2023-10-16T11:28:39.265 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94034.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94034.log.gz 2023-10-16T11:28:39.267 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.137198.log 2023-10-16T11:28:39.271 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.137198.log: /var/log/ceph/ceph-client.admin.164520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138536.log 2023-10-16T11:28:39.272 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.164520.log.gz 2023-10-16T11:28:39.273 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.0.137198.log.gz 2023-10-16T11:28:39.274 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113564.log 2023-10-16T11:28:39.278 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.113564.log: /var/log/ceph/ceph-client.admin.138536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144360.log 2023-10-16T11:28:39.279 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138536.log.gz 2023-10-16T11:28:39.281 INFO:teuthology.orchestra.run.smithi084.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.113564.log.gz 2023-10-16T11:28:39.282 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94021.log 2023-10-16T11:28:39.287 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94021.log: /var/log/ceph/ceph-client.admin.144360.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115365.log 2023-10-16T11:28:39.288 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.144360.log.gz 2023-10-16T11:28:39.289 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94021.log.gz 2023-10-16T11:28:39.290 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101540.log 2023-10-16T11:28:39.295 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.101540.log: /var/log/ceph/ceph-client.admin.115365.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97142.log 2023-10-16T11:28:39.295 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.115365.log.gz 2023-10-16T11:28:39.296 INFO:teuthology.orchestra.run.smithi084.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101540.log.gz 2023-10-16T11:28:39.298 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113184.log 2023-10-16T11:28:39.302 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.113184.log: /var/log/ceph/ceph-client.admin.97142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95491.log 2023-10-16T11:28:39.304 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97142.log.gz 2023-10-16T11:28:39.305 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.113184.log.gz 2023-10-16T11:28:39.306 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98759.log 2023-10-16T11:28:39.310 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98759.log: /var/log/ceph/ceph-client.admin.95491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97715.log 2023-10-16T11:28:39.312 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.97715.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98759.log.gz 2023-10-16T11:28:39.313 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95491.log.gz 2023-10-16T11:28:39.314 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126750.log 2023-10-16T11:28:39.315 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97715.log.gz 2023-10-16T11:28:39.316 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149755.log 2023-10-16T11:28:39.320 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149755.log: /var/log/ceph/ceph-client.admin.126750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.116957.log 2023-10-16T11:28:39.322 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126750.log.gz 2023-10-16T11:28:39.323 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.149755.log.gz 2023-10-16T11:28:39.324 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123499.log 2023-10-16T11:28:39.328 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.123499.log: /var/log/ceph/ceph-client.0.116957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99738.log 2023-10-16T11:28:39.329 INFO:teuthology.orchestra.run.smithi084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.116957.log.gz 2023-10-16T11:28:39.330 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.123499.log.gz 2023-10-16T11:28:39.332 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151323.log 2023-10-16T11:28:39.335 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151323.log: /var/log/ceph/ceph-client.admin.99738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106957.log 2023-10-16T11:28:39.337 INFO:teuthology.orchestra.run.smithi084.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117561.log 2023-10-16T11:28:39.337 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99738.log.gz 2023-10-16T11:28:39.340 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.117561.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151323.log.gz 2023-10-16T11:28:39.340 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117561.log.gz 2023-10-16T11:28:39.342 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.106957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150604.log 2023-10-16T11:28:39.345 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150604.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106957.log.gz 2023-10-16T11:28:39.346 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135538.log 2023-10-16T11:28:39.347 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150604.log.gz 2023-10-16T11:28:39.348 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102760.log 2023-10-16T11:28:39.352 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.102760.log: /var/log/ceph/ceph-client.admin.135538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125997.log 2023-10-16T11:28:39.354 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.135538.log.gz 2023-10-16T11:28:39.354 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.102760.log.gz 2023-10-16T11:28:39.356 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153446.log 2023-10-16T11:28:39.360 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153446.log: /var/log/ceph/ceph-client.admin.125997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130310.log 2023-10-16T11:28:39.361 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.125997.log.gz 2023-10-16T11:28:39.362 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.153446.log.gz 2023-10-16T11:28:39.364 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128952.log 2023-10-16T11:28:39.368 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.128952.log: /var/log/ceph/ceph-client.admin.130310.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.95239.log 2023-10-16T11:28:39.368 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.130310.log.gz 2023-10-16T11:28:39.370 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.128952.log.gz 2023-10-16T11:28:39.371 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157113.log 2023-10-16T11:28:39.374 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157113.log: /var/log/ceph/ceph-client.admin.95239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146960.log 2023-10-16T11:28:39.377 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95239.log.gz 2023-10-16T11:28:39.377 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.157113.log.gz 2023-10-16T11:28:39.378 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106292.log 2023-10-16T11:28:39.381 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.106292.log: /var/log/ceph/ceph-client.admin.146960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101901.log 2023-10-16T11:28:39.383 INFO:teuthology.orchestra.run.smithi084.stderr: 87.5% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.146960.log.gz 2023-10-16T11:28:39.384 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106292.log.gz 2023-10-16T11:28:39.385 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121996.log 2023-10-16T11:28:39.388 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121996.log: /var/log/ceph/ceph-client.admin.101901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120145.log 2023-10-16T11:28:39.390 INFO:teuthology.orchestra.run.smithi084.stderr: 87.5% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101901.log.gz 2023-10-16T11:28:39.391 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140530.log 2023-10-16T11:28:39.391 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.121996.log.gz 2023-10-16T11:28:39.392 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.120145.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.120145.log.gz 2023-10-16T11:28:39.394 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.140530.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.140530.log.gz 2023-10-16T11:28:39.395 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94635.log 2023-10-16T11:28:39.397 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94635.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-16T11:28:39.398 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94635.log.gz 2023-10-16T11:28:39.400 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129918.log 2023-10-16T11:28:39.407 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.129918.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.129918.log.gz 2023-10-16T11:28:39.407 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121641.log 2023-10-16T11:28:39.409 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153607.log 2023-10-16T11:28:39.413 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153607.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.153607.log.gz 2023-10-16T11:28:39.413 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.100583.log 2023-10-16T11:28:39.414 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121641.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.121641.log.gz 2023-10-16T11:28:39.418 INFO:teuthology.orchestra.run.smithi084.stderr:gzip/var/log/ceph/ceph-client.0.100583.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.100583.log.gz 2023-10-16T11:28:39.418 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.121599.log 2023-10-16T11:28:39.421 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139112.log 2023-10-16T11:28:39.423 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.121599.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.121599.log.gz 2023-10-16T11:28:39.428 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139112.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.139112.log.gz 2023-10-16T11:28:39.428 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127242.log 2023-10-16T11:28:39.430 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134651.log 2023-10-16T11:28:39.433 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.127242.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127242.log.gz 2023-10-16T11:28:39.433 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157864.log 2023-10-16T11:28:39.435 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.134651.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.134651.log.gz 2023-10-16T11:28:39.440 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157864.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.157864.log.gz 2023-10-16T11:28:39.440 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153734.log 2023-10-16T11:28:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139537.log 2023-10-16T11:28:39.443 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153734.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.153734.log.gz 2023-10-16T11:28:39.445 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157071.log 2023-10-16T11:28:39.447 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139537.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.139537.log.gz 2023-10-16T11:28:39.451 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139154.log 2023-10-16T11:28:39.452 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157071.log.gz 2023-10-16T11:28:39.456 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98465.log 2023-10-16T11:28:39.458 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139154.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139154.log.gz 2023-10-16T11:28:39.460 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.98465.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98465.log.gz 2023-10-16T11:28:39.462 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153974.log 2023-10-16T11:28:39.465 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112080.log 2023-10-16T11:28:39.467 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153974.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.153974.log.gz 2023-10-16T11:28:39.468 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130352.log 2023-10-16T11:28:39.470 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.112080.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112080.log.gz 2023-10-16T11:28:39.471 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160608.log 2023-10-16T11:28:39.475 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.130352.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.130352.log.gz 2023-10-16T11:28:39.476 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116060.log 2023-10-16T11:28:39.478 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.160608.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.160608.log.gz 2023-10-16T11:28:39.482 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.116060.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.116060.log.gz 2023-10-16T11:28:39.483 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141406.log 2023-10-16T11:28:39.486 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127075.log 2023-10-16T11:28:39.490 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141406.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.141406.log.gz 2023-10-16T11:28:39.490 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94035.log 2023-10-16T11:28:39.491 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.127075.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.127075.log.gz 2023-10-16T11:28:39.492 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139759.log 2023-10-16T11:28:39.493 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.94035.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94035.log.gz 2023-10-16T11:28:39.497 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153526.log 2023-10-16T11:28:39.499 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139759.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.139759.log.gz 2023-10-16T11:28:39.501 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153526.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.153526.log.gz 2023-10-16T11:28:39.503 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95671.log 2023-10-16T11:28:39.506 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150496.log 2023-10-16T11:28:39.508 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.95671.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95671.log.gz 2023-10-16T11:28:39.508 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133479.log 2023-10-16T11:28:39.510 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150496.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.150496.log.gz 2023-10-16T11:28:39.513 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.133479.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.133479.log.gz 2023-10-16T11:28:39.644 INFO:teuthology.orchestra.run.smithi157.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-16T11:28:39.670 INFO:teuthology.orchestra.run.smithi157.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-16T11:28:39.794 INFO:teuthology.orchestra.run.smithi084.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-16T11:28:40.351 INFO:teuthology.orchestra.run.smithi084.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-16T11:28:40.567 INFO:teuthology.orchestra.run.smithi157.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-16T11:28:40.622 INFO:teuthology.orchestra.run.smithi157.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-16T11:28:41.082 INFO:teuthology.orchestra.run.smithi084.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-16T11:28:41.730 INFO:teuthology.orchestra.run.smithi157.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-16T11:28:41.735 INFO:teuthology.orchestra.run.smithi084.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-16T11:28:41.912 INFO:teuthology.orchestra.run.smithi084.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-16T11:28:41.914 INFO:teuthology.orchestra.run.smithi084.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-16T11:28:42.343 INFO:teuthology.orchestra.run.smithi084.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-16T11:28:42.504 INFO:teuthology.orchestra.run.smithi157.stderr: 94.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-16T11:28:42.509 INFO:teuthology.orchestra.run.smithi157.stderr: 2023-10-16T11:28:42.509 INFO:teuthology.orchestra.run.smithi157.stderr:real 0m4.894s 2023-10-16T11:28:42.509 INFO:teuthology.orchestra.run.smithi157.stderr:user 0m18.012s 2023-10-16T11:28:42.510 INFO:teuthology.orchestra.run.smithi157.stderr:sys 0m1.621s 2023-10-16T11:28:43.794 INFO:teuthology.orchestra.run.smithi084.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-16T11:28:43.799 INFO:teuthology.orchestra.run.smithi084.stderr: 2023-10-16T11:28:43.799 INFO:teuthology.orchestra.run.smithi084.stderr:real 0m6.191s 2023-10-16T11:28:43.800 INFO:teuthology.orchestra.run.smithi084.stderr:user 0m22.427s 2023-10-16T11:28:43.800 INFO:teuthology.orchestra.run.smithi084.stderr:sys 0m4.800s 2023-10-16T11:28:43.801 INFO:tasks.ceph:Archiving logs... 2023-10-16T11:28:43.802 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/log/ceph to /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428968/remote/smithi084/log 2023-10-16T11:28:43.803 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-16T11:28:47.495 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428968/remote/smithi157/log 2023-10-16T11:28:47.496 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-16T11:28:49.787 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-16T11:28:49.797 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-16T11:28:49.798 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-16T11:28:49.808 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-16T11:28:49.963 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-16T11:28:49.973 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-16T11:28:49.974 DEBUG:teuthology.orchestra.run.smithi084:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T11:28:49.983 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T11:28:50.032 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2023-10-16T11:28:50.039 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2023-10-16T11:28:50.043 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-16T11:28:50.043 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2023-10-16T11:28:50.043 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 11 +1074us[+1032us] +/- 95ms 2023-10-16T11:28:50.044 INFO:teuthology.orchestra.run.smithi084.stdout:^* hv02.front.sepia.ceph.com 3 6 377 8 -721us[ -763us] +/- 78ms 2023-10-16T11:28:50.044 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 19 -149us[ -191us] +/- 84ms 2023-10-16T11:28:50.044 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T11:28:50.050 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-16T11:28:50.050 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2023-10-16T11:28:50.050 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 83 +1092us[+1093us] +/- 94ms 2023-10-16T11:28:50.051 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv02.front.sepia.ceph.com 3 7 377 19 -683us[ -682us] +/- 77ms 2023-10-16T11:28:50.051 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 17 -146us[ -146us] +/- 84ms 2023-10-16T11:28:50.051 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T11:28:50.053 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-16T11:28:50.062 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-16T11:28:50.063 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-16T11:28:50.071 DEBUG:teuthology.orchestra.run.smithi084:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-16T11:28:50.265 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-16T11:28:50.526 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-16T11:28:50.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T11:28:50.629 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-16T11:28:50.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T11:28:50.738 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-16T11:28:50.747 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-16T11:28:50.756 INFO:teuthology.task.internal:Duration was 6755.362400 seconds 2023-10-16T11:28:50.757 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-16T11:28:50.767 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-16T11:28:50.767 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-16T11:28:50.776 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-16T11:28:50.941 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-16T11:28:50.948 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-16T11:28:51.395 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-16T11:28:51.395 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi084.front.sepia.ceph.com 2023-10-16T11:28:51.396 DEBUG:teuthology.orchestra.run.smithi084:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-16T11:28:51.464 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2023-10-16T11:28:51.464 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-16T11:28:51.548 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-16T11:28:51.548 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-16T11:28:51.557 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-16T11:28:51.652 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-16T11:28:51.653 DEBUG:teuthology.orchestra.run.smithi084:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-16T11:28:51.659 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-16T11:28:51.930 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-16T11:28:51.941 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-16T11:28:51.942 DEBUG:teuthology.orchestra.run.smithi084:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-16T11:28:52.041 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-16T11:28:52.142 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-16T11:28:52.152 DEBUG:teuthology.orchestra.run.smithi084:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-16T11:28:52.161 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-16T11:28:52.267 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = core 2023-10-16T11:28:52.277 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2023-10-16T11:28:52.329 DEBUG:teuthology.orchestra.run.smithi084:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-16T11:28:52.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T11:28:52.383 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-16T11:28:52.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T11:28:52.440 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-16T11:28:52.451 INFO:teuthology.task.internal:Transferring archived files... 2023-10-16T11:28:52.452 DEBUG:teuthology.misc:Transferring archived files from smithi084:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428968/remote/smithi084 2023-10-16T11:28:52.452 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-16T11:28:52.561 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428968/remote/smithi157 2023-10-16T11:28:52.562 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-16T11:28:52.677 INFO:teuthology.task.internal:Removing archive directory... 2023-10-16T11:28:52.678 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-16T11:28:52.687 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-16T11:28:52.741 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-16T11:28:52.755 INFO:teuthology.task.internal:Not uploading archives. 2023-10-16T11:28:52.756 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-16T11:28:52.766 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-16T11:28:52.769 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-16T11:28:52.779 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-16T11:28:52.830 INFO:teuthology.orchestra.run.smithi084.stdout: 393241 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 16 11:28 /home/ubuntu/cephtest 2023-10-16T11:28:52.838 INFO:teuthology.orchestra.run.smithi157.stdout: 393723 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 16 11:28 /home/ubuntu/cephtest 2023-10-16T11:28:52.842 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-16T11:28:52.852 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-16T11:28:52.875 INFO:teuthology.nuke:Checking targets against current locks 2023-10-16T11:28:52.903 DEBUG:teuthology.nuke:shortname: smithi084 2023-10-16T11:28:52.903 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T11:28:52.925 DEBUG:teuthology.nuke:shortname: smithi157 2023-10-16T11:28:52.925 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T11:28:52.929 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428968', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 09:26:43.499039', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJXxfKL1uHMlQo74v+R3GoiPXecQzV8P30woGckwQ84qfJ6EEXNgTDKH3hKwuDEEPsHSPFbQFO1y9ERH4QRY11M='} 2023-10-16T11:28:52.949 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428968', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 09:26:43.500630', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE2h+CtCtnk9zYAofCvqysHgWMJhS6NpRlokOe2x0VCCmhnx8iudgEX75GybhID5gv11OiYGzIRjlOPKuwng4PM='} 2023-10-16T11:28:52.983 INFO:teuthology.orchestra.console.smithi084:Power off 2023-10-16T11:28:52.984 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-16T11:28:53.006 DEBUG:teuthology.orchestra.console.smithi084:power off output: Chassis Power Control: Down/Off 2023-10-16T11:28:53.007 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T11:28:53.021 INFO:teuthology.orchestra.console.smithi157:Power off 2023-10-16T11:28:53.022 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-16T11:28:53.032 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is on 2023-10-16T11:28:53.046 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2023-10-16T11:28:53.047 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T11:28:53.067 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-10-16T11:28:57.034 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T11:28:57.068 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T11:28:57.145 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is on 2023-10-16T11:28:57.178 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-10-16T11:29:01.149 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T11:29:01.179 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T11:29:01.260 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is on 2023-10-16T11:29:01.290 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2023-10-16T11:29:01.391 INFO:teuthology.orchestra.console.smithi157:Power off completed 2023-10-16T11:29:05.264 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T11:29:05.374 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is off 2023-10-16T11:29:05.475 INFO:teuthology.orchestra.console.smithi084:Power off completed 2023-10-16T11:29:05.581 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 client.1-kernel-sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 6755.362399816513 failure_reason: 'Test failure: test_snap_schedule_subvol_and_group_arguments (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments)' flavor: default owner: mchangir sentry_event: https://sentry.ceph.com/organizations/ceph/?query=6c4b54827f9f4f5ebf24f205fad28a20 status: fail success: false 2023-10-16T11:29:05.581 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T11:29:05.663 INFO:teuthology.run:FAIL